2023-11-26T06:58:22.498 INFO:root:teuthology version: 0.0.1.dev205+g2442239 2023-11-26T06:58:22.498 DEBUG:teuthology.run:Teuthology command: teuthology --description fs:volumes/{begin/{0-install 1-ceph 2-logrotate} clusters/1a3s-mds-4c-client conf/{client mds mon osd} distro/{ubuntu_latest} mount/fuse objectstore/bluestore-bitmap overrides/{ignorelist_health ignorelist_wrongly_marked_down no_client_pidfile} tasks/volumes/{overrides test/misc}} --archive /home/teuthworker/archive/rishabh-2023-11-26_06:51:12-fs:volumes-rishabh-mgr-vol-misc-testing-default-smithi/7467343 --name rishabh-2023-11-26_06:51:12-fs:volumes-rishabh-mgr-vol-misc-testing-default-smithi --verbose --owner scheduled_rishabh@teuthology -- /home/teuthworker/archive/rishabh-2023-11-26_06:51:12-fs:volumes-rishabh-mgr-vol-misc-testing-default-smithi/7467343/orig.config.yaml 2023-11-26T06:58:22.541 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2023-11-26T06:58:22.652 INFO:teuthology.run:Config: archive_path: /home/teuthworker/archive/rishabh-2023-11-26_06:51:12-fs:volumes-rishabh-mgr-vol-misc-testing-default-smithi/7467343 branch: rishabh-mgr-vol-misc description: fs:volumes/{begin/{0-install 1-ceph 2-logrotate} clusters/1a3s-mds-4c-client conf/{client mds mon osd} distro/{ubuntu_latest} mount/fuse objectstore/bluestore-bitmap overrides/{ignorelist_health ignorelist_wrongly_marked_down no_client_pidfile} tasks/volumes/{overrides test/misc}} email: null first_in_suite: false job_id: '7467343' kernel: flavor: default kdb: true sha1: 8247bbb5b470f7e2a1f3eefcc16e24cf1338f606 ktype: distro last_in_suite: false log-rotate: ceph-mds: 10G ceph-osd: 10G machine_type: smithi name: rishabh-2023-11-26_06:51:12-fs:volumes-rishabh-mgr-vol-misc-testing-default-smithi no_nested_subset: false nuke-on-error: true openstack: - volumes: count: 4 size: 20 - machine: disk: 200 os_type: ubuntu os_version: '22.04' overrides: admin_socket: branch: rishabh-mgr-vol-misc ceph: conf: client: client mount timeout: 600 debug client: 20 debug ms: 1 pid file: '' rados mon op timeout: 900 rados osd op timeout: 900 mds: debug mds: 20 debug mds balancer: 20 debug ms: 1 mds debug frag: true mds debug scatterstat: true mds op complaint time: 180 mds verify scatter: true osd op complaint time: 180 rados mon op timeout: 900 rados osd op timeout: 900 mgr: debug client: 20 debug finisher: 20 debug mgr: 20 debug ms: 1 mon: debug mon: 20 debug ms: 1 debug paxos: 20 mon op complaint time: 120 osd: bdev async discard: true bdev enable discard: true bluestore allocator: bitmap bluestore block size: 96636764160 bluestore fsck on mount: true debug bluefs: 1/20 debug bluestore: 1/20 debug ms: 1 debug osd: 20 debug rocksdb: 4/10 mon osd backfillfull_ratio: 0.85 mon osd full ratio: 0.9 mon osd nearfull ratio: 0.8 osd failsafe full ratio: 0.95 osd objectstore: bluestore osd op complaint time: 180 flavor: default fs: xfs log-ignorelist: - \(MDS_ALL_DOWN\) - \(MDS_UP_LESS_THAN_MAX\) - overall HEALTH_ - \(FS_DEGRADED\) - \(MDS_FAILED\) - \(MDS_DEGRADED\) - \(FS_WITH_FAILED_MDS\) - \(MDS_DAMAGE\) - \(MDS_ALL_DOWN\) - \(MDS_UP_LESS_THAN_MAX\) - \(FS_INLINE_DATA_DEPRECATED\) - \(POOL_APP_NOT_ENABLED\) - overall HEALTH_ - \(OSD_DOWN\) - \(OSD_ - but it is still running - is not responding - OSD full dropping all updates - OSD near full - pausewr flag - failsafe engaged, dropping updates - failsafe disengaged, no longer dropping - is full \(reached quota - POOL_FULL - POOL_BACKFILLFULL sha1: 44c2237fe77713214148c1fd622c872f346f1b99 ceph-deploy: bluestore: true conf: client: log file: /var/log/ceph/ceph-$name.$pid.log mon: {} osd: bdev async discard: true bdev enable discard: true bluestore block size: 96636764160 bluestore fsck on mount: true debug bluefs: 1/20 debug bluestore: 1/20 debug rocksdb: 4/10 mon osd backfillfull_ratio: 0.85 mon osd full ratio: 0.9 mon osd nearfull ratio: 0.8 osd failsafe full ratio: 0.95 osd objectstore: bluestore fs: xfs install: ceph: flavor: default sha1: 44c2237fe77713214148c1fd622c872f346f1b99 thrashosds: bdev_inject_crash: 2 bdev_inject_crash_probability: 0.5 workunit: branch: rishabh-mgr-vol-misc sha1: 44c2237fe77713214148c1fd622c872f346f1b99 owner: scheduled_rishabh@teuthology priority: 50 repo: https://git.ceph.com/ceph-ci.git roles: - - mon.a - mgr.y - mds.a - mds.c - osd.0 - osd.1 - osd.2 - osd.3 - client.0 - client.2 - - mon.b - mon.c - mgr.x - mds.b - mds.d - osd.4 - osd.5 - osd.6 - osd.7 - client.1 - client.3 seed: 1863 sha1: 44c2237fe77713214148c1fd622c872f346f1b99 sleep_before_teardown: 0 suite: fs:volumes suite_branch: rishabh-mgr-vol-misc suite_path: /home/teuthworker/src/git.ceph.com_ceph-c_44c2237fe77713214148c1fd622c872f346f1b99/qa suite_relpath: qa suite_repo: https://git.ceph.com/ceph-ci.git suite_sha1: 44c2237fe77713214148c1fd622c872f346f1b99 syslog: ignorelist: - WARNING*.*check_session_state - WARNING*.*__ceph_remove_cap targets: smithi143.front.sepia.ceph.com: ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBLMlsYrpI20D7nmuV4Ss3kS8JeSY9qF58zM5ezfrs6+JcgHI73OPCbgwt0ossMtulfaME6pGevN9hNlafUPf9wQ= smithi188.front.sepia.ceph.com: ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBJdbw4I9Ou0oA1kXwuSM4nPA+qQBLn4/ORmmu+HODcBiXZaQj7dzvrfCaWf6g8HzUkhcLo6Ef+7Yycrf+9QMqsM= tasks: - install: extra_packages: deb: - python3-cephfs - cephfs-shell - cephfs-top - cephfs-mirror rpm: - python3-cephfs - cephfs-top - cephfs-mirror extra_system_packages: deb: - bison - flex - libelf-dev - libssl-dev - network-manager - iproute2 - util-linux - dump - indent - libaio-dev - libtool-bin - uuid-dev - xfslibs-dev - postgresql - postgresql-client - postgresql-common - postgresql-contrib rpm: - bison - flex - elfutils-libelf-devel - openssl-devel - NetworkManager - iproute - util-linux - libacl-devel - libaio-devel - libattr-devel - libtool - libuuid-devel - xfsdump - xfsprogs - xfsprogs-devel - libaio-devel - libtool - libuuid-devel - xfsprogs-devel - postgresql - postgresql-server - postgresql-contrib - ceph: create_rbd_pool: false - ceph-fuse: null - cephfs_test_runner: fail_on_skip: false modules: - tasks.cephfs.test_volumes.TestMisc teuthology: fragments_dropped: [] meta: {} postmerge: - local function is_kupstream() return false end - local function is_kdistro() return false end - local function is_fuse() return true end - local function syntax_version() return '' end teuthology_branch: main teuthology_sha1: 2442239d2653456406c25ae0c71b689c8f2657b6 timestamp: 2023-11-26_06:51:12 tube: smithi user: rishabh verbose: false worker_log: /home/teuthworker/archive/worker_logs/dispatcher.smithi.232157 2023-11-26T06:58:22.653 INFO:teuthology.run:suite_path is set to /home/teuthworker/src/git.ceph.com_ceph-c_44c2237fe77713214148c1fd622c872f346f1b99/qa; will attempt to use it 2023-11-26T06:58:22.653 INFO:teuthology.run:Found tasks at /home/teuthworker/src/git.ceph.com_ceph-c_44c2237fe77713214148c1fd622c872f346f1b99/qa/tasks 2023-11-26T06:58:22.653 INFO:teuthology.run_tasks:Running task internal.check_packages... 2023-11-26T06:58:22.655 INFO:teuthology.task.internal:Checking packages... 2023-11-26T06:58:22.676 INFO:teuthology.task.internal:Checking packages for os_type 'ubuntu', flavor 'default' and ceph hash '44c2237fe77713214148c1fd622c872f346f1b99' 2023-11-26T06:58:22.677 WARNING:teuthology.packaging:More than one of ref, tag, branch, or sha1 supplied; using branch 2023-11-26T06:58:22.677 INFO:teuthology.packaging:ref: None 2023-11-26T06:58:22.677 INFO:teuthology.packaging:tag: None 2023-11-26T06:58:22.677 INFO:teuthology.packaging:branch: rishabh-mgr-vol-misc 2023-11-26T06:58:22.677 INFO:teuthology.packaging:sha1: 44c2237fe77713214148c1fd622c872f346f1b99 2023-11-26T06:58:22.677 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=ubuntu%2F22.04%2Fx86_64&ref=rishabh-mgr-vol-misc 2023-11-26T06:58:22.936 INFO:teuthology.task.internal:Found packages for ceph version 18.0.0-7484-g44c2237f-1jammy 2023-11-26T06:58:22.936 INFO:teuthology.run_tasks:Running task internal.buildpackages_prep... 2023-11-26T06:58:22.943 INFO:teuthology.task.internal:no buildpackages task found 2023-11-26T06:58:22.943 INFO:teuthology.run_tasks:Running task internal.save_config... 2023-11-26T06:58:22.962 INFO:teuthology.task.internal:Saving configuration 2023-11-26T06:58:22.978 INFO:teuthology.run_tasks:Running task internal.check_lock... 2023-11-26T06:58:22.984 INFO:teuthology.task.internal.check_lock:Checking locks... 2023-11-26T06:58:23.011 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi143.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/rishabh-2023-11-26_06:51:12-fs:volumes-rishabh-mgr-vol-misc-testing-default-smithi/7467343', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'ubuntu', 'os_version': '22.04', 'arch': 'x86_64', 'locked': True, 'locked_since': '2023-11-26 06:52:34.444332', 'locked_by': 'scheduled_rishabh@teuthology', 'mac_address': None, 'ssh_pub_key': 'ssh-ed25519 AAAAC3NzaC1lZDI1NTE5AAAAILAyMFUd11EVvXltu3xYjH1ZPHWJ6gxOKLCRYQ5a2IrM'} 2023-11-26T06:58:23.032 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi188.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/rishabh-2023-11-26_06:51:12-fs:volumes-rishabh-mgr-vol-misc-testing-default-smithi/7467343', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'ubuntu', 'os_version': '22.04', 'arch': 'x86_64', 'locked': True, 'locked_since': '2023-11-26 06:52:34.445826', 'locked_by': 'scheduled_rishabh@teuthology', 'mac_address': None, 'ssh_pub_key': 'ssh-ed25519 AAAAC3NzaC1lZDI1NTE5AAAAIISuAATZW43RqDi8vY2Q7K6JWR1XOCpaLyBSzuqhMsIC'} 2023-11-26T06:58:23.032 INFO:teuthology.run_tasks:Running task internal.add_remotes... 2023-11-26T06:58:23.039 INFO:teuthology.task.internal:roles: ubuntu@smithi143.front.sepia.ceph.com - ['mon.a', 'mgr.y', 'mds.a', 'mds.c', 'osd.0', 'osd.1', 'osd.2', 'osd.3', 'client.0', 'client.2'] 2023-11-26T06:58:23.039 INFO:teuthology.task.internal:roles: ubuntu@smithi188.front.sepia.ceph.com - ['mon.b', 'mon.c', 'mgr.x', 'mds.b', 'mds.d', 'osd.4', 'osd.5', 'osd.6', 'osd.7', 'client.1', 'client.3'] 2023-11-26T06:58:23.039 INFO:teuthology.run_tasks:Running task console_log... 2023-11-26T06:58:23.107 DEBUG:teuthology.exit:Installing handler: Handler(exiter=, func=.kill_console_loggers at 0x7fe4d8bb00d0>, signals=[15]) 2023-11-26T06:58:23.107 INFO:teuthology.run_tasks:Running task internal.connect... 2023-11-26T06:58:23.114 INFO:teuthology.task.internal:Opening connections... 2023-11-26T06:58:23.114 DEBUG:teuthology.task.internal:connecting to ubuntu@smithi143.front.sepia.ceph.com 2023-11-26T06:58:23.115 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi143.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-11-26T06:58:23.193 DEBUG:teuthology.task.internal:connecting to ubuntu@smithi188.front.sepia.ceph.com 2023-11-26T06:58:23.193 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi188.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-11-26T06:58:23.269 INFO:teuthology.run_tasks:Running task internal.push_inventory... 2023-11-26T06:58:23.276 DEBUG:teuthology.orchestra.run.smithi143:> uname -m 2023-11-26T06:58:23.281 INFO:teuthology.orchestra.run.smithi143.stdout:x86_64 2023-11-26T06:58:23.281 DEBUG:teuthology.orchestra.run.smithi143:> cat /etc/os-release 2023-11-26T06:58:23.325 INFO:teuthology.orchestra.run.smithi143.stdout:PRETTY_NAME="Ubuntu 22.04.1 LTS" 2023-11-26T06:58:23.326 INFO:teuthology.orchestra.run.smithi143.stdout:NAME="Ubuntu" 2023-11-26T06:58:23.326 INFO:teuthology.orchestra.run.smithi143.stdout:VERSION_ID="22.04" 2023-11-26T06:58:23.326 INFO:teuthology.orchestra.run.smithi143.stdout:VERSION="22.04.1 LTS (Jammy Jellyfish)" 2023-11-26T06:58:23.326 INFO:teuthology.orchestra.run.smithi143.stdout:VERSION_CODENAME=jammy 2023-11-26T06:58:23.326 INFO:teuthology.orchestra.run.smithi143.stdout:ID=ubuntu 2023-11-26T06:58:23.326 INFO:teuthology.orchestra.run.smithi143.stdout:ID_LIKE=debian 2023-11-26T06:58:23.326 INFO:teuthology.orchestra.run.smithi143.stdout:HOME_URL="https://www.ubuntu.com/" 2023-11-26T06:58:23.326 INFO:teuthology.orchestra.run.smithi143.stdout:SUPPORT_URL="https://help.ubuntu.com/" 2023-11-26T06:58:23.326 INFO:teuthology.orchestra.run.smithi143.stdout:BUG_REPORT_URL="https://bugs.launchpad.net/ubuntu/" 2023-11-26T06:58:23.326 INFO:teuthology.orchestra.run.smithi143.stdout:PRIVACY_POLICY_URL="https://www.ubuntu.com/legal/terms-and-policies/privacy-policy" 2023-11-26T06:58:23.326 INFO:teuthology.orchestra.run.smithi143.stdout:UBUNTU_CODENAME=jammy 2023-11-26T06:58:23.327 INFO:teuthology.lock.ops:Updating smithi143.front.sepia.ceph.com on lock server 2023-11-26T06:58:23.351 DEBUG:teuthology.orchestra.run.smithi188:> uname -m 2023-11-26T06:58:23.356 INFO:teuthology.orchestra.run.smithi188.stdout:x86_64 2023-11-26T06:58:23.357 DEBUG:teuthology.orchestra.run.smithi188:> cat /etc/os-release 2023-11-26T06:58:23.407 INFO:teuthology.orchestra.run.smithi188.stdout:PRETTY_NAME="Ubuntu 22.04.1 LTS" 2023-11-26T06:58:23.407 INFO:teuthology.orchestra.run.smithi188.stdout:NAME="Ubuntu" 2023-11-26T06:58:23.407 INFO:teuthology.orchestra.run.smithi188.stdout:VERSION_ID="22.04" 2023-11-26T06:58:23.407 INFO:teuthology.orchestra.run.smithi188.stdout:VERSION="22.04.1 LTS (Jammy Jellyfish)" 2023-11-26T06:58:23.407 INFO:teuthology.orchestra.run.smithi188.stdout:VERSION_CODENAME=jammy 2023-11-26T06:58:23.407 INFO:teuthology.orchestra.run.smithi188.stdout:ID=ubuntu 2023-11-26T06:58:23.407 INFO:teuthology.orchestra.run.smithi188.stdout:ID_LIKE=debian 2023-11-26T06:58:23.407 INFO:teuthology.orchestra.run.smithi188.stdout:HOME_URL="https://www.ubuntu.com/" 2023-11-26T06:58:23.408 INFO:teuthology.orchestra.run.smithi188.stdout:SUPPORT_URL="https://help.ubuntu.com/" 2023-11-26T06:58:23.408 INFO:teuthology.orchestra.run.smithi188.stdout:BUG_REPORT_URL="https://bugs.launchpad.net/ubuntu/" 2023-11-26T06:58:23.408 INFO:teuthology.orchestra.run.smithi188.stdout:PRIVACY_POLICY_URL="https://www.ubuntu.com/legal/terms-and-policies/privacy-policy" 2023-11-26T06:58:23.408 INFO:teuthology.orchestra.run.smithi188.stdout:UBUNTU_CODENAME=jammy 2023-11-26T06:58:23.408 INFO:teuthology.lock.ops:Updating smithi188.front.sepia.ceph.com on lock server 2023-11-26T06:58:23.426 INFO:teuthology.run_tasks:Running task internal.serialize_remote_roles... 2023-11-26T06:58:23.436 INFO:teuthology.run_tasks:Running task internal.check_conflict... 2023-11-26T06:58:23.441 INFO:teuthology.task.internal:Checking for old test directory... 2023-11-26T06:58:23.442 DEBUG:teuthology.orchestra.run.smithi143:> test '!' -e /home/ubuntu/cephtest 2023-11-26T06:58:23.444 DEBUG:teuthology.orchestra.run.smithi188:> test '!' -e /home/ubuntu/cephtest 2023-11-26T06:58:23.454 INFO:teuthology.run_tasks:Running task internal.check_ceph_data... 2023-11-26T06:58:23.460 INFO:teuthology.task.internal:Checking for non-empty /var/lib/ceph... 2023-11-26T06:58:23.461 DEBUG:teuthology.orchestra.run.smithi143:> test -z $(ls -A /var/lib/ceph) 2023-11-26T06:58:23.491 DEBUG:teuthology.orchestra.run.smithi188:> test -z $(ls -A /var/lib/ceph) 2023-11-26T06:58:23.505 INFO:teuthology.run_tasks:Running task internal.vm_setup... 2023-11-26T06:58:23.560 INFO:teuthology.run_tasks:Running task kernel... 2023-11-26T06:58:23.574 INFO:teuthology.task.kernel:normalize config orig: {'flavor': 'default', 'kdb': True, 'sha1': '8247bbb5b470f7e2a1f3eefcc16e24cf1338f606'} 2023-11-26T06:58:23.574 INFO:teuthology.task.kernel:config {'mon.a': {'flavor': 'default', 'kdb': True, 'sha1': '8247bbb5b470f7e2a1f3eefcc16e24cf1338f606'}, 'mon.b': {'flavor': 'default', 'kdb': True, 'sha1': '8247bbb5b470f7e2a1f3eefcc16e24cf1338f606'}}, timeout 300 2023-11-26T06:58:23.574 DEBUG:teuthology.orchestra.run.smithi143:> test -f /run/.containerenv -o -f /.dockerenv 2023-11-26T06:58:23.575 DEBUG:teuthology.orchestra.run.smithi188:> test -f /run/.containerenv -o -f /.dockerenv 2023-11-26T06:58:23.579 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-11-26T06:58:23.580 DEBUG:teuthology.task.kernel:sha1 for mon.a is 8247bbb5b470f7e2a1f3eefcc16e24cf1338f606 2023-11-26T06:58:23.580 INFO:teuthology.task.kernel:Checking kernel version of mon.a, want "8247bbb5b470f7e2a1f3eefcc16e24cf1338f606"... 2023-11-26T06:58:23.580 DEBUG:teuthology.orchestra.run.smithi143:> uname -r 2023-11-26T06:58:23.581 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-11-26T06:58:23.581 DEBUG:teuthology.task.kernel:sha1 for mon.b is 8247bbb5b470f7e2a1f3eefcc16e24cf1338f606 2023-11-26T06:58:23.581 INFO:teuthology.task.kernel:Checking kernel version of mon.b, want "8247bbb5b470f7e2a1f3eefcc16e24cf1338f606"... 2023-11-26T06:58:23.581 DEBUG:teuthology.orchestra.run.smithi188:> uname -r 2023-11-26T06:58:23.626 INFO:teuthology.orchestra.run.smithi143.stdout:5.15.0-56-generic 2023-11-26T06:58:23.627 DEBUG:teuthology.task.kernel:current kernel version is 5.15.0-56-generic vs 8247bbb5b470f7e2a1f3eefcc16e24cf1338f606 2023-11-26T06:58:23.627 DEBUG:teuthology.task.kernel:failed to parse current kernel version 2023-11-26T06:58:23.627 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=kernel&flavor=default&distros=ubuntu%2F22.04%2Fx86_64&sha1=8247bbb5b470f7e2a1f3eefcc16e24cf1338f606 2023-11-26T06:58:23.630 INFO:teuthology.orchestra.run.smithi188.stdout:5.15.0-56-generic 2023-11-26T06:58:23.631 DEBUG:teuthology.task.kernel:current kernel version is 5.15.0-56-generic vs 8247bbb5b470f7e2a1f3eefcc16e24cf1338f606 2023-11-26T06:58:23.631 DEBUG:teuthology.task.kernel:failed to parse current kernel version 2023-11-26T06:58:23.631 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=kernel&flavor=default&distros=ubuntu%2F22.04%2Fx86_64&sha1=8247bbb5b470f7e2a1f3eefcc16e24cf1338f606 2023-11-26T06:58:23.767 INFO:teuthology.task.kernel:Installing linux-firmware on mon.a... 2023-11-26T06:58:23.767 DEBUG:teuthology.orchestra.run.smithi143:> cd /lib/firmware/updates && test -d .git && test '!' -s .git/HEAD && sudo rm -rf .git ; sudo install -d -m0755 /lib/firmware/updates && cd /lib/firmware/updates && sudo git init 2023-11-26T06:58:23.769 INFO:teuthology.task.kernel:Installing linux-firmware on mon.b... 2023-11-26T06:58:23.770 DEBUG:teuthology.orchestra.run.smithi188:> cd /lib/firmware/updates && test -d .git && test '!' -s .git/HEAD && sudo rm -rf .git ; sudo install -d -m0755 /lib/firmware/updates && cd /lib/firmware/updates && sudo git init 2023-11-26T06:58:23.780 INFO:teuthology.orchestra.run.smithi188.stderr:bash: line 1: cd: /lib/firmware/updates: No such file or directory 2023-11-26T06:58:23.794 INFO:teuthology.orchestra.run.smithi143.stderr:bash: line 1: cd: /lib/firmware/updates: No such file or directory 2023-11-26T06:58:23.917 INFO:teuthology.orchestra.run.smithi188.stderr:hint: Using 'master' as the name for the initial branch. This default branch name 2023-11-26T06:58:23.917 INFO:teuthology.orchestra.run.smithi188.stderr:hint: is subject to change. To configure the initial branch name to use in all 2023-11-26T06:58:23.917 INFO:teuthology.orchestra.run.smithi188.stderr:hint: of your new repositories, which will suppress this warning, call: 2023-11-26T06:58:23.918 INFO:teuthology.orchestra.run.smithi188.stderr:hint: 2023-11-26T06:58:23.918 INFO:teuthology.orchestra.run.smithi188.stderr:hint: git config --global init.defaultBranch 2023-11-26T06:58:23.918 INFO:teuthology.orchestra.run.smithi188.stderr:hint: 2023-11-26T06:58:23.918 INFO:teuthology.orchestra.run.smithi188.stderr:hint: Names commonly chosen instead of 'master' are 'main', 'trunk' and 2023-11-26T06:58:23.918 INFO:teuthology.orchestra.run.smithi188.stderr:hint: 'development'. The just-created branch can be renamed via this command: 2023-11-26T06:58:23.918 INFO:teuthology.orchestra.run.smithi188.stderr:hint: 2023-11-26T06:58:23.918 INFO:teuthology.orchestra.run.smithi188.stderr:hint: git branch -m 2023-11-26T06:58:23.919 INFO:teuthology.orchestra.run.smithi188.stdout:Initialized empty Git repository in /usr/lib/firmware/updates/.git/ 2023-11-26T06:58:23.919 INFO:teuthology.orchestra.run.smithi143.stderr:hint: Using 'master' as the name for the initial branch. This default branch name 2023-11-26T06:58:23.919 INFO:teuthology.orchestra.run.smithi143.stderr:hint: is subject to change. To configure the initial branch name to use in all 2023-11-26T06:58:23.919 INFO:teuthology.orchestra.run.smithi143.stderr:hint: of your new repositories, which will suppress this warning, call: 2023-11-26T06:58:23.919 INFO:teuthology.orchestra.run.smithi143.stderr:hint: 2023-11-26T06:58:23.919 INFO:teuthology.orchestra.run.smithi143.stderr:hint: git config --global init.defaultBranch 2023-11-26T06:58:23.920 INFO:teuthology.orchestra.run.smithi143.stderr:hint: 2023-11-26T06:58:23.920 INFO:teuthology.orchestra.run.smithi143.stderr:hint: Names commonly chosen instead of 'master' are 'main', 'trunk' and 2023-11-26T06:58:23.920 INFO:teuthology.orchestra.run.smithi143.stderr:hint: 'development'. The just-created branch can be renamed via this command: 2023-11-26T06:58:23.920 INFO:teuthology.orchestra.run.smithi143.stderr:hint: 2023-11-26T06:58:23.920 INFO:teuthology.orchestra.run.smithi143.stderr:hint: git branch -m 2023-11-26T06:58:23.921 DEBUG:teuthology.orchestra.run.smithi188:> sudo git --git-dir=/lib/firmware/updates/.git config --get remote.origin.url >/dev/null || sudo git --git-dir=/lib/firmware/updates/.git remote add origin git://git.kernel.org/pub/scm/linux/kernel/git/firmware/linux-firmware.git 2023-11-26T06:58:23.922 INFO:teuthology.orchestra.run.smithi143.stdout:Initialized empty Git repository in /usr/lib/firmware/updates/.git/ 2023-11-26T06:58:23.925 DEBUG:teuthology.orchestra.run.smithi143:> sudo git --git-dir=/lib/firmware/updates/.git config --get remote.origin.url >/dev/null || sudo git --git-dir=/lib/firmware/updates/.git remote add origin git://git.kernel.org/pub/scm/linux/kernel/git/firmware/linux-firmware.git 2023-11-26T06:58:23.989 DEBUG:teuthology.orchestra.run.smithi143:> sudo git --git-dir=/lib/firmware/updates/.git remote set-url origin git://git.kernel.org/pub/scm/linux/kernel/git/firmware/linux-firmware.git >/dev/null 2023-11-26T06:58:23.990 DEBUG:teuthology.orchestra.run.smithi188:> sudo git --git-dir=/lib/firmware/updates/.git remote set-url origin git://git.kernel.org/pub/scm/linux/kernel/git/firmware/linux-firmware.git >/dev/null 2023-11-26T06:58:24.043 DEBUG:teuthology.orchestra.run.smithi143:> cd /lib/firmware/updates && sudo git fetch origin && sudo git reset --hard origin/main 2023-11-26T06:58:24.047 DEBUG:teuthology.orchestra.run.smithi188:> cd /lib/firmware/updates && sudo git fetch origin && sudo git reset --hard origin/main 2023-11-26T07:02:07.651 INFO:teuthology.orchestra.run.smithi188.stderr:From git://git.kernel.org/pub/scm/linux/kernel/git/firmware/linux-firmware 2023-11-26T07:02:07.651 INFO:teuthology.orchestra.run.smithi188.stderr: * [new branch] main -> origin/main 2023-11-26T07:02:07.682 INFO:teuthology.orchestra.run.smithi188.stderr: * [new tag] 20190312 -> 20190312 2023-11-26T07:02:07.682 INFO:teuthology.orchestra.run.smithi188.stderr: * [new tag] 20190416 -> 20190416 2023-11-26T07:02:07.682 INFO:teuthology.orchestra.run.smithi188.stderr: * [new tag] 20190514 -> 20190514 2023-11-26T07:02:07.682 INFO:teuthology.orchestra.run.smithi188.stderr: * [new tag] 20190618 -> 20190618 2023-11-26T07:02:07.683 INFO:teuthology.orchestra.run.smithi188.stderr: * [new tag] 20190717 -> 20190717 2023-11-26T07:02:07.683 INFO:teuthology.orchestra.run.smithi188.stderr: * [new tag] 20190815 -> 20190815 2023-11-26T07:02:07.683 INFO:teuthology.orchestra.run.smithi188.stderr: * [new tag] 20190923 -> 20190923 2023-11-26T07:02:07.683 INFO:teuthology.orchestra.run.smithi188.stderr: * [new tag] 20191022 -> 20191022 2023-11-26T07:02:07.683 INFO:teuthology.orchestra.run.smithi188.stderr: * [new tag] 20191215 -> 20191215 2023-11-26T07:02:07.683 INFO:teuthology.orchestra.run.smithi188.stderr: * [new tag] 20200122 -> 20200122 2023-11-26T07:02:07.684 INFO:teuthology.orchestra.run.smithi188.stderr: * [new tag] 20200316 -> 20200316 2023-11-26T07:02:07.684 INFO:teuthology.orchestra.run.smithi188.stderr: * [new tag] 20200421 -> 20200421 2023-11-26T07:02:07.684 INFO:teuthology.orchestra.run.smithi188.stderr: * [new tag] 20200519 -> 20200519 2023-11-26T07:02:07.685 INFO:teuthology.orchestra.run.smithi188.stderr: * [new tag] 20200619 -> 20200619 2023-11-26T07:02:07.685 INFO:teuthology.orchestra.run.smithi188.stderr: * [new tag] 20200721 -> 20200721 2023-11-26T07:02:07.685 INFO:teuthology.orchestra.run.smithi188.stderr: * [new tag] 20200817 -> 20200817 2023-11-26T07:02:07.685 INFO:teuthology.orchestra.run.smithi188.stderr: * [new tag] 20200918 -> 20200918 2023-11-26T07:02:07.685 INFO:teuthology.orchestra.run.smithi188.stderr: * [new tag] 20201022 -> 20201022 2023-11-26T07:02:07.686 INFO:teuthology.orchestra.run.smithi188.stderr: * [new tag] 20201118 -> 20201118 2023-11-26T07:02:07.686 INFO:teuthology.orchestra.run.smithi188.stderr: * [new tag] 20201218 -> 20201218 2023-11-26T07:02:07.686 INFO:teuthology.orchestra.run.smithi188.stderr: * [new tag] 20210208 -> 20210208 2023-11-26T07:02:07.686 INFO:teuthology.orchestra.run.smithi188.stderr: * [new tag] 20210315 -> 20210315 2023-11-26T07:02:07.686 INFO:teuthology.orchestra.run.smithi188.stderr: * [new tag] 20210511 -> 20210511 2023-11-26T07:02:07.686 INFO:teuthology.orchestra.run.smithi188.stderr: * [new tag] 20210716 -> 20210716 2023-11-26T07:02:07.686 INFO:teuthology.orchestra.run.smithi188.stderr: * [new tag] 20210818 -> 20210818 2023-11-26T07:02:07.687 INFO:teuthology.orchestra.run.smithi188.stderr: * [new tag] 20210919 -> 20210919 2023-11-26T07:02:07.688 INFO:teuthology.orchestra.run.smithi188.stderr: * [new tag] 20211027 -> 20211027 2023-11-26T07:02:07.688 INFO:teuthology.orchestra.run.smithi188.stderr: * [new tag] 20211216 -> 20211216 2023-11-26T07:02:07.688 INFO:teuthology.orchestra.run.smithi188.stderr: * [new tag] 20220209 -> 20220209 2023-11-26T07:02:07.689 INFO:teuthology.orchestra.run.smithi188.stderr: * [new tag] 20220310 -> 20220310 2023-11-26T07:02:07.689 INFO:teuthology.orchestra.run.smithi188.stderr: * [new tag] 20220411 -> 20220411 2023-11-26T07:02:07.689 INFO:teuthology.orchestra.run.smithi188.stderr: * [new tag] 20220509 -> 20220509 2023-11-26T07:02:07.689 INFO:teuthology.orchestra.run.smithi188.stderr: * [new tag] 20220610 -> 20220610 2023-11-26T07:02:07.690 INFO:teuthology.orchestra.run.smithi188.stderr: * [new tag] 20220708 -> 20220708 2023-11-26T07:02:07.691 INFO:teuthology.orchestra.run.smithi188.stderr: * [new tag] 20220815 -> 20220815 2023-11-26T07:02:07.692 INFO:teuthology.orchestra.run.smithi188.stderr: * [new tag] 20220913 -> 20220913 2023-11-26T07:02:07.692 INFO:teuthology.orchestra.run.smithi188.stderr: * [new tag] 20221012 -> 20221012 2023-11-26T07:02:07.692 INFO:teuthology.orchestra.run.smithi188.stderr: * [new tag] 20221109 -> 20221109 2023-11-26T07:02:07.692 INFO:teuthology.orchestra.run.smithi188.stderr: * [new tag] 20221214 -> 20221214 2023-11-26T07:02:07.693 INFO:teuthology.orchestra.run.smithi188.stderr: * [new tag] 20230117 -> 20230117 2023-11-26T07:02:07.693 INFO:teuthology.orchestra.run.smithi188.stderr: * [new tag] 20230210 -> 20230210 2023-11-26T07:02:07.693 INFO:teuthology.orchestra.run.smithi188.stderr: * [new tag] 20230310 -> 20230310 2023-11-26T07:02:07.693 INFO:teuthology.orchestra.run.smithi188.stderr: * [new tag] 20230404 -> 20230404 2023-11-26T07:02:07.693 INFO:teuthology.orchestra.run.smithi188.stderr: * [new tag] 20230515 -> 20230515 2023-11-26T07:02:07.693 INFO:teuthology.orchestra.run.smithi188.stderr: * [new tag] 20230625 -> 20230625 2023-11-26T07:02:07.693 INFO:teuthology.orchestra.run.smithi188.stderr: * [new tag] 20230804 -> 20230804 2023-11-26T07:02:07.693 INFO:teuthology.orchestra.run.smithi188.stderr: * [new tag] 20230919 -> 20230919 2023-11-26T07:02:07.693 INFO:teuthology.orchestra.run.smithi188.stderr: * [new tag] 20231030 -> 20231030 2023-11-26T07:02:07.693 INFO:teuthology.orchestra.run.smithi188.stderr: * [new tag] 20231111 -> 20231111 2023-11-26T07:02:11.669 INFO:teuthology.orchestra.run.smithi188.stderr:Updating files: 51% (1669/3248) Updating files: 52% (1689/3248) Updating files: 53% (1722/3248) Updating files: 54% (1754/3248) Updating files: 55% (1787/3248) Updating files: 56% (1819/3248) Updating files: 57% (1852/3248) Updating files: 58% (1884/3248) Updating files: 59% (1917/3248) Updating files: 60% (1949/3248) Updating files: 61% (1982/3248) Updating files: 62% (2014/3248) Updating files: 63% (2047/3248) Updating files: 64% (2079/3248) Updating files: 65% (2112/3248) Updating files: 65% (2113/3248) Updating files: 66% (2144/3248) Updating files: 67% (2177/3248) Updating files: 68% (2209/3248) Updating files: 69% (2242/3248) Updating files: 70% (2274/3248) Updating files: 71% (2307/3248) Updating files: 72% (2339/3248) Updating files: 73% (2372/3248) Updating files: 74% (2404/3248) Updating files: 75% (2436/3248) Updating files: 76% (2469/3248) Updating files: 77% (2501/3248) Updating files: 78% (2534/3248) Updating files: 79% (2566/3248) Updating files: 80% (2599/3248) Updating files: 81% (2631/3248) Updating files: 81% (2648/3248) Updating files: 82% (2664/3248) Updating files: 83% (2696/3248) Updating files: 84% (2729/3248) Updating files: 85% (2761/3248) Updating files: 86% (2794/3248) Updating files: 87% (2826/3248) Updating files: 88% (2859/3248) Updating files: 89% (2891/3248) Updating files: 90% (2924/3248) Updating files: 91% (2956/3248) Updating files: 92% (2989/3248) Updating files: 93% (3021/3248) Updating files: 94% (3054/3248) Updating files: 95% (3086/3248) Updating files: 96% (3119/3248) Updating files: 97% (3151/3248) Updating files: 98% (3184/3248) Updating files: 99% (3216/3248) Updating files: 100% (3248/3248) Updating files: 100% (3248/3248), done. 2023-11-26T07:02:11.674 INFO:teuthology.orchestra.run.smithi188.stdout:HEAD is now at 9552083a Merge branch 'robot/pr-0-1700470117' into 'main' 2023-11-26T07:02:11.697 INFO:teuthology.task.kernel:Downloading kernel 8247bbb5b470f7e2a1f3eefcc16e24cf1338f606 on mon.b... 2023-11-26T07:02:11.698 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=kernel&flavor=default&distros=ubuntu%2F22.04%2Fx86_64&sha1=8247bbb5b470f7e2a1f3eefcc16e24cf1338f606 2023-11-26T07:02:11.810 INFO:teuthology.orchestra.run.smithi143.stderr:From git://git.kernel.org/pub/scm/linux/kernel/git/firmware/linux-firmware 2023-11-26T07:02:11.811 INFO:teuthology.orchestra.run.smithi143.stderr: * [new branch] main -> origin/main 2023-11-26T07:02:11.842 INFO:teuthology.orchestra.run.smithi143.stderr: * [new tag] 20190312 -> 20190312 2023-11-26T07:02:11.843 INFO:teuthology.orchestra.run.smithi143.stderr: * [new tag] 20190416 -> 20190416 2023-11-26T07:02:11.843 INFO:teuthology.orchestra.run.smithi143.stderr: * [new tag] 20190514 -> 20190514 2023-11-26T07:02:11.843 INFO:teuthology.orchestra.run.smithi143.stderr: * [new tag] 20190618 -> 20190618 2023-11-26T07:02:11.843 INFO:teuthology.orchestra.run.smithi143.stderr: * [new tag] 20190717 -> 20190717 2023-11-26T07:02:11.843 INFO:teuthology.orchestra.run.smithi143.stderr: * [new tag] 20190815 -> 20190815 2023-11-26T07:02:11.843 INFO:teuthology.orchestra.run.smithi143.stderr: * [new tag] 20190923 -> 20190923 2023-11-26T07:02:11.843 INFO:teuthology.orchestra.run.smithi143.stderr: * [new tag] 20191022 -> 20191022 2023-11-26T07:02:11.844 INFO:teuthology.orchestra.run.smithi143.stderr: * [new tag] 20191215 -> 20191215 2023-11-26T07:02:11.844 INFO:teuthology.orchestra.run.smithi143.stderr: * [new tag] 20200122 -> 20200122 2023-11-26T07:02:11.844 INFO:teuthology.orchestra.run.smithi143.stderr: * [new tag] 20200316 -> 20200316 2023-11-26T07:02:11.850 INFO:teuthology.orchestra.run.smithi143.stderr: * [new tag] 20200421 -> 20200421 2023-11-26T07:02:11.850 INFO:teuthology.orchestra.run.smithi143.stderr: * [new tag] 20200519 -> 20200519 2023-11-26T07:02:11.850 INFO:teuthology.orchestra.run.smithi143.stderr: * [new tag] 20200619 -> 20200619 2023-11-26T07:02:11.850 INFO:teuthology.orchestra.run.smithi143.stderr: * [new tag] 20200721 -> 20200721 2023-11-26T07:02:11.851 INFO:teuthology.orchestra.run.smithi143.stderr: * [new tag] 20200817 -> 20200817 2023-11-26T07:02:11.851 INFO:teuthology.orchestra.run.smithi143.stderr: * [new tag] 20200918 -> 20200918 2023-11-26T07:02:11.851 INFO:teuthology.orchestra.run.smithi143.stderr: * [new tag] 20201022 -> 20201022 2023-11-26T07:02:11.851 INFO:teuthology.orchestra.run.smithi143.stderr: * [new tag] 20201118 -> 20201118 2023-11-26T07:02:11.851 INFO:teuthology.orchestra.run.smithi143.stderr: * [new tag] 20201218 -> 20201218 2023-11-26T07:02:11.851 INFO:teuthology.orchestra.run.smithi143.stderr: * [new tag] 20210208 -> 20210208 2023-11-26T07:02:11.852 INFO:teuthology.orchestra.run.smithi143.stderr: * [new tag] 20210315 -> 20210315 2023-11-26T07:02:11.852 INFO:teuthology.orchestra.run.smithi143.stderr: * [new tag] 20210511 -> 20210511 2023-11-26T07:02:11.852 INFO:teuthology.orchestra.run.smithi143.stderr: * [new tag] 20210716 -> 20210716 2023-11-26T07:02:11.852 INFO:teuthology.orchestra.run.smithi143.stderr: * [new tag] 20210818 -> 20210818 2023-11-26T07:02:11.852 INFO:teuthology.orchestra.run.smithi143.stderr: * [new tag] 20210919 -> 20210919 2023-11-26T07:02:11.852 INFO:teuthology.orchestra.run.smithi143.stderr: * [new tag] 20211027 -> 20211027 2023-11-26T07:02:11.853 INFO:teuthology.orchestra.run.smithi143.stderr: * [new tag] 20211216 -> 20211216 2023-11-26T07:02:11.853 INFO:teuthology.orchestra.run.smithi143.stderr: * [new tag] 20220209 -> 20220209 2023-11-26T07:02:11.853 INFO:teuthology.orchestra.run.smithi143.stderr: * [new tag] 20220310 -> 20220310 2023-11-26T07:02:11.853 INFO:teuthology.orchestra.run.smithi143.stderr: * [new tag] 20220411 -> 20220411 2023-11-26T07:02:11.853 INFO:teuthology.orchestra.run.smithi143.stderr: * [new tag] 20220509 -> 20220509 2023-11-26T07:02:11.853 INFO:teuthology.orchestra.run.smithi143.stderr: * [new tag] 20220610 -> 20220610 2023-11-26T07:02:11.853 INFO:teuthology.orchestra.run.smithi143.stderr: * [new tag] 20220708 -> 20220708 2023-11-26T07:02:11.853 INFO:teuthology.orchestra.run.smithi143.stderr: * [new tag] 20220815 -> 20220815 2023-11-26T07:02:11.854 INFO:teuthology.orchestra.run.smithi143.stderr: * [new tag] 20220913 -> 20220913 2023-11-26T07:02:11.854 INFO:teuthology.orchestra.run.smithi143.stderr: * [new tag] 20221012 -> 20221012 2023-11-26T07:02:11.854 INFO:teuthology.orchestra.run.smithi143.stderr: * [new tag] 20221109 -> 20221109 2023-11-26T07:02:11.854 INFO:teuthology.orchestra.run.smithi143.stderr: * [new tag] 20221214 -> 20221214 2023-11-26T07:02:11.854 INFO:teuthology.orchestra.run.smithi143.stderr: * [new tag] 20230117 -> 20230117 2023-11-26T07:02:11.854 INFO:teuthology.orchestra.run.smithi143.stderr: * [new tag] 20230210 -> 20230210 2023-11-26T07:02:11.854 INFO:teuthology.orchestra.run.smithi143.stderr: * [new tag] 20230310 -> 20230310 2023-11-26T07:02:11.854 INFO:teuthology.orchestra.run.smithi143.stderr: * [new tag] 20230404 -> 20230404 2023-11-26T07:02:11.854 INFO:teuthology.orchestra.run.smithi143.stderr: * [new tag] 20230515 -> 20230515 2023-11-26T07:02:11.854 INFO:teuthology.orchestra.run.smithi143.stderr: * [new tag] 20230625 -> 20230625 2023-11-26T07:02:11.854 INFO:teuthology.orchestra.run.smithi143.stderr: * [new tag] 20230804 -> 20230804 2023-11-26T07:02:11.854 INFO:teuthology.orchestra.run.smithi143.stderr: * [new tag] 20230919 -> 20230919 2023-11-26T07:02:11.855 INFO:teuthology.orchestra.run.smithi143.stderr: * [new tag] 20231030 -> 20231030 2023-11-26T07:02:11.855 INFO:teuthology.orchestra.run.smithi143.stderr: * [new tag] 20231111 -> 20231111 2023-11-26T07:02:11.857 INFO:teuthology.task.kernel:fetching, builder baseurl is https://3.chacra.ceph.com/r/kernel/testing/8247bbb5b470f7e2a1f3eefcc16e24cf1338f606/ubuntu/jammy/flavors/default/pool/main/l/linux-6.6.0-g8247bbb5b470/ 2023-11-26T07:02:11.857 DEBUG:teuthology.orchestra.run.smithi188:> rm -f /tmp/linux-image.deb && echo linux-image-6.6.0-g8247bbb5b470_6.6.0-g8247bbb5b470-1_amd64.deb | wget -nv -O /tmp/linux-image.deb --base=https://3.chacra.ceph.com/r/kernel/testing/8247bbb5b470f7e2a1f3eefcc16e24cf1338f606/ubuntu/jammy/flavors/default/pool/main/l/linux-6.6.0-g8247bbb5b470/ --input-file=- 2023-11-26T07:02:11.860 DEBUG:teuthology.task.kernel:Waiting for download/copy to ubuntu@smithi188.front.sepia.ceph.com to complete... 2023-11-26T07:02:15.836 INFO:teuthology.orchestra.run.smithi143.stderr:Updating files: 51% (1665/3248) Updating files: 52% (1689/3248) Updating files: 53% (1722/3248) Updating files: 54% (1754/3248) Updating files: 55% (1787/3248) Updating files: 56% (1819/3248) Updating files: 57% (1852/3248) Updating files: 58% (1884/3248) Updating files: 59% (1917/3248) Updating files: 60% (1949/3248) Updating files: 61% (1982/3248) Updating files: 62% (2014/3248) Updating files: 63% (2047/3248) Updating files: 64% (2079/3248) Updating files: 64% (2109/3248) Updating files: 65% (2112/3248) Updating files: 66% (2144/3248) Updating files: 67% (2177/3248) Updating files: 68% (2209/3248) Updating files: 69% (2242/3248) Updating files: 70% (2274/3248) Updating files: 71% (2307/3248) Updating files: 72% (2339/3248) Updating files: 73% (2372/3248) Updating files: 74% (2404/3248) Updating files: 75% (2436/3248) Updating files: 76% (2469/3248) Updating files: 77% (2501/3248) Updating files: 78% (2534/3248) Updating files: 79% (2566/3248) Updating files: 80% (2599/3248) Updating files: 81% (2631/3248) Updating files: 81% (2648/3248) Updating files: 82% (2664/3248) Updating files: 83% (2696/3248) Updating files: 84% (2729/3248) Updating files: 85% (2761/3248) Updating files: 86% (2794/3248) Updating files: 87% (2826/3248) Updating files: 88% (2859/3248) Updating files: 89% (2891/3248) Updating files: 90% (2924/3248) Updating files: 91% (2956/3248) Updating files: 92% (2989/3248) Updating files: 93% (3021/3248) Updating files: 94% (3054/3248) Updating files: 95% (3086/3248) Updating files: 96% (3119/3248) Updating files: 97% (3151/3248) Updating files: 98% (3184/3248) Updating files: 99% (3216/3248) Updating files: 100% (3248/3248) Updating files: 100% (3248/3248), done. 2023-11-26T07:02:15.841 INFO:teuthology.orchestra.run.smithi143.stdout:HEAD is now at 9552083a Merge branch 'robot/pr-0-1700470117' into 'main' 2023-11-26T07:02:15.861 INFO:teuthology.task.kernel:Downloading kernel 8247bbb5b470f7e2a1f3eefcc16e24cf1338f606 on mon.a... 2023-11-26T07:02:15.861 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=kernel&flavor=default&distros=ubuntu%2F22.04%2Fx86_64&sha1=8247bbb5b470f7e2a1f3eefcc16e24cf1338f606 2023-11-26T07:02:15.895 INFO:teuthology.orchestra.run.smithi188.stderr:2023-11-26 07:02:15 URL:https://3.chacra.ceph.com/r/kernel/testing/8247bbb5b470f7e2a1f3eefcc16e24cf1338f606/ubuntu/jammy/flavors/default/pool/main/l/linux-6.6.0-g8247bbb5b470/linux-image-6.6.0-g8247bbb5b470_6.6.0-g8247bbb5b470-1_amd64.deb [38938726/38938726] -> "/tmp/linux-image.deb" [1] 2023-11-26T07:02:15.895 INFO:teuthology.orchestra.run.smithi188.stderr:FINISHED --2023-11-26 07:02:15-- 2023-11-26T07:02:15.895 INFO:teuthology.orchestra.run.smithi188.stderr:Total wall clock time: 4.0s 2023-11-26T07:02:15.896 INFO:teuthology.orchestra.run.smithi188.stderr:Downloaded: 1 files, 37M in 3.7s (10.0 MB/s) 2023-11-26T07:02:15.897 INFO:teuthology.task.kernel:Installing kernel 8247bbb5b470f7e2a1f3eefcc16e24cf1338f606 on mon.b... 2023-11-26T07:02:15.898 DEBUG:teuthology.orchestra.run.smithi188:> sudo dpkg -i /tmp/linux-image.deb 2023-11-26T07:02:16.009 INFO:teuthology.task.kernel:fetching, builder baseurl is https://3.chacra.ceph.com/r/kernel/testing/8247bbb5b470f7e2a1f3eefcc16e24cf1338f606/ubuntu/jammy/flavors/default/pool/main/l/linux-6.6.0-g8247bbb5b470/ 2023-11-26T07:02:16.010 DEBUG:teuthology.orchestra.run.smithi143:> rm -f /tmp/linux-image.deb && echo linux-image-6.6.0-g8247bbb5b470_6.6.0-g8247bbb5b470-1_amd64.deb | wget -nv -O /tmp/linux-image.deb --base=https://3.chacra.ceph.com/r/kernel/testing/8247bbb5b470f7e2a1f3eefcc16e24cf1338f606/ubuntu/jammy/flavors/default/pool/main/l/linux-6.6.0-g8247bbb5b470/ --input-file=- 2023-11-26T07:02:16.012 DEBUG:teuthology.task.kernel:Waiting for download/copy to ubuntu@smithi143.front.sepia.ceph.com to complete... 2023-11-26T07:02:16.219 INFO:teuthology.orchestra.run.smithi188.stdout:Selecting previously unselected package linux-image-6.6.0-g8247bbb5b470. 2023-11-26T07:02:18.283 INFO:teuthology.orchestra.run.smithi188.stdout:(Reading database ... 129542 files and directories currently installed.) 2023-11-26T07:02:18.287 INFO:teuthology.orchestra.run.smithi188.stdout:Preparing to unpack /tmp/linux-image.deb ... 2023-11-26T07:02:18.531 INFO:teuthology.orchestra.run.smithi188.stdout:Unpacking linux-image-6.6.0-g8247bbb5b470 (6.6.0-g8247bbb5b470-1) ... 2023-11-26T07:02:18.986 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26 07:02:18 URL:https://3.chacra.ceph.com/r/kernel/testing/8247bbb5b470f7e2a1f3eefcc16e24cf1338f606/ubuntu/jammy/flavors/default/pool/main/l/linux-6.6.0-g8247bbb5b470/linux-image-6.6.0-g8247bbb5b470_6.6.0-g8247bbb5b470-1_amd64.deb [38938726/38938726] -> "/tmp/linux-image.deb" [1] 2023-11-26T07:02:18.987 INFO:teuthology.orchestra.run.smithi143.stderr:FINISHED --2023-11-26 07:02:18-- 2023-11-26T07:02:18.987 INFO:teuthology.orchestra.run.smithi143.stderr:Total wall clock time: 2.9s 2023-11-26T07:02:18.987 INFO:teuthology.orchestra.run.smithi143.stderr:Downloaded: 1 files, 37M in 2.7s (13.6 MB/s) 2023-11-26T07:02:18.990 INFO:teuthology.task.kernel:Installing kernel 8247bbb5b470f7e2a1f3eefcc16e24cf1338f606 on mon.a... 2023-11-26T07:02:18.990 DEBUG:teuthology.orchestra.run.smithi143:> sudo dpkg -i /tmp/linux-image.deb 2023-11-26T07:02:19.275 INFO:teuthology.orchestra.run.smithi143.stdout:Selecting previously unselected package linux-image-6.6.0-g8247bbb5b470. 2023-11-26T07:02:21.331 INFO:teuthology.orchestra.run.smithi143.stdout:(Reading database ... 129542 files and directories currently installed.) 2023-11-26T07:02:21.338 INFO:teuthology.orchestra.run.smithi143.stdout:Preparing to unpack /tmp/linux-image.deb ... 2023-11-26T07:02:21.475 INFO:teuthology.orchestra.run.smithi143.stdout:Unpacking linux-image-6.6.0-g8247bbb5b470 (6.6.0-g8247bbb5b470-1) ... 2023-11-26T07:02:21.797 INFO:teuthology.orchestra.run.smithi188.stdout:Setting up linux-image-6.6.0-g8247bbb5b470 (6.6.0-g8247bbb5b470-1) ... 2023-11-26T07:02:21.898 INFO:teuthology.orchestra.run.smithi188.stderr:update-initramfs: Generating /boot/initrd.img-6.6.0-g8247bbb5b470 2023-11-26T07:02:25.347 INFO:teuthology.orchestra.run.smithi143.stdout:Setting up linux-image-6.6.0-g8247bbb5b470 (6.6.0-g8247bbb5b470-1) ... 2023-11-26T07:02:25.456 INFO:teuthology.orchestra.run.smithi143.stderr:update-initramfs: Generating /boot/initrd.img-6.6.0-g8247bbb5b470 2023-11-26T07:02:43.214 INFO:teuthology.orchestra.run.smithi188.stderr:Sourcing file `/etc/default/grub' 2023-11-26T07:02:43.233 INFO:teuthology.orchestra.run.smithi188.stderr:Sourcing file `/etc/default/grub.d/init-select.cfg' 2023-11-26T07:02:43.271 INFO:teuthology.orchestra.run.smithi188.stderr:Generating grub configuration file ... 2023-11-26T07:02:43.646 INFO:teuthology.orchestra.run.smithi188.stderr:Found linux image: /boot/vmlinuz-6.6.0-g8247bbb5b470 2023-11-26T07:02:43.662 INFO:teuthology.orchestra.run.smithi188.stderr:Found initrd image: /boot/initrd.img-6.6.0-g8247bbb5b470 2023-11-26T07:02:44.032 INFO:teuthology.orchestra.run.smithi188.stderr:Found linux image: /boot/vmlinuz-5.15.0-56-generic 2023-11-26T07:02:44.046 INFO:teuthology.orchestra.run.smithi188.stderr:Found initrd image: /boot/initrd.img-5.15.0-56-generic 2023-11-26T07:02:44.108 INFO:teuthology.orchestra.run.smithi188.stderr:Found linux image: /boot/vmlinuz-5.4.0-135-generic 2023-11-26T07:02:44.114 INFO:teuthology.orchestra.run.smithi188.stderr:Found initrd image: /boot/initrd.img-5.4.0-135-generic 2023-11-26T07:02:44.274 INFO:teuthology.orchestra.run.smithi188.stderr:Warning: os-prober will not be executed to detect other bootable partitions. 2023-11-26T07:02:44.274 INFO:teuthology.orchestra.run.smithi188.stderr:Systems on them will not be added to the GRUB boot configuration. 2023-11-26T07:02:44.274 INFO:teuthology.orchestra.run.smithi188.stderr:Check GRUB_DISABLE_OS_PROBER documentation entry. 2023-11-26T07:02:44.309 INFO:teuthology.orchestra.run.smithi188.stderr:done 2023-11-26T07:02:44.489 DEBUG:teuthology.orchestra.run.smithi188:> dpkg-deb -c /tmp/linux-image.deb 2023-11-26T07:02:44.497 INFO:teuthology.orchestra.run.smithi188.stdout:drwxr-xr-x root/root 0 2023-11-23 03:47 ./ 2023-11-26T07:02:44.497 INFO:teuthology.orchestra.run.smithi188.stdout:drwxr-xr-x root/root 0 2023-11-23 03:47 ./boot/ 2023-11-26T07:02:44.497 INFO:teuthology.orchestra.run.smithi188.stdout:-rw-r--r-- root/root 4415617 2023-11-23 03:47 ./boot/System.map-6.6.0-g8247bbb5b470 2023-11-26T07:02:44.509 INFO:teuthology.orchestra.run.smithi188.stdout:-rw-r--r-- root/root 169214 2023-11-23 03:47 ./boot/config-6.6.0-g8247bbb5b470 2023-11-26T07:02:44.510 INFO:teuthology.orchestra.run.smithi188.stdout:-rw-r--r-- root/root 11352832 2023-11-23 03:47 ./boot/vmlinuz-6.6.0-g8247bbb5b470 2023-11-26T07:02:44.528 INFO:teuthology.orchestra.run.smithi188.stdout:drwxr-xr-x root/root 0 2023-11-23 03:47 ./etc/ 2023-11-26T07:02:44.528 INFO:teuthology.orchestra.run.smithi188.stdout:drwxr-xr-x root/root 0 2023-11-23 03:47 ./etc/kernel/ 2023-11-26T07:02:44.529 INFO:teuthology.orchestra.run.smithi188.stdout:drwxr-xr-x root/root 0 2023-11-23 03:47 ./etc/kernel/postinst.d/ 2023-11-26T07:02:44.529 INFO:teuthology.orchestra.run.smithi188.stdout:drwxr-xr-x root/root 0 2023-11-23 03:47 ./etc/kernel/postrm.d/ 2023-11-26T07:02:44.529 INFO:teuthology.orchestra.run.smithi188.stdout:drwxr-xr-x root/root 0 2023-11-23 03:47 ./etc/kernel/preinst.d/ 2023-11-26T07:02:44.529 INFO:teuthology.orchestra.run.smithi188.stdout:drwxr-xr-x root/root 0 2023-11-23 03:47 ./etc/kernel/prerm.d/ 2023-11-26T07:02:44.529 INFO:teuthology.orchestra.run.smithi188.stdout:drwxr-xr-x root/root 0 2023-11-23 03:47 ./lib/ 2023-11-26T07:02:44.529 INFO:teuthology.orchestra.run.smithi188.stdout:drwxr-xr-x root/root 0 2023-11-23 03:47 ./lib/modules/ 2023-11-26T07:02:44.529 INFO:teuthology.orchestra.run.smithi188.stdout:drwxr-xr-x root/root 0 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/ 2023-11-26T07:02:44.530 INFO:teuthology.orchestra.run.smithi188.stdout:drwxr-xr-x root/root 0 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/ 2023-11-26T07:02:44.530 INFO:teuthology.orchestra.run.smithi188.stdout:drwxr-xr-x root/root 0 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/arch/ 2023-11-26T07:02:44.530 INFO:teuthology.orchestra.run.smithi188.stdout:drwxr-xr-x root/root 0 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/arch/x86/ 2023-11-26T07:02:44.530 INFO:teuthology.orchestra.run.smithi188.stdout:drwxr-xr-x root/root 0 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/arch/x86/crypto/ 2023-11-26T07:02:44.530 INFO:teuthology.orchestra.run.smithi188.stdout:-rw-r--r-- root/root 490233 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/arch/x86/crypto/aesni-intel.ko 2023-11-26T07:02:44.533 INFO:teuthology.orchestra.run.smithi188.stdout:-rw-r--r-- root/root 25329 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/arch/x86/crypto/blowfish-x86_64.ko 2023-11-26T07:02:44.533 INFO:teuthology.orchestra.run.smithi188.stdout:-rw-r--r-- root/root 29529 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/arch/x86/crypto/camellia-aesni-avx-x86_64.ko 2023-11-26T07:02:44.533 INFO:teuthology.orchestra.run.smithi188.stdout:-rw-r--r-- root/root 30313 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/arch/x86/crypto/camellia-aesni-avx2.ko 2023-11-26T07:02:44.533 INFO:teuthology.orchestra.run.smithi188.stdout:-rw-r--r-- root/root 93113 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/arch/x86/crypto/camellia-x86_64.ko 2023-11-26T07:02:44.533 INFO:teuthology.orchestra.run.smithi188.stdout:-rw-r--r-- root/root 106729 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/arch/x86/crypto/cast5-avx-x86_64.ko 2023-11-26T07:02:44.533 INFO:teuthology.orchestra.run.smithi188.stdout:-rw-r--r-- root/root 152241 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/arch/x86/crypto/cast6-avx-x86_64.ko 2023-11-26T07:02:44.533 INFO:teuthology.orchestra.run.smithi188.stdout:-rw-r--r-- root/root 12553 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/arch/x86/crypto/crc32-pclmul.ko 2023-11-26T07:02:44.533 INFO:teuthology.orchestra.run.smithi188.stdout:-rw-r--r-- root/root 13777 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/arch/x86/crypto/crct10dif-pclmul.ko 2023-11-26T07:02:44.534 INFO:teuthology.orchestra.run.smithi188.stdout:-rw-r--r-- root/root 83889 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/arch/x86/crypto/des3_ede-x86_64.ko 2023-11-26T07:02:44.534 INFO:teuthology.orchestra.run.smithi188.stdout:-rw-r--r-- root/root 18625 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/arch/x86/crypto/ghash-clmulni-intel.ko 2023-11-26T07:02:44.534 INFO:teuthology.orchestra.run.smithi188.stdout:-rw-r--r-- root/root 47225 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/arch/x86/crypto/serpent-avx-x86_64.ko 2023-11-26T07:02:44.534 INFO:teuthology.orchestra.run.smithi188.stdout:-rw-r--r-- root/root 47209 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/arch/x86/crypto/serpent-avx2.ko 2023-11-26T07:02:44.534 INFO:teuthology.orchestra.run.smithi188.stdout:-rw-r--r-- root/root 51801 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/arch/x86/crypto/serpent-sse2-x86_64.ko 2023-11-26T07:02:44.534 INFO:teuthology.orchestra.run.smithi188.stdout:-rw-r--r-- root/root 38937 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/arch/x86/crypto/sha1-ssse3.ko 2023-11-26T07:02:44.534 INFO:teuthology.orchestra.run.smithi188.stdout:-rw-r--r-- root/root 40529 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/arch/x86/crypto/sha256-ssse3.ko 2023-11-26T07:02:44.535 INFO:teuthology.orchestra.run.smithi188.stdout:-rw-r--r-- root/root 58585 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/arch/x86/crypto/sha512-ssse3.ko 2023-11-26T07:02:44.535 INFO:teuthology.orchestra.run.smithi188.stdout:-rw-r--r-- root/root 50497 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/arch/x86/crypto/twofish-avx-x86_64.ko 2023-11-26T07:02:44.535 INFO:teuthology.orchestra.run.smithi188.stdout:-rw-r--r-- root/root 32065 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/arch/x86/crypto/twofish-x86_64-3way.ko 2023-11-26T07:02:44.535 INFO:teuthology.orchestra.run.smithi188.stdout:-rw-r--r-- root/root 13625 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/arch/x86/crypto/twofish-x86_64.ko 2023-11-26T07:02:44.535 INFO:teuthology.orchestra.run.smithi188.stdout:drwxr-xr-x root/root 0 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/arch/x86/kernel/ 2023-11-26T07:02:44.535 INFO:teuthology.orchestra.run.smithi188.stdout:drwxr-xr-x root/root 0 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/arch/x86/kernel/cpu/ 2023-11-26T07:02:44.535 INFO:teuthology.orchestra.run.smithi188.stdout:drwxr-xr-x root/root 0 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/arch/x86/kernel/cpu/mce/ 2023-11-26T07:02:44.535 INFO:teuthology.orchestra.run.smithi188.stdout:-rw-r--r-- root/root 40113 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/arch/x86/kernel/cpu/mce/mce-inject.ko 2023-11-26T07:02:44.535 INFO:teuthology.orchestra.run.smithi188.stdout:-rw-r--r-- root/root 13353 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/arch/x86/kernel/cpuid.ko 2023-11-26T07:02:44.535 INFO:teuthology.orchestra.run.smithi188.stdout:-rw-r--r-- root/root 17937 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/arch/x86/kernel/msr.ko 2023-11-26T07:02:44.536 INFO:teuthology.orchestra.run.smithi188.stdout:drwxr-xr-x root/root 0 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/arch/x86/kvm/ 2023-11-26T07:02:44.536 INFO:teuthology.orchestra.run.smithi188.stdout:-rw-r--r-- root/root 354433 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/arch/x86/kvm/kvm-amd.ko 2023-11-26T07:02:44.536 INFO:teuthology.orchestra.run.smithi188.stdout:-rw-r--r-- root/root 881625 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/arch/x86/kvm/kvm-intel.ko 2023-11-26T07:02:44.536 INFO:teuthology.orchestra.run.smithi188.stdout:-rw-r--r-- root/root 2275865 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/arch/x86/kvm/kvm.ko 2023-11-26T07:02:44.607 INFO:teuthology.orchestra.run.smithi188.stdout:drwxr-xr-x root/root 0 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/crypto/ 2023-11-26T07:02:44.607 INFO:teuthology.orchestra.run.smithi188.stdout:-rw-r--r-- root/root 53697 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/crypto/af_alg.ko 2023-11-26T07:02:44.607 INFO:teuthology.orchestra.run.smithi188.stdout:-rw-r--r-- root/root 23097 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/crypto/algif_hash.ko 2023-11-26T07:02:44.607 INFO:teuthology.orchestra.run.smithi188.stdout:-rw-r--r-- root/root 19985 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/crypto/algif_skcipher.ko 2023-11-26T07:02:44.607 INFO:teuthology.orchestra.run.smithi188.stdout:-rw-r--r-- root/root 17209 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/crypto/ansi_cprng.ko 2023-11-26T07:02:44.607 INFO:teuthology.orchestra.run.smithi188.stdout:-rw-r--r-- root/root 19641 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/crypto/anubis.ko 2023-11-26T07:02:44.607 INFO:teuthology.orchestra.run.smithi188.stdout:-rw-r--r-- root/root 10145 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/crypto/arc4.ko 2023-11-26T07:02:44.608 INFO:teuthology.orchestra.run.smithi188.stdout:drwxr-xr-x root/root 0 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/crypto/asymmetric_keys/ 2023-11-26T07:02:44.608 INFO:teuthology.orchestra.run.smithi188.stdout:-rw-r--r-- root/root 10185 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/crypto/asymmetric_keys/pkcs7_test_key.ko 2023-11-26T07:02:44.608 INFO:teuthology.orchestra.run.smithi188.stdout:drwxr-xr-x root/root 0 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/crypto/async_tx/ 2023-11-26T07:02:44.608 INFO:teuthology.orchestra.run.smithi188.stdout:-rw-r--r-- root/root 11249 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/crypto/async_tx/async_memcpy.ko 2023-11-26T07:02:44.608 INFO:teuthology.orchestra.run.smithi188.stdout:-rw-r--r-- root/root 19473 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/crypto/async_tx/async_pq.ko 2023-11-26T07:02:44.608 INFO:teuthology.orchestra.run.smithi188.stdout:-rw-r--r-- root/root 21561 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/crypto/async_tx/async_raid6_recov.ko 2023-11-26T07:02:44.608 INFO:teuthology.orchestra.run.smithi188.stdout:-rw-r--r-- root/root 13825 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/crypto/async_tx/async_tx.ko 2023-11-26T07:02:44.608 INFO:teuthology.orchestra.run.smithi188.stdout:-rw-r--r-- root/root 17745 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/crypto/async_tx/async_xor.ko 2023-11-26T07:02:44.608 INFO:teuthology.orchestra.run.smithi188.stdout:-rw-r--r-- root/root 17921 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/crypto/async_tx/raid6test.ko 2023-11-26T07:02:44.609 INFO:teuthology.orchestra.run.smithi188.stdout:-rw-r--r-- root/root 20489 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/crypto/authenc.ko 2023-11-26T07:02:44.609 INFO:teuthology.orchestra.run.smithi188.stdout:-rw-r--r-- root/root 22577 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/crypto/authencesn.ko 2023-11-26T07:02:44.609 INFO:teuthology.orchestra.run.smithi188.stdout:-rw-r--r-- root/root 21817 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/crypto/blake2b_generic.ko 2023-11-26T07:02:44.609 INFO:teuthology.orchestra.run.smithi188.stdout:-rw-r--r-- root/root 12729 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/crypto/blowfish_common.ko 2023-11-26T07:02:44.609 INFO:teuthology.orchestra.run.smithi188.stdout:-rw-r--r-- root/root 10537 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/crypto/blowfish_generic.ko 2023-11-26T07:02:44.609 INFO:teuthology.orchestra.run.smithi188.stdout:-rw-r--r-- root/root 38809 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/crypto/camellia_generic.ko 2023-11-26T07:02:44.609 INFO:teuthology.orchestra.run.smithi188.stdout:-rw-r--r-- root/root 27321 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/crypto/cast5_generic.ko 2023-11-26T07:02:44.609 INFO:teuthology.orchestra.run.smithi188.stdout:-rw-r--r-- root/root 17745 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/crypto/cast6_generic.ko 2023-11-26T07:02:44.609 INFO:teuthology.orchestra.run.smithi188.stdout:-rw-r--r-- root/root 10537 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/crypto/cast_common.ko 2023-11-26T07:02:44.610 INFO:teuthology.orchestra.run.smithi188.stdout:-rw-r--r-- root/root 35097 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/crypto/ccm.ko 2023-11-26T07:02:44.610 INFO:teuthology.orchestra.run.smithi188.stdout:-rw-r--r-- root/root 14769 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/crypto/cmac.ko 2023-11-26T07:02:44.610 INFO:teuthology.orchestra.run.smithi188.stdout:-rw-r--r-- root/root 9585 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/crypto/crc32_generic.ko 2023-11-26T07:02:44.610 INFO:teuthology.orchestra.run.smithi188.stdout:-rw-r--r-- root/root 47505 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/crypto/cryptd.ko 2023-11-26T07:02:44.610 INFO:teuthology.orchestra.run.smithi188.stdout:-rw-r--r-- root/root 39505 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/crypto/crypto_engine.ko 2023-11-26T07:02:44.610 INFO:teuthology.orchestra.run.smithi188.stdout:-rw-r--r-- root/root 26953 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/crypto/crypto_simd.ko 2023-11-26T07:02:44.610 INFO:teuthology.orchestra.run.smithi188.stdout:-rw-r--r-- root/root 24737 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/crypto/crypto_user.ko 2023-11-26T07:02:44.610 INFO:teuthology.orchestra.run.smithi188.stdout:-rw-r--r-- root/root 17729 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/crypto/cts.ko 2023-11-26T07:02:44.611 INFO:teuthology.orchestra.run.smithi188.stdout:-rw-r--r-- root/root 10913 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/crypto/des_generic.ko 2023-11-26T07:02:44.611 INFO:teuthology.orchestra.run.smithi188.stdout:-rw-r--r-- root/root 15681 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/crypto/dh_generic.ko 2023-11-26T07:02:44.611 INFO:teuthology.orchestra.run.smithi188.stdout:-rw-r--r-- root/root 11001 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/crypto/echainiv.ko 2023-11-26T07:02:44.611 INFO:teuthology.orchestra.run.smithi188.stdout:-rw-r--r-- root/root 25321 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/crypto/essiv.ko 2023-11-26T07:02:44.611 INFO:teuthology.orchestra.run.smithi188.stdout:-rw-r--r-- root/root 17993 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/crypto/fcrypt.ko 2023-11-26T07:02:44.611 INFO:teuthology.orchestra.run.smithi188.stdout:-rw-r--r-- root/root 26897 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/crypto/khazad.ko 2023-11-26T07:02:44.611 INFO:teuthology.orchestra.run.smithi188.stdout:-rw-r--r-- root/root 15649 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/crypto/lrw.ko 2023-11-26T07:02:44.611 INFO:teuthology.orchestra.run.smithi188.stdout:-rw-r--r-- root/root 11513 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/crypto/lz4.ko 2023-11-26T07:02:44.611 INFO:teuthology.orchestra.run.smithi188.stdout:-rw-r--r-- root/root 11529 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/crypto/lz4hc.ko 2023-11-26T07:02:44.611 INFO:teuthology.orchestra.run.smithi188.stdout:-rw-r--r-- root/root 11697 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/crypto/md4.ko 2023-11-26T07:02:44.611 INFO:teuthology.orchestra.run.smithi188.stdout:-rw-r--r-- root/root 9729 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/crypto/michael_mic.ko 2023-11-26T07:02:44.612 INFO:teuthology.orchestra.run.smithi188.stdout:-rw-r--r-- root/root 10673 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/crypto/pcbc.ko 2023-11-26T07:02:44.612 INFO:teuthology.orchestra.run.smithi188.stdout:-rw-r--r-- root/root 18745 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/crypto/pcrypt.ko 2023-11-26T07:02:44.612 INFO:teuthology.orchestra.run.smithi188.stdout:-rw-r--r-- root/root 16049 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/crypto/rmd160.ko 2023-11-26T07:02:44.612 INFO:teuthology.orchestra.run.smithi188.stdout:-rw-r--r-- root/root 29393 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/crypto/seed.ko 2023-11-26T07:02:44.612 INFO:teuthology.orchestra.run.smithi188.stdout:-rw-r--r-- root/root 27081 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/crypto/serpent_generic.ko 2023-11-26T07:02:44.612 INFO:teuthology.orchestra.run.smithi188.stdout:-rw-r--r-- root/root 149193 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/crypto/tcrypt.ko 2023-11-26T07:02:44.612 INFO:teuthology.orchestra.run.smithi188.stdout:-rw-r--r-- root/root 10881 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/crypto/tea.ko 2023-11-26T07:02:44.612 INFO:teuthology.orchestra.run.smithi188.stdout:-rw-r--r-- root/root 28777 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/crypto/twofish_common.ko 2023-11-26T07:02:44.612 INFO:teuthology.orchestra.run.smithi188.stdout:-rw-r--r-- root/root 13169 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/crypto/twofish_generic.ko 2023-11-26T07:02:44.612 INFO:teuthology.orchestra.run.smithi188.stdout:-rw-r--r-- root/root 17401 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/crypto/vmac.ko 2023-11-26T07:02:44.613 INFO:teuthology.orchestra.run.smithi188.stdout:-rw-r--r-- root/root 36185 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/crypto/wp512.ko 2023-11-26T07:02:44.613 INFO:teuthology.orchestra.run.smithi188.stdout:-rw-r--r-- root/root 14105 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/crypto/xcbc.ko 2023-11-26T07:02:44.613 INFO:teuthology.orchestra.run.smithi188.stdout:-rw-r--r-- root/root 26665 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/crypto/xor.ko 2023-11-26T07:02:44.613 INFO:teuthology.orchestra.run.smithi188.stdout:-rw-r--r-- root/root 18713 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/crypto/xts.ko 2023-11-26T07:02:44.613 INFO:teuthology.orchestra.run.smithi188.stdout:-rw-r--r-- root/root 9481 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/crypto/xxhash_generic.ko 2023-11-26T07:02:44.613 INFO:teuthology.orchestra.run.smithi188.stdout:drwxr-xr-x root/root 0 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/ 2023-11-26T07:02:44.613 INFO:teuthology.orchestra.run.smithi188.stdout:drwxr-xr-x root/root 0 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/acpi/ 2023-11-26T07:02:44.613 INFO:teuthology.orchestra.run.smithi188.stdout:-rw-r--r-- root/root 18249 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/acpi/acpi_extlog.ko 2023-11-26T07:02:44.613 INFO:teuthology.orchestra.run.smithi188.stdout:-rw-r--r-- root/root 26985 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/acpi/acpi_ipmi.ko 2023-11-26T07:02:44.613 INFO:teuthology.orchestra.run.smithi188.stdout:-rw-r--r-- root/root 28793 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/acpi/acpi_pad.ko 2023-11-26T07:02:44.614 INFO:teuthology.orchestra.run.smithi188.stdout:drwxr-xr-x root/root 0 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/acpi/apei/ 2023-11-26T07:02:44.614 INFO:teuthology.orchestra.run.smithi188.stdout:-rw-r--r-- root/root 31241 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/acpi/apei/einj.ko 2023-11-26T07:02:44.614 INFO:teuthology.orchestra.run.smithi188.stdout:-rw-r--r-- root/root 12801 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/acpi/ec_sys.ko 2023-11-26T07:02:44.614 INFO:teuthology.orchestra.run.smithi188.stdout:-rw-r--r-- root/root 14937 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/acpi/platform_profile.ko 2023-11-26T07:02:44.614 INFO:teuthology.orchestra.run.smithi188.stdout:-rw-r--r-- root/root 24401 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/acpi/sbs.ko 2023-11-26T07:02:44.614 INFO:teuthology.orchestra.run.smithi188.stdout:-rw-r--r-- root/root 19521 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/acpi/sbshc.ko 2023-11-26T07:02:44.614 INFO:teuthology.orchestra.run.smithi188.stdout:-rw-r--r-- root/root 103025 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/acpi/video.ko 2023-11-26T07:02:44.614 INFO:teuthology.orchestra.run.smithi188.stdout:drwxr-xr-x root/root 0 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/ata/ 2023-11-26T07:02:44.614 INFO:teuthology.orchestra.run.smithi188.stdout:-rw-r--r-- root/root 21601 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/ata/acard-ahci.ko 2023-11-26T07:02:44.614 INFO:teuthology.orchestra.run.smithi188.stdout:-rw-r--r-- root/root 88865 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/ata/ahci.ko 2023-11-26T07:02:44.615 INFO:teuthology.orchestra.run.smithi188.stdout:-rw-r--r-- root/root 14729 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/ata/ahci_platform.ko 2023-11-26T07:02:44.615 INFO:teuthology.orchestra.run.smithi188.stdout:-rw-r--r-- root/root 88289 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/ata/libahci.ko 2023-11-26T07:02:44.615 INFO:teuthology.orchestra.run.smithi188.stdout:-rw-r--r-- root/root 34985 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/ata/libahci_platform.ko 2023-11-26T07:02:44.615 INFO:teuthology.orchestra.run.smithi188.stdout:-rw-r--r-- root/root 17137 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/ata/pata_acpi.ko 2023-11-26T07:02:44.615 INFO:teuthology.orchestra.run.smithi188.stdout:-rw-r--r-- root/root 30433 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/ata/pata_ali.ko 2023-11-26T07:02:44.615 INFO:teuthology.orchestra.run.smithi188.stdout:-rw-r--r-- root/root 33505 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/ata/pata_amd.ko 2023-11-26T07:02:44.615 INFO:teuthology.orchestra.run.smithi188.stdout:-rw-r--r-- root/root 21265 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/ata/pata_artop.ko 2023-11-26T07:02:44.615 INFO:teuthology.orchestra.run.smithi188.stdout:-rw-r--r-- root/root 21121 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/ata/pata_atiixp.ko 2023-11-26T07:02:44.615 INFO:teuthology.orchestra.run.smithi188.stdout:-rw-r--r-- root/root 24945 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/ata/pata_atp867x.ko 2023-11-26T07:02:44.615 INFO:teuthology.orchestra.run.smithi188.stdout:-rw-r--r-- root/root 17657 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/ata/pata_cmd640.ko 2023-11-26T07:02:44.616 INFO:teuthology.orchestra.run.smithi188.stdout:-rw-r--r-- root/root 25185 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/ata/pata_cmd64x.ko 2023-11-26T07:02:44.616 INFO:teuthology.orchestra.run.smithi188.stdout:-rw-r--r-- root/root 15049 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/ata/pata_cypress.ko 2023-11-26T07:02:44.616 INFO:teuthology.orchestra.run.smithi188.stdout:-rw-r--r-- root/root 17849 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/ata/pata_efar.ko 2023-11-26T07:02:44.616 INFO:teuthology.orchestra.run.smithi188.stdout:-rw-r--r-- root/root 21945 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/ata/pata_hpt366.ko 2023-11-26T07:02:44.616 INFO:teuthology.orchestra.run.smithi188.stdout:-rw-r--r-- root/root 33977 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/ata/pata_hpt37x.ko 2023-11-26T07:02:44.616 INFO:teuthology.orchestra.run.smithi188.stdout:-rw-r--r-- root/root 23129 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/ata/pata_hpt3x2n.ko 2023-11-26T07:02:44.616 INFO:teuthology.orchestra.run.smithi188.stdout:-rw-r--r-- root/root 16921 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/ata/pata_hpt3x3.ko 2023-11-26T07:02:44.616 INFO:teuthology.orchestra.run.smithi188.stdout:-rw-r--r-- root/root 16993 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/ata/pata_it8213.ko 2023-11-26T07:02:44.616 INFO:teuthology.orchestra.run.smithi188.stdout:-rw-r--r-- root/root 35273 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/ata/pata_it821x.ko 2023-11-26T07:02:44.616 INFO:teuthology.orchestra.run.smithi188.stdout:-rw-r--r-- root/root 12889 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/ata/pata_jmicron.ko 2023-11-26T07:02:44.616 INFO:teuthology.orchestra.run.smithi188.stdout:-rw-r--r-- root/root 51481 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/ata/pata_legacy.ko 2023-11-26T07:02:44.617 INFO:teuthology.orchestra.run.smithi188.stdout:-rw-r--r-- root/root 15185 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/ata/pata_marvell.ko 2023-11-26T07:02:44.617 INFO:teuthology.orchestra.run.smithi188.stdout:-rw-r--r-- root/root 16281 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/ata/pata_mpiix.ko 2023-11-26T07:02:44.617 INFO:teuthology.orchestra.run.smithi188.stdout:-rw-r--r-- root/root 12833 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/ata/pata_netcell.ko 2023-11-26T07:02:44.617 INFO:teuthology.orchestra.run.smithi188.stdout:-rw-r--r-- root/root 16817 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/ata/pata_ninja32.ko 2023-11-26T07:02:44.617 INFO:teuthology.orchestra.run.smithi188.stdout:-rw-r--r-- root/root 15105 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/ata/pata_ns87410.ko 2023-11-26T07:02:44.617 INFO:teuthology.orchestra.run.smithi188.stdout:-rw-r--r-- root/root 17361 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/ata/pata_ns87415.ko 2023-11-26T07:02:44.617 INFO:teuthology.orchestra.run.smithi188.stdout:-rw-r--r-- root/root 16569 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/ata/pata_oldpiix.ko 2023-11-26T07:02:44.617 INFO:teuthology.orchestra.run.smithi188.stdout:-rw-r--r-- root/root 15377 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/ata/pata_opti.ko 2023-11-26T07:02:44.617 INFO:teuthology.orchestra.run.smithi188.stdout:-rw-r--r-- root/root 20681 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/ata/pata_optidma.ko 2023-11-26T07:02:44.617 INFO:teuthology.orchestra.run.smithi188.stdout:-rw-r--r-- root/root 32585 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/ata/pata_pcmcia.ko 2023-11-26T07:02:44.618 INFO:teuthology.orchestra.run.smithi188.stdout:-rw-r--r-- root/root 37793 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/ata/pata_pdc2027x.ko 2023-11-26T07:02:44.618 INFO:teuthology.orchestra.run.smithi188.stdout:-rw-r--r-- root/root 20561 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/ata/pata_pdc202xx_old.ko 2023-11-26T07:02:44.618 INFO:teuthology.orchestra.run.smithi188.stdout:-rw-r--r-- root/root 14225 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/ata/pata_piccolo.ko 2023-11-26T07:02:44.618 INFO:teuthology.orchestra.run.smithi188.stdout:-rw-r--r-- root/root 15449 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/ata/pata_radisys.ko 2023-11-26T07:02:44.618 INFO:teuthology.orchestra.run.smithi188.stdout:-rw-r--r-- root/root 19521 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/ata/pata_rdc.ko 2023-11-26T07:02:44.618 INFO:teuthology.orchestra.run.smithi188.stdout:-rw-r--r-- root/root 15329 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/ata/pata_rz1000.ko 2023-11-26T07:02:44.618 INFO:teuthology.orchestra.run.smithi188.stdout:-rw-r--r-- root/root 13937 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/ata/pata_sch.ko 2023-11-26T07:02:44.618 INFO:teuthology.orchestra.run.smithi188.stdout:-rw-r--r-- root/root 23097 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/ata/pata_serverworks.ko 2023-11-26T07:02:44.618 INFO:teuthology.orchestra.run.smithi188.stdout:-rw-r--r-- root/root 23457 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/ata/pata_sil680.ko 2023-11-26T07:02:44.618 INFO:teuthology.orchestra.run.smithi188.stdout:-rw-r--r-- root/root 19697 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/ata/pata_sl82c105.ko 2023-11-26T07:02:44.618 INFO:teuthology.orchestra.run.smithi188.stdout:-rw-r--r-- root/root 16009 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/ata/pata_triflex.ko 2023-11-26T07:02:44.619 INFO:teuthology.orchestra.run.smithi188.stdout:-rw-r--r-- root/root 30433 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/ata/pata_via.ko 2023-11-26T07:02:44.619 INFO:teuthology.orchestra.run.smithi188.stdout:-rw-r--r-- root/root 22793 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/ata/pdc_adma.ko 2023-11-26T07:02:44.619 INFO:teuthology.orchestra.run.smithi188.stdout:-rw-r--r-- root/root 29121 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/ata/sata_inic162x.ko 2023-11-26T07:02:44.619 INFO:teuthology.orchestra.run.smithi188.stdout:-rw-r--r-- root/root 83345 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/ata/sata_mv.ko 2023-11-26T07:02:44.619 INFO:teuthology.orchestra.run.smithi188.stdout:-rw-r--r-- root/root 70393 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/ata/sata_nv.ko 2023-11-26T07:02:44.619 INFO:teuthology.orchestra.run.smithi188.stdout:-rw-r--r-- root/root 37377 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/ata/sata_promise.ko 2023-11-26T07:02:44.619 INFO:teuthology.orchestra.run.smithi188.stdout:-rw-r--r-- root/root 23041 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/ata/sata_qstor.ko 2023-11-26T07:02:44.619 INFO:teuthology.orchestra.run.smithi188.stdout:-rw-r--r-- root/root 29273 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/ata/sata_sil.ko 2023-11-26T07:02:44.619 INFO:teuthology.orchestra.run.smithi188.stdout:-rw-r--r-- root/root 38337 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/ata/sata_sil24.ko 2023-11-26T07:02:44.619 INFO:teuthology.orchestra.run.smithi188.stdout:-rw-r--r-- root/root 17945 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/ata/sata_sis.ko 2023-11-26T07:02:44.619 INFO:teuthology.orchestra.run.smithi188.stdout:-rw-r--r-- root/root 20441 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/ata/sata_svw.ko 2023-11-26T07:02:44.620 INFO:teuthology.orchestra.run.smithi188.stdout:-rw-r--r-- root/root 44793 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/ata/sata_sx4.ko 2023-11-26T07:02:44.620 INFO:teuthology.orchestra.run.smithi188.stdout:-rw-r--r-- root/root 15601 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/ata/sata_uli.ko 2023-11-26T07:02:44.620 INFO:teuthology.orchestra.run.smithi188.stdout:-rw-r--r-- root/root 35065 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/ata/sata_via.ko 2023-11-26T07:02:44.620 INFO:teuthology.orchestra.run.smithi188.stdout:-rw-r--r-- root/root 20065 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/ata/sata_vsc.ko 2023-11-26T07:02:44.620 INFO:teuthology.orchestra.run.smithi188.stdout:drwxr-xr-x root/root 0 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/auxdisplay/ 2023-11-26T07:02:44.620 INFO:teuthology.orchestra.run.smithi188.stdout:-rw-r--r-- root/root 16289 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/auxdisplay/ks0108.ko 2023-11-26T07:02:44.620 INFO:teuthology.orchestra.run.smithi188.stdout:drwxr-xr-x root/root 0 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/bcma/ 2023-11-26T07:02:44.620 INFO:teuthology.orchestra.run.smithi188.stdout:-rw-r--r-- root/root 124473 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/bcma/bcma.ko 2023-11-26T07:02:44.620 INFO:teuthology.orchestra.run.smithi188.stdout:drwxr-xr-x root/root 0 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/block/ 2023-11-26T07:02:44.620 INFO:teuthology.orchestra.run.smithi188.stdout:drwxr-xr-x root/root 0 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/block/aoe/ 2023-11-26T07:02:44.621 INFO:teuthology.orchestra.run.smithi188.stdout:-rw-r--r-- root/root 113665 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/block/aoe/aoe.ko 2023-11-26T07:02:44.621 INFO:teuthology.orchestra.run.smithi188.stdout:drwxr-xr-x root/root 0 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/block/drbd/ 2023-11-26T07:02:44.621 INFO:teuthology.orchestra.run.smithi188.stdout:-rw-r--r-- root/root 965585 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/block/drbd/drbd.ko 2023-11-26T07:02:44.621 INFO:teuthology.orchestra.run.smithi188.stdout:-rw-r--r-- root/root 173225 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/block/floppy.ko 2023-11-26T07:02:44.621 INFO:teuthology.orchestra.run.smithi188.stdout:drwxr-xr-x root/root 0 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/block/mtip32xx/ 2023-11-26T07:02:44.621 INFO:teuthology.orchestra.run.smithi188.stdout:-rw-r--r-- root/root 94297 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/block/mtip32xx/mtip32xx.ko 2023-11-26T07:02:44.621 INFO:teuthology.orchestra.run.smithi188.stdout:-rw-r--r-- root/root 115793 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/block/nbd.ko 2023-11-26T07:02:44.621 INFO:teuthology.orchestra.run.smithi188.stdout:drwxr-xr-x root/root 0 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/block/null_blk/ 2023-11-26T07:02:44.621 INFO:teuthology.orchestra.run.smithi188.stdout:-rw-r--r-- root/root 110385 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/block/null_blk/null_blk.ko 2023-11-26T07:02:44.621 INFO:teuthology.orchestra.run.smithi188.stdout:-rw-r--r-- root/root 98193 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/block/pktcdvd.ko 2023-11-26T07:02:44.622 INFO:teuthology.orchestra.run.smithi188.stdout:-rw-r--r-- root/root 263769 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/block/rbd.ko 2023-11-26T07:02:44.622 INFO:teuthology.orchestra.run.smithi188.stdout:drwxr-xr-x root/root 0 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/block/xen-blkback/ 2023-11-26T07:02:44.622 INFO:teuthology.orchestra.run.smithi188.stdout:-rw-r--r-- root/root 98145 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/block/xen-blkback/xen-blkback.ko 2023-11-26T07:02:44.622 INFO:teuthology.orchestra.run.smithi188.stdout:drwxr-xr-x root/root 0 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/block/zram/ 2023-11-26T07:02:44.622 INFO:teuthology.orchestra.run.smithi188.stdout:-rw-r--r-- root/root 60217 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/block/zram/zram.ko 2023-11-26T07:02:44.622 INFO:teuthology.orchestra.run.smithi188.stdout:drwxr-xr-x root/root 0 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/char/ 2023-11-26T07:02:44.622 INFO:teuthology.orchestra.run.smithi188.stdout:-rw-r--r-- root/root 30705 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/char/applicom.ko 2023-11-26T07:02:44.622 INFO:teuthology.orchestra.run.smithi188.stdout:-rw-r--r-- root/root 12009 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/char/hangcheck-timer.ko 2023-11-26T07:02:44.622 INFO:teuthology.orchestra.run.smithi188.stdout:drwxr-xr-x root/root 0 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/char/hw_random/ 2023-11-26T07:02:44.622 INFO:teuthology.orchestra.run.smithi188.stdout:-rw-r--r-- root/root 13889 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/char/hw_random/amd-rng.ko 2023-11-26T07:02:44.623 INFO:teuthology.orchestra.run.smithi188.stdout:-rw-r--r-- root/root 18809 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/char/hw_random/intel-rng.ko 2023-11-26T07:02:44.623 INFO:teuthology.orchestra.run.smithi188.stdout:-rw-r--r-- root/root 13169 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/char/hw_random/timeriomem-rng.ko 2023-11-26T07:02:44.623 INFO:teuthology.orchestra.run.smithi188.stdout:-rw-r--r-- root/root 10673 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/char/hw_random/via-rng.ko 2023-11-26T07:02:44.623 INFO:teuthology.orchestra.run.smithi188.stdout:-rw-r--r-- root/root 15633 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/char/hw_random/virtio-rng.ko 2023-11-26T07:02:44.623 INFO:teuthology.orchestra.run.smithi188.stdout:drwxr-xr-x root/root 0 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/char/ipmi/ 2023-11-26T07:02:44.623 INFO:teuthology.orchestra.run.smithi188.stdout:-rw-r--r-- root/root 30713 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/char/ipmi/ipmi_devintf.ko 2023-11-26T07:02:44.623 INFO:teuthology.orchestra.run.smithi188.stdout:-rw-r--r-- root/root 148601 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/char/ipmi/ipmi_msghandler.ko 2023-11-26T07:02:44.623 INFO:teuthology.orchestra.run.smithi188.stdout:-rw-r--r-- root/root 26569 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/char/ipmi/ipmi_poweroff.ko 2023-11-26T07:02:44.623 INFO:teuthology.orchestra.run.smithi188.stdout:-rw-r--r-- root/root 139313 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/char/ipmi/ipmi_si.ko 2023-11-26T07:02:44.623 INFO:teuthology.orchestra.run.smithi188.stdout:-rw-r--r-- root/root 70921 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/char/ipmi/ipmi_ssif.ko 2023-11-26T07:02:44.624 INFO:teuthology.orchestra.run.smithi188.stdout:-rw-r--r-- root/root 53513 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/char/ipmi/ipmi_watchdog.ko 2023-11-26T07:02:44.624 INFO:teuthology.orchestra.run.smithi188.stdout:-rw-r--r-- root/root 36321 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/char/lp.ko 2023-11-26T07:02:44.624 INFO:teuthology.orchestra.run.smithi188.stdout:drwxr-xr-x root/root 0 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/char/mwave/ 2023-11-26T07:02:44.624 INFO:teuthology.orchestra.run.smithi188.stdout:-rw-r--r-- root/root 84097 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/char/mwave/mwave.ko 2023-11-26T07:02:44.624 INFO:teuthology.orchestra.run.smithi188.stdout:-rw-r--r-- root/root 25873 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/char/nvram.ko 2023-11-26T07:02:44.624 INFO:teuthology.orchestra.run.smithi188.stdout:-rw-r--r-- root/root 32169 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/char/ppdev.ko 2023-11-26T07:02:44.624 INFO:teuthology.orchestra.run.smithi188.stdout:-rw-r--r-- root/root 51113 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/char/tlclk.ko 2023-11-26T07:02:44.624 INFO:teuthology.orchestra.run.smithi188.stdout:drwxr-xr-x root/root 0 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/char/tpm/ 2023-11-26T07:02:44.624 INFO:teuthology.orchestra.run.smithi188.stdout:-rw-r--r-- root/root 16401 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/char/tpm/tpm_atmel.ko 2023-11-26T07:02:44.624 INFO:teuthology.orchestra.run.smithi188.stdout:-rw-r--r-- root/root 15881 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/char/tpm/tpm_i2c_atmel.ko 2023-11-26T07:02:44.624 INFO:teuthology.orchestra.run.smithi188.stdout:-rw-r--r-- root/root 23473 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/char/tpm/tpm_i2c_infineon.ko 2023-11-26T07:02:44.625 INFO:teuthology.orchestra.run.smithi188.stdout:-rw-r--r-- root/root 32985 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/char/tpm/tpm_i2c_nuvoton.ko 2023-11-26T07:02:44.625 INFO:teuthology.orchestra.run.smithi188.stdout:-rw-r--r-- root/root 30121 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/char/tpm/tpm_infineon.ko 2023-11-26T07:02:44.625 INFO:teuthology.orchestra.run.smithi188.stdout:-rw-r--r-- root/root 22697 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/char/tpm/tpm_nsc.ko 2023-11-26T07:02:44.625 INFO:teuthology.orchestra.run.smithi188.stdout:-rw-r--r-- root/root 23561 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/char/tpm/xen-tpmfront.ko 2023-11-26T07:02:44.625 INFO:teuthology.orchestra.run.smithi188.stdout:drwxr-xr-x root/root 0 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/char/xillybus/ 2023-11-26T07:02:44.625 INFO:teuthology.orchestra.run.smithi188.stdout:-rw-r--r-- root/root 17001 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/char/xillybus/xillybus_class.ko 2023-11-26T07:02:44.625 INFO:teuthology.orchestra.run.smithi188.stdout:-rw-r--r-- root/root 50945 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/char/xillybus/xillybus_core.ko 2023-11-26T07:02:44.625 INFO:teuthology.orchestra.run.smithi188.stdout:-rw-r--r-- root/root 12393 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/char/xillybus/xillybus_pcie.ko 2023-11-26T07:02:44.625 INFO:teuthology.orchestra.run.smithi188.stdout:drwxr-xr-x root/root 0 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/clk/ 2023-11-26T07:02:44.625 INFO:teuthology.orchestra.run.smithi188.stdout:-rw-r--r-- root/root 12489 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/clk/clk-palmas.ko 2023-11-26T07:02:44.625 INFO:teuthology.orchestra.run.smithi188.stdout:-rw-r--r-- root/root 47849 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/clk/clk-si5351.ko 2023-11-26T07:02:44.625 INFO:teuthology.orchestra.run.smithi188.stdout:-rw-r--r-- root/root 11457 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/clk/clk-twl6040.ko 2023-11-26T07:02:44.626 INFO:teuthology.orchestra.run.smithi188.stdout:-rw-r--r-- root/root 19249 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/clk/clk-wm831x.ko 2023-11-26T07:02:44.626 INFO:teuthology.orchestra.run.smithi188.stdout:drwxr-xr-x root/root 0 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/cpufreq/ 2023-11-26T07:02:44.626 INFO:teuthology.orchestra.run.smithi188.stdout:-rw-r--r-- root/root 10097 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/cpufreq/amd_freq_sensitivity.ko 2023-11-26T07:02:44.626 INFO:teuthology.orchestra.run.smithi188.stdout:-rw-r--r-- root/root 16833 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/cpufreq/p4-clockmod.ko 2023-11-26T07:02:44.626 INFO:teuthology.orchestra.run.smithi188.stdout:-rw-r--r-- root/root 25297 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/cpufreq/speedstep-lib.ko 2023-11-26T07:02:44.626 INFO:teuthology.orchestra.run.smithi188.stdout:drwxr-xr-x root/root 0 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/crypto/ 2023-11-26T07:02:44.626 INFO:teuthology.orchestra.run.smithi188.stdout:drwxr-xr-x root/root 0 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/crypto/ccp/ 2023-11-26T07:02:44.626 INFO:teuthology.orchestra.run.smithi188.stdout:-rw-r--r-- root/root 70801 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/crypto/ccp/ccp-crypto.ko 2023-11-26T07:02:44.626 INFO:teuthology.orchestra.run.smithi188.stdout:-rw-r--r-- root/root 243025 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/crypto/ccp/ccp.ko 2023-11-26T07:02:44.626 INFO:teuthology.orchestra.run.smithi188.stdout:drwxr-xr-x root/root 0 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/crypto/intel/ 2023-11-26T07:02:44.627 INFO:teuthology.orchestra.run.smithi188.stdout:drwxr-xr-x root/root 0 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/crypto/intel/qat/ 2023-11-26T07:02:44.627 INFO:teuthology.orchestra.run.smithi188.stdout:drwxr-xr-x root/root 0 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/crypto/intel/qat/qat_common/ 2023-11-26T07:02:44.627 INFO:teuthology.orchestra.run.smithi188.stdout:-rw-r--r-- root/root 438761 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/crypto/intel/qat/qat_common/intel_qat.ko 2023-11-26T07:02:44.627 INFO:teuthology.orchestra.run.smithi188.stdout:drwxr-xr-x root/root 0 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/crypto/intel/qat/qat_dh895xcc/ 2023-11-26T07:02:44.627 INFO:teuthology.orchestra.run.smithi188.stdout:-rw-r--r-- root/root 24081 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/crypto/intel/qat/qat_dh895xcc/qat_dh895xcc.ko 2023-11-26T07:02:44.627 INFO:teuthology.orchestra.run.smithi188.stdout:-rw-r--r-- root/root 19937 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/crypto/padlock-aes.ko 2023-11-26T07:02:44.627 INFO:teuthology.orchestra.run.smithi188.stdout:-rw-r--r-- root/root 21537 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/crypto/padlock-sha.ko 2023-11-26T07:02:44.627 INFO:teuthology.orchestra.run.smithi188.stdout:drwxr-xr-x root/root 0 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/crypto/virtio/ 2023-11-26T07:02:44.627 INFO:teuthology.orchestra.run.smithi188.stdout:-rw-r--r-- root/root 70273 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/crypto/virtio/virtio_crypto.ko 2023-11-26T07:02:44.627 INFO:teuthology.orchestra.run.smithi188.stdout:drwxr-xr-x root/root 0 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/dca/ 2023-11-26T07:02:44.627 INFO:teuthology.orchestra.run.smithi188.stdout:-rw-r--r-- root/root 29785 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/dca/dca.ko 2023-11-26T07:02:44.627 INFO:teuthology.orchestra.run.smithi188.stdout:drwxr-xr-x root/root 0 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/dma/ 2023-11-26T07:02:44.628 INFO:teuthology.orchestra.run.smithi188.stdout:drwxr-xr-x root/root 0 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/dma/dw/ 2023-11-26T07:02:44.628 INFO:teuthology.orchestra.run.smithi188.stdout:-rw-r--r-- root/root 15857 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/dma/dw/dw_dmac.ko 2023-11-26T07:02:44.628 INFO:teuthology.orchestra.run.smithi188.stdout:drwxr-xr-x root/root 0 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/dma/ioat/ 2023-11-26T07:02:44.628 INFO:teuthology.orchestra.run.smithi188.stdout:-rw-r--r-- root/root 122569 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/dma/ioat/ioatdma.ko 2023-11-26T07:02:44.628 INFO:teuthology.orchestra.run.smithi188.stdout:drwxr-xr-x root/root 0 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/edac/ 2023-11-26T07:02:44.628 INFO:teuthology.orchestra.run.smithi188.stdout:-rw-r--r-- root/root 79953 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/edac/amd64_edac.ko 2023-11-26T07:02:44.628 INFO:teuthology.orchestra.run.smithi188.stdout:-rw-r--r-- root/root 35769 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/edac/e752x_edac.ko 2023-11-26T07:02:44.628 INFO:teuthology.orchestra.run.smithi188.stdout:-rw-r--r-- root/root 63617 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/edac/edac_mce_amd.ko 2023-11-26T07:02:44.628 INFO:teuthology.orchestra.run.smithi188.stdout:-rw-r--r-- root/root 18105 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/edac/i3000_edac.ko 2023-11-26T07:02:44.628 INFO:teuthology.orchestra.run.smithi188.stdout:-rw-r--r-- root/root 18065 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/edac/i3200_edac.ko 2023-11-26T07:02:44.628 INFO:teuthology.orchestra.run.smithi188.stdout:-rw-r--r-- root/root 22537 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/edac/i5100_edac.ko 2023-11-26T07:02:44.629 INFO:teuthology.orchestra.run.smithi188.stdout:-rw-r--r-- root/root 25657 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/edac/i5400_edac.ko 2023-11-26T07:02:44.629 INFO:teuthology.orchestra.run.smithi188.stdout:-rw-r--r-- root/root 24913 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/edac/i7300_edac.ko 2023-11-26T07:02:44.629 INFO:teuthology.orchestra.run.smithi188.stdout:-rw-r--r-- root/root 52041 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/edac/i7core_edac.ko 2023-11-26T07:02:44.629 INFO:teuthology.orchestra.run.smithi188.stdout:-rw-r--r-- root/root 18433 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/edac/i82975x_edac.ko 2023-11-26T07:02:44.629 INFO:teuthology.orchestra.run.smithi188.stdout:-rw-r--r-- root/root 20017 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/edac/ie31200_edac.ko 2023-11-26T07:02:44.629 INFO:teuthology.orchestra.run.smithi188.stdout:-rw-r--r-- root/root 56825 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/edac/sb_edac.ko 2023-11-26T07:02:44.629 INFO:teuthology.orchestra.run.smithi188.stdout:-rw-r--r-- root/root 17529 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/edac/x38_edac.ko 2023-11-26T07:02:44.629 INFO:teuthology.orchestra.run.smithi188.stdout:drwxr-xr-x root/root 0 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/extcon/ 2023-11-26T07:02:44.629 INFO:teuthology.orchestra.run.smithi188.stdout:-rw-r--r-- root/root 12673 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/extcon/extcon-gpio.ko 2023-11-26T07:02:44.629 INFO:teuthology.orchestra.run.smithi188.stdout:-rw-r--r-- root/root 27185 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/extcon/extcon-max14577.ko 2023-11-26T07:02:44.629 INFO:teuthology.orchestra.run.smithi188.stdout:-rw-r--r-- root/root 29569 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/extcon/extcon-max77693.ko 2023-11-26T07:02:44.630 INFO:teuthology.orchestra.run.smithi188.stdout:-rw-r--r-- root/root 24745 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/extcon/extcon-max8997.ko 2023-11-26T07:02:44.630 INFO:teuthology.orchestra.run.smithi188.stdout:-rw-r--r-- root/root 26537 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/extcon/extcon-palmas.ko 2023-11-26T07:02:44.630 INFO:teuthology.orchestra.run.smithi188.stdout:-rw-r--r-- root/root 23713 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/extcon/extcon-rt8973a.ko 2023-11-26T07:02:44.630 INFO:teuthology.orchestra.run.smithi188.stdout:-rw-r--r-- root/root 27785 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/extcon/extcon-sm5502.ko 2023-11-26T07:02:44.630 INFO:teuthology.orchestra.run.smithi188.stdout:drwxr-xr-x root/root 0 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/firmware/ 2023-11-26T07:02:44.630 INFO:teuthology.orchestra.run.smithi188.stdout:-rw-r--r-- root/root 34977 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/firmware/dmi-sysfs.ko 2023-11-26T07:02:44.630 INFO:teuthology.orchestra.run.smithi188.stdout:drwxr-xr-x root/root 0 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/firmware/efi/ 2023-11-26T07:02:44.630 INFO:teuthology.orchestra.run.smithi188.stdout:-rw-r--r-- root/root 15521 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/firmware/efi/efi-pstore.ko 2023-11-26T07:02:44.630 INFO:teuthology.orchestra.run.smithi188.stdout:-rw-r--r-- root/root 20729 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/firmware/iscsi_ibft.ko 2023-11-26T07:02:44.630 INFO:teuthology.orchestra.run.smithi188.stdout:drwxr-xr-x root/root 0 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/gpio/ 2023-11-26T07:02:44.630 INFO:teuthology.orchestra.run.smithi188.stdout:-rw-r--r-- root/root 12305 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/gpio/gpio-adp5520.ko 2023-11-26T07:02:44.631 INFO:teuthology.orchestra.run.smithi188.stdout:-rw-r--r-- root/root 20153 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/gpio/gpio-amd8111.ko 2023-11-26T07:02:44.631 INFO:teuthology.orchestra.run.smithi188.stdout:-rw-r--r-- root/root 14993 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/gpio/gpio-arizona.ko 2023-11-26T07:02:44.631 INFO:teuthology.orchestra.run.smithi188.stdout:-rw-r--r-- root/root 13841 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/gpio/gpio-da9052.ko 2023-11-26T07:02:44.631 INFO:teuthology.orchestra.run.smithi188.stdout:-rw-r--r-- root/root 11921 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/gpio/gpio-da9055.ko 2023-11-26T07:02:44.631 INFO:teuthology.orchestra.run.smithi188.stdout:-rw-r--r-- root/root 22065 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/gpio/gpio-dln2.ko 2023-11-26T07:02:44.631 INFO:teuthology.orchestra.run.smithi188.stdout:-rw-r--r-- root/root 77849 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/gpio/gpio-f7188x.ko 2023-11-26T07:02:44.631 INFO:teuthology.orchestra.run.smithi188.stdout:-rw-r--r-- root/root 28937 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/gpio/gpio-generic.ko 2023-11-26T07:02:44.631 INFO:teuthology.orchestra.run.smithi188.stdout:-rw-r--r-- root/root 19569 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/gpio/gpio-ich.ko 2023-11-26T07:02:44.631 INFO:teuthology.orchestra.run.smithi188.stdout:-rw-r--r-- root/root 11905 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/gpio/gpio-janz-ttl.ko 2023-11-26T07:02:44.631 INFO:teuthology.orchestra.run.smithi188.stdout:-rw-r--r-- root/root 14113 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/gpio/gpio-kempld.ko 2023-11-26T07:02:44.631 INFO:teuthology.orchestra.run.smithi188.stdout:-rw-r--r-- root/root 12537 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/gpio/gpio-lp3943.ko 2023-11-26T07:02:44.632 INFO:teuthology.orchestra.run.smithi188.stdout:-rw-r--r-- root/root 9505 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/gpio/gpio-max7300.ko 2023-11-26T07:02:44.632 INFO:teuthology.orchestra.run.smithi188.stdout:-rw-r--r-- root/root 11905 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/gpio/gpio-max730x.ko 2023-11-26T07:02:44.632 INFO:teuthology.orchestra.run.smithi188.stdout:-rw-r--r-- root/root 22073 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/gpio/gpio-max732x.ko 2023-11-26T07:02:44.632 INFO:teuthology.orchestra.run.smithi188.stdout:-rw-r--r-- root/root 25145 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/gpio/gpio-ml-ioh.ko 2023-11-26T07:02:44.632 INFO:teuthology.orchestra.run.smithi188.stdout:-rw-r--r-- root/root 47073 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/gpio/gpio-pca953x.ko 2023-11-26T07:02:44.632 INFO:teuthology.orchestra.run.smithi188.stdout:-rw-r--r-- root/root 26825 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/gpio/gpio-pcf857x.ko 2023-11-26T07:02:44.632 INFO:teuthology.orchestra.run.smithi188.stdout:-rw-r--r-- root/root 14377 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/gpio/gpio-rdc321x.ko 2023-11-26T07:02:44.632 INFO:teuthology.orchestra.run.smithi188.stdout:-rw-r--r-- root/root 20593 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/gpio/gpio-sch.ko 2023-11-26T07:02:44.633 INFO:teuthology.orchestra.run.smithi188.stdout:-rw-r--r-- root/root 19505 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/gpio/gpio-sch311x.ko 2023-11-26T07:02:44.633 INFO:teuthology.orchestra.run.smithi188.stdout:-rw-r--r-- root/root 12145 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/gpio/gpio-tps65912.ko 2023-11-26T07:02:44.633 INFO:teuthology.orchestra.run.smithi188.stdout:-rw-r--r-- root/root 21057 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/gpio/gpio-twl4030.ko 2023-11-26T07:02:44.633 INFO:teuthology.orchestra.run.smithi188.stdout:-rw-r--r-- root/root 11425 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/gpio/gpio-twl6040.ko 2023-11-26T07:02:44.633 INFO:teuthology.orchestra.run.smithi188.stdout:-rw-r--r-- root/root 20601 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/gpio/gpio-viperboard.ko 2023-11-26T07:02:44.633 INFO:teuthology.orchestra.run.smithi188.stdout:-rw-r--r-- root/root 15377 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/gpio/gpio-vx855.ko 2023-11-26T07:02:44.633 INFO:teuthology.orchestra.run.smithi188.stdout:-rw-r--r-- root/root 16281 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/gpio/gpio-wm831x.ko 2023-11-26T07:02:44.633 INFO:teuthology.orchestra.run.smithi188.stdout:-rw-r--r-- root/root 11617 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/gpio/gpio-wm8350.ko 2023-11-26T07:02:44.633 INFO:teuthology.orchestra.run.smithi188.stdout:-rw-r--r-- root/root 16753 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/gpio/gpio-wm8994.ko 2023-11-26T07:02:44.633 INFO:teuthology.orchestra.run.smithi188.stdout:drwxr-xr-x root/root 0 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hid/ 2023-11-26T07:02:44.633 INFO:teuthology.orchestra.run.smithi188.stdout:-rw-r--r-- root/root 11577 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hid/hid-a4tech.ko 2023-11-26T07:02:44.634 INFO:teuthology.orchestra.run.smithi188.stdout:-rw-r--r-- root/root 39169 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hid/hid-apple.ko 2023-11-26T07:02:44.634 INFO:teuthology.orchestra.run.smithi188.stdout:-rw-r--r-- root/root 14481 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hid/hid-appleir.ko 2023-11-26T07:02:44.634 INFO:teuthology.orchestra.run.smithi188.stdout:-rw-r--r-- root/root 8353 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hid/hid-aureal.ko 2023-11-26T07:02:44.634 INFO:teuthology.orchestra.run.smithi188.stdout:-rw-r--r-- root/root 15169 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hid/hid-axff.ko 2023-11-26T07:02:44.634 INFO:teuthology.orchestra.run.smithi188.stdout:-rw-r--r-- root/root 11049 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hid/hid-belkin.ko 2023-11-26T07:02:44.634 INFO:teuthology.orchestra.run.smithi188.stdout:-rw-r--r-- root/root 10313 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hid/hid-cherry.ko 2023-11-26T07:02:44.634 INFO:teuthology.orchestra.run.smithi188.stdout:-rw-r--r-- root/root 16041 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hid/hid-chicony.ko 2023-11-26T07:02:44.634 INFO:teuthology.orchestra.run.smithi188.stdout:-rw-r--r-- root/root 60649 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hid/hid-cp2112.ko 2023-11-26T07:02:44.634 INFO:teuthology.orchestra.run.smithi188.stdout:-rw-r--r-- root/root 11049 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hid/hid-cypress.ko 2023-11-26T07:02:44.634 INFO:teuthology.orchestra.run.smithi188.stdout:-rw-r--r-- root/root 16385 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hid/hid-dr.ko 2023-11-26T07:02:44.634 INFO:teuthology.orchestra.run.smithi188.stdout:-rw-r--r-- root/root 10697 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hid/hid-elecom.ko 2023-11-26T07:02:44.635 INFO:teuthology.orchestra.run.smithi188.stdout:-rw-r--r-- root/root 17665 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hid/hid-elo.ko 2023-11-26T07:02:44.635 INFO:teuthology.orchestra.run.smithi188.stdout:-rw-r--r-- root/root 13457 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hid/hid-emsff.ko 2023-11-26T07:02:44.635 INFO:teuthology.orchestra.run.smithi188.stdout:-rw-r--r-- root/root 10105 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hid/hid-ezkey.ko 2023-11-26T07:02:44.635 INFO:teuthology.orchestra.run.smithi188.stdout:-rw-r--r-- root/root 14305 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hid/hid-gaff.ko 2023-11-26T07:02:44.635 INFO:teuthology.orchestra.run.smithi188.stdout:-rw-r--r-- root/root 9225 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hid/hid-generic.ko 2023-11-26T07:02:44.635 INFO:teuthology.orchestra.run.smithi188.stdout:-rw-r--r-- root/root 17129 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hid/hid-gt683r.ko 2023-11-26T07:02:44.635 INFO:teuthology.orchestra.run.smithi188.stdout:-rw-r--r-- root/root 12681 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hid/hid-gyration.ko 2023-11-26T07:02:44.635 INFO:teuthology.orchestra.run.smithi188.stdout:-rw-r--r-- root/root 10161 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hid/hid-holtek-kbd.ko 2023-11-26T07:02:44.635 INFO:teuthology.orchestra.run.smithi188.stdout:-rw-r--r-- root/root 10457 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hid/hid-holtek-mouse.ko 2023-11-26T07:02:44.635 INFO:teuthology.orchestra.run.smithi188.stdout:-rw-r--r-- root/root 14449 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hid/hid-holtekff.ko 2023-11-26T07:02:44.635 INFO:teuthology.orchestra.run.smithi188.stdout:-rw-r--r-- root/root 19105 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hid/hid-hyperv.ko 2023-11-26T07:02:44.636 INFO:teuthology.orchestra.run.smithi188.stdout:-rw-r--r-- root/root 10489 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hid/hid-icade.ko 2023-11-26T07:02:44.636 INFO:teuthology.orchestra.run.smithi188.stdout:-rw-r--r-- root/root 9241 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hid/hid-kensington.ko 2023-11-26T07:02:44.636 INFO:teuthology.orchestra.run.smithi188.stdout:-rw-r--r-- root/root 8545 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hid/hid-keytouch.ko 2023-11-26T07:02:44.636 INFO:teuthology.orchestra.run.smithi188.stdout:-rw-r--r-- root/root 17201 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hid/hid-kye.ko 2023-11-26T07:02:44.636 INFO:teuthology.orchestra.run.smithi188.stdout:-rw-r--r-- root/root 10809 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hid/hid-lcpower.ko 2023-11-26T07:02:44.636 INFO:teuthology.orchestra.run.smithi188.stdout:-rw-r--r-- root/root 20497 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hid/hid-led.ko 2023-11-26T07:02:44.636 INFO:teuthology.orchestra.run.smithi188.stdout:-rw-r--r-- root/root 42297 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hid/hid-lenovo.ko 2023-11-26T07:02:44.636 INFO:teuthology.orchestra.run.smithi188.stdout:-rw-r--r-- root/root 30697 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hid/hid-lg-g15.ko 2023-11-26T07:02:44.636 INFO:teuthology.orchestra.run.smithi188.stdout:-rw-r--r-- root/root 58185 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hid/hid-logitech-dj.ko 2023-11-26T07:02:44.636 INFO:teuthology.orchestra.run.smithi188.stdout:-rw-r--r-- root/root 101481 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hid/hid-logitech-hidpp.ko 2023-11-26T07:02:44.636 INFO:teuthology.orchestra.run.smithi188.stdout:-rw-r--r-- root/root 80993 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hid/hid-logitech.ko 2023-11-26T07:02:44.637 INFO:teuthology.orchestra.run.smithi188.stdout:-rw-r--r-- root/root 30801 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hid/hid-magicmouse.ko 2023-11-26T07:02:44.637 INFO:teuthology.orchestra.run.smithi188.stdout:-rw-r--r-- root/root 21969 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hid/hid-microsoft.ko 2023-11-26T07:02:44.637 INFO:teuthology.orchestra.run.smithi188.stdout:-rw-r--r-- root/root 10361 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hid/hid-monterey.ko 2023-11-26T07:02:44.637 INFO:teuthology.orchestra.run.smithi188.stdout:-rw-r--r-- root/root 50161 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hid/hid-multitouch.ko 2023-11-26T07:02:44.637 INFO:teuthology.orchestra.run.smithi188.stdout:-rw-r--r-- root/root 32593 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hid/hid-ntrig.ko 2023-11-26T07:02:44.637 INFO:teuthology.orchestra.run.smithi188.stdout:-rw-r--r-- root/root 8857 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hid/hid-ortek.ko 2023-11-26T07:02:44.637 INFO:teuthology.orchestra.run.smithi188.stdout:-rw-r--r-- root/root 9121 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hid/hid-penmount.ko 2023-11-26T07:02:44.637 INFO:teuthology.orchestra.run.smithi188.stdout:-rw-r--r-- root/root 12289 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hid/hid-petalynx.ko 2023-11-26T07:02:44.637 INFO:teuthology.orchestra.run.smithi188.stdout:-rw-r--r-- root/root 72937 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hid/hid-picolcd.ko 2023-11-26T07:02:44.637 INFO:teuthology.orchestra.run.smithi188.stdout:-rw-r--r-- root/root 14225 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hid/hid-pl.ko 2023-11-26T07:02:44.637 INFO:teuthology.orchestra.run.smithi188.stdout:-rw-r--r-- root/root 15105 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hid/hid-plantronics.ko 2023-11-26T07:02:44.638 INFO:teuthology.orchestra.run.smithi188.stdout:-rw-r--r-- root/root 8417 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hid/hid-primax.ko 2023-11-26T07:02:44.638 INFO:teuthology.orchestra.run.smithi188.stdout:-rw-r--r-- root/root 35393 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hid/hid-rmi.ko 2023-11-26T07:02:44.638 INFO:teuthology.orchestra.run.smithi188.stdout:-rw-r--r-- root/root 20833 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hid/hid-roccat-arvo.ko 2023-11-26T07:02:44.638 INFO:teuthology.orchestra.run.smithi188.stdout:-rw-r--r-- root/root 14065 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hid/hid-roccat-common.ko 2023-11-26T07:02:44.638 INFO:teuthology.orchestra.run.smithi188.stdout:-rw-r--r-- root/root 31425 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hid/hid-roccat-isku.ko 2023-11-26T07:02:44.638 INFO:teuthology.orchestra.run.smithi188.stdout:-rw-r--r-- root/root 29729 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hid/hid-roccat-kone.ko 2023-11-26T07:02:44.638 INFO:teuthology.orchestra.run.smithi188.stdout:-rw-r--r-- root/root 31129 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hid/hid-roccat-koneplus.ko 2023-11-26T07:02:44.638 INFO:teuthology.orchestra.run.smithi188.stdout:-rw-r--r-- root/root 21137 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hid/hid-roccat-konepure.ko 2023-11-26T07:02:44.638 INFO:teuthology.orchestra.run.smithi188.stdout:-rw-r--r-- root/root 29489 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hid/hid-roccat-kovaplus.ko 2023-11-26T07:02:44.638 INFO:teuthology.orchestra.run.smithi188.stdout:-rw-r--r-- root/root 13305 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hid/hid-roccat-lua.ko 2023-11-26T07:02:44.638 INFO:teuthology.orchestra.run.smithi188.stdout:-rw-r--r-- root/root 29313 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hid/hid-roccat-pyra.ko 2023-11-26T07:02:44.639 INFO:teuthology.orchestra.run.smithi188.stdout:-rw-r--r-- root/root 27937 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hid/hid-roccat-ryos.ko 2023-11-26T07:02:44.639 INFO:teuthology.orchestra.run.smithi188.stdout:-rw-r--r-- root/root 19521 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hid/hid-roccat-savu.ko 2023-11-26T07:02:44.639 INFO:teuthology.orchestra.run.smithi188.stdout:-rw-r--r-- root/root 23337 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hid/hid-roccat.ko 2023-11-26T07:02:44.639 INFO:teuthology.orchestra.run.smithi188.stdout:-rw-r--r-- root/root 13681 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hid/hid-saitek.ko 2023-11-26T07:02:44.639 INFO:teuthology.orchestra.run.smithi188.stdout:-rw-r--r-- root/root 15193 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hid/hid-samsung.ko 2023-11-26T07:02:44.639 INFO:teuthology.orchestra.run.smithi188.stdout:-rw-r--r-- root/root 36817 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hid/hid-sensor-hub.ko 2023-11-26T07:02:44.639 INFO:teuthology.orchestra.run.smithi188.stdout:-rw-r--r-- root/root 13601 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hid/hid-sjoy.ko 2023-11-26T07:02:44.639 INFO:teuthology.orchestra.run.smithi188.stdout:-rw-r--r-- root/root 51249 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hid/hid-sony.ko 2023-11-26T07:02:44.639 INFO:teuthology.orchestra.run.smithi188.stdout:-rw-r--r-- root/root 8313 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hid/hid-speedlink.ko 2023-11-26T07:02:44.639 INFO:teuthology.orchestra.run.smithi188.stdout:-rw-r--r-- root/root 25297 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hid/hid-steelseries.ko 2023-11-26T07:02:44.640 INFO:teuthology.orchestra.run.smithi188.stdout:-rw-r--r-- root/root 9817 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hid/hid-sunplus.ko 2023-11-26T07:02:44.640 INFO:teuthology.orchestra.run.smithi188.stdout:-rw-r--r-- root/root 16993 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hid/hid-thrustmaster.ko 2023-11-26T07:02:44.640 INFO:teuthology.orchestra.run.smithi188.stdout:-rw-r--r-- root/root 10537 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hid/hid-tivo.ko 2023-11-26T07:02:44.640 INFO:teuthology.orchestra.run.smithi188.stdout:-rw-r--r-- root/root 15089 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hid/hid-tmff.ko 2023-11-26T07:02:44.640 INFO:teuthology.orchestra.run.smithi188.stdout:-rw-r--r-- root/root 14385 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hid/hid-topseed.ko 2023-11-26T07:02:44.640 INFO:teuthology.orchestra.run.smithi188.stdout:-rw-r--r-- root/root 16473 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hid/hid-twinhan.ko 2023-11-26T07:02:44.640 INFO:teuthology.orchestra.run.smithi188.stdout:-rw-r--r-- root/root 71073 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hid/hid-uclogic.ko 2023-11-26T07:02:44.640 INFO:teuthology.orchestra.run.smithi188.stdout:-rw-r--r-- root/root 10921 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hid/hid-waltop.ko 2023-11-26T07:02:44.640 INFO:teuthology.orchestra.run.smithi188.stdout:-rw-r--r-- root/root 143473 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hid/hid-wiimote.ko 2023-11-26T07:02:44.640 INFO:teuthology.orchestra.run.smithi188.stdout:-rw-r--r-- root/root 8153 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hid/hid-xinmo.ko 2023-11-26T07:02:44.640 INFO:teuthology.orchestra.run.smithi188.stdout:-rw-r--r-- root/root 13321 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hid/hid-zpff.ko 2023-11-26T07:02:44.641 INFO:teuthology.orchestra.run.smithi188.stdout:-rw-r--r-- root/root 16665 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hid/hid-zydacron.ko 2023-11-26T07:02:44.641 INFO:teuthology.orchestra.run.smithi188.stdout:-rw-r--r-- root/root 267825 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hid/hid.ko 2023-11-26T07:02:44.641 INFO:teuthology.orchestra.run.smithi188.stdout:-rw-r--r-- root/root 32321 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hid/uhid.ko 2023-11-26T07:02:44.641 INFO:teuthology.orchestra.run.smithi188.stdout:drwxr-xr-x root/root 0 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hid/usbhid/ 2023-11-26T07:02:44.641 INFO:teuthology.orchestra.run.smithi188.stdout:-rw-r--r-- root/root 125721 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hid/usbhid/usbhid.ko 2023-11-26T07:02:44.641 INFO:teuthology.orchestra.run.smithi188.stdout:-rw-r--r-- root/root 19137 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hid/usbhid/usbkbd.ko 2023-11-26T07:02:44.641 INFO:teuthology.orchestra.run.smithi188.stdout:-rw-r--r-- root/root 14953 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hid/usbhid/usbmouse.ko 2023-11-26T07:02:44.641 INFO:teuthology.orchestra.run.smithi188.stdout:-rw-r--r-- root/root 217873 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hid/wacom.ko 2023-11-26T07:02:44.641 INFO:teuthology.orchestra.run.smithi188.stdout:drwxr-xr-x root/root 0 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hv/ 2023-11-26T07:02:44.642 INFO:teuthology.orchestra.run.smithi188.stdout:-rw-r--r-- root/root 64201 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hv/hv_balloon.ko 2023-11-26T07:02:44.642 INFO:teuthology.orchestra.run.smithi188.stdout:-rw-r--r-- root/root 89625 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hv/hv_utils.ko 2023-11-26T07:02:44.642 INFO:teuthology.orchestra.run.smithi188.stdout:-rw-r--r-- root/root 305465 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hv/hv_vmbus.ko 2023-11-26T07:02:44.642 INFO:teuthology.orchestra.run.smithi188.stdout:drwxr-xr-x root/root 0 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/ 2023-11-26T07:02:44.642 INFO:teuthology.orchestra.run.smithi188.stdout:-rw-r--r-- root/root 66521 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/abituguru.ko 2023-11-26T07:02:44.642 INFO:teuthology.orchestra.run.smithi188.stdout:-rw-r--r-- root/root 61393 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/abituguru3.ko 2023-11-26T07:02:44.642 INFO:teuthology.orchestra.run.smithi188.stdout:-rw-r--r-- root/root 34745 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/acpi_power_meter.ko 2023-11-26T07:02:44.642 INFO:teuthology.orchestra.run.smithi188.stdout:-rw-r--r-- root/root 17505 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/ad7414.ko 2023-11-26T07:02:44.642 INFO:teuthology.orchestra.run.smithi188.stdout:-rw-r--r-- root/root 19313 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/ad7418.ko 2023-11-26T07:02:44.642 INFO:teuthology.orchestra.run.smithi188.stdout:-rw-r--r-- root/root 25257 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/adc128d818.ko 2023-11-26T07:02:44.643 INFO:teuthology.orchestra.run.smithi188.stdout:-rw-r--r-- root/root 31729 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/adm1025.ko 2023-11-26T07:02:44.643 INFO:teuthology.orchestra.run.smithi188.stdout:-rw-r--r-- root/root 97977 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/adm1026.ko 2023-11-26T07:02:44.643 INFO:teuthology.orchestra.run.smithi188.stdout:-rw-r--r-- root/root 19745 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/adm1029.ko 2023-11-26T07:02:44.643 INFO:teuthology.orchestra.run.smithi188.stdout:-rw-r--r-- root/root 45553 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/adm1031.ko 2023-11-26T07:02:44.643 INFO:teuthology.orchestra.run.smithi188.stdout:-rw-r--r-- root/root 26289 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/adm9240.ko 2023-11-26T07:02:44.643 INFO:teuthology.orchestra.run.smithi188.stdout:-rw-r--r-- root/root 14769 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/ads7828.ko 2023-11-26T07:02:44.643 INFO:teuthology.orchestra.run.smithi188.stdout:-rw-r--r-- root/root 10769 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/adt7410.ko 2023-11-26T07:02:44.643 INFO:teuthology.orchestra.run.smithi188.stdout:-rw-r--r-- root/root 22049 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/adt7411.ko 2023-11-26T07:02:44.643 INFO:teuthology.orchestra.run.smithi188.stdout:-rw-r--r-- root/root 74433 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/adt7462.ko 2023-11-26T07:02:44.643 INFO:teuthology.orchestra.run.smithi188.stdout:-rw-r--r-- root/root 39585 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/adt7470.ko 2023-11-26T07:02:44.643 INFO:teuthology.orchestra.run.smithi188.stdout:-rw-r--r-- root/root 67929 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/adt7475.ko 2023-11-26T07:02:44.644 INFO:teuthology.orchestra.run.smithi188.stdout:-rw-r--r-- root/root 19625 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/adt7x10.ko 2023-11-26T07:02:44.644 INFO:teuthology.orchestra.run.smithi188.stdout:-rw-r--r-- root/root 39073 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/amc6821.ko 2023-11-26T07:02:44.644 INFO:teuthology.orchestra.run.smithi188.stdout:-rw-r--r-- root/root 55201 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/applesmc.ko 2023-11-26T07:02:44.644 INFO:teuthology.orchestra.run.smithi188.stdout:-rw-r--r-- root/root 49913 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/asb100.ko 2023-11-26T07:02:44.644 INFO:teuthology.orchestra.run.smithi188.stdout:-rw-r--r-- root/root 58617 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/asc7621.ko 2023-11-26T07:02:44.644 INFO:teuthology.orchestra.run.smithi188.stdout:-rw-r--r-- root/root 40825 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/asus_atk0110.ko 2023-11-26T07:02:44.644 INFO:teuthology.orchestra.run.smithi188.stdout:-rw-r--r-- root/root 17225 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/atxp1.ko 2023-11-26T07:02:44.644 INFO:teuthology.orchestra.run.smithi188.stdout:-rw-r--r-- root/root 28281 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/coretemp.ko 2023-11-26T07:02:44.644 INFO:teuthology.orchestra.run.smithi188.stdout:-rw-r--r-- root/root 24593 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/da9052-hwmon.ko 2023-11-26T07:02:44.644 INFO:teuthology.orchestra.run.smithi188.stdout:-rw-r--r-- root/root 17265 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/da9055-hwmon.ko 2023-11-26T07:02:44.644 INFO:teuthology.orchestra.run.smithi188.stdout:-rw-r--r-- root/root 44465 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/dell-smm-hwmon.ko 2023-11-26T07:02:44.644 INFO:teuthology.orchestra.run.smithi188.stdout:-rw-r--r-- root/root 99601 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/dme1737.ko 2023-11-26T07:02:44.645 INFO:teuthology.orchestra.run.smithi188.stdout:-rw-r--r-- root/root 19489 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/ds1621.ko 2023-11-26T07:02:44.645 INFO:teuthology.orchestra.run.smithi188.stdout:-rw-r--r-- root/root 15321 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/ds620.ko 2023-11-26T07:02:44.645 INFO:teuthology.orchestra.run.smithi188.stdout:-rw-r--r-- root/root 29777 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/emc1403.ko 2023-11-26T07:02:44.645 INFO:teuthology.orchestra.run.smithi188.stdout:-rw-r--r-- root/root 33185 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/emc2103.ko 2023-11-26T07:02:44.645 INFO:teuthology.orchestra.run.smithi188.stdout:-rw-r--r-- root/root 30321 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/emc6w201.ko 2023-11-26T07:02:44.645 INFO:teuthology.orchestra.run.smithi188.stdout:-rw-r--r-- root/root 66985 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/f71805f.ko 2023-11-26T07:02:44.645 INFO:teuthology.orchestra.run.smithi188.stdout:-rw-r--r-- root/root 94457 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/f71882fg.ko 2023-11-26T07:02:44.645 INFO:teuthology.orchestra.run.smithi188.stdout:-rw-r--r-- root/root 41401 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/f75375s.ko 2023-11-26T07:02:44.645 INFO:teuthology.orchestra.run.smithi188.stdout:-rw-r--r-- root/root 21305 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/fam15h_power.ko 2023-11-26T07:02:44.645 INFO:teuthology.orchestra.run.smithi188.stdout:-rw-r--r-- root/root 55449 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/fschmd.ko 2023-11-26T07:02:44.646 INFO:teuthology.orchestra.run.smithi188.stdout:-rw-r--r-- root/root 14481 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/g760a.ko 2023-11-26T07:02:44.646 INFO:teuthology.orchestra.run.smithi188.stdout:-rw-r--r-- root/root 24017 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/g762.ko 2023-11-26T07:02:44.646 INFO:teuthology.orchestra.run.smithi188.stdout:-rw-r--r-- root/root 46801 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/gl518sm.ko 2023-11-26T07:02:44.646 INFO:teuthology.orchestra.run.smithi188.stdout:-rw-r--r-- root/root 45529 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/gl520sm.ko 2023-11-26T07:02:44.646 INFO:teuthology.orchestra.run.smithi188.stdout:-rw-r--r-- root/root 13273 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/hih6130.ko 2023-11-26T07:02:44.646 INFO:teuthology.orchestra.run.smithi188.stdout:-rw-r--r-- root/root 9745 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/hwmon-vid.ko 2023-11-26T07:02:44.646 INFO:teuthology.orchestra.run.smithi188.stdout:-rw-r--r-- root/root 11553 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/i5500_temp.ko 2023-11-26T07:02:44.646 INFO:teuthology.orchestra.run.smithi188.stdout:-rw-r--r-- root/root 22601 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/i5k_amb.ko 2023-11-26T07:02:44.646 INFO:teuthology.orchestra.run.smithi188.stdout:-rw-r--r-- root/root 34529 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/ibmaem.ko 2023-11-26T07:02:44.646 INFO:teuthology.orchestra.run.smithi188.stdout:-rw-r--r-- root/root 21257 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/ibmpex.ko 2023-11-26T07:02:44.646 INFO:teuthology.orchestra.run.smithi188.stdout:-rw-r--r-- root/root 24873 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/ina209.ko 2023-11-26T07:02:44.646 INFO:teuthology.orchestra.run.smithi188.stdout:-rw-r--r-- root/root 13401 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/ina2xx.ko 2023-11-26T07:02:44.647 INFO:teuthology.orchestra.run.smithi188.stdout:-rw-r--r-- root/root 130449 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/it87.ko 2023-11-26T07:02:44.647 INFO:teuthology.orchestra.run.smithi188.stdout:-rw-r--r-- root/root 18841 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/jc42.ko 2023-11-26T07:02:44.647 INFO:teuthology.orchestra.run.smithi188.stdout:-rw-r--r-- root/root 24329 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/k10temp.ko 2023-11-26T07:02:44.647 INFO:teuthology.orchestra.run.smithi188.stdout:-rw-r--r-- root/root 13553 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/k8temp.ko 2023-11-26T07:02:44.647 INFO:teuthology.orchestra.run.smithi188.stdout:-rw-r--r-- root/root 18929 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/lineage-pem.ko 2023-11-26T07:02:44.647 INFO:teuthology.orchestra.run.smithi188.stdout:-rw-r--r-- root/root 48601 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/lm63.ko 2023-11-26T07:02:44.647 INFO:teuthology.orchestra.run.smithi188.stdout:-rw-r--r-- root/root 16697 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/lm73.ko 2023-11-26T07:02:44.647 INFO:teuthology.orchestra.run.smithi188.stdout:-rw-r--r-- root/root 36777 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/lm75.ko 2023-11-26T07:02:44.647 INFO:teuthology.orchestra.run.smithi188.stdout:-rw-r--r-- root/root 19537 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/lm77.ko 2023-11-26T07:02:44.647 INFO:teuthology.orchestra.run.smithi188.stdout:-rw-r--r-- root/root 34593 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/lm78.ko 2023-11-26T07:02:44.647 INFO:teuthology.orchestra.run.smithi188.stdout:-rw-r--r-- root/root 31745 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/lm80.ko 2023-11-26T07:02:44.648 INFO:teuthology.orchestra.run.smithi188.stdout:-rw-r--r-- root/root 16889 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/lm83.ko 2023-11-26T07:02:44.648 INFO:teuthology.orchestra.run.smithi188.stdout:-rw-r--r-- root/root 72833 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/lm85.ko 2023-11-26T07:02:44.648 INFO:teuthology.orchestra.run.smithi188.stdout:-rw-r--r-- root/root 46153 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/lm87.ko 2023-11-26T07:02:44.648 INFO:teuthology.orchestra.run.smithi188.stdout:-rw-r--r-- root/root 37913 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/lm90.ko 2023-11-26T07:02:44.648 INFO:teuthology.orchestra.run.smithi188.stdout:-rw-r--r-- root/root 19745 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/lm92.ko 2023-11-26T07:02:44.648 INFO:teuthology.orchestra.run.smithi188.stdout:-rw-r--r-- root/root 108841 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/lm93.ko 2023-11-26T07:02:44.648 INFO:teuthology.orchestra.run.smithi188.stdout:-rw-r--r-- root/root 33393 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/lm95234.ko 2023-11-26T07:02:44.649 INFO:teuthology.orchestra.run.smithi188.stdout:-rw-r--r-- root/root 16201 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/lm95241.ko 2023-11-26T07:02:44.649 INFO:teuthology.orchestra.run.smithi188.stdout:-rw-r--r-- root/root 18113 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/lm95245.ko 2023-11-26T07:02:44.649 INFO:teuthology.orchestra.run.smithi188.stdout:-rw-r--r-- root/root 24017 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/ltc2945.ko 2023-11-26T07:02:44.649 INFO:teuthology.orchestra.run.smithi188.stdout:-rw-r--r-- root/root 13793 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/ltc4151.ko 2023-11-26T07:02:44.649 INFO:teuthology.orchestra.run.smithi188.stdout:-rw-r--r-- root/root 15193 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/ltc4215.ko 2023-11-26T07:02:44.649 INFO:teuthology.orchestra.run.smithi188.stdout:-rw-r--r-- root/root 14889 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/ltc4222.ko 2023-11-26T07:02:44.649 INFO:teuthology.orchestra.run.smithi188.stdout:-rw-r--r-- root/root 14825 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/ltc4245.ko 2023-11-26T07:02:44.649 INFO:teuthology.orchestra.run.smithi188.stdout:-rw-r--r-- root/root 13249 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/ltc4260.ko 2023-11-26T07:02:44.649 INFO:teuthology.orchestra.run.smithi188.stdout:-rw-r--r-- root/root 14993 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/ltc4261.ko 2023-11-26T07:02:44.650 INFO:teuthology.orchestra.run.smithi188.stdout:-rw-r--r-- root/root 33489 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/max16065.ko 2023-11-26T07:02:44.650 INFO:teuthology.orchestra.run.smithi188.stdout:-rw-r--r-- root/root 18433 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/max1619.ko 2023-11-26T07:02:44.650 INFO:teuthology.orchestra.run.smithi188.stdout:-rw-r--r-- root/root 24353 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/max1668.ko 2023-11-26T07:02:44.650 INFO:teuthology.orchestra.run.smithi188.stdout:-rw-r--r-- root/root 19969 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/max197.ko 2023-11-26T07:02:44.650 INFO:teuthology.orchestra.run.smithi188.stdout:-rw-r--r-- root/root 27129 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/max6639.ko 2023-11-26T07:02:44.650 INFO:teuthology.orchestra.run.smithi188.stdout:-rw-r--r-- root/root 23321 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/max6650.ko 2023-11-26T07:02:44.650 INFO:teuthology.orchestra.run.smithi188.stdout:-rw-r--r-- root/root 35777 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/max6697.ko 2023-11-26T07:02:44.650 INFO:teuthology.orchestra.run.smithi188.stdout:-rw-r--r-- root/root 18585 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/mc13783-adc.ko 2023-11-26T07:02:44.650 INFO:teuthology.orchestra.run.smithi188.stdout:-rw-r--r-- root/root 10585 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/mcp3021.ko 2023-11-26T07:02:44.650 INFO:teuthology.orchestra.run.smithi188.stdout:-rw-r--r-- root/root 15937 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/menf21bmc_hwmon.ko 2023-11-26T07:02:44.650 INFO:teuthology.orchestra.run.smithi188.stdout:-rw-r--r-- root/root 44129 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/nct6683.ko 2023-11-26T07:02:44.651 INFO:teuthology.orchestra.run.smithi188.stdout:-rw-r--r-- root/root 119817 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/nct6775-core.ko 2023-11-26T07:02:44.651 INFO:teuthology.orchestra.run.smithi188.stdout:-rw-r--r-- root/root 60705 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/nct6775.ko 2023-11-26T07:02:44.651 INFO:teuthology.orchestra.run.smithi188.stdout:-rw-r--r-- root/root 59737 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/nct7802.ko 2023-11-26T07:02:44.651 INFO:teuthology.orchestra.run.smithi188.stdout:-rw-r--r-- root/root 82337 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/pc87360.ko 2023-11-26T07:02:44.651 INFO:teuthology.orchestra.run.smithi188.stdout:-rw-r--r-- root/root 54433 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/pc87427.ko 2023-11-26T07:02:44.651 INFO:teuthology.orchestra.run.smithi188.stdout:-rw-r--r-- root/root 18473 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/pcf8591.ko 2023-11-26T07:02:44.651 INFO:teuthology.orchestra.run.smithi188.stdout:drwxr-xr-x root/root 0 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/pmbus/ 2023-11-26T07:02:44.651 INFO:teuthology.orchestra.run.smithi188.stdout:-rw-r--r-- root/root 21353 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/pmbus/adm1275.ko 2023-11-26T07:02:44.651 INFO:teuthology.orchestra.run.smithi188.stdout:-rw-r--r-- root/root 17065 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/pmbus/lm25066.ko 2023-11-26T07:02:44.651 INFO:teuthology.orchestra.run.smithi188.stdout:-rw-r--r-- root/root 28329 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/pmbus/ltc2978.ko 2023-11-26T07:02:44.651 INFO:teuthology.orchestra.run.smithi188.stdout:-rw-r--r-- root/root 9577 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/pmbus/max16064.ko 2023-11-26T07:02:44.652 INFO:teuthology.orchestra.run.smithi188.stdout:-rw-r--r-- root/root 17657 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/pmbus/max34440.ko 2023-11-26T07:02:44.652 INFO:teuthology.orchestra.run.smithi188.stdout:-rw-r--r-- root/root 10169 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/pmbus/max8688.ko 2023-11-26T07:02:44.652 INFO:teuthology.orchestra.run.smithi188.stdout:-rw-r--r-- root/root 15505 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/pmbus/pmbus.ko 2023-11-26T07:02:44.652 INFO:teuthology.orchestra.run.smithi188.stdout:-rw-r--r-- root/root 82577 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/pmbus/pmbus_core.ko 2023-11-26T07:02:44.652 INFO:teuthology.orchestra.run.smithi188.stdout:-rw-r--r-- root/root 8729 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/pmbus/tps40422.ko 2023-11-26T07:02:44.652 INFO:teuthology.orchestra.run.smithi188.stdout:-rw-r--r-- root/root 30729 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/pmbus/ucd9000.ko 2023-11-26T07:02:44.652 INFO:teuthology.orchestra.run.smithi188.stdout:-rw-r--r-- root/root 15073 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/pmbus/ucd9200.ko 2023-11-26T07:02:44.652 INFO:teuthology.orchestra.run.smithi188.stdout:-rw-r--r-- root/root 20681 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/pmbus/zl6100.ko 2023-11-26T07:02:44.652 INFO:teuthology.orchestra.run.smithi188.stdout:-rw-r--r-- root/root 13017 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/powr1220.ko 2023-11-26T07:02:44.652 INFO:teuthology.orchestra.run.smithi188.stdout:-rw-r--r-- root/root 18705 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/sch5627.ko 2023-11-26T07:02:44.652 INFO:teuthology.orchestra.run.smithi188.stdout:-rw-r--r-- root/root 28817 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/sch5636.ko 2023-11-26T07:02:44.653 INFO:teuthology.orchestra.run.smithi188.stdout:-rw-r--r-- root/root 27145 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/sch56xx-common.ko 2023-11-26T07:02:44.653 INFO:teuthology.orchestra.run.smithi188.stdout:-rw-r--r-- root/root 32137 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/sht15.ko 2023-11-26T07:02:44.653 INFO:teuthology.orchestra.run.smithi188.stdout:-rw-r--r-- root/root 13505 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/sht21.ko 2023-11-26T07:02:44.653 INFO:teuthology.orchestra.run.smithi188.stdout:-rw-r--r-- root/root 16833 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/shtc1.ko 2023-11-26T07:02:44.653 INFO:teuthology.orchestra.run.smithi188.stdout:-rw-r--r-- root/root 40105 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/sis5595.ko 2023-11-26T07:02:44.653 INFO:teuthology.orchestra.run.smithi188.stdout:-rw-r--r-- root/root 19577 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/smsc47b397.ko 2023-11-26T07:02:44.653 INFO:teuthology.orchestra.run.smithi188.stdout:-rw-r--r-- root/root 39193 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/smsc47m1.ko 2023-11-26T07:02:44.653 INFO:teuthology.orchestra.run.smithi188.stdout:-rw-r--r-- root/root 37385 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/smsc47m192.ko 2023-11-26T07:02:44.653 INFO:teuthology.orchestra.run.smithi188.stdout:-rw-r--r-- root/root 26553 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/thmc50.ko 2023-11-26T07:02:44.653 INFO:teuthology.orchestra.run.smithi188.stdout:-rw-r--r-- root/root 17057 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/tmp102.ko 2023-11-26T07:02:44.653 INFO:teuthology.orchestra.run.smithi188.stdout:-rw-r--r-- root/root 14097 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/tmp103.ko 2023-11-26T07:02:44.653 INFO:teuthology.orchestra.run.smithi188.stdout:-rw-r--r-- root/root 21257 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/tmp401.ko 2023-11-26T07:02:44.654 INFO:teuthology.orchestra.run.smithi188.stdout:-rw-r--r-- root/root 19537 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/tmp421.ko 2023-11-26T07:02:44.654 INFO:teuthology.orchestra.run.smithi188.stdout:-rw-r--r-- root/root 19721 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/via-cputemp.ko 2023-11-26T07:02:44.654 INFO:teuthology.orchestra.run.smithi188.stdout:-rw-r--r-- root/root 41945 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/via686a.ko 2023-11-26T07:02:44.654 INFO:teuthology.orchestra.run.smithi188.stdout:-rw-r--r-- root/root 52457 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/vt1211.ko 2023-11-26T07:02:44.654 INFO:teuthology.orchestra.run.smithi188.stdout:-rw-r--r-- root/root 51545 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/vt8231.ko 2023-11-26T07:02:44.654 INFO:teuthology.orchestra.run.smithi188.stdout:-rw-r--r-- root/root 57449 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/w83627ehf.ko 2023-11-26T07:02:44.654 INFO:teuthology.orchestra.run.smithi188.stdout:-rw-r--r-- root/root 78873 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/w83627hf.ko 2023-11-26T07:02:44.654 INFO:teuthology.orchestra.run.smithi188.stdout:-rw-r--r-- root/root 71345 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/w83781d.ko 2023-11-26T07:02:44.654 INFO:teuthology.orchestra.run.smithi188.stdout:-rw-r--r-- root/root 63225 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/w83791d.ko 2023-11-26T07:02:44.654 INFO:teuthology.orchestra.run.smithi188.stdout:-rw-r--r-- root/root 71321 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/w83792d.ko 2023-11-26T07:02:44.654 INFO:teuthology.orchestra.run.smithi188.stdout:-rw-r--r-- root/root 101473 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/w83793.ko 2023-11-26T07:02:44.655 INFO:teuthology.orchestra.run.smithi188.stdout:-rw-r--r-- root/root 119505 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/w83795.ko 2023-11-26T07:02:44.655 INFO:teuthology.orchestra.run.smithi188.stdout:-rw-r--r-- root/root 17313 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/w83l785ts.ko 2023-11-26T07:02:44.655 INFO:teuthology.orchestra.run.smithi188.stdout:-rw-r--r-- root/root 35625 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/w83l786ng.ko 2023-11-26T07:02:44.655 INFO:teuthology.orchestra.run.smithi188.stdout:-rw-r--r-- root/root 13825 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/wm831x-hwmon.ko 2023-11-26T07:02:44.655 INFO:teuthology.orchestra.run.smithi188.stdout:-rw-r--r-- root/root 10625 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/wm8350-hwmon.ko 2023-11-26T07:02:44.655 INFO:teuthology.orchestra.run.smithi188.stdout:drwxr-xr-x root/root 0 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/i2c/ 2023-11-26T07:02:44.655 INFO:teuthology.orchestra.run.smithi188.stdout:drwxr-xr-x root/root 0 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/i2c/algos/ 2023-11-26T07:02:44.655 INFO:teuthology.orchestra.run.smithi188.stdout:-rw-r--r-- root/root 20049 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/i2c/algos/i2c-algo-bit.ko 2023-11-26T07:02:44.655 INFO:teuthology.orchestra.run.smithi188.stdout:-rw-r--r-- root/root 19345 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/i2c/algos/i2c-algo-pca.ko 2023-11-26T07:02:44.655 INFO:teuthology.orchestra.run.smithi188.stdout:drwxr-xr-x root/root 0 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/i2c/busses/ 2023-11-26T07:02:44.656 INFO:teuthology.orchestra.run.smithi188.stdout:-rw-r--r-- root/root 27969 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/i2c/busses/i2c-ali1535.ko 2023-11-26T07:02:44.656 INFO:teuthology.orchestra.run.smithi188.stdout:-rw-r--r-- root/root 28801 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/i2c/busses/i2c-ali1563.ko 2023-11-26T07:02:44.656 INFO:teuthology.orchestra.run.smithi188.stdout:-rw-r--r-- root/root 27777 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/i2c/busses/i2c-ali15x3.ko 2023-11-26T07:02:44.656 INFO:teuthology.orchestra.run.smithi188.stdout:-rw-r--r-- root/root 17937 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/i2c/busses/i2c-amd756-s4882.ko 2023-11-26T07:02:44.656 INFO:teuthology.orchestra.run.smithi188.stdout:-rw-r--r-- root/root 26193 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/i2c/busses/i2c-amd756.ko 2023-11-26T07:02:44.656 INFO:teuthology.orchestra.run.smithi188.stdout:-rw-r--r-- root/root 16201 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/i2c/busses/i2c-amd8111.ko 2023-11-26T07:02:44.656 INFO:teuthology.orchestra.run.smithi188.stdout:-rw-r--r-- root/root 15153 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/i2c/busses/i2c-cbus-gpio.ko 2023-11-26T07:02:44.656 INFO:teuthology.orchestra.run.smithi188.stdout:-rw-r--r-- root/root 6929 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/i2c/busses/i2c-ccgx-ucsi.ko 2023-11-26T07:02:44.656 INFO:teuthology.orchestra.run.smithi188.stdout:-rw-r--r-- root/root 13961 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/i2c/busses/i2c-cros-ec-tunnel.ko 2023-11-26T07:02:44.657 INFO:teuthology.orchestra.run.smithi188.stdout:-rw-r--r-- root/root 50201 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/i2c/busses/i2c-designware-core.ko 2023-11-26T07:02:44.657 INFO:teuthology.orchestra.run.smithi188.stdout:-rw-r--r-- root/root 23905 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/i2c/busses/i2c-designware-pci.ko 2023-11-26T07:02:44.657 INFO:teuthology.orchestra.run.smithi188.stdout:-rw-r--r-- root/root 20001 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/i2c/busses/i2c-designware-platform.ko 2023-11-26T07:02:44.657 INFO:teuthology.orchestra.run.smithi188.stdout:-rw-r--r-- root/root 17337 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/i2c/busses/i2c-diolan-u2c.ko 2023-11-26T07:02:44.657 INFO:teuthology.orchestra.run.smithi188.stdout:-rw-r--r-- root/root 12497 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/i2c/busses/i2c-dln2.ko 2023-11-26T07:02:44.657 INFO:teuthology.orchestra.run.smithi188.stdout:-rw-r--r-- root/root 17585 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/i2c/busses/i2c-gpio.ko 2023-11-26T07:02:44.657 INFO:teuthology.orchestra.run.smithi188.stdout:-rw-r--r-- root/root 62817 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/i2c/busses/i2c-i801.ko 2023-11-26T07:02:44.657 INFO:teuthology.orchestra.run.smithi188.stdout:-rw-r--r-- root/root 22433 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/i2c/busses/i2c-isch.ko 2023-11-26T07:02:44.658 INFO:teuthology.orchestra.run.smithi188.stdout:-rw-r--r-- root/root 47889 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/i2c/busses/i2c-ismt.ko 2023-11-26T07:02:44.658 INFO:teuthology.orchestra.run.smithi188.stdout:-rw-r--r-- root/root 17721 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/i2c/busses/i2c-kempld.ko 2023-11-26T07:02:44.658 INFO:teuthology.orchestra.run.smithi188.stdout:-rw-r--r-- root/root 17881 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/i2c/busses/i2c-nforce2-s4985.ko 2023-11-26T07:02:44.658 INFO:teuthology.orchestra.run.smithi188.stdout:-rw-r--r-- root/root 22337 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/i2c/busses/i2c-nforce2.ko 2023-11-26T07:02:44.658 INFO:teuthology.orchestra.run.smithi188.stdout:-rw-r--r-- root/root 26265 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/i2c/busses/i2c-ocores.ko 2023-11-26T07:02:44.658 INFO:teuthology.orchestra.run.smithi188.stdout:-rw-r--r-- root/root 25145 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/i2c/busses/i2c-parport.ko 2023-11-26T07:02:44.658 INFO:teuthology.orchestra.run.smithi188.stdout:-rw-r--r-- root/root 16497 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/i2c/busses/i2c-pca-platform.ko 2023-11-26T07:02:44.658 INFO:teuthology.orchestra.run.smithi188.stdout:-rw-r--r-- root/root 47241 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/i2c/busses/i2c-piix4.ko 2023-11-26T07:02:44.659 INFO:teuthology.orchestra.run.smithi188.stdout:-rw-r--r-- root/root 15289 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/i2c/busses/i2c-robotfuzz-osif.ko 2023-11-26T07:02:44.659 INFO:teuthology.orchestra.run.smithi188.stdout:-rw-r--r-- root/root 18649 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/i2c/busses/i2c-scmi.ko 2023-11-26T07:02:44.659 INFO:teuthology.orchestra.run.smithi188.stdout:-rw-r--r-- root/root 11809 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/i2c/busses/i2c-simtec.ko 2023-11-26T07:02:44.659 INFO:teuthology.orchestra.run.smithi188.stdout:-rw-r--r-- root/root 22313 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/i2c/busses/i2c-sis5595.ko 2023-11-26T07:02:44.659 INFO:teuthology.orchestra.run.smithi188.stdout:-rw-r--r-- root/root 29361 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/i2c/busses/i2c-sis630.ko 2023-11-26T07:02:44.659 INFO:teuthology.orchestra.run.smithi188.stdout:-rw-r--r-- root/root 20329 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/i2c/busses/i2c-sis96x.ko 2023-11-26T07:02:44.659 INFO:teuthology.orchestra.run.smithi188.stdout:-rw-r--r-- root/root 19241 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/i2c/busses/i2c-taos-evm.ko 2023-11-26T07:02:44.659 INFO:teuthology.orchestra.run.smithi188.stdout:-rw-r--r-- root/root 19809 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/i2c/busses/i2c-tiny-usb.ko 2023-11-26T07:02:44.659 INFO:teuthology.orchestra.run.smithi188.stdout:-rw-r--r-- root/root 13801 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/i2c/busses/i2c-via.ko 2023-11-26T07:02:44.659 INFO:teuthology.orchestra.run.smithi188.stdout:-rw-r--r-- root/root 26081 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/i2c/busses/i2c-viapro.ko 2023-11-26T07:02:44.660 INFO:teuthology.orchestra.run.smithi188.stdout:-rw-r--r-- root/root 19041 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/i2c/busses/i2c-viperboard.ko 2023-11-26T07:02:44.660 INFO:teuthology.orchestra.run.smithi188.stdout:-rw-r--r-- root/root 47657 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/i2c/busses/i2c-xiic.ko 2023-11-26T07:02:44.660 INFO:teuthology.orchestra.run.smithi188.stdout:-rw-r--r-- root/root 22449 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/i2c/i2c-mux.ko 2023-11-26T07:02:44.660 INFO:teuthology.orchestra.run.smithi188.stdout:-rw-r--r-- root/root 18825 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/i2c/i2c-smbus.ko 2023-11-26T07:02:44.660 INFO:teuthology.orchestra.run.smithi188.stdout:-rw-r--r-- root/root 27073 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/i2c/i2c-stub.ko 2023-11-26T07:02:44.660 INFO:teuthology.orchestra.run.smithi188.stdout:drwxr-xr-x root/root 0 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/i2c/muxes/ 2023-11-26T07:02:44.660 INFO:teuthology.orchestra.run.smithi188.stdout:-rw-r--r-- root/root 14921 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/i2c/muxes/i2c-mux-gpio.ko 2023-11-26T07:02:44.660 INFO:teuthology.orchestra.run.smithi188.stdout:-rw-r--r-- root/root 13457 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/i2c/muxes/i2c-mux-pca9541.ko 2023-11-26T07:02:44.660 INFO:teuthology.orchestra.run.smithi188.stdout:-rw-r--r-- root/root 29737 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/i2c/muxes/i2c-mux-pca954x.ko 2023-11-26T07:02:44.660 INFO:teuthology.orchestra.run.smithi188.stdout:drwxr-xr-x root/root 0 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/infiniband/ 2023-11-26T07:02:44.660 INFO:teuthology.orchestra.run.smithi188.stdout:drwxr-xr-x root/root 0 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/infiniband/core/ 2023-11-26T07:02:44.661 INFO:teuthology.orchestra.run.smithi188.stdout:-rw-r--r-- root/root 273609 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/infiniband/core/ib_cm.ko 2023-11-26T07:02:44.661 INFO:teuthology.orchestra.run.smithi188.stdout:-rw-r--r-- root/root 879969 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/infiniband/core/ib_core.ko 2023-11-26T07:02:44.661 INFO:teuthology.orchestra.run.smithi188.stdout:-rw-r--r-- root/root 69273 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/infiniband/core/ib_umad.ko 2023-11-26T07:02:44.661 INFO:teuthology.orchestra.run.smithi188.stdout:-rw-r--r-- root/root 324217 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/infiniband/core/ib_uverbs.ko 2023-11-26T07:02:44.661 INFO:teuthology.orchestra.run.smithi188.stdout:-rw-r--r-- root/root 97985 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/infiniband/core/iw_cm.ko 2023-11-26T07:02:44.661 INFO:teuthology.orchestra.run.smithi188.stdout:-rw-r--r-- root/root 282273 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/infiniband/core/rdma_cm.ko 2023-11-26T07:02:44.661 INFO:teuthology.orchestra.run.smithi188.stdout:-rw-r--r-- root/root 66201 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/infiniband/core/rdma_ucm.ko 2023-11-26T07:02:44.661 INFO:teuthology.orchestra.run.smithi188.stdout:drwxr-xr-x root/root 0 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/infiniband/hw/ 2023-11-26T07:02:44.661 INFO:teuthology.orchestra.run.smithi188.stdout:drwxr-xr-x root/root 0 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/infiniband/hw/cxgb4/ 2023-11-26T07:02:44.661 INFO:teuthology.orchestra.run.smithi188.stdout:-rw-r--r-- root/root 493073 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/infiniband/hw/cxgb4/iw_cxgb4.ko 2023-11-26T07:02:44.661 INFO:teuthology.orchestra.run.smithi188.stdout:drwxr-xr-x root/root 0 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/infiniband/hw/mlx4/ 2023-11-26T07:02:44.662 INFO:teuthology.orchestra.run.smithi188.stdout:-rw-r--r-- root/root 419737 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/infiniband/hw/mlx4/mlx4_ib.ko 2023-11-26T07:02:44.662 INFO:teuthology.orchestra.run.smithi188.stdout:drwxr-xr-x root/root 0 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/infiniband/hw/mlx5/ 2023-11-26T07:02:44.662 INFO:teuthology.orchestra.run.smithi188.stdout:-rw-r--r-- root/root 739025 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/infiniband/hw/mlx5/mlx5_ib.ko 2023-11-26T07:02:44.662 INFO:teuthology.orchestra.run.smithi188.stdout:drwxr-xr-x root/root 0 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/infiniband/hw/mthca/ 2023-11-26T07:02:44.662 INFO:teuthology.orchestra.run.smithi188.stdout:-rw-r--r-- root/root 296625 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/infiniband/hw/mthca/ib_mthca.ko 2023-11-26T07:02:44.662 INFO:teuthology.orchestra.run.smithi188.stdout:drwxr-xr-x root/root 0 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/infiniband/hw/ocrdma/ 2023-11-26T07:02:44.662 INFO:teuthology.orchestra.run.smithi188.stdout:-rw-r--r-- root/root 153697 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/infiniband/hw/ocrdma/ocrdma.ko 2023-11-26T07:02:44.663 INFO:teuthology.orchestra.run.smithi188.stdout:drwxr-xr-x root/root 0 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/infiniband/hw/usnic/ 2023-11-26T07:02:44.663 INFO:teuthology.orchestra.run.smithi188.stdout:-rw-r--r-- root/root 168969 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/infiniband/hw/usnic/usnic_verbs.ko 2023-11-26T07:02:44.663 INFO:teuthology.orchestra.run.smithi188.stdout:drwxr-xr-x root/root 0 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/infiniband/ulp/ 2023-11-26T07:02:44.663 INFO:teuthology.orchestra.run.smithi188.stdout:drwxr-xr-x root/root 0 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/infiniband/ulp/ipoib/ 2023-11-26T07:02:44.663 INFO:teuthology.orchestra.run.smithi188.stdout:-rw-r--r-- root/root 263081 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/infiniband/ulp/ipoib/ib_ipoib.ko 2023-11-26T07:02:44.663 INFO:teuthology.orchestra.run.smithi188.stdout:drwxr-xr-x root/root 0 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/infiniband/ulp/iser/ 2023-11-26T07:02:44.663 INFO:teuthology.orchestra.run.smithi188.stdout:-rw-r--r-- root/root 100241 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/infiniband/ulp/iser/ib_iser.ko 2023-11-26T07:02:44.663 INFO:teuthology.orchestra.run.smithi188.stdout:drwxr-xr-x root/root 0 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/infiniband/ulp/isert/ 2023-11-26T07:02:44.663 INFO:teuthology.orchestra.run.smithi188.stdout:-rw-r--r-- root/root 111425 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/infiniband/ulp/isert/ib_isert.ko 2023-11-26T07:02:44.663 INFO:teuthology.orchestra.run.smithi188.stdout:drwxr-xr-x root/root 0 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/infiniband/ulp/srp/ 2023-11-26T07:02:44.663 INFO:teuthology.orchestra.run.smithi188.stdout:-rw-r--r-- root/root 128753 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/infiniband/ulp/srp/ib_srp.ko 2023-11-26T07:02:44.664 INFO:teuthology.orchestra.run.smithi188.stdout:drwxr-xr-x root/root 0 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/infiniband/ulp/srpt/ 2023-11-26T07:02:44.664 INFO:teuthology.orchestra.run.smithi188.stdout:-rw-r--r-- root/root 129529 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/infiniband/ulp/srpt/ib_srpt.ko 2023-11-26T07:02:44.664 INFO:teuthology.orchestra.run.smithi188.stdout:drwxr-xr-x root/root 0 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/input/ 2023-11-26T07:02:44.664 INFO:teuthology.orchestra.run.smithi188.stdout:-rw-r--r-- root/root 10745 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/input/evbug.ko 2023-11-26T07:02:44.664 INFO:teuthology.orchestra.run.smithi188.stdout:-rw-r--r-- root/root 24129 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/input/ff-memless.ko 2023-11-26T07:02:44.664 INFO:teuthology.orchestra.run.smithi188.stdout:drwxr-xr-x root/root 0 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/input/gameport/ 2023-11-26T07:02:44.664 INFO:teuthology.orchestra.run.smithi188.stdout:-rw-r--r-- root/root 11881 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/input/gameport/emu10k1-gp.ko 2023-11-26T07:02:44.664 INFO:teuthology.orchestra.run.smithi188.stdout:-rw-r--r-- root/root 12145 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/input/gameport/fm801-gp.ko 2023-11-26T07:02:44.664 INFO:teuthology.orchestra.run.smithi188.stdout:-rw-r--r-- root/root 41241 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/input/gameport/gameport.ko 2023-11-26T07:02:44.664 INFO:teuthology.orchestra.run.smithi188.stdout:-rw-r--r-- root/root 41801 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/input/joydev.ko 2023-11-26T07:02:44.664 INFO:teuthology.orchestra.run.smithi188.stdout:drwxr-xr-x root/root 0 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/input/keyboard/ 2023-11-26T07:02:44.665 INFO:teuthology.orchestra.run.smithi188.stdout:-rw-r--r-- root/root 13313 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/input/keyboard/adp5520-keys.ko 2023-11-26T07:02:44.665 INFO:teuthology.orchestra.run.smithi188.stdout:-rw-r--r-- root/root 12801 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/input/keyboard/adp5588-keys.ko 2023-11-26T07:02:44.665 INFO:teuthology.orchestra.run.smithi188.stdout:-rw-r--r-- root/root 26841 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/input/keyboard/adp5589-keys.ko 2023-11-26T07:02:44.665 INFO:teuthology.orchestra.run.smithi188.stdout:-rw-r--r-- root/root 24761 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/input/keyboard/cros_ec_keyb.ko 2023-11-26T07:02:44.665 INFO:teuthology.orchestra.run.smithi188.stdout:-rw-r--r-- root/root 36665 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/input/keyboard/gpio_keys.ko 2023-11-26T07:02:44.665 INFO:teuthology.orchestra.run.smithi188.stdout:-rw-r--r-- root/root 18625 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/input/keyboard/gpio_keys_polled.ko 2023-11-26T07:02:44.665 INFO:teuthology.orchestra.run.smithi188.stdout:-rw-r--r-- root/root 21257 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/input/keyboard/lkkbd.ko 2023-11-26T07:02:44.665 INFO:teuthology.orchestra.run.smithi188.stdout:-rw-r--r-- root/root 26473 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/input/keyboard/lm8323.ko 2023-11-26T07:02:44.665 INFO:teuthology.orchestra.run.smithi188.stdout:-rw-r--r-- root/root 14409 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/input/keyboard/lm8333.ko 2023-11-26T07:02:44.665 INFO:teuthology.orchestra.run.smithi188.stdout:-rw-r--r-- root/root 24265 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/input/keyboard/matrix_keypad.ko 2023-11-26T07:02:44.665 INFO:teuthology.orchestra.run.smithi188.stdout:-rw-r--r-- root/root 18217 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/input/keyboard/max7359_keypad.ko 2023-11-26T07:02:44.665 INFO:teuthology.orchestra.run.smithi188.stdout:-rw-r--r-- root/root 15929 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/input/keyboard/mcs_touchkey.ko 2023-11-26T07:02:44.666 INFO:teuthology.orchestra.run.smithi188.stdout:-rw-r--r-- root/root 16489 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/input/keyboard/mpr121_touchkey.ko 2023-11-26T07:02:44.666 INFO:teuthology.orchestra.run.smithi188.stdout:-rw-r--r-- root/root 11585 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/input/keyboard/newtonkbd.ko 2023-11-26T07:02:44.666 INFO:teuthology.orchestra.run.smithi188.stdout:-rw-r--r-- root/root 10673 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/input/keyboard/opencores-kbd.ko 2023-11-26T07:02:44.666 INFO:teuthology.orchestra.run.smithi188.stdout:-rw-r--r-- root/root 14161 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/input/keyboard/qt1070.ko 2023-11-26T07:02:44.666 INFO:teuthology.orchestra.run.smithi188.stdout:-rw-r--r-- root/root 17929 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/input/keyboard/qt2160.ko 2023-11-26T07:02:44.666 INFO:teuthology.orchestra.run.smithi188.stdout:-rw-r--r-- root/root 23817 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/input/keyboard/samsung-keypad.ko 2023-11-26T07:02:44.666 INFO:teuthology.orchestra.run.smithi188.stdout:-rw-r--r-- root/root 11481 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/input/keyboard/stowaway.ko 2023-11-26T07:02:44.666 INFO:teuthology.orchestra.run.smithi188.stdout:-rw-r--r-- root/root 18289 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/input/keyboard/sunkbd.ko 2023-11-26T07:02:44.666 INFO:teuthology.orchestra.run.smithi188.stdout:-rw-r--r-- root/root 17425 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/input/keyboard/tca6416-keypad.ko 2023-11-26T07:02:44.666 INFO:teuthology.orchestra.run.smithi188.stdout:-rw-r--r-- root/root 16177 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/input/keyboard/tca8418_keypad.ko 2023-11-26T07:02:44.667 INFO:teuthology.orchestra.run.smithi188.stdout:-rw-r--r-- root/root 16801 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/input/keyboard/twl4030_keypad.ko 2023-11-26T07:02:44.667 INFO:teuthology.orchestra.run.smithi188.stdout:-rw-r--r-- root/root 11753 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/input/keyboard/xtkbd.ko 2023-11-26T07:02:44.667 INFO:teuthology.orchestra.run.smithi188.stdout:-rw-r--r-- root/root 11465 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/input/matrix-keymap.ko 2023-11-26T07:02:44.667 INFO:teuthology.orchestra.run.smithi188.stdout:drwxr-xr-x root/root 0 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/input/mouse/ 2023-11-26T07:02:44.667 INFO:teuthology.orchestra.run.smithi188.stdout:-rw-r--r-- root/root 34329 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/input/mouse/appletouch.ko 2023-11-26T07:02:44.667 INFO:teuthology.orchestra.run.smithi188.stdout:-rw-r--r-- root/root 34177 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/input/mouse/bcm5974.ko 2023-11-26T07:02:44.667 INFO:teuthology.orchestra.run.smithi188.stdout:-rw-r--r-- root/root 123681 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/input/mouse/cyapatp.ko 2023-11-26T07:02:44.667 INFO:teuthology.orchestra.run.smithi188.stdout:-rw-r--r-- root/root 89449 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/input/mouse/elan_i2c.ko 2023-11-26T07:02:44.667 INFO:teuthology.orchestra.run.smithi188.stdout:-rw-r--r-- root/root 13961 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/input/mouse/gpio_mouse.ko 2023-11-26T07:02:44.667 INFO:teuthology.orchestra.run.smithi188.stdout:-rw-r--r-- root/root 335929 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/input/mouse/psmouse.ko 2023-11-26T07:02:44.667 INFO:teuthology.orchestra.run.smithi188.stdout:-rw-r--r-- root/root 14681 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/input/mouse/sermouse.ko 2023-11-26T07:02:44.667 INFO:teuthology.orchestra.run.smithi188.stdout:-rw-r--r-- root/root 23881 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/input/mouse/synaptics_i2c.ko 2023-11-26T07:02:44.668 INFO:teuthology.orchestra.run.smithi188.stdout:-rw-r--r-- root/root 22569 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/input/mouse/synaptics_usb.ko 2023-11-26T07:02:44.668 INFO:teuthology.orchestra.run.smithi188.stdout:-rw-r--r-- root/root 15665 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/input/mouse/vsxxxaa.ko 2023-11-26T07:02:44.668 INFO:teuthology.orchestra.run.smithi188.stdout:drwxr-xr-x root/root 0 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/input/rmi4/ 2023-11-26T07:02:44.668 INFO:teuthology.orchestra.run.smithi188.stdout:-rw-r--r-- root/root 109873 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/input/rmi4/rmi_core.ko 2023-11-26T07:02:44.668 INFO:teuthology.orchestra.run.smithi188.stdout:drwxr-xr-x root/root 0 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/input/serio/ 2023-11-26T07:02:44.668 INFO:teuthology.orchestra.run.smithi188.stdout:-rw-r--r-- root/root 11665 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/input/serio/altera_ps2.ko 2023-11-26T07:02:44.668 INFO:teuthology.orchestra.run.smithi188.stdout:-rw-r--r-- root/root 16289 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/input/serio/arc_ps2.ko 2023-11-26T07:02:44.668 INFO:teuthology.orchestra.run.smithi188.stdout:-rw-r--r-- root/root 16425 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/input/serio/ct82c710.ko 2023-11-26T07:02:44.668 INFO:teuthology.orchestra.run.smithi188.stdout:-rw-r--r-- root/root 16953 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/input/serio/hyperv-keyboard.ko 2023-11-26T07:02:44.668 INFO:teuthology.orchestra.run.smithi188.stdout:-rw-r--r-- root/root 15833 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/input/serio/parkbd.ko 2023-11-26T07:02:44.668 INFO:teuthology.orchestra.run.smithi188.stdout:-rw-r--r-- root/root 13225 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/input/serio/pcips2.ko 2023-11-26T07:02:44.668 INFO:teuthology.orchestra.run.smithi188.stdout:-rw-r--r-- root/root 20153 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/input/serio/ps2mult.ko 2023-11-26T07:02:44.669 INFO:teuthology.orchestra.run.smithi188.stdout:-rw-r--r-- root/root 24153 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/input/serio/serio_raw.ko 2023-11-26T07:02:44.669 INFO:teuthology.orchestra.run.smithi188.stdout:-rw-r--r-- root/root 17977 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/input/serio/serport.ko 2023-11-26T07:02:44.669 INFO:teuthology.orchestra.run.smithi188.stdout:-rw-r--r-- root/root 13337 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/input/sparse-keymap.ko 2023-11-26T07:02:44.669 INFO:teuthology.orchestra.run.smithi188.stdout:drwxr-xr-x root/root 0 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/iommu/ 2023-11-26T07:02:44.669 INFO:teuthology.orchestra.run.smithi188.stdout:drwxr-xr-x root/root 0 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/iommu/amd/ 2023-11-26T07:02:44.669 INFO:teuthology.orchestra.run.smithi188.stdout:-rw-r--r-- root/root 39289 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/iommu/amd/iommu_v2.ko 2023-11-26T07:02:44.669 INFO:teuthology.orchestra.run.smithi188.stdout:drwxr-xr-x root/root 0 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/leds/ 2023-11-26T07:02:44.669 INFO:teuthology.orchestra.run.smithi188.stdout:-rw-r--r-- root/root 13817 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/leds/leds-88pm860x.ko 2023-11-26T07:02:44.669 INFO:teuthology.orchestra.run.smithi188.stdout:-rw-r--r-- root/root 11385 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/leds/leds-adp5520.ko 2023-11-26T07:02:44.669 INFO:teuthology.orchestra.run.smithi188.stdout:-rw-r--r-- root/root 59753 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/leds/leds-bd2802.ko 2023-11-26T07:02:44.670 INFO:teuthology.orchestra.run.smithi188.stdout:-rw-r--r-- root/root 24785 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/leds/leds-blinkm.ko 2023-11-26T07:02:44.670 INFO:teuthology.orchestra.run.smithi188.stdout:-rw-r--r-- root/root 10473 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/leds/leds-da903x.ko 2023-11-26T07:02:44.670 INFO:teuthology.orchestra.run.smithi188.stdout:-rw-r--r-- root/root 12241 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/leds/leds-da9052.ko 2023-11-26T07:02:44.670 INFO:teuthology.orchestra.run.smithi188.stdout:-rw-r--r-- root/root 16561 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/leds/leds-gpio.ko 2023-11-26T07:02:44.670 INFO:teuthology.orchestra.run.smithi188.stdout:-rw-r--r-- root/root 15793 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/leds/leds-lm3530.ko 2023-11-26T07:02:44.670 INFO:teuthology.orchestra.run.smithi188.stdout:-rw-r--r-- root/root 30505 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/leds/leds-lm3533.ko 2023-11-26T07:02:44.670 INFO:teuthology.orchestra.run.smithi188.stdout:-rw-r--r-- root/root 19353 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/leds/leds-lm355x.ko 2023-11-26T07:02:44.670 INFO:teuthology.orchestra.run.smithi188.stdout:-rw-r--r-- root/root 19961 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/leds/leds-lm3642.ko 2023-11-26T07:02:44.670 INFO:teuthology.orchestra.run.smithi188.stdout:-rw-r--r-- root/root 16241 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/leds/leds-lp3944.ko 2023-11-26T07:02:44.670 INFO:teuthology.orchestra.run.smithi188.stdout:-rw-r--r-- root/root 11233 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/leds/leds-lp8788.ko 2023-11-26T07:02:44.671 INFO:teuthology.orchestra.run.smithi188.stdout:-rw-r--r-- root/root 14929 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/leds/leds-max8997.ko 2023-11-26T07:02:44.671 INFO:teuthology.orchestra.run.smithi188.stdout:-rw-r--r-- root/root 12545 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/leds/leds-mc13783.ko 2023-11-26T07:02:44.671 INFO:teuthology.orchestra.run.smithi188.stdout:-rw-r--r-- root/root 13385 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/leds/leds-menf21bmc.ko 2023-11-26T07:02:44.671 INFO:teuthology.orchestra.run.smithi188.stdout:-rw-r--r-- root/root 20721 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/leds/leds-pca9532.ko 2023-11-26T07:02:44.671 INFO:teuthology.orchestra.run.smithi188.stdout:-rw-r--r-- root/root 20481 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/leds/leds-pca955x.ko 2023-11-26T07:02:44.671 INFO:teuthology.orchestra.run.smithi188.stdout:-rw-r--r-- root/root 17857 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/leds/leds-pca963x.ko 2023-11-26T07:02:44.671 INFO:teuthology.orchestra.run.smithi188.stdout:-rw-r--r-- root/root 34433 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/leds/leds-ss4200.ko 2023-11-26T07:02:44.671 INFO:teuthology.orchestra.run.smithi188.stdout:-rw-r--r-- root/root 24865 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/leds/leds-tca6507.ko 2023-11-26T07:02:44.671 INFO:teuthology.orchestra.run.smithi188.stdout:-rw-r--r-- root/root 15665 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/leds/leds-wm831x-status.ko 2023-11-26T07:02:44.671 INFO:teuthology.orchestra.run.smithi188.stdout:-rw-r--r-- root/root 11841 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/leds/leds-wm8350.ko 2023-11-26T07:02:44.671 INFO:teuthology.orchestra.run.smithi188.stdout:drwxr-xr-x root/root 0 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/leds/trigger/ 2023-11-26T07:02:44.672 INFO:teuthology.orchestra.run.smithi188.stdout:-rw-r--r-- root/root 9985 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/leds/trigger/ledtrig-audio.ko 2023-11-26T07:02:44.672 INFO:teuthology.orchestra.run.smithi188.stdout:-rw-r--r-- root/root 10873 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/leds/trigger/ledtrig-backlight.ko 2023-11-26T07:02:44.672 INFO:teuthology.orchestra.run.smithi188.stdout:-rw-r--r-- root/root 9089 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/leds/trigger/ledtrig-camera.ko 2023-11-26T07:02:44.672 INFO:teuthology.orchestra.run.smithi188.stdout:-rw-r--r-- root/root 7617 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/leds/trigger/ledtrig-default-on.ko 2023-11-26T07:02:44.672 INFO:teuthology.orchestra.run.smithi188.stdout:-rw-r--r-- root/root 14249 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/leds/trigger/ledtrig-heartbeat.ko 2023-11-26T07:02:44.672 INFO:teuthology.orchestra.run.smithi188.stdout:-rw-r--r-- root/root 14073 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/leds/trigger/ledtrig-oneshot.ko 2023-11-26T07:02:44.672 INFO:teuthology.orchestra.run.smithi188.stdout:-rw-r--r-- root/root 11889 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/leds/trigger/ledtrig-timer.ko 2023-11-26T07:02:44.672 INFO:teuthology.orchestra.run.smithi188.stdout:-rw-r--r-- root/root 14073 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/leds/trigger/ledtrig-transient.ko 2023-11-26T07:02:44.672 INFO:teuthology.orchestra.run.smithi188.stdout:drwxr-xr-x root/root 0 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/md/ 2023-11-26T07:02:44.672 INFO:teuthology.orchestra.run.smithi188.stdout:drwxr-xr-x root/root 0 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/md/bcache/ 2023-11-26T07:02:44.672 INFO:teuthology.orchestra.run.smithi188.stdout:-rw-r--r-- root/root 654969 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/md/bcache/bcache.ko 2023-11-26T07:02:44.673 INFO:teuthology.orchestra.run.smithi188.stdout:-rw-r--r-- root/root 40577 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/md/dm-bio-prison.ko 2023-11-26T07:02:44.673 INFO:teuthology.orchestra.run.smithi188.stdout:-rw-r--r-- root/root 89601 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/md/dm-bufio.ko 2023-11-26T07:02:44.673 INFO:teuthology.orchestra.run.smithi188.stdout:-rw-r--r-- root/root 47529 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/md/dm-cache-smq.ko 2023-11-26T07:02:44.673 INFO:teuthology.orchestra.run.smithi188.stdout:-rw-r--r-- root/root 156009 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/md/dm-cache.ko 2023-11-26T07:02:44.673 INFO:teuthology.orchestra.run.smithi188.stdout:-rw-r--r-- root/root 106897 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/md/dm-crypt.ko 2023-11-26T07:02:44.673 INFO:teuthology.orchestra.run.smithi188.stdout:-rw-r--r-- root/root 21545 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/md/dm-delay.ko 2023-11-26T07:02:44.673 INFO:teuthology.orchestra.run.smithi188.stdout:-rw-r--r-- root/root 53857 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/md/dm-era.ko 2023-11-26T07:02:44.673 INFO:teuthology.orchestra.run.smithi188.stdout:-rw-r--r-- root/root 28465 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/md/dm-flakey.ko 2023-11-26T07:02:44.673 INFO:teuthology.orchestra.run.smithi188.stdout:-rw-r--r-- root/root 41529 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/md/dm-log-userspace.ko 2023-11-26T07:02:44.673 INFO:teuthology.orchestra.run.smithi188.stdout:-rw-r--r-- root/root 32737 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/md/dm-log.ko 2023-11-26T07:02:44.673 INFO:teuthology.orchestra.run.smithi188.stdout:-rw-r--r-- root/root 49545 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/md/dm-mirror.ko 2023-11-26T07:02:44.674 INFO:teuthology.orchestra.run.smithi188.stdout:-rw-r--r-- root/root 78033 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/md/dm-multipath.ko 2023-11-26T07:02:44.674 INFO:teuthology.orchestra.run.smithi188.stdout:-rw-r--r-- root/root 16473 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/md/dm-queue-length.ko 2023-11-26T07:02:44.674 INFO:teuthology.orchestra.run.smithi188.stdout:-rw-r--r-- root/root 70825 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/md/dm-raid.ko 2023-11-26T07:02:44.674 INFO:teuthology.orchestra.run.smithi188.stdout:-rw-r--r-- root/root 36297 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/md/dm-region-hash.ko 2023-11-26T07:02:44.674 INFO:teuthology.orchestra.run.smithi188.stdout:-rw-r--r-- root/root 15873 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/md/dm-round-robin.ko 2023-11-26T07:02:44.674 INFO:teuthology.orchestra.run.smithi188.stdout:-rw-r--r-- root/root 16801 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/md/dm-service-time.ko 2023-11-26T07:02:44.674 INFO:teuthology.orchestra.run.smithi188.stdout:-rw-r--r-- root/root 109529 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/md/dm-snapshot.ko 2023-11-26T07:02:44.674 INFO:teuthology.orchestra.run.smithi188.stdout:-rw-r--r-- root/root 19361 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/md/dm-switch.ko 2023-11-26T07:02:44.674 INFO:teuthology.orchestra.run.smithi188.stdout:-rw-r--r-- root/root 181865 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/md/dm-thin-pool.ko 2023-11-26T07:02:44.674 INFO:teuthology.orchestra.run.smithi188.stdout:-rw-r--r-- root/root 51737 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/md/dm-verity.ko 2023-11-26T07:02:44.674 INFO:teuthology.orchestra.run.smithi188.stdout:-rw-r--r-- root/root 8561 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/md/dm-zero.ko 2023-11-26T07:02:44.675 INFO:teuthology.orchestra.run.smithi188.stdout:-rw-r--r-- root/root 15345 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/md/faulty.ko 2023-11-26T07:02:44.675 INFO:teuthology.orchestra.run.smithi188.stdout:-rw-r--r-- root/root 19625 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/md/linear.ko 2023-11-26T07:02:44.675 INFO:teuthology.orchestra.run.smithi188.stdout:-rw-r--r-- root/root 32417 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/md/multipath.ko 2023-11-26T07:02:44.675 INFO:teuthology.orchestra.run.smithi188.stdout:drwxr-xr-x root/root 0 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/md/persistent-data/ 2023-11-26T07:02:44.675 INFO:teuthology.orchestra.run.smithi188.stdout:-rw-r--r-- root/root 199937 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/md/persistent-data/dm-persistent-data.ko 2023-11-26T07:02:44.675 INFO:teuthology.orchestra.run.smithi188.stdout:-rw-r--r-- root/root 37337 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/md/raid0.ko 2023-11-26T07:02:44.675 INFO:teuthology.orchestra.run.smithi188.stdout:-rw-r--r-- root/root 113073 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/md/raid1.ko 2023-11-26T07:02:44.675 INFO:teuthology.orchestra.run.smithi188.stdout:-rw-r--r-- root/root 158681 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/md/raid10.ko 2023-11-26T07:02:44.675 INFO:teuthology.orchestra.run.smithi188.stdout:-rw-r--r-- root/root 395721 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/md/raid456.ko 2023-11-26T07:02:44.675 INFO:teuthology.orchestra.run.smithi188.stdout:drwxr-xr-x root/root 0 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/message/ 2023-11-26T07:02:44.675 INFO:teuthology.orchestra.run.smithi188.stdout:drwxr-xr-x root/root 0 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/message/fusion/ 2023-11-26T07:02:44.676 INFO:teuthology.orchestra.run.smithi188.stdout:-rw-r--r-- root/root 178137 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/message/fusion/mptbase.ko 2023-11-26T07:02:44.676 INFO:teuthology.orchestra.run.smithi188.stdout:-rw-r--r-- root/root 63385 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/message/fusion/mptctl.ko 2023-11-26T07:02:44.676 INFO:teuthology.orchestra.run.smithi188.stdout:-rw-r--r-- root/root 44665 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/message/fusion/mptfc.ko 2023-11-26T07:02:44.676 INFO:teuthology.orchestra.run.smithi188.stdout:-rw-r--r-- root/root 37225 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/message/fusion/mptlan.ko 2023-11-26T07:02:44.676 INFO:teuthology.orchestra.run.smithi188.stdout:-rw-r--r-- root/root 123321 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/message/fusion/mptsas.ko 2023-11-26T07:02:44.676 INFO:teuthology.orchestra.run.smithi188.stdout:-rw-r--r-- root/root 79041 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/message/fusion/mptscsih.ko 2023-11-26T07:02:44.676 INFO:teuthology.orchestra.run.smithi188.stdout:-rw-r--r-- root/root 53017 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/message/fusion/mptspi.ko 2023-11-26T07:02:44.676 INFO:teuthology.orchestra.run.smithi188.stdout:drwxr-xr-x root/root 0 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/mfd/ 2023-11-26T07:02:44.676 INFO:teuthology.orchestra.run.smithi188.stdout:-rw-r--r-- root/root 17305 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/mfd/88pm800.ko 2023-11-26T07:02:44.676 INFO:teuthology.orchestra.run.smithi188.stdout:-rw-r--r-- root/root 12761 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/mfd/88pm805.ko 2023-11-26T07:02:44.676 INFO:teuthology.orchestra.run.smithi188.stdout:-rw-r--r-- root/root 11193 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/mfd/88pm80x.ko 2023-11-26T07:02:44.676 INFO:teuthology.orchestra.run.smithi188.stdout:-rw-r--r-- root/root 11865 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/mfd/arizona-i2c.ko 2023-11-26T07:02:44.677 INFO:teuthology.orchestra.run.smithi188.stdout:-rw-r--r-- root/root 123721 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/mfd/arizona.ko 2023-11-26T07:02:44.677 INFO:teuthology.orchestra.run.smithi188.stdout:-rw-r--r-- root/root 11961 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/mfd/bcm590xx.ko 2023-11-26T07:02:44.677 INFO:teuthology.orchestra.run.smithi188.stdout:-rw-r--r-- root/root 17465 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/mfd/cros_ec_dev.ko 2023-11-26T07:02:44.677 INFO:teuthology.orchestra.run.smithi188.stdout:-rw-r--r-- root/root 34961 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/mfd/dln2.ko 2023-11-26T07:02:44.677 INFO:teuthology.orchestra.run.smithi188.stdout:-rw-r--r-- root/root 16689 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/mfd/janz-cmodio.ko 2023-11-26T07:02:44.677 INFO:teuthology.orchestra.run.smithi188.stdout:-rw-r--r-- root/root 53585 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/mfd/kempld-core.ko 2023-11-26T07:02:44.677 INFO:teuthology.orchestra.run.smithi188.stdout:-rw-r--r-- root/root 27529 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/mfd/lm3533-core.ko 2023-11-26T07:02:44.677 INFO:teuthology.orchestra.run.smithi188.stdout:-rw-r--r-- root/root 14993 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/mfd/lm3533-ctrlbank.ko 2023-11-26T07:02:44.677 INFO:teuthology.orchestra.run.smithi188.stdout:-rw-r--r-- root/root 13009 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/mfd/lp3943.ko 2023-11-26T07:02:44.677 INFO:teuthology.orchestra.run.smithi188.stdout:-rw-r--r-- root/root 54569 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/mfd/lpc_ich.ko 2023-11-26T07:02:44.677 INFO:teuthology.orchestra.run.smithi188.stdout:-rw-r--r-- root/root 12113 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/mfd/lpc_sch.ko 2023-11-26T07:02:44.678 INFO:teuthology.orchestra.run.smithi188.stdout:-rw-r--r-- root/root 17409 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/mfd/max8907.ko 2023-11-26T07:02:44.678 INFO:teuthology.orchestra.run.smithi188.stdout:-rw-r--r-- root/root 29857 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/mfd/mc13xxx-core.ko 2023-11-26T07:02:44.678 INFO:teuthology.orchestra.run.smithi188.stdout:-rw-r--r-- root/root 11801 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/mfd/mc13xxx-i2c.ko 2023-11-26T07:02:44.678 INFO:teuthology.orchestra.run.smithi188.stdout:-rw-r--r-- root/root 11593 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/mfd/menf21bmc.ko 2023-11-26T07:02:44.678 INFO:teuthology.orchestra.run.smithi188.stdout:-rw-r--r-- root/root 17329 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/mfd/pcf50633-adc.ko 2023-11-26T07:02:44.678 INFO:teuthology.orchestra.run.smithi188.stdout:-rw-r--r-- root/root 9153 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/mfd/pcf50633-gpio.ko 2023-11-26T07:02:44.678 INFO:teuthology.orchestra.run.smithi188.stdout:-rw-r--r-- root/root 34873 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/mfd/pcf50633.ko 2023-11-26T07:02:44.678 INFO:teuthology.orchestra.run.smithi188.stdout:-rw-r--r-- root/root 10105 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/mfd/rdc321x-southbridge.ko 2023-11-26T07:02:44.678 INFO:teuthology.orchestra.run.smithi188.stdout:-rw-r--r-- root/root 20289 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/mfd/retu-mfd.ko 2023-11-26T07:02:44.678 INFO:teuthology.orchestra.run.smithi188.stdout:-rw-r--r-- root/root 80425 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/mfd/si476x-core.ko 2023-11-26T07:02:44.678 INFO:teuthology.orchestra.run.smithi188.stdout:-rw-r--r-- root/root 48457 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/mfd/sm501.ko 2023-11-26T07:02:44.679 INFO:teuthology.orchestra.run.smithi188.stdout:-rw-r--r-- root/root 19825 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/mfd/ti_am335x_tscadc.ko 2023-11-26T07:02:44.679 INFO:teuthology.orchestra.run.smithi188.stdout:-rw-r--r-- root/root 51137 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/mfd/tps65010.ko 2023-11-26T07:02:44.679 INFO:teuthology.orchestra.run.smithi188.stdout:-rw-r--r-- root/root 10353 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/mfd/tps6507x.ko 2023-11-26T07:02:44.679 INFO:teuthology.orchestra.run.smithi188.stdout:-rw-r--r-- root/root 13329 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/mfd/viperboard.ko 2023-11-26T07:02:44.679 INFO:teuthology.orchestra.run.smithi188.stdout:-rw-r--r-- root/root 10537 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/mfd/vx855.ko 2023-11-26T07:02:44.679 INFO:teuthology.orchestra.run.smithi188.stdout:-rw-r--r-- root/root 17953 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/mfd/wl1273-core.ko 2023-11-26T07:02:44.679 INFO:teuthology.orchestra.run.smithi188.stdout:-rw-r--r-- root/root 44345 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/mfd/wm8994.ko 2023-11-26T07:02:44.679 INFO:teuthology.orchestra.run.smithi188.stdout:drwxr-xr-x root/root 0 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/misc/ 2023-11-26T07:02:44.679 INFO:teuthology.orchestra.run.smithi188.stdout:-rw-r--r-- root/root 13833 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/misc/ad525x_dpot-i2c.ko 2023-11-26T07:02:44.679 INFO:teuthology.orchestra.run.smithi188.stdout:-rw-r--r-- root/root 48489 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/misc/ad525x_dpot.ko 2023-11-26T07:02:44.679 INFO:teuthology.orchestra.run.smithi188.stdout:drwxr-xr-x root/root 0 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/misc/altera-stapl/ 2023-11-26T07:02:44.679 INFO:teuthology.orchestra.run.smithi188.stdout:-rw-r--r-- root/root 46585 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/misc/altera-stapl/altera-stapl.ko 2023-11-26T07:02:44.680 INFO:teuthology.orchestra.run.smithi188.stdout:-rw-r--r-- root/root 18393 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/misc/apds9802als.ko 2023-11-26T07:02:44.680 INFO:teuthology.orchestra.run.smithi188.stdout:-rw-r--r-- root/root 36729 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/misc/apds990x.ko 2023-11-26T07:02:44.680 INFO:teuthology.orchestra.run.smithi188.stdout:-rw-r--r-- root/root 45593 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/misc/bh1770glc.ko 2023-11-26T07:02:44.680 INFO:teuthology.orchestra.run.smithi188.stdout:drwxr-xr-x root/root 0 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/misc/c2port/ 2023-11-26T07:02:44.680 INFO:teuthology.orchestra.run.smithi188.stdout:-rw-r--r-- root/root 11145 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/misc/c2port/c2port-duramar2150.ko 2023-11-26T07:02:44.680 INFO:teuthology.orchestra.run.smithi188.stdout:-rw-r--r-- root/root 35281 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/misc/c2port/core.ko 2023-11-26T07:02:44.680 INFO:teuthology.orchestra.run.smithi188.stdout:drwxr-xr-x root/root 0 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/misc/cb710/ 2023-11-26T07:02:44.680 INFO:teuthology.orchestra.run.smithi188.stdout:-rw-r--r-- root/root 25161 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/misc/cb710/cb710.ko 2023-11-26T07:02:44.680 INFO:teuthology.orchestra.run.smithi188.stdout:-rw-r--r-- root/root 17801 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/misc/ds1682.ko 2023-11-26T07:02:44.680 INFO:teuthology.orchestra.run.smithi188.stdout:-rw-r--r-- root/root 8897 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/misc/dummy-irq.ko 2023-11-26T07:02:44.680 INFO:teuthology.orchestra.run.smithi188.stdout:drwxr-xr-x root/root 0 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/misc/echo/ 2023-11-26T07:02:44.680 INFO:teuthology.orchestra.run.smithi188.stdout:-rw-r--r-- root/root 13657 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/misc/echo/echo.ko 2023-11-26T07:02:44.681 INFO:teuthology.orchestra.run.smithi188.stdout:drwxr-xr-x root/root 0 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/misc/eeprom/ 2023-11-26T07:02:44.681 INFO:teuthology.orchestra.run.smithi188.stdout:-rw-r--r-- root/root 37849 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/misc/eeprom/at24.ko 2023-11-26T07:02:44.681 INFO:teuthology.orchestra.run.smithi188.stdout:-rw-r--r-- root/root 14321 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/misc/eeprom/eeprom.ko 2023-11-26T07:02:44.681 INFO:teuthology.orchestra.run.smithi188.stdout:-rw-r--r-- root/root 14769 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/misc/eeprom/eeprom_93cx6.ko 2023-11-26T07:02:44.681 INFO:teuthology.orchestra.run.smithi188.stdout:-rw-r--r-- root/root 13537 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/misc/eeprom/max6875.ko 2023-11-26T07:02:44.681 INFO:teuthology.orchestra.run.smithi188.stdout:-rw-r--r-- root/root 33009 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/misc/enclosure.ko 2023-11-26T07:02:44.681 INFO:teuthology.orchestra.run.smithi188.stdout:drwxr-xr-x root/root 0 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/misc/genwqe/ 2023-11-26T07:02:44.681 INFO:teuthology.orchestra.run.smithi188.stdout:-rw-r--r-- root/root 152201 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/misc/genwqe/genwqe_card.ko 2023-11-26T07:02:44.681 INFO:teuthology.orchestra.run.smithi188.stdout:-rw-r--r-- root/root 14129 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/misc/hmc6352.ko 2023-11-26T07:02:44.681 INFO:teuthology.orchestra.run.smithi188.stdout:-rw-r--r-- root/root 31257 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/misc/hpilo.ko 2023-11-26T07:02:44.681 INFO:teuthology.orchestra.run.smithi188.stdout:drwxr-xr-x root/root 0 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/misc/ibmasm/ 2023-11-26T07:02:44.682 INFO:teuthology.orchestra.run.smithi188.stdout:-rw-r--r-- root/root 72129 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/misc/ibmasm/ibmasm.ko 2023-11-26T07:02:44.682 INFO:teuthology.orchestra.run.smithi188.stdout:-rw-r--r-- root/root 19785 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/misc/ics932s401.ko 2023-11-26T07:02:44.682 INFO:teuthology.orchestra.run.smithi188.stdout:-rw-r--r-- root/root 21681 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/misc/isl29003.ko 2023-11-26T07:02:44.682 INFO:teuthology.orchestra.run.smithi188.stdout:-rw-r--r-- root/root 15753 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/misc/isl29020.ko 2023-11-26T07:02:44.682 INFO:teuthology.orchestra.run.smithi188.stdout:drwxr-xr-x root/root 0 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/misc/lis3lv02d/ 2023-11-26T07:02:44.682 INFO:teuthology.orchestra.run.smithi188.stdout:-rw-r--r-- root/root 42513 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/misc/lis3lv02d/lis3lv02d.ko 2023-11-26T07:02:44.682 INFO:teuthology.orchestra.run.smithi188.stdout:-rw-r--r-- root/root 13857 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/misc/lis3lv02d/lis3lv02d_i2c.ko 2023-11-26T07:02:44.682 INFO:teuthology.orchestra.run.smithi188.stdout:drwxr-xr-x root/root 0 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/misc/mei/ 2023-11-26T07:02:44.682 INFO:teuthology.orchestra.run.smithi188.stdout:-rw-r--r-- root/root 125209 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/misc/mei/mei-me.ko 2023-11-26T07:02:44.682 INFO:teuthology.orchestra.run.smithi188.stdout:-rw-r--r-- root/root 56521 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/misc/mei/mei-txe.ko 2023-11-26T07:02:44.683 INFO:teuthology.orchestra.run.smithi188.stdout:-rw-r--r-- root/root 310113 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/misc/mei/mei.ko 2023-11-26T07:02:44.683 INFO:teuthology.orchestra.run.smithi188.stdout:-rw-r--r-- root/root 31329 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/misc/phantom.ko 2023-11-26T07:02:44.683 INFO:teuthology.orchestra.run.smithi188.stdout:drwxr-xr-x root/root 0 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/misc/ti-st/ 2023-11-26T07:02:44.683 INFO:teuthology.orchestra.run.smithi188.stdout:-rw-r--r-- root/root 74265 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/misc/ti-st/st_drv.ko 2023-11-26T07:02:44.683 INFO:teuthology.orchestra.run.smithi188.stdout:-rw-r--r-- root/root 24305 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/misc/tifm_7xx1.ko 2023-11-26T07:02:44.683 INFO:teuthology.orchestra.run.smithi188.stdout:-rw-r--r-- root/root 28785 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/misc/tifm_core.ko 2023-11-26T07:02:44.683 INFO:teuthology.orchestra.run.smithi188.stdout:-rw-r--r-- root/root 18729 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/misc/tsl2550.ko 2023-11-26T07:02:44.683 INFO:teuthology.orchestra.run.smithi188.stdout:-rw-r--r-- root/root 48697 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/misc/vmw_balloon.ko 2023-11-26T07:02:44.683 INFO:teuthology.orchestra.run.smithi188.stdout:drwxr-xr-x root/root 0 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/misc/vmw_vmci/ 2023-11-26T07:02:44.683 INFO:teuthology.orchestra.run.smithi188.stdout:-rw-r--r-- root/root 183609 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/misc/vmw_vmci/vmw_vmci.ko 2023-11-26T07:02:44.683 INFO:teuthology.orchestra.run.smithi188.stdout:drwxr-xr-x root/root 0 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ 2023-11-26T07:02:44.684 INFO:teuthology.orchestra.run.smithi188.stdout:drwxr-xr-x root/root 0 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/bonding/ 2023-11-26T07:02:44.684 INFO:teuthology.orchestra.run.smithi188.stdout:-rw-r--r-- root/root 490129 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/bonding/bonding.ko 2023-11-26T07:02:44.684 INFO:teuthology.orchestra.run.smithi188.stdout:drwxr-xr-x root/root 0 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/dsa/ 2023-11-26T07:02:44.684 INFO:teuthology.orchestra.run.smithi188.stdout:drwxr-xr-x root/root 0 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/dsa/b53/ 2023-11-26T07:02:44.684 INFO:teuthology.orchestra.run.smithi188.stdout:-rw-r--r-- root/root 106033 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/dsa/b53/b53_common.ko 2023-11-26T07:02:44.684 INFO:teuthology.orchestra.run.smithi188.stdout:-rw-r--r-- root/root 40417 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/dsa/bcm-sf2.ko 2023-11-26T07:02:44.684 INFO:teuthology.orchestra.run.smithi188.stdout:-rw-r--r-- root/root 16041 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/dsa/mv88e6060.ko 2023-11-26T07:02:44.684 INFO:teuthology.orchestra.run.smithi188.stdout:drwxr-xr-x root/root 0 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/dsa/mv88e6xxx/ 2023-11-26T07:02:44.684 INFO:teuthology.orchestra.run.smithi188.stdout:-rw-r--r-- root/root 453737 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/dsa/mv88e6xxx/mv88e6xxx.ko 2023-11-26T07:02:44.758 INFO:teuthology.orchestra.run.smithi188.stdout:-rw-r--r-- root/root 16929 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/dummy.ko 2023-11-26T07:02:44.758 INFO:teuthology.orchestra.run.smithi188.stdout:-rw-r--r-- root/root 17961 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/eql.ko 2023-11-26T07:02:44.758 INFO:teuthology.orchestra.run.smithi188.stdout:drwxr-xr-x root/root 0 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/ 2023-11-26T07:02:44.758 INFO:teuthology.orchestra.run.smithi188.stdout:drwxr-xr-x root/root 0 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/3com/ 2023-11-26T07:02:44.758 INFO:teuthology.orchestra.run.smithi188.stdout:-rw-r--r-- root/root 45785 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/3com/3c574_cs.ko 2023-11-26T07:02:44.758 INFO:teuthology.orchestra.run.smithi188.stdout:-rw-r--r-- root/root 42105 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/3com/3c589_cs.ko 2023-11-26T07:02:44.758 INFO:teuthology.orchestra.run.smithi188.stdout:-rw-r--r-- root/root 130553 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/3com/3c59x.ko 2023-11-26T07:02:44.758 INFO:teuthology.orchestra.run.smithi188.stdout:-rw-r--r-- root/root 68849 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/3com/typhoon.ko 2023-11-26T07:02:44.759 INFO:teuthology.orchestra.run.smithi188.stdout:drwxr-xr-x root/root 0 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/8390/ 2023-11-26T07:02:44.759 INFO:teuthology.orchestra.run.smithi188.stdout:-rw-r--r-- root/root 37649 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/8390/8390.ko 2023-11-26T07:02:44.759 INFO:teuthology.orchestra.run.smithi188.stdout:-rw-r--r-- root/root 59689 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/8390/axnet_cs.ko 2023-11-26T07:02:44.759 INFO:teuthology.orchestra.run.smithi188.stdout:-rw-r--r-- root/root 27201 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/8390/ne2k-pci.ko 2023-11-26T07:02:44.759 INFO:teuthology.orchestra.run.smithi188.stdout:-rw-r--r-- root/root 99049 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/8390/pcnet_cs.ko 2023-11-26T07:02:44.759 INFO:teuthology.orchestra.run.smithi188.stdout:drwxr-xr-x root/root 0 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/adaptec/ 2023-11-26T07:02:44.759 INFO:teuthology.orchestra.run.smithi188.stdout:-rw-r--r-- root/root 54521 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/adaptec/starfire.ko 2023-11-26T07:02:44.759 INFO:teuthology.orchestra.run.smithi188.stdout:drwxr-xr-x root/root 0 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/agere/ 2023-11-26T07:02:44.759 INFO:teuthology.orchestra.run.smithi188.stdout:-rw-r--r-- root/root 69137 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/agere/et131x.ko 2023-11-26T07:02:44.759 INFO:teuthology.orchestra.run.smithi188.stdout:drwxr-xr-x root/root 0 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/alteon/ 2023-11-26T07:02:44.759 INFO:teuthology.orchestra.run.smithi188.stdout:-rw-r--r-- root/root 59585 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/alteon/acenic.ko 2023-11-26T07:02:44.760 INFO:teuthology.orchestra.run.smithi188.stdout:drwxr-xr-x root/root 0 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/altera/ 2023-11-26T07:02:44.760 INFO:teuthology.orchestra.run.smithi188.stdout:-rw-r--r-- root/root 41409 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/altera/altera_tse.ko 2023-11-26T07:02:44.760 INFO:teuthology.orchestra.run.smithi188.stdout:drwxr-xr-x root/root 0 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/amd/ 2023-11-26T07:02:44.760 INFO:teuthology.orchestra.run.smithi188.stdout:-rw-r--r-- root/root 54609 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/amd/amd8111e.ko 2023-11-26T07:02:44.760 INFO:teuthology.orchestra.run.smithi188.stdout:-rw-r--r-- root/root 49289 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/amd/nmclan_cs.ko 2023-11-26T07:02:44.760 INFO:teuthology.orchestra.run.smithi188.stdout:-rw-r--r-- root/root 87297 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/amd/pcnet32.ko 2023-11-26T07:02:44.760 INFO:teuthology.orchestra.run.smithi188.stdout:drwxr-xr-x root/root 0 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/atheros/ 2023-11-26T07:02:44.760 INFO:teuthology.orchestra.run.smithi188.stdout:drwxr-xr-x root/root 0 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/atheros/alx/ 2023-11-26T07:02:44.760 INFO:teuthology.orchestra.run.smithi188.stdout:-rw-r--r-- root/root 97337 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/atheros/alx/alx.ko 2023-11-26T07:02:44.760 INFO:teuthology.orchestra.run.smithi188.stdout:drwxr-xr-x root/root 0 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/atheros/atl1c/ 2023-11-26T07:02:44.761 INFO:teuthology.orchestra.run.smithi188.stdout:-rw-r--r-- root/root 104673 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/atheros/atl1c/atl1c.ko 2023-11-26T07:02:44.761 INFO:teuthology.orchestra.run.smithi188.stdout:drwxr-xr-x root/root 0 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/atheros/atl1e/ 2023-11-26T07:02:44.761 INFO:teuthology.orchestra.run.smithi188.stdout:-rw-r--r-- root/root 90257 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/atheros/atl1e/atl1e.ko 2023-11-26T07:02:44.761 INFO:teuthology.orchestra.run.smithi188.stdout:drwxr-xr-x root/root 0 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/atheros/atlx/ 2023-11-26T07:02:44.761 INFO:teuthology.orchestra.run.smithi188.stdout:-rw-r--r-- root/root 81569 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/atheros/atlx/atl1.ko 2023-11-26T07:02:44.761 INFO:teuthology.orchestra.run.smithi188.stdout:-rw-r--r-- root/root 69729 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/atheros/atlx/atl2.ko 2023-11-26T07:02:44.761 INFO:teuthology.orchestra.run.smithi188.stdout:drwxr-xr-x root/root 0 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/broadcom/ 2023-11-26T07:02:44.761 INFO:teuthology.orchestra.run.smithi188.stdout:-rw-r--r-- root/root 78401 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/broadcom/b44.ko 2023-11-26T07:02:44.761 INFO:teuthology.orchestra.run.smithi188.stdout:-rw-r--r-- root/root 176761 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/broadcom/bnx2.ko 2023-11-26T07:02:44.761 INFO:teuthology.orchestra.run.smithi188.stdout:drwxr-xr-x root/root 0 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/broadcom/bnx2x/ 2023-11-26T07:02:44.761 INFO:teuthology.orchestra.run.smithi188.stdout:-rw-r--r-- root/root 1421497 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/broadcom/bnx2x/bnx2x.ko 2023-11-26T07:02:44.762 INFO:teuthology.orchestra.run.smithi188.stdout:-rw-r--r-- root/root 128569 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/broadcom/cnic.ko 2023-11-26T07:02:44.762 INFO:teuthology.orchestra.run.smithi188.stdout:drwxr-xr-x root/root 0 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/broadcom/genet/ 2023-11-26T07:02:44.762 INFO:teuthology.orchestra.run.smithi188.stdout:-rw-r--r-- root/root 132089 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/broadcom/genet/genet.ko 2023-11-26T07:02:44.762 INFO:teuthology.orchestra.run.smithi188.stdout:-rw-r--r-- root/root 312081 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/broadcom/tg3.ko 2023-11-26T07:02:44.762 INFO:teuthology.orchestra.run.smithi188.stdout:drwxr-xr-x root/root 0 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/brocade/ 2023-11-26T07:02:44.762 INFO:teuthology.orchestra.run.smithi188.stdout:drwxr-xr-x root/root 0 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/brocade/bna/ 2023-11-26T07:02:44.762 INFO:teuthology.orchestra.run.smithi188.stdout:-rw-r--r-- root/root 323513 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/brocade/bna/bna.ko 2023-11-26T07:02:44.762 INFO:teuthology.orchestra.run.smithi188.stdout:drwxr-xr-x root/root 0 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/chelsio/ 2023-11-26T07:02:44.762 INFO:teuthology.orchestra.run.smithi188.stdout:drwxr-xr-x root/root 0 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/chelsio/cxgb/ 2023-11-26T07:02:44.763 INFO:teuthology.orchestra.run.smithi188.stdout:-rw-r--r-- root/root 193497 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/chelsio/cxgb/cxgb.ko 2023-11-26T07:02:44.763 INFO:teuthology.orchestra.run.smithi188.stdout:drwxr-xr-x root/root 0 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/chelsio/cxgb3/ 2023-11-26T07:02:44.763 INFO:teuthology.orchestra.run.smithi188.stdout:-rw-r--r-- root/root 347417 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/chelsio/cxgb3/cxgb3.ko 2023-11-26T07:02:44.763 INFO:teuthology.orchestra.run.smithi188.stdout:drwxr-xr-x root/root 0 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/chelsio/cxgb4/ 2023-11-26T07:02:44.763 INFO:teuthology.orchestra.run.smithi188.stdout:-rw-r--r-- root/root 1044497 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/chelsio/cxgb4/cxgb4.ko 2023-11-26T07:02:44.763 INFO:teuthology.orchestra.run.smithi188.stdout:drwxr-xr-x root/root 0 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/chelsio/cxgb4vf/ 2023-11-26T07:02:44.763 INFO:teuthology.orchestra.run.smithi188.stdout:-rw-r--r-- root/root 166193 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/chelsio/cxgb4vf/cxgb4vf.ko 2023-11-26T07:02:44.763 INFO:teuthology.orchestra.run.smithi188.stdout:drwxr-xr-x root/root 0 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/chelsio/libcxgb/ 2023-11-26T07:02:44.763 INFO:teuthology.orchestra.run.smithi188.stdout:-rw-r--r-- root/root 34881 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/chelsio/libcxgb/libcxgb.ko 2023-11-26T07:02:44.763 INFO:teuthology.orchestra.run.smithi188.stdout:drwxr-xr-x root/root 0 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/cisco/ 2023-11-26T07:02:44.763 INFO:teuthology.orchestra.run.smithi188.stdout:drwxr-xr-x root/root 0 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/cisco/enic/ 2023-11-26T07:02:44.763 INFO:teuthology.orchestra.run.smithi188.stdout:-rw-r--r-- root/root 204345 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/cisco/enic/enic.ko 2023-11-26T07:02:44.764 INFO:teuthology.orchestra.run.smithi188.stdout:drwxr-xr-x root/root 0 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/dec/ 2023-11-26T07:02:44.764 INFO:teuthology.orchestra.run.smithi188.stdout:drwxr-xr-x root/root 0 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/dec/tulip/ 2023-11-26T07:02:44.764 INFO:teuthology.orchestra.run.smithi188.stdout:-rw-r--r-- root/root 69449 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/dec/tulip/de2104x.ko 2023-11-26T07:02:44.764 INFO:teuthology.orchestra.run.smithi188.stdout:-rw-r--r-- root/root 61489 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/dec/tulip/dmfe.ko 2023-11-26T07:02:44.764 INFO:teuthology.orchestra.run.smithi188.stdout:-rw-r--r-- root/root 141225 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/dec/tulip/tulip.ko 2023-11-26T07:02:44.764 INFO:teuthology.orchestra.run.smithi188.stdout:-rw-r--r-- root/root 51497 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/dec/tulip/uli526x.ko 2023-11-26T07:02:44.764 INFO:teuthology.orchestra.run.smithi188.stdout:-rw-r--r-- root/root 62817 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/dec/tulip/winbond-840.ko 2023-11-26T07:02:44.764 INFO:teuthology.orchestra.run.smithi188.stdout:-rw-r--r-- root/root 31369 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/dec/tulip/xircom_cb.ko 2023-11-26T07:02:44.764 INFO:teuthology.orchestra.run.smithi188.stdout:drwxr-xr-x root/root 0 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/dlink/ 2023-11-26T07:02:44.764 INFO:teuthology.orchestra.run.smithi188.stdout:-rw-r--r-- root/root 60873 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/dlink/dl2k.ko 2023-11-26T07:02:44.764 INFO:teuthology.orchestra.run.smithi188.stdout:-rw-r--r-- root/root 62833 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/dlink/sundance.ko 2023-11-26T07:02:44.765 INFO:teuthology.orchestra.run.smithi188.stdout:-rw-r--r-- root/root 45721 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/dnet.ko 2023-11-26T07:02:44.765 INFO:teuthology.orchestra.run.smithi188.stdout:-rw-r--r-- root/root 25185 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/ec_bhf.ko 2023-11-26T07:02:44.765 INFO:teuthology.orchestra.run.smithi188.stdout:drwxr-xr-x root/root 0 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/emulex/ 2023-11-26T07:02:44.765 INFO:teuthology.orchestra.run.smithi188.stdout:drwxr-xr-x root/root 0 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/emulex/benet/ 2023-11-26T07:02:44.765 INFO:teuthology.orchestra.run.smithi188.stdout:-rw-r--r-- root/root 281137 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/emulex/benet/be2net.ko 2023-11-26T07:02:44.765 INFO:teuthology.orchestra.run.smithi188.stdout:-rw-r--r-- root/root 49457 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/ethoc.ko 2023-11-26T07:02:44.765 INFO:teuthology.orchestra.run.smithi188.stdout:-rw-r--r-- root/root 51737 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/fealnx.ko 2023-11-26T07:02:44.765 INFO:teuthology.orchestra.run.smithi188.stdout:drwxr-xr-x root/root 0 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/fujitsu/ 2023-11-26T07:02:44.765 INFO:teuthology.orchestra.run.smithi188.stdout:-rw-r--r-- root/root 45345 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/fujitsu/fmvj18x_cs.ko 2023-11-26T07:02:44.765 INFO:teuthology.orchestra.run.smithi188.stdout:drwxr-xr-x root/root 0 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/intel/ 2023-11-26T07:02:44.765 INFO:teuthology.orchestra.run.smithi188.stdout:-rw-r--r-- root/root 90465 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/intel/e100.ko 2023-11-26T07:02:44.766 INFO:teuthology.orchestra.run.smithi188.stdout:drwxr-xr-x root/root 0 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/intel/e1000/ 2023-11-26T07:02:44.766 INFO:teuthology.orchestra.run.smithi188.stdout:-rw-r--r-- root/root 295577 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/intel/e1000/e1000.ko 2023-11-26T07:02:44.766 INFO:teuthology.orchestra.run.smithi188.stdout:drwxr-xr-x root/root 0 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/intel/e1000e/ 2023-11-26T07:02:44.766 INFO:teuthology.orchestra.run.smithi188.stdout:-rw-r--r-- root/root 584497 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/intel/e1000e/e1000e.ko 2023-11-26T07:02:44.766 INFO:teuthology.orchestra.run.smithi188.stdout:drwxr-xr-x root/root 0 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/intel/fm10k/ 2023-11-26T07:02:44.766 INFO:teuthology.orchestra.run.smithi188.stdout:-rw-r--r-- root/root 246209 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/intel/fm10k/fm10k.ko 2023-11-26T07:02:44.766 INFO:teuthology.orchestra.run.smithi188.stdout:drwxr-xr-x root/root 0 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/intel/i40e/ 2023-11-26T07:02:44.766 INFO:teuthology.orchestra.run.smithi188.stdout:-rw-r--r-- root/root 925825 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/intel/i40e/i40e.ko 2023-11-26T07:02:44.766 INFO:teuthology.orchestra.run.smithi188.stdout:drwxr-xr-x root/root 0 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/intel/iavf/ 2023-11-26T07:02:44.766 INFO:teuthology.orchestra.run.smithi188.stdout:-rw-r--r-- root/root 302937 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/intel/iavf/iavf.ko 2023-11-26T07:02:44.766 INFO:teuthology.orchestra.run.smithi188.stdout:drwxr-xr-x root/root 0 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/intel/igb/ 2023-11-26T07:02:44.767 INFO:teuthology.orchestra.run.smithi188.stdout:-rw-r--r-- root/root 514641 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/intel/igb/igb.ko 2023-11-26T07:02:44.767 INFO:teuthology.orchestra.run.smithi188.stdout:drwxr-xr-x root/root 0 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/intel/igbvf/ 2023-11-26T07:02:44.767 INFO:teuthology.orchestra.run.smithi188.stdout:-rw-r--r-- root/root 93593 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/intel/igbvf/igbvf.ko 2023-11-26T07:02:44.767 INFO:teuthology.orchestra.run.smithi188.stdout:drwxr-xr-x root/root 0 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/intel/ixgbe/ 2023-11-26T07:02:44.767 INFO:teuthology.orchestra.run.smithi188.stdout:-rw-r--r-- root/root 734305 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/intel/ixgbe/ixgbe.ko 2023-11-26T07:02:44.767 INFO:teuthology.orchestra.run.smithi188.stdout:drwxr-xr-x root/root 0 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/intel/ixgbevf/ 2023-11-26T07:02:44.767 INFO:teuthology.orchestra.run.smithi188.stdout:-rw-r--r-- root/root 151769 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/intel/ixgbevf/ixgbevf.ko 2023-11-26T07:02:44.767 INFO:teuthology.orchestra.run.smithi188.stdout:-rw-r--r-- root/root 86921 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/jme.ko 2023-11-26T07:02:44.767 INFO:teuthology.orchestra.run.smithi188.stdout:drwxr-xr-x root/root 0 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/marvell/ 2023-11-26T07:02:44.767 INFO:teuthology.orchestra.run.smithi188.stdout:-rw-r--r-- root/root 20977 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/marvell/mvmdio.ko 2023-11-26T07:02:44.767 INFO:teuthology.orchestra.run.smithi188.stdout:-rw-r--r-- root/root 102289 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/marvell/skge.ko 2023-11-26T07:02:44.767 INFO:teuthology.orchestra.run.smithi188.stdout:-rw-r--r-- root/root 122649 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/marvell/sky2.ko 2023-11-26T07:02:44.768 INFO:teuthology.orchestra.run.smithi188.stdout:drwxr-xr-x root/root 0 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/mellanox/ 2023-11-26T07:02:44.768 INFO:teuthology.orchestra.run.smithi188.stdout:drwxr-xr-x root/root 0 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/mellanox/mlx4/ 2023-11-26T07:02:44.768 INFO:teuthology.orchestra.run.smithi188.stdout:-rw-r--r-- root/root 727601 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/mellanox/mlx4/mlx4_core.ko 2023-11-26T07:02:44.768 INFO:teuthology.orchestra.run.smithi188.stdout:-rw-r--r-- root/root 278937 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/mellanox/mlx4/mlx4_en.ko 2023-11-26T07:02:44.768 INFO:teuthology.orchestra.run.smithi188.stdout:drwxr-xr-x root/root 0 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/mellanox/mlx5/ 2023-11-26T07:02:44.768 INFO:teuthology.orchestra.run.smithi188.stdout:drwxr-xr-x root/root 0 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/mellanox/mlx5/core/ 2023-11-26T07:02:44.768 INFO:teuthology.orchestra.run.smithi188.stdout:-rw-r--r-- root/root 1113681 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/mellanox/mlx5/core/mlx5_core.ko 2023-11-26T07:02:44.768 INFO:teuthology.orchestra.run.smithi188.stdout:drwxr-xr-x root/root 0 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/micrel/ 2023-11-26T07:02:44.768 INFO:teuthology.orchestra.run.smithi188.stdout:-rw-r--r-- root/root 53041 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/micrel/ks8842.ko 2023-11-26T07:02:44.768 INFO:teuthology.orchestra.run.smithi188.stdout:-rw-r--r-- root/root 40993 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/micrel/ks8851_common.ko 2023-11-26T07:02:44.768 INFO:teuthology.orchestra.run.smithi188.stdout:-rw-r--r-- root/root 19489 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/micrel/ks8851_par.ko 2023-11-26T07:02:44.768 INFO:teuthology.orchestra.run.smithi188.stdout:-rw-r--r-- root/root 95665 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/micrel/ksz884x.ko 2023-11-26T07:02:44.769 INFO:teuthology.orchestra.run.smithi188.stdout:drwxr-xr-x root/root 0 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/myricom/ 2023-11-26T07:02:44.769 INFO:teuthology.orchestra.run.smithi188.stdout:drwxr-xr-x root/root 0 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/myricom/myri10ge/ 2023-11-26T07:02:44.769 INFO:teuthology.orchestra.run.smithi188.stdout:-rw-r--r-- root/root 108649 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/myricom/myri10ge/myri10ge.ko 2023-11-26T07:02:44.769 INFO:teuthology.orchestra.run.smithi188.stdout:drwxr-xr-x root/root 0 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/natsemi/ 2023-11-26T07:02:44.769 INFO:teuthology.orchestra.run.smithi188.stdout:-rw-r--r-- root/root 73441 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/natsemi/natsemi.ko 2023-11-26T07:02:44.769 INFO:teuthology.orchestra.run.smithi188.stdout:-rw-r--r-- root/root 47777 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/natsemi/ns83820.ko 2023-11-26T07:02:44.769 INFO:teuthology.orchestra.run.smithi188.stdout:drwxr-xr-x root/root 0 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/neterion/ 2023-11-26T07:02:44.769 INFO:teuthology.orchestra.run.smithi188.stdout:-rw-r--r-- root/root 150905 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/neterion/s2io.ko 2023-11-26T07:02:44.769 INFO:teuthology.orchestra.run.smithi188.stdout:drwxr-xr-x root/root 0 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/nvidia/ 2023-11-26T07:02:44.769 INFO:teuthology.orchestra.run.smithi188.stdout:-rw-r--r-- root/root 136721 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/nvidia/forcedeth.ko 2023-11-26T07:02:44.770 INFO:teuthology.orchestra.run.smithi188.stdout:drwxr-xr-x root/root 0 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/packetengines/ 2023-11-26T07:02:44.770 INFO:teuthology.orchestra.run.smithi188.stdout:-rw-r--r-- root/root 50481 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/packetengines/hamachi.ko 2023-11-26T07:02:44.770 INFO:teuthology.orchestra.run.smithi188.stdout:-rw-r--r-- root/root 43145 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/packetengines/yellowfin.ko 2023-11-26T07:02:44.770 INFO:teuthology.orchestra.run.smithi188.stdout:drwxr-xr-x root/root 0 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/qlogic/ 2023-11-26T07:02:44.770 INFO:teuthology.orchestra.run.smithi188.stdout:drwxr-xr-x root/root 0 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/qlogic/netxen/ 2023-11-26T07:02:44.770 INFO:teuthology.orchestra.run.smithi188.stdout:-rw-r--r-- root/root 227897 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/qlogic/netxen/netxen_nic.ko 2023-11-26T07:02:44.770 INFO:teuthology.orchestra.run.smithi188.stdout:-rw-r--r-- root/root 87081 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/qlogic/qla3xxx.ko 2023-11-26T07:02:44.770 INFO:teuthology.orchestra.run.smithi188.stdout:drwxr-xr-x root/root 0 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/qlogic/qlcnic/ 2023-11-26T07:02:44.770 INFO:teuthology.orchestra.run.smithi188.stdout:-rw-r--r-- root/root 672049 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/qlogic/qlcnic/qlcnic.ko 2023-11-26T07:02:44.771 INFO:teuthology.orchestra.run.smithi188.stdout:drwxr-xr-x root/root 0 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/rdc/ 2023-11-26T07:02:44.771 INFO:teuthology.orchestra.run.smithi188.stdout:-rw-r--r-- root/root 43017 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/rdc/r6040.ko 2023-11-26T07:02:44.771 INFO:teuthology.orchestra.run.smithi188.stdout:drwxr-xr-x root/root 0 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/realtek/ 2023-11-26T07:02:44.771 INFO:teuthology.orchestra.run.smithi188.stdout:-rw-r--r-- root/root 67385 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/realtek/8139cp.ko 2023-11-26T07:02:44.771 INFO:teuthology.orchestra.run.smithi188.stdout:-rw-r--r-- root/root 83985 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/realtek/8139too.ko 2023-11-26T07:02:44.771 INFO:teuthology.orchestra.run.smithi188.stdout:-rw-r--r-- root/root 29321 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/realtek/atp.ko 2023-11-26T07:02:44.771 INFO:teuthology.orchestra.run.smithi188.stdout:-rw-r--r-- root/root 189801 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/realtek/r8169.ko 2023-11-26T07:02:44.771 INFO:teuthology.orchestra.run.smithi188.stdout:drwxr-xr-x root/root 0 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/samsung/ 2023-11-26T07:02:44.771 INFO:teuthology.orchestra.run.smithi188.stdout:drwxr-xr-x root/root 0 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/samsung/sxgbe/ 2023-11-26T07:02:44.771 INFO:teuthology.orchestra.run.smithi188.stdout:-rw-r--r-- root/root 104241 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/samsung/sxgbe/samsung-sxgbe.ko 2023-11-26T07:02:44.771 INFO:teuthology.orchestra.run.smithi188.stdout:drwxr-xr-x root/root 0 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/sfc/ 2023-11-26T07:02:44.772 INFO:teuthology.orchestra.run.smithi188.stdout:-rw-r--r-- root/root 1052497 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/sfc/sfc.ko 2023-11-26T07:02:44.772 INFO:teuthology.orchestra.run.smithi188.stdout:drwxr-xr-x root/root 0 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/silan/ 2023-11-26T07:02:44.772 INFO:teuthology.orchestra.run.smithi188.stdout:-rw-r--r-- root/root 47097 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/silan/sc92031.ko 2023-11-26T07:02:44.772 INFO:teuthology.orchestra.run.smithi188.stdout:drwxr-xr-x root/root 0 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/sis/ 2023-11-26T07:02:44.772 INFO:teuthology.orchestra.run.smithi188.stdout:-rw-r--r-- root/root 54817 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/sis/sis190.ko 2023-11-26T07:02:44.772 INFO:teuthology.orchestra.run.smithi188.stdout:-rw-r--r-- root/root 63337 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/sis/sis900.ko 2023-11-26T07:02:44.772 INFO:teuthology.orchestra.run.smithi188.stdout:drwxr-xr-x root/root 0 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/smsc/ 2023-11-26T07:02:44.772 INFO:teuthology.orchestra.run.smithi188.stdout:-rw-r--r-- root/root 61537 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/smsc/epic100.ko 2023-11-26T07:02:44.772 INFO:teuthology.orchestra.run.smithi188.stdout:-rw-r--r-- root/root 67193 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/smsc/smc91c92_cs.ko 2023-11-26T07:02:44.772 INFO:teuthology.orchestra.run.smithi188.stdout:-rw-r--r-- root/root 73409 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/smsc/smsc911x.ko 2023-11-26T07:02:44.772 INFO:teuthology.orchestra.run.smithi188.stdout:-rw-r--r-- root/root 68177 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/smsc/smsc9420.ko 2023-11-26T07:02:44.772 INFO:teuthology.orchestra.run.smithi188.stdout:drwxr-xr-x root/root 0 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/stmicro/ 2023-11-26T07:02:44.773 INFO:teuthology.orchestra.run.smithi188.stdout:drwxr-xr-x root/root 0 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/stmicro/stmmac/ 2023-11-26T07:02:44.773 INFO:teuthology.orchestra.run.smithi188.stdout:-rw-r--r-- root/root 13441 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/stmicro/stmmac/dwmac-generic.ko 2023-11-26T07:02:44.773 INFO:teuthology.orchestra.run.smithi188.stdout:-rw-r--r-- root/root 39905 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/stmicro/stmmac/dwmac-intel.ko 2023-11-26T07:02:44.773 INFO:teuthology.orchestra.run.smithi188.stdout:-rw-r--r-- root/root 20657 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/stmicro/stmmac/stmmac-platform.ko 2023-11-26T07:02:44.773 INFO:teuthology.orchestra.run.smithi188.stdout:-rw-r--r-- root/root 487025 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/stmicro/stmmac/stmmac.ko 2023-11-26T07:02:44.773 INFO:teuthology.orchestra.run.smithi188.stdout:drwxr-xr-x root/root 0 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/sun/ 2023-11-26T07:02:44.773 INFO:teuthology.orchestra.run.smithi188.stdout:-rw-r--r-- root/root 103337 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/sun/cassini.ko 2023-11-26T07:02:44.773 INFO:teuthology.orchestra.run.smithi188.stdout:-rw-r--r-- root/root 166097 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/sun/niu.ko 2023-11-26T07:02:44.773 INFO:teuthology.orchestra.run.smithi188.stdout:-rw-r--r-- root/root 66329 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/sun/sungem.ko 2023-11-26T07:02:44.773 INFO:teuthology.orchestra.run.smithi188.stdout:-rw-r--r-- root/root 78801 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/sun/sunhme.ko 2023-11-26T07:02:44.773 INFO:teuthology.orchestra.run.smithi188.stdout:drwxr-xr-x root/root 0 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/tehuti/ 2023-11-26T07:02:44.774 INFO:teuthology.orchestra.run.smithi188.stdout:-rw-r--r-- root/root 50057 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/tehuti/tehuti.ko 2023-11-26T07:02:44.774 INFO:teuthology.orchestra.run.smithi188.stdout:drwxr-xr-x root/root 0 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/ti/ 2023-11-26T07:02:44.774 INFO:teuthology.orchestra.run.smithi188.stdout:-rw-r--r-- root/root 86529 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/ti/tlan.ko 2023-11-26T07:02:44.774 INFO:teuthology.orchestra.run.smithi188.stdout:drwxr-xr-x root/root 0 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/via/ 2023-11-26T07:02:44.774 INFO:teuthology.orchestra.run.smithi188.stdout:-rw-r--r-- root/root 80273 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/via/via-rhine.ko 2023-11-26T07:02:44.774 INFO:teuthology.orchestra.run.smithi188.stdout:-rw-r--r-- root/root 83633 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/via/via-velocity.ko 2023-11-26T07:02:44.774 INFO:teuthology.orchestra.run.smithi188.stdout:drwxr-xr-x root/root 0 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/wiznet/ 2023-11-26T07:02:44.774 INFO:teuthology.orchestra.run.smithi188.stdout:-rw-r--r-- root/root 46241 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/wiznet/w5100.ko 2023-11-26T07:02:44.774 INFO:teuthology.orchestra.run.smithi188.stdout:-rw-r--r-- root/root 30849 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/wiznet/w5300.ko 2023-11-26T07:02:44.775 INFO:teuthology.orchestra.run.smithi188.stdout:drwxr-xr-x root/root 0 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/xircom/ 2023-11-26T07:02:44.775 INFO:teuthology.orchestra.run.smithi188.stdout:-rw-r--r-- root/root 59993 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/xircom/xirc2ps_cs.ko 2023-11-26T07:02:44.775 INFO:teuthology.orchestra.run.smithi188.stdout:drwxr-xr-x root/root 0 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/fddi/ 2023-11-26T07:02:44.775 INFO:teuthology.orchestra.run.smithi188.stdout:-rw-r--r-- root/root 41001 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/fddi/defxx.ko 2023-11-26T07:02:44.775 INFO:teuthology.orchestra.run.smithi188.stdout:drwxr-xr-x root/root 0 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/fddi/skfp/ 2023-11-26T07:02:44.775 INFO:teuthology.orchestra.run.smithi188.stdout:-rw-r--r-- root/root 216985 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/fddi/skfp/skfp.ko 2023-11-26T07:02:44.775 INFO:teuthology.orchestra.run.smithi188.stdout:-rw-r--r-- root/root 77345 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/geneve.ko 2023-11-26T07:02:44.775 INFO:teuthology.orchestra.run.smithi188.stdout:drwxr-xr-x root/root 0 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/hyperv/ 2023-11-26T07:02:44.775 INFO:teuthology.orchestra.run.smithi188.stdout:-rw-r--r-- root/root 235521 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/hyperv/hv_netvsc.ko 2023-11-26T07:02:44.775 INFO:teuthology.orchestra.run.smithi188.stdout:drwxr-xr-x root/root 0 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ieee802154/ 2023-11-26T07:02:44.775 INFO:teuthology.orchestra.run.smithi188.stdout:-rw-r--r-- root/root 18273 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ieee802154/fakelb.ko 2023-11-26T07:02:44.776 INFO:teuthology.orchestra.run.smithi188.stdout:-rw-r--r-- root/root 25065 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ifb.ko 2023-11-26T07:02:44.776 INFO:teuthology.orchestra.run.smithi188.stdout:drwxr-xr-x root/root 0 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ipvlan/ 2023-11-26T07:02:44.776 INFO:teuthology.orchestra.run.smithi188.stdout:-rw-r--r-- root/root 83769 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ipvlan/ipvlan.ko 2023-11-26T07:02:44.776 INFO:teuthology.orchestra.run.smithi188.stdout:-rw-r--r-- root/root 62145 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/macvlan.ko 2023-11-26T07:02:44.776 INFO:teuthology.orchestra.run.smithi188.stdout:-rw-r--r-- root/root 15865 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/macvtap.ko 2023-11-26T07:02:44.776 INFO:teuthology.orchestra.run.smithi188.stdout:drwxr-xr-x root/root 0 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/mdio/ 2023-11-26T07:02:44.776 INFO:teuthology.orchestra.run.smithi188.stdout:-rw-r--r-- root/root 20569 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/mdio/mdio-bcm-unimac.ko 2023-11-26T07:02:44.776 INFO:teuthology.orchestra.run.smithi188.stdout:-rw-r--r-- root/root 14185 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/mdio/mdio-bitbang.ko 2023-11-26T07:02:44.776 INFO:teuthology.orchestra.run.smithi188.stdout:-rw-r--r-- root/root 13681 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/mdio/mdio-gpio.ko 2023-11-26T07:02:44.776 INFO:teuthology.orchestra.run.smithi188.stdout:-rw-r--r-- root/root 8825 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/mdio/mdio-regmap.ko 2023-11-26T07:02:44.776 INFO:teuthology.orchestra.run.smithi188.stdout:-rw-r--r-- root/root 16025 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/mdio.ko 2023-11-26T07:02:44.776 INFO:teuthology.orchestra.run.smithi188.stdout:-rw-r--r-- root/root 20513 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/mii.ko 2023-11-26T07:02:44.777 INFO:teuthology.orchestra.run.smithi188.stdout:-rw-r--r-- root/root 43953 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/netconsole.ko 2023-11-26T07:02:44.777 INFO:teuthology.orchestra.run.smithi188.stdout:-rw-r--r-- root/root 13145 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/nlmon.ko 2023-11-26T07:02:44.777 INFO:teuthology.orchestra.run.smithi188.stdout:drwxr-xr-x root/root 0 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/pcs/ 2023-11-26T07:02:44.777 INFO:teuthology.orchestra.run.smithi188.stdout:-rw-r--r-- root/root 17985 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/pcs/pcs-lynx.ko 2023-11-26T07:02:44.777 INFO:teuthology.orchestra.run.smithi188.stdout:-rw-r--r-- root/root 38977 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/pcs/pcs_xpcs.ko 2023-11-26T07:02:44.777 INFO:teuthology.orchestra.run.smithi188.stdout:drwxr-xr-x root/root 0 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/phy/ 2023-11-26T07:02:44.777 INFO:teuthology.orchestra.run.smithi188.stdout:-rw-r--r-- root/root 9737 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/phy/amd.ko 2023-11-26T07:02:44.777 INFO:teuthology.orchestra.run.smithi188.stdout:-rw-r--r-- root/root 50033 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/phy/bcm-phy-lib.ko 2023-11-26T07:02:44.777 INFO:teuthology.orchestra.run.smithi188.stdout:-rw-r--r-- root/root 44721 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/phy/bcm7xxx.ko 2023-11-26T07:02:44.777 INFO:teuthology.orchestra.run.smithi188.stdout:-rw-r--r-- root/root 11521 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/phy/bcm87xx.ko 2023-11-26T07:02:44.777 INFO:teuthology.orchestra.run.smithi188.stdout:-rw-r--r-- root/root 42241 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/phy/broadcom.ko 2023-11-26T07:02:44.778 INFO:teuthology.orchestra.run.smithi188.stdout:-rw-r--r-- root/root 10625 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/phy/cicada.ko 2023-11-26T07:02:44.778 INFO:teuthology.orchestra.run.smithi188.stdout:-rw-r--r-- root/root 13025 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/phy/davicom.ko 2023-11-26T07:02:44.778 INFO:teuthology.orchestra.run.smithi188.stdout:-rw-r--r-- root/root 9449 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/phy/et1011c.ko 2023-11-26T07:02:44.778 INFO:teuthology.orchestra.run.smithi188.stdout:-rw-r--r-- root/root 24969 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/phy/icplus.ko 2023-11-26T07:02:44.778 INFO:teuthology.orchestra.run.smithi188.stdout:-rw-r--r-- root/root 15849 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/phy/lxt.ko 2023-11-26T07:02:44.778 INFO:teuthology.orchestra.run.smithi188.stdout:-rw-r--r-- root/root 82049 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/phy/marvell.ko 2023-11-26T07:02:44.778 INFO:teuthology.orchestra.run.smithi188.stdout:-rw-r--r-- root/root 76113 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/phy/micrel.ko 2023-11-26T07:02:44.778 INFO:teuthology.orchestra.run.smithi188.stdout:-rw-r--r-- root/root 12033 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/phy/national.ko 2023-11-26T07:02:44.778 INFO:teuthology.orchestra.run.smithi188.stdout:-rw-r--r-- root/root 97857 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/phy/phylink.ko 2023-11-26T07:02:44.778 INFO:teuthology.orchestra.run.smithi188.stdout:-rw-r--r-- root/root 10065 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/phy/qsemi.ko 2023-11-26T07:02:44.778 INFO:teuthology.orchestra.run.smithi188.stdout:-rw-r--r-- root/root 50065 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/phy/realtek.ko 2023-11-26T07:02:44.779 INFO:teuthology.orchestra.run.smithi188.stdout:-rw-r--r-- root/root 29985 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/phy/smsc.ko 2023-11-26T07:02:44.779 INFO:teuthology.orchestra.run.smithi188.stdout:-rw-r--r-- root/root 10857 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/phy/ste10Xp.ko 2023-11-26T07:02:44.779 INFO:teuthology.orchestra.run.smithi188.stdout:-rw-r--r-- root/root 24289 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/phy/vitesse.ko 2023-11-26T07:02:44.779 INFO:teuthology.orchestra.run.smithi188.stdout:drwxr-xr-x root/root 0 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/plip/ 2023-11-26T07:02:44.779 INFO:teuthology.orchestra.run.smithi188.stdout:-rw-r--r-- root/root 43505 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/plip/plip.ko 2023-11-26T07:02:44.779 INFO:teuthology.orchestra.run.smithi188.stdout:drwxr-xr-x root/root 0 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ppp/ 2023-11-26T07:02:44.779 INFO:teuthology.orchestra.run.smithi188.stdout:-rw-r--r-- root/root 15857 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ppp/bsd_comp.ko 2023-11-26T07:02:44.779 INFO:teuthology.orchestra.run.smithi188.stdout:-rw-r--r-- root/root 29713 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ppp/ppp_async.ko 2023-11-26T07:02:44.779 INFO:teuthology.orchestra.run.smithi188.stdout:-rw-r--r-- root/root 17665 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ppp/ppp_deflate.ko 2023-11-26T07:02:44.779 INFO:teuthology.orchestra.run.smithi188.stdout:-rw-r--r-- root/root 18625 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ppp/ppp_mppe.ko 2023-11-26T07:02:44.779 INFO:teuthology.orchestra.run.smithi188.stdout:-rw-r--r-- root/root 28729 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ppp/ppp_synctty.ko 2023-11-26T07:02:44.780 INFO:teuthology.orchestra.run.smithi188.stdout:-rw-r--r-- root/root 42545 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ppp/pppoe.ko 2023-11-26T07:02:44.780 INFO:teuthology.orchestra.run.smithi188.stdout:-rw-r--r-- root/root 13441 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ppp/pppox.ko 2023-11-26T07:02:44.780 INFO:teuthology.orchestra.run.smithi188.stdout:-rw-r--r-- root/root 32689 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ppp/pptp.ko 2023-11-26T07:02:44.780 INFO:teuthology.orchestra.run.smithi188.stdout:-rw-r--r-- root/root 28289 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/sb1000.ko 2023-11-26T07:02:44.780 INFO:teuthology.orchestra.run.smithi188.stdout:drwxr-xr-x root/root 0 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/slip/ 2023-11-26T07:02:44.780 INFO:teuthology.orchestra.run.smithi188.stdout:-rw-r--r-- root/root 42465 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/slip/slip.ko 2023-11-26T07:02:44.780 INFO:teuthology.orchestra.run.smithi188.stdout:-rw-r--r-- root/root 26929 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/sungem_phy.ko 2023-11-26T07:02:44.780 INFO:teuthology.orchestra.run.smithi188.stdout:-rw-r--r-- root/root 64449 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/tap.ko 2023-11-26T07:02:44.780 INFO:teuthology.orchestra.run.smithi188.stdout:drwxr-xr-x root/root 0 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/team/ 2023-11-26T07:02:44.780 INFO:teuthology.orchestra.run.smithi188.stdout:-rw-r--r-- root/root 106169 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/team/team.ko 2023-11-26T07:02:44.781 INFO:teuthology.orchestra.run.smithi188.stdout:-rw-r--r-- root/root 13497 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/team/team_mode_activebackup.ko 2023-11-26T07:02:44.781 INFO:teuthology.orchestra.run.smithi188.stdout:-rw-r--r-- root/root 10209 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/team/team_mode_broadcast.ko 2023-11-26T07:02:44.781 INFO:teuthology.orchestra.run.smithi188.stdout:-rw-r--r-- root/root 28881 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/team/team_mode_loadbalance.ko 2023-11-26T07:02:44.781 INFO:teuthology.orchestra.run.smithi188.stdout:-rw-r--r-- root/root 9641 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/team/team_mode_random.ko 2023-11-26T07:02:44.781 INFO:teuthology.orchestra.run.smithi188.stdout:-rw-r--r-- root/root 9321 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/team/team_mode_roundrobin.ko 2023-11-26T07:02:44.781 INFO:teuthology.orchestra.run.smithi188.stdout:-rw-r--r-- root/root 87017 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/veth.ko 2023-11-26T07:02:44.781 INFO:teuthology.orchestra.run.smithi188.stdout:drwxr-xr-x root/root 0 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/vmxnet3/ 2023-11-26T07:02:44.781 INFO:teuthology.orchestra.run.smithi188.stdout:-rw-r--r-- root/root 144297 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/vmxnet3/vmxnet3.ko 2023-11-26T07:02:44.781 INFO:teuthology.orchestra.run.smithi188.stdout:drwxr-xr-x root/root 0 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/vxlan/ 2023-11-26T07:02:44.781 INFO:teuthology.orchestra.run.smithi188.stdout:-rw-r--r-- root/root 297329 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/vxlan/vxlan.ko 2023-11-26T07:02:44.781 INFO:teuthology.orchestra.run.smithi188.stdout:drwxr-xr-x root/root 0 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/xen-netback/ 2023-11-26T07:02:44.782 INFO:teuthology.orchestra.run.smithi188.stdout:-rw-r--r-- root/root 144585 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/xen-netback/xen-netback.ko 2023-11-26T07:02:44.782 INFO:teuthology.orchestra.run.smithi188.stdout:drwxr-xr-x root/root 0 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/ntb/ 2023-11-26T07:02:44.782 INFO:teuthology.orchestra.run.smithi188.stdout:-rw-r--r-- root/root 20969 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/ntb/ntb.ko 2023-11-26T07:02:44.782 INFO:teuthology.orchestra.run.smithi188.stdout:drwxr-xr-x root/root 0 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/nvme/ 2023-11-26T07:02:44.782 INFO:teuthology.orchestra.run.smithi188.stdout:drwxr-xr-x root/root 0 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/nvme/host/ 2023-11-26T07:02:44.782 INFO:teuthology.orchestra.run.smithi188.stdout:-rw-r--r-- root/root 265121 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/nvme/host/nvme-core.ko 2023-11-26T07:02:44.782 INFO:teuthology.orchestra.run.smithi188.stdout:-rw-r--r-- root/root 102209 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/nvme/host/nvme.ko 2023-11-26T07:02:44.782 INFO:teuthology.orchestra.run.smithi188.stdout:drwxr-xr-x root/root 0 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/parport/ 2023-11-26T07:02:44.782 INFO:teuthology.orchestra.run.smithi188.stdout:-rw-r--r-- root/root 133841 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/parport/parport.ko 2023-11-26T07:02:44.782 INFO:teuthology.orchestra.run.smithi188.stdout:-rw-r--r-- root/root 14561 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/parport/parport_cs.ko 2023-11-26T07:02:44.783 INFO:teuthology.orchestra.run.smithi188.stdout:-rw-r--r-- root/root 84465 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/parport/parport_pc.ko 2023-11-26T07:02:44.783 INFO:teuthology.orchestra.run.smithi188.stdout:-rw-r--r-- root/root 26121 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/parport/parport_serial.ko 2023-11-26T07:02:44.783 INFO:teuthology.orchestra.run.smithi188.stdout:drwxr-xr-x root/root 0 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/pci/ 2023-11-26T07:02:44.783 INFO:teuthology.orchestra.run.smithi188.stdout:drwxr-xr-x root/root 0 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/pci/hotplug/ 2023-11-26T07:02:44.783 INFO:teuthology.orchestra.run.smithi188.stdout:-rw-r--r-- root/root 25009 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/pci/hotplug/acpiphp_ibm.ko 2023-11-26T07:02:44.783 INFO:teuthology.orchestra.run.smithi188.stdout:-rw-r--r-- root/root 17849 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/pci/hotplug/cpcihp_generic.ko 2023-11-26T07:02:44.783 INFO:teuthology.orchestra.run.smithi188.stdout:-rw-r--r-- root/root 20137 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/pci/hotplug/cpcihp_zt5550.ko 2023-11-26T07:02:44.783 INFO:teuthology.orchestra.run.smithi188.stdout:-rw-r--r-- root/root 11857 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/pci/pci-stub.ko 2023-11-26T07:02:44.783 INFO:teuthology.orchestra.run.smithi188.stdout:-rw-r--r-- root/root 46465 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/pci/xen-pcifront.ko 2023-11-26T07:02:44.784 INFO:teuthology.orchestra.run.smithi188.stdout:drwxr-xr-x root/root 0 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/pcmcia/ 2023-11-26T07:02:44.784 INFO:teuthology.orchestra.run.smithi188.stdout:-rw-r--r-- root/root 26793 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/pcmcia/i82092.ko 2023-11-26T07:02:44.784 INFO:teuthology.orchestra.run.smithi188.stdout:-rw-r--r-- root/root 145809 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/pcmcia/pcmcia.ko 2023-11-26T07:02:44.784 INFO:teuthology.orchestra.run.smithi188.stdout:-rw-r--r-- root/root 62137 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/pcmcia/pcmcia_core.ko 2023-11-26T07:02:44.784 INFO:teuthology.orchestra.run.smithi188.stdout:-rw-r--r-- root/root 34345 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/pcmcia/pcmcia_rsrc.ko 2023-11-26T07:02:44.784 INFO:teuthology.orchestra.run.smithi188.stdout:-rw-r--r-- root/root 34265 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/pcmcia/pd6729.ko 2023-11-26T07:02:44.784 INFO:teuthology.orchestra.run.smithi188.stdout:-rw-r--r-- root/root 112889 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/pcmcia/yenta_socket.ko 2023-11-26T07:02:44.784 INFO:teuthology.orchestra.run.smithi188.stdout:drwxr-xr-x root/root 0 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/phy/ 2023-11-26T07:02:44.784 INFO:teuthology.orchestra.run.smithi188.stdout:drwxr-xr-x root/root 0 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/phy/broadcom/ 2023-11-26T07:02:44.784 INFO:teuthology.orchestra.run.smithi188.stdout:-rw-r--r-- root/root 10841 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/phy/broadcom/phy-bcm-kona-usb2.ko 2023-11-26T07:02:44.785 INFO:teuthology.orchestra.run.smithi188.stdout:drwxr-xr-x root/root 0 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/phy/samsung/ 2023-11-26T07:02:44.785 INFO:teuthology.orchestra.run.smithi188.stdout:-rw-r--r-- root/root 8441 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/phy/samsung/phy-exynos-usb2.ko 2023-11-26T07:02:44.785 INFO:teuthology.orchestra.run.smithi188.stdout:drwxr-xr-x root/root 0 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/pinctrl/ 2023-11-26T07:02:44.785 INFO:teuthology.orchestra.run.smithi188.stdout:drwxr-xr-x root/root 0 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/pinctrl/intel/ 2023-11-26T07:02:44.785 INFO:teuthology.orchestra.run.smithi188.stdout:-rw-r--r-- root/root 70153 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/pinctrl/intel/pinctrl-cherryview.ko 2023-11-26T07:02:44.785 INFO:teuthology.orchestra.run.smithi188.stdout:drwxr-xr-x root/root 0 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/platform/ 2023-11-26T07:02:44.785 INFO:teuthology.orchestra.run.smithi188.stdout:drwxr-xr-x root/root 0 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/platform/chrome/ 2023-11-26T07:02:44.785 INFO:teuthology.orchestra.run.smithi188.stdout:-rw-r--r-- root/root 50889 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/platform/chrome/cros-ec-sensorhub.ko 2023-11-26T07:02:44.785 INFO:teuthology.orchestra.run.smithi188.stdout:-rw-r--r-- root/root 20849 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/platform/chrome/cros_ec.ko 2023-11-26T07:02:44.785 INFO:teuthology.orchestra.run.smithi188.stdout:-rw-r--r-- root/root 21449 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/platform/chrome/cros_ec_chardev.ko 2023-11-26T07:02:44.786 INFO:teuthology.orchestra.run.smithi188.stdout:-rw-r--r-- root/root 25417 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/platform/chrome/cros_ec_debugfs.ko 2023-11-26T07:02:44.786 INFO:teuthology.orchestra.run.smithi188.stdout:-rw-r--r-- root/root 26201 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/platform/chrome/cros_ec_lightbar.ko 2023-11-26T07:02:44.786 INFO:teuthology.orchestra.run.smithi188.stdout:-rw-r--r-- root/root 18201 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/platform/chrome/cros_ec_sysfs.ko 2023-11-26T07:02:44.786 INFO:teuthology.orchestra.run.smithi188.stdout:-rw-r--r-- root/root 17361 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/platform/chrome/cros_usbpd_notify.ko 2023-11-26T07:02:44.786 INFO:teuthology.orchestra.run.smithi188.stdout:drwxr-xr-x root/root 0 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/platform/x86/ 2023-11-26T07:02:44.786 INFO:teuthology.orchestra.run.smithi188.stdout:-rw-r--r-- root/root 67097 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/platform/x86/acer-wmi.ko 2023-11-26T07:02:44.786 INFO:teuthology.orchestra.run.smithi188.stdout:-rw-r--r-- root/root 39145 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/platform/x86/acerhdf.ko 2023-11-26T07:02:44.786 INFO:teuthology.orchestra.run.smithi188.stdout:-rw-r--r-- root/root 37025 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/platform/x86/apple-gmux.ko 2023-11-26T07:02:44.786 INFO:teuthology.orchestra.run.smithi188.stdout:-rw-r--r-- root/root 59665 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/platform/x86/asus-laptop.ko 2023-11-26T07:02:44.787 INFO:teuthology.orchestra.run.smithi188.stdout:-rw-r--r-- root/root 31161 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/platform/x86/asus-nb-wmi.ko 2023-11-26T07:02:44.787 INFO:teuthology.orchestra.run.smithi188.stdout:-rw-r--r-- root/root 127441 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/platform/x86/asus-wmi.ko 2023-11-26T07:02:44.787 INFO:teuthology.orchestra.run.smithi188.stdout:-rw-r--r-- root/root 36849 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/platform/x86/classmate-laptop.ko 2023-11-26T07:02:44.787 INFO:teuthology.orchestra.run.smithi188.stdout:-rw-r--r-- root/root 49121 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/platform/x86/eeepc-laptop.ko 2023-11-26T07:02:44.787 INFO:teuthology.orchestra.run.smithi188.stdout:-rw-r--r-- root/root 13961 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/platform/x86/eeepc-wmi.ko 2023-11-26T07:02:44.787 INFO:teuthology.orchestra.run.smithi188.stdout:-rw-r--r-- root/root 40265 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/platform/x86/fujitsu-laptop.ko 2023-11-26T07:02:44.787 INFO:teuthology.orchestra.run.smithi188.stdout:-rw-r--r-- root/root 22521 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/platform/x86/fujitsu-tablet.ko 2023-11-26T07:02:44.787 INFO:teuthology.orchestra.run.smithi188.stdout:-rw-r--r-- root/root 36593 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/platform/x86/hdaps.ko 2023-11-26T07:02:44.787 INFO:teuthology.orchestra.run.smithi188.stdout:-rw-r--r-- root/root 19449 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/platform/x86/ibm_rtl.ko 2023-11-26T07:02:44.787 INFO:teuthology.orchestra.run.smithi188.stdout:drwxr-xr-x root/root 0 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/platform/x86/intel/ 2023-11-26T07:02:44.787 INFO:teuthology.orchestra.run.smithi188.stdout:-rw-r--r-- root/root 12073 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/platform/x86/intel/intel-rst.ko 2023-11-26T07:02:44.788 INFO:teuthology.orchestra.run.smithi188.stdout:-rw-r--r-- root/root 8985 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/platform/x86/intel/intel-smartconnect.ko 2023-11-26T07:02:44.788 INFO:teuthology.orchestra.run.smithi188.stdout:-rw-r--r-- root/root 43497 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/platform/x86/intel_ips.ko 2023-11-26T07:02:44.788 INFO:teuthology.orchestra.run.smithi188.stdout:-rw-r--r-- root/root 20273 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/platform/x86/msi-wmi.ko 2023-11-26T07:02:44.788 INFO:teuthology.orchestra.run.smithi188.stdout:-rw-r--r-- root/root 10377 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/platform/x86/mxm-wmi.ko 2023-11-26T07:02:44.788 INFO:teuthology.orchestra.run.smithi188.stdout:-rw-r--r-- root/root 34801 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/platform/x86/panasonic-laptop.ko 2023-11-26T07:02:44.788 INFO:teuthology.orchestra.run.smithi188.stdout:-rw-r--r-- root/root 42329 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/platform/x86/samsung-laptop.ko 2023-11-26T07:02:44.788 INFO:teuthology.orchestra.run.smithi188.stdout:-rw-r--r-- root/root 14169 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/platform/x86/samsung-q10.ko 2023-11-26T07:02:44.788 INFO:teuthology.orchestra.run.smithi188.stdout:-rw-r--r-- root/root 17321 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/platform/x86/topstar-laptop.ko 2023-11-26T07:02:44.788 INFO:teuthology.orchestra.run.smithi188.stdout:-rw-r--r-- root/root 13721 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/platform/x86/toshiba_bluetooth.ko 2023-11-26T07:02:44.788 INFO:teuthology.orchestra.run.smithi188.stdout:-rw-r--r-- root/root 17081 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/platform/x86/toshiba_haps.ko 2023-11-26T07:02:44.789 INFO:teuthology.orchestra.run.smithi188.stdout:-rw-r--r-- root/root 10129 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/platform/x86/wmi-bmof.ko 2023-11-26T07:02:44.789 INFO:teuthology.orchestra.run.smithi188.stdout:-rw-r--r-- root/root 57569 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/platform/x86/wmi.ko 2023-11-26T07:02:44.789 INFO:teuthology.orchestra.run.smithi188.stdout:drwxr-xr-x root/root 0 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/power/ 2023-11-26T07:02:44.789 INFO:teuthology.orchestra.run.smithi188.stdout:drwxr-xr-x root/root 0 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/power/supply/ 2023-11-26T07:02:44.789 INFO:teuthology.orchestra.run.smithi188.stdout:-rw-r--r-- root/root 30281 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/power/supply/88pm860x_battery.ko 2023-11-26T07:02:44.789 INFO:teuthology.orchestra.run.smithi188.stdout:-rw-r--r-- root/root 33985 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/power/supply/88pm860x_charger.ko 2023-11-26T07:02:44.789 INFO:teuthology.orchestra.run.smithi188.stdout:-rw-r--r-- root/root 55057 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/power/supply/bq2415x_charger.ko 2023-11-26T07:02:44.789 INFO:teuthology.orchestra.run.smithi188.stdout:-rw-r--r-- root/root 54081 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/power/supply/bq24190_charger.ko 2023-11-26T07:02:44.789 INFO:teuthology.orchestra.run.smithi188.stdout:-rw-r--r-- root/root 21553 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/power/supply/bq24735-charger.ko 2023-11-26T07:02:44.789 INFO:teuthology.orchestra.run.smithi188.stdout:-rw-r--r-- root/root 17337 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/power/supply/cros_peripheral_charger.ko 2023-11-26T07:02:44.789 INFO:teuthology.orchestra.run.smithi188.stdout:-rw-r--r-- root/root 19609 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/power/supply/da9030_battery.ko 2023-11-26T07:02:44.790 INFO:teuthology.orchestra.run.smithi188.stdout:-rw-r--r-- root/root 19841 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/power/supply/da9052-battery.ko 2023-11-26T07:02:44.790 INFO:teuthology.orchestra.run.smithi188.stdout:-rw-r--r-- root/root 29281 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/power/supply/ds2760_battery.ko 2023-11-26T07:02:44.790 INFO:teuthology.orchestra.run.smithi188.stdout:-rw-r--r-- root/root 23217 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/power/supply/ds2780_battery.ko 2023-11-26T07:02:44.790 INFO:teuthology.orchestra.run.smithi188.stdout:-rw-r--r-- root/root 23353 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/power/supply/ds2781_battery.ko 2023-11-26T07:02:44.790 INFO:teuthology.orchestra.run.smithi188.stdout:-rw-r--r-- root/root 23649 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/power/supply/ds2782_battery.ko 2023-11-26T07:02:44.790 INFO:teuthology.orchestra.run.smithi188.stdout:-rw-r--r-- root/root 20353 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/power/supply/gpio-charger.ko 2023-11-26T07:02:44.790 INFO:teuthology.orchestra.run.smithi188.stdout:-rw-r--r-- root/root 18041 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/power/supply/isp1704_charger.ko 2023-11-26T07:02:44.790 INFO:teuthology.orchestra.run.smithi188.stdout:-rw-r--r-- root/root 20273 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/power/supply/lp8727_charger.ko 2023-11-26T07:02:44.790 INFO:teuthology.orchestra.run.smithi188.stdout:-rw-r--r-- root/root 12577 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/power/supply/max14577_charger.ko 2023-11-26T07:02:44.790 INFO:teuthology.orchestra.run.smithi188.stdout:-rw-r--r-- root/root 25897 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/power/supply/max17040_battery.ko 2023-11-26T07:02:44.791 INFO:teuthology.orchestra.run.smithi188.stdout:-rw-r--r-- root/root 30289 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/power/supply/max17042_battery.ko 2023-11-26T07:02:44.791 INFO:teuthology.orchestra.run.smithi188.stdout:-rw-r--r-- root/root 18585 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/power/supply/max8903_charger.ko 2023-11-26T07:02:44.791 INFO:teuthology.orchestra.run.smithi188.stdout:-rw-r--r-- root/root 23729 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/power/supply/max8925_power.ko 2023-11-26T07:02:44.791 INFO:teuthology.orchestra.run.smithi188.stdout:-rw-r--r-- root/root 21657 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/power/supply/pcf50633-charger.ko 2023-11-26T07:02:44.791 INFO:teuthology.orchestra.run.smithi188.stdout:-rw-r--r-- root/root 38009 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/power/supply/sbs-battery.ko 2023-11-26T07:02:44.791 INFO:teuthology.orchestra.run.smithi188.stdout:-rw-r--r-- root/root 28385 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/power/supply/test_power.ko 2023-11-26T07:02:44.791 INFO:teuthology.orchestra.run.smithi188.stdout:-rw-r--r-- root/root 18865 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/power/supply/tps65090-charger.ko 2023-11-26T07:02:44.791 INFO:teuthology.orchestra.run.smithi188.stdout:-rw-r--r-- root/root 11969 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/power/supply/wm831x_backup.ko 2023-11-26T07:02:44.791 INFO:teuthology.orchestra.run.smithi188.stdout:-rw-r--r-- root/root 24521 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/power/supply/wm831x_power.ko 2023-11-26T07:02:44.791 INFO:teuthology.orchestra.run.smithi188.stdout:-rw-r--r-- root/root 22417 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/power/supply/wm8350_power.ko 2023-11-26T07:02:44.791 INFO:teuthology.orchestra.run.smithi188.stdout:drwxr-xr-x root/root 0 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/powercap/ 2023-11-26T07:02:44.792 INFO:teuthology.orchestra.run.smithi188.stdout:-rw-r--r-- root/root 57729 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/powercap/intel_rapl_common.ko 2023-11-26T07:02:44.792 INFO:teuthology.orchestra.run.smithi188.stdout:-rw-r--r-- root/root 17345 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/powercap/intel_rapl_msr.ko 2023-11-26T07:02:44.792 INFO:teuthology.orchestra.run.smithi188.stdout:drwxr-xr-x root/root 0 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/pps/ 2023-11-26T07:02:44.792 INFO:teuthology.orchestra.run.smithi188.stdout:drwxr-xr-x root/root 0 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/pps/clients/ 2023-11-26T07:02:44.792 INFO:teuthology.orchestra.run.smithi188.stdout:-rw-r--r-- root/root 15521 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/pps/clients/pps-gpio.ko 2023-11-26T07:02:44.792 INFO:teuthology.orchestra.run.smithi188.stdout:-rw-r--r-- root/root 13249 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/pps/clients/pps-ldisc.ko 2023-11-26T07:02:44.792 INFO:teuthology.orchestra.run.smithi188.stdout:-rw-r--r-- root/root 15913 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/pps/clients/pps_parport.ko 2023-11-26T07:02:44.792 INFO:teuthology.orchestra.run.smithi188.stdout:-rw-r--r-- root/root 41873 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/pps/pps_core.ko 2023-11-26T07:02:44.792 INFO:teuthology.orchestra.run.smithi188.stdout:drwxr-xr-x root/root 0 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/ptp/ 2023-11-26T07:02:44.792 INFO:teuthology.orchestra.run.smithi188.stdout:-rw-r--r-- root/root 66385 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/ptp/ptp.ko 2023-11-26T07:02:44.793 INFO:teuthology.orchestra.run.smithi188.stdout:-rw-r--r-- root/root 16449 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/ptp/ptp_kvm.ko 2023-11-26T07:02:44.793 INFO:teuthology.orchestra.run.smithi188.stdout:drwxr-xr-x root/root 0 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/rtc/ 2023-11-26T07:02:44.793 INFO:teuthology.orchestra.run.smithi188.stdout:-rw-r--r-- root/root 23497 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/rtc/rtc-88pm80x.ko 2023-11-26T07:02:44.793 INFO:teuthology.orchestra.run.smithi188.stdout:-rw-r--r-- root/root 24521 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/rtc/rtc-88pm860x.ko 2023-11-26T07:02:44.793 INFO:teuthology.orchestra.run.smithi188.stdout:-rw-r--r-- root/root 15393 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/rtc/rtc-bq32k.ko 2023-11-26T07:02:44.793 INFO:teuthology.orchestra.run.smithi188.stdout:-rw-r--r-- root/root 20105 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/rtc/rtc-da9052.ko 2023-11-26T07:02:44.793 INFO:teuthology.orchestra.run.smithi188.stdout:-rw-r--r-- root/root 18505 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/rtc/rtc-da9055.ko 2023-11-26T07:02:44.793 INFO:teuthology.orchestra.run.smithi188.stdout:-rw-r--r-- root/root 20425 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/rtc/rtc-da9063.ko 2023-11-26T07:02:44.793 INFO:teuthology.orchestra.run.smithi188.stdout:-rw-r--r-- root/root 16457 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/rtc/rtc-ds1286.ko 2023-11-26T07:02:44.793 INFO:teuthology.orchestra.run.smithi188.stdout:-rw-r--r-- root/root 68577 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/rtc/rtc-ds1307.ko 2023-11-26T07:02:44.793 INFO:teuthology.orchestra.run.smithi188.stdout:-rw-r--r-- root/root 20817 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/rtc/rtc-ds1374.ko 2023-11-26T07:02:44.794 INFO:teuthology.orchestra.run.smithi188.stdout:-rw-r--r-- root/root 18217 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/rtc/rtc-ds1511.ko 2023-11-26T07:02:44.794 INFO:teuthology.orchestra.run.smithi188.stdout:-rw-r--r-- root/root 16737 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/rtc/rtc-ds1553.ko 2023-11-26T07:02:44.794 INFO:teuthology.orchestra.run.smithi188.stdout:-rw-r--r-- root/root 13953 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/rtc/rtc-ds1672.ko 2023-11-26T07:02:44.794 INFO:teuthology.orchestra.run.smithi188.stdout:-rw-r--r-- root/root 13369 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/rtc/rtc-ds1742.ko 2023-11-26T07:02:44.794 INFO:teuthology.orchestra.run.smithi188.stdout:-rw-r--r-- root/root 14697 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/rtc/rtc-ds2404.ko 2023-11-26T07:02:44.794 INFO:teuthology.orchestra.run.smithi188.stdout:-rw-r--r-- root/root 23705 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/rtc/rtc-ds3232.ko 2023-11-26T07:02:44.794 INFO:teuthology.orchestra.run.smithi188.stdout:-rw-r--r-- root/root 11081 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/rtc/rtc-em3027.ko 2023-11-26T07:02:44.794 INFO:teuthology.orchestra.run.smithi188.stdout:-rw-r--r-- root/root 24329 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/rtc/rtc-fm3130.ko 2023-11-26T07:02:44.794 INFO:teuthology.orchestra.run.smithi188.stdout:-rw-r--r-- root/root 20729 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/rtc/rtc-isl12022.ko 2023-11-26T07:02:44.794 INFO:teuthology.orchestra.run.smithi188.stdout:-rw-r--r-- root/root 35969 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/rtc/rtc-isl1208.ko 2023-11-26T07:02:44.795 INFO:teuthology.orchestra.run.smithi188.stdout:-rw-r--r-- root/root 14673 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/rtc/rtc-lp8788.ko 2023-11-26T07:02:44.795 INFO:teuthology.orchestra.run.smithi188.stdout:-rw-r--r-- root/root 38097 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/rtc/rtc-m41t80.ko 2023-11-26T07:02:44.795 INFO:teuthology.orchestra.run.smithi188.stdout:-rw-r--r-- root/root 12089 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/rtc/rtc-m48t35.ko 2023-11-26T07:02:44.795 INFO:teuthology.orchestra.run.smithi188.stdout:-rw-r--r-- root/root 25945 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/rtc/rtc-m48t59.ko 2023-11-26T07:02:44.795 INFO:teuthology.orchestra.run.smithi188.stdout:-rw-r--r-- root/root 14841 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/rtc/rtc-m48t86.ko 2023-11-26T07:02:44.795 INFO:teuthology.orchestra.run.smithi188.stdout:-rw-r--r-- root/root 11633 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/rtc/rtc-max6900.ko 2023-11-26T07:02:44.795 INFO:teuthology.orchestra.run.smithi188.stdout:-rw-r--r-- root/root 14673 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/rtc/rtc-max8907.ko 2023-11-26T07:02:44.795 INFO:teuthology.orchestra.run.smithi188.stdout:-rw-r--r-- root/root 15345 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/rtc/rtc-max8925.ko 2023-11-26T07:02:44.795 INFO:teuthology.orchestra.run.smithi188.stdout:-rw-r--r-- root/root 25809 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/rtc/rtc-max8997.ko 2023-11-26T07:02:44.795 INFO:teuthology.orchestra.run.smithi188.stdout:-rw-r--r-- root/root 16681 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/rtc/rtc-max8998.ko 2023-11-26T07:02:44.796 INFO:teuthology.orchestra.run.smithi188.stdout:-rw-r--r-- root/root 18513 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/rtc/rtc-mc13xxx.ko 2023-11-26T07:02:44.796 INFO:teuthology.orchestra.run.smithi188.stdout:-rw-r--r-- root/root 11409 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/rtc/rtc-msm6242.ko 2023-11-26T07:02:44.796 INFO:teuthology.orchestra.run.smithi188.stdout:-rw-r--r-- root/root 19625 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/rtc/rtc-palmas.ko 2023-11-26T07:02:44.796 INFO:teuthology.orchestra.run.smithi188.stdout:-rw-r--r-- root/root 46633 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/rtc/rtc-pcf2127.ko 2023-11-26T07:02:44.796 INFO:teuthology.orchestra.run.smithi188.stdout:-rw-r--r-- root/root 17977 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/rtc/rtc-pcf50633.ko 2023-11-26T07:02:44.796 INFO:teuthology.orchestra.run.smithi188.stdout:-rw-r--r-- root/root 26993 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/rtc/rtc-pcf85063.ko 2023-11-26T07:02:44.796 INFO:teuthology.orchestra.run.smithi188.stdout:-rw-r--r-- root/root 21825 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/rtc/rtc-pcf8523.ko 2023-11-26T07:02:44.796 INFO:teuthology.orchestra.run.smithi188.stdout:-rw-r--r-- root/root 29817 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/rtc/rtc-pcf8563.ko 2023-11-26T07:02:44.796 INFO:teuthology.orchestra.run.smithi188.stdout:-rw-r--r-- root/root 12441 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/rtc/rtc-pcf8583.ko 2023-11-26T07:02:44.797 INFO:teuthology.orchestra.run.smithi188.stdout:-rw-r--r-- root/root 18033 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/rtc/rtc-rc5t583.ko 2023-11-26T07:02:44.797 INFO:teuthology.orchestra.run.smithi188.stdout:-rw-r--r-- root/root 13049 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/rtc/rtc-rp5c01.ko 2023-11-26T07:02:44.797 INFO:teuthology.orchestra.run.smithi188.stdout:-rw-r--r-- root/root 33753 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/rtc/rtc-rs5c372.ko 2023-11-26T07:02:44.797 INFO:teuthology.orchestra.run.smithi188.stdout:-rw-r--r-- root/root 27833 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/rtc/rtc-rv3029c2.ko 2023-11-26T07:02:44.797 INFO:teuthology.orchestra.run.smithi188.stdout:-rw-r--r-- root/root 25257 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/rtc/rtc-rx8025.ko 2023-11-26T07:02:44.797 INFO:teuthology.orchestra.run.smithi188.stdout:-rw-r--r-- root/root 20113 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/rtc/rtc-rx8581.ko 2023-11-26T07:02:44.797 INFO:teuthology.orchestra.run.smithi188.stdout:-rw-r--r-- root/root 21617 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/rtc/rtc-s35390a.ko 2023-11-26T07:02:44.797 INFO:teuthology.orchestra.run.smithi188.stdout:-rw-r--r-- root/root 16825 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/rtc/rtc-stk17ta8.ko 2023-11-26T07:02:44.797 INFO:teuthology.orchestra.run.smithi188.stdout:-rw-r--r-- root/root 17889 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/rtc/rtc-tps6586x.ko 2023-11-26T07:02:44.797 INFO:teuthology.orchestra.run.smithi188.stdout:-rw-r--r-- root/root 21337 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/rtc/rtc-tps65910.ko 2023-11-26T07:02:44.797 INFO:teuthology.orchestra.run.smithi188.stdout:-rw-r--r-- root/root 20545 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/rtc/rtc-wm831x.ko 2023-11-26T07:02:44.797 INFO:teuthology.orchestra.run.smithi188.stdout:-rw-r--r-- root/root 21089 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/rtc/rtc-wm8350.ko 2023-11-26T07:02:44.798 INFO:teuthology.orchestra.run.smithi188.stdout:-rw-r--r-- root/root 27873 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/rtc/rtc-x1205.ko 2023-11-26T07:02:44.798 INFO:teuthology.orchestra.run.smithi188.stdout:drwxr-xr-x root/root 0 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/scsi/ 2023-11-26T07:02:44.798 INFO:teuthology.orchestra.run.smithi188.stdout:-rw-r--r-- root/root 77617 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/scsi/3w-9xxx.ko 2023-11-26T07:02:44.798 INFO:teuthology.orchestra.run.smithi188.stdout:-rw-r--r-- root/root 57665 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/scsi/3w-sas.ko 2023-11-26T07:02:44.798 INFO:teuthology.orchestra.run.smithi188.stdout:-rw-r--r-- root/root 55105 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/scsi/3w-xxxx.ko 2023-11-26T07:02:44.798 INFO:teuthology.orchestra.run.smithi188.stdout:-rw-r--r-- root/root 135185 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/scsi/BusLogic.ko 2023-11-26T07:02:44.798 INFO:teuthology.orchestra.run.smithi188.stdout:-rw-r--r-- root/root 26657 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/scsi/a100u2w.ko 2023-11-26T07:02:44.798 INFO:teuthology.orchestra.run.smithi188.stdout:drwxr-xr-x root/root 0 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/scsi/aacraid/ 2023-11-26T07:02:44.798 INFO:teuthology.orchestra.run.smithi188.stdout:-rw-r--r-- root/root 272305 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/scsi/aacraid/aacraid.ko 2023-11-26T07:02:44.798 INFO:teuthology.orchestra.run.smithi188.stdout:-rw-r--r-- root/root 100617 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/scsi/advansys.ko 2023-11-26T07:02:44.798 INFO:teuthology.orchestra.run.smithi188.stdout:drwxr-xr-x root/root 0 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/scsi/aic7xxx/ 2023-11-26T07:02:44.799 INFO:teuthology.orchestra.run.smithi188.stdout:-rw-r--r-- root/root 356201 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/scsi/aic7xxx/aic79xx.ko 2023-11-26T07:02:44.799 INFO:teuthology.orchestra.run.smithi188.stdout:-rw-r--r-- root/root 295801 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/scsi/aic7xxx/aic7xxx.ko 2023-11-26T07:02:44.799 INFO:teuthology.orchestra.run.smithi188.stdout:drwxr-xr-x root/root 0 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/scsi/aic94xx/ 2023-11-26T07:02:44.799 INFO:teuthology.orchestra.run.smithi188.stdout:-rw-r--r-- root/root 189185 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/scsi/aic94xx/aic94xx.ko 2023-11-26T07:02:44.799 INFO:teuthology.orchestra.run.smithi188.stdout:-rw-r--r-- root/root 24073 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/scsi/am53c974.ko 2023-11-26T07:02:44.799 INFO:teuthology.orchestra.run.smithi188.stdout:drwxr-xr-x root/root 0 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/scsi/arcmsr/ 2023-11-26T07:02:44.799 INFO:teuthology.orchestra.run.smithi188.stdout:-rw-r--r-- root/root 106881 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/scsi/arcmsr/arcmsr.ko 2023-11-26T07:02:44.799 INFO:teuthology.orchestra.run.smithi188.stdout:-rw-r--r-- root/root 38185 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/scsi/atp870u.ko 2023-11-26T07:02:44.799 INFO:teuthology.orchestra.run.smithi188.stdout:drwxr-xr-x root/root 0 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/scsi/be2iscsi/ 2023-11-26T07:02:44.799 INFO:teuthology.orchestra.run.smithi188.stdout:-rw-r--r-- root/root 246985 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/scsi/be2iscsi/be2iscsi.ko 2023-11-26T07:02:44.799 INFO:teuthology.orchestra.run.smithi188.stdout:drwxr-xr-x root/root 0 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/scsi/bfa/ 2023-11-26T07:02:44.799 INFO:teuthology.orchestra.run.smithi188.stdout:-rw-r--r-- root/root 1171945 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/scsi/bfa/bfa.ko 2023-11-26T07:02:44.800 INFO:teuthology.orchestra.run.smithi188.stdout:drwxr-xr-x root/root 0 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/scsi/bnx2fc/ 2023-11-26T07:02:44.800 INFO:teuthology.orchestra.run.smithi188.stdout:-rw-r--r-- root/root 237273 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/scsi/bnx2fc/bnx2fc.ko 2023-11-26T07:02:44.800 INFO:teuthology.orchestra.run.smithi188.stdout:drwxr-xr-x root/root 0 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/scsi/bnx2i/ 2023-11-26T07:02:44.800 INFO:teuthology.orchestra.run.smithi188.stdout:-rw-r--r-- root/root 115369 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/scsi/bnx2i/bnx2i.ko 2023-11-26T07:02:44.800 INFO:teuthology.orchestra.run.smithi188.stdout:-rw-r--r-- root/root 40585 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/scsi/ch.ko 2023-11-26T07:02:44.800 INFO:teuthology.orchestra.run.smithi188.stdout:drwxr-xr-x root/root 0 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/scsi/csiostor/ 2023-11-26T07:02:44.800 INFO:teuthology.orchestra.run.smithi188.stdout:-rw-r--r-- root/root 308921 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/scsi/csiostor/csiostor.ko 2023-11-26T07:02:44.800 INFO:teuthology.orchestra.run.smithi188.stdout:drwxr-xr-x root/root 0 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/scsi/cxgbi/ 2023-11-26T07:02:44.800 INFO:teuthology.orchestra.run.smithi188.stdout:drwxr-xr-x root/root 0 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/scsi/cxgbi/cxgb3i/ 2023-11-26T07:02:44.800 INFO:teuthology.orchestra.run.smithi188.stdout:-rw-r--r-- root/root 76145 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/scsi/cxgbi/cxgb3i/cxgb3i.ko 2023-11-26T07:02:44.800 INFO:teuthology.orchestra.run.smithi188.stdout:drwxr-xr-x root/root 0 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/scsi/cxgbi/cxgb4i/ 2023-11-26T07:02:44.800 INFO:teuthology.orchestra.run.smithi188.stdout:-rw-r--r-- root/root 101449 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/scsi/cxgbi/cxgb4i/cxgb4i.ko 2023-11-26T07:02:44.801 INFO:teuthology.orchestra.run.smithi188.stdout:-rw-r--r-- root/root 146689 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/scsi/cxgbi/libcxgbi.ko 2023-11-26T07:02:44.801 INFO:teuthology.orchestra.run.smithi188.stdout:-rw-r--r-- root/root 75553 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/scsi/dc395x.ko 2023-11-26T07:02:44.801 INFO:teuthology.orchestra.run.smithi188.stdout:-rw-r--r-- root/root 38073 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/scsi/dmx3191d.ko 2023-11-26T07:02:44.801 INFO:teuthology.orchestra.run.smithi188.stdout:drwxr-xr-x root/root 0 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/scsi/esas2r/ 2023-11-26T07:02:44.801 INFO:teuthology.orchestra.run.smithi188.stdout:-rw-r--r-- root/root 212425 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/scsi/esas2r/esas2r.ko 2023-11-26T07:02:44.801 INFO:teuthology.orchestra.run.smithi188.stdout:-rw-r--r-- root/root 60505 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/scsi/esp_scsi.ko 2023-11-26T07:02:44.801 INFO:teuthology.orchestra.run.smithi188.stdout:drwxr-xr-x root/root 0 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/scsi/fcoe/ 2023-11-26T07:02:44.801 INFO:teuthology.orchestra.run.smithi188.stdout:-rw-r--r-- root/root 78209 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/scsi/fcoe/fcoe.ko 2023-11-26T07:02:44.801 INFO:teuthology.orchestra.run.smithi188.stdout:-rw-r--r-- root/root 158505 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/scsi/fcoe/libfcoe.ko 2023-11-26T07:02:44.801 INFO:teuthology.orchestra.run.smithi188.stdout:-rw-r--r-- root/root 18945 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/scsi/fdomain.ko 2023-11-26T07:02:44.801 INFO:teuthology.orchestra.run.smithi188.stdout:drwxr-xr-x root/root 0 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/scsi/fnic/ 2023-11-26T07:02:44.802 INFO:teuthology.orchestra.run.smithi188.stdout:-rw-r--r-- root/root 263489 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/scsi/fnic/fnic.ko 2023-11-26T07:02:44.802 INFO:teuthology.orchestra.run.smithi188.stdout:-rw-r--r-- root/root 201817 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/scsi/hpsa.ko 2023-11-26T07:02:44.802 INFO:teuthology.orchestra.run.smithi188.stdout:-rw-r--r-- root/root 51209 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/scsi/hptiop.ko 2023-11-26T07:02:44.802 INFO:teuthology.orchestra.run.smithi188.stdout:-rw-r--r-- root/root 49041 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/scsi/hv_storvsc.ko 2023-11-26T07:02:44.802 INFO:teuthology.orchestra.run.smithi188.stdout:-rw-r--r-- root/root 35337 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/scsi/imm.ko 2023-11-26T07:02:44.802 INFO:teuthology.orchestra.run.smithi188.stdout:-rw-r--r-- root/root 44057 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/scsi/initio.ko 2023-11-26T07:02:44.802 INFO:teuthology.orchestra.run.smithi188.stdout:-rw-r--r-- root/root 266537 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/scsi/ipr.ko 2023-11-26T07:02:44.802 INFO:teuthology.orchestra.run.smithi188.stdout:-rw-r--r-- root/root 101881 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/scsi/ips.ko 2023-11-26T07:02:44.802 INFO:teuthology.orchestra.run.smithi188.stdout:drwxr-xr-x root/root 0 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/scsi/isci/ 2023-11-26T07:02:44.802 INFO:teuthology.orchestra.run.smithi188.stdout:-rw-r--r-- root/root 317153 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/scsi/isci/isci.ko 2023-11-26T07:02:44.802 INFO:teuthology.orchestra.run.smithi188.stdout:-rw-r--r-- root/root 26385 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/scsi/iscsi_boot_sysfs.ko 2023-11-26T07:02:44.802 INFO:teuthology.orchestra.run.smithi188.stdout:-rw-r--r-- root/root 52961 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/scsi/iscsi_tcp.ko 2023-11-26T07:02:44.803 INFO:teuthology.orchestra.run.smithi188.stdout:drwxr-xr-x root/root 0 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/scsi/libfc/ 2023-11-26T07:02:44.803 INFO:teuthology.orchestra.run.smithi188.stdout:-rw-r--r-- root/root 314865 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/scsi/libfc/libfc.ko 2023-11-26T07:02:44.803 INFO:teuthology.orchestra.run.smithi188.stdout:-rw-r--r-- root/root 145585 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/scsi/libiscsi.ko 2023-11-26T07:02:44.803 INFO:teuthology.orchestra.run.smithi188.stdout:-rw-r--r-- root/root 53521 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/scsi/libiscsi_tcp.ko 2023-11-26T07:02:44.803 INFO:teuthology.orchestra.run.smithi188.stdout:drwxr-xr-x root/root 0 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/scsi/libsas/ 2023-11-26T07:02:44.803 INFO:teuthology.orchestra.run.smithi188.stdout:-rw-r--r-- root/root 216665 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/scsi/libsas/libsas.ko 2023-11-26T07:02:44.803 INFO:teuthology.orchestra.run.smithi188.stdout:drwxr-xr-x root/root 0 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/scsi/lpfc/ 2023-11-26T07:02:44.803 INFO:teuthology.orchestra.run.smithi188.stdout:-rw-r--r-- root/root 2197249 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/scsi/lpfc/lpfc.ko 2023-11-26T07:02:44.803 INFO:teuthology.orchestra.run.smithi188.stdout:drwxr-xr-x root/root 0 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/scsi/megaraid/ 2023-11-26T07:02:44.803 INFO:teuthology.orchestra.run.smithi188.stdout:-rw-r--r-- root/root 73569 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/scsi/megaraid/megaraid_mbox.ko 2023-11-26T07:02:44.803 INFO:teuthology.orchestra.run.smithi188.stdout:-rw-r--r-- root/root 33529 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/scsi/megaraid/megaraid_mm.ko 2023-11-26T07:02:44.803 INFO:teuthology.orchestra.run.smithi188.stdout:-rw-r--r-- root/root 310617 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/scsi/megaraid/megaraid_sas.ko 2023-11-26T07:02:44.804 INFO:teuthology.orchestra.run.smithi188.stdout:-rw-r--r-- root/root 84721 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/scsi/megaraid.ko 2023-11-26T07:02:44.804 INFO:teuthology.orchestra.run.smithi188.stdout:drwxr-xr-x root/root 0 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/scsi/mpt3sas/ 2023-11-26T07:02:44.804 INFO:teuthology.orchestra.run.smithi188.stdout:-rw-r--r-- root/root 659825 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/scsi/mpt3sas/mpt3sas.ko 2023-11-26T07:02:44.804 INFO:teuthology.orchestra.run.smithi188.stdout:drwxr-xr-x root/root 0 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/scsi/mvsas/ 2023-11-26T07:02:44.804 INFO:teuthology.orchestra.run.smithi188.stdout:-rw-r--r-- root/root 119561 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/scsi/mvsas/mvsas.ko 2023-11-26T07:02:44.804 INFO:teuthology.orchestra.run.smithi188.stdout:-rw-r--r-- root/root 78129 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/scsi/mvumi.ko 2023-11-26T07:02:44.804 INFO:teuthology.orchestra.run.smithi188.stdout:drwxr-xr-x root/root 0 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/scsi/pcmcia/ 2023-11-26T07:02:44.804 INFO:teuthology.orchestra.run.smithi188.stdout:-rw-r--r-- root/root 81625 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/scsi/pcmcia/aha152x_cs.ko 2023-11-26T07:02:44.804 INFO:teuthology.orchestra.run.smithi188.stdout:-rw-r--r-- root/root 11713 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/scsi/pcmcia/fdomain_cs.ko 2023-11-26T07:02:44.804 INFO:teuthology.orchestra.run.smithi188.stdout:-rw-r--r-- root/root 20865 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/scsi/pcmcia/qlogic_cs.ko 2023-11-26T07:02:44.804 INFO:teuthology.orchestra.run.smithi188.stdout:-rw-r--r-- root/root 23857 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/scsi/pcmcia/sym53c500_cs.ko 2023-11-26T07:02:44.804 INFO:teuthology.orchestra.run.smithi188.stdout:drwxr-xr-x root/root 0 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/scsi/pm8001/ 2023-11-26T07:02:44.805 INFO:teuthology.orchestra.run.smithi188.stdout:-rw-r--r-- root/root 364105 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/scsi/pm8001/pm80xx.ko 2023-11-26T07:02:44.805 INFO:teuthology.orchestra.run.smithi188.stdout:-rw-r--r-- root/root 123617 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/scsi/pmcraid.ko 2023-11-26T07:02:44.805 INFO:teuthology.orchestra.run.smithi188.stdout:-rw-r--r-- root/root 34513 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/scsi/ppa.ko 2023-11-26T07:02:44.805 INFO:teuthology.orchestra.run.smithi188.stdout:-rw-r--r-- root/root 55361 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/scsi/qla1280.ko 2023-11-26T07:02:44.805 INFO:teuthology.orchestra.run.smithi188.stdout:drwxr-xr-x root/root 0 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/scsi/qla2xxx/ 2023-11-26T07:02:44.805 INFO:teuthology.orchestra.run.smithi188.stdout:-rw-r--r-- root/root 1876689 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/scsi/qla2xxx/qla2xxx.ko 2023-11-26T07:02:44.805 INFO:teuthology.orchestra.run.smithi188.stdout:-rw-r--r-- root/root 85289 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/scsi/qla2xxx/tcm_qla2xxx.ko 2023-11-26T07:02:44.805 INFO:teuthology.orchestra.run.smithi188.stdout:drwxr-xr-x root/root 0 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/scsi/qla4xxx/ 2023-11-26T07:02:44.805 INFO:teuthology.orchestra.run.smithi188.stdout:-rw-r--r-- root/root 559033 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/scsi/qla4xxx/qla4xxx.ko 2023-11-26T07:02:44.805 INFO:teuthology.orchestra.run.smithi188.stdout:-rw-r--r-- root/root 19025 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/scsi/qlogicfas408.ko 2023-11-26T07:02:44.805 INFO:teuthology.orchestra.run.smithi188.stdout:-rw-r--r-- root/root 15993 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/scsi/raid_class.ko 2023-11-26T07:02:44.805 INFO:teuthology.orchestra.run.smithi188.stdout:-rw-r--r-- root/root 238761 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/scsi/scsi_debug.ko 2023-11-26T07:02:44.806 INFO:teuthology.orchestra.run.smithi188.stdout:-rw-r--r-- root/root 173937 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/scsi/scsi_transport_fc.ko 2023-11-26T07:02:44.806 INFO:teuthology.orchestra.run.smithi188.stdout:-rw-r--r-- root/root 279313 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/scsi/scsi_transport_iscsi.ko 2023-11-26T07:02:44.806 INFO:teuthology.orchestra.run.smithi188.stdout:-rw-r--r-- root/root 97729 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/scsi/scsi_transport_sas.ko 2023-11-26T07:02:44.806 INFO:teuthology.orchestra.run.smithi188.stdout:-rw-r--r-- root/root 73881 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/scsi/scsi_transport_spi.ko 2023-11-26T07:02:44.806 INFO:teuthology.orchestra.run.smithi188.stdout:-rw-r--r-- root/root 44993 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/scsi/scsi_transport_srp.ko 2023-11-26T07:02:44.806 INFO:teuthology.orchestra.run.smithi188.stdout:-rw-r--r-- root/root 26249 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/scsi/ses.ko 2023-11-26T07:02:44.806 INFO:teuthology.orchestra.run.smithi188.stdout:-rw-r--r-- root/root 115889 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/scsi/st.ko 2023-11-26T07:02:44.806 INFO:teuthology.orchestra.run.smithi188.stdout:-rw-r--r-- root/root 46825 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/scsi/stex.ko 2023-11-26T07:02:44.806 INFO:teuthology.orchestra.run.smithi188.stdout:drwxr-xr-x root/root 0 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/scsi/sym53c8xx_2/ 2023-11-26T07:02:44.806 INFO:teuthology.orchestra.run.smithi188.stdout:-rw-r--r-- root/root 162121 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/scsi/sym53c8xx_2/sym53c8xx.ko 2023-11-26T07:02:44.806 INFO:teuthology.orchestra.run.smithi188.stdout:-rw-r--r-- root/root 42297 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/scsi/virtio_scsi.ko 2023-11-26T07:02:44.807 INFO:teuthology.orchestra.run.smithi188.stdout:-rw-r--r-- root/root 50321 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/scsi/vmw_pvscsi.ko 2023-11-26T07:02:44.807 INFO:teuthology.orchestra.run.smithi188.stdout:-rw-r--r-- root/root 39961 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/scsi/wd719x.ko 2023-11-26T07:02:44.807 INFO:teuthology.orchestra.run.smithi188.stdout:-rw-r--r-- root/root 50025 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/scsi/xen-scsifront.ko 2023-11-26T07:02:44.807 INFO:teuthology.orchestra.run.smithi188.stdout:drwxr-xr-x root/root 0 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/ssb/ 2023-11-26T07:02:44.807 INFO:teuthology.orchestra.run.smithi188.stdout:-rw-r--r-- root/root 129441 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/ssb/ssb.ko 2023-11-26T07:02:44.807 INFO:teuthology.orchestra.run.smithi188.stdout:drwxr-xr-x root/root 0 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/target/ 2023-11-26T07:02:44.807 INFO:teuthology.orchestra.run.smithi188.stdout:drwxr-xr-x root/root 0 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/target/iscsi/ 2023-11-26T07:02:44.807 INFO:teuthology.orchestra.run.smithi188.stdout:-rw-r--r-- root/root 736601 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/target/iscsi/iscsi_target_mod.ko 2023-11-26T07:02:44.807 INFO:teuthology.orchestra.run.smithi188.stdout:drwxr-xr-x root/root 0 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/target/loopback/ 2023-11-26T07:02:44.807 INFO:teuthology.orchestra.run.smithi188.stdout:-rw-r--r-- root/root 49985 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/target/loopback/tcm_loop.ko 2023-11-26T07:02:44.807 INFO:teuthology.orchestra.run.smithi188.stdout:-rw-r--r-- root/root 39713 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/target/target_core_file.ko 2023-11-26T07:02:44.807 INFO:teuthology.orchestra.run.smithi188.stdout:-rw-r--r-- root/root 43025 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/target/target_core_iblock.ko 2023-11-26T07:02:44.808 INFO:teuthology.orchestra.run.smithi188.stdout:-rw-r--r-- root/root 930257 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/target/target_core_mod.ko 2023-11-26T07:02:44.808 INFO:teuthology.orchestra.run.smithi188.stdout:-rw-r--r-- root/root 44305 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/target/target_core_pscsi.ko 2023-11-26T07:02:44.808 INFO:teuthology.orchestra.run.smithi188.stdout:-rw-r--r-- root/root 122521 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/target/target_core_user.ko 2023-11-26T07:02:44.808 INFO:teuthology.orchestra.run.smithi188.stdout:drwxr-xr-x root/root 0 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/target/tcm_fc/ 2023-11-26T07:02:44.808 INFO:teuthology.orchestra.run.smithi188.stdout:-rw-r--r-- root/root 75593 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/target/tcm_fc/tcm_fc.ko 2023-11-26T07:02:44.808 INFO:teuthology.orchestra.run.smithi188.stdout:drwxr-xr-x root/root 0 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/thermal/ 2023-11-26T07:02:44.808 INFO:teuthology.orchestra.run.smithi188.stdout:drwxr-xr-x root/root 0 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/thermal/intel/ 2023-11-26T07:02:44.808 INFO:teuthology.orchestra.run.smithi188.stdout:drwxr-xr-x root/root 0 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/thermal/intel/int340x_thermal/ 2023-11-26T07:02:44.808 INFO:teuthology.orchestra.run.smithi188.stdout:-rw-r--r-- root/root 23481 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/thermal/intel/int340x_thermal/acpi_thermal_rel.ko 2023-11-26T07:02:44.808 INFO:teuthology.orchestra.run.smithi188.stdout:-rw-r--r-- root/root 29321 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/thermal/intel/int340x_thermal/int3400_thermal.ko 2023-11-26T07:02:44.808 INFO:teuthology.orchestra.run.smithi188.stdout:-rw-r--r-- root/root 9953 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/thermal/intel/int340x_thermal/int3401_thermal.ko 2023-11-26T07:02:44.808 INFO:teuthology.orchestra.run.smithi188.stdout:-rw-r--r-- root/root 10257 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/thermal/intel/int340x_thermal/int3402_thermal.ko 2023-11-26T07:02:44.809 INFO:teuthology.orchestra.run.smithi188.stdout:-rw-r--r-- root/root 15081 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/thermal/intel/int340x_thermal/int3403_thermal.ko 2023-11-26T07:02:44.809 INFO:teuthology.orchestra.run.smithi188.stdout:-rw-r--r-- root/root 16497 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/thermal/intel/int340x_thermal/int340x_thermal_zone.ko 2023-11-26T07:02:44.809 INFO:teuthology.orchestra.run.smithi188.stdout:-rw-r--r-- root/root 26809 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_device.ko 2023-11-26T07:02:44.809 INFO:teuthology.orchestra.run.smithi188.stdout:-rw-r--r-- root/root 20281 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_device_pci.ko 2023-11-26T07:02:44.809 INFO:teuthology.orchestra.run.smithi188.stdout:-rw-r--r-- root/root 14865 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_device_pci_legacy.ko 2023-11-26T07:02:44.809 INFO:teuthology.orchestra.run.smithi188.stdout:-rw-r--r-- root/root 14393 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_mbox.ko 2023-11-26T07:02:44.809 INFO:teuthology.orchestra.run.smithi188.stdout:-rw-r--r-- root/root 13001 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_rapl.ko 2023-11-26T07:02:44.809 INFO:teuthology.orchestra.run.smithi188.stdout:-rw-r--r-- root/root 37065 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_rfim.ko 2023-11-26T07:02:44.809 INFO:teuthology.orchestra.run.smithi188.stdout:-rw-r--r-- root/root 30729 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/thermal/intel/intel_powerclamp.ko 2023-11-26T07:02:44.809 INFO:teuthology.orchestra.run.smithi188.stdout:-rw-r--r-- root/root 18937 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/thermal/intel/intel_soc_dts_iosf.ko 2023-11-26T07:02:44.809 INFO:teuthology.orchestra.run.smithi188.stdout:-rw-r--r-- root/root 11521 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/thermal/intel/intel_soc_dts_thermal.ko 2023-11-26T07:02:44.809 INFO:teuthology.orchestra.run.smithi188.stdout:-rw-r--r-- root/root 26425 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/thermal/intel/x86_pkg_temp_thermal.ko 2023-11-26T07:02:44.809 INFO:teuthology.orchestra.run.smithi188.stdout:drwxr-xr-x root/root 0 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/tty/ 2023-11-26T07:02:44.810 INFO:teuthology.orchestra.run.smithi188.stdout:drwxr-xr-x root/root 0 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/tty/ipwireless/ 2023-11-26T07:02:44.810 INFO:teuthology.orchestra.run.smithi188.stdout:-rw-r--r-- root/root 78425 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/tty/ipwireless/ipwireless.ko 2023-11-26T07:02:44.810 INFO:teuthology.orchestra.run.smithi188.stdout:-rw-r--r-- root/root 57865 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/tty/moxa.ko 2023-11-26T07:02:44.810 INFO:teuthology.orchestra.run.smithi188.stdout:-rw-r--r-- root/root 50953 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/tty/mxser.ko 2023-11-26T07:02:44.810 INFO:teuthology.orchestra.run.smithi188.stdout:-rw-r--r-- root/root 108601 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/tty/n_gsm.ko 2023-11-26T07:02:44.810 INFO:teuthology.orchestra.run.smithi188.stdout:-rw-r--r-- root/root 31961 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/tty/n_hdlc.ko 2023-11-26T07:02:44.810 INFO:teuthology.orchestra.run.smithi188.stdout:-rw-r--r-- root/root 67929 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/tty/nozomi.ko 2023-11-26T07:02:44.810 INFO:teuthology.orchestra.run.smithi188.stdout:drwxr-xr-x root/root 0 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/tty/serial/ 2023-11-26T07:02:44.810 INFO:teuthology.orchestra.run.smithi188.stdout:drwxr-xr-x root/root 0 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/tty/serial/8250/ 2023-11-26T07:02:44.810 INFO:teuthology.orchestra.run.smithi188.stdout:-rw-r--r-- root/root 35753 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/tty/serial/8250/8250_dw.ko 2023-11-26T07:02:44.810 INFO:teuthology.orchestra.run.smithi188.stdout:-rw-r--r-- root/root 59497 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/tty/serial/8250/serial_cs.ko 2023-11-26T07:02:44.811 INFO:teuthology.orchestra.run.smithi188.stdout:-rw-r--r-- root/root 18097 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/tty/serial/altera_jtaguart.ko 2023-11-26T07:02:44.811 INFO:teuthology.orchestra.run.smithi188.stdout:-rw-r--r-- root/root 22577 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/tty/serial/altera_uart.ko 2023-11-26T07:02:44.811 INFO:teuthology.orchestra.run.smithi188.stdout:-rw-r--r-- root/root 9425 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/tty/serial/arc_uart.ko 2023-11-26T07:02:44.811 INFO:teuthology.orchestra.run.smithi188.stdout:-rw-r--r-- root/root 72425 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/tty/serial/fsl_lpuart.ko 2023-11-26T07:02:44.811 INFO:teuthology.orchestra.run.smithi188.stdout:drwxr-xr-x root/root 0 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/tty/serial/jsm/ 2023-11-26T07:02:44.811 INFO:teuthology.orchestra.run.smithi188.stdout:-rw-r--r-- root/root 103073 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/tty/serial/jsm/jsm.ko 2023-11-26T07:02:44.811 INFO:teuthology.orchestra.run.smithi188.stdout:-rw-r--r-- root/root 28785 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/tty/serial/rp2.ko 2023-11-26T07:02:44.811 INFO:teuthology.orchestra.run.smithi188.stdout:-rw-r--r-- root/root 52817 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/tty/serial/sc16is7xx.ko 2023-11-26T07:02:44.811 INFO:teuthology.orchestra.run.smithi188.stdout:drwxr-xr-x root/root 0 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/ufs/ 2023-11-26T07:02:44.811 INFO:teuthology.orchestra.run.smithi188.stdout:drwxr-xr-x root/root 0 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/ufs/core/ 2023-11-26T07:02:44.811 INFO:teuthology.orchestra.run.smithi188.stdout:-rw-r--r-- root/root 531265 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/ufs/core/ufshcd-core.ko 2023-11-26T07:02:44.811 INFO:teuthology.orchestra.run.smithi188.stdout:drwxr-xr-x root/root 0 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/ufs/host/ 2023-11-26T07:02:44.812 INFO:teuthology.orchestra.run.smithi188.stdout:-rw-r--r-- root/root 28481 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/ufs/host/ufshcd-pci.ko 2023-11-26T07:02:44.812 INFO:teuthology.orchestra.run.smithi188.stdout:-rw-r--r-- root/root 14601 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/ufs/host/ufshcd-pltfrm.ko 2023-11-26T07:02:44.812 INFO:teuthology.orchestra.run.smithi188.stdout:drwxr-xr-x root/root 0 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/uio/ 2023-11-26T07:02:44.812 INFO:teuthology.orchestra.run.smithi188.stdout:-rw-r--r-- root/root 44513 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/uio/uio.ko 2023-11-26T07:02:44.812 INFO:teuthology.orchestra.run.smithi188.stdout:-rw-r--r-- root/root 12241 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/uio/uio_aec.ko 2023-11-26T07:02:44.812 INFO:teuthology.orchestra.run.smithi188.stdout:-rw-r--r-- root/root 11281 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/uio/uio_cif.ko 2023-11-26T07:02:44.812 INFO:teuthology.orchestra.run.smithi188.stdout:-rw-r--r-- root/root 18329 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/uio/uio_dmem_genirq.ko 2023-11-26T07:02:44.812 INFO:teuthology.orchestra.run.smithi188.stdout:-rw-r--r-- root/root 12937 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/uio/uio_mf624.ko 2023-11-26T07:02:44.812 INFO:teuthology.orchestra.run.smithi188.stdout:-rw-r--r-- root/root 12177 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/uio/uio_netx.ko 2023-11-26T07:02:44.812 INFO:teuthology.orchestra.run.smithi188.stdout:-rw-r--r-- root/root 10601 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/uio/uio_pci_generic.ko 2023-11-26T07:02:44.812 INFO:teuthology.orchestra.run.smithi188.stdout:-rw-r--r-- root/root 16057 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/uio/uio_pdrv_genirq.ko 2023-11-26T07:02:44.812 INFO:teuthology.orchestra.run.smithi188.stdout:-rw-r--r-- root/root 13633 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/uio/uio_sercos3.ko 2023-11-26T07:02:44.813 INFO:teuthology.orchestra.run.smithi188.stdout:drwxr-xr-x root/root 0 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/ 2023-11-26T07:02:44.813 INFO:teuthology.orchestra.run.smithi188.stdout:drwxr-xr-x root/root 0 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/c67x00/ 2023-11-26T07:02:44.813 INFO:teuthology.orchestra.run.smithi188.stdout:-rw-r--r-- root/root 73297 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/c67x00/c67x00.ko 2023-11-26T07:02:44.813 INFO:teuthology.orchestra.run.smithi188.stdout:drwxr-xr-x root/root 0 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/chipidea/ 2023-11-26T07:02:44.813 INFO:teuthology.orchestra.run.smithi188.stdout:-rw-r--r-- root/root 144377 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/chipidea/ci_hdrc.ko 2023-11-26T07:02:44.813 INFO:teuthology.orchestra.run.smithi188.stdout:-rw-r--r-- root/root 18505 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/chipidea/ci_hdrc_msm.ko 2023-11-26T07:02:44.813 INFO:teuthology.orchestra.run.smithi188.stdout:-rw-r--r-- root/root 12713 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/chipidea/ci_hdrc_pci.ko 2023-11-26T07:02:44.813 INFO:teuthology.orchestra.run.smithi188.stdout:-rw-r--r-- root/root 13057 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/chipidea/ci_hdrc_usb2.ko 2023-11-26T07:02:44.813 INFO:teuthology.orchestra.run.smithi188.stdout:drwxr-xr-x root/root 0 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/class/ 2023-11-26T07:02:44.813 INFO:teuthology.orchestra.run.smithi188.stdout:-rw-r--r-- root/root 87089 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/class/cdc-acm.ko 2023-11-26T07:02:44.813 INFO:teuthology.orchestra.run.smithi188.stdout:-rw-r--r-- root/root 51345 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/class/cdc-wdm.ko 2023-11-26T07:02:44.813 INFO:teuthology.orchestra.run.smithi188.stdout:-rw-r--r-- root/root 47305 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/class/usblp.ko 2023-11-26T07:02:44.814 INFO:teuthology.orchestra.run.smithi188.stdout:-rw-r--r-- root/root 89857 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/class/usbtmc.ko 2023-11-26T07:02:44.814 INFO:teuthology.orchestra.run.smithi188.stdout:drwxr-xr-x root/root 0 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/common/ 2023-11-26T07:02:44.814 INFO:teuthology.orchestra.run.smithi188.stdout:-rw-r--r-- root/root 22953 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/common/ulpi.ko 2023-11-26T07:02:44.814 INFO:teuthology.orchestra.run.smithi188.stdout:drwxr-xr-x root/root 0 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/dwc2/ 2023-11-26T07:02:44.814 INFO:teuthology.orchestra.run.smithi188.stdout:-rw-r--r-- root/root 11529 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/dwc2/dwc2_pci.ko 2023-11-26T07:02:44.814 INFO:teuthology.orchestra.run.smithi188.stdout:drwxr-xr-x root/root 0 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/dwc3/ 2023-11-26T07:02:44.814 INFO:teuthology.orchestra.run.smithi188.stdout:-rw-r--r-- root/root 12089 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/dwc3/dwc3-haps.ko 2023-11-26T07:02:44.814 INFO:teuthology.orchestra.run.smithi188.stdout:-rw-r--r-- root/root 30041 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/dwc3/dwc3-pci.ko 2023-11-26T07:02:44.814 INFO:teuthology.orchestra.run.smithi188.stdout:-rw-r--r-- root/root 448505 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/dwc3/dwc3.ko 2023-11-26T07:02:44.814 INFO:teuthology.orchestra.run.smithi188.stdout:drwxr-xr-x root/root 0 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/gadget/ 2023-11-26T07:02:44.814 INFO:teuthology.orchestra.run.smithi188.stdout:drwxr-xr-x root/root 0 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/gadget/function/ 2023-11-26T07:02:44.815 INFO:teuthology.orchestra.run.smithi188.stdout:-rw-r--r-- root/root 60177 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/gadget/function/u_ether.ko 2023-11-26T07:02:44.815 INFO:teuthology.orchestra.run.smithi188.stdout:-rw-r--r-- root/root 51881 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/gadget/function/u_serial.ko 2023-11-26T07:02:44.815 INFO:teuthology.orchestra.run.smithi188.stdout:-rw-r--r-- root/root 28841 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/gadget/function/usb_f_acm.ko 2023-11-26T07:02:44.815 INFO:teuthology.orchestra.run.smithi188.stdout:-rw-r--r-- root/root 42449 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/gadget/function/usb_f_ecm.ko 2023-11-26T07:02:44.815 INFO:teuthology.orchestra.run.smithi188.stdout:-rw-r--r-- root/root 27953 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/gadget/function/usb_f_ecm_subset.ko 2023-11-26T07:02:44.815 INFO:teuthology.orchestra.run.smithi188.stdout:-rw-r--r-- root/root 36513 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/gadget/function/usb_f_eem.ko 2023-11-26T07:02:44.815 INFO:teuthology.orchestra.run.smithi188.stdout:-rw-r--r-- root/root 92489 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/gadget/function/usb_f_fs.ko 2023-11-26T07:02:44.815 INFO:teuthology.orchestra.run.smithi188.stdout:-rw-r--r-- root/root 51049 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/gadget/function/usb_f_hid.ko 2023-11-26T07:02:44.815 INFO:teuthology.orchestra.run.smithi188.stdout:-rw-r--r-- root/root 122081 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/gadget/function/usb_f_mass_storage.ko 2023-11-26T07:02:44.815 INFO:teuthology.orchestra.run.smithi188.stdout:-rw-r--r-- root/root 57681 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/gadget/function/usb_f_ncm.ko 2023-11-26T07:02:44.815 INFO:teuthology.orchestra.run.smithi188.stdout:-rw-r--r-- root/root 22185 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/gadget/function/usb_f_obex.ko 2023-11-26T07:02:44.816 INFO:teuthology.orchestra.run.smithi188.stdout:-rw-r--r-- root/root 31265 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/gadget/function/usb_f_phonet.ko 2023-11-26T07:02:44.816 INFO:teuthology.orchestra.run.smithi188.stdout:-rw-r--r-- root/root 57729 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/gadget/function/usb_f_printer.ko 2023-11-26T07:02:44.816 INFO:teuthology.orchestra.run.smithi188.stdout:-rw-r--r-- root/root 85609 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/gadget/function/usb_f_rndis.ko 2023-11-26T07:02:44.816 INFO:teuthology.orchestra.run.smithi188.stdout:-rw-r--r-- root/root 19273 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/gadget/function/usb_f_serial.ko 2023-11-26T07:02:44.816 INFO:teuthology.orchestra.run.smithi188.stdout:-rw-r--r-- root/root 58113 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/gadget/function/usb_f_ss_lb.ko 2023-11-26T07:02:44.816 INFO:teuthology.orchestra.run.smithi188.stdout:-rw-r--r-- root/root 67337 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/gadget/function/usb_f_tcm.ko 2023-11-26T07:02:44.816 INFO:teuthology.orchestra.run.smithi188.stdout:drwxr-xr-x root/root 0 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/gadget/legacy/ 2023-11-26T07:02:44.816 INFO:teuthology.orchestra.run.smithi188.stdout:-rw-r--r-- root/root 21265 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/gadget/legacy/g_acm_ms.ko 2023-11-26T07:02:44.816 INFO:teuthology.orchestra.run.smithi188.stdout:-rw-r--r-- root/root 18057 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/gadget/legacy/g_cdc.ko 2023-11-26T07:02:44.816 INFO:teuthology.orchestra.run.smithi188.stdout:-rw-r--r-- root/root 18465 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/gadget/legacy/g_dbgp.ko 2023-11-26T07:02:44.816 INFO:teuthology.orchestra.run.smithi188.stdout:-rw-r--r-- root/root 21657 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/gadget/legacy/g_ether.ko 2023-11-26T07:02:44.816 INFO:teuthology.orchestra.run.smithi188.stdout:-rw-r--r-- root/root 28401 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/gadget/legacy/g_ffs.ko 2023-11-26T07:02:44.817 INFO:teuthology.orchestra.run.smithi188.stdout:-rw-r--r-- root/root 19009 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/gadget/legacy/g_hid.ko 2023-11-26T07:02:44.817 INFO:teuthology.orchestra.run.smithi188.stdout:-rw-r--r-- root/root 20409 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/gadget/legacy/g_mass_storage.ko 2023-11-26T07:02:44.817 INFO:teuthology.orchestra.run.smithi188.stdout:-rw-r--r-- root/root 16977 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/gadget/legacy/g_ncm.ko 2023-11-26T07:02:44.817 INFO:teuthology.orchestra.run.smithi188.stdout:-rw-r--r-- root/root 32073 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/gadget/legacy/g_nokia.ko 2023-11-26T07:02:44.817 INFO:teuthology.orchestra.run.smithi188.stdout:-rw-r--r-- root/root 16793 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/gadget/legacy/g_printer.ko 2023-11-26T07:02:44.817 INFO:teuthology.orchestra.run.smithi188.stdout:-rw-r--r-- root/root 19393 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/gadget/legacy/g_serial.ko 2023-11-26T07:02:44.817 INFO:teuthology.orchestra.run.smithi188.stdout:-rw-r--r-- root/root 27201 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/gadget/legacy/g_zero.ko 2023-11-26T07:02:44.817 INFO:teuthology.orchestra.run.smithi188.stdout:-rw-r--r-- root/root 62185 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/gadget/legacy/gadgetfs.ko 2023-11-26T07:02:44.817 INFO:teuthology.orchestra.run.smithi188.stdout:-rw-r--r-- root/root 15201 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/gadget/legacy/tcm_usb_gadget.ko 2023-11-26T07:02:44.817 INFO:teuthology.orchestra.run.smithi188.stdout:-rw-r--r-- root/root 161225 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/gadget/libcomposite.ko 2023-11-26T07:02:44.817 INFO:teuthology.orchestra.run.smithi188.stdout:drwxr-xr-x root/root 0 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/gadget/udc/ 2023-11-26T07:02:44.817 INFO:teuthology.orchestra.run.smithi188.stdout:-rw-r--r-- root/root 17705 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/gadget/udc/amd5536udc_pci.ko 2023-11-26T07:02:44.818 INFO:teuthology.orchestra.run.smithi188.stdout:drwxr-xr-x root/root 0 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/gadget/udc/bdc/ 2023-11-26T07:02:44.818 INFO:teuthology.orchestra.run.smithi188.stdout:-rw-r--r-- root/root 157681 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/gadget/udc/bdc/bdc.ko 2023-11-26T07:02:44.818 INFO:teuthology.orchestra.run.smithi188.stdout:-rw-r--r-- root/root 36089 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/gadget/udc/goku_udc.ko 2023-11-26T07:02:44.818 INFO:teuthology.orchestra.run.smithi188.stdout:-rw-r--r-- root/root 52721 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/gadget/udc/gr_udc.ko 2023-11-26T07:02:44.818 INFO:teuthology.orchestra.run.smithi188.stdout:-rw-r--r-- root/root 69705 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/gadget/udc/mv_u3d_core.ko 2023-11-26T07:02:44.818 INFO:teuthology.orchestra.run.smithi188.stdout:-rw-r--r-- root/root 56033 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/gadget/udc/mv_udc.ko 2023-11-26T07:02:44.818 INFO:teuthology.orchestra.run.smithi188.stdout:-rw-r--r-- root/root 76057 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/gadget/udc/net2272.ko 2023-11-26T07:02:44.818 INFO:teuthology.orchestra.run.smithi188.stdout:-rw-r--r-- root/root 75073 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/gadget/udc/net2280.ko 2023-11-26T07:02:44.818 INFO:teuthology.orchestra.run.smithi188.stdout:-rw-r--r-- root/root 73169 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/gadget/udc/pch_udc.ko 2023-11-26T07:02:44.818 INFO:teuthology.orchestra.run.smithi188.stdout:-rw-r--r-- root/root 65809 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/gadget/udc/pxa27x_udc.ko 2023-11-26T07:02:44.818 INFO:teuthology.orchestra.run.smithi188.stdout:-rw-r--r-- root/root 66097 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/gadget/udc/r8a66597-udc.ko 2023-11-26T07:02:44.818 INFO:teuthology.orchestra.run.smithi188.stdout:-rw-r--r-- root/root 81505 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/gadget/udc/snps_udc_core.ko 2023-11-26T07:02:44.819 INFO:teuthology.orchestra.run.smithi188.stdout:-rw-r--r-- root/root 156073 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/gadget/udc/udc-core.ko 2023-11-26T07:02:44.819 INFO:teuthology.orchestra.run.smithi188.stdout:drwxr-xr-x root/root 0 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/host/ 2023-11-26T07:02:44.819 INFO:teuthology.orchestra.run.smithi188.stdout:-rw-r--r-- root/root 11585 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/host/bcma-hcd.ko 2023-11-26T07:02:44.819 INFO:teuthology.orchestra.run.smithi188.stdout:-rw-r--r-- root/root 80113 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/host/isp116x-hcd.ko 2023-11-26T07:02:44.819 INFO:teuthology.orchestra.run.smithi188.stdout:-rw-r--r-- root/root 87353 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/host/oxu210hp-hcd.ko 2023-11-26T07:02:44.819 INFO:teuthology.orchestra.run.smithi188.stdout:-rw-r--r-- root/root 78873 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/host/r8a66597-hcd.ko 2023-11-26T07:02:44.819 INFO:teuthology.orchestra.run.smithi188.stdout:-rw-r--r-- root/root 51329 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/host/sl811-hcd.ko 2023-11-26T07:02:44.819 INFO:teuthology.orchestra.run.smithi188.stdout:-rw-r--r-- root/root 15593 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/host/sl811_cs.ko 2023-11-26T07:02:44.819 INFO:teuthology.orchestra.run.smithi188.stdout:-rw-r--r-- root/root 13305 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/host/ssb-hcd.ko 2023-11-26T07:02:44.819 INFO:teuthology.orchestra.run.smithi188.stdout:-rw-r--r-- root/root 25945 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/host/xhci-plat-hcd.ko 2023-11-26T07:02:44.819 INFO:teuthology.orchestra.run.smithi188.stdout:drwxr-xr-x root/root 0 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/image/ 2023-11-26T07:02:44.819 INFO:teuthology.orchestra.run.smithi188.stdout:-rw-r--r-- root/root 34585 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/image/mdc800.ko 2023-11-26T07:02:44.820 INFO:teuthology.orchestra.run.smithi188.stdout:-rw-r--r-- root/root 21081 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/image/microtek.ko 2023-11-26T07:02:44.820 INFO:teuthology.orchestra.run.smithi188.stdout:drwxr-xr-x root/root 0 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/misc/ 2023-11-26T07:02:44.820 INFO:teuthology.orchestra.run.smithi188.stdout:-rw-r--r-- root/root 41721 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/misc/adutux.ko 2023-11-26T07:02:44.820 INFO:teuthology.orchestra.run.smithi188.stdout:-rw-r--r-- root/root 20713 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/misc/appledisplay.ko 2023-11-26T07:02:44.820 INFO:teuthology.orchestra.run.smithi188.stdout:-rw-r--r-- root/root 17721 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/misc/cypress_cy7c63.ko 2023-11-26T07:02:44.820 INFO:teuthology.orchestra.run.smithi188.stdout:-rw-r--r-- root/root 23049 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/misc/cytherm.ko 2023-11-26T07:02:44.820 INFO:teuthology.orchestra.run.smithi188.stdout:-rw-r--r-- root/root 13729 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/misc/ehset.ko 2023-11-26T07:02:44.820 INFO:teuthology.orchestra.run.smithi188.stdout:-rw-r--r-- root/root 15057 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/misc/emi26.ko 2023-11-26T07:02:44.820 INFO:teuthology.orchestra.run.smithi188.stdout:-rw-r--r-- root/root 15985 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/misc/emi62.ko 2023-11-26T07:02:44.820 INFO:teuthology.orchestra.run.smithi188.stdout:-rw-r--r-- root/root 11433 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/misc/ezusb.ko 2023-11-26T07:02:44.820 INFO:teuthology.orchestra.run.smithi188.stdout:-rw-r--r-- root/root 19297 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/misc/idmouse.ko 2023-11-26T07:02:44.820 INFO:teuthology.orchestra.run.smithi188.stdout:-rw-r--r-- root/root 36553 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/misc/iowarrior.ko 2023-11-26T07:02:44.821 INFO:teuthology.orchestra.run.smithi188.stdout:-rw-r--r-- root/root 11593 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/misc/isight_firmware.ko 2023-11-26T07:02:44.821 INFO:teuthology.orchestra.run.smithi188.stdout:-rw-r--r-- root/root 36529 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/misc/ldusb.ko 2023-11-26T07:02:44.821 INFO:teuthology.orchestra.run.smithi188.stdout:-rw-r--r-- root/root 36265 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/misc/legousbtower.ko 2023-11-26T07:02:44.821 INFO:teuthology.orchestra.run.smithi188.stdout:-rw-r--r-- root/root 27793 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/misc/lvstest.ko 2023-11-26T07:02:44.821 INFO:teuthology.orchestra.run.smithi188.stdout:drwxr-xr-x root/root 0 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/misc/sisusbvga/ 2023-11-26T07:02:44.821 INFO:teuthology.orchestra.run.smithi188.stdout:-rw-r--r-- root/root 64841 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/misc/sisusbvga/sisusbvga.ko 2023-11-26T07:02:44.821 INFO:teuthology.orchestra.run.smithi188.stdout:-rw-r--r-- root/root 12937 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/misc/trancevibrator.ko 2023-11-26T07:02:44.821 INFO:teuthology.orchestra.run.smithi188.stdout:-rw-r--r-- root/root 20473 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/misc/usb3503.ko 2023-11-26T07:02:44.821 INFO:teuthology.orchestra.run.smithi188.stdout:-rw-r--r-- root/root 25145 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/misc/usblcd.ko 2023-11-26T07:02:44.821 INFO:teuthology.orchestra.run.smithi188.stdout:-rw-r--r-- root/root 22753 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/misc/usbsevseg.ko 2023-11-26T07:02:44.821 INFO:teuthology.orchestra.run.smithi188.stdout:-rw-r--r-- root/root 70809 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/misc/usbtest.ko 2023-11-26T07:02:44.822 INFO:teuthology.orchestra.run.smithi188.stdout:-rw-r--r-- root/root 41905 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/misc/uss720.ko 2023-11-26T07:02:44.822 INFO:teuthology.orchestra.run.smithi188.stdout:-rw-r--r-- root/root 27617 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/misc/yurex.ko 2023-11-26T07:02:44.822 INFO:teuthology.orchestra.run.smithi188.stdout:drwxr-xr-x root/root 0 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/mon/ 2023-11-26T07:02:44.822 INFO:teuthology.orchestra.run.smithi188.stdout:-rw-r--r-- root/root 72953 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/mon/usbmon.ko 2023-11-26T07:02:44.822 INFO:teuthology.orchestra.run.smithi188.stdout:drwxr-xr-x root/root 0 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/musb/ 2023-11-26T07:02:44.822 INFO:teuthology.orchestra.run.smithi188.stdout:-rw-r--r-- root/root 301697 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/musb/musb_hdrc.ko 2023-11-26T07:02:44.822 INFO:teuthology.orchestra.run.smithi188.stdout:drwxr-xr-x root/root 0 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/phy/ 2023-11-26T07:02:44.822 INFO:teuthology.orchestra.run.smithi188.stdout:-rw-r--r-- root/root 21105 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/phy/phy-generic.ko 2023-11-26T07:02:44.822 INFO:teuthology.orchestra.run.smithi188.stdout:-rw-r--r-- root/root 20177 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/phy/phy-gpio-vbus-usb.ko 2023-11-26T07:02:44.822 INFO:teuthology.orchestra.run.smithi188.stdout:-rw-r--r-- root/root 12257 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/phy/phy-isp1301.ko 2023-11-26T07:02:44.822 INFO:teuthology.orchestra.run.smithi188.stdout:-rw-r--r-- root/root 21745 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/phy/phy-tahvo.ko 2023-11-26T07:02:44.822 INFO:teuthology.orchestra.run.smithi188.stdout:drwxr-xr-x root/root 0 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/serial/ 2023-11-26T07:02:44.823 INFO:teuthology.orchestra.run.smithi188.stdout:-rw-r--r-- root/root 13209 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/serial/aircable.ko 2023-11-26T07:02:44.823 INFO:teuthology.orchestra.run.smithi188.stdout:-rw-r--r-- root/root 33329 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/serial/ark3116.ko 2023-11-26T07:02:44.823 INFO:teuthology.orchestra.run.smithi188.stdout:-rw-r--r-- root/root 26633 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/serial/belkin_sa.ko 2023-11-26T07:02:44.823 INFO:teuthology.orchestra.run.smithi188.stdout:-rw-r--r-- root/root 38105 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/serial/ch341.ko 2023-11-26T07:02:44.823 INFO:teuthology.orchestra.run.smithi188.stdout:-rw-r--r-- root/root 70257 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/serial/cp210x.ko 2023-11-26T07:02:44.823 INFO:teuthology.orchestra.run.smithi188.stdout:-rw-r--r-- root/root 31057 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/serial/cyberjack.ko 2023-11-26T07:02:44.823 INFO:teuthology.orchestra.run.smithi188.stdout:-rw-r--r-- root/root 58217 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/serial/cypress_m8.ko 2023-11-26T07:02:44.823 INFO:teuthology.orchestra.run.smithi188.stdout:-rw-r--r-- root/root 49817 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/serial/digi_acceleport.ko 2023-11-26T07:02:44.823 INFO:teuthology.orchestra.run.smithi188.stdout:-rw-r--r-- root/root 9617 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/serial/empeg.ko 2023-11-26T07:02:44.823 INFO:teuthology.orchestra.run.smithi188.stdout:-rw-r--r-- root/root 44273 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/serial/f81232.ko 2023-11-26T07:02:44.823 INFO:teuthology.orchestra.run.smithi188.stdout:-rw-r--r-- root/root 175137 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/serial/ftdi_sio.ko 2023-11-26T07:02:44.823 INFO:teuthology.orchestra.run.smithi188.stdout:-rw-r--r-- root/root 53113 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/serial/garmin_gps.ko 2023-11-26T07:02:44.824 INFO:teuthology.orchestra.run.smithi188.stdout:-rw-r--r-- root/root 137081 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/serial/io_edgeport.ko 2023-11-26T07:02:44.824 INFO:teuthology.orchestra.run.smithi188.stdout:-rw-r--r-- root/root 124833 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/serial/io_ti.ko 2023-11-26T07:02:44.824 INFO:teuthology.orchestra.run.smithi188.stdout:-rw-r--r-- root/root 67769 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/serial/ipaq.ko 2023-11-26T07:02:44.824 INFO:teuthology.orchestra.run.smithi188.stdout:-rw-r--r-- root/root 19161 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/serial/ipw.ko 2023-11-26T07:02:44.824 INFO:teuthology.orchestra.run.smithi188.stdout:-rw-r--r-- root/root 26897 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/serial/ir-usb.ko 2023-11-26T07:02:44.824 INFO:teuthology.orchestra.run.smithi188.stdout:-rw-r--r-- root/root 58233 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/serial/iuu_phoenix.ko 2023-11-26T07:02:44.824 INFO:teuthology.orchestra.run.smithi188.stdout:-rw-r--r-- root/root 107521 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/serial/keyspan.ko 2023-11-26T07:02:44.824 INFO:teuthology.orchestra.run.smithi188.stdout:-rw-r--r-- root/root 35617 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/serial/keyspan_pda.ko 2023-11-26T07:02:44.824 INFO:teuthology.orchestra.run.smithi188.stdout:-rw-r--r-- root/root 25665 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/serial/kl5kusb105.ko 2023-11-26T07:02:44.824 INFO:teuthology.orchestra.run.smithi188.stdout:-rw-r--r-- root/root 33689 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/serial/kobil_sct.ko 2023-11-26T07:02:44.824 INFO:teuthology.orchestra.run.smithi188.stdout:-rw-r--r-- root/root 32953 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/serial/mct_u232.ko 2023-11-26T07:02:44.824 INFO:teuthology.orchestra.run.smithi188.stdout:-rw-r--r-- root/root 22033 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/serial/metro-usb.ko 2023-11-26T07:02:44.825 INFO:teuthology.orchestra.run.smithi188.stdout:-rw-r--r-- root/root 66449 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/serial/mos7720.ko 2023-11-26T07:02:44.825 INFO:teuthology.orchestra.run.smithi188.stdout:-rw-r--r-- root/root 78217 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/serial/mos7840.ko 2023-11-26T07:02:44.825 INFO:teuthology.orchestra.run.smithi188.stdout:-rw-r--r-- root/root 56337 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/serial/mxuport.ko 2023-11-26T07:02:44.825 INFO:teuthology.orchestra.run.smithi188.stdout:-rw-r--r-- root/root 14025 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/serial/navman.ko 2023-11-26T07:02:44.825 INFO:teuthology.orchestra.run.smithi188.stdout:-rw-r--r-- root/root 12009 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/serial/omninet.ko 2023-11-26T07:02:44.825 INFO:teuthology.orchestra.run.smithi188.stdout:-rw-r--r-- root/root 24633 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/serial/opticon.ko 2023-11-26T07:02:44.825 INFO:teuthology.orchestra.run.smithi188.stdout:-rw-r--r-- root/root 191137 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/serial/option.ko 2023-11-26T07:02:44.825 INFO:teuthology.orchestra.run.smithi188.stdout:-rw-r--r-- root/root 39073 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/serial/oti6858.ko 2023-11-26T07:02:44.825 INFO:teuthology.orchestra.run.smithi188.stdout:-rw-r--r-- root/root 54353 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/serial/pl2303.ko 2023-11-26T07:02:44.825 INFO:teuthology.orchestra.run.smithi188.stdout:-rw-r--r-- root/root 9737 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/serial/qcaux.ko 2023-11-26T07:02:44.825 INFO:teuthology.orchestra.run.smithi188.stdout:-rw-r--r-- root/root 37825 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/serial/qcserial.ko 2023-11-26T07:02:44.826 INFO:teuthology.orchestra.run.smithi188.stdout:-rw-r--r-- root/root 34321 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/serial/quatech2.ko 2023-11-26T07:02:44.826 INFO:teuthology.orchestra.run.smithi188.stdout:-rw-r--r-- root/root 15393 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/serial/safe_serial.ko 2023-11-26T07:02:44.826 INFO:teuthology.orchestra.run.smithi188.stdout:-rw-r--r-- root/root 51625 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/serial/sierra.ko 2023-11-26T07:02:44.826 INFO:teuthology.orchestra.run.smithi188.stdout:-rw-r--r-- root/root 21673 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/serial/spcp8x5.ko 2023-11-26T07:02:44.826 INFO:teuthology.orchestra.run.smithi188.stdout:-rw-r--r-- root/root 29993 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/serial/ssu100.ko 2023-11-26T07:02:44.826 INFO:teuthology.orchestra.run.smithi188.stdout:-rw-r--r-- root/root 17265 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/serial/symbolserial.ko 2023-11-26T07:02:44.826 INFO:teuthology.orchestra.run.smithi188.stdout:-rw-r--r-- root/root 61489 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/serial/ti_usb_3410_5052.ko 2023-11-26T07:02:44.826 INFO:teuthology.orchestra.run.smithi188.stdout:-rw-r--r-- root/root 23729 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/serial/usb-serial-simple.ko 2023-11-26T07:02:44.826 INFO:teuthology.orchestra.run.smithi188.stdout:-rw-r--r-- root/root 10241 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/serial/usb_debug.ko 2023-11-26T07:02:44.826 INFO:teuthology.orchestra.run.smithi188.stdout:-rw-r--r-- root/root 33201 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/serial/usb_wwan.ko 2023-11-26T07:02:44.827 INFO:teuthology.orchestra.run.smithi188.stdout:-rw-r--r-- root/root 125609 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/serial/usbserial.ko 2023-11-26T07:02:44.827 INFO:teuthology.orchestra.run.smithi188.stdout:-rw-r--r-- root/root 28753 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/serial/visor.ko 2023-11-26T07:02:44.827 INFO:teuthology.orchestra.run.smithi188.stdout:-rw-r--r-- root/root 37409 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/serial/whiteheat.ko 2023-11-26T07:02:44.827 INFO:teuthology.orchestra.run.smithi188.stdout:-rw-r--r-- root/root 10537 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/serial/wishbone-serial.ko 2023-11-26T07:02:44.827 INFO:teuthology.orchestra.run.smithi188.stdout:-rw-r--r-- root/root 9289 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/serial/xsens_mt.ko 2023-11-26T07:02:44.827 INFO:teuthology.orchestra.run.smithi188.stdout:drwxr-xr-x root/root 0 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/storage/ 2023-11-26T07:02:44.827 INFO:teuthology.orchestra.run.smithi188.stdout:-rw-r--r-- root/root 52913 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/storage/uas.ko 2023-11-26T07:02:44.827 INFO:teuthology.orchestra.run.smithi188.stdout:-rw-r--r-- root/root 26017 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/storage/ums-alauda.ko 2023-11-26T07:02:44.827 INFO:teuthology.orchestra.run.smithi188.stdout:-rw-r--r-- root/root 13049 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/storage/ums-cypress.ko 2023-11-26T07:02:44.827 INFO:teuthology.orchestra.run.smithi188.stdout:-rw-r--r-- root/root 22049 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/storage/ums-datafab.ko 2023-11-26T07:02:44.827 INFO:teuthology.orchestra.run.smithi188.stdout:-rw-r--r-- root/root 33497 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/storage/ums-eneub6250.ko 2023-11-26T07:02:44.827 INFO:teuthology.orchestra.run.smithi188.stdout:-rw-r--r-- root/root 13745 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/storage/ums-freecom.ko 2023-11-26T07:02:44.828 INFO:teuthology.orchestra.run.smithi188.stdout:-rw-r--r-- root/root 19769 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/storage/ums-isd200.ko 2023-11-26T07:02:44.828 INFO:teuthology.orchestra.run.smithi188.stdout:-rw-r--r-- root/root 19361 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/storage/ums-jumpshot.ko 2023-11-26T07:02:44.828 INFO:teuthology.orchestra.run.smithi188.stdout:-rw-r--r-- root/root 13713 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/storage/ums-karma.ko 2023-11-26T07:02:44.828 INFO:teuthology.orchestra.run.smithi188.stdout:-rw-r--r-- root/root 17841 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/storage/ums-onetouch.ko 2023-11-26T07:02:44.828 INFO:teuthology.orchestra.run.smithi188.stdout:-rw-r--r-- root/root 28441 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/storage/ums-realtek.ko 2023-11-26T07:02:44.828 INFO:teuthology.orchestra.run.smithi188.stdout:-rw-r--r-- root/root 28905 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/storage/ums-sddr09.ko 2023-11-26T07:02:44.828 INFO:teuthology.orchestra.run.smithi188.stdout:-rw-r--r-- root/root 19521 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/storage/ums-sddr55.ko 2023-11-26T07:02:44.828 INFO:teuthology.orchestra.run.smithi188.stdout:-rw-r--r-- root/root 35337 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/storage/ums-usbat.ko 2023-11-26T07:02:44.828 INFO:teuthology.orchestra.run.smithi188.stdout:-rw-r--r-- root/root 179969 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/storage/usb-storage.ko 2023-11-26T07:02:44.828 INFO:teuthology.orchestra.run.smithi188.stdout:drwxr-xr-x root/root 0 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/usbip/ 2023-11-26T07:02:44.828 INFO:teuthology.orchestra.run.smithi188.stdout:-rw-r--r-- root/root 67481 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/usbip/usbip-core.ko 2023-11-26T07:02:44.828 INFO:teuthology.orchestra.run.smithi188.stdout:-rw-r--r-- root/root 72177 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/usbip/usbip-host.ko 2023-11-26T07:02:44.828 INFO:teuthology.orchestra.run.smithi188.stdout:-rw-r--r-- root/root 107001 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/usbip/vhci-hcd.ko 2023-11-26T07:02:44.829 INFO:teuthology.orchestra.run.smithi188.stdout:drwxr-xr-x root/root 0 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/vfio/ 2023-11-26T07:02:44.829 INFO:teuthology.orchestra.run.smithi188.stdout:drwxr-xr-x root/root 0 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/vfio/pci/ 2023-11-26T07:02:44.829 INFO:teuthology.orchestra.run.smithi188.stdout:-rw-r--r-- root/root 154297 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/vfio/pci/vfio-pci-core.ko 2023-11-26T07:02:44.829 INFO:teuthology.orchestra.run.smithi188.stdout:-rw-r--r-- root/root 29537 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/vfio/pci/vfio-pci.ko 2023-11-26T07:02:44.829 INFO:teuthology.orchestra.run.smithi188.stdout:-rw-r--r-- root/root 108737 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/vfio/vfio.ko 2023-11-26T07:02:44.829 INFO:teuthology.orchestra.run.smithi188.stdout:-rw-r--r-- root/root 77609 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/vfio/vfio_iommu_type1.ko 2023-11-26T07:02:44.829 INFO:teuthology.orchestra.run.smithi188.stdout:drwxr-xr-x root/root 0 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/vhost/ 2023-11-26T07:02:44.829 INFO:teuthology.orchestra.run.smithi188.stdout:-rw-r--r-- root/root 113785 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/vhost/vhost.ko 2023-11-26T07:02:44.829 INFO:teuthology.orchestra.run.smithi188.stdout:-rw-r--r-- root/root 17009 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/vhost/vhost_iotlb.ko 2023-11-26T07:02:44.829 INFO:teuthology.orchestra.run.smithi188.stdout:-rw-r--r-- root/root 58089 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/vhost/vhost_net.ko 2023-11-26T07:02:44.829 INFO:teuthology.orchestra.run.smithi188.stdout:-rw-r--r-- root/root 89249 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/vhost/vhost_scsi.ko 2023-11-26T07:02:44.829 INFO:teuthology.orchestra.run.smithi188.stdout:drwxr-xr-x root/root 0 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/video/ 2023-11-26T07:02:44.830 INFO:teuthology.orchestra.run.smithi188.stdout:drwxr-xr-x root/root 0 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/video/backlight/ 2023-11-26T07:02:44.830 INFO:teuthology.orchestra.run.smithi188.stdout:-rw-r--r-- root/root 15417 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/video/backlight/88pm860x_bl.ko 2023-11-26T07:02:44.830 INFO:teuthology.orchestra.run.smithi188.stdout:-rw-r--r-- root/root 12393 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/video/backlight/aat2870_bl.ko 2023-11-26T07:02:44.830 INFO:teuthology.orchestra.run.smithi188.stdout:-rw-r--r-- root/root 23305 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/video/backlight/adp5520_bl.ko 2023-11-26T07:02:44.830 INFO:teuthology.orchestra.run.smithi188.stdout:-rw-r--r-- root/root 32609 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/video/backlight/adp8860_bl.ko 2023-11-26T07:02:44.830 INFO:teuthology.orchestra.run.smithi188.stdout:-rw-r--r-- root/root 39993 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/video/backlight/adp8870_bl.ko 2023-11-26T07:02:44.830 INFO:teuthology.orchestra.run.smithi188.stdout:-rw-r--r-- root/root 16921 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/video/backlight/apple_bl.ko 2023-11-26T07:02:44.830 INFO:teuthology.orchestra.run.smithi188.stdout:-rw-r--r-- root/root 16777 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/video/backlight/as3711_bl.ko 2023-11-26T07:02:44.830 INFO:teuthology.orchestra.run.smithi188.stdout:-rw-r--r-- root/root 12033 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/video/backlight/bd6107.ko 2023-11-26T07:02:44.830 INFO:teuthology.orchestra.run.smithi188.stdout:-rw-r--r-- root/root 11409 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/video/backlight/da903x_bl.ko 2023-11-26T07:02:44.830 INFO:teuthology.orchestra.run.smithi188.stdout:-rw-r--r-- root/root 11273 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/video/backlight/da9052_bl.ko 2023-11-26T07:02:44.830 INFO:teuthology.orchestra.run.smithi188.stdout:-rw-r--r-- root/root 11753 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/video/backlight/gpio_backlight.ko 2023-11-26T07:02:44.831 INFO:teuthology.orchestra.run.smithi188.stdout:-rw-r--r-- root/root 14769 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/video/backlight/kb3886_bl.ko 2023-11-26T07:02:44.831 INFO:teuthology.orchestra.run.smithi188.stdout:-rw-r--r-- root/root 22017 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/video/backlight/lcd.ko 2023-11-26T07:02:44.831 INFO:teuthology.orchestra.run.smithi188.stdout:-rw-r--r-- root/root 23657 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/video/backlight/lm3533_bl.ko 2023-11-26T07:02:44.831 INFO:teuthology.orchestra.run.smithi188.stdout:-rw-r--r-- root/root 19337 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/video/backlight/lm3639_bl.ko 2023-11-26T07:02:44.831 INFO:teuthology.orchestra.run.smithi188.stdout:-rw-r--r-- root/root 11425 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/video/backlight/lv5207lp.ko 2023-11-26T07:02:44.831 INFO:teuthology.orchestra.run.smithi188.stdout:-rw-r--r-- root/root 14345 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/video/backlight/max8925_bl.ko 2023-11-26T07:02:44.831 INFO:teuthology.orchestra.run.smithi188.stdout:-rw-r--r-- root/root 11313 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/video/backlight/pandora_bl.ko 2023-11-26T07:02:44.831 INFO:teuthology.orchestra.run.smithi188.stdout:-rw-r--r-- root/root 10865 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/video/backlight/pcf50633-backlight.ko 2023-11-26T07:02:44.831 INFO:teuthology.orchestra.run.smithi188.stdout:-rw-r--r-- root/root 10897 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/video/backlight/platform_lcd.ko 2023-11-26T07:02:44.831 INFO:teuthology.orchestra.run.smithi188.stdout:-rw-r--r-- root/root 12721 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/video/backlight/wm831x_bl.ko 2023-11-26T07:02:44.831 INFO:teuthology.orchestra.run.smithi188.stdout:drwxr-xr-x root/root 0 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/w1/ 2023-11-26T07:02:44.831 INFO:teuthology.orchestra.run.smithi188.stdout:drwxr-xr-x root/root 0 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/w1/masters/ 2023-11-26T07:02:44.832 INFO:teuthology.orchestra.run.smithi188.stdout:-rw-r--r-- root/root 18785 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/w1/masters/ds2482.ko 2023-11-26T07:02:44.832 INFO:teuthology.orchestra.run.smithi188.stdout:-rw-r--r-- root/root 38985 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/w1/masters/ds2490.ko 2023-11-26T07:02:44.832 INFO:teuthology.orchestra.run.smithi188.stdout:-rw-r--r-- root/root 11265 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/w1/masters/matrox_w1.ko 2023-11-26T07:02:44.832 INFO:teuthology.orchestra.run.smithi188.stdout:-rw-r--r-- root/root 12417 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/w1/masters/w1-gpio.ko 2023-11-26T07:02:44.832 INFO:teuthology.orchestra.run.smithi188.stdout:drwxr-xr-x root/root 0 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/w1/slaves/ 2023-11-26T07:02:44.832 INFO:teuthology.orchestra.run.smithi188.stdout:-rw-r--r-- root/root 12041 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/w1/slaves/w1_ds2406.ko 2023-11-26T07:02:44.832 INFO:teuthology.orchestra.run.smithi188.stdout:-rw-r--r-- root/root 22745 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/w1/slaves/w1_ds2408.ko 2023-11-26T07:02:44.832 INFO:teuthology.orchestra.run.smithi188.stdout:-rw-r--r-- root/root 15113 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/w1/slaves/w1_ds2413.ko 2023-11-26T07:02:44.832 INFO:teuthology.orchestra.run.smithi188.stdout:-rw-r--r-- root/root 10825 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/w1/slaves/w1_ds2423.ko 2023-11-26T07:02:44.832 INFO:teuthology.orchestra.run.smithi188.stdout:-rw-r--r-- root/root 14153 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/w1/slaves/w1_ds2431.ko 2023-11-26T07:02:44.832 INFO:teuthology.orchestra.run.smithi188.stdout:-rw-r--r-- root/root 11929 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/w1/slaves/w1_ds2433.ko 2023-11-26T07:02:44.833 INFO:teuthology.orchestra.run.smithi188.stdout:-rw-r--r-- root/root 14049 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/w1/slaves/w1_ds2780.ko 2023-11-26T07:02:44.833 INFO:teuthology.orchestra.run.smithi188.stdout:-rw-r--r-- root/root 14049 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/w1/slaves/w1_ds2781.ko 2023-11-26T07:02:44.833 INFO:teuthology.orchestra.run.smithi188.stdout:-rw-r--r-- root/root 17921 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/w1/slaves/w1_ds28e04.ko 2023-11-26T07:02:44.833 INFO:teuthology.orchestra.run.smithi188.stdout:-rw-r--r-- root/root 7385 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/w1/slaves/w1_smem.ko 2023-11-26T07:02:44.833 INFO:teuthology.orchestra.run.smithi188.stdout:-rw-r--r-- root/root 56425 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/w1/slaves/w1_therm.ko 2023-11-26T07:02:44.833 INFO:teuthology.orchestra.run.smithi188.stdout:-rw-r--r-- root/root 102673 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/w1/wire.ko 2023-11-26T07:02:44.833 INFO:teuthology.orchestra.run.smithi188.stdout:drwxr-xr-x root/root 0 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/watchdog/ 2023-11-26T07:02:44.833 INFO:teuthology.orchestra.run.smithi188.stdout:-rw-r--r-- root/root 18793 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/watchdog/acquirewdt.ko 2023-11-26T07:02:44.833 INFO:teuthology.orchestra.run.smithi188.stdout:-rw-r--r-- root/root 19977 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/watchdog/advantechwdt.ko 2023-11-26T07:02:44.833 INFO:teuthology.orchestra.run.smithi188.stdout:-rw-r--r-- root/root 18561 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/watchdog/alim1535_wdt.ko 2023-11-26T07:02:44.833 INFO:teuthology.orchestra.run.smithi188.stdout:-rw-r--r-- root/root 22969 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/watchdog/alim7101_wdt.ko 2023-11-26T07:02:44.833 INFO:teuthology.orchestra.run.smithi188.stdout:-rw-r--r-- root/root 19113 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/watchdog/cpu5wdt.ko 2023-11-26T07:02:44.834 INFO:teuthology.orchestra.run.smithi188.stdout:-rw-r--r-- root/root 13617 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/watchdog/da9052_wdt.ko 2023-11-26T07:02:44.834 INFO:teuthology.orchestra.run.smithi188.stdout:-rw-r--r-- root/root 12777 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/watchdog/da9055_wdt.ko 2023-11-26T07:02:44.834 INFO:teuthology.orchestra.run.smithi188.stdout:-rw-r--r-- root/root 17409 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/watchdog/da9063_wdt.ko 2023-11-26T07:02:44.834 INFO:teuthology.orchestra.run.smithi188.stdout:-rw-r--r-- root/root 20169 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/watchdog/dw_wdt.ko 2023-11-26T07:02:44.834 INFO:teuthology.orchestra.run.smithi188.stdout:-rw-r--r-- root/root 20265 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/watchdog/eurotechwdt.ko 2023-11-26T07:02:44.834 INFO:teuthology.orchestra.run.smithi188.stdout:-rw-r--r-- root/root 23865 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/watchdog/f71808e_wdt.ko 2023-11-26T07:02:44.834 INFO:teuthology.orchestra.run.smithi188.stdout:-rw-r--r-- root/root 27401 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/watchdog/hpwdt.ko 2023-11-26T07:02:44.834 INFO:teuthology.orchestra.run.smithi188.stdout:-rw-r--r-- root/root 15577 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/watchdog/i6300esb.ko 2023-11-26T07:02:44.834 INFO:teuthology.orchestra.run.smithi188.stdout:-rw-r--r-- root/root 10833 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/watchdog/iTCO_vendor_support.ko 2023-11-26T07:02:44.834 INFO:teuthology.orchestra.run.smithi188.stdout:-rw-r--r-- root/root 21913 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/watchdog/iTCO_wdt.ko 2023-11-26T07:02:44.834 INFO:teuthology.orchestra.run.smithi188.stdout:-rw-r--r-- root/root 19241 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/watchdog/ib700wdt.ko 2023-11-26T07:02:44.835 INFO:teuthology.orchestra.run.smithi188.stdout:-rw-r--r-- root/root 18689 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/watchdog/ibmasr.ko 2023-11-26T07:02:44.835 INFO:teuthology.orchestra.run.smithi188.stdout:-rw-r--r-- root/root 19713 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/watchdog/ie6xx_wdt.ko 2023-11-26T07:02:44.835 INFO:teuthology.orchestra.run.smithi188.stdout:-rw-r--r-- root/root 21225 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/watchdog/it8712f_wdt.ko 2023-11-26T07:02:44.835 INFO:teuthology.orchestra.run.smithi188.stdout:-rw-r--r-- root/root 13841 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/watchdog/it87_wdt.ko 2023-11-26T07:02:44.835 INFO:teuthology.orchestra.run.smithi188.stdout:-rw-r--r-- root/root 21489 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/watchdog/kempld_wdt.ko 2023-11-26T07:02:44.835 INFO:teuthology.orchestra.run.smithi188.stdout:-rw-r--r-- root/root 18617 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/watchdog/machzwd.ko 2023-11-26T07:02:44.835 INFO:teuthology.orchestra.run.smithi188.stdout:-rw-r--r-- root/root 14721 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/watchdog/mena21_wdt.ko 2023-11-26T07:02:44.835 INFO:teuthology.orchestra.run.smithi188.stdout:-rw-r--r-- root/root 13217 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/watchdog/menf21bmc_wdt.ko 2023-11-26T07:02:44.835 INFO:teuthology.orchestra.run.smithi188.stdout:-rw-r--r-- root/root 24673 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/watchdog/nv_tco.ko 2023-11-26T07:02:44.835 INFO:teuthology.orchestra.run.smithi188.stdout:-rw-r--r-- root/root 17737 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/watchdog/of_xilinx_wdt.ko 2023-11-26T07:02:44.835 INFO:teuthology.orchestra.run.smithi188.stdout:-rw-r--r-- root/root 22265 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/watchdog/pc87413_wdt.ko 2023-11-26T07:02:44.835 INFO:teuthology.orchestra.run.smithi188.stdout:-rw-r--r-- root/root 38657 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/watchdog/pcwd_pci.ko 2023-11-26T07:02:44.836 INFO:teuthology.orchestra.run.smithi188.stdout:-rw-r--r-- root/root 31825 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/watchdog/pcwd_usb.ko 2023-11-26T07:02:44.836 INFO:teuthology.orchestra.run.smithi188.stdout:-rw-r--r-- root/root 13065 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/watchdog/retu_wdt.ko 2023-11-26T07:02:44.836 INFO:teuthology.orchestra.run.smithi188.stdout:-rw-r--r-- root/root 20449 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/watchdog/sbc60xxwdt.ko 2023-11-26T07:02:44.836 INFO:teuthology.orchestra.run.smithi188.stdout:-rw-r--r-- root/root 14129 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/watchdog/sbc_epx_c3.ko 2023-11-26T07:02:44.836 INFO:teuthology.orchestra.run.smithi188.stdout:-rw-r--r-- root/root 17321 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/watchdog/sbc_fitpc2_wdt.ko 2023-11-26T07:02:44.836 INFO:teuthology.orchestra.run.smithi188.stdout:-rw-r--r-- root/root 23809 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/watchdog/sc1200wdt.ko 2023-11-26T07:02:44.836 INFO:teuthology.orchestra.run.smithi188.stdout:-rw-r--r-- root/root 23321 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/watchdog/sch311x_wdt.ko 2023-11-26T07:02:44.836 INFO:teuthology.orchestra.run.smithi188.stdout:-rw-r--r-- root/root 21577 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/watchdog/smsc37b787_wdt.ko 2023-11-26T07:02:44.836 INFO:teuthology.orchestra.run.smithi188.stdout:-rw-r--r-- root/root 17273 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/watchdog/softdog.ko 2023-11-26T07:02:44.836 INFO:teuthology.orchestra.run.smithi188.stdout:-rw-r--r-- root/root 22273 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/watchdog/sp5100_tco.ko 2023-11-26T07:02:44.836 INFO:teuthology.orchestra.run.smithi188.stdout:-rw-r--r-- root/root 12817 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/watchdog/twl4030_wdt.ko 2023-11-26T07:02:44.836 INFO:teuthology.orchestra.run.smithi188.stdout:-rw-r--r-- root/root 17649 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/watchdog/via_wdt.ko 2023-11-26T07:02:44.836 INFO:teuthology.orchestra.run.smithi188.stdout:-rw-r--r-- root/root 19161 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/watchdog/w83627hf_wdt.ko 2023-11-26T07:02:44.837 INFO:teuthology.orchestra.run.smithi188.stdout:-rw-r--r-- root/root 20681 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/watchdog/w83877f_wdt.ko 2023-11-26T07:02:44.837 INFO:teuthology.orchestra.run.smithi188.stdout:-rw-r--r-- root/root 25409 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/watchdog/w83977f_wdt.ko 2023-11-26T07:02:44.837 INFO:teuthology.orchestra.run.smithi188.stdout:-rw-r--r-- root/root 18497 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/watchdog/wafer5823wdt.ko 2023-11-26T07:02:44.837 INFO:teuthology.orchestra.run.smithi188.stdout:-rw-r--r-- root/root 28089 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/watchdog/wdt_pci.ko 2023-11-26T07:02:44.837 INFO:teuthology.orchestra.run.smithi188.stdout:-rw-r--r-- root/root 16257 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/watchdog/wm831x_wdt.ko 2023-11-26T07:02:44.837 INFO:teuthology.orchestra.run.smithi188.stdout:-rw-r--r-- root/root 13785 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/watchdog/wm8350_wdt.ko 2023-11-26T07:02:44.837 INFO:teuthology.orchestra.run.smithi188.stdout:-rw-r--r-- root/root 15697 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/watchdog/xen_wdt.ko 2023-11-26T07:02:44.837 INFO:teuthology.orchestra.run.smithi188.stdout:drwxr-xr-x root/root 0 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/xen/ 2023-11-26T07:02:44.837 INFO:teuthology.orchestra.run.smithi188.stdout:-rw-r--r-- root/root 24729 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/xen/xen-evtchn.ko 2023-11-26T07:02:44.837 INFO:teuthology.orchestra.run.smithi188.stdout:-rw-r--r-- root/root 26769 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/xen/xen-gntalloc.ko 2023-11-26T07:02:44.838 INFO:teuthology.orchestra.run.smithi188.stdout:-rw-r--r-- root/root 47865 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/xen/xen-gntdev.ko 2023-11-26T07:02:44.838 INFO:teuthology.orchestra.run.smithi188.stdout:drwxr-xr-x root/root 0 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/xen/xen-pciback/ 2023-11-26T07:02:44.838 INFO:teuthology.orchestra.run.smithi188.stdout:-rw-r--r-- root/root 176377 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/xen/xen-pciback/xen-pciback.ko 2023-11-26T07:02:44.838 INFO:teuthology.orchestra.run.smithi188.stdout:-rw-r--r-- root/root 40121 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/xen/xen-privcmd.ko 2023-11-26T07:02:44.838 INFO:teuthology.orchestra.run.smithi188.stdout:-rw-r--r-- root/root 71249 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/xen/xen-scsiback.ko 2023-11-26T07:02:44.838 INFO:teuthology.orchestra.run.smithi188.stdout:drwxr-xr-x root/root 0 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/xen/xenfs/ 2023-11-26T07:02:44.838 INFO:teuthology.orchestra.run.smithi188.stdout:-rw-r--r-- root/root 19425 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/xen/xenfs/xenfs.ko 2023-11-26T07:02:44.838 INFO:teuthology.orchestra.run.smithi188.stdout:drwxr-xr-x root/root 0 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/fs/ 2023-11-26T07:02:44.838 INFO:teuthology.orchestra.run.smithi188.stdout:drwxr-xr-x root/root 0 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/fs/9p/ 2023-11-26T07:02:44.838 INFO:teuthology.orchestra.run.smithi188.stdout:-rw-r--r-- root/root 166425 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/fs/9p/9p.ko 2023-11-26T07:02:44.838 INFO:teuthology.orchestra.run.smithi188.stdout:drwxr-xr-x root/root 0 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/fs/adfs/ 2023-11-26T07:02:44.838 INFO:teuthology.orchestra.run.smithi188.stdout:-rw-r--r-- root/root 61073 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/fs/adfs/adfs.ko 2023-11-26T07:02:44.839 INFO:teuthology.orchestra.run.smithi188.stdout:drwxr-xr-x root/root 0 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/fs/affs/ 2023-11-26T07:02:44.839 INFO:teuthology.orchestra.run.smithi188.stdout:-rw-r--r-- root/root 163593 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/fs/affs/affs.ko 2023-11-26T07:02:44.839 INFO:teuthology.orchestra.run.smithi188.stdout:drwxr-xr-x root/root 0 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/fs/afs/ 2023-11-26T07:02:44.839 INFO:teuthology.orchestra.run.smithi188.stdout:-rw-r--r-- root/root 932817 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/fs/afs/kafs.ko 2023-11-26T07:02:44.839 INFO:teuthology.orchestra.run.smithi188.stdout:drwxr-xr-x root/root 0 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/fs/autofs/ 2023-11-26T07:02:44.839 INFO:teuthology.orchestra.run.smithi188.stdout:-rw-r--r-- root/root 101057 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/fs/autofs/autofs4.ko 2023-11-26T07:02:44.839 INFO:teuthology.orchestra.run.smithi188.stdout:drwxr-xr-x root/root 0 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/fs/befs/ 2023-11-26T07:02:44.839 INFO:teuthology.orchestra.run.smithi188.stdout:-rw-r--r-- root/root 64545 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/fs/befs/befs.ko 2023-11-26T07:02:44.839 INFO:teuthology.orchestra.run.smithi188.stdout:drwxr-xr-x root/root 0 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/fs/bfs/ 2023-11-26T07:02:44.839 INFO:teuthology.orchestra.run.smithi188.stdout:-rw-r--r-- root/root 41257 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/fs/bfs/bfs.ko 2023-11-26T07:02:44.839 INFO:teuthology.orchestra.run.smithi188.stdout:-rw-r--r-- root/root 34585 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/fs/binfmt_misc.ko 2023-11-26T07:02:44.839 INFO:teuthology.orchestra.run.smithi188.stdout:drwxr-xr-x root/root 0 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/fs/btrfs/ 2023-11-26T07:02:44.839 INFO:teuthology.orchestra.run.smithi188.stdout:-rw-r--r-- root/root 3402729 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/fs/btrfs/btrfs.ko 2023-11-26T07:02:44.839 INFO:teuthology.orchestra.run.smithi188.stdout:drwxr-xr-x root/root 0 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/fs/cachefiles/ 2023-11-26T07:02:44.840 INFO:teuthology.orchestra.run.smithi188.stdout:-rw-r--r-- root/root 306009 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/fs/cachefiles/cachefiles.ko 2023-11-26T07:02:44.840 INFO:teuthology.orchestra.run.smithi188.stdout:drwxr-xr-x root/root 0 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/fs/ceph/ 2023-11-26T07:02:44.840 INFO:teuthology.orchestra.run.smithi188.stdout:-rw-r--r-- root/root 1333017 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/fs/ceph/ceph.ko 2023-11-26T07:02:44.840 INFO:teuthology.orchestra.run.smithi188.stdout:drwxr-xr-x root/root 0 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/fs/coda/ 2023-11-26T07:02:44.840 INFO:teuthology.orchestra.run.smithi188.stdout:-rw-r--r-- root/root 100801 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/fs/coda/coda.ko 2023-11-26T07:02:44.840 INFO:teuthology.orchestra.run.smithi188.stdout:drwxr-xr-x root/root 0 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/fs/configfs/ 2023-11-26T07:02:44.840 INFO:teuthology.orchestra.run.smithi188.stdout:-rw-r--r-- root/root 108393 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/fs/configfs/configfs.ko 2023-11-26T07:02:44.840 INFO:teuthology.orchestra.run.smithi188.stdout:drwxr-xr-x root/root 0 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/fs/cramfs/ 2023-11-26T07:02:44.840 INFO:teuthology.orchestra.run.smithi188.stdout:-rw-r--r-- root/root 29625 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/fs/cramfs/cramfs.ko 2023-11-26T07:02:44.840 INFO:teuthology.orchestra.run.smithi188.stdout:drwxr-xr-x root/root 0 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/fs/dlm/ 2023-11-26T07:02:44.840 INFO:teuthology.orchestra.run.smithi188.stdout:-rw-r--r-- root/root 642513 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/fs/dlm/dlm.ko 2023-11-26T07:02:44.840 INFO:teuthology.orchestra.run.smithi188.stdout:drwxr-xr-x root/root 0 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/fs/efs/ 2023-11-26T07:02:44.841 INFO:teuthology.orchestra.run.smithi188.stdout:-rw-r--r-- root/root 36649 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/fs/efs/efs.ko 2023-11-26T07:02:44.841 INFO:teuthology.orchestra.run.smithi188.stdout:drwxr-xr-x root/root 0 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/fs/f2fs/ 2023-11-26T07:02:44.841 INFO:teuthology.orchestra.run.smithi188.stdout:-rw-r--r-- root/root 1505025 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/fs/f2fs/f2fs.ko 2023-11-26T07:02:44.841 INFO:teuthology.orchestra.run.smithi188.stdout:drwxr-xr-x root/root 0 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/fs/fat/ 2023-11-26T07:02:44.841 INFO:teuthology.orchestra.run.smithi188.stdout:-rw-r--r-- root/root 27929 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/fs/fat/msdos.ko 2023-11-26T07:02:44.841 INFO:teuthology.orchestra.run.smithi188.stdout:drwxr-xr-x root/root 0 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/fs/freevxfs/ 2023-11-26T07:02:44.841 INFO:teuthology.orchestra.run.smithi188.stdout:-rw-r--r-- root/root 42545 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/fs/freevxfs/freevxfs.ko 2023-11-26T07:02:44.841 INFO:teuthology.orchestra.run.smithi188.stdout:drwxr-xr-x root/root 0 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/fs/fscache/ 2023-11-26T07:02:44.841 INFO:teuthology.orchestra.run.smithi188.stdout:-rw-r--r-- root/root 226481 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/fs/fscache/fscache.ko 2023-11-26T07:02:44.841 INFO:teuthology.orchestra.run.smithi188.stdout:drwxr-xr-x root/root 0 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/fs/fuse/ 2023-11-26T07:02:44.842 INFO:teuthology.orchestra.run.smithi188.stdout:-rw-r--r-- root/root 24489 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/fs/fuse/cuse.ko 2023-11-26T07:02:44.842 INFO:teuthology.orchestra.run.smithi188.stdout:drwxr-xr-x root/root 0 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/fs/gfs2/ 2023-11-26T07:02:44.842 INFO:teuthology.orchestra.run.smithi188.stdout:-rw-r--r-- root/root 1231529 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/fs/gfs2/gfs2.ko 2023-11-26T07:02:44.842 INFO:teuthology.orchestra.run.smithi188.stdout:drwxr-xr-x root/root 0 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/fs/hfs/ 2023-11-26T07:02:44.842 INFO:teuthology.orchestra.run.smithi188.stdout:-rw-r--r-- root/root 141209 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/fs/hfs/hfs.ko 2023-11-26T07:02:44.842 INFO:teuthology.orchestra.run.smithi188.stdout:drwxr-xr-x root/root 0 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/fs/hfsplus/ 2023-11-26T07:02:44.842 INFO:teuthology.orchestra.run.smithi188.stdout:-rw-r--r-- root/root 222817 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/fs/hfsplus/hfsplus.ko 2023-11-26T07:02:44.842 INFO:teuthology.orchestra.run.smithi188.stdout:drwxr-xr-x root/root 0 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/fs/hpfs/ 2023-11-26T07:02:44.842 INFO:teuthology.orchestra.run.smithi188.stdout:-rw-r--r-- root/root 192329 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/fs/hpfs/hpfs.ko 2023-11-26T07:02:44.842 INFO:teuthology.orchestra.run.smithi188.stdout:drwxr-xr-x root/root 0 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/fs/isofs/ 2023-11-26T07:02:44.842 INFO:teuthology.orchestra.run.smithi188.stdout:-rw-r--r-- root/root 74065 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/fs/isofs/isofs.ko 2023-11-26T07:02:44.843 INFO:teuthology.orchestra.run.smithi188.stdout:drwxr-xr-x root/root 0 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/fs/jfs/ 2023-11-26T07:02:44.843 INFO:teuthology.orchestra.run.smithi188.stdout:-rw-r--r-- root/root 419497 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/fs/jfs/jfs.ko 2023-11-26T07:02:44.843 INFO:teuthology.orchestra.run.smithi188.stdout:drwxr-xr-x root/root 0 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/fs/lockd/ 2023-11-26T07:02:44.843 INFO:teuthology.orchestra.run.smithi188.stdout:-rw-r--r-- root/root 280313 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/fs/lockd/lockd.ko 2023-11-26T07:02:44.843 INFO:teuthology.orchestra.run.smithi188.stdout:drwxr-xr-x root/root 0 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/fs/minix/ 2023-11-26T07:02:44.843 INFO:teuthology.orchestra.run.smithi188.stdout:-rw-r--r-- root/root 89193 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/fs/minix/minix.ko 2023-11-26T07:02:44.843 INFO:teuthology.orchestra.run.smithi188.stdout:drwxr-xr-x root/root 0 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/fs/netfs/ 2023-11-26T07:02:44.843 INFO:teuthology.orchestra.run.smithi188.stdout:-rw-r--r-- root/root 124649 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/fs/netfs/netfs.ko 2023-11-26T07:02:44.843 INFO:teuthology.orchestra.run.smithi188.stdout:drwxr-xr-x root/root 0 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/fs/nfs/ 2023-11-26T07:02:44.843 INFO:teuthology.orchestra.run.smithi188.stdout:drwxr-xr-x root/root 0 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/fs/nfs/blocklayout/ 2023-11-26T07:02:44.843 INFO:teuthology.orchestra.run.smithi188.stdout:-rw-r--r-- root/root 81449 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/fs/nfs/blocklayout/blocklayoutdriver.ko 2023-11-26T07:02:44.843 INFO:teuthology.orchestra.run.smithi188.stdout:drwxr-xr-x root/root 0 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/fs/nfs/filelayout/ 2023-11-26T07:02:44.844 INFO:teuthology.orchestra.run.smithi188.stdout:-rw-r--r-- root/root 59449 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/fs/nfs/filelayout/nfs_layout_nfsv41_files.ko 2023-11-26T07:02:44.844 INFO:teuthology.orchestra.run.smithi188.stdout:drwxr-xr-x root/root 0 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/fs/nfs/flexfilelayout/ 2023-11-26T07:02:44.844 INFO:teuthology.orchestra.run.smithi188.stdout:-rw-r--r-- root/root 117465 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/fs/nfs/flexfilelayout/nfs_layout_flexfiles.ko 2023-11-26T07:02:44.844 INFO:teuthology.orchestra.run.smithi188.stdout:-rw-r--r-- root/root 1055721 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/fs/nfs/nfs.ko 2023-11-26T07:02:44.844 INFO:teuthology.orchestra.run.smithi188.stdout:-rw-r--r-- root/root 66441 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/fs/nfs/nfsv2.ko 2023-11-26T07:02:44.844 INFO:teuthology.orchestra.run.smithi188.stdout:-rw-r--r-- root/root 120177 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/fs/nfs/nfsv3.ko 2023-11-26T07:02:44.844 INFO:teuthology.orchestra.run.smithi188.stdout:-rw-r--r-- root/root 2042137 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/fs/nfs/nfsv4.ko 2023-11-26T07:02:44.844 INFO:teuthology.orchestra.run.smithi188.stdout:drwxr-xr-x root/root 0 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/fs/nfs_common/ 2023-11-26T07:02:44.844 INFO:teuthology.orchestra.run.smithi188.stdout:-rw-r--r-- root/root 16929 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/fs/nfs_common/grace.ko 2023-11-26T07:02:44.844 INFO:teuthology.orchestra.run.smithi188.stdout:-rw-r--r-- root/root 16145 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/fs/nfs_common/nfs_acl.ko 2023-11-26T07:02:44.844 INFO:teuthology.orchestra.run.smithi188.stdout:drwxr-xr-x root/root 0 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/fs/nfsd/ 2023-11-26T07:02:44.845 INFO:teuthology.orchestra.run.smithi188.stdout:-rw-r--r-- root/root 1516873 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/fs/nfsd/nfsd.ko 2023-11-26T07:02:44.845 INFO:teuthology.orchestra.run.smithi188.stdout:drwxr-xr-x root/root 0 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/fs/nilfs2/ 2023-11-26T07:02:44.845 INFO:teuthology.orchestra.run.smithi188.stdout:-rw-r--r-- root/root 592833 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/fs/nilfs2/nilfs2.ko 2023-11-26T07:02:44.845 INFO:teuthology.orchestra.run.smithi188.stdout:drwxr-xr-x root/root 0 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/fs/nls/ 2023-11-26T07:02:44.845 INFO:teuthology.orchestra.run.smithi188.stdout:-rw-r--r-- root/root 13793 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/fs/nls/mac-celtic.ko 2023-11-26T07:02:44.845 INFO:teuthology.orchestra.run.smithi188.stdout:-rw-r--r-- root/root 13169 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/fs/nls/mac-centeuro.ko 2023-11-26T07:02:44.845 INFO:teuthology.orchestra.run.smithi188.stdout:-rw-r--r-- root/root 13793 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/fs/nls/mac-croatian.ko 2023-11-26T07:02:44.845 INFO:teuthology.orchestra.run.smithi188.stdout:-rw-r--r-- root/root 12865 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/fs/nls/mac-cyrillic.ko 2023-11-26T07:02:44.845 INFO:teuthology.orchestra.run.smithi188.stdout:-rw-r--r-- root/root 13481 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/fs/nls/mac-gaelic.ko 2023-11-26T07:02:44.845 INFO:teuthology.orchestra.run.smithi188.stdout:-rw-r--r-- root/root 12857 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/fs/nls/mac-greek.ko 2023-11-26T07:02:44.845 INFO:teuthology.orchestra.run.smithi188.stdout:-rw-r--r-- root/root 13793 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/fs/nls/mac-iceland.ko 2023-11-26T07:02:44.845 INFO:teuthology.orchestra.run.smithi188.stdout:-rw-r--r-- root/root 13169 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/fs/nls/mac-inuit.ko 2023-11-26T07:02:44.846 INFO:teuthology.orchestra.run.smithi188.stdout:-rw-r--r-- root/root 14097 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/fs/nls/mac-roman.ko 2023-11-26T07:02:44.846 INFO:teuthology.orchestra.run.smithi188.stdout:-rw-r--r-- root/root 13793 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/fs/nls/mac-romanian.ko 2023-11-26T07:02:44.846 INFO:teuthology.orchestra.run.smithi188.stdout:-rw-r--r-- root/root 13793 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/fs/nls/mac-turkish.ko 2023-11-26T07:02:44.846 INFO:teuthology.orchestra.run.smithi188.stdout:-rw-r--r-- root/root 11297 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/fs/nls/nls_ascii.ko 2023-11-26T07:02:44.846 INFO:teuthology.orchestra.run.smithi188.stdout:-rw-r--r-- root/root 12537 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/fs/nls/nls_cp1250.ko 2023-11-26T07:02:44.846 INFO:teuthology.orchestra.run.smithi188.stdout:-rw-r--r-- root/root 12233 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/fs/nls/nls_cp1251.ko 2023-11-26T07:02:44.846 INFO:teuthology.orchestra.run.smithi188.stdout:-rw-r--r-- root/root 12921 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/fs/nls/nls_cp1255.ko 2023-11-26T07:02:44.846 INFO:teuthology.orchestra.run.smithi188.stdout:-rw-r--r-- root/root 12537 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/fs/nls/nls_cp737.ko 2023-11-26T07:02:44.846 INFO:teuthology.orchestra.run.smithi188.stdout:-rw-r--r-- root/root 12537 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/fs/nls/nls_cp775.ko 2023-11-26T07:02:44.846 INFO:teuthology.orchestra.run.smithi188.stdout:-rw-r--r-- root/root 12225 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/fs/nls/nls_cp850.ko 2023-11-26T07:02:44.846 INFO:teuthology.orchestra.run.smithi188.stdout:-rw-r--r-- root/root 12225 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/fs/nls/nls_cp852.ko 2023-11-26T07:02:44.847 INFO:teuthology.orchestra.run.smithi188.stdout:-rw-r--r-- root/root 12225 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/fs/nls/nls_cp855.ko 2023-11-26T07:02:44.847 INFO:teuthology.orchestra.run.smithi188.stdout:-rw-r--r-- root/root 11913 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/fs/nls/nls_cp857.ko 2023-11-26T07:02:44.847 INFO:teuthology.orchestra.run.smithi188.stdout:-rw-r--r-- root/root 12849 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/fs/nls/nls_cp860.ko 2023-11-26T07:02:44.847 INFO:teuthology.orchestra.run.smithi188.stdout:-rw-r--r-- root/root 13161 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/fs/nls/nls_cp861.ko 2023-11-26T07:02:44.847 INFO:teuthology.orchestra.run.smithi188.stdout:-rw-r--r-- root/root 13473 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/fs/nls/nls_cp862.ko 2023-11-26T07:02:44.847 INFO:teuthology.orchestra.run.smithi188.stdout:-rw-r--r-- root/root 13161 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/fs/nls/nls_cp863.ko 2023-11-26T07:02:44.847 INFO:teuthology.orchestra.run.smithi188.stdout:-rw-r--r-- root/root 12849 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/fs/nls/nls_cp864.ko 2023-11-26T07:02:44.847 INFO:teuthology.orchestra.run.smithi188.stdout:-rw-r--r-- root/root 13161 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/fs/nls/nls_cp865.ko 2023-11-26T07:02:44.847 INFO:teuthology.orchestra.run.smithi188.stdout:-rw-r--r-- root/root 12537 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/fs/nls/nls_cp866.ko 2023-11-26T07:02:44.847 INFO:teuthology.orchestra.run.smithi188.stdout:-rw-r--r-- root/root 12225 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/fs/nls/nls_cp869.ko 2023-11-26T07:02:44.847 INFO:teuthology.orchestra.run.smithi188.stdout:-rw-r--r-- root/root 11985 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/fs/nls/nls_cp874.ko 2023-11-26T07:02:44.847 INFO:teuthology.orchestra.run.smithi188.stdout:-rw-r--r-- root/root 94137 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/fs/nls/nls_cp932.ko 2023-11-26T07:02:44.848 INFO:teuthology.orchestra.run.smithi188.stdout:-rw-r--r-- root/root 142673 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/fs/nls/nls_cp936.ko 2023-11-26T07:02:44.848 INFO:teuthology.orchestra.run.smithi188.stdout:-rw-r--r-- root/root 165745 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/fs/nls/nls_cp949.ko 2023-11-26T07:02:44.848 INFO:teuthology.orchestra.run.smithi188.stdout:-rw-r--r-- root/root 118121 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/fs/nls/nls_cp950.ko 2023-11-26T07:02:44.848 INFO:teuthology.orchestra.run.smithi188.stdout:-rw-r--r-- root/root 12457 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/fs/nls/nls_euc-jp.ko 2023-11-26T07:02:44.848 INFO:teuthology.orchestra.run.smithi188.stdout:-rw-r--r-- root/root 11305 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/fs/nls/nls_iso8859-1.ko 2023-11-26T07:02:44.848 INFO:teuthology.orchestra.run.smithi188.stdout:-rw-r--r-- root/root 11929 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/fs/nls/nls_iso8859-13.ko 2023-11-26T07:02:44.848 INFO:teuthology.orchestra.run.smithi188.stdout:-rw-r--r-- root/root 11929 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/fs/nls/nls_iso8859-14.ko 2023-11-26T07:02:44.848 INFO:teuthology.orchestra.run.smithi188.stdout:-rw-r--r-- root/root 11929 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/fs/nls/nls_iso8859-15.ko 2023-11-26T07:02:44.848 INFO:teuthology.orchestra.run.smithi188.stdout:-rw-r--r-- root/root 11921 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/fs/nls/nls_iso8859-2.ko 2023-11-26T07:02:44.848 INFO:teuthology.orchestra.run.smithi188.stdout:-rw-r--r-- root/root 11921 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/fs/nls/nls_iso8859-3.ko 2023-11-26T07:02:44.848 INFO:teuthology.orchestra.run.smithi188.stdout:-rw-r--r-- root/root 11921 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/fs/nls/nls_iso8859-4.ko 2023-11-26T07:02:44.848 INFO:teuthology.orchestra.run.smithi188.stdout:-rw-r--r-- root/root 11921 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/fs/nls/nls_iso8859-5.ko 2023-11-26T07:02:44.848 INFO:teuthology.orchestra.run.smithi188.stdout:-rw-r--r-- root/root 11617 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/fs/nls/nls_iso8859-6.ko 2023-11-26T07:02:44.849 INFO:teuthology.orchestra.run.smithi188.stdout:-rw-r--r-- root/root 12233 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/fs/nls/nls_iso8859-7.ko 2023-11-26T07:02:44.849 INFO:teuthology.orchestra.run.smithi188.stdout:-rw-r--r-- root/root 11617 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/fs/nls/nls_iso8859-9.ko 2023-11-26T07:02:44.849 INFO:teuthology.orchestra.run.smithi188.stdout:-rw-r--r-- root/root 12537 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/fs/nls/nls_koi8-r.ko 2023-11-26T07:02:44.849 INFO:teuthology.orchestra.run.smithi188.stdout:-rw-r--r-- root/root 8017 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/fs/nls/nls_koi8-ru.ko 2023-11-26T07:02:44.849 INFO:teuthology.orchestra.run.smithi188.stdout:-rw-r--r-- root/root 12537 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/fs/nls/nls_koi8-u.ko 2023-11-26T07:02:44.849 INFO:teuthology.orchestra.run.smithi188.stdout:-rw-r--r-- root/root 7569 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/fs/nls/nls_ucs2_utils.ko 2023-11-26T07:02:44.849 INFO:teuthology.orchestra.run.smithi188.stdout:-rw-r--r-- root/root 8337 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/fs/nls/nls_utf8.ko 2023-11-26T07:02:44.849 INFO:teuthology.orchestra.run.smithi188.stdout:drwxr-xr-x root/root 0 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/fs/ntfs/ 2023-11-26T07:02:44.849 INFO:teuthology.orchestra.run.smithi188.stdout:-rw-r--r-- root/root 219417 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/fs/ntfs/ntfs.ko 2023-11-26T07:02:44.849 INFO:teuthology.orchestra.run.smithi188.stdout:drwxr-xr-x root/root 0 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/fs/ocfs2/ 2023-11-26T07:02:44.849 INFO:teuthology.orchestra.run.smithi188.stdout:drwxr-xr-x root/root 0 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/fs/ocfs2/cluster/ 2023-11-26T07:02:44.849 INFO:teuthology.orchestra.run.smithi188.stdout:-rw-r--r-- root/root 244409 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/fs/ocfs2/cluster/ocfs2_nodemanager.ko 2023-11-26T07:02:44.849 INFO:teuthology.orchestra.run.smithi188.stdout:drwxr-xr-x root/root 0 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/fs/ocfs2/dlm/ 2023-11-26T07:02:44.850 INFO:teuthology.orchestra.run.smithi188.stdout:-rw-r--r-- root/root 440737 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/fs/ocfs2/dlm/ocfs2_dlm.ko 2023-11-26T07:02:44.850 INFO:teuthology.orchestra.run.smithi188.stdout:drwxr-xr-x root/root 0 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/fs/ocfs2/dlmfs/ 2023-11-26T07:02:44.850 INFO:teuthology.orchestra.run.smithi188.stdout:-rw-r--r-- root/root 54049 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/fs/ocfs2/dlmfs/ocfs2_dlmfs.ko 2023-11-26T07:02:44.850 INFO:teuthology.orchestra.run.smithi188.stdout:-rw-r--r-- root/root 2592433 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/fs/ocfs2/ocfs2.ko 2023-11-26T07:02:44.850 INFO:teuthology.orchestra.run.smithi188.stdout:-rw-r--r-- root/root 17905 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/fs/ocfs2/ocfs2_stack_o2cb.ko 2023-11-26T07:02:44.850 INFO:teuthology.orchestra.run.smithi188.stdout:-rw-r--r-- root/root 31729 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/fs/ocfs2/ocfs2_stack_user.ko 2023-11-26T07:02:44.850 INFO:teuthology.orchestra.run.smithi188.stdout:-rw-r--r-- root/root 35897 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/fs/ocfs2/ocfs2_stackglue.ko 2023-11-26T07:02:44.850 INFO:teuthology.orchestra.run.smithi188.stdout:drwxr-xr-x root/root 0 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/fs/omfs/ 2023-11-26T07:02:44.850 INFO:teuthology.orchestra.run.smithi188.stdout:-rw-r--r-- root/root 53433 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/fs/omfs/omfs.ko 2023-11-26T07:02:44.850 INFO:teuthology.orchestra.run.smithi188.stdout:drwxr-xr-x root/root 0 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/fs/overlayfs/ 2023-11-26T07:02:44.850 INFO:teuthology.orchestra.run.smithi188.stdout:-rw-r--r-- root/root 352161 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/fs/overlayfs/overlay.ko 2023-11-26T07:02:44.851 INFO:teuthology.orchestra.run.smithi188.stdout:drwxr-xr-x root/root 0 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/fs/pstore/ 2023-11-26T07:02:44.851 INFO:teuthology.orchestra.run.smithi188.stdout:-rw-r--r-- root/root 54809 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/fs/pstore/ramoops.ko 2023-11-26T07:02:44.851 INFO:teuthology.orchestra.run.smithi188.stdout:drwxr-xr-x root/root 0 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/fs/qnx4/ 2023-11-26T07:02:44.851 INFO:teuthology.orchestra.run.smithi188.stdout:-rw-r--r-- root/root 26865 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/fs/qnx4/qnx4.ko 2023-11-26T07:02:44.851 INFO:teuthology.orchestra.run.smithi188.stdout:drwxr-xr-x root/root 0 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/fs/qnx6/ 2023-11-26T07:02:44.851 INFO:teuthology.orchestra.run.smithi188.stdout:-rw-r--r-- root/root 46689 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/fs/qnx6/qnx6.ko 2023-11-26T07:02:44.851 INFO:teuthology.orchestra.run.smithi188.stdout:drwxr-xr-x root/root 0 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/fs/quota/ 2023-11-26T07:02:44.851 INFO:teuthology.orchestra.run.smithi188.stdout:-rw-r--r-- root/root 30881 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/fs/quota/quota_tree.ko 2023-11-26T07:02:44.851 INFO:teuthology.orchestra.run.smithi188.stdout:-rw-r--r-- root/root 14297 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/fs/quota/quota_v1.ko 2023-11-26T07:02:44.851 INFO:teuthology.orchestra.run.smithi188.stdout:-rw-r--r-- root/root 22017 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/fs/quota/quota_v2.ko 2023-11-26T07:02:44.851 INFO:teuthology.orchestra.run.smithi188.stdout:drwxr-xr-x root/root 0 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/fs/reiserfs/ 2023-11-26T07:02:44.851 INFO:teuthology.orchestra.run.smithi188.stdout:-rw-r--r-- root/root 532673 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/fs/reiserfs/reiserfs.ko 2023-11-26T07:02:44.852 INFO:teuthology.orchestra.run.smithi188.stdout:drwxr-xr-x root/root 0 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/fs/romfs/ 2023-11-26T07:02:44.852 INFO:teuthology.orchestra.run.smithi188.stdout:-rw-r--r-- root/root 28641 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/fs/romfs/romfs.ko 2023-11-26T07:02:44.852 INFO:teuthology.orchestra.run.smithi188.stdout:drwxr-xr-x root/root 0 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/fs/smb/ 2023-11-26T07:02:44.852 INFO:teuthology.orchestra.run.smithi188.stdout:drwxr-xr-x root/root 0 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/fs/smb/client/ 2023-11-26T07:02:44.852 INFO:teuthology.orchestra.run.smithi188.stdout:-rw-r--r-- root/root 2668401 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/fs/smb/client/cifs.ko 2023-11-26T07:02:44.885 INFO:teuthology.orchestra.run.smithi188.stdout:drwxr-xr-x root/root 0 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/fs/smb/common/ 2023-11-26T07:02:44.885 INFO:teuthology.orchestra.run.smithi188.stdout:-rw-r--r-- root/root 7161 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/fs/smb/common/cifs_arc4.ko 2023-11-26T07:02:44.885 INFO:teuthology.orchestra.run.smithi188.stdout:-rw-r--r-- root/root 11257 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/fs/smb/common/cifs_md4.ko 2023-11-26T07:02:44.885 INFO:teuthology.orchestra.run.smithi188.stdout:drwxr-xr-x root/root 0 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/fs/squashfs/ 2023-11-26T07:02:44.885 INFO:teuthology.orchestra.run.smithi188.stdout:-rw-r--r-- root/root 134641 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/fs/squashfs/squashfs.ko 2023-11-26T07:02:44.885 INFO:teuthology.orchestra.run.smithi188.stdout:drwxr-xr-x root/root 0 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/fs/sysv/ 2023-11-26T07:02:44.885 INFO:teuthology.orchestra.run.smithi188.stdout:-rw-r--r-- root/root 90721 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/fs/sysv/sysv.ko 2023-11-26T07:02:44.885 INFO:teuthology.orchestra.run.smithi188.stdout:drwxr-xr-x root/root 0 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/fs/udf/ 2023-11-26T07:02:44.886 INFO:teuthology.orchestra.run.smithi188.stdout:-rw-r--r-- root/root 247737 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/fs/udf/udf.ko 2023-11-26T07:02:44.886 INFO:teuthology.orchestra.run.smithi188.stdout:drwxr-xr-x root/root 0 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/fs/ufs/ 2023-11-26T07:02:44.886 INFO:teuthology.orchestra.run.smithi188.stdout:-rw-r--r-- root/root 162497 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/fs/ufs/ufs.ko 2023-11-26T07:02:44.886 INFO:teuthology.orchestra.run.smithi188.stdout:drwxr-xr-x root/root 0 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/fs/xfs/ 2023-11-26T07:02:44.886 INFO:teuthology.orchestra.run.smithi188.stdout:-rw-r--r-- root/root 4011017 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/fs/xfs/xfs.ko 2023-11-26T07:02:44.886 INFO:teuthology.orchestra.run.smithi188.stdout:drwxr-xr-x root/root 0 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/kernel/ 2023-11-26T07:02:44.886 INFO:teuthology.orchestra.run.smithi188.stdout:drwxr-xr-x root/root 0 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/kernel/locking/ 2023-11-26T07:02:44.886 INFO:teuthology.orchestra.run.smithi188.stdout:-rw-r--r-- root/root 75489 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/kernel/locking/locktorture.ko 2023-11-26T07:02:44.886 INFO:teuthology.orchestra.run.smithi188.stdout:drwxr-xr-x root/root 0 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/kernel/time/ 2023-11-26T07:02:44.886 INFO:teuthology.orchestra.run.smithi188.stdout:-rw-r--r-- root/root 13737 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/kernel/time/test_udelay.ko 2023-11-26T07:02:44.886 INFO:teuthology.orchestra.run.smithi188.stdout:-rw-r--r-- root/root 60553 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/kernel/torture.ko 2023-11-26T07:02:44.886 INFO:teuthology.orchestra.run.smithi188.stdout:drwxr-xr-x root/root 0 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/lib/ 2023-11-26T07:02:44.887 INFO:teuthology.orchestra.run.smithi188.stdout:-rw-r--r-- root/root 6833 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/lib/crc-itu-t.ko 2023-11-26T07:02:44.887 INFO:teuthology.orchestra.run.smithi188.stdout:-rw-r--r-- root/root 6529 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/lib/crc7.ko 2023-11-26T07:02:44.887 INFO:teuthology.orchestra.run.smithi188.stdout:-rw-r--r-- root/root 7025 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/lib/crc8.ko 2023-11-26T07:02:44.887 INFO:teuthology.orchestra.run.smithi188.stdout:drwxr-xr-x root/root 0 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/lib/crypto/ 2023-11-26T07:02:44.887 INFO:teuthology.orchestra.run.smithi188.stdout:-rw-r--r-- root/root 7097 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/lib/crypto/libarc4.ko 2023-11-26T07:02:44.887 INFO:teuthology.orchestra.run.smithi188.stdout:-rw-r--r-- root/root 37081 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/lib/crypto/libdes.ko 2023-11-26T07:02:44.887 INFO:teuthology.orchestra.run.smithi188.stdout:-rw-r--r-- root/root 13313 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/lib/interval_tree_test.ko 2023-11-26T07:02:44.887 INFO:teuthology.orchestra.run.smithi188.stdout:-rw-r--r-- root/root 8425 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/lib/libcrc32c.ko 2023-11-26T07:02:44.887 INFO:teuthology.orchestra.run.smithi188.stdout:-rw-r--r-- root/root 24953 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/lib/lru_cache.ko 2023-11-26T07:02:44.887 INFO:teuthology.orchestra.run.smithi188.stdout:drwxr-xr-x root/root 0 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/lib/lz4/ 2023-11-26T07:02:44.887 INFO:teuthology.orchestra.run.smithi188.stdout:-rw-r--r-- root/root 33801 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/lib/lz4/lz4_compress.ko 2023-11-26T07:02:44.888 INFO:teuthology.orchestra.run.smithi188.stdout:-rw-r--r-- root/root 22633 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/lib/lz4/lz4hc_compress.ko 2023-11-26T07:02:44.888 INFO:teuthology.orchestra.run.smithi188.stdout:drwxr-xr-x root/root 0 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/lib/math/ 2023-11-26T07:02:44.888 INFO:teuthology.orchestra.run.smithi188.stdout:-rw-r--r-- root/root 6617 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/lib/math/cordic.ko 2023-11-26T07:02:44.888 INFO:teuthology.orchestra.run.smithi188.stdout:-rw-r--r-- root/root 8233 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/lib/memory-notifier-error-inject.ko 2023-11-26T07:02:44.888 INFO:teuthology.orchestra.run.smithi188.stdout:-rw-r--r-- root/root 10865 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/lib/notifier-error-inject.ko 2023-11-26T07:02:44.888 INFO:teuthology.orchestra.run.smithi188.stdout:-rw-r--r-- root/root 14089 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/lib/percpu_test.ko 2023-11-26T07:02:44.888 INFO:teuthology.orchestra.run.smithi188.stdout:-rw-r--r-- root/root 8305 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/lib/pm-notifier-error-inject.ko 2023-11-26T07:02:44.888 INFO:teuthology.orchestra.run.smithi188.stdout:drwxr-xr-x root/root 0 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/lib/raid6/ 2023-11-26T07:02:44.888 INFO:teuthology.orchestra.run.smithi188.stdout:-rw-r--r-- root/root 143433 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/lib/raid6/raid6_pq.ko 2023-11-26T07:02:44.888 INFO:teuthology.orchestra.run.smithi188.stdout:-rw-r--r-- root/root 21105 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/lib/rbtree_test.ko 2023-11-26T07:02:44.888 INFO:teuthology.orchestra.run.smithi188.stdout:drwxr-xr-x root/root 0 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/lib/reed_solomon/ 2023-11-26T07:02:44.888 INFO:teuthology.orchestra.run.smithi188.stdout:-rw-r--r-- root/root 18753 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/lib/reed_solomon/reed_solomon.ko 2023-11-26T07:02:44.889 INFO:teuthology.orchestra.run.smithi188.stdout:-rw-r--r-- root/root 38841 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/lib/test-kstrtox.ko 2023-11-26T07:02:44.889 INFO:teuthology.orchestra.run.smithi188.stdout:-rw-r--r-- root/root 26321 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/lib/test-string_helpers.ko 2023-11-26T07:02:44.889 INFO:teuthology.orchestra.run.smithi188.stdout:-rw-r--r-- root/root 4760425 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/lib/test_bpf.ko 2023-11-26T07:02:44.889 INFO:teuthology.orchestra.run.smithi188.stdout:-rw-r--r-- root/root 49089 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/lib/test_firmware.ko 2023-11-26T07:02:44.889 INFO:teuthology.orchestra.run.smithi188.stdout:-rw-r--r-- root/root 6369 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/lib/test_module.ko 2023-11-26T07:02:44.889 INFO:teuthology.orchestra.run.smithi188.stdout:-rw-r--r-- root/root 20801 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/lib/test_user_copy.ko 2023-11-26T07:02:44.889 INFO:teuthology.orchestra.run.smithi188.stdout:-rw-r--r-- root/root 9585 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/lib/ts_bm.ko 2023-11-26T07:02:44.889 INFO:teuthology.orchestra.run.smithi188.stdout:-rw-r--r-- root/root 11121 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/lib/ts_fsm.ko 2023-11-26T07:02:44.889 INFO:teuthology.orchestra.run.smithi188.stdout:-rw-r--r-- root/root 9625 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/lib/ts_kmp.ko 2023-11-26T07:02:44.889 INFO:teuthology.orchestra.run.smithi188.stdout:drwxr-xr-x root/root 0 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/lib/xz/ 2023-11-26T07:02:44.889 INFO:teuthology.orchestra.run.smithi188.stdout:-rw-r--r-- root/root 13545 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/lib/xz/xz_dec_test.ko 2023-11-26T07:02:44.889 INFO:teuthology.orchestra.run.smithi188.stdout:drwxr-xr-x root/root 0 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/mm/ 2023-11-26T07:02:44.889 INFO:teuthology.orchestra.run.smithi188.stdout:-rw-r--r-- root/root 15425 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/mm/hwpoison-inject.ko 2023-11-26T07:02:44.890 INFO:teuthology.orchestra.run.smithi188.stdout:drwxr-xr-x root/root 0 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/net/ 2023-11-26T07:02:44.890 INFO:teuthology.orchestra.run.smithi188.stdout:drwxr-xr-x root/root 0 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/net/6lowpan/ 2023-11-26T07:02:44.890 INFO:teuthology.orchestra.run.smithi188.stdout:-rw-r--r-- root/root 55737 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/net/6lowpan/6lowpan.ko 2023-11-26T07:02:44.890 INFO:teuthology.orchestra.run.smithi188.stdout:-rw-r--r-- root/root 7049 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/net/6lowpan/nhc_dest.ko 2023-11-26T07:02:44.890 INFO:teuthology.orchestra.run.smithi188.stdout:-rw-r--r-- root/root 7065 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/net/6lowpan/nhc_fragment.ko 2023-11-26T07:02:44.890 INFO:teuthology.orchestra.run.smithi188.stdout:-rw-r--r-- root/root 7049 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/net/6lowpan/nhc_hop.ko 2023-11-26T07:02:44.890 INFO:teuthology.orchestra.run.smithi188.stdout:-rw-r--r-- root/root 6985 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/net/6lowpan/nhc_ipv6.ko 2023-11-26T07:02:44.890 INFO:teuthology.orchestra.run.smithi188.stdout:-rw-r--r-- root/root 7065 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/net/6lowpan/nhc_mobility.ko 2023-11-26T07:02:44.890 INFO:teuthology.orchestra.run.smithi188.stdout:-rw-r--r-- root/root 7001 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/net/6lowpan/nhc_routing.ko 2023-11-26T07:02:44.890 INFO:teuthology.orchestra.run.smithi188.stdout:-rw-r--r-- root/root 15329 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/net/6lowpan/nhc_udp.ko 2023-11-26T07:02:44.890 INFO:teuthology.orchestra.run.smithi188.stdout:drwxr-xr-x root/root 0 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/net/802/ 2023-11-26T07:02:44.890 INFO:teuthology.orchestra.run.smithi188.stdout:-rw-r--r-- root/root 30385 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/net/802/garp.ko 2023-11-26T07:02:44.891 INFO:teuthology.orchestra.run.smithi188.stdout:-rw-r--r-- root/root 34809 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/net/802/mrp.ko 2023-11-26T07:02:44.891 INFO:teuthology.orchestra.run.smithi188.stdout:-rw-r--r-- root/root 8545 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/net/802/p8022.ko 2023-11-26T07:02:44.891 INFO:teuthology.orchestra.run.smithi188.stdout:-rw-r--r-- root/root 16345 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/net/802/psnap.ko 2023-11-26T07:02:44.891 INFO:teuthology.orchestra.run.smithi188.stdout:-rw-r--r-- root/root 11537 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/net/802/stp.ko 2023-11-26T07:02:44.891 INFO:teuthology.orchestra.run.smithi188.stdout:drwxr-xr-x root/root 0 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/net/8021q/ 2023-11-26T07:02:44.891 INFO:teuthology.orchestra.run.smithi188.stdout:-rw-r--r-- root/root 81737 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/net/8021q/8021q.ko 2023-11-26T07:02:44.891 INFO:teuthology.orchestra.run.smithi188.stdout:drwxr-xr-x root/root 0 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/net/9p/ 2023-11-26T07:02:44.891 INFO:teuthology.orchestra.run.smithi188.stdout:-rw-r--r-- root/root 198337 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/net/9p/9pnet.ko 2023-11-26T07:02:44.891 INFO:teuthology.orchestra.run.smithi188.stdout:-rw-r--r-- root/root 35977 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/net/9p/9pnet_fd.ko 2023-11-26T07:02:44.891 INFO:teuthology.orchestra.run.smithi188.stdout:-rw-r--r-- root/root 26345 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/net/9p/9pnet_rdma.ko 2023-11-26T07:02:44.891 INFO:teuthology.orchestra.run.smithi188.stdout:-rw-r--r-- root/root 32337 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/net/9p/9pnet_virtio.ko 2023-11-26T07:02:44.892 INFO:teuthology.orchestra.run.smithi188.stdout:drwxr-xr-x root/root 0 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/net/batman-adv/ 2023-11-26T07:02:44.892 INFO:teuthology.orchestra.run.smithi188.stdout:-rw-r--r-- root/root 649297 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/net/batman-adv/batman-adv.ko 2023-11-26T07:02:44.892 INFO:teuthology.orchestra.run.smithi188.stdout:drwxr-xr-x root/root 0 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/net/bridge/ 2023-11-26T07:02:44.892 INFO:teuthology.orchestra.run.smithi188.stdout:-rw-r--r-- root/root 63809 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/net/bridge/br_netfilter.ko 2023-11-26T07:02:44.892 INFO:teuthology.orchestra.run.smithi188.stdout:-rw-r--r-- root/root 777689 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/net/bridge/bridge.ko 2023-11-26T07:02:44.892 INFO:teuthology.orchestra.run.smithi188.stdout:drwxr-xr-x root/root 0 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/net/bridge/netfilter/ 2023-11-26T07:02:44.892 INFO:teuthology.orchestra.run.smithi188.stdout:-rw-r--r-- root/root 7657 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/net/bridge/netfilter/ebt_802_3.ko 2023-11-26T07:02:44.892 INFO:teuthology.orchestra.run.smithi188.stdout:-rw-r--r-- root/root 12385 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/net/bridge/netfilter/ebt_among.ko 2023-11-26T07:02:44.892 INFO:teuthology.orchestra.run.smithi188.stdout:-rw-r--r-- root/root 9169 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/net/bridge/netfilter/ebt_arp.ko 2023-11-26T07:02:44.892 INFO:teuthology.orchestra.run.smithi188.stdout:-rw-r--r-- root/root 9073 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/net/bridge/netfilter/ebt_arpreply.ko 2023-11-26T07:02:44.892 INFO:teuthology.orchestra.run.smithi188.stdout:-rw-r--r-- root/root 9625 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/net/bridge/netfilter/ebt_dnat.ko 2023-11-26T07:02:44.892 INFO:teuthology.orchestra.run.smithi188.stdout:-rw-r--r-- root/root 8737 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/net/bridge/netfilter/ebt_ip.ko 2023-11-26T07:02:44.892 INFO:teuthology.orchestra.run.smithi188.stdout:-rw-r--r-- root/root 9145 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/net/bridge/netfilter/ebt_ip6.ko 2023-11-26T07:02:44.893 INFO:teuthology.orchestra.run.smithi188.stdout:-rw-r--r-- root/root 9825 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/net/bridge/netfilter/ebt_limit.ko 2023-11-26T07:02:44.893 INFO:teuthology.orchestra.run.smithi188.stdout:-rw-r--r-- root/root 13217 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/net/bridge/netfilter/ebt_log.ko 2023-11-26T07:02:44.893 INFO:teuthology.orchestra.run.smithi188.stdout:-rw-r--r-- root/root 7585 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/net/bridge/netfilter/ebt_mark.ko 2023-11-26T07:02:44.893 INFO:teuthology.orchestra.run.smithi188.stdout:-rw-r--r-- root/root 7593 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/net/bridge/netfilter/ebt_mark_m.ko 2023-11-26T07:02:44.893 INFO:teuthology.orchestra.run.smithi188.stdout:-rw-r--r-- root/root 8385 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/net/bridge/netfilter/ebt_nflog.ko 2023-11-26T07:02:44.893 INFO:teuthology.orchestra.run.smithi188.stdout:-rw-r--r-- root/root 7537 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/net/bridge/netfilter/ebt_pkttype.ko 2023-11-26T07:02:44.893 INFO:teuthology.orchestra.run.smithi188.stdout:-rw-r--r-- root/root 9593 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/net/bridge/netfilter/ebt_redirect.ko 2023-11-26T07:02:44.893 INFO:teuthology.orchestra.run.smithi188.stdout:-rw-r--r-- root/root 8649 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/net/bridge/netfilter/ebt_snat.ko 2023-11-26T07:02:44.893 INFO:teuthology.orchestra.run.smithi188.stdout:-rw-r--r-- root/root 8825 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/net/bridge/netfilter/ebt_stp.ko 2023-11-26T07:02:44.893 INFO:teuthology.orchestra.run.smithi188.stdout:-rw-r--r-- root/root 12401 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/net/bridge/netfilter/ebt_vlan.ko 2023-11-26T07:02:44.893 INFO:teuthology.orchestra.run.smithi188.stdout:-rw-r--r-- root/root 11529 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/net/bridge/netfilter/ebtable_broute.ko 2023-11-26T07:02:44.893 INFO:teuthology.orchestra.run.smithi188.stdout:-rw-r--r-- root/root 9817 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/net/bridge/netfilter/ebtable_filter.ko 2023-11-26T07:02:44.893 INFO:teuthology.orchestra.run.smithi188.stdout:-rw-r--r-- root/root 9793 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/net/bridge/netfilter/ebtable_nat.ko 2023-11-26T07:02:44.893 INFO:teuthology.orchestra.run.smithi188.stdout:-rw-r--r-- root/root 55929 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/net/bridge/netfilter/ebtables.ko 2023-11-26T07:02:44.894 INFO:teuthology.orchestra.run.smithi188.stdout:-rw-r--r-- root/root 14041 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/net/bridge/netfilter/nft_meta_bridge.ko 2023-11-26T07:02:44.894 INFO:teuthology.orchestra.run.smithi188.stdout:-rw-r--r-- root/root 13945 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/net/bridge/netfilter/nft_reject_bridge.ko 2023-11-26T07:02:44.894 INFO:teuthology.orchestra.run.smithi188.stdout:drwxr-xr-x root/root 0 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/net/ceph/ 2023-11-26T07:02:44.894 INFO:teuthology.orchestra.run.smithi188.stdout:-rw-r--r-- root/root 1046833 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/net/ceph/libceph.ko 2023-11-26T07:02:44.894 INFO:teuthology.orchestra.run.smithi188.stdout:drwxr-xr-x root/root 0 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/net/core/ 2023-11-26T07:02:44.894 INFO:teuthology.orchestra.run.smithi188.stdout:-rw-r--r-- root/root 125993 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/net/core/pktgen.ko 2023-11-26T07:02:44.894 INFO:teuthology.orchestra.run.smithi188.stdout:drwxr-xr-x root/root 0 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/net/dccp/ 2023-11-26T07:02:44.894 INFO:teuthology.orchestra.run.smithi188.stdout:-rw-r--r-- root/root 194209 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/net/dccp/dccp.ko 2023-11-26T07:02:44.894 INFO:teuthology.orchestra.run.smithi188.stdout:-rw-r--r-- root/root 8665 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/net/dccp/dccp_diag.ko 2023-11-26T07:02:44.894 INFO:teuthology.orchestra.run.smithi188.stdout:-rw-r--r-- root/root 57881 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/net/dccp/dccp_ipv4.ko 2023-11-26T07:02:44.894 INFO:teuthology.orchestra.run.smithi188.stdout:-rw-r--r-- root/root 60337 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/net/dccp/dccp_ipv6.ko 2023-11-26T07:02:44.895 INFO:teuthology.orchestra.run.smithi188.stdout:drwxr-xr-x root/root 0 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/net/dsa/ 2023-11-26T07:02:44.895 INFO:teuthology.orchestra.run.smithi188.stdout:-rw-r--r-- root/root 390473 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/net/dsa/dsa_core.ko 2023-11-26T07:02:44.895 INFO:teuthology.orchestra.run.smithi188.stdout:-rw-r--r-- root/root 13337 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/net/dsa/tag_brcm.ko 2023-11-26T07:02:44.895 INFO:teuthology.orchestra.run.smithi188.stdout:-rw-r--r-- root/root 12577 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/net/dsa/tag_dsa.ko 2023-11-26T07:02:44.895 INFO:teuthology.orchestra.run.smithi188.stdout:-rw-r--r-- root/root 7721 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/net/dsa/tag_none.ko 2023-11-26T07:02:44.895 INFO:teuthology.orchestra.run.smithi188.stdout:-rw-r--r-- root/root 8953 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/net/dsa/tag_trailer.ko 2023-11-26T07:02:44.895 INFO:teuthology.orchestra.run.smithi188.stdout:drwxr-xr-x root/root 0 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/net/hsr/ 2023-11-26T07:02:44.895 INFO:teuthology.orchestra.run.smithi188.stdout:-rw-r--r-- root/root 99729 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/net/hsr/hsr.ko 2023-11-26T07:02:44.895 INFO:teuthology.orchestra.run.smithi188.stdout:drwxr-xr-x root/root 0 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/net/ieee802154/ 2023-11-26T07:02:44.895 INFO:teuthology.orchestra.run.smithi188.stdout:drwxr-xr-x root/root 0 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/net/ieee802154/6lowpan/ 2023-11-26T07:02:44.895 INFO:teuthology.orchestra.run.smithi188.stdout:-rw-r--r-- root/root 45233 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/net/ieee802154/6lowpan/ieee802154_6lowpan.ko 2023-11-26T07:02:44.896 INFO:teuthology.orchestra.run.smithi188.stdout:-rw-r--r-- root/root 254625 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/net/ieee802154/ieee802154.ko 2023-11-26T07:02:44.896 INFO:teuthology.orchestra.run.smithi188.stdout:-rw-r--r-- root/root 47289 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/net/ieee802154/ieee802154_socket.ko 2023-11-26T07:02:44.896 INFO:teuthology.orchestra.run.smithi188.stdout:drwxr-xr-x root/root 0 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/net/ipv4/ 2023-11-26T07:02:44.896 INFO:teuthology.orchestra.run.smithi188.stdout:-rw-r--r-- root/root 24729 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/net/ipv4/ah4.ko 2023-11-26T07:02:44.896 INFO:teuthology.orchestra.run.smithi188.stdout:-rw-r--r-- root/root 38905 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/net/ipv4/esp4.ko 2023-11-26T07:02:44.896 INFO:teuthology.orchestra.run.smithi188.stdout:-rw-r--r-- root/root 48337 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/net/ipv4/fou.ko 2023-11-26T07:02:44.896 INFO:teuthology.orchestra.run.smithi188.stdout:-rw-r--r-- root/root 18121 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/net/ipv4/gre.ko 2023-11-26T07:02:44.896 INFO:teuthology.orchestra.run.smithi188.stdout:-rw-r--r-- root/root 48657 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/net/ipv4/inet_diag.ko 2023-11-26T07:02:44.896 INFO:teuthology.orchestra.run.smithi188.stdout:-rw-r--r-- root/root 60633 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/net/ipv4/ip_gre.ko 2023-11-26T07:02:44.896 INFO:teuthology.orchestra.run.smithi188.stdout:-rw-r--r-- root/root 61225 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/net/ipv4/ip_tunnel.ko 2023-11-26T07:02:44.896 INFO:teuthology.orchestra.run.smithi188.stdout:-rw-r--r-- root/root 34505 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/net/ipv4/ip_vti.ko 2023-11-26T07:02:44.896 INFO:teuthology.orchestra.run.smithi188.stdout:-rw-r--r-- root/root 14001 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/net/ipv4/ipcomp.ko 2023-11-26T07:02:44.897 INFO:teuthology.orchestra.run.smithi188.stdout:-rw-r--r-- root/root 28841 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/net/ipv4/ipip.ko 2023-11-26T07:02:44.897 INFO:teuthology.orchestra.run.smithi188.stdout:drwxr-xr-x root/root 0 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/net/ipv4/netfilter/ 2023-11-26T07:02:44.897 INFO:teuthology.orchestra.run.smithi188.stdout:-rw-r--r-- root/root 37641 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/net/ipv4/netfilter/arp_tables.ko 2023-11-26T07:02:44.897 INFO:teuthology.orchestra.run.smithi188.stdout:-rw-r--r-- root/root 8873 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/net/ipv4/netfilter/arpt_mangle.ko 2023-11-26T07:02:44.897 INFO:teuthology.orchestra.run.smithi188.stdout:-rw-r--r-- root/root 10409 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/net/ipv4/netfilter/arptable_filter.ko 2023-11-26T07:02:44.897 INFO:teuthology.orchestra.run.smithi188.stdout:-rw-r--r-- root/root 41609 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/net/ipv4/netfilter/ip_tables.ko 2023-11-26T07:02:44.897 INFO:teuthology.orchestra.run.smithi188.stdout:-rw-r--r-- root/root 10057 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/net/ipv4/netfilter/ipt_ECN.ko 2023-11-26T07:02:44.897 INFO:teuthology.orchestra.run.smithi188.stdout:-rw-r--r-- root/root 9681 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/net/ipv4/netfilter/ipt_REJECT.ko 2023-11-26T07:02:44.897 INFO:teuthology.orchestra.run.smithi188.stdout:-rw-r--r-- root/root 14121 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/net/ipv4/netfilter/ipt_SYNPROXY.ko 2023-11-26T07:02:44.897 INFO:teuthology.orchestra.run.smithi188.stdout:-rw-r--r-- root/root 11177 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/net/ipv4/netfilter/ipt_ah.ko 2023-11-26T07:02:44.897 INFO:teuthology.orchestra.run.smithi188.stdout:-rw-r--r-- root/root 14881 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/net/ipv4/netfilter/ipt_rpfilter.ko 2023-11-26T07:02:44.897 INFO:teuthology.orchestra.run.smithi188.stdout:-rw-r--r-- root/root 11305 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/net/ipv4/netfilter/iptable_filter.ko 2023-11-26T07:02:44.898 INFO:teuthology.orchestra.run.smithi188.stdout:-rw-r--r-- root/root 11649 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/net/ipv4/netfilter/iptable_mangle.ko 2023-11-26T07:02:44.898 INFO:teuthology.orchestra.run.smithi188.stdout:-rw-r--r-- root/root 14905 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/net/ipv4/netfilter/iptable_nat.ko 2023-11-26T07:02:44.898 INFO:teuthology.orchestra.run.smithi188.stdout:-rw-r--r-- root/root 11545 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/net/ipv4/netfilter/iptable_raw.ko 2023-11-26T07:02:44.898 INFO:teuthology.orchestra.run.smithi188.stdout:-rw-r--r-- root/root 10409 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/net/ipv4/netfilter/iptable_security.ko 2023-11-26T07:02:44.898 INFO:teuthology.orchestra.run.smithi188.stdout:-rw-r--r-- root/root 12425 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/net/ipv4/netfilter/nf_defrag_ipv4.ko 2023-11-26T07:02:44.898 INFO:teuthology.orchestra.run.smithi188.stdout:-rw-r--r-- root/root 8817 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/net/ipv4/netfilter/nf_dup_ipv4.ko 2023-11-26T07:02:44.898 INFO:teuthology.orchestra.run.smithi188.stdout:-rw-r--r-- root/root 27625 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/net/ipv4/netfilter/nf_nat_h323.ko 2023-11-26T07:02:44.898 INFO:teuthology.orchestra.run.smithi188.stdout:-rw-r--r-- root/root 18161 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/net/ipv4/netfilter/nf_nat_pptp.ko 2023-11-26T07:02:44.898 INFO:teuthology.orchestra.run.smithi188.stdout:-rw-r--r-- root/root 13337 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/net/ipv4/netfilter/nf_nat_snmp_basic.ko 2023-11-26T07:02:44.898 INFO:teuthology.orchestra.run.smithi188.stdout:-rw-r--r-- root/root 20561 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/net/ipv4/netfilter/nf_reject_ipv4.ko 2023-11-26T07:02:44.898 INFO:teuthology.orchestra.run.smithi188.stdout:-rw-r--r-- root/root 9329 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/net/ipv4/netfilter/nf_socket_ipv4.ko 2023-11-26T07:02:44.898 INFO:teuthology.orchestra.run.smithi188.stdout:-rw-r--r-- root/root 15177 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/net/ipv4/netfilter/nf_tproxy_ipv4.ko 2023-11-26T07:02:44.899 INFO:teuthology.orchestra.run.smithi188.stdout:-rw-r--r-- root/root 9961 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/net/ipv4/netfilter/nft_dup_ipv4.ko 2023-11-26T07:02:44.899 INFO:teuthology.orchestra.run.smithi188.stdout:-rw-r--r-- root/root 17113 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/net/ipv4/netfilter/nft_fib_ipv4.ko 2023-11-26T07:02:44.899 INFO:teuthology.orchestra.run.smithi188.stdout:-rw-r--r-- root/root 9009 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/net/ipv4/netfilter/nft_reject_ipv4.ko 2023-11-26T07:02:44.899 INFO:teuthology.orchestra.run.smithi188.stdout:-rw-r--r-- root/root 13073 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/net/ipv4/tcp_bic.ko 2023-11-26T07:02:44.899 INFO:teuthology.orchestra.run.smithi188.stdout:-rw-r--r-- root/root 13737 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/net/ipv4/tcp_dctcp.ko 2023-11-26T07:02:44.899 INFO:teuthology.orchestra.run.smithi188.stdout:-rw-r--r-- root/root 15369 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/net/ipv4/tcp_diag.ko 2023-11-26T07:02:44.899 INFO:teuthology.orchestra.run.smithi188.stdout:-rw-r--r-- root/root 9537 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/net/ipv4/tcp_highspeed.ko 2023-11-26T07:02:44.899 INFO:teuthology.orchestra.run.smithi188.stdout:-rw-r--r-- root/root 12401 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/net/ipv4/tcp_htcp.ko 2023-11-26T07:02:44.899 INFO:teuthology.orchestra.run.smithi188.stdout:-rw-r--r-- root/root 10449 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/net/ipv4/tcp_hybla.ko 2023-11-26T07:02:44.899 INFO:teuthology.orchestra.run.smithi188.stdout:-rw-r--r-- root/root 11473 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/net/ipv4/tcp_illinois.ko 2023-11-26T07:02:44.899 INFO:teuthology.orchestra.run.smithi188.stdout:-rw-r--r-- root/root 10433 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/net/ipv4/tcp_lp.ko 2023-11-26T07:02:44.899 INFO:teuthology.orchestra.run.smithi188.stdout:-rw-r--r-- root/root 8329 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/net/ipv4/tcp_scalable.ko 2023-11-26T07:02:44.900 INFO:teuthology.orchestra.run.smithi188.stdout:-rw-r--r-- root/root 14425 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/net/ipv4/tcp_vegas.ko 2023-11-26T07:02:44.900 INFO:teuthology.orchestra.run.smithi188.stdout:-rw-r--r-- root/root 10089 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/net/ipv4/tcp_veno.ko 2023-11-26T07:02:44.900 INFO:teuthology.orchestra.run.smithi188.stdout:-rw-r--r-- root/root 10953 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/net/ipv4/tcp_westwood.ko 2023-11-26T07:02:44.900 INFO:teuthology.orchestra.run.smithi188.stdout:-rw-r--r-- root/root 10185 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/net/ipv4/tcp_yeah.ko 2023-11-26T07:02:44.900 INFO:teuthology.orchestra.run.smithi188.stdout:-rw-r--r-- root/root 22601 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/net/ipv4/tunnel4.ko 2023-11-26T07:02:44.900 INFO:teuthology.orchestra.run.smithi188.stdout:-rw-r--r-- root/root 17097 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/net/ipv4/udp_diag.ko 2023-11-26T07:02:44.900 INFO:teuthology.orchestra.run.smithi188.stdout:-rw-r--r-- root/root 41665 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/net/ipv4/udp_tunnel.ko 2023-11-26T07:02:44.900 INFO:teuthology.orchestra.run.smithi188.stdout:-rw-r--r-- root/root 11361 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/net/ipv4/xfrm4_tunnel.ko 2023-11-26T07:02:44.900 INFO:teuthology.orchestra.run.smithi188.stdout:drwxr-xr-x root/root 0 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/net/ipv6/ 2023-11-26T07:02:44.900 INFO:teuthology.orchestra.run.smithi188.stdout:-rw-r--r-- root/root 26377 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/net/ipv6/ah6.ko 2023-11-26T07:02:44.900 INFO:teuthology.orchestra.run.smithi188.stdout:-rw-r--r-- root/root 39537 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/net/ipv6/esp6.ko 2023-11-26T07:02:44.900 INFO:teuthology.orchestra.run.smithi188.stdout:-rw-r--r-- root/root 14425 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/net/ipv6/fou6.ko 2023-11-26T07:02:44.900 INFO:teuthology.orchestra.run.smithi188.stdout:-rw-r--r-- root/root 80897 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/net/ipv6/ip6_gre.ko 2023-11-26T07:02:44.901 INFO:teuthology.orchestra.run.smithi188.stdout:-rw-r--r-- root/root 93577 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/net/ipv6/ip6_tunnel.ko 2023-11-26T07:02:44.901 INFO:teuthology.orchestra.run.smithi188.stdout:-rw-r--r-- root/root 10825 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/net/ipv6/ip6_udp_tunnel.ko 2023-11-26T07:02:44.901 INFO:teuthology.orchestra.run.smithi188.stdout:-rw-r--r-- root/root 53273 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/net/ipv6/ip6_vti.ko 2023-11-26T07:02:44.901 INFO:teuthology.orchestra.run.smithi188.stdout:-rw-r--r-- root/root 13785 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/net/ipv6/ipcomp6.ko 2023-11-26T07:02:44.901 INFO:teuthology.orchestra.run.smithi188.stdout:-rw-r--r-- root/root 19017 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/net/ipv6/mip6.ko 2023-11-26T07:02:44.901 INFO:teuthology.orchestra.run.smithi188.stdout:drwxr-xr-x root/root 0 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/net/ipv6/netfilter/ 2023-11-26T07:02:44.901 INFO:teuthology.orchestra.run.smithi188.stdout:-rw-r--r-- root/root 42169 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/net/ipv6/netfilter/ip6_tables.ko 2023-11-26T07:02:44.901 INFO:teuthology.orchestra.run.smithi188.stdout:-rw-r--r-- root/root 11753 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/net/ipv6/netfilter/ip6t_NPT.ko 2023-11-26T07:02:44.901 INFO:teuthology.orchestra.run.smithi188.stdout:-rw-r--r-- root/root 9753 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/net/ipv6/netfilter/ip6t_REJECT.ko 2023-11-26T07:02:44.901 INFO:teuthology.orchestra.run.smithi188.stdout:-rw-r--r-- root/root 14137 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/net/ipv6/netfilter/ip6t_SYNPROXY.ko 2023-11-26T07:02:44.901 INFO:teuthology.orchestra.run.smithi188.stdout:-rw-r--r-- root/root 14273 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/net/ipv6/netfilter/ip6t_ah.ko 2023-11-26T07:02:44.902 INFO:teuthology.orchestra.run.smithi188.stdout:-rw-r--r-- root/root 7801 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/net/ipv6/netfilter/ip6t_eui64.ko 2023-11-26T07:02:44.902 INFO:teuthology.orchestra.run.smithi188.stdout:-rw-r--r-- root/root 16209 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/net/ipv6/netfilter/ip6t_frag.ko 2023-11-26T07:02:44.902 INFO:teuthology.orchestra.run.smithi188.stdout:-rw-r--r-- root/root 16033 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/net/ipv6/netfilter/ip6t_hbh.ko 2023-11-26T07:02:44.902 INFO:teuthology.orchestra.run.smithi188.stdout:-rw-r--r-- root/root 8913 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/net/ipv6/netfilter/ip6t_ipv6header.ko 2023-11-26T07:02:44.902 INFO:teuthology.orchestra.run.smithi188.stdout:-rw-r--r-- root/root 9897 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/net/ipv6/netfilter/ip6t_mh.ko 2023-11-26T07:02:44.902 INFO:teuthology.orchestra.run.smithi188.stdout:-rw-r--r-- root/root 10937 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/net/ipv6/netfilter/ip6t_rpfilter.ko 2023-11-26T07:02:44.902 INFO:teuthology.orchestra.run.smithi188.stdout:-rw-r--r-- root/root 10897 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/net/ipv6/netfilter/ip6t_rt.ko 2023-11-26T07:02:44.902 INFO:teuthology.orchestra.run.smithi188.stdout:-rw-r--r-- root/root 11321 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/net/ipv6/netfilter/ip6table_filter.ko 2023-11-26T07:02:44.902 INFO:teuthology.orchestra.run.smithi188.stdout:-rw-r--r-- root/root 11121 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/net/ipv6/netfilter/ip6table_mangle.ko 2023-11-26T07:02:44.902 INFO:teuthology.orchestra.run.smithi188.stdout:-rw-r--r-- root/root 15009 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/net/ipv6/netfilter/ip6table_nat.ko 2023-11-26T07:02:44.902 INFO:teuthology.orchestra.run.smithi188.stdout:-rw-r--r-- root/root 11553 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/net/ipv6/netfilter/ip6table_raw.ko 2023-11-26T07:02:44.902 INFO:teuthology.orchestra.run.smithi188.stdout:-rw-r--r-- root/root 10433 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/net/ipv6/netfilter/ip6table_security.ko 2023-11-26T07:02:44.903 INFO:teuthology.orchestra.run.smithi188.stdout:-rw-r--r-- root/root 39113 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/net/ipv6/netfilter/nf_defrag_ipv6.ko 2023-11-26T07:02:44.903 INFO:teuthology.orchestra.run.smithi188.stdout:-rw-r--r-- root/root 8449 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/net/ipv6/netfilter/nf_dup_ipv6.ko 2023-11-26T07:02:44.903 INFO:teuthology.orchestra.run.smithi188.stdout:-rw-r--r-- root/root 25945 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/net/ipv6/netfilter/nf_reject_ipv6.ko 2023-11-26T07:02:44.903 INFO:teuthology.orchestra.run.smithi188.stdout:-rw-r--r-- root/root 9801 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/net/ipv6/netfilter/nf_socket_ipv6.ko 2023-11-26T07:02:44.903 INFO:teuthology.orchestra.run.smithi188.stdout:-rw-r--r-- root/root 15393 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/net/ipv6/netfilter/nf_tproxy_ipv6.ko 2023-11-26T07:02:44.903 INFO:teuthology.orchestra.run.smithi188.stdout:-rw-r--r-- root/root 9633 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/net/ipv6/netfilter/nft_dup_ipv6.ko 2023-11-26T07:02:44.903 INFO:teuthology.orchestra.run.smithi188.stdout:-rw-r--r-- root/root 16025 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/net/ipv6/netfilter/nft_fib_ipv6.ko 2023-11-26T07:02:44.903 INFO:teuthology.orchestra.run.smithi188.stdout:-rw-r--r-- root/root 9009 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/net/ipv6/netfilter/nft_reject_ipv6.ko 2023-11-26T07:02:44.903 INFO:teuthology.orchestra.run.smithi188.stdout:-rw-r--r-- root/root 74073 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/net/ipv6/sit.ko 2023-11-26T07:02:44.903 INFO:teuthology.orchestra.run.smithi188.stdout:-rw-r--r-- root/root 24193 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/net/ipv6/tunnel6.ko 2023-11-26T07:02:44.903 INFO:teuthology.orchestra.run.smithi188.stdout:-rw-r--r-- root/root 23297 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/net/ipv6/xfrm6_tunnel.ko 2023-11-26T07:02:44.903 INFO:teuthology.orchestra.run.smithi188.stdout:drwxr-xr-x root/root 0 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/net/key/ 2023-11-26T07:02:44.903 INFO:teuthology.orchestra.run.smithi188.stdout:-rw-r--r-- root/root 82833 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/net/key/af_key.ko 2023-11-26T07:02:44.904 INFO:teuthology.orchestra.run.smithi188.stdout:drwxr-xr-x root/root 0 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/net/l2tp/ 2023-11-26T07:02:44.904 INFO:teuthology.orchestra.run.smithi188.stdout:-rw-r--r-- root/root 125281 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/net/l2tp/l2tp_core.ko 2023-11-26T07:02:44.904 INFO:teuthology.orchestra.run.smithi188.stdout:-rw-r--r-- root/root 20177 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/net/l2tp/l2tp_debugfs.ko 2023-11-26T07:02:44.904 INFO:teuthology.orchestra.run.smithi188.stdout:-rw-r--r-- root/root 25953 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/net/l2tp/l2tp_eth.ko 2023-11-26T07:02:44.904 INFO:teuthology.orchestra.run.smithi188.stdout:-rw-r--r-- root/root 35569 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/net/l2tp/l2tp_ip.ko 2023-11-26T07:02:44.904 INFO:teuthology.orchestra.run.smithi188.stdout:-rw-r--r-- root/root 36713 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/net/l2tp/l2tp_ip6.ko 2023-11-26T07:02:44.904 INFO:teuthology.orchestra.run.smithi188.stdout:-rw-r--r-- root/root 33001 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/net/l2tp/l2tp_netlink.ko 2023-11-26T07:02:44.904 INFO:teuthology.orchestra.run.smithi188.stdout:-rw-r--r-- root/root 53241 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/net/l2tp/l2tp_ppp.ko 2023-11-26T07:02:44.904 INFO:teuthology.orchestra.run.smithi188.stdout:drwxr-xr-x root/root 0 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/net/lapb/ 2023-11-26T07:02:44.904 INFO:teuthology.orchestra.run.smithi188.stdout:-rw-r--r-- root/root 47977 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/net/lapb/lapb.ko 2023-11-26T07:02:44.904 INFO:teuthology.orchestra.run.smithi188.stdout:drwxr-xr-x root/root 0 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/net/llc/ 2023-11-26T07:02:44.904 INFO:teuthology.orchestra.run.smithi188.stdout:-rw-r--r-- root/root 23193 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/net/llc/llc.ko 2023-11-26T07:02:44.905 INFO:teuthology.orchestra.run.smithi188.stdout:-rw-r--r-- root/root 267913 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/net/llc/llc2.ko 2023-11-26T07:02:44.905 INFO:teuthology.orchestra.run.smithi188.stdout:drwxr-xr-x root/root 0 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/net/mac802154/ 2023-11-26T07:02:44.905 INFO:teuthology.orchestra.run.smithi188.stdout:-rw-r--r-- root/root 290417 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/net/mac802154/mac802154.ko 2023-11-26T07:02:44.905 INFO:teuthology.orchestra.run.smithi188.stdout:drwxr-xr-x root/root 0 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/net/mpls/ 2023-11-26T07:02:44.905 INFO:teuthology.orchestra.run.smithi188.stdout:-rw-r--r-- root/root 9393 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/net/mpls/mpls_gso.ko 2023-11-26T07:02:44.905 INFO:teuthology.orchestra.run.smithi188.stdout:drwxr-xr-x root/root 0 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/ 2023-11-26T07:02:44.905 INFO:teuthology.orchestra.run.smithi188.stdout:drwxr-xr-x root/root 0 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/ipset/ 2023-11-26T07:02:44.905 INFO:teuthology.orchestra.run.smithi188.stdout:-rw-r--r-- root/root 119585 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/ipset/ip_set.ko 2023-11-26T07:02:44.905 INFO:teuthology.orchestra.run.smithi188.stdout:-rw-r--r-- root/root 28353 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/ipset/ip_set_bitmap_ip.ko 2023-11-26T07:02:44.905 INFO:teuthology.orchestra.run.smithi188.stdout:-rw-r--r-- root/root 26913 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/ipset/ip_set_bitmap_ipmac.ko 2023-11-26T07:02:44.905 INFO:teuthology.orchestra.run.smithi188.stdout:-rw-r--r-- root/root 26369 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/ipset/ip_set_bitmap_port.ko 2023-11-26T07:02:44.905 INFO:teuthology.orchestra.run.smithi188.stdout:-rw-r--r-- root/root 100297 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/ipset/ip_set_hash_ip.ko 2023-11-26T07:02:44.906 INFO:teuthology.orchestra.run.smithi188.stdout:-rw-r--r-- root/root 101441 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/ipset/ip_set_hash_ipmark.ko 2023-11-26T07:02:44.906 INFO:teuthology.orchestra.run.smithi188.stdout:-rw-r--r-- root/root 102817 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/ipset/ip_set_hash_ipport.ko 2023-11-26T07:02:44.906 INFO:teuthology.orchestra.run.smithi188.stdout:-rw-r--r-- root/root 103417 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/ipset/ip_set_hash_ipportip.ko 2023-11-26T07:02:44.906 INFO:teuthology.orchestra.run.smithi188.stdout:-rw-r--r-- root/root 112849 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/ipset/ip_set_hash_ipportnet.ko 2023-11-26T07:02:44.906 INFO:teuthology.orchestra.run.smithi188.stdout:-rw-r--r-- root/root 60329 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/ipset/ip_set_hash_mac.ko 2023-11-26T07:02:44.906 INFO:teuthology.orchestra.run.smithi188.stdout:-rw-r--r-- root/root 109593 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/ipset/ip_set_hash_net.ko 2023-11-26T07:02:44.906 INFO:teuthology.orchestra.run.smithi188.stdout:-rw-r--r-- root/root 113129 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/ipset/ip_set_hash_netiface.ko 2023-11-26T07:02:44.906 INFO:teuthology.orchestra.run.smithi188.stdout:-rw-r--r-- root/root 112585 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/ipset/ip_set_hash_netnet.ko 2023-11-26T07:02:44.906 INFO:teuthology.orchestra.run.smithi188.stdout:-rw-r--r-- root/root 111585 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/ipset/ip_set_hash_netport.ko 2023-11-26T07:02:44.906 INFO:teuthology.orchestra.run.smithi188.stdout:-rw-r--r-- root/root 114465 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/ipset/ip_set_hash_netportnet.ko 2023-11-26T07:02:44.906 INFO:teuthology.orchestra.run.smithi188.stdout:-rw-r--r-- root/root 30025 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/ipset/ip_set_list_set.ko 2023-11-26T07:02:44.907 INFO:teuthology.orchestra.run.smithi188.stdout:drwxr-xr-x root/root 0 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/ipvs/ 2023-11-26T07:02:44.907 INFO:teuthology.orchestra.run.smithi188.stdout:-rw-r--r-- root/root 411729 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/ipvs/ip_vs.ko 2023-11-26T07:02:44.907 INFO:teuthology.orchestra.run.smithi188.stdout:-rw-r--r-- root/root 11961 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/ipvs/ip_vs_dh.ko 2023-11-26T07:02:44.907 INFO:teuthology.orchestra.run.smithi188.stdout:-rw-r--r-- root/root 7625 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/ipvs/ip_vs_fo.ko 2023-11-26T07:02:44.907 INFO:teuthology.orchestra.run.smithi188.stdout:-rw-r--r-- root/root 17729 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/ipvs/ip_vs_ftp.ko 2023-11-26T07:02:44.907 INFO:teuthology.orchestra.run.smithi188.stdout:-rw-r--r-- root/root 17905 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/ipvs/ip_vs_lblc.ko 2023-11-26T07:02:44.907 INFO:teuthology.orchestra.run.smithi188.stdout:-rw-r--r-- root/root 20673 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/ipvs/ip_vs_lblcr.ko 2023-11-26T07:02:44.907 INFO:teuthology.orchestra.run.smithi188.stdout:-rw-r--r-- root/root 7689 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/ipvs/ip_vs_lc.ko 2023-11-26T07:02:44.907 INFO:teuthology.orchestra.run.smithi188.stdout:-rw-r--r-- root/root 7689 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/ipvs/ip_vs_nq.ko 2023-11-26T07:02:44.907 INFO:teuthology.orchestra.run.smithi188.stdout:-rw-r--r-- root/root 10681 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/ipvs/ip_vs_pe_sip.ko 2023-11-26T07:02:44.907 INFO:teuthology.orchestra.run.smithi188.stdout:-rw-r--r-- root/root 9233 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/ipvs/ip_vs_rr.ko 2023-11-26T07:02:44.908 INFO:teuthology.orchestra.run.smithi188.stdout:-rw-r--r-- root/root 7697 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/ipvs/ip_vs_sed.ko 2023-11-26T07:02:44.908 INFO:teuthology.orchestra.run.smithi188.stdout:-rw-r--r-- root/root 13521 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/ipvs/ip_vs_sh.ko 2023-11-26T07:02:44.908 INFO:teuthology.orchestra.run.smithi188.stdout:-rw-r--r-- root/root 7697 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/ipvs/ip_vs_wlc.ko 2023-11-26T07:02:44.908 INFO:teuthology.orchestra.run.smithi188.stdout:-rw-r--r-- root/root 10993 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/ipvs/ip_vs_wrr.ko 2023-11-26T07:02:44.908 INFO:teuthology.orchestra.run.smithi188.stdout:-rw-r--r-- root/root 49257 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/nf_conncount.ko 2023-11-26T07:02:44.908 INFO:teuthology.orchestra.run.smithi188.stdout:-rw-r--r-- root/root 408257 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/nf_conntrack.ko 2023-11-26T07:02:44.908 INFO:teuthology.orchestra.run.smithi188.stdout:-rw-r--r-- root/root 15985 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/nf_conntrack_amanda.ko 2023-11-26T07:02:44.908 INFO:teuthology.orchestra.run.smithi188.stdout:-rw-r--r-- root/root 10177 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/nf_conntrack_broadcast.ko 2023-11-26T07:02:44.908 INFO:teuthology.orchestra.run.smithi188.stdout:-rw-r--r-- root/root 33761 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/nf_conntrack_ftp.ko 2023-11-26T07:02:44.908 INFO:teuthology.orchestra.run.smithi188.stdout:-rw-r--r-- root/root 118145 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/nf_conntrack_h323.ko 2023-11-26T07:02:44.908 INFO:teuthology.orchestra.run.smithi188.stdout:-rw-r--r-- root/root 22185 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/nf_conntrack_irc.ko 2023-11-26T07:02:44.908 INFO:teuthology.orchestra.run.smithi188.stdout:-rw-r--r-- root/root 8817 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/nf_conntrack_netbios_ns.ko 2023-11-26T07:02:44.909 INFO:teuthology.orchestra.run.smithi188.stdout:-rw-r--r-- root/root 107233 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/nf_conntrack_netlink.ko 2023-11-26T07:02:44.909 INFO:teuthology.orchestra.run.smithi188.stdout:-rw-r--r-- root/root 33633 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/nf_conntrack_pptp.ko 2023-11-26T07:02:44.909 INFO:teuthology.orchestra.run.smithi188.stdout:-rw-r--r-- root/root 17401 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/nf_conntrack_sane.ko 2023-11-26T07:02:44.909 INFO:teuthology.orchestra.run.smithi188.stdout:-rw-r--r-- root/root 66641 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/nf_conntrack_sip.ko 2023-11-26T07:02:44.909 INFO:teuthology.orchestra.run.smithi188.stdout:-rw-r--r-- root/root 11233 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/nf_conntrack_snmp.ko 2023-11-26T07:02:44.909 INFO:teuthology.orchestra.run.smithi188.stdout:-rw-r--r-- root/root 19185 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/nf_conntrack_tftp.ko 2023-11-26T07:02:44.909 INFO:teuthology.orchestra.run.smithi188.stdout:-rw-r--r-- root/root 10113 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/nf_dup_netdev.ko 2023-11-26T07:02:44.909 INFO:teuthology.orchestra.run.smithi188.stdout:-rw-r--r-- root/root 89425 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/nf_flow_table.ko 2023-11-26T07:02:44.909 INFO:teuthology.orchestra.run.smithi188.stdout:-rw-r--r-- root/root 9593 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/nf_flow_table_inet.ko 2023-11-26T07:02:44.909 INFO:teuthology.orchestra.run.smithi188.stdout:-rw-r--r-- root/root 37129 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/nf_log_syslog.ko 2023-11-26T07:02:44.909 INFO:teuthology.orchestra.run.smithi188.stdout:-rw-r--r-- root/root 95353 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/nf_nat.ko 2023-11-26T07:02:44.910 INFO:teuthology.orchestra.run.smithi188.stdout:-rw-r--r-- root/root 10129 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/nf_nat_amanda.ko 2023-11-26T07:02:44.910 INFO:teuthology.orchestra.run.smithi188.stdout:-rw-r--r-- root/root 12625 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/nf_nat_ftp.ko 2023-11-26T07:02:44.910 INFO:teuthology.orchestra.run.smithi188.stdout:-rw-r--r-- root/root 11857 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/nf_nat_irc.ko 2023-11-26T07:02:44.910 INFO:teuthology.orchestra.run.smithi188.stdout:-rw-r--r-- root/root 28929 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/nf_nat_sip.ko 2023-11-26T07:02:44.910 INFO:teuthology.orchestra.run.smithi188.stdout:-rw-r--r-- root/root 8809 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/nf_nat_tftp.ko 2023-11-26T07:02:44.910 INFO:teuthology.orchestra.run.smithi188.stdout:-rw-r--r-- root/root 44065 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/nf_synproxy_core.ko 2023-11-26T07:02:44.910 INFO:teuthology.orchestra.run.smithi188.stdout:-rw-r--r-- root/root 615649 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/nf_tables.ko 2023-11-26T07:02:44.910 INFO:teuthology.orchestra.run.smithi188.stdout:-rw-r--r-- root/root 42561 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/nfnetlink.ko 2023-11-26T07:02:44.910 INFO:teuthology.orchestra.run.smithi188.stdout:-rw-r--r-- root/root 30233 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/nfnetlink_acct.ko 2023-11-26T07:02:44.910 INFO:teuthology.orchestra.run.smithi188.stdout:-rw-r--r-- root/root 30569 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/nfnetlink_cttimeout.ko 2023-11-26T07:02:44.910 INFO:teuthology.orchestra.run.smithi188.stdout:-rw-r--r-- root/root 39785 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/nfnetlink_log.ko 2023-11-26T07:02:44.910 INFO:teuthology.orchestra.run.smithi188.stdout:-rw-r--r-- root/root 22921 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/nfnetlink_osf.ko 2023-11-26T07:02:44.911 INFO:teuthology.orchestra.run.smithi188.stdout:-rw-r--r-- root/root 49377 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/nfnetlink_queue.ko 2023-11-26T07:02:44.911 INFO:teuthology.orchestra.run.smithi188.stdout:-rw-r--r-- root/root 10457 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/nft_chain_nat.ko 2023-11-26T07:02:44.911 INFO:teuthology.orchestra.run.smithi188.stdout:-rw-r--r-- root/root 33209 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/nft_compat.ko 2023-11-26T07:02:44.911 INFO:teuthology.orchestra.run.smithi188.stdout:-rw-r--r-- root/root 40721 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/nft_ct.ko 2023-11-26T07:02:44.911 INFO:teuthology.orchestra.run.smithi188.stdout:-rw-r--r-- root/root 9601 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/nft_dup_netdev.ko 2023-11-26T07:02:44.911 INFO:teuthology.orchestra.run.smithi188.stdout:-rw-r--r-- root/root 11401 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/nft_fib.ko 2023-11-26T07:02:44.911 INFO:teuthology.orchestra.run.smithi188.stdout:-rw-r--r-- root/root 13745 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/nft_fwd_netdev.ko 2023-11-26T07:02:44.911 INFO:teuthology.orchestra.run.smithi188.stdout:-rw-r--r-- root/root 14185 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/nft_hash.ko 2023-11-26T07:02:44.911 INFO:teuthology.orchestra.run.smithi188.stdout:-rw-r--r-- root/root 20065 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/nft_limit.ko 2023-11-26T07:02:44.911 INFO:teuthology.orchestra.run.smithi188.stdout:-rw-r--r-- root/root 15401 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/nft_log.ko 2023-11-26T07:02:44.911 INFO:teuthology.orchestra.run.smithi188.stdout:-rw-r--r-- root/root 14801 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/nft_masq.ko 2023-11-26T07:02:44.912 INFO:teuthology.orchestra.run.smithi188.stdout:-rw-r--r-- root/root 14329 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/nft_nat.ko 2023-11-26T07:02:44.912 INFO:teuthology.orchestra.run.smithi188.stdout:-rw-r--r-- root/root 13673 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/nft_numgen.ko 2023-11-26T07:02:44.912 INFO:teuthology.orchestra.run.smithi188.stdout:-rw-r--r-- root/root 11841 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/nft_osf.ko 2023-11-26T07:02:44.912 INFO:teuthology.orchestra.run.smithi188.stdout:-rw-r--r-- root/root 14073 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/nft_queue.ko 2023-11-26T07:02:44.912 INFO:teuthology.orchestra.run.smithi188.stdout:-rw-r--r-- root/root 14193 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/nft_redir.ko 2023-11-26T07:02:44.912 INFO:teuthology.orchestra.run.smithi188.stdout:-rw-r--r-- root/root 10497 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/nft_reject.ko 2023-11-26T07:02:44.912 INFO:teuthology.orchestra.run.smithi188.stdout:-rw-r--r-- root/root 10081 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/nft_reject_inet.ko 2023-11-26T07:02:44.912 INFO:teuthology.orchestra.run.smithi188.stdout:-rw-r--r-- root/root 12737 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/nft_socket.ko 2023-11-26T07:02:44.912 INFO:teuthology.orchestra.run.smithi188.stdout:-rw-r--r-- root/root 20369 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/nft_synproxy.ko 2023-11-26T07:02:44.912 INFO:teuthology.orchestra.run.smithi188.stdout:-rw-r--r-- root/root 14849 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/nft_tproxy.ko 2023-11-26T07:02:44.912 INFO:teuthology.orchestra.run.smithi188.stdout:-rw-r--r-- root/root 89313 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/x_tables.ko 2023-11-26T07:02:44.912 INFO:teuthology.orchestra.run.smithi188.stdout:-rw-r--r-- root/root 12097 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/xt_AUDIT.ko 2023-11-26T07:02:44.913 INFO:teuthology.orchestra.run.smithi188.stdout:-rw-r--r-- root/root 9633 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/xt_CHECKSUM.ko 2023-11-26T07:02:44.913 INFO:teuthology.orchestra.run.smithi188.stdout:-rw-r--r-- root/root 7729 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/xt_CLASSIFY.ko 2023-11-26T07:02:44.913 INFO:teuthology.orchestra.run.smithi188.stdout:-rw-r--r-- root/root 11521 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/xt_CONNSECMARK.ko 2023-11-26T07:02:44.913 INFO:teuthology.orchestra.run.smithi188.stdout:-rw-r--r-- root/root 18169 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/xt_CT.ko 2023-11-26T07:02:44.913 INFO:teuthology.orchestra.run.smithi188.stdout:-rw-r--r-- root/root 10497 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/xt_DSCP.ko 2023-11-26T07:02:44.913 INFO:teuthology.orchestra.run.smithi188.stdout:-rw-r--r-- root/root 9465 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/xt_HL.ko 2023-11-26T07:02:44.913 INFO:teuthology.orchestra.run.smithi188.stdout:-rw-r--r-- root/root 13369 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/xt_HMARK.ko 2023-11-26T07:02:44.913 INFO:teuthology.orchestra.run.smithi188.stdout:-rw-r--r-- root/root 38873 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/xt_IDLETIMER.ko 2023-11-26T07:02:44.913 INFO:teuthology.orchestra.run.smithi188.stdout:-rw-r--r-- root/root 13889 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/xt_LED.ko 2023-11-26T07:02:44.913 INFO:teuthology.orchestra.run.smithi188.stdout:-rw-r--r-- root/root 11489 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/xt_LOG.ko 2023-11-26T07:02:44.913 INFO:teuthology.orchestra.run.smithi188.stdout:-rw-r--r-- root/root 11753 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/xt_MASQUERADE.ko 2023-11-26T07:02:44.914 INFO:teuthology.orchestra.run.smithi188.stdout:-rw-r--r-- root/root 11657 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/xt_NETMAP.ko 2023-11-26T07:02:44.914 INFO:teuthology.orchestra.run.smithi188.stdout:-rw-r--r-- root/root 9473 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/xt_NFLOG.ko 2023-11-26T07:02:44.914 INFO:teuthology.orchestra.run.smithi188.stdout:-rw-r--r-- root/root 13289 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/xt_NFQUEUE.ko 2023-11-26T07:02:44.914 INFO:teuthology.orchestra.run.smithi188.stdout:-rw-r--r-- root/root 19665 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/xt_RATEEST.ko 2023-11-26T07:02:44.914 INFO:teuthology.orchestra.run.smithi188.stdout:-rw-r--r-- root/root 11073 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/xt_REDIRECT.ko 2023-11-26T07:02:44.914 INFO:teuthology.orchestra.run.smithi188.stdout:-rw-r--r-- root/root 14337 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/xt_SECMARK.ko 2023-11-26T07:02:44.914 INFO:teuthology.orchestra.run.smithi188.stdout:-rw-r--r-- root/root 16057 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/xt_TCPMSS.ko 2023-11-26T07:02:44.914 INFO:teuthology.orchestra.run.smithi188.stdout:-rw-r--r-- root/root 10433 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/xt_TCPOPTSTRIP.ko 2023-11-26T07:02:44.914 INFO:teuthology.orchestra.run.smithi188.stdout:-rw-r--r-- root/root 17361 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/xt_TEE.ko 2023-11-26T07:02:44.914 INFO:teuthology.orchestra.run.smithi188.stdout:-rw-r--r-- root/root 16785 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/xt_TPROXY.ko 2023-11-26T07:02:44.914 INFO:teuthology.orchestra.run.smithi188.stdout:-rw-r--r-- root/root 8321 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/xt_TRACE.ko 2023-11-26T07:02:44.914 INFO:teuthology.orchestra.run.smithi188.stdout:-rw-r--r-- root/root 13897 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/xt_addrtype.ko 2023-11-26T07:02:44.915 INFO:teuthology.orchestra.run.smithi188.stdout:-rw-r--r-- root/root 13897 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/xt_bpf.ko 2023-11-26T07:02:44.915 INFO:teuthology.orchestra.run.smithi188.stdout:-rw-r--r-- root/root 15529 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/xt_cgroup.ko 2023-11-26T07:02:44.915 INFO:teuthology.orchestra.run.smithi188.stdout:-rw-r--r-- root/root 11209 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/xt_cluster.ko 2023-11-26T07:02:44.915 INFO:teuthology.orchestra.run.smithi188.stdout:-rw-r--r-- root/root 7505 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/xt_comment.ko 2023-11-26T07:02:44.915 INFO:teuthology.orchestra.run.smithi188.stdout:-rw-r--r-- root/root 10465 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/xt_connbytes.ko 2023-11-26T07:02:44.915 INFO:teuthology.orchestra.run.smithi188.stdout:-rw-r--r-- root/root 11081 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/xt_connlabel.ko 2023-11-26T07:02:44.915 INFO:teuthology.orchestra.run.smithi188.stdout:-rw-r--r-- root/root 9401 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/xt_connlimit.ko 2023-11-26T07:02:44.915 INFO:teuthology.orchestra.run.smithi188.stdout:-rw-r--r-- root/root 13129 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/xt_connmark.ko 2023-11-26T07:02:44.915 INFO:teuthology.orchestra.run.smithi188.stdout:-rw-r--r-- root/root 12401 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/xt_conntrack.ko 2023-11-26T07:02:44.915 INFO:teuthology.orchestra.run.smithi188.stdout:-rw-r--r-- root/root 7817 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/xt_cpu.ko 2023-11-26T07:02:44.915 INFO:teuthology.orchestra.run.smithi188.stdout:-rw-r--r-- root/root 10969 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/xt_dccp.ko 2023-11-26T07:02:44.916 INFO:teuthology.orchestra.run.smithi188.stdout:-rw-r--r-- root/root 7865 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/xt_devgroup.ko 2023-11-26T07:02:44.916 INFO:teuthology.orchestra.run.smithi188.stdout:-rw-r--r-- root/root 8801 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/xt_dscp.ko 2023-11-26T07:02:44.916 INFO:teuthology.orchestra.run.smithi188.stdout:-rw-r--r-- root/root 10657 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/xt_ecn.ko 2023-11-26T07:02:44.916 INFO:teuthology.orchestra.run.smithi188.stdout:-rw-r--r-- root/root 11473 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/xt_esp.ko 2023-11-26T07:02:44.916 INFO:teuthology.orchestra.run.smithi188.stdout:-rw-r--r-- root/root 43289 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/xt_hashlimit.ko 2023-11-26T07:02:44.916 INFO:teuthology.orchestra.run.smithi188.stdout:-rw-r--r-- root/root 11313 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/xt_helper.ko 2023-11-26T07:02:44.916 INFO:teuthology.orchestra.run.smithi188.stdout:-rw-r--r-- root/root 7865 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/xt_hl.ko 2023-11-26T07:02:44.916 INFO:teuthology.orchestra.run.smithi188.stdout:-rw-r--r-- root/root 11921 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/xt_ipcomp.ko 2023-11-26T07:02:44.916 INFO:teuthology.orchestra.run.smithi188.stdout:-rw-r--r-- root/root 11473 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/xt_iprange.ko 2023-11-26T07:02:44.916 INFO:teuthology.orchestra.run.smithi188.stdout:-rw-r--r-- root/root 11785 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/xt_ipvs.ko 2023-11-26T07:02:44.916 INFO:teuthology.orchestra.run.smithi188.stdout:-rw-r--r-- root/root 16041 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/xt_l2tp.ko 2023-11-26T07:02:44.916 INFO:teuthology.orchestra.run.smithi188.stdout:-rw-r--r-- root/root 8001 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/xt_length.ko 2023-11-26T07:02:44.917 INFO:teuthology.orchestra.run.smithi188.stdout:-rw-r--r-- root/root 10209 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/xt_limit.ko 2023-11-26T07:02:44.917 INFO:teuthology.orchestra.run.smithi188.stdout:-rw-r--r-- root/root 7545 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/xt_mac.ko 2023-11-26T07:02:44.917 INFO:teuthology.orchestra.run.smithi188.stdout:-rw-r--r-- root/root 8577 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/xt_mark.ko 2023-11-26T07:02:44.917 INFO:teuthology.orchestra.run.smithi188.stdout:-rw-r--r-- root/root 11825 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/xt_multiport.ko 2023-11-26T07:02:44.917 INFO:teuthology.orchestra.run.smithi188.stdout:-rw-r--r-- root/root 13817 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/xt_nat.ko 2023-11-26T07:02:44.917 INFO:teuthology.orchestra.run.smithi188.stdout:-rw-r--r-- root/root 10105 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/xt_nfacct.ko 2023-11-26T07:02:44.917 INFO:teuthology.orchestra.run.smithi188.stdout:-rw-r--r-- root/root 8297 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/xt_osf.ko 2023-11-26T07:02:44.917 INFO:teuthology.orchestra.run.smithi188.stdout:-rw-r--r-- root/root 9553 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/xt_owner.ko 2023-11-26T07:02:44.917 INFO:teuthology.orchestra.run.smithi188.stdout:-rw-r--r-- root/root 9729 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/xt_physdev.ko 2023-11-26T07:02:44.918 INFO:teuthology.orchestra.run.smithi188.stdout:-rw-r--r-- root/root 7633 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/xt_pkttype.ko 2023-11-26T07:02:44.918 INFO:teuthology.orchestra.run.smithi188.stdout:-rw-r--r-- root/root 11985 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/xt_policy.ko 2023-11-26T07:02:44.918 INFO:teuthology.orchestra.run.smithi188.stdout:-rw-r--r-- root/root 9473 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/xt_quota.ko 2023-11-26T07:02:44.918 INFO:teuthology.orchestra.run.smithi188.stdout:-rw-r--r-- root/root 10409 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/xt_rateest.ko 2023-11-26T07:02:44.918 INFO:teuthology.orchestra.run.smithi188.stdout:-rw-r--r-- root/root 8257 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/xt_realm.ko 2023-11-26T07:02:44.918 INFO:teuthology.orchestra.run.smithi188.stdout:-rw-r--r-- root/root 36433 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/xt_recent.ko 2023-11-26T07:02:44.918 INFO:teuthology.orchestra.run.smithi188.stdout:-rw-r--r-- root/root 13329 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/xt_sctp.ko 2023-11-26T07:02:44.918 INFO:teuthology.orchestra.run.smithi188.stdout:-rw-r--r-- root/root 30089 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/xt_set.ko 2023-11-26T07:02:44.918 INFO:teuthology.orchestra.run.smithi188.stdout:-rw-r--r-- root/root 16089 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/xt_socket.ko 2023-11-26T07:02:44.918 INFO:teuthology.orchestra.run.smithi188.stdout:-rw-r--r-- root/root 9505 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/xt_state.ko 2023-11-26T07:02:44.918 INFO:teuthology.orchestra.run.smithi188.stdout:-rw-r--r-- root/root 9473 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/xt_statistic.ko 2023-11-26T07:02:44.919 INFO:teuthology.orchestra.run.smithi188.stdout:-rw-r--r-- root/root 8809 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/xt_string.ko 2023-11-26T07:02:44.919 INFO:teuthology.orchestra.run.smithi188.stdout:-rw-r--r-- root/root 8553 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/xt_tcpmss.ko 2023-11-26T07:02:44.919 INFO:teuthology.orchestra.run.smithi188.stdout:-rw-r--r-- root/root 15777 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/xt_tcpudp.ko 2023-11-26T07:02:44.919 INFO:teuthology.orchestra.run.smithi188.stdout:-rw-r--r-- root/root 11065 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/xt_time.ko 2023-11-26T07:02:44.919 INFO:teuthology.orchestra.run.smithi188.stdout:-rw-r--r-- root/root 9401 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/xt_u32.ko 2023-11-26T07:02:44.919 INFO:teuthology.orchestra.run.smithi188.stdout:drwxr-xr-x root/root 0 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/net/netlink/ 2023-11-26T07:02:44.919 INFO:teuthology.orchestra.run.smithi188.stdout:-rw-r--r-- root/root 13969 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/net/netlink/netlink_diag.ko 2023-11-26T07:02:44.919 INFO:teuthology.orchestra.run.smithi188.stdout:drwxr-xr-x root/root 0 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/net/nsh/ 2023-11-26T07:02:44.919 INFO:teuthology.orchestra.run.smithi188.stdout:-rw-r--r-- root/root 12425 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/net/nsh/nsh.ko 2023-11-26T07:02:44.919 INFO:teuthology.orchestra.run.smithi188.stdout:drwxr-xr-x root/root 0 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/net/openvswitch/ 2023-11-26T07:02:44.919 INFO:teuthology.orchestra.run.smithi188.stdout:-rw-r--r-- root/root 380033 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/net/openvswitch/openvswitch.ko 2023-11-26T07:02:44.919 INFO:teuthology.orchestra.run.smithi188.stdout:-rw-r--r-- root/root 10257 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/net/openvswitch/vport-geneve.ko 2023-11-26T07:02:44.919 INFO:teuthology.orchestra.run.smithi188.stdout:-rw-r--r-- root/root 9449 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/net/openvswitch/vport-gre.ko 2023-11-26T07:02:44.920 INFO:teuthology.orchestra.run.smithi188.stdout:-rw-r--r-- root/root 11793 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/net/openvswitch/vport-vxlan.ko 2023-11-26T07:02:44.920 INFO:teuthology.orchestra.run.smithi188.stdout:drwxr-xr-x root/root 0 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/net/packet/ 2023-11-26T07:02:44.920 INFO:teuthology.orchestra.run.smithi188.stdout:-rw-r--r-- root/root 13057 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/net/packet/af_packet_diag.ko 2023-11-26T07:02:44.920 INFO:teuthology.orchestra.run.smithi188.stdout:drwxr-xr-x root/root 0 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/net/phonet/ 2023-11-26T07:02:44.920 INFO:teuthology.orchestra.run.smithi188.stdout:-rw-r--r-- root/root 110201 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/net/phonet/phonet.ko 2023-11-26T07:02:44.920 INFO:teuthology.orchestra.run.smithi188.stdout:-rw-r--r-- root/root 54721 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/net/phonet/pn_pep.ko 2023-11-26T07:02:44.920 INFO:teuthology.orchestra.run.smithi188.stdout:drwxr-xr-x root/root 0 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/net/rds/ 2023-11-26T07:02:44.920 INFO:teuthology.orchestra.run.smithi188.stdout:-rw-r--r-- root/root 251481 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/net/rds/rds.ko 2023-11-26T07:02:44.920 INFO:teuthology.orchestra.run.smithi188.stdout:-rw-r--r-- root/root 164937 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/net/rds/rds_rdma.ko 2023-11-26T07:02:44.920 INFO:teuthology.orchestra.run.smithi188.stdout:-rw-r--r-- root/root 67377 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/net/rds/rds_tcp.ko 2023-11-26T07:02:44.920 INFO:teuthology.orchestra.run.smithi188.stdout:drwxr-xr-x root/root 0 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/net/rxrpc/ 2023-11-26T07:02:44.920 INFO:teuthology.orchestra.run.smithi188.stdout:-rw-r--r-- root/root 819657 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/net/rxrpc/rxrpc.ko 2023-11-26T07:02:44.921 INFO:teuthology.orchestra.run.smithi188.stdout:drwxr-xr-x root/root 0 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/net/sched/ 2023-11-26T07:02:44.921 INFO:teuthology.orchestra.run.smithi188.stdout:-rw-r--r-- root/root 32241 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/net/sched/act_csum.ko 2023-11-26T07:02:44.921 INFO:teuthology.orchestra.run.smithi188.stdout:-rw-r--r-- root/root 23577 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/net/sched/act_gact.ko 2023-11-26T07:02:44.921 INFO:teuthology.orchestra.run.smithi188.stdout:-rw-r--r-- root/root 26873 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/net/sched/act_ipt.ko 2023-11-26T07:02:44.921 INFO:teuthology.orchestra.run.smithi188.stdout:-rw-r--r-- root/root 34097 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/net/sched/act_mirred.ko 2023-11-26T07:02:44.921 INFO:teuthology.orchestra.run.smithi188.stdout:-rw-r--r-- root/root 23401 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/net/sched/act_nat.ko 2023-11-26T07:02:44.921 INFO:teuthology.orchestra.run.smithi188.stdout:-rw-r--r-- root/root 43449 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/net/sched/act_pedit.ko 2023-11-26T07:02:44.921 INFO:teuthology.orchestra.run.smithi188.stdout:-rw-r--r-- root/root 31961 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/net/sched/act_police.ko 2023-11-26T07:02:44.921 INFO:teuthology.orchestra.run.smithi188.stdout:-rw-r--r-- root/root 20289 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/net/sched/act_simple.ko 2023-11-26T07:02:44.921 INFO:teuthology.orchestra.run.smithi188.stdout:-rw-r--r-- root/root 35297 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/net/sched/act_skbedit.ko 2023-11-26T07:02:44.921 INFO:teuthology.orchestra.run.smithi188.stdout:-rw-r--r-- root/root 36665 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/net/sched/act_vlan.ko 2023-11-26T07:02:44.921 INFO:teuthology.orchestra.run.smithi188.stdout:-rw-r--r-- root/root 26169 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/net/sched/cls_basic.ko 2023-11-26T07:02:44.922 INFO:teuthology.orchestra.run.smithi188.stdout:-rw-r--r-- root/root 33961 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/net/sched/cls_bpf.ko 2023-11-26T07:02:44.922 INFO:teuthology.orchestra.run.smithi188.stdout:-rw-r--r-- root/root 19169 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/net/sched/cls_cgroup.ko 2023-11-26T07:02:44.922 INFO:teuthology.orchestra.run.smithi188.stdout:-rw-r--r-- root/root 31401 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/net/sched/cls_flow.ko 2023-11-26T07:02:44.922 INFO:teuthology.orchestra.run.smithi188.stdout:-rw-r--r-- root/root 31769 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/net/sched/cls_fw.ko 2023-11-26T07:02:44.922 INFO:teuthology.orchestra.run.smithi188.stdout:-rw-r--r-- root/root 38425 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/net/sched/cls_route.ko 2023-11-26T07:02:44.922 INFO:teuthology.orchestra.run.smithi188.stdout:-rw-r--r-- root/root 61409 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/net/sched/cls_u32.ko 2023-11-26T07:02:44.922 INFO:teuthology.orchestra.run.smithi188.stdout:-rw-r--r-- root/root 7441 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/net/sched/em_cmp.ko 2023-11-26T07:02:44.922 INFO:teuthology.orchestra.run.smithi188.stdout:-rw-r--r-- root/root 13409 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/net/sched/em_ipset.ko 2023-11-26T07:02:44.922 INFO:teuthology.orchestra.run.smithi188.stdout:-rw-r--r-- root/root 33289 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/net/sched/em_meta.ko 2023-11-26T07:02:44.922 INFO:teuthology.orchestra.run.smithi188.stdout:-rw-r--r-- root/root 7897 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/net/sched/em_nbyte.ko 2023-11-26T07:02:44.922 INFO:teuthology.orchestra.run.smithi188.stdout:-rw-r--r-- root/root 10625 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/net/sched/em_text.ko 2023-11-26T07:02:44.923 INFO:teuthology.orchestra.run.smithi188.stdout:-rw-r--r-- root/root 7249 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/net/sched/em_u32.ko 2023-11-26T07:02:44.923 INFO:teuthology.orchestra.run.smithi188.stdout:-rw-r--r-- root/root 21649 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/net/sched/sch_choke.ko 2023-11-26T07:02:44.923 INFO:teuthology.orchestra.run.smithi188.stdout:-rw-r--r-- root/root 20849 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/net/sched/sch_codel.ko 2023-11-26T07:02:44.923 INFO:teuthology.orchestra.run.smithi188.stdout:-rw-r--r-- root/root 34633 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/net/sched/sch_drr.ko 2023-11-26T07:02:44.923 INFO:teuthology.orchestra.run.smithi188.stdout:-rw-r--r-- root/root 33609 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/net/sched/sch_fq.ko 2023-11-26T07:02:44.923 INFO:teuthology.orchestra.run.smithi188.stdout:-rw-r--r-- root/root 31617 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/net/sched/sch_fq_codel.ko 2023-11-26T07:02:44.923 INFO:teuthology.orchestra.run.smithi188.stdout:-rw-r--r-- root/root 35433 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/net/sched/sch_gred.ko 2023-11-26T07:02:44.923 INFO:teuthology.orchestra.run.smithi188.stdout:-rw-r--r-- root/root 49817 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/net/sched/sch_hfsc.ko 2023-11-26T07:02:44.923 INFO:teuthology.orchestra.run.smithi188.stdout:-rw-r--r-- root/root 22089 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/net/sched/sch_hhf.ko 2023-11-26T07:02:44.923 INFO:teuthology.orchestra.run.smithi188.stdout:-rw-r--r-- root/root 66393 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/net/sched/sch_htb.ko 2023-11-26T07:02:44.923 INFO:teuthology.orchestra.run.smithi188.stdout:-rw-r--r-- root/root 26073 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/net/sched/sch_ingress.ko 2023-11-26T07:02:44.924 INFO:teuthology.orchestra.run.smithi188.stdout:-rw-r--r-- root/root 30273 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/net/sched/sch_mqprio.ko 2023-11-26T07:02:44.924 INFO:teuthology.orchestra.run.smithi188.stdout:-rw-r--r-- root/root 10793 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/net/sched/sch_mqprio_lib.ko 2023-11-26T07:02:44.924 INFO:teuthology.orchestra.run.smithi188.stdout:-rw-r--r-- root/root 26065 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/net/sched/sch_multiq.ko 2023-11-26T07:02:44.924 INFO:teuthology.orchestra.run.smithi188.stdout:-rw-r--r-- root/root 36137 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/net/sched/sch_netem.ko 2023-11-26T07:02:44.924 INFO:teuthology.orchestra.run.smithi188.stdout:-rw-r--r-- root/root 27497 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/net/sched/sch_pie.ko 2023-11-26T07:02:44.924 INFO:teuthology.orchestra.run.smithi188.stdout:-rw-r--r-- root/root 10625 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/net/sched/sch_plug.ko 2023-11-26T07:02:44.924 INFO:teuthology.orchestra.run.smithi188.stdout:-rw-r--r-- root/root 24729 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/net/sched/sch_prio.ko 2023-11-26T07:02:44.924 INFO:teuthology.orchestra.run.smithi188.stdout:-rw-r--r-- root/root 48729 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/net/sched/sch_qfq.ko 2023-11-26T07:02:44.924 INFO:teuthology.orchestra.run.smithi188.stdout:-rw-r--r-- root/root 32905 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/net/sched/sch_red.ko 2023-11-26T07:02:44.924 INFO:teuthology.orchestra.run.smithi188.stdout:-rw-r--r-- root/root 26881 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/net/sched/sch_sfb.ko 2023-11-26T07:02:44.924 INFO:teuthology.orchestra.run.smithi188.stdout:-rw-r--r-- root/root 31353 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/net/sched/sch_sfq.ko 2023-11-26T07:02:44.924 INFO:teuthology.orchestra.run.smithi188.stdout:-rw-r--r-- root/root 28897 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/net/sched/sch_tbf.ko 2023-11-26T07:02:44.925 INFO:teuthology.orchestra.run.smithi188.stdout:-rw-r--r-- root/root 28105 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/net/sched/sch_teql.ko 2023-11-26T07:02:44.925 INFO:teuthology.orchestra.run.smithi188.stdout:drwxr-xr-x root/root 0 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/net/sctp/ 2023-11-26T07:02:44.925 INFO:teuthology.orchestra.run.smithi188.stdout:-rw-r--r-- root/root 929745 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/net/sctp/sctp.ko 2023-11-26T07:02:44.925 INFO:teuthology.orchestra.run.smithi188.stdout:-rw-r--r-- root/root 19033 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/net/sctp/sctp_diag.ko 2023-11-26T07:02:44.925 INFO:teuthology.orchestra.run.smithi188.stdout:drwxr-xr-x root/root 0 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/net/sunrpc/ 2023-11-26T07:02:44.925 INFO:teuthology.orchestra.run.smithi188.stdout:drwxr-xr-x root/root 0 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/net/sunrpc/auth_gss/ 2023-11-26T07:02:44.925 INFO:teuthology.orchestra.run.smithi188.stdout:-rw-r--r-- root/root 336529 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/net/sunrpc/auth_gss/auth_rpcgss.ko 2023-11-26T07:02:44.925 INFO:teuthology.orchestra.run.smithi188.stdout:-rw-r--r-- root/root 54737 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/net/sunrpc/auth_gss/rpcsec_gss_krb5.ko 2023-11-26T07:02:44.925 INFO:teuthology.orchestra.run.smithi188.stdout:-rw-r--r-- root/root 1520097 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/net/sunrpc/sunrpc.ko 2023-11-26T07:02:44.925 INFO:teuthology.orchestra.run.smithi188.stdout:drwxr-xr-x root/root 0 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/net/sunrpc/xprtrdma/ 2023-11-26T07:02:44.925 INFO:teuthology.orchestra.run.smithi188.stdout:-rw-r--r-- root/root 707657 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/net/sunrpc/xprtrdma/rpcrdma.ko 2023-11-26T07:02:44.926 INFO:teuthology.orchestra.run.smithi188.stdout:drwxr-xr-x root/root 0 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/net/tipc/ 2023-11-26T07:02:44.926 INFO:teuthology.orchestra.run.smithi188.stdout:-rw-r--r-- root/root 10025 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/net/tipc/diag.ko 2023-11-26T07:02:44.926 INFO:teuthology.orchestra.run.smithi188.stdout:-rw-r--r-- root/root 862665 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/net/tipc/tipc.ko 2023-11-26T07:02:44.926 INFO:teuthology.orchestra.run.smithi188.stdout:drwxr-xr-x root/root 0 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/net/unix/ 2023-11-26T07:02:44.926 INFO:teuthology.orchestra.run.smithi188.stdout:-rw-r--r-- root/root 15729 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/net/unix/unix_diag.ko 2023-11-26T07:02:44.926 INFO:teuthology.orchestra.run.smithi188.stdout:drwxr-xr-x root/root 0 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/net/vmw_vsock/ 2023-11-26T07:02:44.926 INFO:teuthology.orchestra.run.smithi188.stdout:-rw-r--r-- root/root 85409 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/net/vmw_vsock/vmw_vsock_virtio_transport_common.ko 2023-11-26T07:02:44.926 INFO:teuthology.orchestra.run.smithi188.stdout:-rw-r--r-- root/root 68049 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/net/vmw_vsock/vmw_vsock_vmci_transport.ko 2023-11-26T07:02:44.926 INFO:teuthology.orchestra.run.smithi188.stdout:-rw-r--r-- root/root 96193 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/net/vmw_vsock/vsock.ko 2023-11-26T07:02:44.926 INFO:teuthology.orchestra.run.smithi188.stdout:-rw-r--r-- root/root 10705 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/net/vmw_vsock/vsock_diag.ko 2023-11-26T07:02:44.926 INFO:teuthology.orchestra.run.smithi188.stdout:-rw-r--r-- root/root 17521 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/net/vmw_vsock/vsock_loopback.ko 2023-11-26T07:02:44.927 INFO:teuthology.orchestra.run.smithi188.stdout:drwxr-xr-x root/root 0 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/net/x25/ 2023-11-26T07:02:44.927 INFO:teuthology.orchestra.run.smithi188.stdout:-rw-r--r-- root/root 125177 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/net/x25/x25.ko 2023-11-26T07:02:44.927 INFO:teuthology.orchestra.run.smithi188.stdout:drwxr-xr-x root/root 0 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/net/xfrm/ 2023-11-26T07:02:44.927 INFO:teuthology.orchestra.run.smithi188.stdout:-rw-r--r-- root/root 22657 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/net/xfrm/xfrm_algo.ko 2023-11-26T07:02:44.927 INFO:teuthology.orchestra.run.smithi188.stdout:-rw-r--r-- root/root 20649 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/net/xfrm/xfrm_ipcomp.ko 2023-11-26T07:02:44.927 INFO:teuthology.orchestra.run.smithi188.stdout:-rw-r--r-- root/root 102257 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/net/xfrm/xfrm_user.ko 2023-11-26T07:02:44.927 INFO:teuthology.orchestra.run.smithi188.stdout:drwxr-xr-x root/root 0 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/virt/ 2023-11-26T07:02:44.927 INFO:teuthology.orchestra.run.smithi188.stdout:drwxr-xr-x root/root 0 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/virt/lib/ 2023-11-26T07:02:44.927 INFO:teuthology.orchestra.run.smithi188.stdout:-rw-r--r-- root/root 15537 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/virt/lib/irqbypass.ko 2023-11-26T07:02:44.927 INFO:teuthology.orchestra.run.smithi188.stdout:-rw-r--r-- root/root 640081 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/modules.alias 2023-11-26T07:02:44.927 INFO:teuthology.orchestra.run.smithi188.stdout:-rw-r--r-- root/root 624107 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/modules.alias.bin 2023-11-26T07:02:44.928 INFO:teuthology.orchestra.run.smithi188.stdout:-rw-r--r-- root/root 7663 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/modules.builtin 2023-11-26T07:02:44.928 INFO:teuthology.orchestra.run.smithi188.stdout:-rw-r--r-- root/root 23428 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/modules.builtin.alias.bin 2023-11-26T07:02:44.928 INFO:teuthology.orchestra.run.smithi188.stdout:-rw-r--r-- root/root 9821 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/modules.builtin.bin 2023-11-26T07:02:44.928 INFO:teuthology.orchestra.run.smithi188.stdout:-rw-r--r-- root/root 70004 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/modules.builtin.modinfo 2023-11-26T07:02:44.928 INFO:teuthology.orchestra.run.smithi188.stdout:-rw-r--r-- root/root 146313 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/modules.dep 2023-11-26T07:02:44.928 INFO:teuthology.orchestra.run.smithi188.stdout:-rw-r--r-- root/root 232548 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/modules.dep.bin 2023-11-26T07:02:44.928 INFO:teuthology.orchestra.run.smithi188.stdout:-rw-r--r-- root/root 210 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/modules.devname 2023-11-26T07:02:44.928 INFO:teuthology.orchestra.run.smithi188.stdout:-rw-r--r-- root/root 72606 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/modules.order 2023-11-26T07:02:44.928 INFO:teuthology.orchestra.run.smithi188.stdout:-rw-r--r-- root/root 879 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/modules.softdep 2023-11-26T07:02:44.928 INFO:teuthology.orchestra.run.smithi188.stdout:-rw-r--r-- root/root 254438 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/modules.symbols 2023-11-26T07:02:44.929 INFO:teuthology.orchestra.run.smithi188.stdout:-rw-r--r-- root/root 309363 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/modules.symbols.bin 2023-11-26T07:02:44.929 INFO:teuthology.orchestra.run.smithi188.stdout:drwxr-xr-x root/root 0 2023-11-23 03:47 ./usr/ 2023-11-26T07:02:44.929 INFO:teuthology.orchestra.run.smithi188.stdout:drwxr-xr-x root/root 0 2023-11-23 03:47 ./usr/share/ 2023-11-26T07:02:44.929 INFO:teuthology.orchestra.run.smithi188.stdout:drwxr-xr-x root/root 0 2023-11-23 03:47 ./usr/share/doc/ 2023-11-26T07:02:44.929 INFO:teuthology.orchestra.run.smithi188.stdout:drwxr-xr-x root/root 0 2023-11-23 03:47 ./usr/share/doc/linux-image-6.6.0-g8247bbb5b470/ 2023-11-26T07:02:44.929 INFO:teuthology.orchestra.run.smithi188.stdout:-rw-r--r-- root/root 172 2023-11-23 03:47 ./usr/share/doc/linux-image-6.6.0-g8247bbb5b470/changelog.Debian.gz 2023-11-26T07:02:44.929 INFO:teuthology.orchestra.run.smithi188.stdout:-rw-r--r-- root/root 691 2023-11-23 03:47 ./usr/share/doc/linux-image-6.6.0-g8247bbb5b470/copyright 2023-11-26T07:02:44.930 DEBUG:teuthology.task.kernel:get_image_version: 6.6.0-g8247bbb5b470 2023-11-26T07:02:44.930 INFO:teuthology.task.kernel:searching for kernel 6.6.0-g8247bbb5b470 2023-11-26T07:02:44.930 DEBUG:teuthology.orchestra.run.smithi188:> egrep '(submenu|menuentry.*6.6.0-g8247bbb5b470).*{' /boot/grub/grub.cfg 2023-11-26T07:02:44.937 INFO:teuthology.orchestra.run.smithi188.stdout:submenu 'Advanced options for Ubuntu' $menuentry_id_option 'gnulinux-advanced-7e9e2103-8bb7-4452-b35a-3b1230743254' { 2023-11-26T07:02:44.937 INFO:teuthology.orchestra.run.smithi188.stdout: menuentry 'Ubuntu, with Linux 6.6.0-g8247bbb5b470' --class ubuntu --class gnu-linux --class gnu --class os $menuentry_id_option 'gnulinux-6.6.0-g8247bbb5b470-advanced-7e9e2103-8bb7-4452-b35a-3b1230743254' { 2023-11-26T07:02:44.937 INFO:teuthology.orchestra.run.smithi188.stdout: menuentry 'Ubuntu, with Linux 6.6.0-g8247bbb5b470 (recovery mode)' --class ubuntu --class gnu-linux --class gnu --class os $menuentry_id_option 'gnulinux-6.6.0-g8247bbb5b470-recovery-7e9e2103-8bb7-4452-b35a-3b1230743254' { 2023-11-26T07:02:44.938 INFO:teuthology.task.kernel:submenu_title:Advanced options for Ubuntu> 2023-11-26T07:02:44.938 INFO:teuthology.task.kernel:default_title:Ubuntu, with Linux 6.6.0-g8247bbb5b470 2023-11-26T07:02:44.938 DEBUG:teuthology.orchestra.run.smithi188:> /bin/echo -e 'cat </dev/null && sudo chmod a+x -- '/etc/grub.d/01_ceph_kernel.tmp~' && sudo mv -- '/etc/grub.d/01_ceph_kernel.tmp~' /etc/grub.d/01_ceph_kernel && sudo update-grub && rm /tmp/linux-image.deb && ( sleep 1 && sudo shutdown -r now & ) 2023-11-26T07:02:44.983 DEBUG:teuthology.task.kernel:Waiting for install on ubuntu@smithi188.front.sepia.ceph.com to complete... 2023-11-26T07:02:45.450 INFO:teuthology.orchestra.run.smithi188.stderr:Sourcing file `/etc/default/grub' 2023-11-26T07:02:45.451 INFO:teuthology.orchestra.run.smithi188.stderr:Sourcing file `/etc/default/grub.d/init-select.cfg' 2023-11-26T07:02:45.470 INFO:teuthology.orchestra.run.smithi188.stderr:Generating grub configuration file ... 2023-11-26T07:02:45.706 INFO:teuthology.orchestra.run.smithi188.stderr:Found linux image: /boot/vmlinuz-6.6.0-g8247bbb5b470 2023-11-26T07:02:45.711 INFO:teuthology.orchestra.run.smithi188.stderr:Found initrd image: /boot/initrd.img-6.6.0-g8247bbb5b470 2023-11-26T07:02:46.019 INFO:teuthology.orchestra.run.smithi188.stderr:Found linux image: /boot/vmlinuz-5.15.0-56-generic 2023-11-26T07:02:46.024 INFO:teuthology.orchestra.run.smithi188.stderr:Found initrd image: /boot/initrd.img-5.15.0-56-generic 2023-11-26T07:02:46.068 INFO:teuthology.orchestra.run.smithi188.stderr:Found linux image: /boot/vmlinuz-5.4.0-135-generic 2023-11-26T07:02:46.073 INFO:teuthology.orchestra.run.smithi188.stderr:Found initrd image: /boot/initrd.img-5.4.0-135-generic 2023-11-26T07:02:46.252 INFO:teuthology.orchestra.run.smithi188.stderr:Warning: os-prober will not be executed to detect other bootable partitions. 2023-11-26T07:02:46.252 INFO:teuthology.orchestra.run.smithi188.stderr:Systems on them will not be added to the GRUB boot configuration. 2023-11-26T07:02:46.253 INFO:teuthology.orchestra.run.smithi188.stderr:Check GRUB_DISABLE_OS_PROBER documentation entry. 2023-11-26T07:02:46.270 INFO:teuthology.orchestra.run.smithi188.stderr:done 2023-11-26T07:02:47.329 INFO:teuthology.orchestra.run.smithi143.stderr:Sourcing file `/etc/default/grub' 2023-11-26T07:02:47.344 INFO:teuthology.orchestra.run.smithi143.stderr:Sourcing file `/etc/default/grub.d/init-select.cfg' 2023-11-26T07:02:47.369 INFO:teuthology.orchestra.run.smithi143.stderr:Generating grub configuration file ... 2023-11-26T07:02:47.663 INFO:teuthology.orchestra.run.smithi143.stderr:Found linux image: /boot/vmlinuz-6.6.0-g8247bbb5b470 2023-11-26T07:02:47.683 INFO:teuthology.orchestra.run.smithi143.stderr:Found initrd image: /boot/initrd.img-6.6.0-g8247bbb5b470 2023-11-26T07:02:47.986 INFO:teuthology.orchestra.run.smithi143.stderr:Found linux image: /boot/vmlinuz-5.15.0-56-generic 2023-11-26T07:02:48.003 INFO:teuthology.orchestra.run.smithi143.stderr:Found initrd image: /boot/initrd.img-5.15.0-56-generic 2023-11-26T07:02:48.063 INFO:teuthology.orchestra.run.smithi143.stderr:Found linux image: /boot/vmlinuz-5.4.0-135-generic 2023-11-26T07:02:48.067 INFO:teuthology.orchestra.run.smithi143.stderr:Found initrd image: /boot/initrd.img-5.4.0-135-generic 2023-11-26T07:02:48.235 INFO:teuthology.orchestra.run.smithi143.stderr:Warning: os-prober will not be executed to detect other bootable partitions. 2023-11-26T07:02:48.236 INFO:teuthology.orchestra.run.smithi143.stderr:Systems on them will not be added to the GRUB boot configuration. 2023-11-26T07:02:48.236 INFO:teuthology.orchestra.run.smithi143.stderr:Check GRUB_DISABLE_OS_PROBER documentation entry. 2023-11-26T07:02:48.275 INFO:teuthology.orchestra.run.smithi143.stderr:done 2023-11-26T07:02:48.486 DEBUG:teuthology.orchestra.run.smithi143:> dpkg-deb -c /tmp/linux-image.deb 2023-11-26T07:02:48.536 INFO:teuthology.orchestra.run.smithi143.stdout:drwxr-xr-x root/root 0 2023-11-23 03:47 ./ 2023-11-26T07:02:48.536 INFO:teuthology.orchestra.run.smithi143.stdout:drwxr-xr-x root/root 0 2023-11-23 03:47 ./boot/ 2023-11-26T07:02:48.537 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 4415617 2023-11-23 03:47 ./boot/System.map-6.6.0-g8247bbb5b470 2023-11-26T07:02:48.545 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 169214 2023-11-23 03:47 ./boot/config-6.6.0-g8247bbb5b470 2023-11-26T07:02:48.546 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 11352832 2023-11-23 03:47 ./boot/vmlinuz-6.6.0-g8247bbb5b470 2023-11-26T07:02:48.563 INFO:teuthology.orchestra.run.smithi143.stdout:drwxr-xr-x root/root 0 2023-11-23 03:47 ./etc/ 2023-11-26T07:02:48.563 INFO:teuthology.orchestra.run.smithi143.stdout:drwxr-xr-x root/root 0 2023-11-23 03:47 ./etc/kernel/ 2023-11-26T07:02:48.563 INFO:teuthology.orchestra.run.smithi143.stdout:drwxr-xr-x root/root 0 2023-11-23 03:47 ./etc/kernel/postinst.d/ 2023-11-26T07:02:48.563 INFO:teuthology.orchestra.run.smithi143.stdout:drwxr-xr-x root/root 0 2023-11-23 03:47 ./etc/kernel/postrm.d/ 2023-11-26T07:02:48.563 INFO:teuthology.orchestra.run.smithi143.stdout:drwxr-xr-x root/root 0 2023-11-23 03:47 ./etc/kernel/preinst.d/ 2023-11-26T07:02:48.564 INFO:teuthology.orchestra.run.smithi143.stdout:drwxr-xr-x root/root 0 2023-11-23 03:47 ./etc/kernel/prerm.d/ 2023-11-26T07:02:48.564 INFO:teuthology.orchestra.run.smithi143.stdout:drwxr-xr-x root/root 0 2023-11-23 03:47 ./lib/ 2023-11-26T07:02:48.564 INFO:teuthology.orchestra.run.smithi143.stdout:drwxr-xr-x root/root 0 2023-11-23 03:47 ./lib/modules/ 2023-11-26T07:02:48.564 INFO:teuthology.orchestra.run.smithi143.stdout:drwxr-xr-x root/root 0 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/ 2023-11-26T07:02:48.564 INFO:teuthology.orchestra.run.smithi143.stdout:drwxr-xr-x root/root 0 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/ 2023-11-26T07:02:48.564 INFO:teuthology.orchestra.run.smithi143.stdout:drwxr-xr-x root/root 0 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/arch/ 2023-11-26T07:02:48.564 INFO:teuthology.orchestra.run.smithi143.stdout:drwxr-xr-x root/root 0 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/arch/x86/ 2023-11-26T07:02:48.564 INFO:teuthology.orchestra.run.smithi143.stdout:drwxr-xr-x root/root 0 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/arch/x86/crypto/ 2023-11-26T07:02:48.565 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 490233 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/arch/x86/crypto/aesni-intel.ko 2023-11-26T07:02:48.567 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 25329 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/arch/x86/crypto/blowfish-x86_64.ko 2023-11-26T07:02:48.567 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 29529 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/arch/x86/crypto/camellia-aesni-avx-x86_64.ko 2023-11-26T07:02:48.567 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 30313 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/arch/x86/crypto/camellia-aesni-avx2.ko 2023-11-26T07:02:48.568 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 93113 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/arch/x86/crypto/camellia-x86_64.ko 2023-11-26T07:02:48.568 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 106729 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/arch/x86/crypto/cast5-avx-x86_64.ko 2023-11-26T07:02:48.568 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 152241 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/arch/x86/crypto/cast6-avx-x86_64.ko 2023-11-26T07:02:48.568 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 12553 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/arch/x86/crypto/crc32-pclmul.ko 2023-11-26T07:02:48.568 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 13777 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/arch/x86/crypto/crct10dif-pclmul.ko 2023-11-26T07:02:48.568 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 83889 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/arch/x86/crypto/des3_ede-x86_64.ko 2023-11-26T07:02:48.568 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 18625 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/arch/x86/crypto/ghash-clmulni-intel.ko 2023-11-26T07:02:48.568 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 47225 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/arch/x86/crypto/serpent-avx-x86_64.ko 2023-11-26T07:02:48.568 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 47209 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/arch/x86/crypto/serpent-avx2.ko 2023-11-26T07:02:48.568 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 51801 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/arch/x86/crypto/serpent-sse2-x86_64.ko 2023-11-26T07:02:48.569 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 38937 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/arch/x86/crypto/sha1-ssse3.ko 2023-11-26T07:02:48.569 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 40529 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/arch/x86/crypto/sha256-ssse3.ko 2023-11-26T07:02:48.569 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 58585 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/arch/x86/crypto/sha512-ssse3.ko 2023-11-26T07:02:48.569 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 50497 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/arch/x86/crypto/twofish-avx-x86_64.ko 2023-11-26T07:02:48.569 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 32065 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/arch/x86/crypto/twofish-x86_64-3way.ko 2023-11-26T07:02:48.569 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 13625 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/arch/x86/crypto/twofish-x86_64.ko 2023-11-26T07:02:48.569 INFO:teuthology.orchestra.run.smithi143.stdout:drwxr-xr-x root/root 0 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/arch/x86/kernel/ 2023-11-26T07:02:48.569 INFO:teuthology.orchestra.run.smithi143.stdout:drwxr-xr-x root/root 0 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/arch/x86/kernel/cpu/ 2023-11-26T07:02:48.569 INFO:teuthology.orchestra.run.smithi143.stdout:drwxr-xr-x root/root 0 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/arch/x86/kernel/cpu/mce/ 2023-11-26T07:02:48.569 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 40113 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/arch/x86/kernel/cpu/mce/mce-inject.ko 2023-11-26T07:02:48.569 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 13353 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/arch/x86/kernel/cpuid.ko 2023-11-26T07:02:48.570 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 17937 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/arch/x86/kernel/msr.ko 2023-11-26T07:02:48.570 INFO:teuthology.orchestra.run.smithi143.stdout:drwxr-xr-x root/root 0 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/arch/x86/kvm/ 2023-11-26T07:02:48.570 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 354433 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/arch/x86/kvm/kvm-amd.ko 2023-11-26T07:02:48.570 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 881625 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/arch/x86/kvm/kvm-intel.ko 2023-11-26T07:02:48.570 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 2275865 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/arch/x86/kvm/kvm.ko 2023-11-26T07:02:48.635 INFO:teuthology.orchestra.run.smithi143.stdout:drwxr-xr-x root/root 0 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/crypto/ 2023-11-26T07:02:48.635 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 53697 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/crypto/af_alg.ko 2023-11-26T07:02:48.635 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 23097 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/crypto/algif_hash.ko 2023-11-26T07:02:48.635 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 19985 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/crypto/algif_skcipher.ko 2023-11-26T07:02:48.635 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 17209 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/crypto/ansi_cprng.ko 2023-11-26T07:02:48.635 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 19641 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/crypto/anubis.ko 2023-11-26T07:02:48.635 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 10145 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/crypto/arc4.ko 2023-11-26T07:02:48.635 INFO:teuthology.orchestra.run.smithi143.stdout:drwxr-xr-x root/root 0 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/crypto/asymmetric_keys/ 2023-11-26T07:02:48.636 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 10185 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/crypto/asymmetric_keys/pkcs7_test_key.ko 2023-11-26T07:02:48.636 INFO:teuthology.orchestra.run.smithi143.stdout:drwxr-xr-x root/root 0 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/crypto/async_tx/ 2023-11-26T07:02:48.636 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 11249 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/crypto/async_tx/async_memcpy.ko 2023-11-26T07:02:48.636 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 19473 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/crypto/async_tx/async_pq.ko 2023-11-26T07:02:48.636 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 21561 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/crypto/async_tx/async_raid6_recov.ko 2023-11-26T07:02:48.636 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 13825 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/crypto/async_tx/async_tx.ko 2023-11-26T07:02:48.636 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 17745 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/crypto/async_tx/async_xor.ko 2023-11-26T07:02:48.636 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 17921 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/crypto/async_tx/raid6test.ko 2023-11-26T07:02:48.636 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 20489 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/crypto/authenc.ko 2023-11-26T07:02:48.636 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 22577 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/crypto/authencesn.ko 2023-11-26T07:02:48.636 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 21817 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/crypto/blake2b_generic.ko 2023-11-26T07:02:48.637 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 12729 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/crypto/blowfish_common.ko 2023-11-26T07:02:48.637 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 10537 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/crypto/blowfish_generic.ko 2023-11-26T07:02:48.637 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 38809 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/crypto/camellia_generic.ko 2023-11-26T07:02:48.637 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 27321 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/crypto/cast5_generic.ko 2023-11-26T07:02:48.637 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 17745 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/crypto/cast6_generic.ko 2023-11-26T07:02:48.637 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 10537 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/crypto/cast_common.ko 2023-11-26T07:02:48.637 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 35097 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/crypto/ccm.ko 2023-11-26T07:02:48.637 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 14769 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/crypto/cmac.ko 2023-11-26T07:02:48.637 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 9585 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/crypto/crc32_generic.ko 2023-11-26T07:02:48.637 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 47505 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/crypto/cryptd.ko 2023-11-26T07:02:48.637 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 39505 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/crypto/crypto_engine.ko 2023-11-26T07:02:48.637 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 26953 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/crypto/crypto_simd.ko 2023-11-26T07:02:48.638 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 24737 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/crypto/crypto_user.ko 2023-11-26T07:02:48.638 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 17729 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/crypto/cts.ko 2023-11-26T07:02:48.638 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 10913 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/crypto/des_generic.ko 2023-11-26T07:02:48.638 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 15681 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/crypto/dh_generic.ko 2023-11-26T07:02:48.638 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 11001 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/crypto/echainiv.ko 2023-11-26T07:02:48.638 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 25321 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/crypto/essiv.ko 2023-11-26T07:02:48.638 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 17993 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/crypto/fcrypt.ko 2023-11-26T07:02:48.638 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 26897 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/crypto/khazad.ko 2023-11-26T07:02:48.638 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 15649 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/crypto/lrw.ko 2023-11-26T07:02:48.638 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 11513 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/crypto/lz4.ko 2023-11-26T07:02:48.638 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 11529 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/crypto/lz4hc.ko 2023-11-26T07:02:48.639 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 11697 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/crypto/md4.ko 2023-11-26T07:02:48.639 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 9729 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/crypto/michael_mic.ko 2023-11-26T07:02:48.639 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 10673 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/crypto/pcbc.ko 2023-11-26T07:02:48.639 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 18745 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/crypto/pcrypt.ko 2023-11-26T07:02:48.639 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 16049 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/crypto/rmd160.ko 2023-11-26T07:02:48.639 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 29393 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/crypto/seed.ko 2023-11-26T07:02:48.639 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 27081 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/crypto/serpent_generic.ko 2023-11-26T07:02:48.639 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 149193 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/crypto/tcrypt.ko 2023-11-26T07:02:48.639 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 10881 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/crypto/tea.ko 2023-11-26T07:02:48.639 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 28777 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/crypto/twofish_common.ko 2023-11-26T07:02:48.639 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 13169 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/crypto/twofish_generic.ko 2023-11-26T07:02:48.639 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 17401 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/crypto/vmac.ko 2023-11-26T07:02:48.640 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 36185 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/crypto/wp512.ko 2023-11-26T07:02:48.640 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 14105 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/crypto/xcbc.ko 2023-11-26T07:02:48.640 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 26665 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/crypto/xor.ko 2023-11-26T07:02:48.640 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 18713 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/crypto/xts.ko 2023-11-26T07:02:48.640 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 9481 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/crypto/xxhash_generic.ko 2023-11-26T07:02:48.640 INFO:teuthology.orchestra.run.smithi143.stdout:drwxr-xr-x root/root 0 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/ 2023-11-26T07:02:48.640 INFO:teuthology.orchestra.run.smithi143.stdout:drwxr-xr-x root/root 0 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/acpi/ 2023-11-26T07:02:48.640 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 18249 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/acpi/acpi_extlog.ko 2023-11-26T07:02:48.640 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 26985 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/acpi/acpi_ipmi.ko 2023-11-26T07:02:48.640 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 28793 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/acpi/acpi_pad.ko 2023-11-26T07:02:48.641 INFO:teuthology.orchestra.run.smithi143.stdout:drwxr-xr-x root/root 0 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/acpi/apei/ 2023-11-26T07:02:48.641 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 31241 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/acpi/apei/einj.ko 2023-11-26T07:02:48.641 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 12801 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/acpi/ec_sys.ko 2023-11-26T07:02:48.641 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 14937 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/acpi/platform_profile.ko 2023-11-26T07:02:48.641 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 24401 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/acpi/sbs.ko 2023-11-26T07:02:48.641 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 19521 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/acpi/sbshc.ko 2023-11-26T07:02:48.641 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 103025 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/acpi/video.ko 2023-11-26T07:02:48.641 INFO:teuthology.orchestra.run.smithi143.stdout:drwxr-xr-x root/root 0 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/ata/ 2023-11-26T07:02:48.641 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 21601 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/ata/acard-ahci.ko 2023-11-26T07:02:48.641 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 88865 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/ata/ahci.ko 2023-11-26T07:02:48.641 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 14729 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/ata/ahci_platform.ko 2023-11-26T07:02:48.642 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 88289 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/ata/libahci.ko 2023-11-26T07:02:48.642 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 34985 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/ata/libahci_platform.ko 2023-11-26T07:02:48.642 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 17137 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/ata/pata_acpi.ko 2023-11-26T07:02:48.642 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 30433 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/ata/pata_ali.ko 2023-11-26T07:02:48.642 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 33505 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/ata/pata_amd.ko 2023-11-26T07:02:48.642 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 21265 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/ata/pata_artop.ko 2023-11-26T07:02:48.642 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 21121 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/ata/pata_atiixp.ko 2023-11-26T07:02:48.642 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 24945 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/ata/pata_atp867x.ko 2023-11-26T07:02:48.642 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 17657 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/ata/pata_cmd640.ko 2023-11-26T07:02:48.642 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 25185 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/ata/pata_cmd64x.ko 2023-11-26T07:02:48.642 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 15049 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/ata/pata_cypress.ko 2023-11-26T07:02:48.643 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 17849 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/ata/pata_efar.ko 2023-11-26T07:02:48.643 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 21945 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/ata/pata_hpt366.ko 2023-11-26T07:02:48.643 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 33977 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/ata/pata_hpt37x.ko 2023-11-26T07:02:48.643 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 23129 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/ata/pata_hpt3x2n.ko 2023-11-26T07:02:48.643 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 16921 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/ata/pata_hpt3x3.ko 2023-11-26T07:02:48.643 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 16993 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/ata/pata_it8213.ko 2023-11-26T07:02:48.643 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 35273 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/ata/pata_it821x.ko 2023-11-26T07:02:48.643 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 12889 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/ata/pata_jmicron.ko 2023-11-26T07:02:48.643 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 51481 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/ata/pata_legacy.ko 2023-11-26T07:02:48.643 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 15185 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/ata/pata_marvell.ko 2023-11-26T07:02:48.643 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 16281 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/ata/pata_mpiix.ko 2023-11-26T07:02:48.643 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 12833 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/ata/pata_netcell.ko 2023-11-26T07:02:48.644 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 16817 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/ata/pata_ninja32.ko 2023-11-26T07:02:48.644 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 15105 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/ata/pata_ns87410.ko 2023-11-26T07:02:48.644 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 17361 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/ata/pata_ns87415.ko 2023-11-26T07:02:48.644 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 16569 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/ata/pata_oldpiix.ko 2023-11-26T07:02:48.644 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 15377 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/ata/pata_opti.ko 2023-11-26T07:02:48.644 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 20681 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/ata/pata_optidma.ko 2023-11-26T07:02:48.644 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 32585 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/ata/pata_pcmcia.ko 2023-11-26T07:02:48.644 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 37793 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/ata/pata_pdc2027x.ko 2023-11-26T07:02:48.644 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 20561 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/ata/pata_pdc202xx_old.ko 2023-11-26T07:02:48.644 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 14225 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/ata/pata_piccolo.ko 2023-11-26T07:02:48.644 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 15449 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/ata/pata_radisys.ko 2023-11-26T07:02:48.644 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 19521 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/ata/pata_rdc.ko 2023-11-26T07:02:48.644 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 15329 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/ata/pata_rz1000.ko 2023-11-26T07:02:48.645 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 13937 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/ata/pata_sch.ko 2023-11-26T07:02:48.645 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 23097 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/ata/pata_serverworks.ko 2023-11-26T07:02:48.645 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 23457 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/ata/pata_sil680.ko 2023-11-26T07:02:48.645 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 19697 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/ata/pata_sl82c105.ko 2023-11-26T07:02:48.645 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 16009 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/ata/pata_triflex.ko 2023-11-26T07:02:48.645 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 30433 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/ata/pata_via.ko 2023-11-26T07:02:48.645 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 22793 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/ata/pdc_adma.ko 2023-11-26T07:02:48.645 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 29121 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/ata/sata_inic162x.ko 2023-11-26T07:02:48.645 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 83345 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/ata/sata_mv.ko 2023-11-26T07:02:48.645 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 70393 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/ata/sata_nv.ko 2023-11-26T07:02:48.645 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 37377 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/ata/sata_promise.ko 2023-11-26T07:02:48.646 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 23041 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/ata/sata_qstor.ko 2023-11-26T07:02:48.646 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 29273 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/ata/sata_sil.ko 2023-11-26T07:02:48.646 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 38337 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/ata/sata_sil24.ko 2023-11-26T07:02:48.646 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 17945 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/ata/sata_sis.ko 2023-11-26T07:02:48.646 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 20441 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/ata/sata_svw.ko 2023-11-26T07:02:48.646 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 44793 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/ata/sata_sx4.ko 2023-11-26T07:02:48.646 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 15601 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/ata/sata_uli.ko 2023-11-26T07:02:48.646 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 35065 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/ata/sata_via.ko 2023-11-26T07:02:48.646 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 20065 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/ata/sata_vsc.ko 2023-11-26T07:02:48.646 INFO:teuthology.orchestra.run.smithi143.stdout:drwxr-xr-x root/root 0 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/auxdisplay/ 2023-11-26T07:02:48.647 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 16289 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/auxdisplay/ks0108.ko 2023-11-26T07:02:48.647 INFO:teuthology.orchestra.run.smithi143.stdout:drwxr-xr-x root/root 0 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/bcma/ 2023-11-26T07:02:48.647 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 124473 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/bcma/bcma.ko 2023-11-26T07:02:48.647 INFO:teuthology.orchestra.run.smithi143.stdout:drwxr-xr-x root/root 0 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/block/ 2023-11-26T07:02:48.647 INFO:teuthology.orchestra.run.smithi143.stdout:drwxr-xr-x root/root 0 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/block/aoe/ 2023-11-26T07:02:48.647 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 113665 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/block/aoe/aoe.ko 2023-11-26T07:02:48.647 INFO:teuthology.orchestra.run.smithi143.stdout:drwxr-xr-x root/root 0 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/block/drbd/ 2023-11-26T07:02:48.647 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 965585 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/block/drbd/drbd.ko 2023-11-26T07:02:48.647 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 173225 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/block/floppy.ko 2023-11-26T07:02:48.647 INFO:teuthology.orchestra.run.smithi143.stdout:drwxr-xr-x root/root 0 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/block/mtip32xx/ 2023-11-26T07:02:48.647 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 94297 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/block/mtip32xx/mtip32xx.ko 2023-11-26T07:02:48.647 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 115793 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/block/nbd.ko 2023-11-26T07:02:48.648 INFO:teuthology.orchestra.run.smithi143.stdout:drwxr-xr-x root/root 0 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/block/null_blk/ 2023-11-26T07:02:48.648 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 110385 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/block/null_blk/null_blk.ko 2023-11-26T07:02:48.648 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 98193 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/block/pktcdvd.ko 2023-11-26T07:02:48.648 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 263769 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/block/rbd.ko 2023-11-26T07:02:48.648 INFO:teuthology.orchestra.run.smithi143.stdout:drwxr-xr-x root/root 0 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/block/xen-blkback/ 2023-11-26T07:02:48.648 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 98145 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/block/xen-blkback/xen-blkback.ko 2023-11-26T07:02:48.648 INFO:teuthology.orchestra.run.smithi143.stdout:drwxr-xr-x root/root 0 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/block/zram/ 2023-11-26T07:02:48.648 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 60217 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/block/zram/zram.ko 2023-11-26T07:02:48.648 INFO:teuthology.orchestra.run.smithi143.stdout:drwxr-xr-x root/root 0 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/char/ 2023-11-26T07:02:48.648 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 30705 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/char/applicom.ko 2023-11-26T07:02:48.648 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 12009 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/char/hangcheck-timer.ko 2023-11-26T07:02:48.648 INFO:teuthology.orchestra.run.smithi143.stdout:drwxr-xr-x root/root 0 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/char/hw_random/ 2023-11-26T07:02:48.649 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 13889 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/char/hw_random/amd-rng.ko 2023-11-26T07:02:48.649 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 18809 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/char/hw_random/intel-rng.ko 2023-11-26T07:02:48.649 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 13169 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/char/hw_random/timeriomem-rng.ko 2023-11-26T07:02:48.649 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 10673 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/char/hw_random/via-rng.ko 2023-11-26T07:02:48.649 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 15633 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/char/hw_random/virtio-rng.ko 2023-11-26T07:02:48.649 INFO:teuthology.orchestra.run.smithi143.stdout:drwxr-xr-x root/root 0 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/char/ipmi/ 2023-11-26T07:02:48.649 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 30713 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/char/ipmi/ipmi_devintf.ko 2023-11-26T07:02:48.649 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 148601 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/char/ipmi/ipmi_msghandler.ko 2023-11-26T07:02:48.649 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 26569 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/char/ipmi/ipmi_poweroff.ko 2023-11-26T07:02:48.649 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 139313 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/char/ipmi/ipmi_si.ko 2023-11-26T07:02:48.649 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 70921 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/char/ipmi/ipmi_ssif.ko 2023-11-26T07:02:48.649 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 53513 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/char/ipmi/ipmi_watchdog.ko 2023-11-26T07:02:48.650 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 36321 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/char/lp.ko 2023-11-26T07:02:48.650 INFO:teuthology.orchestra.run.smithi143.stdout:drwxr-xr-x root/root 0 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/char/mwave/ 2023-11-26T07:02:48.650 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 84097 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/char/mwave/mwave.ko 2023-11-26T07:02:48.650 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 25873 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/char/nvram.ko 2023-11-26T07:02:48.650 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 32169 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/char/ppdev.ko 2023-11-26T07:02:48.650 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 51113 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/char/tlclk.ko 2023-11-26T07:02:48.650 INFO:teuthology.orchestra.run.smithi143.stdout:drwxr-xr-x root/root 0 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/char/tpm/ 2023-11-26T07:02:48.650 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 16401 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/char/tpm/tpm_atmel.ko 2023-11-26T07:02:48.650 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 15881 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/char/tpm/tpm_i2c_atmel.ko 2023-11-26T07:02:48.650 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 23473 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/char/tpm/tpm_i2c_infineon.ko 2023-11-26T07:02:48.650 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 32985 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/char/tpm/tpm_i2c_nuvoton.ko 2023-11-26T07:02:48.651 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 30121 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/char/tpm/tpm_infineon.ko 2023-11-26T07:02:48.651 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 22697 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/char/tpm/tpm_nsc.ko 2023-11-26T07:02:48.651 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 23561 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/char/tpm/xen-tpmfront.ko 2023-11-26T07:02:48.651 INFO:teuthology.orchestra.run.smithi143.stdout:drwxr-xr-x root/root 0 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/char/xillybus/ 2023-11-26T07:02:48.651 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 17001 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/char/xillybus/xillybus_class.ko 2023-11-26T07:02:48.651 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 50945 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/char/xillybus/xillybus_core.ko 2023-11-26T07:02:48.651 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 12393 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/char/xillybus/xillybus_pcie.ko 2023-11-26T07:02:48.651 INFO:teuthology.orchestra.run.smithi143.stdout:drwxr-xr-x root/root 0 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/clk/ 2023-11-26T07:02:48.651 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 12489 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/clk/clk-palmas.ko 2023-11-26T07:02:48.651 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 47849 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/clk/clk-si5351.ko 2023-11-26T07:02:48.651 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 11457 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/clk/clk-twl6040.ko 2023-11-26T07:02:48.652 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 19249 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/clk/clk-wm831x.ko 2023-11-26T07:02:48.652 INFO:teuthology.orchestra.run.smithi143.stdout:drwxr-xr-x root/root 0 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/cpufreq/ 2023-11-26T07:02:48.652 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 10097 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/cpufreq/amd_freq_sensitivity.ko 2023-11-26T07:02:48.652 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 16833 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/cpufreq/p4-clockmod.ko 2023-11-26T07:02:48.652 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 25297 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/cpufreq/speedstep-lib.ko 2023-11-26T07:02:48.652 INFO:teuthology.orchestra.run.smithi143.stdout:drwxr-xr-x root/root 0 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/crypto/ 2023-11-26T07:02:48.652 INFO:teuthology.orchestra.run.smithi143.stdout:drwxr-xr-x root/root 0 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/crypto/ccp/ 2023-11-26T07:02:48.652 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 70801 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/crypto/ccp/ccp-crypto.ko 2023-11-26T07:02:48.652 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 243025 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/crypto/ccp/ccp.ko 2023-11-26T07:02:48.652 INFO:teuthology.orchestra.run.smithi143.stdout:drwxr-xr-x root/root 0 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/crypto/intel/ 2023-11-26T07:02:48.652 INFO:teuthology.orchestra.run.smithi143.stdout:drwxr-xr-x root/root 0 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/crypto/intel/qat/ 2023-11-26T07:02:48.653 INFO:teuthology.orchestra.run.smithi143.stdout:drwxr-xr-x root/root 0 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/crypto/intel/qat/qat_common/ 2023-11-26T07:02:48.653 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 438761 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/crypto/intel/qat/qat_common/intel_qat.ko 2023-11-26T07:02:48.653 INFO:teuthology.orchestra.run.smithi143.stdout:drwxr-xr-x root/root 0 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/crypto/intel/qat/qat_dh895xcc/ 2023-11-26T07:02:48.653 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 24081 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/crypto/intel/qat/qat_dh895xcc/qat_dh895xcc.ko 2023-11-26T07:02:48.653 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 19937 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/crypto/padlock-aes.ko 2023-11-26T07:02:48.653 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 21537 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/crypto/padlock-sha.ko 2023-11-26T07:02:48.653 INFO:teuthology.orchestra.run.smithi143.stdout:drwxr-xr-x root/root 0 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/crypto/virtio/ 2023-11-26T07:02:48.653 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 70273 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/crypto/virtio/virtio_crypto.ko 2023-11-26T07:02:48.653 INFO:teuthology.orchestra.run.smithi143.stdout:drwxr-xr-x root/root 0 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/dca/ 2023-11-26T07:02:48.653 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 29785 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/dca/dca.ko 2023-11-26T07:02:48.653 INFO:teuthology.orchestra.run.smithi143.stdout:drwxr-xr-x root/root 0 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/dma/ 2023-11-26T07:02:48.653 INFO:teuthology.orchestra.run.smithi143.stdout:drwxr-xr-x root/root 0 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/dma/dw/ 2023-11-26T07:02:48.654 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 15857 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/dma/dw/dw_dmac.ko 2023-11-26T07:02:48.654 INFO:teuthology.orchestra.run.smithi143.stdout:drwxr-xr-x root/root 0 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/dma/ioat/ 2023-11-26T07:02:48.654 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 122569 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/dma/ioat/ioatdma.ko 2023-11-26T07:02:48.654 INFO:teuthology.orchestra.run.smithi143.stdout:drwxr-xr-x root/root 0 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/edac/ 2023-11-26T07:02:48.654 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 79953 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/edac/amd64_edac.ko 2023-11-26T07:02:48.654 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 35769 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/edac/e752x_edac.ko 2023-11-26T07:02:48.654 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 63617 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/edac/edac_mce_amd.ko 2023-11-26T07:02:48.654 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 18105 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/edac/i3000_edac.ko 2023-11-26T07:02:48.654 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 18065 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/edac/i3200_edac.ko 2023-11-26T07:02:48.654 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 22537 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/edac/i5100_edac.ko 2023-11-26T07:02:48.654 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 25657 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/edac/i5400_edac.ko 2023-11-26T07:02:48.655 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 24913 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/edac/i7300_edac.ko 2023-11-26T07:02:48.655 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 52041 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/edac/i7core_edac.ko 2023-11-26T07:02:48.655 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 18433 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/edac/i82975x_edac.ko 2023-11-26T07:02:48.655 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 20017 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/edac/ie31200_edac.ko 2023-11-26T07:02:48.655 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 56825 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/edac/sb_edac.ko 2023-11-26T07:02:48.655 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 17529 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/edac/x38_edac.ko 2023-11-26T07:02:48.655 INFO:teuthology.orchestra.run.smithi143.stdout:drwxr-xr-x root/root 0 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/extcon/ 2023-11-26T07:02:48.655 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 12673 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/extcon/extcon-gpio.ko 2023-11-26T07:02:48.655 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 27185 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/extcon/extcon-max14577.ko 2023-11-26T07:02:48.655 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 29569 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/extcon/extcon-max77693.ko 2023-11-26T07:02:48.655 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 24745 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/extcon/extcon-max8997.ko 2023-11-26T07:02:48.655 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 26537 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/extcon/extcon-palmas.ko 2023-11-26T07:02:48.656 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 23713 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/extcon/extcon-rt8973a.ko 2023-11-26T07:02:48.656 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 27785 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/extcon/extcon-sm5502.ko 2023-11-26T07:02:48.656 INFO:teuthology.orchestra.run.smithi143.stdout:drwxr-xr-x root/root 0 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/firmware/ 2023-11-26T07:02:48.656 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 34977 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/firmware/dmi-sysfs.ko 2023-11-26T07:02:48.656 INFO:teuthology.orchestra.run.smithi143.stdout:drwxr-xr-x root/root 0 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/firmware/efi/ 2023-11-26T07:02:48.656 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 15521 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/firmware/efi/efi-pstore.ko 2023-11-26T07:02:48.656 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 20729 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/firmware/iscsi_ibft.ko 2023-11-26T07:02:48.656 INFO:teuthology.orchestra.run.smithi143.stdout:drwxr-xr-x root/root 0 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/gpio/ 2023-11-26T07:02:48.656 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 12305 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/gpio/gpio-adp5520.ko 2023-11-26T07:02:48.656 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 20153 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/gpio/gpio-amd8111.ko 2023-11-26T07:02:48.656 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 14993 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/gpio/gpio-arizona.ko 2023-11-26T07:02:48.656 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 13841 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/gpio/gpio-da9052.ko 2023-11-26T07:02:48.657 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 11921 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/gpio/gpio-da9055.ko 2023-11-26T07:02:48.657 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 22065 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/gpio/gpio-dln2.ko 2023-11-26T07:02:48.657 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 77849 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/gpio/gpio-f7188x.ko 2023-11-26T07:02:48.657 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 28937 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/gpio/gpio-generic.ko 2023-11-26T07:02:48.657 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 19569 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/gpio/gpio-ich.ko 2023-11-26T07:02:48.657 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 11905 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/gpio/gpio-janz-ttl.ko 2023-11-26T07:02:48.657 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 14113 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/gpio/gpio-kempld.ko 2023-11-26T07:02:48.657 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 12537 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/gpio/gpio-lp3943.ko 2023-11-26T07:02:48.657 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 9505 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/gpio/gpio-max7300.ko 2023-11-26T07:02:48.657 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 11905 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/gpio/gpio-max730x.ko 2023-11-26T07:02:48.657 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 22073 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/gpio/gpio-max732x.ko 2023-11-26T07:02:48.657 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 25145 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/gpio/gpio-ml-ioh.ko 2023-11-26T07:02:48.658 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 47073 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/gpio/gpio-pca953x.ko 2023-11-26T07:02:48.658 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 26825 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/gpio/gpio-pcf857x.ko 2023-11-26T07:02:48.658 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 14377 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/gpio/gpio-rdc321x.ko 2023-11-26T07:02:48.658 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 20593 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/gpio/gpio-sch.ko 2023-11-26T07:02:48.658 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 19505 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/gpio/gpio-sch311x.ko 2023-11-26T07:02:48.658 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 12145 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/gpio/gpio-tps65912.ko 2023-11-26T07:02:48.658 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 21057 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/gpio/gpio-twl4030.ko 2023-11-26T07:02:48.658 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 11425 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/gpio/gpio-twl6040.ko 2023-11-26T07:02:48.658 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 20601 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/gpio/gpio-viperboard.ko 2023-11-26T07:02:48.658 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 15377 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/gpio/gpio-vx855.ko 2023-11-26T07:02:48.658 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 16281 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/gpio/gpio-wm831x.ko 2023-11-26T07:02:48.659 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 11617 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/gpio/gpio-wm8350.ko 2023-11-26T07:02:48.659 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 16753 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/gpio/gpio-wm8994.ko 2023-11-26T07:02:48.659 INFO:teuthology.orchestra.run.smithi143.stdout:drwxr-xr-x root/root 0 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hid/ 2023-11-26T07:02:48.659 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 11577 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hid/hid-a4tech.ko 2023-11-26T07:02:48.659 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 39169 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hid/hid-apple.ko 2023-11-26T07:02:48.659 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 14481 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hid/hid-appleir.ko 2023-11-26T07:02:48.659 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 8353 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hid/hid-aureal.ko 2023-11-26T07:02:48.659 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 15169 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hid/hid-axff.ko 2023-11-26T07:02:48.659 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 11049 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hid/hid-belkin.ko 2023-11-26T07:02:48.659 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 10313 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hid/hid-cherry.ko 2023-11-26T07:02:48.660 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 16041 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hid/hid-chicony.ko 2023-11-26T07:02:48.660 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 60649 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hid/hid-cp2112.ko 2023-11-26T07:02:48.660 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 11049 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hid/hid-cypress.ko 2023-11-26T07:02:48.660 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 16385 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hid/hid-dr.ko 2023-11-26T07:02:48.660 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 10697 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hid/hid-elecom.ko 2023-11-26T07:02:48.660 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 17665 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hid/hid-elo.ko 2023-11-26T07:02:48.660 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 13457 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hid/hid-emsff.ko 2023-11-26T07:02:48.660 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 10105 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hid/hid-ezkey.ko 2023-11-26T07:02:48.660 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 14305 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hid/hid-gaff.ko 2023-11-26T07:02:48.660 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 9225 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hid/hid-generic.ko 2023-11-26T07:02:48.660 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 17129 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hid/hid-gt683r.ko 2023-11-26T07:02:48.661 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 12681 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hid/hid-gyration.ko 2023-11-26T07:02:48.661 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 10161 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hid/hid-holtek-kbd.ko 2023-11-26T07:02:48.661 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 10457 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hid/hid-holtek-mouse.ko 2023-11-26T07:02:48.661 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 14449 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hid/hid-holtekff.ko 2023-11-26T07:02:48.661 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 19105 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hid/hid-hyperv.ko 2023-11-26T07:02:48.661 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 10489 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hid/hid-icade.ko 2023-11-26T07:02:48.661 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 9241 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hid/hid-kensington.ko 2023-11-26T07:02:48.661 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 8545 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hid/hid-keytouch.ko 2023-11-26T07:02:48.661 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 17201 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hid/hid-kye.ko 2023-11-26T07:02:48.661 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 10809 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hid/hid-lcpower.ko 2023-11-26T07:02:48.661 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 20497 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hid/hid-led.ko 2023-11-26T07:02:48.661 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 42297 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hid/hid-lenovo.ko 2023-11-26T07:02:48.661 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 30697 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hid/hid-lg-g15.ko 2023-11-26T07:02:48.662 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 58185 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hid/hid-logitech-dj.ko 2023-11-26T07:02:48.662 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 101481 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hid/hid-logitech-hidpp.ko 2023-11-26T07:02:48.662 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 80993 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hid/hid-logitech.ko 2023-11-26T07:02:48.662 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 30801 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hid/hid-magicmouse.ko 2023-11-26T07:02:48.662 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 21969 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hid/hid-microsoft.ko 2023-11-26T07:02:48.662 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 10361 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hid/hid-monterey.ko 2023-11-26T07:02:48.662 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 50161 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hid/hid-multitouch.ko 2023-11-26T07:02:48.662 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 32593 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hid/hid-ntrig.ko 2023-11-26T07:02:48.662 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 8857 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hid/hid-ortek.ko 2023-11-26T07:02:48.662 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 9121 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hid/hid-penmount.ko 2023-11-26T07:02:48.662 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 12289 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hid/hid-petalynx.ko 2023-11-26T07:02:48.663 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 72937 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hid/hid-picolcd.ko 2023-11-26T07:02:48.665 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 14225 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hid/hid-pl.ko 2023-11-26T07:02:48.665 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 15105 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hid/hid-plantronics.ko 2023-11-26T07:02:48.665 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 8417 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hid/hid-primax.ko 2023-11-26T07:02:48.665 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 35393 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hid/hid-rmi.ko 2023-11-26T07:02:48.665 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 20833 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hid/hid-roccat-arvo.ko 2023-11-26T07:02:48.665 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 14065 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hid/hid-roccat-common.ko 2023-11-26T07:02:48.665 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 31425 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hid/hid-roccat-isku.ko 2023-11-26T07:02:48.665 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 29729 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hid/hid-roccat-kone.ko 2023-11-26T07:02:48.666 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 31129 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hid/hid-roccat-koneplus.ko 2023-11-26T07:02:48.666 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 21137 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hid/hid-roccat-konepure.ko 2023-11-26T07:02:48.666 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 29489 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hid/hid-roccat-kovaplus.ko 2023-11-26T07:02:48.666 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 13305 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hid/hid-roccat-lua.ko 2023-11-26T07:02:48.666 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 29313 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hid/hid-roccat-pyra.ko 2023-11-26T07:02:48.666 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 27937 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hid/hid-roccat-ryos.ko 2023-11-26T07:02:48.666 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 19521 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hid/hid-roccat-savu.ko 2023-11-26T07:02:48.666 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 23337 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hid/hid-roccat.ko 2023-11-26T07:02:48.666 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 13681 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hid/hid-saitek.ko 2023-11-26T07:02:48.666 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 15193 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hid/hid-samsung.ko 2023-11-26T07:02:48.666 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 36817 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hid/hid-sensor-hub.ko 2023-11-26T07:02:48.666 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 13601 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hid/hid-sjoy.ko 2023-11-26T07:02:48.667 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 51249 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hid/hid-sony.ko 2023-11-26T07:02:48.667 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 8313 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hid/hid-speedlink.ko 2023-11-26T07:02:48.667 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 25297 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hid/hid-steelseries.ko 2023-11-26T07:02:48.667 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 9817 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hid/hid-sunplus.ko 2023-11-26T07:02:48.667 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 16993 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hid/hid-thrustmaster.ko 2023-11-26T07:02:48.667 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 10537 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hid/hid-tivo.ko 2023-11-26T07:02:48.667 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 15089 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hid/hid-tmff.ko 2023-11-26T07:02:48.667 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 14385 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hid/hid-topseed.ko 2023-11-26T07:02:48.667 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 16473 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hid/hid-twinhan.ko 2023-11-26T07:02:48.667 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 71073 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hid/hid-uclogic.ko 2023-11-26T07:02:48.667 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 10921 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hid/hid-waltop.ko 2023-11-26T07:02:48.667 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 143473 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hid/hid-wiimote.ko 2023-11-26T07:02:48.668 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 8153 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hid/hid-xinmo.ko 2023-11-26T07:02:48.668 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 13321 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hid/hid-zpff.ko 2023-11-26T07:02:48.668 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 16665 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hid/hid-zydacron.ko 2023-11-26T07:02:48.668 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 267825 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hid/hid.ko 2023-11-26T07:02:48.668 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 32321 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hid/uhid.ko 2023-11-26T07:02:48.668 INFO:teuthology.orchestra.run.smithi143.stdout:drwxr-xr-x root/root 0 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hid/usbhid/ 2023-11-26T07:02:48.668 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 125721 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hid/usbhid/usbhid.ko 2023-11-26T07:02:48.668 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 19137 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hid/usbhid/usbkbd.ko 2023-11-26T07:02:48.668 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 14953 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hid/usbhid/usbmouse.ko 2023-11-26T07:02:48.668 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 217873 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hid/wacom.ko 2023-11-26T07:02:48.668 INFO:teuthology.orchestra.run.smithi143.stdout:drwxr-xr-x root/root 0 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hv/ 2023-11-26T07:02:48.668 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 64201 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hv/hv_balloon.ko 2023-11-26T07:02:48.669 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 89625 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hv/hv_utils.ko 2023-11-26T07:02:48.669 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 305465 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hv/hv_vmbus.ko 2023-11-26T07:02:48.669 INFO:teuthology.orchestra.run.smithi143.stdout:drwxr-xr-x root/root 0 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/ 2023-11-26T07:02:48.669 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 66521 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/abituguru.ko 2023-11-26T07:02:48.669 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 61393 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/abituguru3.ko 2023-11-26T07:02:48.669 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 34745 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/acpi_power_meter.ko 2023-11-26T07:02:48.669 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 17505 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/ad7414.ko 2023-11-26T07:02:48.669 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 19313 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/ad7418.ko 2023-11-26T07:02:48.669 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 25257 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/adc128d818.ko 2023-11-26T07:02:48.669 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 31729 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/adm1025.ko 2023-11-26T07:02:48.669 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 97977 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/adm1026.ko 2023-11-26T07:02:48.670 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 19745 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/adm1029.ko 2023-11-26T07:02:48.670 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 45553 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/adm1031.ko 2023-11-26T07:02:48.670 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 26289 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/adm9240.ko 2023-11-26T07:02:48.670 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 14769 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/ads7828.ko 2023-11-26T07:02:48.670 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 10769 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/adt7410.ko 2023-11-26T07:02:48.670 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 22049 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/adt7411.ko 2023-11-26T07:02:48.670 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 74433 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/adt7462.ko 2023-11-26T07:02:48.670 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 39585 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/adt7470.ko 2023-11-26T07:02:48.670 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 67929 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/adt7475.ko 2023-11-26T07:02:48.671 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 19625 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/adt7x10.ko 2023-11-26T07:02:48.671 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 39073 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/amc6821.ko 2023-11-26T07:02:48.671 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 55201 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/applesmc.ko 2023-11-26T07:02:48.671 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 49913 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/asb100.ko 2023-11-26T07:02:48.671 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 58617 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/asc7621.ko 2023-11-26T07:02:48.671 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 40825 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/asus_atk0110.ko 2023-11-26T07:02:48.671 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 17225 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/atxp1.ko 2023-11-26T07:02:48.671 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 28281 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/coretemp.ko 2023-11-26T07:02:48.671 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 24593 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/da9052-hwmon.ko 2023-11-26T07:02:48.671 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 17265 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/da9055-hwmon.ko 2023-11-26T07:02:48.671 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 44465 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/dell-smm-hwmon.ko 2023-11-26T07:02:48.672 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 99601 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/dme1737.ko 2023-11-26T07:02:48.672 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 19489 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/ds1621.ko 2023-11-26T07:02:48.672 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 15321 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/ds620.ko 2023-11-26T07:02:48.672 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 29777 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/emc1403.ko 2023-11-26T07:02:48.672 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 33185 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/emc2103.ko 2023-11-26T07:02:48.672 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 30321 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/emc6w201.ko 2023-11-26T07:02:48.672 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 66985 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/f71805f.ko 2023-11-26T07:02:48.672 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 94457 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/f71882fg.ko 2023-11-26T07:02:48.672 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 41401 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/f75375s.ko 2023-11-26T07:02:48.672 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 21305 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/fam15h_power.ko 2023-11-26T07:02:48.672 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 55449 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/fschmd.ko 2023-11-26T07:02:48.672 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 14481 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/g760a.ko 2023-11-26T07:02:48.673 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 24017 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/g762.ko 2023-11-26T07:02:48.673 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 46801 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/gl518sm.ko 2023-11-26T07:02:48.673 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 45529 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/gl520sm.ko 2023-11-26T07:02:48.673 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 13273 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/hih6130.ko 2023-11-26T07:02:48.673 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 9745 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/hwmon-vid.ko 2023-11-26T07:02:48.673 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 11553 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/i5500_temp.ko 2023-11-26T07:02:48.673 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 22601 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/i5k_amb.ko 2023-11-26T07:02:48.673 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 34529 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/ibmaem.ko 2023-11-26T07:02:48.673 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 21257 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/ibmpex.ko 2023-11-26T07:02:48.673 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 24873 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/ina209.ko 2023-11-26T07:02:48.673 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 13401 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/ina2xx.ko 2023-11-26T07:02:48.673 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 130449 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/it87.ko 2023-11-26T07:02:48.674 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 18841 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/jc42.ko 2023-11-26T07:02:48.674 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 24329 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/k10temp.ko 2023-11-26T07:02:48.674 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 13553 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/k8temp.ko 2023-11-26T07:02:48.674 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 18929 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/lineage-pem.ko 2023-11-26T07:02:48.674 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 48601 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/lm63.ko 2023-11-26T07:02:48.674 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 16697 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/lm73.ko 2023-11-26T07:02:48.674 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 36777 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/lm75.ko 2023-11-26T07:02:48.674 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 19537 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/lm77.ko 2023-11-26T07:02:48.674 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 34593 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/lm78.ko 2023-11-26T07:02:48.674 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 31745 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/lm80.ko 2023-11-26T07:02:48.674 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 16889 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/lm83.ko 2023-11-26T07:02:48.674 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 72833 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/lm85.ko 2023-11-26T07:02:48.675 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 46153 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/lm87.ko 2023-11-26T07:02:48.675 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 37913 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/lm90.ko 2023-11-26T07:02:48.675 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 19745 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/lm92.ko 2023-11-26T07:02:48.675 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 108841 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/lm93.ko 2023-11-26T07:02:48.675 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 33393 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/lm95234.ko 2023-11-26T07:02:48.675 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 16201 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/lm95241.ko 2023-11-26T07:02:48.675 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 18113 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/lm95245.ko 2023-11-26T07:02:48.675 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 24017 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/ltc2945.ko 2023-11-26T07:02:48.675 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 13793 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/ltc4151.ko 2023-11-26T07:02:48.675 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 15193 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/ltc4215.ko 2023-11-26T07:02:48.675 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 14889 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/ltc4222.ko 2023-11-26T07:02:48.675 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 14825 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/ltc4245.ko 2023-11-26T07:02:48.676 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 13249 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/ltc4260.ko 2023-11-26T07:02:48.676 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 14993 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/ltc4261.ko 2023-11-26T07:02:48.676 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 33489 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/max16065.ko 2023-11-26T07:02:48.676 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 18433 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/max1619.ko 2023-11-26T07:02:48.676 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 24353 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/max1668.ko 2023-11-26T07:02:48.676 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 19969 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/max197.ko 2023-11-26T07:02:48.676 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 27129 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/max6639.ko 2023-11-26T07:02:48.676 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 23321 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/max6650.ko 2023-11-26T07:02:48.676 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 35777 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/max6697.ko 2023-11-26T07:02:48.676 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 18585 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/mc13783-adc.ko 2023-11-26T07:02:48.676 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 10585 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/mcp3021.ko 2023-11-26T07:02:48.676 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 15937 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/menf21bmc_hwmon.ko 2023-11-26T07:02:48.677 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 44129 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/nct6683.ko 2023-11-26T07:02:48.677 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 119817 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/nct6775-core.ko 2023-11-26T07:02:48.677 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 60705 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/nct6775.ko 2023-11-26T07:02:48.677 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 59737 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/nct7802.ko 2023-11-26T07:02:48.677 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 82337 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/pc87360.ko 2023-11-26T07:02:48.677 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 54433 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/pc87427.ko 2023-11-26T07:02:48.677 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 18473 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/pcf8591.ko 2023-11-26T07:02:48.677 INFO:teuthology.orchestra.run.smithi143.stdout:drwxr-xr-x root/root 0 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/pmbus/ 2023-11-26T07:02:48.677 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 21353 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/pmbus/adm1275.ko 2023-11-26T07:02:48.677 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 17065 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/pmbus/lm25066.ko 2023-11-26T07:02:48.677 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 28329 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/pmbus/ltc2978.ko 2023-11-26T07:02:48.677 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 9577 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/pmbus/max16064.ko 2023-11-26T07:02:48.678 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 17657 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/pmbus/max34440.ko 2023-11-26T07:02:48.678 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 10169 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/pmbus/max8688.ko 2023-11-26T07:02:48.678 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 15505 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/pmbus/pmbus.ko 2023-11-26T07:02:48.678 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 82577 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/pmbus/pmbus_core.ko 2023-11-26T07:02:48.678 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 8729 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/pmbus/tps40422.ko 2023-11-26T07:02:48.678 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 30729 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/pmbus/ucd9000.ko 2023-11-26T07:02:48.678 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 15073 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/pmbus/ucd9200.ko 2023-11-26T07:02:48.678 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 20681 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/pmbus/zl6100.ko 2023-11-26T07:02:48.678 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 13017 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/powr1220.ko 2023-11-26T07:02:48.678 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 18705 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/sch5627.ko 2023-11-26T07:02:48.678 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 28817 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/sch5636.ko 2023-11-26T07:02:48.678 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 27145 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/sch56xx-common.ko 2023-11-26T07:02:48.679 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 32137 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/sht15.ko 2023-11-26T07:02:48.679 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 13505 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/sht21.ko 2023-11-26T07:02:48.679 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 16833 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/shtc1.ko 2023-11-26T07:02:48.679 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 40105 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/sis5595.ko 2023-11-26T07:02:48.679 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 19577 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/smsc47b397.ko 2023-11-26T07:02:48.679 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 39193 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/smsc47m1.ko 2023-11-26T07:02:48.679 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 37385 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/smsc47m192.ko 2023-11-26T07:02:48.679 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 26553 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/thmc50.ko 2023-11-26T07:02:48.679 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 17057 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/tmp102.ko 2023-11-26T07:02:48.679 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 14097 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/tmp103.ko 2023-11-26T07:02:48.679 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 21257 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/tmp401.ko 2023-11-26T07:02:48.680 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 19537 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/tmp421.ko 2023-11-26T07:02:48.680 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 19721 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/via-cputemp.ko 2023-11-26T07:02:48.680 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 41945 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/via686a.ko 2023-11-26T07:02:48.680 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 52457 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/vt1211.ko 2023-11-26T07:02:48.680 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 51545 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/vt8231.ko 2023-11-26T07:02:48.680 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 57449 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/w83627ehf.ko 2023-11-26T07:02:48.680 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 78873 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/w83627hf.ko 2023-11-26T07:02:48.680 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 71345 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/w83781d.ko 2023-11-26T07:02:48.680 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 63225 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/w83791d.ko 2023-11-26T07:02:48.680 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 71321 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/w83792d.ko 2023-11-26T07:02:48.680 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 101473 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/w83793.ko 2023-11-26T07:02:48.680 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 119505 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/w83795.ko 2023-11-26T07:02:48.681 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 17313 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/w83l785ts.ko 2023-11-26T07:02:48.681 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 35625 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/w83l786ng.ko 2023-11-26T07:02:48.681 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 13825 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/wm831x-hwmon.ko 2023-11-26T07:02:48.681 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 10625 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/wm8350-hwmon.ko 2023-11-26T07:02:48.681 INFO:teuthology.orchestra.run.smithi143.stdout:drwxr-xr-x root/root 0 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/i2c/ 2023-11-26T07:02:48.681 INFO:teuthology.orchestra.run.smithi143.stdout:drwxr-xr-x root/root 0 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/i2c/algos/ 2023-11-26T07:02:48.681 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 20049 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/i2c/algos/i2c-algo-bit.ko 2023-11-26T07:02:48.681 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 19345 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/i2c/algos/i2c-algo-pca.ko 2023-11-26T07:02:48.681 INFO:teuthology.orchestra.run.smithi143.stdout:drwxr-xr-x root/root 0 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/i2c/busses/ 2023-11-26T07:02:48.681 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 27969 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/i2c/busses/i2c-ali1535.ko 2023-11-26T07:02:48.681 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 28801 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/i2c/busses/i2c-ali1563.ko 2023-11-26T07:02:48.681 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 27777 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/i2c/busses/i2c-ali15x3.ko 2023-11-26T07:02:48.682 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 17937 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/i2c/busses/i2c-amd756-s4882.ko 2023-11-26T07:02:48.682 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 26193 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/i2c/busses/i2c-amd756.ko 2023-11-26T07:02:48.682 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 16201 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/i2c/busses/i2c-amd8111.ko 2023-11-26T07:02:48.682 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 15153 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/i2c/busses/i2c-cbus-gpio.ko 2023-11-26T07:02:48.682 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 6929 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/i2c/busses/i2c-ccgx-ucsi.ko 2023-11-26T07:02:48.682 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 13961 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/i2c/busses/i2c-cros-ec-tunnel.ko 2023-11-26T07:02:48.682 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 50201 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/i2c/busses/i2c-designware-core.ko 2023-11-26T07:02:48.682 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 23905 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/i2c/busses/i2c-designware-pci.ko 2023-11-26T07:02:48.682 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 20001 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/i2c/busses/i2c-designware-platform.ko 2023-11-26T07:02:48.682 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 17337 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/i2c/busses/i2c-diolan-u2c.ko 2023-11-26T07:02:48.682 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 12497 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/i2c/busses/i2c-dln2.ko 2023-11-26T07:02:48.683 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 17585 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/i2c/busses/i2c-gpio.ko 2023-11-26T07:02:48.683 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 62817 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/i2c/busses/i2c-i801.ko 2023-11-26T07:02:48.683 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 22433 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/i2c/busses/i2c-isch.ko 2023-11-26T07:02:48.683 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 47889 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/i2c/busses/i2c-ismt.ko 2023-11-26T07:02:48.683 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 17721 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/i2c/busses/i2c-kempld.ko 2023-11-26T07:02:48.683 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 17881 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/i2c/busses/i2c-nforce2-s4985.ko 2023-11-26T07:02:48.683 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 22337 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/i2c/busses/i2c-nforce2.ko 2023-11-26T07:02:48.683 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 26265 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/i2c/busses/i2c-ocores.ko 2023-11-26T07:02:48.683 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 25145 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/i2c/busses/i2c-parport.ko 2023-11-26T07:02:48.683 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 16497 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/i2c/busses/i2c-pca-platform.ko 2023-11-26T07:02:48.683 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 47241 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/i2c/busses/i2c-piix4.ko 2023-11-26T07:02:48.684 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 15289 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/i2c/busses/i2c-robotfuzz-osif.ko 2023-11-26T07:02:48.684 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 18649 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/i2c/busses/i2c-scmi.ko 2023-11-26T07:02:48.684 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 11809 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/i2c/busses/i2c-simtec.ko 2023-11-26T07:02:48.684 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 22313 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/i2c/busses/i2c-sis5595.ko 2023-11-26T07:02:48.684 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 29361 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/i2c/busses/i2c-sis630.ko 2023-11-26T07:02:48.684 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 20329 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/i2c/busses/i2c-sis96x.ko 2023-11-26T07:02:48.684 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 19241 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/i2c/busses/i2c-taos-evm.ko 2023-11-26T07:02:48.684 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 19809 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/i2c/busses/i2c-tiny-usb.ko 2023-11-26T07:02:48.684 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 13801 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/i2c/busses/i2c-via.ko 2023-11-26T07:02:48.684 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 26081 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/i2c/busses/i2c-viapro.ko 2023-11-26T07:02:48.684 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 19041 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/i2c/busses/i2c-viperboard.ko 2023-11-26T07:02:48.684 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 47657 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/i2c/busses/i2c-xiic.ko 2023-11-26T07:02:48.685 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 22449 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/i2c/i2c-mux.ko 2023-11-26T07:02:48.685 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 18825 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/i2c/i2c-smbus.ko 2023-11-26T07:02:48.685 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 27073 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/i2c/i2c-stub.ko 2023-11-26T07:02:48.685 INFO:teuthology.orchestra.run.smithi143.stdout:drwxr-xr-x root/root 0 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/i2c/muxes/ 2023-11-26T07:02:48.685 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 14921 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/i2c/muxes/i2c-mux-gpio.ko 2023-11-26T07:02:48.685 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 13457 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/i2c/muxes/i2c-mux-pca9541.ko 2023-11-26T07:02:48.685 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 29737 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/i2c/muxes/i2c-mux-pca954x.ko 2023-11-26T07:02:48.685 INFO:teuthology.orchestra.run.smithi143.stdout:drwxr-xr-x root/root 0 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/infiniband/ 2023-11-26T07:02:48.685 INFO:teuthology.orchestra.run.smithi143.stdout:drwxr-xr-x root/root 0 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/infiniband/core/ 2023-11-26T07:02:48.685 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 273609 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/infiniband/core/ib_cm.ko 2023-11-26T07:02:48.685 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 879969 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/infiniband/core/ib_core.ko 2023-11-26T07:02:48.686 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 69273 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/infiniband/core/ib_umad.ko 2023-11-26T07:02:48.686 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 324217 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/infiniband/core/ib_uverbs.ko 2023-11-26T07:02:48.686 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 97985 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/infiniband/core/iw_cm.ko 2023-11-26T07:02:48.686 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 282273 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/infiniband/core/rdma_cm.ko 2023-11-26T07:02:48.686 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 66201 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/infiniband/core/rdma_ucm.ko 2023-11-26T07:02:48.686 INFO:teuthology.orchestra.run.smithi143.stdout:drwxr-xr-x root/root 0 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/infiniband/hw/ 2023-11-26T07:02:48.686 INFO:teuthology.orchestra.run.smithi143.stdout:drwxr-xr-x root/root 0 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/infiniband/hw/cxgb4/ 2023-11-26T07:02:48.686 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 493073 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/infiniband/hw/cxgb4/iw_cxgb4.ko 2023-11-26T07:02:48.686 INFO:teuthology.orchestra.run.smithi143.stdout:drwxr-xr-x root/root 0 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/infiniband/hw/mlx4/ 2023-11-26T07:02:48.686 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 419737 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/infiniband/hw/mlx4/mlx4_ib.ko 2023-11-26T07:02:48.686 INFO:teuthology.orchestra.run.smithi143.stdout:drwxr-xr-x root/root 0 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/infiniband/hw/mlx5/ 2023-11-26T07:02:48.687 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 739025 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/infiniband/hw/mlx5/mlx5_ib.ko 2023-11-26T07:02:48.687 INFO:teuthology.orchestra.run.smithi143.stdout:drwxr-xr-x root/root 0 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/infiniband/hw/mthca/ 2023-11-26T07:02:48.687 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 296625 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/infiniband/hw/mthca/ib_mthca.ko 2023-11-26T07:02:48.687 INFO:teuthology.orchestra.run.smithi143.stdout:drwxr-xr-x root/root 0 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/infiniband/hw/ocrdma/ 2023-11-26T07:02:48.687 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 153697 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/infiniband/hw/ocrdma/ocrdma.ko 2023-11-26T07:02:48.687 INFO:teuthology.orchestra.run.smithi143.stdout:drwxr-xr-x root/root 0 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/infiniband/hw/usnic/ 2023-11-26T07:02:48.687 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 168969 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/infiniband/hw/usnic/usnic_verbs.ko 2023-11-26T07:02:48.687 INFO:teuthology.orchestra.run.smithi143.stdout:drwxr-xr-x root/root 0 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/infiniband/ulp/ 2023-11-26T07:02:48.687 INFO:teuthology.orchestra.run.smithi143.stdout:drwxr-xr-x root/root 0 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/infiniband/ulp/ipoib/ 2023-11-26T07:02:48.687 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 263081 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/infiniband/ulp/ipoib/ib_ipoib.ko 2023-11-26T07:02:48.687 INFO:teuthology.orchestra.run.smithi143.stdout:drwxr-xr-x root/root 0 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/infiniband/ulp/iser/ 2023-11-26T07:02:48.687 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 100241 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/infiniband/ulp/iser/ib_iser.ko 2023-11-26T07:02:48.688 INFO:teuthology.orchestra.run.smithi143.stdout:drwxr-xr-x root/root 0 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/infiniband/ulp/isert/ 2023-11-26T07:02:48.688 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 111425 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/infiniband/ulp/isert/ib_isert.ko 2023-11-26T07:02:48.688 INFO:teuthology.orchestra.run.smithi143.stdout:drwxr-xr-x root/root 0 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/infiniband/ulp/srp/ 2023-11-26T07:02:48.688 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 128753 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/infiniband/ulp/srp/ib_srp.ko 2023-11-26T07:02:48.688 INFO:teuthology.orchestra.run.smithi143.stdout:drwxr-xr-x root/root 0 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/infiniband/ulp/srpt/ 2023-11-26T07:02:48.688 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 129529 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/infiniband/ulp/srpt/ib_srpt.ko 2023-11-26T07:02:48.688 INFO:teuthology.orchestra.run.smithi143.stdout:drwxr-xr-x root/root 0 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/input/ 2023-11-26T07:02:48.688 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 10745 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/input/evbug.ko 2023-11-26T07:02:48.688 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 24129 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/input/ff-memless.ko 2023-11-26T07:02:48.688 INFO:teuthology.orchestra.run.smithi143.stdout:drwxr-xr-x root/root 0 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/input/gameport/ 2023-11-26T07:02:48.688 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 11881 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/input/gameport/emu10k1-gp.ko 2023-11-26T07:02:48.688 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 12145 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/input/gameport/fm801-gp.ko 2023-11-26T07:02:48.689 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 41241 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/input/gameport/gameport.ko 2023-11-26T07:02:48.689 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 41801 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/input/joydev.ko 2023-11-26T07:02:48.689 INFO:teuthology.orchestra.run.smithi143.stdout:drwxr-xr-x root/root 0 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/input/keyboard/ 2023-11-26T07:02:48.689 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 13313 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/input/keyboard/adp5520-keys.ko 2023-11-26T07:02:48.689 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 12801 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/input/keyboard/adp5588-keys.ko 2023-11-26T07:02:48.689 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 26841 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/input/keyboard/adp5589-keys.ko 2023-11-26T07:02:48.689 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 24761 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/input/keyboard/cros_ec_keyb.ko 2023-11-26T07:02:48.689 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 36665 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/input/keyboard/gpio_keys.ko 2023-11-26T07:02:48.689 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 18625 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/input/keyboard/gpio_keys_polled.ko 2023-11-26T07:02:48.689 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 21257 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/input/keyboard/lkkbd.ko 2023-11-26T07:02:48.689 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 26473 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/input/keyboard/lm8323.ko 2023-11-26T07:02:48.690 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 14409 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/input/keyboard/lm8333.ko 2023-11-26T07:02:48.690 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 24265 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/input/keyboard/matrix_keypad.ko 2023-11-26T07:02:48.690 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 18217 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/input/keyboard/max7359_keypad.ko 2023-11-26T07:02:48.690 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 15929 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/input/keyboard/mcs_touchkey.ko 2023-11-26T07:02:48.690 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 16489 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/input/keyboard/mpr121_touchkey.ko 2023-11-26T07:02:48.690 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 11585 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/input/keyboard/newtonkbd.ko 2023-11-26T07:02:48.690 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 10673 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/input/keyboard/opencores-kbd.ko 2023-11-26T07:02:48.690 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 14161 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/input/keyboard/qt1070.ko 2023-11-26T07:02:48.690 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 17929 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/input/keyboard/qt2160.ko 2023-11-26T07:02:48.690 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 23817 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/input/keyboard/samsung-keypad.ko 2023-11-26T07:02:48.690 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 11481 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/input/keyboard/stowaway.ko 2023-11-26T07:02:48.691 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 18289 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/input/keyboard/sunkbd.ko 2023-11-26T07:02:48.691 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 17425 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/input/keyboard/tca6416-keypad.ko 2023-11-26T07:02:48.691 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 16177 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/input/keyboard/tca8418_keypad.ko 2023-11-26T07:02:48.691 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 16801 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/input/keyboard/twl4030_keypad.ko 2023-11-26T07:02:48.691 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 11753 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/input/keyboard/xtkbd.ko 2023-11-26T07:02:48.691 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 11465 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/input/matrix-keymap.ko 2023-11-26T07:02:48.691 INFO:teuthology.orchestra.run.smithi143.stdout:drwxr-xr-x root/root 0 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/input/mouse/ 2023-11-26T07:02:48.691 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 34329 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/input/mouse/appletouch.ko 2023-11-26T07:02:48.691 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 34177 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/input/mouse/bcm5974.ko 2023-11-26T07:02:48.691 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 123681 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/input/mouse/cyapatp.ko 2023-11-26T07:02:48.691 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 89449 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/input/mouse/elan_i2c.ko 2023-11-26T07:02:48.691 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 13961 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/input/mouse/gpio_mouse.ko 2023-11-26T07:02:48.692 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 335929 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/input/mouse/psmouse.ko 2023-11-26T07:02:48.692 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 14681 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/input/mouse/sermouse.ko 2023-11-26T07:02:48.692 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 23881 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/input/mouse/synaptics_i2c.ko 2023-11-26T07:02:48.692 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 22569 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/input/mouse/synaptics_usb.ko 2023-11-26T07:02:48.692 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 15665 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/input/mouse/vsxxxaa.ko 2023-11-26T07:02:48.692 INFO:teuthology.orchestra.run.smithi143.stdout:drwxr-xr-x root/root 0 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/input/rmi4/ 2023-11-26T07:02:48.692 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 109873 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/input/rmi4/rmi_core.ko 2023-11-26T07:02:48.692 INFO:teuthology.orchestra.run.smithi143.stdout:drwxr-xr-x root/root 0 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/input/serio/ 2023-11-26T07:02:48.692 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 11665 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/input/serio/altera_ps2.ko 2023-11-26T07:02:48.692 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 16289 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/input/serio/arc_ps2.ko 2023-11-26T07:02:48.692 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 16425 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/input/serio/ct82c710.ko 2023-11-26T07:02:48.692 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 16953 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/input/serio/hyperv-keyboard.ko 2023-11-26T07:02:48.693 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 15833 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/input/serio/parkbd.ko 2023-11-26T07:02:48.693 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 13225 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/input/serio/pcips2.ko 2023-11-26T07:02:48.693 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 20153 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/input/serio/ps2mult.ko 2023-11-26T07:02:48.693 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 24153 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/input/serio/serio_raw.ko 2023-11-26T07:02:48.693 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 17977 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/input/serio/serport.ko 2023-11-26T07:02:48.693 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 13337 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/input/sparse-keymap.ko 2023-11-26T07:02:48.693 INFO:teuthology.orchestra.run.smithi143.stdout:drwxr-xr-x root/root 0 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/iommu/ 2023-11-26T07:02:48.693 INFO:teuthology.orchestra.run.smithi143.stdout:drwxr-xr-x root/root 0 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/iommu/amd/ 2023-11-26T07:02:48.693 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 39289 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/iommu/amd/iommu_v2.ko 2023-11-26T07:02:48.693 INFO:teuthology.orchestra.run.smithi143.stdout:drwxr-xr-x root/root 0 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/leds/ 2023-11-26T07:02:48.693 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 13817 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/leds/leds-88pm860x.ko 2023-11-26T07:02:48.694 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 11385 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/leds/leds-adp5520.ko 2023-11-26T07:02:48.694 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 59753 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/leds/leds-bd2802.ko 2023-11-26T07:02:48.694 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 24785 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/leds/leds-blinkm.ko 2023-11-26T07:02:48.694 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 10473 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/leds/leds-da903x.ko 2023-11-26T07:02:48.694 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 12241 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/leds/leds-da9052.ko 2023-11-26T07:02:48.694 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 16561 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/leds/leds-gpio.ko 2023-11-26T07:02:48.694 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 15793 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/leds/leds-lm3530.ko 2023-11-26T07:02:48.694 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 30505 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/leds/leds-lm3533.ko 2023-11-26T07:02:48.694 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 19353 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/leds/leds-lm355x.ko 2023-11-26T07:02:48.694 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 19961 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/leds/leds-lm3642.ko 2023-11-26T07:02:48.694 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 16241 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/leds/leds-lp3944.ko 2023-11-26T07:02:48.694 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 11233 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/leds/leds-lp8788.ko 2023-11-26T07:02:48.695 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 14929 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/leds/leds-max8997.ko 2023-11-26T07:02:48.695 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 12545 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/leds/leds-mc13783.ko 2023-11-26T07:02:48.695 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 13385 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/leds/leds-menf21bmc.ko 2023-11-26T07:02:48.695 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 20721 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/leds/leds-pca9532.ko 2023-11-26T07:02:48.695 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 20481 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/leds/leds-pca955x.ko 2023-11-26T07:02:48.695 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 17857 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/leds/leds-pca963x.ko 2023-11-26T07:02:48.695 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 34433 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/leds/leds-ss4200.ko 2023-11-26T07:02:48.695 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 24865 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/leds/leds-tca6507.ko 2023-11-26T07:02:48.695 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 15665 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/leds/leds-wm831x-status.ko 2023-11-26T07:02:48.695 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 11841 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/leds/leds-wm8350.ko 2023-11-26T07:02:48.695 INFO:teuthology.orchestra.run.smithi143.stdout:drwxr-xr-x root/root 0 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/leds/trigger/ 2023-11-26T07:02:48.695 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 9985 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/leds/trigger/ledtrig-audio.ko 2023-11-26T07:02:48.696 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 10873 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/leds/trigger/ledtrig-backlight.ko 2023-11-26T07:02:48.696 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 9089 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/leds/trigger/ledtrig-camera.ko 2023-11-26T07:02:48.696 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 7617 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/leds/trigger/ledtrig-default-on.ko 2023-11-26T07:02:48.696 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 14249 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/leds/trigger/ledtrig-heartbeat.ko 2023-11-26T07:02:48.696 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 14073 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/leds/trigger/ledtrig-oneshot.ko 2023-11-26T07:02:48.696 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 11889 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/leds/trigger/ledtrig-timer.ko 2023-11-26T07:02:48.696 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 14073 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/leds/trigger/ledtrig-transient.ko 2023-11-26T07:02:48.696 INFO:teuthology.orchestra.run.smithi143.stdout:drwxr-xr-x root/root 0 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/md/ 2023-11-26T07:02:48.696 INFO:teuthology.orchestra.run.smithi143.stdout:drwxr-xr-x root/root 0 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/md/bcache/ 2023-11-26T07:02:48.696 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 654969 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/md/bcache/bcache.ko 2023-11-26T07:02:48.696 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 40577 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/md/dm-bio-prison.ko 2023-11-26T07:02:48.696 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 89601 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/md/dm-bufio.ko 2023-11-26T07:02:48.697 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 47529 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/md/dm-cache-smq.ko 2023-11-26T07:02:48.697 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 156009 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/md/dm-cache.ko 2023-11-26T07:02:48.697 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 106897 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/md/dm-crypt.ko 2023-11-26T07:02:48.697 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 21545 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/md/dm-delay.ko 2023-11-26T07:02:48.697 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 53857 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/md/dm-era.ko 2023-11-26T07:02:48.697 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 28465 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/md/dm-flakey.ko 2023-11-26T07:02:48.697 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 41529 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/md/dm-log-userspace.ko 2023-11-26T07:02:48.697 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 32737 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/md/dm-log.ko 2023-11-26T07:02:48.697 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 49545 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/md/dm-mirror.ko 2023-11-26T07:02:48.697 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 78033 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/md/dm-multipath.ko 2023-11-26T07:02:48.697 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 16473 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/md/dm-queue-length.ko 2023-11-26T07:02:48.697 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 70825 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/md/dm-raid.ko 2023-11-26T07:02:48.698 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 36297 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/md/dm-region-hash.ko 2023-11-26T07:02:48.698 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 15873 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/md/dm-round-robin.ko 2023-11-26T07:02:48.698 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 16801 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/md/dm-service-time.ko 2023-11-26T07:02:48.698 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 109529 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/md/dm-snapshot.ko 2023-11-26T07:02:48.698 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 19361 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/md/dm-switch.ko 2023-11-26T07:02:48.698 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 181865 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/md/dm-thin-pool.ko 2023-11-26T07:02:48.698 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 51737 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/md/dm-verity.ko 2023-11-26T07:02:48.698 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 8561 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/md/dm-zero.ko 2023-11-26T07:02:48.698 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 15345 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/md/faulty.ko 2023-11-26T07:02:48.698 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 19625 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/md/linear.ko 2023-11-26T07:02:48.698 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 32417 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/md/multipath.ko 2023-11-26T07:02:48.699 INFO:teuthology.orchestra.run.smithi143.stdout:drwxr-xr-x root/root 0 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/md/persistent-data/ 2023-11-26T07:02:48.699 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 199937 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/md/persistent-data/dm-persistent-data.ko 2023-11-26T07:02:48.699 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 37337 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/md/raid0.ko 2023-11-26T07:02:48.699 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 113073 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/md/raid1.ko 2023-11-26T07:02:48.699 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 158681 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/md/raid10.ko 2023-11-26T07:02:48.699 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 395721 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/md/raid456.ko 2023-11-26T07:02:48.699 INFO:teuthology.orchestra.run.smithi143.stdout:drwxr-xr-x root/root 0 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/message/ 2023-11-26T07:02:48.699 INFO:teuthology.orchestra.run.smithi143.stdout:drwxr-xr-x root/root 0 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/message/fusion/ 2023-11-26T07:02:48.699 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 178137 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/message/fusion/mptbase.ko 2023-11-26T07:02:48.699 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 63385 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/message/fusion/mptctl.ko 2023-11-26T07:02:48.699 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 44665 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/message/fusion/mptfc.ko 2023-11-26T07:02:48.699 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 37225 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/message/fusion/mptlan.ko 2023-11-26T07:02:48.700 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 123321 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/message/fusion/mptsas.ko 2023-11-26T07:02:48.700 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 79041 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/message/fusion/mptscsih.ko 2023-11-26T07:02:48.700 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 53017 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/message/fusion/mptspi.ko 2023-11-26T07:02:48.700 INFO:teuthology.orchestra.run.smithi143.stdout:drwxr-xr-x root/root 0 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/mfd/ 2023-11-26T07:02:48.700 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 17305 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/mfd/88pm800.ko 2023-11-26T07:02:48.700 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 12761 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/mfd/88pm805.ko 2023-11-26T07:02:48.700 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 11193 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/mfd/88pm80x.ko 2023-11-26T07:02:48.700 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 11865 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/mfd/arizona-i2c.ko 2023-11-26T07:02:48.700 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 123721 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/mfd/arizona.ko 2023-11-26T07:02:48.700 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 11961 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/mfd/bcm590xx.ko 2023-11-26T07:02:48.700 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 17465 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/mfd/cros_ec_dev.ko 2023-11-26T07:02:48.700 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 34961 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/mfd/dln2.ko 2023-11-26T07:02:48.701 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 16689 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/mfd/janz-cmodio.ko 2023-11-26T07:02:48.701 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 53585 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/mfd/kempld-core.ko 2023-11-26T07:02:48.701 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 27529 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/mfd/lm3533-core.ko 2023-11-26T07:02:48.701 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 14993 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/mfd/lm3533-ctrlbank.ko 2023-11-26T07:02:48.701 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 13009 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/mfd/lp3943.ko 2023-11-26T07:02:48.701 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 54569 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/mfd/lpc_ich.ko 2023-11-26T07:02:48.701 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 12113 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/mfd/lpc_sch.ko 2023-11-26T07:02:48.701 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 17409 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/mfd/max8907.ko 2023-11-26T07:02:48.701 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 29857 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/mfd/mc13xxx-core.ko 2023-11-26T07:02:48.701 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 11801 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/mfd/mc13xxx-i2c.ko 2023-11-26T07:02:48.701 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 11593 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/mfd/menf21bmc.ko 2023-11-26T07:02:48.701 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 17329 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/mfd/pcf50633-adc.ko 2023-11-26T07:02:48.702 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 9153 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/mfd/pcf50633-gpio.ko 2023-11-26T07:02:48.702 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 34873 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/mfd/pcf50633.ko 2023-11-26T07:02:48.702 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 10105 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/mfd/rdc321x-southbridge.ko 2023-11-26T07:02:48.702 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 20289 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/mfd/retu-mfd.ko 2023-11-26T07:02:48.702 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 80425 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/mfd/si476x-core.ko 2023-11-26T07:02:48.702 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 48457 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/mfd/sm501.ko 2023-11-26T07:02:48.702 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 19825 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/mfd/ti_am335x_tscadc.ko 2023-11-26T07:02:48.702 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 51137 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/mfd/tps65010.ko 2023-11-26T07:02:48.702 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 10353 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/mfd/tps6507x.ko 2023-11-26T07:02:48.702 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 13329 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/mfd/viperboard.ko 2023-11-26T07:02:48.702 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 10537 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/mfd/vx855.ko 2023-11-26T07:02:48.702 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 17953 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/mfd/wl1273-core.ko 2023-11-26T07:02:48.703 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 44345 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/mfd/wm8994.ko 2023-11-26T07:02:48.703 INFO:teuthology.orchestra.run.smithi143.stdout:drwxr-xr-x root/root 0 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/misc/ 2023-11-26T07:02:48.703 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 13833 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/misc/ad525x_dpot-i2c.ko 2023-11-26T07:02:48.703 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 48489 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/misc/ad525x_dpot.ko 2023-11-26T07:02:48.703 INFO:teuthology.orchestra.run.smithi143.stdout:drwxr-xr-x root/root 0 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/misc/altera-stapl/ 2023-11-26T07:02:48.703 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 46585 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/misc/altera-stapl/altera-stapl.ko 2023-11-26T07:02:48.703 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 18393 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/misc/apds9802als.ko 2023-11-26T07:02:48.703 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 36729 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/misc/apds990x.ko 2023-11-26T07:02:48.703 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 45593 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/misc/bh1770glc.ko 2023-11-26T07:02:48.703 INFO:teuthology.orchestra.run.smithi143.stdout:drwxr-xr-x root/root 0 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/misc/c2port/ 2023-11-26T07:02:48.703 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 11145 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/misc/c2port/c2port-duramar2150.ko 2023-11-26T07:02:48.704 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 35281 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/misc/c2port/core.ko 2023-11-26T07:02:48.704 INFO:teuthology.orchestra.run.smithi143.stdout:drwxr-xr-x root/root 0 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/misc/cb710/ 2023-11-26T07:02:48.704 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 25161 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/misc/cb710/cb710.ko 2023-11-26T07:02:48.704 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 17801 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/misc/ds1682.ko 2023-11-26T07:02:48.704 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 8897 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/misc/dummy-irq.ko 2023-11-26T07:02:48.704 INFO:teuthology.orchestra.run.smithi143.stdout:drwxr-xr-x root/root 0 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/misc/echo/ 2023-11-26T07:02:48.704 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 13657 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/misc/echo/echo.ko 2023-11-26T07:02:48.704 INFO:teuthology.orchestra.run.smithi143.stdout:drwxr-xr-x root/root 0 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/misc/eeprom/ 2023-11-26T07:02:48.704 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 37849 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/misc/eeprom/at24.ko 2023-11-26T07:02:48.704 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 14321 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/misc/eeprom/eeprom.ko 2023-11-26T07:02:48.704 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 14769 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/misc/eeprom/eeprom_93cx6.ko 2023-11-26T07:02:48.705 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 13537 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/misc/eeprom/max6875.ko 2023-11-26T07:02:48.705 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 33009 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/misc/enclosure.ko 2023-11-26T07:02:48.705 INFO:teuthology.orchestra.run.smithi143.stdout:drwxr-xr-x root/root 0 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/misc/genwqe/ 2023-11-26T07:02:48.705 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 152201 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/misc/genwqe/genwqe_card.ko 2023-11-26T07:02:48.705 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 14129 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/misc/hmc6352.ko 2023-11-26T07:02:48.705 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 31257 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/misc/hpilo.ko 2023-11-26T07:02:48.705 INFO:teuthology.orchestra.run.smithi143.stdout:drwxr-xr-x root/root 0 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/misc/ibmasm/ 2023-11-26T07:02:48.705 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 72129 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/misc/ibmasm/ibmasm.ko 2023-11-26T07:02:48.705 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 19785 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/misc/ics932s401.ko 2023-11-26T07:02:48.705 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 21681 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/misc/isl29003.ko 2023-11-26T07:02:48.705 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 15753 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/misc/isl29020.ko 2023-11-26T07:02:48.705 INFO:teuthology.orchestra.run.smithi143.stdout:drwxr-xr-x root/root 0 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/misc/lis3lv02d/ 2023-11-26T07:02:48.706 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 42513 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/misc/lis3lv02d/lis3lv02d.ko 2023-11-26T07:02:48.706 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 13857 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/misc/lis3lv02d/lis3lv02d_i2c.ko 2023-11-26T07:02:48.706 INFO:teuthology.orchestra.run.smithi143.stdout:drwxr-xr-x root/root 0 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/misc/mei/ 2023-11-26T07:02:48.706 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 125209 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/misc/mei/mei-me.ko 2023-11-26T07:02:48.706 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 56521 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/misc/mei/mei-txe.ko 2023-11-26T07:02:48.706 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 310113 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/misc/mei/mei.ko 2023-11-26T07:02:48.706 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 31329 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/misc/phantom.ko 2023-11-26T07:02:48.706 INFO:teuthology.orchestra.run.smithi143.stdout:drwxr-xr-x root/root 0 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/misc/ti-st/ 2023-11-26T07:02:48.706 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 74265 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/misc/ti-st/st_drv.ko 2023-11-26T07:02:48.706 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 24305 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/misc/tifm_7xx1.ko 2023-11-26T07:02:48.706 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 28785 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/misc/tifm_core.ko 2023-11-26T07:02:48.706 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 18729 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/misc/tsl2550.ko 2023-11-26T07:02:48.707 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 48697 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/misc/vmw_balloon.ko 2023-11-26T07:02:48.707 INFO:teuthology.orchestra.run.smithi143.stdout:drwxr-xr-x root/root 0 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/misc/vmw_vmci/ 2023-11-26T07:02:48.707 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 183609 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/misc/vmw_vmci/vmw_vmci.ko 2023-11-26T07:02:48.707 INFO:teuthology.orchestra.run.smithi143.stdout:drwxr-xr-x root/root 0 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ 2023-11-26T07:02:48.707 INFO:teuthology.orchestra.run.smithi143.stdout:drwxr-xr-x root/root 0 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/bonding/ 2023-11-26T07:02:48.707 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 490129 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/bonding/bonding.ko 2023-11-26T07:02:48.707 INFO:teuthology.orchestra.run.smithi143.stdout:drwxr-xr-x root/root 0 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/dsa/ 2023-11-26T07:02:48.707 INFO:teuthology.orchestra.run.smithi143.stdout:drwxr-xr-x root/root 0 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/dsa/b53/ 2023-11-26T07:02:48.707 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 106033 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/dsa/b53/b53_common.ko 2023-11-26T07:02:48.707 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 40417 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/dsa/bcm-sf2.ko 2023-11-26T07:02:48.707 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 16041 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/dsa/mv88e6060.ko 2023-11-26T07:02:48.707 INFO:teuthology.orchestra.run.smithi143.stdout:drwxr-xr-x root/root 0 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/dsa/mv88e6xxx/ 2023-11-26T07:02:48.708 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 453737 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/dsa/mv88e6xxx/mv88e6xxx.ko 2023-11-26T07:02:48.708 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 16929 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/dummy.ko 2023-11-26T07:02:48.708 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 17961 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/eql.ko 2023-11-26T07:02:48.708 INFO:teuthology.orchestra.run.smithi143.stdout:drwxr-xr-x root/root 0 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/ 2023-11-26T07:02:48.708 INFO:teuthology.orchestra.run.smithi143.stdout:drwxr-xr-x root/root 0 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/3com/ 2023-11-26T07:02:48.708 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 45785 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/3com/3c574_cs.ko 2023-11-26T07:02:48.708 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 42105 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/3com/3c589_cs.ko 2023-11-26T07:02:48.708 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 130553 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/3com/3c59x.ko 2023-11-26T07:02:48.708 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 68849 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/3com/typhoon.ko 2023-11-26T07:02:48.708 INFO:teuthology.orchestra.run.smithi143.stdout:drwxr-xr-x root/root 0 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/8390/ 2023-11-26T07:02:48.708 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 37649 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/8390/8390.ko 2023-11-26T07:02:48.708 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 59689 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/8390/axnet_cs.ko 2023-11-26T07:02:48.708 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 27201 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/8390/ne2k-pci.ko 2023-11-26T07:02:48.709 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 99049 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/8390/pcnet_cs.ko 2023-11-26T07:02:48.709 INFO:teuthology.orchestra.run.smithi143.stdout:drwxr-xr-x root/root 0 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/adaptec/ 2023-11-26T07:02:48.709 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 54521 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/adaptec/starfire.ko 2023-11-26T07:02:48.709 INFO:teuthology.orchestra.run.smithi143.stdout:drwxr-xr-x root/root 0 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/agere/ 2023-11-26T07:02:48.709 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 69137 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/agere/et131x.ko 2023-11-26T07:02:48.709 INFO:teuthology.orchestra.run.smithi143.stdout:drwxr-xr-x root/root 0 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/alteon/ 2023-11-26T07:02:48.709 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 59585 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/alteon/acenic.ko 2023-11-26T07:02:48.709 INFO:teuthology.orchestra.run.smithi143.stdout:drwxr-xr-x root/root 0 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/altera/ 2023-11-26T07:02:48.709 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 41409 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/altera/altera_tse.ko 2023-11-26T07:02:48.709 INFO:teuthology.orchestra.run.smithi143.stdout:drwxr-xr-x root/root 0 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/amd/ 2023-11-26T07:02:48.709 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 54609 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/amd/amd8111e.ko 2023-11-26T07:02:48.709 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 49289 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/amd/nmclan_cs.ko 2023-11-26T07:02:48.710 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 87297 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/amd/pcnet32.ko 2023-11-26T07:02:48.710 INFO:teuthology.orchestra.run.smithi143.stdout:drwxr-xr-x root/root 0 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/atheros/ 2023-11-26T07:02:48.710 INFO:teuthology.orchestra.run.smithi143.stdout:drwxr-xr-x root/root 0 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/atheros/alx/ 2023-11-26T07:02:48.710 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 97337 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/atheros/alx/alx.ko 2023-11-26T07:02:48.710 INFO:teuthology.orchestra.run.smithi143.stdout:drwxr-xr-x root/root 0 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/atheros/atl1c/ 2023-11-26T07:02:48.710 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 104673 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/atheros/atl1c/atl1c.ko 2023-11-26T07:02:48.710 INFO:teuthology.orchestra.run.smithi143.stdout:drwxr-xr-x root/root 0 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/atheros/atl1e/ 2023-11-26T07:02:48.710 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 90257 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/atheros/atl1e/atl1e.ko 2023-11-26T07:02:48.710 INFO:teuthology.orchestra.run.smithi143.stdout:drwxr-xr-x root/root 0 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/atheros/atlx/ 2023-11-26T07:02:48.710 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 81569 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/atheros/atlx/atl1.ko 2023-11-26T07:02:48.710 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 69729 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/atheros/atlx/atl2.ko 2023-11-26T07:02:48.710 INFO:teuthology.orchestra.run.smithi143.stdout:drwxr-xr-x root/root 0 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/broadcom/ 2023-11-26T07:02:48.711 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 78401 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/broadcom/b44.ko 2023-11-26T07:02:48.711 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 176761 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/broadcom/bnx2.ko 2023-11-26T07:02:48.711 INFO:teuthology.orchestra.run.smithi143.stdout:drwxr-xr-x root/root 0 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/broadcom/bnx2x/ 2023-11-26T07:02:48.711 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 1421497 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/broadcom/bnx2x/bnx2x.ko 2023-11-26T07:02:48.711 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 128569 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/broadcom/cnic.ko 2023-11-26T07:02:48.711 INFO:teuthology.orchestra.run.smithi143.stdout:drwxr-xr-x root/root 0 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/broadcom/genet/ 2023-11-26T07:02:48.711 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 132089 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/broadcom/genet/genet.ko 2023-11-26T07:02:48.711 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 312081 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/broadcom/tg3.ko 2023-11-26T07:02:48.711 INFO:teuthology.orchestra.run.smithi143.stdout:drwxr-xr-x root/root 0 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/brocade/ 2023-11-26T07:02:48.711 INFO:teuthology.orchestra.run.smithi143.stdout:drwxr-xr-x root/root 0 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/brocade/bna/ 2023-11-26T07:02:48.711 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 323513 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/brocade/bna/bna.ko 2023-11-26T07:02:48.712 INFO:teuthology.orchestra.run.smithi143.stdout:drwxr-xr-x root/root 0 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/chelsio/ 2023-11-26T07:02:48.712 INFO:teuthology.orchestra.run.smithi143.stdout:drwxr-xr-x root/root 0 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/chelsio/cxgb/ 2023-11-26T07:02:48.712 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 193497 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/chelsio/cxgb/cxgb.ko 2023-11-26T07:02:48.712 INFO:teuthology.orchestra.run.smithi143.stdout:drwxr-xr-x root/root 0 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/chelsio/cxgb3/ 2023-11-26T07:02:48.712 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 347417 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/chelsio/cxgb3/cxgb3.ko 2023-11-26T07:02:48.712 INFO:teuthology.orchestra.run.smithi143.stdout:drwxr-xr-x root/root 0 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/chelsio/cxgb4/ 2023-11-26T07:02:48.712 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 1044497 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/chelsio/cxgb4/cxgb4.ko 2023-11-26T07:02:48.825 INFO:teuthology.orchestra.run.smithi143.stdout:drwxr-xr-x root/root 0 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/chelsio/cxgb4vf/ 2023-11-26T07:02:48.825 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 166193 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/chelsio/cxgb4vf/cxgb4vf.ko 2023-11-26T07:02:48.825 INFO:teuthology.orchestra.run.smithi143.stdout:drwxr-xr-x root/root 0 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/chelsio/libcxgb/ 2023-11-26T07:02:48.825 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 34881 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/chelsio/libcxgb/libcxgb.ko 2023-11-26T07:02:48.825 INFO:teuthology.orchestra.run.smithi143.stdout:drwxr-xr-x root/root 0 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/cisco/ 2023-11-26T07:02:48.825 INFO:teuthology.orchestra.run.smithi143.stdout:drwxr-xr-x root/root 0 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/cisco/enic/ 2023-11-26T07:02:48.825 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 204345 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/cisco/enic/enic.ko 2023-11-26T07:02:48.826 INFO:teuthology.orchestra.run.smithi143.stdout:drwxr-xr-x root/root 0 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/dec/ 2023-11-26T07:02:48.826 INFO:teuthology.orchestra.run.smithi143.stdout:drwxr-xr-x root/root 0 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/dec/tulip/ 2023-11-26T07:02:48.826 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 69449 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/dec/tulip/de2104x.ko 2023-11-26T07:02:48.826 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 61489 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/dec/tulip/dmfe.ko 2023-11-26T07:02:48.826 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 141225 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/dec/tulip/tulip.ko 2023-11-26T07:02:48.826 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 51497 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/dec/tulip/uli526x.ko 2023-11-26T07:02:48.826 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 62817 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/dec/tulip/winbond-840.ko 2023-11-26T07:02:48.826 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 31369 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/dec/tulip/xircom_cb.ko 2023-11-26T07:02:48.826 INFO:teuthology.orchestra.run.smithi143.stdout:drwxr-xr-x root/root 0 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/dlink/ 2023-11-26T07:02:48.826 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 60873 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/dlink/dl2k.ko 2023-11-26T07:02:48.826 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 62833 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/dlink/sundance.ko 2023-11-26T07:02:48.827 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 45721 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/dnet.ko 2023-11-26T07:02:48.827 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 25185 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/ec_bhf.ko 2023-11-26T07:02:48.827 INFO:teuthology.orchestra.run.smithi143.stdout:drwxr-xr-x root/root 0 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/emulex/ 2023-11-26T07:02:48.827 INFO:teuthology.orchestra.run.smithi143.stdout:drwxr-xr-x root/root 0 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/emulex/benet/ 2023-11-26T07:02:48.827 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 281137 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/emulex/benet/be2net.ko 2023-11-26T07:02:48.827 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 49457 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/ethoc.ko 2023-11-26T07:02:48.827 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 51737 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/fealnx.ko 2023-11-26T07:02:48.827 INFO:teuthology.orchestra.run.smithi143.stdout:drwxr-xr-x root/root 0 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/fujitsu/ 2023-11-26T07:02:48.827 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 45345 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/fujitsu/fmvj18x_cs.ko 2023-11-26T07:02:48.827 INFO:teuthology.orchestra.run.smithi143.stdout:drwxr-xr-x root/root 0 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/intel/ 2023-11-26T07:02:48.827 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 90465 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/intel/e100.ko 2023-11-26T07:02:48.827 INFO:teuthology.orchestra.run.smithi143.stdout:drwxr-xr-x root/root 0 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/intel/e1000/ 2023-11-26T07:02:48.828 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 295577 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/intel/e1000/e1000.ko 2023-11-26T07:02:48.828 INFO:teuthology.orchestra.run.smithi143.stdout:drwxr-xr-x root/root 0 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/intel/e1000e/ 2023-11-26T07:02:48.828 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 584497 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/intel/e1000e/e1000e.ko 2023-11-26T07:02:48.828 INFO:teuthology.orchestra.run.smithi143.stdout:drwxr-xr-x root/root 0 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/intel/fm10k/ 2023-11-26T07:02:48.828 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 246209 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/intel/fm10k/fm10k.ko 2023-11-26T07:02:48.828 INFO:teuthology.orchestra.run.smithi143.stdout:drwxr-xr-x root/root 0 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/intel/i40e/ 2023-11-26T07:02:48.828 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 925825 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/intel/i40e/i40e.ko 2023-11-26T07:02:48.828 INFO:teuthology.orchestra.run.smithi143.stdout:drwxr-xr-x root/root 0 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/intel/iavf/ 2023-11-26T07:02:48.828 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 302937 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/intel/iavf/iavf.ko 2023-11-26T07:02:48.828 INFO:teuthology.orchestra.run.smithi143.stdout:drwxr-xr-x root/root 0 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/intel/igb/ 2023-11-26T07:02:48.828 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 514641 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/intel/igb/igb.ko 2023-11-26T07:02:48.828 INFO:teuthology.orchestra.run.smithi143.stdout:drwxr-xr-x root/root 0 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/intel/igbvf/ 2023-11-26T07:02:48.829 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 93593 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/intel/igbvf/igbvf.ko 2023-11-26T07:02:48.829 INFO:teuthology.orchestra.run.smithi143.stdout:drwxr-xr-x root/root 0 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/intel/ixgbe/ 2023-11-26T07:02:48.829 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 734305 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/intel/ixgbe/ixgbe.ko 2023-11-26T07:02:48.829 INFO:teuthology.orchestra.run.smithi143.stdout:drwxr-xr-x root/root 0 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/intel/ixgbevf/ 2023-11-26T07:02:48.829 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 151769 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/intel/ixgbevf/ixgbevf.ko 2023-11-26T07:02:48.829 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 86921 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/jme.ko 2023-11-26T07:02:48.829 INFO:teuthology.orchestra.run.smithi143.stdout:drwxr-xr-x root/root 0 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/marvell/ 2023-11-26T07:02:48.829 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 20977 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/marvell/mvmdio.ko 2023-11-26T07:02:48.829 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 102289 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/marvell/skge.ko 2023-11-26T07:02:48.829 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 122649 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/marvell/sky2.ko 2023-11-26T07:02:48.829 INFO:teuthology.orchestra.run.smithi143.stdout:drwxr-xr-x root/root 0 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/mellanox/ 2023-11-26T07:02:48.829 INFO:teuthology.orchestra.run.smithi143.stdout:drwxr-xr-x root/root 0 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/mellanox/mlx4/ 2023-11-26T07:02:48.829 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 727601 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/mellanox/mlx4/mlx4_core.ko 2023-11-26T07:02:48.830 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 278937 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/mellanox/mlx4/mlx4_en.ko 2023-11-26T07:02:48.830 INFO:teuthology.orchestra.run.smithi143.stdout:drwxr-xr-x root/root 0 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/mellanox/mlx5/ 2023-11-26T07:02:48.830 INFO:teuthology.orchestra.run.smithi143.stdout:drwxr-xr-x root/root 0 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/mellanox/mlx5/core/ 2023-11-26T07:02:48.830 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 1113681 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/mellanox/mlx5/core/mlx5_core.ko 2023-11-26T07:02:48.830 INFO:teuthology.orchestra.run.smithi143.stdout:drwxr-xr-x root/root 0 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/micrel/ 2023-11-26T07:02:48.830 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 53041 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/micrel/ks8842.ko 2023-11-26T07:02:48.830 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 40993 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/micrel/ks8851_common.ko 2023-11-26T07:02:48.830 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 19489 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/micrel/ks8851_par.ko 2023-11-26T07:02:48.830 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 95665 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/micrel/ksz884x.ko 2023-11-26T07:02:48.830 INFO:teuthology.orchestra.run.smithi143.stdout:drwxr-xr-x root/root 0 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/myricom/ 2023-11-26T07:02:48.831 INFO:teuthology.orchestra.run.smithi143.stdout:drwxr-xr-x root/root 0 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/myricom/myri10ge/ 2023-11-26T07:02:48.831 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 108649 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/myricom/myri10ge/myri10ge.ko 2023-11-26T07:02:48.831 INFO:teuthology.orchestra.run.smithi143.stdout:drwxr-xr-x root/root 0 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/natsemi/ 2023-11-26T07:02:48.831 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 73441 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/natsemi/natsemi.ko 2023-11-26T07:02:48.831 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 47777 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/natsemi/ns83820.ko 2023-11-26T07:02:48.831 INFO:teuthology.orchestra.run.smithi143.stdout:drwxr-xr-x root/root 0 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/neterion/ 2023-11-26T07:02:48.831 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 150905 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/neterion/s2io.ko 2023-11-26T07:02:48.831 INFO:teuthology.orchestra.run.smithi143.stdout:drwxr-xr-x root/root 0 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/nvidia/ 2023-11-26T07:02:48.831 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 136721 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/nvidia/forcedeth.ko 2023-11-26T07:02:48.831 INFO:teuthology.orchestra.run.smithi143.stdout:drwxr-xr-x root/root 0 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/packetengines/ 2023-11-26T07:02:48.831 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 50481 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/packetengines/hamachi.ko 2023-11-26T07:02:48.831 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 43145 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/packetengines/yellowfin.ko 2023-11-26T07:02:48.832 INFO:teuthology.orchestra.run.smithi143.stdout:drwxr-xr-x root/root 0 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/qlogic/ 2023-11-26T07:02:48.832 INFO:teuthology.orchestra.run.smithi143.stdout:drwxr-xr-x root/root 0 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/qlogic/netxen/ 2023-11-26T07:02:48.832 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 227897 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/qlogic/netxen/netxen_nic.ko 2023-11-26T07:02:48.832 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 87081 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/qlogic/qla3xxx.ko 2023-11-26T07:02:48.832 INFO:teuthology.orchestra.run.smithi143.stdout:drwxr-xr-x root/root 0 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/qlogic/qlcnic/ 2023-11-26T07:02:48.832 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 672049 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/qlogic/qlcnic/qlcnic.ko 2023-11-26T07:02:48.832 INFO:teuthology.orchestra.run.smithi143.stdout:drwxr-xr-x root/root 0 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/rdc/ 2023-11-26T07:02:48.832 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 43017 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/rdc/r6040.ko 2023-11-26T07:02:48.832 INFO:teuthology.orchestra.run.smithi143.stdout:drwxr-xr-x root/root 0 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/realtek/ 2023-11-26T07:02:48.832 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 67385 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/realtek/8139cp.ko 2023-11-26T07:02:48.832 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 83985 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/realtek/8139too.ko 2023-11-26T07:02:48.832 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 29321 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/realtek/atp.ko 2023-11-26T07:02:48.833 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 189801 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/realtek/r8169.ko 2023-11-26T07:02:48.833 INFO:teuthology.orchestra.run.smithi143.stdout:drwxr-xr-x root/root 0 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/samsung/ 2023-11-26T07:02:48.833 INFO:teuthology.orchestra.run.smithi143.stdout:drwxr-xr-x root/root 0 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/samsung/sxgbe/ 2023-11-26T07:02:48.833 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 104241 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/samsung/sxgbe/samsung-sxgbe.ko 2023-11-26T07:02:48.833 INFO:teuthology.orchestra.run.smithi143.stdout:drwxr-xr-x root/root 0 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/sfc/ 2023-11-26T07:02:48.833 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 1052497 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/sfc/sfc.ko 2023-11-26T07:02:48.833 INFO:teuthology.orchestra.run.smithi143.stdout:drwxr-xr-x root/root 0 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/silan/ 2023-11-26T07:02:48.833 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 47097 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/silan/sc92031.ko 2023-11-26T07:02:48.833 INFO:teuthology.orchestra.run.smithi143.stdout:drwxr-xr-x root/root 0 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/sis/ 2023-11-26T07:02:48.833 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 54817 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/sis/sis190.ko 2023-11-26T07:02:48.833 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 63337 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/sis/sis900.ko 2023-11-26T07:02:48.833 INFO:teuthology.orchestra.run.smithi143.stdout:drwxr-xr-x root/root 0 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/smsc/ 2023-11-26T07:02:48.834 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 61537 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/smsc/epic100.ko 2023-11-26T07:02:48.834 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 67193 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/smsc/smc91c92_cs.ko 2023-11-26T07:02:48.834 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 73409 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/smsc/smsc911x.ko 2023-11-26T07:02:48.834 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 68177 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/smsc/smsc9420.ko 2023-11-26T07:02:48.834 INFO:teuthology.orchestra.run.smithi143.stdout:drwxr-xr-x root/root 0 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/stmicro/ 2023-11-26T07:02:48.834 INFO:teuthology.orchestra.run.smithi143.stdout:drwxr-xr-x root/root 0 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/stmicro/stmmac/ 2023-11-26T07:02:48.834 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 13441 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/stmicro/stmmac/dwmac-generic.ko 2023-11-26T07:02:48.834 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 39905 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/stmicro/stmmac/dwmac-intel.ko 2023-11-26T07:02:48.834 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 20657 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/stmicro/stmmac/stmmac-platform.ko 2023-11-26T07:02:48.834 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 487025 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/stmicro/stmmac/stmmac.ko 2023-11-26T07:02:48.834 INFO:teuthology.orchestra.run.smithi143.stdout:drwxr-xr-x root/root 0 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/sun/ 2023-11-26T07:02:48.834 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 103337 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/sun/cassini.ko 2023-11-26T07:02:48.835 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 166097 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/sun/niu.ko 2023-11-26T07:02:48.835 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 66329 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/sun/sungem.ko 2023-11-26T07:02:48.835 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 78801 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/sun/sunhme.ko 2023-11-26T07:02:48.835 INFO:teuthology.orchestra.run.smithi143.stdout:drwxr-xr-x root/root 0 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/tehuti/ 2023-11-26T07:02:48.835 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 50057 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/tehuti/tehuti.ko 2023-11-26T07:02:48.835 INFO:teuthology.orchestra.run.smithi143.stdout:drwxr-xr-x root/root 0 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/ti/ 2023-11-26T07:02:48.835 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 86529 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/ti/tlan.ko 2023-11-26T07:02:48.835 INFO:teuthology.orchestra.run.smithi143.stdout:drwxr-xr-x root/root 0 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/via/ 2023-11-26T07:02:48.835 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 80273 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/via/via-rhine.ko 2023-11-26T07:02:48.835 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 83633 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/via/via-velocity.ko 2023-11-26T07:02:48.835 INFO:teuthology.orchestra.run.smithi143.stdout:drwxr-xr-x root/root 0 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/wiznet/ 2023-11-26T07:02:48.836 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 46241 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/wiznet/w5100.ko 2023-11-26T07:02:48.836 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 30849 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/wiznet/w5300.ko 2023-11-26T07:02:48.836 INFO:teuthology.orchestra.run.smithi143.stdout:drwxr-xr-x root/root 0 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/xircom/ 2023-11-26T07:02:48.836 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 59993 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/xircom/xirc2ps_cs.ko 2023-11-26T07:02:48.836 INFO:teuthology.orchestra.run.smithi143.stdout:drwxr-xr-x root/root 0 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/fddi/ 2023-11-26T07:02:48.836 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 41001 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/fddi/defxx.ko 2023-11-26T07:02:48.836 INFO:teuthology.orchestra.run.smithi143.stdout:drwxr-xr-x root/root 0 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/fddi/skfp/ 2023-11-26T07:02:48.836 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 216985 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/fddi/skfp/skfp.ko 2023-11-26T07:02:48.836 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 77345 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/geneve.ko 2023-11-26T07:02:48.836 INFO:teuthology.orchestra.run.smithi143.stdout:drwxr-xr-x root/root 0 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/hyperv/ 2023-11-26T07:02:48.836 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 235521 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/hyperv/hv_netvsc.ko 2023-11-26T07:02:48.836 INFO:teuthology.orchestra.run.smithi143.stdout:drwxr-xr-x root/root 0 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ieee802154/ 2023-11-26T07:02:48.837 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 18273 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ieee802154/fakelb.ko 2023-11-26T07:02:48.837 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 25065 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ifb.ko 2023-11-26T07:02:48.837 INFO:teuthology.orchestra.run.smithi143.stdout:drwxr-xr-x root/root 0 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ipvlan/ 2023-11-26T07:02:48.837 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 83769 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ipvlan/ipvlan.ko 2023-11-26T07:02:48.837 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 62145 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/macvlan.ko 2023-11-26T07:02:48.837 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 15865 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/macvtap.ko 2023-11-26T07:02:48.837 INFO:teuthology.orchestra.run.smithi143.stdout:drwxr-xr-x root/root 0 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/mdio/ 2023-11-26T07:02:48.837 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 20569 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/mdio/mdio-bcm-unimac.ko 2023-11-26T07:02:48.837 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 14185 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/mdio/mdio-bitbang.ko 2023-11-26T07:02:48.838 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 13681 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/mdio/mdio-gpio.ko 2023-11-26T07:02:48.838 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 8825 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/mdio/mdio-regmap.ko 2023-11-26T07:02:48.838 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 16025 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/mdio.ko 2023-11-26T07:02:48.838 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 20513 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/mii.ko 2023-11-26T07:02:48.838 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 43953 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/netconsole.ko 2023-11-26T07:02:48.838 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 13145 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/nlmon.ko 2023-11-26T07:02:48.838 INFO:teuthology.orchestra.run.smithi143.stdout:drwxr-xr-x root/root 0 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/pcs/ 2023-11-26T07:02:48.838 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 17985 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/pcs/pcs-lynx.ko 2023-11-26T07:02:48.838 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 38977 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/pcs/pcs_xpcs.ko 2023-11-26T07:02:48.838 INFO:teuthology.orchestra.run.smithi143.stdout:drwxr-xr-x root/root 0 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/phy/ 2023-11-26T07:02:48.838 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 9737 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/phy/amd.ko 2023-11-26T07:02:48.839 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 50033 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/phy/bcm-phy-lib.ko 2023-11-26T07:02:48.839 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 44721 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/phy/bcm7xxx.ko 2023-11-26T07:02:48.839 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 11521 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/phy/bcm87xx.ko 2023-11-26T07:02:48.839 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 42241 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/phy/broadcom.ko 2023-11-26T07:02:48.839 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 10625 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/phy/cicada.ko 2023-11-26T07:02:48.839 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 13025 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/phy/davicom.ko 2023-11-26T07:02:48.839 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 9449 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/phy/et1011c.ko 2023-11-26T07:02:48.839 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 24969 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/phy/icplus.ko 2023-11-26T07:02:48.839 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 15849 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/phy/lxt.ko 2023-11-26T07:02:48.839 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 82049 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/phy/marvell.ko 2023-11-26T07:02:48.839 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 76113 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/phy/micrel.ko 2023-11-26T07:02:48.840 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 12033 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/phy/national.ko 2023-11-26T07:02:48.840 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 97857 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/phy/phylink.ko 2023-11-26T07:02:48.840 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 10065 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/phy/qsemi.ko 2023-11-26T07:02:48.840 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 50065 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/phy/realtek.ko 2023-11-26T07:02:48.840 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 29985 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/phy/smsc.ko 2023-11-26T07:02:48.840 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 10857 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/phy/ste10Xp.ko 2023-11-26T07:02:48.840 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 24289 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/phy/vitesse.ko 2023-11-26T07:02:48.840 INFO:teuthology.orchestra.run.smithi143.stdout:drwxr-xr-x root/root 0 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/plip/ 2023-11-26T07:02:48.840 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 43505 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/plip/plip.ko 2023-11-26T07:02:48.840 INFO:teuthology.orchestra.run.smithi143.stdout:drwxr-xr-x root/root 0 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ppp/ 2023-11-26T07:02:48.840 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 15857 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ppp/bsd_comp.ko 2023-11-26T07:02:48.840 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 29713 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ppp/ppp_async.ko 2023-11-26T07:02:48.840 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 17665 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ppp/ppp_deflate.ko 2023-11-26T07:02:48.841 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 18625 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ppp/ppp_mppe.ko 2023-11-26T07:02:48.841 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 28729 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ppp/ppp_synctty.ko 2023-11-26T07:02:48.841 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 42545 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ppp/pppoe.ko 2023-11-26T07:02:48.841 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 13441 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ppp/pppox.ko 2023-11-26T07:02:48.841 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 32689 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ppp/pptp.ko 2023-11-26T07:02:48.841 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 28289 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/sb1000.ko 2023-11-26T07:02:48.841 INFO:teuthology.orchestra.run.smithi143.stdout:drwxr-xr-x root/root 0 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/slip/ 2023-11-26T07:02:48.841 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 42465 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/slip/slip.ko 2023-11-26T07:02:48.841 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 26929 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/sungem_phy.ko 2023-11-26T07:02:48.841 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 64449 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/tap.ko 2023-11-26T07:02:48.841 INFO:teuthology.orchestra.run.smithi143.stdout:drwxr-xr-x root/root 0 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/team/ 2023-11-26T07:02:48.841 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 106169 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/team/team.ko 2023-11-26T07:02:48.842 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 13497 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/team/team_mode_activebackup.ko 2023-11-26T07:02:48.842 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 10209 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/team/team_mode_broadcast.ko 2023-11-26T07:02:48.842 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 28881 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/team/team_mode_loadbalance.ko 2023-11-26T07:02:48.842 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 9641 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/team/team_mode_random.ko 2023-11-26T07:02:48.842 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 9321 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/team/team_mode_roundrobin.ko 2023-11-26T07:02:48.842 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 87017 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/veth.ko 2023-11-26T07:02:48.842 INFO:teuthology.orchestra.run.smithi143.stdout:drwxr-xr-x root/root 0 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/vmxnet3/ 2023-11-26T07:02:48.842 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 144297 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/vmxnet3/vmxnet3.ko 2023-11-26T07:02:48.842 INFO:teuthology.orchestra.run.smithi143.stdout:drwxr-xr-x root/root 0 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/vxlan/ 2023-11-26T07:02:48.842 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 297329 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/vxlan/vxlan.ko 2023-11-26T07:02:48.842 INFO:teuthology.orchestra.run.smithi143.stdout:drwxr-xr-x root/root 0 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/xen-netback/ 2023-11-26T07:02:48.843 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 144585 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/xen-netback/xen-netback.ko 2023-11-26T07:02:48.843 INFO:teuthology.orchestra.run.smithi143.stdout:drwxr-xr-x root/root 0 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/ntb/ 2023-11-26T07:02:48.843 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 20969 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/ntb/ntb.ko 2023-11-26T07:02:48.843 INFO:teuthology.orchestra.run.smithi143.stdout:drwxr-xr-x root/root 0 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/nvme/ 2023-11-26T07:02:48.843 INFO:teuthology.orchestra.run.smithi143.stdout:drwxr-xr-x root/root 0 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/nvme/host/ 2023-11-26T07:02:48.843 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 265121 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/nvme/host/nvme-core.ko 2023-11-26T07:02:48.843 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 102209 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/nvme/host/nvme.ko 2023-11-26T07:02:48.843 INFO:teuthology.orchestra.run.smithi143.stdout:drwxr-xr-x root/root 0 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/parport/ 2023-11-26T07:02:48.843 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 133841 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/parport/parport.ko 2023-11-26T07:02:48.843 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 14561 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/parport/parport_cs.ko 2023-11-26T07:02:48.843 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 84465 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/parport/parport_pc.ko 2023-11-26T07:02:48.843 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 26121 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/parport/parport_serial.ko 2023-11-26T07:02:48.844 INFO:teuthology.orchestra.run.smithi143.stdout:drwxr-xr-x root/root 0 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/pci/ 2023-11-26T07:02:48.844 INFO:teuthology.orchestra.run.smithi143.stdout:drwxr-xr-x root/root 0 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/pci/hotplug/ 2023-11-26T07:02:48.844 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 25009 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/pci/hotplug/acpiphp_ibm.ko 2023-11-26T07:02:48.844 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 17849 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/pci/hotplug/cpcihp_generic.ko 2023-11-26T07:02:48.844 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 20137 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/pci/hotplug/cpcihp_zt5550.ko 2023-11-26T07:02:48.844 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 11857 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/pci/pci-stub.ko 2023-11-26T07:02:48.844 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 46465 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/pci/xen-pcifront.ko 2023-11-26T07:02:48.844 INFO:teuthology.orchestra.run.smithi143.stdout:drwxr-xr-x root/root 0 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/pcmcia/ 2023-11-26T07:02:48.844 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 26793 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/pcmcia/i82092.ko 2023-11-26T07:02:48.844 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 145809 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/pcmcia/pcmcia.ko 2023-11-26T07:02:48.844 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 62137 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/pcmcia/pcmcia_core.ko 2023-11-26T07:02:48.844 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 34345 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/pcmcia/pcmcia_rsrc.ko 2023-11-26T07:02:48.845 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 34265 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/pcmcia/pd6729.ko 2023-11-26T07:02:48.845 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 112889 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/pcmcia/yenta_socket.ko 2023-11-26T07:02:48.845 INFO:teuthology.orchestra.run.smithi143.stdout:drwxr-xr-x root/root 0 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/phy/ 2023-11-26T07:02:48.845 INFO:teuthology.orchestra.run.smithi143.stdout:drwxr-xr-x root/root 0 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/phy/broadcom/ 2023-11-26T07:02:48.845 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 10841 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/phy/broadcom/phy-bcm-kona-usb2.ko 2023-11-26T07:02:48.845 INFO:teuthology.orchestra.run.smithi143.stdout:drwxr-xr-x root/root 0 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/phy/samsung/ 2023-11-26T07:02:48.845 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 8441 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/phy/samsung/phy-exynos-usb2.ko 2023-11-26T07:02:48.845 INFO:teuthology.orchestra.run.smithi143.stdout:drwxr-xr-x root/root 0 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/pinctrl/ 2023-11-26T07:02:48.845 INFO:teuthology.orchestra.run.smithi143.stdout:drwxr-xr-x root/root 0 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/pinctrl/intel/ 2023-11-26T07:02:48.845 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 70153 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/pinctrl/intel/pinctrl-cherryview.ko 2023-11-26T07:02:48.845 INFO:teuthology.orchestra.run.smithi143.stdout:drwxr-xr-x root/root 0 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/platform/ 2023-11-26T07:02:48.845 INFO:teuthology.orchestra.run.smithi143.stdout:drwxr-xr-x root/root 0 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/platform/chrome/ 2023-11-26T07:02:48.846 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 50889 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/platform/chrome/cros-ec-sensorhub.ko 2023-11-26T07:02:48.846 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 20849 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/platform/chrome/cros_ec.ko 2023-11-26T07:02:48.846 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 21449 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/platform/chrome/cros_ec_chardev.ko 2023-11-26T07:02:48.846 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 25417 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/platform/chrome/cros_ec_debugfs.ko 2023-11-26T07:02:48.846 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 26201 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/platform/chrome/cros_ec_lightbar.ko 2023-11-26T07:02:48.846 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 18201 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/platform/chrome/cros_ec_sysfs.ko 2023-11-26T07:02:48.846 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 17361 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/platform/chrome/cros_usbpd_notify.ko 2023-11-26T07:02:48.846 INFO:teuthology.orchestra.run.smithi143.stdout:drwxr-xr-x root/root 0 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/platform/x86/ 2023-11-26T07:02:48.846 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 67097 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/platform/x86/acer-wmi.ko 2023-11-26T07:02:48.846 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 39145 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/platform/x86/acerhdf.ko 2023-11-26T07:02:48.846 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 37025 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/platform/x86/apple-gmux.ko 2023-11-26T07:02:48.846 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 59665 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/platform/x86/asus-laptop.ko 2023-11-26T07:02:48.847 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 31161 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/platform/x86/asus-nb-wmi.ko 2023-11-26T07:02:48.847 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 127441 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/platform/x86/asus-wmi.ko 2023-11-26T07:02:48.847 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 36849 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/platform/x86/classmate-laptop.ko 2023-11-26T07:02:48.847 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 49121 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/platform/x86/eeepc-laptop.ko 2023-11-26T07:02:48.847 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 13961 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/platform/x86/eeepc-wmi.ko 2023-11-26T07:02:48.847 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 40265 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/platform/x86/fujitsu-laptop.ko 2023-11-26T07:02:48.847 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 22521 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/platform/x86/fujitsu-tablet.ko 2023-11-26T07:02:48.847 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 36593 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/platform/x86/hdaps.ko 2023-11-26T07:02:48.847 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 19449 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/platform/x86/ibm_rtl.ko 2023-11-26T07:02:48.847 INFO:teuthology.orchestra.run.smithi143.stdout:drwxr-xr-x root/root 0 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/platform/x86/intel/ 2023-11-26T07:02:48.847 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 12073 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/platform/x86/intel/intel-rst.ko 2023-11-26T07:02:48.847 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 8985 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/platform/x86/intel/intel-smartconnect.ko 2023-11-26T07:02:48.848 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 43497 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/platform/x86/intel_ips.ko 2023-11-26T07:02:48.848 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 20273 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/platform/x86/msi-wmi.ko 2023-11-26T07:02:48.848 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 10377 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/platform/x86/mxm-wmi.ko 2023-11-26T07:02:48.848 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 34801 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/platform/x86/panasonic-laptop.ko 2023-11-26T07:02:48.848 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 42329 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/platform/x86/samsung-laptop.ko 2023-11-26T07:02:48.848 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 14169 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/platform/x86/samsung-q10.ko 2023-11-26T07:02:48.848 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 17321 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/platform/x86/topstar-laptop.ko 2023-11-26T07:02:48.848 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 13721 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/platform/x86/toshiba_bluetooth.ko 2023-11-26T07:02:48.848 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 17081 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/platform/x86/toshiba_haps.ko 2023-11-26T07:02:48.848 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 10129 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/platform/x86/wmi-bmof.ko 2023-11-26T07:02:48.848 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 57569 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/platform/x86/wmi.ko 2023-11-26T07:02:48.848 INFO:teuthology.orchestra.run.smithi143.stdout:drwxr-xr-x root/root 0 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/power/ 2023-11-26T07:02:48.849 INFO:teuthology.orchestra.run.smithi143.stdout:drwxr-xr-x root/root 0 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/power/supply/ 2023-11-26T07:02:48.849 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 30281 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/power/supply/88pm860x_battery.ko 2023-11-26T07:02:48.849 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 33985 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/power/supply/88pm860x_charger.ko 2023-11-26T07:02:48.849 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 55057 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/power/supply/bq2415x_charger.ko 2023-11-26T07:02:48.849 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 54081 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/power/supply/bq24190_charger.ko 2023-11-26T07:02:48.849 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 21553 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/power/supply/bq24735-charger.ko 2023-11-26T07:02:48.849 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 17337 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/power/supply/cros_peripheral_charger.ko 2023-11-26T07:02:48.849 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 19609 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/power/supply/da9030_battery.ko 2023-11-26T07:02:48.849 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 19841 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/power/supply/da9052-battery.ko 2023-11-26T07:02:48.849 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 29281 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/power/supply/ds2760_battery.ko 2023-11-26T07:02:48.849 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 23217 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/power/supply/ds2780_battery.ko 2023-11-26T07:02:48.849 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 23353 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/power/supply/ds2781_battery.ko 2023-11-26T07:02:48.850 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 23649 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/power/supply/ds2782_battery.ko 2023-11-26T07:02:48.850 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 20353 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/power/supply/gpio-charger.ko 2023-11-26T07:02:48.850 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 18041 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/power/supply/isp1704_charger.ko 2023-11-26T07:02:48.850 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 20273 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/power/supply/lp8727_charger.ko 2023-11-26T07:02:48.850 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 12577 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/power/supply/max14577_charger.ko 2023-11-26T07:02:48.850 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 25897 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/power/supply/max17040_battery.ko 2023-11-26T07:02:48.850 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 30289 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/power/supply/max17042_battery.ko 2023-11-26T07:02:48.850 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 18585 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/power/supply/max8903_charger.ko 2023-11-26T07:02:48.850 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 23729 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/power/supply/max8925_power.ko 2023-11-26T07:02:48.850 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 21657 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/power/supply/pcf50633-charger.ko 2023-11-26T07:02:48.850 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 38009 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/power/supply/sbs-battery.ko 2023-11-26T07:02:48.850 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 28385 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/power/supply/test_power.ko 2023-11-26T07:02:48.851 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 18865 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/power/supply/tps65090-charger.ko 2023-11-26T07:02:48.851 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 11969 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/power/supply/wm831x_backup.ko 2023-11-26T07:02:48.851 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 24521 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/power/supply/wm831x_power.ko 2023-11-26T07:02:48.851 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 22417 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/power/supply/wm8350_power.ko 2023-11-26T07:02:48.851 INFO:teuthology.orchestra.run.smithi143.stdout:drwxr-xr-x root/root 0 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/powercap/ 2023-11-26T07:02:48.851 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 57729 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/powercap/intel_rapl_common.ko 2023-11-26T07:02:48.851 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 17345 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/powercap/intel_rapl_msr.ko 2023-11-26T07:02:48.851 INFO:teuthology.orchestra.run.smithi143.stdout:drwxr-xr-x root/root 0 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/pps/ 2023-11-26T07:02:48.851 INFO:teuthology.orchestra.run.smithi143.stdout:drwxr-xr-x root/root 0 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/pps/clients/ 2023-11-26T07:02:48.851 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 15521 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/pps/clients/pps-gpio.ko 2023-11-26T07:02:48.851 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 13249 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/pps/clients/pps-ldisc.ko 2023-11-26T07:02:48.851 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 15913 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/pps/clients/pps_parport.ko 2023-11-26T07:02:48.852 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 41873 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/pps/pps_core.ko 2023-11-26T07:02:48.852 INFO:teuthology.orchestra.run.smithi143.stdout:drwxr-xr-x root/root 0 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/ptp/ 2023-11-26T07:02:48.852 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 66385 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/ptp/ptp.ko 2023-11-26T07:02:48.852 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 16449 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/ptp/ptp_kvm.ko 2023-11-26T07:02:48.852 INFO:teuthology.orchestra.run.smithi143.stdout:drwxr-xr-x root/root 0 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/rtc/ 2023-11-26T07:02:48.852 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 23497 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/rtc/rtc-88pm80x.ko 2023-11-26T07:02:48.852 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 24521 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/rtc/rtc-88pm860x.ko 2023-11-26T07:02:48.852 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 15393 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/rtc/rtc-bq32k.ko 2023-11-26T07:02:48.852 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 20105 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/rtc/rtc-da9052.ko 2023-11-26T07:02:48.852 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 18505 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/rtc/rtc-da9055.ko 2023-11-26T07:02:48.852 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 20425 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/rtc/rtc-da9063.ko 2023-11-26T07:02:48.852 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 16457 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/rtc/rtc-ds1286.ko 2023-11-26T07:02:48.853 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 68577 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/rtc/rtc-ds1307.ko 2023-11-26T07:02:48.853 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 20817 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/rtc/rtc-ds1374.ko 2023-11-26T07:02:48.853 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 18217 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/rtc/rtc-ds1511.ko 2023-11-26T07:02:48.853 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 16737 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/rtc/rtc-ds1553.ko 2023-11-26T07:02:48.853 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 13953 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/rtc/rtc-ds1672.ko 2023-11-26T07:02:48.853 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 13369 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/rtc/rtc-ds1742.ko 2023-11-26T07:02:48.853 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 14697 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/rtc/rtc-ds2404.ko 2023-11-26T07:02:48.853 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 23705 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/rtc/rtc-ds3232.ko 2023-11-26T07:02:48.853 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 11081 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/rtc/rtc-em3027.ko 2023-11-26T07:02:48.853 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 24329 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/rtc/rtc-fm3130.ko 2023-11-26T07:02:48.853 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 20729 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/rtc/rtc-isl12022.ko 2023-11-26T07:02:48.853 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 35969 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/rtc/rtc-isl1208.ko 2023-11-26T07:02:48.854 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 14673 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/rtc/rtc-lp8788.ko 2023-11-26T07:02:48.854 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 38097 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/rtc/rtc-m41t80.ko 2023-11-26T07:02:48.854 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 12089 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/rtc/rtc-m48t35.ko 2023-11-26T07:02:48.854 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 25945 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/rtc/rtc-m48t59.ko 2023-11-26T07:02:48.854 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 14841 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/rtc/rtc-m48t86.ko 2023-11-26T07:02:48.854 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 11633 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/rtc/rtc-max6900.ko 2023-11-26T07:02:48.854 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 14673 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/rtc/rtc-max8907.ko 2023-11-26T07:02:48.854 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 15345 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/rtc/rtc-max8925.ko 2023-11-26T07:02:48.854 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 25809 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/rtc/rtc-max8997.ko 2023-11-26T07:02:48.854 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 16681 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/rtc/rtc-max8998.ko 2023-11-26T07:02:48.854 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 18513 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/rtc/rtc-mc13xxx.ko 2023-11-26T07:02:48.855 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 11409 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/rtc/rtc-msm6242.ko 2023-11-26T07:02:48.855 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 19625 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/rtc/rtc-palmas.ko 2023-11-26T07:02:48.855 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 46633 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/rtc/rtc-pcf2127.ko 2023-11-26T07:02:48.855 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 17977 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/rtc/rtc-pcf50633.ko 2023-11-26T07:02:48.855 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 26993 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/rtc/rtc-pcf85063.ko 2023-11-26T07:02:48.855 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 21825 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/rtc/rtc-pcf8523.ko 2023-11-26T07:02:48.855 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 29817 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/rtc/rtc-pcf8563.ko 2023-11-26T07:02:48.855 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 12441 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/rtc/rtc-pcf8583.ko 2023-11-26T07:02:48.855 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 18033 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/rtc/rtc-rc5t583.ko 2023-11-26T07:02:48.855 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 13049 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/rtc/rtc-rp5c01.ko 2023-11-26T07:02:48.855 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 33753 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/rtc/rtc-rs5c372.ko 2023-11-26T07:02:48.855 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 27833 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/rtc/rtc-rv3029c2.ko 2023-11-26T07:02:48.855 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 25257 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/rtc/rtc-rx8025.ko 2023-11-26T07:02:48.856 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 20113 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/rtc/rtc-rx8581.ko 2023-11-26T07:02:48.856 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 21617 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/rtc/rtc-s35390a.ko 2023-11-26T07:02:48.856 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 16825 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/rtc/rtc-stk17ta8.ko 2023-11-26T07:02:48.856 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 17889 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/rtc/rtc-tps6586x.ko 2023-11-26T07:02:48.856 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 21337 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/rtc/rtc-tps65910.ko 2023-11-26T07:02:48.856 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 20545 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/rtc/rtc-wm831x.ko 2023-11-26T07:02:48.856 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 21089 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/rtc/rtc-wm8350.ko 2023-11-26T07:02:48.856 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 27873 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/rtc/rtc-x1205.ko 2023-11-26T07:02:48.856 INFO:teuthology.orchestra.run.smithi143.stdout:drwxr-xr-x root/root 0 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/scsi/ 2023-11-26T07:02:48.856 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 77617 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/scsi/3w-9xxx.ko 2023-11-26T07:02:48.856 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 57665 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/scsi/3w-sas.ko 2023-11-26T07:02:48.856 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 55105 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/scsi/3w-xxxx.ko 2023-11-26T07:02:48.857 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 135185 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/scsi/BusLogic.ko 2023-11-26T07:02:48.857 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 26657 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/scsi/a100u2w.ko 2023-11-26T07:02:48.857 INFO:teuthology.orchestra.run.smithi143.stdout:drwxr-xr-x root/root 0 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/scsi/aacraid/ 2023-11-26T07:02:48.857 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 272305 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/scsi/aacraid/aacraid.ko 2023-11-26T07:02:48.857 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 100617 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/scsi/advansys.ko 2023-11-26T07:02:48.857 INFO:teuthology.orchestra.run.smithi143.stdout:drwxr-xr-x root/root 0 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/scsi/aic7xxx/ 2023-11-26T07:02:48.857 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 356201 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/scsi/aic7xxx/aic79xx.ko 2023-11-26T07:02:48.857 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 295801 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/scsi/aic7xxx/aic7xxx.ko 2023-11-26T07:02:48.857 INFO:teuthology.orchestra.run.smithi143.stdout:drwxr-xr-x root/root 0 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/scsi/aic94xx/ 2023-11-26T07:02:48.857 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 189185 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/scsi/aic94xx/aic94xx.ko 2023-11-26T07:02:48.857 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 24073 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/scsi/am53c974.ko 2023-11-26T07:02:48.858 INFO:teuthology.orchestra.run.smithi143.stdout:drwxr-xr-x root/root 0 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/scsi/arcmsr/ 2023-11-26T07:02:48.858 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 106881 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/scsi/arcmsr/arcmsr.ko 2023-11-26T07:02:48.858 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 38185 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/scsi/atp870u.ko 2023-11-26T07:02:48.858 INFO:teuthology.orchestra.run.smithi143.stdout:drwxr-xr-x root/root 0 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/scsi/be2iscsi/ 2023-11-26T07:02:48.858 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 246985 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/scsi/be2iscsi/be2iscsi.ko 2023-11-26T07:02:48.858 INFO:teuthology.orchestra.run.smithi143.stdout:drwxr-xr-x root/root 0 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/scsi/bfa/ 2023-11-26T07:02:48.858 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 1171945 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/scsi/bfa/bfa.ko 2023-11-26T07:02:48.858 INFO:teuthology.orchestra.run.smithi143.stdout:drwxr-xr-x root/root 0 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/scsi/bnx2fc/ 2023-11-26T07:02:48.859 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 237273 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/scsi/bnx2fc/bnx2fc.ko 2023-11-26T07:02:48.859 INFO:teuthology.orchestra.run.smithi143.stdout:drwxr-xr-x root/root 0 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/scsi/bnx2i/ 2023-11-26T07:02:48.859 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 115369 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/scsi/bnx2i/bnx2i.ko 2023-11-26T07:02:48.859 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 40585 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/scsi/ch.ko 2023-11-26T07:02:48.859 INFO:teuthology.orchestra.run.smithi143.stdout:drwxr-xr-x root/root 0 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/scsi/csiostor/ 2023-11-26T07:02:48.859 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 308921 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/scsi/csiostor/csiostor.ko 2023-11-26T07:02:48.859 INFO:teuthology.orchestra.run.smithi143.stdout:drwxr-xr-x root/root 0 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/scsi/cxgbi/ 2023-11-26T07:02:48.859 INFO:teuthology.orchestra.run.smithi143.stdout:drwxr-xr-x root/root 0 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/scsi/cxgbi/cxgb3i/ 2023-11-26T07:02:48.859 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 76145 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/scsi/cxgbi/cxgb3i/cxgb3i.ko 2023-11-26T07:02:48.860 INFO:teuthology.orchestra.run.smithi143.stdout:drwxr-xr-x root/root 0 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/scsi/cxgbi/cxgb4i/ 2023-11-26T07:02:48.860 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 101449 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/scsi/cxgbi/cxgb4i/cxgb4i.ko 2023-11-26T07:02:48.860 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 146689 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/scsi/cxgbi/libcxgbi.ko 2023-11-26T07:02:48.860 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 75553 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/scsi/dc395x.ko 2023-11-26T07:02:48.861 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 38073 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/scsi/dmx3191d.ko 2023-11-26T07:02:48.861 INFO:teuthology.orchestra.run.smithi143.stdout:drwxr-xr-x root/root 0 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/scsi/esas2r/ 2023-11-26T07:02:48.861 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 212425 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/scsi/esas2r/esas2r.ko 2023-11-26T07:02:48.861 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 60505 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/scsi/esp_scsi.ko 2023-11-26T07:02:48.861 INFO:teuthology.orchestra.run.smithi143.stdout:drwxr-xr-x root/root 0 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/scsi/fcoe/ 2023-11-26T07:02:48.861 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 78209 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/scsi/fcoe/fcoe.ko 2023-11-26T07:02:48.861 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 158505 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/scsi/fcoe/libfcoe.ko 2023-11-26T07:02:48.861 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 18945 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/scsi/fdomain.ko 2023-11-26T07:02:48.862 INFO:teuthology.orchestra.run.smithi143.stdout:drwxr-xr-x root/root 0 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/scsi/fnic/ 2023-11-26T07:02:48.862 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 263489 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/scsi/fnic/fnic.ko 2023-11-26T07:02:48.862 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 201817 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/scsi/hpsa.ko 2023-11-26T07:02:48.862 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 51209 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/scsi/hptiop.ko 2023-11-26T07:02:48.862 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 49041 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/scsi/hv_storvsc.ko 2023-11-26T07:02:48.862 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 35337 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/scsi/imm.ko 2023-11-26T07:02:48.862 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 44057 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/scsi/initio.ko 2023-11-26T07:02:48.862 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 266537 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/scsi/ipr.ko 2023-11-26T07:02:48.863 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 101881 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/scsi/ips.ko 2023-11-26T07:02:48.863 INFO:teuthology.orchestra.run.smithi143.stdout:drwxr-xr-x root/root 0 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/scsi/isci/ 2023-11-26T07:02:48.863 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 317153 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/scsi/isci/isci.ko 2023-11-26T07:02:48.863 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 26385 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/scsi/iscsi_boot_sysfs.ko 2023-11-26T07:02:48.863 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 52961 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/scsi/iscsi_tcp.ko 2023-11-26T07:02:48.863 INFO:teuthology.orchestra.run.smithi143.stdout:drwxr-xr-x root/root 0 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/scsi/libfc/ 2023-11-26T07:02:48.863 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 314865 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/scsi/libfc/libfc.ko 2023-11-26T07:02:48.863 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 145585 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/scsi/libiscsi.ko 2023-11-26T07:02:48.863 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 53521 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/scsi/libiscsi_tcp.ko 2023-11-26T07:02:48.863 INFO:teuthology.orchestra.run.smithi143.stdout:drwxr-xr-x root/root 0 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/scsi/libsas/ 2023-11-26T07:02:48.864 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 216665 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/scsi/libsas/libsas.ko 2023-11-26T07:02:48.864 INFO:teuthology.orchestra.run.smithi143.stdout:drwxr-xr-x root/root 0 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/scsi/lpfc/ 2023-11-26T07:02:48.864 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 2197249 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/scsi/lpfc/lpfc.ko 2023-11-26T07:02:48.864 INFO:teuthology.orchestra.run.smithi143.stdout:drwxr-xr-x root/root 0 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/scsi/megaraid/ 2023-11-26T07:02:48.864 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 73569 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/scsi/megaraid/megaraid_mbox.ko 2023-11-26T07:02:48.864 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 33529 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/scsi/megaraid/megaraid_mm.ko 2023-11-26T07:02:48.864 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 310617 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/scsi/megaraid/megaraid_sas.ko 2023-11-26T07:02:48.864 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 84721 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/scsi/megaraid.ko 2023-11-26T07:02:48.864 INFO:teuthology.orchestra.run.smithi143.stdout:drwxr-xr-x root/root 0 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/scsi/mpt3sas/ 2023-11-26T07:02:48.864 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 659825 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/scsi/mpt3sas/mpt3sas.ko 2023-11-26T07:02:48.864 INFO:teuthology.orchestra.run.smithi143.stdout:drwxr-xr-x root/root 0 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/scsi/mvsas/ 2023-11-26T07:02:48.864 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 119561 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/scsi/mvsas/mvsas.ko 2023-11-26T07:02:48.865 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 78129 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/scsi/mvumi.ko 2023-11-26T07:02:48.865 INFO:teuthology.orchestra.run.smithi143.stdout:drwxr-xr-x root/root 0 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/scsi/pcmcia/ 2023-11-26T07:02:48.865 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 81625 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/scsi/pcmcia/aha152x_cs.ko 2023-11-26T07:02:48.865 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 11713 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/scsi/pcmcia/fdomain_cs.ko 2023-11-26T07:02:48.865 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 20865 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/scsi/pcmcia/qlogic_cs.ko 2023-11-26T07:02:48.865 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 23857 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/scsi/pcmcia/sym53c500_cs.ko 2023-11-26T07:02:48.865 INFO:teuthology.orchestra.run.smithi143.stdout:drwxr-xr-x root/root 0 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/scsi/pm8001/ 2023-11-26T07:02:48.865 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 364105 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/scsi/pm8001/pm80xx.ko 2023-11-26T07:02:48.865 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 123617 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/scsi/pmcraid.ko 2023-11-26T07:02:48.865 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 34513 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/scsi/ppa.ko 2023-11-26T07:02:48.865 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 55361 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/scsi/qla1280.ko 2023-11-26T07:02:48.865 INFO:teuthology.orchestra.run.smithi143.stdout:drwxr-xr-x root/root 0 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/scsi/qla2xxx/ 2023-11-26T07:02:48.865 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 1876689 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/scsi/qla2xxx/qla2xxx.ko 2023-11-26T07:02:48.866 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 85289 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/scsi/qla2xxx/tcm_qla2xxx.ko 2023-11-26T07:02:48.866 INFO:teuthology.orchestra.run.smithi143.stdout:drwxr-xr-x root/root 0 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/scsi/qla4xxx/ 2023-11-26T07:02:48.866 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 559033 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/scsi/qla4xxx/qla4xxx.ko 2023-11-26T07:02:48.866 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 19025 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/scsi/qlogicfas408.ko 2023-11-26T07:02:48.867 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 15993 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/scsi/raid_class.ko 2023-11-26T07:02:48.867 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 238761 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/scsi/scsi_debug.ko 2023-11-26T07:02:48.867 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 173937 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/scsi/scsi_transport_fc.ko 2023-11-26T07:02:48.867 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 279313 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/scsi/scsi_transport_iscsi.ko 2023-11-26T07:02:48.867 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 97729 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/scsi/scsi_transport_sas.ko 2023-11-26T07:02:48.867 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 73881 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/scsi/scsi_transport_spi.ko 2023-11-26T07:02:48.867 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 44993 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/scsi/scsi_transport_srp.ko 2023-11-26T07:02:48.867 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 26249 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/scsi/ses.ko 2023-11-26T07:02:48.867 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 115889 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/scsi/st.ko 2023-11-26T07:02:48.867 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 46825 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/scsi/stex.ko 2023-11-26T07:02:48.867 INFO:teuthology.orchestra.run.smithi143.stdout:drwxr-xr-x root/root 0 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/scsi/sym53c8xx_2/ 2023-11-26T07:02:48.867 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 162121 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/scsi/sym53c8xx_2/sym53c8xx.ko 2023-11-26T07:02:48.868 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 42297 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/scsi/virtio_scsi.ko 2023-11-26T07:02:48.868 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 50321 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/scsi/vmw_pvscsi.ko 2023-11-26T07:02:48.868 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 39961 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/scsi/wd719x.ko 2023-11-26T07:02:48.868 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 50025 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/scsi/xen-scsifront.ko 2023-11-26T07:02:48.868 INFO:teuthology.orchestra.run.smithi143.stdout:drwxr-xr-x root/root 0 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/ssb/ 2023-11-26T07:02:48.868 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 129441 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/ssb/ssb.ko 2023-11-26T07:02:48.868 INFO:teuthology.orchestra.run.smithi143.stdout:drwxr-xr-x root/root 0 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/target/ 2023-11-26T07:02:48.868 INFO:teuthology.orchestra.run.smithi143.stdout:drwxr-xr-x root/root 0 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/target/iscsi/ 2023-11-26T07:02:48.868 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 736601 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/target/iscsi/iscsi_target_mod.ko 2023-11-26T07:02:48.868 INFO:teuthology.orchestra.run.smithi143.stdout:drwxr-xr-x root/root 0 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/target/loopback/ 2023-11-26T07:02:48.868 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 49985 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/target/loopback/tcm_loop.ko 2023-11-26T07:02:48.868 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 39713 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/target/target_core_file.ko 2023-11-26T07:02:48.869 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 43025 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/target/target_core_iblock.ko 2023-11-26T07:02:48.869 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 930257 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/target/target_core_mod.ko 2023-11-26T07:02:48.869 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 44305 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/target/target_core_pscsi.ko 2023-11-26T07:02:48.869 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 122521 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/target/target_core_user.ko 2023-11-26T07:02:48.869 INFO:teuthology.orchestra.run.smithi143.stdout:drwxr-xr-x root/root 0 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/target/tcm_fc/ 2023-11-26T07:02:48.869 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 75593 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/target/tcm_fc/tcm_fc.ko 2023-11-26T07:02:48.869 INFO:teuthology.orchestra.run.smithi143.stdout:drwxr-xr-x root/root 0 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/thermal/ 2023-11-26T07:02:48.869 INFO:teuthology.orchestra.run.smithi143.stdout:drwxr-xr-x root/root 0 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/thermal/intel/ 2023-11-26T07:02:48.869 INFO:teuthology.orchestra.run.smithi143.stdout:drwxr-xr-x root/root 0 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/thermal/intel/int340x_thermal/ 2023-11-26T07:02:48.869 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 23481 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/thermal/intel/int340x_thermal/acpi_thermal_rel.ko 2023-11-26T07:02:48.869 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 29321 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/thermal/intel/int340x_thermal/int3400_thermal.ko 2023-11-26T07:02:48.869 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 9953 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/thermal/intel/int340x_thermal/int3401_thermal.ko 2023-11-26T07:02:48.870 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 10257 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/thermal/intel/int340x_thermal/int3402_thermal.ko 2023-11-26T07:02:48.870 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 15081 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/thermal/intel/int340x_thermal/int3403_thermal.ko 2023-11-26T07:02:48.870 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 16497 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/thermal/intel/int340x_thermal/int340x_thermal_zone.ko 2023-11-26T07:02:48.870 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 26809 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_device.ko 2023-11-26T07:02:48.870 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 20281 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_device_pci.ko 2023-11-26T07:02:48.870 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 14865 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_device_pci_legacy.ko 2023-11-26T07:02:48.870 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 14393 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_mbox.ko 2023-11-26T07:02:48.870 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 13001 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_rapl.ko 2023-11-26T07:02:48.870 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 37065 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_rfim.ko 2023-11-26T07:02:48.870 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 30729 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/thermal/intel/intel_powerclamp.ko 2023-11-26T07:02:48.870 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 18937 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/thermal/intel/intel_soc_dts_iosf.ko 2023-11-26T07:02:48.870 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 11521 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/thermal/intel/intel_soc_dts_thermal.ko 2023-11-26T07:02:48.871 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 26425 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/thermal/intel/x86_pkg_temp_thermal.ko 2023-11-26T07:02:48.871 INFO:teuthology.orchestra.run.smithi143.stdout:drwxr-xr-x root/root 0 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/tty/ 2023-11-26T07:02:48.871 INFO:teuthology.orchestra.run.smithi143.stdout:drwxr-xr-x root/root 0 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/tty/ipwireless/ 2023-11-26T07:02:48.871 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 78425 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/tty/ipwireless/ipwireless.ko 2023-11-26T07:02:48.871 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 57865 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/tty/moxa.ko 2023-11-26T07:02:48.871 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 50953 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/tty/mxser.ko 2023-11-26T07:02:48.871 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 108601 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/tty/n_gsm.ko 2023-11-26T07:02:48.871 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 31961 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/tty/n_hdlc.ko 2023-11-26T07:02:48.871 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 67929 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/tty/nozomi.ko 2023-11-26T07:02:48.871 INFO:teuthology.orchestra.run.smithi143.stdout:drwxr-xr-x root/root 0 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/tty/serial/ 2023-11-26T07:02:48.871 INFO:teuthology.orchestra.run.smithi143.stdout:drwxr-xr-x root/root 0 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/tty/serial/8250/ 2023-11-26T07:02:48.871 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 35753 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/tty/serial/8250/8250_dw.ko 2023-11-26T07:02:48.872 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 59497 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/tty/serial/8250/serial_cs.ko 2023-11-26T07:02:48.872 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 18097 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/tty/serial/altera_jtaguart.ko 2023-11-26T07:02:48.872 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 22577 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/tty/serial/altera_uart.ko 2023-11-26T07:02:48.872 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 9425 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/tty/serial/arc_uart.ko 2023-11-26T07:02:48.872 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 72425 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/tty/serial/fsl_lpuart.ko 2023-11-26T07:02:48.872 INFO:teuthology.orchestra.run.smithi143.stdout:drwxr-xr-x root/root 0 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/tty/serial/jsm/ 2023-11-26T07:02:48.872 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 103073 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/tty/serial/jsm/jsm.ko 2023-11-26T07:02:48.872 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 28785 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/tty/serial/rp2.ko 2023-11-26T07:02:48.872 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 52817 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/tty/serial/sc16is7xx.ko 2023-11-26T07:02:48.872 INFO:teuthology.orchestra.run.smithi143.stdout:drwxr-xr-x root/root 0 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/ufs/ 2023-11-26T07:02:48.872 INFO:teuthology.orchestra.run.smithi143.stdout:drwxr-xr-x root/root 0 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/ufs/core/ 2023-11-26T07:02:48.872 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 531265 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/ufs/core/ufshcd-core.ko 2023-11-26T07:02:48.873 INFO:teuthology.orchestra.run.smithi143.stdout:drwxr-xr-x root/root 0 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/ufs/host/ 2023-11-26T07:02:48.873 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 28481 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/ufs/host/ufshcd-pci.ko 2023-11-26T07:02:48.873 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 14601 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/ufs/host/ufshcd-pltfrm.ko 2023-11-26T07:02:48.873 INFO:teuthology.orchestra.run.smithi143.stdout:drwxr-xr-x root/root 0 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/uio/ 2023-11-26T07:02:48.873 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 44513 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/uio/uio.ko 2023-11-26T07:02:48.873 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 12241 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/uio/uio_aec.ko 2023-11-26T07:02:48.873 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 11281 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/uio/uio_cif.ko 2023-11-26T07:02:48.873 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 18329 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/uio/uio_dmem_genirq.ko 2023-11-26T07:02:48.873 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 12937 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/uio/uio_mf624.ko 2023-11-26T07:02:48.873 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 12177 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/uio/uio_netx.ko 2023-11-26T07:02:48.873 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 10601 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/uio/uio_pci_generic.ko 2023-11-26T07:02:48.873 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 16057 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/uio/uio_pdrv_genirq.ko 2023-11-26T07:02:48.874 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 13633 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/uio/uio_sercos3.ko 2023-11-26T07:02:48.874 INFO:teuthology.orchestra.run.smithi143.stdout:drwxr-xr-x root/root 0 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/ 2023-11-26T07:02:48.874 INFO:teuthology.orchestra.run.smithi143.stdout:drwxr-xr-x root/root 0 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/c67x00/ 2023-11-26T07:02:48.874 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 73297 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/c67x00/c67x00.ko 2023-11-26T07:02:48.874 INFO:teuthology.orchestra.run.smithi143.stdout:drwxr-xr-x root/root 0 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/chipidea/ 2023-11-26T07:02:48.874 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 144377 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/chipidea/ci_hdrc.ko 2023-11-26T07:02:48.874 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 18505 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/chipidea/ci_hdrc_msm.ko 2023-11-26T07:02:48.874 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 12713 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/chipidea/ci_hdrc_pci.ko 2023-11-26T07:02:48.874 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 13057 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/chipidea/ci_hdrc_usb2.ko 2023-11-26T07:02:48.874 INFO:teuthology.orchestra.run.smithi143.stdout:drwxr-xr-x root/root 0 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/class/ 2023-11-26T07:02:48.874 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 87089 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/class/cdc-acm.ko 2023-11-26T07:02:48.874 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 51345 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/class/cdc-wdm.ko 2023-11-26T07:02:48.875 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 47305 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/class/usblp.ko 2023-11-26T07:02:48.875 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 89857 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/class/usbtmc.ko 2023-11-26T07:02:48.875 INFO:teuthology.orchestra.run.smithi143.stdout:drwxr-xr-x root/root 0 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/common/ 2023-11-26T07:02:48.875 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 22953 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/common/ulpi.ko 2023-11-26T07:02:48.875 INFO:teuthology.orchestra.run.smithi143.stdout:drwxr-xr-x root/root 0 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/dwc2/ 2023-11-26T07:02:48.875 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 11529 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/dwc2/dwc2_pci.ko 2023-11-26T07:02:48.875 INFO:teuthology.orchestra.run.smithi143.stdout:drwxr-xr-x root/root 0 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/dwc3/ 2023-11-26T07:02:48.875 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 12089 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/dwc3/dwc3-haps.ko 2023-11-26T07:02:48.875 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 30041 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/dwc3/dwc3-pci.ko 2023-11-26T07:02:48.875 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 448505 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/dwc3/dwc3.ko 2023-11-26T07:02:48.875 INFO:teuthology.orchestra.run.smithi143.stdout:drwxr-xr-x root/root 0 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/gadget/ 2023-11-26T07:02:48.875 INFO:teuthology.orchestra.run.smithi143.stdout:drwxr-xr-x root/root 0 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/gadget/function/ 2023-11-26T07:02:48.876 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 60177 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/gadget/function/u_ether.ko 2023-11-26T07:02:48.876 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 51881 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/gadget/function/u_serial.ko 2023-11-26T07:02:48.876 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 28841 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/gadget/function/usb_f_acm.ko 2023-11-26T07:02:48.876 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 42449 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/gadget/function/usb_f_ecm.ko 2023-11-26T07:02:48.876 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 27953 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/gadget/function/usb_f_ecm_subset.ko 2023-11-26T07:02:48.876 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 36513 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/gadget/function/usb_f_eem.ko 2023-11-26T07:02:48.876 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 92489 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/gadget/function/usb_f_fs.ko 2023-11-26T07:02:48.876 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 51049 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/gadget/function/usb_f_hid.ko 2023-11-26T07:02:48.876 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 122081 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/gadget/function/usb_f_mass_storage.ko 2023-11-26T07:02:48.876 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 57681 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/gadget/function/usb_f_ncm.ko 2023-11-26T07:02:48.876 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 22185 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/gadget/function/usb_f_obex.ko 2023-11-26T07:02:48.876 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 31265 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/gadget/function/usb_f_phonet.ko 2023-11-26T07:02:48.876 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 57729 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/gadget/function/usb_f_printer.ko 2023-11-26T07:02:48.877 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 85609 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/gadget/function/usb_f_rndis.ko 2023-11-26T07:02:48.877 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 19273 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/gadget/function/usb_f_serial.ko 2023-11-26T07:02:48.877 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 58113 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/gadget/function/usb_f_ss_lb.ko 2023-11-26T07:02:48.877 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 67337 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/gadget/function/usb_f_tcm.ko 2023-11-26T07:02:48.877 INFO:teuthology.orchestra.run.smithi143.stdout:drwxr-xr-x root/root 0 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/gadget/legacy/ 2023-11-26T07:02:48.877 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 21265 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/gadget/legacy/g_acm_ms.ko 2023-11-26T07:02:48.877 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 18057 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/gadget/legacy/g_cdc.ko 2023-11-26T07:02:48.877 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 18465 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/gadget/legacy/g_dbgp.ko 2023-11-26T07:02:48.877 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 21657 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/gadget/legacy/g_ether.ko 2023-11-26T07:02:48.877 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 28401 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/gadget/legacy/g_ffs.ko 2023-11-26T07:02:48.878 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 19009 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/gadget/legacy/g_hid.ko 2023-11-26T07:02:48.878 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 20409 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/gadget/legacy/g_mass_storage.ko 2023-11-26T07:02:48.878 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 16977 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/gadget/legacy/g_ncm.ko 2023-11-26T07:02:48.878 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 32073 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/gadget/legacy/g_nokia.ko 2023-11-26T07:02:48.878 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 16793 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/gadget/legacy/g_printer.ko 2023-11-26T07:02:48.878 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 19393 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/gadget/legacy/g_serial.ko 2023-11-26T07:02:48.878 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 27201 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/gadget/legacy/g_zero.ko 2023-11-26T07:02:48.878 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 62185 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/gadget/legacy/gadgetfs.ko 2023-11-26T07:02:48.878 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 15201 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/gadget/legacy/tcm_usb_gadget.ko 2023-11-26T07:02:48.878 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 161225 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/gadget/libcomposite.ko 2023-11-26T07:02:48.879 INFO:teuthology.orchestra.run.smithi143.stdout:drwxr-xr-x root/root 0 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/gadget/udc/ 2023-11-26T07:02:48.879 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 17705 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/gadget/udc/amd5536udc_pci.ko 2023-11-26T07:02:48.879 INFO:teuthology.orchestra.run.smithi143.stdout:drwxr-xr-x root/root 0 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/gadget/udc/bdc/ 2023-11-26T07:02:48.879 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 157681 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/gadget/udc/bdc/bdc.ko 2023-11-26T07:02:48.879 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 36089 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/gadget/udc/goku_udc.ko 2023-11-26T07:02:48.879 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 52721 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/gadget/udc/gr_udc.ko 2023-11-26T07:02:48.879 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 69705 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/gadget/udc/mv_u3d_core.ko 2023-11-26T07:02:48.879 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 56033 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/gadget/udc/mv_udc.ko 2023-11-26T07:02:48.879 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 76057 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/gadget/udc/net2272.ko 2023-11-26T07:02:48.879 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 75073 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/gadget/udc/net2280.ko 2023-11-26T07:02:48.879 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 73169 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/gadget/udc/pch_udc.ko 2023-11-26T07:02:48.880 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 65809 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/gadget/udc/pxa27x_udc.ko 2023-11-26T07:02:48.880 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 66097 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/gadget/udc/r8a66597-udc.ko 2023-11-26T07:02:48.880 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 81505 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/gadget/udc/snps_udc_core.ko 2023-11-26T07:02:48.880 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 156073 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/gadget/udc/udc-core.ko 2023-11-26T07:02:48.880 INFO:teuthology.orchestra.run.smithi143.stdout:drwxr-xr-x root/root 0 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/host/ 2023-11-26T07:02:48.880 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 11585 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/host/bcma-hcd.ko 2023-11-26T07:02:48.880 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 80113 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/host/isp116x-hcd.ko 2023-11-26T07:02:48.880 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 87353 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/host/oxu210hp-hcd.ko 2023-11-26T07:02:48.880 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 78873 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/host/r8a66597-hcd.ko 2023-11-26T07:02:48.880 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 51329 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/host/sl811-hcd.ko 2023-11-26T07:02:48.880 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 15593 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/host/sl811_cs.ko 2023-11-26T07:02:48.881 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 13305 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/host/ssb-hcd.ko 2023-11-26T07:02:48.881 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 25945 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/host/xhci-plat-hcd.ko 2023-11-26T07:02:48.881 INFO:teuthology.orchestra.run.smithi143.stdout:drwxr-xr-x root/root 0 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/image/ 2023-11-26T07:02:48.881 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 34585 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/image/mdc800.ko 2023-11-26T07:02:48.881 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 21081 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/image/microtek.ko 2023-11-26T07:02:48.881 INFO:teuthology.orchestra.run.smithi143.stdout:drwxr-xr-x root/root 0 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/misc/ 2023-11-26T07:02:48.881 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 41721 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/misc/adutux.ko 2023-11-26T07:02:48.881 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 20713 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/misc/appledisplay.ko 2023-11-26T07:02:48.881 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 17721 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/misc/cypress_cy7c63.ko 2023-11-26T07:02:48.881 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 23049 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/misc/cytherm.ko 2023-11-26T07:02:48.881 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 13729 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/misc/ehset.ko 2023-11-26T07:02:48.881 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 15057 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/misc/emi26.ko 2023-11-26T07:02:48.882 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 15985 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/misc/emi62.ko 2023-11-26T07:02:48.882 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 11433 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/misc/ezusb.ko 2023-11-26T07:02:48.882 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 19297 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/misc/idmouse.ko 2023-11-26T07:02:48.882 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 36553 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/misc/iowarrior.ko 2023-11-26T07:02:48.882 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 11593 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/misc/isight_firmware.ko 2023-11-26T07:02:48.882 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 36529 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/misc/ldusb.ko 2023-11-26T07:02:48.882 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 36265 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/misc/legousbtower.ko 2023-11-26T07:02:48.882 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 27793 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/misc/lvstest.ko 2023-11-26T07:02:48.882 INFO:teuthology.orchestra.run.smithi143.stdout:drwxr-xr-x root/root 0 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/misc/sisusbvga/ 2023-11-26T07:02:48.882 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 64841 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/misc/sisusbvga/sisusbvga.ko 2023-11-26T07:02:48.882 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 12937 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/misc/trancevibrator.ko 2023-11-26T07:02:48.882 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 20473 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/misc/usb3503.ko 2023-11-26T07:02:48.883 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 25145 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/misc/usblcd.ko 2023-11-26T07:02:48.883 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 22753 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/misc/usbsevseg.ko 2023-11-26T07:02:48.883 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 70809 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/misc/usbtest.ko 2023-11-26T07:02:48.883 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 41905 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/misc/uss720.ko 2023-11-26T07:02:48.883 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 27617 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/misc/yurex.ko 2023-11-26T07:02:48.883 INFO:teuthology.orchestra.run.smithi143.stdout:drwxr-xr-x root/root 0 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/mon/ 2023-11-26T07:02:48.883 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 72953 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/mon/usbmon.ko 2023-11-26T07:02:48.883 INFO:teuthology.orchestra.run.smithi143.stdout:drwxr-xr-x root/root 0 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/musb/ 2023-11-26T07:02:48.883 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 301697 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/musb/musb_hdrc.ko 2023-11-26T07:02:48.883 INFO:teuthology.orchestra.run.smithi143.stdout:drwxr-xr-x root/root 0 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/phy/ 2023-11-26T07:02:48.883 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 21105 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/phy/phy-generic.ko 2023-11-26T07:02:48.883 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 20177 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/phy/phy-gpio-vbus-usb.ko 2023-11-26T07:02:48.884 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 12257 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/phy/phy-isp1301.ko 2023-11-26T07:02:48.884 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 21745 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/phy/phy-tahvo.ko 2023-11-26T07:02:48.884 INFO:teuthology.orchestra.run.smithi143.stdout:drwxr-xr-x root/root 0 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/serial/ 2023-11-26T07:02:48.884 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 13209 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/serial/aircable.ko 2023-11-26T07:02:48.884 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 33329 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/serial/ark3116.ko 2023-11-26T07:02:48.884 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 26633 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/serial/belkin_sa.ko 2023-11-26T07:02:48.884 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 38105 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/serial/ch341.ko 2023-11-26T07:02:48.884 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 70257 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/serial/cp210x.ko 2023-11-26T07:02:48.884 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 31057 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/serial/cyberjack.ko 2023-11-26T07:02:48.884 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 58217 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/serial/cypress_m8.ko 2023-11-26T07:02:48.884 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 49817 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/serial/digi_acceleport.ko 2023-11-26T07:02:48.884 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 9617 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/serial/empeg.ko 2023-11-26T07:02:48.884 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 44273 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/serial/f81232.ko 2023-11-26T07:02:48.885 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 175137 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/serial/ftdi_sio.ko 2023-11-26T07:02:48.885 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 53113 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/serial/garmin_gps.ko 2023-11-26T07:02:48.885 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 137081 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/serial/io_edgeport.ko 2023-11-26T07:02:48.885 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 124833 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/serial/io_ti.ko 2023-11-26T07:02:48.885 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 67769 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/serial/ipaq.ko 2023-11-26T07:02:48.885 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 19161 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/serial/ipw.ko 2023-11-26T07:02:48.885 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 26897 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/serial/ir-usb.ko 2023-11-26T07:02:48.885 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 58233 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/serial/iuu_phoenix.ko 2023-11-26T07:02:48.885 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 107521 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/serial/keyspan.ko 2023-11-26T07:02:48.885 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 35617 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/serial/keyspan_pda.ko 2023-11-26T07:02:48.885 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 25665 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/serial/kl5kusb105.ko 2023-11-26T07:02:48.886 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 33689 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/serial/kobil_sct.ko 2023-11-26T07:02:48.886 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 32953 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/serial/mct_u232.ko 2023-11-26T07:02:48.886 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 22033 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/serial/metro-usb.ko 2023-11-26T07:02:48.886 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 66449 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/serial/mos7720.ko 2023-11-26T07:02:48.886 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 78217 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/serial/mos7840.ko 2023-11-26T07:02:48.886 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 56337 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/serial/mxuport.ko 2023-11-26T07:02:48.886 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 14025 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/serial/navman.ko 2023-11-26T07:02:48.886 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 12009 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/serial/omninet.ko 2023-11-26T07:02:48.886 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 24633 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/serial/opticon.ko 2023-11-26T07:02:48.886 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 191137 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/serial/option.ko 2023-11-26T07:02:48.886 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 39073 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/serial/oti6858.ko 2023-11-26T07:02:48.886 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 54353 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/serial/pl2303.ko 2023-11-26T07:02:48.887 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 9737 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/serial/qcaux.ko 2023-11-26T07:02:48.887 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 37825 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/serial/qcserial.ko 2023-11-26T07:02:48.887 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 34321 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/serial/quatech2.ko 2023-11-26T07:02:48.887 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 15393 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/serial/safe_serial.ko 2023-11-26T07:02:48.887 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 51625 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/serial/sierra.ko 2023-11-26T07:02:48.887 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 21673 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/serial/spcp8x5.ko 2023-11-26T07:02:48.887 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 29993 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/serial/ssu100.ko 2023-11-26T07:02:48.887 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 17265 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/serial/symbolserial.ko 2023-11-26T07:02:48.887 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 61489 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/serial/ti_usb_3410_5052.ko 2023-11-26T07:02:48.887 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 23729 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/serial/usb-serial-simple.ko 2023-11-26T07:02:48.887 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 10241 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/serial/usb_debug.ko 2023-11-26T07:02:48.887 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 33201 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/serial/usb_wwan.ko 2023-11-26T07:02:48.888 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 125609 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/serial/usbserial.ko 2023-11-26T07:02:48.888 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 28753 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/serial/visor.ko 2023-11-26T07:02:48.888 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 37409 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/serial/whiteheat.ko 2023-11-26T07:02:48.888 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 10537 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/serial/wishbone-serial.ko 2023-11-26T07:02:48.888 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 9289 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/serial/xsens_mt.ko 2023-11-26T07:02:48.888 INFO:teuthology.orchestra.run.smithi143.stdout:drwxr-xr-x root/root 0 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/storage/ 2023-11-26T07:02:48.888 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 52913 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/storage/uas.ko 2023-11-26T07:02:48.888 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 26017 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/storage/ums-alauda.ko 2023-11-26T07:02:48.888 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 13049 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/storage/ums-cypress.ko 2023-11-26T07:02:48.888 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 22049 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/storage/ums-datafab.ko 2023-11-26T07:02:48.888 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 33497 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/storage/ums-eneub6250.ko 2023-11-26T07:02:48.888 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 13745 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/storage/ums-freecom.ko 2023-11-26T07:02:48.889 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 19769 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/storage/ums-isd200.ko 2023-11-26T07:02:48.889 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 19361 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/storage/ums-jumpshot.ko 2023-11-26T07:02:48.889 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 13713 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/storage/ums-karma.ko 2023-11-26T07:02:48.889 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 17841 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/storage/ums-onetouch.ko 2023-11-26T07:02:48.889 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 28441 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/storage/ums-realtek.ko 2023-11-26T07:02:48.889 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 28905 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/storage/ums-sddr09.ko 2023-11-26T07:02:48.889 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 19521 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/storage/ums-sddr55.ko 2023-11-26T07:02:48.889 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 35337 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/storage/ums-usbat.ko 2023-11-26T07:02:48.889 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 179969 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/storage/usb-storage.ko 2023-11-26T07:02:48.889 INFO:teuthology.orchestra.run.smithi143.stdout:drwxr-xr-x root/root 0 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/usbip/ 2023-11-26T07:02:48.889 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 67481 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/usbip/usbip-core.ko 2023-11-26T07:02:48.889 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 72177 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/usbip/usbip-host.ko 2023-11-26T07:02:48.890 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 107001 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/usbip/vhci-hcd.ko 2023-11-26T07:02:48.890 INFO:teuthology.orchestra.run.smithi143.stdout:drwxr-xr-x root/root 0 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/vfio/ 2023-11-26T07:02:48.890 INFO:teuthology.orchestra.run.smithi143.stdout:drwxr-xr-x root/root 0 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/vfio/pci/ 2023-11-26T07:02:48.890 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 154297 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/vfio/pci/vfio-pci-core.ko 2023-11-26T07:02:48.890 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 29537 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/vfio/pci/vfio-pci.ko 2023-11-26T07:02:48.890 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 108737 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/vfio/vfio.ko 2023-11-26T07:02:48.890 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 77609 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/vfio/vfio_iommu_type1.ko 2023-11-26T07:02:48.890 INFO:teuthology.orchestra.run.smithi143.stdout:drwxr-xr-x root/root 0 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/vhost/ 2023-11-26T07:02:48.890 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 113785 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/vhost/vhost.ko 2023-11-26T07:02:48.890 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 17009 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/vhost/vhost_iotlb.ko 2023-11-26T07:02:48.890 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 58089 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/vhost/vhost_net.ko 2023-11-26T07:02:48.890 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 89249 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/vhost/vhost_scsi.ko 2023-11-26T07:02:48.891 INFO:teuthology.orchestra.run.smithi143.stdout:drwxr-xr-x root/root 0 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/video/ 2023-11-26T07:02:48.891 INFO:teuthology.orchestra.run.smithi143.stdout:drwxr-xr-x root/root 0 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/video/backlight/ 2023-11-26T07:02:48.891 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 15417 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/video/backlight/88pm860x_bl.ko 2023-11-26T07:02:48.891 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 12393 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/video/backlight/aat2870_bl.ko 2023-11-26T07:02:48.891 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 23305 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/video/backlight/adp5520_bl.ko 2023-11-26T07:02:48.891 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 32609 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/video/backlight/adp8860_bl.ko 2023-11-26T07:02:48.891 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 39993 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/video/backlight/adp8870_bl.ko 2023-11-26T07:02:48.891 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 16921 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/video/backlight/apple_bl.ko 2023-11-26T07:02:48.891 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 16777 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/video/backlight/as3711_bl.ko 2023-11-26T07:02:48.891 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 12033 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/video/backlight/bd6107.ko 2023-11-26T07:02:48.891 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 11409 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/video/backlight/da903x_bl.ko 2023-11-26T07:02:48.892 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 11273 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/video/backlight/da9052_bl.ko 2023-11-26T07:02:48.892 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 11753 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/video/backlight/gpio_backlight.ko 2023-11-26T07:02:48.892 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 14769 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/video/backlight/kb3886_bl.ko 2023-11-26T07:02:48.892 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 22017 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/video/backlight/lcd.ko 2023-11-26T07:02:48.892 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 23657 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/video/backlight/lm3533_bl.ko 2023-11-26T07:02:48.892 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 19337 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/video/backlight/lm3639_bl.ko 2023-11-26T07:02:48.892 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 11425 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/video/backlight/lv5207lp.ko 2023-11-26T07:02:48.892 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 14345 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/video/backlight/max8925_bl.ko 2023-11-26T07:02:48.892 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 11313 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/video/backlight/pandora_bl.ko 2023-11-26T07:02:48.892 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 10865 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/video/backlight/pcf50633-backlight.ko 2023-11-26T07:02:48.892 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 10897 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/video/backlight/platform_lcd.ko 2023-11-26T07:02:48.892 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 12721 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/video/backlight/wm831x_bl.ko 2023-11-26T07:02:48.892 INFO:teuthology.orchestra.run.smithi143.stdout:drwxr-xr-x root/root 0 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/w1/ 2023-11-26T07:02:48.893 INFO:teuthology.orchestra.run.smithi143.stdout:drwxr-xr-x root/root 0 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/w1/masters/ 2023-11-26T07:02:48.893 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 18785 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/w1/masters/ds2482.ko 2023-11-26T07:02:48.893 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 38985 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/w1/masters/ds2490.ko 2023-11-26T07:02:48.893 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 11265 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/w1/masters/matrox_w1.ko 2023-11-26T07:02:48.893 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 12417 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/w1/masters/w1-gpio.ko 2023-11-26T07:02:48.893 INFO:teuthology.orchestra.run.smithi143.stdout:drwxr-xr-x root/root 0 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/w1/slaves/ 2023-11-26T07:02:48.893 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 12041 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/w1/slaves/w1_ds2406.ko 2023-11-26T07:02:48.893 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 22745 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/w1/slaves/w1_ds2408.ko 2023-11-26T07:02:48.893 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 15113 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/w1/slaves/w1_ds2413.ko 2023-11-26T07:02:48.893 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 10825 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/w1/slaves/w1_ds2423.ko 2023-11-26T07:02:48.893 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 14153 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/w1/slaves/w1_ds2431.ko 2023-11-26T07:02:48.893 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 11929 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/w1/slaves/w1_ds2433.ko 2023-11-26T07:02:48.894 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 14049 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/w1/slaves/w1_ds2780.ko 2023-11-26T07:02:48.894 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 14049 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/w1/slaves/w1_ds2781.ko 2023-11-26T07:02:48.894 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 17921 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/w1/slaves/w1_ds28e04.ko 2023-11-26T07:02:48.894 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 7385 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/w1/slaves/w1_smem.ko 2023-11-26T07:02:48.894 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 56425 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/w1/slaves/w1_therm.ko 2023-11-26T07:02:48.894 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 102673 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/w1/wire.ko 2023-11-26T07:02:48.894 INFO:teuthology.orchestra.run.smithi143.stdout:drwxr-xr-x root/root 0 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/watchdog/ 2023-11-26T07:02:48.894 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 18793 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/watchdog/acquirewdt.ko 2023-11-26T07:02:48.894 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 19977 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/watchdog/advantechwdt.ko 2023-11-26T07:02:48.894 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 18561 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/watchdog/alim1535_wdt.ko 2023-11-26T07:02:48.894 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 22969 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/watchdog/alim7101_wdt.ko 2023-11-26T07:02:48.894 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 19113 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/watchdog/cpu5wdt.ko 2023-11-26T07:02:48.895 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 13617 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/watchdog/da9052_wdt.ko 2023-11-26T07:02:48.895 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 12777 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/watchdog/da9055_wdt.ko 2023-11-26T07:02:48.895 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 17409 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/watchdog/da9063_wdt.ko 2023-11-26T07:02:48.895 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 20169 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/watchdog/dw_wdt.ko 2023-11-26T07:02:48.895 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 20265 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/watchdog/eurotechwdt.ko 2023-11-26T07:02:48.895 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 23865 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/watchdog/f71808e_wdt.ko 2023-11-26T07:02:48.895 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 27401 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/watchdog/hpwdt.ko 2023-11-26T07:02:48.895 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 15577 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/watchdog/i6300esb.ko 2023-11-26T07:02:48.895 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 10833 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/watchdog/iTCO_vendor_support.ko 2023-11-26T07:02:48.895 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 21913 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/watchdog/iTCO_wdt.ko 2023-11-26T07:02:48.895 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 19241 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/watchdog/ib700wdt.ko 2023-11-26T07:02:48.895 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 18689 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/watchdog/ibmasr.ko 2023-11-26T07:02:48.896 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 19713 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/watchdog/ie6xx_wdt.ko 2023-11-26T07:02:48.896 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 21225 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/watchdog/it8712f_wdt.ko 2023-11-26T07:02:48.896 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 13841 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/watchdog/it87_wdt.ko 2023-11-26T07:02:48.896 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 21489 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/watchdog/kempld_wdt.ko 2023-11-26T07:02:48.896 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 18617 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/watchdog/machzwd.ko 2023-11-26T07:02:48.896 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 14721 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/watchdog/mena21_wdt.ko 2023-11-26T07:02:48.896 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 13217 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/watchdog/menf21bmc_wdt.ko 2023-11-26T07:02:48.896 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 24673 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/watchdog/nv_tco.ko 2023-11-26T07:02:48.896 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 17737 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/watchdog/of_xilinx_wdt.ko 2023-11-26T07:02:48.896 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 22265 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/watchdog/pc87413_wdt.ko 2023-11-26T07:02:48.896 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 38657 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/watchdog/pcwd_pci.ko 2023-11-26T07:02:48.896 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 31825 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/watchdog/pcwd_usb.ko 2023-11-26T07:02:48.897 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 13065 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/watchdog/retu_wdt.ko 2023-11-26T07:02:48.897 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 20449 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/watchdog/sbc60xxwdt.ko 2023-11-26T07:02:48.897 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 14129 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/watchdog/sbc_epx_c3.ko 2023-11-26T07:02:48.897 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 17321 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/watchdog/sbc_fitpc2_wdt.ko 2023-11-26T07:02:48.897 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 23809 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/watchdog/sc1200wdt.ko 2023-11-26T07:02:48.897 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 23321 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/watchdog/sch311x_wdt.ko 2023-11-26T07:02:48.897 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 21577 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/watchdog/smsc37b787_wdt.ko 2023-11-26T07:02:48.897 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 17273 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/watchdog/softdog.ko 2023-11-26T07:02:48.897 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 22273 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/watchdog/sp5100_tco.ko 2023-11-26T07:02:48.897 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 12817 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/watchdog/twl4030_wdt.ko 2023-11-26T07:02:48.897 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 17649 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/watchdog/via_wdt.ko 2023-11-26T07:02:48.897 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 19161 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/watchdog/w83627hf_wdt.ko 2023-11-26T07:02:48.897 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 20681 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/watchdog/w83877f_wdt.ko 2023-11-26T07:02:48.898 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 25409 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/watchdog/w83977f_wdt.ko 2023-11-26T07:02:48.898 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 18497 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/watchdog/wafer5823wdt.ko 2023-11-26T07:02:48.898 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 28089 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/watchdog/wdt_pci.ko 2023-11-26T07:02:48.898 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 16257 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/watchdog/wm831x_wdt.ko 2023-11-26T07:02:48.898 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 13785 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/watchdog/wm8350_wdt.ko 2023-11-26T07:02:48.898 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 15697 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/watchdog/xen_wdt.ko 2023-11-26T07:02:48.898 INFO:teuthology.orchestra.run.smithi143.stdout:drwxr-xr-x root/root 0 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/xen/ 2023-11-26T07:02:48.898 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 24729 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/xen/xen-evtchn.ko 2023-11-26T07:02:48.898 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 26769 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/xen/xen-gntalloc.ko 2023-11-26T07:02:48.898 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 47865 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/xen/xen-gntdev.ko 2023-11-26T07:02:48.899 INFO:teuthology.orchestra.run.smithi143.stdout:drwxr-xr-x root/root 0 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/xen/xen-pciback/ 2023-11-26T07:02:48.899 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 176377 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/xen/xen-pciback/xen-pciback.ko 2023-11-26T07:02:48.899 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 40121 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/xen/xen-privcmd.ko 2023-11-26T07:02:48.899 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 71249 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/xen/xen-scsiback.ko 2023-11-26T07:02:48.899 INFO:teuthology.orchestra.run.smithi143.stdout:drwxr-xr-x root/root 0 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/xen/xenfs/ 2023-11-26T07:02:48.899 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 19425 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/xen/xenfs/xenfs.ko 2023-11-26T07:02:48.899 INFO:teuthology.orchestra.run.smithi143.stdout:drwxr-xr-x root/root 0 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/fs/ 2023-11-26T07:02:48.899 INFO:teuthology.orchestra.run.smithi143.stdout:drwxr-xr-x root/root 0 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/fs/9p/ 2023-11-26T07:02:48.899 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 166425 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/fs/9p/9p.ko 2023-11-26T07:02:48.899 INFO:teuthology.orchestra.run.smithi143.stdout:drwxr-xr-x root/root 0 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/fs/adfs/ 2023-11-26T07:02:48.899 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 61073 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/fs/adfs/adfs.ko 2023-11-26T07:02:48.899 INFO:teuthology.orchestra.run.smithi143.stdout:drwxr-xr-x root/root 0 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/fs/affs/ 2023-11-26T07:02:48.900 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 163593 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/fs/affs/affs.ko 2023-11-26T07:02:48.900 INFO:teuthology.orchestra.run.smithi143.stdout:drwxr-xr-x root/root 0 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/fs/afs/ 2023-11-26T07:02:48.900 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 932817 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/fs/afs/kafs.ko 2023-11-26T07:02:48.900 INFO:teuthology.orchestra.run.smithi143.stdout:drwxr-xr-x root/root 0 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/fs/autofs/ 2023-11-26T07:02:48.900 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 101057 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/fs/autofs/autofs4.ko 2023-11-26T07:02:48.900 INFO:teuthology.orchestra.run.smithi143.stdout:drwxr-xr-x root/root 0 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/fs/befs/ 2023-11-26T07:02:48.900 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 64545 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/fs/befs/befs.ko 2023-11-26T07:02:48.900 INFO:teuthology.orchestra.run.smithi143.stdout:drwxr-xr-x root/root 0 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/fs/bfs/ 2023-11-26T07:02:48.900 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 41257 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/fs/bfs/bfs.ko 2023-11-26T07:02:48.900 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 34585 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/fs/binfmt_misc.ko 2023-11-26T07:02:48.900 INFO:teuthology.orchestra.run.smithi143.stdout:drwxr-xr-x root/root 0 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/fs/btrfs/ 2023-11-26T07:02:48.900 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 3402729 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/fs/btrfs/btrfs.ko 2023-11-26T07:02:48.901 INFO:teuthology.orchestra.run.smithi143.stdout:drwxr-xr-x root/root 0 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/fs/cachefiles/ 2023-11-26T07:02:48.901 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 306009 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/fs/cachefiles/cachefiles.ko 2023-11-26T07:02:48.901 INFO:teuthology.orchestra.run.smithi143.stdout:drwxr-xr-x root/root 0 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/fs/ceph/ 2023-11-26T07:02:48.901 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 1333017 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/fs/ceph/ceph.ko 2023-11-26T07:02:48.901 INFO:teuthology.orchestra.run.smithi143.stdout:drwxr-xr-x root/root 0 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/fs/coda/ 2023-11-26T07:02:48.901 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 100801 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/fs/coda/coda.ko 2023-11-26T07:02:48.901 INFO:teuthology.orchestra.run.smithi143.stdout:drwxr-xr-x root/root 0 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/fs/configfs/ 2023-11-26T07:02:48.901 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 108393 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/fs/configfs/configfs.ko 2023-11-26T07:02:48.901 INFO:teuthology.orchestra.run.smithi143.stdout:drwxr-xr-x root/root 0 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/fs/cramfs/ 2023-11-26T07:02:48.901 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 29625 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/fs/cramfs/cramfs.ko 2023-11-26T07:02:48.901 INFO:teuthology.orchestra.run.smithi143.stdout:drwxr-xr-x root/root 0 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/fs/dlm/ 2023-11-26T07:02:48.901 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 642513 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/fs/dlm/dlm.ko 2023-11-26T07:02:48.902 INFO:teuthology.orchestra.run.smithi143.stdout:drwxr-xr-x root/root 0 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/fs/efs/ 2023-11-26T07:02:48.902 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 36649 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/fs/efs/efs.ko 2023-11-26T07:02:48.902 INFO:teuthology.orchestra.run.smithi143.stdout:drwxr-xr-x root/root 0 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/fs/f2fs/ 2023-11-26T07:02:48.902 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 1505025 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/fs/f2fs/f2fs.ko 2023-11-26T07:02:48.902 INFO:teuthology.orchestra.run.smithi143.stdout:drwxr-xr-x root/root 0 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/fs/fat/ 2023-11-26T07:02:48.902 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 27929 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/fs/fat/msdos.ko 2023-11-26T07:02:48.902 INFO:teuthology.orchestra.run.smithi143.stdout:drwxr-xr-x root/root 0 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/fs/freevxfs/ 2023-11-26T07:02:48.902 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 42545 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/fs/freevxfs/freevxfs.ko 2023-11-26T07:02:48.902 INFO:teuthology.orchestra.run.smithi143.stdout:drwxr-xr-x root/root 0 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/fs/fscache/ 2023-11-26T07:02:48.902 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 226481 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/fs/fscache/fscache.ko 2023-11-26T07:02:48.902 INFO:teuthology.orchestra.run.smithi143.stdout:drwxr-xr-x root/root 0 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/fs/fuse/ 2023-11-26T07:02:48.902 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 24489 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/fs/fuse/cuse.ko 2023-11-26T07:02:48.903 INFO:teuthology.orchestra.run.smithi143.stdout:drwxr-xr-x root/root 0 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/fs/gfs2/ 2023-11-26T07:02:48.903 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 1231529 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/fs/gfs2/gfs2.ko 2023-11-26T07:02:48.903 INFO:teuthology.orchestra.run.smithi143.stdout:drwxr-xr-x root/root 0 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/fs/hfs/ 2023-11-26T07:02:48.903 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 141209 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/fs/hfs/hfs.ko 2023-11-26T07:02:48.903 INFO:teuthology.orchestra.run.smithi143.stdout:drwxr-xr-x root/root 0 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/fs/hfsplus/ 2023-11-26T07:02:48.903 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 222817 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/fs/hfsplus/hfsplus.ko 2023-11-26T07:02:48.903 INFO:teuthology.orchestra.run.smithi143.stdout:drwxr-xr-x root/root 0 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/fs/hpfs/ 2023-11-26T07:02:48.903 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 192329 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/fs/hpfs/hpfs.ko 2023-11-26T07:02:48.903 INFO:teuthology.orchestra.run.smithi143.stdout:drwxr-xr-x root/root 0 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/fs/isofs/ 2023-11-26T07:02:48.903 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 74065 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/fs/isofs/isofs.ko 2023-11-26T07:02:48.903 INFO:teuthology.orchestra.run.smithi143.stdout:drwxr-xr-x root/root 0 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/fs/jfs/ 2023-11-26T07:02:48.903 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 419497 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/fs/jfs/jfs.ko 2023-11-26T07:02:48.904 INFO:teuthology.orchestra.run.smithi143.stdout:drwxr-xr-x root/root 0 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/fs/lockd/ 2023-11-26T07:02:48.904 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 280313 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/fs/lockd/lockd.ko 2023-11-26T07:02:48.904 INFO:teuthology.orchestra.run.smithi143.stdout:drwxr-xr-x root/root 0 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/fs/minix/ 2023-11-26T07:02:48.904 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 89193 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/fs/minix/minix.ko 2023-11-26T07:02:48.904 INFO:teuthology.orchestra.run.smithi143.stdout:drwxr-xr-x root/root 0 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/fs/netfs/ 2023-11-26T07:02:48.904 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 124649 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/fs/netfs/netfs.ko 2023-11-26T07:02:48.904 INFO:teuthology.orchestra.run.smithi143.stdout:drwxr-xr-x root/root 0 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/fs/nfs/ 2023-11-26T07:02:48.904 INFO:teuthology.orchestra.run.smithi143.stdout:drwxr-xr-x root/root 0 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/fs/nfs/blocklayout/ 2023-11-26T07:02:48.904 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 81449 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/fs/nfs/blocklayout/blocklayoutdriver.ko 2023-11-26T07:02:48.904 INFO:teuthology.orchestra.run.smithi143.stdout:drwxr-xr-x root/root 0 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/fs/nfs/filelayout/ 2023-11-26T07:02:48.904 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 59449 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/fs/nfs/filelayout/nfs_layout_nfsv41_files.ko 2023-11-26T07:02:48.904 INFO:teuthology.orchestra.run.smithi143.stdout:drwxr-xr-x root/root 0 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/fs/nfs/flexfilelayout/ 2023-11-26T07:02:48.905 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 117465 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/fs/nfs/flexfilelayout/nfs_layout_flexfiles.ko 2023-11-26T07:02:48.905 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 1055721 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/fs/nfs/nfs.ko 2023-11-26T07:02:48.905 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 66441 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/fs/nfs/nfsv2.ko 2023-11-26T07:02:48.905 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 120177 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/fs/nfs/nfsv3.ko 2023-11-26T07:02:48.905 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 2042137 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/fs/nfs/nfsv4.ko 2023-11-26T07:02:48.905 INFO:teuthology.orchestra.run.smithi143.stdout:drwxr-xr-x root/root 0 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/fs/nfs_common/ 2023-11-26T07:02:48.905 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 16929 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/fs/nfs_common/grace.ko 2023-11-26T07:02:48.905 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 16145 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/fs/nfs_common/nfs_acl.ko 2023-11-26T07:02:48.905 INFO:teuthology.orchestra.run.smithi143.stdout:drwxr-xr-x root/root 0 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/fs/nfsd/ 2023-11-26T07:02:48.905 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 1516873 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/fs/nfsd/nfsd.ko 2023-11-26T07:02:48.905 INFO:teuthology.orchestra.run.smithi143.stdout:drwxr-xr-x root/root 0 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/fs/nilfs2/ 2023-11-26T07:02:48.905 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 592833 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/fs/nilfs2/nilfs2.ko 2023-11-26T07:02:48.906 INFO:teuthology.orchestra.run.smithi143.stdout:drwxr-xr-x root/root 0 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/fs/nls/ 2023-11-26T07:02:48.906 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 13793 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/fs/nls/mac-celtic.ko 2023-11-26T07:02:48.906 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 13169 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/fs/nls/mac-centeuro.ko 2023-11-26T07:02:48.906 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 13793 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/fs/nls/mac-croatian.ko 2023-11-26T07:02:48.906 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 12865 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/fs/nls/mac-cyrillic.ko 2023-11-26T07:02:48.906 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 13481 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/fs/nls/mac-gaelic.ko 2023-11-26T07:02:48.906 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 12857 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/fs/nls/mac-greek.ko 2023-11-26T07:02:48.906 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 13793 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/fs/nls/mac-iceland.ko 2023-11-26T07:02:48.906 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 13169 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/fs/nls/mac-inuit.ko 2023-11-26T07:02:48.906 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 14097 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/fs/nls/mac-roman.ko 2023-11-26T07:02:48.906 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 13793 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/fs/nls/mac-romanian.ko 2023-11-26T07:02:48.906 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 13793 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/fs/nls/mac-turkish.ko 2023-11-26T07:02:48.906 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 11297 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/fs/nls/nls_ascii.ko 2023-11-26T07:02:48.907 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 12537 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/fs/nls/nls_cp1250.ko 2023-11-26T07:02:48.907 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 12233 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/fs/nls/nls_cp1251.ko 2023-11-26T07:02:48.907 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 12921 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/fs/nls/nls_cp1255.ko 2023-11-26T07:02:48.907 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 12537 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/fs/nls/nls_cp737.ko 2023-11-26T07:02:48.907 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 12537 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/fs/nls/nls_cp775.ko 2023-11-26T07:02:48.907 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 12225 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/fs/nls/nls_cp850.ko 2023-11-26T07:02:48.907 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 12225 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/fs/nls/nls_cp852.ko 2023-11-26T07:02:48.907 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 12225 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/fs/nls/nls_cp855.ko 2023-11-26T07:02:48.907 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 11913 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/fs/nls/nls_cp857.ko 2023-11-26T07:02:48.907 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 12849 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/fs/nls/nls_cp860.ko 2023-11-26T07:02:48.907 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 13161 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/fs/nls/nls_cp861.ko 2023-11-26T07:02:48.907 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 13473 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/fs/nls/nls_cp862.ko 2023-11-26T07:02:48.908 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 13161 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/fs/nls/nls_cp863.ko 2023-11-26T07:02:48.908 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 12849 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/fs/nls/nls_cp864.ko 2023-11-26T07:02:48.908 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 13161 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/fs/nls/nls_cp865.ko 2023-11-26T07:02:48.908 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 12537 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/fs/nls/nls_cp866.ko 2023-11-26T07:02:48.908 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 12225 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/fs/nls/nls_cp869.ko 2023-11-26T07:02:48.908 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 11985 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/fs/nls/nls_cp874.ko 2023-11-26T07:02:48.908 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 94137 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/fs/nls/nls_cp932.ko 2023-11-26T07:02:48.908 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 142673 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/fs/nls/nls_cp936.ko 2023-11-26T07:02:48.908 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 165745 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/fs/nls/nls_cp949.ko 2023-11-26T07:02:48.908 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 118121 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/fs/nls/nls_cp950.ko 2023-11-26T07:02:48.908 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 12457 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/fs/nls/nls_euc-jp.ko 2023-11-26T07:02:48.908 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 11305 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/fs/nls/nls_iso8859-1.ko 2023-11-26T07:02:48.909 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 11929 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/fs/nls/nls_iso8859-13.ko 2023-11-26T07:02:48.909 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 11929 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/fs/nls/nls_iso8859-14.ko 2023-11-26T07:02:48.909 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 11929 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/fs/nls/nls_iso8859-15.ko 2023-11-26T07:02:48.909 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 11921 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/fs/nls/nls_iso8859-2.ko 2023-11-26T07:02:48.909 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 11921 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/fs/nls/nls_iso8859-3.ko 2023-11-26T07:02:48.909 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 11921 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/fs/nls/nls_iso8859-4.ko 2023-11-26T07:02:48.909 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 11921 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/fs/nls/nls_iso8859-5.ko 2023-11-26T07:02:48.909 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 11617 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/fs/nls/nls_iso8859-6.ko 2023-11-26T07:02:48.909 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 12233 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/fs/nls/nls_iso8859-7.ko 2023-11-26T07:02:48.909 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 11617 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/fs/nls/nls_iso8859-9.ko 2023-11-26T07:02:48.909 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 12537 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/fs/nls/nls_koi8-r.ko 2023-11-26T07:02:48.909 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 8017 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/fs/nls/nls_koi8-ru.ko 2023-11-26T07:02:48.909 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 12537 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/fs/nls/nls_koi8-u.ko 2023-11-26T07:02:48.910 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 7569 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/fs/nls/nls_ucs2_utils.ko 2023-11-26T07:02:48.910 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 8337 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/fs/nls/nls_utf8.ko 2023-11-26T07:02:48.910 INFO:teuthology.orchestra.run.smithi143.stdout:drwxr-xr-x root/root 0 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/fs/ntfs/ 2023-11-26T07:02:48.910 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 219417 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/fs/ntfs/ntfs.ko 2023-11-26T07:02:48.910 INFO:teuthology.orchestra.run.smithi143.stdout:drwxr-xr-x root/root 0 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/fs/ocfs2/ 2023-11-26T07:02:48.910 INFO:teuthology.orchestra.run.smithi143.stdout:drwxr-xr-x root/root 0 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/fs/ocfs2/cluster/ 2023-11-26T07:02:48.910 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 244409 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/fs/ocfs2/cluster/ocfs2_nodemanager.ko 2023-11-26T07:02:48.910 INFO:teuthology.orchestra.run.smithi143.stdout:drwxr-xr-x root/root 0 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/fs/ocfs2/dlm/ 2023-11-26T07:02:48.910 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 440737 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/fs/ocfs2/dlm/ocfs2_dlm.ko 2023-11-26T07:02:48.910 INFO:teuthology.orchestra.run.smithi143.stdout:drwxr-xr-x root/root 0 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/fs/ocfs2/dlmfs/ 2023-11-26T07:02:48.910 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 54049 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/fs/ocfs2/dlmfs/ocfs2_dlmfs.ko 2023-11-26T07:02:48.910 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 2592433 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/fs/ocfs2/ocfs2.ko 2023-11-26T07:02:48.911 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 17905 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/fs/ocfs2/ocfs2_stack_o2cb.ko 2023-11-26T07:02:48.911 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 31729 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/fs/ocfs2/ocfs2_stack_user.ko 2023-11-26T07:02:48.911 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 35897 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/fs/ocfs2/ocfs2_stackglue.ko 2023-11-26T07:02:48.911 INFO:teuthology.orchestra.run.smithi143.stdout:drwxr-xr-x root/root 0 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/fs/omfs/ 2023-11-26T07:02:48.911 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 53433 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/fs/omfs/omfs.ko 2023-11-26T07:02:48.911 INFO:teuthology.orchestra.run.smithi143.stdout:drwxr-xr-x root/root 0 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/fs/overlayfs/ 2023-11-26T07:02:48.911 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 352161 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/fs/overlayfs/overlay.ko 2023-11-26T07:02:48.911 INFO:teuthology.orchestra.run.smithi143.stdout:drwxr-xr-x root/root 0 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/fs/pstore/ 2023-11-26T07:02:48.911 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 54809 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/fs/pstore/ramoops.ko 2023-11-26T07:02:48.911 INFO:teuthology.orchestra.run.smithi143.stdout:drwxr-xr-x root/root 0 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/fs/qnx4/ 2023-11-26T07:02:48.911 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 26865 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/fs/qnx4/qnx4.ko 2023-11-26T07:02:48.911 INFO:teuthology.orchestra.run.smithi143.stdout:drwxr-xr-x root/root 0 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/fs/qnx6/ 2023-11-26T07:02:48.912 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 46689 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/fs/qnx6/qnx6.ko 2023-11-26T07:02:48.912 INFO:teuthology.orchestra.run.smithi143.stdout:drwxr-xr-x root/root 0 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/fs/quota/ 2023-11-26T07:02:48.912 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 30881 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/fs/quota/quota_tree.ko 2023-11-26T07:02:48.912 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 14297 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/fs/quota/quota_v1.ko 2023-11-26T07:02:48.912 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 22017 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/fs/quota/quota_v2.ko 2023-11-26T07:02:48.912 INFO:teuthology.orchestra.run.smithi143.stdout:drwxr-xr-x root/root 0 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/fs/reiserfs/ 2023-11-26T07:02:48.912 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 532673 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/fs/reiserfs/reiserfs.ko 2023-11-26T07:02:48.912 INFO:teuthology.orchestra.run.smithi143.stdout:drwxr-xr-x root/root 0 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/fs/romfs/ 2023-11-26T07:02:48.912 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 28641 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/fs/romfs/romfs.ko 2023-11-26T07:02:48.912 INFO:teuthology.orchestra.run.smithi143.stdout:drwxr-xr-x root/root 0 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/fs/smb/ 2023-11-26T07:02:48.912 INFO:teuthology.orchestra.run.smithi143.stdout:drwxr-xr-x root/root 0 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/fs/smb/client/ 2023-11-26T07:02:48.913 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 2668401 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/fs/smb/client/cifs.ko 2023-11-26T07:02:48.913 INFO:teuthology.orchestra.run.smithi143.stdout:drwxr-xr-x root/root 0 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/fs/smb/common/ 2023-11-26T07:02:48.913 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 7161 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/fs/smb/common/cifs_arc4.ko 2023-11-26T07:02:48.913 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 11257 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/fs/smb/common/cifs_md4.ko 2023-11-26T07:02:48.913 INFO:teuthology.orchestra.run.smithi143.stdout:drwxr-xr-x root/root 0 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/fs/squashfs/ 2023-11-26T07:02:48.913 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 134641 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/fs/squashfs/squashfs.ko 2023-11-26T07:02:48.913 INFO:teuthology.orchestra.run.smithi143.stdout:drwxr-xr-x root/root 0 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/fs/sysv/ 2023-11-26T07:02:48.913 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 90721 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/fs/sysv/sysv.ko 2023-11-26T07:02:48.913 INFO:teuthology.orchestra.run.smithi143.stdout:drwxr-xr-x root/root 0 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/fs/udf/ 2023-11-26T07:02:48.913 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 247737 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/fs/udf/udf.ko 2023-11-26T07:02:48.913 INFO:teuthology.orchestra.run.smithi143.stdout:drwxr-xr-x root/root 0 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/fs/ufs/ 2023-11-26T07:02:48.913 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 162497 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/fs/ufs/ufs.ko 2023-11-26T07:02:48.914 INFO:teuthology.orchestra.run.smithi143.stdout:drwxr-xr-x root/root 0 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/fs/xfs/ 2023-11-26T07:02:48.914 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 4011017 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/fs/xfs/xfs.ko 2023-11-26T07:02:48.914 INFO:teuthology.orchestra.run.smithi143.stdout:drwxr-xr-x root/root 0 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/kernel/ 2023-11-26T07:02:48.914 INFO:teuthology.orchestra.run.smithi143.stdout:drwxr-xr-x root/root 0 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/kernel/locking/ 2023-11-26T07:02:48.914 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 75489 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/kernel/locking/locktorture.ko 2023-11-26T07:02:48.914 INFO:teuthology.orchestra.run.smithi143.stdout:drwxr-xr-x root/root 0 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/kernel/time/ 2023-11-26T07:02:48.914 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 13737 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/kernel/time/test_udelay.ko 2023-11-26T07:02:48.914 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 60553 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/kernel/torture.ko 2023-11-26T07:02:48.914 INFO:teuthology.orchestra.run.smithi143.stdout:drwxr-xr-x root/root 0 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/lib/ 2023-11-26T07:02:48.914 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 6833 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/lib/crc-itu-t.ko 2023-11-26T07:02:48.914 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 6529 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/lib/crc7.ko 2023-11-26T07:02:48.914 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 7025 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/lib/crc8.ko 2023-11-26T07:02:48.915 INFO:teuthology.orchestra.run.smithi143.stdout:drwxr-xr-x root/root 0 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/lib/crypto/ 2023-11-26T07:02:48.915 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 7097 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/lib/crypto/libarc4.ko 2023-11-26T07:02:48.915 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 37081 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/lib/crypto/libdes.ko 2023-11-26T07:02:48.915 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 13313 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/lib/interval_tree_test.ko 2023-11-26T07:02:48.915 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 8425 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/lib/libcrc32c.ko 2023-11-26T07:02:48.915 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 24953 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/lib/lru_cache.ko 2023-11-26T07:02:48.915 INFO:teuthology.orchestra.run.smithi143.stdout:drwxr-xr-x root/root 0 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/lib/lz4/ 2023-11-26T07:02:48.915 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 33801 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/lib/lz4/lz4_compress.ko 2023-11-26T07:02:48.915 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 22633 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/lib/lz4/lz4hc_compress.ko 2023-11-26T07:02:48.915 INFO:teuthology.orchestra.run.smithi143.stdout:drwxr-xr-x root/root 0 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/lib/math/ 2023-11-26T07:02:48.915 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 6617 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/lib/math/cordic.ko 2023-11-26T07:02:48.916 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 8233 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/lib/memory-notifier-error-inject.ko 2023-11-26T07:02:48.916 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 10865 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/lib/notifier-error-inject.ko 2023-11-26T07:02:48.916 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 14089 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/lib/percpu_test.ko 2023-11-26T07:02:48.916 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 8305 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/lib/pm-notifier-error-inject.ko 2023-11-26T07:02:48.916 INFO:teuthology.orchestra.run.smithi143.stdout:drwxr-xr-x root/root 0 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/lib/raid6/ 2023-11-26T07:02:48.916 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 143433 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/lib/raid6/raid6_pq.ko 2023-11-26T07:02:48.916 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 21105 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/lib/rbtree_test.ko 2023-11-26T07:02:48.916 INFO:teuthology.orchestra.run.smithi143.stdout:drwxr-xr-x root/root 0 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/lib/reed_solomon/ 2023-11-26T07:02:48.916 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 18753 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/lib/reed_solomon/reed_solomon.ko 2023-11-26T07:02:48.916 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 38841 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/lib/test-kstrtox.ko 2023-11-26T07:02:48.916 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 26321 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/lib/test-string_helpers.ko 2023-11-26T07:02:48.916 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 4760425 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/lib/test_bpf.ko 2023-11-26T07:02:48.917 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 49089 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/lib/test_firmware.ko 2023-11-26T07:02:48.917 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 6369 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/lib/test_module.ko 2023-11-26T07:02:48.917 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 20801 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/lib/test_user_copy.ko 2023-11-26T07:02:48.917 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 9585 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/lib/ts_bm.ko 2023-11-26T07:02:48.917 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 11121 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/lib/ts_fsm.ko 2023-11-26T07:02:48.917 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 9625 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/lib/ts_kmp.ko 2023-11-26T07:02:48.917 INFO:teuthology.orchestra.run.smithi143.stdout:drwxr-xr-x root/root 0 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/lib/xz/ 2023-11-26T07:02:48.917 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 13545 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/lib/xz/xz_dec_test.ko 2023-11-26T07:02:48.917 INFO:teuthology.orchestra.run.smithi143.stdout:drwxr-xr-x root/root 0 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/mm/ 2023-11-26T07:02:48.917 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 15425 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/mm/hwpoison-inject.ko 2023-11-26T07:02:48.917 INFO:teuthology.orchestra.run.smithi143.stdout:drwxr-xr-x root/root 0 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/net/ 2023-11-26T07:02:48.917 INFO:teuthology.orchestra.run.smithi143.stdout:drwxr-xr-x root/root 0 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/net/6lowpan/ 2023-11-26T07:02:48.918 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 55737 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/net/6lowpan/6lowpan.ko 2023-11-26T07:02:48.918 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 7049 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/net/6lowpan/nhc_dest.ko 2023-11-26T07:02:48.918 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 7065 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/net/6lowpan/nhc_fragment.ko 2023-11-26T07:02:48.918 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 7049 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/net/6lowpan/nhc_hop.ko 2023-11-26T07:02:48.918 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 6985 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/net/6lowpan/nhc_ipv6.ko 2023-11-26T07:02:48.918 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 7065 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/net/6lowpan/nhc_mobility.ko 2023-11-26T07:02:48.918 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 7001 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/net/6lowpan/nhc_routing.ko 2023-11-26T07:02:48.918 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 15329 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/net/6lowpan/nhc_udp.ko 2023-11-26T07:02:48.918 INFO:teuthology.orchestra.run.smithi143.stdout:drwxr-xr-x root/root 0 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/net/802/ 2023-11-26T07:02:48.918 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 30385 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/net/802/garp.ko 2023-11-26T07:02:48.918 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 34809 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/net/802/mrp.ko 2023-11-26T07:02:48.918 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 8545 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/net/802/p8022.ko 2023-11-26T07:02:48.919 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 16345 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/net/802/psnap.ko 2023-11-26T07:02:48.919 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 11537 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/net/802/stp.ko 2023-11-26T07:02:48.919 INFO:teuthology.orchestra.run.smithi143.stdout:drwxr-xr-x root/root 0 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/net/8021q/ 2023-11-26T07:02:48.919 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 81737 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/net/8021q/8021q.ko 2023-11-26T07:02:48.919 INFO:teuthology.orchestra.run.smithi143.stdout:drwxr-xr-x root/root 0 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/net/9p/ 2023-11-26T07:02:48.919 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 198337 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/net/9p/9pnet.ko 2023-11-26T07:02:48.919 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 35977 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/net/9p/9pnet_fd.ko 2023-11-26T07:02:48.919 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 26345 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/net/9p/9pnet_rdma.ko 2023-11-26T07:02:48.919 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 32337 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/net/9p/9pnet_virtio.ko 2023-11-26T07:02:48.919 INFO:teuthology.orchestra.run.smithi143.stdout:drwxr-xr-x root/root 0 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/net/batman-adv/ 2023-11-26T07:02:48.919 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 649297 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/net/batman-adv/batman-adv.ko 2023-11-26T07:02:48.919 INFO:teuthology.orchestra.run.smithi143.stdout:drwxr-xr-x root/root 0 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/net/bridge/ 2023-11-26T07:02:48.920 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 63809 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/net/bridge/br_netfilter.ko 2023-11-26T07:02:48.920 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 777689 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/net/bridge/bridge.ko 2023-11-26T07:02:48.920 INFO:teuthology.orchestra.run.smithi143.stdout:drwxr-xr-x root/root 0 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/net/bridge/netfilter/ 2023-11-26T07:02:48.920 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 7657 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/net/bridge/netfilter/ebt_802_3.ko 2023-11-26T07:02:48.920 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 12385 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/net/bridge/netfilter/ebt_among.ko 2023-11-26T07:02:48.920 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 9169 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/net/bridge/netfilter/ebt_arp.ko 2023-11-26T07:02:48.920 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 9073 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/net/bridge/netfilter/ebt_arpreply.ko 2023-11-26T07:02:48.920 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 9625 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/net/bridge/netfilter/ebt_dnat.ko 2023-11-26T07:02:48.920 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 8737 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/net/bridge/netfilter/ebt_ip.ko 2023-11-26T07:02:48.920 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 9145 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/net/bridge/netfilter/ebt_ip6.ko 2023-11-26T07:02:48.920 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 9825 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/net/bridge/netfilter/ebt_limit.ko 2023-11-26T07:02:48.920 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 13217 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/net/bridge/netfilter/ebt_log.ko 2023-11-26T07:02:48.921 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 7585 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/net/bridge/netfilter/ebt_mark.ko 2023-11-26T07:02:48.921 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 7593 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/net/bridge/netfilter/ebt_mark_m.ko 2023-11-26T07:02:48.921 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 8385 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/net/bridge/netfilter/ebt_nflog.ko 2023-11-26T07:02:48.921 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 7537 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/net/bridge/netfilter/ebt_pkttype.ko 2023-11-26T07:02:48.921 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 9593 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/net/bridge/netfilter/ebt_redirect.ko 2023-11-26T07:02:48.921 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 8649 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/net/bridge/netfilter/ebt_snat.ko 2023-11-26T07:02:48.921 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 8825 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/net/bridge/netfilter/ebt_stp.ko 2023-11-26T07:02:48.921 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 12401 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/net/bridge/netfilter/ebt_vlan.ko 2023-11-26T07:02:48.921 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 11529 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/net/bridge/netfilter/ebtable_broute.ko 2023-11-26T07:02:48.921 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 9817 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/net/bridge/netfilter/ebtable_filter.ko 2023-11-26T07:02:48.921 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 9793 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/net/bridge/netfilter/ebtable_nat.ko 2023-11-26T07:02:48.921 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 55929 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/net/bridge/netfilter/ebtables.ko 2023-11-26T07:02:48.922 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 14041 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/net/bridge/netfilter/nft_meta_bridge.ko 2023-11-26T07:02:48.922 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 13945 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/net/bridge/netfilter/nft_reject_bridge.ko 2023-11-26T07:02:48.922 INFO:teuthology.orchestra.run.smithi143.stdout:drwxr-xr-x root/root 0 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/net/ceph/ 2023-11-26T07:02:48.922 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 1046833 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/net/ceph/libceph.ko 2023-11-26T07:02:48.922 INFO:teuthology.orchestra.run.smithi143.stdout:drwxr-xr-x root/root 0 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/net/core/ 2023-11-26T07:02:48.922 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 125993 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/net/core/pktgen.ko 2023-11-26T07:02:48.922 INFO:teuthology.orchestra.run.smithi143.stdout:drwxr-xr-x root/root 0 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/net/dccp/ 2023-11-26T07:02:48.922 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 194209 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/net/dccp/dccp.ko 2023-11-26T07:02:48.922 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 8665 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/net/dccp/dccp_diag.ko 2023-11-26T07:02:48.922 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 57881 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/net/dccp/dccp_ipv4.ko 2023-11-26T07:02:48.922 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 60337 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/net/dccp/dccp_ipv6.ko 2023-11-26T07:02:48.922 INFO:teuthology.orchestra.run.smithi143.stdout:drwxr-xr-x root/root 0 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/net/dsa/ 2023-11-26T07:02:48.923 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 390473 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/net/dsa/dsa_core.ko 2023-11-26T07:02:48.923 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 13337 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/net/dsa/tag_brcm.ko 2023-11-26T07:02:48.923 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 12577 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/net/dsa/tag_dsa.ko 2023-11-26T07:02:48.923 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 7721 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/net/dsa/tag_none.ko 2023-11-26T07:02:48.923 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 8953 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/net/dsa/tag_trailer.ko 2023-11-26T07:02:48.923 INFO:teuthology.orchestra.run.smithi143.stdout:drwxr-xr-x root/root 0 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/net/hsr/ 2023-11-26T07:02:48.923 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 99729 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/net/hsr/hsr.ko 2023-11-26T07:02:48.923 INFO:teuthology.orchestra.run.smithi143.stdout:drwxr-xr-x root/root 0 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/net/ieee802154/ 2023-11-26T07:02:48.923 INFO:teuthology.orchestra.run.smithi143.stdout:drwxr-xr-x root/root 0 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/net/ieee802154/6lowpan/ 2023-11-26T07:02:48.923 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 45233 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/net/ieee802154/6lowpan/ieee802154_6lowpan.ko 2023-11-26T07:02:48.923 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 254625 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/net/ieee802154/ieee802154.ko 2023-11-26T07:02:48.923 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 47289 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/net/ieee802154/ieee802154_socket.ko 2023-11-26T07:02:48.924 INFO:teuthology.orchestra.run.smithi143.stdout:drwxr-xr-x root/root 0 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/net/ipv4/ 2023-11-26T07:02:48.924 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 24729 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/net/ipv4/ah4.ko 2023-11-26T07:02:48.924 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 38905 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/net/ipv4/esp4.ko 2023-11-26T07:02:48.924 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 48337 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/net/ipv4/fou.ko 2023-11-26T07:02:48.924 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 18121 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/net/ipv4/gre.ko 2023-11-26T07:02:48.924 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 48657 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/net/ipv4/inet_diag.ko 2023-11-26T07:02:48.924 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 60633 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/net/ipv4/ip_gre.ko 2023-11-26T07:02:48.924 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 61225 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/net/ipv4/ip_tunnel.ko 2023-11-26T07:02:48.924 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 34505 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/net/ipv4/ip_vti.ko 2023-11-26T07:02:48.924 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 14001 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/net/ipv4/ipcomp.ko 2023-11-26T07:02:48.924 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 28841 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/net/ipv4/ipip.ko 2023-11-26T07:02:48.924 INFO:teuthology.orchestra.run.smithi143.stdout:drwxr-xr-x root/root 0 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/net/ipv4/netfilter/ 2023-11-26T07:02:48.925 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 37641 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/net/ipv4/netfilter/arp_tables.ko 2023-11-26T07:02:48.925 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 8873 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/net/ipv4/netfilter/arpt_mangle.ko 2023-11-26T07:02:48.925 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 10409 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/net/ipv4/netfilter/arptable_filter.ko 2023-11-26T07:02:48.925 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 41609 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/net/ipv4/netfilter/ip_tables.ko 2023-11-26T07:02:48.925 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 10057 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/net/ipv4/netfilter/ipt_ECN.ko 2023-11-26T07:02:48.925 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 9681 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/net/ipv4/netfilter/ipt_REJECT.ko 2023-11-26T07:02:48.925 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 14121 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/net/ipv4/netfilter/ipt_SYNPROXY.ko 2023-11-26T07:02:48.925 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 11177 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/net/ipv4/netfilter/ipt_ah.ko 2023-11-26T07:02:48.925 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 14881 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/net/ipv4/netfilter/ipt_rpfilter.ko 2023-11-26T07:02:48.925 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 11305 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/net/ipv4/netfilter/iptable_filter.ko 2023-11-26T07:02:48.925 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 11649 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/net/ipv4/netfilter/iptable_mangle.ko 2023-11-26T07:02:48.925 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 14905 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/net/ipv4/netfilter/iptable_nat.ko 2023-11-26T07:02:48.925 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 11545 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/net/ipv4/netfilter/iptable_raw.ko 2023-11-26T07:02:48.926 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 10409 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/net/ipv4/netfilter/iptable_security.ko 2023-11-26T07:02:48.926 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 12425 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/net/ipv4/netfilter/nf_defrag_ipv4.ko 2023-11-26T07:02:48.926 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 8817 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/net/ipv4/netfilter/nf_dup_ipv4.ko 2023-11-26T07:02:48.926 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 27625 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/net/ipv4/netfilter/nf_nat_h323.ko 2023-11-26T07:02:48.926 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 18161 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/net/ipv4/netfilter/nf_nat_pptp.ko 2023-11-26T07:02:48.926 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 13337 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/net/ipv4/netfilter/nf_nat_snmp_basic.ko 2023-11-26T07:02:48.926 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 20561 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/net/ipv4/netfilter/nf_reject_ipv4.ko 2023-11-26T07:02:48.926 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 9329 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/net/ipv4/netfilter/nf_socket_ipv4.ko 2023-11-26T07:02:48.926 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 15177 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/net/ipv4/netfilter/nf_tproxy_ipv4.ko 2023-11-26T07:02:48.926 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 9961 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/net/ipv4/netfilter/nft_dup_ipv4.ko 2023-11-26T07:02:48.926 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 17113 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/net/ipv4/netfilter/nft_fib_ipv4.ko 2023-11-26T07:02:48.926 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 9009 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/net/ipv4/netfilter/nft_reject_ipv4.ko 2023-11-26T07:02:48.927 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 13073 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/net/ipv4/tcp_bic.ko 2023-11-26T07:02:48.927 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 13737 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/net/ipv4/tcp_dctcp.ko 2023-11-26T07:02:48.927 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 15369 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/net/ipv4/tcp_diag.ko 2023-11-26T07:02:48.927 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 9537 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/net/ipv4/tcp_highspeed.ko 2023-11-26T07:02:48.927 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 12401 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/net/ipv4/tcp_htcp.ko 2023-11-26T07:02:48.927 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 10449 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/net/ipv4/tcp_hybla.ko 2023-11-26T07:02:48.927 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 11473 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/net/ipv4/tcp_illinois.ko 2023-11-26T07:02:48.927 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 10433 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/net/ipv4/tcp_lp.ko 2023-11-26T07:02:48.927 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 8329 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/net/ipv4/tcp_scalable.ko 2023-11-26T07:02:48.927 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 14425 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/net/ipv4/tcp_vegas.ko 2023-11-26T07:02:48.927 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 10089 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/net/ipv4/tcp_veno.ko 2023-11-26T07:02:48.927 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 10953 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/net/ipv4/tcp_westwood.ko 2023-11-26T07:02:48.928 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 10185 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/net/ipv4/tcp_yeah.ko 2023-11-26T07:02:48.928 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 22601 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/net/ipv4/tunnel4.ko 2023-11-26T07:02:48.928 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 17097 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/net/ipv4/udp_diag.ko 2023-11-26T07:02:48.928 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 41665 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/net/ipv4/udp_tunnel.ko 2023-11-26T07:02:48.928 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 11361 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/net/ipv4/xfrm4_tunnel.ko 2023-11-26T07:02:48.928 INFO:teuthology.orchestra.run.smithi143.stdout:drwxr-xr-x root/root 0 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/net/ipv6/ 2023-11-26T07:02:48.928 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 26377 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/net/ipv6/ah6.ko 2023-11-26T07:02:48.928 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 39537 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/net/ipv6/esp6.ko 2023-11-26T07:02:48.928 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 14425 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/net/ipv6/fou6.ko 2023-11-26T07:02:48.928 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 80897 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/net/ipv6/ip6_gre.ko 2023-11-26T07:02:48.928 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 93577 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/net/ipv6/ip6_tunnel.ko 2023-11-26T07:02:48.928 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 10825 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/net/ipv6/ip6_udp_tunnel.ko 2023-11-26T07:02:48.929 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 53273 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/net/ipv6/ip6_vti.ko 2023-11-26T07:02:48.929 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 13785 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/net/ipv6/ipcomp6.ko 2023-11-26T07:02:48.929 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 19017 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/net/ipv6/mip6.ko 2023-11-26T07:02:48.929 INFO:teuthology.orchestra.run.smithi143.stdout:drwxr-xr-x root/root 0 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/net/ipv6/netfilter/ 2023-11-26T07:02:48.929 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 42169 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/net/ipv6/netfilter/ip6_tables.ko 2023-11-26T07:02:48.929 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 11753 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/net/ipv6/netfilter/ip6t_NPT.ko 2023-11-26T07:02:48.929 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 9753 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/net/ipv6/netfilter/ip6t_REJECT.ko 2023-11-26T07:02:48.929 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 14137 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/net/ipv6/netfilter/ip6t_SYNPROXY.ko 2023-11-26T07:02:48.929 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 14273 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/net/ipv6/netfilter/ip6t_ah.ko 2023-11-26T07:02:48.929 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 7801 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/net/ipv6/netfilter/ip6t_eui64.ko 2023-11-26T07:02:48.929 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 16209 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/net/ipv6/netfilter/ip6t_frag.ko 2023-11-26T07:02:48.929 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 16033 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/net/ipv6/netfilter/ip6t_hbh.ko 2023-11-26T07:02:48.930 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 8913 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/net/ipv6/netfilter/ip6t_ipv6header.ko 2023-11-26T07:02:48.930 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 9897 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/net/ipv6/netfilter/ip6t_mh.ko 2023-11-26T07:02:48.930 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 10937 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/net/ipv6/netfilter/ip6t_rpfilter.ko 2023-11-26T07:02:48.930 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 10897 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/net/ipv6/netfilter/ip6t_rt.ko 2023-11-26T07:02:48.930 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 11321 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/net/ipv6/netfilter/ip6table_filter.ko 2023-11-26T07:02:48.930 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 11121 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/net/ipv6/netfilter/ip6table_mangle.ko 2023-11-26T07:02:48.930 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 15009 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/net/ipv6/netfilter/ip6table_nat.ko 2023-11-26T07:02:48.930 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 11553 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/net/ipv6/netfilter/ip6table_raw.ko 2023-11-26T07:02:48.930 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 10433 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/net/ipv6/netfilter/ip6table_security.ko 2023-11-26T07:02:48.930 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 39113 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/net/ipv6/netfilter/nf_defrag_ipv6.ko 2023-11-26T07:02:48.930 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 8449 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/net/ipv6/netfilter/nf_dup_ipv6.ko 2023-11-26T07:02:48.930 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 25945 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/net/ipv6/netfilter/nf_reject_ipv6.ko 2023-11-26T07:02:48.931 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 9801 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/net/ipv6/netfilter/nf_socket_ipv6.ko 2023-11-26T07:02:48.931 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 15393 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/net/ipv6/netfilter/nf_tproxy_ipv6.ko 2023-11-26T07:02:48.931 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 9633 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/net/ipv6/netfilter/nft_dup_ipv6.ko 2023-11-26T07:02:48.931 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 16025 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/net/ipv6/netfilter/nft_fib_ipv6.ko 2023-11-26T07:02:48.931 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 9009 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/net/ipv6/netfilter/nft_reject_ipv6.ko 2023-11-26T07:02:48.931 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 74073 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/net/ipv6/sit.ko 2023-11-26T07:02:48.931 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 24193 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/net/ipv6/tunnel6.ko 2023-11-26T07:02:48.931 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 23297 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/net/ipv6/xfrm6_tunnel.ko 2023-11-26T07:02:48.931 INFO:teuthology.orchestra.run.smithi143.stdout:drwxr-xr-x root/root 0 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/net/key/ 2023-11-26T07:02:48.931 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 82833 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/net/key/af_key.ko 2023-11-26T07:02:48.931 INFO:teuthology.orchestra.run.smithi143.stdout:drwxr-xr-x root/root 0 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/net/l2tp/ 2023-11-26T07:02:48.931 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 125281 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/net/l2tp/l2tp_core.ko 2023-11-26T07:02:48.932 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 20177 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/net/l2tp/l2tp_debugfs.ko 2023-11-26T07:02:48.932 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 25953 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/net/l2tp/l2tp_eth.ko 2023-11-26T07:02:48.932 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 35569 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/net/l2tp/l2tp_ip.ko 2023-11-26T07:02:48.932 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 36713 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/net/l2tp/l2tp_ip6.ko 2023-11-26T07:02:48.932 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 33001 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/net/l2tp/l2tp_netlink.ko 2023-11-26T07:02:48.932 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 53241 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/net/l2tp/l2tp_ppp.ko 2023-11-26T07:02:48.932 INFO:teuthology.orchestra.run.smithi143.stdout:drwxr-xr-x root/root 0 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/net/lapb/ 2023-11-26T07:02:48.932 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 47977 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/net/lapb/lapb.ko 2023-11-26T07:02:48.932 INFO:teuthology.orchestra.run.smithi143.stdout:drwxr-xr-x root/root 0 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/net/llc/ 2023-11-26T07:02:48.932 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 23193 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/net/llc/llc.ko 2023-11-26T07:02:48.932 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 267913 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/net/llc/llc2.ko 2023-11-26T07:02:48.932 INFO:teuthology.orchestra.run.smithi143.stdout:drwxr-xr-x root/root 0 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/net/mac802154/ 2023-11-26T07:02:48.933 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 290417 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/net/mac802154/mac802154.ko 2023-11-26T07:02:48.933 INFO:teuthology.orchestra.run.smithi143.stdout:drwxr-xr-x root/root 0 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/net/mpls/ 2023-11-26T07:02:48.933 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 9393 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/net/mpls/mpls_gso.ko 2023-11-26T07:02:48.933 INFO:teuthology.orchestra.run.smithi143.stdout:drwxr-xr-x root/root 0 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/ 2023-11-26T07:02:48.933 INFO:teuthology.orchestra.run.smithi143.stdout:drwxr-xr-x root/root 0 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/ipset/ 2023-11-26T07:02:48.933 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 119585 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/ipset/ip_set.ko 2023-11-26T07:02:48.933 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 28353 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/ipset/ip_set_bitmap_ip.ko 2023-11-26T07:02:48.933 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 26913 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/ipset/ip_set_bitmap_ipmac.ko 2023-11-26T07:02:48.933 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 26369 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/ipset/ip_set_bitmap_port.ko 2023-11-26T07:02:48.933 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 100297 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/ipset/ip_set_hash_ip.ko 2023-11-26T07:02:48.933 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 101441 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/ipset/ip_set_hash_ipmark.ko 2023-11-26T07:02:48.933 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 102817 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/ipset/ip_set_hash_ipport.ko 2023-11-26T07:02:48.933 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 103417 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/ipset/ip_set_hash_ipportip.ko 2023-11-26T07:02:48.934 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 112849 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/ipset/ip_set_hash_ipportnet.ko 2023-11-26T07:02:48.934 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 60329 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/ipset/ip_set_hash_mac.ko 2023-11-26T07:02:48.934 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 109593 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/ipset/ip_set_hash_net.ko 2023-11-26T07:02:48.934 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 113129 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/ipset/ip_set_hash_netiface.ko 2023-11-26T07:02:48.934 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 112585 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/ipset/ip_set_hash_netnet.ko 2023-11-26T07:02:48.934 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 111585 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/ipset/ip_set_hash_netport.ko 2023-11-26T07:02:48.934 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 114465 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/ipset/ip_set_hash_netportnet.ko 2023-11-26T07:02:48.934 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 30025 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/ipset/ip_set_list_set.ko 2023-11-26T07:02:48.934 INFO:teuthology.orchestra.run.smithi143.stdout:drwxr-xr-x root/root 0 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/ipvs/ 2023-11-26T07:02:48.934 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 411729 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/ipvs/ip_vs.ko 2023-11-26T07:02:48.934 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 11961 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/ipvs/ip_vs_dh.ko 2023-11-26T07:02:48.935 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 7625 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/ipvs/ip_vs_fo.ko 2023-11-26T07:02:48.935 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 17729 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/ipvs/ip_vs_ftp.ko 2023-11-26T07:02:48.935 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 17905 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/ipvs/ip_vs_lblc.ko 2023-11-26T07:02:48.935 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 20673 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/ipvs/ip_vs_lblcr.ko 2023-11-26T07:02:48.935 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 7689 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/ipvs/ip_vs_lc.ko 2023-11-26T07:02:48.935 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 7689 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/ipvs/ip_vs_nq.ko 2023-11-26T07:02:48.935 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 10681 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/ipvs/ip_vs_pe_sip.ko 2023-11-26T07:02:48.935 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 9233 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/ipvs/ip_vs_rr.ko 2023-11-26T07:02:48.935 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 7697 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/ipvs/ip_vs_sed.ko 2023-11-26T07:02:48.935 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 13521 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/ipvs/ip_vs_sh.ko 2023-11-26T07:02:48.935 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 7697 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/ipvs/ip_vs_wlc.ko 2023-11-26T07:02:48.936 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 10993 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/ipvs/ip_vs_wrr.ko 2023-11-26T07:02:48.936 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 49257 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/nf_conncount.ko 2023-11-26T07:02:48.936 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 408257 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/nf_conntrack.ko 2023-11-26T07:02:48.936 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 15985 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/nf_conntrack_amanda.ko 2023-11-26T07:02:48.936 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 10177 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/nf_conntrack_broadcast.ko 2023-11-26T07:02:48.936 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 33761 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/nf_conntrack_ftp.ko 2023-11-26T07:02:48.936 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 118145 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/nf_conntrack_h323.ko 2023-11-26T07:02:48.936 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 22185 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/nf_conntrack_irc.ko 2023-11-26T07:02:48.936 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 8817 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/nf_conntrack_netbios_ns.ko 2023-11-26T07:02:48.936 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 107233 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/nf_conntrack_netlink.ko 2023-11-26T07:02:48.936 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 33633 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/nf_conntrack_pptp.ko 2023-11-26T07:02:48.936 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 17401 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/nf_conntrack_sane.ko 2023-11-26T07:02:48.936 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 66641 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/nf_conntrack_sip.ko 2023-11-26T07:02:48.937 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 11233 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/nf_conntrack_snmp.ko 2023-11-26T07:02:48.937 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 19185 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/nf_conntrack_tftp.ko 2023-11-26T07:02:48.937 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 10113 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/nf_dup_netdev.ko 2023-11-26T07:02:48.937 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 89425 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/nf_flow_table.ko 2023-11-26T07:02:48.937 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 9593 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/nf_flow_table_inet.ko 2023-11-26T07:02:48.937 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 37129 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/nf_log_syslog.ko 2023-11-26T07:02:48.937 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 95353 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/nf_nat.ko 2023-11-26T07:02:48.937 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 10129 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/nf_nat_amanda.ko 2023-11-26T07:02:48.937 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 12625 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/nf_nat_ftp.ko 2023-11-26T07:02:48.937 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 11857 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/nf_nat_irc.ko 2023-11-26T07:02:48.937 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 28929 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/nf_nat_sip.ko 2023-11-26T07:02:48.937 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 8809 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/nf_nat_tftp.ko 2023-11-26T07:02:48.938 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 44065 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/nf_synproxy_core.ko 2023-11-26T07:02:48.938 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 615649 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/nf_tables.ko 2023-11-26T07:02:48.938 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 42561 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/nfnetlink.ko 2023-11-26T07:02:48.938 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 30233 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/nfnetlink_acct.ko 2023-11-26T07:02:48.938 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 30569 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/nfnetlink_cttimeout.ko 2023-11-26T07:02:48.938 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 39785 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/nfnetlink_log.ko 2023-11-26T07:02:48.938 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 22921 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/nfnetlink_osf.ko 2023-11-26T07:02:48.938 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 49377 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/nfnetlink_queue.ko 2023-11-26T07:02:48.938 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 10457 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/nft_chain_nat.ko 2023-11-26T07:02:48.938 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 33209 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/nft_compat.ko 2023-11-26T07:02:48.938 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 40721 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/nft_ct.ko 2023-11-26T07:02:48.939 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 9601 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/nft_dup_netdev.ko 2023-11-26T07:02:48.939 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 11401 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/nft_fib.ko 2023-11-26T07:02:48.939 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 13745 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/nft_fwd_netdev.ko 2023-11-26T07:02:48.939 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 14185 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/nft_hash.ko 2023-11-26T07:02:48.939 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 20065 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/nft_limit.ko 2023-11-26T07:02:48.939 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 15401 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/nft_log.ko 2023-11-26T07:02:48.939 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 14801 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/nft_masq.ko 2023-11-26T07:02:48.939 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 14329 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/nft_nat.ko 2023-11-26T07:02:48.939 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 13673 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/nft_numgen.ko 2023-11-26T07:02:48.939 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 11841 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/nft_osf.ko 2023-11-26T07:02:48.939 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 14073 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/nft_queue.ko 2023-11-26T07:02:48.939 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 14193 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/nft_redir.ko 2023-11-26T07:02:48.939 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 10497 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/nft_reject.ko 2023-11-26T07:02:48.940 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 10081 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/nft_reject_inet.ko 2023-11-26T07:02:48.940 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 12737 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/nft_socket.ko 2023-11-26T07:02:48.940 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 20369 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/nft_synproxy.ko 2023-11-26T07:02:48.940 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 14849 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/nft_tproxy.ko 2023-11-26T07:02:48.940 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 89313 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/x_tables.ko 2023-11-26T07:02:48.940 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 12097 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/xt_AUDIT.ko 2023-11-26T07:02:48.940 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 9633 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/xt_CHECKSUM.ko 2023-11-26T07:02:48.940 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 7729 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/xt_CLASSIFY.ko 2023-11-26T07:02:48.940 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 11521 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/xt_CONNSECMARK.ko 2023-11-26T07:02:48.940 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 18169 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/xt_CT.ko 2023-11-26T07:02:48.940 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 10497 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/xt_DSCP.ko 2023-11-26T07:02:48.940 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 9465 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/xt_HL.ko 2023-11-26T07:02:48.941 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 13369 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/xt_HMARK.ko 2023-11-26T07:02:48.941 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 38873 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/xt_IDLETIMER.ko 2023-11-26T07:02:48.941 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 13889 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/xt_LED.ko 2023-11-26T07:02:48.941 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 11489 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/xt_LOG.ko 2023-11-26T07:02:48.941 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 11753 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/xt_MASQUERADE.ko 2023-11-26T07:02:48.941 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 11657 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/xt_NETMAP.ko 2023-11-26T07:02:48.941 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 9473 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/xt_NFLOG.ko 2023-11-26T07:02:48.941 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 13289 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/xt_NFQUEUE.ko 2023-11-26T07:02:48.941 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 19665 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/xt_RATEEST.ko 2023-11-26T07:02:48.941 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 11073 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/xt_REDIRECT.ko 2023-11-26T07:02:48.941 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 14337 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/xt_SECMARK.ko 2023-11-26T07:02:48.941 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 16057 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/xt_TCPMSS.ko 2023-11-26T07:02:48.942 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 10433 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/xt_TCPOPTSTRIP.ko 2023-11-26T07:02:48.942 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 17361 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/xt_TEE.ko 2023-11-26T07:02:48.942 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 16785 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/xt_TPROXY.ko 2023-11-26T07:02:48.942 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 8321 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/xt_TRACE.ko 2023-11-26T07:02:48.942 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 13897 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/xt_addrtype.ko 2023-11-26T07:02:48.942 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 13897 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/xt_bpf.ko 2023-11-26T07:02:48.942 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 15529 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/xt_cgroup.ko 2023-11-26T07:02:48.942 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 11209 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/xt_cluster.ko 2023-11-26T07:02:48.942 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 7505 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/xt_comment.ko 2023-11-26T07:02:48.942 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 10465 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/xt_connbytes.ko 2023-11-26T07:02:48.942 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 11081 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/xt_connlabel.ko 2023-11-26T07:02:48.942 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 9401 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/xt_connlimit.ko 2023-11-26T07:02:48.942 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 13129 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/xt_connmark.ko 2023-11-26T07:02:48.943 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 12401 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/xt_conntrack.ko 2023-11-26T07:02:48.943 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 7817 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/xt_cpu.ko 2023-11-26T07:02:48.943 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 10969 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/xt_dccp.ko 2023-11-26T07:02:48.943 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 7865 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/xt_devgroup.ko 2023-11-26T07:02:48.943 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 8801 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/xt_dscp.ko 2023-11-26T07:02:48.943 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 10657 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/xt_ecn.ko 2023-11-26T07:02:48.943 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 11473 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/xt_esp.ko 2023-11-26T07:02:48.943 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 43289 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/xt_hashlimit.ko 2023-11-26T07:02:48.943 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 11313 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/xt_helper.ko 2023-11-26T07:02:48.943 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 7865 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/xt_hl.ko 2023-11-26T07:02:48.943 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 11921 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/xt_ipcomp.ko 2023-11-26T07:02:48.943 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 11473 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/xt_iprange.ko 2023-11-26T07:02:48.944 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 11785 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/xt_ipvs.ko 2023-11-26T07:02:48.944 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 16041 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/xt_l2tp.ko 2023-11-26T07:02:48.944 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 8001 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/xt_length.ko 2023-11-26T07:02:48.944 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 10209 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/xt_limit.ko 2023-11-26T07:02:48.944 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 7545 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/xt_mac.ko 2023-11-26T07:02:48.944 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 8577 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/xt_mark.ko 2023-11-26T07:02:48.944 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 11825 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/xt_multiport.ko 2023-11-26T07:02:48.944 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 13817 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/xt_nat.ko 2023-11-26T07:02:48.944 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 10105 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/xt_nfacct.ko 2023-11-26T07:02:48.944 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 8297 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/xt_osf.ko 2023-11-26T07:02:48.944 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 9553 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/xt_owner.ko 2023-11-26T07:02:48.944 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 9729 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/xt_physdev.ko 2023-11-26T07:02:48.945 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 7633 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/xt_pkttype.ko 2023-11-26T07:02:48.945 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 11985 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/xt_policy.ko 2023-11-26T07:02:48.945 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 9473 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/xt_quota.ko 2023-11-26T07:02:48.945 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 10409 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/xt_rateest.ko 2023-11-26T07:02:48.945 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 8257 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/xt_realm.ko 2023-11-26T07:02:48.945 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 36433 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/xt_recent.ko 2023-11-26T07:02:48.945 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 13329 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/xt_sctp.ko 2023-11-26T07:02:48.945 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 30089 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/xt_set.ko 2023-11-26T07:02:48.945 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 16089 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/xt_socket.ko 2023-11-26T07:02:48.945 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 9505 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/xt_state.ko 2023-11-26T07:02:48.945 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 9473 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/xt_statistic.ko 2023-11-26T07:02:48.945 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 8809 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/xt_string.ko 2023-11-26T07:02:48.946 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 8553 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/xt_tcpmss.ko 2023-11-26T07:02:48.946 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 15777 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/xt_tcpudp.ko 2023-11-26T07:02:48.946 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 11065 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/xt_time.ko 2023-11-26T07:02:48.946 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 9401 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/xt_u32.ko 2023-11-26T07:02:48.946 INFO:teuthology.orchestra.run.smithi143.stdout:drwxr-xr-x root/root 0 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/net/netlink/ 2023-11-26T07:02:48.946 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 13969 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/net/netlink/netlink_diag.ko 2023-11-26T07:02:48.946 INFO:teuthology.orchestra.run.smithi143.stdout:drwxr-xr-x root/root 0 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/net/nsh/ 2023-11-26T07:02:48.946 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 12425 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/net/nsh/nsh.ko 2023-11-26T07:02:48.946 INFO:teuthology.orchestra.run.smithi143.stdout:drwxr-xr-x root/root 0 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/net/openvswitch/ 2023-11-26T07:02:48.946 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 380033 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/net/openvswitch/openvswitch.ko 2023-11-26T07:02:48.946 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 10257 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/net/openvswitch/vport-geneve.ko 2023-11-26T07:02:48.946 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 9449 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/net/openvswitch/vport-gre.ko 2023-11-26T07:02:48.947 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 11793 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/net/openvswitch/vport-vxlan.ko 2023-11-26T07:02:48.947 INFO:teuthology.orchestra.run.smithi143.stdout:drwxr-xr-x root/root 0 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/net/packet/ 2023-11-26T07:02:48.947 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 13057 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/net/packet/af_packet_diag.ko 2023-11-26T07:02:48.947 INFO:teuthology.orchestra.run.smithi143.stdout:drwxr-xr-x root/root 0 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/net/phonet/ 2023-11-26T07:02:48.947 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 110201 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/net/phonet/phonet.ko 2023-11-26T07:02:48.947 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 54721 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/net/phonet/pn_pep.ko 2023-11-26T07:02:48.947 INFO:teuthology.orchestra.run.smithi143.stdout:drwxr-xr-x root/root 0 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/net/rds/ 2023-11-26T07:02:48.947 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 251481 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/net/rds/rds.ko 2023-11-26T07:02:48.947 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 164937 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/net/rds/rds_rdma.ko 2023-11-26T07:02:48.947 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 67377 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/net/rds/rds_tcp.ko 2023-11-26T07:02:48.947 INFO:teuthology.orchestra.run.smithi143.stdout:drwxr-xr-x root/root 0 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/net/rxrpc/ 2023-11-26T07:02:48.947 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 819657 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/net/rxrpc/rxrpc.ko 2023-11-26T07:02:48.948 INFO:teuthology.orchestra.run.smithi143.stdout:drwxr-xr-x root/root 0 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/net/sched/ 2023-11-26T07:02:48.948 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 32241 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/net/sched/act_csum.ko 2023-11-26T07:02:48.948 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 23577 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/net/sched/act_gact.ko 2023-11-26T07:02:48.948 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 26873 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/net/sched/act_ipt.ko 2023-11-26T07:02:48.948 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 34097 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/net/sched/act_mirred.ko 2023-11-26T07:02:48.948 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 23401 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/net/sched/act_nat.ko 2023-11-26T07:02:48.948 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 43449 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/net/sched/act_pedit.ko 2023-11-26T07:02:48.948 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 31961 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/net/sched/act_police.ko 2023-11-26T07:02:48.948 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 20289 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/net/sched/act_simple.ko 2023-11-26T07:02:48.948 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 35297 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/net/sched/act_skbedit.ko 2023-11-26T07:02:48.948 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 36665 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/net/sched/act_vlan.ko 2023-11-26T07:02:48.948 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 26169 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/net/sched/cls_basic.ko 2023-11-26T07:02:48.949 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 33961 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/net/sched/cls_bpf.ko 2023-11-26T07:02:48.949 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 19169 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/net/sched/cls_cgroup.ko 2023-11-26T07:02:48.949 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 31401 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/net/sched/cls_flow.ko 2023-11-26T07:02:48.949 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 31769 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/net/sched/cls_fw.ko 2023-11-26T07:02:48.949 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 38425 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/net/sched/cls_route.ko 2023-11-26T07:02:48.949 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 61409 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/net/sched/cls_u32.ko 2023-11-26T07:02:48.949 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 7441 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/net/sched/em_cmp.ko 2023-11-26T07:02:48.949 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 13409 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/net/sched/em_ipset.ko 2023-11-26T07:02:48.949 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 33289 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/net/sched/em_meta.ko 2023-11-26T07:02:48.949 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 7897 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/net/sched/em_nbyte.ko 2023-11-26T07:02:48.949 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 10625 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/net/sched/em_text.ko 2023-11-26T07:02:48.949 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 7249 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/net/sched/em_u32.ko 2023-11-26T07:02:48.949 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 21649 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/net/sched/sch_choke.ko 2023-11-26T07:02:48.950 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 20849 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/net/sched/sch_codel.ko 2023-11-26T07:02:48.950 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 34633 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/net/sched/sch_drr.ko 2023-11-26T07:02:48.950 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 33609 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/net/sched/sch_fq.ko 2023-11-26T07:02:48.950 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 31617 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/net/sched/sch_fq_codel.ko 2023-11-26T07:02:48.950 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 35433 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/net/sched/sch_gred.ko 2023-11-26T07:02:48.950 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 49817 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/net/sched/sch_hfsc.ko 2023-11-26T07:02:48.950 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 22089 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/net/sched/sch_hhf.ko 2023-11-26T07:02:48.950 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 66393 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/net/sched/sch_htb.ko 2023-11-26T07:02:48.950 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 26073 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/net/sched/sch_ingress.ko 2023-11-26T07:02:48.950 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 30273 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/net/sched/sch_mqprio.ko 2023-11-26T07:02:48.950 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 10793 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/net/sched/sch_mqprio_lib.ko 2023-11-26T07:02:48.950 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 26065 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/net/sched/sch_multiq.ko 2023-11-26T07:02:48.951 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 36137 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/net/sched/sch_netem.ko 2023-11-26T07:02:48.951 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 27497 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/net/sched/sch_pie.ko 2023-11-26T07:02:48.951 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 10625 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/net/sched/sch_plug.ko 2023-11-26T07:02:48.951 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 24729 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/net/sched/sch_prio.ko 2023-11-26T07:02:48.951 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 48729 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/net/sched/sch_qfq.ko 2023-11-26T07:02:48.951 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 32905 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/net/sched/sch_red.ko 2023-11-26T07:02:48.951 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 26881 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/net/sched/sch_sfb.ko 2023-11-26T07:02:48.951 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 31353 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/net/sched/sch_sfq.ko 2023-11-26T07:02:48.951 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 28897 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/net/sched/sch_tbf.ko 2023-11-26T07:02:48.951 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 28105 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/net/sched/sch_teql.ko 2023-11-26T07:02:48.951 INFO:teuthology.orchestra.run.smithi143.stdout:drwxr-xr-x root/root 0 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/net/sctp/ 2023-11-26T07:02:48.951 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 929745 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/net/sctp/sctp.ko 2023-11-26T07:02:48.951 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 19033 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/net/sctp/sctp_diag.ko 2023-11-26T07:02:48.952 INFO:teuthology.orchestra.run.smithi143.stdout:drwxr-xr-x root/root 0 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/net/sunrpc/ 2023-11-26T07:02:48.952 INFO:teuthology.orchestra.run.smithi143.stdout:drwxr-xr-x root/root 0 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/net/sunrpc/auth_gss/ 2023-11-26T07:02:48.952 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 336529 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/net/sunrpc/auth_gss/auth_rpcgss.ko 2023-11-26T07:02:48.952 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 54737 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/net/sunrpc/auth_gss/rpcsec_gss_krb5.ko 2023-11-26T07:02:48.952 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 1520097 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/net/sunrpc/sunrpc.ko 2023-11-26T07:02:48.952 INFO:teuthology.orchestra.run.smithi143.stdout:drwxr-xr-x root/root 0 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/net/sunrpc/xprtrdma/ 2023-11-26T07:02:48.952 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 707657 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/net/sunrpc/xprtrdma/rpcrdma.ko 2023-11-26T07:02:48.952 INFO:teuthology.orchestra.run.smithi143.stdout:drwxr-xr-x root/root 0 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/net/tipc/ 2023-11-26T07:02:48.952 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 10025 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/net/tipc/diag.ko 2023-11-26T07:02:48.952 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 862665 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/net/tipc/tipc.ko 2023-11-26T07:02:48.952 INFO:teuthology.orchestra.run.smithi143.stdout:drwxr-xr-x root/root 0 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/net/unix/ 2023-11-26T07:02:48.952 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 15729 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/net/unix/unix_diag.ko 2023-11-26T07:02:48.953 INFO:teuthology.orchestra.run.smithi143.stdout:drwxr-xr-x root/root 0 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/net/vmw_vsock/ 2023-11-26T07:02:48.953 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 85409 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/net/vmw_vsock/vmw_vsock_virtio_transport_common.ko 2023-11-26T07:02:48.953 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 68049 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/net/vmw_vsock/vmw_vsock_vmci_transport.ko 2023-11-26T07:02:48.953 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 96193 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/net/vmw_vsock/vsock.ko 2023-11-26T07:02:48.953 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 10705 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/net/vmw_vsock/vsock_diag.ko 2023-11-26T07:02:48.953 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 17521 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/net/vmw_vsock/vsock_loopback.ko 2023-11-26T07:02:48.953 INFO:teuthology.orchestra.run.smithi143.stdout:drwxr-xr-x root/root 0 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/net/x25/ 2023-11-26T07:02:48.953 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 125177 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/net/x25/x25.ko 2023-11-26T07:02:48.953 INFO:teuthology.orchestra.run.smithi143.stdout:drwxr-xr-x root/root 0 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/net/xfrm/ 2023-11-26T07:02:48.953 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 22657 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/net/xfrm/xfrm_algo.ko 2023-11-26T07:02:48.953 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 20649 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/net/xfrm/xfrm_ipcomp.ko 2023-11-26T07:02:48.953 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 102257 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/net/xfrm/xfrm_user.ko 2023-11-26T07:02:48.953 INFO:teuthology.orchestra.run.smithi143.stdout:drwxr-xr-x root/root 0 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/virt/ 2023-11-26T07:02:48.954 INFO:teuthology.orchestra.run.smithi143.stdout:drwxr-xr-x root/root 0 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/virt/lib/ 2023-11-26T07:02:48.954 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 15537 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/virt/lib/irqbypass.ko 2023-11-26T07:02:48.954 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 640081 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/modules.alias 2023-11-26T07:02:48.954 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 624107 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/modules.alias.bin 2023-11-26T07:02:48.954 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 7663 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/modules.builtin 2023-11-26T07:02:48.954 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 23428 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/modules.builtin.alias.bin 2023-11-26T07:02:48.954 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 9821 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/modules.builtin.bin 2023-11-26T07:02:48.954 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 70004 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/modules.builtin.modinfo 2023-11-26T07:02:48.954 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 146313 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/modules.dep 2023-11-26T07:02:48.954 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 232548 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/modules.dep.bin 2023-11-26T07:02:48.954 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 210 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/modules.devname 2023-11-26T07:02:48.954 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 72606 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/modules.order 2023-11-26T07:02:48.955 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 879 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/modules.softdep 2023-11-26T07:02:48.955 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 254438 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/modules.symbols 2023-11-26T07:02:48.955 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 309363 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/modules.symbols.bin 2023-11-26T07:02:48.955 INFO:teuthology.orchestra.run.smithi143.stdout:drwxr-xr-x root/root 0 2023-11-23 03:47 ./usr/ 2023-11-26T07:02:48.955 INFO:teuthology.orchestra.run.smithi143.stdout:drwxr-xr-x root/root 0 2023-11-23 03:47 ./usr/share/ 2023-11-26T07:02:48.955 INFO:teuthology.orchestra.run.smithi143.stdout:drwxr-xr-x root/root 0 2023-11-23 03:47 ./usr/share/doc/ 2023-11-26T07:02:48.955 INFO:teuthology.orchestra.run.smithi143.stdout:drwxr-xr-x root/root 0 2023-11-23 03:47 ./usr/share/doc/linux-image-6.6.0-g8247bbb5b470/ 2023-11-26T07:02:48.955 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 172 2023-11-23 03:47 ./usr/share/doc/linux-image-6.6.0-g8247bbb5b470/changelog.Debian.gz 2023-11-26T07:02:48.955 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 691 2023-11-23 03:47 ./usr/share/doc/linux-image-6.6.0-g8247bbb5b470/copyright 2023-11-26T07:02:48.956 DEBUG:teuthology.task.kernel:get_image_version: 6.6.0-g8247bbb5b470 2023-11-26T07:02:48.956 INFO:teuthology.task.kernel:searching for kernel 6.6.0-g8247bbb5b470 2023-11-26T07:02:48.956 DEBUG:teuthology.orchestra.run.smithi143:> egrep '(submenu|menuentry.*6.6.0-g8247bbb5b470).*{' /boot/grub/grub.cfg 2023-11-26T07:02:48.963 INFO:teuthology.orchestra.run.smithi143.stdout:submenu 'Advanced options for Ubuntu' $menuentry_id_option 'gnulinux-advanced-7e9e2103-8bb7-4452-b35a-3b1230743254' { 2023-11-26T07:02:48.963 INFO:teuthology.orchestra.run.smithi143.stdout: menuentry 'Ubuntu, with Linux 6.6.0-g8247bbb5b470' --class ubuntu --class gnu-linux --class gnu --class os $menuentry_id_option 'gnulinux-6.6.0-g8247bbb5b470-advanced-7e9e2103-8bb7-4452-b35a-3b1230743254' { 2023-11-26T07:02:48.963 INFO:teuthology.orchestra.run.smithi143.stdout: menuentry 'Ubuntu, with Linux 6.6.0-g8247bbb5b470 (recovery mode)' --class ubuntu --class gnu-linux --class gnu --class os $menuentry_id_option 'gnulinux-6.6.0-g8247bbb5b470-recovery-7e9e2103-8bb7-4452-b35a-3b1230743254' { 2023-11-26T07:02:48.964 INFO:teuthology.task.kernel:submenu_title:Advanced options for Ubuntu> 2023-11-26T07:02:48.964 INFO:teuthology.task.kernel:default_title:Ubuntu, with Linux 6.6.0-g8247bbb5b470 2023-11-26T07:02:48.964 DEBUG:teuthology.orchestra.run.smithi143:> /bin/echo -e 'cat </dev/null && sudo chmod a+x -- '/etc/grub.d/01_ceph_kernel.tmp~' && sudo mv -- '/etc/grub.d/01_ceph_kernel.tmp~' /etc/grub.d/01_ceph_kernel && sudo update-grub && rm /tmp/linux-image.deb && ( sleep 1 && sudo shutdown -r now & ) 2023-11-26T07:02:49.006 DEBUG:teuthology.task.kernel:Waiting for install on ubuntu@smithi143.front.sepia.ceph.com to complete... 2023-11-26T07:02:49.421 INFO:teuthology.orchestra.run.smithi143.stderr:Sourcing file `/etc/default/grub' 2023-11-26T07:02:49.422 INFO:teuthology.orchestra.run.smithi143.stderr:Sourcing file `/etc/default/grub.d/init-select.cfg' 2023-11-26T07:02:49.442 INFO:teuthology.orchestra.run.smithi143.stderr:Generating grub configuration file ... 2023-11-26T07:02:49.642 INFO:teuthology.orchestra.run.smithi143.stderr:Found linux image: /boot/vmlinuz-6.6.0-g8247bbb5b470 2023-11-26T07:02:49.649 INFO:teuthology.orchestra.run.smithi143.stderr:Found initrd image: /boot/initrd.img-6.6.0-g8247bbb5b470 2023-11-26T07:02:49.877 INFO:teuthology.orchestra.run.smithi143.stderr:Found linux image: /boot/vmlinuz-5.15.0-56-generic 2023-11-26T07:02:49.882 INFO:teuthology.orchestra.run.smithi143.stderr:Found initrd image: /boot/initrd.img-5.15.0-56-generic 2023-11-26T07:02:49.920 INFO:teuthology.orchestra.run.smithi143.stderr:Found linux image: /boot/vmlinuz-5.4.0-135-generic 2023-11-26T07:02:49.925 INFO:teuthology.orchestra.run.smithi143.stderr:Found initrd image: /boot/initrd.img-5.4.0-135-generic 2023-11-26T07:02:50.045 INFO:teuthology.orchestra.run.smithi143.stderr:Warning: os-prober will not be executed to detect other bootable partitions. 2023-11-26T07:02:50.046 INFO:teuthology.orchestra.run.smithi143.stderr:Systems on them will not be added to the GRUB boot configuration. 2023-11-26T07:02:50.046 INFO:teuthology.orchestra.run.smithi143.stderr:Check GRUB_DISABLE_OS_PROBER documentation entry. 2023-11-26T07:02:50.067 INFO:teuthology.orchestra.run.smithi143.stderr:done 2023-11-26T07:03:17.410 INFO:teuthology.task.kernel:Checking client mon.b for new kernel version... 2023-11-26T07:03:17.411 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi188.front.sepia.ceph.com' 2023-11-26T07:03:17.412 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi188.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-11-26T07:03:21.210 INFO:teuthology.task.kernel:Checking client mon.a for new kernel version... 2023-11-26T07:03:21.210 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi143.front.sepia.ceph.com' 2023-11-26T07:03:21.211 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi143.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-11-26T07:03:51.910 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.188 2023-11-26T07:03:55.746 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.143 2023-11-26T07:04:00.915 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi188.front.sepia.ceph.com' 2023-11-26T07:04:00.915 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi188.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-11-26T07:04:04.750 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi143.front.sepia.ceph.com' 2023-11-26T07:04:04.750 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi143.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-11-26T07:04:19.298 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.188 2023-11-26T07:04:23.138 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.143 2023-11-26T07:04:31.306 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi188.front.sepia.ceph.com' 2023-11-26T07:04:31.307 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi188.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-11-26T07:04:31.860 DEBUG:teuthology.orchestra.run.smithi188:> true 2023-11-26T07:04:33.617 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi188.front.sepia.ceph.com' 2023-11-26T07:04:33.617 INFO:teuthology.task.kernel:Checking kernel version of mon.b, want "6.6.0-g8247bbb5b470"... 2023-11-26T07:04:33.618 DEBUG:teuthology.orchestra.run.smithi188:> uname -r 2023-11-26T07:04:33.666 INFO:teuthology.orchestra.run.smithi188.stdout:6.6.0-g8247bbb5b470 2023-11-26T07:04:33.666 DEBUG:teuthology.task.kernel:current kernel version is 6.6.0-g8247bbb5b470 vs 6.6.0-g8247bbb5b470 2023-11-26T07:04:33.666 DEBUG:teuthology.task.kernel:utsrelease strings match, do not need to install 2023-11-26T07:04:33.666 DEBUG:teuthology.task.kernel:Distro of this test job: ubuntu 2023-11-26T07:04:34.667 INFO:teuthology.task.kernel:Enabling kdb on mon.b... 2023-11-26T07:04:34.668 DEBUG:teuthology.orchestra.run.smithi188:> echo ttyS1 | sudo tee /sys/module/kgdboc/parameters/kgdboc 2023-11-26T07:04:34.756 INFO:teuthology.orchestra.run.smithi188.stdout:ttyS1 2023-11-26T07:04:34.809 DEBUG:teuthology.parallel:result is None 2023-11-26T07:04:35.141 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi143.front.sepia.ceph.com' 2023-11-26T07:04:35.141 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi143.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-11-26T07:04:35.489 DEBUG:teuthology.orchestra.run.smithi143:> true 2023-11-26T07:04:36.916 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi143.front.sepia.ceph.com' 2023-11-26T07:04:36.916 INFO:teuthology.task.kernel:Checking kernel version of mon.a, want "6.6.0-g8247bbb5b470"... 2023-11-26T07:04:36.916 DEBUG:teuthology.orchestra.run.smithi143:> uname -r 2023-11-26T07:04:36.968 INFO:teuthology.orchestra.run.smithi143.stdout:6.6.0-g8247bbb5b470 2023-11-26T07:04:36.969 DEBUG:teuthology.task.kernel:current kernel version is 6.6.0-g8247bbb5b470 vs 6.6.0-g8247bbb5b470 2023-11-26T07:04:36.969 DEBUG:teuthology.task.kernel:utsrelease strings match, do not need to install 2023-11-26T07:04:36.969 DEBUG:teuthology.task.kernel:Distro of this test job: ubuntu 2023-11-26T07:04:37.971 INFO:teuthology.task.kernel:Enabling kdb on mon.a... 2023-11-26T07:04:37.971 DEBUG:teuthology.orchestra.run.smithi143:> echo ttyS1 | sudo tee /sys/module/kgdboc/parameters/kgdboc 2023-11-26T07:04:38.300 INFO:teuthology.orchestra.run.smithi143.stdout:ttyS1 2023-11-26T07:04:38.379 DEBUG:teuthology.parallel:result is None 2023-11-26T07:04:38.380 INFO:teuthology.run_tasks:Running task internal.base... 2023-11-26T07:04:38.389 INFO:teuthology.task.internal:Creating test directory... 2023-11-26T07:04:38.389 DEBUG:teuthology.orchestra.run.smithi143:> mkdir -p -m0755 -- /home/ubuntu/cephtest 2023-11-26T07:04:38.393 DEBUG:teuthology.orchestra.run.smithi188:> mkdir -p -m0755 -- /home/ubuntu/cephtest 2023-11-26T07:04:38.402 INFO:teuthology.run_tasks:Running task internal.archive_upload... 2023-11-26T07:04:38.420 INFO:teuthology.run_tasks:Running task internal.archive... 2023-11-26T07:04:38.434 INFO:teuthology.task.internal:Creating archive directory... 2023-11-26T07:04:38.434 DEBUG:teuthology.orchestra.run.smithi143:> install -d -m0755 -- /home/ubuntu/cephtest/archive 2023-11-26T07:04:38.443 DEBUG:teuthology.orchestra.run.smithi188:> install -d -m0755 -- /home/ubuntu/cephtest/archive 2023-11-26T07:04:38.482 INFO:teuthology.run_tasks:Running task internal.coredump... 2023-11-26T07:04:38.493 INFO:teuthology.task.internal:Enabling coredump saving... 2023-11-26T07:04:38.493 DEBUG:teuthology.orchestra.run.smithi143:> install -d -m0755 -- /home/ubuntu/cephtest/archive/coredump && sudo sysctl -w kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core && echo kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core | sudo tee -a /etc/sysctl.conf 2023-11-26T07:04:38.527 DEBUG:teuthology.orchestra.run.smithi188:> install -d -m0755 -- /home/ubuntu/cephtest/archive/coredump && sudo sysctl -w kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core && echo kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core | sudo tee -a /etc/sysctl.conf 2023-11-26T07:04:38.548 INFO:teuthology.orchestra.run.smithi143.stdout:kernel.core_pattern = /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2023-11-26T07:04:38.556 INFO:teuthology.orchestra.run.smithi188.stdout:kernel.core_pattern = /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2023-11-26T07:04:38.561 INFO:teuthology.orchestra.run.smithi143.stdout:kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2023-11-26T07:04:38.571 INFO:teuthology.orchestra.run.smithi188.stdout:kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2023-11-26T07:04:38.574 INFO:teuthology.run_tasks:Running task internal.sudo... 2023-11-26T07:04:38.582 INFO:teuthology.task.internal:Configuring sudo... 2023-11-26T07:04:38.582 DEBUG:teuthology.orchestra.run.smithi143:> sudo sed -i.orig.teuthology -e 's/^\([^#]*\) \(requiretty\)/\1 !\2/g' -e 's/^\([^#]*\) !\(visiblepw\)/\1 \2/g' /etc/sudoers 2023-11-26T07:04:38.612 DEBUG:teuthology.orchestra.run.smithi188:> sudo sed -i.orig.teuthology -e 's/^\([^#]*\) \(requiretty\)/\1 !\2/g' -e 's/^\([^#]*\) !\(visiblepw\)/\1 \2/g' /etc/sudoers 2023-11-26T07:04:38.643 INFO:teuthology.run_tasks:Running task internal.syslog... 2023-11-26T07:04:38.654 INFO:teuthology.task.internal.syslog:Starting syslog monitoring... 2023-11-26T07:04:38.654 DEBUG:teuthology.orchestra.run.smithi143:> mkdir -p -m0755 -- /home/ubuntu/cephtest/archive/syslog 2023-11-26T07:04:38.679 DEBUG:teuthology.orchestra.run.smithi188:> mkdir -p -m0755 -- /home/ubuntu/cephtest/archive/syslog 2023-11-26T07:04:38.694 DEBUG:teuthology.orchestra.run.smithi143:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/kern.log 2023-11-26T07:04:38.741 DEBUG:teuthology.orchestra.run.smithi143:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/misc.log 2023-11-26T07:04:38.792 DEBUG:teuthology.orchestra.run.smithi143:> set -ex 2023-11-26T07:04:38.793 DEBUG:teuthology.orchestra.run.smithi143:> sudo dd of=/etc/rsyslog.d/80-cephtest.conf 2023-11-26T07:04:38.887 DEBUG:teuthology.orchestra.run.smithi188:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/kern.log 2023-11-26T07:04:38.897 DEBUG:teuthology.orchestra.run.smithi188:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/misc.log 2023-11-26T07:04:38.951 DEBUG:teuthology.orchestra.run.smithi188:> set -ex 2023-11-26T07:04:38.951 DEBUG:teuthology.orchestra.run.smithi188:> sudo dd of=/etc/rsyslog.d/80-cephtest.conf 2023-11-26T07:04:39.017 DEBUG:teuthology.orchestra.run.smithi143:> sudo service rsyslog restart 2023-11-26T07:04:39.020 DEBUG:teuthology.orchestra.run.smithi188:> sudo service rsyslog restart 2023-11-26T07:04:39.139 INFO:teuthology.run_tasks:Running task internal.timer... 2023-11-26T07:04:39.148 INFO:teuthology.task.internal:Starting timer... 2023-11-26T07:04:39.148 INFO:teuthology.run_tasks:Running task pcp... 2023-11-26T07:04:39.164 INFO:teuthology.run_tasks:Running task selinux... 2023-11-26T07:04:39.175 DEBUG:teuthology.task.selinux:Excluding smithi143: OS 'ubuntu' does not support SELinux 2023-11-26T07:04:39.175 DEBUG:teuthology.task.selinux:Excluding smithi188: OS 'ubuntu' does not support SELinux 2023-11-26T07:04:39.175 DEBUG:teuthology.task.selinux:Getting current SELinux state 2023-11-26T07:04:39.175 DEBUG:teuthology.task.selinux:Existing SELinux modes: {} 2023-11-26T07:04:39.175 INFO:teuthology.task.selinux:Putting SELinux into permissive mode 2023-11-26T07:04:39.176 INFO:teuthology.run_tasks:Running task ansible.cephlab... 2023-11-26T07:04:39.190 INFO:teuthology.repo_utils:/home/teuthworker/src/git.ceph.com_ceph-cm-ansible_main was just updated or references a specific commit; assuming it is current 2023-11-26T07:04:39.190 INFO:teuthology.repo_utils:Resetting repo at /home/teuthworker/src/git.ceph.com_ceph-cm-ansible_main to origin/main 2023-11-26T07:04:39.206 INFO:teuthology.task.ansible:Playbook: [{'import_playbook': 'ansible_managed.yml'}, {'import_playbook': 'teuthology.yml'}, {'hosts': 'testnodes', 'tasks': [{'set_fact': {'ran_from_cephlab_playbook': True}}]}, {'import_playbook': 'testnodes.yml'}, {'import_playbook': 'container-host.yml'}, {'import_playbook': 'cobbler.yml'}, {'import_playbook': 'paddles.yml'}, {'import_playbook': 'pulpito.yml'}, {'hosts': 'testnodes', 'become': True, 'tasks': [{'name': 'Touch /ceph-qa-ready', 'file': {'path': '/ceph-qa-ready', 'state': 'touch'}, 'when': 'ran_from_cephlab_playbook|bool'}]}] 2023-11-26T07:04:39.207 DEBUG:teuthology.task.ansible:Running ansible-playbook -v --extra-vars '{"ansible_ssh_user": "ubuntu"}' -i /etc/ansible/hosts --limit smithi143.front.sepia.ceph.com,smithi188.front.sepia.ceph.com /home/teuthworker/src/git.ceph.com_ceph-cm-ansible_main/cephlab.yml 2023-11-26T07:13:20.361 DEBUG:teuthology.task.ansible:Reconnecting to [Remote(name='ubuntu@smithi143.front.sepia.ceph.com'), Remote(name='ubuntu@smithi188.front.sepia.ceph.com')] 2023-11-26T07:13:20.362 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi143.front.sepia.ceph.com' 2023-11-26T07:13:20.363 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi143.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-11-26T07:13:20.440 DEBUG:teuthology.orchestra.run.smithi143:> true 2023-11-26T07:13:20.571 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi143.front.sepia.ceph.com' 2023-11-26T07:13:20.572 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi188.front.sepia.ceph.com' 2023-11-26T07:13:20.572 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi188.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-11-26T07:13:20.650 DEBUG:teuthology.orchestra.run.smithi188:> true 2023-11-26T07:13:20.777 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi188.front.sepia.ceph.com' 2023-11-26T07:13:20.778 INFO:teuthology.run_tasks:Running task clock... 2023-11-26T07:13:20.788 INFO:teuthology.task.clock:Syncing clocks and checking initial clock skew... 2023-11-26T07:13:20.788 INFO:teuthology.orchestra.run:Running command with timeout 360 2023-11-26T07:13:20.789 DEBUG:teuthology.orchestra.run.smithi143:> sudo systemctl stop ntp.service || sudo systemctl stop ntpd.service || sudo systemctl stop chronyd.service ; sudo ntpd -gq || sudo chronyc makestep ; sudo systemctl start ntp.service || sudo systemctl start ntpd.service || sudo systemctl start chronyd.service ; PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2023-11-26T07:13:20.792 INFO:teuthology.orchestra.run:Running command with timeout 360 2023-11-26T07:13:20.792 DEBUG:teuthology.orchestra.run.smithi188:> sudo systemctl stop ntp.service || sudo systemctl stop ntpd.service || sudo systemctl stop chronyd.service ; sudo ntpd -gq || sudo chronyc makestep ; sudo systemctl start ntp.service || sudo systemctl start ntpd.service || sudo systemctl start chronyd.service ; PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2023-11-26T07:13:20.828 INFO:teuthology.orchestra.run.smithi143.stdout:26 Nov 07:13:20 ntpd[27972]: ntpd 4.2.8p15@1.3728-o Wed Feb 16 17:13:02 UTC 2022 (1): Starting 2023-11-26T07:13:20.828 INFO:teuthology.orchestra.run.smithi143.stdout:26 Nov 07:13:20 ntpd[27972]: Command line: ntpd -gq 2023-11-26T07:13:20.829 INFO:teuthology.orchestra.run.smithi143.stdout:26 Nov 07:13:20 ntpd[27972]: ---------------------------------------------------- 2023-11-26T07:13:20.829 INFO:teuthology.orchestra.run.smithi143.stdout:26 Nov 07:13:20 ntpd[27972]: ntp-4 is maintained by Network Time Foundation, 2023-11-26T07:13:20.829 INFO:teuthology.orchestra.run.smithi143.stdout:26 Nov 07:13:20 ntpd[27972]: Inc. (NTF), a non-profit 501(c)(3) public-benefit 2023-11-26T07:13:20.829 INFO:teuthology.orchestra.run.smithi143.stdout:26 Nov 07:13:20 ntpd[27972]: corporation. Support and training for ntp-4 are 2023-11-26T07:13:20.829 INFO:teuthology.orchestra.run.smithi143.stdout:26 Nov 07:13:20 ntpd[27972]: available at https://www.nwtime.org/support 2023-11-26T07:13:20.829 INFO:teuthology.orchestra.run.smithi143.stdout:26 Nov 07:13:20 ntpd[27972]: ---------------------------------------------------- 2023-11-26T07:13:20.829 INFO:teuthology.orchestra.run.smithi143.stdout:26 Nov 07:13:20 ntpd[27972]: proto: precision = 0.042 usec (-24) 2023-11-26T07:13:20.830 INFO:teuthology.orchestra.run.smithi143.stdout:26 Nov 07:13:20 ntpd[27972]: basedate set to 2022-02-04 2023-11-26T07:13:20.830 INFO:teuthology.orchestra.run.smithi143.stdout:26 Nov 07:13:20 ntpd[27972]: gps base set to 2022-02-06 (week 2196) 2023-11-26T07:13:20.830 INFO:teuthology.orchestra.run.smithi143.stderr:restrict 0.0.0.0: KOD does nothing without LIMITED. 2023-11-26T07:13:20.830 INFO:teuthology.orchestra.run.smithi143.stdout:26 Nov 07:13:20 ntpd[27972]: restrict 0.0.0.0: KOD does nothing without LIMITED. 2023-11-26T07:13:20.831 INFO:teuthology.orchestra.run.smithi143.stderr:restrict ::: KOD does nothing without LIMITED. 2023-11-26T07:13:20.832 INFO:teuthology.orchestra.run.smithi143.stdout:26 Nov 07:13:20 ntpd[27972]: restrict ::: KOD does nothing without LIMITED. 2023-11-26T07:13:20.832 INFO:teuthology.orchestra.run.smithi143.stdout:26 Nov 07:13:20 ntpd[27972]: Listen and drop on 0 v6wildcard [::]:123 2023-11-26T07:13:20.832 INFO:teuthology.orchestra.run.smithi143.stdout:26 Nov 07:13:20 ntpd[27972]: Listen and drop on 1 v4wildcard 0.0.0.0:123 2023-11-26T07:13:20.832 INFO:teuthology.orchestra.run.smithi143.stdout:26 Nov 07:13:20 ntpd[27972]: Listen normally on 2 lo 127.0.0.1:123 2023-11-26T07:13:20.832 INFO:teuthology.orchestra.run.smithi143.stdout:26 Nov 07:13:20 ntpd[27972]: Listen normally on 3 enp3s0f1 172.21.15.143:123 2023-11-26T07:13:20.832 INFO:teuthology.orchestra.run.smithi143.stdout:26 Nov 07:13:20 ntpd[27972]: Listen normally on 4 lo [::1]:123 2023-11-26T07:13:20.832 INFO:teuthology.orchestra.run.smithi143.stdout:26 Nov 07:13:20 ntpd[27972]: Listen normally on 5 enp3s0f1 [fe80::ec4:7aff:fe88:7783%5]:123 2023-11-26T07:13:20.832 INFO:teuthology.orchestra.run.smithi143.stdout:26 Nov 07:13:20 ntpd[27972]: Listening on routing socket on fd #22 for interface updates 2023-11-26T07:13:20.863 INFO:teuthology.orchestra.run.smithi188.stdout:26 Nov 07:13:20 ntpd[27968]: ntpd 4.2.8p15@1.3728-o Wed Feb 16 17:13:02 UTC 2022 (1): Starting 2023-11-26T07:13:20.863 INFO:teuthology.orchestra.run.smithi188.stdout:26 Nov 07:13:20 ntpd[27968]: Command line: ntpd -gq 2023-11-26T07:13:20.863 INFO:teuthology.orchestra.run.smithi188.stdout:26 Nov 07:13:20 ntpd[27968]: ---------------------------------------------------- 2023-11-26T07:13:20.863 INFO:teuthology.orchestra.run.smithi188.stdout:26 Nov 07:13:20 ntpd[27968]: ntp-4 is maintained by Network Time Foundation, 2023-11-26T07:13:20.863 INFO:teuthology.orchestra.run.smithi188.stdout:26 Nov 07:13:20 ntpd[27968]: Inc. (NTF), a non-profit 501(c)(3) public-benefit 2023-11-26T07:13:20.863 INFO:teuthology.orchestra.run.smithi188.stdout:26 Nov 07:13:20 ntpd[27968]: corporation. Support and training for ntp-4 are 2023-11-26T07:13:20.863 INFO:teuthology.orchestra.run.smithi188.stdout:26 Nov 07:13:20 ntpd[27968]: available at https://www.nwtime.org/support 2023-11-26T07:13:20.863 INFO:teuthology.orchestra.run.smithi188.stdout:26 Nov 07:13:20 ntpd[27968]: ---------------------------------------------------- 2023-11-26T07:13:20.864 INFO:teuthology.orchestra.run.smithi188.stdout:26 Nov 07:13:20 ntpd[27968]: proto: precision = 0.040 usec (-24) 2023-11-26T07:13:20.864 INFO:teuthology.orchestra.run.smithi188.stdout:26 Nov 07:13:20 ntpd[27968]: basedate set to 2022-02-04 2023-11-26T07:13:20.864 INFO:teuthology.orchestra.run.smithi188.stdout:26 Nov 07:13:20 ntpd[27968]: gps base set to 2022-02-06 (week 2196) 2023-11-26T07:13:20.865 INFO:teuthology.orchestra.run.smithi188.stderr:restrict 0.0.0.0: KOD does nothing without LIMITED. 2023-11-26T07:13:20.865 INFO:teuthology.orchestra.run.smithi188.stdout:26 Nov 07:13:20 ntpd[27968]: restrict 0.0.0.0: KOD does nothing without LIMITED. 2023-11-26T07:13:20.865 INFO:teuthology.orchestra.run.smithi188.stderr:restrict ::: KOD does nothing without LIMITED. 2023-11-26T07:13:20.866 INFO:teuthology.orchestra.run.smithi188.stdout:26 Nov 07:13:20 ntpd[27968]: restrict ::: KOD does nothing without LIMITED. 2023-11-26T07:13:20.866 INFO:teuthology.orchestra.run.smithi188.stdout:26 Nov 07:13:20 ntpd[27968]: Listen and drop on 0 v6wildcard [::]:123 2023-11-26T07:13:20.866 INFO:teuthology.orchestra.run.smithi188.stdout:26 Nov 07:13:20 ntpd[27968]: Listen and drop on 1 v4wildcard 0.0.0.0:123 2023-11-26T07:13:20.867 INFO:teuthology.orchestra.run.smithi188.stdout:26 Nov 07:13:20 ntpd[27968]: Listen normally on 2 lo 127.0.0.1:123 2023-11-26T07:13:20.867 INFO:teuthology.orchestra.run.smithi188.stdout:26 Nov 07:13:20 ntpd[27968]: Listen normally on 3 enp3s0f1 172.21.15.188:123 2023-11-26T07:13:20.867 INFO:teuthology.orchestra.run.smithi188.stdout:26 Nov 07:13:20 ntpd[27968]: Listen normally on 4 lo [::1]:123 2023-11-26T07:13:20.867 INFO:teuthology.orchestra.run.smithi188.stdout:26 Nov 07:13:20 ntpd[27968]: Listen normally on 5 enp3s0f1 [fe80::ec4:7aff:fe88:8039%5]:123 2023-11-26T07:13:20.868 INFO:teuthology.orchestra.run.smithi188.stdout:26 Nov 07:13:20 ntpd[27968]: Listening on routing socket on fd #22 for interface updates 2023-11-26T07:13:21.829 INFO:teuthology.orchestra.run.smithi143.stderr:26 Nov 07:13:21 ntpd[27972]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2023-11-26T07:13:21.829 INFO:teuthology.orchestra.run.smithi143.stderr:26 Nov 07:13:21 ntpd[27972]: can't open /var/log/ntpstats/rawstats.20231126: Permission denied 2023-11-26T07:13:21.830 INFO:teuthology.orchestra.run.smithi143.stderr:26 Nov 07:13:21 ntpd[27972]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2023-11-26T07:13:21.830 INFO:teuthology.orchestra.run.smithi143.stderr:26 Nov 07:13:21 ntpd[27972]: can't open /var/log/ntpstats/rawstats.20231126: Permission denied 2023-11-26T07:13:21.830 INFO:teuthology.orchestra.run.smithi143.stderr:26 Nov 07:13:21 ntpd[27972]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2023-11-26T07:13:21.830 INFO:teuthology.orchestra.run.smithi143.stderr:26 Nov 07:13:21 ntpd[27972]: can't open /var/log/ntpstats/rawstats.20231126: Permission denied 2023-11-26T07:13:21.830 INFO:teuthology.orchestra.run.smithi143.stderr:26 Nov 07:13:21 ntpd[27972]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2023-11-26T07:13:21.830 INFO:teuthology.orchestra.run.smithi143.stderr:26 Nov 07:13:21 ntpd[27972]: can't open /var/log/ntpstats/peerstats.20231126: Permission denied 2023-11-26T07:13:21.863 INFO:teuthology.orchestra.run.smithi188.stderr:26 Nov 07:13:21 ntpd[27968]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2023-11-26T07:13:21.863 INFO:teuthology.orchestra.run.smithi188.stderr:26 Nov 07:13:21 ntpd[27968]: can't open /var/log/ntpstats/rawstats.20231126: Permission denied 2023-11-26T07:13:21.864 INFO:teuthology.orchestra.run.smithi188.stderr:26 Nov 07:13:21 ntpd[27968]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2023-11-26T07:13:21.864 INFO:teuthology.orchestra.run.smithi188.stderr:26 Nov 07:13:21 ntpd[27968]: can't open /var/log/ntpstats/rawstats.20231126: Permission denied 2023-11-26T07:13:21.864 INFO:teuthology.orchestra.run.smithi188.stderr:26 Nov 07:13:21 ntpd[27968]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2023-11-26T07:13:21.864 INFO:teuthology.orchestra.run.smithi188.stderr:26 Nov 07:13:21 ntpd[27968]: can't open /var/log/ntpstats/peerstats.20231126: Permission denied 2023-11-26T07:13:22.829 INFO:teuthology.orchestra.run.smithi143.stderr:26 Nov 07:13:22 ntpd[27972]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2023-11-26T07:13:22.829 INFO:teuthology.orchestra.run.smithi143.stderr:26 Nov 07:13:22 ntpd[27972]: can't open /var/log/ntpstats/rawstats.20231126: Permission denied 2023-11-26T07:13:22.830 INFO:teuthology.orchestra.run.smithi143.stderr:26 Nov 07:13:22 ntpd[27972]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2023-11-26T07:13:22.830 INFO:teuthology.orchestra.run.smithi143.stderr:26 Nov 07:13:22 ntpd[27972]: can't open /var/log/ntpstats/rawstats.20231126: Permission denied 2023-11-26T07:13:22.830 INFO:teuthology.orchestra.run.smithi143.stderr:26 Nov 07:13:22 ntpd[27972]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2023-11-26T07:13:22.830 INFO:teuthology.orchestra.run.smithi143.stderr:26 Nov 07:13:22 ntpd[27972]: can't open /var/log/ntpstats/peerstats.20231126: Permission denied 2023-11-26T07:13:23.829 INFO:teuthology.orchestra.run.smithi143.stderr:26 Nov 07:13:23 ntpd[27972]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2023-11-26T07:13:23.829 INFO:teuthology.orchestra.run.smithi143.stderr:26 Nov 07:13:23 ntpd[27972]: can't open /var/log/ntpstats/rawstats.20231126: Permission denied 2023-11-26T07:13:23.829 INFO:teuthology.orchestra.run.smithi143.stderr:26 Nov 07:13:23 ntpd[27972]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2023-11-26T07:13:23.829 INFO:teuthology.orchestra.run.smithi143.stderr:26 Nov 07:13:23 ntpd[27972]: can't open /var/log/ntpstats/rawstats.20231126: Permission denied 2023-11-26T07:13:23.830 INFO:teuthology.orchestra.run.smithi143.stderr:26 Nov 07:13:23 ntpd[27972]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2023-11-26T07:13:23.830 INFO:teuthology.orchestra.run.smithi143.stderr:26 Nov 07:13:23 ntpd[27972]: can't open /var/log/ntpstats/peerstats.20231126: Permission denied 2023-11-26T07:13:23.864 INFO:teuthology.orchestra.run.smithi188.stderr:26 Nov 07:13:23 ntpd[27968]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2023-11-26T07:13:23.864 INFO:teuthology.orchestra.run.smithi188.stderr:26 Nov 07:13:23 ntpd[27968]: can't open /var/log/ntpstats/rawstats.20231126: Permission denied 2023-11-26T07:13:23.864 INFO:teuthology.orchestra.run.smithi188.stderr:26 Nov 07:13:23 ntpd[27968]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2023-11-26T07:13:23.864 INFO:teuthology.orchestra.run.smithi188.stderr:26 Nov 07:13:23 ntpd[27968]: can't open /var/log/ntpstats/rawstats.20231126: Permission denied 2023-11-26T07:13:23.864 INFO:teuthology.orchestra.run.smithi188.stderr:26 Nov 07:13:23 ntpd[27968]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2023-11-26T07:13:23.865 INFO:teuthology.orchestra.run.smithi188.stderr:26 Nov 07:13:23 ntpd[27968]: can't open /var/log/ntpstats/rawstats.20231126: Permission denied 2023-11-26T07:13:23.865 INFO:teuthology.orchestra.run.smithi188.stderr:26 Nov 07:13:23 ntpd[27968]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2023-11-26T07:13:23.865 INFO:teuthology.orchestra.run.smithi188.stderr:26 Nov 07:13:23 ntpd[27968]: can't open /var/log/ntpstats/rawstats.20231126: Permission denied 2023-11-26T07:13:23.865 INFO:teuthology.orchestra.run.smithi188.stderr:26 Nov 07:13:23 ntpd[27968]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2023-11-26T07:13:23.865 INFO:teuthology.orchestra.run.smithi188.stderr:26 Nov 07:13:23 ntpd[27968]: can't open /var/log/ntpstats/peerstats.20231126: Permission denied 2023-11-26T07:13:23.865 INFO:teuthology.orchestra.run.smithi188.stderr:26 Nov 07:13:23 ntpd[27968]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2023-11-26T07:13:23.865 INFO:teuthology.orchestra.run.smithi188.stderr:26 Nov 07:13:23 ntpd[27968]: can't open /var/log/ntpstats/rawstats.20231126: Permission denied 2023-11-26T07:13:23.865 INFO:teuthology.orchestra.run.smithi188.stderr:26 Nov 07:13:23 ntpd[27968]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2023-11-26T07:13:23.865 INFO:teuthology.orchestra.run.smithi188.stderr:26 Nov 07:13:23 ntpd[27968]: can't open /var/log/ntpstats/peerstats.20231126: Permission denied 2023-11-26T07:13:24.829 INFO:teuthology.orchestra.run.smithi143.stderr:26 Nov 07:13:24 ntpd[27972]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2023-11-26T07:13:24.829 INFO:teuthology.orchestra.run.smithi143.stderr:26 Nov 07:13:24 ntpd[27972]: can't open /var/log/ntpstats/rawstats.20231126: Permission denied 2023-11-26T07:13:24.829 INFO:teuthology.orchestra.run.smithi143.stderr:26 Nov 07:13:24 ntpd[27972]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2023-11-26T07:13:24.829 INFO:teuthology.orchestra.run.smithi143.stderr:26 Nov 07:13:24 ntpd[27972]: can't open /var/log/ntpstats/rawstats.20231126: Permission denied 2023-11-26T07:13:24.829 INFO:teuthology.orchestra.run.smithi143.stderr:26 Nov 07:13:24 ntpd[27972]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2023-11-26T07:13:24.830 INFO:teuthology.orchestra.run.smithi143.stderr:26 Nov 07:13:24 ntpd[27972]: can't open /var/log/ntpstats/peerstats.20231126: Permission denied 2023-11-26T07:13:25.829 INFO:teuthology.orchestra.run.smithi143.stderr:26 Nov 07:13:25 ntpd[27972]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2023-11-26T07:13:25.830 INFO:teuthology.orchestra.run.smithi143.stderr:26 Nov 07:13:25 ntpd[27972]: can't open /var/log/ntpstats/rawstats.20231126: Permission denied 2023-11-26T07:13:25.830 INFO:teuthology.orchestra.run.smithi143.stderr:26 Nov 07:13:25 ntpd[27972]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2023-11-26T07:13:25.830 INFO:teuthology.orchestra.run.smithi143.stderr:26 Nov 07:13:25 ntpd[27972]: can't open /var/log/ntpstats/rawstats.20231126: Permission denied 2023-11-26T07:13:25.830 INFO:teuthology.orchestra.run.smithi143.stderr:26 Nov 07:13:25 ntpd[27972]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2023-11-26T07:13:25.830 INFO:teuthology.orchestra.run.smithi143.stderr:26 Nov 07:13:25 ntpd[27972]: can't open /var/log/ntpstats/rawstats.20231126: Permission denied 2023-11-26T07:13:25.830 INFO:teuthology.orchestra.run.smithi143.stderr:26 Nov 07:13:25 ntpd[27972]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2023-11-26T07:13:25.830 INFO:teuthology.orchestra.run.smithi143.stderr:26 Nov 07:13:25 ntpd[27972]: can't open /var/log/ntpstats/peerstats.20231126: Permission denied 2023-11-26T07:13:25.830 INFO:teuthology.orchestra.run.smithi143.stderr:26 Nov 07:13:25 ntpd[27972]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2023-11-26T07:13:25.830 INFO:teuthology.orchestra.run.smithi143.stderr:26 Nov 07:13:25 ntpd[27972]: can't open /var/log/ntpstats/rawstats.20231126: Permission denied 2023-11-26T07:13:25.830 INFO:teuthology.orchestra.run.smithi143.stderr:26 Nov 07:13:25 ntpd[27972]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2023-11-26T07:13:25.830 INFO:teuthology.orchestra.run.smithi143.stderr:26 Nov 07:13:25 ntpd[27972]: can't open /var/log/ntpstats/peerstats.20231126: Permission denied 2023-11-26T07:13:25.864 INFO:teuthology.orchestra.run.smithi188.stderr:26 Nov 07:13:25 ntpd[27968]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2023-11-26T07:13:25.865 INFO:teuthology.orchestra.run.smithi188.stderr:26 Nov 07:13:25 ntpd[27968]: can't open /var/log/ntpstats/rawstats.20231126: Permission denied 2023-11-26T07:13:25.865 INFO:teuthology.orchestra.run.smithi188.stderr:26 Nov 07:13:25 ntpd[27968]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2023-11-26T07:13:25.865 INFO:teuthology.orchestra.run.smithi188.stderr:26 Nov 07:13:25 ntpd[27968]: can't open /var/log/ntpstats/rawstats.20231126: Permission denied 2023-11-26T07:13:25.865 INFO:teuthology.orchestra.run.smithi188.stderr:26 Nov 07:13:25 ntpd[27968]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2023-11-26T07:13:25.865 INFO:teuthology.orchestra.run.smithi188.stderr:26 Nov 07:13:25 ntpd[27968]: can't open /var/log/ntpstats/rawstats.20231126: Permission denied 2023-11-26T07:13:25.865 INFO:teuthology.orchestra.run.smithi188.stderr:26 Nov 07:13:25 ntpd[27968]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2023-11-26T07:13:25.865 INFO:teuthology.orchestra.run.smithi188.stderr:26 Nov 07:13:25 ntpd[27968]: can't open /var/log/ntpstats/rawstats.20231126: Permission denied 2023-11-26T07:13:25.865 INFO:teuthology.orchestra.run.smithi188.stderr:26 Nov 07:13:25 ntpd[27968]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2023-11-26T07:13:25.865 INFO:teuthology.orchestra.run.smithi188.stderr:26 Nov 07:13:25 ntpd[27968]: can't open /var/log/ntpstats/peerstats.20231126: Permission denied 2023-11-26T07:13:25.865 INFO:teuthology.orchestra.run.smithi188.stderr:26 Nov 07:13:25 ntpd[27968]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2023-11-26T07:13:25.865 INFO:teuthology.orchestra.run.smithi188.stderr:26 Nov 07:13:25 ntpd[27968]: can't open /var/log/ntpstats/rawstats.20231126: Permission denied 2023-11-26T07:13:25.865 INFO:teuthology.orchestra.run.smithi188.stderr:26 Nov 07:13:25 ntpd[27968]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2023-11-26T07:13:25.866 INFO:teuthology.orchestra.run.smithi188.stderr:26 Nov 07:13:25 ntpd[27968]: can't open /var/log/ntpstats/peerstats.20231126: Permission denied 2023-11-26T07:13:25.866 INFO:teuthology.orchestra.run.smithi188.stderr:26 Nov 07:13:25 ntpd[27968]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2023-11-26T07:13:25.866 INFO:teuthology.orchestra.run.smithi188.stderr:26 Nov 07:13:25 ntpd[27968]: can't open /var/log/ntpstats/rawstats.20231126: Permission denied 2023-11-26T07:13:25.866 INFO:teuthology.orchestra.run.smithi188.stderr:26 Nov 07:13:25 ntpd[27968]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2023-11-26T07:13:25.866 INFO:teuthology.orchestra.run.smithi188.stderr:26 Nov 07:13:25 ntpd[27968]: can't open /var/log/ntpstats/peerstats.20231126: Permission denied 2023-11-26T07:13:26.829 INFO:teuthology.orchestra.run.smithi143.stderr:26 Nov 07:13:26 ntpd[27972]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2023-11-26T07:13:26.829 INFO:teuthology.orchestra.run.smithi143.stderr:26 Nov 07:13:26 ntpd[27972]: can't open /var/log/ntpstats/rawstats.20231126: Permission denied 2023-11-26T07:13:26.830 INFO:teuthology.orchestra.run.smithi143.stderr:26 Nov 07:13:26 ntpd[27972]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2023-11-26T07:13:26.830 INFO:teuthology.orchestra.run.smithi143.stderr:26 Nov 07:13:26 ntpd[27972]: can't open /var/log/ntpstats/rawstats.20231126: Permission denied 2023-11-26T07:13:26.830 INFO:teuthology.orchestra.run.smithi143.stderr:26 Nov 07:13:26 ntpd[27972]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2023-11-26T07:13:26.830 INFO:teuthology.orchestra.run.smithi143.stderr:26 Nov 07:13:26 ntpd[27972]: can't open /var/log/ntpstats/peerstats.20231126: Permission denied 2023-11-26T07:13:27.829 INFO:teuthology.orchestra.run.smithi143.stderr:26 Nov 07:13:27 ntpd[27972]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2023-11-26T07:13:27.830 INFO:teuthology.orchestra.run.smithi143.stderr:26 Nov 07:13:27 ntpd[27972]: can't open /var/log/ntpstats/rawstats.20231126: Permission denied 2023-11-26T07:13:27.830 INFO:teuthology.orchestra.run.smithi143.stderr:26 Nov 07:13:27 ntpd[27972]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2023-11-26T07:13:27.830 INFO:teuthology.orchestra.run.smithi143.stderr:26 Nov 07:13:27 ntpd[27972]: can't open /var/log/ntpstats/rawstats.20231126: Permission denied 2023-11-26T07:13:27.830 INFO:teuthology.orchestra.run.smithi143.stderr:26 Nov 07:13:27 ntpd[27972]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2023-11-26T07:13:27.830 INFO:teuthology.orchestra.run.smithi143.stderr:26 Nov 07:13:27 ntpd[27972]: can't open /var/log/ntpstats/rawstats.20231126: Permission denied 2023-11-26T07:13:27.830 INFO:teuthology.orchestra.run.smithi143.stderr:26 Nov 07:13:27 ntpd[27972]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2023-11-26T07:13:27.831 INFO:teuthology.orchestra.run.smithi143.stderr:26 Nov 07:13:27 ntpd[27972]: can't open /var/log/ntpstats/peerstats.20231126: Permission denied 2023-11-26T07:13:27.831 INFO:teuthology.orchestra.run.smithi143.stdout:26 Nov 07:13:27 ntpd[27972]: ntpd: time slew -0.000047 s 2023-11-26T07:13:27.831 INFO:teuthology.orchestra.run.smithi143.stdout:ntpd: time slew -0.000047s 2023-11-26T07:13:27.832 INFO:teuthology.orchestra.run.smithi143.stderr:26 Nov 07:13:27 ntpd[27972]: couldn't unlink /var/log/ntpstats/loopstats: Permission denied 2023-11-26T07:13:27.832 INFO:teuthology.orchestra.run.smithi143.stderr:26 Nov 07:13:27 ntpd[27972]: can't open /var/log/ntpstats/loopstats.20231126: Permission denied 2023-11-26T07:13:27.865 INFO:teuthology.orchestra.run.smithi188.stderr:26 Nov 07:13:27 ntpd[27968]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2023-11-26T07:13:27.865 INFO:teuthology.orchestra.run.smithi188.stderr:26 Nov 07:13:27 ntpd[27968]: can't open /var/log/ntpstats/rawstats.20231126: Permission denied 2023-11-26T07:13:27.865 INFO:teuthology.orchestra.run.smithi188.stderr:26 Nov 07:13:27 ntpd[27968]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2023-11-26T07:13:27.865 INFO:teuthology.orchestra.run.smithi188.stderr:26 Nov 07:13:27 ntpd[27968]: can't open /var/log/ntpstats/rawstats.20231126: Permission denied 2023-11-26T07:13:27.866 INFO:teuthology.orchestra.run.smithi188.stderr:26 Nov 07:13:27 ntpd[27968]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2023-11-26T07:13:27.866 INFO:teuthology.orchestra.run.smithi188.stderr:26 Nov 07:13:27 ntpd[27968]: can't open /var/log/ntpstats/rawstats.20231126: Permission denied 2023-11-26T07:13:27.866 INFO:teuthology.orchestra.run.smithi188.stderr:26 Nov 07:13:27 ntpd[27968]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2023-11-26T07:13:27.866 INFO:teuthology.orchestra.run.smithi188.stderr:26 Nov 07:13:27 ntpd[27968]: can't open /var/log/ntpstats/rawstats.20231126: Permission denied 2023-11-26T07:13:27.867 INFO:teuthology.orchestra.run.smithi188.stderr:26 Nov 07:13:27 ntpd[27968]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2023-11-26T07:13:27.867 INFO:teuthology.orchestra.run.smithi188.stderr:26 Nov 07:13:27 ntpd[27968]: can't open /var/log/ntpstats/peerstats.20231126: Permission denied 2023-11-26T07:13:27.867 INFO:teuthology.orchestra.run.smithi188.stderr:26 Nov 07:13:27 ntpd[27968]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2023-11-26T07:13:27.867 INFO:teuthology.orchestra.run.smithi188.stderr:26 Nov 07:13:27 ntpd[27968]: can't open /var/log/ntpstats/rawstats.20231126: Permission denied 2023-11-26T07:13:27.867 INFO:teuthology.orchestra.run.smithi188.stderr:26 Nov 07:13:27 ntpd[27968]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2023-11-26T07:13:27.868 INFO:teuthology.orchestra.run.smithi188.stderr:26 Nov 07:13:27 ntpd[27968]: can't open /var/log/ntpstats/peerstats.20231126: Permission denied 2023-11-26T07:13:27.868 INFO:teuthology.orchestra.run.smithi188.stdout:26 Nov 07:13:27 ntpd[27968]: ntpd: time slew -0.002990 s 2023-11-26T07:13:27.868 INFO:teuthology.orchestra.run.smithi188.stdout:ntpd: time slew -0.002990s 2023-11-26T07:13:27.869 INFO:teuthology.orchestra.run.smithi188.stderr:26 Nov 07:13:27 ntpd[27968]: couldn't unlink /var/log/ntpstats/loopstats: Permission denied 2023-11-26T07:13:27.869 INFO:teuthology.orchestra.run.smithi188.stderr:26 Nov 07:13:27 ntpd[27968]: can't open /var/log/ntpstats/loopstats.20231126: Permission denied 2023-11-26T07:13:27.924 INFO:teuthology.orchestra.run.smithi143.stdout: remote refid st t when poll reach delay offset jitter 2023-11-26T07:13:27.924 INFO:teuthology.orchestra.run.smithi143.stdout:============================================================================== 2023-11-26T07:13:27.924 INFO:teuthology.orchestra.run.smithi143.stdout: hv01.front.sepi .INIT. 16 u - 64 0 0.000 +0.000 0.000 2023-11-26T07:13:27.925 INFO:teuthology.orchestra.run.smithi143.stdout: hv02.front.sepi .INIT. 16 u - 64 0 0.000 +0.000 0.000 2023-11-26T07:13:27.925 INFO:teuthology.orchestra.run.smithi143.stdout: hv03.front.sepi .INIT. 16 u - 64 0 0.000 +0.000 0.000 2023-11-26T07:13:27.925 INFO:teuthology.orchestra.run.smithi143.stdout: hv04.front.sepi .INIT. 16 u - 64 0 0.000 +0.000 0.000 2023-11-26T07:13:27.967 INFO:teuthology.orchestra.run.smithi188.stdout: remote refid st t when poll reach delay offset jitter 2023-11-26T07:13:27.967 INFO:teuthology.orchestra.run.smithi188.stdout:============================================================================== 2023-11-26T07:13:27.967 INFO:teuthology.orchestra.run.smithi188.stdout: hv01.front.sepi .INIT. 16 u - 64 0 0.000 +0.000 0.000 2023-11-26T07:13:27.967 INFO:teuthology.orchestra.run.smithi188.stdout: hv02.front.sepi .INIT. 16 u - 64 0 0.000 +0.000 0.000 2023-11-26T07:13:27.967 INFO:teuthology.orchestra.run.smithi188.stdout: hv03.front.sepi .INIT. 16 u - 64 0 0.000 +0.000 0.000 2023-11-26T07:13:27.967 INFO:teuthology.orchestra.run.smithi188.stdout: hv04.front.sepi .INIT. 16 u - 64 0 0.000 +0.000 0.000 2023-11-26T07:13:27.968 INFO:teuthology.run_tasks:Running task install... 2023-11-26T07:13:27.977 DEBUG:teuthology.task.install:project ceph 2023-11-26T07:13:27.978 DEBUG:teuthology.task.install:INSTALL overrides: {'ceph': {'flavor': 'default', 'sha1': '44c2237fe77713214148c1fd622c872f346f1b99'}} 2023-11-26T07:13:27.978 DEBUG:teuthology.task.install:config {'extra_packages': {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-top', 'cephfs-mirror']}, 'extra_system_packages': {'deb': ['bison', 'flex', 'libelf-dev', 'libssl-dev', 'network-manager', 'iproute2', 'util-linux', 'dump', 'indent', 'libaio-dev', 'libtool-bin', 'uuid-dev', 'xfslibs-dev', 'postgresql', 'postgresql-client', 'postgresql-common', 'postgresql-contrib'], 'rpm': ['bison', 'flex', 'elfutils-libelf-devel', 'openssl-devel', 'NetworkManager', 'iproute', 'util-linux', 'libacl-devel', 'libaio-devel', 'libattr-devel', 'libtool', 'libuuid-devel', 'xfsdump', 'xfsprogs', 'xfsprogs-devel', 'libaio-devel', 'libtool', 'libuuid-devel', 'xfsprogs-devel', 'postgresql', 'postgresql-server', 'postgresql-contrib']}, 'flavor': 'default', 'sha1': '44c2237fe77713214148c1fd622c872f346f1b99'} 2023-11-26T07:13:27.978 INFO:teuthology.task.install:Using flavor: default 2023-11-26T07:13:27.983 DEBUG:teuthology.task.install:Package list is: {'deb': ['ceph', 'cephadm', 'ceph-mds', 'ceph-mgr', 'ceph-common', 'ceph-fuse', 'ceph-test', 'ceph-volume', 'radosgw', 'python3-rados', 'python3-rgw', 'python3-cephfs', 'python3-rbd', 'libcephfs2', 'libcephfs-dev', 'librados2', 'librbd1', 'rbd-fuse'], 'rpm': ['ceph-radosgw', 'ceph-test', 'ceph', 'ceph-base', 'cephadm', 'ceph-immutable-object-cache', 'ceph-mgr', 'ceph-mgr-dashboard', 'ceph-mgr-diskprediction-local', 'ceph-mgr-rook', 'ceph-mgr-cephadm', 'ceph-fuse', 'ceph-volume', 'librados-devel', 'libcephfs2', 'libcephfs-devel', 'librados2', 'librbd1', 'python3-rados', 'python3-rgw', 'python3-cephfs', 'python3-rbd', 'rbd-fuse', 'rbd-mirror', 'rbd-nbd']} 2023-11-26T07:13:27.983 INFO:teuthology.task.install:extra packages: {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-top', 'cephfs-mirror']} 2023-11-26T07:13:27.984 DEBUG:teuthology.orchestra.run.smithi143:> sudo apt-key list | grep Ceph 2023-11-26T07:13:27.984 DEBUG:teuthology.orchestra.run.smithi188:> sudo apt-key list | grep Ceph 2023-11-26T07:13:28.089 INFO:teuthology.orchestra.run.smithi143.stderr:Warning: apt-key is deprecated. Manage keyring files in trusted.gpg.d instead (see apt-key(8)). 2023-11-26T07:13:28.121 INFO:teuthology.orchestra.run.smithi188.stderr:Warning: apt-key is deprecated. Manage keyring files in trusted.gpg.d instead (see apt-key(8)). 2023-11-26T07:13:28.142 INFO:teuthology.orchestra.run.smithi143.stdout:uid [ unknown] Ceph automated package build (Ceph automated package build) 2023-11-26T07:13:28.142 INFO:teuthology.orchestra.run.smithi143.stdout:uid [ unknown] Ceph.com (release key) 2023-11-26T07:13:28.145 INFO:teuthology.task.install.deb:Installing packages: ceph, cephadm, ceph-mds, ceph-mgr, ceph-common, ceph-fuse, ceph-test, ceph-volume, radosgw, python3-rados, python3-rgw, python3-cephfs, python3-rbd, libcephfs2, libcephfs-dev, librados2, librbd1, rbd-fuse, python3-cephfs, cephfs-shell, cephfs-top, cephfs-mirror on remote deb x86_64 2023-11-26T07:13:28.145 INFO:teuthology.task.install.deb:Installing system (non-project) packages: bison, flex, libelf-dev, libssl-dev, network-manager, iproute2, util-linux, dump, indent, libaio-dev, libtool-bin, uuid-dev, xfslibs-dev, postgresql, postgresql-client, postgresql-common, postgresql-contrib on remote deb x86_64 2023-11-26T07:13:28.146 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=ubuntu%2F22.04%2Fx86_64&sha1=44c2237fe77713214148c1fd622c872f346f1b99 2023-11-26T07:13:28.180 INFO:teuthology.orchestra.run.smithi188.stdout:uid [ unknown] Ceph automated package build (Ceph automated package build) 2023-11-26T07:13:28.180 INFO:teuthology.orchestra.run.smithi188.stdout:uid [ unknown] Ceph.com (release key) 2023-11-26T07:13:28.180 INFO:teuthology.task.install.deb:Installing packages: ceph, cephadm, ceph-mds, ceph-mgr, ceph-common, ceph-fuse, ceph-test, ceph-volume, radosgw, python3-rados, python3-rgw, python3-cephfs, python3-rbd, libcephfs2, libcephfs-dev, librados2, librbd1, rbd-fuse, python3-cephfs, cephfs-shell, cephfs-top, cephfs-mirror on remote deb x86_64 2023-11-26T07:13:28.181 INFO:teuthology.task.install.deb:Installing system (non-project) packages: bison, flex, libelf-dev, libssl-dev, network-manager, iproute2, util-linux, dump, indent, libaio-dev, libtool-bin, uuid-dev, xfslibs-dev, postgresql, postgresql-client, postgresql-common, postgresql-contrib on remote deb x86_64 2023-11-26T07:13:28.181 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=ubuntu%2F22.04%2Fx86_64&sha1=44c2237fe77713214148c1fd622c872f346f1b99 2023-11-26T07:13:28.292 INFO:teuthology.task.install.deb:Pulling from https://4.chacra.ceph.com/r/ceph/rishabh-mgr-vol-misc/44c2237fe77713214148c1fd622c872f346f1b99/ubuntu/jammy/flavors/default/ 2023-11-26T07:13:28.293 INFO:teuthology.task.install.deb:Package version is 18.0.0-7484-g44c2237f-1jammy 2023-11-26T07:13:28.392 INFO:teuthology.task.install.deb:Pulling from https://4.chacra.ceph.com/r/ceph/rishabh-mgr-vol-misc/44c2237fe77713214148c1fd622c872f346f1b99/ubuntu/jammy/flavors/default/ 2023-11-26T07:13:28.393 INFO:teuthology.task.install.deb:Package version is 18.0.0-7484-g44c2237f-1jammy 2023-11-26T07:13:28.494 DEBUG:teuthology.orchestra.run.smithi188:> set -ex 2023-11-26T07:13:28.494 DEBUG:teuthology.orchestra.run.smithi188:> sudo dd of=/etc/apt/sources.list.d/ceph.list 2023-11-26T07:13:28.496 DEBUG:teuthology.orchestra.run.smithi143:> set -ex 2023-11-26T07:13:28.497 DEBUG:teuthology.orchestra.run.smithi143:> sudo dd of=/etc/apt/sources.list.d/ceph.list 2023-11-26T07:13:28.517 DEBUG:teuthology.orchestra.run.smithi143:> sudo apt-get update 2023-11-26T07:13:28.518 DEBUG:teuthology.orchestra.run.smithi188:> sudo apt-get update 2023-11-26T07:13:28.710 INFO:teuthology.orchestra.run.smithi143.stdout:Hit:1 http://archive.ubuntu.com/ubuntu jammy InRelease 2023-11-26T07:13:28.711 INFO:teuthology.orchestra.run.smithi188.stdout:Hit:1 http://security.ubuntu.com/ubuntu jammy-security InRelease 2023-11-26T07:13:28.711 INFO:teuthology.orchestra.run.smithi188.stdout:Hit:2 http://archive.ubuntu.com/ubuntu jammy InRelease 2023-11-26T07:13:28.719 INFO:teuthology.orchestra.run.smithi143.stdout:Hit:2 http://archive.ubuntu.com/ubuntu jammy-updates InRelease 2023-11-26T07:13:28.721 INFO:teuthology.orchestra.run.smithi188.stdout:Hit:3 http://archive.ubuntu.com/ubuntu jammy-updates InRelease 2023-11-26T07:13:28.738 INFO:teuthology.orchestra.run.smithi143.stdout:Hit:3 http://archive.ubuntu.com/ubuntu jammy-backports InRelease 2023-11-26T07:13:28.741 INFO:teuthology.orchestra.run.smithi188.stdout:Hit:4 http://archive.ubuntu.com/ubuntu jammy-backports InRelease 2023-11-26T07:13:28.757 INFO:teuthology.orchestra.run.smithi143.stdout:Ign:4 https://4.chacra.ceph.com/r/ceph/rishabh-mgr-vol-misc/44c2237fe77713214148c1fd622c872f346f1b99/ubuntu/jammy/flavors/default jammy InRelease 2023-11-26T07:13:28.779 INFO:teuthology.orchestra.run.smithi143.stdout:Get:5 https://4.chacra.ceph.com/r/ceph/rishabh-mgr-vol-misc/44c2237fe77713214148c1fd622c872f346f1b99/ubuntu/jammy/flavors/default jammy Release [7,662 B] 2023-11-26T07:13:28.801 INFO:teuthology.orchestra.run.smithi143.stdout:Ign:6 https://4.chacra.ceph.com/r/ceph/rishabh-mgr-vol-misc/44c2237fe77713214148c1fd622c872f346f1b99/ubuntu/jammy/flavors/default jammy Release.gpg 2023-11-26T07:13:28.818 INFO:teuthology.orchestra.run.smithi188.stdout:Ign:5 https://4.chacra.ceph.com/r/ceph/rishabh-mgr-vol-misc/44c2237fe77713214148c1fd622c872f346f1b99/ubuntu/jammy/flavors/default jammy InRelease 2023-11-26T07:13:28.822 INFO:teuthology.orchestra.run.smithi143.stdout:Get:7 https://4.chacra.ceph.com/r/ceph/rishabh-mgr-vol-misc/44c2237fe77713214148c1fd622c872f346f1b99/ubuntu/jammy/flavors/default jammy/main i386 Packages [3,419 B] 2023-11-26T07:13:28.833 INFO:teuthology.orchestra.run.smithi143.stdout:Hit:8 http://security.ubuntu.com/ubuntu jammy-security InRelease 2023-11-26T07:13:28.855 INFO:teuthology.orchestra.run.smithi143.stdout:Get:9 https://4.chacra.ceph.com/r/ceph/rishabh-mgr-vol-misc/44c2237fe77713214148c1fd622c872f346f1b99/ubuntu/jammy/flavors/default jammy/main amd64 Packages [17.5 kB] 2023-11-26T07:13:28.855 INFO:teuthology.orchestra.run.smithi188.stdout:Get:6 https://4.chacra.ceph.com/r/ceph/rishabh-mgr-vol-misc/44c2237fe77713214148c1fd622c872f346f1b99/ubuntu/jammy/flavors/default jammy Release [7,662 B] 2023-11-26T07:13:28.891 INFO:teuthology.orchestra.run.smithi188.stdout:Ign:7 https://4.chacra.ceph.com/r/ceph/rishabh-mgr-vol-misc/44c2237fe77713214148c1fd622c872f346f1b99/ubuntu/jammy/flavors/default jammy Release.gpg 2023-11-26T07:13:28.928 INFO:teuthology.orchestra.run.smithi188.stdout:Get:8 https://4.chacra.ceph.com/r/ceph/rishabh-mgr-vol-misc/44c2237fe77713214148c1fd622c872f346f1b99/ubuntu/jammy/flavors/default jammy/main i386 Packages [3,419 B] 2023-11-26T07:13:28.976 INFO:teuthology.orchestra.run.smithi188.stdout:Get:9 https://4.chacra.ceph.com/r/ceph/rishabh-mgr-vol-misc/44c2237fe77713214148c1fd622c872f346f1b99/ubuntu/jammy/flavors/default jammy/main amd64 Packages [17.5 kB] 2023-11-26T07:13:29.315 INFO:teuthology.orchestra.run.smithi143.stdout:Fetched 28.5 kB in 1s (49.9 kB/s) 2023-11-26T07:13:29.317 INFO:teuthology.orchestra.run.smithi188.stdout:Fetched 28.5 kB in 1s (49.8 kB/s) 2023-11-26T07:13:30.236 INFO:teuthology.orchestra.run.smithi143.stdout:Reading package lists... 2023-11-26T07:13:30.237 INFO:teuthology.orchestra.run.smithi188.stdout:Reading package lists... 2023-11-26T07:13:30.251 DEBUG:teuthology.orchestra.run.smithi143:> sudo DEBIAN_FRONTEND=noninteractive apt-get -y --force-yes -o Dpkg::Options::="--force-confdef" -o Dpkg::Options::="--force-confold" install ceph=18.0.0-7484-g44c2237f-1jammy cephadm=18.0.0-7484-g44c2237f-1jammy ceph-mds=18.0.0-7484-g44c2237f-1jammy ceph-mgr=18.0.0-7484-g44c2237f-1jammy ceph-common=18.0.0-7484-g44c2237f-1jammy ceph-fuse=18.0.0-7484-g44c2237f-1jammy ceph-test=18.0.0-7484-g44c2237f-1jammy ceph-volume=18.0.0-7484-g44c2237f-1jammy radosgw=18.0.0-7484-g44c2237f-1jammy python3-rados=18.0.0-7484-g44c2237f-1jammy python3-rgw=18.0.0-7484-g44c2237f-1jammy python3-cephfs=18.0.0-7484-g44c2237f-1jammy python3-rbd=18.0.0-7484-g44c2237f-1jammy libcephfs2=18.0.0-7484-g44c2237f-1jammy libcephfs-dev=18.0.0-7484-g44c2237f-1jammy librados2=18.0.0-7484-g44c2237f-1jammy librbd1=18.0.0-7484-g44c2237f-1jammy rbd-fuse=18.0.0-7484-g44c2237f-1jammy python3-cephfs=18.0.0-7484-g44c2237f-1jammy cephfs-shell=18.0.0-7484-g44c2237f-1jammy cephfs-top=18.0.0-7484-g44c2237f-1jammy cephfs-mirror=18.0.0-7484-g44c2237f-1jammy 2023-11-26T07:13:30.252 DEBUG:teuthology.orchestra.run.smithi188:> sudo DEBIAN_FRONTEND=noninteractive apt-get -y --force-yes -o Dpkg::Options::="--force-confdef" -o Dpkg::Options::="--force-confold" install ceph=18.0.0-7484-g44c2237f-1jammy cephadm=18.0.0-7484-g44c2237f-1jammy ceph-mds=18.0.0-7484-g44c2237f-1jammy ceph-mgr=18.0.0-7484-g44c2237f-1jammy ceph-common=18.0.0-7484-g44c2237f-1jammy ceph-fuse=18.0.0-7484-g44c2237f-1jammy ceph-test=18.0.0-7484-g44c2237f-1jammy ceph-volume=18.0.0-7484-g44c2237f-1jammy radosgw=18.0.0-7484-g44c2237f-1jammy python3-rados=18.0.0-7484-g44c2237f-1jammy python3-rgw=18.0.0-7484-g44c2237f-1jammy python3-cephfs=18.0.0-7484-g44c2237f-1jammy python3-rbd=18.0.0-7484-g44c2237f-1jammy libcephfs2=18.0.0-7484-g44c2237f-1jammy libcephfs-dev=18.0.0-7484-g44c2237f-1jammy librados2=18.0.0-7484-g44c2237f-1jammy librbd1=18.0.0-7484-g44c2237f-1jammy rbd-fuse=18.0.0-7484-g44c2237f-1jammy python3-cephfs=18.0.0-7484-g44c2237f-1jammy cephfs-shell=18.0.0-7484-g44c2237f-1jammy cephfs-top=18.0.0-7484-g44c2237f-1jammy cephfs-mirror=18.0.0-7484-g44c2237f-1jammy 2023-11-26T07:13:30.327 INFO:teuthology.orchestra.run.smithi143.stdout:Reading package lists... 2023-11-26T07:13:30.327 INFO:teuthology.orchestra.run.smithi188.stdout:Reading package lists... 2023-11-26T07:13:30.493 INFO:teuthology.orchestra.run.smithi188.stdout:Building dependency tree... 2023-11-26T07:13:30.494 INFO:teuthology.orchestra.run.smithi188.stdout:Reading state information... 2023-11-26T07:13:30.498 INFO:teuthology.orchestra.run.smithi143.stdout:Building dependency tree... 2023-11-26T07:13:30.498 INFO:teuthology.orchestra.run.smithi143.stdout:Reading state information... 2023-11-26T07:13:30.645 INFO:teuthology.orchestra.run.smithi188.stdout:The following packages were automatically installed and are no longer required: 2023-11-26T07:13:30.645 INFO:teuthology.orchestra.run.smithi188.stdout: libboost-iostreams1.74.0 libboost-thread1.74.0 libgfapi0 libgfrpc0 libgfxdr0 2023-11-26T07:13:30.645 INFO:teuthology.orchestra.run.smithi188.stdout: libglusterfs0 libiscsi7 libpmemobj1 libpython2-dev libpython2-stdlib 2023-11-26T07:13:30.645 INFO:teuthology.orchestra.run.smithi188.stdout: libpython2.7 libpython2.7-dev libpython2.7-minimal libpython2.7-stdlib 2023-11-26T07:13:30.645 INFO:teuthology.orchestra.run.smithi188.stdout: linux-image-5.4.0-135-generic linux-modules-5.4.0-135-generic python2 2023-11-26T07:13:30.646 INFO:teuthology.orchestra.run.smithi188.stdout: python2-dev python2-minimal python2.7 python2.7-dev python2.7-minimal 2023-11-26T07:13:30.646 INFO:teuthology.orchestra.run.smithi188.stdout:Use 'sudo apt autoremove' to remove them. 2023-11-26T07:13:30.646 INFO:teuthology.orchestra.run.smithi188.stdout:The following additional packages will be installed: 2023-11-26T07:13:30.647 INFO:teuthology.orchestra.run.smithi188.stdout: ceph-base ceph-mgr-cephadm ceph-mgr-dashboard ceph-mgr-diskprediction-local 2023-11-26T07:13:30.647 INFO:teuthology.orchestra.run.smithi188.stdout: ceph-mgr-k8sevents ceph-mgr-modules-core ceph-mon ceph-osd jq 2023-11-26T07:13:30.647 INFO:teuthology.orchestra.run.smithi188.stdout: libdouble-conversion3 libjq1 liblttng-ust1 liboath0 libonig5 libpcre2-16-0 2023-11-26T07:13:30.647 INFO:teuthology.orchestra.run.smithi188.stdout: libqt5core5a libqt5dbus5 libqt5network5 libradosstriper1 librdkafka1 librgw2 2023-11-26T07:13:30.648 INFO:teuthology.orchestra.run.smithi188.stdout: libsqlite3-mod-ceph libthrift-0.16.0 nvme-cli python-asyncssh-doc 2023-11-26T07:13:30.648 INFO:teuthology.orchestra.run.smithi188.stdout: python-babel-localedata python-pastedeploy-tpl python3-asyncssh 2023-11-26T07:13:30.648 INFO:teuthology.orchestra.run.smithi188.stdout: python3-babel python3-bcrypt python3-cachetools python3-ceph-argparse 2023-11-26T07:13:30.648 INFO:teuthology.orchestra.run.smithi188.stdout: python3-ceph-common python3-certifi python3-cheroot python3-cherrypy3 2023-11-26T07:13:30.648 INFO:teuthology.orchestra.run.smithi188.stdout: python3-cmd2 python3-colorama python3-google-auth python3-idna 2023-11-26T07:13:30.649 INFO:teuthology.orchestra.run.smithi188.stdout: python3-iniconfig python3-jaraco.classes python3-jaraco.collections 2023-11-26T07:13:30.649 INFO:teuthology.orchestra.run.smithi188.stdout: python3-jaraco.functools python3-jaraco.text python3-jinja2 python3-joblib 2023-11-26T07:13:30.649 INFO:teuthology.orchestra.run.smithi188.stdout: python3-kubernetes python3-logutils python3-mako python3-markupsafe 2023-11-26T07:13:30.649 INFO:teuthology.orchestra.run.smithi188.stdout: python3-natsort python3-openssl python3-paste python3-pastedeploy 2023-11-26T07:13:30.649 INFO:teuthology.orchestra.run.smithi188.stdout: python3-pastescript python3-pecan python3-pluggy python3-portend 2023-11-26T07:13:30.649 INFO:teuthology.orchestra.run.smithi188.stdout: python3-prettytable python3-psutil python3-py python3-pyasn1 2023-11-26T07:13:30.649 INFO:teuthology.orchestra.run.smithi188.stdout: python3-pyasn1-modules python3-pygments python3-pyinotify python3-pyperclip 2023-11-26T07:13:30.649 INFO:teuthology.orchestra.run.smithi188.stdout: python3-pytest python3-repoze.lru python3-requests python3-requests-oauthlib 2023-11-26T07:13:30.649 INFO:teuthology.orchestra.run.smithi188.stdout: python3-routes python3-rsa python3-simplegeneric python3-simplejson 2023-11-26T07:13:30.649 INFO:teuthology.orchestra.run.smithi188.stdout: python3-singledispatch python3-sklearn python3-sklearn-lib python3-tempita 2023-11-26T07:13:30.649 INFO:teuthology.orchestra.run.smithi188.stdout: python3-tempora python3-threadpoolctl python3-toml python3-urllib3 2023-11-26T07:13:30.649 INFO:teuthology.orchestra.run.smithi188.stdout: python3-waitress python3-wcwidth python3-webob python3-websocket 2023-11-26T07:13:30.650 INFO:teuthology.orchestra.run.smithi188.stdout: python3-webtest python3-werkzeug python3-zc.lockfile qttranslations5-l10n 2023-11-26T07:13:30.650 INFO:teuthology.orchestra.run.smithi188.stdout: socat xmlstarlet 2023-11-26T07:13:30.650 INFO:teuthology.orchestra.run.smithi188.stdout:Suggested packages: 2023-11-26T07:13:30.650 INFO:teuthology.orchestra.run.smithi188.stdout: python3-influxdb python-jinja2-doc python3-beaker python-mako-doc 2023-11-26T07:13:30.650 INFO:teuthology.orchestra.run.smithi188.stdout: python-natsort-doc python-openssl-doc python3-openssl-dbg httpd-wsgi 2023-11-26T07:13:30.650 INFO:teuthology.orchestra.run.smithi188.stdout: libapache2-mod-python libapache2-mod-scgi libjs-mochikit python-pecan-doc 2023-11-26T07:13:30.650 INFO:teuthology.orchestra.run.smithi188.stdout: python-psutil-doc subversion python-pygments-doc python-pyinotify-doc 2023-11-26T07:13:30.650 INFO:teuthology.orchestra.run.smithi188.stdout: python3-socks python-requests-doc python3-dap python-sklearn-doc ipython3 2023-11-26T07:13:30.650 INFO:teuthology.orchestra.run.smithi188.stdout: python-waitress-doc python-webob-doc python-webtest-doc python-werkzeug-doc 2023-11-26T07:13:30.650 INFO:teuthology.orchestra.run.smithi188.stdout: python3-watchdog 2023-11-26T07:13:30.650 INFO:teuthology.orchestra.run.smithi188.stdout:Recommended packages: 2023-11-26T07:13:30.650 INFO:teuthology.orchestra.run.smithi188.stdout: btrfs-tools 2023-11-26T07:13:30.652 INFO:teuthology.orchestra.run.smithi143.stdout:The following packages were automatically installed and are no longer required: 2023-11-26T07:13:30.652 INFO:teuthology.orchestra.run.smithi143.stdout: libboost-iostreams1.74.0 libboost-thread1.74.0 libgfapi0 libgfrpc0 libgfxdr0 2023-11-26T07:13:30.652 INFO:teuthology.orchestra.run.smithi143.stdout: libglusterfs0 libiscsi7 libpmemobj1 libpython2-dev libpython2-stdlib 2023-11-26T07:13:30.652 INFO:teuthology.orchestra.run.smithi143.stdout: libpython2.7 libpython2.7-dev libpython2.7-minimal libpython2.7-stdlib 2023-11-26T07:13:30.653 INFO:teuthology.orchestra.run.smithi143.stdout: linux-image-5.4.0-135-generic linux-modules-5.4.0-135-generic python2 2023-11-26T07:13:30.653 INFO:teuthology.orchestra.run.smithi143.stdout: python2-dev python2-minimal python2.7 python2.7-dev python2.7-minimal 2023-11-26T07:13:30.653 INFO:teuthology.orchestra.run.smithi143.stdout:Use 'sudo apt autoremove' to remove them. 2023-11-26T07:13:30.654 INFO:teuthology.orchestra.run.smithi143.stdout:The following additional packages will be installed: 2023-11-26T07:13:30.654 INFO:teuthology.orchestra.run.smithi143.stdout: ceph-base ceph-mgr-cephadm ceph-mgr-dashboard ceph-mgr-diskprediction-local 2023-11-26T07:13:30.654 INFO:teuthology.orchestra.run.smithi143.stdout: ceph-mgr-k8sevents ceph-mgr-modules-core ceph-mon ceph-osd jq 2023-11-26T07:13:30.654 INFO:teuthology.orchestra.run.smithi143.stdout: libdouble-conversion3 libjq1 liblttng-ust1 liboath0 libonig5 libpcre2-16-0 2023-11-26T07:13:30.654 INFO:teuthology.orchestra.run.smithi143.stdout: libqt5core5a libqt5dbus5 libqt5network5 libradosstriper1 librdkafka1 librgw2 2023-11-26T07:13:30.655 INFO:teuthology.orchestra.run.smithi143.stdout: libsqlite3-mod-ceph libthrift-0.16.0 nvme-cli python-asyncssh-doc 2023-11-26T07:13:30.655 INFO:teuthology.orchestra.run.smithi143.stdout: python-babel-localedata python-pastedeploy-tpl python3-asyncssh 2023-11-26T07:13:30.655 INFO:teuthology.orchestra.run.smithi143.stdout: python3-babel python3-bcrypt python3-cachetools python3-ceph-argparse 2023-11-26T07:13:30.656 INFO:teuthology.orchestra.run.smithi143.stdout: python3-ceph-common python3-certifi python3-cheroot python3-cherrypy3 2023-11-26T07:13:30.656 INFO:teuthology.orchestra.run.smithi143.stdout: python3-cmd2 python3-colorama python3-google-auth python3-idna 2023-11-26T07:13:30.656 INFO:teuthology.orchestra.run.smithi143.stdout: python3-iniconfig python3-jaraco.classes python3-jaraco.collections 2023-11-26T07:13:30.656 INFO:teuthology.orchestra.run.smithi143.stdout: python3-jaraco.functools python3-jaraco.text python3-jinja2 python3-joblib 2023-11-26T07:13:30.656 INFO:teuthology.orchestra.run.smithi143.stdout: python3-kubernetes python3-logutils python3-mako python3-markupsafe 2023-11-26T07:13:30.656 INFO:teuthology.orchestra.run.smithi143.stdout: python3-natsort python3-openssl python3-paste python3-pastedeploy 2023-11-26T07:13:30.656 INFO:teuthology.orchestra.run.smithi143.stdout: python3-pastescript python3-pecan python3-pluggy python3-portend 2023-11-26T07:13:30.656 INFO:teuthology.orchestra.run.smithi143.stdout: python3-prettytable python3-psutil python3-py python3-pyasn1 2023-11-26T07:13:30.656 INFO:teuthology.orchestra.run.smithi143.stdout: python3-pyasn1-modules python3-pygments python3-pyinotify python3-pyperclip 2023-11-26T07:13:30.656 INFO:teuthology.orchestra.run.smithi143.stdout: python3-pytest python3-repoze.lru python3-requests python3-requests-oauthlib 2023-11-26T07:13:30.656 INFO:teuthology.orchestra.run.smithi143.stdout: python3-routes python3-rsa python3-simplegeneric python3-simplejson 2023-11-26T07:13:30.656 INFO:teuthology.orchestra.run.smithi143.stdout: python3-singledispatch python3-sklearn python3-sklearn-lib python3-tempita 2023-11-26T07:13:30.656 INFO:teuthology.orchestra.run.smithi143.stdout: python3-tempora python3-threadpoolctl python3-toml python3-urllib3 2023-11-26T07:13:30.657 INFO:teuthology.orchestra.run.smithi143.stdout: python3-waitress python3-wcwidth python3-webob python3-websocket 2023-11-26T07:13:30.657 INFO:teuthology.orchestra.run.smithi143.stdout: python3-webtest python3-werkzeug python3-zc.lockfile qttranslations5-l10n 2023-11-26T07:13:30.657 INFO:teuthology.orchestra.run.smithi143.stdout: socat xmlstarlet 2023-11-26T07:13:30.657 INFO:teuthology.orchestra.run.smithi143.stdout:Suggested packages: 2023-11-26T07:13:30.657 INFO:teuthology.orchestra.run.smithi143.stdout: python3-influxdb python-jinja2-doc python3-beaker python-mako-doc 2023-11-26T07:13:30.657 INFO:teuthology.orchestra.run.smithi143.stdout: python-natsort-doc python-openssl-doc python3-openssl-dbg httpd-wsgi 2023-11-26T07:13:30.657 INFO:teuthology.orchestra.run.smithi143.stdout: libapache2-mod-python libapache2-mod-scgi libjs-mochikit python-pecan-doc 2023-11-26T07:13:30.657 INFO:teuthology.orchestra.run.smithi143.stdout: python-psutil-doc subversion python-pygments-doc python-pyinotify-doc 2023-11-26T07:13:30.657 INFO:teuthology.orchestra.run.smithi143.stdout: python3-socks python-requests-doc python3-dap python-sklearn-doc ipython3 2023-11-26T07:13:30.657 INFO:teuthology.orchestra.run.smithi143.stdout: python-waitress-doc python-webob-doc python-webtest-doc python-werkzeug-doc 2023-11-26T07:13:30.657 INFO:teuthology.orchestra.run.smithi143.stdout: python3-watchdog 2023-11-26T07:13:30.657 INFO:teuthology.orchestra.run.smithi143.stdout:Recommended packages: 2023-11-26T07:13:30.658 INFO:teuthology.orchestra.run.smithi143.stdout: btrfs-tools 2023-11-26T07:13:30.792 INFO:teuthology.orchestra.run.smithi188.stdout:The following NEW packages will be installed: 2023-11-26T07:13:30.792 INFO:teuthology.orchestra.run.smithi188.stdout: ceph ceph-base ceph-common ceph-fuse ceph-mds ceph-mgr ceph-mgr-cephadm 2023-11-26T07:13:30.792 INFO:teuthology.orchestra.run.smithi188.stdout: ceph-mgr-dashboard ceph-mgr-diskprediction-local ceph-mgr-k8sevents 2023-11-26T07:13:30.792 INFO:teuthology.orchestra.run.smithi188.stdout: ceph-mgr-modules-core ceph-mon ceph-osd ceph-test ceph-volume cephadm 2023-11-26T07:13:30.793 INFO:teuthology.orchestra.run.smithi188.stdout: cephfs-mirror cephfs-shell cephfs-top jq libcephfs-dev libcephfs2 2023-11-26T07:13:30.793 INFO:teuthology.orchestra.run.smithi188.stdout: libdouble-conversion3 libjq1 liblttng-ust1 liboath0 libonig5 libpcre2-16-0 2023-11-26T07:13:30.793 INFO:teuthology.orchestra.run.smithi188.stdout: libqt5core5a libqt5dbus5 libqt5network5 librados2 libradosstriper1 librbd1 2023-11-26T07:13:30.794 INFO:teuthology.orchestra.run.smithi188.stdout: librdkafka1 librgw2 libsqlite3-mod-ceph libthrift-0.16.0 nvme-cli 2023-11-26T07:13:30.794 INFO:teuthology.orchestra.run.smithi188.stdout: python-asyncssh-doc python-babel-localedata python-pastedeploy-tpl 2023-11-26T07:13:30.794 INFO:teuthology.orchestra.run.smithi188.stdout: python3-asyncssh python3-babel python3-bcrypt python3-cachetools 2023-11-26T07:13:30.794 INFO:teuthology.orchestra.run.smithi188.stdout: python3-ceph-argparse python3-ceph-common python3-cephfs python3-certifi 2023-11-26T07:13:30.794 INFO:teuthology.orchestra.run.smithi188.stdout: python3-cheroot python3-cherrypy3 python3-cmd2 python3-colorama 2023-11-26T07:13:30.794 INFO:teuthology.orchestra.run.smithi188.stdout: python3-google-auth python3-idna python3-iniconfig python3-jaraco.classes 2023-11-26T07:13:30.794 INFO:teuthology.orchestra.run.smithi188.stdout: python3-jaraco.collections python3-jaraco.functools python3-jaraco.text 2023-11-26T07:13:30.794 INFO:teuthology.orchestra.run.smithi188.stdout: python3-jinja2 python3-joblib python3-kubernetes python3-logutils 2023-11-26T07:13:30.794 INFO:teuthology.orchestra.run.smithi188.stdout: python3-mako python3-markupsafe python3-natsort python3-openssl 2023-11-26T07:13:30.794 INFO:teuthology.orchestra.run.smithi188.stdout: python3-paste python3-pastedeploy python3-pastescript python3-pecan 2023-11-26T07:13:30.795 INFO:teuthology.orchestra.run.smithi188.stdout: python3-pluggy python3-portend python3-prettytable python3-psutil python3-py 2023-11-26T07:13:30.795 INFO:teuthology.orchestra.run.smithi188.stdout: python3-pyasn1 python3-pyasn1-modules python3-pygments python3-pyinotify 2023-11-26T07:13:30.795 INFO:teuthology.orchestra.run.smithi188.stdout: python3-pyperclip python3-pytest python3-rados python3-rbd 2023-11-26T07:13:30.795 INFO:teuthology.orchestra.run.smithi188.stdout: python3-repoze.lru python3-requests python3-requests-oauthlib python3-rgw 2023-11-26T07:13:30.795 INFO:teuthology.orchestra.run.smithi188.stdout: python3-routes python3-rsa python3-simplegeneric python3-simplejson 2023-11-26T07:13:30.795 INFO:teuthology.orchestra.run.smithi188.stdout: python3-singledispatch python3-sklearn python3-sklearn-lib python3-tempita 2023-11-26T07:13:30.795 INFO:teuthology.orchestra.run.smithi188.stdout: python3-tempora python3-threadpoolctl python3-toml python3-urllib3 2023-11-26T07:13:30.795 INFO:teuthology.orchestra.run.smithi188.stdout: python3-waitress python3-wcwidth python3-webob python3-websocket 2023-11-26T07:13:30.795 INFO:teuthology.orchestra.run.smithi188.stdout: python3-webtest python3-werkzeug python3-zc.lockfile qttranslations5-l10n 2023-11-26T07:13:30.795 INFO:teuthology.orchestra.run.smithi188.stdout: radosgw rbd-fuse socat xmlstarlet 2023-11-26T07:13:30.799 INFO:teuthology.orchestra.run.smithi143.stdout:The following NEW packages will be installed: 2023-11-26T07:13:30.799 INFO:teuthology.orchestra.run.smithi143.stdout: ceph ceph-base ceph-common ceph-fuse ceph-mds ceph-mgr ceph-mgr-cephadm 2023-11-26T07:13:30.799 INFO:teuthology.orchestra.run.smithi143.stdout: ceph-mgr-dashboard ceph-mgr-diskprediction-local ceph-mgr-k8sevents 2023-11-26T07:13:30.799 INFO:teuthology.orchestra.run.smithi143.stdout: ceph-mgr-modules-core ceph-mon ceph-osd ceph-test ceph-volume cephadm 2023-11-26T07:13:30.799 INFO:teuthology.orchestra.run.smithi143.stdout: cephfs-mirror cephfs-shell cephfs-top jq libcephfs-dev libcephfs2 2023-11-26T07:13:30.800 INFO:teuthology.orchestra.run.smithi143.stdout: libdouble-conversion3 libjq1 liblttng-ust1 liboath0 libonig5 libpcre2-16-0 2023-11-26T07:13:30.800 INFO:teuthology.orchestra.run.smithi143.stdout: libqt5core5a libqt5dbus5 libqt5network5 librados2 libradosstriper1 librbd1 2023-11-26T07:13:30.801 INFO:teuthology.orchestra.run.smithi143.stdout: librdkafka1 librgw2 libsqlite3-mod-ceph libthrift-0.16.0 nvme-cli 2023-11-26T07:13:30.801 INFO:teuthology.orchestra.run.smithi143.stdout: python-asyncssh-doc python-babel-localedata python-pastedeploy-tpl 2023-11-26T07:13:30.801 INFO:teuthology.orchestra.run.smithi143.stdout: python3-asyncssh python3-babel python3-bcrypt python3-cachetools 2023-11-26T07:13:30.801 INFO:teuthology.orchestra.run.smithi143.stdout: python3-ceph-argparse python3-ceph-common python3-cephfs python3-certifi 2023-11-26T07:13:30.801 INFO:teuthology.orchestra.run.smithi143.stdout: python3-cheroot python3-cherrypy3 python3-cmd2 python3-colorama 2023-11-26T07:13:30.801 INFO:teuthology.orchestra.run.smithi143.stdout: python3-google-auth python3-idna python3-iniconfig python3-jaraco.classes 2023-11-26T07:13:30.801 INFO:teuthology.orchestra.run.smithi143.stdout: python3-jaraco.collections python3-jaraco.functools python3-jaraco.text 2023-11-26T07:13:30.801 INFO:teuthology.orchestra.run.smithi143.stdout: python3-jinja2 python3-joblib python3-kubernetes python3-logutils 2023-11-26T07:13:30.801 INFO:teuthology.orchestra.run.smithi143.stdout: python3-mako python3-markupsafe python3-natsort python3-openssl 2023-11-26T07:13:30.801 INFO:teuthology.orchestra.run.smithi143.stdout: python3-paste python3-pastedeploy python3-pastescript python3-pecan 2023-11-26T07:13:30.801 INFO:teuthology.orchestra.run.smithi143.stdout: python3-pluggy python3-portend python3-prettytable python3-psutil python3-py 2023-11-26T07:13:30.801 INFO:teuthology.orchestra.run.smithi143.stdout: python3-pyasn1 python3-pyasn1-modules python3-pygments python3-pyinotify 2023-11-26T07:13:30.801 INFO:teuthology.orchestra.run.smithi143.stdout: python3-pyperclip python3-pytest python3-rados python3-rbd 2023-11-26T07:13:30.801 INFO:teuthology.orchestra.run.smithi143.stdout: python3-repoze.lru python3-requests python3-requests-oauthlib python3-rgw 2023-11-26T07:13:30.801 INFO:teuthology.orchestra.run.smithi143.stdout: python3-routes python3-rsa python3-simplegeneric python3-simplejson 2023-11-26T07:13:30.802 INFO:teuthology.orchestra.run.smithi143.stdout: python3-singledispatch python3-sklearn python3-sklearn-lib python3-tempita 2023-11-26T07:13:30.802 INFO:teuthology.orchestra.run.smithi143.stdout: python3-tempora python3-threadpoolctl python3-toml python3-urllib3 2023-11-26T07:13:30.802 INFO:teuthology.orchestra.run.smithi143.stdout: python3-waitress python3-wcwidth python3-webob python3-websocket 2023-11-26T07:13:30.802 INFO:teuthology.orchestra.run.smithi143.stdout: python3-webtest python3-werkzeug python3-zc.lockfile qttranslations5-l10n 2023-11-26T07:13:30.802 INFO:teuthology.orchestra.run.smithi143.stdout: radosgw rbd-fuse socat xmlstarlet 2023-11-26T07:13:30.863 INFO:teuthology.orchestra.run.smithi188.stdout:0 upgraded, 114 newly installed, 0 to remove and 264 not upgraded. 2023-11-26T07:13:30.863 INFO:teuthology.orchestra.run.smithi188.stdout:Need to get 186 MB of archives. 2023-11-26T07:13:30.863 INFO:teuthology.orchestra.run.smithi188.stdout:After this operation, 829 MB of additional disk space will be used. 2023-11-26T07:13:30.863 INFO:teuthology.orchestra.run.smithi188.stdout:Get:1 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-pyperclip all 1.8.2-2 [11.5 kB] 2023-11-26T07:13:30.880 INFO:teuthology.orchestra.run.smithi188.stdout:Get:2 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-wcwidth all 0.2.5+dfsg1-1 [21.9 kB] 2023-11-26T07:13:30.891 INFO:teuthology.orchestra.run.smithi188.stdout:Get:3 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-cmd2 all 2.3.3-1 [124 kB] 2023-11-26T07:13:30.931 INFO:teuthology.orchestra.run.smithi188.stdout:Get:4 http://archive.ubuntu.com/ubuntu jammy/main amd64 liblttng-ust1 amd64 2.13.1-1ubuntu1 [190 kB] 2023-11-26T07:13:30.950 INFO:teuthology.orchestra.run.smithi188.stdout:Get:5 http://archive.ubuntu.com/ubuntu jammy/universe amd64 libdouble-conversion3 amd64 3.1.7-4 [39.0 kB] 2023-11-26T07:13:30.953 INFO:teuthology.orchestra.run.smithi188.stdout:Get:6 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 libpcre2-16-0 amd64 10.39-3ubuntu0.1 [203 kB] 2023-11-26T07:13:30.964 INFO:teuthology.orchestra.run.smithi188.stdout:Get:7 http://archive.ubuntu.com/ubuntu jammy-updates/universe amd64 libqt5core5a amd64 5.15.3+dfsg-2ubuntu0.2 [2,006 kB] 2023-11-26T07:13:31.001 INFO:teuthology.orchestra.run.smithi188.stdout:Get:8 http://archive.ubuntu.com/ubuntu jammy-updates/universe amd64 libqt5dbus5 amd64 5.15.3+dfsg-2ubuntu0.2 [222 kB] 2023-11-26T07:13:31.003 INFO:teuthology.orchestra.run.smithi188.stdout:Get:9 http://archive.ubuntu.com/ubuntu jammy-updates/universe amd64 libqt5network5 amd64 5.15.3+dfsg-2ubuntu0.2 [731 kB] 2023-11-26T07:13:31.010 INFO:teuthology.orchestra.run.smithi188.stdout:Get:10 http://archive.ubuntu.com/ubuntu jammy/universe amd64 libthrift-0.16.0 amd64 0.16.0-2 [267 kB] 2023-11-26T07:13:31.013 INFO:teuthology.orchestra.run.smithi188.stdout:Get:11 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-prettytable all 2.5.0-2 [31.3 kB] 2023-11-26T07:13:31.014 INFO:teuthology.orchestra.run.smithi188.stdout:Get:12 http://archive.ubuntu.com/ubuntu jammy/universe amd64 librdkafka1 amd64 1.8.0-1build1 [633 kB] 2023-11-26T07:13:31.020 INFO:teuthology.orchestra.run.smithi188.stdout:Get:13 http://archive.ubuntu.com/ubuntu jammy/main amd64 liboath0 amd64 2.6.7-3build1 [41.1 kB] 2023-11-26T07:13:31.020 INFO:teuthology.orchestra.run.smithi188.stdout:Get:14 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-jaraco.functools all 3.4.0-2 [9,030 B] 2023-11-26T07:13:31.021 INFO:teuthology.orchestra.run.smithi188.stdout:Get:15 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 python3-cheroot all 8.5.2+ds1-1ubuntu3.1 [71.1 kB] 2023-11-26T07:13:31.022 INFO:teuthology.orchestra.run.smithi188.stdout:Get:16 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-jaraco.classes all 3.2.1-3 [6,452 B] 2023-11-26T07:13:31.022 INFO:teuthology.orchestra.run.smithi188.stdout:Get:17 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-jaraco.text all 3.6.0-2 [8,716 B] 2023-11-26T07:13:31.037 INFO:teuthology.orchestra.run.smithi188.stdout:Get:18 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-jaraco.collections all 3.4.0-2 [11.4 kB] 2023-11-26T07:13:31.037 INFO:teuthology.orchestra.run.smithi188.stdout:Get:19 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-tempora all 4.1.2-1 [14.8 kB] 2023-11-26T07:13:31.037 INFO:teuthology.orchestra.run.smithi188.stdout:Get:20 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-portend all 3.0.0-1 [7,240 B] 2023-11-26T07:13:31.038 INFO:teuthology.orchestra.run.smithi188.stdout:Get:21 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-zc.lockfile all 2.0-1 [8,980 B] 2023-11-26T07:13:31.053 INFO:teuthology.orchestra.run.smithi188.stdout:Get:22 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-cherrypy3 all 18.6.1-4 [208 kB] 2023-11-26T07:13:31.055 INFO:teuthology.orchestra.run.smithi188.stdout:Get:23 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-natsort all 8.0.2-1 [35.3 kB] 2023-11-26T07:13:31.056 INFO:teuthology.orchestra.run.smithi188.stdout:Get:24 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-logutils all 0.3.3-8 [17.6 kB] 2023-11-26T07:13:31.056 INFO:teuthology.orchestra.run.smithi188.stdout:Get:25 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-markupsafe amd64 2.0.1-2build1 [12.7 kB] 2023-11-26T07:13:31.057 INFO:teuthology.orchestra.run.smithi188.stdout:Get:26 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 python3-mako all 1.1.3+ds1-2ubuntu0.1 [60.5 kB] 2023-11-26T07:13:31.058 INFO:teuthology.orchestra.run.smithi188.stdout:Get:27 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-simplegeneric all 0.8.1-3 [11.3 kB] 2023-11-26T07:13:31.070 INFO:teuthology.orchestra.run.smithi188.stdout:Get:28 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-singledispatch all 3.4.0.3-3 [7,320 B] 2023-11-26T07:13:31.070 INFO:teuthology.orchestra.run.smithi188.stdout:Get:29 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-webob all 1:1.8.6-1.1 [85.2 kB] 2023-11-26T07:13:31.071 INFO:teuthology.orchestra.run.smithi188.stdout:Get:30 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-waitress all 1.4.4-1.1ubuntu1 [47.3 kB] 2023-11-26T07:13:31.072 INFO:teuthology.orchestra.run.smithi188.stdout:Get:31 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-tempita all 0.5.2-6ubuntu1 [15.1 kB] 2023-11-26T07:13:31.086 INFO:teuthology.orchestra.run.smithi188.stdout:Get:32 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-paste all 3.5.0+dfsg1-1 [456 kB] 2023-11-26T07:13:31.091 INFO:teuthology.orchestra.run.smithi188.stdout:Get:33 http://archive.ubuntu.com/ubuntu jammy/main amd64 python-pastedeploy-tpl all 2.1.1-1 [4,892 B] 2023-11-26T07:13:31.091 INFO:teuthology.orchestra.run.smithi188.stdout:Get:34 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-pastedeploy all 2.1.1-1 [26.6 kB] 2023-11-26T07:13:31.092 INFO:teuthology.orchestra.run.smithi188.stdout:Get:35 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-webtest all 2.0.35-1 [28.5 kB] 2023-11-26T07:13:31.092 INFO:teuthology.orchestra.run.smithi188.stdout:Get:36 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-pecan all 1.3.3-4ubuntu2 [87.3 kB] 2023-11-26T07:13:31.093 INFO:teuthology.orchestra.run.smithi188.stdout:Get:37 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-certifi all 2020.6.20-1 [150 kB] 2023-11-26T07:13:31.103 INFO:teuthology.orchestra.run.smithi188.stdout:Get:38 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-idna all 3.3-1 [49.3 kB] 2023-11-26T07:13:31.104 INFO:teuthology.orchestra.run.smithi188.stdout:Get:39 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 python3-urllib3 all 1.26.5-1~exp1ubuntu0.1 [98.2 kB] 2023-11-26T07:13:31.105 INFO:teuthology.orchestra.run.smithi188.stdout:Get:40 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 python3-requests all 2.25.1+dfsg-2ubuntu0.1 [48.8 kB] 2023-11-26T07:13:31.106 INFO:teuthology.orchestra.run.smithi188.stdout:Get:41 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 python3-werkzeug all 2.0.2+dfsg1-1ubuntu0.22.04.1 [181 kB] 2023-11-26T07:13:31.120 INFO:teuthology.orchestra.run.smithi188.stdout:Get:42 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-openssl all 21.0.0-1 [45.2 kB] 2023-11-26T07:13:31.121 INFO:teuthology.orchestra.run.smithi188.stdout:Get:43 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-asyncssh all 2.5.0-1 [185 kB] 2023-11-26T07:13:31.123 INFO:teuthology.orchestra.run.smithi188.stdout:Get:44 http://archive.ubuntu.com/ubuntu jammy/main amd64 python-babel-localedata all 2.8.0+dfsg.1-7 [4,982 kB] 2023-11-26T07:13:31.141 INFO:teuthology.orchestra.run.smithi188.stdout:Get:45 https://4.chacra.ceph.com/r/ceph/rishabh-mgr-vol-misc/44c2237fe77713214148c1fd622c872f346f1b99/ubuntu/jammy/flavors/default jammy/main amd64 librados2 amd64 18.0.0-7484-g44c2237f-1jammy [3,468 kB] 2023-11-26T07:13:31.168 INFO:teuthology.orchestra.run.smithi188.stdout:Get:46 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-babel all 2.8.0+dfsg.1-7 [85.1 kB] 2023-11-26T07:13:31.169 INFO:teuthology.orchestra.run.smithi188.stdout:Get:47 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-jinja2 all 3.0.3-1 [108 kB] 2023-11-26T07:13:31.170 INFO:teuthology.orchestra.run.smithi188.stdout:Get:48 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-bcrypt amd64 3.2.0-1build1 [32.7 kB] 2023-11-26T07:13:31.171 INFO:teuthology.orchestra.run.smithi188.stdout:Get:49 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-repoze.lru all 0.7-2 [12.1 kB] 2023-11-26T07:13:31.171 INFO:teuthology.orchestra.run.smithi188.stdout:Get:50 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-routes all 2.5.1-1ubuntu1 [89.0 kB] 2023-11-26T07:13:31.172 INFO:teuthology.orchestra.run.smithi188.stdout:Get:51 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-sklearn-lib amd64 0.23.2-5ubuntu6 [2,058 kB] 2023-11-26T07:13:31.192 INFO:teuthology.orchestra.run.smithi188.stdout:Get:52 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-joblib all 0.17.0-4ubuntu1 [204 kB] 2023-11-26T07:13:31.194 INFO:teuthology.orchestra.run.smithi188.stdout:Get:53 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-threadpoolctl all 3.1.0-1 [21.3 kB] 2023-11-26T07:13:31.195 INFO:teuthology.orchestra.run.smithi188.stdout:Get:54 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-sklearn all 0.23.2-5ubuntu6 [1,829 kB] 2023-11-26T07:13:31.212 INFO:teuthology.orchestra.run.smithi188.stdout:Get:55 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-cachetools all 5.0.0-1 [9,722 B] 2023-11-26T07:13:31.212 INFO:teuthology.orchestra.run.smithi188.stdout:Get:56 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-pyasn1 all 0.4.8-1 [50.9 kB] 2023-11-26T07:13:31.213 INFO:teuthology.orchestra.run.smithi188.stdout:Get:57 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-pyasn1-modules all 0.2.1-1 [33.0 kB] 2023-11-26T07:13:31.213 INFO:teuthology.orchestra.run.smithi188.stdout:Get:58 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-rsa all 4.8-1 [28.4 kB] 2023-11-26T07:13:31.214 INFO:teuthology.orchestra.run.smithi188.stdout:Get:59 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-google-auth all 1.5.1-3 [35.7 kB] 2023-11-26T07:13:31.214 INFO:teuthology.orchestra.run.smithi188.stdout:Get:60 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-requests-oauthlib all 1.3.0+ds-0.1 [18.7 kB] 2023-11-26T07:13:31.224 INFO:teuthology.orchestra.run.smithi143.stdout:0 upgraded, 114 newly installed, 0 to remove and 264 not upgraded. 2023-11-26T07:13:31.224 INFO:teuthology.orchestra.run.smithi143.stdout:Need to get 186 MB of archives. 2023-11-26T07:13:31.224 INFO:teuthology.orchestra.run.smithi143.stdout:After this operation, 829 MB of additional disk space will be used. 2023-11-26T07:13:31.224 INFO:teuthology.orchestra.run.smithi143.stdout:Get:1 https://4.chacra.ceph.com/r/ceph/rishabh-mgr-vol-misc/44c2237fe77713214148c1fd622c872f346f1b99/ubuntu/jammy/flavors/default jammy/main amd64 librados2 amd64 18.0.0-7484-g44c2237f-1jammy [3,468 kB] 2023-11-26T07:13:31.225 INFO:teuthology.orchestra.run.smithi188.stdout:Get:61 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-websocket all 1.2.3-1 [34.7 kB] 2023-11-26T07:13:31.225 INFO:teuthology.orchestra.run.smithi188.stdout:Get:62 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-kubernetes all 12.0.1-1ubuntu1 [353 kB] 2023-11-26T07:13:31.229 INFO:teuthology.orchestra.run.smithi188.stdout:Get:63 http://archive.ubuntu.com/ubuntu jammy/main amd64 libonig5 amd64 6.9.7.1-2build1 [172 kB] 2023-11-26T07:13:31.241 INFO:teuthology.orchestra.run.smithi188.stdout:Get:64 http://archive.ubuntu.com/ubuntu jammy/main amd64 libjq1 amd64 1.6-2.1ubuntu3 [133 kB] 2023-11-26T07:13:31.243 INFO:teuthology.orchestra.run.smithi188.stdout:Get:65 http://archive.ubuntu.com/ubuntu jammy/main amd64 jq amd64 1.6-2.1ubuntu3 [52.5 kB] 2023-11-26T07:13:31.244 INFO:teuthology.orchestra.run.smithi188.stdout:Get:66 http://archive.ubuntu.com/ubuntu jammy/main amd64 socat amd64 1.7.4.1-3ubuntu4 [349 kB] 2023-11-26T07:13:31.247 INFO:teuthology.orchestra.run.smithi188.stdout:Get:67 http://archive.ubuntu.com/ubuntu jammy/universe amd64 xmlstarlet amd64 1.6.1-2.1 [265 kB] 2023-11-26T07:13:31.250 INFO:teuthology.orchestra.run.smithi188.stdout:Get:68 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-colorama all 0.4.4-1 [24.5 kB] 2023-11-26T07:13:31.250 INFO:teuthology.orchestra.run.smithi188.stdout:Get:69 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 nvme-cli amd64 1.16-3ubuntu0.1 [474 kB] 2023-11-26T07:13:31.255 INFO:teuthology.orchestra.run.smithi188.stdout:Get:70 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python-asyncssh-doc all 2.5.0-1 [318 kB] 2023-11-26T07:13:31.258 INFO:teuthology.orchestra.run.smithi188.stdout:Get:71 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-iniconfig all 1.1.1-2 [6,024 B] 2023-11-26T07:13:31.259 INFO:teuthology.orchestra.run.smithi188.stdout:Get:72 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-pastescript all 2.0.2-4 [54.6 kB] 2023-11-26T07:13:31.260 INFO:teuthology.orchestra.run.smithi188.stdout:Get:73 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-pluggy all 0.13.0-7.1 [19.0 kB] 2023-11-26T07:13:31.275 INFO:teuthology.orchestra.run.smithi188.stdout:Get:74 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-psutil amd64 5.9.0-1build1 [158 kB] 2023-11-26T07:13:31.277 INFO:teuthology.orchestra.run.smithi188.stdout:Get:75 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-py all 1.10.0-1 [71.9 kB] 2023-11-26T07:13:31.278 INFO:teuthology.orchestra.run.smithi188.stdout:Get:76 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-pygments all 2.11.2+dfsg-2 [750 kB] 2023-11-26T07:13:31.284 INFO:teuthology.orchestra.run.smithi188.stdout:Get:77 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-pyinotify all 0.9.6-1.3 [24.8 kB] 2023-11-26T07:13:31.285 INFO:teuthology.orchestra.run.smithi188.stdout:Get:78 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-toml all 0.10.2-1 [16.5 kB] 2023-11-26T07:13:31.285 INFO:teuthology.orchestra.run.smithi188.stdout:Get:79 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-pytest all 6.2.5-1ubuntu2 [214 kB] 2023-11-26T07:13:31.292 INFO:teuthology.orchestra.run.smithi188.stdout:Get:80 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-simplejson amd64 3.17.6-1build1 [54.7 kB] 2023-11-26T07:13:31.293 INFO:teuthology.orchestra.run.smithi188.stdout:Get:81 http://archive.ubuntu.com/ubuntu jammy/universe amd64 qttranslations5-l10n all 5.15.3-1 [1,983 kB] 2023-11-26T07:13:32.360 INFO:teuthology.orchestra.run.smithi143.stdout:Get:2 https://4.chacra.ceph.com/r/ceph/rishabh-mgr-vol-misc/44c2237fe77713214148c1fd622c872f346f1b99/ubuntu/jammy/flavors/default jammy/main amd64 librbd1 amd64 18.0.0-7484-g44c2237f-1jammy [3,112 kB] 2023-11-26T07:13:32.846 INFO:teuthology.orchestra.run.smithi143.stdout:Get:3 https://4.chacra.ceph.com/r/ceph/rishabh-mgr-vol-misc/44c2237fe77713214148c1fd622c872f346f1b99/ubuntu/jammy/flavors/default jammy/main amd64 libcephfs2 amd64 18.0.0-7484-g44c2237f-1jammy [691 kB] 2023-11-26T07:13:32.943 INFO:teuthology.orchestra.run.smithi143.stdout:Get:4 https://4.chacra.ceph.com/r/ceph/rishabh-mgr-vol-misc/44c2237fe77713214148c1fd622c872f346f1b99/ubuntu/jammy/flavors/default jammy/main amd64 python3-rados amd64 18.0.0-7484-g44c2237f-1jammy [356 kB] 2023-11-26T07:13:33.008 INFO:teuthology.orchestra.run.smithi143.stdout:Get:5 https://4.chacra.ceph.com/r/ceph/rishabh-mgr-vol-misc/44c2237fe77713214148c1fd622c872f346f1b99/ubuntu/jammy/flavors/default jammy/main amd64 python3-ceph-argparse all 18.0.0-7484-g44c2237f-1jammy [32.8 kB] 2023-11-26T07:13:33.010 INFO:teuthology.orchestra.run.smithi143.stdout:Get:6 https://4.chacra.ceph.com/r/ceph/rishabh-mgr-vol-misc/44c2237fe77713214148c1fd622c872f346f1b99/ubuntu/jammy/flavors/default jammy/main amd64 python3-cephfs amd64 18.0.0-7484-g44c2237f-1jammy [180 kB] 2023-11-26T07:13:33.025 INFO:teuthology.orchestra.run.smithi143.stdout:Get:7 https://4.chacra.ceph.com/r/ceph/rishabh-mgr-vol-misc/44c2237fe77713214148c1fd622c872f346f1b99/ubuntu/jammy/flavors/default jammy/main amd64 python3-ceph-common all 18.0.0-7484-g44c2237f-1jammy [63.5 kB] 2023-11-26T07:13:33.031 INFO:teuthology.orchestra.run.smithi143.stdout:Get:8 https://4.chacra.ceph.com/r/ceph/rishabh-mgr-vol-misc/44c2237fe77713214148c1fd622c872f346f1b99/ubuntu/jammy/flavors/default jammy/main amd64 python3-rbd amd64 18.0.0-7484-g44c2237f-1jammy [330 kB] 2023-11-26T07:13:33.066 INFO:teuthology.orchestra.run.smithi188.stdout:Get:82 https://4.chacra.ceph.com/r/ceph/rishabh-mgr-vol-misc/44c2237fe77713214148c1fd622c872f346f1b99/ubuntu/jammy/flavors/default jammy/main amd64 librbd1 amd64 18.0.0-7484-g44c2237f-1jammy [3,112 kB] 2023-11-26T07:13:33.079 INFO:teuthology.orchestra.run.smithi143.stdout:Get:9 https://4.chacra.ceph.com/r/ceph/rishabh-mgr-vol-misc/44c2237fe77713214148c1fd622c872f346f1b99/ubuntu/jammy/flavors/default jammy/main amd64 librgw2 amd64 18.0.0-7484-g44c2237f-1jammy [6,230 kB] 2023-11-26T07:13:33.910 INFO:teuthology.orchestra.run.smithi143.stdout:Get:10 https://4.chacra.ceph.com/r/ceph/rishabh-mgr-vol-misc/44c2237fe77713214148c1fd622c872f346f1b99/ubuntu/jammy/flavors/default jammy/main amd64 python3-rgw amd64 18.0.0-7484-g44c2237f-1jammy [112 kB] 2023-11-26T07:13:33.919 INFO:teuthology.orchestra.run.smithi143.stdout:Get:11 https://4.chacra.ceph.com/r/ceph/rishabh-mgr-vol-misc/44c2237fe77713214148c1fd622c872f346f1b99/ubuntu/jammy/flavors/default jammy/main amd64 libradosstriper1 amd64 18.0.0-7484-g44c2237f-1jammy [439 kB] 2023-11-26T07:13:33.973 INFO:teuthology.orchestra.run.smithi188.stdout:Get:83 https://4.chacra.ceph.com/r/ceph/rishabh-mgr-vol-misc/44c2237fe77713214148c1fd622c872f346f1b99/ubuntu/jammy/flavors/default jammy/main amd64 libcephfs2 amd64 18.0.0-7484-g44c2237f-1jammy [691 kB] 2023-11-26T07:13:33.983 INFO:teuthology.orchestra.run.smithi143.stdout:Get:12 https://4.chacra.ceph.com/r/ceph/rishabh-mgr-vol-misc/44c2237fe77713214148c1fd622c872f346f1b99/ubuntu/jammy/flavors/default jammy/main amd64 ceph-common amd64 18.0.0-7484-g44c2237f-1jammy [22.6 MB] 2023-11-26T07:13:34.160 INFO:teuthology.orchestra.run.smithi188.stdout:Get:84 https://4.chacra.ceph.com/r/ceph/rishabh-mgr-vol-misc/44c2237fe77713214148c1fd622c872f346f1b99/ubuntu/jammy/flavors/default jammy/main amd64 python3-rados amd64 18.0.0-7484-g44c2237f-1jammy [356 kB] 2023-11-26T07:13:34.230 INFO:teuthology.orchestra.run.smithi188.stdout:Get:85 https://4.chacra.ceph.com/r/ceph/rishabh-mgr-vol-misc/44c2237fe77713214148c1fd622c872f346f1b99/ubuntu/jammy/flavors/default jammy/main amd64 python3-ceph-argparse all 18.0.0-7484-g44c2237f-1jammy [32.8 kB] 2023-11-26T07:13:34.231 INFO:teuthology.orchestra.run.smithi188.stdout:Get:86 https://4.chacra.ceph.com/r/ceph/rishabh-mgr-vol-misc/44c2237fe77713214148c1fd622c872f346f1b99/ubuntu/jammy/flavors/default jammy/main amd64 python3-cephfs amd64 18.0.0-7484-g44c2237f-1jammy [180 kB] 2023-11-26T07:13:34.292 INFO:teuthology.orchestra.run.smithi188.stdout:Get:87 https://4.chacra.ceph.com/r/ceph/rishabh-mgr-vol-misc/44c2237fe77713214148c1fd622c872f346f1b99/ubuntu/jammy/flavors/default jammy/main amd64 python3-ceph-common all 18.0.0-7484-g44c2237f-1jammy [63.5 kB] 2023-11-26T07:13:34.347 INFO:teuthology.orchestra.run.smithi188.stdout:Get:88 https://4.chacra.ceph.com/r/ceph/rishabh-mgr-vol-misc/44c2237fe77713214148c1fd622c872f346f1b99/ubuntu/jammy/flavors/default jammy/main amd64 python3-rbd amd64 18.0.0-7484-g44c2237f-1jammy [330 kB] 2023-11-26T07:13:34.415 INFO:teuthology.orchestra.run.smithi188.stdout:Get:89 https://4.chacra.ceph.com/r/ceph/rishabh-mgr-vol-misc/44c2237fe77713214148c1fd622c872f346f1b99/ubuntu/jammy/flavors/default jammy/main amd64 librgw2 amd64 18.0.0-7484-g44c2237f-1jammy [6,230 kB] 2023-11-26T07:13:34.857 INFO:teuthology.orchestra.run.smithi143.stdout:Get:13 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-pyperclip all 1.8.2-2 [11.5 kB] 2023-11-26T07:13:34.877 INFO:teuthology.orchestra.run.smithi143.stdout:Get:14 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-wcwidth all 0.2.5+dfsg1-1 [21.9 kB] 2023-11-26T07:13:34.888 INFO:teuthology.orchestra.run.smithi143.stdout:Get:15 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-cmd2 all 2.3.3-1 [124 kB] 2023-11-26T07:13:34.937 INFO:teuthology.orchestra.run.smithi143.stdout:Get:16 http://archive.ubuntu.com/ubuntu jammy/main amd64 liblttng-ust1 amd64 2.13.1-1ubuntu1 [190 kB] 2023-11-26T07:13:34.961 INFO:teuthology.orchestra.run.smithi143.stdout:Get:17 http://archive.ubuntu.com/ubuntu jammy/universe amd64 libdouble-conversion3 amd64 3.1.7-4 [39.0 kB] 2023-11-26T07:13:34.964 INFO:teuthology.orchestra.run.smithi143.stdout:Get:18 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 libpcre2-16-0 amd64 10.39-3ubuntu0.1 [203 kB] 2023-11-26T07:13:34.977 INFO:teuthology.orchestra.run.smithi143.stdout:Get:19 http://archive.ubuntu.com/ubuntu jammy-updates/universe amd64 libqt5core5a amd64 5.15.3+dfsg-2ubuntu0.2 [2,006 kB] 2023-11-26T07:13:35.021 INFO:teuthology.orchestra.run.smithi143.stdout:Get:20 http://archive.ubuntu.com/ubuntu jammy-updates/universe amd64 libqt5dbus5 amd64 5.15.3+dfsg-2ubuntu0.2 [222 kB] 2023-11-26T07:13:35.023 INFO:teuthology.orchestra.run.smithi143.stdout:Get:21 http://archive.ubuntu.com/ubuntu jammy-updates/universe amd64 libqt5network5 amd64 5.15.3+dfsg-2ubuntu0.2 [731 kB] 2023-11-26T07:13:35.031 INFO:teuthology.orchestra.run.smithi143.stdout:Get:22 http://archive.ubuntu.com/ubuntu jammy/universe amd64 libthrift-0.16.0 amd64 0.16.0-2 [267 kB] 2023-11-26T07:13:35.033 INFO:teuthology.orchestra.run.smithi143.stdout:Get:23 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-prettytable all 2.5.0-2 [31.3 kB] 2023-11-26T07:13:35.034 INFO:teuthology.orchestra.run.smithi143.stdout:Get:24 http://archive.ubuntu.com/ubuntu jammy/universe amd64 librdkafka1 amd64 1.8.0-1build1 [633 kB] 2023-11-26T07:13:35.040 INFO:teuthology.orchestra.run.smithi143.stdout:Get:25 http://archive.ubuntu.com/ubuntu jammy/main amd64 liboath0 amd64 2.6.7-3build1 [41.1 kB] 2023-11-26T07:13:35.040 INFO:teuthology.orchestra.run.smithi143.stdout:Get:26 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-jaraco.functools all 3.4.0-2 [9,030 B] 2023-11-26T07:13:35.041 INFO:teuthology.orchestra.run.smithi143.stdout:Get:27 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 python3-cheroot all 8.5.2+ds1-1ubuntu3.1 [71.1 kB] 2023-11-26T07:13:35.042 INFO:teuthology.orchestra.run.smithi143.stdout:Get:28 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-jaraco.classes all 3.2.1-3 [6,452 B] 2023-11-26T07:13:35.042 INFO:teuthology.orchestra.run.smithi143.stdout:Get:29 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-jaraco.text all 3.6.0-2 [8,716 B] 2023-11-26T07:13:35.062 INFO:teuthology.orchestra.run.smithi143.stdout:Get:30 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-jaraco.collections all 3.4.0-2 [11.4 kB] 2023-11-26T07:13:35.062 INFO:teuthology.orchestra.run.smithi143.stdout:Get:31 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-tempora all 4.1.2-1 [14.8 kB] 2023-11-26T07:13:35.063 INFO:teuthology.orchestra.run.smithi143.stdout:Get:32 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-portend all 3.0.0-1 [7,240 B] 2023-11-26T07:13:35.063 INFO:teuthology.orchestra.run.smithi143.stdout:Get:33 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-zc.lockfile all 2.0-1 [8,980 B] 2023-11-26T07:13:35.064 INFO:teuthology.orchestra.run.smithi143.stdout:Get:34 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-cherrypy3 all 18.6.1-4 [208 kB] 2023-11-26T07:13:35.066 INFO:teuthology.orchestra.run.smithi143.stdout:Get:35 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-natsort all 8.0.2-1 [35.3 kB] 2023-11-26T07:13:35.067 INFO:teuthology.orchestra.run.smithi143.stdout:Get:36 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-logutils all 0.3.3-8 [17.6 kB] 2023-11-26T07:13:35.067 INFO:teuthology.orchestra.run.smithi143.stdout:Get:37 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-markupsafe amd64 2.0.1-2build1 [12.7 kB] 2023-11-26T07:13:35.067 INFO:teuthology.orchestra.run.smithi143.stdout:Get:38 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 python3-mako all 1.1.3+ds1-2ubuntu0.1 [60.5 kB] 2023-11-26T07:13:35.081 INFO:teuthology.orchestra.run.smithi143.stdout:Get:39 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-simplegeneric all 0.8.1-3 [11.3 kB] 2023-11-26T07:13:35.088 INFO:teuthology.orchestra.run.smithi143.stdout:Get:40 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-singledispatch all 3.4.0.3-3 [7,320 B] 2023-11-26T07:13:35.088 INFO:teuthology.orchestra.run.smithi143.stdout:Get:41 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-webob all 1:1.8.6-1.1 [85.2 kB] 2023-11-26T07:13:35.089 INFO:teuthology.orchestra.run.smithi143.stdout:Get:42 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-waitress all 1.4.4-1.1ubuntu1 [47.3 kB] 2023-11-26T07:13:35.090 INFO:teuthology.orchestra.run.smithi143.stdout:Get:43 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-tempita all 0.5.2-6ubuntu1 [15.1 kB] 2023-11-26T07:13:35.090 INFO:teuthology.orchestra.run.smithi143.stdout:Get:44 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-paste all 3.5.0+dfsg1-1 [456 kB] 2023-11-26T07:13:35.095 INFO:teuthology.orchestra.run.smithi143.stdout:Get:45 http://archive.ubuntu.com/ubuntu jammy/main amd64 python-pastedeploy-tpl all 2.1.1-1 [4,892 B] 2023-11-26T07:13:35.095 INFO:teuthology.orchestra.run.smithi143.stdout:Get:46 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-pastedeploy all 2.1.1-1 [26.6 kB] 2023-11-26T07:13:35.096 INFO:teuthology.orchestra.run.smithi143.stdout:Get:47 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-webtest all 2.0.35-1 [28.5 kB] 2023-11-26T07:13:35.101 INFO:teuthology.orchestra.run.smithi143.stdout:Get:48 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-pecan all 1.3.3-4ubuntu2 [87.3 kB] 2023-11-26T07:13:35.116 INFO:teuthology.orchestra.run.smithi143.stdout:Get:49 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-certifi all 2020.6.20-1 [150 kB] 2023-11-26T07:13:35.117 INFO:teuthology.orchestra.run.smithi143.stdout:Get:50 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-idna all 3.3-1 [49.3 kB] 2023-11-26T07:13:35.118 INFO:teuthology.orchestra.run.smithi143.stdout:Get:51 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 python3-urllib3 all 1.26.5-1~exp1ubuntu0.1 [98.2 kB] 2023-11-26T07:13:35.119 INFO:teuthology.orchestra.run.smithi143.stdout:Get:52 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 python3-requests all 2.25.1+dfsg-2ubuntu0.1 [48.8 kB] 2023-11-26T07:13:35.120 INFO:teuthology.orchestra.run.smithi143.stdout:Get:53 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 python3-werkzeug all 2.0.2+dfsg1-1ubuntu0.22.04.1 [181 kB] 2023-11-26T07:13:35.122 INFO:teuthology.orchestra.run.smithi143.stdout:Get:54 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-openssl all 21.0.0-1 [45.2 kB] 2023-11-26T07:13:35.122 INFO:teuthology.orchestra.run.smithi143.stdout:Get:55 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-asyncssh all 2.5.0-1 [185 kB] 2023-11-26T07:13:35.125 INFO:teuthology.orchestra.run.smithi143.stdout:Get:56 http://archive.ubuntu.com/ubuntu jammy/main amd64 python-babel-localedata all 2.8.0+dfsg.1-7 [4,982 kB] 2023-11-26T07:13:35.175 INFO:teuthology.orchestra.run.smithi143.stdout:Get:57 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-babel all 2.8.0+dfsg.1-7 [85.1 kB] 2023-11-26T07:13:35.176 INFO:teuthology.orchestra.run.smithi143.stdout:Get:58 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-jinja2 all 3.0.3-1 [108 kB] 2023-11-26T07:13:35.177 INFO:teuthology.orchestra.run.smithi143.stdout:Get:59 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-bcrypt amd64 3.2.0-1build1 [32.7 kB] 2023-11-26T07:13:35.177 INFO:teuthology.orchestra.run.smithi143.stdout:Get:60 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-repoze.lru all 0.7-2 [12.1 kB] 2023-11-26T07:13:35.178 INFO:teuthology.orchestra.run.smithi143.stdout:Get:61 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-routes all 2.5.1-1ubuntu1 [89.0 kB] 2023-11-26T07:13:35.179 INFO:teuthology.orchestra.run.smithi143.stdout:Get:62 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-sklearn-lib amd64 0.23.2-5ubuntu6 [2,058 kB] 2023-11-26T07:13:35.201 INFO:teuthology.orchestra.run.smithi143.stdout:Get:63 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-joblib all 0.17.0-4ubuntu1 [204 kB] 2023-11-26T07:13:35.203 INFO:teuthology.orchestra.run.smithi143.stdout:Get:64 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-threadpoolctl all 3.1.0-1 [21.3 kB] 2023-11-26T07:13:35.203 INFO:teuthology.orchestra.run.smithi143.stdout:Get:65 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-sklearn all 0.23.2-5ubuntu6 [1,829 kB] 2023-11-26T07:13:35.225 INFO:teuthology.orchestra.run.smithi143.stdout:Get:66 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-cachetools all 5.0.0-1 [9,722 B] 2023-11-26T07:13:35.226 INFO:teuthology.orchestra.run.smithi143.stdout:Get:67 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-pyasn1 all 0.4.8-1 [50.9 kB] 2023-11-26T07:13:35.226 INFO:teuthology.orchestra.run.smithi143.stdout:Get:68 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-pyasn1-modules all 0.2.1-1 [33.0 kB] 2023-11-26T07:13:35.227 INFO:teuthology.orchestra.run.smithi143.stdout:Get:69 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-rsa all 4.8-1 [28.4 kB] 2023-11-26T07:13:35.227 INFO:teuthology.orchestra.run.smithi143.stdout:Get:70 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-google-auth all 1.5.1-3 [35.7 kB] 2023-11-26T07:13:35.228 INFO:teuthology.orchestra.run.smithi143.stdout:Get:71 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-requests-oauthlib all 1.3.0+ds-0.1 [18.7 kB] 2023-11-26T07:13:35.243 INFO:teuthology.orchestra.run.smithi143.stdout:Get:72 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-websocket all 1.2.3-1 [34.7 kB] 2023-11-26T07:13:35.244 INFO:teuthology.orchestra.run.smithi143.stdout:Get:73 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-kubernetes all 12.0.1-1ubuntu1 [353 kB] 2023-11-26T07:13:35.247 INFO:teuthology.orchestra.run.smithi143.stdout:Get:74 http://archive.ubuntu.com/ubuntu jammy/main amd64 libonig5 amd64 6.9.7.1-2build1 [172 kB] 2023-11-26T07:13:35.263 INFO:teuthology.orchestra.run.smithi143.stdout:Get:75 http://archive.ubuntu.com/ubuntu jammy/main amd64 libjq1 amd64 1.6-2.1ubuntu3 [133 kB] 2023-11-26T07:13:35.264 INFO:teuthology.orchestra.run.smithi143.stdout:Get:76 http://archive.ubuntu.com/ubuntu jammy/main amd64 jq amd64 1.6-2.1ubuntu3 [52.5 kB] 2023-11-26T07:13:35.265 INFO:teuthology.orchestra.run.smithi143.stdout:Get:77 http://archive.ubuntu.com/ubuntu jammy/main amd64 socat amd64 1.7.4.1-3ubuntu4 [349 kB] 2023-11-26T07:13:35.268 INFO:teuthology.orchestra.run.smithi143.stdout:Get:78 http://archive.ubuntu.com/ubuntu jammy/universe amd64 xmlstarlet amd64 1.6.1-2.1 [265 kB] 2023-11-26T07:13:35.271 INFO:teuthology.orchestra.run.smithi143.stdout:Get:79 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-colorama all 0.4.4-1 [24.5 kB] 2023-11-26T07:13:35.271 INFO:teuthology.orchestra.run.smithi143.stdout:Get:80 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 nvme-cli amd64 1.16-3ubuntu0.1 [474 kB] 2023-11-26T07:13:35.276 INFO:teuthology.orchestra.run.smithi143.stdout:Get:81 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python-asyncssh-doc all 2.5.0-1 [318 kB] 2023-11-26T07:13:35.282 INFO:teuthology.orchestra.run.smithi143.stdout:Get:82 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-iniconfig all 1.1.1-2 [6,024 B] 2023-11-26T07:13:35.282 INFO:teuthology.orchestra.run.smithi143.stdout:Get:83 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-pastescript all 2.0.2-4 [54.6 kB] 2023-11-26T07:13:35.283 INFO:teuthology.orchestra.run.smithi143.stdout:Get:84 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-pluggy all 0.13.0-7.1 [19.0 kB] 2023-11-26T07:13:35.302 INFO:teuthology.orchestra.run.smithi143.stdout:Get:85 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-psutil amd64 5.9.0-1build1 [158 kB] 2023-11-26T07:13:35.303 INFO:teuthology.orchestra.run.smithi143.stdout:Get:86 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-py all 1.10.0-1 [71.9 kB] 2023-11-26T07:13:35.304 INFO:teuthology.orchestra.run.smithi143.stdout:Get:87 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-pygments all 2.11.2+dfsg-2 [750 kB] 2023-11-26T07:13:35.311 INFO:teuthology.orchestra.run.smithi143.stdout:Get:88 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-pyinotify all 0.9.6-1.3 [24.8 kB] 2023-11-26T07:13:35.312 INFO:teuthology.orchestra.run.smithi143.stdout:Get:89 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-toml all 0.10.2-1 [16.5 kB] 2023-11-26T07:13:35.312 INFO:teuthology.orchestra.run.smithi143.stdout:Get:90 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-pytest all 6.2.5-1ubuntu2 [214 kB] 2023-11-26T07:13:35.314 INFO:teuthology.orchestra.run.smithi143.stdout:Get:91 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-simplejson amd64 3.17.6-1build1 [54.7 kB] 2023-11-26T07:13:35.324 INFO:teuthology.orchestra.run.smithi143.stdout:Get:92 http://archive.ubuntu.com/ubuntu jammy/universe amd64 qttranslations5-l10n all 5.15.3-1 [1,983 kB] 2023-11-26T07:13:36.045 INFO:teuthology.orchestra.run.smithi188.stdout:Get:90 https://4.chacra.ceph.com/r/ceph/rishabh-mgr-vol-misc/44c2237fe77713214148c1fd622c872f346f1b99/ubuntu/jammy/flavors/default jammy/main amd64 python3-rgw amd64 18.0.0-7484-g44c2237f-1jammy [112 kB] 2023-11-26T07:13:36.096 INFO:teuthology.orchestra.run.smithi188.stdout:Get:91 https://4.chacra.ceph.com/r/ceph/rishabh-mgr-vol-misc/44c2237fe77713214148c1fd622c872f346f1b99/ubuntu/jammy/flavors/default jammy/main amd64 libradosstriper1 amd64 18.0.0-7484-g44c2237f-1jammy [439 kB] 2023-11-26T07:13:36.186 INFO:teuthology.orchestra.run.smithi188.stdout:Get:92 https://4.chacra.ceph.com/r/ceph/rishabh-mgr-vol-misc/44c2237fe77713214148c1fd622c872f346f1b99/ubuntu/jammy/flavors/default jammy/main amd64 ceph-common amd64 18.0.0-7484-g44c2237f-1jammy [22.6 MB] 2023-11-26T07:13:36.835 INFO:teuthology.orchestra.run.smithi143.stdout:Get:93 https://4.chacra.ceph.com/r/ceph/rishabh-mgr-vol-misc/44c2237fe77713214148c1fd622c872f346f1b99/ubuntu/jammy/flavors/default jammy/main amd64 ceph-base amd64 18.0.0-7484-g44c2237f-1jammy [6,900 kB] 2023-11-26T07:13:37.656 INFO:teuthology.orchestra.run.smithi143.stdout:Get:94 https://4.chacra.ceph.com/r/ceph/rishabh-mgr-vol-misc/44c2237fe77713214148c1fd622c872f346f1b99/ubuntu/jammy/flavors/default jammy/main amd64 ceph-mgr-modules-core all 18.0.0-7484-g44c2237f-1jammy [238 kB] 2023-11-26T07:13:37.694 INFO:teuthology.orchestra.run.smithi143.stdout:Get:95 https://4.chacra.ceph.com/r/ceph/rishabh-mgr-vol-misc/44c2237fe77713214148c1fd622c872f346f1b99/ubuntu/jammy/flavors/default jammy/main amd64 libsqlite3-mod-ceph amd64 18.0.0-7484-g44c2237f-1jammy [124 kB] 2023-11-26T07:13:37.706 INFO:teuthology.orchestra.run.smithi143.stdout:Get:96 https://4.chacra.ceph.com/r/ceph/rishabh-mgr-vol-misc/44c2237fe77713214148c1fd622c872f346f1b99/ubuntu/jammy/flavors/default jammy/main amd64 ceph-mgr amd64 18.0.0-7484-g44c2237f-1jammy [1,496 kB] 2023-11-26T07:13:37.886 INFO:teuthology.orchestra.run.smithi143.stdout:Get:97 https://4.chacra.ceph.com/r/ceph/rishabh-mgr-vol-misc/44c2237fe77713214148c1fd622c872f346f1b99/ubuntu/jammy/flavors/default jammy/main amd64 ceph-mon amd64 18.0.0-7484-g44c2237f-1jammy [9,955 kB] 2023-11-26T07:13:39.129 INFO:teuthology.orchestra.run.smithi143.stdout:Get:98 https://4.chacra.ceph.com/r/ceph/rishabh-mgr-vol-misc/44c2237fe77713214148c1fd622c872f346f1b99/ubuntu/jammy/flavors/default jammy/main amd64 ceph-osd amd64 18.0.0-7484-g44c2237f-1jammy [31.8 MB] 2023-11-26T07:13:43.584 INFO:teuthology.orchestra.run.smithi188.stdout:Get:93 https://4.chacra.ceph.com/r/ceph/rishabh-mgr-vol-misc/44c2237fe77713214148c1fd622c872f346f1b99/ubuntu/jammy/flavors/default jammy/main amd64 ceph-base amd64 18.0.0-7484-g44c2237f-1jammy [6,900 kB] 2023-11-26T07:13:44.738 INFO:teuthology.orchestra.run.smithi143.stdout:Get:99 https://4.chacra.ceph.com/r/ceph/rishabh-mgr-vol-misc/44c2237fe77713214148c1fd622c872f346f1b99/ubuntu/jammy/flavors/default jammy/main amd64 ceph amd64 18.0.0-7484-g44c2237f-1jammy [14.0 kB] 2023-11-26T07:13:44.739 INFO:teuthology.orchestra.run.smithi143.stdout:Get:100 https://4.chacra.ceph.com/r/ceph/rishabh-mgr-vol-misc/44c2237fe77713214148c1fd622c872f346f1b99/ubuntu/jammy/flavors/default jammy/main amd64 ceph-fuse amd64 18.0.0-7484-g44c2237f-1jammy [839 kB] 2023-11-26T07:13:44.853 INFO:teuthology.orchestra.run.smithi143.stdout:Get:101 https://4.chacra.ceph.com/r/ceph/rishabh-mgr-vol-misc/44c2237fe77713214148c1fd622c872f346f1b99/ubuntu/jammy/flavors/default jammy/main amd64 ceph-mds amd64 18.0.0-7484-g44c2237f-1jammy [2,144 kB] 2023-11-26T07:13:45.242 INFO:teuthology.orchestra.run.smithi143.stdout:Get:102 https://4.chacra.ceph.com/r/ceph/rishabh-mgr-vol-misc/44c2237fe77713214148c1fd622c872f346f1b99/ubuntu/jammy/flavors/default jammy/main amd64 cephadm amd64 18.0.0-7484-g44c2237f-1jammy [805 kB] 2023-11-26T07:13:45.390 INFO:teuthology.orchestra.run.smithi143.stdout:Get:103 https://4.chacra.ceph.com/r/ceph/rishabh-mgr-vol-misc/44c2237fe77713214148c1fd622c872f346f1b99/ubuntu/jammy/flavors/default jammy/main amd64 ceph-mgr-cephadm all 18.0.0-7484-g44c2237f-1jammy [139 kB] 2023-11-26T07:13:45.416 INFO:teuthology.orchestra.run.smithi143.stdout:Get:104 https://4.chacra.ceph.com/r/ceph/rishabh-mgr-vol-misc/44c2237fe77713214148c1fd622c872f346f1b99/ubuntu/jammy/flavors/default jammy/main amd64 ceph-mgr-dashboard all 18.0.0-7484-g44c2237f-1jammy [1,900 kB] 2023-11-26T07:13:45.806 INFO:teuthology.orchestra.run.smithi143.stdout:Get:105 https://4.chacra.ceph.com/r/ceph/rishabh-mgr-vol-misc/44c2237fe77713214148c1fd622c872f346f1b99/ubuntu/jammy/flavors/default jammy/main amd64 ceph-mgr-diskprediction-local all 18.0.0-7484-g44c2237f-1jammy [8,625 kB] 2023-11-26T07:13:45.938 INFO:teuthology.orchestra.run.smithi188.stdout:Get:94 https://4.chacra.ceph.com/r/ceph/rishabh-mgr-vol-misc/44c2237fe77713214148c1fd622c872f346f1b99/ubuntu/jammy/flavors/default jammy/main amd64 ceph-mgr-modules-core all 18.0.0-7484-g44c2237f-1jammy [238 kB] 2023-11-26T07:13:46.037 INFO:teuthology.orchestra.run.smithi188.stdout:Get:95 https://4.chacra.ceph.com/r/ceph/rishabh-mgr-vol-misc/44c2237fe77713214148c1fd622c872f346f1b99/ubuntu/jammy/flavors/default jammy/main amd64 libsqlite3-mod-ceph amd64 18.0.0-7484-g44c2237f-1jammy [124 kB] 2023-11-26T07:13:46.098 INFO:teuthology.orchestra.run.smithi188.stdout:Get:96 https://4.chacra.ceph.com/r/ceph/rishabh-mgr-vol-misc/44c2237fe77713214148c1fd622c872f346f1b99/ubuntu/jammy/flavors/default jammy/main amd64 ceph-mgr amd64 18.0.0-7484-g44c2237f-1jammy [1,496 kB] 2023-11-26T07:13:46.638 INFO:teuthology.orchestra.run.smithi188.stdout:Get:97 https://4.chacra.ceph.com/r/ceph/rishabh-mgr-vol-misc/44c2237fe77713214148c1fd622c872f346f1b99/ubuntu/jammy/flavors/default jammy/main amd64 ceph-mon amd64 18.0.0-7484-g44c2237f-1jammy [9,955 kB] 2023-11-26T07:13:47.711 INFO:teuthology.orchestra.run.smithi143.stdout:Get:106 https://4.chacra.ceph.com/r/ceph/rishabh-mgr-vol-misc/44c2237fe77713214148c1fd622c872f346f1b99/ubuntu/jammy/flavors/default jammy/main amd64 ceph-mgr-k8sevents all 18.0.0-7484-g44c2237f-1jammy [14.2 kB] 2023-11-26T07:13:47.712 INFO:teuthology.orchestra.run.smithi143.stdout:Get:107 https://4.chacra.ceph.com/r/ceph/rishabh-mgr-vol-misc/44c2237fe77713214148c1fd622c872f346f1b99/ubuntu/jammy/flavors/default jammy/main amd64 ceph-test amd64 18.0.0-7484-g44c2237f-1jammy [48.7 MB] 2023-11-26T07:13:50.104 INFO:teuthology.orchestra.run.smithi188.stdout:Get:98 https://4.chacra.ceph.com/r/ceph/rishabh-mgr-vol-misc/44c2237fe77713214148c1fd622c872f346f1b99/ubuntu/jammy/flavors/default jammy/main amd64 ceph-osd amd64 18.0.0-7484-g44c2237f-1jammy [31.8 MB] 2023-11-26T07:14:02.444 INFO:teuthology.orchestra.run.smithi143.stdout:Get:108 https://4.chacra.ceph.com/r/ceph/rishabh-mgr-vol-misc/44c2237fe77713214148c1fd622c872f346f1b99/ubuntu/jammy/flavors/default jammy/main amd64 ceph-volume all 18.0.0-7484-g44c2237f-1jammy [121 kB] 2023-11-26T07:14:02.452 INFO:teuthology.orchestra.run.smithi143.stdout:Get:109 https://4.chacra.ceph.com/r/ceph/rishabh-mgr-vol-misc/44c2237fe77713214148c1fd622c872f346f1b99/ubuntu/jammy/flavors/default jammy/main amd64 cephfs-mirror amd64 18.0.0-7484-g44c2237f-1jammy [225 kB] 2023-11-26T07:14:02.544 INFO:teuthology.orchestra.run.smithi143.stdout:Get:110 https://4.chacra.ceph.com/r/ceph/rishabh-mgr-vol-misc/44c2237fe77713214148c1fd622c872f346f1b99/ubuntu/jammy/flavors/default jammy/main amd64 cephfs-shell all 18.0.0-7484-g44c2237f-1jammy [34.0 kB] 2023-11-26T07:14:02.545 INFO:teuthology.orchestra.run.smithi143.stdout:Get:111 https://4.chacra.ceph.com/r/ceph/rishabh-mgr-vol-misc/44c2237fe77713214148c1fd622c872f346f1b99/ubuntu/jammy/flavors/default jammy/main amd64 cephfs-top all 18.0.0-7484-g44c2237f-1jammy [23.4 kB] 2023-11-26T07:14:02.547 INFO:teuthology.orchestra.run.smithi143.stdout:Get:112 https://4.chacra.ceph.com/r/ceph/rishabh-mgr-vol-misc/44c2237fe77713214148c1fd622c872f346f1b99/ubuntu/jammy/flavors/default jammy/main amd64 libcephfs-dev amd64 18.0.0-7484-g44c2237f-1jammy [38.3 kB] 2023-11-26T07:14:02.550 INFO:teuthology.orchestra.run.smithi143.stdout:Get:113 https://4.chacra.ceph.com/r/ceph/rishabh-mgr-vol-misc/44c2237fe77713214148c1fd622c872f346f1b99/ubuntu/jammy/flavors/default jammy/main amd64 radosgw amd64 18.0.0-7484-g44c2237f-1jammy [12.3 MB] 2023-11-26T07:14:06.950 INFO:teuthology.orchestra.run.smithi143.stdout:Get:114 https://4.chacra.ceph.com/r/ceph/rishabh-mgr-vol-misc/44c2237fe77713214148c1fd622c872f346f1b99/ubuntu/jammy/flavors/default jammy/main amd64 rbd-fuse amd64 18.0.0-7484-g44c2237f-1jammy [91.2 kB] 2023-11-26T07:14:07.378 INFO:teuthology.orchestra.run.smithi188.stdout:Get:99 https://4.chacra.ceph.com/r/ceph/rishabh-mgr-vol-misc/44c2237fe77713214148c1fd622c872f346f1b99/ubuntu/jammy/flavors/default jammy/main amd64 ceph amd64 18.0.0-7484-g44c2237f-1jammy [14.0 kB] 2023-11-26T07:14:07.378 INFO:teuthology.orchestra.run.smithi188.stdout:Get:100 https://4.chacra.ceph.com/r/ceph/rishabh-mgr-vol-misc/44c2237fe77713214148c1fd622c872f346f1b99/ubuntu/jammy/flavors/default jammy/main amd64 ceph-fuse amd64 18.0.0-7484-g44c2237f-1jammy [839 kB] 2023-11-26T07:14:07.554 INFO:teuthology.orchestra.run.smithi143.stdout:Fetched 186 MB in 36s (5,135 kB/s) 2023-11-26T07:14:07.697 INFO:teuthology.orchestra.run.smithi143.stdout:Selecting previously unselected package python3-pyperclip. 2023-11-26T07:14:07.766 INFO:teuthology.orchestra.run.smithi143.stdout:(Reading database ... (Reading database ... 5% (Reading database ... 10% (Reading database ... 15% (Reading database ... 20% (Reading database ... 25% (Reading database ... 30% (Reading database ... 35% (Reading database ... 40% (Reading database ... 45% (Reading database ... 50% (Reading database ... 55% (Reading database ... 60% (Reading database ... 65% (Reading database ... 70% (Reading database ... 75% (Reading database ... 80% (Reading database ... 85% (Reading database ... 90% (Reading database ... 95% (Reading database ... 100% (Reading database ... 132930 files and directories currently installed.) 2023-11-26T07:14:07.770 INFO:teuthology.orchestra.run.smithi143.stdout:Preparing to unpack .../000-python3-pyperclip_1.8.2-2_all.deb ... 2023-11-26T07:14:07.813 INFO:teuthology.orchestra.run.smithi143.stdout:Unpacking python3-pyperclip (1.8.2-2) ... 2023-11-26T07:14:07.841 INFO:teuthology.orchestra.run.smithi188.stdout:Get:101 https://4.chacra.ceph.com/r/ceph/rishabh-mgr-vol-misc/44c2237fe77713214148c1fd622c872f346f1b99/ubuntu/jammy/flavors/default jammy/main amd64 ceph-mds amd64 18.0.0-7484-g44c2237f-1jammy [2,144 kB] 2023-11-26T07:14:08.124 INFO:teuthology.orchestra.run.smithi143.stdout:Selecting previously unselected package python3-wcwidth. 2023-11-26T07:14:08.137 INFO:teuthology.orchestra.run.smithi143.stdout:Preparing to unpack .../001-python3-wcwidth_0.2.5+dfsg1-1_all.deb ... 2023-11-26T07:14:08.173 INFO:teuthology.orchestra.run.smithi143.stdout:Unpacking python3-wcwidth (0.2.5+dfsg1-1) ... 2023-11-26T07:14:08.518 INFO:teuthology.orchestra.run.smithi143.stdout:Selecting previously unselected package python3-cmd2. 2023-11-26T07:14:08.531 INFO:teuthology.orchestra.run.smithi143.stdout:Preparing to unpack .../002-python3-cmd2_2.3.3-1_all.deb ... 2023-11-26T07:14:08.576 INFO:teuthology.orchestra.run.smithi143.stdout:Unpacking python3-cmd2 (2.3.3-1) ... 2023-11-26T07:14:08.887 INFO:teuthology.orchestra.run.smithi143.stdout:Selecting previously unselected package liblttng-ust1:amd64. 2023-11-26T07:14:08.900 INFO:teuthology.orchestra.run.smithi143.stdout:Preparing to unpack .../003-liblttng-ust1_2.13.1-1ubuntu1_amd64.deb ... 2023-11-26T07:14:08.987 INFO:teuthology.orchestra.run.smithi143.stdout:Unpacking liblttng-ust1:amd64 (2.13.1-1ubuntu1) ... 2023-11-26T07:14:09.077 INFO:teuthology.orchestra.run.smithi188.stdout:Get:102 https://4.chacra.ceph.com/r/ceph/rishabh-mgr-vol-misc/44c2237fe77713214148c1fd622c872f346f1b99/ubuntu/jammy/flavors/default jammy/main amd64 cephadm amd64 18.0.0-7484-g44c2237f-1jammy [805 kB] 2023-11-26T07:14:09.307 INFO:teuthology.orchestra.run.smithi143.stdout:Selecting previously unselected package libdouble-conversion3:amd64. 2023-11-26T07:14:09.320 INFO:teuthology.orchestra.run.smithi143.stdout:Preparing to unpack .../004-libdouble-conversion3_3.1.7-4_amd64.deb ... 2023-11-26T07:14:09.349 INFO:teuthology.orchestra.run.smithi143.stdout:Unpacking libdouble-conversion3:amd64 (3.1.7-4) ... 2023-11-26T07:14:09.506 INFO:teuthology.orchestra.run.smithi188.stdout:Get:103 https://4.chacra.ceph.com/r/ceph/rishabh-mgr-vol-misc/44c2237fe77713214148c1fd622c872f346f1b99/ubuntu/jammy/flavors/default jammy/main amd64 ceph-mgr-cephadm all 18.0.0-7484-g44c2237f-1jammy [139 kB] 2023-11-26T07:14:09.571 INFO:teuthology.orchestra.run.smithi188.stdout:Get:104 https://4.chacra.ceph.com/r/ceph/rishabh-mgr-vol-misc/44c2237fe77713214148c1fd622c872f346f1b99/ubuntu/jammy/flavors/default jammy/main amd64 ceph-mgr-dashboard all 18.0.0-7484-g44c2237f-1jammy [1,900 kB] 2023-11-26T07:14:09.677 INFO:teuthology.orchestra.run.smithi143.stdout:Selecting previously unselected package libpcre2-16-0:amd64. 2023-11-26T07:14:09.690 INFO:teuthology.orchestra.run.smithi143.stdout:Preparing to unpack .../005-libpcre2-16-0_10.39-3ubuntu0.1_amd64.deb ... 2023-11-26T07:14:09.735 INFO:teuthology.orchestra.run.smithi143.stdout:Unpacking libpcre2-16-0:amd64 (10.39-3ubuntu0.1) ... 2023-11-26T07:14:10.196 INFO:teuthology.orchestra.run.smithi143.stdout:Selecting previously unselected package libqt5core5a:amd64. 2023-11-26T07:14:10.210 INFO:teuthology.orchestra.run.smithi143.stdout:Preparing to unpack .../006-libqt5core5a_5.15.3+dfsg-2ubuntu0.2_amd64.deb ... 2023-11-26T07:14:10.262 INFO:teuthology.orchestra.run.smithi143.stdout:Unpacking libqt5core5a:amd64 (5.15.3+dfsg-2ubuntu0.2) ... 2023-11-26T07:14:10.477 INFO:teuthology.orchestra.run.smithi188.stdout:Get:105 https://4.chacra.ceph.com/r/ceph/rishabh-mgr-vol-misc/44c2237fe77713214148c1fd622c872f346f1b99/ubuntu/jammy/flavors/default jammy/main amd64 ceph-mgr-diskprediction-local all 18.0.0-7484-g44c2237f-1jammy [8,625 kB] 2023-11-26T07:14:10.608 INFO:teuthology.orchestra.run.smithi143.stdout:Selecting previously unselected package libqt5dbus5:amd64. 2023-11-26T07:14:10.621 INFO:teuthology.orchestra.run.smithi143.stdout:Preparing to unpack .../007-libqt5dbus5_5.15.3+dfsg-2ubuntu0.2_amd64.deb ... 2023-11-26T07:14:10.649 INFO:teuthology.orchestra.run.smithi143.stdout:Unpacking libqt5dbus5:amd64 (5.15.3+dfsg-2ubuntu0.2) ... 2023-11-26T07:14:10.977 INFO:teuthology.orchestra.run.smithi143.stdout:Selecting previously unselected package libqt5network5:amd64. 2023-11-26T07:14:10.990 INFO:teuthology.orchestra.run.smithi143.stdout:Preparing to unpack .../008-libqt5network5_5.15.3+dfsg-2ubuntu0.2_amd64.deb ... 2023-11-26T07:14:11.027 INFO:teuthology.orchestra.run.smithi143.stdout:Unpacking libqt5network5:amd64 (5.15.3+dfsg-2ubuntu0.2) ... 2023-11-26T07:14:11.405 INFO:teuthology.orchestra.run.smithi143.stdout:Selecting previously unselected package libthrift-0.16.0:amd64. 2023-11-26T07:14:11.418 INFO:teuthology.orchestra.run.smithi143.stdout:Preparing to unpack .../009-libthrift-0.16.0_0.16.0-2_amd64.deb ... 2023-11-26T07:14:11.463 INFO:teuthology.orchestra.run.smithi143.stdout:Unpacking libthrift-0.16.0:amd64 (0.16.0-2) ... 2023-11-26T07:14:11.783 INFO:teuthology.orchestra.run.smithi143.stdout:Selecting previously unselected package librados2. 2023-11-26T07:14:11.796 INFO:teuthology.orchestra.run.smithi143.stdout:Preparing to unpack .../010-librados2_18.0.0-7484-g44c2237f-1jammy_amd64.deb ... 2023-11-26T07:14:11.841 INFO:teuthology.orchestra.run.smithi143.stdout:Unpacking librados2 (18.0.0-7484-g44c2237f-1jammy) ... 2023-11-26T07:14:12.277 INFO:teuthology.orchestra.run.smithi143.stdout:Selecting previously unselected package librbd1. 2023-11-26T07:14:12.290 INFO:teuthology.orchestra.run.smithi143.stdout:Preparing to unpack .../011-librbd1_18.0.0-7484-g44c2237f-1jammy_amd64.deb ... 2023-11-26T07:14:12.335 INFO:teuthology.orchestra.run.smithi143.stdout:Unpacking librbd1 (18.0.0-7484-g44c2237f-1jammy) ... 2023-11-26T07:14:12.737 INFO:teuthology.orchestra.run.smithi143.stdout:Selecting previously unselected package libcephfs2. 2023-11-26T07:14:12.750 INFO:teuthology.orchestra.run.smithi143.stdout:Preparing to unpack .../012-libcephfs2_18.0.0-7484-g44c2237f-1jammy_amd64.deb ... 2023-11-26T07:14:12.787 INFO:teuthology.orchestra.run.smithi143.stdout:Unpacking libcephfs2 (18.0.0-7484-g44c2237f-1jammy) ... 2023-11-26T07:14:13.039 INFO:teuthology.orchestra.run.smithi143.stdout:Selecting previously unselected package python3-rados. 2023-11-26T07:14:13.053 INFO:teuthology.orchestra.run.smithi143.stdout:Preparing to unpack .../013-python3-rados_18.0.0-7484-g44c2237f-1jammy_amd64.deb ... 2023-11-26T07:14:13.090 INFO:teuthology.orchestra.run.smithi143.stdout:Unpacking python3-rados (18.0.0-7484-g44c2237f-1jammy) ... 2023-11-26T07:14:13.575 INFO:teuthology.orchestra.run.smithi143.stdout:Selecting previously unselected package python3-ceph-argparse. 2023-11-26T07:14:13.589 INFO:teuthology.orchestra.run.smithi143.stdout:Preparing to unpack .../014-python3-ceph-argparse_18.0.0-7484-g44c2237f-1jammy_all.deb ... 2023-11-26T07:14:13.634 INFO:teuthology.orchestra.run.smithi143.stdout:Unpacking python3-ceph-argparse (18.0.0-7484-g44c2237f-1jammy) ... 2023-11-26T07:14:13.928 INFO:teuthology.orchestra.run.smithi143.stdout:Selecting previously unselected package python3-cephfs. 2023-11-26T07:14:13.942 INFO:teuthology.orchestra.run.smithi143.stdout:Preparing to unpack .../015-python3-cephfs_18.0.0-7484-g44c2237f-1jammy_amd64.deb ... 2023-11-26T07:14:13.995 INFO:teuthology.orchestra.run.smithi143.stdout:Unpacking python3-cephfs (18.0.0-7484-g44c2237f-1jammy) ... 2023-11-26T07:14:14.339 INFO:teuthology.orchestra.run.smithi143.stdout:Selecting previously unselected package python3-ceph-common. 2023-11-26T07:14:14.353 INFO:teuthology.orchestra.run.smithi143.stdout:Preparing to unpack .../016-python3-ceph-common_18.0.0-7484-g44c2237f-1jammy_all.deb ... 2023-11-26T07:14:14.388 INFO:teuthology.orchestra.run.smithi188.stdout:Get:106 https://4.chacra.ceph.com/r/ceph/rishabh-mgr-vol-misc/44c2237fe77713214148c1fd622c872f346f1b99/ubuntu/jammy/flavors/default jammy/main amd64 ceph-mgr-k8sevents all 18.0.0-7484-g44c2237f-1jammy [14.2 kB] 2023-11-26T07:14:14.391 INFO:teuthology.orchestra.run.smithi188.stdout:Get:107 https://4.chacra.ceph.com/r/ceph/rishabh-mgr-vol-misc/44c2237fe77713214148c1fd622c872f346f1b99/ubuntu/jammy/flavors/default jammy/main amd64 ceph-test amd64 18.0.0-7484-g44c2237f-1jammy [48.7 MB] 2023-11-26T07:14:14.397 INFO:teuthology.orchestra.run.smithi143.stdout:Unpacking python3-ceph-common (18.0.0-7484-g44c2237f-1jammy) ... 2023-11-26T07:14:14.758 INFO:teuthology.orchestra.run.smithi143.stdout:Selecting previously unselected package python3-prettytable. 2023-11-26T07:14:14.773 INFO:teuthology.orchestra.run.smithi143.stdout:Preparing to unpack .../017-python3-prettytable_2.5.0-2_all.deb ... 2023-11-26T07:14:14.825 INFO:teuthology.orchestra.run.smithi143.stdout:Unpacking python3-prettytable (2.5.0-2) ... 2023-11-26T07:14:15.135 INFO:teuthology.orchestra.run.smithi143.stdout:Selecting previously unselected package python3-rbd. 2023-11-26T07:14:15.150 INFO:teuthology.orchestra.run.smithi143.stdout:Preparing to unpack .../018-python3-rbd_18.0.0-7484-g44c2237f-1jammy_amd64.deb ... 2023-11-26T07:14:15.202 INFO:teuthology.orchestra.run.smithi143.stdout:Unpacking python3-rbd (18.0.0-7484-g44c2237f-1jammy) ... 2023-11-26T07:14:15.647 INFO:teuthology.orchestra.run.smithi143.stdout:Selecting previously unselected package librdkafka1:amd64. 2023-11-26T07:14:15.661 INFO:teuthology.orchestra.run.smithi143.stdout:Preparing to unpack .../019-librdkafka1_1.8.0-1build1_amd64.deb ... 2023-11-26T07:14:15.713 INFO:teuthology.orchestra.run.smithi143.stdout:Unpacking librdkafka1:amd64 (1.8.0-1build1) ... 2023-11-26T07:14:16.091 INFO:teuthology.orchestra.run.smithi143.stdout:Selecting previously unselected package librgw2. 2023-11-26T07:14:16.106 INFO:teuthology.orchestra.run.smithi143.stdout:Preparing to unpack .../020-librgw2_18.0.0-7484-g44c2237f-1jammy_amd64.deb ... 2023-11-26T07:14:16.149 INFO:teuthology.orchestra.run.smithi143.stdout:Unpacking librgw2 (18.0.0-7484-g44c2237f-1jammy) ... 2023-11-26T07:14:16.593 INFO:teuthology.orchestra.run.smithi143.stdout:Selecting previously unselected package python3-rgw. 2023-11-26T07:14:16.608 INFO:teuthology.orchestra.run.smithi143.stdout:Preparing to unpack .../021-python3-rgw_18.0.0-7484-g44c2237f-1jammy_amd64.deb ... 2023-11-26T07:14:16.651 INFO:teuthology.orchestra.run.smithi143.stdout:Unpacking python3-rgw (18.0.0-7484-g44c2237f-1jammy) ... 2023-11-26T07:14:17.004 INFO:teuthology.orchestra.run.smithi143.stdout:Selecting previously unselected package liboath0:amd64. 2023-11-26T07:14:17.019 INFO:teuthology.orchestra.run.smithi143.stdout:Preparing to unpack .../022-liboath0_2.6.7-3build1_amd64.deb ... 2023-11-26T07:14:17.062 INFO:teuthology.orchestra.run.smithi143.stdout:Unpacking liboath0:amd64 (2.6.7-3build1) ... 2023-11-26T07:14:17.365 INFO:teuthology.orchestra.run.smithi143.stdout:Selecting previously unselected package libradosstriper1. 2023-11-26T07:14:17.380 INFO:teuthology.orchestra.run.smithi143.stdout:Preparing to unpack .../023-libradosstriper1_18.0.0-7484-g44c2237f-1jammy_amd64.deb ... 2023-11-26T07:14:17.423 INFO:teuthology.orchestra.run.smithi143.stdout:Unpacking libradosstriper1 (18.0.0-7484-g44c2237f-1jammy) ... 2023-11-26T07:14:17.834 INFO:teuthology.orchestra.run.smithi143.stdout:Selecting previously unselected package ceph-common. 2023-11-26T07:14:17.849 INFO:teuthology.orchestra.run.smithi143.stdout:Preparing to unpack .../024-ceph-common_18.0.0-7484-g44c2237f-1jammy_amd64.deb ... 2023-11-26T07:14:17.892 INFO:teuthology.orchestra.run.smithi143.stdout:Unpacking ceph-common (18.0.0-7484-g44c2237f-1jammy) ... 2023-11-26T07:14:18.998 INFO:teuthology.orchestra.run.smithi143.stdout:Selecting previously unselected package ceph-base. 2023-11-26T07:14:19.013 INFO:teuthology.orchestra.run.smithi143.stdout:Preparing to unpack .../025-ceph-base_18.0.0-7484-g44c2237f-1jammy_amd64.deb ... 2023-11-26T07:14:19.065 INFO:teuthology.orchestra.run.smithi143.stdout:Unpacking ceph-base (18.0.0-7484-g44c2237f-1jammy) ... 2023-11-26T07:14:19.785 INFO:teuthology.orchestra.run.smithi143.stdout:Selecting previously unselected package python3-jaraco.functools. 2023-11-26T07:14:19.799 INFO:teuthology.orchestra.run.smithi143.stdout:Preparing to unpack .../026-python3-jaraco.functools_3.4.0-2_all.deb ... 2023-11-26T07:14:19.843 INFO:teuthology.orchestra.run.smithi143.stdout:Unpacking python3-jaraco.functools (3.4.0-2) ... 2023-11-26T07:14:20.129 INFO:teuthology.orchestra.run.smithi143.stdout:Selecting previously unselected package python3-cheroot. 2023-11-26T07:14:20.144 INFO:teuthology.orchestra.run.smithi143.stdout:Preparing to unpack .../027-python3-cheroot_8.5.2+ds1-1ubuntu3.1_all.deb ... 2023-11-26T07:14:20.187 INFO:teuthology.orchestra.run.smithi143.stdout:Unpacking python3-cheroot (8.5.2+ds1-1ubuntu3.1) ... 2023-11-26T07:14:20.499 INFO:teuthology.orchestra.run.smithi143.stdout:Selecting previously unselected package python3-jaraco.classes. 2023-11-26T07:14:20.514 INFO:teuthology.orchestra.run.smithi143.stdout:Preparing to unpack .../028-python3-jaraco.classes_3.2.1-3_all.deb ... 2023-11-26T07:14:20.549 INFO:teuthology.orchestra.run.smithi143.stdout:Unpacking python3-jaraco.classes (3.2.1-3) ... 2023-11-26T07:14:20.768 INFO:teuthology.orchestra.run.smithi143.stdout:Selecting previously unselected package python3-jaraco.text. 2023-11-26T07:14:20.783 INFO:teuthology.orchestra.run.smithi143.stdout:Preparing to unpack .../029-python3-jaraco.text_3.6.0-2_all.deb ... 2023-11-26T07:14:20.818 INFO:teuthology.orchestra.run.smithi143.stdout:Unpacking python3-jaraco.text (3.6.0-2) ... 2023-11-26T07:14:21.145 INFO:teuthology.orchestra.run.smithi143.stdout:Selecting previously unselected package python3-jaraco.collections. 2023-11-26T07:14:21.160 INFO:teuthology.orchestra.run.smithi143.stdout:Preparing to unpack .../030-python3-jaraco.collections_3.4.0-2_all.deb ... 2023-11-26T07:14:21.203 INFO:teuthology.orchestra.run.smithi143.stdout:Unpacking python3-jaraco.collections (3.4.0-2) ... 2023-11-26T07:14:21.498 INFO:teuthology.orchestra.run.smithi143.stdout:Selecting previously unselected package python3-tempora. 2023-11-26T07:14:21.513 INFO:teuthology.orchestra.run.smithi143.stdout:Preparing to unpack .../031-python3-tempora_4.1.2-1_all.deb ... 2023-11-26T07:14:21.556 INFO:teuthology.orchestra.run.smithi143.stdout:Unpacking python3-tempora (4.1.2-1) ... 2023-11-26T07:14:21.842 INFO:teuthology.orchestra.run.smithi143.stdout:Selecting previously unselected package python3-portend. 2023-11-26T07:14:21.857 INFO:teuthology.orchestra.run.smithi143.stdout:Preparing to unpack .../032-python3-portend_3.0.0-1_all.deb ... 2023-11-26T07:14:21.892 INFO:teuthology.orchestra.run.smithi143.stdout:Unpacking python3-portend (3.0.0-1) ... 2023-11-26T07:14:22.112 INFO:teuthology.orchestra.run.smithi143.stdout:Selecting previously unselected package python3-zc.lockfile. 2023-11-26T07:14:22.126 INFO:teuthology.orchestra.run.smithi143.stdout:Preparing to unpack .../033-python3-zc.lockfile_2.0-1_all.deb ... 2023-11-26T07:14:22.169 INFO:teuthology.orchestra.run.smithi143.stdout:Unpacking python3-zc.lockfile (2.0-1) ... 2023-11-26T07:14:22.506 INFO:teuthology.orchestra.run.smithi143.stdout:Selecting previously unselected package python3-cherrypy3. 2023-11-26T07:14:22.521 INFO:teuthology.orchestra.run.smithi143.stdout:Preparing to unpack .../034-python3-cherrypy3_18.6.1-4_all.deb ... 2023-11-26T07:14:22.564 INFO:teuthology.orchestra.run.smithi143.stdout:Unpacking python3-cherrypy3 (18.6.1-4) ... 2023-11-26T07:14:22.901 INFO:teuthology.orchestra.run.smithi143.stdout:Selecting previously unselected package python3-natsort. 2023-11-26T07:14:22.915 INFO:teuthology.orchestra.run.smithi143.stdout:Preparing to unpack .../035-python3-natsort_8.0.2-1_all.deb ... 2023-11-26T07:14:22.959 INFO:teuthology.orchestra.run.smithi143.stdout:Unpacking python3-natsort (8.0.2-1) ... 2023-11-26T07:14:23.270 INFO:teuthology.orchestra.run.smithi143.stdout:Selecting previously unselected package python3-logutils. 2023-11-26T07:14:23.285 INFO:teuthology.orchestra.run.smithi143.stdout:Preparing to unpack .../036-python3-logutils_0.3.3-8_all.deb ... 2023-11-26T07:14:23.328 INFO:teuthology.orchestra.run.smithi143.stdout:Unpacking python3-logutils (0.3.3-8) ... 2023-11-26T07:14:23.714 INFO:teuthology.orchestra.run.smithi143.stdout:Selecting previously unselected package python3-markupsafe. 2023-11-26T07:14:23.729 INFO:teuthology.orchestra.run.smithi143.stdout:Preparing to unpack .../037-python3-markupsafe_2.0.1-2build1_amd64.deb ... 2023-11-26T07:14:23.823 INFO:teuthology.orchestra.run.smithi143.stdout:Unpacking python3-markupsafe (2.0.1-2build1) ... 2023-11-26T07:14:24.209 INFO:teuthology.orchestra.run.smithi143.stdout:Selecting previously unselected package python3-mako. 2023-11-26T07:14:24.224 INFO:teuthology.orchestra.run.smithi143.stdout:Preparing to unpack .../038-python3-mako_1.1.3+ds1-2ubuntu0.1_all.deb ... 2023-11-26T07:14:24.275 INFO:teuthology.orchestra.run.smithi143.stdout:Unpacking python3-mako (1.1.3+ds1-2ubuntu0.1) ... 2023-11-26T07:14:24.678 INFO:teuthology.orchestra.run.smithi143.stdout:Selecting previously unselected package python3-simplegeneric. 2023-11-26T07:14:24.694 INFO:teuthology.orchestra.run.smithi143.stdout:Preparing to unpack .../039-python3-simplegeneric_0.8.1-3_all.deb ... 2023-11-26T07:14:24.745 INFO:teuthology.orchestra.run.smithi143.stdout:Unpacking python3-simplegeneric (0.8.1-3) ... 2023-11-26T07:14:25.081 INFO:teuthology.orchestra.run.smithi143.stdout:Selecting previously unselected package python3-singledispatch. 2023-11-26T07:14:25.096 INFO:teuthology.orchestra.run.smithi143.stdout:Preparing to unpack .../040-python3-singledispatch_3.4.0.3-3_all.deb ... 2023-11-26T07:14:25.164 INFO:teuthology.orchestra.run.smithi143.stdout:Unpacking python3-singledispatch (3.4.0.3-3) ... 2023-11-26T07:14:25.525 INFO:teuthology.orchestra.run.smithi143.stdout:Selecting previously unselected package python3-webob. 2023-11-26T07:14:25.540 INFO:teuthology.orchestra.run.smithi143.stdout:Preparing to unpack .../041-python3-webob_1%3a1.8.6-1.1_all.deb ... 2023-11-26T07:14:25.592 INFO:teuthology.orchestra.run.smithi143.stdout:Unpacking python3-webob (1:1.8.6-1.1) ... 2023-11-26T07:14:26.011 INFO:teuthology.orchestra.run.smithi143.stdout:Selecting previously unselected package python3-waitress. 2023-11-26T07:14:26.026 INFO:teuthology.orchestra.run.smithi143.stdout:Preparing to unpack .../042-python3-waitress_1.4.4-1.1ubuntu1_all.deb ... 2023-11-26T07:14:26.080 INFO:teuthology.orchestra.run.smithi143.stdout:Unpacking python3-waitress (1.4.4-1.1ubuntu1) ... 2023-11-26T07:14:26.489 INFO:teuthology.orchestra.run.smithi143.stdout:Selecting previously unselected package python3-tempita. 2023-11-26T07:14:26.504 INFO:teuthology.orchestra.run.smithi143.stdout:Preparing to unpack .../043-python3-tempita_0.5.2-6ubuntu1_all.deb ... 2023-11-26T07:14:26.564 INFO:teuthology.orchestra.run.smithi143.stdout:Unpacking python3-tempita (0.5.2-6ubuntu1) ... 2023-11-26T07:14:26.850 INFO:teuthology.orchestra.run.smithi143.stdout:Selecting previously unselected package python3-paste. 2023-11-26T07:14:26.865 INFO:teuthology.orchestra.run.smithi143.stdout:Preparing to unpack .../044-python3-paste_3.5.0+dfsg1-1_all.deb ... 2023-11-26T07:14:26.908 INFO:teuthology.orchestra.run.smithi143.stdout:Unpacking python3-paste (3.5.0+dfsg1-1) ... 2023-11-26T07:14:27.219 INFO:teuthology.orchestra.run.smithi143.stdout:Selecting previously unselected package python-pastedeploy-tpl. 2023-11-26T07:14:27.234 INFO:teuthology.orchestra.run.smithi143.stdout:Preparing to unpack .../045-python-pastedeploy-tpl_2.1.1-1_all.deb ... 2023-11-26T07:14:27.277 INFO:teuthology.orchestra.run.smithi143.stdout:Unpacking python-pastedeploy-tpl (2.1.1-1) ... 2023-11-26T07:14:27.597 INFO:teuthology.orchestra.run.smithi143.stdout:Selecting previously unselected package python3-pastedeploy. 2023-11-26T07:14:27.611 INFO:teuthology.orchestra.run.smithi143.stdout:Preparing to unpack .../046-python3-pastedeploy_2.1.1-1_all.deb ... 2023-11-26T07:14:27.655 INFO:teuthology.orchestra.run.smithi143.stdout:Unpacking python3-pastedeploy (2.1.1-1) ... 2023-11-26T07:14:27.966 INFO:teuthology.orchestra.run.smithi143.stdout:Selecting previously unselected package python3-webtest. 2023-11-26T07:14:27.981 INFO:teuthology.orchestra.run.smithi143.stdout:Preparing to unpack .../047-python3-webtest_2.0.35-1_all.deb ... 2023-11-26T07:14:28.016 INFO:teuthology.orchestra.run.smithi143.stdout:Unpacking python3-webtest (2.0.35-1) ... 2023-11-26T07:14:28.293 INFO:teuthology.orchestra.run.smithi143.stdout:Selecting previously unselected package python3-pecan. 2023-11-26T07:14:28.308 INFO:teuthology.orchestra.run.smithi143.stdout:Preparing to unpack .../048-python3-pecan_1.3.3-4ubuntu2_all.deb ... 2023-11-26T07:14:28.343 INFO:teuthology.orchestra.run.smithi143.stdout:Unpacking python3-pecan (1.3.3-4ubuntu2) ... 2023-11-26T07:14:28.673 INFO:teuthology.orchestra.run.smithi143.stdout:Selecting previously unselected package python3-certifi. 2023-11-26T07:14:28.687 INFO:teuthology.orchestra.run.smithi143.stdout:Preparing to unpack .../049-python3-certifi_2020.6.20-1_all.deb ... 2023-11-26T07:14:28.722 INFO:teuthology.orchestra.run.smithi143.stdout:Unpacking python3-certifi (2020.6.20-1) ... 2023-11-26T07:14:28.958 INFO:teuthology.orchestra.run.smithi188.stdout:Get:108 https://4.chacra.ceph.com/r/ceph/rishabh-mgr-vol-misc/44c2237fe77713214148c1fd622c872f346f1b99/ubuntu/jammy/flavors/default jammy/main amd64 ceph-volume all 18.0.0-7484-g44c2237f-1jammy [121 kB] 2023-11-26T07:14:28.959 INFO:teuthology.orchestra.run.smithi188.stdout:Get:109 https://4.chacra.ceph.com/r/ceph/rishabh-mgr-vol-misc/44c2237fe77713214148c1fd622c872f346f1b99/ubuntu/jammy/flavors/default jammy/main amd64 cephfs-mirror amd64 18.0.0-7484-g44c2237f-1jammy [225 kB] 2023-11-26T07:14:28.987 INFO:teuthology.orchestra.run.smithi188.stdout:Get:110 https://4.chacra.ceph.com/r/ceph/rishabh-mgr-vol-misc/44c2237fe77713214148c1fd622c872f346f1b99/ubuntu/jammy/flavors/default jammy/main amd64 cephfs-shell all 18.0.0-7484-g44c2237f-1jammy [34.0 kB] 2023-11-26T07:14:29.008 INFO:teuthology.orchestra.run.smithi188.stdout:Get:111 https://4.chacra.ceph.com/r/ceph/rishabh-mgr-vol-misc/44c2237fe77713214148c1fd622c872f346f1b99/ubuntu/jammy/flavors/default jammy/main amd64 cephfs-top all 18.0.0-7484-g44c2237f-1jammy [23.4 kB] 2023-11-26T07:14:29.021 INFO:teuthology.orchestra.run.smithi188.stdout:Get:112 https://4.chacra.ceph.com/r/ceph/rishabh-mgr-vol-misc/44c2237fe77713214148c1fd622c872f346f1b99/ubuntu/jammy/flavors/default jammy/main amd64 libcephfs-dev amd64 18.0.0-7484-g44c2237f-1jammy [38.3 kB] 2023-11-26T07:14:29.023 INFO:teuthology.orchestra.run.smithi188.stdout:Get:113 https://4.chacra.ceph.com/r/ceph/rishabh-mgr-vol-misc/44c2237fe77713214148c1fd622c872f346f1b99/ubuntu/jammy/flavors/default jammy/main amd64 radosgw amd64 18.0.0-7484-g44c2237f-1jammy [12.3 MB] 2023-11-26T07:14:29.209 INFO:teuthology.orchestra.run.smithi143.stdout:Selecting previously unselected package python3-idna. 2023-11-26T07:14:29.224 INFO:teuthology.orchestra.run.smithi143.stdout:Preparing to unpack .../050-python3-idna_3.3-1_all.deb ... 2023-11-26T07:14:29.292 INFO:teuthology.orchestra.run.smithi143.stdout:Unpacking python3-idna (3.3-1) ... 2023-11-26T07:14:29.603 INFO:teuthology.orchestra.run.smithi143.stdout:Selecting previously unselected package python3-urllib3. 2023-11-26T07:14:29.618 INFO:teuthology.orchestra.run.smithi143.stdout:Preparing to unpack .../051-python3-urllib3_1.26.5-1~exp1ubuntu0.1_all.deb ... 2023-11-26T07:14:29.661 INFO:teuthology.orchestra.run.smithi143.stdout:Unpacking python3-urllib3 (1.26.5-1~exp1ubuntu0.1) ... 2023-11-26T07:14:29.981 INFO:teuthology.orchestra.run.smithi143.stdout:Selecting previously unselected package python3-requests. 2023-11-26T07:14:29.996 INFO:teuthology.orchestra.run.smithi143.stdout:Preparing to unpack .../052-python3-requests_2.25.1+dfsg-2ubuntu0.1_all.deb ... 2023-11-26T07:14:30.039 INFO:teuthology.orchestra.run.smithi143.stdout:Unpacking python3-requests (2.25.1+dfsg-2ubuntu0.1) ... 2023-11-26T07:14:30.375 INFO:teuthology.orchestra.run.smithi143.stdout:Selecting previously unselected package python3-werkzeug. 2023-11-26T07:14:30.390 INFO:teuthology.orchestra.run.smithi143.stdout:Preparing to unpack .../053-python3-werkzeug_2.0.2+dfsg1-1ubuntu0.22.04.1_all.deb ... 2023-11-26T07:14:30.433 INFO:teuthology.orchestra.run.smithi143.stdout:Unpacking python3-werkzeug (2.0.2+dfsg1-1ubuntu0.22.04.1) ... 2023-11-26T07:14:30.744 INFO:teuthology.orchestra.run.smithi143.stdout:Selecting previously unselected package ceph-mgr-modules-core. 2023-11-26T07:14:30.759 INFO:teuthology.orchestra.run.smithi143.stdout:Preparing to unpack .../054-ceph-mgr-modules-core_18.0.0-7484-g44c2237f-1jammy_all.deb ... 2023-11-26T07:14:30.803 INFO:teuthology.orchestra.run.smithi143.stdout:Unpacking ceph-mgr-modules-core (18.0.0-7484-g44c2237f-1jammy) ... 2023-11-26T07:14:31.013 INFO:teuthology.orchestra.run.smithi188.stdout:Get:114 https://4.chacra.ceph.com/r/ceph/rishabh-mgr-vol-misc/44c2237fe77713214148c1fd622c872f346f1b99/ubuntu/jammy/flavors/default jammy/main amd64 rbd-fuse amd64 18.0.0-7484-g44c2237f-1jammy [91.2 kB] 2023-11-26T07:14:31.157 INFO:teuthology.orchestra.run.smithi143.stdout:Selecting previously unselected package libsqlite3-mod-ceph. 2023-11-26T07:14:31.172 INFO:teuthology.orchestra.run.smithi143.stdout:Preparing to unpack .../055-libsqlite3-mod-ceph_18.0.0-7484-g44c2237f-1jammy_amd64.deb ... 2023-11-26T07:14:31.207 INFO:teuthology.orchestra.run.smithi143.stdout:Unpacking libsqlite3-mod-ceph (18.0.0-7484-g44c2237f-1jammy) ... 2023-11-26T07:14:31.476 INFO:teuthology.orchestra.run.smithi143.stdout:Selecting previously unselected package python3-openssl. 2023-11-26T07:14:31.491 INFO:teuthology.orchestra.run.smithi143.stdout:Preparing to unpack .../056-python3-openssl_21.0.0-1_all.deb ... 2023-11-26T07:14:31.534 INFO:teuthology.orchestra.run.smithi143.stdout:Unpacking python3-openssl (21.0.0-1) ... 2023-11-26T07:14:31.594 INFO:teuthology.orchestra.run.smithi188.stdout:Fetched 186 MB in 60s (3,085 kB/s) 2023-11-26T07:14:31.704 INFO:teuthology.orchestra.run.smithi188.stdout:Selecting previously unselected package python3-pyperclip. 2023-11-26T07:14:31.773 INFO:teuthology.orchestra.run.smithi188.stdout:(Reading database ... (Reading database ... 5% (Reading database ... 10% (Reading database ... 15% (Reading database ... 20% (Reading database ... 25% (Reading database ... 30% (Reading database ... 35% (Reading database ... 40% (Reading database ... 45% (Reading database ... 50% (Reading database ... 55% (Reading database ... 60% (Reading database ... 65% (Reading database ... 70% (Reading database ... 75% (Reading database ... 80% (Reading database ... 85% (Reading database ... 90% (Reading database ... 95% (Reading database ... 100% (Reading database ... 132930 files and directories currently installed.) 2023-11-26T07:14:31.776 INFO:teuthology.orchestra.run.smithi188.stdout:Preparing to unpack .../000-python3-pyperclip_1.8.2-2_all.deb ... 2023-11-26T07:14:31.820 INFO:teuthology.orchestra.run.smithi188.stdout:Unpacking python3-pyperclip (1.8.2-2) ... 2023-11-26T07:14:31.887 INFO:teuthology.orchestra.run.smithi143.stdout:Selecting previously unselected package ceph-mgr. 2023-11-26T07:14:31.902 INFO:teuthology.orchestra.run.smithi143.stdout:Preparing to unpack .../057-ceph-mgr_18.0.0-7484-g44c2237f-1jammy_amd64.deb ... 2023-11-26T07:14:31.945 INFO:teuthology.orchestra.run.smithi143.stdout:Unpacking ceph-mgr (18.0.0-7484-g44c2237f-1jammy) ... 2023-11-26T07:14:32.156 INFO:teuthology.orchestra.run.smithi188.stdout:Selecting previously unselected package python3-wcwidth. 2023-11-26T07:14:32.169 INFO:teuthology.orchestra.run.smithi188.stdout:Preparing to unpack .../001-python3-wcwidth_0.2.5+dfsg1-1_all.deb ... 2023-11-26T07:14:32.214 INFO:teuthology.orchestra.run.smithi188.stdout:Unpacking python3-wcwidth (0.2.5+dfsg1-1) ... 2023-11-26T07:14:32.365 INFO:teuthology.orchestra.run.smithi143.stdout:Selecting previously unselected package ceph-mon. 2023-11-26T07:14:32.380 INFO:teuthology.orchestra.run.smithi143.stdout:Preparing to unpack .../058-ceph-mon_18.0.0-7484-g44c2237f-1jammy_amd64.deb ... 2023-11-26T07:14:32.423 INFO:teuthology.orchestra.run.smithi143.stdout:Unpacking ceph-mon (18.0.0-7484-g44c2237f-1jammy) ... 2023-11-26T07:14:32.525 INFO:teuthology.orchestra.run.smithi188.stdout:Selecting previously unselected package python3-cmd2. 2023-11-26T07:14:32.539 INFO:teuthology.orchestra.run.smithi188.stdout:Preparing to unpack .../002-python3-cmd2_2.3.3-1_all.deb ... 2023-11-26T07:14:32.609 INFO:teuthology.orchestra.run.smithi188.stdout:Unpacking python3-cmd2 (2.3.3-1) ... 2023-11-26T07:14:32.953 INFO:teuthology.orchestra.run.smithi188.stdout:Selecting previously unselected package liblttng-ust1:amd64. 2023-11-26T07:14:32.967 INFO:teuthology.orchestra.run.smithi188.stdout:Preparing to unpack .../003-liblttng-ust1_2.13.1-1ubuntu1_amd64.deb ... 2023-11-26T07:14:33.054 INFO:teuthology.orchestra.run.smithi188.stdout:Unpacking liblttng-ust1:amd64 (2.13.1-1ubuntu1) ... 2023-11-26T07:14:33.176 INFO:teuthology.orchestra.run.smithi143.stdout:Selecting previously unselected package ceph-osd. 2023-11-26T07:14:33.192 INFO:teuthology.orchestra.run.smithi143.stdout:Preparing to unpack .../059-ceph-osd_18.0.0-7484-g44c2237f-1jammy_amd64.deb ... 2023-11-26T07:14:33.226 INFO:teuthology.orchestra.run.smithi143.stdout:Unpacking ceph-osd (18.0.0-7484-g44c2237f-1jammy) ... 2023-11-26T07:14:33.390 INFO:teuthology.orchestra.run.smithi188.stdout:Selecting previously unselected package libdouble-conversion3:amd64. 2023-11-26T07:14:33.404 INFO:teuthology.orchestra.run.smithi188.stdout:Preparing to unpack .../004-libdouble-conversion3_3.1.7-4_amd64.deb ... 2023-11-26T07:14:33.449 INFO:teuthology.orchestra.run.smithi188.stdout:Unpacking libdouble-conversion3:amd64 (3.1.7-4) ... 2023-11-26T07:14:33.810 INFO:teuthology.orchestra.run.smithi188.stdout:Selecting previously unselected package libpcre2-16-0:amd64. 2023-11-26T07:14:33.825 INFO:teuthology.orchestra.run.smithi188.stdout:Preparing to unpack .../005-libpcre2-16-0_10.39-3ubuntu0.1_amd64.deb ... 2023-11-26T07:14:33.877 INFO:teuthology.orchestra.run.smithi188.stdout:Unpacking libpcre2-16-0:amd64 (10.39-3ubuntu0.1) ... 2023-11-26T07:14:34.297 INFO:teuthology.orchestra.run.smithi188.stdout:Selecting previously unselected package libqt5core5a:amd64. 2023-11-26T07:14:34.311 INFO:teuthology.orchestra.run.smithi188.stdout:Preparing to unpack .../006-libqt5core5a_5.15.3+dfsg-2ubuntu0.2_amd64.deb ... 2023-11-26T07:14:34.380 INFO:teuthology.orchestra.run.smithi188.stdout:Unpacking libqt5core5a:amd64 (5.15.3+dfsg-2ubuntu0.2) ... 2023-11-26T07:14:34.454 INFO:teuthology.orchestra.run.smithi143.stdout:Selecting previously unselected package ceph. 2023-11-26T07:14:34.469 INFO:teuthology.orchestra.run.smithi143.stdout:Preparing to unpack .../060-ceph_18.0.0-7484-g44c2237f-1jammy_amd64.deb ... 2023-11-26T07:14:34.512 INFO:teuthology.orchestra.run.smithi143.stdout:Unpacking ceph (18.0.0-7484-g44c2237f-1jammy) ... 2023-11-26T07:14:34.767 INFO:teuthology.orchestra.run.smithi188.stdout:Selecting previously unselected package libqt5dbus5:amd64. 2023-11-26T07:14:34.780 INFO:teuthology.orchestra.run.smithi188.stdout:Preparing to unpack .../007-libqt5dbus5_5.15.3+dfsg-2ubuntu0.2_amd64.deb ... 2023-11-26T07:14:34.826 INFO:teuthology.orchestra.run.smithi188.stdout:Unpacking libqt5dbus5:amd64 (5.15.3+dfsg-2ubuntu0.2) ... 2023-11-26T07:14:34.840 INFO:teuthology.orchestra.run.smithi143.stdout:Selecting previously unselected package ceph-fuse. 2023-11-26T07:14:34.855 INFO:teuthology.orchestra.run.smithi143.stdout:Preparing to unpack .../061-ceph-fuse_18.0.0-7484-g44c2237f-1jammy_amd64.deb ... 2023-11-26T07:14:34.898 INFO:teuthology.orchestra.run.smithi143.stdout:Unpacking ceph-fuse (18.0.0-7484-g44c2237f-1jammy) ... 2023-11-26T07:14:35.179 INFO:teuthology.orchestra.run.smithi188.stdout:Selecting previously unselected package libqt5network5:amd64. 2023-11-26T07:14:35.192 INFO:teuthology.orchestra.run.smithi188.stdout:Preparing to unpack .../008-libqt5network5_5.15.3+dfsg-2ubuntu0.2_amd64.deb ... 2023-11-26T07:14:35.228 INFO:teuthology.orchestra.run.smithi188.stdout:Unpacking libqt5network5:amd64 (5.15.3+dfsg-2ubuntu0.2) ... 2023-11-26T07:14:35.285 INFO:teuthology.orchestra.run.smithi143.stdout:Selecting previously unselected package ceph-mds. 2023-11-26T07:14:35.299 INFO:teuthology.orchestra.run.smithi143.stdout:Preparing to unpack .../062-ceph-mds_18.0.0-7484-g44c2237f-1jammy_amd64.deb ... 2023-11-26T07:14:35.360 INFO:teuthology.orchestra.run.smithi143.stdout:Unpacking ceph-mds (18.0.0-7484-g44c2237f-1jammy) ... 2023-11-26T07:14:35.581 INFO:teuthology.orchestra.run.smithi188.stdout:Selecting previously unselected package libthrift-0.16.0:amd64. 2023-11-26T07:14:35.595 INFO:teuthology.orchestra.run.smithi188.stdout:Preparing to unpack .../009-libthrift-0.16.0_0.16.0-2_amd64.deb ... 2023-11-26T07:14:35.640 INFO:teuthology.orchestra.run.smithi188.stdout:Unpacking libthrift-0.16.0:amd64 (0.16.0-2) ... 2023-11-26T07:14:35.679 INFO:teuthology.orchestra.run.smithi143.stdout:Selecting previously unselected package cephadm. 2023-11-26T07:14:35.694 INFO:teuthology.orchestra.run.smithi143.stdout:Preparing to unpack .../063-cephadm_18.0.0-7484-g44c2237f-1jammy_amd64.deb ... 2023-11-26T07:14:35.746 INFO:teuthology.orchestra.run.smithi143.stdout:Unpacking cephadm (18.0.0-7484-g44c2237f-1jammy) ... 2023-11-26T07:14:35.976 INFO:teuthology.orchestra.run.smithi188.stdout:Selecting previously unselected package librados2. 2023-11-26T07:14:35.990 INFO:teuthology.orchestra.run.smithi188.stdout:Preparing to unpack .../010-librados2_18.0.0-7484-g44c2237f-1jammy_amd64.deb ... 2023-11-26T07:14:36.057 INFO:teuthology.orchestra.run.smithi143.stdout:Selecting previously unselected package python3-asyncssh. 2023-11-26T07:14:36.072 INFO:teuthology.orchestra.run.smithi143.stdout:Preparing to unpack .../064-python3-asyncssh_2.5.0-1_all.deb ... 2023-11-26T07:14:36.080 INFO:teuthology.orchestra.run.smithi188.stdout:Unpacking librados2 (18.0.0-7484-g44c2237f-1jammy) ... 2023-11-26T07:14:36.115 INFO:teuthology.orchestra.run.smithi143.stdout:Unpacking python3-asyncssh (2.5.0-1) ... 2023-11-26T07:14:36.385 INFO:teuthology.orchestra.run.smithi143.stdout:Selecting previously unselected package python-babel-localedata. 2023-11-26T07:14:36.400 INFO:teuthology.orchestra.run.smithi143.stdout:Preparing to unpack .../065-python-babel-localedata_2.8.0+dfsg.1-7_all.deb ... 2023-11-26T07:14:36.451 INFO:teuthology.orchestra.run.smithi143.stdout:Unpacking python-babel-localedata (2.8.0+dfsg.1-7) ... 2023-11-26T07:14:36.729 INFO:teuthology.orchestra.run.smithi188.stdout:Selecting previously unselected package librbd1. 2023-11-26T07:14:36.743 INFO:teuthology.orchestra.run.smithi188.stdout:Preparing to unpack .../011-librbd1_18.0.0-7484-g44c2237f-1jammy_amd64.deb ... 2023-11-26T07:14:36.787 INFO:teuthology.orchestra.run.smithi188.stdout:Unpacking librbd1 (18.0.0-7484-g44c2237f-1jammy) ... 2023-11-26T07:14:37.173 INFO:teuthology.orchestra.run.smithi188.stdout:Selecting previously unselected package libcephfs2. 2023-11-26T07:14:37.187 INFO:teuthology.orchestra.run.smithi188.stdout:Preparing to unpack .../012-libcephfs2_18.0.0-7484-g44c2237f-1jammy_amd64.deb ... 2023-11-26T07:14:37.249 INFO:teuthology.orchestra.run.smithi188.stdout:Unpacking libcephfs2 (18.0.0-7484-g44c2237f-1jammy) ... 2023-11-26T07:14:37.464 INFO:teuthology.orchestra.run.smithi143.stdout:Selecting previously unselected package python3-babel. 2023-11-26T07:14:37.479 INFO:teuthology.orchestra.run.smithi143.stdout:Preparing to unpack .../066-python3-babel_2.8.0+dfsg.1-7_all.deb ... 2023-11-26T07:14:37.518 INFO:teuthology.orchestra.run.smithi188.stdout:Selecting previously unselected package python3-rados. 2023-11-26T07:14:37.531 INFO:teuthology.orchestra.run.smithi143.stdout:Unpacking python3-babel (2.8.0+dfsg.1-7) ... 2023-11-26T07:14:37.531 INFO:teuthology.orchestra.run.smithi188.stdout:Preparing to unpack .../013-python3-rados_18.0.0-7484-g44c2237f-1jammy_amd64.deb ... 2023-11-26T07:14:37.576 INFO:teuthology.orchestra.run.smithi188.stdout:Unpacking python3-rados (18.0.0-7484-g44c2237f-1jammy) ... 2023-11-26T07:14:37.834 INFO:teuthology.orchestra.run.smithi143.stdout:Selecting previously unselected package python3-jinja2. 2023-11-26T07:14:37.848 INFO:teuthology.orchestra.run.smithi143.stdout:Preparing to unpack .../067-python3-jinja2_3.0.3-1_all.deb ... 2023-11-26T07:14:37.901 INFO:teuthology.orchestra.run.smithi143.stdout:Unpacking python3-jinja2 (3.0.3-1) ... 2023-11-26T07:14:37.929 INFO:teuthology.orchestra.run.smithi188.stdout:Selecting previously unselected package python3-ceph-argparse. 2023-11-26T07:14:37.942 INFO:teuthology.orchestra.run.smithi188.stdout:Preparing to unpack .../014-python3-ceph-argparse_18.0.0-7484-g44c2237f-1jammy_all.deb ... 2023-11-26T07:14:37.987 INFO:teuthology.orchestra.run.smithi188.stdout:Unpacking python3-ceph-argparse (18.0.0-7484-g44c2237f-1jammy) ... 2023-11-26T07:14:38.239 INFO:teuthology.orchestra.run.smithi188.stdout:Selecting previously unselected package python3-cephfs. 2023-11-26T07:14:38.253 INFO:teuthology.orchestra.run.smithi188.stdout:Preparing to unpack .../015-python3-cephfs_18.0.0-7484-g44c2237f-1jammy_amd64.deb ... 2023-11-26T07:14:38.295 INFO:teuthology.orchestra.run.smithi143.stdout:Selecting previously unselected package ceph-mgr-cephadm. 2023-11-26T07:14:38.298 INFO:teuthology.orchestra.run.smithi188.stdout:Unpacking python3-cephfs (18.0.0-7484-g44c2237f-1jammy) ... 2023-11-26T07:14:38.310 INFO:teuthology.orchestra.run.smithi143.stdout:Preparing to unpack .../068-ceph-mgr-cephadm_18.0.0-7484-g44c2237f-1jammy_all.deb ... 2023-11-26T07:14:38.353 INFO:teuthology.orchestra.run.smithi143.stdout:Unpacking ceph-mgr-cephadm (18.0.0-7484-g44c2237f-1jammy) ... 2023-11-26T07:14:38.634 INFO:teuthology.orchestra.run.smithi188.stdout:Selecting previously unselected package python3-ceph-common. 2023-11-26T07:14:38.648 INFO:teuthology.orchestra.run.smithi188.stdout:Preparing to unpack .../016-python3-ceph-common_18.0.0-7484-g44c2237f-1jammy_all.deb ... 2023-11-26T07:14:38.690 INFO:teuthology.orchestra.run.smithi143.stdout:Selecting previously unselected package python3-bcrypt. 2023-11-26T07:14:38.692 INFO:teuthology.orchestra.run.smithi188.stdout:Unpacking python3-ceph-common (18.0.0-7484-g44c2237f-1jammy) ... 2023-11-26T07:14:38.705 INFO:teuthology.orchestra.run.smithi143.stdout:Preparing to unpack .../069-python3-bcrypt_3.2.0-1build1_amd64.deb ... 2023-11-26T07:14:38.748 INFO:teuthology.orchestra.run.smithi143.stdout:Unpacking python3-bcrypt (3.2.0-1build1) ... 2023-11-26T07:14:39.045 INFO:teuthology.orchestra.run.smithi188.stdout:Selecting previously unselected package python3-prettytable. 2023-11-26T07:14:39.051 INFO:teuthology.orchestra.run.smithi143.stdout:Selecting previously unselected package python3-repoze.lru. 2023-11-26T07:14:39.059 INFO:teuthology.orchestra.run.smithi188.stdout:Preparing to unpack .../017-python3-prettytable_2.5.0-2_all.deb ... 2023-11-26T07:14:39.067 INFO:teuthology.orchestra.run.smithi143.stdout:Preparing to unpack .../070-python3-repoze.lru_0.7-2_all.deb ... 2023-11-26T07:14:39.104 INFO:teuthology.orchestra.run.smithi188.stdout:Unpacking python3-prettytable (2.5.0-2) ... 2023-11-26T07:14:39.109 INFO:teuthology.orchestra.run.smithi143.stdout:Unpacking python3-repoze.lru (0.7-2) ... 2023-11-26T07:14:39.340 INFO:teuthology.orchestra.run.smithi188.stdout:Selecting previously unselected package python3-rbd. 2023-11-26T07:14:39.353 INFO:teuthology.orchestra.run.smithi188.stdout:Preparing to unpack .../018-python3-rbd_18.0.0-7484-g44c2237f-1jammy_amd64.deb ... 2023-11-26T07:14:39.390 INFO:teuthology.orchestra.run.smithi188.stdout:Unpacking python3-rbd (18.0.0-7484-g44c2237f-1jammy) ... 2023-11-26T07:14:39.395 INFO:teuthology.orchestra.run.smithi143.stdout:Selecting previously unselected package python3-routes. 2023-11-26T07:14:39.410 INFO:teuthology.orchestra.run.smithi143.stdout:Preparing to unpack .../071-python3-routes_2.5.1-1ubuntu1_all.deb ... 2023-11-26T07:14:39.445 INFO:teuthology.orchestra.run.smithi143.stdout:Unpacking python3-routes (2.5.1-1ubuntu1) ... 2023-11-26T07:14:39.743 INFO:teuthology.orchestra.run.smithi188.stdout:Selecting previously unselected package librdkafka1:amd64. 2023-11-26T07:14:39.756 INFO:teuthology.orchestra.run.smithi188.stdout:Preparing to unpack .../019-librdkafka1_1.8.0-1build1_amd64.deb ... 2023-11-26T07:14:39.793 INFO:teuthology.orchestra.run.smithi188.stdout:Unpacking librdkafka1:amd64 (1.8.0-1build1) ... 2023-11-26T07:14:39.848 INFO:teuthology.orchestra.run.smithi143.stdout:Selecting previously unselected package ceph-mgr-dashboard. 2023-11-26T07:14:39.863 INFO:teuthology.orchestra.run.smithi143.stdout:Preparing to unpack .../072-ceph-mgr-dashboard_18.0.0-7484-g44c2237f-1jammy_all.deb ... 2023-11-26T07:14:39.915 INFO:teuthology.orchestra.run.smithi143.stdout:Unpacking ceph-mgr-dashboard (18.0.0-7484-g44c2237f-1jammy) ... 2023-11-26T07:14:40.162 INFO:teuthology.orchestra.run.smithi188.stdout:Selecting previously unselected package librgw2. 2023-11-26T07:14:40.176 INFO:teuthology.orchestra.run.smithi188.stdout:Preparing to unpack .../020-librgw2_18.0.0-7484-g44c2237f-1jammy_amd64.deb ... 2023-11-26T07:14:40.221 INFO:teuthology.orchestra.run.smithi188.stdout:Unpacking librgw2 (18.0.0-7484-g44c2237f-1jammy) ... 2023-11-26T07:14:40.343 INFO:teuthology.orchestra.run.smithi143.stdout:Selecting previously unselected package python3-sklearn-lib:amd64. 2023-11-26T07:14:40.358 INFO:teuthology.orchestra.run.smithi143.stdout:Preparing to unpack .../073-python3-sklearn-lib_0.23.2-5ubuntu6_amd64.deb ... 2023-11-26T07:14:40.401 INFO:teuthology.orchestra.run.smithi143.stdout:Unpacking python3-sklearn-lib:amd64 (0.23.2-5ubuntu6) ... 2023-11-26T07:14:40.698 INFO:teuthology.orchestra.run.smithi188.stdout:Selecting previously unselected package python3-rgw. 2023-11-26T07:14:40.712 INFO:teuthology.orchestra.run.smithi188.stdout:Preparing to unpack .../021-python3-rgw_18.0.0-7484-g44c2237f-1jammy_amd64.deb ... 2023-11-26T07:14:40.756 INFO:teuthology.orchestra.run.smithi188.stdout:Unpacking python3-rgw (18.0.0-7484-g44c2237f-1jammy) ... 2023-11-26T07:14:40.913 INFO:teuthology.orchestra.run.smithi143.stdout:Selecting previously unselected package python3-joblib. 2023-11-26T07:14:40.928 INFO:teuthology.orchestra.run.smithi143.stdout:Preparing to unpack .../074-python3-joblib_0.17.0-4ubuntu1_all.deb ... 2023-11-26T07:14:40.971 INFO:teuthology.orchestra.run.smithi143.stdout:Unpacking python3-joblib (0.17.0-4ubuntu1) ... 2023-11-26T07:14:41.059 INFO:teuthology.orchestra.run.smithi188.stdout:Selecting previously unselected package liboath0:amd64. 2023-11-26T07:14:41.074 INFO:teuthology.orchestra.run.smithi188.stdout:Preparing to unpack .../022-liboath0_2.6.7-3build1_amd64.deb ... 2023-11-26T07:14:41.109 INFO:teuthology.orchestra.run.smithi188.stdout:Unpacking liboath0:amd64 (2.6.7-3build1) ... 2023-11-26T07:14:41.333 INFO:teuthology.orchestra.run.smithi143.stdout:Selecting previously unselected package python3-threadpoolctl. 2023-11-26T07:14:41.348 INFO:teuthology.orchestra.run.smithi143.stdout:Preparing to unpack .../075-python3-threadpoolctl_3.1.0-1_all.deb ... 2023-11-26T07:14:41.382 INFO:teuthology.orchestra.run.smithi143.stdout:Unpacking python3-threadpoolctl (3.1.0-1) ... 2023-11-26T07:14:41.446 INFO:teuthology.orchestra.run.smithi188.stdout:Selecting previously unselected package libradosstriper1. 2023-11-26T07:14:41.460 INFO:teuthology.orchestra.run.smithi188.stdout:Preparing to unpack .../023-libradosstriper1_18.0.0-7484-g44c2237f-1jammy_amd64.deb ... 2023-11-26T07:14:41.504 INFO:teuthology.orchestra.run.smithi188.stdout:Unpacking libradosstriper1 (18.0.0-7484-g44c2237f-1jammy) ... 2023-11-26T07:14:41.652 INFO:teuthology.orchestra.run.smithi143.stdout:Selecting previously unselected package python3-sklearn. 2023-11-26T07:14:41.667 INFO:teuthology.orchestra.run.smithi143.stdout:Preparing to unpack .../076-python3-sklearn_0.23.2-5ubuntu6_all.deb ... 2023-11-26T07:14:41.727 INFO:teuthology.orchestra.run.smithi143.stdout:Unpacking python3-sklearn (0.23.2-5ubuntu6) ... 2023-11-26T07:14:41.974 INFO:teuthology.orchestra.run.smithi188.stdout:Selecting previously unselected package ceph-common. 2023-11-26T07:14:41.988 INFO:teuthology.orchestra.run.smithi188.stdout:Preparing to unpack .../024-ceph-common_18.0.0-7484-g44c2237f-1jammy_amd64.deb ... 2023-11-26T07:14:42.032 INFO:teuthology.orchestra.run.smithi188.stdout:Unpacking ceph-common (18.0.0-7484-g44c2237f-1jammy) ... 2023-11-26T07:14:42.412 INFO:teuthology.orchestra.run.smithi143.stdout:Selecting previously unselected package ceph-mgr-diskprediction-local. 2023-11-26T07:14:42.426 INFO:teuthology.orchestra.run.smithi143.stdout:Preparing to unpack .../077-ceph-mgr-diskprediction-local_18.0.0-7484-g44c2237f-1jammy_all.deb ... 2023-11-26T07:14:42.470 INFO:teuthology.orchestra.run.smithi143.stdout:Unpacking ceph-mgr-diskprediction-local (18.0.0-7484-g44c2237f-1jammy) ... 2023-11-26T07:14:43.113 INFO:teuthology.orchestra.run.smithi188.stdout:Selecting previously unselected package ceph-base. 2023-11-26T07:14:43.128 INFO:teuthology.orchestra.run.smithi188.stdout:Preparing to unpack .../025-ceph-base_18.0.0-7484-g44c2237f-1jammy_amd64.deb ... 2023-11-26T07:14:43.180 INFO:teuthology.orchestra.run.smithi188.stdout:Unpacking ceph-base (18.0.0-7484-g44c2237f-1jammy) ... 2023-11-26T07:14:43.365 INFO:teuthology.orchestra.run.smithi143.stdout:Selecting previously unselected package python3-cachetools. 2023-11-26T07:14:43.381 INFO:teuthology.orchestra.run.smithi143.stdout:Preparing to unpack .../078-python3-cachetools_5.0.0-1_all.deb ... 2023-11-26T07:14:43.415 INFO:teuthology.orchestra.run.smithi143.stdout:Unpacking python3-cachetools (5.0.0-1) ... 2023-11-26T07:14:43.711 INFO:teuthology.orchestra.run.smithi143.stdout:Selecting previously unselected package python3-pyasn1. 2023-11-26T07:14:43.726 INFO:teuthology.orchestra.run.smithi143.stdout:Preparing to unpack .../079-python3-pyasn1_0.4.8-1_all.deb ... 2023-11-26T07:14:43.768 INFO:teuthology.orchestra.run.smithi143.stdout:Unpacking python3-pyasn1 (0.4.8-1) ... 2023-11-26T07:14:43.892 INFO:teuthology.orchestra.run.smithi188.stdout:Selecting previously unselected package python3-jaraco.functools. 2023-11-26T07:14:43.906 INFO:teuthology.orchestra.run.smithi188.stdout:Preparing to unpack .../026-python3-jaraco.functools_3.4.0-2_all.deb ... 2023-11-26T07:14:43.950 INFO:teuthology.orchestra.run.smithi188.stdout:Unpacking python3-jaraco.functools (3.4.0-2) ... 2023-11-26T07:14:44.114 INFO:teuthology.orchestra.run.smithi143.stdout:Selecting previously unselected package python3-pyasn1-modules. 2023-11-26T07:14:44.130 INFO:teuthology.orchestra.run.smithi143.stdout:Preparing to unpack .../080-python3-pyasn1-modules_0.2.1-1_all.deb ... 2023-11-26T07:14:44.172 INFO:teuthology.orchestra.run.smithi143.stdout:Unpacking python3-pyasn1-modules (0.2.1-1) ... 2023-11-26T07:14:44.278 INFO:teuthology.orchestra.run.smithi188.stdout:Selecting previously unselected package python3-cheroot. 2023-11-26T07:14:44.292 INFO:teuthology.orchestra.run.smithi188.stdout:Preparing to unpack .../027-python3-cheroot_8.5.2+ds1-1ubuntu3.1_all.deb ... 2023-11-26T07:14:44.337 INFO:teuthology.orchestra.run.smithi188.stdout:Unpacking python3-cheroot (8.5.2+ds1-1ubuntu3.1) ... 2023-11-26T07:14:44.500 INFO:teuthology.orchestra.run.smithi143.stdout:Selecting previously unselected package python3-rsa. 2023-11-26T07:14:44.515 INFO:teuthology.orchestra.run.smithi143.stdout:Preparing to unpack .../081-python3-rsa_4.8-1_all.deb ... 2023-11-26T07:14:44.558 INFO:teuthology.orchestra.run.smithi143.stdout:Unpacking python3-rsa (4.8-1) ... 2023-11-26T07:14:44.623 INFO:teuthology.orchestra.run.smithi188.stdout:Selecting previously unselected package python3-jaraco.classes. 2023-11-26T07:14:44.636 INFO:teuthology.orchestra.run.smithi188.stdout:Preparing to unpack .../028-python3-jaraco.classes_3.2.1-3_all.deb ... 2023-11-26T07:14:44.681 INFO:teuthology.orchestra.run.smithi188.stdout:Unpacking python3-jaraco.classes (3.2.1-3) ... 2023-11-26T07:14:44.879 INFO:teuthology.orchestra.run.smithi143.stdout:Selecting previously unselected package python3-google-auth. 2023-11-26T07:14:44.894 INFO:teuthology.orchestra.run.smithi143.stdout:Preparing to unpack .../082-python3-google-auth_1.5.1-3_all.deb ... 2023-11-26T07:14:44.936 INFO:teuthology.orchestra.run.smithi143.stdout:Unpacking python3-google-auth (1.5.1-3) ... 2023-11-26T07:14:45.018 INFO:teuthology.orchestra.run.smithi188.stdout:Selecting previously unselected package python3-jaraco.text. 2023-11-26T07:14:45.031 INFO:teuthology.orchestra.run.smithi188.stdout:Preparing to unpack .../029-python3-jaraco.text_3.6.0-2_all.deb ... 2023-11-26T07:14:45.076 INFO:teuthology.orchestra.run.smithi188.stdout:Unpacking python3-jaraco.text (3.6.0-2) ... 2023-11-26T07:14:45.367 INFO:teuthology.orchestra.run.smithi143.stdout:Selecting previously unselected package python3-requests-oauthlib. 2023-11-26T07:14:45.382 INFO:teuthology.orchestra.run.smithi143.stdout:Preparing to unpack .../083-python3-requests-oauthlib_1.3.0+ds-0.1_all.deb ... 2023-11-26T07:14:45.413 INFO:teuthology.orchestra.run.smithi188.stdout:Selecting previously unselected package python3-jaraco.collections. 2023-11-26T07:14:45.425 INFO:teuthology.orchestra.run.smithi143.stdout:Unpacking python3-requests-oauthlib (1.3.0+ds-0.1) ... 2023-11-26T07:14:45.426 INFO:teuthology.orchestra.run.smithi188.stdout:Preparing to unpack .../030-python3-jaraco.collections_3.4.0-2_all.deb ... 2023-11-26T07:14:45.470 INFO:teuthology.orchestra.run.smithi188.stdout:Unpacking python3-jaraco.collections (3.4.0-2) ... 2023-11-26T07:14:45.704 INFO:teuthology.orchestra.run.smithi143.stdout:Selecting previously unselected package python3-websocket. 2023-11-26T07:14:45.719 INFO:teuthology.orchestra.run.smithi143.stdout:Preparing to unpack .../084-python3-websocket_1.2.3-1_all.deb ... 2023-11-26T07:14:45.731 INFO:teuthology.orchestra.run.smithi188.stdout:Selecting previously unselected package python3-tempora. 2023-11-26T07:14:45.746 INFO:teuthology.orchestra.run.smithi188.stdout:Preparing to unpack .../031-python3-tempora_4.1.2-1_all.deb ... 2023-11-26T07:14:45.762 INFO:teuthology.orchestra.run.smithi143.stdout:Unpacking python3-websocket (1.2.3-1) ... 2023-11-26T07:14:45.790 INFO:teuthology.orchestra.run.smithi188.stdout:Unpacking python3-tempora (4.1.2-1) ... 2023-11-26T07:14:46.082 INFO:teuthology.orchestra.run.smithi143.stdout:Selecting previously unselected package python3-kubernetes. 2023-11-26T07:14:46.098 INFO:teuthology.orchestra.run.smithi143.stdout:Preparing to unpack .../085-python3-kubernetes_12.0.1-1ubuntu1_all.deb ... 2023-11-26T07:14:46.126 INFO:teuthology.orchestra.run.smithi188.stdout:Selecting previously unselected package python3-portend. 2023-11-26T07:14:46.140 INFO:teuthology.orchestra.run.smithi143.stdout:Unpacking python3-kubernetes (12.0.1-1ubuntu1) ... 2023-11-26T07:14:46.141 INFO:teuthology.orchestra.run.smithi188.stdout:Preparing to unpack .../032-python3-portend_3.0.0-1_all.deb ... 2023-11-26T07:14:46.184 INFO:teuthology.orchestra.run.smithi188.stdout:Unpacking python3-portend (3.0.0-1) ... 2023-11-26T07:14:46.521 INFO:teuthology.orchestra.run.smithi188.stdout:Selecting previously unselected package python3-zc.lockfile. 2023-11-26T07:14:46.544 INFO:teuthology.orchestra.run.smithi188.stdout:Preparing to unpack .../033-python3-zc.lockfile_2.0-1_all.deb ... 2023-11-26T07:14:46.729 INFO:teuthology.orchestra.run.smithi188.stdout:Unpacking python3-zc.lockfile (2.0-1) ... 2023-11-26T07:14:46.821 INFO:teuthology.orchestra.run.smithi143.stdout:Selecting previously unselected package ceph-mgr-k8sevents. 2023-11-26T07:14:46.836 INFO:teuthology.orchestra.run.smithi143.stdout:Preparing to unpack .../086-ceph-mgr-k8sevents_18.0.0-7484-g44c2237f-1jammy_all.deb ... 2023-11-26T07:14:46.871 INFO:teuthology.orchestra.run.smithi143.stdout:Unpacking ceph-mgr-k8sevents (18.0.0-7484-g44c2237f-1jammy) ... 2023-11-26T07:14:47.032 INFO:teuthology.orchestra.run.smithi188.stdout:Selecting previously unselected package python3-cherrypy3. 2023-11-26T07:14:47.047 INFO:teuthology.orchestra.run.smithi188.stdout:Preparing to unpack .../034-python3-cherrypy3_18.6.1-4_all.deb ... 2023-11-26T07:14:47.107 INFO:teuthology.orchestra.run.smithi188.stdout:Unpacking python3-cherrypy3 (18.6.1-4) ... 2023-11-26T07:14:47.174 INFO:teuthology.orchestra.run.smithi143.stdout:Selecting previously unselected package libonig5:amd64. 2023-11-26T07:14:47.188 INFO:teuthology.orchestra.run.smithi143.stdout:Preparing to unpack .../087-libonig5_6.9.7.1-2build1_amd64.deb ... 2023-11-26T07:14:47.232 INFO:teuthology.orchestra.run.smithi143.stdout:Unpacking libonig5:amd64 (6.9.7.1-2build1) ... 2023-11-26T07:14:47.419 INFO:teuthology.orchestra.run.smithi188.stdout:Selecting previously unselected package python3-natsort. 2023-11-26T07:14:47.433 INFO:teuthology.orchestra.run.smithi188.stdout:Preparing to unpack .../035-python3-natsort_8.0.2-1_all.deb ... 2023-11-26T07:14:47.477 INFO:teuthology.orchestra.run.smithi188.stdout:Unpacking python3-natsort (8.0.2-1) ... 2023-11-26T07:14:47.577 INFO:teuthology.orchestra.run.smithi143.stdout:Selecting previously unselected package libjq1:amd64. 2023-11-26T07:14:47.592 INFO:teuthology.orchestra.run.smithi143.stdout:Preparing to unpack .../088-libjq1_1.6-2.1ubuntu3_amd64.deb ... 2023-11-26T07:14:47.636 INFO:teuthology.orchestra.run.smithi143.stdout:Unpacking libjq1:amd64 (1.6-2.1ubuntu3) ... 2023-11-26T07:14:47.797 INFO:teuthology.orchestra.run.smithi188.stdout:Selecting previously unselected package python3-logutils. 2023-11-26T07:14:47.812 INFO:teuthology.orchestra.run.smithi188.stdout:Preparing to unpack .../036-python3-logutils_0.3.3-8_all.deb ... 2023-11-26T07:14:47.855 INFO:teuthology.orchestra.run.smithi188.stdout:Unpacking python3-logutils (0.3.3-8) ... 2023-11-26T07:14:47.880 INFO:teuthology.orchestra.run.smithi143.stdout:Selecting previously unselected package jq. 2023-11-26T07:14:47.895 INFO:teuthology.orchestra.run.smithi143.stdout:Preparing to unpack .../089-jq_1.6-2.1ubuntu3_amd64.deb ... 2023-11-26T07:14:47.930 INFO:teuthology.orchestra.run.smithi143.stdout:Unpacking jq (1.6-2.1ubuntu3) ... 2023-11-26T07:14:48.158 INFO:teuthology.orchestra.run.smithi188.stdout:Selecting previously unselected package python3-markupsafe. 2023-11-26T07:14:48.173 INFO:teuthology.orchestra.run.smithi188.stdout:Preparing to unpack .../037-python3-markupsafe_2.0.1-2build1_amd64.deb ... 2023-11-26T07:14:48.183 INFO:teuthology.orchestra.run.smithi143.stdout:Selecting previously unselected package socat. 2023-11-26T07:14:48.198 INFO:teuthology.orchestra.run.smithi143.stdout:Preparing to unpack .../090-socat_1.7.4.1-3ubuntu4_amd64.deb ... 2023-11-26T07:14:48.241 INFO:teuthology.orchestra.run.smithi143.stdout:Unpacking socat (1.7.4.1-3ubuntu4) ... 2023-11-26T07:14:48.250 INFO:teuthology.orchestra.run.smithi188.stdout:Unpacking python3-markupsafe (2.0.1-2build1) ... 2023-11-26T07:14:48.536 INFO:teuthology.orchestra.run.smithi188.stdout:Selecting previously unselected package python3-mako. 2023-11-26T07:14:48.545 INFO:teuthology.orchestra.run.smithi143.stdout:Selecting previously unselected package xmlstarlet. 2023-11-26T07:14:48.551 INFO:teuthology.orchestra.run.smithi188.stdout:Preparing to unpack .../038-python3-mako_1.1.3+ds1-2ubuntu0.1_all.deb ... 2023-11-26T07:14:48.561 INFO:teuthology.orchestra.run.smithi143.stdout:Preparing to unpack .../091-xmlstarlet_1.6.1-2.1_amd64.deb ... 2023-11-26T07:14:48.594 INFO:teuthology.orchestra.run.smithi188.stdout:Unpacking python3-mako (1.1.3+ds1-2ubuntu0.1) ... 2023-11-26T07:14:48.603 INFO:teuthology.orchestra.run.smithi143.stdout:Unpacking xmlstarlet (1.6.1-2.1) ... 2023-11-26T07:14:48.940 INFO:teuthology.orchestra.run.smithi188.stdout:Selecting previously unselected package python3-simplegeneric. 2023-11-26T07:14:48.949 INFO:teuthology.orchestra.run.smithi143.stdout:Selecting previously unselected package ceph-test. 2023-11-26T07:14:48.954 INFO:teuthology.orchestra.run.smithi188.stdout:Preparing to unpack .../039-python3-simplegeneric_0.8.1-3_all.deb ... 2023-11-26T07:14:48.964 INFO:teuthology.orchestra.run.smithi143.stdout:Preparing to unpack .../092-ceph-test_18.0.0-7484-g44c2237f-1jammy_amd64.deb ... 2023-11-26T07:14:48.989 INFO:teuthology.orchestra.run.smithi188.stdout:Unpacking python3-simplegeneric (0.8.1-3) ... 2023-11-26T07:14:49.007 INFO:teuthology.orchestra.run.smithi143.stdout:Unpacking ceph-test (18.0.0-7484-g44c2237f-1jammy) ... 2023-11-26T07:14:49.276 INFO:teuthology.orchestra.run.smithi188.stdout:Selecting previously unselected package python3-singledispatch. 2023-11-26T07:14:49.291 INFO:teuthology.orchestra.run.smithi188.stdout:Preparing to unpack .../040-python3-singledispatch_3.4.0.3-3_all.deb ... 2023-11-26T07:14:49.334 INFO:teuthology.orchestra.run.smithi188.stdout:Unpacking python3-singledispatch (3.4.0.3-3) ... 2023-11-26T07:14:49.636 INFO:teuthology.orchestra.run.smithi188.stdout:Selecting previously unselected package python3-webob. 2023-11-26T07:14:49.651 INFO:teuthology.orchestra.run.smithi188.stdout:Preparing to unpack .../041-python3-webob_1%3a1.8.6-1.1_all.deb ... 2023-11-26T07:14:49.695 INFO:teuthology.orchestra.run.smithi188.stdout:Unpacking python3-webob (1:1.8.6-1.1) ... 2023-11-26T07:14:50.073 INFO:teuthology.orchestra.run.smithi188.stdout:Selecting previously unselected package python3-waitress. 2023-11-26T07:14:50.088 INFO:teuthology.orchestra.run.smithi188.stdout:Preparing to unpack .../042-python3-waitress_1.4.4-1.1ubuntu1_all.deb ... 2023-11-26T07:14:50.159 INFO:teuthology.orchestra.run.smithi188.stdout:Unpacking python3-waitress (1.4.4-1.1ubuntu1) ... 2023-11-26T07:14:50.443 INFO:teuthology.orchestra.run.smithi188.stdout:Selecting previously unselected package python3-tempita. 2023-11-26T07:14:50.457 INFO:teuthology.orchestra.run.smithi188.stdout:Preparing to unpack .../043-python3-tempita_0.5.2-6ubuntu1_all.deb ... 2023-11-26T07:14:50.525 INFO:teuthology.orchestra.run.smithi188.stdout:Unpacking python3-tempita (0.5.2-6ubuntu1) ... 2023-11-26T07:14:50.812 INFO:teuthology.orchestra.run.smithi188.stdout:Selecting previously unselected package python3-paste. 2023-11-26T07:14:50.827 INFO:teuthology.orchestra.run.smithi188.stdout:Preparing to unpack .../044-python3-paste_3.5.0+dfsg1-1_all.deb ... 2023-11-26T07:14:50.870 INFO:teuthology.orchestra.run.smithi188.stdout:Unpacking python3-paste (3.5.0+dfsg1-1) ... 2023-11-26T07:14:51.178 INFO:teuthology.orchestra.run.smithi143.stdout:Selecting previously unselected package ceph-volume. 2023-11-26T07:14:51.194 INFO:teuthology.orchestra.run.smithi143.stdout:Preparing to unpack .../093-ceph-volume_18.0.0-7484-g44c2237f-1jammy_all.deb ... 2023-11-26T07:14:51.207 INFO:teuthology.orchestra.run.smithi188.stdout:Selecting previously unselected package python-pastedeploy-tpl. 2023-11-26T07:14:51.221 INFO:teuthology.orchestra.run.smithi188.stdout:Preparing to unpack .../045-python-pastedeploy-tpl_2.1.1-1_all.deb ... 2023-11-26T07:14:51.228 INFO:teuthology.orchestra.run.smithi143.stdout:Unpacking ceph-volume (18.0.0-7484-g44c2237f-1jammy) ... 2023-11-26T07:14:51.265 INFO:teuthology.orchestra.run.smithi188.stdout:Unpacking python-pastedeploy-tpl (2.1.1-1) ... 2023-11-26T07:14:51.535 INFO:teuthology.orchestra.run.smithi188.stdout:Selecting previously unselected package python3-pastedeploy. 2023-11-26T07:14:51.550 INFO:teuthology.orchestra.run.smithi188.stdout:Preparing to unpack .../046-python3-pastedeploy_2.1.1-1_all.deb ... 2023-11-26T07:14:51.582 INFO:teuthology.orchestra.run.smithi143.stdout:Selecting previously unselected package cephfs-mirror. 2023-11-26T07:14:51.593 INFO:teuthology.orchestra.run.smithi188.stdout:Unpacking python3-pastedeploy (2.1.1-1) ... 2023-11-26T07:14:51.597 INFO:teuthology.orchestra.run.smithi143.stdout:Preparing to unpack .../094-cephfs-mirror_18.0.0-7484-g44c2237f-1jammy_amd64.deb ... 2023-11-26T07:14:51.641 INFO:teuthology.orchestra.run.smithi143.stdout:Unpacking cephfs-mirror (18.0.0-7484-g44c2237f-1jammy) ... 2023-11-26T07:14:51.929 INFO:teuthology.orchestra.run.smithi188.stdout:Selecting previously unselected package python3-webtest. 2023-11-26T07:14:51.936 INFO:teuthology.orchestra.run.smithi143.stdout:Selecting previously unselected package python3-colorama. 2023-11-26T07:14:51.944 INFO:teuthology.orchestra.run.smithi188.stdout:Preparing to unpack .../047-python3-webtest_2.0.35-1_all.deb ... 2023-11-26T07:14:51.952 INFO:teuthology.orchestra.run.smithi143.stdout:Preparing to unpack .../095-python3-colorama_0.4.4-1_all.deb ... 2023-11-26T07:14:51.987 INFO:teuthology.orchestra.run.smithi188.stdout:Unpacking python3-webtest (2.0.35-1) ... 2023-11-26T07:14:51.994 INFO:teuthology.orchestra.run.smithi143.stdout:Unpacking python3-colorama (0.4.4-1) ... 2023-11-26T07:14:52.230 INFO:teuthology.orchestra.run.smithi143.stdout:Selecting previously unselected package cephfs-shell. 2023-11-26T07:14:52.246 INFO:teuthology.orchestra.run.smithi143.stdout:Preparing to unpack .../096-cephfs-shell_18.0.0-7484-g44c2237f-1jammy_all.deb ... 2023-11-26T07:14:52.280 INFO:teuthology.orchestra.run.smithi143.stdout:Unpacking cephfs-shell (18.0.0-7484-g44c2237f-1jammy) ... 2023-11-26T07:14:52.324 INFO:teuthology.orchestra.run.smithi188.stdout:Selecting previously unselected package python3-pecan. 2023-11-26T07:14:52.339 INFO:teuthology.orchestra.run.smithi188.stdout:Preparing to unpack .../048-python3-pecan_1.3.3-4ubuntu2_all.deb ... 2023-11-26T07:14:52.382 INFO:teuthology.orchestra.run.smithi188.stdout:Unpacking python3-pecan (1.3.3-4ubuntu2) ... 2023-11-26T07:14:52.533 INFO:teuthology.orchestra.run.smithi143.stdout:Selecting previously unselected package cephfs-top. 2023-11-26T07:14:52.549 INFO:teuthology.orchestra.run.smithi143.stdout:Preparing to unpack .../097-cephfs-top_18.0.0-7484-g44c2237f-1jammy_all.deb ... 2023-11-26T07:14:52.591 INFO:teuthology.orchestra.run.smithi143.stdout:Unpacking cephfs-top (18.0.0-7484-g44c2237f-1jammy) ... 2023-11-26T07:14:52.729 INFO:teuthology.orchestra.run.smithi188.stdout:Selecting previously unselected package python3-certifi. 2023-11-26T07:14:52.743 INFO:teuthology.orchestra.run.smithi188.stdout:Preparing to unpack .../049-python3-certifi_2020.6.20-1_all.deb ... 2023-11-26T07:14:52.787 INFO:teuthology.orchestra.run.smithi188.stdout:Unpacking python3-certifi (2020.6.20-1) ... 2023-11-26T07:14:52.845 INFO:teuthology.orchestra.run.smithi143.stdout:Selecting previously unselected package libcephfs-dev. 2023-11-26T07:14:52.860 INFO:teuthology.orchestra.run.smithi143.stdout:Preparing to unpack .../098-libcephfs-dev_18.0.0-7484-g44c2237f-1jammy_amd64.deb ... 2023-11-26T07:14:52.903 INFO:teuthology.orchestra.run.smithi143.stdout:Unpacking libcephfs-dev (18.0.0-7484-g44c2237f-1jammy) ... 2023-11-26T07:14:53.131 INFO:teuthology.orchestra.run.smithi188.stdout:Selecting previously unselected package python3-idna. 2023-11-26T07:14:53.145 INFO:teuthology.orchestra.run.smithi188.stdout:Preparing to unpack .../050-python3-idna_3.3-1_all.deb ... 2023-11-26T07:14:53.190 INFO:teuthology.orchestra.run.smithi188.stdout:Unpacking python3-idna (3.3-1) ... 2023-11-26T07:14:53.282 INFO:teuthology.orchestra.run.smithi143.stdout:Selecting previously unselected package nvme-cli. 2023-11-26T07:14:53.297 INFO:teuthology.orchestra.run.smithi143.stdout:Preparing to unpack .../099-nvme-cli_1.16-3ubuntu0.1_amd64.deb ... 2023-11-26T07:14:53.339 INFO:teuthology.orchestra.run.smithi143.stdout:Unpacking nvme-cli (1.16-3ubuntu0.1) ... 2023-11-26T07:14:53.518 INFO:teuthology.orchestra.run.smithi188.stdout:Selecting previously unselected package python3-urllib3. 2023-11-26T07:14:53.532 INFO:teuthology.orchestra.run.smithi188.stdout:Preparing to unpack .../051-python3-urllib3_1.26.5-1~exp1ubuntu0.1_all.deb ... 2023-11-26T07:14:53.576 INFO:teuthology.orchestra.run.smithi188.stdout:Unpacking python3-urllib3 (1.26.5-1~exp1ubuntu0.1) ... 2023-11-26T07:14:53.692 INFO:teuthology.orchestra.run.smithi143.stdout:Selecting previously unselected package python-asyncssh-doc. 2023-11-26T07:14:53.707 INFO:teuthology.orchestra.run.smithi143.stdout:Preparing to unpack .../100-python-asyncssh-doc_2.5.0-1_all.deb ... 2023-11-26T07:14:53.753 INFO:teuthology.orchestra.run.smithi143.stdout:Unpacking python-asyncssh-doc (2.5.0-1) ... 2023-11-26T07:14:53.887 INFO:teuthology.orchestra.run.smithi188.stdout:Selecting previously unselected package python3-requests. 2023-11-26T07:14:53.902 INFO:teuthology.orchestra.run.smithi188.stdout:Preparing to unpack .../052-python3-requests_2.25.1+dfsg-2ubuntu0.1_all.deb ... 2023-11-26T07:14:53.945 INFO:teuthology.orchestra.run.smithi188.stdout:Unpacking python3-requests (2.25.1+dfsg-2ubuntu0.1) ... 2023-11-26T07:14:54.173 INFO:teuthology.orchestra.run.smithi143.stdout:Selecting previously unselected package python3-iniconfig. 2023-11-26T07:14:54.188 INFO:teuthology.orchestra.run.smithi143.stdout:Preparing to unpack .../101-python3-iniconfig_1.1.1-2_all.deb ... 2023-11-26T07:14:54.223 INFO:teuthology.orchestra.run.smithi143.stdout:Unpacking python3-iniconfig (1.1.1-2) ... 2023-11-26T07:14:54.273 INFO:teuthology.orchestra.run.smithi188.stdout:Selecting previously unselected package python3-werkzeug. 2023-11-26T07:14:54.289 INFO:teuthology.orchestra.run.smithi188.stdout:Preparing to unpack .../053-python3-werkzeug_2.0.2+dfsg1-1ubuntu0.22.04.1_all.deb ... 2023-11-26T07:14:54.332 INFO:teuthology.orchestra.run.smithi188.stdout:Unpacking python3-werkzeug (2.0.2+dfsg1-1ubuntu0.22.04.1) ... 2023-11-26T07:14:54.493 INFO:teuthology.orchestra.run.smithi143.stdout:Selecting previously unselected package python3-pastescript. 2023-11-26T07:14:54.508 INFO:teuthology.orchestra.run.smithi143.stdout:Preparing to unpack .../102-python3-pastescript_2.0.2-4_all.deb ... 2023-11-26T07:14:54.551 INFO:teuthology.orchestra.run.smithi143.stdout:Unpacking python3-pastescript (2.0.2-4) ... 2023-11-26T07:14:54.652 INFO:teuthology.orchestra.run.smithi188.stdout:Selecting previously unselected package ceph-mgr-modules-core. 2023-11-26T07:14:54.667 INFO:teuthology.orchestra.run.smithi188.stdout:Preparing to unpack .../054-ceph-mgr-modules-core_18.0.0-7484-g44c2237f-1jammy_all.deb ... 2023-11-26T07:14:54.710 INFO:teuthology.orchestra.run.smithi188.stdout:Unpacking ceph-mgr-modules-core (18.0.0-7484-g44c2237f-1jammy) ... 2023-11-26T07:14:54.889 INFO:teuthology.orchestra.run.smithi143.stdout:Selecting previously unselected package python3-pluggy. 2023-11-26T07:14:54.904 INFO:teuthology.orchestra.run.smithi143.stdout:Preparing to unpack .../103-python3-pluggy_0.13.0-7.1_all.deb ... 2023-11-26T07:14:54.946 INFO:teuthology.orchestra.run.smithi143.stdout:Unpacking python3-pluggy (0.13.0-7.1) ... 2023-11-26T07:14:55.148 INFO:teuthology.orchestra.run.smithi188.stdout:Selecting previously unselected package libsqlite3-mod-ceph. 2023-11-26T07:14:55.163 INFO:teuthology.orchestra.run.smithi188.stdout:Preparing to unpack .../055-libsqlite3-mod-ceph_18.0.0-7484-g44c2237f-1jammy_amd64.deb ... 2023-11-26T07:14:55.206 INFO:teuthology.orchestra.run.smithi188.stdout:Unpacking libsqlite3-mod-ceph (18.0.0-7484-g44c2237f-1jammy) ... 2023-11-26T07:14:55.283 INFO:teuthology.orchestra.run.smithi143.stdout:Selecting previously unselected package python3-psutil. 2023-11-26T07:14:55.298 INFO:teuthology.orchestra.run.smithi143.stdout:Preparing to unpack .../104-python3-psutil_5.9.0-1build1_amd64.deb ... 2023-11-26T07:14:55.341 INFO:teuthology.orchestra.run.smithi143.stdout:Unpacking python3-psutil (5.9.0-1build1) ... 2023-11-26T07:14:55.484 INFO:teuthology.orchestra.run.smithi188.stdout:Selecting previously unselected package python3-openssl. 2023-11-26T07:14:55.499 INFO:teuthology.orchestra.run.smithi188.stdout:Preparing to unpack .../056-python3-openssl_21.0.0-1_all.deb ... 2023-11-26T07:14:55.543 INFO:teuthology.orchestra.run.smithi188.stdout:Unpacking python3-openssl (21.0.0-1) ... 2023-11-26T07:14:55.703 INFO:teuthology.orchestra.run.smithi143.stdout:Selecting previously unselected package python3-py. 2023-11-26T07:14:55.718 INFO:teuthology.orchestra.run.smithi143.stdout:Preparing to unpack .../105-python3-py_1.10.0-1_all.deb ... 2023-11-26T07:14:55.760 INFO:teuthology.orchestra.run.smithi143.stdout:Unpacking python3-py (1.10.0-1) ... 2023-11-26T07:14:55.912 INFO:teuthology.orchestra.run.smithi188.stdout:Selecting previously unselected package ceph-mgr. 2023-11-26T07:14:55.928 INFO:teuthology.orchestra.run.smithi188.stdout:Preparing to unpack .../057-ceph-mgr_18.0.0-7484-g44c2237f-1jammy_amd64.deb ... 2023-11-26T07:14:55.962 INFO:teuthology.orchestra.run.smithi188.stdout:Unpacking ceph-mgr (18.0.0-7484-g44c2237f-1jammy) ... 2023-11-26T07:14:56.089 INFO:teuthology.orchestra.run.smithi143.stdout:Selecting previously unselected package python3-pygments. 2023-11-26T07:14:56.104 INFO:teuthology.orchestra.run.smithi143.stdout:Preparing to unpack .../106-python3-pygments_2.11.2+dfsg-2_all.deb ... 2023-11-26T07:14:56.147 INFO:teuthology.orchestra.run.smithi143.stdout:Unpacking python3-pygments (2.11.2+dfsg-2) ... 2023-11-26T07:14:56.374 INFO:teuthology.orchestra.run.smithi188.stdout:Selecting previously unselected package ceph-mon. 2023-11-26T07:14:56.389 INFO:teuthology.orchestra.run.smithi188.stdout:Preparing to unpack .../058-ceph-mon_18.0.0-7484-g44c2237f-1jammy_amd64.deb ... 2023-11-26T07:14:56.432 INFO:teuthology.orchestra.run.smithi188.stdout:Unpacking ceph-mon (18.0.0-7484-g44c2237f-1jammy) ... 2023-11-26T07:14:56.652 INFO:teuthology.orchestra.run.smithi143.stdout:Selecting previously unselected package python3-pyinotify. 2023-11-26T07:14:56.667 INFO:teuthology.orchestra.run.smithi143.stdout:Preparing to unpack .../107-python3-pyinotify_0.9.6-1.3_all.deb ... 2023-11-26T07:14:56.710 INFO:teuthology.orchestra.run.smithi143.stdout:Unpacking python3-pyinotify (0.9.6-1.3) ... 2023-11-26T07:14:57.013 INFO:teuthology.orchestra.run.smithi143.stdout:Selecting previously unselected package python3-toml. 2023-11-26T07:14:57.028 INFO:teuthology.orchestra.run.smithi143.stdout:Preparing to unpack .../108-python3-toml_0.10.2-1_all.deb ... 2023-11-26T07:14:57.071 INFO:teuthology.orchestra.run.smithi143.stdout:Unpacking python3-toml (0.10.2-1) ... 2023-11-26T07:14:57.252 INFO:teuthology.orchestra.run.smithi188.stdout:Selecting previously unselected package ceph-osd. 2023-11-26T07:14:57.267 INFO:teuthology.orchestra.run.smithi188.stdout:Preparing to unpack .../059-ceph-osd_18.0.0-7484-g44c2237f-1jammy_amd64.deb ... 2023-11-26T07:14:57.310 INFO:teuthology.orchestra.run.smithi188.stdout:Unpacking ceph-osd (18.0.0-7484-g44c2237f-1jammy) ... 2023-11-26T07:14:57.358 INFO:teuthology.orchestra.run.smithi143.stdout:Selecting previously unselected package python3-pytest. 2023-11-26T07:14:57.373 INFO:teuthology.orchestra.run.smithi143.stdout:Preparing to unpack .../109-python3-pytest_6.2.5-1ubuntu2_all.deb ... 2023-11-26T07:14:57.407 INFO:teuthology.orchestra.run.smithi143.stdout:Unpacking python3-pytest (6.2.5-1ubuntu2) ... 2023-11-26T07:14:57.753 INFO:teuthology.orchestra.run.smithi143.stdout:Selecting previously unselected package python3-simplejson. 2023-11-26T07:14:57.769 INFO:teuthology.orchestra.run.smithi143.stdout:Preparing to unpack .../110-python3-simplejson_3.17.6-1build1_amd64.deb ... 2023-11-26T07:14:57.811 INFO:teuthology.orchestra.run.smithi143.stdout:Unpacking python3-simplejson (3.17.6-1build1) ... 2023-11-26T07:14:58.123 INFO:teuthology.orchestra.run.smithi143.stdout:Selecting previously unselected package qttranslations5-l10n. 2023-11-26T07:14:58.138 INFO:teuthology.orchestra.run.smithi143.stdout:Preparing to unpack .../111-qttranslations5-l10n_5.15.3-1_all.deb ... 2023-11-26T07:14:58.189 INFO:teuthology.orchestra.run.smithi143.stdout:Unpacking qttranslations5-l10n (5.15.3-1) ... 2023-11-26T07:14:58.547 INFO:teuthology.orchestra.run.smithi188.stdout:Selecting previously unselected package ceph. 2023-11-26T07:14:58.562 INFO:teuthology.orchestra.run.smithi188.stdout:Preparing to unpack .../060-ceph_18.0.0-7484-g44c2237f-1jammy_amd64.deb ... 2023-11-26T07:14:58.605 INFO:teuthology.orchestra.run.smithi188.stdout:Unpacking ceph (18.0.0-7484-g44c2237f-1jammy) ... 2023-11-26T07:14:58.918 INFO:teuthology.orchestra.run.smithi143.stdout:Selecting previously unselected package radosgw. 2023-11-26T07:14:58.932 INFO:teuthology.orchestra.run.smithi143.stdout:Preparing to unpack .../112-radosgw_18.0.0-7484-g44c2237f-1jammy_amd64.deb ... 2023-11-26T07:14:58.950 INFO:teuthology.orchestra.run.smithi188.stdout:Selecting previously unselected package ceph-fuse. 2023-11-26T07:14:58.965 INFO:teuthology.orchestra.run.smithi188.stdout:Preparing to unpack .../061-ceph-fuse_18.0.0-7484-g44c2237f-1jammy_amd64.deb ... 2023-11-26T07:14:58.968 INFO:teuthology.orchestra.run.smithi143.stdout:Unpacking radosgw (18.0.0-7484-g44c2237f-1jammy) ... 2023-11-26T07:14:59.025 INFO:teuthology.orchestra.run.smithi188.stdout:Unpacking ceph-fuse (18.0.0-7484-g44c2237f-1jammy) ... 2023-11-26T07:14:59.403 INFO:teuthology.orchestra.run.smithi188.stdout:Selecting previously unselected package ceph-mds. 2023-11-26T07:14:59.418 INFO:teuthology.orchestra.run.smithi188.stdout:Preparing to unpack .../062-ceph-mds_18.0.0-7484-g44c2237f-1jammy_amd64.deb ... 2023-11-26T07:14:59.461 INFO:teuthology.orchestra.run.smithi188.stdout:Unpacking ceph-mds (18.0.0-7484-g44c2237f-1jammy) ... 2023-11-26T07:14:59.555 INFO:teuthology.orchestra.run.smithi143.stdout:Selecting previously unselected package rbd-fuse. 2023-11-26T07:14:59.570 INFO:teuthology.orchestra.run.smithi143.stdout:Preparing to unpack .../113-rbd-fuse_18.0.0-7484-g44c2237f-1jammy_amd64.deb ... 2023-11-26T07:14:59.613 INFO:teuthology.orchestra.run.smithi143.stdout:Unpacking rbd-fuse (18.0.0-7484-g44c2237f-1jammy) ... 2023-11-26T07:14:59.831 INFO:teuthology.orchestra.run.smithi188.stdout:Selecting previously unselected package cephadm. 2023-11-26T07:14:59.846 INFO:teuthology.orchestra.run.smithi188.stdout:Preparing to unpack .../063-cephadm_18.0.0-7484-g44c2237f-1jammy_amd64.deb ... 2023-11-26T07:14:59.905 INFO:teuthology.orchestra.run.smithi143.stdout:Setting up python3-iniconfig (1.1.1-2) ... 2023-11-26T07:14:59.922 INFO:teuthology.orchestra.run.smithi188.stdout:Unpacking cephadm (18.0.0-7484-g44c2237f-1jammy) ... 2023-11-26T07:15:00.185 INFO:teuthology.orchestra.run.smithi143.stdout:Setting up libdouble-conversion3:amd64 (3.1.7-4) ... 2023-11-26T07:15:00.211 INFO:teuthology.orchestra.run.smithi188.stdout:Selecting previously unselected package python3-asyncssh. 2023-11-26T07:15:00.225 INFO:teuthology.orchestra.run.smithi188.stdout:Preparing to unpack .../064-python3-asyncssh_2.5.0-1_all.deb ... 2023-11-26T07:15:00.261 INFO:teuthology.orchestra.run.smithi188.stdout:Unpacking python3-asyncssh (2.5.0-1) ... 2023-11-26T07:15:00.295 INFO:teuthology.orchestra.run.smithi143.stdout:Setting up nvme-cli (1.16-3ubuntu0.1) ... 2023-11-26T07:15:00.556 INFO:teuthology.orchestra.run.smithi143.stdout:Created symlink /etc/systemd/system/default.target.wants/nvmefc-boot-connections.service -> /lib/systemd/system/nvmefc-boot-connections.service. 2023-11-26T07:15:00.589 INFO:teuthology.orchestra.run.smithi188.stdout:Selecting previously unselected package python-babel-localedata. 2023-11-26T07:15:00.603 INFO:teuthology.orchestra.run.smithi188.stdout:Preparing to unpack .../065-python-babel-localedata_2.8.0+dfsg.1-7_all.deb ... 2023-11-26T07:15:00.647 INFO:teuthology.orchestra.run.smithi188.stdout:Unpacking python-babel-localedata (2.8.0+dfsg.1-7) ... 2023-11-26T07:15:00.944 INFO:teuthology.orchestra.run.smithi143.stdout:Created symlink /etc/systemd/system/default.target.wants/nvmf-autoconnect.service -> /lib/systemd/system/nvmf-autoconnect.service. 2023-11-26T07:15:01.560 INFO:teuthology.orchestra.run.smithi143.stdout:nvmf-connect.target is a disabled or a static unit, not starting it. 2023-11-26T07:15:01.617 INFO:teuthology.orchestra.run.smithi143.stdout:Could not execute systemctl: at /usr/bin/deb-systemd-invoke line 142. 2023-11-26T07:15:01.620 INFO:teuthology.orchestra.run.smithi188.stdout:Selecting previously unselected package python3-babel. 2023-11-26T07:15:01.634 INFO:teuthology.orchestra.run.smithi188.stdout:Preparing to unpack .../066-python3-babel_2.8.0+dfsg.1-7_all.deb ... 2023-11-26T07:15:01.663 INFO:teuthology.orchestra.run.smithi143.stdout:Setting up cephadm (18.0.0-7484-g44c2237f-1jammy) ... 2023-11-26T07:15:01.677 INFO:teuthology.orchestra.run.smithi188.stdout:Unpacking python3-babel (2.8.0+dfsg.1-7) ... 2023-11-26T07:15:02.164 INFO:teuthology.orchestra.run.smithi188.stdout:Selecting previously unselected package python3-jinja2. 2023-11-26T07:15:02.178 INFO:teuthology.orchestra.run.smithi188.stdout:Preparing to unpack .../067-python3-jinja2_3.0.3-1_all.deb ... 2023-11-26T07:15:02.217 INFO:teuthology.orchestra.run.smithi143.stdout:Adding system user cephadm....done 2023-11-26T07:15:02.239 INFO:teuthology.orchestra.run.smithi188.stdout:Unpacking python3-jinja2 (3.0.3-1) ... 2023-11-26T07:15:02.378 INFO:teuthology.orchestra.run.smithi143.stdout:Setting up python3-waitress (1.4.4-1.1ubuntu1) ... 2023-11-26T07:15:02.558 INFO:teuthology.orchestra.run.smithi188.stdout:Selecting previously unselected package ceph-mgr-cephadm. 2023-11-26T07:15:02.572 INFO:teuthology.orchestra.run.smithi188.stdout:Preparing to unpack .../068-ceph-mgr-cephadm_18.0.0-7484-g44c2237f-1jammy_all.deb ... 2023-11-26T07:15:02.617 INFO:teuthology.orchestra.run.smithi188.stdout:Unpacking ceph-mgr-cephadm (18.0.0-7484-g44c2237f-1jammy) ... 2023-11-26T07:15:02.654 INFO:teuthology.orchestra.run.smithi143.stdout:Setting up python3-jaraco.classes (3.2.1-3) ... 2023-11-26T07:15:02.923 INFO:teuthology.orchestra.run.smithi143.stdout:Setting up python-asyncssh-doc (2.5.0-1) ... 2023-11-26T07:15:02.945 INFO:teuthology.orchestra.run.smithi188.stdout:Selecting previously unselected package python3-bcrypt. 2023-11-26T07:15:02.959 INFO:teuthology.orchestra.run.smithi188.stdout:Preparing to unpack .../069-python3-bcrypt_3.2.0-1build1_amd64.deb ... 2023-11-26T07:15:03.004 INFO:teuthology.orchestra.run.smithi188.stdout:Unpacking python3-bcrypt (3.2.0-1build1) ... 2023-11-26T07:15:03.023 INFO:teuthology.orchestra.run.smithi143.stdout:Setting up python3-jaraco.functools (3.4.0-2) ... 2023-11-26T07:15:03.265 INFO:teuthology.orchestra.run.smithi188.stdout:Selecting previously unselected package python3-repoze.lru. 2023-11-26T07:15:08.165 INFO:teuthology.orchestra.run.smithi143.stdout:Setting up python3-repoze.lru (0.7-2) ... 2023-11-26T07:15:08.165 INFO:teuthology.orchestra.run.smithi143.stdout:Setting up liboath0:amd64 (2.6.7-3build1) ... 2023-11-26T07:15:08.165 INFO:teuthology.orchestra.run.smithi143.stdout:Setting up python3-py (1.10.0-1) ... 2023-11-26T07:15:08.166 INFO:teuthology.orchestra.run.smithi143.stdout:Setting up python3-joblib (0.17.0-4ubuntu1) ... 2023-11-26T07:15:08.166 INFO:teuthology.orchestra.run.smithi143.stdout:Setting up python3-cachetools (5.0.0-1) ... 2023-11-26T07:15:08.166 INFO:teuthology.orchestra.run.smithi143.stdout:Setting up python3-colorama (0.4.4-1) ... 2023-11-26T07:15:08.166 INFO:teuthology.orchestra.run.smithi143.stdout:Setting up python3-openssl (21.0.0-1) ... 2023-11-26T07:15:08.166 INFO:teuthology.orchestra.run.smithi143.stdout:Setting up python3-bcrypt (3.2.0-1build1) ... 2023-11-26T07:15:08.166 INFO:teuthology.orchestra.run.smithi143.stdout:Setting up python3-pyinotify (0.9.6-1.3) ... 2023-11-26T07:15:08.166 INFO:teuthology.orchestra.run.smithi143.stdout:Setting up python3-threadpoolctl (3.1.0-1) ... 2023-11-26T07:15:08.166 INFO:teuthology.orchestra.run.smithi143.stdout:Setting up python3-ceph-argparse (18.0.0-7484-g44c2237f-1jammy) ... 2023-11-26T07:15:08.166 INFO:teuthology.orchestra.run.smithi143.stdout:Setting up python3-sklearn-lib:amd64 (0.23.2-5ubuntu6) ... 2023-11-26T07:15:08.166 INFO:teuthology.orchestra.run.smithi143.stdout:Setting up python3-pyperclip (1.8.2-2) ... 2023-11-26T07:15:08.166 INFO:teuthology.orchestra.run.smithi143.stdout:Setting up python3-markupsafe (2.0.1-2build1) ... 2023-11-26T07:15:08.166 INFO:teuthology.orchestra.run.smithi143.stdout:Setting up libpcre2-16-0:amd64 (10.39-3ubuntu0.1) ... 2023-11-26T07:15:08.166 INFO:teuthology.orchestra.run.smithi143.stdout:Setting up python3-psutil (5.9.0-1build1) ... 2023-11-26T07:15:08.166 INFO:teuthology.orchestra.run.smithi143.stdout:Setting up python-babel-localedata (2.8.0+dfsg.1-7) ... 2023-11-26T07:15:08.167 INFO:teuthology.orchestra.run.smithi143.stdout:Setting up python3-natsort (8.0.2-1) ... 2023-11-26T07:15:08.167 INFO:teuthology.orchestra.run.smithi143.stdout:Setting up python3-routes (2.5.1-1ubuntu1) ... 2023-11-26T07:15:08.167 INFO:teuthology.orchestra.run.smithi188.stdout:Preparing to unpack .../070-python3-repoze.lru_0.7-2_all.deb ... 2023-11-26T07:15:08.168 INFO:teuthology.orchestra.run.smithi188.stdout:Unpacking python3-repoze.lru (0.7-2) ... 2023-11-26T07:15:08.168 INFO:teuthology.orchestra.run.smithi188.stdout:Selecting previously unselected package python3-routes. 2023-11-26T07:15:08.168 INFO:teuthology.orchestra.run.smithi188.stdout:Preparing to unpack .../071-python3-routes_2.5.1-1ubuntu1_all.deb ... 2023-11-26T07:15:08.168 INFO:teuthology.orchestra.run.smithi188.stdout:Unpacking python3-routes (2.5.1-1ubuntu1) ... 2023-11-26T07:15:08.168 INFO:teuthology.orchestra.run.smithi188.stdout:Selecting previously unselected package ceph-mgr-dashboard. 2023-11-26T07:15:08.168 INFO:teuthology.orchestra.run.smithi188.stdout:Preparing to unpack .../072-ceph-mgr-dashboard_18.0.0-7484-g44c2237f-1jammy_all.deb ... 2023-11-26T07:15:08.168 INFO:teuthology.orchestra.run.smithi188.stdout:Unpacking ceph-mgr-dashboard (18.0.0-7484-g44c2237f-1jammy) ... 2023-11-26T07:15:08.168 INFO:teuthology.orchestra.run.smithi188.stdout:Selecting previously unselected package python3-sklearn-lib:amd64. 2023-11-26T07:15:08.168 INFO:teuthology.orchestra.run.smithi188.stdout:Preparing to unpack .../073-python3-sklearn-lib_0.23.2-5ubuntu6_amd64.deb ... 2023-11-26T07:15:08.168 INFO:teuthology.orchestra.run.smithi188.stdout:Unpacking python3-sklearn-lib:amd64 (0.23.2-5ubuntu6) ... 2023-11-26T07:15:08.168 INFO:teuthology.orchestra.run.smithi188.stdout:Selecting previously unselected package python3-joblib. 2023-11-26T07:15:08.168 INFO:teuthology.orchestra.run.smithi188.stdout:Preparing to unpack .../074-python3-joblib_0.17.0-4ubuntu1_all.deb ... 2023-11-26T07:15:08.168 INFO:teuthology.orchestra.run.smithi188.stdout:Unpacking python3-joblib (0.17.0-4ubuntu1) ... 2023-11-26T07:15:08.168 INFO:teuthology.orchestra.run.smithi188.stdout:Selecting previously unselected package python3-threadpoolctl. 2023-11-26T07:15:08.169 INFO:teuthology.orchestra.run.smithi188.stdout:Preparing to unpack .../075-python3-threadpoolctl_3.1.0-1_all.deb ... 2023-11-26T07:15:08.169 INFO:teuthology.orchestra.run.smithi188.stdout:Unpacking python3-threadpoolctl (3.1.0-1) ... 2023-11-26T07:15:08.169 INFO:teuthology.orchestra.run.smithi188.stdout:Selecting previously unselected package python3-sklearn. 2023-11-26T07:15:08.169 INFO:teuthology.orchestra.run.smithi188.stdout:Preparing to unpack .../076-python3-sklearn_0.23.2-5ubuntu6_all.deb ... 2023-11-26T07:15:08.169 INFO:teuthology.orchestra.run.smithi188.stdout:Unpacking python3-sklearn (0.23.2-5ubuntu6) ... 2023-11-26T07:15:08.169 INFO:teuthology.orchestra.run.smithi188.stdout:Selecting previously unselected package ceph-mgr-diskprediction-local. 2023-11-26T07:15:08.169 INFO:teuthology.orchestra.run.smithi188.stdout:Preparing to unpack .../077-ceph-mgr-diskprediction-local_18.0.0-7484-g44c2237f-1jammy_all.deb ... 2023-11-26T07:15:08.169 INFO:teuthology.orchestra.run.smithi188.stdout:Unpacking ceph-mgr-diskprediction-local (18.0.0-7484-g44c2237f-1jammy) ... 2023-11-26T07:15:08.169 INFO:teuthology.orchestra.run.smithi188.stdout:Selecting previously unselected package python3-cachetools. 2023-11-26T07:15:08.169 INFO:teuthology.orchestra.run.smithi188.stdout:Preparing to unpack .../078-python3-cachetools_5.0.0-1_all.deb ... 2023-11-26T07:15:08.169 INFO:teuthology.orchestra.run.smithi188.stdout:Unpacking python3-cachetools (5.0.0-1) ... 2023-11-26T07:15:08.169 INFO:teuthology.orchestra.run.smithi188.stdout:Selecting previously unselected package python3-pyasn1. 2023-11-26T07:15:08.182 INFO:teuthology.orchestra.run.smithi188.stdout:Preparing to unpack .../079-python3-pyasn1_0.4.8-1_all.deb ... 2023-11-26T07:15:08.182 INFO:teuthology.orchestra.run.smithi143.stdout:Setting up python3-simplejson (3.17.6-1build1) ... 2023-11-26T07:15:08.224 INFO:teuthology.orchestra.run.smithi188.stdout:Unpacking python3-pyasn1 (0.4.8-1) ... 2023-11-26T07:15:08.492 INFO:teuthology.orchestra.run.smithi143.stdout:Setting up python3-pygments (2.11.2+dfsg-2) ... 2023-11-26T07:15:08.544 INFO:teuthology.orchestra.run.smithi188.stdout:Selecting previously unselected package python3-pyasn1-modules. 2023-11-26T07:15:08.560 INFO:teuthology.orchestra.run.smithi188.stdout:Preparing to unpack .../080-python3-pyasn1-modules_0.2.1-1_all.deb ... 2023-11-26T07:15:08.594 INFO:teuthology.orchestra.run.smithi188.stdout:Unpacking python3-pyasn1-modules (0.2.1-1) ... 2023-11-26T07:15:08.931 INFO:teuthology.orchestra.run.smithi188.stdout:Selecting previously unselected package python3-rsa. 2023-11-26T07:15:08.946 INFO:teuthology.orchestra.run.smithi188.stdout:Preparing to unpack .../081-python3-rsa_4.8-1_all.deb ... 2023-11-26T07:15:08.980 INFO:teuthology.orchestra.run.smithi188.stdout:Unpacking python3-rsa (4.8-1) ... 2023-11-26T07:15:09.169 INFO:teuthology.orchestra.run.smithi143.stdout:Setting up python3-tempita (0.5.2-6ubuntu1) ... 2023-11-26T07:15:09.317 INFO:teuthology.orchestra.run.smithi188.stdout:Selecting previously unselected package python3-google-auth. 2023-11-26T07:15:09.333 INFO:teuthology.orchestra.run.smithi188.stdout:Preparing to unpack .../082-python3-google-auth_1.5.1-3_all.deb ... 2023-11-26T07:15:09.375 INFO:teuthology.orchestra.run.smithi188.stdout:Unpacking python3-google-auth (1.5.1-3) ... 2023-11-26T07:15:09.437 INFO:teuthology.orchestra.run.smithi143.stdout:Setting up python-pastedeploy-tpl (2.1.1-1) ... 2023-11-26T07:15:09.563 INFO:teuthology.orchestra.run.smithi143.stdout:Setting up qttranslations5-l10n (5.15.3-1) ... 2023-11-26T07:15:09.689 INFO:teuthology.orchestra.run.smithi143.stdout:Setting up python3-wcwidth (0.2.5+dfsg1-1) ... 2023-11-26T07:15:09.805 INFO:teuthology.orchestra.run.smithi188.stdout:Selecting previously unselected package python3-requests-oauthlib. 2023-11-26T07:15:09.821 INFO:teuthology.orchestra.run.smithi188.stdout:Preparing to unpack .../083-python3-requests-oauthlib_1.3.0+ds-0.1_all.deb ... 2023-11-26T07:15:09.863 INFO:teuthology.orchestra.run.smithi188.stdout:Unpacking python3-requests-oauthlib (1.3.0+ds-0.1) ... 2023-11-26T07:15:10.015 INFO:teuthology.orchestra.run.smithi143.stdout:Setting up python3-asyncssh (2.5.0-1) ... 2023-11-26T07:15:10.150 INFO:teuthology.orchestra.run.smithi188.stdout:Selecting previously unselected package python3-websocket. 2023-11-26T07:15:10.165 INFO:teuthology.orchestra.run.smithi188.stdout:Preparing to unpack .../084-python3-websocket_1.2.3-1_all.deb ... 2023-11-26T07:15:10.208 INFO:teuthology.orchestra.run.smithi188.stdout:Unpacking python3-websocket (1.2.3-1) ... 2023-11-26T07:15:10.408 INFO:teuthology.orchestra.run.smithi143.stdout:Setting up python3-certifi (2020.6.20-1) ... 2023-11-26T07:15:10.562 INFO:teuthology.orchestra.run.smithi188.stdout:Selecting previously unselected package python3-kubernetes. 2023-11-26T07:15:10.577 INFO:teuthology.orchestra.run.smithi188.stdout:Preparing to unpack .../085-python3-kubernetes_12.0.1-1ubuntu1_all.deb ... 2023-11-26T07:15:10.620 INFO:teuthology.orchestra.run.smithi188.stdout:Unpacking python3-kubernetes (12.0.1-1ubuntu1) ... 2023-11-26T07:15:10.651 INFO:teuthology.orchestra.run.smithi143.stdout:Setting up python3-paste (3.5.0+dfsg1-1) ... 2023-11-26T07:15:11.044 INFO:teuthology.orchestra.run.smithi143.stdout:Setting up python3-cheroot (8.5.2+ds1-1ubuntu3.1) ... 2023-11-26T07:15:11.334 INFO:teuthology.orchestra.run.smithi188.stdout:Selecting previously unselected package ceph-mgr-k8sevents. 2023-11-26T07:15:11.346 INFO:teuthology.orchestra.run.smithi143.stdout:Setting up python3-cmd2 (2.3.3-1) ... 2023-11-26T07:15:11.348 INFO:teuthology.orchestra.run.smithi188.stdout:Preparing to unpack .../086-ceph-mgr-k8sevents_18.0.0-7484-g44c2237f-1jammy_all.deb ... 2023-11-26T07:15:11.392 INFO:teuthology.orchestra.run.smithi188.stdout:Unpacking ceph-mgr-k8sevents (18.0.0-7484-g44c2237f-1jammy) ... 2023-11-26T07:15:11.672 INFO:teuthology.orchestra.run.smithi143.stdout:Setting up python3-werkzeug (2.0.2+dfsg1-1ubuntu0.22.04.1) ... 2023-11-26T07:15:11.736 INFO:teuthology.orchestra.run.smithi188.stdout:Selecting previously unselected package libonig5:amd64. 2023-11-26T07:15:11.750 INFO:teuthology.orchestra.run.smithi188.stdout:Preparing to unpack .../087-libonig5_6.9.7.1-2build1_amd64.deb ... 2023-11-26T07:15:11.795 INFO:teuthology.orchestra.run.smithi188.stdout:Unpacking libonig5:amd64 (6.9.7.1-2build1) ... 2023-11-26T07:15:12.040 INFO:teuthology.orchestra.run.smithi143.stdout:Setting up python3-jaraco.text (3.6.0-2) ... 2023-11-26T07:15:12.081 INFO:teuthology.orchestra.run.smithi188.stdout:Selecting previously unselected package libjq1:amd64. 2023-11-26T07:15:12.095 INFO:teuthology.orchestra.run.smithi188.stdout:Preparing to unpack .../088-libjq1_1.6-2.1ubuntu3_amd64.deb ... 2023-11-26T07:15:12.164 INFO:teuthology.orchestra.run.smithi188.stdout:Unpacking libjq1:amd64 (1.6-2.1ubuntu3) ... 2023-11-26T07:15:12.341 INFO:teuthology.orchestra.run.smithi143.stdout:Setting up socat (1.7.4.1-3ubuntu4) ... 2023-11-26T07:15:12.383 INFO:teuthology.orchestra.run.smithi188.stdout:Selecting previously unselected package jq. 2023-11-26T07:15:12.397 INFO:teuthology.orchestra.run.smithi188.stdout:Preparing to unpack .../089-jq_1.6-2.1ubuntu3_amd64.deb ... 2023-11-26T07:15:12.442 INFO:teuthology.orchestra.run.smithi188.stdout:Unpacking jq (1.6-2.1ubuntu3) ... 2023-11-26T07:15:12.459 INFO:teuthology.orchestra.run.smithi143.stdout:Setting up python3-ceph-common (18.0.0-7484-g44c2237f-1jammy) ... 2023-11-26T07:15:12.695 INFO:teuthology.orchestra.run.smithi188.stdout:Selecting previously unselected package socat. 2023-11-26T07:15:12.709 INFO:teuthology.orchestra.run.smithi188.stdout:Preparing to unpack .../090-socat_1.7.4.1-3ubuntu4_amd64.deb ... 2023-11-26T07:15:12.744 INFO:teuthology.orchestra.run.smithi143.stdout:Setting up python3-idna (3.3-1) ... 2023-11-26T07:15:12.770 INFO:teuthology.orchestra.run.smithi188.stdout:Unpacking socat (1.7.4.1-3ubuntu4) ... 2023-11-26T07:15:13.048 INFO:teuthology.orchestra.run.smithi188.stdout:Selecting previously unselected package xmlstarlet. 2023-11-26T07:15:13.062 INFO:teuthology.orchestra.run.smithi188.stdout:Preparing to unpack .../091-xmlstarlet_1.6.1-2.1_amd64.deb ... 2023-11-26T07:15:13.098 INFO:teuthology.orchestra.run.smithi188.stdout:Unpacking xmlstarlet (1.6.1-2.1) ... 2023-11-26T07:15:13.120 INFO:teuthology.orchestra.run.smithi143.stdout:Setting up python3-sklearn (0.23.2-5ubuntu6) ... 2023-11-26T07:15:13.460 INFO:teuthology.orchestra.run.smithi188.stdout:Selecting previously unselected package ceph-test. 2023-11-26T07:15:13.475 INFO:teuthology.orchestra.run.smithi188.stdout:Preparing to unpack .../092-ceph-test_18.0.0-7484-g44c2237f-1jammy_amd64.deb ... 2023-11-26T07:15:13.519 INFO:teuthology.orchestra.run.smithi188.stdout:Unpacking ceph-test (18.0.0-7484-g44c2237f-1jammy) ... 2023-11-26T07:15:14.343 INFO:teuthology.orchestra.run.smithi143.stdout:Setting up libqt5core5a:amd64 (5.15.3+dfsg-2ubuntu0.2) ... 2023-11-26T07:15:14.502 INFO:teuthology.orchestra.run.smithi143.stdout:Setting up python3-toml (0.10.2-1) ... 2023-11-26T07:15:14.779 INFO:teuthology.orchestra.run.smithi143.stdout:Setting up librdkafka1:amd64 (1.8.0-1build1) ... 2023-11-26T07:15:14.904 INFO:teuthology.orchestra.run.smithi143.stdout:Setting up xmlstarlet (1.6.1-2.1) ... 2023-11-26T07:15:15.031 INFO:teuthology.orchestra.run.smithi143.stdout:Setting up python3-urllib3 (1.26.5-1~exp1ubuntu0.1) ... 2023-11-26T07:15:15.324 INFO:teuthology.orchestra.run.smithi143.stdout:Setting up python3-pluggy (0.13.0-7.1) ... 2023-11-26T07:15:15.389 INFO:teuthology.orchestra.run.smithi188.stdout:Selecting previously unselected package ceph-volume. 2023-11-26T07:15:15.405 INFO:teuthology.orchestra.run.smithi188.stdout:Preparing to unpack .../093-ceph-volume_18.0.0-7484-g44c2237f-1jammy_all.deb ... 2023-11-26T07:15:15.447 INFO:teuthology.orchestra.run.smithi188.stdout:Unpacking ceph-volume (18.0.0-7484-g44c2237f-1jammy) ... 2023-11-26T07:15:15.567 INFO:teuthology.orchestra.run.smithi143.stdout:Setting up python3-zc.lockfile (2.0-1) ... 2023-11-26T07:15:15.818 INFO:teuthology.orchestra.run.smithi143.stdout:Setting up libqt5dbus5:amd64 (5.15.3+dfsg-2ubuntu0.2) ... 2023-11-26T07:15:15.843 INFO:teuthology.orchestra.run.smithi188.stdout:Selecting previously unselected package cephfs-mirror. 2023-11-26T07:15:15.859 INFO:teuthology.orchestra.run.smithi188.stdout:Preparing to unpack .../094-cephfs-mirror_18.0.0-7484-g44c2237f-1jammy_amd64.deb ... 2023-11-26T07:15:15.901 INFO:teuthology.orchestra.run.smithi188.stdout:Unpacking cephfs-mirror (18.0.0-7484-g44c2237f-1jammy) ... 2023-11-26T07:15:15.977 INFO:teuthology.orchestra.run.smithi143.stdout:Setting up python3-pyasn1 (0.4.8-1) ... 2023-11-26T07:15:16.197 INFO:teuthology.orchestra.run.smithi188.stdout:Selecting previously unselected package python3-colorama. 2023-11-26T07:15:16.213 INFO:teuthology.orchestra.run.smithi188.stdout:Preparing to unpack .../095-python3-colorama_0.4.4-1_all.deb ... 2023-11-26T07:15:16.254 INFO:teuthology.orchestra.run.smithi188.stdout:Unpacking python3-colorama (0.4.4-1) ... 2023-11-26T07:15:16.296 INFO:teuthology.orchestra.run.smithi143.stdout:Setting up python3-singledispatch (3.4.0.3-3) ... 2023-11-26T07:15:16.563 INFO:teuthology.orchestra.run.smithi143.stdout:Setting up python3-logutils (0.3.3-8) ... 2023-11-26T07:15:16.566 INFO:teuthology.orchestra.run.smithi188.stdout:Selecting previously unselected package cephfs-shell. 2023-11-26T07:15:16.581 INFO:teuthology.orchestra.run.smithi188.stdout:Preparing to unpack .../096-cephfs-shell_18.0.0-7484-g44c2237f-1jammy_all.deb ... 2023-11-26T07:15:16.624 INFO:teuthology.orchestra.run.smithi188.stdout:Unpacking cephfs-shell (18.0.0-7484-g44c2237f-1jammy) ... 2023-11-26T07:15:16.840 INFO:teuthology.orchestra.run.smithi143.stdout:Setting up python3-tempora (4.1.2-1) ... 2023-11-26T07:15:16.927 INFO:teuthology.orchestra.run.smithi188.stdout:Selecting previously unselected package cephfs-top. 2023-11-26T07:15:16.943 INFO:teuthology.orchestra.run.smithi188.stdout:Preparing to unpack .../097-cephfs-top_18.0.0-7484-g44c2237f-1jammy_all.deb ... 2023-11-26T07:15:16.985 INFO:teuthology.orchestra.run.smithi188.stdout:Unpacking cephfs-top (18.0.0-7484-g44c2237f-1jammy) ... 2023-11-26T07:15:17.091 INFO:teuthology.orchestra.run.smithi143.stdout:Setting up python3-simplegeneric (0.8.1-3) ... 2023-11-26T07:15:17.221 INFO:teuthology.orchestra.run.smithi188.stdout:Selecting previously unselected package libcephfs-dev. 2023-11-26T07:15:17.237 INFO:teuthology.orchestra.run.smithi188.stdout:Preparing to unpack .../098-libcephfs-dev_18.0.0-7484-g44c2237f-1jammy_amd64.deb ... 2023-11-26T07:15:17.333 INFO:teuthology.orchestra.run.smithi143.stdout:Setting up python3-prettytable (2.5.0-2) ... 2023-11-26T07:15:17.504 INFO:teuthology.orchestra.run.smithi188.stdout:Unpacking libcephfs-dev (18.0.0-7484-g44c2237f-1jammy) ... 2023-11-26T07:15:17.602 INFO:teuthology.orchestra.run.smithi143.stdout:Setting up liblttng-ust1:amd64 (2.13.1-1ubuntu1) ... 2023-11-26T07:15:17.728 INFO:teuthology.orchestra.run.smithi143.stdout:Setting up python3-websocket (1.2.3-1) ... 2023-11-26T07:15:17.874 INFO:teuthology.orchestra.run.smithi188.stdout:Selecting previously unselected package nvme-cli. 2023-11-26T07:15:17.890 INFO:teuthology.orchestra.run.smithi188.stdout:Preparing to unpack .../099-nvme-cli_1.16-3ubuntu0.1_amd64.deb ... 2023-11-26T07:15:17.932 INFO:teuthology.orchestra.run.smithi188.stdout:Unpacking nvme-cli (1.16-3ubuntu0.1) ... 2023-11-26T07:15:18.012 INFO:teuthology.orchestra.run.smithi143.stdout:Setting up libonig5:amd64 (6.9.7.1-2build1) ... 2023-11-26T07:15:18.138 INFO:teuthology.orchestra.run.smithi143.stdout:Setting up python3-mako (1.1.3+ds1-2ubuntu0.1) ... 2023-11-26T07:15:18.271 INFO:teuthology.orchestra.run.smithi188.stdout:Selecting previously unselected package python-asyncssh-doc. 2023-11-26T07:15:18.285 INFO:teuthology.orchestra.run.smithi188.stdout:Preparing to unpack .../100-python-asyncssh-doc_2.5.0-1_all.deb ... 2023-11-26T07:15:18.329 INFO:teuthology.orchestra.run.smithi188.stdout:Unpacking python-asyncssh-doc (2.5.0-1) ... 2023-11-26T07:15:18.464 INFO:teuthology.orchestra.run.smithi143.stdout:Setting up python3-webob (1:1.8.6-1.1) ... 2023-11-26T07:15:18.765 INFO:teuthology.orchestra.run.smithi188.stdout:Selecting previously unselected package python3-iniconfig. 2023-11-26T07:15:18.774 INFO:teuthology.orchestra.run.smithi143.stdout:Setting up python3-jaraco.collections (3.4.0-2) ... 2023-11-26T07:15:18.780 INFO:teuthology.orchestra.run.smithi188.stdout:Preparing to unpack .../101-python3-iniconfig_1.1.1-2_all.deb ... 2023-11-26T07:15:18.824 INFO:teuthology.orchestra.run.smithi188.stdout:Unpacking python3-iniconfig (1.1.1-2) ... 2023-11-26T07:15:19.025 INFO:teuthology.orchestra.run.smithi143.stdout:Setting up python3-babel (2.8.0+dfsg.1-7) ... 2023-11-26T07:15:19.110 INFO:teuthology.orchestra.run.smithi188.stdout:Selecting previously unselected package python3-pastescript. 2023-11-26T07:15:19.125 INFO:teuthology.orchestra.run.smithi188.stdout:Preparing to unpack .../102-python3-pastescript_2.0.2-4_all.deb ... 2023-11-26T07:15:19.207 INFO:teuthology.orchestra.run.smithi188.stdout:Unpacking python3-pastescript (2.0.2-4) ... 2023-11-26T07:15:19.417 INFO:teuthology.orchestra.run.smithi143.stdout:update-alternatives: using /usr/bin/pybabel-python3 to provide /usr/bin/pybabel (pybabel) in auto mode 2023-11-26T07:15:19.513 INFO:teuthology.orchestra.run.smithi188.stdout:Selecting previously unselected package python3-pluggy. 2023-11-26T07:15:19.528 INFO:teuthology.orchestra.run.smithi188.stdout:Preparing to unpack .../103-python3-pluggy_0.13.0-7.1_all.deb ... 2023-11-26T07:15:19.571 INFO:teuthology.orchestra.run.smithi188.stdout:Unpacking python3-pluggy (0.13.0-7.1) ... 2023-11-26T07:15:19.662 INFO:teuthology.orchestra.run.smithi143.stdout:Setting up libjq1:amd64 (1.6-2.1ubuntu3) ... 2023-11-26T07:15:19.788 INFO:teuthology.orchestra.run.smithi143.stdout:Setting up python3-pytest (6.2.5-1ubuntu2) ... 2023-11-26T07:15:19.908 INFO:teuthology.orchestra.run.smithi188.stdout:Selecting previously unselected package python3-psutil. 2023-11-26T07:15:19.922 INFO:teuthology.orchestra.run.smithi188.stdout:Preparing to unpack .../104-python3-psutil_5.9.0-1build1_amd64.deb ... 2023-11-26T07:15:19.966 INFO:teuthology.orchestra.run.smithi188.stdout:Unpacking python3-psutil (5.9.0-1build1) ... 2023-11-26T07:15:20.181 INFO:teuthology.orchestra.run.smithi143.stdout:Setting up python3-pastedeploy (2.1.1-1) ... 2023-11-26T07:15:20.311 INFO:teuthology.orchestra.run.smithi188.stdout:Selecting previously unselected package python3-py. 2023-11-26T07:15:20.326 INFO:teuthology.orchestra.run.smithi188.stdout:Preparing to unpack .../105-python3-py_1.10.0-1_all.deb ... 2023-11-26T07:15:20.394 INFO:teuthology.orchestra.run.smithi188.stdout:Unpacking python3-py (1.10.0-1) ... 2023-11-26T07:15:20.457 INFO:teuthology.orchestra.run.smithi143.stdout:Setting up python3-portend (3.0.0-1) ... 2023-11-26T07:15:20.708 INFO:teuthology.orchestra.run.smithi143.stdout:Setting up libqt5network5:amd64 (5.15.3+dfsg-2ubuntu0.2) ... 2023-11-26T07:15:20.722 INFO:teuthology.orchestra.run.smithi188.stdout:Selecting previously unselected package python3-pygments. 2023-11-26T07:15:20.738 INFO:teuthology.orchestra.run.smithi188.stdout:Preparing to unpack .../106-python3-pygments_2.11.2+dfsg-2_all.deb ... 2023-11-26T07:15:20.780 INFO:teuthology.orchestra.run.smithi188.stdout:Unpacking python3-pygments (2.11.2+dfsg-2) ... 2023-11-26T07:15:20.843 INFO:teuthology.orchestra.run.smithi143.stdout:Setting up python3-jinja2 (3.0.3-1) ... 2023-11-26T07:15:21.169 INFO:teuthology.orchestra.run.smithi143.stdout:Setting up python3-pyasn1-modules (0.2.1-1) ... 2023-11-26T07:15:21.259 INFO:teuthology.orchestra.run.smithi188.stdout:Selecting previously unselected package python3-pyinotify. 2023-11-26T07:15:21.274 INFO:teuthology.orchestra.run.smithi188.stdout:Preparing to unpack .../107-python3-pyinotify_0.9.6-1.3_all.deb ... 2023-11-26T07:15:21.318 INFO:teuthology.orchestra.run.smithi188.stdout:Unpacking python3-pyinotify (0.9.6-1.3) ... 2023-11-26T07:15:21.487 INFO:teuthology.orchestra.run.smithi143.stdout:Setting up python3-requests (2.25.1+dfsg-2ubuntu0.1) ... 2023-11-26T07:15:21.638 INFO:teuthology.orchestra.run.smithi188.stdout:Selecting previously unselected package python3-toml. 2023-11-26T07:15:21.652 INFO:teuthology.orchestra.run.smithi188.stdout:Preparing to unpack .../108-python3-toml_0.10.2-1_all.deb ... 2023-11-26T07:15:21.695 INFO:teuthology.orchestra.run.smithi188.stdout:Unpacking python3-toml (0.10.2-1) ... 2023-11-26T07:15:21.772 INFO:teuthology.orchestra.run.smithi143.stdout:Setting up jq (1.6-2.1ubuntu3) ... 2023-11-26T07:15:21.898 INFO:teuthology.orchestra.run.smithi143.stdout:Setting up python3-rsa (4.8-1) ... 2023-11-26T07:15:22.006 INFO:teuthology.orchestra.run.smithi188.stdout:Selecting previously unselected package python3-pytest. 2023-11-26T07:15:22.022 INFO:teuthology.orchestra.run.smithi188.stdout:Preparing to unpack .../109-python3-pytest_6.2.5-1ubuntu2_all.deb ... 2023-11-26T07:15:22.064 INFO:teuthology.orchestra.run.smithi188.stdout:Unpacking python3-pytest (6.2.5-1ubuntu2) ... 2023-11-26T07:15:22.165 INFO:teuthology.orchestra.run.smithi143.stdout:Setting up python3-webtest (2.0.35-1) ... 2023-11-26T07:15:22.425 INFO:teuthology.orchestra.run.smithi143.stdout:Setting up python3-requests-oauthlib (1.3.0+ds-0.1) ... 2023-11-26T07:15:22.426 INFO:teuthology.orchestra.run.smithi188.stdout:Selecting previously unselected package python3-simplejson. 2023-11-26T07:15:22.442 INFO:teuthology.orchestra.run.smithi188.stdout:Preparing to unpack .../110-python3-simplejson_3.17.6-1build1_amd64.deb ... 2023-11-26T07:15:22.485 INFO:teuthology.orchestra.run.smithi188.stdout:Unpacking python3-simplejson (3.17.6-1build1) ... 2023-11-26T07:15:22.676 INFO:teuthology.orchestra.run.smithi143.stdout:Setting up python3-cherrypy3 (18.6.1-4) ... 2023-11-26T07:15:22.787 INFO:teuthology.orchestra.run.smithi188.stdout:Selecting previously unselected package qttranslations5-l10n. 2023-11-26T07:15:22.803 INFO:teuthology.orchestra.run.smithi188.stdout:Preparing to unpack .../111-qttranslations5-l10n_5.15.3-1_all.deb ... 2023-11-26T07:15:22.846 INFO:teuthology.orchestra.run.smithi188.stdout:Unpacking qttranslations5-l10n (5.15.3-1) ... 2023-11-26T07:15:23.095 INFO:teuthology.orchestra.run.smithi143.stdout:Setting up python3-pastescript (2.0.2-4) ... 2023-11-26T07:15:23.388 INFO:teuthology.orchestra.run.smithi143.stdout:Setting up python3-pecan (1.3.3-4ubuntu2) ... 2023-11-26T07:15:23.574 INFO:teuthology.orchestra.run.smithi188.stdout:Selecting previously unselected package radosgw. 2023-11-26T07:15:23.589 INFO:teuthology.orchestra.run.smithi188.stdout:Preparing to unpack .../112-radosgw_18.0.0-7484-g44c2237f-1jammy_amd64.deb ... 2023-11-26T07:15:23.632 INFO:teuthology.orchestra.run.smithi188.stdout:Unpacking radosgw (18.0.0-7484-g44c2237f-1jammy) ... 2023-11-26T07:15:23.740 INFO:teuthology.orchestra.run.smithi143.stdout:Setting up libthrift-0.16.0:amd64 (0.16.0-2) ... 2023-11-26T07:15:23.858 INFO:teuthology.orchestra.run.smithi143.stdout:Setting up librados2 (18.0.0-7484-g44c2237f-1jammy) ... 2023-11-26T07:15:23.967 INFO:teuthology.orchestra.run.smithi143.stdout:Setting up librgw2 (18.0.0-7484-g44c2237f-1jammy) ... 2023-11-26T07:15:24.093 INFO:teuthology.orchestra.run.smithi143.stdout:Setting up libsqlite3-mod-ceph (18.0.0-7484-g44c2237f-1jammy) ... 2023-11-26T07:15:24.253 INFO:teuthology.orchestra.run.smithi188.stdout:Selecting previously unselected package rbd-fuse. 2023-11-26T07:15:24.267 INFO:teuthology.orchestra.run.smithi188.stdout:Preparing to unpack .../113-rbd-fuse_18.0.0-7484-g44c2237f-1jammy_amd64.deb ... 2023-11-26T07:15:24.311 INFO:teuthology.orchestra.run.smithi188.stdout:Unpacking rbd-fuse (18.0.0-7484-g44c2237f-1jammy) ... 2023-11-26T07:15:24.338 INFO:teuthology.orchestra.run.smithi143.stdout:Setting up libcephfs2 (18.0.0-7484-g44c2237f-1jammy) ... 2023-11-26T07:15:24.473 INFO:teuthology.orchestra.run.smithi143.stdout:Setting up libradosstriper1 (18.0.0-7484-g44c2237f-1jammy) ... 2023-11-26T07:15:24.599 INFO:teuthology.orchestra.run.smithi143.stdout:Setting up python3-google-auth (1.5.1-3) ... 2023-11-26T07:15:24.628 INFO:teuthology.orchestra.run.smithi188.stdout:Setting up python3-iniconfig (1.1.1-2) ... 2023-11-26T07:15:24.882 INFO:teuthology.orchestra.run.smithi188.stdout:Setting up libdouble-conversion3:amd64 (3.1.7-4) ... 2023-11-26T07:15:24.884 INFO:teuthology.orchestra.run.smithi143.stdout:Setting up librbd1 (18.0.0-7484-g44c2237f-1jammy) ... 2023-11-26T07:15:25.010 INFO:teuthology.orchestra.run.smithi143.stdout:Setting up ceph-mgr-modules-core (18.0.0-7484-g44c2237f-1jammy) ... 2023-11-26T07:15:25.050 INFO:teuthology.orchestra.run.smithi188.stdout:Setting up nvme-cli (1.16-3ubuntu0.1) ... 2023-11-26T07:15:25.136 INFO:teuthology.orchestra.run.smithi143.stdout:Setting up ceph-fuse (18.0.0-7484-g44c2237f-1jammy) ... 2023-11-26T07:15:25.306 INFO:teuthology.orchestra.run.smithi188.stdout:Created symlink /etc/systemd/system/default.target.wants/nvmefc-boot-connections.service -> /lib/systemd/system/nvmefc-boot-connections.service. 2023-11-26T07:15:25.389 INFO:teuthology.orchestra.run.smithi143.stdout:Created symlink /etc/systemd/system/remote-fs.target.wants/ceph-fuse.target -> /lib/systemd/system/ceph-fuse.target. 2023-11-26T07:15:25.390 INFO:teuthology.orchestra.run.smithi143.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-fuse.target -> /lib/systemd/system/ceph-fuse.target. 2023-11-26T07:15:25.724 INFO:teuthology.orchestra.run.smithi188.stdout:Created symlink /etc/systemd/system/default.target.wants/nvmf-autoconnect.service -> /lib/systemd/system/nvmf-autoconnect.service. 2023-11-26T07:15:26.013 INFO:teuthology.orchestra.run.smithi143.stdout:Setting up libcephfs-dev (18.0.0-7484-g44c2237f-1jammy) ... 2023-11-26T07:15:26.139 INFO:teuthology.orchestra.run.smithi143.stdout:Setting up python3-rados (18.0.0-7484-g44c2237f-1jammy) ... 2023-11-26T07:15:26.265 INFO:teuthology.orchestra.run.smithi143.stdout:Setting up cephfs-top (18.0.0-7484-g44c2237f-1jammy) ... 2023-11-26T07:15:26.319 INFO:teuthology.orchestra.run.smithi188.stdout:nvmf-connect.target is a disabled or a static unit, not starting it. 2023-11-26T07:15:26.353 INFO:teuthology.orchestra.run.smithi188.stdout:Could not execute systemctl: at /usr/bin/deb-systemd-invoke line 142. 2023-11-26T07:15:26.383 INFO:teuthology.orchestra.run.smithi143.stdout:Setting up python3-kubernetes (12.0.1-1ubuntu1) ... 2023-11-26T07:15:26.410 INFO:teuthology.orchestra.run.smithi188.stdout:Setting up cephadm (18.0.0-7484-g44c2237f-1jammy) ... 2023-11-26T07:15:26.964 INFO:teuthology.orchestra.run.smithi188.stdout:Adding system user cephadm....done 2023-11-26T07:15:27.124 INFO:teuthology.orchestra.run.smithi188.stdout:Setting up python3-waitress (1.4.4-1.1ubuntu1) ... 2023-11-26T07:15:27.409 INFO:teuthology.orchestra.run.smithi188.stdout:Setting up python3-jaraco.classes (3.2.1-3) ... 2023-11-26T07:15:27.655 INFO:teuthology.orchestra.run.smithi143.stdout:Setting up python3-rbd (18.0.0-7484-g44c2237f-1jammy) ... 2023-11-26T07:15:27.661 INFO:teuthology.orchestra.run.smithi188.stdout:Setting up python-asyncssh-doc (2.5.0-1) ... 2023-11-26T07:15:27.781 INFO:teuthology.orchestra.run.smithi143.stdout:Setting up rbd-fuse (18.0.0-7484-g44c2237f-1jammy) ... 2023-11-26T07:15:27.786 INFO:teuthology.orchestra.run.smithi188.stdout:Setting up python3-jaraco.functools (3.4.0-2) ... 2023-11-26T07:15:27.907 INFO:teuthology.orchestra.run.smithi143.stdout:Setting up python3-rgw (18.0.0-7484-g44c2237f-1jammy) ... 2023-11-26T07:15:28.025 INFO:teuthology.orchestra.run.smithi143.stdout:Setting up python3-cephfs (18.0.0-7484-g44c2237f-1jammy) ... 2023-11-26T07:15:28.063 INFO:teuthology.orchestra.run.smithi188.stdout:Setting up python3-repoze.lru (0.7-2) ... 2023-11-26T07:15:28.134 INFO:teuthology.orchestra.run.smithi143.stdout:Setting up cephfs-shell (18.0.0-7484-g44c2237f-1jammy) ... 2023-11-26T07:15:28.271 INFO:teuthology.orchestra.run.smithi143.stdout:Setting up ceph-common (18.0.0-7484-g44c2237f-1jammy) ... 2023-11-26T07:15:28.331 INFO:teuthology.orchestra.run.smithi188.stdout:Setting up liboath0:amd64 (2.6.7-3build1) ... 2023-11-26T07:15:28.490 INFO:teuthology.orchestra.run.smithi188.stdout:Setting up python3-py (1.10.0-1) ... 2023-11-26T07:15:28.809 INFO:teuthology.orchestra.run.smithi188.stdout:Setting up python3-joblib (0.17.0-4ubuntu1) ... 2023-11-26T07:15:28.823 INFO:teuthology.orchestra.run.smithi143.stdout:Adding group ceph....done 2023-11-26T07:15:29.167 INFO:teuthology.orchestra.run.smithi143.stdout:Adding system user ceph....done 2023-11-26T07:15:29.177 INFO:teuthology.orchestra.run.smithi188.stdout:Setting up python3-cachetools (5.0.0-1) ... 2023-11-26T07:15:29.337 INFO:teuthology.orchestra.run.smithi143.stdout:Setting system user ceph properties....done 2023-11-26T07:15:29.355 INFO:teuthology.orchestra.run.smithi143.stdout:chown: cannot access '/var/log/ceph/*.log*': No such file or directory 2023-11-26T07:15:29.437 INFO:teuthology.orchestra.run.smithi188.stdout:Setting up python3-colorama (0.4.4-1) ... 2023-11-26T07:15:29.497 INFO:teuthology.orchestra.run.smithi143.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph.target -> /lib/systemd/system/ceph.target. 2023-11-26T07:15:29.705 INFO:teuthology.orchestra.run.smithi188.stdout:Setting up python3-openssl (21.0.0-1) ... 2023-11-26T07:15:29.863 INFO:teuthology.orchestra.run.smithi143.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/rbdmap.service -> /lib/systemd/system/rbdmap.service. 2023-11-26T07:15:29.989 INFO:teuthology.orchestra.run.smithi188.stdout:Setting up python3-bcrypt (3.2.0-1build1) ... 2023-11-26T07:15:30.248 INFO:teuthology.orchestra.run.smithi188.stdout:Setting up python3-pyinotify (0.9.6-1.3) ... 2023-11-26T07:15:30.524 INFO:teuthology.orchestra.run.smithi188.stdout:Setting up python3-threadpoolctl (3.1.0-1) ... 2023-11-26T07:15:30.548 INFO:teuthology.orchestra.run.smithi143.stdout:Setting up cephfs-mirror (18.0.0-7484-g44c2237f-1jammy) ... 2023-11-26T07:15:30.740 INFO:teuthology.orchestra.run.smithi143.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/cephfs-mirror.target -> /lib/systemd/system/cephfs-mirror.target. 2023-11-26T07:15:30.740 INFO:teuthology.orchestra.run.smithi143.stdout:Created symlink /etc/systemd/system/ceph.target.wants/cephfs-mirror.target -> /lib/systemd/system/cephfs-mirror.target. 2023-11-26T07:15:30.900 INFO:teuthology.orchestra.run.smithi188.stdout:Setting up python3-ceph-argparse (18.0.0-7484-g44c2237f-1jammy) ... 2023-11-26T07:15:31.177 INFO:teuthology.orchestra.run.smithi188.stdout:Setting up python3-sklearn-lib:amd64 (0.23.2-5ubuntu6) ... 2023-11-26T07:15:31.286 INFO:teuthology.orchestra.run.smithi188.stdout:Setting up python3-pyperclip (1.8.2-2) ... 2023-11-26T07:15:31.374 INFO:teuthology.orchestra.run.smithi143.stdout:Setting up ceph-test (18.0.0-7484-g44c2237f-1jammy) ... 2023-11-26T07:15:31.483 INFO:teuthology.orchestra.run.smithi143.stdout:Setting up radosgw (18.0.0-7484-g44c2237f-1jammy) ... 2023-11-26T07:15:31.587 INFO:teuthology.orchestra.run.smithi188.stdout:Setting up python3-markupsafe (2.0.1-2build1) ... 2023-11-26T07:15:31.846 INFO:teuthology.orchestra.run.smithi188.stdout:Setting up libpcre2-16-0:amd64 (10.39-3ubuntu0.1) ... 2023-11-26T07:15:31.999 INFO:teuthology.orchestra.run.smithi143.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-radosgw.target -> /lib/systemd/system/ceph-radosgw.target. 2023-11-26T07:15:31.999 INFO:teuthology.orchestra.run.smithi143.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-radosgw.target -> /lib/systemd/system/ceph-radosgw.target. 2023-11-26T07:15:32.005 INFO:teuthology.orchestra.run.smithi188.stdout:Setting up python3-psutil (5.9.0-1build1) ... 2023-11-26T07:15:32.373 INFO:teuthology.orchestra.run.smithi188.stdout:Setting up python-babel-localedata (2.8.0+dfsg.1-7) ... 2023-11-26T07:15:32.525 INFO:teuthology.orchestra.run.smithi188.stdout:Setting up python3-natsort (8.0.2-1) ... 2023-11-26T07:15:32.627 INFO:teuthology.orchestra.run.smithi143.stdout:Setting up ceph-base (18.0.0-7484-g44c2237f-1jammy) ... 2023-11-26T07:15:32.893 INFO:teuthology.orchestra.run.smithi188.stdout:Setting up python3-routes (2.5.1-1ubuntu1) ... 2023-11-26T07:15:32.985 INFO:teuthology.orchestra.run.smithi143.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-crash.service -> /lib/systemd/system/ceph-crash.service. 2023-11-26T07:15:33.160 INFO:teuthology.orchestra.run.smithi188.stdout:Setting up python3-simplejson (3.17.6-1build1) ... 2023-11-26T07:15:33.478 INFO:teuthology.orchestra.run.smithi188.stdout:Setting up python3-pygments (2.11.2+dfsg-2) ... 2023-11-26T07:15:33.646 INFO:teuthology.orchestra.run.smithi143.stdout:Setting up ceph-mds (18.0.0-7484-g44c2237f-1jammy) ... 2023-11-26T07:15:33.862 INFO:teuthology.orchestra.run.smithi143.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mds.target -> /lib/systemd/system/ceph-mds.target. 2023-11-26T07:15:33.862 INFO:teuthology.orchestra.run.smithi143.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mds.target -> /lib/systemd/system/ceph-mds.target. 2023-11-26T07:15:34.164 INFO:teuthology.orchestra.run.smithi188.stdout:Setting up python3-tempita (0.5.2-6ubuntu1) ... 2023-11-26T07:15:34.456 INFO:teuthology.orchestra.run.smithi188.stdout:Setting up python-pastedeploy-tpl (2.1.1-1) ... 2023-11-26T07:15:34.506 INFO:teuthology.orchestra.run.smithi143.stdout:Setting up ceph-mgr (18.0.0-7484-g44c2237f-1jammy) ... 2023-11-26T07:15:34.583 INFO:teuthology.orchestra.run.smithi188.stdout:Setting up qttranslations5-l10n (5.15.3-1) ... 2023-11-26T07:15:34.717 INFO:teuthology.orchestra.run.smithi188.stdout:Setting up python3-wcwidth (0.2.5+dfsg1-1) ... 2023-11-26T07:15:34.721 INFO:teuthology.orchestra.run.smithi143.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mgr.target -> /lib/systemd/system/ceph-mgr.target. 2023-11-26T07:15:34.721 INFO:teuthology.orchestra.run.smithi143.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mgr.target -> /lib/systemd/system/ceph-mgr.target. 2023-11-26T07:15:35.068 INFO:teuthology.orchestra.run.smithi188.stdout:Setting up python3-asyncssh (2.5.0-1) ... 2023-11-26T07:15:35.333 INFO:teuthology.orchestra.run.smithi143.stdout:Setting up ceph-osd (18.0.0-7484-g44c2237f-1jammy) ... 2023-11-26T07:15:35.469 INFO:teuthology.orchestra.run.smithi188.stdout:Setting up python3-certifi (2020.6.20-1) ... 2023-11-26T07:15:35.671 INFO:teuthology.orchestra.run.smithi143.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-osd.target -> /lib/systemd/system/ceph-osd.target. 2023-11-26T07:15:35.671 INFO:teuthology.orchestra.run.smithi143.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-osd.target -> /lib/systemd/system/ceph-osd.target. 2023-11-26T07:15:35.728 INFO:teuthology.orchestra.run.smithi188.stdout:Setting up python3-paste (3.5.0+dfsg1-1) ... 2023-11-26T07:15:36.114 INFO:teuthology.orchestra.run.smithi188.stdout:Setting up python3-cheroot (8.5.2+ds1-1ubuntu3.1) ... 2023-11-26T07:15:36.339 INFO:teuthology.orchestra.run.smithi143.stdout:Setting up ceph-mgr-k8sevents (18.0.0-7484-g44c2237f-1jammy) ... 2023-11-26T07:15:36.415 INFO:teuthology.orchestra.run.smithi188.stdout:Setting up python3-cmd2 (2.3.3-1) ... 2023-11-26T07:15:36.448 INFO:teuthology.orchestra.run.smithi143.stdout:Setting up ceph-mgr-diskprediction-local (18.0.0-7484-g44c2237f-1jammy) ... 2023-11-26T07:15:36.633 INFO:teuthology.orchestra.run.smithi143.stdout:Setting up ceph-mon (18.0.0-7484-g44c2237f-1jammy) ... 2023-11-26T07:15:36.741 INFO:teuthology.orchestra.run.smithi188.stdout:Setting up python3-werkzeug (2.0.2+dfsg1-1ubuntu0.22.04.1) ... 2023-11-26T07:15:36.842 INFO:teuthology.orchestra.run.smithi143.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mon.target -> /lib/systemd/system/ceph-mon.target. 2023-11-26T07:15:36.842 INFO:teuthology.orchestra.run.smithi143.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mon.target -> /lib/systemd/system/ceph-mon.target. 2023-11-26T07:15:37.134 INFO:teuthology.orchestra.run.smithi188.stdout:Setting up python3-jaraco.text (3.6.0-2) ... 2023-11-26T07:15:37.386 INFO:teuthology.orchestra.run.smithi188.stdout:Setting up socat (1.7.4.1-3ubuntu4) ... 2023-11-26T07:15:37.476 INFO:teuthology.orchestra.run.smithi143.stdout:Setting up ceph-mgr-cephadm (18.0.0-7484-g44c2237f-1jammy) ... 2023-11-26T07:15:37.511 INFO:teuthology.orchestra.run.smithi188.stdout:Setting up python3-ceph-common (18.0.0-7484-g44c2237f-1jammy) ... 2023-11-26T07:15:37.610 INFO:teuthology.orchestra.run.smithi143.stdout:Setting up ceph (18.0.0-7484-g44c2237f-1jammy) ... 2023-11-26T07:15:37.737 INFO:teuthology.orchestra.run.smithi143.stdout:Setting up ceph-mgr-dashboard (18.0.0-7484-g44c2237f-1jammy) ... 2023-11-26T07:15:37.805 INFO:teuthology.orchestra.run.smithi188.stdout:Setting up python3-idna (3.3-1) ... 2023-11-26T07:15:37.879 INFO:teuthology.orchestra.run.smithi143.stdout:Setting up ceph-volume (18.0.0-7484-g44c2237f-1jammy) ... 2023-11-26T07:15:38.147 INFO:teuthology.orchestra.run.smithi188.stdout:Setting up python3-sklearn (0.23.2-5ubuntu6) ... 2023-11-26T07:15:38.266 INFO:teuthology.orchestra.run.smithi143.stdout:Processing triggers for man-db (2.10.2-1) ... 2023-11-26T07:15:39.367 INFO:teuthology.orchestra.run.smithi143.stdout:Processing triggers for libc-bin (2.35-0ubuntu3.1) ... 2023-11-26T07:15:39.378 INFO:teuthology.orchestra.run.smithi188.stdout:Setting up libqt5core5a:amd64 (5.15.3+dfsg-2ubuntu0.2) ... 2023-11-26T07:15:39.520 INFO:teuthology.orchestra.run.smithi188.stdout:Setting up python3-toml (0.10.2-1) ... 2023-11-26T07:15:39.797 INFO:teuthology.orchestra.run.smithi188.stdout:Setting up librdkafka1:amd64 (1.8.0-1build1) ... 2023-11-26T07:15:39.828 INFO:teuthology.orchestra.run.smithi143.stderr:W: --force-yes is deprecated, use one of the options starting with --allow instead. 2023-11-26T07:15:39.832 DEBUG:teuthology.orchestra.run.smithi143:> sudo DEBIAN_FRONTEND=noninteractive apt-get -y --force-yes -o Dpkg::Options::="--force-confdef" -o Dpkg::Options::="--force-confold" install bison flex libelf-dev libssl-dev network-manager iproute2 util-linux dump indent libaio-dev libtool-bin uuid-dev xfslibs-dev postgresql postgresql-client postgresql-common postgresql-contrib 2023-11-26T07:15:39.907 INFO:teuthology.orchestra.run.smithi143.stdout:Reading package lists... 2023-11-26T07:15:39.922 INFO:teuthology.orchestra.run.smithi188.stdout:Setting up xmlstarlet (1.6.1-2.1) ... 2023-11-26T07:15:40.073 INFO:teuthology.orchestra.run.smithi143.stdout:Building dependency tree... 2023-11-26T07:15:40.074 INFO:teuthology.orchestra.run.smithi143.stdout:Reading state information... 2023-11-26T07:15:40.082 INFO:teuthology.orchestra.run.smithi188.stdout:Setting up python3-urllib3 (1.26.5-1~exp1ubuntu0.1) ... 2023-11-26T07:15:40.223 INFO:teuthology.orchestra.run.smithi143.stdout:iproute2 is already the newest version (5.15.0-1ubuntu2). 2023-11-26T07:15:40.223 INFO:teuthology.orchestra.run.smithi143.stdout:libtool-bin is already the newest version (2.4.6-15build2). 2023-11-26T07:15:40.224 INFO:teuthology.orchestra.run.smithi143.stdout:util-linux is already the newest version (2.37.2-4ubuntu3). 2023-11-26T07:15:40.224 INFO:teuthology.orchestra.run.smithi143.stdout:uuid-dev is already the newest version (2.37.2-4ubuntu3). 2023-11-26T07:15:40.224 INFO:teuthology.orchestra.run.smithi143.stdout:xfslibs-dev is already the newest version (5.13.0-1ubuntu2). 2023-11-26T07:15:40.224 INFO:teuthology.orchestra.run.smithi143.stdout:The following packages were automatically installed and are no longer required: 2023-11-26T07:15:40.224 INFO:teuthology.orchestra.run.smithi143.stdout: libboost-iostreams1.74.0 libboost-thread1.74.0 libgfapi0 libgfrpc0 libgfxdr0 2023-11-26T07:15:40.224 INFO:teuthology.orchestra.run.smithi143.stdout: libglusterfs0 libiscsi7 libpmemobj1 libpython2-dev libpython2-stdlib 2023-11-26T07:15:40.224 INFO:teuthology.orchestra.run.smithi143.stdout: libpython2.7 libpython2.7-dev libpython2.7-minimal libpython2.7-stdlib 2023-11-26T07:15:40.225 INFO:teuthology.orchestra.run.smithi143.stdout: linux-image-5.4.0-135-generic linux-modules-5.4.0-135-generic python2 2023-11-26T07:15:40.226 INFO:teuthology.orchestra.run.smithi143.stdout: python2-dev python2-minimal python2.7 python2.7-dev python2.7-minimal 2023-11-26T07:15:40.226 INFO:teuthology.orchestra.run.smithi143.stdout:Use 'sudo apt autoremove' to remove them. 2023-11-26T07:15:40.226 INFO:teuthology.orchestra.run.smithi143.stdout:The following additional packages will be installed: 2023-11-26T07:15:40.226 INFO:teuthology.orchestra.run.smithi143.stdout: libbluetooth3 libfl-dev libfl2 libjson-perl libllvm14 liblzo2-2 2023-11-26T07:15:40.226 INFO:teuthology.orchestra.run.smithi143.stdout: libmbim-glib4 libmbim-proxy libmm-glib0 libndp0 libnm0 libpq5 libqmi-glib5 2023-11-26T07:15:40.226 INFO:teuthology.orchestra.run.smithi143.stdout: libqmi-proxy libssl3 libteamdctl0 modemmanager network-manager-pptp 2023-11-26T07:15:40.227 INFO:teuthology.orchestra.run.smithi143.stdout: postgresql-14 postgresql-client-14 postgresql-client-common ppp pptp-linux 2023-11-26T07:15:40.227 INFO:teuthology.orchestra.run.smithi143.stdout: tcl tcl8.6 usb-modeswitch usb-modeswitch-data wpasupplicant 2023-11-26T07:15:40.227 INFO:teuthology.orchestra.run.smithi143.stdout:Suggested packages: 2023-11-26T07:15:40.227 INFO:teuthology.orchestra.run.smithi143.stdout: bison-doc flex-doc indent-doc libssl-doc avahi-autoipd libteam-utils 2023-11-26T07:15:40.227 INFO:teuthology.orchestra.run.smithi143.stdout: postgresql-doc postgresql-doc-14 tcl-tclreadline comgt wvdial wpagui 2023-11-26T07:15:40.228 INFO:teuthology.orchestra.run.smithi143.stdout: libengine-pkcs11-openssl 2023-11-26T07:15:40.275 INFO:teuthology.orchestra.run.smithi143.stdout:The following NEW packages will be installed: 2023-11-26T07:15:40.276 INFO:teuthology.orchestra.run.smithi143.stdout: bison dump flex indent libaio-dev libbluetooth3 libelf-dev libfl-dev libfl2 2023-11-26T07:15:40.276 INFO:teuthology.orchestra.run.smithi143.stdout: libjson-perl libllvm14 liblzo2-2 libmbim-glib4 libmbim-proxy libmm-glib0 2023-11-26T07:15:40.276 INFO:teuthology.orchestra.run.smithi143.stdout: libndp0 libnm0 libpq5 libqmi-glib5 libqmi-proxy libssl-dev libteamdctl0 2023-11-26T07:15:40.276 INFO:teuthology.orchestra.run.smithi143.stdout: modemmanager network-manager network-manager-pptp postgresql postgresql-14 2023-11-26T07:15:40.276 INFO:teuthology.orchestra.run.smithi143.stdout: postgresql-client postgresql-client-14 postgresql-client-common 2023-11-26T07:15:40.277 INFO:teuthology.orchestra.run.smithi143.stdout: postgresql-common postgresql-contrib ppp pptp-linux tcl tcl8.6 2023-11-26T07:15:40.277 INFO:teuthology.orchestra.run.smithi143.stdout: usb-modeswitch usb-modeswitch-data wpasupplicant 2023-11-26T07:15:40.277 INFO:teuthology.orchestra.run.smithi143.stdout:The following packages will be upgraded: 2023-11-26T07:15:40.278 INFO:teuthology.orchestra.run.smithi143.stdout: libssl3 2023-11-26T07:15:40.408 INFO:teuthology.orchestra.run.smithi188.stdout:Setting up python3-pluggy (0.13.0-7.1) ... 2023-11-26T07:15:40.462 INFO:teuthology.orchestra.run.smithi143.stdout:1 upgraded, 39 newly installed, 0 to remove and 263 not upgraded. 2023-11-26T07:15:40.462 INFO:teuthology.orchestra.run.smithi143.stdout:Need to get 54.6 MB of archives. 2023-11-26T07:15:40.462 INFO:teuthology.orchestra.run.smithi143.stdout:After this operation, 202 MB of additional disk space will be used. 2023-11-26T07:15:40.462 INFO:teuthology.orchestra.run.smithi143.stdout:Get:1 http://archive.ubuntu.com/ubuntu jammy/main amd64 flex amd64 2.6.4-8build2 [307 kB] 2023-11-26T07:15:40.668 INFO:teuthology.orchestra.run.smithi188.stdout:Setting up python3-zc.lockfile (2.0-1) ... 2023-11-26T07:15:40.928 INFO:teuthology.orchestra.run.smithi188.stdout:Setting up libqt5dbus5:amd64 (5.15.3+dfsg-2ubuntu0.2) ... 2023-11-26T07:15:40.959 INFO:teuthology.orchestra.run.smithi143.stdout:Get:2 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 libssl3 amd64 3.0.2-0ubuntu1.12 [1,901 kB] 2023-11-26T07:15:41.036 INFO:teuthology.orchestra.run.smithi188.stdout:Setting up python3-pyasn1 (0.4.8-1) ... 2023-11-26T07:15:41.217 INFO:teuthology.orchestra.run.smithi143.stdout:Get:3 http://archive.ubuntu.com/ubuntu jammy/main amd64 bison amd64 2:3.8.2+dfsg-1build1 [748 kB] 2023-11-26T07:15:41.251 INFO:teuthology.orchestra.run.smithi143.stdout:Get:4 http://archive.ubuntu.com/ubuntu jammy/main amd64 liblzo2-2 amd64 2.10-2build3 [53.7 kB] 2023-11-26T07:15:41.253 INFO:teuthology.orchestra.run.smithi143.stdout:Get:5 http://archive.ubuntu.com/ubuntu jammy/universe amd64 dump amd64 0.4b47-3 [133 kB] 2023-11-26T07:15:41.258 INFO:teuthology.orchestra.run.smithi143.stdout:Get:6 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 indent amd64 2.2.12-1ubuntu0.22.04.1 [90.8 kB] 2023-11-26T07:15:41.261 INFO:teuthology.orchestra.run.smithi143.stdout:Get:7 http://archive.ubuntu.com/ubuntu jammy/main amd64 libaio-dev amd64 0.3.112-13build1 [21.2 kB] 2023-11-26T07:15:41.262 INFO:teuthology.orchestra.run.smithi143.stdout:Get:8 http://archive.ubuntu.com/ubuntu jammy/main amd64 libbluetooth3 amd64 5.64-0ubuntu1 [86.9 kB] 2023-11-26T07:15:41.265 INFO:teuthology.orchestra.run.smithi143.stdout:Get:9 http://archive.ubuntu.com/ubuntu jammy/main amd64 libelf-dev amd64 0.186-1build1 [64.4 kB] 2023-11-26T07:15:41.268 INFO:teuthology.orchestra.run.smithi143.stdout:Get:10 http://archive.ubuntu.com/ubuntu jammy/main amd64 libfl2 amd64 2.6.4-8build2 [10.7 kB] 2023-11-26T07:15:41.330 INFO:teuthology.orchestra.run.smithi188.stdout:Setting up python3-singledispatch (3.4.0.3-3) ... 2023-11-26T07:15:41.433 INFO:teuthology.orchestra.run.smithi143.stdout:Get:11 http://archive.ubuntu.com/ubuntu jammy/main amd64 libfl-dev amd64 2.6.4-8build2 [6,236 B] 2023-11-26T07:15:41.433 INFO:teuthology.orchestra.run.smithi143.stdout:Get:12 http://archive.ubuntu.com/ubuntu jammy/main amd64 libjson-perl all 4.04000-1 [81.8 kB] 2023-11-26T07:15:41.598 INFO:teuthology.orchestra.run.smithi188.stdout:Setting up python3-logutils (0.3.3-8) ... 2023-11-26T07:15:41.684 INFO:teuthology.orchestra.run.smithi143.stdout:Get:13 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 libllvm14 amd64 1:14.0.0-1ubuntu1.1 [24.0 MB] 2023-11-26T07:15:41.865 INFO:teuthology.orchestra.run.smithi188.stdout:Setting up python3-tempora (4.1.2-1) ... 2023-11-26T07:15:42.134 INFO:teuthology.orchestra.run.smithi188.stdout:Setting up python3-simplegeneric (0.8.1-3) ... 2023-11-26T07:15:42.393 INFO:teuthology.orchestra.run.smithi188.stdout:Setting up python3-prettytable (2.5.0-2) ... 2023-11-26T07:15:42.669 INFO:teuthology.orchestra.run.smithi188.stdout:Setting up liblttng-ust1:amd64 (2.13.1-1ubuntu1) ... 2023-11-26T07:15:42.693 INFO:teuthology.orchestra.run.smithi143.stdout:Get:14 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 libmbim-glib4 amd64 1.28.0-1~ubuntu20.04.1 [191 kB] 2023-11-26T07:15:42.695 INFO:teuthology.orchestra.run.smithi143.stdout:Get:15 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 libmbim-proxy amd64 1.28.0-1~ubuntu20.04.1 [6,130 B] 2023-11-26T07:15:42.696 INFO:teuthology.orchestra.run.smithi143.stdout:Get:16 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 libmm-glib0 amd64 1.20.0-1~ubuntu22.04.2 [263 kB] 2023-11-26T07:15:42.698 INFO:teuthology.orchestra.run.smithi143.stdout:Get:17 http://archive.ubuntu.com/ubuntu jammy/main amd64 libndp0 amd64 1.8-0ubuntu3 [12.3 kB] 2023-11-26T07:15:42.698 INFO:teuthology.orchestra.run.smithi143.stdout:Get:18 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 libnm0 amd64 1.36.6-0ubuntu2 [456 kB] 2023-11-26T07:15:42.725 INFO:teuthology.orchestra.run.smithi143.stdout:Get:19 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 libpq5 amd64 14.9-0ubuntu0.22.04.1 [141 kB] 2023-11-26T07:15:42.727 INFO:teuthology.orchestra.run.smithi143.stdout:Get:20 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 libqmi-glib5 amd64 1.32.0-1ubuntu0.22.04.1 [772 kB] 2023-11-26T07:15:42.734 INFO:teuthology.orchestra.run.smithi143.stdout:Get:21 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 libqmi-proxy amd64 1.32.0-1ubuntu0.22.04.1 [6,072 B] 2023-11-26T07:15:42.794 INFO:teuthology.orchestra.run.smithi188.stdout:Setting up python3-websocket (1.2.3-1) ... 2023-11-26T07:15:42.898 INFO:teuthology.orchestra.run.smithi143.stdout:Get:22 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 libssl-dev amd64 3.0.2-0ubuntu1.12 [2,373 kB] 2023-11-26T07:15:43.112 INFO:teuthology.orchestra.run.smithi188.stdout:Setting up libonig5:amd64 (6.9.7.1-2build1) ... 2023-11-26T07:15:43.238 INFO:teuthology.orchestra.run.smithi188.stdout:Setting up python3-mako (1.1.3+ds1-2ubuntu0.1) ... 2023-11-26T07:15:43.556 INFO:teuthology.orchestra.run.smithi188.stdout:Setting up python3-webob (1:1.8.6-1.1) ... 2023-11-26T07:15:43.638 INFO:teuthology.orchestra.run.smithi143.stdout:Get:23 http://archive.ubuntu.com/ubuntu jammy/main amd64 libteamdctl0 amd64 1.31-1build2 [12.6 kB] 2023-11-26T07:15:43.639 INFO:teuthology.orchestra.run.smithi143.stdout:Get:24 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 modemmanager amd64 1.20.0-1~ubuntu22.04.2 [1,094 kB] 2023-11-26T07:15:43.665 INFO:teuthology.orchestra.run.smithi143.stdout:Get:25 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 network-manager amd64 1.36.6-0ubuntu2 [2,203 kB] 2023-11-26T07:15:43.742 INFO:teuthology.orchestra.run.smithi143.stdout:Get:26 http://archive.ubuntu.com/ubuntu jammy/main amd64 ppp amd64 2.4.9-1+1ubuntu3 [363 kB] 2023-11-26T07:15:43.751 INFO:teuthology.orchestra.run.smithi143.stdout:Get:27 http://archive.ubuntu.com/ubuntu jammy/main amd64 pptp-linux amd64 1.10.0-1build3 [42.5 kB] 2023-11-26T07:15:43.751 INFO:teuthology.orchestra.run.smithi143.stdout:Get:28 http://archive.ubuntu.com/ubuntu jammy/main amd64 network-manager-pptp amd64 1.2.10-1 [30.9 kB] 2023-11-26T07:15:43.752 INFO:teuthology.orchestra.run.smithi143.stdout:Get:29 http://archive.ubuntu.com/ubuntu jammy/main amd64 postgresql-client-common all 238 [29.6 kB] 2023-11-26T07:15:43.753 INFO:teuthology.orchestra.run.smithi143.stdout:Get:30 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 postgresql-client-14 amd64 14.9-0ubuntu0.22.04.1 [1,222 kB] 2023-11-26T07:15:43.781 INFO:teuthology.orchestra.run.smithi143.stdout:Get:31 http://archive.ubuntu.com/ubuntu jammy/main amd64 postgresql-common all 238 [169 kB] 2023-11-26T07:15:43.784 INFO:teuthology.orchestra.run.smithi143.stdout:Get:32 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 postgresql-14 amd64 14.9-0ubuntu0.22.04.1 [16.1 MB] 2023-11-26T07:15:43.874 INFO:teuthology.orchestra.run.smithi188.stdout:Setting up python3-jaraco.collections (3.4.0-2) ... 2023-11-26T07:15:44.151 INFO:teuthology.orchestra.run.smithi188.stdout:Setting up python3-babel (2.8.0+dfsg.1-7) ... 2023-11-26T07:15:44.410 INFO:teuthology.orchestra.run.smithi143.stdout:Get:33 http://archive.ubuntu.com/ubuntu jammy/main amd64 postgresql all 14+238 [3,288 B] 2023-11-26T07:15:44.492 INFO:teuthology.orchestra.run.smithi143.stdout:Get:34 http://archive.ubuntu.com/ubuntu jammy/main amd64 postgresql-client all 14+238 [3,292 B] 2023-11-26T07:15:44.492 INFO:teuthology.orchestra.run.smithi143.stdout:Get:35 http://archive.ubuntu.com/ubuntu jammy/main amd64 postgresql-contrib all 14+238 [3,292 B] 2023-11-26T07:15:44.492 INFO:teuthology.orchestra.run.smithi143.stdout:Get:36 http://archive.ubuntu.com/ubuntu jammy/main amd64 tcl8.6 amd64 8.6.12+dfsg-1build1 [15.0 kB] 2023-11-26T07:15:44.547 INFO:teuthology.orchestra.run.smithi188.stdout:update-alternatives: using /usr/bin/pybabel-python3 to provide /usr/bin/pybabel (pybabel) in auto mode 2023-11-26T07:15:44.584 INFO:teuthology.orchestra.run.smithi143.stdout:Get:37 http://archive.ubuntu.com/ubuntu jammy/main amd64 tcl amd64 8.6.11+1build2 [4,678 B] 2023-11-26T07:15:44.598 INFO:teuthology.orchestra.run.smithi143.stdout:Get:38 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 wpasupplicant amd64 2:2.10-6ubuntu2 [1,481 kB] 2023-11-26T07:15:44.820 INFO:teuthology.orchestra.run.smithi188.stdout:Setting up libjq1:amd64 (1.6-2.1ubuntu3) ... 2023-11-26T07:15:44.946 INFO:teuthology.orchestra.run.smithi188.stdout:Setting up python3-pytest (6.2.5-1ubuntu2) ... 2023-11-26T07:15:45.051 INFO:teuthology.orchestra.run.smithi143.stdout:Get:39 http://archive.ubuntu.com/ubuntu jammy/main amd64 usb-modeswitch-data all 20191128-4 [33.2 kB] 2023-11-26T07:15:45.052 INFO:teuthology.orchestra.run.smithi143.stdout:Get:40 http://archive.ubuntu.com/ubuntu jammy/main amd64 usb-modeswitch amd64 2.6.1-3ubuntu2 [46.0 kB] 2023-11-26T07:15:45.348 INFO:teuthology.orchestra.run.smithi188.stdout:Setting up python3-pastedeploy (2.1.1-1) ... 2023-11-26T07:15:45.442 INFO:teuthology.orchestra.run.smithi143.stdout:Preconfiguring packages ... 2023-11-26T07:15:45.591 INFO:teuthology.orchestra.run.smithi143.stdout:Fetched 54.6 MB in 5s (11.4 MB/s) 2023-11-26T07:15:45.624 INFO:teuthology.orchestra.run.smithi188.stdout:Setting up python3-portend (3.0.0-1) ... 2023-11-26T07:15:45.733 INFO:teuthology.orchestra.run.smithi143.stdout:Selecting previously unselected package flex. 2023-11-26T07:15:45.806 INFO:teuthology.orchestra.run.smithi143.stdout:(Reading database ... (Reading database ... 5% (Reading database ... 10% (Reading database ... 15% (Reading database ... 20% (Reading database ... 25% (Reading database ... 30% (Reading database ... 35% (Reading database ... 40% (Reading database ... 45% (Reading database ... 50% (Reading database ... 55% (Reading database ... 60% (Reading database ... 65% (Reading database ... 70% (Reading database ... 75% (Reading database ... 80% (Reading database ... 85% (Reading database ... 90% (Reading database ... 95% (Reading database ... 100% (Reading database ... 139845 files and directories currently installed.) 2023-11-26T07:15:45.810 INFO:teuthology.orchestra.run.smithi143.stdout:Preparing to unpack .../flex_2.6.4-8build2_amd64.deb ... 2023-11-26T07:15:45.849 INFO:teuthology.orchestra.run.smithi143.stdout:Unpacking flex (2.6.4-8build2) ... 2023-11-26T07:15:45.884 INFO:teuthology.orchestra.run.smithi188.stdout:Setting up libqt5network5:amd64 (5.15.3+dfsg-2ubuntu0.2) ... 2023-11-26T07:15:46.010 INFO:teuthology.orchestra.run.smithi188.stdout:Setting up python3-jinja2 (3.0.3-1) ... 2023-11-26T07:15:46.360 INFO:teuthology.orchestra.run.smithi188.stdout:Setting up python3-pyasn1-modules (0.2.1-1) ... 2023-11-26T07:15:46.410 INFO:teuthology.orchestra.run.smithi143.stdout:Preparing to unpack .../libssl3_3.0.2-0ubuntu1.12_amd64.deb ... 2023-11-26T07:15:46.580 INFO:teuthology.orchestra.run.smithi143.stdout:Unpacking libssl3:amd64 (3.0.2-0ubuntu1.12) over (3.0.2-0ubuntu1.7) ... 2023-11-26T07:15:46.670 INFO:teuthology.orchestra.run.smithi188.stdout:Setting up python3-requests (2.25.1+dfsg-2ubuntu0.1) ... 2023-11-26T07:15:46.954 INFO:teuthology.orchestra.run.smithi188.stdout:Setting up jq (1.6-2.1ubuntu3) ... 2023-11-26T07:15:46.985 INFO:teuthology.orchestra.run.smithi143.stdout:Setting up libssl3:amd64 (3.0.2-0ubuntu1.12) ... 2023-11-26T07:15:47.080 INFO:teuthology.orchestra.run.smithi188.stdout:Setting up python3-rsa (4.8-1) ... 2023-11-26T07:15:47.357 INFO:teuthology.orchestra.run.smithi188.stdout:Setting up python3-webtest (2.0.35-1) ... 2023-11-26T07:15:47.464 INFO:teuthology.orchestra.run.smithi143.stdout:Selecting previously unselected package bison. 2023-11-26T07:15:47.537 INFO:teuthology.orchestra.run.smithi143.stdout:(Reading database ... (Reading database ... 5% (Reading database ... 10% (Reading database ... 15% (Reading database ... 20% (Reading database ... 25% (Reading database ... 30% (Reading database ... 35% (Reading database ... 40% (Reading database ... 45% (Reading database ... 50% (Reading database ... 55% (Reading database ... 60% (Reading database ... 65% (Reading database ... 70% (Reading database ... 75% (Reading database ... 80% (Reading database ... 85% (Reading database ... 90% (Reading database ... 95% (Reading database ... 100% (Reading database ... 139918 files and directories currently installed.) 2023-11-26T07:15:47.541 INFO:teuthology.orchestra.run.smithi143.stdout:Preparing to unpack .../00-bison_2%3a3.8.2+dfsg-1build1_amd64.deb ... 2023-11-26T07:15:47.611 INFO:teuthology.orchestra.run.smithi143.stdout:Unpacking bison (2:3.8.2+dfsg-1build1) ... 2023-11-26T07:15:47.643 INFO:teuthology.orchestra.run.smithi188.stdout:Setting up python3-requests-oauthlib (1.3.0+ds-0.1) ... 2023-11-26T07:15:47.911 INFO:teuthology.orchestra.run.smithi188.stdout:Setting up python3-cherrypy3 (18.6.1-4) ... 2023-11-26T07:15:48.026 INFO:teuthology.orchestra.run.smithi143.stdout:Selecting previously unselected package liblzo2-2:amd64. 2023-11-26T07:15:48.040 INFO:teuthology.orchestra.run.smithi143.stdout:Preparing to unpack .../01-liblzo2-2_2.10-2build3_amd64.deb ... 2023-11-26T07:15:48.084 INFO:teuthology.orchestra.run.smithi143.stdout:Unpacking liblzo2-2:amd64 (2.10-2build3) ... 2023-11-26T07:15:48.412 INFO:teuthology.orchestra.run.smithi143.stdout:Selecting previously unselected package dump. 2023-11-26T07:15:48.427 INFO:teuthology.orchestra.run.smithi143.stdout:Preparing to unpack .../02-dump_0.4b47-3_amd64.deb ... 2023-11-26T07:15:48.462 INFO:teuthology.orchestra.run.smithi188.stdout:Setting up python3-pastescript (2.0.2-4) ... 2023-11-26T07:15:48.471 INFO:teuthology.orchestra.run.smithi143.stdout:Unpacking dump (0.4b47-3) ... 2023-11-26T07:15:48.724 INFO:teuthology.orchestra.run.smithi143.stdout:Selecting previously unselected package indent. 2023-11-26T07:15:48.739 INFO:teuthology.orchestra.run.smithi143.stdout:Preparing to unpack .../03-indent_2.2.12-1ubuntu0.22.04.1_amd64.deb ... 2023-11-26T07:15:48.772 INFO:teuthology.orchestra.run.smithi188.stdout:Setting up python3-pecan (1.3.3-4ubuntu2) ... 2023-11-26T07:15:48.783 INFO:teuthology.orchestra.run.smithi143.stdout:Unpacking indent (2.2.12-1ubuntu0.22.04.1) ... 2023-11-26T07:15:49.069 INFO:teuthology.orchestra.run.smithi143.stdout:Selecting previously unselected package libaio-dev:amd64. 2023-11-26T07:15:49.083 INFO:teuthology.orchestra.run.smithi143.stdout:Preparing to unpack .../04-libaio-dev_0.3.112-13build1_amd64.deb ... 2023-11-26T07:15:49.124 INFO:teuthology.orchestra.run.smithi188.stdout:Setting up libthrift-0.16.0:amd64 (0.16.0-2) ... 2023-11-26T07:15:49.127 INFO:teuthology.orchestra.run.smithi143.stdout:Unpacking libaio-dev:amd64 (0.3.112-13build1) ... 2023-11-26T07:15:49.283 INFO:teuthology.orchestra.run.smithi188.stdout:Setting up librados2 (18.0.0-7484-g44c2237f-1jammy) ... 2023-11-26T07:15:49.409 INFO:teuthology.orchestra.run.smithi188.stdout:Setting up librgw2 (18.0.0-7484-g44c2237f-1jammy) ... 2023-11-26T07:15:49.456 INFO:teuthology.orchestra.run.smithi143.stdout:Selecting previously unselected package libbluetooth3:amd64. 2023-11-26T07:15:49.471 INFO:teuthology.orchestra.run.smithi143.stdout:Preparing to unpack .../05-libbluetooth3_5.64-0ubuntu1_amd64.deb ... 2023-11-26T07:15:49.515 INFO:teuthology.orchestra.run.smithi143.stdout:Unpacking libbluetooth3:amd64 (5.64-0ubuntu1) ... 2023-11-26T07:15:49.518 INFO:teuthology.orchestra.run.smithi188.stdout:Setting up libsqlite3-mod-ceph (18.0.0-7484-g44c2237f-1jammy) ... 2023-11-26T07:15:49.713 INFO:teuthology.orchestra.run.smithi188.stdout:Setting up libcephfs2 (18.0.0-7484-g44c2237f-1jammy) ... 2023-11-26T07:15:49.759 INFO:teuthology.orchestra.run.smithi143.stdout:Selecting previously unselected package libelf-dev:amd64. 2023-11-26T07:15:49.773 INFO:teuthology.orchestra.run.smithi143.stdout:Preparing to unpack .../06-libelf-dev_0.186-1build1_amd64.deb ... 2023-11-26T07:15:49.809 INFO:teuthology.orchestra.run.smithi143.stdout:Unpacking libelf-dev:amd64 (0.186-1build1) ... 2023-11-26T07:15:49.864 INFO:teuthology.orchestra.run.smithi188.stdout:Setting up libradosstriper1 (18.0.0-7484-g44c2237f-1jammy) ... 2023-11-26T07:15:49.990 INFO:teuthology.orchestra.run.smithi188.stdout:Setting up python3-google-auth (1.5.1-3) ... 2023-11-26T07:15:50.137 INFO:teuthology.orchestra.run.smithi143.stdout:Selecting previously unselected package libfl2:amd64. 2023-11-26T07:15:50.152 INFO:teuthology.orchestra.run.smithi143.stdout:Preparing to unpack .../07-libfl2_2.6.4-8build2_amd64.deb ... 2023-11-26T07:15:50.195 INFO:teuthology.orchestra.run.smithi143.stdout:Unpacking libfl2:amd64 (2.6.4-8build2) ... 2023-11-26T07:15:50.275 INFO:teuthology.orchestra.run.smithi188.stdout:Setting up librbd1 (18.0.0-7484-g44c2237f-1jammy) ... 2023-11-26T07:15:50.426 INFO:teuthology.orchestra.run.smithi188.stdout:Setting up ceph-mgr-modules-core (18.0.0-7484-g44c2237f-1jammy) ... 2023-11-26T07:15:50.473 INFO:teuthology.orchestra.run.smithi143.stdout:Selecting previously unselected package libfl-dev:amd64. 2023-11-26T07:15:50.488 INFO:teuthology.orchestra.run.smithi143.stdout:Preparing to unpack .../08-libfl-dev_2.6.4-8build2_amd64.deb ... 2023-11-26T07:15:50.532 INFO:teuthology.orchestra.run.smithi143.stdout:Unpacking libfl-dev:amd64 (2.6.4-8build2) ... 2023-11-26T07:15:50.544 INFO:teuthology.orchestra.run.smithi188.stdout:Setting up ceph-fuse (18.0.0-7484-g44c2237f-1jammy) ... 2023-11-26T07:15:50.736 INFO:teuthology.orchestra.run.smithi188.stdout:Created symlink /etc/systemd/system/remote-fs.target.wants/ceph-fuse.target -> /lib/systemd/system/ceph-fuse.target. 2023-11-26T07:15:50.737 INFO:teuthology.orchestra.run.smithi188.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-fuse.target -> /lib/systemd/system/ceph-fuse.target. 2023-11-26T07:15:50.776 INFO:teuthology.orchestra.run.smithi143.stdout:Selecting previously unselected package libjson-perl. 2023-11-26T07:15:50.790 INFO:teuthology.orchestra.run.smithi143.stdout:Preparing to unpack .../09-libjson-perl_4.04000-1_all.deb ... 2023-11-26T07:15:50.826 INFO:teuthology.orchestra.run.smithi143.stdout:Unpacking libjson-perl (4.04000-1) ... 2023-11-26T07:15:51.129 INFO:teuthology.orchestra.run.smithi143.stdout:Selecting previously unselected package libllvm14:amd64. 2023-11-26T07:15:51.145 INFO:teuthology.orchestra.run.smithi143.stdout:Preparing to unpack .../10-libllvm14_1%3a14.0.0-1ubuntu1.1_amd64.deb ... 2023-11-26T07:15:51.188 INFO:teuthology.orchestra.run.smithi143.stdout:Unpacking libllvm14:amd64 (1:14.0.0-1ubuntu1.1) ... 2023-11-26T07:15:51.395 INFO:teuthology.orchestra.run.smithi188.stdout:Setting up libcephfs-dev (18.0.0-7484-g44c2237f-1jammy) ... 2023-11-26T07:15:51.555 INFO:teuthology.orchestra.run.smithi188.stdout:Setting up python3-rados (18.0.0-7484-g44c2237f-1jammy) ... 2023-11-26T07:15:51.681 INFO:teuthology.orchestra.run.smithi188.stdout:Setting up cephfs-top (18.0.0-7484-g44c2237f-1jammy) ... 2023-11-26T07:15:51.807 INFO:teuthology.orchestra.run.smithi188.stdout:Setting up python3-kubernetes (12.0.1-1ubuntu1) ... 2023-11-26T07:15:52.599 INFO:teuthology.orchestra.run.smithi143.stdout:Selecting previously unselected package libmbim-glib4:amd64. 2023-11-26T07:15:52.614 INFO:teuthology.orchestra.run.smithi143.stdout:Preparing to unpack .../11-libmbim-glib4_1.28.0-1~ubuntu20.04.1_amd64.deb ... 2023-11-26T07:15:52.657 INFO:teuthology.orchestra.run.smithi143.stdout:Unpacking libmbim-glib4:amd64 (1.28.0-1~ubuntu20.04.1) ... 2023-11-26T07:15:52.893 INFO:teuthology.orchestra.run.smithi143.stdout:Selecting previously unselected package libmbim-proxy. 2023-11-26T07:15:52.909 INFO:teuthology.orchestra.run.smithi143.stdout:Preparing to unpack .../12-libmbim-proxy_1.28.0-1~ubuntu20.04.1_amd64.deb ... 2023-11-26T07:15:52.943 INFO:teuthology.orchestra.run.smithi143.stdout:Unpacking libmbim-proxy (1.28.0-1~ubuntu20.04.1) ... 2023-11-26T07:15:53.069 INFO:teuthology.orchestra.run.smithi188.stdout:Setting up python3-rbd (18.0.0-7484-g44c2237f-1jammy) ... 2023-11-26T07:15:53.246 INFO:teuthology.orchestra.run.smithi143.stdout:Selecting previously unselected package libmm-glib0:amd64. 2023-11-26T07:15:53.254 INFO:teuthology.orchestra.run.smithi188.stdout:Setting up rbd-fuse (18.0.0-7484-g44c2237f-1jammy) ... 2023-11-26T07:15:53.262 INFO:teuthology.orchestra.run.smithi143.stdout:Preparing to unpack .../13-libmm-glib0_1.20.0-1~ubuntu22.04.2_amd64.deb ... 2023-11-26T07:15:53.304 INFO:teuthology.orchestra.run.smithi143.stdout:Unpacking libmm-glib0:amd64 (1.20.0-1~ubuntu22.04.2) ... 2023-11-26T07:15:53.379 INFO:teuthology.orchestra.run.smithi188.stdout:Setting up python3-rgw (18.0.0-7484-g44c2237f-1jammy) ... 2023-11-26T07:15:53.506 INFO:teuthology.orchestra.run.smithi188.stdout:Setting up python3-cephfs (18.0.0-7484-g44c2237f-1jammy) ... 2023-11-26T07:15:53.623 INFO:teuthology.orchestra.run.smithi188.stdout:Setting up cephfs-shell (18.0.0-7484-g44c2237f-1jammy) ... 2023-11-26T07:15:53.665 INFO:teuthology.orchestra.run.smithi143.stdout:Selecting previously unselected package libndp0:amd64. 2023-11-26T07:15:53.681 INFO:teuthology.orchestra.run.smithi143.stdout:Preparing to unpack .../14-libndp0_1.8-0ubuntu3_amd64.deb ... 2023-11-26T07:15:53.724 INFO:teuthology.orchestra.run.smithi143.stdout:Unpacking libndp0:amd64 (1.8-0ubuntu3) ... 2023-11-26T07:15:53.741 INFO:teuthology.orchestra.run.smithi188.stdout:Setting up ceph-common (18.0.0-7484-g44c2237f-1jammy) ... 2023-11-26T07:15:54.051 INFO:teuthology.orchestra.run.smithi143.stdout:Selecting previously unselected package libnm0:amd64. 2023-11-26T07:15:54.067 INFO:teuthology.orchestra.run.smithi143.stdout:Preparing to unpack .../15-libnm0_1.36.6-0ubuntu2_amd64.deb ... 2023-11-26T07:15:54.110 INFO:teuthology.orchestra.run.smithi143.stdout:Unpacking libnm0:amd64 (1.36.6-0ubuntu2) ... 2023-11-26T07:15:54.345 INFO:teuthology.orchestra.run.smithi188.stdout:Adding group ceph....done 2023-11-26T07:15:54.438 INFO:teuthology.orchestra.run.smithi143.stdout:Selecting previously unselected package libpq5:amd64. 2023-11-26T07:15:54.454 INFO:teuthology.orchestra.run.smithi143.stdout:Preparing to unpack .../16-libpq5_14.9-0ubuntu0.22.04.1_amd64.deb ... 2023-11-26T07:15:54.496 INFO:teuthology.orchestra.run.smithi143.stdout:Unpacking libpq5:amd64 (14.9-0ubuntu0.22.04.1) ... 2023-11-26T07:15:54.636 INFO:teuthology.orchestra.run.smithi188.stdout:Adding system user ceph....done 2023-11-26T07:15:54.808 INFO:teuthology.orchestra.run.smithi143.stdout:Selecting previously unselected package libqmi-glib5:amd64. 2023-11-26T07:15:54.823 INFO:teuthology.orchestra.run.smithi188.stdout:Setting system user ceph properties....done 2023-11-26T07:15:54.824 INFO:teuthology.orchestra.run.smithi143.stdout:Preparing to unpack .../17-libqmi-glib5_1.32.0-1ubuntu0.22.04.1_amd64.deb ... 2023-11-26T07:15:54.837 INFO:teuthology.orchestra.run.smithi188.stdout:chown: cannot access '/var/log/ceph/*.log*': No such file or directory 2023-11-26T07:15:54.866 INFO:teuthology.orchestra.run.smithi143.stdout:Unpacking libqmi-glib5:amd64 (1.32.0-1ubuntu0.22.04.1) ... 2023-11-26T07:15:54.979 INFO:teuthology.orchestra.run.smithi188.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph.target -> /lib/systemd/system/ceph.target. 2023-11-26T07:15:55.177 INFO:teuthology.orchestra.run.smithi143.stdout:Selecting previously unselected package libqmi-proxy. 2023-11-26T07:15:55.192 INFO:teuthology.orchestra.run.smithi143.stdout:Preparing to unpack .../18-libqmi-proxy_1.32.0-1ubuntu0.22.04.1_amd64.deb ... 2023-11-26T07:15:55.235 INFO:teuthology.orchestra.run.smithi143.stdout:Unpacking libqmi-proxy (1.32.0-1ubuntu0.22.04.1) ... 2023-11-26T07:15:55.369 INFO:teuthology.orchestra.run.smithi188.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/rbdmap.service -> /lib/systemd/system/rbdmap.service. 2023-11-26T07:15:55.488 INFO:teuthology.orchestra.run.smithi143.stdout:Selecting previously unselected package libssl-dev:amd64. 2023-11-26T07:15:55.503 INFO:teuthology.orchestra.run.smithi143.stdout:Preparing to unpack .../19-libssl-dev_3.0.2-0ubuntu1.12_amd64.deb ... 2023-11-26T07:15:55.546 INFO:teuthology.orchestra.run.smithi143.stdout:Unpacking libssl-dev:amd64 (3.0.2-0ubuntu1.12) ... 2023-11-26T07:15:56.050 INFO:teuthology.orchestra.run.smithi143.stdout:Selecting previously unselected package libteamdctl0:amd64. 2023-11-26T07:15:56.064 INFO:teuthology.orchestra.run.smithi143.stdout:Preparing to unpack .../20-libteamdctl0_1.31-1build2_amd64.deb ... 2023-11-26T07:15:56.067 INFO:teuthology.orchestra.run.smithi188.stdout:Setting up cephfs-mirror (18.0.0-7484-g44c2237f-1jammy) ... 2023-11-26T07:15:56.108 INFO:teuthology.orchestra.run.smithi143.stdout:Unpacking libteamdctl0:amd64 (1.31-1build2) ... 2023-11-26T07:15:56.318 INFO:teuthology.orchestra.run.smithi188.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/cephfs-mirror.target -> /lib/systemd/system/cephfs-mirror.target. 2023-11-26T07:15:56.318 INFO:teuthology.orchestra.run.smithi188.stdout:Created symlink /etc/systemd/system/ceph.target.wants/cephfs-mirror.target -> /lib/systemd/system/cephfs-mirror.target. 2023-11-26T07:15:56.436 INFO:teuthology.orchestra.run.smithi143.stdout:Selecting previously unselected package modemmanager. 2023-11-26T07:15:56.451 INFO:teuthology.orchestra.run.smithi143.stdout:Preparing to unpack .../21-modemmanager_1.20.0-1~ubuntu22.04.2_amd64.deb ... 2023-11-26T07:15:56.494 INFO:teuthology.orchestra.run.smithi143.stdout:Unpacking modemmanager (1.20.0-1~ubuntu22.04.2) ... 2023-11-26T07:15:56.918 INFO:teuthology.orchestra.run.smithi188.stdout:Setting up ceph-test (18.0.0-7484-g44c2237f-1jammy) ... 2023-11-26T07:15:57.044 INFO:teuthology.orchestra.run.smithi188.stdout:Setting up radosgw (18.0.0-7484-g44c2237f-1jammy) ... 2023-11-26T07:15:57.118 INFO:teuthology.orchestra.run.smithi143.stdout:Selecting previously unselected package network-manager. 2023-11-26T07:15:57.133 INFO:teuthology.orchestra.run.smithi143.stdout:Preparing to unpack .../22-network-manager_1.36.6-0ubuntu2_amd64.deb ... 2023-11-26T07:15:57.184 INFO:teuthology.orchestra.run.smithi143.stdout:Unpacking network-manager (1.36.6-0ubuntu2) ... 2023-11-26T07:15:57.571 INFO:teuthology.orchestra.run.smithi188.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-radosgw.target -> /lib/systemd/system/ceph-radosgw.target. 2023-11-26T07:15:57.571 INFO:teuthology.orchestra.run.smithi188.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-radosgw.target -> /lib/systemd/system/ceph-radosgw.target. 2023-11-26T07:15:57.742 INFO:teuthology.orchestra.run.smithi143.stdout:Selecting previously unselected package ppp. 2023-11-26T07:15:57.758 INFO:teuthology.orchestra.run.smithi143.stdout:Preparing to unpack .../23-ppp_2.4.9-1+1ubuntu3_amd64.deb ... 2023-11-26T07:15:58.129 INFO:teuthology.orchestra.run.smithi143.stdout:Unpacking ppp (2.4.9-1+1ubuntu3) ... 2023-11-26T07:15:58.237 INFO:teuthology.orchestra.run.smithi188.stdout:Setting up ceph-base (18.0.0-7484-g44c2237f-1jammy) ... 2023-11-26T07:15:58.447 INFO:teuthology.orchestra.run.smithi143.stdout:Selecting previously unselected package pptp-linux. 2023-11-26T07:15:58.462 INFO:teuthology.orchestra.run.smithi143.stdout:Preparing to unpack .../24-pptp-linux_1.10.0-1build3_amd64.deb ... 2023-11-26T07:15:58.506 INFO:teuthology.orchestra.run.smithi143.stdout:Unpacking pptp-linux (1.10.0-1build3) ... 2023-11-26T07:15:58.633 INFO:teuthology.orchestra.run.smithi188.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-crash.service -> /lib/systemd/system/ceph-crash.service. 2023-11-26T07:15:58.758 INFO:teuthology.orchestra.run.smithi143.stdout:Selecting previously unselected package network-manager-pptp. 2023-11-26T07:15:58.773 INFO:teuthology.orchestra.run.smithi143.stdout:Preparing to unpack .../25-network-manager-pptp_1.2.10-1_amd64.deb ... 2023-11-26T07:15:58.817 INFO:teuthology.orchestra.run.smithi143.stdout:Unpacking network-manager-pptp (1.2.10-1) ... 2023-11-26T07:15:59.103 INFO:teuthology.orchestra.run.smithi143.stdout:Selecting previously unselected package postgresql-client-common. 2023-11-26T07:15:59.118 INFO:teuthology.orchestra.run.smithi143.stdout:Preparing to unpack .../26-postgresql-client-common_238_all.deb ... 2023-11-26T07:15:59.202 INFO:teuthology.orchestra.run.smithi143.stdout:Unpacking postgresql-client-common (238) ... 2023-11-26T07:15:59.289 INFO:teuthology.orchestra.run.smithi188.stdout:Setting up ceph-mds (18.0.0-7484-g44c2237f-1jammy) ... 2023-11-26T07:15:59.504 INFO:teuthology.orchestra.run.smithi188.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mds.target -> /lib/systemd/system/ceph-mds.target. 2023-11-26T07:15:59.504 INFO:teuthology.orchestra.run.smithi188.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mds.target -> /lib/systemd/system/ceph-mds.target. 2023-11-26T07:15:59.507 INFO:teuthology.orchestra.run.smithi143.stdout:Selecting previously unselected package postgresql-client-14. 2023-11-26T07:15:59.524 INFO:teuthology.orchestra.run.smithi143.stdout:Preparing to unpack .../27-postgresql-client-14_14.9-0ubuntu0.22.04.1_amd64.deb ... 2023-11-26T07:15:59.566 INFO:teuthology.orchestra.run.smithi143.stdout:Unpacking postgresql-client-14 (14.9-0ubuntu0.22.04.1) ... 2023-11-26T07:16:00.121 INFO:teuthology.orchestra.run.smithi143.stdout:Selecting previously unselected package postgresql-common. 2023-11-26T07:16:00.136 INFO:teuthology.orchestra.run.smithi143.stdout:Preparing to unpack .../28-postgresql-common_238_all.deb ... 2023-11-26T07:16:00.174 INFO:teuthology.orchestra.run.smithi188.stdout:Setting up ceph-mgr (18.0.0-7484-g44c2237f-1jammy) ... 2023-11-26T07:16:00.188 INFO:teuthology.orchestra.run.smithi143.stdout:Adding 'diversion of /usr/bin/pg_config to /usr/bin/pg_config.libpq-dev by postgresql-common' 2023-11-26T07:16:00.219 INFO:teuthology.orchestra.run.smithi143.stdout:Unpacking postgresql-common (238) ... 2023-11-26T07:16:00.387 INFO:teuthology.orchestra.run.smithi188.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mgr.target -> /lib/systemd/system/ceph-mgr.target. 2023-11-26T07:16:00.387 INFO:teuthology.orchestra.run.smithi188.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mgr.target -> /lib/systemd/system/ceph-mgr.target. 2023-11-26T07:16:00.696 INFO:teuthology.orchestra.run.smithi143.stdout:Selecting previously unselected package postgresql-14. 2023-11-26T07:16:00.710 INFO:teuthology.orchestra.run.smithi143.stdout:Preparing to unpack .../29-postgresql-14_14.9-0ubuntu0.22.04.1_amd64.deb ... 2023-11-26T07:16:00.756 INFO:teuthology.orchestra.run.smithi143.stdout:Unpacking postgresql-14 (14.9-0ubuntu0.22.04.1) ... 2023-11-26T07:16:01.000 INFO:teuthology.orchestra.run.smithi188.stdout:Setting up ceph-osd (18.0.0-7484-g44c2237f-1jammy) ... 2023-11-26T07:16:01.426 INFO:teuthology.orchestra.run.smithi188.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-osd.target -> /lib/systemd/system/ceph-osd.target. 2023-11-26T07:16:01.426 INFO:teuthology.orchestra.run.smithi188.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-osd.target -> /lib/systemd/system/ceph-osd.target. 2023-11-26T07:16:01.733 INFO:teuthology.orchestra.run.smithi143.stdout:Selecting previously unselected package postgresql. 2023-11-26T07:16:01.748 INFO:teuthology.orchestra.run.smithi143.stdout:Preparing to unpack .../30-postgresql_14+238_all.deb ... 2023-11-26T07:16:01.791 INFO:teuthology.orchestra.run.smithi143.stdout:Unpacking postgresql (14+238) ... 2023-11-26T07:16:02.052 INFO:teuthology.orchestra.run.smithi188.stdout:Setting up ceph-mgr-k8sevents (18.0.0-7484-g44c2237f-1jammy) ... 2023-11-26T07:16:02.076 INFO:teuthology.orchestra.run.smithi143.stdout:Selecting previously unselected package postgresql-client. 2023-11-26T07:16:02.091 INFO:teuthology.orchestra.run.smithi143.stdout:Preparing to unpack .../31-postgresql-client_14+238_all.deb ... 2023-11-26T07:16:02.135 INFO:teuthology.orchestra.run.smithi143.stdout:Unpacking postgresql-client (14+238) ... 2023-11-26T07:16:02.178 INFO:teuthology.orchestra.run.smithi188.stdout:Setting up ceph-mgr-diskprediction-local (18.0.0-7484-g44c2237f-1jammy) ... 2023-11-26T07:16:02.320 INFO:teuthology.orchestra.run.smithi188.stdout:Setting up ceph-mon (18.0.0-7484-g44c2237f-1jammy) ... 2023-11-26T07:16:02.463 INFO:teuthology.orchestra.run.smithi143.stdout:Selecting previously unselected package postgresql-contrib. 2023-11-26T07:16:02.477 INFO:teuthology.orchestra.run.smithi143.stdout:Preparing to unpack .../32-postgresql-contrib_14+238_all.deb ... 2023-11-26T07:16:02.521 INFO:teuthology.orchestra.run.smithi143.stdout:Unpacking postgresql-contrib (14+238) ... 2023-11-26T07:16:02.528 INFO:teuthology.orchestra.run.smithi188.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mon.target -> /lib/systemd/system/ceph-mon.target. 2023-11-26T07:16:02.529 INFO:teuthology.orchestra.run.smithi188.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mon.target -> /lib/systemd/system/ceph-mon.target. 2023-11-26T07:16:02.807 INFO:teuthology.orchestra.run.smithi143.stdout:Selecting previously unselected package tcl8.6. 2023-11-26T07:16:02.822 INFO:teuthology.orchestra.run.smithi143.stdout:Preparing to unpack .../33-tcl8.6_8.6.12+dfsg-1build1_amd64.deb ... 2023-11-26T07:16:02.857 INFO:teuthology.orchestra.run.smithi143.stdout:Unpacking tcl8.6 (8.6.12+dfsg-1build1) ... 2023-11-26T07:16:03.126 INFO:teuthology.orchestra.run.smithi143.stdout:Selecting previously unselected package tcl. 2023-11-26T07:16:03.141 INFO:teuthology.orchestra.run.smithi143.stdout:Preparing to unpack .../34-tcl_8.6.11+1build2_amd64.deb ... 2023-11-26T07:16:03.163 INFO:teuthology.orchestra.run.smithi188.stdout:Setting up ceph-mgr-cephadm (18.0.0-7484-g44c2237f-1jammy) ... 2023-11-26T07:16:03.189 INFO:teuthology.orchestra.run.smithi143.stdout:Unpacking tcl (8.6.11+1build2) ... 2023-11-26T07:16:03.305 INFO:teuthology.orchestra.run.smithi188.stdout:Setting up ceph (18.0.0-7484-g44c2237f-1jammy) ... 2023-11-26T07:16:03.440 INFO:teuthology.orchestra.run.smithi188.stdout:Setting up ceph-mgr-dashboard (18.0.0-7484-g44c2237f-1jammy) ... 2023-11-26T07:16:03.563 INFO:teuthology.orchestra.run.smithi143.stdout:Selecting previously unselected package wpasupplicant. 2023-11-26T07:16:03.578 INFO:teuthology.orchestra.run.smithi143.stdout:Preparing to unpack .../35-wpasupplicant_2%3a2.10-6ubuntu2_amd64.deb ... 2023-11-26T07:16:03.607 INFO:teuthology.orchestra.run.smithi188.stdout:Setting up ceph-volume (18.0.0-7484-g44c2237f-1jammy) ... 2023-11-26T07:16:03.621 INFO:teuthology.orchestra.run.smithi143.stdout:Unpacking wpasupplicant (2:2.10-6ubuntu2) ... 2023-11-26T07:16:03.934 INFO:teuthology.orchestra.run.smithi143.stdout:Selecting previously unselected package usb-modeswitch-data. 2023-11-26T07:16:03.949 INFO:teuthology.orchestra.run.smithi143.stdout:Preparing to unpack .../36-usb-modeswitch-data_20191128-4_all.deb ... 2023-11-26T07:16:03.992 INFO:teuthology.orchestra.run.smithi143.stdout:Unpacking usb-modeswitch-data (20191128-4) ... 2023-11-26T07:16:04.011 INFO:teuthology.orchestra.run.smithi188.stdout:Processing triggers for man-db (2.10.2-1) ... 2023-11-26T07:16:04.379 INFO:teuthology.orchestra.run.smithi143.stdout:Selecting previously unselected package usb-modeswitch. 2023-11-26T07:16:04.395 INFO:teuthology.orchestra.run.smithi143.stdout:Preparing to unpack .../37-usb-modeswitch_2.6.1-3ubuntu2_amd64.deb ... 2023-11-26T07:16:04.446 INFO:teuthology.orchestra.run.smithi143.stdout:Unpacking usb-modeswitch (2.6.1-3ubuntu2) ... 2023-11-26T07:16:04.721 INFO:teuthology.orchestra.run.smithi143.stdout:Setting up postgresql-client-common (238) ... 2023-11-26T07:16:04.902 INFO:teuthology.orchestra.run.smithi143.stdout:Setting up flex (2.6.4-8build2) ... 2023-11-26T07:16:05.020 INFO:teuthology.orchestra.run.smithi143.stdout:Setting up tcl8.6 (8.6.12+dfsg-1build1) ... 2023-11-26T07:16:05.145 INFO:teuthology.orchestra.run.smithi188.stdout:Processing triggers for libc-bin (2.35-0ubuntu3.1) ... 2023-11-26T07:16:05.154 INFO:teuthology.orchestra.run.smithi143.stdout:Setting up libpq5:amd64 (14.9-0ubuntu0.22.04.1) ... 2023-11-26T07:16:05.281 INFO:teuthology.orchestra.run.smithi143.stdout:Setting up liblzo2-2:amd64 (2.10-2build3) ... 2023-11-26T07:16:05.407 INFO:teuthology.orchestra.run.smithi143.stdout:Setting up libaio-dev:amd64 (0.3.112-13build1) ... 2023-11-26T07:16:05.532 INFO:teuthology.orchestra.run.smithi143.stdout:Setting up postgresql-client-14 (14.9-0ubuntu0.22.04.1) ... 2023-11-26T07:16:05.648 INFO:teuthology.orchestra.run.smithi188.stderr:W: --force-yes is deprecated, use one of the options starting with --allow instead. 2023-11-26T07:16:05.652 DEBUG:teuthology.orchestra.run.smithi188:> sudo DEBIAN_FRONTEND=noninteractive apt-get -y --force-yes -o Dpkg::Options::="--force-confdef" -o Dpkg::Options::="--force-confold" install bison flex libelf-dev libssl-dev network-manager iproute2 util-linux dump indent libaio-dev libtool-bin uuid-dev xfslibs-dev postgresql postgresql-client postgresql-common postgresql-contrib 2023-11-26T07:16:05.727 INFO:teuthology.orchestra.run.smithi188.stdout:Reading package lists... 2023-11-26T07:16:05.894 INFO:teuthology.orchestra.run.smithi188.stdout:Building dependency tree... 2023-11-26T07:16:05.895 INFO:teuthology.orchestra.run.smithi188.stdout:Reading state information... 2023-11-26T07:16:06.045 INFO:teuthology.orchestra.run.smithi188.stdout:iproute2 is already the newest version (5.15.0-1ubuntu2). 2023-11-26T07:16:06.045 INFO:teuthology.orchestra.run.smithi188.stdout:libtool-bin is already the newest version (2.4.6-15build2). 2023-11-26T07:16:06.045 INFO:teuthology.orchestra.run.smithi188.stdout:util-linux is already the newest version (2.37.2-4ubuntu3). 2023-11-26T07:16:06.045 INFO:teuthology.orchestra.run.smithi188.stdout:uuid-dev is already the newest version (2.37.2-4ubuntu3). 2023-11-26T07:16:06.046 INFO:teuthology.orchestra.run.smithi188.stdout:xfslibs-dev is already the newest version (5.13.0-1ubuntu2). 2023-11-26T07:16:06.046 INFO:teuthology.orchestra.run.smithi188.stdout:The following packages were automatically installed and are no longer required: 2023-11-26T07:16:06.046 INFO:teuthology.orchestra.run.smithi188.stdout: libboost-iostreams1.74.0 libboost-thread1.74.0 libgfapi0 libgfrpc0 libgfxdr0 2023-11-26T07:16:06.046 INFO:teuthology.orchestra.run.smithi188.stdout: libglusterfs0 libiscsi7 libpmemobj1 libpython2-dev libpython2-stdlib 2023-11-26T07:16:06.046 INFO:teuthology.orchestra.run.smithi188.stdout: libpython2.7 libpython2.7-dev libpython2.7-minimal libpython2.7-stdlib 2023-11-26T07:16:06.046 INFO:teuthology.orchestra.run.smithi188.stdout: linux-image-5.4.0-135-generic linux-modules-5.4.0-135-generic python2 2023-11-26T07:16:06.047 INFO:teuthology.orchestra.run.smithi188.stdout: python2-dev python2-minimal python2.7 python2.7-dev python2.7-minimal 2023-11-26T07:16:06.047 INFO:teuthology.orchestra.run.smithi188.stdout:Use 'sudo apt autoremove' to remove them. 2023-11-26T07:16:06.047 INFO:teuthology.orchestra.run.smithi188.stdout:The following additional packages will be installed: 2023-11-26T07:16:06.047 INFO:teuthology.orchestra.run.smithi188.stdout: libbluetooth3 libfl-dev libfl2 libjson-perl libllvm14 liblzo2-2 2023-11-26T07:16:06.047 INFO:teuthology.orchestra.run.smithi188.stdout: libmbim-glib4 libmbim-proxy libmm-glib0 libndp0 libnm0 libpq5 libqmi-glib5 2023-11-26T07:16:06.048 INFO:teuthology.orchestra.run.smithi188.stdout: libqmi-proxy libssl3 libteamdctl0 modemmanager network-manager-pptp 2023-11-26T07:16:06.048 INFO:teuthology.orchestra.run.smithi188.stdout: postgresql-14 postgresql-client-14 postgresql-client-common ppp pptp-linux 2023-11-26T07:16:06.048 INFO:teuthology.orchestra.run.smithi188.stdout: tcl tcl8.6 usb-modeswitch usb-modeswitch-data wpasupplicant 2023-11-26T07:16:06.048 INFO:teuthology.orchestra.run.smithi188.stdout:Suggested packages: 2023-11-26T07:16:06.048 INFO:teuthology.orchestra.run.smithi188.stdout: bison-doc flex-doc indent-doc libssl-doc avahi-autoipd libteam-utils 2023-11-26T07:16:06.048 INFO:teuthology.orchestra.run.smithi188.stdout: postgresql-doc postgresql-doc-14 tcl-tclreadline comgt wvdial wpagui 2023-11-26T07:16:06.048 INFO:teuthology.orchestra.run.smithi188.stdout: libengine-pkcs11-openssl 2023-11-26T07:16:06.097 INFO:teuthology.orchestra.run.smithi188.stdout:The following NEW packages will be installed: 2023-11-26T07:16:06.098 INFO:teuthology.orchestra.run.smithi188.stdout: bison dump flex indent libaio-dev libbluetooth3 libelf-dev libfl-dev libfl2 2023-11-26T07:16:06.098 INFO:teuthology.orchestra.run.smithi188.stdout: libjson-perl libllvm14 liblzo2-2 libmbim-glib4 libmbim-proxy libmm-glib0 2023-11-26T07:16:06.099 INFO:teuthology.orchestra.run.smithi188.stdout: libndp0 libnm0 libpq5 libqmi-glib5 libqmi-proxy libssl-dev libteamdctl0 2023-11-26T07:16:06.099 INFO:teuthology.orchestra.run.smithi188.stdout: modemmanager network-manager network-manager-pptp postgresql postgresql-14 2023-11-26T07:16:06.099 INFO:teuthology.orchestra.run.smithi188.stdout: postgresql-client postgresql-client-14 postgresql-client-common 2023-11-26T07:16:06.099 INFO:teuthology.orchestra.run.smithi188.stdout: postgresql-common postgresql-contrib ppp pptp-linux tcl tcl8.6 2023-11-26T07:16:06.099 INFO:teuthology.orchestra.run.smithi188.stdout: usb-modeswitch usb-modeswitch-data wpasupplicant 2023-11-26T07:16:06.100 INFO:teuthology.orchestra.run.smithi188.stdout:The following packages will be upgraded: 2023-11-26T07:16:06.101 INFO:teuthology.orchestra.run.smithi188.stdout: libssl3 2023-11-26T07:16:06.156 INFO:teuthology.orchestra.run.smithi188.stdout:1 upgraded, 39 newly installed, 0 to remove and 263 not upgraded. 2023-11-26T07:16:06.156 INFO:teuthology.orchestra.run.smithi188.stdout:Need to get 54.6 MB of archives. 2023-11-26T07:16:06.156 INFO:teuthology.orchestra.run.smithi188.stdout:After this operation, 202 MB of additional disk space will be used. 2023-11-26T07:16:06.156 INFO:teuthology.orchestra.run.smithi188.stdout:Get:1 http://archive.ubuntu.com/ubuntu jammy/main amd64 flex amd64 2.6.4-8build2 [307 kB] 2023-11-26T07:16:06.258 INFO:teuthology.orchestra.run.smithi188.stdout:Get:2 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 libssl3 amd64 3.0.2-0ubuntu1.12 [1,901 kB] 2023-11-26T07:16:06.313 INFO:teuthology.orchestra.run.smithi188.stdout:Get:3 http://archive.ubuntu.com/ubuntu jammy/main amd64 bison amd64 2:3.8.2+dfsg-1build1 [748 kB] 2023-11-26T07:16:06.320 INFO:teuthology.orchestra.run.smithi188.stdout:Get:4 http://archive.ubuntu.com/ubuntu jammy/main amd64 liblzo2-2 amd64 2.10-2build3 [53.7 kB] 2023-11-26T07:16:06.322 INFO:teuthology.orchestra.run.smithi188.stdout:Get:5 http://archive.ubuntu.com/ubuntu jammy/universe amd64 dump amd64 0.4b47-3 [133 kB] 2023-11-26T07:16:06.323 INFO:teuthology.orchestra.run.smithi188.stdout:Get:6 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 indent amd64 2.2.12-1ubuntu0.22.04.1 [90.8 kB] 2023-11-26T07:16:06.324 INFO:teuthology.orchestra.run.smithi188.stdout:Get:7 http://archive.ubuntu.com/ubuntu jammy/main amd64 libaio-dev amd64 0.3.112-13build1 [21.2 kB] 2023-11-26T07:16:06.325 INFO:teuthology.orchestra.run.smithi188.stdout:Get:8 http://archive.ubuntu.com/ubuntu jammy/main amd64 libbluetooth3 amd64 5.64-0ubuntu1 [86.9 kB] 2023-11-26T07:16:06.326 INFO:teuthology.orchestra.run.smithi188.stdout:Get:9 http://archive.ubuntu.com/ubuntu jammy/main amd64 libelf-dev amd64 0.186-1build1 [64.4 kB] 2023-11-26T07:16:06.327 INFO:teuthology.orchestra.run.smithi188.stdout:Get:10 http://archive.ubuntu.com/ubuntu jammy/main amd64 libfl2 amd64 2.6.4-8build2 [10.7 kB] 2023-11-26T07:16:06.327 INFO:teuthology.orchestra.run.smithi188.stdout:Get:11 http://archive.ubuntu.com/ubuntu jammy/main amd64 libfl-dev amd64 2.6.4-8build2 [6,236 B] 2023-11-26T07:16:06.332 INFO:teuthology.orchestra.run.smithi188.stdout:Get:12 http://archive.ubuntu.com/ubuntu jammy/main amd64 libjson-perl all 4.04000-1 [81.8 kB] 2023-11-26T07:16:06.344 INFO:teuthology.orchestra.run.smithi188.stdout:Get:13 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 libllvm14 amd64 1:14.0.0-1ubuntu1.1 [24.0 MB] 2023-11-26T07:16:06.536 INFO:teuthology.orchestra.run.smithi143.stdout:update-alternatives: using /usr/share/postgresql/14/man/man1/psql.1.gz to provide /usr/share/man/man1/psql.1.gz (psql.1.gz) in auto mode 2023-11-26T07:16:06.557 INFO:teuthology.orchestra.run.smithi188.stdout:Get:14 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 libmbim-glib4 amd64 1.28.0-1~ubuntu20.04.1 [191 kB] 2023-11-26T07:16:06.559 INFO:teuthology.orchestra.run.smithi188.stdout:Get:15 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 libmbim-proxy amd64 1.28.0-1~ubuntu20.04.1 [6,130 B] 2023-11-26T07:16:06.560 INFO:teuthology.orchestra.run.smithi188.stdout:Get:16 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 libmm-glib0 amd64 1.20.0-1~ubuntu22.04.2 [263 kB] 2023-11-26T07:16:06.562 INFO:teuthology.orchestra.run.smithi188.stdout:Get:17 http://archive.ubuntu.com/ubuntu jammy/main amd64 libndp0 amd64 1.8-0ubuntu3 [12.3 kB] 2023-11-26T07:16:06.563 INFO:teuthology.orchestra.run.smithi188.stdout:Get:18 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 libnm0 amd64 1.36.6-0ubuntu2 [456 kB] 2023-11-26T07:16:06.567 INFO:teuthology.orchestra.run.smithi188.stdout:Get:19 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 libpq5 amd64 14.9-0ubuntu0.22.04.1 [141 kB] 2023-11-26T07:16:06.569 INFO:teuthology.orchestra.run.smithi188.stdout:Get:20 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 libqmi-glib5 amd64 1.32.0-1ubuntu0.22.04.1 [772 kB] 2023-11-26T07:16:06.576 INFO:teuthology.orchestra.run.smithi188.stdout:Get:21 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 libqmi-proxy amd64 1.32.0-1ubuntu0.22.04.1 [6,072 B] 2023-11-26T07:16:06.576 INFO:teuthology.orchestra.run.smithi188.stdout:Get:22 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 libssl-dev amd64 3.0.2-0ubuntu1.12 [2,373 kB] 2023-11-26T07:16:06.598 INFO:teuthology.orchestra.run.smithi188.stdout:Get:23 http://archive.ubuntu.com/ubuntu jammy/main amd64 libteamdctl0 amd64 1.31-1build2 [12.6 kB] 2023-11-26T07:16:06.598 INFO:teuthology.orchestra.run.smithi188.stdout:Get:24 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 modemmanager amd64 1.20.0-1~ubuntu22.04.2 [1,094 kB] 2023-11-26T07:16:06.608 INFO:teuthology.orchestra.run.smithi188.stdout:Get:25 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 network-manager amd64 1.36.6-0ubuntu2 [2,203 kB] 2023-11-26T07:16:06.629 INFO:teuthology.orchestra.run.smithi188.stdout:Get:26 http://archive.ubuntu.com/ubuntu jammy/main amd64 ppp amd64 2.4.9-1+1ubuntu3 [363 kB] 2023-11-26T07:16:06.633 INFO:teuthology.orchestra.run.smithi188.stdout:Get:27 http://archive.ubuntu.com/ubuntu jammy/main amd64 pptp-linux amd64 1.10.0-1build3 [42.5 kB] 2023-11-26T07:16:06.633 INFO:teuthology.orchestra.run.smithi188.stdout:Get:28 http://archive.ubuntu.com/ubuntu jammy/main amd64 network-manager-pptp amd64 1.2.10-1 [30.9 kB] 2023-11-26T07:16:06.634 INFO:teuthology.orchestra.run.smithi188.stdout:Get:29 http://archive.ubuntu.com/ubuntu jammy/main amd64 postgresql-client-common all 238 [29.6 kB] 2023-11-26T07:16:06.634 INFO:teuthology.orchestra.run.smithi188.stdout:Get:30 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 postgresql-client-14 amd64 14.9-0ubuntu0.22.04.1 [1,222 kB] 2023-11-26T07:16:06.646 INFO:teuthology.orchestra.run.smithi188.stdout:Get:31 http://archive.ubuntu.com/ubuntu jammy/main amd64 postgresql-common all 238 [169 kB] 2023-11-26T07:16:06.647 INFO:teuthology.orchestra.run.smithi188.stdout:Get:32 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 postgresql-14 amd64 14.9-0ubuntu0.22.04.1 [16.1 MB] 2023-11-26T07:16:06.794 INFO:teuthology.orchestra.run.smithi188.stdout:Get:33 http://archive.ubuntu.com/ubuntu jammy/main amd64 postgresql all 14+238 [3,288 B] 2023-11-26T07:16:06.794 INFO:teuthology.orchestra.run.smithi188.stdout:Get:34 http://archive.ubuntu.com/ubuntu jammy/main amd64 postgresql-client all 14+238 [3,292 B] 2023-11-26T07:16:06.795 INFO:teuthology.orchestra.run.smithi188.stdout:Get:35 http://archive.ubuntu.com/ubuntu jammy/main amd64 postgresql-contrib all 14+238 [3,292 B] 2023-11-26T07:16:06.795 INFO:teuthology.orchestra.run.smithi188.stdout:Get:36 http://archive.ubuntu.com/ubuntu jammy/main amd64 tcl8.6 amd64 8.6.12+dfsg-1build1 [15.0 kB] 2023-11-26T07:16:06.795 INFO:teuthology.orchestra.run.smithi188.stdout:Get:37 http://archive.ubuntu.com/ubuntu jammy/main amd64 tcl amd64 8.6.11+1build2 [4,678 B] 2023-11-26T07:16:06.795 INFO:teuthology.orchestra.run.smithi188.stdout:Get:38 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 wpasupplicant amd64 2:2.10-6ubuntu2 [1,481 kB] 2023-11-26T07:16:06.808 INFO:teuthology.orchestra.run.smithi188.stdout:Get:39 http://archive.ubuntu.com/ubuntu jammy/main amd64 usb-modeswitch-data all 20191128-4 [33.2 kB] 2023-11-26T07:16:06.809 INFO:teuthology.orchestra.run.smithi188.stdout:Get:40 http://archive.ubuntu.com/ubuntu jammy/main amd64 usb-modeswitch amd64 2.6.1-3ubuntu2 [46.0 kB] 2023-11-26T07:16:06.972 INFO:teuthology.orchestra.run.smithi143.stdout:Setting up wpasupplicant (2:2.10-6ubuntu2) ... 2023-11-26T07:16:07.186 INFO:teuthology.orchestra.run.smithi188.stdout:Preconfiguring packages ... 2023-11-26T07:16:07.343 INFO:teuthology.orchestra.run.smithi188.stdout:Fetched 54.6 MB in 1s (77.9 MB/s) 2023-11-26T07:16:07.353 INFO:teuthology.orchestra.run.smithi143.stdout:Created symlink /etc/systemd/system/dbus-fi.w1.wpa_supplicant1.service -> /lib/systemd/system/wpa_supplicant.service. 2023-11-26T07:16:07.353 INFO:teuthology.orchestra.run.smithi143.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/wpa_supplicant.service -> /lib/systemd/system/wpa_supplicant.service. 2023-11-26T07:16:07.485 INFO:teuthology.orchestra.run.smithi188.stdout:Selecting previously unselected package flex. 2023-11-26T07:16:07.556 INFO:teuthology.orchestra.run.smithi188.stdout:(Reading database ... (Reading database ... 5% (Reading database ... 10% (Reading database ... 15% (Reading database ... 20% (Reading database ... 25% (Reading database ... 30% (Reading database ... 35% (Reading database ... 40% (Reading database ... 45% (Reading database ... 50% (Reading database ... 55% (Reading database ... 60% (Reading database ... 65% (Reading database ... 70% (Reading database ... 75% (Reading database ... 80% (Reading database ... 85% (Reading database ... 90% (Reading database ... 95% (Reading database ... 100% (Reading database ... 139845 files and directories currently installed.) 2023-11-26T07:16:07.560 INFO:teuthology.orchestra.run.smithi188.stdout:Preparing to unpack .../flex_2.6.4-8build2_amd64.deb ... 2023-11-26T07:16:07.600 INFO:teuthology.orchestra.run.smithi188.stdout:Unpacking flex (2.6.4-8build2) ... 2023-11-26T07:16:08.015 INFO:teuthology.orchestra.run.smithi143.stdout:Setting up libteamdctl0:amd64 (1.31-1build2) ... 2023-11-26T07:16:08.141 INFO:teuthology.orchestra.run.smithi143.stdout:Setting up libssl-dev:amd64 (3.0.2-0ubuntu1.12) ... 2023-11-26T07:16:08.145 INFO:teuthology.orchestra.run.smithi188.stdout:Preparing to unpack .../libssl3_3.0.2-0ubuntu1.12_amd64.deb ... 2023-11-26T07:16:08.251 INFO:teuthology.orchestra.run.smithi143.stdout:Setting up usb-modeswitch-data (20191128-4) ... 2023-11-26T07:16:08.339 INFO:teuthology.orchestra.run.smithi188.stdout:Unpacking libssl3:amd64 (3.0.2-0ubuntu1.12) over (3.0.2-0ubuntu1.7) ... 2023-11-26T07:16:08.352 INFO:teuthology.orchestra.run.smithi143.stdout:Setting up libelf-dev:amd64 (0.186-1build1) ... 2023-11-26T07:16:08.453 INFO:teuthology.orchestra.run.smithi143.stdout:Setting up libfl2:amd64 (2.6.4-8build2) ... 2023-11-26T07:16:08.579 INFO:teuthology.orchestra.run.smithi143.stdout:Setting up libnm0:amd64 (1.36.6-0ubuntu2) ... 2023-11-26T07:16:08.705 INFO:teuthology.orchestra.run.smithi143.stdout:Setting up ppp (2.4.9-1+1ubuntu3) ... 2023-11-26T07:16:08.908 INFO:teuthology.orchestra.run.smithi188.stdout:Setting up libssl3:amd64 (3.0.2-0ubuntu1.12) ... 2023-11-26T07:16:09.369 INFO:teuthology.orchestra.run.smithi143.stdout:Setting up libllvm14:amd64 (1:14.0.0-1ubuntu1.1) ... 2023-11-26T07:16:09.467 INFO:teuthology.orchestra.run.smithi188.stdout:Selecting previously unselected package bison. 2023-11-26T07:16:09.495 INFO:teuthology.orchestra.run.smithi143.stdout:Setting up dump (0.4b47-3) ... 2023-11-26T07:16:09.541 INFO:teuthology.orchestra.run.smithi188.stdout:(Reading database ... (Reading database ... 5% (Reading database ... 10% (Reading database ... 15% (Reading database ... 20% (Reading database ... 25% (Reading database ... 30% (Reading database ... 35% (Reading database ... 40% (Reading database ... 45% (Reading database ... 50% (Reading database ... 55% (Reading database ... 60% (Reading database ... 65% (Reading database ... 70% (Reading database ... 75% (Reading database ... 80% (Reading database ... 85% (Reading database ... 90% (Reading database ... 95% (Reading database ... 100% (Reading database ... 139918 files and directories currently installed.) 2023-11-26T07:16:09.545 INFO:teuthology.orchestra.run.smithi188.stdout:Preparing to unpack .../00-bison_2%3a3.8.2+dfsg-1build1_amd64.deb ... 2023-11-26T07:16:09.586 INFO:teuthology.orchestra.run.smithi188.stdout:Unpacking bison (2:3.8.2+dfsg-1build1) ... 2023-11-26T07:16:09.591 INFO:teuthology.orchestra.run.smithi143.stdout:update-alternatives: using /usr/sbin/rmt-dump to provide /usr/sbin/rmt (rmt) in auto mode 2023-11-26T07:16:09.647 INFO:teuthology.orchestra.run.smithi143.stdout:Setting up libmm-glib0:amd64 (1.20.0-1~ubuntu22.04.2) ... 2023-11-26T07:16:09.773 INFO:teuthology.orchestra.run.smithi143.stdout:Setting up libbluetooth3:amd64 (5.64-0ubuntu1) ... 2023-11-26T07:16:09.899 INFO:teuthology.orchestra.run.smithi143.stdout:Setting up tcl (8.6.11+1build2) ... 2023-11-26T07:16:10.017 INFO:teuthology.orchestra.run.smithi143.stdout:Setting up libjson-perl (4.04000-1) ... 2023-11-26T07:16:10.045 INFO:teuthology.orchestra.run.smithi188.stdout:Selecting previously unselected package liblzo2-2:amd64. 2023-11-26T07:16:10.059 INFO:teuthology.orchestra.run.smithi188.stdout:Preparing to unpack .../01-liblzo2-2_2.10-2build3_amd64.deb ... 2023-11-26T07:16:10.103 INFO:teuthology.orchestra.run.smithi188.stdout:Unpacking liblzo2-2:amd64 (2.10-2build3) ... 2023-11-26T07:16:10.135 INFO:teuthology.orchestra.run.smithi143.stdout:Setting up bison (2:3.8.2+dfsg-1build1) ... 2023-11-26T07:16:10.234 INFO:teuthology.orchestra.run.smithi143.stdout:update-alternatives: using /usr/bin/bison.yacc to provide /usr/bin/yacc (yacc) in auto mode 2023-11-26T07:16:10.312 INFO:teuthology.orchestra.run.smithi143.stdout:Setting up indent (2.2.12-1ubuntu0.22.04.1) ... 2023-11-26T07:16:10.398 INFO:teuthology.orchestra.run.smithi188.stdout:Selecting previously unselected package dump. 2023-11-26T07:16:10.412 INFO:teuthology.orchestra.run.smithi188.stdout:Preparing to unpack .../02-dump_0.4b47-3_amd64.deb ... 2023-11-26T07:16:10.446 INFO:teuthology.orchestra.run.smithi143.stdout:Setting up usb-modeswitch (2.6.1-3ubuntu2) ... 2023-11-26T07:16:10.456 INFO:teuthology.orchestra.run.smithi188.stdout:Unpacking dump (0.4b47-3) ... 2023-11-26T07:16:10.622 INFO:teuthology.orchestra.run.smithi143.stdout:Setting up libndp0:amd64 (1.8-0ubuntu3) ... 2023-11-26T07:16:10.749 INFO:teuthology.orchestra.run.smithi143.stdout:Setting up libmbim-glib4:amd64 (1.28.0-1~ubuntu20.04.1) ... 2023-11-26T07:16:10.759 INFO:teuthology.orchestra.run.smithi188.stdout:Selecting previously unselected package indent. 2023-11-26T07:16:10.775 INFO:teuthology.orchestra.run.smithi188.stdout:Preparing to unpack .../03-indent_2.2.12-1ubuntu0.22.04.1_amd64.deb ... 2023-11-26T07:16:10.818 INFO:teuthology.orchestra.run.smithi188.stdout:Unpacking indent (2.2.12-1ubuntu0.22.04.1) ... 2023-11-26T07:16:10.875 INFO:teuthology.orchestra.run.smithi143.stdout:Setting up pptp-linux (1.10.0-1build3) ... 2023-11-26T07:16:11.034 INFO:teuthology.orchestra.run.smithi143.stdout:Setting up postgresql-client (14+238) ... 2023-11-26T07:16:11.053 INFO:teuthology.orchestra.run.smithi188.stdout:Selecting previously unselected package libaio-dev:amd64. 2023-11-26T07:16:11.069 INFO:teuthology.orchestra.run.smithi188.stdout:Preparing to unpack .../04-libaio-dev_0.3.112-13build1_amd64.deb ... 2023-11-26T07:16:11.112 INFO:teuthology.orchestra.run.smithi188.stdout:Unpacking libaio-dev:amd64 (0.3.112-13build1) ... 2023-11-26T07:16:11.135 INFO:teuthology.orchestra.run.smithi143.stdout:Setting up network-manager (1.36.6-0ubuntu2) ... 2023-11-26T07:16:11.423 INFO:teuthology.orchestra.run.smithi188.stdout:Selecting previously unselected package libbluetooth3:amd64. 2023-11-26T07:16:11.439 INFO:teuthology.orchestra.run.smithi188.stdout:Preparing to unpack .../05-libbluetooth3_5.64-0ubuntu1_amd64.deb ... 2023-11-26T07:16:11.474 INFO:teuthology.orchestra.run.smithi188.stdout:Unpacking libbluetooth3:amd64 (5.64-0ubuntu1) ... 2023-11-26T07:16:11.749 INFO:teuthology.orchestra.run.smithi143.stdout: 2023-11-26T07:16:11.749 INFO:teuthology.orchestra.run.smithi143.stdout:The following network interfaces were found in /etc/network/interfaces 2023-11-26T07:16:11.749 INFO:teuthology.orchestra.run.smithi143.stdout:which means they are currently configured by ifupdown: 2023-11-26T07:16:11.749 INFO:teuthology.orchestra.run.smithi143.stdout:- enp3s0f1 2023-11-26T07:16:11.749 INFO:teuthology.orchestra.run.smithi143.stdout:If you want to manage those interfaces with NetworkManager instead 2023-11-26T07:16:11.749 INFO:teuthology.orchestra.run.smithi143.stdout:remove their configuration from /etc/network/interfaces. 2023-11-26T07:16:11.749 INFO:teuthology.orchestra.run.smithi143.stdout: 2023-11-26T07:16:11.881 INFO:teuthology.orchestra.run.smithi143.stdout:Created symlink /etc/systemd/system/dbus-org.freedesktop.nm-dispatcher.service -> /lib/systemd/system/NetworkManager-dispatcher.service. 2023-11-26T07:16:11.909 INFO:teuthology.orchestra.run.smithi188.stdout:Selecting previously unselected package libelf-dev:amd64. 2023-11-26T07:16:11.925 INFO:teuthology.orchestra.run.smithi188.stdout:Preparing to unpack .../06-libelf-dev_0.186-1build1_amd64.deb ... 2023-11-26T07:16:11.967 INFO:teuthology.orchestra.run.smithi188.stdout:Unpacking libelf-dev:amd64 (0.186-1build1) ... 2023-11-26T07:16:12.273 INFO:teuthology.orchestra.run.smithi143.stdout:Created symlink /etc/systemd/system/network-online.target.wants/NetworkManager-wait-online.service -> /lib/systemd/system/NetworkManager-wait-online.service. 2023-11-26T07:16:12.337 INFO:teuthology.orchestra.run.smithi188.stdout:Selecting previously unselected package libfl2:amd64. 2023-11-26T07:16:12.353 INFO:teuthology.orchestra.run.smithi188.stdout:Preparing to unpack .../07-libfl2_2.6.4-8build2_amd64.deb ... 2023-11-26T07:16:12.395 INFO:teuthology.orchestra.run.smithi188.stdout:Unpacking libfl2:amd64 (2.6.4-8build2) ... 2023-11-26T07:16:12.648 INFO:teuthology.orchestra.run.smithi188.stdout:Selecting previously unselected package libfl-dev:amd64. 2023-11-26T07:16:12.663 INFO:teuthology.orchestra.run.smithi188.stdout:Preparing to unpack .../08-libfl-dev_2.6.4-8build2_amd64.deb ... 2023-11-26T07:16:12.686 INFO:teuthology.orchestra.run.smithi143.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/NetworkManager.service -> /lib/systemd/system/NetworkManager.service. 2023-11-26T07:16:12.706 INFO:teuthology.orchestra.run.smithi188.stdout:Unpacking libfl-dev:amd64 (2.6.4-8build2) ... 2023-11-26T07:16:12.983 INFO:teuthology.orchestra.run.smithi188.stdout:Selecting previously unselected package libjson-perl. 2023-11-26T07:16:12.999 INFO:teuthology.orchestra.run.smithi188.stdout:Preparing to unpack .../09-libjson-perl_4.04000-1_all.deb ... 2023-11-26T07:16:13.033 INFO:teuthology.orchestra.run.smithi188.stdout:Unpacking libjson-perl (4.04000-1) ... 2023-11-26T07:16:13.311 INFO:teuthology.orchestra.run.smithi188.stdout:Selecting previously unselected package libllvm14:amd64. 2023-11-26T07:16:13.327 INFO:teuthology.orchestra.run.smithi188.stdout:Preparing to unpack .../10-libllvm14_1%3a14.0.0-1ubuntu1.1_amd64.deb ... 2023-11-26T07:16:13.369 INFO:teuthology.orchestra.run.smithi188.stdout:Unpacking libllvm14:amd64 (1:14.0.0-1ubuntu1.1) ... 2023-11-26T07:16:13.447 INFO:teuthology.orchestra.run.smithi143.stdout:Setting up libmbim-proxy (1.28.0-1~ubuntu20.04.1) ... 2023-11-26T07:16:13.598 INFO:teuthology.orchestra.run.smithi143.stdout:Setting up libfl-dev:amd64 (2.6.4-8build2) ... 2023-11-26T07:16:13.725 INFO:teuthology.orchestra.run.smithi143.stdout:Setting up network-manager-pptp (1.2.10-1) ... 2023-11-26T07:16:13.834 INFO:teuthology.orchestra.run.smithi143.stdout:Setting up postgresql-common (238) ... 2023-11-26T07:16:14.741 INFO:teuthology.orchestra.run.smithi143.stdout:Adding user postgres to group ssl-cert 2023-11-26T07:16:14.848 INFO:teuthology.orchestra.run.smithi188.stdout:Selecting previously unselected package libmbim-glib4:amd64. 2023-11-26T07:16:14.864 INFO:teuthology.orchestra.run.smithi188.stdout:Preparing to unpack .../11-libmbim-glib4_1.28.0-1~ubuntu20.04.1_amd64.deb ... 2023-11-26T07:16:14.914 INFO:teuthology.orchestra.run.smithi188.stdout:Unpacking libmbim-glib4:amd64 (1.28.0-1~ubuntu20.04.1) ... 2023-11-26T07:16:15.004 INFO:teuthology.orchestra.run.smithi143.stdout: 2023-11-26T07:16:15.004 INFO:teuthology.orchestra.run.smithi143.stdout:Creating config file /etc/postgresql-common/createcluster.conf with new version 2023-11-26T07:16:15.200 INFO:teuthology.orchestra.run.smithi188.stdout:Selecting previously unselected package libmbim-proxy. 2023-11-26T07:16:15.216 INFO:teuthology.orchestra.run.smithi188.stdout:Preparing to unpack .../12-libmbim-proxy_1.28.0-1~ubuntu20.04.1_amd64.deb ... 2023-11-26T07:16:15.257 INFO:teuthology.orchestra.run.smithi143.stdout:Building PostgreSQL dictionaries from installed myspell/hunspell packages... 2023-11-26T07:16:15.257 INFO:teuthology.orchestra.run.smithi143.stdout:Removing obsolete dictionary files: 2023-11-26T07:16:15.258 INFO:teuthology.orchestra.run.smithi188.stdout:Unpacking libmbim-proxy (1.28.0-1~ubuntu20.04.1) ... 2023-11-26T07:16:15.594 INFO:teuthology.orchestra.run.smithi188.stdout:Selecting previously unselected package libmm-glib0:amd64. 2023-11-26T07:16:15.610 INFO:teuthology.orchestra.run.smithi188.stdout:Preparing to unpack .../13-libmm-glib0_1.20.0-1~ubuntu22.04.2_amd64.deb ... 2023-11-26T07:16:15.653 INFO:teuthology.orchestra.run.smithi188.stdout:Unpacking libmm-glib0:amd64 (1.20.0-1~ubuntu22.04.2) ... 2023-11-26T07:16:15.941 INFO:teuthology.orchestra.run.smithi143.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/postgresql.service -> /lib/systemd/system/postgresql.service. 2023-11-26T07:16:15.980 INFO:teuthology.orchestra.run.smithi188.stdout:Selecting previously unselected package libndp0:amd64. 2023-11-26T07:16:15.996 INFO:teuthology.orchestra.run.smithi188.stdout:Preparing to unpack .../14-libndp0_1.8-0ubuntu3_amd64.deb ... 2023-11-26T07:16:16.038 INFO:teuthology.orchestra.run.smithi188.stdout:Unpacking libndp0:amd64 (1.8-0ubuntu3) ... 2023-11-26T07:16:16.400 INFO:teuthology.orchestra.run.smithi188.stdout:Selecting previously unselected package libnm0:amd64. 2023-11-26T07:16:16.404 INFO:teuthology.orchestra.run.smithi143.stdout:Setting up postgresql-14 (14.9-0ubuntu0.22.04.1) ... 2023-11-26T07:16:16.415 INFO:teuthology.orchestra.run.smithi188.stdout:Preparing to unpack .../15-libnm0_1.36.6-0ubuntu2_amd64.deb ... 2023-11-26T07:16:16.458 INFO:teuthology.orchestra.run.smithi188.stdout:Unpacking libnm0:amd64 (1.36.6-0ubuntu2) ... 2023-11-26T07:16:16.749 INFO:teuthology.orchestra.run.smithi143.stdout:Creating new PostgreSQL cluster 14/main ... 2023-11-26T07:16:16.769 INFO:teuthology.orchestra.run.smithi143.stdout:/usr/lib/postgresql/14/bin/initdb -D /var/lib/postgresql/14/main --auth-local peer --auth-host scram-sha-256 --no-instructions 2023-11-26T07:16:16.811 INFO:teuthology.orchestra.run.smithi188.stdout:Selecting previously unselected package libpq5:amd64. 2023-11-26T07:16:16.812 INFO:teuthology.orchestra.run.smithi143.stdout:The files belonging to this database system will be owned by user "postgres". 2023-11-26T07:16:16.812 INFO:teuthology.orchestra.run.smithi143.stdout:This user must also own the server process. 2023-11-26T07:16:16.812 INFO:teuthology.orchestra.run.smithi143.stdout: 2023-11-26T07:16:16.812 INFO:teuthology.orchestra.run.smithi143.stdout:The database cluster will be initialized with locale "en_US". 2023-11-26T07:16:16.812 INFO:teuthology.orchestra.run.smithi143.stdout:The default database encoding has accordingly been set to "LATIN1". 2023-11-26T07:16:16.812 INFO:teuthology.orchestra.run.smithi143.stdout:The default text search configuration will be set to "english". 2023-11-26T07:16:16.812 INFO:teuthology.orchestra.run.smithi143.stdout: 2023-11-26T07:16:16.812 INFO:teuthology.orchestra.run.smithi143.stdout:Data page checksums are disabled. 2023-11-26T07:16:16.812 INFO:teuthology.orchestra.run.smithi143.stdout: 2023-11-26T07:16:16.812 INFO:teuthology.orchestra.run.smithi143.stdout:fixing permissions on existing directory /var/lib/postgresql/14/main ... ok 2023-11-26T07:16:16.813 INFO:teuthology.orchestra.run.smithi143.stdout:creating subdirectories ... ok 2023-11-26T07:16:16.813 INFO:teuthology.orchestra.run.smithi143.stdout:selecting dynamic shared memory implementation ... posix 2023-11-26T07:16:16.826 INFO:teuthology.orchestra.run.smithi188.stdout:Preparing to unpack .../16-libpq5_14.9-0ubuntu0.22.04.1_amd64.deb ... 2023-11-26T07:16:16.837 INFO:teuthology.orchestra.run.smithi143.stdout:selecting default max_connections ... 100 2023-11-26T07:16:16.861 INFO:teuthology.orchestra.run.smithi188.stdout:Unpacking libpq5:amd64 (14.9-0ubuntu0.22.04.1) ... 2023-11-26T07:16:16.872 INFO:teuthology.orchestra.run.smithi143.stdout:selecting default shared_buffers ... 128MB 2023-11-26T07:16:16.886 INFO:teuthology.orchestra.run.smithi143.stdout:selecting default time zone ... Etc/UTC 2023-11-26T07:16:16.887 INFO:teuthology.orchestra.run.smithi143.stdout:creating configuration files ... ok 2023-11-26T07:16:17.055 INFO:teuthology.orchestra.run.smithi143.stdout:running bootstrap script ... ok 2023-11-26T07:16:17.163 INFO:teuthology.orchestra.run.smithi188.stdout:Selecting previously unselected package libqmi-glib5:amd64. 2023-11-26T07:16:17.179 INFO:teuthology.orchestra.run.smithi188.stdout:Preparing to unpack .../17-libqmi-glib5_1.32.0-1ubuntu0.22.04.1_amd64.deb ... 2023-11-26T07:16:17.222 INFO:teuthology.orchestra.run.smithi188.stdout:Unpacking libqmi-glib5:amd64 (1.32.0-1ubuntu0.22.04.1) ... 2023-11-26T07:16:17.549 INFO:teuthology.orchestra.run.smithi188.stdout:Selecting previously unselected package libqmi-proxy. 2023-11-26T07:16:17.565 INFO:teuthology.orchestra.run.smithi188.stdout:Preparing to unpack .../18-libqmi-proxy_1.32.0-1ubuntu0.22.04.1_amd64.deb ... 2023-11-26T07:16:17.607 INFO:teuthology.orchestra.run.smithi188.stdout:Unpacking libqmi-proxy (1.32.0-1ubuntu0.22.04.1) ... 2023-11-26T07:16:17.700 INFO:teuthology.orchestra.run.smithi143.stdout:performing post-bootstrap initialization ... ok 2023-11-26T07:16:17.884 INFO:teuthology.orchestra.run.smithi188.stdout:Selecting previously unselected package libssl-dev:amd64. 2023-11-26T07:16:17.900 INFO:teuthology.orchestra.run.smithi188.stdout:Preparing to unpack .../19-libssl-dev_3.0.2-0ubuntu1.12_amd64.deb ... 2023-11-26T07:16:17.943 INFO:teuthology.orchestra.run.smithi188.stdout:Unpacking libssl-dev:amd64 (3.0.2-0ubuntu1.12) ... 2023-11-26T07:16:18.211 INFO:teuthology.orchestra.run.smithi143.stdout:syncing data to disk ... ok 2023-11-26T07:16:18.471 INFO:teuthology.orchestra.run.smithi188.stdout:Selecting previously unselected package libteamdctl0:amd64. 2023-11-26T07:16:18.486 INFO:teuthology.orchestra.run.smithi188.stdout:Preparing to unpack .../20-libteamdctl0_1.31-1build2_amd64.deb ... 2023-11-26T07:16:18.529 INFO:teuthology.orchestra.run.smithi188.stdout:Unpacking libteamdctl0:amd64 (1.31-1build2) ... 2023-11-26T07:16:18.735 INFO:teuthology.orchestra.run.smithi143.stdout:update-alternatives: using /usr/share/postgresql/14/man/man1/postmaster.1.gz to provide /usr/share/man/man1/postmaster.1.gz (postmaster.1.gz) in auto mode 2023-11-26T07:16:18.865 INFO:teuthology.orchestra.run.smithi188.stdout:Selecting previously unselected package modemmanager. 2023-11-26T07:16:18.881 INFO:teuthology.orchestra.run.smithi188.stdout:Preparing to unpack .../21-modemmanager_1.20.0-1~ubuntu22.04.2_amd64.deb ... 2023-11-26T07:16:18.923 INFO:teuthology.orchestra.run.smithi188.stdout:Unpacking modemmanager (1.20.0-1~ubuntu22.04.2) ... 2023-11-26T07:16:20.014 INFO:teuthology.orchestra.run.smithi188.stdout:Selecting previously unselected package network-manager. 2023-11-26T07:16:20.029 INFO:teuthology.orchestra.run.smithi188.stdout:Preparing to unpack .../22-network-manager_1.36.6-0ubuntu2_amd64.deb ... 2023-11-26T07:16:20.080 INFO:teuthology.orchestra.run.smithi188.stdout:Unpacking network-manager (1.36.6-0ubuntu2) ... 2023-11-26T07:16:20.645 INFO:teuthology.orchestra.run.smithi188.stdout:Selecting previously unselected package ppp. 2023-11-26T07:16:20.661 INFO:teuthology.orchestra.run.smithi188.stdout:Preparing to unpack .../23-ppp_2.4.9-1+1ubuntu3_amd64.deb ... 2023-11-26T07:16:21.011 INFO:teuthology.orchestra.run.smithi188.stdout:Unpacking ppp (2.4.9-1+1ubuntu3) ... 2023-11-26T07:16:21.341 INFO:teuthology.orchestra.run.smithi188.stdout:Selecting previously unselected package pptp-linux. 2023-11-26T07:16:21.355 INFO:teuthology.orchestra.run.smithi188.stdout:Preparing to unpack .../24-pptp-linux_1.10.0-1build3_amd64.deb ... 2023-11-26T07:16:21.400 INFO:teuthology.orchestra.run.smithi188.stdout:Unpacking pptp-linux (1.10.0-1build3) ... 2023-11-26T07:16:21.685 INFO:teuthology.orchestra.run.smithi188.stdout:Selecting previously unselected package network-manager-pptp. 2023-11-26T07:16:21.700 INFO:teuthology.orchestra.run.smithi188.stdout:Preparing to unpack .../25-network-manager-pptp_1.2.10-1_amd64.deb ... 2023-11-26T07:16:21.729 INFO:teuthology.orchestra.run.smithi143.stdout:Setting up libqmi-glib5:amd64 (1.32.0-1ubuntu0.22.04.1) ... 2023-11-26T07:16:21.743 INFO:teuthology.orchestra.run.smithi188.stdout:Unpacking network-manager-pptp (1.2.10-1) ... 2023-11-26T07:16:21.855 INFO:teuthology.orchestra.run.smithi143.stdout:Setting up postgresql-contrib (14+238) ... 2023-11-26T07:16:21.981 INFO:teuthology.orchestra.run.smithi143.stdout:Setting up postgresql (14+238) ... 2023-11-26T07:16:22.080 INFO:teuthology.orchestra.run.smithi188.stdout:Selecting previously unselected package postgresql-client-common. 2023-11-26T07:16:22.095 INFO:teuthology.orchestra.run.smithi188.stdout:Preparing to unpack .../26-postgresql-client-common_238_all.deb ... 2023-11-26T07:16:22.107 INFO:teuthology.orchestra.run.smithi143.stdout:Setting up libqmi-proxy (1.32.0-1ubuntu0.22.04.1) ... 2023-11-26T07:16:22.138 INFO:teuthology.orchestra.run.smithi188.stdout:Unpacking postgresql-client-common (238) ... 2023-11-26T07:16:22.225 INFO:teuthology.orchestra.run.smithi143.stdout:Setting up modemmanager (1.20.0-1~ubuntu22.04.2) ... 2023-11-26T07:16:22.442 INFO:teuthology.orchestra.run.smithi188.stdout:Selecting previously unselected package postgresql-client-14. 2023-11-26T07:16:22.458 INFO:teuthology.orchestra.run.smithi188.stdout:Preparing to unpack .../27-postgresql-client-14_14.9-0ubuntu0.22.04.1_amd64.deb ... 2023-11-26T07:16:22.461 INFO:teuthology.orchestra.run.smithi143.stdout:Created symlink /etc/systemd/system/dbus-org.freedesktop.ModemManager1.service -> /lib/systemd/system/ModemManager.service. 2023-11-26T07:16:22.461 INFO:teuthology.orchestra.run.smithi143.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ModemManager.service -> /lib/systemd/system/ModemManager.service. 2023-11-26T07:16:22.500 INFO:teuthology.orchestra.run.smithi188.stdout:Unpacking postgresql-client-14 (14.9-0ubuntu0.22.04.1) ... 2023-11-26T07:16:23.080 INFO:teuthology.orchestra.run.smithi188.stdout:Selecting previously unselected package postgresql-common. 2023-11-26T07:16:23.095 INFO:teuthology.orchestra.run.smithi188.stdout:Preparing to unpack .../28-postgresql-common_238_all.deb ... 2023-11-26T07:16:23.156 INFO:teuthology.orchestra.run.smithi188.stdout:Adding 'diversion of /usr/bin/pg_config to /usr/bin/pg_config.libpq-dev by postgresql-common' 2023-11-26T07:16:23.187 INFO:teuthology.orchestra.run.smithi188.stdout:Unpacking postgresql-common (238) ... 2023-11-26T07:16:23.202 INFO:teuthology.orchestra.run.smithi143.stdout:Processing triggers for hicolor-icon-theme (0.17-2) ... 2023-11-26T07:16:23.513 INFO:teuthology.orchestra.run.smithi143.stdout:Processing triggers for libc-bin (2.35-0ubuntu3.1) ... 2023-11-26T07:16:23.648 INFO:teuthology.orchestra.run.smithi143.stdout:Processing triggers for man-db (2.10.2-1) ... 2023-11-26T07:16:23.678 INFO:teuthology.orchestra.run.smithi188.stdout:Selecting previously unselected package postgresql-14. 2023-11-26T07:16:23.694 INFO:teuthology.orchestra.run.smithi188.stdout:Preparing to unpack .../29-postgresql-14_14.9-0ubuntu0.22.04.1_amd64.deb ... 2023-11-26T07:16:23.739 INFO:teuthology.orchestra.run.smithi188.stdout:Unpacking postgresql-14 (14.9-0ubuntu0.22.04.1) ... 2023-11-26T07:16:25.108 INFO:teuthology.orchestra.run.smithi188.stdout:Selecting previously unselected package postgresql. 2023-11-26T07:16:25.123 INFO:teuthology.orchestra.run.smithi188.stdout:Preparing to unpack .../30-postgresql_14+238_all.deb ... 2023-11-26T07:16:25.166 INFO:teuthology.orchestra.run.smithi188.stdout:Unpacking postgresql (14+238) ... 2023-11-26T07:16:25.211 INFO:teuthology.orchestra.run.smithi143.stdout:Processing triggers for dbus (1.12.20-2ubuntu4.1) ... 2023-11-26T07:16:25.311 INFO:teuthology.orchestra.run.smithi143.stdout:Processing triggers for install-info (6.8-4build1) ... 2023-11-26T07:16:25.410 INFO:teuthology.orchestra.run.smithi188.stdout:Selecting previously unselected package postgresql-client. 2023-11-26T07:16:25.424 INFO:teuthology.orchestra.run.smithi188.stdout:Preparing to unpack .../31-postgresql-client_14+238_all.deb ... 2023-11-26T07:16:25.459 INFO:teuthology.orchestra.run.smithi188.stdout:Unpacking postgresql-client (14+238) ... 2023-11-26T07:16:25.703 INFO:teuthology.orchestra.run.smithi188.stdout:Selecting previously unselected package postgresql-contrib. 2023-11-26T07:16:25.718 INFO:teuthology.orchestra.run.smithi188.stdout:Preparing to unpack .../32-postgresql-contrib_14+238_all.deb ... 2023-11-26T07:16:25.762 INFO:teuthology.orchestra.run.smithi188.stdout:Unpacking postgresql-contrib (14+238) ... 2023-11-26T07:16:26.001 INFO:teuthology.orchestra.run.smithi143.stderr:W: --force-yes is deprecated, use one of the options starting with --allow instead. 2023-11-26T07:16:26.006 DEBUG:teuthology.parallel:result is None 2023-11-26T07:16:26.081 INFO:teuthology.orchestra.run.smithi188.stdout:Selecting previously unselected package tcl8.6. 2023-11-26T07:16:26.096 INFO:teuthology.orchestra.run.smithi188.stdout:Preparing to unpack .../33-tcl8.6_8.6.12+dfsg-1build1_amd64.deb ... 2023-11-26T07:16:26.162 INFO:teuthology.orchestra.run.smithi188.stdout:Unpacking tcl8.6 (8.6.12+dfsg-1build1) ... 2023-11-26T07:16:26.451 INFO:teuthology.orchestra.run.smithi188.stdout:Selecting previously unselected package tcl. 2023-11-26T07:16:26.466 INFO:teuthology.orchestra.run.smithi188.stdout:Preparing to unpack .../34-tcl_8.6.11+1build2_amd64.deb ... 2023-11-26T07:16:26.505 INFO:teuthology.orchestra.run.smithi188.stdout:Unpacking tcl (8.6.11+1build2) ... 2023-11-26T07:16:26.870 INFO:teuthology.orchestra.run.smithi188.stdout:Selecting previously unselected package wpasupplicant. 2023-11-26T07:16:26.886 INFO:teuthology.orchestra.run.smithi188.stdout:Preparing to unpack .../35-wpasupplicant_2%3a2.10-6ubuntu2_amd64.deb ... 2023-11-26T07:16:26.928 INFO:teuthology.orchestra.run.smithi188.stdout:Unpacking wpasupplicant (2:2.10-6ubuntu2) ... 2023-11-26T07:16:27.258 INFO:teuthology.orchestra.run.smithi188.stdout:Selecting previously unselected package usb-modeswitch-data. 2023-11-26T07:16:27.273 INFO:teuthology.orchestra.run.smithi188.stdout:Preparing to unpack .../36-usb-modeswitch-data_20191128-4_all.deb ... 2023-11-26T07:16:27.341 INFO:teuthology.orchestra.run.smithi188.stdout:Unpacking usb-modeswitch-data (20191128-4) ... 2023-11-26T07:16:27.710 INFO:teuthology.orchestra.run.smithi188.stdout:Selecting previously unselected package usb-modeswitch. 2023-11-26T07:16:27.726 INFO:teuthology.orchestra.run.smithi188.stdout:Preparing to unpack .../37-usb-modeswitch_2.6.1-3ubuntu2_amd64.deb ... 2023-11-26T07:16:27.777 INFO:teuthology.orchestra.run.smithi188.stdout:Unpacking usb-modeswitch (2.6.1-3ubuntu2) ... 2023-11-26T07:16:28.094 INFO:teuthology.orchestra.run.smithi188.stdout:Setting up postgresql-client-common (238) ... 2023-11-26T07:16:28.308 INFO:teuthology.orchestra.run.smithi188.stdout:Setting up flex (2.6.4-8build2) ... 2023-11-26T07:16:28.426 INFO:teuthology.orchestra.run.smithi188.stdout:Setting up tcl8.6 (8.6.12+dfsg-1build1) ... 2023-11-26T07:16:28.568 INFO:teuthology.orchestra.run.smithi188.stdout:Setting up libpq5:amd64 (14.9-0ubuntu0.22.04.1) ... 2023-11-26T07:16:28.686 INFO:teuthology.orchestra.run.smithi188.stdout:Setting up liblzo2-2:amd64 (2.10-2build3) ... 2023-11-26T07:16:28.812 INFO:teuthology.orchestra.run.smithi188.stdout:Setting up libaio-dev:amd64 (0.3.112-13build1) ... 2023-11-26T07:16:28.938 INFO:teuthology.orchestra.run.smithi188.stdout:Setting up postgresql-client-14 (14.9-0ubuntu0.22.04.1) ... 2023-11-26T07:16:29.963 INFO:teuthology.orchestra.run.smithi188.stdout:update-alternatives: using /usr/share/postgresql/14/man/man1/psql.1.gz to provide /usr/share/man/man1/psql.1.gz (psql.1.gz) in auto mode 2023-11-26T07:16:30.051 INFO:teuthology.orchestra.run.smithi188.stdout:Setting up wpasupplicant (2:2.10-6ubuntu2) ... 2023-11-26T07:16:30.472 INFO:teuthology.orchestra.run.smithi188.stdout:Created symlink /etc/systemd/system/dbus-fi.w1.wpa_supplicant1.service -> /lib/systemd/system/wpa_supplicant.service. 2023-11-26T07:16:30.472 INFO:teuthology.orchestra.run.smithi188.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/wpa_supplicant.service -> /lib/systemd/system/wpa_supplicant.service. 2023-11-26T07:16:31.163 INFO:teuthology.orchestra.run.smithi188.stdout:Setting up libteamdctl0:amd64 (1.31-1build2) ... 2023-11-26T07:16:31.289 INFO:teuthology.orchestra.run.smithi188.stdout:Setting up libssl-dev:amd64 (3.0.2-0ubuntu1.12) ... 2023-11-26T07:16:31.440 INFO:teuthology.orchestra.run.smithi188.stdout:Setting up usb-modeswitch-data (20191128-4) ... 2023-11-26T07:16:31.566 INFO:teuthology.orchestra.run.smithi188.stdout:Setting up libelf-dev:amd64 (0.186-1build1) ... 2023-11-26T07:16:31.692 INFO:teuthology.orchestra.run.smithi188.stdout:Setting up libfl2:amd64 (2.6.4-8build2) ... 2023-11-26T07:16:31.817 INFO:teuthology.orchestra.run.smithi188.stdout:Setting up libnm0:amd64 (1.36.6-0ubuntu2) ... 2023-11-26T07:16:31.994 INFO:teuthology.orchestra.run.smithi188.stdout:Setting up ppp (2.4.9-1+1ubuntu3) ... 2023-11-26T07:16:32.742 INFO:teuthology.orchestra.run.smithi188.stdout:Setting up libllvm14:amd64 (1:14.0.0-1ubuntu1.1) ... 2023-11-26T07:16:32.868 INFO:teuthology.orchestra.run.smithi188.stdout:Setting up dump (0.4b47-3) ... 2023-11-26T07:16:32.963 INFO:teuthology.orchestra.run.smithi188.stdout:update-alternatives: using /usr/sbin/rmt-dump to provide /usr/sbin/rmt (rmt) in auto mode 2023-11-26T07:16:33.028 INFO:teuthology.orchestra.run.smithi188.stdout:Setting up libmm-glib0:amd64 (1.20.0-1~ubuntu22.04.2) ... 2023-11-26T07:16:33.187 INFO:teuthology.orchestra.run.smithi188.stdout:Setting up libbluetooth3:amd64 (5.64-0ubuntu1) ... 2023-11-26T07:16:33.305 INFO:teuthology.orchestra.run.smithi188.stdout:Setting up tcl (8.6.11+1build2) ... 2023-11-26T07:16:33.406 INFO:teuthology.orchestra.run.smithi188.stdout:Setting up libjson-perl (4.04000-1) ... 2023-11-26T07:16:33.524 INFO:teuthology.orchestra.run.smithi188.stdout:Setting up bison (2:3.8.2+dfsg-1build1) ... 2023-11-26T07:16:33.614 INFO:teuthology.orchestra.run.smithi188.stdout:update-alternatives: using /usr/bin/bison.yacc to provide /usr/bin/yacc (yacc) in auto mode 2023-11-26T07:16:33.709 INFO:teuthology.orchestra.run.smithi188.stdout:Setting up indent (2.2.12-1ubuntu0.22.04.1) ... 2023-11-26T07:16:33.835 INFO:teuthology.orchestra.run.smithi188.stdout:Setting up usb-modeswitch (2.6.1-3ubuntu2) ... 2023-11-26T07:16:34.012 INFO:teuthology.orchestra.run.smithi188.stdout:Setting up libndp0:amd64 (1.8-0ubuntu3) ... 2023-11-26T07:16:34.138 INFO:teuthology.orchestra.run.smithi188.stdout:Setting up libmbim-glib4:amd64 (1.28.0-1~ubuntu20.04.1) ... 2023-11-26T07:16:34.288 INFO:teuthology.orchestra.run.smithi188.stdout:Setting up pptp-linux (1.10.0-1build3) ... 2023-11-26T07:16:34.432 INFO:teuthology.orchestra.run.smithi188.stdout:Setting up postgresql-client (14+238) ... 2023-11-26T07:16:34.549 INFO:teuthology.orchestra.run.smithi188.stdout:Setting up network-manager (1.36.6-0ubuntu2) ... 2023-11-26T07:16:35.134 INFO:teuthology.orchestra.run.smithi188.stdout: 2023-11-26T07:16:35.134 INFO:teuthology.orchestra.run.smithi188.stdout:The following network interfaces were found in /etc/network/interfaces 2023-11-26T07:16:35.134 INFO:teuthology.orchestra.run.smithi188.stdout:which means they are currently configured by ifupdown: 2023-11-26T07:16:35.134 INFO:teuthology.orchestra.run.smithi188.stdout:- enp3s0f1 2023-11-26T07:16:35.134 INFO:teuthology.orchestra.run.smithi188.stdout:If you want to manage those interfaces with NetworkManager instead 2023-11-26T07:16:35.134 INFO:teuthology.orchestra.run.smithi188.stdout:remove their configuration from /etc/network/interfaces. 2023-11-26T07:16:35.134 INFO:teuthology.orchestra.run.smithi188.stdout: 2023-11-26T07:16:35.266 INFO:teuthology.orchestra.run.smithi188.stdout:Created symlink /etc/systemd/system/dbus-org.freedesktop.nm-dispatcher.service -> /lib/systemd/system/NetworkManager-dispatcher.service. 2023-11-26T07:16:35.680 INFO:teuthology.orchestra.run.smithi188.stdout:Created symlink /etc/systemd/system/network-online.target.wants/NetworkManager-wait-online.service -> /lib/systemd/system/NetworkManager-wait-online.service. 2023-11-26T07:16:36.096 INFO:teuthology.orchestra.run.smithi188.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/NetworkManager.service -> /lib/systemd/system/NetworkManager.service. 2023-11-26T07:16:36.810 INFO:teuthology.orchestra.run.smithi188.stdout:Setting up libmbim-proxy (1.28.0-1~ubuntu20.04.1) ... 2023-11-26T07:16:36.962 INFO:teuthology.orchestra.run.smithi188.stdout:Setting up libfl-dev:amd64 (2.6.4-8build2) ... 2023-11-26T07:16:37.109 INFO:teuthology.orchestra.run.smithi188.stdout:Setting up network-manager-pptp (1.2.10-1) ... 2023-11-26T07:16:37.256 INFO:teuthology.orchestra.run.smithi188.stdout:Setting up postgresql-common (238) ... 2023-11-26T07:16:38.190 INFO:teuthology.orchestra.run.smithi188.stdout:Adding user postgres to group ssl-cert 2023-11-26T07:16:38.489 INFO:teuthology.orchestra.run.smithi188.stdout: 2023-11-26T07:16:38.490 INFO:teuthology.orchestra.run.smithi188.stdout:Creating config file /etc/postgresql-common/createcluster.conf with new version 2023-11-26T07:16:38.754 INFO:teuthology.orchestra.run.smithi188.stdout:Building PostgreSQL dictionaries from installed myspell/hunspell packages... 2023-11-26T07:16:38.754 INFO:teuthology.orchestra.run.smithi188.stdout:Removing obsolete dictionary files: 2023-11-26T07:16:39.493 INFO:teuthology.orchestra.run.smithi188.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/postgresql.service -> /lib/systemd/system/postgresql.service. 2023-11-26T07:16:40.108 INFO:teuthology.orchestra.run.smithi188.stdout:Setting up postgresql-14 (14.9-0ubuntu0.22.04.1) ... 2023-11-26T07:16:40.436 INFO:teuthology.orchestra.run.smithi188.stdout:Creating new PostgreSQL cluster 14/main ... 2023-11-26T07:16:40.458 INFO:teuthology.orchestra.run.smithi188.stdout:/usr/lib/postgresql/14/bin/initdb -D /var/lib/postgresql/14/main --auth-local peer --auth-host scram-sha-256 --no-instructions 2023-11-26T07:16:40.506 INFO:teuthology.orchestra.run.smithi188.stdout:The files belonging to this database system will be owned by user "postgres". 2023-11-26T07:16:40.506 INFO:teuthology.orchestra.run.smithi188.stdout:This user must also own the server process. 2023-11-26T07:16:40.507 INFO:teuthology.orchestra.run.smithi188.stdout: 2023-11-26T07:16:40.507 INFO:teuthology.orchestra.run.smithi188.stdout:The database cluster will be initialized with locale "en_US". 2023-11-26T07:16:40.507 INFO:teuthology.orchestra.run.smithi188.stdout:The default database encoding has accordingly been set to "LATIN1". 2023-11-26T07:16:40.507 INFO:teuthology.orchestra.run.smithi188.stdout:The default text search configuration will be set to "english". 2023-11-26T07:16:40.507 INFO:teuthology.orchestra.run.smithi188.stdout: 2023-11-26T07:16:40.507 INFO:teuthology.orchestra.run.smithi188.stdout:Data page checksums are disabled. 2023-11-26T07:16:40.507 INFO:teuthology.orchestra.run.smithi188.stdout: 2023-11-26T07:16:40.507 INFO:teuthology.orchestra.run.smithi188.stdout:fixing permissions on existing directory /var/lib/postgresql/14/main ... ok 2023-11-26T07:16:40.508 INFO:teuthology.orchestra.run.smithi188.stdout:creating subdirectories ... ok 2023-11-26T07:16:40.508 INFO:teuthology.orchestra.run.smithi188.stdout:selecting dynamic shared memory implementation ... posix 2023-11-26T07:16:40.532 INFO:teuthology.orchestra.run.smithi188.stdout:selecting default max_connections ... 100 2023-11-26T07:16:40.567 INFO:teuthology.orchestra.run.smithi188.stdout:selecting default shared_buffers ... 128MB 2023-11-26T07:16:40.581 INFO:teuthology.orchestra.run.smithi188.stdout:selecting default time zone ... Etc/UTC 2023-11-26T07:16:40.582 INFO:teuthology.orchestra.run.smithi188.stdout:creating configuration files ... ok 2023-11-26T07:16:40.750 INFO:teuthology.orchestra.run.smithi188.stdout:running bootstrap script ... ok 2023-11-26T07:16:41.370 INFO:teuthology.orchestra.run.smithi188.stdout:performing post-bootstrap initialization ... ok 2023-11-26T07:16:41.917 INFO:teuthology.orchestra.run.smithi188.stdout:syncing data to disk ... ok 2023-11-26T07:16:42.445 INFO:teuthology.orchestra.run.smithi188.stdout:update-alternatives: using /usr/share/postgresql/14/man/man1/postmaster.1.gz to provide /usr/share/man/man1/postmaster.1.gz (postmaster.1.gz) in auto mode 2023-11-26T07:16:45.757 INFO:teuthology.orchestra.run.smithi188.stdout:Setting up libqmi-glib5:amd64 (1.32.0-1ubuntu0.22.04.1) ... 2023-11-26T07:16:45.916 INFO:teuthology.orchestra.run.smithi188.stdout:Setting up postgresql-contrib (14+238) ... 2023-11-26T07:16:46.042 INFO:teuthology.orchestra.run.smithi188.stdout:Setting up postgresql (14+238) ... 2023-11-26T07:16:46.168 INFO:teuthology.orchestra.run.smithi188.stdout:Setting up libqmi-proxy (1.32.0-1ubuntu0.22.04.1) ... 2023-11-26T07:16:46.294 INFO:teuthology.orchestra.run.smithi188.stdout:Setting up modemmanager (1.20.0-1~ubuntu22.04.2) ... 2023-11-26T07:16:46.581 INFO:teuthology.orchestra.run.smithi188.stdout:Created symlink /etc/systemd/system/dbus-org.freedesktop.ModemManager1.service -> /lib/systemd/system/ModemManager.service. 2023-11-26T07:16:46.581 INFO:teuthology.orchestra.run.smithi188.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ModemManager.service -> /lib/systemd/system/ModemManager.service. 2023-11-26T07:16:47.338 INFO:teuthology.orchestra.run.smithi188.stdout:Processing triggers for hicolor-icon-theme (0.17-2) ... 2023-11-26T07:16:47.640 INFO:teuthology.orchestra.run.smithi188.stdout:Processing triggers for libc-bin (2.35-0ubuntu3.1) ... 2023-11-26T07:16:47.792 INFO:teuthology.orchestra.run.smithi188.stdout:Processing triggers for man-db (2.10.2-1) ... 2023-11-26T07:16:49.162 INFO:teuthology.orchestra.run.smithi188.stdout:Processing triggers for dbus (1.12.20-2ubuntu4.1) ... 2023-11-26T07:16:49.279 INFO:teuthology.orchestra.run.smithi188.stdout:Processing triggers for install-info (6.8-4build1) ... 2023-11-26T07:16:49.946 INFO:teuthology.orchestra.run.smithi188.stderr:W: --force-yes is deprecated, use one of the options starting with --allow instead. 2023-11-26T07:16:49.951 DEBUG:teuthology.parallel:result is None 2023-11-26T07:16:49.951 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=ubuntu%2F22.04%2Fx86_64&sha1=44c2237fe77713214148c1fd622c872f346f1b99 2023-11-26T07:16:50.214 DEBUG:teuthology.orchestra.run.smithi143:> dpkg-query -W -f '${Version}' ceph 2023-11-26T07:16:50.237 INFO:teuthology.orchestra.run.smithi143.stdout:18.0.0-7484-g44c2237f-1jammy 2023-11-26T07:16:50.238 INFO:teuthology.packaging:The installed version of ceph is 18.0.0-7484-g44c2237f-1jammy 2023-11-26T07:16:50.238 INFO:teuthology.task.install:The correct ceph version 18.0.0-7484-g44c2237f-1jammy is installed. 2023-11-26T07:16:50.238 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=ubuntu%2F22.04%2Fx86_64&sha1=44c2237fe77713214148c1fd622c872f346f1b99 2023-11-26T07:16:50.371 DEBUG:teuthology.orchestra.run.smithi188:> dpkg-query -W -f '${Version}' ceph 2023-11-26T07:16:50.393 INFO:teuthology.orchestra.run.smithi188.stdout:18.0.0-7484-g44c2237f-1jammy 2023-11-26T07:16:50.393 INFO:teuthology.packaging:The installed version of ceph is 18.0.0-7484-g44c2237f-1jammy 2023-11-26T07:16:50.393 INFO:teuthology.task.install:The correct ceph version 18.0.0-7484-g44c2237f-1jammy is installed. 2023-11-26T07:16:50.393 INFO:teuthology.task.install.util:Shipping valgrind.supp... 2023-11-26T07:16:50.393 DEBUG:teuthology.orchestra.run.smithi143:> set -ex 2023-11-26T07:16:50.393 DEBUG:teuthology.orchestra.run.smithi143:> sudo dd of=/home/ubuntu/cephtest/valgrind.supp 2023-11-26T07:16:50.416 DEBUG:teuthology.orchestra.run.smithi188:> set -ex 2023-11-26T07:16:50.416 DEBUG:teuthology.orchestra.run.smithi188:> sudo dd of=/home/ubuntu/cephtest/valgrind.supp 2023-11-26T07:16:50.459 INFO:teuthology.task.install.util:Shipping 'daemon-helper'... 2023-11-26T07:16:50.459 DEBUG:teuthology.orchestra.run.smithi143:> set -ex 2023-11-26T07:16:50.460 DEBUG:teuthology.orchestra.run.smithi143:> sudo dd of=/usr/bin/daemon-helper 2023-11-26T07:16:50.482 DEBUG:teuthology.orchestra.run.smithi143:> sudo chmod a=rx -- /usr/bin/daemon-helper 2023-11-26T07:16:50.546 DEBUG:teuthology.orchestra.run.smithi188:> set -ex 2023-11-26T07:16:50.546 DEBUG:teuthology.orchestra.run.smithi188:> sudo dd of=/usr/bin/daemon-helper 2023-11-26T07:16:50.567 DEBUG:teuthology.orchestra.run.smithi188:> sudo chmod a=rx -- /usr/bin/daemon-helper 2023-11-26T07:16:50.630 INFO:teuthology.task.install.util:Shipping 'adjust-ulimits'... 2023-11-26T07:16:50.630 DEBUG:teuthology.orchestra.run.smithi143:> set -ex 2023-11-26T07:16:50.631 DEBUG:teuthology.orchestra.run.smithi143:> sudo dd of=/usr/bin/adjust-ulimits 2023-11-26T07:16:50.653 DEBUG:teuthology.orchestra.run.smithi143:> sudo chmod a=rx -- /usr/bin/adjust-ulimits 2023-11-26T07:16:50.717 DEBUG:teuthology.orchestra.run.smithi188:> set -ex 2023-11-26T07:16:50.717 DEBUG:teuthology.orchestra.run.smithi188:> sudo dd of=/usr/bin/adjust-ulimits 2023-11-26T07:16:50.738 DEBUG:teuthology.orchestra.run.smithi188:> sudo chmod a=rx -- /usr/bin/adjust-ulimits 2023-11-26T07:16:50.798 INFO:teuthology.task.install.util:Shipping 'stdin-killer'... 2023-11-26T07:16:50.798 DEBUG:teuthology.orchestra.run.smithi143:> set -ex 2023-11-26T07:16:50.798 DEBUG:teuthology.orchestra.run.smithi143:> sudo dd of=/usr/bin/stdin-killer 2023-11-26T07:16:50.819 DEBUG:teuthology.orchestra.run.smithi143:> sudo chmod a=rx -- /usr/bin/stdin-killer 2023-11-26T07:16:50.881 DEBUG:teuthology.orchestra.run.smithi188:> set -ex 2023-11-26T07:16:50.881 DEBUG:teuthology.orchestra.run.smithi188:> sudo dd of=/usr/bin/stdin-killer 2023-11-26T07:16:50.902 DEBUG:teuthology.orchestra.run.smithi188:> sudo chmod a=rx -- /usr/bin/stdin-killer 2023-11-26T07:16:50.962 INFO:teuthology.run_tasks:Running task ceph... 2023-11-26T07:16:51.025 INFO:tasks.ceph:Making ceph log dir writeable by non-root... 2023-11-26T07:16:51.025 DEBUG:teuthology.orchestra.run.smithi143:> sudo chmod 777 /var/log/ceph 2023-11-26T07:16:51.029 DEBUG:teuthology.orchestra.run.smithi188:> sudo chmod 777 /var/log/ceph 2023-11-26T07:16:51.049 INFO:tasks.ceph:Disabling ceph logrotate... 2023-11-26T07:16:51.049 DEBUG:teuthology.orchestra.run.smithi143:> sudo rm -f -- /etc/logrotate.d/ceph 2023-11-26T07:16:51.096 DEBUG:teuthology.orchestra.run.smithi188:> sudo rm -f -- /etc/logrotate.d/ceph 2023-11-26T07:16:51.116 INFO:tasks.ceph:Creating extra log directories... 2023-11-26T07:16:51.116 DEBUG:teuthology.orchestra.run.smithi143:> sudo install -d -m0777 -- /var/log/ceph/valgrind /var/log/ceph/profiling-logger 2023-11-26T07:16:51.156 DEBUG:teuthology.orchestra.run.smithi188:> sudo install -d -m0777 -- /var/log/ceph/valgrind /var/log/ceph/profiling-logger 2023-11-26T07:16:51.189 INFO:tasks.ceph:Setting up log rotation with {'ceph-mds': '10G', 'ceph-osd': '10G'} 2023-11-26T07:16:51.189 INFO:tasks.ceph:writing logrotate stanza for ceph-mds 2023-11-26T07:16:51.189 INFO:tasks.ceph:writing logrotate stanza for ceph-osd 2023-11-26T07:16:51.189 DEBUG:teuthology.orchestra.run.smithi143:> set -ex 2023-11-26T07:16:51.189 DEBUG:teuthology.orchestra.run.smithi143:> dd of=/home/ubuntu/cephtest/logrotate.ceph-test.conf 2023-11-26T07:16:51.229 DEBUG:teuthology.orchestra.run.smithi143:> sudo mv /home/ubuntu/cephtest/logrotate.ceph-test.conf /etc/logrotate.d/ceph-test.conf && sudo chmod 0644 /etc/logrotate.d/ceph-test.conf && sudo chown root.root /etc/logrotate.d/ceph-test.conf 2023-11-26T07:16:51.324 DEBUG:teuthology.orchestra.run.smithi188:> set -ex 2023-11-26T07:16:51.324 DEBUG:teuthology.orchestra.run.smithi188:> dd of=/home/ubuntu/cephtest/logrotate.ceph-test.conf 2023-11-26T07:16:51.333 DEBUG:teuthology.orchestra.run.smithi188:> sudo mv /home/ubuntu/cephtest/logrotate.ceph-test.conf /etc/logrotate.d/ceph-test.conf && sudo chmod 0644 /etc/logrotate.d/ceph-test.conf && sudo chown root.root /etc/logrotate.d/ceph-test.conf 2023-11-26T07:16:51.428 INFO:tasks.ceph:Creating ceph cluster ceph... 2023-11-26T07:16:51.428 INFO:tasks.ceph:config {'conf': {'client': {'client mount timeout': 600, 'debug client': 20, 'debug ms': 1, 'pid file': '', 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'mds': {'debug mds': 20, 'debug mds balancer': 20, 'debug ms': 1, 'mds debug frag': True, 'mds debug scatterstat': True, 'mds op complaint time': 180, 'mds verify scatter': True, 'osd op complaint time': 180, 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'mgr': {'debug client': 20, 'debug finisher': 20, 'debug mgr': 20, 'debug ms': 1}, 'mon': {'debug mon': 20, 'debug ms': 1, 'debug paxos': 20, 'mon op complaint time': 120}, 'osd': {'bdev async discard': True, 'bdev enable discard': True, 'bluestore allocator': 'bitmap', 'bluestore block size': 96636764160, 'bluestore fsck on mount': True, 'debug bluefs': '1/20', 'debug bluestore': '1/20', 'debug ms': 1, 'debug osd': 20, 'debug rocksdb': '4/10', 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd failsafe full ratio': 0.95, 'osd objectstore': 'bluestore', 'osd op complaint time': 180}}, 'fs': 'xfs', 'mkfs_options': None, 'mount_options': None, 'skip_mgr_daemons': False, 'log_ignorelist': ['\\(MDS_ALL_DOWN\\)', '\\(MDS_UP_LESS_THAN_MAX\\)', 'overall HEALTH_', '\\(FS_DEGRADED\\)', '\\(MDS_FAILED\\)', '\\(MDS_DEGRADED\\)', '\\(FS_WITH_FAILED_MDS\\)', '\\(MDS_DAMAGE\\)', '\\(MDS_ALL_DOWN\\)', '\\(MDS_UP_LESS_THAN_MAX\\)', '\\(FS_INLINE_DATA_DEPRECATED\\)', '\\(POOL_APP_NOT_ENABLED\\)', 'overall HEALTH_', '\\(OSD_DOWN\\)', '\\(OSD_', 'but it is still running', 'is not responding', 'OSD full dropping all updates', 'OSD near full', 'pausewr flag', 'failsafe engaged, dropping updates', 'failsafe disengaged, no longer dropping', 'is full \\(reached quota', 'POOL_FULL', 'POOL_BACKFILLFULL'], 'cpu_profile': set(), 'cluster': 'ceph', 'mon_bind_msgr2': True, 'mon_bind_addrvec': True} 2023-11-26T07:16:51.429 INFO:tasks.ceph:ctx.config {'archive_path': '/home/teuthworker/archive/rishabh-2023-11-26_06:51:12-fs:volumes-rishabh-mgr-vol-misc-testing-default-smithi/7467343', 'branch': 'rishabh-mgr-vol-misc', 'description': 'fs:volumes/{begin/{0-install 1-ceph 2-logrotate} clusters/1a3s-mds-4c-client conf/{client mds mon osd} distro/{ubuntu_latest} mount/fuse objectstore/bluestore-bitmap overrides/{ignorelist_health ignorelist_wrongly_marked_down no_client_pidfile} tasks/volumes/{overrides test/misc}}', 'email': None, 'first_in_suite': False, 'job_id': '7467343', 'kernel': {'flavor': 'default', 'kdb': True, 'sha1': '8247bbb5b470f7e2a1f3eefcc16e24cf1338f606'}, 'ktype': 'distro', 'last_in_suite': False, 'log-rotate': {'ceph-mds': '10G', 'ceph-osd': '10G'}, 'machine_type': 'smithi', 'name': 'rishabh-2023-11-26_06:51:12-fs:volumes-rishabh-mgr-vol-misc-testing-default-smithi', 'no_nested_subset': False, 'nuke-on-error': True, 'openstack': [{'volumes': {'count': 4, 'size': 20}}, {'machine': {'disk': 200}}], 'os_type': 'ubuntu', 'os_version': '22.04', 'overrides': {'admin_socket': {'branch': 'rishabh-mgr-vol-misc'}, 'ceph': {'conf': {'client': {'client mount timeout': 600, 'debug client': 20, 'debug ms': 1, 'pid file': '', 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'mds': {'debug mds': 20, 'debug mds balancer': 20, 'debug ms': 1, 'mds debug frag': True, 'mds debug scatterstat': True, 'mds op complaint time': 180, 'mds verify scatter': True, 'osd op complaint time': 180, 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'mgr': {'debug client': 20, 'debug finisher': 20, 'debug mgr': 20, 'debug ms': 1}, 'mon': {'debug mon': 20, 'debug ms': 1, 'debug paxos': 20, 'mon op complaint time': 120}, 'osd': {'bdev async discard': True, 'bdev enable discard': True, 'bluestore allocator': 'bitmap', 'bluestore block size': 96636764160, 'bluestore fsck on mount': True, 'debug bluefs': '1/20', 'debug bluestore': '1/20', 'debug ms': 1, 'debug osd': 20, 'debug rocksdb': '4/10', 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd failsafe full ratio': 0.95, 'osd objectstore': 'bluestore', 'osd op complaint time': 180}}, 'flavor': 'default', 'fs': 'xfs', 'log-ignorelist': ['\\(MDS_ALL_DOWN\\)', '\\(MDS_UP_LESS_THAN_MAX\\)', 'overall HEALTH_', '\\(FS_DEGRADED\\)', '\\(MDS_FAILED\\)', '\\(MDS_DEGRADED\\)', '\\(FS_WITH_FAILED_MDS\\)', '\\(MDS_DAMAGE\\)', '\\(MDS_ALL_DOWN\\)', '\\(MDS_UP_LESS_THAN_MAX\\)', '\\(FS_INLINE_DATA_DEPRECATED\\)', '\\(POOL_APP_NOT_ENABLED\\)', 'overall HEALTH_', '\\(OSD_DOWN\\)', '\\(OSD_', 'but it is still running', 'is not responding', 'OSD full dropping all updates', 'OSD near full', 'pausewr flag', 'failsafe engaged, dropping updates', 'failsafe disengaged, no longer dropping', 'is full \\(reached quota', 'POOL_FULL', 'POOL_BACKFILLFULL'], 'sha1': '44c2237fe77713214148c1fd622c872f346f1b99'}, 'ceph-deploy': {'bluestore': True, 'conf': {'client': {'log file': '/var/log/ceph/ceph-$name.$pid.log'}, 'mon': {}, 'osd': {'bdev async discard': True, 'bdev enable discard': True, 'bluestore block size': 96636764160, 'bluestore fsck on mount': True, 'debug bluefs': '1/20', 'debug bluestore': '1/20', 'debug rocksdb': '4/10', 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd failsafe full ratio': 0.95, 'osd objectstore': 'bluestore'}}, 'fs': 'xfs'}, 'install': {'ceph': {'flavor': 'default', 'sha1': '44c2237fe77713214148c1fd622c872f346f1b99'}}, 'thrashosds': {'bdev_inject_crash': 2, 'bdev_inject_crash_probability': 0.5}, 'workunit': {'branch': 'rishabh-mgr-vol-misc', 'sha1': '44c2237fe77713214148c1fd622c872f346f1b99'}}, 'owner': 'scheduled_rishabh@teuthology', 'priority': 50, 'repo': 'https://git.ceph.com/ceph-ci.git', 'roles': [['mon.a', 'mgr.y', 'mds.a', 'mds.c', 'osd.0', 'osd.1', 'osd.2', 'osd.3', 'client.0', 'client.2'], ['mon.b', 'mon.c', 'mgr.x', 'mds.b', 'mds.d', 'osd.4', 'osd.5', 'osd.6', 'osd.7', 'client.1', 'client.3']], 'seed': 1863, 'sha1': '44c2237fe77713214148c1fd622c872f346f1b99', 'sleep_before_teardown': 0, 'suite': 'fs:volumes', 'suite_branch': 'rishabh-mgr-vol-misc', 'suite_path': '/home/teuthworker/src/git.ceph.com_ceph-c_44c2237fe77713214148c1fd622c872f346f1b99/qa', 'suite_relpath': 'qa', 'suite_repo': 'https://git.ceph.com/ceph-ci.git', 'suite_sha1': '44c2237fe77713214148c1fd622c872f346f1b99', 'syslog': {'ignorelist': ['WARNING*.*check_session_state', 'WARNING*.*__ceph_remove_cap']}, 'targets': {'smithi143.front.sepia.ceph.com': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBLMlsYrpI20D7nmuV4Ss3kS8JeSY9qF58zM5ezfrs6+JcgHI73OPCbgwt0ossMtulfaME6pGevN9hNlafUPf9wQ=', 'smithi188.front.sepia.ceph.com': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBJdbw4I9Ou0oA1kXwuSM4nPA+qQBLn4/ORmmu+HODcBiXZaQj7dzvrfCaWf6g8HzUkhcLo6Ef+7Yycrf+9QMqsM='}, 'tasks': [{'internal.check_packages': None}, {'internal.buildpackages_prep': None}, {'internal.save_config': None}, {'internal.check_lock': None}, {'internal.add_remotes': None}, {'console_log': None}, {'internal.connect': None}, {'internal.push_inventory': None}, {'internal.serialize_remote_roles': None}, {'internal.check_conflict': None}, {'internal.check_ceph_data': None}, {'internal.vm_setup': None}, {'kernel': {'flavor': 'default', 'kdb': True, 'sha1': '8247bbb5b470f7e2a1f3eefcc16e24cf1338f606'}}, {'internal.base': None}, {'internal.archive_upload': None}, {'internal.archive': None}, {'internal.coredump': None}, {'internal.sudo': None}, {'internal.syslog': None}, {'internal.timer': None}, {'pcp': None}, {'selinux': None}, {'ansible.cephlab': None}, {'clock': None}, {'install': {'extra_packages': {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-top', 'cephfs-mirror']}, 'extra_system_packages': {'deb': ['bison', 'flex', 'libelf-dev', 'libssl-dev', 'network-manager', 'iproute2', 'util-linux', 'dump', 'indent', 'libaio-dev', 'libtool-bin', 'uuid-dev', 'xfslibs-dev', 'postgresql', 'postgresql-client', 'postgresql-common', 'postgresql-contrib'], 'rpm': ['bison', 'flex', 'elfutils-libelf-devel', 'openssl-devel', 'NetworkManager', 'iproute', 'util-linux', 'libacl-devel', 'libaio-devel', 'libattr-devel', 'libtool', 'libuuid-devel', 'xfsdump', 'xfsprogs', 'xfsprogs-devel', 'libaio-devel', 'libtool', 'libuuid-devel', 'xfsprogs-devel', 'postgresql', 'postgresql-server', 'postgresql-contrib']}, 'flavor': 'default', 'sha1': '44c2237fe77713214148c1fd622c872f346f1b99'}}, {'ceph': {'create_rbd_pool': False, 'conf': {'client': {'client mount timeout': 600, 'debug client': 20, 'debug ms': 1, 'pid file': '', 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'mds': {'debug mds': 20, 'debug mds balancer': 20, 'debug ms': 1, 'mds debug frag': True, 'mds debug scatterstat': True, 'mds op complaint time': 180, 'mds verify scatter': True, 'osd op complaint time': 180, 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'mgr': {'debug client': 20, 'debug finisher': 20, 'debug mgr': 20, 'debug ms': 1}, 'mon': {'debug mon': 20, 'debug ms': 1, 'debug paxos': 20, 'mon op complaint time': 120}, 'osd': {'bdev async discard': True, 'bdev enable discard': True, 'bluestore allocator': 'bitmap', 'bluestore block size': 96636764160, 'bluestore fsck on mount': True, 'debug bluefs': '1/20', 'debug bluestore': '1/20', 'debug ms': 1, 'debug osd': 20, 'debug rocksdb': '4/10', 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd failsafe full ratio': 0.95, 'osd objectstore': 'bluestore', 'osd op complaint time': 180}}, 'flavor': 'default', 'fs': 'xfs', 'log-ignorelist': ['\\(MDS_ALL_DOWN\\)', '\\(MDS_UP_LESS_THAN_MAX\\)', 'overall HEALTH_', '\\(FS_DEGRADED\\)', '\\(MDS_FAILED\\)', '\\(MDS_DEGRADED\\)', '\\(FS_WITH_FAILED_MDS\\)', '\\(MDS_DAMAGE\\)', '\\(MDS_ALL_DOWN\\)', '\\(MDS_UP_LESS_THAN_MAX\\)', '\\(FS_INLINE_DATA_DEPRECATED\\)', '\\(POOL_APP_NOT_ENABLED\\)', 'overall HEALTH_', '\\(OSD_DOWN\\)', '\\(OSD_', 'but it is still running', 'is not responding', 'OSD full dropping all updates', 'OSD near full', 'pausewr flag', 'failsafe engaged, dropping updates', 'failsafe disengaged, no longer dropping', 'is full \\(reached quota', 'POOL_FULL', 'POOL_BACKFILLFULL'], 'sha1': '44c2237fe77713214148c1fd622c872f346f1b99', 'cluster': 'ceph'}}, {'ceph-fuse': None}, {'cephfs_test_runner': {'fail_on_skip': False, 'modules': ['tasks.cephfs.test_volumes.TestMisc']}}], 'teuthology': {'fragments_dropped': [], 'meta': {}, 'postmerge': ['local function is_kupstream() return false end', 'local function is_kdistro() return false end', 'local function is_fuse() return true end', "local function syntax_version() return '' end"]}, 'teuthology_branch': 'main', 'teuthology_sha1': '2442239d2653456406c25ae0c71b689c8f2657b6', 'timestamp': '2023-11-26_06:51:12', 'tube': 'smithi', 'user': 'rishabh', 'verbose': False, 'worker_log': '/home/teuthworker/archive/worker_logs/dispatcher.smithi.232157'} 2023-11-26T07:16:51.429 DEBUG:teuthology.orchestra.run.smithi143:> install -d -m0755 -- /home/ubuntu/cephtest/ceph.data 2023-11-26T07:16:51.433 DEBUG:teuthology.orchestra.run.smithi188:> install -d -m0755 -- /home/ubuntu/cephtest/ceph.data 2023-11-26T07:16:51.442 DEBUG:teuthology.orchestra.run.smithi143:> sudo install -d -m0777 -- /var/run/ceph 2023-11-26T07:16:51.488 DEBUG:teuthology.orchestra.run.smithi188:> sudo install -d -m0777 -- /var/run/ceph 2023-11-26T07:16:51.509 DEBUG:teuthology.orchestra.run.smithi143:> set -ex 2023-11-26T07:16:51.509 DEBUG:teuthology.orchestra.run.smithi143:> dd if=/scratch_devs of=/dev/stdout 2023-11-26T07:16:51.553 DEBUG:teuthology.misc:devs=['/dev/vg_nvme/lv_1', '/dev/vg_nvme/lv_2', '/dev/vg_nvme/lv_3', '/dev/vg_nvme/lv_4'] 2023-11-26T07:16:51.553 DEBUG:teuthology.orchestra.run.smithi143:> stat /dev/vg_nvme/lv_1 2023-11-26T07:16:51.605 INFO:teuthology.orchestra.run.smithi143.stdout: File: /dev/vg_nvme/lv_1 -> ../dm-0 2023-11-26T07:16:51.606 INFO:teuthology.orchestra.run.smithi143.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2023-11-26T07:16:51.606 INFO:teuthology.orchestra.run.smithi143.stdout:Device: 5h/5d Inode: 897 Links: 1 2023-11-26T07:16:51.606 INFO:teuthology.orchestra.run.smithi143.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2023-11-26T07:16:51.606 INFO:teuthology.orchestra.run.smithi143.stdout:Access: 2023-11-26 07:09:42.228799807 +0000 2023-11-26T07:16:51.606 INFO:teuthology.orchestra.run.smithi143.stdout:Modify: 2023-11-26 07:09:41.820803451 +0000 2023-11-26T07:16:51.606 INFO:teuthology.orchestra.run.smithi143.stdout:Change: 2023-11-26 07:09:41.820803451 +0000 2023-11-26T07:16:51.606 INFO:teuthology.orchestra.run.smithi143.stdout: Birth: 2023-11-26 07:09:41.820803451 +0000 2023-11-26T07:16:51.606 DEBUG:teuthology.orchestra.run.smithi143:> sudo dd if=/dev/vg_nvme/lv_1 of=/dev/null count=1 2023-11-26T07:16:51.666 INFO:teuthology.orchestra.run.smithi143.stderr:1+0 records in 2023-11-26T07:16:51.666 INFO:teuthology.orchestra.run.smithi143.stderr:1+0 records out 2023-11-26T07:16:51.666 INFO:teuthology.orchestra.run.smithi143.stderr:512 bytes copied, 0.000349403 s, 1.5 MB/s 2023-11-26T07:16:51.668 DEBUG:teuthology.orchestra.run.smithi143:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_1 2023-11-26T07:16:51.722 DEBUG:teuthology.orchestra.run.smithi143:> stat /dev/vg_nvme/lv_2 2023-11-26T07:16:51.773 INFO:teuthology.orchestra.run.smithi143.stdout: File: /dev/vg_nvme/lv_2 -> ../dm-1 2023-11-26T07:16:51.773 INFO:teuthology.orchestra.run.smithi143.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2023-11-26T07:16:51.773 INFO:teuthology.orchestra.run.smithi143.stdout:Device: 5h/5d Inode: 914 Links: 1 2023-11-26T07:16:51.773 INFO:teuthology.orchestra.run.smithi143.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2023-11-26T07:16:51.773 INFO:teuthology.orchestra.run.smithi143.stdout:Access: 2023-11-26 07:09:42.920793626 +0000 2023-11-26T07:16:51.773 INFO:teuthology.orchestra.run.smithi143.stdout:Modify: 2023-11-26 07:09:42.512797270 +0000 2023-11-26T07:16:51.774 INFO:teuthology.orchestra.run.smithi143.stdout:Change: 2023-11-26 07:09:42.512797270 +0000 2023-11-26T07:16:51.774 INFO:teuthology.orchestra.run.smithi143.stdout: Birth: 2023-11-26 07:09:42.512797270 +0000 2023-11-26T07:16:51.774 DEBUG:teuthology.orchestra.run.smithi143:> sudo dd if=/dev/vg_nvme/lv_2 of=/dev/null count=1 2023-11-26T07:16:51.839 INFO:teuthology.orchestra.run.smithi143.stderr:1+0 records in 2023-11-26T07:16:51.839 INFO:teuthology.orchestra.run.smithi143.stderr:1+0 records out 2023-11-26T07:16:51.839 INFO:teuthology.orchestra.run.smithi143.stderr:512 bytes copied, 0.000355248 s, 1.4 MB/s 2023-11-26T07:16:51.840 DEBUG:teuthology.orchestra.run.smithi143:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_2 2023-11-26T07:16:51.891 DEBUG:teuthology.orchestra.run.smithi143:> stat /dev/vg_nvme/lv_3 2023-11-26T07:16:51.941 INFO:teuthology.orchestra.run.smithi143.stdout: File: /dev/vg_nvme/lv_3 -> ../dm-2 2023-11-26T07:16:51.941 INFO:teuthology.orchestra.run.smithi143.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2023-11-26T07:16:51.941 INFO:teuthology.orchestra.run.smithi143.stdout:Device: 5h/5d Inode: 929 Links: 1 2023-11-26T07:16:51.941 INFO:teuthology.orchestra.run.smithi143.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2023-11-26T07:16:51.941 INFO:teuthology.orchestra.run.smithi143.stdout:Access: 2023-11-26 07:09:43.572787802 +0000 2023-11-26T07:16:51.941 INFO:teuthology.orchestra.run.smithi143.stdout:Modify: 2023-11-26 07:09:43.192791196 +0000 2023-11-26T07:16:51.941 INFO:teuthology.orchestra.run.smithi143.stdout:Change: 2023-11-26 07:09:43.192791196 +0000 2023-11-26T07:16:51.941 INFO:teuthology.orchestra.run.smithi143.stdout: Birth: 2023-11-26 07:09:43.192791196 +0000 2023-11-26T07:16:51.942 DEBUG:teuthology.orchestra.run.smithi143:> sudo dd if=/dev/vg_nvme/lv_3 of=/dev/null count=1 2023-11-26T07:16:52.002 INFO:teuthology.orchestra.run.smithi143.stderr:1+0 records in 2023-11-26T07:16:52.002 INFO:teuthology.orchestra.run.smithi143.stderr:1+0 records out 2023-11-26T07:16:52.002 INFO:teuthology.orchestra.run.smithi143.stderr:512 bytes copied, 0.000355588 s, 1.4 MB/s 2023-11-26T07:16:52.004 DEBUG:teuthology.orchestra.run.smithi143:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_3 2023-11-26T07:16:52.059 DEBUG:teuthology.orchestra.run.smithi143:> stat /dev/vg_nvme/lv_4 2023-11-26T07:16:52.109 INFO:teuthology.orchestra.run.smithi143.stdout: File: /dev/vg_nvme/lv_4 -> ../dm-3 2023-11-26T07:16:52.109 INFO:teuthology.orchestra.run.smithi143.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2023-11-26T07:16:52.109 INFO:teuthology.orchestra.run.smithi143.stdout:Device: 5h/5d Inode: 947 Links: 1 2023-11-26T07:16:52.109 INFO:teuthology.orchestra.run.smithi143.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2023-11-26T07:16:52.109 INFO:teuthology.orchestra.run.smithi143.stdout:Access: 2023-11-26 07:09:44.276781514 +0000 2023-11-26T07:16:52.109 INFO:teuthology.orchestra.run.smithi143.stdout:Modify: 2023-11-26 07:09:43.872785123 +0000 2023-11-26T07:16:52.109 INFO:teuthology.orchestra.run.smithi143.stdout:Change: 2023-11-26 07:09:43.872785123 +0000 2023-11-26T07:16:52.110 INFO:teuthology.orchestra.run.smithi143.stdout: Birth: 2023-11-26 07:09:43.872785123 +0000 2023-11-26T07:16:52.110 DEBUG:teuthology.orchestra.run.smithi143:> sudo dd if=/dev/vg_nvme/lv_4 of=/dev/null count=1 2023-11-26T07:16:52.175 INFO:teuthology.orchestra.run.smithi143.stderr:1+0 records in 2023-11-26T07:16:52.175 INFO:teuthology.orchestra.run.smithi143.stderr:1+0 records out 2023-11-26T07:16:52.175 INFO:teuthology.orchestra.run.smithi143.stderr:512 bytes copied, 0.000364451 s, 1.4 MB/s 2023-11-26T07:16:52.176 DEBUG:teuthology.orchestra.run.smithi143:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_4 2023-11-26T07:16:52.231 INFO:tasks.ceph:osd dev map: {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'} 2023-11-26T07:16:52.231 DEBUG:teuthology.orchestra.run.smithi188:> set -ex 2023-11-26T07:16:52.231 DEBUG:teuthology.orchestra.run.smithi188:> dd if=/scratch_devs of=/dev/stdout 2023-11-26T07:16:52.240 DEBUG:teuthology.misc:devs=['/dev/vg_nvme/lv_1', '/dev/vg_nvme/lv_2', '/dev/vg_nvme/lv_3', '/dev/vg_nvme/lv_4'] 2023-11-26T07:16:52.240 DEBUG:teuthology.orchestra.run.smithi188:> stat /dev/vg_nvme/lv_1 2023-11-26T07:16:52.295 INFO:teuthology.orchestra.run.smithi188.stdout: File: /dev/vg_nvme/lv_1 -> ../dm-0 2023-11-26T07:16:52.295 INFO:teuthology.orchestra.run.smithi188.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2023-11-26T07:16:52.295 INFO:teuthology.orchestra.run.smithi188.stdout:Device: 5h/5d Inode: 889 Links: 1 2023-11-26T07:16:52.295 INFO:teuthology.orchestra.run.smithi188.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2023-11-26T07:16:52.296 INFO:teuthology.orchestra.run.smithi188.stdout:Access: 2023-11-26 07:09:43.833255344 +0000 2023-11-26T07:16:52.296 INFO:teuthology.orchestra.run.smithi188.stdout:Modify: 2023-11-26 07:09:43.441258555 +0000 2023-11-26T07:16:52.296 INFO:teuthology.orchestra.run.smithi188.stdout:Change: 2023-11-26 07:09:43.441258555 +0000 2023-11-26T07:16:52.296 INFO:teuthology.orchestra.run.smithi188.stdout: Birth: 2023-11-26 07:09:43.441258555 +0000 2023-11-26T07:16:52.296 DEBUG:teuthology.orchestra.run.smithi188:> sudo dd if=/dev/vg_nvme/lv_1 of=/dev/null count=1 2023-11-26T07:16:52.358 INFO:teuthology.orchestra.run.smithi188.stderr:1+0 records in 2023-11-26T07:16:52.358 INFO:teuthology.orchestra.run.smithi188.stderr:1+0 records out 2023-11-26T07:16:52.358 INFO:teuthology.orchestra.run.smithi188.stderr:512 bytes copied, 0.000401666 s, 1.3 MB/s 2023-11-26T07:16:52.360 DEBUG:teuthology.orchestra.run.smithi188:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_1 2023-11-26T07:16:52.413 DEBUG:teuthology.orchestra.run.smithi188:> stat /dev/vg_nvme/lv_2 2023-11-26T07:16:52.463 INFO:teuthology.orchestra.run.smithi188.stdout: File: /dev/vg_nvme/lv_2 -> ../dm-1 2023-11-26T07:16:52.463 INFO:teuthology.orchestra.run.smithi188.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2023-11-26T07:16:52.463 INFO:teuthology.orchestra.run.smithi188.stdout:Device: 5h/5d Inode: 904 Links: 1 2023-11-26T07:16:52.463 INFO:teuthology.orchestra.run.smithi188.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2023-11-26T07:16:52.463 INFO:teuthology.orchestra.run.smithi188.stdout:Access: 2023-11-26 07:09:44.521249706 +0000 2023-11-26T07:16:52.463 INFO:teuthology.orchestra.run.smithi188.stdout:Modify: 2023-11-26 07:09:44.109253082 +0000 2023-11-26T07:16:52.463 INFO:teuthology.orchestra.run.smithi188.stdout:Change: 2023-11-26 07:09:44.109253082 +0000 2023-11-26T07:16:52.463 INFO:teuthology.orchestra.run.smithi188.stdout: Birth: 2023-11-26 07:09:44.109253082 +0000 2023-11-26T07:16:52.464 DEBUG:teuthology.orchestra.run.smithi188:> sudo dd if=/dev/vg_nvme/lv_2 of=/dev/null count=1 2023-11-26T07:16:52.526 INFO:teuthology.orchestra.run.smithi188.stderr:1+0 records in 2023-11-26T07:16:52.526 INFO:teuthology.orchestra.run.smithi188.stderr:1+0 records out 2023-11-26T07:16:52.526 INFO:teuthology.orchestra.run.smithi188.stderr:512 bytes copied, 0.000638587 s, 802 kB/s 2023-11-26T07:16:52.528 DEBUG:teuthology.orchestra.run.smithi188:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_2 2023-11-26T07:16:52.581 DEBUG:teuthology.orchestra.run.smithi188:> stat /dev/vg_nvme/lv_3 2023-11-26T07:16:52.635 INFO:teuthology.orchestra.run.smithi188.stdout: File: /dev/vg_nvme/lv_3 -> ../dm-2 2023-11-26T07:16:52.635 INFO:teuthology.orchestra.run.smithi188.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2023-11-26T07:16:52.635 INFO:teuthology.orchestra.run.smithi188.stdout:Device: 5h/5d Inode: 921 Links: 1 2023-11-26T07:16:52.635 INFO:teuthology.orchestra.run.smithi188.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2023-11-26T07:16:52.636 INFO:teuthology.orchestra.run.smithi188.stdout:Access: 2023-11-26 07:09:45.169244395 +0000 2023-11-26T07:16:52.636 INFO:teuthology.orchestra.run.smithi188.stdout:Modify: 2023-11-26 07:09:44.785247542 +0000 2023-11-26T07:16:52.636 INFO:teuthology.orchestra.run.smithi188.stdout:Change: 2023-11-26 07:09:44.785247542 +0000 2023-11-26T07:16:52.636 INFO:teuthology.orchestra.run.smithi188.stdout: Birth: 2023-11-26 07:09:44.785247542 +0000 2023-11-26T07:16:52.636 DEBUG:teuthology.orchestra.run.smithi188:> sudo dd if=/dev/vg_nvme/lv_3 of=/dev/null count=1 2023-11-26T07:16:52.702 INFO:teuthology.orchestra.run.smithi188.stderr:1+0 records in 2023-11-26T07:16:52.702 INFO:teuthology.orchestra.run.smithi188.stderr:1+0 records out 2023-11-26T07:16:52.702 INFO:teuthology.orchestra.run.smithi188.stderr:512 bytes copied, 0.00038369 s, 1.3 MB/s 2023-11-26T07:16:52.704 DEBUG:teuthology.orchestra.run.smithi188:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_3 2023-11-26T07:16:52.757 DEBUG:teuthology.orchestra.run.smithi188:> stat /dev/vg_nvme/lv_4 2023-11-26T07:16:52.811 INFO:teuthology.orchestra.run.smithi188.stdout: File: /dev/vg_nvme/lv_4 -> ../dm-3 2023-11-26T07:16:52.811 INFO:teuthology.orchestra.run.smithi188.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2023-11-26T07:16:52.811 INFO:teuthology.orchestra.run.smithi188.stdout:Device: 5h/5d Inode: 939 Links: 1 2023-11-26T07:16:52.811 INFO:teuthology.orchestra.run.smithi188.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2023-11-26T07:16:52.811 INFO:teuthology.orchestra.run.smithi188.stdout:Access: 2023-11-26 07:09:45.897238426 +0000 2023-11-26T07:16:52.812 INFO:teuthology.orchestra.run.smithi188.stdout:Modify: 2023-11-26 07:09:45.465241968 +0000 2023-11-26T07:16:52.812 INFO:teuthology.orchestra.run.smithi188.stdout:Change: 2023-11-26 07:09:45.465241968 +0000 2023-11-26T07:16:52.812 INFO:teuthology.orchestra.run.smithi188.stdout: Birth: 2023-11-26 07:09:45.465241968 +0000 2023-11-26T07:16:52.812 DEBUG:teuthology.orchestra.run.smithi188:> sudo dd if=/dev/vg_nvme/lv_4 of=/dev/null count=1 2023-11-26T07:16:52.873 INFO:teuthology.orchestra.run.smithi188.stderr:1+0 records in 2023-11-26T07:16:52.874 INFO:teuthology.orchestra.run.smithi188.stderr:1+0 records out 2023-11-26T07:16:52.874 INFO:teuthology.orchestra.run.smithi188.stderr:512 bytes copied, 0.000383026 s, 1.3 MB/s 2023-11-26T07:16:52.876 DEBUG:teuthology.orchestra.run.smithi188:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_4 2023-11-26T07:16:52.929 INFO:tasks.ceph:osd dev map: {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'} 2023-11-26T07:16:52.929 INFO:tasks.ceph:remote_to_roles_to_devs: {Remote(name='ubuntu@smithi143.front.sepia.ceph.com'): {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'}, Remote(name='ubuntu@smithi188.front.sepia.ceph.com'): {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'}} 2023-11-26T07:16:52.929 INFO:tasks.ceph:Generating config... 2023-11-26T07:16:52.931 INFO:tasks.ceph:[client] client mount timeout = 600 2023-11-26T07:16:52.931 INFO:tasks.ceph:[client] debug client = 20 2023-11-26T07:16:52.931 INFO:tasks.ceph:[client] debug ms = 1 2023-11-26T07:16:52.931 INFO:tasks.ceph:[client] pid file = 2023-11-26T07:16:52.931 INFO:tasks.ceph:[client] rados mon op timeout = 900 2023-11-26T07:16:52.931 INFO:tasks.ceph:[client] rados osd op timeout = 900 2023-11-26T07:16:52.931 INFO:tasks.ceph:[mds] debug mds = 20 2023-11-26T07:16:52.931 INFO:tasks.ceph:[mds] debug mds balancer = 20 2023-11-26T07:16:52.931 INFO:tasks.ceph:[mds] debug ms = 1 2023-11-26T07:16:52.932 INFO:tasks.ceph:[mds] mds debug frag = True 2023-11-26T07:16:52.932 INFO:tasks.ceph:[mds] mds debug scatterstat = True 2023-11-26T07:16:52.932 INFO:tasks.ceph:[mds] mds op complaint time = 180 2023-11-26T07:16:52.932 INFO:tasks.ceph:[mds] mds verify scatter = True 2023-11-26T07:16:52.932 INFO:tasks.ceph:[mds] osd op complaint time = 180 2023-11-26T07:16:52.932 INFO:tasks.ceph:[mds] rados mon op timeout = 900 2023-11-26T07:16:52.932 INFO:tasks.ceph:[mds] rados osd op timeout = 900 2023-11-26T07:16:52.932 INFO:tasks.ceph:[mgr] debug client = 20 2023-11-26T07:16:52.932 INFO:tasks.ceph:[mgr] debug finisher = 20 2023-11-26T07:16:52.932 INFO:tasks.ceph:[mgr] debug mgr = 20 2023-11-26T07:16:52.933 INFO:tasks.ceph:[mgr] debug ms = 1 2023-11-26T07:16:52.933 INFO:tasks.ceph:[mon] debug mon = 20 2023-11-26T07:16:52.933 INFO:tasks.ceph:[mon] debug ms = 1 2023-11-26T07:16:52.933 INFO:tasks.ceph:[mon] debug paxos = 20 2023-11-26T07:16:52.933 INFO:tasks.ceph:[mon] mon op complaint time = 120 2023-11-26T07:16:52.933 INFO:tasks.ceph:[osd] bdev async discard = True 2023-11-26T07:16:52.933 INFO:tasks.ceph:[osd] bdev enable discard = True 2023-11-26T07:16:52.933 INFO:tasks.ceph:[osd] bluestore allocator = bitmap 2023-11-26T07:16:52.933 INFO:tasks.ceph:[osd] bluestore block size = 96636764160 2023-11-26T07:16:52.933 INFO:tasks.ceph:[osd] bluestore fsck on mount = True 2023-11-26T07:16:52.933 INFO:tasks.ceph:[osd] debug bluefs = 1/20 2023-11-26T07:16:52.933 INFO:tasks.ceph:[osd] debug bluestore = 1/20 2023-11-26T07:16:52.934 INFO:tasks.ceph:[osd] debug ms = 1 2023-11-26T07:16:52.934 INFO:tasks.ceph:[osd] debug osd = 20 2023-11-26T07:16:52.934 INFO:tasks.ceph:[osd] debug rocksdb = 4/10 2023-11-26T07:16:52.934 INFO:tasks.ceph:[osd] mon osd backfillfull_ratio = 0.85 2023-11-26T07:16:52.934 INFO:tasks.ceph:[osd] mon osd full ratio = 0.9 2023-11-26T07:16:52.934 INFO:tasks.ceph:[osd] mon osd nearfull ratio = 0.8 2023-11-26T07:16:52.934 INFO:tasks.ceph:[osd] osd failsafe full ratio = 0.95 2023-11-26T07:16:52.934 INFO:tasks.ceph:[osd] osd objectstore = bluestore 2023-11-26T07:16:52.934 INFO:tasks.ceph:[osd] osd op complaint time = 180 2023-11-26T07:16:52.935 INFO:tasks.ceph:Setting up mon.a... 2023-11-26T07:16:52.935 DEBUG:teuthology.orchestra.run.smithi143:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring /etc/ceph/ceph.keyring 2023-11-26T07:16:52.979 INFO:teuthology.orchestra.run.smithi143.stdout:creating /etc/ceph/ceph.keyring 2023-11-26T07:16:52.991 DEBUG:teuthology.orchestra.run.smithi143:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --gen-key --name=mon. /etc/ceph/ceph.keyring 2023-11-26T07:16:53.080 DEBUG:teuthology.orchestra.run.smithi143:> sudo chmod 0644 /etc/ceph/ceph.keyring 2023-11-26T07:16:53.100 DEBUG:tasks.ceph:Ceph mon addresses: [('mon.a', '172.21.15.143'), ('mon.b', '172.21.15.188'), ('mon.c', '[v2:172.21.15.188:3301,v1:172.21.15.188:6790]')] 2023-11-26T07:16:53.100 DEBUG:tasks.ceph:writing out conf {'global': {'chdir': '', 'pid file': '/var/run/ceph/$cluster-$name.pid', 'auth supported': 'cephx', 'filestore xattr use omap': 'true', 'mon clock drift allowed': '1.000', 'osd crush chooseleaf type': '0', 'auth debug': 'true', 'ms die on old message': 'true', 'ms die on bug': 'true', 'mon max pg per osd': '10000', 'mon pg warn max object skew': '0', 'osd_pool_default_pg_autoscale_mode': 'off', 'osd pool default size': '2', 'mon osd allow primary affinity': 'true', 'mon osd allow pg remap': 'true', 'mon warn on legacy crush tunables': 'false', 'mon warn on crush straw calc version zero': 'false', 'mon warn on no sortbitwise': 'false', 'mon warn on osd down out interval zero': 'false', 'mon warn on too few osds': 'false', 'mon_warn_on_pool_pg_num_not_power_of_two': 'false', 'mon_warn_on_pool_no_redundancy': 'false', 'mon_allow_pool_size_one': 'true', 'osd pool default erasure code profile': 'plugin=jerasure technique=reed_sol_van k=2 m=1 crush-failure-domain=osd', 'osd default data pool replay window': '5', 'mon allow pool delete': 'true', 'mon cluster log file level': 'debug', 'debug asserts on shutdown': 'true', 'mon health detail to clog': 'false', 'mon host': '172.21.15.143,172.21.15.188,[v2:172.21.15.188:3301,v1:172.21.15.188:6790]'}, 'osd': {'osd journal size': '100', 'osd scrub load threshold': '5.0', 'osd scrub max interval': '600', 'osd mclock profile': 'high_recovery_ops', 'osd recover clone overlap': 'true', 'osd recovery max chunk': '1048576', 'osd debug shutdown': 'true', 'osd debug op order': 'true', 'osd debug verify stray on activate': 'true', 'osd debug trim objects': 'true', 'osd open classes on start': 'true', 'osd debug pg log writeout': 'true', 'osd deep scrub update digest min age': '30', 'osd map max advance': '10', 'journal zero on create': 'true', 'filestore ondisk finisher threads': '3', 'filestore apply finisher threads': '3', 'bdev debug aio': 'true', 'osd debug misdirected ops': 'true', 'bdev async discard': True, 'bdev enable discard': True, 'bluestore allocator': 'bitmap', 'bluestore block size': 96636764160, 'bluestore fsck on mount': True, 'debug bluefs': '1/20', 'debug bluestore': '1/20', 'debug ms': 1, 'debug osd': 20, 'debug rocksdb': '4/10', 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd failsafe full ratio': 0.95, 'osd objectstore': 'bluestore', 'osd op complaint time': 180}, 'mgr': {'debug ms': 1, 'debug mgr': 20, 'debug mon': '20', 'debug auth': '20', 'mon reweight min pgs per osd': '4', 'mon reweight min bytes per osd': '10', 'mgr/telemetry/nag': 'false', 'debug client': 20, 'debug finisher': 20}, 'mon': {'debug ms': 1, 'debug mon': 20, 'debug paxos': 20, 'debug auth': '20', 'mon data avail warn': '5', 'mon mgr mkfs grace': '240', 'mon reweight min pgs per osd': '4', 'mon osd reporter subtree level': 'osd', 'mon osd prime pg temp': 'true', 'mon reweight min bytes per osd': '10', 'auth mon ticket ttl': '660', 'auth service ticket ttl': '240', 'mon_warn_on_insecure_global_id_reclaim': 'false', 'mon_warn_on_insecure_global_id_reclaim_allowed': 'false', 'mon_down_mkfs_grace': '2m', 'mon_warn_on_filestore_osds': 'false', 'mon op complaint time': 120}, 'client': {'rgw cache enabled': 'true', 'rgw enable ops log': 'true', 'rgw enable usage log': 'true', 'log file': '/var/log/ceph/$cluster-$name.$pid.log', 'admin socket': '/var/run/ceph/$cluster-$name.$pid.asok', 'client mount timeout': 600, 'debug client': 20, 'debug ms': 1, 'pid file': '', 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'mon.a': {}, 'mon.b': {}, 'mon.c': {}, 'mds.a': {}, 'mds.c': {}, 'mds.b': {}, 'mds.d': {}, 'mds': {'debug mds': 20, 'debug mds balancer': 20, 'debug ms': 1, 'mds debug frag': True, 'mds debug scatterstat': True, 'mds op complaint time': 180, 'mds verify scatter': True, 'osd op complaint time': 180, 'rados mon op timeout': 900, 'rados osd op timeout': 900}} 2023-11-26T07:16:53.101 DEBUG:teuthology.orchestra.run.smithi143:> set -ex 2023-11-26T07:16:53.101 DEBUG:teuthology.orchestra.run.smithi143:> dd of=/home/ubuntu/cephtest/ceph.tmp.conf 2023-11-26T07:16:53.149 DEBUG:teuthology.orchestra.run.smithi143:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage monmaptool -c /home/ubuntu/cephtest/ceph.tmp.conf --create --clobber --enable-all-features --add a 172.21.15.143 --add b 172.21.15.188 --addv c '[v2:172.21.15.188:3301,v1:172.21.15.188:6790]' --print /home/ubuntu/cephtest/ceph.monmap 2023-11-26T07:16:53.243 INFO:teuthology.orchestra.run.smithi143.stdout:monmaptool: monmap file /home/ubuntu/cephtest/ceph.monmap 2023-11-26T07:16:53.243 INFO:teuthology.orchestra.run.smithi143.stdout:monmaptool: generated fsid f5033060-2469-4a07-95fe-2b76f5404bc1 2023-11-26T07:16:53.243 INFO:teuthology.orchestra.run.smithi143.stdout:setting min_mon_release = pacific 2023-11-26T07:16:53.243 INFO:teuthology.orchestra.run.smithi143.stdout:epoch 0 2023-11-26T07:16:53.243 INFO:teuthology.orchestra.run.smithi143.stdout:fsid f5033060-2469-4a07-95fe-2b76f5404bc1 2023-11-26T07:16:53.243 INFO:teuthology.orchestra.run.smithi143.stdout:last_changed 2023-11-26T07:16:53.246261+0000 2023-11-26T07:16:53.243 INFO:teuthology.orchestra.run.smithi143.stdout:created 2023-11-26T07:16:53.246261+0000 2023-11-26T07:16:53.243 INFO:teuthology.orchestra.run.smithi143.stdout:min_mon_release 16 (pacific) 2023-11-26T07:16:53.244 INFO:teuthology.orchestra.run.smithi143.stdout:election_strategy: 1 2023-11-26T07:16:53.244 INFO:teuthology.orchestra.run.smithi143.stdout:0: [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] mon.a 2023-11-26T07:16:53.244 INFO:teuthology.orchestra.run.smithi143.stdout:1: [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] mon.b 2023-11-26T07:16:53.244 INFO:teuthology.orchestra.run.smithi143.stdout:2: [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] mon.c 2023-11-26T07:16:53.244 INFO:teuthology.orchestra.run.smithi143.stdout:monmaptool: writing epoch 0 to /home/ubuntu/cephtest/ceph.monmap (3 monitors) 2023-11-26T07:16:53.246 DEBUG:teuthology.orchestra.run.smithi143:> rm -- /home/ubuntu/cephtest/ceph.tmp.conf 2023-11-26T07:16:53.254 INFO:tasks.ceph:Writing /etc/ceph/ceph.conf for FSID f5033060-2469-4a07-95fe-2b76f5404bc1... 2023-11-26T07:16:53.255 DEBUG:teuthology.orchestra.run.smithi143:> sudo mkdir -p /etc/ceph && sudo chmod 0755 /etc/ceph && sudo tee /etc/ceph/ceph.conf && sudo chmod 0644 /etc/ceph/ceph.conf > /dev/null 2023-11-26T07:16:53.300 DEBUG:teuthology.orchestra.run.smithi188:> sudo mkdir -p /etc/ceph && sudo chmod 0755 /etc/ceph && sudo tee /etc/ceph/ceph.conf && sudo chmod 0644 /etc/ceph/ceph.conf > /dev/null 2023-11-26T07:16:53.340 INFO:teuthology.orchestra.run.smithi143.stdout:[global] 2023-11-26T07:16:53.340 INFO:teuthology.orchestra.run.smithi143.stdout: chdir = "" 2023-11-26T07:16:53.341 INFO:teuthology.orchestra.run.smithi143.stdout: pid file = /var/run/ceph/$cluster-$name.pid 2023-11-26T07:16:53.341 INFO:teuthology.orchestra.run.smithi143.stdout: auth supported = cephx 2023-11-26T07:16:53.341 INFO:teuthology.orchestra.run.smithi143.stdout: 2023-11-26T07:16:53.341 INFO:teuthology.orchestra.run.smithi143.stdout: filestore xattr use omap = true 2023-11-26T07:16:53.341 INFO:teuthology.orchestra.run.smithi143.stdout: 2023-11-26T07:16:53.341 INFO:teuthology.orchestra.run.smithi143.stdout: mon clock drift allowed = 1.000 2023-11-26T07:16:53.341 INFO:teuthology.orchestra.run.smithi143.stdout: 2023-11-26T07:16:53.341 INFO:teuthology.orchestra.run.smithi143.stdout: osd crush chooseleaf type = 0 2023-11-26T07:16:53.341 INFO:teuthology.orchestra.run.smithi143.stdout: auth debug = true 2023-11-26T07:16:53.341 INFO:teuthology.orchestra.run.smithi143.stdout: 2023-11-26T07:16:53.341 INFO:teuthology.orchestra.run.smithi143.stdout: ms die on old message = true 2023-11-26T07:16:53.341 INFO:teuthology.orchestra.run.smithi143.stdout: ms die on bug = true 2023-11-26T07:16:53.342 INFO:teuthology.orchestra.run.smithi143.stdout: 2023-11-26T07:16:53.342 INFO:teuthology.orchestra.run.smithi143.stdout: mon max pg per osd = 10000 # >= luminous 2023-11-26T07:16:53.342 INFO:teuthology.orchestra.run.smithi143.stdout: mon pg warn max object skew = 0 2023-11-26T07:16:53.342 INFO:teuthology.orchestra.run.smithi143.stdout: 2023-11-26T07:16:53.342 INFO:teuthology.orchestra.run.smithi143.stdout: # disable pg_autoscaler by default for new pools 2023-11-26T07:16:53.342 INFO:teuthology.orchestra.run.smithi143.stdout: osd_pool_default_pg_autoscale_mode = off 2023-11-26T07:16:53.342 INFO:teuthology.orchestra.run.smithi143.stdout: 2023-11-26T07:16:53.342 INFO:teuthology.orchestra.run.smithi143.stdout: osd pool default size = 2 2023-11-26T07:16:53.342 INFO:teuthology.orchestra.run.smithi143.stdout: 2023-11-26T07:16:53.342 INFO:teuthology.orchestra.run.smithi143.stdout: mon osd allow primary affinity = true 2023-11-26T07:16:53.342 INFO:teuthology.orchestra.run.smithi143.stdout: mon osd allow pg remap = true 2023-11-26T07:16:53.342 INFO:teuthology.orchestra.run.smithi143.stdout: mon warn on legacy crush tunables = false 2023-11-26T07:16:53.343 INFO:teuthology.orchestra.run.smithi143.stdout: mon warn on crush straw calc version zero = false 2023-11-26T07:16:53.343 INFO:teuthology.orchestra.run.smithi143.stdout: mon warn on no sortbitwise = false 2023-11-26T07:16:53.343 INFO:teuthology.orchestra.run.smithi143.stdout: mon warn on osd down out interval zero = false 2023-11-26T07:16:53.343 INFO:teuthology.orchestra.run.smithi143.stdout: mon warn on too few osds = false 2023-11-26T07:16:53.343 INFO:teuthology.orchestra.run.smithi143.stdout: mon_warn_on_pool_pg_num_not_power_of_two = false 2023-11-26T07:16:53.343 INFO:teuthology.orchestra.run.smithi143.stdout: mon_warn_on_pool_no_redundancy = false 2023-11-26T07:16:53.343 INFO:teuthology.orchestra.run.smithi143.stdout: mon_allow_pool_size_one = true 2023-11-26T07:16:53.343 INFO:teuthology.orchestra.run.smithi143.stdout: 2023-11-26T07:16:53.343 INFO:teuthology.orchestra.run.smithi143.stdout: osd pool default erasure code profile = plugin=jerasure technique=reed_sol_van k=2 m=1 crush-failure-domain=osd 2023-11-26T07:16:53.343 INFO:teuthology.orchestra.run.smithi143.stdout: 2023-11-26T07:16:53.343 INFO:teuthology.orchestra.run.smithi143.stdout: osd default data pool replay window = 5 2023-11-26T07:16:53.343 INFO:teuthology.orchestra.run.smithi143.stdout: 2023-11-26T07:16:53.343 INFO:teuthology.orchestra.run.smithi143.stdout: mon allow pool delete = true 2023-11-26T07:16:53.344 INFO:teuthology.orchestra.run.smithi143.stdout: 2023-11-26T07:16:53.344 INFO:teuthology.orchestra.run.smithi143.stdout: mon cluster log file level = debug 2023-11-26T07:16:53.344 INFO:teuthology.orchestra.run.smithi143.stdout: debug asserts on shutdown = true 2023-11-26T07:16:53.344 INFO:teuthology.orchestra.run.smithi143.stdout: mon health detail to clog = false 2023-11-26T07:16:53.344 INFO:teuthology.orchestra.run.smithi143.stdout: mon host = "172.21.15.143,172.21.15.188,[v2:172.21.15.188:3301,v1:172.21.15.188:6790]" 2023-11-26T07:16:53.344 INFO:teuthology.orchestra.run.smithi143.stdout: fsid = f5033060-2469-4a07-95fe-2b76f5404bc1 2023-11-26T07:16:53.344 INFO:teuthology.orchestra.run.smithi143.stdout: 2023-11-26T07:16:53.344 INFO:teuthology.orchestra.run.smithi143.stdout:[osd] 2023-11-26T07:16:53.344 INFO:teuthology.orchestra.run.smithi143.stdout: osd journal size = 100 2023-11-26T07:16:53.344 INFO:teuthology.orchestra.run.smithi143.stdout: 2023-11-26T07:16:53.344 INFO:teuthology.orchestra.run.smithi143.stdout: osd scrub load threshold = 5.0 2023-11-26T07:16:53.344 INFO:teuthology.orchestra.run.smithi143.stdout: osd scrub max interval = 600 2023-11-26T07:16:53.344 INFO:teuthology.orchestra.run.smithi143.stdout: osd mclock profile = high_recovery_ops 2023-11-26T07:16:53.345 INFO:teuthology.orchestra.run.smithi143.stdout: 2023-11-26T07:16:53.345 INFO:teuthology.orchestra.run.smithi143.stdout: osd recover clone overlap = true 2023-11-26T07:16:53.345 INFO:teuthology.orchestra.run.smithi143.stdout: osd recovery max chunk = 1048576 2023-11-26T07:16:53.345 INFO:teuthology.orchestra.run.smithi143.stdout: 2023-11-26T07:16:53.345 INFO:teuthology.orchestra.run.smithi143.stdout: osd debug shutdown = true 2023-11-26T07:16:53.345 INFO:teuthology.orchestra.run.smithi143.stdout: osd debug op order = true 2023-11-26T07:16:53.345 INFO:teuthology.orchestra.run.smithi143.stdout: osd debug verify stray on activate = true 2023-11-26T07:16:53.345 INFO:teuthology.orchestra.run.smithi143.stdout: osd debug trim objects = true 2023-11-26T07:16:53.345 INFO:teuthology.orchestra.run.smithi143.stdout: 2023-11-26T07:16:53.345 INFO:teuthology.orchestra.run.smithi143.stdout: osd open classes on start = true 2023-11-26T07:16:53.345 INFO:teuthology.orchestra.run.smithi143.stdout: osd debug pg log writeout = true 2023-11-26T07:16:53.345 INFO:teuthology.orchestra.run.smithi143.stdout: 2023-11-26T07:16:53.345 INFO:teuthology.orchestra.run.smithi143.stdout: osd deep scrub update digest min age = 30 2023-11-26T07:16:53.346 INFO:teuthology.orchestra.run.smithi143.stdout: 2023-11-26T07:16:53.346 INFO:teuthology.orchestra.run.smithi143.stdout: osd map max advance = 10 2023-11-26T07:16:53.346 INFO:teuthology.orchestra.run.smithi143.stdout: 2023-11-26T07:16:53.346 INFO:teuthology.orchestra.run.smithi143.stdout: journal zero on create = true 2023-11-26T07:16:53.346 INFO:teuthology.orchestra.run.smithi143.stdout: 2023-11-26T07:16:53.346 INFO:teuthology.orchestra.run.smithi143.stdout: filestore ondisk finisher threads = 3 2023-11-26T07:16:53.346 INFO:teuthology.orchestra.run.smithi143.stdout: filestore apply finisher threads = 3 2023-11-26T07:16:53.346 INFO:teuthology.orchestra.run.smithi143.stdout: 2023-11-26T07:16:53.346 INFO:teuthology.orchestra.run.smithi143.stdout: bdev debug aio = true 2023-11-26T07:16:53.346 INFO:teuthology.orchestra.run.smithi143.stdout: osd debug misdirected ops = true 2023-11-26T07:16:53.346 INFO:teuthology.orchestra.run.smithi143.stdout: bdev async discard = True 2023-11-26T07:16:53.346 INFO:teuthology.orchestra.run.smithi143.stdout: bdev enable discard = True 2023-11-26T07:16:53.347 INFO:teuthology.orchestra.run.smithi143.stdout: bluestore allocator = bitmap 2023-11-26T07:16:53.347 INFO:teuthology.orchestra.run.smithi143.stdout: bluestore block size = 96636764160 2023-11-26T07:16:53.347 INFO:teuthology.orchestra.run.smithi143.stdout: bluestore fsck on mount = True 2023-11-26T07:16:53.347 INFO:teuthology.orchestra.run.smithi143.stdout: debug bluefs = 1/20 2023-11-26T07:16:53.347 INFO:teuthology.orchestra.run.smithi143.stdout: debug bluestore = 1/20 2023-11-26T07:16:53.347 INFO:teuthology.orchestra.run.smithi143.stdout: debug ms = 1 2023-11-26T07:16:53.347 INFO:teuthology.orchestra.run.smithi143.stdout: debug osd = 20 2023-11-26T07:16:53.347 INFO:teuthology.orchestra.run.smithi143.stdout: debug rocksdb = 4/10 2023-11-26T07:16:53.347 INFO:teuthology.orchestra.run.smithi143.stdout: mon osd backfillfull_ratio = 0.85 2023-11-26T07:16:53.347 INFO:teuthology.orchestra.run.smithi143.stdout: mon osd full ratio = 0.9 2023-11-26T07:16:53.347 INFO:teuthology.orchestra.run.smithi143.stdout: mon osd nearfull ratio = 0.8 2023-11-26T07:16:53.347 INFO:teuthology.orchestra.run.smithi143.stdout: osd failsafe full ratio = 0.95 2023-11-26T07:16:53.347 INFO:teuthology.orchestra.run.smithi143.stdout: osd objectstore = bluestore 2023-11-26T07:16:53.347 INFO:teuthology.orchestra.run.smithi143.stdout: osd op complaint time = 180 2023-11-26T07:16:53.348 INFO:teuthology.orchestra.run.smithi143.stdout: 2023-11-26T07:16:53.348 INFO:teuthology.orchestra.run.smithi143.stdout:[mgr] 2023-11-26T07:16:53.348 INFO:teuthology.orchestra.run.smithi143.stdout: debug ms = 1 2023-11-26T07:16:53.348 INFO:teuthology.orchestra.run.smithi143.stdout: debug mgr = 20 2023-11-26T07:16:53.348 INFO:teuthology.orchestra.run.smithi143.stdout: debug mon = 20 2023-11-26T07:16:53.348 INFO:teuthology.orchestra.run.smithi143.stdout: debug auth = 20 2023-11-26T07:16:53.348 INFO:teuthology.orchestra.run.smithi143.stdout: mon reweight min pgs per osd = 4 2023-11-26T07:16:53.348 INFO:teuthology.orchestra.run.smithi143.stdout: mon reweight min bytes per osd = 10 2023-11-26T07:16:53.348 INFO:teuthology.orchestra.run.smithi143.stdout: mgr/telemetry/nag = false 2023-11-26T07:16:53.348 INFO:teuthology.orchestra.run.smithi143.stdout: debug client = 20 2023-11-26T07:16:53.348 INFO:teuthology.orchestra.run.smithi143.stdout: debug finisher = 20 2023-11-26T07:16:53.348 INFO:teuthology.orchestra.run.smithi143.stdout: 2023-11-26T07:16:53.349 INFO:teuthology.orchestra.run.smithi143.stdout:[mon] 2023-11-26T07:16:53.349 INFO:teuthology.orchestra.run.smithi143.stdout: debug ms = 1 2023-11-26T07:16:53.349 INFO:teuthology.orchestra.run.smithi143.stdout: debug mon = 20 2023-11-26T07:16:53.349 INFO:teuthology.orchestra.run.smithi143.stdout: debug paxos = 20 2023-11-26T07:16:53.349 INFO:teuthology.orchestra.run.smithi143.stdout: debug auth = 20 2023-11-26T07:16:53.349 INFO:teuthology.orchestra.run.smithi143.stdout: mon data avail warn = 5 2023-11-26T07:16:53.349 INFO:teuthology.orchestra.run.smithi143.stdout: mon mgr mkfs grace = 240 2023-11-26T07:16:53.349 INFO:teuthology.orchestra.run.smithi143.stdout: mon reweight min pgs per osd = 4 2023-11-26T07:16:53.349 INFO:teuthology.orchestra.run.smithi143.stdout: mon osd reporter subtree level = osd 2023-11-26T07:16:53.349 INFO:teuthology.orchestra.run.smithi143.stdout: mon osd prime pg temp = true 2023-11-26T07:16:53.349 INFO:teuthology.orchestra.run.smithi143.stdout: mon reweight min bytes per osd = 10 2023-11-26T07:16:53.349 INFO:teuthology.orchestra.run.smithi143.stdout: 2023-11-26T07:16:53.349 INFO:teuthology.orchestra.run.smithi143.stdout: # rotate auth tickets quickly to exercise renewal paths 2023-11-26T07:16:53.350 INFO:teuthology.orchestra.run.smithi143.stdout: auth mon ticket ttl = 660 # 11m 2023-11-26T07:16:53.350 INFO:teuthology.orchestra.run.smithi143.stdout: auth service ticket ttl = 240 # 4m 2023-11-26T07:16:53.350 INFO:teuthology.orchestra.run.smithi143.stdout: 2023-11-26T07:16:53.350 INFO:teuthology.orchestra.run.smithi143.stdout: # don't complain about insecure global_id in the test suite 2023-11-26T07:16:53.350 INFO:teuthology.orchestra.run.smithi143.stdout: mon_warn_on_insecure_global_id_reclaim = false 2023-11-26T07:16:53.350 INFO:teuthology.orchestra.run.smithi143.stdout: mon_warn_on_insecure_global_id_reclaim_allowed = false 2023-11-26T07:16:53.350 INFO:teuthology.orchestra.run.smithi143.stdout: 2023-11-26T07:16:53.350 INFO:teuthology.orchestra.run.smithi143.stdout: # 1m isn't quite enough 2023-11-26T07:16:53.350 INFO:teuthology.orchestra.run.smithi143.stdout: mon_down_mkfs_grace = 2m 2023-11-26T07:16:53.350 INFO:teuthology.orchestra.run.smithi143.stdout: 2023-11-26T07:16:53.350 INFO:teuthology.orchestra.run.smithi143.stdout: mon_warn_on_filestore_osds = false 2023-11-26T07:16:53.350 INFO:teuthology.orchestra.run.smithi143.stdout: mon op complaint time = 120 2023-11-26T07:16:53.350 INFO:teuthology.orchestra.run.smithi143.stdout: 2023-11-26T07:16:53.351 INFO:teuthology.orchestra.run.smithi143.stdout:[client] 2023-11-26T07:16:53.351 INFO:teuthology.orchestra.run.smithi143.stdout: rgw cache enabled = true 2023-11-26T07:16:53.351 INFO:teuthology.orchestra.run.smithi143.stdout: rgw enable ops log = true 2023-11-26T07:16:53.351 INFO:teuthology.orchestra.run.smithi143.stdout: rgw enable usage log = true 2023-11-26T07:16:53.351 INFO:teuthology.orchestra.run.smithi143.stdout: log file = /var/log/ceph/$cluster-$name.$pid.log 2023-11-26T07:16:53.351 INFO:teuthology.orchestra.run.smithi143.stdout: admin socket = /var/run/ceph/$cluster-$name.$pid.asok 2023-11-26T07:16:53.351 INFO:teuthology.orchestra.run.smithi143.stdout: client mount timeout = 600 2023-11-26T07:16:53.351 INFO:teuthology.orchestra.run.smithi143.stdout: debug client = 20 2023-11-26T07:16:53.351 INFO:teuthology.orchestra.run.smithi143.stdout: debug ms = 1 2023-11-26T07:16:53.351 INFO:teuthology.orchestra.run.smithi143.stdout: pid file = "" 2023-11-26T07:16:53.351 INFO:teuthology.orchestra.run.smithi143.stdout: rados mon op timeout = 900 2023-11-26T07:16:53.351 INFO:teuthology.orchestra.run.smithi143.stdout: rados osd op timeout = 900 2023-11-26T07:16:53.351 INFO:teuthology.orchestra.run.smithi143.stdout:[mon.a] 2023-11-26T07:16:53.352 INFO:teuthology.orchestra.run.smithi143.stdout:[mon.b] 2023-11-26T07:16:53.352 INFO:teuthology.orchestra.run.smithi143.stdout:[mon.c] 2023-11-26T07:16:53.352 INFO:teuthology.orchestra.run.smithi143.stdout:[mds.a] 2023-11-26T07:16:53.352 INFO:teuthology.orchestra.run.smithi143.stdout:[mds.c] 2023-11-26T07:16:53.352 INFO:teuthology.orchestra.run.smithi143.stdout:[mds.b] 2023-11-26T07:16:53.352 INFO:teuthology.orchestra.run.smithi143.stdout:[mds.d] 2023-11-26T07:16:53.352 INFO:teuthology.orchestra.run.smithi143.stdout:[mds] 2023-11-26T07:16:53.352 INFO:teuthology.orchestra.run.smithi143.stdout: debug mds = 20 2023-11-26T07:16:53.352 INFO:teuthology.orchestra.run.smithi143.stdout: debug mds balancer = 20 2023-11-26T07:16:53.352 INFO:teuthology.orchestra.run.smithi143.stdout: debug ms = 1 2023-11-26T07:16:53.352 INFO:teuthology.orchestra.run.smithi143.stdout: mds debug frag = True 2023-11-26T07:16:53.352 INFO:teuthology.orchestra.run.smithi143.stdout: mds debug scatterstat = True 2023-11-26T07:16:53.352 INFO:teuthology.orchestra.run.smithi143.stdout: mds op complaint time = 180 2023-11-26T07:16:53.353 INFO:teuthology.orchestra.run.smithi143.stdout: mds verify scatter = True 2023-11-26T07:16:53.353 INFO:teuthology.orchestra.run.smithi143.stdout: osd op complaint time = 180 2023-11-26T07:16:53.353 INFO:teuthology.orchestra.run.smithi143.stdout: rados mon op timeout = 900 2023-11-26T07:16:53.353 INFO:teuthology.orchestra.run.smithi143.stdout: rados osd op timeout = 900 2023-11-26T07:16:53.353 INFO:teuthology.orchestra.run.smithi188.stdout:[global] 2023-11-26T07:16:53.353 INFO:teuthology.orchestra.run.smithi188.stdout: chdir = "" 2023-11-26T07:16:53.353 INFO:teuthology.orchestra.run.smithi188.stdout: pid file = /var/run/ceph/$cluster-$name.pid 2023-11-26T07:16:53.353 INFO:teuthology.orchestra.run.smithi188.stdout: auth supported = cephx 2023-11-26T07:16:53.353 INFO:teuthology.orchestra.run.smithi188.stdout: 2023-11-26T07:16:53.354 INFO:teuthology.orchestra.run.smithi188.stdout: filestore xattr use omap = true 2023-11-26T07:16:53.354 INFO:teuthology.orchestra.run.smithi188.stdout: 2023-11-26T07:16:53.354 INFO:teuthology.orchestra.run.smithi188.stdout: mon clock drift allowed = 1.000 2023-11-26T07:16:53.354 INFO:teuthology.orchestra.run.smithi188.stdout: 2023-11-26T07:16:53.354 INFO:teuthology.orchestra.run.smithi188.stdout: osd crush chooseleaf type = 0 2023-11-26T07:16:53.354 INFO:teuthology.orchestra.run.smithi188.stdout: auth debug = true 2023-11-26T07:16:53.354 INFO:teuthology.orchestra.run.smithi188.stdout: 2023-11-26T07:16:53.354 INFO:teuthology.orchestra.run.smithi188.stdout: ms die on old message = true 2023-11-26T07:16:53.354 INFO:teuthology.orchestra.run.smithi188.stdout: ms die on bug = true 2023-11-26T07:16:53.354 INFO:teuthology.orchestra.run.smithi188.stdout: 2023-11-26T07:16:53.354 INFO:teuthology.orchestra.run.smithi188.stdout: mon max pg per osd = 10000 # >= luminous 2023-11-26T07:16:53.354 INFO:teuthology.orchestra.run.smithi188.stdout: mon pg warn max object skew = 0 2023-11-26T07:16:53.355 INFO:teuthology.orchestra.run.smithi188.stdout: 2023-11-26T07:16:53.355 INFO:teuthology.orchestra.run.smithi188.stdout: # disable pg_autoscaler by default for new pools 2023-11-26T07:16:53.355 INFO:teuthology.orchestra.run.smithi188.stdout: osd_pool_default_pg_autoscale_mode = off 2023-11-26T07:16:53.355 INFO:teuthology.orchestra.run.smithi188.stdout: 2023-11-26T07:16:53.355 INFO:teuthology.orchestra.run.smithi188.stdout: osd pool default size = 2 2023-11-26T07:16:53.355 INFO:teuthology.orchestra.run.smithi188.stdout: 2023-11-26T07:16:53.355 INFO:teuthology.orchestra.run.smithi188.stdout: mon osd allow primary affinity = true 2023-11-26T07:16:53.355 INFO:teuthology.orchestra.run.smithi188.stdout: mon osd allow pg remap = true 2023-11-26T07:16:53.355 INFO:teuthology.orchestra.run.smithi188.stdout: mon warn on legacy crush tunables = false 2023-11-26T07:16:53.355 INFO:teuthology.orchestra.run.smithi188.stdout: mon warn on crush straw calc version zero = false 2023-11-26T07:16:53.355 INFO:teuthology.orchestra.run.smithi188.stdout: mon warn on no sortbitwise = false 2023-11-26T07:16:53.355 INFO:teuthology.orchestra.run.smithi188.stdout: mon warn on osd down out interval zero = false 2023-11-26T07:16:53.355 INFO:teuthology.orchestra.run.smithi188.stdout: mon warn on too few osds = false 2023-11-26T07:16:53.356 INFO:teuthology.orchestra.run.smithi188.stdout: mon_warn_on_pool_pg_num_not_power_of_two = false 2023-11-26T07:16:53.356 INFO:teuthology.orchestra.run.smithi188.stdout: mon_warn_on_pool_no_redundancy = false 2023-11-26T07:16:53.356 INFO:teuthology.orchestra.run.smithi188.stdout: mon_allow_pool_size_one = true 2023-11-26T07:16:53.356 INFO:teuthology.orchestra.run.smithi188.stdout: 2023-11-26T07:16:53.356 INFO:teuthology.orchestra.run.smithi188.stdout: osd pool default erasure code profile = plugin=jerasure technique=reed_sol_van k=2 m=1 crush-failure-domain=osd 2023-11-26T07:16:53.356 INFO:teuthology.orchestra.run.smithi188.stdout: 2023-11-26T07:16:53.356 INFO:teuthology.orchestra.run.smithi188.stdout: osd default data pool replay window = 5 2023-11-26T07:16:53.356 INFO:teuthology.orchestra.run.smithi188.stdout: 2023-11-26T07:16:53.356 INFO:teuthology.orchestra.run.smithi188.stdout: mon allow pool delete = true 2023-11-26T07:16:53.356 INFO:teuthology.orchestra.run.smithi188.stdout: 2023-11-26T07:16:53.356 INFO:teuthology.orchestra.run.smithi188.stdout: mon cluster log file level = debug 2023-11-26T07:16:53.356 INFO:teuthology.orchestra.run.smithi188.stdout: debug asserts on shutdown = true 2023-11-26T07:16:53.356 INFO:teuthology.orchestra.run.smithi188.stdout: mon health detail to clog = false 2023-11-26T07:16:53.357 INFO:teuthology.orchestra.run.smithi188.stdout: mon host = "172.21.15.143,172.21.15.188,[v2:172.21.15.188:3301,v1:172.21.15.188:6790]" 2023-11-26T07:16:53.357 INFO:teuthology.orchestra.run.smithi188.stdout: fsid = f5033060-2469-4a07-95fe-2b76f5404bc1 2023-11-26T07:16:53.357 INFO:teuthology.orchestra.run.smithi188.stdout: 2023-11-26T07:16:53.357 INFO:teuthology.orchestra.run.smithi188.stdout:[osd] 2023-11-26T07:16:53.357 INFO:teuthology.orchestra.run.smithi188.stdout: osd journal size = 100 2023-11-26T07:16:53.357 INFO:teuthology.orchestra.run.smithi188.stdout: 2023-11-26T07:16:53.357 INFO:teuthology.orchestra.run.smithi188.stdout: osd scrub load threshold = 5.0 2023-11-26T07:16:53.357 INFO:teuthology.orchestra.run.smithi188.stdout: osd scrub max interval = 600 2023-11-26T07:16:53.357 INFO:teuthology.orchestra.run.smithi188.stdout: osd mclock profile = high_recovery_ops 2023-11-26T07:16:53.357 INFO:teuthology.orchestra.run.smithi188.stdout: 2023-11-26T07:16:53.357 INFO:teuthology.orchestra.run.smithi188.stdout: osd recover clone overlap = true 2023-11-26T07:16:53.357 INFO:teuthology.orchestra.run.smithi188.stdout: osd recovery max chunk = 1048576 2023-11-26T07:16:53.357 INFO:teuthology.orchestra.run.smithi188.stdout: 2023-11-26T07:16:53.358 INFO:teuthology.orchestra.run.smithi188.stdout: osd debug shutdown = true 2023-11-26T07:16:53.358 INFO:teuthology.orchestra.run.smithi188.stdout: osd debug op order = true 2023-11-26T07:16:53.358 INFO:teuthology.orchestra.run.smithi188.stdout: osd debug verify stray on activate = true 2023-11-26T07:16:53.358 INFO:teuthology.orchestra.run.smithi188.stdout: osd debug trim objects = true 2023-11-26T07:16:53.358 INFO:teuthology.orchestra.run.smithi188.stdout: 2023-11-26T07:16:53.358 INFO:teuthology.orchestra.run.smithi188.stdout: osd open classes on start = true 2023-11-26T07:16:53.358 INFO:teuthology.orchestra.run.smithi188.stdout: osd debug pg log writeout = true 2023-11-26T07:16:53.358 INFO:teuthology.orchestra.run.smithi188.stdout: 2023-11-26T07:16:53.358 INFO:teuthology.orchestra.run.smithi188.stdout: osd deep scrub update digest min age = 30 2023-11-26T07:16:53.358 INFO:teuthology.orchestra.run.smithi188.stdout: 2023-11-26T07:16:53.358 INFO:teuthology.orchestra.run.smithi188.stdout: osd map max advance = 10 2023-11-26T07:16:53.358 INFO:teuthology.orchestra.run.smithi188.stdout: 2023-11-26T07:16:53.358 INFO:teuthology.orchestra.run.smithi188.stdout: journal zero on create = true 2023-11-26T07:16:53.359 INFO:teuthology.orchestra.run.smithi188.stdout: 2023-11-26T07:16:53.359 INFO:teuthology.orchestra.run.smithi188.stdout: filestore ondisk finisher threads = 3 2023-11-26T07:16:53.359 INFO:teuthology.orchestra.run.smithi188.stdout: filestore apply finisher threads = 3 2023-11-26T07:16:53.359 INFO:teuthology.orchestra.run.smithi188.stdout: 2023-11-26T07:16:53.359 INFO:teuthology.orchestra.run.smithi188.stdout: bdev debug aio = true 2023-11-26T07:16:53.359 INFO:teuthology.orchestra.run.smithi188.stdout: osd debug misdirected ops = true 2023-11-26T07:16:53.359 INFO:teuthology.orchestra.run.smithi188.stdout: bdev async discard = True 2023-11-26T07:16:53.359 INFO:teuthology.orchestra.run.smithi188.stdout: bdev enable discard = True 2023-11-26T07:16:53.359 INFO:teuthology.orchestra.run.smithi188.stdout: bluestore allocator = bitmap 2023-11-26T07:16:53.359 INFO:teuthology.orchestra.run.smithi188.stdout: bluestore block size = 96636764160 2023-11-26T07:16:53.359 INFO:teuthology.orchestra.run.smithi188.stdout: bluestore fsck on mount = True 2023-11-26T07:16:53.359 INFO:teuthology.orchestra.run.smithi188.stdout: debug bluefs = 1/20 2023-11-26T07:16:53.359 INFO:teuthology.orchestra.run.smithi188.stdout: debug bluestore = 1/20 2023-11-26T07:16:53.360 INFO:teuthology.orchestra.run.smithi188.stdout: debug ms = 1 2023-11-26T07:16:53.360 INFO:teuthology.orchestra.run.smithi188.stdout: debug osd = 20 2023-11-26T07:16:53.360 INFO:teuthology.orchestra.run.smithi188.stdout: debug rocksdb = 4/10 2023-11-26T07:16:53.360 INFO:teuthology.orchestra.run.smithi188.stdout: mon osd backfillfull_ratio = 0.85 2023-11-26T07:16:53.360 INFO:teuthology.orchestra.run.smithi188.stdout: mon osd full ratio = 0.9 2023-11-26T07:16:53.360 INFO:teuthology.orchestra.run.smithi188.stdout: mon osd nearfull ratio = 0.8 2023-11-26T07:16:53.360 INFO:teuthology.orchestra.run.smithi188.stdout: osd failsafe full ratio = 0.95 2023-11-26T07:16:53.360 INFO:teuthology.orchestra.run.smithi188.stdout: osd objectstore = bluestore 2023-11-26T07:16:53.360 INFO:teuthology.orchestra.run.smithi188.stdout: osd op complaint time = 180 2023-11-26T07:16:53.360 INFO:teuthology.orchestra.run.smithi188.stdout: 2023-11-26T07:16:53.360 INFO:teuthology.orchestra.run.smithi188.stdout:[mgr] 2023-11-26T07:16:53.360 INFO:teuthology.orchestra.run.smithi188.stdout: debug ms = 1 2023-11-26T07:16:53.361 INFO:teuthology.orchestra.run.smithi188.stdout: debug mgr = 20 2023-11-26T07:16:53.361 INFO:teuthology.orchestra.run.smithi188.stdout: debug mon = 20 2023-11-26T07:16:53.361 INFO:teuthology.orchestra.run.smithi188.stdout: debug auth = 20 2023-11-26T07:16:53.361 INFO:teuthology.orchestra.run.smithi188.stdout: mon reweight min pgs per osd = 4 2023-11-26T07:16:53.361 INFO:teuthology.orchestra.run.smithi188.stdout: mon reweight min bytes per osd = 10 2023-11-26T07:16:53.361 INFO:teuthology.orchestra.run.smithi188.stdout: mgr/telemetry/nag = false 2023-11-26T07:16:53.361 INFO:teuthology.orchestra.run.smithi188.stdout: debug client = 20 2023-11-26T07:16:53.361 INFO:teuthology.orchestra.run.smithi188.stdout: debug finisher = 20 2023-11-26T07:16:53.361 INFO:teuthology.orchestra.run.smithi188.stdout: 2023-11-26T07:16:53.361 INFO:teuthology.orchestra.run.smithi188.stdout:[mon] 2023-11-26T07:16:53.361 INFO:teuthology.orchestra.run.smithi188.stdout: debug ms = 1 2023-11-26T07:16:53.361 INFO:teuthology.orchestra.run.smithi188.stdout: debug mon = 20 2023-11-26T07:16:53.361 INFO:teuthology.orchestra.run.smithi188.stdout: debug paxos = 20 2023-11-26T07:16:53.362 INFO:teuthology.orchestra.run.smithi188.stdout: debug auth = 20 2023-11-26T07:16:53.362 INFO:teuthology.orchestra.run.smithi188.stdout: mon data avail warn = 5 2023-11-26T07:16:53.362 INFO:teuthology.orchestra.run.smithi188.stdout: mon mgr mkfs grace = 240 2023-11-26T07:16:53.362 INFO:teuthology.orchestra.run.smithi188.stdout: mon reweight min pgs per osd = 4 2023-11-26T07:16:53.362 INFO:teuthology.orchestra.run.smithi188.stdout: mon osd reporter subtree level = osd 2023-11-26T07:16:53.362 INFO:teuthology.orchestra.run.smithi188.stdout: mon osd prime pg temp = true 2023-11-26T07:16:53.362 INFO:teuthology.orchestra.run.smithi188.stdout: mon reweight min bytes per osd = 10 2023-11-26T07:16:53.362 INFO:teuthology.orchestra.run.smithi188.stdout: 2023-11-26T07:16:53.362 INFO:teuthology.orchestra.run.smithi188.stdout: # rotate auth tickets quickly to exercise renewal paths 2023-11-26T07:16:53.362 INFO:teuthology.orchestra.run.smithi188.stdout: auth mon ticket ttl = 660 # 11m 2023-11-26T07:16:53.362 INFO:teuthology.orchestra.run.smithi188.stdout: auth service ticket ttl = 240 # 4m 2023-11-26T07:16:53.362 INFO:teuthology.orchestra.run.smithi188.stdout: 2023-11-26T07:16:53.362 INFO:teuthology.orchestra.run.smithi188.stdout: # don't complain about insecure global_id in the test suite 2023-11-26T07:16:53.363 INFO:teuthology.orchestra.run.smithi188.stdout: mon_warn_on_insecure_global_id_reclaim = false 2023-11-26T07:16:53.363 INFO:teuthology.orchestra.run.smithi188.stdout: mon_warn_on_insecure_global_id_reclaim_allowed = false 2023-11-26T07:16:53.363 INFO:teuthology.orchestra.run.smithi188.stdout: 2023-11-26T07:16:53.363 INFO:teuthology.orchestra.run.smithi188.stdout: # 1m isn't quite enough 2023-11-26T07:16:53.363 INFO:teuthology.orchestra.run.smithi188.stdout: mon_down_mkfs_grace = 2m 2023-11-26T07:16:53.363 INFO:teuthology.orchestra.run.smithi188.stdout: 2023-11-26T07:16:53.363 INFO:teuthology.orchestra.run.smithi188.stdout: mon_warn_on_filestore_osds = false 2023-11-26T07:16:53.363 INFO:teuthology.orchestra.run.smithi188.stdout: mon op complaint time = 120 2023-11-26T07:16:53.363 INFO:teuthology.orchestra.run.smithi188.stdout: 2023-11-26T07:16:53.363 INFO:teuthology.orchestra.run.smithi188.stdout:[client] 2023-11-26T07:16:53.363 INFO:teuthology.orchestra.run.smithi188.stdout: rgw cache enabled = true 2023-11-26T07:16:53.363 INFO:teuthology.orchestra.run.smithi188.stdout: rgw enable ops log = true 2023-11-26T07:16:53.363 INFO:teuthology.orchestra.run.smithi188.stdout: rgw enable usage log = true 2023-11-26T07:16:53.363 INFO:teuthology.orchestra.run.smithi188.stdout: log file = /var/log/ceph/$cluster-$name.$pid.log 2023-11-26T07:16:53.364 INFO:teuthology.orchestra.run.smithi188.stdout: admin socket = /var/run/ceph/$cluster-$name.$pid.asok 2023-11-26T07:16:53.364 INFO:teuthology.orchestra.run.smithi188.stdout: client mount timeout = 600 2023-11-26T07:16:53.364 INFO:teuthology.orchestra.run.smithi188.stdout: debug client = 20 2023-11-26T07:16:53.364 INFO:teuthology.orchestra.run.smithi188.stdout: debug ms = 1 2023-11-26T07:16:53.364 INFO:teuthology.orchestra.run.smithi188.stdout: pid file = "" 2023-11-26T07:16:53.364 INFO:teuthology.orchestra.run.smithi188.stdout: rados mon op timeout = 900 2023-11-26T07:16:53.364 INFO:teuthology.orchestra.run.smithi188.stdout: rados osd op timeout = 900 2023-11-26T07:16:53.364 INFO:teuthology.orchestra.run.smithi188.stdout:[mon.a] 2023-11-26T07:16:53.364 INFO:teuthology.orchestra.run.smithi188.stdout:[mon.b] 2023-11-26T07:16:53.364 INFO:teuthology.orchestra.run.smithi188.stdout:[mon.c] 2023-11-26T07:16:53.364 INFO:teuthology.orchestra.run.smithi188.stdout:[mds.a] 2023-11-26T07:16:53.364 INFO:teuthology.orchestra.run.smithi188.stdout:[mds.c] 2023-11-26T07:16:53.364 INFO:teuthology.orchestra.run.smithi188.stdout:[mds.b] 2023-11-26T07:16:53.365 INFO:teuthology.orchestra.run.smithi188.stdout:[mds.d] 2023-11-26T07:16:53.365 INFO:teuthology.orchestra.run.smithi188.stdout:[mds] 2023-11-26T07:16:53.365 INFO:teuthology.orchestra.run.smithi188.stdout: debug mds = 20 2023-11-26T07:16:53.365 INFO:teuthology.orchestra.run.smithi188.stdout: debug mds balancer = 20 2023-11-26T07:16:53.365 INFO:teuthology.orchestra.run.smithi188.stdout: debug ms = 1 2023-11-26T07:16:53.365 INFO:teuthology.orchestra.run.smithi188.stdout: mds debug frag = True 2023-11-26T07:16:53.365 INFO:teuthology.orchestra.run.smithi188.stdout: mds debug scatterstat = True 2023-11-26T07:16:53.365 INFO:teuthology.orchestra.run.smithi188.stdout: mds op complaint time = 180 2023-11-26T07:16:53.365 INFO:teuthology.orchestra.run.smithi188.stdout: mds verify scatter = True 2023-11-26T07:16:53.365 INFO:teuthology.orchestra.run.smithi188.stdout: osd op complaint time = 180 2023-11-26T07:16:53.365 INFO:teuthology.orchestra.run.smithi188.stdout: rados mon op timeout = 900 2023-11-26T07:16:53.365 INFO:teuthology.orchestra.run.smithi188.stdout: rados osd op timeout = 900 2023-11-26T07:16:53.366 INFO:tasks.ceph:Creating admin key on mon.a... 2023-11-26T07:16:53.367 DEBUG:teuthology.orchestra.run.smithi143:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --gen-key --name=client.admin --cap mon 'allow *' --cap osd 'allow *' --cap mds 'allow *' --cap mgr 'allow *' /etc/ceph/ceph.keyring 2023-11-26T07:16:53.470 INFO:tasks.ceph:Copying monmap to all nodes... 2023-11-26T07:16:53.470 DEBUG:teuthology.orchestra.run.smithi143:> set -ex 2023-11-26T07:16:53.470 DEBUG:teuthology.orchestra.run.smithi143:> dd if=/etc/ceph/ceph.keyring of=/dev/stdout 2023-11-26T07:16:53.479 DEBUG:teuthology.orchestra.run.smithi143:> set -ex 2023-11-26T07:16:53.479 DEBUG:teuthology.orchestra.run.smithi143:> dd if=/home/ubuntu/cephtest/ceph.monmap of=/dev/stdout 2023-11-26T07:16:53.533 INFO:tasks.ceph:Sending monmap to node ubuntu@smithi143.front.sepia.ceph.com 2023-11-26T07:16:53.533 DEBUG:teuthology.orchestra.run.smithi143:> set -ex 2023-11-26T07:16:53.533 DEBUG:teuthology.orchestra.run.smithi143:> sudo dd of=/etc/ceph/ceph.keyring 2023-11-26T07:16:53.533 DEBUG:teuthology.orchestra.run.smithi143:> sudo chmod 0644 /etc/ceph/ceph.keyring 2023-11-26T07:16:53.609 DEBUG:teuthology.orchestra.run.smithi143:> set -ex 2023-11-26T07:16:53.609 DEBUG:teuthology.orchestra.run.smithi143:> dd of=/home/ubuntu/cephtest/ceph.monmap 2023-11-26T07:16:53.661 INFO:tasks.ceph:Sending monmap to node ubuntu@smithi188.front.sepia.ceph.com 2023-11-26T07:16:53.661 DEBUG:teuthology.orchestra.run.smithi188:> set -ex 2023-11-26T07:16:53.661 DEBUG:teuthology.orchestra.run.smithi188:> sudo dd of=/etc/ceph/ceph.keyring 2023-11-26T07:16:53.661 DEBUG:teuthology.orchestra.run.smithi188:> sudo chmod 0644 /etc/ceph/ceph.keyring 2023-11-26T07:16:53.697 DEBUG:teuthology.orchestra.run.smithi188:> set -ex 2023-11-26T07:16:53.697 DEBUG:teuthology.orchestra.run.smithi188:> dd of=/home/ubuntu/cephtest/ceph.monmap 2023-11-26T07:16:53.751 INFO:tasks.ceph:Setting up mon nodes... 2023-11-26T07:16:53.751 INFO:tasks.ceph:Setting up mgr nodes... 2023-11-26T07:16:53.751 DEBUG:teuthology.orchestra.run.smithi143:> sudo mkdir -p /var/lib/ceph/mgr/ceph-y && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=mgr.y /var/lib/ceph/mgr/ceph-y/keyring 2023-11-26T07:16:53.807 INFO:teuthology.orchestra.run.smithi143.stdout:creating /var/lib/ceph/mgr/ceph-y/keyring 2023-11-26T07:16:53.812 DEBUG:teuthology.orchestra.run.smithi188:> sudo mkdir -p /var/lib/ceph/mgr/ceph-x && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=mgr.x /var/lib/ceph/mgr/ceph-x/keyring 2023-11-26T07:16:53.873 INFO:teuthology.orchestra.run.smithi188.stdout:creating /var/lib/ceph/mgr/ceph-x/keyring 2023-11-26T07:16:53.887 INFO:tasks.ceph:Setting up mds nodes... 2023-11-26T07:16:53.887 DEBUG:teuthology.orchestra.run.smithi143:> sudo mkdir -p /var/lib/ceph/mds/ceph-a && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=mds.a /var/lib/ceph/mds/ceph-a/keyring 2023-11-26T07:16:53.943 INFO:teuthology.orchestra.run.smithi143.stdout:creating /var/lib/ceph/mds/ceph-a/keyring 2023-11-26T07:16:53.947 DEBUG:teuthology.orchestra.run.smithi143:> sudo chown -R ceph:ceph /var/lib/ceph/mds/ceph-a 2023-11-26T07:16:53.967 DEBUG:teuthology.orchestra.run.smithi143:> sudo mkdir -p /var/lib/ceph/mds/ceph-c && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=mds.c /var/lib/ceph/mds/ceph-c/keyring 2023-11-26T07:16:54.065 INFO:teuthology.orchestra.run.smithi143.stdout:creating /var/lib/ceph/mds/ceph-c/keyring 2023-11-26T07:16:54.069 DEBUG:teuthology.orchestra.run.smithi143:> sudo chown -R ceph:ceph /var/lib/ceph/mds/ceph-c 2023-11-26T07:16:54.089 DEBUG:teuthology.orchestra.run.smithi188:> sudo mkdir -p /var/lib/ceph/mds/ceph-b && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=mds.b /var/lib/ceph/mds/ceph-b/keyring 2023-11-26T07:16:54.148 INFO:teuthology.orchestra.run.smithi188.stdout:creating /var/lib/ceph/mds/ceph-b/keyring 2023-11-26T07:16:54.152 DEBUG:teuthology.orchestra.run.smithi188:> sudo chown -R ceph:ceph /var/lib/ceph/mds/ceph-b 2023-11-26T07:16:54.173 DEBUG:teuthology.orchestra.run.smithi188:> sudo mkdir -p /var/lib/ceph/mds/ceph-d && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=mds.d /var/lib/ceph/mds/ceph-d/keyring 2023-11-26T07:16:54.274 INFO:teuthology.orchestra.run.smithi188.stdout:creating /var/lib/ceph/mds/ceph-d/keyring 2023-11-26T07:16:54.278 DEBUG:teuthology.orchestra.run.smithi188:> sudo chown -R ceph:ceph /var/lib/ceph/mds/ceph-d 2023-11-26T07:16:54.298 INFO:tasks.ceph_client:Setting up client nodes... 2023-11-26T07:16:54.298 DEBUG:teuthology.orchestra.run.smithi143:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=client.0 /etc/ceph/ceph.client.0.keyring && sudo chmod 0644 /etc/ceph/ceph.client.0.keyring 2023-11-26T07:16:54.340 INFO:teuthology.orchestra.run.smithi143.stdout:creating /etc/ceph/ceph.client.0.keyring 2023-11-26T07:16:54.357 DEBUG:teuthology.orchestra.run.smithi143:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=client.2 /etc/ceph/ceph.client.2.keyring && sudo chmod 0644 /etc/ceph/ceph.client.2.keyring 2023-11-26T07:16:54.444 INFO:teuthology.orchestra.run.smithi143.stdout:creating /etc/ceph/ceph.client.2.keyring 2023-11-26T07:16:54.460 DEBUG:teuthology.orchestra.run.smithi188:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=client.1 /etc/ceph/ceph.client.1.keyring && sudo chmod 0644 /etc/ceph/ceph.client.1.keyring 2023-11-26T07:16:54.503 INFO:teuthology.orchestra.run.smithi188.stdout:creating /etc/ceph/ceph.client.1.keyring 2023-11-26T07:16:54.520 DEBUG:teuthology.orchestra.run.smithi188:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=client.3 /etc/ceph/ceph.client.3.keyring && sudo chmod 0644 /etc/ceph/ceph.client.3.keyring 2023-11-26T07:16:54.608 INFO:teuthology.orchestra.run.smithi188.stdout:creating /etc/ceph/ceph.client.3.keyring 2023-11-26T07:16:54.625 INFO:tasks.ceph:Running mkfs on osd nodes... 2023-11-26T07:16:54.625 INFO:tasks.ceph:ctx.disk_config.remote_to_roles_to_dev: {Remote(name='ubuntu@smithi143.front.sepia.ceph.com'): {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'}, Remote(name='ubuntu@smithi188.front.sepia.ceph.com'): {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'}} 2023-11-26T07:16:54.625 DEBUG:teuthology.orchestra.run.smithi143:> sudo mkdir -p /var/lib/ceph/osd/ceph-0 2023-11-26T07:16:54.645 INFO:tasks.ceph:roles_to_devs: {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'} 2023-11-26T07:16:54.645 INFO:tasks.ceph:role: osd.0 2023-11-26T07:16:54.645 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_1 on ubuntu@smithi143.front.sepia.ceph.com 2023-11-26T07:16:54.646 DEBUG:teuthology.orchestra.run.smithi143:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_1 2023-11-26T07:16:54.716 INFO:teuthology.orchestra.run.smithi143.stdout:meta-data=/dev/vg_nvme/lv_1 isize=2048 agcount=4, agsize=5859072 blks 2023-11-26T07:16:54.716 INFO:teuthology.orchestra.run.smithi143.stdout: = sectsz=512 attr=2, projid32bit=1 2023-11-26T07:16:54.717 INFO:teuthology.orchestra.run.smithi143.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2023-11-26T07:16:54.717 INFO:teuthology.orchestra.run.smithi143.stdout: = reflink=1 bigtime=0 inobtcount=0 2023-11-26T07:16:54.717 INFO:teuthology.orchestra.run.smithi143.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2023-11-26T07:16:54.717 INFO:teuthology.orchestra.run.smithi143.stdout: = sunit=0 swidth=0 blks 2023-11-26T07:16:54.717 INFO:teuthology.orchestra.run.smithi143.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2023-11-26T07:16:54.717 INFO:teuthology.orchestra.run.smithi143.stdout:log =internal log bsize=4096 blocks=11443, version=2 2023-11-26T07:16:54.718 INFO:teuthology.orchestra.run.smithi143.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2023-11-26T07:16:54.718 INFO:teuthology.orchestra.run.smithi143.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2023-11-26T07:16:55.186 INFO:teuthology.orchestra.run.smithi143.stdout:Discarding blocks...Done. 2023-11-26T07:16:55.188 INFO:tasks.ceph:mount /dev/vg_nvme/lv_1 on ubuntu@smithi143.front.sepia.ceph.com -o noatime 2023-11-26T07:16:55.188 DEBUG:teuthology.orchestra.run.smithi143:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_1 /var/lib/ceph/osd/ceph-0 2023-11-26T07:16:55.356 DEBUG:teuthology.orchestra.run.smithi143:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-0 2023-11-26T07:16:55.372 INFO:teuthology.orchestra.run.smithi143.stderr:sudo: /sbin/restorecon: command not found 2023-11-26T07:16:55.373 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-11-26T07:16:55.373 DEBUG:teuthology.orchestra.run.smithi143:> sudo mkdir -p /var/lib/ceph/osd/ceph-1 2023-11-26T07:16:55.441 INFO:tasks.ceph:roles_to_devs: {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'} 2023-11-26T07:16:55.441 INFO:tasks.ceph:role: osd.1 2023-11-26T07:16:55.441 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_2 on ubuntu@smithi143.front.sepia.ceph.com 2023-11-26T07:16:55.441 DEBUG:teuthology.orchestra.run.smithi143:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_2 2023-11-26T07:16:55.509 INFO:teuthology.orchestra.run.smithi143.stdout:meta-data=/dev/vg_nvme/lv_2 isize=2048 agcount=4, agsize=5859072 blks 2023-11-26T07:16:55.509 INFO:teuthology.orchestra.run.smithi143.stdout: = sectsz=512 attr=2, projid32bit=1 2023-11-26T07:16:55.509 INFO:teuthology.orchestra.run.smithi143.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2023-11-26T07:16:55.509 INFO:teuthology.orchestra.run.smithi143.stdout: = reflink=1 bigtime=0 inobtcount=0 2023-11-26T07:16:55.509 INFO:teuthology.orchestra.run.smithi143.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2023-11-26T07:16:55.509 INFO:teuthology.orchestra.run.smithi143.stdout: = sunit=0 swidth=0 blks 2023-11-26T07:16:55.509 INFO:teuthology.orchestra.run.smithi143.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2023-11-26T07:16:55.509 INFO:teuthology.orchestra.run.smithi143.stdout:log =internal log bsize=4096 blocks=11443, version=2 2023-11-26T07:16:55.509 INFO:teuthology.orchestra.run.smithi143.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2023-11-26T07:16:55.509 INFO:teuthology.orchestra.run.smithi143.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2023-11-26T07:16:55.944 INFO:teuthology.orchestra.run.smithi143.stdout:Discarding blocks...Done. 2023-11-26T07:16:55.947 INFO:tasks.ceph:mount /dev/vg_nvme/lv_2 on ubuntu@smithi143.front.sepia.ceph.com -o noatime 2023-11-26T07:16:55.947 DEBUG:teuthology.orchestra.run.smithi143:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_2 /var/lib/ceph/osd/ceph-1 2023-11-26T07:16:56.039 DEBUG:teuthology.orchestra.run.smithi143:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-1 2023-11-26T07:16:56.055 INFO:teuthology.orchestra.run.smithi143.stderr:sudo: /sbin/restorecon: command not found 2023-11-26T07:16:56.056 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-11-26T07:16:56.056 DEBUG:teuthology.orchestra.run.smithi143:> sudo mkdir -p /var/lib/ceph/osd/ceph-2 2023-11-26T07:16:56.117 INFO:tasks.ceph:roles_to_devs: {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'} 2023-11-26T07:16:56.117 INFO:tasks.ceph:role: osd.2 2023-11-26T07:16:56.117 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_3 on ubuntu@smithi143.front.sepia.ceph.com 2023-11-26T07:16:56.117 DEBUG:teuthology.orchestra.run.smithi143:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_3 2023-11-26T07:16:56.185 INFO:teuthology.orchestra.run.smithi143.stdout:meta-data=/dev/vg_nvme/lv_3 isize=2048 agcount=4, agsize=5859072 blks 2023-11-26T07:16:56.185 INFO:teuthology.orchestra.run.smithi143.stdout: = sectsz=512 attr=2, projid32bit=1 2023-11-26T07:16:56.185 INFO:teuthology.orchestra.run.smithi143.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2023-11-26T07:16:56.185 INFO:teuthology.orchestra.run.smithi143.stdout: = reflink=1 bigtime=0 inobtcount=0 2023-11-26T07:16:56.185 INFO:teuthology.orchestra.run.smithi143.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2023-11-26T07:16:56.185 INFO:teuthology.orchestra.run.smithi143.stdout: = sunit=0 swidth=0 blks 2023-11-26T07:16:56.185 INFO:teuthology.orchestra.run.smithi143.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2023-11-26T07:16:56.185 INFO:teuthology.orchestra.run.smithi143.stdout:log =internal log bsize=4096 blocks=11443, version=2 2023-11-26T07:16:56.185 INFO:teuthology.orchestra.run.smithi143.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2023-11-26T07:16:56.186 INFO:teuthology.orchestra.run.smithi143.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2023-11-26T07:16:56.654 INFO:teuthology.orchestra.run.smithi143.stdout:Discarding blocks...Done. 2023-11-26T07:16:56.657 INFO:tasks.ceph:mount /dev/vg_nvme/lv_3 on ubuntu@smithi143.front.sepia.ceph.com -o noatime 2023-11-26T07:16:56.657 DEBUG:teuthology.orchestra.run.smithi143:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_3 /var/lib/ceph/osd/ceph-2 2023-11-26T07:16:56.810 DEBUG:teuthology.orchestra.run.smithi143:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-2 2023-11-26T07:16:56.827 INFO:teuthology.orchestra.run.smithi143.stderr:sudo: /sbin/restorecon: command not found 2023-11-26T07:16:56.828 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-11-26T07:16:56.828 DEBUG:teuthology.orchestra.run.smithi143:> sudo mkdir -p /var/lib/ceph/osd/ceph-3 2023-11-26T07:16:56.894 INFO:tasks.ceph:roles_to_devs: {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'} 2023-11-26T07:16:56.894 INFO:tasks.ceph:role: osd.3 2023-11-26T07:16:56.894 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_4 on ubuntu@smithi143.front.sepia.ceph.com 2023-11-26T07:16:56.894 DEBUG:teuthology.orchestra.run.smithi143:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_4 2023-11-26T07:16:56.967 INFO:teuthology.orchestra.run.smithi143.stdout:meta-data=/dev/vg_nvme/lv_4 isize=2048 agcount=4, agsize=5859072 blks 2023-11-26T07:16:56.967 INFO:teuthology.orchestra.run.smithi143.stdout: = sectsz=512 attr=2, projid32bit=1 2023-11-26T07:16:56.967 INFO:teuthology.orchestra.run.smithi143.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2023-11-26T07:16:56.967 INFO:teuthology.orchestra.run.smithi143.stdout: = reflink=1 bigtime=0 inobtcount=0 2023-11-26T07:16:56.967 INFO:teuthology.orchestra.run.smithi143.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2023-11-26T07:16:56.967 INFO:teuthology.orchestra.run.smithi143.stdout: = sunit=0 swidth=0 blks 2023-11-26T07:16:56.967 INFO:teuthology.orchestra.run.smithi143.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2023-11-26T07:16:56.967 INFO:teuthology.orchestra.run.smithi143.stdout:log =internal log bsize=4096 blocks=11443, version=2 2023-11-26T07:16:56.967 INFO:teuthology.orchestra.run.smithi143.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2023-11-26T07:16:56.968 INFO:teuthology.orchestra.run.smithi143.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2023-11-26T07:16:57.468 INFO:teuthology.orchestra.run.smithi143.stdout:Discarding blocks...Done. 2023-11-26T07:16:57.471 INFO:tasks.ceph:mount /dev/vg_nvme/lv_4 on ubuntu@smithi143.front.sepia.ceph.com -o noatime 2023-11-26T07:16:57.471 DEBUG:teuthology.orchestra.run.smithi143:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_4 /var/lib/ceph/osd/ceph-3 2023-11-26T07:16:57.616 DEBUG:teuthology.orchestra.run.smithi143:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-3 2023-11-26T07:16:57.634 INFO:teuthology.orchestra.run.smithi143.stderr:sudo: /sbin/restorecon: command not found 2023-11-26T07:16:57.635 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-11-26T07:16:57.635 DEBUG:teuthology.orchestra.run.smithi143:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 0 --monmap /home/ubuntu/cephtest/ceph.monmap 2023-11-26T07:16:57.770 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:16:57.770+0000 7f5c8d85b640 -1 auth: error reading file: /var/lib/ceph/osd/ceph-0/keyring: can't open /var/lib/ceph/osd/ceph-0/keyring: (2) No such file or directory 2023-11-26T07:16:57.774 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:16:57.770+0000 7f5c8d85b640 -1 created new key in keyring /var/lib/ceph/osd/ceph-0/keyring 2023-11-26T07:16:57.774 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:16:57.770+0000 7f5c8d85b640 -1 bluestore(/var/lib/ceph/osd/ceph-0/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-0/block: (2) No such file or directory 2023-11-26T07:16:57.774 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:16:57.770+0000 7f5c8d85b640 -1 bluestore(/var/lib/ceph/osd/ceph-0/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-0/block: (2) No such file or directory 2023-11-26T07:16:57.774 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:16:57.770+0000 7f5c8d85b640 -1 bluestore(/var/lib/ceph/osd/ceph-0/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-0/block: (2) No such file or directory 2023-11-26T07:16:57.777 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:16:57.774+0000 7f5c8d85b640 -1 bluestore(/var/lib/ceph/osd/ceph-0) _read_fsid unparsable uuid 2023-11-26T07:17:00.759 DEBUG:teuthology.orchestra.run.smithi143:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-0 2023-11-26T07:17:00.779 DEBUG:teuthology.orchestra.run.smithi143:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 1 --monmap /home/ubuntu/cephtest/ceph.monmap 2023-11-26T07:17:00.880 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:17:00.878+0000 7fa363c2b640 -1 auth: error reading file: /var/lib/ceph/osd/ceph-1/keyring: can't open /var/lib/ceph/osd/ceph-1/keyring: (2) No such file or directory 2023-11-26T07:17:00.881 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:17:00.878+0000 7fa363c2b640 -1 created new key in keyring /var/lib/ceph/osd/ceph-1/keyring 2023-11-26T07:17:00.881 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:17:00.878+0000 7fa363c2b640 -1 bluestore(/var/lib/ceph/osd/ceph-1/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-1/block: (2) No such file or directory 2023-11-26T07:17:00.881 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:17:00.878+0000 7fa363c2b640 -1 bluestore(/var/lib/ceph/osd/ceph-1/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-1/block: (2) No such file or directory 2023-11-26T07:17:00.881 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:17:00.878+0000 7fa363c2b640 -1 bluestore(/var/lib/ceph/osd/ceph-1/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-1/block: (2) No such file or directory 2023-11-26T07:17:00.882 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:17:00.878+0000 7fa363c2b640 -1 bluestore(/var/lib/ceph/osd/ceph-1) _read_fsid unparsable uuid 2023-11-26T07:17:03.862 DEBUG:teuthology.orchestra.run.smithi143:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-1 2023-11-26T07:17:03.882 DEBUG:teuthology.orchestra.run.smithi143:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 2 --monmap /home/ubuntu/cephtest/ceph.monmap 2023-11-26T07:17:03.983 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:17:03.981+0000 7f8127101640 -1 auth: error reading file: /var/lib/ceph/osd/ceph-2/keyring: can't open /var/lib/ceph/osd/ceph-2/keyring: (2) No such file or directory 2023-11-26T07:17:03.984 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:17:03.981+0000 7f8127101640 -1 created new key in keyring /var/lib/ceph/osd/ceph-2/keyring 2023-11-26T07:17:03.984 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:17:03.981+0000 7f8127101640 -1 bluestore(/var/lib/ceph/osd/ceph-2/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-2/block: (2) No such file or directory 2023-11-26T07:17:03.984 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:17:03.981+0000 7f8127101640 -1 bluestore(/var/lib/ceph/osd/ceph-2/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-2/block: (2) No such file or directory 2023-11-26T07:17:03.985 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:17:03.981+0000 7f8127101640 -1 bluestore(/var/lib/ceph/osd/ceph-2/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-2/block: (2) No such file or directory 2023-11-26T07:17:03.985 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:17:03.981+0000 7f8127101640 -1 bluestore(/var/lib/ceph/osd/ceph-2) _read_fsid unparsable uuid 2023-11-26T07:17:06.997 DEBUG:teuthology.orchestra.run.smithi143:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-2 2023-11-26T07:17:07.019 DEBUG:teuthology.orchestra.run.smithi143:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 3 --monmap /home/ubuntu/cephtest/ceph.monmap 2023-11-26T07:17:07.120 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:17:07.117+0000 7fa8f48b6640 -1 auth: error reading file: /var/lib/ceph/osd/ceph-3/keyring: can't open /var/lib/ceph/osd/ceph-3/keyring: (2) No such file or directory 2023-11-26T07:17:07.121 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:17:07.117+0000 7fa8f48b6640 -1 created new key in keyring /var/lib/ceph/osd/ceph-3/keyring 2023-11-26T07:17:07.122 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:17:07.117+0000 7fa8f48b6640 -1 bluestore(/var/lib/ceph/osd/ceph-3/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-3/block: (2) No such file or directory 2023-11-26T07:17:07.122 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:17:07.117+0000 7fa8f48b6640 -1 bluestore(/var/lib/ceph/osd/ceph-3/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-3/block: (2) No such file or directory 2023-11-26T07:17:07.122 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:17:07.117+0000 7fa8f48b6640 -1 bluestore(/var/lib/ceph/osd/ceph-3/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-3/block: (2) No such file or directory 2023-11-26T07:17:07.123 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:17:07.121+0000 7fa8f48b6640 -1 bluestore(/var/lib/ceph/osd/ceph-3) _read_fsid unparsable uuid 2023-11-26T07:17:10.114 DEBUG:teuthology.orchestra.run.smithi143:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-3 2023-11-26T07:17:10.134 DEBUG:teuthology.orchestra.run.smithi188:> sudo mkdir -p /var/lib/ceph/osd/ceph-4 2023-11-26T07:17:10.157 INFO:tasks.ceph:roles_to_devs: {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'} 2023-11-26T07:17:10.157 INFO:tasks.ceph:role: osd.4 2023-11-26T07:17:10.157 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_1 on ubuntu@smithi188.front.sepia.ceph.com 2023-11-26T07:17:10.157 DEBUG:teuthology.orchestra.run.smithi188:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_1 2023-11-26T07:17:10.233 INFO:teuthology.orchestra.run.smithi188.stdout:meta-data=/dev/vg_nvme/lv_1 isize=2048 agcount=4, agsize=5859072 blks 2023-11-26T07:17:10.233 INFO:teuthology.orchestra.run.smithi188.stdout: = sectsz=512 attr=2, projid32bit=1 2023-11-26T07:17:10.233 INFO:teuthology.orchestra.run.smithi188.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2023-11-26T07:17:10.233 INFO:teuthology.orchestra.run.smithi188.stdout: = reflink=1 bigtime=0 inobtcount=0 2023-11-26T07:17:10.233 INFO:teuthology.orchestra.run.smithi188.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2023-11-26T07:17:10.233 INFO:teuthology.orchestra.run.smithi188.stdout: = sunit=0 swidth=0 blks 2023-11-26T07:17:10.233 INFO:teuthology.orchestra.run.smithi188.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2023-11-26T07:17:10.233 INFO:teuthology.orchestra.run.smithi188.stdout:log =internal log bsize=4096 blocks=11443, version=2 2023-11-26T07:17:10.233 INFO:teuthology.orchestra.run.smithi188.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2023-11-26T07:17:10.234 INFO:teuthology.orchestra.run.smithi188.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2023-11-26T07:17:10.692 INFO:teuthology.orchestra.run.smithi188.stdout:Discarding blocks...Done. 2023-11-26T07:17:10.696 INFO:tasks.ceph:mount /dev/vg_nvme/lv_1 on ubuntu@smithi188.front.sepia.ceph.com -o noatime 2023-11-26T07:17:10.696 DEBUG:teuthology.orchestra.run.smithi188:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_1 /var/lib/ceph/osd/ceph-4 2023-11-26T07:17:10.825 DEBUG:teuthology.orchestra.run.smithi188:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-4 2023-11-26T07:17:10.843 INFO:teuthology.orchestra.run.smithi188.stderr:sudo: /sbin/restorecon: command not found 2023-11-26T07:17:10.844 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-11-26T07:17:10.844 DEBUG:teuthology.orchestra.run.smithi188:> sudo mkdir -p /var/lib/ceph/osd/ceph-5 2023-11-26T07:17:10.913 INFO:tasks.ceph:roles_to_devs: {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'} 2023-11-26T07:17:10.913 INFO:tasks.ceph:role: osd.5 2023-11-26T07:17:10.913 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_2 on ubuntu@smithi188.front.sepia.ceph.com 2023-11-26T07:17:10.913 DEBUG:teuthology.orchestra.run.smithi188:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_2 2023-11-26T07:17:10.985 INFO:teuthology.orchestra.run.smithi188.stdout:meta-data=/dev/vg_nvme/lv_2 isize=2048 agcount=4, agsize=5859072 blks 2023-11-26T07:17:10.985 INFO:teuthology.orchestra.run.smithi188.stdout: = sectsz=512 attr=2, projid32bit=1 2023-11-26T07:17:10.985 INFO:teuthology.orchestra.run.smithi188.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2023-11-26T07:17:10.985 INFO:teuthology.orchestra.run.smithi188.stdout: = reflink=1 bigtime=0 inobtcount=0 2023-11-26T07:17:10.986 INFO:teuthology.orchestra.run.smithi188.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2023-11-26T07:17:10.986 INFO:teuthology.orchestra.run.smithi188.stdout: = sunit=0 swidth=0 blks 2023-11-26T07:17:10.986 INFO:teuthology.orchestra.run.smithi188.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2023-11-26T07:17:10.986 INFO:teuthology.orchestra.run.smithi188.stdout:log =internal log bsize=4096 blocks=11443, version=2 2023-11-26T07:17:10.986 INFO:teuthology.orchestra.run.smithi188.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2023-11-26T07:17:10.986 INFO:teuthology.orchestra.run.smithi188.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2023-11-26T07:17:11.458 INFO:teuthology.orchestra.run.smithi188.stdout:Discarding blocks...Done. 2023-11-26T07:17:11.461 INFO:tasks.ceph:mount /dev/vg_nvme/lv_2 on ubuntu@smithi188.front.sepia.ceph.com -o noatime 2023-11-26T07:17:11.461 DEBUG:teuthology.orchestra.run.smithi188:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_2 /var/lib/ceph/osd/ceph-5 2023-11-26T07:17:11.589 DEBUG:teuthology.orchestra.run.smithi188:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-5 2023-11-26T07:17:11.608 INFO:teuthology.orchestra.run.smithi188.stderr:sudo: /sbin/restorecon: command not found 2023-11-26T07:17:11.608 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-11-26T07:17:11.609 DEBUG:teuthology.orchestra.run.smithi188:> sudo mkdir -p /var/lib/ceph/osd/ceph-6 2023-11-26T07:17:11.676 INFO:tasks.ceph:roles_to_devs: {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'} 2023-11-26T07:17:11.676 INFO:tasks.ceph:role: osd.6 2023-11-26T07:17:11.677 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_3 on ubuntu@smithi188.front.sepia.ceph.com 2023-11-26T07:17:11.677 DEBUG:teuthology.orchestra.run.smithi188:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_3 2023-11-26T07:17:11.752 INFO:teuthology.orchestra.run.smithi188.stdout:meta-data=/dev/vg_nvme/lv_3 isize=2048 agcount=4, agsize=5859072 blks 2023-11-26T07:17:11.752 INFO:teuthology.orchestra.run.smithi188.stdout: = sectsz=512 attr=2, projid32bit=1 2023-11-26T07:17:11.752 INFO:teuthology.orchestra.run.smithi188.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2023-11-26T07:17:11.753 INFO:teuthology.orchestra.run.smithi188.stdout: = reflink=1 bigtime=0 inobtcount=0 2023-11-26T07:17:11.753 INFO:teuthology.orchestra.run.smithi188.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2023-11-26T07:17:11.753 INFO:teuthology.orchestra.run.smithi188.stdout: = sunit=0 swidth=0 blks 2023-11-26T07:17:11.753 INFO:teuthology.orchestra.run.smithi188.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2023-11-26T07:17:11.753 INFO:teuthology.orchestra.run.smithi188.stdout:log =internal log bsize=4096 blocks=11443, version=2 2023-11-26T07:17:11.753 INFO:teuthology.orchestra.run.smithi188.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2023-11-26T07:17:11.754 INFO:teuthology.orchestra.run.smithi188.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2023-11-26T07:17:12.231 INFO:teuthology.orchestra.run.smithi188.stdout:Discarding blocks...Done. 2023-11-26T07:17:12.234 INFO:tasks.ceph:mount /dev/vg_nvme/lv_3 on ubuntu@smithi188.front.sepia.ceph.com -o noatime 2023-11-26T07:17:12.235 DEBUG:teuthology.orchestra.run.smithi188:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_3 /var/lib/ceph/osd/ceph-6 2023-11-26T07:17:12.357 DEBUG:teuthology.orchestra.run.smithi188:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-6 2023-11-26T07:17:12.374 INFO:teuthology.orchestra.run.smithi188.stderr:sudo: /sbin/restorecon: command not found 2023-11-26T07:17:12.375 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-11-26T07:17:12.375 DEBUG:teuthology.orchestra.run.smithi188:> sudo mkdir -p /var/lib/ceph/osd/ceph-7 2023-11-26T07:17:12.436 INFO:tasks.ceph:roles_to_devs: {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'} 2023-11-26T07:17:12.436 INFO:tasks.ceph:role: osd.7 2023-11-26T07:17:12.436 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_4 on ubuntu@smithi188.front.sepia.ceph.com 2023-11-26T07:17:12.436 DEBUG:teuthology.orchestra.run.smithi188:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_4 2023-11-26T07:17:12.508 INFO:teuthology.orchestra.run.smithi188.stdout:meta-data=/dev/vg_nvme/lv_4 isize=2048 agcount=4, agsize=5859072 blks 2023-11-26T07:17:12.508 INFO:teuthology.orchestra.run.smithi188.stdout: = sectsz=512 attr=2, projid32bit=1 2023-11-26T07:17:12.508 INFO:teuthology.orchestra.run.smithi188.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2023-11-26T07:17:12.508 INFO:teuthology.orchestra.run.smithi188.stdout: = reflink=1 bigtime=0 inobtcount=0 2023-11-26T07:17:12.508 INFO:teuthology.orchestra.run.smithi188.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2023-11-26T07:17:12.508 INFO:teuthology.orchestra.run.smithi188.stdout: = sunit=0 swidth=0 blks 2023-11-26T07:17:12.508 INFO:teuthology.orchestra.run.smithi188.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2023-11-26T07:17:12.509 INFO:teuthology.orchestra.run.smithi188.stdout:log =internal log bsize=4096 blocks=11443, version=2 2023-11-26T07:17:12.509 INFO:teuthology.orchestra.run.smithi188.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2023-11-26T07:17:12.509 INFO:teuthology.orchestra.run.smithi188.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2023-11-26T07:17:12.955 INFO:teuthology.orchestra.run.smithi188.stdout:Discarding blocks...Done. 2023-11-26T07:17:12.958 INFO:tasks.ceph:mount /dev/vg_nvme/lv_4 on ubuntu@smithi188.front.sepia.ceph.com -o noatime 2023-11-26T07:17:12.958 DEBUG:teuthology.orchestra.run.smithi188:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_4 /var/lib/ceph/osd/ceph-7 2023-11-26T07:17:13.066 DEBUG:teuthology.orchestra.run.smithi188:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-7 2023-11-26T07:17:13.083 INFO:teuthology.orchestra.run.smithi188.stderr:sudo: /sbin/restorecon: command not found 2023-11-26T07:17:13.083 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-11-26T07:17:13.084 DEBUG:teuthology.orchestra.run.smithi188:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 4 --monmap /home/ubuntu/cephtest/ceph.monmap 2023-11-26T07:17:13.208 INFO:teuthology.orchestra.run.smithi188.stderr:2023-11-26T07:17:13.204+0000 7f8c5424a640 -1 auth: error reading file: /var/lib/ceph/osd/ceph-4/keyring: can't open /var/lib/ceph/osd/ceph-4/keyring: (2) No such file or directory 2023-11-26T07:17:13.212 INFO:teuthology.orchestra.run.smithi188.stderr:2023-11-26T07:17:13.204+0000 7f8c5424a640 -1 created new key in keyring /var/lib/ceph/osd/ceph-4/keyring 2023-11-26T07:17:13.212 INFO:teuthology.orchestra.run.smithi188.stderr:2023-11-26T07:17:13.204+0000 7f8c5424a640 -1 bluestore(/var/lib/ceph/osd/ceph-4/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-4/block: (2) No such file or directory 2023-11-26T07:17:13.212 INFO:teuthology.orchestra.run.smithi188.stderr:2023-11-26T07:17:13.204+0000 7f8c5424a640 -1 bluestore(/var/lib/ceph/osd/ceph-4/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-4/block: (2) No such file or directory 2023-11-26T07:17:13.212 INFO:teuthology.orchestra.run.smithi188.stderr:2023-11-26T07:17:13.204+0000 7f8c5424a640 -1 bluestore(/var/lib/ceph/osd/ceph-4/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-4/block: (2) No such file or directory 2023-11-26T07:17:13.215 INFO:teuthology.orchestra.run.smithi188.stderr:2023-11-26T07:17:13.208+0000 7f8c5424a640 -1 bluestore(/var/lib/ceph/osd/ceph-4) _read_fsid unparsable uuid 2023-11-26T07:17:16.205 DEBUG:teuthology.orchestra.run.smithi188:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-4 2023-11-26T07:17:16.226 DEBUG:teuthology.orchestra.run.smithi188:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 5 --monmap /home/ubuntu/cephtest/ceph.monmap 2023-11-26T07:17:16.321 INFO:teuthology.orchestra.run.smithi188.stderr:2023-11-26T07:17:16.316+0000 7ff7ba3b6640 -1 auth: error reading file: /var/lib/ceph/osd/ceph-5/keyring: can't open /var/lib/ceph/osd/ceph-5/keyring: (2) No such file or directory 2023-11-26T07:17:16.322 INFO:teuthology.orchestra.run.smithi188.stderr:2023-11-26T07:17:16.316+0000 7ff7ba3b6640 -1 created new key in keyring /var/lib/ceph/osd/ceph-5/keyring 2023-11-26T07:17:16.323 INFO:teuthology.orchestra.run.smithi188.stderr:2023-11-26T07:17:16.316+0000 7ff7ba3b6640 -1 bluestore(/var/lib/ceph/osd/ceph-5/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-5/block: (2) No such file or directory 2023-11-26T07:17:16.323 INFO:teuthology.orchestra.run.smithi188.stderr:2023-11-26T07:17:16.316+0000 7ff7ba3b6640 -1 bluestore(/var/lib/ceph/osd/ceph-5/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-5/block: (2) No such file or directory 2023-11-26T07:17:16.323 INFO:teuthology.orchestra.run.smithi188.stderr:2023-11-26T07:17:16.316+0000 7ff7ba3b6640 -1 bluestore(/var/lib/ceph/osd/ceph-5/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-5/block: (2) No such file or directory 2023-11-26T07:17:16.324 INFO:teuthology.orchestra.run.smithi188.stderr:2023-11-26T07:17:16.316+0000 7ff7ba3b6640 -1 bluestore(/var/lib/ceph/osd/ceph-5) _read_fsid unparsable uuid 2023-11-26T07:17:19.308 DEBUG:teuthology.orchestra.run.smithi188:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-5 2023-11-26T07:17:19.329 DEBUG:teuthology.orchestra.run.smithi188:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 6 --monmap /home/ubuntu/cephtest/ceph.monmap 2023-11-26T07:17:19.425 INFO:teuthology.orchestra.run.smithi188.stderr:2023-11-26T07:17:19.420+0000 7f55d2b05640 -1 auth: error reading file: /var/lib/ceph/osd/ceph-6/keyring: can't open /var/lib/ceph/osd/ceph-6/keyring: (2) No such file or directory 2023-11-26T07:17:19.427 INFO:teuthology.orchestra.run.smithi188.stderr:2023-11-26T07:17:19.420+0000 7f55d2b05640 -1 created new key in keyring /var/lib/ceph/osd/ceph-6/keyring 2023-11-26T07:17:19.427 INFO:teuthology.orchestra.run.smithi188.stderr:2023-11-26T07:17:19.420+0000 7f55d2b05640 -1 bluestore(/var/lib/ceph/osd/ceph-6/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-6/block: (2) No such file or directory 2023-11-26T07:17:19.427 INFO:teuthology.orchestra.run.smithi188.stderr:2023-11-26T07:17:19.420+0000 7f55d2b05640 -1 bluestore(/var/lib/ceph/osd/ceph-6/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-6/block: (2) No such file or directory 2023-11-26T07:17:19.427 INFO:teuthology.orchestra.run.smithi188.stderr:2023-11-26T07:17:19.420+0000 7f55d2b05640 -1 bluestore(/var/lib/ceph/osd/ceph-6/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-6/block: (2) No such file or directory 2023-11-26T07:17:19.428 INFO:teuthology.orchestra.run.smithi188.stderr:2023-11-26T07:17:19.420+0000 7f55d2b05640 -1 bluestore(/var/lib/ceph/osd/ceph-6) _read_fsid unparsable uuid 2023-11-26T07:17:21.430 DEBUG:teuthology.orchestra.run.smithi143:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-26T07:17:21.434 DEBUG:teuthology.orchestra.run.smithi188:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-26T07:17:22.433 DEBUG:teuthology.orchestra.run.smithi188:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-6 2023-11-26T07:17:22.456 DEBUG:teuthology.orchestra.run.smithi188:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 7 --monmap /home/ubuntu/cephtest/ceph.monmap 2023-11-26T07:17:22.553 INFO:teuthology.orchestra.run.smithi188.stderr:2023-11-26T07:17:22.548+0000 7fc699f57640 -1 auth: error reading file: /var/lib/ceph/osd/ceph-7/keyring: can't open /var/lib/ceph/osd/ceph-7/keyring: (2) No such file or directory 2023-11-26T07:17:22.555 INFO:teuthology.orchestra.run.smithi188.stderr:2023-11-26T07:17:22.548+0000 7fc699f57640 -1 created new key in keyring /var/lib/ceph/osd/ceph-7/keyring 2023-11-26T07:17:22.555 INFO:teuthology.orchestra.run.smithi188.stderr:2023-11-26T07:17:22.548+0000 7fc699f57640 -1 bluestore(/var/lib/ceph/osd/ceph-7/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-7/block: (2) No such file or directory 2023-11-26T07:17:22.555 INFO:teuthology.orchestra.run.smithi188.stderr:2023-11-26T07:17:22.548+0000 7fc699f57640 -1 bluestore(/var/lib/ceph/osd/ceph-7/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-7/block: (2) No such file or directory 2023-11-26T07:17:22.555 INFO:teuthology.orchestra.run.smithi188.stderr:2023-11-26T07:17:22.548+0000 7fc699f57640 -1 bluestore(/var/lib/ceph/osd/ceph-7/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-7/block: (2) No such file or directory 2023-11-26T07:17:22.556 INFO:teuthology.orchestra.run.smithi188.stderr:2023-11-26T07:17:22.548+0000 7fc699f57640 -1 bluestore(/var/lib/ceph/osd/ceph-7) _read_fsid unparsable uuid 2023-11-26T07:17:25.551 DEBUG:teuthology.orchestra.run.smithi188:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-7 2023-11-26T07:17:25.572 INFO:tasks.ceph:Reading keys from all nodes... 2023-11-26T07:17:25.572 DEBUG:teuthology.orchestra.run.smithi143:> set -ex 2023-11-26T07:17:25.572 DEBUG:teuthology.orchestra.run.smithi143:> sudo dd if=/var/lib/ceph/mgr/ceph-y/keyring of=/dev/stdout 2023-11-26T07:17:25.595 DEBUG:teuthology.orchestra.run.smithi143:> set -ex 2023-11-26T07:17:25.595 DEBUG:teuthology.orchestra.run.smithi143:> sudo dd if=/var/lib/ceph/mds/ceph-a/keyring of=/dev/stdout 2023-11-26T07:17:25.661 DEBUG:teuthology.orchestra.run.smithi143:> set -ex 2023-11-26T07:17:25.661 DEBUG:teuthology.orchestra.run.smithi143:> sudo dd if=/var/lib/ceph/mds/ceph-c/keyring of=/dev/stdout 2023-11-26T07:17:25.724 DEBUG:teuthology.orchestra.run.smithi143:> set -ex 2023-11-26T07:17:25.725 DEBUG:teuthology.orchestra.run.smithi143:> sudo dd if=/var/lib/ceph/osd/ceph-0/keyring of=/dev/stdout 2023-11-26T07:17:25.788 DEBUG:teuthology.orchestra.run.smithi143:> set -ex 2023-11-26T07:17:25.789 DEBUG:teuthology.orchestra.run.smithi143:> sudo dd if=/var/lib/ceph/osd/ceph-1/keyring of=/dev/stdout 2023-11-26T07:17:25.852 DEBUG:teuthology.orchestra.run.smithi143:> set -ex 2023-11-26T07:17:25.852 DEBUG:teuthology.orchestra.run.smithi143:> sudo dd if=/var/lib/ceph/osd/ceph-2/keyring of=/dev/stdout 2023-11-26T07:17:25.917 DEBUG:teuthology.orchestra.run.smithi143:> set -ex 2023-11-26T07:17:25.917 DEBUG:teuthology.orchestra.run.smithi143:> sudo dd if=/var/lib/ceph/osd/ceph-3/keyring of=/dev/stdout 2023-11-26T07:17:25.980 DEBUG:teuthology.orchestra.run.smithi188:> set -ex 2023-11-26T07:17:25.981 DEBUG:teuthology.orchestra.run.smithi188:> sudo dd if=/var/lib/ceph/mgr/ceph-x/keyring of=/dev/stdout 2023-11-26T07:17:26.005 DEBUG:teuthology.orchestra.run.smithi188:> set -ex 2023-11-26T07:17:26.005 DEBUG:teuthology.orchestra.run.smithi188:> sudo dd if=/var/lib/ceph/mds/ceph-b/keyring of=/dev/stdout 2023-11-26T07:17:26.068 DEBUG:teuthology.orchestra.run.smithi188:> set -ex 2023-11-26T07:17:26.068 DEBUG:teuthology.orchestra.run.smithi188:> sudo dd if=/var/lib/ceph/mds/ceph-d/keyring of=/dev/stdout 2023-11-26T07:17:26.132 DEBUG:teuthology.orchestra.run.smithi188:> set -ex 2023-11-26T07:17:26.133 DEBUG:teuthology.orchestra.run.smithi188:> sudo dd if=/var/lib/ceph/osd/ceph-4/keyring of=/dev/stdout 2023-11-26T07:17:26.200 DEBUG:teuthology.orchestra.run.smithi188:> set -ex 2023-11-26T07:17:26.200 DEBUG:teuthology.orchestra.run.smithi188:> sudo dd if=/var/lib/ceph/osd/ceph-5/keyring of=/dev/stdout 2023-11-26T07:17:26.264 DEBUG:teuthology.orchestra.run.smithi188:> set -ex 2023-11-26T07:17:26.264 DEBUG:teuthology.orchestra.run.smithi188:> sudo dd if=/var/lib/ceph/osd/ceph-6/keyring of=/dev/stdout 2023-11-26T07:17:26.328 DEBUG:teuthology.orchestra.run.smithi188:> set -ex 2023-11-26T07:17:26.328 DEBUG:teuthology.orchestra.run.smithi188:> sudo dd if=/var/lib/ceph/osd/ceph-7/keyring of=/dev/stdout 2023-11-26T07:17:26.387 DEBUG:teuthology.orchestra.run.smithi143:> set -ex 2023-11-26T07:17:26.388 DEBUG:teuthology.orchestra.run.smithi143:> dd if=/etc/ceph/ceph.client.0.keyring of=/dev/stdout 2023-11-26T07:17:26.397 DEBUG:teuthology.orchestra.run.smithi143:> set -ex 2023-11-26T07:17:26.397 DEBUG:teuthology.orchestra.run.smithi143:> dd if=/etc/ceph/ceph.client.2.keyring of=/dev/stdout 2023-11-26T07:17:26.452 DEBUG:teuthology.orchestra.run.smithi188:> set -ex 2023-11-26T07:17:26.452 DEBUG:teuthology.orchestra.run.smithi188:> dd if=/etc/ceph/ceph.client.1.keyring of=/dev/stdout 2023-11-26T07:17:26.460 DEBUG:teuthology.orchestra.run.smithi188:> set -ex 2023-11-26T07:17:26.461 DEBUG:teuthology.orchestra.run.smithi188:> dd if=/etc/ceph/ceph.client.3.keyring of=/dev/stdout 2023-11-26T07:17:26.512 INFO:tasks.ceph:Adding keys to all mons... 2023-11-26T07:17:26.512 DEBUG:teuthology.orchestra.run.smithi143:> sudo tee -a /etc/ceph/ceph.keyring 2023-11-26T07:17:26.516 DEBUG:teuthology.orchestra.run.smithi188:> sudo tee -a /etc/ceph/ceph.keyring 2023-11-26T07:17:26.560 INFO:teuthology.orchestra.run.smithi143.stdout:[mgr.y] 2023-11-26T07:17:26.560 INFO:teuthology.orchestra.run.smithi143.stdout: key = AQDl8GJlfuFPMBAA3TYEFvdfk4DyGsBkbpv0hg== 2023-11-26T07:17:26.560 INFO:teuthology.orchestra.run.smithi143.stdout:[mds.a] 2023-11-26T07:17:26.560 INFO:teuthology.orchestra.run.smithi143.stdout: key = AQDl8GJldeBsOBAAhk75XZnQRURKcr0MrW17Lg== 2023-11-26T07:17:26.560 INFO:teuthology.orchestra.run.smithi143.stdout:[mds.c] 2023-11-26T07:17:26.560 INFO:teuthology.orchestra.run.smithi143.stdout: key = AQDm8GJluQ0PBBAA23EmVdD3bVk52h6izUXDRQ== 2023-11-26T07:17:26.560 INFO:teuthology.orchestra.run.smithi143.stdout:[osd.0] 2023-11-26T07:17:26.561 INFO:teuthology.orchestra.run.smithi143.stdout: key = AQDp8GJlNscZLhAA31zK4vTcnxwiRIRgxzewpA== 2023-11-26T07:17:26.561 INFO:teuthology.orchestra.run.smithi143.stdout:[osd.1] 2023-11-26T07:17:26.561 INFO:teuthology.orchestra.run.smithi143.stdout: key = AQDs8GJlsyKUNBAA+dQ1ElGPrdP1GV7lk4JQjQ== 2023-11-26T07:17:26.561 INFO:teuthology.orchestra.run.smithi143.stdout:[osd.2] 2023-11-26T07:17:26.561 INFO:teuthology.orchestra.run.smithi143.stdout: key = AQDv8GJlXsy7OhAAJJdldsodgGZVtyV2HuyC4A== 2023-11-26T07:17:26.561 INFO:teuthology.orchestra.run.smithi143.stdout:[osd.3] 2023-11-26T07:17:26.561 INFO:teuthology.orchestra.run.smithi143.stdout: key = AQDz8GJloLhMBxAAOWD9ec2y4c8nkdJmQnne8Q== 2023-11-26T07:17:26.561 INFO:teuthology.orchestra.run.smithi143.stdout:[mgr.x] 2023-11-26T07:17:26.561 INFO:teuthology.orchestra.run.smithi143.stdout: key = AQDl8GJlXy8cNBAA4f5RDC1MASCqZPcJQvnUDA== 2023-11-26T07:17:26.561 INFO:teuthology.orchestra.run.smithi143.stdout:[mds.b] 2023-11-26T07:17:26.561 INFO:teuthology.orchestra.run.smithi143.stdout: key = AQDm8GJl0lvhCBAAXuvHKoB8TAhp0bJ4DcV94g== 2023-11-26T07:17:26.561 INFO:teuthology.orchestra.run.smithi143.stdout:[mds.d] 2023-11-26T07:17:26.561 INFO:teuthology.orchestra.run.smithi143.stdout: key = AQDm8GJlkt9nEBAAg+LYtKVNHAN/fwYjEcNJHQ== 2023-11-26T07:17:26.561 INFO:teuthology.orchestra.run.smithi143.stdout:[osd.4] 2023-11-26T07:17:26.562 INFO:teuthology.orchestra.run.smithi143.stdout: key = AQD58GJlDlZsDBAAZNauAPPPfAZ+KIqSmgSQhg== 2023-11-26T07:17:26.562 INFO:teuthology.orchestra.run.smithi143.stdout:[osd.5] 2023-11-26T07:17:26.562 INFO:teuthology.orchestra.run.smithi143.stdout: key = AQD88GJlNE4mExAA5jYkCa0tE3oR6rgK1Dt5cA== 2023-11-26T07:17:26.562 INFO:teuthology.orchestra.run.smithi143.stdout:[osd.6] 2023-11-26T07:17:26.562 INFO:teuthology.orchestra.run.smithi143.stdout: key = AQD/8GJlh0lYGRAArHalCyZ36UvBOVMZor+N/g== 2023-11-26T07:17:26.562 INFO:teuthology.orchestra.run.smithi143.stdout:[osd.7] 2023-11-26T07:17:26.562 INFO:teuthology.orchestra.run.smithi143.stdout: key = AQAC8WJlJavtIBAAKMvnR4AnnrDVOXlzN7j05w== 2023-11-26T07:17:26.562 INFO:teuthology.orchestra.run.smithi143.stdout:[client.0] 2023-11-26T07:17:26.562 INFO:teuthology.orchestra.run.smithi143.stdout: key = AQDm8GJlWMh9FBAAslVzi6fm4/wt7vr/rHCHHQ== 2023-11-26T07:17:26.562 INFO:teuthology.orchestra.run.smithi143.stdout:[client.2] 2023-11-26T07:17:26.562 INFO:teuthology.orchestra.run.smithi143.stdout: key = AQDm8GJlW4KlGhAANFKsyVJems0ZoE6bo3WmfQ== 2023-11-26T07:17:26.562 INFO:teuthology.orchestra.run.smithi143.stdout:[client.1] 2023-11-26T07:17:26.562 INFO:teuthology.orchestra.run.smithi143.stdout: key = AQDm8GJl5vkYHhAAgP+gKH2+lpl3aTlgqZNIXg== 2023-11-26T07:17:26.563 INFO:teuthology.orchestra.run.smithi143.stdout:[client.3] 2023-11-26T07:17:26.563 INFO:teuthology.orchestra.run.smithi143.stdout: key = AQDm8GJlAJZSJBAAUSUNcwRBY01hOuilBJjlFQ== 2023-11-26T07:17:26.574 INFO:teuthology.orchestra.run.smithi188.stdout:[mgr.y] 2023-11-26T07:17:26.574 INFO:teuthology.orchestra.run.smithi188.stdout: key = AQDl8GJlfuFPMBAA3TYEFvdfk4DyGsBkbpv0hg== 2023-11-26T07:17:26.574 INFO:teuthology.orchestra.run.smithi188.stdout:[mds.a] 2023-11-26T07:17:26.574 INFO:teuthology.orchestra.run.smithi188.stdout: key = AQDl8GJldeBsOBAAhk75XZnQRURKcr0MrW17Lg== 2023-11-26T07:17:26.574 INFO:teuthology.orchestra.run.smithi188.stdout:[mds.c] 2023-11-26T07:17:26.574 INFO:teuthology.orchestra.run.smithi188.stdout: key = AQDm8GJluQ0PBBAA23EmVdD3bVk52h6izUXDRQ== 2023-11-26T07:17:26.574 INFO:teuthology.orchestra.run.smithi188.stdout:[osd.0] 2023-11-26T07:17:26.574 INFO:teuthology.orchestra.run.smithi188.stdout: key = AQDp8GJlNscZLhAA31zK4vTcnxwiRIRgxzewpA== 2023-11-26T07:17:26.574 INFO:teuthology.orchestra.run.smithi188.stdout:[osd.1] 2023-11-26T07:17:26.575 INFO:teuthology.orchestra.run.smithi188.stdout: key = AQDs8GJlsyKUNBAA+dQ1ElGPrdP1GV7lk4JQjQ== 2023-11-26T07:17:26.575 INFO:teuthology.orchestra.run.smithi188.stdout:[osd.2] 2023-11-26T07:17:26.575 INFO:teuthology.orchestra.run.smithi188.stdout: key = AQDv8GJlXsy7OhAAJJdldsodgGZVtyV2HuyC4A== 2023-11-26T07:17:26.575 INFO:teuthology.orchestra.run.smithi188.stdout:[osd.3] 2023-11-26T07:17:26.575 INFO:teuthology.orchestra.run.smithi188.stdout: key = AQDz8GJloLhMBxAAOWD9ec2y4c8nkdJmQnne8Q== 2023-11-26T07:17:26.575 INFO:teuthology.orchestra.run.smithi188.stdout:[mgr.x] 2023-11-26T07:17:26.575 INFO:teuthology.orchestra.run.smithi188.stdout: key = AQDl8GJlXy8cNBAA4f5RDC1MASCqZPcJQvnUDA== 2023-11-26T07:17:26.575 INFO:teuthology.orchestra.run.smithi188.stdout:[mds.b] 2023-11-26T07:17:26.575 INFO:teuthology.orchestra.run.smithi188.stdout: key = AQDm8GJl0lvhCBAAXuvHKoB8TAhp0bJ4DcV94g== 2023-11-26T07:17:26.575 INFO:teuthology.orchestra.run.smithi188.stdout:[mds.d] 2023-11-26T07:17:26.575 INFO:teuthology.orchestra.run.smithi188.stdout: key = AQDm8GJlkt9nEBAAg+LYtKVNHAN/fwYjEcNJHQ== 2023-11-26T07:17:26.575 INFO:teuthology.orchestra.run.smithi188.stdout:[osd.4] 2023-11-26T07:17:26.575 INFO:teuthology.orchestra.run.smithi188.stdout: key = AQD58GJlDlZsDBAAZNauAPPPfAZ+KIqSmgSQhg== 2023-11-26T07:17:26.575 INFO:teuthology.orchestra.run.smithi188.stdout:[osd.5] 2023-11-26T07:17:26.575 INFO:teuthology.orchestra.run.smithi188.stdout: key = AQD88GJlNE4mExAA5jYkCa0tE3oR6rgK1Dt5cA== 2023-11-26T07:17:26.576 INFO:teuthology.orchestra.run.smithi188.stdout:[osd.6] 2023-11-26T07:17:26.576 INFO:teuthology.orchestra.run.smithi188.stdout: key = AQD/8GJlh0lYGRAArHalCyZ36UvBOVMZor+N/g== 2023-11-26T07:17:26.576 INFO:teuthology.orchestra.run.smithi188.stdout:[osd.7] 2023-11-26T07:17:26.576 INFO:teuthology.orchestra.run.smithi188.stdout: key = AQAC8WJlJavtIBAAKMvnR4AnnrDVOXlzN7j05w== 2023-11-26T07:17:26.576 INFO:teuthology.orchestra.run.smithi188.stdout:[client.0] 2023-11-26T07:17:26.576 INFO:teuthology.orchestra.run.smithi188.stdout: key = AQDm8GJlWMh9FBAAslVzi6fm4/wt7vr/rHCHHQ== 2023-11-26T07:17:26.576 INFO:teuthology.orchestra.run.smithi188.stdout:[client.2] 2023-11-26T07:17:26.576 INFO:teuthology.orchestra.run.smithi188.stdout: key = AQDm8GJlW4KlGhAANFKsyVJems0ZoE6bo3WmfQ== 2023-11-26T07:17:26.576 INFO:teuthology.orchestra.run.smithi188.stdout:[client.1] 2023-11-26T07:17:26.576 INFO:teuthology.orchestra.run.smithi188.stdout: key = AQDm8GJl5vkYHhAAgP+gKH2+lpl3aTlgqZNIXg== 2023-11-26T07:17:26.576 INFO:teuthology.orchestra.run.smithi188.stdout:[client.3] 2023-11-26T07:17:26.576 INFO:teuthology.orchestra.run.smithi188.stdout: key = AQDm8GJlAJZSJBAAUSUNcwRBY01hOuilBJjlFQ== 2023-11-26T07:17:26.577 DEBUG:teuthology.orchestra.run.smithi143:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mgr.y --cap mon 'allow profile mgr' --cap osd 'allow *' --cap mds 'allow *' 2023-11-26T07:17:26.611 DEBUG:teuthology.orchestra.run.smithi188:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mgr.y --cap mon 'allow profile mgr' --cap osd 'allow *' --cap mds 'allow *' 2023-11-26T07:17:26.675 DEBUG:teuthology.orchestra.run.smithi143:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.a --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2023-11-26T07:17:26.679 DEBUG:teuthology.orchestra.run.smithi188:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.a --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2023-11-26T07:17:26.737 DEBUG:teuthology.orchestra.run.smithi143:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.c --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2023-11-26T07:17:26.741 DEBUG:teuthology.orchestra.run.smithi188:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.c --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2023-11-26T07:17:26.799 DEBUG:teuthology.orchestra.run.smithi143:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.0 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2023-11-26T07:17:26.803 DEBUG:teuthology.orchestra.run.smithi188:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.0 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2023-11-26T07:17:26.858 DEBUG:teuthology.orchestra.run.smithi143:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.1 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2023-11-26T07:17:26.861 DEBUG:teuthology.orchestra.run.smithi188:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.1 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2023-11-26T07:17:26.916 DEBUG:teuthology.orchestra.run.smithi143:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.2 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2023-11-26T07:17:26.920 DEBUG:teuthology.orchestra.run.smithi188:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.2 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2023-11-26T07:17:26.975 DEBUG:teuthology.orchestra.run.smithi143:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.3 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2023-11-26T07:17:26.978 DEBUG:teuthology.orchestra.run.smithi188:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.3 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2023-11-26T07:17:27.033 DEBUG:teuthology.orchestra.run.smithi143:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mgr.x --cap mon 'allow profile mgr' --cap osd 'allow *' --cap mds 'allow *' 2023-11-26T07:17:27.037 DEBUG:teuthology.orchestra.run.smithi188:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mgr.x --cap mon 'allow profile mgr' --cap osd 'allow *' --cap mds 'allow *' 2023-11-26T07:17:27.092 DEBUG:teuthology.orchestra.run.smithi143:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.b --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2023-11-26T07:17:27.095 DEBUG:teuthology.orchestra.run.smithi188:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.b --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2023-11-26T07:17:27.152 DEBUG:teuthology.orchestra.run.smithi143:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.d --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2023-11-26T07:17:27.155 DEBUG:teuthology.orchestra.run.smithi188:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.d --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2023-11-26T07:17:27.212 DEBUG:teuthology.orchestra.run.smithi143:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.4 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2023-11-26T07:17:27.215 DEBUG:teuthology.orchestra.run.smithi188:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.4 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2023-11-26T07:17:27.270 DEBUG:teuthology.orchestra.run.smithi143:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.5 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2023-11-26T07:17:27.273 DEBUG:teuthology.orchestra.run.smithi188:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.5 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2023-11-26T07:17:27.328 DEBUG:teuthology.orchestra.run.smithi143:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.6 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2023-11-26T07:17:27.332 DEBUG:teuthology.orchestra.run.smithi188:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.6 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2023-11-26T07:17:27.386 DEBUG:teuthology.orchestra.run.smithi143:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.7 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2023-11-26T07:17:27.390 DEBUG:teuthology.orchestra.run.smithi188:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.7 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2023-11-26T07:17:27.445 DEBUG:teuthology.orchestra.run.smithi143:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.0 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2023-11-26T07:17:27.449 DEBUG:teuthology.orchestra.run.smithi188:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.0 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2023-11-26T07:17:27.507 DEBUG:teuthology.orchestra.run.smithi143:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.2 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2023-11-26T07:17:27.510 DEBUG:teuthology.orchestra.run.smithi188:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.2 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2023-11-26T07:17:27.568 DEBUG:teuthology.orchestra.run.smithi143:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.1 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2023-11-26T07:17:27.571 DEBUG:teuthology.orchestra.run.smithi188:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.1 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2023-11-26T07:17:27.629 DEBUG:teuthology.orchestra.run.smithi143:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.3 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2023-11-26T07:17:27.632 DEBUG:teuthology.orchestra.run.smithi188:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.3 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2023-11-26T07:17:27.689 INFO:tasks.ceph:Running mkfs on mon nodes... 2023-11-26T07:17:27.689 DEBUG:teuthology.orchestra.run.smithi143:> sudo mkdir -p /var/lib/ceph/mon/ceph-a 2023-11-26T07:17:27.710 DEBUG:teuthology.orchestra.run.smithi143:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-mon --cluster ceph --mkfs -i a --monmap /home/ubuntu/cephtest/ceph.monmap --keyring /etc/ceph/ceph.keyring 2023-11-26T07:17:27.825 DEBUG:teuthology.orchestra.run.smithi143:> sudo chown -R ceph:ceph /var/lib/ceph/mon/ceph-a 2023-11-26T07:17:27.845 DEBUG:teuthology.orchestra.run.smithi188:> sudo mkdir -p /var/lib/ceph/mon/ceph-b 2023-11-26T07:17:27.865 DEBUG:teuthology.orchestra.run.smithi188:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-mon --cluster ceph --mkfs -i b --monmap /home/ubuntu/cephtest/ceph.monmap --keyring /etc/ceph/ceph.keyring 2023-11-26T07:17:27.981 DEBUG:teuthology.orchestra.run.smithi188:> sudo chown -R ceph:ceph /var/lib/ceph/mon/ceph-b 2023-11-26T07:17:28.001 DEBUG:teuthology.orchestra.run.smithi188:> sudo mkdir -p /var/lib/ceph/mon/ceph-c 2023-11-26T07:17:28.067 DEBUG:teuthology.orchestra.run.smithi188:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-mon --cluster ceph --mkfs -i c --monmap /home/ubuntu/cephtest/ceph.monmap --keyring /etc/ceph/ceph.keyring 2023-11-26T07:17:28.181 DEBUG:teuthology.orchestra.run.smithi188:> sudo chown -R ceph:ceph /var/lib/ceph/mon/ceph-c 2023-11-26T07:17:28.201 DEBUG:teuthology.orchestra.run.smithi143:> rm -- /home/ubuntu/cephtest/ceph.monmap 2023-11-26T07:17:28.204 DEBUG:teuthology.orchestra.run.smithi188:> rm -- /home/ubuntu/cephtest/ceph.monmap 2023-11-26T07:17:28.252 INFO:tasks.ceph:Starting mon daemons in cluster ceph... 2023-11-26T07:17:28.252 INFO:tasks.ceph.mon.a:Restarting daemon 2023-11-26T07:17:28.252 DEBUG:teuthology.orchestra.run.smithi143:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mon -f --cluster ceph -i a 2023-11-26T07:17:28.256 INFO:tasks.ceph.mon.a:Started 2023-11-26T07:17:28.256 INFO:tasks.ceph.mon.b:Restarting daemon 2023-11-26T07:17:28.256 DEBUG:teuthology.orchestra.run.smithi188:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mon -f --cluster ceph -i b 2023-11-26T07:17:28.295 INFO:tasks.ceph.mon.b:Started 2023-11-26T07:17:28.296 INFO:tasks.ceph.mon.c:Restarting daemon 2023-11-26T07:17:28.296 DEBUG:teuthology.orchestra.run.smithi188:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mon -f --cluster ceph -i c 2023-11-26T07:17:28.299 INFO:tasks.ceph.mon.c:Started 2023-11-26T07:17:28.299 INFO:tasks.ceph:Starting mgr daemons in cluster ceph... 2023-11-26T07:17:28.299 INFO:tasks.ceph.mgr.y:Restarting daemon 2023-11-26T07:17:28.299 DEBUG:teuthology.orchestra.run.smithi143:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mgr -f --cluster ceph -i y 2023-11-26T07:17:28.303 INFO:tasks.ceph.mgr.y:Started 2023-11-26T07:17:28.303 INFO:tasks.ceph.mgr.x:Restarting daemon 2023-11-26T07:17:28.303 DEBUG:teuthology.orchestra.run.smithi188:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mgr -f --cluster ceph -i x 2023-11-26T07:17:28.306 INFO:tasks.ceph.mgr.x:Started 2023-11-26T07:17:28.306 INFO:tasks.ceph:Setting crush tunables to default 2023-11-26T07:17:28.307 DEBUG:teuthology.orchestra.run.smithi143:> sudo ceph --cluster ceph osd crush tunables default 2023-11-26T07:17:28.446 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:17:28.444+0000 7f86d01aa640 1 Processor -- start 2023-11-26T07:17:28.446 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:17:28.444+0000 7f86d01aa640 1 -- start start 2023-11-26T07:17:28.448 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:17:28.444+0000 7f86d01aa640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f86c806f3e0 0x7f86c8161690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:17:28.448 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:17:28.444+0000 7f86d01aa640 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7f86c8099a70 0x7f86c8161bd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:17:28.448 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:17:28.444+0000 7f86d01aa640 1 -- --> v1:172.21.15.143:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f86c80cf260 con 0x7f86c806f000 2023-11-26T07:17:28.449 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:17:28.444+0000 7f86d01aa640 1 -- --> v2:172.21.15.143:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f86c809a010 con 0x7f86c8099a70 2023-11-26T07:17:28.449 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:17:28.444+0000 7f86d01aa640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f86c809a2f0 con 0x7f86c806f3e0 2023-11-26T07:17:28.449 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:17:28.448+0000 7f86cdf46640 1 --1- >> v1:172.21.15.143:6789/0 conn(0x7f86c806f000 0x7f86c806d030 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.143:6789/0 says I am v1:172.21.15.143:51820/0 (socket says 172.21.15.143:51820) 2023-11-26T07:17:28.449 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:17:28.448+0000 7f86cdf46640 1 -- 172.21.15.143:0/2577620390 learned_addr learned my addr 172.21.15.143:0/2577620390 (peer_addr_for_me v1:172.21.15.143:0/0) 2023-11-26T07:17:28.449 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:17:28.448+0000 7f86ce747640 1 --2- 172.21.15.143:0/2577620390 >> v2:172.21.15.143:3300/0 conn(0x7f86c8099a70 0x7f86c8161bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:17:28.449 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:17:28.448+0000 7f86ce747640 1 -- 172.21.15.143:0/2577620390 >> v2:172.21.15.143:3300/0 conn(0x7f86c8099a70 msgr2=0x7f86c8161bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 15 2023-11-26T07:17:28.449 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:17:28.448+0000 7f86ce747640 1 -- 172.21.15.143:0/2577620390 >> v2:172.21.15.143:3300/0 conn(0x7f86c8099a70 msgr2=0x7f86c8161bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2023-11-26T07:17:28.449 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:17:28.448+0000 7f86ce747640 1 --2- 172.21.15.143:0/2577620390 >> v2:172.21.15.143:3300/0 conn(0x7f86c8099a70 0x7f86c8161bd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2023-11-26T07:17:28.449 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:17:28.448+0000 7f86cd745640 1 --2- 172.21.15.143:0/2577620390 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f86c806f3e0 0x7f86c8161690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:17:28.449 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:17:28.448+0000 7f86ce747640 1 --2- 172.21.15.143:0/2577620390 >> v2:172.21.15.143:3300/0 conn(0x7f86c8099a70 0x7f86c8161bd0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-11-26T07:17:28.449 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:17:28.448+0000 7f86cd745640 1 -- 172.21.15.143:0/2577620390 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f86c806f3e0 msgr2=0x7f86c8161690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 14 2023-11-26T07:17:28.449 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:17:28.448+0000 7f86cd745640 1 -- 172.21.15.143:0/2577620390 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f86c806f3e0 msgr2=0x7f86c8161690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2023-11-26T07:17:28.449 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:17:28.448+0000 7f86cd745640 1 --2- 172.21.15.143:0/2577620390 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f86c806f3e0 0x7f86c8161690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2023-11-26T07:17:28.450 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:17:28.448+0000 7f86cd745640 1 --2- 172.21.15.143:0/2577620390 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f86c806f3e0 0x7f86c8161690 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-11-26T07:17:28.631 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:17:28.628+0000 7f86ccf44640 1 -- 172.21.15.143:0/2577620390 <== mon.0 v1:172.21.15.143:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1520912614 0 0) 0x7f86c80cf260 con 0x7f86c806f000 2023-11-26T07:17:28.631 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:17:28.628+0000 7f86ccf44640 1 -- 172.21.15.143:0/2577620390 --> v1:172.21.15.143:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f86b4003530 con 0x7f86c806f000 2023-11-26T07:17:28.633 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:17:28.632+0000 7f86ccf44640 1 -- 172.21.15.143:0/2577620390 <== mon.0 v1:172.21.15.143:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1810331267 0 0) 0x7f86b4003530 con 0x7f86c806f000 2023-11-26T07:17:28.633 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:17:28.632+0000 7f86ccf44640 1 -- 172.21.15.143:0/2577620390 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f86c806f3e0 msgr2=0x7f86c8161690 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2023-11-26T07:17:28.633 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:17:28.632+0000 7f86ccf44640 1 --2- 172.21.15.143:0/2577620390 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f86c806f3e0 0x7f86c8161690 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:17:28.633 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:17:28.632+0000 7f86ccf44640 1 -- 172.21.15.143:0/2577620390 >> v2:172.21.15.143:3300/0 conn(0x7f86c8099a70 msgr2=0x7f86c8161bd0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2023-11-26T07:17:28.633 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:17:28.632+0000 7f86ccf44640 1 --2- 172.21.15.143:0/2577620390 >> v2:172.21.15.143:3300/0 conn(0x7f86c8099a70 0x7f86c8161bd0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:17:28.633 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:17:28.632+0000 7f86ccf44640 1 -- 172.21.15.143:0/2577620390 --> v1:172.21.15.143:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f86c809a5d0 con 0x7f86c806f000 2023-11-26T07:17:28.633 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:17:28.632+0000 7f86ccf44640 1 -- 172.21.15.143:0/2577620390 <== mon.0 v1:172.21.15.143:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2137882523 0 0) 0x7f86c4001680 con 0x7f86c806f000 2023-11-26T07:17:28.633 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:17:28.632+0000 7f86ccf44640 1 -- 172.21.15.143:0/2577620390 >> v1:172.21.15.143:6789/0 conn(0x7f86c806f000 legacy=0x7f86c806d030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:17:28.633 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:17:28.632+0000 7f86ccf44640 1 --2- 172.21.15.143:0/2577620390 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f86b4003980 0x7f86b4003da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:17:28.633 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:17:28.632+0000 7f86ccf44640 1 --2- 172.21.15.143:0/2577620390 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f86b40047d0 0x7f86b4004c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:17:28.633 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:17:28.632+0000 7f86ccf44640 1 --2- 172.21.15.143:0/2577620390 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f86c806f3e0 0x7f86b40051b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:17:28.634 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:17:28.632+0000 7f86ccf44640 1 -- 172.21.15.143:0/2577620390 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f86c809a010 con 0x7f86b40047d0 2023-11-26T07:17:28.634 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:17:28.632+0000 7f86ccf44640 1 -- 172.21.15.143:0/2577620390 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f86c809a2f0 con 0x7f86b4003980 2023-11-26T07:17:28.634 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:17:28.632+0000 7f86ccf44640 1 -- 172.21.15.143:0/2577620390 --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f86b4014360 con 0x7f86c806f3e0 2023-11-26T07:17:28.634 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:17:28.632+0000 7f86cdf46640 1 --2- 172.21.15.143:0/2577620390 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f86b4003980 0x7f86b4003da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:17:28.634 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:17:28.632+0000 7f86cd745640 1 --2- 172.21.15.143:0/2577620390 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f86b40047d0 0x7f86b4004c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:17:28.634 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:17:28.632+0000 7f86ce747640 1 --2- 172.21.15.143:0/2577620390 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f86c806f3e0 0x7f86b40051b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:17:28.635 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:17:28.632+0000 7f86cd745640 1 -- 172.21.15.143:0/2577620390 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f86c806f3e0 msgr2=0x7f86b40051b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-26T07:17:28.635 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:17:28.632+0000 7f86cd745640 1 --2- 172.21.15.143:0/2577620390 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f86c806f3e0 0x7f86b40051b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:17:28.635 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:17:28.632+0000 7f86cd745640 1 -- 172.21.15.143:0/2577620390 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f86b4003980 msgr2=0x7f86b4003da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-26T07:17:28.635 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:17:28.632+0000 7f86cd745640 1 --2- 172.21.15.143:0/2577620390 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f86b4003980 0x7f86b4003da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:17:28.635 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:17:28.632+0000 7f86cd745640 1 -- 172.21.15.143:0/2577620390 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f86b4003530 con 0x7f86b40047d0 2023-11-26T07:17:28.636 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:17:28.636+0000 7f86cd745640 1 --2- 172.21.15.143:0/2577620390 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f86b40047d0 0x7f86b4004c50 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7f86b800c820 tx=0x7f86b801c4f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=aeb97a5c6b3d74a7 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:17:28.637 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:17:28.636+0000 7f86ccf44640 1 -- 172.21.15.143:0/2577620390 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f86b80ab600 con 0x7f86b40047d0 2023-11-26T07:17:28.637 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:17:28.636+0000 7f86ccf44640 1 -- 172.21.15.143:0/2577620390 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f86b80a7050 con 0x7f86b40047d0 2023-11-26T07:17:28.637 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:17:28.636+0000 7f86ccf44640 1 -- 172.21.15.143:0/2577620390 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f86b80ab9a0 con 0x7f86b40047d0 2023-11-26T07:17:28.638 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:17:28.636+0000 7f86d01aa640 1 -- 172.21.15.143:0/2577620390 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f86b40047d0 msgr2=0x7f86b4004c50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:17:28.638 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:17:28.636+0000 7f86d01aa640 1 --2- 172.21.15.143:0/2577620390 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f86b40047d0 0x7f86b4004c50 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7f86b800c820 tx=0x7f86b801c4f0 comp rx=0 tx=0).stop 2023-11-26T07:17:28.638 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:17:28.636+0000 7f86ce747640 1 -- 172.21.15.143:0/2577620390 reap_dead start 2023-11-26T07:17:28.638 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:17:28.636+0000 7f86d01aa640 1 -- 172.21.15.143:0/2577620390 shutdown_connections 2023-11-26T07:17:28.638 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:17:28.636+0000 7f86d01aa640 1 -- 172.21.15.143:0/2577620390 >> 172.21.15.143:0/2577620390 conn(0x7f86c80b2d10 msgr2=0x7f86c80b3120 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:17:28.638 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:17:28.636+0000 7f86d01aa640 1 -- 172.21.15.143:0/2577620390 shutdown_connections 2023-11-26T07:17:28.638 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:17:28.636+0000 7f86d01aa640 1 -- 172.21.15.143:0/2577620390 wait complete. 2023-11-26T07:17:28.639 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:17:28.636+0000 7f86d01aa640 1 Processor -- start 2023-11-26T07:17:28.639 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:17:28.636+0000 7f86d01aa640 1 -- start start 2023-11-26T07:17:28.641 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:17:28.640+0000 7f86d01aa640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f86b40047d0 0x7f86c80b4c40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:17:28.642 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:17:28.640+0000 7f86d01aa640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f86c81559c0 0x7f86c8155e40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:17:28.642 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:17:28.640+0000 7f86d01aa640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f86c8159b80 0x7f86c815a000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:17:28.642 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:17:28.640+0000 7f86d01aa640 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f86c80b4820 con 0x7f86c8159b80 2023-11-26T07:17:28.642 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:17:28.640+0000 7f86d01aa640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f86c8156380 con 0x7f86c81559c0 2023-11-26T07:17:28.642 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:17:28.640+0000 7f86d01aa640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f86c815a540 con 0x7f86b40047d0 2023-11-26T07:17:28.642 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:17:28.640+0000 7f86ce747640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f86c8159b80 0x7f86c815a000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:17:28.642 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:17:28.640+0000 7f86ce747640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f86c8159b80 0x7f86c815a000 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:47150/0 (socket says 172.21.15.143:47150) 2023-11-26T07:17:28.642 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:17:28.640+0000 7f86ce747640 1 -- 172.21.15.143:0/2602844123 learned_addr learned my addr 172.21.15.143:0/2602844123 (peer_addr_for_me v2:172.21.15.143:0/0) 2023-11-26T07:17:28.642 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:17:28.640+0000 7f86cd745640 1 --2- 172.21.15.143:0/2602844123 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f86c81559c0 0x7f86c8155e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:17:28.642 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:17:28.640+0000 7f86ce747640 1 -- 172.21.15.143:0/2602844123 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f86b40047d0 msgr2=0x7f86c80b4c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:17:28.642 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:17:28.640+0000 7f86ce747640 1 --2- 172.21.15.143:0/2602844123 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f86b40047d0 0x7f86c80b4c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:17:28.642 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:17:28.640+0000 7f86ce747640 1 -- 172.21.15.143:0/2602844123 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f86c81559c0 msgr2=0x7f86c8155e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:17:28.643 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:17:28.640+0000 7f86ce747640 1 --2- 172.21.15.143:0/2602844123 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f86c81559c0 0x7f86c8155e40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:17:28.643 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:17:28.640+0000 7f86ce747640 1 -- 172.21.15.143:0/2602844123 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f86c80b60c0 con 0x7f86c8159b80 2023-11-26T07:17:28.643 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:17:28.640+0000 7f86ce747640 1 --2- 172.21.15.143:0/2602844123 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f86c8159b80 0x7f86c815a000 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7f86c8071760 tx=0x7f86c000f520 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:17:28.644 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:17:28.640+0000 7f86be7fc640 1 -- 172.21.15.143:0/2602844123 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f86c0003ba0 con 0x7f86c8159b80 2023-11-26T07:17:28.644 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:17:28.640+0000 7f86be7fc640 1 -- 172.21.15.143:0/2602844123 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f86c0003d40 con 0x7f86c8159b80 2023-11-26T07:17:28.644 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:17:28.640+0000 7f86d01aa640 1 -- 172.21.15.143:0/2602844123 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f86c80b41a0 con 0x7f86c8159b80 2023-11-26T07:17:28.644 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:17:28.640+0000 7f86be7fc640 1 -- 172.21.15.143:0/2602844123 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f86c0004270 con 0x7f86c8159b80 2023-11-26T07:17:28.644 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:17:28.640+0000 7f86d01aa640 1 -- 172.21.15.143:0/2602844123 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f86c80b4440 con 0x7f86c8159b80 2023-11-26T07:17:28.644 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:17:28.640+0000 7f86be7fc640 1 -- 172.21.15.143:0/2602844123 <== mon.0 v2:172.21.15.143:3300/0 4 ==== mgrmap(e 1) v1 ==== 660+0+0 (secure 0 0 0) 0x7f86c00043d0 con 0x7f86c8159b80 2023-11-26T07:17:28.644 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:17:28.640+0000 7f86be7fc640 1 -- 172.21.15.143:0/2602844123 <== mon.0 v2:172.21.15.143:3300/0 5 ==== osd_map(1..1 src has 1..1) v4 ==== 725+0+0 (secure 0 0 0) 0x7f86c001cbb0 con 0x7f86c8159b80 2023-11-26T07:17:28.644 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:17:28.644+0000 7f86d01aa640 1 -- 172.21.15.143:0/2602844123 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f86c8067680 con 0x7f86c8159b80 2023-11-26T07:17:28.647 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:17:28.644+0000 7f86be7fc640 1 -- 172.21.15.143:0/2602844123 <== mon.0 v2:172.21.15.143:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+72898 (secure 0 0 0) 0x7f86c0004810 con 0x7f86c8159b80 2023-11-26T07:17:28.708 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:17:28.704+0000 7f86d01aa640 1 -- 172.21.15.143:0/2602844123 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "osd crush tunables", "profile": "default"} v 0) v1 -- 0x7f86c80b5b70 con 0x7f86c8159b80 2023-11-26T07:17:28.710 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:17:28.708+0000 7f86be7fc640 1 -- 172.21.15.143:0/2602844123 <== mon.0 v2:172.21.15.143:3300/0 7 ==== mon_command_ack([{"prefix": "osd crush tunables", "profile": "default"}]=0 adjusted tunables profile to default v2) v1 ==== 124+0+0 (secure 0 0 0) 0x7f86c001cd70 con 0x7f86c8159b80 2023-11-26T07:17:28.710 INFO:teuthology.orchestra.run.smithi143.stderr:adjusted tunables profile to default 2023-11-26T07:17:28.711 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:17:28.708+0000 7f86d01aa640 1 -- 172.21.15.143:0/2602844123 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f86c8159b80 msgr2=0x7f86c815a000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:17:28.711 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:17:28.708+0000 7f86d01aa640 1 --2- 172.21.15.143:0/2602844123 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f86c8159b80 0x7f86c815a000 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7f86c8071760 tx=0x7f86c000f520 comp rx=0 tx=0).stop 2023-11-26T07:17:28.711 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:17:28.708+0000 7f86d01aa640 1 -- 172.21.15.143:0/2602844123 shutdown_connections 2023-11-26T07:17:28.711 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:17:28.708+0000 7f86d01aa640 1 --2- 172.21.15.143:0/2602844123 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f86c8159b80 0x7f86c815a000 unknown :-1 s=CLOSED pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:17:28.711 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:17:28.708+0000 7f86d01aa640 1 --2- 172.21.15.143:0/2602844123 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f86c81559c0 0x7f86c8155e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:17:28.711 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:17:28.708+0000 7f86d01aa640 1 --2- 172.21.15.143:0/2602844123 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f86b40047d0 0x7f86c80b4c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:17:28.712 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:17:28.708+0000 7f86d01aa640 1 -- 172.21.15.143:0/2602844123 >> 172.21.15.143:0/2602844123 conn(0x7f86c80b2d10 msgr2=0x7f86c80c7660 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:17:28.712 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:17:28.708+0000 7f86d01aa640 1 -- 172.21.15.143:0/2602844123 shutdown_connections 2023-11-26T07:17:28.712 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:17:28.708+0000 7f86d01aa640 1 -- 172.21.15.143:0/2602844123 wait complete. 2023-11-26T07:17:28.729 INFO:tasks.ceph:check_enable_crimson: False 2023-11-26T07:17:28.730 INFO:tasks.ceph:Starting osd daemons in cluster ceph... 2023-11-26T07:17:28.730 DEBUG:teuthology.orchestra.run.smithi143:> set -ex 2023-11-26T07:17:28.730 DEBUG:teuthology.orchestra.run.smithi143:> sudo dd if=/var/lib/ceph/osd/ceph-0/fsid of=/dev/stdout 2023-11-26T07:17:28.751 DEBUG:teuthology.orchestra.run.smithi143:> set -ex 2023-11-26T07:17:28.751 DEBUG:teuthology.orchestra.run.smithi143:> sudo dd if=/var/lib/ceph/osd/ceph-1/fsid of=/dev/stdout 2023-11-26T07:17:28.814 DEBUG:teuthology.orchestra.run.smithi143:> set -ex 2023-11-26T07:17:28.814 DEBUG:teuthology.orchestra.run.smithi143:> sudo dd if=/var/lib/ceph/osd/ceph-2/fsid of=/dev/stdout 2023-11-26T07:17:28.858 INFO:tasks.ceph.mgr.x.smithi188.stderr:2023-11-26T07:17:28.855+0000 7f052864e180 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2023-11-26T07:17:28.867 INFO:tasks.ceph.mgr.y.smithi143.stderr:2023-11-26T07:17:28.864+0000 7f50c328d180 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2023-11-26T07:17:28.876 DEBUG:teuthology.orchestra.run.smithi143:> set -ex 2023-11-26T07:17:28.877 DEBUG:teuthology.orchestra.run.smithi143:> sudo dd if=/var/lib/ceph/osd/ceph-3/fsid of=/dev/stdout 2023-11-26T07:17:28.939 DEBUG:teuthology.orchestra.run.smithi188:> set -ex 2023-11-26T07:17:28.940 DEBUG:teuthology.orchestra.run.smithi188:> sudo dd if=/var/lib/ceph/osd/ceph-4/fsid of=/dev/stdout 2023-11-26T07:17:28.961 DEBUG:teuthology.orchestra.run.smithi188:> set -ex 2023-11-26T07:17:28.961 DEBUG:teuthology.orchestra.run.smithi188:> sudo dd if=/var/lib/ceph/osd/ceph-5/fsid of=/dev/stdout 2023-11-26T07:17:29.024 DEBUG:teuthology.orchestra.run.smithi188:> set -ex 2023-11-26T07:17:29.024 DEBUG:teuthology.orchestra.run.smithi188:> sudo dd if=/var/lib/ceph/osd/ceph-6/fsid of=/dev/stdout 2023-11-26T07:17:29.088 DEBUG:teuthology.orchestra.run.smithi188:> set -ex 2023-11-26T07:17:29.088 DEBUG:teuthology.orchestra.run.smithi188:> sudo dd if=/var/lib/ceph/osd/ceph-7/fsid of=/dev/stdout 2023-11-26T07:17:29.148 DEBUG:teuthology.orchestra.run.smithi188:> sudo ceph --cluster ceph osd new ebbee2c6-023e-44b9-9f1d-6f7bfd309730 0 2023-11-26T07:17:29.329 INFO:teuthology.orchestra.run.smithi188.stderr:2023-11-26T07:17:29.327+0000 7f05df301640 1 Processor -- start 2023-11-26T07:17:29.330 INFO:teuthology.orchestra.run.smithi188.stderr:2023-11-26T07:17:29.327+0000 7f05df301640 1 -- start start 2023-11-26T07:17:29.331 INFO:teuthology.orchestra.run.smithi188.stderr:2023-11-26T07:17:29.327+0000 7f05df301640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f05d815d6e0 0x7f05d80d0df0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:17:29.331 INFO:teuthology.orchestra.run.smithi188.stderr:2023-11-26T07:17:29.327+0000 7f05df301640 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7f05d80d1bc0 0x7f05d80b0750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:17:29.331 INFO:teuthology.orchestra.run.smithi188.stderr:2023-11-26T07:17:29.327+0000 7f05df301640 1 -- --> v1:172.21.15.188:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f05d80cf0e0 con 0x7f05d80b2160 2023-11-26T07:17:29.331 INFO:teuthology.orchestra.run.smithi188.stderr:2023-11-26T07:17:29.327+0000 7f05df301640 1 -- --> v2:172.21.15.143:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f05d815dac0 con 0x7f05d80d1bc0 2023-11-26T07:17:29.331 INFO:teuthology.orchestra.run.smithi188.stderr:2023-11-26T07:17:29.327+0000 7f05df301640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f05d80ceb20 con 0x7f05d815d6e0 2023-11-26T07:17:29.331 INFO:teuthology.orchestra.run.smithi188.stderr:2023-11-26T07:17:29.327+0000 7f05dd09d640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f05d815d6e0 0x7f05d80d0df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:17:29.332 INFO:teuthology.orchestra.run.smithi188.stderr:2023-11-26T07:17:29.327+0000 7f05dc89c640 1 --1- >> v1:172.21.15.188:6789/0 conn(0x7f05d80b2160 0x7f05d80d14b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.188:6789/0 says I am v1:172.21.15.188:41038/0 (socket says 172.21.15.188:41038) 2023-11-26T07:17:29.332 INFO:teuthology.orchestra.run.smithi188.stderr:2023-11-26T07:17:29.327+0000 7f05dc89c640 1 -- 172.21.15.188:0/70130614 learned_addr learned my addr 172.21.15.188:0/70130614 (peer_addr_for_me v1:172.21.15.188:0/0) 2023-11-26T07:17:29.332 INFO:teuthology.orchestra.run.smithi188.stderr:2023-11-26T07:17:29.327+0000 7f05dd89e640 1 --2- 172.21.15.188:0/70130614 >> v2:172.21.15.143:3300/0 conn(0x7f05d80d1bc0 0x7f05d80b0750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:17:29.332 INFO:teuthology.orchestra.run.smithi188.stderr:2023-11-26T07:17:29.327+0000 7f05dd09d640 1 -- 172.21.15.188:0/70130614 >> v2:172.21.15.143:3300/0 conn(0x7f05d80d1bc0 msgr2=0x7f05d80b0750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-26T07:17:29.332 INFO:teuthology.orchestra.run.smithi188.stderr:2023-11-26T07:17:29.327+0000 7f05dd09d640 1 --2- 172.21.15.188:0/70130614 >> v2:172.21.15.143:3300/0 conn(0x7f05d80d1bc0 0x7f05d80b0750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:17:29.332 INFO:teuthology.orchestra.run.smithi188.stderr:2023-11-26T07:17:29.327+0000 7f05dd09d640 1 -- 172.21.15.188:0/70130614 >> v1:172.21.15.188:6789/0 conn(0x7f05d80b2160 legacy=0x7f05d80d14b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:17:29.332 INFO:teuthology.orchestra.run.smithi188.stderr:2023-11-26T07:17:29.327+0000 7f05dd09d640 1 -- 172.21.15.188:0/70130614 --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f05d80cee00 con 0x7f05d815d6e0 2023-11-26T07:17:29.333 INFO:teuthology.orchestra.run.smithi188.stderr:2023-11-26T07:17:29.327+0000 7f05cffff640 1 -- 172.21.15.188:0/70130614 <== mon.1 v1:172.21.15.188:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2256509267 0 0) 0x7f05d80cf0e0 con 0x7f05d80b2160 2023-11-26T07:17:29.334 INFO:teuthology.orchestra.run.smithi188.stderr:2023-11-26T07:17:29.331+0000 7f05dd09d640 1 --2- 172.21.15.188:0/70130614 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f05d815d6e0 0x7f05d80d0df0 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f05d40097c0 tx=0x7f05d401c480 comp rx=0 tx=0).ready entity=mon.2 client_cookie=a75658ca0e67c13a server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:17:29.334 INFO:teuthology.orchestra.run.smithi188.stderr:2023-11-26T07:17:29.331+0000 7f05cffff640 1 -- 172.21.15.188:0/70130614 <== mon.2 v2:172.21.15.188:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f05d40ab600 con 0x7f05d815d6e0 2023-11-26T07:17:29.335 INFO:teuthology.orchestra.run.smithi188.stderr:2023-11-26T07:17:29.331+0000 7f05cffff640 1 -- 172.21.15.188:0/70130614 <== mon.2 v2:172.21.15.188:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f05d40a7050 con 0x7f05d815d6e0 2023-11-26T07:17:29.335 INFO:teuthology.orchestra.run.smithi188.stderr:2023-11-26T07:17:29.331+0000 7f05cffff640 1 -- 172.21.15.188:0/70130614 <== mon.2 v2:172.21.15.188:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f05d40ab980 con 0x7f05d815d6e0 2023-11-26T07:17:29.336 INFO:teuthology.orchestra.run.smithi188.stderr:2023-11-26T07:17:29.331+0000 7f05df301640 1 -- 172.21.15.188:0/70130614 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f05d815d6e0 msgr2=0x7f05d80d0df0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:17:29.336 INFO:teuthology.orchestra.run.smithi188.stderr:2023-11-26T07:17:29.331+0000 7f05df301640 1 --2- 172.21.15.188:0/70130614 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f05d815d6e0 0x7f05d80d0df0 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f05d40097c0 tx=0x7f05d401c480 comp rx=0 tx=0).stop 2023-11-26T07:17:29.336 INFO:teuthology.orchestra.run.smithi188.stderr:2023-11-26T07:17:29.331+0000 7f05df301640 1 -- 172.21.15.188:0/70130614 shutdown_connections 2023-11-26T07:17:29.336 INFO:teuthology.orchestra.run.smithi188.stderr:2023-11-26T07:17:29.331+0000 7f05df301640 1 --2- 172.21.15.188:0/70130614 >> v2:172.21.15.143:3300/0 conn(0x7f05d80d1bc0 0x7f05d80b0750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:17:29.336 INFO:teuthology.orchestra.run.smithi188.stderr:2023-11-26T07:17:29.331+0000 7f05df301640 1 --2- 172.21.15.188:0/70130614 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f05d815d6e0 0x7f05d80d0df0 unknown :-1 s=CLOSED pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:17:29.336 INFO:teuthology.orchestra.run.smithi188.stderr:2023-11-26T07:17:29.331+0000 7f05df301640 1 -- 172.21.15.188:0/70130614 >> 172.21.15.188:0/70130614 conn(0x7f05d809c550 msgr2=0x7f05d80d3220 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:17:29.336 INFO:teuthology.orchestra.run.smithi188.stderr:2023-11-26T07:17:29.331+0000 7f05df301640 1 -- 172.21.15.188:0/70130614 shutdown_connections 2023-11-26T07:17:29.336 INFO:teuthology.orchestra.run.smithi188.stderr:2023-11-26T07:17:29.331+0000 7f05df301640 1 -- 172.21.15.188:0/70130614 wait complete. 2023-11-26T07:17:29.337 INFO:teuthology.orchestra.run.smithi188.stderr:2023-11-26T07:17:29.335+0000 7f05df301640 1 Processor -- start 2023-11-26T07:17:29.337 INFO:teuthology.orchestra.run.smithi188.stderr:2023-11-26T07:17:29.335+0000 7f05df301640 1 -- start start 2023-11-26T07:17:29.340 INFO:teuthology.orchestra.run.smithi188.stderr:2023-11-26T07:17:29.335+0000 7f05df301640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f05d80b2160 0x7f05d8159920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:17:29.340 INFO:teuthology.orchestra.run.smithi188.stderr:2023-11-26T07:17:29.335+0000 7f05df301640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f05d80d1bc0 0x7f05d8157c40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:17:29.340 INFO:teuthology.orchestra.run.smithi188.stderr:2023-11-26T07:17:29.335+0000 7f05df301640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f05d815d6e0 0x7f05d81583e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:17:29.340 INFO:teuthology.orchestra.run.smithi188.stderr:2023-11-26T07:17:29.335+0000 7f05df301640 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f05d8158c10 con 0x7f05d80b2160 2023-11-26T07:17:29.340 INFO:teuthology.orchestra.run.smithi188.stderr:2023-11-26T07:17:29.335+0000 7f05df301640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f05d8158ef0 con 0x7f05d815d6e0 2023-11-26T07:17:29.340 INFO:teuthology.orchestra.run.smithi188.stderr:2023-11-26T07:17:29.335+0000 7f05df301640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f05d81591d0 con 0x7f05d80d1bc0 2023-11-26T07:17:29.340 INFO:teuthology.orchestra.run.smithi188.stderr:2023-11-26T07:17:29.335+0000 7f05dc89c640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f05d80d1bc0 0x7f05d8157c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:17:29.340 INFO:teuthology.orchestra.run.smithi188.stderr:2023-11-26T07:17:29.335+0000 7f05dd89e640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f05d815d6e0 0x7f05d81583e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:17:29.341 INFO:teuthology.orchestra.run.smithi188.stderr:2023-11-26T07:17:29.335+0000 7f05dd09d640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f05d80b2160 0x7f05d8159920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:17:29.341 INFO:teuthology.orchestra.run.smithi188.stderr:2023-11-26T07:17:29.335+0000 7f05dc89c640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f05d80d1bc0 0x7f05d8157c40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.188:3301/0 says I am v2:172.21.15.188:35040/0 (socket says 172.21.15.188:35040) 2023-11-26T07:17:29.341 INFO:teuthology.orchestra.run.smithi188.stderr:2023-11-26T07:17:29.335+0000 7f05dc89c640 1 -- 172.21.15.188:0/3074337995 learned_addr learned my addr 172.21.15.188:0/3074337995 (peer_addr_for_me v2:172.21.15.188:0/0) 2023-11-26T07:17:29.341 INFO:teuthology.orchestra.run.smithi188.stderr:2023-11-26T07:17:29.335+0000 7f05dc89c640 1 -- 172.21.15.188:0/3074337995 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f05d815d6e0 msgr2=0x7f05d81583e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:17:29.341 INFO:teuthology.orchestra.run.smithi188.stderr:2023-11-26T07:17:29.335+0000 7f05dc89c640 1 --2- 172.21.15.188:0/3074337995 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f05d815d6e0 0x7f05d81583e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:17:29.341 INFO:teuthology.orchestra.run.smithi188.stderr:2023-11-26T07:17:29.335+0000 7f05dc89c640 1 -- 172.21.15.188:0/3074337995 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f05d80b2160 msgr2=0x7f05d8159920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:17:29.341 INFO:teuthology.orchestra.run.smithi188.stderr:2023-11-26T07:17:29.335+0000 7f05dc89c640 1 --2- 172.21.15.188:0/3074337995 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f05d80b2160 0x7f05d8159920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:17:29.341 INFO:teuthology.orchestra.run.smithi188.stderr:2023-11-26T07:17:29.335+0000 7f05dc89c640 1 -- 172.21.15.188:0/3074337995 --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f05d4009420 con 0x7f05d80d1bc0 2023-11-26T07:17:29.341 INFO:teuthology.orchestra.run.smithi188.stderr:2023-11-26T07:17:29.335+0000 7f05dd09d640 1 --2- 172.21.15.188:0/3074337995 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f05d80b2160 0x7f05d8159920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-26T07:17:29.341 INFO:teuthology.orchestra.run.smithi188.stderr:2023-11-26T07:17:29.335+0000 7f05dd89e640 1 --2- 172.21.15.188:0/3074337995 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f05d815d6e0 0x7f05d81583e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-26T07:17:29.341 INFO:teuthology.orchestra.run.smithi188.stderr:2023-11-26T07:17:29.335+0000 7f05dc89c640 1 --2- 172.21.15.188:0/3074337995 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f05d80d1bc0 0x7f05d8157c40 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7f05c80011f0 tx=0x7f05c800c9b0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:17:29.341 INFO:teuthology.orchestra.run.smithi188.stderr:2023-11-26T07:17:29.335+0000 7f05cdffb640 1 -- 172.21.15.188:0/3074337995 <== mon.2 v2:172.21.15.188:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f05c80074e0 con 0x7f05d80d1bc0 2023-11-26T07:17:29.341 INFO:teuthology.orchestra.run.smithi188.stderr:2023-11-26T07:17:29.335+0000 7f05cdffb640 1 -- 172.21.15.188:0/3074337995 <== mon.2 v2:172.21.15.188:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f05c8007640 con 0x7f05d80d1bc0 2023-11-26T07:17:29.341 INFO:teuthology.orchestra.run.smithi188.stderr:2023-11-26T07:17:29.335+0000 7f05df301640 1 -- 172.21.15.188:0/3074337995 --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f05d81594b0 con 0x7f05d80d1bc0 2023-11-26T07:17:29.342 INFO:teuthology.orchestra.run.smithi188.stderr:2023-11-26T07:17:29.335+0000 7f05cdffb640 1 -- 172.21.15.188:0/3074337995 <== mon.2 v2:172.21.15.188:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f05c8004a10 con 0x7f05d80d1bc0 2023-11-26T07:17:29.342 INFO:teuthology.orchestra.run.smithi188.stderr:2023-11-26T07:17:29.335+0000 7f05cdffb640 1 -- 172.21.15.188:0/3074337995 <== mon.2 v2:172.21.15.188:3301/0 4 ==== mgrmap(e 1) v1 ==== 660+0+0 (secure 0 0 0) 0x7f05c801c3b0 con 0x7f05d80d1bc0 2023-11-26T07:17:29.342 INFO:teuthology.orchestra.run.smithi188.stderr:2023-11-26T07:17:29.335+0000 7f05df301640 1 -- 172.21.15.188:0/3074337995 --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f05d8158920 con 0x7f05d80d1bc0 2023-11-26T07:17:29.342 INFO:teuthology.orchestra.run.smithi188.stderr:2023-11-26T07:17:29.335+0000 7f05cdffb640 1 -- 172.21.15.188:0/3074337995 <== mon.2 v2:172.21.15.188:3301/0 5 ==== osd_map(2..2 src has 1..2) v4 ==== 725+0+0 (secure 0 0 0) 0x7f05c801c9a0 con 0x7f05d80d1bc0 2023-11-26T07:17:29.342 INFO:teuthology.orchestra.run.smithi188.stderr:2023-11-26T07:17:29.339+0000 7f05df301640 1 -- 172.21.15.188:0/3074337995 --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f05d8067680 con 0x7f05d80d1bc0 2023-11-26T07:17:29.343 INFO:teuthology.orchestra.run.smithi188.stderr:2023-11-26T07:17:29.339+0000 7f05cdffb640 1 -- 172.21.15.188:0/3074337995 <== mon.2 v2:172.21.15.188:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+72898 (secure 0 0 0) 0x7f05c8018070 con 0x7f05d80d1bc0 2023-11-26T07:17:29.402 INFO:tasks.ceph.mgr.x.smithi188.stderr:2023-11-26T07:17:29.399+0000 7f052864e180 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2023-11-26T07:17:29.409 INFO:teuthology.orchestra.run.smithi188.stderr:2023-11-26T07:17:29.407+0000 7f05df301640 1 -- 172.21.15.188:0/3074337995 --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_command({"prefix": "osd new", "uuid": "ebbee2c6-023e-44b9-9f1d-6f7bfd309730", "id": 0} v 0) v1 -- 0x7f05d80b4530 con 0x7f05d80d1bc0 2023-11-26T07:17:29.417 INFO:teuthology.orchestra.run.smithi188.stderr:2023-11-26T07:17:29.415+0000 7f05cdffb640 1 -- 172.21.15.188:0/3074337995 <== mon.2 v2:172.21.15.188:3301/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "ebbee2c6-023e-44b9-9f1d-6f7bfd309730", "id": 0}]=0 v3) v1 ==== 112+0+2 (secure 0 0 0) 0x7f05c801cd00 con 0x7f05d80d1bc0 2023-11-26T07:17:29.417 INFO:teuthology.orchestra.run.smithi188.stdout:0 2023-11-26T07:17:29.417 INFO:tasks.ceph.mgr.y.smithi143.stderr:2023-11-26T07:17:29.416+0000 7f50c328d180 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2023-11-26T07:17:29.418 INFO:teuthology.orchestra.run.smithi188.stderr:2023-11-26T07:17:29.415+0000 7f05df301640 1 -- 172.21.15.188:0/3074337995 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f05d80d1bc0 msgr2=0x7f05d8157c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:17:29.418 INFO:teuthology.orchestra.run.smithi188.stderr:2023-11-26T07:17:29.415+0000 7f05df301640 1 --2- 172.21.15.188:0/3074337995 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f05d80d1bc0 0x7f05d8157c40 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7f05c80011f0 tx=0x7f05c800c9b0 comp rx=0 tx=0).stop 2023-11-26T07:17:29.418 INFO:teuthology.orchestra.run.smithi188.stderr:2023-11-26T07:17:29.415+0000 7f05df301640 1 -- 172.21.15.188:0/3074337995 shutdown_connections 2023-11-26T07:17:29.419 INFO:teuthology.orchestra.run.smithi188.stderr:2023-11-26T07:17:29.415+0000 7f05df301640 1 --2- 172.21.15.188:0/3074337995 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f05d815d6e0 0x7f05d81583e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:17:29.419 INFO:teuthology.orchestra.run.smithi188.stderr:2023-11-26T07:17:29.415+0000 7f05df301640 1 --2- 172.21.15.188:0/3074337995 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f05d80d1bc0 0x7f05d8157c40 unknown :-1 s=CLOSED pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:17:29.419 INFO:teuthology.orchestra.run.smithi188.stderr:2023-11-26T07:17:29.415+0000 7f05df301640 1 --2- 172.21.15.188:0/3074337995 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f05d80b2160 0x7f05d8159920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:17:29.419 INFO:teuthology.orchestra.run.smithi188.stderr:2023-11-26T07:17:29.415+0000 7f05df301640 1 -- 172.21.15.188:0/3074337995 >> 172.21.15.188:0/3074337995 conn(0x7f05d809c550 msgr2=0x7f05d80b26c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:17:29.419 INFO:teuthology.orchestra.run.smithi188.stderr:2023-11-26T07:17:29.415+0000 7f05df301640 1 -- 172.21.15.188:0/3074337995 shutdown_connections 2023-11-26T07:17:29.419 INFO:teuthology.orchestra.run.smithi188.stderr:2023-11-26T07:17:29.415+0000 7f05df301640 1 -- 172.21.15.188:0/3074337995 wait complete. 2023-11-26T07:17:29.436 DEBUG:teuthology.orchestra.run.smithi188:> sudo ceph --cluster ceph osd new fec4b8be-7122-49e9-9a19-5850a0700001 1 2023-11-26T07:17:29.572 INFO:teuthology.orchestra.run.smithi188.stderr:2023-11-26T07:17:29.567+0000 7f8afabdf640 1 Processor -- start 2023-11-26T07:17:29.572 INFO:teuthology.orchestra.run.smithi188.stderr:2023-11-26T07:17:29.567+0000 7f8afabdf640 1 -- start start 2023-11-26T07:17:29.574 INFO:teuthology.orchestra.run.smithi188.stderr:2023-11-26T07:17:29.571+0000 7f8afabdf640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f8af406f000 0x7f8af406d030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:17:29.574 INFO:teuthology.orchestra.run.smithi188.stderr:2023-11-26T07:17:29.571+0000 7f8afabdf640 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7f8af406f3e0 0x7f8af406b5a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:17:29.574 INFO:teuthology.orchestra.run.smithi188.stderr:2023-11-26T07:17:29.571+0000 7f8afabdf640 1 -- --> v1:172.21.15.143:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8af40cf260 con 0x7f8af4099a70 2023-11-26T07:17:29.574 INFO:teuthology.orchestra.run.smithi188.stderr:2023-11-26T07:17:29.571+0000 7f8afabdf640 1 -- --> v2:172.21.15.143:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8af406d570 con 0x7f8af406f3e0 2023-11-26T07:17:29.574 INFO:teuthology.orchestra.run.smithi188.stderr:2023-11-26T07:17:29.571+0000 7f8afabdf640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8af409a010 con 0x7f8af406f000 2023-11-26T07:17:29.574 INFO:teuthology.orchestra.run.smithi188.stderr:2023-11-26T07:17:29.571+0000 7f8af897b640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f8af406f000 0x7f8af406d030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:17:29.574 INFO:teuthology.orchestra.run.smithi188.stderr:2023-11-26T07:17:29.571+0000 7f8af897b640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f8af406f000 0x7f8af406d030 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.188:3301/0 says I am v2:172.21.15.188:35050/0 (socket says 172.21.15.188:35050) 2023-11-26T07:17:29.574 INFO:teuthology.orchestra.run.smithi188.stderr:2023-11-26T07:17:29.571+0000 7f8af897b640 1 -- 172.21.15.188:0/4159551905 learned_addr learned my addr 172.21.15.188:0/4159551905 (peer_addr_for_me v2:172.21.15.188:0/0) 2023-11-26T07:17:29.574 INFO:teuthology.orchestra.run.smithi188.stderr:2023-11-26T07:17:29.571+0000 7f8aebfff640 1 --2- 172.21.15.188:0/4159551905 >> v2:172.21.15.143:3300/0 conn(0x7f8af406f3e0 0x7f8af406b5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:17:29.574 INFO:teuthology.orchestra.run.smithi188.stderr:2023-11-26T07:17:29.571+0000 7f8af897b640 1 -- 172.21.15.188:0/4159551905 >> v2:172.21.15.143:3300/0 conn(0x7f8af406f3e0 msgr2=0x7f8af406b5a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-26T07:17:29.575 INFO:teuthology.orchestra.run.smithi188.stderr:2023-11-26T07:17:29.571+0000 7f8af897b640 1 --2- 172.21.15.188:0/4159551905 >> v2:172.21.15.143:3300/0 conn(0x7f8af406f3e0 0x7f8af406b5a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:17:29.575 INFO:teuthology.orchestra.run.smithi188.stderr:2023-11-26T07:17:29.571+0000 7f8af897b640 1 -- 172.21.15.188:0/4159551905 >> v1:172.21.15.143:6789/0 conn(0x7f8af4099a70 legacy=0x7f8af406bae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:17:29.575 INFO:teuthology.orchestra.run.smithi188.stderr:2023-11-26T07:17:29.571+0000 7f8af897b640 1 -- 172.21.15.188:0/4159551905 --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8af409a2f0 con 0x7f8af406f000 2023-11-26T07:17:29.577 INFO:teuthology.orchestra.run.smithi188.stderr:2023-11-26T07:17:29.575+0000 7f8af897b640 1 --2- 172.21.15.188:0/4159551905 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f8af406f000 0x7f8af406d030 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7f8af0009870 tx=0x7f8af001c540 comp rx=0 tx=0).ready entity=mon.2 client_cookie=1c67194977be4a88 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:17:29.577 INFO:teuthology.orchestra.run.smithi188.stderr:2023-11-26T07:17:29.575+0000 7f8aeb7fe640 1 -- 172.21.15.188:0/4159551905 <== mon.2 v2:172.21.15.188:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8af00a7070 con 0x7f8af406f000 2023-11-26T07:17:29.577 INFO:teuthology.orchestra.run.smithi188.stderr:2023-11-26T07:17:29.575+0000 7f8aeb7fe640 1 -- 172.21.15.188:0/4159551905 <== mon.2 v2:172.21.15.188:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8af00ab430 con 0x7f8af406f000 2023-11-26T07:17:29.577 INFO:teuthology.orchestra.run.smithi188.stderr:2023-11-26T07:17:29.575+0000 7f8aeb7fe640 1 -- 172.21.15.188:0/4159551905 <== mon.2 v2:172.21.15.188:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8af00ab7c0 con 0x7f8af406f000 2023-11-26T07:17:29.578 INFO:teuthology.orchestra.run.smithi188.stderr:2023-11-26T07:17:29.575+0000 7f8afabdf640 1 -- 172.21.15.188:0/4159551905 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f8af406f000 msgr2=0x7f8af406d030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:17:29.578 INFO:teuthology.orchestra.run.smithi188.stderr:2023-11-26T07:17:29.575+0000 7f8afabdf640 1 --2- 172.21.15.188:0/4159551905 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f8af406f000 0x7f8af406d030 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7f8af0009870 tx=0x7f8af001c540 comp rx=0 tx=0).stop 2023-11-26T07:17:29.578 INFO:teuthology.orchestra.run.smithi188.stderr:2023-11-26T07:17:29.575+0000 7f8afabdf640 1 -- 172.21.15.188:0/4159551905 shutdown_connections 2023-11-26T07:17:29.578 INFO:teuthology.orchestra.run.smithi188.stderr:2023-11-26T07:17:29.575+0000 7f8afabdf640 1 --2- 172.21.15.188:0/4159551905 >> v2:172.21.15.143:3300/0 conn(0x7f8af406f3e0 0x7f8af406b5a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:17:29.578 INFO:teuthology.orchestra.run.smithi188.stderr:2023-11-26T07:17:29.575+0000 7f8afabdf640 1 --2- 172.21.15.188:0/4159551905 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f8af406f000 0x7f8af406d030 unknown :-1 s=CLOSED pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:17:29.578 INFO:teuthology.orchestra.run.smithi188.stderr:2023-11-26T07:17:29.575+0000 7f8afabdf640 1 -- 172.21.15.188:0/4159551905 >> 172.21.15.188:0/4159551905 conn(0x7f8af40b2d10 msgr2=0x7f8af40b3120 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:17:29.578 INFO:teuthology.orchestra.run.smithi188.stderr:2023-11-26T07:17:29.575+0000 7f8afabdf640 1 -- 172.21.15.188:0/4159551905 shutdown_connections 2023-11-26T07:17:29.579 INFO:teuthology.orchestra.run.smithi188.stderr:2023-11-26T07:17:29.575+0000 7f8afabdf640 1 -- 172.21.15.188:0/4159551905 wait complete. 2023-11-26T07:17:29.579 INFO:teuthology.orchestra.run.smithi188.stderr:2023-11-26T07:17:29.575+0000 7f8afabdf640 1 Processor -- start 2023-11-26T07:17:29.579 INFO:teuthology.orchestra.run.smithi188.stderr:2023-11-26T07:17:29.575+0000 7f8afabdf640 1 -- start start 2023-11-26T07:17:29.582 INFO:teuthology.orchestra.run.smithi188.stderr:2023-11-26T07:17:29.575+0000 7f8afabdf640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f8af406f000 0x7f8af40cc230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:17:29.583 INFO:teuthology.orchestra.run.smithi188.stderr:2023-11-26T07:17:29.575+0000 7f8afabdf640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f8af406f3e0 0x7f8af40cde60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:17:29.583 INFO:teuthology.orchestra.run.smithi188.stderr:2023-11-26T07:17:29.575+0000 7f8afabdf640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f8af4099a70 0x7f8af40ce3a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:17:29.583 INFO:teuthology.orchestra.run.smithi188.stderr:2023-11-26T07:17:29.579+0000 7f8afabdf640 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8af41f9f80 con 0x7f8af406f3e0 2023-11-26T07:17:29.583 INFO:teuthology.orchestra.run.smithi188.stderr:2023-11-26T07:17:29.579+0000 7f8afabdf640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8af41f9340 con 0x7f8af406f000 2023-11-26T07:17:29.583 INFO:teuthology.orchestra.run.smithi188.stderr:2023-11-26T07:17:29.579+0000 7f8afabdf640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8af41f95e0 con 0x7f8af4099a70 2023-11-26T07:17:29.583 INFO:teuthology.orchestra.run.smithi188.stderr:2023-11-26T07:17:29.579+0000 7f8af897b640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f8af406f000 0x7f8af40cc230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:17:29.583 INFO:teuthology.orchestra.run.smithi188.stderr:2023-11-26T07:17:29.579+0000 7f8af917c640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f8af4099a70 0x7f8af40ce3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:17:29.583 INFO:teuthology.orchestra.run.smithi188.stderr:2023-11-26T07:17:29.579+0000 7f8af897b640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f8af406f000 0x7f8af40cc230 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.188:3300/0 says I am v2:172.21.15.188:37022/0 (socket says 172.21.15.188:37022) 2023-11-26T07:17:29.583 INFO:teuthology.orchestra.run.smithi188.stderr:2023-11-26T07:17:29.579+0000 7f8af897b640 1 -- 172.21.15.188:0/661139764 learned_addr learned my addr 172.21.15.188:0/661139764 (peer_addr_for_me v2:172.21.15.188:0/0) 2023-11-26T07:17:29.583 INFO:teuthology.orchestra.run.smithi188.stderr:2023-11-26T07:17:29.579+0000 7f8aebfff640 1 --2- 172.21.15.188:0/661139764 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f8af406f3e0 0x7f8af40cde60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:17:29.583 INFO:teuthology.orchestra.run.smithi188.stderr:2023-11-26T07:17:29.579+0000 7f8af917c640 1 -- 172.21.15.188:0/661139764 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f8af406f000 msgr2=0x7f8af40cc230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:17:29.584 INFO:teuthology.orchestra.run.smithi188.stderr:2023-11-26T07:17:29.579+0000 7f8af917c640 1 --2- 172.21.15.188:0/661139764 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f8af406f000 0x7f8af40cc230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:17:29.584 INFO:teuthology.orchestra.run.smithi188.stderr:2023-11-26T07:17:29.579+0000 7f8af917c640 1 -- 172.21.15.188:0/661139764 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f8af406f3e0 msgr2=0x7f8af40cde60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:17:29.584 INFO:teuthology.orchestra.run.smithi188.stderr:2023-11-26T07:17:29.579+0000 7f8af917c640 1 --2- 172.21.15.188:0/661139764 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f8af406f3e0 0x7f8af40cde60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:17:29.584 INFO:teuthology.orchestra.run.smithi188.stderr:2023-11-26T07:17:29.579+0000 7f8af917c640 1 -- 172.21.15.188:0/661139764 --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8af00094d0 con 0x7f8af4099a70 2023-11-26T07:17:29.584 INFO:teuthology.orchestra.run.smithi188.stderr:2023-11-26T07:17:29.579+0000 7f8af897b640 1 --2- 172.21.15.188:0/661139764 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f8af406f000 0x7f8af40cc230 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-26T07:17:29.584 INFO:teuthology.orchestra.run.smithi188.stderr:2023-11-26T07:17:29.579+0000 7f8af917c640 1 --2- 172.21.15.188:0/661139764 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f8af4099a70 0x7f8af40ce3a0 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7f8aec00c700 tx=0x7f8aec00cbc0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:17:29.584 INFO:teuthology.orchestra.run.smithi188.stderr:2023-11-26T07:17:29.579+0000 7f8ae97fa640 1 -- 172.21.15.188:0/661139764 <== mon.2 v2:172.21.15.188:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8aec003b80 con 0x7f8af4099a70 2023-11-26T07:17:29.584 INFO:teuthology.orchestra.run.smithi188.stderr:2023-11-26T07:17:29.579+0000 7f8ae97fa640 1 -- 172.21.15.188:0/661139764 <== mon.2 v2:172.21.15.188:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8aec003ce0 con 0x7f8af4099a70 2023-11-26T07:17:29.584 INFO:teuthology.orchestra.run.smithi188.stderr:2023-11-26T07:17:29.579+0000 7f8afabdf640 1 -- 172.21.15.188:0/661139764 --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8af41f98c0 con 0x7f8af4099a70 2023-11-26T07:17:29.584 INFO:teuthology.orchestra.run.smithi188.stderr:2023-11-26T07:17:29.579+0000 7f8ae97fa640 1 -- 172.21.15.188:0/661139764 <== mon.2 v2:172.21.15.188:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8aec005310 con 0x7f8af4099a70 2023-11-26T07:17:29.584 INFO:teuthology.orchestra.run.smithi188.stderr:2023-11-26T07:17:29.579+0000 7f8afabdf640 1 -- 172.21.15.188:0/661139764 --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8af41620c0 con 0x7f8af4099a70 2023-11-26T07:17:29.584 INFO:teuthology.orchestra.run.smithi188.stderr:2023-11-26T07:17:29.579+0000 7f8ae97fa640 1 -- 172.21.15.188:0/661139764 <== mon.2 v2:172.21.15.188:3301/0 4 ==== mgrmap(e 1) v1 ==== 660+0+0 (secure 0 0 0) 0x7f8aec00a640 con 0x7f8af4099a70 2023-11-26T07:17:29.584 INFO:teuthology.orchestra.run.smithi188.stderr:2023-11-26T07:17:29.579+0000 7f8ae97fa640 1 -- 172.21.15.188:0/661139764 <== mon.2 v2:172.21.15.188:3301/0 5 ==== osd_map(3..3 src has 1..3) v4 ==== 840+0+0 (secure 0 0 0) 0x7f8aec012480 con 0x7f8af4099a70 2023-11-26T07:17:29.585 INFO:teuthology.orchestra.run.smithi188.stderr:2023-11-26T07:17:29.579+0000 7f8afabdf640 1 -- 172.21.15.188:0/661139764 --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8af41fa0c0 con 0x7f8af4099a70 2023-11-26T07:17:29.585 INFO:teuthology.orchestra.run.smithi188.stderr:2023-11-26T07:17:29.583+0000 7f8ae97fa640 1 -- 172.21.15.188:0/661139764 <== mon.2 v2:172.21.15.188:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+72898 (secure 0 0 0) 0x7f8af41fa0c0 con 0x7f8af4099a70 2023-11-26T07:17:29.622 INFO:tasks.ceph.mgr.y.smithi143.stderr:2023-11-26T07:17:29.620+0000 7f50c328d180 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2023-11-26T07:17:29.653 INFO:teuthology.orchestra.run.smithi188.stderr:2023-11-26T07:17:29.651+0000 7f8afabdf640 1 -- 172.21.15.188:0/661139764 --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_command({"prefix": "osd new", "uuid": "fec4b8be-7122-49e9-9a19-5850a0700001", "id": 1} v 0) v1 -- 0x7f8af4067680 con 0x7f8af4099a70 2023-11-26T07:17:29.660 INFO:teuthology.orchestra.run.smithi188.stderr:2023-11-26T07:17:29.655+0000 7f8ae97fa640 1 -- 172.21.15.188:0/661139764 <== mon.2 v2:172.21.15.188:3301/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "fec4b8be-7122-49e9-9a19-5850a0700001", "id": 1}]=0 v4) v1 ==== 112+0+2 (secure 0 0 0) 0x7f8aec00aad0 con 0x7f8af4099a70 2023-11-26T07:17:29.660 INFO:teuthology.orchestra.run.smithi188.stdout:1 2023-11-26T07:17:29.661 INFO:teuthology.orchestra.run.smithi188.stderr:2023-11-26T07:17:29.659+0000 7f8afabdf640 1 -- 172.21.15.188:0/661139764 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f8af4099a70 msgr2=0x7f8af40ce3a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:17:29.661 INFO:teuthology.orchestra.run.smithi188.stderr:2023-11-26T07:17:29.659+0000 7f8afabdf640 1 --2- 172.21.15.188:0/661139764 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f8af4099a70 0x7f8af40ce3a0 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7f8aec00c700 tx=0x7f8aec00cbc0 comp rx=0 tx=0).stop 2023-11-26T07:17:29.661 INFO:teuthology.orchestra.run.smithi188.stderr:2023-11-26T07:17:29.659+0000 7f8afabdf640 1 -- 172.21.15.188:0/661139764 shutdown_connections 2023-11-26T07:17:29.661 INFO:teuthology.orchestra.run.smithi188.stderr:2023-11-26T07:17:29.659+0000 7f8afabdf640 1 --2- 172.21.15.188:0/661139764 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f8af4099a70 0x7f8af40ce3a0 unknown :-1 s=CLOSED pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:17:29.661 INFO:teuthology.orchestra.run.smithi188.stderr:2023-11-26T07:17:29.659+0000 7f8afabdf640 1 --2- 172.21.15.188:0/661139764 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f8af406f3e0 0x7f8af40cde60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:17:29.661 INFO:teuthology.orchestra.run.smithi188.stderr:2023-11-26T07:17:29.659+0000 7f8afabdf640 1 --2- 172.21.15.188:0/661139764 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f8af406f000 0x7f8af40cc230 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:17:29.661 INFO:teuthology.orchestra.run.smithi188.stderr:2023-11-26T07:17:29.659+0000 7f8afabdf640 1 -- 172.21.15.188:0/661139764 >> 172.21.15.188:0/661139764 conn(0x7f8af40b2d10 msgr2=0x7f8af40c5bc0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:17:29.662 INFO:teuthology.orchestra.run.smithi188.stderr:2023-11-26T07:17:29.659+0000 7f8afabdf640 1 -- 172.21.15.188:0/661139764 shutdown_connections 2023-11-26T07:17:29.662 INFO:teuthology.orchestra.run.smithi188.stderr:2023-11-26T07:17:29.659+0000 7f8afabdf640 1 -- 172.21.15.188:0/661139764 wait complete. 2023-11-26T07:17:29.681 DEBUG:teuthology.orchestra.run.smithi188:> sudo ceph --cluster ceph osd new 2888af5b-f933-46ca-8f95-6dd494ff43f6 2 2023-11-26T07:17:29.733 INFO:tasks.ceph.mgr.x.smithi188.stderr:2023-11-26T07:17:29.731+0000 7f052864e180 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2023-11-26T07:17:29.817 INFO:teuthology.orchestra.run.smithi188.stderr:2023-11-26T07:17:29.815+0000 7fd8c6ec9640 1 Processor -- start 2023-11-26T07:17:29.818 INFO:teuthology.orchestra.run.smithi188.stderr:2023-11-26T07:17:29.815+0000 7fd8c6ec9640 1 -- start start 2023-11-26T07:17:29.819 INFO:teuthology.orchestra.run.smithi188.stderr:2023-11-26T07:17:29.815+0000 7fd8c6ec9640 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7fd8c00d1bc0 0x7fd8c00d0df0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:17:29.820 INFO:teuthology.orchestra.run.smithi188.stderr:2023-11-26T07:17:29.815+0000 7fd8c6ec9640 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7fd8c00b2160 0x7fd8c00b0750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:17:29.820 INFO:teuthology.orchestra.run.smithi188.stderr:2023-11-26T07:17:29.815+0000 7fd8c6ec9640 1 -- --> v1:172.21.15.188:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd8c00cf0e0 con 0x7fd8c015d6e0 2023-11-26T07:17:29.820 INFO:teuthology.orchestra.run.smithi188.stderr:2023-11-26T07:17:29.815+0000 7fd8c6ec9640 1 -- --> v2:172.21.15.143:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd8c015dac0 con 0x7fd8c00b2160 2023-11-26T07:17:29.820 INFO:teuthology.orchestra.run.smithi188.stderr:2023-11-26T07:17:29.815+0000 7fd8c6ec9640 1 -- --> v2:172.21.15.188:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd8c00ceb20 con 0x7fd8c00d1bc0 2023-11-26T07:17:29.820 INFO:teuthology.orchestra.run.smithi188.stderr:2023-11-26T07:17:29.815+0000 7fd8bffff640 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7fd8c00d1bc0 0x7fd8c00d0df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:17:29.820 INFO:teuthology.orchestra.run.smithi188.stderr:2023-11-26T07:17:29.815+0000 7fd8bffff640 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7fd8c00d1bc0 0x7fd8c00d0df0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.188:3300/0 says I am v2:172.21.15.188:37038/0 (socket says 172.21.15.188:37038) 2023-11-26T07:17:29.820 INFO:teuthology.orchestra.run.smithi188.stderr:2023-11-26T07:17:29.815+0000 7fd8bffff640 1 -- 172.21.15.188:0/2225991412 learned_addr learned my addr 172.21.15.188:0/2225991412 (peer_addr_for_me v2:172.21.15.188:0/0) 2023-11-26T07:17:29.820 INFO:teuthology.orchestra.run.smithi188.stderr:2023-11-26T07:17:29.815+0000 7fd8c5466640 1 --2- 172.21.15.188:0/2225991412 >> v2:172.21.15.143:3300/0 conn(0x7fd8c00b2160 0x7fd8c00b0750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:17:29.820 INFO:teuthology.orchestra.run.smithi188.stderr:2023-11-26T07:17:29.815+0000 7fd8bffff640 1 -- 172.21.15.188:0/2225991412 >> v2:172.21.15.143:3300/0 conn(0x7fd8c00b2160 msgr2=0x7fd8c00b0750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-26T07:17:29.820 INFO:teuthology.orchestra.run.smithi188.stderr:2023-11-26T07:17:29.815+0000 7fd8bffff640 1 --2- 172.21.15.188:0/2225991412 >> v2:172.21.15.143:3300/0 conn(0x7fd8c00b2160 0x7fd8c00b0750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:17:29.820 INFO:teuthology.orchestra.run.smithi188.stderr:2023-11-26T07:17:29.815+0000 7fd8bffff640 1 -- 172.21.15.188:0/2225991412 >> v1:172.21.15.188:6789/0 conn(0x7fd8c015d6e0 legacy=0x7fd8c00d14b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:17:29.820 INFO:teuthology.orchestra.run.smithi188.stderr:2023-11-26T07:17:29.815+0000 7fd8bffff640 1 -- 172.21.15.188:0/2225991412 --> v2:172.21.15.188:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd8c00cee00 con 0x7fd8c00d1bc0 2023-11-26T07:17:29.822 INFO:teuthology.orchestra.run.smithi188.stderr:2023-11-26T07:17:29.819+0000 7fd8bffff640 1 --2- 172.21.15.188:0/2225991412 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fd8c00d1bc0 0x7fd8c00d0df0 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7fd8c00ca2a0 tx=0x7fd8ac01c500 comp rx=0 tx=0).ready entity=mon.1 client_cookie=a94b60ed826647ff server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:17:29.823 INFO:teuthology.orchestra.run.smithi188.stderr:2023-11-26T07:17:29.819+0000 7fd8bf7fe640 1 -- 172.21.15.188:0/2225991412 <== mon.1 v2:172.21.15.188:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd8ac0ab5f0 con 0x7fd8c00d1bc0 2023-11-26T07:17:29.823 INFO:teuthology.orchestra.run.smithi188.stderr:2023-11-26T07:17:29.819+0000 7fd8bf7fe640 1 -- 172.21.15.188:0/2225991412 <== mon.1 v2:172.21.15.188:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd8ac0ab750 con 0x7fd8c00d1bc0 2023-11-26T07:17:29.823 INFO:teuthology.orchestra.run.smithi188.stderr:2023-11-26T07:17:29.819+0000 7fd8bf7fe640 1 -- 172.21.15.188:0/2225991412 <== mon.1 v2:172.21.15.188:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd8ac0abae0 con 0x7fd8c00d1bc0 2023-11-26T07:17:29.824 INFO:teuthology.orchestra.run.smithi188.stderr:2023-11-26T07:17:29.819+0000 7fd8c6ec9640 1 -- 172.21.15.188:0/2225991412 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fd8c00d1bc0 msgr2=0x7fd8c00d0df0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:17:29.824 INFO:teuthology.orchestra.run.smithi188.stderr:2023-11-26T07:17:29.819+0000 7fd8c6ec9640 1 --2- 172.21.15.188:0/2225991412 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fd8c00d1bc0 0x7fd8c00d0df0 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7fd8c00ca2a0 tx=0x7fd8ac01c500 comp rx=0 tx=0).stop 2023-11-26T07:17:29.824 INFO:teuthology.orchestra.run.smithi188.stderr:2023-11-26T07:17:29.819+0000 7fd8c6ec9640 1 -- 172.21.15.188:0/2225991412 shutdown_connections 2023-11-26T07:17:29.824 INFO:teuthology.orchestra.run.smithi188.stderr:2023-11-26T07:17:29.819+0000 7fd8c6ec9640 1 --2- 172.21.15.188:0/2225991412 >> v2:172.21.15.143:3300/0 conn(0x7fd8c00b2160 0x7fd8c00b0750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:17:29.824 INFO:teuthology.orchestra.run.smithi188.stderr:2023-11-26T07:17:29.819+0000 7fd8c6ec9640 1 --2- 172.21.15.188:0/2225991412 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fd8c00d1bc0 0x7fd8c00d0df0 unknown :-1 s=CLOSED pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:17:29.824 INFO:teuthology.orchestra.run.smithi188.stderr:2023-11-26T07:17:29.819+0000 7fd8c6ec9640 1 -- 172.21.15.188:0/2225991412 >> 172.21.15.188:0/2225991412 conn(0x7fd8c009c550 msgr2=0x7fd8c00d3220 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:17:29.824 INFO:teuthology.orchestra.run.smithi188.stderr:2023-11-26T07:17:29.819+0000 7fd8c6ec9640 1 -- 172.21.15.188:0/2225991412 shutdown_connections 2023-11-26T07:17:29.824 INFO:teuthology.orchestra.run.smithi188.stderr:2023-11-26T07:17:29.819+0000 7fd8c6ec9640 1 -- 172.21.15.188:0/2225991412 wait complete. 2023-11-26T07:17:29.826 INFO:teuthology.orchestra.run.smithi188.stderr:2023-11-26T07:17:29.823+0000 7fd8c6ec9640 1 Processor -- start 2023-11-26T07:17:29.826 INFO:teuthology.orchestra.run.smithi188.stderr:2023-11-26T07:17:29.823+0000 7fd8c6ec9640 1 -- start start 2023-11-26T07:17:29.829 INFO:teuthology.orchestra.run.smithi188.stderr:2023-11-26T07:17:29.823+0000 7fd8c6ec9640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fd8c00b2160 0x7fd8c00ccbe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:17:29.830 INFO:teuthology.orchestra.run.smithi188.stderr:2023-11-26T07:17:29.823+0000 7fd8c6ec9640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fd8c00d1bc0 0x7fd8c015f430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:17:29.830 INFO:teuthology.orchestra.run.smithi188.stderr:2023-11-26T07:17:29.823+0000 7fd8c6ec9640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fd8c015d6e0 0x7fd8c015f970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:17:29.830 INFO:teuthology.orchestra.run.smithi188.stderr:2023-11-26T07:17:29.823+0000 7fd8c6ec9640 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd8c00ca2f0 con 0x7fd8c00b2160 2023-11-26T07:17:29.830 INFO:teuthology.orchestra.run.smithi188.stderr:2023-11-26T07:17:29.823+0000 7fd8c6ec9640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd8c00cd120 con 0x7fd8c00d1bc0 2023-11-26T07:17:29.830 INFO:teuthology.orchestra.run.smithi188.stderr:2023-11-26T07:17:29.823+0000 7fd8c6ec9640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd8c015aa30 con 0x7fd8c015d6e0 2023-11-26T07:17:29.830 INFO:teuthology.orchestra.run.smithi188.stderr:2023-11-26T07:17:29.823+0000 7fd8bffff640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fd8c00d1bc0 0x7fd8c015f430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:17:29.830 INFO:teuthology.orchestra.run.smithi188.stderr:2023-11-26T07:17:29.823+0000 7fd8c5466640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fd8c015d6e0 0x7fd8c015f970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:17:29.830 INFO:teuthology.orchestra.run.smithi188.stderr:2023-11-26T07:17:29.823+0000 7fd8c5466640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fd8c015d6e0 0x7fd8c015f970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.188:3301/0 says I am v2:172.21.15.188:35058/0 (socket says 172.21.15.188:35058) 2023-11-26T07:17:29.830 INFO:teuthology.orchestra.run.smithi188.stderr:2023-11-26T07:17:29.823+0000 7fd8bffff640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fd8c00d1bc0 0x7fd8c015f430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.188:3300/0 says I am v2:172.21.15.188:37052/0 (socket says 172.21.15.188:37052) 2023-11-26T07:17:29.830 INFO:teuthology.orchestra.run.smithi188.stderr:2023-11-26T07:17:29.823+0000 7fd8c5466640 1 -- 172.21.15.188:0/1146767053 learned_addr learned my addr 172.21.15.188:0/1146767053 (peer_addr_for_me v2:172.21.15.188:0/0) 2023-11-26T07:17:29.830 INFO:teuthology.orchestra.run.smithi188.stderr:2023-11-26T07:17:29.823+0000 7fd8c4c65640 1 --2- 172.21.15.188:0/1146767053 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fd8c00b2160 0x7fd8c00ccbe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:17:29.830 INFO:teuthology.orchestra.run.smithi188.stderr:2023-11-26T07:17:29.827+0000 7fd8c5466640 1 -- 172.21.15.188:0/1146767053 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fd8c00d1bc0 msgr2=0x7fd8c015f430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:17:29.831 INFO:teuthology.orchestra.run.smithi188.stderr:2023-11-26T07:17:29.827+0000 7fd8c5466640 1 --2- 172.21.15.188:0/1146767053 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fd8c00d1bc0 0x7fd8c015f430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:17:29.831 INFO:teuthology.orchestra.run.smithi188.stderr:2023-11-26T07:17:29.827+0000 7fd8c5466640 1 -- 172.21.15.188:0/1146767053 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fd8c00b2160 msgr2=0x7fd8c00ccbe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:17:29.831 INFO:teuthology.orchestra.run.smithi188.stderr:2023-11-26T07:17:29.827+0000 7fd8c5466640 1 --2- 172.21.15.188:0/1146767053 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fd8c00b2160 0x7fd8c00ccbe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:17:29.831 INFO:teuthology.orchestra.run.smithi188.stderr:2023-11-26T07:17:29.827+0000 7fd8c5466640 1 -- 172.21.15.188:0/1146767053 --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd8ac0094d0 con 0x7fd8c015d6e0 2023-11-26T07:17:29.831 INFO:teuthology.orchestra.run.smithi188.stderr:2023-11-26T07:17:29.827+0000 7fd8bffff640 1 --2- 172.21.15.188:0/1146767053 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fd8c00d1bc0 0x7fd8c015f430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-26T07:17:29.831 INFO:teuthology.orchestra.run.smithi188.stderr:2023-11-26T07:17:29.827+0000 7fd8c5466640 1 --2- 172.21.15.188:0/1146767053 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fd8c015d6e0 0x7fd8c015f970 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7fd8b8007e30 tx=0x7fd8b800e4b0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:17:29.831 INFO:teuthology.orchestra.run.smithi188.stderr:2023-11-26T07:17:29.827+0000 7fd8bd7fa640 1 -- 172.21.15.188:0/1146767053 <== mon.2 v2:172.21.15.188:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd8b8014620 con 0x7fd8c015d6e0 2023-11-26T07:17:29.831 INFO:teuthology.orchestra.run.smithi188.stderr:2023-11-26T07:17:29.827+0000 7fd8bd7fa640 1 -- 172.21.15.188:0/1146767053 <== mon.2 v2:172.21.15.188:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd8b800ee90 con 0x7fd8c015d6e0 2023-11-26T07:17:29.831 INFO:teuthology.orchestra.run.smithi188.stderr:2023-11-26T07:17:29.827+0000 7fd8bd7fa640 1 -- 172.21.15.188:0/1146767053 <== mon.2 v2:172.21.15.188:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd8b80149c0 con 0x7fd8c015d6e0 2023-11-26T07:17:29.831 INFO:teuthology.orchestra.run.smithi188.stderr:2023-11-26T07:17:29.827+0000 7fd8c6ec9640 1 -- 172.21.15.188:0/1146767053 --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd8c015df80 con 0x7fd8c015d6e0 2023-11-26T07:17:29.831 INFO:teuthology.orchestra.run.smithi188.stderr:2023-11-26T07:17:29.827+0000 7fd8c6ec9640 1 -- 172.21.15.188:0/1146767053 --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd8c015a6c0 con 0x7fd8c015d6e0 2023-11-26T07:17:29.831 INFO:teuthology.orchestra.run.smithi188.stderr:2023-11-26T07:17:29.827+0000 7fd8bd7fa640 1 -- 172.21.15.188:0/1146767053 <== mon.2 v2:172.21.15.188:3301/0 4 ==== mgrmap(e 1) v1 ==== 660+0+0 (secure 0 0 0) 0x7fd8b801e460 con 0x7fd8c015d6e0 2023-11-26T07:17:29.832 INFO:teuthology.orchestra.run.smithi188.stderr:2023-11-26T07:17:29.827+0000 7fd8bd7fa640 1 -- 172.21.15.188:0/1146767053 <== mon.2 v2:172.21.15.188:3301/0 5 ==== osd_map(4..4 src has 1..4) v4 ==== 955+0+0 (secure 0 0 0) 0x7fd8b801eb30 con 0x7fd8c015d6e0 2023-11-26T07:17:29.832 INFO:teuthology.orchestra.run.smithi188.stderr:2023-11-26T07:17:29.827+0000 7fd8c6ec9640 1 -- 172.21.15.188:0/1146767053 --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd8c015aba0 con 0x7fd8c015d6e0 2023-11-26T07:17:29.832 INFO:teuthology.orchestra.run.smithi188.stderr:2023-11-26T07:17:29.831+0000 7fd8bd7fa640 1 -- 172.21.15.188:0/1146767053 <== mon.2 v2:172.21.15.188:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+72898 (secure 0 0 0) 0x7fd8b801ecf0 con 0x7fd8c015d6e0 2023-11-26T07:17:29.898 INFO:teuthology.orchestra.run.smithi188.stderr:2023-11-26T07:17:29.895+0000 7fd8c6ec9640 1 -- 172.21.15.188:0/1146767053 --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_command({"prefix": "osd new", "uuid": "2888af5b-f933-46ca-8f95-6dd494ff43f6", "id": 2} v 0) v1 -- 0x7fd8c0067680 con 0x7fd8c015d6e0 2023-11-26T07:17:29.904 INFO:teuthology.orchestra.run.smithi188.stderr:2023-11-26T07:17:29.899+0000 7fd8bd7fa640 1 -- 172.21.15.188:0/1146767053 <== mon.2 v2:172.21.15.188:3301/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "2888af5b-f933-46ca-8f95-6dd494ff43f6", "id": 2}]=0 v5) v1 ==== 112+0+2 (secure 0 0 0) 0x7fd8b801e750 con 0x7fd8c015d6e0 2023-11-26T07:17:29.904 INFO:teuthology.orchestra.run.smithi188.stdout:2 2023-11-26T07:17:29.905 INFO:teuthology.orchestra.run.smithi188.stderr:2023-11-26T07:17:29.903+0000 7fd8c6ec9640 1 -- 172.21.15.188:0/1146767053 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fd8c015d6e0 msgr2=0x7fd8c015f970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:17:29.906 INFO:teuthology.orchestra.run.smithi188.stderr:2023-11-26T07:17:29.903+0000 7fd8c6ec9640 1 --2- 172.21.15.188:0/1146767053 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fd8c015d6e0 0x7fd8c015f970 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7fd8b8007e30 tx=0x7fd8b800e4b0 comp rx=0 tx=0).stop 2023-11-26T07:17:29.906 INFO:teuthology.orchestra.run.smithi188.stderr:2023-11-26T07:17:29.903+0000 7fd8c6ec9640 1 -- 172.21.15.188:0/1146767053 shutdown_connections 2023-11-26T07:17:29.906 INFO:teuthology.orchestra.run.smithi188.stderr:2023-11-26T07:17:29.903+0000 7fd8c6ec9640 1 --2- 172.21.15.188:0/1146767053 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fd8c015d6e0 0x7fd8c015f970 unknown :-1 s=CLOSED pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:17:29.906 INFO:teuthology.orchestra.run.smithi188.stderr:2023-11-26T07:17:29.903+0000 7fd8c6ec9640 1 --2- 172.21.15.188:0/1146767053 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fd8c00d1bc0 0x7fd8c015f430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:17:29.906 INFO:teuthology.orchestra.run.smithi188.stderr:2023-11-26T07:17:29.903+0000 7fd8c6ec9640 1 --2- 172.21.15.188:0/1146767053 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fd8c00b2160 0x7fd8c00ccbe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:17:29.906 INFO:teuthology.orchestra.run.smithi188.stderr:2023-11-26T07:17:29.903+0000 7fd8c6ec9640 1 -- 172.21.15.188:0/1146767053 >> 172.21.15.188:0/1146767053 conn(0x7fd8c009c550 msgr2=0x7fd8c00734a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:17:29.906 INFO:teuthology.orchestra.run.smithi188.stderr:2023-11-26T07:17:29.903+0000 7fd8c6ec9640 1 -- 172.21.15.188:0/1146767053 shutdown_connections 2023-11-26T07:17:29.906 INFO:teuthology.orchestra.run.smithi188.stderr:2023-11-26T07:17:29.903+0000 7fd8c6ec9640 1 -- 172.21.15.188:0/1146767053 wait complete. 2023-11-26T07:17:29.923 DEBUG:teuthology.orchestra.run.smithi188:> sudo ceph --cluster ceph osd new d1f52435-7bc1-4081-9550-9cd1ba55f549 3 2023-11-26T07:17:30.059 INFO:teuthology.orchestra.run.smithi188.stderr:2023-11-26T07:17:30.055+0000 7fa72032a640 1 Processor -- start 2023-11-26T07:17:30.059 INFO:teuthology.orchestra.run.smithi188.stderr:2023-11-26T07:17:30.055+0000 7fa72032a640 1 -- start start 2023-11-26T07:17:30.060 INFO:teuthology.orchestra.run.smithi188.stderr:2023-11-26T07:17:30.055+0000 7fa72032a640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fa71815d5b0 0x7fa7180d1820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:17:30.060 INFO:teuthology.orchestra.run.smithi188.stderr:2023-11-26T07:17:30.055+0000 7fa72032a640 1 -- --> v1:172.21.15.143:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa7180d21c0 con 0x7fa7180d1d60 2023-11-26T07:17:30.060 INFO:teuthology.orchestra.run.smithi188.stderr:2023-11-26T07:17:30.055+0000 7fa72032a640 1 -- --> v1:172.21.15.188:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa7180ce590 con 0x7fa7180b3880 2023-11-26T07:17:30.060 INFO:teuthology.orchestra.run.smithi188.stderr:2023-11-26T07:17:30.055+0000 7fa72032a640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa7180cdf80 con 0x7fa71815d5b0 2023-11-26T07:17:30.060 INFO:teuthology.orchestra.run.smithi188.stderr:2023-11-26T07:17:30.055+0000 7fa71e0c6640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fa71815d5b0 0x7fa7180d1820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:17:30.061 INFO:teuthology.orchestra.run.smithi188.stderr:2023-11-26T07:17:30.055+0000 7fa71e8c7640 1 --1- >> v1:172.21.15.188:6789/0 conn(0x7fa7180b3880 0x7fa7180cea20 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.188:6789/0 says I am v1:172.21.15.188:41060/0 (socket says 172.21.15.188:41060) 2023-11-26T07:17:30.061 INFO:teuthology.orchestra.run.smithi188.stderr:2023-11-26T07:17:30.055+0000 7fa71e8c7640 1 -- 172.21.15.188:0/2053583403 learned_addr learned my addr 172.21.15.188:0/2053583403 (peer_addr_for_me v1:172.21.15.188:0/0) 2023-11-26T07:17:30.061 INFO:teuthology.orchestra.run.smithi188.stderr:2023-11-26T07:17:30.059+0000 7fa71d0c4640 1 -- 172.21.15.188:0/2053583403 <== mon.1 v1:172.21.15.188:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4212115514 0 0) 0x7fa7180ce590 con 0x7fa7180b3880 2023-11-26T07:17:30.061 INFO:teuthology.orchestra.run.smithi188.stderr:2023-11-26T07:17:30.059+0000 7fa71e0c6640 1 -- 172.21.15.188:0/2053583403 >> v1:172.21.15.188:6789/0 conn(0x7fa7180b3880 legacy=0x7fa7180cea20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:17:30.061 INFO:teuthology.orchestra.run.smithi188.stderr:2023-11-26T07:17:30.059+0000 7fa71e0c6640 1 -- 172.21.15.188:0/2053583403 >> v1:172.21.15.143:6789/0 conn(0x7fa7180d1d60 legacy=0x7fa7180b3170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:17:30.061 INFO:teuthology.orchestra.run.smithi188.stderr:2023-11-26T07:17:30.059+0000 7fa71e0c6640 1 -- 172.21.15.188:0/2053583403 --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa7180b3ea0 con 0x7fa71815d5b0 2023-11-26T07:17:30.064 INFO:teuthology.orchestra.run.smithi188.stderr:2023-11-26T07:17:30.059+0000 7fa71e0c6640 1 --2- 172.21.15.188:0/2053583403 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fa71815d5b0 0x7fa7180d1820 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0x7fa7140097f0 tx=0x7fa71401c350 comp rx=0 tx=0).ready entity=mon.2 client_cookie=c8ff4f773f7a3c0f server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:17:30.064 INFO:teuthology.orchestra.run.smithi188.stderr:2023-11-26T07:17:30.059+0000 7fa71d0c4640 1 -- 172.21.15.188:0/2053583403 <== mon.2 v2:172.21.15.188:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa7140ab600 con 0x7fa71815d5b0 2023-11-26T07:17:30.064 INFO:teuthology.orchestra.run.smithi188.stderr:2023-11-26T07:17:30.059+0000 7fa71d0c4640 1 -- 172.21.15.188:0/2053583403 <== mon.2 v2:172.21.15.188:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa7140a7050 con 0x7fa71815d5b0 2023-11-26T07:17:30.064 INFO:teuthology.orchestra.run.smithi188.stderr:2023-11-26T07:17:30.059+0000 7fa71d0c4640 1 -- 172.21.15.188:0/2053583403 <== mon.2 v2:172.21.15.188:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa7140ab980 con 0x7fa71815d5b0 2023-11-26T07:17:30.065 INFO:teuthology.orchestra.run.smithi188.stderr:2023-11-26T07:17:30.059+0000 7fa72032a640 1 -- 172.21.15.188:0/2053583403 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fa71815d5b0 msgr2=0x7fa7180d1820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:17:30.065 INFO:teuthology.orchestra.run.smithi188.stderr:2023-11-26T07:17:30.059+0000 7fa72032a640 1 --2- 172.21.15.188:0/2053583403 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fa71815d5b0 0x7fa7180d1820 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0x7fa7140097f0 tx=0x7fa71401c350 comp rx=0 tx=0).stop 2023-11-26T07:17:30.065 INFO:teuthology.orchestra.run.smithi188.stderr:2023-11-26T07:17:30.063+0000 7fa72032a640 1 -- 172.21.15.188:0/2053583403 shutdown_connections 2023-11-26T07:17:30.065 INFO:teuthology.orchestra.run.smithi188.stderr:2023-11-26T07:17:30.063+0000 7fa72032a640 1 --2- 172.21.15.188:0/2053583403 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fa71815d5b0 0x7fa7180d1820 unknown :-1 s=CLOSED pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:17:30.065 INFO:teuthology.orchestra.run.smithi188.stderr:2023-11-26T07:17:30.063+0000 7fa72032a640 1 -- 172.21.15.188:0/2053583403 >> 172.21.15.188:0/2053583403 conn(0x7fa71808cc60 msgr2=0x7fa71809c580 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:17:30.065 INFO:teuthology.orchestra.run.smithi188.stderr:2023-11-26T07:17:30.063+0000 7fa72032a640 1 -- 172.21.15.188:0/2053583403 shutdown_connections 2023-11-26T07:17:30.065 INFO:teuthology.orchestra.run.smithi188.stderr:2023-11-26T07:17:30.063+0000 7fa72032a640 1 -- 172.21.15.188:0/2053583403 wait complete. 2023-11-26T07:17:30.066 INFO:teuthology.orchestra.run.smithi188.stderr:2023-11-26T07:17:30.063+0000 7fa72032a640 1 Processor -- start 2023-11-26T07:17:30.067 INFO:teuthology.orchestra.run.smithi188.stderr:2023-11-26T07:17:30.063+0000 7fa72032a640 1 -- start start 2023-11-26T07:17:30.070 INFO:teuthology.orchestra.run.smithi188.stderr:2023-11-26T07:17:30.063+0000 7fa72032a640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fa7180b3880 0x7fa7181f2190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:17:30.070 INFO:teuthology.orchestra.run.smithi188.stderr:2023-11-26T07:17:30.063+0000 7fa72032a640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fa7180d1d60 0x7fa7180b1e50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:17:30.070 INFO:teuthology.orchestra.run.smithi188.stderr:2023-11-26T07:17:30.063+0000 7fa72032a640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fa71815d5b0 0x7fa7180b2390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:17:30.070 INFO:teuthology.orchestra.run.smithi188.stderr:2023-11-26T07:17:30.063+0000 7fa72032a640 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa718157210 con 0x7fa7180b3880 2023-11-26T07:17:30.070 INFO:teuthology.orchestra.run.smithi188.stderr:2023-11-26T07:17:30.063+0000 7fa72032a640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa71815f230 con 0x7fa7180d1d60 2023-11-26T07:17:30.070 INFO:teuthology.orchestra.run.smithi188.stderr:2023-11-26T07:17:30.063+0000 7fa72032a640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa7180b0880 con 0x7fa71815d5b0 2023-11-26T07:17:30.070 INFO:teuthology.orchestra.run.smithi188.stderr:2023-11-26T07:17:30.063+0000 7fa71e8c7640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fa71815d5b0 0x7fa7180b2390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:17:30.070 INFO:teuthology.orchestra.run.smithi188.stderr:2023-11-26T07:17:30.063+0000 7fa71d8c5640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fa7180d1d60 0x7fa7180b1e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:17:30.070 INFO:teuthology.orchestra.run.smithi188.stderr:2023-11-26T07:17:30.063+0000 7fa71e8c7640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fa71815d5b0 0x7fa7180b2390 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.188:3301/0 says I am v2:172.21.15.188:35082/0 (socket says 172.21.15.188:35082) 2023-11-26T07:17:30.071 INFO:teuthology.orchestra.run.smithi188.stderr:2023-11-26T07:17:30.063+0000 7fa71e0c6640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fa7180b3880 0x7fa7181f2190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:17:30.071 INFO:teuthology.orchestra.run.smithi188.stderr:2023-11-26T07:17:30.063+0000 7fa71e8c7640 1 -- 172.21.15.188:0/1746955769 learned_addr learned my addr 172.21.15.188:0/1746955769 (peer_addr_for_me v2:172.21.15.188:0/0) 2023-11-26T07:17:30.071 INFO:teuthology.orchestra.run.smithi188.stderr:2023-11-26T07:17:30.063+0000 7fa71e8c7640 1 -- 172.21.15.188:0/1746955769 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fa7180d1d60 msgr2=0x7fa7180b1e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:17:30.071 INFO:teuthology.orchestra.run.smithi188.stderr:2023-11-26T07:17:30.063+0000 7fa71e8c7640 1 --2- 172.21.15.188:0/1746955769 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fa7180d1d60 0x7fa7180b1e50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:17:30.071 INFO:teuthology.orchestra.run.smithi188.stderr:2023-11-26T07:17:30.063+0000 7fa71e8c7640 1 -- 172.21.15.188:0/1746955769 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fa7180b3880 msgr2=0x7fa7181f2190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:17:30.071 INFO:teuthology.orchestra.run.smithi188.stderr:2023-11-26T07:17:30.063+0000 7fa71e8c7640 1 --2- 172.21.15.188:0/1746955769 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fa7180b3880 0x7fa7181f2190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:17:30.071 INFO:teuthology.orchestra.run.smithi188.stderr:2023-11-26T07:17:30.063+0000 7fa71e8c7640 1 -- 172.21.15.188:0/1746955769 --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa714009420 con 0x7fa71815d5b0 2023-11-26T07:17:30.071 INFO:teuthology.orchestra.run.smithi188.stderr:2023-11-26T07:17:30.063+0000 7fa71d8c5640 1 --2- 172.21.15.188:0/1746955769 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fa7180d1d60 0x7fa7180b1e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-26T07:17:30.071 INFO:teuthology.orchestra.run.smithi188.stderr:2023-11-26T07:17:30.067+0000 7fa71e8c7640 1 --2- 172.21.15.188:0/1746955769 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fa71815d5b0 0x7fa7180b2390 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7fa71000c810 tx=0x7fa71000ccd0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:17:30.071 INFO:teuthology.orchestra.run.smithi188.stderr:2023-11-26T07:17:30.067+0000 7fa70effd640 1 -- 172.21.15.188:0/1746955769 <== mon.2 v2:172.21.15.188:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa71000a640 con 0x7fa71815d5b0 2023-11-26T07:17:30.071 INFO:teuthology.orchestra.run.smithi188.stderr:2023-11-26T07:17:30.067+0000 7fa70effd640 1 -- 172.21.15.188:0/1746955769 <== mon.2 v2:172.21.15.188:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa71000a7c0 con 0x7fa71815d5b0 2023-11-26T07:17:30.071 INFO:teuthology.orchestra.run.smithi188.stderr:2023-11-26T07:17:30.067+0000 7fa70effd640 1 -- 172.21.15.188:0/1746955769 <== mon.2 v2:172.21.15.188:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa71000ab70 con 0x7fa71815d5b0 2023-11-26T07:17:30.071 INFO:teuthology.orchestra.run.smithi188.stderr:2023-11-26T07:17:30.067+0000 7fa72032a640 1 -- 172.21.15.188:0/1746955769 --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa7180b0b60 con 0x7fa71815d5b0 2023-11-26T07:17:30.071 INFO:teuthology.orchestra.run.smithi188.stderr:2023-11-26T07:17:30.067+0000 7fa72032a640 1 -- 172.21.15.188:0/1746955769 --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa7180b1120 con 0x7fa71815d5b0 2023-11-26T07:17:30.071 INFO:teuthology.orchestra.run.smithi188.stderr:2023-11-26T07:17:30.067+0000 7fa70effd640 1 -- 172.21.15.188:0/1746955769 <== mon.2 v2:172.21.15.188:3301/0 4 ==== mgrmap(e 1) v1 ==== 660+0+0 (secure 0 0 0) 0x7fa7100070d0 con 0x7fa71815d5b0 2023-11-26T07:17:30.072 INFO:teuthology.orchestra.run.smithi188.stderr:2023-11-26T07:17:30.067+0000 7fa70effd640 1 -- 172.21.15.188:0/1746955769 <== mon.2 v2:172.21.15.188:3301/0 5 ==== osd_map(5..5 src has 1..5) v4 ==== 1070+0+0 (secure 0 0 0) 0x7fa710007810 con 0x7fa71815d5b0 2023-11-26T07:17:30.072 INFO:teuthology.orchestra.run.smithi188.stderr:2023-11-26T07:17:30.067+0000 7fa72032a640 1 -- 172.21.15.188:0/1746955769 --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa7180b1340 con 0x7fa71815d5b0 2023-11-26T07:17:30.072 INFO:teuthology.orchestra.run.smithi188.stderr:2023-11-26T07:17:30.067+0000 7fa70effd640 1 -- 172.21.15.188:0/1746955769 <== mon.2 v2:172.21.15.188:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+72898 (secure 0 0 0) 0x7fa710018070 con 0x7fa71815d5b0 2023-11-26T07:17:30.138 INFO:teuthology.orchestra.run.smithi188.stderr:2023-11-26T07:17:30.135+0000 7fa72032a640 1 -- 172.21.15.188:0/1746955769 --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_command({"prefix": "osd new", "uuid": "d1f52435-7bc1-4081-9550-9cd1ba55f549", "id": 3} v 0) v1 -- 0x7fa718067680 con 0x7fa71815d5b0 2023-11-26T07:17:30.145 INFO:teuthology.orchestra.run.smithi188.stderr:2023-11-26T07:17:30.143+0000 7fa70effd640 1 -- 172.21.15.188:0/1746955769 <== mon.2 v2:172.21.15.188:3301/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "d1f52435-7bc1-4081-9550-9cd1ba55f549", "id": 3}]=0 v6) v1 ==== 112+0+2 (secure 0 0 0) 0x7fa710007380 con 0x7fa71815d5b0 2023-11-26T07:17:30.145 INFO:teuthology.orchestra.run.smithi188.stdout:3 2023-11-26T07:17:30.147 INFO:teuthology.orchestra.run.smithi188.stderr:2023-11-26T07:17:30.143+0000 7fa72032a640 1 -- 172.21.15.188:0/1746955769 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fa71815d5b0 msgr2=0x7fa7180b2390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:17:30.147 INFO:teuthology.orchestra.run.smithi188.stderr:2023-11-26T07:17:30.143+0000 7fa72032a640 1 --2- 172.21.15.188:0/1746955769 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fa71815d5b0 0x7fa7180b2390 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7fa71000c810 tx=0x7fa71000ccd0 comp rx=0 tx=0).stop 2023-11-26T07:17:30.147 INFO:teuthology.orchestra.run.smithi188.stderr:2023-11-26T07:17:30.143+0000 7fa72032a640 1 -- 172.21.15.188:0/1746955769 shutdown_connections 2023-11-26T07:17:30.147 INFO:teuthology.orchestra.run.smithi188.stderr:2023-11-26T07:17:30.143+0000 7fa72032a640 1 --2- 172.21.15.188:0/1746955769 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fa71815d5b0 0x7fa7180b2390 unknown :-1 s=CLOSED pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:17:30.147 INFO:teuthology.orchestra.run.smithi188.stderr:2023-11-26T07:17:30.143+0000 7fa72032a640 1 --2- 172.21.15.188:0/1746955769 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fa7180d1d60 0x7fa7180b1e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:17:30.147 INFO:teuthology.orchestra.run.smithi188.stderr:2023-11-26T07:17:30.143+0000 7fa72032a640 1 --2- 172.21.15.188:0/1746955769 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fa7180b3880 0x7fa7181f2190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:17:30.147 INFO:teuthology.orchestra.run.smithi188.stderr:2023-11-26T07:17:30.143+0000 7fa72032a640 1 -- 172.21.15.188:0/1746955769 >> 172.21.15.188:0/1746955769 conn(0x7fa71808cc60 msgr2=0x7fa7180c9850 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:17:30.147 INFO:teuthology.orchestra.run.smithi188.stderr:2023-11-26T07:17:30.143+0000 7fa72032a640 1 -- 172.21.15.188:0/1746955769 shutdown_connections 2023-11-26T07:17:30.147 INFO:teuthology.orchestra.run.smithi188.stderr:2023-11-26T07:17:30.143+0000 7fa72032a640 1 -- 172.21.15.188:0/1746955769 wait complete. 2023-11-26T07:17:30.165 DEBUG:teuthology.orchestra.run.smithi188:> sudo ceph --cluster ceph osd new e2ae8b6e-6e25-49b2-bcbf-aacce68f06a3 4 2023-11-26T07:17:30.258 INFO:tasks.ceph.mgr.y.smithi143.stderr:2023-11-26T07:17:30.255+0000 7f50c328d180 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2023-11-26T07:17:30.304 INFO:teuthology.orchestra.run.smithi188.stderr:2023-11-26T07:17:30.299+0000 7f67c5a32640 1 Processor -- start 2023-11-26T07:17:30.304 INFO:teuthology.orchestra.run.smithi188.stderr:2023-11-26T07:17:30.299+0000 7f67c5a32640 1 -- start start 2023-11-26T07:17:30.305 INFO:teuthology.orchestra.run.smithi188.stderr:2023-11-26T07:17:30.303+0000 7f67c5a32640 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7f67c00d1bc0 0x7f67c00d0df0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:17:30.305 INFO:teuthology.orchestra.run.smithi188.stderr:2023-11-26T07:17:30.303+0000 7f67c5a32640 1 -- --> v1:172.21.15.143:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f67c00cf0e0 con 0x7f67c015d6e0 2023-11-26T07:17:30.305 INFO:teuthology.orchestra.run.smithi188.stderr:2023-11-26T07:17:30.303+0000 7f67c5a32640 1 -- --> v1:172.21.15.188:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f67c00ceb20 con 0x7f67c00b2160 2023-11-26T07:17:30.305 INFO:teuthology.orchestra.run.smithi188.stderr:2023-11-26T07:17:30.303+0000 7f67c5a32640 1 -- --> v2:172.21.15.143:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f67c015dac0 con 0x7f67c00d1bc0 2023-11-26T07:17:30.305 INFO:teuthology.orchestra.run.smithi188.stderr:2023-11-26T07:17:30.303+0000 7f67bf7fe640 1 --1- >> v1:172.21.15.188:6789/0 conn(0x7f67c00b2160 0x7f67c00b0750 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.188:6789/0 says I am v1:172.21.15.188:41066/0 (socket says 172.21.15.188:41066) 2023-11-26T07:17:30.306 INFO:teuthology.orchestra.run.smithi188.stderr:2023-11-26T07:17:30.303+0000 7f67bf7fe640 1 -- 172.21.15.188:0/3508725789 learned_addr learned my addr 172.21.15.188:0/3508725789 (peer_addr_for_me v1:172.21.15.188:0/0) 2023-11-26T07:17:30.306 INFO:teuthology.orchestra.run.smithi188.stderr:2023-11-26T07:17:30.303+0000 7f67b7fff640 1 --2- 172.21.15.188:0/3508725789 >> v2:172.21.15.143:3300/0 conn(0x7f67c00d1bc0 0x7f67c00d0df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:17:30.307 INFO:teuthology.orchestra.run.smithi188.stderr:2023-11-26T07:17:30.303+0000 7f67be7fc640 1 -- 172.21.15.188:0/3508725789 <== mon.0 v1:172.21.15.143:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 55735905 0 0) 0x7f67c00cf0e0 con 0x7f67c015d6e0 2023-11-26T07:17:30.307 INFO:teuthology.orchestra.run.smithi188.stderr:2023-11-26T07:17:30.303+0000 7f67be7fc640 1 -- 172.21.15.188:0/3508725789 --> v1:172.21.15.143:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f67ac003530 con 0x7f67c015d6e0 2023-11-26T07:17:30.307 INFO:teuthology.orchestra.run.smithi188.stderr:2023-11-26T07:17:30.303+0000 7f67be7fc640 1 -- 172.21.15.188:0/3508725789 <== mon.1 v1:172.21.15.188:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3378280069 0 0) 0x7f67c00ceb20 con 0x7f67c00b2160 2023-11-26T07:17:30.307 INFO:teuthology.orchestra.run.smithi188.stderr:2023-11-26T07:17:30.303+0000 7f67be7fc640 1 -- 172.21.15.188:0/3508725789 --> v1:172.21.15.188:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f67c00cf0e0 con 0x7f67c00b2160 2023-11-26T07:17:30.307 INFO:teuthology.orchestra.run.smithi188.stderr:2023-11-26T07:17:30.303+0000 7f67b7fff640 1 -- 172.21.15.188:0/3508725789 >> v1:172.21.15.188:6789/0 conn(0x7f67c00b2160 legacy=0x7f67c00b0750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:17:30.307 INFO:teuthology.orchestra.run.smithi188.stderr:2023-11-26T07:17:30.303+0000 7f67b7fff640 1 -- 172.21.15.188:0/3508725789 >> v1:172.21.15.143:6789/0 conn(0x7f67c015d6e0 legacy=0x7f67c00d14b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:17:30.307 INFO:teuthology.orchestra.run.smithi188.stderr:2023-11-26T07:17:30.303+0000 7f67b7fff640 1 -- 172.21.15.188:0/3508725789 --> v2:172.21.15.143:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f67c00cee00 con 0x7f67c00d1bc0 2023-11-26T07:17:30.309 INFO:teuthology.orchestra.run.smithi188.stderr:2023-11-26T07:17:30.307+0000 7f67b7fff640 1 --2- 172.21.15.188:0/3508725789 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f67c00d1bc0 0x7f67c00d0df0 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7f67a80097c0 tx=0x7f67a801c480 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9a41706b9373eb58 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:17:30.309 INFO:teuthology.orchestra.run.smithi188.stderr:2023-11-26T07:17:30.307+0000 7f67be7fc640 1 -- 172.21.15.188:0/3508725789 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f67a80ab600 con 0x7f67c00d1bc0 2023-11-26T07:17:30.309 INFO:teuthology.orchestra.run.smithi188.stderr:2023-11-26T07:17:30.307+0000 7f67be7fc640 1 -- 172.21.15.188:0/3508725789 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f67a80a7050 con 0x7f67c00d1bc0 2023-11-26T07:17:30.310 INFO:teuthology.orchestra.run.smithi188.stderr:2023-11-26T07:17:30.307+0000 7f67be7fc640 1 -- 172.21.15.188:0/3508725789 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f67a80ab960 con 0x7f67c00d1bc0 2023-11-26T07:17:30.310 INFO:teuthology.orchestra.run.smithi188.stderr:2023-11-26T07:17:30.307+0000 7f67c5a32640 1 -- 172.21.15.188:0/3508725789 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f67c00d1bc0 msgr2=0x7f67c00d0df0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:17:30.310 INFO:teuthology.orchestra.run.smithi188.stderr:2023-11-26T07:17:30.307+0000 7f67c5a32640 1 --2- 172.21.15.188:0/3508725789 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f67c00d1bc0 0x7f67c00d0df0 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7f67a80097c0 tx=0x7f67a801c480 comp rx=0 tx=0).stop 2023-11-26T07:17:30.310 INFO:teuthology.orchestra.run.smithi188.stderr:2023-11-26T07:17:30.307+0000 7f67c5a32640 1 -- 172.21.15.188:0/3508725789 shutdown_connections 2023-11-26T07:17:30.310 INFO:teuthology.orchestra.run.smithi188.stderr:2023-11-26T07:17:30.307+0000 7f67c5a32640 1 --2- 172.21.15.188:0/3508725789 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f67c00d1bc0 0x7f67c00d0df0 unknown :-1 s=CLOSED pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:17:30.311 INFO:teuthology.orchestra.run.smithi188.stderr:2023-11-26T07:17:30.307+0000 7f67c5a32640 1 -- 172.21.15.188:0/3508725789 >> 172.21.15.188:0/3508725789 conn(0x7f67c009c550 msgr2=0x7f67c00d3220 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:17:30.311 INFO:teuthology.orchestra.run.smithi188.stderr:2023-11-26T07:17:30.307+0000 7f67c5a32640 1 -- 172.21.15.188:0/3508725789 shutdown_connections 2023-11-26T07:17:30.311 INFO:teuthology.orchestra.run.smithi188.stderr:2023-11-26T07:17:30.307+0000 7f67c5a32640 1 -- 172.21.15.188:0/3508725789 wait complete. 2023-11-26T07:17:30.312 INFO:teuthology.orchestra.run.smithi188.stderr:2023-11-26T07:17:30.307+0000 7f67c5a32640 1 Processor -- start 2023-11-26T07:17:30.312 INFO:teuthology.orchestra.run.smithi188.stderr:2023-11-26T07:17:30.307+0000 7f67c5a32640 1 -- start start 2023-11-26T07:17:30.313 INFO:teuthology.orchestra.run.smithi188.stderr:2023-11-26T07:17:30.311+0000 7f67c5a32640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f67c00b2160 0x7f67c00b8890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:17:30.313 INFO:teuthology.orchestra.run.smithi188.stderr:2023-11-26T07:17:30.311+0000 7f67c5a32640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f67c00d1bc0 0x7f67c0151690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:17:30.314 INFO:teuthology.orchestra.run.smithi188.stderr:2023-11-26T07:17:30.311+0000 7f67c5a32640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f67c015d6e0 0x7f67c006b9f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:17:30.314 INFO:teuthology.orchestra.run.smithi188.stderr:2023-11-26T07:17:30.311+0000 7f67c5a32640 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f67c00b7eb0 con 0x7f67c00d1bc0 2023-11-26T07:17:30.314 INFO:teuthology.orchestra.run.smithi188.stderr:2023-11-26T07:17:30.311+0000 7f67c5a32640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f67c00b7230 con 0x7f67c00b2160 2023-11-26T07:17:30.314 INFO:teuthology.orchestra.run.smithi188.stderr:2023-11-26T07:17:30.311+0000 7f67c5a32640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f67c00b7510 con 0x7f67c015d6e0 2023-11-26T07:17:30.314 INFO:teuthology.orchestra.run.smithi188.stderr:2023-11-26T07:17:30.311+0000 7f67beffd640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f67c00b2160 0x7f67c00b8890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:17:30.314 INFO:teuthology.orchestra.run.smithi188.stderr:2023-11-26T07:17:30.311+0000 7f67beffd640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f67c00b2160 0x7f67c00b8890 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.188:3300/0 says I am v2:172.21.15.188:37076/0 (socket says 172.21.15.188:37076) 2023-11-26T07:17:30.314 INFO:teuthology.orchestra.run.smithi188.stderr:2023-11-26T07:17:30.311+0000 7f67beffd640 1 -- 172.21.15.188:0/2676891189 learned_addr learned my addr 172.21.15.188:0/2676891189 (peer_addr_for_me v2:172.21.15.188:0/0) 2023-11-26T07:17:30.314 INFO:teuthology.orchestra.run.smithi188.stderr:2023-11-26T07:17:30.311+0000 7f67bf7fe640 1 --2- 172.21.15.188:0/2676891189 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f67c015d6e0 0x7f67c006b9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:17:30.314 INFO:teuthology.orchestra.run.smithi188.stderr:2023-11-26T07:17:30.311+0000 7f67b7fff640 1 --2- 172.21.15.188:0/2676891189 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f67c00d1bc0 0x7f67c0151690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:17:30.315 INFO:teuthology.orchestra.run.smithi188.stderr:2023-11-26T07:17:30.311+0000 7f67beffd640 1 -- 172.21.15.188:0/2676891189 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f67c015d6e0 msgr2=0x7f67c006b9f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:17:30.316 INFO:teuthology.orchestra.run.smithi188.stderr:2023-11-26T07:17:30.311+0000 7f67beffd640 1 --2- 172.21.15.188:0/2676891189 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f67c015d6e0 0x7f67c006b9f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:17:30.316 INFO:teuthology.orchestra.run.smithi188.stderr:2023-11-26T07:17:30.311+0000 7f67beffd640 1 -- 172.21.15.188:0/2676891189 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f67c00d1bc0 msgr2=0x7f67c0151690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:17:30.316 INFO:teuthology.orchestra.run.smithi188.stderr:2023-11-26T07:17:30.311+0000 7f67beffd640 1 --2- 172.21.15.188:0/2676891189 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f67c00d1bc0 0x7f67c0151690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:17:30.316 INFO:teuthology.orchestra.run.smithi188.stderr:2023-11-26T07:17:30.311+0000 7f67beffd640 1 -- 172.21.15.188:0/2676891189 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f67a8009420 con 0x7f67c00b2160 2023-11-26T07:17:30.316 INFO:teuthology.orchestra.run.smithi188.stderr:2023-11-26T07:17:30.311+0000 7f67beffd640 1 --2- 172.21.15.188:0/2676891189 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f67c00b2160 0x7f67c00b8890 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0x7f67b000c3c0 tx=0x7f67b000c880 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:17:30.316 INFO:teuthology.orchestra.run.smithi188.stderr:2023-11-26T07:17:30.311+0000 7f67bcff9640 1 -- 172.21.15.188:0/2676891189 <== mon.1 v2:172.21.15.188:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f67b00052d0 con 0x7f67c00b2160 2023-11-26T07:17:30.316 INFO:teuthology.orchestra.run.smithi188.stderr:2023-11-26T07:17:30.311+0000 7f67bcff9640 1 -- 172.21.15.188:0/2676891189 <== mon.1 v2:172.21.15.188:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f67b0005430 con 0x7f67c00b2160 2023-11-26T07:17:30.316 INFO:teuthology.orchestra.run.smithi188.stderr:2023-11-26T07:17:30.311+0000 7f67c5a32640 1 -- 172.21.15.188:0/2676891189 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f67c00b77f0 con 0x7f67c00b2160 2023-11-26T07:17:30.316 INFO:teuthology.orchestra.run.smithi188.stderr:2023-11-26T07:17:30.311+0000 7f67bcff9640 1 -- 172.21.15.188:0/2676891189 <== mon.1 v2:172.21.15.188:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f67b0003840 con 0x7f67c00b2160 2023-11-26T07:17:30.316 INFO:teuthology.orchestra.run.smithi188.stderr:2023-11-26T07:17:30.311+0000 7f67c5a32640 1 -- 172.21.15.188:0/2676891189 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f67c0151bd0 con 0x7f67c00b2160 2023-11-26T07:17:30.316 INFO:teuthology.orchestra.run.smithi188.stderr:2023-11-26T07:17:30.311+0000 7f67bcff9640 1 -- 172.21.15.188:0/2676891189 <== mon.1 v2:172.21.15.188:3300/0 4 ==== mgrmap(e 1) v1 ==== 660+0+0 (secure 0 0 0) 0x7f67b0003ca0 con 0x7f67c00b2160 2023-11-26T07:17:30.316 INFO:teuthology.orchestra.run.smithi188.stderr:2023-11-26T07:17:30.311+0000 7f67bcff9640 1 -- 172.21.15.188:0/2676891189 <== mon.1 v2:172.21.15.188:3300/0 5 ==== osd_map(6..6 src has 1..6) v4 ==== 1185+0+0 (secure 0 0 0) 0x7f67b0011aa0 con 0x7f67c00b2160 2023-11-26T07:17:30.316 INFO:teuthology.orchestra.run.smithi188.stderr:2023-11-26T07:17:30.311+0000 7f67c5a32640 1 -- 172.21.15.188:0/2676891189 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f67c00b1bb0 con 0x7f67c00b2160 2023-11-26T07:17:30.318 INFO:teuthology.orchestra.run.smithi188.stderr:2023-11-26T07:17:30.315+0000 7f67bcff9640 1 -- 172.21.15.188:0/2676891189 <== mon.1 v2:172.21.15.188:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+72898 (secure 0 0 0) 0x7f67b0017070 con 0x7f67c00b2160 2023-11-26T07:17:30.321 INFO:tasks.ceph.mgr.y.smithi143.stderr:2023-11-26T07:17:30.319+0000 7f50c328d180 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2023-11-26T07:17:30.381 INFO:tasks.ceph.mgr.y.smithi143.stderr:2023-11-26T07:17:30.379+0000 7f50c328d180 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2023-11-26T07:17:30.384 INFO:teuthology.orchestra.run.smithi188.stderr:2023-11-26T07:17:30.379+0000 7f67c5a32640 1 -- 172.21.15.188:0/2676891189 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_command({"prefix": "osd new", "uuid": "e2ae8b6e-6e25-49b2-bcbf-aacce68f06a3", "id": 4} v 0) v1 -- 0x7f67c0067660 con 0x7f67c00b2160 2023-11-26T07:17:30.391 INFO:teuthology.orchestra.run.smithi188.stderr:2023-11-26T07:17:30.387+0000 7f67bcff9640 1 -- 172.21.15.188:0/2676891189 <== mon.1 v2:172.21.15.188:3300/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "e2ae8b6e-6e25-49b2-bcbf-aacce68f06a3", "id": 4}]=0 v7) v1 ==== 112+0+2 (secure 0 0 0) 0x7f67b0011590 con 0x7f67c00b2160 2023-11-26T07:17:30.391 INFO:teuthology.orchestra.run.smithi188.stdout:4 2023-11-26T07:17:30.392 INFO:teuthology.orchestra.run.smithi188.stderr:2023-11-26T07:17:30.387+0000 7f67c5a32640 1 -- 172.21.15.188:0/2676891189 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f67c00b2160 msgr2=0x7f67c00b8890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:17:30.393 INFO:teuthology.orchestra.run.smithi188.stderr:2023-11-26T07:17:30.387+0000 7f67c5a32640 1 --2- 172.21.15.188:0/2676891189 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f67c00b2160 0x7f67c00b8890 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0x7f67b000c3c0 tx=0x7f67b000c880 comp rx=0 tx=0).stop 2023-11-26T07:17:30.393 INFO:teuthology.orchestra.run.smithi188.stderr:2023-11-26T07:17:30.387+0000 7f67c5a32640 1 -- 172.21.15.188:0/2676891189 shutdown_connections 2023-11-26T07:17:30.393 INFO:teuthology.orchestra.run.smithi188.stderr:2023-11-26T07:17:30.387+0000 7f67c5a32640 1 --2- 172.21.15.188:0/2676891189 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f67c015d6e0 0x7f67c006b9f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:17:30.393 INFO:teuthology.orchestra.run.smithi188.stderr:2023-11-26T07:17:30.387+0000 7f67c5a32640 1 --2- 172.21.15.188:0/2676891189 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f67c00d1bc0 0x7f67c0151690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:17:30.393 INFO:teuthology.orchestra.run.smithi188.stderr:2023-11-26T07:17:30.387+0000 7f67c5a32640 1 --2- 172.21.15.188:0/2676891189 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f67c00b2160 0x7f67c00b8890 unknown :-1 s=CLOSED pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:17:30.393 INFO:teuthology.orchestra.run.smithi188.stderr:2023-11-26T07:17:30.387+0000 7f67c5a32640 1 -- 172.21.15.188:0/2676891189 >> 172.21.15.188:0/2676891189 conn(0x7f67c009c550 msgr2=0x7f67c00734a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:17:30.393 INFO:teuthology.orchestra.run.smithi188.stderr:2023-11-26T07:17:30.387+0000 7f67c5a32640 1 -- 172.21.15.188:0/2676891189 shutdown_connections 2023-11-26T07:17:30.393 INFO:teuthology.orchestra.run.smithi188.stderr:2023-11-26T07:17:30.387+0000 7f67c5a32640 1 -- 172.21.15.188:0/2676891189 wait complete. 2023-11-26T07:17:30.405 INFO:tasks.ceph.mgr.x.smithi188.stderr:2023-11-26T07:17:30.399+0000 7f052864e180 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2023-11-26T07:17:30.411 DEBUG:teuthology.orchestra.run.smithi188:> sudo ceph --cluster ceph osd new 3cec8a32-7698-4c36-8e89-e4ba11934f33 5 2023-11-26T07:17:30.470 INFO:tasks.ceph.mgr.y.smithi143.stderr:2023-11-26T07:17:30.467+0000 7f50c328d180 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2023-11-26T07:17:30.472 INFO:tasks.ceph.mgr.x.smithi188.stderr:2023-11-26T07:17:30.467+0000 7f052864e180 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2023-11-26T07:17:30.529 INFO:tasks.ceph.mgr.y.smithi143.stderr:2023-11-26T07:17:30.527+0000 7f50c328d180 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2023-11-26T07:17:30.535 INFO:tasks.ceph.mgr.x.smithi188.stderr:2023-11-26T07:17:30.531+0000 7f052864e180 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2023-11-26T07:17:30.548 INFO:teuthology.orchestra.run.smithi188.stderr:2023-11-26T07:17:30.543+0000 7f972be6b640 1 Processor -- start 2023-11-26T07:17:30.548 INFO:teuthology.orchestra.run.smithi188.stderr:2023-11-26T07:17:30.543+0000 7f972be6b640 1 -- start start 2023-11-26T07:17:30.550 INFO:teuthology.orchestra.run.smithi188.stderr:2023-11-26T07:17:30.547+0000 7f972be6b640 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7f972415d6e0 0x7f97240d0df0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:17:30.550 INFO:teuthology.orchestra.run.smithi188.stderr:2023-11-26T07:17:30.547+0000 7f972be6b640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f97240d1bc0 0x7f97240b0750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:17:30.550 INFO:teuthology.orchestra.run.smithi188.stderr:2023-11-26T07:17:30.547+0000 7f972be6b640 1 -- --> v1:172.21.15.188:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f97240cf0e0 con 0x7f97240b2160 2023-11-26T07:17:30.550 INFO:teuthology.orchestra.run.smithi188.stderr:2023-11-26T07:17:30.547+0000 7f972be6b640 1 -- --> v2:172.21.15.143:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f972415dac0 con 0x7f972415d6e0 2023-11-26T07:17:30.550 INFO:teuthology.orchestra.run.smithi188.stderr:2023-11-26T07:17:30.547+0000 7f972be6b640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f97240ceb20 con 0x7f97240d1bc0 2023-11-26T07:17:30.551 INFO:teuthology.orchestra.run.smithi188.stderr:2023-11-26T07:17:30.547+0000 7f972a408640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f97240d1bc0 0x7f97240b0750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:17:30.551 INFO:teuthology.orchestra.run.smithi188.stderr:2023-11-26T07:17:30.547+0000 7f9729406640 1 --1- >> v1:172.21.15.188:6789/0 conn(0x7f97240b2160 0x7f97240d14b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.188:6789/0 says I am v1:172.21.15.188:41070/0 (socket says 172.21.15.188:41070) 2023-11-26T07:17:30.551 INFO:teuthology.orchestra.run.smithi188.stderr:2023-11-26T07:17:30.547+0000 7f9729406640 1 -- 172.21.15.188:0/2022761568 learned_addr learned my addr 172.21.15.188:0/2022761568 (peer_addr_for_me v1:172.21.15.188:0/0) 2023-11-26T07:17:30.551 INFO:teuthology.orchestra.run.smithi188.stderr:2023-11-26T07:17:30.547+0000 7f9729c07640 1 --2- 172.21.15.188:0/2022761568 >> v2:172.21.15.143:3300/0 conn(0x7f972415d6e0 0x7f97240d0df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:17:30.551 INFO:teuthology.orchestra.run.smithi188.stderr:2023-11-26T07:17:30.547+0000 7f972a408640 1 -- 172.21.15.188:0/2022761568 >> v2:172.21.15.143:3300/0 conn(0x7f972415d6e0 msgr2=0x7f97240d0df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-26T07:17:30.551 INFO:teuthology.orchestra.run.smithi188.stderr:2023-11-26T07:17:30.547+0000 7f972a408640 1 --2- 172.21.15.188:0/2022761568 >> v2:172.21.15.143:3300/0 conn(0x7f972415d6e0 0x7f97240d0df0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:17:30.551 INFO:teuthology.orchestra.run.smithi188.stderr:2023-11-26T07:17:30.547+0000 7f972a408640 1 -- 172.21.15.188:0/2022761568 >> v1:172.21.15.188:6789/0 conn(0x7f97240b2160 legacy=0x7f97240d14b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:17:30.552 INFO:teuthology.orchestra.run.smithi188.stderr:2023-11-26T07:17:30.547+0000 7f972a408640 1 -- 172.21.15.188:0/2022761568 --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f97240cee00 con 0x7f97240d1bc0 2023-11-26T07:17:30.552 INFO:teuthology.orchestra.run.smithi188.stderr:2023-11-26T07:17:30.547+0000 7f9728c05640 1 -- 172.21.15.188:0/2022761568 <== mon.1 v1:172.21.15.188:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3367035792 0 0) 0x7f97240cf0e0 con 0x7f97240b2160 2023-11-26T07:17:30.554 INFO:teuthology.orchestra.run.smithi188.stderr:2023-11-26T07:17:30.551+0000 7f972a408640 1 --2- 172.21.15.188:0/2022761568 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f97240d1bc0 0x7f97240b0750 secure :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0x7f971c009d90 tx=0x7f971c01c950 comp rx=0 tx=0).ready entity=mon.2 client_cookie=2f613cbaf42ff2f4 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:17:30.554 INFO:teuthology.orchestra.run.smithi188.stderr:2023-11-26T07:17:30.551+0000 7f9728c05640 1 -- 172.21.15.188:0/2022761568 <== mon.2 v2:172.21.15.188:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f971c0a5900 con 0x7f97240d1bc0 2023-11-26T07:17:30.554 INFO:teuthology.orchestra.run.smithi188.stderr:2023-11-26T07:17:30.551+0000 7f9728c05640 1 -- 172.21.15.188:0/2022761568 <== mon.2 v2:172.21.15.188:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f971c0a5a60 con 0x7f97240d1bc0 2023-11-26T07:17:30.554 INFO:teuthology.orchestra.run.smithi188.stderr:2023-11-26T07:17:30.551+0000 7f9728c05640 1 -- 172.21.15.188:0/2022761568 <== mon.2 v2:172.21.15.188:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f971c0a5df0 con 0x7f97240d1bc0 2023-11-26T07:17:30.555 INFO:teuthology.orchestra.run.smithi188.stderr:2023-11-26T07:17:30.551+0000 7f972be6b640 1 -- 172.21.15.188:0/2022761568 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f97240d1bc0 msgr2=0x7f97240b0750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:17:30.555 INFO:teuthology.orchestra.run.smithi188.stderr:2023-11-26T07:17:30.551+0000 7f972be6b640 1 --2- 172.21.15.188:0/2022761568 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f97240d1bc0 0x7f97240b0750 secure :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0x7f971c009d90 tx=0x7f971c01c950 comp rx=0 tx=0).stop 2023-11-26T07:17:30.555 INFO:teuthology.orchestra.run.smithi188.stderr:2023-11-26T07:17:30.551+0000 7f972be6b640 1 -- 172.21.15.188:0/2022761568 shutdown_connections 2023-11-26T07:17:30.555 INFO:teuthology.orchestra.run.smithi188.stderr:2023-11-26T07:17:30.551+0000 7f972be6b640 1 --2- 172.21.15.188:0/2022761568 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f97240d1bc0 0x7f97240b0750 unknown :-1 s=CLOSED pgs=15 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:17:30.555 INFO:teuthology.orchestra.run.smithi188.stderr:2023-11-26T07:17:30.551+0000 7f972be6b640 1 --2- 172.21.15.188:0/2022761568 >> v2:172.21.15.143:3300/0 conn(0x7f972415d6e0 0x7f97240d0df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:17:30.555 INFO:teuthology.orchestra.run.smithi188.stderr:2023-11-26T07:17:30.551+0000 7f972be6b640 1 -- 172.21.15.188:0/2022761568 >> 172.21.15.188:0/2022761568 conn(0x7f972409c550 msgr2=0x7f97240d3220 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:17:30.556 INFO:teuthology.orchestra.run.smithi188.stderr:2023-11-26T07:17:30.551+0000 7f972be6b640 1 -- 172.21.15.188:0/2022761568 shutdown_connections 2023-11-26T07:17:30.556 INFO:teuthology.orchestra.run.smithi188.stderr:2023-11-26T07:17:30.551+0000 7f972be6b640 1 -- 172.21.15.188:0/2022761568 wait complete. 2023-11-26T07:17:30.557 INFO:teuthology.orchestra.run.smithi188.stderr:2023-11-26T07:17:30.555+0000 7f972be6b640 1 Processor -- start 2023-11-26T07:17:30.557 INFO:teuthology.orchestra.run.smithi188.stderr:2023-11-26T07:17:30.555+0000 7f972be6b640 1 -- start start 2023-11-26T07:17:30.558 INFO:teuthology.orchestra.run.smithi188.stderr:2023-11-26T07:17:30.555+0000 7f972be6b640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f97240b2160 0x7f972415a710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:17:30.558 INFO:teuthology.orchestra.run.smithi188.stderr:2023-11-26T07:17:30.555+0000 7f972be6b640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f97240d1bc0 0x7f972415c550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:17:30.558 INFO:teuthology.orchestra.run.smithi188.stderr:2023-11-26T07:17:30.555+0000 7f972be6b640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f972415d6e0 0x7f972414e720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:17:30.558 INFO:teuthology.orchestra.run.smithi188.stderr:2023-11-26T07:17:30.555+0000 7f972be6b640 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9724151100 con 0x7f97240b2160 2023-11-26T07:17:30.558 INFO:teuthology.orchestra.run.smithi188.stderr:2023-11-26T07:17:30.555+0000 7f972be6b640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f972414fcb0 con 0x7f97240d1bc0 2023-11-26T07:17:30.558 INFO:teuthology.orchestra.run.smithi188.stderr:2023-11-26T07:17:30.555+0000 7f972be6b640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f972414ff50 con 0x7f972415d6e0 2023-11-26T07:17:30.560 INFO:teuthology.orchestra.run.smithi188.stderr:2023-11-26T07:17:30.555+0000 7f9729406640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f97240d1bc0 0x7f972415c550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:17:30.560 INFO:teuthology.orchestra.run.smithi188.stderr:2023-11-26T07:17:30.555+0000 7f9729406640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f97240d1bc0 0x7f972415c550 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.188:3300/0 says I am v2:172.21.15.188:37082/0 (socket says 172.21.15.188:37082) 2023-11-26T07:17:30.560 INFO:teuthology.orchestra.run.smithi188.stderr:2023-11-26T07:17:30.555+0000 7f9729406640 1 -- 172.21.15.188:0/1529161 learned_addr learned my addr 172.21.15.188:0/1529161 (peer_addr_for_me v2:172.21.15.188:0/0) 2023-11-26T07:17:30.560 INFO:teuthology.orchestra.run.smithi188.stderr:2023-11-26T07:17:30.555+0000 7f972a408640 1 --2- 172.21.15.188:0/1529161 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f972415d6e0 0x7f972414e720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:17:30.560 INFO:teuthology.orchestra.run.smithi188.stderr:2023-11-26T07:17:30.555+0000 7f9729c07640 1 --2- 172.21.15.188:0/1529161 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f97240b2160 0x7f972415a710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:17:30.560 INFO:teuthology.orchestra.run.smithi188.stderr:2023-11-26T07:17:30.555+0000 7f972a408640 1 -- 172.21.15.188:0/1529161 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f97240d1bc0 msgr2=0x7f972415c550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:17:30.560 INFO:teuthology.orchestra.run.smithi188.stderr:2023-11-26T07:17:30.555+0000 7f972a408640 1 --2- 172.21.15.188:0/1529161 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f97240d1bc0 0x7f972415c550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:17:30.560 INFO:teuthology.orchestra.run.smithi188.stderr:2023-11-26T07:17:30.555+0000 7f972a408640 1 -- 172.21.15.188:0/1529161 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f97240b2160 msgr2=0x7f972415a710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:17:30.560 INFO:teuthology.orchestra.run.smithi188.stderr:2023-11-26T07:17:30.555+0000 7f972a408640 1 --2- 172.21.15.188:0/1529161 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f97240b2160 0x7f972415a710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:17:30.561 INFO:teuthology.orchestra.run.smithi188.stderr:2023-11-26T07:17:30.555+0000 7f972a408640 1 -- 172.21.15.188:0/1529161 --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f971c009a70 con 0x7f972415d6e0 2023-11-26T07:17:30.561 INFO:teuthology.orchestra.run.smithi188.stderr:2023-11-26T07:17:30.555+0000 7f9729406640 1 --2- 172.21.15.188:0/1529161 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f97240d1bc0 0x7f972415c550 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-26T07:17:30.561 INFO:teuthology.orchestra.run.smithi188.stderr:2023-11-26T07:17:30.555+0000 7f972a408640 1 --2- 172.21.15.188:0/1529161 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f972415d6e0 0x7f972414e720 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7f971c0ac000 tx=0x7f971c0196f0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:17:30.561 INFO:teuthology.orchestra.run.smithi188.stderr:2023-11-26T07:17:30.555+0000 7f971a7fc640 1 -- 172.21.15.188:0/1529161 <== mon.2 v2:172.21.15.188:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f971c0a58f0 con 0x7f972415d6e0 2023-11-26T07:17:30.561 INFO:teuthology.orchestra.run.smithi188.stderr:2023-11-26T07:17:30.555+0000 7f971a7fc640 1 -- 172.21.15.188:0/1529161 <== mon.2 v2:172.21.15.188:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f971c0b9660 con 0x7f972415d6e0 2023-11-26T07:17:30.561 INFO:teuthology.orchestra.run.smithi188.stderr:2023-11-26T07:17:30.555+0000 7f971a7fc640 1 -- 172.21.15.188:0/1529161 <== mon.2 v2:172.21.15.188:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f971c0b99f0 con 0x7f972415d6e0 2023-11-26T07:17:30.561 INFO:teuthology.orchestra.run.smithi188.stderr:2023-11-26T07:17:30.555+0000 7f972be6b640 1 -- 172.21.15.188:0/1529161 --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9724150230 con 0x7f972415d6e0 2023-11-26T07:17:30.561 INFO:teuthology.orchestra.run.smithi188.stderr:2023-11-26T07:17:30.555+0000 7f972be6b640 1 -- 172.21.15.188:0/1529161 --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f972415ac50 con 0x7f972415d6e0 2023-11-26T07:17:30.561 INFO:teuthology.orchestra.run.smithi188.stderr:2023-11-26T07:17:30.555+0000 7f971a7fc640 1 -- 172.21.15.188:0/1529161 <== mon.2 v2:172.21.15.188:3301/0 4 ==== mgrmap(e 1) v1 ==== 660+0+0 (secure 0 0 0) 0x7f971c0ad440 con 0x7f972415d6e0 2023-11-26T07:17:30.561 INFO:teuthology.orchestra.run.smithi188.stderr:2023-11-26T07:17:30.555+0000 7f971a7fc640 1 -- 172.21.15.188:0/1529161 <== mon.2 v2:172.21.15.188:3301/0 5 ==== osd_map(7..7 src has 1..7) v4 ==== 1300+0+0 (secure 0 0 0) 0x7f971c0c2990 con 0x7f972415d6e0 2023-11-26T07:17:30.562 INFO:teuthology.orchestra.run.smithi188.stderr:2023-11-26T07:17:30.555+0000 7f972be6b640 1 -- 172.21.15.188:0/1529161 --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f972406cd50 con 0x7f972415d6e0 2023-11-26T07:17:30.562 INFO:teuthology.orchestra.run.smithi188.stderr:2023-11-26T07:17:30.559+0000 7f971a7fc640 1 -- 172.21.15.188:0/1529161 <== mon.2 v2:172.21.15.188:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+72898 (secure 0 0 0) 0x7f971c0bd030 con 0x7f972415d6e0 2023-11-26T07:17:30.618 INFO:tasks.ceph.mgr.y.smithi143.stderr:2023-11-26T07:17:30.615+0000 7f50c328d180 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2023-11-26T07:17:30.641 INFO:tasks.ceph.mgr.x.smithi188.stderr:2023-11-26T07:17:30.639+0000 7f052864e180 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2023-11-26T07:17:30.644 INFO:teuthology.orchestra.run.smithi188.stderr:2023-11-26T07:17:30.639+0000 7f972be6b640 1 -- 172.21.15.188:0/1529161 --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_command({"prefix": "osd new", "uuid": "3cec8a32-7698-4c36-8e89-e4ba11934f33", "id": 5} v 0) v1 -- 0x7f9724151270 con 0x7f972415d6e0 2023-11-26T07:17:30.649 INFO:teuthology.orchestra.run.smithi188.stderr:2023-11-26T07:17:30.647+0000 7f971a7fc640 1 -- 172.21.15.188:0/1529161 <== mon.2 v2:172.21.15.188:3301/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "3cec8a32-7698-4c36-8e89-e4ba11934f33", "id": 5}]=0 v8) v1 ==== 112+0+2 (secure 0 0 0) 0x7f971c0d10b0 con 0x7f972415d6e0 2023-11-26T07:17:30.649 INFO:teuthology.orchestra.run.smithi188.stdout:5 2023-11-26T07:17:30.651 INFO:teuthology.orchestra.run.smithi188.stderr:2023-11-26T07:17:30.647+0000 7f972be6b640 1 -- 172.21.15.188:0/1529161 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f972415d6e0 msgr2=0x7f972414e720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:17:30.651 INFO:teuthology.orchestra.run.smithi188.stderr:2023-11-26T07:17:30.647+0000 7f972be6b640 1 --2- 172.21.15.188:0/1529161 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f972415d6e0 0x7f972414e720 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7f971c0ac000 tx=0x7f971c0196f0 comp rx=0 tx=0).stop 2023-11-26T07:17:30.651 INFO:teuthology.orchestra.run.smithi188.stderr:2023-11-26T07:17:30.647+0000 7f972be6b640 1 -- 172.21.15.188:0/1529161 shutdown_connections 2023-11-26T07:17:30.651 INFO:teuthology.orchestra.run.smithi188.stderr:2023-11-26T07:17:30.647+0000 7f972be6b640 1 --2- 172.21.15.188:0/1529161 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f972415d6e0 0x7f972414e720 unknown :-1 s=CLOSED pgs=16 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:17:30.651 INFO:teuthology.orchestra.run.smithi188.stderr:2023-11-26T07:17:30.647+0000 7f972be6b640 1 --2- 172.21.15.188:0/1529161 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f97240d1bc0 0x7f972415c550 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:17:30.651 INFO:teuthology.orchestra.run.smithi188.stderr:2023-11-26T07:17:30.647+0000 7f972be6b640 1 --2- 172.21.15.188:0/1529161 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f97240b2160 0x7f972415a710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:17:30.651 INFO:teuthology.orchestra.run.smithi188.stderr:2023-11-26T07:17:30.647+0000 7f972be6b640 1 -- 172.21.15.188:0/1529161 >> 172.21.15.188:0/1529161 conn(0x7f972409c550 msgr2=0x7f9724159c10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:17:30.651 INFO:teuthology.orchestra.run.smithi188.stderr:2023-11-26T07:17:30.647+0000 7f972be6b640 1 -- 172.21.15.188:0/1529161 shutdown_connections 2023-11-26T07:17:30.651 INFO:teuthology.orchestra.run.smithi188.stderr:2023-11-26T07:17:30.647+0000 7f972be6b640 1 -- 172.21.15.188:0/1529161 wait complete. 2023-11-26T07:17:30.669 DEBUG:teuthology.orchestra.run.smithi188:> sudo ceph --cluster ceph osd new 58a65d5d-e090-473a-81b4-ad38e85029f9 6 2023-11-26T07:17:30.680 INFO:tasks.ceph.mgr.y.smithi143.stderr:2023-11-26T07:17:30.675+0000 7f50c328d180 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2023-11-26T07:17:30.706 INFO:tasks.ceph.mgr.x.smithi188.stderr:2023-11-26T07:17:30.703+0000 7f052864e180 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2023-11-26T07:17:30.798 INFO:tasks.ceph.mgr.x.smithi188.stderr:2023-11-26T07:17:30.795+0000 7f052864e180 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2023-11-26T07:17:30.806 INFO:teuthology.orchestra.run.smithi188.stderr:2023-11-26T07:17:30.803+0000 7fc1ce58d640 1 Processor -- start 2023-11-26T07:17:30.806 INFO:teuthology.orchestra.run.smithi188.stderr:2023-11-26T07:17:30.803+0000 7fc1ce58d640 1 -- start start 2023-11-26T07:17:30.808 INFO:teuthology.orchestra.run.smithi188.stderr:2023-11-26T07:17:30.803+0000 7fc1ce58d640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fc1c815d6e0 0x7fc1c80d0df0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:17:30.808 INFO:teuthology.orchestra.run.smithi188.stderr:2023-11-26T07:17:30.803+0000 7fc1ce58d640 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7fc1c80b2160 0x7fc1c80d14b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:17:30.808 INFO:teuthology.orchestra.run.smithi188.stderr:2023-11-26T07:17:30.803+0000 7fc1ce58d640 1 -- --> v1:172.21.15.188:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc1c80cf0e0 con 0x7fc1c80d19f0 2023-11-26T07:17:30.808 INFO:teuthology.orchestra.run.smithi188.stderr:2023-11-26T07:17:30.803+0000 7fc1ce58d640 1 -- --> v2:172.21.15.143:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc1c815dac0 con 0x7fc1c80b2160 2023-11-26T07:17:30.808 INFO:teuthology.orchestra.run.smithi188.stderr:2023-11-26T07:17:30.803+0000 7fc1ce58d640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc1c80d1e80 con 0x7fc1c815d6e0 2023-11-26T07:17:30.808 INFO:teuthology.orchestra.run.smithi188.stderr:2023-11-26T07:17:30.803+0000 7fc1c7fff640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fc1c815d6e0 0x7fc1c80d0df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:17:30.808 INFO:teuthology.orchestra.run.smithi188.stderr:2023-11-26T07:17:30.803+0000 7fc1c7fff640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fc1c815d6e0 0x7fc1c80d0df0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.188:3301/0 says I am v2:172.21.15.188:35114/0 (socket says 172.21.15.188:35114) 2023-11-26T07:17:30.808 INFO:teuthology.orchestra.run.smithi188.stderr:2023-11-26T07:17:30.803+0000 7fc1c7fff640 1 -- 172.21.15.188:0/952034118 learned_addr learned my addr 172.21.15.188:0/952034118 (peer_addr_for_me v2:172.21.15.188:0/0) 2023-11-26T07:17:30.808 INFO:teuthology.orchestra.run.smithi188.stderr:2023-11-26T07:17:30.803+0000 7fc1c77fe640 1 --2- 172.21.15.188:0/952034118 >> v2:172.21.15.143:3300/0 conn(0x7fc1c80b2160 0x7fc1c80d14b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:17:30.808 INFO:teuthology.orchestra.run.smithi188.stderr:2023-11-26T07:17:30.803+0000 7fc1c7fff640 1 -- 172.21.15.188:0/952034118 >> v2:172.21.15.143:3300/0 conn(0x7fc1c80b2160 msgr2=0x7fc1c80d14b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-26T07:17:30.808 INFO:teuthology.orchestra.run.smithi188.stderr:2023-11-26T07:17:30.803+0000 7fc1c7fff640 1 --2- 172.21.15.188:0/952034118 >> v2:172.21.15.143:3300/0 conn(0x7fc1c80b2160 0x7fc1c80d14b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:17:30.808 INFO:teuthology.orchestra.run.smithi188.stderr:2023-11-26T07:17:30.803+0000 7fc1c7fff640 1 -- 172.21.15.188:0/952034118 >> v1:172.21.15.188:6789/0 conn(0x7fc1c80d19f0 legacy=0x7fc1c80cfbf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:17:30.809 INFO:teuthology.orchestra.run.smithi188.stderr:2023-11-26T07:17:30.803+0000 7fc1c7fff640 1 -- 172.21.15.188:0/952034118 --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc1c80ceb20 con 0x7fc1c815d6e0 2023-11-26T07:17:30.810 INFO:teuthology.orchestra.run.smithi188.stderr:2023-11-26T07:17:30.807+0000 7fc1c7fff640 1 --2- 172.21.15.188:0/952034118 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fc1c815d6e0 0x7fc1c80d0df0 secure :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0x7fc1b8009870 tx=0x7fc1b801c540 comp rx=0 tx=0).ready entity=mon.2 client_cookie=3b6e00d2783b038 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:17:30.811 INFO:teuthology.orchestra.run.smithi188.stderr:2023-11-26T07:17:30.807+0000 7fc1c6ffd640 1 -- 172.21.15.188:0/952034118 <== mon.2 v2:172.21.15.188:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc1b80a7070 con 0x7fc1c815d6e0 2023-11-26T07:17:30.811 INFO:teuthology.orchestra.run.smithi188.stderr:2023-11-26T07:17:30.807+0000 7fc1c6ffd640 1 -- 172.21.15.188:0/952034118 <== mon.2 v2:172.21.15.188:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc1b80ab430 con 0x7fc1c815d6e0 2023-11-26T07:17:30.811 INFO:teuthology.orchestra.run.smithi188.stderr:2023-11-26T07:17:30.807+0000 7fc1c6ffd640 1 -- 172.21.15.188:0/952034118 <== mon.2 v2:172.21.15.188:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc1b80ab7c0 con 0x7fc1c815d6e0 2023-11-26T07:17:30.812 INFO:teuthology.orchestra.run.smithi188.stderr:2023-11-26T07:17:30.807+0000 7fc1ce58d640 1 -- 172.21.15.188:0/952034118 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fc1c815d6e0 msgr2=0x7fc1c80d0df0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:17:30.812 INFO:teuthology.orchestra.run.smithi188.stderr:2023-11-26T07:17:30.807+0000 7fc1ce58d640 1 --2- 172.21.15.188:0/952034118 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fc1c815d6e0 0x7fc1c80d0df0 secure :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0x7fc1b8009870 tx=0x7fc1b801c540 comp rx=0 tx=0).stop 2023-11-26T07:17:30.812 INFO:teuthology.orchestra.run.smithi188.stderr:2023-11-26T07:17:30.807+0000 7fc1ce58d640 1 -- 172.21.15.188:0/952034118 shutdown_connections 2023-11-26T07:17:30.812 INFO:teuthology.orchestra.run.smithi188.stderr:2023-11-26T07:17:30.807+0000 7fc1ce58d640 1 --2- 172.21.15.188:0/952034118 >> v2:172.21.15.143:3300/0 conn(0x7fc1c80b2160 0x7fc1c80d14b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:17:30.812 INFO:teuthology.orchestra.run.smithi188.stderr:2023-11-26T07:17:30.807+0000 7fc1ce58d640 1 --2- 172.21.15.188:0/952034118 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fc1c815d6e0 0x7fc1c80d0df0 unknown :-1 s=CLOSED pgs=17 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:17:30.812 INFO:teuthology.orchestra.run.smithi188.stderr:2023-11-26T07:17:30.807+0000 7fc1ce58d640 1 -- 172.21.15.188:0/952034118 >> 172.21.15.188:0/952034118 conn(0x7fc1c809c550 msgr2=0x7fc1c80d3220 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:17:30.813 INFO:teuthology.orchestra.run.smithi188.stderr:2023-11-26T07:17:30.807+0000 7fc1ce58d640 1 -- 172.21.15.188:0/952034118 shutdown_connections 2023-11-26T07:17:30.813 INFO:teuthology.orchestra.run.smithi188.stderr:2023-11-26T07:17:30.807+0000 7fc1ce58d640 1 -- 172.21.15.188:0/952034118 wait complete. 2023-11-26T07:17:30.813 INFO:teuthology.orchestra.run.smithi188.stderr:2023-11-26T07:17:30.811+0000 7fc1ce58d640 1 Processor -- start 2023-11-26T07:17:30.814 INFO:teuthology.orchestra.run.smithi188.stderr:2023-11-26T07:17:30.811+0000 7fc1ce58d640 1 -- start start 2023-11-26T07:17:30.815 INFO:teuthology.orchestra.run.smithi188.stderr:2023-11-26T07:17:30.811+0000 7fc1ce58d640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fc1c80b2160 0x7fc1c8153ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:17:30.815 INFO:teuthology.orchestra.run.smithi188.stderr:2023-11-26T07:17:30.811+0000 7fc1ce58d640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fc1c80d19f0 0x7fc1c8155450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:17:30.815 INFO:teuthology.orchestra.run.smithi188.stderr:2023-11-26T07:17:30.811+0000 7fc1ce58d640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fc1c815d6e0 0x7fc1c809cfc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:17:30.816 INFO:teuthology.orchestra.run.smithi188.stderr:2023-11-26T07:17:30.811+0000 7fc1ce58d640 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc1c80b7d70 con 0x7fc1c80d19f0 2023-11-26T07:17:30.816 INFO:teuthology.orchestra.run.smithi188.stderr:2023-11-26T07:17:30.811+0000 7fc1ce58d640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc1c80b8050 con 0x7fc1c815d6e0 2023-11-26T07:17:30.816 INFO:teuthology.orchestra.run.smithi188.stderr:2023-11-26T07:17:30.811+0000 7fc1ce58d640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc1c80b8330 con 0x7fc1c80b2160 2023-11-26T07:17:30.816 INFO:teuthology.orchestra.run.smithi188.stderr:2023-11-26T07:17:30.811+0000 7fc1c7fff640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fc1c80b2160 0x7fc1c8153ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:17:30.816 INFO:teuthology.orchestra.run.smithi188.stderr:2023-11-26T07:17:30.811+0000 7fc1ccb2a640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fc1c815d6e0 0x7fc1c809cfc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:17:30.816 INFO:teuthology.orchestra.run.smithi188.stderr:2023-11-26T07:17:30.811+0000 7fc1c7fff640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fc1c80b2160 0x7fc1c8153ad0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.188:3301/0 says I am v2:172.21.15.188:35118/0 (socket says 172.21.15.188:35118) 2023-11-26T07:17:30.816 INFO:teuthology.orchestra.run.smithi188.stderr:2023-11-26T07:17:30.811+0000 7fc1c7fff640 1 -- 172.21.15.188:0/392596585 learned_addr learned my addr 172.21.15.188:0/392596585 (peer_addr_for_me v2:172.21.15.188:0/0) 2023-11-26T07:17:30.816 INFO:teuthology.orchestra.run.smithi188.stderr:2023-11-26T07:17:30.811+0000 7fc1c77fe640 1 --2- 172.21.15.188:0/392596585 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fc1c80d19f0 0x7fc1c8155450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:17:30.816 INFO:teuthology.orchestra.run.smithi188.stderr:2023-11-26T07:17:30.811+0000 7fc1c7fff640 1 -- 172.21.15.188:0/392596585 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fc1c815d6e0 msgr2=0x7fc1c809cfc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:17:30.816 INFO:teuthology.orchestra.run.smithi188.stderr:2023-11-26T07:17:30.811+0000 7fc1c7fff640 1 --2- 172.21.15.188:0/392596585 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fc1c815d6e0 0x7fc1c809cfc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:17:30.816 INFO:teuthology.orchestra.run.smithi188.stderr:2023-11-26T07:17:30.811+0000 7fc1c7fff640 1 -- 172.21.15.188:0/392596585 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fc1c80d19f0 msgr2=0x7fc1c8155450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:17:30.816 INFO:teuthology.orchestra.run.smithi188.stderr:2023-11-26T07:17:30.811+0000 7fc1c7fff640 1 --2- 172.21.15.188:0/392596585 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fc1c80d19f0 0x7fc1c8155450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:17:30.816 INFO:teuthology.orchestra.run.smithi188.stderr:2023-11-26T07:17:30.811+0000 7fc1c7fff640 1 -- 172.21.15.188:0/392596585 --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc1b80094d0 con 0x7fc1c80b2160 2023-11-26T07:17:30.817 INFO:teuthology.orchestra.run.smithi188.stderr:2023-11-26T07:17:30.811+0000 7fc1c7fff640 1 --2- 172.21.15.188:0/392596585 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fc1c80b2160 0x7fc1c8153ad0 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7fc1b80ad040 tx=0x7fc1b80192e0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:17:30.817 INFO:teuthology.orchestra.run.smithi188.stderr:2023-11-26T07:17:30.811+0000 7fc1c4ff9640 1 -- 172.21.15.188:0/392596585 <== mon.2 v2:172.21.15.188:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc1b80bc070 con 0x7fc1c80b2160 2023-11-26T07:17:30.818 INFO:teuthology.orchestra.run.smithi188.stderr:2023-11-26T07:17:30.811+0000 7fc1c4ff9640 1 -- 172.21.15.188:0/392596585 <== mon.2 v2:172.21.15.188:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc1b80b4420 con 0x7fc1c80b2160 2023-11-26T07:17:30.818 INFO:teuthology.orchestra.run.smithi188.stderr:2023-11-26T07:17:30.811+0000 7fc1ce58d640 1 -- 172.21.15.188:0/392596585 --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc1c80b8610 con 0x7fc1c80b2160 2023-11-26T07:17:30.818 INFO:teuthology.orchestra.run.smithi188.stderr:2023-11-26T07:17:30.811+0000 7fc1c4ff9640 1 -- 172.21.15.188:0/392596585 <== mon.2 v2:172.21.15.188:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc1b80b47d0 con 0x7fc1c80b2160 2023-11-26T07:17:30.818 INFO:teuthology.orchestra.run.smithi188.stderr:2023-11-26T07:17:30.811+0000 7fc1ce58d640 1 -- 172.21.15.188:0/392596585 --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc1c8154010 con 0x7fc1c80b2160 2023-11-26T07:17:30.818 INFO:teuthology.orchestra.run.smithi188.stderr:2023-11-26T07:17:30.815+0000 7fc1c4ff9640 1 -- 172.21.15.188:0/392596585 <== mon.2 v2:172.21.15.188:3301/0 4 ==== mgrmap(e 1) v1 ==== 660+0+0 (secure 0 0 0) 0x7fc1b80b4c30 con 0x7fc1c80b2160 2023-11-26T07:17:30.818 INFO:teuthology.orchestra.run.smithi188.stderr:2023-11-26T07:17:30.815+0000 7fc1c4ff9640 1 -- 172.21.15.188:0/392596585 <== mon.2 v2:172.21.15.188:3301/0 5 ==== osd_map(8..8 src has 1..8) v4 ==== 1415+0+0 (secure 0 0 0) 0x7fc1b80c1c30 con 0x7fc1c80b2160 2023-11-26T07:17:30.818 INFO:teuthology.orchestra.run.smithi188.stderr:2023-11-26T07:17:30.815+0000 7fc1ce58d640 1 -- 172.21.15.188:0/392596585 --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc1c8067680 con 0x7fc1c80b2160 2023-11-26T07:17:30.819 INFO:teuthology.orchestra.run.smithi188.stderr:2023-11-26T07:17:30.815+0000 7fc1c4ff9640 1 -- 172.21.15.188:0/392596585 <== mon.2 v2:172.21.15.188:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+72898 (secure 0 0 0) 0x7fc1b80c1410 con 0x7fc1c80b2160 2023-11-26T07:17:30.836 INFO:tasks.ceph.mgr.y.smithi143.stderr:2023-11-26T07:17:30.835+0000 7f50c328d180 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2023-11-26T07:17:30.870 INFO:tasks.ceph.mgr.x.smithi188.stderr:2023-11-26T07:17:30.867+0000 7f052864e180 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2023-11-26T07:17:30.895 INFO:teuthology.orchestra.run.smithi188.stderr:2023-11-26T07:17:30.891+0000 7fc1ce58d640 1 -- 172.21.15.188:0/392596585 --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_command({"prefix": "osd new", "uuid": "58a65d5d-e090-473a-81b4-ad38e85029f9", "id": 6} v 0) v1 -- 0x7fc1c80b8470 con 0x7fc1c80b2160 2023-11-26T07:17:30.901 INFO:teuthology.orchestra.run.smithi188.stderr:2023-11-26T07:17:30.899+0000 7fc1c4ff9640 1 -- 172.21.15.188:0/392596585 <== mon.2 v2:172.21.15.188:3301/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "58a65d5d-e090-473a-81b4-ad38e85029f9", "id": 6}]=0 v9) v1 ==== 112+0+2 (secure 0 0 0) 0x7fc1c80b8470 con 0x7fc1c80b2160 2023-11-26T07:17:30.901 INFO:teuthology.orchestra.run.smithi188.stdout:6 2023-11-26T07:17:30.902 INFO:teuthology.orchestra.run.smithi188.stderr:2023-11-26T07:17:30.899+0000 7fc1ce58d640 1 -- 172.21.15.188:0/392596585 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fc1c80b2160 msgr2=0x7fc1c8153ad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:17:30.902 INFO:teuthology.orchestra.run.smithi188.stderr:2023-11-26T07:17:30.899+0000 7fc1ce58d640 1 --2- 172.21.15.188:0/392596585 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fc1c80b2160 0x7fc1c8153ad0 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7fc1b80ad040 tx=0x7fc1b80192e0 comp rx=0 tx=0).stop 2023-11-26T07:17:30.902 INFO:teuthology.orchestra.run.smithi188.stderr:2023-11-26T07:17:30.899+0000 7fc1ce58d640 1 -- 172.21.15.188:0/392596585 shutdown_connections 2023-11-26T07:17:30.903 INFO:teuthology.orchestra.run.smithi188.stderr:2023-11-26T07:17:30.899+0000 7fc1ce58d640 1 --2- 172.21.15.188:0/392596585 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fc1c815d6e0 0x7fc1c809cfc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:17:30.903 INFO:teuthology.orchestra.run.smithi188.stderr:2023-11-26T07:17:30.899+0000 7fc1ce58d640 1 --2- 172.21.15.188:0/392596585 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fc1c80d19f0 0x7fc1c8155450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:17:30.903 INFO:teuthology.orchestra.run.smithi188.stderr:2023-11-26T07:17:30.899+0000 7fc1ce58d640 1 --2- 172.21.15.188:0/392596585 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fc1c80b2160 0x7fc1c8153ad0 unknown :-1 s=CLOSED pgs=18 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:17:30.903 INFO:teuthology.orchestra.run.smithi188.stderr:2023-11-26T07:17:30.899+0000 7fc1ce58d640 1 -- 172.21.15.188:0/392596585 >> 172.21.15.188:0/392596585 conn(0x7fc1c809c550 msgr2=0x7fc1c80b2600 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:17:30.903 INFO:teuthology.orchestra.run.smithi188.stderr:2023-11-26T07:17:30.899+0000 7fc1ce58d640 1 -- 172.21.15.188:0/392596585 shutdown_connections 2023-11-26T07:17:30.903 INFO:teuthology.orchestra.run.smithi188.stderr:2023-11-26T07:17:30.899+0000 7fc1ce58d640 1 -- 172.21.15.188:0/392596585 wait complete. 2023-11-26T07:17:30.905 INFO:tasks.ceph.mgr.y.smithi143.stderr:2023-11-26T07:17:30.903+0000 7f50c328d180 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2023-11-26T07:17:30.920 DEBUG:teuthology.orchestra.run.smithi188:> sudo ceph --cluster ceph osd new ea5b7f70-3554-4cf8-ac30-f7c59b80678d 7 2023-11-26T07:17:31.032 INFO:tasks.ceph.mgr.x.smithi188.stderr:2023-11-26T07:17:31.027+0000 7f052864e180 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2023-11-26T07:17:31.100 INFO:teuthology.orchestra.run.smithi188.stderr:2023-11-26T07:17:31.095+0000 7ff31eead640 1 Processor -- start 2023-11-26T07:17:31.100 INFO:teuthology.orchestra.run.smithi188.stderr:2023-11-26T07:17:31.095+0000 7ff31eead640 1 -- start start 2023-11-26T07:17:31.101 INFO:teuthology.orchestra.run.smithi188.stderr:2023-11-26T07:17:31.099+0000 7ff31eead640 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7ff3180d1f30 0x7ff3180b3170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:17:31.101 INFO:teuthology.orchestra.run.smithi188.stderr:2023-11-26T07:17:31.099+0000 7ff31eead640 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7ff3180b36b0 0x7ff3180cea20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:17:31.101 INFO:teuthology.orchestra.run.smithi188.stderr:2023-11-26T07:17:31.099+0000 7ff31eead640 1 -- --> v1:172.21.15.188:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff3180ce590 con 0x7ff31815d5b0 2023-11-26T07:17:31.101 INFO:teuthology.orchestra.run.smithi188.stderr:2023-11-26T07:17:31.099+0000 7ff31eead640 1 -- --> v2:172.21.15.143:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff3180cdf80 con 0x7ff3180b36b0 2023-11-26T07:17:31.102 INFO:teuthology.orchestra.run.smithi188.stderr:2023-11-26T07:17:31.099+0000 7ff31eead640 1 -- --> v2:172.21.15.188:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff3180b3ea0 con 0x7ff3180d1f30 2023-11-26T07:17:31.102 INFO:teuthology.orchestra.run.smithi188.stderr:2023-11-26T07:17:31.099+0000 7ff317fff640 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7ff3180d1f30 0x7ff3180b3170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:17:31.102 INFO:teuthology.orchestra.run.smithi188.stderr:2023-11-26T07:17:31.099+0000 7ff317fff640 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7ff3180d1f30 0x7ff3180b3170 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.188:3300/0 says I am v2:172.21.15.188:37100/0 (socket says 172.21.15.188:37100) 2023-11-26T07:17:31.102 INFO:teuthology.orchestra.run.smithi188.stderr:2023-11-26T07:17:31.099+0000 7ff317fff640 1 -- 172.21.15.188:0/1281945810 learned_addr learned my addr 172.21.15.188:0/1281945810 (peer_addr_for_me v2:172.21.15.188:0/0) 2023-11-26T07:17:31.102 INFO:teuthology.orchestra.run.smithi188.stderr:2023-11-26T07:17:31.099+0000 7ff317fff640 1 -- 172.21.15.188:0/1281945810 >> v2:172.21.15.143:3300/0 conn(0x7ff3180b36b0 msgr2=0x7ff3180cea20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-26T07:17:31.103 INFO:teuthology.orchestra.run.smithi188.stderr:2023-11-26T07:17:31.099+0000 7ff31d44a640 1 --2- 172.21.15.188:0/1281945810 >> v2:172.21.15.143:3300/0 conn(0x7ff3180b36b0 0x7ff3180cea20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:17:31.103 INFO:teuthology.orchestra.run.smithi188.stderr:2023-11-26T07:17:31.099+0000 7ff317fff640 1 --2- 172.21.15.188:0/1281945810 >> v2:172.21.15.143:3300/0 conn(0x7ff3180b36b0 0x7ff3180cea20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:17:31.103 INFO:teuthology.orchestra.run.smithi188.stderr:2023-11-26T07:17:31.099+0000 7ff317fff640 1 -- 172.21.15.188:0/1281945810 >> v1:172.21.15.188:6789/0 conn(0x7ff31815d5b0 legacy=0x7ff3180d1820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:17:31.103 INFO:teuthology.orchestra.run.smithi188.stderr:2023-11-26T07:17:31.099+0000 7ff317fff640 1 -- 172.21.15.188:0/1281945810 --> v2:172.21.15.188:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff3180ce220 con 0x7ff3180d1f30 2023-11-26T07:17:31.103 INFO:teuthology.orchestra.run.smithi188.stderr:2023-11-26T07:17:31.099+0000 7ff31d44a640 1 --2- 172.21.15.188:0/1281945810 >> v2:172.21.15.143:3300/0 conn(0x7ff3180b36b0 0x7ff3180cea20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-26T07:17:31.103 INFO:tasks.ceph.mgr.x.smithi188.stderr:2023-11-26T07:17:31.099+0000 7f052864e180 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2023-11-26T07:17:31.104 INFO:teuthology.orchestra.run.smithi188.stderr:2023-11-26T07:17:31.103+0000 7ff317fff640 1 --2- 172.21.15.188:0/1281945810 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7ff3180d1f30 0x7ff3180b3170 secure :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0x7ff31806b120 tx=0x7ff30401c500 comp rx=0 tx=0).ready entity=mon.1 client_cookie=22ed1133e854fa82 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:17:31.105 INFO:teuthology.orchestra.run.smithi188.stderr:2023-11-26T07:17:31.103+0000 7ff3177fe640 1 -- 172.21.15.188:0/1281945810 <== mon.1 v2:172.21.15.188:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff3040a7070 con 0x7ff3180d1f30 2023-11-26T07:17:31.105 INFO:teuthology.orchestra.run.smithi188.stderr:2023-11-26T07:17:31.103+0000 7ff3177fe640 1 -- 172.21.15.188:0/1281945810 <== mon.1 v2:172.21.15.188:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff3040ab430 con 0x7ff3180d1f30 2023-11-26T07:17:31.105 INFO:teuthology.orchestra.run.smithi188.stderr:2023-11-26T07:17:31.103+0000 7ff3177fe640 1 -- 172.21.15.188:0/1281945810 <== mon.1 v2:172.21.15.188:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff3040ab7c0 con 0x7ff3180d1f30 2023-11-26T07:17:31.106 INFO:teuthology.orchestra.run.smithi188.stderr:2023-11-26T07:17:31.103+0000 7ff31eead640 1 -- 172.21.15.188:0/1281945810 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7ff3180d1f30 msgr2=0x7ff3180b3170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:17:31.106 INFO:teuthology.orchestra.run.smithi188.stderr:2023-11-26T07:17:31.103+0000 7ff31eead640 1 --2- 172.21.15.188:0/1281945810 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7ff3180d1f30 0x7ff3180b3170 secure :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0x7ff31806b120 tx=0x7ff30401c500 comp rx=0 tx=0).stop 2023-11-26T07:17:31.106 INFO:teuthology.orchestra.run.smithi188.stderr:2023-11-26T07:17:31.103+0000 7ff31eead640 1 -- 172.21.15.188:0/1281945810 shutdown_connections 2023-11-26T07:17:31.106 INFO:teuthology.orchestra.run.smithi188.stderr:2023-11-26T07:17:31.103+0000 7ff31eead640 1 --2- 172.21.15.188:0/1281945810 >> v2:172.21.15.143:3300/0 conn(0x7ff3180b36b0 0x7ff3180cea20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:17:31.106 INFO:teuthology.orchestra.run.smithi188.stderr:2023-11-26T07:17:31.103+0000 7ff31eead640 1 --2- 172.21.15.188:0/1281945810 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7ff3180d1f30 0x7ff3180b3170 unknown :-1 s=CLOSED pgs=17 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:17:31.106 INFO:teuthology.orchestra.run.smithi188.stderr:2023-11-26T07:17:31.103+0000 7ff31eead640 1 -- 172.21.15.188:0/1281945810 >> 172.21.15.188:0/1281945810 conn(0x7ff31808cc60 msgr2=0x7ff31809c580 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:17:31.106 INFO:teuthology.orchestra.run.smithi188.stderr:2023-11-26T07:17:31.103+0000 7ff31eead640 1 -- 172.21.15.188:0/1281945810 shutdown_connections 2023-11-26T07:17:31.107 INFO:teuthology.orchestra.run.smithi188.stderr:2023-11-26T07:17:31.103+0000 7ff31eead640 1 -- 172.21.15.188:0/1281945810 wait complete. 2023-11-26T07:17:31.107 INFO:teuthology.orchestra.run.smithi188.stderr:2023-11-26T07:17:31.103+0000 7ff31eead640 1 Processor -- start 2023-11-26T07:17:31.107 INFO:teuthology.orchestra.run.smithi188.stderr:2023-11-26T07:17:31.103+0000 7ff31eead640 1 -- start start 2023-11-26T07:17:31.109 INFO:teuthology.orchestra.run.smithi188.stderr:2023-11-26T07:17:31.103+0000 7ff31eead640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7ff3180b36b0 0x7ff3181572d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:17:31.109 INFO:teuthology.orchestra.run.smithi188.stderr:2023-11-26T07:17:31.103+0000 7ff31eead640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7ff3180d1f30 0x7ff31815dc90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:17:31.110 INFO:teuthology.orchestra.run.smithi188.stderr:2023-11-26T07:17:31.107+0000 7ff31eead640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7ff31815d5b0 0x7ff3181536d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:17:31.110 INFO:teuthology.orchestra.run.smithi188.stderr:2023-11-26T07:17:31.107+0000 7ff31eead640 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff31815e1d0 con 0x7ff3180d1f30 2023-11-26T07:17:31.110 INFO:teuthology.orchestra.run.smithi188.stderr:2023-11-26T07:17:31.107+0000 7ff31eead640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff318157920 con 0x7ff3180b36b0 2023-11-26T07:17:31.110 INFO:teuthology.orchestra.run.smithi188.stderr:2023-11-26T07:17:31.107+0000 7ff31eead640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff318154b40 con 0x7ff31815d5b0 2023-11-26T07:17:31.110 INFO:teuthology.orchestra.run.smithi188.stderr:2023-11-26T07:17:31.107+0000 7ff31cc49640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7ff3180b36b0 0x7ff3181572d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:17:31.110 INFO:teuthology.orchestra.run.smithi188.stderr:2023-11-26T07:17:31.107+0000 7ff31cc49640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7ff3180b36b0 0x7ff3181572d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.188:3300/0 says I am v2:172.21.15.188:37110/0 (socket says 172.21.15.188:37110) 2023-11-26T07:17:31.110 INFO:teuthology.orchestra.run.smithi188.stderr:2023-11-26T07:17:31.107+0000 7ff31cc49640 1 -- 172.21.15.188:0/50672539 learned_addr learned my addr 172.21.15.188:0/50672539 (peer_addr_for_me v2:172.21.15.188:0/0) 2023-11-26T07:17:31.110 INFO:teuthology.orchestra.run.smithi188.stderr:2023-11-26T07:17:31.107+0000 7ff31d44a640 1 --2- 172.21.15.188:0/50672539 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7ff31815d5b0 0x7ff3181536d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:17:31.110 INFO:teuthology.orchestra.run.smithi188.stderr:2023-11-26T07:17:31.107+0000 7ff317fff640 1 --2- 172.21.15.188:0/50672539 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7ff3180d1f30 0x7ff31815dc90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:17:31.110 INFO:teuthology.orchestra.run.smithi188.stderr:2023-11-26T07:17:31.107+0000 7ff31cc49640 1 -- 172.21.15.188:0/50672539 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7ff31815d5b0 msgr2=0x7ff3181536d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:17:31.110 INFO:teuthology.orchestra.run.smithi188.stderr:2023-11-26T07:17:31.107+0000 7ff31cc49640 1 --2- 172.21.15.188:0/50672539 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7ff31815d5b0 0x7ff3181536d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:17:31.110 INFO:teuthology.orchestra.run.smithi188.stderr:2023-11-26T07:17:31.107+0000 7ff31cc49640 1 -- 172.21.15.188:0/50672539 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7ff3180d1f30 msgr2=0x7ff31815dc90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:17:31.110 INFO:teuthology.orchestra.run.smithi188.stderr:2023-11-26T07:17:31.107+0000 7ff31cc49640 1 --2- 172.21.15.188:0/50672539 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7ff3180d1f30 0x7ff31815dc90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:17:31.111 INFO:teuthology.orchestra.run.smithi188.stderr:2023-11-26T07:17:31.107+0000 7ff31cc49640 1 -- 172.21.15.188:0/50672539 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff3040094d0 con 0x7ff3180b36b0 2023-11-26T07:17:31.111 INFO:teuthology.orchestra.run.smithi188.stderr:2023-11-26T07:17:31.107+0000 7ff317fff640 1 --2- 172.21.15.188:0/50672539 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7ff3180d1f30 0x7ff31815dc90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-26T07:17:31.111 INFO:teuthology.orchestra.run.smithi188.stderr:2023-11-26T07:17:31.107+0000 7ff31cc49640 1 --2- 172.21.15.188:0/50672539 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7ff3180b36b0 0x7ff3181572d0 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7ff30c00c3c0 tx=0x7ff30c00c880 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:17:31.112 INFO:teuthology.orchestra.run.smithi188.stderr:2023-11-26T07:17:31.107+0000 7ff3157fa640 1 -- 172.21.15.188:0/50672539 <== mon.1 v2:172.21.15.188:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff30c0052d0 con 0x7ff3180b36b0 2023-11-26T07:17:31.112 INFO:teuthology.orchestra.run.smithi188.stderr:2023-11-26T07:17:31.107+0000 7ff3157fa640 1 -- 172.21.15.188:0/50672539 <== mon.1 v2:172.21.15.188:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff30c005430 con 0x7ff3180b36b0 2023-11-26T07:17:31.112 INFO:teuthology.orchestra.run.smithi188.stderr:2023-11-26T07:17:31.107+0000 7ff3157fa640 1 -- 172.21.15.188:0/50672539 <== mon.1 v2:172.21.15.188:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff30c003840 con 0x7ff3180b36b0 2023-11-26T07:17:31.112 INFO:teuthology.orchestra.run.smithi188.stderr:2023-11-26T07:17:31.107+0000 7ff31eead640 1 -- 172.21.15.188:0/50672539 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff318154e20 con 0x7ff3180b36b0 2023-11-26T07:17:31.112 INFO:teuthology.orchestra.run.smithi188.stderr:2023-11-26T07:17:31.107+0000 7ff31eead640 1 -- 172.21.15.188:0/50672539 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff318153c10 con 0x7ff3180b36b0 2023-11-26T07:17:31.112 INFO:teuthology.orchestra.run.smithi188.stderr:2023-11-26T07:17:31.107+0000 7ff3157fa640 1 -- 172.21.15.188:0/50672539 <== mon.1 v2:172.21.15.188:3300/0 4 ==== mgrmap(e 1) v1 ==== 660+0+0 (secure 0 0 0) 0x7ff30c003ca0 con 0x7ff3180b36b0 2023-11-26T07:17:31.112 INFO:teuthology.orchestra.run.smithi188.stderr:2023-11-26T07:17:31.107+0000 7ff3157fa640 1 -- 172.21.15.188:0/50672539 <== mon.1 v2:172.21.15.188:3300/0 5 ==== osd_map(9..9 src has 1..9) v4 ==== 1530+0+0 (secure 0 0 0) 0x7ff30c011ea0 con 0x7ff3180b36b0 2023-11-26T07:17:31.112 INFO:teuthology.orchestra.run.smithi188.stderr:2023-11-26T07:17:31.107+0000 7ff31eead640 1 -- 172.21.15.188:0/50672539 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff318067680 con 0x7ff3180b36b0 2023-11-26T07:17:31.113 INFO:teuthology.orchestra.run.smithi188.stderr:2023-11-26T07:17:31.111+0000 7ff3157fa640 1 -- 172.21.15.188:0/50672539 <== mon.1 v2:172.21.15.188:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+72898 (secure 0 0 0) 0x7ff30c017070 con 0x7ff3180b36b0 2023-11-26T07:17:31.180 INFO:teuthology.orchestra.run.smithi188.stderr:2023-11-26T07:17:31.175+0000 7ff31eead640 1 -- 172.21.15.188:0/50672539 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_command({"prefix": "osd new", "uuid": "ea5b7f70-3554-4cf8-ac30-f7c59b80678d", "id": 7} v 0) v1 -- 0x7ff318156d40 con 0x7ff3180b36b0 2023-11-26T07:17:31.186 INFO:teuthology.orchestra.run.smithi188.stderr:2023-11-26T07:17:31.183+0000 7ff3157fa640 1 -- 172.21.15.188:0/50672539 <== mon.1 v2:172.21.15.188:3300/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "ea5b7f70-3554-4cf8-ac30-f7c59b80678d", "id": 7}]=0 v10) v1 ==== 112+0+2 (secure 0 0 0) 0x7ff30c012bc0 con 0x7ff3180b36b0 2023-11-26T07:17:31.186 INFO:teuthology.orchestra.run.smithi188.stdout:7 2023-11-26T07:17:31.187 INFO:teuthology.orchestra.run.smithi188.stderr:2023-11-26T07:17:31.183+0000 7ff31eead640 1 -- 172.21.15.188:0/50672539 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7ff3180b36b0 msgr2=0x7ff3181572d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:17:31.188 INFO:teuthology.orchestra.run.smithi188.stderr:2023-11-26T07:17:31.183+0000 7ff31eead640 1 --2- 172.21.15.188:0/50672539 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7ff3180b36b0 0x7ff3181572d0 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7ff30c00c3c0 tx=0x7ff30c00c880 comp rx=0 tx=0).stop 2023-11-26T07:17:31.188 INFO:teuthology.orchestra.run.smithi188.stderr:2023-11-26T07:17:31.183+0000 7ff31eead640 1 -- 172.21.15.188:0/50672539 shutdown_connections 2023-11-26T07:17:31.188 INFO:teuthology.orchestra.run.smithi188.stderr:2023-11-26T07:17:31.183+0000 7ff31eead640 1 --2- 172.21.15.188:0/50672539 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7ff31815d5b0 0x7ff3181536d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:17:31.188 INFO:teuthology.orchestra.run.smithi188.stderr:2023-11-26T07:17:31.183+0000 7ff31eead640 1 --2- 172.21.15.188:0/50672539 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7ff3180d1f30 0x7ff31815dc90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:17:31.188 INFO:teuthology.orchestra.run.smithi188.stderr:2023-11-26T07:17:31.183+0000 7ff31eead640 1 --2- 172.21.15.188:0/50672539 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7ff3180b36b0 0x7ff3181572d0 unknown :-1 s=CLOSED pgs=18 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:17:31.188 INFO:teuthology.orchestra.run.smithi188.stderr:2023-11-26T07:17:31.183+0000 7ff31eead640 1 -- 172.21.15.188:0/50672539 >> 172.21.15.188:0/50672539 conn(0x7ff31808cc60 msgr2=0x7ff3180cfb60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:17:31.188 INFO:teuthology.orchestra.run.smithi188.stderr:2023-11-26T07:17:31.183+0000 7ff31eead640 1 -- 172.21.15.188:0/50672539 shutdown_connections 2023-11-26T07:17:31.188 INFO:teuthology.orchestra.run.smithi188.stderr:2023-11-26T07:17:31.183+0000 7ff31eead640 1 -- 172.21.15.188:0/50672539 wait complete. 2023-11-26T07:17:31.206 INFO:tasks.ceph.osd.0:Restarting daemon 2023-11-26T07:17:31.206 DEBUG:teuthology.orchestra.run.smithi143:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 0 2023-11-26T07:17:31.210 INFO:tasks.ceph.osd.0:Started 2023-11-26T07:17:31.210 INFO:tasks.ceph.osd.1:Restarting daemon 2023-11-26T07:17:31.210 DEBUG:teuthology.orchestra.run.smithi143:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 1 2023-11-26T07:17:31.213 INFO:tasks.ceph.osd.1:Started 2023-11-26T07:17:31.214 INFO:tasks.ceph.osd.2:Restarting daemon 2023-11-26T07:17:31.214 DEBUG:teuthology.orchestra.run.smithi143:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 2 2023-11-26T07:17:31.218 INFO:tasks.ceph.osd.2:Started 2023-11-26T07:17:31.219 INFO:tasks.ceph.osd.3:Restarting daemon 2023-11-26T07:17:31.219 DEBUG:teuthology.orchestra.run.smithi143:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 3 2023-11-26T07:17:31.229 INFO:tasks.ceph.osd.3:Started 2023-11-26T07:17:31.230 INFO:tasks.ceph.osd.4:Restarting daemon 2023-11-26T07:17:31.230 DEBUG:teuthology.orchestra.run.smithi188:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 4 2023-11-26T07:17:31.235 INFO:tasks.ceph.osd.4:Started 2023-11-26T07:17:31.235 INFO:tasks.ceph.osd.5:Restarting daemon 2023-11-26T07:17:31.235 DEBUG:teuthology.orchestra.run.smithi188:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 5 2023-11-26T07:17:31.240 INFO:tasks.ceph.osd.5:Started 2023-11-26T07:17:31.240 INFO:tasks.ceph.osd.6:Restarting daemon 2023-11-26T07:17:31.240 DEBUG:teuthology.orchestra.run.smithi188:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 6 2023-11-26T07:17:31.245 INFO:tasks.ceph.osd.6:Started 2023-11-26T07:17:31.245 INFO:tasks.ceph.osd.7:Restarting daemon 2023-11-26T07:17:31.245 DEBUG:teuthology.orchestra.run.smithi188:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 7 2023-11-26T07:17:31.250 INFO:tasks.ceph.osd.7:Started 2023-11-26T07:17:31.250 DEBUG:teuthology.orchestra.run.smithi143:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-11-26T07:17:31.441 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:17:31.439+0000 7f69e7901640 1 Processor -- start 2023-11-26T07:17:31.441 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:17:31.439+0000 7f69e7901640 1 -- start start 2023-11-26T07:17:31.443 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:17:31.439+0000 7f69e7901640 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7f69e015d5b0 0x7f69e00d1820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:17:31.443 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:17:31.439+0000 7f69e7901640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f69e00b3880 0x7f69e00cea20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:17:31.443 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:17:31.439+0000 7f69e7901640 1 -- --> v1:172.21.15.188:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f69e00d21c0 con 0x7f69e00d1d60 2023-11-26T07:17:31.443 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:17:31.439+0000 7f69e7901640 1 -- --> v2:172.21.15.188:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f69e00ce590 con 0x7f69e015d5b0 2023-11-26T07:17:31.444 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:17:31.439+0000 7f69e7901640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f69e00cdf80 con 0x7f69e00b3880 2023-11-26T07:17:31.444 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:17:31.439+0000 7f69e569d640 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7f69e015d5b0 0x7f69e00d1820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:17:31.444 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:17:31.439+0000 7f69e569d640 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7f69e015d5b0 0x7f69e00d1820 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.188:3300/0 says I am v2:172.21.15.143:48414/0 (socket says 172.21.15.143:48414) 2023-11-26T07:17:31.444 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:17:31.439+0000 7f69e569d640 1 -- 172.21.15.143:0/2952147572 learned_addr learned my addr 172.21.15.143:0/2952147572 (peer_addr_for_me v2:172.21.15.143:0/0) 2023-11-26T07:17:31.444 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:17:31.439+0000 7f69e4e9c640 1 --1- 172.21.15.143:0/2952147572 >> v1:172.21.15.188:6789/0 conn(0x7f69e00d1d60 0x7f69e00b3170 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.188:6789/0 says I am v1:172.21.15.143:40622/0 (socket says 172.21.15.143:40622) 2023-11-26T07:17:31.444 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:17:31.439+0000 7f69e5e9e640 1 --2- 172.21.15.143:0/2952147572 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f69e00b3880 0x7f69e00cea20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:17:31.445 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:17:31.439+0000 7f69e569d640 1 -- 172.21.15.143:0/2952147572 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f69e00b3880 msgr2=0x7f69e00cea20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-26T07:17:31.445 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:17:31.439+0000 7f69e569d640 1 --2- 172.21.15.143:0/2952147572 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f69e00b3880 0x7f69e00cea20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:17:31.445 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:17:31.439+0000 7f69e569d640 1 -- 172.21.15.143:0/2952147572 >> v1:172.21.15.188:6789/0 conn(0x7f69e00d1d60 legacy=0x7f69e00b3170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:17:31.445 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:17:31.439+0000 7f69e569d640 1 -- 172.21.15.143:0/2952147572 --> v2:172.21.15.188:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f69e00b3ea0 con 0x7f69e015d5b0 2023-11-26T07:17:31.446 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:17:31.443+0000 7f69e569d640 1 --2- 172.21.15.143:0/2952147572 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f69e015d5b0 0x7f69e00d1820 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0x7f69dc009870 tx=0x7f69dc01c510 comp rx=0 tx=0).ready entity=mon.1 client_cookie=9b7fc287537bfcb7 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:17:31.447 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:17:31.443+0000 7f69d7fff640 1 -- 172.21.15.143:0/2952147572 <== mon.1 v2:172.21.15.188:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f69dc0ab620 con 0x7f69e015d5b0 2023-11-26T07:17:31.447 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:17:31.443+0000 7f69d7fff640 1 -- 172.21.15.143:0/2952147572 <== mon.1 v2:172.21.15.188:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f69dc0ab780 con 0x7f69e015d5b0 2023-11-26T07:17:31.447 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:17:31.443+0000 7f69d7fff640 1 -- 172.21.15.143:0/2952147572 <== mon.1 v2:172.21.15.188:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f69dc0abb10 con 0x7f69e015d5b0 2023-11-26T07:17:31.448 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:17:31.447+0000 7f69e7901640 1 -- 172.21.15.143:0/2952147572 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f69e015d5b0 msgr2=0x7f69e00d1820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:17:31.448 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:17:31.447+0000 7f69e7901640 1 --2- 172.21.15.143:0/2952147572 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f69e015d5b0 0x7f69e00d1820 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0x7f69dc009870 tx=0x7f69dc01c510 comp rx=0 tx=0).stop 2023-11-26T07:17:31.448 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:17:31.447+0000 7f69e7901640 1 -- 172.21.15.143:0/2952147572 shutdown_connections 2023-11-26T07:17:31.448 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:17:31.447+0000 7f69e7901640 1 --2- 172.21.15.143:0/2952147572 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f69e00b3880 0x7f69e00cea20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:17:31.448 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:17:31.447+0000 7f69e7901640 1 --2- 172.21.15.143:0/2952147572 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f69e015d5b0 0x7f69e00d1820 unknown :-1 s=CLOSED pgs=29 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:17:31.448 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:17:31.447+0000 7f69e7901640 1 -- 172.21.15.143:0/2952147572 >> 172.21.15.143:0/2952147572 conn(0x7f69e008cc60 msgr2=0x7f69e009c580 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:17:31.449 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:17:31.447+0000 7f69e7901640 1 -- 172.21.15.143:0/2952147572 shutdown_connections 2023-11-26T07:17:31.449 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:17:31.447+0000 7f69e7901640 1 -- 172.21.15.143:0/2952147572 wait complete. 2023-11-26T07:17:31.450 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:17:31.447+0000 7f69e7901640 1 Processor -- start 2023-11-26T07:17:31.450 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:17:31.447+0000 7f69e7901640 1 -- start start 2023-11-26T07:17:31.452 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:17:31.447+0000 7f69e7901640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f69e00b3880 0x7f69e00ccb20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:17:31.453 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:17:31.447+0000 7f69e7901640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f69e00d1d60 0x7f69e015c070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:17:31.453 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:17:31.447+0000 7f69e7901640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f69e015d5b0 0x7f69e00cb180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:17:31.453 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:17:31.447+0000 7f69e7901640 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f69e015ad80 con 0x7f69e00d1d60 2023-11-26T07:17:31.453 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:17:31.447+0000 7f69e7901640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f69e015afd0 con 0x7f69e00b3880 2023-11-26T07:17:31.453 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:17:31.447+0000 7f69e7901640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f69e015b2b0 con 0x7f69e015d5b0 2023-11-26T07:17:31.453 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:17:31.447+0000 7f69e4e9c640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f69e00d1d60 0x7f69e015c070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:17:31.453 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:17:31.447+0000 7f69e4e9c640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f69e00d1d60 0x7f69e015c070 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:47184/0 (socket says 172.21.15.143:47184) 2023-11-26T07:17:31.453 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:17:31.447+0000 7f69e4e9c640 1 -- 172.21.15.143:0/614016620 learned_addr learned my addr 172.21.15.143:0/614016620 (peer_addr_for_me v2:172.21.15.143:0/0) 2023-11-26T07:17:31.453 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:17:31.447+0000 7f69e569d640 1 --2- 172.21.15.143:0/614016620 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f69e00b3880 0x7f69e00ccb20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:17:31.453 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:17:31.447+0000 7f69e5e9e640 1 --2- 172.21.15.143:0/614016620 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f69e015d5b0 0x7f69e00cb180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:17:31.453 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:17:31.447+0000 7f69e4e9c640 1 -- 172.21.15.143:0/614016620 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f69e015d5b0 msgr2=0x7f69e00cb180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:17:31.454 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:17:31.447+0000 7f69e4e9c640 1 --2- 172.21.15.143:0/614016620 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f69e015d5b0 0x7f69e00cb180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:17:31.454 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:17:31.447+0000 7f69e4e9c640 1 -- 172.21.15.143:0/614016620 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f69e00b3880 msgr2=0x7f69e00ccb20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:17:31.454 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:17:31.447+0000 7f69e4e9c640 1 --2- 172.21.15.143:0/614016620 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f69e00b3880 0x7f69e00ccb20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:17:31.454 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:17:31.447+0000 7f69e4e9c640 1 -- 172.21.15.143:0/614016620 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f69dc0094d0 con 0x7f69e00d1d60 2023-11-26T07:17:31.454 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:17:31.451+0000 7f69e4e9c640 1 --2- 172.21.15.143:0/614016620 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f69e00d1d60 0x7f69e015c070 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7f69d000c490 tx=0x7f69d000c950 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:17:31.454 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:17:31.451+0000 7f69d5ffb640 1 -- 172.21.15.143:0/614016620 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f69d0007480 con 0x7f69e00d1d60 2023-11-26T07:17:31.454 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:17:31.451+0000 7f69d5ffb640 1 -- 172.21.15.143:0/614016620 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f69d00075e0 con 0x7f69e00d1d60 2023-11-26T07:17:31.454 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:17:31.451+0000 7f69e7901640 1 -- 172.21.15.143:0/614016620 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f69e015b590 con 0x7f69e00d1d60 2023-11-26T07:17:31.454 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:17:31.451+0000 7f69d5ffb640 1 -- 172.21.15.143:0/614016620 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f69d0004a10 con 0x7f69e00d1d60 2023-11-26T07:17:31.454 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:17:31.451+0000 7f69d5ffb640 1 -- 172.21.15.143:0/614016620 <== mon.0 v2:172.21.15.143:3300/0 4 ==== mgrmap(e 1) v1 ==== 660+0+0 (secure 0 0 0) 0x7f69d0012440 con 0x7f69e00d1d60 2023-11-26T07:17:31.454 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:17:31.451+0000 7f69e7901640 1 -- 172.21.15.143:0/614016620 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f69e015bb50 con 0x7f69e00d1d60 2023-11-26T07:17:31.454 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:17:31.451+0000 7f69d5ffb640 1 -- 172.21.15.143:0/614016620 <== mon.0 v2:172.21.15.143:3300/0 5 ==== osd_map(10..10 src has 1..10) v4 ==== 1645+0+0 (secure 0 0 0) 0x7f69d0012dc0 con 0x7f69e00d1d60 2023-11-26T07:17:31.455 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:17:31.451+0000 7f69e7901640 1 -- 172.21.15.143:0/614016620 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f69e0067680 con 0x7f69e00d1d60 2023-11-26T07:17:31.465 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:17:31.463+0000 7f69d5ffb640 1 -- 172.21.15.143:0/614016620 <== mon.0 v2:172.21.15.143:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+72898 (secure 0 0 0) 0x7f69d0018070 con 0x7f69e00d1d60 2023-11-26T07:17:31.522 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:17:31.519+0000 7f69e7901640 1 -- 172.21.15.143:0/614016620 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f69e0155fc0 con 0x7f69e00d1d60 2023-11-26T07:17:31.522 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:17:31.519+0000 7f69d5ffb640 1 -- 172.21.15.143:0/614016620 <== mon.0 v2:172.21.15.143:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v10) v1 ==== 74+0+7057 (secure 0 0 0) 0x7f69d00126f0 con 0x7f69e00d1d60 2023-11-26T07:17:31.523 INFO:teuthology.orchestra.run.smithi143.stdout: 2023-11-26T07:17:31.523 INFO:teuthology.orchestra.run.smithi143.stdout:{"epoch":10,"fsid":"f5033060-2469-4a07-95fe-2b76f5404bc1","created":"2023-11-26T07:17:28.613663+0000","modified":"2023-11-26T07:17:31.181658+0000","last_up_change":"0.000000","last_in_change":"2023-11-26T07:17:31.181658+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":2,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":0,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[],"osds":[{"osd":0,"uuid":"ebbee2c6-023e-44b9-9f1d-6f7bfd309730","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":1,"uuid":"fec4b8be-7122-49e9-9a19-5850a0700001","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":2,"uuid":"2888af5b-f933-46ca-8f95-6dd494ff43f6","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":3,"uuid":"d1f52435-7bc1-4081-9550-9cd1ba55f549","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":4,"uuid":"e2ae8b6e-6e25-49b2-bcbf-aacce68f06a3","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":5,"uuid":"3cec8a32-7698-4c36-8e89-e4ba11934f33","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":6,"uuid":"58a65d5d-e090-473a-81b4-ad38e85029f9","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":7,"uuid":"ea5b7f70-3554-4cf8-ac30-f7c59b80678d","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-11-26T07:17:31.523 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:17:31.519+0000 7f69e7901640 1 -- 172.21.15.143:0/614016620 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f69e00d1d60 msgr2=0x7f69e015c070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:17:31.524 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:17:31.519+0000 7f69e7901640 1 --2- 172.21.15.143:0/614016620 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f69e00d1d60 0x7f69e015c070 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7f69d000c490 tx=0x7f69d000c950 comp rx=0 tx=0).stop 2023-11-26T07:17:31.524 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:17:31.519+0000 7f69e7901640 1 -- 172.21.15.143:0/614016620 shutdown_connections 2023-11-26T07:17:31.524 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:17:31.519+0000 7f69e7901640 1 --2- 172.21.15.143:0/614016620 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f69e015d5b0 0x7f69e00cb180 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:17:31.524 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:17:31.519+0000 7f69e7901640 1 --2- 172.21.15.143:0/614016620 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f69e00d1d60 0x7f69e015c070 unknown :-1 s=CLOSED pgs=20 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:17:31.524 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:17:31.519+0000 7f69e7901640 1 --2- 172.21.15.143:0/614016620 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f69e00b3880 0x7f69e00ccb20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:17:31.524 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:17:31.519+0000 7f69e7901640 1 -- 172.21.15.143:0/614016620 >> 172.21.15.143:0/614016620 conn(0x7f69e008cc60 msgr2=0x7f69e00c70c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:17:31.524 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:17:31.519+0000 7f69e7901640 1 -- 172.21.15.143:0/614016620 shutdown_connections 2023-11-26T07:17:31.524 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:17:31.519+0000 7f69e7901640 1 -- 172.21.15.143:0/614016620 wait complete. 2023-11-26T07:17:31.543 INFO:tasks.ceph.ceph_manager.ceph:[] 2023-11-26T07:17:31.543 INFO:tasks.ceph:Waiting for OSDs to come up 2023-11-26T07:17:31.663 INFO:tasks.ceph.mgr.y.smithi143.stderr:/usr/lib/python3/dist-packages/scipy/__init__.py:67: UserWarning: NumPy was imported from a Python sub-interpreter but NumPy does not properly support sub-interpreters. This will likely work for most users but might cause hard to track down issues or subtle bugs. A common user of the rare sub-interpreter feature is wsgi which also allows single-interpreter mode. 2023-11-26T07:17:31.663 INFO:tasks.ceph.mgr.y.smithi143.stderr:Improvements in the case of bugs are welcome, but is not on the NumPy roadmap, and full support may require significant effort to achieve. 2023-11-26T07:17:31.663 INFO:tasks.ceph.mgr.y.smithi143.stderr: from numpy import show_config as show_numpy_config 2023-11-26T07:17:31.710 INFO:tasks.ceph.mgr.y.smithi143.stderr:2023-11-26T07:17:31.707+0000 7f50c328d180 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2023-11-26T07:17:31.773 INFO:tasks.ceph.mgr.x.smithi188.stderr:/usr/lib/python3/dist-packages/scipy/__init__.py:67: UserWarning: NumPy was imported from a Python sub-interpreter but NumPy does not properly support sub-interpreters. This will likely work for most users but might cause hard to track down issues or subtle bugs. A common user of the rare sub-interpreter feature is wsgi which also allows single-interpreter mode. 2023-11-26T07:17:31.773 INFO:tasks.ceph.mgr.x.smithi188.stderr:Improvements in the case of bugs are welcome, but is not on the NumPy roadmap, and full support may require significant effort to achieve. 2023-11-26T07:17:31.773 INFO:tasks.ceph.mgr.x.smithi188.stderr: from numpy import show_config as show_numpy_config 2023-11-26T07:17:31.817 INFO:tasks.ceph.mgr.x.smithi188.stderr:2023-11-26T07:17:31.815+0000 7f052864e180 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2023-11-26T07:17:31.846 INFO:tasks.ceph.mgr.y.smithi143.stderr:2023-11-26T07:17:31.843+0000 7f50c328d180 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2023-11-26T07:17:31.882 INFO:tasks.ceph.osd.4.smithi188.stderr:2023-11-26T07:17:31.879+0000 7ffbb2cd4640 -1 Falling back to public interface 2023-11-26T07:17:31.894 INFO:tasks.ceph.osd.6.smithi188.stderr:2023-11-26T07:17:31.891+0000 7f593edef640 -1 Falling back to public interface 2023-11-26T07:17:31.898 INFO:tasks.ceph.osd.5.smithi188.stderr:2023-11-26T07:17:31.895+0000 7f5361495640 -1 Falling back to public interface 2023-11-26T07:17:31.898 INFO:tasks.ceph.osd.6.smithi188.stderr:2023-11-26T07:17:31.895+0000 7f593ddb2640 -1 PosixStack listen unable to listen on v1:0.0.0.0:6813/0: (98) Address already in use 2023-11-26T07:17:31.917 INFO:tasks.ceph.osd.3.smithi143.stderr:2023-11-26T07:17:31.915+0000 7f947d02b640 -1 Falling back to public interface 2023-11-26T07:17:31.918 INFO:tasks.ceph.osd.7.smithi188.stderr:2023-11-26T07:17:31.915+0000 7f6742a2e640 -1 Falling back to public interface 2023-11-26T07:17:31.921 INFO:tasks.ceph.osd.2.smithi143.stderr:2023-11-26T07:17:31.919+0000 7f657d5a2640 -1 Falling back to public interface 2023-11-26T07:17:31.929 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-11-26T07:17:31.927+0000 7f87f3f31640 -1 Falling back to public interface 2023-11-26T07:17:31.929 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-11-26T07:17:31.927+0000 7f0cb09d8640 -1 Falling back to public interface 2023-11-26T07:17:31.959 INFO:tasks.ceph.mgr.x.smithi188.stderr:2023-11-26T07:17:31.955+0000 7f052864e180 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2023-11-26T07:17:32.019 INFO:tasks.ceph.mgr.y.smithi143.stderr:2023-11-26T07:17:32.015+0000 7f50c328d180 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2023-11-26T07:17:32.092 INFO:tasks.ceph.mgr.y.smithi143.stderr:2023-11-26T07:17:32.087+0000 7f50c328d180 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2023-11-26T07:17:32.131 INFO:tasks.ceph.mgr.x.smithi188.stderr:2023-11-26T07:17:32.127+0000 7f052864e180 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2023-11-26T07:17:32.205 INFO:tasks.ceph.mgr.x.smithi188.stderr:2023-11-26T07:17:32.203+0000 7f052864e180 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2023-11-26T07:17:32.229 INFO:tasks.ceph.mgr.y.smithi143.stderr:2023-11-26T07:17:32.227+0000 7f50c328d180 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2023-11-26T07:17:32.344 INFO:tasks.ceph.mgr.x.smithi188.stderr:2023-11-26T07:17:32.339+0000 7f052864e180 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2023-11-26T07:17:32.347 DEBUG:teuthology.orchestra.run.smithi143:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph --cluster ceph osd dump --format=json 2023-11-26T07:17:32.476 INFO:teuthology.misc.health.smithi143.stderr:2023-11-26T07:17:32.471+0000 7f5adcf1d640 1 Processor -- start 2023-11-26T07:17:32.476 INFO:teuthology.misc.health.smithi143.stderr:2023-11-26T07:17:32.471+0000 7f5adcf1d640 1 -- start start 2023-11-26T07:17:32.477 INFO:teuthology.misc.health.smithi143.stderr:2023-11-26T07:17:32.475+0000 7f5adcf1d640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f5ad80d1e20 0x7f5ad80cde70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:17:32.477 INFO:teuthology.misc.health.smithi143.stderr:2023-11-26T07:17:32.475+0000 7f5adcf1d640 1 -- --> v1:172.21.15.143:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5ad80ce470 con 0x7f5ad80d1a40 2023-11-26T07:17:32.477 INFO:teuthology.misc.health.smithi143.stderr:2023-11-26T07:17:32.475+0000 7f5adcf1d640 1 -- --> v1:172.21.15.188:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5ad80ce750 con 0x7f5ad80cf510 2023-11-26T07:17:32.477 INFO:teuthology.misc.health.smithi143.stderr:2023-11-26T07:17:32.475+0000 7f5adcf1d640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5ad80cea30 con 0x7f5ad80d1e20 2023-11-26T07:17:32.478 INFO:teuthology.misc.health.smithi143.stderr:2023-11-26T07:17:32.475+0000 7f5ad659c640 1 --1- >> v1:172.21.15.143:6789/0 conn(0x7f5ad80d1a40 0x7f5ad80b4540 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.143:6789/0 says I am v1:172.21.15.143:51836/0 (socket says 172.21.15.143:51836) 2023-11-26T07:17:32.478 INFO:teuthology.misc.health.smithi143.stderr:2023-11-26T07:17:32.475+0000 7f5ad659c640 1 -- 172.21.15.143:0/1841764466 learned_addr learned my addr 172.21.15.143:0/1841764466 (peer_addr_for_me v1:172.21.15.143:0/0) 2023-11-26T07:17:32.478 INFO:teuthology.misc.health.smithi143.stderr:2023-11-26T07:17:32.475+0000 7f5acffff640 1 --2- 172.21.15.143:0/1841764466 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f5ad80d1e20 0x7f5ad80cde70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:17:32.479 INFO:teuthology.misc.health.smithi143.stderr:2023-11-26T07:17:32.475+0000 7f5ad5d9b640 1 -- 172.21.15.143:0/1841764466 <== mon.0 v1:172.21.15.143:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3987492015 0 0) 0x7f5ad80ce470 con 0x7f5ad80d1a40 2023-11-26T07:17:32.479 INFO:teuthology.misc.health.smithi143.stderr:2023-11-26T07:17:32.475+0000 7f5ad5d9b640 1 -- 172.21.15.143:0/1841764466 --> v1:172.21.15.143:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f5ac4003530 con 0x7f5ad80d1a40 2023-11-26T07:17:32.479 INFO:teuthology.misc.health.smithi143.stderr:2023-11-26T07:17:32.475+0000 7f5ad5d9b640 1 -- 172.21.15.143:0/1841764466 <== mon.1 v1:172.21.15.188:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 835930808 0 0) 0x7f5ad80ce750 con 0x7f5ad80cf510 2023-11-26T07:17:32.479 INFO:teuthology.misc.health.smithi143.stderr:2023-11-26T07:17:32.475+0000 7f5ad5d9b640 1 -- 172.21.15.143:0/1841764466 --> v1:172.21.15.188:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f5ad80ce470 con 0x7f5ad80cf510 2023-11-26T07:17:32.479 INFO:teuthology.misc.health.smithi143.stderr:2023-11-26T07:17:32.475+0000 7f5acffff640 1 -- 172.21.15.143:0/1841764466 >> v1:172.21.15.188:6789/0 conn(0x7f5ad80cf510 legacy=0x7f5ad815d170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:17:32.479 INFO:teuthology.misc.health.smithi143.stderr:2023-11-26T07:17:32.475+0000 7f5acffff640 1 -- 172.21.15.143:0/1841764466 >> v1:172.21.15.143:6789/0 conn(0x7f5ad80d1a40 legacy=0x7f5ad80b4540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:17:32.479 INFO:teuthology.misc.health.smithi143.stderr:2023-11-26T07:17:32.475+0000 7f5acffff640 1 -- 172.21.15.143:0/1841764466 --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5ad80ced10 con 0x7f5ad80d1e20 2023-11-26T07:17:32.481 INFO:teuthology.misc.health.smithi143.stderr:2023-11-26T07:17:32.479+0000 7f5acffff640 1 --2- 172.21.15.143:0/1841764466 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f5ad80d1e20 0x7f5ad80cde70 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0x7f5ac00097f0 tx=0x7f5ac001ecb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=37136ca85c385e00 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:17:32.481 INFO:teuthology.misc.health.smithi143.stderr:2023-11-26T07:17:32.479+0000 7f5ad5d9b640 1 -- 172.21.15.143:0/1841764466 <== mon.2 v2:172.21.15.188:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5ac00a89b0 con 0x7f5ad80d1e20 2023-11-26T07:17:32.481 INFO:teuthology.misc.health.smithi143.stderr:2023-11-26T07:17:32.479+0000 7f5ad5d9b640 1 -- 172.21.15.143:0/1841764466 <== mon.2 v2:172.21.15.188:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5ac00a8b10 con 0x7f5ad80d1e20 2023-11-26T07:17:32.481 INFO:teuthology.misc.health.smithi143.stderr:2023-11-26T07:17:32.479+0000 7f5ad5d9b640 1 -- 172.21.15.143:0/1841764466 <== mon.2 v2:172.21.15.188:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5ac00a8ea0 con 0x7f5ad80d1e20 2023-11-26T07:17:32.482 INFO:teuthology.misc.health.smithi143.stderr:2023-11-26T07:17:32.479+0000 7f5adcf1d640 1 -- 172.21.15.143:0/1841764466 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f5ad80d1e20 msgr2=0x7f5ad80cde70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:17:32.482 INFO:teuthology.misc.health.smithi143.stderr:2023-11-26T07:17:32.479+0000 7f5adcf1d640 1 --2- 172.21.15.143:0/1841764466 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f5ad80d1e20 0x7f5ad80cde70 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0x7f5ac00097f0 tx=0x7f5ac001ecb0 comp rx=0 tx=0).stop 2023-11-26T07:17:32.482 INFO:teuthology.misc.health.smithi143.stderr:2023-11-26T07:17:32.479+0000 7f5adcf1d640 1 -- 172.21.15.143:0/1841764466 shutdown_connections 2023-11-26T07:17:32.482 INFO:teuthology.misc.health.smithi143.stderr:2023-11-26T07:17:32.479+0000 7f5adcf1d640 1 --2- 172.21.15.143:0/1841764466 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f5ad80d1e20 0x7f5ad80cde70 unknown :-1 s=CLOSED pgs=23 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:17:32.482 INFO:teuthology.misc.health.smithi143.stderr:2023-11-26T07:17:32.479+0000 7f5adcf1d640 1 -- 172.21.15.143:0/1841764466 >> 172.21.15.143:0/1841764466 conn(0x7f5ad808ccb0 msgr2=0x7f5ad809c5d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:17:32.483 INFO:teuthology.misc.health.smithi143.stderr:2023-11-26T07:17:32.479+0000 7f5adcf1d640 1 -- 172.21.15.143:0/1841764466 shutdown_connections 2023-11-26T07:17:32.483 INFO:teuthology.misc.health.smithi143.stderr:2023-11-26T07:17:32.479+0000 7f5adcf1d640 1 -- 172.21.15.143:0/1841764466 wait complete. 2023-11-26T07:17:32.484 INFO:teuthology.misc.health.smithi143.stderr:2023-11-26T07:17:32.483+0000 7f5adcf1d640 1 Processor -- start 2023-11-26T07:17:32.484 INFO:teuthology.misc.health.smithi143.stderr:2023-11-26T07:17:32.483+0000 7f5adcf1d640 1 -- start start 2023-11-26T07:17:32.487 INFO:teuthology.misc.health.smithi143.stderr:2023-11-26T07:17:32.483+0000 7f5adcf1d640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f5ad80cf510 0x7f5ad80cad40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:17:32.487 INFO:teuthology.misc.health.smithi143.stderr:2023-11-26T07:17:32.483+0000 7f5adcf1d640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f5ad80d1a40 0x7f5ad80b8b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:17:32.487 INFO:teuthology.misc.health.smithi143.stderr:2023-11-26T07:17:32.483+0000 7f5adcf1d640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f5ad80d1e20 0x7f5ad814d600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:17:32.487 INFO:teuthology.misc.health.smithi143.stderr:2023-11-26T07:17:32.483+0000 7f5adcf1d640 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5ad80b90d0 con 0x7f5ad80d1e20 2023-11-26T07:17:32.487 INFO:teuthology.misc.health.smithi143.stderr:2023-11-26T07:17:32.483+0000 7f5adcf1d640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5ad815aec0 con 0x7f5ad80cf510 2023-11-26T07:17:32.487 INFO:teuthology.misc.health.smithi143.stderr:2023-11-26T07:17:32.483+0000 7f5adcf1d640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5ad80cb800 con 0x7f5ad80d1a40 2023-11-26T07:17:32.487 INFO:teuthology.misc.health.smithi143.stderr:2023-11-26T07:17:32.483+0000 7f5ad6d9d640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f5ad80d1e20 0x7f5ad814d600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:17:32.488 INFO:teuthology.misc.health.smithi143.stderr:2023-11-26T07:17:32.483+0000 7f5ad659c640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f5ad80cf510 0x7f5ad80cad40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:17:32.488 INFO:teuthology.misc.health.smithi143.stderr:2023-11-26T07:17:32.483+0000 7f5ad6d9d640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f5ad80d1e20 0x7f5ad814d600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:47192/0 (socket says 172.21.15.143:47192) 2023-11-26T07:17:32.488 INFO:teuthology.misc.health.smithi143.stderr:2023-11-26T07:17:32.483+0000 7f5ad6d9d640 1 -- 172.21.15.143:0/2589138121 learned_addr learned my addr 172.21.15.143:0/2589138121 (peer_addr_for_me v2:172.21.15.143:0/0) 2023-11-26T07:17:32.488 INFO:teuthology.misc.health.smithi143.stderr:2023-11-26T07:17:32.483+0000 7f5acffff640 1 --2- 172.21.15.143:0/2589138121 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f5ad80d1a40 0x7f5ad80b8b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:17:32.488 INFO:teuthology.misc.health.smithi143.stderr:2023-11-26T07:17:32.483+0000 7f5ad659c640 1 -- 172.21.15.143:0/2589138121 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f5ad80d1a40 msgr2=0x7f5ad80b8b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:17:32.488 INFO:teuthology.misc.health.smithi143.stderr:2023-11-26T07:17:32.483+0000 7f5ad659c640 1 --2- 172.21.15.143:0/2589138121 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f5ad80d1a40 0x7f5ad80b8b90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:17:32.488 INFO:teuthology.misc.health.smithi143.stderr:2023-11-26T07:17:32.483+0000 7f5ad659c640 1 -- 172.21.15.143:0/2589138121 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f5ad80d1e20 msgr2=0x7f5ad814d600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:17:32.488 INFO:teuthology.misc.health.smithi143.stderr:2023-11-26T07:17:32.483+0000 7f5ad659c640 1 --2- 172.21.15.143:0/2589138121 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f5ad80d1e20 0x7f5ad814d600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:17:32.488 INFO:teuthology.misc.health.smithi143.stderr:2023-11-26T07:17:32.483+0000 7f5ad659c640 1 -- 172.21.15.143:0/2589138121 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5ac0009420 con 0x7f5ad80cf510 2023-11-26T07:17:32.488 INFO:teuthology.misc.health.smithi143.stderr:2023-11-26T07:17:32.483+0000 7f5ad6d9d640 1 --2- 172.21.15.143:0/2589138121 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f5ad80d1e20 0x7f5ad814d600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-26T07:17:32.488 INFO:teuthology.misc.health.smithi143.stderr:2023-11-26T07:17:32.483+0000 7f5ad659c640 1 --2- 172.21.15.143:0/2589138121 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f5ad80cf510 0x7f5ad80cad40 secure :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0x7f5ac800c400 tx=0x7f5ac800c8c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:17:32.489 INFO:teuthology.misc.health.smithi143.stderr:2023-11-26T07:17:32.483+0000 7f5aceffd640 1 -- 172.21.15.143:0/2589138121 <== mon.1 v2:172.21.15.188:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5ac8005cf0 con 0x7f5ad80cf510 2023-11-26T07:17:32.489 INFO:teuthology.misc.health.smithi143.stderr:2023-11-26T07:17:32.483+0000 7f5aceffd640 1 -- 172.21.15.143:0/2589138121 <== mon.1 v2:172.21.15.188:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5ac8005e50 con 0x7f5ad80cf510 2023-11-26T07:17:32.489 INFO:teuthology.misc.health.smithi143.stderr:2023-11-26T07:17:32.483+0000 7f5aceffd640 1 -- 172.21.15.143:0/2589138121 <== mon.1 v2:172.21.15.188:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5ac80072d0 con 0x7f5ad80cf510 2023-11-26T07:17:32.489 INFO:teuthology.misc.health.smithi143.stderr:2023-11-26T07:17:32.483+0000 7f5adcf1d640 1 -- 172.21.15.143:0/2589138121 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5ad80cbae0 con 0x7f5ad80cf510 2023-11-26T07:17:32.489 INFO:teuthology.misc.health.smithi143.stderr:2023-11-26T07:17:32.483+0000 7f5adcf1d640 1 -- 172.21.15.143:0/2589138121 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5ad80cc0a0 con 0x7f5ad80cf510 2023-11-26T07:17:32.489 INFO:teuthology.misc.health.smithi143.stderr:2023-11-26T07:17:32.483+0000 7f5aceffd640 1 -- 172.21.15.143:0/2589138121 <== mon.1 v2:172.21.15.188:3300/0 4 ==== mgrmap(e 1) v1 ==== 660+0+0 (secure 0 0 0) 0x7f5ac801c440 con 0x7f5ad80cf510 2023-11-26T07:17:32.489 INFO:teuthology.misc.health.smithi143.stderr:2023-11-26T07:17:32.483+0000 7f5aceffd640 1 -- 172.21.15.143:0/2589138121 <== mon.1 v2:172.21.15.188:3300/0 5 ==== osd_map(10..10 src has 1..10) v4 ==== 1645+0+0 (secure 0 0 0) 0x7f5ac801cdc0 con 0x7f5ad80cf510 2023-11-26T07:17:32.490 INFO:teuthology.misc.health.smithi143.stderr:2023-11-26T07:17:32.483+0000 7f5adcf1d640 1 -- 172.21.15.143:0/2589138121 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5ad8067680 con 0x7f5ad80cf510 2023-11-26T07:17:32.490 INFO:teuthology.misc.health.smithi143.stderr:2023-11-26T07:17:32.487+0000 7f5aceffd640 1 -- 172.21.15.143:0/2589138121 <== mon.1 v2:172.21.15.188:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+72898 (secure 0 0 0) 0x7f5ac8021060 con 0x7f5ad80cf510 2023-11-26T07:17:32.547 INFO:teuthology.misc.health.smithi143.stderr:2023-11-26T07:17:32.543+0000 7f5adcf1d640 1 -- 172.21.15.143:0/2589138121 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f5ad80b4220 con 0x7f5ad80cf510 2023-11-26T07:17:32.547 INFO:teuthology.misc.health.smithi143.stderr:2023-11-26T07:17:32.543+0000 7f5aceffd640 1 -- 172.21.15.143:0/2589138121 <== mon.1 v2:172.21.15.188:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v10) v1 ==== 74+0+7057 (secure 0 0 0) 0x7f5ac8017070 con 0x7f5ad80cf510 2023-11-26T07:17:32.548 INFO:teuthology.misc.health.smithi143.stdout: 2023-11-26T07:17:32.548 INFO:teuthology.misc.health.smithi143.stdout:{"epoch":10,"fsid":"f5033060-2469-4a07-95fe-2b76f5404bc1","created":"2023-11-26T07:17:28.613663+0000","modified":"2023-11-26T07:17:31.181658+0000","last_up_change":"0.000000","last_in_change":"2023-11-26T07:17:31.181658+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":2,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":0,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[],"osds":[{"osd":0,"uuid":"ebbee2c6-023e-44b9-9f1d-6f7bfd309730","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":1,"uuid":"fec4b8be-7122-49e9-9a19-5850a0700001","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":2,"uuid":"2888af5b-f933-46ca-8f95-6dd494ff43f6","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":3,"uuid":"d1f52435-7bc1-4081-9550-9cd1ba55f549","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":4,"uuid":"e2ae8b6e-6e25-49b2-bcbf-aacce68f06a3","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":5,"uuid":"3cec8a32-7698-4c36-8e89-e4ba11934f33","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":6,"uuid":"58a65d5d-e090-473a-81b4-ad38e85029f9","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":7,"uuid":"ea5b7f70-3554-4cf8-ac30-f7c59b80678d","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-11-26T07:17:32.549 INFO:teuthology.misc.health.smithi143.stderr:2023-11-26T07:17:32.547+0000 7f5adcf1d640 1 -- 172.21.15.143:0/2589138121 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f5ad80cf510 msgr2=0x7f5ad80cad40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:17:32.549 INFO:teuthology.misc.health.smithi143.stderr:2023-11-26T07:17:32.547+0000 7f5adcf1d640 1 --2- 172.21.15.143:0/2589138121 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f5ad80cf510 0x7f5ad80cad40 secure :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0x7f5ac800c400 tx=0x7f5ac800c8c0 comp rx=0 tx=0).stop 2023-11-26T07:17:32.549 INFO:teuthology.misc.health.smithi143.stderr:2023-11-26T07:17:32.547+0000 7f5adcf1d640 1 -- 172.21.15.143:0/2589138121 shutdown_connections 2023-11-26T07:17:32.549 INFO:teuthology.misc.health.smithi143.stderr:2023-11-26T07:17:32.547+0000 7f5adcf1d640 1 --2- 172.21.15.143:0/2589138121 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f5ad80d1e20 0x7f5ad814d600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:17:32.549 INFO:teuthology.misc.health.smithi143.stderr:2023-11-26T07:17:32.547+0000 7f5adcf1d640 1 --2- 172.21.15.143:0/2589138121 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f5ad80d1a40 0x7f5ad80b8b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:17:32.549 INFO:teuthology.misc.health.smithi143.stderr:2023-11-26T07:17:32.547+0000 7f5adcf1d640 1 --2- 172.21.15.143:0/2589138121 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f5ad80cf510 0x7f5ad80cad40 unknown :-1 s=CLOSED pgs=31 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:17:32.549 INFO:teuthology.misc.health.smithi143.stderr:2023-11-26T07:17:32.547+0000 7f5adcf1d640 1 -- 172.21.15.143:0/2589138121 >> 172.21.15.143:0/2589138121 conn(0x7f5ad808ccb0 msgr2=0x7f5ad80c7c80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:17:32.550 INFO:teuthology.misc.health.smithi143.stderr:2023-11-26T07:17:32.547+0000 7f5adcf1d640 1 -- 172.21.15.143:0/2589138121 shutdown_connections 2023-11-26T07:17:32.550 INFO:teuthology.misc.health.smithi143.stderr:2023-11-26T07:17:32.547+0000 7f5adcf1d640 1 -- 172.21.15.143:0/2589138121 wait complete. 2023-11-26T07:17:32.567 DEBUG:teuthology.misc:0 of 8 OSDs are up 2023-11-26T07:17:32.922 INFO:tasks.ceph.mgr.y.smithi143.stderr:2023-11-26T07:17:32.919+0000 7f50c328d180 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2023-11-26T07:17:33.007 INFO:tasks.ceph.mgr.y.smithi143.stderr:2023-11-26T07:17:33.003+0000 7f50c328d180 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2023-11-26T07:17:33.028 INFO:tasks.ceph.mgr.x.smithi188.stderr:2023-11-26T07:17:33.023+0000 7f052864e180 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2023-11-26T07:17:33.116 INFO:tasks.ceph.mgr.x.smithi188.stderr:2023-11-26T07:17:33.111+0000 7f052864e180 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2023-11-26T07:17:33.198 INFO:tasks.ceph.mgr.y.smithi143.stderr:2023-11-26T07:17:33.195+0000 7f50c328d180 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2023-11-26T07:17:33.282 INFO:tasks.ceph.mgr.y.smithi143.stderr:2023-11-26T07:17:33.279+0000 7f50c328d180 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2023-11-26T07:17:33.308 INFO:tasks.ceph.mgr.x.smithi188.stderr:2023-11-26T07:17:33.303+0000 7f052864e180 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2023-11-26T07:17:33.394 INFO:tasks.ceph.mgr.x.smithi188.stderr:2023-11-26T07:17:33.391+0000 7f052864e180 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2023-11-26T07:17:33.608 INFO:tasks.ceph.mgr.y.smithi143.stderr:2023-11-26T07:17:33.607+0000 7f50c328d180 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2023-11-26T07:17:33.676 INFO:tasks.ceph.mgr.y.smithi143.stderr:2023-11-26T07:17:33.675+0000 7f50c328d180 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2023-11-26T07:17:33.730 INFO:tasks.ceph.mgr.x.smithi188.stderr:2023-11-26T07:17:33.727+0000 7f052864e180 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2023-11-26T07:17:33.799 INFO:tasks.ceph.mgr.x.smithi188.stderr:2023-11-26T07:17:33.795+0000 7f052864e180 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2023-11-26T07:17:34.700 INFO:tasks.ceph.mgr.y.smithi143.stderr:2023-11-26T07:17:34.695+0000 7f506ac0a640 -1 mgr.server handle_report got status from non-daemon mon.c 2023-11-26T07:17:34.700 INFO:tasks.ceph.mgr.y.smithi143.stderr:2023-11-26T07:17:34.695+0000 7f506ac0a640 -1 mgr.server handle_report got status from non-daemon mon.b 2023-11-26T07:17:35.709 INFO:tasks.ceph.osd.4.smithi188.stderr:2023-11-26T07:17:35.707+0000 7ffbb2cd4640 -1 osd.4 0 log_to_monitors true 2023-11-26T07:17:35.796 INFO:tasks.ceph.osd.6.smithi188.stderr:2023-11-26T07:17:35.791+0000 7f593edef640 -1 osd.6 0 log_to_monitors true 2023-11-26T07:17:35.824 INFO:tasks.ceph.osd.3.smithi143.stderr:2023-11-26T07:17:35.819+0000 7f947d02b640 -1 osd.3 0 log_to_monitors true 2023-11-26T07:17:35.854 INFO:tasks.ceph.osd.7.smithi188.stderr:2023-11-26T07:17:35.851+0000 7f6742a2e640 -1 osd.7 0 log_to_monitors true 2023-11-26T07:17:35.869 INFO:tasks.ceph.osd.2.smithi143.stderr:2023-11-26T07:17:35.867+0000 7f657d5a2640 -1 osd.2 0 log_to_monitors true 2023-11-26T07:17:35.911 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-11-26T07:17:35.907+0000 7f87f3f31640 -1 osd.1 0 log_to_monitors true 2023-11-26T07:17:35.929 INFO:tasks.ceph.osd.5.smithi188.stderr:2023-11-26T07:17:35.927+0000 7f5361495640 -1 osd.5 0 log_to_monitors true 2023-11-26T07:17:35.958 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-11-26T07:17:35.955+0000 7f0cb09d8640 -1 osd.0 0 log_to_monitors true 2023-11-26T07:17:39.373 DEBUG:teuthology.orchestra.run.smithi143:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph --cluster ceph osd dump --format=json 2023-11-26T07:17:39.591 INFO:teuthology.misc.health.smithi143.stderr:2023-11-26T07:17:39.587+0000 7f046da46640 1 Processor -- start 2023-11-26T07:17:39.591 INFO:teuthology.misc.health.smithi143.stderr:2023-11-26T07:17:39.587+0000 7f046da46640 1 -- start start 2023-11-26T07:17:39.592 INFO:teuthology.misc.health.smithi143.stderr:2023-11-26T07:17:39.591+0000 7f046da46640 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7f04600e4a70 0x7f04600e4e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:17:39.593 INFO:teuthology.misc.health.smithi143.stderr:2023-11-26T07:17:39.591+0000 7f046da46640 1 -- --> v1:172.21.15.143:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f04600085a0 con 0x7f04600e1150 2023-11-26T07:17:39.593 INFO:teuthology.misc.health.smithi143.stderr:2023-11-26T07:17:39.591+0000 7f046da46640 1 -- --> v1:172.21.15.188:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f04600e5980 con 0x7f04600570c0 2023-11-26T07:17:39.593 INFO:teuthology.misc.health.smithi143.stderr:2023-11-26T07:17:39.591+0000 7f046da46640 1 -- --> v2:172.21.15.188:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f04600018b0 con 0x7f04600e4a70 2023-11-26T07:17:39.594 INFO:teuthology.misc.health.smithi143.stderr:2023-11-26T07:17:39.591+0000 7f04667fc640 1 --1- >> v1:172.21.15.143:6789/0 conn(0x7f04600e1150 0x7f0460007780 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.143:6789/0 says I am v1:172.21.15.143:42736/0 (socket says 172.21.15.143:42736) 2023-11-26T07:17:39.594 INFO:teuthology.misc.health.smithi143.stderr:2023-11-26T07:17:39.591+0000 7f04667fc640 1 -- 172.21.15.143:0/3109677209 learned_addr learned my addr 172.21.15.143:0/3109677209 (peer_addr_for_me v1:172.21.15.143:0/0) 2023-11-26T07:17:39.594 INFO:teuthology.misc.health.smithi143.stderr:2023-11-26T07:17:39.591+0000 7f0466ffd640 1 --2- 172.21.15.143:0/3109677209 >> v2:172.21.15.188:3300/0 conn(0x7f04600e4a70 0x7f04600e4e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:17:39.594 INFO:teuthology.misc.health.smithi143.stderr:2023-11-26T07:17:39.591+0000 7f0465ffb640 1 -- 172.21.15.143:0/3109677209 <== mon.0 v1:172.21.15.143:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3020480364 0 0) 0x7f04600085a0 con 0x7f04600e1150 2023-11-26T07:17:39.595 INFO:teuthology.misc.health.smithi143.stderr:2023-11-26T07:17:39.591+0000 7f0465ffb640 1 -- 172.21.15.143:0/3109677209 --> v1:172.21.15.143:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f0450003530 con 0x7f04600e1150 2023-11-26T07:17:39.596 INFO:teuthology.misc.health.smithi143.stderr:2023-11-26T07:17:39.591+0000 7f0465ffb640 1 -- 172.21.15.143:0/3109677209 <== mon.0 v1:172.21.15.143:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 896302438 0 0) 0x7f0450003530 con 0x7f04600e1150 2023-11-26T07:17:39.596 INFO:teuthology.misc.health.smithi143.stderr:2023-11-26T07:17:39.591+0000 7f0465ffb640 1 -- 172.21.15.143:0/3109677209 >> v2:172.21.15.188:3300/0 conn(0x7f04600e4a70 msgr2=0x7f04600e4e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-26T07:17:39.596 INFO:teuthology.misc.health.smithi143.stderr:2023-11-26T07:17:39.591+0000 7f0465ffb640 1 --2- 172.21.15.143:0/3109677209 >> v2:172.21.15.188:3300/0 conn(0x7f04600e4a70 0x7f04600e4e50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:17:39.596 INFO:teuthology.misc.health.smithi143.stderr:2023-11-26T07:17:39.591+0000 7f0465ffb640 1 -- 172.21.15.143:0/3109677209 >> v1:172.21.15.188:6789/0 conn(0x7f04600570c0 legacy=0x7f0460007e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:17:39.596 INFO:teuthology.misc.health.smithi143.stderr:2023-11-26T07:17:39.591+0000 7f0465ffb640 1 -- 172.21.15.143:0/3109677209 --> v1:172.21.15.143:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0460001b90 con 0x7f04600e1150 2023-11-26T07:17:39.596 INFO:teuthology.misc.health.smithi143.stderr:2023-11-26T07:17:39.591+0000 7f0465ffb640 1 -- 172.21.15.143:0/3109677209 <== mon.0 v1:172.21.15.143:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2137882523 0 0) 0x7f045c001680 con 0x7f04600e1150 2023-11-26T07:17:39.596 INFO:teuthology.misc.health.smithi143.stderr:2023-11-26T07:17:39.591+0000 7f0465ffb640 1 -- 172.21.15.143:0/3109677209 >> v1:172.21.15.143:6789/0 conn(0x7f04600e1150 legacy=0x7f0460007780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:17:39.596 INFO:teuthology.misc.health.smithi143.stderr:2023-11-26T07:17:39.591+0000 7f0465ffb640 1 --2- 172.21.15.143:0/3109677209 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f0450003980 0x7f0450003da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:17:39.596 INFO:teuthology.misc.health.smithi143.stderr:2023-11-26T07:17:39.591+0000 7f0465ffb640 1 --2- 172.21.15.143:0/3109677209 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f04500047d0 0x7f0450004c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:17:39.596 INFO:teuthology.misc.health.smithi143.stderr:2023-11-26T07:17:39.591+0000 7f0465ffb640 1 --2- 172.21.15.143:0/3109677209 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f0450005190 0x7f0450014590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:17:39.596 INFO:teuthology.misc.health.smithi143.stderr:2023-11-26T07:17:39.591+0000 7f0465ffb640 1 -- 172.21.15.143:0/3109677209 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f04600018b0 con 0x7f0450005190 2023-11-26T07:17:39.597 INFO:teuthology.misc.health.smithi143.stderr:2023-11-26T07:17:39.591+0000 7f0465ffb640 1 -- 172.21.15.143:0/3109677209 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0450009a90 con 0x7f04500047d0 2023-11-26T07:17:39.597 INFO:teuthology.misc.health.smithi143.stderr:2023-11-26T07:17:39.591+0000 7f0465ffb640 1 -- 172.21.15.143:0/3109677209 --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0450014ad0 con 0x7f0450003980 2023-11-26T07:17:39.597 INFO:teuthology.misc.health.smithi143.stderr:2023-11-26T07:17:39.591+0000 7f0467fff640 1 --2- 172.21.15.143:0/3109677209 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f0450005190 0x7f0450014590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:17:39.597 INFO:teuthology.misc.health.smithi143.stderr:2023-11-26T07:17:39.591+0000 7f04667fc640 1 --2- 172.21.15.143:0/3109677209 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f04500047d0 0x7f0450004c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:17:39.597 INFO:teuthology.misc.health.smithi143.stderr:2023-11-26T07:17:39.595+0000 7f0467fff640 1 -- 172.21.15.143:0/3109677209 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f0450003980 msgr2=0x7f0450003da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-26T07:17:39.597 INFO:teuthology.misc.health.smithi143.stderr:2023-11-26T07:17:39.595+0000 7f0467fff640 1 --2- 172.21.15.143:0/3109677209 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f0450003980 0x7f0450003da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:17:39.597 INFO:teuthology.misc.health.smithi143.stderr:2023-11-26T07:17:39.595+0000 7f0467fff640 1 -- 172.21.15.143:0/3109677209 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f04500047d0 msgr2=0x7f0450004c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-26T07:17:39.598 INFO:teuthology.misc.health.smithi143.stderr:2023-11-26T07:17:39.595+0000 7f0467fff640 1 --2- 172.21.15.143:0/3109677209 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f04500047d0 0x7f0450004c50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:17:39.598 INFO:teuthology.misc.health.smithi143.stderr:2023-11-26T07:17:39.595+0000 7f0467fff640 1 -- 172.21.15.143:0/3109677209 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0460001b90 con 0x7f0450005190 2023-11-26T07:17:39.599 INFO:teuthology.misc.health.smithi143.stderr:2023-11-26T07:17:39.595+0000 7f0467fff640 1 -- 172.21.15.143:0/3109677209 reap_dead start 2023-11-26T07:17:39.599 INFO:teuthology.misc.health.smithi143.stderr:2023-11-26T07:17:39.595+0000 7f0467fff640 1 --2- 172.21.15.143:0/3109677209 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f0450005190 0x7f0450014590 secure :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0x7f0468067bb0 tx=0x7f0468096b00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7ab4960be0fd4994 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:17:39.600 INFO:teuthology.misc.health.smithi143.stderr:2023-11-26T07:17:39.595+0000 7f0465ffb640 1 -- 172.21.15.143:0/3109677209 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f046811f880 con 0x7f0450005190 2023-11-26T07:17:39.600 INFO:teuthology.misc.health.smithi143.stderr:2023-11-26T07:17:39.595+0000 7f0465ffb640 1 -- 172.21.15.143:0/3109677209 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f046811fa20 con 0x7f0450005190 2023-11-26T07:17:39.600 INFO:teuthology.misc.health.smithi143.stderr:2023-11-26T07:17:39.595+0000 7f0465ffb640 1 -- 172.21.15.143:0/3109677209 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f046811fdd0 con 0x7f0450005190 2023-11-26T07:17:39.600 INFO:teuthology.misc.health.smithi143.stderr:2023-11-26T07:17:39.599+0000 7f046da46640 1 -- 172.21.15.143:0/3109677209 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f0450005190 msgr2=0x7f0450014590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:17:39.601 INFO:teuthology.misc.health.smithi143.stderr:2023-11-26T07:17:39.599+0000 7f046da46640 1 --2- 172.21.15.143:0/3109677209 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f0450005190 0x7f0450014590 secure :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0x7f0468067bb0 tx=0x7f0468096b00 comp rx=0 tx=0).stop 2023-11-26T07:17:39.601 INFO:teuthology.misc.health.smithi143.stderr:2023-11-26T07:17:39.599+0000 7f046da46640 1 -- 172.21.15.143:0/3109677209 shutdown_connections 2023-11-26T07:17:39.601 INFO:teuthology.misc.health.smithi143.stderr:2023-11-26T07:17:39.599+0000 7f046da46640 1 --2- 172.21.15.143:0/3109677209 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f0450005190 0x7f0450014590 unknown :-1 s=CLOSED pgs=33 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:17:39.601 INFO:teuthology.misc.health.smithi143.stderr:2023-11-26T07:17:39.599+0000 7f046da46640 1 -- 172.21.15.143:0/3109677209 >> 172.21.15.143:0/3109677209 conn(0x7f04600563b0 msgr2=0x7f04600567c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:17:39.602 INFO:teuthology.misc.health.smithi143.stderr:2023-11-26T07:17:39.599+0000 7f046da46640 1 -- 172.21.15.143:0/3109677209 shutdown_connections 2023-11-26T07:17:39.602 INFO:teuthology.misc.health.smithi143.stderr:2023-11-26T07:17:39.599+0000 7f046da46640 1 -- 172.21.15.143:0/3109677209 wait complete. 2023-11-26T07:17:39.603 INFO:teuthology.misc.health.smithi143.stderr:2023-11-26T07:17:39.599+0000 7f046da46640 1 Processor -- start 2023-11-26T07:17:39.603 INFO:teuthology.misc.health.smithi143.stderr:2023-11-26T07:17:39.599+0000 7f046da46640 1 -- start start 2023-11-26T07:17:39.604 INFO:teuthology.misc.health.smithi143.stderr:2023-11-26T07:17:39.603+0000 7f046da46640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f0450005190 0x7f04600e1530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:17:39.604 INFO:teuthology.misc.health.smithi143.stderr:2023-11-26T07:17:39.603+0000 7f046da46640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f04600e0630 0x7f0460053f60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:17:39.605 INFO:teuthology.misc.health.smithi143.stderr:2023-11-26T07:17:39.603+0000 7f046da46640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f04600d9a90 0x7f04600544a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:17:39.605 INFO:teuthology.misc.health.smithi143.stderr:2023-11-26T07:17:39.603+0000 7f046da46640 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f04600dd160 con 0x7f04600d9a90 2023-11-26T07:17:39.605 INFO:teuthology.misc.health.smithi143.stderr:2023-11-26T07:17:39.603+0000 7f046da46640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f04600e1f50 con 0x7f04600e0630 2023-11-26T07:17:39.605 INFO:teuthology.misc.health.smithi143.stderr:2023-11-26T07:17:39.603+0000 7f046da46640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f04600e2230 con 0x7f0450005190 2023-11-26T07:17:39.606 INFO:teuthology.misc.health.smithi143.stderr:2023-11-26T07:17:39.603+0000 7f0467fff640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f04600d9a90 0x7f04600544a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:17:39.606 INFO:teuthology.misc.health.smithi143.stderr:2023-11-26T07:17:39.603+0000 7f0467fff640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f04600d9a90 0x7f04600544a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:45194/0 (socket says 172.21.15.143:45194) 2023-11-26T07:17:39.606 INFO:teuthology.misc.health.smithi143.stderr:2023-11-26T07:17:39.603+0000 7f0467fff640 1 -- 172.21.15.143:0/203832382 learned_addr learned my addr 172.21.15.143:0/203832382 (peer_addr_for_me v2:172.21.15.143:0/0) 2023-11-26T07:17:39.606 INFO:teuthology.misc.health.smithi143.stderr:2023-11-26T07:17:39.603+0000 7f0466ffd640 1 --2- 172.21.15.143:0/203832382 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f0450005190 0x7f04600e1530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:17:39.606 INFO:teuthology.misc.health.smithi143.stderr:2023-11-26T07:17:39.603+0000 7f0467fff640 1 -- 172.21.15.143:0/203832382 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f0450005190 msgr2=0x7f04600e1530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:17:39.606 INFO:teuthology.misc.health.smithi143.stderr:2023-11-26T07:17:39.603+0000 7f0467fff640 1 --2- 172.21.15.143:0/203832382 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f0450005190 0x7f04600e1530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:17:39.606 INFO:teuthology.misc.health.smithi143.stderr:2023-11-26T07:17:39.603+0000 7f0467fff640 1 -- 172.21.15.143:0/203832382 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f04600e0630 msgr2=0x7f0460053f60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:17:39.606 INFO:teuthology.misc.health.smithi143.stderr:2023-11-26T07:17:39.603+0000 7f0467fff640 1 --2- 172.21.15.143:0/203832382 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f04600e0630 0x7f0460053f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:17:39.606 INFO:teuthology.misc.health.smithi143.stderr:2023-11-26T07:17:39.603+0000 7f0467fff640 1 -- 172.21.15.143:0/203832382 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f04600e2510 con 0x7f04600d9a90 2023-11-26T07:17:39.606 INFO:teuthology.misc.health.smithi143.stderr:2023-11-26T07:17:39.603+0000 7f0466ffd640 1 --2- 172.21.15.143:0/203832382 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f0450005190 0x7f04600e1530 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-26T07:17:39.607 INFO:teuthology.misc.health.smithi143.stderr:2023-11-26T07:17:39.603+0000 7f0467fff640 1 --2- 172.21.15.143:0/203832382 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f04600d9a90 0x7f04600544a0 secure :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0x7f046804fe40 tx=0x7f0450005800 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:17:39.607 INFO:teuthology.misc.health.smithi143.stderr:2023-11-26T07:17:39.603+0000 7f044ffff640 1 -- 172.21.15.143:0/203832382 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0468127850 con 0x7f04600d9a90 2023-11-26T07:17:39.607 INFO:teuthology.misc.health.smithi143.stderr:2023-11-26T07:17:39.603+0000 7f044ffff640 1 -- 172.21.15.143:0/203832382 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f04681279b0 con 0x7f04600d9a90 2023-11-26T07:17:39.607 INFO:teuthology.misc.health.smithi143.stderr:2023-11-26T07:17:39.603+0000 7f044ffff640 1 -- 172.21.15.143:0/203832382 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0468127d80 con 0x7f04600d9a90 2023-11-26T07:17:39.607 INFO:teuthology.misc.health.smithi143.stderr:2023-11-26T07:17:39.603+0000 7f046da46640 1 -- 172.21.15.143:0/203832382 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f04600e27f0 con 0x7f04600d9a90 2023-11-26T07:17:39.608 INFO:teuthology.misc.health.smithi143.stderr:2023-11-26T07:17:39.603+0000 7f046da46640 1 -- 172.21.15.143:0/203832382 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0460035e10 con 0x7f04600d9a90 2023-11-26T07:17:39.610 INFO:teuthology.misc.health.smithi143.stderr:2023-11-26T07:17:39.607+0000 7f044ffff640 1 -- 172.21.15.143:0/203832382 <== mon.0 v2:172.21.15.143:3300/0 4 ==== mgrmap(e 4) v1 ==== 85796+0+0 (secure 0 0 0) 0x7f0468131460 con 0x7f04600d9a90 2023-11-26T07:17:39.610 INFO:teuthology.misc.health.smithi143.stderr:2023-11-26T07:17:39.607+0000 7f046da46640 1 -- 172.21.15.143:0/203832382 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0460006c80 con 0x7f04600d9a90 2023-11-26T07:17:39.610 INFO:teuthology.misc.health.smithi143.stderr:2023-11-26T07:17:39.607+0000 7f044ffff640 1 --2- 172.21.15.143:0/203832382 >> [v2:172.21.15.143:6832/2150557544,v1:172.21.15.143:6833/2150557544] conn(0x7f0454066460 0x7f0454068920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:17:39.610 INFO:teuthology.misc.health.smithi143.stderr:2023-11-26T07:17:39.607+0000 7f0466ffd640 1 --2- 172.21.15.143:0/203832382 >> [v2:172.21.15.143:6832/2150557544,v1:172.21.15.143:6833/2150557544] conn(0x7f0454066460 0x7f0454068920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:17:39.610 INFO:teuthology.misc.health.smithi143.stderr:2023-11-26T07:17:39.607+0000 7f044ffff640 1 -- 172.21.15.143:0/203832382 <== mon.0 v2:172.21.15.143:3300/0 5 ==== osd_map(12..12 src has 1..12) v4 ==== 2231+0+0 (secure 0 0 0) 0x7f04681692b0 con 0x7f04600d9a90 2023-11-26T07:17:39.611 INFO:teuthology.misc.health.smithi143.stderr:2023-11-26T07:17:39.607+0000 7f0466ffd640 1 --2- 172.21.15.143:0/203832382 >> [v2:172.21.15.143:6832/2150557544,v1:172.21.15.143:6833/2150557544] conn(0x7f0454066460 0x7f0454068920 secure :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0x7f04580047c0 tx=0x7f0458008040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:17:39.619 INFO:teuthology.misc.health.smithi143.stderr:2023-11-26T07:17:39.615+0000 7f044ffff640 1 -- 172.21.15.143:0/203832382 <== mon.0 v2:172.21.15.143:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172649 (secure 0 0 0) 0x7f0468165680 con 0x7f04600d9a90 2023-11-26T07:17:39.955 INFO:teuthology.misc.health.smithi143.stdout: 2023-11-26T07:17:39.955 INFO:teuthology.misc.health.smithi143.stdout:{"epoch":12,"fsid":"f5033060-2469-4a07-95fe-2b76f5404bc1","created":"2023-11-26T07:17:28.613663+0000","modified":"2023-11-26T07:17:37.700940+0000","last_up_change":"0.000000","last_in_change":"2023-11-26T07:17:31.181658+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":0,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[],"osds":[{"osd":0,"uuid":"ebbee2c6-023e-44b9-9f1d-6f7bfd309730","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":1,"uuid":"fec4b8be-7122-49e9-9a19-5850a0700001","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":2,"uuid":"2888af5b-f933-46ca-8f95-6dd494ff43f6","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":3,"uuid":"d1f52435-7bc1-4081-9550-9cd1ba55f549","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":4,"uuid":"e2ae8b6e-6e25-49b2-bcbf-aacce68f06a3","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":5,"uuid":"3cec8a32-7698-4c36-8e89-e4ba11934f33","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":6,"uuid":"58a65d5d-e090-473a-81b4-ad38e85029f9","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":7,"uuid":"ea5b7f70-3554-4cf8-ac30-f7c59b80678d","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-11-26T07:17:39.955 INFO:teuthology.misc.health.smithi143.stderr:2023-11-26T07:17:39.951+0000 7f046da46640 1 -- 172.21.15.143:0/203832382 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f04600e2370 con 0x7f04600d9a90 2023-11-26T07:17:39.955 INFO:teuthology.misc.health.smithi143.stderr:2023-11-26T07:17:39.951+0000 7f044ffff640 1 -- 172.21.15.143:0/203832382 <== mon.0 v2:172.21.15.143:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v12) v1 ==== 74+0+7057 (secure 0 0 0) 0x7f04600e2370 con 0x7f04600d9a90 2023-11-26T07:17:39.958 INFO:teuthology.misc.health.smithi143.stderr:2023-11-26T07:17:39.955+0000 7f046da46640 1 -- 172.21.15.143:0/203832382 >> [v2:172.21.15.143:6832/2150557544,v1:172.21.15.143:6833/2150557544] conn(0x7f0454066460 msgr2=0x7f0454068920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:17:39.958 INFO:teuthology.misc.health.smithi143.stderr:2023-11-26T07:17:39.955+0000 7f046da46640 1 --2- 172.21.15.143:0/203832382 >> [v2:172.21.15.143:6832/2150557544,v1:172.21.15.143:6833/2150557544] conn(0x7f0454066460 0x7f0454068920 secure :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0x7f04580047c0 tx=0x7f0458008040 comp rx=0 tx=0).stop 2023-11-26T07:17:39.958 INFO:teuthology.misc.health.smithi143.stderr:2023-11-26T07:17:39.955+0000 7f046da46640 1 -- 172.21.15.143:0/203832382 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f04600d9a90 msgr2=0x7f04600544a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:17:39.958 INFO:teuthology.misc.health.smithi143.stderr:2023-11-26T07:17:39.955+0000 7f046da46640 1 --2- 172.21.15.143:0/203832382 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f04600d9a90 0x7f04600544a0 secure :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0x7f046804fe40 tx=0x7f0450005800 comp rx=0 tx=0).stop 2023-11-26T07:17:39.959 INFO:teuthology.misc.health.smithi143.stderr:2023-11-26T07:17:39.955+0000 7f046da46640 1 -- 172.21.15.143:0/203832382 shutdown_connections 2023-11-26T07:17:39.959 INFO:teuthology.misc.health.smithi143.stderr:2023-11-26T07:17:39.955+0000 7f046da46640 1 --2- 172.21.15.143:0/203832382 >> [v2:172.21.15.143:6832/2150557544,v1:172.21.15.143:6833/2150557544] conn(0x7f0454066460 0x7f0454068920 unknown :-1 s=CLOSED pgs=25 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:17:39.959 INFO:teuthology.misc.health.smithi143.stderr:2023-11-26T07:17:39.955+0000 7f046da46640 1 --2- 172.21.15.143:0/203832382 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f04600d9a90 0x7f04600544a0 unknown :-1 s=CLOSED pgs=34 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:17:39.959 INFO:teuthology.misc.health.smithi143.stderr:2023-11-26T07:17:39.955+0000 7f046da46640 1 --2- 172.21.15.143:0/203832382 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f04600e0630 0x7f0460053f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:17:39.959 INFO:teuthology.misc.health.smithi143.stderr:2023-11-26T07:17:39.955+0000 7f046da46640 1 --2- 172.21.15.143:0/203832382 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f0450005190 0x7f04600e1530 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:17:39.959 INFO:teuthology.misc.health.smithi143.stderr:2023-11-26T07:17:39.955+0000 7f046da46640 1 -- 172.21.15.143:0/203832382 >> 172.21.15.143:0/203832382 conn(0x7f04600563b0 msgr2=0x7f046003afc0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:17:39.960 INFO:teuthology.misc.health.smithi143.stderr:2023-11-26T07:17:39.955+0000 7f046da46640 1 -- 172.21.15.143:0/203832382 shutdown_connections 2023-11-26T07:17:39.960 INFO:teuthology.misc.health.smithi143.stderr:2023-11-26T07:17:39.955+0000 7f046da46640 1 -- 172.21.15.143:0/203832382 wait complete. 2023-11-26T07:17:39.988 DEBUG:teuthology.misc:0 of 8 OSDs are up 2023-11-26T07:17:40.130 INFO:tasks.ceph.osd.4.smithi188.stderr:2023-11-26T07:17:40.127+0000 7ffbafc91640 -1 osd.4 0 waiting for initial osdmap 2023-11-26T07:17:40.135 INFO:tasks.ceph.osd.5.smithi188.stderr:2023-11-26T07:17:40.131+0000 7f535cc1e640 -1 osd.5 0 waiting for initial osdmap 2023-11-26T07:17:40.154 INFO:tasks.ceph.osd.7.smithi188.stderr:2023-11-26T07:17:40.151+0000 7f673d801640 -1 osd.7 0 waiting for initial osdmap 2023-11-26T07:17:40.159 INFO:tasks.ceph.osd.4.smithi188.stderr:2023-11-26T07:17:40.155+0000 7ffba6aa8640 -1 osd.4 12 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2023-11-26T07:17:40.170 INFO:tasks.ceph.osd.5.smithi188.stderr:2023-11-26T07:17:40.167+0000 7f535526b640 -1 osd.5 12 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2023-11-26T07:17:40.187 INFO:tasks.ceph.osd.6.smithi188.stderr:2023-11-26T07:17:40.183+0000 7f593a578640 -1 osd.6 0 waiting for initial osdmap 2023-11-26T07:17:40.188 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-11-26T07:17:40.187+0000 7f0cac15f640 -1 osd.0 0 waiting for initial osdmap 2023-11-26T07:17:40.199 INFO:tasks.ceph.osd.7.smithi188.stderr:2023-11-26T07:17:40.195+0000 7f6735e4e640 -1 osd.7 12 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2023-11-26T07:17:40.206 INFO:tasks.ceph.osd.2.smithi143.stderr:2023-11-26T07:17:40.203+0000 7f6578d29640 -1 osd.2 0 waiting for initial osdmap 2023-11-26T07:17:40.214 INFO:tasks.ceph.osd.6.smithi188.stderr:2023-11-26T07:17:40.211+0000 7f5932bc5640 -1 osd.6 12 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2023-11-26T07:17:40.225 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-11-26T07:17:40.223+0000 7f0ca47ac640 -1 osd.0 12 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2023-11-26T07:17:40.245 INFO:tasks.ceph.osd.2.smithi143.stderr:2023-11-26T07:17:40.243+0000 7f6571376640 -1 osd.2 12 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2023-11-26T07:17:40.249 INFO:tasks.ceph.osd.3.smithi143.stderr:2023-11-26T07:17:40.247+0000 7f9477d67640 -1 osd.3 0 waiting for initial osdmap 2023-11-26T07:17:40.262 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-11-26T07:17:40.259+0000 7f87f0eee640 -1 osd.1 0 waiting for initial osdmap 2023-11-26T07:17:40.289 INFO:tasks.ceph.osd.3.smithi143.stderr:2023-11-26T07:17:40.287+0000 7f94703b4640 -1 osd.3 12 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2023-11-26T07:17:40.301 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-11-26T07:17:40.299+0000 7f87e7d05640 -1 osd.1 12 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2023-11-26T07:17:46.794 DEBUG:teuthology.orchestra.run.smithi143:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph --cluster ceph osd dump --format=json 2023-11-26T07:17:46.923 INFO:teuthology.misc.health.smithi143.stderr:2023-11-26T07:17:46.919+0000 7fc0b8279640 1 Processor -- start 2023-11-26T07:17:46.923 INFO:teuthology.misc.health.smithi143.stderr:2023-11-26T07:17:46.919+0000 7fc0b8279640 1 -- start start 2023-11-26T07:17:46.927 INFO:teuthology.misc.health.smithi143.stderr:2023-11-26T07:17:46.923+0000 7fc0b8279640 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7fc0b00ceac0 0x7fc0b00b3bd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:17:46.927 INFO:teuthology.misc.health.smithi143.stderr:2023-11-26T07:17:46.923+0000 7fc0b8279640 1 -- --> v1:172.21.15.143:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc0b00b3410 con 0x7fc0b00ce630 2023-11-26T07:17:46.927 INFO:teuthology.misc.health.smithi143.stderr:2023-11-26T07:17:46.923+0000 7fc0b8279640 1 -- --> v1:172.21.15.188:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc0b015a5c0 con 0x7fc0b015d2f0 2023-11-26T07:17:46.927 INFO:teuthology.misc.health.smithi143.stderr:2023-11-26T07:17:46.923+0000 7fc0b8279640 1 -- --> v2:172.21.15.188:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc0b006b320 con 0x7fc0b00ceac0 2023-11-26T07:17:46.927 INFO:teuthology.misc.health.smithi143.stderr:2023-11-26T07:17:46.923+0000 7fc0b5814640 1 --1- >> v1:172.21.15.143:6789/0 conn(0x7fc0b00ce630 0x7fc0b006ac10 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.143:6789/0 says I am v1:172.21.15.143:57432/0 (socket says 172.21.15.143:57432) 2023-11-26T07:17:46.927 INFO:teuthology.misc.health.smithi143.stderr:2023-11-26T07:17:46.923+0000 7fc0b5814640 1 -- 172.21.15.143:0/3859484734 learned_addr learned my addr 172.21.15.143:0/3859484734 (peer_addr_for_me v1:172.21.15.143:0/0) 2023-11-26T07:17:46.927 INFO:teuthology.misc.health.smithi143.stderr:2023-11-26T07:17:46.923+0000 7fc0b6816640 1 --2- 172.21.15.143:0/3859484734 >> v2:172.21.15.188:3300/0 conn(0x7fc0b00ceac0 0x7fc0b00b3bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:17:46.927 INFO:teuthology.misc.health.smithi143.stderr:2023-11-26T07:17:46.923+0000 7fc0b5013640 1 -- 172.21.15.143:0/3859484734 <== mon.0 v1:172.21.15.143:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 414592880 0 0) 0x7fc0b00b3410 con 0x7fc0b00ce630 2023-11-26T07:17:46.927 INFO:teuthology.misc.health.smithi143.stderr:2023-11-26T07:17:46.923+0000 7fc0b5013640 1 -- 172.21.15.143:0/3859484734 --> v1:172.21.15.143:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fc09c003530 con 0x7fc0b00ce630 2023-11-26T07:17:46.928 INFO:teuthology.misc.health.smithi143.stderr:2023-11-26T07:17:46.923+0000 7fc0b5013640 1 -- 172.21.15.143:0/3859484734 <== mon.0 v1:172.21.15.143:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1079351470 0 0) 0x7fc09c003530 con 0x7fc0b00ce630 2023-11-26T07:17:46.928 INFO:teuthology.misc.health.smithi143.stderr:2023-11-26T07:17:46.923+0000 7fc0b5013640 1 -- 172.21.15.143:0/3859484734 >> v2:172.21.15.188:3300/0 conn(0x7fc0b00ceac0 msgr2=0x7fc0b00b3bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-26T07:17:46.928 INFO:teuthology.misc.health.smithi143.stderr:2023-11-26T07:17:46.923+0000 7fc0b5013640 1 --2- 172.21.15.143:0/3859484734 >> v2:172.21.15.188:3300/0 conn(0x7fc0b00ceac0 0x7fc0b00b3bd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:17:46.928 INFO:teuthology.misc.health.smithi143.stderr:2023-11-26T07:17:46.923+0000 7fc0b5013640 1 -- 172.21.15.143:0/3859484734 >> v1:172.21.15.188:6789/0 conn(0x7fc0b015d2f0 legacy=0x7fc0b0159d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:17:46.928 INFO:teuthology.misc.health.smithi143.stderr:2023-11-26T07:17:46.923+0000 7fc0b5013640 1 -- 172.21.15.143:0/3859484734 --> v1:172.21.15.143:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc0b015a8a0 con 0x7fc0b00ce630 2023-11-26T07:17:46.928 INFO:teuthology.misc.health.smithi143.stderr:2023-11-26T07:17:46.923+0000 7fc0b5013640 1 -- 172.21.15.143:0/3859484734 <== mon.0 v1:172.21.15.143:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2137882523 0 0) 0x7fc0a0001680 con 0x7fc0b00ce630 2023-11-26T07:17:46.928 INFO:teuthology.misc.health.smithi143.stderr:2023-11-26T07:17:46.923+0000 7fc0b5013640 1 -- 172.21.15.143:0/3859484734 >> v1:172.21.15.143:6789/0 conn(0x7fc0b00ce630 legacy=0x7fc0b006ac10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:17:46.928 INFO:teuthology.misc.health.smithi143.stderr:2023-11-26T07:17:46.923+0000 7fc0b5013640 1 --2- 172.21.15.143:0/3859484734 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fc09c003980 0x7fc09c003da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:17:46.928 INFO:teuthology.misc.health.smithi143.stderr:2023-11-26T07:17:46.923+0000 7fc0b5013640 1 --2- 172.21.15.143:0/3859484734 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fc09c0047d0 0x7fc09c004c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:17:46.928 INFO:teuthology.misc.health.smithi143.stderr:2023-11-26T07:17:46.923+0000 7fc0b5013640 1 --2- 172.21.15.143:0/3859484734 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fc09c005190 0x7fc09c00b990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:17:46.928 INFO:teuthology.misc.health.smithi143.stderr:2023-11-26T07:17:46.923+0000 7fc0b5013640 1 -- 172.21.15.143:0/3859484734 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc0b006b320 con 0x7fc09c003980 2023-11-26T07:17:46.928 INFO:teuthology.misc.health.smithi143.stderr:2023-11-26T07:17:46.923+0000 7fc0b5013640 1 -- 172.21.15.143:0/3859484734 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc09c00c130 con 0x7fc09c0047d0 2023-11-26T07:17:46.929 INFO:teuthology.misc.health.smithi143.stderr:2023-11-26T07:17:46.923+0000 7fc0b5013640 1 -- 172.21.15.143:0/3859484734 --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc09c00c2f0 con 0x7fc09c005190 2023-11-26T07:17:46.929 INFO:teuthology.misc.health.smithi143.stderr:2023-11-26T07:17:46.923+0000 7fc0b6015640 1 --2- 172.21.15.143:0/3859484734 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fc09c003980 0x7fc09c003da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:17:46.929 INFO:teuthology.misc.health.smithi143.stderr:2023-11-26T07:17:46.923+0000 7fc0b5814640 1 --2- 172.21.15.143:0/3859484734 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fc09c0047d0 0x7fc09c004c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:17:46.929 INFO:teuthology.misc.health.smithi143.stderr:2023-11-26T07:17:46.923+0000 7fc0b6816640 1 --2- 172.21.15.143:0/3859484734 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fc09c005190 0x7fc09c00b990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:17:46.929 INFO:teuthology.misc.health.smithi143.stderr:2023-11-26T07:17:46.923+0000 7fc0b6015640 1 -- 172.21.15.143:0/3859484734 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fc09c005190 msgr2=0x7fc09c00b990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-26T07:17:46.929 INFO:teuthology.misc.health.smithi143.stderr:2023-11-26T07:17:46.923+0000 7fc0b6015640 1 --2- 172.21.15.143:0/3859484734 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fc09c005190 0x7fc09c00b990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:17:46.929 INFO:teuthology.misc.health.smithi143.stderr:2023-11-26T07:17:46.923+0000 7fc0b6015640 1 -- 172.21.15.143:0/3859484734 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fc09c0047d0 msgr2=0x7fc09c004c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-26T07:17:46.929 INFO:teuthology.misc.health.smithi143.stderr:2023-11-26T07:17:46.923+0000 7fc0b6015640 1 --2- 172.21.15.143:0/3859484734 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fc09c0047d0 0x7fc09c004c50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:17:46.929 INFO:teuthology.misc.health.smithi143.stderr:2023-11-26T07:17:46.923+0000 7fc0b6015640 1 -- 172.21.15.143:0/3859484734 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc09c003530 con 0x7fc09c003980 2023-11-26T07:17:46.929 INFO:teuthology.misc.health.smithi143.stderr:2023-11-26T07:17:46.923+0000 7fc0b6816640 1 -- 172.21.15.143:0/3859484734 reap_dead start 2023-11-26T07:17:46.930 INFO:teuthology.misc.health.smithi143.stderr:2023-11-26T07:17:46.927+0000 7fc0b6015640 1 --2- 172.21.15.143:0/3859484734 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fc09c003980 0x7fc09c003da0 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7fc0ac001780 tx=0x7fc0ac01c400 comp rx=0 tx=0).ready entity=mon.0 client_cookie=56d48b5fdcc3c96a server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:17:46.930 INFO:teuthology.misc.health.smithi143.stderr:2023-11-26T07:17:46.927+0000 7fc0b5013640 1 -- 172.21.15.143:0/3859484734 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc0ac0052d0 con 0x7fc09c003980 2023-11-26T07:17:46.930 INFO:teuthology.misc.health.smithi143.stderr:2023-11-26T07:17:46.927+0000 7fc0b5013640 1 -- 172.21.15.143:0/3859484734 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc0ac0a7050 con 0x7fc09c003980 2023-11-26T07:17:46.931 INFO:teuthology.misc.health.smithi143.stderr:2023-11-26T07:17:46.927+0000 7fc0b5013640 1 -- 172.21.15.143:0/3859484734 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc0ac003600 con 0x7fc09c003980 2023-11-26T07:17:46.931 INFO:teuthology.misc.health.smithi143.stderr:2023-11-26T07:17:46.927+0000 7fc0b8279640 1 -- 172.21.15.143:0/3859484734 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fc09c003980 msgr2=0x7fc09c003da0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:17:46.931 INFO:teuthology.misc.health.smithi143.stderr:2023-11-26T07:17:46.927+0000 7fc0b8279640 1 --2- 172.21.15.143:0/3859484734 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fc09c003980 0x7fc09c003da0 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7fc0ac001780 tx=0x7fc0ac01c400 comp rx=0 tx=0).stop 2023-11-26T07:17:46.931 INFO:teuthology.misc.health.smithi143.stderr:2023-11-26T07:17:46.927+0000 7fc0b8279640 1 -- 172.21.15.143:0/3859484734 shutdown_connections 2023-11-26T07:17:46.931 INFO:teuthology.misc.health.smithi143.stderr:2023-11-26T07:17:46.927+0000 7fc0b8279640 1 --2- 172.21.15.143:0/3859484734 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fc09c003980 0x7fc09c003da0 unknown :-1 s=CLOSED pgs=39 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:17:46.931 INFO:teuthology.misc.health.smithi143.stderr:2023-11-26T07:17:46.927+0000 7fc0b8279640 1 -- 172.21.15.143:0/3859484734 >> 172.21.15.143:0/3859484734 conn(0x7fc0b006dc00 msgr2=0x7fc0b006a500 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:17:46.931 INFO:teuthology.misc.health.smithi143.stderr:2023-11-26T07:17:46.927+0000 7fc0b8279640 1 -- 172.21.15.143:0/3859484734 shutdown_connections 2023-11-26T07:17:46.931 INFO:teuthology.misc.health.smithi143.stderr:2023-11-26T07:17:46.927+0000 7fc0b8279640 1 -- 172.21.15.143:0/3859484734 wait complete. 2023-11-26T07:17:46.932 INFO:teuthology.misc.health.smithi143.stderr:2023-11-26T07:17:46.931+0000 7fc0b8279640 1 Processor -- start 2023-11-26T07:17:46.932 INFO:teuthology.misc.health.smithi143.stderr:2023-11-26T07:17:46.931+0000 7fc0b8279640 1 -- start start 2023-11-26T07:17:46.935 INFO:teuthology.misc.health.smithi143.stderr:2023-11-26T07:17:46.931+0000 7fc0b8279640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fc09c003980 0x7fc0b013d650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:17:46.935 INFO:teuthology.misc.health.smithi143.stderr:2023-11-26T07:17:46.931+0000 7fc0b8279640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fc0b0142860 0x7fc0b0144090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:17:46.935 INFO:teuthology.misc.health.smithi143.stderr:2023-11-26T07:17:46.931+0000 7fc0b8279640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fc0b0153670 0x7fc0b0145de0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:17:46.935 INFO:teuthology.misc.health.smithi143.stderr:2023-11-26T07:17:46.931+0000 7fc0b8279640 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc0b0142440 con 0x7fc09c003980 2023-11-26T07:17:46.935 INFO:teuthology.misc.health.smithi143.stderr:2023-11-26T07:17:46.931+0000 7fc0b8279640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc0b01445d0 con 0x7fc0b0142860 2023-11-26T07:17:46.935 INFO:teuthology.misc.health.smithi143.stderr:2023-11-26T07:17:46.931+0000 7fc0b8279640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc0b0144850 con 0x7fc0b0153670 2023-11-26T07:17:46.935 INFO:teuthology.misc.health.smithi143.stderr:2023-11-26T07:17:46.931+0000 7fc0b6015640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fc09c003980 0x7fc0b013d650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:17:46.935 INFO:teuthology.misc.health.smithi143.stderr:2023-11-26T07:17:46.931+0000 7fc0b6015640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fc09c003980 0x7fc0b013d650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:52952/0 (socket says 172.21.15.143:52952) 2023-11-26T07:17:46.936 INFO:teuthology.misc.health.smithi143.stderr:2023-11-26T07:17:46.931+0000 7fc0b6015640 1 -- 172.21.15.143:0/1827931550 learned_addr learned my addr 172.21.15.143:0/1827931550 (peer_addr_for_me v2:172.21.15.143:0/0) 2023-11-26T07:17:46.936 INFO:teuthology.misc.health.smithi143.stderr:2023-11-26T07:17:46.931+0000 7fc0b6816640 1 --2- 172.21.15.143:0/1827931550 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fc0b0153670 0x7fc0b0145de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:17:46.936 INFO:teuthology.misc.health.smithi143.stderr:2023-11-26T07:17:46.931+0000 7fc0b5814640 1 --2- 172.21.15.143:0/1827931550 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fc0b0142860 0x7fc0b0144090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:17:46.936 INFO:teuthology.misc.health.smithi143.stderr:2023-11-26T07:17:46.931+0000 7fc0b6015640 1 -- 172.21.15.143:0/1827931550 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fc0b0153670 msgr2=0x7fc0b0145de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:17:46.936 INFO:teuthology.misc.health.smithi143.stderr:2023-11-26T07:17:46.931+0000 7fc0b6015640 1 --2- 172.21.15.143:0/1827931550 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fc0b0153670 0x7fc0b0145de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:17:46.936 INFO:teuthology.misc.health.smithi143.stderr:2023-11-26T07:17:46.931+0000 7fc0b6015640 1 -- 172.21.15.143:0/1827931550 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fc0b0142860 msgr2=0x7fc0b0144090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:17:46.936 INFO:teuthology.misc.health.smithi143.stderr:2023-11-26T07:17:46.931+0000 7fc0b6015640 1 --2- 172.21.15.143:0/1827931550 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fc0b0142860 0x7fc0b0144090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:17:46.936 INFO:teuthology.misc.health.smithi143.stderr:2023-11-26T07:17:46.931+0000 7fc0b6015640 1 -- 172.21.15.143:0/1827931550 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc0b0141aa0 con 0x7fc09c003980 2023-11-26T07:17:46.936 INFO:teuthology.misc.health.smithi143.stderr:2023-11-26T07:17:46.931+0000 7fc0b6015640 1 --2- 172.21.15.143:0/1827931550 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fc09c003980 0x7fc0b013d650 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7fc0ac01c230 tx=0x7fc09c005800 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:17:46.936 INFO:teuthology.misc.health.smithi143.stderr:2023-11-26T07:17:46.931+0000 7fc0a6ffd640 1 -- 172.21.15.143:0/1827931550 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc0ac007220 con 0x7fc09c003980 2023-11-26T07:17:46.936 INFO:teuthology.misc.health.smithi143.stderr:2023-11-26T07:17:46.931+0000 7fc0a6ffd640 1 -- 172.21.15.143:0/1827931550 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc0ac007380 con 0x7fc09c003980 2023-11-26T07:17:46.936 INFO:teuthology.misc.health.smithi143.stderr:2023-11-26T07:17:46.931+0000 7fc0a6ffd640 1 -- 172.21.15.143:0/1827931550 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc0ac0067e0 con 0x7fc09c003980 2023-11-26T07:17:46.937 INFO:teuthology.misc.health.smithi143.stderr:2023-11-26T07:17:46.931+0000 7fc0b8279640 1 -- 172.21.15.143:0/1827931550 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc0b0141d80 con 0x7fc09c003980 2023-11-26T07:17:46.937 INFO:teuthology.misc.health.smithi143.stderr:2023-11-26T07:17:46.931+0000 7fc0b8279640 1 -- 172.21.15.143:0/1827931550 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc0b0142cc0 con 0x7fc09c003980 2023-11-26T07:17:46.937 INFO:teuthology.misc.health.smithi143.stderr:2023-11-26T07:17:46.931+0000 7fc0b8279640 1 -- 172.21.15.143:0/1827931550 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc0b0067680 con 0x7fc09c003980 2023-11-26T07:17:46.938 INFO:teuthology.misc.health.smithi143.stderr:2023-11-26T07:17:46.935+0000 7fc0a6ffd640 1 -- 172.21.15.143:0/1827931550 <== mon.0 v2:172.21.15.143:3300/0 4 ==== mgrmap(e 5) v1 ==== 85852+0+0 (secure 0 0 0) 0x7fc0ac006940 con 0x7fc09c003980 2023-11-26T07:17:46.938 INFO:teuthology.misc.health.smithi143.stderr:2023-11-26T07:17:46.935+0000 7fc0a6ffd640 1 --2- 172.21.15.143:0/1827931550 >> [v2:172.21.15.143:6832/2150557544,v1:172.21.15.143:6833/2150557544] conn(0x7fc0940664a0 0x7fc094068960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:17:46.938 INFO:teuthology.misc.health.smithi143.stderr:2023-11-26T07:17:46.935+0000 7fc0a6ffd640 1 -- 172.21.15.143:0/1827931550 <== mon.0 v2:172.21.15.143:3300/0 5 ==== osd_map(16..16 src has 1..16) v4 ==== 4857+0+0 (secure 0 0 0) 0x7fc0ac11d6e0 con 0x7fc09c003980 2023-11-26T07:17:46.938 INFO:teuthology.misc.health.smithi143.stderr:2023-11-26T07:17:46.935+0000 7fc0b5814640 1 --2- 172.21.15.143:0/1827931550 >> [v2:172.21.15.143:6832/2150557544,v1:172.21.15.143:6833/2150557544] conn(0x7fc0940664a0 0x7fc094068960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:17:46.938 INFO:teuthology.misc.health.smithi143.stderr:2023-11-26T07:17:46.935+0000 7fc0b5814640 1 --2- 172.21.15.143:0/1827931550 >> [v2:172.21.15.143:6832/2150557544,v1:172.21.15.143:6833/2150557544] conn(0x7fc0940664a0 0x7fc094068960 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7fc0b00b0730 tx=0x7fc0a0009040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:17:46.942 INFO:teuthology.misc.health.smithi143.stderr:2023-11-26T07:17:46.939+0000 7fc0a6ffd640 1 -- 172.21.15.143:0/1827931550 <== mon.0 v2:172.21.15.143:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172649 (secure 0 0 0) 0x7fc0ac0eaea0 con 0x7fc09c003980 2023-11-26T07:17:47.094 INFO:teuthology.misc.health.smithi143.stderr:2023-11-26T07:17:47.091+0000 7fc0b8279640 1 -- 172.21.15.143:0/1827931550 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fc0b0067d30 con 0x7fc09c003980 2023-11-26T07:17:47.095 INFO:teuthology.misc.health.smithi143.stderr:2023-11-26T07:17:47.091+0000 7fc0a6ffd640 1 -- 172.21.15.143:0/1827931550 <== mon.0 v2:172.21.15.143:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v16) v1 ==== 74+0+12886 (secure 0 0 0) 0x7fc0ac0ee4c0 con 0x7fc09c003980 2023-11-26T07:17:47.095 INFO:teuthology.misc.health.smithi143.stdout: 2023-11-26T07:17:47.096 INFO:teuthology.misc.health.smithi143.stdout:{"epoch":16,"fsid":"f5033060-2469-4a07-95fe-2b76f5404bc1","created":"2023-11-26T07:17:28.613663+0000","modified":"2023-11-26T07:17:44.413749+0000","last_up_change":"2023-11-26T07:17:41.157813+0000","last_in_change":"2023-11-26T07:17:31.181658+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":1,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-11-26T07:17:41.758597+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"ebbee2c6-023e-44b9-9f1d-6f7bfd309730","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6817","nonce":639807369},{"type":"v1","addr":"172.21.15.143:6819","nonce":639807369}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6821","nonce":639807369},{"type":"v1","addr":"172.21.15.143:6823","nonce":639807369}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6830","nonce":639807369},{"type":"v1","addr":"172.21.15.143:6831","nonce":639807369}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6825","nonce":639807369},{"type":"v1","addr":"172.21.15.143:6828","nonce":639807369}]},"public_addr":"172.21.15.143:6819/639807369","cluster_addr":"172.21.15.143:6823/639807369","heartbeat_back_addr":"172.21.15.143:6831/639807369","heartbeat_front_addr":"172.21.15.143:6828/639807369","state":["exists","up"]},{"osd":1,"uuid":"fec4b8be-7122-49e9-9a19-5850a0700001","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6816","nonce":1144942530},{"type":"v1","addr":"172.21.15.143:6818","nonce":1144942530}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6820","nonce":1144942530},{"type":"v1","addr":"172.21.15.143:6822","nonce":1144942530}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6827","nonce":1144942530},{"type":"v1","addr":"172.21.15.143:6829","nonce":1144942530}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6824","nonce":1144942530},{"type":"v1","addr":"172.21.15.143:6826","nonce":1144942530}]},"public_addr":"172.21.15.143:6818/1144942530","cluster_addr":"172.21.15.143:6822/1144942530","heartbeat_back_addr":"172.21.15.143:6829/1144942530","heartbeat_front_addr":"172.21.15.143:6826/1144942530","state":["exists","up"]},{"osd":2,"uuid":"2888af5b-f933-46ca-8f95-6dd494ff43f6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6808","nonce":3909095186},{"type":"v1","addr":"172.21.15.143:6809","nonce":3909095186}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6810","nonce":3909095186},{"type":"v1","addr":"172.21.15.143:6811","nonce":3909095186}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6814","nonce":3909095186},{"type":"v1","addr":"172.21.15.143:6815","nonce":3909095186}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6812","nonce":3909095186},{"type":"v1","addr":"172.21.15.143:6813","nonce":3909095186}]},"public_addr":"172.21.15.143:6809/3909095186","cluster_addr":"172.21.15.143:6811/3909095186","heartbeat_back_addr":"172.21.15.143:6815/3909095186","heartbeat_front_addr":"172.21.15.143:6813/3909095186","state":["exists","up"]},{"osd":3,"uuid":"d1f52435-7bc1-4081-9550-9cd1ba55f549","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6800","nonce":3854210991},{"type":"v1","addr":"172.21.15.143:6801","nonce":3854210991}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6802","nonce":3854210991},{"type":"v1","addr":"172.21.15.143:6803","nonce":3854210991}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6806","nonce":3854210991},{"type":"v1","addr":"172.21.15.143:6807","nonce":3854210991}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6804","nonce":3854210991},{"type":"v1","addr":"172.21.15.143:6805","nonce":3854210991}]},"public_addr":"172.21.15.143:6801/3854210991","cluster_addr":"172.21.15.143:6803/3854210991","heartbeat_back_addr":"172.21.15.143:6807/3854210991","heartbeat_front_addr":"172.21.15.143:6805/3854210991","state":["exists","up"]},{"osd":4,"uuid":"e2ae8b6e-6e25-49b2-bcbf-aacce68f06a3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6800","nonce":3924606902},{"type":"v1","addr":"172.21.15.188:6801","nonce":3924606902}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6802","nonce":3924606902},{"type":"v1","addr":"172.21.15.188:6803","nonce":3924606902}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6806","nonce":3924606902},{"type":"v1","addr":"172.21.15.188:6807","nonce":3924606902}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6804","nonce":3924606902},{"type":"v1","addr":"172.21.15.188:6805","nonce":3924606902}]},"public_addr":"172.21.15.188:6801/3924606902","cluster_addr":"172.21.15.188:6803/3924606902","heartbeat_back_addr":"172.21.15.188:6807/3924606902","heartbeat_front_addr":"172.21.15.188:6805/3924606902","state":["exists","up"]},{"osd":5,"uuid":"3cec8a32-7698-4c36-8e89-e4ba11934f33","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6813","nonce":186841544},{"type":"v1","addr":"172.21.15.188:6815","nonce":186841544}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6817","nonce":186841544},{"type":"v1","addr":"172.21.15.188:6818","nonce":186841544}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6822","nonce":186841544},{"type":"v1","addr":"172.21.15.188:6823","nonce":186841544}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6820","nonce":186841544},{"type":"v1","addr":"172.21.15.188:6821","nonce":186841544}]},"public_addr":"172.21.15.188:6815/186841544","cluster_addr":"172.21.15.188:6818/186841544","heartbeat_back_addr":"172.21.15.188:6823/186841544","heartbeat_front_addr":"172.21.15.188:6821/186841544","state":["exists","up"]},{"osd":6,"uuid":"58a65d5d-e090-473a-81b4-ad38e85029f9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6808","nonce":2153499405},{"type":"v1","addr":"172.21.15.188:6809","nonce":2153499405}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6810","nonce":2153499405},{"type":"v1","addr":"172.21.15.188:6811","nonce":2153499405}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6816","nonce":2153499405},{"type":"v1","addr":"172.21.15.188:6819","nonce":2153499405}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6812","nonce":2153499405},{"type":"v1","addr":"172.21.15.188:6814","nonce":2153499405}]},"public_addr":"172.21.15.188:6809/2153499405","cluster_addr":"172.21.15.188:6811/2153499405","heartbeat_back_addr":"172.21.15.188:6819/2153499405","heartbeat_front_addr":"172.21.15.188:6814/2153499405","state":["exists","up"]},{"osd":7,"uuid":"ea5b7f70-3554-4cf8-ac30-f7c59b80678d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":14,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6824","nonce":2722096729},{"type":"v1","addr":"172.21.15.188:6825","nonce":2722096729}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6826","nonce":2722096729},{"type":"v1","addr":"172.21.15.188:6827","nonce":2722096729}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6830","nonce":2722096729},{"type":"v1","addr":"172.21.15.188:6831","nonce":2722096729}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6828","nonce":2722096729},{"type":"v1","addr":"172.21.15.188:6829","nonce":2722096729}]},"public_addr":"172.21.15.188:6825/2722096729","cluster_addr":"172.21.15.188:6827/2722096729","heartbeat_back_addr":"172.21.15.188:6831/2722096729","heartbeat_front_addr":"172.21.15.188:6829/2722096729","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-26T07:17:36.995032+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-26T07:17:36.932266+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-26T07:17:36.829304+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-26T07:17:36.804297+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-26T07:17:36.667892+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-26T07:17:36.964868+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-26T07:17:36.827332+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-26T07:17:36.893179+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-11-26T07:17:47.097 INFO:teuthology.misc.health.smithi143.stderr:2023-11-26T07:17:47.095+0000 7fc0b8279640 1 -- 172.21.15.143:0/1827931550 >> [v2:172.21.15.143:6832/2150557544,v1:172.21.15.143:6833/2150557544] conn(0x7fc0940664a0 msgr2=0x7fc094068960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:17:47.098 INFO:teuthology.misc.health.smithi143.stderr:2023-11-26T07:17:47.095+0000 7fc0b8279640 1 --2- 172.21.15.143:0/1827931550 >> [v2:172.21.15.143:6832/2150557544,v1:172.21.15.143:6833/2150557544] conn(0x7fc0940664a0 0x7fc094068960 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7fc0b00b0730 tx=0x7fc0a0009040 comp rx=0 tx=0).stop 2023-11-26T07:17:47.098 INFO:teuthology.misc.health.smithi143.stderr:2023-11-26T07:17:47.095+0000 7fc0b8279640 1 -- 172.21.15.143:0/1827931550 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fc09c003980 msgr2=0x7fc0b013d650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:17:47.098 INFO:teuthology.misc.health.smithi143.stderr:2023-11-26T07:17:47.095+0000 7fc0b8279640 1 --2- 172.21.15.143:0/1827931550 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fc09c003980 0x7fc0b013d650 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7fc0ac01c230 tx=0x7fc09c005800 comp rx=0 tx=0).stop 2023-11-26T07:17:47.098 INFO:teuthology.misc.health.smithi143.stderr:2023-11-26T07:17:47.095+0000 7fc0b8279640 1 -- 172.21.15.143:0/1827931550 shutdown_connections 2023-11-26T07:17:47.098 INFO:teuthology.misc.health.smithi143.stderr:2023-11-26T07:17:47.095+0000 7fc0b8279640 1 --2- 172.21.15.143:0/1827931550 >> [v2:172.21.15.143:6832/2150557544,v1:172.21.15.143:6833/2150557544] conn(0x7fc0940664a0 0x7fc094068960 unknown :-1 s=CLOSED pgs=51 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:17:47.098 INFO:teuthology.misc.health.smithi143.stderr:2023-11-26T07:17:47.095+0000 7fc0b8279640 1 --2- 172.21.15.143:0/1827931550 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fc0b0153670 0x7fc0b0145de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:17:47.098 INFO:teuthology.misc.health.smithi143.stderr:2023-11-26T07:17:47.095+0000 7fc0b8279640 1 --2- 172.21.15.143:0/1827931550 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fc0b0142860 0x7fc0b0144090 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:17:47.098 INFO:teuthology.misc.health.smithi143.stderr:2023-11-26T07:17:47.095+0000 7fc0b8279640 1 --2- 172.21.15.143:0/1827931550 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fc09c003980 0x7fc0b013d650 unknown :-1 s=CLOSED pgs=40 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:17:47.098 INFO:teuthology.misc.health.smithi143.stderr:2023-11-26T07:17:47.095+0000 7fc0b8279640 1 -- 172.21.15.143:0/1827931550 >> 172.21.15.143:0/1827931550 conn(0x7fc0b006dc00 msgr2=0x7fc0b014e6d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:17:47.098 INFO:teuthology.misc.health.smithi143.stderr:2023-11-26T07:17:47.095+0000 7fc0b8279640 1 -- 172.21.15.143:0/1827931550 shutdown_connections 2023-11-26T07:17:47.098 INFO:teuthology.misc.health.smithi143.stderr:2023-11-26T07:17:47.095+0000 7fc0b8279640 1 -- 172.21.15.143:0/1827931550 wait complete. 2023-11-26T07:17:47.115 DEBUG:teuthology.misc:8 of 8 OSDs are up 2023-11-26T07:17:47.115 INFO:tasks.ceph:Starting mds daemons in cluster ceph... 2023-11-26T07:17:47.115 INFO:tasks.ceph.mds.a:Restarting daemon 2023-11-26T07:17:47.115 DEBUG:teuthology.orchestra.run.smithi143:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i a 2023-11-26T07:17:47.119 INFO:tasks.ceph.mds.a:Started 2023-11-26T07:17:47.119 INFO:tasks.ceph.mds.c:Restarting daemon 2023-11-26T07:17:47.119 DEBUG:teuthology.orchestra.run.smithi143:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i c 2023-11-26T07:17:47.123 INFO:tasks.ceph.mds.c:Started 2023-11-26T07:17:47.123 INFO:tasks.ceph.mds.b:Restarting daemon 2023-11-26T07:17:47.123 DEBUG:teuthology.orchestra.run.smithi188:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i b 2023-11-26T07:17:47.127 INFO:tasks.ceph.mds.b:Started 2023-11-26T07:17:47.127 INFO:tasks.ceph.mds.d:Restarting daemon 2023-11-26T07:17:47.127 DEBUG:teuthology.orchestra.run.smithi188:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i d 2023-11-26T07:17:47.132 INFO:tasks.ceph.mds.d:Started 2023-11-26T07:17:47.132 INFO:tasks.ceph:Setting up CephFS filesystem(s)... 2023-11-26T07:17:47.132 DEBUG:teuthology.orchestra.run.smithi143:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-11-26T07:17:47.222 INFO:tasks.ceph.mds.c.smithi143.stdout:starting mds.c at 2023-11-26T07:17:47.223 INFO:tasks.ceph.mds.a.smithi143.stdout:starting mds.a at 2023-11-26T07:17:47.241 INFO:tasks.ceph.mds.d.smithi188.stdout:starting mds.d at 2023-11-26T07:17:47.242 INFO:tasks.ceph.mds.b.smithi188.stdout:starting mds.b at 2023-11-26T07:17:47.279 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:17:47.275+0000 7f83061c1640 1 Processor -- start 2023-11-26T07:17:47.280 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:17:47.275+0000 7f83061c1640 1 -- start start 2023-11-26T07:17:47.281 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:17:47.279+0000 7f83061c1640 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7f83000ce680 0x7f8300071850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:17:47.281 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:17:47.279+0000 7f83061c1640 1 -- --> v1:172.21.15.143:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f830006c750 con 0x7f83001596d0 2023-11-26T07:17:47.281 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:17:47.279+0000 7f83061c1640 1 -- --> v1:172.21.15.188:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f83000b3550 con 0x7f83000b2f80 2023-11-26T07:17:47.282 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:17:47.279+0000 7f83061c1640 1 -- --> v2:172.21.15.188:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f83000cbe00 con 0x7f83000ce680 2023-11-26T07:17:47.282 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:17:47.279+0000 7f82ff7fe640 1 --1- >> v1:172.21.15.143:6789/0 conn(0x7f83001596d0 0x7f83000b2870 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.143:6789/0 says I am v1:172.21.15.143:57438/0 (socket says 172.21.15.143:57438) 2023-11-26T07:17:47.282 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:17:47.279+0000 7f82ff7fe640 1 -- 172.21.15.143:0/3294516417 learned_addr learned my addr 172.21.15.143:0/3294516417 (peer_addr_for_me v1:172.21.15.143:0/0) 2023-11-26T07:17:47.282 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:17:47.279+0000 7f82fffff640 1 --2- 172.21.15.143:0/3294516417 >> v2:172.21.15.188:3300/0 conn(0x7f83000ce680 0x7f8300071850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:17:47.282 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:17:47.279+0000 7f82fe7fc640 1 -- 172.21.15.143:0/3294516417 <== mon.0 v1:172.21.15.143:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2364081091 0 0) 0x7f830006c750 con 0x7f83001596d0 2023-11-26T07:17:47.283 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:17:47.279+0000 7f82fe7fc640 1 -- 172.21.15.143:0/3294516417 --> v1:172.21.15.143:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f82e8005740 con 0x7f83001596d0 2023-11-26T07:17:47.283 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:17:47.279+0000 7f82fffff640 1 -- 172.21.15.143:0/3294516417 >> v1:172.21.15.188:6789/0 conn(0x7f83000b2f80 legacy=0x7f830006cbe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:17:47.283 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:17:47.279+0000 7f82fe7fc640 1 -- 172.21.15.143:0/3294516417 <== mon.1 v1:172.21.15.188:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1759319281 0 0) 0x7f83000b3550 con 0x7f83000b2f80 2023-11-26T07:17:47.283 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:17:47.279+0000 7f82fffff640 1 -- 172.21.15.143:0/3294516417 >> v1:172.21.15.143:6789/0 conn(0x7f83001596d0 legacy=0x7f83000b2870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:17:47.283 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:17:47.279+0000 7f82fffff640 1 -- 172.21.15.143:0/3294516417 --> v2:172.21.15.188:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8300071e80 con 0x7f83000ce680 2023-11-26T07:17:47.285 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:17:47.283+0000 7f82fffff640 1 --2- 172.21.15.143:0/3294516417 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f83000ce680 0x7f8300071850 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7f83000cbf40 tx=0x7f82f801c8a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=1e65c2095bb55414 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:17:47.285 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:17:47.283+0000 7f82fe7fc640 1 -- 172.21.15.143:0/3294516417 <== mon.1 v2:172.21.15.188:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f82f80a57d0 con 0x7f83000ce680 2023-11-26T07:17:47.285 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:17:47.283+0000 7f82fe7fc640 1 -- 172.21.15.143:0/3294516417 <== mon.1 v2:172.21.15.188:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f82f80a5930 con 0x7f83000ce680 2023-11-26T07:17:47.285 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:17:47.283+0000 7f82fe7fc640 1 -- 172.21.15.143:0/3294516417 <== mon.1 v2:172.21.15.188:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f82f80a5cc0 con 0x7f83000ce680 2023-11-26T07:17:47.286 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:17:47.283+0000 7f83061c1640 1 -- 172.21.15.143:0/3294516417 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f83000ce680 msgr2=0x7f8300071850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:17:47.286 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:17:47.283+0000 7f83061c1640 1 --2- 172.21.15.143:0/3294516417 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f83000ce680 0x7f8300071850 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7f83000cbf40 tx=0x7f82f801c8a0 comp rx=0 tx=0).stop 2023-11-26T07:17:47.286 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:17:47.283+0000 7f83061c1640 1 -- 172.21.15.143:0/3294516417 shutdown_connections 2023-11-26T07:17:47.286 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:17:47.283+0000 7f83061c1640 1 --2- 172.21.15.143:0/3294516417 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f83000ce680 0x7f8300071850 unknown :-1 s=CLOSED pgs=39 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:17:47.286 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:17:47.283+0000 7f83061c1640 1 -- 172.21.15.143:0/3294516417 >> 172.21.15.143:0/3294516417 conn(0x7f830006a890 msgr2=0x7f830006aca0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:17:47.286 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:17:47.283+0000 7f83061c1640 1 -- 172.21.15.143:0/3294516417 shutdown_connections 2023-11-26T07:17:47.286 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:17:47.283+0000 7f83061c1640 1 -- 172.21.15.143:0/3294516417 wait complete. 2023-11-26T07:17:47.287 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:17:47.283+0000 7f83061c1640 1 Processor -- start 2023-11-26T07:17:47.287 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:17:47.283+0000 7f83061c1640 1 -- start start 2023-11-26T07:17:47.288 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:17:47.287+0000 7f83061c1640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f83000b2f80 0x7f83000c47e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:17:47.289 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:17:47.287+0000 7f83061c1640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f83000ce680 0x7f83000ac8e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:17:47.289 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:17:47.287+0000 7f83061c1640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f83001596d0 0x7f83000ace20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:17:47.289 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:17:47.287+0000 7f83061c1640 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f830014f970 con 0x7f83000b2f80 2023-11-26T07:17:47.289 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:17:47.287+0000 7f83061c1640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f830014ea80 con 0x7f83000ce680 2023-11-26T07:17:47.289 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:17:47.287+0000 7f83061c1640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f830014ed60 con 0x7f83001596d0 2023-11-26T07:17:47.289 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:17:47.287+0000 7f82ff7fe640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f83000b2f80 0x7f83000c47e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:17:47.289 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:17:47.287+0000 7f82ff7fe640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f83000b2f80 0x7f83000c47e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:52978/0 (socket says 172.21.15.143:52978) 2023-11-26T07:17:47.289 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:17:47.287+0000 7f82ff7fe640 1 -- 172.21.15.143:0/976834882 learned_addr learned my addr 172.21.15.143:0/976834882 (peer_addr_for_me v2:172.21.15.143:0/0) 2023-11-26T07:17:47.290 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:17:47.287+0000 7f82fffff640 1 --2- 172.21.15.143:0/976834882 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f83001596d0 0x7f83000ace20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:17:47.290 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:17:47.287+0000 7f82feffd640 1 --2- 172.21.15.143:0/976834882 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f83000ce680 0x7f83000ac8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:17:47.291 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:17:47.287+0000 7f82ff7fe640 1 -- 172.21.15.143:0/976834882 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f83001596d0 msgr2=0x7f83000ace20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:17:47.291 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:17:47.287+0000 7f82ff7fe640 1 --2- 172.21.15.143:0/976834882 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f83001596d0 0x7f83000ace20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:17:47.291 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:17:47.287+0000 7f82ff7fe640 1 -- 172.21.15.143:0/976834882 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f83000ce680 msgr2=0x7f83000ac8e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:17:47.291 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:17:47.287+0000 7f82ff7fe640 1 --2- 172.21.15.143:0/976834882 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f83000ce680 0x7f83000ac8e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:17:47.291 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:17:47.287+0000 7f82ff7fe640 1 -- 172.21.15.143:0/976834882 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f82f8009a70 con 0x7f83000b2f80 2023-11-26T07:17:47.291 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:17:47.287+0000 7f82fffff640 1 --2- 172.21.15.143:0/976834882 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f83001596d0 0x7f83000ace20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-26T07:17:47.291 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:17:47.287+0000 7f82feffd640 1 --2- 172.21.15.143:0/976834882 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f83000ce680 0x7f83000ac8e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-26T07:17:47.291 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:17:47.287+0000 7f82ff7fe640 1 --2- 172.21.15.143:0/976834882 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f83000b2f80 0x7f83000c47e0 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7f82f000c3c0 tx=0x7f82f000c880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:17:47.291 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:17:47.287+0000 7f82e7fff640 1 -- 172.21.15.143:0/976834882 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f82f0005cf0 con 0x7f83000b2f80 2023-11-26T07:17:47.291 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:17:47.287+0000 7f82e7fff640 1 -- 172.21.15.143:0/976834882 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f82f0005e50 con 0x7f83000b2f80 2023-11-26T07:17:47.291 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:17:47.287+0000 7f82e7fff640 1 -- 172.21.15.143:0/976834882 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f82f00072f0 con 0x7f83000b2f80 2023-11-26T07:17:47.291 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:17:47.287+0000 7f83061c1640 1 -- 172.21.15.143:0/976834882 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f830014f040 con 0x7f83000b2f80 2023-11-26T07:17:47.292 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:17:47.287+0000 7f83061c1640 1 -- 172.21.15.143:0/976834882 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f830014f600 con 0x7f83000b2f80 2023-11-26T07:17:47.292 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:17:47.287+0000 7f83061c1640 1 -- 172.21.15.143:0/976834882 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f83000cb680 con 0x7f83000b2f80 2023-11-26T07:17:47.292 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:17:47.287+0000 7f82e7fff640 1 -- 172.21.15.143:0/976834882 <== mon.0 v2:172.21.15.143:3300/0 4 ==== mgrmap(e 5) v1 ==== 85852+0+0 (secure 0 0 0) 0x7f82f0007450 con 0x7f83000b2f80 2023-11-26T07:17:47.293 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:17:47.291+0000 7f82e7fff640 1 --2- 172.21.15.143:0/976834882 >> [v2:172.21.15.143:6832/2150557544,v1:172.21.15.143:6833/2150557544] conn(0x7f82ec066470 0x7f82ec068930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:17:47.293 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:17:47.291+0000 7f82e7fff640 1 -- 172.21.15.143:0/976834882 <== mon.0 v2:172.21.15.143:3300/0 5 ==== osd_map(16..16 src has 1..16) v4 ==== 4857+0+0 (secure 0 0 0) 0x7f82f0083190 con 0x7f83000b2f80 2023-11-26T07:17:47.293 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:17:47.291+0000 7f82feffd640 1 --2- 172.21.15.143:0/976834882 >> [v2:172.21.15.143:6832/2150557544,v1:172.21.15.143:6833/2150557544] conn(0x7f82ec066470 0x7f82ec068930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:17:47.293 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:17:47.291+0000 7f82feffd640 1 --2- 172.21.15.143:0/976834882 >> [v2:172.21.15.143:6832/2150557544,v1:172.21.15.143:6833/2150557544] conn(0x7f82ec066470 0x7f82ec068930 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7f83000c4560 tx=0x7f82f4009040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:17:47.297 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:17:47.295+0000 7f82e7fff640 1 -- 172.21.15.143:0/976834882 <== mon.0 v2:172.21.15.143:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172649 (secure 0 0 0) 0x7f82f00508c0 con 0x7f83000b2f80 2023-11-26T07:17:47.448 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:17:47.443+0000 7f83061c1640 1 -- 172.21.15.143:0/976834882 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f8300067680 con 0x7f83000b2f80 2023-11-26T07:17:47.449 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:17:47.447+0000 7f82e7fff640 1 -- 172.21.15.143:0/976834882 <== mon.0 v2:172.21.15.143:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v16) v1 ==== 74+0+12886 (secure 0 0 0) 0x7f82f0017090 con 0x7f83000b2f80 2023-11-26T07:17:47.449 INFO:teuthology.orchestra.run.smithi143.stdout: 2023-11-26T07:17:47.449 INFO:teuthology.orchestra.run.smithi143.stdout:{"epoch":16,"fsid":"f5033060-2469-4a07-95fe-2b76f5404bc1","created":"2023-11-26T07:17:28.613663+0000","modified":"2023-11-26T07:17:44.413749+0000","last_up_change":"2023-11-26T07:17:41.157813+0000","last_in_change":"2023-11-26T07:17:31.181658+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":1,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-11-26T07:17:41.758597+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"ebbee2c6-023e-44b9-9f1d-6f7bfd309730","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6817","nonce":639807369},{"type":"v1","addr":"172.21.15.143:6819","nonce":639807369}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6821","nonce":639807369},{"type":"v1","addr":"172.21.15.143:6823","nonce":639807369}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6830","nonce":639807369},{"type":"v1","addr":"172.21.15.143:6831","nonce":639807369}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6825","nonce":639807369},{"type":"v1","addr":"172.21.15.143:6828","nonce":639807369}]},"public_addr":"172.21.15.143:6819/639807369","cluster_addr":"172.21.15.143:6823/639807369","heartbeat_back_addr":"172.21.15.143:6831/639807369","heartbeat_front_addr":"172.21.15.143:6828/639807369","state":["exists","up"]},{"osd":1,"uuid":"fec4b8be-7122-49e9-9a19-5850a0700001","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6816","nonce":1144942530},{"type":"v1","addr":"172.21.15.143:6818","nonce":1144942530}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6820","nonce":1144942530},{"type":"v1","addr":"172.21.15.143:6822","nonce":1144942530}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6827","nonce":1144942530},{"type":"v1","addr":"172.21.15.143:6829","nonce":1144942530}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6824","nonce":1144942530},{"type":"v1","addr":"172.21.15.143:6826","nonce":1144942530}]},"public_addr":"172.21.15.143:6818/1144942530","cluster_addr":"172.21.15.143:6822/1144942530","heartbeat_back_addr":"172.21.15.143:6829/1144942530","heartbeat_front_addr":"172.21.15.143:6826/1144942530","state":["exists","up"]},{"osd":2,"uuid":"2888af5b-f933-46ca-8f95-6dd494ff43f6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6808","nonce":3909095186},{"type":"v1","addr":"172.21.15.143:6809","nonce":3909095186}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6810","nonce":3909095186},{"type":"v1","addr":"172.21.15.143:6811","nonce":3909095186}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6814","nonce":3909095186},{"type":"v1","addr":"172.21.15.143:6815","nonce":3909095186}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6812","nonce":3909095186},{"type":"v1","addr":"172.21.15.143:6813","nonce":3909095186}]},"public_addr":"172.21.15.143:6809/3909095186","cluster_addr":"172.21.15.143:6811/3909095186","heartbeat_back_addr":"172.21.15.143:6815/3909095186","heartbeat_front_addr":"172.21.15.143:6813/3909095186","state":["exists","up"]},{"osd":3,"uuid":"d1f52435-7bc1-4081-9550-9cd1ba55f549","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6800","nonce":3854210991},{"type":"v1","addr":"172.21.15.143:6801","nonce":3854210991}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6802","nonce":3854210991},{"type":"v1","addr":"172.21.15.143:6803","nonce":3854210991}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6806","nonce":3854210991},{"type":"v1","addr":"172.21.15.143:6807","nonce":3854210991}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6804","nonce":3854210991},{"type":"v1","addr":"172.21.15.143:6805","nonce":3854210991}]},"public_addr":"172.21.15.143:6801/3854210991","cluster_addr":"172.21.15.143:6803/3854210991","heartbeat_back_addr":"172.21.15.143:6807/3854210991","heartbeat_front_addr":"172.21.15.143:6805/3854210991","state":["exists","up"]},{"osd":4,"uuid":"e2ae8b6e-6e25-49b2-bcbf-aacce68f06a3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6800","nonce":3924606902},{"type":"v1","addr":"172.21.15.188:6801","nonce":3924606902}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6802","nonce":3924606902},{"type":"v1","addr":"172.21.15.188:6803","nonce":3924606902}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6806","nonce":3924606902},{"type":"v1","addr":"172.21.15.188:6807","nonce":3924606902}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6804","nonce":3924606902},{"type":"v1","addr":"172.21.15.188:6805","nonce":3924606902}]},"public_addr":"172.21.15.188:6801/3924606902","cluster_addr":"172.21.15.188:6803/3924606902","heartbeat_back_addr":"172.21.15.188:6807/3924606902","heartbeat_front_addr":"172.21.15.188:6805/3924606902","state":["exists","up"]},{"osd":5,"uuid":"3cec8a32-7698-4c36-8e89-e4ba11934f33","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6813","nonce":186841544},{"type":"v1","addr":"172.21.15.188:6815","nonce":186841544}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6817","nonce":186841544},{"type":"v1","addr":"172.21.15.188:6818","nonce":186841544}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6822","nonce":186841544},{"type":"v1","addr":"172.21.15.188:6823","nonce":186841544}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6820","nonce":186841544},{"type":"v1","addr":"172.21.15.188:6821","nonce":186841544}]},"public_addr":"172.21.15.188:6815/186841544","cluster_addr":"172.21.15.188:6818/186841544","heartbeat_back_addr":"172.21.15.188:6823/186841544","heartbeat_front_addr":"172.21.15.188:6821/186841544","state":["exists","up"]},{"osd":6,"uuid":"58a65d5d-e090-473a-81b4-ad38e85029f9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6808","nonce":2153499405},{"type":"v1","addr":"172.21.15.188:6809","nonce":2153499405}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6810","nonce":2153499405},{"type":"v1","addr":"172.21.15.188:6811","nonce":2153499405}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6816","nonce":2153499405},{"type":"v1","addr":"172.21.15.188:6819","nonce":2153499405}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6812","nonce":2153499405},{"type":"v1","addr":"172.21.15.188:6814","nonce":2153499405}]},"public_addr":"172.21.15.188:6809/2153499405","cluster_addr":"172.21.15.188:6811/2153499405","heartbeat_back_addr":"172.21.15.188:6819/2153499405","heartbeat_front_addr":"172.21.15.188:6814/2153499405","state":["exists","up"]},{"osd":7,"uuid":"ea5b7f70-3554-4cf8-ac30-f7c59b80678d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":14,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6824","nonce":2722096729},{"type":"v1","addr":"172.21.15.188:6825","nonce":2722096729}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6826","nonce":2722096729},{"type":"v1","addr":"172.21.15.188:6827","nonce":2722096729}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6830","nonce":2722096729},{"type":"v1","addr":"172.21.15.188:6831","nonce":2722096729}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6828","nonce":2722096729},{"type":"v1","addr":"172.21.15.188:6829","nonce":2722096729}]},"public_addr":"172.21.15.188:6825/2722096729","cluster_addr":"172.21.15.188:6827/2722096729","heartbeat_back_addr":"172.21.15.188:6831/2722096729","heartbeat_front_addr":"172.21.15.188:6829/2722096729","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-26T07:17:36.995032+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-26T07:17:36.932266+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-26T07:17:36.829304+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-26T07:17:36.804297+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-26T07:17:36.667892+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-26T07:17:36.964868+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-26T07:17:36.827332+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-26T07:17:36.893179+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-11-26T07:17:47.451 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:17:47.447+0000 7f83061c1640 1 -- 172.21.15.143:0/976834882 >> [v2:172.21.15.143:6832/2150557544,v1:172.21.15.143:6833/2150557544] conn(0x7f82ec066470 msgr2=0x7f82ec068930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:17:47.451 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:17:47.447+0000 7f83061c1640 1 --2- 172.21.15.143:0/976834882 >> [v2:172.21.15.143:6832/2150557544,v1:172.21.15.143:6833/2150557544] conn(0x7f82ec066470 0x7f82ec068930 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7f83000c4560 tx=0x7f82f4009040 comp rx=0 tx=0).stop 2023-11-26T07:17:47.451 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:17:47.447+0000 7f83061c1640 1 -- 172.21.15.143:0/976834882 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f83000b2f80 msgr2=0x7f83000c47e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:17:47.451 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:17:47.447+0000 7f83061c1640 1 --2- 172.21.15.143:0/976834882 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f83000b2f80 0x7f83000c47e0 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7f82f000c3c0 tx=0x7f82f000c880 comp rx=0 tx=0).stop 2023-11-26T07:17:47.452 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:17:47.447+0000 7f83061c1640 1 -- 172.21.15.143:0/976834882 shutdown_connections 2023-11-26T07:17:47.452 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:17:47.447+0000 7f83061c1640 1 --2- 172.21.15.143:0/976834882 >> [v2:172.21.15.143:6832/2150557544,v1:172.21.15.143:6833/2150557544] conn(0x7f82ec066470 0x7f82ec068930 unknown :-1 s=CLOSED pgs=53 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:17:47.452 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:17:47.447+0000 7f83061c1640 1 --2- 172.21.15.143:0/976834882 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f83001596d0 0x7f83000ace20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:17:47.452 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:17:47.447+0000 7f83061c1640 1 --2- 172.21.15.143:0/976834882 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f83000ce680 0x7f83000ac8e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:17:47.452 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:17:47.447+0000 7f83061c1640 1 --2- 172.21.15.143:0/976834882 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f83000b2f80 0x7f83000c47e0 unknown :-1 s=CLOSED pgs=48 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:17:47.452 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:17:47.447+0000 7f83061c1640 1 -- 172.21.15.143:0/976834882 >> 172.21.15.143:0/976834882 conn(0x7f830006a890 msgr2=0x7f83000715d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:17:47.452 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:17:47.447+0000 7f83061c1640 1 -- 172.21.15.143:0/976834882 shutdown_connections 2023-11-26T07:17:47.452 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:17:47.447+0000 7f83061c1640 1 -- 172.21.15.143:0/976834882 wait complete. 2023-11-26T07:17:47.471 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-11-26T07:17:41.758597+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2023-11-26T07:17:47.472 DEBUG:teuthology.orchestra.run.smithi143:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-11-26T07:17:47.612 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:17:47.607+0000 7fcff76ea640 1 Processor -- start 2023-11-26T07:17:47.612 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:17:47.611+0000 7fcff76ea640 1 -- start start 2023-11-26T07:17:47.614 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:17:47.611+0000 7fcff76ea640 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7fcff0143ac0 0x7fcff0070370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:17:47.614 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:17:47.611+0000 7fcff76ea640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fcff00708b0 0x7fcff00cf280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:17:47.614 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:17:47.611+0000 7fcff76ea640 1 -- --> v1:172.21.15.188:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fcff00cfbf0 con 0x7fcff00cf7c0 2023-11-26T07:17:47.614 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:17:47.611+0000 7fcff76ea640 1 -- --> v2:172.21.15.143:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fcff0143ea0 con 0x7fcff0143ac0 2023-11-26T07:17:47.614 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:17:47.611+0000 7fcff76ea640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcff00cc730 con 0x7fcff00708b0 2023-11-26T07:17:47.615 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:17:47.611+0000 7fcff5486640 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7fcff0143ac0 0x7fcff0070370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:17:47.615 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:17:47.611+0000 7fcff5486640 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7fcff0143ac0 0x7fcff0070370 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:52994/0 (socket says 172.21.15.143:52994) 2023-11-26T07:17:47.615 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:17:47.611+0000 7fcff5486640 1 -- 172.21.15.143:0/4151228173 learned_addr learned my addr 172.21.15.143:0/4151228173 (peer_addr_for_me v2:172.21.15.143:0/0) 2023-11-26T07:17:47.615 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:17:47.611+0000 7fcff4c85640 1 --2- 172.21.15.143:0/4151228173 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fcff00708b0 0x7fcff00cf280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:17:47.615 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:17:47.611+0000 7fcff5486640 1 -- 172.21.15.143:0/4151228173 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fcff00708b0 msgr2=0x7fcff00cf280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-26T07:17:47.615 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:17:47.611+0000 7fcff5486640 1 --2- 172.21.15.143:0/4151228173 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fcff00708b0 0x7fcff00cf280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:17:47.615 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:17:47.611+0000 7fcff5486640 1 -- 172.21.15.143:0/4151228173 >> v1:172.21.15.188:6789/0 conn(0x7fcff00cf7c0 legacy=0x7fcff00b1ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:17:47.615 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:17:47.611+0000 7fcff5486640 1 -- 172.21.15.143:0/4151228173 --> v2:172.21.15.143:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcff0070ee0 con 0x7fcff0143ac0 2023-11-26T07:17:47.617 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:17:47.615+0000 7fcff5486640 1 --2- 172.21.15.143:0/4151228173 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fcff0143ac0 0x7fcff0070370 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7fcfec00bb10 tx=0x7fcfec01ec10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=89d84fd289622349 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:17:47.617 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:17:47.615+0000 7fcfe7fff640 1 -- 172.21.15.143:0/4151228173 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcfec0a7bf0 con 0x7fcff0143ac0 2023-11-26T07:17:47.617 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:17:47.615+0000 7fcfe7fff640 1 -- 172.21.15.143:0/4151228173 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcfec0a7d50 con 0x7fcff0143ac0 2023-11-26T07:17:47.617 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:17:47.615+0000 7fcfe7fff640 1 -- 172.21.15.143:0/4151228173 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcfec0af660 con 0x7fcff0143ac0 2023-11-26T07:17:47.618 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:17:47.615+0000 7fcff76ea640 1 -- 172.21.15.143:0/4151228173 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fcff0143ac0 msgr2=0x7fcff0070370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:17:47.618 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:17:47.615+0000 7fcff76ea640 1 --2- 172.21.15.143:0/4151228173 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fcff0143ac0 0x7fcff0070370 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7fcfec00bb10 tx=0x7fcfec01ec10 comp rx=0 tx=0).stop 2023-11-26T07:17:47.618 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:17:47.615+0000 7fcff76ea640 1 -- 172.21.15.143:0/4151228173 shutdown_connections 2023-11-26T07:17:47.618 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:17:47.615+0000 7fcff76ea640 1 --2- 172.21.15.143:0/4151228173 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fcff00708b0 0x7fcff00cf280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:17:47.619 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:17:47.615+0000 7fcff76ea640 1 --2- 172.21.15.143:0/4151228173 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fcff0143ac0 0x7fcff0070370 unknown :-1 s=CLOSED pgs=49 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:17:47.619 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:17:47.615+0000 7fcff76ea640 1 -- 172.21.15.143:0/4151228173 >> 172.21.15.143:0/4151228173 conn(0x7fcff006a890 msgr2=0x7fcff006aca0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:17:47.619 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:17:47.615+0000 7fcff76ea640 1 -- 172.21.15.143:0/4151228173 shutdown_connections 2023-11-26T07:17:47.619 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:17:47.615+0000 7fcff76ea640 1 -- 172.21.15.143:0/4151228173 wait complete. 2023-11-26T07:17:47.620 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:17:47.615+0000 7fcff76ea640 1 Processor -- start 2023-11-26T07:17:47.620 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:17:47.615+0000 7fcff76ea640 1 -- start start 2023-11-26T07:17:47.622 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:17:47.619+0000 7fcff76ea640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fcff00708b0 0x7fcff015efb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:17:47.622 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:17:47.619+0000 7fcff76ea640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fcff00cf7c0 0x7fcff00b0590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:17:47.622 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:17:47.619+0000 7fcff76ea640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fcff0143ac0 0x7fcff013e970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:17:47.622 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:17:47.619+0000 7fcff76ea640 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcff00b0ad0 con 0x7fcff00cf7c0 2023-11-26T07:17:47.623 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:17:47.619+0000 7fcff76ea640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcff015f670 con 0x7fcff00708b0 2023-11-26T07:17:47.623 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:17:47.619+0000 7fcff76ea640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcff015f950 con 0x7fcff0143ac0 2023-11-26T07:17:47.623 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:17:47.619+0000 7fcff4c85640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fcff00cf7c0 0x7fcff00b0590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:17:47.623 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:17:47.619+0000 7fcff4c85640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fcff00cf7c0 0x7fcff00b0590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:53010/0 (socket says 172.21.15.143:53010) 2023-11-26T07:17:47.623 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:17:47.619+0000 7fcff4c85640 1 -- 172.21.15.143:0/1299802963 learned_addr learned my addr 172.21.15.143:0/1299802963 (peer_addr_for_me v2:172.21.15.143:0/0) 2023-11-26T07:17:47.623 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:17:47.619+0000 7fcff5486640 1 --2- 172.21.15.143:0/1299802963 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fcff00708b0 0x7fcff015efb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:17:47.623 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:17:47.619+0000 7fcff5c87640 1 --2- 172.21.15.143:0/1299802963 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fcff0143ac0 0x7fcff013e970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:17:47.623 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:17:47.619+0000 7fcff4c85640 1 -- 172.21.15.143:0/1299802963 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fcff0143ac0 msgr2=0x7fcff013e970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:17:47.623 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:17:47.619+0000 7fcff4c85640 1 --2- 172.21.15.143:0/1299802963 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fcff0143ac0 0x7fcff013e970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:17:47.623 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:17:47.619+0000 7fcff4c85640 1 -- 172.21.15.143:0/1299802963 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fcff00708b0 msgr2=0x7fcff015efb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:17:47.623 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:17:47.619+0000 7fcff4c85640 1 --2- 172.21.15.143:0/1299802963 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fcff00708b0 0x7fcff015efb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:17:47.624 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:17:47.619+0000 7fcff4c85640 1 -- 172.21.15.143:0/1299802963 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcfec00b770 con 0x7fcff00cf7c0 2023-11-26T07:17:47.624 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:17:47.619+0000 7fcff4c85640 1 --2- 172.21.15.143:0/1299802963 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fcff00cf7c0 0x7fcff00b0590 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7fcfe000e930 tx=0x7fcfe000edf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:17:47.624 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:17:47.619+0000 7fcfe5ffb640 1 -- 172.21.15.143:0/1299802963 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcfe0009690 con 0x7fcff00cf7c0 2023-11-26T07:17:47.624 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:17:47.619+0000 7fcfe5ffb640 1 -- 172.21.15.143:0/1299802963 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcfe000ca00 con 0x7fcff00cf7c0 2023-11-26T07:17:47.624 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:17:47.619+0000 7fcff76ea640 1 -- 172.21.15.143:0/1299802963 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcff015fc30 con 0x7fcff00cf7c0 2023-11-26T07:17:47.624 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:17:47.619+0000 7fcfe5ffb640 1 -- 172.21.15.143:0/1299802963 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcfe000cd70 con 0x7fcff00cf7c0 2023-11-26T07:17:47.624 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:17:47.619+0000 7fcff76ea640 1 -- 172.21.15.143:0/1299802963 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcff01601f0 con 0x7fcff00cf7c0 2023-11-26T07:17:47.625 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:17:47.619+0000 7fcff76ea640 1 -- 172.21.15.143:0/1299802963 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fcff0067680 con 0x7fcff00cf7c0 2023-11-26T07:17:47.625 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:17:47.623+0000 7fcfe5ffb640 1 -- 172.21.15.143:0/1299802963 <== mon.0 v2:172.21.15.143:3300/0 4 ==== mgrmap(e 5) v1 ==== 85852+0+0 (secure 0 0 0) 0x7fcfe0013470 con 0x7fcff00cf7c0 2023-11-26T07:17:47.625 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:17:47.623+0000 7fcfe5ffb640 1 --2- 172.21.15.143:0/1299802963 >> [v2:172.21.15.143:6832/2150557544,v1:172.21.15.143:6833/2150557544] conn(0x7fcfd4066470 0x7fcfd4068930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:17:47.625 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:17:47.623+0000 7fcfe5ffb640 1 -- 172.21.15.143:0/1299802963 <== mon.0 v2:172.21.15.143:3300/0 5 ==== osd_map(16..16 src has 1..16) v4 ==== 4857+0+0 (secure 0 0 0) 0x7fcfe0083de0 con 0x7fcff00cf7c0 2023-11-26T07:17:47.625 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:17:47.623+0000 7fcff5486640 1 --2- 172.21.15.143:0/1299802963 >> [v2:172.21.15.143:6832/2150557544,v1:172.21.15.143:6833/2150557544] conn(0x7fcfd4066470 0x7fcfd4068930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:17:47.626 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:17:47.623+0000 7fcff5486640 1 --2- 172.21.15.143:0/1299802963 >> [v2:172.21.15.143:6832/2150557544,v1:172.21.15.143:6833/2150557544] conn(0x7fcfd4066470 0x7fcfd4068930 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7fcfec0b1000 tx=0x7fcfec009040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:17:47.629 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:17:47.627+0000 7fcfe5ffb640 1 -- 172.21.15.143:0/1299802963 <== mon.0 v2:172.21.15.143:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172649 (secure 0 0 0) 0x7fcfe0051680 con 0x7fcff00cf7c0 2023-11-26T07:17:47.781 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:17:47.779+0000 7fcff76ea640 1 -- 172.21.15.143:0/1299802963 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7fcff00cb730 con 0x7fcff00cf7c0 2023-11-26T07:17:47.781 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:17:47.779+0000 7fcfe5ffb640 1 -- 172.21.15.143:0/1299802963 <== mon.0 v2:172.21.15.143:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v16) v1 ==== 93+0+10 (secure 0 0 0) 0x7fcfe00170a0 con 0x7fcff00cf7c0 2023-11-26T07:17:47.781 INFO:teuthology.orchestra.run.smithi143.stdout:pg_num: 1 2023-11-26T07:17:47.783 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:17:47.779+0000 7fcff76ea640 1 -- 172.21.15.143:0/1299802963 >> [v2:172.21.15.143:6832/2150557544,v1:172.21.15.143:6833/2150557544] conn(0x7fcfd4066470 msgr2=0x7fcfd4068930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:17:47.784 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:17:47.779+0000 7fcff76ea640 1 --2- 172.21.15.143:0/1299802963 >> [v2:172.21.15.143:6832/2150557544,v1:172.21.15.143:6833/2150557544] conn(0x7fcfd4066470 0x7fcfd4068930 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7fcfec0b1000 tx=0x7fcfec009040 comp rx=0 tx=0).stop 2023-11-26T07:17:47.784 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:17:47.779+0000 7fcff76ea640 1 -- 172.21.15.143:0/1299802963 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fcff00cf7c0 msgr2=0x7fcff00b0590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:17:47.784 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:17:47.779+0000 7fcff76ea640 1 --2- 172.21.15.143:0/1299802963 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fcff00cf7c0 0x7fcff00b0590 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7fcfe000e930 tx=0x7fcfe000edf0 comp rx=0 tx=0).stop 2023-11-26T07:17:47.784 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:17:47.779+0000 7fcff76ea640 1 -- 172.21.15.143:0/1299802963 shutdown_connections 2023-11-26T07:17:47.784 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:17:47.779+0000 7fcff76ea640 1 --2- 172.21.15.143:0/1299802963 >> [v2:172.21.15.143:6832/2150557544,v1:172.21.15.143:6833/2150557544] conn(0x7fcfd4066470 0x7fcfd4068930 unknown :-1 s=CLOSED pgs=54 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:17:47.784 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:17:47.779+0000 7fcff76ea640 1 --2- 172.21.15.143:0/1299802963 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fcff0143ac0 0x7fcff013e970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:17:47.784 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:17:47.779+0000 7fcff76ea640 1 --2- 172.21.15.143:0/1299802963 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fcff00cf7c0 0x7fcff00b0590 unknown :-1 s=CLOSED pgs=50 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:17:47.784 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:17:47.779+0000 7fcff76ea640 1 --2- 172.21.15.143:0/1299802963 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fcff00708b0 0x7fcff015efb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:17:47.784 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:17:47.779+0000 7fcff76ea640 1 -- 172.21.15.143:0/1299802963 >> 172.21.15.143:0/1299802963 conn(0x7fcff006a890 msgr2=0x7fcff00b1cb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:17:47.784 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:17:47.779+0000 7fcff76ea640 1 -- 172.21.15.143:0/1299802963 shutdown_connections 2023-11-26T07:17:47.784 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:17:47.779+0000 7fcff76ea640 1 -- 172.21.15.143:0/1299802963 wait complete. 2023-11-26T07:17:47.802 DEBUG:teuthology.orchestra.run.smithi143:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-11-26T07:17:47.939 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:17:47.935+0000 7fca4c3d1640 1 Processor -- start 2023-11-26T07:17:47.939 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:17:47.935+0000 7fca4c3d1640 1 -- start start 2023-11-26T07:17:47.941 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:17:47.939+0000 7fca4c3d1640 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7fca44071850 0x7fca441596d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:17:47.941 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:17:47.939+0000 7fca4c3d1640 1 -- --> v1:172.21.15.143:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fca4406c750 con 0x7fca440ce680 2023-11-26T07:17:47.941 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:17:47.939+0000 7fca4c3d1640 1 -- --> v1:172.21.15.188:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fca440b3550 con 0x7fca440b2870 2023-11-26T07:17:47.941 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:17:47.939+0000 7fca4c3d1640 1 -- --> v2:172.21.15.188:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fca440cbe00 con 0x7fca44071850 2023-11-26T07:17:47.942 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:17:47.939+0000 7fca4a16d640 1 --1- >> v1:172.21.15.143:6789/0 conn(0x7fca440ce680 0x7fca440cf280 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.143:6789/0 says I am v1:172.21.15.143:57450/0 (socket says 172.21.15.143:57450) 2023-11-26T07:17:47.942 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:17:47.939+0000 7fca4a16d640 1 -- 172.21.15.143:0/4145280154 learned_addr learned my addr 172.21.15.143:0/4145280154 (peer_addr_for_me v1:172.21.15.143:0/0) 2023-11-26T07:17:47.942 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:17:47.939+0000 7fca4996c640 1 --2- 172.21.15.143:0/4145280154 >> v2:172.21.15.188:3300/0 conn(0x7fca44071850 0x7fca441596d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:17:47.943 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:17:47.939+0000 7fca4916b640 1 -- 172.21.15.143:0/4145280154 <== mon.0 v1:172.21.15.143:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 321132656 0 0) 0x7fca4406c750 con 0x7fca440ce680 2023-11-26T07:17:47.943 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:17:47.939+0000 7fca4916b640 1 -- 172.21.15.143:0/4145280154 --> v1:172.21.15.143:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fca30005740 con 0x7fca440ce680 2023-11-26T07:17:47.943 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:17:47.939+0000 7fca4996c640 1 -- 172.21.15.143:0/4145280154 >> v1:172.21.15.188:6789/0 conn(0x7fca440b2870 legacy=0x7fca44164650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:17:47.943 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:17:47.939+0000 7fca4916b640 1 -- 172.21.15.143:0/4145280154 <== mon.1 v1:172.21.15.188:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4021598759 0 0) 0x7fca440b3550 con 0x7fca440b2870 2023-11-26T07:17:47.943 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:17:47.939+0000 7fca4996c640 1 -- 172.21.15.143:0/4145280154 >> v1:172.21.15.143:6789/0 conn(0x7fca440ce680 legacy=0x7fca440cf280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:17:47.943 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:17:47.939+0000 7fca4996c640 1 -- 172.21.15.143:0/4145280154 --> v2:172.21.15.188:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fca44071e80 con 0x7fca44071850 2023-11-26T07:17:47.945 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:17:47.943+0000 7fca4996c640 1 --2- 172.21.15.143:0/4145280154 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fca44071850 0x7fca441596d0 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7fca340097c0 tx=0x7fca3401c2c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=a06aae62f61581b5 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:17:47.945 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:17:47.943+0000 7fca4916b640 1 -- 172.21.15.143:0/4145280154 <== mon.1 v2:172.21.15.188:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fca40001680 con 0x7fca44071850 2023-11-26T07:17:47.945 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:17:47.943+0000 7fca4916b640 1 -- 172.21.15.143:0/4145280154 <== mon.1 v2:172.21.15.188:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fca340a7050 con 0x7fca44071850 2023-11-26T07:17:47.945 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:17:47.943+0000 7fca4916b640 1 -- 172.21.15.143:0/4145280154 <== mon.1 v2:172.21.15.188:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fca340ab620 con 0x7fca44071850 2023-11-26T07:17:47.946 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:17:47.943+0000 7fca4c3d1640 1 -- 172.21.15.143:0/4145280154 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fca44071850 msgr2=0x7fca441596d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:17:47.946 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:17:47.943+0000 7fca4c3d1640 1 --2- 172.21.15.143:0/4145280154 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fca44071850 0x7fca441596d0 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7fca340097c0 tx=0x7fca3401c2c0 comp rx=0 tx=0).stop 2023-11-26T07:17:47.947 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:17:47.943+0000 7fca4c3d1640 1 -- 172.21.15.143:0/4145280154 shutdown_connections 2023-11-26T07:17:47.947 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:17:47.943+0000 7fca4c3d1640 1 --2- 172.21.15.143:0/4145280154 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fca44071850 0x7fca441596d0 unknown :-1 s=CLOSED pgs=42 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:17:47.947 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:17:47.943+0000 7fca4c3d1640 1 -- 172.21.15.143:0/4145280154 >> 172.21.15.143:0/4145280154 conn(0x7fca4406a890 msgr2=0x7fca4406aca0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:17:47.947 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:17:47.943+0000 7fca4c3d1640 1 -- 172.21.15.143:0/4145280154 shutdown_connections 2023-11-26T07:17:47.947 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:17:47.943+0000 7fca4c3d1640 1 -- 172.21.15.143:0/4145280154 wait complete. 2023-11-26T07:17:47.948 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:17:47.943+0000 7fca4c3d1640 1 Processor -- start 2023-11-26T07:17:47.948 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:17:47.943+0000 7fca4c3d1640 1 -- start start 2023-11-26T07:17:47.950 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:17:47.947+0000 7fca4c3d1640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fca44071850 0x7fca440ab070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:17:47.950 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:17:47.947+0000 7fca4c3d1640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fca440b2870 0x7fca4413b500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:17:47.950 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:17:47.947+0000 7fca4c3d1640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fca440ce680 0x7fca440af300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:17:47.951 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:17:47.947+0000 7fca4c3d1640 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fca441394b0 con 0x7fca44071850 2023-11-26T07:17:47.951 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:17:47.947+0000 7fca4c3d1640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fca4413bd20 con 0x7fca440ce680 2023-11-26T07:17:47.951 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:17:47.947+0000 7fca4c3d1640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fca4413c000 con 0x7fca440b2870 2023-11-26T07:17:47.951 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:17:47.947+0000 7fca4a16d640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fca44071850 0x7fca440ab070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:17:47.951 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:17:47.947+0000 7fca4a16d640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fca44071850 0x7fca440ab070 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:53016/0 (socket says 172.21.15.143:53016) 2023-11-26T07:17:47.951 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:17:47.947+0000 7fca4a16d640 1 -- 172.21.15.143:0/134453305 learned_addr learned my addr 172.21.15.143:0/134453305 (peer_addr_for_me v2:172.21.15.143:0/0) 2023-11-26T07:17:47.951 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:17:47.947+0000 7fca4996c640 1 --2- 172.21.15.143:0/134453305 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fca440b2870 0x7fca4413b500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:17:47.951 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:17:47.947+0000 7fca4a96e640 1 --2- 172.21.15.143:0/134453305 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fca440ce680 0x7fca440af300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:17:47.951 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:17:47.947+0000 7fca4a16d640 1 -- 172.21.15.143:0/134453305 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fca440b2870 msgr2=0x7fca4413b500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:17:47.951 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:17:47.947+0000 7fca4a16d640 1 --2- 172.21.15.143:0/134453305 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fca440b2870 0x7fca4413b500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:17:47.951 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:17:47.947+0000 7fca4a16d640 1 -- 172.21.15.143:0/134453305 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fca440ce680 msgr2=0x7fca440af300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:17:47.952 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:17:47.947+0000 7fca4a16d640 1 --2- 172.21.15.143:0/134453305 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fca440ce680 0x7fca440af300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:17:47.952 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:17:47.947+0000 7fca4a16d640 1 -- 172.21.15.143:0/134453305 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fca34009420 con 0x7fca44071850 2023-11-26T07:17:47.952 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:17:47.947+0000 7fca4a96e640 1 --2- 172.21.15.143:0/134453305 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fca440ce680 0x7fca440af300 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-26T07:17:47.952 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:17:47.947+0000 7fca4a16d640 1 --2- 172.21.15.143:0/134453305 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fca44071850 0x7fca440ab070 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7fca40001ec0 tx=0x7fca4000cc00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:17:47.952 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:17:47.947+0000 7fca3affd640 1 -- 172.21.15.143:0/134453305 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fca400074e0 con 0x7fca44071850 2023-11-26T07:17:47.952 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:17:47.947+0000 7fca3affd640 1 -- 172.21.15.143:0/134453305 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fca40007640 con 0x7fca44071850 2023-11-26T07:17:47.952 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:17:47.947+0000 7fca3affd640 1 -- 172.21.15.143:0/134453305 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fca40004a10 con 0x7fca44071850 2023-11-26T07:17:47.952 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:17:47.947+0000 7fca4c3d1640 1 -- 172.21.15.143:0/134453305 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fca4413c2e0 con 0x7fca44071850 2023-11-26T07:17:47.952 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:17:47.947+0000 7fca4c3d1640 1 -- 172.21.15.143:0/134453305 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fca4413ba40 con 0x7fca44071850 2023-11-26T07:17:47.953 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:17:47.947+0000 7fca4c3d1640 1 -- 172.21.15.143:0/134453305 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fca44139900 con 0x7fca44071850 2023-11-26T07:17:47.953 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:17:47.951+0000 7fca3affd640 1 -- 172.21.15.143:0/134453305 <== mon.0 v2:172.21.15.143:3300/0 4 ==== mgrmap(e 5) v1 ==== 85852+0+0 (secure 0 0 0) 0x7fca40004b70 con 0x7fca44071850 2023-11-26T07:17:47.953 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:17:47.951+0000 7fca3affd640 1 --2- 172.21.15.143:0/134453305 >> [v2:172.21.15.143:6832/2150557544,v1:172.21.15.143:6833/2150557544] conn(0x7fca28066470 0x7fca28068930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:17:47.953 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:17:47.951+0000 7fca3affd640 1 -- 172.21.15.143:0/134453305 <== mon.0 v2:172.21.15.143:3300/0 5 ==== osd_map(16..16 src has 1..16) v4 ==== 4857+0+0 (secure 0 0 0) 0x7fca400831d0 con 0x7fca44071850 2023-11-26T07:17:47.953 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:17:47.951+0000 7fca4996c640 1 --2- 172.21.15.143:0/134453305 >> [v2:172.21.15.143:6832/2150557544,v1:172.21.15.143:6833/2150557544] conn(0x7fca28066470 0x7fca28068930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:17:47.954 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:17:47.951+0000 7fca4996c640 1 --2- 172.21.15.143:0/134453305 >> [v2:172.21.15.143:6832/2150557544,v1:172.21.15.143:6833/2150557544] conn(0x7fca28066470 0x7fca28068930 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7fca340ad000 tx=0x7fca340a6000 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:17:47.957 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:17:47.955+0000 7fca3affd640 1 -- 172.21.15.143:0/134453305 <== mon.0 v2:172.21.15.143:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172649 (secure 0 0 0) 0x7fca44139900 con 0x7fca44071850 2023-11-26T07:17:48.137 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:17:48.135+0000 7fca4c3d1640 1 -- 172.21.15.143:0/134453305 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fca4413c470 con 0x7fca44071850 2023-11-26T07:17:48.137 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:17:48.135+0000 7fca3affd640 1 -- 172.21.15.143:0/134453305 <== mon.0 v2:172.21.15.143:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 2 v2) v1 ==== 93+0+1327 (secure 0 0 0) 0x7fca40018020 con 0x7fca44071850 2023-11-26T07:17:48.138 INFO:teuthology.orchestra.run.smithi143.stdout: 2023-11-26T07:17:48.138 INFO:teuthology.orchestra.run.smithi143.stdout:{"epoch":2,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4266,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.143:6835/3850184704","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6834","nonce":3850184704},{"type":"v1","addr":"172.21.15.143:6835","nonce":3850184704}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":2}],"filesystems":[]} 2023-11-26T07:17:48.138 INFO:teuthology.orchestra.run.smithi143.stderr:dumped fsmap epoch 2 2023-11-26T07:17:48.139 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:17:48.135+0000 7fca4c3d1640 1 -- 172.21.15.143:0/134453305 >> [v2:172.21.15.143:6832/2150557544,v1:172.21.15.143:6833/2150557544] conn(0x7fca28066470 msgr2=0x7fca28068930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:17:48.139 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:17:48.135+0000 7fca4c3d1640 1 --2- 172.21.15.143:0/134453305 >> [v2:172.21.15.143:6832/2150557544,v1:172.21.15.143:6833/2150557544] conn(0x7fca28066470 0x7fca28068930 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7fca340ad000 tx=0x7fca340a6000 comp rx=0 tx=0).stop 2023-11-26T07:17:48.140 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:17:48.135+0000 7fca4c3d1640 1 -- 172.21.15.143:0/134453305 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fca44071850 msgr2=0x7fca440ab070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:17:48.140 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:17:48.135+0000 7fca4c3d1640 1 --2- 172.21.15.143:0/134453305 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fca44071850 0x7fca440ab070 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7fca40001ec0 tx=0x7fca4000cc00 comp rx=0 tx=0).stop 2023-11-26T07:17:48.140 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:17:48.135+0000 7fca4c3d1640 1 -- 172.21.15.143:0/134453305 shutdown_connections 2023-11-26T07:17:48.140 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:17:48.135+0000 7fca4c3d1640 1 --2- 172.21.15.143:0/134453305 >> [v2:172.21.15.143:6832/2150557544,v1:172.21.15.143:6833/2150557544] conn(0x7fca28066470 0x7fca28068930 unknown :-1 s=CLOSED pgs=55 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:17:48.140 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:17:48.135+0000 7fca4c3d1640 1 --2- 172.21.15.143:0/134453305 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fca440ce680 0x7fca440af300 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:17:48.140 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:17:48.135+0000 7fca4c3d1640 1 --2- 172.21.15.143:0/134453305 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fca440b2870 0x7fca4413b500 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:17:48.140 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:17:48.135+0000 7fca4c3d1640 1 --2- 172.21.15.143:0/134453305 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fca44071850 0x7fca440ab070 unknown :-1 s=CLOSED pgs=52 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:17:48.140 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:17:48.135+0000 7fca4c3d1640 1 -- 172.21.15.143:0/134453305 >> 172.21.15.143:0/134453305 conn(0x7fca4406a890 msgr2=0x7fca4415dc70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:17:48.140 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:17:48.135+0000 7fca4c3d1640 1 -- 172.21.15.143:0/134453305 shutdown_connections 2023-11-26T07:17:48.140 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:17:48.135+0000 7fca4c3d1640 1 -- 172.21.15.143:0/134453305 wait complete. 2023-11-26T07:17:50.158 DEBUG:teuthology.orchestra.run.smithi143:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-11-26T07:17:50.297 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:17:50.295+0000 7f504dee3640 1 Processor -- start 2023-11-26T07:17:50.298 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:17:50.295+0000 7f504dee3640 1 -- start start 2023-11-26T07:17:50.299 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:17:50.295+0000 7f504dee3640 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7f50480d1bc0 0x7f50480d0df0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:17:50.299 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:17:50.295+0000 7f504dee3640 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7f50480b2160 0x7f50480b0750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:17:50.299 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:17:50.295+0000 7f504dee3640 1 -- --> v1:172.21.15.188:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f50480cf0e0 con 0x7f504815d6e0 2023-11-26T07:17:50.300 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:17:50.295+0000 7f504dee3640 1 -- --> v2:172.21.15.143:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f504815dac0 con 0x7f50480b2160 2023-11-26T07:17:50.300 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:17:50.295+0000 7f504dee3640 1 -- --> v2:172.21.15.188:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f50480ceb20 con 0x7f50480d1bc0 2023-11-26T07:17:50.300 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:17:50.295+0000 7f5047fff640 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7f50480b2160 0x7f50480b0750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:17:50.300 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:17:50.295+0000 7f5047fff640 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7f50480b2160 0x7f50480b0750 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:53028/0 (socket says 172.21.15.143:53028) 2023-11-26T07:17:50.300 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:17:50.295+0000 7f5047fff640 1 -- 172.21.15.143:0/2020232433 learned_addr learned my addr 172.21.15.143:0/2020232433 (peer_addr_for_me v2:172.21.15.143:0/0) 2023-11-26T07:17:50.300 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:17:50.295+0000 7f5046ffd640 1 --2- 172.21.15.143:0/2020232433 >> v2:172.21.15.188:3300/0 conn(0x7f50480d1bc0 0x7f50480d0df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:17:50.300 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:17:50.295+0000 7f5047fff640 1 -- 172.21.15.143:0/2020232433 >> v2:172.21.15.188:3300/0 conn(0x7f50480d1bc0 msgr2=0x7f50480d0df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-26T07:17:50.300 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:17:50.295+0000 7f5047fff640 1 --2- 172.21.15.143:0/2020232433 >> v2:172.21.15.188:3300/0 conn(0x7f50480d1bc0 0x7f50480d0df0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:17:50.300 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:17:50.295+0000 7f5047fff640 1 -- 172.21.15.143:0/2020232433 >> v1:172.21.15.188:6789/0 conn(0x7f504815d6e0 legacy=0x7f50480d14b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:17:50.300 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:17:50.295+0000 7f5047fff640 1 -- 172.21.15.143:0/2020232433 --> v2:172.21.15.143:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f50480cee00 con 0x7f50480b2160 2023-11-26T07:17:50.302 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:17:50.299+0000 7f5047fff640 1 --2- 172.21.15.143:0/2020232433 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f50480b2160 0x7f50480b0750 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7f50480ca2a0 tx=0x7f504001c9d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ef9a8631d55a285d server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:17:50.302 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:17:50.299+0000 7f50467fc640 1 -- 172.21.15.143:0/2020232433 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f50400a5780 con 0x7f50480b2160 2023-11-26T07:17:50.302 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:17:50.299+0000 7f50467fc640 1 -- 172.21.15.143:0/2020232433 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f50400a58e0 con 0x7f50480b2160 2023-11-26T07:17:50.303 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:17:50.299+0000 7f50467fc640 1 -- 172.21.15.143:0/2020232433 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f50400a5c70 con 0x7f50480b2160 2023-11-26T07:17:50.303 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:17:50.299+0000 7f504dee3640 1 -- 172.21.15.143:0/2020232433 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f50480b2160 msgr2=0x7f50480b0750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:17:50.303 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:17:50.299+0000 7f504dee3640 1 --2- 172.21.15.143:0/2020232433 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f50480b2160 0x7f50480b0750 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7f50480ca2a0 tx=0x7f504001c9d0 comp rx=0 tx=0).stop 2023-11-26T07:17:50.303 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:17:50.299+0000 7f504dee3640 1 -- 172.21.15.143:0/2020232433 shutdown_connections 2023-11-26T07:17:50.304 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:17:50.299+0000 7f504dee3640 1 --2- 172.21.15.143:0/2020232433 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f50480b2160 0x7f50480b0750 unknown :-1 s=CLOSED pgs=53 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:17:50.304 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:17:50.299+0000 7f504dee3640 1 --2- 172.21.15.143:0/2020232433 >> v2:172.21.15.188:3300/0 conn(0x7f50480d1bc0 0x7f50480d0df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:17:50.304 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:17:50.299+0000 7f504dee3640 1 -- 172.21.15.143:0/2020232433 >> 172.21.15.143:0/2020232433 conn(0x7f504809c550 msgr2=0x7f50480d3220 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:17:50.304 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:17:50.299+0000 7f504dee3640 1 -- 172.21.15.143:0/2020232433 shutdown_connections 2023-11-26T07:17:50.304 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:17:50.299+0000 7f504dee3640 1 -- 172.21.15.143:0/2020232433 wait complete. 2023-11-26T07:17:50.304 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:17:50.303+0000 7f504dee3640 1 Processor -- start 2023-11-26T07:17:50.305 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:17:50.303+0000 7f504dee3640 1 -- start start 2023-11-26T07:17:50.309 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:17:50.303+0000 7f504dee3640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f50480b2160 0x7f50480c6d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:17:50.310 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:17:50.303+0000 7f504dee3640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f50480d1bc0 0x7f50481585c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:17:50.310 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:17:50.303+0000 7f504dee3640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f504815d6e0 0x7f50480c89c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:17:50.310 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:17:50.303+0000 7f504dee3640 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f50481590a0 con 0x7f50480d1bc0 2023-11-26T07:17:50.310 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:17:50.303+0000 7f504dee3640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f50480c7330 con 0x7f504815d6e0 2023-11-26T07:17:50.311 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:17:50.303+0000 7f504dee3640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f50480cbfa0 con 0x7f50480b2160 2023-11-26T07:17:50.311 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:17:50.303+0000 7f5046ffd640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f50480d1bc0 0x7f50481585c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:17:50.311 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:17:50.303+0000 7f5046ffd640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f50480d1bc0 0x7f50481585c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:53032/0 (socket says 172.21.15.143:53032) 2023-11-26T07:17:50.311 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:17:50.303+0000 7f5046ffd640 1 -- 172.21.15.143:0/4010095788 learned_addr learned my addr 172.21.15.143:0/4010095788 (peer_addr_for_me v2:172.21.15.143:0/0) 2023-11-26T07:17:50.311 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:17:50.303+0000 7f5047fff640 1 --2- 172.21.15.143:0/4010095788 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f504815d6e0 0x7f50480c89c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:17:50.312 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:17:50.303+0000 7f50477fe640 1 --2- 172.21.15.143:0/4010095788 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f50480b2160 0x7f50480c6d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:17:50.312 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:17:50.303+0000 7f5046ffd640 1 -- 172.21.15.143:0/4010095788 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f50480b2160 msgr2=0x7f50480c6d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:17:50.312 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:17:50.303+0000 7f5046ffd640 1 --2- 172.21.15.143:0/4010095788 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f50480b2160 0x7f50480c6d50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:17:50.312 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:17:50.303+0000 7f5046ffd640 1 -- 172.21.15.143:0/4010095788 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f504815d6e0 msgr2=0x7f50480c89c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:17:50.312 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:17:50.303+0000 7f5046ffd640 1 --2- 172.21.15.143:0/4010095788 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f504815d6e0 0x7f50480c89c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:17:50.313 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:17:50.303+0000 7f5046ffd640 1 -- 172.21.15.143:0/4010095788 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5040009b30 con 0x7f50480d1bc0 2023-11-26T07:17:50.313 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:17:50.303+0000 7f5046ffd640 1 --2- 172.21.15.143:0/4010095788 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f50480d1bc0 0x7f50481585c0 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7f503c00d800 tx=0x7f503c00dcc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:17:50.313 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:17:50.303+0000 7f502ffff640 1 -- 172.21.15.143:0/4010095788 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f503c00b6c0 con 0x7f50480d1bc0 2023-11-26T07:17:50.313 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:17:50.303+0000 7f502ffff640 1 -- 172.21.15.143:0/4010095788 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f503c00b820 con 0x7f50480d1bc0 2023-11-26T07:17:50.313 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:17:50.303+0000 7f504dee3640 1 -- 172.21.15.143:0/4010095788 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f50480cc280 con 0x7f50480d1bc0 2023-11-26T07:17:50.313 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:17:50.303+0000 7f502ffff640 1 -- 172.21.15.143:0/4010095788 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f503c00bb90 con 0x7f50480d1bc0 2023-11-26T07:17:50.314 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:17:50.303+0000 7f504dee3640 1 -- 172.21.15.143:0/4010095788 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5048158b00 con 0x7f50480d1bc0 2023-11-26T07:17:50.314 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:17:50.307+0000 7f504dee3640 1 -- 172.21.15.143:0/4010095788 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5048067680 con 0x7f50480d1bc0 2023-11-26T07:17:50.314 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:17:50.307+0000 7f502ffff640 1 -- 172.21.15.143:0/4010095788 <== mon.0 v2:172.21.15.143:3300/0 4 ==== mgrmap(e 5) v1 ==== 85852+0+0 (secure 0 0 0) 0x7f503c00bcf0 con 0x7f50480d1bc0 2023-11-26T07:17:50.315 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:17:50.307+0000 7f502ffff640 1 --2- 172.21.15.143:0/4010095788 >> [v2:172.21.15.143:6832/2150557544,v1:172.21.15.143:6833/2150557544] conn(0x7f50340664a0 0x7f5034068960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:17:50.315 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:17:50.307+0000 7f502ffff640 1 -- 172.21.15.143:0/4010095788 <== mon.0 v2:172.21.15.143:3300/0 5 ==== osd_map(16..16 src has 1..16) v4 ==== 4857+0+0 (secure 0 0 0) 0x7f503c083e70 con 0x7f50480d1bc0 2023-11-26T07:17:50.316 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:17:50.307+0000 7f50477fe640 1 --2- 172.21.15.143:0/4010095788 >> [v2:172.21.15.143:6832/2150557544,v1:172.21.15.143:6833/2150557544] conn(0x7f50340664a0 0x7f5034068960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:17:50.316 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:17:50.307+0000 7f50477fe640 1 --2- 172.21.15.143:0/4010095788 >> [v2:172.21.15.143:6832/2150557544,v1:172.21.15.143:6833/2150557544] conn(0x7f50340664a0 0x7f5034068960 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7f5048073860 tx=0x7f5038009040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:17:50.316 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:17:50.311+0000 7f502ffff640 1 -- 172.21.15.143:0/4010095788 <== mon.0 v2:172.21.15.143:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172649 (secure 0 0 0) 0x7f503c051780 con 0x7f50480d1bc0 2023-11-26T07:17:50.502 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:17:50.499+0000 7f504dee3640 1 -- 172.21.15.143:0/4010095788 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f5048073b10 con 0x7f50480d1bc0 2023-11-26T07:17:50.503 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:17:50.499+0000 7f502ffff640 1 -- 172.21.15.143:0/4010095788 <== mon.0 v2:172.21.15.143:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 3 v3) v1 ==== 93+0+3700 (secure 0 0 0) 0x7f503c054da0 con 0x7f50480d1bc0 2023-11-26T07:17:50.503 INFO:teuthology.orchestra.run.smithi143.stdout: 2023-11-26T07:17:50.503 INFO:teuthology.orchestra.run.smithi143.stdout:{"epoch":3,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4241,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.188:6834/652687322","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6832","nonce":652687322},{"type":"v1","addr":"172.21.15.188:6834","nonce":652687322}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":4266,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.143:6835/3850184704","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6834","nonce":3850184704},{"type":"v1","addr":"172.21.15.143:6835","nonce":3850184704}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":2},{"gid":4273,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.143:6837/1695478651","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6836","nonce":1695478651},{"type":"v1","addr":"172.21.15.143:6837","nonce":1695478651}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":4285,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.188:6835/994617268","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6833","nonce":994617268},{"type":"v1","addr":"172.21.15.188:6835","nonce":994617268}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3}],"filesystems":[]} 2023-11-26T07:17:50.504 INFO:teuthology.orchestra.run.smithi143.stderr:dumped fsmap epoch 3 2023-11-26T07:17:50.505 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:17:50.503+0000 7f504dee3640 1 -- 172.21.15.143:0/4010095788 >> [v2:172.21.15.143:6832/2150557544,v1:172.21.15.143:6833/2150557544] conn(0x7f50340664a0 msgr2=0x7f5034068960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:17:50.505 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:17:50.503+0000 7f504dee3640 1 --2- 172.21.15.143:0/4010095788 >> [v2:172.21.15.143:6832/2150557544,v1:172.21.15.143:6833/2150557544] conn(0x7f50340664a0 0x7f5034068960 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7f5048073860 tx=0x7f5038009040 comp rx=0 tx=0).stop 2023-11-26T07:17:50.505 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:17:50.503+0000 7f504dee3640 1 -- 172.21.15.143:0/4010095788 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f50480d1bc0 msgr2=0x7f50481585c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:17:50.505 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:17:50.503+0000 7f504dee3640 1 --2- 172.21.15.143:0/4010095788 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f50480d1bc0 0x7f50481585c0 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7f503c00d800 tx=0x7f503c00dcc0 comp rx=0 tx=0).stop 2023-11-26T07:17:50.505 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:17:50.503+0000 7f504dee3640 1 -- 172.21.15.143:0/4010095788 shutdown_connections 2023-11-26T07:17:50.505 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:17:50.503+0000 7f504dee3640 1 --2- 172.21.15.143:0/4010095788 >> [v2:172.21.15.143:6832/2150557544,v1:172.21.15.143:6833/2150557544] conn(0x7f50340664a0 0x7f5034068960 unknown :-1 s=CLOSED pgs=59 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:17:50.505 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:17:50.503+0000 7f504dee3640 1 --2- 172.21.15.143:0/4010095788 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f504815d6e0 0x7f50480c89c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:17:50.506 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:17:50.503+0000 7f504dee3640 1 --2- 172.21.15.143:0/4010095788 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f50480d1bc0 0x7f50481585c0 unknown :-1 s=CLOSED pgs=54 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:17:50.506 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:17:50.503+0000 7f504dee3640 1 --2- 172.21.15.143:0/4010095788 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f50480b2160 0x7f50480c6d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:17:50.506 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:17:50.503+0000 7f504dee3640 1 -- 172.21.15.143:0/4010095788 >> 172.21.15.143:0/4010095788 conn(0x7f504809c550 msgr2=0x7f50480b25e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:17:50.506 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:17:50.503+0000 7f504dee3640 1 -- 172.21.15.143:0/4010095788 shutdown_connections 2023-11-26T07:17:50.506 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:17:50.503+0000 7f504dee3640 1 -- 172.21.15.143:0/4010095788 wait complete. 2023-11-26T07:17:50.524 DEBUG:teuthology.orchestra.run.smithi143:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-11-26T07:17:50.665 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:17:50.663+0000 7f4e84c9a640 1 Processor -- start 2023-11-26T07:17:50.665 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:17:50.663+0000 7f4e84c9a640 1 -- start start 2023-11-26T07:17:50.666 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:17:50.663+0000 7f4e84c9a640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f4e8015d6e0 0x7f4e800d0df0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:17:50.667 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:17:50.663+0000 7f4e84c9a640 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7f4e800b2160 0x7f4e800d14b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:17:50.667 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:17:50.663+0000 7f4e84c9a640 1 -- --> v1:172.21.15.143:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4e800cf0e0 con 0x7f4e800d19f0 2023-11-26T07:17:50.667 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:17:50.663+0000 7f4e84c9a640 1 -- --> v2:172.21.15.143:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4e8015dac0 con 0x7f4e800b2160 2023-11-26T07:17:50.667 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:17:50.663+0000 7f4e84c9a640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4e800d1e80 con 0x7f4e8015d6e0 2023-11-26T07:17:50.667 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:17:50.663+0000 7f4e7dd9b640 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7f4e800b2160 0x7f4e800d14b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:17:50.667 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:17:50.663+0000 7f4e7dd9b640 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7f4e800b2160 0x7f4e800d14b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:53038/0 (socket says 172.21.15.143:53038) 2023-11-26T07:17:50.667 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:17:50.663+0000 7f4e7dd9b640 1 -- 172.21.15.143:0/1529268615 learned_addr learned my addr 172.21.15.143:0/1529268615 (peer_addr_for_me v2:172.21.15.143:0/0) 2023-11-26T07:17:50.667 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:17:50.663+0000 7f4e7ed9d640 1 --1- 172.21.15.143:0/1529268615 >> v1:172.21.15.143:6789/0 conn(0x7f4e800d19f0 0x7f4e800cfbf0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.143:6789/0 says I am v1:172.21.15.143:57456/0 (socket says 172.21.15.143:57456) 2023-11-26T07:17:50.668 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:17:50.663+0000 7f4e7e59c640 1 --2- 172.21.15.143:0/1529268615 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f4e8015d6e0 0x7f4e800d0df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:17:50.668 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:17:50.663+0000 7f4e7dd9b640 1 -- 172.21.15.143:0/1529268615 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f4e8015d6e0 msgr2=0x7f4e800d0df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-26T07:17:50.668 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:17:50.663+0000 7f4e7dd9b640 1 --2- 172.21.15.143:0/1529268615 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f4e8015d6e0 0x7f4e800d0df0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:17:50.668 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:17:50.663+0000 7f4e7dd9b640 1 -- 172.21.15.143:0/1529268615 >> v1:172.21.15.143:6789/0 conn(0x7f4e800d19f0 legacy=0x7f4e800cfbf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:17:50.668 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:17:50.663+0000 7f4e7dd9b640 1 -- 172.21.15.143:0/1529268615 --> v2:172.21.15.143:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4e800ceb20 con 0x7f4e800b2160 2023-11-26T07:17:50.668 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:17:50.663+0000 7f4e7d59a640 1 -- 172.21.15.143:0/1529268615 <== mon.0 v1:172.21.15.143:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3999512906 0 0) 0x7f4e800cf0e0 con 0x7f4e800d19f0 2023-11-26T07:17:50.668 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:17:50.663+0000 7f4e7e59c640 1 --2- 172.21.15.143:0/1529268615 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f4e8015d6e0 0x7f4e800d0df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-26T07:17:50.670 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:17:50.667+0000 7f4e7dd9b640 1 --2- 172.21.15.143:0/1529268615 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f4e800b2160 0x7f4e800d14b0 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7f4e74009870 tx=0x7f4e7401c540 comp rx=0 tx=0).ready entity=mon.0 client_cookie=599dd6a1ac76535a server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:17:50.670 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:17:50.667+0000 7f4e7d59a640 1 -- 172.21.15.143:0/1529268615 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4e740a7070 con 0x7f4e800b2160 2023-11-26T07:17:50.670 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:17:50.667+0000 7f4e7d59a640 1 -- 172.21.15.143:0/1529268615 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4e740ab430 con 0x7f4e800b2160 2023-11-26T07:17:50.670 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:17:50.667+0000 7f4e7d59a640 1 -- 172.21.15.143:0/1529268615 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4e740ab7c0 con 0x7f4e800b2160 2023-11-26T07:17:50.671 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:17:50.667+0000 7f4e84c9a640 1 -- 172.21.15.143:0/1529268615 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f4e800b2160 msgr2=0x7f4e800d14b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:17:50.671 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:17:50.667+0000 7f4e84c9a640 1 --2- 172.21.15.143:0/1529268615 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f4e800b2160 0x7f4e800d14b0 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7f4e74009870 tx=0x7f4e7401c540 comp rx=0 tx=0).stop 2023-11-26T07:17:50.671 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:17:50.667+0000 7f4e84c9a640 1 -- 172.21.15.143:0/1529268615 shutdown_connections 2023-11-26T07:17:50.671 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:17:50.667+0000 7f4e84c9a640 1 --2- 172.21.15.143:0/1529268615 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f4e800b2160 0x7f4e800d14b0 unknown :-1 s=CLOSED pgs=56 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:17:50.671 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:17:50.667+0000 7f4e84c9a640 1 --2- 172.21.15.143:0/1529268615 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f4e8015d6e0 0x7f4e800d0df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:17:50.671 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:17:50.667+0000 7f4e84c9a640 1 -- 172.21.15.143:0/1529268615 >> 172.21.15.143:0/1529268615 conn(0x7f4e8009c550 msgr2=0x7f4e800d3220 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:17:50.672 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:17:50.667+0000 7f4e84c9a640 1 -- 172.21.15.143:0/1529268615 shutdown_connections 2023-11-26T07:17:50.672 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:17:50.667+0000 7f4e84c9a640 1 -- 172.21.15.143:0/1529268615 wait complete. 2023-11-26T07:17:50.673 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:17:50.671+0000 7f4e84c9a640 1 Processor -- start 2023-11-26T07:17:50.673 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:17:50.671+0000 7f4e84c9a640 1 -- start start 2023-11-26T07:17:50.675 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:17:50.671+0000 7f4e84c9a640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f4e800b2160 0x7f4e800ca2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:17:50.675 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:17:50.671+0000 7f4e84c9a640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f4e800d19f0 0x7f4e800cb970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:17:50.675 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:17:50.671+0000 7f4e84c9a640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f4e8015d6e0 0x7f4e800b5420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:17:50.675 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:17:50.671+0000 7f4e84c9a640 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4e800cb550 con 0x7f4e800b2160 2023-11-26T07:17:50.675 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:17:50.671+0000 7f4e84c9a640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4e800ca8d0 con 0x7f4e8015d6e0 2023-11-26T07:17:50.676 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:17:50.671+0000 7f4e84c9a640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4e800cabb0 con 0x7f4e800d19f0 2023-11-26T07:17:50.676 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:17:50.671+0000 7f4e7e59c640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f4e800b2160 0x7f4e800ca2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:17:50.676 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:17:50.671+0000 7f4e7e59c640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f4e800b2160 0x7f4e800ca2f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:53052/0 (socket says 172.21.15.143:53052) 2023-11-26T07:17:50.676 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:17:50.671+0000 7f4e7e59c640 1 -- 172.21.15.143:0/2776720540 learned_addr learned my addr 172.21.15.143:0/2776720540 (peer_addr_for_me v2:172.21.15.143:0/0) 2023-11-26T07:17:50.676 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:17:50.671+0000 7f4e7dd9b640 1 --2- 172.21.15.143:0/2776720540 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f4e800d19f0 0x7f4e800cb970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:17:50.676 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:17:50.671+0000 7f4e7ed9d640 1 --2- 172.21.15.143:0/2776720540 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f4e8015d6e0 0x7f4e800b5420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:17:50.676 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:17:50.671+0000 7f4e7e59c640 1 -- 172.21.15.143:0/2776720540 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f4e800d19f0 msgr2=0x7f4e800cb970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:17:50.676 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:17:50.671+0000 7f4e7e59c640 1 --2- 172.21.15.143:0/2776720540 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f4e800d19f0 0x7f4e800cb970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:17:50.676 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:17:50.671+0000 7f4e7e59c640 1 -- 172.21.15.143:0/2776720540 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f4e8015d6e0 msgr2=0x7f4e800b5420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:17:50.676 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:17:50.671+0000 7f4e7e59c640 1 --2- 172.21.15.143:0/2776720540 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f4e8015d6e0 0x7f4e800b5420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:17:50.676 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:17:50.671+0000 7f4e7e59c640 1 -- 172.21.15.143:0/2776720540 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4e740094d0 con 0x7f4e800b2160 2023-11-26T07:17:50.676 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:17:50.671+0000 7f4e7dd9b640 1 --2- 172.21.15.143:0/2776720540 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f4e800d19f0 0x7f4e800cb970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-26T07:17:50.677 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:17:50.671+0000 7f4e7e59c640 1 --2- 172.21.15.143:0/2776720540 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f4e800b2160 0x7f4e800ca2f0 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7f4e7000c7f0 tx=0x7f4e7000ccb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:17:50.677 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:17:50.671+0000 7f4e6effd640 1 -- 172.21.15.143:0/2776720540 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4e700077c0 con 0x7f4e800b2160 2023-11-26T07:17:50.677 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:17:50.671+0000 7f4e6effd640 1 -- 172.21.15.143:0/2776720540 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4e70007920 con 0x7f4e800b2160 2023-11-26T07:17:50.677 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:17:50.671+0000 7f4e6effd640 1 -- 172.21.15.143:0/2776720540 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4e70007cd0 con 0x7f4e800b2160 2023-11-26T07:17:50.677 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:17:50.671+0000 7f4e84c9a640 1 -- 172.21.15.143:0/2776720540 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4e800cae90 con 0x7f4e800b2160 2023-11-26T07:17:50.677 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:17:50.671+0000 7f4e84c9a640 1 -- 172.21.15.143:0/2776720540 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4e800b5960 con 0x7f4e800b2160 2023-11-26T07:17:50.677 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:17:50.675+0000 7f4e84c9a640 1 -- 172.21.15.143:0/2776720540 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4e800cb7c0 con 0x7f4e800b2160 2023-11-26T07:17:50.678 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:17:50.675+0000 7f4e6effd640 1 -- 172.21.15.143:0/2776720540 <== mon.0 v2:172.21.15.143:3300/0 4 ==== mgrmap(e 5) v1 ==== 85852+0+0 (secure 0 0 0) 0x7f4e700113f0 con 0x7f4e800b2160 2023-11-26T07:17:50.678 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:17:50.675+0000 7f4e6effd640 1 --2- 172.21.15.143:0/2776720540 >> [v2:172.21.15.143:6832/2150557544,v1:172.21.15.143:6833/2150557544] conn(0x7f4e60066470 0x7f4e60068930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:17:50.678 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:17:50.675+0000 7f4e6effd640 1 -- 172.21.15.143:0/2776720540 <== mon.0 v2:172.21.15.143:3300/0 5 ==== osd_map(16..16 src has 1..16) v4 ==== 4857+0+0 (secure 0 0 0) 0x7f4e70083c60 con 0x7f4e800b2160 2023-11-26T07:17:50.678 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:17:50.675+0000 7f4e7dd9b640 1 --2- 172.21.15.143:0/2776720540 >> [v2:172.21.15.143:6832/2150557544,v1:172.21.15.143:6833/2150557544] conn(0x7f4e60066470 0x7f4e60068930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:17:50.678 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:17:50.675+0000 7f4e7dd9b640 1 --2- 172.21.15.143:0/2776720540 >> [v2:172.21.15.143:6832/2150557544,v1:172.21.15.143:6833/2150557544] conn(0x7f4e60066470 0x7f4e60068930 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7f4e740af000 tx=0x7f4e74008040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:17:50.682 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:17:50.679+0000 7f4e6effd640 1 -- 172.21.15.143:0/2776720540 <== mon.0 v2:172.21.15.143:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172649 (secure 0 0 0) 0x7f4e70051420 con 0x7f4e800b2160 2023-11-26T07:17:50.835 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:17:50.831+0000 7f4e84c9a640 1 -- 172.21.15.143:0/2776720540 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f4e80067680 con 0x7f4e800b2160 2023-11-26T07:17:50.836 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:17:50.831+0000 7f4e6effd640 1 -- 172.21.15.143:0/2776720540 <== mon.0 v2:172.21.15.143:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v16) v1 ==== 74+0+12886 (secure 0 0 0) 0x7f4e70054a40 con 0x7f4e800b2160 2023-11-26T07:17:50.836 INFO:teuthology.orchestra.run.smithi143.stdout: 2023-11-26T07:17:50.836 INFO:teuthology.orchestra.run.smithi143.stdout:{"epoch":16,"fsid":"f5033060-2469-4a07-95fe-2b76f5404bc1","created":"2023-11-26T07:17:28.613663+0000","modified":"2023-11-26T07:17:44.413749+0000","last_up_change":"2023-11-26T07:17:41.157813+0000","last_in_change":"2023-11-26T07:17:31.181658+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":1,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-11-26T07:17:41.758597+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"ebbee2c6-023e-44b9-9f1d-6f7bfd309730","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6817","nonce":639807369},{"type":"v1","addr":"172.21.15.143:6819","nonce":639807369}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6821","nonce":639807369},{"type":"v1","addr":"172.21.15.143:6823","nonce":639807369}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6830","nonce":639807369},{"type":"v1","addr":"172.21.15.143:6831","nonce":639807369}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6825","nonce":639807369},{"type":"v1","addr":"172.21.15.143:6828","nonce":639807369}]},"public_addr":"172.21.15.143:6819/639807369","cluster_addr":"172.21.15.143:6823/639807369","heartbeat_back_addr":"172.21.15.143:6831/639807369","heartbeat_front_addr":"172.21.15.143:6828/639807369","state":["exists","up"]},{"osd":1,"uuid":"fec4b8be-7122-49e9-9a19-5850a0700001","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6816","nonce":1144942530},{"type":"v1","addr":"172.21.15.143:6818","nonce":1144942530}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6820","nonce":1144942530},{"type":"v1","addr":"172.21.15.143:6822","nonce":1144942530}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6827","nonce":1144942530},{"type":"v1","addr":"172.21.15.143:6829","nonce":1144942530}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6824","nonce":1144942530},{"type":"v1","addr":"172.21.15.143:6826","nonce":1144942530}]},"public_addr":"172.21.15.143:6818/1144942530","cluster_addr":"172.21.15.143:6822/1144942530","heartbeat_back_addr":"172.21.15.143:6829/1144942530","heartbeat_front_addr":"172.21.15.143:6826/1144942530","state":["exists","up"]},{"osd":2,"uuid":"2888af5b-f933-46ca-8f95-6dd494ff43f6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6808","nonce":3909095186},{"type":"v1","addr":"172.21.15.143:6809","nonce":3909095186}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6810","nonce":3909095186},{"type":"v1","addr":"172.21.15.143:6811","nonce":3909095186}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6814","nonce":3909095186},{"type":"v1","addr":"172.21.15.143:6815","nonce":3909095186}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6812","nonce":3909095186},{"type":"v1","addr":"172.21.15.143:6813","nonce":3909095186}]},"public_addr":"172.21.15.143:6809/3909095186","cluster_addr":"172.21.15.143:6811/3909095186","heartbeat_back_addr":"172.21.15.143:6815/3909095186","heartbeat_front_addr":"172.21.15.143:6813/3909095186","state":["exists","up"]},{"osd":3,"uuid":"d1f52435-7bc1-4081-9550-9cd1ba55f549","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6800","nonce":3854210991},{"type":"v1","addr":"172.21.15.143:6801","nonce":3854210991}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6802","nonce":3854210991},{"type":"v1","addr":"172.21.15.143:6803","nonce":3854210991}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6806","nonce":3854210991},{"type":"v1","addr":"172.21.15.143:6807","nonce":3854210991}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6804","nonce":3854210991},{"type":"v1","addr":"172.21.15.143:6805","nonce":3854210991}]},"public_addr":"172.21.15.143:6801/3854210991","cluster_addr":"172.21.15.143:6803/3854210991","heartbeat_back_addr":"172.21.15.143:6807/3854210991","heartbeat_front_addr":"172.21.15.143:6805/3854210991","state":["exists","up"]},{"osd":4,"uuid":"e2ae8b6e-6e25-49b2-bcbf-aacce68f06a3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6800","nonce":3924606902},{"type":"v1","addr":"172.21.15.188:6801","nonce":3924606902}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6802","nonce":3924606902},{"type":"v1","addr":"172.21.15.188:6803","nonce":3924606902}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6806","nonce":3924606902},{"type":"v1","addr":"172.21.15.188:6807","nonce":3924606902}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6804","nonce":3924606902},{"type":"v1","addr":"172.21.15.188:6805","nonce":3924606902}]},"public_addr":"172.21.15.188:6801/3924606902","cluster_addr":"172.21.15.188:6803/3924606902","heartbeat_back_addr":"172.21.15.188:6807/3924606902","heartbeat_front_addr":"172.21.15.188:6805/3924606902","state":["exists","up"]},{"osd":5,"uuid":"3cec8a32-7698-4c36-8e89-e4ba11934f33","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6813","nonce":186841544},{"type":"v1","addr":"172.21.15.188:6815","nonce":186841544}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6817","nonce":186841544},{"type":"v1","addr":"172.21.15.188:6818","nonce":186841544}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6822","nonce":186841544},{"type":"v1","addr":"172.21.15.188:6823","nonce":186841544}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6820","nonce":186841544},{"type":"v1","addr":"172.21.15.188:6821","nonce":186841544}]},"public_addr":"172.21.15.188:6815/186841544","cluster_addr":"172.21.15.188:6818/186841544","heartbeat_back_addr":"172.21.15.188:6823/186841544","heartbeat_front_addr":"172.21.15.188:6821/186841544","state":["exists","up"]},{"osd":6,"uuid":"58a65d5d-e090-473a-81b4-ad38e85029f9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6808","nonce":2153499405},{"type":"v1","addr":"172.21.15.188:6809","nonce":2153499405}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6810","nonce":2153499405},{"type":"v1","addr":"172.21.15.188:6811","nonce":2153499405}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6816","nonce":2153499405},{"type":"v1","addr":"172.21.15.188:6819","nonce":2153499405}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6812","nonce":2153499405},{"type":"v1","addr":"172.21.15.188:6814","nonce":2153499405}]},"public_addr":"172.21.15.188:6809/2153499405","cluster_addr":"172.21.15.188:6811/2153499405","heartbeat_back_addr":"172.21.15.188:6819/2153499405","heartbeat_front_addr":"172.21.15.188:6814/2153499405","state":["exists","up"]},{"osd":7,"uuid":"ea5b7f70-3554-4cf8-ac30-f7c59b80678d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":14,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6824","nonce":2722096729},{"type":"v1","addr":"172.21.15.188:6825","nonce":2722096729}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6826","nonce":2722096729},{"type":"v1","addr":"172.21.15.188:6827","nonce":2722096729}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6830","nonce":2722096729},{"type":"v1","addr":"172.21.15.188:6831","nonce":2722096729}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6828","nonce":2722096729},{"type":"v1","addr":"172.21.15.188:6829","nonce":2722096729}]},"public_addr":"172.21.15.188:6825/2722096729","cluster_addr":"172.21.15.188:6827/2722096729","heartbeat_back_addr":"172.21.15.188:6831/2722096729","heartbeat_front_addr":"172.21.15.188:6829/2722096729","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-26T07:17:36.995032+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-26T07:17:36.932266+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-26T07:17:36.829304+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-26T07:17:36.804297+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-26T07:17:36.667892+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-26T07:17:36.964868+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-26T07:17:36.827332+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-26T07:17:36.893179+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-11-26T07:17:50.838 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:17:50.835+0000 7f4e84c9a640 1 -- 172.21.15.143:0/2776720540 >> [v2:172.21.15.143:6832/2150557544,v1:172.21.15.143:6833/2150557544] conn(0x7f4e60066470 msgr2=0x7f4e60068930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:17:50.838 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:17:50.835+0000 7f4e84c9a640 1 --2- 172.21.15.143:0/2776720540 >> [v2:172.21.15.143:6832/2150557544,v1:172.21.15.143:6833/2150557544] conn(0x7f4e60066470 0x7f4e60068930 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7f4e740af000 tx=0x7f4e74008040 comp rx=0 tx=0).stop 2023-11-26T07:17:50.838 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:17:50.835+0000 7f4e84c9a640 1 -- 172.21.15.143:0/2776720540 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f4e800b2160 msgr2=0x7f4e800ca2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:17:50.838 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:17:50.835+0000 7f4e84c9a640 1 --2- 172.21.15.143:0/2776720540 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f4e800b2160 0x7f4e800ca2f0 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7f4e7000c7f0 tx=0x7f4e7000ccb0 comp rx=0 tx=0).stop 2023-11-26T07:17:50.839 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:17:50.835+0000 7f4e84c9a640 1 -- 172.21.15.143:0/2776720540 shutdown_connections 2023-11-26T07:17:50.839 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:17:50.835+0000 7f4e84c9a640 1 --2- 172.21.15.143:0/2776720540 >> [v2:172.21.15.143:6832/2150557544,v1:172.21.15.143:6833/2150557544] conn(0x7f4e60066470 0x7f4e60068930 unknown :-1 s=CLOSED pgs=60 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:17:50.839 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:17:50.835+0000 7f4e84c9a640 1 --2- 172.21.15.143:0/2776720540 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f4e8015d6e0 0x7f4e800b5420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:17:50.839 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:17:50.835+0000 7f4e84c9a640 1 --2- 172.21.15.143:0/2776720540 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f4e800d19f0 0x7f4e800cb970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:17:50.839 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:17:50.835+0000 7f4e84c9a640 1 --2- 172.21.15.143:0/2776720540 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f4e800b2160 0x7f4e800ca2f0 unknown :-1 s=CLOSED pgs=57 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:17:50.839 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:17:50.835+0000 7f4e84c9a640 1 -- 172.21.15.143:0/2776720540 >> 172.21.15.143:0/2776720540 conn(0x7f4e8009c550 msgr2=0x7f4e800b7e20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:17:50.839 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:17:50.835+0000 7f4e84c9a640 1 -- 172.21.15.143:0/2776720540 shutdown_connections 2023-11-26T07:17:50.839 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:17:50.835+0000 7f4e84c9a640 1 -- 172.21.15.143:0/2776720540 wait complete. 2023-11-26T07:17:50.857 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-11-26T07:17:41.758597+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2023-11-26T07:17:50.858 DEBUG:teuthology.orchestra.run.smithi143:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-11-26T07:17:50.998 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:17:50.995+0000 7f3c76fc1640 1 Processor -- start 2023-11-26T07:17:50.998 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:17:50.995+0000 7f3c76fc1640 1 -- start start 2023-11-26T07:17:50.999 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:17:50.995+0000 7f3c76fc1640 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7f3c700b2160 0x7f3c700d0df0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:17:50.999 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:17:50.995+0000 7f3c76fc1640 1 -- --> v1:172.21.15.143:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3c700cf0e0 con 0x7f3c700d1bc0 2023-11-26T07:17:50.999 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:17:50.995+0000 7f3c76fc1640 1 -- --> v1:172.21.15.188:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3c700ceb20 con 0x7f3c7015d6e0 2023-11-26T07:17:50.999 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:17:50.995+0000 7f3c76fc1640 1 -- --> v2:172.21.15.188:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3c7015dac0 con 0x7f3c700b2160 2023-11-26T07:17:51.000 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:17:50.995+0000 7f3c67fff640 1 --1- >> v1:172.21.15.143:6789/0 conn(0x7f3c700d1bc0 0x7f3c700b0750 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.143:6789/0 says I am v1:172.21.15.143:57470/0 (socket says 172.21.15.143:57470) 2023-11-26T07:17:51.000 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:17:50.995+0000 7f3c67fff640 1 -- 172.21.15.143:0/2217918249 learned_addr learned my addr 172.21.15.143:0/2217918249 (peer_addr_for_me v1:172.21.15.143:0/0) 2023-11-26T07:17:51.000 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:17:50.995+0000 7f3c7555e640 1 --2- 172.21.15.143:0/2217918249 >> v2:172.21.15.188:3300/0 conn(0x7f3c700b2160 0x7f3c700d0df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:17:51.000 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:17:50.995+0000 7f3c677fe640 1 -- 172.21.15.143:0/2217918249 <== mon.0 v1:172.21.15.143:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3661439995 0 0) 0x7f3c700cf0e0 con 0x7f3c700d1bc0 2023-11-26T07:17:51.001 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:17:50.995+0000 7f3c677fe640 1 -- 172.21.15.143:0/2217918249 --> v1:172.21.15.143:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f3c58003530 con 0x7f3c700d1bc0 2023-11-26T07:17:51.001 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:17:50.995+0000 7f3c677fe640 1 -- 172.21.15.143:0/2217918249 <== mon.1 v1:172.21.15.188:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1198396921 0 0) 0x7f3c700ceb20 con 0x7f3c7015d6e0 2023-11-26T07:17:51.001 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:17:50.995+0000 7f3c677fe640 1 -- 172.21.15.143:0/2217918249 --> v1:172.21.15.188:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f3c700cf0e0 con 0x7f3c7015d6e0 2023-11-26T07:17:51.001 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:17:50.995+0000 7f3c7555e640 1 -- 172.21.15.143:0/2217918249 >> v1:172.21.15.188:6789/0 conn(0x7f3c7015d6e0 legacy=0x7f3c700d14b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:17:51.001 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:17:50.995+0000 7f3c7555e640 1 -- 172.21.15.143:0/2217918249 >> v1:172.21.15.143:6789/0 conn(0x7f3c700d1bc0 legacy=0x7f3c700b0750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:17:51.001 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:17:50.995+0000 7f3c7555e640 1 -- 172.21.15.143:0/2217918249 --> v2:172.21.15.188:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3c700cee00 con 0x7f3c700b2160 2023-11-26T07:17:51.003 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:17:50.999+0000 7f3c7555e640 1 --2- 172.21.15.143:0/2217918249 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f3c700b2160 0x7f3c700d0df0 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7f3c68009d90 tx=0x7f3c6801c950 comp rx=0 tx=0).ready entity=mon.1 client_cookie=45fa6e42e0d13519 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:17:51.003 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:17:50.999+0000 7f3c677fe640 1 -- 172.21.15.143:0/2217918249 <== mon.1 v2:172.21.15.188:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3c680a5900 con 0x7f3c700b2160 2023-11-26T07:17:51.003 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:17:50.999+0000 7f3c677fe640 1 -- 172.21.15.143:0/2217918249 <== mon.1 v2:172.21.15.188:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3c680a5a60 con 0x7f3c700b2160 2023-11-26T07:17:51.003 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:17:50.999+0000 7f3c677fe640 1 -- 172.21.15.143:0/2217918249 <== mon.1 v2:172.21.15.188:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3c680a5dd0 con 0x7f3c700b2160 2023-11-26T07:17:51.004 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:17:50.999+0000 7f3c76fc1640 1 -- 172.21.15.143:0/2217918249 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f3c700b2160 msgr2=0x7f3c700d0df0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:17:51.004 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:17:50.999+0000 7f3c76fc1640 1 --2- 172.21.15.143:0/2217918249 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f3c700b2160 0x7f3c700d0df0 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7f3c68009d90 tx=0x7f3c6801c950 comp rx=0 tx=0).stop 2023-11-26T07:17:51.004 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:17:50.999+0000 7f3c76fc1640 1 -- 172.21.15.143:0/2217918249 shutdown_connections 2023-11-26T07:17:51.004 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:17:50.999+0000 7f3c76fc1640 1 --2- 172.21.15.143:0/2217918249 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f3c700b2160 0x7f3c700d0df0 unknown :-1 s=CLOSED pgs=45 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:17:51.005 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:17:50.999+0000 7f3c76fc1640 1 -- 172.21.15.143:0/2217918249 >> 172.21.15.143:0/2217918249 conn(0x7f3c7009c550 msgr2=0x7f3c700d3220 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:17:51.005 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:17:50.999+0000 7f3c76fc1640 1 -- 172.21.15.143:0/2217918249 shutdown_connections 2023-11-26T07:17:51.005 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:17:50.999+0000 7f3c76fc1640 1 -- 172.21.15.143:0/2217918249 wait complete. 2023-11-26T07:17:51.005 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:17:51.003+0000 7f3c76fc1640 1 Processor -- start 2023-11-26T07:17:51.006 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:17:51.003+0000 7f3c76fc1640 1 -- start start 2023-11-26T07:17:51.007 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:17:51.003+0000 7f3c76fc1640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f3c700b2160 0x7f3c7015bcd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:17:51.008 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:17:51.003+0000 7f3c76fc1640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f3c700d1bc0 0x7f3c7015e050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:17:51.008 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:17:51.003+0000 7f3c76fc1640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f3c7015d6e0 0x7f3c7006c110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:17:51.008 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:17:51.003+0000 7f3c76fc1640 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3c7015e590 con 0x7f3c7015d6e0 2023-11-26T07:17:51.008 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:17:51.003+0000 7f3c76fc1640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3c701560b0 con 0x7f3c700d1bc0 2023-11-26T07:17:51.008 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:17:51.003+0000 7f3c76fc1640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3c7015c300 con 0x7f3c700b2160 2023-11-26T07:17:51.008 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:17:51.003+0000 7f3c7555e640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f3c7015d6e0 0x7f3c7006c110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:17:51.008 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:17:51.003+0000 7f3c7555e640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f3c7015d6e0 0x7f3c7006c110 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:53068/0 (socket says 172.21.15.143:53068) 2023-11-26T07:17:51.008 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:17:51.003+0000 7f3c7555e640 1 -- 172.21.15.143:0/345069385 learned_addr learned my addr 172.21.15.143:0/345069385 (peer_addr_for_me v2:172.21.15.143:0/0) 2023-11-26T07:17:51.008 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:17:51.003+0000 7f3c67fff640 1 --2- 172.21.15.143:0/345069385 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f3c700d1bc0 0x7f3c7015e050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:17:51.008 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:17:51.003+0000 7f3c74d5d640 1 --2- 172.21.15.143:0/345069385 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f3c700b2160 0x7f3c7015bcd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:17:51.008 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:17:51.003+0000 7f3c7555e640 1 -- 172.21.15.143:0/345069385 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f3c700b2160 msgr2=0x7f3c7015bcd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:17:51.008 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:17:51.003+0000 7f3c7555e640 1 --2- 172.21.15.143:0/345069385 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f3c700b2160 0x7f3c7015bcd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:17:51.008 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:17:51.003+0000 7f3c7555e640 1 -- 172.21.15.143:0/345069385 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f3c700d1bc0 msgr2=0x7f3c7015e050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:17:51.009 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:17:51.003+0000 7f3c7555e640 1 --2- 172.21.15.143:0/345069385 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f3c700d1bc0 0x7f3c7015e050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:17:51.009 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:17:51.003+0000 7f3c7555e640 1 -- 172.21.15.143:0/345069385 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3c68009a70 con 0x7f3c7015d6e0 2023-11-26T07:17:51.009 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:17:51.003+0000 7f3c7555e640 1 --2- 172.21.15.143:0/345069385 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f3c7015d6e0 0x7f3c7006c110 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7f3c6801a580 tx=0x7f3c680196f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:17:51.010 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:17:51.007+0000 7f3c657fa640 1 -- 172.21.15.143:0/345069385 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3c680ad640 con 0x7f3c7015d6e0 2023-11-26T07:17:51.010 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:17:51.007+0000 7f3c657fa640 1 -- 172.21.15.143:0/345069385 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3c680ad7a0 con 0x7f3c7015d6e0 2023-11-26T07:17:51.010 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:17:51.007+0000 7f3c76fc1640 1 -- 172.21.15.143:0/345069385 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3c70154c40 con 0x7f3c7015d6e0 2023-11-26T07:17:51.010 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:17:51.007+0000 7f3c657fa640 1 -- 172.21.15.143:0/345069385 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3c680adb30 con 0x7f3c7015d6e0 2023-11-26T07:17:51.010 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:17:51.007+0000 7f3c76fc1640 1 -- 172.21.15.143:0/345069385 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3c7015b500 con 0x7f3c7015d6e0 2023-11-26T07:17:51.010 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:17:51.007+0000 7f3c76fc1640 1 -- 172.21.15.143:0/345069385 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3c70156240 con 0x7f3c7015d6e0 2023-11-26T07:17:51.010 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:17:51.007+0000 7f3c657fa640 1 -- 172.21.15.143:0/345069385 <== mon.0 v2:172.21.15.143:3300/0 4 ==== mgrmap(e 5) v1 ==== 85852+0+0 (secure 0 0 0) 0x7f3c680b4070 con 0x7f3c7015d6e0 2023-11-26T07:17:51.011 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:17:51.007+0000 7f3c657fa640 1 --2- 172.21.15.143:0/345069385 >> [v2:172.21.15.143:6832/2150557544,v1:172.21.15.143:6833/2150557544] conn(0x7f3c5c066470 0x7f3c5c068930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:17:51.011 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:17:51.007+0000 7f3c657fa640 1 -- 172.21.15.143:0/345069385 <== mon.0 v2:172.21.15.143:3300/0 5 ==== osd_map(16..16 src has 1..16) v4 ==== 4857+0+0 (secure 0 0 0) 0x7f3c6811e8d0 con 0x7f3c7015d6e0 2023-11-26T07:17:51.011 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:17:51.007+0000 7f3c74d5d640 1 --2- 172.21.15.143:0/345069385 >> [v2:172.21.15.143:6832/2150557544,v1:172.21.15.143:6833/2150557544] conn(0x7f3c5c066470 0x7f3c5c068930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:17:51.012 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:17:51.007+0000 7f3c74d5d640 1 --2- 172.21.15.143:0/345069385 >> [v2:172.21.15.143:6832/2150557544,v1:172.21.15.143:6833/2150557544] conn(0x7f3c5c066470 0x7f3c5c068930 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7f3c70159120 tx=0x7f3c6c009040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:17:51.015 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:17:51.011+0000 7f3c657fa640 1 -- 172.21.15.143:0/345069385 <== mon.0 v2:172.21.15.143:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172649 (secure 0 0 0) 0x7f3c680ec160 con 0x7f3c7015d6e0 2023-11-26T07:17:51.165 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:17:51.163+0000 7f3c76fc1640 1 -- 172.21.15.143:0/345069385 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f3c70067680 con 0x7f3c7015d6e0 2023-11-26T07:17:51.166 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:17:51.163+0000 7f3c657fa640 1 -- 172.21.15.143:0/345069385 <== mon.0 v2:172.21.15.143:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v16) v1 ==== 93+0+10 (secure 0 0 0) 0x7f3c680ef780 con 0x7f3c7015d6e0 2023-11-26T07:17:51.166 INFO:teuthology.orchestra.run.smithi143.stdout:pg_num: 1 2023-11-26T07:17:51.168 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:17:51.163+0000 7f3c76fc1640 1 -- 172.21.15.143:0/345069385 >> [v2:172.21.15.143:6832/2150557544,v1:172.21.15.143:6833/2150557544] conn(0x7f3c5c066470 msgr2=0x7f3c5c068930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:17:51.168 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:17:51.163+0000 7f3c76fc1640 1 --2- 172.21.15.143:0/345069385 >> [v2:172.21.15.143:6832/2150557544,v1:172.21.15.143:6833/2150557544] conn(0x7f3c5c066470 0x7f3c5c068930 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7f3c70159120 tx=0x7f3c6c009040 comp rx=0 tx=0).stop 2023-11-26T07:17:51.168 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:17:51.163+0000 7f3c76fc1640 1 -- 172.21.15.143:0/345069385 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f3c7015d6e0 msgr2=0x7f3c7006c110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:17:51.168 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:17:51.163+0000 7f3c76fc1640 1 --2- 172.21.15.143:0/345069385 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f3c7015d6e0 0x7f3c7006c110 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7f3c6801a580 tx=0x7f3c680196f0 comp rx=0 tx=0).stop 2023-11-26T07:17:51.168 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:17:51.163+0000 7f3c76fc1640 1 -- 172.21.15.143:0/345069385 shutdown_connections 2023-11-26T07:17:51.168 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:17:51.163+0000 7f3c76fc1640 1 --2- 172.21.15.143:0/345069385 >> [v2:172.21.15.143:6832/2150557544,v1:172.21.15.143:6833/2150557544] conn(0x7f3c5c066470 0x7f3c5c068930 unknown :-1 s=CLOSED pgs=61 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:17:51.168 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:17:51.163+0000 7f3c76fc1640 1 --2- 172.21.15.143:0/345069385 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f3c7015d6e0 0x7f3c7006c110 unknown :-1 s=CLOSED pgs=59 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:17:51.168 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:17:51.163+0000 7f3c76fc1640 1 --2- 172.21.15.143:0/345069385 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f3c700d1bc0 0x7f3c7015e050 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:17:51.168 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:17:51.163+0000 7f3c76fc1640 1 --2- 172.21.15.143:0/345069385 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f3c700b2160 0x7f3c7015bcd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:17:51.168 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:17:51.163+0000 7f3c76fc1640 1 -- 172.21.15.143:0/345069385 >> 172.21.15.143:0/345069385 conn(0x7f3c7009c550 msgr2=0x7f3c700cfd20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:17:51.168 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:17:51.163+0000 7f3c76fc1640 1 -- 172.21.15.143:0/345069385 shutdown_connections 2023-11-26T07:17:51.169 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:17:51.163+0000 7f3c76fc1640 1 -- 172.21.15.143:0/345069385 wait complete. 2023-11-26T07:17:51.187 DEBUG:teuthology.orchestra.run.smithi143:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --format=json-pretty osd lspools 2023-11-26T07:17:51.325 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:17:51.323+0000 7f56cd09f640 1 Processor -- start 2023-11-26T07:17:51.325 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:17:51.323+0000 7f56cd09f640 1 -- start start 2023-11-26T07:17:51.326 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:17:51.323+0000 7f56cd09f640 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7f56c80d1590 0x7f56c80ca170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:17:51.326 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:17:51.323+0000 7f56cd09f640 1 -- --> v1:172.21.15.143:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f56c80cf100 con 0x7f56c815d940 2023-11-26T07:17:51.326 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:17:51.323+0000 7f56cd09f640 1 -- --> v1:172.21.15.188:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f56c80d3c30 con 0x7f56c815d560 2023-11-26T07:17:51.326 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:17:51.323+0000 7f56cd09f640 1 -- --> v2:172.21.15.188:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f56c80d1a50 con 0x7f56c80d1590 2023-11-26T07:17:51.326 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:17:51.323+0000 7f56c77fe640 1 --1- >> v1:172.21.15.143:6789/0 conn(0x7f56c815d940 0x7f56c80d0e80 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.143:6789/0 says I am v1:172.21.15.143:57476/0 (socket says 172.21.15.143:57476) 2023-11-26T07:17:51.326 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:17:51.323+0000 7f56c77fe640 1 -- 172.21.15.143:0/3110856181 learned_addr learned my addr 172.21.15.143:0/3110856181 (peer_addr_for_me v1:172.21.15.143:0/0) 2023-11-26T07:17:51.327 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:17:51.323+0000 7f56cc89e640 1 --2- 172.21.15.143:0/3110856181 >> v2:172.21.15.188:3300/0 conn(0x7f56c80d1590 0x7f56c80ca170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:17:51.328 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:17:51.323+0000 7f56c6ffd640 1 -- 172.21.15.143:0/3110856181 <== mon.0 v1:172.21.15.143:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 600162990 0 0) 0x7f56c80cf100 con 0x7f56c815d940 2023-11-26T07:17:51.328 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:17:51.323+0000 7f56c6ffd640 1 -- 172.21.15.143:0/3110856181 --> v1:172.21.15.143:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f56b0003530 con 0x7f56c815d940 2023-11-26T07:17:51.329 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:17:51.323+0000 7f56c6ffd640 1 -- 172.21.15.143:0/3110856181 <== mon.1 v1:172.21.15.188:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 564274282 0 0) 0x7f56c80d3c30 con 0x7f56c815d560 2023-11-26T07:17:51.329 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:17:51.323+0000 7f56c6ffd640 1 -- 172.21.15.143:0/3110856181 --> v1:172.21.15.188:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f56c80cf100 con 0x7f56c815d560 2023-11-26T07:17:51.329 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:17:51.323+0000 7f56c6ffd640 1 -- 172.21.15.143:0/3110856181 <== mon.0 v1:172.21.15.143:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1800486222 0 0) 0x7f56b0003530 con 0x7f56c815d940 2023-11-26T07:17:51.329 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:17:51.323+0000 7f56c6ffd640 1 -- 172.21.15.143:0/3110856181 >> v2:172.21.15.188:3300/0 conn(0x7f56c80d1590 msgr2=0x7f56c80ca170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-26T07:17:51.329 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:17:51.323+0000 7f56c6ffd640 1 --2- 172.21.15.143:0/3110856181 >> v2:172.21.15.188:3300/0 conn(0x7f56c80d1590 0x7f56c80ca170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:17:51.329 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:17:51.323+0000 7f56c6ffd640 1 -- 172.21.15.143:0/3110856181 >> v1:172.21.15.188:6789/0 conn(0x7f56c815d560 legacy=0x7f56c80ce9f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:17:51.329 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:17:51.323+0000 7f56c6ffd640 1 -- 172.21.15.143:0/3110856181 --> v1:172.21.15.143:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f56c80d1d30 con 0x7f56c815d940 2023-11-26T07:17:51.329 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:17:51.323+0000 7f56c6ffd640 1 -- 172.21.15.143:0/3110856181 <== mon.0 v1:172.21.15.143:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2137882523 0 0) 0x7f56bc001680 con 0x7f56c815d940 2023-11-26T07:17:51.329 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:17:51.323+0000 7f56cc89e640 1 --2- 172.21.15.143:0/3110856181 >> v2:172.21.15.188:3300/0 conn(0x7f56c80d1590 0x7f56c80ca170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-26T07:17:51.329 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:17:51.323+0000 7f56c6ffd640 1 -- 172.21.15.143:0/3110856181 >> v1:172.21.15.143:6789/0 conn(0x7f56c815d940 legacy=0x7f56c80d0e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:17:51.329 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:17:51.323+0000 7f56c6ffd640 1 --2- 172.21.15.143:0/3110856181 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f56b0003b10 0x7f56b0003f10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:17:51.329 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:17:51.323+0000 7f56c6ffd640 1 --2- 172.21.15.143:0/3110856181 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f56b00047d0 0x7f56b0004c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:17:51.329 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:17:51.323+0000 7f56c6ffd640 1 --2- 172.21.15.143:0/3110856181 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f56b0005190 0x7f56b000b990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:17:51.329 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:17:51.323+0000 7f56c6ffd640 1 -- 172.21.15.143:0/3110856181 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f56c80d1a50 con 0x7f56b00047d0 2023-11-26T07:17:51.330 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:17:51.323+0000 7f56c6ffd640 1 -- 172.21.15.143:0/3110856181 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f56b000c010 con 0x7f56b0005190 2023-11-26T07:17:51.330 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:17:51.323+0000 7f56c6ffd640 1 -- 172.21.15.143:0/3110856181 --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f56b000c1d0 con 0x7f56b0003b10 2023-11-26T07:17:51.330 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:17:51.323+0000 7f56c77fe640 1 --2- 172.21.15.143:0/3110856181 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f56b00047d0 0x7f56b0004c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:17:51.330 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:17:51.323+0000 7f56c7fff640 1 --2- 172.21.15.143:0/3110856181 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f56b0003b10 0x7f56b0003f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:17:51.330 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:17:51.323+0000 7f56cc89e640 1 --2- 172.21.15.143:0/3110856181 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f56b0005190 0x7f56b000b990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:17:51.330 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:17:51.323+0000 7f56c77fe640 1 -- 172.21.15.143:0/3110856181 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f56b0003b10 msgr2=0x7f56b0003f10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-26T07:17:51.330 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:17:51.323+0000 7f56c77fe640 1 --2- 172.21.15.143:0/3110856181 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f56b0003b10 0x7f56b0003f10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:17:51.330 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:17:51.323+0000 7f56c77fe640 1 -- 172.21.15.143:0/3110856181 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f56b0005190 msgr2=0x7f56b000b990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-26T07:17:51.330 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:17:51.323+0000 7f56c77fe640 1 --2- 172.21.15.143:0/3110856181 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f56b0005190 0x7f56b000b990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:17:51.330 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:17:51.323+0000 7f56c77fe640 1 -- 172.21.15.143:0/3110856181 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f56b0003530 con 0x7f56b00047d0 2023-11-26T07:17:51.330 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:17:51.323+0000 7f56cc89e640 1 --2- 172.21.15.143:0/3110856181 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f56b0005190 0x7f56b000b990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-26T07:17:51.330 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:17:51.323+0000 7f56cc89e640 1 -- 172.21.15.143:0/3110856181 reap_dead start 2023-11-26T07:17:51.331 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:17:51.327+0000 7f56c77fe640 1 --2- 172.21.15.143:0/3110856181 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f56b00047d0 0x7f56b0004c50 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7f56bc00c160 tx=0x7f56bc01cba0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7f1a16ae1727a490 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:17:51.331 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:17:51.327+0000 7f56c6ffd640 1 -- 172.21.15.143:0/3110856181 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f56bc0058e0 con 0x7f56b00047d0 2023-11-26T07:17:51.331 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:17:51.327+0000 7f56c6ffd640 1 -- 172.21.15.143:0/3110856181 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f56bc005a60 con 0x7f56b00047d0 2023-11-26T07:17:51.332 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:17:51.327+0000 7f56c6ffd640 1 -- 172.21.15.143:0/3110856181 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f56bc005e50 con 0x7f56b00047d0 2023-11-26T07:17:51.332 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:17:51.327+0000 7f56cd09f640 1 -- 172.21.15.143:0/3110856181 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f56b00047d0 msgr2=0x7f56b0004c50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:17:51.332 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:17:51.327+0000 7f56cd09f640 1 --2- 172.21.15.143:0/3110856181 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f56b00047d0 0x7f56b0004c50 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7f56bc00c160 tx=0x7f56bc01cba0 comp rx=0 tx=0).stop 2023-11-26T07:17:51.332 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:17:51.327+0000 7f56cd09f640 1 -- 172.21.15.143:0/3110856181 shutdown_connections 2023-11-26T07:17:51.332 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:17:51.327+0000 7f56cd09f640 1 --2- 172.21.15.143:0/3110856181 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f56b00047d0 0x7f56b0004c50 unknown :-1 s=CLOSED pgs=61 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:17:51.332 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:17:51.327+0000 7f56cd09f640 1 -- 172.21.15.143:0/3110856181 >> 172.21.15.143:0/3110856181 conn(0x7f56c80d30a0 msgr2=0x7f56c80d3480 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:17:51.332 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:17:51.327+0000 7f56cd09f640 1 -- 172.21.15.143:0/3110856181 shutdown_connections 2023-11-26T07:17:51.332 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:17:51.327+0000 7f56cd09f640 1 -- 172.21.15.143:0/3110856181 wait complete. 2023-11-26T07:17:51.333 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:17:51.331+0000 7f56cd09f640 1 Processor -- start 2023-11-26T07:17:51.333 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:17:51.331+0000 7f56cd09f640 1 -- start start 2023-11-26T07:17:51.335 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:17:51.331+0000 7f56cd09f640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f56b00047d0 0x7f56c81e3f60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:17:51.335 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:17:51.331+0000 7f56cd09f640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f56c80b7940 0x7f56c80b7dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:17:51.335 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:17:51.331+0000 7f56cd09f640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f56c815a160 0x7f56c815a5e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:17:51.335 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:17:51.331+0000 7f56cd09f640 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f56c815ab20 con 0x7f56c80b7940 2023-11-26T07:17:51.335 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:17:51.331+0000 7f56cd09f640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f56c80b5cc0 con 0x7f56b00047d0 2023-11-26T07:17:51.335 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:17:51.331+0000 7f56cd09f640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f56c80b5fa0 con 0x7f56c815a160 2023-11-26T07:17:51.335 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:17:51.331+0000 7f56c77fe640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f56c80b7940 0x7f56c80b7dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:17:51.335 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:17:51.331+0000 7f56c77fe640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f56c80b7940 0x7f56c80b7dc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:53094/0 (socket says 172.21.15.143:53094) 2023-11-26T07:17:51.336 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:17:51.331+0000 7f56c77fe640 1 -- 172.21.15.143:0/4165343798 learned_addr learned my addr 172.21.15.143:0/4165343798 (peer_addr_for_me v2:172.21.15.143:0/0) 2023-11-26T07:17:51.336 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:17:51.331+0000 7f56cc89e640 1 --2- 172.21.15.143:0/4165343798 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f56c815a160 0x7f56c815a5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:17:51.336 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:17:51.331+0000 7f56c7fff640 1 --2- 172.21.15.143:0/4165343798 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f56b00047d0 0x7f56c81e3f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:17:51.336 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:17:51.331+0000 7f56c77fe640 1 -- 172.21.15.143:0/4165343798 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f56c815a160 msgr2=0x7f56c815a5e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:17:51.336 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:17:51.331+0000 7f56c77fe640 1 --2- 172.21.15.143:0/4165343798 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f56c815a160 0x7f56c815a5e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:17:51.336 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:17:51.331+0000 7f56c77fe640 1 -- 172.21.15.143:0/4165343798 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f56b00047d0 msgr2=0x7f56c81e3f60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:17:51.336 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:17:51.331+0000 7f56c77fe640 1 --2- 172.21.15.143:0/4165343798 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f56b00047d0 0x7f56c81e3f60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:17:51.336 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:17:51.331+0000 7f56c77fe640 1 -- 172.21.15.143:0/4165343798 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f56c80b6280 con 0x7f56c80b7940 2023-11-26T07:17:51.336 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:17:51.331+0000 7f56c77fe640 1 --2- 172.21.15.143:0/4165343798 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f56c80b7940 0x7f56c80b7dc0 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7f56bc01c9d0 tx=0x7f56b0005800 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:17:51.337 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:17:51.331+0000 7f56c4ff9640 1 -- 172.21.15.143:0/4165343798 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f56bc0a5cb0 con 0x7f56c80b7940 2023-11-26T07:17:51.337 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:17:51.331+0000 7f56c4ff9640 1 -- 172.21.15.143:0/4165343798 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f56bc0a5e10 con 0x7f56c80b7940 2023-11-26T07:17:51.337 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:17:51.331+0000 7f56c4ff9640 1 -- 172.21.15.143:0/4165343798 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f56bc004a10 con 0x7f56c80b7940 2023-11-26T07:17:51.337 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:17:51.331+0000 7f56cd09f640 1 -- 172.21.15.143:0/4165343798 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f56c80b6560 con 0x7f56c80b7940 2023-11-26T07:17:51.337 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:17:51.331+0000 7f56cd09f640 1 -- 172.21.15.143:0/4165343798 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f56c80b6b20 con 0x7f56c80b7940 2023-11-26T07:17:51.337 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:17:51.335+0000 7f56cd09f640 1 -- 172.21.15.143:0/4165343798 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f56c80afc60 con 0x7f56c80b7940 2023-11-26T07:17:51.338 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:17:51.335+0000 7f56c4ff9640 1 -- 172.21.15.143:0/4165343798 <== mon.0 v2:172.21.15.143:3300/0 4 ==== mgrmap(e 5) v1 ==== 85852+0+0 (secure 0 0 0) 0x7f56bc004b70 con 0x7f56c80b7940 2023-11-26T07:17:51.338 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:17:51.335+0000 7f56c4ff9640 1 --2- 172.21.15.143:0/4165343798 >> [v2:172.21.15.143:6832/2150557544,v1:172.21.15.143:6833/2150557544] conn(0x7f56b4066470 0x7f56b4068930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:17:51.338 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:17:51.335+0000 7f56c4ff9640 1 -- 172.21.15.143:0/4165343798 <== mon.0 v2:172.21.15.143:3300/0 5 ==== osd_map(16..16 src has 1..16) v4 ==== 4857+0+0 (secure 0 0 0) 0x7f56b8001d30 con 0x7f56c80b7940 2023-11-26T07:17:51.339 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:17:51.335+0000 7f56c7fff640 1 --2- 172.21.15.143:0/4165343798 >> [v2:172.21.15.143:6832/2150557544,v1:172.21.15.143:6833/2150557544] conn(0x7f56b4066470 0x7f56b4068930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:17:51.339 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:17:51.335+0000 7f56c7fff640 1 --2- 172.21.15.143:0/4165343798 >> [v2:172.21.15.143:6832/2150557544,v1:172.21.15.143:6833/2150557544] conn(0x7f56b4066470 0x7f56b4068930 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7f56b8007860 tx=0x7f56b80077f0 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:17:51.342 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:17:51.339+0000 7f56c4ff9640 1 -- 172.21.15.143:0/4165343798 <== mon.0 v2:172.21.15.143:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172649 (secure 0 0 0) 0x7f56bc0eb310 con 0x7f56c80b7940 2023-11-26T07:17:51.497 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:17:51.495+0000 7f56cd09f640 1 -- 172.21.15.143:0/4165343798 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "osd lspools", "format": "json-pretty"} v 0) v1 -- 0x7f56c8067500 con 0x7f56c80b7940 2023-11-26T07:17:51.497 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:17:51.495+0000 7f56c4ff9640 1 -- 172.21.15.143:0/4165343798 <== mon.0 v2:172.21.15.143:3300/0 7 ==== mon_command_ack([{"prefix": "osd lspools", "format": "json-pretty"}]=0 v16) v1 ==== 84+0+65 (secure 0 0 0) 0x7f56bc0f3e60 con 0x7f56c80b7940 2023-11-26T07:17:51.497 INFO:teuthology.orchestra.run.smithi143.stdout: 2023-11-26T07:17:51.497 INFO:teuthology.orchestra.run.smithi143.stdout:[ 2023-11-26T07:17:51.497 INFO:teuthology.orchestra.run.smithi143.stdout: { 2023-11-26T07:17:51.498 INFO:teuthology.orchestra.run.smithi143.stdout: "poolnum": 1, 2023-11-26T07:17:51.498 INFO:teuthology.orchestra.run.smithi143.stdout: "poolname": ".mgr" 2023-11-26T07:17:51.498 INFO:teuthology.orchestra.run.smithi143.stdout: } 2023-11-26T07:17:51.498 INFO:teuthology.orchestra.run.smithi143.stdout:] 2023-11-26T07:17:51.499 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:17:51.495+0000 7f56cd09f640 1 -- 172.21.15.143:0/4165343798 >> [v2:172.21.15.143:6832/2150557544,v1:172.21.15.143:6833/2150557544] conn(0x7f56b4066470 msgr2=0x7f56b4068930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:17:51.499 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:17:51.495+0000 7f56cd09f640 1 --2- 172.21.15.143:0/4165343798 >> [v2:172.21.15.143:6832/2150557544,v1:172.21.15.143:6833/2150557544] conn(0x7f56b4066470 0x7f56b4068930 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7f56b8007860 tx=0x7f56b80077f0 comp rx=0 tx=0).stop 2023-11-26T07:17:51.500 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:17:51.495+0000 7f56cd09f640 1 -- 172.21.15.143:0/4165343798 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f56c80b7940 msgr2=0x7f56c80b7dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:17:51.500 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:17:51.495+0000 7f56cd09f640 1 --2- 172.21.15.143:0/4165343798 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f56c80b7940 0x7f56c80b7dc0 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7f56bc01c9d0 tx=0x7f56b0005800 comp rx=0 tx=0).stop 2023-11-26T07:17:51.500 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:17:51.495+0000 7f56cd09f640 1 -- 172.21.15.143:0/4165343798 shutdown_connections 2023-11-26T07:17:51.500 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:17:51.495+0000 7f56cd09f640 1 --2- 172.21.15.143:0/4165343798 >> [v2:172.21.15.143:6832/2150557544,v1:172.21.15.143:6833/2150557544] conn(0x7f56b4066470 0x7f56b4068930 unknown :-1 s=CLOSED pgs=62 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:17:51.500 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:17:51.495+0000 7f56cd09f640 1 --2- 172.21.15.143:0/4165343798 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f56c815a160 0x7f56c815a5e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:17:51.500 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:17:51.495+0000 7f56cd09f640 1 --2- 172.21.15.143:0/4165343798 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f56c80b7940 0x7f56c80b7dc0 unknown :-1 s=CLOSED pgs=62 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:17:51.500 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:17:51.495+0000 7f56cd09f640 1 --2- 172.21.15.143:0/4165343798 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f56b00047d0 0x7f56c81e3f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:17:51.500 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:17:51.495+0000 7f56cd09f640 1 -- 172.21.15.143:0/4165343798 >> 172.21.15.143:0/4165343798 conn(0x7f56c80d30a0 msgr2=0x7f56c80cca20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:17:51.500 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:17:51.495+0000 7f56cd09f640 1 -- 172.21.15.143:0/4165343798 shutdown_connections 2023-11-26T07:17:51.500 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:17:51.495+0000 7f56cd09f640 1 -- 172.21.15.143:0/4165343798 wait complete. 2023-11-26T07:17:51.519 DEBUG:tasks.cephfs.filesystem:Creating filesystem 'cephfs' 2023-11-26T07:17:51.519 DEBUG:teuthology.orchestra.run.smithi143:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_metadata --pg_num_min 64 2023-11-26T07:17:51.540 DEBUG:teuthology.orchestra.run.smithi143:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-26T07:17:51.567 DEBUG:teuthology.orchestra.run.smithi188:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-26T07:17:51.661 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:17:51.659+0000 7f74c6157640 1 Processor -- start 2023-11-26T07:17:51.661 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:17:51.659+0000 7f74c6157640 1 -- start start 2023-11-26T07:17:51.662 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:17:51.659+0000 7f74c6157640 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7f74c00cf710 0x7f74c00cd8c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:17:51.662 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:17:51.659+0000 7f74c6157640 1 -- --> v1:172.21.15.143:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f74c00cc730 con 0x7f74c0143ac0 2023-11-26T07:17:51.663 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:17:51.659+0000 7f74c6157640 1 -- --> v1:172.21.15.188:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f74c0070ee0 con 0x7f74c00cf280 2023-11-26T07:17:51.663 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:17:51.659+0000 7f74c6157640 1 -- --> v2:172.21.15.143:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f74c0143ea0 con 0x7f74c00cf710 2023-11-26T07:17:51.663 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:17:51.659+0000 7f74bf7fe640 1 --1- >> v1:172.21.15.143:6789/0 conn(0x7f74c0143ac0 0x7f74c0070370 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.143:6789/0 says I am v1:172.21.15.143:57488/0 (socket says 172.21.15.143:57488) 2023-11-26T07:17:51.663 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:17:51.659+0000 7f74bf7fe640 1 -- 172.21.15.143:0/1119075601 learned_addr learned my addr 172.21.15.143:0/1119075601 (peer_addr_for_me v1:172.21.15.143:0/0) 2023-11-26T07:17:51.663 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:17:51.659+0000 7f74bffff640 1 --2- 172.21.15.143:0/1119075601 >> v2:172.21.15.143:3300/0 conn(0x7f74c00cf710 0x7f74c00cd8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:17:51.663 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:17:51.659+0000 7f74be7fc640 1 -- 172.21.15.143:0/1119075601 <== mon.0 v1:172.21.15.143:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1204646847 0 0) 0x7f74c00cc730 con 0x7f74c0143ac0 2023-11-26T07:17:51.663 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:17:51.659+0000 7f74be7fc640 1 -- 172.21.15.143:0/1119075601 --> v1:172.21.15.143:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f74a8003530 con 0x7f74c0143ac0 2023-11-26T07:17:51.664 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:17:51.659+0000 7f74bffff640 1 -- 172.21.15.143:0/1119075601 >> v1:172.21.15.188:6789/0 conn(0x7f74c00cf280 legacy=0x7f74c00cd1b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-26T07:17:51.664 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:17:51.659+0000 7f74bffff640 1 -- 172.21.15.143:0/1119075601 >> v1:172.21.15.143:6789/0 conn(0x7f74c0143ac0 legacy=0x7f74c0070370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:17:51.664 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:17:51.659+0000 7f74bffff640 1 -- 172.21.15.143:0/1119075601 --> v2:172.21.15.143:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f74c00711c0 con 0x7f74c00cf710 2023-11-26T07:17:51.666 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:17:51.663+0000 7f74bffff640 1 --2- 172.21.15.143:0/1119075601 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f74c00cf710 0x7f74c00cd8c0 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7f74b8009d90 tx=0x7f74b801c910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=348711624cf12727 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:17:51.666 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:17:51.663+0000 7f74be7fc640 1 -- 172.21.15.143:0/1119075601 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f74b80a59d0 con 0x7f74c00cf710 2023-11-26T07:17:51.666 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:17:51.663+0000 7f74be7fc640 1 -- 172.21.15.143:0/1119075601 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f74b80a5b30 con 0x7f74c00cf710 2023-11-26T07:17:51.666 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:17:51.663+0000 7f74be7fc640 1 -- 172.21.15.143:0/1119075601 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f74b80ad440 con 0x7f74c00cf710 2023-11-26T07:17:51.667 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:17:51.663+0000 7f74c6157640 1 -- 172.21.15.143:0/1119075601 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f74c00cf710 msgr2=0x7f74c00cd8c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:17:51.667 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:17:51.663+0000 7f74c6157640 1 --2- 172.21.15.143:0/1119075601 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f74c00cf710 0x7f74c00cd8c0 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7f74b8009d90 tx=0x7f74b801c910 comp rx=0 tx=0).stop 2023-11-26T07:17:51.667 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:17:51.663+0000 7f74c6157640 1 -- 172.21.15.143:0/1119075601 shutdown_connections 2023-11-26T07:17:51.667 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:17:51.663+0000 7f74c6157640 1 --2- 172.21.15.143:0/1119075601 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f74c00cf710 0x7f74c00cd8c0 unknown :-1 s=CLOSED pgs=64 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:17:51.667 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:17:51.663+0000 7f74c6157640 1 -- 172.21.15.143:0/1119075601 >> 172.21.15.143:0/1119075601 conn(0x7f74c006a890 msgr2=0x7f74c006aca0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:17:51.667 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:17:51.663+0000 7f74c6157640 1 -- 172.21.15.143:0/1119075601 shutdown_connections 2023-11-26T07:17:51.667 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:17:51.663+0000 7f74c6157640 1 -- 172.21.15.143:0/1119075601 wait complete. 2023-11-26T07:17:51.668 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:17:51.667+0000 7f74c6157640 1 Processor -- start 2023-11-26T07:17:51.669 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:17:51.667+0000 7f74c6157640 1 -- start start 2023-11-26T07:17:51.670 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:17:51.667+0000 7f74c6157640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f74c00cf280 0x7f74c0133a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:17:51.671 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:17:51.667+0000 7f74c6157640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f74c00cf710 0x7f74c0136c60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:17:51.671 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:17:51.667+0000 7f74c6157640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f74c0143ac0 0x7f74c01371a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:17:51.671 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:17:51.667+0000 7f74c6157640 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f74c01376e0 con 0x7f74c00cf710 2023-11-26T07:17:51.671 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:17:51.667+0000 7f74c6157640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f74c0157520 con 0x7f74c0143ac0 2023-11-26T07:17:51.671 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:17:51.667+0000 7f74c6157640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f74c01400e0 con 0x7f74c00cf280 2023-11-26T07:17:51.671 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:17:51.667+0000 7f74beffd640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f74c00cf710 0x7f74c0136c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:17:51.671 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:17:51.667+0000 7f74beffd640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f74c00cf710 0x7f74c0136c60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:53110/0 (socket says 172.21.15.143:53110) 2023-11-26T07:17:51.671 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:17:51.667+0000 7f74beffd640 1 -- 172.21.15.143:0/3742090982 learned_addr learned my addr 172.21.15.143:0/3742090982 (peer_addr_for_me v2:172.21.15.143:0/0) 2023-11-26T07:17:51.671 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:17:51.667+0000 7f74bf7fe640 1 --2- 172.21.15.143:0/3742090982 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f74c00cf280 0x7f74c0133a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:17:51.671 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:17:51.667+0000 7f74bffff640 1 --2- 172.21.15.143:0/3742090982 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f74c0143ac0 0x7f74c01371a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:17:51.671 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:17:51.667+0000 7f74beffd640 1 -- 172.21.15.143:0/3742090982 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f74c00cf280 msgr2=0x7f74c0133a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:17:51.671 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:17:51.667+0000 7f74beffd640 1 --2- 172.21.15.143:0/3742090982 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f74c00cf280 0x7f74c0133a70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:17:51.671 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:17:51.667+0000 7f74beffd640 1 -- 172.21.15.143:0/3742090982 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f74c0143ac0 msgr2=0x7f74c01371a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:17:51.672 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:17:51.667+0000 7f74beffd640 1 --2- 172.21.15.143:0/3742090982 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f74c0143ac0 0x7f74c01371a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:17:51.672 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:17:51.667+0000 7f74beffd640 1 -- 172.21.15.143:0/3742090982 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f74b8009a70 con 0x7f74c00cf710 2023-11-26T07:17:51.672 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:17:51.667+0000 7f74bffff640 1 --2- 172.21.15.143:0/3742090982 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f74c0143ac0 0x7f74c01371a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-26T07:17:51.672 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:17:51.667+0000 7f74beffd640 1 --2- 172.21.15.143:0/3742090982 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f74c00cf710 0x7f74c0136c60 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7f74b400b410 tx=0x7f74b400b8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:17:51.672 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:17:51.667+0000 7f74a7fff640 1 -- 172.21.15.143:0/3742090982 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f74b4004020 con 0x7f74c00cf710 2023-11-26T07:17:51.673 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:17:51.667+0000 7f74a7fff640 1 -- 172.21.15.143:0/3742090982 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f74b4004180 con 0x7f74c00cf710 2023-11-26T07:17:51.673 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:17:51.667+0000 7f74c6157640 1 -- 172.21.15.143:0/3742090982 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f74c01577e0 con 0x7f74c00cf710 2023-11-26T07:17:51.673 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:17:51.667+0000 7f74a7fff640 1 -- 172.21.15.143:0/3742090982 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f74b4011630 con 0x7f74c00cf710 2023-11-26T07:17:51.673 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:17:51.667+0000 7f74c6157640 1 -- 172.21.15.143:0/3742090982 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f74c0133fb0 con 0x7f74c00cf710 2023-11-26T07:17:51.673 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:17:51.671+0000 7f74c6157640 1 -- 172.21.15.143:0/3742090982 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f74c0067680 con 0x7f74c00cf710 2023-11-26T07:17:51.673 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:17:51.671+0000 7f74a7fff640 1 -- 172.21.15.143:0/3742090982 <== mon.0 v2:172.21.15.143:3300/0 4 ==== mgrmap(e 5) v1 ==== 85852+0+0 (secure 0 0 0) 0x7f74b4011790 con 0x7f74c00cf710 2023-11-26T07:17:51.674 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:17:51.671+0000 7f74a7fff640 1 --2- 172.21.15.143:0/3742090982 >> [v2:172.21.15.143:6832/2150557544,v1:172.21.15.143:6833/2150557544] conn(0x7f74ac066470 0x7f74ac068930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:17:51.674 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:17:51.671+0000 7f74a7fff640 1 -- 172.21.15.143:0/3742090982 <== mon.0 v2:172.21.15.143:3300/0 5 ==== osd_map(16..16 src has 1..16) v4 ==== 4857+0+0 (secure 0 0 0) 0x7f74b4083870 con 0x7f74c00cf710 2023-11-26T07:17:51.674 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:17:51.671+0000 7f74bf7fe640 1 --2- 172.21.15.143:0/3742090982 >> [v2:172.21.15.143:6832/2150557544,v1:172.21.15.143:6833/2150557544] conn(0x7f74ac066470 0x7f74ac068930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:17:51.674 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:17:51.671+0000 7f74bf7fe640 1 --2- 172.21.15.143:0/3742090982 >> [v2:172.21.15.143:6832/2150557544,v1:172.21.15.143:6833/2150557544] conn(0x7f74ac066470 0x7f74ac068930 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7f74b00011f0 tx=0x7f74b0007d40 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:17:51.678 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:17:51.675+0000 7f74a7fff640 1 -- 172.21.15.143:0/3742090982 <== mon.0 v2:172.21.15.143:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172649 (secure 0 0 0) 0x7f74b4050fb0 con 0x7f74c00cf710 2023-11-26T07:17:51.834 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:17:51.831+0000 7f74c6157640 1 -- 172.21.15.143:0/3742090982 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64} v 0) v1 -- 0x7f74c0157c90 con 0x7f74c00cf710 2023-11-26T07:17:52.483 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:17:52.479+0000 7f74a7fff640 1 -- 172.21.15.143:0/3742090982 <== mon.0 v2:172.21.15.143:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]=0 pool 'cephfs_metadata' created v17) v1 ==== 138+0+0 (secure 0 0 0) 0x7f74b4015020 con 0x7f74c00cf710 2023-11-26T07:17:52.483 INFO:teuthology.orchestra.run.smithi143.stderr:pool 'cephfs_metadata' created 2023-11-26T07:17:52.486 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:17:52.483+0000 7f74c6157640 1 -- 172.21.15.143:0/3742090982 >> [v2:172.21.15.143:6832/2150557544,v1:172.21.15.143:6833/2150557544] conn(0x7f74ac066470 msgr2=0x7f74ac068930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:17:52.486 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:17:52.483+0000 7f74c6157640 1 --2- 172.21.15.143:0/3742090982 >> [v2:172.21.15.143:6832/2150557544,v1:172.21.15.143:6833/2150557544] conn(0x7f74ac066470 0x7f74ac068930 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7f74b00011f0 tx=0x7f74b0007d40 comp rx=0 tx=0).stop 2023-11-26T07:17:52.486 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:17:52.483+0000 7f74c6157640 1 -- 172.21.15.143:0/3742090982 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f74c00cf710 msgr2=0x7f74c0136c60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:17:52.487 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:17:52.483+0000 7f74c6157640 1 --2- 172.21.15.143:0/3742090982 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f74c00cf710 0x7f74c0136c60 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7f74b400b410 tx=0x7f74b400b8d0 comp rx=0 tx=0).stop 2023-11-26T07:17:52.487 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:17:52.483+0000 7f74c6157640 1 -- 172.21.15.143:0/3742090982 shutdown_connections 2023-11-26T07:17:52.487 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:17:52.483+0000 7f74c6157640 1 --2- 172.21.15.143:0/3742090982 >> [v2:172.21.15.143:6832/2150557544,v1:172.21.15.143:6833/2150557544] conn(0x7f74ac066470 0x7f74ac068930 unknown :-1 s=CLOSED pgs=63 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:17:52.487 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:17:52.483+0000 7f74c6157640 1 --2- 172.21.15.143:0/3742090982 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f74c0143ac0 0x7f74c01371a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:17:52.487 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:17:52.483+0000 7f74c6157640 1 --2- 172.21.15.143:0/3742090982 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f74c00cf710 0x7f74c0136c60 unknown :-1 s=CLOSED pgs=65 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:17:52.487 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:17:52.483+0000 7f74c6157640 1 --2- 172.21.15.143:0/3742090982 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f74c00cf280 0x7f74c0133a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:17:52.487 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:17:52.483+0000 7f74c6157640 1 -- 172.21.15.143:0/3742090982 >> 172.21.15.143:0/3742090982 conn(0x7f74c006a890 msgr2=0x7f74c00b2280 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:17:52.487 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:17:52.483+0000 7f74c6157640 1 -- 172.21.15.143:0/3742090982 shutdown_connections 2023-11-26T07:17:52.487 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:17:52.483+0000 7f74c6157640 1 -- 172.21.15.143:0/3742090982 wait complete. 2023-11-26T07:17:52.513 DEBUG:teuthology.orchestra.run.smithi143:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_data 64 --pg_num_min 64 --target_size_ratio 0.9 2023-11-26T07:17:52.658 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:17:52.655+0000 7f9f9fd35640 1 Processor -- start 2023-11-26T07:17:52.658 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:17:52.655+0000 7f9f9fd35640 1 -- start start 2023-11-26T07:17:52.660 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:17:52.655+0000 7f9f9fd35640 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7f9f98070390 0x7f9f980b1ef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:17:52.660 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:17:52.655+0000 7f9f9fd35640 1 -- --> v1:172.21.15.143:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9f980b25f0 con 0x7f9f98071550 2023-11-26T07:17:52.660 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:17:52.655+0000 7f9f9fd35640 1 -- --> v1:172.21.15.188:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9f980b28d0 con 0x7f9f98143c00 2023-11-26T07:17:52.660 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:17:52.655+0000 7f9f9fd35640 1 -- --> v2:172.21.15.188:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9f980b2bb0 con 0x7f9f98070390 2023-11-26T07:17:52.660 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:17:52.655+0000 7f9f9e2d2640 1 --1- >> v1:172.21.15.143:6789/0 conn(0x7f9f98071550 0x7f9f980b3690 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.143:6789/0 says I am v1:172.21.15.143:57498/0 (socket says 172.21.15.143:57498) 2023-11-26T07:17:52.660 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:17:52.655+0000 7f9f9e2d2640 1 -- 172.21.15.143:0/466968520 learned_addr learned my addr 172.21.15.143:0/466968520 (peer_addr_for_me v1:172.21.15.143:0/0) 2023-11-26T07:17:52.660 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:17:52.655+0000 7f9f9d2d0640 1 --2- 172.21.15.143:0/466968520 >> v2:172.21.15.188:3300/0 conn(0x7f9f98070390 0x7f9f980b1ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:17:52.661 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:17:52.655+0000 7f9f9cacf640 1 -- 172.21.15.143:0/466968520 <== mon.0 v1:172.21.15.143:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4061302022 0 0) 0x7f9f980b25f0 con 0x7f9f98071550 2023-11-26T07:17:52.661 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:17:52.659+0000 7f9f9cacf640 1 -- 172.21.15.143:0/466968520 --> v1:172.21.15.143:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f9f84005740 con 0x7f9f98071550 2023-11-26T07:17:52.661 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:17:52.659+0000 7f9f9cacf640 1 -- 172.21.15.143:0/466968520 <== mon.1 v1:172.21.15.188:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 369542340 0 0) 0x7f9f980b28d0 con 0x7f9f98143c00 2023-11-26T07:17:52.661 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:17:52.659+0000 7f9f9cacf640 1 -- 172.21.15.143:0/466968520 --> v1:172.21.15.188:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f9f980b25f0 con 0x7f9f98143c00 2023-11-26T07:17:52.661 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:17:52.659+0000 7f9f9d2d0640 1 -- 172.21.15.143:0/466968520 >> v1:172.21.15.188:6789/0 conn(0x7f9f98143c00 legacy=0x7f9f980cc7a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:17:52.661 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:17:52.659+0000 7f9f9d2d0640 1 -- 172.21.15.143:0/466968520 >> v1:172.21.15.143:6789/0 conn(0x7f9f98071550 legacy=0x7f9f980b3690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:17:52.661 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:17:52.659+0000 7f9f9d2d0640 1 -- 172.21.15.143:0/466968520 --> v2:172.21.15.188:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9f980b2e90 con 0x7f9f98070390 2023-11-26T07:17:52.663 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:17:52.659+0000 7f9f9d2d0640 1 --2- 172.21.15.143:0/466968520 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f9f98070390 0x7f9f980b1ef0 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7f9f880097c0 tx=0x7f9f8801c3f0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=42c5bf50fe110b81 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:17:52.664 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:17:52.659+0000 7f9f9cacf640 1 -- 172.21.15.143:0/466968520 <== mon.1 v2:172.21.15.188:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9f880ac640 con 0x7f9f98070390 2023-11-26T07:17:52.664 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:17:52.659+0000 7f9f9cacf640 1 -- 172.21.15.143:0/466968520 <== mon.1 v2:172.21.15.188:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9f880ac7a0 con 0x7f9f98070390 2023-11-26T07:17:52.664 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:17:52.659+0000 7f9f9cacf640 1 -- 172.21.15.143:0/466968520 <== mon.1 v2:172.21.15.188:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9f880acb30 con 0x7f9f98070390 2023-11-26T07:17:52.664 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:17:52.659+0000 7f9f9fd35640 1 -- 172.21.15.143:0/466968520 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f9f98070390 msgr2=0x7f9f980b1ef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:17:52.664 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:17:52.659+0000 7f9f9fd35640 1 --2- 172.21.15.143:0/466968520 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f9f98070390 0x7f9f980b1ef0 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7f9f880097c0 tx=0x7f9f8801c3f0 comp rx=0 tx=0).stop 2023-11-26T07:17:52.665 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:17:52.663+0000 7f9f9fd35640 1 -- 172.21.15.143:0/466968520 shutdown_connections 2023-11-26T07:17:52.665 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:17:52.663+0000 7f9f9fd35640 1 --2- 172.21.15.143:0/466968520 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f9f98070390 0x7f9f980b1ef0 unknown :-1 s=CLOSED pgs=49 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:17:52.665 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:17:52.663+0000 7f9f9fd35640 1 -- 172.21.15.143:0/466968520 >> 172.21.15.143:0/466968520 conn(0x7f9f980b0bf0 msgr2=0x7f9f980b1000 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:17:52.665 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:17:52.663+0000 7f9f9fd35640 1 -- 172.21.15.143:0/466968520 shutdown_connections 2023-11-26T07:17:52.665 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:17:52.663+0000 7f9f9fd35640 1 -- 172.21.15.143:0/466968520 wait complete. 2023-11-26T07:17:52.666 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:17:52.663+0000 7f9f9fd35640 1 Processor -- start 2023-11-26T07:17:52.666 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:17:52.663+0000 7f9f9fd35640 1 -- start start 2023-11-26T07:17:52.668 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:17:52.663+0000 7f9f9fd35640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f9f98070390 0x7f9f98157f40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:17:52.668 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:17:52.663+0000 7f9f9fd35640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f9f98071550 0x7f9f9813d1a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:17:52.668 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:17:52.663+0000 7f9f9fd35640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f9f98143c00 0x7f9f980ad530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:17:52.668 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:17:52.663+0000 7f9f9fd35640 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9f98158480 con 0x7f9f98143c00 2023-11-26T07:17:52.668 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:17:52.663+0000 7f9f9fd35640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9f9813d6e0 con 0x7f9f98070390 2023-11-26T07:17:52.668 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:17:52.663+0000 7f9f9fd35640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9f980ae1a0 con 0x7f9f98071550 2023-11-26T07:17:52.668 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:17:52.663+0000 7f9f9e2d2640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f9f98143c00 0x7f9f980ad530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:17:52.668 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:17:52.663+0000 7f9f9e2d2640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f9f98143c00 0x7f9f980ad530 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:53114/0 (socket says 172.21.15.143:53114) 2023-11-26T07:17:52.668 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:17:52.663+0000 7f9f9e2d2640 1 -- 172.21.15.143:0/2214435580 learned_addr learned my addr 172.21.15.143:0/2214435580 (peer_addr_for_me v2:172.21.15.143:0/0) 2023-11-26T07:17:52.668 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:17:52.663+0000 7f9f9dad1640 1 --2- 172.21.15.143:0/2214435580 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f9f98070390 0x7f9f98157f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:17:52.669 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:17:52.663+0000 7f9f9d2d0640 1 --2- 172.21.15.143:0/2214435580 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f9f98071550 0x7f9f9813d1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:17:52.669 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:17:52.663+0000 7f9f9e2d2640 1 -- 172.21.15.143:0/2214435580 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f9f98071550 msgr2=0x7f9f9813d1a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:17:52.669 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:17:52.663+0000 7f9f9e2d2640 1 --2- 172.21.15.143:0/2214435580 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f9f98071550 0x7f9f9813d1a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:17:52.669 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:17:52.663+0000 7f9f9e2d2640 1 -- 172.21.15.143:0/2214435580 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f9f98070390 msgr2=0x7f9f98157f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:17:52.669 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:17:52.663+0000 7f9f9e2d2640 1 --2- 172.21.15.143:0/2214435580 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f9f98070390 0x7f9f98157f40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:17:52.669 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:17:52.663+0000 7f9f9e2d2640 1 -- 172.21.15.143:0/2214435580 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9f88009420 con 0x7f9f98143c00 2023-11-26T07:17:52.669 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:17:52.663+0000 7f9f9e2d2640 1 --2- 172.21.15.143:0/2214435580 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f9f98143c00 0x7f9f980ad530 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7f9f9000cc00 tx=0x7f9f9000a430 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:17:52.670 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:17:52.663+0000 7f9f8e7fc640 1 -- 172.21.15.143:0/2214435580 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9f9000ce10 con 0x7f9f98143c00 2023-11-26T07:17:52.670 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:17:52.667+0000 7f9f8e7fc640 1 -- 172.21.15.143:0/2214435580 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9f90012400 con 0x7f9f98143c00 2023-11-26T07:17:52.670 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:17:52.667+0000 7f9f8e7fc640 1 -- 172.21.15.143:0/2214435580 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9f90012790 con 0x7f9f98143c00 2023-11-26T07:17:52.670 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:17:52.667+0000 7f9f9fd35640 1 -- 172.21.15.143:0/2214435580 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9f98137dc0 con 0x7f9f98143c00 2023-11-26T07:17:52.670 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:17:52.667+0000 7f9f9fd35640 1 -- 172.21.15.143:0/2214435580 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9f98138380 con 0x7f9f98143c00 2023-11-26T07:17:52.670 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:17:52.667+0000 7f9f9fd35640 1 -- 172.21.15.143:0/2214435580 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9f98067720 con 0x7f9f98143c00 2023-11-26T07:17:52.671 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:17:52.667+0000 7f9f8e7fc640 1 -- 172.21.15.143:0/2214435580 <== mon.0 v2:172.21.15.143:3300/0 4 ==== mgrmap(e 5) v1 ==== 85852+0+0 (secure 0 0 0) 0x7f9f90007070 con 0x7f9f98143c00 2023-11-26T07:17:52.671 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:17:52.667+0000 7f9f8e7fc640 1 --2- 172.21.15.143:0/2214435580 >> [v2:172.21.15.143:6832/2150557544,v1:172.21.15.143:6833/2150557544] conn(0x7f9f7c066470 0x7f9f7c068930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:17:52.671 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:17:52.667+0000 7f9f8e7fc640 1 -- 172.21.15.143:0/2214435580 <== mon.0 v2:172.21.15.143:3300/0 5 ==== osd_map(17..17 src has 1..17) v4 ==== 5227+0+0 (secure 0 0 0) 0x7f9f90083d90 con 0x7f9f98143c00 2023-11-26T07:17:52.671 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:17:52.667+0000 7f9f9dad1640 1 --2- 172.21.15.143:0/2214435580 >> [v2:172.21.15.143:6832/2150557544,v1:172.21.15.143:6833/2150557544] conn(0x7f9f7c066470 0x7f9f7c068930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:17:52.672 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:17:52.671+0000 7f9f9dad1640 1 --2- 172.21.15.143:0/2214435580 >> [v2:172.21.15.143:6832/2150557544,v1:172.21.15.143:6833/2150557544] conn(0x7f9f7c066470 0x7f9f7c068930 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7f9f940011f0 tx=0x7f9f94009040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:17:52.676 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:17:52.671+0000 7f9f8e7fc640 1 -- 172.21.15.143:0/2214435580 <== mon.0 v2:172.21.15.143:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172649 (secure 0 0 0) 0x7f9f90051350 con 0x7f9f98143c00 2023-11-26T07:17:52.828 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:17:52.823+0000 7f9f9fd35640 1 -- 172.21.15.143:0/2214435580 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9} v 0) v1 -- 0x7f9f981589d0 con 0x7f9f98143c00 2023-11-26T07:17:53.491 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:17:53.487+0000 7f9f8e7fc640 1 -- 172.21.15.143:0/2214435580 <== mon.0 v2:172.21.15.143:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]=0 pool 'cephfs_data' created v18) v1 ==== 170+0+0 (secure 0 0 0) 0x7f9f900170d0 con 0x7f9f98143c00 2023-11-26T07:17:53.491 INFO:teuthology.orchestra.run.smithi143.stderr:pool 'cephfs_data' created 2023-11-26T07:17:53.494 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:17:53.491+0000 7f9f9fd35640 1 -- 172.21.15.143:0/2214435580 >> [v2:172.21.15.143:6832/2150557544,v1:172.21.15.143:6833/2150557544] conn(0x7f9f7c066470 msgr2=0x7f9f7c068930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:17:53.494 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:17:53.491+0000 7f9f9fd35640 1 --2- 172.21.15.143:0/2214435580 >> [v2:172.21.15.143:6832/2150557544,v1:172.21.15.143:6833/2150557544] conn(0x7f9f7c066470 0x7f9f7c068930 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7f9f940011f0 tx=0x7f9f94009040 comp rx=0 tx=0).stop 2023-11-26T07:17:53.494 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:17:53.491+0000 7f9f9fd35640 1 -- 172.21.15.143:0/2214435580 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f9f98143c00 msgr2=0x7f9f980ad530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:17:53.494 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:17:53.491+0000 7f9f9fd35640 1 --2- 172.21.15.143:0/2214435580 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f9f98143c00 0x7f9f980ad530 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7f9f9000cc00 tx=0x7f9f9000a430 comp rx=0 tx=0).stop 2023-11-26T07:17:53.495 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:17:53.491+0000 7f9f9fd35640 1 -- 172.21.15.143:0/2214435580 shutdown_connections 2023-11-26T07:17:53.495 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:17:53.491+0000 7f9f9fd35640 1 --2- 172.21.15.143:0/2214435580 >> [v2:172.21.15.143:6832/2150557544,v1:172.21.15.143:6833/2150557544] conn(0x7f9f7c066470 0x7f9f7c068930 unknown :-1 s=CLOSED pgs=64 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:17:53.495 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:17:53.491+0000 7f9f9fd35640 1 --2- 172.21.15.143:0/2214435580 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f9f98143c00 0x7f9f980ad530 unknown :-1 s=CLOSED pgs=67 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:17:53.495 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:17:53.491+0000 7f9f9fd35640 1 --2- 172.21.15.143:0/2214435580 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f9f98071550 0x7f9f9813d1a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:17:53.495 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:17:53.491+0000 7f9f9fd35640 1 --2- 172.21.15.143:0/2214435580 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f9f98070390 0x7f9f98157f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:17:53.495 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:17:53.491+0000 7f9f9fd35640 1 -- 172.21.15.143:0/2214435580 >> 172.21.15.143:0/2214435580 conn(0x7f9f980b0bf0 msgr2=0x7f9f980cd8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:17:53.495 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:17:53.491+0000 7f9f9fd35640 1 -- 172.21.15.143:0/2214435580 shutdown_connections 2023-11-26T07:17:53.496 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:17:53.491+0000 7f9f9fd35640 1 -- 172.21.15.143:0/2214435580 wait complete. 2023-11-26T07:17:53.536 DEBUG:teuthology.orchestra.run.smithi143:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs new cephfs cephfs_metadata cephfs_data 2023-11-26T07:17:53.677 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:17:53.674+0000 7f455f0e3640 1 Processor -- start 2023-11-26T07:17:53.677 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:17:53.674+0000 7f455f0e3640 1 -- start start 2023-11-26T07:17:53.678 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:17:53.674+0000 7f455f0e3640 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7f45580b2f80 0x7f4558071850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:17:53.678 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:17:53.674+0000 7f455f0e3640 1 -- --> v1:172.21.15.143:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f455806c750 con 0x7f45581596d0 2023-11-26T07:17:53.678 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:17:53.674+0000 7f455f0e3640 1 -- --> v1:172.21.15.188:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f45580b3550 con 0x7f45580ce680 2023-11-26T07:17:53.678 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:17:53.674+0000 7f455f0e3640 1 -- --> v2:172.21.15.143:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f45580cbe00 con 0x7f45580b2f80 2023-11-26T07:17:53.679 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:17:53.674+0000 7f455ce7f640 1 --1- >> v1:172.21.15.143:6789/0 conn(0x7f45581596d0 0x7f45580b2870 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.143:6789/0 says I am v1:172.21.15.143:57512/0 (socket says 172.21.15.143:57512) 2023-11-26T07:17:53.679 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:17:53.674+0000 7f455ce7f640 1 -- 172.21.15.143:0/1822623641 learned_addr learned my addr 172.21.15.143:0/1822623641 (peer_addr_for_me v1:172.21.15.143:0/0) 2023-11-26T07:17:53.679 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:17:53.674+0000 7f4557fff640 1 --2- 172.21.15.143:0/1822623641 >> v2:172.21.15.143:3300/0 conn(0x7f45580b2f80 0x7f4558071850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:17:53.679 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:17:53.674+0000 7f45577fe640 1 -- 172.21.15.143:0/1822623641 <== mon.0 v1:172.21.15.143:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2870219493 0 0) 0x7f455806c750 con 0x7f45581596d0 2023-11-26T07:17:53.680 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:17:53.674+0000 7f45577fe640 1 -- 172.21.15.143:0/1822623641 --> v1:172.21.15.143:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f4548003530 con 0x7f45581596d0 2023-11-26T07:17:53.680 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:17:53.674+0000 7f4557fff640 1 -- 172.21.15.143:0/1822623641 >> v1:172.21.15.188:6789/0 conn(0x7f45580ce680 legacy=0x7f455806cbe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:17:53.680 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:17:53.674+0000 7f4557fff640 1 -- 172.21.15.143:0/1822623641 >> v1:172.21.15.143:6789/0 conn(0x7f45581596d0 legacy=0x7f45580b2870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:17:53.680 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:17:53.674+0000 7f4557fff640 1 -- 172.21.15.143:0/1822623641 --> v2:172.21.15.143:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4558071e80 con 0x7f45580b2f80 2023-11-26T07:17:53.681 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:17:53.678+0000 7f4557fff640 1 --2- 172.21.15.143:0/1822623641 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f45580b2f80 0x7f4558071850 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7f45580cbf40 tx=0x7f454401d7c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3dc59a3e790d543a server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:17:53.682 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:17:53.678+0000 7f45577fe640 1 -- 172.21.15.143:0/1822623641 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f45440a76f0 con 0x7f45580b2f80 2023-11-26T07:17:53.682 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:17:53.678+0000 7f45577fe640 1 -- 172.21.15.143:0/1822623641 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f45440a7850 con 0x7f45580b2f80 2023-11-26T07:17:53.682 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:17:53.678+0000 7f45577fe640 1 -- 172.21.15.143:0/1822623641 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f45440a7bc0 con 0x7f45580b2f80 2023-11-26T07:17:53.683 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:17:53.678+0000 7f455f0e3640 1 -- 172.21.15.143:0/1822623641 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f45580b2f80 msgr2=0x7f4558071850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:17:53.683 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:17:53.678+0000 7f455f0e3640 1 --2- 172.21.15.143:0/1822623641 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f45580b2f80 0x7f4558071850 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7f45580cbf40 tx=0x7f454401d7c0 comp rx=0 tx=0).stop 2023-11-26T07:17:53.683 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:17:53.678+0000 7f455f0e3640 1 -- 172.21.15.143:0/1822623641 shutdown_connections 2023-11-26T07:17:53.683 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:17:53.678+0000 7f455f0e3640 1 --2- 172.21.15.143:0/1822623641 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f45580b2f80 0x7f4558071850 unknown :-1 s=CLOSED pgs=69 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:17:53.683 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:17:53.678+0000 7f455f0e3640 1 -- 172.21.15.143:0/1822623641 >> 172.21.15.143:0/1822623641 conn(0x7f455806a890 msgr2=0x7f455806aca0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:17:53.683 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:17:53.678+0000 7f455f0e3640 1 -- 172.21.15.143:0/1822623641 shutdown_connections 2023-11-26T07:17:53.683 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:17:53.678+0000 7f455f0e3640 1 -- 172.21.15.143:0/1822623641 wait complete. 2023-11-26T07:17:53.684 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:17:53.678+0000 7f455f0e3640 1 Processor -- start 2023-11-26T07:17:53.684 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:17:53.682+0000 7f455f0e3640 1 -- start start 2023-11-26T07:17:53.686 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:17:53.682+0000 7f455f0e3640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f45580b2f80 0x7f455814c0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:17:53.686 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:17:53.682+0000 7f455f0e3640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f45580ce680 0x7f4558153180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:17:53.687 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:17:53.682+0000 7f455f0e3640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f45581596d0 0x7f45581376d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:17:53.687 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:17:53.682+0000 7f455f0e3640 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f45581536c0 con 0x7f45580b2f80 2023-11-26T07:17:53.687 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:17:53.682+0000 7f455f0e3640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4558157bc0 con 0x7f45581596d0 2023-11-26T07:17:53.687 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:17:53.682+0000 7f455f0e3640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f455814c720 con 0x7f45580ce680 2023-11-26T07:17:53.687 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:17:53.682+0000 7f455ce7f640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f45580b2f80 0x7f455814c0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:17:53.687 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:17:53.682+0000 7f455ce7f640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f45580b2f80 0x7f455814c0d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:53134/0 (socket says 172.21.15.143:53134) 2023-11-26T07:17:53.687 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:17:53.682+0000 7f455ce7f640 1 -- 172.21.15.143:0/4122341970 learned_addr learned my addr 172.21.15.143:0/4122341970 (peer_addr_for_me v2:172.21.15.143:0/0) 2023-11-26T07:17:53.687 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:17:53.682+0000 7f455ce7f640 1 -- 172.21.15.143:0/4122341970 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f45580ce680 msgr2=0x7f4558153180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:17:53.687 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:17:53.682+0000 7f455ce7f640 1 --2- 172.21.15.143:0/4122341970 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f45580ce680 0x7f4558153180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:17:53.687 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:17:53.682+0000 7f455ce7f640 1 -- 172.21.15.143:0/4122341970 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f45581596d0 msgr2=0x7f45581376d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:17:53.687 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:17:53.682+0000 7f455ce7f640 1 --2- 172.21.15.143:0/4122341970 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f45581596d0 0x7f45581376d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:17:53.688 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:17:53.682+0000 7f455ce7f640 1 -- 172.21.15.143:0/4122341970 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f454400a5c0 con 0x7f45580b2f80 2023-11-26T07:17:53.688 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:17:53.682+0000 7f455ce7f640 1 --2- 172.21.15.143:0/4122341970 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f45580b2f80 0x7f455814c0d0 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7f454c00c450 tx=0x7f454c00c910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:17:53.688 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:17:53.682+0000 7f45557fa640 1 -- 172.21.15.143:0/4122341970 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f454c005cf0 con 0x7f45580b2f80 2023-11-26T07:17:53.688 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:17:53.682+0000 7f45557fa640 1 -- 172.21.15.143:0/4122341970 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f454c005e50 con 0x7f45580b2f80 2023-11-26T07:17:53.688 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:17:53.682+0000 7f455f0e3640 1 -- 172.21.15.143:0/4122341970 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f455814ca00 con 0x7f45580b2f80 2023-11-26T07:17:53.688 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:17:53.682+0000 7f45557fa640 1 -- 172.21.15.143:0/4122341970 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f454c007290 con 0x7f45580b2f80 2023-11-26T07:17:53.688 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:17:53.682+0000 7f455f0e3640 1 -- 172.21.15.143:0/4122341970 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f455814cfc0 con 0x7f45580b2f80 2023-11-26T07:17:53.688 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:17:53.682+0000 7f455f0e3640 1 -- 172.21.15.143:0/4122341970 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f45580709f0 con 0x7f45580b2f80 2023-11-26T07:17:53.689 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:17:53.686+0000 7f45557fa640 1 -- 172.21.15.143:0/4122341970 <== mon.0 v2:172.21.15.143:3300/0 4 ==== mgrmap(e 5) v1 ==== 85852+0+0 (secure 0 0 0) 0x7f454c0073f0 con 0x7f45580b2f80 2023-11-26T07:17:53.690 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:17:53.686+0000 7f45557fa640 1 --2- 172.21.15.143:0/4122341970 >> [v2:172.21.15.143:6832/2150557544,v1:172.21.15.143:6833/2150557544] conn(0x7f453c066470 0x7f453c068930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:17:53.690 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:17:53.686+0000 7f45557fa640 1 -- 172.21.15.143:0/4122341970 <== mon.0 v2:172.21.15.143:3300/0 5 ==== osd_map(18..18 src has 1..18) v4 ==== 5609+0+0 (secure 0 0 0) 0x7f454c083440 con 0x7f45580b2f80 2023-11-26T07:17:53.690 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:17:53.686+0000 7f4557fff640 1 --2- 172.21.15.143:0/4122341970 >> [v2:172.21.15.143:6832/2150557544,v1:172.21.15.143:6833/2150557544] conn(0x7f453c066470 0x7f453c068930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:17:53.690 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:17:53.686+0000 7f4557fff640 1 --2- 172.21.15.143:0/4122341970 >> [v2:172.21.15.143:6832/2150557544,v1:172.21.15.143:6833/2150557544] conn(0x7f453c066470 0x7f453c068930 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7f45440b0000 tx=0x7f4544008040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:17:53.694 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:17:53.690+0000 7f45557fa640 1 -- 172.21.15.143:0/4122341970 <== mon.0 v2:172.21.15.143:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172649 (secure 0 0 0) 0x7f454c050880 con 0x7f45580b2f80 2023-11-26T07:17:53.876 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:17:53.870+0000 7f455f0e3640 1 -- 172.21.15.143:0/4122341970 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"} v 0) v1 -- 0x7f4558067660 con 0x7f45580b2f80 2023-11-26T07:17:54.505 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:17:54.502+0000 7f45557fa640 1 -- 172.21.15.143:0/4122341970 <== mon.0 v2:172.21.15.143:3300/0 7 ==== mon_command_ack([{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]=0 new fs with metadata pool 2 and data pool 3 v4) v1 ==== 172+0+0 (secure 0 0 0) 0x7f454c053ea0 con 0x7f45580b2f80 2023-11-26T07:17:54.506 INFO:teuthology.orchestra.run.smithi143.stderr:new fs with metadata pool 2 and data pool 3 2023-11-26T07:17:54.511 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:17:54.506+0000 7f455f0e3640 1 -- 172.21.15.143:0/4122341970 >> [v2:172.21.15.143:6832/2150557544,v1:172.21.15.143:6833/2150557544] conn(0x7f453c066470 msgr2=0x7f453c068930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:17:54.511 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:17:54.506+0000 7f455f0e3640 1 --2- 172.21.15.143:0/4122341970 >> [v2:172.21.15.143:6832/2150557544,v1:172.21.15.143:6833/2150557544] conn(0x7f453c066470 0x7f453c068930 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7f45440b0000 tx=0x7f4544008040 comp rx=0 tx=0).stop 2023-11-26T07:17:54.511 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:17:54.506+0000 7f455f0e3640 1 -- 172.21.15.143:0/4122341970 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f45580b2f80 msgr2=0x7f455814c0d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:17:54.511 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:17:54.506+0000 7f455f0e3640 1 --2- 172.21.15.143:0/4122341970 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f45580b2f80 0x7f455814c0d0 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7f454c00c450 tx=0x7f454c00c910 comp rx=0 tx=0).stop 2023-11-26T07:17:54.512 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:17:54.506+0000 7f455f0e3640 1 -- 172.21.15.143:0/4122341970 shutdown_connections 2023-11-26T07:17:54.512 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:17:54.506+0000 7f455f0e3640 1 --2- 172.21.15.143:0/4122341970 >> [v2:172.21.15.143:6832/2150557544,v1:172.21.15.143:6833/2150557544] conn(0x7f453c066470 0x7f453c068930 unknown :-1 s=CLOSED pgs=65 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:17:54.512 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:17:54.506+0000 7f455f0e3640 1 --2- 172.21.15.143:0/4122341970 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f45581596d0 0x7f45581376d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:17:54.512 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:17:54.506+0000 7f455f0e3640 1 --2- 172.21.15.143:0/4122341970 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f45580ce680 0x7f4558153180 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:17:54.512 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:17:54.506+0000 7f455f0e3640 1 --2- 172.21.15.143:0/4122341970 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f45580b2f80 0x7f455814c0d0 unknown :-1 s=CLOSED pgs=70 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:17:54.513 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:17:54.506+0000 7f455f0e3640 1 -- 172.21.15.143:0/4122341970 >> 172.21.15.143:0/4122341970 conn(0x7f455806a890 msgr2=0x7f455814f1a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:17:54.513 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:17:54.510+0000 7f455f0e3640 1 -- 172.21.15.143:0/4122341970 shutdown_connections 2023-11-26T07:17:54.513 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:17:54.510+0000 7f455f0e3640 1 -- 172.21.15.143:0/4122341970 wait complete. 2023-11-26T07:17:54.555 DEBUG:teuthology.orchestra.run.smithi143:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-11-26T07:17:54.694 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:17:54.690+0000 7f61c6817640 1 Processor -- start 2023-11-26T07:17:54.694 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:17:54.690+0000 7f61c6817640 1 -- start start 2023-11-26T07:17:54.696 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:17:54.690+0000 7f61c6817640 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7f61c00d1bc0 0x7f61c00d0df0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:17:54.696 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:17:54.690+0000 7f61c6817640 1 -- --> v1:172.21.15.143:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f61c00cf0e0 con 0x7f61c00b2160 2023-11-26T07:17:54.696 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:17:54.690+0000 7f61c6817640 1 -- --> v1:172.21.15.188:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f61c00ceb20 con 0x7f61c015d6e0 2023-11-26T07:17:54.696 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:17:54.690+0000 7f61c6817640 1 -- --> v2:172.21.15.188:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f61c015dac0 con 0x7f61c00d1bc0 2023-11-26T07:17:54.696 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:17:54.690+0000 7f61c4db4640 1 --1- >> v1:172.21.15.143:6789/0 conn(0x7f61c00b2160 0x7f61c00b0750 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.143:6789/0 says I am v1:172.21.15.143:40078/0 (socket says 172.21.15.143:40078) 2023-11-26T07:17:54.696 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:17:54.690+0000 7f61c4db4640 1 -- 172.21.15.143:0/3819040580 learned_addr learned my addr 172.21.15.143:0/3819040580 (peer_addr_for_me v1:172.21.15.143:0/0) 2023-11-26T07:17:54.696 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:17:54.690+0000 7f61bf7fe640 1 --2- 172.21.15.143:0/3819040580 >> v2:172.21.15.188:3300/0 conn(0x7f61c00d1bc0 0x7f61c00d0df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:17:54.697 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:17:54.690+0000 7f61beffd640 1 -- 172.21.15.143:0/3819040580 <== mon.0 v1:172.21.15.143:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2794590042 0 0) 0x7f61c00cf0e0 con 0x7f61c00b2160 2023-11-26T07:17:54.697 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:17:54.690+0000 7f61beffd640 1 -- 172.21.15.143:0/3819040580 --> v1:172.21.15.143:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f61a8003530 con 0x7f61c00b2160 2023-11-26T07:17:54.697 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:17:54.690+0000 7f61beffd640 1 -- 172.21.15.143:0/3819040580 <== mon.1 v1:172.21.15.188:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 351032075 0 0) 0x7f61c00ceb20 con 0x7f61c015d6e0 2023-11-26T07:17:54.697 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:17:54.690+0000 7f61beffd640 1 -- 172.21.15.143:0/3819040580 --> v1:172.21.15.188:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f61c00cf0e0 con 0x7f61c015d6e0 2023-11-26T07:17:54.697 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:17:54.690+0000 7f61bf7fe640 1 -- 172.21.15.143:0/3819040580 >> v1:172.21.15.188:6789/0 conn(0x7f61c015d6e0 legacy=0x7f61c00d14b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:17:54.697 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:17:54.690+0000 7f61bf7fe640 1 -- 172.21.15.143:0/3819040580 >> v1:172.21.15.143:6789/0 conn(0x7f61c00b2160 legacy=0x7f61c00b0750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:17:54.697 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:17:54.690+0000 7f61bf7fe640 1 -- 172.21.15.143:0/3819040580 --> v2:172.21.15.188:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f61c00cee00 con 0x7f61c00d1bc0 2023-11-26T07:17:54.699 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:17:54.694+0000 7f61bf7fe640 1 --2- 172.21.15.143:0/3819040580 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f61c00d1bc0 0x7f61c00d0df0 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7f61b40097c0 tx=0x7f61b401c480 comp rx=0 tx=0).ready entity=mon.1 client_cookie=69897f215dce98af server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:17:54.699 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:17:54.694+0000 7f61beffd640 1 -- 172.21.15.143:0/3819040580 <== mon.1 v2:172.21.15.188:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f61b40ab600 con 0x7f61c00d1bc0 2023-11-26T07:17:54.699 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:17:54.694+0000 7f61beffd640 1 -- 172.21.15.143:0/3819040580 <== mon.1 v2:172.21.15.188:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f61b40a7050 con 0x7f61c00d1bc0 2023-11-26T07:17:54.699 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:17:54.694+0000 7f61beffd640 1 -- 172.21.15.143:0/3819040580 <== mon.1 v2:172.21.15.188:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f61b40ab980 con 0x7f61c00d1bc0 2023-11-26T07:17:54.700 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:17:54.694+0000 7f61c6817640 1 -- 172.21.15.143:0/3819040580 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f61c00d1bc0 msgr2=0x7f61c00d0df0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:17:54.700 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:17:54.694+0000 7f61c6817640 1 --2- 172.21.15.143:0/3819040580 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f61c00d1bc0 0x7f61c00d0df0 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7f61b40097c0 tx=0x7f61b401c480 comp rx=0 tx=0).stop 2023-11-26T07:17:54.700 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:17:54.694+0000 7f61c6817640 1 -- 172.21.15.143:0/3819040580 shutdown_connections 2023-11-26T07:17:54.700 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:17:54.694+0000 7f61c6817640 1 --2- 172.21.15.143:0/3819040580 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f61c00d1bc0 0x7f61c00d0df0 unknown :-1 s=CLOSED pgs=52 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:17:54.700 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:17:54.694+0000 7f61c6817640 1 -- 172.21.15.143:0/3819040580 >> 172.21.15.143:0/3819040580 conn(0x7f61c009c550 msgr2=0x7f61c00d3220 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:17:54.700 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:17:54.694+0000 7f61c6817640 1 -- 172.21.15.143:0/3819040580 shutdown_connections 2023-11-26T07:17:54.700 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:17:54.694+0000 7f61c6817640 1 -- 172.21.15.143:0/3819040580 wait complete. 2023-11-26T07:17:54.701 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:17:54.698+0000 7f61c6817640 1 Processor -- start 2023-11-26T07:17:54.702 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:17:54.698+0000 7f61c6817640 1 -- start start 2023-11-26T07:17:54.704 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:17:54.698+0000 7f61c6817640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f61c00b2160 0x7f61c00c7760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:17:54.704 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:17:54.698+0000 7f61c6817640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f61c00d1bc0 0x7f61c0153e90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:17:54.704 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:17:54.698+0000 7f61c6817640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f61c015d6e0 0x7f61c01543d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:17:54.704 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:17:54.698+0000 7f61c6817640 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f61c0154910 con 0x7f61c00b2160 2023-11-26T07:17:54.704 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:17:54.698+0000 7f61c6817640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f61c00cbcc0 con 0x7f61c00d1bc0 2023-11-26T07:17:54.704 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:17:54.698+0000 7f61c6817640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f61c00cb0b0 con 0x7f61c015d6e0 2023-11-26T07:17:54.704 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:17:54.698+0000 7f61bffff640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f61c00b2160 0x7f61c00c7760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:17:54.704 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:17:54.698+0000 7f61bffff640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f61c00b2160 0x7f61c00c7760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:46156/0 (socket says 172.21.15.143:46156) 2023-11-26T07:17:54.704 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:17:54.698+0000 7f61bf7fe640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f61c00d1bc0 0x7f61c0153e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:17:54.704 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:17:54.698+0000 7f61bffff640 1 -- 172.21.15.143:0/1369169686 learned_addr learned my addr 172.21.15.143:0/1369169686 (peer_addr_for_me v2:172.21.15.143:0/0) 2023-11-26T07:17:54.705 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:17:54.698+0000 7f61c4db4640 1 --2- 172.21.15.143:0/1369169686 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f61c015d6e0 0x7f61c01543d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:17:54.705 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:17:54.698+0000 7f61bffff640 1 -- 172.21.15.143:0/1369169686 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f61c015d6e0 msgr2=0x7f61c01543d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:17:54.705 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:17:54.698+0000 7f61bffff640 1 --2- 172.21.15.143:0/1369169686 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f61c015d6e0 0x7f61c01543d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:17:54.705 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:17:54.698+0000 7f61bffff640 1 -- 172.21.15.143:0/1369169686 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f61c00d1bc0 msgr2=0x7f61c0153e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:17:54.705 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:17:54.698+0000 7f61bffff640 1 --2- 172.21.15.143:0/1369169686 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f61c00d1bc0 0x7f61c0153e90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:17:54.705 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:17:54.698+0000 7f61bffff640 1 -- 172.21.15.143:0/1369169686 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f61b4009420 con 0x7f61c00b2160 2023-11-26T07:17:54.705 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:17:54.698+0000 7f61bffff640 1 --2- 172.21.15.143:0/1369169686 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f61c00b2160 0x7f61c00c7760 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7f61b000c430 tx=0x7f61b000c8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:17:54.706 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:17:54.702+0000 7f61bcff9640 1 -- 172.21.15.143:0/1369169686 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f61b00052d0 con 0x7f61c00b2160 2023-11-26T07:17:54.706 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:17:54.702+0000 7f61bcff9640 1 -- 172.21.15.143:0/1369169686 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f61b0005450 con 0x7f61c00b2160 2023-11-26T07:17:54.706 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:17:54.702+0000 7f61bcff9640 1 -- 172.21.15.143:0/1369169686 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f61b0003840 con 0x7f61c00b2160 2023-11-26T07:17:54.706 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:17:54.702+0000 7f61c6817640 1 -- 172.21.15.143:0/1369169686 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f61c00cb390 con 0x7f61c00b2160 2023-11-26T07:17:54.706 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:17:54.702+0000 7f61c6817640 1 -- 172.21.15.143:0/1369169686 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f61c00cb950 con 0x7f61c00b2160 2023-11-26T07:17:54.706 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:17:54.702+0000 7f61c6817640 1 -- 172.21.15.143:0/1369169686 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f61c0067680 con 0x7f61c00b2160 2023-11-26T07:17:54.706 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:17:54.702+0000 7f61bcff9640 1 -- 172.21.15.143:0/1369169686 <== mon.0 v2:172.21.15.143:3300/0 4 ==== mgrmap(e 5) v1 ==== 85852+0+0 (secure 0 0 0) 0x7f61b00039a0 con 0x7f61c00b2160 2023-11-26T07:17:54.707 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:17:54.702+0000 7f61bcff9640 1 --2- 172.21.15.143:0/1369169686 >> [v2:172.21.15.143:6832/2150557544,v1:172.21.15.143:6833/2150557544] conn(0x7f61ac066470 0x7f61ac068930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:17:54.707 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:17:54.702+0000 7f61bcff9640 1 -- 172.21.15.143:0/1369169686 <== mon.0 v2:172.21.15.143:3300/0 5 ==== osd_map(19..19 src has 1..19) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f61b0083360 con 0x7f61c00b2160 2023-11-26T07:17:54.707 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:17:54.702+0000 7f61bf7fe640 1 --2- 172.21.15.143:0/1369169686 >> [v2:172.21.15.143:6832/2150557544,v1:172.21.15.143:6833/2150557544] conn(0x7f61ac066470 0x7f61ac068930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:17:54.708 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:17:54.702+0000 7f61bf7fe640 1 --2- 172.21.15.143:0/1369169686 >> [v2:172.21.15.143:6832/2150557544,v1:172.21.15.143:6833/2150557544] conn(0x7f61ac066470 0x7f61ac068930 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7f61b40ad000 tx=0x7f61b4008040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:17:54.711 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:17:54.706+0000 7f61bcff9640 1 -- 172.21.15.143:0/1369169686 <== mon.0 v2:172.21.15.143:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172649 (secure 0 0 0) 0x7f61b0050920 con 0x7f61c00b2160 2023-11-26T07:17:54.862 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:17:54.858+0000 7f61c6817640 1 -- 172.21.15.143:0/1369169686 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f61c00b47a0 con 0x7f61c00b2160 2023-11-26T07:17:54.863 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:17:54.858+0000 7f61bcff9640 1 -- 172.21.15.143:0/1369169686 <== mon.0 v2:172.21.15.143:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v19) v1 ==== 74+0+16554 (secure 0 0 0) 0x7f61b0053f40 con 0x7f61c00b2160 2023-11-26T07:17:54.863 INFO:teuthology.orchestra.run.smithi143.stdout: 2023-11-26T07:17:54.864 INFO:teuthology.orchestra.run.smithi143.stdout:{"epoch":19,"fsid":"f5033060-2469-4a07-95fe-2b76f5404bc1","created":"2023-11-26T07:17:28.613663+0000","modified":"2023-11-26T07:17:53.876678+0000","last_up_change":"2023-11-26T07:17:41.157813+0000","last_in_change":"2023-11-26T07:17:31.181658+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-11-26T07:17:41.758597+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2023-11-26T07:17:51.878332+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2023-11-26T07:17:52.871396+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"ebbee2c6-023e-44b9-9f1d-6f7bfd309730","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6817","nonce":639807369},{"type":"v1","addr":"172.21.15.143:6819","nonce":639807369}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6821","nonce":639807369},{"type":"v1","addr":"172.21.15.143:6823","nonce":639807369}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6830","nonce":639807369},{"type":"v1","addr":"172.21.15.143:6831","nonce":639807369}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6825","nonce":639807369},{"type":"v1","addr":"172.21.15.143:6828","nonce":639807369}]},"public_addr":"172.21.15.143:6819/639807369","cluster_addr":"172.21.15.143:6823/639807369","heartbeat_back_addr":"172.21.15.143:6831/639807369","heartbeat_front_addr":"172.21.15.143:6828/639807369","state":["exists","up"]},{"osd":1,"uuid":"fec4b8be-7122-49e9-9a19-5850a0700001","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6816","nonce":1144942530},{"type":"v1","addr":"172.21.15.143:6818","nonce":1144942530}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6820","nonce":1144942530},{"type":"v1","addr":"172.21.15.143:6822","nonce":1144942530}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6827","nonce":1144942530},{"type":"v1","addr":"172.21.15.143:6829","nonce":1144942530}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6824","nonce":1144942530},{"type":"v1","addr":"172.21.15.143:6826","nonce":1144942530}]},"public_addr":"172.21.15.143:6818/1144942530","cluster_addr":"172.21.15.143:6822/1144942530","heartbeat_back_addr":"172.21.15.143:6829/1144942530","heartbeat_front_addr":"172.21.15.143:6826/1144942530","state":["exists","up"]},{"osd":2,"uuid":"2888af5b-f933-46ca-8f95-6dd494ff43f6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6808","nonce":3909095186},{"type":"v1","addr":"172.21.15.143:6809","nonce":3909095186}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6810","nonce":3909095186},{"type":"v1","addr":"172.21.15.143:6811","nonce":3909095186}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6814","nonce":3909095186},{"type":"v1","addr":"172.21.15.143:6815","nonce":3909095186}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6812","nonce":3909095186},{"type":"v1","addr":"172.21.15.143:6813","nonce":3909095186}]},"public_addr":"172.21.15.143:6809/3909095186","cluster_addr":"172.21.15.143:6811/3909095186","heartbeat_back_addr":"172.21.15.143:6815/3909095186","heartbeat_front_addr":"172.21.15.143:6813/3909095186","state":["exists","up"]},{"osd":3,"uuid":"d1f52435-7bc1-4081-9550-9cd1ba55f549","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6800","nonce":3854210991},{"type":"v1","addr":"172.21.15.143:6801","nonce":3854210991}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6802","nonce":3854210991},{"type":"v1","addr":"172.21.15.143:6803","nonce":3854210991}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6806","nonce":3854210991},{"type":"v1","addr":"172.21.15.143:6807","nonce":3854210991}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6804","nonce":3854210991},{"type":"v1","addr":"172.21.15.143:6805","nonce":3854210991}]},"public_addr":"172.21.15.143:6801/3854210991","cluster_addr":"172.21.15.143:6803/3854210991","heartbeat_back_addr":"172.21.15.143:6807/3854210991","heartbeat_front_addr":"172.21.15.143:6805/3854210991","state":["exists","up"]},{"osd":4,"uuid":"e2ae8b6e-6e25-49b2-bcbf-aacce68f06a3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6800","nonce":3924606902},{"type":"v1","addr":"172.21.15.188:6801","nonce":3924606902}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6802","nonce":3924606902},{"type":"v1","addr":"172.21.15.188:6803","nonce":3924606902}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6806","nonce":3924606902},{"type":"v1","addr":"172.21.15.188:6807","nonce":3924606902}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6804","nonce":3924606902},{"type":"v1","addr":"172.21.15.188:6805","nonce":3924606902}]},"public_addr":"172.21.15.188:6801/3924606902","cluster_addr":"172.21.15.188:6803/3924606902","heartbeat_back_addr":"172.21.15.188:6807/3924606902","heartbeat_front_addr":"172.21.15.188:6805/3924606902","state":["exists","up"]},{"osd":5,"uuid":"3cec8a32-7698-4c36-8e89-e4ba11934f33","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6813","nonce":186841544},{"type":"v1","addr":"172.21.15.188:6815","nonce":186841544}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6817","nonce":186841544},{"type":"v1","addr":"172.21.15.188:6818","nonce":186841544}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6822","nonce":186841544},{"type":"v1","addr":"172.21.15.188:6823","nonce":186841544}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6820","nonce":186841544},{"type":"v1","addr":"172.21.15.188:6821","nonce":186841544}]},"public_addr":"172.21.15.188:6815/186841544","cluster_addr":"172.21.15.188:6818/186841544","heartbeat_back_addr":"172.21.15.188:6823/186841544","heartbeat_front_addr":"172.21.15.188:6821/186841544","state":["exists","up"]},{"osd":6,"uuid":"58a65d5d-e090-473a-81b4-ad38e85029f9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6808","nonce":2153499405},{"type":"v1","addr":"172.21.15.188:6809","nonce":2153499405}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6810","nonce":2153499405},{"type":"v1","addr":"172.21.15.188:6811","nonce":2153499405}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6816","nonce":2153499405},{"type":"v1","addr":"172.21.15.188:6819","nonce":2153499405}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6812","nonce":2153499405},{"type":"v1","addr":"172.21.15.188:6814","nonce":2153499405}]},"public_addr":"172.21.15.188:6809/2153499405","cluster_addr":"172.21.15.188:6811/2153499405","heartbeat_back_addr":"172.21.15.188:6819/2153499405","heartbeat_front_addr":"172.21.15.188:6814/2153499405","state":["exists","up"]},{"osd":7,"uuid":"ea5b7f70-3554-4cf8-ac30-f7c59b80678d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6824","nonce":2722096729},{"type":"v1","addr":"172.21.15.188:6825","nonce":2722096729}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6826","nonce":2722096729},{"type":"v1","addr":"172.21.15.188:6827","nonce":2722096729}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6830","nonce":2722096729},{"type":"v1","addr":"172.21.15.188:6831","nonce":2722096729}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6828","nonce":2722096729},{"type":"v1","addr":"172.21.15.188:6829","nonce":2722096729}]},"public_addr":"172.21.15.188:6825/2722096729","cluster_addr":"172.21.15.188:6827/2722096729","heartbeat_back_addr":"172.21.15.188:6831/2722096729","heartbeat_front_addr":"172.21.15.188:6829/2722096729","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-26T07:17:36.995032+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-26T07:17:36.932266+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-26T07:17:36.829304+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-26T07:17:36.804297+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-26T07:17:36.667892+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-26T07:17:36.964868+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-26T07:17:36.827332+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-26T07:17:36.893179+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-11-26T07:17:54.865 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:17:54.862+0000 7f61c6817640 1 -- 172.21.15.143:0/1369169686 >> [v2:172.21.15.143:6832/2150557544,v1:172.21.15.143:6833/2150557544] conn(0x7f61ac066470 msgr2=0x7f61ac068930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:17:54.865 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:17:54.862+0000 7f61c6817640 1 --2- 172.21.15.143:0/1369169686 >> [v2:172.21.15.143:6832/2150557544,v1:172.21.15.143:6833/2150557544] conn(0x7f61ac066470 0x7f61ac068930 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7f61b40ad000 tx=0x7f61b4008040 comp rx=0 tx=0).stop 2023-11-26T07:17:54.866 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:17:54.862+0000 7f61c6817640 1 -- 172.21.15.143:0/1369169686 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f61c00b2160 msgr2=0x7f61c00c7760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:17:54.866 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:17:54.862+0000 7f61c6817640 1 --2- 172.21.15.143:0/1369169686 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f61c00b2160 0x7f61c00c7760 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7f61b000c430 tx=0x7f61b000c8f0 comp rx=0 tx=0).stop 2023-11-26T07:17:54.866 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:17:54.862+0000 7f61c6817640 1 -- 172.21.15.143:0/1369169686 shutdown_connections 2023-11-26T07:17:54.866 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:17:54.862+0000 7f61c6817640 1 --2- 172.21.15.143:0/1369169686 >> [v2:172.21.15.143:6832/2150557544,v1:172.21.15.143:6833/2150557544] conn(0x7f61ac066470 0x7f61ac068930 unknown :-1 s=CLOSED pgs=66 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:17:54.866 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:17:54.862+0000 7f61c6817640 1 --2- 172.21.15.143:0/1369169686 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f61c015d6e0 0x7f61c01543d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:17:54.866 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:17:54.862+0000 7f61c6817640 1 --2- 172.21.15.143:0/1369169686 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f61c00d1bc0 0x7f61c0153e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:17:54.866 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:17:54.862+0000 7f61c6817640 1 --2- 172.21.15.143:0/1369169686 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f61c00b2160 0x7f61c00c7760 unknown :-1 s=CLOSED pgs=72 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:17:54.866 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:17:54.862+0000 7f61c6817640 1 -- 172.21.15.143:0/1369169686 >> 172.21.15.143:0/1369169686 conn(0x7f61c009c550 msgr2=0x7f61c006c820 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:17:54.866 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:17:54.862+0000 7f61c6817640 1 -- 172.21.15.143:0/1369169686 shutdown_connections 2023-11-26T07:17:54.866 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:17:54.862+0000 7f61c6817640 1 -- 172.21.15.143:0/1369169686 wait complete. 2023-11-26T07:17:54.885 DEBUG:teuthology.orchestra.run.smithi143:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-11-26T07:17:55.023 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:17:55.018+0000 7f255492e640 1 Processor -- start 2023-11-26T07:17:55.023 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:17:55.018+0000 7f255492e640 1 -- start start 2023-11-26T07:17:55.025 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:17:55.018+0000 7f255492e640 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7f25500cf990 0x7f255015efa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:17:55.025 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:17:55.018+0000 7f255492e640 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7f255015f4e0 0x7f25500cd7b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:17:55.025 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:17:55.018+0000 7f255492e640 1 -- --> v1:172.21.15.188:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f255006c520 con 0x7f255013e290 2023-11-26T07:17:55.025 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:17:55.018+0000 7f255492e640 1 -- --> v2:172.21.15.143:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f255006c800 con 0x7f25500cf990 2023-11-26T07:17:55.025 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:17:55.018+0000 7f255492e640 1 -- --> v2:172.21.15.188:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f255006cae0 con 0x7f255015f4e0 2023-11-26T07:17:55.025 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:17:55.022+0000 7f2547fff640 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7f25500cf990 0x7f255015efa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:17:55.025 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:17:55.022+0000 7f2547fff640 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7f25500cf990 0x7f255015efa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:46168/0 (socket says 172.21.15.143:46168) 2023-11-26T07:17:55.025 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:17:55.022+0000 7f2547fff640 1 -- 172.21.15.143:0/1505001929 learned_addr learned my addr 172.21.15.143:0/1505001929 (peer_addr_for_me v2:172.21.15.143:0/0) 2023-11-26T07:17:55.025 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:17:55.022+0000 7f254ed9d640 1 --2- 172.21.15.143:0/1505001929 >> v2:172.21.15.188:3300/0 conn(0x7f255015f4e0 0x7f25500cd7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:17:55.025 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:17:55.022+0000 7f2547fff640 1 -- 172.21.15.143:0/1505001929 >> v2:172.21.15.188:3300/0 conn(0x7f255015f4e0 msgr2=0x7f25500cd7b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-26T07:17:55.025 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:17:55.022+0000 7f2547fff640 1 --2- 172.21.15.143:0/1505001929 >> v2:172.21.15.188:3300/0 conn(0x7f255015f4e0 0x7f25500cd7b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:17:55.025 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:17:55.022+0000 7f2547fff640 1 -- 172.21.15.143:0/1505001929 >> v1:172.21.15.188:6789/0 conn(0x7f255013e290 legacy=0x7f25500cf280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:17:55.025 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:17:55.022+0000 7f2547fff640 1 -- 172.21.15.143:0/1505001929 --> v2:172.21.15.143:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f255006cdc0 con 0x7f25500cf990 2023-11-26T07:17:55.028 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:17:55.022+0000 7f2547fff640 1 --2- 172.21.15.143:0/1505001929 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f25500cf990 0x7f255015efa0 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7f2550155cf0 tx=0x7f253801c4d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=208067ac1a2ba05b server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:17:55.028 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:17:55.022+0000 7f254dd9b640 1 -- 172.21.15.143:0/1505001929 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f25380aa650 con 0x7f25500cf990 2023-11-26T07:17:55.028 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:17:55.022+0000 7f254dd9b640 1 -- 172.21.15.143:0/1505001929 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f25380aa7b0 con 0x7f25500cf990 2023-11-26T07:17:55.028 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:17:55.022+0000 7f254dd9b640 1 -- 172.21.15.143:0/1505001929 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f25380aab40 con 0x7f25500cf990 2023-11-26T07:17:55.029 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:17:55.026+0000 7f255492e640 1 -- 172.21.15.143:0/1505001929 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f25500cf990 msgr2=0x7f255015efa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:17:55.030 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:17:55.026+0000 7f255492e640 1 --2- 172.21.15.143:0/1505001929 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f25500cf990 0x7f255015efa0 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7f2550155cf0 tx=0x7f253801c4d0 comp rx=0 tx=0).stop 2023-11-26T07:17:55.030 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:17:55.026+0000 7f255492e640 1 -- 172.21.15.143:0/1505001929 shutdown_connections 2023-11-26T07:17:55.030 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:17:55.026+0000 7f255492e640 1 --2- 172.21.15.143:0/1505001929 >> v2:172.21.15.188:3300/0 conn(0x7f255015f4e0 0x7f25500cd7b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:17:55.030 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:17:55.026+0000 7f255492e640 1 --2- 172.21.15.143:0/1505001929 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f25500cf990 0x7f255015efa0 unknown :-1 s=CLOSED pgs=73 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:17:55.030 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:17:55.026+0000 7f255492e640 1 -- 172.21.15.143:0/1505001929 >> 172.21.15.143:0/1505001929 conn(0x7f255006a890 msgr2=0x7f255006aca0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:17:55.030 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:17:55.026+0000 7f255492e640 1 -- 172.21.15.143:0/1505001929 shutdown_connections 2023-11-26T07:17:55.030 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:17:55.026+0000 7f255492e640 1 -- 172.21.15.143:0/1505001929 wait complete. 2023-11-26T07:17:55.032 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:17:55.026+0000 7f255492e640 1 Processor -- start 2023-11-26T07:17:55.032 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:17:55.026+0000 7f255492e640 1 -- start start 2023-11-26T07:17:55.032 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:17:55.026+0000 7f255492e640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f25500cf990 0x7f25501390a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:17:55.032 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:17:55.026+0000 7f255492e640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f255013e290 0x7f255014fc20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:17:55.032 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:17:55.026+0000 7f255492e640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f255015f4e0 0x7f25501f9dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:17:55.032 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:17:55.026+0000 7f255492e640 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f25500b0140 con 0x7f255013e290 2023-11-26T07:17:55.033 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:17:55.026+0000 7f255492e640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2550150990 con 0x7f255015f4e0 2023-11-26T07:17:55.033 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:17:55.026+0000 7f255492e640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2550150c70 con 0x7f25500cf990 2023-11-26T07:17:55.033 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:17:55.026+0000 7f2547fff640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f255013e290 0x7f255014fc20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:17:55.033 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:17:55.026+0000 7f2547fff640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f255013e290 0x7f255014fc20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:46170/0 (socket says 172.21.15.143:46170) 2023-11-26T07:17:55.033 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:17:55.026+0000 7f2547fff640 1 -- 172.21.15.143:0/2431191351 learned_addr learned my addr 172.21.15.143:0/2431191351 (peer_addr_for_me v2:172.21.15.143:0/0) 2023-11-26T07:17:55.033 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:17:55.026+0000 7f254ed9d640 1 --2- 172.21.15.143:0/2431191351 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f255015f4e0 0x7f25501f9dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:17:55.033 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:17:55.026+0000 7f254e59c640 1 --2- 172.21.15.143:0/2431191351 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f25500cf990 0x7f25501390a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:17:55.033 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:17:55.026+0000 7f2547fff640 1 -- 172.21.15.143:0/2431191351 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f25500cf990 msgr2=0x7f25501390a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:17:55.033 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:17:55.026+0000 7f2547fff640 1 --2- 172.21.15.143:0/2431191351 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f25500cf990 0x7f25501390a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:17:55.033 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:17:55.026+0000 7f2547fff640 1 -- 172.21.15.143:0/2431191351 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f255015f4e0 msgr2=0x7f25501f9dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:17:55.033 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:17:55.026+0000 7f2547fff640 1 --2- 172.21.15.143:0/2431191351 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f255015f4e0 0x7f25501f9dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:17:55.033 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:17:55.026+0000 7f2547fff640 1 -- 172.21.15.143:0/2431191351 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f25380094d0 con 0x7f255013e290 2023-11-26T07:17:55.034 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:17:55.026+0000 7f254ed9d640 1 --2- 172.21.15.143:0/2431191351 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f255015f4e0 0x7f25501f9dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-26T07:17:55.034 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:17:55.030+0000 7f2547fff640 1 --2- 172.21.15.143:0/2431191351 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f255013e290 0x7f255014fc20 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7f25380190e0 tx=0x7f253801ad10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:17:55.034 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:17:55.030+0000 7f2546ffd640 1 -- 172.21.15.143:0/2431191351 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f25380b3800 con 0x7f255013e290 2023-11-26T07:17:55.035 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:17:55.030+0000 7f2546ffd640 1 -- 172.21.15.143:0/2431191351 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f25380b3960 con 0x7f255013e290 2023-11-26T07:17:55.035 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:17:55.030+0000 7f2546ffd640 1 -- 172.21.15.143:0/2431191351 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f25380b3cd0 con 0x7f255013e290 2023-11-26T07:17:55.035 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:17:55.030+0000 7f255492e640 1 -- 172.21.15.143:0/2431191351 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2550150f50 con 0x7f255013e290 2023-11-26T07:17:55.035 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:17:55.030+0000 7f255492e640 1 -- 172.21.15.143:0/2431191351 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2550150560 con 0x7f255013e290 2023-11-26T07:17:55.035 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:17:55.030+0000 7f255492e640 1 -- 172.21.15.143:0/2431191351 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2550150750 con 0x7f255013e290 2023-11-26T07:17:55.035 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:17:55.030+0000 7f2546ffd640 1 -- 172.21.15.143:0/2431191351 <== mon.0 v2:172.21.15.143:3300/0 4 ==== mgrmap(e 5) v1 ==== 85852+0+0 (secure 0 0 0) 0x7f25380b2380 con 0x7f255013e290 2023-11-26T07:17:55.036 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:17:55.030+0000 7f2546ffd640 1 --2- 172.21.15.143:0/2431191351 >> [v2:172.21.15.143:6832/2150557544,v1:172.21.15.143:6833/2150557544] conn(0x7f25300664a0 0x7f2530068960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:17:55.036 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:17:55.030+0000 7f2546ffd640 1 -- 172.21.15.143:0/2431191351 <== mon.0 v2:172.21.15.143:3300/0 5 ==== osd_map(19..19 src has 1..19) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f255015fb00 con 0x7f255013e290 2023-11-26T07:17:55.036 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:17:55.030+0000 7f254e59c640 1 --2- 172.21.15.143:0/2431191351 >> [v2:172.21.15.143:6832/2150557544,v1:172.21.15.143:6833/2150557544] conn(0x7f25300664a0 0x7f2530068960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:17:55.036 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:17:55.034+0000 7f254e59c640 1 --2- 172.21.15.143:0/2431191351 >> [v2:172.21.15.143:6832/2150557544,v1:172.21.15.143:6833/2150557544] conn(0x7f25300664a0 0x7f2530068960 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7f2540000f70 tx=0x7f2540009040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:17:55.040 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:17:55.034+0000 7f2546ffd640 1 -- 172.21.15.143:0/2431191351 <== mon.0 v2:172.21.15.143:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172649 (secure 0 0 0) 0x7f2550150750 con 0x7f255013e290 2023-11-26T07:17:55.191 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:17:55.186+0000 7f255492e640 1 -- 172.21.15.143:0/2431191351 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f2550150750 con 0x7f255013e290 2023-11-26T07:17:55.193 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:17:55.190+0000 7f2546ffd640 1 -- 172.21.15.143:0/2431191351 <== mon.0 v2:172.21.15.143:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v19) v1 ==== 74+0+16554 (secure 0 0 0) 0x7f2550150750 con 0x7f255013e290 2023-11-26T07:17:55.193 INFO:teuthology.orchestra.run.smithi143.stdout: 2023-11-26T07:17:55.193 INFO:teuthology.orchestra.run.smithi143.stdout:{"epoch":19,"fsid":"f5033060-2469-4a07-95fe-2b76f5404bc1","created":"2023-11-26T07:17:28.613663+0000","modified":"2023-11-26T07:17:53.876678+0000","last_up_change":"2023-11-26T07:17:41.157813+0000","last_in_change":"2023-11-26T07:17:31.181658+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-11-26T07:17:41.758597+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2023-11-26T07:17:51.878332+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2023-11-26T07:17:52.871396+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"ebbee2c6-023e-44b9-9f1d-6f7bfd309730","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6817","nonce":639807369},{"type":"v1","addr":"172.21.15.143:6819","nonce":639807369}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6821","nonce":639807369},{"type":"v1","addr":"172.21.15.143:6823","nonce":639807369}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6830","nonce":639807369},{"type":"v1","addr":"172.21.15.143:6831","nonce":639807369}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6825","nonce":639807369},{"type":"v1","addr":"172.21.15.143:6828","nonce":639807369}]},"public_addr":"172.21.15.143:6819/639807369","cluster_addr":"172.21.15.143:6823/639807369","heartbeat_back_addr":"172.21.15.143:6831/639807369","heartbeat_front_addr":"172.21.15.143:6828/639807369","state":["exists","up"]},{"osd":1,"uuid":"fec4b8be-7122-49e9-9a19-5850a0700001","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6816","nonce":1144942530},{"type":"v1","addr":"172.21.15.143:6818","nonce":1144942530}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6820","nonce":1144942530},{"type":"v1","addr":"172.21.15.143:6822","nonce":1144942530}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6827","nonce":1144942530},{"type":"v1","addr":"172.21.15.143:6829","nonce":1144942530}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6824","nonce":1144942530},{"type":"v1","addr":"172.21.15.143:6826","nonce":1144942530}]},"public_addr":"172.21.15.143:6818/1144942530","cluster_addr":"172.21.15.143:6822/1144942530","heartbeat_back_addr":"172.21.15.143:6829/1144942530","heartbeat_front_addr":"172.21.15.143:6826/1144942530","state":["exists","up"]},{"osd":2,"uuid":"2888af5b-f933-46ca-8f95-6dd494ff43f6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6808","nonce":3909095186},{"type":"v1","addr":"172.21.15.143:6809","nonce":3909095186}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6810","nonce":3909095186},{"type":"v1","addr":"172.21.15.143:6811","nonce":3909095186}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6814","nonce":3909095186},{"type":"v1","addr":"172.21.15.143:6815","nonce":3909095186}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6812","nonce":3909095186},{"type":"v1","addr":"172.21.15.143:6813","nonce":3909095186}]},"public_addr":"172.21.15.143:6809/3909095186","cluster_addr":"172.21.15.143:6811/3909095186","heartbeat_back_addr":"172.21.15.143:6815/3909095186","heartbeat_front_addr":"172.21.15.143:6813/3909095186","state":["exists","up"]},{"osd":3,"uuid":"d1f52435-7bc1-4081-9550-9cd1ba55f549","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6800","nonce":3854210991},{"type":"v1","addr":"172.21.15.143:6801","nonce":3854210991}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6802","nonce":3854210991},{"type":"v1","addr":"172.21.15.143:6803","nonce":3854210991}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6806","nonce":3854210991},{"type":"v1","addr":"172.21.15.143:6807","nonce":3854210991}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6804","nonce":3854210991},{"type":"v1","addr":"172.21.15.143:6805","nonce":3854210991}]},"public_addr":"172.21.15.143:6801/3854210991","cluster_addr":"172.21.15.143:6803/3854210991","heartbeat_back_addr":"172.21.15.143:6807/3854210991","heartbeat_front_addr":"172.21.15.143:6805/3854210991","state":["exists","up"]},{"osd":4,"uuid":"e2ae8b6e-6e25-49b2-bcbf-aacce68f06a3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6800","nonce":3924606902},{"type":"v1","addr":"172.21.15.188:6801","nonce":3924606902}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6802","nonce":3924606902},{"type":"v1","addr":"172.21.15.188:6803","nonce":3924606902}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6806","nonce":3924606902},{"type":"v1","addr":"172.21.15.188:6807","nonce":3924606902}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6804","nonce":3924606902},{"type":"v1","addr":"172.21.15.188:6805","nonce":3924606902}]},"public_addr":"172.21.15.188:6801/3924606902","cluster_addr":"172.21.15.188:6803/3924606902","heartbeat_back_addr":"172.21.15.188:6807/3924606902","heartbeat_front_addr":"172.21.15.188:6805/3924606902","state":["exists","up"]},{"osd":5,"uuid":"3cec8a32-7698-4c36-8e89-e4ba11934f33","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6813","nonce":186841544},{"type":"v1","addr":"172.21.15.188:6815","nonce":186841544}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6817","nonce":186841544},{"type":"v1","addr":"172.21.15.188:6818","nonce":186841544}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6822","nonce":186841544},{"type":"v1","addr":"172.21.15.188:6823","nonce":186841544}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6820","nonce":186841544},{"type":"v1","addr":"172.21.15.188:6821","nonce":186841544}]},"public_addr":"172.21.15.188:6815/186841544","cluster_addr":"172.21.15.188:6818/186841544","heartbeat_back_addr":"172.21.15.188:6823/186841544","heartbeat_front_addr":"172.21.15.188:6821/186841544","state":["exists","up"]},{"osd":6,"uuid":"58a65d5d-e090-473a-81b4-ad38e85029f9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6808","nonce":2153499405},{"type":"v1","addr":"172.21.15.188:6809","nonce":2153499405}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6810","nonce":2153499405},{"type":"v1","addr":"172.21.15.188:6811","nonce":2153499405}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6816","nonce":2153499405},{"type":"v1","addr":"172.21.15.188:6819","nonce":2153499405}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6812","nonce":2153499405},{"type":"v1","addr":"172.21.15.188:6814","nonce":2153499405}]},"public_addr":"172.21.15.188:6809/2153499405","cluster_addr":"172.21.15.188:6811/2153499405","heartbeat_back_addr":"172.21.15.188:6819/2153499405","heartbeat_front_addr":"172.21.15.188:6814/2153499405","state":["exists","up"]},{"osd":7,"uuid":"ea5b7f70-3554-4cf8-ac30-f7c59b80678d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6824","nonce":2722096729},{"type":"v1","addr":"172.21.15.188:6825","nonce":2722096729}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6826","nonce":2722096729},{"type":"v1","addr":"172.21.15.188:6827","nonce":2722096729}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6830","nonce":2722096729},{"type":"v1","addr":"172.21.15.188:6831","nonce":2722096729}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6828","nonce":2722096729},{"type":"v1","addr":"172.21.15.188:6829","nonce":2722096729}]},"public_addr":"172.21.15.188:6825/2722096729","cluster_addr":"172.21.15.188:6827/2722096729","heartbeat_back_addr":"172.21.15.188:6831/2722096729","heartbeat_front_addr":"172.21.15.188:6829/2722096729","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-26T07:17:36.995032+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-26T07:17:36.932266+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-26T07:17:36.829304+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-26T07:17:36.804297+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-26T07:17:36.667892+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-26T07:17:36.964868+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-26T07:17:36.827332+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-26T07:17:36.893179+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-11-26T07:17:55.195 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:17:55.190+0000 7f255492e640 1 -- 172.21.15.143:0/2431191351 >> [v2:172.21.15.143:6832/2150557544,v1:172.21.15.143:6833/2150557544] conn(0x7f25300664a0 msgr2=0x7f2530068960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:17:55.195 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:17:55.190+0000 7f255492e640 1 --2- 172.21.15.143:0/2431191351 >> [v2:172.21.15.143:6832/2150557544,v1:172.21.15.143:6833/2150557544] conn(0x7f25300664a0 0x7f2530068960 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7f2540000f70 tx=0x7f2540009040 comp rx=0 tx=0).stop 2023-11-26T07:17:55.195 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:17:55.190+0000 7f255492e640 1 -- 172.21.15.143:0/2431191351 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f255013e290 msgr2=0x7f255014fc20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:17:55.195 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:17:55.190+0000 7f255492e640 1 --2- 172.21.15.143:0/2431191351 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f255013e290 0x7f255014fc20 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7f25380190e0 tx=0x7f253801ad10 comp rx=0 tx=0).stop 2023-11-26T07:17:55.195 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:17:55.190+0000 7f255492e640 1 -- 172.21.15.143:0/2431191351 shutdown_connections 2023-11-26T07:17:55.196 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:17:55.190+0000 7f255492e640 1 --2- 172.21.15.143:0/2431191351 >> [v2:172.21.15.143:6832/2150557544,v1:172.21.15.143:6833/2150557544] conn(0x7f25300664a0 0x7f2530068960 unknown :-1 s=CLOSED pgs=67 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:17:55.196 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:17:55.190+0000 7f255492e640 1 --2- 172.21.15.143:0/2431191351 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f255015f4e0 0x7f25501f9dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:17:55.196 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:17:55.190+0000 7f255492e640 1 --2- 172.21.15.143:0/2431191351 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f255013e290 0x7f255014fc20 unknown :-1 s=CLOSED pgs=74 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:17:55.196 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:17:55.190+0000 7f255492e640 1 --2- 172.21.15.143:0/2431191351 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f25500cf990 0x7f25501390a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:17:55.196 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:17:55.190+0000 7f255492e640 1 -- 172.21.15.143:0/2431191351 >> 172.21.15.143:0/2431191351 conn(0x7f255006a890 msgr2=0x7f25500b25e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:17:55.196 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:17:55.190+0000 7f255492e640 1 -- 172.21.15.143:0/2431191351 shutdown_connections 2023-11-26T07:17:55.197 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:17:55.190+0000 7f255492e640 1 -- 172.21.15.143:0/2431191351 wait complete. 2023-11-26T07:17:55.214 DEBUG:teuthology.orchestra.run.smithi143:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs standby_count_wanted 0 2023-11-26T07:17:55.355 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:17:55.350+0000 7f0a445b8640 1 Processor -- start 2023-11-26T07:17:55.355 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:17:55.350+0000 7f0a445b8640 1 -- start start 2023-11-26T07:17:55.357 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:17:55.354+0000 7f0a445b8640 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7f0a3c143ac0 0x7f0a3c070370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:17:55.357 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:17:55.354+0000 7f0a445b8640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f0a3c0708b0 0x7f0a3c0cf280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:17:55.357 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:17:55.354+0000 7f0a445b8640 1 -- --> v1:172.21.15.188:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0a3c0cfbf0 con 0x7f0a3c0cf7c0 2023-11-26T07:17:55.357 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:17:55.354+0000 7f0a445b8640 1 -- --> v2:172.21.15.143:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0a3c143ea0 con 0x7f0a3c143ac0 2023-11-26T07:17:55.357 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:17:55.354+0000 7f0a445b8640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0a3c0cc730 con 0x7f0a3c0708b0 2023-11-26T07:17:55.357 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:17:55.354+0000 7f0a42354640 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7f0a3c143ac0 0x7f0a3c070370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:17:55.357 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:17:55.354+0000 7f0a42354640 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7f0a3c143ac0 0x7f0a3c070370 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:46182/0 (socket says 172.21.15.143:46182) 2023-11-26T07:17:55.357 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:17:55.354+0000 7f0a42354640 1 -- 172.21.15.143:0/1069645138 learned_addr learned my addr 172.21.15.143:0/1069645138 (peer_addr_for_me v2:172.21.15.143:0/0) 2023-11-26T07:17:55.357 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:17:55.354+0000 7f0a41b53640 1 --2- 172.21.15.143:0/1069645138 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f0a3c0708b0 0x7f0a3c0cf280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:17:55.358 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:17:55.354+0000 7f0a42354640 1 -- 172.21.15.143:0/1069645138 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f0a3c0708b0 msgr2=0x7f0a3c0cf280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-26T07:17:55.358 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:17:55.354+0000 7f0a42354640 1 --2- 172.21.15.143:0/1069645138 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f0a3c0708b0 0x7f0a3c0cf280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:17:55.358 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:17:55.354+0000 7f0a42354640 1 -- 172.21.15.143:0/1069645138 >> v1:172.21.15.188:6789/0 conn(0x7f0a3c0cf7c0 legacy=0x7f0a3c0b1ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:17:55.358 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:17:55.354+0000 7f0a42354640 1 -- 172.21.15.143:0/1069645138 --> v2:172.21.15.143:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0a3c070ee0 con 0x7f0a3c143ac0 2023-11-26T07:17:55.360 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:17:55.354+0000 7f0a42354640 1 --2- 172.21.15.143:0/1069645138 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f0a3c143ac0 0x7f0a3c070370 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7f0a38009870 tx=0x7f0a3801c540 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ad1f42e70a484a8b server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:17:55.360 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:17:55.358+0000 7f0a41352640 1 -- 172.21.15.143:0/1069645138 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0a380ab650 con 0x7f0a3c143ac0 2023-11-26T07:17:55.360 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:17:55.358+0000 7f0a41352640 1 -- 172.21.15.143:0/1069645138 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0a380ab7b0 con 0x7f0a3c143ac0 2023-11-26T07:17:55.361 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:17:55.358+0000 7f0a41352640 1 -- 172.21.15.143:0/1069645138 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0a380abb40 con 0x7f0a3c143ac0 2023-11-26T07:17:55.361 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:17:55.358+0000 7f0a445b8640 1 -- 172.21.15.143:0/1069645138 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f0a3c143ac0 msgr2=0x7f0a3c070370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:17:55.361 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:17:55.358+0000 7f0a445b8640 1 --2- 172.21.15.143:0/1069645138 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f0a3c143ac0 0x7f0a3c070370 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7f0a38009870 tx=0x7f0a3801c540 comp rx=0 tx=0).stop 2023-11-26T07:17:55.361 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:17:55.358+0000 7f0a445b8640 1 -- 172.21.15.143:0/1069645138 shutdown_connections 2023-11-26T07:17:55.361 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:17:55.358+0000 7f0a445b8640 1 --2- 172.21.15.143:0/1069645138 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f0a3c0708b0 0x7f0a3c0cf280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:17:55.362 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:17:55.358+0000 7f0a445b8640 1 --2- 172.21.15.143:0/1069645138 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f0a3c143ac0 0x7f0a3c070370 unknown :-1 s=CLOSED pgs=75 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:17:55.362 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:17:55.358+0000 7f0a445b8640 1 -- 172.21.15.143:0/1069645138 >> 172.21.15.143:0/1069645138 conn(0x7f0a3c06a890 msgr2=0x7f0a3c06aca0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:17:55.362 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:17:55.358+0000 7f0a445b8640 1 -- 172.21.15.143:0/1069645138 shutdown_connections 2023-11-26T07:17:55.362 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:17:55.358+0000 7f0a445b8640 1 -- 172.21.15.143:0/1069645138 wait complete. 2023-11-26T07:17:55.363 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:17:55.358+0000 7f0a445b8640 1 Processor -- start 2023-11-26T07:17:55.363 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:17:55.358+0000 7f0a445b8640 1 -- start start 2023-11-26T07:17:55.365 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:17:55.358+0000 7f0a445b8640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f0a3c0708b0 0x7f0a3c156be0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:17:55.365 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:17:55.358+0000 7f0a445b8640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f0a3c0cf7c0 0x7f0a3c15c0e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:17:55.365 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:17:55.358+0000 7f0a445b8640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f0a3c143ac0 0x7f0a3c15c620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:17:55.365 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:17:55.358+0000 7f0a445b8640 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0a3c13f0a0 con 0x7f0a3c0cf7c0 2023-11-26T07:17:55.365 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:17:55.358+0000 7f0a445b8640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0a3c138640 con 0x7f0a3c143ac0 2023-11-26T07:17:55.366 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:17:55.358+0000 7f0a445b8640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0a3c138920 con 0x7f0a3c0708b0 2023-11-26T07:17:55.366 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:17:55.358+0000 7f0a41b53640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f0a3c0cf7c0 0x7f0a3c15c0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:17:55.366 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:17:55.358+0000 7f0a41b53640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f0a3c0cf7c0 0x7f0a3c15c0e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:46198/0 (socket says 172.21.15.143:46198) 2023-11-26T07:17:55.366 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:17:55.358+0000 7f0a41b53640 1 -- 172.21.15.143:0/3434274859 learned_addr learned my addr 172.21.15.143:0/3434274859 (peer_addr_for_me v2:172.21.15.143:0/0) 2023-11-26T07:17:55.366 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:17:55.362+0000 7f0a42b55640 1 --2- 172.21.15.143:0/3434274859 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f0a3c143ac0 0x7f0a3c15c620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:17:55.366 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:17:55.362+0000 7f0a42354640 1 --2- 172.21.15.143:0/3434274859 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f0a3c0708b0 0x7f0a3c156be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:17:55.366 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:17:55.362+0000 7f0a41b53640 1 -- 172.21.15.143:0/3434274859 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f0a3c0708b0 msgr2=0x7f0a3c156be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:17:55.366 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:17:55.362+0000 7f0a41b53640 1 --2- 172.21.15.143:0/3434274859 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f0a3c0708b0 0x7f0a3c156be0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:17:55.366 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:17:55.362+0000 7f0a41b53640 1 -- 172.21.15.143:0/3434274859 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f0a3c143ac0 msgr2=0x7f0a3c15c620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:17:55.366 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:17:55.362+0000 7f0a41b53640 1 --2- 172.21.15.143:0/3434274859 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f0a3c143ac0 0x7f0a3c15c620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:17:55.366 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:17:55.362+0000 7f0a41b53640 1 -- 172.21.15.143:0/3434274859 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0a380094d0 con 0x7f0a3c0cf7c0 2023-11-26T07:17:55.366 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:17:55.362+0000 7f0a41b53640 1 --2- 172.21.15.143:0/3434274859 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f0a3c0cf7c0 0x7f0a3c15c0e0 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7f0a2c00d800 tx=0x7f0a2c00dcc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:17:55.367 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:17:55.362+0000 7f0a32ffd640 1 -- 172.21.15.143:0/3434274859 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0a2c00b6c0 con 0x7f0a3c0cf7c0 2023-11-26T07:17:55.367 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:17:55.362+0000 7f0a32ffd640 1 -- 172.21.15.143:0/3434274859 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0a2c00b860 con 0x7f0a3c0cf7c0 2023-11-26T07:17:55.367 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:17:55.362+0000 7f0a445b8640 1 -- 172.21.15.143:0/3434274859 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0a3c138c00 con 0x7f0a3c0cf7c0 2023-11-26T07:17:55.367 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:17:55.362+0000 7f0a32ffd640 1 -- 172.21.15.143:0/3434274859 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0a2c00bc10 con 0x7f0a3c0cf7c0 2023-11-26T07:17:55.367 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:17:55.362+0000 7f0a445b8640 1 -- 172.21.15.143:0/3434274859 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0a3c0b19e0 con 0x7f0a3c0cf7c0 2023-11-26T07:17:55.367 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:17:55.362+0000 7f0a445b8640 1 -- 172.21.15.143:0/3434274859 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0a3c139120 con 0x7f0a3c0cf7c0 2023-11-26T07:17:55.367 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:17:55.362+0000 7f0a32ffd640 1 -- 172.21.15.143:0/3434274859 <== mon.0 v2:172.21.15.143:3300/0 4 ==== mgrmap(e 5) v1 ==== 85852+0+0 (secure 0 0 0) 0x7f0a2c01e460 con 0x7f0a3c0cf7c0 2023-11-26T07:17:55.368 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:17:55.362+0000 7f0a32ffd640 1 --2- 172.21.15.143:0/3434274859 >> [v2:172.21.15.143:6832/2150557544,v1:172.21.15.143:6833/2150557544] conn(0x7f0a20066470 0x7f0a20068930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:17:55.368 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:17:55.362+0000 7f0a32ffd640 1 -- 172.21.15.143:0/3434274859 <== mon.0 v2:172.21.15.143:3300/0 5 ==== osd_map(19..19 src has 1..19) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f0a2c084400 con 0x7f0a3c0cf7c0 2023-11-26T07:17:55.368 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:17:55.366+0000 7f0a42354640 1 --2- 172.21.15.143:0/3434274859 >> [v2:172.21.15.143:6832/2150557544,v1:172.21.15.143:6833/2150557544] conn(0x7f0a20066470 0x7f0a20068930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:17:55.369 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:17:55.366+0000 7f0a42354640 1 --2- 172.21.15.143:0/3434274859 >> [v2:172.21.15.143:6832/2150557544,v1:172.21.15.143:6833/2150557544] conn(0x7f0a20066470 0x7f0a20068930 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7f0a380af000 tx=0x7f0a380b6040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:17:55.372 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:17:55.366+0000 7f0a32ffd640 1 -- 172.21.15.143:0/3434274859 <== mon.0 v2:172.21.15.143:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172649 (secure 0 0 0) 0x7f0a2c0518a0 con 0x7f0a3c0cf7c0 2023-11-26T07:17:55.560 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:17:55.554+0000 7f0a445b8640 1 -- 172.21.15.143:0/3434274859 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"} v 0) v1 -- 0x7f0a3c0670b0 con 0x7f0a3c0cf7c0 2023-11-26T07:17:56.519 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:17:56.514+0000 7f0a32ffd640 1 -- 172.21.15.143:0/3434274859 <== mon.0 v2:172.21.15.143:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]=0 v7) v1 ==== 118+0+0 (secure 0 0 0) 0x7f0a2c054ec0 con 0x7f0a3c0cf7c0 2023-11-26T07:17:56.522 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:17:56.518+0000 7f0a445b8640 1 -- 172.21.15.143:0/3434274859 >> [v2:172.21.15.143:6832/2150557544,v1:172.21.15.143:6833/2150557544] conn(0x7f0a20066470 msgr2=0x7f0a20068930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:17:56.522 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:17:56.518+0000 7f0a445b8640 1 --2- 172.21.15.143:0/3434274859 >> [v2:172.21.15.143:6832/2150557544,v1:172.21.15.143:6833/2150557544] conn(0x7f0a20066470 0x7f0a20068930 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7f0a380af000 tx=0x7f0a380b6040 comp rx=0 tx=0).stop 2023-11-26T07:17:56.522 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:17:56.518+0000 7f0a445b8640 1 -- 172.21.15.143:0/3434274859 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f0a3c0cf7c0 msgr2=0x7f0a3c15c0e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:17:56.523 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:17:56.518+0000 7f0a445b8640 1 --2- 172.21.15.143:0/3434274859 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f0a3c0cf7c0 0x7f0a3c15c0e0 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7f0a2c00d800 tx=0x7f0a2c00dcc0 comp rx=0 tx=0).stop 2023-11-26T07:17:56.523 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:17:56.518+0000 7f0a445b8640 1 -- 172.21.15.143:0/3434274859 shutdown_connections 2023-11-26T07:17:56.523 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:17:56.518+0000 7f0a445b8640 1 --2- 172.21.15.143:0/3434274859 >> [v2:172.21.15.143:6832/2150557544,v1:172.21.15.143:6833/2150557544] conn(0x7f0a20066470 0x7f0a20068930 unknown :-1 s=CLOSED pgs=68 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:17:56.523 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:17:56.518+0000 7f0a445b8640 1 --2- 172.21.15.143:0/3434274859 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f0a3c143ac0 0x7f0a3c15c620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:17:56.523 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:17:56.518+0000 7f0a445b8640 1 --2- 172.21.15.143:0/3434274859 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f0a3c0cf7c0 0x7f0a3c15c0e0 unknown :-1 s=CLOSED pgs=76 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:17:56.523 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:17:56.518+0000 7f0a445b8640 1 --2- 172.21.15.143:0/3434274859 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f0a3c0708b0 0x7f0a3c156be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:17:56.523 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:17:56.518+0000 7f0a445b8640 1 -- 172.21.15.143:0/3434274859 >> 172.21.15.143:0/3434274859 conn(0x7f0a3c06a890 msgr2=0x7f0a3c13e880 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:17:56.523 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:17:56.518+0000 7f0a445b8640 1 -- 172.21.15.143:0/3434274859 shutdown_connections 2023-11-26T07:17:56.523 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:17:56.518+0000 7f0a445b8640 1 -- 172.21.15.143:0/3434274859 wait complete. 2023-11-26T07:17:56.543 DEBUG:tasks.cephfs.filesystem:fs_config: {} 2023-11-26T07:17:56.543 DEBUG:teuthology.orchestra.run.smithi143:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs allow_standby_replay false 2023-11-26T07:17:56.686 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:17:56.682+0000 7fdd673e7640 1 Processor -- start 2023-11-26T07:17:56.686 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:17:56.682+0000 7fdd673e7640 1 -- start start 2023-11-26T07:17:56.687 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:17:56.682+0000 7fdd673e7640 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7fdd600cf280 0x7fdd600cf710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:17:56.687 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:17:56.682+0000 7fdd673e7640 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7fdd600cd1b0 0x7fdd600cd5c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:17:56.688 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:17:56.682+0000 7fdd673e7640 1 -- --> v1:172.21.15.188:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fdd600cc730 con 0x7fdd60143ac0 2023-11-26T07:17:56.688 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:17:56.682+0000 7fdd673e7640 1 -- --> v2:172.21.15.143:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fdd60143ea0 con 0x7fdd600cd1b0 2023-11-26T07:17:56.688 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:17:56.682+0000 7fdd673e7640 1 -- --> v2:172.21.15.188:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fdd600cfc50 con 0x7fdd600cf280 2023-11-26T07:17:56.688 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:17:56.682+0000 7fdd65984640 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7fdd600cd1b0 0x7fdd600cd5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:17:56.688 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:17:56.682+0000 7fdd65984640 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7fdd600cd1b0 0x7fdd600cd5c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:46204/0 (socket says 172.21.15.143:46204) 2023-11-26T07:17:56.688 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:17:56.682+0000 7fdd65984640 1 -- 172.21.15.143:0/1371428322 learned_addr learned my addr 172.21.15.143:0/1371428322 (peer_addr_for_me v2:172.21.15.143:0/0) 2023-11-26T07:17:56.688 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:17:56.682+0000 7fdd65984640 1 -- 172.21.15.143:0/1371428322 >> v2:172.21.15.188:3300/0 conn(0x7fdd600cf280 msgr2=0x7fdd600cf710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-26T07:17:56.689 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:17:56.682+0000 7fdd65984640 1 --2- 172.21.15.143:0/1371428322 >> v2:172.21.15.188:3300/0 conn(0x7fdd600cf280 0x7fdd600cf710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:17:56.689 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:17:56.682+0000 7fdd65984640 1 -- 172.21.15.143:0/1371428322 >> v1:172.21.15.188:6789/0 conn(0x7fdd60143ac0 legacy=0x7fdd60070370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-26T07:17:56.689 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:17:56.682+0000 7fdd65984640 1 -- 172.21.15.143:0/1371428322 --> v2:172.21.15.143:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdd60070ee0 con 0x7fdd600cd1b0 2023-11-26T07:17:56.691 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:17:56.686+0000 7fdd65984640 1 --2- 172.21.15.143:0/1371428322 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fdd600cd1b0 0x7fdd600cd5c0 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7fdd600aeb10 tx=0x7fdd5801c990 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a084743cadc740d4 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:17:56.691 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:17:56.686+0000 7fdd57fff640 1 -- 172.21.15.143:0/1371428322 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdd580a5850 con 0x7fdd600cd1b0 2023-11-26T07:17:56.691 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:17:56.686+0000 7fdd57fff640 1 -- 172.21.15.143:0/1371428322 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdd580a59b0 con 0x7fdd600cd1b0 2023-11-26T07:17:56.691 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:17:56.686+0000 7fdd57fff640 1 -- 172.21.15.143:0/1371428322 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdd580a5d40 con 0x7fdd600cd1b0 2023-11-26T07:17:56.692 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:17:56.686+0000 7fdd673e7640 1 -- 172.21.15.143:0/1371428322 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fdd600cd1b0 msgr2=0x7fdd600cd5c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:17:56.692 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:17:56.686+0000 7fdd673e7640 1 --2- 172.21.15.143:0/1371428322 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fdd600cd1b0 0x7fdd600cd5c0 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7fdd600aeb10 tx=0x7fdd5801c990 comp rx=0 tx=0).stop 2023-11-26T07:17:56.692 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:17:56.686+0000 7fdd673e7640 1 -- 172.21.15.143:0/1371428322 shutdown_connections 2023-11-26T07:17:56.692 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:17:56.686+0000 7fdd673e7640 1 --2- 172.21.15.143:0/1371428322 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fdd600cd1b0 0x7fdd600cd5c0 unknown :-1 s=CLOSED pgs=77 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:17:56.692 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:17:56.686+0000 7fdd673e7640 1 --2- 172.21.15.143:0/1371428322 >> v2:172.21.15.188:3300/0 conn(0x7fdd600cf280 0x7fdd600cf710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:17:56.692 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:17:56.686+0000 7fdd673e7640 1 -- 172.21.15.143:0/1371428322 >> 172.21.15.143:0/1371428322 conn(0x7fdd6006a890 msgr2=0x7fdd6006aca0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:17:56.692 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:17:56.686+0000 7fdd673e7640 1 -- 172.21.15.143:0/1371428322 shutdown_connections 2023-11-26T07:17:56.692 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:17:56.686+0000 7fdd673e7640 1 -- 172.21.15.143:0/1371428322 wait complete. 2023-11-26T07:17:56.693 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:17:56.690+0000 7fdd673e7640 1 Processor -- start 2023-11-26T07:17:56.693 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:17:56.690+0000 7fdd673e7640 1 -- start start 2023-11-26T07:17:56.695 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:17:56.690+0000 7fdd673e7640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fdd600cd1b0 0x7fdd60136f90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:17:56.695 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:17:56.690+0000 7fdd673e7640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fdd600cf280 0x7fdd60159b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:17:56.695 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:17:56.690+0000 7fdd673e7640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fdd60143ac0 0x7fdd6006c7b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:17:56.696 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:17:56.690+0000 7fdd673e7640 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdd6015a060 con 0x7fdd600cf280 2023-11-26T07:17:56.696 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:17:56.690+0000 7fdd673e7640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdd60135000 con 0x7fdd60143ac0 2023-11-26T07:17:56.696 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:17:56.690+0000 7fdd673e7640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdd600afc90 con 0x7fdd600cd1b0 2023-11-26T07:17:56.696 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:17:56.690+0000 7fdd64982640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fdd600cf280 0x7fdd60159b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:17:56.696 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:17:56.690+0000 7fdd64982640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fdd600cf280 0x7fdd60159b20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:46220/0 (socket says 172.21.15.143:46220) 2023-11-26T07:17:56.696 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:17:56.690+0000 7fdd64982640 1 -- 172.21.15.143:0/765789361 learned_addr learned my addr 172.21.15.143:0/765789361 (peer_addr_for_me v2:172.21.15.143:0/0) 2023-11-26T07:17:56.696 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:17:56.690+0000 7fdd65984640 1 --2- 172.21.15.143:0/765789361 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fdd60143ac0 0x7fdd6006c7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:17:56.696 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:17:56.690+0000 7fdd65183640 1 --2- 172.21.15.143:0/765789361 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fdd600cd1b0 0x7fdd60136f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:17:56.696 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:17:56.690+0000 7fdd64982640 1 -- 172.21.15.143:0/765789361 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fdd600cd1b0 msgr2=0x7fdd60136f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:17:56.696 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:17:56.690+0000 7fdd64982640 1 --2- 172.21.15.143:0/765789361 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fdd600cd1b0 0x7fdd60136f90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:17:56.696 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:17:56.690+0000 7fdd64982640 1 -- 172.21.15.143:0/765789361 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fdd60143ac0 msgr2=0x7fdd6006c7b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:17:56.696 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:17:56.690+0000 7fdd64982640 1 --2- 172.21.15.143:0/765789361 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fdd60143ac0 0x7fdd6006c7b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:17:56.697 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:17:56.690+0000 7fdd64982640 1 -- 172.21.15.143:0/765789361 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdd58009b30 con 0x7fdd600cf280 2023-11-26T07:17:56.697 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:17:56.690+0000 7fdd65984640 1 --2- 172.21.15.143:0/765789361 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fdd60143ac0 0x7fdd6006c7b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-26T07:17:56.697 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:17:56.690+0000 7fdd64982640 1 --2- 172.21.15.143:0/765789361 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fdd600cf280 0x7fdd60159b20 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7fdd5000b580 tx=0x7fdd5000ba40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:17:56.697 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:17:56.690+0000 7fdd55ffb640 1 -- 172.21.15.143:0/765789361 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdd50010870 con 0x7fdd600cf280 2023-11-26T07:17:56.698 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:17:56.690+0000 7fdd55ffb640 1 -- 172.21.15.143:0/765789361 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdd500109f0 con 0x7fdd600cf280 2023-11-26T07:17:56.698 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:17:56.690+0000 7fdd673e7640 1 -- 172.21.15.143:0/765789361 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdd600b0810 con 0x7fdd600cf280 2023-11-26T07:17:56.698 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:17:56.690+0000 7fdd55ffb640 1 -- 172.21.15.143:0/765789361 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdd50010da0 con 0x7fdd600cf280 2023-11-26T07:17:56.698 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:17:56.690+0000 7fdd673e7640 1 -- 172.21.15.143:0/765789361 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdd600b01c0 con 0x7fdd600cf280 2023-11-26T07:17:56.698 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:17:56.694+0000 7fdd673e7640 1 -- 172.21.15.143:0/765789361 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fdd60067680 con 0x7fdd600cf280 2023-11-26T07:17:56.698 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:17:56.694+0000 7fdd55ffb640 1 -- 172.21.15.143:0/765789361 <== mon.0 v2:172.21.15.143:3300/0 4 ==== mgrmap(e 5) v1 ==== 85852+0+0 (secure 0 0 0) 0x7fdd5000f3c0 con 0x7fdd600cf280 2023-11-26T07:17:56.699 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:17:56.694+0000 7fdd55ffb640 1 --2- 172.21.15.143:0/765789361 >> [v2:172.21.15.143:6832/2150557544,v1:172.21.15.143:6833/2150557544] conn(0x7fdd440664a0 0x7fdd44068960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:17:56.699 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:17:56.694+0000 7fdd55ffb640 1 -- 172.21.15.143:0/765789361 <== mon.0 v2:172.21.15.143:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fdd50083f70 con 0x7fdd600cf280 2023-11-26T07:17:56.699 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:17:56.694+0000 7fdd65183640 1 --2- 172.21.15.143:0/765789361 >> [v2:172.21.15.143:6832/2150557544,v1:172.21.15.143:6833/2150557544] conn(0x7fdd440664a0 0x7fdd44068960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:17:56.699 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:17:56.694+0000 7fdd65183640 1 --2- 172.21.15.143:0/765789361 >> [v2:172.21.15.143:6832/2150557544,v1:172.21.15.143:6833/2150557544] conn(0x7fdd440664a0 0x7fdd44068960 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7fdd60139c40 tx=0x7fdd5c009040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:17:56.702 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:17:56.698+0000 7fdd55ffb640 1 -- 172.21.15.143:0/765789361 <== mon.0 v2:172.21.15.143:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172649 (secure 0 0 0) 0x7fdd50051360 con 0x7fdd600cf280 2023-11-26T07:17:56.885 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:17:56.882+0000 7fdd673e7640 1 -- 172.21.15.143:0/765789361 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"} v 0) v1 -- 0x7fdd600cfa10 con 0x7fdd600cf280 2023-11-26T07:17:57.527 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:17:57.522+0000 7fdd55ffb640 1 -- 172.21.15.143:0/765789361 <== mon.0 v2:172.21.15.143:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"}]=0 v8) v1 ==== 122+0+0 (secure 0 0 0) 0x7fdd50054980 con 0x7fdd600cf280 2023-11-26T07:17:57.530 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:17:57.526+0000 7fdd673e7640 1 -- 172.21.15.143:0/765789361 >> [v2:172.21.15.143:6832/2150557544,v1:172.21.15.143:6833/2150557544] conn(0x7fdd440664a0 msgr2=0x7fdd44068960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:17:57.530 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:17:57.526+0000 7fdd673e7640 1 --2- 172.21.15.143:0/765789361 >> [v2:172.21.15.143:6832/2150557544,v1:172.21.15.143:6833/2150557544] conn(0x7fdd440664a0 0x7fdd44068960 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7fdd60139c40 tx=0x7fdd5c009040 comp rx=0 tx=0).stop 2023-11-26T07:17:57.530 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:17:57.526+0000 7fdd673e7640 1 -- 172.21.15.143:0/765789361 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fdd600cf280 msgr2=0x7fdd60159b20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:17:57.530 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:17:57.526+0000 7fdd673e7640 1 --2- 172.21.15.143:0/765789361 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fdd600cf280 0x7fdd60159b20 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7fdd5000b580 tx=0x7fdd5000ba40 comp rx=0 tx=0).stop 2023-11-26T07:17:57.530 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:17:57.526+0000 7fdd673e7640 1 -- 172.21.15.143:0/765789361 shutdown_connections 2023-11-26T07:17:57.530 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:17:57.526+0000 7fdd673e7640 1 --2- 172.21.15.143:0/765789361 >> [v2:172.21.15.143:6832/2150557544,v1:172.21.15.143:6833/2150557544] conn(0x7fdd440664a0 0x7fdd44068960 unknown :-1 s=CLOSED pgs=69 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:17:57.530 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:17:57.526+0000 7fdd673e7640 1 --2- 172.21.15.143:0/765789361 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fdd60143ac0 0x7fdd6006c7b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:17:57.530 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:17:57.526+0000 7fdd673e7640 1 --2- 172.21.15.143:0/765789361 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fdd600cf280 0x7fdd60159b20 unknown :-1 s=CLOSED pgs=78 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:17:57.530 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:17:57.526+0000 7fdd673e7640 1 --2- 172.21.15.143:0/765789361 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fdd600cd1b0 0x7fdd60136f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:17:57.530 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:17:57.526+0000 7fdd673e7640 1 -- 172.21.15.143:0/765789361 >> 172.21.15.143:0/765789361 conn(0x7fdd6006a890 msgr2=0x7fdd600cb0e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:17:57.530 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:17:57.526+0000 7fdd673e7640 1 -- 172.21.15.143:0/765789361 shutdown_connections 2023-11-26T07:17:57.531 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:17:57.526+0000 7fdd673e7640 1 -- 172.21.15.143:0/765789361 wait complete. 2023-11-26T07:17:57.551 DEBUG:tasks.cephfs.filesystem:Not Creating any subvols for filesystem 'cephfs' 2023-11-26T07:17:57.551 DEBUG:teuthology.orchestra.run.smithi143:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-11-26T07:17:57.690 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:17:57.686+0000 7f0046aba640 1 Processor -- start 2023-11-26T07:17:57.690 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:17:57.686+0000 7f0046aba640 1 -- start start 2023-11-26T07:17:57.692 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:17:57.686+0000 7f0046aba640 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7f0040153b90 0x7f004015a600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:17:57.692 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:17:57.686+0000 7f0046aba640 1 -- --> v1:172.21.15.143:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f00400b1e10 con 0x7f00400b2b40 2023-11-26T07:17:57.692 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:17:57.686+0000 7f0046aba640 1 -- --> v1:172.21.15.188:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f004015ab40 con 0x7f004006d820 2023-11-26T07:17:57.692 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:17:57.686+0000 7f0046aba640 1 -- --> v2:172.21.15.143:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f00400b4160 con 0x7f0040153b90 2023-11-26T07:17:57.692 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:17:57.686+0000 7f0045ab8640 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7f0040153b90 0x7f004015a600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:17:57.692 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:17:57.686+0000 7f00452b7640 1 --1- >> v1:172.21.15.143:6789/0 conn(0x7f00400b2b40 0x7f004006d110 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.143:6789/0 says I am v1:172.21.15.143:40092/0 (socket says 172.21.15.143:40092) 2023-11-26T07:17:57.692 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:17:57.686+0000 7f0045ab8640 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7f0040153b90 0x7f004015a600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:46232/0 (socket says 172.21.15.143:46232) 2023-11-26T07:17:57.692 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:17:57.686+0000 7f00452b7640 1 -- 172.21.15.143:0/865390105 learned_addr learned my addr 172.21.15.143:0/865390105 (peer_addr_for_me v1:172.21.15.143:0/0) 2023-11-26T07:17:57.693 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:17:57.690+0000 7f0045ab8640 1 -- 172.21.15.143:0/865390105 >> v1:172.21.15.188:6789/0 conn(0x7f004006d820 legacy=0x7f00400ccc80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-26T07:17:57.693 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:17:57.690+0000 7f0045ab8640 1 -- 172.21.15.143:0/865390105 >> v1:172.21.15.143:6789/0 conn(0x7f00400b2b40 legacy=0x7f004006d110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:17:57.693 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:17:57.690+0000 7f0045ab8640 1 -- 172.21.15.143:0/865390105 --> v2:172.21.15.143:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f00400cea00 con 0x7f0040153b90 2023-11-26T07:17:57.693 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:17:57.690+0000 7f0044ab6640 1 -- 172.21.15.143:0/865390105 <== mon.0 v1:172.21.15.143:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 453420453 0 0) 0x7f00400b1e10 con 0x7f00400b2b40 2023-11-26T07:17:57.695 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:17:57.690+0000 7f0045ab8640 1 --2- 172.21.15.143:0/865390105 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f0040153b90 0x7f004015a600 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7f003c0097c0 tx=0x7f003c01c410 comp rx=0 tx=0).ready entity=mon.0 client_cookie=43804c7faa3ca731 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:17:57.695 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:17:57.690+0000 7f0044ab6640 1 -- 172.21.15.143:0/865390105 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f003c0a8050 con 0x7f0040153b90 2023-11-26T07:17:57.696 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:17:57.690+0000 7f0044ab6640 1 -- 172.21.15.143:0/865390105 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f003c0ac5f0 con 0x7f0040153b90 2023-11-26T07:17:57.696 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:17:57.690+0000 7f0044ab6640 1 -- 172.21.15.143:0/865390105 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f003c0ac980 con 0x7f0040153b90 2023-11-26T07:17:57.696 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:17:57.690+0000 7f0046aba640 1 -- 172.21.15.143:0/865390105 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f0040153b90 msgr2=0x7f004015a600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:17:57.696 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:17:57.690+0000 7f0046aba640 1 --2- 172.21.15.143:0/865390105 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f0040153b90 0x7f004015a600 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7f003c0097c0 tx=0x7f003c01c410 comp rx=0 tx=0).stop 2023-11-26T07:17:57.697 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:17:57.690+0000 7f0046aba640 1 -- 172.21.15.143:0/865390105 shutdown_connections 2023-11-26T07:17:57.697 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:17:57.690+0000 7f0046aba640 1 --2- 172.21.15.143:0/865390105 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f0040153b90 0x7f004015a600 unknown :-1 s=CLOSED pgs=80 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:17:57.697 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:17:57.690+0000 7f0046aba640 1 -- 172.21.15.143:0/865390105 >> 172.21.15.143:0/865390105 conn(0x7f004006a890 msgr2=0x7f004006aca0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:17:57.697 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:17:57.690+0000 7f0046aba640 1 -- 172.21.15.143:0/865390105 shutdown_connections 2023-11-26T07:17:57.697 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:17:57.690+0000 7f0046aba640 1 -- 172.21.15.143:0/865390105 wait complete. 2023-11-26T07:17:57.698 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:17:57.694+0000 7f0046aba640 1 Processor -- start 2023-11-26T07:17:57.698 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:17:57.694+0000 7f0046aba640 1 -- start start 2023-11-26T07:17:57.700 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:17:57.694+0000 7f0046aba640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f004006d820 0x7f004014b820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:17:57.700 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:17:57.694+0000 7f0046aba640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f00400b2b40 0x7f00401577a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:17:57.700 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:17:57.694+0000 7f0046aba640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f0040153b90 0x7f0040167530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:17:57.700 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:17:57.694+0000 7f0046aba640 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f004014bd60 con 0x7f0040153b90 2023-11-26T07:17:57.700 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:17:57.694+0000 7f0046aba640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0040158260 con 0x7f004006d820 2023-11-26T07:17:57.700 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:17:57.694+0000 7f0046aba640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0040158540 con 0x7f00400b2b40 2023-11-26T07:17:57.701 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:17:57.694+0000 7f00462b9640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f0040153b90 0x7f0040167530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:17:57.701 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:17:57.694+0000 7f00462b9640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f0040153b90 0x7f0040167530 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:46242/0 (socket says 172.21.15.143:46242) 2023-11-26T07:17:57.701 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:17:57.694+0000 7f00462b9640 1 -- 172.21.15.143:0/3439641727 learned_addr learned my addr 172.21.15.143:0/3439641727 (peer_addr_for_me v2:172.21.15.143:0/0) 2023-11-26T07:17:57.701 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:17:57.694+0000 7f0045ab8640 1 --2- 172.21.15.143:0/3439641727 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f004006d820 0x7f004014b820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:17:57.701 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:17:57.694+0000 7f00452b7640 1 --2- 172.21.15.143:0/3439641727 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f00400b2b40 0x7f00401577a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:17:57.701 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:17:57.694+0000 7f00462b9640 1 -- 172.21.15.143:0/3439641727 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f00400b2b40 msgr2=0x7f00401577a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:17:57.701 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:17:57.694+0000 7f00462b9640 1 --2- 172.21.15.143:0/3439641727 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f00400b2b40 0x7f00401577a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:17:57.701 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:17:57.694+0000 7f00462b9640 1 -- 172.21.15.143:0/3439641727 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f004006d820 msgr2=0x7f004014b820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:17:57.701 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:17:57.694+0000 7f00462b9640 1 --2- 172.21.15.143:0/3439641727 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f004006d820 0x7f004014b820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:17:57.701 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:17:57.694+0000 7f00462b9640 1 -- 172.21.15.143:0/3439641727 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f003c009420 con 0x7f0040153b90 2023-11-26T07:17:57.701 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:17:57.694+0000 7f00462b9640 1 --2- 172.21.15.143:0/3439641727 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f0040153b90 0x7f0040167530 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7f003800c780 tx=0x7f003800cc40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:17:57.701 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:17:57.694+0000 7f00367fc640 1 -- 172.21.15.143:0/3439641727 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0038003d90 con 0x7f0040153b90 2023-11-26T07:17:57.702 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:17:57.694+0000 7f00367fc640 1 -- 172.21.15.143:0/3439641727 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f003800ce90 con 0x7f0040153b90 2023-11-26T07:17:57.702 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:17:57.694+0000 7f00367fc640 1 -- 172.21.15.143:0/3439641727 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f00380052b0 con 0x7f0040153b90 2023-11-26T07:17:57.702 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:17:57.694+0000 7f0046aba640 1 -- 172.21.15.143:0/3439641727 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0040158820 con 0x7f0040153b90 2023-11-26T07:17:57.702 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:17:57.694+0000 7f0046aba640 1 -- 172.21.15.143:0/3439641727 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0040158de0 con 0x7f0040153b90 2023-11-26T07:17:57.702 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:17:57.698+0000 7f0046aba640 1 -- 172.21.15.143:0/3439641727 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0040159000 con 0x7f0040153b90 2023-11-26T07:17:57.702 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:17:57.698+0000 7f00367fc640 1 -- 172.21.15.143:0/3439641727 <== mon.0 v2:172.21.15.143:3300/0 4 ==== mgrmap(e 5) v1 ==== 85852+0+0 (secure 0 0 0) 0x7f0038005410 con 0x7f0040153b90 2023-11-26T07:17:57.703 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:17:57.698+0000 7f00367fc640 1 --2- 172.21.15.143:0/3439641727 >> [v2:172.21.15.143:6832/2150557544,v1:172.21.15.143:6833/2150557544] conn(0x7f0024066470 0x7f0024068930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:17:57.703 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:17:57.698+0000 7f00367fc640 1 -- 172.21.15.143:0/3439641727 <== mon.0 v2:172.21.15.143:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f0038083490 con 0x7f0040153b90 2023-11-26T07:17:57.703 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:17:57.698+0000 7f0045ab8640 1 --2- 172.21.15.143:0/3439641727 >> [v2:172.21.15.143:6832/2150557544,v1:172.21.15.143:6833/2150557544] conn(0x7f0024066470 0x7f0024068930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:17:57.703 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:17:57.698+0000 7f0045ab8640 1 --2- 172.21.15.143:0/3439641727 >> [v2:172.21.15.143:6832/2150557544,v1:172.21.15.143:6833/2150557544] conn(0x7f0024066470 0x7f0024068930 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7f003c0ae000 tx=0x7f003c0a7000 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:17:57.707 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:17:57.702+0000 7f00367fc640 1 -- 172.21.15.143:0/3439641727 <== mon.0 v2:172.21.15.143:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172649 (secure 0 0 0) 0x7f00380507e0 con 0x7f0040153b90 2023-11-26T07:17:57.886 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:17:57.882+0000 7f0046aba640 1 -- 172.21.15.143:0/3439641727 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f0040067500 con 0x7f0040153b90 2023-11-26T07:17:57.887 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:17:57.882+0000 7f00367fc640 1 -- 172.21.15.143:0/3439641727 <== mon.0 v2:172.21.15.143:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 8 v8) v1 ==== 93+0+4913 (secure 0 0 0) 0x7f0038017090 con 0x7f0040153b90 2023-11-26T07:17:57.887 INFO:teuthology.orchestra.run.smithi143.stdout: 2023-11-26T07:17:57.888 INFO:teuthology.orchestra.run.smithi143.stdout:{"epoch":8,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4241,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.188:6834/652687322","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6832","nonce":652687322},{"type":"v1","addr":"172.21.15.188:6834","nonce":652687322}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":4266,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.143:6835/3850184704","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6834","nonce":3850184704},{"type":"v1","addr":"172.21.15.143:6835","nonce":3850184704}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":2},{"gid":4273,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.143:6837/1695478651","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6836","nonce":1695478651},{"type":"v1","addr":"172.21.15.143:6837","nonce":1695478651}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":8,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-11-26T07:17:53.876989+0000","modified":"2023-11-26T07:17:56.885608+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":4285},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4285":{"gid":4285,"name":"b","rank":0,"incarnation":4,"state":"up:active","state_seq":3,"addr":"172.21.15.188:6835/994617268","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6833","nonce":994617268},{"type":"v1","addr":"172.21.15.188:6835","nonce":994617268}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2023-11-26T07:17:57.888 INFO:teuthology.orchestra.run.smithi143.stderr:dumped fsmap epoch 8 2023-11-26T07:17:57.889 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:17:57.886+0000 7f0046aba640 1 -- 172.21.15.143:0/3439641727 >> [v2:172.21.15.143:6832/2150557544,v1:172.21.15.143:6833/2150557544] conn(0x7f0024066470 msgr2=0x7f0024068930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:17:57.889 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:17:57.886+0000 7f0046aba640 1 --2- 172.21.15.143:0/3439641727 >> [v2:172.21.15.143:6832/2150557544,v1:172.21.15.143:6833/2150557544] conn(0x7f0024066470 0x7f0024068930 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7f003c0ae000 tx=0x7f003c0a7000 comp rx=0 tx=0).stop 2023-11-26T07:17:57.889 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:17:57.886+0000 7f0046aba640 1 -- 172.21.15.143:0/3439641727 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f0040153b90 msgr2=0x7f0040167530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:17:57.889 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:17:57.886+0000 7f0046aba640 1 --2- 172.21.15.143:0/3439641727 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f0040153b90 0x7f0040167530 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7f003800c780 tx=0x7f003800cc40 comp rx=0 tx=0).stop 2023-11-26T07:17:57.889 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:17:57.886+0000 7f0046aba640 1 -- 172.21.15.143:0/3439641727 shutdown_connections 2023-11-26T07:17:57.890 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:17:57.886+0000 7f0046aba640 1 --2- 172.21.15.143:0/3439641727 >> [v2:172.21.15.143:6832/2150557544,v1:172.21.15.143:6833/2150557544] conn(0x7f0024066470 0x7f0024068930 unknown :-1 s=CLOSED pgs=70 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:17:57.890 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:17:57.886+0000 7f0046aba640 1 --2- 172.21.15.143:0/3439641727 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f0040153b90 0x7f0040167530 unknown :-1 s=CLOSED pgs=81 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:17:57.890 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:17:57.886+0000 7f0046aba640 1 --2- 172.21.15.143:0/3439641727 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f00400b2b40 0x7f00401577a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:17:57.890 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:17:57.886+0000 7f0046aba640 1 --2- 172.21.15.143:0/3439641727 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f004006d820 0x7f004014b820 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:17:57.890 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:17:57.886+0000 7f0046aba640 1 -- 172.21.15.143:0/3439641727 >> 172.21.15.143:0/3439641727 conn(0x7f004006a890 msgr2=0x7f004015afe0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:17:57.890 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:17:57.886+0000 7f0046aba640 1 -- 172.21.15.143:0/3439641727 shutdown_connections 2023-11-26T07:17:57.890 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:17:57.886+0000 7f0046aba640 1 -- 172.21.15.143:0/3439641727 wait complete. 2023-11-26T07:17:57.908 DEBUG:teuthology.orchestra.run.smithi143:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-11-26T07:17:58.045 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:17:58.042+0000 7f5c86c38640 1 Processor -- start 2023-11-26T07:17:58.045 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:17:58.042+0000 7f5c86c38640 1 -- start start 2023-11-26T07:17:58.046 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:17:58.042+0000 7f5c86c38640 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7f5c800b7160 0x7f5c800d3be0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:17:58.046 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:17:58.042+0000 7f5c86c38640 1 -- --> v1:172.21.15.143:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5c800ce3b0 con 0x7f5c800d01b0 2023-11-26T07:17:58.046 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:17:58.042+0000 7f5c86c38640 1 -- --> v1:172.21.15.188:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5c800ce690 con 0x7f5c8015d230 2023-11-26T07:17:58.046 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:17:58.042+0000 7f5c86c38640 1 -- --> v2:172.21.15.188:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5c800ce970 con 0x7f5c800b7160 2023-11-26T07:17:58.046 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:17:58.042+0000 7f5c85435640 1 --1- >> v1:172.21.15.143:6789/0 conn(0x7f5c800d01b0 0x7f5c800b3cc0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.143:6789/0 says I am v1:172.21.15.143:40094/0 (socket says 172.21.15.143:40094) 2023-11-26T07:17:58.047 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:17:58.042+0000 7f5c85435640 1 -- 172.21.15.143:0/804578113 learned_addr learned my addr 172.21.15.143:0/804578113 (peer_addr_for_me v1:172.21.15.143:0/0) 2023-11-26T07:17:58.049 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:17:58.042+0000 7f5c86437640 1 --2- 172.21.15.143:0/804578113 >> v2:172.21.15.188:3300/0 conn(0x7f5c800b7160 0x7f5c800d3be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:17:58.049 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:17:58.042+0000 7f5c84c34640 1 -- 172.21.15.143:0/804578113 <== mon.0 v1:172.21.15.143:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3992150813 0 0) 0x7f5c800ce3b0 con 0x7f5c800d01b0 2023-11-26T07:17:58.049 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:17:58.042+0000 7f5c84c34640 1 -- 172.21.15.143:0/804578113 --> v1:172.21.15.143:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f5c6c003530 con 0x7f5c800d01b0 2023-11-26T07:17:58.049 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:17:58.042+0000 7f5c84c34640 1 -- 172.21.15.143:0/804578113 <== mon.1 v1:172.21.15.188:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2830247836 0 0) 0x7f5c800ce690 con 0x7f5c8015d230 2023-11-26T07:17:58.049 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:17:58.042+0000 7f5c84c34640 1 -- 172.21.15.143:0/804578113 --> v1:172.21.15.188:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f5c800ce3b0 con 0x7f5c8015d230 2023-11-26T07:17:58.049 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:17:58.042+0000 7f5c84c34640 1 -- 172.21.15.143:0/804578113 <== mon.0 v1:172.21.15.143:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2559284005 0 0) 0x7f5c6c003530 con 0x7f5c800d01b0 2023-11-26T07:17:58.049 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:17:58.042+0000 7f5c84c34640 1 -- 172.21.15.143:0/804578113 >> v2:172.21.15.188:3300/0 conn(0x7f5c800b7160 msgr2=0x7f5c800d3be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-26T07:17:58.049 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:17:58.042+0000 7f5c84c34640 1 --2- 172.21.15.143:0/804578113 >> v2:172.21.15.188:3300/0 conn(0x7f5c800b7160 0x7f5c800d3be0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:17:58.049 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:17:58.042+0000 7f5c84c34640 1 -- 172.21.15.143:0/804578113 >> v1:172.21.15.188:6789/0 conn(0x7f5c8015d230 legacy=0x7f5c800cfaa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:17:58.049 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:17:58.042+0000 7f5c84c34640 1 -- 172.21.15.143:0/804578113 --> v1:172.21.15.143:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5c800cec50 con 0x7f5c800d01b0 2023-11-26T07:17:58.049 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:17:58.042+0000 7f5c84c34640 1 -- 172.21.15.143:0/804578113 <== mon.0 v1:172.21.15.143:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2137882523 0 0) 0x7f5c70001680 con 0x7f5c800d01b0 2023-11-26T07:17:58.049 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:17:58.042+0000 7f5c84c34640 1 -- 172.21.15.143:0/804578113 >> v1:172.21.15.143:6789/0 conn(0x7f5c800d01b0 legacy=0x7f5c800b3cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:17:58.050 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:17:58.042+0000 7f5c84c34640 1 --2- 172.21.15.143:0/804578113 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f5c6c003b40 0x7f5c6c003f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:17:58.050 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:17:58.042+0000 7f5c84c34640 1 --2- 172.21.15.143:0/804578113 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f5c6c0047d0 0x7f5c6c004c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:17:58.050 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:17:58.042+0000 7f5c84c34640 1 --2- 172.21.15.143:0/804578113 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f5c6c005190 0x7f5c6c00b990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:17:58.050 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:17:58.042+0000 7f5c84c34640 1 -- 172.21.15.143:0/804578113 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5c800ce970 con 0x7f5c6c0047d0 2023-11-26T07:17:58.050 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:17:58.042+0000 7f5c84c34640 1 -- 172.21.15.143:0/804578113 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5c6c00c0e0 con 0x7f5c6c005190 2023-11-26T07:17:58.050 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:17:58.042+0000 7f5c84c34640 1 -- 172.21.15.143:0/804578113 --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5c6c00c2a0 con 0x7f5c6c003b40 2023-11-26T07:17:58.050 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:17:58.042+0000 7f5c85435640 1 --2- 172.21.15.143:0/804578113 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f5c6c0047d0 0x7f5c6c004c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:17:58.050 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:17:58.042+0000 7f5c85c36640 1 --2- 172.21.15.143:0/804578113 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f5c6c003b40 0x7f5c6c003f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:17:58.050 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:17:58.042+0000 7f5c86437640 1 --2- 172.21.15.143:0/804578113 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f5c6c005190 0x7f5c6c00b990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:17:58.050 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:17:58.042+0000 7f5c85435640 1 -- 172.21.15.143:0/804578113 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f5c6c003b40 msgr2=0x7f5c6c003f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-26T07:17:58.050 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:17:58.042+0000 7f5c85435640 1 --2- 172.21.15.143:0/804578113 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f5c6c003b40 0x7f5c6c003f40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:17:58.051 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:17:58.042+0000 7f5c85435640 1 -- 172.21.15.143:0/804578113 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f5c6c005190 msgr2=0x7f5c6c00b990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-26T07:17:58.051 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:17:58.042+0000 7f5c85435640 1 --2- 172.21.15.143:0/804578113 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f5c6c005190 0x7f5c6c00b990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:17:58.051 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:17:58.042+0000 7f5c85435640 1 -- 172.21.15.143:0/804578113 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5c6c003530 con 0x7f5c6c0047d0 2023-11-26T07:17:58.051 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:17:58.042+0000 7f5c86437640 1 -- 172.21.15.143:0/804578113 reap_dead start 2023-11-26T07:17:58.052 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:17:58.046+0000 7f5c85435640 1 --2- 172.21.15.143:0/804578113 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f5c6c0047d0 0x7f5c6c004c50 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7f5c7000c190 tx=0x7f5c7001cba0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8dddca7300b7bb4e server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:17:58.052 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:17:58.046+0000 7f5c84c34640 1 -- 172.21.15.143:0/804578113 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5c700058e0 con 0x7f5c6c0047d0 2023-11-26T07:17:58.052 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:17:58.046+0000 7f5c84c34640 1 -- 172.21.15.143:0/804578113 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5c70005a40 con 0x7f5c6c0047d0 2023-11-26T07:17:58.052 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:17:58.046+0000 7f5c84c34640 1 -- 172.21.15.143:0/804578113 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5c70005e50 con 0x7f5c6c0047d0 2023-11-26T07:17:58.052 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:17:58.046+0000 7f5c86c38640 1 -- 172.21.15.143:0/804578113 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f5c6c0047d0 msgr2=0x7f5c6c004c50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:17:58.052 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:17:58.046+0000 7f5c86c38640 1 --2- 172.21.15.143:0/804578113 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f5c6c0047d0 0x7f5c6c004c50 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7f5c7000c190 tx=0x7f5c7001cba0 comp rx=0 tx=0).stop 2023-11-26T07:17:58.053 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:17:58.046+0000 7f5c86c38640 1 -- 172.21.15.143:0/804578113 shutdown_connections 2023-11-26T07:17:58.053 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:17:58.046+0000 7f5c86c38640 1 --2- 172.21.15.143:0/804578113 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f5c6c0047d0 0x7f5c6c004c50 unknown :-1 s=CLOSED pgs=83 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:17:58.053 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:17:58.046+0000 7f5c86c38640 1 -- 172.21.15.143:0/804578113 >> 172.21.15.143:0/804578113 conn(0x7f5c800d30c0 msgr2=0x7f5c800d34d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:17:58.053 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:17:58.046+0000 7f5c86c38640 1 -- 172.21.15.143:0/804578113 shutdown_connections 2023-11-26T07:17:58.053 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:17:58.046+0000 7f5c86c38640 1 -- 172.21.15.143:0/804578113 wait complete. 2023-11-26T07:17:58.053 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:17:58.050+0000 7f5c86c38640 1 Processor -- start 2023-11-26T07:17:58.053 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:17:58.050+0000 7f5c86c38640 1 -- start start 2023-11-26T07:17:58.056 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:17:58.050+0000 7f5c86c38640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f5c6c0047d0 0x7f5c800ca7e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:17:58.056 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:17:58.050+0000 7f5c86c38640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f5c80142160 0x7f5c801425c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:17:58.056 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:17:58.050+0000 7f5c86c38640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f5c8015fe80 0x7f5c80160330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:17:58.056 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:17:58.050+0000 7f5c86c38640 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5c80140120 con 0x7f5c80142160 2023-11-26T07:17:58.056 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:17:58.050+0000 7f5c86c38640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5c80142b00 con 0x7f5c6c0047d0 2023-11-26T07:17:58.056 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:17:58.050+0000 7f5c86c38640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5c80160870 con 0x7f5c8015fe80 2023-11-26T07:17:58.056 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:17:58.050+0000 7f5c85435640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f5c80142160 0x7f5c801425c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:17:58.056 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:17:58.050+0000 7f5c85435640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f5c80142160 0x7f5c801425c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:46260/0 (socket says 172.21.15.143:46260) 2023-11-26T07:17:58.056 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:17:58.050+0000 7f5c85435640 1 -- 172.21.15.143:0/3850923971 learned_addr learned my addr 172.21.15.143:0/3850923971 (peer_addr_for_me v2:172.21.15.143:0/0) 2023-11-26T07:17:58.057 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:17:58.050+0000 7f5c85c36640 1 --2- 172.21.15.143:0/3850923971 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f5c6c0047d0 0x7f5c800ca7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:17:58.057 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:17:58.050+0000 7f5c86437640 1 --2- 172.21.15.143:0/3850923971 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f5c8015fe80 0x7f5c80160330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:17:58.057 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:17:58.050+0000 7f5c85435640 1 -- 172.21.15.143:0/3850923971 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f5c8015fe80 msgr2=0x7f5c80160330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:17:58.057 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:17:58.050+0000 7f5c85435640 1 --2- 172.21.15.143:0/3850923971 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f5c8015fe80 0x7f5c80160330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:17:58.057 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:17:58.050+0000 7f5c85435640 1 -- 172.21.15.143:0/3850923971 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f5c6c0047d0 msgr2=0x7f5c800ca7e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:17:58.057 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:17:58.050+0000 7f5c85435640 1 --2- 172.21.15.143:0/3850923971 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f5c6c0047d0 0x7f5c800ca7e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:17:58.057 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:17:58.050+0000 7f5c85435640 1 -- 172.21.15.143:0/3850923971 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5c801ea3b0 con 0x7f5c80142160 2023-11-26T07:17:58.057 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:17:58.050+0000 7f5c85c36640 1 --2- 172.21.15.143:0/3850923971 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f5c6c0047d0 0x7f5c800ca7e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-26T07:17:58.057 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:17:58.050+0000 7f5c85435640 1 --2- 172.21.15.143:0/3850923971 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f5c80142160 0x7f5c801425c0 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7f5c700ab040 tx=0x7f5c6c005800 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:17:58.057 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:17:58.050+0000 7f5c767fc640 1 -- 172.21.15.143:0/3850923971 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5c700a58a0 con 0x7f5c80142160 2023-11-26T07:17:58.057 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:17:58.050+0000 7f5c767fc640 1 -- 172.21.15.143:0/3850923971 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5c700a5a00 con 0x7f5c80142160 2023-11-26T07:17:58.058 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:17:58.050+0000 7f5c767fc640 1 -- 172.21.15.143:0/3850923971 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5c700a5d70 con 0x7f5c80142160 2023-11-26T07:17:58.058 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:17:58.050+0000 7f5c86c38640 1 -- 172.21.15.143:0/3850923971 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5c801ea690 con 0x7f5c80142160 2023-11-26T07:17:58.058 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:17:58.050+0000 7f5c86c38640 1 -- 172.21.15.143:0/3850923971 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5c80140650 con 0x7f5c80142160 2023-11-26T07:17:58.058 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:17:58.054+0000 7f5c86c38640 1 -- 172.21.15.143:0/3850923971 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5c800cfb40 con 0x7f5c80142160 2023-11-26T07:17:58.058 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:17:58.054+0000 7f5c767fc640 1 -- 172.21.15.143:0/3850923971 <== mon.0 v2:172.21.15.143:3300/0 4 ==== mgrmap(e 5) v1 ==== 85852+0+0 (secure 0 0 0) 0x7f5c700047f0 con 0x7f5c80142160 2023-11-26T07:17:58.059 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:17:58.054+0000 7f5c767fc640 1 --2- 172.21.15.143:0/3850923971 >> [v2:172.21.15.143:6832/2150557544,v1:172.21.15.143:6833/2150557544] conn(0x7f5c640664a0 0x7f5c64068960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:17:58.059 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:17:58.054+0000 7f5c767fc640 1 -- 172.21.15.143:0/3850923971 <== mon.0 v2:172.21.15.143:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f5c7011d510 con 0x7f5c80142160 2023-11-26T07:17:58.059 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:17:58.054+0000 7f5c85c36640 1 --2- 172.21.15.143:0/3850923971 >> [v2:172.21.15.143:6832/2150557544,v1:172.21.15.143:6833/2150557544] conn(0x7f5c640664a0 0x7f5c64068960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:17:58.059 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:17:58.054+0000 7f5c85c36640 1 --2- 172.21.15.143:0/3850923971 >> [v2:172.21.15.143:6832/2150557544,v1:172.21.15.143:6833/2150557544] conn(0x7f5c640664a0 0x7f5c64068960 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f5c80067bb0 tx=0x7f5c7c009040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:17:58.062 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:17:58.058+0000 7f5c767fc640 1 -- 172.21.15.143:0/3850923971 <== mon.0 v2:172.21.15.143:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172649 (secure 0 0 0) 0x7f5c700ea870 con 0x7f5c80142160 2023-11-26T07:17:58.212 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:17:58.210+0000 7f5c86c38640 1 -- 172.21.15.143:0/3850923971 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f5c80067c10 con 0x7f5c80142160 2023-11-26T07:17:58.214 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:17:58.210+0000 7f5c767fc640 1 -- 172.21.15.143:0/3850923971 <== mon.0 v2:172.21.15.143:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v20) v1 ==== 74+0+16541 (secure 0 0 0) 0x7f5c700ede90 con 0x7f5c80142160 2023-11-26T07:17:58.214 INFO:teuthology.orchestra.run.smithi143.stdout: 2023-11-26T07:17:58.214 INFO:teuthology.orchestra.run.smithi143.stdout:{"epoch":20,"fsid":"f5033060-2469-4a07-95fe-2b76f5404bc1","created":"2023-11-26T07:17:28.613663+0000","modified":"2023-11-26T07:17:55.505435+0000","last_up_change":"2023-11-26T07:17:41.157813+0000","last_in_change":"2023-11-26T07:17:31.181658+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-11-26T07:17:41.758597+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2023-11-26T07:17:51.878332+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2023-11-26T07:17:52.871396+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"ebbee2c6-023e-44b9-9f1d-6f7bfd309730","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6817","nonce":639807369},{"type":"v1","addr":"172.21.15.143:6819","nonce":639807369}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6821","nonce":639807369},{"type":"v1","addr":"172.21.15.143:6823","nonce":639807369}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6830","nonce":639807369},{"type":"v1","addr":"172.21.15.143:6831","nonce":639807369}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6825","nonce":639807369},{"type":"v1","addr":"172.21.15.143:6828","nonce":639807369}]},"public_addr":"172.21.15.143:6819/639807369","cluster_addr":"172.21.15.143:6823/639807369","heartbeat_back_addr":"172.21.15.143:6831/639807369","heartbeat_front_addr":"172.21.15.143:6828/639807369","state":["exists","up"]},{"osd":1,"uuid":"fec4b8be-7122-49e9-9a19-5850a0700001","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6816","nonce":1144942530},{"type":"v1","addr":"172.21.15.143:6818","nonce":1144942530}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6820","nonce":1144942530},{"type":"v1","addr":"172.21.15.143:6822","nonce":1144942530}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6827","nonce":1144942530},{"type":"v1","addr":"172.21.15.143:6829","nonce":1144942530}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6824","nonce":1144942530},{"type":"v1","addr":"172.21.15.143:6826","nonce":1144942530}]},"public_addr":"172.21.15.143:6818/1144942530","cluster_addr":"172.21.15.143:6822/1144942530","heartbeat_back_addr":"172.21.15.143:6829/1144942530","heartbeat_front_addr":"172.21.15.143:6826/1144942530","state":["exists","up"]},{"osd":2,"uuid":"2888af5b-f933-46ca-8f95-6dd494ff43f6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6808","nonce":3909095186},{"type":"v1","addr":"172.21.15.143:6809","nonce":3909095186}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6810","nonce":3909095186},{"type":"v1","addr":"172.21.15.143:6811","nonce":3909095186}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6814","nonce":3909095186},{"type":"v1","addr":"172.21.15.143:6815","nonce":3909095186}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6812","nonce":3909095186},{"type":"v1","addr":"172.21.15.143:6813","nonce":3909095186}]},"public_addr":"172.21.15.143:6809/3909095186","cluster_addr":"172.21.15.143:6811/3909095186","heartbeat_back_addr":"172.21.15.143:6815/3909095186","heartbeat_front_addr":"172.21.15.143:6813/3909095186","state":["exists","up"]},{"osd":3,"uuid":"d1f52435-7bc1-4081-9550-9cd1ba55f549","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6800","nonce":3854210991},{"type":"v1","addr":"172.21.15.143:6801","nonce":3854210991}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6802","nonce":3854210991},{"type":"v1","addr":"172.21.15.143:6803","nonce":3854210991}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6806","nonce":3854210991},{"type":"v1","addr":"172.21.15.143:6807","nonce":3854210991}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6804","nonce":3854210991},{"type":"v1","addr":"172.21.15.143:6805","nonce":3854210991}]},"public_addr":"172.21.15.143:6801/3854210991","cluster_addr":"172.21.15.143:6803/3854210991","heartbeat_back_addr":"172.21.15.143:6807/3854210991","heartbeat_front_addr":"172.21.15.143:6805/3854210991","state":["exists","up"]},{"osd":4,"uuid":"e2ae8b6e-6e25-49b2-bcbf-aacce68f06a3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6800","nonce":3924606902},{"type":"v1","addr":"172.21.15.188:6801","nonce":3924606902}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6802","nonce":3924606902},{"type":"v1","addr":"172.21.15.188:6803","nonce":3924606902}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6806","nonce":3924606902},{"type":"v1","addr":"172.21.15.188:6807","nonce":3924606902}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6804","nonce":3924606902},{"type":"v1","addr":"172.21.15.188:6805","nonce":3924606902}]},"public_addr":"172.21.15.188:6801/3924606902","cluster_addr":"172.21.15.188:6803/3924606902","heartbeat_back_addr":"172.21.15.188:6807/3924606902","heartbeat_front_addr":"172.21.15.188:6805/3924606902","state":["exists","up"]},{"osd":5,"uuid":"3cec8a32-7698-4c36-8e89-e4ba11934f33","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6813","nonce":186841544},{"type":"v1","addr":"172.21.15.188:6815","nonce":186841544}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6817","nonce":186841544},{"type":"v1","addr":"172.21.15.188:6818","nonce":186841544}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6822","nonce":186841544},{"type":"v1","addr":"172.21.15.188:6823","nonce":186841544}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6820","nonce":186841544},{"type":"v1","addr":"172.21.15.188:6821","nonce":186841544}]},"public_addr":"172.21.15.188:6815/186841544","cluster_addr":"172.21.15.188:6818/186841544","heartbeat_back_addr":"172.21.15.188:6823/186841544","heartbeat_front_addr":"172.21.15.188:6821/186841544","state":["exists","up"]},{"osd":6,"uuid":"58a65d5d-e090-473a-81b4-ad38e85029f9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6808","nonce":2153499405},{"type":"v1","addr":"172.21.15.188:6809","nonce":2153499405}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6810","nonce":2153499405},{"type":"v1","addr":"172.21.15.188:6811","nonce":2153499405}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6816","nonce":2153499405},{"type":"v1","addr":"172.21.15.188:6819","nonce":2153499405}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6812","nonce":2153499405},{"type":"v1","addr":"172.21.15.188:6814","nonce":2153499405}]},"public_addr":"172.21.15.188:6809/2153499405","cluster_addr":"172.21.15.188:6811/2153499405","heartbeat_back_addr":"172.21.15.188:6819/2153499405","heartbeat_front_addr":"172.21.15.188:6814/2153499405","state":["exists","up"]},{"osd":7,"uuid":"ea5b7f70-3554-4cf8-ac30-f7c59b80678d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6824","nonce":2722096729},{"type":"v1","addr":"172.21.15.188:6825","nonce":2722096729}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6826","nonce":2722096729},{"type":"v1","addr":"172.21.15.188:6827","nonce":2722096729}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6830","nonce":2722096729},{"type":"v1","addr":"172.21.15.188:6831","nonce":2722096729}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6828","nonce":2722096729},{"type":"v1","addr":"172.21.15.188:6829","nonce":2722096729}]},"public_addr":"172.21.15.188:6825/2722096729","cluster_addr":"172.21.15.188:6827/2722096729","heartbeat_back_addr":"172.21.15.188:6831/2722096729","heartbeat_front_addr":"172.21.15.188:6829/2722096729","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-26T07:17:36.995032+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-26T07:17:36.932266+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-26T07:17:36.829304+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-26T07:17:36.804297+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-26T07:17:36.667892+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-26T07:17:36.964868+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-26T07:17:36.827332+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-26T07:17:36.893179+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-11-26T07:17:58.216 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:17:58.210+0000 7f5c86c38640 1 -- 172.21.15.143:0/3850923971 >> [v2:172.21.15.143:6832/2150557544,v1:172.21.15.143:6833/2150557544] conn(0x7f5c640664a0 msgr2=0x7f5c64068960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:17:58.216 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:17:58.210+0000 7f5c86c38640 1 --2- 172.21.15.143:0/3850923971 >> [v2:172.21.15.143:6832/2150557544,v1:172.21.15.143:6833/2150557544] conn(0x7f5c640664a0 0x7f5c64068960 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f5c80067bb0 tx=0x7f5c7c009040 comp rx=0 tx=0).stop 2023-11-26T07:17:58.216 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:17:58.210+0000 7f5c86c38640 1 -- 172.21.15.143:0/3850923971 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f5c80142160 msgr2=0x7f5c801425c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:17:58.216 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:17:58.210+0000 7f5c86c38640 1 --2- 172.21.15.143:0/3850923971 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f5c80142160 0x7f5c801425c0 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7f5c700ab040 tx=0x7f5c6c005800 comp rx=0 tx=0).stop 2023-11-26T07:17:58.216 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:17:58.210+0000 7f5c86c38640 1 -- 172.21.15.143:0/3850923971 shutdown_connections 2023-11-26T07:17:58.216 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:17:58.210+0000 7f5c86c38640 1 --2- 172.21.15.143:0/3850923971 >> [v2:172.21.15.143:6832/2150557544,v1:172.21.15.143:6833/2150557544] conn(0x7f5c640664a0 0x7f5c64068960 unknown :-1 s=CLOSED pgs=71 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:17:58.217 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:17:58.210+0000 7f5c86c38640 1 --2- 172.21.15.143:0/3850923971 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f5c8015fe80 0x7f5c80160330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:17:58.217 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:17:58.210+0000 7f5c86c38640 1 --2- 172.21.15.143:0/3850923971 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f5c80142160 0x7f5c801425c0 unknown :-1 s=CLOSED pgs=84 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:17:58.217 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:17:58.210+0000 7f5c86c38640 1 --2- 172.21.15.143:0/3850923971 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f5c6c0047d0 0x7f5c800ca7e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:17:58.217 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:17:58.210+0000 7f5c86c38640 1 -- 172.21.15.143:0/3850923971 >> 172.21.15.143:0/3850923971 conn(0x7f5c800d30c0 msgr2=0x7f5c8006c690 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:17:58.217 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:17:58.210+0000 7f5c86c38640 1 -- 172.21.15.143:0/3850923971 shutdown_connections 2023-11-26T07:17:58.217 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:17:58.210+0000 7f5c86c38640 1 -- 172.21.15.143:0/3850923971 wait complete. 2023-11-26T07:17:58.235 INFO:tasks.cephfs.filesystem.ceph_manager:waiting for clean 2023-11-26T07:17:58.235 DEBUG:teuthology.orchestra.run.smithi143:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-11-26T07:17:58.244 INFO:tasks.ceph.mds.b.smithi188.stderr:2023-11-26T07:17:58.238+0000 7fde7bce6640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2023-11-26T07:17:58.372 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:17:58.366+0000 7f291c46b640 1 Processor -- start 2023-11-26T07:17:58.372 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:17:58.366+0000 7f291c46b640 1 -- start start 2023-11-26T07:17:58.374 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:17:58.370+0000 7f291c46b640 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7f29140cdb60 0x7f29140aead0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:17:58.374 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:17:58.370+0000 7f291c46b640 1 -- --> v1:172.21.15.143:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f29140b3690 con 0x7f2914070ba0 2023-11-26T07:17:58.374 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:17:58.370+0000 7f291c46b640 1 -- --> v1:172.21.15.188:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f29140cf080 con 0x7f291406a490 2023-11-26T07:17:58.374 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:17:58.370+0000 7f291c46b640 1 -- --> v2:172.21.15.143:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f29140b3ca0 con 0x7f29140cdb60 2023-11-26T07:17:58.374 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:17:58.370+0000 7f2919a06640 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7f29140cdb60 0x7f29140aead0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:17:58.374 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:17:58.370+0000 7f291a207640 1 --1- >> v1:172.21.15.143:6789/0 conn(0x7f2914070ba0 0x7f2914070f80 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.143:6789/0 says I am v1:172.21.15.143:40102/0 (socket says 172.21.15.143:40102) 2023-11-26T07:17:58.374 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:17:58.370+0000 7f2919a06640 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7f29140cdb60 0x7f29140aead0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:46276/0 (socket says 172.21.15.143:46276) 2023-11-26T07:17:58.374 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:17:58.370+0000 7f291a207640 1 -- 172.21.15.143:0/1806008803 learned_addr learned my addr 172.21.15.143:0/1806008803 (peer_addr_for_me v1:172.21.15.143:0/0) 2023-11-26T07:17:58.375 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:17:58.370+0000 7f2919a06640 1 -- 172.21.15.143:0/1806008803 >> v1:172.21.15.188:6789/0 conn(0x7f291406a490 legacy=0x7f29140b40c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-26T07:17:58.375 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:17:58.370+0000 7f2919205640 1 -- 172.21.15.143:0/1806008803 <== mon.0 v1:172.21.15.143:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1175135752 0 0) 0x7f29140b3690 con 0x7f2914070ba0 2023-11-26T07:17:58.375 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:17:58.370+0000 7f2919a06640 1 -- 172.21.15.143:0/1806008803 >> v1:172.21.15.143:6789/0 conn(0x7f2914070ba0 legacy=0x7f2914070f80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:17:58.375 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:17:58.370+0000 7f2919a06640 1 -- 172.21.15.143:0/1806008803 --> v2:172.21.15.143:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f29140cd8c0 con 0x7f29140cdb60 2023-11-26T07:17:58.377 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:17:58.374+0000 7f2919a06640 1 --2- 172.21.15.143:0/1806008803 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f29140cdb60 0x7f29140aead0 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7f29040097c0 tx=0x7f290401c3b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5ca1f85b28ea0291 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:17:58.378 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:17:58.374+0000 7f2919205640 1 -- 172.21.15.143:0/1806008803 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f29040ab600 con 0x7f29140cdb60 2023-11-26T07:17:58.378 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:17:58.374+0000 7f2919205640 1 -- 172.21.15.143:0/1806008803 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f29040a7050 con 0x7f29140cdb60 2023-11-26T07:17:58.378 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:17:58.374+0000 7f2919205640 1 -- 172.21.15.143:0/1806008803 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f29040ab980 con 0x7f29140cdb60 2023-11-26T07:17:58.378 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:17:58.374+0000 7f291c46b640 1 -- 172.21.15.143:0/1806008803 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f29140cdb60 msgr2=0x7f29140aead0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:17:58.378 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:17:58.374+0000 7f291c46b640 1 --2- 172.21.15.143:0/1806008803 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f29140cdb60 0x7f29140aead0 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7f29040097c0 tx=0x7f290401c3b0 comp rx=0 tx=0).stop 2023-11-26T07:17:58.379 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:17:58.374+0000 7f291c46b640 1 -- 172.21.15.143:0/1806008803 shutdown_connections 2023-11-26T07:17:58.379 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:17:58.374+0000 7f291c46b640 1 --2- 172.21.15.143:0/1806008803 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f29140cdb60 0x7f29140aead0 unknown :-1 s=CLOSED pgs=86 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:17:58.379 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:17:58.374+0000 7f291c46b640 1 -- 172.21.15.143:0/1806008803 >> 172.21.15.143:0/1806008803 conn(0x7f29140cf710 msgr2=0x7f29140cfb20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:17:58.379 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:17:58.374+0000 7f291c46b640 1 -- 172.21.15.143:0/1806008803 shutdown_connections 2023-11-26T07:17:58.379 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:17:58.374+0000 7f291c46b640 1 -- 172.21.15.143:0/1806008803 wait complete. 2023-11-26T07:17:58.380 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:17:58.374+0000 7f291c46b640 1 Processor -- start 2023-11-26T07:17:58.380 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:17:58.374+0000 7f291c46b640 1 -- start start 2023-11-26T07:17:58.381 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:17:58.378+0000 7f291c46b640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f291406a490 0x7f2914151d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:17:58.381 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:17:58.378+0000 7f291c46b640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f2914070ba0 0x7f291414f120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:17:58.382 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:17:58.378+0000 7f291c46b640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f29140cdb60 0x7f291414f8a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:17:58.382 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:17:58.378+0000 7f291c46b640 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2914158970 con 0x7f29140cdb60 2023-11-26T07:17:58.382 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:17:58.378+0000 7f291c46b640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2914152580 con 0x7f291406a490 2023-11-26T07:17:58.382 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:17:58.378+0000 7f291c46b640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f291414f660 con 0x7f2914070ba0 2023-11-26T07:17:58.382 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:17:58.378+0000 7f291aa08640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f29140cdb60 0x7f291414f8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:17:58.382 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:17:58.378+0000 7f291a207640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f291406a490 0x7f2914151d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:17:58.382 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:17:58.378+0000 7f291aa08640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f29140cdb60 0x7f291414f8a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:46282/0 (socket says 172.21.15.143:46282) 2023-11-26T07:17:58.382 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:17:58.378+0000 7f291aa08640 1 -- 172.21.15.143:0/2504444290 learned_addr learned my addr 172.21.15.143:0/2504444290 (peer_addr_for_me v2:172.21.15.143:0/0) 2023-11-26T07:17:58.382 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:17:58.378+0000 7f2919a06640 1 --2- 172.21.15.143:0/2504444290 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f2914070ba0 0x7f291414f120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:17:58.383 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:17:58.378+0000 7f291aa08640 1 -- 172.21.15.143:0/2504444290 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f2914070ba0 msgr2=0x7f291414f120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:17:58.383 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:17:58.378+0000 7f291aa08640 1 --2- 172.21.15.143:0/2504444290 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f2914070ba0 0x7f291414f120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:17:58.383 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:17:58.378+0000 7f291aa08640 1 -- 172.21.15.143:0/2504444290 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f291406a490 msgr2=0x7f2914151d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:17:58.384 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:17:58.378+0000 7f291aa08640 1 --2- 172.21.15.143:0/2504444290 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f291406a490 0x7f2914151d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:17:58.384 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:17:58.378+0000 7f291aa08640 1 -- 172.21.15.143:0/2504444290 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2904009420 con 0x7f29140cdb60 2023-11-26T07:17:58.384 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:17:58.378+0000 7f291aa08640 1 --2- 172.21.15.143:0/2504444290 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f29140cdb60 0x7f291414f8a0 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7f290c00c3c0 tx=0x7f290c00c880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:17:58.384 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:17:58.378+0000 7f290affd640 1 -- 172.21.15.143:0/2504444290 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f290c011790 con 0x7f29140cdb60 2023-11-26T07:17:58.384 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:17:58.378+0000 7f290affd640 1 -- 172.21.15.143:0/2504444290 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f290c0118f0 con 0x7f29140cdb60 2023-11-26T07:17:58.384 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:17:58.378+0000 7f290affd640 1 -- 172.21.15.143:0/2504444290 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f290c011ca0 con 0x7f29140cdb60 2023-11-26T07:17:58.384 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:17:58.378+0000 7f291c46b640 1 -- 172.21.15.143:0/2504444290 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2914159e80 con 0x7f29140cdb60 2023-11-26T07:17:58.384 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:17:58.378+0000 7f291c46b640 1 -- 172.21.15.143:0/2504444290 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2914158590 con 0x7f29140cdb60 2023-11-26T07:17:58.384 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:17:58.378+0000 7f291c46b640 1 -- 172.21.15.143:0/2504444290 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2914158ab0 con 0x7f29140cdb60 2023-11-26T07:17:58.384 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:17:58.378+0000 7f290affd640 1 -- 172.21.15.143:0/2504444290 <== mon.0 v2:172.21.15.143:3300/0 4 ==== mgrmap(e 5) v1 ==== 85852+0+0 (secure 0 0 0) 0x7f290c01c460 con 0x7f29140cdb60 2023-11-26T07:17:58.385 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:17:58.382+0000 7f290affd640 1 --2- 172.21.15.143:0/2504444290 >> [v2:172.21.15.143:6832/2150557544,v1:172.21.15.143:6833/2150557544] conn(0x7f28f8066470 0x7f28f8068930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:17:58.385 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:17:58.382+0000 7f290affd640 1 -- 172.21.15.143:0/2504444290 <== mon.0 v2:172.21.15.143:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f290c083c20 con 0x7f29140cdb60 2023-11-26T07:17:58.385 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:17:58.382+0000 7f291a207640 1 --2- 172.21.15.143:0/2504444290 >> [v2:172.21.15.143:6832/2150557544,v1:172.21.15.143:6833/2150557544] conn(0x7f28f8066470 0x7f28f8068930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:17:58.386 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:17:58.382+0000 7f291a207640 1 --2- 172.21.15.143:0/2504444290 >> [v2:172.21.15.143:6832/2150557544,v1:172.21.15.143:6833/2150557544] conn(0x7f28f8066470 0x7f28f8068930 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7f29100012f0 tx=0x7f2910001770 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:17:58.389 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:17:58.386+0000 7f290affd640 1 -- 172.21.15.143:0/2504444290 <== mon.0 v2:172.21.15.143:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172649 (secure 0 0 0) 0x7f2914158ab0 con 0x7f29140cdb60 2023-11-26T07:17:58.541 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:17:58.538+0000 7f291c46b640 1 -- 172.21.15.143:0/2504444290 --> [v2:172.21.15.143:6832/2150557544,v1:172.21.15.143:6833/2150557544] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f2914158ab0 con 0x7f28f8066470 2023-11-26T07:17:58.549 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:17:58.546+0000 7f290affd640 1 -- 172.21.15.143:0/2504444290 <== mgr.4100 v2:172.21.15.143:6832/2150557544 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+264699 (secure 0 0 0) 0x7f2914158ab0 con 0x7f28f8066470 2023-11-26T07:17:58.551 INFO:teuthology.orchestra.run.smithi143.stdout: 2023-11-26T07:17:58.554 INFO:teuthology.orchestra.run.smithi143.stderr:dumped all 2023-11-26T07:17:58.555 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:17:58.550+0000 7f291c46b640 1 -- 172.21.15.143:0/2504444290 >> [v2:172.21.15.143:6832/2150557544,v1:172.21.15.143:6833/2150557544] conn(0x7f28f8066470 msgr2=0x7f28f8068930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:17:58.555 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:17:58.550+0000 7f291c46b640 1 --2- 172.21.15.143:0/2504444290 >> [v2:172.21.15.143:6832/2150557544,v1:172.21.15.143:6833/2150557544] conn(0x7f28f8066470 0x7f28f8068930 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7f29100012f0 tx=0x7f2910001770 comp rx=0 tx=0).stop 2023-11-26T07:17:58.555 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:17:58.550+0000 7f291c46b640 1 -- 172.21.15.143:0/2504444290 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f29140cdb60 msgr2=0x7f291414f8a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:17:58.555 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:17:58.550+0000 7f291c46b640 1 --2- 172.21.15.143:0/2504444290 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f29140cdb60 0x7f291414f8a0 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7f290c00c3c0 tx=0x7f290c00c880 comp rx=0 tx=0).stop 2023-11-26T07:17:58.555 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:17:58.550+0000 7f291c46b640 1 -- 172.21.15.143:0/2504444290 shutdown_connections 2023-11-26T07:17:58.555 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:17:58.550+0000 7f291c46b640 1 --2- 172.21.15.143:0/2504444290 >> [v2:172.21.15.143:6832/2150557544,v1:172.21.15.143:6833/2150557544] conn(0x7f28f8066470 0x7f28f8068930 unknown :-1 s=CLOSED pgs=72 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:17:58.555 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:17:58.550+0000 7f291c46b640 1 --2- 172.21.15.143:0/2504444290 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f29140cdb60 0x7f291414f8a0 unknown :-1 s=CLOSED pgs=87 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:17:58.555 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:17:58.550+0000 7f291c46b640 1 --2- 172.21.15.143:0/2504444290 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f2914070ba0 0x7f291414f120 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:17:58.555 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:17:58.550+0000 7f291c46b640 1 --2- 172.21.15.143:0/2504444290 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f291406a490 0x7f2914151d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:17:58.556 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:17:58.550+0000 7f291c46b640 1 -- 172.21.15.143:0/2504444290 >> 172.21.15.143:0/2504444290 conn(0x7f29140cf710 msgr2=0x7f291415bc40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:17:58.556 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:17:58.550+0000 7f291c46b640 1 -- 172.21.15.143:0/2504444290 shutdown_connections 2023-11-26T07:17:58.556 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:17:58.550+0000 7f291c46b640 1 -- 172.21.15.143:0/2504444290 wait complete. 2023-11-26T07:17:58.572 INFO:teuthology.orchestra.run.smithi143.stdout:{"pg_ready":true,"pg_map":{"version":21,"stamp":"2023-11-26T07:17:57.694444+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592778,"num_objects":23,"num_object_clones":0,"num_object_copies":46,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":23,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":203,"num_write_kb":2430,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":155,"ondisk_log_size":155,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":178,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":218964,"kb_used_data":2788,"kb_used_omap":12,"kb_used_meta":216051,"kb_avail":754755756,"statfs":{"total":773094113280,"available":772869894144,"internally_reserved":0,"allocated":2854912,"data_stored":1733936,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12749,"internal_metadata":221236787},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":40,"apply_latency_ms":40,"commit_latency_ns":40000000,"apply_latency_ns":40000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2410,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"5.209089"},"pg_stats":[{"pgid":"3.3f","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-26T07:17:55.764359+0000","last_change":"2023-11-26T07:17:54.530144+0000","last_active":"2023-11-26T07:17:55.764359+0000","last_peered":"2023-11-26T07:17:55.764359+0000","last_clean":"2023-11-26T07:17:55.764359+0000","last_became_active":"2023-11-26T07:17:54.529816+0000","last_became_peered":"2023-11-26T07:17:54.529816+0000","last_unstale":"2023-11-26T07:17:55.764359+0000","last_undegraded":"2023-11-26T07:17:55.764359+0000","last_fullsized":"2023-11-26T07:17:55.764359+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:17:53.483873+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:17:53.483873+0000","last_clean_scrub_stamp":"2023-11-26T07:17:53.483873+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T08:53:05.773632+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-26T07:17:55.516522+0000","last_change":"2023-11-26T07:17:54.523453+0000","last_active":"2023-11-26T07:17:55.516522+0000","last_peered":"2023-11-26T07:17:55.516522+0000","last_clean":"2023-11-26T07:17:55.516522+0000","last_became_active":"2023-11-26T07:17:54.523104+0000","last_became_peered":"2023-11-26T07:17:54.523104+0000","last_unstale":"2023-11-26T07:17:55.516522+0000","last_undegraded":"2023-11-26T07:17:55.516522+0000","last_fullsized":"2023-11-26T07:17:55.516522+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:17:53.483873+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:17:53.483873+0000","last_clean_scrub_stamp":"2023-11-26T07:17:53.483873+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T11:16:25.801280+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-26T07:17:56.068788+0000","last_change":"2023-11-26T07:17:54.514169+0000","last_active":"2023-11-26T07:17:56.068788+0000","last_peered":"2023-11-26T07:17:56.068788+0000","last_clean":"2023-11-26T07:17:56.068788+0000","last_became_active":"2023-11-26T07:17:54.513836+0000","last_became_peered":"2023-11-26T07:17:54.513836+0000","last_unstale":"2023-11-26T07:17:56.068788+0000","last_undegraded":"2023-11-26T07:17:56.068788+0000","last_fullsized":"2023-11-26T07:17:56.068788+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:17:53.483873+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:17:53.483873+0000","last_clean_scrub_stamp":"2023-11-26T07:17:53.483873+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T09:53:29.244002+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-26T07:17:55.538957+0000","last_change":"2023-11-26T07:17:54.528554+0000","last_active":"2023-11-26T07:17:55.538957+0000","last_peered":"2023-11-26T07:17:55.538957+0000","last_clean":"2023-11-26T07:17:55.538957+0000","last_became_active":"2023-11-26T07:17:54.527864+0000","last_became_peered":"2023-11-26T07:17:54.527864+0000","last_unstale":"2023-11-26T07:17:55.538957+0000","last_undegraded":"2023-11-26T07:17:55.538957+0000","last_fullsized":"2023-11-26T07:17:55.538957+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:17:53.483873+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:17:53.483873+0000","last_clean_scrub_stamp":"2023-11-26T07:17:53.483873+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T15:49:39.889795+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-26T07:17:55.763887+0000","last_change":"2023-11-26T07:17:54.527144+0000","last_active":"2023-11-26T07:17:55.763887+0000","last_peered":"2023-11-26T07:17:55.763887+0000","last_clean":"2023-11-26T07:17:55.763887+0000","last_became_active":"2023-11-26T07:17:54.525279+0000","last_became_peered":"2023-11-26T07:17:54.525279+0000","last_unstale":"2023-11-26T07:17:55.763887+0000","last_undegraded":"2023-11-26T07:17:55.763887+0000","last_fullsized":"2023-11-26T07:17:55.763887+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:17:53.483873+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:17:53.483873+0000","last_clean_scrub_stamp":"2023-11-26T07:17:53.483873+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T11:28:10.566680+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-26T07:17:55.538672+0000","last_change":"2023-11-26T07:17:54.522607+0000","last_active":"2023-11-26T07:17:55.538672+0000","last_peered":"2023-11-26T07:17:55.538672+0000","last_clean":"2023-11-26T07:17:55.538672+0000","last_became_active":"2023-11-26T07:17:54.522297+0000","last_became_peered":"2023-11-26T07:17:54.522297+0000","last_unstale":"2023-11-26T07:17:55.538672+0000","last_undegraded":"2023-11-26T07:17:55.538672+0000","last_fullsized":"2023-11-26T07:17:55.538672+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:17:53.483873+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:17:53.483873+0000","last_clean_scrub_stamp":"2023-11-26T07:17:53.483873+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T13:18:15.521322+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-26T07:17:55.763706+0000","last_change":"2023-11-26T07:17:54.524898+0000","last_active":"2023-11-26T07:17:55.763706+0000","last_peered":"2023-11-26T07:17:55.763706+0000","last_clean":"2023-11-26T07:17:55.763706+0000","last_became_active":"2023-11-26T07:17:54.524611+0000","last_became_peered":"2023-11-26T07:17:54.524611+0000","last_unstale":"2023-11-26T07:17:55.763706+0000","last_undegraded":"2023-11-26T07:17:55.763706+0000","last_fullsized":"2023-11-26T07:17:55.763706+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:17:53.483873+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:17:53.483873+0000","last_clean_scrub_stamp":"2023-11-26T07:17:53.483873+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T17:14:14.930365+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-26T07:17:56.640574+0000","last_change":"2023-11-26T07:17:54.530342+0000","last_active":"2023-11-26T07:17:56.640574+0000","last_peered":"2023-11-26T07:17:56.640574+0000","last_clean":"2023-11-26T07:17:56.640574+0000","last_became_active":"2023-11-26T07:17:54.529960+0000","last_became_peered":"2023-11-26T07:17:54.529960+0000","last_unstale":"2023-11-26T07:17:56.640574+0000","last_undegraded":"2023-11-26T07:17:56.640574+0000","last_fullsized":"2023-11-26T07:17:56.640574+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:17:53.483873+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:17:53.483873+0000","last_clean_scrub_stamp":"2023-11-26T07:17:53.483873+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T08:42:29.375032+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-26T07:17:55.764389+0000","last_change":"2023-11-26T07:17:54.539062+0000","last_active":"2023-11-26T07:17:55.764389+0000","last_peered":"2023-11-26T07:17:55.764389+0000","last_clean":"2023-11-26T07:17:55.764389+0000","last_became_active":"2023-11-26T07:17:54.538119+0000","last_became_peered":"2023-11-26T07:17:54.538119+0000","last_unstale":"2023-11-26T07:17:55.764389+0000","last_undegraded":"2023-11-26T07:17:55.764389+0000","last_fullsized":"2023-11-26T07:17:55.764389+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:17:53.483873+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:17:53.483873+0000","last_clean_scrub_stamp":"2023-11-26T07:17:53.483873+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T13:57:07.286285+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-26T07:17:56.640341+0000","last_change":"2023-11-26T07:17:54.524564+0000","last_active":"2023-11-26T07:17:56.640341+0000","last_peered":"2023-11-26T07:17:56.640341+0000","last_clean":"2023-11-26T07:17:56.640341+0000","last_became_active":"2023-11-26T07:17:54.524255+0000","last_became_peered":"2023-11-26T07:17:54.524255+0000","last_unstale":"2023-11-26T07:17:56.640341+0000","last_undegraded":"2023-11-26T07:17:56.640341+0000","last_fullsized":"2023-11-26T07:17:56.640341+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:17:53.483873+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:17:53.483873+0000","last_clean_scrub_stamp":"2023-11-26T07:17:53.483873+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T11:16:25.801280+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-26T07:17:56.639652+0000","last_change":"2023-11-26T07:17:54.535310+0000","last_active":"2023-11-26T07:17:56.639652+0000","last_peered":"2023-11-26T07:17:56.639652+0000","last_clean":"2023-11-26T07:17:56.639652+0000","last_became_active":"2023-11-26T07:17:54.534986+0000","last_became_peered":"2023-11-26T07:17:54.534986+0000","last_unstale":"2023-11-26T07:17:56.639652+0000","last_undegraded":"2023-11-26T07:17:56.639652+0000","last_fullsized":"2023-11-26T07:17:56.639652+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:17:53.483873+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:17:53.483873+0000","last_clean_scrub_stamp":"2023-11-26T07:17:53.483873+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T15:18:17.309782+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-26T07:17:56.640026+0000","last_change":"2023-11-26T07:17:54.512408+0000","last_active":"2023-11-26T07:17:56.640026+0000","last_peered":"2023-11-26T07:17:56.640026+0000","last_clean":"2023-11-26T07:17:56.640026+0000","last_became_active":"2023-11-26T07:17:54.512005+0000","last_became_peered":"2023-11-26T07:17:54.512005+0000","last_unstale":"2023-11-26T07:17:56.640026+0000","last_undegraded":"2023-11-26T07:17:56.640026+0000","last_fullsized":"2023-11-26T07:17:56.640026+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:17:53.483873+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:17:53.483873+0000","last_clean_scrub_stamp":"2023-11-26T07:17:53.483873+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T09:07:53.947975+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-26T07:17:56.640756+0000","last_change":"2023-11-26T07:17:54.514825+0000","last_active":"2023-11-26T07:17:56.640756+0000","last_peered":"2023-11-26T07:17:56.640756+0000","last_clean":"2023-11-26T07:17:56.640756+0000","last_became_active":"2023-11-26T07:17:54.514502+0000","last_became_peered":"2023-11-26T07:17:54.514502+0000","last_unstale":"2023-11-26T07:17:56.640756+0000","last_undegraded":"2023-11-26T07:17:56.640756+0000","last_fullsized":"2023-11-26T07:17:56.640756+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:17:53.483873+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:17:53.483873+0000","last_clean_scrub_stamp":"2023-11-26T07:17:53.483873+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T07:50:04.472214+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-26T07:17:56.068691+0000","last_change":"2023-11-26T07:17:54.521737+0000","last_active":"2023-11-26T07:17:56.068691+0000","last_peered":"2023-11-26T07:17:56.068691+0000","last_clean":"2023-11-26T07:17:56.068691+0000","last_became_active":"2023-11-26T07:17:54.521387+0000","last_became_peered":"2023-11-26T07:17:54.521387+0000","last_unstale":"2023-11-26T07:17:56.068691+0000","last_undegraded":"2023-11-26T07:17:56.068691+0000","last_fullsized":"2023-11-26T07:17:56.068691+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:17:53.483873+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:17:53.483873+0000","last_clean_scrub_stamp":"2023-11-26T07:17:53.483873+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T18:33:32.653907+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-26T07:17:55.516698+0000","last_change":"2023-11-26T07:17:54.533852+0000","last_active":"2023-11-26T07:17:55.516698+0000","last_peered":"2023-11-26T07:17:55.516698+0000","last_clean":"2023-11-26T07:17:55.516698+0000","last_became_active":"2023-11-26T07:17:54.533532+0000","last_became_peered":"2023-11-26T07:17:54.533532+0000","last_unstale":"2023-11-26T07:17:55.516698+0000","last_undegraded":"2023-11-26T07:17:55.516698+0000","last_fullsized":"2023-11-26T07:17:55.516698+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:17:53.483873+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:17:53.483873+0000","last_clean_scrub_stamp":"2023-11-26T07:17:53.483873+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T14:45:51.557148+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-26T07:17:55.538556+0000","last_change":"2023-11-26T07:17:54.523130+0000","last_active":"2023-11-26T07:17:55.538556+0000","last_peered":"2023-11-26T07:17:55.538556+0000","last_clean":"2023-11-26T07:17:55.538556+0000","last_became_active":"2023-11-26T07:17:54.522879+0000","last_became_peered":"2023-11-26T07:17:54.522879+0000","last_unstale":"2023-11-26T07:17:55.538556+0000","last_undegraded":"2023-11-26T07:17:55.538556+0000","last_fullsized":"2023-11-26T07:17:55.538556+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:17:53.483873+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:17:53.483873+0000","last_clean_scrub_stamp":"2023-11-26T07:17:53.483873+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T11:16:25.801280+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-26T07:17:55.539151+0000","last_change":"2023-11-26T07:17:54.534352+0000","last_active":"2023-11-26T07:17:55.539151+0000","last_peered":"2023-11-26T07:17:55.539151+0000","last_clean":"2023-11-26T07:17:55.539151+0000","last_became_active":"2023-11-26T07:17:54.534004+0000","last_became_peered":"2023-11-26T07:17:54.534004+0000","last_unstale":"2023-11-26T07:17:55.539151+0000","last_undegraded":"2023-11-26T07:17:55.539151+0000","last_fullsized":"2023-11-26T07:17:55.539151+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:17:53.483873+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:17:53.483873+0000","last_clean_scrub_stamp":"2023-11-26T07:17:53.483873+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T18:27:00.305240+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-26T07:17:55.514964+0000","last_change":"2023-11-26T07:17:54.515085+0000","last_active":"2023-11-26T07:17:55.514964+0000","last_peered":"2023-11-26T07:17:55.514964+0000","last_clean":"2023-11-26T07:17:55.514964+0000","last_became_active":"2023-11-26T07:17:54.514782+0000","last_became_peered":"2023-11-26T07:17:54.514782+0000","last_unstale":"2023-11-26T07:17:55.514964+0000","last_undegraded":"2023-11-26T07:17:55.514964+0000","last_fullsized":"2023-11-26T07:17:55.514964+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:17:53.483873+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:17:53.483873+0000","last_clean_scrub_stamp":"2023-11-26T07:17:53.483873+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T13:45:34.765962+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-26T07:17:55.764203+0000","last_change":"2023-11-26T07:17:54.529137+0000","last_active":"2023-11-26T07:17:55.764203+0000","last_peered":"2023-11-26T07:17:55.764203+0000","last_clean":"2023-11-26T07:17:55.764203+0000","last_became_active":"2023-11-26T07:17:54.528496+0000","last_became_peered":"2023-11-26T07:17:54.528496+0000","last_unstale":"2023-11-26T07:17:55.764203+0000","last_undegraded":"2023-11-26T07:17:55.764203+0000","last_fullsized":"2023-11-26T07:17:55.764203+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:17:53.483873+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:17:53.483873+0000","last_clean_scrub_stamp":"2023-11-26T07:17:53.483873+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T17:27:24.347347+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-26T07:17:56.639612+0000","last_change":"2023-11-26T07:17:54.521609+0000","last_active":"2023-11-26T07:17:56.639612+0000","last_peered":"2023-11-26T07:17:56.639612+0000","last_clean":"2023-11-26T07:17:56.639612+0000","last_became_active":"2023-11-26T07:17:54.521298+0000","last_became_peered":"2023-11-26T07:17:54.521298+0000","last_unstale":"2023-11-26T07:17:56.639612+0000","last_undegraded":"2023-11-26T07:17:56.639612+0000","last_fullsized":"2023-11-26T07:17:56.639612+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:17:53.483873+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:17:53.483873+0000","last_clean_scrub_stamp":"2023-11-26T07:17:53.483873+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T08:42:29.375032+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-26T07:17:56.068908+0000","last_change":"2023-11-26T07:17:54.526550+0000","last_active":"2023-11-26T07:17:56.068908+0000","last_peered":"2023-11-26T07:17:56.068908+0000","last_clean":"2023-11-26T07:17:56.068908+0000","last_became_active":"2023-11-26T07:17:54.525639+0000","last_became_peered":"2023-11-26T07:17:54.525639+0000","last_unstale":"2023-11-26T07:17:56.068908+0000","last_undegraded":"2023-11-26T07:17:56.068908+0000","last_fullsized":"2023-11-26T07:17:56.068908+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:17:53.483873+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:17:53.483873+0000","last_clean_scrub_stamp":"2023-11-26T07:17:53.483873+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T07:50:04.472214+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-26T07:17:55.538640+0000","last_change":"2023-11-26T07:17:54.534209+0000","last_active":"2023-11-26T07:17:55.538640+0000","last_peered":"2023-11-26T07:17:55.538640+0000","last_clean":"2023-11-26T07:17:55.538640+0000","last_became_active":"2023-11-26T07:17:54.533932+0000","last_became_peered":"2023-11-26T07:17:54.533932+0000","last_unstale":"2023-11-26T07:17:55.538640+0000","last_undegraded":"2023-11-26T07:17:55.538640+0000","last_fullsized":"2023-11-26T07:17:55.538640+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:17:53.483873+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:17:53.483873+0000","last_clean_scrub_stamp":"2023-11-26T07:17:53.483873+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T18:02:15.969376+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-26T07:17:56.640577+0000","last_change":"2023-11-26T07:17:54.526358+0000","last_active":"2023-11-26T07:17:56.640577+0000","last_peered":"2023-11-26T07:17:56.640577+0000","last_clean":"2023-11-26T07:17:56.640577+0000","last_became_active":"2023-11-26T07:17:54.525756+0000","last_became_peered":"2023-11-26T07:17:54.525756+0000","last_unstale":"2023-11-26T07:17:56.640577+0000","last_undegraded":"2023-11-26T07:17:56.640577+0000","last_fullsized":"2023-11-26T07:17:56.640577+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:17:53.483873+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:17:53.483873+0000","last_clean_scrub_stamp":"2023-11-26T07:17:53.483873+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T15:18:17.309782+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-26T07:17:55.538626+0000","last_change":"2023-11-26T07:17:54.526328+0000","last_active":"2023-11-26T07:17:55.538626+0000","last_peered":"2023-11-26T07:17:55.538626+0000","last_clean":"2023-11-26T07:17:55.538626+0000","last_became_active":"2023-11-26T07:17:54.526026+0000","last_became_peered":"2023-11-26T07:17:54.526026+0000","last_unstale":"2023-11-26T07:17:55.538626+0000","last_undegraded":"2023-11-26T07:17:55.538626+0000","last_fullsized":"2023-11-26T07:17:55.538626+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:17:53.483873+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:17:53.483873+0000","last_clean_scrub_stamp":"2023-11-26T07:17:53.483873+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T14:45:51.557148+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-26T07:17:56.641174+0000","last_change":"2023-11-26T07:17:54.521949+0000","last_active":"2023-11-26T07:17:56.641174+0000","last_peered":"2023-11-26T07:17:56.641174+0000","last_clean":"2023-11-26T07:17:56.641174+0000","last_became_active":"2023-11-26T07:17:54.521649+0000","last_became_peered":"2023-11-26T07:17:54.521649+0000","last_unstale":"2023-11-26T07:17:56.641174+0000","last_undegraded":"2023-11-26T07:17:56.641174+0000","last_fullsized":"2023-11-26T07:17:56.641174+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:17:53.483873+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:17:53.483873+0000","last_clean_scrub_stamp":"2023-11-26T07:17:53.483873+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T17:14:14.930365+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-26T07:17:56.640304+0000","last_change":"2023-11-26T07:17:54.526758+0000","last_active":"2023-11-26T07:17:56.640304+0000","last_peered":"2023-11-26T07:17:56.640304+0000","last_clean":"2023-11-26T07:17:56.640304+0000","last_became_active":"2023-11-26T07:17:54.525563+0000","last_became_peered":"2023-11-26T07:17:54.525563+0000","last_unstale":"2023-11-26T07:17:56.640304+0000","last_undegraded":"2023-11-26T07:17:56.640304+0000","last_fullsized":"2023-11-26T07:17:56.640304+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:17:53.483873+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:17:53.483873+0000","last_clean_scrub_stamp":"2023-11-26T07:17:53.483873+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T07:50:04.472214+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-26T07:17:55.516791+0000","last_change":"2023-11-26T07:17:54.520560+0000","last_active":"2023-11-26T07:17:55.516791+0000","last_peered":"2023-11-26T07:17:55.516791+0000","last_clean":"2023-11-26T07:17:55.516791+0000","last_became_active":"2023-11-26T07:17:54.520377+0000","last_became_peered":"2023-11-26T07:17:54.520377+0000","last_unstale":"2023-11-26T07:17:55.516791+0000","last_undegraded":"2023-11-26T07:17:55.516791+0000","last_fullsized":"2023-11-26T07:17:55.516791+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:17:53.483873+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:17:53.483873+0000","last_clean_scrub_stamp":"2023-11-26T07:17:53.483873+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T18:02:15.969376+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-26T07:17:56.640998+0000","last_change":"2023-11-26T07:17:54.527041+0000","last_active":"2023-11-26T07:17:56.640998+0000","last_peered":"2023-11-26T07:17:56.640998+0000","last_clean":"2023-11-26T07:17:56.640998+0000","last_became_active":"2023-11-26T07:17:54.526391+0000","last_became_peered":"2023-11-26T07:17:54.526391+0000","last_unstale":"2023-11-26T07:17:56.640998+0000","last_undegraded":"2023-11-26T07:17:56.640998+0000","last_fullsized":"2023-11-26T07:17:56.640998+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:17:53.483873+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:17:53.483873+0000","last_clean_scrub_stamp":"2023-11-26T07:17:53.483873+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T18:02:15.969376+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-26T07:17:56.068828+0000","last_change":"2023-11-26T07:17:54.512264+0000","last_active":"2023-11-26T07:17:56.068828+0000","last_peered":"2023-11-26T07:17:56.068828+0000","last_clean":"2023-11-26T07:17:56.068828+0000","last_became_active":"2023-11-26T07:17:54.511963+0000","last_became_peered":"2023-11-26T07:17:54.511963+0000","last_unstale":"2023-11-26T07:17:56.068828+0000","last_undegraded":"2023-11-26T07:17:56.068828+0000","last_fullsized":"2023-11-26T07:17:56.068828+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:17:53.483873+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:17:53.483873+0000","last_clean_scrub_stamp":"2023-11-26T07:17:53.483873+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T08:42:29.375032+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-26T07:17:55.538793+0000","last_change":"2023-11-26T07:17:54.528162+0000","last_active":"2023-11-26T07:17:55.538793+0000","last_peered":"2023-11-26T07:17:55.538793+0000","last_clean":"2023-11-26T07:17:55.538793+0000","last_became_active":"2023-11-26T07:17:54.527657+0000","last_became_peered":"2023-11-26T07:17:54.527657+0000","last_unstale":"2023-11-26T07:17:55.538793+0000","last_undegraded":"2023-11-26T07:17:55.538793+0000","last_fullsized":"2023-11-26T07:17:55.538793+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:17:53.483873+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:17:53.483873+0000","last_clean_scrub_stamp":"2023-11-26T07:17:53.483873+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T14:49:27.883374+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-26T07:17:55.514728+0000","last_change":"2023-11-26T07:17:54.519107+0000","last_active":"2023-11-26T07:17:55.514728+0000","last_peered":"2023-11-26T07:17:55.514728+0000","last_clean":"2023-11-26T07:17:55.514728+0000","last_became_active":"2023-11-26T07:17:54.518822+0000","last_became_peered":"2023-11-26T07:17:54.518822+0000","last_unstale":"2023-11-26T07:17:55.514728+0000","last_undegraded":"2023-11-26T07:17:55.514728+0000","last_fullsized":"2023-11-26T07:17:55.514728+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:17:53.483873+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:17:53.483873+0000","last_clean_scrub_stamp":"2023-11-26T07:17:53.483873+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T09:07:53.947975+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-26T07:17:56.639246+0000","last_change":"2023-11-26T07:17:54.513368+0000","last_active":"2023-11-26T07:17:56.639246+0000","last_peered":"2023-11-26T07:17:56.639246+0000","last_clean":"2023-11-26T07:17:56.639246+0000","last_became_active":"2023-11-26T07:17:54.511290+0000","last_became_peered":"2023-11-26T07:17:54.511290+0000","last_unstale":"2023-11-26T07:17:56.639246+0000","last_undegraded":"2023-11-26T07:17:56.639246+0000","last_fullsized":"2023-11-26T07:17:56.639246+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:17:53.483873+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:17:53.483873+0000","last_clean_scrub_stamp":"2023-11-26T07:17:53.483873+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T18:33:32.653907+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-26T07:17:56.639579+0000","last_change":"2023-11-26T07:17:54.513190+0000","last_active":"2023-11-26T07:17:56.639579+0000","last_peered":"2023-11-26T07:17:56.639579+0000","last_clean":"2023-11-26T07:17:56.639579+0000","last_became_active":"2023-11-26T07:17:54.512872+0000","last_became_peered":"2023-11-26T07:17:54.512872+0000","last_unstale":"2023-11-26T07:17:56.639579+0000","last_undegraded":"2023-11-26T07:17:56.639579+0000","last_fullsized":"2023-11-26T07:17:56.639579+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:17:53.483873+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:17:53.483873+0000","last_clean_scrub_stamp":"2023-11-26T07:17:53.483873+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T12:32:48.249974+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1b","version":"19'1","reported_seq":13,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-26T07:17:56.640973+0000","last_change":"2023-11-26T07:17:53.525183+0000","last_active":"2023-11-26T07:17:56.640973+0000","last_peered":"2023-11-26T07:17:56.640973+0000","last_clean":"2023-11-26T07:17:56.640973+0000","last_became_active":"2023-11-26T07:17:53.524904+0000","last_became_peered":"2023-11-26T07:17:53.524904+0000","last_unstale":"2023-11-26T07:17:56.640973+0000","last_undegraded":"2023-11-26T07:17:56.640973+0000","last_fullsized":"2023-11-26T07:17:56.640973+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:17:52.477276+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:17:52.477276+0000","last_clean_scrub_stamp":"2023-11-26T07:17:52.477276+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T13:59:52.977195+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-26T07:17:56.068970+0000","last_change":"2023-11-26T07:17:54.522974+0000","last_active":"2023-11-26T07:17:56.068970+0000","last_peered":"2023-11-26T07:17:56.068970+0000","last_clean":"2023-11-26T07:17:56.068970+0000","last_became_active":"2023-11-26T07:17:54.521283+0000","last_became_peered":"2023-11-26T07:17:54.521283+0000","last_unstale":"2023-11-26T07:17:56.068970+0000","last_undegraded":"2023-11-26T07:17:56.068970+0000","last_fullsized":"2023-11-26T07:17:56.068970+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:17:53.483873+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:17:53.483873+0000","last_clean_scrub_stamp":"2023-11-26T07:17:53.483873+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T11:16:25.801280+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1a","version":"19'1","reported_seq":13,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-26T07:17:56.639442+0000","last_change":"2023-11-26T07:17:53.516099+0000","last_active":"2023-11-26T07:17:56.639442+0000","last_peered":"2023-11-26T07:17:56.639442+0000","last_clean":"2023-11-26T07:17:56.639442+0000","last_became_active":"2023-11-26T07:17:53.515687+0000","last_became_peered":"2023-11-26T07:17:53.515687+0000","last_unstale":"2023-11-26T07:17:56.639442+0000","last_undegraded":"2023-11-26T07:17:56.639442+0000","last_fullsized":"2023-11-26T07:17:56.639442+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:17:52.477276+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:17:52.477276+0000","last_clean_scrub_stamp":"2023-11-26T07:17:52.477276+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T09:07:52.941378+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.19","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-26T07:17:56.640738+0000","last_change":"2023-11-26T07:17:53.523159+0000","last_active":"2023-11-26T07:17:56.640738+0000","last_peered":"2023-11-26T07:17:56.640738+0000","last_clean":"2023-11-26T07:17:56.640738+0000","last_became_active":"2023-11-26T07:17:53.522824+0000","last_became_peered":"2023-11-26T07:17:53.522824+0000","last_unstale":"2023-11-26T07:17:56.640738+0000","last_undegraded":"2023-11-26T07:17:56.640738+0000","last_fullsized":"2023-11-26T07:17:56.640738+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:17:52.477276+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:17:52.477276+0000","last_clean_scrub_stamp":"2023-11-26T07:17:52.477276+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T13:45:33.759365+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-26T07:17:56.641065+0000","last_change":"2023-11-26T07:17:54.515953+0000","last_active":"2023-11-26T07:17:56.641065+0000","last_peered":"2023-11-26T07:17:56.641065+0000","last_clean":"2023-11-26T07:17:56.641065+0000","last_became_active":"2023-11-26T07:17:54.515662+0000","last_became_peered":"2023-11-26T07:17:54.515662+0000","last_unstale":"2023-11-26T07:17:56.641065+0000","last_undegraded":"2023-11-26T07:17:56.641065+0000","last_fullsized":"2023-11-26T07:17:56.641065+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:17:53.483873+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:17:53.483873+0000","last_clean_scrub_stamp":"2023-11-26T07:17:53.483873+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T10:18:52.982852+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.18","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-26T07:17:55.538849+0000","last_change":"2023-11-26T07:17:53.516308+0000","last_active":"2023-11-26T07:17:55.538849+0000","last_peered":"2023-11-26T07:17:55.538849+0000","last_clean":"2023-11-26T07:17:55.538849+0000","last_became_active":"2023-11-26T07:17:53.516027+0000","last_became_peered":"2023-11-26T07:17:53.516027+0000","last_unstale":"2023-11-26T07:17:55.538849+0000","last_undegraded":"2023-11-26T07:17:55.538849+0000","last_fullsized":"2023-11-26T07:17:55.538849+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:17:52.477276+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:17:52.477276+0000","last_clean_scrub_stamp":"2023-11-26T07:17:52.477276+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T08:42:28.368435+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-26T07:17:55.516385+0000","last_change":"2023-11-26T07:17:54.521790+0000","last_active":"2023-11-26T07:17:55.516385+0000","last_peered":"2023-11-26T07:17:55.516385+0000","last_clean":"2023-11-26T07:17:55.516385+0000","last_became_active":"2023-11-26T07:17:54.521475+0000","last_became_peered":"2023-11-26T07:17:54.521475+0000","last_unstale":"2023-11-26T07:17:55.516385+0000","last_undegraded":"2023-11-26T07:17:55.516385+0000","last_fullsized":"2023-11-26T07:17:55.516385+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:17:53.483873+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:17:53.483873+0000","last_clean_scrub_stamp":"2023-11-26T07:17:53.483873+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T15:18:17.309782+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.17","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-26T07:17:56.640306+0000","last_change":"2023-11-26T07:17:53.514460+0000","last_active":"2023-11-26T07:17:56.640306+0000","last_peered":"2023-11-26T07:17:56.640306+0000","last_clean":"2023-11-26T07:17:56.640306+0000","last_became_active":"2023-11-26T07:17:53.514170+0000","last_became_peered":"2023-11-26T07:17:53.514170+0000","last_unstale":"2023-11-26T07:17:56.640306+0000","last_undegraded":"2023-11-26T07:17:56.640306+0000","last_fullsized":"2023-11-26T07:17:56.640306+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:17:52.477276+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:17:52.477276+0000","last_clean_scrub_stamp":"2023-11-26T07:17:52.477276+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T15:47:35.416235+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-26T07:17:55.538995+0000","last_change":"2023-11-26T07:17:54.533776+0000","last_active":"2023-11-26T07:17:55.538995+0000","last_peered":"2023-11-26T07:17:55.538995+0000","last_clean":"2023-11-26T07:17:55.538995+0000","last_became_active":"2023-11-26T07:17:54.533279+0000","last_became_peered":"2023-11-26T07:17:54.533279+0000","last_unstale":"2023-11-26T07:17:55.538995+0000","last_undegraded":"2023-11-26T07:17:55.538995+0000","last_fullsized":"2023-11-26T07:17:55.538995+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:17:53.483873+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:17:53.483873+0000","last_clean_scrub_stamp":"2023-11-26T07:17:53.483873+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T15:38:12.078699+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.16","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-26T07:17:55.539107+0000","last_change":"2023-11-26T07:17:53.520128+0000","last_active":"2023-11-26T07:17:55.539107+0000","last_peered":"2023-11-26T07:17:55.539107+0000","last_clean":"2023-11-26T07:17:55.539107+0000","last_became_active":"2023-11-26T07:17:53.519399+0000","last_became_peered":"2023-11-26T07:17:53.519399+0000","last_unstale":"2023-11-26T07:17:55.539107+0000","last_undegraded":"2023-11-26T07:17:55.539107+0000","last_fullsized":"2023-11-26T07:17:55.539107+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:17:52.477276+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:17:52.477276+0000","last_clean_scrub_stamp":"2023-11-26T07:17:52.477276+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T15:18:16.303185+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-26T07:17:56.068974+0000","last_change":"2023-11-26T07:17:54.528621+0000","last_active":"2023-11-26T07:17:56.068974+0000","last_peered":"2023-11-26T07:17:56.068974+0000","last_clean":"2023-11-26T07:17:56.068974+0000","last_became_active":"2023-11-26T07:17:54.528333+0000","last_became_peered":"2023-11-26T07:17:54.528333+0000","last_unstale":"2023-11-26T07:17:56.068974+0000","last_undegraded":"2023-11-26T07:17:56.068974+0000","last_fullsized":"2023-11-26T07:17:56.068974+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:17:53.483873+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:17:53.483873+0000","last_clean_scrub_stamp":"2023-11-26T07:17:53.483873+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T13:18:15.521322+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.15","version":"19'3","reported_seq":15,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-26T07:17:55.516929+0000","last_change":"2023-11-26T07:17:53.515987+0000","last_active":"2023-11-26T07:17:55.516929+0000","last_peered":"2023-11-26T07:17:55.516929+0000","last_clean":"2023-11-26T07:17:55.516929+0000","last_became_active":"2023-11-26T07:17:53.515360+0000","last_became_peered":"2023-11-26T07:17:53.515360+0000","last_unstale":"2023-11-26T07:17:55.516929+0000","last_undegraded":"2023-11-26T07:17:55.516929+0000","last_fullsized":"2023-11-26T07:17:55.516929+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:17:52.477276+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:17:52.477276+0000","last_clean_scrub_stamp":"2023-11-26T07:17:52.477276+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T19:07:41.542569+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-26T07:17:56.640029+0000","last_change":"2023-11-26T07:17:54.530537+0000","last_active":"2023-11-26T07:17:56.640029+0000","last_peered":"2023-11-26T07:17:56.640029+0000","last_clean":"2023-11-26T07:17:56.640029+0000","last_became_active":"2023-11-26T07:17:54.530074+0000","last_became_peered":"2023-11-26T07:17:54.530074+0000","last_unstale":"2023-11-26T07:17:56.640029+0000","last_undegraded":"2023-11-26T07:17:56.640029+0000","last_fullsized":"2023-11-26T07:17:56.640029+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:17:53.483873+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:17:53.483873+0000","last_clean_scrub_stamp":"2023-11-26T07:17:53.483873+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T13:45:34.765962+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.14","version":"19'6","reported_seq":18,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-26T07:17:56.639634+0000","last_change":"2023-11-26T07:17:53.522850+0000","last_active":"2023-11-26T07:17:56.639634+0000","last_peered":"2023-11-26T07:17:56.639634+0000","last_clean":"2023-11-26T07:17:56.639634+0000","last_became_active":"2023-11-26T07:17:53.522377+0000","last_became_peered":"2023-11-26T07:17:53.522377+0000","last_unstale":"2023-11-26T07:17:56.639634+0000","last_undegraded":"2023-11-26T07:17:56.639634+0000","last_fullsized":"2023-11-26T07:17:56.639634+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:17:52.477276+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:17:52.477276+0000","last_clean_scrub_stamp":"2023-11-26T07:17:52.477276+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T12:51:16.332936+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-26T07:17:55.763972+0000","last_change":"2023-11-26T07:17:54.529408+0000","last_active":"2023-11-26T07:17:55.763972+0000","last_peered":"2023-11-26T07:17:55.763972+0000","last_clean":"2023-11-26T07:17:55.763972+0000","last_became_active":"2023-11-26T07:17:54.528825+0000","last_became_peered":"2023-11-26T07:17:54.528825+0000","last_unstale":"2023-11-26T07:17:55.763972+0000","last_undegraded":"2023-11-26T07:17:55.763972+0000","last_fullsized":"2023-11-26T07:17:55.763972+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:17:53.483873+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:17:53.483873+0000","last_clean_scrub_stamp":"2023-11-26T07:17:53.483873+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T12:21:10.834625+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.13","version":"19'2","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-26T07:17:56.068998+0000","last_change":"2023-11-26T07:17:53.517650+0000","last_active":"2023-11-26T07:17:56.068998+0000","last_peered":"2023-11-26T07:17:56.068998+0000","last_clean":"2023-11-26T07:17:56.068998+0000","last_became_active":"2023-11-26T07:17:53.517349+0000","last_became_peered":"2023-11-26T07:17:53.517349+0000","last_unstale":"2023-11-26T07:17:56.068998+0000","last_undegraded":"2023-11-26T07:17:56.068998+0000","last_fullsized":"2023-11-26T07:17:56.068998+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:17:52.477276+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:17:52.477276+0000","last_clean_scrub_stamp":"2023-11-26T07:17:52.477276+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T12:51:16.332936+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-26T07:17:56.068804+0000","last_change":"2023-11-26T07:17:54.531742+0000","last_active":"2023-11-26T07:17:56.068804+0000","last_peered":"2023-11-26T07:17:56.068804+0000","last_clean":"2023-11-26T07:17:56.068804+0000","last_became_active":"2023-11-26T07:17:54.531136+0000","last_became_peered":"2023-11-26T07:17:54.531136+0000","last_unstale":"2023-11-26T07:17:56.068804+0000","last_undegraded":"2023-11-26T07:17:56.068804+0000","last_fullsized":"2023-11-26T07:17:56.068804+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:17:53.483873+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:17:53.483873+0000","last_clean_scrub_stamp":"2023-11-26T07:17:53.483873+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T16:07:46.594040+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.12","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-26T07:17:55.538769+0000","last_change":"2023-11-26T07:17:53.522311+0000","last_active":"2023-11-26T07:17:55.538769+0000","last_peered":"2023-11-26T07:17:55.538769+0000","last_clean":"2023-11-26T07:17:55.538769+0000","last_became_active":"2023-11-26T07:17:53.522017+0000","last_became_peered":"2023-11-26T07:17:53.522017+0000","last_unstale":"2023-11-26T07:17:55.538769+0000","last_undegraded":"2023-11-26T07:17:55.538769+0000","last_fullsized":"2023-11-26T07:17:55.538769+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:17:52.477276+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:17:52.477276+0000","last_clean_scrub_stamp":"2023-11-26T07:17:52.477276+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T09:53:28.237406+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-26T07:17:55.763876+0000","last_change":"2023-11-26T07:17:54.531973+0000","last_active":"2023-11-26T07:17:55.763876+0000","last_peered":"2023-11-26T07:17:55.763876+0000","last_clean":"2023-11-26T07:17:55.763876+0000","last_became_active":"2023-11-26T07:17:54.531710+0000","last_became_peered":"2023-11-26T07:17:54.531710+0000","last_unstale":"2023-11-26T07:17:55.763876+0000","last_undegraded":"2023-11-26T07:17:55.763876+0000","last_fullsized":"2023-11-26T07:17:55.763876+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:17:53.483873+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:17:53.483873+0000","last_clean_scrub_stamp":"2023-11-26T07:17:53.483873+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T15:38:12.078699+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.11","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-26T07:17:56.639192+0000","last_change":"2023-11-26T07:17:53.519520+0000","last_active":"2023-11-26T07:17:56.639192+0000","last_peered":"2023-11-26T07:17:56.639192+0000","last_clean":"2023-11-26T07:17:56.639192+0000","last_became_active":"2023-11-26T07:17:53.516001+0000","last_became_peered":"2023-11-26T07:17:53.516001+0000","last_unstale":"2023-11-26T07:17:56.639192+0000","last_undegraded":"2023-11-26T07:17:56.639192+0000","last_fullsized":"2023-11-26T07:17:56.639192+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:17:52.477276+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:17:52.477276+0000","last_clean_scrub_stamp":"2023-11-26T07:17:52.477276+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T19:07:41.542569+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-26T07:17:56.639395+0000","last_change":"2023-11-26T07:17:54.511603+0000","last_active":"2023-11-26T07:17:56.639395+0000","last_peered":"2023-11-26T07:17:56.639395+0000","last_clean":"2023-11-26T07:17:56.639395+0000","last_became_active":"2023-11-26T07:17:54.510939+0000","last_became_peered":"2023-11-26T07:17:54.510939+0000","last_unstale":"2023-11-26T07:17:56.639395+0000","last_undegraded":"2023-11-26T07:17:56.639395+0000","last_fullsized":"2023-11-26T07:17:56.639395+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:17:53.483873+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:17:53.483873+0000","last_clean_scrub_stamp":"2023-11-26T07:17:53.483873+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T09:53:29.244002+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.10","version":"19'8","reported_seq":20,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-26T07:17:55.514684+0000","last_change":"2023-11-26T07:17:53.506514+0000","last_active":"2023-11-26T07:17:55.514684+0000","last_peered":"2023-11-26T07:17:55.514684+0000","last_clean":"2023-11-26T07:17:55.514684+0000","last_became_active":"2023-11-26T07:17:53.506081+0000","last_became_peered":"2023-11-26T07:17:53.506081+0000","last_unstale":"2023-11-26T07:17:55.514684+0000","last_undegraded":"2023-11-26T07:17:55.514684+0000","last_fullsized":"2023-11-26T07:17:55.514684+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:17:52.477276+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:17:52.477276+0000","last_clean_scrub_stamp":"2023-11-26T07:17:52.477276+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T09:43:57.945669+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-26T07:17:55.763817+0000","last_change":"2023-11-26T07:17:54.532538+0000","last_active":"2023-11-26T07:17:55.763817+0000","last_peered":"2023-11-26T07:17:55.763817+0000","last_clean":"2023-11-26T07:17:55.763817+0000","last_became_active":"2023-11-26T07:17:54.531186+0000","last_became_peered":"2023-11-26T07:17:54.531186+0000","last_unstale":"2023-11-26T07:17:55.763817+0000","last_undegraded":"2023-11-26T07:17:55.763817+0000","last_fullsized":"2023-11-26T07:17:55.763817+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:17:53.483873+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:17:53.483873+0000","last_clean_scrub_stamp":"2023-11-26T07:17:53.483873+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T15:49:39.889795+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.f","version":"19'1","reported_seq":13,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-26T07:17:56.640634+0000","last_change":"2023-11-26T07:17:53.509212+0000","last_active":"2023-11-26T07:17:56.640634+0000","last_peered":"2023-11-26T07:17:56.640634+0000","last_clean":"2023-11-26T07:17:56.640634+0000","last_became_active":"2023-11-26T07:17:53.508602+0000","last_became_peered":"2023-11-26T07:17:53.508602+0000","last_unstale":"2023-11-26T07:17:56.640634+0000","last_undegraded":"2023-11-26T07:17:56.640634+0000","last_fullsized":"2023-11-26T07:17:56.640634+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:17:52.477276+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:17:52.477276+0000","last_clean_scrub_stamp":"2023-11-26T07:17:52.477276+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T10:16:37.754573+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-26T07:17:55.764257+0000","last_change":"2023-11-26T07:17:54.531723+0000","last_active":"2023-11-26T07:17:55.764257+0000","last_peered":"2023-11-26T07:17:55.764257+0000","last_clean":"2023-11-26T07:17:55.764257+0000","last_became_active":"2023-11-26T07:17:54.531321+0000","last_became_peered":"2023-11-26T07:17:54.531321+0000","last_unstale":"2023-11-26T07:17:55.764257+0000","last_undegraded":"2023-11-26T07:17:55.764257+0000","last_fullsized":"2023-11-26T07:17:55.764257+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:17:53.483873+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:17:53.483873+0000","last_clean_scrub_stamp":"2023-11-26T07:17:53.483873+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T15:19:56.111921+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.e","version":"19'2","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-26T07:17:55.514998+0000","last_change":"2023-11-26T07:17:53.524243+0000","last_active":"2023-11-26T07:17:55.514998+0000","last_peered":"2023-11-26T07:17:55.514998+0000","last_clean":"2023-11-26T07:17:55.514998+0000","last_became_active":"2023-11-26T07:17:53.523933+0000","last_became_peered":"2023-11-26T07:17:53.523933+0000","last_unstale":"2023-11-26T07:17:55.514998+0000","last_undegraded":"2023-11-26T07:17:55.514998+0000","last_fullsized":"2023-11-26T07:17:55.514998+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:17:52.477276+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:17:52.477276+0000","last_clean_scrub_stamp":"2023-11-26T07:17:52.477276+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T15:44:14.149131+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-26T07:17:55.764439+0000","last_change":"2023-11-26T07:17:54.531576+0000","last_active":"2023-11-26T07:17:55.764439+0000","last_peered":"2023-11-26T07:17:55.764439+0000","last_clean":"2023-11-26T07:17:55.764439+0000","last_became_active":"2023-11-26T07:17:54.531261+0000","last_became_peered":"2023-11-26T07:17:54.531261+0000","last_unstale":"2023-11-26T07:17:55.764439+0000","last_undegraded":"2023-11-26T07:17:55.764439+0000","last_fullsized":"2023-11-26T07:17:55.764439+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:17:53.483873+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:17:53.483873+0000","last_clean_scrub_stamp":"2023-11-26T07:17:53.483873+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T11:46:17.783756+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.d","version":"19'1","reported_seq":13,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-26T07:17:55.516854+0000","last_change":"2023-11-26T07:17:53.512500+0000","last_active":"2023-11-26T07:17:55.516854+0000","last_peered":"2023-11-26T07:17:55.516854+0000","last_clean":"2023-11-26T07:17:55.516854+0000","last_became_active":"2023-11-26T07:17:53.512082+0000","last_became_peered":"2023-11-26T07:17:53.512082+0000","last_unstale":"2023-11-26T07:17:55.516854+0000","last_undegraded":"2023-11-26T07:17:55.516854+0000","last_fullsized":"2023-11-26T07:17:55.516854+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:17:52.477276+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:17:52.477276+0000","last_clean_scrub_stamp":"2023-11-26T07:17:52.477276+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T18:33:31.647310+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-26T07:17:55.539068+0000","last_change":"2023-11-26T07:17:54.528819+0000","last_active":"2023-11-26T07:17:55.539068+0000","last_peered":"2023-11-26T07:17:55.539068+0000","last_clean":"2023-11-26T07:17:55.539068+0000","last_became_active":"2023-11-26T07:17:54.528290+0000","last_became_peered":"2023-11-26T07:17:54.528290+0000","last_unstale":"2023-11-26T07:17:55.539068+0000","last_undegraded":"2023-11-26T07:17:55.539068+0000","last_fullsized":"2023-11-26T07:17:55.539068+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:17:53.483873+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:17:53.483873+0000","last_clean_scrub_stamp":"2023-11-26T07:17:53.483873+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T12:21:10.834625+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.c","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-26T07:17:55.514878+0000","last_change":"2023-11-26T07:17:53.514524+0000","last_active":"2023-11-26T07:17:55.514878+0000","last_peered":"2023-11-26T07:17:55.514878+0000","last_clean":"2023-11-26T07:17:55.514878+0000","last_became_active":"2023-11-26T07:17:53.514070+0000","last_became_peered":"2023-11-26T07:17:53.514070+0000","last_unstale":"2023-11-26T07:17:55.514878+0000","last_undegraded":"2023-11-26T07:17:55.514878+0000","last_fullsized":"2023-11-26T07:17:55.514878+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:17:52.477276+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:17:52.477276+0000","last_clean_scrub_stamp":"2023-11-26T07:17:52.477276+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T15:47:35.416235+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-26T07:17:55.763997+0000","last_change":"2023-11-26T07:17:54.535603+0000","last_active":"2023-11-26T07:17:55.763997+0000","last_peered":"2023-11-26T07:17:55.763997+0000","last_clean":"2023-11-26T07:17:55.763997+0000","last_became_active":"2023-11-26T07:17:54.535343+0000","last_became_peered":"2023-11-26T07:17:54.535343+0000","last_unstale":"2023-11-26T07:17:55.763997+0000","last_undegraded":"2023-11-26T07:17:55.763997+0000","last_fullsized":"2023-11-26T07:17:55.763997+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:17:53.483873+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:17:53.483873+0000","last_clean_scrub_stamp":"2023-11-26T07:17:53.483873+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T18:27:00.305240+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.0","version":"15'117","reported_seq":165,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-11-26T07:17:54.507822+0000","last_change":"2023-11-26T07:17:43.827802+0000","last_active":"2023-11-26T07:17:54.507822+0000","last_peered":"2023-11-26T07:17:54.507822+0000","last_clean":"2023-11-26T07:17:54.507822+0000","last_became_active":"2023-11-26T07:17:43.827308+0000","last_became_peered":"2023-11-26T07:17:43.827308+0000","last_unstale":"2023-11-26T07:17:54.507822+0000","last_undegraded":"2023-11-26T07:17:54.507822+0000","last_fullsized":"2023-11-26T07:17:54.507822+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:17:42.160945+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:17:42.160945+0000","last_clean_scrub_stamp":"2023-11-26T07:17:42.160945+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T08:17:54.167366+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-26T07:17:55.538622+0000","last_change":"2023-11-26T07:17:53.514397+0000","last_active":"2023-11-26T07:17:55.538622+0000","last_peered":"2023-11-26T07:17:55.538622+0000","last_clean":"2023-11-26T07:17:55.538622+0000","last_became_active":"2023-11-26T07:17:53.513876+0000","last_became_peered":"2023-11-26T07:17:53.513876+0000","last_unstale":"2023-11-26T07:17:55.538622+0000","last_undegraded":"2023-11-26T07:17:55.538622+0000","last_fullsized":"2023-11-26T07:17:55.538622+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:17:52.477276+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:17:52.477276+0000","last_clean_scrub_stamp":"2023-11-26T07:17:52.477276+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T18:33:31.647310+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-26T07:17:56.640762+0000","last_change":"2023-11-26T07:17:54.522327+0000","last_active":"2023-11-26T07:17:56.640762+0000","last_peered":"2023-11-26T07:17:56.640762+0000","last_clean":"2023-11-26T07:17:56.640762+0000","last_became_active":"2023-11-26T07:17:54.522058+0000","last_became_peered":"2023-11-26T07:17:54.522058+0000","last_unstale":"2023-11-26T07:17:56.640762+0000","last_undegraded":"2023-11-26T07:17:56.640762+0000","last_fullsized":"2023-11-26T07:17:56.640762+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:17:53.483873+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:17:53.483873+0000","last_clean_scrub_stamp":"2023-11-26T07:17:53.483873+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T11:16:25.801280+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-26T07:17:55.763573+0000","last_change":"2023-11-26T07:17:53.522494+0000","last_active":"2023-11-26T07:17:55.763573+0000","last_peered":"2023-11-26T07:17:55.763573+0000","last_clean":"2023-11-26T07:17:55.763573+0000","last_became_active":"2023-11-26T07:17:53.522221+0000","last_became_peered":"2023-11-26T07:17:53.522221+0000","last_unstale":"2023-11-26T07:17:55.763573+0000","last_undegraded":"2023-11-26T07:17:55.763573+0000","last_fullsized":"2023-11-26T07:17:55.763573+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:17:52.477276+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:17:52.477276+0000","last_clean_scrub_stamp":"2023-11-26T07:17:52.477276+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T13:45:33.759365+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-26T07:17:56.068604+0000","last_change":"2023-11-26T07:17:54.505916+0000","last_active":"2023-11-26T07:17:56.068604+0000","last_peered":"2023-11-26T07:17:56.068604+0000","last_clean":"2023-11-26T07:17:56.068604+0000","last_became_active":"2023-11-26T07:17:54.505618+0000","last_became_peered":"2023-11-26T07:17:54.505618+0000","last_unstale":"2023-11-26T07:17:56.068604+0000","last_undegraded":"2023-11-26T07:17:56.068604+0000","last_fullsized":"2023-11-26T07:17:56.068604+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:17:53.483873+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:17:53.483873+0000","last_clean_scrub_stamp":"2023-11-26T07:17:53.483873+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T19:07:42.549166+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-26T07:17:55.514749+0000","last_change":"2023-11-26T07:17:53.523403+0000","last_active":"2023-11-26T07:17:55.514749+0000","last_peered":"2023-11-26T07:17:55.514749+0000","last_clean":"2023-11-26T07:17:55.514749+0000","last_became_active":"2023-11-26T07:17:53.523061+0000","last_became_peered":"2023-11-26T07:17:53.523061+0000","last_unstale":"2023-11-26T07:17:55.514749+0000","last_undegraded":"2023-11-26T07:17:55.514749+0000","last_fullsized":"2023-11-26T07:17:55.514749+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:17:52.477276+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:17:52.477276+0000","last_clean_scrub_stamp":"2023-11-26T07:17:52.477276+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T09:18:47.261084+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-26T07:17:55.516400+0000","last_change":"2023-11-26T07:17:54.519128+0000","last_active":"2023-11-26T07:17:55.516400+0000","last_peered":"2023-11-26T07:17:55.516400+0000","last_clean":"2023-11-26T07:17:55.516400+0000","last_became_active":"2023-11-26T07:17:54.518918+0000","last_became_peered":"2023-11-26T07:17:54.518918+0000","last_unstale":"2023-11-26T07:17:55.516400+0000","last_undegraded":"2023-11-26T07:17:55.516400+0000","last_fullsized":"2023-11-26T07:17:55.516400+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:17:53.483873+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:17:53.483873+0000","last_clean_scrub_stamp":"2023-11-26T07:17:53.483873+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T07:50:04.472214+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-26T07:17:55.538911+0000","last_change":"2023-11-26T07:17:53.517034+0000","last_active":"2023-11-26T07:17:55.538911+0000","last_peered":"2023-11-26T07:17:55.538911+0000","last_clean":"2023-11-26T07:17:55.538911+0000","last_became_active":"2023-11-26T07:17:53.516689+0000","last_became_peered":"2023-11-26T07:17:53.516689+0000","last_unstale":"2023-11-26T07:17:55.538911+0000","last_undegraded":"2023-11-26T07:17:55.538911+0000","last_fullsized":"2023-11-26T07:17:55.538911+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:17:52.477276+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:17:52.477276+0000","last_clean_scrub_stamp":"2023-11-26T07:17:52.477276+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T16:07:45.587443+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-26T07:17:56.640131+0000","last_change":"2023-11-26T07:17:54.501979+0000","last_active":"2023-11-26T07:17:56.640131+0000","last_peered":"2023-11-26T07:17:56.640131+0000","last_clean":"2023-11-26T07:17:56.640131+0000","last_became_active":"2023-11-26T07:17:54.501682+0000","last_became_peered":"2023-11-26T07:17:54.501682+0000","last_unstale":"2023-11-26T07:17:56.640131+0000","last_undegraded":"2023-11-26T07:17:56.640131+0000","last_fullsized":"2023-11-26T07:17:56.640131+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:17:53.483873+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:17:53.483873+0000","last_clean_scrub_stamp":"2023-11-26T07:17:53.483873+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T12:51:17.339533+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-26T07:17:55.516646+0000","last_change":"2023-11-26T07:17:53.509606+0000","last_active":"2023-11-26T07:17:55.516646+0000","last_peered":"2023-11-26T07:17:55.516646+0000","last_clean":"2023-11-26T07:17:55.516646+0000","last_became_active":"2023-11-26T07:17:53.509268+0000","last_became_peered":"2023-11-26T07:17:53.509268+0000","last_unstale":"2023-11-26T07:17:55.516646+0000","last_undegraded":"2023-11-26T07:17:55.516646+0000","last_fullsized":"2023-11-26T07:17:55.516646+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:17:52.477276+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:17:52.477276+0000","last_clean_scrub_stamp":"2023-11-26T07:17:52.477276+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T16:07:45.587443+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-26T07:17:55.538908+0000","last_change":"2023-11-26T07:17:54.527293+0000","last_active":"2023-11-26T07:17:55.538908+0000","last_peered":"2023-11-26T07:17:55.538908+0000","last_clean":"2023-11-26T07:17:55.538908+0000","last_became_active":"2023-11-26T07:17:54.526666+0000","last_became_peered":"2023-11-26T07:17:54.526666+0000","last_unstale":"2023-11-26T07:17:55.538908+0000","last_undegraded":"2023-11-26T07:17:55.538908+0000","last_fullsized":"2023-11-26T07:17:55.538908+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:17:53.483873+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:17:53.483873+0000","last_clean_scrub_stamp":"2023-11-26T07:17:53.483873+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T17:14:14.930365+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-26T07:17:55.764220+0000","last_change":"2023-11-26T07:17:53.527907+0000","last_active":"2023-11-26T07:17:55.764220+0000","last_peered":"2023-11-26T07:17:55.764220+0000","last_clean":"2023-11-26T07:17:55.764220+0000","last_became_active":"2023-11-26T07:17:53.527624+0000","last_became_peered":"2023-11-26T07:17:53.527624+0000","last_unstale":"2023-11-26T07:17:55.764220+0000","last_undegraded":"2023-11-26T07:17:55.764220+0000","last_fullsized":"2023-11-26T07:17:55.764220+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:17:52.477276+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:17:52.477276+0000","last_clean_scrub_stamp":"2023-11-26T07:17:52.477276+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T11:16:24.794684+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-26T07:17:55.516576+0000","last_change":"2023-11-26T07:17:54.522960+0000","last_active":"2023-11-26T07:17:55.516576+0000","last_peered":"2023-11-26T07:17:55.516576+0000","last_clean":"2023-11-26T07:17:55.516576+0000","last_became_active":"2023-11-26T07:17:54.522658+0000","last_became_peered":"2023-11-26T07:17:54.522658+0000","last_unstale":"2023-11-26T07:17:55.516576+0000","last_undegraded":"2023-11-26T07:17:55.516576+0000","last_fullsized":"2023-11-26T07:17:55.516576+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:17:53.483873+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:17:53.483873+0000","last_clean_scrub_stamp":"2023-11-26T07:17:53.483873+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T13:18:15.521322+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-26T07:17:55.516521+0000","last_change":"2023-11-26T07:17:53.513529+0000","last_active":"2023-11-26T07:17:55.516521+0000","last_peered":"2023-11-26T07:17:55.516521+0000","last_clean":"2023-11-26T07:17:55.516521+0000","last_became_active":"2023-11-26T07:17:53.513125+0000","last_became_peered":"2023-11-26T07:17:53.513125+0000","last_unstale":"2023-11-26T07:17:55.516521+0000","last_undegraded":"2023-11-26T07:17:55.516521+0000","last_fullsized":"2023-11-26T07:17:55.516521+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:17:52.477276+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:17:52.477276+0000","last_clean_scrub_stamp":"2023-11-26T07:17:52.477276+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T09:53:28.237406+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-26T07:17:56.641191+0000","last_change":"2023-11-26T07:17:54.525518+0000","last_active":"2023-11-26T07:17:56.641191+0000","last_peered":"2023-11-26T07:17:56.641191+0000","last_clean":"2023-11-26T07:17:56.641191+0000","last_became_active":"2023-11-26T07:17:54.524972+0000","last_became_peered":"2023-11-26T07:17:54.524972+0000","last_unstale":"2023-11-26T07:17:56.641191+0000","last_undegraded":"2023-11-26T07:17:56.641191+0000","last_fullsized":"2023-11-26T07:17:56.641191+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:17:53.483873+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:17:53.483873+0000","last_clean_scrub_stamp":"2023-11-26T07:17:53.483873+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T15:49:39.889795+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.7","version":"19'2","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-26T07:17:56.640410+0000","last_change":"2023-11-26T07:17:53.523424+0000","last_active":"2023-11-26T07:17:56.640410+0000","last_peered":"2023-11-26T07:17:56.640410+0000","last_clean":"2023-11-26T07:17:56.640410+0000","last_became_active":"2023-11-26T07:17:53.523139+0000","last_became_peered":"2023-11-26T07:17:53.523139+0000","last_unstale":"2023-11-26T07:17:56.640410+0000","last_undegraded":"2023-11-26T07:17:56.640410+0000","last_fullsized":"2023-11-26T07:17:56.640410+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:17:52.477276+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:17:52.477276+0000","last_clean_scrub_stamp":"2023-11-26T07:17:52.477276+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T09:18:47.261084+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-26T07:17:56.069026+0000","last_change":"2023-11-26T07:17:54.523889+0000","last_active":"2023-11-26T07:17:56.069026+0000","last_peered":"2023-11-26T07:17:56.069026+0000","last_clean":"2023-11-26T07:17:56.069026+0000","last_became_active":"2023-11-26T07:17:54.523610+0000","last_became_peered":"2023-11-26T07:17:54.523610+0000","last_unstale":"2023-11-26T07:17:56.069026+0000","last_undegraded":"2023-11-26T07:17:56.069026+0000","last_fullsized":"2023-11-26T07:17:56.069026+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:17:53.483873+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:17:53.483873+0000","last_clean_scrub_stamp":"2023-11-26T07:17:53.483873+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T14:45:51.557148+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-26T07:17:55.763603+0000","last_change":"2023-11-26T07:17:53.521942+0000","last_active":"2023-11-26T07:17:55.763603+0000","last_peered":"2023-11-26T07:17:55.763603+0000","last_clean":"2023-11-26T07:17:55.763603+0000","last_became_active":"2023-11-26T07:17:53.521487+0000","last_became_peered":"2023-11-26T07:17:53.521487+0000","last_unstale":"2023-11-26T07:17:55.763603+0000","last_undegraded":"2023-11-26T07:17:55.763603+0000","last_fullsized":"2023-11-26T07:17:55.763603+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:17:52.477276+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:17:52.477276+0000","last_clean_scrub_stamp":"2023-11-26T07:17:52.477276+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T09:07:52.941378+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-26T07:17:56.640134+0000","last_change":"2023-11-26T07:17:54.516390+0000","last_active":"2023-11-26T07:17:56.640134+0000","last_peered":"2023-11-26T07:17:56.640134+0000","last_clean":"2023-11-26T07:17:56.640134+0000","last_became_active":"2023-11-26T07:17:54.515967+0000","last_became_peered":"2023-11-26T07:17:54.515967+0000","last_unstale":"2023-11-26T07:17:56.640134+0000","last_undegraded":"2023-11-26T07:17:56.640134+0000","last_fullsized":"2023-11-26T07:17:56.640134+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:17:53.483873+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:17:53.483873+0000","last_clean_scrub_stamp":"2023-11-26T07:17:53.483873+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T17:42:29.554668+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.9","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-26T07:17:55.516770+0000","last_change":"2023-11-26T07:17:53.520545+0000","last_active":"2023-11-26T07:17:55.516770+0000","last_peered":"2023-11-26T07:17:55.516770+0000","last_clean":"2023-11-26T07:17:55.516770+0000","last_became_active":"2023-11-26T07:17:53.520076+0000","last_became_peered":"2023-11-26T07:17:53.520076+0000","last_unstale":"2023-11-26T07:17:55.516770+0000","last_undegraded":"2023-11-26T07:17:55.516770+0000","last_fullsized":"2023-11-26T07:17:55.516770+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:17:52.477276+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:17:52.477276+0000","last_clean_scrub_stamp":"2023-11-26T07:17:52.477276+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T12:51:16.332936+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-26T07:17:56.641019+0000","last_change":"2023-11-26T07:17:54.512904+0000","last_active":"2023-11-26T07:17:56.641019+0000","last_peered":"2023-11-26T07:17:56.641019+0000","last_clean":"2023-11-26T07:17:56.641019+0000","last_became_active":"2023-11-26T07:17:54.512619+0000","last_became_peered":"2023-11-26T07:17:54.512619+0000","last_unstale":"2023-11-26T07:17:56.641019+0000","last_undegraded":"2023-11-26T07:17:56.641019+0000","last_fullsized":"2023-11-26T07:17:56.641019+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:17:53.483873+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:17:53.483873+0000","last_clean_scrub_stamp":"2023-11-26T07:17:53.483873+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T14:49:27.883374+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.a","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-26T07:17:55.516544+0000","last_change":"2023-11-26T07:17:53.523572+0000","last_active":"2023-11-26T07:17:55.516544+0000","last_peered":"2023-11-26T07:17:55.516544+0000","last_clean":"2023-11-26T07:17:55.516544+0000","last_became_active":"2023-11-26T07:17:53.523266+0000","last_became_peered":"2023-11-26T07:17:53.523266+0000","last_unstale":"2023-11-26T07:17:55.516544+0000","last_undegraded":"2023-11-26T07:17:55.516544+0000","last_fullsized":"2023-11-26T07:17:55.516544+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:17:52.477276+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:17:52.477276+0000","last_clean_scrub_stamp":"2023-11-26T07:17:52.477276+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T13:59:52.977195+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-26T07:17:56.640963+0000","last_change":"2023-11-26T07:17:54.515437+0000","last_active":"2023-11-26T07:17:56.640963+0000","last_peered":"2023-11-26T07:17:56.640963+0000","last_clean":"2023-11-26T07:17:56.640963+0000","last_became_active":"2023-11-26T07:17:54.515127+0000","last_became_peered":"2023-11-26T07:17:54.515127+0000","last_unstale":"2023-11-26T07:17:56.640963+0000","last_undegraded":"2023-11-26T07:17:56.640963+0000","last_fullsized":"2023-11-26T07:17:56.640963+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:17:53.483873+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:17:53.483873+0000","last_clean_scrub_stamp":"2023-11-26T07:17:53.483873+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T14:45:51.557148+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.b","version":"19'2","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-26T07:17:55.763968+0000","last_change":"2023-11-26T07:17:53.525249+0000","last_active":"2023-11-26T07:17:55.763968+0000","last_peered":"2023-11-26T07:17:55.763968+0000","last_clean":"2023-11-26T07:17:55.763968+0000","last_became_active":"2023-11-26T07:17:53.524957+0000","last_became_peered":"2023-11-26T07:17:53.524957+0000","last_unstale":"2023-11-26T07:17:55.763968+0000","last_undegraded":"2023-11-26T07:17:55.763968+0000","last_fullsized":"2023-11-26T07:17:55.763968+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:17:52.477276+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:17:52.477276+0000","last_clean_scrub_stamp":"2023-11-26T07:17:52.477276+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T19:07:41.542569+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-26T07:17:56.639556+0000","last_change":"2023-11-26T07:17:54.521923+0000","last_active":"2023-11-26T07:17:56.639556+0000","last_peered":"2023-11-26T07:17:56.639556+0000","last_clean":"2023-11-26T07:17:56.639556+0000","last_became_active":"2023-11-26T07:17:54.521503+0000","last_became_peered":"2023-11-26T07:17:54.521503+0000","last_unstale":"2023-11-26T07:17:56.639556+0000","last_undegraded":"2023-11-26T07:17:56.639556+0000","last_fullsized":"2023-11-26T07:17:56.639556+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:17:53.483873+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:17:53.483873+0000","last_clean_scrub_stamp":"2023-11-26T07:17:53.483873+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T16:07:46.594040+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-26T07:17:55.538848+0000","last_change":"2023-11-26T07:17:54.526945+0000","last_active":"2023-11-26T07:17:55.538848+0000","last_peered":"2023-11-26T07:17:55.538848+0000","last_clean":"2023-11-26T07:17:55.538848+0000","last_became_active":"2023-11-26T07:17:54.526110+0000","last_became_peered":"2023-11-26T07:17:54.526110+0000","last_unstale":"2023-11-26T07:17:55.538848+0000","last_undegraded":"2023-11-26T07:17:55.538848+0000","last_fullsized":"2023-11-26T07:17:55.538848+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:17:53.483873+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:17:53.483873+0000","last_clean_scrub_stamp":"2023-11-26T07:17:53.483873+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T10:18:52.982852+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1c","version":"19'2","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-26T07:17:55.763845+0000","last_change":"2023-11-26T07:17:53.524780+0000","last_active":"2023-11-26T07:17:55.763845+0000","last_peered":"2023-11-26T07:17:55.763845+0000","last_clean":"2023-11-26T07:17:55.763845+0000","last_became_active":"2023-11-26T07:17:53.524429+0000","last_became_peered":"2023-11-26T07:17:53.524429+0000","last_unstale":"2023-11-26T07:17:55.763845+0000","last_undegraded":"2023-11-26T07:17:55.763845+0000","last_fullsized":"2023-11-26T07:17:55.763845+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:17:52.477276+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:17:52.477276+0000","last_clean_scrub_stamp":"2023-11-26T07:17:52.477276+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T09:53:28.237406+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-26T07:17:55.539011+0000","last_change":"2023-11-26T07:17:54.523997+0000","last_active":"2023-11-26T07:17:55.539011+0000","last_peered":"2023-11-26T07:17:55.539011+0000","last_clean":"2023-11-26T07:17:55.539011+0000","last_became_active":"2023-11-26T07:17:54.523690+0000","last_became_peered":"2023-11-26T07:17:54.523690+0000","last_unstale":"2023-11-26T07:17:55.539011+0000","last_undegraded":"2023-11-26T07:17:55.539011+0000","last_fullsized":"2023-11-26T07:17:55.539011+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:17:53.483873+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:17:53.483873+0000","last_clean_scrub_stamp":"2023-11-26T07:17:53.483873+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T11:28:10.566680+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1d","version":"19'2","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-26T07:17:55.764098+0000","last_change":"2023-11-26T07:17:53.526233+0000","last_active":"2023-11-26T07:17:55.764098+0000","last_peered":"2023-11-26T07:17:55.764098+0000","last_clean":"2023-11-26T07:17:55.764098+0000","last_became_active":"2023-11-26T07:17:53.525918+0000","last_became_peered":"2023-11-26T07:17:53.525918+0000","last_unstale":"2023-11-26T07:17:55.764098+0000","last_undegraded":"2023-11-26T07:17:55.764098+0000","last_fullsized":"2023-11-26T07:17:55.764098+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:17:52.477276+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:17:52.477276+0000","last_clean_scrub_stamp":"2023-11-26T07:17:52.477276+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T15:18:16.303185+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-26T07:17:56.068892+0000","last_change":"2023-11-26T07:17:54.528760+0000","last_active":"2023-11-26T07:17:56.068892+0000","last_peered":"2023-11-26T07:17:56.068892+0000","last_clean":"2023-11-26T07:17:56.068892+0000","last_became_active":"2023-11-26T07:17:54.528456+0000","last_became_peered":"2023-11-26T07:17:54.528456+0000","last_unstale":"2023-11-26T07:17:56.068892+0000","last_undegraded":"2023-11-26T07:17:56.068892+0000","last_fullsized":"2023-11-26T07:17:56.068892+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:17:53.483873+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:17:53.483873+0000","last_clean_scrub_stamp":"2023-11-26T07:17:53.483873+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T15:18:17.309782+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1e","version":"19'1","reported_seq":13,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-26T07:17:56.641018+0000","last_change":"2023-11-26T07:17:53.523797+0000","last_active":"2023-11-26T07:17:56.641018+0000","last_peered":"2023-11-26T07:17:56.641018+0000","last_clean":"2023-11-26T07:17:56.641018+0000","last_became_active":"2023-11-26T07:17:53.523521+0000","last_became_peered":"2023-11-26T07:17:53.523521+0000","last_unstale":"2023-11-26T07:17:56.641018+0000","last_undegraded":"2023-11-26T07:17:56.641018+0000","last_fullsized":"2023-11-26T07:17:56.641018+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:17:52.477276+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:17:52.477276+0000","last_clean_scrub_stamp":"2023-11-26T07:17:52.477276+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T09:53:28.237406+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-26T07:17:56.640929+0000","last_change":"2023-11-26T07:17:54.517290+0000","last_active":"2023-11-26T07:17:56.640929+0000","last_peered":"2023-11-26T07:17:56.640929+0000","last_clean":"2023-11-26T07:17:56.640929+0000","last_became_active":"2023-11-26T07:17:54.516998+0000","last_became_peered":"2023-11-26T07:17:54.516998+0000","last_unstale":"2023-11-26T07:17:56.640929+0000","last_undegraded":"2023-11-26T07:17:56.640929+0000","last_fullsized":"2023-11-26T07:17:56.640929+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:17:53.483873+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:17:53.483873+0000","last_clean_scrub_stamp":"2023-11-26T07:17:53.483873+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T13:18:15.521322+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1f","version":"19'4","reported_seq":16,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-26T07:17:56.069044+0000","last_change":"2023-11-26T07:17:53.524902+0000","last_active":"2023-11-26T07:17:56.069044+0000","last_peered":"2023-11-26T07:17:56.069044+0000","last_clean":"2023-11-26T07:17:56.069044+0000","last_became_active":"2023-11-26T07:17:53.524530+0000","last_became_peered":"2023-11-26T07:17:53.524530+0000","last_unstale":"2023-11-26T07:17:56.069044+0000","last_undegraded":"2023-11-26T07:17:56.069044+0000","last_fullsized":"2023-11-26T07:17:56.069044+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:17:52.477276+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:17:52.477276+0000","last_clean_scrub_stamp":"2023-11-26T07:17:52.477276+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T09:18:47.261084+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":2410,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":38,"ondisk_log_size":38,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574852,"num_pgs":30,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27808,"kb_used_data":784,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344032,"statfs":{"total":96636764160,"available":96608288768,"internally_reserved":0,"allocated":802816,"data_stored":661860,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":7,"apply_latency_ms":7,"commit_latency_ns":7000000,"apply_latency_ns":7000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.77300000000000002}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.80500000000000005}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.61199999999999999}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.52000000000000002}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.66300000000000003}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.72699999999999998}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.78600000000000003}]}]},{"osd":6,"up_from":13,"seq":55834574854,"num_pgs":16,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27220,"kb_used_data":200,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344620,"statfs":{"total":96636764160,"available":96608890880,"internally_reserved":0,"allocated":204800,"data_stored":66222,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.76400000000000001}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.66500000000000004}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.70299999999999996}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.57599999999999996}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.48299999999999998}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.59299999999999997}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.71799999999999997}]}]},{"osd":5,"up_from":13,"seq":55834574852,"num_pgs":30,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27228,"kb_used_data":204,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344612,"statfs":{"total":96636764160,"available":96608882688,"internally_reserved":0,"allocated":208896,"data_stored":71000,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.86799999999999999}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.80700000000000005}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.76300000000000001}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.83199999999999996}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.66600000000000004}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.54000000000000004}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.93200000000000005}]}]},{"osd":4,"up_from":13,"seq":55834574854,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27220,"kb_used_data":200,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344620,"statfs":{"total":96636764160,"available":96608890880,"internally_reserved":0,"allocated":204800,"data_stored":64736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":4.1689999999999996}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.66100000000000003}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":2.7850000000000001}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.591}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.496}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":2.8330000000000002}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":4.3689999999999998}]}]},{"osd":3,"up_from":13,"seq":55834574855,"num_pgs":19,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27220,"kb_used_data":200,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344620,"statfs":{"total":96636764160,"available":96608890880,"internally_reserved":0,"allocated":204800,"data_stored":66222,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1592,"internal_metadata":27654600},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.39700000000000002}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.32200000000000001}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.41999999999999998}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.878}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.77700000000000002}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.82799999999999996}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.006}]}]},{"osd":2,"up_from":13,"seq":55834574855,"num_pgs":13,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27224,"kb_used_data":200,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344616,"statfs":{"total":96636764160,"available":96608886784,"internally_reserved":0,"allocated":204800,"data_stored":70910,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.42699999999999999}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.47599999999999998}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.40600000000000003}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.89800000000000002}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.70099999999999996}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.80300000000000005}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.88400000000000001}]}]},{"osd":1,"up_from":13,"seq":55834574854,"num_pgs":19,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27232,"kb_used_data":208,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344608,"statfs":{"total":96636764160,"available":96608878592,"internally_reserved":0,"allocated":212992,"data_stored":71070,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.443}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.36499999999999999}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.52600000000000002}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.63700000000000001}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.51300000000000001}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.66700000000000004}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.70699999999999996}]}]},{"osd":0,"up_from":13,"seq":55834574854,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27812,"kb_used_data":792,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344028,"statfs":{"total":96636764160,"available":96608284672,"internally_reserved":0,"allocated":811008,"data_stored":661916,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.45400000000000001}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.40899999999999997}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.51400000000000001}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.70799999999999996}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.871}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.85299999999999998}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.89900000000000002}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":638,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":160,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":126,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-11-26T07:17:58.577 DEBUG:teuthology.orchestra.run.smithi143:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-11-26T07:17:58.757 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:17:58.754+0000 7f2080032640 1 Processor -- start 2023-11-26T07:17:58.757 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:17:58.754+0000 7f2080032640 1 -- start start 2023-11-26T07:17:58.759 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:17:58.754+0000 7f2080032640 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7f207806b190 0x7f2078146530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:17:58.759 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:17:58.754+0000 7f2080032640 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7f207806a490 0x7f207806a900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:17:58.759 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:17:58.754+0000 7f2080032640 1 -- --> v1:172.21.15.143:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2078146380 con 0x7f2078145fa0 2023-11-26T07:17:58.759 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:17:58.754+0000 7f2080032640 1 -- --> v2:172.21.15.143:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2078071480 con 0x7f207806a490 2023-11-26T07:17:58.759 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:17:58.754+0000 7f2080032640 1 -- --> v2:172.21.15.188:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2078072000 con 0x7f207806b190 2023-11-26T07:17:58.759 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:17:58.754+0000 7f207ddce640 1 --1- >> v1:172.21.15.143:6789/0 conn(0x7f2078145fa0 0x7f2078072420 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.143:6789/0 says I am v1:172.21.15.143:40108/0 (socket says 172.21.15.143:40108) 2023-11-26T07:17:58.759 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:17:58.754+0000 7f207ddce640 1 -- 172.21.15.143:0/4196732564 learned_addr learned my addr 172.21.15.143:0/4196732564 (peer_addr_for_me v1:172.21.15.143:0/0) 2023-11-26T07:17:58.759 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:17:58.754+0000 7f207e5cf640 1 --2- 172.21.15.143:0/4196732564 >> v2:172.21.15.143:3300/0 conn(0x7f207806a490 0x7f207806a900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:17:58.760 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:17:58.754+0000 7f207d5cd640 1 --2- 172.21.15.143:0/4196732564 >> v2:172.21.15.188:3300/0 conn(0x7f207806b190 0x7f2078146530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:17:58.760 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:17:58.754+0000 7f207cdcc640 1 -- 172.21.15.143:0/4196732564 <== mon.0 v1:172.21.15.143:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3794344617 0 0) 0x7f2078146380 con 0x7f2078145fa0 2023-11-26T07:17:58.760 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:17:58.754+0000 7f207cdcc640 1 -- 172.21.15.143:0/4196732564 --> v1:172.21.15.143:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f2064003530 con 0x7f2078145fa0 2023-11-26T07:17:58.760 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:17:58.754+0000 7f207e5cf640 1 -- 172.21.15.143:0/4196732564 >> v2:172.21.15.188:3300/0 conn(0x7f207806b190 msgr2=0x7f2078146530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-26T07:17:58.760 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:17:58.754+0000 7f207e5cf640 1 --2- 172.21.15.143:0/4196732564 >> v2:172.21.15.188:3300/0 conn(0x7f207806b190 0x7f2078146530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:17:58.760 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:17:58.754+0000 7f207e5cf640 1 -- 172.21.15.143:0/4196732564 >> v1:172.21.15.143:6789/0 conn(0x7f2078145fa0 legacy=0x7f2078072420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:17:58.761 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:17:58.754+0000 7f207e5cf640 1 -- 172.21.15.143:0/4196732564 --> v2:172.21.15.143:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2078071660 con 0x7f207806a490 2023-11-26T07:17:58.762 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:17:58.758+0000 7f207e5cf640 1 --2- 172.21.15.143:0/4196732564 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f207806a490 0x7f207806a900 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7f20780b3e80 tx=0x7f207001c910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=922349dccd01d254 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:17:58.762 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:17:58.758+0000 7f207cdcc640 1 -- 172.21.15.143:0/4196732564 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f20700a58c0 con 0x7f207806a490 2023-11-26T07:17:58.762 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:17:58.758+0000 7f207cdcc640 1 -- 172.21.15.143:0/4196732564 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f20700a5a20 con 0x7f207806a490 2023-11-26T07:17:58.763 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:17:58.758+0000 7f207cdcc640 1 -- 172.21.15.143:0/4196732564 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f20700a5db0 con 0x7f207806a490 2023-11-26T07:17:58.763 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:17:58.758+0000 7f2080032640 1 -- 172.21.15.143:0/4196732564 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f207806a490 msgr2=0x7f207806a900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:17:58.763 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:17:58.758+0000 7f2080032640 1 --2- 172.21.15.143:0/4196732564 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f207806a490 0x7f207806a900 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7f20780b3e80 tx=0x7f207001c910 comp rx=0 tx=0).stop 2023-11-26T07:17:58.763 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:17:58.758+0000 7f2080032640 1 -- 172.21.15.143:0/4196732564 shutdown_connections 2023-11-26T07:17:58.764 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:17:58.758+0000 7f2080032640 1 --2- 172.21.15.143:0/4196732564 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f207806a490 0x7f207806a900 unknown :-1 s=CLOSED pgs=89 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:17:58.764 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:17:58.758+0000 7f2080032640 1 --2- 172.21.15.143:0/4196732564 >> v2:172.21.15.188:3300/0 conn(0x7f207806b190 0x7f2078146530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:17:58.764 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:17:58.758+0000 7f2080032640 1 -- 172.21.15.143:0/4196732564 >> 172.21.15.143:0/4196732564 conn(0x7f20780cff00 msgr2=0x7f20780d0310 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:17:58.764 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:17:58.758+0000 7f2080032640 1 -- 172.21.15.143:0/4196732564 shutdown_connections 2023-11-26T07:17:58.764 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:17:58.758+0000 7f2080032640 1 -- 172.21.15.143:0/4196732564 wait complete. 2023-11-26T07:17:58.765 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:17:58.762+0000 7f2080032640 1 Processor -- start 2023-11-26T07:17:58.765 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:17:58.762+0000 7f2080032640 1 -- start start 2023-11-26T07:17:58.766 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:17:58.762+0000 7f2080032640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f207806a490 0x7f20780cc460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:17:58.767 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:17:58.762+0000 7f2080032640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f207806b190 0x7f207815b4a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:17:58.767 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:17:58.762+0000 7f2080032640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f2078145fa0 0x7f20780cd230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:17:58.767 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:17:58.762+0000 7f2080032640 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2078144960 con 0x7f2078145fa0 2023-11-26T07:17:58.767 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:17:58.762+0000 7f2080032640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2078143d70 con 0x7f207806b190 2023-11-26T07:17:58.767 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:17:58.762+0000 7f2080032640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2078143fc0 con 0x7f207806a490 2023-11-26T07:17:58.767 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:17:58.762+0000 7f207e5cf640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f2078145fa0 0x7f20780cd230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:17:58.767 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:17:58.762+0000 7f207e5cf640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f2078145fa0 0x7f20780cd230 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:46298/0 (socket says 172.21.15.143:46298) 2023-11-26T07:17:58.767 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:17:58.762+0000 7f207e5cf640 1 -- 172.21.15.143:0/4076719933 learned_addr learned my addr 172.21.15.143:0/4076719933 (peer_addr_for_me v2:172.21.15.143:0/0) 2023-11-26T07:17:58.767 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:17:58.762+0000 7f207ddce640 1 --2- 172.21.15.143:0/4076719933 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f207806a490 0x7f20780cc460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:17:58.767 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:17:58.762+0000 7f207d5cd640 1 --2- 172.21.15.143:0/4076719933 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f207806b190 0x7f207815b4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:17:58.767 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:17:58.762+0000 7f207e5cf640 1 -- 172.21.15.143:0/4076719933 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f207806a490 msgr2=0x7f20780cc460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:17:58.768 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:17:58.762+0000 7f207e5cf640 1 --2- 172.21.15.143:0/4076719933 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f207806a490 0x7f20780cc460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:17:58.768 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:17:58.762+0000 7f207e5cf640 1 -- 172.21.15.143:0/4076719933 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f207806b190 msgr2=0x7f207815b4a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:17:58.768 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:17:58.762+0000 7f207e5cf640 1 --2- 172.21.15.143:0/4076719933 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f207806b190 0x7f207815b4a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:17:58.768 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:17:58.762+0000 7f207e5cf640 1 -- 172.21.15.143:0/4076719933 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2070009a70 con 0x7f2078145fa0 2023-11-26T07:17:58.769 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:17:58.762+0000 7f207e5cf640 1 --2- 172.21.15.143:0/4076719933 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f2078145fa0 0x7f20780cd230 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7f2070019520 tx=0x7f20700196b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:17:58.769 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:17:58.762+0000 7f206e7fc640 1 -- 172.21.15.143:0/4076719933 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f20700b9860 con 0x7f2078145fa0 2023-11-26T07:17:58.769 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:17:58.762+0000 7f206e7fc640 1 -- 172.21.15.143:0/4076719933 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f20700b99c0 con 0x7f2078145fa0 2023-11-26T07:17:58.769 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:17:58.762+0000 7f206e7fc640 1 -- 172.21.15.143:0/4076719933 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f20700b9d70 con 0x7f2078145fa0 2023-11-26T07:17:58.769 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:17:58.762+0000 7f2080032640 1 -- 172.21.15.143:0/4076719933 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f20781442a0 con 0x7f2078145fa0 2023-11-26T07:17:58.769 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:17:58.762+0000 7f2080032640 1 -- 172.21.15.143:0/4076719933 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f20780cf5d0 con 0x7f2078145fa0 2023-11-26T07:17:58.769 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:17:58.762+0000 7f2080032640 1 -- 172.21.15.143:0/4076719933 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2078144aa0 con 0x7f2078145fa0 2023-11-26T07:17:58.769 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:17:58.766+0000 7f206e7fc640 1 -- 172.21.15.143:0/4076719933 <== mon.0 v2:172.21.15.143:3300/0 4 ==== mgrmap(e 5) v1 ==== 85852+0+0 (secure 0 0 0) 0x7f20700ad440 con 0x7f2078145fa0 2023-11-26T07:17:58.770 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:17:58.766+0000 7f206e7fc640 1 --2- 172.21.15.143:0/4076719933 >> [v2:172.21.15.143:6832/2150557544,v1:172.21.15.143:6833/2150557544] conn(0x7f205c066470 0x7f205c068930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:17:58.770 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:17:58.766+0000 7f206e7fc640 1 -- 172.21.15.143:0/4076719933 <== mon.0 v2:172.21.15.143:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f207011f0b0 con 0x7f2078145fa0 2023-11-26T07:17:58.770 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:17:58.766+0000 7f207ddce640 1 --2- 172.21.15.143:0/4076719933 >> [v2:172.21.15.143:6832/2150557544,v1:172.21.15.143:6833/2150557544] conn(0x7f205c066470 0x7f205c068930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:17:58.770 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:17:58.766+0000 7f207ddce640 1 --2- 172.21.15.143:0/4076719933 >> [v2:172.21.15.143:6832/2150557544,v1:172.21.15.143:6833/2150557544] conn(0x7f205c066470 0x7f205c068930 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7f20740011f0 tx=0x7f2074007d40 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:17:58.774 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:17:58.770+0000 7f206e7fc640 1 -- 172.21.15.143:0/4076719933 <== mon.0 v2:172.21.15.143:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172649 (secure 0 0 0) 0x7f2078144aa0 con 0x7f2078145fa0 2023-11-26T07:17:58.920 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:17:58.914+0000 7f2080032640 1 -- 172.21.15.143:0/4076719933 --> [v2:172.21.15.143:6832/2150557544,v1:172.21.15.143:6833/2150557544] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f2078144aa0 con 0x7f205c066470 2023-11-26T07:17:58.927 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:17:58.922+0000 7f206e7fc640 1 -- 172.21.15.143:0/4076719933 <== mgr.4100 v2:172.21.15.143:6832/2150557544 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+264699 (secure 0 0 0) 0x7f2078144aa0 con 0x7f205c066470 2023-11-26T07:17:58.930 INFO:teuthology.orchestra.run.smithi143.stdout: 2023-11-26T07:17:58.931 INFO:teuthology.orchestra.run.smithi143.stderr:dumped all 2023-11-26T07:17:58.932 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:17:58.926+0000 7f2080032640 1 -- 172.21.15.143:0/4076719933 >> [v2:172.21.15.143:6832/2150557544,v1:172.21.15.143:6833/2150557544] conn(0x7f205c066470 msgr2=0x7f205c068930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:17:58.932 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:17:58.926+0000 7f2080032640 1 --2- 172.21.15.143:0/4076719933 >> [v2:172.21.15.143:6832/2150557544,v1:172.21.15.143:6833/2150557544] conn(0x7f205c066470 0x7f205c068930 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7f20740011f0 tx=0x7f2074007d40 comp rx=0 tx=0).stop 2023-11-26T07:17:58.932 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:17:58.926+0000 7f2080032640 1 -- 172.21.15.143:0/4076719933 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f2078145fa0 msgr2=0x7f20780cd230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:17:58.932 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:17:58.926+0000 7f2080032640 1 --2- 172.21.15.143:0/4076719933 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f2078145fa0 0x7f20780cd230 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7f2070019520 tx=0x7f20700196b0 comp rx=0 tx=0).stop 2023-11-26T07:17:58.932 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:17:58.926+0000 7f2080032640 1 -- 172.21.15.143:0/4076719933 shutdown_connections 2023-11-26T07:17:58.933 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:17:58.926+0000 7f2080032640 1 --2- 172.21.15.143:0/4076719933 >> [v2:172.21.15.143:6832/2150557544,v1:172.21.15.143:6833/2150557544] conn(0x7f205c066470 0x7f205c068930 unknown :-1 s=CLOSED pgs=73 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:17:58.933 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:17:58.926+0000 7f2080032640 1 --2- 172.21.15.143:0/4076719933 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f2078145fa0 0x7f20780cd230 unknown :-1 s=CLOSED pgs=90 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:17:58.933 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:17:58.926+0000 7f2080032640 1 --2- 172.21.15.143:0/4076719933 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f207806b190 0x7f207815b4a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:17:58.933 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:17:58.926+0000 7f2080032640 1 --2- 172.21.15.143:0/4076719933 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f207806a490 0x7f20780cc460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:17:58.933 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:17:58.926+0000 7f2080032640 1 -- 172.21.15.143:0/4076719933 >> 172.21.15.143:0/4076719933 conn(0x7f20780cff00 msgr2=0x7f20780aef70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:17:58.933 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:17:58.926+0000 7f2080032640 1 -- 172.21.15.143:0/4076719933 shutdown_connections 2023-11-26T07:17:58.933 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:17:58.926+0000 7f2080032640 1 -- 172.21.15.143:0/4076719933 wait complete. 2023-11-26T07:17:58.950 INFO:teuthology.orchestra.run.smithi143.stdout:{"pg_ready":true,"pg_map":{"version":21,"stamp":"2023-11-26T07:17:57.694444+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592778,"num_objects":23,"num_object_clones":0,"num_object_copies":46,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":23,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":203,"num_write_kb":2430,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":155,"ondisk_log_size":155,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":178,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":218964,"kb_used_data":2788,"kb_used_omap":12,"kb_used_meta":216051,"kb_avail":754755756,"statfs":{"total":773094113280,"available":772869894144,"internally_reserved":0,"allocated":2854912,"data_stored":1733936,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12749,"internal_metadata":221236787},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":40,"apply_latency_ms":40,"commit_latency_ns":40000000,"apply_latency_ns":40000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2410,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"5.209089"},"pg_stats":[{"pgid":"3.3f","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-26T07:17:55.764359+0000","last_change":"2023-11-26T07:17:54.530144+0000","last_active":"2023-11-26T07:17:55.764359+0000","last_peered":"2023-11-26T07:17:55.764359+0000","last_clean":"2023-11-26T07:17:55.764359+0000","last_became_active":"2023-11-26T07:17:54.529816+0000","last_became_peered":"2023-11-26T07:17:54.529816+0000","last_unstale":"2023-11-26T07:17:55.764359+0000","last_undegraded":"2023-11-26T07:17:55.764359+0000","last_fullsized":"2023-11-26T07:17:55.764359+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:17:53.483873+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:17:53.483873+0000","last_clean_scrub_stamp":"2023-11-26T07:17:53.483873+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T08:53:05.773632+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-26T07:17:55.516522+0000","last_change":"2023-11-26T07:17:54.523453+0000","last_active":"2023-11-26T07:17:55.516522+0000","last_peered":"2023-11-26T07:17:55.516522+0000","last_clean":"2023-11-26T07:17:55.516522+0000","last_became_active":"2023-11-26T07:17:54.523104+0000","last_became_peered":"2023-11-26T07:17:54.523104+0000","last_unstale":"2023-11-26T07:17:55.516522+0000","last_undegraded":"2023-11-26T07:17:55.516522+0000","last_fullsized":"2023-11-26T07:17:55.516522+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:17:53.483873+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:17:53.483873+0000","last_clean_scrub_stamp":"2023-11-26T07:17:53.483873+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T11:16:25.801280+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-26T07:17:56.068788+0000","last_change":"2023-11-26T07:17:54.514169+0000","last_active":"2023-11-26T07:17:56.068788+0000","last_peered":"2023-11-26T07:17:56.068788+0000","last_clean":"2023-11-26T07:17:56.068788+0000","last_became_active":"2023-11-26T07:17:54.513836+0000","last_became_peered":"2023-11-26T07:17:54.513836+0000","last_unstale":"2023-11-26T07:17:56.068788+0000","last_undegraded":"2023-11-26T07:17:56.068788+0000","last_fullsized":"2023-11-26T07:17:56.068788+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:17:53.483873+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:17:53.483873+0000","last_clean_scrub_stamp":"2023-11-26T07:17:53.483873+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T09:53:29.244002+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-26T07:17:55.538957+0000","last_change":"2023-11-26T07:17:54.528554+0000","last_active":"2023-11-26T07:17:55.538957+0000","last_peered":"2023-11-26T07:17:55.538957+0000","last_clean":"2023-11-26T07:17:55.538957+0000","last_became_active":"2023-11-26T07:17:54.527864+0000","last_became_peered":"2023-11-26T07:17:54.527864+0000","last_unstale":"2023-11-26T07:17:55.538957+0000","last_undegraded":"2023-11-26T07:17:55.538957+0000","last_fullsized":"2023-11-26T07:17:55.538957+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:17:53.483873+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:17:53.483873+0000","last_clean_scrub_stamp":"2023-11-26T07:17:53.483873+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T15:49:39.889795+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-26T07:17:55.763887+0000","last_change":"2023-11-26T07:17:54.527144+0000","last_active":"2023-11-26T07:17:55.763887+0000","last_peered":"2023-11-26T07:17:55.763887+0000","last_clean":"2023-11-26T07:17:55.763887+0000","last_became_active":"2023-11-26T07:17:54.525279+0000","last_became_peered":"2023-11-26T07:17:54.525279+0000","last_unstale":"2023-11-26T07:17:55.763887+0000","last_undegraded":"2023-11-26T07:17:55.763887+0000","last_fullsized":"2023-11-26T07:17:55.763887+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:17:53.483873+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:17:53.483873+0000","last_clean_scrub_stamp":"2023-11-26T07:17:53.483873+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T11:28:10.566680+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-26T07:17:55.538672+0000","last_change":"2023-11-26T07:17:54.522607+0000","last_active":"2023-11-26T07:17:55.538672+0000","last_peered":"2023-11-26T07:17:55.538672+0000","last_clean":"2023-11-26T07:17:55.538672+0000","last_became_active":"2023-11-26T07:17:54.522297+0000","last_became_peered":"2023-11-26T07:17:54.522297+0000","last_unstale":"2023-11-26T07:17:55.538672+0000","last_undegraded":"2023-11-26T07:17:55.538672+0000","last_fullsized":"2023-11-26T07:17:55.538672+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:17:53.483873+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:17:53.483873+0000","last_clean_scrub_stamp":"2023-11-26T07:17:53.483873+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T13:18:15.521322+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-26T07:17:55.763706+0000","last_change":"2023-11-26T07:17:54.524898+0000","last_active":"2023-11-26T07:17:55.763706+0000","last_peered":"2023-11-26T07:17:55.763706+0000","last_clean":"2023-11-26T07:17:55.763706+0000","last_became_active":"2023-11-26T07:17:54.524611+0000","last_became_peered":"2023-11-26T07:17:54.524611+0000","last_unstale":"2023-11-26T07:17:55.763706+0000","last_undegraded":"2023-11-26T07:17:55.763706+0000","last_fullsized":"2023-11-26T07:17:55.763706+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:17:53.483873+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:17:53.483873+0000","last_clean_scrub_stamp":"2023-11-26T07:17:53.483873+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T17:14:14.930365+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-26T07:17:56.640574+0000","last_change":"2023-11-26T07:17:54.530342+0000","last_active":"2023-11-26T07:17:56.640574+0000","last_peered":"2023-11-26T07:17:56.640574+0000","last_clean":"2023-11-26T07:17:56.640574+0000","last_became_active":"2023-11-26T07:17:54.529960+0000","last_became_peered":"2023-11-26T07:17:54.529960+0000","last_unstale":"2023-11-26T07:17:56.640574+0000","last_undegraded":"2023-11-26T07:17:56.640574+0000","last_fullsized":"2023-11-26T07:17:56.640574+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:17:53.483873+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:17:53.483873+0000","last_clean_scrub_stamp":"2023-11-26T07:17:53.483873+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T08:42:29.375032+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-26T07:17:55.764389+0000","last_change":"2023-11-26T07:17:54.539062+0000","last_active":"2023-11-26T07:17:55.764389+0000","last_peered":"2023-11-26T07:17:55.764389+0000","last_clean":"2023-11-26T07:17:55.764389+0000","last_became_active":"2023-11-26T07:17:54.538119+0000","last_became_peered":"2023-11-26T07:17:54.538119+0000","last_unstale":"2023-11-26T07:17:55.764389+0000","last_undegraded":"2023-11-26T07:17:55.764389+0000","last_fullsized":"2023-11-26T07:17:55.764389+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:17:53.483873+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:17:53.483873+0000","last_clean_scrub_stamp":"2023-11-26T07:17:53.483873+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T13:57:07.286285+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-26T07:17:56.640341+0000","last_change":"2023-11-26T07:17:54.524564+0000","last_active":"2023-11-26T07:17:56.640341+0000","last_peered":"2023-11-26T07:17:56.640341+0000","last_clean":"2023-11-26T07:17:56.640341+0000","last_became_active":"2023-11-26T07:17:54.524255+0000","last_became_peered":"2023-11-26T07:17:54.524255+0000","last_unstale":"2023-11-26T07:17:56.640341+0000","last_undegraded":"2023-11-26T07:17:56.640341+0000","last_fullsized":"2023-11-26T07:17:56.640341+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:17:53.483873+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:17:53.483873+0000","last_clean_scrub_stamp":"2023-11-26T07:17:53.483873+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T11:16:25.801280+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-26T07:17:56.639652+0000","last_change":"2023-11-26T07:17:54.535310+0000","last_active":"2023-11-26T07:17:56.639652+0000","last_peered":"2023-11-26T07:17:56.639652+0000","last_clean":"2023-11-26T07:17:56.639652+0000","last_became_active":"2023-11-26T07:17:54.534986+0000","last_became_peered":"2023-11-26T07:17:54.534986+0000","last_unstale":"2023-11-26T07:17:56.639652+0000","last_undegraded":"2023-11-26T07:17:56.639652+0000","last_fullsized":"2023-11-26T07:17:56.639652+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:17:53.483873+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:17:53.483873+0000","last_clean_scrub_stamp":"2023-11-26T07:17:53.483873+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T15:18:17.309782+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-26T07:17:56.640026+0000","last_change":"2023-11-26T07:17:54.512408+0000","last_active":"2023-11-26T07:17:56.640026+0000","last_peered":"2023-11-26T07:17:56.640026+0000","last_clean":"2023-11-26T07:17:56.640026+0000","last_became_active":"2023-11-26T07:17:54.512005+0000","last_became_peered":"2023-11-26T07:17:54.512005+0000","last_unstale":"2023-11-26T07:17:56.640026+0000","last_undegraded":"2023-11-26T07:17:56.640026+0000","last_fullsized":"2023-11-26T07:17:56.640026+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:17:53.483873+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:17:53.483873+0000","last_clean_scrub_stamp":"2023-11-26T07:17:53.483873+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T09:07:53.947975+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-26T07:17:56.640756+0000","last_change":"2023-11-26T07:17:54.514825+0000","last_active":"2023-11-26T07:17:56.640756+0000","last_peered":"2023-11-26T07:17:56.640756+0000","last_clean":"2023-11-26T07:17:56.640756+0000","last_became_active":"2023-11-26T07:17:54.514502+0000","last_became_peered":"2023-11-26T07:17:54.514502+0000","last_unstale":"2023-11-26T07:17:56.640756+0000","last_undegraded":"2023-11-26T07:17:56.640756+0000","last_fullsized":"2023-11-26T07:17:56.640756+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:17:53.483873+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:17:53.483873+0000","last_clean_scrub_stamp":"2023-11-26T07:17:53.483873+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T07:50:04.472214+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-26T07:17:56.068691+0000","last_change":"2023-11-26T07:17:54.521737+0000","last_active":"2023-11-26T07:17:56.068691+0000","last_peered":"2023-11-26T07:17:56.068691+0000","last_clean":"2023-11-26T07:17:56.068691+0000","last_became_active":"2023-11-26T07:17:54.521387+0000","last_became_peered":"2023-11-26T07:17:54.521387+0000","last_unstale":"2023-11-26T07:17:56.068691+0000","last_undegraded":"2023-11-26T07:17:56.068691+0000","last_fullsized":"2023-11-26T07:17:56.068691+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:17:53.483873+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:17:53.483873+0000","last_clean_scrub_stamp":"2023-11-26T07:17:53.483873+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T18:33:32.653907+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-26T07:17:55.516698+0000","last_change":"2023-11-26T07:17:54.533852+0000","last_active":"2023-11-26T07:17:55.516698+0000","last_peered":"2023-11-26T07:17:55.516698+0000","last_clean":"2023-11-26T07:17:55.516698+0000","last_became_active":"2023-11-26T07:17:54.533532+0000","last_became_peered":"2023-11-26T07:17:54.533532+0000","last_unstale":"2023-11-26T07:17:55.516698+0000","last_undegraded":"2023-11-26T07:17:55.516698+0000","last_fullsized":"2023-11-26T07:17:55.516698+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:17:53.483873+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:17:53.483873+0000","last_clean_scrub_stamp":"2023-11-26T07:17:53.483873+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T14:45:51.557148+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-26T07:17:55.538556+0000","last_change":"2023-11-26T07:17:54.523130+0000","last_active":"2023-11-26T07:17:55.538556+0000","last_peered":"2023-11-26T07:17:55.538556+0000","last_clean":"2023-11-26T07:17:55.538556+0000","last_became_active":"2023-11-26T07:17:54.522879+0000","last_became_peered":"2023-11-26T07:17:54.522879+0000","last_unstale":"2023-11-26T07:17:55.538556+0000","last_undegraded":"2023-11-26T07:17:55.538556+0000","last_fullsized":"2023-11-26T07:17:55.538556+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:17:53.483873+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:17:53.483873+0000","last_clean_scrub_stamp":"2023-11-26T07:17:53.483873+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T11:16:25.801280+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-26T07:17:55.539151+0000","last_change":"2023-11-26T07:17:54.534352+0000","last_active":"2023-11-26T07:17:55.539151+0000","last_peered":"2023-11-26T07:17:55.539151+0000","last_clean":"2023-11-26T07:17:55.539151+0000","last_became_active":"2023-11-26T07:17:54.534004+0000","last_became_peered":"2023-11-26T07:17:54.534004+0000","last_unstale":"2023-11-26T07:17:55.539151+0000","last_undegraded":"2023-11-26T07:17:55.539151+0000","last_fullsized":"2023-11-26T07:17:55.539151+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:17:53.483873+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:17:53.483873+0000","last_clean_scrub_stamp":"2023-11-26T07:17:53.483873+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T18:27:00.305240+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-26T07:17:55.514964+0000","last_change":"2023-11-26T07:17:54.515085+0000","last_active":"2023-11-26T07:17:55.514964+0000","last_peered":"2023-11-26T07:17:55.514964+0000","last_clean":"2023-11-26T07:17:55.514964+0000","last_became_active":"2023-11-26T07:17:54.514782+0000","last_became_peered":"2023-11-26T07:17:54.514782+0000","last_unstale":"2023-11-26T07:17:55.514964+0000","last_undegraded":"2023-11-26T07:17:55.514964+0000","last_fullsized":"2023-11-26T07:17:55.514964+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:17:53.483873+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:17:53.483873+0000","last_clean_scrub_stamp":"2023-11-26T07:17:53.483873+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T13:45:34.765962+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-26T07:17:55.764203+0000","last_change":"2023-11-26T07:17:54.529137+0000","last_active":"2023-11-26T07:17:55.764203+0000","last_peered":"2023-11-26T07:17:55.764203+0000","last_clean":"2023-11-26T07:17:55.764203+0000","last_became_active":"2023-11-26T07:17:54.528496+0000","last_became_peered":"2023-11-26T07:17:54.528496+0000","last_unstale":"2023-11-26T07:17:55.764203+0000","last_undegraded":"2023-11-26T07:17:55.764203+0000","last_fullsized":"2023-11-26T07:17:55.764203+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:17:53.483873+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:17:53.483873+0000","last_clean_scrub_stamp":"2023-11-26T07:17:53.483873+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T17:27:24.347347+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-26T07:17:56.639612+0000","last_change":"2023-11-26T07:17:54.521609+0000","last_active":"2023-11-26T07:17:56.639612+0000","last_peered":"2023-11-26T07:17:56.639612+0000","last_clean":"2023-11-26T07:17:56.639612+0000","last_became_active":"2023-11-26T07:17:54.521298+0000","last_became_peered":"2023-11-26T07:17:54.521298+0000","last_unstale":"2023-11-26T07:17:56.639612+0000","last_undegraded":"2023-11-26T07:17:56.639612+0000","last_fullsized":"2023-11-26T07:17:56.639612+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:17:53.483873+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:17:53.483873+0000","last_clean_scrub_stamp":"2023-11-26T07:17:53.483873+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T08:42:29.375032+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-26T07:17:56.068908+0000","last_change":"2023-11-26T07:17:54.526550+0000","last_active":"2023-11-26T07:17:56.068908+0000","last_peered":"2023-11-26T07:17:56.068908+0000","last_clean":"2023-11-26T07:17:56.068908+0000","last_became_active":"2023-11-26T07:17:54.525639+0000","last_became_peered":"2023-11-26T07:17:54.525639+0000","last_unstale":"2023-11-26T07:17:56.068908+0000","last_undegraded":"2023-11-26T07:17:56.068908+0000","last_fullsized":"2023-11-26T07:17:56.068908+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:17:53.483873+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:17:53.483873+0000","last_clean_scrub_stamp":"2023-11-26T07:17:53.483873+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T07:50:04.472214+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-26T07:17:55.538640+0000","last_change":"2023-11-26T07:17:54.534209+0000","last_active":"2023-11-26T07:17:55.538640+0000","last_peered":"2023-11-26T07:17:55.538640+0000","last_clean":"2023-11-26T07:17:55.538640+0000","last_became_active":"2023-11-26T07:17:54.533932+0000","last_became_peered":"2023-11-26T07:17:54.533932+0000","last_unstale":"2023-11-26T07:17:55.538640+0000","last_undegraded":"2023-11-26T07:17:55.538640+0000","last_fullsized":"2023-11-26T07:17:55.538640+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:17:53.483873+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:17:53.483873+0000","last_clean_scrub_stamp":"2023-11-26T07:17:53.483873+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T18:02:15.969376+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-26T07:17:56.640577+0000","last_change":"2023-11-26T07:17:54.526358+0000","last_active":"2023-11-26T07:17:56.640577+0000","last_peered":"2023-11-26T07:17:56.640577+0000","last_clean":"2023-11-26T07:17:56.640577+0000","last_became_active":"2023-11-26T07:17:54.525756+0000","last_became_peered":"2023-11-26T07:17:54.525756+0000","last_unstale":"2023-11-26T07:17:56.640577+0000","last_undegraded":"2023-11-26T07:17:56.640577+0000","last_fullsized":"2023-11-26T07:17:56.640577+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:17:53.483873+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:17:53.483873+0000","last_clean_scrub_stamp":"2023-11-26T07:17:53.483873+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T15:18:17.309782+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-26T07:17:55.538626+0000","last_change":"2023-11-26T07:17:54.526328+0000","last_active":"2023-11-26T07:17:55.538626+0000","last_peered":"2023-11-26T07:17:55.538626+0000","last_clean":"2023-11-26T07:17:55.538626+0000","last_became_active":"2023-11-26T07:17:54.526026+0000","last_became_peered":"2023-11-26T07:17:54.526026+0000","last_unstale":"2023-11-26T07:17:55.538626+0000","last_undegraded":"2023-11-26T07:17:55.538626+0000","last_fullsized":"2023-11-26T07:17:55.538626+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:17:53.483873+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:17:53.483873+0000","last_clean_scrub_stamp":"2023-11-26T07:17:53.483873+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T14:45:51.557148+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-26T07:17:56.641174+0000","last_change":"2023-11-26T07:17:54.521949+0000","last_active":"2023-11-26T07:17:56.641174+0000","last_peered":"2023-11-26T07:17:56.641174+0000","last_clean":"2023-11-26T07:17:56.641174+0000","last_became_active":"2023-11-26T07:17:54.521649+0000","last_became_peered":"2023-11-26T07:17:54.521649+0000","last_unstale":"2023-11-26T07:17:56.641174+0000","last_undegraded":"2023-11-26T07:17:56.641174+0000","last_fullsized":"2023-11-26T07:17:56.641174+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:17:53.483873+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:17:53.483873+0000","last_clean_scrub_stamp":"2023-11-26T07:17:53.483873+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T17:14:14.930365+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-26T07:17:56.640304+0000","last_change":"2023-11-26T07:17:54.526758+0000","last_active":"2023-11-26T07:17:56.640304+0000","last_peered":"2023-11-26T07:17:56.640304+0000","last_clean":"2023-11-26T07:17:56.640304+0000","last_became_active":"2023-11-26T07:17:54.525563+0000","last_became_peered":"2023-11-26T07:17:54.525563+0000","last_unstale":"2023-11-26T07:17:56.640304+0000","last_undegraded":"2023-11-26T07:17:56.640304+0000","last_fullsized":"2023-11-26T07:17:56.640304+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:17:53.483873+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:17:53.483873+0000","last_clean_scrub_stamp":"2023-11-26T07:17:53.483873+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T07:50:04.472214+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-26T07:17:55.516791+0000","last_change":"2023-11-26T07:17:54.520560+0000","last_active":"2023-11-26T07:17:55.516791+0000","last_peered":"2023-11-26T07:17:55.516791+0000","last_clean":"2023-11-26T07:17:55.516791+0000","last_became_active":"2023-11-26T07:17:54.520377+0000","last_became_peered":"2023-11-26T07:17:54.520377+0000","last_unstale":"2023-11-26T07:17:55.516791+0000","last_undegraded":"2023-11-26T07:17:55.516791+0000","last_fullsized":"2023-11-26T07:17:55.516791+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:17:53.483873+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:17:53.483873+0000","last_clean_scrub_stamp":"2023-11-26T07:17:53.483873+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T18:02:15.969376+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-26T07:17:56.640998+0000","last_change":"2023-11-26T07:17:54.527041+0000","last_active":"2023-11-26T07:17:56.640998+0000","last_peered":"2023-11-26T07:17:56.640998+0000","last_clean":"2023-11-26T07:17:56.640998+0000","last_became_active":"2023-11-26T07:17:54.526391+0000","last_became_peered":"2023-11-26T07:17:54.526391+0000","last_unstale":"2023-11-26T07:17:56.640998+0000","last_undegraded":"2023-11-26T07:17:56.640998+0000","last_fullsized":"2023-11-26T07:17:56.640998+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:17:53.483873+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:17:53.483873+0000","last_clean_scrub_stamp":"2023-11-26T07:17:53.483873+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T18:02:15.969376+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-26T07:17:56.068828+0000","last_change":"2023-11-26T07:17:54.512264+0000","last_active":"2023-11-26T07:17:56.068828+0000","last_peered":"2023-11-26T07:17:56.068828+0000","last_clean":"2023-11-26T07:17:56.068828+0000","last_became_active":"2023-11-26T07:17:54.511963+0000","last_became_peered":"2023-11-26T07:17:54.511963+0000","last_unstale":"2023-11-26T07:17:56.068828+0000","last_undegraded":"2023-11-26T07:17:56.068828+0000","last_fullsized":"2023-11-26T07:17:56.068828+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:17:53.483873+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:17:53.483873+0000","last_clean_scrub_stamp":"2023-11-26T07:17:53.483873+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T08:42:29.375032+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-26T07:17:55.538793+0000","last_change":"2023-11-26T07:17:54.528162+0000","last_active":"2023-11-26T07:17:55.538793+0000","last_peered":"2023-11-26T07:17:55.538793+0000","last_clean":"2023-11-26T07:17:55.538793+0000","last_became_active":"2023-11-26T07:17:54.527657+0000","last_became_peered":"2023-11-26T07:17:54.527657+0000","last_unstale":"2023-11-26T07:17:55.538793+0000","last_undegraded":"2023-11-26T07:17:55.538793+0000","last_fullsized":"2023-11-26T07:17:55.538793+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:17:53.483873+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:17:53.483873+0000","last_clean_scrub_stamp":"2023-11-26T07:17:53.483873+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T14:49:27.883374+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-26T07:17:55.514728+0000","last_change":"2023-11-26T07:17:54.519107+0000","last_active":"2023-11-26T07:17:55.514728+0000","last_peered":"2023-11-26T07:17:55.514728+0000","last_clean":"2023-11-26T07:17:55.514728+0000","last_became_active":"2023-11-26T07:17:54.518822+0000","last_became_peered":"2023-11-26T07:17:54.518822+0000","last_unstale":"2023-11-26T07:17:55.514728+0000","last_undegraded":"2023-11-26T07:17:55.514728+0000","last_fullsized":"2023-11-26T07:17:55.514728+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:17:53.483873+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:17:53.483873+0000","last_clean_scrub_stamp":"2023-11-26T07:17:53.483873+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T09:07:53.947975+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-26T07:17:56.639246+0000","last_change":"2023-11-26T07:17:54.513368+0000","last_active":"2023-11-26T07:17:56.639246+0000","last_peered":"2023-11-26T07:17:56.639246+0000","last_clean":"2023-11-26T07:17:56.639246+0000","last_became_active":"2023-11-26T07:17:54.511290+0000","last_became_peered":"2023-11-26T07:17:54.511290+0000","last_unstale":"2023-11-26T07:17:56.639246+0000","last_undegraded":"2023-11-26T07:17:56.639246+0000","last_fullsized":"2023-11-26T07:17:56.639246+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:17:53.483873+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:17:53.483873+0000","last_clean_scrub_stamp":"2023-11-26T07:17:53.483873+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T18:33:32.653907+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-26T07:17:56.639579+0000","last_change":"2023-11-26T07:17:54.513190+0000","last_active":"2023-11-26T07:17:56.639579+0000","last_peered":"2023-11-26T07:17:56.639579+0000","last_clean":"2023-11-26T07:17:56.639579+0000","last_became_active":"2023-11-26T07:17:54.512872+0000","last_became_peered":"2023-11-26T07:17:54.512872+0000","last_unstale":"2023-11-26T07:17:56.639579+0000","last_undegraded":"2023-11-26T07:17:56.639579+0000","last_fullsized":"2023-11-26T07:17:56.639579+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:17:53.483873+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:17:53.483873+0000","last_clean_scrub_stamp":"2023-11-26T07:17:53.483873+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T12:32:48.249974+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1b","version":"19'1","reported_seq":13,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-26T07:17:56.640973+0000","last_change":"2023-11-26T07:17:53.525183+0000","last_active":"2023-11-26T07:17:56.640973+0000","last_peered":"2023-11-26T07:17:56.640973+0000","last_clean":"2023-11-26T07:17:56.640973+0000","last_became_active":"2023-11-26T07:17:53.524904+0000","last_became_peered":"2023-11-26T07:17:53.524904+0000","last_unstale":"2023-11-26T07:17:56.640973+0000","last_undegraded":"2023-11-26T07:17:56.640973+0000","last_fullsized":"2023-11-26T07:17:56.640973+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:17:52.477276+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:17:52.477276+0000","last_clean_scrub_stamp":"2023-11-26T07:17:52.477276+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T13:59:52.977195+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-26T07:17:56.068970+0000","last_change":"2023-11-26T07:17:54.522974+0000","last_active":"2023-11-26T07:17:56.068970+0000","last_peered":"2023-11-26T07:17:56.068970+0000","last_clean":"2023-11-26T07:17:56.068970+0000","last_became_active":"2023-11-26T07:17:54.521283+0000","last_became_peered":"2023-11-26T07:17:54.521283+0000","last_unstale":"2023-11-26T07:17:56.068970+0000","last_undegraded":"2023-11-26T07:17:56.068970+0000","last_fullsized":"2023-11-26T07:17:56.068970+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:17:53.483873+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:17:53.483873+0000","last_clean_scrub_stamp":"2023-11-26T07:17:53.483873+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T11:16:25.801280+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1a","version":"19'1","reported_seq":13,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-26T07:17:56.639442+0000","last_change":"2023-11-26T07:17:53.516099+0000","last_active":"2023-11-26T07:17:56.639442+0000","last_peered":"2023-11-26T07:17:56.639442+0000","last_clean":"2023-11-26T07:17:56.639442+0000","last_became_active":"2023-11-26T07:17:53.515687+0000","last_became_peered":"2023-11-26T07:17:53.515687+0000","last_unstale":"2023-11-26T07:17:56.639442+0000","last_undegraded":"2023-11-26T07:17:56.639442+0000","last_fullsized":"2023-11-26T07:17:56.639442+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:17:52.477276+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:17:52.477276+0000","last_clean_scrub_stamp":"2023-11-26T07:17:52.477276+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T09:07:52.941378+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.19","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-26T07:17:56.640738+0000","last_change":"2023-11-26T07:17:53.523159+0000","last_active":"2023-11-26T07:17:56.640738+0000","last_peered":"2023-11-26T07:17:56.640738+0000","last_clean":"2023-11-26T07:17:56.640738+0000","last_became_active":"2023-11-26T07:17:53.522824+0000","last_became_peered":"2023-11-26T07:17:53.522824+0000","last_unstale":"2023-11-26T07:17:56.640738+0000","last_undegraded":"2023-11-26T07:17:56.640738+0000","last_fullsized":"2023-11-26T07:17:56.640738+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:17:52.477276+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:17:52.477276+0000","last_clean_scrub_stamp":"2023-11-26T07:17:52.477276+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T13:45:33.759365+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-26T07:17:56.641065+0000","last_change":"2023-11-26T07:17:54.515953+0000","last_active":"2023-11-26T07:17:56.641065+0000","last_peered":"2023-11-26T07:17:56.641065+0000","last_clean":"2023-11-26T07:17:56.641065+0000","last_became_active":"2023-11-26T07:17:54.515662+0000","last_became_peered":"2023-11-26T07:17:54.515662+0000","last_unstale":"2023-11-26T07:17:56.641065+0000","last_undegraded":"2023-11-26T07:17:56.641065+0000","last_fullsized":"2023-11-26T07:17:56.641065+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:17:53.483873+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:17:53.483873+0000","last_clean_scrub_stamp":"2023-11-26T07:17:53.483873+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T10:18:52.982852+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.18","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-26T07:17:55.538849+0000","last_change":"2023-11-26T07:17:53.516308+0000","last_active":"2023-11-26T07:17:55.538849+0000","last_peered":"2023-11-26T07:17:55.538849+0000","last_clean":"2023-11-26T07:17:55.538849+0000","last_became_active":"2023-11-26T07:17:53.516027+0000","last_became_peered":"2023-11-26T07:17:53.516027+0000","last_unstale":"2023-11-26T07:17:55.538849+0000","last_undegraded":"2023-11-26T07:17:55.538849+0000","last_fullsized":"2023-11-26T07:17:55.538849+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:17:52.477276+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:17:52.477276+0000","last_clean_scrub_stamp":"2023-11-26T07:17:52.477276+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T08:42:28.368435+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-26T07:17:55.516385+0000","last_change":"2023-11-26T07:17:54.521790+0000","last_active":"2023-11-26T07:17:55.516385+0000","last_peered":"2023-11-26T07:17:55.516385+0000","last_clean":"2023-11-26T07:17:55.516385+0000","last_became_active":"2023-11-26T07:17:54.521475+0000","last_became_peered":"2023-11-26T07:17:54.521475+0000","last_unstale":"2023-11-26T07:17:55.516385+0000","last_undegraded":"2023-11-26T07:17:55.516385+0000","last_fullsized":"2023-11-26T07:17:55.516385+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:17:53.483873+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:17:53.483873+0000","last_clean_scrub_stamp":"2023-11-26T07:17:53.483873+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T15:18:17.309782+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.17","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-26T07:17:56.640306+0000","last_change":"2023-11-26T07:17:53.514460+0000","last_active":"2023-11-26T07:17:56.640306+0000","last_peered":"2023-11-26T07:17:56.640306+0000","last_clean":"2023-11-26T07:17:56.640306+0000","last_became_active":"2023-11-26T07:17:53.514170+0000","last_became_peered":"2023-11-26T07:17:53.514170+0000","last_unstale":"2023-11-26T07:17:56.640306+0000","last_undegraded":"2023-11-26T07:17:56.640306+0000","last_fullsized":"2023-11-26T07:17:56.640306+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:17:52.477276+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:17:52.477276+0000","last_clean_scrub_stamp":"2023-11-26T07:17:52.477276+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T15:47:35.416235+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-26T07:17:55.538995+0000","last_change":"2023-11-26T07:17:54.533776+0000","last_active":"2023-11-26T07:17:55.538995+0000","last_peered":"2023-11-26T07:17:55.538995+0000","last_clean":"2023-11-26T07:17:55.538995+0000","last_became_active":"2023-11-26T07:17:54.533279+0000","last_became_peered":"2023-11-26T07:17:54.533279+0000","last_unstale":"2023-11-26T07:17:55.538995+0000","last_undegraded":"2023-11-26T07:17:55.538995+0000","last_fullsized":"2023-11-26T07:17:55.538995+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:17:53.483873+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:17:53.483873+0000","last_clean_scrub_stamp":"2023-11-26T07:17:53.483873+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T15:38:12.078699+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.16","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-26T07:17:55.539107+0000","last_change":"2023-11-26T07:17:53.520128+0000","last_active":"2023-11-26T07:17:55.539107+0000","last_peered":"2023-11-26T07:17:55.539107+0000","last_clean":"2023-11-26T07:17:55.539107+0000","last_became_active":"2023-11-26T07:17:53.519399+0000","last_became_peered":"2023-11-26T07:17:53.519399+0000","last_unstale":"2023-11-26T07:17:55.539107+0000","last_undegraded":"2023-11-26T07:17:55.539107+0000","last_fullsized":"2023-11-26T07:17:55.539107+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:17:52.477276+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:17:52.477276+0000","last_clean_scrub_stamp":"2023-11-26T07:17:52.477276+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T15:18:16.303185+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-26T07:17:56.068974+0000","last_change":"2023-11-26T07:17:54.528621+0000","last_active":"2023-11-26T07:17:56.068974+0000","last_peered":"2023-11-26T07:17:56.068974+0000","last_clean":"2023-11-26T07:17:56.068974+0000","last_became_active":"2023-11-26T07:17:54.528333+0000","last_became_peered":"2023-11-26T07:17:54.528333+0000","last_unstale":"2023-11-26T07:17:56.068974+0000","last_undegraded":"2023-11-26T07:17:56.068974+0000","last_fullsized":"2023-11-26T07:17:56.068974+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:17:53.483873+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:17:53.483873+0000","last_clean_scrub_stamp":"2023-11-26T07:17:53.483873+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T13:18:15.521322+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.15","version":"19'3","reported_seq":15,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-26T07:17:55.516929+0000","last_change":"2023-11-26T07:17:53.515987+0000","last_active":"2023-11-26T07:17:55.516929+0000","last_peered":"2023-11-26T07:17:55.516929+0000","last_clean":"2023-11-26T07:17:55.516929+0000","last_became_active":"2023-11-26T07:17:53.515360+0000","last_became_peered":"2023-11-26T07:17:53.515360+0000","last_unstale":"2023-11-26T07:17:55.516929+0000","last_undegraded":"2023-11-26T07:17:55.516929+0000","last_fullsized":"2023-11-26T07:17:55.516929+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:17:52.477276+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:17:52.477276+0000","last_clean_scrub_stamp":"2023-11-26T07:17:52.477276+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T19:07:41.542569+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-26T07:17:56.640029+0000","last_change":"2023-11-26T07:17:54.530537+0000","last_active":"2023-11-26T07:17:56.640029+0000","last_peered":"2023-11-26T07:17:56.640029+0000","last_clean":"2023-11-26T07:17:56.640029+0000","last_became_active":"2023-11-26T07:17:54.530074+0000","last_became_peered":"2023-11-26T07:17:54.530074+0000","last_unstale":"2023-11-26T07:17:56.640029+0000","last_undegraded":"2023-11-26T07:17:56.640029+0000","last_fullsized":"2023-11-26T07:17:56.640029+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:17:53.483873+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:17:53.483873+0000","last_clean_scrub_stamp":"2023-11-26T07:17:53.483873+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T13:45:34.765962+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.14","version":"19'6","reported_seq":18,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-26T07:17:56.639634+0000","last_change":"2023-11-26T07:17:53.522850+0000","last_active":"2023-11-26T07:17:56.639634+0000","last_peered":"2023-11-26T07:17:56.639634+0000","last_clean":"2023-11-26T07:17:56.639634+0000","last_became_active":"2023-11-26T07:17:53.522377+0000","last_became_peered":"2023-11-26T07:17:53.522377+0000","last_unstale":"2023-11-26T07:17:56.639634+0000","last_undegraded":"2023-11-26T07:17:56.639634+0000","last_fullsized":"2023-11-26T07:17:56.639634+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:17:52.477276+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:17:52.477276+0000","last_clean_scrub_stamp":"2023-11-26T07:17:52.477276+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T12:51:16.332936+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-26T07:17:55.763972+0000","last_change":"2023-11-26T07:17:54.529408+0000","last_active":"2023-11-26T07:17:55.763972+0000","last_peered":"2023-11-26T07:17:55.763972+0000","last_clean":"2023-11-26T07:17:55.763972+0000","last_became_active":"2023-11-26T07:17:54.528825+0000","last_became_peered":"2023-11-26T07:17:54.528825+0000","last_unstale":"2023-11-26T07:17:55.763972+0000","last_undegraded":"2023-11-26T07:17:55.763972+0000","last_fullsized":"2023-11-26T07:17:55.763972+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:17:53.483873+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:17:53.483873+0000","last_clean_scrub_stamp":"2023-11-26T07:17:53.483873+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T12:21:10.834625+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.13","version":"19'2","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-26T07:17:56.068998+0000","last_change":"2023-11-26T07:17:53.517650+0000","last_active":"2023-11-26T07:17:56.068998+0000","last_peered":"2023-11-26T07:17:56.068998+0000","last_clean":"2023-11-26T07:17:56.068998+0000","last_became_active":"2023-11-26T07:17:53.517349+0000","last_became_peered":"2023-11-26T07:17:53.517349+0000","last_unstale":"2023-11-26T07:17:56.068998+0000","last_undegraded":"2023-11-26T07:17:56.068998+0000","last_fullsized":"2023-11-26T07:17:56.068998+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:17:52.477276+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:17:52.477276+0000","last_clean_scrub_stamp":"2023-11-26T07:17:52.477276+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T12:51:16.332936+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-26T07:17:56.068804+0000","last_change":"2023-11-26T07:17:54.531742+0000","last_active":"2023-11-26T07:17:56.068804+0000","last_peered":"2023-11-26T07:17:56.068804+0000","last_clean":"2023-11-26T07:17:56.068804+0000","last_became_active":"2023-11-26T07:17:54.531136+0000","last_became_peered":"2023-11-26T07:17:54.531136+0000","last_unstale":"2023-11-26T07:17:56.068804+0000","last_undegraded":"2023-11-26T07:17:56.068804+0000","last_fullsized":"2023-11-26T07:17:56.068804+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:17:53.483873+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:17:53.483873+0000","last_clean_scrub_stamp":"2023-11-26T07:17:53.483873+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T16:07:46.594040+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.12","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-26T07:17:55.538769+0000","last_change":"2023-11-26T07:17:53.522311+0000","last_active":"2023-11-26T07:17:55.538769+0000","last_peered":"2023-11-26T07:17:55.538769+0000","last_clean":"2023-11-26T07:17:55.538769+0000","last_became_active":"2023-11-26T07:17:53.522017+0000","last_became_peered":"2023-11-26T07:17:53.522017+0000","last_unstale":"2023-11-26T07:17:55.538769+0000","last_undegraded":"2023-11-26T07:17:55.538769+0000","last_fullsized":"2023-11-26T07:17:55.538769+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:17:52.477276+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:17:52.477276+0000","last_clean_scrub_stamp":"2023-11-26T07:17:52.477276+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T09:53:28.237406+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-26T07:17:55.763876+0000","last_change":"2023-11-26T07:17:54.531973+0000","last_active":"2023-11-26T07:17:55.763876+0000","last_peered":"2023-11-26T07:17:55.763876+0000","last_clean":"2023-11-26T07:17:55.763876+0000","last_became_active":"2023-11-26T07:17:54.531710+0000","last_became_peered":"2023-11-26T07:17:54.531710+0000","last_unstale":"2023-11-26T07:17:55.763876+0000","last_undegraded":"2023-11-26T07:17:55.763876+0000","last_fullsized":"2023-11-26T07:17:55.763876+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:17:53.483873+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:17:53.483873+0000","last_clean_scrub_stamp":"2023-11-26T07:17:53.483873+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T15:38:12.078699+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.11","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-26T07:17:56.639192+0000","last_change":"2023-11-26T07:17:53.519520+0000","last_active":"2023-11-26T07:17:56.639192+0000","last_peered":"2023-11-26T07:17:56.639192+0000","last_clean":"2023-11-26T07:17:56.639192+0000","last_became_active":"2023-11-26T07:17:53.516001+0000","last_became_peered":"2023-11-26T07:17:53.516001+0000","last_unstale":"2023-11-26T07:17:56.639192+0000","last_undegraded":"2023-11-26T07:17:56.639192+0000","last_fullsized":"2023-11-26T07:17:56.639192+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:17:52.477276+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:17:52.477276+0000","last_clean_scrub_stamp":"2023-11-26T07:17:52.477276+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T19:07:41.542569+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-26T07:17:56.639395+0000","last_change":"2023-11-26T07:17:54.511603+0000","last_active":"2023-11-26T07:17:56.639395+0000","last_peered":"2023-11-26T07:17:56.639395+0000","last_clean":"2023-11-26T07:17:56.639395+0000","last_became_active":"2023-11-26T07:17:54.510939+0000","last_became_peered":"2023-11-26T07:17:54.510939+0000","last_unstale":"2023-11-26T07:17:56.639395+0000","last_undegraded":"2023-11-26T07:17:56.639395+0000","last_fullsized":"2023-11-26T07:17:56.639395+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:17:53.483873+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:17:53.483873+0000","last_clean_scrub_stamp":"2023-11-26T07:17:53.483873+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T09:53:29.244002+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.10","version":"19'8","reported_seq":20,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-26T07:17:55.514684+0000","last_change":"2023-11-26T07:17:53.506514+0000","last_active":"2023-11-26T07:17:55.514684+0000","last_peered":"2023-11-26T07:17:55.514684+0000","last_clean":"2023-11-26T07:17:55.514684+0000","last_became_active":"2023-11-26T07:17:53.506081+0000","last_became_peered":"2023-11-26T07:17:53.506081+0000","last_unstale":"2023-11-26T07:17:55.514684+0000","last_undegraded":"2023-11-26T07:17:55.514684+0000","last_fullsized":"2023-11-26T07:17:55.514684+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:17:52.477276+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:17:52.477276+0000","last_clean_scrub_stamp":"2023-11-26T07:17:52.477276+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T09:43:57.945669+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-26T07:17:55.763817+0000","last_change":"2023-11-26T07:17:54.532538+0000","last_active":"2023-11-26T07:17:55.763817+0000","last_peered":"2023-11-26T07:17:55.763817+0000","last_clean":"2023-11-26T07:17:55.763817+0000","last_became_active":"2023-11-26T07:17:54.531186+0000","last_became_peered":"2023-11-26T07:17:54.531186+0000","last_unstale":"2023-11-26T07:17:55.763817+0000","last_undegraded":"2023-11-26T07:17:55.763817+0000","last_fullsized":"2023-11-26T07:17:55.763817+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:17:53.483873+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:17:53.483873+0000","last_clean_scrub_stamp":"2023-11-26T07:17:53.483873+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T15:49:39.889795+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.f","version":"19'1","reported_seq":13,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-26T07:17:56.640634+0000","last_change":"2023-11-26T07:17:53.509212+0000","last_active":"2023-11-26T07:17:56.640634+0000","last_peered":"2023-11-26T07:17:56.640634+0000","last_clean":"2023-11-26T07:17:56.640634+0000","last_became_active":"2023-11-26T07:17:53.508602+0000","last_became_peered":"2023-11-26T07:17:53.508602+0000","last_unstale":"2023-11-26T07:17:56.640634+0000","last_undegraded":"2023-11-26T07:17:56.640634+0000","last_fullsized":"2023-11-26T07:17:56.640634+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:17:52.477276+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:17:52.477276+0000","last_clean_scrub_stamp":"2023-11-26T07:17:52.477276+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T10:16:37.754573+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-26T07:17:55.764257+0000","last_change":"2023-11-26T07:17:54.531723+0000","last_active":"2023-11-26T07:17:55.764257+0000","last_peered":"2023-11-26T07:17:55.764257+0000","last_clean":"2023-11-26T07:17:55.764257+0000","last_became_active":"2023-11-26T07:17:54.531321+0000","last_became_peered":"2023-11-26T07:17:54.531321+0000","last_unstale":"2023-11-26T07:17:55.764257+0000","last_undegraded":"2023-11-26T07:17:55.764257+0000","last_fullsized":"2023-11-26T07:17:55.764257+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:17:53.483873+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:17:53.483873+0000","last_clean_scrub_stamp":"2023-11-26T07:17:53.483873+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T15:19:56.111921+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.e","version":"19'2","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-26T07:17:55.514998+0000","last_change":"2023-11-26T07:17:53.524243+0000","last_active":"2023-11-26T07:17:55.514998+0000","last_peered":"2023-11-26T07:17:55.514998+0000","last_clean":"2023-11-26T07:17:55.514998+0000","last_became_active":"2023-11-26T07:17:53.523933+0000","last_became_peered":"2023-11-26T07:17:53.523933+0000","last_unstale":"2023-11-26T07:17:55.514998+0000","last_undegraded":"2023-11-26T07:17:55.514998+0000","last_fullsized":"2023-11-26T07:17:55.514998+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:17:52.477276+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:17:52.477276+0000","last_clean_scrub_stamp":"2023-11-26T07:17:52.477276+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T15:44:14.149131+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-26T07:17:55.764439+0000","last_change":"2023-11-26T07:17:54.531576+0000","last_active":"2023-11-26T07:17:55.764439+0000","last_peered":"2023-11-26T07:17:55.764439+0000","last_clean":"2023-11-26T07:17:55.764439+0000","last_became_active":"2023-11-26T07:17:54.531261+0000","last_became_peered":"2023-11-26T07:17:54.531261+0000","last_unstale":"2023-11-26T07:17:55.764439+0000","last_undegraded":"2023-11-26T07:17:55.764439+0000","last_fullsized":"2023-11-26T07:17:55.764439+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:17:53.483873+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:17:53.483873+0000","last_clean_scrub_stamp":"2023-11-26T07:17:53.483873+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T11:46:17.783756+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.d","version":"19'1","reported_seq":13,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-26T07:17:55.516854+0000","last_change":"2023-11-26T07:17:53.512500+0000","last_active":"2023-11-26T07:17:55.516854+0000","last_peered":"2023-11-26T07:17:55.516854+0000","last_clean":"2023-11-26T07:17:55.516854+0000","last_became_active":"2023-11-26T07:17:53.512082+0000","last_became_peered":"2023-11-26T07:17:53.512082+0000","last_unstale":"2023-11-26T07:17:55.516854+0000","last_undegraded":"2023-11-26T07:17:55.516854+0000","last_fullsized":"2023-11-26T07:17:55.516854+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:17:52.477276+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:17:52.477276+0000","last_clean_scrub_stamp":"2023-11-26T07:17:52.477276+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T18:33:31.647310+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-26T07:17:55.539068+0000","last_change":"2023-11-26T07:17:54.528819+0000","last_active":"2023-11-26T07:17:55.539068+0000","last_peered":"2023-11-26T07:17:55.539068+0000","last_clean":"2023-11-26T07:17:55.539068+0000","last_became_active":"2023-11-26T07:17:54.528290+0000","last_became_peered":"2023-11-26T07:17:54.528290+0000","last_unstale":"2023-11-26T07:17:55.539068+0000","last_undegraded":"2023-11-26T07:17:55.539068+0000","last_fullsized":"2023-11-26T07:17:55.539068+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:17:53.483873+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:17:53.483873+0000","last_clean_scrub_stamp":"2023-11-26T07:17:53.483873+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T12:21:10.834625+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.c","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-26T07:17:55.514878+0000","last_change":"2023-11-26T07:17:53.514524+0000","last_active":"2023-11-26T07:17:55.514878+0000","last_peered":"2023-11-26T07:17:55.514878+0000","last_clean":"2023-11-26T07:17:55.514878+0000","last_became_active":"2023-11-26T07:17:53.514070+0000","last_became_peered":"2023-11-26T07:17:53.514070+0000","last_unstale":"2023-11-26T07:17:55.514878+0000","last_undegraded":"2023-11-26T07:17:55.514878+0000","last_fullsized":"2023-11-26T07:17:55.514878+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:17:52.477276+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:17:52.477276+0000","last_clean_scrub_stamp":"2023-11-26T07:17:52.477276+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T15:47:35.416235+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-26T07:17:55.763997+0000","last_change":"2023-11-26T07:17:54.535603+0000","last_active":"2023-11-26T07:17:55.763997+0000","last_peered":"2023-11-26T07:17:55.763997+0000","last_clean":"2023-11-26T07:17:55.763997+0000","last_became_active":"2023-11-26T07:17:54.535343+0000","last_became_peered":"2023-11-26T07:17:54.535343+0000","last_unstale":"2023-11-26T07:17:55.763997+0000","last_undegraded":"2023-11-26T07:17:55.763997+0000","last_fullsized":"2023-11-26T07:17:55.763997+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:17:53.483873+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:17:53.483873+0000","last_clean_scrub_stamp":"2023-11-26T07:17:53.483873+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T18:27:00.305240+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.0","version":"15'117","reported_seq":165,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-11-26T07:17:54.507822+0000","last_change":"2023-11-26T07:17:43.827802+0000","last_active":"2023-11-26T07:17:54.507822+0000","last_peered":"2023-11-26T07:17:54.507822+0000","last_clean":"2023-11-26T07:17:54.507822+0000","last_became_active":"2023-11-26T07:17:43.827308+0000","last_became_peered":"2023-11-26T07:17:43.827308+0000","last_unstale":"2023-11-26T07:17:54.507822+0000","last_undegraded":"2023-11-26T07:17:54.507822+0000","last_fullsized":"2023-11-26T07:17:54.507822+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:17:42.160945+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:17:42.160945+0000","last_clean_scrub_stamp":"2023-11-26T07:17:42.160945+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T08:17:54.167366+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-26T07:17:55.538622+0000","last_change":"2023-11-26T07:17:53.514397+0000","last_active":"2023-11-26T07:17:55.538622+0000","last_peered":"2023-11-26T07:17:55.538622+0000","last_clean":"2023-11-26T07:17:55.538622+0000","last_became_active":"2023-11-26T07:17:53.513876+0000","last_became_peered":"2023-11-26T07:17:53.513876+0000","last_unstale":"2023-11-26T07:17:55.538622+0000","last_undegraded":"2023-11-26T07:17:55.538622+0000","last_fullsized":"2023-11-26T07:17:55.538622+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:17:52.477276+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:17:52.477276+0000","last_clean_scrub_stamp":"2023-11-26T07:17:52.477276+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T18:33:31.647310+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-26T07:17:56.640762+0000","last_change":"2023-11-26T07:17:54.522327+0000","last_active":"2023-11-26T07:17:56.640762+0000","last_peered":"2023-11-26T07:17:56.640762+0000","last_clean":"2023-11-26T07:17:56.640762+0000","last_became_active":"2023-11-26T07:17:54.522058+0000","last_became_peered":"2023-11-26T07:17:54.522058+0000","last_unstale":"2023-11-26T07:17:56.640762+0000","last_undegraded":"2023-11-26T07:17:56.640762+0000","last_fullsized":"2023-11-26T07:17:56.640762+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:17:53.483873+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:17:53.483873+0000","last_clean_scrub_stamp":"2023-11-26T07:17:53.483873+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T11:16:25.801280+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-26T07:17:55.763573+0000","last_change":"2023-11-26T07:17:53.522494+0000","last_active":"2023-11-26T07:17:55.763573+0000","last_peered":"2023-11-26T07:17:55.763573+0000","last_clean":"2023-11-26T07:17:55.763573+0000","last_became_active":"2023-11-26T07:17:53.522221+0000","last_became_peered":"2023-11-26T07:17:53.522221+0000","last_unstale":"2023-11-26T07:17:55.763573+0000","last_undegraded":"2023-11-26T07:17:55.763573+0000","last_fullsized":"2023-11-26T07:17:55.763573+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:17:52.477276+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:17:52.477276+0000","last_clean_scrub_stamp":"2023-11-26T07:17:52.477276+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T13:45:33.759365+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-26T07:17:56.068604+0000","last_change":"2023-11-26T07:17:54.505916+0000","last_active":"2023-11-26T07:17:56.068604+0000","last_peered":"2023-11-26T07:17:56.068604+0000","last_clean":"2023-11-26T07:17:56.068604+0000","last_became_active":"2023-11-26T07:17:54.505618+0000","last_became_peered":"2023-11-26T07:17:54.505618+0000","last_unstale":"2023-11-26T07:17:56.068604+0000","last_undegraded":"2023-11-26T07:17:56.068604+0000","last_fullsized":"2023-11-26T07:17:56.068604+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:17:53.483873+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:17:53.483873+0000","last_clean_scrub_stamp":"2023-11-26T07:17:53.483873+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T19:07:42.549166+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-26T07:17:55.514749+0000","last_change":"2023-11-26T07:17:53.523403+0000","last_active":"2023-11-26T07:17:55.514749+0000","last_peered":"2023-11-26T07:17:55.514749+0000","last_clean":"2023-11-26T07:17:55.514749+0000","last_became_active":"2023-11-26T07:17:53.523061+0000","last_became_peered":"2023-11-26T07:17:53.523061+0000","last_unstale":"2023-11-26T07:17:55.514749+0000","last_undegraded":"2023-11-26T07:17:55.514749+0000","last_fullsized":"2023-11-26T07:17:55.514749+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:17:52.477276+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:17:52.477276+0000","last_clean_scrub_stamp":"2023-11-26T07:17:52.477276+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T09:18:47.261084+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-26T07:17:55.516400+0000","last_change":"2023-11-26T07:17:54.519128+0000","last_active":"2023-11-26T07:17:55.516400+0000","last_peered":"2023-11-26T07:17:55.516400+0000","last_clean":"2023-11-26T07:17:55.516400+0000","last_became_active":"2023-11-26T07:17:54.518918+0000","last_became_peered":"2023-11-26T07:17:54.518918+0000","last_unstale":"2023-11-26T07:17:55.516400+0000","last_undegraded":"2023-11-26T07:17:55.516400+0000","last_fullsized":"2023-11-26T07:17:55.516400+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:17:53.483873+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:17:53.483873+0000","last_clean_scrub_stamp":"2023-11-26T07:17:53.483873+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T07:50:04.472214+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-26T07:17:55.538911+0000","last_change":"2023-11-26T07:17:53.517034+0000","last_active":"2023-11-26T07:17:55.538911+0000","last_peered":"2023-11-26T07:17:55.538911+0000","last_clean":"2023-11-26T07:17:55.538911+0000","last_became_active":"2023-11-26T07:17:53.516689+0000","last_became_peered":"2023-11-26T07:17:53.516689+0000","last_unstale":"2023-11-26T07:17:55.538911+0000","last_undegraded":"2023-11-26T07:17:55.538911+0000","last_fullsized":"2023-11-26T07:17:55.538911+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:17:52.477276+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:17:52.477276+0000","last_clean_scrub_stamp":"2023-11-26T07:17:52.477276+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T16:07:45.587443+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-26T07:17:56.640131+0000","last_change":"2023-11-26T07:17:54.501979+0000","last_active":"2023-11-26T07:17:56.640131+0000","last_peered":"2023-11-26T07:17:56.640131+0000","last_clean":"2023-11-26T07:17:56.640131+0000","last_became_active":"2023-11-26T07:17:54.501682+0000","last_became_peered":"2023-11-26T07:17:54.501682+0000","last_unstale":"2023-11-26T07:17:56.640131+0000","last_undegraded":"2023-11-26T07:17:56.640131+0000","last_fullsized":"2023-11-26T07:17:56.640131+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:17:53.483873+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:17:53.483873+0000","last_clean_scrub_stamp":"2023-11-26T07:17:53.483873+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T12:51:17.339533+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-26T07:17:55.516646+0000","last_change":"2023-11-26T07:17:53.509606+0000","last_active":"2023-11-26T07:17:55.516646+0000","last_peered":"2023-11-26T07:17:55.516646+0000","last_clean":"2023-11-26T07:17:55.516646+0000","last_became_active":"2023-11-26T07:17:53.509268+0000","last_became_peered":"2023-11-26T07:17:53.509268+0000","last_unstale":"2023-11-26T07:17:55.516646+0000","last_undegraded":"2023-11-26T07:17:55.516646+0000","last_fullsized":"2023-11-26T07:17:55.516646+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:17:52.477276+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:17:52.477276+0000","last_clean_scrub_stamp":"2023-11-26T07:17:52.477276+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T16:07:45.587443+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-26T07:17:55.538908+0000","last_change":"2023-11-26T07:17:54.527293+0000","last_active":"2023-11-26T07:17:55.538908+0000","last_peered":"2023-11-26T07:17:55.538908+0000","last_clean":"2023-11-26T07:17:55.538908+0000","last_became_active":"2023-11-26T07:17:54.526666+0000","last_became_peered":"2023-11-26T07:17:54.526666+0000","last_unstale":"2023-11-26T07:17:55.538908+0000","last_undegraded":"2023-11-26T07:17:55.538908+0000","last_fullsized":"2023-11-26T07:17:55.538908+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:17:53.483873+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:17:53.483873+0000","last_clean_scrub_stamp":"2023-11-26T07:17:53.483873+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T17:14:14.930365+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-26T07:17:55.764220+0000","last_change":"2023-11-26T07:17:53.527907+0000","last_active":"2023-11-26T07:17:55.764220+0000","last_peered":"2023-11-26T07:17:55.764220+0000","last_clean":"2023-11-26T07:17:55.764220+0000","last_became_active":"2023-11-26T07:17:53.527624+0000","last_became_peered":"2023-11-26T07:17:53.527624+0000","last_unstale":"2023-11-26T07:17:55.764220+0000","last_undegraded":"2023-11-26T07:17:55.764220+0000","last_fullsized":"2023-11-26T07:17:55.764220+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:17:52.477276+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:17:52.477276+0000","last_clean_scrub_stamp":"2023-11-26T07:17:52.477276+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T11:16:24.794684+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-26T07:17:55.516576+0000","last_change":"2023-11-26T07:17:54.522960+0000","last_active":"2023-11-26T07:17:55.516576+0000","last_peered":"2023-11-26T07:17:55.516576+0000","last_clean":"2023-11-26T07:17:55.516576+0000","last_became_active":"2023-11-26T07:17:54.522658+0000","last_became_peered":"2023-11-26T07:17:54.522658+0000","last_unstale":"2023-11-26T07:17:55.516576+0000","last_undegraded":"2023-11-26T07:17:55.516576+0000","last_fullsized":"2023-11-26T07:17:55.516576+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:17:53.483873+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:17:53.483873+0000","last_clean_scrub_stamp":"2023-11-26T07:17:53.483873+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T13:18:15.521322+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-26T07:17:55.516521+0000","last_change":"2023-11-26T07:17:53.513529+0000","last_active":"2023-11-26T07:17:55.516521+0000","last_peered":"2023-11-26T07:17:55.516521+0000","last_clean":"2023-11-26T07:17:55.516521+0000","last_became_active":"2023-11-26T07:17:53.513125+0000","last_became_peered":"2023-11-26T07:17:53.513125+0000","last_unstale":"2023-11-26T07:17:55.516521+0000","last_undegraded":"2023-11-26T07:17:55.516521+0000","last_fullsized":"2023-11-26T07:17:55.516521+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:17:52.477276+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:17:52.477276+0000","last_clean_scrub_stamp":"2023-11-26T07:17:52.477276+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T09:53:28.237406+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-26T07:17:56.641191+0000","last_change":"2023-11-26T07:17:54.525518+0000","last_active":"2023-11-26T07:17:56.641191+0000","last_peered":"2023-11-26T07:17:56.641191+0000","last_clean":"2023-11-26T07:17:56.641191+0000","last_became_active":"2023-11-26T07:17:54.524972+0000","last_became_peered":"2023-11-26T07:17:54.524972+0000","last_unstale":"2023-11-26T07:17:56.641191+0000","last_undegraded":"2023-11-26T07:17:56.641191+0000","last_fullsized":"2023-11-26T07:17:56.641191+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:17:53.483873+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:17:53.483873+0000","last_clean_scrub_stamp":"2023-11-26T07:17:53.483873+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T15:49:39.889795+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.7","version":"19'2","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-26T07:17:56.640410+0000","last_change":"2023-11-26T07:17:53.523424+0000","last_active":"2023-11-26T07:17:56.640410+0000","last_peered":"2023-11-26T07:17:56.640410+0000","last_clean":"2023-11-26T07:17:56.640410+0000","last_became_active":"2023-11-26T07:17:53.523139+0000","last_became_peered":"2023-11-26T07:17:53.523139+0000","last_unstale":"2023-11-26T07:17:56.640410+0000","last_undegraded":"2023-11-26T07:17:56.640410+0000","last_fullsized":"2023-11-26T07:17:56.640410+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:17:52.477276+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:17:52.477276+0000","last_clean_scrub_stamp":"2023-11-26T07:17:52.477276+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T09:18:47.261084+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-26T07:17:56.069026+0000","last_change":"2023-11-26T07:17:54.523889+0000","last_active":"2023-11-26T07:17:56.069026+0000","last_peered":"2023-11-26T07:17:56.069026+0000","last_clean":"2023-11-26T07:17:56.069026+0000","last_became_active":"2023-11-26T07:17:54.523610+0000","last_became_peered":"2023-11-26T07:17:54.523610+0000","last_unstale":"2023-11-26T07:17:56.069026+0000","last_undegraded":"2023-11-26T07:17:56.069026+0000","last_fullsized":"2023-11-26T07:17:56.069026+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:17:53.483873+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:17:53.483873+0000","last_clean_scrub_stamp":"2023-11-26T07:17:53.483873+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T14:45:51.557148+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-26T07:17:55.763603+0000","last_change":"2023-11-26T07:17:53.521942+0000","last_active":"2023-11-26T07:17:55.763603+0000","last_peered":"2023-11-26T07:17:55.763603+0000","last_clean":"2023-11-26T07:17:55.763603+0000","last_became_active":"2023-11-26T07:17:53.521487+0000","last_became_peered":"2023-11-26T07:17:53.521487+0000","last_unstale":"2023-11-26T07:17:55.763603+0000","last_undegraded":"2023-11-26T07:17:55.763603+0000","last_fullsized":"2023-11-26T07:17:55.763603+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:17:52.477276+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:17:52.477276+0000","last_clean_scrub_stamp":"2023-11-26T07:17:52.477276+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T09:07:52.941378+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-26T07:17:56.640134+0000","last_change":"2023-11-26T07:17:54.516390+0000","last_active":"2023-11-26T07:17:56.640134+0000","last_peered":"2023-11-26T07:17:56.640134+0000","last_clean":"2023-11-26T07:17:56.640134+0000","last_became_active":"2023-11-26T07:17:54.515967+0000","last_became_peered":"2023-11-26T07:17:54.515967+0000","last_unstale":"2023-11-26T07:17:56.640134+0000","last_undegraded":"2023-11-26T07:17:56.640134+0000","last_fullsized":"2023-11-26T07:17:56.640134+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:17:53.483873+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:17:53.483873+0000","last_clean_scrub_stamp":"2023-11-26T07:17:53.483873+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T17:42:29.554668+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.9","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-26T07:17:55.516770+0000","last_change":"2023-11-26T07:17:53.520545+0000","last_active":"2023-11-26T07:17:55.516770+0000","last_peered":"2023-11-26T07:17:55.516770+0000","last_clean":"2023-11-26T07:17:55.516770+0000","last_became_active":"2023-11-26T07:17:53.520076+0000","last_became_peered":"2023-11-26T07:17:53.520076+0000","last_unstale":"2023-11-26T07:17:55.516770+0000","last_undegraded":"2023-11-26T07:17:55.516770+0000","last_fullsized":"2023-11-26T07:17:55.516770+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:17:52.477276+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:17:52.477276+0000","last_clean_scrub_stamp":"2023-11-26T07:17:52.477276+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T12:51:16.332936+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-26T07:17:56.641019+0000","last_change":"2023-11-26T07:17:54.512904+0000","last_active":"2023-11-26T07:17:56.641019+0000","last_peered":"2023-11-26T07:17:56.641019+0000","last_clean":"2023-11-26T07:17:56.641019+0000","last_became_active":"2023-11-26T07:17:54.512619+0000","last_became_peered":"2023-11-26T07:17:54.512619+0000","last_unstale":"2023-11-26T07:17:56.641019+0000","last_undegraded":"2023-11-26T07:17:56.641019+0000","last_fullsized":"2023-11-26T07:17:56.641019+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:17:53.483873+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:17:53.483873+0000","last_clean_scrub_stamp":"2023-11-26T07:17:53.483873+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T14:49:27.883374+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.a","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-26T07:17:55.516544+0000","last_change":"2023-11-26T07:17:53.523572+0000","last_active":"2023-11-26T07:17:55.516544+0000","last_peered":"2023-11-26T07:17:55.516544+0000","last_clean":"2023-11-26T07:17:55.516544+0000","last_became_active":"2023-11-26T07:17:53.523266+0000","last_became_peered":"2023-11-26T07:17:53.523266+0000","last_unstale":"2023-11-26T07:17:55.516544+0000","last_undegraded":"2023-11-26T07:17:55.516544+0000","last_fullsized":"2023-11-26T07:17:55.516544+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:17:52.477276+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:17:52.477276+0000","last_clean_scrub_stamp":"2023-11-26T07:17:52.477276+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T13:59:52.977195+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-26T07:17:56.640963+0000","last_change":"2023-11-26T07:17:54.515437+0000","last_active":"2023-11-26T07:17:56.640963+0000","last_peered":"2023-11-26T07:17:56.640963+0000","last_clean":"2023-11-26T07:17:56.640963+0000","last_became_active":"2023-11-26T07:17:54.515127+0000","last_became_peered":"2023-11-26T07:17:54.515127+0000","last_unstale":"2023-11-26T07:17:56.640963+0000","last_undegraded":"2023-11-26T07:17:56.640963+0000","last_fullsized":"2023-11-26T07:17:56.640963+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:17:53.483873+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:17:53.483873+0000","last_clean_scrub_stamp":"2023-11-26T07:17:53.483873+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T14:45:51.557148+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.b","version":"19'2","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-26T07:17:55.763968+0000","last_change":"2023-11-26T07:17:53.525249+0000","last_active":"2023-11-26T07:17:55.763968+0000","last_peered":"2023-11-26T07:17:55.763968+0000","last_clean":"2023-11-26T07:17:55.763968+0000","last_became_active":"2023-11-26T07:17:53.524957+0000","last_became_peered":"2023-11-26T07:17:53.524957+0000","last_unstale":"2023-11-26T07:17:55.763968+0000","last_undegraded":"2023-11-26T07:17:55.763968+0000","last_fullsized":"2023-11-26T07:17:55.763968+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:17:52.477276+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:17:52.477276+0000","last_clean_scrub_stamp":"2023-11-26T07:17:52.477276+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T19:07:41.542569+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-26T07:17:56.639556+0000","last_change":"2023-11-26T07:17:54.521923+0000","last_active":"2023-11-26T07:17:56.639556+0000","last_peered":"2023-11-26T07:17:56.639556+0000","last_clean":"2023-11-26T07:17:56.639556+0000","last_became_active":"2023-11-26T07:17:54.521503+0000","last_became_peered":"2023-11-26T07:17:54.521503+0000","last_unstale":"2023-11-26T07:17:56.639556+0000","last_undegraded":"2023-11-26T07:17:56.639556+0000","last_fullsized":"2023-11-26T07:17:56.639556+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:17:53.483873+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:17:53.483873+0000","last_clean_scrub_stamp":"2023-11-26T07:17:53.483873+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T16:07:46.594040+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-26T07:17:55.538848+0000","last_change":"2023-11-26T07:17:54.526945+0000","last_active":"2023-11-26T07:17:55.538848+0000","last_peered":"2023-11-26T07:17:55.538848+0000","last_clean":"2023-11-26T07:17:55.538848+0000","last_became_active":"2023-11-26T07:17:54.526110+0000","last_became_peered":"2023-11-26T07:17:54.526110+0000","last_unstale":"2023-11-26T07:17:55.538848+0000","last_undegraded":"2023-11-26T07:17:55.538848+0000","last_fullsized":"2023-11-26T07:17:55.538848+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:17:53.483873+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:17:53.483873+0000","last_clean_scrub_stamp":"2023-11-26T07:17:53.483873+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T10:18:52.982852+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1c","version":"19'2","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-26T07:17:55.763845+0000","last_change":"2023-11-26T07:17:53.524780+0000","last_active":"2023-11-26T07:17:55.763845+0000","last_peered":"2023-11-26T07:17:55.763845+0000","last_clean":"2023-11-26T07:17:55.763845+0000","last_became_active":"2023-11-26T07:17:53.524429+0000","last_became_peered":"2023-11-26T07:17:53.524429+0000","last_unstale":"2023-11-26T07:17:55.763845+0000","last_undegraded":"2023-11-26T07:17:55.763845+0000","last_fullsized":"2023-11-26T07:17:55.763845+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:17:52.477276+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:17:52.477276+0000","last_clean_scrub_stamp":"2023-11-26T07:17:52.477276+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T09:53:28.237406+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-26T07:17:55.539011+0000","last_change":"2023-11-26T07:17:54.523997+0000","last_active":"2023-11-26T07:17:55.539011+0000","last_peered":"2023-11-26T07:17:55.539011+0000","last_clean":"2023-11-26T07:17:55.539011+0000","last_became_active":"2023-11-26T07:17:54.523690+0000","last_became_peered":"2023-11-26T07:17:54.523690+0000","last_unstale":"2023-11-26T07:17:55.539011+0000","last_undegraded":"2023-11-26T07:17:55.539011+0000","last_fullsized":"2023-11-26T07:17:55.539011+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:17:53.483873+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:17:53.483873+0000","last_clean_scrub_stamp":"2023-11-26T07:17:53.483873+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T11:28:10.566680+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1d","version":"19'2","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-26T07:17:55.764098+0000","last_change":"2023-11-26T07:17:53.526233+0000","last_active":"2023-11-26T07:17:55.764098+0000","last_peered":"2023-11-26T07:17:55.764098+0000","last_clean":"2023-11-26T07:17:55.764098+0000","last_became_active":"2023-11-26T07:17:53.525918+0000","last_became_peered":"2023-11-26T07:17:53.525918+0000","last_unstale":"2023-11-26T07:17:55.764098+0000","last_undegraded":"2023-11-26T07:17:55.764098+0000","last_fullsized":"2023-11-26T07:17:55.764098+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:17:52.477276+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:17:52.477276+0000","last_clean_scrub_stamp":"2023-11-26T07:17:52.477276+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T15:18:16.303185+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-26T07:17:56.068892+0000","last_change":"2023-11-26T07:17:54.528760+0000","last_active":"2023-11-26T07:17:56.068892+0000","last_peered":"2023-11-26T07:17:56.068892+0000","last_clean":"2023-11-26T07:17:56.068892+0000","last_became_active":"2023-11-26T07:17:54.528456+0000","last_became_peered":"2023-11-26T07:17:54.528456+0000","last_unstale":"2023-11-26T07:17:56.068892+0000","last_undegraded":"2023-11-26T07:17:56.068892+0000","last_fullsized":"2023-11-26T07:17:56.068892+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:17:53.483873+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:17:53.483873+0000","last_clean_scrub_stamp":"2023-11-26T07:17:53.483873+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T15:18:17.309782+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1e","version":"19'1","reported_seq":13,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-26T07:17:56.641018+0000","last_change":"2023-11-26T07:17:53.523797+0000","last_active":"2023-11-26T07:17:56.641018+0000","last_peered":"2023-11-26T07:17:56.641018+0000","last_clean":"2023-11-26T07:17:56.641018+0000","last_became_active":"2023-11-26T07:17:53.523521+0000","last_became_peered":"2023-11-26T07:17:53.523521+0000","last_unstale":"2023-11-26T07:17:56.641018+0000","last_undegraded":"2023-11-26T07:17:56.641018+0000","last_fullsized":"2023-11-26T07:17:56.641018+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:17:52.477276+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:17:52.477276+0000","last_clean_scrub_stamp":"2023-11-26T07:17:52.477276+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T09:53:28.237406+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-26T07:17:56.640929+0000","last_change":"2023-11-26T07:17:54.517290+0000","last_active":"2023-11-26T07:17:56.640929+0000","last_peered":"2023-11-26T07:17:56.640929+0000","last_clean":"2023-11-26T07:17:56.640929+0000","last_became_active":"2023-11-26T07:17:54.516998+0000","last_became_peered":"2023-11-26T07:17:54.516998+0000","last_unstale":"2023-11-26T07:17:56.640929+0000","last_undegraded":"2023-11-26T07:17:56.640929+0000","last_fullsized":"2023-11-26T07:17:56.640929+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:17:53.483873+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:17:53.483873+0000","last_clean_scrub_stamp":"2023-11-26T07:17:53.483873+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T13:18:15.521322+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1f","version":"19'4","reported_seq":16,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-26T07:17:56.069044+0000","last_change":"2023-11-26T07:17:53.524902+0000","last_active":"2023-11-26T07:17:56.069044+0000","last_peered":"2023-11-26T07:17:56.069044+0000","last_clean":"2023-11-26T07:17:56.069044+0000","last_became_active":"2023-11-26T07:17:53.524530+0000","last_became_peered":"2023-11-26T07:17:53.524530+0000","last_unstale":"2023-11-26T07:17:56.069044+0000","last_undegraded":"2023-11-26T07:17:56.069044+0000","last_fullsized":"2023-11-26T07:17:56.069044+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:17:52.477276+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:17:52.477276+0000","last_clean_scrub_stamp":"2023-11-26T07:17:52.477276+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T09:18:47.261084+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":2410,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":38,"ondisk_log_size":38,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574852,"num_pgs":30,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27808,"kb_used_data":784,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344032,"statfs":{"total":96636764160,"available":96608288768,"internally_reserved":0,"allocated":802816,"data_stored":661860,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":7,"apply_latency_ms":7,"commit_latency_ns":7000000,"apply_latency_ns":7000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.77300000000000002}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.80500000000000005}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.61199999999999999}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.52000000000000002}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.66300000000000003}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.72699999999999998}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.78600000000000003}]}]},{"osd":6,"up_from":13,"seq":55834574854,"num_pgs":16,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27220,"kb_used_data":200,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344620,"statfs":{"total":96636764160,"available":96608890880,"internally_reserved":0,"allocated":204800,"data_stored":66222,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.76400000000000001}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.66500000000000004}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.70299999999999996}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.57599999999999996}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.48299999999999998}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.59299999999999997}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.71799999999999997}]}]},{"osd":5,"up_from":13,"seq":55834574852,"num_pgs":30,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27228,"kb_used_data":204,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344612,"statfs":{"total":96636764160,"available":96608882688,"internally_reserved":0,"allocated":208896,"data_stored":71000,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.86799999999999999}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.80700000000000005}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.76300000000000001}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.83199999999999996}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.66600000000000004}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.54000000000000004}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.93200000000000005}]}]},{"osd":4,"up_from":13,"seq":55834574854,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27220,"kb_used_data":200,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344620,"statfs":{"total":96636764160,"available":96608890880,"internally_reserved":0,"allocated":204800,"data_stored":64736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":4.1689999999999996}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.66100000000000003}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":2.7850000000000001}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.591}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.496}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":2.8330000000000002}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":4.3689999999999998}]}]},{"osd":3,"up_from":13,"seq":55834574855,"num_pgs":19,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27220,"kb_used_data":200,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344620,"statfs":{"total":96636764160,"available":96608890880,"internally_reserved":0,"allocated":204800,"data_stored":66222,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1592,"internal_metadata":27654600},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.39700000000000002}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.32200000000000001}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.41999999999999998}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.878}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.77700000000000002}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.82799999999999996}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.006}]}]},{"osd":2,"up_from":13,"seq":55834574855,"num_pgs":13,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27224,"kb_used_data":200,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344616,"statfs":{"total":96636764160,"available":96608886784,"internally_reserved":0,"allocated":204800,"data_stored":70910,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.42699999999999999}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.47599999999999998}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.40600000000000003}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.89800000000000002}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.70099999999999996}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.80300000000000005}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.88400000000000001}]}]},{"osd":1,"up_from":13,"seq":55834574854,"num_pgs":19,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27232,"kb_used_data":208,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344608,"statfs":{"total":96636764160,"available":96608878592,"internally_reserved":0,"allocated":212992,"data_stored":71070,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.443}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.36499999999999999}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.52600000000000002}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.63700000000000001}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.51300000000000001}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.66700000000000004}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.70699999999999996}]}]},{"osd":0,"up_from":13,"seq":55834574854,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27812,"kb_used_data":792,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344028,"statfs":{"total":96636764160,"available":96608284672,"internally_reserved":0,"allocated":811008,"data_stored":661916,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.45400000000000001}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.40899999999999997}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.51400000000000001}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.70799999999999996}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.871}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.85299999999999998}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.89900000000000002}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":638,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":160,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":126,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-11-26T07:17:58.954 INFO:tasks.cephfs.filesystem.ceph_manager:clean! 2023-11-26T07:17:58.954 INFO:tasks.ceph:Waiting until ceph daemons up and pgs clean... 2023-11-26T07:17:58.954 INFO:tasks.ceph.ceph_manager.ceph:waiting for mgr available 2023-11-26T07:17:58.954 DEBUG:teuthology.orchestra.run.smithi143:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph mgr dump --format=json 2023-11-26T07:17:58.955 INFO:tasks.daemonwatchdog.daemon_watchdog:watchdog starting 2023-11-26T07:17:59.132 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:17:59.126+0000 7f7ebc501640 1 Processor -- start 2023-11-26T07:17:59.132 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:17:59.126+0000 7f7ebc501640 1 -- start start 2023-11-26T07:17:59.134 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:17:59.130+0000 7f7ebc501640 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7f7eb413e290 0x7f7eb40cf280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:17:59.134 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:17:59.130+0000 7f7ebc501640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f7eb40cf7c0 0x7f7eb415efa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:17:59.134 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:17:59.130+0000 7f7ebc501640 1 -- --> v1:172.21.15.188:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7eb406c520 con 0x7f7eb415f4e0 2023-11-26T07:17:59.134 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:17:59.130+0000 7f7ebc501640 1 -- --> v2:172.21.15.188:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7eb40cfc40 con 0x7f7eb413e290 2023-11-26T07:17:59.134 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:17:59.130+0000 7f7ebc501640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7eb406c800 con 0x7f7eb40cf7c0 2023-11-26T07:17:59.135 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:17:59.130+0000 7f7eba29d640 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7f7eb413e290 0x7f7eb40cf280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:17:59.135 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:17:59.130+0000 7f7eb9a9c640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f7eb40cf7c0 0x7f7eb415efa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:17:59.135 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:17:59.130+0000 7f7ebaa9e640 1 --1- >> v1:172.21.15.188:6789/0 conn(0x7f7eb415f4e0 0x7f7eb41408e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.188:6789/0 says I am v1:172.21.15.143:39902/0 (socket says 172.21.15.143:39902) 2023-11-26T07:17:59.135 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:17:59.130+0000 7f7ebaa9e640 1 -- 172.21.15.143:0/4039684936 learned_addr learned my addr 172.21.15.143:0/4039684936 (peer_addr_for_me v1:172.21.15.143:0/0) 2023-11-26T07:17:59.135 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:17:59.130+0000 7f7eba29d640 1 --2- 172.21.15.143:0/4039684936 >> v2:172.21.15.188:3300/0 conn(0x7f7eb413e290 0x7f7eb40cf280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.188:3300/0 says I am v2:172.21.15.143:45488/0 (socket says 172.21.15.143:45488) 2023-11-26T07:17:59.136 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:17:59.130+0000 7f7eb9a9c640 1 -- 172.21.15.143:0/4039684936 >> v2:172.21.15.188:3300/0 conn(0x7f7eb413e290 msgr2=0x7f7eb40cf280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-26T07:17:59.136 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:17:59.130+0000 7f7eb9a9c640 1 --2- 172.21.15.143:0/4039684936 >> v2:172.21.15.188:3300/0 conn(0x7f7eb413e290 0x7f7eb40cf280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:17:59.136 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:17:59.130+0000 7f7eb9a9c640 1 -- 172.21.15.143:0/4039684936 >> v1:172.21.15.188:6789/0 conn(0x7f7eb415f4e0 legacy=0x7f7eb41408e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:17:59.136 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:17:59.130+0000 7f7eb9a9c640 1 -- 172.21.15.143:0/4039684936 --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7eb406cae0 con 0x7f7eb40cf7c0 2023-11-26T07:17:59.136 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:17:59.130+0000 7f7eb929b640 1 -- 172.21.15.143:0/4039684936 <== mon.1 v1:172.21.15.188:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 834310503 0 0) 0x7f7eb406c520 con 0x7f7eb415f4e0 2023-11-26T07:17:59.137 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:17:59.134+0000 7f7eb9a9c640 1 --2- 172.21.15.143:0/4039684936 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f7eb40cf7c0 0x7f7eb415efa0 secure :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0x7f7ea40097c0 tx=0x7f7ea401c450 comp rx=0 tx=0).ready entity=mon.2 client_cookie=41849576c8d0218e server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:17:59.138 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:17:59.134+0000 7f7eb929b640 1 -- 172.21.15.143:0/4039684936 <== mon.2 v2:172.21.15.188:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7ea40ab650 con 0x7f7eb40cf7c0 2023-11-26T07:17:59.138 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:17:59.134+0000 7f7eb929b640 1 -- 172.21.15.143:0/4039684936 <== mon.2 v2:172.21.15.188:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7ea40ab7b0 con 0x7f7eb40cf7c0 2023-11-26T07:17:59.138 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:17:59.134+0000 7f7eb929b640 1 -- 172.21.15.143:0/4039684936 <== mon.2 v2:172.21.15.188:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7ea40abb40 con 0x7f7eb40cf7c0 2023-11-26T07:17:59.139 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:17:59.134+0000 7f7ebc501640 1 -- 172.21.15.143:0/4039684936 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f7eb40cf7c0 msgr2=0x7f7eb415efa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:17:59.139 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:17:59.134+0000 7f7ebc501640 1 --2- 172.21.15.143:0/4039684936 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f7eb40cf7c0 0x7f7eb415efa0 secure :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0x7f7ea40097c0 tx=0x7f7ea401c450 comp rx=0 tx=0).stop 2023-11-26T07:17:59.139 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:17:59.134+0000 7f7ebc501640 1 -- 172.21.15.143:0/4039684936 shutdown_connections 2023-11-26T07:17:59.140 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:17:59.134+0000 7f7ebc501640 1 --2- 172.21.15.143:0/4039684936 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f7eb40cf7c0 0x7f7eb415efa0 unknown :-1 s=CLOSED pgs=32 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:17:59.140 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:17:59.134+0000 7f7ebc501640 1 --2- 172.21.15.143:0/4039684936 >> v2:172.21.15.188:3300/0 conn(0x7f7eb413e290 0x7f7eb40cf280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:17:59.140 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:17:59.134+0000 7f7ebc501640 1 -- 172.21.15.143:0/4039684936 >> 172.21.15.143:0/4039684936 conn(0x7f7eb406a890 msgr2=0x7f7eb406aca0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:17:59.140 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:17:59.134+0000 7f7ebc501640 1 -- 172.21.15.143:0/4039684936 shutdown_connections 2023-11-26T07:17:59.140 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:17:59.134+0000 7f7ebc501640 1 -- 172.21.15.143:0/4039684936 wait complete. 2023-11-26T07:17:59.141 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:17:59.138+0000 7f7ebc501640 1 Processor -- start 2023-11-26T07:17:59.141 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:17:59.138+0000 7f7ebc501640 1 -- start start 2023-11-26T07:17:59.143 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:17:59.138+0000 7f7ebc501640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f7eb40cf7c0 0x7f7eb41373b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:17:59.143 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:17:59.138+0000 7f7ebc501640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f7eb413e290 0x7f7eb40aeb60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:17:59.144 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:17:59.138+0000 7f7ebc501640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f7eb415f4e0 0x7f7eb4071ba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:17:59.144 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:17:59.138+0000 7f7ebc501640 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7eb4137cc0 con 0x7f7eb40cf7c0 2023-11-26T07:17:59.144 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:17:59.138+0000 7f7ebc501640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7eb40af0a0 con 0x7f7eb415f4e0 2023-11-26T07:17:59.144 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:17:59.138+0000 7f7ebc501640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7eb40b1070 con 0x7f7eb413e290 2023-11-26T07:17:59.144 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:17:59.138+0000 7f7eba29d640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f7eb40cf7c0 0x7f7eb41373b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:17:59.144 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:17:59.138+0000 7f7eba29d640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f7eb40cf7c0 0x7f7eb41373b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:46300/0 (socket says 172.21.15.143:46300) 2023-11-26T07:17:59.144 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:17:59.138+0000 7f7eba29d640 1 -- 172.21.15.143:0/1214269546 learned_addr learned my addr 172.21.15.143:0/1214269546 (peer_addr_for_me v2:172.21.15.143:0/0) 2023-11-26T07:17:59.144 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:17:59.138+0000 7f7eb9a9c640 1 --2- 172.21.15.143:0/1214269546 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f7eb413e290 0x7f7eb40aeb60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:17:59.144 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:17:59.138+0000 7f7ebaa9e640 1 --2- 172.21.15.143:0/1214269546 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f7eb415f4e0 0x7f7eb4071ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:17:59.144 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:17:59.138+0000 7f7eba29d640 1 -- 172.21.15.143:0/1214269546 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f7eb413e290 msgr2=0x7f7eb40aeb60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:17:59.144 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:17:59.138+0000 7f7eba29d640 1 --2- 172.21.15.143:0/1214269546 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f7eb413e290 0x7f7eb40aeb60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:17:59.145 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:17:59.138+0000 7f7eba29d640 1 -- 172.21.15.143:0/1214269546 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f7eb415f4e0 msgr2=0x7f7eb4071ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:17:59.145 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:17:59.138+0000 7f7eba29d640 1 --2- 172.21.15.143:0/1214269546 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f7eb415f4e0 0x7f7eb4071ba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:17:59.145 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:17:59.138+0000 7f7eba29d640 1 -- 172.21.15.143:0/1214269546 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7ea4009420 con 0x7f7eb40cf7c0 2023-11-26T07:17:59.145 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:17:59.138+0000 7f7ebaa9e640 1 --2- 172.21.15.143:0/1214269546 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f7eb415f4e0 0x7f7eb4071ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-26T07:17:59.145 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:17:59.138+0000 7f7eba29d640 1 --2- 172.21.15.143:0/1214269546 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f7eb40cf7c0 0x7f7eb41373b0 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7f7eb000e930 tx=0x7f7eb000edf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:17:59.145 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:17:59.138+0000 7f7eaaffd640 1 -- 172.21.15.143:0/1214269546 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7eb0009690 con 0x7f7eb40cf7c0 2023-11-26T07:17:59.145 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:17:59.138+0000 7f7eaaffd640 1 -- 172.21.15.143:0/1214269546 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7eb000c9a0 con 0x7f7eb40cf7c0 2023-11-26T07:17:59.145 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:17:59.138+0000 7f7eaaffd640 1 -- 172.21.15.143:0/1214269546 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7eb000cd10 con 0x7f7eb40cf7c0 2023-11-26T07:17:59.145 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:17:59.138+0000 7f7ebc501640 1 -- 172.21.15.143:0/1214269546 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7eb415dd60 con 0x7f7eb40cf7c0 2023-11-26T07:17:59.145 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:17:59.138+0000 7f7ebc501640 1 -- 172.21.15.143:0/1214269546 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7eb415e2c0 con 0x7f7eb40cf7c0 2023-11-26T07:17:59.146 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:17:59.142+0000 7f7ebc501640 1 -- 172.21.15.143:0/1214269546 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7eb4067680 con 0x7f7eb40cf7c0 2023-11-26T07:17:59.146 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:17:59.142+0000 7f7eaaffd640 1 -- 172.21.15.143:0/1214269546 <== mon.0 v2:172.21.15.143:3300/0 4 ==== mgrmap(e 5) v1 ==== 85852+0+0 (secure 0 0 0) 0x7f7eb0013470 con 0x7f7eb40cf7c0 2023-11-26T07:17:59.147 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:17:59.142+0000 7f7eaaffd640 1 --2- 172.21.15.143:0/1214269546 >> [v2:172.21.15.143:6832/2150557544,v1:172.21.15.143:6833/2150557544] conn(0x7f7e98066470 0x7f7e98068930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:17:59.147 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:17:59.142+0000 7f7eaaffd640 1 -- 172.21.15.143:0/1214269546 <== mon.0 v2:172.21.15.143:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f7eb0083ff0 con 0x7f7eb40cf7c0 2023-11-26T07:17:59.147 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:17:59.142+0000 7f7eb9a9c640 1 --2- 172.21.15.143:0/1214269546 >> [v2:172.21.15.143:6832/2150557544,v1:172.21.15.143:6833/2150557544] conn(0x7f7e98066470 0x7f7e98068930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:17:59.147 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:17:59.142+0000 7f7eb9a9c640 1 --2- 172.21.15.143:0/1214269546 >> [v2:172.21.15.143:6832/2150557544,v1:172.21.15.143:6833/2150557544] conn(0x7f7e98066470 0x7f7e98068930 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7f7ea40af000 tx=0x7f7ea40a7000 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:17:59.151 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:17:59.146+0000 7f7eaaffd640 1 -- 172.21.15.143:0/1214269546 <== mon.0 v2:172.21.15.143:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172649 (secure 0 0 0) 0x7f7eb00515d0 con 0x7f7eb40cf7c0 2023-11-26T07:17:59.336 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:17:59.330+0000 7f7ebc501640 1 -- 172.21.15.143:0/1214269546 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "mgr dump", "format": "json"} v 0) v1 -- 0x7f7eb406df50 con 0x7f7eb40cf7c0 2023-11-26T07:17:59.339 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:17:59.334+0000 7f7eaaffd640 1 -- 172.21.15.143:0/1214269546 <== mon.0 v2:172.21.15.143:3300/0 7 ==== mon_command_ack([{"prefix": "mgr dump", "format": "json"}]=0 v5) v1 ==== 74+0+164439 (secure 0 0 0) 0x7f7eb0017090 con 0x7f7eb40cf7c0 2023-11-26T07:17:59.340 INFO:teuthology.orchestra.run.smithi143.stdout: 2023-11-26T07:17:59.342 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:17:59.338+0000 7f7ebc501640 1 -- 172.21.15.143:0/1214269546 >> [v2:172.21.15.143:6832/2150557544,v1:172.21.15.143:6833/2150557544] conn(0x7f7e98066470 msgr2=0x7f7e98068930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:17:59.342 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:17:59.338+0000 7f7ebc501640 1 --2- 172.21.15.143:0/1214269546 >> [v2:172.21.15.143:6832/2150557544,v1:172.21.15.143:6833/2150557544] conn(0x7f7e98066470 0x7f7e98068930 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7f7ea40af000 tx=0x7f7ea40a7000 comp rx=0 tx=0).stop 2023-11-26T07:17:59.342 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:17:59.338+0000 7f7ebc501640 1 -- 172.21.15.143:0/1214269546 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f7eb40cf7c0 msgr2=0x7f7eb41373b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:17:59.342 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:17:59.338+0000 7f7ebc501640 1 --2- 172.21.15.143:0/1214269546 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f7eb40cf7c0 0x7f7eb41373b0 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7f7eb000e930 tx=0x7f7eb000edf0 comp rx=0 tx=0).stop 2023-11-26T07:17:59.342 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:17:59.338+0000 7f7ebc501640 1 -- 172.21.15.143:0/1214269546 shutdown_connections 2023-11-26T07:17:59.343 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:17:59.338+0000 7f7ebc501640 1 --2- 172.21.15.143:0/1214269546 >> [v2:172.21.15.143:6832/2150557544,v1:172.21.15.143:6833/2150557544] conn(0x7f7e98066470 0x7f7e98068930 unknown :-1 s=CLOSED pgs=74 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:17:59.343 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:17:59.338+0000 7f7ebc501640 1 --2- 172.21.15.143:0/1214269546 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f7eb415f4e0 0x7f7eb4071ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:17:59.343 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:17:59.338+0000 7f7ebc501640 1 --2- 172.21.15.143:0/1214269546 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f7eb413e290 0x7f7eb40aeb60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:17:59.343 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:17:59.338+0000 7f7ebc501640 1 --2- 172.21.15.143:0/1214269546 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f7eb40cf7c0 0x7f7eb41373b0 unknown :-1 s=CLOSED pgs=91 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:17:59.343 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:17:59.338+0000 7f7ebc501640 1 -- 172.21.15.143:0/1214269546 >> 172.21.15.143:0/1214269546 conn(0x7f7eb406a890 msgr2=0x7f7eb415a150 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:17:59.343 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:17:59.338+0000 7f7ebc501640 1 -- 172.21.15.143:0/1214269546 shutdown_connections 2023-11-26T07:17:59.343 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:17:59.338+0000 7f7ebc501640 1 -- 172.21.15.143:0/1214269546 wait complete. 2023-11-26T07:17:59.361 INFO:teuthology.orchestra.run.smithi143.stdout:{"epoch":5,"active_gid":4100,"active_name":"y","active_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6832","nonce":2150557544},{"type":"v1","addr":"172.21.15.143:6833","nonce":2150557544}]},"active_addr":"172.21.15.143:6833/2150557544","active_change":"2023-11-26T07:17:33.680133+0000","active_mgr_features":4540138322906710015,"available":true,"standbys":[{"gid":4108,"name":"x","mgr_features":4540138322906710015,"available_modules":[{"name":"alerts","can_run":true,"error_string":"","module_options":{"interval":{"name":"interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"How frequently to reexamine health status","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"smtp_destination":{"name":"smtp_destination","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Email address to send alerts to","long_desc":"","tags":[],"see_also":[]},"smtp_from_name":{"name":"smtp_from_name","type":"str","level":"advanced","flags":1,"default_value":"Ceph","min":"","max":"","enum_allowed":[],"desc":"Email From: name","long_desc":"","tags":[],"see_also":[]},"smtp_host":{"name":"smtp_host","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_password":{"name":"smtp_password","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Password to authenticate with","long_desc":"","tags":[],"see_also":[]},"smtp_port":{"name":"smtp_port","type":"int","level":"advanced","flags":1,"default_value":"465","min":"","max":"","enum_allowed":[],"desc":"SMTP port","long_desc":"","tags":[],"see_also":[]},"smtp_sender":{"name":"smtp_sender","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP envelope sender","long_desc":"","tags":[],"see_also":[]},"smtp_ssl":{"name":"smtp_ssl","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Use SSL to connect to SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_user":{"name":"smtp_user","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"User to authenticate as","long_desc":"","tags":[],"see_also":[]}}},{"name":"balancer","can_run":true,"error_string":"","module_options":{"active":{"name":"active","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"automatically balance PGs across cluster","long_desc":"","tags":[],"see_also":[]},"begin_time":{"name":"begin_time","type":"str","level":"advanced","flags":1,"default_value":"0000","min":"","max":"","enum_allowed":[],"desc":"beginning time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"begin_weekday":{"name":"begin_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to this day of the week or later","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"crush_compat_max_iterations":{"name":"crush_compat_max_iterations","type":"uint","level":"advanced","flags":1,"default_value":"25","min":"1","max":"250","enum_allowed":[],"desc":"maximum number of iterations to attempt optimization","long_desc":"","tags":[],"see_also":[]},"crush_compat_metrics":{"name":"crush_compat_metrics","type":"str","level":"advanced","flags":1,"default_value":"pgs,objects,bytes","min":"","max":"","enum_allowed":[],"desc":"metrics with which to calculate OSD utilization","long_desc":"Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.","tags":[],"see_also":[]},"crush_compat_step":{"name":"crush_compat_step","type":"float","level":"advanced","flags":1,"default_value":"0.5","min":"0.001","max":"0.999","enum_allowed":[],"desc":"aggressiveness of optimization","long_desc":".99 is very aggressive, .01 is less aggressive","tags":[],"see_also":[]},"end_time":{"name":"end_time","type":"str","level":"advanced","flags":1,"default_value":"2359","min":"","max":"","enum_allowed":[],"desc":"ending time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"end_weekday":{"name":"end_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to days of the week earlier than this","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_score":{"name":"min_score","type":"float","level":"advanced","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"minimum score, below which no optimization is attempted","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":1,"default_value":"upmap","min":"","max":"","enum_allowed":["crush-compat","none","upmap"],"desc":"Balancer mode","long_desc":"","tags":[],"see_also":[]},"pool_ids":{"name":"pool_ids","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"pools which the automatic balancing will be limited to","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and attempt optimization","long_desc":"","tags":[],"see_also":[]},"upmap_max_deviation":{"name":"upmap_max_deviation","type":"int","level":"advanced","flags":1,"default_value":"5","min":"1","max":"","enum_allowed":[],"desc":"deviation below which no optimization is attempted","long_desc":"If the number of PGs are within this count then no optimization is attempted","tags":[],"see_also":[]},"upmap_max_optimizations":{"name":"upmap_max_optimizations","type":"uint","level":"advanced","flags":1,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"maximum upmap optimizations to make per attempt","long_desc":"","tags":[],"see_also":[]}}},{"name":"cephadm","can_run":true,"error_string":"","module_options":{"agent_down_multiplier":{"name":"agent_down_multiplier","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"","max":"","enum_allowed":[],"desc":"Multiplied by agent refresh rate to calculate how long agent must not report before being marked down","long_desc":"","tags":[],"see_also":[]},"agent_refresh_rate":{"name":"agent_refresh_rate","type":"secs","level":"advanced","flags":0,"default_value":"20","min":"","max":"","enum_allowed":[],"desc":"How often agent on each host will try to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"agent_starting_port":{"name":"agent_starting_port","type":"int","level":"advanced","flags":0,"default_value":"4721","min":"","max":"","enum_allowed":[],"desc":"First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)","long_desc":"","tags":[],"see_also":[]},"allow_ptrace":{"name":"allow_ptrace","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow SYS_PTRACE capability on ceph containers","long_desc":"The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.","tags":[],"see_also":[]},"autotune_interval":{"name":"autotune_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to autotune daemon memory","long_desc":"","tags":[],"see_also":[]},"autotune_memory_target_ratio":{"name":"autotune_memory_target_ratio","type":"float","level":"advanced","flags":0,"default_value":"0.7","min":"","max":"","enum_allowed":[],"desc":"ratio of total system memory to divide amongst autotuned daemons","long_desc":"","tags":[],"see_also":[]},"cephadm_log_destination":{"name":"cephadm_log_destination","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":["file","file,syslog","syslog"],"desc":"Destination for cephadm command's persistent logging","long_desc":"","tags":[],"see_also":[]},"cgroups_split":{"name":"cgroups_split","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Pass --cgroups=split when cephadm creates containers (currently podman only)","long_desc":"","tags":[],"see_also":[]},"config_checks_enabled":{"name":"config_checks_enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable or disable the cephadm configuration analysis","long_desc":"","tags":[],"see_also":[]},"config_dashboard":{"name":"config_dashboard","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"manage configs like API endpoints in Dashboard.","long_desc":"","tags":[],"see_also":[]},"container_image_alertmanager":{"name":"container_image_alertmanager","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/alertmanager:v0.25.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_base":{"name":"container_image_base","type":"str","level":"advanced","flags":1,"default_value":"quay.io/ceph/ceph","min":"","max":"","enum_allowed":[],"desc":"Container image name, without the tag","long_desc":"","tags":[],"see_also":[]},"container_image_elasticsearch":{"name":"container_image_elasticsearch","type":"str","level":"advanced","flags":0,"default_value":"quay.io/omrizeneva/elasticsearch:6.8.23","min":"","max":"","enum_allowed":[],"desc":"elasticsearch container image","long_desc":"","tags":[],"see_also":[]},"container_image_grafana":{"name":"container_image_grafana","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/ceph-grafana:9.4.7","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_haproxy":{"name":"container_image_haproxy","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/haproxy:2.3","min":"","max":"","enum_allowed":[],"desc":"HAproxy container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_agent":{"name":"container_image_jaeger_agent","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-agent:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger agent container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_collector":{"name":"container_image_jaeger_collector","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-collector:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger collector container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_query":{"name":"container_image_jaeger_query","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-query:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger query container image","long_desc":"","tags":[],"see_also":[]},"container_image_keepalived":{"name":"container_image_keepalived","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/keepalived:2.2.4","min":"","max":"","enum_allowed":[],"desc":"Keepalived container image","long_desc":"","tags":[],"see_also":[]},"container_image_loki":{"name":"container_image_loki","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/loki:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Loki container image","long_desc":"","tags":[],"see_also":[]},"container_image_node_exporter":{"name":"container_image_node_exporter","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/node-exporter:v1.5.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_nvmeof":{"name":"container_image_nvmeof","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/nvmeof:latest","min":"","max":"","enum_allowed":[],"desc":"Nvme-of container image","long_desc":"","tags":[],"see_also":[]},"container_image_prometheus":{"name":"container_image_prometheus","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/prometheus:v2.43.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_promtail":{"name":"container_image_promtail","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/promtail:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Promtail container image","long_desc":"","tags":[],"see_also":[]},"container_image_snmp_gateway":{"name":"container_image_snmp_gateway","type":"str","level":"advanced","flags":0,"default_value":"docker.io/maxwo/snmp-notifier:v1.2.1","min":"","max":"","enum_allowed":[],"desc":"SNMP Gateway container image","long_desc":"","tags":[],"see_also":[]},"container_init":{"name":"container_init","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Run podman/docker with `--init`","long_desc":"","tags":[],"see_also":[]},"daemon_cache_timeout":{"name":"daemon_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"seconds to cache service (daemon) inventory","long_desc":"","tags":[],"see_also":[]},"default_cephadm_command_timeout":{"name":"default_cephadm_command_timeout","type":"secs","level":"advanced","flags":0,"default_value":"900","min":"","max":"","enum_allowed":[],"desc":"Default timeout applied to cephadm commands run directly on the host (in seconds)","long_desc":"","tags":[],"see_also":[]},"default_registry":{"name":"default_registry","type":"str","level":"advanced","flags":0,"default_value":"docker.io","min":"","max":"","enum_allowed":[],"desc":"Search-registry to which we should normalize unqualified image names. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"device_cache_timeout":{"name":"device_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"1800","min":"","max":"","enum_allowed":[],"desc":"seconds to cache device inventory","long_desc":"","tags":[],"see_also":[]},"device_enhanced_scan":{"name":"device_enhanced_scan","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use libstoragemgmt during device scans","long_desc":"","tags":[],"see_also":[]},"facts_cache_timeout":{"name":"facts_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"seconds to cache host facts data","long_desc":"","tags":[],"see_also":[]},"host_check_interval":{"name":"host_check_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to perform a host check","long_desc":"","tags":[],"see_also":[]},"inventory_list_all":{"name":"inventory_list_all","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_refresh_metadata":{"name":"log_refresh_metadata","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"log to the \"cephadm\" cluster log channel\"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf":{"name":"manage_etc_ceph_ceph_conf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Manage and own /etc/ceph/ceph.conf on the hosts.","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf_hosts":{"name":"manage_etc_ceph_ceph_conf_hosts","type":"str","level":"advanced","flags":0,"default_value":"*","min":"","max":"","enum_allowed":[],"desc":"PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf","long_desc":"","tags":[],"see_also":[]},"max_count_per_host":{"name":"max_count_per_host","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of daemons per service per host","long_desc":"","tags":[],"see_also":[]},"max_osd_draining_count":{"name":"max_osd_draining_count","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of osds that will be drained simultaneously when osds are removed","long_desc":"","tags":[],"see_also":[]},"migration_current":{"name":"migration_current","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"internal - do not modify","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":0,"default_value":"root","min":"","max":"","enum_allowed":["cephadm-package","root"],"desc":"mode for remote execution of cephadm","long_desc":"","tags":[],"see_also":[]},"prometheus_alerts_path":{"name":"prometheus_alerts_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/prometheus/ceph/ceph_default_alerts.yml","min":"","max":"","enum_allowed":[],"desc":"location of alerts to include in prometheus deployments","long_desc":"","tags":[],"see_also":[]},"registry_insecure":{"name":"registry_insecure","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Registry is to be considered insecure (no TLS available). Only for development purposes.","long_desc":"","tags":[],"see_also":[]},"registry_password":{"name":"registry_password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository password. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"registry_url":{"name":"registry_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Registry url for login purposes. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"registry_username":{"name":"registry_username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository username. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"secure_monitoring_stack":{"name":"secure_monitoring_stack","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable TLS security for all the monitoring stack daemons","long_desc":"","tags":[],"see_also":[]},"service_discovery_port":{"name":"service_discovery_port","type":"int","level":"advanced","flags":0,"default_value":"8765","min":"","max":"","enum_allowed":[],"desc":"cephadm service discovery port","long_desc":"","tags":[],"see_also":[]},"ssh_config_file":{"name":"ssh_config_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"customized SSH config file to connect to managed hosts","long_desc":"","tags":[],"see_also":[]},"use_agent":{"name":"use_agent","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use cephadm agent on each host to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"use_repo_digest":{"name":"use_repo_digest","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Automatically convert image tags to image digest. Make sure all daemons use the same image","long_desc":"","tags":[],"see_also":[]},"warn_on_failed_host_check":{"name":"warn_on_failed_host_check","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if the host check fails","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_daemons":{"name":"warn_on_stray_daemons","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected that are not managed by cephadm","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_hosts":{"name":"warn_on_stray_hosts","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected on a host that is not managed by cephadm","long_desc":"","tags":[],"see_also":[]}}},{"name":"crash","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"retain_interval":{"name":"retain_interval","type":"secs","level":"advanced","flags":1,"default_value":"31536000","min":"","max":"","enum_allowed":[],"desc":"how long to retain crashes before pruning them","long_desc":"","tags":[],"see_also":[]},"warn_recent_interval":{"name":"warn_recent_interval","type":"secs","level":"advanced","flags":1,"default_value":"1209600","min":"","max":"","enum_allowed":[],"desc":"time interval in which to warn about recent crashes","long_desc":"","tags":[],"see_also":[]}}},{"name":"dashboard","can_run":true,"error_string":"","module_options":{"ACCOUNT_LOCKOUT_ATTEMPTS":{"name":"ACCOUNT_LOCKOUT_ATTEMPTS","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_HOST":{"name":"ALERTMANAGER_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_SSL_VERIFY":{"name":"ALERTMANAGER_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_ENABLED":{"name":"AUDIT_API_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_LOG_PAYLOAD":{"name":"AUDIT_API_LOG_PAYLOAD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ENABLE_BROWSABLE_API":{"name":"ENABLE_BROWSABLE_API","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_CEPHFS":{"name":"FEATURE_TOGGLE_CEPHFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_DASHBOARD":{"name":"FEATURE_TOGGLE_DASHBOARD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_ISCSI":{"name":"FEATURE_TOGGLE_ISCSI","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_MIRRORING":{"name":"FEATURE_TOGGLE_MIRRORING","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_NFS":{"name":"FEATURE_TOGGLE_NFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RBD":{"name":"FEATURE_TOGGLE_RBD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RGW":{"name":"FEATURE_TOGGLE_RGW","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE":{"name":"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_PASSWORD":{"name":"GRAFANA_API_PASSWORD","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_SSL_VERIFY":{"name":"GRAFANA_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_URL":{"name":"GRAFANA_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_USERNAME":{"name":"GRAFANA_API_USERNAME","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_FRONTEND_API_URL":{"name":"GRAFANA_FRONTEND_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_UPDATE_DASHBOARDS":{"name":"GRAFANA_UPDATE_DASHBOARDS","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISCSI_API_SSL_VERIFICATION":{"name":"ISCSI_API_SSL_VERIFICATION","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISSUE_TRACKER_API_KEY":{"name":"ISSUE_TRACKER_API_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_HOST":{"name":"PROMETHEUS_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_SSL_VERIFY":{"name":"PROMETHEUS_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_COMPLEXITY_ENABLED":{"name":"PWD_POLICY_CHECK_COMPLEXITY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED":{"name":"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_LENGTH_ENABLED":{"name":"PWD_POLICY_CHECK_LENGTH_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_OLDPWD_ENABLED":{"name":"PWD_POLICY_CHECK_OLDPWD_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_USERNAME_ENABLED":{"name":"PWD_POLICY_CHECK_USERNAME_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_ENABLED":{"name":"PWD_POLICY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_EXCLUSION_LIST":{"name":"PWD_POLICY_EXCLUSION_LIST","type":"str","level":"advanced","flags":0,"default_value":"osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_COMPLEXITY":{"name":"PWD_POLICY_MIN_COMPLEXITY","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_LENGTH":{"name":"PWD_POLICY_MIN_LENGTH","type":"int","level":"advanced","flags":0,"default_value":"8","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"REST_REQUESTS_TIMEOUT":{"name":"REST_REQUESTS_TIMEOUT","type":"int","level":"advanced","flags":0,"default_value":"45","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ACCESS_KEY":{"name":"RGW_API_ACCESS_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ADMIN_RESOURCE":{"name":"RGW_API_ADMIN_RESOURCE","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SECRET_KEY":{"name":"RGW_API_SECRET_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SSL_VERIFY":{"name":"RGW_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"UNSAFE_TLS_v1_2":{"name":"UNSAFE_TLS_v1_2","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_SPAN":{"name":"USER_PWD_EXPIRATION_SPAN","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_1":{"name":"USER_PWD_EXPIRATION_WARNING_1","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_2":{"name":"USER_PWD_EXPIRATION_WARNING_2","type":"int","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"cross_origin_url":{"name":"cross_origin_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"crt_file":{"name":"crt_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"debug":{"name":"debug","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable/disable debug options","long_desc":"","tags":[],"see_also":[]},"jwt_token_ttl":{"name":"jwt_token_ttl","type":"int","level":"advanced","flags":0,"default_value":"28800","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"motd":{"name":"motd","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"The message of the day","long_desc":"","tags":[],"see_also":[]},"redirect_resolve_ip_addr":{"name":"redirect_resolve_ip_addr","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":0,"default_value":"8080","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl_server_port":{"name":"ssl_server_port","type":"int","level":"advanced","flags":0,"default_value":"8443","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":0,"default_value":"redirect","min":"","max":"","enum_allowed":["error","redirect"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":0,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url_prefix":{"name":"url_prefix","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"devicehealth","can_run":true,"error_string":"","module_options":{"enable_monitoring":{"name":"enable_monitoring","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"monitor device health metrics","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mark_out_threshold":{"name":"mark_out_threshold","type":"secs","level":"advanced","flags":1,"default_value":"2419200","min":"","max":"","enum_allowed":[],"desc":"automatically mark OSD if it may fail before this long","long_desc":"","tags":[],"see_also":[]},"pool_name":{"name":"pool_name","type":"str","level":"advanced","flags":1,"default_value":"device_health_metrics","min":"","max":"","enum_allowed":[],"desc":"name of pool in which to store device health metrics","long_desc":"","tags":[],"see_also":[]},"retention_period":{"name":"retention_period","type":"secs","level":"advanced","flags":1,"default_value":"15552000","min":"","max":"","enum_allowed":[],"desc":"how long to retain device health metrics","long_desc":"","tags":[],"see_also":[]},"scrape_frequency":{"name":"scrape_frequency","type":"secs","level":"advanced","flags":1,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"how frequently to scrape device health metrics","long_desc":"","tags":[],"see_also":[]},"self_heal":{"name":"self_heal","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"preemptively heal cluster around devices that may fail","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and check device health","long_desc":"","tags":[],"see_also":[]},"warn_threshold":{"name":"warn_threshold","type":"secs","level":"advanced","flags":1,"default_value":"7257600","min":"","max":"","enum_allowed":[],"desc":"raise health warning if OSD may fail before this long","long_desc":"","tags":[],"see_also":[]}}},{"name":"diskprediction_local","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predict_interval":{"name":"predict_interval","type":"str","level":"advanced","flags":0,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predictor_model":{"name":"predictor_model","type":"str","level":"advanced","flags":0,"default_value":"prophetstor","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"str","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"influx","can_run":false,"error_string":"influxdb python module not found","module_options":{"batch_size":{"name":"batch_size","type":"int","level":"advanced","flags":0,"default_value":"5000","min":"","max":"","enum_allowed":[],"desc":"How big batches of data points should be when sending to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"database":{"name":"database","type":"str","level":"advanced","flags":0,"default_value":"ceph","min":"","max":"","enum_allowed":[],"desc":"InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.","long_desc":"","tags":[],"see_also":[]},"hostname":{"name":"hostname","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server hostname","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"30","min":"5","max":"","enum_allowed":[],"desc":"Time between reports to InfluxDB. Default 30 seconds.","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"password":{"name":"password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"password of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"port":{"name":"port","type":"int","level":"advanced","flags":0,"default_value":"8086","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server port","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"str","level":"advanced","flags":0,"default_value":"false","min":"","max":"","enum_allowed":[],"desc":"Use https connection for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]},"threads":{"name":"threads","type":"int","level":"advanced","flags":0,"default_value":"5","min":"1","max":"32","enum_allowed":[],"desc":"How many worker threads should be spawned for sending data to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"username":{"name":"username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"username of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"verify_ssl":{"name":"verify_ssl","type":"str","level":"advanced","flags":0,"default_value":"true","min":"","max":"","enum_allowed":[],"desc":"Verify https cert for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]}}},{"name":"insights","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"iostat","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"localpool","can_run":true,"error_string":"","module_options":{"failure_domain":{"name":"failure_domain","type":"str","level":"advanced","flags":1,"default_value":"host","min":"","max":"","enum_allowed":[],"desc":"failure domain for any created local pool","long_desc":"what failure domain we should separate data replicas across.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_size":{"name":"min_size","type":"int","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"default min_size for any created local pool","long_desc":"value to set min_size to (unchanged from Ceph's default if this option is not set)","tags":[],"see_also":[]},"num_rep":{"name":"num_rep","type":"int","level":"advanced","flags":1,"default_value":"3","min":"","max":"","enum_allowed":[],"desc":"default replica count for any created local pool","long_desc":"","tags":[],"see_also":[]},"pg_num":{"name":"pg_num","type":"int","level":"advanced","flags":1,"default_value":"128","min":"","max":"","enum_allowed":[],"desc":"default pg_num for any created local pool","long_desc":"","tags":[],"see_also":[]},"prefix":{"name":"prefix","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"name prefix for any created local pool","long_desc":"","tags":[],"see_also":[]},"subtree":{"name":"subtree","type":"str","level":"advanced","flags":1,"default_value":"rack","min":"","max":"","enum_allowed":[],"desc":"CRUSH level for which to create a local pool","long_desc":"which CRUSH subtree type the module should create a pool for.","tags":[],"see_also":[]}}},{"name":"mirroring","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"nfs","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"orchestrator","can_run":true,"error_string":"","module_options":{"fail_fs":{"name":"fail_fs","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Fail filesystem for rapid multi-rank mds upgrade","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"orchestrator":{"name":"orchestrator","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["cephadm","rook","test_orchestrator"],"desc":"Orchestrator backend","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_perf_query","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"pg_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"threshold":{"name":"threshold","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"1.0","max":"","enum_allowed":[],"desc":"scaling threshold","long_desc":"The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0","tags":[],"see_also":[]}}},{"name":"progress","can_run":true,"error_string":"","module_options":{"allow_pg_recovery_event":{"name":"allow_pg_recovery_event","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow the module to show pg recovery progress","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_completed_events":{"name":"max_completed_events","type":"int","level":"advanced","flags":1,"default_value":"50","min":"","max":"","enum_allowed":[],"desc":"number of past completed events to remember","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"how long the module is going to sleep","long_desc":"","tags":[],"see_also":[]}}},{"name":"prometheus","can_run":true,"error_string":"","module_options":{"cache":{"name":"cache","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"exclude_perf_counters":{"name":"exclude_perf_counters","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Do not include perf-counters in the metrics output","long_desc":"Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools":{"name":"rbd_stats_pools","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools_refresh_interval":{"name":"rbd_stats_pools_refresh_interval","type":"int","level":"advanced","flags":0,"default_value":"300","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"scrape_interval":{"name":"scrape_interval","type":"float","level":"advanced","flags":0,"default_value":"15.0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"the IPv4 or IPv6 address on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":1,"default_value":"9283","min":"","max":"","enum_allowed":[],"desc":"the port on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"stale_cache_strategy":{"name":"stale_cache_strategy","type":"str","level":"advanced","flags":0,"default_value":"log","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":1,"default_value":"default","min":"","max":"","enum_allowed":["default","error"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":1,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rbd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_snap_create":{"name":"max_concurrent_snap_create","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mirror_snapshot_schedule":{"name":"mirror_snapshot_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"trash_purge_schedule":{"name":"trash_purge_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"restful","can_run":true,"error_string":"","module_options":{"enable_auth":{"name":"enable_auth","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"selftest","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption1":{"name":"roption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption2":{"name":"roption2","type":"str","level":"advanced","flags":0,"default_value":"xyz","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption1":{"name":"rwoption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption2":{"name":"rwoption2","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption3":{"name":"rwoption3","type":"float","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption4":{"name":"rwoption4","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption5":{"name":"rwoption5","type":"bool","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption6":{"name":"rwoption6","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption7":{"name":"rwoption7","type":"int","level":"advanced","flags":0,"default_value":"","min":"1","max":"42","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testkey":{"name":"testkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testlkey":{"name":"testlkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testnewline":{"name":"testnewline","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"snap_schedule","can_run":true,"error_string":"","module_options":{"allow_m_granularity":{"name":"allow_m_granularity","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow minute scheduled snapshots","long_desc":"","tags":[],"see_also":[]},"dump_on_update":{"name":"dump_on_update","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"dump database to debug log on update","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"stats","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"status","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telegraf","can_run":true,"error_string":"","module_options":{"address":{"name":"address","type":"str","level":"advanced","flags":0,"default_value":"unixgram:///tmp/telegraf.sock","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"15","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telemetry","can_run":true,"error_string":"","module_options":{"channel_basic":{"name":"channel_basic","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share basic cluster information (size, version)","long_desc":"","tags":[],"see_also":[]},"channel_crash":{"name":"channel_crash","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share metadata about Ceph daemon crashes (version, stack straces, etc)","long_desc":"","tags":[],"see_also":[]},"channel_device":{"name":"channel_device","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)","long_desc":"","tags":[],"see_also":[]},"channel_ident":{"name":"channel_ident","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share a user-provided description and/or contact email for the cluster","long_desc":"","tags":[],"see_also":[]},"channel_perf":{"name":"channel_perf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share various performance metrics of a cluster","long_desc":"","tags":[],"see_also":[]},"contact":{"name":"contact","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"description":{"name":"description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"device_url":{"name":"device_url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/device","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"int","level":"advanced","flags":0,"default_value":"24","min":"8","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"last_opt_revision":{"name":"last_opt_revision","type":"int","level":"advanced","flags":0,"default_value":"1","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard":{"name":"leaderboard","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard_description":{"name":"leaderboard_description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"organization":{"name":"organization","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"proxy":{"name":"proxy","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url":{"name":"url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/report","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"test_orchestrator","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"volumes","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_clones":{"name":"max_concurrent_clones","type":"int","level":"advanced","flags":0,"default_value":"4","min":"","max":"","enum_allowed":[],"desc":"Number of asynchronous cloner threads","long_desc":"","tags":[],"see_also":[]},"periodic_async_work":{"name":"periodic_async_work","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Periodically check for async work","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_delay":{"name":"snapshot_clone_delay","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"Delay clone begin operation by snapshot_clone_delay seconds","long_desc":"","tags":[],"see_also":[]}}},{"name":"zabbix","can_run":true,"error_string":"","module_options":{"discovery_interval":{"name":"discovery_interval","type":"uint","level":"advanced","flags":0,"default_value":"100","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"identifier":{"name":"identifier","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_host":{"name":"zabbix_host","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_port":{"name":"zabbix_port","type":"int","level":"advanced","flags":0,"default_value":"10051","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_sender":{"name":"zabbix_sender","type":"str","level":"advanced","flags":0,"default_value":"/usr/bin/zabbix_sender","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}}]}],"modules":["iostat","nfs","restful"],"available_modules":[{"name":"alerts","can_run":true,"error_string":"","module_options":{"interval":{"name":"interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"How frequently to reexamine health status","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"smtp_destination":{"name":"smtp_destination","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Email address to send alerts to","long_desc":"","tags":[],"see_also":[]},"smtp_from_name":{"name":"smtp_from_name","type":"str","level":"advanced","flags":1,"default_value":"Ceph","min":"","max":"","enum_allowed":[],"desc":"Email From: name","long_desc":"","tags":[],"see_also":[]},"smtp_host":{"name":"smtp_host","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_password":{"name":"smtp_password","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Password to authenticate with","long_desc":"","tags":[],"see_also":[]},"smtp_port":{"name":"smtp_port","type":"int","level":"advanced","flags":1,"default_value":"465","min":"","max":"","enum_allowed":[],"desc":"SMTP port","long_desc":"","tags":[],"see_also":[]},"smtp_sender":{"name":"smtp_sender","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP envelope sender","long_desc":"","tags":[],"see_also":[]},"smtp_ssl":{"name":"smtp_ssl","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Use SSL to connect to SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_user":{"name":"smtp_user","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"User to authenticate as","long_desc":"","tags":[],"see_also":[]}}},{"name":"balancer","can_run":true,"error_string":"","module_options":{"active":{"name":"active","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"automatically balance PGs across cluster","long_desc":"","tags":[],"see_also":[]},"begin_time":{"name":"begin_time","type":"str","level":"advanced","flags":1,"default_value":"0000","min":"","max":"","enum_allowed":[],"desc":"beginning time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"begin_weekday":{"name":"begin_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to this day of the week or later","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"crush_compat_max_iterations":{"name":"crush_compat_max_iterations","type":"uint","level":"advanced","flags":1,"default_value":"25","min":"1","max":"250","enum_allowed":[],"desc":"maximum number of iterations to attempt optimization","long_desc":"","tags":[],"see_also":[]},"crush_compat_metrics":{"name":"crush_compat_metrics","type":"str","level":"advanced","flags":1,"default_value":"pgs,objects,bytes","min":"","max":"","enum_allowed":[],"desc":"metrics with which to calculate OSD utilization","long_desc":"Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.","tags":[],"see_also":[]},"crush_compat_step":{"name":"crush_compat_step","type":"float","level":"advanced","flags":1,"default_value":"0.5","min":"0.001","max":"0.999","enum_allowed":[],"desc":"aggressiveness of optimization","long_desc":".99 is very aggressive, .01 is less aggressive","tags":[],"see_also":[]},"end_time":{"name":"end_time","type":"str","level":"advanced","flags":1,"default_value":"2359","min":"","max":"","enum_allowed":[],"desc":"ending time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"end_weekday":{"name":"end_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to days of the week earlier than this","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_score":{"name":"min_score","type":"float","level":"advanced","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"minimum score, below which no optimization is attempted","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":1,"default_value":"upmap","min":"","max":"","enum_allowed":["crush-compat","none","upmap"],"desc":"Balancer mode","long_desc":"","tags":[],"see_also":[]},"pool_ids":{"name":"pool_ids","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"pools which the automatic balancing will be limited to","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and attempt optimization","long_desc":"","tags":[],"see_also":[]},"upmap_max_deviation":{"name":"upmap_max_deviation","type":"int","level":"advanced","flags":1,"default_value":"5","min":"1","max":"","enum_allowed":[],"desc":"deviation below which no optimization is attempted","long_desc":"If the number of PGs are within this count then no optimization is attempted","tags":[],"see_also":[]},"upmap_max_optimizations":{"name":"upmap_max_optimizations","type":"uint","level":"advanced","flags":1,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"maximum upmap optimizations to make per attempt","long_desc":"","tags":[],"see_also":[]}}},{"name":"cephadm","can_run":true,"error_string":"","module_options":{"agent_down_multiplier":{"name":"agent_down_multiplier","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"","max":"","enum_allowed":[],"desc":"Multiplied by agent refresh rate to calculate how long agent must not report before being marked down","long_desc":"","tags":[],"see_also":[]},"agent_refresh_rate":{"name":"agent_refresh_rate","type":"secs","level":"advanced","flags":0,"default_value":"20","min":"","max":"","enum_allowed":[],"desc":"How often agent on each host will try to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"agent_starting_port":{"name":"agent_starting_port","type":"int","level":"advanced","flags":0,"default_value":"4721","min":"","max":"","enum_allowed":[],"desc":"First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)","long_desc":"","tags":[],"see_also":[]},"allow_ptrace":{"name":"allow_ptrace","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow SYS_PTRACE capability on ceph containers","long_desc":"The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.","tags":[],"see_also":[]},"autotune_interval":{"name":"autotune_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to autotune daemon memory","long_desc":"","tags":[],"see_also":[]},"autotune_memory_target_ratio":{"name":"autotune_memory_target_ratio","type":"float","level":"advanced","flags":0,"default_value":"0.7","min":"","max":"","enum_allowed":[],"desc":"ratio of total system memory to divide amongst autotuned daemons","long_desc":"","tags":[],"see_also":[]},"cephadm_log_destination":{"name":"cephadm_log_destination","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":["file","file,syslog","syslog"],"desc":"Destination for cephadm command's persistent logging","long_desc":"","tags":[],"see_also":[]},"cgroups_split":{"name":"cgroups_split","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Pass --cgroups=split when cephadm creates containers (currently podman only)","long_desc":"","tags":[],"see_also":[]},"config_checks_enabled":{"name":"config_checks_enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable or disable the cephadm configuration analysis","long_desc":"","tags":[],"see_also":[]},"config_dashboard":{"name":"config_dashboard","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"manage configs like API endpoints in Dashboard.","long_desc":"","tags":[],"see_also":[]},"container_image_alertmanager":{"name":"container_image_alertmanager","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/alertmanager:v0.25.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_base":{"name":"container_image_base","type":"str","level":"advanced","flags":1,"default_value":"quay.io/ceph/ceph","min":"","max":"","enum_allowed":[],"desc":"Container image name, without the tag","long_desc":"","tags":[],"see_also":[]},"container_image_elasticsearch":{"name":"container_image_elasticsearch","type":"str","level":"advanced","flags":0,"default_value":"quay.io/omrizeneva/elasticsearch:6.8.23","min":"","max":"","enum_allowed":[],"desc":"elasticsearch container image","long_desc":"","tags":[],"see_also":[]},"container_image_grafana":{"name":"container_image_grafana","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/ceph-grafana:9.4.7","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_haproxy":{"name":"container_image_haproxy","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/haproxy:2.3","min":"","max":"","enum_allowed":[],"desc":"HAproxy container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_agent":{"name":"container_image_jaeger_agent","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-agent:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger agent container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_collector":{"name":"container_image_jaeger_collector","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-collector:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger collector container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_query":{"name":"container_image_jaeger_query","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-query:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger query container image","long_desc":"","tags":[],"see_also":[]},"container_image_keepalived":{"name":"container_image_keepalived","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/keepalived:2.2.4","min":"","max":"","enum_allowed":[],"desc":"Keepalived container image","long_desc":"","tags":[],"see_also":[]},"container_image_loki":{"name":"container_image_loki","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/loki:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Loki container image","long_desc":"","tags":[],"see_also":[]},"container_image_node_exporter":{"name":"container_image_node_exporter","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/node-exporter:v1.5.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_nvmeof":{"name":"container_image_nvmeof","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/nvmeof:latest","min":"","max":"","enum_allowed":[],"desc":"Nvme-of container image","long_desc":"","tags":[],"see_also":[]},"container_image_prometheus":{"name":"container_image_prometheus","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/prometheus:v2.43.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_promtail":{"name":"container_image_promtail","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/promtail:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Promtail container image","long_desc":"","tags":[],"see_also":[]},"container_image_snmp_gateway":{"name":"container_image_snmp_gateway","type":"str","level":"advanced","flags":0,"default_value":"docker.io/maxwo/snmp-notifier:v1.2.1","min":"","max":"","enum_allowed":[],"desc":"SNMP Gateway container image","long_desc":"","tags":[],"see_also":[]},"container_init":{"name":"container_init","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Run podman/docker with `--init`","long_desc":"","tags":[],"see_also":[]},"daemon_cache_timeout":{"name":"daemon_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"seconds to cache service (daemon) inventory","long_desc":"","tags":[],"see_also":[]},"default_cephadm_command_timeout":{"name":"default_cephadm_command_timeout","type":"secs","level":"advanced","flags":0,"default_value":"900","min":"","max":"","enum_allowed":[],"desc":"Default timeout applied to cephadm commands run directly on the host (in seconds)","long_desc":"","tags":[],"see_also":[]},"default_registry":{"name":"default_registry","type":"str","level":"advanced","flags":0,"default_value":"docker.io","min":"","max":"","enum_allowed":[],"desc":"Search-registry to which we should normalize unqualified image names. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"device_cache_timeout":{"name":"device_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"1800","min":"","max":"","enum_allowed":[],"desc":"seconds to cache device inventory","long_desc":"","tags":[],"see_also":[]},"device_enhanced_scan":{"name":"device_enhanced_scan","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use libstoragemgmt during device scans","long_desc":"","tags":[],"see_also":[]},"facts_cache_timeout":{"name":"facts_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"seconds to cache host facts data","long_desc":"","tags":[],"see_also":[]},"host_check_interval":{"name":"host_check_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to perform a host check","long_desc":"","tags":[],"see_also":[]},"inventory_list_all":{"name":"inventory_list_all","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_refresh_metadata":{"name":"log_refresh_metadata","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"log to the \"cephadm\" cluster log channel\"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf":{"name":"manage_etc_ceph_ceph_conf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Manage and own /etc/ceph/ceph.conf on the hosts.","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf_hosts":{"name":"manage_etc_ceph_ceph_conf_hosts","type":"str","level":"advanced","flags":0,"default_value":"*","min":"","max":"","enum_allowed":[],"desc":"PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf","long_desc":"","tags":[],"see_also":[]},"max_count_per_host":{"name":"max_count_per_host","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of daemons per service per host","long_desc":"","tags":[],"see_also":[]},"max_osd_draining_count":{"name":"max_osd_draining_count","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of osds that will be drained simultaneously when osds are removed","long_desc":"","tags":[],"see_also":[]},"migration_current":{"name":"migration_current","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"internal - do not modify","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":0,"default_value":"root","min":"","max":"","enum_allowed":["cephadm-package","root"],"desc":"mode for remote execution of cephadm","long_desc":"","tags":[],"see_also":[]},"prometheus_alerts_path":{"name":"prometheus_alerts_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/prometheus/ceph/ceph_default_alerts.yml","min":"","max":"","enum_allowed":[],"desc":"location of alerts to include in prometheus deployments","long_desc":"","tags":[],"see_also":[]},"registry_insecure":{"name":"registry_insecure","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Registry is to be considered insecure (no TLS available). Only for development purposes.","long_desc":"","tags":[],"see_also":[]},"registry_password":{"name":"registry_password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository password. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"registry_url":{"name":"registry_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Registry url for login purposes. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"registry_username":{"name":"registry_username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository username. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"secure_monitoring_stack":{"name":"secure_monitoring_stack","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable TLS security for all the monitoring stack daemons","long_desc":"","tags":[],"see_also":[]},"service_discovery_port":{"name":"service_discovery_port","type":"int","level":"advanced","flags":0,"default_value":"8765","min":"","max":"","enum_allowed":[],"desc":"cephadm service discovery port","long_desc":"","tags":[],"see_also":[]},"ssh_config_file":{"name":"ssh_config_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"customized SSH config file to connect to managed hosts","long_desc":"","tags":[],"see_also":[]},"use_agent":{"name":"use_agent","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use cephadm agent on each host to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"use_repo_digest":{"name":"use_repo_digest","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Automatically convert image tags to image digest. Make sure all daemons use the same image","long_desc":"","tags":[],"see_also":[]},"warn_on_failed_host_check":{"name":"warn_on_failed_host_check","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if the host check fails","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_daemons":{"name":"warn_on_stray_daemons","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected that are not managed by cephadm","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_hosts":{"name":"warn_on_stray_hosts","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected on a host that is not managed by cephadm","long_desc":"","tags":[],"see_also":[]}}},{"name":"crash","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"retain_interval":{"name":"retain_interval","type":"secs","level":"advanced","flags":1,"default_value":"31536000","min":"","max":"","enum_allowed":[],"desc":"how long to retain crashes before pruning them","long_desc":"","tags":[],"see_also":[]},"warn_recent_interval":{"name":"warn_recent_interval","type":"secs","level":"advanced","flags":1,"default_value":"1209600","min":"","max":"","enum_allowed":[],"desc":"time interval in which to warn about recent crashes","long_desc":"","tags":[],"see_also":[]}}},{"name":"dashboard","can_run":true,"error_string":"","module_options":{"ACCOUNT_LOCKOUT_ATTEMPTS":{"name":"ACCOUNT_LOCKOUT_ATTEMPTS","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_HOST":{"name":"ALERTMANAGER_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_SSL_VERIFY":{"name":"ALERTMANAGER_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_ENABLED":{"name":"AUDIT_API_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_LOG_PAYLOAD":{"name":"AUDIT_API_LOG_PAYLOAD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ENABLE_BROWSABLE_API":{"name":"ENABLE_BROWSABLE_API","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_CEPHFS":{"name":"FEATURE_TOGGLE_CEPHFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_DASHBOARD":{"name":"FEATURE_TOGGLE_DASHBOARD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_ISCSI":{"name":"FEATURE_TOGGLE_ISCSI","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_MIRRORING":{"name":"FEATURE_TOGGLE_MIRRORING","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_NFS":{"name":"FEATURE_TOGGLE_NFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RBD":{"name":"FEATURE_TOGGLE_RBD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RGW":{"name":"FEATURE_TOGGLE_RGW","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE":{"name":"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_PASSWORD":{"name":"GRAFANA_API_PASSWORD","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_SSL_VERIFY":{"name":"GRAFANA_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_URL":{"name":"GRAFANA_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_USERNAME":{"name":"GRAFANA_API_USERNAME","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_FRONTEND_API_URL":{"name":"GRAFANA_FRONTEND_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_UPDATE_DASHBOARDS":{"name":"GRAFANA_UPDATE_DASHBOARDS","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISCSI_API_SSL_VERIFICATION":{"name":"ISCSI_API_SSL_VERIFICATION","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISSUE_TRACKER_API_KEY":{"name":"ISSUE_TRACKER_API_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_HOST":{"name":"PROMETHEUS_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_SSL_VERIFY":{"name":"PROMETHEUS_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_COMPLEXITY_ENABLED":{"name":"PWD_POLICY_CHECK_COMPLEXITY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED":{"name":"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_LENGTH_ENABLED":{"name":"PWD_POLICY_CHECK_LENGTH_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_OLDPWD_ENABLED":{"name":"PWD_POLICY_CHECK_OLDPWD_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_USERNAME_ENABLED":{"name":"PWD_POLICY_CHECK_USERNAME_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_ENABLED":{"name":"PWD_POLICY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_EXCLUSION_LIST":{"name":"PWD_POLICY_EXCLUSION_LIST","type":"str","level":"advanced","flags":0,"default_value":"osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_COMPLEXITY":{"name":"PWD_POLICY_MIN_COMPLEXITY","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_LENGTH":{"name":"PWD_POLICY_MIN_LENGTH","type":"int","level":"advanced","flags":0,"default_value":"8","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"REST_REQUESTS_TIMEOUT":{"name":"REST_REQUESTS_TIMEOUT","type":"int","level":"advanced","flags":0,"default_value":"45","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ACCESS_KEY":{"name":"RGW_API_ACCESS_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ADMIN_RESOURCE":{"name":"RGW_API_ADMIN_RESOURCE","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SECRET_KEY":{"name":"RGW_API_SECRET_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SSL_VERIFY":{"name":"RGW_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"UNSAFE_TLS_v1_2":{"name":"UNSAFE_TLS_v1_2","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_SPAN":{"name":"USER_PWD_EXPIRATION_SPAN","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_1":{"name":"USER_PWD_EXPIRATION_WARNING_1","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_2":{"name":"USER_PWD_EXPIRATION_WARNING_2","type":"int","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"cross_origin_url":{"name":"cross_origin_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"crt_file":{"name":"crt_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"debug":{"name":"debug","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable/disable debug options","long_desc":"","tags":[],"see_also":[]},"jwt_token_ttl":{"name":"jwt_token_ttl","type":"int","level":"advanced","flags":0,"default_value":"28800","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"motd":{"name":"motd","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"The message of the day","long_desc":"","tags":[],"see_also":[]},"redirect_resolve_ip_addr":{"name":"redirect_resolve_ip_addr","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":0,"default_value":"8080","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl_server_port":{"name":"ssl_server_port","type":"int","level":"advanced","flags":0,"default_value":"8443","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":0,"default_value":"redirect","min":"","max":"","enum_allowed":["error","redirect"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":0,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url_prefix":{"name":"url_prefix","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"devicehealth","can_run":true,"error_string":"","module_options":{"enable_monitoring":{"name":"enable_monitoring","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"monitor device health metrics","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mark_out_threshold":{"name":"mark_out_threshold","type":"secs","level":"advanced","flags":1,"default_value":"2419200","min":"","max":"","enum_allowed":[],"desc":"automatically mark OSD if it may fail before this long","long_desc":"","tags":[],"see_also":[]},"pool_name":{"name":"pool_name","type":"str","level":"advanced","flags":1,"default_value":"device_health_metrics","min":"","max":"","enum_allowed":[],"desc":"name of pool in which to store device health metrics","long_desc":"","tags":[],"see_also":[]},"retention_period":{"name":"retention_period","type":"secs","level":"advanced","flags":1,"default_value":"15552000","min":"","max":"","enum_allowed":[],"desc":"how long to retain device health metrics","long_desc":"","tags":[],"see_also":[]},"scrape_frequency":{"name":"scrape_frequency","type":"secs","level":"advanced","flags":1,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"how frequently to scrape device health metrics","long_desc":"","tags":[],"see_also":[]},"self_heal":{"name":"self_heal","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"preemptively heal cluster around devices that may fail","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and check device health","long_desc":"","tags":[],"see_also":[]},"warn_threshold":{"name":"warn_threshold","type":"secs","level":"advanced","flags":1,"default_value":"7257600","min":"","max":"","enum_allowed":[],"desc":"raise health warning if OSD may fail before this long","long_desc":"","tags":[],"see_also":[]}}},{"name":"diskprediction_local","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predict_interval":{"name":"predict_interval","type":"str","level":"advanced","flags":0,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predictor_model":{"name":"predictor_model","type":"str","level":"advanced","flags":0,"default_value":"prophetstor","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"str","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"influx","can_run":false,"error_string":"influxdb python module not found","module_options":{"batch_size":{"name":"batch_size","type":"int","level":"advanced","flags":0,"default_value":"5000","min":"","max":"","enum_allowed":[],"desc":"How big batches of data points should be when sending to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"database":{"name":"database","type":"str","level":"advanced","flags":0,"default_value":"ceph","min":"","max":"","enum_allowed":[],"desc":"InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.","long_desc":"","tags":[],"see_also":[]},"hostname":{"name":"hostname","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server hostname","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"30","min":"5","max":"","enum_allowed":[],"desc":"Time between reports to InfluxDB. Default 30 seconds.","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"password":{"name":"password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"password of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"port":{"name":"port","type":"int","level":"advanced","flags":0,"default_value":"8086","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server port","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"str","level":"advanced","flags":0,"default_value":"false","min":"","max":"","enum_allowed":[],"desc":"Use https connection for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]},"threads":{"name":"threads","type":"int","level":"advanced","flags":0,"default_value":"5","min":"1","max":"32","enum_allowed":[],"desc":"How many worker threads should be spawned for sending data to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"username":{"name":"username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"username of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"verify_ssl":{"name":"verify_ssl","type":"str","level":"advanced","flags":0,"default_value":"true","min":"","max":"","enum_allowed":[],"desc":"Verify https cert for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]}}},{"name":"insights","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"iostat","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"localpool","can_run":true,"error_string":"","module_options":{"failure_domain":{"name":"failure_domain","type":"str","level":"advanced","flags":1,"default_value":"host","min":"","max":"","enum_allowed":[],"desc":"failure domain for any created local pool","long_desc":"what failure domain we should separate data replicas across.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_size":{"name":"min_size","type":"int","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"default min_size for any created local pool","long_desc":"value to set min_size to (unchanged from Ceph's default if this option is not set)","tags":[],"see_also":[]},"num_rep":{"name":"num_rep","type":"int","level":"advanced","flags":1,"default_value":"3","min":"","max":"","enum_allowed":[],"desc":"default replica count for any created local pool","long_desc":"","tags":[],"see_also":[]},"pg_num":{"name":"pg_num","type":"int","level":"advanced","flags":1,"default_value":"128","min":"","max":"","enum_allowed":[],"desc":"default pg_num for any created local pool","long_desc":"","tags":[],"see_also":[]},"prefix":{"name":"prefix","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"name prefix for any created local pool","long_desc":"","tags":[],"see_also":[]},"subtree":{"name":"subtree","type":"str","level":"advanced","flags":1,"default_value":"rack","min":"","max":"","enum_allowed":[],"desc":"CRUSH level for which to create a local pool","long_desc":"which CRUSH subtree type the module should create a pool for.","tags":[],"see_also":[]}}},{"name":"mirroring","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"nfs","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"orchestrator","can_run":true,"error_string":"","module_options":{"fail_fs":{"name":"fail_fs","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Fail filesystem for rapid multi-rank mds upgrade","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"orchestrator":{"name":"orchestrator","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["cephadm","rook","test_orchestrator"],"desc":"Orchestrator backend","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_perf_query","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"pg_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"threshold":{"name":"threshold","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"1.0","max":"","enum_allowed":[],"desc":"scaling threshold","long_desc":"The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0","tags":[],"see_also":[]}}},{"name":"progress","can_run":true,"error_string":"","module_options":{"allow_pg_recovery_event":{"name":"allow_pg_recovery_event","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow the module to show pg recovery progress","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_completed_events":{"name":"max_completed_events","type":"int","level":"advanced","flags":1,"default_value":"50","min":"","max":"","enum_allowed":[],"desc":"number of past completed events to remember","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"how long the module is going to sleep","long_desc":"","tags":[],"see_also":[]}}},{"name":"prometheus","can_run":true,"error_string":"","module_options":{"cache":{"name":"cache","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"exclude_perf_counters":{"name":"exclude_perf_counters","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Do not include perf-counters in the metrics output","long_desc":"Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools":{"name":"rbd_stats_pools","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools_refresh_interval":{"name":"rbd_stats_pools_refresh_interval","type":"int","level":"advanced","flags":0,"default_value":"300","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"scrape_interval":{"name":"scrape_interval","type":"float","level":"advanced","flags":0,"default_value":"15.0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"the IPv4 or IPv6 address on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":1,"default_value":"9283","min":"","max":"","enum_allowed":[],"desc":"the port on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"stale_cache_strategy":{"name":"stale_cache_strategy","type":"str","level":"advanced","flags":0,"default_value":"log","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":1,"default_value":"default","min":"","max":"","enum_allowed":["default","error"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":1,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rbd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_snap_create":{"name":"max_concurrent_snap_create","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mirror_snapshot_schedule":{"name":"mirror_snapshot_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"trash_purge_schedule":{"name":"trash_purge_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"restful","can_run":true,"error_string":"","module_options":{"enable_auth":{"name":"enable_auth","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"selftest","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption1":{"name":"roption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption2":{"name":"roption2","type":"str","level":"advanced","flags":0,"default_value":"xyz","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption1":{"name":"rwoption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption2":{"name":"rwoption2","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption3":{"name":"rwoption3","type":"float","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption4":{"name":"rwoption4","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption5":{"name":"rwoption5","type":"bool","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption6":{"name":"rwoption6","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption7":{"name":"rwoption7","type":"int","level":"advanced","flags":0,"default_value":"","min":"1","max":"42","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testkey":{"name":"testkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testlkey":{"name":"testlkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testnewline":{"name":"testnewline","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"snap_schedule","can_run":true,"error_string":"","module_options":{"allow_m_granularity":{"name":"allow_m_granularity","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow minute scheduled snapshots","long_desc":"","tags":[],"see_also":[]},"dump_on_update":{"name":"dump_on_update","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"dump database to debug log on update","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"stats","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"status","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telegraf","can_run":true,"error_string":"","module_options":{"address":{"name":"address","type":"str","level":"advanced","flags":0,"default_value":"unixgram:///tmp/telegraf.sock","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"15","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telemetry","can_run":true,"error_string":"","module_options":{"channel_basic":{"name":"channel_basic","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share basic cluster information (size, version)","long_desc":"","tags":[],"see_also":[]},"channel_crash":{"name":"channel_crash","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share metadata about Ceph daemon crashes (version, stack straces, etc)","long_desc":"","tags":[],"see_also":[]},"channel_device":{"name":"channel_device","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)","long_desc":"","tags":[],"see_also":[]},"channel_ident":{"name":"channel_ident","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share a user-provided description and/or contact email for the cluster","long_desc":"","tags":[],"see_also":[]},"channel_perf":{"name":"channel_perf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share various performance metrics of a cluster","long_desc":"","tags":[],"see_also":[]},"contact":{"name":"contact","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"description":{"name":"description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"device_url":{"name":"device_url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/device","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"int","level":"advanced","flags":0,"default_value":"24","min":"8","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"last_opt_revision":{"name":"last_opt_revision","type":"int","level":"advanced","flags":0,"default_value":"1","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard":{"name":"leaderboard","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard_description":{"name":"leaderboard_description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"organization":{"name":"organization","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"proxy":{"name":"proxy","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url":{"name":"url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/report","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"test_orchestrator","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"volumes","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_clones":{"name":"max_concurrent_clones","type":"int","level":"advanced","flags":0,"default_value":"4","min":"","max":"","enum_allowed":[],"desc":"Number of asynchronous cloner threads","long_desc":"","tags":[],"see_also":[]},"periodic_async_work":{"name":"periodic_async_work","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Periodically check for async work","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_delay":{"name":"snapshot_clone_delay","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"Delay clone begin operation by snapshot_clone_delay seconds","long_desc":"","tags":[],"see_also":[]}}},{"name":"zabbix","can_run":true,"error_string":"","module_options":{"discovery_interval":{"name":"discovery_interval","type":"uint","level":"advanced","flags":0,"default_value":"100","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"identifier":{"name":"identifier","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_host":{"name":"zabbix_host","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_port":{"name":"zabbix_port","type":"int","level":"advanced","flags":0,"default_value":"10051","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_sender":{"name":"zabbix_sender","type":"str","level":"advanced","flags":0,"default_value":"/usr/bin/zabbix_sender","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}}],"services":{},"always_on_modules":{"octopus":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"pacific":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"quincy":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"reef":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"]},"last_failure_osd_epoch":0,"active_clients":[{"name":"devicehealth","addrvec":[{"type":"v2","addr":"172.21.15.143:0","nonce":3516040872}]},{"name":"libcephsqlite","addrvec":[{"type":"v2","addr":"172.21.15.143:0","nonce":1844383386}]},{"name":"rbd_support","addrvec":[{"type":"v2","addr":"172.21.15.143:0","nonce":3813484262}]},{"name":"volumes","addrvec":[{"type":"v2","addr":"172.21.15.143:0","nonce":2591285564}]}]} 2023-11-26T07:17:59.364 INFO:tasks.ceph.ceph_manager.ceph:mgr available! 2023-11-26T07:17:59.364 INFO:tasks.ceph.ceph_manager.ceph:waiting for all up 2023-11-26T07:17:59.364 DEBUG:teuthology.orchestra.run.smithi143:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-11-26T07:17:59.501 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:17:59.498+0000 7fd253091640 1 Processor -- start 2023-11-26T07:17:59.501 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:17:59.498+0000 7fd253091640 1 -- start start 2023-11-26T07:17:59.503 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:17:59.498+0000 7fd253091640 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7fd24c1596d0 0x7fd24c071850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:17:59.503 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:17:59.498+0000 7fd253091640 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7fd24c0b2f80 0x7fd24c06cbe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:17:59.504 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:17:59.498+0000 7fd253091640 1 -- --> v1:172.21.15.188:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd24c06c750 con 0x7fd24c0ce680 2023-11-26T07:17:59.504 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:17:59.498+0000 7fd253091640 1 -- --> v2:172.21.15.143:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd24c0cbe00 con 0x7fd24c0b2f80 2023-11-26T07:17:59.504 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:17:59.498+0000 7fd253091640 1 -- --> v2:172.21.15.188:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd24c0b3550 con 0x7fd24c1596d0 2023-11-26T07:17:59.504 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:17:59.498+0000 7fd25162e640 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7fd24c0b2f80 0x7fd24c06cbe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:17:59.504 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:17:59.498+0000 7fd25162e640 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7fd24c0b2f80 0x7fd24c06cbe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:46314/0 (socket says 172.21.15.143:46314) 2023-11-26T07:17:59.504 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:17:59.498+0000 7fd25162e640 1 -- 172.21.15.143:0/2632592763 learned_addr learned my addr 172.21.15.143:0/2632592763 (peer_addr_for_me v2:172.21.15.143:0/0) 2023-11-26T07:17:59.504 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:17:59.498+0000 7fd250e2d640 1 --2- 172.21.15.143:0/2632592763 >> v2:172.21.15.188:3300/0 conn(0x7fd24c1596d0 0x7fd24c071850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:17:59.504 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:17:59.498+0000 7fd25162e640 1 -- 172.21.15.143:0/2632592763 >> v2:172.21.15.188:3300/0 conn(0x7fd24c1596d0 msgr2=0x7fd24c071850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-26T07:17:59.504 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:17:59.498+0000 7fd25162e640 1 --2- 172.21.15.143:0/2632592763 >> v2:172.21.15.188:3300/0 conn(0x7fd24c1596d0 0x7fd24c071850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:17:59.504 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:17:59.498+0000 7fd25162e640 1 -- 172.21.15.143:0/2632592763 >> v1:172.21.15.188:6789/0 conn(0x7fd24c0ce680 legacy=0x7fd24c0b2870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:17:59.504 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:17:59.498+0000 7fd25162e640 1 -- 172.21.15.143:0/2632592763 --> v2:172.21.15.143:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd24c071e80 con 0x7fd24c0b2f80 2023-11-26T07:17:59.506 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:17:59.502+0000 7fd25162e640 1 --2- 172.21.15.143:0/2632592763 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fd24c0b2f80 0x7fd24c06cbe0 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7fd24c06e4d0 tx=0x7fd24401c9d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a7391a7e8f131cc9 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:17:59.506 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:17:59.502+0000 7fd2437fe640 1 -- 172.21.15.143:0/2632592763 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd2440a55f0 con 0x7fd24c0b2f80 2023-11-26T07:17:59.507 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:17:59.502+0000 7fd2437fe640 1 -- 172.21.15.143:0/2632592763 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd2440a5750 con 0x7fd24c0b2f80 2023-11-26T07:17:59.507 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:17:59.502+0000 7fd2437fe640 1 -- 172.21.15.143:0/2632592763 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd2440a5ae0 con 0x7fd24c0b2f80 2023-11-26T07:17:59.507 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:17:59.502+0000 7fd253091640 1 -- 172.21.15.143:0/2632592763 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fd24c0b2f80 msgr2=0x7fd24c06cbe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:17:59.508 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:17:59.502+0000 7fd253091640 1 --2- 172.21.15.143:0/2632592763 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fd24c0b2f80 0x7fd24c06cbe0 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7fd24c06e4d0 tx=0x7fd24401c9d0 comp rx=0 tx=0).stop 2023-11-26T07:17:59.508 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:17:59.502+0000 7fd253091640 1 -- 172.21.15.143:0/2632592763 shutdown_connections 2023-11-26T07:17:59.508 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:17:59.502+0000 7fd253091640 1 --2- 172.21.15.143:0/2632592763 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fd24c0b2f80 0x7fd24c06cbe0 unknown :-1 s=CLOSED pgs=92 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:17:59.508 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:17:59.502+0000 7fd253091640 1 --2- 172.21.15.143:0/2632592763 >> v2:172.21.15.188:3300/0 conn(0x7fd24c1596d0 0x7fd24c071850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:17:59.508 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:17:59.502+0000 7fd253091640 1 -- 172.21.15.143:0/2632592763 >> 172.21.15.143:0/2632592763 conn(0x7fd24c06a890 msgr2=0x7fd24c06aca0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:17:59.508 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:17:59.502+0000 7fd253091640 1 -- 172.21.15.143:0/2632592763 shutdown_connections 2023-11-26T07:17:59.508 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:17:59.502+0000 7fd253091640 1 -- 172.21.15.143:0/2632592763 wait complete. 2023-11-26T07:17:59.509 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:17:59.506+0000 7fd253091640 1 Processor -- start 2023-11-26T07:17:59.509 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:17:59.506+0000 7fd253091640 1 -- start start 2023-11-26T07:17:59.511 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:17:59.506+0000 7fd253091640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fd24c0b2f80 0x7fd24c15c730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:17:59.511 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:17:59.506+0000 7fd253091640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fd24c0ce680 0x7fd24c0afb80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:17:59.511 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:17:59.506+0000 7fd253091640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fd24c1596d0 0x7fd24c13efa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:17:59.511 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:17:59.506+0000 7fd253091640 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd24c14eb80 con 0x7fd24c1596d0 2023-11-26T07:17:59.511 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:17:59.506+0000 7fd253091640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd24c15b230 con 0x7fd24c0b2f80 2023-11-26T07:17:59.511 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:17:59.506+0000 7fd253091640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd24c0af140 con 0x7fd24c0ce680 2023-11-26T07:17:59.511 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:17:59.506+0000 7fd25162e640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fd24c1596d0 0x7fd24c13efa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:17:59.511 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:17:59.506+0000 7fd25162e640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fd24c1596d0 0x7fd24c13efa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:46320/0 (socket says 172.21.15.143:46320) 2023-11-26T07:17:59.511 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:17:59.506+0000 7fd25162e640 1 -- 172.21.15.143:0/812127034 learned_addr learned my addr 172.21.15.143:0/812127034 (peer_addr_for_me v2:172.21.15.143:0/0) 2023-11-26T07:17:59.511 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:17:59.506+0000 7fd250e2d640 1 --2- 172.21.15.143:0/812127034 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fd24c0b2f80 0x7fd24c15c730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:17:59.512 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:17:59.506+0000 7fd243fff640 1 --2- 172.21.15.143:0/812127034 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fd24c0ce680 0x7fd24c0afb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:17:59.512 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:17:59.506+0000 7fd25162e640 1 -- 172.21.15.143:0/812127034 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fd24c0ce680 msgr2=0x7fd24c0afb80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:17:59.512 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:17:59.506+0000 7fd25162e640 1 --2- 172.21.15.143:0/812127034 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fd24c0ce680 0x7fd24c0afb80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:17:59.512 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:17:59.506+0000 7fd25162e640 1 -- 172.21.15.143:0/812127034 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fd24c0b2f80 msgr2=0x7fd24c15c730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:17:59.512 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:17:59.506+0000 7fd25162e640 1 --2- 172.21.15.143:0/812127034 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fd24c0b2f80 0x7fd24c15c730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:17:59.512 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:17:59.506+0000 7fd25162e640 1 -- 172.21.15.143:0/812127034 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd244009b30 con 0x7fd24c1596d0 2023-11-26T07:17:59.513 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:17:59.506+0000 7fd25162e640 1 --2- 172.21.15.143:0/812127034 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fd24c1596d0 0x7fd24c13efa0 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7fd2440af040 tx=0x7fd244019770 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:17:59.513 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:17:59.506+0000 7fd2417fa640 1 -- 172.21.15.143:0/812127034 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd2440bd070 con 0x7fd24c1596d0 2023-11-26T07:17:59.513 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:17:59.506+0000 7fd2417fa640 1 -- 172.21.15.143:0/812127034 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd244009510 con 0x7fd24c1596d0 2023-11-26T07:17:59.513 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:17:59.506+0000 7fd253091640 1 -- 172.21.15.143:0/812127034 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd24c14e5c0 con 0x7fd24c1596d0 2023-11-26T07:17:59.513 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:17:59.506+0000 7fd2417fa640 1 -- 172.21.15.143:0/812127034 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd2440b86b0 con 0x7fd24c1596d0 2023-11-26T07:17:59.514 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:17:59.506+0000 7fd253091640 1 -- 172.21.15.143:0/812127034 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd24c0af380 con 0x7fd24c1596d0 2023-11-26T07:17:59.514 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:17:59.506+0000 7fd253091640 1 -- 172.21.15.143:0/812127034 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd24c067680 con 0x7fd24c1596d0 2023-11-26T07:17:59.514 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:17:59.510+0000 7fd2417fa640 1 -- 172.21.15.143:0/812127034 <== mon.0 v2:172.21.15.143:3300/0 4 ==== mgrmap(e 5) v1 ==== 85852+0+0 (secure 0 0 0) 0x7fd2440b8810 con 0x7fd24c1596d0 2023-11-26T07:17:59.515 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:17:59.510+0000 7fd2417fa640 1 --2- 172.21.15.143:0/812127034 >> [v2:172.21.15.143:6832/2150557544,v1:172.21.15.143:6833/2150557544] conn(0x7fd2380664a0 0x7fd238068960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:17:59.515 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:17:59.510+0000 7fd2417fa640 1 -- 172.21.15.143:0/812127034 <== mon.0 v2:172.21.15.143:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fd24411ec70 con 0x7fd24c1596d0 2023-11-26T07:17:59.515 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:17:59.510+0000 7fd250e2d640 1 --2- 172.21.15.143:0/812127034 >> [v2:172.21.15.143:6832/2150557544,v1:172.21.15.143:6833/2150557544] conn(0x7fd2380664a0 0x7fd238068960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:17:59.515 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:17:59.510+0000 7fd250e2d640 1 --2- 172.21.15.143:0/812127034 >> [v2:172.21.15.143:6832/2150557544,v1:172.21.15.143:6833/2150557544] conn(0x7fd2380664a0 0x7fd238068960 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7fd248009ac0 tx=0x7fd248009190 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:17:59.519 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:17:59.514+0000 7fd2417fa640 1 -- 172.21.15.143:0/812127034 <== mon.0 v2:172.21.15.143:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172649 (secure 0 0 0) 0x7fd2440ec2e0 con 0x7fd24c1596d0 2023-11-26T07:17:59.672 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:17:59.666+0000 7fd253091640 1 -- 172.21.15.143:0/812127034 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fd24c0709f0 con 0x7fd24c1596d0 2023-11-26T07:17:59.673 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:17:59.670+0000 7fd2417fa640 1 -- 172.21.15.143:0/812127034 <== mon.0 v2:172.21.15.143:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v20) v1 ==== 74+0+16541 (secure 0 0 0) 0x7fd2440ef900 con 0x7fd24c1596d0 2023-11-26T07:17:59.673 INFO:teuthology.orchestra.run.smithi143.stdout: 2023-11-26T07:17:59.674 INFO:teuthology.orchestra.run.smithi143.stdout:{"epoch":20,"fsid":"f5033060-2469-4a07-95fe-2b76f5404bc1","created":"2023-11-26T07:17:28.613663+0000","modified":"2023-11-26T07:17:55.505435+0000","last_up_change":"2023-11-26T07:17:41.157813+0000","last_in_change":"2023-11-26T07:17:31.181658+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-11-26T07:17:41.758597+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2023-11-26T07:17:51.878332+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2023-11-26T07:17:52.871396+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"ebbee2c6-023e-44b9-9f1d-6f7bfd309730","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6817","nonce":639807369},{"type":"v1","addr":"172.21.15.143:6819","nonce":639807369}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6821","nonce":639807369},{"type":"v1","addr":"172.21.15.143:6823","nonce":639807369}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6830","nonce":639807369},{"type":"v1","addr":"172.21.15.143:6831","nonce":639807369}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6825","nonce":639807369},{"type":"v1","addr":"172.21.15.143:6828","nonce":639807369}]},"public_addr":"172.21.15.143:6819/639807369","cluster_addr":"172.21.15.143:6823/639807369","heartbeat_back_addr":"172.21.15.143:6831/639807369","heartbeat_front_addr":"172.21.15.143:6828/639807369","state":["exists","up"]},{"osd":1,"uuid":"fec4b8be-7122-49e9-9a19-5850a0700001","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6816","nonce":1144942530},{"type":"v1","addr":"172.21.15.143:6818","nonce":1144942530}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6820","nonce":1144942530},{"type":"v1","addr":"172.21.15.143:6822","nonce":1144942530}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6827","nonce":1144942530},{"type":"v1","addr":"172.21.15.143:6829","nonce":1144942530}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6824","nonce":1144942530},{"type":"v1","addr":"172.21.15.143:6826","nonce":1144942530}]},"public_addr":"172.21.15.143:6818/1144942530","cluster_addr":"172.21.15.143:6822/1144942530","heartbeat_back_addr":"172.21.15.143:6829/1144942530","heartbeat_front_addr":"172.21.15.143:6826/1144942530","state":["exists","up"]},{"osd":2,"uuid":"2888af5b-f933-46ca-8f95-6dd494ff43f6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6808","nonce":3909095186},{"type":"v1","addr":"172.21.15.143:6809","nonce":3909095186}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6810","nonce":3909095186},{"type":"v1","addr":"172.21.15.143:6811","nonce":3909095186}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6814","nonce":3909095186},{"type":"v1","addr":"172.21.15.143:6815","nonce":3909095186}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6812","nonce":3909095186},{"type":"v1","addr":"172.21.15.143:6813","nonce":3909095186}]},"public_addr":"172.21.15.143:6809/3909095186","cluster_addr":"172.21.15.143:6811/3909095186","heartbeat_back_addr":"172.21.15.143:6815/3909095186","heartbeat_front_addr":"172.21.15.143:6813/3909095186","state":["exists","up"]},{"osd":3,"uuid":"d1f52435-7bc1-4081-9550-9cd1ba55f549","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6800","nonce":3854210991},{"type":"v1","addr":"172.21.15.143:6801","nonce":3854210991}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6802","nonce":3854210991},{"type":"v1","addr":"172.21.15.143:6803","nonce":3854210991}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6806","nonce":3854210991},{"type":"v1","addr":"172.21.15.143:6807","nonce":3854210991}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6804","nonce":3854210991},{"type":"v1","addr":"172.21.15.143:6805","nonce":3854210991}]},"public_addr":"172.21.15.143:6801/3854210991","cluster_addr":"172.21.15.143:6803/3854210991","heartbeat_back_addr":"172.21.15.143:6807/3854210991","heartbeat_front_addr":"172.21.15.143:6805/3854210991","state":["exists","up"]},{"osd":4,"uuid":"e2ae8b6e-6e25-49b2-bcbf-aacce68f06a3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6800","nonce":3924606902},{"type":"v1","addr":"172.21.15.188:6801","nonce":3924606902}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6802","nonce":3924606902},{"type":"v1","addr":"172.21.15.188:6803","nonce":3924606902}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6806","nonce":3924606902},{"type":"v1","addr":"172.21.15.188:6807","nonce":3924606902}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6804","nonce":3924606902},{"type":"v1","addr":"172.21.15.188:6805","nonce":3924606902}]},"public_addr":"172.21.15.188:6801/3924606902","cluster_addr":"172.21.15.188:6803/3924606902","heartbeat_back_addr":"172.21.15.188:6807/3924606902","heartbeat_front_addr":"172.21.15.188:6805/3924606902","state":["exists","up"]},{"osd":5,"uuid":"3cec8a32-7698-4c36-8e89-e4ba11934f33","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6813","nonce":186841544},{"type":"v1","addr":"172.21.15.188:6815","nonce":186841544}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6817","nonce":186841544},{"type":"v1","addr":"172.21.15.188:6818","nonce":186841544}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6822","nonce":186841544},{"type":"v1","addr":"172.21.15.188:6823","nonce":186841544}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6820","nonce":186841544},{"type":"v1","addr":"172.21.15.188:6821","nonce":186841544}]},"public_addr":"172.21.15.188:6815/186841544","cluster_addr":"172.21.15.188:6818/186841544","heartbeat_back_addr":"172.21.15.188:6823/186841544","heartbeat_front_addr":"172.21.15.188:6821/186841544","state":["exists","up"]},{"osd":6,"uuid":"58a65d5d-e090-473a-81b4-ad38e85029f9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6808","nonce":2153499405},{"type":"v1","addr":"172.21.15.188:6809","nonce":2153499405}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6810","nonce":2153499405},{"type":"v1","addr":"172.21.15.188:6811","nonce":2153499405}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6816","nonce":2153499405},{"type":"v1","addr":"172.21.15.188:6819","nonce":2153499405}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6812","nonce":2153499405},{"type":"v1","addr":"172.21.15.188:6814","nonce":2153499405}]},"public_addr":"172.21.15.188:6809/2153499405","cluster_addr":"172.21.15.188:6811/2153499405","heartbeat_back_addr":"172.21.15.188:6819/2153499405","heartbeat_front_addr":"172.21.15.188:6814/2153499405","state":["exists","up"]},{"osd":7,"uuid":"ea5b7f70-3554-4cf8-ac30-f7c59b80678d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6824","nonce":2722096729},{"type":"v1","addr":"172.21.15.188:6825","nonce":2722096729}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6826","nonce":2722096729},{"type":"v1","addr":"172.21.15.188:6827","nonce":2722096729}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6830","nonce":2722096729},{"type":"v1","addr":"172.21.15.188:6831","nonce":2722096729}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6828","nonce":2722096729},{"type":"v1","addr":"172.21.15.188:6829","nonce":2722096729}]},"public_addr":"172.21.15.188:6825/2722096729","cluster_addr":"172.21.15.188:6827/2722096729","heartbeat_back_addr":"172.21.15.188:6831/2722096729","heartbeat_front_addr":"172.21.15.188:6829/2722096729","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-26T07:17:36.995032+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-26T07:17:36.932266+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-26T07:17:36.829304+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-26T07:17:36.804297+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-26T07:17:36.667892+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-26T07:17:36.964868+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-26T07:17:36.827332+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-26T07:17:36.893179+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-11-26T07:17:59.675 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:17:59.670+0000 7fd253091640 1 -- 172.21.15.143:0/812127034 >> [v2:172.21.15.143:6832/2150557544,v1:172.21.15.143:6833/2150557544] conn(0x7fd2380664a0 msgr2=0x7fd238068960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:17:59.676 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:17:59.670+0000 7fd253091640 1 --2- 172.21.15.143:0/812127034 >> [v2:172.21.15.143:6832/2150557544,v1:172.21.15.143:6833/2150557544] conn(0x7fd2380664a0 0x7fd238068960 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7fd248009ac0 tx=0x7fd248009190 comp rx=0 tx=0).stop 2023-11-26T07:17:59.676 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:17:59.670+0000 7fd253091640 1 -- 172.21.15.143:0/812127034 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fd24c1596d0 msgr2=0x7fd24c13efa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:17:59.676 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:17:59.670+0000 7fd253091640 1 --2- 172.21.15.143:0/812127034 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fd24c1596d0 0x7fd24c13efa0 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7fd2440af040 tx=0x7fd244019770 comp rx=0 tx=0).stop 2023-11-26T07:17:59.676 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:17:59.670+0000 7fd253091640 1 -- 172.21.15.143:0/812127034 shutdown_connections 2023-11-26T07:17:59.676 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:17:59.670+0000 7fd253091640 1 --2- 172.21.15.143:0/812127034 >> [v2:172.21.15.143:6832/2150557544,v1:172.21.15.143:6833/2150557544] conn(0x7fd2380664a0 0x7fd238068960 unknown :-1 s=CLOSED pgs=75 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:17:59.676 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:17:59.670+0000 7fd253091640 1 --2- 172.21.15.143:0/812127034 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fd24c1596d0 0x7fd24c13efa0 unknown :-1 s=CLOSED pgs=93 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:17:59.676 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:17:59.670+0000 7fd253091640 1 --2- 172.21.15.143:0/812127034 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fd24c0ce680 0x7fd24c0afb80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:17:59.676 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:17:59.670+0000 7fd253091640 1 --2- 172.21.15.143:0/812127034 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fd24c0b2f80 0x7fd24c15c730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:17:59.676 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:17:59.670+0000 7fd253091640 1 -- 172.21.15.143:0/812127034 >> 172.21.15.143:0/812127034 conn(0x7fd24c06a890 msgr2=0x7fd24c166800 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:17:59.676 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:17:59.670+0000 7fd253091640 1 -- 172.21.15.143:0/812127034 shutdown_connections 2023-11-26T07:17:59.676 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:17:59.670+0000 7fd253091640 1 -- 172.21.15.143:0/812127034 wait complete. 2023-11-26T07:17:59.694 INFO:tasks.ceph.ceph_manager.ceph:all up! 2023-11-26T07:17:59.695 DEBUG:teuthology.orchestra.run.smithi143:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-11-26T07:17:59.833 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:17:59.830+0000 7f43f953f640 1 Processor -- start 2023-11-26T07:17:59.833 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:17:59.830+0000 7f43f953f640 1 -- start start 2023-11-26T07:17:59.835 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:17:59.830+0000 7f43f953f640 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7f43f40d0e40 0x7f43f40ce430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:17:59.835 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:17:59.830+0000 7f43f953f640 1 -- --> v1:172.21.15.143:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f43f40d3d20 con 0x7f43f40b1860 2023-11-26T07:17:59.835 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:17:59.830+0000 7f43f953f640 1 -- --> v1:172.21.15.188:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f43f40afd70 con 0x7f43f40cfa70 2023-11-26T07:17:59.835 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:17:59.830+0000 7f43f953f640 1 -- --> v2:172.21.15.143:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f43f40d4000 con 0x7f43f40d0e40 2023-11-26T07:17:59.835 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:17:59.830+0000 7f43f8d3e640 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7f43f40d0e40 0x7f43f40ce430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:17:59.835 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:17:59.830+0000 7f43f8d3e640 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7f43f40d0e40 0x7f43f40ce430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:46328/0 (socket says 172.21.15.143:46328) 2023-11-26T07:17:59.836 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:17:59.830+0000 7f43f8d3e640 1 -- 172.21.15.143:0/2977291205 learned_addr learned my addr 172.21.15.143:0/2977291205 (peer_addr_for_me v2:172.21.15.143:0/0) 2023-11-26T07:17:59.836 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:17:59.830+0000 7f43f8d3e640 1 -- 172.21.15.143:0/2977291205 >> v1:172.21.15.188:6789/0 conn(0x7f43f40cfa70 legacy=0x7f43f40cfe50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:17:59.836 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:17:59.830+0000 7f43f8d3e640 1 -- 172.21.15.143:0/2977291205 >> v1:172.21.15.143:6789/0 conn(0x7f43f40b1860 legacy=0x7f43f40cdd20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:17:59.836 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:17:59.830+0000 7f43f8d3e640 1 -- 172.21.15.143:0/2977291205 --> v2:172.21.15.143:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f43f40b0050 con 0x7f43f40d0e40 2023-11-26T07:17:59.837 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:17:59.830+0000 7f43f2ffd640 1 -- 172.21.15.143:0/2977291205 <== mon.0 v1:172.21.15.143:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3640839000 0 0) 0x7f43f40d3d20 con 0x7f43f40b1860 2023-11-26T07:17:59.838 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:17:59.834+0000 7f43f8d3e640 1 --2- 172.21.15.143:0/2977291205 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f43f40d0e40 0x7f43f40ce430 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7f43ec009e50 tx=0x7f43ec01ca10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2f63b87cba3bbd65 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:17:59.838 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:17:59.834+0000 7f43f2ffd640 1 -- 172.21.15.143:0/2977291205 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f43ec0a57f0 con 0x7f43f40d0e40 2023-11-26T07:17:59.838 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:17:59.834+0000 7f43f2ffd640 1 -- 172.21.15.143:0/2977291205 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f43ec0a5950 con 0x7f43f40d0e40 2023-11-26T07:17:59.839 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:17:59.834+0000 7f43f2ffd640 1 -- 172.21.15.143:0/2977291205 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f43ec0a5ce0 con 0x7f43f40d0e40 2023-11-26T07:17:59.839 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:17:59.834+0000 7f43f953f640 1 -- 172.21.15.143:0/2977291205 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f43f40d0e40 msgr2=0x7f43f40ce430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:17:59.840 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:17:59.834+0000 7f43f953f640 1 --2- 172.21.15.143:0/2977291205 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f43f40d0e40 0x7f43f40ce430 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7f43ec009e50 tx=0x7f43ec01ca10 comp rx=0 tx=0).stop 2023-11-26T07:17:59.840 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:17:59.834+0000 7f43f953f640 1 -- 172.21.15.143:0/2977291205 shutdown_connections 2023-11-26T07:17:59.840 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:17:59.834+0000 7f43f953f640 1 --2- 172.21.15.143:0/2977291205 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f43f40d0e40 0x7f43f40ce430 unknown :-1 s=CLOSED pgs=95 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:17:59.840 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:17:59.834+0000 7f43f953f640 1 -- 172.21.15.143:0/2977291205 >> 172.21.15.143:0/2977291205 conn(0x7f43f40d3060 msgr2=0x7f43f40d3470 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:17:59.840 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:17:59.834+0000 7f43f953f640 1 -- 172.21.15.143:0/2977291205 shutdown_connections 2023-11-26T07:17:59.840 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:17:59.834+0000 7f43f953f640 1 -- 172.21.15.143:0/2977291205 wait complete. 2023-11-26T07:17:59.841 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:17:59.838+0000 7f43f953f640 1 Processor -- start 2023-11-26T07:17:59.841 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:17:59.838+0000 7f43f953f640 1 -- start start 2023-11-26T07:17:59.842 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:17:59.838+0000 7f43f953f640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f43f40b1860 0x7f43f4154d70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:17:59.843 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:17:59.838+0000 7f43f953f640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f43f40cfa70 0x7f43f4159520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:17:59.843 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:17:59.838+0000 7f43f953f640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f43f40d0e40 0x7f43f415c040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:17:59.843 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:17:59.838+0000 7f43f953f640 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f43f4159a60 con 0x7f43f40cfa70 2023-11-26T07:17:59.843 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:17:59.838+0000 7f43f953f640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f43f41548e0 con 0x7f43f40d0e40 2023-11-26T07:17:59.843 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:17:59.838+0000 7f43f953f640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f43f4158c40 con 0x7f43f40b1860 2023-11-26T07:17:59.843 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:17:59.838+0000 7f43f37fe640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f43f40cfa70 0x7f43f4159520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:17:59.843 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:17:59.838+0000 7f43f37fe640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f43f40cfa70 0x7f43f4159520 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:46332/0 (socket says 172.21.15.143:46332) 2023-11-26T07:17:59.844 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:17:59.838+0000 7f43f37fe640 1 -- 172.21.15.143:0/1382956314 learned_addr learned my addr 172.21.15.143:0/1382956314 (peer_addr_for_me v2:172.21.15.143:0/0) 2023-11-26T07:17:59.844 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:17:59.838+0000 7f43f8d3e640 1 --2- 172.21.15.143:0/1382956314 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f43f40d0e40 0x7f43f415c040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:17:59.844 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:17:59.838+0000 7f43f3fff640 1 --2- 172.21.15.143:0/1382956314 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f43f40b1860 0x7f43f4154d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:17:59.844 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:17:59.838+0000 7f43f37fe640 1 -- 172.21.15.143:0/1382956314 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f43f40b1860 msgr2=0x7f43f4154d70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:17:59.844 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:17:59.838+0000 7f43f37fe640 1 --2- 172.21.15.143:0/1382956314 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f43f40b1860 0x7f43f4154d70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:17:59.845 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:17:59.838+0000 7f43f37fe640 1 -- 172.21.15.143:0/1382956314 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f43f40d0e40 msgr2=0x7f43f415c040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:17:59.845 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:17:59.838+0000 7f43f37fe640 1 --2- 172.21.15.143:0/1382956314 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f43f40d0e40 0x7f43f415c040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:17:59.845 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:17:59.838+0000 7f43f37fe640 1 -- 172.21.15.143:0/1382956314 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f43ec009b30 con 0x7f43f40cfa70 2023-11-26T07:17:59.845 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:17:59.838+0000 7f43f37fe640 1 --2- 172.21.15.143:0/1382956314 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f43f40cfa70 0x7f43f4159520 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7f43e800c480 tx=0x7f43e800c940 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:17:59.846 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:17:59.838+0000 7f43f0ff9640 1 -- 172.21.15.143:0/1382956314 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f43e8005480 con 0x7f43f40cfa70 2023-11-26T07:17:59.846 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:17:59.838+0000 7f43f0ff9640 1 -- 172.21.15.143:0/1382956314 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f43e8003600 con 0x7f43f40cfa70 2023-11-26T07:17:59.847 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:17:59.838+0000 7f43f0ff9640 1 -- 172.21.15.143:0/1382956314 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f43e80039b0 con 0x7f43f40cfa70 2023-11-26T07:17:59.847 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:17:59.838+0000 7f43f953f640 1 -- 172.21.15.143:0/1382956314 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f43f4158f20 con 0x7f43f40cfa70 2023-11-26T07:17:59.847 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:17:59.838+0000 7f43f953f640 1 -- 172.21.15.143:0/1382956314 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f43f4154570 con 0x7f43f40cfa70 2023-11-26T07:17:59.847 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:17:59.838+0000 7f43f953f640 1 -- 172.21.15.143:0/1382956314 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f43f4067500 con 0x7f43f40cfa70 2023-11-26T07:17:59.847 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:17:59.842+0000 7f43f0ff9640 1 -- 172.21.15.143:0/1382956314 <== mon.0 v2:172.21.15.143:3300/0 4 ==== mgrmap(e 5) v1 ==== 85852+0+0 (secure 0 0 0) 0x7f43e8003b10 con 0x7f43f40cfa70 2023-11-26T07:17:59.847 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:17:59.842+0000 7f43f0ff9640 1 --2- 172.21.15.143:0/1382956314 >> [v2:172.21.15.143:6832/2150557544,v1:172.21.15.143:6833/2150557544] conn(0x7f43e0066470 0x7f43e0068930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:17:59.847 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:17:59.842+0000 7f43f0ff9640 1 -- 172.21.15.143:0/1382956314 <== mon.0 v2:172.21.15.143:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f43e80839c0 con 0x7f43f40cfa70 2023-11-26T07:17:59.848 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:17:59.842+0000 7f43f3fff640 1 --2- 172.21.15.143:0/1382956314 >> [v2:172.21.15.143:6832/2150557544,v1:172.21.15.143:6833/2150557544] conn(0x7f43e0066470 0x7f43e0068930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:17:59.848 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:17:59.842+0000 7f43f3fff640 1 --2- 172.21.15.143:0/1382956314 >> [v2:172.21.15.143:6832/2150557544,v1:172.21.15.143:6833/2150557544] conn(0x7f43e0066470 0x7f43e0068930 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7f43e4000f70 tx=0x7f43e4009040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:17:59.850 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:17:59.846+0000 7f43f0ff9640 1 -- 172.21.15.143:0/1382956314 <== mon.0 v2:172.21.15.143:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172649 (secure 0 0 0) 0x7f43e8050e30 con 0x7f43f40cfa70 2023-11-26T07:18:00.000 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:17:59.998+0000 7f43f953f640 1 -- 172.21.15.143:0/1382956314 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f43f40cd640 con 0x7f43f40cfa70 2023-11-26T07:18:00.002 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:17:59.998+0000 7f43f0ff9640 1 -- 172.21.15.143:0/1382956314 <== mon.0 v2:172.21.15.143:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v20) v1 ==== 74+0+16541 (secure 0 0 0) 0x7f43e8054450 con 0x7f43f40cfa70 2023-11-26T07:18:00.002 INFO:teuthology.orchestra.run.smithi143.stdout: 2023-11-26T07:18:00.003 INFO:teuthology.orchestra.run.smithi143.stdout:{"epoch":20,"fsid":"f5033060-2469-4a07-95fe-2b76f5404bc1","created":"2023-11-26T07:17:28.613663+0000","modified":"2023-11-26T07:17:55.505435+0000","last_up_change":"2023-11-26T07:17:41.157813+0000","last_in_change":"2023-11-26T07:17:31.181658+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-11-26T07:17:41.758597+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2023-11-26T07:17:51.878332+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2023-11-26T07:17:52.871396+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"ebbee2c6-023e-44b9-9f1d-6f7bfd309730","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6817","nonce":639807369},{"type":"v1","addr":"172.21.15.143:6819","nonce":639807369}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6821","nonce":639807369},{"type":"v1","addr":"172.21.15.143:6823","nonce":639807369}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6830","nonce":639807369},{"type":"v1","addr":"172.21.15.143:6831","nonce":639807369}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6825","nonce":639807369},{"type":"v1","addr":"172.21.15.143:6828","nonce":639807369}]},"public_addr":"172.21.15.143:6819/639807369","cluster_addr":"172.21.15.143:6823/639807369","heartbeat_back_addr":"172.21.15.143:6831/639807369","heartbeat_front_addr":"172.21.15.143:6828/639807369","state":["exists","up"]},{"osd":1,"uuid":"fec4b8be-7122-49e9-9a19-5850a0700001","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6816","nonce":1144942530},{"type":"v1","addr":"172.21.15.143:6818","nonce":1144942530}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6820","nonce":1144942530},{"type":"v1","addr":"172.21.15.143:6822","nonce":1144942530}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6827","nonce":1144942530},{"type":"v1","addr":"172.21.15.143:6829","nonce":1144942530}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6824","nonce":1144942530},{"type":"v1","addr":"172.21.15.143:6826","nonce":1144942530}]},"public_addr":"172.21.15.143:6818/1144942530","cluster_addr":"172.21.15.143:6822/1144942530","heartbeat_back_addr":"172.21.15.143:6829/1144942530","heartbeat_front_addr":"172.21.15.143:6826/1144942530","state":["exists","up"]},{"osd":2,"uuid":"2888af5b-f933-46ca-8f95-6dd494ff43f6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6808","nonce":3909095186},{"type":"v1","addr":"172.21.15.143:6809","nonce":3909095186}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6810","nonce":3909095186},{"type":"v1","addr":"172.21.15.143:6811","nonce":3909095186}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6814","nonce":3909095186},{"type":"v1","addr":"172.21.15.143:6815","nonce":3909095186}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6812","nonce":3909095186},{"type":"v1","addr":"172.21.15.143:6813","nonce":3909095186}]},"public_addr":"172.21.15.143:6809/3909095186","cluster_addr":"172.21.15.143:6811/3909095186","heartbeat_back_addr":"172.21.15.143:6815/3909095186","heartbeat_front_addr":"172.21.15.143:6813/3909095186","state":["exists","up"]},{"osd":3,"uuid":"d1f52435-7bc1-4081-9550-9cd1ba55f549","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6800","nonce":3854210991},{"type":"v1","addr":"172.21.15.143:6801","nonce":3854210991}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6802","nonce":3854210991},{"type":"v1","addr":"172.21.15.143:6803","nonce":3854210991}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6806","nonce":3854210991},{"type":"v1","addr":"172.21.15.143:6807","nonce":3854210991}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6804","nonce":3854210991},{"type":"v1","addr":"172.21.15.143:6805","nonce":3854210991}]},"public_addr":"172.21.15.143:6801/3854210991","cluster_addr":"172.21.15.143:6803/3854210991","heartbeat_back_addr":"172.21.15.143:6807/3854210991","heartbeat_front_addr":"172.21.15.143:6805/3854210991","state":["exists","up"]},{"osd":4,"uuid":"e2ae8b6e-6e25-49b2-bcbf-aacce68f06a3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6800","nonce":3924606902},{"type":"v1","addr":"172.21.15.188:6801","nonce":3924606902}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6802","nonce":3924606902},{"type":"v1","addr":"172.21.15.188:6803","nonce":3924606902}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6806","nonce":3924606902},{"type":"v1","addr":"172.21.15.188:6807","nonce":3924606902}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6804","nonce":3924606902},{"type":"v1","addr":"172.21.15.188:6805","nonce":3924606902}]},"public_addr":"172.21.15.188:6801/3924606902","cluster_addr":"172.21.15.188:6803/3924606902","heartbeat_back_addr":"172.21.15.188:6807/3924606902","heartbeat_front_addr":"172.21.15.188:6805/3924606902","state":["exists","up"]},{"osd":5,"uuid":"3cec8a32-7698-4c36-8e89-e4ba11934f33","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6813","nonce":186841544},{"type":"v1","addr":"172.21.15.188:6815","nonce":186841544}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6817","nonce":186841544},{"type":"v1","addr":"172.21.15.188:6818","nonce":186841544}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6822","nonce":186841544},{"type":"v1","addr":"172.21.15.188:6823","nonce":186841544}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6820","nonce":186841544},{"type":"v1","addr":"172.21.15.188:6821","nonce":186841544}]},"public_addr":"172.21.15.188:6815/186841544","cluster_addr":"172.21.15.188:6818/186841544","heartbeat_back_addr":"172.21.15.188:6823/186841544","heartbeat_front_addr":"172.21.15.188:6821/186841544","state":["exists","up"]},{"osd":6,"uuid":"58a65d5d-e090-473a-81b4-ad38e85029f9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6808","nonce":2153499405},{"type":"v1","addr":"172.21.15.188:6809","nonce":2153499405}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6810","nonce":2153499405},{"type":"v1","addr":"172.21.15.188:6811","nonce":2153499405}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6816","nonce":2153499405},{"type":"v1","addr":"172.21.15.188:6819","nonce":2153499405}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6812","nonce":2153499405},{"type":"v1","addr":"172.21.15.188:6814","nonce":2153499405}]},"public_addr":"172.21.15.188:6809/2153499405","cluster_addr":"172.21.15.188:6811/2153499405","heartbeat_back_addr":"172.21.15.188:6819/2153499405","heartbeat_front_addr":"172.21.15.188:6814/2153499405","state":["exists","up"]},{"osd":7,"uuid":"ea5b7f70-3554-4cf8-ac30-f7c59b80678d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6824","nonce":2722096729},{"type":"v1","addr":"172.21.15.188:6825","nonce":2722096729}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6826","nonce":2722096729},{"type":"v1","addr":"172.21.15.188:6827","nonce":2722096729}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6830","nonce":2722096729},{"type":"v1","addr":"172.21.15.188:6831","nonce":2722096729}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6828","nonce":2722096729},{"type":"v1","addr":"172.21.15.188:6829","nonce":2722096729}]},"public_addr":"172.21.15.188:6825/2722096729","cluster_addr":"172.21.15.188:6827/2722096729","heartbeat_back_addr":"172.21.15.188:6831/2722096729","heartbeat_front_addr":"172.21.15.188:6829/2722096729","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-26T07:17:36.995032+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-26T07:17:36.932266+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-26T07:17:36.829304+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-26T07:17:36.804297+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-26T07:17:36.667892+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-26T07:17:36.964868+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-26T07:17:36.827332+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-26T07:17:36.893179+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-11-26T07:18:00.005 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:00.002+0000 7f43f953f640 1 -- 172.21.15.143:0/1382956314 >> [v2:172.21.15.143:6832/2150557544,v1:172.21.15.143:6833/2150557544] conn(0x7f43e0066470 msgr2=0x7f43e0068930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:18:00.005 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:00.002+0000 7f43f953f640 1 --2- 172.21.15.143:0/1382956314 >> [v2:172.21.15.143:6832/2150557544,v1:172.21.15.143:6833/2150557544] conn(0x7f43e0066470 0x7f43e0068930 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7f43e4000f70 tx=0x7f43e4009040 comp rx=0 tx=0).stop 2023-11-26T07:18:00.005 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:00.002+0000 7f43f953f640 1 -- 172.21.15.143:0/1382956314 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f43f40cfa70 msgr2=0x7f43f4159520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:18:00.006 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:00.002+0000 7f43f953f640 1 --2- 172.21.15.143:0/1382956314 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f43f40cfa70 0x7f43f4159520 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7f43e800c480 tx=0x7f43e800c940 comp rx=0 tx=0).stop 2023-11-26T07:18:00.006 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:00.002+0000 7f43f953f640 1 -- 172.21.15.143:0/1382956314 shutdown_connections 2023-11-26T07:18:00.006 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:00.002+0000 7f43f953f640 1 --2- 172.21.15.143:0/1382956314 >> [v2:172.21.15.143:6832/2150557544,v1:172.21.15.143:6833/2150557544] conn(0x7f43e0066470 0x7f43e0068930 unknown :-1 s=CLOSED pgs=76 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:18:00.006 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:00.002+0000 7f43f953f640 1 --2- 172.21.15.143:0/1382956314 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f43f40d0e40 0x7f43f415c040 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:18:00.006 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:00.002+0000 7f43f953f640 1 --2- 172.21.15.143:0/1382956314 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f43f40cfa70 0x7f43f4159520 unknown :-1 s=CLOSED pgs=96 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:18:00.006 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:00.002+0000 7f43f953f640 1 --2- 172.21.15.143:0/1382956314 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f43f40b1860 0x7f43f4154d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:18:00.007 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:00.002+0000 7f43f953f640 1 -- 172.21.15.143:0/1382956314 >> 172.21.15.143:0/1382956314 conn(0x7f43f40d3060 msgr2=0x7f43f40d3440 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:18:00.007 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:00.002+0000 7f43f953f640 1 -- 172.21.15.143:0/1382956314 shutdown_connections 2023-11-26T07:18:00.007 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:00.002+0000 7f43f953f640 1 -- 172.21.15.143:0/1382956314 wait complete. 2023-11-26T07:18:00.025 DEBUG:teuthology.orchestra.run.smithi143:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.0 flush_pg_stats 2023-11-26T07:18:00.026 DEBUG:teuthology.orchestra.run.smithi143:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.1 flush_pg_stats 2023-11-26T07:18:00.027 DEBUG:teuthology.orchestra.run.smithi143:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.2 flush_pg_stats 2023-11-26T07:18:00.027 DEBUG:teuthology.orchestra.run.smithi143:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.3 flush_pg_stats 2023-11-26T07:18:00.028 DEBUG:teuthology.orchestra.run.smithi143:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.4 flush_pg_stats 2023-11-26T07:18:00.028 DEBUG:teuthology.orchestra.run.smithi143:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.5 flush_pg_stats 2023-11-26T07:18:00.029 DEBUG:teuthology.orchestra.run.smithi143:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.6 flush_pg_stats 2023-11-26T07:18:00.029 DEBUG:teuthology.orchestra.run.smithi143:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.7 flush_pg_stats 2023-11-26T07:18:00.270 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:00.266+0000 7f4c2f335640 1 Processor -- start 2023-11-26T07:18:00.271 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:00.266+0000 7f4c2f335640 1 -- start start 2023-11-26T07:18:00.272 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:00.266+0000 7fcea9d35640 1 Processor -- start 2023-11-26T07:18:00.273 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:00.266+0000 7fcea9d35640 1 -- start start 2023-11-26T07:18:00.273 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:00.266+0000 7fbb85b94640 1 Processor -- start 2023-11-26T07:18:00.273 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:00.270+0000 7f4c2f335640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f4c280b2790 0x7f4c281592f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:18:00.274 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:00.266+0000 7fbb85b94640 1 -- start start 2023-11-26T07:18:00.274 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:00.270+0000 7f4c2f335640 1 -- --> v1:172.21.15.143:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4c280ac720 con 0x7f4c28072620 2023-11-26T07:18:00.275 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:00.270+0000 7f4c2f335640 1 -- --> v1:172.21.15.188:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4c280aca00 con 0x7f4c280745d0 2023-11-26T07:18:00.275 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:00.270+0000 7f4c2f335640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4c280acce0 con 0x7f4c280b2790 2023-11-26T07:18:00.275 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:00.270+0000 7fcea9d35640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fcea414afa0 0x7fcea406ce00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:18:00.275 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:00.270+0000 7fcea9d35640 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7fcea40b3f80 0x7fcea40ccc90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:18:00.276 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:00.270+0000 7fcea9d35640 1 -- --> v1:172.21.15.143:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fcea40cc800 con 0x7fcea406d340 2023-11-26T07:18:00.276 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:00.270+0000 7fcea9d35640 1 -- --> v2:172.21.15.143:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fcea414b380 con 0x7fcea40b3f80 2023-11-26T07:18:00.276 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:00.270+0000 7fcea9d35640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcea406d7c0 con 0x7fcea414afa0 2023-11-26T07:18:00.276 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:00.270+0000 7f4c2c8d0640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f4c280b2790 0x7f4c281592f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:18:00.276 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:00.270+0000 7f4c2c8d0640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f4c280b2790 0x7f4c281592f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.188:3301/0 says I am v2:172.21.15.143:44998/0 (socket says 172.21.15.143:44998) 2023-11-26T07:18:00.277 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:00.270+0000 7f4c2c8d0640 1 -- 172.21.15.143:0/1455233037 learned_addr learned my addr 172.21.15.143:0/1455233037 (peer_addr_for_me v2:172.21.15.143:0/0) 2023-11-26T07:18:00.277 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:00.270+0000 7fcea2ffd640 1 --1- >> v1:172.21.15.143:6789/0 conn(0x7fcea406d340 0x7fcea40b3870 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.143:6789/0 says I am v1:172.21.15.143:40142/0 (socket says 172.21.15.143:40142) 2023-11-26T07:18:00.277 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:00.270+0000 7fcea2ffd640 1 -- 172.21.15.143:0/866294174 learned_addr learned my addr 172.21.15.143:0/866294174 (peer_addr_for_me v1:172.21.15.143:0/0) 2023-11-26T07:18:00.277 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:00.270+0000 7fbb85b94640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fbb800d1e20 0x7fbb800c6df0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:18:00.278 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:00.270+0000 7fcea3fff640 1 --2- 172.21.15.143:0/866294174 >> v2:172.21.15.143:3300/0 conn(0x7fcea40b3f80 0x7fcea40ccc90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:18:00.278 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:00.270+0000 7fbb85b94640 1 -- --> v1:172.21.15.143:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbb800d3b20 con 0x7fbb80160d60 2023-11-26T07:18:00.278 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:00.270+0000 7fbb85b94640 1 -- --> v1:172.21.15.188:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbb800ce750 con 0x7fbb800cf9c0 2023-11-26T07:18:00.279 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:00.270+0000 7fbb85b94640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbb800b4820 con 0x7fbb800d1e20 2023-11-26T07:18:00.279 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:00.270+0000 7f4c2c8d0640 1 -- 172.21.15.143:0/1455233037 >> v1:172.21.15.188:6789/0 conn(0x7f4c280745d0 legacy=0x7f4c280b2080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:18:00.279 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:00.270+0000 7f4c2c8d0640 1 -- 172.21.15.143:0/1455233037 >> v1:172.21.15.143:6789/0 conn(0x7f4c28072620 legacy=0x7f4c28159830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:18:00.279 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:00.270+0000 7f4c2c8d0640 1 -- 172.21.15.143:0/1455233037 --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4c280acfc0 con 0x7f4c280b2790 2023-11-26T07:18:00.279 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:00.270+0000 7fbb77fff640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fbb800d1e20 0x7fbb800c6df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:18:00.280 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:00.270+0000 7fcea3fff640 1 -- 172.21.15.143:0/866294174 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fcea414afa0 msgr2=0x7fcea406ce00 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2023-11-26T07:18:00.280 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:00.270+0000 7fcea3fff640 1 --2- 172.21.15.143:0/866294174 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fcea414afa0 0x7fcea406ce00 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:18:00.280 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:00.270+0000 7fcea3fff640 1 -- 172.21.15.143:0/866294174 >> v1:172.21.15.143:6789/0 conn(0x7fcea406d340 legacy=0x7fcea40b3870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:18:00.280 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:00.270+0000 7fcea3fff640 1 -- 172.21.15.143:0/866294174 --> v2:172.21.15.143:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcea406daa0 con 0x7fcea40b3f80 2023-11-26T07:18:00.281 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:00.270+0000 7fbb77fff640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fbb800d1e20 0x7fbb800c6df0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.188:3301/0 says I am v2:172.21.15.143:45024/0 (socket says 172.21.15.143:45024) 2023-11-26T07:18:00.281 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:00.270+0000 7fbb77fff640 1 -- 172.21.15.143:0/3359646625 learned_addr learned my addr 172.21.15.143:0/3359646625 (peer_addr_for_me v2:172.21.15.143:0/0) 2023-11-26T07:18:00.281 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:00.270+0000 7fbb7effd640 1 -- 172.21.15.143:0/3359646625 <== mon.1 v1:172.21.15.188:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2888266936 0 0) 0x7fbb800ce750 con 0x7fbb800cf9c0 2023-11-26T07:18:00.281 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:00.270+0000 7fbb77fff640 1 -- 172.21.15.143:0/3359646625 >> v1:172.21.15.188:6789/0 conn(0x7fbb800cf9c0 legacy=0x7fbb80160650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:18:00.281 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:00.270+0000 7fbb77fff640 1 -- 172.21.15.143:0/3359646625 >> v1:172.21.15.143:6789/0 conn(0x7fbb80160d60 legacy=0x7fbb800c7330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:18:00.281 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:00.270+0000 7fbb77fff640 1 -- 172.21.15.143:0/3359646625 --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbb800d3d70 con 0x7fbb800d1e20 2023-11-26T07:18:00.282 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:00.274+0000 7fcea3fff640 1 --2- 172.21.15.143:0/866294174 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fcea40b3f80 0x7fcea40ccc90 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7fce9c0097c0 tx=0x7fce9c01c4a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d467aae6f26f0623 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:18:00.282 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:00.274+0000 7fcea27fc640 1 -- 172.21.15.143:0/866294174 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fce9c0ab600 con 0x7fcea40b3f80 2023-11-26T07:18:00.282 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:00.274+0000 7f4c2c8d0640 1 --2- 172.21.15.143:0/1455233037 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f4c280b2790 0x7f4c281592f0 secure :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0x7f4c1c009e80 tx=0x7f4c1c01c8e0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=38534199e2a38197 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:18:00.282 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:00.274+0000 7fcea27fc640 1 -- 172.21.15.143:0/866294174 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fce9c0a7050 con 0x7fcea40b3f80 2023-11-26T07:18:00.282 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:00.274+0000 7fcea27fc640 1 -- 172.21.15.143:0/866294174 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fce9c0ab980 con 0x7fcea40b3f80 2023-11-26T07:18:00.283 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:00.278+0000 7f4c27fff640 1 -- 172.21.15.143:0/1455233037 <== mon.2 v2:172.21.15.188:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4c1c0a5630 con 0x7f4c280b2790 2023-11-26T07:18:00.283 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:00.278+0000 7f4c27fff640 1 -- 172.21.15.143:0/1455233037 <== mon.2 v2:172.21.15.188:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4c1c0a5790 con 0x7f4c280b2790 2023-11-26T07:18:00.283 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:00.278+0000 7f4c27fff640 1 -- 172.21.15.143:0/1455233037 <== mon.2 v2:172.21.15.188:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4c1c0a5b20 con 0x7f4c280b2790 2023-11-26T07:18:00.283 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:00.278+0000 7fbb77fff640 1 --2- 172.21.15.143:0/3359646625 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fbb800d1e20 0x7fbb800c6df0 secure :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0x7fbb680098a0 tx=0x7fbb6801c450 comp rx=0 tx=0).ready entity=mon.2 client_cookie=dc046b9adc5addfa server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:18:00.284 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:00.278+0000 7fcea9d35640 1 -- 172.21.15.143:0/866294174 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fcea40b3f80 msgr2=0x7fcea40ccc90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:18:00.284 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:00.278+0000 7fcea9d35640 1 --2- 172.21.15.143:0/866294174 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fcea40b3f80 0x7fcea40ccc90 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7fce9c0097c0 tx=0x7fce9c01c4a0 comp rx=0 tx=0).stop 2023-11-26T07:18:00.284 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:00.278+0000 7fcea9d35640 1 -- 172.21.15.143:0/866294174 shutdown_connections 2023-11-26T07:18:00.284 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:00.278+0000 7fcea9d35640 1 --2- 172.21.15.143:0/866294174 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fcea40b3f80 0x7fcea40ccc90 unknown :-1 s=CLOSED pgs=100 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:18:00.284 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:00.278+0000 7fcea9d35640 1 --2- 172.21.15.143:0/866294174 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fcea414afa0 0x7fcea406ce00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:18:00.284 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:00.278+0000 7fcea9d35640 1 -- 172.21.15.143:0/866294174 >> 172.21.15.143:0/866294174 conn(0x7fcea40aea10 msgr2=0x7fcea40aee20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:18:00.284 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:00.278+0000 7fcea9d35640 1 -- 172.21.15.143:0/866294174 shutdown_connections 2023-11-26T07:18:00.285 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:00.278+0000 7f4c2f335640 1 -- 172.21.15.143:0/1455233037 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f4c280b2790 msgr2=0x7f4c281592f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:18:00.285 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:00.278+0000 7f4c2f335640 1 --2- 172.21.15.143:0/1455233037 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f4c280b2790 0x7f4c281592f0 secure :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0x7f4c1c009e80 tx=0x7f4c1c01c8e0 comp rx=0 tx=0).stop 2023-11-26T07:18:00.285 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:00.278+0000 7fbb7effd640 1 -- 172.21.15.143:0/3359646625 <== mon.2 v2:172.21.15.188:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbb680ab610 con 0x7fbb800d1e20 2023-11-26T07:18:00.285 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:00.278+0000 7fbb7effd640 1 -- 172.21.15.143:0/3359646625 <== mon.2 v2:172.21.15.188:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbb680ab770 con 0x7fbb800d1e20 2023-11-26T07:18:00.285 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:00.278+0000 7fbb7effd640 1 -- 172.21.15.143:0/3359646625 <== mon.2 v2:172.21.15.188:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbb680abb00 con 0x7fbb800d1e20 2023-11-26T07:18:00.286 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:00.278+0000 7fcea9d35640 1 -- 172.21.15.143:0/866294174 wait complete. 2023-11-26T07:18:00.286 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:00.278+0000 7fbb85b94640 1 -- 172.21.15.143:0/3359646625 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fbb800d1e20 msgr2=0x7fbb800c6df0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:18:00.286 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:00.278+0000 7fbb85b94640 1 --2- 172.21.15.143:0/3359646625 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fbb800d1e20 0x7fbb800c6df0 secure :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0x7fbb680098a0 tx=0x7fbb6801c450 comp rx=0 tx=0).stop 2023-11-26T07:18:00.286 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:00.278+0000 7f4c2f335640 1 -- 172.21.15.143:0/1455233037 shutdown_connections 2023-11-26T07:18:00.287 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:00.278+0000 7f4c2f335640 1 --2- 172.21.15.143:0/1455233037 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f4c280b2790 0x7f4c281592f0 unknown :-1 s=CLOSED pgs=33 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:18:00.287 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:00.278+0000 7f4c2f335640 1 -- 172.21.15.143:0/1455233037 >> 172.21.15.143:0/1455233037 conn(0x7f4c28071a90 msgr2=0x7f4c28071ea0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:18:00.287 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:00.278+0000 7fbb85b94640 1 -- 172.21.15.143:0/3359646625 shutdown_connections 2023-11-26T07:18:00.287 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:00.278+0000 7fbb85b94640 1 --2- 172.21.15.143:0/3359646625 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fbb800d1e20 0x7fbb800c6df0 unknown :-1 s=CLOSED pgs=34 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:18:00.287 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:00.278+0000 7fbb85b94640 1 -- 172.21.15.143:0/3359646625 >> 172.21.15.143:0/3359646625 conn(0x7fbb800d3030 msgr2=0x7fbb800d3410 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:18:00.287 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:00.278+0000 7fbb85b94640 1 -- 172.21.15.143:0/3359646625 shutdown_connections 2023-11-26T07:18:00.288 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:00.278+0000 7f4c2f335640 1 -- 172.21.15.143:0/1455233037 shutdown_connections 2023-11-26T07:18:00.288 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:00.278+0000 7fbb85b94640 1 -- 172.21.15.143:0/3359646625 wait complete. 2023-11-26T07:18:00.288 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:00.278+0000 7f4c2f335640 1 -- 172.21.15.143:0/1455233037 wait complete. 2023-11-26T07:18:00.289 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:00.278+0000 7fcea9d35640 1 Processor -- start 2023-11-26T07:18:00.289 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:00.282+0000 7fcea9d35640 1 -- start start 2023-11-26T07:18:00.289 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:00.282+0000 7fbb85b94640 1 Processor -- start 2023-11-26T07:18:00.289 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:00.282+0000 7fcea9d35640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fcea406d340 0x7fcea4160a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:18:00.289 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:00.282+0000 7fcea9d35640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fcea40b3f80 0x7fcea415f200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:18:00.290 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:00.282+0000 7fcea9d35640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fcea414afa0 0x7fcea415f740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:18:00.290 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:00.282+0000 7fcea9d35640 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcea41514e0 con 0x7fcea414afa0 2023-11-26T07:18:00.290 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:00.282+0000 7fcea9d35640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcea414e670 con 0x7fcea406d340 2023-11-26T07:18:00.290 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:00.282+0000 7fcea9d35640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcea415fde0 con 0x7fcea40b3f80 2023-11-26T07:18:00.290 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:00.282+0000 7fbb85b94640 1 -- start start 2023-11-26T07:18:00.290 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:00.282+0000 7f4c2f335640 1 Processor -- start 2023-11-26T07:18:00.291 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:00.282+0000 7fcea37fe640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fcea406d340 0x7fcea4160a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:18:00.291 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:00.282+0000 7fcea37fe640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fcea406d340 0x7fcea4160a10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.188:3300/0 says I am v2:172.21.15.143:45546/0 (socket says 172.21.15.143:45546) 2023-11-26T07:18:00.291 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:00.282+0000 7fcea37fe640 1 -- 172.21.15.143:0/4090735067 learned_addr learned my addr 172.21.15.143:0/4090735067 (peer_addr_for_me v2:172.21.15.143:0/0) 2023-11-26T07:18:00.291 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:00.282+0000 7f4c2f335640 1 -- start start 2023-11-26T07:18:00.292 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:00.282+0000 7fcea37fe640 1 -- 172.21.15.143:0/4090735067 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fcea40b3f80 msgr2=0x7fcea415f200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:18:00.292 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:00.282+0000 7fcea2ffd640 1 --2- 172.21.15.143:0/4090735067 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fcea40b3f80 0x7fcea415f200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:18:00.292 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:00.282+0000 7fcea37fe640 1 --2- 172.21.15.143:0/4090735067 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fcea40b3f80 0x7fcea415f200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:18:00.292 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:00.282+0000 7fcea37fe640 1 -- 172.21.15.143:0/4090735067 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fcea414afa0 msgr2=0x7fcea415f740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:18:00.292 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:00.282+0000 7fcea37fe640 1 --2- 172.21.15.143:0/4090735067 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fcea414afa0 0x7fcea415f740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:18:00.292 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:00.282+0000 7fcea37fe640 1 -- 172.21.15.143:0/4090735067 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fce9c009420 con 0x7fcea406d340 2023-11-26T07:18:00.292 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:00.282+0000 7fcea2ffd640 1 --2- 172.21.15.143:0/4090735067 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fcea40b3f80 0x7fcea415f200 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-26T07:18:00.292 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:00.282+0000 7fcea37fe640 1 --2- 172.21.15.143:0/4090735067 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fcea406d340 0x7fcea4160a10 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7fce9800b3e0 tx=0x7fce9800b8a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:18:00.293 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:00.282+0000 7fce8bfff640 1 -- 172.21.15.143:0/4090735067 <== mon.1 v2:172.21.15.188:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fce98011830 con 0x7fcea406d340 2023-11-26T07:18:00.293 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:00.282+0000 7fce8bfff640 1 -- 172.21.15.143:0/4090735067 <== mon.1 v2:172.21.15.188:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fce980119b0 con 0x7fcea406d340 2023-11-26T07:18:00.293 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:00.282+0000 7fce8bfff640 1 -- 172.21.15.143:0/4090735067 <== mon.1 v2:172.21.15.188:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fce98011d60 con 0x7fcea406d340 2023-11-26T07:18:00.293 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:00.282+0000 7fbb85b94640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fbb800cf9c0 0x7fbb800c5de0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:18:00.293 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:00.286+0000 7fbb85b94640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fbb800d1e20 0x7fbb800cb2d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:18:00.293 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:00.286+0000 7fbb85b94640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fbb80160d60 0x7fbb800b7910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:18:00.293 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:00.286+0000 7fbb85b94640 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbb801556f0 con 0x7fbb800cf9c0 2023-11-26T07:18:00.294 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:00.286+0000 7fbb85b94640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbb800ca230 con 0x7fbb800d1e20 2023-11-26T07:18:00.294 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:00.286+0000 7fbb85b94640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbb800ca510 con 0x7fbb80160d60 2023-11-26T07:18:00.294 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:00.286+0000 7fcea9d35640 1 -- 172.21.15.143:0/4090735067 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcea41600c0 con 0x7fcea406d340 2023-11-26T07:18:00.294 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:00.286+0000 7fcea9d35640 1 -- 172.21.15.143:0/4090735067 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcea414e8c0 con 0x7fcea406d340 2023-11-26T07:18:00.294 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:00.286+0000 7f4c2f335640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f4c28072620 0x7f4c282029b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:18:00.295 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:00.286+0000 7fbb7ffff640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fbb80160d60 0x7fbb800b7910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:18:00.295 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:00.286+0000 7fbb7ffff640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fbb80160d60 0x7fbb800b7910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.188:3301/0 says I am v2:172.21.15.143:45052/0 (socket says 172.21.15.143:45052) 2023-11-26T07:18:00.295 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:00.286+0000 7fbb7ffff640 1 -- 172.21.15.143:0/3220663692 learned_addr learned my addr 172.21.15.143:0/3220663692 (peer_addr_for_me v2:172.21.15.143:0/0) 2023-11-26T07:18:00.295 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:00.286+0000 7fbb77fff640 1 --2- 172.21.15.143:0/3220663692 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fbb800d1e20 0x7fbb800cb2d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:18:00.295 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:00.286+0000 7fbbe5957640 1 Processor -- start 2023-11-26T07:18:00.296 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:00.286+0000 7f4c2f335640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f4c280745d0 0x7f4c2814a700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:18:00.296 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:00.286+0000 7fbb7ffff640 1 -- 172.21.15.143:0/3220663692 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fbb800d1e20 msgr2=0x7fbb800cb2d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:18:00.296 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:00.286+0000 7fbb7ffff640 1 --2- 172.21.15.143:0/3220663692 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fbb800d1e20 0x7fbb800cb2d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:18:00.296 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:00.286+0000 7fbb7ffff640 1 -- 172.21.15.143:0/3220663692 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fbb800cf9c0 msgr2=0x7fbb800c5de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:18:00.297 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:00.286+0000 7fbb7ffff640 1 --2- 172.21.15.143:0/3220663692 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fbb800cf9c0 0x7fbb800c5de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:18:00.297 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:00.286+0000 7fbb7ffff640 1 -- 172.21.15.143:0/3220663692 --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbb680094d0 con 0x7fbb80160d60 2023-11-26T07:18:00.297 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:00.286+0000 7f4c2f335640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f4c280b2790 0x7f4c28205320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:18:00.297 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:00.286+0000 7fcea9d35640 1 -- 172.21.15.143:0/4090735067 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7fcea4067500 con 0x7fcea406d340 2023-11-26T07:18:00.297 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:00.286+0000 7fbbe5957640 1 -- start start 2023-11-26T07:18:00.298 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:00.286+0000 7f4c2d0d1640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f4c28072620 0x7f4c282029b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:18:00.298 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:00.286+0000 7f4c2d0d1640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f4c28072620 0x7f4c282029b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.188:3300/0 says I am v2:172.21.15.143:45552/0 (socket says 172.21.15.143:45552) 2023-11-26T07:18:00.298 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:00.286+0000 7f4c2c8d0640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f4c280745d0 0x7f4c2814a700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:18:00.298 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:00.286+0000 7fbb7ffff640 1 --2- 172.21.15.143:0/3220663692 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fbb80160d60 0x7fbb800b7910 secure :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0x7fbb7800c770 tx=0x7fbb7800cc30 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:18:00.299 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:00.286+0000 7f4c2c8d0640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f4c280745d0 0x7f4c2814a700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:46350/0 (socket says 172.21.15.143:46350) 2023-11-26T07:18:00.299 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:00.286+0000 7f4c2c8d0640 1 -- 172.21.15.143:0/1462997878 learned_addr learned my addr 172.21.15.143:0/1462997878 (peer_addr_for_me v2:172.21.15.143:0/0) 2023-11-26T07:18:00.299 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:00.286+0000 7f4c2d8d2640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f4c280b2790 0x7f4c28205320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:18:00.299 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:00.286+0000 7f4c2f335640 1 -- 172.21.15.143:0/1462997878 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4c28148940 con 0x7f4c280745d0 2023-11-26T07:18:00.299 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:00.286+0000 7f4c2f335640 1 -- 172.21.15.143:0/1462997878 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4c28202ef0 con 0x7f4c28072620 2023-11-26T07:18:00.299 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:00.286+0000 7f4c2f335640 1 -- 172.21.15.143:0/1462997878 --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4c28147cc0 con 0x7f4c280b2790 2023-11-26T07:18:00.300 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:00.286+0000 7fbb7cff9640 1 -- 172.21.15.143:0/3220663692 <== mon.2 v2:172.21.15.188:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbb78003b80 con 0x7fbb80160d60 2023-11-26T07:18:00.300 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:00.286+0000 7fbb7cff9640 1 -- 172.21.15.143:0/3220663692 <== mon.2 v2:172.21.15.188:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbb78003ce0 con 0x7fbb80160d60 2023-11-26T07:18:00.300 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:00.286+0000 7fbb7cff9640 1 -- 172.21.15.143:0/3220663692 <== mon.2 v2:172.21.15.188:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbb7800aa90 con 0x7fbb80160d60 2023-11-26T07:18:00.300 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:00.286+0000 7f4c2d8d2640 1 -- 172.21.15.143:0/1462997878 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f4c28072620 msgr2=0x7f4c282029b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:18:00.300 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:00.286+0000 7fbb85b94640 1 -- 172.21.15.143:0/3220663692 --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbb800ca7f0 con 0x7fbb80160d60 2023-11-26T07:18:00.300 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:00.286+0000 7fbb85b94640 1 -- 172.21.15.143:0/3220663692 --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbb800cadb0 con 0x7fbb80160d60 2023-11-26T07:18:00.301 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:00.286+0000 7fbbe5957640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fbbe0159170 0x7fbbe006d680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:18:00.301 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:00.286+0000 7f4c2d8d2640 1 --2- 172.21.15.143:0/1462997878 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f4c28072620 0x7f4c282029b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:18:00.301 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:00.286+0000 7f4c2d8d2640 1 -- 172.21.15.143:0/1462997878 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f4c280745d0 msgr2=0x7f4c2814a700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:18:00.301 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:00.286+0000 7f4c2d8d2640 1 --2- 172.21.15.143:0/1462997878 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f4c280745d0 0x7f4c2814a700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:18:00.301 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:00.286+0000 7f4c2d8d2640 1 -- 172.21.15.143:0/1462997878 --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4c1c009b30 con 0x7f4c280b2790 2023-11-26T07:18:00.302 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:00.286+0000 7fce8bfff640 1 -- 172.21.15.143:0/4090735067 <== mon.1 v2:172.21.15.188:3300/0 4 ==== mgrmap(e 5) v1 ==== 85852+0+0 (secure 0 0 0) 0x7fce9801a400 con 0x7fcea406d340 2023-11-26T07:18:00.302 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:00.286+0000 7f4c2d8d2640 1 --2- 172.21.15.143:0/1462997878 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f4c280b2790 0x7f4c28205320 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7f4c1400c3c0 tx=0x7f4c1400c880 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:18:00.302 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:00.286+0000 7fbbdeffd640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fbbe0159170 0x7fbbe006d680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:18:00.302 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:00.286+0000 7fbbe5957640 1 -- --> v1:172.21.15.143:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbbe006cf30 con 0x7fbbe00ce310 2023-11-26T07:18:00.302 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:00.286+0000 7fbbe5957640 1 -- --> v1:172.21.15.188:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbbe00cb2f0 con 0x7fbbe0070a80 2023-11-26T07:18:00.303 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:00.286+0000 7fbbdeffd640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fbbe0159170 0x7fbbe006d680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.188:3301/0 says I am v2:172.21.15.143:45066/0 (socket says 172.21.15.143:45066) 2023-11-26T07:18:00.303 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:00.290+0000 7fbbdeffd640 1 -- 172.21.15.143:0/993593581 learned_addr learned my addr 172.21.15.143:0/993593581 (peer_addr_for_me v2:172.21.15.143:0/0) 2023-11-26T07:18:00.303 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:00.290+0000 7fbbe5957640 1 -- 172.21.15.143:0/993593581 --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbbe00cb0e0 con 0x7fbbe0159170 2023-11-26T07:18:00.303 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:00.290+0000 7fbb85b94640 1 -- 172.21.15.143:0/3220663692 --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7fbb80067c10 con 0x7fbb80160d60 2023-11-26T07:18:00.303 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:00.290+0000 7f4c25ffb640 1 -- 172.21.15.143:0/1462997878 <== mon.2 v2:172.21.15.188:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4c140052d0 con 0x7f4c280b2790 2023-11-26T07:18:00.304 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:00.290+0000 7f4c25ffb640 1 -- 172.21.15.143:0/1462997878 <== mon.2 v2:172.21.15.188:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4c14005450 con 0x7f4c280b2790 2023-11-26T07:18:00.304 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:00.290+0000 7f4c25ffb640 1 -- 172.21.15.143:0/1462997878 <== mon.2 v2:172.21.15.188:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4c14007260 con 0x7f4c280b2790 2023-11-26T07:18:00.304 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:00.290+0000 7f4c2f335640 1 -- 172.21.15.143:0/1462997878 --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4c28147fa0 con 0x7f4c280b2790 2023-11-26T07:18:00.304 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:00.290+0000 7fce8bfff640 1 --2- 172.21.15.143:0/4090735067 >> [v2:172.21.15.143:6832/2150557544,v1:172.21.15.143:6833/2150557544] conn(0x7fce900664c0 0x7fce90068980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:18:00.304 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:00.290+0000 7fce8bfff640 1 -- 172.21.15.143:0/4090735067 <== mon.1 v2:172.21.15.188:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fce98083f40 con 0x7fcea406d340 2023-11-26T07:18:00.304 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:00.290+0000 7f4c2f335640 1 -- 172.21.15.143:0/1462997878 --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4c28148560 con 0x7f4c280b2790 2023-11-26T07:18:00.305 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:00.290+0000 7f262d9aa640 1 Processor -- start 2023-11-26T07:18:00.305 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:00.290+0000 7fbbddffb640 1 -- 172.21.15.143:0/993593581 <== mon.1 v1:172.21.15.188:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3552969499 0 0) 0x7fbbe00cb2f0 con 0x7fbbe0070a80 2023-11-26T07:18:00.305 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:00.290+0000 7fbbddffb640 1 -- 172.21.15.143:0/993593581 --> v1:172.21.15.188:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fbbc8003530 con 0x7fbbe0070a80 2023-11-26T07:18:00.305 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:00.290+0000 7fce8bfff640 1 --2- 172.21.15.143:0/4090735067 >> [v2:172.21.15.143:6817/639807369,v1:172.21.15.143:6819/639807369] conn(0x7fce9006e660 0x7fce90070a80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:18:00.305 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:00.290+0000 7fce8bfff640 1 -- 172.21.15.143:0/4090735067 --> [v2:172.21.15.143:6817/639807369,v1:172.21.15.143:6819/639807369] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7fce90071130 con 0x7fce9006e660 2023-11-26T07:18:00.306 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:00.290+0000 7fce8bfff640 1 -- 172.21.15.143:0/4090735067 <== mon.1 v2:172.21.15.188:3300/0 6 ==== mon_get_version_reply(handle=1 version=20) v2 ==== 24+0+0 (secure 0 0 0) 0x7fce98009210 con 0x7fcea406d340 2023-11-26T07:18:00.306 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:00.290+0000 7fbbdeffd640 1 -- 172.21.15.143:0/993593581 >> v1:172.21.15.188:6789/0 conn(0x7fbbe0070a80 legacy=0x7fbbe015cea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:18:00.306 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:00.290+0000 7fbbdeffd640 1 -- 172.21.15.143:0/993593581 >> v1:172.21.15.143:6789/0 conn(0x7fbbe00ce310 legacy=0x7fbbe0070370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:18:00.306 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:00.290+0000 7fbbdeffd640 1 -- 172.21.15.143:0/993593581 --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbbe00cb5d0 con 0x7fbbe0159170 2023-11-26T07:18:00.306 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:00.290+0000 7fcea2ffd640 1 --2- 172.21.15.143:0/4090735067 >> [v2:172.21.15.143:6832/2150557544,v1:172.21.15.143:6833/2150557544] conn(0x7fce900664c0 0x7fce90068980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:18:00.307 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:00.290+0000 7fcea3fff640 1 --2- 172.21.15.143:0/4090735067 >> [v2:172.21.15.143:6817/639807369,v1:172.21.15.143:6819/639807369] conn(0x7fce9006e660 0x7fce90070a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:18:00.307 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:00.290+0000 7f4c2f335640 1 -- 172.21.15.143:0/1462997878 --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f4c28067500 con 0x7f4c280b2790 2023-11-26T07:18:00.307 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:00.290+0000 7fbb7cff9640 1 -- 172.21.15.143:0/3220663692 <== mon.2 v2:172.21.15.188:3301/0 4 ==== mgrmap(e 5) v1 ==== 85852+0+0 (secure 0 0 0) 0x7fbb7800abf0 con 0x7fbb80160d60 2023-11-26T07:18:00.307 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:00.290+0000 7f262d9aa640 1 -- start start 2023-11-26T07:18:00.308 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:00.290+0000 7f55c0432640 1 Processor -- start 2023-11-26T07:18:00.308 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:00.290+0000 7f55c0432640 1 -- start start 2023-11-26T07:18:00.308 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:00.286+0000 7f447f52a640 1 Processor -- start 2023-11-26T07:18:00.308 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:00.290+0000 7f447f52a640 1 -- start start 2023-11-26T07:18:00.308 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:00.290+0000 7f447f52a640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f4478159380 0x7f4478072180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:18:00.308 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:00.290+0000 7f447d2c6640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f4478159380 0x7f4478072180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:18:00.308 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:00.290+0000 7f447d2c6640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f4478159380 0x7f4478072180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.188:3301/0 says I am v2:172.21.15.143:45080/0 (socket says 172.21.15.143:45080) 2023-11-26T07:18:00.309 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:00.290+0000 7f447dac7640 1 --1- >> v1:172.21.15.188:6789/0 conn(0x7f44780cfac0 0x7f44780cb9f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.188:6789/0 says I am v1:172.21.15.143:39964/0 (socket says 172.21.15.143:39964) 2023-11-26T07:18:00.309 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:00.290+0000 7f447d2c6640 1 -- 172.21.15.143:0/5125820 learned_addr learned my addr 172.21.15.143:0/5125820 (peer_addr_for_me v2:172.21.15.143:0/0) 2023-11-26T07:18:00.309 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:00.290+0000 7f447f52a640 1 -- 172.21.15.143:0/5125820 --> v1:172.21.15.143:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4478159760 con 0x7f44780ce950 2023-11-26T07:18:00.309 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:00.290+0000 7f447f52a640 1 -- 172.21.15.143:0/5125820 --> v1:172.21.15.188:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4478071cf0 con 0x7f44780cfac0 2023-11-26T07:18:00.309 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:00.290+0000 7f447f52a640 1 -- 172.21.15.143:0/5125820 --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f44780ce520 con 0x7f4478159380 2023-11-26T07:18:00.309 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:00.290+0000 7f446ffff640 1 -- 172.21.15.143:0/5125820 <== mon.1 v1:172.21.15.188:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3832453359 0 0) 0x7f4478071cf0 con 0x7f44780cfac0 2023-11-26T07:18:00.309 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:00.290+0000 7f446ffff640 1 -- 172.21.15.143:0/5125820 --> v1:172.21.15.188:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f4464003530 con 0x7f44780cfac0 2023-11-26T07:18:00.309 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:00.290+0000 7fcea3fff640 1 --2- 172.21.15.143:0/4090735067 >> [v2:172.21.15.143:6817/639807369,v1:172.21.15.143:6819/639807369] conn(0x7fce9006e660 0x7fce90070a80 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:18:00.309 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:00.294+0000 7fcea2ffd640 1 --2- 172.21.15.143:0/4090735067 >> [v2:172.21.15.143:6832/2150557544,v1:172.21.15.143:6833/2150557544] conn(0x7fce900664c0 0x7fce90068980 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7fcea4151800 tx=0x7fce94009040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:18:00.310 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:00.294+0000 7f262d9aa640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f26280ce950 0x7f2628072180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:18:00.310 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:00.294+0000 7f262d9aa640 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7f2628070a00 0x7f26280cf3b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:18:00.310 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:00.294+0000 7f262d9aa640 1 -- --> v1:172.21.15.188:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2628071cf0 con 0x7f262806e460 2023-11-26T07:18:00.310 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:00.294+0000 7f262d9aa640 1 -- --> v2:172.21.15.188:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2628070e90 con 0x7f2628070a00 2023-11-26T07:18:00.310 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:00.294+0000 7f262d9aa640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f26280ce520 con 0x7f26280ce950 2023-11-26T07:18:00.310 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:00.294+0000 7f446ffff640 1 -- 172.21.15.143:0/5125820 <== mon.1 v1:172.21.15.188:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2617234794 0 0) 0x7f4464003530 con 0x7f44780cfac0 2023-11-26T07:18:00.310 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:00.294+0000 7f446ffff640 1 -- 172.21.15.143:0/5125820 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f4478159380 msgr2=0x7f4478072180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-26T07:18:00.311 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:00.294+0000 7f446ffff640 1 --2- 172.21.15.143:0/5125820 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f4478159380 0x7f4478072180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:18:00.311 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:00.294+0000 7f446ffff640 1 -- 172.21.15.143:0/5125820 >> v1:172.21.15.143:6789/0 conn(0x7f44780ce950 legacy=0x7f44780cf3b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:18:00.311 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:00.294+0000 7f446ffff640 1 -- 172.21.15.143:0/5125820 --> v1:172.21.15.188:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f44780710e0 con 0x7f44780cfac0 2023-11-26T07:18:00.311 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:00.294+0000 7f446ffff640 1 -- 172.21.15.143:0/5125820 <== mon.1 v1:172.21.15.188:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2137882523 0 0) 0x7f4474001680 con 0x7f44780cfac0 2023-11-26T07:18:00.311 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:00.294+0000 7f446ffff640 1 -- 172.21.15.143:0/5125820 >> v1:172.21.15.188:6789/0 conn(0x7f44780cfac0 legacy=0x7f44780cb9f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:18:00.311 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:00.294+0000 7f446ffff640 1 --2- 172.21.15.143:0/5125820 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f4464003980 0x7f4464003da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:18:00.311 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:00.294+0000 7f446ffff640 1 --2- 172.21.15.143:0/5125820 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f44640047d0 0x7f4464004c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:18:00.311 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:00.294+0000 7f447d2c6640 1 --2- 172.21.15.143:0/5125820 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f4478159380 0x7f4478072180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-26T07:18:00.311 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:00.294+0000 7fce8bfff640 1 -- 172.21.15.143:0/4090735067 <== osd.0 v2:172.21.15.143:6817/639807369 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+26965 (crc 0 0 0) 0x7fce90071130 con 0x7fce9006e660 2023-11-26T07:18:00.312 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:00.294+0000 7f4c25ffb640 1 -- 172.21.15.143:0/1462997878 <== mon.2 v2:172.21.15.188:3301/0 4 ==== mgrmap(e 5) v1 ==== 85852+0+0 (secure 0 0 0) 0x7f4c140073c0 con 0x7f4c280b2790 2023-11-26T07:18:00.312 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:00.294+0000 7fbb7cff9640 1 --2- 172.21.15.143:0/3220663692 >> [v2:172.21.15.143:6832/2150557544,v1:172.21.15.143:6833/2150557544] conn(0x7fbb60066470 0x7fbb60068930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:18:00.312 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:00.294+0000 7fbb7cff9640 1 -- 172.21.15.143:0/3220663692 <== mon.2 v2:172.21.15.188:3301/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fbb78084840 con 0x7fbb80160d60 2023-11-26T07:18:00.312 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:00.294+0000 7fbb7cff9640 1 --2- 172.21.15.143:0/3220663692 >> [v2:172.21.15.143:6808/3909095186,v1:172.21.15.143:6809/3909095186] conn(0x7fbb6006e690 0x7fbb60070ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:18:00.312 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:00.294+0000 7fbb7f7fe640 1 --2- 172.21.15.143:0/3220663692 >> [v2:172.21.15.143:6832/2150557544,v1:172.21.15.143:6833/2150557544] conn(0x7fbb60066470 0x7fbb60068930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:18:00.312 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:00.294+0000 7f2626ffd640 1 --1- >> v1:172.21.15.188:6789/0 conn(0x7f262806e460 0x7f2628159120 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.188:6789/0 says I am v1:172.21.15.143:39968/0 (socket says 172.21.15.143:39968) 2023-11-26T07:18:00.313 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:00.294+0000 7f2626ffd640 1 -- 172.21.15.143:0/270683067 learned_addr learned my addr 172.21.15.143:0/270683067 (peer_addr_for_me v1:172.21.15.143:0/0) 2023-11-26T07:18:00.313 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:00.294+0000 7f26277fe640 1 --2- 172.21.15.143:0/270683067 >> v2:172.21.15.188:3300/0 conn(0x7f2628070a00 0x7f26280cf3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:18:00.313 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:00.294+0000 7f26267fc640 1 --2- 172.21.15.143:0/270683067 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f26280ce950 0x7f2628072180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:18:00.313 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:00.294+0000 7f26277fe640 1 -- 172.21.15.143:0/270683067 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f26280ce950 msgr2=0x7f2628072180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-26T07:18:00.313 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:00.294+0000 7f26277fe640 1 --2- 172.21.15.143:0/270683067 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f26280ce950 0x7f2628072180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:18:00.313 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:00.294+0000 7f26277fe640 1 -- 172.21.15.143:0/270683067 >> v1:172.21.15.188:6789/0 conn(0x7f262806e460 legacy=0x7f2628159120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:18:00.313 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:00.294+0000 7f55c0432640 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7f55b806d3b0 0x7f55b80b2e80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:18:00.313 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:00.294+0000 7f55c0432640 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7f55b80b33c0 0x7f55b816b310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:18:00.313 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:00.294+0000 7f55c0432640 1 -- --> v1:172.21.15.188:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f55b8070a80 con 0x7f55b80b2aa0 2023-11-26T07:18:00.314 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:00.294+0000 7f55c0432640 1 -- --> v2:172.21.15.143:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f55b8071d80 con 0x7f55b806d3b0 2023-11-26T07:18:00.314 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:00.294+0000 7f55c0432640 1 -- --> v2:172.21.15.188:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f55b8070ce0 con 0x7f55b80b33c0 2023-11-26T07:18:00.314 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:00.294+0000 7f55be1ce640 1 --1- >> v1:172.21.15.188:6789/0 conn(0x7f55b80b2aa0 0x7f55b8070370 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.188:6789/0 says I am v1:172.21.15.143:39974/0 (socket says 172.21.15.143:39974) 2023-11-26T07:18:00.314 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:00.294+0000 7f55be1ce640 1 -- 172.21.15.143:0/3475104171 learned_addr learned my addr 172.21.15.143:0/3475104171 (peer_addr_for_me v1:172.21.15.143:0/0) 2023-11-26T07:18:00.314 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:00.294+0000 7f55be9cf640 1 --2- 172.21.15.143:0/3475104171 >> v2:172.21.15.188:3300/0 conn(0x7f55b80b33c0 0x7f55b816b310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:18:00.314 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:00.294+0000 7f446ffff640 1 --2- 172.21.15.143:0/5125820 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f4478159380 0x7f4464005190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:18:00.314 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:00.294+0000 7f446ffff640 1 -- 172.21.15.143:0/5125820 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f44780ce520 con 0x7f4464003980 2023-11-26T07:18:00.315 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:00.294+0000 7f446ffff640 1 -- 172.21.15.143:0/5125820 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4464007ad0 con 0x7f44640047d0 2023-11-26T07:18:00.315 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:00.294+0000 7f446ffff640 1 -- 172.21.15.143:0/5125820 --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4464007c90 con 0x7f4478159380 2023-11-26T07:18:00.315 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:00.294+0000 7f447d2c6640 1 --2- 172.21.15.143:0/5125820 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f4464003980 0x7f4464003da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:18:00.315 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:00.294+0000 7f447dac7640 1 --2- 172.21.15.143:0/5125820 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f4478159380 0x7f4464005190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:18:00.315 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:00.294+0000 7f55bd1cc640 1 -- 172.21.15.143:0/3475104171 <== mon.1 v1:172.21.15.188:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1046073993 0 0) 0x7f55b8070a80 con 0x7f55b80b2aa0 2023-11-26T07:18:00.315 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:00.294+0000 7f55bd1cc640 1 -- 172.21.15.143:0/3475104171 --> v1:172.21.15.188:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f55a4003530 con 0x7f55b80b2aa0 2023-11-26T07:18:00.316 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:00.294+0000 7fbb7cff9640 1 -- 172.21.15.143:0/3220663692 --> [v2:172.21.15.143:6808/3909095186,v1:172.21.15.143:6809/3909095186] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7fbb60071160 con 0x7fbb6006e690 2023-11-26T07:18:00.316 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:00.294+0000 7fbb7cff9640 1 -- 172.21.15.143:0/3220663692 <== mon.2 v2:172.21.15.188:3301/0 6 ==== mon_get_version_reply(handle=1 version=20) v2 ==== 24+0+0 (secure 0 0 0) 0x7fbb78084bf0 con 0x7fbb80160d60 2023-11-26T07:18:00.316 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:00.294+0000 7f26277fe640 1 -- 172.21.15.143:0/270683067 --> v2:172.21.15.188:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f26280713c0 con 0x7f2628070a00 2023-11-26T07:18:00.316 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:00.294+0000 7fbb77fff640 1 --2- 172.21.15.143:0/3220663692 >> [v2:172.21.15.143:6808/3909095186,v1:172.21.15.143:6809/3909095186] conn(0x7fbb6006e690 0x7fbb60070ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:18:00.317 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:00.294+0000 7f55be9cf640 1 -- 172.21.15.143:0/3475104171 >> v2:172.21.15.143:3300/0 conn(0x7f55b806d3b0 msgr2=0x7f55b80b2e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-26T07:18:00.317 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:00.294+0000 7f55be9cf640 1 --2- 172.21.15.143:0/3475104171 >> v2:172.21.15.143:3300/0 conn(0x7f55b806d3b0 0x7f55b80b2e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:18:00.317 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:00.294+0000 7f55be9cf640 1 -- 172.21.15.143:0/3475104171 >> v1:172.21.15.188:6789/0 conn(0x7f55b80b2aa0 legacy=0x7f55b8070370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:18:00.317 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:00.294+0000 7f55be9cf640 1 -- 172.21.15.143:0/3475104171 --> v2:172.21.15.188:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f55b8070fc0 con 0x7f55b80b33c0 2023-11-26T07:18:00.317 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:00.294+0000 7f447dac7640 1 -- 172.21.15.143:0/5125820 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f44640047d0 msgr2=0x7f4464004c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-26T07:18:00.317 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:00.294+0000 7f447dac7640 1 --2- 172.21.15.143:0/5125820 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f44640047d0 0x7f4464004c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:18:00.317 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:00.294+0000 7f447dac7640 1 -- 172.21.15.143:0/5125820 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f4464003980 msgr2=0x7f4464003da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-26T07:18:00.318 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:00.294+0000 7f447dac7640 1 --2- 172.21.15.143:0/5125820 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f4464003980 0x7f4464003da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:18:00.318 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:00.294+0000 7f447dac7640 1 -- 172.21.15.143:0/5125820 --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4464003530 con 0x7f4478159380 2023-11-26T07:18:00.318 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:00.294+0000 7fbb7f7fe640 1 --2- 172.21.15.143:0/3220663692 >> [v2:172.21.15.143:6832/2150557544,v1:172.21.15.143:6833/2150557544] conn(0x7fbb60066470 0x7fbb60068930 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7fbb70007fd0 tx=0x7fbb70007940 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:18:00.318 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:00.294+0000 7fbb77fff640 1 --2- 172.21.15.143:0/3220663692 >> [v2:172.21.15.143:6808/3909095186,v1:172.21.15.143:6809/3909095186] conn(0x7fbb6006e690 0x7fbb60070ab0 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:18:00.318 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:00.298+0000 7fbbdeffd640 1 --2- 172.21.15.143:0/993593581 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fbbe0159170 0x7fbbe006d680 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7fbbd000cb90 tx=0x7fbbd001e890 comp rx=0 tx=0).ready entity=mon.2 client_cookie=979469da4313cfb0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:18:00.319 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:00.294+0000 7f4c25ffb640 1 --2- 172.21.15.143:0/1462997878 >> [v2:172.21.15.143:6832/2150557544,v1:172.21.15.143:6833/2150557544] conn(0x7f4c0c0664c0 0x7f4c0c068980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:18:00.319 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:00.294+0000 7f4c25ffb640 1 -- 172.21.15.143:0/1462997878 <== mon.2 v2:172.21.15.188:3301/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f4c14082dc0 con 0x7f4c280b2790 2023-11-26T07:18:00.319 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:00.298+0000 7f4c25ffb640 1 --2- 172.21.15.143:0/1462997878 >> [v2:172.21.15.143:6816/1144942530,v1:172.21.15.143:6818/1144942530] conn(0x7f4c0c06e6e0 0x7f4c0c070b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:18:00.319 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:00.298+0000 7f4c25ffb640 1 -- 172.21.15.143:0/1462997878 --> [v2:172.21.15.143:6816/1144942530,v1:172.21.15.143:6818/1144942530] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f4c0c0711b0 con 0x7f4c0c06e6e0 2023-11-26T07:18:00.319 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:00.298+0000 7f4c25ffb640 1 -- 172.21.15.143:0/1462997878 <== mon.2 v2:172.21.15.188:3301/0 6 ==== mon_get_version_reply(handle=1 version=20) v2 ==== 24+0+0 (secure 0 0 0) 0x7f4c14083140 con 0x7f4c280b2790 2023-11-26T07:18:00.320 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:00.298+0000 7f26267fc640 1 --2- 172.21.15.143:0/270683067 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f26280ce950 0x7f2628072180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-26T07:18:00.320 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:00.298+0000 7fbbddffb640 1 -- 172.21.15.143:0/993593581 <== mon.2 v2:172.21.15.188:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbbd000c2d0 con 0x7fbbe0159170 2023-11-26T07:18:00.320 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:00.298+0000 7fbbddffb640 1 -- 172.21.15.143:0/993593581 <== mon.2 v2:172.21.15.188:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbbd00a7700 con 0x7fbbe0159170 2023-11-26T07:18:00.320 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:00.298+0000 7fbbddffb640 1 -- 172.21.15.143:0/993593581 <== mon.2 v2:172.21.15.188:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbbd00a7a90 con 0x7fbbe0159170 2023-11-26T07:18:00.320 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:00.298+0000 7f4c2c8d0640 1 --2- 172.21.15.143:0/1462997878 >> [v2:172.21.15.143:6816/1144942530,v1:172.21.15.143:6818/1144942530] conn(0x7f4c0c06e6e0 0x7f4c0c070b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:18:00.320 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:00.298+0000 7f4c2d0d1640 1 --2- 172.21.15.143:0/1462997878 >> [v2:172.21.15.143:6832/2150557544,v1:172.21.15.143:6833/2150557544] conn(0x7f4c0c0664c0 0x7f4c0c068980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:18:00.321 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:00.298+0000 7fbb7cff9640 1 -- 172.21.15.143:0/3220663692 <== osd.2 v2:172.21.15.143:6808/3909095186 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+26965 (crc 0 0 0) 0x7fbb60071160 con 0x7fbb6006e690 2023-11-26T07:18:00.321 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:00.298+0000 7fbbe5957640 1 -- 172.21.15.143:0/993593581 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fbbe0159170 msgr2=0x7fbbe006d680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:18:00.321 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:00.298+0000 7fbbe5957640 1 --2- 172.21.15.143:0/993593581 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fbbe0159170 0x7fbbe006d680 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7fbbd000cb90 tx=0x7fbbd001e890 comp rx=0 tx=0).stop 2023-11-26T07:18:00.321 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:00.298+0000 7fbbe5957640 1 -- 172.21.15.143:0/993593581 shutdown_connections 2023-11-26T07:18:00.321 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:00.298+0000 7fbbe5957640 1 --2- 172.21.15.143:0/993593581 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fbbe0159170 0x7fbbe006d680 unknown :-1 s=CLOSED pgs=37 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:18:00.321 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:00.298+0000 7fbbe5957640 1 -- 172.21.15.143:0/993593581 >> 172.21.15.143:0/993593581 conn(0x7fbbe00b0bf0 msgr2=0x7fbbe00b1000 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:18:00.322 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:00.298+0000 7f4c2d0d1640 1 --2- 172.21.15.143:0/1462997878 >> [v2:172.21.15.143:6832/2150557544,v1:172.21.15.143:6833/2150557544] conn(0x7f4c0c0664c0 0x7f4c0c068980 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7f4c20007fd0 tx=0x7f4c20009040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:18:00.322 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:00.298+0000 7fbbe5957640 1 -- 172.21.15.143:0/993593581 shutdown_connections 2023-11-26T07:18:00.322 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:00.298+0000 7fbbe5957640 1 -- 172.21.15.143:0/993593581 wait complete. 2023-11-26T07:18:00.322 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:00.298+0000 7f8707d35640 1 Processor -- start 2023-11-26T07:18:00.323 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:00.298+0000 7f55be9cf640 1 --2- 172.21.15.143:0/3475104171 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f55b80b33c0 0x7f55b816b310 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7f55b80b18f0 tx=0x7f55b001c900 comp rx=0 tx=0).ready entity=mon.1 client_cookie=ffbfd9f49a01384b server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:18:00.323 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:00.298+0000 7f4c2c8d0640 1 --2- 172.21.15.143:0/1462997878 >> [v2:172.21.15.143:6816/1144942530,v1:172.21.15.143:6818/1144942530] conn(0x7f4c0c06e6e0 0x7f4c0c070b00 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:18:00.323 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:00.302+0000 7fbbe5957640 1 Processor -- start 2023-11-26T07:18:00.324 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:00.298+0000 7f8707d35640 1 -- start start 2023-11-26T07:18:00.324 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:00.302+0000 7f26277fe640 1 --2- 172.21.15.143:0/270683067 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f2628070a00 0x7f26280cf3b0 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7f261c0097c0 tx=0x7f261c01c3c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=6e99f36cc0129c8e server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:18:00.324 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:00.302+0000 7f55bd1cc640 1 -- 172.21.15.143:0/3475104171 <== mon.1 v2:172.21.15.188:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f55b00a59a0 con 0x7f55b80b33c0 2023-11-26T07:18:00.324 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:00.302+0000 7f55bd1cc640 1 -- 172.21.15.143:0/3475104171 <== mon.1 v2:172.21.15.188:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f55b00a5b00 con 0x7f55b80b33c0 2023-11-26T07:18:00.324 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:00.302+0000 7f55bd1cc640 1 -- 172.21.15.143:0/3475104171 <== mon.1 v2:172.21.15.188:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f55b00a5e90 con 0x7f55b80b33c0 2023-11-26T07:18:00.325 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:00.302+0000 7fbbe5957640 1 -- start start 2023-11-26T07:18:00.325 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:00.302+0000 7f2625ffb640 1 -- 172.21.15.143:0/270683067 <== mon.1 v2:172.21.15.188:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f261c0ab600 con 0x7f2628070a00 2023-11-26T07:18:00.325 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:00.302+0000 7f2625ffb640 1 -- 172.21.15.143:0/270683067 <== mon.1 v2:172.21.15.188:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f261c0a7050 con 0x7f2628070a00 2023-11-26T07:18:00.325 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:00.302+0000 7f2625ffb640 1 -- 172.21.15.143:0/270683067 <== mon.1 v2:172.21.15.188:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f261c0ab980 con 0x7f2628070a00 2023-11-26T07:18:00.326 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:00.302+0000 7f447dac7640 1 --2- 172.21.15.143:0/5125820 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f4478159380 0x7f4464005190 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7f447806d5a0 tx=0x7f4474027a00 comp rx=0 tx=0).ready entity=mon.2 client_cookie=a2b506082a6c8a66 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:18:00.326 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:00.302+0000 7f446ffff640 1 -- 172.21.15.143:0/5125820 <== mon.2 v2:172.21.15.188:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4474007270 con 0x7f4478159380 2023-11-26T07:18:00.326 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:00.302+0000 7f446ffff640 1 -- 172.21.15.143:0/5125820 <== mon.2 v2:172.21.15.188:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f44740073d0 con 0x7f4478159380 2023-11-26T07:18:00.326 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:00.302+0000 7f446ffff640 1 -- 172.21.15.143:0/5125820 <== mon.2 v2:172.21.15.188:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4474005820 con 0x7f4478159380 2023-11-26T07:18:00.326 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:00.302+0000 7f4c25ffb640 1 -- 172.21.15.143:0/1462997878 <== osd.1 v2:172.21.15.143:6816/1144942530 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+26965 (crc 0 0 0) 0x7f4c0c0711b0 con 0x7f4c0c06e6e0 2023-11-26T07:18:00.327 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:00.302+0000 7f55c0432640 1 -- 172.21.15.143:0/3475104171 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f55b80b33c0 msgr2=0x7f55b816b310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:18:00.327 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:00.302+0000 7f55c0432640 1 --2- 172.21.15.143:0/3475104171 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f55b80b33c0 0x7f55b816b310 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7f55b80b18f0 tx=0x7f55b001c900 comp rx=0 tx=0).stop 2023-11-26T07:18:00.327 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:00.302+0000 7f55c0432640 1 -- 172.21.15.143:0/3475104171 shutdown_connections 2023-11-26T07:18:00.327 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:00.302+0000 7f55c0432640 1 --2- 172.21.15.143:0/3475104171 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f55b80b33c0 0x7f55b816b310 unknown :-1 s=CLOSED pgs=67 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:18:00.327 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:00.302+0000 7f55c0432640 1 --2- 172.21.15.143:0/3475104171 >> v2:172.21.15.143:3300/0 conn(0x7f55b806d3b0 0x7f55b80b2e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:18:00.327 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:00.302+0000 7f55c0432640 1 -- 172.21.15.143:0/3475104171 >> 172.21.15.143:0/3475104171 conn(0x7f55b80b0bf0 msgr2=0x7f55b80b1000 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:18:00.327 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:00.302+0000 7f262d9aa640 1 -- 172.21.15.143:0/270683067 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f2628070a00 msgr2=0x7f26280cf3b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:18:00.327 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:00.302+0000 7f262d9aa640 1 --2- 172.21.15.143:0/270683067 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f2628070a00 0x7f26280cf3b0 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7f261c0097c0 tx=0x7f261c01c3c0 comp rx=0 tx=0).stop 2023-11-26T07:18:00.328 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:00.302+0000 7f55c0432640 1 -- 172.21.15.143:0/3475104171 shutdown_connections 2023-11-26T07:18:00.328 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:00.302+0000 7f447f52a640 1 -- 172.21.15.143:0/5125820 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f4478159380 msgr2=0x7f4464005190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:18:00.328 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:00.302+0000 7f447f52a640 1 --2- 172.21.15.143:0/5125820 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f4478159380 0x7f4464005190 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7f447806d5a0 tx=0x7f4474027a00 comp rx=0 tx=0).stop 2023-11-26T07:18:00.328 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:00.302+0000 7f447dac7640 1 -- 172.21.15.143:0/5125820 reap_dead start 2023-11-26T07:18:00.328 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:00.302+0000 7f55c0432640 1 -- 172.21.15.143:0/3475104171 wait complete. 2023-11-26T07:18:00.329 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:00.302+0000 7f262d9aa640 1 -- 172.21.15.143:0/270683067 shutdown_connections 2023-11-26T07:18:00.329 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:00.302+0000 7f262d9aa640 1 --2- 172.21.15.143:0/270683067 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f2628070a00 0x7f26280cf3b0 unknown :-1 s=CLOSED pgs=68 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:18:00.329 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:00.302+0000 7f262d9aa640 1 --2- 172.21.15.143:0/270683067 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f26280ce950 0x7f2628072180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:18:00.329 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:00.302+0000 7f262d9aa640 1 -- 172.21.15.143:0/270683067 >> 172.21.15.143:0/270683067 conn(0x7f262806a890 msgr2=0x7f262806aca0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:18:00.329 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:00.302+0000 7f262d9aa640 1 -- 172.21.15.143:0/270683067 shutdown_connections 2023-11-26T07:18:00.329 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:00.302+0000 7f262d9aa640 1 -- 172.21.15.143:0/270683067 wait complete. 2023-11-26T07:18:00.330 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:00.302+0000 7f8707d35640 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7f8700159170 0x7f870006d680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:18:00.330 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:00.302+0000 7f8707d35640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f87000ce310 0x7f8700070370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:18:00.330 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:00.302+0000 7f8707d35640 1 -- --> v1:172.21.15.188:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f870006cf30 con 0x7f87000708b0 2023-11-26T07:18:00.330 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:00.302+0000 7f8707d35640 1 -- --> v2:172.21.15.143:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f87000cb0e0 con 0x7f8700159170 2023-11-26T07:18:00.330 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:00.302+0000 7f8707d35640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f87000cb2f0 con 0x7f87000ce310 2023-11-26T07:18:00.330 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:00.302+0000 7f87052d0640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f87000ce310 0x7f8700070370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:18:00.330 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:00.302+0000 7f87052d0640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f87000ce310 0x7f8700070370 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.188:3301/0 says I am v2:172.21.15.143:45092/0 (socket says 172.21.15.143:45092) 2023-11-26T07:18:00.330 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:00.302+0000 7f87052d0640 1 -- 172.21.15.143:0/4293057541 learned_addr learned my addr 172.21.15.143:0/4293057541 (peer_addr_for_me v2:172.21.15.143:0/0) 2023-11-26T07:18:00.330 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:00.302+0000 7f87052d0640 1 -- 172.21.15.143:0/4293057541 >> v2:172.21.15.143:3300/0 conn(0x7f8700159170 msgr2=0x7f870006d680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-26T07:18:00.331 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:00.302+0000 7f87052d0640 1 --2- 172.21.15.143:0/4293057541 >> v2:172.21.15.143:3300/0 conn(0x7f8700159170 0x7f870006d680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:18:00.331 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:00.302+0000 7f87052d0640 1 -- 172.21.15.143:0/4293057541 >> v1:172.21.15.188:6789/0 conn(0x7f87000708b0 legacy=0x7f870015cea0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2023-11-26T07:18:00.331 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:00.302+0000 7f87052d0640 1 -- 172.21.15.143:0/4293057541 --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f87000cb5d0 con 0x7f87000ce310 2023-11-26T07:18:00.331 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:00.302+0000 7fbbe5957640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fbbe0070a80 0x7fbbe0146840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:18:00.331 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:00.302+0000 7fbbe5957640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fbbe00ce310 0x7fbbe014c150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:18:00.331 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:00.302+0000 7fbbe5957640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fbbe0159170 0x7fbbe014c690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:18:00.331 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:00.302+0000 7fbbe5957640 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbbe0145c10 con 0x7fbbe00ce310 2023-11-26T07:18:00.331 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:00.302+0000 7fbbe5957640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbbe014cda0 con 0x7fbbe0070a80 2023-11-26T07:18:00.331 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:00.302+0000 7fbbe5957640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbbe014aa40 con 0x7fbbe0159170 2023-11-26T07:18:00.332 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:00.306+0000 7f447f52a640 1 -- 172.21.15.143:0/5125820 shutdown_connections 2023-11-26T07:18:00.332 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:00.306+0000 7f447f52a640 1 -- 172.21.15.143:0/5125820 >> 172.21.15.143:0/5125820 conn(0x7f447806a890 msgr2=0x7f447806aca0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:18:00.332 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:00.302+0000 7fbbdf7fe640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fbbe0159170 0x7fbbe014c690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:18:00.332 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:00.302+0000 7fbbdf7fe640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fbbe0159170 0x7fbbe014c690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.188:3301/0 says I am v2:172.21.15.143:45094/0 (socket says 172.21.15.143:45094) 2023-11-26T07:18:00.332 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:00.306+0000 7fbbdf7fe640 1 -- 172.21.15.143:0/2843155256 learned_addr learned my addr 172.21.15.143:0/2843155256 (peer_addr_for_me v2:172.21.15.143:0/0) 2023-11-26T07:18:00.333 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:00.306+0000 7fbbdf7fe640 1 -- 172.21.15.143:0/2843155256 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fbbe0070a80 msgr2=0x7fbbe0146840 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-11-26T07:18:00.333 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:00.306+0000 7fbbdf7fe640 1 --2- 172.21.15.143:0/2843155256 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fbbe0070a80 0x7fbbe0146840 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:18:00.333 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:00.306+0000 7fbbdf7fe640 1 -- 172.21.15.143:0/2843155256 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fbbe00ce310 msgr2=0x7fbbe014c150 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-11-26T07:18:00.333 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:00.306+0000 7fbbdf7fe640 1 --2- 172.21.15.143:0/2843155256 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fbbe00ce310 0x7fbbe014c150 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:18:00.333 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:00.306+0000 7fbbdf7fe640 1 -- 172.21.15.143:0/2843155256 --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbbd000c7d0 con 0x7fbbe0159170 2023-11-26T07:18:00.333 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:00.306+0000 7f262d9aa640 1 Processor -- start 2023-11-26T07:18:00.333 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:00.306+0000 7f447f52a640 1 -- 172.21.15.143:0/5125820 shutdown_connections 2023-11-26T07:18:00.334 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:00.306+0000 7f447f52a640 1 -- 172.21.15.143:0/5125820 wait complete. 2023-11-26T07:18:00.334 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:00.306+0000 7f262d9aa640 1 -- start start 2023-11-26T07:18:00.334 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:00.306+0000 7fbbdf7fe640 1 --2- 172.21.15.143:0/2843155256 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fbbe0159170 0x7fbbe014c690 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7fbbd800c7b0 tx=0x7fbbd800cc70 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:18:00.335 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:00.306+0000 7f55c0432640 1 Processor -- start 2023-11-26T07:18:00.335 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:00.306+0000 7f55c0432640 1 -- start start 2023-11-26T07:18:00.335 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:00.306+0000 7fbbc7fff640 1 -- 172.21.15.143:0/2843155256 <== mon.2 v2:172.21.15.188:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbbd8003db0 con 0x7fbbe0159170 2023-11-26T07:18:00.335 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:00.306+0000 7fbbc7fff640 1 -- 172.21.15.143:0/2843155256 <== mon.2 v2:172.21.15.188:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbbd8005070 con 0x7fbbe0159170 2023-11-26T07:18:00.335 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:00.306+0000 7fbbc7fff640 1 -- 172.21.15.143:0/2843155256 <== mon.2 v2:172.21.15.188:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbbd8005400 con 0x7fbbe0159170 2023-11-26T07:18:00.336 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:00.306+0000 7f262d9aa640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f262806e460 0x7f262816ac20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:18:00.336 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:00.306+0000 7f262d9aa640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f2628070a00 0x7f26281407e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:18:00.336 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:00.306+0000 7f262d9aa640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f26280ce950 0x7f26281fab40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:18:00.336 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:00.306+0000 7f262d9aa640 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f26281434c0 con 0x7f2628070a00 2023-11-26T07:18:00.336 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:00.306+0000 7f262d9aa640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f262816a920 con 0x7f262806e460 2023-11-26T07:18:00.336 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:00.306+0000 7f262d9aa640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f26281425e0 con 0x7f26280ce950 2023-11-26T07:18:00.337 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:00.306+0000 7f87052d0640 1 --2- 172.21.15.143:0/4293057541 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f87000ce310 0x7f8700070370 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7f86f0009870 tx=0x7f86f001c440 comp rx=0 tx=0).ready entity=mon.2 client_cookie=97bbcac2750c8ae8 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:18:00.337 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:00.306+0000 7f8704acf640 1 -- 172.21.15.143:0/4293057541 <== mon.2 v2:172.21.15.188:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f86f00ab620 con 0x7f87000ce310 2023-11-26T07:18:00.337 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:00.310+0000 7f8704acf640 1 -- 172.21.15.143:0/4293057541 <== mon.2 v2:172.21.15.188:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f86f00ab780 con 0x7f87000ce310 2023-11-26T07:18:00.337 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:00.310+0000 7f8704acf640 1 -- 172.21.15.143:0/4293057541 <== mon.2 v2:172.21.15.188:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f86f00abb10 con 0x7f87000ce310 2023-11-26T07:18:00.337 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:00.310+0000 7fbbe5957640 1 -- 172.21.15.143:0/2843155256 --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbbe014ad20 con 0x7fbbe0159170 2023-11-26T07:18:00.337 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:00.310+0000 7f26267fc640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f2628070a00 0x7f26281407e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:18:00.338 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:00.310+0000 7f26267fc640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f2628070a00 0x7f26281407e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:46398/0 (socket says 172.21.15.143:46398) 2023-11-26T07:18:00.338 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:00.310+0000 7f26267fc640 1 -- 172.21.15.143:0/1619865464 learned_addr learned my addr 172.21.15.143:0/1619865464 (peer_addr_for_me v2:172.21.15.143:0/0) 2023-11-26T07:18:00.338 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:00.310+0000 7fbbe5957640 1 -- 172.21.15.143:0/2843155256 --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbbe0146d80 con 0x7fbbe0159170 2023-11-26T07:18:00.338 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:00.310+0000 7f8707d35640 1 -- 172.21.15.143:0/4293057541 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f87000ce310 msgr2=0x7f8700070370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:18:00.338 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:00.310+0000 7f8707d35640 1 --2- 172.21.15.143:0/4293057541 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f87000ce310 0x7f8700070370 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7f86f0009870 tx=0x7f86f001c440 comp rx=0 tx=0).stop 2023-11-26T07:18:00.339 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:00.310+0000 7f8707d35640 1 -- 172.21.15.143:0/4293057541 shutdown_connections 2023-11-26T07:18:00.339 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:00.310+0000 7f8707d35640 1 --2- 172.21.15.143:0/4293057541 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f87000ce310 0x7f8700070370 unknown :-1 s=CLOSED pgs=40 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:18:00.339 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:00.310+0000 7f8707d35640 1 --2- 172.21.15.143:0/4293057541 >> v2:172.21.15.143:3300/0 conn(0x7f8700159170 0x7f870006d680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:18:00.339 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:00.310+0000 7f8707d35640 1 -- 172.21.15.143:0/4293057541 >> 172.21.15.143:0/4293057541 conn(0x7f87000b0bf0 msgr2=0x7f87000b1000 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:18:00.339 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:00.310+0000 7f55c0432640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f55b806d3b0 0x7f55b8144be0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:18:00.340 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:00.310+0000 7f2626ffd640 1 --2- 172.21.15.143:0/1619865464 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f262806e460 0x7f262816ac20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:18:00.340 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:00.310+0000 7f26277fe640 1 --2- 172.21.15.143:0/1619865464 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f26280ce950 0x7f26281fab40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:18:00.340 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:00.310+0000 7f26277fe640 1 -- 172.21.15.143:0/1619865464 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f262806e460 msgr2=0x7f262816ac20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:18:00.340 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:00.310+0000 7f26277fe640 1 --2- 172.21.15.143:0/1619865464 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f262806e460 0x7f262816ac20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:18:00.340 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:00.310+0000 7f26277fe640 1 -- 172.21.15.143:0/1619865464 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f2628070a00 msgr2=0x7f26281407e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:18:00.340 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:00.310+0000 7f26277fe640 1 --2- 172.21.15.143:0/1619865464 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f2628070a00 0x7f26281407e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:18:00.341 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:00.310+0000 7f26277fe640 1 -- 172.21.15.143:0/1619865464 --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f261c009420 con 0x7f26280ce950 2023-11-26T07:18:00.341 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:00.310+0000 7f2626ffd640 1 --2- 172.21.15.143:0/1619865464 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f262806e460 0x7f262816ac20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-26T07:18:00.341 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:00.310+0000 7f26277fe640 1 --2- 172.21.15.143:0/1619865464 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f26280ce950 0x7f26281fab40 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7f262806d800 tx=0x7f261c019160 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:18:00.341 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:00.310+0000 7f260ffff640 1 -- 172.21.15.143:0/1619865464 <== mon.2 v2:172.21.15.188:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f261c0aa8f0 con 0x7f26280ce950 2023-11-26T07:18:00.341 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:00.310+0000 7f260ffff640 1 -- 172.21.15.143:0/1619865464 <== mon.2 v2:172.21.15.188:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f261c0aaa50 con 0x7f26280ce950 2023-11-26T07:18:00.341 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:00.310+0000 7f262d9aa640 1 -- 172.21.15.143:0/1619865464 --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2628142880 con 0x7f26280ce950 2023-11-26T07:18:00.341 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:00.310+0000 7f260ffff640 1 -- 172.21.15.143:0/1619865464 <== mon.2 v2:172.21.15.188:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f261c0aae00 con 0x7f26280ce950 2023-11-26T07:18:00.341 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:00.310+0000 7f262d9aa640 1 -- 172.21.15.143:0/1619865464 --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2628142e00 con 0x7f26280ce950 2023-11-26T07:18:00.341 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:00.310+0000 7f55be1ce640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f55b806d3b0 0x7f55b8144be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:18:00.342 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:00.310+0000 7f55be1ce640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f55b806d3b0 0x7f55b8144be0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:46408/0 (socket says 172.21.15.143:46408) 2023-11-26T07:18:00.342 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:00.310+0000 7f262d9aa640 1 -- 172.21.15.143:0/1619865464 --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f2608000fd0 con 0x7f26280ce950 2023-11-26T07:18:00.342 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:00.310+0000 7f447f52a640 1 Processor -- start 2023-11-26T07:18:00.342 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:00.310+0000 7f447f52a640 1 -- start start 2023-11-26T07:18:00.342 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:00.314+0000 7f55c0432640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f55b80b2aa0 0x7f55b81571c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:18:00.343 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:00.314+0000 7f55c0432640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f55b80b33c0 0x7f55b8157700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:18:00.343 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:00.314+0000 7f55c0432640 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f55b815a130 con 0x7f55b806d3b0 2023-11-26T07:18:00.343 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:00.314+0000 7f55c0432640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f55b8159520 con 0x7f55b80b2aa0 2023-11-26T07:18:00.343 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:00.314+0000 7f55c0432640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f55b8159800 con 0x7f55b80b33c0 2023-11-26T07:18:00.343 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:00.314+0000 7f8707d35640 1 -- 172.21.15.143:0/4293057541 shutdown_connections 2023-11-26T07:18:00.343 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:00.314+0000 7f8707d35640 1 -- 172.21.15.143:0/4293057541 wait complete. 2023-11-26T07:18:00.343 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:00.314+0000 7f55be1ce640 1 -- 172.21.15.143:0/978563229 learned_addr learned my addr 172.21.15.143:0/978563229 (peer_addr_for_me v2:172.21.15.143:0/0) 2023-11-26T07:18:00.344 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:00.310+0000 7fbbc7fff640 1 -- 172.21.15.143:0/2843155256 <== mon.2 v2:172.21.15.188:3301/0 4 ==== mgrmap(e 5) v1 ==== 85852+0+0 (secure 0 0 0) 0x7fbbd8005560 con 0x7fbbe0159170 2023-11-26T07:18:00.344 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:00.310+0000 7fbbe5957640 1 -- 172.21.15.143:0/2843155256 --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7fbbe0072340 con 0x7fbbe0159170 2023-11-26T07:18:00.344 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:00.314+0000 7fbbc7fff640 1 --2- 172.21.15.143:0/2843155256 >> [v2:172.21.15.143:6832/2150557544,v1:172.21.15.143:6833/2150557544] conn(0x7fbbcc0664f0 0x7fbbcc0689b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:18:00.344 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:00.314+0000 7fbbc7fff640 1 -- 172.21.15.143:0/2843155256 <== mon.2 v2:172.21.15.188:3301/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fbbd8083720 con 0x7fbbe0159170 2023-11-26T07:18:00.344 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:00.314+0000 7fbbc7fff640 1 --2- 172.21.15.143:0/2843155256 >> [v2:172.21.15.188:6808/2153499405,v1:172.21.15.188:6809/2153499405] conn(0x7fbbcc06e710 0x7fbbcc070b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:18:00.344 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:00.314+0000 7fbbc7fff640 1 -- 172.21.15.143:0/2843155256 --> [v2:172.21.15.188:6808/2153499405,v1:172.21.15.188:6809/2153499405] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7fbbcc0711e0 con 0x7fbbcc06e710 2023-11-26T07:18:00.344 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:00.314+0000 7fbbc7fff640 1 -- 172.21.15.143:0/2843155256 <== mon.2 v2:172.21.15.188:3301/0 6 ==== mon_get_version_reply(handle=1 version=20) v2 ==== 24+0+0 (secure 0 0 0) 0x7fbbd8085d00 con 0x7fbbe0159170 2023-11-26T07:18:00.345 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:00.314+0000 7f55be9cf640 1 --2- 172.21.15.143:0/978563229 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f55b80b33c0 0x7f55b8157700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:18:00.345 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:00.314+0000 7f55bd9cd640 1 --2- 172.21.15.143:0/978563229 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f55b80b2aa0 0x7f55b81571c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:18:00.345 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:00.314+0000 7f447f52a640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f4478159380 0x7f4478146f60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:18:00.345 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:00.314+0000 7f447f52a640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f44780aeb60 0x7f447815a340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:18:00.345 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:00.314+0000 7fbbde7fc640 1 --2- 172.21.15.143:0/2843155256 >> [v2:172.21.15.188:6808/2153499405,v1:172.21.15.188:6809/2153499405] conn(0x7fbbcc06e710 0x7fbbcc070b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:18:00.346 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:00.314+0000 7f55be1ce640 1 -- 172.21.15.143:0/978563229 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f55b80b33c0 msgr2=0x7f55b8157700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:18:00.346 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:00.314+0000 7f55be1ce640 1 --2- 172.21.15.143:0/978563229 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f55b80b33c0 0x7f55b8157700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:18:00.346 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:00.314+0000 7f55be1ce640 1 -- 172.21.15.143:0/978563229 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f55b80b2aa0 msgr2=0x7f55b81571c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:18:00.346 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:00.314+0000 7f55be1ce640 1 --2- 172.21.15.143:0/978563229 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f55b80b2aa0 0x7f55b81571c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:18:00.346 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:00.314+0000 7f55be1ce640 1 -- 172.21.15.143:0/978563229 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f55b0009a70 con 0x7f55b806d3b0 2023-11-26T07:18:00.346 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:00.314+0000 7f447f52a640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f44780b2540 0x7f44780b29d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:18:00.346 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:00.314+0000 7f447d2c6640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f4478159380 0x7f4478146f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:18:00.347 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:00.314+0000 7f8707d35640 1 Processor -- start 2023-11-26T07:18:00.347 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:00.314+0000 7f8707d35640 1 -- start start 2023-11-26T07:18:00.347 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:00.314+0000 7f55be1ce640 1 --2- 172.21.15.143:0/978563229 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f55b806d3b0 0x7f55b8144be0 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7f55b40010f0 tx=0x7f55b400c8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:18:00.347 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:00.314+0000 7fbbde7fc640 1 --2- 172.21.15.143:0/2843155256 >> [v2:172.21.15.188:6808/2153499405,v1:172.21.15.188:6809/2153499405] conn(0x7fbbcc06e710 0x7fbbcc070b30 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:18:00.348 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:00.318+0000 7f260ffff640 1 -- 172.21.15.143:0/1619865464 <== mon.2 v2:172.21.15.188:3301/0 4 ==== mgrmap(e 5) v1 ==== 85852+0+0 (secure 0 0 0) 0x7f261c0b2040 con 0x7f26280ce950 2023-11-26T07:18:00.348 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:00.314+0000 7f447d2c6640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f4478159380 0x7f4478146f60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:46418/0 (socket says 172.21.15.143:46418) 2023-11-26T07:18:00.348 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:00.318+0000 7fbbdeffd640 1 --2- 172.21.15.143:0/2843155256 >> [v2:172.21.15.143:6832/2150557544,v1:172.21.15.143:6833/2150557544] conn(0x7fbbcc0664f0 0x7fbbcc0689b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:18:00.348 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:00.318+0000 7f55aeffd640 1 -- 172.21.15.143:0/978563229 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f55b4005cf0 con 0x7f55b806d3b0 2023-11-26T07:18:00.349 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:00.314+0000 7f447cac5640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f44780aeb60 0x7f447815a340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:18:00.349 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:00.314+0000 7f447cac5640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f44780aeb60 0x7f447815a340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.188:3300/0 says I am v2:172.21.15.143:45612/0 (socket says 172.21.15.143:45612) 2023-11-26T07:18:00.349 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:00.314+0000 7f447d2c6640 1 -- 172.21.15.143:0/2503894875 learned_addr learned my addr 172.21.15.143:0/2503894875 (peer_addr_for_me v2:172.21.15.143:0/0) 2023-11-26T07:18:00.349 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:00.314+0000 7f447f52a640 1 -- 172.21.15.143:0/2503894875 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f44780b2f10 con 0x7f4478159380 2023-11-26T07:18:00.349 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:00.314+0000 7f447f52a640 1 -- 172.21.15.143:0/2503894875 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f44780b0100 con 0x7f44780aeb60 2023-11-26T07:18:00.349 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:00.314+0000 7f447f52a640 1 -- 172.21.15.143:0/2503894875 --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f44780afc70 con 0x7f44780b2540 2023-11-26T07:18:00.349 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:00.314+0000 7f447dac7640 1 --2- 172.21.15.143:0/2503894875 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f44780b2540 0x7f44780b29d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:18:00.350 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:00.318+0000 7f8707d35640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f87000708b0 0x7f8700149ec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:18:00.350 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:00.318+0000 7f8707d35640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f8700159170 0x7f870014aaf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:18:00.350 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:00.318+0000 7f8707d35640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f870014dba0 0x7f870014b8a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:18:00.350 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:00.318+0000 7f8707d35640 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f870014b4d0 con 0x7f87000708b0 2023-11-26T07:18:00.350 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:00.318+0000 7f8707d35640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f870014e050 con 0x7f8700159170 2023-11-26T07:18:00.350 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:00.318+0000 7f8707d35640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f870014b1f0 con 0x7f870014dba0 2023-11-26T07:18:00.350 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:00.314+0000 7f447cac5640 1 -- 172.21.15.143:0/2503894875 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f44780b2540 msgr2=0x7f44780b29d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:18:00.351 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:00.314+0000 7f447cac5640 1 --2- 172.21.15.143:0/2503894875 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f44780b2540 0x7f44780b29d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:18:00.351 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:00.314+0000 7f447cac5640 1 -- 172.21.15.143:0/2503894875 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f4478159380 msgr2=0x7f4478146f60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:18:00.351 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:00.314+0000 7f447cac5640 1 --2- 172.21.15.143:0/2503894875 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f4478159380 0x7f4478146f60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:18:00.351 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:00.314+0000 7f447cac5640 1 -- 172.21.15.143:0/2503894875 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f44780af060 con 0x7f44780aeb60 2023-11-26T07:18:00.351 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:00.318+0000 7f447d2c6640 1 --2- 172.21.15.143:0/2503894875 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f4478159380 0x7f4478146f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-26T07:18:00.351 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:00.318+0000 7f447cac5640 1 --2- 172.21.15.143:0/2503894875 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f44780aeb60 0x7f447815a340 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7f447000c760 tx=0x7f447000cc20 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:18:00.351 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:00.318+0000 7f260ffff640 1 --2- 172.21.15.143:0/1619865464 >> [v2:172.21.15.143:6832/2150557544,v1:172.21.15.143:6833/2150557544] conn(0x7f2614066470 0x7f2614068930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:18:00.351 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:00.318+0000 7f260ffff640 1 -- 172.21.15.143:0/1619865464 <== mon.2 v2:172.21.15.188:3301/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f261c11ec80 con 0x7f26280ce950 2023-11-26T07:18:00.352 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:00.318+0000 7f260ffff640 1 --2- 172.21.15.143:0/1619865464 >> [v2:172.21.15.143:6800/3854210991,v1:172.21.15.143:6801/3854210991] conn(0x7f261406e690 0x7f2614070ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:18:00.352 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:00.318+0000 7f260ffff640 1 -- 172.21.15.143:0/1619865464 --> [v2:172.21.15.143:6800/3854210991,v1:172.21.15.143:6801/3854210991] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f2614071160 con 0x7f261406e690 2023-11-26T07:18:00.352 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:00.318+0000 7f260ffff640 1 -- 172.21.15.143:0/1619865464 <== mon.2 v2:172.21.15.188:3301/0 6 ==== mon_get_version_reply(handle=1 version=20) v2 ==== 24+0+0 (secure 0 0 0) 0x7f261c11f030 con 0x7f26280ce950 2023-11-26T07:18:00.352 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:00.322+0000 7f2626ffd640 1 --2- 172.21.15.143:0/1619865464 >> [v2:172.21.15.143:6832/2150557544,v1:172.21.15.143:6833/2150557544] conn(0x7f2614066470 0x7f2614068930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:18:00.352 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:00.322+0000 7f26267fc640 1 --2- 172.21.15.143:0/1619865464 >> [v2:172.21.15.143:6800/3854210991,v1:172.21.15.143:6801/3854210991] conn(0x7f261406e690 0x7f2614070ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:18:00.352 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:00.318+0000 7f55c0432640 1 -- 172.21.15.143:0/978563229 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f55b8159ae0 con 0x7f55b806d3b0 2023-11-26T07:18:00.352 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:00.318+0000 7f55c0432640 1 -- 172.21.15.143:0/978563229 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f55b8145120 con 0x7f55b806d3b0 2023-11-26T07:18:00.352 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:00.318+0000 7f55aeffd640 1 -- 172.21.15.143:0/978563229 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f55b4005e70 con 0x7f55b806d3b0 2023-11-26T07:18:00.352 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:00.318+0000 7f55aeffd640 1 -- 172.21.15.143:0/978563229 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f55b40072d0 con 0x7f55b806d3b0 2023-11-26T07:18:00.352 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:00.318+0000 7f55acff9640 1 -- 172.21.15.143:0/978563229 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f5594000fd0 con 0x7f55b806d3b0 2023-11-26T07:18:00.353 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:00.318+0000 7f446dffb640 1 -- 172.21.15.143:0/2503894875 <== mon.1 v2:172.21.15.188:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4470003b80 con 0x7f44780aeb60 2023-11-26T07:18:00.353 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:00.318+0000 7f446dffb640 1 -- 172.21.15.143:0/2503894875 <== mon.1 v2:172.21.15.188:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4470003d00 con 0x7f44780aeb60 2023-11-26T07:18:00.353 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:00.318+0000 7f446dffb640 1 -- 172.21.15.143:0/2503894875 <== mon.1 v2:172.21.15.188:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f447000aa90 con 0x7f44780aeb60 2023-11-26T07:18:00.353 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:00.318+0000 7f447f52a640 1 -- 172.21.15.143:0/2503894875 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f44780af340 con 0x7f44780aeb60 2023-11-26T07:18:00.353 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:00.318+0000 7f447f52a640 1 -- 172.21.15.143:0/2503894875 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f44780af900 con 0x7f44780aeb60 2023-11-26T07:18:00.353 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:00.318+0000 7fbbdeffd640 1 --2- 172.21.15.143:0/2843155256 >> [v2:172.21.15.143:6832/2150557544,v1:172.21.15.143:6833/2150557544] conn(0x7fbbcc0664f0 0x7fbbcc0689b0 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7fbbd00b0000 tx=0x7fbbd00a9070 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:18:00.353 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:00.318+0000 7fbbc7fff640 1 -- 172.21.15.143:0/2843155256 <== osd.6 v2:172.21.15.188:6808/2153499405 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+26965 (crc 0 0 0) 0x7fbbcc0711e0 con 0x7fbbcc06e710 2023-11-26T07:18:00.354 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:00.318+0000 7f87062d2640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f870014dba0 0x7f870014b8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:18:00.354 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:00.318+0000 7f87062d2640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f870014dba0 0x7f870014b8a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.188:3301/0 says I am v2:172.21.15.143:45136/0 (socket says 172.21.15.143:45136) 2023-11-26T07:18:00.354 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:00.318+0000 7f87062d2640 1 -- 172.21.15.143:0/3010467576 learned_addr learned my addr 172.21.15.143:0/3010467576 (peer_addr_for_me v2:172.21.15.143:0/0) 2023-11-26T07:18:00.354 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:00.318+0000 7f8705ad1640 1 --2- 172.21.15.143:0/3010467576 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f87000708b0 0x7f8700149ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:18:00.354 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:00.318+0000 7f87052d0640 1 --2- 172.21.15.143:0/3010467576 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f8700159170 0x7f870014aaf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:18:00.354 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:00.318+0000 7f87062d2640 1 -- 172.21.15.143:0/3010467576 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f8700159170 msgr2=0x7f870014aaf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:18:00.354 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:00.318+0000 7f87062d2640 1 --2- 172.21.15.143:0/3010467576 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f8700159170 0x7f870014aaf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:18:00.354 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:00.318+0000 7f87062d2640 1 -- 172.21.15.143:0/3010467576 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f87000708b0 msgr2=0x7f8700149ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:18:00.354 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:00.318+0000 7f87062d2640 1 --2- 172.21.15.143:0/3010467576 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f87000708b0 0x7f8700149ec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:18:00.355 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:00.318+0000 7f87062d2640 1 -- 172.21.15.143:0/3010467576 --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f86f00094d0 con 0x7f870014dba0 2023-11-26T07:18:00.355 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:00.318+0000 7f87062d2640 1 --2- 172.21.15.143:0/3010467576 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f870014dba0 0x7f870014b8a0 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7f86f800b9c0 tx=0x7f86f800be80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:18:00.355 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:00.318+0000 7f86f67fc640 1 -- 172.21.15.143:0/3010467576 <== mon.2 v2:172.21.15.188:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f86f800cba0 con 0x7f870014dba0 2023-11-26T07:18:00.355 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:00.318+0000 7f86f67fc640 1 -- 172.21.15.143:0/3010467576 <== mon.2 v2:172.21.15.188:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f86f800cd00 con 0x7f870014dba0 2023-11-26T07:18:00.355 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:00.318+0000 7f86f67fc640 1 -- 172.21.15.143:0/3010467576 <== mon.2 v2:172.21.15.188:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f86f8014680 con 0x7f870014dba0 2023-11-26T07:18:00.355 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:00.322+0000 7f447f52a640 1 -- 172.21.15.143:0/2503894875 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f4478071980 con 0x7f44780aeb60 2023-11-26T07:18:00.355 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:00.322+0000 7f8707d35640 1 -- 172.21.15.143:0/3010467576 --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8700149400 con 0x7f870014dba0 2023-11-26T07:18:00.356 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:00.322+0000 7f8707d35640 1 -- 172.21.15.143:0/3010467576 --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f87001499c0 con 0x7f870014dba0 2023-11-26T07:18:00.356 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:00.322+0000 7f26267fc640 1 --2- 172.21.15.143:0/1619865464 >> [v2:172.21.15.143:6800/3854210991,v1:172.21.15.143:6801/3854210991] conn(0x7f261406e690 0x7f2614070ab0 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:18:00.356 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:00.322+0000 7f2626ffd640 1 --2- 172.21.15.143:0/1619865464 >> [v2:172.21.15.143:6832/2150557544,v1:172.21.15.143:6833/2150557544] conn(0x7f2614066470 0x7f2614068930 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7f2620000f90 tx=0x7f262000c190 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:18:00.356 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:00.322+0000 7f446dffb640 1 -- 172.21.15.143:0/2503894875 <== mon.1 v2:172.21.15.188:3300/0 4 ==== mgrmap(e 5) v1 ==== 85852+0+0 (secure 0 0 0) 0x7f447000abf0 con 0x7f44780aeb60 2023-11-26T07:18:00.356 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:00.326+0000 7f8707d35640 1 -- 172.21.15.143:0/3010467576 --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f8700070d40 con 0x7f870014dba0 2023-11-26T07:18:00.357 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:00.326+0000 7f86f67fc640 1 -- 172.21.15.143:0/3010467576 <== mon.2 v2:172.21.15.188:3301/0 4 ==== mgrmap(e 5) v1 ==== 85852+0+0 (secure 0 0 0) 0x7f86f801d3b0 con 0x7f870014dba0 2023-11-26T07:18:00.357 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:00.326+0000 7fbb85b94640 1 -- 172.21.15.143:0/3220663692 --> [v2:172.21.15.143:6808/3909095186,v1:172.21.15.143:6809/3909095186] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7fbb800cafa0 con 0x7fbb6006e690 2023-11-26T07:18:00.357 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:00.326+0000 7f446dffb640 1 --2- 172.21.15.143:0/2503894875 >> [v2:172.21.15.143:6832/2150557544,v1:172.21.15.143:6833/2150557544] conn(0x7f445c066570 0x7f445c068a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:18:00.357 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:00.326+0000 7f260ffff640 1 -- 172.21.15.143:0/1619865464 <== osd.3 v2:172.21.15.143:6800/3854210991 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+26965 (crc 0 0 0) 0x7f2614071160 con 0x7f261406e690 2023-11-26T07:18:00.358 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:00.326+0000 7f4c2f335640 1 -- 172.21.15.143:0/1462997878 --> [v2:172.21.15.143:6816/1144942530,v1:172.21.15.143:6818/1144942530] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7f4c28148a80 con 0x7f4c0c06e6e0 2023-11-26T07:18:00.358 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:00.326+0000 7f446dffb640 1 -- 172.21.15.143:0/2503894875 <== mon.1 v2:172.21.15.188:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f4470083ac0 con 0x7f44780aeb60 2023-11-26T07:18:00.358 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:00.326+0000 7f446dffb640 1 --2- 172.21.15.143:0/2503894875 >> [v2:172.21.15.188:6813/186841544,v1:172.21.15.188:6815/186841544] conn(0x7f445c06e790 0x7f445c070bb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:18:00.358 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:00.326+0000 7f446dffb640 1 -- 172.21.15.143:0/2503894875 --> [v2:172.21.15.188:6813/186841544,v1:172.21.15.188:6815/186841544] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f445c071260 con 0x7f445c06e790 2023-11-26T07:18:00.358 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:00.326+0000 7f446dffb640 1 -- 172.21.15.143:0/2503894875 <== mon.1 v2:172.21.15.188:3300/0 6 ==== mon_get_version_reply(handle=1 version=20) v2 ==== 24+0+0 (secure 0 0 0) 0x7f44700882a0 con 0x7f44780aeb60 2023-11-26T07:18:00.359 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:00.326+0000 7f86f67fc640 1 --2- 172.21.15.143:0/3010467576 >> [v2:172.21.15.143:6832/2150557544,v1:172.21.15.143:6833/2150557544] conn(0x7f86e4066470 0x7f86e4068930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:18:00.359 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:00.326+0000 7fcea9d35640 1 -- 172.21.15.143:0/4090735067 --> [v2:172.21.15.143:6817/639807369,v1:172.21.15.143:6819/639807369] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7fcea414eae0 con 0x7fce9006e660 2023-11-26T07:18:00.359 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:00.326+0000 7f4c25ffb640 1 -- 172.21.15.143:0/1462997878 <== osd.1 v2:172.21.15.143:6816/1144942530 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+11 (crc 0 0 0) 0x7f4c28148a80 con 0x7f4c0c06e6e0 2023-11-26T07:18:00.359 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:00.326+0000 7f4c2f335640 1 -- 172.21.15.143:0/1462997878 >> [v2:172.21.15.143:6816/1144942530,v1:172.21.15.143:6818/1144942530] conn(0x7f4c0c06e6e0 msgr2=0x7f4c0c070b00 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:18:00.360 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:00.326+0000 7fbb7cff9640 1 -- 172.21.15.143:0/3220663692 <== osd.2 v2:172.21.15.143:6808/3909095186 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+11 (crc 0 0 0) 0x7fbb800cafa0 con 0x7fbb6006e690 2023-11-26T07:18:00.360 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:00.326+0000 7fbb85b94640 1 -- 172.21.15.143:0/3220663692 >> [v2:172.21.15.143:6808/3909095186,v1:172.21.15.143:6809/3909095186] conn(0x7fbb6006e690 msgr2=0x7fbb60070ab0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:18:00.360 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:00.326+0000 7fbb85b94640 1 --2- 172.21.15.143:0/3220663692 >> [v2:172.21.15.143:6808/3909095186,v1:172.21.15.143:6809/3909095186] conn(0x7fbb6006e690 0x7fbb60070ab0 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:18:00.360 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:00.326+0000 7fbb85b94640 1 -- 172.21.15.143:0/3220663692 >> [v2:172.21.15.143:6832/2150557544,v1:172.21.15.143:6833/2150557544] conn(0x7fbb60066470 msgr2=0x7fbb60068930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:18:00.360 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:00.326+0000 7fbb85b94640 1 --2- 172.21.15.143:0/3220663692 >> [v2:172.21.15.143:6832/2150557544,v1:172.21.15.143:6833/2150557544] conn(0x7fbb60066470 0x7fbb60068930 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7fbb70007fd0 tx=0x7fbb70007940 comp rx=0 tx=0).stop 2023-11-26T07:18:00.360 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:00.326+0000 7fbb85b94640 1 -- 172.21.15.143:0/3220663692 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fbb80160d60 msgr2=0x7fbb800b7910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:18:00.360 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:00.326+0000 7fbb85b94640 1 --2- 172.21.15.143:0/3220663692 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fbb80160d60 0x7fbb800b7910 secure :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0x7fbb7800c770 tx=0x7fbb7800cc30 comp rx=0 tx=0).stop 2023-11-26T07:18:00.360 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:00.326+0000 7fbb7ffff640 1 -- 172.21.15.143:0/3220663692 reap_dead start 2023-11-26T07:18:00.361 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:00.326+0000 7f447d2c6640 1 --2- 172.21.15.143:0/2503894875 >> [v2:172.21.15.143:6832/2150557544,v1:172.21.15.143:6833/2150557544] conn(0x7f445c066570 0x7f445c068a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:18:00.361 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:00.326+0000 7f447dac7640 1 --2- 172.21.15.143:0/2503894875 >> [v2:172.21.15.188:6813/186841544,v1:172.21.15.188:6815/186841544] conn(0x7f445c06e790 0x7f445c070bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:18:00.361 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:00.326+0000 7f86f67fc640 1 -- 172.21.15.143:0/3010467576 <== mon.2 v2:172.21.15.188:3301/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f86f8083e20 con 0x7f870014dba0 2023-11-26T07:18:00.361 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:00.326+0000 7f86f67fc640 1 --2- 172.21.15.143:0/3010467576 >> [v2:172.21.15.188:6824/2722096729,v1:172.21.15.188:6825/2722096729] conn(0x7f86e406e690 0x7f86e4070ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:18:00.361 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:00.326+0000 7f86f67fc640 1 -- 172.21.15.143:0/3010467576 --> [v2:172.21.15.188:6824/2722096729,v1:172.21.15.188:6825/2722096729] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f86e4071160 con 0x7f86e406e690 2023-11-26T07:18:00.361 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:00.326+0000 7f86f67fc640 1 -- 172.21.15.143:0/3010467576 <== mon.2 v2:172.21.15.188:3301/0 6 ==== mon_get_version_reply(handle=1 version=20) v2 ==== 24+0+0 (secure 0 0 0) 0x7f86f80893b0 con 0x7f870014dba0 2023-11-26T07:18:00.362 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:00.326+0000 7f4c2f335640 1 --2- 172.21.15.143:0/1462997878 >> [v2:172.21.15.143:6816/1144942530,v1:172.21.15.143:6818/1144942530] conn(0x7f4c0c06e6e0 0x7f4c0c070b00 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:18:00.362 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:00.326+0000 7f4c2f335640 1 -- 172.21.15.143:0/1462997878 >> [v2:172.21.15.143:6832/2150557544,v1:172.21.15.143:6833/2150557544] conn(0x7f4c0c0664c0 msgr2=0x7f4c0c068980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:18:00.362 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:00.326+0000 7f4c2f335640 1 --2- 172.21.15.143:0/1462997878 >> [v2:172.21.15.143:6832/2150557544,v1:172.21.15.143:6833/2150557544] conn(0x7f4c0c0664c0 0x7f4c0c068980 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7f4c20007fd0 tx=0x7f4c20009040 comp rx=0 tx=0).stop 2023-11-26T07:18:00.362 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:00.326+0000 7f4c2f335640 1 -- 172.21.15.143:0/1462997878 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f4c280b2790 msgr2=0x7f4c28205320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:18:00.362 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:00.326+0000 7f4c2f335640 1 --2- 172.21.15.143:0/1462997878 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f4c280b2790 0x7f4c28205320 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7f4c1400c3c0 tx=0x7f4c1400c880 comp rx=0 tx=0).stop 2023-11-26T07:18:00.362 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:00.326+0000 7f4c2d8d2640 1 -- 172.21.15.143:0/1462997878 reap_dead start 2023-11-26T07:18:00.362 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:00.326+0000 7f8705ad1640 1 --2- 172.21.15.143:0/3010467576 >> [v2:172.21.15.143:6832/2150557544,v1:172.21.15.143:6833/2150557544] conn(0x7f86e4066470 0x7f86e4068930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:18:00.363 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:00.326+0000 7f4c2f335640 1 -- 172.21.15.143:0/1462997878 shutdown_connections 2023-11-26T07:18:00.363 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:00.326+0000 7f4c2f335640 1 -- 172.21.15.143:0/1462997878 >> 172.21.15.143:0/1462997878 conn(0x7f4c28071a90 msgr2=0x7f4c28157690 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:18:00.363 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:00.326+0000 7f55aeffd640 1 -- 172.21.15.143:0/978563229 <== mon.0 v2:172.21.15.143:3300/0 4 ==== mgrmap(e 5) v1 ==== 85852+0+0 (secure 0 0 0) 0x7f55b4007430 con 0x7f55b806d3b0 2023-11-26T07:18:00.363 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:00.326+0000 7fbb85b94640 1 -- 172.21.15.143:0/3220663692 shutdown_connections 2023-11-26T07:18:00.363 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:00.326+0000 7fbb85b94640 1 -- 172.21.15.143:0/3220663692 >> 172.21.15.143:0/3220663692 conn(0x7fbb800d3030 msgr2=0x7fbb800d3410 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:18:00.364 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:00.330+0000 7f4c2f335640 1 -- 172.21.15.143:0/1462997878 shutdown_connections 2023-11-26T07:18:00.364 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:00.330+0000 7f87052d0640 1 --2- 172.21.15.143:0/3010467576 >> [v2:172.21.15.188:6824/2722096729,v1:172.21.15.188:6825/2722096729] conn(0x7f86e406e690 0x7f86e4070ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:18:00.364 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:00.330+0000 7fbb85b94640 1 -- 172.21.15.143:0/3220663692 shutdown_connections 2023-11-26T07:18:00.364 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:00.330+0000 7fce8bfff640 1 -- 172.21.15.143:0/4090735067 <== osd.0 v2:172.21.15.143:6817/639807369 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+11 (crc 0 0 0) 0x7fcea414eae0 con 0x7fce9006e660 2023-11-26T07:18:00.365 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:00.330+0000 7fbb85b94640 1 -- 172.21.15.143:0/3220663692 wait complete. 2023-11-26T07:18:00.365 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:00.330+0000 7f8705ad1640 1 --2- 172.21.15.143:0/3010467576 >> [v2:172.21.15.143:6832/2150557544,v1:172.21.15.143:6833/2150557544] conn(0x7f86e4066470 0x7f86e4068930 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7f86fc009990 tx=0x7f86fc009110 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:18:00.366 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:00.330+0000 7fcea9d35640 1 -- 172.21.15.143:0/4090735067 >> [v2:172.21.15.143:6817/639807369,v1:172.21.15.143:6819/639807369] conn(0x7fce9006e660 msgr2=0x7fce90070a80 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:18:00.366 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:00.330+0000 7fcea9d35640 1 --2- 172.21.15.143:0/4090735067 >> [v2:172.21.15.143:6817/639807369,v1:172.21.15.143:6819/639807369] conn(0x7fce9006e660 0x7fce90070a80 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:18:00.366 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:00.330+0000 7fcea9d35640 1 -- 172.21.15.143:0/4090735067 >> [v2:172.21.15.143:6832/2150557544,v1:172.21.15.143:6833/2150557544] conn(0x7fce900664c0 msgr2=0x7fce90068980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:18:00.366 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:00.330+0000 7fcea9d35640 1 --2- 172.21.15.143:0/4090735067 >> [v2:172.21.15.143:6832/2150557544,v1:172.21.15.143:6833/2150557544] conn(0x7fce900664c0 0x7fce90068980 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7fcea4151800 tx=0x7fce94009040 comp rx=0 tx=0).stop 2023-11-26T07:18:00.366 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:00.330+0000 7fcea9d35640 1 -- 172.21.15.143:0/4090735067 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fcea406d340 msgr2=0x7fcea4160a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:18:00.366 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:00.330+0000 7fcea9d35640 1 --2- 172.21.15.143:0/4090735067 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fcea406d340 0x7fcea4160a10 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7fce9800b3e0 tx=0x7fce9800b8a0 comp rx=0 tx=0).stop 2023-11-26T07:18:00.366 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:00.330+0000 7f87052d0640 1 --2- 172.21.15.143:0/3010467576 >> [v2:172.21.15.188:6824/2722096729,v1:172.21.15.188:6825/2722096729] conn(0x7f86e406e690 0x7f86e4070ab0 crc :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.7 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:18:00.367 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:00.330+0000 7fcea3fff640 1 -- 172.21.15.143:0/4090735067 reap_dead start 2023-11-26T07:18:00.367 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:00.330+0000 7fcea9d35640 1 -- 172.21.15.143:0/4090735067 shutdown_connections 2023-11-26T07:18:00.367 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:00.330+0000 7fcea9d35640 1 -- 172.21.15.143:0/4090735067 >> 172.21.15.143:0/4090735067 conn(0x7fcea40aea10 msgr2=0x7fcea4071270 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:18:00.367 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:00.330+0000 7f447dac7640 1 --2- 172.21.15.143:0/2503894875 >> [v2:172.21.15.188:6813/186841544,v1:172.21.15.188:6815/186841544] conn(0x7f445c06e790 0x7f445c070bb0 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.5 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:18:00.367 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:00.330+0000 7f55aeffd640 1 --2- 172.21.15.143:0/978563229 >> [v2:172.21.15.143:6832/2150557544,v1:172.21.15.143:6833/2150557544] conn(0x7f559c0664a0 0x7f559c068960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:18:00.368 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:00.330+0000 7fcea9d35640 1 -- 172.21.15.143:0/4090735067 shutdown_connections 2023-11-26T07:18:00.368 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:00.330+0000 7fcea9d35640 1 -- 172.21.15.143:0/4090735067 wait complete. 2023-11-26T07:18:00.368 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:00.330+0000 7f86f67fc640 1 -- 172.21.15.143:0/3010467576 <== osd.7 v2:172.21.15.188:6824/2722096729 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+26965 (crc 0 0 0) 0x7f86e4071160 con 0x7f86e406e690 2023-11-26T07:18:00.368 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:00.330+0000 7f55aeffd640 1 -- 172.21.15.143:0/978563229 <== mon.0 v2:172.21.15.143:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f55b4083ca0 con 0x7f55b806d3b0 2023-11-26T07:18:00.369 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:00.330+0000 7f447d2c6640 1 --2- 172.21.15.143:0/2503894875 >> [v2:172.21.15.143:6832/2150557544,v1:172.21.15.143:6833/2150557544] conn(0x7f445c066570 0x7f445c068a30 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7f4468006cd0 tx=0x7f4468008040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:18:00.369 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:00.330+0000 7f55aeffd640 1 --2- 172.21.15.143:0/978563229 >> [v2:172.21.15.188:6800/3924606902,v1:172.21.15.188:6801/3924606902] conn(0x7f559c06e6c0 0x7f559c070ae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:18:00.369 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:00.330+0000 7f55aeffd640 1 -- 172.21.15.143:0/978563229 --> [v2:172.21.15.188:6800/3924606902,v1:172.21.15.188:6801/3924606902] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f559c071190 con 0x7f559c06e6c0 2023-11-26T07:18:00.369 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:00.330+0000 7f55aeffd640 1 -- 172.21.15.143:0/978563229 <== mon.0 v2:172.21.15.143:3300/0 6 ==== mon_get_version_reply(handle=1 version=20) v2 ==== 24+0+0 (secure 0 0 0) 0x7f55b4084050 con 0x7f55b806d3b0 2023-11-26T07:18:00.369 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:00.330+0000 7f446dffb640 1 -- 172.21.15.143:0/2503894875 <== osd.5 v2:172.21.15.188:6813/186841544 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+26965 (crc 0 0 0) 0x7f445c071260 con 0x7f445c06e790 2023-11-26T07:18:00.370 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:00.330+0000 7f55bd9cd640 1 --2- 172.21.15.143:0/978563229 >> [v2:172.21.15.143:6832/2150557544,v1:172.21.15.143:6833/2150557544] conn(0x7f559c0664a0 0x7f559c068960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:18:00.370 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:00.330+0000 7f4c2f335640 1 -- 172.21.15.143:0/1462997878 wait complete. 2023-11-26T07:18:00.370 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:00.334+0000 7f55bd9cd640 1 --2- 172.21.15.143:0/978563229 >> [v2:172.21.15.143:6832/2150557544,v1:172.21.15.143:6833/2150557544] conn(0x7f559c0664a0 0x7f559c068960 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7f55a8006fd0 tx=0x7f55a8008040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:18:00.371 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:00.334+0000 7f55be9cf640 1 --2- 172.21.15.143:0/978563229 >> [v2:172.21.15.188:6800/3924606902,v1:172.21.15.188:6801/3924606902] conn(0x7f559c06e6c0 0x7f559c070ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:18:00.371 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:00.338+0000 7f55be9cf640 1 --2- 172.21.15.143:0/978563229 >> [v2:172.21.15.188:6800/3924606902,v1:172.21.15.188:6801/3924606902] conn(0x7f559c06e6c0 0x7f559c070ae0 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.4 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:18:00.371 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:00.338+0000 7f55aeffd640 1 -- 172.21.15.143:0/978563229 <== osd.4 v2:172.21.15.188:6800/3924606902 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+26965 (crc 0 0 0) 0x7f559c071190 con 0x7f559c06e6c0 2023-11-26T07:18:00.371 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:00.354+0000 7fbbe5957640 1 -- 172.21.15.143:0/2843155256 --> [v2:172.21.15.188:6808/2153499405,v1:172.21.15.188:6809/2153499405] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7fbbe014a8c0 con 0x7fbbcc06e710 2023-11-26T07:18:00.371 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:00.354+0000 7f262d9aa640 1 -- 172.21.15.143:0/1619865464 --> [v2:172.21.15.143:6800/3854210991,v1:172.21.15.143:6801/3854210991] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7f2608002cc0 con 0x7f261406e690 2023-11-26T07:18:00.371 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:00.354+0000 7f260ffff640 1 -- 172.21.15.143:0/1619865464 <== osd.3 v2:172.21.15.143:6800/3854210991 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+11 (crc 0 0 0) 0x7f2608002cc0 con 0x7f261406e690 2023-11-26T07:18:00.372 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:00.354+0000 7f262d9aa640 1 -- 172.21.15.143:0/1619865464 >> [v2:172.21.15.143:6800/3854210991,v1:172.21.15.143:6801/3854210991] conn(0x7f261406e690 msgr2=0x7f2614070ab0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:18:00.372 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:00.354+0000 7f262d9aa640 1 --2- 172.21.15.143:0/1619865464 >> [v2:172.21.15.143:6800/3854210991,v1:172.21.15.143:6801/3854210991] conn(0x7f261406e690 0x7f2614070ab0 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:18:00.372 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:00.358+0000 7f262d9aa640 1 -- 172.21.15.143:0/1619865464 >> [v2:172.21.15.143:6832/2150557544,v1:172.21.15.143:6833/2150557544] conn(0x7f2614066470 msgr2=0x7f2614068930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:18:00.372 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:00.358+0000 7f262d9aa640 1 --2- 172.21.15.143:0/1619865464 >> [v2:172.21.15.143:6832/2150557544,v1:172.21.15.143:6833/2150557544] conn(0x7f2614066470 0x7f2614068930 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7f2620000f90 tx=0x7f262000c190 comp rx=0 tx=0).stop 2023-11-26T07:18:00.372 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:00.358+0000 7f262d9aa640 1 -- 172.21.15.143:0/1619865464 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f26280ce950 msgr2=0x7f26281fab40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:18:00.373 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:00.358+0000 7f262d9aa640 1 --2- 172.21.15.143:0/1619865464 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f26280ce950 0x7f26281fab40 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7f262806d800 tx=0x7f261c019160 comp rx=0 tx=0).stop 2023-11-26T07:18:00.373 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:00.358+0000 7f26277fe640 1 -- 172.21.15.143:0/1619865464 reap_dead start 2023-11-26T07:18:00.373 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:00.358+0000 7f262d9aa640 1 -- 172.21.15.143:0/1619865464 shutdown_connections 2023-11-26T07:18:00.373 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:00.358+0000 7f262d9aa640 1 -- 172.21.15.143:0/1619865464 >> 172.21.15.143:0/1619865464 conn(0x7f262806a890 msgr2=0x7f2628146f70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:18:00.373 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:00.358+0000 7fbbc7fff640 1 -- 172.21.15.143:0/2843155256 <== osd.6 v2:172.21.15.188:6808/2153499405 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+11 (crc 0 0 0) 0x7fbbd40067e0 con 0x7fbbcc06e710 2023-11-26T07:18:00.373 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:00.358+0000 7f262d9aa640 1 -- 172.21.15.143:0/1619865464 shutdown_connections 2023-11-26T07:18:00.373 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:00.358+0000 7f262d9aa640 1 -- 172.21.15.143:0/1619865464 wait complete. 2023-11-26T07:18:00.374 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:00.358+0000 7fbbe5957640 1 -- 172.21.15.143:0/2843155256 >> [v2:172.21.15.188:6808/2153499405,v1:172.21.15.188:6809/2153499405] conn(0x7fbbcc06e710 msgr2=0x7fbbcc070b30 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:18:00.374 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:00.358+0000 7fbbe5957640 1 --2- 172.21.15.143:0/2843155256 >> [v2:172.21.15.188:6808/2153499405,v1:172.21.15.188:6809/2153499405] conn(0x7fbbcc06e710 0x7fbbcc070b30 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:18:00.374 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:00.362+0000 7fbbe5957640 1 -- 172.21.15.143:0/2843155256 >> [v2:172.21.15.143:6832/2150557544,v1:172.21.15.143:6833/2150557544] conn(0x7fbbcc0664f0 msgr2=0x7fbbcc0689b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:18:00.374 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:00.362+0000 7fbbe5957640 1 --2- 172.21.15.143:0/2843155256 >> [v2:172.21.15.143:6832/2150557544,v1:172.21.15.143:6833/2150557544] conn(0x7fbbcc0664f0 0x7fbbcc0689b0 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7fbbd00b0000 tx=0x7fbbd00a9070 comp rx=0 tx=0).stop 2023-11-26T07:18:00.374 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:00.362+0000 7fbbe5957640 1 -- 172.21.15.143:0/2843155256 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fbbe0159170 msgr2=0x7fbbe014c690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:18:00.375 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:00.362+0000 7fbbe5957640 1 --2- 172.21.15.143:0/2843155256 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fbbe0159170 0x7fbbe014c690 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7fbbd800c7b0 tx=0x7fbbd800cc70 comp rx=0 tx=0).stop 2023-11-26T07:18:00.375 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:00.362+0000 7fbbdf7fe640 1 -- 172.21.15.143:0/2843155256 reap_dead start 2023-11-26T07:18:00.375 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:00.362+0000 7fbbe5957640 1 -- 172.21.15.143:0/2843155256 shutdown_connections 2023-11-26T07:18:00.375 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:00.362+0000 7fbbe5957640 1 -- 172.21.15.143:0/2843155256 >> 172.21.15.143:0/2843155256 conn(0x7fbbe00b0bf0 msgr2=0x7fbbe00afc80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:18:00.375 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:00.362+0000 7f8707d35640 1 -- 172.21.15.143:0/3010467576 --> [v2:172.21.15.188:6824/2722096729,v1:172.21.15.188:6825/2722096729] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7f8700067680 con 0x7f86e406e690 2023-11-26T07:18:00.375 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:00.362+0000 7f447f52a640 1 -- 172.21.15.143:0/2503894875 --> [v2:172.21.15.188:6813/186841544,v1:172.21.15.188:6815/186841544] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7f44780716a0 con 0x7f445c06e790 2023-11-26T07:18:00.376 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:00.362+0000 7f86f67fc640 1 -- 172.21.15.143:0/3010467576 <== osd.7 v2:172.21.15.188:6824/2722096729 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+11 (crc 0 0 0) 0x7f8700067680 con 0x7f86e406e690 2023-11-26T07:18:00.376 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:00.362+0000 7f446dffb640 1 -- 172.21.15.143:0/2503894875 <== osd.5 v2:172.21.15.188:6813/186841544 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+11 (crc 0 0 0) 0x7f4474007540 con 0x7f445c06e790 2023-11-26T07:18:00.376 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:00.362+0000 7f8707d35640 1 -- 172.21.15.143:0/3010467576 >> [v2:172.21.15.188:6824/2722096729,v1:172.21.15.188:6825/2722096729] conn(0x7f86e406e690 msgr2=0x7f86e4070ab0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:18:00.376 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:00.362+0000 7f8707d35640 1 --2- 172.21.15.143:0/3010467576 >> [v2:172.21.15.188:6824/2722096729,v1:172.21.15.188:6825/2722096729] conn(0x7f86e406e690 0x7f86e4070ab0 crc :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:18:00.377 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:00.362+0000 7f8707d35640 1 -- 172.21.15.143:0/3010467576 >> [v2:172.21.15.143:6832/2150557544,v1:172.21.15.143:6833/2150557544] conn(0x7f86e4066470 msgr2=0x7f86e4068930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:18:00.377 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:00.362+0000 7f8707d35640 1 --2- 172.21.15.143:0/3010467576 >> [v2:172.21.15.143:6832/2150557544,v1:172.21.15.143:6833/2150557544] conn(0x7f86e4066470 0x7f86e4068930 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7f86fc009990 tx=0x7f86fc009110 comp rx=0 tx=0).stop 2023-11-26T07:18:00.377 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:00.362+0000 7f8707d35640 1 -- 172.21.15.143:0/3010467576 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f870014dba0 msgr2=0x7f870014b8a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:18:00.377 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:00.362+0000 7f8707d35640 1 --2- 172.21.15.143:0/3010467576 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f870014dba0 0x7f870014b8a0 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7f86f800b9c0 tx=0x7f86f800be80 comp rx=0 tx=0).stop 2023-11-26T07:18:00.377 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:00.362+0000 7f87062d2640 1 -- 172.21.15.143:0/3010467576 reap_dead start 2023-11-26T07:18:00.377 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:00.362+0000 7f8707d35640 1 -- 172.21.15.143:0/3010467576 shutdown_connections 2023-11-26T07:18:00.377 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:00.362+0000 7f8707d35640 1 -- 172.21.15.143:0/3010467576 >> 172.21.15.143:0/3010467576 conn(0x7f87000b0bf0 msgr2=0x7f87000b0110 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:18:00.378 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:00.362+0000 7f447f52a640 1 -- 172.21.15.143:0/2503894875 >> [v2:172.21.15.188:6813/186841544,v1:172.21.15.188:6815/186841544] conn(0x7f445c06e790 msgr2=0x7f445c070bb0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:18:00.378 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:00.362+0000 7f447f52a640 1 --2- 172.21.15.143:0/2503894875 >> [v2:172.21.15.188:6813/186841544,v1:172.21.15.188:6815/186841544] conn(0x7f445c06e790 0x7f445c070bb0 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:18:00.378 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:00.362+0000 7f8707d35640 1 -- 172.21.15.143:0/3010467576 shutdown_connections 2023-11-26T07:18:00.378 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:00.362+0000 7f447f52a640 1 -- 172.21.15.143:0/2503894875 >> [v2:172.21.15.143:6832/2150557544,v1:172.21.15.143:6833/2150557544] conn(0x7f445c066570 msgr2=0x7f445c068a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:18:00.379 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:00.362+0000 7f447f52a640 1 --2- 172.21.15.143:0/2503894875 >> [v2:172.21.15.143:6832/2150557544,v1:172.21.15.143:6833/2150557544] conn(0x7f445c066570 0x7f445c068a30 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7f4468006cd0 tx=0x7f4468008040 comp rx=0 tx=0).stop 2023-11-26T07:18:00.379 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:00.362+0000 7f447f52a640 1 -- 172.21.15.143:0/2503894875 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f44780aeb60 msgr2=0x7f447815a340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:18:00.379 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:00.362+0000 7f447f52a640 1 --2- 172.21.15.143:0/2503894875 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f44780aeb60 0x7f447815a340 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7f447000c760 tx=0x7f447000cc20 comp rx=0 tx=0).stop 2023-11-26T07:18:00.379 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:00.362+0000 7f8707d35640 1 -- 172.21.15.143:0/3010467576 wait complete. 2023-11-26T07:18:00.380 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:00.362+0000 7f447dac7640 1 -- 172.21.15.143:0/2503894875 reap_dead start 2023-11-26T07:18:00.380 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:00.362+0000 7f447f52a640 1 -- 172.21.15.143:0/2503894875 shutdown_connections 2023-11-26T07:18:00.380 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:00.362+0000 7f447f52a640 1 -- 172.21.15.143:0/2503894875 >> 172.21.15.143:0/2503894875 conn(0x7f447806a890 msgr2=0x7f44781566f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:18:00.380 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:00.362+0000 7f447f52a640 1 -- 172.21.15.143:0/2503894875 shutdown_connections 2023-11-26T07:18:00.380 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:00.362+0000 7f447f52a640 1 -- 172.21.15.143:0/2503894875 wait complete. 2023-11-26T07:18:00.380 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:00.366+0000 7fbbe5957640 1 -- 172.21.15.143:0/2843155256 shutdown_connections 2023-11-26T07:18:00.380 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:00.366+0000 7fbbe5957640 1 -- 172.21.15.143:0/2843155256 wait complete. 2023-11-26T07:18:00.381 INFO:teuthology.orchestra.run.smithi143.stdout:55834574856 2023-11-26T07:18:00.381 DEBUG:teuthology.orchestra.run.smithi143:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.2 2023-11-26T07:18:00.382 INFO:teuthology.orchestra.run.smithi143.stdout:55834574855 2023-11-26T07:18:00.382 DEBUG:teuthology.orchestra.run.smithi143:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.1 2023-11-26T07:18:00.383 INFO:teuthology.orchestra.run.smithi143.stdout:55834574855 2023-11-26T07:18:00.383 DEBUG:teuthology.orchestra.run.smithi143:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.0 2023-11-26T07:18:00.384 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:00.370+0000 7f55acff9640 1 -- 172.21.15.143:0/978563229 --> [v2:172.21.15.188:6800/3924606902,v1:172.21.15.188:6801/3924606902] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7f5594002cc0 con 0x7f559c06e6c0 2023-11-26T07:18:00.384 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:00.370+0000 7f55aeffd640 1 -- 172.21.15.143:0/978563229 <== osd.4 v2:172.21.15.188:6800/3924606902 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+11 (crc 0 0 0) 0x7f5594002cc0 con 0x7f559c06e6c0 2023-11-26T07:18:00.385 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:00.370+0000 7f55acff9640 1 -- 172.21.15.143:0/978563229 >> [v2:172.21.15.188:6800/3924606902,v1:172.21.15.188:6801/3924606902] conn(0x7f559c06e6c0 msgr2=0x7f559c070ae0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:18:00.385 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:00.370+0000 7f55acff9640 1 --2- 172.21.15.143:0/978563229 >> [v2:172.21.15.188:6800/3924606902,v1:172.21.15.188:6801/3924606902] conn(0x7f559c06e6c0 0x7f559c070ae0 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:18:00.385 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:00.370+0000 7f55acff9640 1 -- 172.21.15.143:0/978563229 >> [v2:172.21.15.143:6832/2150557544,v1:172.21.15.143:6833/2150557544] conn(0x7f559c0664a0 msgr2=0x7f559c068960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:18:00.385 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:00.370+0000 7f55acff9640 1 --2- 172.21.15.143:0/978563229 >> [v2:172.21.15.143:6832/2150557544,v1:172.21.15.143:6833/2150557544] conn(0x7f559c0664a0 0x7f559c068960 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7f55a8006fd0 tx=0x7f55a8008040 comp rx=0 tx=0).stop 2023-11-26T07:18:00.385 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:00.370+0000 7f55acff9640 1 -- 172.21.15.143:0/978563229 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f55b806d3b0 msgr2=0x7f55b8144be0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:18:00.385 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:00.370+0000 7f55acff9640 1 --2- 172.21.15.143:0/978563229 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f55b806d3b0 0x7f55b8144be0 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7f55b40010f0 tx=0x7f55b400c8f0 comp rx=0 tx=0).stop 2023-11-26T07:18:00.386 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:00.370+0000 7f55be9cf640 1 -- 172.21.15.143:0/978563229 reap_dead start 2023-11-26T07:18:00.386 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:00.374+0000 7f55acff9640 1 -- 172.21.15.143:0/978563229 shutdown_connections 2023-11-26T07:18:00.386 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:00.374+0000 7f55acff9640 1 -- 172.21.15.143:0/978563229 >> 172.21.15.143:0/978563229 conn(0x7f55b80b0bf0 msgr2=0x7f55b81485b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:18:00.386 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:00.374+0000 7f55acff9640 1 -- 172.21.15.143:0/978563229 shutdown_connections 2023-11-26T07:18:00.386 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:00.374+0000 7f55acff9640 1 -- 172.21.15.143:0/978563229 wait complete. 2023-11-26T07:18:00.398 INFO:teuthology.orchestra.run.smithi143.stdout:55834574856 2023-11-26T07:18:00.398 DEBUG:teuthology.orchestra.run.smithi143:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.3 2023-11-26T07:18:00.406 INFO:teuthology.orchestra.run.smithi143.stdout:55834574855 2023-11-26T07:18:00.407 INFO:teuthology.orchestra.run.smithi143.stdout:55834574855 2023-11-26T07:18:00.407 DEBUG:teuthology.orchestra.run.smithi143:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.4 2023-11-26T07:18:00.408 DEBUG:teuthology.orchestra.run.smithi143:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.6 2023-11-26T07:18:00.413 INFO:teuthology.orchestra.run.smithi143.stdout:55834574853 2023-11-26T07:18:00.414 INFO:teuthology.orchestra.run.smithi143.stdout:55834574853 2023-11-26T07:18:00.414 DEBUG:teuthology.orchestra.run.smithi143:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.5 2023-11-26T07:18:00.415 DEBUG:teuthology.orchestra.run.smithi143:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.7 2023-11-26T07:18:00.615 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:00.610+0000 7fe18046b640 1 Processor -- start 2023-11-26T07:18:00.615 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:00.610+0000 7fe18046b640 1 -- start start 2023-11-26T07:18:00.617 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:00.614+0000 7fe18046b640 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7fe1780ce310 0x7fe178070370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:18:00.617 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:00.614+0000 7fe18046b640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fe1780708b0 0x7fe17815cea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:18:00.618 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:00.614+0000 7fe18046b640 1 -- --> v1:172.21.15.143:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe17806cf30 con 0x7fe178159170 2023-11-26T07:18:00.618 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:00.614+0000 7fe17da06640 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7fe1780ce310 0x7fe178070370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:18:00.618 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:00.614+0000 7fe17da06640 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7fe1780ce310 0x7fe178070370 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:46438/0 (socket says 172.21.15.143:46438) 2023-11-26T07:18:00.618 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:00.614+0000 7fe17da06640 1 -- 172.21.15.143:0/2383465475 learned_addr learned my addr 172.21.15.143:0/2383465475 (peer_addr_for_me v2:172.21.15.143:0/0) 2023-11-26T07:18:00.618 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:00.614+0000 7fe17ea08640 1 --2- 172.21.15.143:0/2383465475 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fe1780708b0 0x7fe17815cea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:18:00.619 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:00.614+0000 7fe18046b640 1 -- 172.21.15.143:0/2383465475 --> v2:172.21.15.143:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe1780cb0e0 con 0x7fe1780ce310 2023-11-26T07:18:00.619 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:00.614+0000 7fe18046b640 1 -- 172.21.15.143:0/2383465475 --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe1780cb2f0 con 0x7fe1780708b0 2023-11-26T07:18:00.619 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:00.614+0000 7fe17da06640 1 -- 172.21.15.143:0/2383465475 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fe1780708b0 msgr2=0x7fe17815cea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-26T07:18:00.620 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:00.614+0000 7fe17da06640 1 --2- 172.21.15.143:0/2383465475 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fe1780708b0 0x7fe17815cea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:18:00.620 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:00.614+0000 7fe17da06640 1 -- 172.21.15.143:0/2383465475 >> v1:172.21.15.143:6789/0 conn(0x7fe178159170 legacy=0x7fe17806d680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:18:00.620 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:00.614+0000 7fe17da06640 1 -- 172.21.15.143:0/2383465475 --> v2:172.21.15.143:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe1780cb5d0 con 0x7fe1780ce310 2023-11-26T07:18:00.621 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:00.618+0000 7f7f6a2c9640 1 Processor -- start 2023-11-26T07:18:00.621 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:00.618+0000 7f72253a0640 1 Processor -- start 2023-11-26T07:18:00.621 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:00.618+0000 7f7f6a2c9640 1 -- start start 2023-11-26T07:18:00.622 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:00.618+0000 7f72253a0640 1 -- start start 2023-11-26T07:18:00.624 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:00.618+0000 7f2146962640 1 Processor -- start 2023-11-26T07:18:00.624 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:00.618+0000 7f2146962640 1 -- start start 2023-11-26T07:18:00.624 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:00.618+0000 7f7f6a2c9640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f7f64159760 0x7f7f640b3af0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:18:00.624 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:00.618+0000 7f7f6a2c9640 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7f7f64071e20 0x7f7f64070370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:18:00.624 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:00.618+0000 7f7f6a2c9640 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7f7f640708b0 0x7f7f6406d1d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:18:00.624 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:00.618+0000 7f7f6a2c9640 1 -- --> v2:172.21.15.143:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7f64070d80 con 0x7f7f64071e20 2023-11-26T07:18:00.624 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:00.618+0000 7f7f6a2c9640 1 -- --> v2:172.21.15.188:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7f64071060 con 0x7f7f640708b0 2023-11-26T07:18:00.625 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:00.618+0000 7f7f6a2c9640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7f64071340 con 0x7f7f64159760 2023-11-26T07:18:00.625 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:00.618+0000 7f72253a0640 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7f72180e30a0 0x7f7218054200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:18:00.625 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:00.618+0000 7f72253a0640 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7f72180e6c00 0x7f72180e7090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:18:00.625 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:00.618+0000 7f72253a0640 1 -- --> v1:172.21.15.188:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7218037b00 con 0x7f72180e77f0 2023-11-26T07:18:00.625 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:00.618+0000 7f72253a0640 1 -- --> v2:172.21.15.143:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f72180e75d0 con 0x7f72180e6c00 2023-11-26T07:18:00.625 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:00.618+0000 7f72253a0640 1 -- --> v2:172.21.15.188:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f721804fb10 con 0x7f72180e30a0 2023-11-26T07:18:00.626 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:00.618+0000 7f7f63fff640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f7f64159760 0x7f7f640b3af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:18:00.626 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:00.618+0000 7f7f63fff640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f7f64159760 0x7f7f640b3af0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.188:3301/0 says I am v2:172.21.15.143:45148/0 (socket says 172.21.15.143:45148) 2023-11-26T07:18:00.626 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:00.618+0000 7f7f63fff640 1 -- 172.21.15.143:0/351260781 learned_addr learned my addr 172.21.15.143:0/351260781 (peer_addr_for_me v2:172.21.15.143:0/0) 2023-11-26T07:18:00.626 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:00.622+0000 7f7f637fe640 1 --2- 172.21.15.143:0/351260781 >> v2:172.21.15.143:3300/0 conn(0x7f7f64071e20 0x7f7f64070370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:18:00.626 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:00.622+0000 7f7f68866640 1 --2- 172.21.15.143:0/351260781 >> v2:172.21.15.188:3300/0 conn(0x7f7f640708b0 0x7f7f6406d1d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:18:00.626 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:00.622+0000 7f721ffff640 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7f72180e30a0 0x7f7218054200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:18:00.626 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:00.622+0000 7f721f7fe640 1 --1- >> v1:172.21.15.188:6789/0 conn(0x7f72180e77f0 0x7f721804f060 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.188:6789/0 says I am v1:172.21.15.143:39984/0 (socket says 172.21.15.143:39984) 2023-11-26T07:18:00.627 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:00.622+0000 7f721f7fe640 1 -- 172.21.15.143:0/706388380 learned_addr learned my addr 172.21.15.143:0/706388380 (peer_addr_for_me v1:172.21.15.143:0/0) 2023-11-26T07:18:00.627 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:00.622+0000 7fe17da06640 1 --2- 172.21.15.143:0/2383465475 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fe1780ce310 0x7fe178070370 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7fe170009d40 tx=0x7fe17001d9b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ed84e7cc2b0c2abd server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:18:00.627 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:00.622+0000 7f7f63fff640 1 -- 172.21.15.143:0/351260781 >> v2:172.21.15.188:3300/0 conn(0x7f7f640708b0 msgr2=0x7f7f6406d1d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-26T07:18:00.627 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:00.622+0000 7f7f63fff640 1 --2- 172.21.15.143:0/351260781 >> v2:172.21.15.188:3300/0 conn(0x7f7f640708b0 0x7f7f6406d1d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:18:00.627 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:00.622+0000 7f7f63fff640 1 -- 172.21.15.143:0/351260781 >> v2:172.21.15.143:3300/0 conn(0x7f7f64071e20 msgr2=0x7f7f64070370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-26T07:18:00.629 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:00.622+0000 7f2146962640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f214006e2d0 0x7f21400b2fd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:18:00.629 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:00.622+0000 7f2146962640 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7f214006d010 0x7f21400cc770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:18:00.630 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:00.622+0000 7f7224b9f640 1 --2- 172.21.15.143:0/706388380 >> v2:172.21.15.143:3300/0 conn(0x7f72180e6c00 0x7f72180e7090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:18:00.630 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:00.622+0000 7f721ffff640 1 -- 172.21.15.143:0/706388380 >> v2:172.21.15.143:3300/0 conn(0x7f72180e6c00 msgr2=0x7f72180e7090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-26T07:18:00.630 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:00.622+0000 7f721effd640 1 -- 172.21.15.143:0/706388380 <== mon.1 v1:172.21.15.188:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4065208103 0 0) 0x7f7218037b00 con 0x7f72180e77f0 2023-11-26T07:18:00.631 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:00.622+0000 7f2146962640 1 -- --> v1:172.21.15.188:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f21400ce410 con 0x7f2140159270 2023-11-26T07:18:00.631 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:00.622+0000 7f2146962640 1 -- --> v2:172.21.15.143:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f21400b28e0 con 0x7f214006d010 2023-11-26T07:18:00.631 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:00.622+0000 7f2146962640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f21400b3740 con 0x7f214006e2d0 2023-11-26T07:18:00.631 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:00.622+0000 7f214515f640 1 --1- >> v1:172.21.15.188:6789/0 conn(0x7f2140159270 0x7f214006c900 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.188:6789/0 says I am v1:172.21.15.143:39988/0 (socket says 172.21.15.143:39988) 2023-11-26T07:18:00.631 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:00.622+0000 7f214515f640 1 -- 172.21.15.143:0/3903593325 learned_addr learned my addr 172.21.15.143:0/3903593325 (peer_addr_for_me v1:172.21.15.143:0/0) 2023-11-26T07:18:00.631 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:00.622+0000 7f2145960640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f214006e2d0 0x7f21400b2fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:18:00.631 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:00.622+0000 7f2145960640 1 -- 172.21.15.143:0/3903593325 >> v2:172.21.15.143:3300/0 conn(0x7f214006d010 msgr2=0x7f21400cc770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-26T07:18:00.631 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:00.622+0000 7f2145960640 1 --2- 172.21.15.143:0/3903593325 >> v2:172.21.15.143:3300/0 conn(0x7f214006d010 0x7f21400cc770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:18:00.631 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:00.622+0000 7f2145960640 1 -- 172.21.15.143:0/3903593325 >> v1:172.21.15.188:6789/0 conn(0x7f2140159270 legacy=0x7f214006c900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:18:00.631 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:00.622+0000 7f2145960640 1 -- 172.21.15.143:0/3903593325 --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f21400b3510 con 0x7f214006e2d0 2023-11-26T07:18:00.632 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:00.622+0000 7f7f63fff640 1 --2- 172.21.15.143:0/351260781 >> v2:172.21.15.143:3300/0 conn(0x7f7f64071e20 0x7f7f64070370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:18:00.632 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:00.622+0000 7f7f63fff640 1 -- 172.21.15.143:0/351260781 --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7f64071620 con 0x7f7f64159760 2023-11-26T07:18:00.632 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:00.626+0000 7f7f68866640 1 --2- 172.21.15.143:0/351260781 >> v2:172.21.15.188:3300/0 conn(0x7f7f640708b0 0x7f7f6406d1d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-26T07:18:00.632 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:00.622+0000 7f721ffff640 1 --2- 172.21.15.143:0/706388380 >> v2:172.21.15.143:3300/0 conn(0x7f72180e6c00 0x7f72180e7090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:18:00.632 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:00.622+0000 7f721ffff640 1 -- 172.21.15.143:0/706388380 >> v1:172.21.15.188:6789/0 conn(0x7f72180e77f0 legacy=0x7f721804f060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:18:00.632 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:00.622+0000 7f721ffff640 1 -- 172.21.15.143:0/706388380 --> v2:172.21.15.188:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f721804f770 con 0x7f72180e30a0 2023-11-26T07:18:00.632 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:00.622+0000 7f7224b9f640 1 --2- 172.21.15.143:0/706388380 >> v2:172.21.15.143:3300/0 conn(0x7f72180e6c00 0x7f72180e7090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-26T07:18:00.633 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:00.626+0000 7f7f637fe640 1 --2- 172.21.15.143:0/351260781 >> v2:172.21.15.143:3300/0 conn(0x7f7f64071e20 0x7f7f64070370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-26T07:18:00.633 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:00.626+0000 7fe17d205640 1 -- 172.21.15.143:0/2383465475 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe1780cebe0 con 0x7fe1780ce310 2023-11-26T07:18:00.633 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:00.626+0000 7fe17d205640 1 -- 172.21.15.143:0/2383465475 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe1700a68b0 con 0x7fe1780ce310 2023-11-26T07:18:00.633 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:00.626+0000 7fe17d205640 1 -- 172.21.15.143:0/2383465475 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe1780cebe0 con 0x7fe1780ce310 2023-11-26T07:18:00.634 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:00.626+0000 7f7f63fff640 1 --2- 172.21.15.143:0/351260781 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f7f64159760 0x7f7f640b3af0 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7f7f6406eb90 tx=0x7f7f5401c510 comp rx=0 tx=0).ready entity=mon.2 client_cookie=3a2a89d794d5ddc1 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:18:00.634 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:00.626+0000 7fe18046b640 1 -- 172.21.15.143:0/2383465475 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fe1780ce310 msgr2=0x7fe178070370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:18:00.634 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:00.626+0000 7fe18046b640 1 --2- 172.21.15.143:0/2383465475 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fe1780ce310 0x7fe178070370 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7fe170009d40 tx=0x7fe17001d9b0 comp rx=0 tx=0).stop 2023-11-26T07:18:00.634 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:00.626+0000 7f7f62ffd640 1 -- 172.21.15.143:0/351260781 <== mon.2 v2:172.21.15.188:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7f540ac600 con 0x7f7f64159760 2023-11-26T07:18:00.634 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:00.626+0000 7f7f62ffd640 1 -- 172.21.15.143:0/351260781 <== mon.2 v2:172.21.15.188:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7f540a8050 con 0x7f7f64159760 2023-11-26T07:18:00.635 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:00.626+0000 7f7f62ffd640 1 -- 172.21.15.143:0/351260781 <== mon.2 v2:172.21.15.188:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7f540ac980 con 0x7f7f64159760 2023-11-26T07:18:00.635 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:00.626+0000 7f721ffff640 1 --2- 172.21.15.143:0/706388380 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f72180e30a0 0x7f7218054200 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7f72180079e0 tx=0x7f721401ec90 comp rx=0 tx=0).ready entity=mon.1 client_cookie=7cbf8e7a1549a71e server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:18:00.635 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:00.626+0000 7f721effd640 1 -- 172.21.15.143:0/706388380 <== mon.1 v2:172.21.15.188:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f72140a8970 con 0x7f72180e30a0 2023-11-26T07:18:00.635 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:00.626+0000 7f721effd640 1 -- 172.21.15.143:0/706388380 <== mon.1 v2:172.21.15.188:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f72140a8ad0 con 0x7f72180e30a0 2023-11-26T07:18:00.635 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:00.626+0000 7f721effd640 1 -- 172.21.15.143:0/706388380 <== mon.1 v2:172.21.15.188:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f72140a8ea0 con 0x7f72180e30a0 2023-11-26T07:18:00.636 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:00.630+0000 7fe18046b640 1 -- 172.21.15.143:0/2383465475 shutdown_connections 2023-11-26T07:18:00.636 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:00.630+0000 7fe18046b640 1 --2- 172.21.15.143:0/2383465475 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fe1780708b0 0x7fe17815cea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:18:00.636 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:00.630+0000 7fe18046b640 1 --2- 172.21.15.143:0/2383465475 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fe1780ce310 0x7fe178070370 unknown :-1 s=CLOSED pgs=105 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:18:00.636 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:00.630+0000 7fe18046b640 1 -- 172.21.15.143:0/2383465475 >> 172.21.15.143:0/2383465475 conn(0x7fe1780b0bf0 msgr2=0x7fe1780b1000 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:18:00.636 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:00.630+0000 7f7f6a2c9640 1 -- 172.21.15.143:0/351260781 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f7f64159760 msgr2=0x7f7f640b3af0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:18:00.637 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:00.630+0000 7f7f6a2c9640 1 --2- 172.21.15.143:0/351260781 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f7f64159760 0x7f7f640b3af0 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7f7f6406eb90 tx=0x7f7f5401c510 comp rx=0 tx=0).stop 2023-11-26T07:18:00.637 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:00.630+0000 7f2145960640 1 --2- 172.21.15.143:0/3903593325 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f214006e2d0 0x7f21400b2fd0 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7f21300097c0 tx=0x7f213001c250 comp rx=0 tx=0).ready entity=mon.2 client_cookie=d1fc698ddcfd94ee server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:18:00.637 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:00.630+0000 7f7f6a2c9640 1 -- 172.21.15.143:0/351260781 shutdown_connections 2023-11-26T07:18:00.637 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:00.630+0000 7f7f6a2c9640 1 --2- 172.21.15.143:0/351260781 >> v2:172.21.15.188:3300/0 conn(0x7f7f640708b0 0x7f7f6406d1d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:18:00.637 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:00.630+0000 7f7f6a2c9640 1 --2- 172.21.15.143:0/351260781 >> v2:172.21.15.143:3300/0 conn(0x7f7f64071e20 0x7f7f64070370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:18:00.637 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:00.630+0000 7f7f6a2c9640 1 --2- 172.21.15.143:0/351260781 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f7f64159760 0x7f7f640b3af0 unknown :-1 s=CLOSED pgs=43 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:18:00.637 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:00.630+0000 7f7f6a2c9640 1 -- 172.21.15.143:0/351260781 >> 172.21.15.143:0/351260781 conn(0x7f7f640b0bf0 msgr2=0x7f7f640b1000 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:18:00.638 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:00.630+0000 7fe18046b640 1 -- 172.21.15.143:0/2383465475 shutdown_connections 2023-11-26T07:18:00.638 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:00.630+0000 7f72253a0640 1 -- 172.21.15.143:0/706388380 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f72180e30a0 msgr2=0x7f7218054200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:18:00.638 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:00.630+0000 7f72253a0640 1 --2- 172.21.15.143:0/706388380 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f72180e30a0 0x7f7218054200 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7f72180079e0 tx=0x7f721401ec90 comp rx=0 tx=0).stop 2023-11-26T07:18:00.638 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:00.630+0000 7fe18046b640 1 -- 172.21.15.143:0/2383465475 wait complete. 2023-11-26T07:18:00.639 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:00.630+0000 7f214495e640 1 -- 172.21.15.143:0/3903593325 <== mon.2 v2:172.21.15.188:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f21300ab600 con 0x7f214006e2d0 2023-11-26T07:18:00.639 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:00.630+0000 7f7f6a2c9640 1 -- 172.21.15.143:0/351260781 shutdown_connections 2023-11-26T07:18:00.639 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:00.630+0000 7f7f6a2c9640 1 -- 172.21.15.143:0/351260781 wait complete. 2023-11-26T07:18:00.639 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:00.630+0000 7f214495e640 1 -- 172.21.15.143:0/3903593325 <== mon.2 v2:172.21.15.188:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f21300a7050 con 0x7f214006e2d0 2023-11-26T07:18:00.640 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:00.630+0000 7f214495e640 1 -- 172.21.15.143:0/3903593325 <== mon.2 v2:172.21.15.188:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f21300ab980 con 0x7f214006e2d0 2023-11-26T07:18:00.640 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:00.630+0000 7f2146962640 1 -- 172.21.15.143:0/3903593325 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f214006e2d0 msgr2=0x7f21400b2fd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:18:00.640 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:00.630+0000 7f2146962640 1 --2- 172.21.15.143:0/3903593325 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f214006e2d0 0x7f21400b2fd0 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7f21300097c0 tx=0x7f213001c250 comp rx=0 tx=0).stop 2023-11-26T07:18:00.640 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:00.630+0000 7f72253a0640 1 -- 172.21.15.143:0/706388380 shutdown_connections 2023-11-26T07:18:00.640 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:00.630+0000 7f72253a0640 1 --2- 172.21.15.143:0/706388380 >> v2:172.21.15.143:3300/0 conn(0x7f72180e6c00 0x7f72180e7090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:18:00.640 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:00.630+0000 7f72253a0640 1 --2- 172.21.15.143:0/706388380 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f72180e30a0 0x7f7218054200 unknown :-1 s=CLOSED pgs=72 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:18:00.640 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:00.630+0000 7f72253a0640 1 -- 172.21.15.143:0/706388380 >> 172.21.15.143:0/706388380 conn(0x7f7218009f90 msgr2=0x7f721800a3a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:18:00.641 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:00.630+0000 7f72253a0640 1 -- 172.21.15.143:0/706388380 shutdown_connections 2023-11-26T07:18:00.641 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:00.630+0000 7f72253a0640 1 -- 172.21.15.143:0/706388380 wait complete. 2023-11-26T07:18:00.641 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:00.630+0000 7f2146962640 1 -- 172.21.15.143:0/3903593325 shutdown_connections 2023-11-26T07:18:00.641 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:00.630+0000 7f2146962640 1 --2- 172.21.15.143:0/3903593325 >> v2:172.21.15.143:3300/0 conn(0x7f214006d010 0x7f21400cc770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:18:00.641 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:00.630+0000 7f2146962640 1 --2- 172.21.15.143:0/3903593325 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f214006e2d0 0x7f21400b2fd0 secure :-1 s=CLOSED pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7f21300097c0 tx=0x7f213001c250 comp rx=0 tx=0).stop 2023-11-26T07:18:00.641 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:00.630+0000 7f2146962640 1 -- 172.21.15.143:0/3903593325 >> 172.21.15.143:0/3903593325 conn(0x7f214006a890 msgr2=0x7f214006aca0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:18:00.642 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:00.630+0000 7f7f6a2c9640 1 Processor -- start 2023-11-26T07:18:00.642 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:00.634+0000 7f7f6a2c9640 1 -- start start 2023-11-26T07:18:00.642 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:00.634+0000 7f72253a0640 1 Processor -- start 2023-11-26T07:18:00.642 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:00.634+0000 7f72253a0640 1 -- start start 2023-11-26T07:18:00.642 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:00.634+0000 7fe18046b640 1 Processor -- start 2023-11-26T07:18:00.643 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:00.630+0000 7f2146962640 1 -- 172.21.15.143:0/3903593325 shutdown_connections 2023-11-26T07:18:00.643 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:00.630+0000 7f2146962640 1 -- 172.21.15.143:0/3903593325 wait complete. 2023-11-26T07:18:00.643 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:00.634+0000 7fe18046b640 1 -- start start 2023-11-26T07:18:00.643 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:00.634+0000 7f2146962640 1 Processor -- start 2023-11-26T07:18:00.644 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:00.634+0000 7f7f6a2c9640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f7f640708b0 0x7f7f6414ffd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:18:00.644 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:00.634+0000 7f72253a0640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f72180e30a0 0x7f72180ce5f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:18:00.644 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:00.634+0000 7fe18046b640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fe1780708b0 0x7fe178163500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:18:00.644 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:00.634+0000 7fe18046b640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fe1780ce310 0x7fe1781542b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:18:00.644 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:00.634+0000 7fe18046b640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fe178159170 0x7fe1781547f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:18:00.645 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:00.634+0000 7f72253a0640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f72180e6c00 0x7f721804ad30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:18:00.645 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:00.634+0000 7f72253a0640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f72180e77f0 0x7f721817bff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:18:00.645 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:00.638+0000 7f721ffff640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f72180e30a0 0x7f72180ce5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:18:00.645 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:00.638+0000 7f721ffff640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f72180e30a0 0x7f72180ce5f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.188:3301/0 says I am v2:172.21.15.143:45156/0 (socket says 172.21.15.143:45156) 2023-11-26T07:18:00.645 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:00.638+0000 7f72253a0640 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7218036000 con 0x7f72180e77f0 2023-11-26T07:18:00.645 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:00.638+0000 7f72253a0640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f72180362e0 con 0x7f72180e6c00 2023-11-26T07:18:00.645 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:00.638+0000 7f72253a0640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f72180365c0 con 0x7f72180e30a0 2023-11-26T07:18:00.646 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:00.634+0000 7f7f6a2c9640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f7f64071e20 0x7f7f641fe680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:18:00.646 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:00.638+0000 7f721ffff640 1 -- 172.21.15.143:0/1951363296 learned_addr learned my addr 172.21.15.143:0/1951363296 (peer_addr_for_me v2:172.21.15.143:0/0) 2023-11-26T07:18:00.646 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:00.634+0000 7f7f6a2c9640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f7f64159760 0x7f7f641febc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:18:00.646 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:00.638+0000 7f7f637fe640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f7f64071e20 0x7f7f641fe680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:18:00.646 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:00.638+0000 7f7f68866640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f7f64159760 0x7f7f641febc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:18:00.647 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:00.638+0000 7f7f68866640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f7f64159760 0x7f7f641febc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.188:3301/0 says I am v2:172.21.15.143:45172/0 (socket says 172.21.15.143:45172) 2023-11-26T07:18:00.647 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:00.638+0000 7f7f637fe640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f7f64071e20 0x7f7f641fe680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.188:3300/0 says I am v2:172.21.15.143:45640/0 (socket says 172.21.15.143:45640) 2023-11-26T07:18:00.647 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:00.638+0000 7f721f7fe640 1 --2- 172.21.15.143:0/1951363296 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f72180e6c00 0x7f721804ad30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:18:00.647 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:00.638+0000 7f7f68866640 1 -- 172.21.15.143:0/2197969909 learned_addr learned my addr 172.21.15.143:0/2197969909 (peer_addr_for_me v2:172.21.15.143:0/0) 2023-11-26T07:18:00.648 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:00.638+0000 7f7f6a2c9640 1 -- 172.21.15.143:0/2197969909 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7f6415bcf0 con 0x7f7f640708b0 2023-11-26T07:18:00.648 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:00.638+0000 7f7f6a2c9640 1 -- 172.21.15.143:0/2197969909 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7f6414f7c0 con 0x7f7f64071e20 2023-11-26T07:18:00.648 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:00.638+0000 7f7f6a2c9640 1 -- 172.21.15.143:0/2197969909 --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7f6415bf60 con 0x7f7f64159760 2023-11-26T07:18:00.648 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:00.638+0000 7fe17e207640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fe1780708b0 0x7fe178163500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:18:00.648 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:00.638+0000 7fe18046b640 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe178158040 con 0x7fe1780ce310 2023-11-26T07:18:00.648 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:00.638+0000 7fe18046b640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe178144b50 con 0x7fe1780708b0 2023-11-26T07:18:00.648 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:00.638+0000 7fe18046b640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe178144f70 con 0x7fe178159170 2023-11-26T07:18:00.648 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:00.638+0000 7fe17da06640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fe1780ce310 0x7fe1781542b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:18:00.648 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:00.638+0000 7fe17e207640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fe1780708b0 0x7fe178163500 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.188:3300/0 says I am v2:172.21.15.143:45646/0 (socket says 172.21.15.143:45646) 2023-11-26T07:18:00.649 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:00.638+0000 7fe17e207640 1 -- 172.21.15.143:0/1551758953 learned_addr learned my addr 172.21.15.143:0/1551758953 (peer_addr_for_me v2:172.21.15.143:0/0) 2023-11-26T07:18:00.649 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:00.638+0000 7f7224b9f640 1 --2- 172.21.15.143:0/1951363296 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f72180e77f0 0x7f721817bff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:18:00.649 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:00.638+0000 7fe17ea08640 1 --2- 172.21.15.143:0/1551758953 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fe178159170 0x7fe1781547f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:18:00.649 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:00.638+0000 7f721ffff640 1 -- 172.21.15.143:0/1951363296 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f72180e6c00 msgr2=0x7f721804ad30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:18:00.649 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:00.638+0000 7f721ffff640 1 --2- 172.21.15.143:0/1951363296 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f72180e6c00 0x7f721804ad30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:18:00.650 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:00.638+0000 7f721ffff640 1 -- 172.21.15.143:0/1951363296 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f72180e77f0 msgr2=0x7f721817bff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:18:00.650 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:00.638+0000 7f721ffff640 1 --2- 172.21.15.143:0/1951363296 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f72180e77f0 0x7f721817bff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:18:00.650 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:00.638+0000 7f721ffff640 1 -- 172.21.15.143:0/1951363296 --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7214009420 con 0x7f72180e30a0 2023-11-26T07:18:00.650 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:00.638+0000 7fe17ea08640 1 -- 172.21.15.143:0/1551758953 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fe1780708b0 msgr2=0x7fe178163500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:18:00.650 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:00.638+0000 7fe17ea08640 1 --2- 172.21.15.143:0/1551758953 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fe1780708b0 0x7fe178163500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:18:00.650 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:00.638+0000 7fe17ea08640 1 -- 172.21.15.143:0/1551758953 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fe1780ce310 msgr2=0x7fe1781542b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:18:00.650 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:00.638+0000 7fe17ea08640 1 --2- 172.21.15.143:0/1551758953 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fe1780ce310 0x7fe1781542b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:18:00.650 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:00.638+0000 7fe17ea08640 1 -- 172.21.15.143:0/1551758953 --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe17000d040 con 0x7fe178159170 2023-11-26T07:18:00.651 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:00.638+0000 7f7224b9f640 1 --2- 172.21.15.143:0/1951363296 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f72180e77f0 0x7f721817bff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-26T07:18:00.651 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:00.634+0000 7f2146962640 1 -- start start 2023-11-26T07:18:00.651 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:00.638+0000 7fe17ea08640 1 --2- 172.21.15.143:0/1551758953 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fe178159170 0x7fe1781547f0 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7fe17400dad0 tx=0x7fe17400df90 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:18:00.652 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:00.638+0000 7f2146962640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f214006d010 0x7f21401545e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:18:00.652 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:00.638+0000 7f2146962640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f2140159270 0x7f2140160c20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:18:00.652 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:00.638+0000 7f2146962640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f2140161160 0x7f2140164890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:18:00.652 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:00.638+0000 7f2146962640 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f21401652c0 con 0x7f2140161160 2023-11-26T07:18:00.652 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:00.638+0000 7f2146962640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2140154b20 con 0x7f214006d010 2023-11-26T07:18:00.652 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:00.638+0000 7f2146962640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f214015fe60 con 0x7f2140159270 2023-11-26T07:18:00.653 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:00.638+0000 7f2145960640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f214006d010 0x7f21401545e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:18:00.653 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:00.638+0000 7f2145960640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f214006d010 0x7f21401545e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.188:3300/0 says I am v2:172.21.15.143:45656/0 (socket says 172.21.15.143:45656) 2023-11-26T07:18:00.653 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:00.638+0000 7f2145960640 1 -- 172.21.15.143:0/3175475790 learned_addr learned my addr 172.21.15.143:0/3175475790 (peer_addr_for_me v2:172.21.15.143:0/0) 2023-11-26T07:18:00.653 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:00.638+0000 7f214515f640 1 --2- 172.21.15.143:0/3175475790 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f2140159270 0x7f2140160c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:18:00.653 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:00.638+0000 7f2145960640 1 -- 172.21.15.143:0/3175475790 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f2140159270 msgr2=0x7f2140160c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:18:00.653 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:00.638+0000 7f2145960640 1 --2- 172.21.15.143:0/3175475790 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f2140159270 0x7f2140160c20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:18:00.653 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:00.638+0000 7f2145960640 1 -- 172.21.15.143:0/3175475790 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f2140161160 msgr2=0x7f2140164890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:18:00.653 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:00.638+0000 7f2145960640 1 --2- 172.21.15.143:0/3175475790 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f2140161160 0x7f2140164890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:18:00.653 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:00.638+0000 7f2145960640 1 -- 172.21.15.143:0/3175475790 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2130009420 con 0x7f214006d010 2023-11-26T07:18:00.653 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:00.638+0000 7f2145960640 1 --2- 172.21.15.143:0/3175475790 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f214006d010 0x7f21401545e0 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7f2130019e80 tx=0x7f213001aa90 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:18:00.653 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:00.638+0000 7f21367fc640 1 -- 172.21.15.143:0/3175475790 <== mon.1 v2:172.21.15.188:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f21300b7650 con 0x7f214006d010 2023-11-26T07:18:00.654 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:00.638+0000 7fe16effd640 1 -- 172.21.15.143:0/1551758953 <== mon.2 v2:172.21.15.188:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe174012070 con 0x7fe178159170 2023-11-26T07:18:00.654 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:00.638+0000 7fe16effd640 1 -- 172.21.15.143:0/1551758953 <== mon.2 v2:172.21.15.188:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe17400cd20 con 0x7fe178159170 2023-11-26T07:18:00.654 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:00.638+0000 7fe16effd640 1 -- 172.21.15.143:0/1551758953 <== mon.2 v2:172.21.15.188:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe17400a650 con 0x7fe178159170 2023-11-26T07:18:00.654 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:00.638+0000 7f7f63fff640 1 --2- 172.21.15.143:0/2197969909 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f7f640708b0 0x7f7f6414ffd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:18:00.654 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:00.638+0000 7f7f68866640 1 -- 172.21.15.143:0/2197969909 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f7f64071e20 msgr2=0x7f7f641fe680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:18:00.655 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:00.638+0000 7f721ffff640 1 --2- 172.21.15.143:0/1951363296 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f72180e30a0 0x7f72180ce5f0 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7f721401c8c0 tx=0x7f721401ba30 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:18:00.655 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:00.638+0000 7f21367fc640 1 -- 172.21.15.143:0/3175475790 <== mon.1 v2:172.21.15.188:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f21300abe90 con 0x7f214006d010 2023-11-26T07:18:00.655 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:00.642+0000 7fe18046b640 1 -- 172.21.15.143:0/1551758953 --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe178141f80 con 0x7fe178159170 2023-11-26T07:18:00.655 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:00.642+0000 7fe18046b640 1 -- 172.21.15.143:0/1551758953 --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe178144490 con 0x7fe178159170 2023-11-26T07:18:00.656 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:00.638+0000 7f7f68866640 1 --2- 172.21.15.143:0/2197969909 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f7f64071e20 0x7f7f641fe680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:18:00.656 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:00.638+0000 7f7f68866640 1 -- 172.21.15.143:0/2197969909 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f7f640708b0 msgr2=0x7f7f6414ffd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:18:00.656 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:00.638+0000 7f7f68866640 1 --2- 172.21.15.143:0/2197969909 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f7f640708b0 0x7f7f6414ffd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:18:00.656 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:00.638+0000 7f7f68866640 1 -- 172.21.15.143:0/2197969909 --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7f540094d0 con 0x7f7f64159760 2023-11-26T07:18:00.656 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:00.638+0000 7f7f637fe640 1 --2- 172.21.15.143:0/2197969909 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f7f64071e20 0x7f7f641fe680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-26T07:18:00.657 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:00.638+0000 7f7f68866640 1 --2- 172.21.15.143:0/2197969909 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f7f64159760 0x7f7f641febc0 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7f7f5c00cef0 tx=0x7f7f5c00a710 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:18:00.657 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:00.638+0000 7f7f60ff9640 1 -- 172.21.15.143:0/2197969909 <== mon.2 v2:172.21.15.188:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7f5c00d840 con 0x7f7f64159760 2023-11-26T07:18:00.657 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:00.638+0000 7f7f60ff9640 1 -- 172.21.15.143:0/2197969909 <== mon.2 v2:172.21.15.188:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7f5c00d9c0 con 0x7f7f64159760 2023-11-26T07:18:00.657 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:00.638+0000 7f7f60ff9640 1 -- 172.21.15.143:0/2197969909 <== mon.2 v2:172.21.15.188:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7f5c00dd70 con 0x7f7f64159760 2023-11-26T07:18:00.657 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:00.638+0000 7f7f6a2c9640 1 -- 172.21.15.143:0/2197969909 --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7f6415c240 con 0x7f7f64159760 2023-11-26T07:18:00.657 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:00.638+0000 7f7f6a2c9640 1 -- 172.21.15.143:0/2197969909 --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7f6414f470 con 0x7f7f64159760 2023-11-26T07:18:00.657 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:00.642+0000 7f7f6a2c9640 1 -- 172.21.15.143:0/2197969909 --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7f6415c760 con 0x7f7f64159760 2023-11-26T07:18:00.657 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:00.642+0000 7f721cff9640 1 -- 172.21.15.143:0/1951363296 <== mon.2 v2:172.21.15.188:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f72140b0e00 con 0x7f72180e30a0 2023-11-26T07:18:00.658 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:00.642+0000 7f21367fc640 1 -- 172.21.15.143:0/3175475790 <== mon.1 v2:172.21.15.188:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f21300b79f0 con 0x7f214006d010 2023-11-26T07:18:00.658 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:00.642+0000 7fe18046b640 1 -- 172.21.15.143:0/1551758953 --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe178144c90 con 0x7fe178159170 2023-11-26T07:18:00.658 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:00.642+0000 7f721cff9640 1 -- 172.21.15.143:0/1951363296 <== mon.2 v2:172.21.15.188:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f72140b9400 con 0x7f72180e30a0 2023-11-26T07:18:00.659 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:00.642+0000 7f721cff9640 1 -- 172.21.15.143:0/1951363296 <== mon.2 v2:172.21.15.188:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f72140b0e00 con 0x7f72180e30a0 2023-11-26T07:18:00.659 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:00.642+0000 7f72253a0640 1 -- 172.21.15.143:0/1951363296 --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f72180368a0 con 0x7f72180e30a0 2023-11-26T07:18:00.659 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:00.642+0000 7f72253a0640 1 -- 172.21.15.143:0/1951363296 --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f721804b270 con 0x7f72180e30a0 2023-11-26T07:18:00.659 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:00.642+0000 7f2146962640 1 -- 172.21.15.143:0/3175475790 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2140160140 con 0x7f214006d010 2023-11-26T07:18:00.659 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:00.642+0000 7f72253a0640 1 -- 172.21.15.143:0/1951363296 --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7200005260 con 0x7f72180e30a0 2023-11-26T07:18:00.660 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:00.642+0000 7f2146962640 1 -- 172.21.15.143:0/3175475790 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2140160700 con 0x7f214006d010 2023-11-26T07:18:00.660 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:00.646+0000 7fe16effd640 1 -- 172.21.15.143:0/1551758953 <== mon.2 v2:172.21.15.188:3301/0 4 ==== mgrmap(e 5) v1 ==== 85852+0+0 (secure 0 0 0) 0x7fe17400a7b0 con 0x7fe178159170 2023-11-26T07:18:00.660 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:00.642+0000 7f7f60ff9640 1 -- 172.21.15.143:0/2197969909 <== mon.2 v2:172.21.15.188:3301/0 4 ==== mgrmap(e 5) v1 ==== 85852+0+0 (secure 0 0 0) 0x7f7f5c01e3b0 con 0x7f7f64159760 2023-11-26T07:18:00.660 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:00.646+0000 7f7f60ff9640 1 --2- 172.21.15.143:0/2197969909 >> [v2:172.21.15.143:6832/2150557544,v1:172.21.15.143:6833/2150557544] conn(0x7f7f44077550 0x7f7f44079a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:18:00.661 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:00.646+0000 7f2146962640 1 -- 172.21.15.143:0/3175475790 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2140160920 con 0x7f214006d010 2023-11-26T07:18:00.661 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:00.646+0000 7fe16effd640 1 --2- 172.21.15.143:0/1551758953 >> [v2:172.21.15.143:6832/2150557544,v1:172.21.15.143:6833/2150557544] conn(0x7fe15c0664c0 0x7fe15c068980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:18:00.661 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:00.646+0000 7fe16effd640 1 -- 172.21.15.143:0/1551758953 <== mon.2 v2:172.21.15.188:3301/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fe1740848b0 con 0x7fe178159170 2023-11-26T07:18:00.662 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:00.646+0000 7f7f63fff640 1 --2- 172.21.15.143:0/2197969909 >> [v2:172.21.15.143:6832/2150557544,v1:172.21.15.143:6833/2150557544] conn(0x7f7f44077550 0x7f7f44079a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:18:00.662 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:00.646+0000 7f7f60ff9640 1 -- 172.21.15.143:0/2197969909 <== mon.2 v2:172.21.15.188:3301/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f7f5c0848e0 con 0x7f7f64159760 2023-11-26T07:18:00.662 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:00.646+0000 7fe17e207640 1 --2- 172.21.15.143:0/1551758953 >> [v2:172.21.15.143:6832/2150557544,v1:172.21.15.143:6833/2150557544] conn(0x7fe15c0664c0 0x7fe15c068980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:18:00.662 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:00.646+0000 7f21367fc640 1 -- 172.21.15.143:0/3175475790 <== mon.1 v2:172.21.15.188:3300/0 4 ==== mgrmap(e 5) v1 ==== 85852+0+0 (secure 0 0 0) 0x7f21300b7b50 con 0x7f214006d010 2023-11-26T07:18:00.663 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:00.646+0000 7fe17e207640 1 --2- 172.21.15.143:0/1551758953 >> [v2:172.21.15.143:6832/2150557544,v1:172.21.15.143:6833/2150557544] conn(0x7fe15c0664c0 0x7fe15c068980 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7fe168007fd0 tx=0x7fe168009040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:18:00.663 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:00.646+0000 7f7f63fff640 1 --2- 172.21.15.143:0/2197969909 >> [v2:172.21.15.143:6832/2150557544,v1:172.21.15.143:6833/2150557544] conn(0x7f7f44077550 0x7f7f44079a10 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7f7f540ae000 tx=0x7f7f54008040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:18:00.663 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:00.650+0000 7f21367fc640 1 --2- 172.21.15.143:0/3175475790 >> [v2:172.21.15.143:6832/2150557544,v1:172.21.15.143:6833/2150557544] conn(0x7f2124066470 0x7f2124068930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:18:00.664 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:00.650+0000 7f21367fc640 1 -- 172.21.15.143:0/3175475790 <== mon.1 v2:172.21.15.188:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f213011e800 con 0x7f214006d010 2023-11-26T07:18:00.664 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:00.650+0000 7f214515f640 1 --2- 172.21.15.143:0/3175475790 >> [v2:172.21.15.143:6832/2150557544,v1:172.21.15.143:6833/2150557544] conn(0x7f2124066470 0x7f2124068930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:18:00.664 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:00.650+0000 7f214515f640 1 --2- 172.21.15.143:0/3175475790 >> [v2:172.21.15.143:6832/2150557544,v1:172.21.15.143:6833/2150557544] conn(0x7f2124066470 0x7f2124068930 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7f2140158a10 tx=0x7f213c007c30 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:18:00.688 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:00.682+0000 7f7f60ff9640 1 -- 172.21.15.143:0/2197969909 <== mon.2 v2:172.21.15.188:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172649 (secure 0 0 0) 0x7f7f6415c760 con 0x7f7f64159760 2023-11-26T07:18:00.689 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:00.682+0000 7f8407832640 1 Processor -- start 2023-11-26T07:18:00.689 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:00.686+0000 7f8407832640 1 -- start start 2023-11-26T07:18:00.690 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:00.686+0000 7f7690b65640 1 Processor -- start 2023-11-26T07:18:00.690 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:00.686+0000 7f721cff9640 1 -- 172.21.15.143:0/1951363296 <== mon.2 v2:172.21.15.188:3301/0 4 ==== mgrmap(e 5) v1 ==== 85852+0+0 (secure 0 0 0) 0x7f72140b7040 con 0x7f72180e30a0 2023-11-26T07:18:00.690 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:00.686+0000 7f7690b65640 1 -- start start 2023-11-26T07:18:00.692 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:00.686+0000 7f21367fc640 1 -- 172.21.15.143:0/3175475790 <== mon.1 v2:172.21.15.188:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172649 (secure 0 0 0) 0x7f21300ebc70 con 0x7f214006d010 2023-11-26T07:18:00.692 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:00.686+0000 7f8407832640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f8400163520 0x7f840016c3a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:18:00.692 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:00.686+0000 7f8407832640 1 -- --> v1:172.21.15.143:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8400162080 con 0x7f840014b1e0 2023-11-26T07:18:00.692 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:00.686+0000 7f8407832640 1 -- --> v1:172.21.15.188:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8400161d70 con 0x7f84000cf990 2023-11-26T07:18:00.692 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:00.686+0000 7f8407832640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f840014b5c0 con 0x7f8400163520 2023-11-26T07:18:00.693 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:00.686+0000 7f7690b65640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f768c15d5b0 0x7f768c0d1820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:18:00.693 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:00.686+0000 7f7690b65640 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7f768c0b3880 0x7f768c0cea20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:18:00.693 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:00.686+0000 7f7690b65640 1 -- --> v1:172.21.15.188:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f768c0d21c0 con 0x7f768c0d1d60 2023-11-26T07:18:00.693 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:00.686+0000 7f7690b65640 1 -- --> v2:172.21.15.188:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f768c0ce590 con 0x7f768c0b3880 2023-11-26T07:18:00.693 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:00.686+0000 7f7690b65640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f768c0cdf80 con 0x7f768c15d5b0 2023-11-26T07:18:00.693 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:00.686+0000 7f721cff9640 1 --2- 172.21.15.143:0/1951363296 >> [v2:172.21.15.143:6832/2150557544,v1:172.21.15.143:6833/2150557544] conn(0x7f7204066530 0x7f72040689f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:18:00.694 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:00.686+0000 7f7689d9b640 1 --1- >> v1:172.21.15.188:6789/0 conn(0x7f768c0d1d60 0x7f768c0b3170 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.188:6789/0 says I am v1:172.21.15.143:40004/0 (socket says 172.21.15.143:40004) 2023-11-26T07:18:00.694 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:00.686+0000 7f7689d9b640 1 -- 172.21.15.143:0/4025619962 learned_addr learned my addr 172.21.15.143:0/4025619962 (peer_addr_for_me v1:172.21.15.143:0/0) 2023-11-26T07:18:00.694 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:00.686+0000 7f8404dcd640 1 --1- >> v1:172.21.15.188:6789/0 conn(0x7f84000cf990 0x7f84000708c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.188:6789/0 says I am v1:172.21.15.143:39994/0 (socket says 172.21.15.143:39994) 2023-11-26T07:18:00.694 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:00.686+0000 7f8404dcd640 1 -- 172.21.15.143:0/3952528700 learned_addr learned my addr 172.21.15.143:0/3952528700 (peer_addr_for_me v1:172.21.15.143:0/0) 2023-11-26T07:18:00.694 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:00.690+0000 7f83f7fff640 1 -- 172.21.15.143:0/3952528700 <== mon.1 v1:172.21.15.188:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3099899318 0 0) 0x7f8400161d70 con 0x7f84000cf990 2023-11-26T07:18:00.695 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:00.686+0000 7f768a59c640 1 --2- 172.21.15.143:0/4025619962 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f768c15d5b0 0x7f768c0d1820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:18:00.695 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:00.690+0000 7f768ad9d640 1 --2- 172.21.15.143:0/4025619962 >> v2:172.21.15.188:3300/0 conn(0x7f768c0b3880 0x7f768c0cea20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:18:00.695 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:00.690+0000 7f768959a640 1 -- 172.21.15.143:0/4025619962 <== mon.1 v1:172.21.15.188:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 820226147 0 0) 0x7f768c0d21c0 con 0x7f768c0d1d60 2023-11-26T07:18:00.695 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:00.690+0000 7f768959a640 1 -- 172.21.15.143:0/4025619962 --> v1:172.21.15.188:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f7674003530 con 0x7f768c0d1d60 2023-11-26T07:18:00.695 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:00.690+0000 7f8405dcf640 1 --2- 172.21.15.143:0/3952528700 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f8400163520 0x7f840016c3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:18:00.695 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:00.690+0000 7f83f7fff640 1 -- 172.21.15.143:0/3952528700 --> v1:172.21.15.188:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f83ec003530 con 0x7f84000cf990 2023-11-26T07:18:00.696 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:00.690+0000 7f768959a640 1 -- 172.21.15.143:0/4025619962 <== mon.1 v1:172.21.15.188:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3133308305 0 0) 0x7f7674003530 con 0x7f768c0d1d60 2023-11-26T07:18:00.696 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:00.690+0000 7f768959a640 1 -- 172.21.15.143:0/4025619962 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f768c15d5b0 msgr2=0x7f768c0d1820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-26T07:18:00.696 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:00.690+0000 7f768959a640 1 --2- 172.21.15.143:0/4025619962 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f768c15d5b0 0x7f768c0d1820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:18:00.696 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:00.690+0000 7f768959a640 1 -- 172.21.15.143:0/4025619962 >> v2:172.21.15.188:3300/0 conn(0x7f768c0b3880 msgr2=0x7f768c0cea20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-26T07:18:00.696 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:00.690+0000 7f768959a640 1 --2- 172.21.15.143:0/4025619962 >> v2:172.21.15.188:3300/0 conn(0x7f768c0b3880 0x7f768c0cea20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:18:00.696 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:00.690+0000 7f768959a640 1 -- 172.21.15.143:0/4025619962 --> v1:172.21.15.188:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f768c0b3ea0 con 0x7f768c0d1d60 2023-11-26T07:18:00.696 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:00.690+0000 7f768ad9d640 1 --2- 172.21.15.143:0/4025619962 >> v2:172.21.15.188:3300/0 conn(0x7f768c0b3880 0x7f768c0cea20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-26T07:18:00.696 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:00.690+0000 7f768959a640 1 -- 172.21.15.143:0/4025619962 <== mon.1 v1:172.21.15.188:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2137882523 0 0) 0x7f7680001680 con 0x7f768c0d1d60 2023-11-26T07:18:00.696 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:00.690+0000 7f768959a640 1 -- 172.21.15.143:0/4025619962 >> v1:172.21.15.188:6789/0 conn(0x7f768c0d1d60 legacy=0x7f768c0b3170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:18:00.697 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:00.690+0000 7f721cff9640 1 -- 172.21.15.143:0/1951363296 <== mon.2 v2:172.21.15.188:3301/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f72140f38f0 con 0x7f72180e30a0 2023-11-26T07:18:00.697 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:00.690+0000 7f83f7fff640 1 -- 172.21.15.143:0/3952528700 <== mon.1 v1:172.21.15.188:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2258785848 0 0) 0x7f83ec003530 con 0x7f84000cf990 2023-11-26T07:18:00.697 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:00.690+0000 7f83f7fff640 1 -- 172.21.15.143:0/3952528700 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f8400163520 msgr2=0x7f840016c3a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-26T07:18:00.697 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:00.690+0000 7f83f7fff640 1 --2- 172.21.15.143:0/3952528700 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f8400163520 0x7f840016c3a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:18:00.697 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:00.690+0000 7f83f7fff640 1 -- 172.21.15.143:0/3952528700 >> v1:172.21.15.143:6789/0 conn(0x7f840014b1e0 legacy=0x7f84000cf280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-26T07:18:00.697 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:00.690+0000 7f83f7fff640 1 -- 172.21.15.143:0/3952528700 --> v1:172.21.15.188:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8400071580 con 0x7f84000cf990 2023-11-26T07:18:00.697 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:00.690+0000 7f83f7fff640 1 -- 172.21.15.143:0/3952528700 <== mon.1 v1:172.21.15.188:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2137882523 0 0) 0x7f83f0001680 con 0x7f84000cf990 2023-11-26T07:18:00.698 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:00.690+0000 7f768959a640 1 --2- 172.21.15.143:0/4025619962 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f76740039a0 0x7f7674003de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:18:00.698 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:00.690+0000 7f768959a640 1 --2- 172.21.15.143:0/4025619962 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f76740047d0 0x7f7674004c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:18:00.698 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:00.690+0000 7f768959a640 1 --2- 172.21.15.143:0/4025619962 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f768c15d5b0 0x7f7674005190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:18:00.698 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:00.690+0000 7f768959a640 1 -- 172.21.15.143:0/4025619962 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f768c0ce590 con 0x7f76740047d0 2023-11-26T07:18:00.698 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:00.690+0000 7f768959a640 1 -- 172.21.15.143:0/4025619962 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f768c0cdf80 con 0x7f76740039a0 2023-11-26T07:18:00.698 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:00.690+0000 7f768959a640 1 -- 172.21.15.143:0/4025619962 --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f767400b990 con 0x7f768c15d5b0 2023-11-26T07:18:00.698 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:00.690+0000 7f7689d9b640 1 --2- 172.21.15.143:0/4025619962 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f76740047d0 0x7f7674004c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:18:00.698 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:00.690+0000 7f768a59c640 1 --2- 172.21.15.143:0/4025619962 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f76740039a0 0x7f7674003de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:18:00.698 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:00.690+0000 7f768ad9d640 1 --2- 172.21.15.143:0/4025619962 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f768c15d5b0 0x7f7674005190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:18:00.699 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:00.690+0000 7f721f7fe640 1 --2- 172.21.15.143:0/1951363296 >> [v2:172.21.15.143:6832/2150557544,v1:172.21.15.143:6833/2150557544] conn(0x7f7204066530 0x7f72040689f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:18:00.699 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:00.690+0000 7f768ad9d640 1 -- 172.21.15.143:0/4025619962 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f76740039a0 msgr2=0x7f7674003de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-26T07:18:00.699 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:00.690+0000 7f768ad9d640 1 --2- 172.21.15.143:0/4025619962 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f76740039a0 0x7f7674003de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:18:00.699 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:00.690+0000 7f768ad9d640 1 -- 172.21.15.143:0/4025619962 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f76740047d0 msgr2=0x7f7674004c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-26T07:18:00.700 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:00.690+0000 7f768ad9d640 1 --2- 172.21.15.143:0/4025619962 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f76740047d0 0x7f7674004c50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:18:00.700 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:00.690+0000 7f768ad9d640 1 -- 172.21.15.143:0/4025619962 --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f768c0d21c0 con 0x7f768c15d5b0 2023-11-26T07:18:00.700 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:00.690+0000 7f83f7fff640 1 -- 172.21.15.143:0/3952528700 >> v1:172.21.15.188:6789/0 conn(0x7f84000cf990 legacy=0x7f84000708c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:18:00.700 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:00.690+0000 7f83f7fff640 1 --2- 172.21.15.143:0/3952528700 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f83ec003980 0x7f83ec003d80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:18:00.700 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:00.690+0000 7f83f7fff640 1 --2- 172.21.15.143:0/3952528700 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f83ec0047d0 0x7f83ec004c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:18:00.700 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:00.690+0000 7f83f7fff640 1 --2- 172.21.15.143:0/3952528700 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f8400163520 0x7f83ec005190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:18:00.700 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:00.690+0000 7f83f7fff640 1 -- 172.21.15.143:0/3952528700 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f840014b5c0 con 0x7f83ec003980 2023-11-26T07:18:00.700 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:00.690+0000 7f83f7fff640 1 -- 172.21.15.143:0/3952528700 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f83ec00ba90 con 0x7f83ec0047d0 2023-11-26T07:18:00.700 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:00.690+0000 7f83f7fff640 1 -- 172.21.15.143:0/3952528700 --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f83ec00bc50 con 0x7f8400163520 2023-11-26T07:18:00.701 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:00.690+0000 7f84055ce640 1 --2- 172.21.15.143:0/3952528700 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f83ec003980 0x7f83ec003d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:18:00.701 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:00.690+0000 7f8404dcd640 1 --2- 172.21.15.143:0/3952528700 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f83ec0047d0 0x7f83ec004c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:18:00.701 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:00.690+0000 7f8405dcf640 1 --2- 172.21.15.143:0/3952528700 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f8400163520 0x7f83ec005190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:18:00.701 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:00.690+0000 7f8404dcd640 1 -- 172.21.15.143:0/3952528700 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f8400163520 msgr2=0x7f83ec005190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-26T07:18:00.701 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:00.690+0000 7f8404dcd640 1 --2- 172.21.15.143:0/3952528700 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f8400163520 0x7f83ec005190 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:18:00.701 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:00.690+0000 7f8404dcd640 1 -- 172.21.15.143:0/3952528700 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f83ec003980 msgr2=0x7f83ec003d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-26T07:18:00.701 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:00.690+0000 7f8404dcd640 1 --2- 172.21.15.143:0/3952528700 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f83ec003980 0x7f83ec003d80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:18:00.701 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:00.690+0000 7f8404dcd640 1 -- 172.21.15.143:0/3952528700 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8400071580 con 0x7f83ec0047d0 2023-11-26T07:18:00.701 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:00.690+0000 7fe16effd640 1 -- 172.21.15.143:0/1551758953 <== mon.2 v2:172.21.15.188:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172649 (secure 0 0 0) 0x7fe178144c90 con 0x7fe178159170 2023-11-26T07:18:00.702 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:00.694+0000 7f721f7fe640 1 --2- 172.21.15.143:0/1951363296 >> [v2:172.21.15.143:6832/2150557544,v1:172.21.15.143:6833/2150557544] conn(0x7f7204066530 0x7f72040689f0 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7f7218033950 tx=0x7f720c007b10 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:18:00.702 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:00.694+0000 7f721cff9640 1 -- 172.21.15.143:0/1951363296 <== mon.2 v2:172.21.15.188:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172649 (secure 0 0 0) 0x7f72140ef530 con 0x7f72180e30a0 2023-11-26T07:18:00.702 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:00.698+0000 7f8404dcd640 1 --2- 172.21.15.143:0/3952528700 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f83ec0047d0 0x7f83ec004c50 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7f83f000c290 tx=0x7f83f001cc70 comp rx=0 tx=0).ready entity=mon.1 client_cookie=3d50ecb622869273 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:18:00.702 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:00.698+0000 7f768ad9d640 1 --2- 172.21.15.143:0/4025619962 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f768c15d5b0 0x7f7674005190 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7f768400e350 tx=0x7f768401ce60 comp rx=0 tx=0).ready entity=mon.2 client_cookie=c5c55d5529d2d4ec server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:18:00.703 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:00.698+0000 7f83f7fff640 1 -- 172.21.15.143:0/3952528700 <== mon.1 v2:172.21.15.188:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f83f0005ae0 con 0x7f83ec0047d0 2023-11-26T07:18:00.703 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:00.698+0000 7f768959a640 1 -- 172.21.15.143:0/4025619962 <== mon.2 v2:172.21.15.188:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f76840a5d60 con 0x7f768c15d5b0 2023-11-26T07:18:00.703 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:00.698+0000 7f768959a640 1 -- 172.21.15.143:0/4025619962 <== mon.2 v2:172.21.15.188:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f768400f040 con 0x7f768c15d5b0 2023-11-26T07:18:00.703 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:00.698+0000 7f768959a640 1 -- 172.21.15.143:0/4025619962 <== mon.2 v2:172.21.15.188:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f76840ad640 con 0x7f768c15d5b0 2023-11-26T07:18:00.703 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:00.698+0000 7f7690b65640 1 -- 172.21.15.143:0/4025619962 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f768c15d5b0 msgr2=0x7f7674005190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:18:00.703 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:00.698+0000 7f7690b65640 1 --2- 172.21.15.143:0/4025619962 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f768c15d5b0 0x7f7674005190 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7f768400e350 tx=0x7f768401ce60 comp rx=0 tx=0).stop 2023-11-26T07:18:00.704 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:00.698+0000 7f83f7fff640 1 -- 172.21.15.143:0/3952528700 <== mon.1 v2:172.21.15.188:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f83f0005c40 con 0x7f83ec0047d0 2023-11-26T07:18:00.704 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:00.698+0000 7f768ad9d640 1 -- 172.21.15.143:0/4025619962 reap_dead start 2023-11-26T07:18:00.704 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:00.698+0000 7f8407832640 1 -- 172.21.15.143:0/3952528700 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f83ec0047d0 msgr2=0x7f83ec004c50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:18:00.704 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:00.698+0000 7f8407832640 1 --2- 172.21.15.143:0/3952528700 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f83ec0047d0 0x7f83ec004c50 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7f83f000c290 tx=0x7f83f001cc70 comp rx=0 tx=0).stop 2023-11-26T07:18:00.705 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:00.698+0000 7f8405dcf640 1 -- 172.21.15.143:0/3952528700 reap_dead start 2023-11-26T07:18:00.705 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:00.698+0000 7f7690b65640 1 -- 172.21.15.143:0/4025619962 shutdown_connections 2023-11-26T07:18:00.705 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:00.698+0000 7f7690b65640 1 -- 172.21.15.143:0/4025619962 >> 172.21.15.143:0/4025619962 conn(0x7f768c08cc60 msgr2=0x7f768c09c580 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:18:00.705 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:00.698+0000 7f8407832640 1 -- 172.21.15.143:0/3952528700 shutdown_connections 2023-11-26T07:18:00.705 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:00.698+0000 7f8407832640 1 -- 172.21.15.143:0/3952528700 >> 172.21.15.143:0/3952528700 conn(0x7f840006a890 msgr2=0x7f840006aca0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:18:00.706 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:00.698+0000 7f7690b65640 1 -- 172.21.15.143:0/4025619962 shutdown_connections 2023-11-26T07:18:00.706 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:00.698+0000 7f8407832640 1 -- 172.21.15.143:0/3952528700 shutdown_connections 2023-11-26T07:18:00.706 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:00.698+0000 7f7690b65640 1 -- 172.21.15.143:0/4025619962 wait complete. 2023-11-26T07:18:00.707 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:00.698+0000 7f8407832640 1 -- 172.21.15.143:0/3952528700 wait complete. 2023-11-26T07:18:00.707 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:00.702+0000 7f7690b65640 1 Processor -- start 2023-11-26T07:18:00.707 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:00.702+0000 7f7690b65640 1 -- start start 2023-11-26T07:18:00.707 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:00.702+0000 7f8407832640 1 Processor -- start 2023-11-26T07:18:00.708 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:00.702+0000 7f8407832640 1 -- start start 2023-11-26T07:18:00.708 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:00.702+0000 7f7690b65640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f768c15d5b0 0x7f768c15a9f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:18:00.708 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:00.702+0000 7f7690b65640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f768c0b1460 0x7f768c0b18e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:18:00.708 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:00.702+0000 7f7690b65640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f768c0ca310 0x7f768c0ca7c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:18:00.709 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:00.702+0000 7f7690b65640 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f768c0cad00 con 0x7f768c0b1460 2023-11-26T07:18:00.709 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:00.702+0000 7f7690b65640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f768c0b1e20 con 0x7f768c15d5b0 2023-11-26T07:18:00.709 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:00.702+0000 7f7690b65640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f768c0b8c00 con 0x7f768c0ca310 2023-11-26T07:18:00.709 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:00.706+0000 7f768ad9d640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f768c0ca310 0x7f768c0ca7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:18:00.709 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:00.706+0000 7f768a59c640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f768c15d5b0 0x7f768c15a9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:18:00.709 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:00.706+0000 7f768a59c640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f768c15d5b0 0x7f768c15a9f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.188:3300/0 says I am v2:172.21.15.143:45690/0 (socket says 172.21.15.143:45690) 2023-11-26T07:18:00.710 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:00.706+0000 7f768a59c640 1 -- 172.21.15.143:0/713141229 learned_addr learned my addr 172.21.15.143:0/713141229 (peer_addr_for_me v2:172.21.15.143:0/0) 2023-11-26T07:18:00.710 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:00.706+0000 7f8407832640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f83ec0047d0 0x7f840015fed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:18:00.710 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:00.706+0000 7f8407832640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f8400160810 0x7f84000acda0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:18:00.710 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:00.706+0000 7f8407832640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f84000aa640 0x7f84000aaac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:18:00.710 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:00.706+0000 7f8407832640 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f84000a98c0 con 0x7f84000aa640 2023-11-26T07:18:00.711 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:00.706+0000 7f8407832640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f84000ab000 con 0x7f83ec0047d0 2023-11-26T07:18:00.711 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:00.706+0000 7f8407832640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f84000ab210 con 0x7f8400160810 2023-11-26T07:18:00.711 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:00.706+0000 7f768a59c640 1 -- 172.21.15.143:0/713141229 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f768c0ca310 msgr2=0x7f768c0ca7c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:18:00.711 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:00.706+0000 7f768a59c640 1 --2- 172.21.15.143:0/713141229 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f768c0ca310 0x7f768c0ca7c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:18:00.711 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:00.706+0000 7f768a59c640 1 -- 172.21.15.143:0/713141229 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f768c0b1460 msgr2=0x7f768c0b18e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:18:00.711 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:00.706+0000 7f768a59c640 1 --2- 172.21.15.143:0/713141229 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f768c0b1460 0x7f768c0b18e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:18:00.712 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:00.706+0000 7f84055ce640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f83ec0047d0 0x7f840015fed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:18:00.712 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:00.706+0000 7f8404dcd640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f8400160810 0x7f84000acda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:18:00.712 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:00.706+0000 7f8404dcd640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f8400160810 0x7f84000acda0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.188:3301/0 says I am v2:172.21.15.143:45236/0 (socket says 172.21.15.143:45236) 2023-11-26T07:18:00.712 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:00.706+0000 7f8404dcd640 1 -- 172.21.15.143:0/3921722920 learned_addr learned my addr 172.21.15.143:0/3921722920 (peer_addr_for_me v2:172.21.15.143:0/0) 2023-11-26T07:18:00.713 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:00.706+0000 7f768a59c640 1 -- 172.21.15.143:0/713141229 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f768c0caef0 con 0x7f768c15d5b0 2023-11-26T07:18:00.713 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:00.706+0000 7f768ad9d640 1 --2- 172.21.15.143:0/713141229 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f768c0ca310 0x7f768c0ca7c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-26T07:18:00.713 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:00.706+0000 7f768a59c640 1 --2- 172.21.15.143:0/713141229 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f768c15d5b0 0x7f768c15a9f0 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7f767c011960 tx=0x7f767c011e20 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:18:00.713 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:00.706+0000 7f767affd640 1 -- 172.21.15.143:0/713141229 <== mon.1 v2:172.21.15.188:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f767c00faf0 con 0x7f768c15d5b0 2023-11-26T07:18:00.713 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:00.706+0000 7f767affd640 1 -- 172.21.15.143:0/713141229 <== mon.1 v2:172.21.15.188:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f767c00fc50 con 0x7f768c15d5b0 2023-11-26T07:18:00.713 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:00.706+0000 7f767affd640 1 -- 172.21.15.143:0/713141229 <== mon.1 v2:172.21.15.188:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f767c014440 con 0x7f768c15d5b0 2023-11-26T07:18:00.713 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:00.706+0000 7f7690b65640 1 -- 172.21.15.143:0/713141229 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f768c0cb1d0 con 0x7f768c15d5b0 2023-11-26T07:18:00.713 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:00.706+0000 7f7690b65640 1 -- 172.21.15.143:0/713141229 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f768c0cb790 con 0x7f768c15d5b0 2023-11-26T07:18:00.714 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:00.706+0000 7f8404dcd640 1 -- 172.21.15.143:0/3921722920 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f83ec0047d0 msgr2=0x7f840015fed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:18:00.714 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:00.706+0000 7f8404dcd640 1 --2- 172.21.15.143:0/3921722920 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f83ec0047d0 0x7f840015fed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:18:00.714 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:00.706+0000 7f8404dcd640 1 -- 172.21.15.143:0/3921722920 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f84000aa640 msgr2=0x7f84000aaac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:18:00.714 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:00.706+0000 7f8404dcd640 1 --2- 172.21.15.143:0/3921722920 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f84000aa640 0x7f84000aaac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:18:00.714 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:00.706+0000 7f8404dcd640 1 -- 172.21.15.143:0/3921722920 --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f84000a9ce0 con 0x7f8400160810 2023-11-26T07:18:00.714 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:00.706+0000 7f84055ce640 1 --2- 172.21.15.143:0/3921722920 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f83ec0047d0 0x7f840015fed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-26T07:18:00.714 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:00.710+0000 7f7690b65640 1 -- 172.21.15.143:0/713141229 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f768c067680 con 0x7f768c15d5b0 2023-11-26T07:18:00.715 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:00.710+0000 7f767affd640 1 -- 172.21.15.143:0/713141229 <== mon.1 v2:172.21.15.188:3300/0 4 ==== mgrmap(e 5) v1 ==== 85852+0+0 (secure 0 0 0) 0x7f767c0145a0 con 0x7f768c15d5b0 2023-11-26T07:18:00.715 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:00.710+0000 7fec9fe5a640 1 Processor -- start 2023-11-26T07:18:00.715 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:00.710+0000 7f767affd640 1 --2- 172.21.15.143:0/713141229 >> [v2:172.21.15.143:6832/2150557544,v1:172.21.15.143:6833/2150557544] conn(0x7f766c066470 0x7f766c068930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:18:00.716 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:00.710+0000 7fec9fe5a640 1 -- start start 2023-11-26T07:18:00.716 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:00.710+0000 7f767affd640 1 -- 172.21.15.143:0/713141229 <== mon.1 v2:172.21.15.188:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f767c085c20 con 0x7f768c15d5b0 2023-11-26T07:18:00.716 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:00.710+0000 7f8ff28f5640 1 Processor -- start 2023-11-26T07:18:00.716 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:00.710+0000 7f8404dcd640 1 --2- 172.21.15.143:0/3921722920 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f8400160810 0x7f84000acda0 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7f83f000c290 tx=0x7f83ec005800 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:18:00.717 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:00.710+0000 7f7689d9b640 1 --2- 172.21.15.143:0/713141229 >> [v2:172.21.15.143:6832/2150557544,v1:172.21.15.143:6833/2150557544] conn(0x7f766c066470 0x7f766c068930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:18:00.717 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:00.714+0000 7f7689d9b640 1 --2- 172.21.15.143:0/713141229 >> [v2:172.21.15.143:6832/2150557544,v1:172.21.15.143:6833/2150557544] conn(0x7f766c066470 0x7f766c068930 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7f7680007cf0 tx=0x7f7680007c60 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:18:00.717 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:00.714+0000 7f83f5ffb640 1 -- 172.21.15.143:0/3921722920 <== mon.2 v2:172.21.15.188:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f83f00b4070 con 0x7f8400160810 2023-11-26T07:18:00.718 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:00.714+0000 7fec9fe5a640 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7fec98070d10 0x7fec9815d900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:18:00.718 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:00.714+0000 7fec9fe5a640 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7fec9806cd50 0x7fec980b2180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:18:00.718 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:00.714+0000 7fec9fe5a640 1 -- --> v1:172.21.15.143:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fec9815b2f0 con 0x7fec980cf280 2023-11-26T07:18:00.718 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:00.714+0000 7fec9fe5a640 1 -- --> v2:172.21.15.143:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fec9806c5c0 con 0x7fec9806cd50 2023-11-26T07:18:00.719 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:00.714+0000 7fec9fe5a640 1 -- --> v2:172.21.15.188:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fec9815e1a0 con 0x7fec98070d10 2023-11-26T07:18:00.719 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:00.714+0000 7f83f5ffb640 1 -- 172.21.15.143:0/3921722920 <== mon.2 v2:172.21.15.188:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f83f0001600 con 0x7f8400160810 2023-11-26T07:18:00.719 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:00.714+0000 7f83f5ffb640 1 -- 172.21.15.143:0/3921722920 <== mon.2 v2:172.21.15.188:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f83f00a5c50 con 0x7f8400160810 2023-11-26T07:18:00.719 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:00.714+0000 7f8407832640 1 -- 172.21.15.143:0/3921722920 --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f84000ab4b0 con 0x7f8400160810 2023-11-26T07:18:00.719 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:00.714+0000 7f8ff28f5640 1 -- start start 2023-11-26T07:18:00.720 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:00.714+0000 7f8407832640 1 -- 172.21.15.143:0/3921722920 --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8400160410 con 0x7f8400160810 2023-11-26T07:18:00.720 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:00.714+0000 7fec9d3f5640 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7fec98070d10 0x7fec9815d900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:18:00.720 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:00.714+0000 7fec9d3f5640 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7fec98070d10 0x7fec9815d900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.188:3300/0 says I am v2:172.21.15.143:45716/0 (socket says 172.21.15.143:45716) 2023-11-26T07:18:00.720 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:00.714+0000 7fec9d3f5640 1 -- 172.21.15.143:0/3495777311 learned_addr learned my addr 172.21.15.143:0/3495777311 (peer_addr_for_me v2:172.21.15.143:0/0) 2023-11-26T07:18:00.720 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:00.714+0000 7fec9e3f7640 1 --2- 172.21.15.143:0/3495777311 >> v2:172.21.15.143:3300/0 conn(0x7fec9806cd50 0x7fec980b2180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:18:00.721 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:00.714+0000 7f8ff28f5640 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7f8fec0722f0 0x7f8fec153240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:18:00.721 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:00.714+0000 7f8ff28f5640 1 -- --> v1:172.21.15.143:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8fec15d5b0 con 0x7f8fec0b34f0 2023-11-26T07:18:00.721 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:00.714+0000 7f8ff28f5640 1 -- --> v1:172.21.15.188:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8fec15d890 con 0x7f8fec0b3110 2023-11-26T07:18:00.721 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:00.714+0000 7f8ff28f5640 1 -- --> v2:172.21.15.143:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8fec15db70 con 0x7f8fec0722f0 2023-11-26T07:18:00.725 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:00.714+0000 7fec9e3f7640 1 -- 172.21.15.143:0/3495777311 >> v2:172.21.15.188:3300/0 conn(0x7fec98070d10 msgr2=0x7fec9815d900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-26T07:18:00.725 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:00.714+0000 7fec9e3f7640 1 --2- 172.21.15.143:0/3495777311 >> v2:172.21.15.188:3300/0 conn(0x7fec98070d10 0x7fec9815d900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:18:00.725 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:00.714+0000 7fec9e3f7640 1 -- 172.21.15.143:0/3495777311 >> v1:172.21.15.143:6789/0 conn(0x7fec980cf280 legacy=0x7fec980cf660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-26T07:18:00.725 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:00.718+0000 7fec9e3f7640 1 -- 172.21.15.143:0/3495777311 --> v2:172.21.15.143:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fec9815e480 con 0x7fec9806cd50 2023-11-26T07:18:00.725 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:00.718+0000 7f8407832640 1 -- 172.21.15.143:0/3921722920 --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f84000ab210 con 0x7f8400160810 2023-11-26T07:18:00.725 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:00.718+0000 7f83f5ffb640 1 -- 172.21.15.143:0/3921722920 <== mon.2 v2:172.21.15.188:3301/0 4 ==== mgrmap(e 5) v1 ==== 85852+0+0 (secure 0 0 0) 0x7f83f0007070 con 0x7f8400160810 2023-11-26T07:18:00.726 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:00.718+0000 7f83f5ffb640 1 --2- 172.21.15.143:0/3921722920 >> [v2:172.21.15.143:6832/2150557544,v1:172.21.15.143:6833/2150557544] conn(0x7f83e40664f0 0x7f83e40689b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:18:00.726 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:00.718+0000 7f83f5ffb640 1 -- 172.21.15.143:0/3921722920 <== mon.2 v2:172.21.15.188:3301/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f83f011d610 con 0x7f8400160810 2023-11-26T07:18:00.726 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:00.718+0000 7f84055ce640 1 --2- 172.21.15.143:0/3921722920 >> [v2:172.21.15.143:6832/2150557544,v1:172.21.15.143:6833/2150557544] conn(0x7f83e40664f0 0x7f83e40689b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:18:00.726 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:00.714+0000 7f767affd640 1 -- 172.21.15.143:0/713141229 <== mon.1 v2:172.21.15.188:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172649 (secure 0 0 0) 0x7f767c089060 con 0x7f768c15d5b0 2023-11-26T07:18:00.727 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:00.718+0000 7f8ff18f3640 1 --1- >> v1:172.21.15.188:6789/0 conn(0x7f8fec0b3110 0x7f8fec15cd60 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.188:6789/0 says I am v1:172.21.15.143:40014/0 (socket says 172.21.15.143:40014) 2023-11-26T07:18:00.727 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:00.718+0000 7f8ff18f3640 1 -- 172.21.15.143:0/1718063979 learned_addr learned my addr 172.21.15.143:0/1718063979 (peer_addr_for_me v1:172.21.15.143:0/0) 2023-11-26T07:18:00.727 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:00.718+0000 7f8ff08f1640 1 -- 172.21.15.143:0/1718063979 <== mon.1 v1:172.21.15.188:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 926243717 0 0) 0x7f8fec15d890 con 0x7f8fec0b3110 2023-11-26T07:18:00.727 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:00.718+0000 7f8ff08f1640 1 -- 172.21.15.143:0/1718063979 --> v1:172.21.15.188:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f8fd8003530 con 0x7f8fec0b3110 2023-11-26T07:18:00.727 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:00.718+0000 7f8ff08f1640 1 -- 172.21.15.143:0/1718063979 <== mon.1 v1:172.21.15.188:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1619540557 0 0) 0x7f8fd8003530 con 0x7f8fec0b3110 2023-11-26T07:18:00.727 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:00.718+0000 7f8ff08f1640 1 -- 172.21.15.143:0/1718063979 >> v2:172.21.15.143:3300/0 conn(0x7f8fec0722f0 msgr2=0x7f8fec153240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-26T07:18:00.727 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:00.718+0000 7f8ff08f1640 1 --2- 172.21.15.143:0/1718063979 >> v2:172.21.15.143:3300/0 conn(0x7f8fec0722f0 0x7f8fec153240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:18:00.727 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:00.718+0000 7f8ff08f1640 1 -- 172.21.15.143:0/1718063979 >> v1:172.21.15.143:6789/0 conn(0x7f8fec0b34f0 legacy=0x7f8fec15a080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-26T07:18:00.727 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:00.718+0000 7f8ff08f1640 1 -- 172.21.15.143:0/1718063979 --> v1:172.21.15.188:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8fec159880 con 0x7f8fec0b3110 2023-11-26T07:18:00.727 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:00.718+0000 7f8ff08f1640 1 -- 172.21.15.143:0/1718063979 <== mon.1 v1:172.21.15.188:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2137882523 0 0) 0x7f8fe4001a00 con 0x7f8fec0b3110 2023-11-26T07:18:00.728 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:00.722+0000 7f8ff08f1640 1 -- 172.21.15.143:0/1718063979 >> v1:172.21.15.188:6789/0 conn(0x7f8fec0b3110 legacy=0x7f8fec15cd60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:18:00.728 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:00.722+0000 7f8ff08f1640 1 --2- 172.21.15.143:0/1718063979 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f8fd8003770 0x7f8fd8003b70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:18:00.728 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:00.722+0000 7f8ff08f1640 1 --2- 172.21.15.143:0/1718063979 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f8fd80041d0 0x7f8fd800b020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:18:00.728 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:00.722+0000 7f8ff08f1640 1 --2- 172.21.15.143:0/1718063979 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f8fd800b660 0x7f8fd800db20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:18:00.728 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:00.722+0000 7f8ff08f1640 1 -- 172.21.15.143:0/1718063979 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8fec15db70 con 0x7f8fd8003770 2023-11-26T07:18:00.728 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:00.722+0000 7f8ff08f1640 1 -- 172.21.15.143:0/1718063979 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8fd800e400 con 0x7f8fd800b660 2023-11-26T07:18:00.728 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:00.722+0000 7f8ff08f1640 1 -- 172.21.15.143:0/1718063979 --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8fd800e5c0 con 0x7f8fd80041d0 2023-11-26T07:18:00.728 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:00.722+0000 7f8ff20f4640 1 --2- 172.21.15.143:0/1718063979 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f8fd800b660 0x7f8fd800db20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:18:00.728 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:00.722+0000 7f84055ce640 1 --2- 172.21.15.143:0/3921722920 >> [v2:172.21.15.143:6832/2150557544,v1:172.21.15.143:6833/2150557544] conn(0x7f83e40664f0 0x7f83e40689b0 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7f83fc007fd0 tx=0x7f83fc007cd0 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:18:00.729 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:00.722+0000 7f8ff20f4640 1 -- 172.21.15.143:0/1718063979 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f8fd80041d0 msgr2=0x7f8fd800b020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-26T07:18:00.729 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:00.722+0000 7f8ff10f2640 1 --2- 172.21.15.143:0/1718063979 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f8fd80041d0 0x7f8fd800b020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:18:00.729 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:00.722+0000 7fec9e3f7640 1 --2- 172.21.15.143:0/3495777311 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fec9806cd50 0x7fec980b2180 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7fec9806e480 tx=0x7fec9001f0c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7bf68c317f03a235 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:18:00.729 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:00.722+0000 7f83f5ffb640 1 -- 172.21.15.143:0/3921722920 <== mon.2 v2:172.21.15.188:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172649 (secure 0 0 0) 0x7f83f00eabb0 con 0x7f8400160810 2023-11-26T07:18:00.730 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:00.722+0000 7fec9cbf4640 1 -- 172.21.15.143:0/3495777311 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fec900b0440 con 0x7fec9806cd50 2023-11-26T07:18:00.730 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:00.722+0000 7fec9cbf4640 1 -- 172.21.15.143:0/3495777311 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fec900b05a0 con 0x7fec9806cd50 2023-11-26T07:18:00.730 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:00.722+0000 7fec9cbf4640 1 -- 172.21.15.143:0/3495777311 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fec900b0930 con 0x7fec9806cd50 2023-11-26T07:18:00.730 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:00.726+0000 7fec9fe5a640 1 -- 172.21.15.143:0/3495777311 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fec9806cd50 msgr2=0x7fec980b2180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:18:00.731 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:00.726+0000 7fec9fe5a640 1 --2- 172.21.15.143:0/3495777311 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fec9806cd50 0x7fec980b2180 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7fec9806e480 tx=0x7fec9001f0c0 comp rx=0 tx=0).stop 2023-11-26T07:18:00.731 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:00.726+0000 7fec9fe5a640 1 -- 172.21.15.143:0/3495777311 shutdown_connections 2023-11-26T07:18:00.731 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:00.722+0000 7f8ff20f4640 1 --2- 172.21.15.143:0/1718063979 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f8fd80041d0 0x7f8fd800b020 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:18:00.731 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:00.722+0000 7f8ff20f4640 1 -- 172.21.15.143:0/1718063979 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f8fd8003770 msgr2=0x7f8fd8003b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-26T07:18:00.731 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:00.722+0000 7f8ff20f4640 1 --2- 172.21.15.143:0/1718063979 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f8fd8003770 0x7f8fd8003b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:18:00.731 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:00.722+0000 7f8ff20f4640 1 -- 172.21.15.143:0/1718063979 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8fd8003530 con 0x7f8fd800b660 2023-11-26T07:18:00.731 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:00.726+0000 7f8ff20f4640 1 -- 172.21.15.143:0/1718063979 reap_dead start 2023-11-26T07:18:00.732 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:00.726+0000 7fec9fe5a640 1 --2- 172.21.15.143:0/3495777311 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fec9806cd50 0x7fec980b2180 unknown :-1 s=CLOSED pgs=108 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:18:00.732 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:00.726+0000 7fec9fe5a640 1 --2- 172.21.15.143:0/3495777311 >> v2:172.21.15.188:3300/0 conn(0x7fec98070d10 0x7fec9815d900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:18:00.732 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:00.726+0000 7fec9fe5a640 1 -- 172.21.15.143:0/3495777311 >> 172.21.15.143:0/3495777311 conn(0x7fec9806a890 msgr2=0x7fec9806aca0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:18:00.732 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:00.726+0000 7f8ff20f4640 1 --2- 172.21.15.143:0/1718063979 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f8fd800b660 0x7f8fd800db20 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7f8fe8009420 tx=0x7f8fe8020770 comp rx=0 tx=0).ready entity=mon.1 client_cookie=fb8bfc3e588eb9ba server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:18:00.733 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:00.726+0000 7fec9fe5a640 1 -- 172.21.15.143:0/3495777311 shutdown_connections 2023-11-26T07:18:00.733 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:00.726+0000 7fec9fe5a640 1 -- 172.21.15.143:0/3495777311 wait complete. 2023-11-26T07:18:00.734 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:00.726+0000 7f8ff08f1640 1 -- 172.21.15.143:0/1718063979 <== mon.1 v2:172.21.15.188:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8fe80aa040 con 0x7f8fd800b660 2023-11-26T07:18:00.734 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:00.726+0000 7f8ff08f1640 1 -- 172.21.15.143:0/1718063979 <== mon.1 v2:172.21.15.188:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8fe8004240 con 0x7f8fd800b660 2023-11-26T07:18:00.734 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:00.726+0000 7f8ff08f1640 1 -- 172.21.15.143:0/1718063979 <== mon.1 v2:172.21.15.188:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8fe8003600 con 0x7f8fd800b660 2023-11-26T07:18:00.734 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:00.730+0000 7f8ff28f5640 1 -- 172.21.15.143:0/1718063979 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f8fd800b660 msgr2=0x7f8fd800db20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:18:00.734 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:00.730+0000 7f8ff28f5640 1 --2- 172.21.15.143:0/1718063979 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f8fd800b660 0x7f8fd800db20 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7f8fe8009420 tx=0x7f8fe8020770 comp rx=0 tx=0).stop 2023-11-26T07:18:00.734 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:00.730+0000 7f8ff28f5640 1 -- 172.21.15.143:0/1718063979 shutdown_connections 2023-11-26T07:18:00.734 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:00.730+0000 7f8ff28f5640 1 --2- 172.21.15.143:0/1718063979 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f8fd800b660 0x7f8fd800db20 unknown :-1 s=CLOSED pgs=79 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:18:00.734 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:00.730+0000 7f8ff28f5640 1 -- 172.21.15.143:0/1718063979 >> 172.21.15.143:0/1718063979 conn(0x7f8fec0b0aa0 msgr2=0x7f8fec0b0eb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:18:00.735 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:00.730+0000 7fec9fe5a640 1 Processor -- start 2023-11-26T07:18:00.735 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:00.730+0000 7f8ff28f5640 1 -- 172.21.15.143:0/1718063979 shutdown_connections 2023-11-26T07:18:00.735 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:00.730+0000 7f8ff28f5640 1 -- 172.21.15.143:0/1718063979 wait complete. 2023-11-26T07:18:00.735 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:00.730+0000 7fec9fe5a640 1 -- start start 2023-11-26T07:18:00.737 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:00.734+0000 7fec9fe5a640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fec9806cd50 0x7fec98159b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:18:00.737 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:00.734+0000 7fec9fe5a640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fec98070d10 0x7fec980b08d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:18:00.737 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:00.734+0000 7fec9dbf6640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fec9806cd50 0x7fec98159b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:18:00.737 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:00.734+0000 7f8ff28f5640 1 Processor -- start 2023-11-26T07:18:00.738 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:00.734+0000 7fec9dbf6640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fec9806cd50 0x7fec98159b00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.188:3300/0 says I am v2:172.21.15.143:45732/0 (socket says 172.21.15.143:45732) 2023-11-26T07:18:00.738 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:00.734+0000 7fec9d3f5640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fec98070d10 0x7fec980b08d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:18:00.738 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:00.734+0000 7fec9d3f5640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fec98070d10 0x7fec980b08d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:46604/0 (socket says 172.21.15.143:46604) 2023-11-26T07:18:00.739 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:00.734+0000 7fec9dbf6640 1 -- 172.21.15.143:0/286529952 learned_addr learned my addr 172.21.15.143:0/286529952 (peer_addr_for_me v2:172.21.15.143:0/0) 2023-11-26T07:18:00.739 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:00.734+0000 7fec9fe5a640 1 --2- 172.21.15.143:0/286529952 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fec980cf280 0x7fec980ab8a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:18:00.739 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:00.734+0000 7fec9fe5a640 1 -- 172.21.15.143:0/286529952 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fec980ab490 con 0x7fec98070d10 2023-11-26T07:18:00.739 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:00.734+0000 7fec9fe5a640 1 -- 172.21.15.143:0/286529952 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fec981f88c0 con 0x7fec9806cd50 2023-11-26T07:18:00.739 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:00.734+0000 7fec9fe5a640 1 -- 172.21.15.143:0/286529952 --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fec980aa910 con 0x7fec980cf280 2023-11-26T07:18:00.739 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:00.734+0000 7f8ff28f5640 1 -- start start 2023-11-26T07:18:00.740 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:00.734+0000 7fec9e3f7640 1 --2- 172.21.15.143:0/286529952 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fec980cf280 0x7fec980ab8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:18:00.740 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:00.734+0000 7fec9e3f7640 1 -- 172.21.15.143:0/286529952 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fec9806cd50 msgr2=0x7fec98159b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:18:00.740 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:00.734+0000 7fec9e3f7640 1 --2- 172.21.15.143:0/286529952 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fec9806cd50 0x7fec98159b00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:18:00.740 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:00.734+0000 7fec9e3f7640 1 -- 172.21.15.143:0/286529952 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fec98070d10 msgr2=0x7fec980b08d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:18:00.740 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:00.734+0000 7fec9e3f7640 1 --2- 172.21.15.143:0/286529952 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fec98070d10 0x7fec980b08d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:18:00.740 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:00.734+0000 7fec9e3f7640 1 -- 172.21.15.143:0/286529952 --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fec90009a70 con 0x7fec980cf280 2023-11-26T07:18:00.740 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:00.734+0000 7fec9d3f5640 1 --2- 172.21.15.143:0/286529952 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fec98070d10 0x7fec980b08d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-26T07:18:00.741 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:00.734+0000 7f8ff28f5640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f8fd800b660 0x7f8fec2029f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:18:00.741 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:00.734+0000 7fec9e3f7640 1 --2- 172.21.15.143:0/286529952 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fec980cf280 0x7fec980ab8a0 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7fec900af000 tx=0x7fec9001be60 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:18:00.742 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:00.734+0000 7f8ff18f3640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f8fd800b660 0x7f8fec2029f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:18:00.742 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:00.738+0000 7f8ff18f3640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f8fd800b660 0x7f8fec2029f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:46608/0 (socket says 172.21.15.143:46608) 2023-11-26T07:18:00.742 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:00.738+0000 7f8ff18f3640 1 -- 172.21.15.143:0/519090295 learned_addr learned my addr 172.21.15.143:0/519090295 (peer_addr_for_me v2:172.21.15.143:0/0) 2023-11-26T07:18:00.742 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:00.738+0000 7f8ff28f5640 1 --2- 172.21.15.143:0/519090295 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f8fec0ca870 0x7f8fec154230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:18:00.742 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:00.738+0000 7fec8e7fc640 1 -- 172.21.15.143:0/286529952 <== mon.2 v2:172.21.15.188:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fec900c1070 con 0x7fec980cf280 2023-11-26T07:18:00.742 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:00.738+0000 7fec8e7fc640 1 -- 172.21.15.143:0/286529952 <== mon.2 v2:172.21.15.188:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fec900b0e90 con 0x7fec980cf280 2023-11-26T07:18:00.742 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:00.738+0000 7fec8e7fc640 1 -- 172.21.15.143:0/286529952 <== mon.2 v2:172.21.15.188:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fec900b0440 con 0x7fec980cf280 2023-11-26T07:18:00.742 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:00.738+0000 7fec9fe5a640 1 -- 172.21.15.143:0/286529952 --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fec980aabf0 con 0x7fec980cf280 2023-11-26T07:18:00.743 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:00.738+0000 7fec9fe5a640 1 -- 172.21.15.143:0/286529952 --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fec980ab1b0 con 0x7fec980cf280 2023-11-26T07:18:00.743 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:00.738+0000 7f8ff28f5640 1 --2- 172.21.15.143:0/519090295 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f8fec154770 0x7f8fec154bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:18:00.743 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:00.738+0000 7f8ff28f5640 1 -- 172.21.15.143:0/519090295 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8fec202f30 con 0x7f8fd800b660 2023-11-26T07:18:00.743 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:00.738+0000 7f8ff28f5640 1 -- 172.21.15.143:0/519090295 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8fec0ca670 con 0x7f8fec154770 2023-11-26T07:18:00.744 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:00.738+0000 7f8ff28f5640 1 -- 172.21.15.143:0/519090295 --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8fec1590d0 con 0x7f8fec0ca870 2023-11-26T07:18:00.744 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:00.738+0000 7f8ff20f4640 1 --2- 172.21.15.143:0/519090295 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f8fec154770 0x7f8fec154bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:18:00.744 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:00.738+0000 7f8ff10f2640 1 --2- 172.21.15.143:0/519090295 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f8fec0ca870 0x7f8fec154230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:18:00.744 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:00.738+0000 7f8ff20f4640 1 -- 172.21.15.143:0/519090295 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f8fec0ca870 msgr2=0x7f8fec154230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:18:00.744 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:00.738+0000 7f8ff20f4640 1 --2- 172.21.15.143:0/519090295 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f8fec0ca870 0x7f8fec154230 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:18:00.744 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:00.738+0000 7f8ff20f4640 1 -- 172.21.15.143:0/519090295 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f8fd800b660 msgr2=0x7f8fec2029f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:18:00.744 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:00.738+0000 7fec9fe5a640 1 -- 172.21.15.143:0/286529952 --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fec98067680 con 0x7fec980cf280 2023-11-26T07:18:00.745 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:00.738+0000 7f8ff20f4640 1 --2- 172.21.15.143:0/519090295 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f8fd800b660 0x7f8fec2029f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:18:00.745 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:00.738+0000 7f8ff20f4640 1 -- 172.21.15.143:0/519090295 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8fec158b10 con 0x7f8fec154770 2023-11-26T07:18:00.745 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:00.738+0000 7f8ff20f4640 1 --2- 172.21.15.143:0/519090295 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f8fec154770 0x7f8fec154bf0 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7f8fec0b1e50 tx=0x7f8fe801d510 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:18:00.745 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:00.738+0000 7f8fe27fc640 1 -- 172.21.15.143:0/519090295 <== mon.1 v2:172.21.15.188:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8fe8010380 con 0x7f8fec154770 2023-11-26T07:18:00.745 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:00.738+0000 7f8fe27fc640 1 -- 172.21.15.143:0/519090295 <== mon.1 v2:172.21.15.188:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8fe8003b80 con 0x7f8fec154770 2023-11-26T07:18:00.746 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:00.738+0000 7f8fe27fc640 1 -- 172.21.15.143:0/519090295 <== mon.1 v2:172.21.15.188:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8fe8003ef0 con 0x7f8fec154770 2023-11-26T07:18:00.746 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:00.738+0000 7f8ff28f5640 1 -- 172.21.15.143:0/519090295 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8fec158df0 con 0x7f8fec154770 2023-11-26T07:18:00.746 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:00.738+0000 7f8ff28f5640 1 -- 172.21.15.143:0/519090295 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8fec0ca320 con 0x7f8fec154770 2023-11-26T07:18:00.746 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:00.742+0000 7f8ff18f3640 1 --2- 172.21.15.143:0/519090295 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f8fd800b660 0x7f8fec2029f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-26T07:18:00.748 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:00.742+0000 7fec8e7fc640 1 -- 172.21.15.143:0/286529952 <== mon.2 v2:172.21.15.188:3301/0 4 ==== mgrmap(e 5) v1 ==== 85852+0+0 (secure 0 0 0) 0x7fec900bc880 con 0x7fec980cf280 2023-11-26T07:18:00.748 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:00.742+0000 7fec8e7fc640 1 --2- 172.21.15.143:0/286529952 >> [v2:172.21.15.143:6832/2150557544,v1:172.21.15.143:6833/2150557544] conn(0x7fec7c066540 0x7fec7c068a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:18:00.748 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:00.742+0000 7fec8e7fc640 1 -- 172.21.15.143:0/286529952 <== mon.2 v2:172.21.15.188:3301/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fec90121fa0 con 0x7fec980cf280 2023-11-26T07:18:00.748 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:00.742+0000 7fec9dbf6640 1 --2- 172.21.15.143:0/286529952 >> [v2:172.21.15.143:6832/2150557544,v1:172.21.15.143:6833/2150557544] conn(0x7fec7c066540 0x7fec7c068a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:18:00.749 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:00.742+0000 7f8fe27fc640 1 -- 172.21.15.143:0/519090295 <== mon.1 v2:172.21.15.188:3300/0 4 ==== mgrmap(e 5) v1 ==== 85852+0+0 (secure 0 0 0) 0x7f8fe80b8450 con 0x7f8fec154770 2023-11-26T07:18:00.749 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:00.742+0000 7f8ff28f5640 1 -- 172.21.15.143:0/519090295 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8fec067500 con 0x7f8fec154770 2023-11-26T07:18:00.749 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:00.742+0000 7f8fe27fc640 1 --2- 172.21.15.143:0/519090295 >> [v2:172.21.15.143:6832/2150557544,v1:172.21.15.143:6833/2150557544] conn(0x7f8fd00664f0 0x7f8fd00689b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:18:00.749 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:00.746+0000 7f8ff18f3640 1 --2- 172.21.15.143:0/519090295 >> [v2:172.21.15.143:6832/2150557544,v1:172.21.15.143:6833/2150557544] conn(0x7f8fd00664f0 0x7f8fd00689b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:18:00.749 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:00.746+0000 7f8fe27fc640 1 -- 172.21.15.143:0/519090295 <== mon.1 v2:172.21.15.188:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f8fe811d970 con 0x7f8fec154770 2023-11-26T07:18:00.750 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:00.746+0000 7f8ff18f3640 1 --2- 172.21.15.143:0/519090295 >> [v2:172.21.15.143:6832/2150557544,v1:172.21.15.143:6833/2150557544] conn(0x7f8fd00664f0 0x7f8fd00689b0 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7f8fe4001340 tx=0x7f8fe40018f0 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:18:00.750 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:00.746+0000 7fec8e7fc640 1 -- 172.21.15.143:0/286529952 <== mon.2 v2:172.21.15.188:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172649 (secure 0 0 0) 0x7fec900ef660 con 0x7fec980cf280 2023-11-26T07:18:00.755 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:00.750+0000 7fec9dbf6640 1 --2- 172.21.15.143:0/286529952 >> [v2:172.21.15.143:6832/2150557544,v1:172.21.15.143:6833/2150557544] conn(0x7fec7c066540 0x7fec7c068a00 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7fec940050b0 tx=0x7fec94003700 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:18:00.756 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:00.750+0000 7f8fe27fc640 1 -- 172.21.15.143:0/519090295 <== mon.1 v2:172.21.15.188:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172649 (secure 0 0 0) 0x7f8fe80eaf60 con 0x7f8fec154770 2023-11-26T07:18:01.006 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:01.002+0000 7f2146962640 1 -- 172.21.15.143:0/3175475790 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 3} v 0) v1 -- 0x7f2140067500 con 0x7f214006d010 2023-11-26T07:18:01.006 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:01.002+0000 7f21367fc640 1 -- 172.21.15.143:0/3175475790 <== mon.1 v2:172.21.15.188:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 3}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7f21300bb090 con 0x7f214006d010 2023-11-26T07:18:01.006 INFO:teuthology.orchestra.run.smithi143.stdout:55834574855 2023-11-26T07:18:01.009 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:01.006+0000 7f2146962640 1 -- 172.21.15.143:0/3175475790 >> [v2:172.21.15.143:6832/2150557544,v1:172.21.15.143:6833/2150557544] conn(0x7f2124066470 msgr2=0x7f2124068930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:18:01.009 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:01.006+0000 7f2146962640 1 --2- 172.21.15.143:0/3175475790 >> [v2:172.21.15.143:6832/2150557544,v1:172.21.15.143:6833/2150557544] conn(0x7f2124066470 0x7f2124068930 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7f2140158a10 tx=0x7f213c007c30 comp rx=0 tx=0).stop 2023-11-26T07:18:01.009 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:01.006+0000 7f2146962640 1 -- 172.21.15.143:0/3175475790 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f214006d010 msgr2=0x7f21401545e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:18:01.009 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:01.006+0000 7f2146962640 1 --2- 172.21.15.143:0/3175475790 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f214006d010 0x7f21401545e0 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7f2130019e80 tx=0x7f213001aa90 comp rx=0 tx=0).stop 2023-11-26T07:18:01.013 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:01.010+0000 7f2146962640 1 -- 172.21.15.143:0/3175475790 shutdown_connections 2023-11-26T07:18:01.013 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:01.010+0000 7f2146962640 1 --2- 172.21.15.143:0/3175475790 >> [v2:172.21.15.143:6832/2150557544,v1:172.21.15.143:6833/2150557544] conn(0x7f2124066470 0x7f2124068930 unknown :-1 s=CLOSED pgs=87 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:18:01.013 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:01.010+0000 7f2146962640 1 --2- 172.21.15.143:0/3175475790 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f2140161160 0x7f2140164890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:18:01.013 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:01.010+0000 7f2146962640 1 --2- 172.21.15.143:0/3175475790 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f2140159270 0x7f2140160c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:18:01.013 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:01.010+0000 7f2146962640 1 --2- 172.21.15.143:0/3175475790 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f214006d010 0x7f21401545e0 unknown :-1 s=CLOSED pgs=73 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:18:01.013 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:01.010+0000 7f2146962640 1 -- 172.21.15.143:0/3175475790 >> 172.21.15.143:0/3175475790 conn(0x7f214006a890 msgr2=0x7f21400b0a80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:18:01.013 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:01.010+0000 7f2146962640 1 -- 172.21.15.143:0/3175475790 shutdown_connections 2023-11-26T07:18:01.013 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:01.010+0000 7f2146962640 1 -- 172.21.15.143:0/3175475790 wait complete. 2023-11-26T07:18:01.017 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:01.014+0000 7f7f6a2c9640 1 -- 172.21.15.143:0/2197969909 --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 2} v 0) v1 -- 0x7f7f6415c760 con 0x7f7f64159760 2023-11-26T07:18:01.018 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:01.014+0000 7f7f60ff9640 1 -- 172.21.15.143:0/2197969909 <== mon.2 v2:172.21.15.188:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 2}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7f7f5c01a020 con 0x7f7f64159760 2023-11-26T07:18:01.018 INFO:teuthology.orchestra.run.smithi143.stdout:55834574855 2023-11-26T07:18:01.022 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:01.018+0000 7f7f6a2c9640 1 -- 172.21.15.143:0/2197969909 >> [v2:172.21.15.143:6832/2150557544,v1:172.21.15.143:6833/2150557544] conn(0x7f7f44077550 msgr2=0x7f7f44079a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:18:01.022 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:01.018+0000 7f7f6a2c9640 1 --2- 172.21.15.143:0/2197969909 >> [v2:172.21.15.143:6832/2150557544,v1:172.21.15.143:6833/2150557544] conn(0x7f7f44077550 0x7f7f44079a10 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7f7f540ae000 tx=0x7f7f54008040 comp rx=0 tx=0).stop 2023-11-26T07:18:01.022 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:01.018+0000 7f7f6a2c9640 1 -- 172.21.15.143:0/2197969909 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f7f64159760 msgr2=0x7f7f641febc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:18:01.022 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:01.018+0000 7f7f6a2c9640 1 --2- 172.21.15.143:0/2197969909 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f7f64159760 0x7f7f641febc0 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7f7f5c00cef0 tx=0x7f7f5c00a710 comp rx=0 tx=0).stop 2023-11-26T07:18:01.023 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:01.018+0000 7f7f6a2c9640 1 -- 172.21.15.143:0/2197969909 shutdown_connections 2023-11-26T07:18:01.023 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:01.018+0000 7f7f6a2c9640 1 --2- 172.21.15.143:0/2197969909 >> [v2:172.21.15.143:6832/2150557544,v1:172.21.15.143:6833/2150557544] conn(0x7f7f44077550 0x7f7f44079a10 unknown :-1 s=CLOSED pgs=85 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:18:01.023 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:01.018+0000 7f7f6a2c9640 1 --2- 172.21.15.143:0/2197969909 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f7f64159760 0x7f7f641febc0 unknown :-1 s=CLOSED pgs=45 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:18:01.023 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:01.018+0000 7f7f6a2c9640 1 --2- 172.21.15.143:0/2197969909 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f7f64071e20 0x7f7f641fe680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:18:01.024 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:01.018+0000 7f7f6a2c9640 1 --2- 172.21.15.143:0/2197969909 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f7f640708b0 0x7f7f6414ffd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:18:01.024 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:01.018+0000 7f7f6a2c9640 1 -- 172.21.15.143:0/2197969909 >> 172.21.15.143:0/2197969909 conn(0x7f7f640b0bf0 msgr2=0x7f7f640cec10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:18:01.024 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:01.018+0000 7f7f6a2c9640 1 -- 172.21.15.143:0/2197969909 shutdown_connections 2023-11-26T07:18:01.024 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:01.018+0000 7f7f6a2c9640 1 -- 172.21.15.143:0/2197969909 wait complete. 2023-11-26T07:18:01.024 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:01.018+0000 7f72253a0640 1 -- 172.21.15.143:0/1951363296 --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 0} v 0) v1 -- 0x7f72000057e0 con 0x7f72180e30a0 2023-11-26T07:18:01.025 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:01.018+0000 7f721cff9640 1 -- 172.21.15.143:0/1951363296 <== mon.2 v2:172.21.15.188:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 0}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7f72140b5030 con 0x7f72180e30a0 2023-11-26T07:18:01.025 INFO:teuthology.orchestra.run.smithi143.stdout:55834574854 2023-11-26T07:18:01.025 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:01.022+0000 7fe18046b640 1 -- 172.21.15.143:0/1551758953 --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 1} v 0) v1 -- 0x7fe178144c90 con 0x7fe178159170 2023-11-26T07:18:01.026 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:01.022+0000 7fe16effd640 1 -- 172.21.15.143:0/1551758953 <== mon.2 v2:172.21.15.188:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 1}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7fe178144c90 con 0x7fe178159170 2023-11-26T07:18:01.028 INFO:teuthology.orchestra.run.smithi143.stdout:55834574854 2023-11-26T07:18:01.029 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:01.026+0000 7f72253a0640 1 -- 172.21.15.143:0/1951363296 >> [v2:172.21.15.143:6832/2150557544,v1:172.21.15.143:6833/2150557544] conn(0x7f7204066530 msgr2=0x7f72040689f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:18:01.029 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:01.026+0000 7fe18046b640 1 -- 172.21.15.143:0/1551758953 >> [v2:172.21.15.143:6832/2150557544,v1:172.21.15.143:6833/2150557544] conn(0x7fe15c0664c0 msgr2=0x7fe15c068980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:18:01.029 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:01.026+0000 7fe18046b640 1 --2- 172.21.15.143:0/1551758953 >> [v2:172.21.15.143:6832/2150557544,v1:172.21.15.143:6833/2150557544] conn(0x7fe15c0664c0 0x7fe15c068980 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7fe168007fd0 tx=0x7fe168009040 comp rx=0 tx=0).stop 2023-11-26T07:18:01.029 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:01.026+0000 7fe18046b640 1 -- 172.21.15.143:0/1551758953 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fe178159170 msgr2=0x7fe1781547f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:18:01.029 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:01.026+0000 7fe18046b640 1 --2- 172.21.15.143:0/1551758953 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fe178159170 0x7fe1781547f0 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7fe17400dad0 tx=0x7fe17400df90 comp rx=0 tx=0).stop 2023-11-26T07:18:01.030 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:01.026+0000 7fe18046b640 1 -- 172.21.15.143:0/1551758953 shutdown_connections 2023-11-26T07:18:01.030 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:01.026+0000 7fe18046b640 1 --2- 172.21.15.143:0/1551758953 >> [v2:172.21.15.143:6832/2150557544,v1:172.21.15.143:6833/2150557544] conn(0x7fe15c0664c0 0x7fe15c068980 unknown :-1 s=CLOSED pgs=86 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:18:01.030 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:01.026+0000 7fe18046b640 1 --2- 172.21.15.143:0/1551758953 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fe178159170 0x7fe1781547f0 unknown :-1 s=CLOSED pgs=47 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:18:01.030 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:01.026+0000 7fe18046b640 1 --2- 172.21.15.143:0/1551758953 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fe1780ce310 0x7fe1781542b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:18:01.030 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:01.026+0000 7fe18046b640 1 --2- 172.21.15.143:0/1551758953 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fe1780708b0 0x7fe178163500 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:18:01.030 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:01.026+0000 7fe18046b640 1 -- 172.21.15.143:0/1551758953 >> 172.21.15.143:0/1551758953 conn(0x7fe1780b0bf0 msgr2=0x7fe17814f8c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:18:01.031 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:01.026+0000 7fe18046b640 1 -- 172.21.15.143:0/1551758953 shutdown_connections 2023-11-26T07:18:01.031 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:01.026+0000 7fe18046b640 1 -- 172.21.15.143:0/1551758953 wait complete. 2023-11-26T07:18:01.032 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:01.026+0000 7f72253a0640 1 --2- 172.21.15.143:0/1951363296 >> [v2:172.21.15.143:6832/2150557544,v1:172.21.15.143:6833/2150557544] conn(0x7f7204066530 0x7f72040689f0 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7f7218033950 tx=0x7f720c007b10 comp rx=0 tx=0).stop 2023-11-26T07:18:01.033 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:01.026+0000 7f72253a0640 1 -- 172.21.15.143:0/1951363296 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f72180e30a0 msgr2=0x7f72180ce5f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:18:01.033 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:01.030+0000 7f72253a0640 1 --2- 172.21.15.143:0/1951363296 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f72180e30a0 0x7f72180ce5f0 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7f721401c8c0 tx=0x7f721401ba30 comp rx=0 tx=0).stop 2023-11-26T07:18:01.034 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:01.030+0000 7f72253a0640 1 -- 172.21.15.143:0/1951363296 shutdown_connections 2023-11-26T07:18:01.034 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:01.030+0000 7f72253a0640 1 --2- 172.21.15.143:0/1951363296 >> [v2:172.21.15.143:6832/2150557544,v1:172.21.15.143:6833/2150557544] conn(0x7f7204066530 0x7f72040689f0 unknown :-1 s=CLOSED pgs=88 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:18:01.034 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:01.030+0000 7f72253a0640 1 --2- 172.21.15.143:0/1951363296 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f72180e77f0 0x7f721817bff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:18:01.035 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:01.030+0000 7f72253a0640 1 --2- 172.21.15.143:0/1951363296 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f72180e6c00 0x7f721804ad30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:18:01.035 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:01.030+0000 7f72253a0640 1 --2- 172.21.15.143:0/1951363296 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f72180e30a0 0x7f72180ce5f0 unknown :-1 s=CLOSED pgs=46 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:18:01.035 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:01.030+0000 7f72253a0640 1 -- 172.21.15.143:0/1951363296 >> 172.21.15.143:0/1951363296 conn(0x7f7218009f90 msgr2=0x7f72180397c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:18:01.035 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:01.030+0000 7f72253a0640 1 -- 172.21.15.143:0/1951363296 shutdown_connections 2023-11-26T07:18:01.035 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:01.030+0000 7f72253a0640 1 -- 172.21.15.143:0/1951363296 wait complete. 2023-11-26T07:18:01.050 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574856 got 55834574855 for osd.3 2023-11-26T07:18:01.051 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:01.046+0000 7f7690b65640 1 -- 172.21.15.143:0/713141229 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 7} v 0) v1 -- 0x7f768c0b2ea0 con 0x7f768c15d5b0 2023-11-26T07:18:01.051 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:01.046+0000 7f767affd640 1 -- 172.21.15.143:0/713141229 <== mon.1 v2:172.21.15.188:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 7}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7f767c089020 con 0x7f768c15d5b0 2023-11-26T07:18:01.051 INFO:teuthology.orchestra.run.smithi143.stdout:55834574852 2023-11-26T07:18:01.053 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:01.050+0000 7f7690b65640 1 -- 172.21.15.143:0/713141229 >> [v2:172.21.15.143:6832/2150557544,v1:172.21.15.143:6833/2150557544] conn(0x7f766c066470 msgr2=0x7f766c068930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:18:01.054 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:01.050+0000 7f7690b65640 1 --2- 172.21.15.143:0/713141229 >> [v2:172.21.15.143:6832/2150557544,v1:172.21.15.143:6833/2150557544] conn(0x7f766c066470 0x7f766c068930 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7f7680007cf0 tx=0x7f7680007c60 comp rx=0 tx=0).stop 2023-11-26T07:18:01.054 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:01.050+0000 7f7690b65640 1 -- 172.21.15.143:0/713141229 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f768c15d5b0 msgr2=0x7f768c15a9f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:18:01.054 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:01.050+0000 7f7690b65640 1 --2- 172.21.15.143:0/713141229 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f768c15d5b0 0x7f768c15a9f0 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7f767c011960 tx=0x7f767c011e20 comp rx=0 tx=0).stop 2023-11-26T07:18:01.054 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:01.050+0000 7f7690b65640 1 -- 172.21.15.143:0/713141229 shutdown_connections 2023-11-26T07:18:01.054 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:01.050+0000 7f7690b65640 1 --2- 172.21.15.143:0/713141229 >> [v2:172.21.15.143:6832/2150557544,v1:172.21.15.143:6833/2150557544] conn(0x7f766c066470 0x7f766c068930 unknown :-1 s=CLOSED pgs=89 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:18:01.054 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:01.050+0000 7f7690b65640 1 --2- 172.21.15.143:0/713141229 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f768c0ca310 0x7f768c0ca7c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:18:01.054 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:01.050+0000 7f7690b65640 1 --2- 172.21.15.143:0/713141229 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f768c0b1460 0x7f768c0b18e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:18:01.055 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:01.050+0000 7f7690b65640 1 --2- 172.21.15.143:0/713141229 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f768c15d5b0 0x7f768c15a9f0 unknown :-1 s=CLOSED pgs=77 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:18:01.055 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:01.050+0000 7f7690b65640 1 -- 172.21.15.143:0/713141229 >> 172.21.15.143:0/713141229 conn(0x7f768c08cc60 msgr2=0x7f768c0b3170 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:18:01.055 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:01.050+0000 7f8407832640 1 -- 172.21.15.143:0/3921722920 --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 5} v 0) v1 -- 0x7f84000a9a00 con 0x7f8400160810 2023-11-26T07:18:01.055 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:01.050+0000 7f7690b65640 1 -- 172.21.15.143:0/713141229 shutdown_connections 2023-11-26T07:18:01.055 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:01.050+0000 7f7690b65640 1 -- 172.21.15.143:0/713141229 wait complete. 2023-11-26T07:18:01.056 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:01.050+0000 7f83f5ffb640 1 -- 172.21.15.143:0/3921722920 <== mon.2 v2:172.21.15.188:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 5}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7f84000a9a00 con 0x7f8400160810 2023-11-26T07:18:01.056 INFO:teuthology.orchestra.run.smithi143.stdout:55834574852 2023-11-26T07:18:01.061 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:01.058+0000 7f8407832640 1 -- 172.21.15.143:0/3921722920 >> [v2:172.21.15.143:6832/2150557544,v1:172.21.15.143:6833/2150557544] conn(0x7f83e40664f0 msgr2=0x7f83e40689b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:18:01.061 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:01.058+0000 7f8407832640 1 --2- 172.21.15.143:0/3921722920 >> [v2:172.21.15.143:6832/2150557544,v1:172.21.15.143:6833/2150557544] conn(0x7f83e40664f0 0x7f83e40689b0 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7f83fc007fd0 tx=0x7f83fc007cd0 comp rx=0 tx=0).stop 2023-11-26T07:18:01.061 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:01.058+0000 7f8407832640 1 -- 172.21.15.143:0/3921722920 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f8400160810 msgr2=0x7f84000acda0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:18:01.061 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:01.058+0000 7f8407832640 1 --2- 172.21.15.143:0/3921722920 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f8400160810 0x7f84000acda0 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7f83f000c290 tx=0x7f83ec005800 comp rx=0 tx=0).stop 2023-11-26T07:18:01.062 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:01.058+0000 7f8407832640 1 -- 172.21.15.143:0/3921722920 shutdown_connections 2023-11-26T07:18:01.062 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:01.058+0000 7f8407832640 1 --2- 172.21.15.143:0/3921722920 >> [v2:172.21.15.143:6832/2150557544,v1:172.21.15.143:6833/2150557544] conn(0x7f83e40664f0 0x7f83e40689b0 unknown :-1 s=CLOSED pgs=90 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:18:01.062 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:01.058+0000 7f8407832640 1 --2- 172.21.15.143:0/3921722920 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f84000aa640 0x7f84000aaac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:18:01.062 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:01.058+0000 7f8407832640 1 --2- 172.21.15.143:0/3921722920 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f8400160810 0x7f84000acda0 secure :-1 s=CLOSED pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7f83f000c290 tx=0x7f83ec005800 comp rx=0 tx=0).stop 2023-11-26T07:18:01.062 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:01.058+0000 7f8407832640 1 --2- 172.21.15.143:0/3921722920 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f83ec0047d0 0x7f840015fed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:18:01.062 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:01.058+0000 7f8407832640 1 -- 172.21.15.143:0/3921722920 >> 172.21.15.143:0/3921722920 conn(0x7f840006a890 msgr2=0x7f84000cf250 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:18:01.062 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:01.058+0000 7f8407832640 1 -- 172.21.15.143:0/3921722920 shutdown_connections 2023-11-26T07:18:01.062 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:01.058+0000 7f8407832640 1 -- 172.21.15.143:0/3921722920 wait complete. 2023-11-26T07:18:01.064 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574856 got 55834574855 for osd.2 2023-11-26T07:18:01.068 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574855 got 55834574854 for osd.1 2023-11-26T07:18:01.072 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:01.066+0000 7fec9fe5a640 1 -- 172.21.15.143:0/286529952 --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 4} v 0) v1 -- 0x7fec9815ba90 con 0x7fec980cf280 2023-11-26T07:18:01.073 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:01.070+0000 7fec8e7fc640 1 -- 172.21.15.143:0/286529952 <== mon.2 v2:172.21.15.188:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 4}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7fec900b5020 con 0x7fec980cf280 2023-11-26T07:18:01.073 INFO:teuthology.orchestra.run.smithi143.stdout:55834574854 2023-11-26T07:18:01.074 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574855 got 55834574854 for osd.0 2023-11-26T07:18:01.077 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:01.070+0000 7fec9fe5a640 1 -- 172.21.15.143:0/286529952 >> [v2:172.21.15.143:6832/2150557544,v1:172.21.15.143:6833/2150557544] conn(0x7fec7c066540 msgr2=0x7fec7c068a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:18:01.077 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:01.070+0000 7fec9fe5a640 1 --2- 172.21.15.143:0/286529952 >> [v2:172.21.15.143:6832/2150557544,v1:172.21.15.143:6833/2150557544] conn(0x7fec7c066540 0x7fec7c068a00 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7fec940050b0 tx=0x7fec94003700 comp rx=0 tx=0).stop 2023-11-26T07:18:01.077 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:01.070+0000 7fec9fe5a640 1 -- 172.21.15.143:0/286529952 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fec980cf280 msgr2=0x7fec980ab8a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:18:01.077 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:01.070+0000 7fec9fe5a640 1 --2- 172.21.15.143:0/286529952 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fec980cf280 0x7fec980ab8a0 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7fec900af000 tx=0x7fec9001be60 comp rx=0 tx=0).stop 2023-11-26T07:18:01.077 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:01.070+0000 7fec9fe5a640 1 -- 172.21.15.143:0/286529952 shutdown_connections 2023-11-26T07:18:01.077 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:01.070+0000 7fec9fe5a640 1 --2- 172.21.15.143:0/286529952 >> [v2:172.21.15.143:6832/2150557544,v1:172.21.15.143:6833/2150557544] conn(0x7fec7c066540 0x7fec7c068a00 unknown :-1 s=CLOSED pgs=92 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:18:01.077 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:01.070+0000 7fec9fe5a640 1 --2- 172.21.15.143:0/286529952 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fec980cf280 0x7fec980ab8a0 unknown :-1 s=CLOSED pgs=50 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:18:01.077 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:01.070+0000 7fec9fe5a640 1 --2- 172.21.15.143:0/286529952 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fec98070d10 0x7fec980b08d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:18:01.077 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:01.070+0000 7fec9fe5a640 1 --2- 172.21.15.143:0/286529952 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fec9806cd50 0x7fec98159b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:18:01.077 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:01.070+0000 7fec9fe5a640 1 -- 172.21.15.143:0/286529952 >> 172.21.15.143:0/286529952 conn(0x7fec9806a890 msgr2=0x7fec9815cdd0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:18:01.077 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:01.074+0000 7fec9fe5a640 1 -- 172.21.15.143:0/286529952 shutdown_connections 2023-11-26T07:18:01.078 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:01.074+0000 7fec9fe5a640 1 -- 172.21.15.143:0/286529952 wait complete. 2023-11-26T07:18:01.078 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:01.074+0000 7f8ff28f5640 1 -- 172.21.15.143:0/519090295 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 6} v 0) v1 -- 0x7f8fec071990 con 0x7f8fec154770 2023-11-26T07:18:01.078 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:01.074+0000 7f8fe27fc640 1 -- 172.21.15.143:0/519090295 <== mon.1 v2:172.21.15.188:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 6}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7f8fe80ee580 con 0x7f8fec154770 2023-11-26T07:18:01.078 INFO:teuthology.orchestra.run.smithi143.stdout:55834574854 2023-11-26T07:18:01.080 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574853 got 55834574852 for osd.7 2023-11-26T07:18:01.081 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:01.074+0000 7f8ff28f5640 1 -- 172.21.15.143:0/519090295 >> [v2:172.21.15.143:6832/2150557544,v1:172.21.15.143:6833/2150557544] conn(0x7f8fd00664f0 msgr2=0x7f8fd00689b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:18:01.081 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:01.074+0000 7f8ff28f5640 1 --2- 172.21.15.143:0/519090295 >> [v2:172.21.15.143:6832/2150557544,v1:172.21.15.143:6833/2150557544] conn(0x7f8fd00664f0 0x7f8fd00689b0 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7f8fe4001340 tx=0x7f8fe40018f0 comp rx=0 tx=0).stop 2023-11-26T07:18:01.081 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:01.074+0000 7f8ff28f5640 1 -- 172.21.15.143:0/519090295 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f8fec154770 msgr2=0x7f8fec154bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:18:01.081 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:01.074+0000 7f8ff28f5640 1 --2- 172.21.15.143:0/519090295 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f8fec154770 0x7f8fec154bf0 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7f8fec0b1e50 tx=0x7f8fe801d510 comp rx=0 tx=0).stop 2023-11-26T07:18:01.081 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:01.074+0000 7f8ff28f5640 1 -- 172.21.15.143:0/519090295 shutdown_connections 2023-11-26T07:18:01.081 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:01.074+0000 7f8ff28f5640 1 --2- 172.21.15.143:0/519090295 >> [v2:172.21.15.143:6832/2150557544,v1:172.21.15.143:6833/2150557544] conn(0x7f8fd00664f0 0x7f8fd00689b0 unknown :-1 s=CLOSED pgs=91 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:18:01.081 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:01.074+0000 7f8ff28f5640 1 --2- 172.21.15.143:0/519090295 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f8fec154770 0x7f8fec154bf0 unknown :-1 s=CLOSED pgs=80 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:18:01.081 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:01.074+0000 7f8ff28f5640 1 --2- 172.21.15.143:0/519090295 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f8fec0ca870 0x7f8fec154230 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:18:01.081 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:01.074+0000 7f8ff28f5640 1 --2- 172.21.15.143:0/519090295 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f8fd800b660 0x7f8fec2029f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:18:01.081 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:01.074+0000 7f8ff28f5640 1 -- 172.21.15.143:0/519090295 >> 172.21.15.143:0/519090295 conn(0x7f8fec0b0aa0 msgr2=0x7f8fec1527b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:18:01.081 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:01.074+0000 7f8ff28f5640 1 -- 172.21.15.143:0/519090295 shutdown_connections 2023-11-26T07:18:01.082 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:01.074+0000 7f8ff28f5640 1 -- 172.21.15.143:0/519090295 wait complete. 2023-11-26T07:18:01.096 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574853 got 55834574852 for osd.5 2023-11-26T07:18:01.103 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574855 got 55834574854 for osd.6 2023-11-26T07:18:01.104 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574855 got 55834574854 for osd.4 2023-11-26T07:18:02.052 DEBUG:teuthology.orchestra.run.smithi143:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.3 2023-11-26T07:18:02.065 DEBUG:teuthology.orchestra.run.smithi143:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.2 2023-11-26T07:18:02.068 DEBUG:teuthology.orchestra.run.smithi143:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.1 2023-11-26T07:18:02.075 DEBUG:teuthology.orchestra.run.smithi143:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.0 2023-11-26T07:18:02.081 DEBUG:teuthology.orchestra.run.smithi143:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.7 2023-11-26T07:18:02.097 DEBUG:teuthology.orchestra.run.smithi143:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.5 2023-11-26T07:18:02.104 DEBUG:teuthology.orchestra.run.smithi143:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.6 2023-11-26T07:18:02.105 DEBUG:teuthology.orchestra.run.smithi143:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.4 2023-11-26T07:18:02.266 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:02.262+0000 7f00c18e3640 1 Processor -- start 2023-11-26T07:18:02.268 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:02.262+0000 7f00c18e3640 1 -- start start 2023-11-26T07:18:02.269 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:02.262+0000 7f00c18e3640 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7f00bc06db20 0x7f00bc0b3e60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:18:02.269 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:02.262+0000 7f00c18e3640 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7f00bc0b20b0 0x7f00bc0b2520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:18:02.269 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:02.262+0000 7f00c18e3640 1 -- --> v1:172.21.15.143:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f00bc0cc790 con 0x7f00bc15a440 2023-11-26T07:18:02.269 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:02.262+0000 7f00c18e3640 1 -- --> v2:172.21.15.143:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f00bc0b4ae0 con 0x7f00bc0b20b0 2023-11-26T07:18:02.269 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:02.262+0000 7f00c18e3640 1 -- --> v2:172.21.15.188:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f00bc0b4800 con 0x7f00bc06db20 2023-11-26T07:18:02.270 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:02.266+0000 7f00baffd640 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7f00bc06db20 0x7f00bc0b3e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:18:02.270 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:02.266+0000 7f00baffd640 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7f00bc06db20 0x7f00bc0b3e60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.188:3300/0 says I am v2:172.21.15.143:45744/0 (socket says 172.21.15.143:45744) 2023-11-26T07:18:02.276 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:02.266+0000 7f00baffd640 1 -- 172.21.15.143:0/1995992616 learned_addr learned my addr 172.21.15.143:0/1995992616 (peer_addr_for_me v2:172.21.15.143:0/0) 2023-11-26T07:18:02.276 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:02.266+0000 7f00baffd640 1 -- 172.21.15.143:0/1995992616 >> v2:172.21.15.143:3300/0 conn(0x7f00bc0b20b0 msgr2=0x7f00bc0b2520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-26T07:18:02.276 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:02.266+0000 7f00baffd640 1 --2- 172.21.15.143:0/1995992616 >> v2:172.21.15.143:3300/0 conn(0x7f00bc0b20b0 0x7f00bc0b2520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:18:02.276 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:02.266+0000 7f00baffd640 1 -- 172.21.15.143:0/1995992616 >> v1:172.21.15.143:6789/0 conn(0x7f00bc15a440 legacy=0x7f00bc0cd4a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-26T07:18:02.276 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:02.266+0000 7f00baffd640 1 -- 172.21.15.143:0/1995992616 --> v2:172.21.15.188:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f00bc0cc9c0 con 0x7f00bc06db20 2023-11-26T07:18:02.276 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:02.270+0000 7f00baffd640 1 --2- 172.21.15.143:0/1995992616 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f00bc06db20 0x7f00bc0b3e60 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7f00bc0b5060 tx=0x7f00ac01e9c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=5af12c6bb37485c server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:18:02.278 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:02.274+0000 7f00b9ffb640 1 -- 172.21.15.143:0/1995992616 <== mon.1 v2:172.21.15.188:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f00ac0a7a90 con 0x7f00bc06db20 2023-11-26T07:18:02.278 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:02.274+0000 7f00b9ffb640 1 -- 172.21.15.143:0/1995992616 <== mon.1 v2:172.21.15.188:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f00ac0ab070 con 0x7f00bc06db20 2023-11-26T07:18:02.278 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:02.274+0000 7f00b9ffb640 1 -- 172.21.15.143:0/1995992616 <== mon.1 v2:172.21.15.188:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f00ac0a7e50 con 0x7f00bc06db20 2023-11-26T07:18:02.279 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:02.274+0000 7f00c18e3640 1 -- 172.21.15.143:0/1995992616 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f00bc06db20 msgr2=0x7f00bc0b3e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:18:02.279 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:02.274+0000 7f00c18e3640 1 --2- 172.21.15.143:0/1995992616 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f00bc06db20 0x7f00bc0b3e60 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7f00bc0b5060 tx=0x7f00ac01e9c0 comp rx=0 tx=0).stop 2023-11-26T07:18:02.280 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:02.274+0000 7f00c18e3640 1 -- 172.21.15.143:0/1995992616 shutdown_connections 2023-11-26T07:18:02.280 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:02.274+0000 7f00c18e3640 1 --2- 172.21.15.143:0/1995992616 >> v2:172.21.15.143:3300/0 conn(0x7f00bc0b20b0 0x7f00bc0b2520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:18:02.280 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:02.274+0000 7f00c18e3640 1 --2- 172.21.15.143:0/1995992616 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f00bc06db20 0x7f00bc0b3e60 secure :-1 s=CLOSED pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7f00bc0b5060 tx=0x7f00ac01e9c0 comp rx=0 tx=0).stop 2023-11-26T07:18:02.280 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:02.274+0000 7f00c18e3640 1 -- 172.21.15.143:0/1995992616 >> 172.21.15.143:0/1995992616 conn(0x7f00bc0aea10 msgr2=0x7f00bc0aee20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:18:02.280 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:02.274+0000 7f00c18e3640 1 -- 172.21.15.143:0/1995992616 shutdown_connections 2023-11-26T07:18:02.280 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:02.274+0000 7f00c18e3640 1 -- 172.21.15.143:0/1995992616 wait complete. 2023-11-26T07:18:02.283 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:02.278+0000 7f00c18e3640 1 Processor -- start 2023-11-26T07:18:02.283 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:02.278+0000 7f00c18e3640 1 -- start start 2023-11-26T07:18:02.285 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:02.282+0000 7f00c18e3640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f00bc0b20b0 0x7f00bc137b10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:18:02.285 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:02.282+0000 7f00c18e3640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f00bc15a440 0x7f00bc0a8950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:18:02.286 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:02.282+0000 7f00baffd640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f00bc0b20b0 0x7f00bc137b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:18:02.286 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:02.282+0000 7f00ba7fc640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f00bc15a440 0x7f00bc0a8950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:18:02.287 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:02.282+0000 7f00ba7fc640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f00bc15a440 0x7f00bc0a8950 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:46628/0 (socket says 172.21.15.143:46628) 2023-11-26T07:18:02.287 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:02.282+0000 7f00ba7fc640 1 -- 172.21.15.143:0/906989677 learned_addr learned my addr 172.21.15.143:0/906989677 (peer_addr_for_me v2:172.21.15.143:0/0) 2023-11-26T07:18:02.287 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:02.282+0000 7f00c18e3640 1 --2- 172.21.15.143:0/906989677 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f00bc0a8e90 0x7f00bc151850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:18:02.288 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:02.282+0000 7f00c18e3640 1 -- 172.21.15.143:0/906989677 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f00bc14d020 con 0x7f00bc15a440 2023-11-26T07:18:02.288 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:02.282+0000 7f00c18e3640 1 -- 172.21.15.143:0/906989677 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f00bc14c410 con 0x7f00bc0b20b0 2023-11-26T07:18:02.288 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:02.282+0000 7f00c18e3640 1 -- 172.21.15.143:0/906989677 --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f00bc14c6f0 con 0x7f00bc0a8e90 2023-11-26T07:18:02.288 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:02.282+0000 7f00bb7fe640 1 --2- 172.21.15.143:0/906989677 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f00bc0a8e90 0x7f00bc151850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:18:02.288 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:02.282+0000 7f00bb7fe640 1 -- 172.21.15.143:0/906989677 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f00bc0b20b0 msgr2=0x7f00bc137b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:18:02.288 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:02.282+0000 7f00bb7fe640 1 --2- 172.21.15.143:0/906989677 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f00bc0b20b0 0x7f00bc137b10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:18:02.289 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:02.282+0000 7f00bb7fe640 1 -- 172.21.15.143:0/906989677 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f00bc15a440 msgr2=0x7f00bc0a8950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:18:02.289 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:02.282+0000 7f00bb7fe640 1 --2- 172.21.15.143:0/906989677 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f00bc15a440 0x7f00bc0a8950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:18:02.289 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:02.282+0000 7f00bb7fe640 1 -- 172.21.15.143:0/906989677 --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f00ac008550 con 0x7f00bc0a8e90 2023-11-26T07:18:02.289 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:02.282+0000 7f00ba7fc640 1 --2- 172.21.15.143:0/906989677 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f00bc15a440 0x7f00bc0a8950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-26T07:18:02.289 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:02.286+0000 7f00bb7fe640 1 --2- 172.21.15.143:0/906989677 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f00bc0a8e90 0x7f00bc151850 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7f00b000b3e0 tx=0x7f00b0009f90 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:18:02.290 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:02.286+0000 7f00a3fff640 1 -- 172.21.15.143:0/906989677 <== mon.2 v2:172.21.15.188:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f00b0011830 con 0x7f00bc0a8e90 2023-11-26T07:18:02.290 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:02.286+0000 7f00a3fff640 1 -- 172.21.15.143:0/906989677 <== mon.2 v2:172.21.15.188:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f00b0011990 con 0x7f00bc0a8e90 2023-11-26T07:18:02.290 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:02.286+0000 7f00a3fff640 1 -- 172.21.15.143:0/906989677 <== mon.2 v2:172.21.15.188:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f00b0011d00 con 0x7f00bc0a8e90 2023-11-26T07:18:02.291 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:02.286+0000 7f00c18e3640 1 -- 172.21.15.143:0/906989677 --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f00bc14c9d0 con 0x7f00bc0a8e90 2023-11-26T07:18:02.291 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:02.286+0000 7f00c18e3640 1 -- 172.21.15.143:0/906989677 --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f00bc14da20 con 0x7f00bc0a8e90 2023-11-26T07:18:02.292 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:02.286+0000 7f00c18e3640 1 -- 172.21.15.143:0/906989677 --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f00bc14d190 con 0x7f00bc0a8e90 2023-11-26T07:18:02.296 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:02.290+0000 7f00a3fff640 1 -- 172.21.15.143:0/906989677 <== mon.2 v2:172.21.15.188:3301/0 4 ==== mgrmap(e 5) v1 ==== 85852+0+0 (secure 0 0 0) 0x7f00b001a400 con 0x7f00bc0a8e90 2023-11-26T07:18:02.296 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:02.290+0000 7f00a3fff640 1 --2- 172.21.15.143:0/906989677 >> [v2:172.21.15.143:6832/2150557544,v1:172.21.15.143:6833/2150557544] conn(0x7f00a8066470 0x7f00a8068930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:18:02.297 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:02.290+0000 7f00a3fff640 1 -- 172.21.15.143:0/906989677 <== mon.2 v2:172.21.15.188:3301/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f00b0083e60 con 0x7f00bc0a8e90 2023-11-26T07:18:02.297 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:02.294+0000 7f00baffd640 1 --2- 172.21.15.143:0/906989677 >> [v2:172.21.15.143:6832/2150557544,v1:172.21.15.143:6833/2150557544] conn(0x7f00a8066470 0x7f00a8068930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:18:02.298 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:02.294+0000 7f00baffd640 1 --2- 172.21.15.143:0/906989677 >> [v2:172.21.15.143:6832/2150557544,v1:172.21.15.143:6833/2150557544] conn(0x7f00a8066470 0x7f00a8068930 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7f00ac0b2000 tx=0x7f00ac00a040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:18:02.299 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:02.294+0000 7f00a3fff640 1 -- 172.21.15.143:0/906989677 <== mon.2 v2:172.21.15.188:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172649 (secure 0 0 0) 0x7f00b00512f0 con 0x7f00bc0a8e90 2023-11-26T07:18:02.329 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:02.326+0000 7f41900d6640 1 Processor -- start 2023-11-26T07:18:02.329 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:02.326+0000 7f41900d6640 1 -- start start 2023-11-26T07:18:02.331 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:02.326+0000 7f41900d6640 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7f41800e30a0 0x7f4180054200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:18:02.331 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:02.326+0000 7f41900d6640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f41800e77f0 0x7f418004f060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:18:02.332 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:02.326+0000 7f41900d6640 1 -- --> v1:172.21.15.188:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4180037b00 con 0x7f418004f5a0 2023-11-26T07:18:02.332 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:02.326+0000 7f41900d6640 1 -- --> v2:172.21.15.143:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f418004fb10 con 0x7f41800e30a0 2023-11-26T07:18:02.332 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:02.326+0000 7f41900d6640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f418003a420 con 0x7f41800e77f0 2023-11-26T07:18:02.332 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:02.326+0000 7f418de72640 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7f41800e30a0 0x7f4180054200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:18:02.333 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:02.326+0000 7f418de72640 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7f41800e30a0 0x7f4180054200 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:46636/0 (socket says 172.21.15.143:46636) 2023-11-26T07:18:02.333 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:02.326+0000 7f418de72640 1 -- 172.21.15.143:0/2358955393 learned_addr learned my addr 172.21.15.143:0/2358955393 (peer_addr_for_me v2:172.21.15.143:0/0) 2023-11-26T07:18:02.333 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:02.330+0000 7f418d671640 1 --2- 172.21.15.143:0/2358955393 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f41800e77f0 0x7f418004f060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:18:02.334 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:02.330+0000 7f418de72640 1 -- 172.21.15.143:0/2358955393 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f41800e77f0 msgr2=0x7f418004f060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-26T07:18:02.334 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:02.330+0000 7f418de72640 1 --2- 172.21.15.143:0/2358955393 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f41800e77f0 0x7f418004f060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:18:02.334 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:02.330+0000 7f418de72640 1 -- 172.21.15.143:0/2358955393 >> v1:172.21.15.188:6789/0 conn(0x7f418004f5a0 legacy=0x7f41800e6c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-26T07:18:02.334 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:02.330+0000 7f418de72640 1 -- 172.21.15.143:0/2358955393 --> v2:172.21.15.143:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f418003a700 con 0x7f41800e30a0 2023-11-26T07:18:02.338 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:02.334+0000 7f418de72640 1 --2- 172.21.15.143:0/2358955393 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f41800e30a0 0x7f4180054200 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7f4184009870 tx=0x7f418401ecf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a6ab24614118df30 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:18:02.339 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:02.334+0000 7f418ce70640 1 -- 172.21.15.143:0/2358955393 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f41840a89b0 con 0x7f41800e30a0 2023-11-26T07:18:02.339 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:02.334+0000 7f418ce70640 1 -- 172.21.15.143:0/2358955393 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f41840a8b10 con 0x7f41800e30a0 2023-11-26T07:18:02.339 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:02.334+0000 7f418ce70640 1 -- 172.21.15.143:0/2358955393 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f41840a8ea0 con 0x7f41800e30a0 2023-11-26T07:18:02.340 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:02.334+0000 7f41900d6640 1 -- 172.21.15.143:0/2358955393 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f41800e30a0 msgr2=0x7f4180054200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:18:02.340 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:02.334+0000 7f41900d6640 1 --2- 172.21.15.143:0/2358955393 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f41800e30a0 0x7f4180054200 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7f4184009870 tx=0x7f418401ecf0 comp rx=0 tx=0).stop 2023-11-26T07:18:02.340 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:02.338+0000 7f41900d6640 1 -- 172.21.15.143:0/2358955393 shutdown_connections 2023-11-26T07:18:02.341 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:02.338+0000 7f41900d6640 1 --2- 172.21.15.143:0/2358955393 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f41800e77f0 0x7f418004f060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:18:02.341 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:02.338+0000 7f41900d6640 1 --2- 172.21.15.143:0/2358955393 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f41800e30a0 0x7f4180054200 secure :-1 s=CLOSED pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7f4184009870 tx=0x7f418401ecf0 comp rx=0 tx=0).stop 2023-11-26T07:18:02.341 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:02.338+0000 7f41900d6640 1 -- 172.21.15.143:0/2358955393 >> 172.21.15.143:0/2358955393 conn(0x7f4180009f90 msgr2=0x7f418000a3a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:18:02.345 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:02.334+0000 7fef7f59e640 1 Processor -- start 2023-11-26T07:18:02.346 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:02.334+0000 7fef7f59e640 1 -- start start 2023-11-26T07:18:02.346 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:02.338+0000 7fef7f59e640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fef80159270 0x7fef800b2fd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:18:02.346 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:02.338+0000 7fef7f59e640 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7fef800ce7c0 0x7fef8006c900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:18:02.346 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:02.338+0000 7fef7f59e640 1 -- --> v1:172.21.15.188:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fef800ce410 con 0x7fef8006ce40 2023-11-26T07:18:02.346 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:02.338+0000 7fef7f59e640 1 -- --> v2:172.21.15.143:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fef800b28e0 con 0x7fef800ce7c0 2023-11-26T07:18:02.346 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:02.338+0000 7fef7f59e640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fef800b3740 con 0x7fef80159270 2023-11-26T07:18:02.346 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:02.338+0000 7fef7ed9d640 1 --1- >> v1:172.21.15.188:6789/0 conn(0x7fef8006ce40 0x7fef80070210 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.188:6789/0 says I am v1:172.21.15.143:40026/0 (socket says 172.21.15.143:40026) 2023-11-26T07:18:02.346 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:02.338+0000 7fef7ed9d640 1 -- 172.21.15.143:0/1482293730 learned_addr learned my addr 172.21.15.143:0/1482293730 (peer_addr_for_me v1:172.21.15.143:0/0) 2023-11-26T07:18:02.346 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:02.338+0000 7fef7e59c640 1 --2- 172.21.15.143:0/1482293730 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fef80159270 0x7fef800b2fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:18:02.346 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:02.338+0000 7fef7d59a640 1 -- 172.21.15.143:0/1482293730 <== mon.1 v1:172.21.15.188:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2358324340 0 0) 0x7fef800ce410 con 0x7fef8006ce40 2023-11-26T07:18:02.347 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:02.338+0000 7fef7d59a640 1 -- 172.21.15.143:0/1482293730 --> v1:172.21.15.188:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fef68003530 con 0x7fef8006ce40 2023-11-26T07:18:02.347 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:02.338+0000 7fef7e59c640 1 -- 172.21.15.143:0/1482293730 >> v2:172.21.15.143:3300/0 conn(0x7fef800ce7c0 msgr2=0x7fef8006c900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-26T07:18:02.347 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:02.338+0000 7fef7e59c640 1 --2- 172.21.15.143:0/1482293730 >> v2:172.21.15.143:3300/0 conn(0x7fef800ce7c0 0x7fef8006c900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:18:02.347 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:02.338+0000 7fef7e59c640 1 -- 172.21.15.143:0/1482293730 >> v1:172.21.15.188:6789/0 conn(0x7fef8006ce40 legacy=0x7fef80070210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:18:02.347 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:02.338+0000 7fef7e59c640 1 -- 172.21.15.143:0/1482293730 --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fef800b3510 con 0x7fef80159270 2023-11-26T07:18:02.347 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:02.342+0000 7f41900d6640 1 -- 172.21.15.143:0/2358955393 shutdown_connections 2023-11-26T07:18:02.347 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:02.342+0000 7f41900d6640 1 -- 172.21.15.143:0/2358955393 wait complete. 2023-11-26T07:18:02.348 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:02.342+0000 7f6299817640 1 Processor -- start 2023-11-26T07:18:02.348 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:02.342+0000 7fef7e59c640 1 --2- 172.21.15.143:0/1482293730 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fef80159270 0x7fef800b2fd0 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7fef700097c0 tx=0x7fef7001c3f0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=c320c12daa55784b server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:18:02.348 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:02.342+0000 7f6299817640 1 -- start start 2023-11-26T07:18:02.349 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:02.342+0000 7fef7d59a640 1 -- 172.21.15.143:0/1482293730 <== mon.2 v2:172.21.15.188:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fef700ab600 con 0x7fef80159270 2023-11-26T07:18:02.350 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:02.342+0000 7fef7d59a640 1 -- 172.21.15.143:0/1482293730 <== mon.2 v2:172.21.15.188:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fef700a7050 con 0x7fef80159270 2023-11-26T07:18:02.350 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:02.346+0000 7fef7f59e640 1 -- 172.21.15.143:0/1482293730 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fef80159270 msgr2=0x7fef800b2fd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:18:02.350 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:02.346+0000 7fef7f59e640 1 --2- 172.21.15.143:0/1482293730 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fef80159270 0x7fef800b2fd0 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7fef700097c0 tx=0x7fef7001c3f0 comp rx=0 tx=0).stop 2023-11-26T07:18:02.351 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:02.346+0000 7f41900d6640 1 Processor -- start 2023-11-26T07:18:02.351 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:02.346+0000 7f41900d6640 1 -- start start 2023-11-26T07:18:02.351 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:02.346+0000 7f6299817640 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7f6294070370 0x7f62940707b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:18:02.351 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:02.342+0000 7f5753c5a640 1 Processor -- start 2023-11-26T07:18:02.352 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:02.346+0000 7f62927fc640 1 --1- >> v1:172.21.15.143:6789/0 conn(0x7f62940ce310 0x7f6294158f10 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.143:6789/0 says I am v1:172.21.15.143:40226/0 (socket says 172.21.15.143:40226) 2023-11-26T07:18:02.352 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:02.346+0000 7f62927fc640 1 -- 172.21.15.143:0/1317493407 learned_addr learned my addr 172.21.15.143:0/1317493407 (peer_addr_for_me v1:172.21.15.143:0/0) 2023-11-26T07:18:02.352 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:02.346+0000 7f62937fe640 1 --2- 172.21.15.143:0/1317493407 >> v2:172.21.15.143:3300/0 conn(0x7f6294070370 0x7f62940707b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:18:02.352 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:02.346+0000 7f6299817640 1 -- 172.21.15.143:0/1317493407 --> v1:172.21.15.143:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f629406cf30 con 0x7f62940ce310 2023-11-26T07:18:02.352 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:02.346+0000 7f6299817640 1 -- 172.21.15.143:0/1317493407 --> v1:172.21.15.188:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f62940cb5d0 con 0x7f62940b1710 2023-11-26T07:18:02.352 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:02.346+0000 7f6299817640 1 -- 172.21.15.143:0/1317493407 --> v2:172.21.15.143:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f62940cb0e0 con 0x7f6294070370 2023-11-26T07:18:02.352 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:02.346+0000 7fef7f59e640 1 -- 172.21.15.143:0/1482293730 shutdown_connections 2023-11-26T07:18:02.353 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:02.346+0000 7fef7f59e640 1 --2- 172.21.15.143:0/1482293730 >> v2:172.21.15.143:3300/0 conn(0x7fef800ce7c0 0x7fef8006c900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:18:02.353 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:02.346+0000 7fef7f59e640 1 --2- 172.21.15.143:0/1482293730 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fef80159270 0x7fef800b2fd0 unknown :-1 s=CLOSED pgs=52 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:18:02.353 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:02.346+0000 7fef7f59e640 1 -- 172.21.15.143:0/1482293730 >> 172.21.15.143:0/1482293730 conn(0x7fef8006a890 msgr2=0x7fef8006aca0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:18:02.353 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:02.346+0000 7f6291ffb640 1 -- 172.21.15.143:0/1317493407 <== mon.0 v1:172.21.15.143:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 548769966 0 0) 0x7f629406cf30 con 0x7f62940ce310 2023-11-26T07:18:02.353 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:02.346+0000 7f6291ffb640 1 -- 172.21.15.143:0/1317493407 --> v1:172.21.15.143:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f6280003530 con 0x7f62940ce310 2023-11-26T07:18:02.353 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:02.346+0000 7fef7f59e640 1 -- 172.21.15.143:0/1482293730 shutdown_connections 2023-11-26T07:18:02.354 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:02.346+0000 7fef7f59e640 1 -- 172.21.15.143:0/1482293730 wait complete. 2023-11-26T07:18:02.354 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:02.346+0000 7f62937fe640 1 -- 172.21.15.143:0/1317493407 >> v1:172.21.15.188:6789/0 conn(0x7f62940b1710 legacy=0x7f629406d680 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2023-11-26T07:18:02.354 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:02.346+0000 7f62937fe640 1 -- 172.21.15.143:0/1317493407 >> v1:172.21.15.143:6789/0 conn(0x7f62940ce310 legacy=0x7f6294158f10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:18:02.354 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:02.346+0000 7f62937fe640 1 -- 172.21.15.143:0/1317493407 --> v2:172.21.15.143:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f629406d180 con 0x7f6294070370 2023-11-26T07:18:02.355 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:02.346+0000 7f41900d6640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f41800e77f0 0x7f4180034360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:18:02.355 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:02.346+0000 7f41900d6640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f4180035ae0 0x7f41800ccd80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:18:02.355 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:02.346+0000 7f41900d6640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f41800d6cd0 0x7f41800da3a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:18:02.355 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:02.346+0000 7f41900d6640 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f41800cd2c0 con 0x7f4180035ae0 2023-11-26T07:18:02.355 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:02.346+0000 7f41900d6640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f41800374b0 con 0x7f41800e77f0 2023-11-26T07:18:02.355 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:02.346+0000 7f41900d6640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4180034a40 con 0x7f41800d6cd0 2023-11-26T07:18:02.355 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:02.346+0000 7f418de72640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f41800e77f0 0x7f4180034360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:18:02.355 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:02.346+0000 7f418de72640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f41800e77f0 0x7f4180034360 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.188:3300/0 says I am v2:172.21.15.143:45770/0 (socket says 172.21.15.143:45770) 2023-11-26T07:18:02.355 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:02.346+0000 7f418de72640 1 -- 172.21.15.143:0/3086785269 learned_addr learned my addr 172.21.15.143:0/3086785269 (peer_addr_for_me v2:172.21.15.143:0/0) 2023-11-26T07:18:02.356 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:02.346+0000 7f418e673640 1 --2- 172.21.15.143:0/3086785269 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f41800d6cd0 0x7f41800da3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:18:02.356 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:02.346+0000 7f418d671640 1 --2- 172.21.15.143:0/3086785269 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f4180035ae0 0x7f41800ccd80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:18:02.356 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:02.346+0000 7f418de72640 1 -- 172.21.15.143:0/3086785269 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f41800d6cd0 msgr2=0x7f41800da3a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:18:02.356 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:02.346+0000 7f418de72640 1 --2- 172.21.15.143:0/3086785269 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f41800d6cd0 0x7f41800da3a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:18:02.356 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:02.346+0000 7f418de72640 1 -- 172.21.15.143:0/3086785269 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f4180035ae0 msgr2=0x7f41800ccd80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:18:02.356 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:02.346+0000 7f418de72640 1 --2- 172.21.15.143:0/3086785269 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f4180035ae0 0x7f41800ccd80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:18:02.356 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:02.350+0000 7f418de72640 1 -- 172.21.15.143:0/3086785269 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f41840094d0 con 0x7f41800e77f0 2023-11-26T07:18:02.356 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:02.346+0000 7f5753c5a640 1 -- start start 2023-11-26T07:18:02.356 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:02.346+0000 7f5753c5a640 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7f574c14b1e0 0x7f574c0cf280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:18:02.357 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:02.346+0000 7f5753c5a640 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7f574c070fd0 0x7f574c162510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:18:02.357 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:02.346+0000 7f5753c5a640 1 -- --> v1:172.21.15.188:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f574c0cfc20 con 0x7f574c0cf7c0 2023-11-26T07:18:02.357 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:02.346+0000 7f5753c5a640 1 -- --> v2:172.21.15.143:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f574c162080 con 0x7f574c14b1e0 2023-11-26T07:18:02.357 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:02.346+0000 7f5753c5a640 1 -- --> v2:172.21.15.188:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f574c161d70 con 0x7f574c070fd0 2023-11-26T07:18:02.357 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:02.346+0000 7f57521f7640 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7f574c070fd0 0x7f574c162510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:18:02.357 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:02.346+0000 7f57521f7640 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7f574c070fd0 0x7f574c162510 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.188:3300/0 says I am v2:172.21.15.143:45764/0 (socket says 172.21.15.143:45764) 2023-11-26T07:18:02.357 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:02.346+0000 7f57521f7640 1 -- 172.21.15.143:0/2087901506 learned_addr learned my addr 172.21.15.143:0/2087901506 (peer_addr_for_me v2:172.21.15.143:0/0) 2023-11-26T07:18:02.357 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:02.346+0000 7f57519f6640 1 --2- 172.21.15.143:0/2087901506 >> v2:172.21.15.143:3300/0 conn(0x7f574c14b1e0 0x7f574c0cf280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:18:02.357 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:02.346+0000 7f57521f7640 1 -- 172.21.15.143:0/2087901506 >> v2:172.21.15.143:3300/0 conn(0x7f574c14b1e0 msgr2=0x7f574c0cf280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-26T07:18:02.357 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:02.346+0000 7f57521f7640 1 --2- 172.21.15.143:0/2087901506 >> v2:172.21.15.143:3300/0 conn(0x7f574c14b1e0 0x7f574c0cf280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:18:02.357 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:02.346+0000 7f57521f7640 1 -- 172.21.15.143:0/2087901506 >> v1:172.21.15.188:6789/0 conn(0x7f574c0cf7c0 legacy=0x7f574c0708c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-26T07:18:02.358 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:02.346+0000 7f57521f7640 1 -- 172.21.15.143:0/2087901506 --> v2:172.21.15.188:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f574c14b5c0 con 0x7f574c070fd0 2023-11-26T07:18:02.358 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:02.350+0000 7f418de72640 1 --2- 172.21.15.143:0/3086785269 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f41800e77f0 0x7f4180034360 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7f41840b3040 tx=0x7f418401ba90 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:18:02.358 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:02.350+0000 7f417e7fc640 1 -- 172.21.15.143:0/3086785269 <== mon.1 v2:172.21.15.188:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f41840b0e40 con 0x7f41800e77f0 2023-11-26T07:18:02.358 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:02.350+0000 7f417e7fc640 1 -- 172.21.15.143:0/3086785269 <== mon.1 v2:172.21.15.188:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f41840bb470 con 0x7f41800e77f0 2023-11-26T07:18:02.358 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:02.350+0000 7f417e7fc640 1 -- 172.21.15.143:0/3086785269 <== mon.1 v2:172.21.15.188:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f41840b0e40 con 0x7f41800e77f0 2023-11-26T07:18:02.359 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:02.350+0000 7fef7f59e640 1 Processor -- start 2023-11-26T07:18:02.359 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:02.350+0000 7fef7f59e640 1 -- start start 2023-11-26T07:18:02.359 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:02.350+0000 7fef7f59e640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fef8006ce40 0x7fef801415f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:18:02.359 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:02.350+0000 7fef7f59e640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fef800ce7c0 0x7fef80166580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:18:02.360 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:02.354+0000 7fef7f59e640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fef80159270 0x7fef80150310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:18:02.360 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:02.354+0000 7fef7f59e640 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fef8015ce50 con 0x7fef800ce7c0 2023-11-26T07:18:02.360 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:02.354+0000 7fef7f59e640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fef8015d410 con 0x7fef8006ce40 2023-11-26T07:18:02.360 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:02.354+0000 7fef7f59e640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fef80150850 con 0x7fef80159270 2023-11-26T07:18:02.360 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:02.354+0000 7fef7dd9b640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fef800ce7c0 0x7fef80166580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:18:02.360 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:02.354+0000 7fef7dd9b640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fef800ce7c0 0x7fef80166580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:46684/0 (socket says 172.21.15.143:46684) 2023-11-26T07:18:02.360 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:02.354+0000 7fef7dd9b640 1 -- 172.21.15.143:0/1859405342 learned_addr learned my addr 172.21.15.143:0/1859405342 (peer_addr_for_me v2:172.21.15.143:0/0) 2023-11-26T07:18:02.360 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:02.354+0000 7fef7e59c640 1 --2- 172.21.15.143:0/1859405342 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fef8006ce40 0x7fef801415f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:18:02.360 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:02.350+0000 7f41900d6640 1 -- 172.21.15.143:0/3086785269 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4180034d20 con 0x7f41800e77f0 2023-11-26T07:18:02.361 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:02.354+0000 7f41900d6640 1 -- 172.21.15.143:0/3086785269 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f41800352e0 con 0x7f41800e77f0 2023-11-26T07:18:02.361 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:02.354+0000 7fef7ed9d640 1 --2- 172.21.15.143:0/1859405342 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fef80159270 0x7fef80150310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:18:02.361 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:02.354+0000 7f62937fe640 1 --2- 172.21.15.143:0/1317493407 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f6294070370 0x7f62940707b0 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7f62840097c0 tx=0x7f628401c3b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e27965791607fa6b server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:18:02.361 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:02.354+0000 7fef7e59c640 1 -- 172.21.15.143:0/1859405342 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fef80159270 msgr2=0x7fef80150310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:18:02.362 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:02.354+0000 7fef7e59c640 1 --2- 172.21.15.143:0/1859405342 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fef80159270 0x7fef80150310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:18:02.362 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:02.354+0000 7fef7e59c640 1 -- 172.21.15.143:0/1859405342 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fef800ce7c0 msgr2=0x7fef80166580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:18:02.362 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:02.354+0000 7fef7e59c640 1 --2- 172.21.15.143:0/1859405342 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fef800ce7c0 0x7fef80166580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:18:02.362 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:02.354+0000 7fef7e59c640 1 -- 172.21.15.143:0/1859405342 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fef70009420 con 0x7fef8006ce40 2023-11-26T07:18:02.362 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:02.354+0000 7fef7ed9d640 1 --2- 172.21.15.143:0/1859405342 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fef80159270 0x7fef80150310 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-26T07:18:02.362 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:02.354+0000 7f41900d6640 1 -- 172.21.15.143:0/3086785269 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f418003a9c0 con 0x7f41800e77f0 2023-11-26T07:18:02.363 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:02.354+0000 7f57521f7640 1 --2- 172.21.15.143:0/2087901506 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f574c070fd0 0x7f574c162510 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7f574c072110 tx=0x7f573c01c490 comp rx=0 tx=0).ready entity=mon.1 client_cookie=62c6add93865f94a server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:18:02.363 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:02.354+0000 7fef7e59c640 1 --2- 172.21.15.143:0/1859405342 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fef8006ce40 0x7fef801415f0 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7fef700ab000 tx=0x7fef70019190 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:18:02.363 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:02.354+0000 7f6291ffb640 1 -- 172.21.15.143:0/1317493407 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f62940cebe0 con 0x7f6294070370 2023-11-26T07:18:02.363 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:02.354+0000 7f6291ffb640 1 -- 172.21.15.143:0/1317493407 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f62840a7050 con 0x7f6294070370 2023-11-26T07:18:02.364 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:02.354+0000 7f6299817640 1 -- 172.21.15.143:0/1317493407 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f6294070370 msgr2=0x7f62940707b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:18:02.364 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:02.354+0000 7f6299817640 1 --2- 172.21.15.143:0/1317493407 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f6294070370 0x7f62940707b0 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7f62840097c0 tx=0x7f628401c3b0 comp rx=0 tx=0).stop 2023-11-26T07:18:02.364 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:02.354+0000 7f417e7fc640 1 -- 172.21.15.143:0/3086785269 <== mon.1 v2:172.21.15.188:3300/0 4 ==== mgrmap(e 5) v1 ==== 85852+0+0 (secure 0 0 0) 0x7f41840b7070 con 0x7f41800e77f0 2023-11-26T07:18:02.364 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:02.354+0000 7f6299817640 1 -- 172.21.15.143:0/1317493407 shutdown_connections 2023-11-26T07:18:02.365 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:02.354+0000 7f6299817640 1 --2- 172.21.15.143:0/1317493407 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f6294070370 0x7f62940707b0 unknown :-1 s=CLOSED pgs=112 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:18:02.365 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:02.354+0000 7f6299817640 1 -- 172.21.15.143:0/1317493407 >> 172.21.15.143:0/1317493407 conn(0x7f62940b0bf0 msgr2=0x7f62940b1000 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:18:02.365 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:02.358+0000 7f6299817640 1 -- 172.21.15.143:0/1317493407 shutdown_connections 2023-11-26T07:18:02.365 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:02.358+0000 7f6299817640 1 -- 172.21.15.143:0/1317493407 wait complete. 2023-11-26T07:18:02.365 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:02.358+0000 7f417e7fc640 1 --2- 172.21.15.143:0/3086785269 >> [v2:172.21.15.143:6832/2150557544,v1:172.21.15.143:6833/2150557544] conn(0x7f416c066500 0x7f416c0689c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:18:02.365 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:02.358+0000 7f417e7fc640 1 -- 172.21.15.143:0/3086785269 <== mon.1 v2:172.21.15.188:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f4184122860 con 0x7f41800e77f0 2023-11-26T07:18:02.366 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:02.358+0000 7f6299817640 1 Processor -- start 2023-11-26T07:18:02.366 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:02.358+0000 7f418d671640 1 --2- 172.21.15.143:0/3086785269 >> [v2:172.21.15.143:6832/2150557544,v1:172.21.15.143:6833/2150557544] conn(0x7f416c066500 0x7f416c0689c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:18:02.366 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:02.358+0000 7fa401a68640 1 Processor -- start 2023-11-26T07:18:02.366 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:02.358+0000 7fa401a68640 1 -- start start 2023-11-26T07:18:02.367 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:02.358+0000 7f6299817640 1 -- start start 2023-11-26T07:18:02.367 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:02.358+0000 7f57509f4640 1 -- 172.21.15.143:0/2087901506 <== mon.1 v2:172.21.15.188:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f573c0aa5f0 con 0x7f574c070fd0 2023-11-26T07:18:02.367 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:02.358+0000 7fef6effd640 1 -- 172.21.15.143:0/1859405342 <== mon.1 v2:172.21.15.188:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fef700b58e0 con 0x7fef8006ce40 2023-11-26T07:18:02.367 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:02.358+0000 7f6299817640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f6294070370 0x7f629415c6e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:18:02.368 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:02.358+0000 7fef6effd640 1 -- 172.21.15.143:0/1859405342 <== mon.1 v2:172.21.15.188:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fef700a4e90 con 0x7fef8006ce40 2023-11-26T07:18:02.368 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:02.358+0000 7fef6effd640 1 -- 172.21.15.143:0/1859405342 <== mon.1 v2:172.21.15.188:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fef700b5c80 con 0x7fef8006ce40 2023-11-26T07:18:02.368 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:02.358+0000 7f417e7fc640 1 -- 172.21.15.143:0/3086785269 <== mon.1 v2:172.21.15.188:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172649 (secure 0 0 0) 0x7f41840efcc0 con 0x7f41800e77f0 2023-11-26T07:18:02.368 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:02.358+0000 7f57509f4640 1 -- 172.21.15.143:0/2087901506 <== mon.1 v2:172.21.15.188:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f573c0aa750 con 0x7f574c070fd0 2023-11-26T07:18:02.369 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:02.358+0000 7f57509f4640 1 -- 172.21.15.143:0/2087901506 <== mon.1 v2:172.21.15.188:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f573c0aaae0 con 0x7f574c070fd0 2023-11-26T07:18:02.369 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:02.358+0000 7fef7f59e640 1 -- 172.21.15.143:0/1859405342 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fef80166c80 con 0x7fef8006ce40 2023-11-26T07:18:02.369 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:02.358+0000 7fef7f59e640 1 -- 172.21.15.143:0/1859405342 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fef8015d0a0 con 0x7fef8006ce40 2023-11-26T07:18:02.369 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:02.362+0000 7f418d671640 1 --2- 172.21.15.143:0/3086785269 >> [v2:172.21.15.143:6832/2150557544,v1:172.21.15.143:6833/2150557544] conn(0x7f416c066500 0x7f416c0689c0 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7f41800daff0 tx=0x7f4178008040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:18:02.370 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:02.362+0000 7f5753c5a640 1 -- 172.21.15.143:0/2087901506 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f574c070fd0 msgr2=0x7f574c162510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:18:02.370 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:02.362+0000 7f5753c5a640 1 --2- 172.21.15.143:0/2087901506 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f574c070fd0 0x7f574c162510 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7f574c072110 tx=0x7f573c01c490 comp rx=0 tx=0).stop 2023-11-26T07:18:02.370 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:02.362+0000 7fa401a68640 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7fa3fc0b2160 0x7fa3fc0d0df0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:18:02.370 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:02.362+0000 7fa401a68640 1 -- --> v1:172.21.15.143:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa3fc0cf0e0 con 0x7fa3fc0d1bc0 2023-11-26T07:18:02.370 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:02.362+0000 7f6299817640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f62940b1710 0x7f629415b6f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:18:02.370 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:02.362+0000 7f6299817640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f62940ce310 0x7f6294156a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:18:02.371 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:02.362+0000 7f6299817640 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f629415cc20 con 0x7f6294070370 2023-11-26T07:18:02.371 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:02.362+0000 7f6299817640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6294157760 con 0x7f62940ce310 2023-11-26T07:18:02.371 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:02.362+0000 7f6299817640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6294157480 con 0x7f62940b1710 2023-11-26T07:18:02.371 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:02.362+0000 7f62937fe640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f62940ce310 0x7f6294156a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:18:02.371 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:02.362+0000 7f62937fe640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f62940ce310 0x7f6294156a00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.188:3300/0 says I am v2:172.21.15.143:45800/0 (socket says 172.21.15.143:45800) 2023-11-26T07:18:02.371 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:02.362+0000 7f62937fe640 1 -- 172.21.15.143:0/1201588543 learned_addr learned my addr 172.21.15.143:0/1201588543 (peer_addr_for_me v2:172.21.15.143:0/0) 2023-11-26T07:18:02.371 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:02.362+0000 7fa3fb7fe640 1 --1- >> v1:172.21.15.188:6789/0 conn(0x7fa3fc15d6e0 0x7fa3fc0d14b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.188:6789/0 says I am v1:172.21.15.143:40054/0 (socket says 172.21.15.143:40054) 2023-11-26T07:18:02.371 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:02.362+0000 7fa3fb7fe640 1 -- 172.21.15.143:0/888415134 learned_addr learned my addr 172.21.15.143:0/888415134 (peer_addr_for_me v1:172.21.15.143:0/0) 2023-11-26T07:18:02.372 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:02.362+0000 7fef6cff9640 1 -- 172.21.15.143:0/1859405342 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fef58005260 con 0x7fef8006ce40 2023-11-26T07:18:02.372 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:02.362+0000 7fef6effd640 1 -- 172.21.15.143:0/1859405342 <== mon.1 v2:172.21.15.188:3300/0 4 ==== mgrmap(e 5) v1 ==== 85852+0+0 (secure 0 0 0) 0x7fef700b2050 con 0x7fef8006ce40 2023-11-26T07:18:02.372 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:02.362+0000 7f5753c5a640 1 -- 172.21.15.143:0/2087901506 shutdown_connections 2023-11-26T07:18:02.372 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:02.362+0000 7f5753c5a640 1 --2- 172.21.15.143:0/2087901506 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f574c070fd0 0x7f574c162510 unknown :-1 s=CLOSED pgs=86 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:18:02.372 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:02.362+0000 7f5753c5a640 1 --2- 172.21.15.143:0/2087901506 >> v2:172.21.15.143:3300/0 conn(0x7f574c14b1e0 0x7f574c0cf280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:18:02.372 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:02.362+0000 7f5753c5a640 1 -- 172.21.15.143:0/2087901506 >> 172.21.15.143:0/2087901506 conn(0x7f574c06a890 msgr2=0x7f574c06aca0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:18:02.373 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:02.362+0000 7fa401a68640 1 -- 172.21.15.143:0/888415134 --> v1:172.21.15.188:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa3fc0ceb20 con 0x7fa3fc15d6e0 2023-11-26T07:18:02.373 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:02.362+0000 7fa401a68640 1 -- 172.21.15.143:0/888415134 --> v2:172.21.15.188:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa3fc15dac0 con 0x7fa3fc0b2160 2023-11-26T07:18:02.373 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:02.362+0000 7fa3fbfff640 1 --2- 172.21.15.143:0/888415134 >> v2:172.21.15.188:3300/0 conn(0x7fa3fc0b2160 0x7fa3fc0d0df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:18:02.373 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:02.362+0000 7fa3fbfff640 1 -- 172.21.15.143:0/888415134 >> v1:172.21.15.188:6789/0 conn(0x7fa3fc15d6e0 legacy=0x7fa3fc0d14b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:18:02.374 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:02.362+0000 7f62927fc640 1 --2- 172.21.15.143:0/1201588543 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f62940b1710 0x7f629415b6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:18:02.374 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:02.362+0000 7f62927fc640 1 -- 172.21.15.143:0/1201588543 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f62940ce310 msgr2=0x7f6294156a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:18:02.374 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:02.362+0000 7f62927fc640 1 --2- 172.21.15.143:0/1201588543 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f62940ce310 0x7f6294156a00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:18:02.374 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:02.362+0000 7f62927fc640 1 -- 172.21.15.143:0/1201588543 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f6294070370 msgr2=0x7f629415c6e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:18:02.374 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:02.362+0000 7f62927fc640 1 --2- 172.21.15.143:0/1201588543 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f6294070370 0x7f629415c6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:18:02.374 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:02.366+0000 7fef6effd640 1 --2- 172.21.15.143:0/1859405342 >> [v2:172.21.15.143:6832/2150557544,v1:172.21.15.143:6833/2150557544] conn(0x7fef600664a0 0x7fef60068960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:18:02.375 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:02.362+0000 7f62927fc640 1 -- 172.21.15.143:0/1201588543 --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6284009420 con 0x7f62940b1710 2023-11-26T07:18:02.375 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:02.362+0000 7f62927fc640 1 --2- 172.21.15.143:0/1201588543 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f62940b1710 0x7f629415b6f0 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7f627c00c400 tx=0x7f627c00c8c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:18:02.375 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:02.366+0000 7fef6effd640 1 -- 172.21.15.143:0/1859405342 <== mon.1 v2:172.21.15.188:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fef7011d070 con 0x7fef8006ce40 2023-11-26T07:18:02.375 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:02.366+0000 7fef7dd9b640 1 --2- 172.21.15.143:0/1859405342 >> [v2:172.21.15.143:6832/2150557544,v1:172.21.15.143:6833/2150557544] conn(0x7fef600664a0 0x7fef60068960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:18:02.376 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:02.366+0000 7f5753c5a640 1 -- 172.21.15.143:0/2087901506 shutdown_connections 2023-11-26T07:18:02.376 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:02.366+0000 7f5753c5a640 1 -- 172.21.15.143:0/2087901506 wait complete. 2023-11-26T07:18:02.376 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:02.366+0000 7fef7dd9b640 1 --2- 172.21.15.143:0/1859405342 >> [v2:172.21.15.143:6832/2150557544,v1:172.21.15.143:6833/2150557544] conn(0x7fef600664a0 0x7fef60068960 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7fef74006fd0 tx=0x7fef74008040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:18:02.376 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:02.362+0000 7fa3fbfff640 1 -- 172.21.15.143:0/888415134 >> v1:172.21.15.143:6789/0 conn(0x7fa3fc0d1bc0 legacy=0x7fa3fc0b0750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:18:02.376 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:02.362+0000 7fa3fbfff640 1 -- 172.21.15.143:0/888415134 --> v2:172.21.15.188:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa3fc0cee00 con 0x7fa3fc0b2160 2023-11-26T07:18:02.377 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:02.366+0000 7f5753c5a640 1 Processor -- start 2023-11-26T07:18:02.377 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:02.366+0000 7f628bfff640 1 -- 172.21.15.143:0/1201588543 <== mon.2 v2:172.21.15.188:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f627c005cf0 con 0x7f62940b1710 2023-11-26T07:18:02.377 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:02.366+0000 7f628bfff640 1 -- 172.21.15.143:0/1201588543 <== mon.2 v2:172.21.15.188:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f627c005e50 con 0x7f62940b1710 2023-11-26T07:18:02.377 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:02.366+0000 7f628bfff640 1 -- 172.21.15.143:0/1201588543 <== mon.2 v2:172.21.15.188:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f627c0072d0 con 0x7f62940b1710 2023-11-26T07:18:02.377 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:02.366+0000 7f6299817640 1 -- 172.21.15.143:0/1201588543 --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f629415ac10 con 0x7f62940b1710 2023-11-26T07:18:02.377 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:02.366+0000 7f6299817640 1 -- 172.21.15.143:0/1201588543 --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f629415b1d0 con 0x7f62940b1710 2023-11-26T07:18:02.378 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:02.366+0000 7f5753c5a640 1 -- start start 2023-11-26T07:18:02.378 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:02.370+0000 7fef6effd640 1 -- 172.21.15.143:0/1859405342 <== mon.1 v2:172.21.15.188:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172649 (secure 0 0 0) 0x7fef700ea800 con 0x7fef8006ce40 2023-11-26T07:18:02.378 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:02.370+0000 7f6299817640 1 -- 172.21.15.143:0/1201588543 --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6278005260 con 0x7f62940b1710 2023-11-26T07:18:02.378 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:02.370+0000 7f5753c5a640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f574c070fd0 0x7f574c0add50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:18:02.379 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:02.370+0000 7f5753c5a640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f574c0cf7c0 0x7f574c0cd410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:18:02.379 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:02.370+0000 7f5753c5a640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f574c14b1e0 0x7f574c0cd950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:18:02.379 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:02.370+0000 7fa3fbfff640 1 --2- 172.21.15.143:0/888415134 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fa3fc0b2160 0x7fa3fc0d0df0 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7fa3f4009d90 tx=0x7fa3f401c970 comp rx=0 tx=0).ready entity=mon.1 client_cookie=b48eac73337fba49 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:18:02.379 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:02.370+0000 7f57511f5640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f574c0cf7c0 0x7f574c0cd410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:18:02.379 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:02.370+0000 7f57511f5640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f574c0cf7c0 0x7f574c0cd410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.188:3301/0 says I am v2:172.21.15.143:45342/0 (socket says 172.21.15.143:45342) 2023-11-26T07:18:02.380 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:02.370+0000 7f57519f6640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f574c070fd0 0x7f574c0add50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:18:02.380 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:02.370+0000 7f628bfff640 1 -- 172.21.15.143:0/1201588543 <== mon.2 v2:172.21.15.188:3301/0 4 ==== mgrmap(e 5) v1 ==== 85852+0+0 (secure 0 0 0) 0x7f627c007430 con 0x7f62940b1710 2023-11-26T07:18:02.380 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:02.370+0000 7f57519f6640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f574c070fd0 0x7f574c0add50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.188:3300/0 says I am v2:172.21.15.143:45804/0 (socket says 172.21.15.143:45804) 2023-11-26T07:18:02.380 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:02.370+0000 7f5753c5a640 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f574c13b030 con 0x7f574c14b1e0 2023-11-26T07:18:02.381 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:02.370+0000 7f5753c5a640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f574c13a420 con 0x7f574c070fd0 2023-11-26T07:18:02.381 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:02.370+0000 7f5753c5a640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f574c13a700 con 0x7f574c0cf7c0 2023-11-26T07:18:02.381 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:02.370+0000 7f57511f5640 1 -- 172.21.15.143:0/1893192248 learned_addr learned my addr 172.21.15.143:0/1893192248 (peer_addr_for_me v2:172.21.15.143:0/0) 2023-11-26T07:18:02.381 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:02.370+0000 7f57521f7640 1 --2- 172.21.15.143:0/1893192248 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f574c14b1e0 0x7f574c0cd950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:18:02.381 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:02.370+0000 7f57519f6640 1 -- 172.21.15.143:0/1893192248 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f574c0cf7c0 msgr2=0x7f574c0cd410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:18:02.381 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:02.370+0000 7f57519f6640 1 --2- 172.21.15.143:0/1893192248 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f574c0cf7c0 0x7f574c0cd410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:18:02.381 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:02.370+0000 7f57519f6640 1 -- 172.21.15.143:0/1893192248 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f574c14b1e0 msgr2=0x7f574c0cd950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:18:02.381 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:02.370+0000 7f57519f6640 1 --2- 172.21.15.143:0/1893192248 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f574c14b1e0 0x7f574c0cd950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:18:02.381 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:02.370+0000 7f57519f6640 1 -- 172.21.15.143:0/1893192248 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f573c0094d0 con 0x7f574c070fd0 2023-11-26T07:18:02.381 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:02.370+0000 7f57519f6640 1 --2- 172.21.15.143:0/1893192248 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f574c070fd0 0x7f574c0add50 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7f574800c540 tx=0x7f574800ca00 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:18:02.382 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:02.370+0000 7f628bfff640 1 --2- 172.21.15.143:0/1201588543 >> [v2:172.21.15.143:6832/2150557544,v1:172.21.15.143:6833/2150557544] conn(0x7f62740664c0 0x7f6274068980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:18:02.382 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:02.374+0000 7f628bfff640 1 -- 172.21.15.143:0/1201588543 <== mon.2 v2:172.21.15.188:3301/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f627c083240 con 0x7f62940b1710 2023-11-26T07:18:02.382 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:02.374+0000 7fa3fa7fc640 1 -- 172.21.15.143:0/888415134 <== mon.1 v2:172.21.15.188:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa3f40a5900 con 0x7fa3fc0b2160 2023-11-26T07:18:02.382 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:02.374+0000 7fa3fa7fc640 1 -- 172.21.15.143:0/888415134 <== mon.1 v2:172.21.15.188:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa3f40a5a60 con 0x7fa3fc0b2160 2023-11-26T07:18:02.382 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:02.374+0000 7fa3fa7fc640 1 -- 172.21.15.143:0/888415134 <== mon.1 v2:172.21.15.188:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa3f40a5df0 con 0x7fa3fc0b2160 2023-11-26T07:18:02.383 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:02.374+0000 7f57427fc640 1 -- 172.21.15.143:0/1893192248 <== mon.1 v2:172.21.15.188:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5748011800 con 0x7f574c070fd0 2023-11-26T07:18:02.383 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:02.374+0000 7f6292ffd640 1 --2- 172.21.15.143:0/1201588543 >> [v2:172.21.15.143:6832/2150557544,v1:172.21.15.143:6833/2150557544] conn(0x7f62740664c0 0x7f6274068980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:18:02.383 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:02.374+0000 7f57427fc640 1 -- 172.21.15.143:0/1893192248 <== mon.1 v2:172.21.15.188:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5748011960 con 0x7f574c070fd0 2023-11-26T07:18:02.383 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:02.374+0000 7f57427fc640 1 -- 172.21.15.143:0/1893192248 <== mon.1 v2:172.21.15.188:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5748011d10 con 0x7f574c070fd0 2023-11-26T07:18:02.384 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:02.374+0000 7fa401a68640 1 -- 172.21.15.143:0/888415134 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fa3fc0b2160 msgr2=0x7fa3fc0d0df0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:18:02.384 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:02.374+0000 7fa401a68640 1 --2- 172.21.15.143:0/888415134 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fa3fc0b2160 0x7fa3fc0d0df0 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7fa3f4009d90 tx=0x7fa3f401c970 comp rx=0 tx=0).stop 2023-11-26T07:18:02.384 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:02.374+0000 7fa401a68640 1 -- 172.21.15.143:0/888415134 shutdown_connections 2023-11-26T07:18:02.384 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:02.374+0000 7fa401a68640 1 --2- 172.21.15.143:0/888415134 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fa3fc0b2160 0x7fa3fc0d0df0 unknown :-1 s=CLOSED pgs=89 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:18:02.385 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:02.374+0000 7fa401a68640 1 -- 172.21.15.143:0/888415134 >> 172.21.15.143:0/888415134 conn(0x7fa3fc09c550 msgr2=0x7fa3fc0d3220 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:18:02.385 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:02.374+0000 7fa401a68640 1 -- 172.21.15.143:0/888415134 shutdown_connections 2023-11-26T07:18:02.385 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:02.374+0000 7fa401a68640 1 -- 172.21.15.143:0/888415134 wait complete. 2023-11-26T07:18:02.385 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:02.374+0000 7f5753c5a640 1 -- 172.21.15.143:0/1893192248 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f574c13a9e0 con 0x7f574c070fd0 2023-11-26T07:18:02.385 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:02.374+0000 7f5753c5a640 1 -- 172.21.15.143:0/1893192248 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f574c158c80 con 0x7f574c070fd0 2023-11-26T07:18:02.385 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:02.374+0000 7f6292ffd640 1 --2- 172.21.15.143:0/1201588543 >> [v2:172.21.15.143:6832/2150557544,v1:172.21.15.143:6833/2150557544] conn(0x7f62740664c0 0x7f6274068980 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7f628c00abb0 tx=0x7f628c009250 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:18:02.386 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:02.374+0000 7f5753c5a640 1 -- 172.21.15.143:0/1893192248 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f574c067680 con 0x7f574c070fd0 2023-11-26T07:18:02.386 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:02.378+0000 7fa401a68640 1 Processor -- start 2023-11-26T07:18:02.386 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:02.378+0000 7f628bfff640 1 -- 172.21.15.143:0/1201588543 <== mon.2 v2:172.21.15.188:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172649 (secure 0 0 0) 0x7f627c054370 con 0x7f62940b1710 2023-11-26T07:18:02.387 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:02.378+0000 7f57427fc640 1 -- 172.21.15.143:0/1893192248 <== mon.1 v2:172.21.15.188:3300/0 4 ==== mgrmap(e 5) v1 ==== 85852+0+0 (secure 0 0 0) 0x7f5748022350 con 0x7f574c070fd0 2023-11-26T07:18:02.387 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:02.378+0000 7f57427fc640 1 --2- 172.21.15.143:0/1893192248 >> [v2:172.21.15.143:6832/2150557544,v1:172.21.15.143:6833/2150557544] conn(0x7f57300664c0 0x7f5730068980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:18:02.387 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:02.378+0000 7f57427fc640 1 -- 172.21.15.143:0/1893192248 <== mon.1 v2:172.21.15.188:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f5748083cd0 con 0x7f574c070fd0 2023-11-26T07:18:02.387 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:02.378+0000 7f57511f5640 1 --2- 172.21.15.143:0/1893192248 >> [v2:172.21.15.143:6832/2150557544,v1:172.21.15.143:6833/2150557544] conn(0x7f57300664c0 0x7f5730068980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:18:02.387 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:02.382+0000 7f57511f5640 1 --2- 172.21.15.143:0/1893192248 >> [v2:172.21.15.143:6832/2150557544,v1:172.21.15.143:6833/2150557544] conn(0x7f57300664c0 0x7f5730068980 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7f574c157a40 tx=0x7f5744003e70 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:18:02.388 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:02.382+0000 7fa401a68640 1 -- start start 2023-11-26T07:18:02.388 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:02.382+0000 7fa401a68640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fa3fc0b2160 0x7fa3fc154500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:18:02.388 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:02.382+0000 7fa401a68640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fa3fc0d1bc0 0x7fa3fc1fd340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:18:02.388 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:02.382+0000 7f57427fc640 1 -- 172.21.15.143:0/1893192248 <== mon.1 v2:172.21.15.188:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172649 (secure 0 0 0) 0x7f57480512e0 con 0x7f574c070fd0 2023-11-26T07:18:02.389 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:02.382+0000 7fa401a68640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fa3fc15d6e0 0x7fa3fc1fd880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:18:02.389 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:02.382+0000 7fa401a68640 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa3fc0cbbe0 con 0x7fa3fc0b2160 2023-11-26T07:18:02.389 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:02.382+0000 7fa401a68640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa3fc0cb5d0 con 0x7fa3fc15d6e0 2023-11-26T07:18:02.389 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:02.382+0000 7fa401a68640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa3fc0cce50 con 0x7fa3fc0d1bc0 2023-11-26T07:18:02.389 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:02.382+0000 7fa3fb7fe640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fa3fc0b2160 0x7fa3fc154500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:18:02.389 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:02.382+0000 7fa3fb7fe640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fa3fc0b2160 0x7fa3fc154500 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:46706/0 (socket says 172.21.15.143:46706) 2023-11-26T07:18:02.389 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:02.382+0000 7fa3fb7fe640 1 -- 172.21.15.143:0/27832242 learned_addr learned my addr 172.21.15.143:0/27832242 (peer_addr_for_me v2:172.21.15.143:0/0) 2023-11-26T07:18:02.390 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:02.382+0000 7fa3faffd640 1 --2- 172.21.15.143:0/27832242 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fa3fc0d1bc0 0x7fa3fc1fd340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:18:02.390 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:02.382+0000 7fa3fbfff640 1 --2- 172.21.15.143:0/27832242 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fa3fc15d6e0 0x7fa3fc1fd880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:18:02.390 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:02.382+0000 7fa3fb7fe640 1 -- 172.21.15.143:0/27832242 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fa3fc0d1bc0 msgr2=0x7fa3fc1fd340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:18:02.392 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:02.390+0000 7fa3fb7fe640 1 --2- 172.21.15.143:0/27832242 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fa3fc0d1bc0 0x7fa3fc1fd340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:18:02.393 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:02.390+0000 7fa3fb7fe640 1 -- 172.21.15.143:0/27832242 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fa3fc15d6e0 msgr2=0x7fa3fc1fd880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:18:02.393 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:02.390+0000 7fa3fb7fe640 1 --2- 172.21.15.143:0/27832242 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fa3fc15d6e0 0x7fa3fc1fd880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:18:02.393 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:02.390+0000 7fa3fb7fe640 1 -- 172.21.15.143:0/27832242 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa3f4009a70 con 0x7fa3fc0b2160 2023-11-26T07:18:02.393 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:02.390+0000 7fa3fbfff640 1 --2- 172.21.15.143:0/27832242 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fa3fc15d6e0 0x7fa3fc1fd880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-26T07:18:02.393 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:02.390+0000 7fa3faffd640 1 --2- 172.21.15.143:0/27832242 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fa3fc0d1bc0 0x7fa3fc1fd340 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-26T07:18:02.397 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:02.390+0000 7fa3fb7fe640 1 --2- 172.21.15.143:0/27832242 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fa3fc0b2160 0x7fa3fc154500 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7fa3f000c400 tx=0x7fa3f000c8c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:18:02.397 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:02.390+0000 7fa3e3fff640 1 -- 172.21.15.143:0/27832242 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa3f0005cf0 con 0x7fa3fc0b2160 2023-11-26T07:18:02.397 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:02.390+0000 7fa401a68640 1 -- 172.21.15.143:0/27832242 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa3fc0cb870 con 0x7fa3fc0b2160 2023-11-26T07:18:02.397 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:02.390+0000 7fa401a68640 1 -- 172.21.15.143:0/27832242 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa3fc0cd130 con 0x7fa3fc0b2160 2023-11-26T07:18:02.397 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:02.390+0000 7fa3e3fff640 1 -- 172.21.15.143:0/27832242 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa3f0005e50 con 0x7fa3fc0b2160 2023-11-26T07:18:02.397 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:02.390+0000 7fa3e3fff640 1 -- 172.21.15.143:0/27832242 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa3f0005cf0 con 0x7fa3fc0b2160 2023-11-26T07:18:02.397 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:02.394+0000 7fa401a68640 1 -- 172.21.15.143:0/27832242 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa3fc15f3c0 con 0x7fa3fc0b2160 2023-11-26T07:18:02.399 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:02.394+0000 7fa3e3fff640 1 -- 172.21.15.143:0/27832242 <== mon.0 v2:172.21.15.143:3300/0 4 ==== mgrmap(e 5) v1 ==== 85852+0+0 (secure 0 0 0) 0x7fa3f0007070 con 0x7fa3fc0b2160 2023-11-26T07:18:02.401 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:02.398+0000 7fa3e3fff640 1 --2- 172.21.15.143:0/27832242 >> [v2:172.21.15.143:6832/2150557544,v1:172.21.15.143:6833/2150557544] conn(0x7fa3e80664d0 0x7fa3e8068990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:18:02.401 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:02.398+0000 7fa3e3fff640 1 -- 172.21.15.143:0/27832242 <== mon.0 v2:172.21.15.143:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fa3f0083a20 con 0x7fa3fc0b2160 2023-11-26T07:18:02.401 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:02.398+0000 7fa3faffd640 1 --2- 172.21.15.143:0/27832242 >> [v2:172.21.15.143:6832/2150557544,v1:172.21.15.143:6833/2150557544] conn(0x7fa3e80664d0 0x7fa3e8068990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:18:02.403 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:02.398+0000 7fa3faffd640 1 --2- 172.21.15.143:0/27832242 >> [v2:172.21.15.143:6832/2150557544,v1:172.21.15.143:6833/2150557544] conn(0x7fa3e80664d0 0x7fa3e8068990 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7fa3fc071b90 tx=0x7fa3ec009040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:18:02.408 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:02.402+0000 7fa3e3fff640 1 -- 172.21.15.143:0/27832242 <== mon.0 v2:172.21.15.143:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172649 (secure 0 0 0) 0x7fa3f0050460 con 0x7fa3fc0b2160 2023-11-26T07:18:02.411 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:02.406+0000 7fa59ae17640 1 Processor -- start 2023-11-26T07:18:02.411 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:02.406+0000 7fa40bfff640 1 Processor -- start 2023-11-26T07:18:02.411 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:02.406+0000 7fa40bfff640 1 -- start start 2023-11-26T07:18:02.411 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:02.406+0000 7fa59ae17640 1 -- start start 2023-11-26T07:18:02.414 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:02.410+0000 7fa59ae17640 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7fa5940cc3f0 0x7fa5940ce560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:18:02.414 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:02.410+0000 7fa59ae17640 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7fa5940720d0 0x7fa5940cc7d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:18:02.414 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:02.410+0000 7fa59ae17640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fa59415f3d0 0x7fa59415b400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:18:02.414 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:02.410+0000 7fa59ae17640 1 -- --> v2:172.21.15.143:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa5940ccd10 con 0x7fa5940720d0 2023-11-26T07:18:02.414 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:02.410+0000 7fa59ae17640 1 -- --> v2:172.21.15.188:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa594071310 con 0x7fa5940cc3f0 2023-11-26T07:18:02.414 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:02.410+0000 7fa59ae17640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa5940715f0 con 0x7fa59415f3d0 2023-11-26T07:18:02.415 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:02.410+0000 7fa599614640 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7fa5940720d0 0x7fa5940cc7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:18:02.415 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:02.410+0000 7fa599e15640 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7fa5940cc3f0 0x7fa5940ce560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:18:02.415 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:02.410+0000 7fa599e15640 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7fa5940cc3f0 0x7fa5940ce560 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.188:3300/0 says I am v2:172.21.15.143:45816/0 (socket says 172.21.15.143:45816) 2023-11-26T07:18:02.415 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:02.410+0000 7fa599e15640 1 -- 172.21.15.143:0/4272479411 learned_addr learned my addr 172.21.15.143:0/4272479411 (peer_addr_for_me v2:172.21.15.143:0/0) 2023-11-26T07:18:02.416 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:02.410+0000 7fa599614640 1 -- 172.21.15.143:0/4272479411 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fa59415f3d0 msgr2=0x7fa59415b400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-26T07:18:02.416 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:02.410+0000 7fa599614640 1 --2- 172.21.15.143:0/4272479411 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fa59415f3d0 0x7fa59415b400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:18:02.416 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:02.410+0000 7fa599614640 1 -- 172.21.15.143:0/4272479411 >> v2:172.21.15.188:3300/0 conn(0x7fa5940cc3f0 msgr2=0x7fa5940ce560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-26T07:18:02.416 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:02.410+0000 7fa599614640 1 --2- 172.21.15.143:0/4272479411 >> v2:172.21.15.188:3300/0 conn(0x7fa5940cc3f0 0x7fa5940ce560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:18:02.416 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:02.410+0000 7fa599614640 1 -- 172.21.15.143:0/4272479411 --> v2:172.21.15.143:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa5940718d0 con 0x7fa5940720d0 2023-11-26T07:18:02.417 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:02.414+0000 7fa40bfff640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fa40c153550 0x7fa40c06cda0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:18:02.417 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:02.414+0000 7fa40bfff640 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7fa40c159690 0x7fa40c0b38d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:18:02.417 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:02.414+0000 7fa40bfff640 1 -- --> v1:172.21.15.188:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa40c159c00 con 0x7fa40c153930 2023-11-26T07:18:02.417 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:02.414+0000 7fa40bfff640 1 -- --> v2:172.21.15.188:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa40c0b1e10 con 0x7fa40c159690 2023-11-26T07:18:02.417 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:02.414+0000 7fa40bfff640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa40c159ee0 con 0x7fa40c153550 2023-11-26T07:18:02.418 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:02.414+0000 7fa40affd640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fa40c153550 0x7fa40c06cda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:18:02.418 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:02.414+0000 7fa40affd640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fa40c153550 0x7fa40c06cda0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.188:3301/0 says I am v2:172.21.15.143:45358/0 (socket says 172.21.15.143:45358) 2023-11-26T07:18:02.418 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:02.414+0000 7fa40affd640 1 -- 172.21.15.143:0/1867617141 learned_addr learned my addr 172.21.15.143:0/1867617141 (peer_addr_for_me v2:172.21.15.143:0/0) 2023-11-26T07:18:02.418 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:02.414+0000 7fa40b7fe640 1 --2- 172.21.15.143:0/1867617141 >> v2:172.21.15.188:3300/0 conn(0x7fa40c159690 0x7fa40c0b38d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:18:02.420 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:02.414+0000 7fa40affd640 1 -- 172.21.15.143:0/1867617141 >> v2:172.21.15.188:3300/0 conn(0x7fa40c159690 msgr2=0x7fa40c0b38d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-26T07:18:02.420 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:02.414+0000 7fa40affd640 1 --2- 172.21.15.143:0/1867617141 >> v2:172.21.15.188:3300/0 conn(0x7fa40c159690 0x7fa40c0b38d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:18:02.421 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:02.414+0000 7fa40affd640 1 -- 172.21.15.143:0/1867617141 >> v1:172.21.15.188:6789/0 conn(0x7fa40c153930 legacy=0x7fa40c0cc800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-26T07:18:02.421 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:02.414+0000 7fa40affd640 1 -- 172.21.15.143:0/1867617141 --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa40c15a1c0 con 0x7fa40c153550 2023-11-26T07:18:02.421 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:02.418+0000 7fa599614640 1 --2- 172.21.15.143:0/4272479411 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fa5940720d0 0x7fa5940cc7d0 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7fa5940b4340 tx=0x7fa58401c470 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cec0baeb635fef24 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:18:02.422 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:02.418+0000 7fa598e13640 1 -- 172.21.15.143:0/4272479411 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa5840ab600 con 0x7fa5940720d0 2023-11-26T07:18:02.422 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:02.418+0000 7fa598e13640 1 -- 172.21.15.143:0/4272479411 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa5840a7050 con 0x7fa5940720d0 2023-11-26T07:18:02.422 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:02.418+0000 7fa598e13640 1 -- 172.21.15.143:0/4272479411 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa5840ab980 con 0x7fa5940720d0 2023-11-26T07:18:02.423 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:02.418+0000 7fa59ae17640 1 -- 172.21.15.143:0/4272479411 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fa5940720d0 msgr2=0x7fa5940cc7d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:18:02.423 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:02.418+0000 7fa59ae17640 1 --2- 172.21.15.143:0/4272479411 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fa5940720d0 0x7fa5940cc7d0 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7fa5940b4340 tx=0x7fa58401c470 comp rx=0 tx=0).stop 2023-11-26T07:18:02.424 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:02.418+0000 7fa59ae17640 1 -- 172.21.15.143:0/4272479411 shutdown_connections 2023-11-26T07:18:02.424 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:02.418+0000 7fa59ae17640 1 --2- 172.21.15.143:0/4272479411 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fa59415f3d0 0x7fa59415b400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:18:02.424 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:02.418+0000 7fa59ae17640 1 --2- 172.21.15.143:0/4272479411 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fa5940720d0 0x7fa5940cc7d0 unknown :-1 s=CLOSED pgs=115 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:18:02.424 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:02.418+0000 7fa59ae17640 1 --2- 172.21.15.143:0/4272479411 >> v2:172.21.15.188:3300/0 conn(0x7fa5940cc3f0 0x7fa5940ce560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:18:02.424 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:02.418+0000 7fa40affd640 1 --2- 172.21.15.143:0/1867617141 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fa40c153550 0x7fa40c06cda0 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7fa3fc009870 tx=0x7fa3fc01c520 comp rx=0 tx=0).ready entity=mon.2 client_cookie=6c0a70f192d1f160 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:18:02.424 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:02.418+0000 7fa409ffb640 1 -- 172.21.15.143:0/1867617141 <== mon.2 v2:172.21.15.188:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa3fc0ab600 con 0x7fa40c153550 2023-11-26T07:18:02.425 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:02.418+0000 7fa409ffb640 1 -- 172.21.15.143:0/1867617141 <== mon.2 v2:172.21.15.188:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa3fc0a7050 con 0x7fa40c153550 2023-11-26T07:18:02.425 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:02.418+0000 7fa409ffb640 1 -- 172.21.15.143:0/1867617141 <== mon.2 v2:172.21.15.188:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa3fc0ab600 con 0x7fa40c153550 2023-11-26T07:18:02.425 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:02.418+0000 7fa59ae17640 1 -- 172.21.15.143:0/4272479411 >> 172.21.15.143:0/4272479411 conn(0x7fa59406a890 msgr2=0x7fa59406aca0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:18:02.426 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:02.422+0000 7fa59ae17640 1 -- 172.21.15.143:0/4272479411 shutdown_connections 2023-11-26T07:18:02.426 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:02.422+0000 7fa40bfff640 1 -- 172.21.15.143:0/1867617141 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fa40c153550 msgr2=0x7fa40c06cda0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:18:02.427 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:02.422+0000 7fa40bfff640 1 --2- 172.21.15.143:0/1867617141 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fa40c153550 0x7fa40c06cda0 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7fa3fc009870 tx=0x7fa3fc01c520 comp rx=0 tx=0).stop 2023-11-26T07:18:02.427 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:02.422+0000 7fa40bfff640 1 -- 172.21.15.143:0/1867617141 shutdown_connections 2023-11-26T07:18:02.427 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:02.422+0000 7fa40bfff640 1 --2- 172.21.15.143:0/1867617141 >> v2:172.21.15.188:3300/0 conn(0x7fa40c159690 0x7fa40c0b38d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:18:02.427 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:02.422+0000 7fa40bfff640 1 --2- 172.21.15.143:0/1867617141 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fa40c153550 0x7fa40c06cda0 unknown :-1 s=CLOSED pgs=54 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:18:02.427 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:02.422+0000 7fa40bfff640 1 -- 172.21.15.143:0/1867617141 >> 172.21.15.143:0/1867617141 conn(0x7fa40c06a890 msgr2=0x7fa40c06aca0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:18:02.427 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:02.422+0000 7fa40bfff640 1 -- 172.21.15.143:0/1867617141 shutdown_connections 2023-11-26T07:18:02.427 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:02.422+0000 7fa40bfff640 1 -- 172.21.15.143:0/1867617141 wait complete. 2023-11-26T07:18:02.429 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:02.422+0000 7fa59ae17640 1 -- 172.21.15.143:0/4272479411 wait complete. 2023-11-26T07:18:02.429 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:02.426+0000 7fa59ae17640 1 Processor -- start 2023-11-26T07:18:02.429 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:02.426+0000 7fa59ae17640 1 -- start start 2023-11-26T07:18:02.430 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:02.426+0000 7fa40bfff640 1 Processor -- start 2023-11-26T07:18:02.430 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:02.426+0000 7fa40bfff640 1 -- start start 2023-11-26T07:18:02.431 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:02.426+0000 7fa59ae17640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fa5940cc3f0 0x7fa59415aa30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:18:02.432 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:02.426+0000 7fa59ae17640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fa59415f3d0 0x7fa59415fb20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:18:02.432 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:02.430+0000 7fa40bfff640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fa40c153550 0x7fa40c1ef780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:18:02.433 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:02.430+0000 7fa40bfff640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fa40c153930 0x7fa40c152890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:18:02.433 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:02.430+0000 7fa40bfff640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fa40c159690 0x7fa40c1ee1d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:18:02.433 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:02.430+0000 7fa40bfff640 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa40c1ef360 con 0x7fa40c153930 2023-11-26T07:18:02.433 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:02.430+0000 7fa40affd640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fa40c153550 0x7fa40c1ef780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:18:02.434 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:02.430+0000 7fa40bfff640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa40c0c4ee0 con 0x7fa40c153550 2023-11-26T07:18:02.434 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:02.430+0000 7fa40bfff640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa40c152dd0 con 0x7fa40c159690 2023-11-26T07:18:02.434 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:02.430+0000 7fa40affd640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fa40c153550 0x7fa40c1ef780 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.188:3300/0 says I am v2:172.21.15.143:45832/0 (socket says 172.21.15.143:45832) 2023-11-26T07:18:02.434 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:02.430+0000 7fa40affd640 1 -- 172.21.15.143:0/2257136655 learned_addr learned my addr 172.21.15.143:0/2257136655 (peer_addr_for_me v2:172.21.15.143:0/0) 2023-11-26T07:18:02.434 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:02.426+0000 7fa599e15640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fa5940cc3f0 0x7fa59415aa30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:18:02.434 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:02.426+0000 7fa599e15640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fa5940cc3f0 0x7fa59415aa30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.188:3301/0 says I am v2:172.21.15.143:45368/0 (socket says 172.21.15.143:45368) 2023-11-26T07:18:02.435 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:02.430+0000 7fa599614640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fa59415f3d0 0x7fa59415fb20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:18:02.435 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:02.430+0000 7fa59ae17640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fa594160060 0x7fa59415c930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:18:02.435 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:02.430+0000 7fa59ae17640 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa594163180 con 0x7fa59415f3d0 2023-11-26T07:18:02.435 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:02.430+0000 7fa40b7fe640 1 --2- 172.21.15.143:0/2257136655 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fa40c159690 0x7fa40c1ee1d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:18:02.435 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:02.430+0000 7fa40affd640 1 -- 172.21.15.143:0/2257136655 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fa40c159690 msgr2=0x7fa40c1ee1d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:18:02.435 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:02.430+0000 7fa40affd640 1 --2- 172.21.15.143:0/2257136655 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fa40c159690 0x7fa40c1ee1d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:18:02.435 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:02.430+0000 7fa40affd640 1 -- 172.21.15.143:0/2257136655 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fa40c153930 msgr2=0x7fa40c152890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:18:02.436 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:02.430+0000 7fa40affd640 1 --2- 172.21.15.143:0/2257136655 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fa40c153930 0x7fa40c152890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:18:02.436 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:02.430+0000 7fa40affd640 1 -- 172.21.15.143:0/2257136655 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa3fc0094d0 con 0x7fa40c153550 2023-11-26T07:18:02.436 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:02.430+0000 7fa599e15640 1 -- 172.21.15.143:0/1355198621 learned_addr learned my addr 172.21.15.143:0/1355198621 (peer_addr_for_me v2:172.21.15.143:0/0) 2023-11-26T07:18:02.436 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:02.430+0000 7fa59ae17640 1 -- 172.21.15.143:0/1355198621 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa5941f3ca0 con 0x7fa594160060 2023-11-26T07:18:02.436 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:02.430+0000 7fa59ae17640 1 -- 172.21.15.143:0/1355198621 --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa594163420 con 0x7fa5940cc3f0 2023-11-26T07:18:02.436 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:02.430+0000 7fa59a616640 1 --2- 172.21.15.143:0/1355198621 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fa594160060 0x7fa59415c930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:18:02.436 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:02.430+0000 7fa599614640 1 -- 172.21.15.143:0/1355198621 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fa5940cc3f0 msgr2=0x7fa59415aa30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:18:02.437 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:02.430+0000 7fa599614640 1 --2- 172.21.15.143:0/1355198621 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fa5940cc3f0 0x7fa59415aa30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:18:02.437 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:02.430+0000 7fa599614640 1 -- 172.21.15.143:0/1355198621 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fa594160060 msgr2=0x7fa59415c930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:18:02.437 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:02.430+0000 7fa599614640 1 --2- 172.21.15.143:0/1355198621 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fa594160060 0x7fa59415c930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:18:02.437 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:02.430+0000 7fa599614640 1 -- 172.21.15.143:0/1355198621 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa584009420 con 0x7fa59415f3d0 2023-11-26T07:18:02.437 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:02.430+0000 7fa599e15640 1 --2- 172.21.15.143:0/1355198621 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fa5940cc3f0 0x7fa59415aa30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-26T07:18:02.437 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:02.430+0000 7fa59a616640 1 --2- 172.21.15.143:0/1355198621 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fa594160060 0x7fa59415c930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-26T07:18:02.437 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:02.430+0000 7fa40affd640 1 --2- 172.21.15.143:0/2257136655 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fa40c153550 0x7fa40c1ef780 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7fa3fc0ae040 tx=0x7fa3fc0192c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:18:02.437 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:02.430+0000 7fa3f37fe640 1 -- 172.21.15.143:0/2257136655 <== mon.1 v2:172.21.15.188:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa3fc003ab0 con 0x7fa40c153550 2023-11-26T07:18:02.438 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:02.430+0000 7fa599614640 1 --2- 172.21.15.143:0/1355198621 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fa59415f3d0 0x7fa59415fb20 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7fa59406e380 tx=0x7fa584019210 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:18:02.438 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:02.430+0000 7fa58a7fc640 1 -- 172.21.15.143:0/1355198621 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa5840b4810 con 0x7fa59415f3d0 2023-11-26T07:18:02.438 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:02.430+0000 7fa58a7fc640 1 -- 172.21.15.143:0/1355198621 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa5840b4970 con 0x7fa59415f3d0 2023-11-26T07:18:02.438 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:02.430+0000 7fa58a7fc640 1 -- 172.21.15.143:0/1355198621 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa5840b4d20 con 0x7fa59415f3d0 2023-11-26T07:18:02.441 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:02.434+0000 7fa40bfff640 1 -- 172.21.15.143:0/2257136655 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa40c0c40b0 con 0x7fa40c153550 2023-11-26T07:18:02.441 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:02.434+0000 7fa40bfff640 1 -- 172.21.15.143:0/2257136655 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa40c1eecc0 con 0x7fa40c153550 2023-11-26T07:18:02.441 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:02.434+0000 7fa3f37fe640 1 -- 172.21.15.143:0/2257136655 <== mon.1 v2:172.21.15.188:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa3fc003c10 con 0x7fa40c153550 2023-11-26T07:18:02.442 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:02.434+0000 7fa3f37fe640 1 -- 172.21.15.143:0/2257136655 <== mon.1 v2:172.21.15.188:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa3fc003ab0 con 0x7fa40c153550 2023-11-26T07:18:02.442 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:02.434+0000 7fa59ae17640 1 -- 172.21.15.143:0/1355198621 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa594160e60 con 0x7fa59415f3d0 2023-11-26T07:18:02.442 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:02.434+0000 7fa59ae17640 1 -- 172.21.15.143:0/1355198621 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa5941608a0 con 0x7fa59415f3d0 2023-11-26T07:18:02.442 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:02.434+0000 7fa40bfff640 1 -- 172.21.15.143:0/2257136655 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa3ec005260 con 0x7fa40c153550 2023-11-26T07:18:02.444 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:02.438+0000 7fa3f37fe640 1 -- 172.21.15.143:0/2257136655 <== mon.1 v2:172.21.15.188:3300/0 4 ==== mgrmap(e 5) v1 ==== 85852+0+0 (secure 0 0 0) 0x7fa3fc0b2070 con 0x7fa40c153550 2023-11-26T07:18:02.444 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:02.438+0000 7fa58a7fc640 1 -- 172.21.15.143:0/1355198621 <== mon.0 v2:172.21.15.143:3300/0 4 ==== mgrmap(e 5) v1 ==== 85852+0+0 (secure 0 0 0) 0x7fa5840b2040 con 0x7fa59415f3d0 2023-11-26T07:18:02.444 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:02.438+0000 7fa59ae17640 1 -- 172.21.15.143:0/1355198621 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa594163590 con 0x7fa59415f3d0 2023-11-26T07:18:02.444 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:02.438+0000 7fa58a7fc640 1 --2- 172.21.15.143:0/1355198621 >> [v2:172.21.15.143:6832/2150557544,v1:172.21.15.143:6833/2150557544] conn(0x7fa5780664a0 0x7fa578068960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:18:02.444 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:02.438+0000 7fa58a7fc640 1 -- 172.21.15.143:0/1355198621 <== mon.0 v2:172.21.15.143:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fa58411df90 con 0x7fa59415f3d0 2023-11-26T07:18:02.444 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:02.438+0000 7fa599e15640 1 --2- 172.21.15.143:0/1355198621 >> [v2:172.21.15.143:6832/2150557544,v1:172.21.15.143:6833/2150557544] conn(0x7fa5780664a0 0x7fa578068960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:18:02.445 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:02.442+0000 7fa599e15640 1 --2- 172.21.15.143:0/1355198621 >> [v2:172.21.15.143:6832/2150557544,v1:172.21.15.143:6833/2150557544] conn(0x7fa5780664a0 0x7fa578068960 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7fa5940706a0 tx=0x7fa590009240 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:18:02.445 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:02.442+0000 7fa3f37fe640 1 --2- 172.21.15.143:0/2257136655 >> [v2:172.21.15.143:6832/2150557544,v1:172.21.15.143:6833/2150557544] conn(0x7fa3f80664d0 0x7fa3f8068990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:18:02.446 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:02.442+0000 7fa40a7fc640 1 --2- 172.21.15.143:0/2257136655 >> [v2:172.21.15.143:6832/2150557544,v1:172.21.15.143:6833/2150557544] conn(0x7fa3f80664d0 0x7fa3f8068990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:18:02.448 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:02.442+0000 7fa3f37fe640 1 -- 172.21.15.143:0/2257136655 <== mon.1 v2:172.21.15.188:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fa3fc11cd40 con 0x7fa40c153550 2023-11-26T07:18:02.448 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:02.442+0000 7fa40a7fc640 1 --2- 172.21.15.143:0/2257136655 >> [v2:172.21.15.143:6832/2150557544,v1:172.21.15.143:6833/2150557544] conn(0x7fa3f80664d0 0x7fa3f8068990 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7fa40c162ff0 tx=0x7fa4000079c0 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:18:02.448 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:02.446+0000 7fa3f37fe640 1 -- 172.21.15.143:0/2257136655 <== mon.1 v2:172.21.15.188:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172649 (secure 0 0 0) 0x7fa3fc0e99f0 con 0x7fa40c153550 2023-11-26T07:18:02.452 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:02.446+0000 7fa58a7fc640 1 -- 172.21.15.143:0/1355198621 <== mon.0 v2:172.21.15.143:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172649 (secure 0 0 0) 0x7fa5840eb4c0 con 0x7fa59415f3d0 2023-11-26T07:18:02.648 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:02.642+0000 7f00c18e3640 1 -- 172.21.15.143:0/906989677 --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 3} v 0) v1 -- 0x7f00bc14c830 con 0x7f00bc0a8e90 2023-11-26T07:18:02.649 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:02.646+0000 7f00a3fff640 1 -- 172.21.15.143:0/906989677 <== mon.2 v2:172.21.15.188:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 3}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7f00bc14c830 con 0x7f00bc0a8e90 2023-11-26T07:18:02.650 INFO:teuthology.orchestra.run.smithi143.stdout:55834574856 2023-11-26T07:18:02.653 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:02.650+0000 7f00c18e3640 1 -- 172.21.15.143:0/906989677 >> [v2:172.21.15.143:6832/2150557544,v1:172.21.15.143:6833/2150557544] conn(0x7f00a8066470 msgr2=0x7f00a8068930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:18:02.653 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:02.650+0000 7f00c18e3640 1 --2- 172.21.15.143:0/906989677 >> [v2:172.21.15.143:6832/2150557544,v1:172.21.15.143:6833/2150557544] conn(0x7f00a8066470 0x7f00a8068930 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7f00ac0b2000 tx=0x7f00ac00a040 comp rx=0 tx=0).stop 2023-11-26T07:18:02.653 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:02.650+0000 7f00c18e3640 1 -- 172.21.15.143:0/906989677 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f00bc0a8e90 msgr2=0x7f00bc151850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:18:02.654 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:02.650+0000 7f00c18e3640 1 --2- 172.21.15.143:0/906989677 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f00bc0a8e90 0x7f00bc151850 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7f00b000b3e0 tx=0x7f00b0009f90 comp rx=0 tx=0).stop 2023-11-26T07:18:02.654 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:02.650+0000 7f00c18e3640 1 -- 172.21.15.143:0/906989677 shutdown_connections 2023-11-26T07:18:02.654 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:02.650+0000 7f00c18e3640 1 --2- 172.21.15.143:0/906989677 >> [v2:172.21.15.143:6832/2150557544,v1:172.21.15.143:6833/2150557544] conn(0x7f00a8066470 0x7f00a8068930 unknown :-1 s=CLOSED pgs=93 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:18:02.655 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:02.650+0000 7f00c18e3640 1 --2- 172.21.15.143:0/906989677 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f00bc0a8e90 0x7f00bc151850 unknown :-1 s=CLOSED pgs=51 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:18:02.655 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:02.650+0000 7f00c18e3640 1 --2- 172.21.15.143:0/906989677 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f00bc15a440 0x7f00bc0a8950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:18:02.655 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:02.650+0000 7f00c18e3640 1 --2- 172.21.15.143:0/906989677 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f00bc0b20b0 0x7f00bc137b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:18:02.655 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:02.650+0000 7f00c18e3640 1 -- 172.21.15.143:0/906989677 >> 172.21.15.143:0/906989677 conn(0x7f00bc0aea10 msgr2=0x7f00bc0aedf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:18:02.655 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:02.650+0000 7f00c18e3640 1 -- 172.21.15.143:0/906989677 shutdown_connections 2023-11-26T07:18:02.655 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:02.650+0000 7f00c18e3640 1 -- 172.21.15.143:0/906989677 wait complete. 2023-11-26T07:18:02.690 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574856 got 55834574856 for osd.3 2023-11-26T07:18:02.690 DEBUG:teuthology.parallel:result is None 2023-11-26T07:18:02.703 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:02.698+0000 7fef6cff9640 1 -- 172.21.15.143:0/1859405342 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 2} v 0) v1 -- 0x7fef58005510 con 0x7fef8006ce40 2023-11-26T07:18:02.704 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:02.702+0000 7fef6effd640 1 -- 172.21.15.143:0/1859405342 <== mon.1 v2:172.21.15.188:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 2}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7fef700b0090 con 0x7fef8006ce40 2023-11-26T07:18:02.709 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:02.706+0000 7f41900d6640 1 -- 172.21.15.143:0/3086785269 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 1} v 0) v1 -- 0x7f418003a9c0 con 0x7f41800e77f0 2023-11-26T07:18:02.709 INFO:teuthology.orchestra.run.smithi143.stdout:55834574856 2023-11-26T07:18:02.709 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:02.706+0000 7f417e7fc640 1 -- 172.21.15.143:0/3086785269 <== mon.1 v2:172.21.15.188:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 1}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7f418003a9c0 con 0x7f41800e77f0 2023-11-26T07:18:02.710 INFO:teuthology.orchestra.run.smithi143.stdout:55834574855 2023-11-26T07:18:02.713 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:02.706+0000 7f41900d6640 1 -- 172.21.15.143:0/3086785269 >> [v2:172.21.15.143:6832/2150557544,v1:172.21.15.143:6833/2150557544] conn(0x7f416c066500 msgr2=0x7f416c0689c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:18:02.713 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:02.706+0000 7f41900d6640 1 --2- 172.21.15.143:0/3086785269 >> [v2:172.21.15.143:6832/2150557544,v1:172.21.15.143:6833/2150557544] conn(0x7f416c066500 0x7f416c0689c0 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7f41800daff0 tx=0x7f4178008040 comp rx=0 tx=0).stop 2023-11-26T07:18:02.713 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:02.706+0000 7f41900d6640 1 -- 172.21.15.143:0/3086785269 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f41800e77f0 msgr2=0x7f4180034360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:18:02.713 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:02.706+0000 7f41900d6640 1 --2- 172.21.15.143:0/3086785269 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f41800e77f0 0x7f4180034360 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7f41840b3040 tx=0x7f418401ba90 comp rx=0 tx=0).stop 2023-11-26T07:18:02.714 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:02.706+0000 7f41900d6640 1 -- 172.21.15.143:0/3086785269 shutdown_connections 2023-11-26T07:18:02.714 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:02.710+0000 7f41900d6640 1 --2- 172.21.15.143:0/3086785269 >> [v2:172.21.15.143:6832/2150557544,v1:172.21.15.143:6833/2150557544] conn(0x7f416c066500 0x7f416c0689c0 unknown :-1 s=CLOSED pgs=94 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:18:02.714 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:02.710+0000 7f41900d6640 1 --2- 172.21.15.143:0/3086785269 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f41800d6cd0 0x7f41800da3a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:18:02.714 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:02.710+0000 7f41900d6640 1 --2- 172.21.15.143:0/3086785269 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f4180035ae0 0x7f41800ccd80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:18:02.714 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:02.710+0000 7f41900d6640 1 --2- 172.21.15.143:0/3086785269 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f41800e77f0 0x7f4180034360 unknown :-1 s=CLOSED pgs=85 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:18:02.714 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:02.710+0000 7f41900d6640 1 -- 172.21.15.143:0/3086785269 >> 172.21.15.143:0/3086785269 conn(0x7f4180009f90 msgr2=0x7f4180049810 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:18:02.714 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:02.710+0000 7f41900d6640 1 -- 172.21.15.143:0/3086785269 shutdown_connections 2023-11-26T07:18:02.715 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:02.710+0000 7f41900d6640 1 -- 172.21.15.143:0/3086785269 wait complete. 2023-11-26T07:18:02.715 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:02.710+0000 7fef6cff9640 1 -- 172.21.15.143:0/1859405342 >> [v2:172.21.15.143:6832/2150557544,v1:172.21.15.143:6833/2150557544] conn(0x7fef600664a0 msgr2=0x7fef60068960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:18:02.715 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:02.710+0000 7fef6cff9640 1 --2- 172.21.15.143:0/1859405342 >> [v2:172.21.15.143:6832/2150557544,v1:172.21.15.143:6833/2150557544] conn(0x7fef600664a0 0x7fef60068960 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7fef74006fd0 tx=0x7fef74008040 comp rx=0 tx=0).stop 2023-11-26T07:18:02.715 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:02.710+0000 7fef6cff9640 1 -- 172.21.15.143:0/1859405342 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fef8006ce40 msgr2=0x7fef801415f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:18:02.716 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:02.710+0000 7fef6cff9640 1 --2- 172.21.15.143:0/1859405342 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fef8006ce40 0x7fef801415f0 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7fef700ab000 tx=0x7fef70019190 comp rx=0 tx=0).stop 2023-11-26T07:18:02.716 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:02.710+0000 7f5753c5a640 1 -- 172.21.15.143:0/1893192248 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 7} v 0) v1 -- 0x7f574c06d920 con 0x7f574c070fd0 2023-11-26T07:18:02.716 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:02.710+0000 7fef6cff9640 1 -- 172.21.15.143:0/1859405342 shutdown_connections 2023-11-26T07:18:02.716 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:02.710+0000 7fef6cff9640 1 --2- 172.21.15.143:0/1859405342 >> [v2:172.21.15.143:6832/2150557544,v1:172.21.15.143:6833/2150557544] conn(0x7fef600664a0 0x7fef60068960 unknown :-1 s=CLOSED pgs=95 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:18:02.717 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:02.710+0000 7fef6cff9640 1 --2- 172.21.15.143:0/1859405342 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fef80159270 0x7fef80150310 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:18:02.717 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:02.710+0000 7fef6cff9640 1 --2- 172.21.15.143:0/1859405342 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fef800ce7c0 0x7fef80166580 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:18:02.717 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:02.710+0000 7fef6cff9640 1 --2- 172.21.15.143:0/1859405342 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fef8006ce40 0x7fef801415f0 unknown :-1 s=CLOSED pgs=87 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:18:02.717 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:02.710+0000 7fef6cff9640 1 -- 172.21.15.143:0/1859405342 >> 172.21.15.143:0/1859405342 conn(0x7fef8006a890 msgr2=0x7fef8015bd20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:18:02.717 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:02.710+0000 7f6299817640 1 -- 172.21.15.143:0/1201588543 --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 0} v 0) v1 -- 0x7f62780057e0 con 0x7f62940b1710 2023-11-26T07:18:02.718 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:02.710+0000 7fef6cff9640 1 -- 172.21.15.143:0/1859405342 shutdown_connections 2023-11-26T07:18:02.718 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:02.710+0000 7fef6cff9640 1 -- 172.21.15.143:0/1859405342 wait complete. 2023-11-26T07:18:02.718 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:02.710+0000 7f57427fc640 1 -- 172.21.15.143:0/1893192248 <== mon.1 v2:172.21.15.188:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 7}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7f5748054900 con 0x7f574c070fd0 2023-11-26T07:18:02.719 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:02.710+0000 7f628bfff640 1 -- 172.21.15.143:0/1201588543 <== mon.2 v2:172.21.15.188:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 0}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7f627c053f00 con 0x7f62940b1710 2023-11-26T07:18:02.719 INFO:teuthology.orchestra.run.smithi143.stdout:55834574855 2023-11-26T07:18:02.719 INFO:teuthology.orchestra.run.smithi143.stdout:55834574853 2023-11-26T07:18:02.719 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:02.714+0000 7f6299817640 1 -- 172.21.15.143:0/1201588543 >> [v2:172.21.15.143:6832/2150557544,v1:172.21.15.143:6833/2150557544] conn(0x7f62740664c0 msgr2=0x7f6274068980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:18:02.720 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:02.714+0000 7f6299817640 1 --2- 172.21.15.143:0/1201588543 >> [v2:172.21.15.143:6832/2150557544,v1:172.21.15.143:6833/2150557544] conn(0x7f62740664c0 0x7f6274068980 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7f628c00abb0 tx=0x7f628c009250 comp rx=0 tx=0).stop 2023-11-26T07:18:02.720 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:02.714+0000 7f6299817640 1 -- 172.21.15.143:0/1201588543 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f62940b1710 msgr2=0x7f629415b6f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:18:02.720 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:02.714+0000 7f6299817640 1 --2- 172.21.15.143:0/1201588543 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f62940b1710 0x7f629415b6f0 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7f627c00c400 tx=0x7f627c00c8c0 comp rx=0 tx=0).stop 2023-11-26T07:18:02.720 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:02.714+0000 7f6299817640 1 -- 172.21.15.143:0/1201588543 shutdown_connections 2023-11-26T07:18:02.720 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:02.714+0000 7f6299817640 1 --2- 172.21.15.143:0/1201588543 >> [v2:172.21.15.143:6832/2150557544,v1:172.21.15.143:6833/2150557544] conn(0x7f62740664c0 0x7f6274068980 unknown :-1 s=CLOSED pgs=96 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:18:02.720 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:02.714+0000 7f6299817640 1 --2- 172.21.15.143:0/1201588543 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f62940ce310 0x7f6294156a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:18:02.720 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:02.714+0000 7f6299817640 1 --2- 172.21.15.143:0/1201588543 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f62940b1710 0x7f629415b6f0 secure :-1 s=CLOSED pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7f627c00c400 tx=0x7f627c00c8c0 comp rx=0 tx=0).stop 2023-11-26T07:18:02.720 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:02.714+0000 7f6299817640 1 --2- 172.21.15.143:0/1201588543 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f6294070370 0x7f629415c6e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:18:02.720 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:02.714+0000 7f6299817640 1 -- 172.21.15.143:0/1201588543 >> 172.21.15.143:0/1201588543 conn(0x7f62940b0bf0 msgr2=0x7f6294166de0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:18:02.721 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:02.714+0000 7f6299817640 1 -- 172.21.15.143:0/1201588543 shutdown_connections 2023-11-26T07:18:02.721 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:02.714+0000 7f6299817640 1 -- 172.21.15.143:0/1201588543 wait complete. 2023-11-26T07:18:02.721 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:02.714+0000 7f5753c5a640 1 -- 172.21.15.143:0/1893192248 >> [v2:172.21.15.143:6832/2150557544,v1:172.21.15.143:6833/2150557544] conn(0x7f57300664c0 msgr2=0x7f5730068980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:18:02.721 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:02.714+0000 7f5753c5a640 1 --2- 172.21.15.143:0/1893192248 >> [v2:172.21.15.143:6832/2150557544,v1:172.21.15.143:6833/2150557544] conn(0x7f57300664c0 0x7f5730068980 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7f574c157a40 tx=0x7f5744003e70 comp rx=0 tx=0).stop 2023-11-26T07:18:02.721 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:02.714+0000 7f5753c5a640 1 -- 172.21.15.143:0/1893192248 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f574c070fd0 msgr2=0x7f574c0add50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:18:02.722 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:02.714+0000 7f5753c5a640 1 --2- 172.21.15.143:0/1893192248 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f574c070fd0 0x7f574c0add50 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7f574800c540 tx=0x7f574800ca00 comp rx=0 tx=0).stop 2023-11-26T07:18:02.722 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:02.714+0000 7f5753c5a640 1 -- 172.21.15.143:0/1893192248 shutdown_connections 2023-11-26T07:18:02.722 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:02.714+0000 7f5753c5a640 1 --2- 172.21.15.143:0/1893192248 >> [v2:172.21.15.143:6832/2150557544,v1:172.21.15.143:6833/2150557544] conn(0x7f57300664c0 0x7f5730068980 unknown :-1 s=CLOSED pgs=97 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:18:02.722 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:02.714+0000 7f5753c5a640 1 --2- 172.21.15.143:0/1893192248 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f574c14b1e0 0x7f574c0cd950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:18:02.722 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:02.714+0000 7f5753c5a640 1 --2- 172.21.15.143:0/1893192248 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f574c0cf7c0 0x7f574c0cd410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:18:02.722 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:02.714+0000 7f5753c5a640 1 --2- 172.21.15.143:0/1893192248 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f574c070fd0 0x7f574c0add50 unknown :-1 s=CLOSED pgs=90 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:18:02.722 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:02.714+0000 7f5753c5a640 1 -- 172.21.15.143:0/1893192248 >> 172.21.15.143:0/1893192248 conn(0x7f574c06a890 msgr2=0x7f574c16cbc0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:18:02.722 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:02.718+0000 7f5753c5a640 1 -- 172.21.15.143:0/1893192248 shutdown_connections 2023-11-26T07:18:02.722 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:02.718+0000 7f5753c5a640 1 -- 172.21.15.143:0/1893192248 wait complete. 2023-11-26T07:18:02.735 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:02.730+0000 7fa401a68640 1 -- 172.21.15.143:0/27832242 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 5} v 0) v1 -- 0x7fa3fc067680 con 0x7fa3fc0b2160 2023-11-26T07:18:02.736 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:02.730+0000 7fa3e3fff640 1 -- 172.21.15.143:0/27832242 <== mon.0 v2:172.21.15.143:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 5}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7fa3f0053a80 con 0x7fa3fc0b2160 2023-11-26T07:18:02.736 INFO:teuthology.orchestra.run.smithi143.stdout:55834574853 2023-11-26T07:18:02.740 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:02.734+0000 7fa401a68640 1 -- 172.21.15.143:0/27832242 >> [v2:172.21.15.143:6832/2150557544,v1:172.21.15.143:6833/2150557544] conn(0x7fa3e80664d0 msgr2=0x7fa3e8068990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:18:02.740 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:02.734+0000 7fa401a68640 1 --2- 172.21.15.143:0/27832242 >> [v2:172.21.15.143:6832/2150557544,v1:172.21.15.143:6833/2150557544] conn(0x7fa3e80664d0 0x7fa3e8068990 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7fa3fc071b90 tx=0x7fa3ec009040 comp rx=0 tx=0).stop 2023-11-26T07:18:02.740 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:02.734+0000 7fa401a68640 1 -- 172.21.15.143:0/27832242 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fa3fc0b2160 msgr2=0x7fa3fc154500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:18:02.740 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:02.734+0000 7fa401a68640 1 --2- 172.21.15.143:0/27832242 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fa3fc0b2160 0x7fa3fc154500 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7fa3f000c400 tx=0x7fa3f000c8c0 comp rx=0 tx=0).stop 2023-11-26T07:18:02.741 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:02.738+0000 7fa401a68640 1 -- 172.21.15.143:0/27832242 shutdown_connections 2023-11-26T07:18:02.741 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:02.738+0000 7fa401a68640 1 --2- 172.21.15.143:0/27832242 >> [v2:172.21.15.143:6832/2150557544,v1:172.21.15.143:6833/2150557544] conn(0x7fa3e80664d0 0x7fa3e8068990 unknown :-1 s=CLOSED pgs=98 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:18:02.741 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:02.738+0000 7fa401a68640 1 --2- 172.21.15.143:0/27832242 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fa3fc15d6e0 0x7fa3fc1fd880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:18:02.741 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:02.738+0000 7fa401a68640 1 --2- 172.21.15.143:0/27832242 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fa3fc0d1bc0 0x7fa3fc1fd340 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:18:02.741 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:02.738+0000 7fa401a68640 1 --2- 172.21.15.143:0/27832242 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fa3fc0b2160 0x7fa3fc154500 unknown :-1 s=CLOSED pgs=114 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:18:02.741 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:02.738+0000 7fa401a68640 1 -- 172.21.15.143:0/27832242 >> 172.21.15.143:0/27832242 conn(0x7fa3fc09c550 msgr2=0x7fa3fc06c590 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:18:02.745 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:02.742+0000 7fa401a68640 1 -- 172.21.15.143:0/27832242 shutdown_connections 2023-11-26T07:18:02.745 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:02.742+0000 7fa401a68640 1 -- 172.21.15.143:0/27832242 wait complete. 2023-11-26T07:18:02.746 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:02.742+0000 7fa40bfff640 1 -- 172.21.15.143:0/2257136655 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 6} v 0) v1 -- 0x7fa3ec005510 con 0x7fa40c153550 2023-11-26T07:18:02.746 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:02.742+0000 7fa3f37fe640 1 -- 172.21.15.143:0/2257136655 <== mon.1 v2:172.21.15.188:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 6}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7fa3fc0b0090 con 0x7fa40c153550 2023-11-26T07:18:02.748 INFO:teuthology.orchestra.run.smithi143.stdout:55834574855 2023-11-26T07:18:02.749 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:02.746+0000 7fa40bfff640 1 -- 172.21.15.143:0/2257136655 >> [v2:172.21.15.143:6832/2150557544,v1:172.21.15.143:6833/2150557544] conn(0x7fa3f80664d0 msgr2=0x7fa3f8068990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:18:02.749 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:02.746+0000 7fa40bfff640 1 --2- 172.21.15.143:0/2257136655 >> [v2:172.21.15.143:6832/2150557544,v1:172.21.15.143:6833/2150557544] conn(0x7fa3f80664d0 0x7fa3f8068990 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7fa40c162ff0 tx=0x7fa4000079c0 comp rx=0 tx=0).stop 2023-11-26T07:18:02.750 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:02.746+0000 7fa40bfff640 1 -- 172.21.15.143:0/2257136655 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fa40c153550 msgr2=0x7fa40c1ef780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:18:02.750 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:02.746+0000 7fa40bfff640 1 --2- 172.21.15.143:0/2257136655 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fa40c153550 0x7fa40c1ef780 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7fa3fc0ae040 tx=0x7fa3fc0192c0 comp rx=0 tx=0).stop 2023-11-26T07:18:02.750 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:02.746+0000 7fa40bfff640 1 -- 172.21.15.143:0/2257136655 shutdown_connections 2023-11-26T07:18:02.750 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:02.746+0000 7fa40bfff640 1 --2- 172.21.15.143:0/2257136655 >> [v2:172.21.15.143:6832/2150557544,v1:172.21.15.143:6833/2150557544] conn(0x7fa3f80664d0 0x7fa3f8068990 unknown :-1 s=CLOSED pgs=100 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:18:02.750 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:02.746+0000 7fa40bfff640 1 --2- 172.21.15.143:0/2257136655 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fa40c159690 0x7fa40c1ee1d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:18:02.750 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:02.746+0000 7fa40bfff640 1 --2- 172.21.15.143:0/2257136655 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fa40c153930 0x7fa40c152890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:18:02.750 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:02.746+0000 7fa40bfff640 1 --2- 172.21.15.143:0/2257136655 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fa40c153550 0x7fa40c1ef780 secure :-1 s=CLOSED pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7fa3fc0ae040 tx=0x7fa3fc0192c0 comp rx=0 tx=0).stop 2023-11-26T07:18:02.751 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:02.746+0000 7fa40bfff640 1 -- 172.21.15.143:0/2257136655 >> 172.21.15.143:0/2257136655 conn(0x7fa40c06a890 msgr2=0x7fa40c163020 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:18:02.751 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:02.746+0000 7fa40bfff640 1 -- 172.21.15.143:0/2257136655 shutdown_connections 2023-11-26T07:18:02.751 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:02.746+0000 7fa40bfff640 1 -- 172.21.15.143:0/2257136655 wait complete. 2023-11-26T07:18:02.754 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574855 got 55834574855 for osd.1 2023-11-26T07:18:02.754 DEBUG:teuthology.parallel:result is None 2023-11-26T07:18:02.756 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574856 got 55834574856 for osd.2 2023-11-26T07:18:02.757 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:02.750+0000 7fa59ae17640 1 -- 172.21.15.143:0/1355198621 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 4} v 0) v1 -- 0x7fa594067500 con 0x7fa59415f3d0 2023-11-26T07:18:02.757 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:02.750+0000 7fa58a7fc640 1 -- 172.21.15.143:0/1355198621 <== mon.0 v2:172.21.15.143:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 4}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7fa5840b0090 con 0x7fa59415f3d0 2023-11-26T07:18:02.757 DEBUG:teuthology.parallel:result is None 2023-11-26T07:18:02.757 INFO:teuthology.orchestra.run.smithi143.stdout:55834574855 2023-11-26T07:18:02.759 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574855 got 55834574855 for osd.0 2023-11-26T07:18:02.759 DEBUG:teuthology.parallel:result is None 2023-11-26T07:18:02.760 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574853 got 55834574853 for osd.7 2023-11-26T07:18:02.760 DEBUG:teuthology.parallel:result is None 2023-11-26T07:18:02.761 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:02.754+0000 7fa59ae17640 1 -- 172.21.15.143:0/1355198621 >> [v2:172.21.15.143:6832/2150557544,v1:172.21.15.143:6833/2150557544] conn(0x7fa5780664a0 msgr2=0x7fa578068960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:18:02.761 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:02.754+0000 7fa59ae17640 1 --2- 172.21.15.143:0/1355198621 >> [v2:172.21.15.143:6832/2150557544,v1:172.21.15.143:6833/2150557544] conn(0x7fa5780664a0 0x7fa578068960 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7fa5940706a0 tx=0x7fa590009240 comp rx=0 tx=0).stop 2023-11-26T07:18:02.761 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:02.754+0000 7fa59ae17640 1 -- 172.21.15.143:0/1355198621 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fa59415f3d0 msgr2=0x7fa59415fb20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:18:02.761 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:02.754+0000 7fa59ae17640 1 --2- 172.21.15.143:0/1355198621 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fa59415f3d0 0x7fa59415fb20 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7fa59406e380 tx=0x7fa584019210 comp rx=0 tx=0).stop 2023-11-26T07:18:02.761 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:02.754+0000 7fa59ae17640 1 -- 172.21.15.143:0/1355198621 shutdown_connections 2023-11-26T07:18:02.761 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:02.754+0000 7fa59ae17640 1 --2- 172.21.15.143:0/1355198621 >> [v2:172.21.15.143:6832/2150557544,v1:172.21.15.143:6833/2150557544] conn(0x7fa5780664a0 0x7fa578068960 unknown :-1 s=CLOSED pgs=99 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:18:02.761 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:02.754+0000 7fa59ae17640 1 --2- 172.21.15.143:0/1355198621 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fa594160060 0x7fa59415c930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:18:02.761 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:02.754+0000 7fa59ae17640 1 --2- 172.21.15.143:0/1355198621 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fa59415f3d0 0x7fa59415fb20 unknown :-1 s=CLOSED pgs=116 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:18:02.761 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:02.754+0000 7fa59ae17640 1 --2- 172.21.15.143:0/1355198621 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fa5940cc3f0 0x7fa59415aa30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:18:02.761 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:02.754+0000 7fa59ae17640 1 -- 172.21.15.143:0/1355198621 >> 172.21.15.143:0/1355198621 conn(0x7fa59406a890 msgr2=0x7fa59406ac70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:18:02.762 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:02.754+0000 7fa59ae17640 1 -- 172.21.15.143:0/1355198621 shutdown_connections 2023-11-26T07:18:02.762 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:02.754+0000 7fa59ae17640 1 -- 172.21.15.143:0/1355198621 wait complete. 2023-11-26T07:18:02.773 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574853 got 55834574853 for osd.5 2023-11-26T07:18:02.773 DEBUG:teuthology.parallel:result is None 2023-11-26T07:18:02.777 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574855 got 55834574855 for osd.6 2023-11-26T07:18:02.778 DEBUG:teuthology.parallel:result is None 2023-11-26T07:18:02.782 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574855 got 55834574855 for osd.4 2023-11-26T07:18:02.783 DEBUG:teuthology.parallel:result is None 2023-11-26T07:18:02.783 INFO:tasks.ceph.ceph_manager.ceph:waiting for clean 2023-11-26T07:18:02.783 DEBUG:teuthology.orchestra.run.smithi143:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-11-26T07:18:02.965 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:02.962+0000 7f4c3c225640 1 Processor -- start 2023-11-26T07:18:02.965 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:02.962+0000 7f4c3c225640 1 -- start start 2023-11-26T07:18:02.967 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:02.962+0000 7f4c3c225640 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7f4c3406f000 0x7f4c3406d030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:18:02.967 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:02.962+0000 7f4c3c225640 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7f4c34099a70 0x7f4c3406bcb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:18:02.967 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:02.962+0000 7f4c3c225640 1 -- --> v1:172.21.15.188:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4c340cf260 con 0x7f4c3406f3e0 2023-11-26T07:18:02.967 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:02.962+0000 7f4c3c225640 1 -- --> v2:172.21.15.143:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4c3406d570 con 0x7f4c3406f000 2023-11-26T07:18:02.967 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:02.962+0000 7f4c3c225640 1 -- --> v2:172.21.15.188:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4c3409a010 con 0x7f4c34099a70 2023-11-26T07:18:02.967 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:02.962+0000 7f4c39fc1640 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7f4c3406f000 0x7f4c3406d030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:18:02.967 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:02.962+0000 7f4c39fc1640 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7f4c3406f000 0x7f4c3406d030 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:46758/0 (socket says 172.21.15.143:46758) 2023-11-26T07:18:02.967 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:02.962+0000 7f4c39fc1640 1 -- 172.21.15.143:0/409266349 learned_addr learned my addr 172.21.15.143:0/409266349 (peer_addr_for_me v2:172.21.15.143:0/0) 2023-11-26T07:18:02.968 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:02.962+0000 7f4c3a7c2640 1 --2- 172.21.15.143:0/409266349 >> v2:172.21.15.188:3300/0 conn(0x7f4c34099a70 0x7f4c3406bcb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:18:02.968 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:02.962+0000 7f4c39fc1640 1 -- 172.21.15.143:0/409266349 >> v2:172.21.15.188:3300/0 conn(0x7f4c34099a70 msgr2=0x7f4c3406bcb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-26T07:18:02.968 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:02.962+0000 7f4c39fc1640 1 --2- 172.21.15.143:0/409266349 >> v2:172.21.15.188:3300/0 conn(0x7f4c34099a70 0x7f4c3406bcb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:18:02.968 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:02.962+0000 7f4c39fc1640 1 -- 172.21.15.143:0/409266349 >> v1:172.21.15.188:6789/0 conn(0x7f4c3406f3e0 legacy=0x7f4c3406b5a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-26T07:18:02.968 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:02.962+0000 7f4c39fc1640 1 -- 172.21.15.143:0/409266349 --> v2:172.21.15.143:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4c3409a2f0 con 0x7f4c3406f000 2023-11-26T07:18:02.968 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:02.962+0000 7f4c3a7c2640 1 --2- 172.21.15.143:0/409266349 >> v2:172.21.15.188:3300/0 conn(0x7f4c34099a70 0x7f4c3406bcb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-26T07:18:02.970 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:02.966+0000 7f4c39fc1640 1 --2- 172.21.15.143:0/409266349 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f4c3406f000 0x7f4c3406d030 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7f4c340cb800 tx=0x7f4c3001ecb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d15c52a0c2fbacc8 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:18:02.970 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:02.966+0000 7f4c38fbf640 1 -- 172.21.15.143:0/409266349 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4c300a89b0 con 0x7f4c3406f000 2023-11-26T07:18:02.971 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:02.966+0000 7f4c38fbf640 1 -- 172.21.15.143:0/409266349 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4c300a8b10 con 0x7f4c3406f000 2023-11-26T07:18:02.971 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:02.966+0000 7f4c38fbf640 1 -- 172.21.15.143:0/409266349 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4c300a8ea0 con 0x7f4c3406f000 2023-11-26T07:18:02.971 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:02.966+0000 7f4c3c225640 1 -- 172.21.15.143:0/409266349 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f4c3406f000 msgr2=0x7f4c3406d030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:18:02.972 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:02.966+0000 7f4c3c225640 1 --2- 172.21.15.143:0/409266349 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f4c3406f000 0x7f4c3406d030 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7f4c340cb800 tx=0x7f4c3001ecb0 comp rx=0 tx=0).stop 2023-11-26T07:18:02.972 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:02.966+0000 7f4c3c225640 1 -- 172.21.15.143:0/409266349 shutdown_connections 2023-11-26T07:18:02.972 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:02.966+0000 7f4c3c225640 1 --2- 172.21.15.143:0/409266349 >> v2:172.21.15.188:3300/0 conn(0x7f4c34099a70 0x7f4c3406bcb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:18:02.972 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:02.966+0000 7f4c3c225640 1 --2- 172.21.15.143:0/409266349 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f4c3406f000 0x7f4c3406d030 unknown :-1 s=CLOSED pgs=117 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:18:02.972 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:02.966+0000 7f4c3c225640 1 -- 172.21.15.143:0/409266349 >> 172.21.15.143:0/409266349 conn(0x7f4c340b2d10 msgr2=0x7f4c340b3120 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:18:02.972 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:02.966+0000 7f4c3c225640 1 -- 172.21.15.143:0/409266349 shutdown_connections 2023-11-26T07:18:02.972 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:02.966+0000 7f4c3c225640 1 -- 172.21.15.143:0/409266349 wait complete. 2023-11-26T07:18:02.973 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:02.970+0000 7f4c3c225640 1 Processor -- start 2023-11-26T07:18:02.973 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:02.970+0000 7f4c3c225640 1 -- start start 2023-11-26T07:18:02.975 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:02.970+0000 7f4c3c225640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f4c3406f000 0x7f4c340cb850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:18:02.975 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:02.970+0000 7f4c3c225640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f4c3406f3e0 0x7f4c3415cc20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:18:02.975 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:02.970+0000 7f4c3c225640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f4c34099a70 0x7f4c3415fea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:18:02.975 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:02.970+0000 7f4c3c225640 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4c340ccd60 con 0x7f4c3406f3e0 2023-11-26T07:18:02.975 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:02.970+0000 7f4c3c225640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4c340cd1f0 con 0x7f4c34099a70 2023-11-26T07:18:02.975 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:02.970+0000 7f4c3c225640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4c340cc150 con 0x7f4c3406f000 2023-11-26T07:18:02.975 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:02.970+0000 7f4c397c0640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f4c3406f3e0 0x7f4c3415cc20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:18:02.975 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:02.970+0000 7f4c397c0640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f4c3406f3e0 0x7f4c3415cc20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:46762/0 (socket says 172.21.15.143:46762) 2023-11-26T07:18:02.975 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:02.970+0000 7f4c397c0640 1 -- 172.21.15.143:0/122321169 learned_addr learned my addr 172.21.15.143:0/122321169 (peer_addr_for_me v2:172.21.15.143:0/0) 2023-11-26T07:18:02.976 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:02.970+0000 7f4c39fc1640 1 --2- 172.21.15.143:0/122321169 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f4c3406f000 0x7f4c340cb850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:18:02.976 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:02.970+0000 7f4c3a7c2640 1 --2- 172.21.15.143:0/122321169 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f4c34099a70 0x7f4c3415fea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:18:02.976 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:02.970+0000 7f4c397c0640 1 -- 172.21.15.143:0/122321169 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f4c3406f000 msgr2=0x7f4c340cb850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:18:02.976 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:02.970+0000 7f4c397c0640 1 --2- 172.21.15.143:0/122321169 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f4c3406f000 0x7f4c340cb850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:18:02.976 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:02.970+0000 7f4c397c0640 1 -- 172.21.15.143:0/122321169 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f4c34099a70 msgr2=0x7f4c3415fea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:18:02.976 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:02.970+0000 7f4c397c0640 1 --2- 172.21.15.143:0/122321169 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f4c34099a70 0x7f4c3415fea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:18:02.976 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:02.970+0000 7f4c397c0640 1 -- 172.21.15.143:0/122321169 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4c300094d0 con 0x7f4c3406f3e0 2023-11-26T07:18:02.977 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:02.970+0000 7f4c397c0640 1 --2- 172.21.15.143:0/122321169 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f4c3406f3e0 0x7f4c3415cc20 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7f4c2400c450 tx=0x7f4c2400c910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:18:02.977 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:02.970+0000 7f4c2a7fc640 1 -- 172.21.15.143:0/122321169 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4c24012830 con 0x7f4c3406f3e0 2023-11-26T07:18:02.977 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:02.970+0000 7f4c2a7fc640 1 -- 172.21.15.143:0/122321169 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4c240129b0 con 0x7f4c3406f3e0 2023-11-26T07:18:02.977 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:02.970+0000 7f4c2a7fc640 1 -- 172.21.15.143:0/122321169 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4c24012d60 con 0x7f4c3406f3e0 2023-11-26T07:18:02.977 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:02.970+0000 7f4c3c225640 1 -- 172.21.15.143:0/122321169 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4c340cc430 con 0x7f4c3406f3e0 2023-11-26T07:18:02.977 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:02.970+0000 7f4c3c225640 1 -- 172.21.15.143:0/122321169 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4c340cc9f0 con 0x7f4c3406f3e0 2023-11-26T07:18:02.978 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:02.974+0000 7f4c3c225640 1 -- 172.21.15.143:0/122321169 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4c340ccf10 con 0x7f4c3406f3e0 2023-11-26T07:18:02.978 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:02.974+0000 7f4c2a7fc640 1 -- 172.21.15.143:0/122321169 <== mon.0 v2:172.21.15.143:3300/0 4 ==== mgrmap(e 5) v1 ==== 85852+0+0 (secure 0 0 0) 0x7f4c2401c460 con 0x7f4c3406f3e0 2023-11-26T07:18:02.978 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:02.974+0000 7f4c2a7fc640 1 --2- 172.21.15.143:0/122321169 >> [v2:172.21.15.143:6832/2150557544,v1:172.21.15.143:6833/2150557544] conn(0x7f4c18066470 0x7f4c18068930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:18:02.978 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:02.974+0000 7f4c2a7fc640 1 -- 172.21.15.143:0/122321169 <== mon.0 v2:172.21.15.143:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f4c24083c60 con 0x7f4c3406f3e0 2023-11-26T07:18:02.979 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:02.974+0000 7f4c39fc1640 1 --2- 172.21.15.143:0/122321169 >> [v2:172.21.15.143:6832/2150557544,v1:172.21.15.143:6833/2150557544] conn(0x7f4c18066470 0x7f4c18068930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:18:02.979 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:02.974+0000 7f4c39fc1640 1 --2- 172.21.15.143:0/122321169 >> [v2:172.21.15.143:6832/2150557544,v1:172.21.15.143:6833/2150557544] conn(0x7f4c18066470 0x7f4c18068930 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7f4c300b2000 tx=0x7f4c30009040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:18:02.982 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:02.978+0000 7f4c2a7fc640 1 -- 172.21.15.143:0/122321169 <== mon.0 v2:172.21.15.143:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172649 (secure 0 0 0) 0x7f4c340ccf10 con 0x7f4c3406f3e0 2023-11-26T07:18:03.127 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:03.122+0000 7f4c3c225640 1 -- 172.21.15.143:0/122321169 --> [v2:172.21.15.143:6832/2150557544,v1:172.21.15.143:6833/2150557544] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f4c340cf490 con 0x7f4c18066470 2023-11-26T07:18:03.134 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:03.130+0000 7f4c2a7fc640 1 -- 172.21.15.143:0/122321169 <== mgr.4100 v2:172.21.15.143:6832/2150557544 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+264593 (secure 0 0 0) 0x7f4c300ae340 con 0x7f4c18066470 2023-11-26T07:18:03.137 INFO:teuthology.orchestra.run.smithi143.stdout: 2023-11-26T07:18:03.140 INFO:teuthology.orchestra.run.smithi143.stderr:dumped all 2023-11-26T07:18:03.141 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:03.134+0000 7f4c3c225640 1 -- 172.21.15.143:0/122321169 >> [v2:172.21.15.143:6832/2150557544,v1:172.21.15.143:6833/2150557544] conn(0x7f4c18066470 msgr2=0x7f4c18068930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:18:03.141 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:03.134+0000 7f4c3c225640 1 --2- 172.21.15.143:0/122321169 >> [v2:172.21.15.143:6832/2150557544,v1:172.21.15.143:6833/2150557544] conn(0x7f4c18066470 0x7f4c18068930 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7f4c300b2000 tx=0x7f4c30009040 comp rx=0 tx=0).stop 2023-11-26T07:18:03.141 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:03.134+0000 7f4c3c225640 1 -- 172.21.15.143:0/122321169 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f4c3406f3e0 msgr2=0x7f4c3415cc20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:18:03.141 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:03.134+0000 7f4c3c225640 1 --2- 172.21.15.143:0/122321169 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f4c3406f3e0 0x7f4c3415cc20 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7f4c2400c450 tx=0x7f4c2400c910 comp rx=0 tx=0).stop 2023-11-26T07:18:03.141 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:03.134+0000 7f4c3c225640 1 -- 172.21.15.143:0/122321169 shutdown_connections 2023-11-26T07:18:03.141 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:03.134+0000 7f4c3c225640 1 --2- 172.21.15.143:0/122321169 >> [v2:172.21.15.143:6832/2150557544,v1:172.21.15.143:6833/2150557544] conn(0x7f4c18066470 0x7f4c18068930 unknown :-1 s=CLOSED pgs=101 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:18:03.141 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:03.134+0000 7f4c3c225640 1 --2- 172.21.15.143:0/122321169 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f4c34099a70 0x7f4c3415fea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:18:03.141 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:03.134+0000 7f4c3c225640 1 --2- 172.21.15.143:0/122321169 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f4c3406f3e0 0x7f4c3415cc20 unknown :-1 s=CLOSED pgs=118 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:18:03.141 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:03.134+0000 7f4c3c225640 1 --2- 172.21.15.143:0/122321169 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f4c3406f000 0x7f4c340cb850 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:18:03.142 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:03.134+0000 7f4c3c225640 1 -- 172.21.15.143:0/122321169 >> 172.21.15.143:0/122321169 conn(0x7f4c340b2d10 msgr2=0x7f4c340b6390 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:18:03.142 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:03.134+0000 7f4c3c225640 1 -- 172.21.15.143:0/122321169 shutdown_connections 2023-11-26T07:18:03.142 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:03.134+0000 7f4c3c225640 1 -- 172.21.15.143:0/122321169 wait complete. 2023-11-26T07:18:03.158 INFO:teuthology.orchestra.run.smithi143.stdout:{"pg_ready":true,"pg_map":{"version":23,"stamp":"2023-11-26T07:18:01.695219+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592778,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":204,"num_write_kb":2430,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":156,"ondisk_log_size":156,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":192,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":219012,"kb_used_data":2824,"kb_used_omap":12,"kb_used_meta":216051,"kb_avail":754755708,"statfs":{"total":773094113280,"available":772869844992,"internally_reserved":0,"allocated":2891776,"data_stored":1752836,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12749,"internal_metadata":221236787},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"8.001398"},"pg_stats":[{"pgid":"3.3f","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-26T07:17:55.764359+0000","last_change":"2023-11-26T07:17:54.530144+0000","last_active":"2023-11-26T07:17:55.764359+0000","last_peered":"2023-11-26T07:17:55.764359+0000","last_clean":"2023-11-26T07:17:55.764359+0000","last_became_active":"2023-11-26T07:17:54.529816+0000","last_became_peered":"2023-11-26T07:17:54.529816+0000","last_unstale":"2023-11-26T07:17:55.764359+0000","last_undegraded":"2023-11-26T07:17:55.764359+0000","last_fullsized":"2023-11-26T07:17:55.764359+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:17:53.483873+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:17:53.483873+0000","last_clean_scrub_stamp":"2023-11-26T07:17:53.483873+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T08:53:05.773632+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-26T07:17:55.516522+0000","last_change":"2023-11-26T07:17:54.523453+0000","last_active":"2023-11-26T07:17:55.516522+0000","last_peered":"2023-11-26T07:17:55.516522+0000","last_clean":"2023-11-26T07:17:55.516522+0000","last_became_active":"2023-11-26T07:17:54.523104+0000","last_became_peered":"2023-11-26T07:17:54.523104+0000","last_unstale":"2023-11-26T07:17:55.516522+0000","last_undegraded":"2023-11-26T07:17:55.516522+0000","last_fullsized":"2023-11-26T07:17:55.516522+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:17:53.483873+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:17:53.483873+0000","last_clean_scrub_stamp":"2023-11-26T07:17:53.483873+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T11:16:25.801280+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-26T07:17:56.068788+0000","last_change":"2023-11-26T07:17:54.514169+0000","last_active":"2023-11-26T07:17:56.068788+0000","last_peered":"2023-11-26T07:17:56.068788+0000","last_clean":"2023-11-26T07:17:56.068788+0000","last_became_active":"2023-11-26T07:17:54.513836+0000","last_became_peered":"2023-11-26T07:17:54.513836+0000","last_unstale":"2023-11-26T07:17:56.068788+0000","last_undegraded":"2023-11-26T07:17:56.068788+0000","last_fullsized":"2023-11-26T07:17:56.068788+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:17:53.483873+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:17:53.483873+0000","last_clean_scrub_stamp":"2023-11-26T07:17:53.483873+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T09:53:29.244002+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-26T07:17:55.538957+0000","last_change":"2023-11-26T07:17:54.528554+0000","last_active":"2023-11-26T07:17:55.538957+0000","last_peered":"2023-11-26T07:17:55.538957+0000","last_clean":"2023-11-26T07:17:55.538957+0000","last_became_active":"2023-11-26T07:17:54.527864+0000","last_became_peered":"2023-11-26T07:17:54.527864+0000","last_unstale":"2023-11-26T07:17:55.538957+0000","last_undegraded":"2023-11-26T07:17:55.538957+0000","last_fullsized":"2023-11-26T07:17:55.538957+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:17:53.483873+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:17:53.483873+0000","last_clean_scrub_stamp":"2023-11-26T07:17:53.483873+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T15:49:39.889795+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-26T07:17:55.763887+0000","last_change":"2023-11-26T07:17:54.527144+0000","last_active":"2023-11-26T07:17:55.763887+0000","last_peered":"2023-11-26T07:17:55.763887+0000","last_clean":"2023-11-26T07:17:55.763887+0000","last_became_active":"2023-11-26T07:17:54.525279+0000","last_became_peered":"2023-11-26T07:17:54.525279+0000","last_unstale":"2023-11-26T07:17:55.763887+0000","last_undegraded":"2023-11-26T07:17:55.763887+0000","last_fullsized":"2023-11-26T07:17:55.763887+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:17:53.483873+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:17:53.483873+0000","last_clean_scrub_stamp":"2023-11-26T07:17:53.483873+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T11:28:10.566680+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-26T07:17:55.538672+0000","last_change":"2023-11-26T07:17:54.522607+0000","last_active":"2023-11-26T07:17:55.538672+0000","last_peered":"2023-11-26T07:17:55.538672+0000","last_clean":"2023-11-26T07:17:55.538672+0000","last_became_active":"2023-11-26T07:17:54.522297+0000","last_became_peered":"2023-11-26T07:17:54.522297+0000","last_unstale":"2023-11-26T07:17:55.538672+0000","last_undegraded":"2023-11-26T07:17:55.538672+0000","last_fullsized":"2023-11-26T07:17:55.538672+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:17:53.483873+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:17:53.483873+0000","last_clean_scrub_stamp":"2023-11-26T07:17:53.483873+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T13:18:15.521322+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-26T07:17:55.763706+0000","last_change":"2023-11-26T07:17:54.524898+0000","last_active":"2023-11-26T07:17:55.763706+0000","last_peered":"2023-11-26T07:17:55.763706+0000","last_clean":"2023-11-26T07:17:55.763706+0000","last_became_active":"2023-11-26T07:17:54.524611+0000","last_became_peered":"2023-11-26T07:17:54.524611+0000","last_unstale":"2023-11-26T07:17:55.763706+0000","last_undegraded":"2023-11-26T07:17:55.763706+0000","last_fullsized":"2023-11-26T07:17:55.763706+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:17:53.483873+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:17:53.483873+0000","last_clean_scrub_stamp":"2023-11-26T07:17:53.483873+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T17:14:14.930365+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-26T07:17:56.640574+0000","last_change":"2023-11-26T07:17:54.530342+0000","last_active":"2023-11-26T07:17:56.640574+0000","last_peered":"2023-11-26T07:17:56.640574+0000","last_clean":"2023-11-26T07:17:56.640574+0000","last_became_active":"2023-11-26T07:17:54.529960+0000","last_became_peered":"2023-11-26T07:17:54.529960+0000","last_unstale":"2023-11-26T07:17:56.640574+0000","last_undegraded":"2023-11-26T07:17:56.640574+0000","last_fullsized":"2023-11-26T07:17:56.640574+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:17:53.483873+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:17:53.483873+0000","last_clean_scrub_stamp":"2023-11-26T07:17:53.483873+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T08:42:29.375032+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-26T07:17:55.764389+0000","last_change":"2023-11-26T07:17:54.539062+0000","last_active":"2023-11-26T07:17:55.764389+0000","last_peered":"2023-11-26T07:17:55.764389+0000","last_clean":"2023-11-26T07:17:55.764389+0000","last_became_active":"2023-11-26T07:17:54.538119+0000","last_became_peered":"2023-11-26T07:17:54.538119+0000","last_unstale":"2023-11-26T07:17:55.764389+0000","last_undegraded":"2023-11-26T07:17:55.764389+0000","last_fullsized":"2023-11-26T07:17:55.764389+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:17:53.483873+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:17:53.483873+0000","last_clean_scrub_stamp":"2023-11-26T07:17:53.483873+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T13:57:07.286285+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-26T07:17:56.640341+0000","last_change":"2023-11-26T07:17:54.524564+0000","last_active":"2023-11-26T07:17:56.640341+0000","last_peered":"2023-11-26T07:17:56.640341+0000","last_clean":"2023-11-26T07:17:56.640341+0000","last_became_active":"2023-11-26T07:17:54.524255+0000","last_became_peered":"2023-11-26T07:17:54.524255+0000","last_unstale":"2023-11-26T07:17:56.640341+0000","last_undegraded":"2023-11-26T07:17:56.640341+0000","last_fullsized":"2023-11-26T07:17:56.640341+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:17:53.483873+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:17:53.483873+0000","last_clean_scrub_stamp":"2023-11-26T07:17:53.483873+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T11:16:25.801280+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-26T07:17:56.639652+0000","last_change":"2023-11-26T07:17:54.535310+0000","last_active":"2023-11-26T07:17:56.639652+0000","last_peered":"2023-11-26T07:17:56.639652+0000","last_clean":"2023-11-26T07:17:56.639652+0000","last_became_active":"2023-11-26T07:17:54.534986+0000","last_became_peered":"2023-11-26T07:17:54.534986+0000","last_unstale":"2023-11-26T07:17:56.639652+0000","last_undegraded":"2023-11-26T07:17:56.639652+0000","last_fullsized":"2023-11-26T07:17:56.639652+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:17:53.483873+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:17:53.483873+0000","last_clean_scrub_stamp":"2023-11-26T07:17:53.483873+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T15:18:17.309782+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-26T07:17:56.640026+0000","last_change":"2023-11-26T07:17:54.512408+0000","last_active":"2023-11-26T07:17:56.640026+0000","last_peered":"2023-11-26T07:17:56.640026+0000","last_clean":"2023-11-26T07:17:56.640026+0000","last_became_active":"2023-11-26T07:17:54.512005+0000","last_became_peered":"2023-11-26T07:17:54.512005+0000","last_unstale":"2023-11-26T07:17:56.640026+0000","last_undegraded":"2023-11-26T07:17:56.640026+0000","last_fullsized":"2023-11-26T07:17:56.640026+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:17:53.483873+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:17:53.483873+0000","last_clean_scrub_stamp":"2023-11-26T07:17:53.483873+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T09:07:53.947975+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-26T07:17:56.640756+0000","last_change":"2023-11-26T07:17:54.514825+0000","last_active":"2023-11-26T07:17:56.640756+0000","last_peered":"2023-11-26T07:17:56.640756+0000","last_clean":"2023-11-26T07:17:56.640756+0000","last_became_active":"2023-11-26T07:17:54.514502+0000","last_became_peered":"2023-11-26T07:17:54.514502+0000","last_unstale":"2023-11-26T07:17:56.640756+0000","last_undegraded":"2023-11-26T07:17:56.640756+0000","last_fullsized":"2023-11-26T07:17:56.640756+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:17:53.483873+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:17:53.483873+0000","last_clean_scrub_stamp":"2023-11-26T07:17:53.483873+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T07:50:04.472214+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-26T07:17:56.068691+0000","last_change":"2023-11-26T07:17:54.521737+0000","last_active":"2023-11-26T07:17:56.068691+0000","last_peered":"2023-11-26T07:17:56.068691+0000","last_clean":"2023-11-26T07:17:56.068691+0000","last_became_active":"2023-11-26T07:17:54.521387+0000","last_became_peered":"2023-11-26T07:17:54.521387+0000","last_unstale":"2023-11-26T07:17:56.068691+0000","last_undegraded":"2023-11-26T07:17:56.068691+0000","last_fullsized":"2023-11-26T07:17:56.068691+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:17:53.483873+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:17:53.483873+0000","last_clean_scrub_stamp":"2023-11-26T07:17:53.483873+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T18:33:32.653907+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-26T07:17:55.516698+0000","last_change":"2023-11-26T07:17:54.533852+0000","last_active":"2023-11-26T07:17:55.516698+0000","last_peered":"2023-11-26T07:17:55.516698+0000","last_clean":"2023-11-26T07:17:55.516698+0000","last_became_active":"2023-11-26T07:17:54.533532+0000","last_became_peered":"2023-11-26T07:17:54.533532+0000","last_unstale":"2023-11-26T07:17:55.516698+0000","last_undegraded":"2023-11-26T07:17:55.516698+0000","last_fullsized":"2023-11-26T07:17:55.516698+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:17:53.483873+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:17:53.483873+0000","last_clean_scrub_stamp":"2023-11-26T07:17:53.483873+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T14:45:51.557148+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-26T07:17:55.538556+0000","last_change":"2023-11-26T07:17:54.523130+0000","last_active":"2023-11-26T07:17:55.538556+0000","last_peered":"2023-11-26T07:17:55.538556+0000","last_clean":"2023-11-26T07:17:55.538556+0000","last_became_active":"2023-11-26T07:17:54.522879+0000","last_became_peered":"2023-11-26T07:17:54.522879+0000","last_unstale":"2023-11-26T07:17:55.538556+0000","last_undegraded":"2023-11-26T07:17:55.538556+0000","last_fullsized":"2023-11-26T07:17:55.538556+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:17:53.483873+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:17:53.483873+0000","last_clean_scrub_stamp":"2023-11-26T07:17:53.483873+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T11:16:25.801280+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-26T07:17:55.539151+0000","last_change":"2023-11-26T07:17:54.534352+0000","last_active":"2023-11-26T07:17:55.539151+0000","last_peered":"2023-11-26T07:17:55.539151+0000","last_clean":"2023-11-26T07:17:55.539151+0000","last_became_active":"2023-11-26T07:17:54.534004+0000","last_became_peered":"2023-11-26T07:17:54.534004+0000","last_unstale":"2023-11-26T07:17:55.539151+0000","last_undegraded":"2023-11-26T07:17:55.539151+0000","last_fullsized":"2023-11-26T07:17:55.539151+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:17:53.483873+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:17:53.483873+0000","last_clean_scrub_stamp":"2023-11-26T07:17:53.483873+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T18:27:00.305240+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-26T07:17:55.514964+0000","last_change":"2023-11-26T07:17:54.515085+0000","last_active":"2023-11-26T07:17:55.514964+0000","last_peered":"2023-11-26T07:17:55.514964+0000","last_clean":"2023-11-26T07:17:55.514964+0000","last_became_active":"2023-11-26T07:17:54.514782+0000","last_became_peered":"2023-11-26T07:17:54.514782+0000","last_unstale":"2023-11-26T07:17:55.514964+0000","last_undegraded":"2023-11-26T07:17:55.514964+0000","last_fullsized":"2023-11-26T07:17:55.514964+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:17:53.483873+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:17:53.483873+0000","last_clean_scrub_stamp":"2023-11-26T07:17:53.483873+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T13:45:34.765962+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-26T07:17:55.764203+0000","last_change":"2023-11-26T07:17:54.529137+0000","last_active":"2023-11-26T07:17:55.764203+0000","last_peered":"2023-11-26T07:17:55.764203+0000","last_clean":"2023-11-26T07:17:55.764203+0000","last_became_active":"2023-11-26T07:17:54.528496+0000","last_became_peered":"2023-11-26T07:17:54.528496+0000","last_unstale":"2023-11-26T07:17:55.764203+0000","last_undegraded":"2023-11-26T07:17:55.764203+0000","last_fullsized":"2023-11-26T07:17:55.764203+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:17:53.483873+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:17:53.483873+0000","last_clean_scrub_stamp":"2023-11-26T07:17:53.483873+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T17:27:24.347347+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-26T07:17:56.639612+0000","last_change":"2023-11-26T07:17:54.521609+0000","last_active":"2023-11-26T07:17:56.639612+0000","last_peered":"2023-11-26T07:17:56.639612+0000","last_clean":"2023-11-26T07:17:56.639612+0000","last_became_active":"2023-11-26T07:17:54.521298+0000","last_became_peered":"2023-11-26T07:17:54.521298+0000","last_unstale":"2023-11-26T07:17:56.639612+0000","last_undegraded":"2023-11-26T07:17:56.639612+0000","last_fullsized":"2023-11-26T07:17:56.639612+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:17:53.483873+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:17:53.483873+0000","last_clean_scrub_stamp":"2023-11-26T07:17:53.483873+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T08:42:29.375032+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-26T07:17:56.068908+0000","last_change":"2023-11-26T07:17:54.526550+0000","last_active":"2023-11-26T07:17:56.068908+0000","last_peered":"2023-11-26T07:17:56.068908+0000","last_clean":"2023-11-26T07:17:56.068908+0000","last_became_active":"2023-11-26T07:17:54.525639+0000","last_became_peered":"2023-11-26T07:17:54.525639+0000","last_unstale":"2023-11-26T07:17:56.068908+0000","last_undegraded":"2023-11-26T07:17:56.068908+0000","last_fullsized":"2023-11-26T07:17:56.068908+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:17:53.483873+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:17:53.483873+0000","last_clean_scrub_stamp":"2023-11-26T07:17:53.483873+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T07:50:04.472214+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-26T07:17:55.538640+0000","last_change":"2023-11-26T07:17:54.534209+0000","last_active":"2023-11-26T07:17:55.538640+0000","last_peered":"2023-11-26T07:17:55.538640+0000","last_clean":"2023-11-26T07:17:55.538640+0000","last_became_active":"2023-11-26T07:17:54.533932+0000","last_became_peered":"2023-11-26T07:17:54.533932+0000","last_unstale":"2023-11-26T07:17:55.538640+0000","last_undegraded":"2023-11-26T07:17:55.538640+0000","last_fullsized":"2023-11-26T07:17:55.538640+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:17:53.483873+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:17:53.483873+0000","last_clean_scrub_stamp":"2023-11-26T07:17:53.483873+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T18:02:15.969376+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-26T07:17:56.640577+0000","last_change":"2023-11-26T07:17:54.526358+0000","last_active":"2023-11-26T07:17:56.640577+0000","last_peered":"2023-11-26T07:17:56.640577+0000","last_clean":"2023-11-26T07:17:56.640577+0000","last_became_active":"2023-11-26T07:17:54.525756+0000","last_became_peered":"2023-11-26T07:17:54.525756+0000","last_unstale":"2023-11-26T07:17:56.640577+0000","last_undegraded":"2023-11-26T07:17:56.640577+0000","last_fullsized":"2023-11-26T07:17:56.640577+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:17:53.483873+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:17:53.483873+0000","last_clean_scrub_stamp":"2023-11-26T07:17:53.483873+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T15:18:17.309782+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-26T07:17:55.538626+0000","last_change":"2023-11-26T07:17:54.526328+0000","last_active":"2023-11-26T07:17:55.538626+0000","last_peered":"2023-11-26T07:17:55.538626+0000","last_clean":"2023-11-26T07:17:55.538626+0000","last_became_active":"2023-11-26T07:17:54.526026+0000","last_became_peered":"2023-11-26T07:17:54.526026+0000","last_unstale":"2023-11-26T07:17:55.538626+0000","last_undegraded":"2023-11-26T07:17:55.538626+0000","last_fullsized":"2023-11-26T07:17:55.538626+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:17:53.483873+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:17:53.483873+0000","last_clean_scrub_stamp":"2023-11-26T07:17:53.483873+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T14:45:51.557148+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-26T07:17:56.641174+0000","last_change":"2023-11-26T07:17:54.521949+0000","last_active":"2023-11-26T07:17:56.641174+0000","last_peered":"2023-11-26T07:17:56.641174+0000","last_clean":"2023-11-26T07:17:56.641174+0000","last_became_active":"2023-11-26T07:17:54.521649+0000","last_became_peered":"2023-11-26T07:17:54.521649+0000","last_unstale":"2023-11-26T07:17:56.641174+0000","last_undegraded":"2023-11-26T07:17:56.641174+0000","last_fullsized":"2023-11-26T07:17:56.641174+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:17:53.483873+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:17:53.483873+0000","last_clean_scrub_stamp":"2023-11-26T07:17:53.483873+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T17:14:14.930365+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-26T07:17:56.640304+0000","last_change":"2023-11-26T07:17:54.526758+0000","last_active":"2023-11-26T07:17:56.640304+0000","last_peered":"2023-11-26T07:17:56.640304+0000","last_clean":"2023-11-26T07:17:56.640304+0000","last_became_active":"2023-11-26T07:17:54.525563+0000","last_became_peered":"2023-11-26T07:17:54.525563+0000","last_unstale":"2023-11-26T07:17:56.640304+0000","last_undegraded":"2023-11-26T07:17:56.640304+0000","last_fullsized":"2023-11-26T07:17:56.640304+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:17:53.483873+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:17:53.483873+0000","last_clean_scrub_stamp":"2023-11-26T07:17:53.483873+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T07:50:04.472214+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-26T07:17:55.516791+0000","last_change":"2023-11-26T07:17:54.520560+0000","last_active":"2023-11-26T07:17:55.516791+0000","last_peered":"2023-11-26T07:17:55.516791+0000","last_clean":"2023-11-26T07:17:55.516791+0000","last_became_active":"2023-11-26T07:17:54.520377+0000","last_became_peered":"2023-11-26T07:17:54.520377+0000","last_unstale":"2023-11-26T07:17:55.516791+0000","last_undegraded":"2023-11-26T07:17:55.516791+0000","last_fullsized":"2023-11-26T07:17:55.516791+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:17:53.483873+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:17:53.483873+0000","last_clean_scrub_stamp":"2023-11-26T07:17:53.483873+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T18:02:15.969376+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-26T07:17:56.640998+0000","last_change":"2023-11-26T07:17:54.527041+0000","last_active":"2023-11-26T07:17:56.640998+0000","last_peered":"2023-11-26T07:17:56.640998+0000","last_clean":"2023-11-26T07:17:56.640998+0000","last_became_active":"2023-11-26T07:17:54.526391+0000","last_became_peered":"2023-11-26T07:17:54.526391+0000","last_unstale":"2023-11-26T07:17:56.640998+0000","last_undegraded":"2023-11-26T07:17:56.640998+0000","last_fullsized":"2023-11-26T07:17:56.640998+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:17:53.483873+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:17:53.483873+0000","last_clean_scrub_stamp":"2023-11-26T07:17:53.483873+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T18:02:15.969376+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-26T07:17:56.068828+0000","last_change":"2023-11-26T07:17:54.512264+0000","last_active":"2023-11-26T07:17:56.068828+0000","last_peered":"2023-11-26T07:17:56.068828+0000","last_clean":"2023-11-26T07:17:56.068828+0000","last_became_active":"2023-11-26T07:17:54.511963+0000","last_became_peered":"2023-11-26T07:17:54.511963+0000","last_unstale":"2023-11-26T07:17:56.068828+0000","last_undegraded":"2023-11-26T07:17:56.068828+0000","last_fullsized":"2023-11-26T07:17:56.068828+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:17:53.483873+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:17:53.483873+0000","last_clean_scrub_stamp":"2023-11-26T07:17:53.483873+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T08:42:29.375032+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-26T07:17:55.538793+0000","last_change":"2023-11-26T07:17:54.528162+0000","last_active":"2023-11-26T07:17:55.538793+0000","last_peered":"2023-11-26T07:17:55.538793+0000","last_clean":"2023-11-26T07:17:55.538793+0000","last_became_active":"2023-11-26T07:17:54.527657+0000","last_became_peered":"2023-11-26T07:17:54.527657+0000","last_unstale":"2023-11-26T07:17:55.538793+0000","last_undegraded":"2023-11-26T07:17:55.538793+0000","last_fullsized":"2023-11-26T07:17:55.538793+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:17:53.483873+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:17:53.483873+0000","last_clean_scrub_stamp":"2023-11-26T07:17:53.483873+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T14:49:27.883374+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-26T07:17:55.514728+0000","last_change":"2023-11-26T07:17:54.519107+0000","last_active":"2023-11-26T07:17:55.514728+0000","last_peered":"2023-11-26T07:17:55.514728+0000","last_clean":"2023-11-26T07:17:55.514728+0000","last_became_active":"2023-11-26T07:17:54.518822+0000","last_became_peered":"2023-11-26T07:17:54.518822+0000","last_unstale":"2023-11-26T07:17:55.514728+0000","last_undegraded":"2023-11-26T07:17:55.514728+0000","last_fullsized":"2023-11-26T07:17:55.514728+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:17:53.483873+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:17:53.483873+0000","last_clean_scrub_stamp":"2023-11-26T07:17:53.483873+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T09:07:53.947975+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-26T07:17:56.639246+0000","last_change":"2023-11-26T07:17:54.513368+0000","last_active":"2023-11-26T07:17:56.639246+0000","last_peered":"2023-11-26T07:17:56.639246+0000","last_clean":"2023-11-26T07:17:56.639246+0000","last_became_active":"2023-11-26T07:17:54.511290+0000","last_became_peered":"2023-11-26T07:17:54.511290+0000","last_unstale":"2023-11-26T07:17:56.639246+0000","last_undegraded":"2023-11-26T07:17:56.639246+0000","last_fullsized":"2023-11-26T07:17:56.639246+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:17:53.483873+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:17:53.483873+0000","last_clean_scrub_stamp":"2023-11-26T07:17:53.483873+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T18:33:32.653907+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-26T07:17:56.639579+0000","last_change":"2023-11-26T07:17:54.513190+0000","last_active":"2023-11-26T07:17:56.639579+0000","last_peered":"2023-11-26T07:17:56.639579+0000","last_clean":"2023-11-26T07:17:56.639579+0000","last_became_active":"2023-11-26T07:17:54.512872+0000","last_became_peered":"2023-11-26T07:17:54.512872+0000","last_unstale":"2023-11-26T07:17:56.639579+0000","last_undegraded":"2023-11-26T07:17:56.639579+0000","last_fullsized":"2023-11-26T07:17:56.639579+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:17:53.483873+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:17:53.483873+0000","last_clean_scrub_stamp":"2023-11-26T07:17:53.483873+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T12:32:48.249974+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1b","version":"19'1","reported_seq":13,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-26T07:17:56.640973+0000","last_change":"2023-11-26T07:17:53.525183+0000","last_active":"2023-11-26T07:17:56.640973+0000","last_peered":"2023-11-26T07:17:56.640973+0000","last_clean":"2023-11-26T07:17:56.640973+0000","last_became_active":"2023-11-26T07:17:53.524904+0000","last_became_peered":"2023-11-26T07:17:53.524904+0000","last_unstale":"2023-11-26T07:17:56.640973+0000","last_undegraded":"2023-11-26T07:17:56.640973+0000","last_fullsized":"2023-11-26T07:17:56.640973+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:17:52.477276+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:17:52.477276+0000","last_clean_scrub_stamp":"2023-11-26T07:17:52.477276+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T13:59:52.977195+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-26T07:17:56.068970+0000","last_change":"2023-11-26T07:17:54.522974+0000","last_active":"2023-11-26T07:17:56.068970+0000","last_peered":"2023-11-26T07:17:56.068970+0000","last_clean":"2023-11-26T07:17:56.068970+0000","last_became_active":"2023-11-26T07:17:54.521283+0000","last_became_peered":"2023-11-26T07:17:54.521283+0000","last_unstale":"2023-11-26T07:17:56.068970+0000","last_undegraded":"2023-11-26T07:17:56.068970+0000","last_fullsized":"2023-11-26T07:17:56.068970+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:17:53.483873+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:17:53.483873+0000","last_clean_scrub_stamp":"2023-11-26T07:17:53.483873+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T11:16:25.801280+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1a","version":"19'1","reported_seq":13,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-26T07:17:56.639442+0000","last_change":"2023-11-26T07:17:53.516099+0000","last_active":"2023-11-26T07:17:56.639442+0000","last_peered":"2023-11-26T07:17:56.639442+0000","last_clean":"2023-11-26T07:17:56.639442+0000","last_became_active":"2023-11-26T07:17:53.515687+0000","last_became_peered":"2023-11-26T07:17:53.515687+0000","last_unstale":"2023-11-26T07:17:56.639442+0000","last_undegraded":"2023-11-26T07:17:56.639442+0000","last_fullsized":"2023-11-26T07:17:56.639442+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:17:52.477276+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:17:52.477276+0000","last_clean_scrub_stamp":"2023-11-26T07:17:52.477276+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T09:07:52.941378+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.19","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-26T07:17:56.640738+0000","last_change":"2023-11-26T07:17:53.523159+0000","last_active":"2023-11-26T07:17:56.640738+0000","last_peered":"2023-11-26T07:17:56.640738+0000","last_clean":"2023-11-26T07:17:56.640738+0000","last_became_active":"2023-11-26T07:17:53.522824+0000","last_became_peered":"2023-11-26T07:17:53.522824+0000","last_unstale":"2023-11-26T07:17:56.640738+0000","last_undegraded":"2023-11-26T07:17:56.640738+0000","last_fullsized":"2023-11-26T07:17:56.640738+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:17:52.477276+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:17:52.477276+0000","last_clean_scrub_stamp":"2023-11-26T07:17:52.477276+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T13:45:33.759365+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-26T07:17:56.641065+0000","last_change":"2023-11-26T07:17:54.515953+0000","last_active":"2023-11-26T07:17:56.641065+0000","last_peered":"2023-11-26T07:17:56.641065+0000","last_clean":"2023-11-26T07:17:56.641065+0000","last_became_active":"2023-11-26T07:17:54.515662+0000","last_became_peered":"2023-11-26T07:17:54.515662+0000","last_unstale":"2023-11-26T07:17:56.641065+0000","last_undegraded":"2023-11-26T07:17:56.641065+0000","last_fullsized":"2023-11-26T07:17:56.641065+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:17:53.483873+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:17:53.483873+0000","last_clean_scrub_stamp":"2023-11-26T07:17:53.483873+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T10:18:52.982852+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.18","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-26T07:17:55.538849+0000","last_change":"2023-11-26T07:17:53.516308+0000","last_active":"2023-11-26T07:17:55.538849+0000","last_peered":"2023-11-26T07:17:55.538849+0000","last_clean":"2023-11-26T07:17:55.538849+0000","last_became_active":"2023-11-26T07:17:53.516027+0000","last_became_peered":"2023-11-26T07:17:53.516027+0000","last_unstale":"2023-11-26T07:17:55.538849+0000","last_undegraded":"2023-11-26T07:17:55.538849+0000","last_fullsized":"2023-11-26T07:17:55.538849+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:17:52.477276+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:17:52.477276+0000","last_clean_scrub_stamp":"2023-11-26T07:17:52.477276+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T08:42:28.368435+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-26T07:17:55.516385+0000","last_change":"2023-11-26T07:17:54.521790+0000","last_active":"2023-11-26T07:17:55.516385+0000","last_peered":"2023-11-26T07:17:55.516385+0000","last_clean":"2023-11-26T07:17:55.516385+0000","last_became_active":"2023-11-26T07:17:54.521475+0000","last_became_peered":"2023-11-26T07:17:54.521475+0000","last_unstale":"2023-11-26T07:17:55.516385+0000","last_undegraded":"2023-11-26T07:17:55.516385+0000","last_fullsized":"2023-11-26T07:17:55.516385+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:17:53.483873+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:17:53.483873+0000","last_clean_scrub_stamp":"2023-11-26T07:17:53.483873+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T15:18:17.309782+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.17","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-26T07:17:56.640306+0000","last_change":"2023-11-26T07:17:53.514460+0000","last_active":"2023-11-26T07:17:56.640306+0000","last_peered":"2023-11-26T07:17:56.640306+0000","last_clean":"2023-11-26T07:17:56.640306+0000","last_became_active":"2023-11-26T07:17:53.514170+0000","last_became_peered":"2023-11-26T07:17:53.514170+0000","last_unstale":"2023-11-26T07:17:56.640306+0000","last_undegraded":"2023-11-26T07:17:56.640306+0000","last_fullsized":"2023-11-26T07:17:56.640306+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:17:52.477276+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:17:52.477276+0000","last_clean_scrub_stamp":"2023-11-26T07:17:52.477276+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T15:47:35.416235+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-26T07:17:55.538995+0000","last_change":"2023-11-26T07:17:54.533776+0000","last_active":"2023-11-26T07:17:55.538995+0000","last_peered":"2023-11-26T07:17:55.538995+0000","last_clean":"2023-11-26T07:17:55.538995+0000","last_became_active":"2023-11-26T07:17:54.533279+0000","last_became_peered":"2023-11-26T07:17:54.533279+0000","last_unstale":"2023-11-26T07:17:55.538995+0000","last_undegraded":"2023-11-26T07:17:55.538995+0000","last_fullsized":"2023-11-26T07:17:55.538995+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:17:53.483873+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:17:53.483873+0000","last_clean_scrub_stamp":"2023-11-26T07:17:53.483873+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T15:38:12.078699+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.16","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-26T07:17:55.539107+0000","last_change":"2023-11-26T07:17:53.520128+0000","last_active":"2023-11-26T07:17:55.539107+0000","last_peered":"2023-11-26T07:17:55.539107+0000","last_clean":"2023-11-26T07:17:55.539107+0000","last_became_active":"2023-11-26T07:17:53.519399+0000","last_became_peered":"2023-11-26T07:17:53.519399+0000","last_unstale":"2023-11-26T07:17:55.539107+0000","last_undegraded":"2023-11-26T07:17:55.539107+0000","last_fullsized":"2023-11-26T07:17:55.539107+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:17:52.477276+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:17:52.477276+0000","last_clean_scrub_stamp":"2023-11-26T07:17:52.477276+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T15:18:16.303185+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-26T07:17:56.068974+0000","last_change":"2023-11-26T07:17:54.528621+0000","last_active":"2023-11-26T07:17:56.068974+0000","last_peered":"2023-11-26T07:17:56.068974+0000","last_clean":"2023-11-26T07:17:56.068974+0000","last_became_active":"2023-11-26T07:17:54.528333+0000","last_became_peered":"2023-11-26T07:17:54.528333+0000","last_unstale":"2023-11-26T07:17:56.068974+0000","last_undegraded":"2023-11-26T07:17:56.068974+0000","last_fullsized":"2023-11-26T07:17:56.068974+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:17:53.483873+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:17:53.483873+0000","last_clean_scrub_stamp":"2023-11-26T07:17:53.483873+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T13:18:15.521322+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.15","version":"19'3","reported_seq":15,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-26T07:17:55.516929+0000","last_change":"2023-11-26T07:17:53.515987+0000","last_active":"2023-11-26T07:17:55.516929+0000","last_peered":"2023-11-26T07:17:55.516929+0000","last_clean":"2023-11-26T07:17:55.516929+0000","last_became_active":"2023-11-26T07:17:53.515360+0000","last_became_peered":"2023-11-26T07:17:53.515360+0000","last_unstale":"2023-11-26T07:17:55.516929+0000","last_undegraded":"2023-11-26T07:17:55.516929+0000","last_fullsized":"2023-11-26T07:17:55.516929+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:17:52.477276+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:17:52.477276+0000","last_clean_scrub_stamp":"2023-11-26T07:17:52.477276+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T19:07:41.542569+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-26T07:17:56.640029+0000","last_change":"2023-11-26T07:17:54.530537+0000","last_active":"2023-11-26T07:17:56.640029+0000","last_peered":"2023-11-26T07:17:56.640029+0000","last_clean":"2023-11-26T07:17:56.640029+0000","last_became_active":"2023-11-26T07:17:54.530074+0000","last_became_peered":"2023-11-26T07:17:54.530074+0000","last_unstale":"2023-11-26T07:17:56.640029+0000","last_undegraded":"2023-11-26T07:17:56.640029+0000","last_fullsized":"2023-11-26T07:17:56.640029+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:17:53.483873+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:17:53.483873+0000","last_clean_scrub_stamp":"2023-11-26T07:17:53.483873+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T13:45:34.765962+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.14","version":"19'6","reported_seq":18,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-26T07:17:56.639634+0000","last_change":"2023-11-26T07:17:53.522850+0000","last_active":"2023-11-26T07:17:56.639634+0000","last_peered":"2023-11-26T07:17:56.639634+0000","last_clean":"2023-11-26T07:17:56.639634+0000","last_became_active":"2023-11-26T07:17:53.522377+0000","last_became_peered":"2023-11-26T07:17:53.522377+0000","last_unstale":"2023-11-26T07:17:56.639634+0000","last_undegraded":"2023-11-26T07:17:56.639634+0000","last_fullsized":"2023-11-26T07:17:56.639634+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:17:52.477276+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:17:52.477276+0000","last_clean_scrub_stamp":"2023-11-26T07:17:52.477276+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T12:51:16.332936+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-26T07:17:55.763972+0000","last_change":"2023-11-26T07:17:54.529408+0000","last_active":"2023-11-26T07:17:55.763972+0000","last_peered":"2023-11-26T07:17:55.763972+0000","last_clean":"2023-11-26T07:17:55.763972+0000","last_became_active":"2023-11-26T07:17:54.528825+0000","last_became_peered":"2023-11-26T07:17:54.528825+0000","last_unstale":"2023-11-26T07:17:55.763972+0000","last_undegraded":"2023-11-26T07:17:55.763972+0000","last_fullsized":"2023-11-26T07:17:55.763972+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:17:53.483873+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:17:53.483873+0000","last_clean_scrub_stamp":"2023-11-26T07:17:53.483873+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T12:21:10.834625+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.13","version":"19'2","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-26T07:17:56.068998+0000","last_change":"2023-11-26T07:17:53.517650+0000","last_active":"2023-11-26T07:17:56.068998+0000","last_peered":"2023-11-26T07:17:56.068998+0000","last_clean":"2023-11-26T07:17:56.068998+0000","last_became_active":"2023-11-26T07:17:53.517349+0000","last_became_peered":"2023-11-26T07:17:53.517349+0000","last_unstale":"2023-11-26T07:17:56.068998+0000","last_undegraded":"2023-11-26T07:17:56.068998+0000","last_fullsized":"2023-11-26T07:17:56.068998+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:17:52.477276+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:17:52.477276+0000","last_clean_scrub_stamp":"2023-11-26T07:17:52.477276+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T12:51:16.332936+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-26T07:17:56.068804+0000","last_change":"2023-11-26T07:17:54.531742+0000","last_active":"2023-11-26T07:17:56.068804+0000","last_peered":"2023-11-26T07:17:56.068804+0000","last_clean":"2023-11-26T07:17:56.068804+0000","last_became_active":"2023-11-26T07:17:54.531136+0000","last_became_peered":"2023-11-26T07:17:54.531136+0000","last_unstale":"2023-11-26T07:17:56.068804+0000","last_undegraded":"2023-11-26T07:17:56.068804+0000","last_fullsized":"2023-11-26T07:17:56.068804+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:17:53.483873+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:17:53.483873+0000","last_clean_scrub_stamp":"2023-11-26T07:17:53.483873+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T16:07:46.594040+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.12","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-26T07:17:55.538769+0000","last_change":"2023-11-26T07:17:53.522311+0000","last_active":"2023-11-26T07:17:55.538769+0000","last_peered":"2023-11-26T07:17:55.538769+0000","last_clean":"2023-11-26T07:17:55.538769+0000","last_became_active":"2023-11-26T07:17:53.522017+0000","last_became_peered":"2023-11-26T07:17:53.522017+0000","last_unstale":"2023-11-26T07:17:55.538769+0000","last_undegraded":"2023-11-26T07:17:55.538769+0000","last_fullsized":"2023-11-26T07:17:55.538769+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:17:52.477276+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:17:52.477276+0000","last_clean_scrub_stamp":"2023-11-26T07:17:52.477276+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T09:53:28.237406+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-26T07:17:55.763876+0000","last_change":"2023-11-26T07:17:54.531973+0000","last_active":"2023-11-26T07:17:55.763876+0000","last_peered":"2023-11-26T07:17:55.763876+0000","last_clean":"2023-11-26T07:17:55.763876+0000","last_became_active":"2023-11-26T07:17:54.531710+0000","last_became_peered":"2023-11-26T07:17:54.531710+0000","last_unstale":"2023-11-26T07:17:55.763876+0000","last_undegraded":"2023-11-26T07:17:55.763876+0000","last_fullsized":"2023-11-26T07:17:55.763876+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:17:53.483873+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:17:53.483873+0000","last_clean_scrub_stamp":"2023-11-26T07:17:53.483873+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T15:38:12.078699+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.11","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-26T07:17:56.639192+0000","last_change":"2023-11-26T07:17:53.519520+0000","last_active":"2023-11-26T07:17:56.639192+0000","last_peered":"2023-11-26T07:17:56.639192+0000","last_clean":"2023-11-26T07:17:56.639192+0000","last_became_active":"2023-11-26T07:17:53.516001+0000","last_became_peered":"2023-11-26T07:17:53.516001+0000","last_unstale":"2023-11-26T07:17:56.639192+0000","last_undegraded":"2023-11-26T07:17:56.639192+0000","last_fullsized":"2023-11-26T07:17:56.639192+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:17:52.477276+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:17:52.477276+0000","last_clean_scrub_stamp":"2023-11-26T07:17:52.477276+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T19:07:41.542569+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-26T07:17:56.639395+0000","last_change":"2023-11-26T07:17:54.511603+0000","last_active":"2023-11-26T07:17:56.639395+0000","last_peered":"2023-11-26T07:17:56.639395+0000","last_clean":"2023-11-26T07:17:56.639395+0000","last_became_active":"2023-11-26T07:17:54.510939+0000","last_became_peered":"2023-11-26T07:17:54.510939+0000","last_unstale":"2023-11-26T07:17:56.639395+0000","last_undegraded":"2023-11-26T07:17:56.639395+0000","last_fullsized":"2023-11-26T07:17:56.639395+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:17:53.483873+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:17:53.483873+0000","last_clean_scrub_stamp":"2023-11-26T07:17:53.483873+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T09:53:29.244002+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.10","version":"19'8","reported_seq":20,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-26T07:17:55.514684+0000","last_change":"2023-11-26T07:17:53.506514+0000","last_active":"2023-11-26T07:17:55.514684+0000","last_peered":"2023-11-26T07:17:55.514684+0000","last_clean":"2023-11-26T07:17:55.514684+0000","last_became_active":"2023-11-26T07:17:53.506081+0000","last_became_peered":"2023-11-26T07:17:53.506081+0000","last_unstale":"2023-11-26T07:17:55.514684+0000","last_undegraded":"2023-11-26T07:17:55.514684+0000","last_fullsized":"2023-11-26T07:17:55.514684+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:17:52.477276+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:17:52.477276+0000","last_clean_scrub_stamp":"2023-11-26T07:17:52.477276+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T09:43:57.945669+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-26T07:17:55.763817+0000","last_change":"2023-11-26T07:17:54.532538+0000","last_active":"2023-11-26T07:17:55.763817+0000","last_peered":"2023-11-26T07:17:55.763817+0000","last_clean":"2023-11-26T07:17:55.763817+0000","last_became_active":"2023-11-26T07:17:54.531186+0000","last_became_peered":"2023-11-26T07:17:54.531186+0000","last_unstale":"2023-11-26T07:17:55.763817+0000","last_undegraded":"2023-11-26T07:17:55.763817+0000","last_fullsized":"2023-11-26T07:17:55.763817+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:17:53.483873+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:17:53.483873+0000","last_clean_scrub_stamp":"2023-11-26T07:17:53.483873+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T15:49:39.889795+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.f","version":"19'1","reported_seq":13,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-26T07:17:56.640634+0000","last_change":"2023-11-26T07:17:53.509212+0000","last_active":"2023-11-26T07:17:56.640634+0000","last_peered":"2023-11-26T07:17:56.640634+0000","last_clean":"2023-11-26T07:17:56.640634+0000","last_became_active":"2023-11-26T07:17:53.508602+0000","last_became_peered":"2023-11-26T07:17:53.508602+0000","last_unstale":"2023-11-26T07:17:56.640634+0000","last_undegraded":"2023-11-26T07:17:56.640634+0000","last_fullsized":"2023-11-26T07:17:56.640634+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:17:52.477276+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:17:52.477276+0000","last_clean_scrub_stamp":"2023-11-26T07:17:52.477276+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T10:16:37.754573+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-26T07:17:55.764257+0000","last_change":"2023-11-26T07:17:54.531723+0000","last_active":"2023-11-26T07:17:55.764257+0000","last_peered":"2023-11-26T07:17:55.764257+0000","last_clean":"2023-11-26T07:17:55.764257+0000","last_became_active":"2023-11-26T07:17:54.531321+0000","last_became_peered":"2023-11-26T07:17:54.531321+0000","last_unstale":"2023-11-26T07:17:55.764257+0000","last_undegraded":"2023-11-26T07:17:55.764257+0000","last_fullsized":"2023-11-26T07:17:55.764257+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:17:53.483873+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:17:53.483873+0000","last_clean_scrub_stamp":"2023-11-26T07:17:53.483873+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T15:19:56.111921+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.e","version":"19'2","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-26T07:17:55.514998+0000","last_change":"2023-11-26T07:17:53.524243+0000","last_active":"2023-11-26T07:17:55.514998+0000","last_peered":"2023-11-26T07:17:55.514998+0000","last_clean":"2023-11-26T07:17:55.514998+0000","last_became_active":"2023-11-26T07:17:53.523933+0000","last_became_peered":"2023-11-26T07:17:53.523933+0000","last_unstale":"2023-11-26T07:17:55.514998+0000","last_undegraded":"2023-11-26T07:17:55.514998+0000","last_fullsized":"2023-11-26T07:17:55.514998+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:17:52.477276+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:17:52.477276+0000","last_clean_scrub_stamp":"2023-11-26T07:17:52.477276+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T15:44:14.149131+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-26T07:17:55.764439+0000","last_change":"2023-11-26T07:17:54.531576+0000","last_active":"2023-11-26T07:17:55.764439+0000","last_peered":"2023-11-26T07:17:55.764439+0000","last_clean":"2023-11-26T07:17:55.764439+0000","last_became_active":"2023-11-26T07:17:54.531261+0000","last_became_peered":"2023-11-26T07:17:54.531261+0000","last_unstale":"2023-11-26T07:17:55.764439+0000","last_undegraded":"2023-11-26T07:17:55.764439+0000","last_fullsized":"2023-11-26T07:17:55.764439+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:17:53.483873+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:17:53.483873+0000","last_clean_scrub_stamp":"2023-11-26T07:17:53.483873+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T11:46:17.783756+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.d","version":"19'1","reported_seq":13,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-26T07:17:55.516854+0000","last_change":"2023-11-26T07:17:53.512500+0000","last_active":"2023-11-26T07:17:55.516854+0000","last_peered":"2023-11-26T07:17:55.516854+0000","last_clean":"2023-11-26T07:17:55.516854+0000","last_became_active":"2023-11-26T07:17:53.512082+0000","last_became_peered":"2023-11-26T07:17:53.512082+0000","last_unstale":"2023-11-26T07:17:55.516854+0000","last_undegraded":"2023-11-26T07:17:55.516854+0000","last_fullsized":"2023-11-26T07:17:55.516854+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:17:52.477276+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:17:52.477276+0000","last_clean_scrub_stamp":"2023-11-26T07:17:52.477276+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T18:33:31.647310+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-26T07:17:55.539068+0000","last_change":"2023-11-26T07:17:54.528819+0000","last_active":"2023-11-26T07:17:55.539068+0000","last_peered":"2023-11-26T07:17:55.539068+0000","last_clean":"2023-11-26T07:17:55.539068+0000","last_became_active":"2023-11-26T07:17:54.528290+0000","last_became_peered":"2023-11-26T07:17:54.528290+0000","last_unstale":"2023-11-26T07:17:55.539068+0000","last_undegraded":"2023-11-26T07:17:55.539068+0000","last_fullsized":"2023-11-26T07:17:55.539068+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:17:53.483873+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:17:53.483873+0000","last_clean_scrub_stamp":"2023-11-26T07:17:53.483873+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T12:21:10.834625+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.c","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-26T07:17:55.514878+0000","last_change":"2023-11-26T07:17:53.514524+0000","last_active":"2023-11-26T07:17:55.514878+0000","last_peered":"2023-11-26T07:17:55.514878+0000","last_clean":"2023-11-26T07:17:55.514878+0000","last_became_active":"2023-11-26T07:17:53.514070+0000","last_became_peered":"2023-11-26T07:17:53.514070+0000","last_unstale":"2023-11-26T07:17:55.514878+0000","last_undegraded":"2023-11-26T07:17:55.514878+0000","last_fullsized":"2023-11-26T07:17:55.514878+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:17:52.477276+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:17:52.477276+0000","last_clean_scrub_stamp":"2023-11-26T07:17:52.477276+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T15:47:35.416235+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-26T07:17:55.763997+0000","last_change":"2023-11-26T07:17:54.535603+0000","last_active":"2023-11-26T07:17:55.763997+0000","last_peered":"2023-11-26T07:17:55.763997+0000","last_clean":"2023-11-26T07:17:55.763997+0000","last_became_active":"2023-11-26T07:17:54.535343+0000","last_became_peered":"2023-11-26T07:17:54.535343+0000","last_unstale":"2023-11-26T07:17:55.763997+0000","last_undegraded":"2023-11-26T07:17:55.763997+0000","last_fullsized":"2023-11-26T07:17:55.763997+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:17:53.483873+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:17:53.483873+0000","last_clean_scrub_stamp":"2023-11-26T07:17:53.483873+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T18:27:00.305240+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.0","version":"15'117","reported_seq":165,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-11-26T07:17:54.507822+0000","last_change":"2023-11-26T07:17:43.827802+0000","last_active":"2023-11-26T07:17:54.507822+0000","last_peered":"2023-11-26T07:17:54.507822+0000","last_clean":"2023-11-26T07:17:54.507822+0000","last_became_active":"2023-11-26T07:17:43.827308+0000","last_became_peered":"2023-11-26T07:17:43.827308+0000","last_unstale":"2023-11-26T07:17:54.507822+0000","last_undegraded":"2023-11-26T07:17:54.507822+0000","last_fullsized":"2023-11-26T07:17:54.507822+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:17:42.160945+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:17:42.160945+0000","last_clean_scrub_stamp":"2023-11-26T07:17:42.160945+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T08:17:54.167366+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-26T07:17:55.538622+0000","last_change":"2023-11-26T07:17:53.514397+0000","last_active":"2023-11-26T07:17:55.538622+0000","last_peered":"2023-11-26T07:17:55.538622+0000","last_clean":"2023-11-26T07:17:55.538622+0000","last_became_active":"2023-11-26T07:17:53.513876+0000","last_became_peered":"2023-11-26T07:17:53.513876+0000","last_unstale":"2023-11-26T07:17:55.538622+0000","last_undegraded":"2023-11-26T07:17:55.538622+0000","last_fullsized":"2023-11-26T07:17:55.538622+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:17:52.477276+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:17:52.477276+0000","last_clean_scrub_stamp":"2023-11-26T07:17:52.477276+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T18:33:31.647310+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-26T07:17:56.640762+0000","last_change":"2023-11-26T07:17:54.522327+0000","last_active":"2023-11-26T07:17:56.640762+0000","last_peered":"2023-11-26T07:17:56.640762+0000","last_clean":"2023-11-26T07:17:56.640762+0000","last_became_active":"2023-11-26T07:17:54.522058+0000","last_became_peered":"2023-11-26T07:17:54.522058+0000","last_unstale":"2023-11-26T07:17:56.640762+0000","last_undegraded":"2023-11-26T07:17:56.640762+0000","last_fullsized":"2023-11-26T07:17:56.640762+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:17:53.483873+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:17:53.483873+0000","last_clean_scrub_stamp":"2023-11-26T07:17:53.483873+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T11:16:25.801280+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-26T07:17:55.763573+0000","last_change":"2023-11-26T07:17:53.522494+0000","last_active":"2023-11-26T07:17:55.763573+0000","last_peered":"2023-11-26T07:17:55.763573+0000","last_clean":"2023-11-26T07:17:55.763573+0000","last_became_active":"2023-11-26T07:17:53.522221+0000","last_became_peered":"2023-11-26T07:17:53.522221+0000","last_unstale":"2023-11-26T07:17:55.763573+0000","last_undegraded":"2023-11-26T07:17:55.763573+0000","last_fullsized":"2023-11-26T07:17:55.763573+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:17:52.477276+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:17:52.477276+0000","last_clean_scrub_stamp":"2023-11-26T07:17:52.477276+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T13:45:33.759365+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-26T07:17:56.068604+0000","last_change":"2023-11-26T07:17:54.505916+0000","last_active":"2023-11-26T07:17:56.068604+0000","last_peered":"2023-11-26T07:17:56.068604+0000","last_clean":"2023-11-26T07:17:56.068604+0000","last_became_active":"2023-11-26T07:17:54.505618+0000","last_became_peered":"2023-11-26T07:17:54.505618+0000","last_unstale":"2023-11-26T07:17:56.068604+0000","last_undegraded":"2023-11-26T07:17:56.068604+0000","last_fullsized":"2023-11-26T07:17:56.068604+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:17:53.483873+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:17:53.483873+0000","last_clean_scrub_stamp":"2023-11-26T07:17:53.483873+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T19:07:42.549166+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-26T07:17:55.514749+0000","last_change":"2023-11-26T07:17:53.523403+0000","last_active":"2023-11-26T07:17:55.514749+0000","last_peered":"2023-11-26T07:17:55.514749+0000","last_clean":"2023-11-26T07:17:55.514749+0000","last_became_active":"2023-11-26T07:17:53.523061+0000","last_became_peered":"2023-11-26T07:17:53.523061+0000","last_unstale":"2023-11-26T07:17:55.514749+0000","last_undegraded":"2023-11-26T07:17:55.514749+0000","last_fullsized":"2023-11-26T07:17:55.514749+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:17:52.477276+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:17:52.477276+0000","last_clean_scrub_stamp":"2023-11-26T07:17:52.477276+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T09:18:47.261084+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-26T07:17:55.516400+0000","last_change":"2023-11-26T07:17:54.519128+0000","last_active":"2023-11-26T07:17:55.516400+0000","last_peered":"2023-11-26T07:17:55.516400+0000","last_clean":"2023-11-26T07:17:55.516400+0000","last_became_active":"2023-11-26T07:17:54.518918+0000","last_became_peered":"2023-11-26T07:17:54.518918+0000","last_unstale":"2023-11-26T07:17:55.516400+0000","last_undegraded":"2023-11-26T07:17:55.516400+0000","last_fullsized":"2023-11-26T07:17:55.516400+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:17:53.483873+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:17:53.483873+0000","last_clean_scrub_stamp":"2023-11-26T07:17:53.483873+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T07:50:04.472214+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-26T07:17:55.538911+0000","last_change":"2023-11-26T07:17:53.517034+0000","last_active":"2023-11-26T07:17:55.538911+0000","last_peered":"2023-11-26T07:17:55.538911+0000","last_clean":"2023-11-26T07:17:55.538911+0000","last_became_active":"2023-11-26T07:17:53.516689+0000","last_became_peered":"2023-11-26T07:17:53.516689+0000","last_unstale":"2023-11-26T07:17:55.538911+0000","last_undegraded":"2023-11-26T07:17:55.538911+0000","last_fullsized":"2023-11-26T07:17:55.538911+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:17:52.477276+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:17:52.477276+0000","last_clean_scrub_stamp":"2023-11-26T07:17:52.477276+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T16:07:45.587443+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-26T07:17:56.640131+0000","last_change":"2023-11-26T07:17:54.501979+0000","last_active":"2023-11-26T07:17:56.640131+0000","last_peered":"2023-11-26T07:17:56.640131+0000","last_clean":"2023-11-26T07:17:56.640131+0000","last_became_active":"2023-11-26T07:17:54.501682+0000","last_became_peered":"2023-11-26T07:17:54.501682+0000","last_unstale":"2023-11-26T07:17:56.640131+0000","last_undegraded":"2023-11-26T07:17:56.640131+0000","last_fullsized":"2023-11-26T07:17:56.640131+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:17:53.483873+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:17:53.483873+0000","last_clean_scrub_stamp":"2023-11-26T07:17:53.483873+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T12:51:17.339533+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-26T07:17:55.516646+0000","last_change":"2023-11-26T07:17:53.509606+0000","last_active":"2023-11-26T07:17:55.516646+0000","last_peered":"2023-11-26T07:17:55.516646+0000","last_clean":"2023-11-26T07:17:55.516646+0000","last_became_active":"2023-11-26T07:17:53.509268+0000","last_became_peered":"2023-11-26T07:17:53.509268+0000","last_unstale":"2023-11-26T07:17:55.516646+0000","last_undegraded":"2023-11-26T07:17:55.516646+0000","last_fullsized":"2023-11-26T07:17:55.516646+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:17:52.477276+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:17:52.477276+0000","last_clean_scrub_stamp":"2023-11-26T07:17:52.477276+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T16:07:45.587443+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-26T07:17:55.538908+0000","last_change":"2023-11-26T07:17:54.527293+0000","last_active":"2023-11-26T07:17:55.538908+0000","last_peered":"2023-11-26T07:17:55.538908+0000","last_clean":"2023-11-26T07:17:55.538908+0000","last_became_active":"2023-11-26T07:17:54.526666+0000","last_became_peered":"2023-11-26T07:17:54.526666+0000","last_unstale":"2023-11-26T07:17:55.538908+0000","last_undegraded":"2023-11-26T07:17:55.538908+0000","last_fullsized":"2023-11-26T07:17:55.538908+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:17:53.483873+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:17:53.483873+0000","last_clean_scrub_stamp":"2023-11-26T07:17:53.483873+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T17:14:14.930365+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-26T07:17:55.764220+0000","last_change":"2023-11-26T07:17:53.527907+0000","last_active":"2023-11-26T07:17:55.764220+0000","last_peered":"2023-11-26T07:17:55.764220+0000","last_clean":"2023-11-26T07:17:55.764220+0000","last_became_active":"2023-11-26T07:17:53.527624+0000","last_became_peered":"2023-11-26T07:17:53.527624+0000","last_unstale":"2023-11-26T07:17:55.764220+0000","last_undegraded":"2023-11-26T07:17:55.764220+0000","last_fullsized":"2023-11-26T07:17:55.764220+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:17:52.477276+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:17:52.477276+0000","last_clean_scrub_stamp":"2023-11-26T07:17:52.477276+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T11:16:24.794684+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-26T07:17:55.516576+0000","last_change":"2023-11-26T07:17:54.522960+0000","last_active":"2023-11-26T07:17:55.516576+0000","last_peered":"2023-11-26T07:17:55.516576+0000","last_clean":"2023-11-26T07:17:55.516576+0000","last_became_active":"2023-11-26T07:17:54.522658+0000","last_became_peered":"2023-11-26T07:17:54.522658+0000","last_unstale":"2023-11-26T07:17:55.516576+0000","last_undegraded":"2023-11-26T07:17:55.516576+0000","last_fullsized":"2023-11-26T07:17:55.516576+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:17:53.483873+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:17:53.483873+0000","last_clean_scrub_stamp":"2023-11-26T07:17:53.483873+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T13:18:15.521322+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"20'1","reported_seq":13,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-26T07:17:56.715464+0000","last_change":"2023-11-26T07:17:53.513529+0000","last_active":"2023-11-26T07:17:56.715464+0000","last_peered":"2023-11-26T07:17:56.715464+0000","last_clean":"2023-11-26T07:17:56.715464+0000","last_became_active":"2023-11-26T07:17:53.513125+0000","last_became_peered":"2023-11-26T07:17:53.513125+0000","last_unstale":"2023-11-26T07:17:56.715464+0000","last_undegraded":"2023-11-26T07:17:56.715464+0000","last_fullsized":"2023-11-26T07:17:56.715464+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:17:52.477276+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:17:52.477276+0000","last_clean_scrub_stamp":"2023-11-26T07:17:52.477276+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T09:53:28.237406+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-26T07:17:56.641191+0000","last_change":"2023-11-26T07:17:54.525518+0000","last_active":"2023-11-26T07:17:56.641191+0000","last_peered":"2023-11-26T07:17:56.641191+0000","last_clean":"2023-11-26T07:17:56.641191+0000","last_became_active":"2023-11-26T07:17:54.524972+0000","last_became_peered":"2023-11-26T07:17:54.524972+0000","last_unstale":"2023-11-26T07:17:56.641191+0000","last_undegraded":"2023-11-26T07:17:56.641191+0000","last_fullsized":"2023-11-26T07:17:56.641191+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:17:53.483873+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:17:53.483873+0000","last_clean_scrub_stamp":"2023-11-26T07:17:53.483873+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T15:49:39.889795+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.7","version":"19'2","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-26T07:17:56.640410+0000","last_change":"2023-11-26T07:17:53.523424+0000","last_active":"2023-11-26T07:17:56.640410+0000","last_peered":"2023-11-26T07:17:56.640410+0000","last_clean":"2023-11-26T07:17:56.640410+0000","last_became_active":"2023-11-26T07:17:53.523139+0000","last_became_peered":"2023-11-26T07:17:53.523139+0000","last_unstale":"2023-11-26T07:17:56.640410+0000","last_undegraded":"2023-11-26T07:17:56.640410+0000","last_fullsized":"2023-11-26T07:17:56.640410+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:17:52.477276+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:17:52.477276+0000","last_clean_scrub_stamp":"2023-11-26T07:17:52.477276+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T09:18:47.261084+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-26T07:17:56.069026+0000","last_change":"2023-11-26T07:17:54.523889+0000","last_active":"2023-11-26T07:17:56.069026+0000","last_peered":"2023-11-26T07:17:56.069026+0000","last_clean":"2023-11-26T07:17:56.069026+0000","last_became_active":"2023-11-26T07:17:54.523610+0000","last_became_peered":"2023-11-26T07:17:54.523610+0000","last_unstale":"2023-11-26T07:17:56.069026+0000","last_undegraded":"2023-11-26T07:17:56.069026+0000","last_fullsized":"2023-11-26T07:17:56.069026+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:17:53.483873+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:17:53.483873+0000","last_clean_scrub_stamp":"2023-11-26T07:17:53.483873+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T14:45:51.557148+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-26T07:17:55.763603+0000","last_change":"2023-11-26T07:17:53.521942+0000","last_active":"2023-11-26T07:17:55.763603+0000","last_peered":"2023-11-26T07:17:55.763603+0000","last_clean":"2023-11-26T07:17:55.763603+0000","last_became_active":"2023-11-26T07:17:53.521487+0000","last_became_peered":"2023-11-26T07:17:53.521487+0000","last_unstale":"2023-11-26T07:17:55.763603+0000","last_undegraded":"2023-11-26T07:17:55.763603+0000","last_fullsized":"2023-11-26T07:17:55.763603+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:17:52.477276+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:17:52.477276+0000","last_clean_scrub_stamp":"2023-11-26T07:17:52.477276+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T09:07:52.941378+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-26T07:17:56.640134+0000","last_change":"2023-11-26T07:17:54.516390+0000","last_active":"2023-11-26T07:17:56.640134+0000","last_peered":"2023-11-26T07:17:56.640134+0000","last_clean":"2023-11-26T07:17:56.640134+0000","last_became_active":"2023-11-26T07:17:54.515967+0000","last_became_peered":"2023-11-26T07:17:54.515967+0000","last_unstale":"2023-11-26T07:17:56.640134+0000","last_undegraded":"2023-11-26T07:17:56.640134+0000","last_fullsized":"2023-11-26T07:17:56.640134+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:17:53.483873+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:17:53.483873+0000","last_clean_scrub_stamp":"2023-11-26T07:17:53.483873+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T17:42:29.554668+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.9","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-26T07:17:55.516770+0000","last_change":"2023-11-26T07:17:53.520545+0000","last_active":"2023-11-26T07:17:55.516770+0000","last_peered":"2023-11-26T07:17:55.516770+0000","last_clean":"2023-11-26T07:17:55.516770+0000","last_became_active":"2023-11-26T07:17:53.520076+0000","last_became_peered":"2023-11-26T07:17:53.520076+0000","last_unstale":"2023-11-26T07:17:55.516770+0000","last_undegraded":"2023-11-26T07:17:55.516770+0000","last_fullsized":"2023-11-26T07:17:55.516770+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:17:52.477276+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:17:52.477276+0000","last_clean_scrub_stamp":"2023-11-26T07:17:52.477276+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T12:51:16.332936+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-26T07:17:56.641019+0000","last_change":"2023-11-26T07:17:54.512904+0000","last_active":"2023-11-26T07:17:56.641019+0000","last_peered":"2023-11-26T07:17:56.641019+0000","last_clean":"2023-11-26T07:17:56.641019+0000","last_became_active":"2023-11-26T07:17:54.512619+0000","last_became_peered":"2023-11-26T07:17:54.512619+0000","last_unstale":"2023-11-26T07:17:56.641019+0000","last_undegraded":"2023-11-26T07:17:56.641019+0000","last_fullsized":"2023-11-26T07:17:56.641019+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:17:53.483873+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:17:53.483873+0000","last_clean_scrub_stamp":"2023-11-26T07:17:53.483873+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T14:49:27.883374+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.a","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-26T07:17:55.516544+0000","last_change":"2023-11-26T07:17:53.523572+0000","last_active":"2023-11-26T07:17:55.516544+0000","last_peered":"2023-11-26T07:17:55.516544+0000","last_clean":"2023-11-26T07:17:55.516544+0000","last_became_active":"2023-11-26T07:17:53.523266+0000","last_became_peered":"2023-11-26T07:17:53.523266+0000","last_unstale":"2023-11-26T07:17:55.516544+0000","last_undegraded":"2023-11-26T07:17:55.516544+0000","last_fullsized":"2023-11-26T07:17:55.516544+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:17:52.477276+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:17:52.477276+0000","last_clean_scrub_stamp":"2023-11-26T07:17:52.477276+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T13:59:52.977195+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-26T07:17:56.640963+0000","last_change":"2023-11-26T07:17:54.515437+0000","last_active":"2023-11-26T07:17:56.640963+0000","last_peered":"2023-11-26T07:17:56.640963+0000","last_clean":"2023-11-26T07:17:56.640963+0000","last_became_active":"2023-11-26T07:17:54.515127+0000","last_became_peered":"2023-11-26T07:17:54.515127+0000","last_unstale":"2023-11-26T07:17:56.640963+0000","last_undegraded":"2023-11-26T07:17:56.640963+0000","last_fullsized":"2023-11-26T07:17:56.640963+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:17:53.483873+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:17:53.483873+0000","last_clean_scrub_stamp":"2023-11-26T07:17:53.483873+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T14:45:51.557148+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.b","version":"19'2","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-26T07:17:55.763968+0000","last_change":"2023-11-26T07:17:53.525249+0000","last_active":"2023-11-26T07:17:55.763968+0000","last_peered":"2023-11-26T07:17:55.763968+0000","last_clean":"2023-11-26T07:17:55.763968+0000","last_became_active":"2023-11-26T07:17:53.524957+0000","last_became_peered":"2023-11-26T07:17:53.524957+0000","last_unstale":"2023-11-26T07:17:55.763968+0000","last_undegraded":"2023-11-26T07:17:55.763968+0000","last_fullsized":"2023-11-26T07:17:55.763968+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:17:52.477276+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:17:52.477276+0000","last_clean_scrub_stamp":"2023-11-26T07:17:52.477276+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T19:07:41.542569+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-26T07:17:56.639556+0000","last_change":"2023-11-26T07:17:54.521923+0000","last_active":"2023-11-26T07:17:56.639556+0000","last_peered":"2023-11-26T07:17:56.639556+0000","last_clean":"2023-11-26T07:17:56.639556+0000","last_became_active":"2023-11-26T07:17:54.521503+0000","last_became_peered":"2023-11-26T07:17:54.521503+0000","last_unstale":"2023-11-26T07:17:56.639556+0000","last_undegraded":"2023-11-26T07:17:56.639556+0000","last_fullsized":"2023-11-26T07:17:56.639556+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:17:53.483873+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:17:53.483873+0000","last_clean_scrub_stamp":"2023-11-26T07:17:53.483873+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T16:07:46.594040+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-26T07:17:55.538848+0000","last_change":"2023-11-26T07:17:54.526945+0000","last_active":"2023-11-26T07:17:55.538848+0000","last_peered":"2023-11-26T07:17:55.538848+0000","last_clean":"2023-11-26T07:17:55.538848+0000","last_became_active":"2023-11-26T07:17:54.526110+0000","last_became_peered":"2023-11-26T07:17:54.526110+0000","last_unstale":"2023-11-26T07:17:55.538848+0000","last_undegraded":"2023-11-26T07:17:55.538848+0000","last_fullsized":"2023-11-26T07:17:55.538848+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:17:53.483873+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:17:53.483873+0000","last_clean_scrub_stamp":"2023-11-26T07:17:53.483873+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T10:18:52.982852+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1c","version":"19'2","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-26T07:17:55.763845+0000","last_change":"2023-11-26T07:17:53.524780+0000","last_active":"2023-11-26T07:17:55.763845+0000","last_peered":"2023-11-26T07:17:55.763845+0000","last_clean":"2023-11-26T07:17:55.763845+0000","last_became_active":"2023-11-26T07:17:53.524429+0000","last_became_peered":"2023-11-26T07:17:53.524429+0000","last_unstale":"2023-11-26T07:17:55.763845+0000","last_undegraded":"2023-11-26T07:17:55.763845+0000","last_fullsized":"2023-11-26T07:17:55.763845+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:17:52.477276+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:17:52.477276+0000","last_clean_scrub_stamp":"2023-11-26T07:17:52.477276+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T09:53:28.237406+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-26T07:17:55.539011+0000","last_change":"2023-11-26T07:17:54.523997+0000","last_active":"2023-11-26T07:17:55.539011+0000","last_peered":"2023-11-26T07:17:55.539011+0000","last_clean":"2023-11-26T07:17:55.539011+0000","last_became_active":"2023-11-26T07:17:54.523690+0000","last_became_peered":"2023-11-26T07:17:54.523690+0000","last_unstale":"2023-11-26T07:17:55.539011+0000","last_undegraded":"2023-11-26T07:17:55.539011+0000","last_fullsized":"2023-11-26T07:17:55.539011+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:17:53.483873+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:17:53.483873+0000","last_clean_scrub_stamp":"2023-11-26T07:17:53.483873+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T11:28:10.566680+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1d","version":"19'2","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-26T07:17:55.764098+0000","last_change":"2023-11-26T07:17:53.526233+0000","last_active":"2023-11-26T07:17:55.764098+0000","last_peered":"2023-11-26T07:17:55.764098+0000","last_clean":"2023-11-26T07:17:55.764098+0000","last_became_active":"2023-11-26T07:17:53.525918+0000","last_became_peered":"2023-11-26T07:17:53.525918+0000","last_unstale":"2023-11-26T07:17:55.764098+0000","last_undegraded":"2023-11-26T07:17:55.764098+0000","last_fullsized":"2023-11-26T07:17:55.764098+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:17:52.477276+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:17:52.477276+0000","last_clean_scrub_stamp":"2023-11-26T07:17:52.477276+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T15:18:16.303185+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-26T07:17:56.068892+0000","last_change":"2023-11-26T07:17:54.528760+0000","last_active":"2023-11-26T07:17:56.068892+0000","last_peered":"2023-11-26T07:17:56.068892+0000","last_clean":"2023-11-26T07:17:56.068892+0000","last_became_active":"2023-11-26T07:17:54.528456+0000","last_became_peered":"2023-11-26T07:17:54.528456+0000","last_unstale":"2023-11-26T07:17:56.068892+0000","last_undegraded":"2023-11-26T07:17:56.068892+0000","last_fullsized":"2023-11-26T07:17:56.068892+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:17:53.483873+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:17:53.483873+0000","last_clean_scrub_stamp":"2023-11-26T07:17:53.483873+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T15:18:17.309782+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1e","version":"19'1","reported_seq":13,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-26T07:17:56.641018+0000","last_change":"2023-11-26T07:17:53.523797+0000","last_active":"2023-11-26T07:17:56.641018+0000","last_peered":"2023-11-26T07:17:56.641018+0000","last_clean":"2023-11-26T07:17:56.641018+0000","last_became_active":"2023-11-26T07:17:53.523521+0000","last_became_peered":"2023-11-26T07:17:53.523521+0000","last_unstale":"2023-11-26T07:17:56.641018+0000","last_undegraded":"2023-11-26T07:17:56.641018+0000","last_fullsized":"2023-11-26T07:17:56.641018+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:17:52.477276+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:17:52.477276+0000","last_clean_scrub_stamp":"2023-11-26T07:17:52.477276+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T09:53:28.237406+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-26T07:17:56.640929+0000","last_change":"2023-11-26T07:17:54.517290+0000","last_active":"2023-11-26T07:17:56.640929+0000","last_peered":"2023-11-26T07:17:56.640929+0000","last_clean":"2023-11-26T07:17:56.640929+0000","last_became_active":"2023-11-26T07:17:54.516998+0000","last_became_peered":"2023-11-26T07:17:54.516998+0000","last_unstale":"2023-11-26T07:17:56.640929+0000","last_undegraded":"2023-11-26T07:17:56.640929+0000","last_fullsized":"2023-11-26T07:17:56.640929+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:17:53.483873+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:17:53.483873+0000","last_clean_scrub_stamp":"2023-11-26T07:17:53.483873+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T13:18:15.521322+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1f","version":"19'4","reported_seq":16,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-26T07:17:56.069044+0000","last_change":"2023-11-26T07:17:53.524902+0000","last_active":"2023-11-26T07:17:56.069044+0000","last_peered":"2023-11-26T07:17:56.069044+0000","last_clean":"2023-11-26T07:17:56.069044+0000","last_became_active":"2023-11-26T07:17:53.524530+0000","last_became_peered":"2023-11-26T07:17:53.524530+0000","last_unstale":"2023-11-26T07:17:56.069044+0000","last_undegraded":"2023-11-26T07:17:56.069044+0000","last_fullsized":"2023-11-26T07:17:56.069044+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:17:52.477276+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:17:52.477276+0000","last_clean_scrub_stamp":"2023-11-26T07:17:52.477276+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T09:18:47.261084+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":39,"ondisk_log_size":39,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574853,"num_pgs":30,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27808,"kb_used_data":784,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344032,"statfs":{"total":96636764160,"available":96608288768,"internally_reserved":0,"allocated":802816,"data_stored":661860,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.76200000000000001}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.60999999999999999}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.80900000000000005}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.73299999999999998}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.57999999999999996}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.65200000000000002}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.83399999999999996}]}]},{"osd":6,"up_from":13,"seq":55834574855,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27236,"kb_used_data":212,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344604,"statfs":{"total":96636764160,"available":96608874496,"internally_reserved":0,"allocated":217088,"data_stored":72522,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.7549999999999999}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":5.4240000000000004}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.8520000000000001}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":5.4390000000000001}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.47199999999999998}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.45600000000000002}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.57799999999999996}]}]},{"osd":5,"up_from":13,"seq":55834574853,"num_pgs":30,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27228,"kb_used_data":204,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344612,"statfs":{"total":96636764160,"available":96608882688,"internally_reserved":0,"allocated":208896,"data_stored":71000,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.69999999999999996}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.75900000000000001}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.85699999999999998}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.78200000000000003}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.748}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.81299999999999994}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.89500000000000002}]}]},{"osd":4,"up_from":13,"seq":55834574855,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27236,"kb_used_data":212,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344604,"statfs":{"total":96636764160,"available":96608874496,"internally_reserved":0,"allocated":217088,"data_stored":71036,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":4.1689999999999996}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.66100000000000003}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":2.7850000000000001}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.591}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.496}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":2.8330000000000002}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":4.3689999999999998}]}]},{"osd":3,"up_from":13,"seq":55834574856,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27236,"kb_used_data":212,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344604,"statfs":{"total":96636764160,"available":96608874496,"internally_reserved":0,"allocated":217088,"data_stored":72522,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1592,"internal_metadata":27654600},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.33200000000000002}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.36499999999999999}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.40300000000000002}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.67200000000000004}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.65500000000000003}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.64200000000000002}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.70099999999999996}]}]},{"osd":2,"up_from":13,"seq":55834574856,"num_pgs":13,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27224,"kb_used_data":200,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344616,"statfs":{"total":96636764160,"available":96608886784,"internally_reserved":0,"allocated":204800,"data_stored":70910,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":4.7030000000000003}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":4.2690000000000001}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.70499999999999996}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":4.4269999999999996}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.627}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":4.0449999999999999}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":4.1219999999999999}]}]},{"osd":1,"up_from":13,"seq":55834574855,"num_pgs":19,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27232,"kb_used_data":208,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344608,"statfs":{"total":96636764160,"available":96608878592,"internally_reserved":0,"allocated":212992,"data_stored":71070,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.502}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.40600000000000003}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.59799999999999998}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.69299999999999995}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.58199999999999996}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.66200000000000003}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.55100000000000005}]}]},{"osd":0,"up_from":13,"seq":55834574855,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27812,"kb_used_data":792,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344028,"statfs":{"total":96636764160,"available":96608284672,"internally_reserved":0,"allocated":811008,"data_stored":661916,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.28000000000000003}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.41999999999999998}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.48199999999999998}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.69199999999999995}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.73299999999999998}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.83699999999999997}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.89100000000000001}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":638,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":160,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":126,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-11-26T07:18:03.162 DEBUG:teuthology.orchestra.run.smithi143:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-11-26T07:18:03.338 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:03.334+0000 7f94855f8640 1 Processor -- start 2023-11-26T07:18:03.338 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:03.334+0000 7f94855f8640 1 -- start start 2023-11-26T07:18:03.339 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:03.334+0000 7f94855f8640 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7f948015aef0 0x7f948006b320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:18:03.339 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:03.334+0000 7f94855f8640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f948006c560 0x7f94800b2290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:18:03.339 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:03.334+0000 7f94855f8640 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7f94800b27d0 0x7f9480071aa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:18:03.339 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:03.334+0000 7f94855f8640 1 -- --> v2:172.21.15.143:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f948015f7c0 con 0x7f94800b27d0 2023-11-26T07:18:03.339 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:03.334+0000 7f94855f8640 1 -- --> v2:172.21.15.188:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f94800b2c30 con 0x7f948015aef0 2023-11-26T07:18:03.340 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:03.334+0000 7f94855f8640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f948006c310 con 0x7f948006c560 2023-11-26T07:18:03.340 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:03.334+0000 7f947ffff640 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7f94800b27d0 0x7f9480071aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:18:03.340 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:03.334+0000 7f947ffff640 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7f94800b27d0 0x7f9480071aa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:46774/0 (socket says 172.21.15.143:46774) 2023-11-26T07:18:03.340 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:03.334+0000 7f947ffff640 1 -- 172.21.15.143:0/3457015976 learned_addr learned my addr 172.21.15.143:0/3457015976 (peer_addr_for_me v2:172.21.15.143:0/0) 2023-11-26T07:18:03.340 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:03.334+0000 7f947effd640 1 --2- 172.21.15.143:0/3457015976 >> v2:172.21.15.188:3300/0 conn(0x7f948015aef0 0x7f948006b320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:18:03.341 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:03.334+0000 7f947e7fc640 1 --2- 172.21.15.143:0/3457015976 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f948006c560 0x7f94800b2290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:18:03.341 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:03.334+0000 7f947ffff640 1 -- 172.21.15.143:0/3457015976 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f948006c560 msgr2=0x7f94800b2290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-26T07:18:03.341 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:03.334+0000 7f947ffff640 1 --2- 172.21.15.143:0/3457015976 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f948006c560 0x7f94800b2290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:18:03.341 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:03.334+0000 7f947ffff640 1 -- 172.21.15.143:0/3457015976 >> v2:172.21.15.188:3300/0 conn(0x7f948015aef0 msgr2=0x7f948006b320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-26T07:18:03.341 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:03.334+0000 7f947ffff640 1 --2- 172.21.15.143:0/3457015976 >> v2:172.21.15.188:3300/0 conn(0x7f948015aef0 0x7f948006b320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:18:03.341 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:03.334+0000 7f947ffff640 1 -- 172.21.15.143:0/3457015976 --> v2:172.21.15.143:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f948015ebb0 con 0x7f94800b27d0 2023-11-26T07:18:03.342 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:03.338+0000 7f947ffff640 1 --2- 172.21.15.143:0/3457015976 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f94800b27d0 0x7f9480071aa0 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7f948006ffb0 tx=0x7f947801c6e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a4efd7f768da88c5 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:18:03.343 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:03.338+0000 7f947dffb640 1 -- 172.21.15.143:0/3457015976 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f94780a5560 con 0x7f94800b27d0 2023-11-26T07:18:03.343 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:03.338+0000 7f947dffb640 1 -- 172.21.15.143:0/3457015976 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f94780a56c0 con 0x7f94800b27d0 2023-11-26T07:18:03.343 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:03.338+0000 7f947dffb640 1 -- 172.21.15.143:0/3457015976 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f94780a5a50 con 0x7f94800b27d0 2023-11-26T07:18:03.344 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:03.338+0000 7f94855f8640 1 -- 172.21.15.143:0/3457015976 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f94800b27d0 msgr2=0x7f9480071aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:18:03.344 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:03.338+0000 7f94855f8640 1 --2- 172.21.15.143:0/3457015976 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f94800b27d0 0x7f9480071aa0 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7f948006ffb0 tx=0x7f947801c6e0 comp rx=0 tx=0).stop 2023-11-26T07:18:03.344 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:03.338+0000 7f94855f8640 1 -- 172.21.15.143:0/3457015976 shutdown_connections 2023-11-26T07:18:03.344 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:03.338+0000 7f94855f8640 1 --2- 172.21.15.143:0/3457015976 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f94800b27d0 0x7f9480071aa0 unknown :-1 s=CLOSED pgs=119 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:18:03.344 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:03.338+0000 7f94855f8640 1 --2- 172.21.15.143:0/3457015976 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f948006c560 0x7f94800b2290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:18:03.344 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:03.338+0000 7f94855f8640 1 --2- 172.21.15.143:0/3457015976 >> v2:172.21.15.188:3300/0 conn(0x7f948015aef0 0x7f948006b320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:18:03.344 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:03.338+0000 7f94855f8640 1 -- 172.21.15.143:0/3457015976 >> 172.21.15.143:0/3457015976 conn(0x7f9480066720 msgr2=0x7f948006dc10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:18:03.344 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:03.338+0000 7f94855f8640 1 -- 172.21.15.143:0/3457015976 shutdown_connections 2023-11-26T07:18:03.344 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:03.338+0000 7f94855f8640 1 -- 172.21.15.143:0/3457015976 wait complete. 2023-11-26T07:18:03.345 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:03.342+0000 7f94855f8640 1 Processor -- start 2023-11-26T07:18:03.345 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:03.342+0000 7f94855f8640 1 -- start start 2023-11-26T07:18:03.347 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:03.342+0000 7f94855f8640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f948006c560 0x7f9480149ba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:18:03.347 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:03.342+0000 7f94855f8640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f94800b27d0 0x7f94801493d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:18:03.347 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:03.342+0000 7f94855f8640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f948015aef0 0x7f94800d0660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:18:03.347 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:03.342+0000 7f94855f8640 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f948014c890 con 0x7f948015aef0 2023-11-26T07:18:03.347 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:03.342+0000 7f94855f8640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f948014bf60 con 0x7f948006c560 2023-11-26T07:18:03.347 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:03.342+0000 7f94855f8640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f948014c240 con 0x7f94800b27d0 2023-11-26T07:18:03.348 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:03.342+0000 7f947ffff640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f948015aef0 0x7f94800d0660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:18:03.348 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:03.342+0000 7f947ffff640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f948015aef0 0x7f94800d0660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:46790/0 (socket says 172.21.15.143:46790) 2023-11-26T07:18:03.348 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:03.342+0000 7f947ffff640 1 -- 172.21.15.143:0/2926621393 learned_addr learned my addr 172.21.15.143:0/2926621393 (peer_addr_for_me v2:172.21.15.143:0/0) 2023-11-26T07:18:03.348 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:03.342+0000 7f947ffff640 1 -- 172.21.15.143:0/2926621393 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f94800b27d0 msgr2=0x7f94801493d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:18:03.348 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:03.342+0000 7f947ffff640 1 --2- 172.21.15.143:0/2926621393 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f94800b27d0 0x7f94801493d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:18:03.348 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:03.342+0000 7f947effd640 1 --2- 172.21.15.143:0/2926621393 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f948006c560 0x7f9480149ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:18:03.348 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:03.342+0000 7f947ffff640 1 -- 172.21.15.143:0/2926621393 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f948006c560 msgr2=0x7f9480149ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:18:03.348 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:03.342+0000 7f947ffff640 1 --2- 172.21.15.143:0/2926621393 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f948006c560 0x7f9480149ba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:18:03.348 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:03.342+0000 7f947ffff640 1 -- 172.21.15.143:0/2926621393 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9478009b30 con 0x7f948015aef0 2023-11-26T07:18:03.348 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:03.342+0000 7f947effd640 1 --2- 172.21.15.143:0/2926621393 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f948006c560 0x7f9480149ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-26T07:18:03.348 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:03.342+0000 7f947ffff640 1 --2- 172.21.15.143:0/2926621393 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f948015aef0 0x7f94800d0660 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7f94780af040 tx=0x7f9478019480 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:18:03.349 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:03.342+0000 7f94677fe640 1 -- 172.21.15.143:0/2926621393 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f94780b8670 con 0x7f948015aef0 2023-11-26T07:18:03.349 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:03.342+0000 7f94677fe640 1 -- 172.21.15.143:0/2926621393 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f94780b87d0 con 0x7f948015aef0 2023-11-26T07:18:03.349 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:03.342+0000 7f94677fe640 1 -- 172.21.15.143:0/2926621393 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f94780b8b80 con 0x7f948015aef0 2023-11-26T07:18:03.350 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:03.342+0000 7f94855f8640 1 -- 172.21.15.143:0/2926621393 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f948014c520 con 0x7f948015aef0 2023-11-26T07:18:03.350 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:03.342+0000 7f94855f8640 1 -- 172.21.15.143:0/2926621393 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f948014a320 con 0x7f948015aef0 2023-11-26T07:18:03.350 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:03.346+0000 7f94855f8640 1 -- 172.21.15.143:0/2926621393 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f948014ca00 con 0x7f948015aef0 2023-11-26T07:18:03.350 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:03.346+0000 7f94677fe640 1 -- 172.21.15.143:0/2926621393 <== mon.0 v2:172.21.15.143:3300/0 4 ==== mgrmap(e 5) v1 ==== 85852+0+0 (secure 0 0 0) 0x7f94780b8ce0 con 0x7f948015aef0 2023-11-26T07:18:03.351 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:03.346+0000 7f94677fe640 1 --2- 172.21.15.143:0/2926621393 >> [v2:172.21.15.143:6832/2150557544,v1:172.21.15.143:6833/2150557544] conn(0x7f946c066470 0x7f946c068930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:18:03.351 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:03.346+0000 7f94677fe640 1 -- 172.21.15.143:0/2926621393 <== mon.0 v2:172.21.15.143:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f947811f2e0 con 0x7f948015aef0 2023-11-26T07:18:03.351 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:03.346+0000 7f947effd640 1 --2- 172.21.15.143:0/2926621393 >> [v2:172.21.15.143:6832/2150557544,v1:172.21.15.143:6833/2150557544] conn(0x7f946c066470 0x7f946c068930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:18:03.351 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:03.346+0000 7f947effd640 1 --2- 172.21.15.143:0/2926621393 >> [v2:172.21.15.143:6832/2150557544,v1:172.21.15.143:6833/2150557544] conn(0x7f946c066470 0x7f946c068930 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7f94801f9290 tx=0x7f9470008040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:18:03.355 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:03.350+0000 7f94677fe640 1 -- 172.21.15.143:0/2926621393 <== mon.0 v2:172.21.15.143:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172649 (secure 0 0 0) 0x7f94780ec8c0 con 0x7f948015aef0 2023-11-26T07:18:03.502 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:03.498+0000 7f94855f8640 1 -- 172.21.15.143:0/2926621393 --> [v2:172.21.15.143:6832/2150557544,v1:172.21.15.143:6833/2150557544] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f948015b2d0 con 0x7f946c066470 2023-11-26T07:18:03.508 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:03.502+0000 7f94677fe640 1 -- 172.21.15.143:0/2926621393 <== mgr.4100 v2:172.21.15.143:6832/2150557544 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+264593 (secure 0 0 0) 0x7f9470009d70 con 0x7f946c066470 2023-11-26T07:18:03.511 INFO:teuthology.orchestra.run.smithi143.stdout: 2023-11-26T07:18:03.513 INFO:teuthology.orchestra.run.smithi143.stderr:dumped all 2023-11-26T07:18:03.514 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:03.506+0000 7f94855f8640 1 -- 172.21.15.143:0/2926621393 >> [v2:172.21.15.143:6832/2150557544,v1:172.21.15.143:6833/2150557544] conn(0x7f946c066470 msgr2=0x7f946c068930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:18:03.514 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:03.506+0000 7f94855f8640 1 --2- 172.21.15.143:0/2926621393 >> [v2:172.21.15.143:6832/2150557544,v1:172.21.15.143:6833/2150557544] conn(0x7f946c066470 0x7f946c068930 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7f94801f9290 tx=0x7f9470008040 comp rx=0 tx=0).stop 2023-11-26T07:18:03.514 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:03.506+0000 7f94855f8640 1 -- 172.21.15.143:0/2926621393 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f948015aef0 msgr2=0x7f94800d0660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:18:03.514 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:03.506+0000 7f94855f8640 1 --2- 172.21.15.143:0/2926621393 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f948015aef0 0x7f94800d0660 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7f94780af040 tx=0x7f9478019480 comp rx=0 tx=0).stop 2023-11-26T07:18:03.514 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:03.506+0000 7f94855f8640 1 -- 172.21.15.143:0/2926621393 shutdown_connections 2023-11-26T07:18:03.514 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:03.506+0000 7f94855f8640 1 --2- 172.21.15.143:0/2926621393 >> [v2:172.21.15.143:6832/2150557544,v1:172.21.15.143:6833/2150557544] conn(0x7f946c066470 0x7f946c068930 unknown :-1 s=CLOSED pgs=102 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:18:03.514 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:03.506+0000 7f94855f8640 1 --2- 172.21.15.143:0/2926621393 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f948015aef0 0x7f94800d0660 unknown :-1 s=CLOSED pgs=120 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:18:03.514 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:03.506+0000 7f94855f8640 1 --2- 172.21.15.143:0/2926621393 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f94800b27d0 0x7f94801493d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:18:03.514 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:03.506+0000 7f94855f8640 1 --2- 172.21.15.143:0/2926621393 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f948006c560 0x7f9480149ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:18:03.514 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:03.506+0000 7f94855f8640 1 -- 172.21.15.143:0/2926621393 >> 172.21.15.143:0/2926621393 conn(0x7f9480066720 msgr2=0x7f948015b650 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:18:03.515 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:03.506+0000 7f94855f8640 1 -- 172.21.15.143:0/2926621393 shutdown_connections 2023-11-26T07:18:03.515 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:03.506+0000 7f94855f8640 1 -- 172.21.15.143:0/2926621393 wait complete. 2023-11-26T07:18:03.531 INFO:teuthology.orchestra.run.smithi143.stdout:{"pg_ready":true,"pg_map":{"version":23,"stamp":"2023-11-26T07:18:01.695219+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592778,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":204,"num_write_kb":2430,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":156,"ondisk_log_size":156,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":192,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":219012,"kb_used_data":2824,"kb_used_omap":12,"kb_used_meta":216051,"kb_avail":754755708,"statfs":{"total":773094113280,"available":772869844992,"internally_reserved":0,"allocated":2891776,"data_stored":1752836,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12749,"internal_metadata":221236787},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"8.001398"},"pg_stats":[{"pgid":"3.3f","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-26T07:17:55.764359+0000","last_change":"2023-11-26T07:17:54.530144+0000","last_active":"2023-11-26T07:17:55.764359+0000","last_peered":"2023-11-26T07:17:55.764359+0000","last_clean":"2023-11-26T07:17:55.764359+0000","last_became_active":"2023-11-26T07:17:54.529816+0000","last_became_peered":"2023-11-26T07:17:54.529816+0000","last_unstale":"2023-11-26T07:17:55.764359+0000","last_undegraded":"2023-11-26T07:17:55.764359+0000","last_fullsized":"2023-11-26T07:17:55.764359+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:17:53.483873+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:17:53.483873+0000","last_clean_scrub_stamp":"2023-11-26T07:17:53.483873+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T08:53:05.773632+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-26T07:17:55.516522+0000","last_change":"2023-11-26T07:17:54.523453+0000","last_active":"2023-11-26T07:17:55.516522+0000","last_peered":"2023-11-26T07:17:55.516522+0000","last_clean":"2023-11-26T07:17:55.516522+0000","last_became_active":"2023-11-26T07:17:54.523104+0000","last_became_peered":"2023-11-26T07:17:54.523104+0000","last_unstale":"2023-11-26T07:17:55.516522+0000","last_undegraded":"2023-11-26T07:17:55.516522+0000","last_fullsized":"2023-11-26T07:17:55.516522+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:17:53.483873+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:17:53.483873+0000","last_clean_scrub_stamp":"2023-11-26T07:17:53.483873+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T11:16:25.801280+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-26T07:17:56.068788+0000","last_change":"2023-11-26T07:17:54.514169+0000","last_active":"2023-11-26T07:17:56.068788+0000","last_peered":"2023-11-26T07:17:56.068788+0000","last_clean":"2023-11-26T07:17:56.068788+0000","last_became_active":"2023-11-26T07:17:54.513836+0000","last_became_peered":"2023-11-26T07:17:54.513836+0000","last_unstale":"2023-11-26T07:17:56.068788+0000","last_undegraded":"2023-11-26T07:17:56.068788+0000","last_fullsized":"2023-11-26T07:17:56.068788+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:17:53.483873+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:17:53.483873+0000","last_clean_scrub_stamp":"2023-11-26T07:17:53.483873+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T09:53:29.244002+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-26T07:17:55.538957+0000","last_change":"2023-11-26T07:17:54.528554+0000","last_active":"2023-11-26T07:17:55.538957+0000","last_peered":"2023-11-26T07:17:55.538957+0000","last_clean":"2023-11-26T07:17:55.538957+0000","last_became_active":"2023-11-26T07:17:54.527864+0000","last_became_peered":"2023-11-26T07:17:54.527864+0000","last_unstale":"2023-11-26T07:17:55.538957+0000","last_undegraded":"2023-11-26T07:17:55.538957+0000","last_fullsized":"2023-11-26T07:17:55.538957+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:17:53.483873+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:17:53.483873+0000","last_clean_scrub_stamp":"2023-11-26T07:17:53.483873+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T15:49:39.889795+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-26T07:17:55.763887+0000","last_change":"2023-11-26T07:17:54.527144+0000","last_active":"2023-11-26T07:17:55.763887+0000","last_peered":"2023-11-26T07:17:55.763887+0000","last_clean":"2023-11-26T07:17:55.763887+0000","last_became_active":"2023-11-26T07:17:54.525279+0000","last_became_peered":"2023-11-26T07:17:54.525279+0000","last_unstale":"2023-11-26T07:17:55.763887+0000","last_undegraded":"2023-11-26T07:17:55.763887+0000","last_fullsized":"2023-11-26T07:17:55.763887+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:17:53.483873+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:17:53.483873+0000","last_clean_scrub_stamp":"2023-11-26T07:17:53.483873+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T11:28:10.566680+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-26T07:17:55.538672+0000","last_change":"2023-11-26T07:17:54.522607+0000","last_active":"2023-11-26T07:17:55.538672+0000","last_peered":"2023-11-26T07:17:55.538672+0000","last_clean":"2023-11-26T07:17:55.538672+0000","last_became_active":"2023-11-26T07:17:54.522297+0000","last_became_peered":"2023-11-26T07:17:54.522297+0000","last_unstale":"2023-11-26T07:17:55.538672+0000","last_undegraded":"2023-11-26T07:17:55.538672+0000","last_fullsized":"2023-11-26T07:17:55.538672+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:17:53.483873+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:17:53.483873+0000","last_clean_scrub_stamp":"2023-11-26T07:17:53.483873+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T13:18:15.521322+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-26T07:17:55.763706+0000","last_change":"2023-11-26T07:17:54.524898+0000","last_active":"2023-11-26T07:17:55.763706+0000","last_peered":"2023-11-26T07:17:55.763706+0000","last_clean":"2023-11-26T07:17:55.763706+0000","last_became_active":"2023-11-26T07:17:54.524611+0000","last_became_peered":"2023-11-26T07:17:54.524611+0000","last_unstale":"2023-11-26T07:17:55.763706+0000","last_undegraded":"2023-11-26T07:17:55.763706+0000","last_fullsized":"2023-11-26T07:17:55.763706+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:17:53.483873+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:17:53.483873+0000","last_clean_scrub_stamp":"2023-11-26T07:17:53.483873+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T17:14:14.930365+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-26T07:17:56.640574+0000","last_change":"2023-11-26T07:17:54.530342+0000","last_active":"2023-11-26T07:17:56.640574+0000","last_peered":"2023-11-26T07:17:56.640574+0000","last_clean":"2023-11-26T07:17:56.640574+0000","last_became_active":"2023-11-26T07:17:54.529960+0000","last_became_peered":"2023-11-26T07:17:54.529960+0000","last_unstale":"2023-11-26T07:17:56.640574+0000","last_undegraded":"2023-11-26T07:17:56.640574+0000","last_fullsized":"2023-11-26T07:17:56.640574+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:17:53.483873+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:17:53.483873+0000","last_clean_scrub_stamp":"2023-11-26T07:17:53.483873+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T08:42:29.375032+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-26T07:17:55.764389+0000","last_change":"2023-11-26T07:17:54.539062+0000","last_active":"2023-11-26T07:17:55.764389+0000","last_peered":"2023-11-26T07:17:55.764389+0000","last_clean":"2023-11-26T07:17:55.764389+0000","last_became_active":"2023-11-26T07:17:54.538119+0000","last_became_peered":"2023-11-26T07:17:54.538119+0000","last_unstale":"2023-11-26T07:17:55.764389+0000","last_undegraded":"2023-11-26T07:17:55.764389+0000","last_fullsized":"2023-11-26T07:17:55.764389+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:17:53.483873+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:17:53.483873+0000","last_clean_scrub_stamp":"2023-11-26T07:17:53.483873+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T13:57:07.286285+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-26T07:17:56.640341+0000","last_change":"2023-11-26T07:17:54.524564+0000","last_active":"2023-11-26T07:17:56.640341+0000","last_peered":"2023-11-26T07:17:56.640341+0000","last_clean":"2023-11-26T07:17:56.640341+0000","last_became_active":"2023-11-26T07:17:54.524255+0000","last_became_peered":"2023-11-26T07:17:54.524255+0000","last_unstale":"2023-11-26T07:17:56.640341+0000","last_undegraded":"2023-11-26T07:17:56.640341+0000","last_fullsized":"2023-11-26T07:17:56.640341+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:17:53.483873+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:17:53.483873+0000","last_clean_scrub_stamp":"2023-11-26T07:17:53.483873+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T11:16:25.801280+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-26T07:17:56.639652+0000","last_change":"2023-11-26T07:17:54.535310+0000","last_active":"2023-11-26T07:17:56.639652+0000","last_peered":"2023-11-26T07:17:56.639652+0000","last_clean":"2023-11-26T07:17:56.639652+0000","last_became_active":"2023-11-26T07:17:54.534986+0000","last_became_peered":"2023-11-26T07:17:54.534986+0000","last_unstale":"2023-11-26T07:17:56.639652+0000","last_undegraded":"2023-11-26T07:17:56.639652+0000","last_fullsized":"2023-11-26T07:17:56.639652+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:17:53.483873+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:17:53.483873+0000","last_clean_scrub_stamp":"2023-11-26T07:17:53.483873+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T15:18:17.309782+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-26T07:17:56.640026+0000","last_change":"2023-11-26T07:17:54.512408+0000","last_active":"2023-11-26T07:17:56.640026+0000","last_peered":"2023-11-26T07:17:56.640026+0000","last_clean":"2023-11-26T07:17:56.640026+0000","last_became_active":"2023-11-26T07:17:54.512005+0000","last_became_peered":"2023-11-26T07:17:54.512005+0000","last_unstale":"2023-11-26T07:17:56.640026+0000","last_undegraded":"2023-11-26T07:17:56.640026+0000","last_fullsized":"2023-11-26T07:17:56.640026+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:17:53.483873+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:17:53.483873+0000","last_clean_scrub_stamp":"2023-11-26T07:17:53.483873+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T09:07:53.947975+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-26T07:17:56.640756+0000","last_change":"2023-11-26T07:17:54.514825+0000","last_active":"2023-11-26T07:17:56.640756+0000","last_peered":"2023-11-26T07:17:56.640756+0000","last_clean":"2023-11-26T07:17:56.640756+0000","last_became_active":"2023-11-26T07:17:54.514502+0000","last_became_peered":"2023-11-26T07:17:54.514502+0000","last_unstale":"2023-11-26T07:17:56.640756+0000","last_undegraded":"2023-11-26T07:17:56.640756+0000","last_fullsized":"2023-11-26T07:17:56.640756+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:17:53.483873+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:17:53.483873+0000","last_clean_scrub_stamp":"2023-11-26T07:17:53.483873+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T07:50:04.472214+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-26T07:17:56.068691+0000","last_change":"2023-11-26T07:17:54.521737+0000","last_active":"2023-11-26T07:17:56.068691+0000","last_peered":"2023-11-26T07:17:56.068691+0000","last_clean":"2023-11-26T07:17:56.068691+0000","last_became_active":"2023-11-26T07:17:54.521387+0000","last_became_peered":"2023-11-26T07:17:54.521387+0000","last_unstale":"2023-11-26T07:17:56.068691+0000","last_undegraded":"2023-11-26T07:17:56.068691+0000","last_fullsized":"2023-11-26T07:17:56.068691+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:17:53.483873+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:17:53.483873+0000","last_clean_scrub_stamp":"2023-11-26T07:17:53.483873+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T18:33:32.653907+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-26T07:17:55.516698+0000","last_change":"2023-11-26T07:17:54.533852+0000","last_active":"2023-11-26T07:17:55.516698+0000","last_peered":"2023-11-26T07:17:55.516698+0000","last_clean":"2023-11-26T07:17:55.516698+0000","last_became_active":"2023-11-26T07:17:54.533532+0000","last_became_peered":"2023-11-26T07:17:54.533532+0000","last_unstale":"2023-11-26T07:17:55.516698+0000","last_undegraded":"2023-11-26T07:17:55.516698+0000","last_fullsized":"2023-11-26T07:17:55.516698+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:17:53.483873+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:17:53.483873+0000","last_clean_scrub_stamp":"2023-11-26T07:17:53.483873+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T14:45:51.557148+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-26T07:17:55.538556+0000","last_change":"2023-11-26T07:17:54.523130+0000","last_active":"2023-11-26T07:17:55.538556+0000","last_peered":"2023-11-26T07:17:55.538556+0000","last_clean":"2023-11-26T07:17:55.538556+0000","last_became_active":"2023-11-26T07:17:54.522879+0000","last_became_peered":"2023-11-26T07:17:54.522879+0000","last_unstale":"2023-11-26T07:17:55.538556+0000","last_undegraded":"2023-11-26T07:17:55.538556+0000","last_fullsized":"2023-11-26T07:17:55.538556+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:17:53.483873+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:17:53.483873+0000","last_clean_scrub_stamp":"2023-11-26T07:17:53.483873+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T11:16:25.801280+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-26T07:17:55.539151+0000","last_change":"2023-11-26T07:17:54.534352+0000","last_active":"2023-11-26T07:17:55.539151+0000","last_peered":"2023-11-26T07:17:55.539151+0000","last_clean":"2023-11-26T07:17:55.539151+0000","last_became_active":"2023-11-26T07:17:54.534004+0000","last_became_peered":"2023-11-26T07:17:54.534004+0000","last_unstale":"2023-11-26T07:17:55.539151+0000","last_undegraded":"2023-11-26T07:17:55.539151+0000","last_fullsized":"2023-11-26T07:17:55.539151+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:17:53.483873+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:17:53.483873+0000","last_clean_scrub_stamp":"2023-11-26T07:17:53.483873+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T18:27:00.305240+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-26T07:17:55.514964+0000","last_change":"2023-11-26T07:17:54.515085+0000","last_active":"2023-11-26T07:17:55.514964+0000","last_peered":"2023-11-26T07:17:55.514964+0000","last_clean":"2023-11-26T07:17:55.514964+0000","last_became_active":"2023-11-26T07:17:54.514782+0000","last_became_peered":"2023-11-26T07:17:54.514782+0000","last_unstale":"2023-11-26T07:17:55.514964+0000","last_undegraded":"2023-11-26T07:17:55.514964+0000","last_fullsized":"2023-11-26T07:17:55.514964+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:17:53.483873+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:17:53.483873+0000","last_clean_scrub_stamp":"2023-11-26T07:17:53.483873+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T13:45:34.765962+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-26T07:17:55.764203+0000","last_change":"2023-11-26T07:17:54.529137+0000","last_active":"2023-11-26T07:17:55.764203+0000","last_peered":"2023-11-26T07:17:55.764203+0000","last_clean":"2023-11-26T07:17:55.764203+0000","last_became_active":"2023-11-26T07:17:54.528496+0000","last_became_peered":"2023-11-26T07:17:54.528496+0000","last_unstale":"2023-11-26T07:17:55.764203+0000","last_undegraded":"2023-11-26T07:17:55.764203+0000","last_fullsized":"2023-11-26T07:17:55.764203+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:17:53.483873+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:17:53.483873+0000","last_clean_scrub_stamp":"2023-11-26T07:17:53.483873+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T17:27:24.347347+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-26T07:17:56.639612+0000","last_change":"2023-11-26T07:17:54.521609+0000","last_active":"2023-11-26T07:17:56.639612+0000","last_peered":"2023-11-26T07:17:56.639612+0000","last_clean":"2023-11-26T07:17:56.639612+0000","last_became_active":"2023-11-26T07:17:54.521298+0000","last_became_peered":"2023-11-26T07:17:54.521298+0000","last_unstale":"2023-11-26T07:17:56.639612+0000","last_undegraded":"2023-11-26T07:17:56.639612+0000","last_fullsized":"2023-11-26T07:17:56.639612+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:17:53.483873+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:17:53.483873+0000","last_clean_scrub_stamp":"2023-11-26T07:17:53.483873+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T08:42:29.375032+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-26T07:17:56.068908+0000","last_change":"2023-11-26T07:17:54.526550+0000","last_active":"2023-11-26T07:17:56.068908+0000","last_peered":"2023-11-26T07:17:56.068908+0000","last_clean":"2023-11-26T07:17:56.068908+0000","last_became_active":"2023-11-26T07:17:54.525639+0000","last_became_peered":"2023-11-26T07:17:54.525639+0000","last_unstale":"2023-11-26T07:17:56.068908+0000","last_undegraded":"2023-11-26T07:17:56.068908+0000","last_fullsized":"2023-11-26T07:17:56.068908+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:17:53.483873+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:17:53.483873+0000","last_clean_scrub_stamp":"2023-11-26T07:17:53.483873+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T07:50:04.472214+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-26T07:17:55.538640+0000","last_change":"2023-11-26T07:17:54.534209+0000","last_active":"2023-11-26T07:17:55.538640+0000","last_peered":"2023-11-26T07:17:55.538640+0000","last_clean":"2023-11-26T07:17:55.538640+0000","last_became_active":"2023-11-26T07:17:54.533932+0000","last_became_peered":"2023-11-26T07:17:54.533932+0000","last_unstale":"2023-11-26T07:17:55.538640+0000","last_undegraded":"2023-11-26T07:17:55.538640+0000","last_fullsized":"2023-11-26T07:17:55.538640+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:17:53.483873+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:17:53.483873+0000","last_clean_scrub_stamp":"2023-11-26T07:17:53.483873+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T18:02:15.969376+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-26T07:17:56.640577+0000","last_change":"2023-11-26T07:17:54.526358+0000","last_active":"2023-11-26T07:17:56.640577+0000","last_peered":"2023-11-26T07:17:56.640577+0000","last_clean":"2023-11-26T07:17:56.640577+0000","last_became_active":"2023-11-26T07:17:54.525756+0000","last_became_peered":"2023-11-26T07:17:54.525756+0000","last_unstale":"2023-11-26T07:17:56.640577+0000","last_undegraded":"2023-11-26T07:17:56.640577+0000","last_fullsized":"2023-11-26T07:17:56.640577+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:17:53.483873+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:17:53.483873+0000","last_clean_scrub_stamp":"2023-11-26T07:17:53.483873+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T15:18:17.309782+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-26T07:17:55.538626+0000","last_change":"2023-11-26T07:17:54.526328+0000","last_active":"2023-11-26T07:17:55.538626+0000","last_peered":"2023-11-26T07:17:55.538626+0000","last_clean":"2023-11-26T07:17:55.538626+0000","last_became_active":"2023-11-26T07:17:54.526026+0000","last_became_peered":"2023-11-26T07:17:54.526026+0000","last_unstale":"2023-11-26T07:17:55.538626+0000","last_undegraded":"2023-11-26T07:17:55.538626+0000","last_fullsized":"2023-11-26T07:17:55.538626+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:17:53.483873+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:17:53.483873+0000","last_clean_scrub_stamp":"2023-11-26T07:17:53.483873+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T14:45:51.557148+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-26T07:17:56.641174+0000","last_change":"2023-11-26T07:17:54.521949+0000","last_active":"2023-11-26T07:17:56.641174+0000","last_peered":"2023-11-26T07:17:56.641174+0000","last_clean":"2023-11-26T07:17:56.641174+0000","last_became_active":"2023-11-26T07:17:54.521649+0000","last_became_peered":"2023-11-26T07:17:54.521649+0000","last_unstale":"2023-11-26T07:17:56.641174+0000","last_undegraded":"2023-11-26T07:17:56.641174+0000","last_fullsized":"2023-11-26T07:17:56.641174+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:17:53.483873+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:17:53.483873+0000","last_clean_scrub_stamp":"2023-11-26T07:17:53.483873+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T17:14:14.930365+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-26T07:17:56.640304+0000","last_change":"2023-11-26T07:17:54.526758+0000","last_active":"2023-11-26T07:17:56.640304+0000","last_peered":"2023-11-26T07:17:56.640304+0000","last_clean":"2023-11-26T07:17:56.640304+0000","last_became_active":"2023-11-26T07:17:54.525563+0000","last_became_peered":"2023-11-26T07:17:54.525563+0000","last_unstale":"2023-11-26T07:17:56.640304+0000","last_undegraded":"2023-11-26T07:17:56.640304+0000","last_fullsized":"2023-11-26T07:17:56.640304+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:17:53.483873+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:17:53.483873+0000","last_clean_scrub_stamp":"2023-11-26T07:17:53.483873+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T07:50:04.472214+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-26T07:17:55.516791+0000","last_change":"2023-11-26T07:17:54.520560+0000","last_active":"2023-11-26T07:17:55.516791+0000","last_peered":"2023-11-26T07:17:55.516791+0000","last_clean":"2023-11-26T07:17:55.516791+0000","last_became_active":"2023-11-26T07:17:54.520377+0000","last_became_peered":"2023-11-26T07:17:54.520377+0000","last_unstale":"2023-11-26T07:17:55.516791+0000","last_undegraded":"2023-11-26T07:17:55.516791+0000","last_fullsized":"2023-11-26T07:17:55.516791+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:17:53.483873+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:17:53.483873+0000","last_clean_scrub_stamp":"2023-11-26T07:17:53.483873+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T18:02:15.969376+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-26T07:17:56.640998+0000","last_change":"2023-11-26T07:17:54.527041+0000","last_active":"2023-11-26T07:17:56.640998+0000","last_peered":"2023-11-26T07:17:56.640998+0000","last_clean":"2023-11-26T07:17:56.640998+0000","last_became_active":"2023-11-26T07:17:54.526391+0000","last_became_peered":"2023-11-26T07:17:54.526391+0000","last_unstale":"2023-11-26T07:17:56.640998+0000","last_undegraded":"2023-11-26T07:17:56.640998+0000","last_fullsized":"2023-11-26T07:17:56.640998+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:17:53.483873+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:17:53.483873+0000","last_clean_scrub_stamp":"2023-11-26T07:17:53.483873+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T18:02:15.969376+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-26T07:17:56.068828+0000","last_change":"2023-11-26T07:17:54.512264+0000","last_active":"2023-11-26T07:17:56.068828+0000","last_peered":"2023-11-26T07:17:56.068828+0000","last_clean":"2023-11-26T07:17:56.068828+0000","last_became_active":"2023-11-26T07:17:54.511963+0000","last_became_peered":"2023-11-26T07:17:54.511963+0000","last_unstale":"2023-11-26T07:17:56.068828+0000","last_undegraded":"2023-11-26T07:17:56.068828+0000","last_fullsized":"2023-11-26T07:17:56.068828+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:17:53.483873+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:17:53.483873+0000","last_clean_scrub_stamp":"2023-11-26T07:17:53.483873+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T08:42:29.375032+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-26T07:17:55.538793+0000","last_change":"2023-11-26T07:17:54.528162+0000","last_active":"2023-11-26T07:17:55.538793+0000","last_peered":"2023-11-26T07:17:55.538793+0000","last_clean":"2023-11-26T07:17:55.538793+0000","last_became_active":"2023-11-26T07:17:54.527657+0000","last_became_peered":"2023-11-26T07:17:54.527657+0000","last_unstale":"2023-11-26T07:17:55.538793+0000","last_undegraded":"2023-11-26T07:17:55.538793+0000","last_fullsized":"2023-11-26T07:17:55.538793+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:17:53.483873+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:17:53.483873+0000","last_clean_scrub_stamp":"2023-11-26T07:17:53.483873+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T14:49:27.883374+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-26T07:17:55.514728+0000","last_change":"2023-11-26T07:17:54.519107+0000","last_active":"2023-11-26T07:17:55.514728+0000","last_peered":"2023-11-26T07:17:55.514728+0000","last_clean":"2023-11-26T07:17:55.514728+0000","last_became_active":"2023-11-26T07:17:54.518822+0000","last_became_peered":"2023-11-26T07:17:54.518822+0000","last_unstale":"2023-11-26T07:17:55.514728+0000","last_undegraded":"2023-11-26T07:17:55.514728+0000","last_fullsized":"2023-11-26T07:17:55.514728+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:17:53.483873+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:17:53.483873+0000","last_clean_scrub_stamp":"2023-11-26T07:17:53.483873+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T09:07:53.947975+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-26T07:17:56.639246+0000","last_change":"2023-11-26T07:17:54.513368+0000","last_active":"2023-11-26T07:17:56.639246+0000","last_peered":"2023-11-26T07:17:56.639246+0000","last_clean":"2023-11-26T07:17:56.639246+0000","last_became_active":"2023-11-26T07:17:54.511290+0000","last_became_peered":"2023-11-26T07:17:54.511290+0000","last_unstale":"2023-11-26T07:17:56.639246+0000","last_undegraded":"2023-11-26T07:17:56.639246+0000","last_fullsized":"2023-11-26T07:17:56.639246+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:17:53.483873+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:17:53.483873+0000","last_clean_scrub_stamp":"2023-11-26T07:17:53.483873+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T18:33:32.653907+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-26T07:17:56.639579+0000","last_change":"2023-11-26T07:17:54.513190+0000","last_active":"2023-11-26T07:17:56.639579+0000","last_peered":"2023-11-26T07:17:56.639579+0000","last_clean":"2023-11-26T07:17:56.639579+0000","last_became_active":"2023-11-26T07:17:54.512872+0000","last_became_peered":"2023-11-26T07:17:54.512872+0000","last_unstale":"2023-11-26T07:17:56.639579+0000","last_undegraded":"2023-11-26T07:17:56.639579+0000","last_fullsized":"2023-11-26T07:17:56.639579+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:17:53.483873+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:17:53.483873+0000","last_clean_scrub_stamp":"2023-11-26T07:17:53.483873+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T12:32:48.249974+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1b","version":"19'1","reported_seq":13,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-26T07:17:56.640973+0000","last_change":"2023-11-26T07:17:53.525183+0000","last_active":"2023-11-26T07:17:56.640973+0000","last_peered":"2023-11-26T07:17:56.640973+0000","last_clean":"2023-11-26T07:17:56.640973+0000","last_became_active":"2023-11-26T07:17:53.524904+0000","last_became_peered":"2023-11-26T07:17:53.524904+0000","last_unstale":"2023-11-26T07:17:56.640973+0000","last_undegraded":"2023-11-26T07:17:56.640973+0000","last_fullsized":"2023-11-26T07:17:56.640973+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:17:52.477276+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:17:52.477276+0000","last_clean_scrub_stamp":"2023-11-26T07:17:52.477276+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T13:59:52.977195+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-26T07:17:56.068970+0000","last_change":"2023-11-26T07:17:54.522974+0000","last_active":"2023-11-26T07:17:56.068970+0000","last_peered":"2023-11-26T07:17:56.068970+0000","last_clean":"2023-11-26T07:17:56.068970+0000","last_became_active":"2023-11-26T07:17:54.521283+0000","last_became_peered":"2023-11-26T07:17:54.521283+0000","last_unstale":"2023-11-26T07:17:56.068970+0000","last_undegraded":"2023-11-26T07:17:56.068970+0000","last_fullsized":"2023-11-26T07:17:56.068970+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:17:53.483873+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:17:53.483873+0000","last_clean_scrub_stamp":"2023-11-26T07:17:53.483873+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T11:16:25.801280+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1a","version":"19'1","reported_seq":13,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-26T07:17:56.639442+0000","last_change":"2023-11-26T07:17:53.516099+0000","last_active":"2023-11-26T07:17:56.639442+0000","last_peered":"2023-11-26T07:17:56.639442+0000","last_clean":"2023-11-26T07:17:56.639442+0000","last_became_active":"2023-11-26T07:17:53.515687+0000","last_became_peered":"2023-11-26T07:17:53.515687+0000","last_unstale":"2023-11-26T07:17:56.639442+0000","last_undegraded":"2023-11-26T07:17:56.639442+0000","last_fullsized":"2023-11-26T07:17:56.639442+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:17:52.477276+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:17:52.477276+0000","last_clean_scrub_stamp":"2023-11-26T07:17:52.477276+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T09:07:52.941378+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.19","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-26T07:17:56.640738+0000","last_change":"2023-11-26T07:17:53.523159+0000","last_active":"2023-11-26T07:17:56.640738+0000","last_peered":"2023-11-26T07:17:56.640738+0000","last_clean":"2023-11-26T07:17:56.640738+0000","last_became_active":"2023-11-26T07:17:53.522824+0000","last_became_peered":"2023-11-26T07:17:53.522824+0000","last_unstale":"2023-11-26T07:17:56.640738+0000","last_undegraded":"2023-11-26T07:17:56.640738+0000","last_fullsized":"2023-11-26T07:17:56.640738+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:17:52.477276+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:17:52.477276+0000","last_clean_scrub_stamp":"2023-11-26T07:17:52.477276+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T13:45:33.759365+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-26T07:17:56.641065+0000","last_change":"2023-11-26T07:17:54.515953+0000","last_active":"2023-11-26T07:17:56.641065+0000","last_peered":"2023-11-26T07:17:56.641065+0000","last_clean":"2023-11-26T07:17:56.641065+0000","last_became_active":"2023-11-26T07:17:54.515662+0000","last_became_peered":"2023-11-26T07:17:54.515662+0000","last_unstale":"2023-11-26T07:17:56.641065+0000","last_undegraded":"2023-11-26T07:17:56.641065+0000","last_fullsized":"2023-11-26T07:17:56.641065+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:17:53.483873+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:17:53.483873+0000","last_clean_scrub_stamp":"2023-11-26T07:17:53.483873+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T10:18:52.982852+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.18","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-26T07:17:55.538849+0000","last_change":"2023-11-26T07:17:53.516308+0000","last_active":"2023-11-26T07:17:55.538849+0000","last_peered":"2023-11-26T07:17:55.538849+0000","last_clean":"2023-11-26T07:17:55.538849+0000","last_became_active":"2023-11-26T07:17:53.516027+0000","last_became_peered":"2023-11-26T07:17:53.516027+0000","last_unstale":"2023-11-26T07:17:55.538849+0000","last_undegraded":"2023-11-26T07:17:55.538849+0000","last_fullsized":"2023-11-26T07:17:55.538849+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:17:52.477276+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:17:52.477276+0000","last_clean_scrub_stamp":"2023-11-26T07:17:52.477276+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T08:42:28.368435+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-26T07:17:55.516385+0000","last_change":"2023-11-26T07:17:54.521790+0000","last_active":"2023-11-26T07:17:55.516385+0000","last_peered":"2023-11-26T07:17:55.516385+0000","last_clean":"2023-11-26T07:17:55.516385+0000","last_became_active":"2023-11-26T07:17:54.521475+0000","last_became_peered":"2023-11-26T07:17:54.521475+0000","last_unstale":"2023-11-26T07:17:55.516385+0000","last_undegraded":"2023-11-26T07:17:55.516385+0000","last_fullsized":"2023-11-26T07:17:55.516385+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:17:53.483873+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:17:53.483873+0000","last_clean_scrub_stamp":"2023-11-26T07:17:53.483873+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T15:18:17.309782+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.17","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-26T07:17:56.640306+0000","last_change":"2023-11-26T07:17:53.514460+0000","last_active":"2023-11-26T07:17:56.640306+0000","last_peered":"2023-11-26T07:17:56.640306+0000","last_clean":"2023-11-26T07:17:56.640306+0000","last_became_active":"2023-11-26T07:17:53.514170+0000","last_became_peered":"2023-11-26T07:17:53.514170+0000","last_unstale":"2023-11-26T07:17:56.640306+0000","last_undegraded":"2023-11-26T07:17:56.640306+0000","last_fullsized":"2023-11-26T07:17:56.640306+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:17:52.477276+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:17:52.477276+0000","last_clean_scrub_stamp":"2023-11-26T07:17:52.477276+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T15:47:35.416235+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-26T07:17:55.538995+0000","last_change":"2023-11-26T07:17:54.533776+0000","last_active":"2023-11-26T07:17:55.538995+0000","last_peered":"2023-11-26T07:17:55.538995+0000","last_clean":"2023-11-26T07:17:55.538995+0000","last_became_active":"2023-11-26T07:17:54.533279+0000","last_became_peered":"2023-11-26T07:17:54.533279+0000","last_unstale":"2023-11-26T07:17:55.538995+0000","last_undegraded":"2023-11-26T07:17:55.538995+0000","last_fullsized":"2023-11-26T07:17:55.538995+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:17:53.483873+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:17:53.483873+0000","last_clean_scrub_stamp":"2023-11-26T07:17:53.483873+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T15:38:12.078699+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.16","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-26T07:17:55.539107+0000","last_change":"2023-11-26T07:17:53.520128+0000","last_active":"2023-11-26T07:17:55.539107+0000","last_peered":"2023-11-26T07:17:55.539107+0000","last_clean":"2023-11-26T07:17:55.539107+0000","last_became_active":"2023-11-26T07:17:53.519399+0000","last_became_peered":"2023-11-26T07:17:53.519399+0000","last_unstale":"2023-11-26T07:17:55.539107+0000","last_undegraded":"2023-11-26T07:17:55.539107+0000","last_fullsized":"2023-11-26T07:17:55.539107+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:17:52.477276+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:17:52.477276+0000","last_clean_scrub_stamp":"2023-11-26T07:17:52.477276+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T15:18:16.303185+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-26T07:17:56.068974+0000","last_change":"2023-11-26T07:17:54.528621+0000","last_active":"2023-11-26T07:17:56.068974+0000","last_peered":"2023-11-26T07:17:56.068974+0000","last_clean":"2023-11-26T07:17:56.068974+0000","last_became_active":"2023-11-26T07:17:54.528333+0000","last_became_peered":"2023-11-26T07:17:54.528333+0000","last_unstale":"2023-11-26T07:17:56.068974+0000","last_undegraded":"2023-11-26T07:17:56.068974+0000","last_fullsized":"2023-11-26T07:17:56.068974+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:17:53.483873+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:17:53.483873+0000","last_clean_scrub_stamp":"2023-11-26T07:17:53.483873+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T13:18:15.521322+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.15","version":"19'3","reported_seq":15,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-26T07:17:55.516929+0000","last_change":"2023-11-26T07:17:53.515987+0000","last_active":"2023-11-26T07:17:55.516929+0000","last_peered":"2023-11-26T07:17:55.516929+0000","last_clean":"2023-11-26T07:17:55.516929+0000","last_became_active":"2023-11-26T07:17:53.515360+0000","last_became_peered":"2023-11-26T07:17:53.515360+0000","last_unstale":"2023-11-26T07:17:55.516929+0000","last_undegraded":"2023-11-26T07:17:55.516929+0000","last_fullsized":"2023-11-26T07:17:55.516929+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:17:52.477276+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:17:52.477276+0000","last_clean_scrub_stamp":"2023-11-26T07:17:52.477276+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T19:07:41.542569+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-26T07:17:56.640029+0000","last_change":"2023-11-26T07:17:54.530537+0000","last_active":"2023-11-26T07:17:56.640029+0000","last_peered":"2023-11-26T07:17:56.640029+0000","last_clean":"2023-11-26T07:17:56.640029+0000","last_became_active":"2023-11-26T07:17:54.530074+0000","last_became_peered":"2023-11-26T07:17:54.530074+0000","last_unstale":"2023-11-26T07:17:56.640029+0000","last_undegraded":"2023-11-26T07:17:56.640029+0000","last_fullsized":"2023-11-26T07:17:56.640029+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:17:53.483873+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:17:53.483873+0000","last_clean_scrub_stamp":"2023-11-26T07:17:53.483873+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T13:45:34.765962+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.14","version":"19'6","reported_seq":18,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-26T07:17:56.639634+0000","last_change":"2023-11-26T07:17:53.522850+0000","last_active":"2023-11-26T07:17:56.639634+0000","last_peered":"2023-11-26T07:17:56.639634+0000","last_clean":"2023-11-26T07:17:56.639634+0000","last_became_active":"2023-11-26T07:17:53.522377+0000","last_became_peered":"2023-11-26T07:17:53.522377+0000","last_unstale":"2023-11-26T07:17:56.639634+0000","last_undegraded":"2023-11-26T07:17:56.639634+0000","last_fullsized":"2023-11-26T07:17:56.639634+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:17:52.477276+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:17:52.477276+0000","last_clean_scrub_stamp":"2023-11-26T07:17:52.477276+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T12:51:16.332936+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-26T07:17:55.763972+0000","last_change":"2023-11-26T07:17:54.529408+0000","last_active":"2023-11-26T07:17:55.763972+0000","last_peered":"2023-11-26T07:17:55.763972+0000","last_clean":"2023-11-26T07:17:55.763972+0000","last_became_active":"2023-11-26T07:17:54.528825+0000","last_became_peered":"2023-11-26T07:17:54.528825+0000","last_unstale":"2023-11-26T07:17:55.763972+0000","last_undegraded":"2023-11-26T07:17:55.763972+0000","last_fullsized":"2023-11-26T07:17:55.763972+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:17:53.483873+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:17:53.483873+0000","last_clean_scrub_stamp":"2023-11-26T07:17:53.483873+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T12:21:10.834625+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.13","version":"19'2","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-26T07:17:56.068998+0000","last_change":"2023-11-26T07:17:53.517650+0000","last_active":"2023-11-26T07:17:56.068998+0000","last_peered":"2023-11-26T07:17:56.068998+0000","last_clean":"2023-11-26T07:17:56.068998+0000","last_became_active":"2023-11-26T07:17:53.517349+0000","last_became_peered":"2023-11-26T07:17:53.517349+0000","last_unstale":"2023-11-26T07:17:56.068998+0000","last_undegraded":"2023-11-26T07:17:56.068998+0000","last_fullsized":"2023-11-26T07:17:56.068998+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:17:52.477276+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:17:52.477276+0000","last_clean_scrub_stamp":"2023-11-26T07:17:52.477276+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T12:51:16.332936+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-26T07:17:56.068804+0000","last_change":"2023-11-26T07:17:54.531742+0000","last_active":"2023-11-26T07:17:56.068804+0000","last_peered":"2023-11-26T07:17:56.068804+0000","last_clean":"2023-11-26T07:17:56.068804+0000","last_became_active":"2023-11-26T07:17:54.531136+0000","last_became_peered":"2023-11-26T07:17:54.531136+0000","last_unstale":"2023-11-26T07:17:56.068804+0000","last_undegraded":"2023-11-26T07:17:56.068804+0000","last_fullsized":"2023-11-26T07:17:56.068804+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:17:53.483873+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:17:53.483873+0000","last_clean_scrub_stamp":"2023-11-26T07:17:53.483873+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T16:07:46.594040+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.12","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-26T07:17:55.538769+0000","last_change":"2023-11-26T07:17:53.522311+0000","last_active":"2023-11-26T07:17:55.538769+0000","last_peered":"2023-11-26T07:17:55.538769+0000","last_clean":"2023-11-26T07:17:55.538769+0000","last_became_active":"2023-11-26T07:17:53.522017+0000","last_became_peered":"2023-11-26T07:17:53.522017+0000","last_unstale":"2023-11-26T07:17:55.538769+0000","last_undegraded":"2023-11-26T07:17:55.538769+0000","last_fullsized":"2023-11-26T07:17:55.538769+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:17:52.477276+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:17:52.477276+0000","last_clean_scrub_stamp":"2023-11-26T07:17:52.477276+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T09:53:28.237406+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-26T07:17:55.763876+0000","last_change":"2023-11-26T07:17:54.531973+0000","last_active":"2023-11-26T07:17:55.763876+0000","last_peered":"2023-11-26T07:17:55.763876+0000","last_clean":"2023-11-26T07:17:55.763876+0000","last_became_active":"2023-11-26T07:17:54.531710+0000","last_became_peered":"2023-11-26T07:17:54.531710+0000","last_unstale":"2023-11-26T07:17:55.763876+0000","last_undegraded":"2023-11-26T07:17:55.763876+0000","last_fullsized":"2023-11-26T07:17:55.763876+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:17:53.483873+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:17:53.483873+0000","last_clean_scrub_stamp":"2023-11-26T07:17:53.483873+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T15:38:12.078699+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.11","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-26T07:17:56.639192+0000","last_change":"2023-11-26T07:17:53.519520+0000","last_active":"2023-11-26T07:17:56.639192+0000","last_peered":"2023-11-26T07:17:56.639192+0000","last_clean":"2023-11-26T07:17:56.639192+0000","last_became_active":"2023-11-26T07:17:53.516001+0000","last_became_peered":"2023-11-26T07:17:53.516001+0000","last_unstale":"2023-11-26T07:17:56.639192+0000","last_undegraded":"2023-11-26T07:17:56.639192+0000","last_fullsized":"2023-11-26T07:17:56.639192+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:17:52.477276+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:17:52.477276+0000","last_clean_scrub_stamp":"2023-11-26T07:17:52.477276+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T19:07:41.542569+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-26T07:17:56.639395+0000","last_change":"2023-11-26T07:17:54.511603+0000","last_active":"2023-11-26T07:17:56.639395+0000","last_peered":"2023-11-26T07:17:56.639395+0000","last_clean":"2023-11-26T07:17:56.639395+0000","last_became_active":"2023-11-26T07:17:54.510939+0000","last_became_peered":"2023-11-26T07:17:54.510939+0000","last_unstale":"2023-11-26T07:17:56.639395+0000","last_undegraded":"2023-11-26T07:17:56.639395+0000","last_fullsized":"2023-11-26T07:17:56.639395+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:17:53.483873+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:17:53.483873+0000","last_clean_scrub_stamp":"2023-11-26T07:17:53.483873+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T09:53:29.244002+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.10","version":"19'8","reported_seq":20,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-26T07:17:55.514684+0000","last_change":"2023-11-26T07:17:53.506514+0000","last_active":"2023-11-26T07:17:55.514684+0000","last_peered":"2023-11-26T07:17:55.514684+0000","last_clean":"2023-11-26T07:17:55.514684+0000","last_became_active":"2023-11-26T07:17:53.506081+0000","last_became_peered":"2023-11-26T07:17:53.506081+0000","last_unstale":"2023-11-26T07:17:55.514684+0000","last_undegraded":"2023-11-26T07:17:55.514684+0000","last_fullsized":"2023-11-26T07:17:55.514684+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:17:52.477276+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:17:52.477276+0000","last_clean_scrub_stamp":"2023-11-26T07:17:52.477276+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T09:43:57.945669+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-26T07:17:55.763817+0000","last_change":"2023-11-26T07:17:54.532538+0000","last_active":"2023-11-26T07:17:55.763817+0000","last_peered":"2023-11-26T07:17:55.763817+0000","last_clean":"2023-11-26T07:17:55.763817+0000","last_became_active":"2023-11-26T07:17:54.531186+0000","last_became_peered":"2023-11-26T07:17:54.531186+0000","last_unstale":"2023-11-26T07:17:55.763817+0000","last_undegraded":"2023-11-26T07:17:55.763817+0000","last_fullsized":"2023-11-26T07:17:55.763817+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:17:53.483873+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:17:53.483873+0000","last_clean_scrub_stamp":"2023-11-26T07:17:53.483873+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T15:49:39.889795+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.f","version":"19'1","reported_seq":13,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-26T07:17:56.640634+0000","last_change":"2023-11-26T07:17:53.509212+0000","last_active":"2023-11-26T07:17:56.640634+0000","last_peered":"2023-11-26T07:17:56.640634+0000","last_clean":"2023-11-26T07:17:56.640634+0000","last_became_active":"2023-11-26T07:17:53.508602+0000","last_became_peered":"2023-11-26T07:17:53.508602+0000","last_unstale":"2023-11-26T07:17:56.640634+0000","last_undegraded":"2023-11-26T07:17:56.640634+0000","last_fullsized":"2023-11-26T07:17:56.640634+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:17:52.477276+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:17:52.477276+0000","last_clean_scrub_stamp":"2023-11-26T07:17:52.477276+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T10:16:37.754573+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-26T07:17:55.764257+0000","last_change":"2023-11-26T07:17:54.531723+0000","last_active":"2023-11-26T07:17:55.764257+0000","last_peered":"2023-11-26T07:17:55.764257+0000","last_clean":"2023-11-26T07:17:55.764257+0000","last_became_active":"2023-11-26T07:17:54.531321+0000","last_became_peered":"2023-11-26T07:17:54.531321+0000","last_unstale":"2023-11-26T07:17:55.764257+0000","last_undegraded":"2023-11-26T07:17:55.764257+0000","last_fullsized":"2023-11-26T07:17:55.764257+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:17:53.483873+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:17:53.483873+0000","last_clean_scrub_stamp":"2023-11-26T07:17:53.483873+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T15:19:56.111921+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.e","version":"19'2","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-26T07:17:55.514998+0000","last_change":"2023-11-26T07:17:53.524243+0000","last_active":"2023-11-26T07:17:55.514998+0000","last_peered":"2023-11-26T07:17:55.514998+0000","last_clean":"2023-11-26T07:17:55.514998+0000","last_became_active":"2023-11-26T07:17:53.523933+0000","last_became_peered":"2023-11-26T07:17:53.523933+0000","last_unstale":"2023-11-26T07:17:55.514998+0000","last_undegraded":"2023-11-26T07:17:55.514998+0000","last_fullsized":"2023-11-26T07:17:55.514998+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:17:52.477276+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:17:52.477276+0000","last_clean_scrub_stamp":"2023-11-26T07:17:52.477276+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T15:44:14.149131+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-26T07:17:55.764439+0000","last_change":"2023-11-26T07:17:54.531576+0000","last_active":"2023-11-26T07:17:55.764439+0000","last_peered":"2023-11-26T07:17:55.764439+0000","last_clean":"2023-11-26T07:17:55.764439+0000","last_became_active":"2023-11-26T07:17:54.531261+0000","last_became_peered":"2023-11-26T07:17:54.531261+0000","last_unstale":"2023-11-26T07:17:55.764439+0000","last_undegraded":"2023-11-26T07:17:55.764439+0000","last_fullsized":"2023-11-26T07:17:55.764439+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:17:53.483873+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:17:53.483873+0000","last_clean_scrub_stamp":"2023-11-26T07:17:53.483873+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T11:46:17.783756+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.d","version":"19'1","reported_seq":13,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-26T07:17:55.516854+0000","last_change":"2023-11-26T07:17:53.512500+0000","last_active":"2023-11-26T07:17:55.516854+0000","last_peered":"2023-11-26T07:17:55.516854+0000","last_clean":"2023-11-26T07:17:55.516854+0000","last_became_active":"2023-11-26T07:17:53.512082+0000","last_became_peered":"2023-11-26T07:17:53.512082+0000","last_unstale":"2023-11-26T07:17:55.516854+0000","last_undegraded":"2023-11-26T07:17:55.516854+0000","last_fullsized":"2023-11-26T07:17:55.516854+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:17:52.477276+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:17:52.477276+0000","last_clean_scrub_stamp":"2023-11-26T07:17:52.477276+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T18:33:31.647310+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-26T07:17:55.539068+0000","last_change":"2023-11-26T07:17:54.528819+0000","last_active":"2023-11-26T07:17:55.539068+0000","last_peered":"2023-11-26T07:17:55.539068+0000","last_clean":"2023-11-26T07:17:55.539068+0000","last_became_active":"2023-11-26T07:17:54.528290+0000","last_became_peered":"2023-11-26T07:17:54.528290+0000","last_unstale":"2023-11-26T07:17:55.539068+0000","last_undegraded":"2023-11-26T07:17:55.539068+0000","last_fullsized":"2023-11-26T07:17:55.539068+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:17:53.483873+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:17:53.483873+0000","last_clean_scrub_stamp":"2023-11-26T07:17:53.483873+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T12:21:10.834625+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.c","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-26T07:17:55.514878+0000","last_change":"2023-11-26T07:17:53.514524+0000","last_active":"2023-11-26T07:17:55.514878+0000","last_peered":"2023-11-26T07:17:55.514878+0000","last_clean":"2023-11-26T07:17:55.514878+0000","last_became_active":"2023-11-26T07:17:53.514070+0000","last_became_peered":"2023-11-26T07:17:53.514070+0000","last_unstale":"2023-11-26T07:17:55.514878+0000","last_undegraded":"2023-11-26T07:17:55.514878+0000","last_fullsized":"2023-11-26T07:17:55.514878+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:17:52.477276+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:17:52.477276+0000","last_clean_scrub_stamp":"2023-11-26T07:17:52.477276+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T15:47:35.416235+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-26T07:17:55.763997+0000","last_change":"2023-11-26T07:17:54.535603+0000","last_active":"2023-11-26T07:17:55.763997+0000","last_peered":"2023-11-26T07:17:55.763997+0000","last_clean":"2023-11-26T07:17:55.763997+0000","last_became_active":"2023-11-26T07:17:54.535343+0000","last_became_peered":"2023-11-26T07:17:54.535343+0000","last_unstale":"2023-11-26T07:17:55.763997+0000","last_undegraded":"2023-11-26T07:17:55.763997+0000","last_fullsized":"2023-11-26T07:17:55.763997+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:17:53.483873+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:17:53.483873+0000","last_clean_scrub_stamp":"2023-11-26T07:17:53.483873+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T18:27:00.305240+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.0","version":"15'117","reported_seq":165,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-11-26T07:17:54.507822+0000","last_change":"2023-11-26T07:17:43.827802+0000","last_active":"2023-11-26T07:17:54.507822+0000","last_peered":"2023-11-26T07:17:54.507822+0000","last_clean":"2023-11-26T07:17:54.507822+0000","last_became_active":"2023-11-26T07:17:43.827308+0000","last_became_peered":"2023-11-26T07:17:43.827308+0000","last_unstale":"2023-11-26T07:17:54.507822+0000","last_undegraded":"2023-11-26T07:17:54.507822+0000","last_fullsized":"2023-11-26T07:17:54.507822+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:17:42.160945+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:17:42.160945+0000","last_clean_scrub_stamp":"2023-11-26T07:17:42.160945+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T08:17:54.167366+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-26T07:17:55.538622+0000","last_change":"2023-11-26T07:17:53.514397+0000","last_active":"2023-11-26T07:17:55.538622+0000","last_peered":"2023-11-26T07:17:55.538622+0000","last_clean":"2023-11-26T07:17:55.538622+0000","last_became_active":"2023-11-26T07:17:53.513876+0000","last_became_peered":"2023-11-26T07:17:53.513876+0000","last_unstale":"2023-11-26T07:17:55.538622+0000","last_undegraded":"2023-11-26T07:17:55.538622+0000","last_fullsized":"2023-11-26T07:17:55.538622+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:17:52.477276+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:17:52.477276+0000","last_clean_scrub_stamp":"2023-11-26T07:17:52.477276+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T18:33:31.647310+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-26T07:17:56.640762+0000","last_change":"2023-11-26T07:17:54.522327+0000","last_active":"2023-11-26T07:17:56.640762+0000","last_peered":"2023-11-26T07:17:56.640762+0000","last_clean":"2023-11-26T07:17:56.640762+0000","last_became_active":"2023-11-26T07:17:54.522058+0000","last_became_peered":"2023-11-26T07:17:54.522058+0000","last_unstale":"2023-11-26T07:17:56.640762+0000","last_undegraded":"2023-11-26T07:17:56.640762+0000","last_fullsized":"2023-11-26T07:17:56.640762+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:17:53.483873+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:17:53.483873+0000","last_clean_scrub_stamp":"2023-11-26T07:17:53.483873+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T11:16:25.801280+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-26T07:17:55.763573+0000","last_change":"2023-11-26T07:17:53.522494+0000","last_active":"2023-11-26T07:17:55.763573+0000","last_peered":"2023-11-26T07:17:55.763573+0000","last_clean":"2023-11-26T07:17:55.763573+0000","last_became_active":"2023-11-26T07:17:53.522221+0000","last_became_peered":"2023-11-26T07:17:53.522221+0000","last_unstale":"2023-11-26T07:17:55.763573+0000","last_undegraded":"2023-11-26T07:17:55.763573+0000","last_fullsized":"2023-11-26T07:17:55.763573+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:17:52.477276+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:17:52.477276+0000","last_clean_scrub_stamp":"2023-11-26T07:17:52.477276+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T13:45:33.759365+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-26T07:17:56.068604+0000","last_change":"2023-11-26T07:17:54.505916+0000","last_active":"2023-11-26T07:17:56.068604+0000","last_peered":"2023-11-26T07:17:56.068604+0000","last_clean":"2023-11-26T07:17:56.068604+0000","last_became_active":"2023-11-26T07:17:54.505618+0000","last_became_peered":"2023-11-26T07:17:54.505618+0000","last_unstale":"2023-11-26T07:17:56.068604+0000","last_undegraded":"2023-11-26T07:17:56.068604+0000","last_fullsized":"2023-11-26T07:17:56.068604+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:17:53.483873+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:17:53.483873+0000","last_clean_scrub_stamp":"2023-11-26T07:17:53.483873+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T19:07:42.549166+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-26T07:17:55.514749+0000","last_change":"2023-11-26T07:17:53.523403+0000","last_active":"2023-11-26T07:17:55.514749+0000","last_peered":"2023-11-26T07:17:55.514749+0000","last_clean":"2023-11-26T07:17:55.514749+0000","last_became_active":"2023-11-26T07:17:53.523061+0000","last_became_peered":"2023-11-26T07:17:53.523061+0000","last_unstale":"2023-11-26T07:17:55.514749+0000","last_undegraded":"2023-11-26T07:17:55.514749+0000","last_fullsized":"2023-11-26T07:17:55.514749+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:17:52.477276+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:17:52.477276+0000","last_clean_scrub_stamp":"2023-11-26T07:17:52.477276+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T09:18:47.261084+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-26T07:17:55.516400+0000","last_change":"2023-11-26T07:17:54.519128+0000","last_active":"2023-11-26T07:17:55.516400+0000","last_peered":"2023-11-26T07:17:55.516400+0000","last_clean":"2023-11-26T07:17:55.516400+0000","last_became_active":"2023-11-26T07:17:54.518918+0000","last_became_peered":"2023-11-26T07:17:54.518918+0000","last_unstale":"2023-11-26T07:17:55.516400+0000","last_undegraded":"2023-11-26T07:17:55.516400+0000","last_fullsized":"2023-11-26T07:17:55.516400+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:17:53.483873+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:17:53.483873+0000","last_clean_scrub_stamp":"2023-11-26T07:17:53.483873+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T07:50:04.472214+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-26T07:17:55.538911+0000","last_change":"2023-11-26T07:17:53.517034+0000","last_active":"2023-11-26T07:17:55.538911+0000","last_peered":"2023-11-26T07:17:55.538911+0000","last_clean":"2023-11-26T07:17:55.538911+0000","last_became_active":"2023-11-26T07:17:53.516689+0000","last_became_peered":"2023-11-26T07:17:53.516689+0000","last_unstale":"2023-11-26T07:17:55.538911+0000","last_undegraded":"2023-11-26T07:17:55.538911+0000","last_fullsized":"2023-11-26T07:17:55.538911+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:17:52.477276+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:17:52.477276+0000","last_clean_scrub_stamp":"2023-11-26T07:17:52.477276+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T16:07:45.587443+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-26T07:17:56.640131+0000","last_change":"2023-11-26T07:17:54.501979+0000","last_active":"2023-11-26T07:17:56.640131+0000","last_peered":"2023-11-26T07:17:56.640131+0000","last_clean":"2023-11-26T07:17:56.640131+0000","last_became_active":"2023-11-26T07:17:54.501682+0000","last_became_peered":"2023-11-26T07:17:54.501682+0000","last_unstale":"2023-11-26T07:17:56.640131+0000","last_undegraded":"2023-11-26T07:17:56.640131+0000","last_fullsized":"2023-11-26T07:17:56.640131+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:17:53.483873+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:17:53.483873+0000","last_clean_scrub_stamp":"2023-11-26T07:17:53.483873+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T12:51:17.339533+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-26T07:17:55.516646+0000","last_change":"2023-11-26T07:17:53.509606+0000","last_active":"2023-11-26T07:17:55.516646+0000","last_peered":"2023-11-26T07:17:55.516646+0000","last_clean":"2023-11-26T07:17:55.516646+0000","last_became_active":"2023-11-26T07:17:53.509268+0000","last_became_peered":"2023-11-26T07:17:53.509268+0000","last_unstale":"2023-11-26T07:17:55.516646+0000","last_undegraded":"2023-11-26T07:17:55.516646+0000","last_fullsized":"2023-11-26T07:17:55.516646+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:17:52.477276+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:17:52.477276+0000","last_clean_scrub_stamp":"2023-11-26T07:17:52.477276+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T16:07:45.587443+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-26T07:17:55.538908+0000","last_change":"2023-11-26T07:17:54.527293+0000","last_active":"2023-11-26T07:17:55.538908+0000","last_peered":"2023-11-26T07:17:55.538908+0000","last_clean":"2023-11-26T07:17:55.538908+0000","last_became_active":"2023-11-26T07:17:54.526666+0000","last_became_peered":"2023-11-26T07:17:54.526666+0000","last_unstale":"2023-11-26T07:17:55.538908+0000","last_undegraded":"2023-11-26T07:17:55.538908+0000","last_fullsized":"2023-11-26T07:17:55.538908+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:17:53.483873+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:17:53.483873+0000","last_clean_scrub_stamp":"2023-11-26T07:17:53.483873+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T17:14:14.930365+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-26T07:17:55.764220+0000","last_change":"2023-11-26T07:17:53.527907+0000","last_active":"2023-11-26T07:17:55.764220+0000","last_peered":"2023-11-26T07:17:55.764220+0000","last_clean":"2023-11-26T07:17:55.764220+0000","last_became_active":"2023-11-26T07:17:53.527624+0000","last_became_peered":"2023-11-26T07:17:53.527624+0000","last_unstale":"2023-11-26T07:17:55.764220+0000","last_undegraded":"2023-11-26T07:17:55.764220+0000","last_fullsized":"2023-11-26T07:17:55.764220+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:17:52.477276+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:17:52.477276+0000","last_clean_scrub_stamp":"2023-11-26T07:17:52.477276+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T11:16:24.794684+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-26T07:17:55.516576+0000","last_change":"2023-11-26T07:17:54.522960+0000","last_active":"2023-11-26T07:17:55.516576+0000","last_peered":"2023-11-26T07:17:55.516576+0000","last_clean":"2023-11-26T07:17:55.516576+0000","last_became_active":"2023-11-26T07:17:54.522658+0000","last_became_peered":"2023-11-26T07:17:54.522658+0000","last_unstale":"2023-11-26T07:17:55.516576+0000","last_undegraded":"2023-11-26T07:17:55.516576+0000","last_fullsized":"2023-11-26T07:17:55.516576+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:17:53.483873+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:17:53.483873+0000","last_clean_scrub_stamp":"2023-11-26T07:17:53.483873+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T13:18:15.521322+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"20'1","reported_seq":13,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-26T07:17:56.715464+0000","last_change":"2023-11-26T07:17:53.513529+0000","last_active":"2023-11-26T07:17:56.715464+0000","last_peered":"2023-11-26T07:17:56.715464+0000","last_clean":"2023-11-26T07:17:56.715464+0000","last_became_active":"2023-11-26T07:17:53.513125+0000","last_became_peered":"2023-11-26T07:17:53.513125+0000","last_unstale":"2023-11-26T07:17:56.715464+0000","last_undegraded":"2023-11-26T07:17:56.715464+0000","last_fullsized":"2023-11-26T07:17:56.715464+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:17:52.477276+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:17:52.477276+0000","last_clean_scrub_stamp":"2023-11-26T07:17:52.477276+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T09:53:28.237406+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-26T07:17:56.641191+0000","last_change":"2023-11-26T07:17:54.525518+0000","last_active":"2023-11-26T07:17:56.641191+0000","last_peered":"2023-11-26T07:17:56.641191+0000","last_clean":"2023-11-26T07:17:56.641191+0000","last_became_active":"2023-11-26T07:17:54.524972+0000","last_became_peered":"2023-11-26T07:17:54.524972+0000","last_unstale":"2023-11-26T07:17:56.641191+0000","last_undegraded":"2023-11-26T07:17:56.641191+0000","last_fullsized":"2023-11-26T07:17:56.641191+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:17:53.483873+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:17:53.483873+0000","last_clean_scrub_stamp":"2023-11-26T07:17:53.483873+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T15:49:39.889795+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.7","version":"19'2","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-26T07:17:56.640410+0000","last_change":"2023-11-26T07:17:53.523424+0000","last_active":"2023-11-26T07:17:56.640410+0000","last_peered":"2023-11-26T07:17:56.640410+0000","last_clean":"2023-11-26T07:17:56.640410+0000","last_became_active":"2023-11-26T07:17:53.523139+0000","last_became_peered":"2023-11-26T07:17:53.523139+0000","last_unstale":"2023-11-26T07:17:56.640410+0000","last_undegraded":"2023-11-26T07:17:56.640410+0000","last_fullsized":"2023-11-26T07:17:56.640410+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:17:52.477276+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:17:52.477276+0000","last_clean_scrub_stamp":"2023-11-26T07:17:52.477276+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T09:18:47.261084+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-26T07:17:56.069026+0000","last_change":"2023-11-26T07:17:54.523889+0000","last_active":"2023-11-26T07:17:56.069026+0000","last_peered":"2023-11-26T07:17:56.069026+0000","last_clean":"2023-11-26T07:17:56.069026+0000","last_became_active":"2023-11-26T07:17:54.523610+0000","last_became_peered":"2023-11-26T07:17:54.523610+0000","last_unstale":"2023-11-26T07:17:56.069026+0000","last_undegraded":"2023-11-26T07:17:56.069026+0000","last_fullsized":"2023-11-26T07:17:56.069026+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:17:53.483873+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:17:53.483873+0000","last_clean_scrub_stamp":"2023-11-26T07:17:53.483873+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T14:45:51.557148+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-26T07:17:55.763603+0000","last_change":"2023-11-26T07:17:53.521942+0000","last_active":"2023-11-26T07:17:55.763603+0000","last_peered":"2023-11-26T07:17:55.763603+0000","last_clean":"2023-11-26T07:17:55.763603+0000","last_became_active":"2023-11-26T07:17:53.521487+0000","last_became_peered":"2023-11-26T07:17:53.521487+0000","last_unstale":"2023-11-26T07:17:55.763603+0000","last_undegraded":"2023-11-26T07:17:55.763603+0000","last_fullsized":"2023-11-26T07:17:55.763603+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:17:52.477276+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:17:52.477276+0000","last_clean_scrub_stamp":"2023-11-26T07:17:52.477276+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T09:07:52.941378+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-26T07:17:56.640134+0000","last_change":"2023-11-26T07:17:54.516390+0000","last_active":"2023-11-26T07:17:56.640134+0000","last_peered":"2023-11-26T07:17:56.640134+0000","last_clean":"2023-11-26T07:17:56.640134+0000","last_became_active":"2023-11-26T07:17:54.515967+0000","last_became_peered":"2023-11-26T07:17:54.515967+0000","last_unstale":"2023-11-26T07:17:56.640134+0000","last_undegraded":"2023-11-26T07:17:56.640134+0000","last_fullsized":"2023-11-26T07:17:56.640134+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:17:53.483873+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:17:53.483873+0000","last_clean_scrub_stamp":"2023-11-26T07:17:53.483873+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T17:42:29.554668+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.9","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-26T07:17:55.516770+0000","last_change":"2023-11-26T07:17:53.520545+0000","last_active":"2023-11-26T07:17:55.516770+0000","last_peered":"2023-11-26T07:17:55.516770+0000","last_clean":"2023-11-26T07:17:55.516770+0000","last_became_active":"2023-11-26T07:17:53.520076+0000","last_became_peered":"2023-11-26T07:17:53.520076+0000","last_unstale":"2023-11-26T07:17:55.516770+0000","last_undegraded":"2023-11-26T07:17:55.516770+0000","last_fullsized":"2023-11-26T07:17:55.516770+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:17:52.477276+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:17:52.477276+0000","last_clean_scrub_stamp":"2023-11-26T07:17:52.477276+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T12:51:16.332936+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-26T07:17:56.641019+0000","last_change":"2023-11-26T07:17:54.512904+0000","last_active":"2023-11-26T07:17:56.641019+0000","last_peered":"2023-11-26T07:17:56.641019+0000","last_clean":"2023-11-26T07:17:56.641019+0000","last_became_active":"2023-11-26T07:17:54.512619+0000","last_became_peered":"2023-11-26T07:17:54.512619+0000","last_unstale":"2023-11-26T07:17:56.641019+0000","last_undegraded":"2023-11-26T07:17:56.641019+0000","last_fullsized":"2023-11-26T07:17:56.641019+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:17:53.483873+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:17:53.483873+0000","last_clean_scrub_stamp":"2023-11-26T07:17:53.483873+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T14:49:27.883374+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.a","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-26T07:17:55.516544+0000","last_change":"2023-11-26T07:17:53.523572+0000","last_active":"2023-11-26T07:17:55.516544+0000","last_peered":"2023-11-26T07:17:55.516544+0000","last_clean":"2023-11-26T07:17:55.516544+0000","last_became_active":"2023-11-26T07:17:53.523266+0000","last_became_peered":"2023-11-26T07:17:53.523266+0000","last_unstale":"2023-11-26T07:17:55.516544+0000","last_undegraded":"2023-11-26T07:17:55.516544+0000","last_fullsized":"2023-11-26T07:17:55.516544+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:17:52.477276+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:17:52.477276+0000","last_clean_scrub_stamp":"2023-11-26T07:17:52.477276+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T13:59:52.977195+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-26T07:17:56.640963+0000","last_change":"2023-11-26T07:17:54.515437+0000","last_active":"2023-11-26T07:17:56.640963+0000","last_peered":"2023-11-26T07:17:56.640963+0000","last_clean":"2023-11-26T07:17:56.640963+0000","last_became_active":"2023-11-26T07:17:54.515127+0000","last_became_peered":"2023-11-26T07:17:54.515127+0000","last_unstale":"2023-11-26T07:17:56.640963+0000","last_undegraded":"2023-11-26T07:17:56.640963+0000","last_fullsized":"2023-11-26T07:17:56.640963+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:17:53.483873+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:17:53.483873+0000","last_clean_scrub_stamp":"2023-11-26T07:17:53.483873+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T14:45:51.557148+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.b","version":"19'2","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-26T07:17:55.763968+0000","last_change":"2023-11-26T07:17:53.525249+0000","last_active":"2023-11-26T07:17:55.763968+0000","last_peered":"2023-11-26T07:17:55.763968+0000","last_clean":"2023-11-26T07:17:55.763968+0000","last_became_active":"2023-11-26T07:17:53.524957+0000","last_became_peered":"2023-11-26T07:17:53.524957+0000","last_unstale":"2023-11-26T07:17:55.763968+0000","last_undegraded":"2023-11-26T07:17:55.763968+0000","last_fullsized":"2023-11-26T07:17:55.763968+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:17:52.477276+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:17:52.477276+0000","last_clean_scrub_stamp":"2023-11-26T07:17:52.477276+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T19:07:41.542569+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-26T07:17:56.639556+0000","last_change":"2023-11-26T07:17:54.521923+0000","last_active":"2023-11-26T07:17:56.639556+0000","last_peered":"2023-11-26T07:17:56.639556+0000","last_clean":"2023-11-26T07:17:56.639556+0000","last_became_active":"2023-11-26T07:17:54.521503+0000","last_became_peered":"2023-11-26T07:17:54.521503+0000","last_unstale":"2023-11-26T07:17:56.639556+0000","last_undegraded":"2023-11-26T07:17:56.639556+0000","last_fullsized":"2023-11-26T07:17:56.639556+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:17:53.483873+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:17:53.483873+0000","last_clean_scrub_stamp":"2023-11-26T07:17:53.483873+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T16:07:46.594040+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-26T07:17:55.538848+0000","last_change":"2023-11-26T07:17:54.526945+0000","last_active":"2023-11-26T07:17:55.538848+0000","last_peered":"2023-11-26T07:17:55.538848+0000","last_clean":"2023-11-26T07:17:55.538848+0000","last_became_active":"2023-11-26T07:17:54.526110+0000","last_became_peered":"2023-11-26T07:17:54.526110+0000","last_unstale":"2023-11-26T07:17:55.538848+0000","last_undegraded":"2023-11-26T07:17:55.538848+0000","last_fullsized":"2023-11-26T07:17:55.538848+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:17:53.483873+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:17:53.483873+0000","last_clean_scrub_stamp":"2023-11-26T07:17:53.483873+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T10:18:52.982852+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1c","version":"19'2","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-26T07:17:55.763845+0000","last_change":"2023-11-26T07:17:53.524780+0000","last_active":"2023-11-26T07:17:55.763845+0000","last_peered":"2023-11-26T07:17:55.763845+0000","last_clean":"2023-11-26T07:17:55.763845+0000","last_became_active":"2023-11-26T07:17:53.524429+0000","last_became_peered":"2023-11-26T07:17:53.524429+0000","last_unstale":"2023-11-26T07:17:55.763845+0000","last_undegraded":"2023-11-26T07:17:55.763845+0000","last_fullsized":"2023-11-26T07:17:55.763845+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:17:52.477276+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:17:52.477276+0000","last_clean_scrub_stamp":"2023-11-26T07:17:52.477276+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T09:53:28.237406+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-26T07:17:55.539011+0000","last_change":"2023-11-26T07:17:54.523997+0000","last_active":"2023-11-26T07:17:55.539011+0000","last_peered":"2023-11-26T07:17:55.539011+0000","last_clean":"2023-11-26T07:17:55.539011+0000","last_became_active":"2023-11-26T07:17:54.523690+0000","last_became_peered":"2023-11-26T07:17:54.523690+0000","last_unstale":"2023-11-26T07:17:55.539011+0000","last_undegraded":"2023-11-26T07:17:55.539011+0000","last_fullsized":"2023-11-26T07:17:55.539011+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:17:53.483873+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:17:53.483873+0000","last_clean_scrub_stamp":"2023-11-26T07:17:53.483873+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T11:28:10.566680+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1d","version":"19'2","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-26T07:17:55.764098+0000","last_change":"2023-11-26T07:17:53.526233+0000","last_active":"2023-11-26T07:17:55.764098+0000","last_peered":"2023-11-26T07:17:55.764098+0000","last_clean":"2023-11-26T07:17:55.764098+0000","last_became_active":"2023-11-26T07:17:53.525918+0000","last_became_peered":"2023-11-26T07:17:53.525918+0000","last_unstale":"2023-11-26T07:17:55.764098+0000","last_undegraded":"2023-11-26T07:17:55.764098+0000","last_fullsized":"2023-11-26T07:17:55.764098+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:17:52.477276+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:17:52.477276+0000","last_clean_scrub_stamp":"2023-11-26T07:17:52.477276+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T15:18:16.303185+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-26T07:17:56.068892+0000","last_change":"2023-11-26T07:17:54.528760+0000","last_active":"2023-11-26T07:17:56.068892+0000","last_peered":"2023-11-26T07:17:56.068892+0000","last_clean":"2023-11-26T07:17:56.068892+0000","last_became_active":"2023-11-26T07:17:54.528456+0000","last_became_peered":"2023-11-26T07:17:54.528456+0000","last_unstale":"2023-11-26T07:17:56.068892+0000","last_undegraded":"2023-11-26T07:17:56.068892+0000","last_fullsized":"2023-11-26T07:17:56.068892+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:17:53.483873+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:17:53.483873+0000","last_clean_scrub_stamp":"2023-11-26T07:17:53.483873+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T15:18:17.309782+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1e","version":"19'1","reported_seq":13,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-26T07:17:56.641018+0000","last_change":"2023-11-26T07:17:53.523797+0000","last_active":"2023-11-26T07:17:56.641018+0000","last_peered":"2023-11-26T07:17:56.641018+0000","last_clean":"2023-11-26T07:17:56.641018+0000","last_became_active":"2023-11-26T07:17:53.523521+0000","last_became_peered":"2023-11-26T07:17:53.523521+0000","last_unstale":"2023-11-26T07:17:56.641018+0000","last_undegraded":"2023-11-26T07:17:56.641018+0000","last_fullsized":"2023-11-26T07:17:56.641018+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:17:52.477276+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:17:52.477276+0000","last_clean_scrub_stamp":"2023-11-26T07:17:52.477276+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T09:53:28.237406+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-26T07:17:56.640929+0000","last_change":"2023-11-26T07:17:54.517290+0000","last_active":"2023-11-26T07:17:56.640929+0000","last_peered":"2023-11-26T07:17:56.640929+0000","last_clean":"2023-11-26T07:17:56.640929+0000","last_became_active":"2023-11-26T07:17:54.516998+0000","last_became_peered":"2023-11-26T07:17:54.516998+0000","last_unstale":"2023-11-26T07:17:56.640929+0000","last_undegraded":"2023-11-26T07:17:56.640929+0000","last_fullsized":"2023-11-26T07:17:56.640929+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:17:53.483873+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:17:53.483873+0000","last_clean_scrub_stamp":"2023-11-26T07:17:53.483873+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T13:18:15.521322+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1f","version":"19'4","reported_seq":16,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-26T07:17:56.069044+0000","last_change":"2023-11-26T07:17:53.524902+0000","last_active":"2023-11-26T07:17:56.069044+0000","last_peered":"2023-11-26T07:17:56.069044+0000","last_clean":"2023-11-26T07:17:56.069044+0000","last_became_active":"2023-11-26T07:17:53.524530+0000","last_became_peered":"2023-11-26T07:17:53.524530+0000","last_unstale":"2023-11-26T07:17:56.069044+0000","last_undegraded":"2023-11-26T07:17:56.069044+0000","last_fullsized":"2023-11-26T07:17:56.069044+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:17:52.477276+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:17:52.477276+0000","last_clean_scrub_stamp":"2023-11-26T07:17:52.477276+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T09:18:47.261084+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":39,"ondisk_log_size":39,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574853,"num_pgs":30,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27808,"kb_used_data":784,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344032,"statfs":{"total":96636764160,"available":96608288768,"internally_reserved":0,"allocated":802816,"data_stored":661860,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.76200000000000001}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.60999999999999999}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.80900000000000005}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.73299999999999998}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.57999999999999996}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.65200000000000002}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.83399999999999996}]}]},{"osd":6,"up_from":13,"seq":55834574855,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27236,"kb_used_data":212,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344604,"statfs":{"total":96636764160,"available":96608874496,"internally_reserved":0,"allocated":217088,"data_stored":72522,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.7549999999999999}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":5.4240000000000004}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.8520000000000001}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":5.4390000000000001}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.47199999999999998}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.45600000000000002}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.57799999999999996}]}]},{"osd":5,"up_from":13,"seq":55834574853,"num_pgs":30,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27228,"kb_used_data":204,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344612,"statfs":{"total":96636764160,"available":96608882688,"internally_reserved":0,"allocated":208896,"data_stored":71000,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.69999999999999996}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.75900000000000001}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.85699999999999998}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.78200000000000003}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.748}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.81299999999999994}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.89500000000000002}]}]},{"osd":4,"up_from":13,"seq":55834574855,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27236,"kb_used_data":212,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344604,"statfs":{"total":96636764160,"available":96608874496,"internally_reserved":0,"allocated":217088,"data_stored":71036,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":4.1689999999999996}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.66100000000000003}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":2.7850000000000001}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.591}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.496}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":2.8330000000000002}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":4.3689999999999998}]}]},{"osd":3,"up_from":13,"seq":55834574856,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27236,"kb_used_data":212,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344604,"statfs":{"total":96636764160,"available":96608874496,"internally_reserved":0,"allocated":217088,"data_stored":72522,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1592,"internal_metadata":27654600},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.33200000000000002}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.36499999999999999}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.40300000000000002}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.67200000000000004}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.65500000000000003}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.64200000000000002}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.70099999999999996}]}]},{"osd":2,"up_from":13,"seq":55834574856,"num_pgs":13,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27224,"kb_used_data":200,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344616,"statfs":{"total":96636764160,"available":96608886784,"internally_reserved":0,"allocated":204800,"data_stored":70910,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":4.7030000000000003}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":4.2690000000000001}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.70499999999999996}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":4.4269999999999996}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.627}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":4.0449999999999999}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":4.1219999999999999}]}]},{"osd":1,"up_from":13,"seq":55834574855,"num_pgs":19,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27232,"kb_used_data":208,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344608,"statfs":{"total":96636764160,"available":96608878592,"internally_reserved":0,"allocated":212992,"data_stored":71070,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.502}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.40600000000000003}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.59799999999999998}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.69299999999999995}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.58199999999999996}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.66200000000000003}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.55100000000000005}]}]},{"osd":0,"up_from":13,"seq":55834574855,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27812,"kb_used_data":792,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344028,"statfs":{"total":96636764160,"available":96608284672,"internally_reserved":0,"allocated":811008,"data_stored":661916,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.28000000000000003}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.41999999999999998}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.48199999999999998}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.69199999999999995}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.73299999999999998}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.83699999999999997}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.89100000000000001}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":638,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":160,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":126,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-11-26T07:18:03.536 INFO:tasks.ceph.ceph_manager.ceph:clean! 2023-11-26T07:18:03.536 INFO:tasks.ceph:Waiting until ceph cluster ceph is healthy... 2023-11-26T07:18:03.536 INFO:tasks.ceph.ceph_manager.ceph:wait_until_healthy 2023-11-26T07:18:03.536 DEBUG:teuthology.orchestra.run.smithi143:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph health --format=json 2023-11-26T07:18:03.713 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:03.710+0000 7fd6e9cbb640 1 Processor -- start 2023-11-26T07:18:03.713 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:03.710+0000 7fd6e9cbb640 1 -- start start 2023-11-26T07:18:03.715 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:03.710+0000 7fd6e9cbb640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fd6e4145fa0 0x7fd6e4072420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:18:03.715 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:03.710+0000 7fd6e9cbb640 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7fd6e406a490 0x7fd6e406a900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:18:03.715 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:03.710+0000 7fd6e9cbb640 1 -- --> v1:172.21.15.143:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd6e4146380 con 0x7fd6e406b190 2023-11-26T07:18:03.715 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:03.710+0000 7fd6e9cbb640 1 -- --> v2:172.21.15.143:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd6e4071480 con 0x7fd6e406a490 2023-11-26T07:18:03.715 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:03.710+0000 7fd6e9cbb640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd6e4072000 con 0x7fd6e4145fa0 2023-11-26T07:18:03.715 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:03.710+0000 7fd6e3fff640 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7fd6e406a490 0x7fd6e406a900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:18:03.716 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:03.710+0000 7fd6e3fff640 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7fd6e406a490 0x7fd6e406a900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:46806/0 (socket says 172.21.15.143:46806) 2023-11-26T07:18:03.716 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:03.710+0000 7fd6e2ffd640 1 --1- >> v1:172.21.15.143:6789/0 conn(0x7fd6e406b190 0x7fd6e4146530 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.143:6789/0 says I am v1:172.21.15.143:40248/0 (socket says 172.21.15.143:40248) 2023-11-26T07:18:03.716 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:03.710+0000 7fd6e3fff640 1 -- 172.21.15.143:0/1370094383 learned_addr learned my addr 172.21.15.143:0/1370094383 (peer_addr_for_me v2:172.21.15.143:0/0) 2023-11-26T07:18:03.716 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:03.710+0000 7fd6e37fe640 1 --2- 172.21.15.143:0/1370094383 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fd6e4145fa0 0x7fd6e4072420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:18:03.716 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:03.710+0000 7fd6e3fff640 1 -- 172.21.15.143:0/1370094383 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fd6e4145fa0 msgr2=0x7fd6e4072420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-26T07:18:03.716 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:03.710+0000 7fd6e3fff640 1 --2- 172.21.15.143:0/1370094383 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fd6e4145fa0 0x7fd6e4072420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:18:03.716 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:03.710+0000 7fd6e3fff640 1 -- 172.21.15.143:0/1370094383 >> v1:172.21.15.143:6789/0 conn(0x7fd6e406b190 legacy=0x7fd6e4146530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:18:03.716 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:03.710+0000 7fd6e3fff640 1 -- 172.21.15.143:0/1370094383 --> v2:172.21.15.143:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd6e4146c40 con 0x7fd6e406a490 2023-11-26T07:18:03.718 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:03.714+0000 7fd6e3fff640 1 --2- 172.21.15.143:0/1370094383 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fd6e406a490 0x7fd6e406a900 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7fd6dc009e50 tx=0x7fd6dc01c8a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2eb42eb1b3fb0703 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:18:03.718 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:03.714+0000 7fd6e27fc640 1 -- 172.21.15.143:0/1370094383 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd6dc0a56c0 con 0x7fd6e406a490 2023-11-26T07:18:03.718 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:03.714+0000 7fd6e27fc640 1 -- 172.21.15.143:0/1370094383 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd6dc0a5820 con 0x7fd6e406a490 2023-11-26T07:18:03.718 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:03.714+0000 7fd6e27fc640 1 -- 172.21.15.143:0/1370094383 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd6dc0a5bb0 con 0x7fd6e406a490 2023-11-26T07:18:03.719 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:03.714+0000 7fd6e9cbb640 1 -- 172.21.15.143:0/1370094383 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fd6e406a490 msgr2=0x7fd6e406a900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:18:03.719 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:03.714+0000 7fd6e9cbb640 1 --2- 172.21.15.143:0/1370094383 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fd6e406a490 0x7fd6e406a900 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7fd6dc009e50 tx=0x7fd6dc01c8a0 comp rx=0 tx=0).stop 2023-11-26T07:18:03.719 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:03.714+0000 7fd6e9cbb640 1 -- 172.21.15.143:0/1370094383 shutdown_connections 2023-11-26T07:18:03.719 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:03.714+0000 7fd6e9cbb640 1 --2- 172.21.15.143:0/1370094383 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fd6e406a490 0x7fd6e406a900 unknown :-1 s=CLOSED pgs=122 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:18:03.720 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:03.714+0000 7fd6e9cbb640 1 --2- 172.21.15.143:0/1370094383 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fd6e4145fa0 0x7fd6e4072420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:18:03.720 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:03.714+0000 7fd6e9cbb640 1 -- 172.21.15.143:0/1370094383 >> 172.21.15.143:0/1370094383 conn(0x7fd6e40cff00 msgr2=0x7fd6e40d0310 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:18:03.720 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:03.714+0000 7fd6e9cbb640 1 -- 172.21.15.143:0/1370094383 shutdown_connections 2023-11-26T07:18:03.720 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:03.714+0000 7fd6e9cbb640 1 -- 172.21.15.143:0/1370094383 wait complete. 2023-11-26T07:18:03.720 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:03.718+0000 7fd6e9cbb640 1 Processor -- start 2023-11-26T07:18:03.721 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:03.718+0000 7fd6e9cbb640 1 -- start start 2023-11-26T07:18:03.722 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:03.718+0000 7fd6e9cbb640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fd6e406a490 0x7fd6e413ef60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:18:03.722 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:03.718+0000 7fd6e9cbb640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fd6e406b190 0x7fd6e4140db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:18:03.723 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:03.718+0000 7fd6e9cbb640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fd6e4145fa0 0x7fd6e41f2310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:18:03.723 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:03.718+0000 7fd6e9cbb640 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd6e41412f0 con 0x7fd6e4145fa0 2023-11-26T07:18:03.723 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:03.718+0000 7fd6e9cbb640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd6e4141590 con 0x7fd6e406b190 2023-11-26T07:18:03.723 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:03.718+0000 7fd6e9cbb640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd6e4141870 con 0x7fd6e406a490 2023-11-26T07:18:03.723 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:03.718+0000 7fd6e3fff640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fd6e4145fa0 0x7fd6e41f2310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:18:03.723 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:03.718+0000 7fd6e3fff640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fd6e4145fa0 0x7fd6e41f2310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:46816/0 (socket says 172.21.15.143:46816) 2023-11-26T07:18:03.723 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:03.718+0000 7fd6e3fff640 1 -- 172.21.15.143:0/4136684108 learned_addr learned my addr 172.21.15.143:0/4136684108 (peer_addr_for_me v2:172.21.15.143:0/0) 2023-11-26T07:18:03.723 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:03.718+0000 7fd6e37fe640 1 --2- 172.21.15.143:0/4136684108 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fd6e406a490 0x7fd6e413ef60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:18:03.723 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:03.718+0000 7fd6e2ffd640 1 --2- 172.21.15.143:0/4136684108 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fd6e406b190 0x7fd6e4140db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:18:03.723 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:03.718+0000 7fd6e3fff640 1 -- 172.21.15.143:0/4136684108 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fd6e406a490 msgr2=0x7fd6e413ef60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:18:03.723 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:03.718+0000 7fd6e3fff640 1 --2- 172.21.15.143:0/4136684108 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fd6e406a490 0x7fd6e413ef60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:18:03.723 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:03.718+0000 7fd6e3fff640 1 -- 172.21.15.143:0/4136684108 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fd6e406b190 msgr2=0x7fd6e4140db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:18:03.724 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:03.718+0000 7fd6e3fff640 1 --2- 172.21.15.143:0/4136684108 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fd6e406b190 0x7fd6e4140db0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:18:03.724 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:03.718+0000 7fd6e3fff640 1 -- 172.21.15.143:0/4136684108 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd6dc009b30 con 0x7fd6e4145fa0 2023-11-26T07:18:03.724 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:03.718+0000 7fd6e3fff640 1 --2- 172.21.15.143:0/4136684108 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fd6e4145fa0 0x7fd6e41f2310 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7fd6dc0af040 tx=0x7fd6dc019640 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:18:03.725 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:03.718+0000 7fd6cbfff640 1 -- 172.21.15.143:0/4136684108 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd6dc0a56c0 con 0x7fd6e4145fa0 2023-11-26T07:18:03.725 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:03.718+0000 7fd6cbfff640 1 -- 172.21.15.143:0/4136684108 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd6dc007bf0 con 0x7fd6e4145fa0 2023-11-26T07:18:03.725 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:03.718+0000 7fd6cbfff640 1 -- 172.21.15.143:0/4136684108 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd6dc009510 con 0x7fd6e4145fa0 2023-11-26T07:18:03.725 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:03.718+0000 7fd6e9cbb640 1 -- 172.21.15.143:0/4136684108 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd6e4141b50 con 0x7fd6e4145fa0 2023-11-26T07:18:03.725 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:03.718+0000 7fd6e9cbb640 1 -- 172.21.15.143:0/4136684108 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd6e4142110 con 0x7fd6e4145fa0 2023-11-26T07:18:03.725 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:03.718+0000 7fd6e9cbb640 1 -- 172.21.15.143:0/4136684108 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd6e4067680 con 0x7fd6e4145fa0 2023-11-26T07:18:03.725 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:03.722+0000 7fd6cbfff640 1 -- 172.21.15.143:0/4136684108 <== mon.0 v2:172.21.15.143:3300/0 4 ==== mgrmap(e 5) v1 ==== 85852+0+0 (secure 0 0 0) 0x7fd6dc0b8470 con 0x7fd6e4145fa0 2023-11-26T07:18:03.726 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:03.722+0000 7fd6cbfff640 1 --2- 172.21.15.143:0/4136684108 >> [v2:172.21.15.143:6832/2150557544,v1:172.21.15.143:6833/2150557544] conn(0x7fd6d0066470 0x7fd6d0068930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:18:03.726 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:03.722+0000 7fd6cbfff640 1 -- 172.21.15.143:0/4136684108 <== mon.0 v2:172.21.15.143:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fd6dc11f640 con 0x7fd6e4145fa0 2023-11-26T07:18:03.726 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:03.722+0000 7fd6e37fe640 1 --2- 172.21.15.143:0/4136684108 >> [v2:172.21.15.143:6832/2150557544,v1:172.21.15.143:6833/2150557544] conn(0x7fd6d0066470 0x7fd6d0068930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:18:03.726 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:03.722+0000 7fd6e37fe640 1 --2- 172.21.15.143:0/4136684108 >> [v2:172.21.15.143:6832/2150557544,v1:172.21.15.143:6833/2150557544] conn(0x7fd6d0066470 0x7fd6d0068930 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7fd6d400b9d0 tx=0x7fd6d4008040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:18:03.730 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:03.726+0000 7fd6cbfff640 1 -- 172.21.15.143:0/4136684108 <== mon.0 v2:172.21.15.143:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172649 (secure 0 0 0) 0x7fd6dc0ecad0 con 0x7fd6e4145fa0 2023-11-26T07:18:03.914 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:03.910+0000 7fd6e9cbb640 1 -- 172.21.15.143:0/4136684108 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "health", "format": "json"} v 0) v1 -- 0x7fd6e4146c40 con 0x7fd6e4145fa0 2023-11-26T07:18:03.914 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:03.910+0000 7fd6cbfff640 1 -- 172.21.15.143:0/4136684108 <== mon.0 v2:172.21.15.143:3300/0 7 ==== mon_command_ack([{"prefix": "health", "format": "json"}]=0 v0) v1 ==== 72+0+46 (secure 0 0 0) 0x7fd6e4146c40 con 0x7fd6e4145fa0 2023-11-26T07:18:03.914 INFO:teuthology.orchestra.run.smithi143.stdout: 2023-11-26T07:18:03.914 INFO:teuthology.orchestra.run.smithi143.stdout:{"status":"HEALTH_OK","checks":{},"mutes":[]} 2023-11-26T07:18:03.916 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:03.910+0000 7fd6e9cbb640 1 -- 172.21.15.143:0/4136684108 >> [v2:172.21.15.143:6832/2150557544,v1:172.21.15.143:6833/2150557544] conn(0x7fd6d0066470 msgr2=0x7fd6d0068930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:18:03.916 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:03.910+0000 7fd6e9cbb640 1 --2- 172.21.15.143:0/4136684108 >> [v2:172.21.15.143:6832/2150557544,v1:172.21.15.143:6833/2150557544] conn(0x7fd6d0066470 0x7fd6d0068930 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7fd6d400b9d0 tx=0x7fd6d4008040 comp rx=0 tx=0).stop 2023-11-26T07:18:03.916 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:03.910+0000 7fd6e9cbb640 1 -- 172.21.15.143:0/4136684108 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fd6e4145fa0 msgr2=0x7fd6e41f2310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:18:03.916 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:03.910+0000 7fd6e9cbb640 1 --2- 172.21.15.143:0/4136684108 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fd6e4145fa0 0x7fd6e41f2310 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7fd6dc0af040 tx=0x7fd6dc019640 comp rx=0 tx=0).stop 2023-11-26T07:18:03.916 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:03.910+0000 7fd6e9cbb640 1 -- 172.21.15.143:0/4136684108 shutdown_connections 2023-11-26T07:18:03.916 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:03.910+0000 7fd6e9cbb640 1 --2- 172.21.15.143:0/4136684108 >> [v2:172.21.15.143:6832/2150557544,v1:172.21.15.143:6833/2150557544] conn(0x7fd6d0066470 0x7fd6d0068930 unknown :-1 s=CLOSED pgs=103 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:18:03.916 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:03.910+0000 7fd6e9cbb640 1 --2- 172.21.15.143:0/4136684108 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fd6e4145fa0 0x7fd6e41f2310 unknown :-1 s=CLOSED pgs=123 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:18:03.917 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:03.910+0000 7fd6e9cbb640 1 --2- 172.21.15.143:0/4136684108 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fd6e406b190 0x7fd6e4140db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:18:03.917 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:03.910+0000 7fd6e9cbb640 1 --2- 172.21.15.143:0/4136684108 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fd6e406a490 0x7fd6e413ef60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:18:03.917 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:03.910+0000 7fd6e9cbb640 1 -- 172.21.15.143:0/4136684108 >> 172.21.15.143:0/4136684108 conn(0x7fd6e40cff00 msgr2=0x7fd6e40ac8e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:18:03.917 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:03.910+0000 7fd6e9cbb640 1 -- 172.21.15.143:0/4136684108 shutdown_connections 2023-11-26T07:18:03.917 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:03.910+0000 7fd6e9cbb640 1 -- 172.21.15.143:0/4136684108 wait complete. 2023-11-26T07:18:03.935 INFO:tasks.ceph.ceph_manager.ceph:wait_until_healthy done 2023-11-26T07:18:03.935 DEBUG:teuthology.orchestra.run.smithi143:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-11-26T07:18:04.072 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:04.066+0000 7f33e293e640 1 Processor -- start 2023-11-26T07:18:04.072 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:04.066+0000 7f33e293e640 1 -- start start 2023-11-26T07:18:04.074 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:04.070+0000 7f33e293e640 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7f33dc0cf990 0x7f33dc15efa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:18:04.074 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:04.070+0000 7f33e293e640 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7f33dc15f4e0 0x7f33dc0cd7b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:18:04.074 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:04.070+0000 7f33e293e640 1 -- --> v1:172.21.15.188:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f33dc06c520 con 0x7f33dc13e290 2023-11-26T07:18:04.074 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:04.070+0000 7f33e293e640 1 -- --> v2:172.21.15.143:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f33dc06c800 con 0x7f33dc15f4e0 2023-11-26T07:18:04.074 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:04.070+0000 7f33e293e640 1 -- --> v2:172.21.15.188:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f33dc06cae0 con 0x7f33dc0cf990 2023-11-26T07:18:04.075 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:04.070+0000 7f33e0edb640 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7f33dc15f4e0 0x7f33dc0cd7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:18:04.075 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:04.070+0000 7f33e0edb640 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7f33dc15f4e0 0x7f33dc0cd7b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:46824/0 (socket says 172.21.15.143:46824) 2023-11-26T07:18:04.075 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:04.070+0000 7f33e0edb640 1 -- 172.21.15.143:0/1787840956 learned_addr learned my addr 172.21.15.143:0/1787840956 (peer_addr_for_me v2:172.21.15.143:0/0) 2023-11-26T07:18:04.075 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:04.070+0000 7f33db7fe640 1 --2- 172.21.15.143:0/1787840956 >> v2:172.21.15.188:3300/0 conn(0x7f33dc0cf990 0x7f33dc15efa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:18:04.075 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:04.070+0000 7f33e0edb640 1 -- 172.21.15.143:0/1787840956 >> v2:172.21.15.188:3300/0 conn(0x7f33dc0cf990 msgr2=0x7f33dc15efa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-26T07:18:04.075 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:04.070+0000 7f33e0edb640 1 --2- 172.21.15.143:0/1787840956 >> v2:172.21.15.188:3300/0 conn(0x7f33dc0cf990 0x7f33dc15efa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:18:04.075 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:04.070+0000 7f33e0edb640 1 -- 172.21.15.143:0/1787840956 >> v1:172.21.15.188:6789/0 conn(0x7f33dc13e290 legacy=0x7f33dc0cf280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:18:04.075 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:04.070+0000 7f33e0edb640 1 -- 172.21.15.143:0/1787840956 --> v2:172.21.15.143:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f33dc06cdc0 con 0x7f33dc15f4e0 2023-11-26T07:18:04.077 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:04.074+0000 7f33e0edb640 1 --2- 172.21.15.143:0/1787840956 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f33dc15f4e0 0x7f33dc0cd7b0 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7f33dc0cfe40 tx=0x7f33d401c9a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e82256b0ac4d20c4 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:18:04.077 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:04.074+0000 7f33daffd640 1 -- 172.21.15.143:0/1787840956 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f33dc15fb60 con 0x7f33dc15f4e0 2023-11-26T07:18:04.077 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:04.074+0000 7f33daffd640 1 -- 172.21.15.143:0/1787840956 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f33d40a5880 con 0x7f33dc15f4e0 2023-11-26T07:18:04.078 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:04.074+0000 7f33daffd640 1 -- 172.21.15.143:0/1787840956 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f33d40a5c10 con 0x7f33dc15f4e0 2023-11-26T07:18:04.078 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:04.074+0000 7f33e293e640 1 -- 172.21.15.143:0/1787840956 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f33dc15f4e0 msgr2=0x7f33dc0cd7b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:18:04.078 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:04.074+0000 7f33e293e640 1 --2- 172.21.15.143:0/1787840956 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f33dc15f4e0 0x7f33dc0cd7b0 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7f33dc0cfe40 tx=0x7f33d401c9a0 comp rx=0 tx=0).stop 2023-11-26T07:18:04.079 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:04.074+0000 7f33e293e640 1 -- 172.21.15.143:0/1787840956 shutdown_connections 2023-11-26T07:18:04.079 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:04.074+0000 7f33e293e640 1 --2- 172.21.15.143:0/1787840956 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f33dc15f4e0 0x7f33dc0cd7b0 unknown :-1 s=CLOSED pgs=124 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:18:04.079 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:04.074+0000 7f33e293e640 1 --2- 172.21.15.143:0/1787840956 >> v2:172.21.15.188:3300/0 conn(0x7f33dc0cf990 0x7f33dc15efa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:18:04.079 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:04.074+0000 7f33e293e640 1 -- 172.21.15.143:0/1787840956 >> 172.21.15.143:0/1787840956 conn(0x7f33dc06a890 msgr2=0x7f33dc06aca0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:18:04.079 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:04.074+0000 7f33e293e640 1 -- 172.21.15.143:0/1787840956 shutdown_connections 2023-11-26T07:18:04.079 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:04.074+0000 7f33e293e640 1 -- 172.21.15.143:0/1787840956 wait complete. 2023-11-26T07:18:04.079 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:04.074+0000 7f33e293e640 1 Processor -- start 2023-11-26T07:18:04.080 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:04.074+0000 7f33e293e640 1 -- start start 2023-11-26T07:18:04.082 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:04.078+0000 7f33e293e640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f33dc0cf990 0x7f33dc15a1f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:18:04.082 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:04.078+0000 7f33e293e640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f33dc13e290 0x7f33dc1f4fd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:18:04.082 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:04.078+0000 7f33e293e640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f33dc15f4e0 0x7f33dc1f5510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:18:04.083 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:04.078+0000 7f33e293e640 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f33dc1f5a50 con 0x7f33dc15f4e0 2023-11-26T07:18:04.083 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:04.078+0000 7f33e293e640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f33dc0b15c0 con 0x7f33dc0cf990 2023-11-26T07:18:04.083 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:04.078+0000 7f33e293e640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f33dc15b510 con 0x7f33dc13e290 2023-11-26T07:18:04.083 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:04.078+0000 7f33e0edb640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f33dc15f4e0 0x7f33dc1f5510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:18:04.083 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:04.078+0000 7f33e0edb640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f33dc15f4e0 0x7f33dc1f5510 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:46826/0 (socket says 172.21.15.143:46826) 2023-11-26T07:18:04.083 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:04.078+0000 7f33e0edb640 1 -- 172.21.15.143:0/2383855465 learned_addr learned my addr 172.21.15.143:0/2383855465 (peer_addr_for_me v2:172.21.15.143:0/0) 2023-11-26T07:18:04.083 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:04.078+0000 7f33dbfff640 1 --2- 172.21.15.143:0/2383855465 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f33dc0cf990 0x7f33dc15a1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:18:04.083 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:04.078+0000 7f33e0edb640 1 -- 172.21.15.143:0/2383855465 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f33dc13e290 msgr2=0x7f33dc1f4fd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:18:04.083 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:04.078+0000 7f33db7fe640 1 --2- 172.21.15.143:0/2383855465 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f33dc13e290 0x7f33dc1f4fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:18:04.083 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:04.078+0000 7f33e0edb640 1 --2- 172.21.15.143:0/2383855465 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f33dc13e290 0x7f33dc1f4fd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:18:04.083 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:04.078+0000 7f33e0edb640 1 -- 172.21.15.143:0/2383855465 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f33dc0cf990 msgr2=0x7f33dc15a1f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:18:04.084 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:04.078+0000 7f33e0edb640 1 --2- 172.21.15.143:0/2383855465 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f33dc0cf990 0x7f33dc15a1f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:18:04.084 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:04.078+0000 7f33e0edb640 1 -- 172.21.15.143:0/2383855465 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f33d4009b30 con 0x7f33dc15f4e0 2023-11-26T07:18:04.084 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:04.078+0000 7f33dbfff640 1 --2- 172.21.15.143:0/2383855465 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f33dc0cf990 0x7f33dc15a1f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-26T07:18:04.084 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:04.078+0000 7f33e0edb640 1 --2- 172.21.15.143:0/2383855465 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f33dc15f4e0 0x7f33dc1f5510 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7f33d40af040 tx=0x7f33d4019740 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:18:04.084 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:04.078+0000 7f33d8ff9640 1 -- 172.21.15.143:0/2383855465 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f33d40bd070 con 0x7f33dc15f4e0 2023-11-26T07:18:04.084 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:04.078+0000 7f33d8ff9640 1 -- 172.21.15.143:0/2383855465 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f33d40b8470 con 0x7f33dc15f4e0 2023-11-26T07:18:04.084 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:04.078+0000 7f33d8ff9640 1 -- 172.21.15.143:0/2383855465 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f33d40b8840 con 0x7f33dc15f4e0 2023-11-26T07:18:04.084 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:04.078+0000 7f33e293e640 1 -- 172.21.15.143:0/2383855465 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f33dc153eb0 con 0x7f33dc15f4e0 2023-11-26T07:18:04.084 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:04.078+0000 7f33e293e640 1 -- 172.21.15.143:0/2383855465 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f33dc154470 con 0x7f33dc15f4e0 2023-11-26T07:18:04.085 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:04.078+0000 7f33e293e640 1 -- 172.21.15.143:0/2383855465 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f33dc15b680 con 0x7f33dc15f4e0 2023-11-26T07:18:04.085 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:04.078+0000 7f33d8ff9640 1 -- 172.21.15.143:0/2383855465 <== mon.0 v2:172.21.15.143:3300/0 4 ==== mgrmap(e 5) v1 ==== 85852+0+0 (secure 0 0 0) 0x7f33d40b89a0 con 0x7f33dc15f4e0 2023-11-26T07:18:04.085 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:04.082+0000 7f33d8ff9640 1 --2- 172.21.15.143:0/2383855465 >> [v2:172.21.15.143:6832/2150557544,v1:172.21.15.143:6833/2150557544] conn(0x7f33c8066470 0x7f33c8068930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:18:04.085 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:04.082+0000 7f33d8ff9640 1 -- 172.21.15.143:0/2383855465 <== mon.0 v2:172.21.15.143:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f33d411efe0 con 0x7f33dc15f4e0 2023-11-26T07:18:04.085 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:04.082+0000 7f33dbfff640 1 --2- 172.21.15.143:0/2383855465 >> [v2:172.21.15.143:6832/2150557544,v1:172.21.15.143:6833/2150557544] conn(0x7f33c8066470 0x7f33c8068930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:18:04.086 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:04.082+0000 7f33dbfff640 1 --2- 172.21.15.143:0/2383855465 >> [v2:172.21.15.143:6832/2150557544,v1:172.21.15.143:6833/2150557544] conn(0x7f33c8066470 0x7f33c8068930 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7f33cc000f70 tx=0x7f33cc009040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:18:04.089 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:04.086+0000 7f33d8ff9640 1 -- 172.21.15.143:0/2383855465 <== mon.0 v2:172.21.15.143:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172649 (secure 0 0 0) 0x7f33dc15b680 con 0x7f33dc15f4e0 2023-11-26T07:18:04.240 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:04.234+0000 7f33e293e640 1 -- 172.21.15.143:0/2383855465 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f33dc15b680 con 0x7f33dc15f4e0 2023-11-26T07:18:04.242 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:04.238+0000 7f33d8ff9640 1 -- 172.21.15.143:0/2383855465 <== mon.0 v2:172.21.15.143:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v20) v1 ==== 74+0+16541 (secure 0 0 0) 0x7f33dc15b680 con 0x7f33dc15f4e0 2023-11-26T07:18:04.242 INFO:teuthology.orchestra.run.smithi143.stdout: 2023-11-26T07:18:04.242 INFO:teuthology.orchestra.run.smithi143.stdout:{"epoch":20,"fsid":"f5033060-2469-4a07-95fe-2b76f5404bc1","created":"2023-11-26T07:17:28.613663+0000","modified":"2023-11-26T07:17:55.505435+0000","last_up_change":"2023-11-26T07:17:41.157813+0000","last_in_change":"2023-11-26T07:17:31.181658+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-11-26T07:17:41.758597+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2023-11-26T07:17:51.878332+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2023-11-26T07:17:52.871396+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"ebbee2c6-023e-44b9-9f1d-6f7bfd309730","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6817","nonce":639807369},{"type":"v1","addr":"172.21.15.143:6819","nonce":639807369}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6821","nonce":639807369},{"type":"v1","addr":"172.21.15.143:6823","nonce":639807369}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6830","nonce":639807369},{"type":"v1","addr":"172.21.15.143:6831","nonce":639807369}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6825","nonce":639807369},{"type":"v1","addr":"172.21.15.143:6828","nonce":639807369}]},"public_addr":"172.21.15.143:6819/639807369","cluster_addr":"172.21.15.143:6823/639807369","heartbeat_back_addr":"172.21.15.143:6831/639807369","heartbeat_front_addr":"172.21.15.143:6828/639807369","state":["exists","up"]},{"osd":1,"uuid":"fec4b8be-7122-49e9-9a19-5850a0700001","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6816","nonce":1144942530},{"type":"v1","addr":"172.21.15.143:6818","nonce":1144942530}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6820","nonce":1144942530},{"type":"v1","addr":"172.21.15.143:6822","nonce":1144942530}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6827","nonce":1144942530},{"type":"v1","addr":"172.21.15.143:6829","nonce":1144942530}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6824","nonce":1144942530},{"type":"v1","addr":"172.21.15.143:6826","nonce":1144942530}]},"public_addr":"172.21.15.143:6818/1144942530","cluster_addr":"172.21.15.143:6822/1144942530","heartbeat_back_addr":"172.21.15.143:6829/1144942530","heartbeat_front_addr":"172.21.15.143:6826/1144942530","state":["exists","up"]},{"osd":2,"uuid":"2888af5b-f933-46ca-8f95-6dd494ff43f6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6808","nonce":3909095186},{"type":"v1","addr":"172.21.15.143:6809","nonce":3909095186}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6810","nonce":3909095186},{"type":"v1","addr":"172.21.15.143:6811","nonce":3909095186}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6814","nonce":3909095186},{"type":"v1","addr":"172.21.15.143:6815","nonce":3909095186}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6812","nonce":3909095186},{"type":"v1","addr":"172.21.15.143:6813","nonce":3909095186}]},"public_addr":"172.21.15.143:6809/3909095186","cluster_addr":"172.21.15.143:6811/3909095186","heartbeat_back_addr":"172.21.15.143:6815/3909095186","heartbeat_front_addr":"172.21.15.143:6813/3909095186","state":["exists","up"]},{"osd":3,"uuid":"d1f52435-7bc1-4081-9550-9cd1ba55f549","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6800","nonce":3854210991},{"type":"v1","addr":"172.21.15.143:6801","nonce":3854210991}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6802","nonce":3854210991},{"type":"v1","addr":"172.21.15.143:6803","nonce":3854210991}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6806","nonce":3854210991},{"type":"v1","addr":"172.21.15.143:6807","nonce":3854210991}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6804","nonce":3854210991},{"type":"v1","addr":"172.21.15.143:6805","nonce":3854210991}]},"public_addr":"172.21.15.143:6801/3854210991","cluster_addr":"172.21.15.143:6803/3854210991","heartbeat_back_addr":"172.21.15.143:6807/3854210991","heartbeat_front_addr":"172.21.15.143:6805/3854210991","state":["exists","up"]},{"osd":4,"uuid":"e2ae8b6e-6e25-49b2-bcbf-aacce68f06a3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6800","nonce":3924606902},{"type":"v1","addr":"172.21.15.188:6801","nonce":3924606902}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6802","nonce":3924606902},{"type":"v1","addr":"172.21.15.188:6803","nonce":3924606902}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6806","nonce":3924606902},{"type":"v1","addr":"172.21.15.188:6807","nonce":3924606902}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6804","nonce":3924606902},{"type":"v1","addr":"172.21.15.188:6805","nonce":3924606902}]},"public_addr":"172.21.15.188:6801/3924606902","cluster_addr":"172.21.15.188:6803/3924606902","heartbeat_back_addr":"172.21.15.188:6807/3924606902","heartbeat_front_addr":"172.21.15.188:6805/3924606902","state":["exists","up"]},{"osd":5,"uuid":"3cec8a32-7698-4c36-8e89-e4ba11934f33","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6813","nonce":186841544},{"type":"v1","addr":"172.21.15.188:6815","nonce":186841544}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6817","nonce":186841544},{"type":"v1","addr":"172.21.15.188:6818","nonce":186841544}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6822","nonce":186841544},{"type":"v1","addr":"172.21.15.188:6823","nonce":186841544}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6820","nonce":186841544},{"type":"v1","addr":"172.21.15.188:6821","nonce":186841544}]},"public_addr":"172.21.15.188:6815/186841544","cluster_addr":"172.21.15.188:6818/186841544","heartbeat_back_addr":"172.21.15.188:6823/186841544","heartbeat_front_addr":"172.21.15.188:6821/186841544","state":["exists","up"]},{"osd":6,"uuid":"58a65d5d-e090-473a-81b4-ad38e85029f9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6808","nonce":2153499405},{"type":"v1","addr":"172.21.15.188:6809","nonce":2153499405}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6810","nonce":2153499405},{"type":"v1","addr":"172.21.15.188:6811","nonce":2153499405}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6816","nonce":2153499405},{"type":"v1","addr":"172.21.15.188:6819","nonce":2153499405}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6812","nonce":2153499405},{"type":"v1","addr":"172.21.15.188:6814","nonce":2153499405}]},"public_addr":"172.21.15.188:6809/2153499405","cluster_addr":"172.21.15.188:6811/2153499405","heartbeat_back_addr":"172.21.15.188:6819/2153499405","heartbeat_front_addr":"172.21.15.188:6814/2153499405","state":["exists","up"]},{"osd":7,"uuid":"ea5b7f70-3554-4cf8-ac30-f7c59b80678d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6824","nonce":2722096729},{"type":"v1","addr":"172.21.15.188:6825","nonce":2722096729}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6826","nonce":2722096729},{"type":"v1","addr":"172.21.15.188:6827","nonce":2722096729}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6830","nonce":2722096729},{"type":"v1","addr":"172.21.15.188:6831","nonce":2722096729}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6828","nonce":2722096729},{"type":"v1","addr":"172.21.15.188:6829","nonce":2722096729}]},"public_addr":"172.21.15.188:6825/2722096729","cluster_addr":"172.21.15.188:6827/2722096729","heartbeat_back_addr":"172.21.15.188:6831/2722096729","heartbeat_front_addr":"172.21.15.188:6829/2722096729","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-26T07:17:36.995032+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-26T07:17:36.932266+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-26T07:17:36.829304+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-26T07:17:36.804297+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-26T07:17:36.667892+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-26T07:17:36.964868+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-26T07:17:36.827332+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-26T07:17:36.893179+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-11-26T07:18:04.244 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:04.238+0000 7f33e293e640 1 -- 172.21.15.143:0/2383855465 >> [v2:172.21.15.143:6832/2150557544,v1:172.21.15.143:6833/2150557544] conn(0x7f33c8066470 msgr2=0x7f33c8068930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:18:04.244 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:04.238+0000 7f33e293e640 1 --2- 172.21.15.143:0/2383855465 >> [v2:172.21.15.143:6832/2150557544,v1:172.21.15.143:6833/2150557544] conn(0x7f33c8066470 0x7f33c8068930 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7f33cc000f70 tx=0x7f33cc009040 comp rx=0 tx=0).stop 2023-11-26T07:18:04.244 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:04.238+0000 7f33e293e640 1 -- 172.21.15.143:0/2383855465 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f33dc15f4e0 msgr2=0x7f33dc1f5510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:18:04.244 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:04.238+0000 7f33e293e640 1 --2- 172.21.15.143:0/2383855465 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f33dc15f4e0 0x7f33dc1f5510 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7f33d40af040 tx=0x7f33d4019740 comp rx=0 tx=0).stop 2023-11-26T07:18:04.244 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:04.238+0000 7f33e293e640 1 -- 172.21.15.143:0/2383855465 shutdown_connections 2023-11-26T07:18:04.244 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:04.238+0000 7f33e293e640 1 --2- 172.21.15.143:0/2383855465 >> [v2:172.21.15.143:6832/2150557544,v1:172.21.15.143:6833/2150557544] conn(0x7f33c8066470 0x7f33c8068930 unknown :-1 s=CLOSED pgs=104 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:18:04.244 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:04.238+0000 7f33e293e640 1 --2- 172.21.15.143:0/2383855465 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f33dc15f4e0 0x7f33dc1f5510 unknown :-1 s=CLOSED pgs=125 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:18:04.244 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:04.238+0000 7f33e293e640 1 --2- 172.21.15.143:0/2383855465 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f33dc13e290 0x7f33dc1f4fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:18:04.245 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:04.238+0000 7f33e293e640 1 --2- 172.21.15.143:0/2383855465 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f33dc0cf990 0x7f33dc15a1f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:18:04.245 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:04.238+0000 7f33e293e640 1 -- 172.21.15.143:0/2383855465 >> 172.21.15.143:0/2383855465 conn(0x7f33dc06a890 msgr2=0x7f33dc0cf250 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:18:04.245 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:04.238+0000 7f33e293e640 1 -- 172.21.15.143:0/2383855465 shutdown_connections 2023-11-26T07:18:04.245 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:04.238+0000 7f33e293e640 1 -- 172.21.15.143:0/2383855465 wait complete. 2023-11-26T07:18:04.263 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-11-26T07:17:41.758597+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2023-11-26T07:17:51.878332+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2023-11-26T07:17:52.871396+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '20', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2023-11-26T07:18:04.264 DEBUG:teuthology.orchestra.run.smithi143:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-11-26T07:18:04.403 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:04.398+0000 7fe629157640 1 Processor -- start 2023-11-26T07:18:04.403 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:04.398+0000 7fe629157640 1 -- start start 2023-11-26T07:18:04.406 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:04.398+0000 7fe629157640 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7fe6240cf990 0x7fe62415efa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:18:04.406 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:04.398+0000 7fe629157640 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7fe62415f4e0 0x7fe6240cd7b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:18:04.406 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:04.398+0000 7fe629157640 1 -- --> v1:172.21.15.188:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe62406c520 con 0x7fe62413e290 2023-11-26T07:18:04.406 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:04.398+0000 7fe629157640 1 -- --> v2:172.21.15.143:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe62406c800 con 0x7fe6240cf990 2023-11-26T07:18:04.406 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:04.398+0000 7fe629157640 1 -- --> v2:172.21.15.188:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe62406cae0 con 0x7fe62415f4e0 2023-11-26T07:18:04.407 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:04.398+0000 7fe61bfff640 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7fe6240cf990 0x7fe62415efa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:18:04.407 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:04.398+0000 7fe61bfff640 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7fe6240cf990 0x7fe62415efa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:46834/0 (socket says 172.21.15.143:46834) 2023-11-26T07:18:04.407 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:04.398+0000 7fe61bfff640 1 -- 172.21.15.143:0/1813028087 learned_addr learned my addr 172.21.15.143:0/1813028087 (peer_addr_for_me v2:172.21.15.143:0/0) 2023-11-26T07:18:04.407 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:04.402+0000 7fe62359e640 1 --2- 172.21.15.143:0/1813028087 >> v2:172.21.15.188:3300/0 conn(0x7fe62415f4e0 0x7fe6240cd7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:18:04.407 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:04.402+0000 7fe61bfff640 1 -- 172.21.15.143:0/1813028087 >> v2:172.21.15.188:3300/0 conn(0x7fe62415f4e0 msgr2=0x7fe6240cd7b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-26T07:18:04.408 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:04.402+0000 7fe61bfff640 1 --2- 172.21.15.143:0/1813028087 >> v2:172.21.15.188:3300/0 conn(0x7fe62415f4e0 0x7fe6240cd7b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:18:04.408 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:04.402+0000 7fe61bfff640 1 -- 172.21.15.143:0/1813028087 >> v1:172.21.15.188:6789/0 conn(0x7fe62413e290 legacy=0x7fe6240cf280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-26T07:18:04.408 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:04.402+0000 7fe61bfff640 1 -- 172.21.15.143:0/1813028087 --> v2:172.21.15.143:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe62406cdc0 con 0x7fe6240cf990 2023-11-26T07:18:04.408 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:04.402+0000 7fe62359e640 1 --2- 172.21.15.143:0/1813028087 >> v2:172.21.15.188:3300/0 conn(0x7fe62415f4e0 0x7fe6240cd7b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-26T07:18:04.410 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:04.402+0000 7fe61bfff640 1 --2- 172.21.15.143:0/1813028087 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fe6240cf990 0x7fe62415efa0 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7fe624155cf0 tx=0x7fe60c01c490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c256ad3a87b65a3e server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:18:04.410 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:04.402+0000 7fe62259c640 1 -- 172.21.15.143:0/1813028087 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe60c0aa610 con 0x7fe6240cf990 2023-11-26T07:18:04.410 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:04.402+0000 7fe62259c640 1 -- 172.21.15.143:0/1813028087 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe60c0a6040 con 0x7fe6240cf990 2023-11-26T07:18:04.410 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:04.402+0000 7fe62259c640 1 -- 172.21.15.143:0/1813028087 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe60c0aa990 con 0x7fe6240cf990 2023-11-26T07:18:04.411 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:04.402+0000 7fe629157640 1 -- 172.21.15.143:0/1813028087 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fe6240cf990 msgr2=0x7fe62415efa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:18:04.411 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:04.402+0000 7fe629157640 1 --2- 172.21.15.143:0/1813028087 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fe6240cf990 0x7fe62415efa0 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7fe624155cf0 tx=0x7fe60c01c490 comp rx=0 tx=0).stop 2023-11-26T07:18:04.411 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:04.406+0000 7fe629157640 1 -- 172.21.15.143:0/1813028087 shutdown_connections 2023-11-26T07:18:04.411 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:04.406+0000 7fe629157640 1 --2- 172.21.15.143:0/1813028087 >> v2:172.21.15.188:3300/0 conn(0x7fe62415f4e0 0x7fe6240cd7b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:18:04.411 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:04.406+0000 7fe629157640 1 --2- 172.21.15.143:0/1813028087 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fe6240cf990 0x7fe62415efa0 unknown :-1 s=CLOSED pgs=126 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:18:04.411 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:04.406+0000 7fe629157640 1 -- 172.21.15.143:0/1813028087 >> 172.21.15.143:0/1813028087 conn(0x7fe62406a890 msgr2=0x7fe62406aca0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:18:04.411 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:04.406+0000 7fe629157640 1 -- 172.21.15.143:0/1813028087 shutdown_connections 2023-11-26T07:18:04.411 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:04.406+0000 7fe629157640 1 -- 172.21.15.143:0/1813028087 wait complete. 2023-11-26T07:18:04.414 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:04.406+0000 7fe629157640 1 Processor -- start 2023-11-26T07:18:04.415 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:04.406+0000 7fe629157640 1 -- start start 2023-11-26T07:18:04.415 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:04.406+0000 7fe629157640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fe6240cf990 0x7fe624154a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:18:04.415 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:04.406+0000 7fe629157640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fe62413e290 0x7fe6240aa2d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:18:04.415 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:04.406+0000 7fe629157640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fe62415f4e0 0x7fe62415bfc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:18:04.415 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:04.406+0000 7fe629157640 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe624137cc0 con 0x7fe6240cf990 2023-11-26T07:18:04.416 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:04.406+0000 7fe629157640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe624154f70 con 0x7fe62415f4e0 2023-11-26T07:18:04.416 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:04.406+0000 7fe629157640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe6240aa810 con 0x7fe62413e290 2023-11-26T07:18:04.416 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:04.406+0000 7fe622d9d640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fe6240cf990 0x7fe624154a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:18:04.416 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:04.406+0000 7fe622d9d640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fe6240cf990 0x7fe624154a30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:48136/0 (socket says 172.21.15.143:48136) 2023-11-26T07:18:04.416 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:04.406+0000 7fe622d9d640 1 -- 172.21.15.143:0/151772150 learned_addr learned my addr 172.21.15.143:0/151772150 (peer_addr_for_me v2:172.21.15.143:0/0) 2023-11-26T07:18:04.416 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:04.406+0000 7fe61bfff640 1 --2- 172.21.15.143:0/151772150 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fe62413e290 0x7fe6240aa2d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:18:04.417 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:04.406+0000 7fe62359e640 1 --2- 172.21.15.143:0/151772150 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fe62415f4e0 0x7fe62415bfc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:18:04.417 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:04.406+0000 7fe622d9d640 1 -- 172.21.15.143:0/151772150 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fe62413e290 msgr2=0x7fe6240aa2d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:18:04.417 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:04.406+0000 7fe622d9d640 1 --2- 172.21.15.143:0/151772150 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fe62413e290 0x7fe6240aa2d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:18:04.417 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:04.406+0000 7fe622d9d640 1 -- 172.21.15.143:0/151772150 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fe62415f4e0 msgr2=0x7fe62415bfc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:18:04.417 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:04.406+0000 7fe622d9d640 1 --2- 172.21.15.143:0/151772150 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fe62415f4e0 0x7fe62415bfc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:18:04.418 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:04.406+0000 7fe622d9d640 1 -- 172.21.15.143:0/151772150 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe60c0094d0 con 0x7fe6240cf990 2023-11-26T07:18:04.418 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:04.406+0000 7fe61bfff640 1 --2- 172.21.15.143:0/151772150 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fe62413e290 0x7fe6240aa2d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-26T07:18:04.418 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:04.406+0000 7fe622d9d640 1 --2- 172.21.15.143:0/151772150 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fe6240cf990 0x7fe624154a30 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7fe61400c450 tx=0x7fe61400c910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:18:04.418 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:04.410+0000 7fe61b7fe640 1 -- 172.21.15.143:0/151772150 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe614012830 con 0x7fe6240cf990 2023-11-26T07:18:04.418 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:04.410+0000 7fe61b7fe640 1 -- 172.21.15.143:0/151772150 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe6140129b0 con 0x7fe6240cf990 2023-11-26T07:18:04.419 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:04.410+0000 7fe61b7fe640 1 -- 172.21.15.143:0/151772150 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe614012d60 con 0x7fe6240cf990 2023-11-26T07:18:04.419 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:04.410+0000 7fe629157640 1 -- 172.21.15.143:0/151772150 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe62415cc40 con 0x7fe6240cf990 2023-11-26T07:18:04.419 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:04.410+0000 7fe629157640 1 -- 172.21.15.143:0/151772150 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe62415d200 con 0x7fe6240cf990 2023-11-26T07:18:04.419 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:04.410+0000 7fe629157640 1 -- 172.21.15.143:0/151772150 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe624067680 con 0x7fe6240cf990 2023-11-26T07:18:04.419 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:04.410+0000 7fe61b7fe640 1 -- 172.21.15.143:0/151772150 <== mon.0 v2:172.21.15.143:3300/0 4 ==== mgrmap(e 5) v1 ==== 85852+0+0 (secure 0 0 0) 0x7fe61401c460 con 0x7fe6240cf990 2023-11-26T07:18:04.421 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:04.410+0000 7fe61b7fe640 1 --2- 172.21.15.143:0/151772150 >> [v2:172.21.15.143:6832/2150557544,v1:172.21.15.143:6833/2150557544] conn(0x7fe604066470 0x7fe604068930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:18:04.421 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:04.410+0000 7fe61b7fe640 1 -- 172.21.15.143:0/151772150 <== mon.0 v2:172.21.15.143:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fe614083d40 con 0x7fe6240cf990 2023-11-26T07:18:04.421 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:04.410+0000 7fe61bfff640 1 --2- 172.21.15.143:0/151772150 >> [v2:172.21.15.143:6832/2150557544,v1:172.21.15.143:6833/2150557544] conn(0x7fe604066470 0x7fe604068930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:18:04.421 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:04.410+0000 7fe61bfff640 1 --2- 172.21.15.143:0/151772150 >> [v2:172.21.15.143:6832/2150557544,v1:172.21.15.143:6833/2150557544] conn(0x7fe604066470 0x7fe604068930 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7fe60c0ae000 tx=0x7fe60c0a7000 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:18:04.421 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:04.414+0000 7fe61b7fe640 1 -- 172.21.15.143:0/151772150 <== mon.0 v2:172.21.15.143:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172649 (secure 0 0 0) 0x7fe614051120 con 0x7fe6240cf990 2023-11-26T07:18:04.568 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:04.562+0000 7fe629157640 1 -- 172.21.15.143:0/151772150 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7fe62406df50 con 0x7fe6240cf990 2023-11-26T07:18:04.568 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:04.562+0000 7fe61b7fe640 1 -- 172.21.15.143:0/151772150 <== mon.0 v2:172.21.15.143:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v20) v1 ==== 93+0+10 (secure 0 0 0) 0x7fe614054740 con 0x7fe6240cf990 2023-11-26T07:18:04.569 INFO:teuthology.orchestra.run.smithi143.stdout:pg_num: 1 2023-11-26T07:18:04.570 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:04.566+0000 7fe629157640 1 -- 172.21.15.143:0/151772150 >> [v2:172.21.15.143:6832/2150557544,v1:172.21.15.143:6833/2150557544] conn(0x7fe604066470 msgr2=0x7fe604068930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:18:04.571 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:04.566+0000 7fe629157640 1 --2- 172.21.15.143:0/151772150 >> [v2:172.21.15.143:6832/2150557544,v1:172.21.15.143:6833/2150557544] conn(0x7fe604066470 0x7fe604068930 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7fe60c0ae000 tx=0x7fe60c0a7000 comp rx=0 tx=0).stop 2023-11-26T07:18:04.571 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:04.566+0000 7fe629157640 1 -- 172.21.15.143:0/151772150 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fe6240cf990 msgr2=0x7fe624154a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:18:04.571 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:04.566+0000 7fe629157640 1 --2- 172.21.15.143:0/151772150 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fe6240cf990 0x7fe624154a30 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7fe61400c450 tx=0x7fe61400c910 comp rx=0 tx=0).stop 2023-11-26T07:18:04.571 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:04.566+0000 7fe629157640 1 -- 172.21.15.143:0/151772150 shutdown_connections 2023-11-26T07:18:04.571 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:04.566+0000 7fe629157640 1 --2- 172.21.15.143:0/151772150 >> [v2:172.21.15.143:6832/2150557544,v1:172.21.15.143:6833/2150557544] conn(0x7fe604066470 0x7fe604068930 unknown :-1 s=CLOSED pgs=105 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:18:04.571 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:04.566+0000 7fe629157640 1 --2- 172.21.15.143:0/151772150 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fe62415f4e0 0x7fe62415bfc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:18:04.571 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:04.566+0000 7fe629157640 1 --2- 172.21.15.143:0/151772150 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fe62413e290 0x7fe6240aa2d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:18:04.571 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:04.566+0000 7fe629157640 1 --2- 172.21.15.143:0/151772150 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fe6240cf990 0x7fe624154a30 unknown :-1 s=CLOSED pgs=127 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:18:04.571 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:04.566+0000 7fe629157640 1 -- 172.21.15.143:0/151772150 >> 172.21.15.143:0/151772150 conn(0x7fe62406a890 msgr2=0x7fe62406ac70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:18:04.571 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:04.566+0000 7fe629157640 1 -- 172.21.15.143:0/151772150 shutdown_connections 2023-11-26T07:18:04.571 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:04.566+0000 7fe629157640 1 -- 172.21.15.143:0/151772150 wait complete. 2023-11-26T07:18:04.590 DEBUG:teuthology.orchestra.run.smithi143:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-11-26T07:18:04.731 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:04.726+0000 7fb87372a640 1 Processor -- start 2023-11-26T07:18:04.731 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:04.726+0000 7fb87372a640 1 -- start start 2023-11-26T07:18:04.733 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:04.726+0000 7fb87372a640 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7fb86c0cf990 0x7fb86c15efa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:18:04.733 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:04.726+0000 7fb87372a640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fb86c15f4e0 0x7fb86c0cd7b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:18:04.733 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:04.726+0000 7fb87372a640 1 -- --> v1:172.21.15.188:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb86c06c520 con 0x7fb86c13e290 2023-11-26T07:18:04.733 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:04.726+0000 7fb87372a640 1 -- --> v2:172.21.15.143:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb86c06c800 con 0x7fb86c0cf990 2023-11-26T07:18:04.733 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:04.726+0000 7fb87372a640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb86c06cae0 con 0x7fb86c15f4e0 2023-11-26T07:18:04.733 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:04.726+0000 7fb870cc5640 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7fb86c0cf990 0x7fb86c15efa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:18:04.733 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:04.730+0000 7fb870cc5640 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7fb86c0cf990 0x7fb86c15efa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:48146/0 (socket says 172.21.15.143:48146) 2023-11-26T07:18:04.733 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:04.730+0000 7fb870cc5640 1 -- 172.21.15.143:0/2183547910 learned_addr learned my addr 172.21.15.143:0/2183547910 (peer_addr_for_me v2:172.21.15.143:0/0) 2023-11-26T07:18:04.734 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:04.730+0000 7fb871cc7640 1 --2- 172.21.15.143:0/2183547910 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fb86c15f4e0 0x7fb86c0cd7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:18:04.734 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:04.730+0000 7fb870cc5640 1 -- 172.21.15.143:0/2183547910 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fb86c15f4e0 msgr2=0x7fb86c0cd7b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-26T07:18:04.734 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:04.730+0000 7fb870cc5640 1 --2- 172.21.15.143:0/2183547910 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fb86c15f4e0 0x7fb86c0cd7b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:18:04.734 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:04.730+0000 7fb870cc5640 1 -- 172.21.15.143:0/2183547910 >> v1:172.21.15.188:6789/0 conn(0x7fb86c13e290 legacy=0x7fb86c0cf280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:18:04.734 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:04.730+0000 7fb870cc5640 1 -- 172.21.15.143:0/2183547910 --> v2:172.21.15.143:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb86c06cdc0 con 0x7fb86c0cf990 2023-11-26T07:18:04.735 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:04.730+0000 7fb870cc5640 1 --2- 172.21.15.143:0/2183547910 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fb86c0cf990 0x7fb86c15efa0 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7fb85c009870 tx=0x7fb85c01c510 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c7e6d0ed65a54ab1 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:18:04.736 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:04.730+0000 7fb863fff640 1 -- 172.21.15.143:0/2183547910 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb85c0a95f0 con 0x7fb86c0cf990 2023-11-26T07:18:04.736 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:04.730+0000 7fb863fff640 1 -- 172.21.15.143:0/2183547910 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb85c0a9750 con 0x7fb86c0cf990 2023-11-26T07:18:04.736 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:04.730+0000 7fb863fff640 1 -- 172.21.15.143:0/2183547910 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb85c0a9ae0 con 0x7fb86c0cf990 2023-11-26T07:18:04.737 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:04.730+0000 7fb87372a640 1 -- 172.21.15.143:0/2183547910 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fb86c0cf990 msgr2=0x7fb86c15efa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:18:04.737 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:04.730+0000 7fb87372a640 1 --2- 172.21.15.143:0/2183547910 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fb86c0cf990 0x7fb86c15efa0 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7fb85c009870 tx=0x7fb85c01c510 comp rx=0 tx=0).stop 2023-11-26T07:18:04.737 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:04.734+0000 7fb87372a640 1 -- 172.21.15.143:0/2183547910 shutdown_connections 2023-11-26T07:18:04.737 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:04.734+0000 7fb87372a640 1 --2- 172.21.15.143:0/2183547910 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fb86c0cf990 0x7fb86c15efa0 unknown :-1 s=CLOSED pgs=128 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:18:04.737 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:04.734+0000 7fb87372a640 1 --2- 172.21.15.143:0/2183547910 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fb86c15f4e0 0x7fb86c0cd7b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:18:04.738 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:04.734+0000 7fb87372a640 1 -- 172.21.15.143:0/2183547910 >> 172.21.15.143:0/2183547910 conn(0x7fb86c06a890 msgr2=0x7fb86c06aca0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:18:04.738 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:04.734+0000 7fb87372a640 1 -- 172.21.15.143:0/2183547910 shutdown_connections 2023-11-26T07:18:04.738 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:04.734+0000 7fb87372a640 1 -- 172.21.15.143:0/2183547910 wait complete. 2023-11-26T07:18:04.738 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:04.734+0000 7fb87372a640 1 Processor -- start 2023-11-26T07:18:04.738 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:04.734+0000 7fb87372a640 1 -- start start 2023-11-26T07:18:04.740 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:04.734+0000 7fb87372a640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fb86c0cf990 0x7fb86c1f59f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:18:04.740 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:04.734+0000 7fb87372a640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fb86c13e290 0x7fb86c0b2b10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:18:04.740 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:04.734+0000 7fb87372a640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fb86c15f4e0 0x7fb86c0b1470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:18:04.740 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:04.734+0000 7fb87372a640 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb86c1f5f30 con 0x7fb86c15f4e0 2023-11-26T07:18:04.740 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:04.734+0000 7fb87372a640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb86c0b3050 con 0x7fb86c0cf990 2023-11-26T07:18:04.740 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:04.734+0000 7fb87372a640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb86c071750 con 0x7fb86c13e290 2023-11-26T07:18:04.741 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:04.734+0000 7fb871cc7640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fb86c15f4e0 0x7fb86c0b1470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:18:04.741 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:04.734+0000 7fb8714c6640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fb86c0cf990 0x7fb86c1f59f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:18:04.741 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:04.734+0000 7fb871cc7640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fb86c15f4e0 0x7fb86c0b1470 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:48154/0 (socket says 172.21.15.143:48154) 2023-11-26T07:18:04.741 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:04.734+0000 7fb871cc7640 1 -- 172.21.15.143:0/1186765553 learned_addr learned my addr 172.21.15.143:0/1186765553 (peer_addr_for_me v2:172.21.15.143:0/0) 2023-11-26T07:18:04.741 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:04.734+0000 7fb870cc5640 1 --2- 172.21.15.143:0/1186765553 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fb86c13e290 0x7fb86c0b2b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:18:04.741 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:04.734+0000 7fb871cc7640 1 -- 172.21.15.143:0/1186765553 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fb86c13e290 msgr2=0x7fb86c0b2b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:18:04.741 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:04.734+0000 7fb871cc7640 1 --2- 172.21.15.143:0/1186765553 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fb86c13e290 0x7fb86c0b2b10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:18:04.741 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:04.734+0000 7fb871cc7640 1 -- 172.21.15.143:0/1186765553 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fb86c0cf990 msgr2=0x7fb86c1f59f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:18:04.741 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:04.734+0000 7fb871cc7640 1 --2- 172.21.15.143:0/1186765553 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fb86c0cf990 0x7fb86c1f59f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:18:04.741 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:04.734+0000 7fb871cc7640 1 -- 172.21.15.143:0/1186765553 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb85c0094d0 con 0x7fb86c15f4e0 2023-11-26T07:18:04.742 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:04.734+0000 7fb871cc7640 1 --2- 172.21.15.143:0/1186765553 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fb86c15f4e0 0x7fb86c0b1470 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7fb864007e40 tx=0x7fb86400e4b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:18:04.742 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:04.738+0000 7fb861ffb640 1 -- 172.21.15.143:0/1186765553 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb864014620 con 0x7fb86c15f4e0 2023-11-26T07:18:04.742 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:04.738+0000 7fb861ffb640 1 -- 172.21.15.143:0/1186765553 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb86400ee90 con 0x7fb86c15f4e0 2023-11-26T07:18:04.743 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:04.738+0000 7fb87372a640 1 -- 172.21.15.143:0/1186765553 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb86c071a30 con 0x7fb86c15f4e0 2023-11-26T07:18:04.743 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:04.738+0000 7fb861ffb640 1 -- 172.21.15.143:0/1186765553 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb8640149e0 con 0x7fb86c15f4e0 2023-11-26T07:18:04.743 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:04.738+0000 7fb87372a640 1 -- 172.21.15.143:0/1186765553 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb86c071ff0 con 0x7fb86c15f4e0 2023-11-26T07:18:04.743 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:04.738+0000 7fb87372a640 1 -- 172.21.15.143:0/1186765553 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb86c067680 con 0x7fb86c15f4e0 2023-11-26T07:18:04.743 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:04.738+0000 7fb861ffb640 1 -- 172.21.15.143:0/1186765553 <== mon.0 v2:172.21.15.143:3300/0 4 ==== mgrmap(e 5) v1 ==== 85852+0+0 (secure 0 0 0) 0x7fb864014b40 con 0x7fb86c15f4e0 2023-11-26T07:18:04.744 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:04.738+0000 7fb861ffb640 1 --2- 172.21.15.143:0/1186765553 >> [v2:172.21.15.143:6832/2150557544,v1:172.21.15.143:6833/2150557544] conn(0x7fb850066470 0x7fb850068930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:18:04.744 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:04.738+0000 7fb861ffb640 1 -- 172.21.15.143:0/1186765553 <== mon.0 v2:172.21.15.143:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fb864084700 con 0x7fb86c15f4e0 2023-11-26T07:18:04.744 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:04.738+0000 7fb8714c6640 1 --2- 172.21.15.143:0/1186765553 >> [v2:172.21.15.143:6832/2150557544,v1:172.21.15.143:6833/2150557544] conn(0x7fb850066470 0x7fb850068930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:18:04.744 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:04.738+0000 7fb8714c6640 1 --2- 172.21.15.143:0/1186765553 >> [v2:172.21.15.143:6832/2150557544,v1:172.21.15.143:6833/2150557544] conn(0x7fb850066470 0x7fb850068930 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7fb868007fd0 tx=0x7fb868009040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:18:04.747 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:04.742+0000 7fb861ffb640 1 -- 172.21.15.143:0/1186765553 <== mon.0 v2:172.21.15.143:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172649 (secure 0 0 0) 0x7fb864051c20 con 0x7fb86c15f4e0 2023-11-26T07:18:04.898 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:04.894+0000 7fb87372a640 1 -- 172.21.15.143:0/1186765553 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7fb86c13da50 con 0x7fb86c15f4e0 2023-11-26T07:18:04.898 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:04.894+0000 7fb861ffb640 1 -- 172.21.15.143:0/1186765553 <== mon.0 v2:172.21.15.143:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v20) v1 ==== 104+0+11 (secure 0 0 0) 0x7fb864055240 con 0x7fb86c15f4e0 2023-11-26T07:18:04.898 INFO:teuthology.orchestra.run.smithi143.stdout:pg_num: 32 2023-11-26T07:18:04.900 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:04.894+0000 7fb87372a640 1 -- 172.21.15.143:0/1186765553 >> [v2:172.21.15.143:6832/2150557544,v1:172.21.15.143:6833/2150557544] conn(0x7fb850066470 msgr2=0x7fb850068930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:18:04.900 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:04.894+0000 7fb87372a640 1 --2- 172.21.15.143:0/1186765553 >> [v2:172.21.15.143:6832/2150557544,v1:172.21.15.143:6833/2150557544] conn(0x7fb850066470 0x7fb850068930 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7fb868007fd0 tx=0x7fb868009040 comp rx=0 tx=0).stop 2023-11-26T07:18:04.900 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:04.894+0000 7fb87372a640 1 -- 172.21.15.143:0/1186765553 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fb86c15f4e0 msgr2=0x7fb86c0b1470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:18:04.900 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:04.894+0000 7fb87372a640 1 --2- 172.21.15.143:0/1186765553 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fb86c15f4e0 0x7fb86c0b1470 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7fb864007e40 tx=0x7fb86400e4b0 comp rx=0 tx=0).stop 2023-11-26T07:18:04.900 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:04.894+0000 7fb87372a640 1 -- 172.21.15.143:0/1186765553 shutdown_connections 2023-11-26T07:18:04.901 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:04.894+0000 7fb87372a640 1 --2- 172.21.15.143:0/1186765553 >> [v2:172.21.15.143:6832/2150557544,v1:172.21.15.143:6833/2150557544] conn(0x7fb850066470 0x7fb850068930 unknown :-1 s=CLOSED pgs=106 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:18:04.901 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:04.894+0000 7fb87372a640 1 --2- 172.21.15.143:0/1186765553 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fb86c15f4e0 0x7fb86c0b1470 unknown :-1 s=CLOSED pgs=129 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:18:04.901 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:04.894+0000 7fb87372a640 1 --2- 172.21.15.143:0/1186765553 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fb86c13e290 0x7fb86c0b2b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:18:04.901 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:04.894+0000 7fb87372a640 1 --2- 172.21.15.143:0/1186765553 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fb86c0cf990 0x7fb86c1f59f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:18:04.901 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:04.894+0000 7fb87372a640 1 -- 172.21.15.143:0/1186765553 >> 172.21.15.143:0/1186765553 conn(0x7fb86c06a890 msgr2=0x7fb86c13c720 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:18:04.901 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:04.894+0000 7fb87372a640 1 -- 172.21.15.143:0/1186765553 shutdown_connections 2023-11-26T07:18:04.901 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:04.894+0000 7fb87372a640 1 -- 172.21.15.143:0/1186765553 wait complete. 2023-11-26T07:18:04.920 DEBUG:teuthology.orchestra.run.smithi143:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-11-26T07:18:05.060 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:05.054+0000 7f81a692a640 1 Processor -- start 2023-11-26T07:18:05.060 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:05.054+0000 7f81a692a640 1 -- start start 2023-11-26T07:18:05.062 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:05.058+0000 7f81a692a640 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7f81a00d17f0 0x7f81a00d1c30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:18:05.062 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:05.058+0000 7f81a692a640 1 -- --> v1:172.21.15.143:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f81a00d0bc0 con 0x7f81a00b0490 2023-11-26T07:18:05.062 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:05.058+0000 7f81a692a640 1 -- --> v1:172.21.15.188:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f81a00d0320 con 0x7f81a015d580 2023-11-26T07:18:05.062 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:05.058+0000 7f81a692a640 1 -- --> v2:172.21.15.188:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f81a00d0600 con 0x7f81a00d17f0 2023-11-26T07:18:05.062 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:05.058+0000 7f81a4ec7640 1 --1- >> v1:172.21.15.143:6789/0 conn(0x7f81a00b0490 0x7f81a00b0920 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.143:6789/0 says I am v1:172.21.15.143:52196/0 (socket says 172.21.15.143:52196) 2023-11-26T07:18:05.062 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:05.058+0000 7f81a4ec7640 1 -- 172.21.15.143:0/761022641 learned_addr learned my addr 172.21.15.143:0/761022641 (peer_addr_for_me v1:172.21.15.143:0/0) 2023-11-26T07:18:05.062 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:05.058+0000 7f819f7fe640 1 --2- 172.21.15.143:0/761022641 >> v2:172.21.15.188:3300/0 conn(0x7f81a00d17f0 0x7f81a00d1c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:18:05.064 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:05.058+0000 7f819effd640 1 -- 172.21.15.143:0/761022641 <== mon.0 v1:172.21.15.143:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 833009468 0 0) 0x7f81a00d0bc0 con 0x7f81a00b0490 2023-11-26T07:18:05.064 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:05.058+0000 7f819effd640 1 -- 172.21.15.143:0/761022641 --> v1:172.21.15.143:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f8188003530 con 0x7f81a00b0490 2023-11-26T07:18:05.065 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:05.058+0000 7f819effd640 1 -- 172.21.15.143:0/761022641 <== mon.1 v1:172.21.15.188:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 87437029 0 0) 0x7f81a00d0320 con 0x7f81a015d580 2023-11-26T07:18:05.065 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:05.058+0000 7f819effd640 1 -- 172.21.15.143:0/761022641 --> v1:172.21.15.188:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f81a00d0bc0 con 0x7f81a015d580 2023-11-26T07:18:05.065 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:05.058+0000 7f819effd640 1 -- 172.21.15.143:0/761022641 <== mon.0 v1:172.21.15.143:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1545385864 0 0) 0x7f8188003530 con 0x7f81a00b0490 2023-11-26T07:18:05.065 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:05.058+0000 7f819effd640 1 -- 172.21.15.143:0/761022641 >> v2:172.21.15.188:3300/0 conn(0x7f81a00d17f0 msgr2=0x7f81a00d1c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-26T07:18:05.065 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:05.058+0000 7f819effd640 1 --2- 172.21.15.143:0/761022641 >> v2:172.21.15.188:3300/0 conn(0x7f81a00d17f0 0x7f81a00d1c30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:18:05.065 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:05.058+0000 7f819effd640 1 -- 172.21.15.143:0/761022641 >> v1:172.21.15.188:6789/0 conn(0x7f81a015d580 legacy=0x7f81a00d3920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:18:05.065 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:05.058+0000 7f819effd640 1 -- 172.21.15.143:0/761022641 --> v1:172.21.15.143:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f81a00d08e0 con 0x7f81a00b0490 2023-11-26T07:18:05.065 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:05.058+0000 7f819effd640 1 -- 172.21.15.143:0/761022641 <== mon.0 v1:172.21.15.143:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2137882523 0 0) 0x7f8198001a00 con 0x7f81a00b0490 2023-11-26T07:18:05.066 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:05.058+0000 7f819effd640 1 -- 172.21.15.143:0/761022641 >> v1:172.21.15.143:6789/0 conn(0x7f81a00b0490 legacy=0x7f81a00b0920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:18:05.066 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:05.058+0000 7f819effd640 1 --2- 172.21.15.143:0/761022641 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f8188003b10 0x7f8188003f10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:18:05.066 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:05.058+0000 7f819effd640 1 --2- 172.21.15.143:0/761022641 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f81880047d0 0x7f8188004c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:18:05.066 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:05.058+0000 7f819effd640 1 --2- 172.21.15.143:0/761022641 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f8188005190 0x7f818800b990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:18:05.066 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:05.058+0000 7f819effd640 1 -- 172.21.15.143:0/761022641 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f81a00d0600 con 0x7f8188003b10 2023-11-26T07:18:05.066 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:05.058+0000 7f819effd640 1 -- 172.21.15.143:0/761022641 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f818800c010 con 0x7f81880047d0 2023-11-26T07:18:05.066 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:05.058+0000 7f819effd640 1 -- 172.21.15.143:0/761022641 --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f818800c1d0 con 0x7f8188005190 2023-11-26T07:18:05.066 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:05.058+0000 7f819ffff640 1 --2- 172.21.15.143:0/761022641 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f8188003b10 0x7f8188003f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:18:05.067 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:05.058+0000 7f819f7fe640 1 --2- 172.21.15.143:0/761022641 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f81880047d0 0x7f8188004c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:18:05.067 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:05.058+0000 7f81a4ec7640 1 --2- 172.21.15.143:0/761022641 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f8188005190 0x7f818800b990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:18:05.067 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:05.058+0000 7f819ffff640 1 -- 172.21.15.143:0/761022641 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f8188005190 msgr2=0x7f818800b990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-26T07:18:05.067 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:05.058+0000 7f819ffff640 1 --2- 172.21.15.143:0/761022641 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f8188005190 0x7f818800b990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:18:05.067 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:05.058+0000 7f819ffff640 1 -- 172.21.15.143:0/761022641 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f81880047d0 msgr2=0x7f8188004c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-26T07:18:05.067 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:05.058+0000 7f819ffff640 1 --2- 172.21.15.143:0/761022641 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f81880047d0 0x7f8188004c50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:18:05.067 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:05.058+0000 7f819ffff640 1 -- 172.21.15.143:0/761022641 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8188003530 con 0x7f8188003b10 2023-11-26T07:18:05.067 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:05.058+0000 7f81a4ec7640 1 -- 172.21.15.143:0/761022641 reap_dead start 2023-11-26T07:18:05.068 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:05.062+0000 7f819ffff640 1 --2- 172.21.15.143:0/761022641 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f8188003b10 0x7f8188003f10 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7f81900018d0 tx=0x7f819001c4f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e45134a28d6c1ace server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:18:05.068 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:05.062+0000 7f819effd640 1 -- 172.21.15.143:0/761022641 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f81900052d0 con 0x7f8188003b10 2023-11-26T07:18:05.068 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:05.062+0000 7f819effd640 1 -- 172.21.15.143:0/761022641 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f81900a7050 con 0x7f8188003b10 2023-11-26T07:18:05.068 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:05.062+0000 7f819effd640 1 -- 172.21.15.143:0/761022641 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8190003600 con 0x7f8188003b10 2023-11-26T07:18:05.068 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:05.062+0000 7f81a692a640 1 -- 172.21.15.143:0/761022641 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f8188003b10 msgr2=0x7f8188003f10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:18:05.069 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:05.062+0000 7f81a692a640 1 --2- 172.21.15.143:0/761022641 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f8188003b10 0x7f8188003f10 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7f81900018d0 tx=0x7f819001c4f0 comp rx=0 tx=0).stop 2023-11-26T07:18:05.069 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:05.062+0000 7f81a692a640 1 -- 172.21.15.143:0/761022641 shutdown_connections 2023-11-26T07:18:05.069 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:05.062+0000 7f81a692a640 1 --2- 172.21.15.143:0/761022641 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f8188003b10 0x7f8188003f10 unknown :-1 s=CLOSED pgs=131 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:18:05.069 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:05.062+0000 7f81a692a640 1 -- 172.21.15.143:0/761022641 >> 172.21.15.143:0/761022641 conn(0x7f81a009c550 msgr2=0x7f81a00d3210 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:18:05.069 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:05.062+0000 7f81a692a640 1 -- 172.21.15.143:0/761022641 shutdown_connections 2023-11-26T07:18:05.069 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:05.062+0000 7f81a692a640 1 -- 172.21.15.143:0/761022641 wait complete. 2023-11-26T07:18:05.069 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:05.066+0000 7f81a692a640 1 Processor -- start 2023-11-26T07:18:05.070 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:05.066+0000 7f81a692a640 1 -- start start 2023-11-26T07:18:05.072 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:05.066+0000 7f81a692a640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f8188003b10 0x7f81a00ccd70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:18:05.072 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:05.066+0000 7f81a692a640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f81a00b7e70 0x7f81a0152690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:18:05.072 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:05.066+0000 7f81a692a640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f81a01f9600 0x7f81a01f9a80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:18:05.072 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:05.066+0000 7f81a692a640 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f81a01f9fc0 con 0x7f8188003b10 2023-11-26T07:18:05.072 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:05.066+0000 7f81a692a640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f81a00ca7f0 con 0x7f81a01f9600 2023-11-26T07:18:05.072 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:05.066+0000 7f81a692a640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f81a00b85e0 con 0x7f81a00b7e70 2023-11-26T07:18:05.072 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:05.066+0000 7f819ffff640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f8188003b10 0x7f81a00ccd70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:18:05.072 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:05.066+0000 7f819ffff640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f8188003b10 0x7f81a00ccd70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:48176/0 (socket says 172.21.15.143:48176) 2023-11-26T07:18:05.072 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:05.066+0000 7f819ffff640 1 -- 172.21.15.143:0/1705342442 learned_addr learned my addr 172.21.15.143:0/1705342442 (peer_addr_for_me v2:172.21.15.143:0/0) 2023-11-26T07:18:05.072 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:05.066+0000 7f81a4ec7640 1 --2- 172.21.15.143:0/1705342442 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f81a01f9600 0x7f81a01f9a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:18:05.072 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:05.066+0000 7f819f7fe640 1 --2- 172.21.15.143:0/1705342442 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f81a00b7e70 0x7f81a0152690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:18:05.072 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:05.066+0000 7f819ffff640 1 -- 172.21.15.143:0/1705342442 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f81a00b7e70 msgr2=0x7f81a0152690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:18:05.073 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:05.066+0000 7f819ffff640 1 --2- 172.21.15.143:0/1705342442 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f81a00b7e70 0x7f81a0152690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:18:05.073 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:05.066+0000 7f819ffff640 1 -- 172.21.15.143:0/1705342442 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f81a01f9600 msgr2=0x7f81a01f9a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:18:05.073 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:05.066+0000 7f819ffff640 1 --2- 172.21.15.143:0/1705342442 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f81a01f9600 0x7f81a01f9a80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:18:05.073 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:05.066+0000 7f819ffff640 1 -- 172.21.15.143:0/1705342442 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f81a00b88c0 con 0x7f8188003b10 2023-11-26T07:18:05.073 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:05.066+0000 7f819f7fe640 1 --2- 172.21.15.143:0/1705342442 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f81a00b7e70 0x7f81a0152690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-26T07:18:05.073 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:05.066+0000 7f819ffff640 1 --2- 172.21.15.143:0/1705342442 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f8188003b10 0x7f81a00ccd70 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7f8190005040 tx=0x7f8188005800 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:18:05.073 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:05.066+0000 7f819cff9640 1 -- 172.21.15.143:0/1705342442 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f81900ba070 con 0x7f8188003b10 2023-11-26T07:18:05.073 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:05.066+0000 7f819cff9640 1 -- 172.21.15.143:0/1705342442 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8190007020 con 0x7f8188003b10 2023-11-26T07:18:05.073 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:05.066+0000 7f819cff9640 1 -- 172.21.15.143:0/1705342442 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f81900073d0 con 0x7f8188003b10 2023-11-26T07:18:05.073 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:05.066+0000 7f81a692a640 1 -- 172.21.15.143:0/1705342442 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f81a00ca480 con 0x7f8188003b10 2023-11-26T07:18:05.074 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:05.066+0000 7f81a692a640 1 -- 172.21.15.143:0/1705342442 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f81a0152bd0 con 0x7f8188003b10 2023-11-26T07:18:05.074 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:05.070+0000 7f81a692a640 1 -- 172.21.15.143:0/1705342442 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f81a00ca2c0 con 0x7f8188003b10 2023-11-26T07:18:05.074 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:05.070+0000 7f819cff9640 1 -- 172.21.15.143:0/1705342442 <== mon.0 v2:172.21.15.143:3300/0 4 ==== mgrmap(e 5) v1 ==== 85852+0+0 (secure 0 0 0) 0x7f8190007530 con 0x7f8188003b10 2023-11-26T07:18:05.075 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:05.070+0000 7f819cff9640 1 --2- 172.21.15.143:0/1705342442 >> [v2:172.21.15.143:6832/2150557544,v1:172.21.15.143:6833/2150557544] conn(0x7f818c0664a0 0x7f818c068960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:18:05.075 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:05.070+0000 7f819cff9640 1 -- 172.21.15.143:0/1705342442 <== mon.0 v2:172.21.15.143:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f819011d750 con 0x7f8188003b10 2023-11-26T07:18:05.075 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:05.070+0000 7f819f7fe640 1 --2- 172.21.15.143:0/1705342442 >> [v2:172.21.15.143:6832/2150557544,v1:172.21.15.143:6833/2150557544] conn(0x7f818c0664a0 0x7f818c068960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:18:05.075 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:05.070+0000 7f819f7fe640 1 --2- 172.21.15.143:0/1705342442 >> [v2:172.21.15.143:6832/2150557544,v1:172.21.15.143:6833/2150557544] conn(0x7f818c0664a0 0x7f818c068960 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7f81a00b4ca0 tx=0x7f8194008040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:18:05.079 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:05.074+0000 7f819cff9640 1 -- 172.21.15.143:0/1705342442 <== mon.0 v2:172.21.15.143:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172649 (secure 0 0 0) 0x7f81900eabb0 con 0x7f8188003b10 2023-11-26T07:18:05.228 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:05.222+0000 7f81a692a640 1 -- 172.21.15.143:0/1705342442 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f81a00ca2c0 con 0x7f8188003b10 2023-11-26T07:18:05.229 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:05.226+0000 7f819cff9640 1 -- 172.21.15.143:0/1705342442 <== mon.0 v2:172.21.15.143:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v20) v1 ==== 100+0+11 (secure 0 0 0) 0x7f81900ee1d0 con 0x7f8188003b10 2023-11-26T07:18:05.229 INFO:teuthology.orchestra.run.smithi143.stdout:pg_num: 64 2023-11-26T07:18:05.231 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:05.226+0000 7f81a692a640 1 -- 172.21.15.143:0/1705342442 >> [v2:172.21.15.143:6832/2150557544,v1:172.21.15.143:6833/2150557544] conn(0x7f818c0664a0 msgr2=0x7f818c068960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:18:05.231 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:05.226+0000 7f81a692a640 1 --2- 172.21.15.143:0/1705342442 >> [v2:172.21.15.143:6832/2150557544,v1:172.21.15.143:6833/2150557544] conn(0x7f818c0664a0 0x7f818c068960 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7f81a00b4ca0 tx=0x7f8194008040 comp rx=0 tx=0).stop 2023-11-26T07:18:05.231 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:05.226+0000 7f81a692a640 1 -- 172.21.15.143:0/1705342442 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f8188003b10 msgr2=0x7f81a00ccd70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:18:05.231 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:05.226+0000 7f81a692a640 1 --2- 172.21.15.143:0/1705342442 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f8188003b10 0x7f81a00ccd70 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7f8190005040 tx=0x7f8188005800 comp rx=0 tx=0).stop 2023-11-26T07:18:05.231 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:05.226+0000 7f81a692a640 1 -- 172.21.15.143:0/1705342442 shutdown_connections 2023-11-26T07:18:05.232 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:05.226+0000 7f81a692a640 1 --2- 172.21.15.143:0/1705342442 >> [v2:172.21.15.143:6832/2150557544,v1:172.21.15.143:6833/2150557544] conn(0x7f818c0664a0 0x7f818c068960 unknown :-1 s=CLOSED pgs=107 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:18:05.232 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:05.226+0000 7f81a692a640 1 --2- 172.21.15.143:0/1705342442 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f81a01f9600 0x7f81a01f9a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:18:05.232 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:05.226+0000 7f81a692a640 1 --2- 172.21.15.143:0/1705342442 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f81a00b7e70 0x7f81a0152690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:18:05.232 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:05.226+0000 7f81a692a640 1 --2- 172.21.15.143:0/1705342442 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f8188003b10 0x7f81a00ccd70 unknown :-1 s=CLOSED pgs=132 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:18:05.232 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:05.226+0000 7f81a692a640 1 -- 172.21.15.143:0/1705342442 >> 172.21.15.143:0/1705342442 conn(0x7f81a009c550 msgr2=0x7f81a00ce820 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:18:05.232 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:05.226+0000 7f81a692a640 1 -- 172.21.15.143:0/1705342442 shutdown_connections 2023-11-26T07:18:05.232 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:05.226+0000 7f81a692a640 1 -- 172.21.15.143:0/1705342442 wait complete. 2023-11-26T07:18:05.250 DEBUG:teuthology.orchestra.run.smithi143:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-11-26T07:18:05.385 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:05.382+0000 7fae135c1640 1 Processor -- start 2023-11-26T07:18:05.385 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:05.382+0000 7fae135c1640 1 -- start start 2023-11-26T07:18:05.387 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:05.382+0000 7fae135c1640 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7fae0c143ac0 0x7fae0c070370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:18:05.388 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:05.382+0000 7fae135c1640 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7fae0c0708b0 0x7fae0c0cf280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:18:05.388 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:05.382+0000 7fae135c1640 1 -- --> v1:172.21.15.143:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fae0c0cc730 con 0x7fae0c0cf7c0 2023-11-26T07:18:05.388 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:05.382+0000 7fae135c1640 1 -- --> v2:172.21.15.143:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fae0c143ea0 con 0x7fae0c0708b0 2023-11-26T07:18:05.388 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:05.382+0000 7fae135c1640 1 -- --> v2:172.21.15.188:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fae0c0cfbd0 con 0x7fae0c143ac0 2023-11-26T07:18:05.388 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:05.382+0000 7fae10b5c640 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7fae0c0708b0 0x7fae0c0cf280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:18:05.388 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:05.382+0000 7fae11b5e640 1 --1- >> v1:172.21.15.143:6789/0 conn(0x7fae0c0cf7c0 0x7fae0c0b1ef0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.143:6789/0 says I am v1:172.21.15.143:52200/0 (socket says 172.21.15.143:52200) 2023-11-26T07:18:05.388 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:05.382+0000 7fae11b5e640 1 -- 172.21.15.143:0/694928977 learned_addr learned my addr 172.21.15.143:0/694928977 (peer_addr_for_me v1:172.21.15.143:0/0) 2023-11-26T07:18:05.388 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:05.382+0000 7fae1135d640 1 --2- 172.21.15.143:0/694928977 >> v2:172.21.15.188:3300/0 conn(0x7fae0c143ac0 0x7fae0c070370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:18:05.388 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:05.382+0000 7fae10b5c640 1 -- 172.21.15.143:0/694928977 >> v2:172.21.15.188:3300/0 conn(0x7fae0c143ac0 msgr2=0x7fae0c070370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-26T07:18:05.388 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:05.382+0000 7fae10b5c640 1 --2- 172.21.15.143:0/694928977 >> v2:172.21.15.188:3300/0 conn(0x7fae0c143ac0 0x7fae0c070370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:18:05.388 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:05.382+0000 7fae10b5c640 1 -- 172.21.15.143:0/694928977 >> v1:172.21.15.143:6789/0 conn(0x7fae0c0cf7c0 legacy=0x7fae0c0b1ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:18:05.388 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:05.382+0000 7fae10b5c640 1 -- 172.21.15.143:0/694928977 --> v2:172.21.15.143:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fae0c070ee0 con 0x7fae0c0708b0 2023-11-26T07:18:05.390 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:05.386+0000 7fae10b5c640 1 --2- 172.21.15.143:0/694928977 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fae0c0708b0 0x7fae0c0cf280 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7fae0c06dca0 tx=0x7fadfc01c460 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5b9696e108be2495 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:18:05.391 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:05.386+0000 7fae03fff640 1 -- 172.21.15.143:0/694928977 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fadfc0ab660 con 0x7fae0c0708b0 2023-11-26T07:18:05.391 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:05.386+0000 7fae03fff640 1 -- 172.21.15.143:0/694928977 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fadfc0ab7c0 con 0x7fae0c0708b0 2023-11-26T07:18:05.391 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:05.386+0000 7fae03fff640 1 -- 172.21.15.143:0/694928977 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fadfc0abb50 con 0x7fae0c0708b0 2023-11-26T07:18:05.392 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:05.386+0000 7fae135c1640 1 -- 172.21.15.143:0/694928977 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fae0c0708b0 msgr2=0x7fae0c0cf280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:18:05.392 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:05.386+0000 7fae135c1640 1 --2- 172.21.15.143:0/694928977 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fae0c0708b0 0x7fae0c0cf280 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7fae0c06dca0 tx=0x7fadfc01c460 comp rx=0 tx=0).stop 2023-11-26T07:18:05.392 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:05.386+0000 7fae135c1640 1 -- 172.21.15.143:0/694928977 shutdown_connections 2023-11-26T07:18:05.392 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:05.386+0000 7fae135c1640 1 --2- 172.21.15.143:0/694928977 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fae0c0708b0 0x7fae0c0cf280 unknown :-1 s=CLOSED pgs=134 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:18:05.392 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:05.386+0000 7fae135c1640 1 --2- 172.21.15.143:0/694928977 >> v2:172.21.15.188:3300/0 conn(0x7fae0c143ac0 0x7fae0c070370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:18:05.392 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:05.386+0000 7fae135c1640 1 -- 172.21.15.143:0/694928977 >> 172.21.15.143:0/694928977 conn(0x7fae0c06a890 msgr2=0x7fae0c06aca0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:18:05.392 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:05.386+0000 7fae135c1640 1 -- 172.21.15.143:0/694928977 shutdown_connections 2023-11-26T07:18:05.392 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:05.386+0000 7fae135c1640 1 -- 172.21.15.143:0/694928977 wait complete. 2023-11-26T07:18:05.393 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:05.390+0000 7fae135c1640 1 Processor -- start 2023-11-26T07:18:05.393 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:05.390+0000 7fae135c1640 1 -- start start 2023-11-26T07:18:05.395 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:05.390+0000 7fae135c1640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fae0c0708b0 0x7fae0c159280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:18:05.396 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:05.390+0000 7fae135c1640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fae0c0cf7c0 0x7fae0c13ad20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:18:05.396 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:05.390+0000 7fae135c1640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fae0c143ac0 0x7fae0c142420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:18:05.396 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:05.390+0000 7fae135c1640 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fae0c1fa570 con 0x7fae0c0708b0 2023-11-26T07:18:05.396 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:05.390+0000 7fae135c1640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fae0c1597c0 con 0x7fae0c0cf7c0 2023-11-26T07:18:05.396 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:05.390+0000 7fae135c1640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fae0c1f98f0 con 0x7fae0c143ac0 2023-11-26T07:18:05.396 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:05.390+0000 7fae1135d640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fae0c0708b0 0x7fae0c159280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:18:05.396 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:05.390+0000 7fae1135d640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fae0c0708b0 0x7fae0c159280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:48192/0 (socket says 172.21.15.143:48192) 2023-11-26T07:18:05.396 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:05.390+0000 7fae1135d640 1 -- 172.21.15.143:0/2271048125 learned_addr learned my addr 172.21.15.143:0/2271048125 (peer_addr_for_me v2:172.21.15.143:0/0) 2023-11-26T07:18:05.396 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:05.390+0000 7fae10b5c640 1 --2- 172.21.15.143:0/2271048125 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fae0c0cf7c0 0x7fae0c13ad20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:18:05.396 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:05.390+0000 7fae11b5e640 1 --2- 172.21.15.143:0/2271048125 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fae0c143ac0 0x7fae0c142420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:18:05.397 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:05.390+0000 7fae1135d640 1 -- 172.21.15.143:0/2271048125 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fae0c143ac0 msgr2=0x7fae0c142420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:18:05.397 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:05.390+0000 7fae1135d640 1 --2- 172.21.15.143:0/2271048125 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fae0c143ac0 0x7fae0c142420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:18:05.397 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:05.390+0000 7fae1135d640 1 -- 172.21.15.143:0/2271048125 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fae0c0cf7c0 msgr2=0x7fae0c13ad20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:18:05.397 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:05.390+0000 7fae1135d640 1 --2- 172.21.15.143:0/2271048125 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fae0c0cf7c0 0x7fae0c13ad20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:18:05.397 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:05.390+0000 7fae1135d640 1 -- 172.21.15.143:0/2271048125 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fadfc009420 con 0x7fae0c0708b0 2023-11-26T07:18:05.397 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:05.390+0000 7fae11b5e640 1 --2- 172.21.15.143:0/2271048125 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fae0c143ac0 0x7fae0c142420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-26T07:18:05.397 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:05.390+0000 7fae1135d640 1 --2- 172.21.15.143:0/2271048125 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fae0c0708b0 0x7fae0c159280 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7fae0800d830 tx=0x7fae0800dcf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:18:05.397 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:05.390+0000 7fae01ffb640 1 -- 172.21.15.143:0/2271048125 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fae0800b6c0 con 0x7fae0c0708b0 2023-11-26T07:18:05.397 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:05.390+0000 7fae01ffb640 1 -- 172.21.15.143:0/2271048125 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fae0800b820 con 0x7fae0c0708b0 2023-11-26T07:18:05.397 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:05.390+0000 7fae01ffb640 1 -- 172.21.15.143:0/2271048125 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fae0800bb90 con 0x7fae0c0708b0 2023-11-26T07:18:05.397 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:05.390+0000 7fae135c1640 1 -- 172.21.15.143:0/2271048125 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fae0c1f9bd0 con 0x7fae0c0708b0 2023-11-26T07:18:05.397 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:05.390+0000 7fae135c1640 1 -- 172.21.15.143:0/2271048125 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fae0c1fa190 con 0x7fae0c0708b0 2023-11-26T07:18:05.398 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:05.394+0000 7fae135c1640 1 -- 172.21.15.143:0/2271048125 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fae0c067680 con 0x7fae0c0708b0 2023-11-26T07:18:05.398 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:05.394+0000 7fae01ffb640 1 -- 172.21.15.143:0/2271048125 <== mon.0 v2:172.21.15.143:3300/0 4 ==== mgrmap(e 5) v1 ==== 85852+0+0 (secure 0 0 0) 0x7fae0801e460 con 0x7fae0c0708b0 2023-11-26T07:18:05.399 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:05.394+0000 7fae01ffb640 1 --2- 172.21.15.143:0/2271048125 >> [v2:172.21.15.143:6832/2150557544,v1:172.21.15.143:6833/2150557544] conn(0x7fadf0066470 0x7fadf0068930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:18:05.399 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:05.394+0000 7fae01ffb640 1 -- 172.21.15.143:0/2271048125 <== mon.0 v2:172.21.15.143:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fae080845d0 con 0x7fae0c0708b0 2023-11-26T07:18:05.399 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:05.394+0000 7fae10b5c640 1 --2- 172.21.15.143:0/2271048125 >> [v2:172.21.15.143:6832/2150557544,v1:172.21.15.143:6833/2150557544] conn(0x7fadf0066470 0x7fadf0068930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:18:05.399 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:05.394+0000 7fae10b5c640 1 --2- 172.21.15.143:0/2271048125 >> [v2:172.21.15.143:6832/2150557544,v1:172.21.15.143:6833/2150557544] conn(0x7fadf0066470 0x7fadf0068930 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7fadfc0af000 tx=0x7fadfc0a7000 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:18:05.402 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:05.398+0000 7fae01ffb640 1 -- 172.21.15.143:0/2271048125 <== mon.0 v2:172.21.15.143:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172649 (secure 0 0 0) 0x7fae08051a40 con 0x7fae0c0708b0 2023-11-26T07:18:05.580 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:05.574+0000 7fae135c1640 1 -- 172.21.15.143:0/2271048125 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fae0c06c6c0 con 0x7fae0c0708b0 2023-11-26T07:18:05.581 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:05.578+0000 7fae01ffb640 1 -- 172.21.15.143:0/2271048125 <== mon.0 v2:172.21.15.143:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 8 v8) v1 ==== 93+0+4913 (secure 0 0 0) 0x7fae08055060 con 0x7fae0c0708b0 2023-11-26T07:18:05.581 INFO:teuthology.orchestra.run.smithi143.stdout: 2023-11-26T07:18:05.581 INFO:teuthology.orchestra.run.smithi143.stdout:{"epoch":8,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4241,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.188:6834/652687322","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6832","nonce":652687322},{"type":"v1","addr":"172.21.15.188:6834","nonce":652687322}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":4266,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.143:6835/3850184704","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6834","nonce":3850184704},{"type":"v1","addr":"172.21.15.143:6835","nonce":3850184704}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":2},{"gid":4273,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.143:6837/1695478651","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6836","nonce":1695478651},{"type":"v1","addr":"172.21.15.143:6837","nonce":1695478651}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":8,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-11-26T07:17:53.876989+0000","modified":"2023-11-26T07:17:56.885608+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":4285},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4285":{"gid":4285,"name":"b","rank":0,"incarnation":4,"state":"up:active","state_seq":3,"addr":"172.21.15.188:6835/994617268","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6833","nonce":994617268},{"type":"v1","addr":"172.21.15.188:6835","nonce":994617268}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2023-11-26T07:18:05.582 INFO:teuthology.orchestra.run.smithi143.stderr:dumped fsmap epoch 8 2023-11-26T07:18:05.583 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:05.578+0000 7fae135c1640 1 -- 172.21.15.143:0/2271048125 >> [v2:172.21.15.143:6832/2150557544,v1:172.21.15.143:6833/2150557544] conn(0x7fadf0066470 msgr2=0x7fadf0068930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:18:05.583 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:05.578+0000 7fae135c1640 1 --2- 172.21.15.143:0/2271048125 >> [v2:172.21.15.143:6832/2150557544,v1:172.21.15.143:6833/2150557544] conn(0x7fadf0066470 0x7fadf0068930 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7fadfc0af000 tx=0x7fadfc0a7000 comp rx=0 tx=0).stop 2023-11-26T07:18:05.584 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:05.578+0000 7fae135c1640 1 -- 172.21.15.143:0/2271048125 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fae0c0708b0 msgr2=0x7fae0c159280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:18:05.584 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:05.578+0000 7fae135c1640 1 --2- 172.21.15.143:0/2271048125 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fae0c0708b0 0x7fae0c159280 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7fae0800d830 tx=0x7fae0800dcf0 comp rx=0 tx=0).stop 2023-11-26T07:18:05.584 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:05.578+0000 7fae135c1640 1 -- 172.21.15.143:0/2271048125 shutdown_connections 2023-11-26T07:18:05.584 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:05.578+0000 7fae135c1640 1 --2- 172.21.15.143:0/2271048125 >> [v2:172.21.15.143:6832/2150557544,v1:172.21.15.143:6833/2150557544] conn(0x7fadf0066470 0x7fadf0068930 unknown :-1 s=CLOSED pgs=108 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:18:05.584 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:05.578+0000 7fae135c1640 1 --2- 172.21.15.143:0/2271048125 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fae0c143ac0 0x7fae0c142420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:18:05.584 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:05.578+0000 7fae135c1640 1 --2- 172.21.15.143:0/2271048125 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fae0c0cf7c0 0x7fae0c13ad20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:18:05.584 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:05.578+0000 7fae135c1640 1 --2- 172.21.15.143:0/2271048125 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fae0c0708b0 0x7fae0c159280 unknown :-1 s=CLOSED pgs=135 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:18:05.584 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:05.578+0000 7fae135c1640 1 -- 172.21.15.143:0/2271048125 >> 172.21.15.143:0/2271048125 conn(0x7fae0c06a890 msgr2=0x7fae0c06ca30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:18:05.584 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:05.578+0000 7fae135c1640 1 -- 172.21.15.143:0/2271048125 shutdown_connections 2023-11-26T07:18:05.584 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:05.578+0000 7fae135c1640 1 -- 172.21.15.143:0/2271048125 wait complete. 2023-11-26T07:18:05.603 DEBUG:teuthology.orchestra.run.smithi143:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-11-26T07:18:05.741 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:05.734+0000 7f7f605df640 1 Processor -- start 2023-11-26T07:18:05.741 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:05.734+0000 7f7f605df640 1 -- start start 2023-11-26T07:18:05.742 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:05.738+0000 7f7f605df640 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7f7f58143ac0 0x7f7f58070370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:18:05.742 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:05.738+0000 7f7f605df640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f7f580cf990 0x7f7f580cd1b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:18:05.742 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:05.738+0000 7f7f605df640 1 -- --> v1:172.21.15.188:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7f580cc730 con 0x7f7f580708b0 2023-11-26T07:18:05.742 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:05.738+0000 7f7f605df640 1 -- --> v2:172.21.15.188:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7f58143ea0 con 0x7f7f58143ac0 2023-11-26T07:18:05.742 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:05.738+0000 7f7f605df640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7f58070ee0 con 0x7f7f580cf990 2023-11-26T07:18:05.743 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:05.738+0000 7f7f5e37b640 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7f7f58143ac0 0x7f7f58070370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:18:05.743 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:05.738+0000 7f7f5db7a640 1 --1- >> v1:172.21.15.188:6789/0 conn(0x7f7f580708b0 0x7f7f580cf280 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.188:6789/0 says I am v1:172.21.15.143:39214/0 (socket says 172.21.15.143:39214) 2023-11-26T07:18:05.743 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:05.738+0000 7f7f5eb7c640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f7f580cf990 0x7f7f580cd1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:18:05.743 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:05.738+0000 7f7f5db7a640 1 -- 172.21.15.143:0/2214876175 learned_addr learned my addr 172.21.15.143:0/2214876175 (peer_addr_for_me v1:172.21.15.143:0/0) 2023-11-26T07:18:05.743 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:05.738+0000 7f7f5d379640 1 -- 172.21.15.143:0/2214876175 <== mon.1 v1:172.21.15.188:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 998492193 0 0) 0x7f7f580cc730 con 0x7f7f580708b0 2023-11-26T07:18:05.743 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:05.738+0000 7f7f5d379640 1 -- 172.21.15.143:0/2214876175 --> v1:172.21.15.188:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f7f44003530 con 0x7f7f580708b0 2023-11-26T07:18:05.744 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:05.738+0000 7f7f5e37b640 1 -- 172.21.15.143:0/2214876175 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f7f580cf990 msgr2=0x7f7f580cd1b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-26T07:18:05.744 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:05.738+0000 7f7f5e37b640 1 --2- 172.21.15.143:0/2214876175 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f7f580cf990 0x7f7f580cd1b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:18:05.744 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:05.738+0000 7f7f5e37b640 1 -- 172.21.15.143:0/2214876175 >> v1:172.21.15.188:6789/0 conn(0x7f7f580708b0 legacy=0x7f7f580cf280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:18:05.744 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:05.738+0000 7f7f5e37b640 1 -- 172.21.15.143:0/2214876175 --> v2:172.21.15.188:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7f580711c0 con 0x7f7f58143ac0 2023-11-26T07:18:05.744 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:05.738+0000 7f7f5eb7c640 1 --2- 172.21.15.143:0/2214876175 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f7f580cf990 0x7f7f580cd1b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-26T07:18:05.746 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:05.742+0000 7f7f5e37b640 1 --2- 172.21.15.143:0/2214876175 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f7f58143ac0 0x7f7f58070370 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7f7f540097c0 tx=0x7f7f5401c480 comp rx=0 tx=0).ready entity=mon.1 client_cookie=cf46777cf986e63c server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:18:05.746 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:05.742+0000 7f7f5d379640 1 -- 172.21.15.143:0/2214876175 <== mon.1 v2:172.21.15.188:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7f540ad640 con 0x7f7f58143ac0 2023-11-26T07:18:05.746 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:05.742+0000 7f7f5d379640 1 -- 172.21.15.143:0/2214876175 <== mon.1 v2:172.21.15.188:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7f540ad7a0 con 0x7f7f58143ac0 2023-11-26T07:18:05.746 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:05.742+0000 7f7f5d379640 1 -- 172.21.15.143:0/2214876175 <== mon.1 v2:172.21.15.188:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7f540adb30 con 0x7f7f58143ac0 2023-11-26T07:18:05.747 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:05.742+0000 7f7f605df640 1 -- 172.21.15.143:0/2214876175 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f7f58143ac0 msgr2=0x7f7f58070370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:18:05.747 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:05.742+0000 7f7f605df640 1 --2- 172.21.15.143:0/2214876175 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f7f58143ac0 0x7f7f58070370 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7f7f540097c0 tx=0x7f7f5401c480 comp rx=0 tx=0).stop 2023-11-26T07:18:05.747 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:05.742+0000 7f7f605df640 1 -- 172.21.15.143:0/2214876175 shutdown_connections 2023-11-26T07:18:05.747 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:05.742+0000 7f7f605df640 1 --2- 172.21.15.143:0/2214876175 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f7f580cf990 0x7f7f580cd1b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:18:05.748 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:05.742+0000 7f7f605df640 1 --2- 172.21.15.143:0/2214876175 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f7f58143ac0 0x7f7f58070370 unknown :-1 s=CLOSED pgs=99 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:18:05.748 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:05.742+0000 7f7f605df640 1 -- 172.21.15.143:0/2214876175 >> 172.21.15.143:0/2214876175 conn(0x7f7f5806a890 msgr2=0x7f7f5806aca0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:18:05.748 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:05.742+0000 7f7f605df640 1 -- 172.21.15.143:0/2214876175 shutdown_connections 2023-11-26T07:18:05.748 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:05.742+0000 7f7f605df640 1 -- 172.21.15.143:0/2214876175 wait complete. 2023-11-26T07:18:05.749 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:05.742+0000 7f7f605df640 1 Processor -- start 2023-11-26T07:18:05.749 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:05.742+0000 7f7f605df640 1 -- start start 2023-11-26T07:18:05.751 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:05.746+0000 7f7f605df640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f7f580708b0 0x7f7f5815bdc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:18:05.751 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:05.746+0000 7f7f605df640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f7f580cf990 0x7f7f5813a990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:18:05.751 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:05.746+0000 7f7f605df640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f7f58143ac0 0x7f7f580b0150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:18:05.752 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:05.746+0000 7f7f605df640 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7f5815c300 con 0x7f7f58143ac0 2023-11-26T07:18:05.752 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:05.746+0000 7f7f605df640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7f580b13d0 con 0x7f7f580708b0 2023-11-26T07:18:05.752 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:05.746+0000 7f7f605df640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7f5814dfd0 con 0x7f7f580cf990 2023-11-26T07:18:05.752 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:05.746+0000 7f7f5eb7c640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f7f58143ac0 0x7f7f580b0150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:18:05.752 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:05.746+0000 7f7f5eb7c640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f7f58143ac0 0x7f7f580b0150 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:48196/0 (socket says 172.21.15.143:48196) 2023-11-26T07:18:05.752 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:05.746+0000 7f7f5eb7c640 1 -- 172.21.15.143:0/2357922524 learned_addr learned my addr 172.21.15.143:0/2357922524 (peer_addr_for_me v2:172.21.15.143:0/0) 2023-11-26T07:18:05.752 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:05.746+0000 7f7f5db7a640 1 --2- 172.21.15.143:0/2357922524 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f7f580cf990 0x7f7f5813a990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:18:05.752 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:05.746+0000 7f7f5e37b640 1 --2- 172.21.15.143:0/2357922524 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f7f580708b0 0x7f7f5815bdc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:18:05.754 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:05.746+0000 7f7f5eb7c640 1 -- 172.21.15.143:0/2357922524 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f7f580cf990 msgr2=0x7f7f5813a990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:18:05.754 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:05.746+0000 7f7f5eb7c640 1 --2- 172.21.15.143:0/2357922524 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f7f580cf990 0x7f7f5813a990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:18:05.754 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:05.746+0000 7f7f5eb7c640 1 -- 172.21.15.143:0/2357922524 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f7f580708b0 msgr2=0x7f7f5815bdc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:18:05.754 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:05.746+0000 7f7f5eb7c640 1 --2- 172.21.15.143:0/2357922524 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f7f580708b0 0x7f7f5815bdc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:18:05.754 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:05.746+0000 7f7f5eb7c640 1 -- 172.21.15.143:0/2357922524 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7f50009a70 con 0x7f7f58143ac0 2023-11-26T07:18:05.755 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:05.746+0000 7f7f5eb7c640 1 --2- 172.21.15.143:0/2357922524 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f7f58143ac0 0x7f7f580b0150 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7f7f50009f00 tx=0x7f7f5000c490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:18:05.755 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:05.746+0000 7f7f4effd640 1 -- 172.21.15.143:0/2357922524 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7f50009d70 con 0x7f7f58143ac0 2023-11-26T07:18:05.755 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:05.746+0000 7f7f4effd640 1 -- 172.21.15.143:0/2357922524 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7f5000f040 con 0x7f7f58143ac0 2023-11-26T07:18:05.755 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:05.746+0000 7f7f605df640 1 -- 172.21.15.143:0/2357922524 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7f54009420 con 0x7f7f58143ac0 2023-11-26T07:18:05.755 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:05.746+0000 7f7f4effd640 1 -- 172.21.15.143:0/2357922524 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7f50003bc0 con 0x7f7f58143ac0 2023-11-26T07:18:05.756 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:05.746+0000 7f7f605df640 1 -- 172.21.15.143:0/2357922524 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7f5813b690 con 0x7f7f58143ac0 2023-11-26T07:18:05.756 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:05.746+0000 7f7f605df640 1 -- 172.21.15.143:0/2357922524 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7f580b1b20 con 0x7f7f58143ac0 2023-11-26T07:18:05.756 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:05.750+0000 7f7f4effd640 1 -- 172.21.15.143:0/2357922524 <== mon.0 v2:172.21.15.143:3300/0 4 ==== mgrmap(e 5) v1 ==== 85852+0+0 (secure 0 0 0) 0x7f7f50014480 con 0x7f7f58143ac0 2023-11-26T07:18:05.757 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:05.750+0000 7f7f4effd640 1 --2- 172.21.15.143:0/2357922524 >> [v2:172.21.15.143:6832/2150557544,v1:172.21.15.143:6833/2150557544] conn(0x7f7f3c066470 0x7f7f3c068930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:18:05.757 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:05.750+0000 7f7f4effd640 1 -- 172.21.15.143:0/2357922524 <== mon.0 v2:172.21.15.143:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f7f50084ad0 con 0x7f7f58143ac0 2023-11-26T07:18:05.757 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:05.750+0000 7f7f5e37b640 1 --2- 172.21.15.143:0/2357922524 >> [v2:172.21.15.143:6832/2150557544,v1:172.21.15.143:6833/2150557544] conn(0x7f7f3c066470 0x7f7f3c068930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:18:05.757 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:05.750+0000 7f7f5e37b640 1 --2- 172.21.15.143:0/2357922524 >> [v2:172.21.15.143:6832/2150557544,v1:172.21.15.143:6833/2150557544] conn(0x7f7f3c066470 0x7f7f3c068930 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7f7f540af000 tx=0x7f7f54008040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:18:05.759 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:05.754+0000 7f7f4effd640 1 -- 172.21.15.143:0/2357922524 <== mon.0 v2:172.21.15.143:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172649 (secure 0 0 0) 0x7f7f50051ed0 con 0x7f7f58143ac0 2023-11-26T07:18:05.907 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:05.902+0000 7f7f605df640 1 -- 172.21.15.143:0/2357922524 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f7f580b1580 con 0x7f7f58143ac0 2023-11-26T07:18:05.908 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:05.902+0000 7f7f4effd640 1 -- 172.21.15.143:0/2357922524 <== mon.0 v2:172.21.15.143:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v20) v1 ==== 74+0+16541 (secure 0 0 0) 0x7f7f580b1580 con 0x7f7f58143ac0 2023-11-26T07:18:05.908 INFO:teuthology.orchestra.run.smithi143.stdout: 2023-11-26T07:18:05.908 INFO:teuthology.orchestra.run.smithi143.stdout:{"epoch":20,"fsid":"f5033060-2469-4a07-95fe-2b76f5404bc1","created":"2023-11-26T07:17:28.613663+0000","modified":"2023-11-26T07:17:55.505435+0000","last_up_change":"2023-11-26T07:17:41.157813+0000","last_in_change":"2023-11-26T07:17:31.181658+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-11-26T07:17:41.758597+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2023-11-26T07:17:51.878332+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2023-11-26T07:17:52.871396+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"ebbee2c6-023e-44b9-9f1d-6f7bfd309730","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6817","nonce":639807369},{"type":"v1","addr":"172.21.15.143:6819","nonce":639807369}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6821","nonce":639807369},{"type":"v1","addr":"172.21.15.143:6823","nonce":639807369}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6830","nonce":639807369},{"type":"v1","addr":"172.21.15.143:6831","nonce":639807369}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6825","nonce":639807369},{"type":"v1","addr":"172.21.15.143:6828","nonce":639807369}]},"public_addr":"172.21.15.143:6819/639807369","cluster_addr":"172.21.15.143:6823/639807369","heartbeat_back_addr":"172.21.15.143:6831/639807369","heartbeat_front_addr":"172.21.15.143:6828/639807369","state":["exists","up"]},{"osd":1,"uuid":"fec4b8be-7122-49e9-9a19-5850a0700001","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6816","nonce":1144942530},{"type":"v1","addr":"172.21.15.143:6818","nonce":1144942530}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6820","nonce":1144942530},{"type":"v1","addr":"172.21.15.143:6822","nonce":1144942530}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6827","nonce":1144942530},{"type":"v1","addr":"172.21.15.143:6829","nonce":1144942530}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6824","nonce":1144942530},{"type":"v1","addr":"172.21.15.143:6826","nonce":1144942530}]},"public_addr":"172.21.15.143:6818/1144942530","cluster_addr":"172.21.15.143:6822/1144942530","heartbeat_back_addr":"172.21.15.143:6829/1144942530","heartbeat_front_addr":"172.21.15.143:6826/1144942530","state":["exists","up"]},{"osd":2,"uuid":"2888af5b-f933-46ca-8f95-6dd494ff43f6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6808","nonce":3909095186},{"type":"v1","addr":"172.21.15.143:6809","nonce":3909095186}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6810","nonce":3909095186},{"type":"v1","addr":"172.21.15.143:6811","nonce":3909095186}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6814","nonce":3909095186},{"type":"v1","addr":"172.21.15.143:6815","nonce":3909095186}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6812","nonce":3909095186},{"type":"v1","addr":"172.21.15.143:6813","nonce":3909095186}]},"public_addr":"172.21.15.143:6809/3909095186","cluster_addr":"172.21.15.143:6811/3909095186","heartbeat_back_addr":"172.21.15.143:6815/3909095186","heartbeat_front_addr":"172.21.15.143:6813/3909095186","state":["exists","up"]},{"osd":3,"uuid":"d1f52435-7bc1-4081-9550-9cd1ba55f549","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6800","nonce":3854210991},{"type":"v1","addr":"172.21.15.143:6801","nonce":3854210991}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6802","nonce":3854210991},{"type":"v1","addr":"172.21.15.143:6803","nonce":3854210991}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6806","nonce":3854210991},{"type":"v1","addr":"172.21.15.143:6807","nonce":3854210991}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6804","nonce":3854210991},{"type":"v1","addr":"172.21.15.143:6805","nonce":3854210991}]},"public_addr":"172.21.15.143:6801/3854210991","cluster_addr":"172.21.15.143:6803/3854210991","heartbeat_back_addr":"172.21.15.143:6807/3854210991","heartbeat_front_addr":"172.21.15.143:6805/3854210991","state":["exists","up"]},{"osd":4,"uuid":"e2ae8b6e-6e25-49b2-bcbf-aacce68f06a3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6800","nonce":3924606902},{"type":"v1","addr":"172.21.15.188:6801","nonce":3924606902}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6802","nonce":3924606902},{"type":"v1","addr":"172.21.15.188:6803","nonce":3924606902}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6806","nonce":3924606902},{"type":"v1","addr":"172.21.15.188:6807","nonce":3924606902}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6804","nonce":3924606902},{"type":"v1","addr":"172.21.15.188:6805","nonce":3924606902}]},"public_addr":"172.21.15.188:6801/3924606902","cluster_addr":"172.21.15.188:6803/3924606902","heartbeat_back_addr":"172.21.15.188:6807/3924606902","heartbeat_front_addr":"172.21.15.188:6805/3924606902","state":["exists","up"]},{"osd":5,"uuid":"3cec8a32-7698-4c36-8e89-e4ba11934f33","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6813","nonce":186841544},{"type":"v1","addr":"172.21.15.188:6815","nonce":186841544}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6817","nonce":186841544},{"type":"v1","addr":"172.21.15.188:6818","nonce":186841544}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6822","nonce":186841544},{"type":"v1","addr":"172.21.15.188:6823","nonce":186841544}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6820","nonce":186841544},{"type":"v1","addr":"172.21.15.188:6821","nonce":186841544}]},"public_addr":"172.21.15.188:6815/186841544","cluster_addr":"172.21.15.188:6818/186841544","heartbeat_back_addr":"172.21.15.188:6823/186841544","heartbeat_front_addr":"172.21.15.188:6821/186841544","state":["exists","up"]},{"osd":6,"uuid":"58a65d5d-e090-473a-81b4-ad38e85029f9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6808","nonce":2153499405},{"type":"v1","addr":"172.21.15.188:6809","nonce":2153499405}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6810","nonce":2153499405},{"type":"v1","addr":"172.21.15.188:6811","nonce":2153499405}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6816","nonce":2153499405},{"type":"v1","addr":"172.21.15.188:6819","nonce":2153499405}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6812","nonce":2153499405},{"type":"v1","addr":"172.21.15.188:6814","nonce":2153499405}]},"public_addr":"172.21.15.188:6809/2153499405","cluster_addr":"172.21.15.188:6811/2153499405","heartbeat_back_addr":"172.21.15.188:6819/2153499405","heartbeat_front_addr":"172.21.15.188:6814/2153499405","state":["exists","up"]},{"osd":7,"uuid":"ea5b7f70-3554-4cf8-ac30-f7c59b80678d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6824","nonce":2722096729},{"type":"v1","addr":"172.21.15.188:6825","nonce":2722096729}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6826","nonce":2722096729},{"type":"v1","addr":"172.21.15.188:6827","nonce":2722096729}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6830","nonce":2722096729},{"type":"v1","addr":"172.21.15.188:6831","nonce":2722096729}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6828","nonce":2722096729},{"type":"v1","addr":"172.21.15.188:6829","nonce":2722096729}]},"public_addr":"172.21.15.188:6825/2722096729","cluster_addr":"172.21.15.188:6827/2722096729","heartbeat_back_addr":"172.21.15.188:6831/2722096729","heartbeat_front_addr":"172.21.15.188:6829/2722096729","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-26T07:17:36.995032+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-26T07:17:36.932266+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-26T07:17:36.829304+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-26T07:17:36.804297+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-26T07:17:36.667892+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-26T07:17:36.964868+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-26T07:17:36.827332+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-26T07:17:36.893179+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-11-26T07:18:05.910 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:05.906+0000 7f7f605df640 1 -- 172.21.15.143:0/2357922524 >> [v2:172.21.15.143:6832/2150557544,v1:172.21.15.143:6833/2150557544] conn(0x7f7f3c066470 msgr2=0x7f7f3c068930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:18:05.910 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:05.906+0000 7f7f605df640 1 --2- 172.21.15.143:0/2357922524 >> [v2:172.21.15.143:6832/2150557544,v1:172.21.15.143:6833/2150557544] conn(0x7f7f3c066470 0x7f7f3c068930 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7f7f540af000 tx=0x7f7f54008040 comp rx=0 tx=0).stop 2023-11-26T07:18:05.910 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:05.906+0000 7f7f605df640 1 -- 172.21.15.143:0/2357922524 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f7f58143ac0 msgr2=0x7f7f580b0150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:18:05.910 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:05.906+0000 7f7f605df640 1 --2- 172.21.15.143:0/2357922524 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f7f58143ac0 0x7f7f580b0150 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7f7f50009f00 tx=0x7f7f5000c490 comp rx=0 tx=0).stop 2023-11-26T07:18:05.910 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:05.906+0000 7f7f605df640 1 -- 172.21.15.143:0/2357922524 shutdown_connections 2023-11-26T07:18:05.911 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:05.906+0000 7f7f605df640 1 --2- 172.21.15.143:0/2357922524 >> [v2:172.21.15.143:6832/2150557544,v1:172.21.15.143:6833/2150557544] conn(0x7f7f3c066470 0x7f7f3c068930 unknown :-1 s=CLOSED pgs=109 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:18:05.911 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:05.906+0000 7f7f605df640 1 --2- 172.21.15.143:0/2357922524 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f7f58143ac0 0x7f7f580b0150 unknown :-1 s=CLOSED pgs=136 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:18:05.911 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:05.906+0000 7f7f605df640 1 --2- 172.21.15.143:0/2357922524 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f7f580cf990 0x7f7f5813a990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:18:05.911 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:05.906+0000 7f7f605df640 1 --2- 172.21.15.143:0/2357922524 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f7f580708b0 0x7f7f5815bdc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:18:05.911 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:05.906+0000 7f7f605df640 1 -- 172.21.15.143:0/2357922524 >> 172.21.15.143:0/2357922524 conn(0x7f7f5806a890 msgr2=0x7f7f5806cc00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:18:05.911 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:05.906+0000 7f7f605df640 1 -- 172.21.15.143:0/2357922524 shutdown_connections 2023-11-26T07:18:05.911 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:05.906+0000 7f7f605df640 1 -- 172.21.15.143:0/2357922524 wait complete. 2023-11-26T07:18:05.930 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-11-26T07:17:41.758597+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2023-11-26T07:17:51.878332+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2023-11-26T07:17:52.871396+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '20', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2023-11-26T07:18:05.933 DEBUG:teuthology.orchestra.run.smithi143:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-11-26T07:18:06.077 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:06.074+0000 7f5183216640 1 Processor -- start 2023-11-26T07:18:06.077 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:06.074+0000 7f5183216640 1 -- start start 2023-11-26T07:18:06.079 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:06.074+0000 7f5183216640 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7f517c1614b0 0x7f517c0cd610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:18:06.079 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:06.074+0000 7f5183216640 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7f517c071970 0x7f517c161890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:18:06.079 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:06.074+0000 7f5183216640 1 -- --> v1:172.21.15.143:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f517c15d970 con 0x7f517c070ba0 2023-11-26T07:18:06.079 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:06.074+0000 7f5183216640 1 -- --> v2:172.21.15.143:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f517c161dd0 con 0x7f517c1614b0 2023-11-26T07:18:06.079 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:06.074+0000 7f5183216640 1 -- --> v2:172.21.15.188:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f517c15dc30 con 0x7f517c071970 2023-11-26T07:18:06.079 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:06.074+0000 7f5180fb2640 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7f517c1614b0 0x7f517c0cd610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:18:06.080 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:06.074+0000 7f51817b3640 1 --1- >> v1:172.21.15.143:6789/0 conn(0x7f517c070ba0 0x7f517c06dd10 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.143:6789/0 says I am v1:172.21.15.143:52214/0 (socket says 172.21.15.143:52214) 2023-11-26T07:18:06.080 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:06.074+0000 7f51817b3640 1 -- 172.21.15.143:0/665454661 learned_addr learned my addr 172.21.15.143:0/665454661 (peer_addr_for_me v1:172.21.15.143:0/0) 2023-11-26T07:18:06.081 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:06.074+0000 7f5173fff640 1 --2- 172.21.15.143:0/665454661 >> v2:172.21.15.188:3300/0 conn(0x7f517c071970 0x7f517c161890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:18:06.081 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:06.074+0000 7f5180fb2640 1 -- 172.21.15.143:0/665454661 >> v2:172.21.15.188:3300/0 conn(0x7f517c071970 msgr2=0x7f517c161890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-26T07:18:06.081 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:06.074+0000 7f5180fb2640 1 --2- 172.21.15.143:0/665454661 >> v2:172.21.15.188:3300/0 conn(0x7f517c071970 0x7f517c161890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:18:06.081 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:06.074+0000 7f5180fb2640 1 -- 172.21.15.143:0/665454661 >> v1:172.21.15.143:6789/0 conn(0x7f517c070ba0 legacy=0x7f517c06dd10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:18:06.081 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:06.074+0000 7f5180fb2640 1 -- 172.21.15.143:0/665454661 --> v2:172.21.15.143:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f517c15df10 con 0x7f517c1614b0 2023-11-26T07:18:06.081 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:06.074+0000 7f51737fe640 1 -- 172.21.15.143:0/665454661 <== mon.0 v1:172.21.15.143:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 599271935 0 0) 0x7f517c15d970 con 0x7f517c070ba0 2023-11-26T07:18:06.083 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:06.078+0000 7f5180fb2640 1 --2- 172.21.15.143:0/665454661 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f517c1614b0 0x7f517c0cd610 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7f517c06d3c0 tx=0x7f517801c430 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c445d0d3760df174 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:18:06.083 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:06.078+0000 7f51737fe640 1 -- 172.21.15.143:0/665454661 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f51780ab600 con 0x7f517c1614b0 2023-11-26T07:18:06.084 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:06.078+0000 7f51737fe640 1 -- 172.21.15.143:0/665454661 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f51780ab760 con 0x7f517c1614b0 2023-11-26T07:18:06.084 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:06.078+0000 7f51737fe640 1 -- 172.21.15.143:0/665454661 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f51780abaf0 con 0x7f517c1614b0 2023-11-26T07:18:06.084 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:06.078+0000 7f5183216640 1 -- 172.21.15.143:0/665454661 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f517c1614b0 msgr2=0x7f517c0cd610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:18:06.084 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:06.078+0000 7f5183216640 1 --2- 172.21.15.143:0/665454661 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f517c1614b0 0x7f517c0cd610 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7f517c06d3c0 tx=0x7f517801c430 comp rx=0 tx=0).stop 2023-11-26T07:18:06.084 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:06.078+0000 7f5183216640 1 -- 172.21.15.143:0/665454661 shutdown_connections 2023-11-26T07:18:06.084 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:06.078+0000 7f5183216640 1 --2- 172.21.15.143:0/665454661 >> v2:172.21.15.188:3300/0 conn(0x7f517c071970 0x7f517c161890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:18:06.084 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:06.078+0000 7f5183216640 1 --2- 172.21.15.143:0/665454661 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f517c1614b0 0x7f517c0cd610 unknown :-1 s=CLOSED pgs=138 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:18:06.085 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:06.078+0000 7f5183216640 1 -- 172.21.15.143:0/665454661 >> 172.21.15.143:0/665454661 conn(0x7f517c0cf710 msgr2=0x7f517c0cfb20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:18:06.085 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:06.078+0000 7f5183216640 1 -- 172.21.15.143:0/665454661 shutdown_connections 2023-11-26T07:18:06.085 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:06.078+0000 7f5183216640 1 -- 172.21.15.143:0/665454661 wait complete. 2023-11-26T07:18:06.088 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:06.078+0000 7f5183216640 1 Processor -- start 2023-11-26T07:18:06.088 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:06.078+0000 7f5183216640 1 -- start start 2023-11-26T07:18:06.088 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:06.082+0000 7f5183216640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f517c070ba0 0x7f517c155c40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:18:06.088 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:06.082+0000 7f5183216640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f517c071970 0x7f517c156180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:18:06.089 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:06.082+0000 7f5183216640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f517c1614b0 0x7f517c1566c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:18:06.089 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:06.082+0000 7f5183216640 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f517c0ad660 con 0x7f517c070ba0 2023-11-26T07:18:06.089 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:06.082+0000 7f5183216640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f517c0ad940 con 0x7f517c071970 2023-11-26T07:18:06.089 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:06.082+0000 7f5183216640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f517c0adc20 con 0x7f517c1614b0 2023-11-26T07:18:06.089 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:06.082+0000 7f5180fb2640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f517c070ba0 0x7f517c155c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:18:06.089 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:06.082+0000 7f5180fb2640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f517c070ba0 0x7f517c155c40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:48216/0 (socket says 172.21.15.143:48216) 2023-11-26T07:18:06.089 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:06.082+0000 7f5180fb2640 1 -- 172.21.15.143:0/640996201 learned_addr learned my addr 172.21.15.143:0/640996201 (peer_addr_for_me v2:172.21.15.143:0/0) 2023-11-26T07:18:06.089 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:06.082+0000 7f51817b3640 1 --2- 172.21.15.143:0/640996201 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f517c1614b0 0x7f517c1566c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:18:06.090 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:06.082+0000 7f5173fff640 1 --2- 172.21.15.143:0/640996201 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f517c071970 0x7f517c156180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:18:06.090 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:06.082+0000 7f5180fb2640 1 -- 172.21.15.143:0/640996201 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f517c1614b0 msgr2=0x7f517c1566c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:18:06.090 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:06.082+0000 7f5180fb2640 1 --2- 172.21.15.143:0/640996201 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f517c1614b0 0x7f517c1566c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:18:06.090 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:06.082+0000 7f5180fb2640 1 -- 172.21.15.143:0/640996201 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f517c071970 msgr2=0x7f517c156180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:18:06.090 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:06.082+0000 7f5180fb2640 1 --2- 172.21.15.143:0/640996201 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f517c071970 0x7f517c156180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:18:06.090 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:06.082+0000 7f5180fb2640 1 -- 172.21.15.143:0/640996201 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5178009420 con 0x7f517c070ba0 2023-11-26T07:18:06.090 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:06.082+0000 7f5180fb2640 1 --2- 172.21.15.143:0/640996201 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f517c070ba0 0x7f517c155c40 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7f5178019040 tx=0x7f517801ac70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:18:06.090 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:06.082+0000 7f51717fa640 1 -- 172.21.15.143:0/640996201 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f51780b5810 con 0x7f517c070ba0 2023-11-26T07:18:06.090 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:06.082+0000 7f51717fa640 1 -- 172.21.15.143:0/640996201 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f51780b5970 con 0x7f517c070ba0 2023-11-26T07:18:06.091 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:06.082+0000 7f5183216640 1 -- 172.21.15.143:0/640996201 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f517c071540 con 0x7f517c070ba0 2023-11-26T07:18:06.091 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:06.082+0000 7f51717fa640 1 -- 172.21.15.143:0/640996201 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f51780b5d20 con 0x7f517c070ba0 2023-11-26T07:18:06.091 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:06.082+0000 7f5183216640 1 -- 172.21.15.143:0/640996201 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f517c0ae1e0 con 0x7f517c070ba0 2023-11-26T07:18:06.091 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:06.082+0000 7f5183216640 1 -- 172.21.15.143:0/640996201 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f517c067680 con 0x7f517c070ba0 2023-11-26T07:18:06.092 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:06.086+0000 7f51717fa640 1 -- 172.21.15.143:0/640996201 <== mon.0 v2:172.21.15.143:3300/0 4 ==== mgrmap(e 5) v1 ==== 85852+0+0 (secure 0 0 0) 0x7f51780bf410 con 0x7f517c070ba0 2023-11-26T07:18:06.092 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:06.086+0000 7f51717fa640 1 --2- 172.21.15.143:0/640996201 >> [v2:172.21.15.143:6832/2150557544,v1:172.21.15.143:6833/2150557544] conn(0x7f5168066470 0x7f5168068930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:18:06.092 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:06.086+0000 7f51717fa640 1 -- 172.21.15.143:0/640996201 <== mon.0 v2:172.21.15.143:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f517811ec20 con 0x7f517c070ba0 2023-11-26T07:18:06.093 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:06.086+0000 7f5173fff640 1 --2- 172.21.15.143:0/640996201 >> [v2:172.21.15.143:6832/2150557544,v1:172.21.15.143:6833/2150557544] conn(0x7f5168066470 0x7f5168068930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:18:06.093 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:06.086+0000 7f5173fff640 1 --2- 172.21.15.143:0/640996201 >> [v2:172.21.15.143:6832/2150557544,v1:172.21.15.143:6833/2150557544] conn(0x7f5168066470 0x7f5168068930 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7f517c0cf130 tx=0x7f516c008040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:18:06.094 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:06.090+0000 7f51717fa640 1 -- 172.21.15.143:0/640996201 <== mon.0 v2:172.21.15.143:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172649 (secure 0 0 0) 0x7f51780ec1a0 con 0x7f517c070ba0 2023-11-26T07:18:06.243 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:06.238+0000 7f5183216640 1 -- 172.21.15.143:0/640996201 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f517c15eb40 con 0x7f517c070ba0 2023-11-26T07:18:06.243 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:06.238+0000 7f51717fa640 1 -- 172.21.15.143:0/640996201 <== mon.0 v2:172.21.15.143:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v20) v1 ==== 93+0+10 (secure 0 0 0) 0x7f51780ab5c0 con 0x7f517c070ba0 2023-11-26T07:18:06.243 INFO:teuthology.orchestra.run.smithi143.stdout:pg_num: 1 2023-11-26T07:18:06.245 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:06.242+0000 7f5183216640 1 -- 172.21.15.143:0/640996201 >> [v2:172.21.15.143:6832/2150557544,v1:172.21.15.143:6833/2150557544] conn(0x7f5168066470 msgr2=0x7f5168068930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:18:06.245 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:06.242+0000 7f5183216640 1 --2- 172.21.15.143:0/640996201 >> [v2:172.21.15.143:6832/2150557544,v1:172.21.15.143:6833/2150557544] conn(0x7f5168066470 0x7f5168068930 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7f517c0cf130 tx=0x7f516c008040 comp rx=0 tx=0).stop 2023-11-26T07:18:06.245 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:06.242+0000 7f5183216640 1 -- 172.21.15.143:0/640996201 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f517c070ba0 msgr2=0x7f517c155c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:18:06.246 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:06.242+0000 7f5183216640 1 --2- 172.21.15.143:0/640996201 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f517c070ba0 0x7f517c155c40 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7f5178019040 tx=0x7f517801ac70 comp rx=0 tx=0).stop 2023-11-26T07:18:06.246 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:06.242+0000 7f5183216640 1 -- 172.21.15.143:0/640996201 shutdown_connections 2023-11-26T07:18:06.246 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:06.242+0000 7f5183216640 1 --2- 172.21.15.143:0/640996201 >> [v2:172.21.15.143:6832/2150557544,v1:172.21.15.143:6833/2150557544] conn(0x7f5168066470 0x7f5168068930 unknown :-1 s=CLOSED pgs=110 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:18:06.246 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:06.242+0000 7f5183216640 1 --2- 172.21.15.143:0/640996201 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f517c1614b0 0x7f517c1566c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:18:06.246 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:06.242+0000 7f5183216640 1 --2- 172.21.15.143:0/640996201 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f517c071970 0x7f517c156180 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:18:06.246 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:06.242+0000 7f5183216640 1 --2- 172.21.15.143:0/640996201 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f517c070ba0 0x7f517c155c40 unknown :-1 s=CLOSED pgs=139 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:18:06.246 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:06.242+0000 7f5183216640 1 -- 172.21.15.143:0/640996201 >> 172.21.15.143:0/640996201 conn(0x7f517c0cf710 msgr2=0x7f517c1511d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:18:06.246 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:06.242+0000 7f5183216640 1 -- 172.21.15.143:0/640996201 shutdown_connections 2023-11-26T07:18:06.246 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:06.242+0000 7f5183216640 1 -- 172.21.15.143:0/640996201 wait complete. 2023-11-26T07:18:06.264 DEBUG:teuthology.orchestra.run.smithi143:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-11-26T07:18:06.405 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:06.402+0000 7fc861bdf640 1 Processor -- start 2023-11-26T07:18:06.405 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:06.402+0000 7fc861bdf640 1 -- start start 2023-11-26T07:18:06.406 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:06.402+0000 7fc861bdf640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fc85c070ba0 0x7fc85c06bae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:18:06.406 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:06.402+0000 7fc861bdf640 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7fc85c161cf0 0x7fc85c070f80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:18:06.406 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:06.402+0000 7fc861bdf640 1 -- --> v1:172.21.15.143:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc85c0714c0 con 0x7fc85c0cdd10 2023-11-26T07:18:06.407 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:06.402+0000 7fc861bdf640 1 -- --> v2:172.21.15.143:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc85c0cd610 con 0x7fc85c161cf0 2023-11-26T07:18:06.407 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:06.402+0000 7fc861bdf640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc85c0cd8f0 con 0x7fc85c070ba0 2023-11-26T07:18:06.407 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:06.402+0000 7fc85affd640 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7fc85c161cf0 0x7fc85c070f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:18:06.408 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:06.402+0000 7fc85affd640 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7fc85c161cf0 0x7fc85c070f80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:48228/0 (socket says 172.21.15.143:48228) 2023-11-26T07:18:06.408 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:06.402+0000 7fc85affd640 1 -- 172.21.15.143:0/4281505177 learned_addr learned my addr 172.21.15.143:0/4281505177 (peer_addr_for_me v2:172.21.15.143:0/0) 2023-11-26T07:18:06.408 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:06.402+0000 7fc85bfff640 1 --1- 172.21.15.143:0/4281505177 >> v1:172.21.15.143:6789/0 conn(0x7fc85c0cdd10 0x7fc85c06a490 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.143:6789/0 says I am v1:172.21.15.143:52220/0 (socket says 172.21.15.143:52220) 2023-11-26T07:18:06.408 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:06.402+0000 7fc85b7fe640 1 --2- 172.21.15.143:0/4281505177 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fc85c070ba0 0x7fc85c06bae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:18:06.408 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:06.402+0000 7fc85affd640 1 -- 172.21.15.143:0/4281505177 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fc85c070ba0 msgr2=0x7fc85c06bae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-26T07:18:06.408 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:06.402+0000 7fc85affd640 1 --2- 172.21.15.143:0/4281505177 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fc85c070ba0 0x7fc85c06bae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:18:06.408 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:06.402+0000 7fc85affd640 1 -- 172.21.15.143:0/4281505177 >> v1:172.21.15.143:6789/0 conn(0x7fc85c0cdd10 legacy=0x7fc85c06a490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:18:06.408 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:06.402+0000 7fc85affd640 1 -- 172.21.15.143:0/4281505177 --> v2:172.21.15.143:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc85c161210 con 0x7fc85c161cf0 2023-11-26T07:18:06.408 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:06.402+0000 7fc85a7fc640 1 -- 172.21.15.143:0/4281505177 <== mon.0 v1:172.21.15.143:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 484110918 0 0) 0x7fc85c0714c0 con 0x7fc85c0cdd10 2023-11-26T07:18:06.410 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:06.406+0000 7fc85affd640 1 --2- 172.21.15.143:0/4281505177 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fc85c161cf0 0x7fc85c070f80 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7fc850009870 tx=0x7fc85001c540 comp rx=0 tx=0).ready entity=mon.0 client_cookie=34f0e14510e8268c server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:18:06.410 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:06.406+0000 7fc85a7fc640 1 -- 172.21.15.143:0/4281505177 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc8500ab600 con 0x7fc85c161cf0 2023-11-26T07:18:06.410 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:06.406+0000 7fc85a7fc640 1 -- 172.21.15.143:0/4281505177 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc8500a7050 con 0x7fc85c161cf0 2023-11-26T07:18:06.410 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:06.406+0000 7fc85a7fc640 1 -- 172.21.15.143:0/4281505177 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc8500ab980 con 0x7fc85c161cf0 2023-11-26T07:18:06.411 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:06.406+0000 7fc861bdf640 1 -- 172.21.15.143:0/4281505177 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fc85c161cf0 msgr2=0x7fc85c070f80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:18:06.411 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:06.406+0000 7fc861bdf640 1 --2- 172.21.15.143:0/4281505177 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fc85c161cf0 0x7fc85c070f80 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7fc850009870 tx=0x7fc85001c540 comp rx=0 tx=0).stop 2023-11-26T07:18:06.411 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:06.406+0000 7fc861bdf640 1 -- 172.21.15.143:0/4281505177 shutdown_connections 2023-11-26T07:18:06.411 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:06.406+0000 7fc861bdf640 1 --2- 172.21.15.143:0/4281505177 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fc85c161cf0 0x7fc85c070f80 unknown :-1 s=CLOSED pgs=141 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:18:06.411 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:06.406+0000 7fc861bdf640 1 --2- 172.21.15.143:0/4281505177 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fc85c070ba0 0x7fc85c06bae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:18:06.411 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:06.406+0000 7fc861bdf640 1 -- 172.21.15.143:0/4281505177 >> 172.21.15.143:0/4281505177 conn(0x7fc85c0cf710 msgr2=0x7fc85c0cfb20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:18:06.412 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:06.406+0000 7fc861bdf640 1 -- 172.21.15.143:0/4281505177 shutdown_connections 2023-11-26T07:18:06.412 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:06.406+0000 7fc861bdf640 1 -- 172.21.15.143:0/4281505177 wait complete. 2023-11-26T07:18:06.412 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:06.410+0000 7fc861bdf640 1 Processor -- start 2023-11-26T07:18:06.413 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:06.410+0000 7fc861bdf640 1 -- start start 2023-11-26T07:18:06.414 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:06.410+0000 7fc861bdf640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fc85c070ba0 0x7fc85c15f490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:18:06.415 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:06.410+0000 7fc861bdf640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fc85c0cdd10 0x7fc85c0b3960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:18:06.415 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:06.410+0000 7fc861bdf640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fc85c161cf0 0x7fc85c151fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:18:06.415 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:06.410+0000 7fc861bdf640 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc85c1575d0 con 0x7fc85c0cdd10 2023-11-26T07:18:06.415 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:06.410+0000 7fc861bdf640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc85c155540 con 0x7fc85c161cf0 2023-11-26T07:18:06.415 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:06.410+0000 7fc861bdf640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc85c154c10 con 0x7fc85c070ba0 2023-11-26T07:18:06.415 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:06.410+0000 7fc85affd640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fc85c0cdd10 0x7fc85c0b3960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:18:06.415 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:06.410+0000 7fc85affd640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fc85c0cdd10 0x7fc85c0b3960 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:48242/0 (socket says 172.21.15.143:48242) 2023-11-26T07:18:06.415 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:06.410+0000 7fc85affd640 1 -- 172.21.15.143:0/3259641573 learned_addr learned my addr 172.21.15.143:0/3259641573 (peer_addr_for_me v2:172.21.15.143:0/0) 2023-11-26T07:18:06.415 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:06.410+0000 7fc85b7fe640 1 --2- 172.21.15.143:0/3259641573 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fc85c070ba0 0x7fc85c15f490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:18:06.415 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:06.410+0000 7fc85bfff640 1 --2- 172.21.15.143:0/3259641573 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fc85c161cf0 0x7fc85c151fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:18:06.415 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:06.410+0000 7fc85affd640 1 -- 172.21.15.143:0/3259641573 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fc85c070ba0 msgr2=0x7fc85c15f490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:18:06.416 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:06.410+0000 7fc85affd640 1 --2- 172.21.15.143:0/3259641573 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fc85c070ba0 0x7fc85c15f490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:18:06.416 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:06.410+0000 7fc85affd640 1 -- 172.21.15.143:0/3259641573 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fc85c161cf0 msgr2=0x7fc85c151fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:18:06.416 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:06.410+0000 7fc85affd640 1 --2- 172.21.15.143:0/3259641573 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fc85c161cf0 0x7fc85c151fe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:18:06.416 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:06.410+0000 7fc85affd640 1 -- 172.21.15.143:0/3259641573 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc8500094d0 con 0x7fc85c0cdd10 2023-11-26T07:18:06.416 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:06.410+0000 7fc85affd640 1 --2- 172.21.15.143:0/3259641573 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fc85c0cdd10 0x7fc85c0b3960 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7fc85001abf0 tx=0x7fc8500192e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:18:06.417 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:06.410+0000 7fc843fff640 1 -- 172.21.15.143:0/3259641573 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc8500bc070 con 0x7fc85c0cdd10 2023-11-26T07:18:06.417 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:06.410+0000 7fc843fff640 1 -- 172.21.15.143:0/3259641573 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc8500ace80 con 0x7fc85c0cdd10 2023-11-26T07:18:06.417 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:06.410+0000 7fc843fff640 1 -- 172.21.15.143:0/3259641573 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc8500b6640 con 0x7fc85c0cdd10 2023-11-26T07:18:06.417 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:06.410+0000 7fc861bdf640 1 -- 172.21.15.143:0/3259641573 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc85c154ef0 con 0x7fc85c0cdd10 2023-11-26T07:18:06.417 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:06.410+0000 7fc861bdf640 1 -- 172.21.15.143:0/3259641573 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc85c15f9d0 con 0x7fc85c0cdd10 2023-11-26T07:18:06.417 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:06.410+0000 7fc861bdf640 1 -- 172.21.15.143:0/3259641573 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc85c15fbc0 con 0x7fc85c0cdd10 2023-11-26T07:18:06.418 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:06.414+0000 7fc843fff640 1 -- 172.21.15.143:0/3259641573 <== mon.0 v2:172.21.15.143:3300/0 4 ==== mgrmap(e 5) v1 ==== 85852+0+0 (secure 0 0 0) 0x7fc8500b2040 con 0x7fc85c0cdd10 2023-11-26T07:18:06.418 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:06.414+0000 7fc843fff640 1 --2- 172.21.15.143:0/3259641573 >> [v2:172.21.15.143:6832/2150557544,v1:172.21.15.143:6833/2150557544] conn(0x7fc848066470 0x7fc848068930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:18:06.418 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:06.414+0000 7fc843fff640 1 -- 172.21.15.143:0/3259641573 <== mon.0 v2:172.21.15.143:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fc85011d840 con 0x7fc85c0cdd10 2023-11-26T07:18:06.418 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:06.414+0000 7fc85b7fe640 1 --2- 172.21.15.143:0/3259641573 >> [v2:172.21.15.143:6832/2150557544,v1:172.21.15.143:6833/2150557544] conn(0x7fc848066470 0x7fc848068930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:18:06.419 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:06.414+0000 7fc85b7fe640 1 --2- 172.21.15.143:0/3259641573 >> [v2:172.21.15.143:6832/2150557544,v1:172.21.15.143:6833/2150557544] conn(0x7fc848066470 0x7fc848068930 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7fc84c007a60 tx=0x7fc84c008040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:18:06.422 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:06.418+0000 7fc843fff640 1 -- 172.21.15.143:0/3259641573 <== mon.0 v2:172.21.15.143:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172649 (secure 0 0 0) 0x7fc85c15fbc0 con 0x7fc85c0cdd10 2023-11-26T07:18:06.571 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:06.566+0000 7fc861bdf640 1 -- 172.21.15.143:0/3259641573 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7fc85c15fbc0 con 0x7fc85c0cdd10 2023-11-26T07:18:06.571 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:06.566+0000 7fc843fff640 1 -- 172.21.15.143:0/3259641573 <== mon.0 v2:172.21.15.143:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v20) v1 ==== 104+0+11 (secure 0 0 0) 0x7fc85c15fbc0 con 0x7fc85c0cdd10 2023-11-26T07:18:06.571 INFO:teuthology.orchestra.run.smithi143.stdout:pg_num: 32 2023-11-26T07:18:06.573 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:06.570+0000 7fc861bdf640 1 -- 172.21.15.143:0/3259641573 >> [v2:172.21.15.143:6832/2150557544,v1:172.21.15.143:6833/2150557544] conn(0x7fc848066470 msgr2=0x7fc848068930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:18:06.573 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:06.570+0000 7fc861bdf640 1 --2- 172.21.15.143:0/3259641573 >> [v2:172.21.15.143:6832/2150557544,v1:172.21.15.143:6833/2150557544] conn(0x7fc848066470 0x7fc848068930 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7fc84c007a60 tx=0x7fc84c008040 comp rx=0 tx=0).stop 2023-11-26T07:18:06.573 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:06.570+0000 7fc861bdf640 1 -- 172.21.15.143:0/3259641573 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fc85c0cdd10 msgr2=0x7fc85c0b3960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:18:06.574 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:06.570+0000 7fc861bdf640 1 --2- 172.21.15.143:0/3259641573 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fc85c0cdd10 0x7fc85c0b3960 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7fc85001abf0 tx=0x7fc8500192e0 comp rx=0 tx=0).stop 2023-11-26T07:18:06.574 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:06.570+0000 7fc861bdf640 1 -- 172.21.15.143:0/3259641573 shutdown_connections 2023-11-26T07:18:06.574 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:06.570+0000 7fc861bdf640 1 --2- 172.21.15.143:0/3259641573 >> [v2:172.21.15.143:6832/2150557544,v1:172.21.15.143:6833/2150557544] conn(0x7fc848066470 0x7fc848068930 unknown :-1 s=CLOSED pgs=111 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:18:06.574 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:06.570+0000 7fc861bdf640 1 --2- 172.21.15.143:0/3259641573 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fc85c161cf0 0x7fc85c151fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:18:06.574 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:06.570+0000 7fc861bdf640 1 --2- 172.21.15.143:0/3259641573 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fc85c0cdd10 0x7fc85c0b3960 unknown :-1 s=CLOSED pgs=142 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:18:06.574 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:06.570+0000 7fc861bdf640 1 --2- 172.21.15.143:0/3259641573 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fc85c070ba0 0x7fc85c15f490 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:18:06.574 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:06.570+0000 7fc861bdf640 1 -- 172.21.15.143:0/3259641573 >> 172.21.15.143:0/3259641573 conn(0x7fc85c0cf710 msgr2=0x7fc85c0ae4e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:18:06.574 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:06.570+0000 7fc861bdf640 1 -- 172.21.15.143:0/3259641573 shutdown_connections 2023-11-26T07:18:06.574 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:06.570+0000 7fc861bdf640 1 -- 172.21.15.143:0/3259641573 wait complete. 2023-11-26T07:18:06.593 DEBUG:teuthology.orchestra.run.smithi143:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-11-26T07:18:06.734 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:06.730+0000 7f24498c9640 1 Processor -- start 2023-11-26T07:18:06.734 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:06.730+0000 7f24498c9640 1 -- start start 2023-11-26T07:18:06.736 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:06.730+0000 7f24498c9640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f244415d3f0 0x7f24440cfb10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:18:06.736 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:06.730+0000 7f24498c9640 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7f24440d0050 0x7f244406bf20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:18:06.736 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:06.730+0000 7f24498c9640 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7f24440b0490 0x7f244406c460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:18:06.736 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:06.730+0000 7f24498c9640 1 -- --> v2:172.21.15.143:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f24440cb480 con 0x7f24440d0050 2023-11-26T07:18:06.736 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:06.730+0000 7f24498c9640 1 -- --> v2:172.21.15.188:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f24440cb6d0 con 0x7f24440b0490 2023-11-26T07:18:06.736 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:06.730+0000 7f24498c9640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f24440cb9b0 con 0x7f244415d3f0 2023-11-26T07:18:06.736 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:06.730+0000 7f243bfff640 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7f24440d0050 0x7f244406bf20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:18:06.737 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:06.730+0000 7f243bfff640 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7f24440d0050 0x7f244406bf20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:48252/0 (socket says 172.21.15.143:48252) 2023-11-26T07:18:06.737 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:06.730+0000 7f243bfff640 1 -- 172.21.15.143:0/3845939601 learned_addr learned my addr 172.21.15.143:0/3845939601 (peer_addr_for_me v2:172.21.15.143:0/0) 2023-11-26T07:18:06.737 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:06.730+0000 7f2442ffd640 1 --2- 172.21.15.143:0/3845939601 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f244415d3f0 0x7f24440cfb10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:18:06.737 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:06.730+0000 7f24437fe640 1 --2- 172.21.15.143:0/3845939601 >> v2:172.21.15.188:3300/0 conn(0x7f24440b0490 0x7f244406c460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:18:06.737 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:06.730+0000 7f243bfff640 1 -- 172.21.15.143:0/3845939601 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f244415d3f0 msgr2=0x7f24440cfb10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-26T07:18:06.737 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:06.730+0000 7f243bfff640 1 --2- 172.21.15.143:0/3845939601 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f244415d3f0 0x7f24440cfb10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:18:06.737 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:06.730+0000 7f243bfff640 1 -- 172.21.15.143:0/3845939601 >> v2:172.21.15.188:3300/0 conn(0x7f24440b0490 msgr2=0x7f244406c460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-26T07:18:06.737 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:06.730+0000 7f243bfff640 1 --2- 172.21.15.143:0/3845939601 >> v2:172.21.15.188:3300/0 conn(0x7f24440b0490 0x7f244406c460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:18:06.737 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:06.730+0000 7f243bfff640 1 -- 172.21.15.143:0/3845939601 --> v2:172.21.15.143:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f24440cbc90 con 0x7f24440d0050 2023-11-26T07:18:06.739 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:06.734+0000 7f243bfff640 1 --2- 172.21.15.143:0/3845939601 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f24440d0050 0x7f244406bf20 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7f24440716e0 tx=0x7f242c01c4d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a6bdf34ee66c7a10 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:18:06.739 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:06.734+0000 7f24427fc640 1 -- 172.21.15.143:0/3845939601 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f242c0aa610 con 0x7f24440d0050 2023-11-26T07:18:06.739 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:06.734+0000 7f24427fc640 1 -- 172.21.15.143:0/3845939601 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f242c0a6040 con 0x7f24440d0050 2023-11-26T07:18:06.739 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:06.734+0000 7f24427fc640 1 -- 172.21.15.143:0/3845939601 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f242c0aa990 con 0x7f24440d0050 2023-11-26T07:18:06.740 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:06.734+0000 7f24498c9640 1 -- 172.21.15.143:0/3845939601 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f24440d0050 msgr2=0x7f244406bf20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:18:06.740 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:06.734+0000 7f24498c9640 1 --2- 172.21.15.143:0/3845939601 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f24440d0050 0x7f244406bf20 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7f24440716e0 tx=0x7f242c01c4d0 comp rx=0 tx=0).stop 2023-11-26T07:18:06.740 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:06.734+0000 7f24498c9640 1 -- 172.21.15.143:0/3845939601 shutdown_connections 2023-11-26T07:18:06.740 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:06.734+0000 7f24498c9640 1 --2- 172.21.15.143:0/3845939601 >> v2:172.21.15.188:3300/0 conn(0x7f24440b0490 0x7f244406c460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:18:06.740 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:06.734+0000 7f24498c9640 1 --2- 172.21.15.143:0/3845939601 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f24440d0050 0x7f244406bf20 unknown :-1 s=CLOSED pgs=143 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:18:06.740 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:06.734+0000 7f24498c9640 1 --2- 172.21.15.143:0/3845939601 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f244415d3f0 0x7f24440cfb10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:18:06.740 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:06.734+0000 7f24498c9640 1 -- 172.21.15.143:0/3845939601 >> 172.21.15.143:0/3845939601 conn(0x7f24440d0e30 msgr2=0x7f24440d1240 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:18:06.741 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:06.734+0000 7f24498c9640 1 -- 172.21.15.143:0/3845939601 shutdown_connections 2023-11-26T07:18:06.741 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:06.734+0000 7f24498c9640 1 -- 172.21.15.143:0/3845939601 wait complete. 2023-11-26T07:18:06.741 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:06.738+0000 7f24498c9640 1 Processor -- start 2023-11-26T07:18:06.741 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:06.738+0000 7f24498c9640 1 -- start start 2023-11-26T07:18:06.743 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:06.738+0000 7f24498c9640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f24440b0490 0x7f24440b4560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:18:06.743 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:06.738+0000 7f24498c9640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f24440d0050 0x7f24441f8ef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:18:06.743 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:06.738+0000 7f24498c9640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f244415d3f0 0x7f24441f9430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:18:06.744 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:06.738+0000 7f24498c9640 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f24440caf10 con 0x7f24440d0050 2023-11-26T07:18:06.744 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:06.738+0000 7f24498c9640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f24440c9030 con 0x7f244415d3f0 2023-11-26T07:18:06.744 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:06.738+0000 7f24498c9640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f244415be10 con 0x7f24440b0490 2023-11-26T07:18:06.744 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:06.738+0000 7f243bfff640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f24440d0050 0x7f24441f8ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:18:06.744 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:06.738+0000 7f243bfff640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f24440d0050 0x7f24441f8ef0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:48254/0 (socket says 172.21.15.143:48254) 2023-11-26T07:18:06.744 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:06.738+0000 7f243bfff640 1 -- 172.21.15.143:0/2719911269 learned_addr learned my addr 172.21.15.143:0/2719911269 (peer_addr_for_me v2:172.21.15.143:0/0) 2023-11-26T07:18:06.744 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:06.738+0000 7f2442ffd640 1 --2- 172.21.15.143:0/2719911269 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f24440b0490 0x7f24440b4560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:18:06.744 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:06.738+0000 7f24437fe640 1 --2- 172.21.15.143:0/2719911269 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f244415d3f0 0x7f24441f9430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:18:06.744 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:06.738+0000 7f243bfff640 1 -- 172.21.15.143:0/2719911269 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f24440b0490 msgr2=0x7f24440b4560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:18:06.744 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:06.738+0000 7f243bfff640 1 --2- 172.21.15.143:0/2719911269 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f24440b0490 0x7f24440b4560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:18:06.744 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:06.738+0000 7f243bfff640 1 -- 172.21.15.143:0/2719911269 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f244415d3f0 msgr2=0x7f24441f9430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:18:06.745 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:06.738+0000 7f243bfff640 1 --2- 172.21.15.143:0/2719911269 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f244415d3f0 0x7f24441f9430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:18:06.745 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:06.738+0000 7f243bfff640 1 -- 172.21.15.143:0/2719911269 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f242c0094d0 con 0x7f24440d0050 2023-11-26T07:18:06.745 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:06.738+0000 7f243bfff640 1 --2- 172.21.15.143:0/2719911269 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f24440d0050 0x7f24441f8ef0 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7f242c0ac040 tx=0x7f242c01ad10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:18:06.745 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:06.738+0000 7f2440ff9640 1 -- 172.21.15.143:0/2719911269 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f242c0bd3f0 con 0x7f24440d0050 2023-11-26T07:18:06.746 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:06.738+0000 7f2440ff9640 1 -- 172.21.15.143:0/2719911269 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f242c0bd550 con 0x7f24440d0050 2023-11-26T07:18:06.746 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:06.738+0000 7f2440ff9640 1 -- 172.21.15.143:0/2719911269 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f242c0bd900 con 0x7f24440d0050 2023-11-26T07:18:06.746 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:06.738+0000 7f24498c9640 1 -- 172.21.15.143:0/2719911269 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f244415c0f0 con 0x7f24440d0050 2023-11-26T07:18:06.746 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:06.738+0000 7f24498c9640 1 -- 172.21.15.143:0/2719911269 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f24440b4aa0 con 0x7f24440d0050 2023-11-26T07:18:06.746 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:06.742+0000 7f24498c9640 1 -- 172.21.15.143:0/2719911269 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f24440b4c90 con 0x7f24440d0050 2023-11-26T07:18:06.746 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:06.742+0000 7f2440ff9640 1 -- 172.21.15.143:0/2719911269 <== mon.0 v2:172.21.15.143:3300/0 4 ==== mgrmap(e 5) v1 ==== 85852+0+0 (secure 0 0 0) 0x7f242c0b33f0 con 0x7f24440d0050 2023-11-26T07:18:06.747 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:06.742+0000 7f2440ff9640 1 --2- 172.21.15.143:0/2719911269 >> [v2:172.21.15.143:6832/2150557544,v1:172.21.15.143:6833/2150557544] conn(0x7f24240664a0 0x7f2424068960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:18:06.747 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:06.742+0000 7f2440ff9640 1 -- 172.21.15.143:0/2719911269 <== mon.0 v2:172.21.15.143:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f242c11e280 con 0x7f24440d0050 2023-11-26T07:18:06.747 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:06.742+0000 7f2442ffd640 1 --2- 172.21.15.143:0/2719911269 >> [v2:172.21.15.143:6832/2150557544,v1:172.21.15.143:6833/2150557544] conn(0x7f24240664a0 0x7f2424068960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:18:06.747 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:06.742+0000 7f2442ffd640 1 --2- 172.21.15.143:0/2719911269 >> [v2:172.21.15.143:6832/2150557544,v1:172.21.15.143:6833/2150557544] conn(0x7f24240664a0 0x7f2424068960 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7f2434009a40 tx=0x7f2434009190 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:18:06.750 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:06.746+0000 7f2440ff9640 1 -- 172.21.15.143:0/2719911269 <== mon.0 v2:172.21.15.143:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172649 (secure 0 0 0) 0x7f24440b4c90 con 0x7f24440d0050 2023-11-26T07:18:06.900 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:06.894+0000 7f24498c9640 1 -- 172.21.15.143:0/2719911269 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f24440b4c90 con 0x7f24440d0050 2023-11-26T07:18:06.901 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:06.898+0000 7f2440ff9640 1 -- 172.21.15.143:0/2719911269 <== mon.0 v2:172.21.15.143:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v20) v1 ==== 100+0+11 (secure 0 0 0) 0x7f24440b4c90 con 0x7f24440d0050 2023-11-26T07:18:06.901 INFO:teuthology.orchestra.run.smithi143.stdout:pg_num: 64 2023-11-26T07:18:06.903 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:06.898+0000 7f24498c9640 1 -- 172.21.15.143:0/2719911269 >> [v2:172.21.15.143:6832/2150557544,v1:172.21.15.143:6833/2150557544] conn(0x7f24240664a0 msgr2=0x7f2424068960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:18:06.903 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:06.898+0000 7f24498c9640 1 --2- 172.21.15.143:0/2719911269 >> [v2:172.21.15.143:6832/2150557544,v1:172.21.15.143:6833/2150557544] conn(0x7f24240664a0 0x7f2424068960 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7f2434009a40 tx=0x7f2434009190 comp rx=0 tx=0).stop 2023-11-26T07:18:06.903 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:06.898+0000 7f24498c9640 1 -- 172.21.15.143:0/2719911269 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f24440d0050 msgr2=0x7f24441f8ef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:18:06.903 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:06.898+0000 7f24498c9640 1 --2- 172.21.15.143:0/2719911269 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f24440d0050 0x7f24441f8ef0 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7f242c0ac040 tx=0x7f242c01ad10 comp rx=0 tx=0).stop 2023-11-26T07:18:06.903 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:06.898+0000 7f24498c9640 1 -- 172.21.15.143:0/2719911269 shutdown_connections 2023-11-26T07:18:06.903 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:06.898+0000 7f24498c9640 1 --2- 172.21.15.143:0/2719911269 >> [v2:172.21.15.143:6832/2150557544,v1:172.21.15.143:6833/2150557544] conn(0x7f24240664a0 0x7f2424068960 unknown :-1 s=CLOSED pgs=112 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:18:06.903 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:06.898+0000 7f24498c9640 1 --2- 172.21.15.143:0/2719911269 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f244415d3f0 0x7f24441f9430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:18:06.903 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:06.898+0000 7f24498c9640 1 --2- 172.21.15.143:0/2719911269 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f24440d0050 0x7f24441f8ef0 unknown :-1 s=CLOSED pgs=144 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:18:06.904 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:06.898+0000 7f24498c9640 1 --2- 172.21.15.143:0/2719911269 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f24440b0490 0x7f24440b4560 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:18:06.904 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:06.898+0000 7f24498c9640 1 -- 172.21.15.143:0/2719911269 >> 172.21.15.143:0/2719911269 conn(0x7f24440d0e30 msgr2=0x7f24440ccd10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:18:06.904 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:06.898+0000 7f24498c9640 1 -- 172.21.15.143:0/2719911269 shutdown_connections 2023-11-26T07:18:06.904 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:06.898+0000 7f24498c9640 1 -- 172.21.15.143:0/2719911269 wait complete. 2023-11-26T07:18:06.922 DEBUG:teuthology.orchestra.run.smithi143:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-11-26T07:18:07.058 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:07.054+0000 7fdb0b5b8640 1 Processor -- start 2023-11-26T07:18:07.059 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:07.054+0000 7fdb0b5b8640 1 -- start start 2023-11-26T07:18:07.060 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:07.054+0000 7fdb0b5b8640 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7fdb04143ac0 0x7fdb04070370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:18:07.060 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:07.054+0000 7fdb0b5b8640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fdb040cf990 0x7fdb040cd1b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:18:07.060 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:07.054+0000 7fdb0b5b8640 1 -- --> v1:172.21.15.143:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fdb040cc730 con 0x7fdb040708b0 2023-11-26T07:18:07.061 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:07.054+0000 7fdb0b5b8640 1 -- --> v2:172.21.15.143:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fdb04143ea0 con 0x7fdb04143ac0 2023-11-26T07:18:07.061 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:07.054+0000 7fdb0b5b8640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdb04070ee0 con 0x7fdb040cf990 2023-11-26T07:18:07.061 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:07.054+0000 7fdb09354640 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7fdb04143ac0 0x7fdb04070370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:18:07.061 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:07.054+0000 7fdb08b53640 1 --1- >> v1:172.21.15.143:6789/0 conn(0x7fdb040708b0 0x7fdb040cf280 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.143:6789/0 says I am v1:172.21.15.143:52226/0 (socket says 172.21.15.143:52226) 2023-11-26T07:18:07.061 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:07.054+0000 7fdb08b53640 1 -- 172.21.15.143:0/2167157069 learned_addr learned my addr 172.21.15.143:0/2167157069 (peer_addr_for_me v1:172.21.15.143:0/0) 2023-11-26T07:18:07.061 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:07.054+0000 7fdb09b55640 1 --2- 172.21.15.143:0/2167157069 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fdb040cf990 0x7fdb040cd1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:18:07.062 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:07.058+0000 7fdb09354640 1 -- 172.21.15.143:0/2167157069 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fdb040cf990 msgr2=0x7fdb040cd1b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-26T07:18:07.062 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:07.058+0000 7fdb09354640 1 --2- 172.21.15.143:0/2167157069 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fdb040cf990 0x7fdb040cd1b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:18:07.062 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:07.058+0000 7fdb09354640 1 -- 172.21.15.143:0/2167157069 >> v1:172.21.15.143:6789/0 conn(0x7fdb040708b0 legacy=0x7fdb040cf280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:18:07.062 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:07.058+0000 7fdb09354640 1 -- 172.21.15.143:0/2167157069 --> v2:172.21.15.143:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdb040711c0 con 0x7fdb04143ac0 2023-11-26T07:18:07.062 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:07.058+0000 7fdafbfff640 1 -- 172.21.15.143:0/2167157069 <== mon.0 v1:172.21.15.143:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 835005704 0 0) 0x7fdb040cc730 con 0x7fdb040708b0 2023-11-26T07:18:07.064 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:07.058+0000 7fdb09354640 1 --2- 172.21.15.143:0/2167157069 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fdb04143ac0 0x7fdb04070370 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7fdb000097c0 tx=0x7fdb0001c480 comp rx=0 tx=0).ready entity=mon.0 client_cookie=48ffe1f70a9a07e0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:18:07.064 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:07.058+0000 7fdafbfff640 1 -- 172.21.15.143:0/2167157069 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdb000ad640 con 0x7fdb04143ac0 2023-11-26T07:18:07.064 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:07.058+0000 7fdafbfff640 1 -- 172.21.15.143:0/2167157069 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdb000ad7a0 con 0x7fdb04143ac0 2023-11-26T07:18:07.064 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:07.058+0000 7fdafbfff640 1 -- 172.21.15.143:0/2167157069 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdb000adb30 con 0x7fdb04143ac0 2023-11-26T07:18:07.065 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:07.058+0000 7fdb0b5b8640 1 -- 172.21.15.143:0/2167157069 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fdb04143ac0 msgr2=0x7fdb04070370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:18:07.065 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:07.058+0000 7fdb0b5b8640 1 --2- 172.21.15.143:0/2167157069 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fdb04143ac0 0x7fdb04070370 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7fdb000097c0 tx=0x7fdb0001c480 comp rx=0 tx=0).stop 2023-11-26T07:18:07.065 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:07.058+0000 7fdb0b5b8640 1 -- 172.21.15.143:0/2167157069 shutdown_connections 2023-11-26T07:18:07.065 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:07.058+0000 7fdb0b5b8640 1 --2- 172.21.15.143:0/2167157069 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fdb040cf990 0x7fdb040cd1b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:18:07.065 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:07.058+0000 7fdb0b5b8640 1 --2- 172.21.15.143:0/2167157069 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fdb04143ac0 0x7fdb04070370 unknown :-1 s=CLOSED pgs=146 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:18:07.065 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:07.058+0000 7fdb0b5b8640 1 -- 172.21.15.143:0/2167157069 >> 172.21.15.143:0/2167157069 conn(0x7fdb0406a890 msgr2=0x7fdb0406aca0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:18:07.065 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:07.062+0000 7fdb0b5b8640 1 -- 172.21.15.143:0/2167157069 shutdown_connections 2023-11-26T07:18:07.065 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:07.062+0000 7fdb0b5b8640 1 -- 172.21.15.143:0/2167157069 wait complete. 2023-11-26T07:18:07.066 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:07.062+0000 7fdb0b5b8640 1 Processor -- start 2023-11-26T07:18:07.066 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:07.062+0000 7fdb0b5b8640 1 -- start start 2023-11-26T07:18:07.069 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:07.062+0000 7fdb0b5b8640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fdb040708b0 0x7fdb0415bb30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:18:07.069 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:07.062+0000 7fdb0b5b8640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fdb040cf990 0x7fdb04159e60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:18:07.069 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:07.062+0000 7fdb0b5b8640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fdb04143ac0 0x7fdb0413bc60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:18:07.069 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:07.062+0000 7fdb0b5b8640 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdb0415b6a0 con 0x7fdb040cf990 2023-11-26T07:18:07.069 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:07.062+0000 7fdb0b5b8640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdb0415aa90 con 0x7fdb040708b0 2023-11-26T07:18:07.069 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:07.062+0000 7fdb0b5b8640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdb0415ad70 con 0x7fdb04143ac0 2023-11-26T07:18:07.069 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:07.062+0000 7fdb08b53640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fdb040cf990 0x7fdb04159e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:18:07.069 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:07.062+0000 7fdb08b53640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fdb040cf990 0x7fdb04159e60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:48280/0 (socket says 172.21.15.143:48280) 2023-11-26T07:18:07.069 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:07.062+0000 7fdb08b53640 1 -- 172.21.15.143:0/171265258 learned_addr learned my addr 172.21.15.143:0/171265258 (peer_addr_for_me v2:172.21.15.143:0/0) 2023-11-26T07:18:07.070 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:07.062+0000 7fdb09b55640 1 --2- 172.21.15.143:0/171265258 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fdb04143ac0 0x7fdb0413bc60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:18:07.070 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:07.062+0000 7fdb09354640 1 --2- 172.21.15.143:0/171265258 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fdb040708b0 0x7fdb0415bb30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:18:07.070 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:07.062+0000 7fdb08b53640 1 -- 172.21.15.143:0/171265258 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fdb04143ac0 msgr2=0x7fdb0413bc60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:18:07.070 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:07.062+0000 7fdb08b53640 1 --2- 172.21.15.143:0/171265258 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fdb04143ac0 0x7fdb0413bc60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:18:07.070 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:07.062+0000 7fdb08b53640 1 -- 172.21.15.143:0/171265258 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fdb040708b0 msgr2=0x7fdb0415bb30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:18:07.070 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:07.062+0000 7fdb08b53640 1 --2- 172.21.15.143:0/171265258 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fdb040708b0 0x7fdb0415bb30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:18:07.070 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:07.062+0000 7fdb08b53640 1 -- 172.21.15.143:0/171265258 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdb00009420 con 0x7fdb040cf990 2023-11-26T07:18:07.070 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:07.062+0000 7fdb09b55640 1 --2- 172.21.15.143:0/171265258 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fdb04143ac0 0x7fdb0413bc60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-26T07:18:07.070 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:07.062+0000 7fdb09354640 1 --2- 172.21.15.143:0/171265258 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fdb040708b0 0x7fdb0415bb30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-26T07:18:07.070 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:07.062+0000 7fdb08b53640 1 --2- 172.21.15.143:0/171265258 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fdb040cf990 0x7fdb04159e60 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7fdaf400c550 tx=0x7fdaf400ca10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:18:07.070 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:07.062+0000 7fdaf9ffb640 1 -- 172.21.15.143:0/171265258 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdaf4005cf0 con 0x7fdb040cf990 2023-11-26T07:18:07.070 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:07.062+0000 7fdaf9ffb640 1 -- 172.21.15.143:0/171265258 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdaf4005e70 con 0x7fdb040cf990 2023-11-26T07:18:07.071 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:07.062+0000 7fdaf9ffb640 1 -- 172.21.15.143:0/171265258 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdaf4007310 con 0x7fdb040cf990 2023-11-26T07:18:07.071 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:07.066+0000 7fdb0b5b8640 1 -- 172.21.15.143:0/171265258 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdb0415b050 con 0x7fdb040cf990 2023-11-26T07:18:07.071 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:07.066+0000 7fdb0b5b8640 1 -- 172.21.15.143:0/171265258 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdb0413c1a0 con 0x7fdb040cf990 2023-11-26T07:18:07.071 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:07.066+0000 7fdb0b5b8640 1 -- 172.21.15.143:0/171265258 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fdb0415b850 con 0x7fdb040cf990 2023-11-26T07:18:07.072 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:07.066+0000 7fdaf9ffb640 1 -- 172.21.15.143:0/171265258 <== mon.0 v2:172.21.15.143:3300/0 4 ==== mgrmap(e 5) v1 ==== 85852+0+0 (secure 0 0 0) 0x7fdaf4007470 con 0x7fdb040cf990 2023-11-26T07:18:07.072 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:07.066+0000 7fdaf9ffb640 1 --2- 172.21.15.143:0/171265258 >> [v2:172.21.15.143:6832/2150557544,v1:172.21.15.143:6833/2150557544] conn(0x7fdae8066470 0x7fdae8068930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:18:07.072 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:07.066+0000 7fdaf9ffb640 1 -- 172.21.15.143:0/171265258 <== mon.0 v2:172.21.15.143:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fdaf40834e0 con 0x7fdb040cf990 2023-11-26T07:18:07.072 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:07.066+0000 7fdb09354640 1 --2- 172.21.15.143:0/171265258 >> [v2:172.21.15.143:6832/2150557544,v1:172.21.15.143:6833/2150557544] conn(0x7fdae8066470 0x7fdae8068930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:18:07.072 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:07.066+0000 7fdb09354640 1 --2- 172.21.15.143:0/171265258 >> [v2:172.21.15.143:6832/2150557544,v1:172.21.15.143:6833/2150557544] conn(0x7fdae8066470 0x7fdae8068930 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7fdb000af000 tx=0x7fdb00008040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:18:07.075 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:07.070+0000 7fdaf9ffb640 1 -- 172.21.15.143:0/171265258 <== mon.0 v2:172.21.15.143:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172649 (secure 0 0 0) 0x7fdb0415b850 con 0x7fdb040cf990 2023-11-26T07:18:07.258 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:07.254+0000 7fdb0b5b8640 1 -- 172.21.15.143:0/171265258 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fdb04067680 con 0x7fdb040cf990 2023-11-26T07:18:07.259 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:07.254+0000 7fdaf9ffb640 1 -- 172.21.15.143:0/171265258 <== mon.0 v2:172.21.15.143:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 8 v8) v1 ==== 93+0+4913 (secure 0 0 0) 0x7fdaf4050950 con 0x7fdb040cf990 2023-11-26T07:18:07.259 INFO:teuthology.orchestra.run.smithi143.stdout: 2023-11-26T07:18:07.259 INFO:teuthology.orchestra.run.smithi143.stdout:{"epoch":8,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4241,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.188:6834/652687322","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6832","nonce":652687322},{"type":"v1","addr":"172.21.15.188:6834","nonce":652687322}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":4266,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.143:6835/3850184704","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6834","nonce":3850184704},{"type":"v1","addr":"172.21.15.143:6835","nonce":3850184704}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":2},{"gid":4273,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.143:6837/1695478651","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6836","nonce":1695478651},{"type":"v1","addr":"172.21.15.143:6837","nonce":1695478651}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":8,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-11-26T07:17:53.876989+0000","modified":"2023-11-26T07:17:56.885608+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":4285},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4285":{"gid":4285,"name":"b","rank":0,"incarnation":4,"state":"up:active","state_seq":3,"addr":"172.21.15.188:6835/994617268","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6833","nonce":994617268},{"type":"v1","addr":"172.21.15.188:6835","nonce":994617268}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2023-11-26T07:18:07.259 INFO:teuthology.orchestra.run.smithi143.stderr:dumped fsmap epoch 8 2023-11-26T07:18:07.261 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:07.258+0000 7fdb0b5b8640 1 -- 172.21.15.143:0/171265258 >> [v2:172.21.15.143:6832/2150557544,v1:172.21.15.143:6833/2150557544] conn(0x7fdae8066470 msgr2=0x7fdae8068930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:18:07.261 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:07.258+0000 7fdb0b5b8640 1 --2- 172.21.15.143:0/171265258 >> [v2:172.21.15.143:6832/2150557544,v1:172.21.15.143:6833/2150557544] conn(0x7fdae8066470 0x7fdae8068930 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7fdb000af000 tx=0x7fdb00008040 comp rx=0 tx=0).stop 2023-11-26T07:18:07.261 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:07.258+0000 7fdb0b5b8640 1 -- 172.21.15.143:0/171265258 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fdb040cf990 msgr2=0x7fdb04159e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:18:07.261 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:07.258+0000 7fdb0b5b8640 1 --2- 172.21.15.143:0/171265258 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fdb040cf990 0x7fdb04159e60 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7fdaf400c550 tx=0x7fdaf400ca10 comp rx=0 tx=0).stop 2023-11-26T07:18:07.261 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:07.258+0000 7fdb0b5b8640 1 -- 172.21.15.143:0/171265258 shutdown_connections 2023-11-26T07:18:07.262 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:07.258+0000 7fdb0b5b8640 1 --2- 172.21.15.143:0/171265258 >> [v2:172.21.15.143:6832/2150557544,v1:172.21.15.143:6833/2150557544] conn(0x7fdae8066470 0x7fdae8068930 unknown :-1 s=CLOSED pgs=113 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:18:07.262 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:07.258+0000 7fdb0b5b8640 1 --2- 172.21.15.143:0/171265258 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fdb04143ac0 0x7fdb0413bc60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:18:07.262 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:07.258+0000 7fdb0b5b8640 1 --2- 172.21.15.143:0/171265258 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fdb040cf990 0x7fdb04159e60 unknown :-1 s=CLOSED pgs=147 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:18:07.262 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:07.258+0000 7fdb0b5b8640 1 --2- 172.21.15.143:0/171265258 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fdb040708b0 0x7fdb0415bb30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:18:07.262 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:07.258+0000 7fdb0b5b8640 1 -- 172.21.15.143:0/171265258 >> 172.21.15.143:0/171265258 conn(0x7fdb0406a890 msgr2=0x7fdb040acdc0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:18:07.262 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:07.258+0000 7fdb0b5b8640 1 -- 172.21.15.143:0/171265258 shutdown_connections 2023-11-26T07:18:07.262 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:07.258+0000 7fdb0b5b8640 1 -- 172.21.15.143:0/171265258 wait complete. 2023-11-26T07:18:07.280 DEBUG:teuthology.orchestra.run.smithi143:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-11-26T07:18:07.416 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:07.410+0000 7f9cfca32640 1 Processor -- start 2023-11-26T07:18:07.416 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:07.410+0000 7f9cfca32640 1 -- start start 2023-11-26T07:18:07.417 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:07.414+0000 7f9cfca32640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f9cf80d1bc0 0x7f9cf80d0df0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:18:07.418 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:07.414+0000 7f9cfca32640 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7f9cf80b2160 0x7f9cf80b0750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:18:07.418 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:07.414+0000 7f9cfca32640 1 -- --> v1:172.21.15.143:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9cf80cf0e0 con 0x7f9cf815d6e0 2023-11-26T07:18:07.418 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:07.414+0000 7f9cfca32640 1 -- --> v2:172.21.15.188:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9cf815dac0 con 0x7f9cf80b2160 2023-11-26T07:18:07.418 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:07.414+0000 7f9cfca32640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9cf80ceb20 con 0x7f9cf80d1bc0 2023-11-26T07:18:07.418 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:07.414+0000 7f9cf659c640 1 --1- >> v1:172.21.15.143:6789/0 conn(0x7f9cf815d6e0 0x7f9cf80d14b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.143:6789/0 says I am v1:172.21.15.143:52230/0 (socket says 172.21.15.143:52230) 2023-11-26T07:18:07.418 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:07.414+0000 7f9cf659c640 1 -- 172.21.15.143:0/903852335 learned_addr learned my addr 172.21.15.143:0/903852335 (peer_addr_for_me v1:172.21.15.143:0/0) 2023-11-26T07:18:07.419 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:07.414+0000 7f9cedd9b640 1 --2- 172.21.15.143:0/903852335 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f9cf80d1bc0 0x7f9cf80d0df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:18:07.419 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:07.414+0000 7f9cf6d9d640 1 --2- 172.21.15.143:0/903852335 >> v2:172.21.15.188:3300/0 conn(0x7f9cf80b2160 0x7f9cf80b0750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:18:07.419 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:07.414+0000 7f9cf5d9b640 1 -- 172.21.15.143:0/903852335 <== mon.0 v1:172.21.15.143:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1757339694 0 0) 0x7f9cf80cf0e0 con 0x7f9cf815d6e0 2023-11-26T07:18:07.419 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:07.414+0000 7f9cf5d9b640 1 -- 172.21.15.143:0/903852335 --> v1:172.21.15.143:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f9ce4003530 con 0x7f9cf815d6e0 2023-11-26T07:18:07.419 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:07.414+0000 7f9cf6d9d640 1 -- 172.21.15.143:0/903852335 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f9cf80d1bc0 msgr2=0x7f9cf80d0df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-26T07:18:07.419 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:07.414+0000 7f9cf6d9d640 1 --2- 172.21.15.143:0/903852335 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f9cf80d1bc0 0x7f9cf80d0df0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:18:07.419 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:07.414+0000 7f9cf6d9d640 1 -- 172.21.15.143:0/903852335 >> v1:172.21.15.143:6789/0 conn(0x7f9cf815d6e0 legacy=0x7f9cf80d14b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:18:07.419 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:07.414+0000 7f9cf6d9d640 1 -- 172.21.15.143:0/903852335 --> v2:172.21.15.188:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9cf80cee00 con 0x7f9cf80b2160 2023-11-26T07:18:07.421 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:07.418+0000 7f9cf6d9d640 1 --2- 172.21.15.143:0/903852335 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f9cf80b2160 0x7f9cf80b0750 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7f9cf0009d90 tx=0x7f9cf001c950 comp rx=0 tx=0).ready entity=mon.1 client_cookie=761b8cc5bd01c4a1 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:18:07.423 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:07.418+0000 7f9cf5d9b640 1 -- 172.21.15.143:0/903852335 <== mon.1 v2:172.21.15.188:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9cf00a5900 con 0x7f9cf80b2160 2023-11-26T07:18:07.423 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:07.418+0000 7f9cf5d9b640 1 -- 172.21.15.143:0/903852335 <== mon.1 v2:172.21.15.188:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9cf00a5a60 con 0x7f9cf80b2160 2023-11-26T07:18:07.423 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:07.418+0000 7f9cf5d9b640 1 -- 172.21.15.143:0/903852335 <== mon.1 v2:172.21.15.188:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9cf00a5df0 con 0x7f9cf80b2160 2023-11-26T07:18:07.423 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:07.418+0000 7f9cfca32640 1 -- 172.21.15.143:0/903852335 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f9cf80b2160 msgr2=0x7f9cf80b0750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:18:07.423 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:07.418+0000 7f9cfca32640 1 --2- 172.21.15.143:0/903852335 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f9cf80b2160 0x7f9cf80b0750 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7f9cf0009d90 tx=0x7f9cf001c950 comp rx=0 tx=0).stop 2023-11-26T07:18:07.423 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:07.418+0000 7f9cfca32640 1 -- 172.21.15.143:0/903852335 shutdown_connections 2023-11-26T07:18:07.424 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:07.418+0000 7f9cfca32640 1 --2- 172.21.15.143:0/903852335 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f9cf80b2160 0x7f9cf80b0750 unknown :-1 s=CLOSED pgs=100 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:18:07.424 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:07.418+0000 7f9cfca32640 1 --2- 172.21.15.143:0/903852335 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f9cf80d1bc0 0x7f9cf80d0df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:18:07.424 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:07.418+0000 7f9cfca32640 1 -- 172.21.15.143:0/903852335 >> 172.21.15.143:0/903852335 conn(0x7f9cf809c550 msgr2=0x7f9cf80d3220 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:18:07.424 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:07.418+0000 7f9cfca32640 1 -- 172.21.15.143:0/903852335 shutdown_connections 2023-11-26T07:18:07.424 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:07.418+0000 7f9cfca32640 1 -- 172.21.15.143:0/903852335 wait complete. 2023-11-26T07:18:07.424 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:07.418+0000 7f9cfca32640 1 Processor -- start 2023-11-26T07:18:07.424 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:07.418+0000 7f9cfca32640 1 -- start start 2023-11-26T07:18:07.426 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:07.422+0000 7f9cfca32640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f9cf80b2160 0x7f9cf80ca550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:18:07.426 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:07.422+0000 7f9cfca32640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f9cf80d1bc0 0x7f9cf80b6940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:18:07.426 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:07.422+0000 7f9cfca32640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f9cf815d6e0 0x7f9cf8154bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:18:07.426 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:07.422+0000 7f9cfca32640 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9cf815ec50 con 0x7f9cf80b2160 2023-11-26T07:18:07.427 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:07.422+0000 7f9cfca32640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9cf815f790 con 0x7f9cf80d1bc0 2023-11-26T07:18:07.427 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:07.422+0000 7f9cfca32640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9cf815ee60 con 0x7f9cf815d6e0 2023-11-26T07:18:07.427 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:07.422+0000 7f9cf659c640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f9cf80b2160 0x7f9cf80ca550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:18:07.427 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:07.422+0000 7f9cf659c640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f9cf80b2160 0x7f9cf80ca550 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:48286/0 (socket says 172.21.15.143:48286) 2023-11-26T07:18:07.427 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:07.422+0000 7f9cf659c640 1 -- 172.21.15.143:0/2632672354 learned_addr learned my addr 172.21.15.143:0/2632672354 (peer_addr_for_me v2:172.21.15.143:0/0) 2023-11-26T07:18:07.427 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:07.422+0000 7f9cedd9b640 1 --2- 172.21.15.143:0/2632672354 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f9cf80d1bc0 0x7f9cf80b6940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:18:07.427 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:07.422+0000 7f9cf6d9d640 1 --2- 172.21.15.143:0/2632672354 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f9cf815d6e0 0x7f9cf8154bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:18:07.427 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:07.422+0000 7f9cf659c640 1 -- 172.21.15.143:0/2632672354 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f9cf815d6e0 msgr2=0x7f9cf8154bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:18:07.427 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:07.422+0000 7f9cf659c640 1 --2- 172.21.15.143:0/2632672354 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f9cf815d6e0 0x7f9cf8154bf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:18:07.427 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:07.422+0000 7f9cf659c640 1 -- 172.21.15.143:0/2632672354 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f9cf80d1bc0 msgr2=0x7f9cf80b6940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:18:07.427 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:07.422+0000 7f9cf659c640 1 --2- 172.21.15.143:0/2632672354 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f9cf80d1bc0 0x7f9cf80b6940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:18:07.428 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:07.422+0000 7f9cf659c640 1 -- 172.21.15.143:0/2632672354 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9cf0009a70 con 0x7f9cf80b2160 2023-11-26T07:18:07.428 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:07.422+0000 7f9cf659c640 1 --2- 172.21.15.143:0/2632672354 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f9cf80b2160 0x7f9cf80ca550 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7f9ce800c420 tx=0x7f9ce800c8e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:18:07.428 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:07.422+0000 7f9cef7fe640 1 -- 172.21.15.143:0/2632672354 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9ce8005cf0 con 0x7f9cf80b2160 2023-11-26T07:18:07.429 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:07.422+0000 7f9cef7fe640 1 -- 172.21.15.143:0/2632672354 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9ce8005e50 con 0x7f9cf80b2160 2023-11-26T07:18:07.429 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:07.422+0000 7f9cfca32640 1 -- 172.21.15.143:0/2632672354 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9cf815f140 con 0x7f9cf80b2160 2023-11-26T07:18:07.429 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:07.422+0000 7f9cef7fe640 1 -- 172.21.15.143:0/2632672354 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9ce80072d0 con 0x7f9cf80b2160 2023-11-26T07:18:07.429 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:07.422+0000 7f9cfca32640 1 -- 172.21.15.143:0/2632672354 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9cf8155960 con 0x7f9cf80b2160 2023-11-26T07:18:07.429 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:07.422+0000 7f9cfca32640 1 -- 172.21.15.143:0/2632672354 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9cf806c7e0 con 0x7f9cf80b2160 2023-11-26T07:18:07.429 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:07.426+0000 7f9cef7fe640 1 -- 172.21.15.143:0/2632672354 <== mon.0 v2:172.21.15.143:3300/0 4 ==== mgrmap(e 5) v1 ==== 85852+0+0 (secure 0 0 0) 0x7f9ce8007430 con 0x7f9cf80b2160 2023-11-26T07:18:07.430 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:07.426+0000 7f9cef7fe640 1 --2- 172.21.15.143:0/2632672354 >> [v2:172.21.15.143:6832/2150557544,v1:172.21.15.143:6833/2150557544] conn(0x7f9cd8066470 0x7f9cd8068930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:18:07.430 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:07.426+0000 7f9cef7fe640 1 -- 172.21.15.143:0/2632672354 <== mon.0 v2:172.21.15.143:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f9ce8083320 con 0x7f9cf80b2160 2023-11-26T07:18:07.430 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:07.426+0000 7f9cedd9b640 1 --2- 172.21.15.143:0/2632672354 >> [v2:172.21.15.143:6832/2150557544,v1:172.21.15.143:6833/2150557544] conn(0x7f9cd8066470 0x7f9cd8068930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:18:07.430 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:07.426+0000 7f9cedd9b640 1 --2- 172.21.15.143:0/2632672354 >> [v2:172.21.15.143:6832/2150557544,v1:172.21.15.143:6833/2150557544] conn(0x7f9cd8066470 0x7f9cd8068930 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7f9cf8159310 tx=0x7f9ce0009190 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:18:07.434 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:07.430+0000 7f9cef7fe640 1 -- 172.21.15.143:0/2632672354 <== mon.0 v2:172.21.15.143:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172649 (secure 0 0 0) 0x7f9ce8050840 con 0x7f9cf80b2160 2023-11-26T07:18:07.583 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:07.578+0000 7f9cfca32640 1 -- 172.21.15.143:0/2632672354 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f9cf80d29a0 con 0x7f9cf80b2160 2023-11-26T07:18:07.584 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:07.578+0000 7f9cef7fe640 1 -- 172.21.15.143:0/2632672354 <== mon.0 v2:172.21.15.143:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v20) v1 ==== 74+0+16541 (secure 0 0 0) 0x7f9ce8053e60 con 0x7f9cf80b2160 2023-11-26T07:18:07.584 INFO:teuthology.orchestra.run.smithi143.stdout: 2023-11-26T07:18:07.584 INFO:teuthology.orchestra.run.smithi143.stdout:{"epoch":20,"fsid":"f5033060-2469-4a07-95fe-2b76f5404bc1","created":"2023-11-26T07:17:28.613663+0000","modified":"2023-11-26T07:17:55.505435+0000","last_up_change":"2023-11-26T07:17:41.157813+0000","last_in_change":"2023-11-26T07:17:31.181658+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-11-26T07:17:41.758597+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2023-11-26T07:17:51.878332+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2023-11-26T07:17:52.871396+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"ebbee2c6-023e-44b9-9f1d-6f7bfd309730","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6817","nonce":639807369},{"type":"v1","addr":"172.21.15.143:6819","nonce":639807369}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6821","nonce":639807369},{"type":"v1","addr":"172.21.15.143:6823","nonce":639807369}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6830","nonce":639807369},{"type":"v1","addr":"172.21.15.143:6831","nonce":639807369}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6825","nonce":639807369},{"type":"v1","addr":"172.21.15.143:6828","nonce":639807369}]},"public_addr":"172.21.15.143:6819/639807369","cluster_addr":"172.21.15.143:6823/639807369","heartbeat_back_addr":"172.21.15.143:6831/639807369","heartbeat_front_addr":"172.21.15.143:6828/639807369","state":["exists","up"]},{"osd":1,"uuid":"fec4b8be-7122-49e9-9a19-5850a0700001","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6816","nonce":1144942530},{"type":"v1","addr":"172.21.15.143:6818","nonce":1144942530}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6820","nonce":1144942530},{"type":"v1","addr":"172.21.15.143:6822","nonce":1144942530}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6827","nonce":1144942530},{"type":"v1","addr":"172.21.15.143:6829","nonce":1144942530}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6824","nonce":1144942530},{"type":"v1","addr":"172.21.15.143:6826","nonce":1144942530}]},"public_addr":"172.21.15.143:6818/1144942530","cluster_addr":"172.21.15.143:6822/1144942530","heartbeat_back_addr":"172.21.15.143:6829/1144942530","heartbeat_front_addr":"172.21.15.143:6826/1144942530","state":["exists","up"]},{"osd":2,"uuid":"2888af5b-f933-46ca-8f95-6dd494ff43f6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6808","nonce":3909095186},{"type":"v1","addr":"172.21.15.143:6809","nonce":3909095186}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6810","nonce":3909095186},{"type":"v1","addr":"172.21.15.143:6811","nonce":3909095186}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6814","nonce":3909095186},{"type":"v1","addr":"172.21.15.143:6815","nonce":3909095186}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6812","nonce":3909095186},{"type":"v1","addr":"172.21.15.143:6813","nonce":3909095186}]},"public_addr":"172.21.15.143:6809/3909095186","cluster_addr":"172.21.15.143:6811/3909095186","heartbeat_back_addr":"172.21.15.143:6815/3909095186","heartbeat_front_addr":"172.21.15.143:6813/3909095186","state":["exists","up"]},{"osd":3,"uuid":"d1f52435-7bc1-4081-9550-9cd1ba55f549","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6800","nonce":3854210991},{"type":"v1","addr":"172.21.15.143:6801","nonce":3854210991}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6802","nonce":3854210991},{"type":"v1","addr":"172.21.15.143:6803","nonce":3854210991}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6806","nonce":3854210991},{"type":"v1","addr":"172.21.15.143:6807","nonce":3854210991}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6804","nonce":3854210991},{"type":"v1","addr":"172.21.15.143:6805","nonce":3854210991}]},"public_addr":"172.21.15.143:6801/3854210991","cluster_addr":"172.21.15.143:6803/3854210991","heartbeat_back_addr":"172.21.15.143:6807/3854210991","heartbeat_front_addr":"172.21.15.143:6805/3854210991","state":["exists","up"]},{"osd":4,"uuid":"e2ae8b6e-6e25-49b2-bcbf-aacce68f06a3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6800","nonce":3924606902},{"type":"v1","addr":"172.21.15.188:6801","nonce":3924606902}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6802","nonce":3924606902},{"type":"v1","addr":"172.21.15.188:6803","nonce":3924606902}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6806","nonce":3924606902},{"type":"v1","addr":"172.21.15.188:6807","nonce":3924606902}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6804","nonce":3924606902},{"type":"v1","addr":"172.21.15.188:6805","nonce":3924606902}]},"public_addr":"172.21.15.188:6801/3924606902","cluster_addr":"172.21.15.188:6803/3924606902","heartbeat_back_addr":"172.21.15.188:6807/3924606902","heartbeat_front_addr":"172.21.15.188:6805/3924606902","state":["exists","up"]},{"osd":5,"uuid":"3cec8a32-7698-4c36-8e89-e4ba11934f33","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6813","nonce":186841544},{"type":"v1","addr":"172.21.15.188:6815","nonce":186841544}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6817","nonce":186841544},{"type":"v1","addr":"172.21.15.188:6818","nonce":186841544}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6822","nonce":186841544},{"type":"v1","addr":"172.21.15.188:6823","nonce":186841544}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6820","nonce":186841544},{"type":"v1","addr":"172.21.15.188:6821","nonce":186841544}]},"public_addr":"172.21.15.188:6815/186841544","cluster_addr":"172.21.15.188:6818/186841544","heartbeat_back_addr":"172.21.15.188:6823/186841544","heartbeat_front_addr":"172.21.15.188:6821/186841544","state":["exists","up"]},{"osd":6,"uuid":"58a65d5d-e090-473a-81b4-ad38e85029f9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6808","nonce":2153499405},{"type":"v1","addr":"172.21.15.188:6809","nonce":2153499405}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6810","nonce":2153499405},{"type":"v1","addr":"172.21.15.188:6811","nonce":2153499405}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6816","nonce":2153499405},{"type":"v1","addr":"172.21.15.188:6819","nonce":2153499405}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6812","nonce":2153499405},{"type":"v1","addr":"172.21.15.188:6814","nonce":2153499405}]},"public_addr":"172.21.15.188:6809/2153499405","cluster_addr":"172.21.15.188:6811/2153499405","heartbeat_back_addr":"172.21.15.188:6819/2153499405","heartbeat_front_addr":"172.21.15.188:6814/2153499405","state":["exists","up"]},{"osd":7,"uuid":"ea5b7f70-3554-4cf8-ac30-f7c59b80678d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6824","nonce":2722096729},{"type":"v1","addr":"172.21.15.188:6825","nonce":2722096729}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6826","nonce":2722096729},{"type":"v1","addr":"172.21.15.188:6827","nonce":2722096729}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6830","nonce":2722096729},{"type":"v1","addr":"172.21.15.188:6831","nonce":2722096729}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6828","nonce":2722096729},{"type":"v1","addr":"172.21.15.188:6829","nonce":2722096729}]},"public_addr":"172.21.15.188:6825/2722096729","cluster_addr":"172.21.15.188:6827/2722096729","heartbeat_back_addr":"172.21.15.188:6831/2722096729","heartbeat_front_addr":"172.21.15.188:6829/2722096729","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-26T07:17:36.995032+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-26T07:17:36.932266+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-26T07:17:36.829304+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-26T07:17:36.804297+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-26T07:17:36.667892+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-26T07:17:36.964868+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-26T07:17:36.827332+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-26T07:17:36.893179+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-11-26T07:18:07.586 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:07.582+0000 7f9cfca32640 1 -- 172.21.15.143:0/2632672354 >> [v2:172.21.15.143:6832/2150557544,v1:172.21.15.143:6833/2150557544] conn(0x7f9cd8066470 msgr2=0x7f9cd8068930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:18:07.586 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:07.582+0000 7f9cfca32640 1 --2- 172.21.15.143:0/2632672354 >> [v2:172.21.15.143:6832/2150557544,v1:172.21.15.143:6833/2150557544] conn(0x7f9cd8066470 0x7f9cd8068930 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7f9cf8159310 tx=0x7f9ce0009190 comp rx=0 tx=0).stop 2023-11-26T07:18:07.587 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:07.582+0000 7f9cfca32640 1 -- 172.21.15.143:0/2632672354 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f9cf80b2160 msgr2=0x7f9cf80ca550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:18:07.587 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:07.582+0000 7f9cfca32640 1 --2- 172.21.15.143:0/2632672354 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f9cf80b2160 0x7f9cf80ca550 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7f9ce800c420 tx=0x7f9ce800c8e0 comp rx=0 tx=0).stop 2023-11-26T07:18:07.587 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:07.582+0000 7f9cfca32640 1 -- 172.21.15.143:0/2632672354 shutdown_connections 2023-11-26T07:18:07.587 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:07.582+0000 7f9cfca32640 1 --2- 172.21.15.143:0/2632672354 >> [v2:172.21.15.143:6832/2150557544,v1:172.21.15.143:6833/2150557544] conn(0x7f9cd8066470 0x7f9cd8068930 unknown :-1 s=CLOSED pgs=114 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:18:07.587 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:07.582+0000 7f9cfca32640 1 --2- 172.21.15.143:0/2632672354 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f9cf815d6e0 0x7f9cf8154bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:18:07.587 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:07.582+0000 7f9cfca32640 1 --2- 172.21.15.143:0/2632672354 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f9cf80d1bc0 0x7f9cf80b6940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:18:07.587 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:07.582+0000 7f9cfca32640 1 --2- 172.21.15.143:0/2632672354 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f9cf80b2160 0x7f9cf80ca550 unknown :-1 s=CLOSED pgs=149 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:18:07.587 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:07.582+0000 7f9cfca32640 1 -- 172.21.15.143:0/2632672354 >> 172.21.15.143:0/2632672354 conn(0x7f9cf809c550 msgr2=0x7f9cf80b25a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:18:07.587 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:07.582+0000 7f9cfca32640 1 -- 172.21.15.143:0/2632672354 shutdown_connections 2023-11-26T07:18:07.587 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:07.582+0000 7f9cfca32640 1 -- 172.21.15.143:0/2632672354 wait complete. 2023-11-26T07:18:07.606 DEBUG:teuthology.orchestra.run.smithi143:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-11-26T07:18:07.744 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:07.738+0000 7fb4be646640 1 Processor -- start 2023-11-26T07:18:07.744 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:07.738+0000 7fb4be646640 1 -- start start 2023-11-26T07:18:07.745 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:07.742+0000 7fb4be646640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fb4b813e290 0x7fb4b80cf280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:18:07.745 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:07.742+0000 7fb4be646640 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7fb4b80cf7c0 0x7fb4b815efa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:18:07.746 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:07.742+0000 7fb4be646640 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7fb4b815f4e0 0x7fb4b81408e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:18:07.746 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:07.742+0000 7fb4be646640 1 -- --> v2:172.21.15.143:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb4b80cfc40 con 0x7fb4b80cf7c0 2023-11-26T07:18:07.746 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:07.742+0000 7fb4be646640 1 -- --> v2:172.21.15.188:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb4b806c520 con 0x7fb4b815f4e0 2023-11-26T07:18:07.746 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:07.742+0000 7fb4be646640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb4b806c800 con 0x7fb4b813e290 2023-11-26T07:18:07.746 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:07.742+0000 7fb4b77fe640 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7fb4b80cf7c0 0x7fb4b815efa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:18:07.746 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:07.742+0000 7fb4b77fe640 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7fb4b80cf7c0 0x7fb4b815efa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:48288/0 (socket says 172.21.15.143:48288) 2023-11-26T07:18:07.746 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:07.742+0000 7fb4b77fe640 1 -- 172.21.15.143:0/3697507212 learned_addr learned my addr 172.21.15.143:0/3697507212 (peer_addr_for_me v2:172.21.15.143:0/0) 2023-11-26T07:18:07.747 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:07.742+0000 7fb4b7fff640 1 --2- 172.21.15.143:0/3697507212 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fb4b813e290 0x7fb4b80cf280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:18:07.747 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:07.742+0000 7fb4bcbe3640 1 --2- 172.21.15.143:0/3697507212 >> v2:172.21.15.188:3300/0 conn(0x7fb4b815f4e0 0x7fb4b81408e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:18:07.747 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:07.742+0000 7fb4b77fe640 1 -- 172.21.15.143:0/3697507212 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fb4b813e290 msgr2=0x7fb4b80cf280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-26T07:18:07.747 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:07.742+0000 7fb4b77fe640 1 --2- 172.21.15.143:0/3697507212 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fb4b813e290 0x7fb4b80cf280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:18:07.747 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:07.742+0000 7fb4b77fe640 1 -- 172.21.15.143:0/3697507212 >> v2:172.21.15.188:3300/0 conn(0x7fb4b815f4e0 msgr2=0x7fb4b81408e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-26T07:18:07.747 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:07.742+0000 7fb4b77fe640 1 --2- 172.21.15.143:0/3697507212 >> v2:172.21.15.188:3300/0 conn(0x7fb4b815f4e0 0x7fb4b81408e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:18:07.747 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:07.742+0000 7fb4b77fe640 1 -- 172.21.15.143:0/3697507212 --> v2:172.21.15.143:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb4b806cae0 con 0x7fb4b80cf7c0 2023-11-26T07:18:07.748 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:07.742+0000 7fb4bcbe3640 1 --2- 172.21.15.143:0/3697507212 >> v2:172.21.15.188:3300/0 conn(0x7fb4b815f4e0 0x7fb4b81408e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-26T07:18:07.749 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:07.746+0000 7fb4b77fe640 1 --2- 172.21.15.143:0/3697507212 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fb4b80cf7c0 0x7fb4b815efa0 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7fb4b8072660 tx=0x7fb4ac01c210 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c519812c867c7707 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:18:07.749 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:07.746+0000 7fb4b6ffd640 1 -- 172.21.15.143:0/3697507212 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb4ac0ab600 con 0x7fb4b80cf7c0 2023-11-26T07:18:07.749 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:07.746+0000 7fb4b6ffd640 1 -- 172.21.15.143:0/3697507212 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb4ac0a7050 con 0x7fb4b80cf7c0 2023-11-26T07:18:07.749 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:07.746+0000 7fb4b6ffd640 1 -- 172.21.15.143:0/3697507212 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb4ac0ab980 con 0x7fb4b80cf7c0 2023-11-26T07:18:07.750 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:07.746+0000 7fb4be646640 1 -- 172.21.15.143:0/3697507212 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fb4b80cf7c0 msgr2=0x7fb4b815efa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:18:07.750 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:07.746+0000 7fb4be646640 1 --2- 172.21.15.143:0/3697507212 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fb4b80cf7c0 0x7fb4b815efa0 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7fb4b8072660 tx=0x7fb4ac01c210 comp rx=0 tx=0).stop 2023-11-26T07:18:07.750 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:07.746+0000 7fb4be646640 1 -- 172.21.15.143:0/3697507212 shutdown_connections 2023-11-26T07:18:07.750 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:07.746+0000 7fb4be646640 1 --2- 172.21.15.143:0/3697507212 >> v2:172.21.15.188:3300/0 conn(0x7fb4b815f4e0 0x7fb4b81408e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:18:07.750 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:07.746+0000 7fb4be646640 1 --2- 172.21.15.143:0/3697507212 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fb4b80cf7c0 0x7fb4b815efa0 unknown :-1 s=CLOSED pgs=150 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:18:07.750 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:07.746+0000 7fb4be646640 1 --2- 172.21.15.143:0/3697507212 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fb4b813e290 0x7fb4b80cf280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:18:07.750 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:07.746+0000 7fb4be646640 1 -- 172.21.15.143:0/3697507212 >> 172.21.15.143:0/3697507212 conn(0x7fb4b806a890 msgr2=0x7fb4b806aca0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:18:07.751 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:07.746+0000 7fb4be646640 1 -- 172.21.15.143:0/3697507212 shutdown_connections 2023-11-26T07:18:07.751 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:07.746+0000 7fb4be646640 1 -- 172.21.15.143:0/3697507212 wait complete. 2023-11-26T07:18:07.751 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:07.746+0000 7fb4be646640 1 Processor -- start 2023-11-26T07:18:07.751 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:07.746+0000 7fb4be646640 1 -- start start 2023-11-26T07:18:07.754 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:07.746+0000 7fb4be646640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fb4b80cf7c0 0x7fb4b8137140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:18:07.754 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:07.746+0000 7fb4be646640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fb4b813e290 0x7fb4b813c8e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:18:07.754 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:07.750+0000 7fb4be646640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fb4b815f4e0 0x7fb4b813ce20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:18:07.754 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:07.750+0000 7fb4be646640 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb4b8072040 con 0x7fb4b815f4e0 2023-11-26T07:18:07.754 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:07.750+0000 7fb4be646640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb4b80704a0 con 0x7fb4b813e290 2023-11-26T07:18:07.754 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:07.750+0000 7fb4be646640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb4b8070780 con 0x7fb4b80cf7c0 2023-11-26T07:18:07.754 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:07.750+0000 7fb4bcbe3640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fb4b815f4e0 0x7fb4b813ce20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:18:07.754 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:07.750+0000 7fb4bcbe3640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fb4b815f4e0 0x7fb4b813ce20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:48296/0 (socket says 172.21.15.143:48296) 2023-11-26T07:18:07.755 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:07.750+0000 7fb4bcbe3640 1 -- 172.21.15.143:0/422788972 learned_addr learned my addr 172.21.15.143:0/422788972 (peer_addr_for_me v2:172.21.15.143:0/0) 2023-11-26T07:18:07.755 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:07.750+0000 7fb4b7fff640 1 --2- 172.21.15.143:0/422788972 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fb4b80cf7c0 0x7fb4b8137140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:18:07.755 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:07.750+0000 7fb4b77fe640 1 --2- 172.21.15.143:0/422788972 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fb4b813e290 0x7fb4b813c8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:18:07.755 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:07.750+0000 7fb4bcbe3640 1 -- 172.21.15.143:0/422788972 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fb4b80cf7c0 msgr2=0x7fb4b8137140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:18:07.755 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:07.750+0000 7fb4bcbe3640 1 --2- 172.21.15.143:0/422788972 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fb4b80cf7c0 0x7fb4b8137140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:18:07.755 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:07.750+0000 7fb4bcbe3640 1 -- 172.21.15.143:0/422788972 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fb4b813e290 msgr2=0x7fb4b813c8e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:18:07.755 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:07.750+0000 7fb4bcbe3640 1 --2- 172.21.15.143:0/422788972 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fb4b813e290 0x7fb4b813c8e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:18:07.755 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:07.750+0000 7fb4bcbe3640 1 -- 172.21.15.143:0/422788972 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb4ac0094d0 con 0x7fb4b815f4e0 2023-11-26T07:18:07.755 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:07.750+0000 7fb4bcbe3640 1 --2- 172.21.15.143:0/422788972 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fb4b815f4e0 0x7fb4b813ce20 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7fb4b000cd50 tx=0x7fb4b000a650 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:18:07.755 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:07.750+0000 7fb4b4ff9640 1 -- 172.21.15.143:0/422788972 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb4b000f680 con 0x7fb4b815f4e0 2023-11-26T07:18:07.755 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:07.750+0000 7fb4b4ff9640 1 -- 172.21.15.143:0/422788972 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb4b000f800 con 0x7fb4b815f4e0 2023-11-26T07:18:07.756 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:07.750+0000 7fb4be646640 1 -- 172.21.15.143:0/422788972 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb4b8070a60 con 0x7fb4b815f4e0 2023-11-26T07:18:07.756 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:07.750+0000 7fb4b4ff9640 1 -- 172.21.15.143:0/422788972 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb4b000fbb0 con 0x7fb4b815f4e0 2023-11-26T07:18:07.756 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:07.750+0000 7fb4be646640 1 -- 172.21.15.143:0/422788972 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb4b8137680 con 0x7fb4b815f4e0 2023-11-26T07:18:07.756 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:07.750+0000 7fb4be646640 1 -- 172.21.15.143:0/422788972 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb4b8070f80 con 0x7fb4b815f4e0 2023-11-26T07:18:07.756 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:07.750+0000 7fb4b4ff9640 1 -- 172.21.15.143:0/422788972 <== mon.0 v2:172.21.15.143:3300/0 4 ==== mgrmap(e 5) v1 ==== 85852+0+0 (secure 0 0 0) 0x7fb4b000b040 con 0x7fb4b815f4e0 2023-11-26T07:18:07.756 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:07.754+0000 7fb4b4ff9640 1 --2- 172.21.15.143:0/422788972 >> [v2:172.21.15.143:6832/2150557544,v1:172.21.15.143:6833/2150557544] conn(0x7fb4a40664a0 0x7fb4a4068960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:18:07.757 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:07.754+0000 7fb4b4ff9640 1 -- 172.21.15.143:0/422788972 <== mon.0 v2:172.21.15.143:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fb4b00847e0 con 0x7fb4b815f4e0 2023-11-26T07:18:07.757 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:07.754+0000 7fb4b7fff640 1 --2- 172.21.15.143:0/422788972 >> [v2:172.21.15.143:6832/2150557544,v1:172.21.15.143:6833/2150557544] conn(0x7fb4a40664a0 0x7fb4a4068960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:18:07.757 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:07.754+0000 7fb4b7fff640 1 --2- 172.21.15.143:0/422788972 >> [v2:172.21.15.143:6832/2150557544,v1:172.21.15.143:6833/2150557544] conn(0x7fb4a40664a0 0x7fb4a4068960 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7fb4a80045f0 tx=0x7fb4a8008040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:18:07.760 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:07.754+0000 7fb4b4ff9640 1 -- 172.21.15.143:0/422788972 <== mon.0 v2:172.21.15.143:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172649 (secure 0 0 0) 0x7fb4b8070f80 con 0x7fb4b815f4e0 2023-11-26T07:18:07.942 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:07.938+0000 7fb4be646640 1 -- 172.21.15.143:0/422788972 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fb4b8070f80 con 0x7fb4b815f4e0 2023-11-26T07:18:07.943 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:07.938+0000 7fb4b4ff9640 1 -- 172.21.15.143:0/422788972 <== mon.0 v2:172.21.15.143:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 8 v8) v1 ==== 93+0+4913 (secure 0 0 0) 0x7fb4b8070f80 con 0x7fb4b815f4e0 2023-11-26T07:18:07.943 INFO:teuthology.orchestra.run.smithi143.stdout: 2023-11-26T07:18:07.943 INFO:teuthology.orchestra.run.smithi143.stdout:{"epoch":8,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4241,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.188:6834/652687322","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6832","nonce":652687322},{"type":"v1","addr":"172.21.15.188:6834","nonce":652687322}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":4266,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.143:6835/3850184704","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6834","nonce":3850184704},{"type":"v1","addr":"172.21.15.143:6835","nonce":3850184704}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":2},{"gid":4273,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.143:6837/1695478651","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6836","nonce":1695478651},{"type":"v1","addr":"172.21.15.143:6837","nonce":1695478651}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":8,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-11-26T07:17:53.876989+0000","modified":"2023-11-26T07:17:56.885608+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":4285},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4285":{"gid":4285,"name":"b","rank":0,"incarnation":4,"state":"up:active","state_seq":3,"addr":"172.21.15.188:6835/994617268","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6833","nonce":994617268},{"type":"v1","addr":"172.21.15.188:6835","nonce":994617268}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2023-11-26T07:18:07.943 INFO:teuthology.orchestra.run.smithi143.stderr:dumped fsmap epoch 8 2023-11-26T07:18:07.944 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:07.938+0000 7fb4be646640 1 -- 172.21.15.143:0/422788972 >> [v2:172.21.15.143:6832/2150557544,v1:172.21.15.143:6833/2150557544] conn(0x7fb4a40664a0 msgr2=0x7fb4a4068960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:18:07.945 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:07.938+0000 7fb4be646640 1 --2- 172.21.15.143:0/422788972 >> [v2:172.21.15.143:6832/2150557544,v1:172.21.15.143:6833/2150557544] conn(0x7fb4a40664a0 0x7fb4a4068960 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7fb4a80045f0 tx=0x7fb4a8008040 comp rx=0 tx=0).stop 2023-11-26T07:18:07.945 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:07.938+0000 7fb4be646640 1 -- 172.21.15.143:0/422788972 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fb4b815f4e0 msgr2=0x7fb4b813ce20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:18:07.945 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:07.938+0000 7fb4be646640 1 --2- 172.21.15.143:0/422788972 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fb4b815f4e0 0x7fb4b813ce20 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7fb4b000cd50 tx=0x7fb4b000a650 comp rx=0 tx=0).stop 2023-11-26T07:18:07.945 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:07.938+0000 7fb4be646640 1 -- 172.21.15.143:0/422788972 shutdown_connections 2023-11-26T07:18:07.945 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:07.938+0000 7fb4be646640 1 --2- 172.21.15.143:0/422788972 >> [v2:172.21.15.143:6832/2150557544,v1:172.21.15.143:6833/2150557544] conn(0x7fb4a40664a0 0x7fb4a4068960 unknown :-1 s=CLOSED pgs=115 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:18:07.945 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:07.938+0000 7fb4be646640 1 --2- 172.21.15.143:0/422788972 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fb4b815f4e0 0x7fb4b813ce20 unknown :-1 s=CLOSED pgs=151 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:18:07.945 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:07.938+0000 7fb4be646640 1 --2- 172.21.15.143:0/422788972 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fb4b813e290 0x7fb4b813c8e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:18:07.945 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:07.938+0000 7fb4be646640 1 --2- 172.21.15.143:0/422788972 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fb4b80cf7c0 0x7fb4b8137140 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:18:07.945 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:07.938+0000 7fb4be646640 1 -- 172.21.15.143:0/422788972 >> 172.21.15.143:0/422788972 conn(0x7fb4b806a890 msgr2=0x7fb4b80b3cc0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:18:07.945 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:07.938+0000 7fb4be646640 1 -- 172.21.15.143:0/422788972 shutdown_connections 2023-11-26T07:18:07.945 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:07.938+0000 7fb4be646640 1 -- 172.21.15.143:0/422788972 wait complete. 2023-11-26T07:18:08.367 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 8, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2023-11-26T07:17:53.876989+0000', 'modified': '2023-11-26T07:17:56.885608+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 4285}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_4285': {'gid': 4285, 'name': 'b', 'rank': 0, 'incarnation': 4, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.188:6835/994617268', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.188:6833', 'nonce': 994617268}, {'type': 'v1', 'addr': '172.21.15.188:6835', 'nonce': 994617268}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138322906710015, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}}}, 'data_pools': [3], 'metadata_pool': 2, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0} 2023-11-26T07:18:08.367 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2023-11-26T07:18:08.368 DEBUG:teuthology.orchestra.run.smithi143:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.b status 2023-11-26T07:18:08.509 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:08.506+0000 7fe42ff94640 1 Processor -- start 2023-11-26T07:18:08.509 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:08.506+0000 7fe42ff94640 1 -- start start 2023-11-26T07:18:08.511 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:08.506+0000 7fe42ff94640 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7fe428143ac0 0x7fe428070370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:18:08.511 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:08.506+0000 7fe42ff94640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fe4280cf990 0x7fe4280cd1b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:18:08.511 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:08.506+0000 7fe42ff94640 1 -- --> v1:172.21.15.188:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe4280cc730 con 0x7fe4280708b0 2023-11-26T07:18:08.511 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:08.506+0000 7fe42ff94640 1 -- --> v2:172.21.15.143:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe428143ea0 con 0x7fe428143ac0 2023-11-26T07:18:08.511 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:08.506+0000 7fe42ff94640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe428070ee0 con 0x7fe4280cf990 2023-11-26T07:18:08.511 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:08.506+0000 7fe42dd30640 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7fe428143ac0 0x7fe428070370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:18:08.512 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:08.506+0000 7fe42dd30640 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7fe428143ac0 0x7fe428070370 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:48304/0 (socket says 172.21.15.143:48304) 2023-11-26T07:18:08.512 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:08.506+0000 7fe42dd30640 1 -- 172.21.15.143:0/3015987252 learned_addr learned my addr 172.21.15.143:0/3015987252 (peer_addr_for_me v2:172.21.15.143:0/0) 2023-11-26T07:18:08.512 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:08.506+0000 7fe42e531640 1 --2- 172.21.15.143:0/3015987252 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fe4280cf990 0x7fe4280cd1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:18:08.512 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:08.506+0000 7fe42dd30640 1 -- 172.21.15.143:0/3015987252 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fe4280cf990 msgr2=0x7fe4280cd1b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-26T07:18:08.512 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:08.506+0000 7fe42dd30640 1 --2- 172.21.15.143:0/3015987252 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fe4280cf990 0x7fe4280cd1b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:18:08.512 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:08.506+0000 7fe42dd30640 1 -- 172.21.15.143:0/3015987252 >> v1:172.21.15.188:6789/0 conn(0x7fe4280708b0 legacy=0x7fe4280cf280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-26T07:18:08.512 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:08.506+0000 7fe42dd30640 1 -- 172.21.15.143:0/3015987252 --> v2:172.21.15.143:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe4280711c0 con 0x7fe428143ac0 2023-11-26T07:18:08.514 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:08.510+0000 7fe42dd30640 1 --2- 172.21.15.143:0/3015987252 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fe428143ac0 0x7fe428070370 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7fe424009870 tx=0x7fe42401c540 comp rx=0 tx=0).ready entity=mon.0 client_cookie=99f1d1e33e51031 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:18:08.514 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:08.510+0000 7fe42cd2e640 1 -- 172.21.15.143:0/3015987252 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe4240ab650 con 0x7fe428143ac0 2023-11-26T07:18:08.514 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:08.510+0000 7fe42cd2e640 1 -- 172.21.15.143:0/3015987252 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe4240ab7b0 con 0x7fe428143ac0 2023-11-26T07:18:08.514 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:08.510+0000 7fe42cd2e640 1 -- 172.21.15.143:0/3015987252 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe4240abb40 con 0x7fe428143ac0 2023-11-26T07:18:08.515 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:08.510+0000 7fe42ff94640 1 -- 172.21.15.143:0/3015987252 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fe428143ac0 msgr2=0x7fe428070370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:18:08.515 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:08.510+0000 7fe42ff94640 1 --2- 172.21.15.143:0/3015987252 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fe428143ac0 0x7fe428070370 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7fe424009870 tx=0x7fe42401c540 comp rx=0 tx=0).stop 2023-11-26T07:18:08.515 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:08.510+0000 7fe42ff94640 1 -- 172.21.15.143:0/3015987252 shutdown_connections 2023-11-26T07:18:08.516 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:08.510+0000 7fe42ff94640 1 --2- 172.21.15.143:0/3015987252 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fe4280cf990 0x7fe4280cd1b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:18:08.516 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:08.510+0000 7fe42ff94640 1 --2- 172.21.15.143:0/3015987252 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fe428143ac0 0x7fe428070370 unknown :-1 s=CLOSED pgs=152 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:18:08.516 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:08.510+0000 7fe42ff94640 1 -- 172.21.15.143:0/3015987252 >> 172.21.15.143:0/3015987252 conn(0x7fe42806a890 msgr2=0x7fe42806aca0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:18:08.516 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:08.510+0000 7fe42ff94640 1 -- 172.21.15.143:0/3015987252 shutdown_connections 2023-11-26T07:18:08.516 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:08.510+0000 7fe42ff94640 1 -- 172.21.15.143:0/3015987252 wait complete. 2023-11-26T07:18:08.516 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:08.514+0000 7fe42ff94640 1 Processor -- start 2023-11-26T07:18:08.517 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:08.514+0000 7fe42ff94640 1 -- start start 2023-11-26T07:18:08.519 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:08.514+0000 7fe42ff94640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fe4280708b0 0x7fe428155790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:18:08.519 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:08.514+0000 7fe42ff94640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fe4280cf990 0x7fe428166d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:18:08.519 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:08.514+0000 7fe42ff94640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fe428143ac0 0x7fe42814c4c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:18:08.520 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:08.514+0000 7fe42ff94640 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe428152130 con 0x7fe428143ac0 2023-11-26T07:18:08.520 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:08.514+0000 7fe42ff94640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe428166570 con 0x7fe4280cf990 2023-11-26T07:18:08.520 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:08.514+0000 7fe42ff94640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe4280b12f0 con 0x7fe4280708b0 2023-11-26T07:18:08.520 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:08.514+0000 7fe42e531640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fe428143ac0 0x7fe42814c4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:18:08.520 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:08.514+0000 7fe42e531640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fe428143ac0 0x7fe42814c4c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:48320/0 (socket says 172.21.15.143:48320) 2023-11-26T07:18:08.521 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:08.514+0000 7fe42e531640 1 -- 172.21.15.143:0/4282803129 learned_addr learned my addr 172.21.15.143:0/4282803129 (peer_addr_for_me v2:172.21.15.143:0/0) 2023-11-26T07:18:08.521 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:08.514+0000 7fe42dd30640 1 --2- 172.21.15.143:0/4282803129 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fe4280708b0 0x7fe428155790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:18:08.521 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:08.514+0000 7fe42d52f640 1 --2- 172.21.15.143:0/4282803129 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fe4280cf990 0x7fe428166d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:18:08.521 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:08.514+0000 7fe42e531640 1 -- 172.21.15.143:0/4282803129 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fe4280708b0 msgr2=0x7fe428155790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:18:08.521 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:08.514+0000 7fe42e531640 1 --2- 172.21.15.143:0/4282803129 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fe4280708b0 0x7fe428155790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:18:08.521 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:08.514+0000 7fe42e531640 1 -- 172.21.15.143:0/4282803129 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fe4280cf990 msgr2=0x7fe428166d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:18:08.521 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:08.514+0000 7fe42e531640 1 --2- 172.21.15.143:0/4282803129 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fe4280cf990 0x7fe428166d10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:18:08.521 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:08.514+0000 7fe42e531640 1 -- 172.21.15.143:0/4282803129 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe4240094d0 con 0x7fe428143ac0 2023-11-26T07:18:08.521 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:08.514+0000 7fe42dd30640 1 --2- 172.21.15.143:0/4282803129 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fe4280708b0 0x7fe428155790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-26T07:18:08.521 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:08.514+0000 7fe42e531640 1 --2- 172.21.15.143:0/4282803129 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fe428143ac0 0x7fe42814c4c0 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7fe420007e40 tx=0x7fe42000e4b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:18:08.522 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:08.514+0000 7fe41e7fc640 1 -- 172.21.15.143:0/4282803129 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe420014620 con 0x7fe428143ac0 2023-11-26T07:18:08.522 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:08.514+0000 7fe41e7fc640 1 -- 172.21.15.143:0/4282803129 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe42000ee90 con 0x7fe428143ac0 2023-11-26T07:18:08.522 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:08.514+0000 7fe42ff94640 1 -- 172.21.15.143:0/4282803129 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe4280b1590 con 0x7fe428143ac0 2023-11-26T07:18:08.522 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:08.514+0000 7fe41e7fc640 1 -- 172.21.15.143:0/4282803129 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe4200149e0 con 0x7fe428143ac0 2023-11-26T07:18:08.522 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:08.514+0000 7fe42ff94640 1 -- 172.21.15.143:0/4282803129 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe4281667f0 con 0x7fe428143ac0 2023-11-26T07:18:08.523 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:08.518+0000 7fe41e7fc640 1 -- 172.21.15.143:0/4282803129 <== mon.0 v2:172.21.15.143:3300/0 4 ==== mgrmap(e 5) v1 ==== 85852+0+0 (secure 0 0 0) 0x7fe420014b40 con 0x7fe428143ac0 2023-11-26T07:18:08.523 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:08.518+0000 7fe41e7fc640 1 --2- 172.21.15.143:0/4282803129 >> [v2:172.21.15.143:6832/2150557544,v1:172.21.15.143:6833/2150557544] conn(0x7fe40c066470 0x7fe40c068930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:18:08.524 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:08.518+0000 7fe41e7fc640 1 -- 172.21.15.143:0/4282803129 <== mon.0 v2:172.21.15.143:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fe420084800 con 0x7fe428143ac0 2023-11-26T07:18:08.524 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:08.518+0000 7fe42dd30640 1 --2- 172.21.15.143:0/4282803129 >> [v2:172.21.15.143:6832/2150557544,v1:172.21.15.143:6833/2150557544] conn(0x7fe40c066470 0x7fe40c068930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:18:08.524 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:08.518+0000 7fe433301000 1 Processor -- start 2023-11-26T07:18:08.524 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:08.518+0000 7fe433301000 1 -- start start 2023-11-26T07:18:08.524 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:08.518+0000 7fe42dd30640 1 --2- 172.21.15.143:0/4282803129 >> [v2:172.21.15.143:6832/2150557544,v1:172.21.15.143:6833/2150557544] conn(0x7fe40c066470 0x7fe40c068930 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7fe4240af000 tx=0x7fe4240a7000 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:18:08.527 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:08.518+0000 7fe433301000 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x559069523c90 0x559069524070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:18:08.527 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:08.518+0000 7fe433301000 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x5590695245b0 0x5590694d9de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:18:08.527 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:08.518+0000 7fe433301000 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x5590694da320 0x5590694da700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:18:08.527 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:08.518+0000 7fe433301000 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x559069344130 con 0x5590694da320 2023-11-26T07:18:08.527 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:08.518+0000 7fe433301000 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x5590693319a0 con 0x559069523c90 2023-11-26T07:18:08.528 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:08.518+0000 7fe433301000 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x55906932eed0 con 0x5590695245b0 2023-11-26T07:18:08.528 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:08.518+0000 7fe42e531640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x5590694da320 0x5590694da700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:18:08.528 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:08.518+0000 7fe42e531640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x5590694da320 0x5590694da700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:48334/0 (socket says 172.21.15.143:48334) 2023-11-26T07:18:08.528 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:08.518+0000 7fe42e531640 1 -- 172.21.15.143:0/1014834990 learned_addr learned my addr 172.21.15.143:0/1014834990 (peer_addr_for_me v2:172.21.15.143:0/0) 2023-11-26T07:18:08.528 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:08.518+0000 7fe42d52f640 1 --2- 172.21.15.143:0/1014834990 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x5590695245b0 0x5590694d9de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:18:08.528 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:08.518+0000 7fe42dd30640 1 --2- 172.21.15.143:0/1014834990 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x559069523c90 0x559069524070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:18:08.528 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:08.522+0000 7fe42e531640 1 -- 172.21.15.143:0/1014834990 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x5590695245b0 msgr2=0x5590694d9de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-26T07:18:08.528 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:08.522+0000 7fe42e531640 1 --2- 172.21.15.143:0/1014834990 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x5590695245b0 0x5590694d9de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:18:08.528 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:08.522+0000 7fe42e531640 1 -- 172.21.15.143:0/1014834990 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x559069523c90 msgr2=0x559069524070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-26T07:18:08.529 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:08.522+0000 7fe42e531640 1 --2- 172.21.15.143:0/1014834990 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x559069523c90 0x559069524070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:18:08.529 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:08.522+0000 7fe42e531640 1 -- 172.21.15.143:0/1014834990 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x559069022890 con 0x5590694da320 2023-11-26T07:18:08.529 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:08.522+0000 7fe42dd30640 1 --2- 172.21.15.143:0/1014834990 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x559069523c90 0x559069524070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-26T07:18:08.529 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:08.522+0000 7fe42e531640 1 --2- 172.21.15.143:0/1014834990 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x5590694da320 0x5590694da700 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7fe42000c040 tx=0x7fe420011320 comp rx=0 tx=0).ready entity=mon.0 client_cookie=73ad7cbc36fc42e4 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:18:08.529 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:08.522+0000 7fe40b7fe640 1 -- 172.21.15.143:0/1014834990 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe420059a80 con 0x5590694da320 2023-11-26T07:18:08.529 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:08.522+0000 7fe40b7fe640 1 -- 172.21.15.143:0/1014834990 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe42005dc30 con 0x5590694da320 2023-11-26T07:18:08.529 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:08.522+0000 7fe40b7fe640 1 -- 172.21.15.143:0/1014834990 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe420053a70 con 0x5590694da320 2023-11-26T07:18:08.530 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:08.522+0000 7fe433301000 1 -- 172.21.15.143:0/1014834990 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x5590694da320 msgr2=0x5590694da700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:18:08.530 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:08.522+0000 7fe433301000 1 --2- 172.21.15.143:0/1014834990 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x5590694da320 0x5590694da700 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7fe42000c040 tx=0x7fe420011320 comp rx=0 tx=0).stop 2023-11-26T07:18:08.530 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:08.522+0000 7fe433301000 1 -- 172.21.15.143:0/1014834990 shutdown_connections 2023-11-26T07:18:08.530 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:08.522+0000 7fe433301000 1 --2- 172.21.15.143:0/1014834990 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x5590694da320 0x5590694da700 unknown :-1 s=CLOSED pgs=154 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:18:08.531 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:08.522+0000 7fe433301000 1 --2- 172.21.15.143:0/1014834990 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x5590695245b0 0x5590694d9de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:18:08.532 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:08.522+0000 7fe433301000 1 --2- 172.21.15.143:0/1014834990 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x559069523c90 0x559069524070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:18:08.532 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:08.522+0000 7fe433301000 1 -- 172.21.15.143:0/1014834990 >> 172.21.15.143:0/1014834990 conn(0x5590694d62d0 msgr2=0x5590694d86c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:18:08.532 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:08.522+0000 7fe433301000 1 -- 172.21.15.143:0/1014834990 shutdown_connections 2023-11-26T07:18:08.532 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:08.522+0000 7fe433301000 1 -- 172.21.15.143:0/1014834990 wait complete. 2023-11-26T07:18:08.532 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:08.522+0000 7fe433301000 1 -- start start 2023-11-26T07:18:08.532 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:08.522+0000 7fe433301000 1 Processor -- start 2023-11-26T07:18:08.535 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:08.522+0000 7fe433301000 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x5590694da320 0x5590694ded40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:18:08.536 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:08.522+0000 7fe433301000 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x559069523c90 0x5590694df280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:18:08.536 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:08.522+0000 7fe433301000 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x5590695245b0 0x5590694f2910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:18:08.536 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:08.522+0000 7fe433301000 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x55906932a7c0 con 0x5590695245b0 2023-11-26T07:18:08.536 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:08.522+0000 7fe433301000 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x5590692e2bf0 con 0x559069523c90 2023-11-26T07:18:08.536 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:08.522+0000 7fe433301000 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x5590692e8a90 con 0x5590694da320 2023-11-26T07:18:08.536 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:08.522+0000 7fe433301000 10 client.0 _ll_register_callbacks cb 0x5590694c53c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-11-26T07:18:08.536 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:08.522+0000 7fe42e531640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x5590695245b0 0x5590694f2910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:18:08.536 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:08.522+0000 7fe42e531640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x5590695245b0 0x5590694f2910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:48336/0 (socket says 172.21.15.143:48336) 2023-11-26T07:18:08.536 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:08.522+0000 7fe42e531640 1 -- 172.21.15.143:0/605772457 learned_addr learned my addr 172.21.15.143:0/605772457 (peer_addr_for_me v2:172.21.15.143:0/0) 2023-11-26T07:18:08.536 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:08.522+0000 7fe42dd30640 1 --2- 172.21.15.143:0/605772457 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x5590694da320 0x5590694ded40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:18:08.536 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:08.522+0000 7fe42d52f640 1 --2- 172.21.15.143:0/605772457 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x559069523c90 0x5590694df280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:18:08.537 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:08.526+0000 7fe42e531640 1 -- 172.21.15.143:0/605772457 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x5590694da320 msgr2=0x5590694ded40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-26T07:18:08.537 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:08.526+0000 7fe42e531640 1 --2- 172.21.15.143:0/605772457 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x5590694da320 0x5590694ded40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:18:08.537 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:08.526+0000 7fe42e531640 1 -- 172.21.15.143:0/605772457 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x559069523c90 msgr2=0x5590694df280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-26T07:18:08.537 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:08.526+0000 7fe42e531640 1 --2- 172.21.15.143:0/605772457 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x559069523c90 0x5590694df280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:18:08.537 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:08.526+0000 7fe42e531640 1 -- 172.21.15.143:0/605772457 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe4281667f0 con 0x5590695245b0 2023-11-26T07:18:08.537 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:08.526+0000 7fe42d52f640 1 --2- 172.21.15.143:0/605772457 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x559069523c90 0x5590694df280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-26T07:18:08.537 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:08.526+0000 7fe42e531640 1 --2- 172.21.15.143:0/605772457 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x5590695245b0 0x5590694f2910 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7fe420011410 tx=0x7fe420016000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2ee709ba48c3f2d2 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:18:08.537 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:08.526+0000 7fe408ff9640 10 client.0 ms_handle_connect on v2:172.21.15.143:3300/0 2023-11-26T07:18:08.537 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:08.526+0000 7fe408ff9640 1 -- 172.21.15.143:0/605772457 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe42005dc30 con 0x5590695245b0 2023-11-26T07:18:08.537 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:08.526+0000 7fe408ff9640 1 -- 172.21.15.143:0/605772457 <== mon.0 v2:172.21.15.143:3300/0 2 ==== osd_map(20..20 src has 1..20) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fe420053cd0 con 0x5590695245b0 2023-11-26T07:18:08.537 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:08.526+0000 7fe433301000 1 -- 172.21.15.143:0/605772457 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x5590692ec470 con 0x5590695245b0 2023-11-26T07:18:08.537 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:08.526+0000 7fe408ff9640 1 -- 172.21.15.143:0/605772457 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_get_version_reply(handle=1 version=8) v2 ==== 24+0+0 (secure 0 0 0) 0x7fe420052040 con 0x5590695245b0 2023-11-26T07:18:08.538 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:08.526+0000 7fe433301000 10 client.4617 fetch_fsmap learned FSMap version 8 2023-11-26T07:18:08.538 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:08.526+0000 7fe433301000 1 -- 172.21.15.143:0/605772457 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,fsmap=8,monmap=2+}) v3 -- 0x5590692eb190 con 0x5590695245b0 2023-11-26T07:18:08.538 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:08.526+0000 7fe408ff9640 1 -- 172.21.15.143:0/605772457 <== mon.0 v2:172.21.15.143:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe42007b030 con 0x5590695245b0 2023-11-26T07:18:08.538 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:08.526+0000 7fe408ff9640 1 -- 172.21.15.143:0/605772457 <== mon.0 v2:172.21.15.143:3300/0 5 ==== fsmap(e 8) v1 ==== 3294+0+0 (secure 0 0 0) 0x7fe420005d20 con 0x5590695245b0 2023-11-26T07:18:08.538 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:08.526+0000 7fe433301000 10 client.4617 fetch_fsmap finished waiting for FSMap version 8 2023-11-26T07:18:08.538 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:08.526+0000 7fe433301000 10 client.4617 resolve_mds: resolved name 'b' to daemon mds.b 2023-11-26T07:18:08.538 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:08.526+0000 7fe433301000 20 client.4617 populate_metadata read hostname 'smithi143' 2023-11-26T07:18:08.538 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:08.526+0000 7fe433301000 1 --2- 172.21.15.143:0/605772457 >> [v2:172.21.15.188:6833/994617268,v1:172.21.15.188:6835/994617268] conn(0x5590694f3320 0x559069535f50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:18:08.538 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:08.526+0000 7fe433301000 4 client.4617 mds_command: new command op to 4285 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2023-11-26T07:18:08.538 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:08.526+0000 7fe433301000 1 -- 172.21.15.143:0/605772457 --> [v2:172.21.15.188:6833/994617268,v1:172.21.15.188:6835/994617268] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x5590692efb50 con 0x5590694f3320 2023-11-26T07:18:08.538 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:08.526+0000 7fe42dd30640 1 --2- 172.21.15.143:0/605772457 >> [v2:172.21.15.188:6833/994617268,v1:172.21.15.188:6835/994617268] conn(0x5590694f3320 0x559069535f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:18:08.538 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:08.526+0000 7fe42dd30640 1 --2- 172.21.15.143:0/605772457 >> [v2:172.21.15.188:6833/994617268,v1:172.21.15.188:6835/994617268] conn(0x5590694f3320 0x559069535f50 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=49ed0b17ab338db8 server_cookie=40408f9a9c26daaa in_seq=0 out_seq=0 2023-11-26T07:18:08.539 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:08.526+0000 7fe408ff9640 10 client.4617 ms_handle_connect on v2:172.21.15.188:6833/994617268 2023-11-26T07:18:08.539 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:08.526+0000 7fe408ff9640 1 -- 172.21.15.143:0/605772457 <== mds.0 v2:172.21.15.188:6833/994617268 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x5590692efb50 con 0x5590694f3320 2023-11-26T07:18:08.539 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:08.526+0000 7fe408ff9640 10 client.4617 handle_command_reply: tid=0 2023-11-26T07:18:08.539 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:08.526+0000 7fe433301000 1 client.4617 shutdown 2023-11-26T07:18:08.539 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:08.530+0000 7fe433301000 1 -- 172.21.15.143:0/605772457 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x5590695245b0 msgr2=0x5590694f2910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:18:08.539 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:08.530+0000 7fe433301000 1 --2- 172.21.15.143:0/605772457 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x5590695245b0 0x5590694f2910 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7fe420011410 tx=0x7fe420016000 comp rx=0 tx=0).stop 2023-11-26T07:18:08.539 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:08.530+0000 7fe433301000 1 -- 172.21.15.143:0/605772457 shutdown_connections 2023-11-26T07:18:08.539 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:08.530+0000 7fe433301000 1 --2- 172.21.15.143:0/605772457 >> [v2:172.21.15.188:6833/994617268,v1:172.21.15.188:6835/994617268] conn(0x5590694f3320 0x559069535f50 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:18:08.539 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:08.530+0000 7fe433301000 1 --2- 172.21.15.143:0/605772457 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x5590695245b0 0x5590694f2910 unknown :-1 s=CLOSED pgs=155 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:18:08.539 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:08.530+0000 7fe433301000 1 --2- 172.21.15.143:0/605772457 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x559069523c90 0x5590694df280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:18:08.539 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:08.530+0000 7fe433301000 1 --2- 172.21.15.143:0/605772457 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x5590694da320 0x5590694ded40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:18:08.539 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:08.530+0000 7fe408ff9640 0 client.4617 ms_handle_reset on v2:172.21.15.188:6833/994617268 2023-11-26T07:18:08.540 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:08.530+0000 7fe433301000 1 -- 172.21.15.143:0/605772457 >> 172.21.15.143:0/605772457 conn(0x5590694d62d0 msgr2=0x5590694d86c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:18:08.540 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:08.530+0000 7fe433301000 1 -- 172.21.15.143:0/605772457 shutdown_connections 2023-11-26T07:18:08.540 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:08.530+0000 7fe433301000 1 -- 172.21.15.143:0/605772457 wait complete. 2023-11-26T07:18:08.540 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:08.530+0000 7fe433301000 20 client.4617 trim_cache size 0 max 16384 2023-11-26T07:18:08.548 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:08.542+0000 7fe433301000 1 Processor -- start 2023-11-26T07:18:08.548 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:08.542+0000 7fe433301000 1 -- start start 2023-11-26T07:18:08.551 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:08.546+0000 7fe433301000 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5590694da320 0x559069522c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:18:08.551 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:08.546+0000 7fe433301000 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x559069523c90 0x559069521b30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:18:08.551 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:08.546+0000 7fe433301000 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x5590695245b0 0x559069522070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:18:08.551 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:08.546+0000 7fe433301000 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x559068fe3940 con 0x5590695245b0 2023-11-26T07:18:08.551 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:08.546+0000 7fe433301000 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x559069324ac0 con 0x5590694da320 2023-11-26T07:18:08.551 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:08.546+0000 7fe433301000 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x55906932d320 con 0x559069523c90 2023-11-26T07:18:08.552 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:08.546+0000 7fe42e531640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x5590695245b0 0x559069522070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:18:08.552 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:08.546+0000 7fe42e531640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x5590695245b0 0x559069522070 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:48344/0 (socket says 172.21.15.143:48344) 2023-11-26T07:18:08.552 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:08.546+0000 7fe42e531640 1 -- 172.21.15.143:0/556465967 learned_addr learned my addr 172.21.15.143:0/556465967 (peer_addr_for_me v2:172.21.15.143:0/0) 2023-11-26T07:18:08.552 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:08.546+0000 7fe42d52f640 1 --2- 172.21.15.143:0/556465967 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x559069523c90 0x559069521b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:18:08.552 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:08.546+0000 7fe42dd30640 1 --2- 172.21.15.143:0/556465967 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5590694da320 0x559069522c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:18:08.552 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:08.546+0000 7fe42e531640 1 -- 172.21.15.143:0/556465967 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x559069523c90 msgr2=0x559069521b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-26T07:18:08.552 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:08.546+0000 7fe42e531640 1 --2- 172.21.15.143:0/556465967 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x559069523c90 0x559069521b30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:18:08.552 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:08.546+0000 7fe42e531640 1 -- 172.21.15.143:0/556465967 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5590694da320 msgr2=0x559069522c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-26T07:18:08.552 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:08.546+0000 7fe42e531640 1 --2- 172.21.15.143:0/556465967 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5590694da320 0x559069522c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:18:08.552 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:08.546+0000 7fe42e531640 1 -- 172.21.15.143:0/556465967 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5590692d1490 con 0x5590695245b0 2023-11-26T07:18:08.552 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:08.546+0000 7fe42e531640 1 --2- 172.21.15.143:0/556465967 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x5590695245b0 0x559069522070 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7fe420007e10 tx=0x7fe42001f000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7a23e7102da001cd server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:18:08.553 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:08.546+0000 7fe40bfff640 1 -- 172.21.15.143:0/556465967 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe420052040 con 0x5590695245b0 2023-11-26T07:18:08.553 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:08.546+0000 7fe40bfff640 1 -- 172.21.15.143:0/556465967 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe42007b070 con 0x5590695245b0 2023-11-26T07:18:08.553 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:08.546+0000 7fe40bfff640 1 -- 172.21.15.143:0/556465967 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe42005dc30 con 0x5590695245b0 2023-11-26T07:18:08.553 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:08.546+0000 7fe433301000 1 -- 172.21.15.143:0/556465967 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x5590695245b0 msgr2=0x559069522070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:18:08.553 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:08.546+0000 7fe433301000 1 --2- 172.21.15.143:0/556465967 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x5590695245b0 0x559069522070 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7fe420007e10 tx=0x7fe42001f000 comp rx=0 tx=0).stop 2023-11-26T07:18:08.553 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:08.546+0000 7fe433301000 1 -- 172.21.15.143:0/556465967 shutdown_connections 2023-11-26T07:18:08.553 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:08.546+0000 7fe433301000 1 --2- 172.21.15.143:0/556465967 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x5590695245b0 0x559069522070 unknown :-1 s=CLOSED pgs=156 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:18:08.553 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:08.546+0000 7fe433301000 1 --2- 172.21.15.143:0/556465967 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x559069523c90 0x559069521b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:18:08.553 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:08.546+0000 7fe433301000 1 --2- 172.21.15.143:0/556465967 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5590694da320 0x559069522c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:18:08.553 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:08.546+0000 7fe433301000 1 -- 172.21.15.143:0/556465967 >> 172.21.15.143:0/556465967 conn(0x5590694d62d0 msgr2=0x559069531280 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:18:08.553 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:08.546+0000 7fe433301000 1 -- 172.21.15.143:0/556465967 shutdown_connections 2023-11-26T07:18:08.554 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:08.546+0000 7fe433301000 1 -- 172.21.15.143:0/556465967 wait complete. 2023-11-26T07:18:08.558 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:08.546+0000 7fe433301000 1 -- start start 2023-11-26T07:18:08.558 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:08.546+0000 7fe433301000 1 Processor -- start 2023-11-26T07:18:08.558 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:08.550+0000 7fe433301000 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x5590694da320 0x5590694dba90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:18:08.558 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:08.550+0000 7fe433301000 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x559069523c90 0x559069536fa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:18:08.558 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:08.550+0000 7fe433301000 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5590695245b0 0x5590695374e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:18:08.558 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:08.550+0000 7fe433301000 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x5590692c58a0 con 0x5590694da320 2023-11-26T07:18:08.558 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:08.550+0000 7fe433301000 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x5590694dffb0 con 0x5590695245b0 2023-11-26T07:18:08.558 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:08.550+0000 7fe433301000 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x5590692e3d60 con 0x559069523c90 2023-11-26T07:18:08.558 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:08.550+0000 7fe433301000 10 client.0 _ll_register_callbacks cb 0x559069378c00 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-11-26T07:18:08.559 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:08.550+0000 7fe42dd30640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x5590694da320 0x5590694dba90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:18:08.559 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:08.550+0000 7fe42dd30640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x5590694da320 0x5590694dba90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:48358/0 (socket says 172.21.15.143:48358) 2023-11-26T07:18:08.559 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:08.550+0000 7fe42dd30640 1 -- 172.21.15.143:0/3762335080 learned_addr learned my addr 172.21.15.143:0/3762335080 (peer_addr_for_me v2:172.21.15.143:0/0) 2023-11-26T07:18:08.559 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:08.550+0000 7fe42e531640 1 --2- 172.21.15.143:0/3762335080 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5590695245b0 0x5590695374e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:18:08.559 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:08.550+0000 7fe42d52f640 1 --2- 172.21.15.143:0/3762335080 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x559069523c90 0x559069536fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:18:08.559 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:08.550+0000 7fe42dd30640 1 -- 172.21.15.143:0/3762335080 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x559069523c90 msgr2=0x559069536fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-26T07:18:08.559 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:08.550+0000 7fe42dd30640 1 --2- 172.21.15.143:0/3762335080 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x559069523c90 0x559069536fa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:18:08.559 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:08.550+0000 7fe42dd30640 1 -- 172.21.15.143:0/3762335080 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5590695245b0 msgr2=0x5590695374e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-26T07:18:08.559 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:08.550+0000 7fe42dd30640 1 --2- 172.21.15.143:0/3762335080 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5590695245b0 0x5590695374e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:18:08.559 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:08.550+0000 7fe42dd30640 1 -- 172.21.15.143:0/3762335080 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x5590692eb190 con 0x5590694da320 2023-11-26T07:18:08.559 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:08.550+0000 7fe42e531640 1 --2- 172.21.15.143:0/3762335080 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5590695245b0 0x5590695374e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-26T07:18:08.559 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:08.550+0000 7fe42dd30640 1 --2- 172.21.15.143:0/3762335080 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x5590694da320 0x5590694dba90 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7fe4240ad040 tx=0x7fe42401ad80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=740ea3a889fc477f server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:18:08.560 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:08.550+0000 7fe408ff9640 10 client.0 ms_handle_connect on v2:172.21.15.143:3300/0 2023-11-26T07:18:08.560 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:08.550+0000 7fe408ff9640 1 -- 172.21.15.143:0/3762335080 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe4240c5040 con 0x5590694da320 2023-11-26T07:18:08.560 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:08.550+0000 7fe408ff9640 1 -- 172.21.15.143:0/3762335080 <== mon.0 v2:172.21.15.143:3300/0 2 ==== osd_map(20..20 src has 1..20) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fe4240cc5c0 con 0x5590694da320 2023-11-26T07:18:08.560 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:08.550+0000 7fe433301000 1 -- 172.21.15.143:0/3762335080 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x559069329110 con 0x5590694da320 2023-11-26T07:18:08.560 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:08.550+0000 7fe408ff9640 1 -- 172.21.15.143:0/3762335080 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_get_version_reply(handle=1 version=8) v2 ==== 24+0+0 (secure 0 0 0) 0x7fe4240abe90 con 0x5590694da320 2023-11-26T07:18:08.560 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:08.550+0000 7fe433301000 10 client.4623 fetch_fsmap learned FSMap version 8 2023-11-26T07:18:08.560 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:08.550+0000 7fe433301000 1 -- 172.21.15.143:0/3762335080 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,fsmap=8,monmap=2+}) v3 -- 0x559069022cc0 con 0x5590694da320 2023-11-26T07:18:08.560 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:08.550+0000 7fe408ff9640 1 -- 172.21.15.143:0/3762335080 <== mon.0 v2:172.21.15.143:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x559069329110 con 0x5590694da320 2023-11-26T07:18:08.560 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:08.550+0000 7fe408ff9640 1 -- 172.21.15.143:0/3762335080 <== mon.0 v2:172.21.15.143:3300/0 5 ==== fsmap(e 8) v1 ==== 3294+0+0 (secure 0 0 0) 0x7fe4240cc780 con 0x5590694da320 2023-11-26T07:18:08.560 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:08.550+0000 7fe433301000 10 client.4623 fetch_fsmap finished waiting for FSMap version 8 2023-11-26T07:18:08.560 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:08.550+0000 7fe433301000 10 client.4623 resolve_mds: resolved name 'b' to daemon mds.b 2023-11-26T07:18:08.560 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:08.550+0000 7fe433301000 20 client.4623 populate_metadata read hostname 'smithi143' 2023-11-26T07:18:08.561 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:08.550+0000 7fe433301000 1 --2- 172.21.15.143:0/3762335080 >> [v2:172.21.15.188:6833/994617268,v1:172.21.15.188:6835/994617268] conn(0x55906958c820 0x55906958cc00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:18:08.561 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:08.550+0000 7fe433301000 4 client.4623 mds_command: new command op to 4285 tid=0 multi_id=0 [{"prefix": "status"}] 2023-11-26T07:18:08.561 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:08.550+0000 7fe433301000 1 -- 172.21.15.143:0/3762335080 --> [v2:172.21.15.188:6833/994617268,v1:172.21.15.188:6835/994617268] -- command(tid 0: {"prefix": "status"}) v1 -- 0x5590693589a0 con 0x55906958c820 2023-11-26T07:18:08.561 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:08.550+0000 7fe42d52f640 1 --2- 172.21.15.143:0/3762335080 >> [v2:172.21.15.188:6833/994617268,v1:172.21.15.188:6835/994617268] conn(0x55906958c820 0x55906958cc00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:18:08.561 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:08.550+0000 7fe42d52f640 1 --2- 172.21.15.143:0/3762335080 >> [v2:172.21.15.188:6833/994617268,v1:172.21.15.188:6835/994617268] conn(0x55906958c820 0x55906958cc00 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=2932197f0b880944 server_cookie=f8a18e118a69d6ba in_seq=0 out_seq=0 2023-11-26T07:18:08.561 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:08.550+0000 7fe408ff9640 10 client.4623 ms_handle_connect on v2:172.21.15.188:6833/994617268 2023-11-26T07:18:08.561 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:08.550+0000 7fe408ff9640 1 -- 172.21.15.143:0/3762335080 <== mds.0 v2:172.21.15.188:6833/994617268 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+324 (crc 0 0 0) 0x5590693589a0 con 0x55906958c820 2023-11-26T07:18:08.561 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:08.550+0000 7fe408ff9640 10 client.4623 handle_command_reply: tid=0 2023-11-26T07:18:08.561 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:08.550+0000 7fe433301000 1 client.4623 shutdown 2023-11-26T07:18:08.561 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:08.550+0000 7fe433301000 1 -- 172.21.15.143:0/3762335080 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x5590694da320 msgr2=0x5590694dba90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:18:08.561 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:08.550+0000 7fe433301000 1 --2- 172.21.15.143:0/3762335080 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x5590694da320 0x5590694dba90 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7fe4240ad040 tx=0x7fe42401ad80 comp rx=0 tx=0).stop 2023-11-26T07:18:08.561 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:08.550+0000 7fe433301000 1 -- 172.21.15.143:0/3762335080 shutdown_connections 2023-11-26T07:18:08.561 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:08.550+0000 7fe433301000 1 --2- 172.21.15.143:0/3762335080 >> [v2:172.21.15.188:6833/994617268,v1:172.21.15.188:6835/994617268] conn(0x55906958c820 0x55906958cc00 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:18:08.562 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:08.550+0000 7fe433301000 1 --2- 172.21.15.143:0/3762335080 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5590695245b0 0x5590695374e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:18:08.562 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:08.550+0000 7fe433301000 1 --2- 172.21.15.143:0/3762335080 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x559069523c90 0x559069536fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:18:08.562 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:08.550+0000 7fe408ff9640 0 client.4623 ms_handle_reset on v2:172.21.15.188:6833/994617268 2023-11-26T07:18:08.562 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:08.550+0000 7fe433301000 1 --2- 172.21.15.143:0/3762335080 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x5590694da320 0x5590694dba90 unknown :-1 s=CLOSED pgs=157 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:18:08.562 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:08.550+0000 7fe433301000 1 -- 172.21.15.143:0/3762335080 >> 172.21.15.143:0/3762335080 conn(0x5590694d62d0 msgr2=0x5590694d86c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:18:08.562 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:08.550+0000 7fe433301000 1 -- 172.21.15.143:0/3762335080 shutdown_connections 2023-11-26T07:18:08.562 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:08.550+0000 7fe433301000 1 -- 172.21.15.143:0/3762335080 wait complete. 2023-11-26T07:18:08.562 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:08.554+0000 7fe433301000 20 client.4623 trim_cache size 0 max 16384 2023-11-26T07:18:08.562 INFO:teuthology.orchestra.run.smithi143.stdout:{ 2023-11-26T07:18:08.563 INFO:teuthology.orchestra.run.smithi143.stdout: "cluster_fsid": "f5033060-2469-4a07-95fe-2b76f5404bc1", 2023-11-26T07:18:08.563 INFO:teuthology.orchestra.run.smithi143.stdout: "whoami": 0, 2023-11-26T07:18:08.563 INFO:teuthology.orchestra.run.smithi143.stdout: "id": 4285, 2023-11-26T07:18:08.563 INFO:teuthology.orchestra.run.smithi143.stdout: "want_state": "up:active", 2023-11-26T07:18:08.563 INFO:teuthology.orchestra.run.smithi143.stdout: "state": "up:active", 2023-11-26T07:18:08.563 INFO:teuthology.orchestra.run.smithi143.stdout: "fs_name": "cephfs", 2023-11-26T07:18:08.563 INFO:teuthology.orchestra.run.smithi143.stdout: "rank_uptime": 14.044982123, 2023-11-26T07:18:08.563 INFO:teuthology.orchestra.run.smithi143.stdout: "mdsmap_epoch": 8, 2023-11-26T07:18:08.563 INFO:teuthology.orchestra.run.smithi143.stdout: "osdmap_epoch": 20, 2023-11-26T07:18:08.563 INFO:teuthology.orchestra.run.smithi143.stdout: "osdmap_epoch_barrier": 20, 2023-11-26T07:18:08.563 INFO:teuthology.orchestra.run.smithi143.stdout: "uptime": 21.308718805000002 2023-11-26T07:18:08.563 INFO:teuthology.orchestra.run.smithi143.stdout:} 2023-11-26T07:18:08.564 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:08.554+0000 7fe42ff94640 1 -- 172.21.15.143:0/4282803129 >> [v2:172.21.15.143:6832/2150557544,v1:172.21.15.143:6833/2150557544] conn(0x7fe40c066470 msgr2=0x7fe40c068930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:18:08.564 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:08.554+0000 7fe42ff94640 1 --2- 172.21.15.143:0/4282803129 >> [v2:172.21.15.143:6832/2150557544,v1:172.21.15.143:6833/2150557544] conn(0x7fe40c066470 0x7fe40c068930 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7fe4240af000 tx=0x7fe4240a7000 comp rx=0 tx=0).stop 2023-11-26T07:18:08.564 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:08.554+0000 7fe42ff94640 1 -- 172.21.15.143:0/4282803129 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fe428143ac0 msgr2=0x7fe42814c4c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:18:08.564 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:08.554+0000 7fe42ff94640 1 --2- 172.21.15.143:0/4282803129 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fe428143ac0 0x7fe42814c4c0 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7fe420007e40 tx=0x7fe42000e4b0 comp rx=0 tx=0).stop 2023-11-26T07:18:08.564 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:08.554+0000 7fe42ff94640 1 -- 172.21.15.143:0/4282803129 shutdown_connections 2023-11-26T07:18:08.565 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:08.554+0000 7fe42ff94640 1 --2- 172.21.15.143:0/4282803129 >> [v2:172.21.15.143:6832/2150557544,v1:172.21.15.143:6833/2150557544] conn(0x7fe40c066470 0x7fe40c068930 unknown :-1 s=CLOSED pgs=116 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:18:08.565 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:08.554+0000 7fe42ff94640 1 --2- 172.21.15.143:0/4282803129 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fe428143ac0 0x7fe42814c4c0 unknown :-1 s=CLOSED pgs=153 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:18:08.565 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:08.554+0000 7fe42ff94640 1 --2- 172.21.15.143:0/4282803129 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fe4280cf990 0x7fe428166d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:18:08.565 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:08.554+0000 7fe42ff94640 1 --2- 172.21.15.143:0/4282803129 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fe4280708b0 0x7fe428155790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:18:08.565 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:08.554+0000 7fe42ff94640 1 -- 172.21.15.143:0/4282803129 >> 172.21.15.143:0/4282803129 conn(0x7fe42806a890 msgr2=0x7fe42815bc70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:18:08.565 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:08.554+0000 7fe42ff94640 1 -- 172.21.15.143:0/4282803129 shutdown_connections 2023-11-26T07:18:08.565 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:08.554+0000 7fe42ff94640 1 -- 172.21.15.143:0/4282803129 wait complete. 2023-11-26T07:18:08.576 INFO:teuthology.run_tasks:Running task ceph-fuse... 2023-11-26T07:18:08.604 INFO:tasks.ceph_fuse:Running ceph_fuse task... 2023-11-26T07:18:08.604 INFO:tasks.ceph_fuse:config is {'client.0': {}, 'client.2': {}, 'client.1': {}, 'client.3': {}} 2023-11-26T07:18:08.604 INFO:tasks.ceph_fuse:client.0 config is {} 2023-11-26T07:18:08.604 INFO:tasks.cephfs.mount:cephfs_mntpt = None 2023-11-26T07:18:08.604 INFO:tasks.cephfs.mount:hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2023-11-26T07:18:08.605 INFO:tasks.ceph_fuse:client.2 config is {} 2023-11-26T07:18:08.605 INFO:tasks.cephfs.mount:cephfs_mntpt = None 2023-11-26T07:18:08.605 INFO:tasks.cephfs.mount:hostfs_mntpt = /home/ubuntu/cephtest/mnt.2 2023-11-26T07:18:08.605 INFO:tasks.ceph_fuse:client.1 config is {} 2023-11-26T07:18:08.605 INFO:tasks.cephfs.mount:cephfs_mntpt = None 2023-11-26T07:18:08.605 INFO:tasks.cephfs.mount:hostfs_mntpt = /home/ubuntu/cephtest/mnt.1 2023-11-26T07:18:08.605 INFO:tasks.ceph_fuse:client.3 config is {} 2023-11-26T07:18:08.605 INFO:tasks.cephfs.mount:cephfs_mntpt = None 2023-11-26T07:18:08.605 INFO:tasks.cephfs.mount:hostfs_mntpt = /home/ubuntu/cephtest/mnt.3 2023-11-26T07:18:08.605 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-26T07:18:08.606 DEBUG:teuthology.orchestra.run.smithi188:> ip netns list 2023-11-26T07:18:08.617 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-26T07:18:08.617 DEBUG:teuthology.orchestra.run.smithi188:> sudo ip link delete ceph-brx 2023-11-26T07:18:08.682 INFO:teuthology.orchestra.run.smithi188.stderr:Cannot find device "ceph-brx" 2023-11-26T07:18:08.683 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-11-26T07:18:08.684 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-26T07:18:08.684 DEBUG:teuthology.orchestra.run.smithi143:> ip netns list 2023-11-26T07:18:08.694 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-26T07:18:08.694 DEBUG:teuthology.orchestra.run.smithi143:> sudo ip link delete ceph-brx 2023-11-26T07:18:08.763 INFO:teuthology.orchestra.run.smithi143.stderr:Cannot find device "ceph-brx" 2023-11-26T07:18:08.765 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-11-26T07:18:08.765 INFO:tasks.ceph_fuse:Mounting ceph-fuse clients... 2023-11-26T07:18:08.765 DEBUG:teuthology.orchestra.run.smithi143:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2023-11-26T07:18:08.947 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:08.942+0000 7f70d5632640 1 Processor -- start 2023-11-26T07:18:08.947 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:08.942+0000 7f70d5632640 1 -- start start 2023-11-26T07:18:08.948 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:08.942+0000 7f70d5632640 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7f70d015d580 0x7f70d00d3920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:18:08.948 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:08.942+0000 7f70d5632640 1 -- --> v1:172.21.15.143:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f70d00d0bc0 con 0x7f70d00d3e60 2023-11-26T07:18:08.948 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:08.942+0000 7f70d5632640 1 -- --> v1:172.21.15.188:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f70d00d0320 con 0x7f70d00d1f00 2023-11-26T07:18:08.948 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:08.942+0000 7f70d5632640 1 -- --> v2:172.21.15.188:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f70d00d0600 con 0x7f70d015d580 2023-11-26T07:18:08.949 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:08.942+0000 7f70c67fc640 1 --1- >> v1:172.21.15.143:6789/0 conn(0x7f70d00d3e60 0x7f70d00d17f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.143:6789/0 says I am v1:172.21.15.143:52240/0 (socket says 172.21.15.143:52240) 2023-11-26T07:18:08.949 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:08.942+0000 7f70c67fc640 1 -- 172.21.15.143:0/2800306999 learned_addr learned my addr 172.21.15.143:0/2800306999 (peer_addr_for_me v1:172.21.15.143:0/0) 2023-11-26T07:18:08.949 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:08.942+0000 7f70ceffd640 1 --2- 172.21.15.143:0/2800306999 >> v2:172.21.15.188:3300/0 conn(0x7f70d015d580 0x7f70d00d3920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:18:08.951 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:08.942+0000 7f70ce7fc640 1 -- 172.21.15.143:0/2800306999 <== mon.0 v1:172.21.15.143:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3212338253 0 0) 0x7f70d00d0bc0 con 0x7f70d00d3e60 2023-11-26T07:18:08.952 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:08.942+0000 7f70ce7fc640 1 -- 172.21.15.143:0/2800306999 --> v1:172.21.15.143:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f70bc003530 con 0x7f70d00d3e60 2023-11-26T07:18:08.952 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:08.946+0000 7f70ce7fc640 1 -- 172.21.15.143:0/2800306999 <== mon.0 v1:172.21.15.143:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3510647415 0 0) 0x7f70bc003530 con 0x7f70d00d3e60 2023-11-26T07:18:08.952 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:08.946+0000 7f70ce7fc640 1 -- 172.21.15.143:0/2800306999 >> v2:172.21.15.188:3300/0 conn(0x7f70d015d580 msgr2=0x7f70d00d3920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-26T07:18:08.952 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:08.946+0000 7f70ce7fc640 1 --2- 172.21.15.143:0/2800306999 >> v2:172.21.15.188:3300/0 conn(0x7f70d015d580 0x7f70d00d3920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:18:08.952 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:08.946+0000 7f70ce7fc640 1 -- 172.21.15.143:0/2800306999 >> v1:172.21.15.188:6789/0 conn(0x7f70d00d1f00 legacy=0x7f70d00b0490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:18:08.952 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:08.946+0000 7f70ce7fc640 1 -- 172.21.15.143:0/2800306999 --> v1:172.21.15.143:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f70d00d08e0 con 0x7f70d00d3e60 2023-11-26T07:18:08.952 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:08.946+0000 7f70ce7fc640 1 -- 172.21.15.143:0/2800306999 <== mon.0 v1:172.21.15.143:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2137882523 0 0) 0x7f70b8001680 con 0x7f70d00d3e60 2023-11-26T07:18:08.952 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:08.946+0000 7f70ceffd640 1 --2- 172.21.15.143:0/2800306999 >> v2:172.21.15.188:3300/0 conn(0x7f70d015d580 0x7f70d00d3920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-26T07:18:08.952 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:08.946+0000 7f70ce7fc640 1 -- 172.21.15.143:0/2800306999 >> v1:172.21.15.143:6789/0 conn(0x7f70d00d3e60 legacy=0x7f70d00d17f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:18:08.952 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:08.946+0000 7f70ce7fc640 1 --2- 172.21.15.143:0/2800306999 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f70bc003980 0x7f70bc003da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:18:08.953 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:08.946+0000 7f70ce7fc640 1 --2- 172.21.15.143:0/2800306999 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f70bc0047d0 0x7f70bc004c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:18:08.953 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:08.946+0000 7f70ce7fc640 1 --2- 172.21.15.143:0/2800306999 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f70bc005190 0x7f70bc00b990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:18:08.953 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:08.946+0000 7f70ce7fc640 1 -- 172.21.15.143:0/2800306999 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f70d00d0600 con 0x7f70bc0047d0 2023-11-26T07:18:08.953 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:08.946+0000 7f70ce7fc640 1 -- 172.21.15.143:0/2800306999 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f70bc00c030 con 0x7f70bc003980 2023-11-26T07:18:08.953 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:08.946+0000 7f70ce7fc640 1 -- 172.21.15.143:0/2800306999 --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f70bc00c1f0 con 0x7f70bc005190 2023-11-26T07:18:08.953 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:08.946+0000 7f70c67fc640 1 --2- 172.21.15.143:0/2800306999 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f70bc0047d0 0x7f70bc004c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:18:08.953 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:08.946+0000 7f70ceffd640 1 --2- 172.21.15.143:0/2800306999 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f70bc003980 0x7f70bc003da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:18:08.953 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:08.946+0000 7f70cf7fe640 1 --2- 172.21.15.143:0/2800306999 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f70bc005190 0x7f70bc00b990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:18:08.953 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:08.946+0000 7f70c67fc640 1 -- 172.21.15.143:0/2800306999 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f70bc005190 msgr2=0x7f70bc00b990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-26T07:18:08.953 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:08.946+0000 7f70c67fc640 1 --2- 172.21.15.143:0/2800306999 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f70bc005190 0x7f70bc00b990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:18:08.953 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:08.946+0000 7f70c67fc640 1 -- 172.21.15.143:0/2800306999 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f70bc003980 msgr2=0x7f70bc003da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-26T07:18:08.953 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:08.946+0000 7f70c67fc640 1 --2- 172.21.15.143:0/2800306999 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f70bc003980 0x7f70bc003da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:18:08.954 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:08.946+0000 7f70c67fc640 1 -- 172.21.15.143:0/2800306999 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f70bc003530 con 0x7f70bc0047d0 2023-11-26T07:18:08.954 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:08.946+0000 7f70cf7fe640 1 -- 172.21.15.143:0/2800306999 reap_dead start 2023-11-26T07:18:08.955 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:08.950+0000 7f70c67fc640 1 --2- 172.21.15.143:0/2800306999 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f70bc0047d0 0x7f70bc004c50 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7f70b800c240 tx=0x7f70b801f3b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f1e32d062bb8dffc server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:18:08.955 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:08.950+0000 7f70ce7fc640 1 -- 172.21.15.143:0/2800306999 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f70b8005ae0 con 0x7f70bc0047d0 2023-11-26T07:18:08.955 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:08.950+0000 7f70ce7fc640 1 -- 172.21.15.143:0/2800306999 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f70b80ab070 con 0x7f70bc0047d0 2023-11-26T07:18:08.955 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:08.950+0000 7f70ce7fc640 1 -- 172.21.15.143:0/2800306999 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f70b8007070 con 0x7f70bc0047d0 2023-11-26T07:18:08.955 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:08.950+0000 7f70d5632640 1 -- 172.21.15.143:0/2800306999 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f70bc0047d0 msgr2=0x7f70bc004c50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:18:08.955 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:08.950+0000 7f70d5632640 1 --2- 172.21.15.143:0/2800306999 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f70bc0047d0 0x7f70bc004c50 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7f70b800c240 tx=0x7f70b801f3b0 comp rx=0 tx=0).stop 2023-11-26T07:18:08.955 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:08.950+0000 7f70d5632640 1 -- 172.21.15.143:0/2800306999 shutdown_connections 2023-11-26T07:18:08.955 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:08.950+0000 7f70d5632640 1 --2- 172.21.15.143:0/2800306999 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f70bc0047d0 0x7f70bc004c50 unknown :-1 s=CLOSED pgs=159 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:18:08.955 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:08.950+0000 7f70d5632640 1 -- 172.21.15.143:0/2800306999 >> 172.21.15.143:0/2800306999 conn(0x7f70d009c550 msgr2=0x7f70d00d3210 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:18:08.956 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:08.950+0000 7f70d5632640 1 -- 172.21.15.143:0/2800306999 shutdown_connections 2023-11-26T07:18:08.956 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:08.950+0000 7f70d5632640 1 -- 172.21.15.143:0/2800306999 wait complete. 2023-11-26T07:18:08.958 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:08.950+0000 7f70d5632640 1 Processor -- start 2023-11-26T07:18:08.958 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:08.950+0000 7f70d5632640 1 -- start start 2023-11-26T07:18:08.958 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:08.950+0000 7f70d5632640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f70bc0047d0 0x7f70d00b20f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:18:08.958 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:08.950+0000 7f70d5632640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f70d00b7eb0 0x7f70d009d360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:18:08.958 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:08.950+0000 7f70d5632640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f70d009d8a0 0x7f70d00b3890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:18:08.958 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:08.950+0000 7f70d5632640 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f70d0152a40 con 0x7f70d009d8a0 2023-11-26T07:18:08.958 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:08.950+0000 7f70d5632640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f70d00b27f0 con 0x7f70d00b7eb0 2023-11-26T07:18:08.959 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:08.950+0000 7f70d5632640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f70d00b2ad0 con 0x7f70bc0047d0 2023-11-26T07:18:08.959 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:08.954+0000 7f70cf7fe640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f70d009d8a0 0x7f70d00b3890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:18:08.959 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:08.954+0000 7f70cf7fe640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f70d009d8a0 0x7f70d00b3890 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:48372/0 (socket says 172.21.15.143:48372) 2023-11-26T07:18:08.959 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:08.954+0000 7f70cf7fe640 1 -- 172.21.15.143:0/1911951423 learned_addr learned my addr 172.21.15.143:0/1911951423 (peer_addr_for_me v2:172.21.15.143:0/0) 2023-11-26T07:18:08.959 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:08.954+0000 7f70ceffd640 1 --2- 172.21.15.143:0/1911951423 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f70bc0047d0 0x7f70d00b20f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:18:08.959 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:08.954+0000 7f70c67fc640 1 --2- 172.21.15.143:0/1911951423 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f70d00b7eb0 0x7f70d009d360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:18:08.959 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:08.954+0000 7f70cf7fe640 1 -- 172.21.15.143:0/1911951423 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f70bc0047d0 msgr2=0x7f70d00b20f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:18:08.959 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:08.954+0000 7f70cf7fe640 1 --2- 172.21.15.143:0/1911951423 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f70bc0047d0 0x7f70d00b20f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:18:08.959 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:08.954+0000 7f70cf7fe640 1 -- 172.21.15.143:0/1911951423 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f70d00b7eb0 msgr2=0x7f70d009d360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:18:08.959 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:08.954+0000 7f70cf7fe640 1 --2- 172.21.15.143:0/1911951423 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f70d00b7eb0 0x7f70d009d360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:18:08.959 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:08.954+0000 7f70cf7fe640 1 -- 172.21.15.143:0/1911951423 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f70d00b2db0 con 0x7f70d009d8a0 2023-11-26T07:18:08.959 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:08.954+0000 7f70ceffd640 1 --2- 172.21.15.143:0/1911951423 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f70bc0047d0 0x7f70d00b20f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-26T07:18:08.959 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:08.954+0000 7f70c67fc640 1 --2- 172.21.15.143:0/1911951423 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f70d00b7eb0 0x7f70d009d360 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-26T07:18:08.960 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:08.954+0000 7f70cf7fe640 1 --2- 172.21.15.143:0/1911951423 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f70d009d8a0 0x7f70d00b3890 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7f70d00b9240 tx=0x7f70c800eb90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:18:08.960 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:08.954+0000 7f70c7fff640 1 -- 172.21.15.143:0/1911951423 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f70c800eed0 con 0x7f70d009d8a0 2023-11-26T07:18:08.960 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:08.954+0000 7f70c7fff640 1 -- 172.21.15.143:0/1911951423 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f70c8003980 con 0x7f70d009d8a0 2023-11-26T07:18:08.960 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:08.954+0000 7f70d5632640 1 -- 172.21.15.143:0/1911951423 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f70d00b3090 con 0x7f70d009d8a0 2023-11-26T07:18:08.960 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:08.954+0000 7f70c7fff640 1 -- 172.21.15.143:0/1911951423 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f70c8003d70 con 0x7f70d009d8a0 2023-11-26T07:18:08.960 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:08.954+0000 7f70d5632640 1 -- 172.21.15.143:0/1911951423 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f70d015bc90 con 0x7f70d009d8a0 2023-11-26T07:18:08.961 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:08.954+0000 7f70d5632640 1 -- 172.21.15.143:0/1911951423 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f70d0067680 con 0x7f70d009d8a0 2023-11-26T07:18:08.961 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:08.954+0000 7f70c7fff640 1 -- 172.21.15.143:0/1911951423 <== mon.0 v2:172.21.15.143:3300/0 4 ==== mgrmap(e 5) v1 ==== 85852+0+0 (secure 0 0 0) 0x7f70c80050d0 con 0x7f70d009d8a0 2023-11-26T07:18:08.961 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:08.954+0000 7f70c7fff640 1 --2- 172.21.15.143:0/1911951423 >> [v2:172.21.15.143:6832/2150557544,v1:172.21.15.143:6833/2150557544] conn(0x7f70b0066470 0x7f70b0068930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:18:08.961 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:08.954+0000 7f70c7fff640 1 -- 172.21.15.143:0/1911951423 <== mon.0 v2:172.21.15.143:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f70c8083b60 con 0x7f70d009d8a0 2023-11-26T07:18:08.961 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:08.954+0000 7f70ceffd640 1 --2- 172.21.15.143:0/1911951423 >> [v2:172.21.15.143:6832/2150557544,v1:172.21.15.143:6833/2150557544] conn(0x7f70b0066470 0x7f70b0068930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:18:08.961 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:08.958+0000 7f70ceffd640 1 --2- 172.21.15.143:0/1911951423 >> [v2:172.21.15.143:6832/2150557544,v1:172.21.15.143:6833/2150557544] conn(0x7f70b0066470 0x7f70b0068930 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7f70d00b89a0 tx=0x7f70c0008040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:18:08.964 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:08.962+0000 7f70c7fff640 1 -- 172.21.15.143:0/1911951423 <== mon.0 v2:172.21.15.143:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172649 (secure 0 0 0) 0x7f70c8050fd0 con 0x7f70d009d8a0 2023-11-26T07:18:09.144 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:09.138+0000 7f70d5632640 1 -- 172.21.15.143:0/1911951423 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "fs ls"} v 0) v1 -- 0x7f70d00b3540 con 0x7f70d009d8a0 2023-11-26T07:18:09.145 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:09.138+0000 7f70c7fff640 1 -- 172.21.15.143:0/1911951423 <== mon.0 v2:172.21.15.143:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v8) v1 ==== 53+0+73 (secure 0 0 0) 0x7f70c80545f0 con 0x7f70d009d8a0 2023-11-26T07:18:09.145 INFO:teuthology.orchestra.run.smithi143.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2023-11-26T07:18:09.146 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:09.142+0000 7f70d5632640 1 -- 172.21.15.143:0/1911951423 >> [v2:172.21.15.143:6832/2150557544,v1:172.21.15.143:6833/2150557544] conn(0x7f70b0066470 msgr2=0x7f70b0068930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:18:09.146 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:09.142+0000 7f70d5632640 1 --2- 172.21.15.143:0/1911951423 >> [v2:172.21.15.143:6832/2150557544,v1:172.21.15.143:6833/2150557544] conn(0x7f70b0066470 0x7f70b0068930 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7f70d00b89a0 tx=0x7f70c0008040 comp rx=0 tx=0).stop 2023-11-26T07:18:09.147 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:09.142+0000 7f70d5632640 1 -- 172.21.15.143:0/1911951423 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f70d009d8a0 msgr2=0x7f70d00b3890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:18:09.147 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:09.142+0000 7f70d5632640 1 --2- 172.21.15.143:0/1911951423 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f70d009d8a0 0x7f70d00b3890 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7f70d00b9240 tx=0x7f70c800eb90 comp rx=0 tx=0).stop 2023-11-26T07:18:09.147 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:09.142+0000 7f70d5632640 1 -- 172.21.15.143:0/1911951423 shutdown_connections 2023-11-26T07:18:09.147 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:09.142+0000 7f70d5632640 1 --2- 172.21.15.143:0/1911951423 >> [v2:172.21.15.143:6832/2150557544,v1:172.21.15.143:6833/2150557544] conn(0x7f70b0066470 0x7f70b0068930 unknown :-1 s=CLOSED pgs=117 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:18:09.147 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:09.142+0000 7f70d5632640 1 --2- 172.21.15.143:0/1911951423 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f70d009d8a0 0x7f70d00b3890 unknown :-1 s=CLOSED pgs=160 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:18:09.147 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:09.142+0000 7f70d5632640 1 --2- 172.21.15.143:0/1911951423 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f70d00b7eb0 0x7f70d009d360 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:18:09.147 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:09.142+0000 7f70d5632640 1 --2- 172.21.15.143:0/1911951423 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f70bc0047d0 0x7f70d00b20f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:18:09.147 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:09.142+0000 7f70d5632640 1 -- 172.21.15.143:0/1911951423 >> 172.21.15.143:0/1911951423 conn(0x7f70d009c550 msgr2=0x7f70d00cc2f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:18:09.147 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:09.142+0000 7f70d5632640 1 -- 172.21.15.143:0/1911951423 shutdown_connections 2023-11-26T07:18:09.147 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:09.142+0000 7f70d5632640 1 -- 172.21.15.143:0/1911951423 wait complete. 2023-11-26T07:18:09.165 INFO:tasks.cephfs.mount:Mounting default Ceph FS; just confirmed its presence on cluster 2023-11-26T07:18:09.166 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2023-11-26T07:18:09.166 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi143.front.sepia.ceph.com 2023-11-26T07:18:09.166 INFO:tasks.cephfs.mount:self.client.name = client.0 2023-11-26T07:18:09.166 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2023-11-26T07:18:09.166 INFO:tasks.cephfs.mount:self.cephfs_name = None 2023-11-26T07:18:09.166 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2023-11-26T07:18:09.166 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2023-11-26T07:18:09.166 INFO:tasks.cephfs.mount:Setting the 'None' netns for '/home/ubuntu/cephtest/mnt.0' 2023-11-26T07:18:09.166 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-26T07:18:09.167 DEBUG:teuthology.orchestra.run.smithi143:> ip addr 2023-11-26T07:18:09.175 INFO:teuthology.orchestra.run.smithi143.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2023-11-26T07:18:09.176 INFO:teuthology.orchestra.run.smithi143.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2023-11-26T07:18:09.176 INFO:teuthology.orchestra.run.smithi143.stdout: inet 127.0.0.1/8 scope host lo 2023-11-26T07:18:09.176 INFO:teuthology.orchestra.run.smithi143.stdout: valid_lft forever preferred_lft forever 2023-11-26T07:18:09.176 INFO:teuthology.orchestra.run.smithi143.stdout: inet6 ::1/128 scope host 2023-11-26T07:18:09.176 INFO:teuthology.orchestra.run.smithi143.stdout: valid_lft forever preferred_lft forever 2023-11-26T07:18:09.176 INFO:teuthology.orchestra.run.smithi143.stdout:2: eno1: mtu 1500 qdisc noop state DOWN group default qlen 1000 2023-11-26T07:18:09.176 INFO:teuthology.orchestra.run.smithi143.stdout: link/ether 0c:c4:7a:d9:3b:94 brd ff:ff:ff:ff:ff:ff 2023-11-26T07:18:09.176 INFO:teuthology.orchestra.run.smithi143.stdout: altname enp7s0f0 2023-11-26T07:18:09.176 INFO:teuthology.orchestra.run.smithi143.stdout:3: eno2: mtu 1500 qdisc noop state DOWN group default qlen 1000 2023-11-26T07:18:09.176 INFO:teuthology.orchestra.run.smithi143.stdout: link/ether 0c:c4:7a:d9:3b:95 brd ff:ff:ff:ff:ff:ff 2023-11-26T07:18:09.176 INFO:teuthology.orchestra.run.smithi143.stdout: altname enp7s0f1 2023-11-26T07:18:09.176 INFO:teuthology.orchestra.run.smithi143.stdout:4: enp3s0f0: mtu 1500 qdisc noop state DOWN group default qlen 1000 2023-11-26T07:18:09.177 INFO:teuthology.orchestra.run.smithi143.stdout: link/ether 0c:c4:7a:88:77:82 brd ff:ff:ff:ff:ff:ff 2023-11-26T07:18:09.177 INFO:teuthology.orchestra.run.smithi143.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2023-11-26T07:18:09.177 INFO:teuthology.orchestra.run.smithi143.stdout: link/ether 0c:c4:7a:88:77:83 brd ff:ff:ff:ff:ff:ff 2023-11-26T07:18:09.177 INFO:teuthology.orchestra.run.smithi143.stdout: inet 172.21.15.143/20 metric 100 brd 172.21.15.255 scope global dynamic enp3s0f1 2023-11-26T07:18:09.177 INFO:teuthology.orchestra.run.smithi143.stdout: valid_lft 42384sec preferred_lft 42384sec 2023-11-26T07:18:09.177 INFO:teuthology.orchestra.run.smithi143.stdout: inet6 fe80::ec4:7aff:fe88:7783/64 scope link 2023-11-26T07:18:09.177 INFO:teuthology.orchestra.run.smithi143.stdout: valid_lft forever preferred_lft forever 2023-11-26T07:18:09.177 INFO:teuthology.orchestra.run.smithi143.stdout:6: docker0: mtu 1500 qdisc noqueue state DOWN group default 2023-11-26T07:18:09.177 INFO:teuthology.orchestra.run.smithi143.stdout: link/ether 02:42:c4:b4:2f:eb brd ff:ff:ff:ff:ff:ff 2023-11-26T07:18:09.177 INFO:teuthology.orchestra.run.smithi143.stdout: inet 172.17.0.1/16 brd 172.17.255.255 scope global docker0 2023-11-26T07:18:09.177 INFO:teuthology.orchestra.run.smithi143.stdout: valid_lft forever preferred_lft forever 2023-11-26T07:18:09.179 INFO:tasks.cephfs.mount:Setuping the 'ceph-brx' with 192.168.255.254/16 2023-11-26T07:18:09.179 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-26T07:18:09.179 DEBUG:teuthology.orchestra.run.smithi143:> (cd / && exec bash -c ' 2023-11-26T07:18:09.179 DEBUG:teuthology.orchestra.run.smithi143:> set -e 2023-11-26T07:18:09.179 DEBUG:teuthology.orchestra.run.smithi143:> sudo ip link add name ceph-brx type bridge 2023-11-26T07:18:09.179 DEBUG:teuthology.orchestra.run.smithi143:> sudo ip addr flush dev ceph-brx 2023-11-26T07:18:09.179 DEBUG:teuthology.orchestra.run.smithi143:> sudo ip link set ceph-brx up 2023-11-26T07:18:09.180 DEBUG:teuthology.orchestra.run.smithi143:> sudo ip addr add 192.168.255.254/16 brd 192.168.255.255 dev ceph-brx 2023-11-26T07:18:09.180 DEBUG:teuthology.orchestra.run.smithi143:> ') 2023-11-26T07:18:09.290 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-26T07:18:09.290 DEBUG:teuthology.orchestra.run.smithi143:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2023-11-26T07:18:09.310 INFO:teuthology.orchestra.run.smithi143.stdout:1 2023-11-26T07:18:09.312 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-26T07:18:09.312 DEBUG:teuthology.orchestra.run.smithi143:> route 2023-11-26T07:18:09.388 INFO:teuthology.orchestra.run.smithi143.stdout:Kernel IP routing table 2023-11-26T07:18:09.389 INFO:teuthology.orchestra.run.smithi143.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2023-11-26T07:18:09.389 INFO:teuthology.orchestra.run.smithi143.stdout:default _gateway 0.0.0.0 UG 0 0 0 enp3s0f1 2023-11-26T07:18:09.389 INFO:teuthology.orchestra.run.smithi143.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2023-11-26T07:18:09.389 INFO:teuthology.orchestra.run.smithi143.stdout:172.17.0.0 0.0.0.0 255.255.0.0 U 0 0 0 docker0 2023-11-26T07:18:09.389 INFO:teuthology.orchestra.run.smithi143.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 enp3s0f1 2023-11-26T07:18:09.389 INFO:teuthology.orchestra.run.smithi143.stdout:ns1.front.sepia 0.0.0.0 255.255.255.255 UH 100 0 0 enp3s0f1 2023-11-26T07:18:09.389 INFO:teuthology.orchestra.run.smithi143.stdout:172.21.0.2 0.0.0.0 255.255.255.255 UH 100 0 0 enp3s0f1 2023-11-26T07:18:09.389 INFO:teuthology.orchestra.run.smithi143.stdout:_gateway 0.0.0.0 255.255.255.255 UH 100 0 0 enp3s0f1 2023-11-26T07:18:09.389 INFO:teuthology.orchestra.run.smithi143.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2023-11-26T07:18:09.390 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-26T07:18:09.390 DEBUG:teuthology.orchestra.run.smithi143:> (cd / && exec bash -c ' 2023-11-26T07:18:09.390 DEBUG:teuthology.orchestra.run.smithi143:> set -e 2023-11-26T07:18:09.390 DEBUG:teuthology.orchestra.run.smithi143:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2023-11-26T07:18:09.390 DEBUG:teuthology.orchestra.run.smithi143:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2023-11-26T07:18:09.390 DEBUG:teuthology.orchestra.run.smithi143:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2023-11-26T07:18:09.390 DEBUG:teuthology.orchestra.run.smithi143:> ') 2023-11-26T07:18:09.482 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-26T07:18:09.482 DEBUG:teuthology.orchestra.run.smithi143:> ip netns list 2023-11-26T07:18:09.492 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-26T07:18:09.492 DEBUG:teuthology.orchestra.run.smithi143:> ip netns list-id 2023-11-26T07:18:09.545 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-26T07:18:09.545 DEBUG:teuthology.orchestra.run.smithi143:> (cd / && exec bash -c ' 2023-11-26T07:18:09.546 DEBUG:teuthology.orchestra.run.smithi143:> set -e 2023-11-26T07:18:09.546 DEBUG:teuthology.orchestra.run.smithi143:> sudo ip netns add ceph-ns--home-ubuntu-cephtest-mnt.0 2023-11-26T07:18:09.546 DEBUG:teuthology.orchestra.run.smithi143:> sudo ip netns set ceph-ns--home-ubuntu-cephtest-mnt.0 0 2023-11-26T07:18:09.546 DEBUG:teuthology.orchestra.run.smithi143:> ') 2023-11-26T07:18:09.635 INFO:tasks.cephfs.mount:Setuping the netns 'ceph-ns--home-ubuntu-cephtest-mnt.0' with 192.168.0.1/16 2023-11-26T07:18:09.635 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-26T07:18:09.635 DEBUG:teuthology.orchestra.run.smithi143:> (cd / && exec bash -c ' 2023-11-26T07:18:09.635 DEBUG:teuthology.orchestra.run.smithi143:> set -e 2023-11-26T07:18:09.636 DEBUG:teuthology.orchestra.run.smithi143:> sudo ip link add veth0 netns ceph-ns--home-ubuntu-cephtest-mnt.0 type veth peer name brx.0 2023-11-26T07:18:09.636 DEBUG:teuthology.orchestra.run.smithi143:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip addr add 192.168.0.1/16 brd 192.168.255.255 dev veth0 2023-11-26T07:18:09.636 DEBUG:teuthology.orchestra.run.smithi143:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip link set veth0 up 2023-11-26T07:18:09.636 DEBUG:teuthology.orchestra.run.smithi143:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip link set lo up 2023-11-26T07:18:09.636 DEBUG:teuthology.orchestra.run.smithi143:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip route add default via 192.168.255.254 2023-11-26T07:18:09.636 DEBUG:teuthology.orchestra.run.smithi143:> ') 2023-11-26T07:18:09.802 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-26T07:18:09.802 DEBUG:teuthology.orchestra.run.smithi143:> (cd / && exec bash -c ' 2023-11-26T07:18:09.802 DEBUG:teuthology.orchestra.run.smithi143:> set -e 2023-11-26T07:18:09.803 DEBUG:teuthology.orchestra.run.smithi143:> sudo ip link set brx.0 up 2023-11-26T07:18:09.803 DEBUG:teuthology.orchestra.run.smithi143:> sudo ip link set dev brx.0 master ceph-brx 2023-11-26T07:18:09.803 DEBUG:teuthology.orchestra.run.smithi143:> ') 2023-11-26T07:18:09.844 INFO:tasks.cephfs.fuse_mount:Client client.0 config is {} 2023-11-26T07:18:09.844 INFO:teuthology.orchestra.run:Running command with timeout 60 2023-11-26T07:18:09.845 DEBUG:teuthology.orchestra.run.smithi143:> mkdir -p -v /home/ubuntu/cephtest/mnt.0 2023-11-26T07:18:09.901 INFO:teuthology.orchestra.run.smithi143.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.0' 2023-11-26T07:18:09.901 INFO:teuthology.orchestra.run:Running command with timeout 60 2023-11-26T07:18:09.901 DEBUG:teuthology.orchestra.run.smithi143:> chmod 0000 /home/ubuntu/cephtest/mnt.0 2023-11-26T07:18:09.952 DEBUG:teuthology.orchestra.run.smithi143:> sudo modprobe fuse 2023-11-26T07:18:10.014 DEBUG:teuthology.orchestra.run.smithi143:> cat /proc/self/mounts | awk '{print $2}' 2023-11-26T07:18:10.070 INFO:teuthology.orchestra.run.smithi143.stdout:/sys 2023-11-26T07:18:10.070 INFO:teuthology.orchestra.run.smithi143.stdout:/proc 2023-11-26T07:18:10.071 INFO:teuthology.orchestra.run.smithi143.stdout:/dev 2023-11-26T07:18:10.071 INFO:teuthology.orchestra.run.smithi143.stdout:/dev/pts 2023-11-26T07:18:10.071 INFO:teuthology.orchestra.run.smithi143.stdout:/run 2023-11-26T07:18:10.071 INFO:teuthology.orchestra.run.smithi143.stdout:/ 2023-11-26T07:18:10.071 INFO:teuthology.orchestra.run.smithi143.stdout:/sys/kernel/security 2023-11-26T07:18:10.071 INFO:teuthology.orchestra.run.smithi143.stdout:/dev/shm 2023-11-26T07:18:10.071 INFO:teuthology.orchestra.run.smithi143.stdout:/run/lock 2023-11-26T07:18:10.071 INFO:teuthology.orchestra.run.smithi143.stdout:/sys/fs/cgroup 2023-11-26T07:18:10.071 INFO:teuthology.orchestra.run.smithi143.stdout:/sys/fs/pstore 2023-11-26T07:18:10.071 INFO:teuthology.orchestra.run.smithi143.stdout:/sys/fs/bpf 2023-11-26T07:18:10.071 INFO:teuthology.orchestra.run.smithi143.stdout:/proc/sys/fs/binfmt_misc 2023-11-26T07:18:10.072 INFO:teuthology.orchestra.run.smithi143.stdout:/dev/hugepages 2023-11-26T07:18:10.072 INFO:teuthology.orchestra.run.smithi143.stdout:/dev/mqueue 2023-11-26T07:18:10.072 INFO:teuthology.orchestra.run.smithi143.stdout:/sys/kernel/debug 2023-11-26T07:18:10.072 INFO:teuthology.orchestra.run.smithi143.stdout:/sys/kernel/tracing 2023-11-26T07:18:10.072 INFO:teuthology.orchestra.run.smithi143.stdout:/proc/fs/nfsd 2023-11-26T07:18:10.072 INFO:teuthology.orchestra.run.smithi143.stdout:/sys/fs/fuse/connections 2023-11-26T07:18:10.072 INFO:teuthology.orchestra.run.smithi143.stdout:/sys/kernel/config 2023-11-26T07:18:10.072 INFO:teuthology.orchestra.run.smithi143.stdout:/run/credentials/systemd-sysusers.service 2023-11-26T07:18:10.072 INFO:teuthology.orchestra.run.smithi143.stdout:/run/rpc_pipefs 2023-11-26T07:18:10.073 INFO:teuthology.orchestra.run.smithi143.stdout:/run/user/1000 2023-11-26T07:18:10.073 INFO:teuthology.orchestra.run.smithi143.stdout:/var/lib/ceph 2023-11-26T07:18:10.073 INFO:teuthology.orchestra.run.smithi143.stdout:/var/lib/ceph/osd/ceph-0 2023-11-26T07:18:10.073 INFO:teuthology.orchestra.run.smithi143.stdout:/var/lib/ceph/osd/ceph-1 2023-11-26T07:18:10.073 INFO:teuthology.orchestra.run.smithi143.stdout:/var/lib/ceph/osd/ceph-2 2023-11-26T07:18:10.073 INFO:teuthology.orchestra.run.smithi143.stdout:/var/lib/ceph/osd/ceph-3 2023-11-26T07:18:10.074 INFO:teuthology.orchestra.run.smithi143.stdout:/run/netns 2023-11-26T07:18:10.074 INFO:teuthology.orchestra.run.smithi143.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2023-11-26T07:18:10.074 INFO:teuthology.orchestra.run.smithi143.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2023-11-26T07:18:10.075 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-26T07:18:10.075 DEBUG:teuthology.orchestra.run.smithi143:> ls /sys/fs/fuse/connections 2023-11-26T07:18:10.128 INFO:tasks.cephfs.fuse_mount:Pre-mount connections: [] 2023-11-26T07:18:10.129 DEBUG:teuthology.orchestra.run.smithi143:> (cd /home/ubuntu/cephtest && exec sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-fuse -f --admin-socket '/var/run/ceph/$cluster-$name.$pid.asok' /home/ubuntu/cephtest/mnt.0 --id 0) 2023-11-26T07:18:10.179 DEBUG:teuthology.orchestra.run.smithi143:> sudo modprobe fuse 2023-11-26T07:18:10.200 DEBUG:teuthology.orchestra.run.smithi143:> cat /proc/self/mounts | awk '{print $2}' 2023-11-26T07:18:10.255 INFO:teuthology.orchestra.run.smithi143.stdout:/sys 2023-11-26T07:18:10.255 INFO:teuthology.orchestra.run.smithi143.stdout:/proc 2023-11-26T07:18:10.255 INFO:teuthology.orchestra.run.smithi143.stdout:/dev 2023-11-26T07:18:10.255 INFO:teuthology.orchestra.run.smithi143.stdout:/dev/pts 2023-11-26T07:18:10.255 INFO:teuthology.orchestra.run.smithi143.stdout:/run 2023-11-26T07:18:10.255 INFO:teuthology.orchestra.run.smithi143.stdout:/ 2023-11-26T07:18:10.256 INFO:teuthology.orchestra.run.smithi143.stdout:/sys/kernel/security 2023-11-26T07:18:10.256 INFO:teuthology.orchestra.run.smithi143.stdout:/dev/shm 2023-11-26T07:18:10.256 INFO:teuthology.orchestra.run.smithi143.stdout:/run/lock 2023-11-26T07:18:10.256 INFO:teuthology.orchestra.run.smithi143.stdout:/sys/fs/cgroup 2023-11-26T07:18:10.256 INFO:teuthology.orchestra.run.smithi143.stdout:/sys/fs/pstore 2023-11-26T07:18:10.256 INFO:teuthology.orchestra.run.smithi143.stdout:/sys/fs/bpf 2023-11-26T07:18:10.256 INFO:teuthology.orchestra.run.smithi143.stdout:/proc/sys/fs/binfmt_misc 2023-11-26T07:18:10.256 INFO:teuthology.orchestra.run.smithi143.stdout:/dev/hugepages 2023-11-26T07:18:10.256 INFO:teuthology.orchestra.run.smithi143.stdout:/dev/mqueue 2023-11-26T07:18:10.256 INFO:teuthology.orchestra.run.smithi143.stdout:/sys/kernel/debug 2023-11-26T07:18:10.256 INFO:teuthology.orchestra.run.smithi143.stdout:/sys/kernel/tracing 2023-11-26T07:18:10.257 INFO:teuthology.orchestra.run.smithi143.stdout:/proc/fs/nfsd 2023-11-26T07:18:10.257 INFO:teuthology.orchestra.run.smithi143.stdout:/sys/fs/fuse/connections 2023-11-26T07:18:10.257 INFO:teuthology.orchestra.run.smithi143.stdout:/sys/kernel/config 2023-11-26T07:18:10.257 INFO:teuthology.orchestra.run.smithi143.stdout:/run/credentials/systemd-sysusers.service 2023-11-26T07:18:10.257 INFO:teuthology.orchestra.run.smithi143.stdout:/run/rpc_pipefs 2023-11-26T07:18:10.257 INFO:teuthology.orchestra.run.smithi143.stdout:/run/user/1000 2023-11-26T07:18:10.257 INFO:teuthology.orchestra.run.smithi143.stdout:/var/lib/ceph 2023-11-26T07:18:10.257 INFO:teuthology.orchestra.run.smithi143.stdout:/var/lib/ceph/osd/ceph-0 2023-11-26T07:18:10.257 INFO:teuthology.orchestra.run.smithi143.stdout:/var/lib/ceph/osd/ceph-1 2023-11-26T07:18:10.257 INFO:teuthology.orchestra.run.smithi143.stdout:/var/lib/ceph/osd/ceph-2 2023-11-26T07:18:10.257 INFO:teuthology.orchestra.run.smithi143.stdout:/var/lib/ceph/osd/ceph-3 2023-11-26T07:18:10.257 INFO:teuthology.orchestra.run.smithi143.stdout:/run/netns 2023-11-26T07:18:10.257 INFO:teuthology.orchestra.run.smithi143.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2023-11-26T07:18:10.258 INFO:teuthology.orchestra.run.smithi143.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2023-11-26T07:18:10.258 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-26T07:18:10.259 DEBUG:teuthology.orchestra.run.smithi143:> ls /sys/fs/fuse/connections 2023-11-26T07:18:10.287 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi143.stderr:2023-11-26T07:18:10.282+0000 7f6306bad580 -1 init, newargv = 0x5562566f01b0 newargc=15 2023-11-26T07:18:10.287 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi143.stderr:ceph-fuse[45847]: starting ceph client 2023-11-26T07:18:10.296 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi143.stderr:ceph-fuse[45847]: starting fuse 2023-11-26T07:18:10.297 INFO:teuthology.orchestra.run.smithi143.stdout:43 2023-11-26T07:18:10.297 INFO:tasks.cephfs.fuse_mount:Post-mount connections: [43] 2023-11-26T07:18:10.297 DEBUG:teuthology.orchestra.run.smithi143:> sudo stdin-killer -- python3 -c ' 2023-11-26T07:18:10.297 DEBUG:teuthology.orchestra.run.smithi143:> import glob 2023-11-26T07:18:10.297 DEBUG:teuthology.orchestra.run.smithi143:> import re 2023-11-26T07:18:10.297 DEBUG:teuthology.orchestra.run.smithi143:> import os 2023-11-26T07:18:10.297 DEBUG:teuthology.orchestra.run.smithi143:> import subprocess 2023-11-26T07:18:10.297 DEBUG:teuthology.orchestra.run.smithi143:> 2023-11-26T07:18:10.297 DEBUG:teuthology.orchestra.run.smithi143:> def _find_admin_socket(client_name): 2023-11-26T07:18:10.298 DEBUG:teuthology.orchestra.run.smithi143:> asok_path = "/var/run/ceph/ceph-client.0.*.asok" 2023-11-26T07:18:10.298 DEBUG:teuthology.orchestra.run.smithi143:> files = glob.glob(asok_path) 2023-11-26T07:18:10.298 DEBUG:teuthology.orchestra.run.smithi143:> mountpoint = "/home/ubuntu/cephtest/mnt.0" 2023-11-26T07:18:10.298 DEBUG:teuthology.orchestra.run.smithi143:> 2023-11-26T07:18:10.298 DEBUG:teuthology.orchestra.run.smithi143:> # Given a non-glob path, it better be there 2023-11-26T07:18:10.298 DEBUG:teuthology.orchestra.run.smithi143:> if "*" not in asok_path: 2023-11-26T07:18:10.298 DEBUG:teuthology.orchestra.run.smithi143:> assert(len(files) == 1) 2023-11-26T07:18:10.298 DEBUG:teuthology.orchestra.run.smithi143:> return files[0] 2023-11-26T07:18:10.298 DEBUG:teuthology.orchestra.run.smithi143:> 2023-11-26T07:18:10.298 DEBUG:teuthology.orchestra.run.smithi143:> for f in files: 2023-11-26T07:18:10.298 DEBUG:teuthology.orchestra.run.smithi143:> pid = re.match(".*\.(\d+)\.asok$", f).group(1) 2023-11-26T07:18:10.298 DEBUG:teuthology.orchestra.run.smithi143:> if os.path.exists("/proc/{0}".format(pid)): 2023-11-26T07:18:10.298 DEBUG:teuthology.orchestra.run.smithi143:> with open("/proc/{0}/cmdline".format(pid), '"'"'r'"'"') as proc_f: 2023-11-26T07:18:10.299 DEBUG:teuthology.orchestra.run.smithi143:> contents = proc_f.read() 2023-11-26T07:18:10.299 DEBUG:teuthology.orchestra.run.smithi143:> if mountpoint in contents: 2023-11-26T07:18:10.299 DEBUG:teuthology.orchestra.run.smithi143:> return f 2023-11-26T07:18:10.299 DEBUG:teuthology.orchestra.run.smithi143:> raise RuntimeError("Client socket {0} not found".format(client_name)) 2023-11-26T07:18:10.299 DEBUG:teuthology.orchestra.run.smithi143:> 2023-11-26T07:18:10.299 DEBUG:teuthology.orchestra.run.smithi143:> print(_find_admin_socket("client.0")) 2023-11-26T07:18:10.299 DEBUG:teuthology.orchestra.run.smithi143:> ' 2023-11-26T07:18:10.424 INFO:teuthology.orchestra.run.smithi143.stdout:/var/run/ceph/ceph-client.0.45847.asok 2023-11-26T07:18:10.427 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:10 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2023-11-26T07:18:10.435 INFO:tasks.cephfs.fuse_mount:Found client admin socket at /var/run/ceph/ceph-client.0.45847.asok 2023-11-26T07:18:10.435 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-26T07:18:10.435 DEBUG:teuthology.orchestra.run.smithi143:> sudo ceph --admin-daemon /var/run/ceph/ceph-client.0.45847.asok status 2023-11-26T07:18:10.609 INFO:teuthology.orchestra.run.smithi143.stdout:{ 2023-11-26T07:18:10.610 INFO:teuthology.orchestra.run.smithi143.stdout: "metadata": { 2023-11-26T07:18:10.610 INFO:teuthology.orchestra.run.smithi143.stdout: "ceph_sha1": "44c2237fe77713214148c1fd622c872f346f1b99", 2023-11-26T07:18:10.610 INFO:teuthology.orchestra.run.smithi143.stdout: "ceph_version": "ceph version 18.0.0-7484-g44c2237f (44c2237fe77713214148c1fd622c872f346f1b99) reef (dev)", 2023-11-26T07:18:10.610 INFO:teuthology.orchestra.run.smithi143.stdout: "entity_id": "0", 2023-11-26T07:18:10.610 INFO:teuthology.orchestra.run.smithi143.stdout: "hostname": "smithi143", 2023-11-26T07:18:10.610 INFO:teuthology.orchestra.run.smithi143.stdout: "mount_point": "/home/ubuntu/cephtest/mnt.0", 2023-11-26T07:18:10.610 INFO:teuthology.orchestra.run.smithi143.stdout: "pid": "45847", 2023-11-26T07:18:10.610 INFO:teuthology.orchestra.run.smithi143.stdout: "root": "/" 2023-11-26T07:18:10.610 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2023-11-26T07:18:10.610 INFO:teuthology.orchestra.run.smithi143.stdout: "dentry_count": 0, 2023-11-26T07:18:10.610 INFO:teuthology.orchestra.run.smithi143.stdout: "dentry_pinned_count": 0, 2023-11-26T07:18:10.611 INFO:teuthology.orchestra.run.smithi143.stdout: "id": 4638, 2023-11-26T07:18:10.611 INFO:teuthology.orchestra.run.smithi143.stdout: "inst": { 2023-11-26T07:18:10.611 INFO:teuthology.orchestra.run.smithi143.stdout: "name": { 2023-11-26T07:18:10.611 INFO:teuthology.orchestra.run.smithi143.stdout: "type": "client", 2023-11-26T07:18:10.611 INFO:teuthology.orchestra.run.smithi143.stdout: "num": 4638 2023-11-26T07:18:10.611 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2023-11-26T07:18:10.611 INFO:teuthology.orchestra.run.smithi143.stdout: "addr": { 2023-11-26T07:18:10.611 INFO:teuthology.orchestra.run.smithi143.stdout: "type": "v1", 2023-11-26T07:18:10.611 INFO:teuthology.orchestra.run.smithi143.stdout: "addr": "192.168.0.1:0", 2023-11-26T07:18:10.611 INFO:teuthology.orchestra.run.smithi143.stdout: "nonce": 1363899290 2023-11-26T07:18:10.611 INFO:teuthology.orchestra.run.smithi143.stdout: } 2023-11-26T07:18:10.611 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2023-11-26T07:18:10.611 INFO:teuthology.orchestra.run.smithi143.stdout: "addr": { 2023-11-26T07:18:10.612 INFO:teuthology.orchestra.run.smithi143.stdout: "type": "v1", 2023-11-26T07:18:10.612 INFO:teuthology.orchestra.run.smithi143.stdout: "addr": "192.168.0.1:0", 2023-11-26T07:18:10.612 INFO:teuthology.orchestra.run.smithi143.stdout: "nonce": 1363899290 2023-11-26T07:18:10.612 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2023-11-26T07:18:10.612 INFO:teuthology.orchestra.run.smithi143.stdout: "inst_str": "client.4638 192.168.0.1:0/1363899290", 2023-11-26T07:18:10.612 INFO:teuthology.orchestra.run.smithi143.stdout: "addr_str": "192.168.0.1:0/1363899290", 2023-11-26T07:18:10.612 INFO:teuthology.orchestra.run.smithi143.stdout: "inode_count": 1, 2023-11-26T07:18:10.612 INFO:teuthology.orchestra.run.smithi143.stdout: "mds_epoch": 8, 2023-11-26T07:18:10.612 INFO:teuthology.orchestra.run.smithi143.stdout: "osd_epoch": 20, 2023-11-26T07:18:10.612 INFO:teuthology.orchestra.run.smithi143.stdout: "osd_epoch_barrier": 0, 2023-11-26T07:18:10.612 INFO:teuthology.orchestra.run.smithi143.stdout: "blocklisted": false, 2023-11-26T07:18:10.612 INFO:teuthology.orchestra.run.smithi143.stdout: "fs_name": "cephfs" 2023-11-26T07:18:10.612 INFO:teuthology.orchestra.run.smithi143.stdout:} 2023-11-26T07:18:10.627 DEBUG:teuthology.orchestra.run.smithi143:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2023-11-26T07:18:10.764 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:10.758+0000 7f30c4bdf640 1 Processor -- start 2023-11-26T07:18:10.765 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:10.758+0000 7f30c4bdf640 1 -- start start 2023-11-26T07:18:10.766 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:10.762+0000 7f30c4bdf640 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7f30c013e290 0x7f30c00cf280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:18:10.766 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:10.762+0000 7f30c4bdf640 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7f30c015f6b0 0x7f30c00cd7b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:18:10.766 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:10.762+0000 7f30c4bdf640 1 -- --> v1:172.21.15.143:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f30c015fb20 con 0x7f30c00cf7c0 2023-11-26T07:18:10.766 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:10.762+0000 7f30c4bdf640 1 -- --> v2:172.21.15.143:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f30c00cfc40 con 0x7f30c015f6b0 2023-11-26T07:18:10.766 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:10.762+0000 7f30c4bdf640 1 -- --> v2:172.21.15.188:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f30c006c520 con 0x7f30c013e290 2023-11-26T07:18:10.766 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:10.762+0000 7f30b7fff640 1 --1- >> v1:172.21.15.143:6789/0 conn(0x7f30c00cf7c0 0x7f30c015efa0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.143:6789/0 says I am v1:172.21.15.143:52254/0 (socket says 172.21.15.143:52254) 2023-11-26T07:18:10.766 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:10.762+0000 7f30bed9d640 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7f30c015f6b0 0x7f30c00cd7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:18:10.766 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:10.762+0000 7f30b7fff640 1 -- 172.21.15.143:0/2166667269 learned_addr learned my addr 172.21.15.143:0/2166667269 (peer_addr_for_me v1:172.21.15.143:0/0) 2023-11-26T07:18:10.767 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:10.762+0000 7f30be59c640 1 --2- 172.21.15.143:0/2166667269 >> v2:172.21.15.188:3300/0 conn(0x7f30c013e290 0x7f30c00cf280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:18:10.767 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:10.762+0000 7f30bed9d640 1 -- 172.21.15.143:0/2166667269 >> v2:172.21.15.188:3300/0 conn(0x7f30c013e290 msgr2=0x7f30c00cf280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-26T07:18:10.767 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:10.762+0000 7f30bed9d640 1 --2- 172.21.15.143:0/2166667269 >> v2:172.21.15.188:3300/0 conn(0x7f30c013e290 0x7f30c00cf280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:18:10.767 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:10.762+0000 7f30bed9d640 1 -- 172.21.15.143:0/2166667269 >> v1:172.21.15.143:6789/0 conn(0x7f30c00cf7c0 legacy=0x7f30c015efa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:18:10.767 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:10.762+0000 7f30bed9d640 1 -- 172.21.15.143:0/2166667269 --> v2:172.21.15.143:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f30c006c800 con 0x7f30c015f6b0 2023-11-26T07:18:10.769 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:10.766+0000 7f30bed9d640 1 --2- 172.21.15.143:0/2166667269 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f30c015f6b0 0x7f30c00cd7b0 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7f30c00725a0 tx=0x7f30b801c8e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a0f1f414d5d9bdb4 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:18:10.769 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:10.766+0000 7f30bdd9b640 1 -- 172.21.15.143:0/2166667269 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f30b80a59a0 con 0x7f30c015f6b0 2023-11-26T07:18:10.769 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:10.766+0000 7f30bdd9b640 1 -- 172.21.15.143:0/2166667269 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f30b80a5b00 con 0x7f30c015f6b0 2023-11-26T07:18:10.769 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:10.766+0000 7f30bdd9b640 1 -- 172.21.15.143:0/2166667269 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f30b80a5e90 con 0x7f30c015f6b0 2023-11-26T07:18:10.770 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:10.766+0000 7f30c4bdf640 1 -- 172.21.15.143:0/2166667269 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f30c015f6b0 msgr2=0x7f30c00cd7b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:18:10.770 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:10.766+0000 7f30c4bdf640 1 --2- 172.21.15.143:0/2166667269 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f30c015f6b0 0x7f30c00cd7b0 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7f30c00725a0 tx=0x7f30b801c8e0 comp rx=0 tx=0).stop 2023-11-26T07:18:10.770 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:10.766+0000 7f30c4bdf640 1 -- 172.21.15.143:0/2166667269 shutdown_connections 2023-11-26T07:18:10.770 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:10.766+0000 7f30c4bdf640 1 --2- 172.21.15.143:0/2166667269 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f30c015f6b0 0x7f30c00cd7b0 unknown :-1 s=CLOSED pgs=165 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:18:10.771 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:10.766+0000 7f30c4bdf640 1 --2- 172.21.15.143:0/2166667269 >> v2:172.21.15.188:3300/0 conn(0x7f30c013e290 0x7f30c00cf280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:18:10.771 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:10.766+0000 7f30c4bdf640 1 -- 172.21.15.143:0/2166667269 >> 172.21.15.143:0/2166667269 conn(0x7f30c006a890 msgr2=0x7f30c006aca0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:18:10.771 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:10.766+0000 7f30c4bdf640 1 -- 172.21.15.143:0/2166667269 shutdown_connections 2023-11-26T07:18:10.771 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:10.766+0000 7f30c4bdf640 1 -- 172.21.15.143:0/2166667269 wait complete. 2023-11-26T07:18:10.772 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:10.766+0000 7f30c4bdf640 1 Processor -- start 2023-11-26T07:18:10.772 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:10.766+0000 7f30c4bdf640 1 -- start start 2023-11-26T07:18:10.774 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:10.770+0000 7f30c4bdf640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f30c00cf7c0 0x7f30c01f8500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:18:10.774 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:10.770+0000 7f30c4bdf640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f30c013e290 0x7f30c015a120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:18:10.774 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:10.770+0000 7f30c4bdf640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f30c015f6b0 0x7f30c015a660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:18:10.774 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:10.770+0000 7f30c4bdf640 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f30c00b1ab0 con 0x7f30c013e290 2023-11-26T07:18:10.774 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:10.770+0000 7f30c4bdf640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f30c01f56d0 con 0x7f30c015f6b0 2023-11-26T07:18:10.774 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:10.770+0000 7f30c4bdf640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f30c0072360 con 0x7f30c00cf7c0 2023-11-26T07:18:10.774 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:10.770+0000 7f30b7fff640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f30c013e290 0x7f30c015a120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:18:10.774 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:10.770+0000 7f30b7fff640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f30c013e290 0x7f30c015a120 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:48382/0 (socket says 172.21.15.143:48382) 2023-11-26T07:18:10.774 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:10.770+0000 7f30b7fff640 1 -- 172.21.15.143:0/1321479332 learned_addr learned my addr 172.21.15.143:0/1321479332 (peer_addr_for_me v2:172.21.15.143:0/0) 2023-11-26T07:18:10.775 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:10.770+0000 7f30be59c640 1 --2- 172.21.15.143:0/1321479332 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f30c00cf7c0 0x7f30c01f8500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:18:10.775 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:10.770+0000 7f30bed9d640 1 --2- 172.21.15.143:0/1321479332 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f30c015f6b0 0x7f30c015a660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:18:10.775 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:10.770+0000 7f30b7fff640 1 -- 172.21.15.143:0/1321479332 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f30c00cf7c0 msgr2=0x7f30c01f8500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:18:10.775 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:10.770+0000 7f30b7fff640 1 --2- 172.21.15.143:0/1321479332 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f30c00cf7c0 0x7f30c01f8500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:18:10.775 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:10.770+0000 7f30b7fff640 1 -- 172.21.15.143:0/1321479332 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f30c015f6b0 msgr2=0x7f30c015a660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:18:10.775 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:10.770+0000 7f30b7fff640 1 --2- 172.21.15.143:0/1321479332 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f30c015f6b0 0x7f30c015a660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:18:10.775 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:10.770+0000 7f30b7fff640 1 -- 172.21.15.143:0/1321479332 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f30b8009a70 con 0x7f30c013e290 2023-11-26T07:18:10.775 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:10.770+0000 7f30b7fff640 1 --2- 172.21.15.143:0/1321479332 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f30c013e290 0x7f30c015a120 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7f30a8005890 tx=0x7f30a800c960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:18:10.776 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:10.770+0000 7f30b6ffd640 1 -- 172.21.15.143:0/1321479332 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f30a8005cf0 con 0x7f30c013e290 2023-11-26T07:18:10.776 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:10.770+0000 7f30b6ffd640 1 -- 172.21.15.143:0/1321479332 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f30a8005e50 con 0x7f30c013e290 2023-11-26T07:18:10.776 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:10.770+0000 7f30c4bdf640 1 -- 172.21.15.143:0/1321479332 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f30c00717e0 con 0x7f30c013e290 2023-11-26T07:18:10.776 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:10.770+0000 7f30b6ffd640 1 -- 172.21.15.143:0/1321479332 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f30a80072b0 con 0x7f30c013e290 2023-11-26T07:18:10.776 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:10.770+0000 7f30c4bdf640 1 -- 172.21.15.143:0/1321479332 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f30c0072080 con 0x7f30c013e290 2023-11-26T07:18:10.776 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:10.770+0000 7f30c4bdf640 1 -- 172.21.15.143:0/1321479332 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f30c0071be0 con 0x7f30c013e290 2023-11-26T07:18:10.776 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:10.770+0000 7f30b6ffd640 1 -- 172.21.15.143:0/1321479332 <== mon.0 v2:172.21.15.143:3300/0 4 ==== mgrmap(e 5) v1 ==== 85852+0+0 (secure 0 0 0) 0x7f30a8007410 con 0x7f30c013e290 2023-11-26T07:18:10.778 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:10.774+0000 7f30b6ffd640 1 --2- 172.21.15.143:0/1321479332 >> [v2:172.21.15.143:6832/2150557544,v1:172.21.15.143:6833/2150557544] conn(0x7f30a0066470 0x7f30a0068930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:18:10.778 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:10.774+0000 7f30b6ffd640 1 -- 172.21.15.143:0/1321479332 <== mon.0 v2:172.21.15.143:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f30a80831f0 con 0x7f30c013e290 2023-11-26T07:18:10.778 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:10.774+0000 7f30be59c640 1 --2- 172.21.15.143:0/1321479332 >> [v2:172.21.15.143:6832/2150557544,v1:172.21.15.143:6833/2150557544] conn(0x7f30a0066470 0x7f30a0068930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:18:10.778 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:10.774+0000 7f30be59c640 1 --2- 172.21.15.143:0/1321479332 >> [v2:172.21.15.143:6832/2150557544,v1:172.21.15.143:6833/2150557544] conn(0x7f30a0066470 0x7f30a0068930 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7f30b0007a60 tx=0x7f30b0008040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:18:10.782 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:10.778+0000 7f30b6ffd640 1 -- 172.21.15.143:0/1321479332 <== mon.0 v2:172.21.15.143:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172649 (secure 0 0 0) 0x7f30a8050680 con 0x7f30c013e290 2023-11-26T07:18:10.963 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:10.958+0000 7f30c4bdf640 1 -- 172.21.15.143:0/1321479332 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "fs ls"} v 0) v1 -- 0x7f30c015fb20 con 0x7f30c013e290 2023-11-26T07:18:10.963 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:10.958+0000 7f30b6ffd640 1 -- 172.21.15.143:0/1321479332 <== mon.0 v2:172.21.15.143:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v8) v1 ==== 53+0+73 (secure 0 0 0) 0x7f30a8017060 con 0x7f30c013e290 2023-11-26T07:18:10.963 INFO:teuthology.orchestra.run.smithi143.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2023-11-26T07:18:10.965 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:10.962+0000 7f30c4bdf640 1 -- 172.21.15.143:0/1321479332 >> [v2:172.21.15.143:6832/2150557544,v1:172.21.15.143:6833/2150557544] conn(0x7f30a0066470 msgr2=0x7f30a0068930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:18:10.965 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:10.962+0000 7f30c4bdf640 1 --2- 172.21.15.143:0/1321479332 >> [v2:172.21.15.143:6832/2150557544,v1:172.21.15.143:6833/2150557544] conn(0x7f30a0066470 0x7f30a0068930 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7f30b0007a60 tx=0x7f30b0008040 comp rx=0 tx=0).stop 2023-11-26T07:18:10.965 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:10.962+0000 7f30c4bdf640 1 -- 172.21.15.143:0/1321479332 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f30c013e290 msgr2=0x7f30c015a120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:18:10.965 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:10.962+0000 7f30c4bdf640 1 --2- 172.21.15.143:0/1321479332 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f30c013e290 0x7f30c015a120 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7f30a8005890 tx=0x7f30a800c960 comp rx=0 tx=0).stop 2023-11-26T07:18:10.965 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:10.962+0000 7f30c4bdf640 1 -- 172.21.15.143:0/1321479332 shutdown_connections 2023-11-26T07:18:10.966 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:10.962+0000 7f30c4bdf640 1 --2- 172.21.15.143:0/1321479332 >> [v2:172.21.15.143:6832/2150557544,v1:172.21.15.143:6833/2150557544] conn(0x7f30a0066470 0x7f30a0068930 unknown :-1 s=CLOSED pgs=118 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:18:10.966 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:10.962+0000 7f30c4bdf640 1 --2- 172.21.15.143:0/1321479332 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f30c015f6b0 0x7f30c015a660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:18:10.966 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:10.962+0000 7f30c4bdf640 1 --2- 172.21.15.143:0/1321479332 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f30c013e290 0x7f30c015a120 unknown :-1 s=CLOSED pgs=166 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:18:10.966 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:10.962+0000 7f30c4bdf640 1 --2- 172.21.15.143:0/1321479332 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f30c00cf7c0 0x7f30c01f8500 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:18:10.966 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:10.962+0000 7f30c4bdf640 1 -- 172.21.15.143:0/1321479332 >> 172.21.15.143:0/1321479332 conn(0x7f30c006a890 msgr2=0x7f30c0139f30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:18:10.966 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:10.962+0000 7f30c4bdf640 1 -- 172.21.15.143:0/1321479332 shutdown_connections 2023-11-26T07:18:10.966 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:10.962+0000 7f30c4bdf640 1 -- 172.21.15.143:0/1321479332 wait complete. 2023-11-26T07:18:10.985 INFO:tasks.cephfs.mount:Mounting default Ceph FS; just confirmed its presence on cluster 2023-11-26T07:18:10.985 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2023-11-26T07:18:10.985 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi143.front.sepia.ceph.com 2023-11-26T07:18:10.985 INFO:tasks.cephfs.mount:self.client.name = client.2 2023-11-26T07:18:10.985 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.2 2023-11-26T07:18:10.985 INFO:tasks.cephfs.mount:self.cephfs_name = None 2023-11-26T07:18:10.985 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2023-11-26T07:18:10.985 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2023-11-26T07:18:10.985 INFO:tasks.cephfs.mount:Setting the 'None' netns for '/home/ubuntu/cephtest/mnt.2' 2023-11-26T07:18:10.985 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-26T07:18:10.986 DEBUG:teuthology.orchestra.run.smithi143:> ip addr 2023-11-26T07:18:10.996 INFO:teuthology.orchestra.run.smithi143.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2023-11-26T07:18:10.996 INFO:teuthology.orchestra.run.smithi143.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2023-11-26T07:18:10.996 INFO:teuthology.orchestra.run.smithi143.stdout: inet 127.0.0.1/8 scope host lo 2023-11-26T07:18:10.996 INFO:teuthology.orchestra.run.smithi143.stdout: valid_lft forever preferred_lft forever 2023-11-26T07:18:10.996 INFO:teuthology.orchestra.run.smithi143.stdout: inet6 ::1/128 scope host 2023-11-26T07:18:10.996 INFO:teuthology.orchestra.run.smithi143.stdout: valid_lft forever preferred_lft forever 2023-11-26T07:18:10.997 INFO:teuthology.orchestra.run.smithi143.stdout:2: eno1: mtu 1500 qdisc noop state DOWN group default qlen 1000 2023-11-26T07:18:10.997 INFO:teuthology.orchestra.run.smithi143.stdout: link/ether 0c:c4:7a:d9:3b:94 brd ff:ff:ff:ff:ff:ff 2023-11-26T07:18:10.997 INFO:teuthology.orchestra.run.smithi143.stdout: altname enp7s0f0 2023-11-26T07:18:10.997 INFO:teuthology.orchestra.run.smithi143.stdout:3: eno2: mtu 1500 qdisc noop state DOWN group default qlen 1000 2023-11-26T07:18:10.997 INFO:teuthology.orchestra.run.smithi143.stdout: link/ether 0c:c4:7a:d9:3b:95 brd ff:ff:ff:ff:ff:ff 2023-11-26T07:18:10.997 INFO:teuthology.orchestra.run.smithi143.stdout: altname enp7s0f1 2023-11-26T07:18:10.997 INFO:teuthology.orchestra.run.smithi143.stdout:4: enp3s0f0: mtu 1500 qdisc noop state DOWN group default qlen 1000 2023-11-26T07:18:10.997 INFO:teuthology.orchestra.run.smithi143.stdout: link/ether 0c:c4:7a:88:77:82 brd ff:ff:ff:ff:ff:ff 2023-11-26T07:18:10.997 INFO:teuthology.orchestra.run.smithi143.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2023-11-26T07:18:10.998 INFO:teuthology.orchestra.run.smithi143.stdout: link/ether 0c:c4:7a:88:77:83 brd ff:ff:ff:ff:ff:ff 2023-11-26T07:18:10.998 INFO:teuthology.orchestra.run.smithi143.stdout: inet 172.21.15.143/20 metric 100 brd 172.21.15.255 scope global dynamic enp3s0f1 2023-11-26T07:18:10.998 INFO:teuthology.orchestra.run.smithi143.stdout: valid_lft 42383sec preferred_lft 42383sec 2023-11-26T07:18:10.998 INFO:teuthology.orchestra.run.smithi143.stdout: inet6 fe80::ec4:7aff:fe88:7783/64 scope link 2023-11-26T07:18:10.998 INFO:teuthology.orchestra.run.smithi143.stdout: valid_lft forever preferred_lft forever 2023-11-26T07:18:10.998 INFO:teuthology.orchestra.run.smithi143.stdout:6: docker0: mtu 1500 qdisc noqueue state DOWN group default 2023-11-26T07:18:10.998 INFO:teuthology.orchestra.run.smithi143.stdout: link/ether 02:42:c4:b4:2f:eb brd ff:ff:ff:ff:ff:ff 2023-11-26T07:18:10.998 INFO:teuthology.orchestra.run.smithi143.stdout: inet 172.17.0.1/16 brd 172.17.255.255 scope global docker0 2023-11-26T07:18:10.998 INFO:teuthology.orchestra.run.smithi143.stdout: valid_lft forever preferred_lft forever 2023-11-26T07:18:10.998 INFO:teuthology.orchestra.run.smithi143.stdout:7: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2023-11-26T07:18:10.998 INFO:teuthology.orchestra.run.smithi143.stdout: link/ether 5e:1f:84:6d:79:ae brd ff:ff:ff:ff:ff:ff 2023-11-26T07:18:10.998 INFO:teuthology.orchestra.run.smithi143.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2023-11-26T07:18:10.999 INFO:teuthology.orchestra.run.smithi143.stdout: valid_lft forever preferred_lft forever 2023-11-26T07:18:10.999 INFO:teuthology.orchestra.run.smithi143.stdout: inet6 fe80::5c1f:84ff:fe6d:79ae/64 scope link tentative 2023-11-26T07:18:10.999 INFO:teuthology.orchestra.run.smithi143.stdout: valid_lft forever preferred_lft forever 2023-11-26T07:18:10.999 INFO:teuthology.orchestra.run.smithi143.stdout:8: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2023-11-26T07:18:10.999 INFO:teuthology.orchestra.run.smithi143.stdout: link/ether 0e:5c:f6:1f:82:2f brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.0 2023-11-26T07:18:10.999 INFO:teuthology.orchestra.run.smithi143.stdout: inet6 fe80::c5c:f6ff:fe1f:822f/64 scope link tentative 2023-11-26T07:18:10.999 INFO:teuthology.orchestra.run.smithi143.stdout: valid_lft forever preferred_lft forever 2023-11-26T07:18:10.999 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-26T07:18:10.999 DEBUG:teuthology.orchestra.run.smithi143:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2023-11-26T07:18:11.059 INFO:teuthology.orchestra.run.smithi143.stdout:1 2023-11-26T07:18:11.061 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-26T07:18:11.061 DEBUG:teuthology.orchestra.run.smithi143:> route 2023-11-26T07:18:11.119 INFO:teuthology.orchestra.run.smithi143.stdout:Kernel IP routing table 2023-11-26T07:18:11.119 INFO:teuthology.orchestra.run.smithi143.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2023-11-26T07:18:11.119 INFO:teuthology.orchestra.run.smithi143.stdout:default _gateway 0.0.0.0 UG 0 0 0 enp3s0f1 2023-11-26T07:18:11.119 INFO:teuthology.orchestra.run.smithi143.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2023-11-26T07:18:11.119 INFO:teuthology.orchestra.run.smithi143.stdout:172.17.0.0 0.0.0.0 255.255.0.0 U 0 0 0 docker0 2023-11-26T07:18:11.120 INFO:teuthology.orchestra.run.smithi143.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 enp3s0f1 2023-11-26T07:18:11.120 INFO:teuthology.orchestra.run.smithi143.stdout:ns1.front.sepia 0.0.0.0 255.255.255.255 UH 100 0 0 enp3s0f1 2023-11-26T07:18:11.120 INFO:teuthology.orchestra.run.smithi143.stdout:172.21.0.2 0.0.0.0 255.255.255.255 UH 100 0 0 enp3s0f1 2023-11-26T07:18:11.120 INFO:teuthology.orchestra.run.smithi143.stdout:_gateway 0.0.0.0 255.255.255.255 UH 100 0 0 enp3s0f1 2023-11-26T07:18:11.120 INFO:teuthology.orchestra.run.smithi143.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2023-11-26T07:18:11.120 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-26T07:18:11.120 DEBUG:teuthology.orchestra.run.smithi143:> (cd / && exec bash -c ' 2023-11-26T07:18:11.120 DEBUG:teuthology.orchestra.run.smithi143:> set -e 2023-11-26T07:18:11.120 DEBUG:teuthology.orchestra.run.smithi143:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2023-11-26T07:18:11.120 DEBUG:teuthology.orchestra.run.smithi143:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2023-11-26T07:18:11.121 DEBUG:teuthology.orchestra.run.smithi143:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2023-11-26T07:18:11.121 DEBUG:teuthology.orchestra.run.smithi143:> ') 2023-11-26T07:18:11.214 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-26T07:18:11.215 DEBUG:teuthology.orchestra.run.smithi143:> ip netns list 2023-11-26T07:18:11.223 INFO:teuthology.orchestra.run.smithi143.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2023-11-26T07:18:11.224 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-26T07:18:11.224 DEBUG:teuthology.orchestra.run.smithi143:> ip netns list-id 2023-11-26T07:18:11.276 INFO:teuthology.orchestra.run.smithi143.stdout:nsid 0 (iproute2 netns name: ceph-ns--home-ubuntu-cephtest-mnt.0) 2023-11-26T07:18:11.277 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-26T07:18:11.277 DEBUG:teuthology.orchestra.run.smithi143:> (cd / && exec bash -c ' 2023-11-26T07:18:11.277 DEBUG:teuthology.orchestra.run.smithi143:> set -e 2023-11-26T07:18:11.277 DEBUG:teuthology.orchestra.run.smithi143:> sudo ip netns add ceph-ns--home-ubuntu-cephtest-mnt.2 2023-11-26T07:18:11.278 DEBUG:teuthology.orchestra.run.smithi143:> sudo ip netns set ceph-ns--home-ubuntu-cephtest-mnt.2 1 2023-11-26T07:18:11.278 DEBUG:teuthology.orchestra.run.smithi143:> ') 2023-11-26T07:18:11.360 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-26T07:18:11.360 DEBUG:teuthology.orchestra.run.smithi143:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip addr 2023-11-26T07:18:11.425 INFO:teuthology.orchestra.run.smithi143.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2023-11-26T07:18:11.425 INFO:teuthology.orchestra.run.smithi143.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2023-11-26T07:18:11.426 INFO:teuthology.orchestra.run.smithi143.stdout: inet 127.0.0.1/8 scope host lo 2023-11-26T07:18:11.426 INFO:teuthology.orchestra.run.smithi143.stdout: valid_lft forever preferred_lft forever 2023-11-26T07:18:11.426 INFO:teuthology.orchestra.run.smithi143.stdout: inet6 ::1/128 scope host 2023-11-26T07:18:11.426 INFO:teuthology.orchestra.run.smithi143.stdout: valid_lft forever preferred_lft forever 2023-11-26T07:18:11.426 INFO:teuthology.orchestra.run.smithi143.stdout:2: veth0@if8: mtu 1500 qdisc noqueue state UP group default qlen 1000 2023-11-26T07:18:11.426 INFO:teuthology.orchestra.run.smithi143.stdout: link/ether c2:3b:2a:64:79:73 brd ff:ff:ff:ff:ff:ff link-netnsid 0 2023-11-26T07:18:11.426 INFO:teuthology.orchestra.run.smithi143.stdout: inet 192.168.0.1/16 brd 192.168.255.255 scope global veth0 2023-11-26T07:18:11.426 INFO:teuthology.orchestra.run.smithi143.stdout: valid_lft forever preferred_lft forever 2023-11-26T07:18:11.426 INFO:teuthology.orchestra.run.smithi143.stdout: inet6 fe80::c03b:2aff:fe64:7973/64 scope link 2023-11-26T07:18:11.426 INFO:teuthology.orchestra.run.smithi143.stdout: valid_lft forever preferred_lft forever 2023-11-26T07:18:11.427 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-26T07:18:11.427 DEBUG:teuthology.orchestra.run.smithi143:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip addr 2023-11-26T07:18:11.493 INFO:teuthology.orchestra.run.smithi143.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2023-11-26T07:18:11.493 INFO:teuthology.orchestra.run.smithi143.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2023-11-26T07:18:11.493 INFO:teuthology.orchestra.run.smithi143.stdout: inet 127.0.0.1/8 scope host lo 2023-11-26T07:18:11.493 INFO:teuthology.orchestra.run.smithi143.stdout: valid_lft forever preferred_lft forever 2023-11-26T07:18:11.493 INFO:teuthology.orchestra.run.smithi143.stdout: inet6 ::1/128 scope host 2023-11-26T07:18:11.493 INFO:teuthology.orchestra.run.smithi143.stdout: valid_lft forever preferred_lft forever 2023-11-26T07:18:11.493 INFO:teuthology.orchestra.run.smithi143.stdout:2: veth0@if8: mtu 1500 qdisc noqueue state UP group default qlen 1000 2023-11-26T07:18:11.493 INFO:teuthology.orchestra.run.smithi143.stdout: link/ether c2:3b:2a:64:79:73 brd ff:ff:ff:ff:ff:ff link-netnsid 0 2023-11-26T07:18:11.493 INFO:teuthology.orchestra.run.smithi143.stdout: inet 192.168.0.1/16 brd 192.168.255.255 scope global veth0 2023-11-26T07:18:11.493 INFO:teuthology.orchestra.run.smithi143.stdout: valid_lft forever preferred_lft forever 2023-11-26T07:18:11.494 INFO:teuthology.orchestra.run.smithi143.stdout: inet6 fe80::c03b:2aff:fe64:7973/64 scope link 2023-11-26T07:18:11.494 INFO:teuthology.orchestra.run.smithi143.stdout: valid_lft forever preferred_lft forever 2023-11-26T07:18:11.495 INFO:tasks.cephfs.mount:Setuping the netns 'ceph-ns--home-ubuntu-cephtest-mnt.2' with 192.168.0.2/16 2023-11-26T07:18:11.495 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-26T07:18:11.495 DEBUG:teuthology.orchestra.run.smithi143:> (cd / && exec bash -c ' 2023-11-26T07:18:11.495 DEBUG:teuthology.orchestra.run.smithi143:> set -e 2023-11-26T07:18:11.495 DEBUG:teuthology.orchestra.run.smithi143:> sudo ip link add veth0 netns ceph-ns--home-ubuntu-cephtest-mnt.2 type veth peer name brx.1 2023-11-26T07:18:11.495 DEBUG:teuthology.orchestra.run.smithi143:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.2 ip addr add 192.168.0.2/16 brd 192.168.255.255 dev veth0 2023-11-26T07:18:11.495 DEBUG:teuthology.orchestra.run.smithi143:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.2 ip link set veth0 up 2023-11-26T07:18:11.496 DEBUG:teuthology.orchestra.run.smithi143:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.2 ip link set lo up 2023-11-26T07:18:11.496 DEBUG:teuthology.orchestra.run.smithi143:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.2 ip route add default via 192.168.255.254 2023-11-26T07:18:11.496 DEBUG:teuthology.orchestra.run.smithi143:> ') 2023-11-26T07:18:11.633 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-26T07:18:11.634 DEBUG:teuthology.orchestra.run.smithi143:> (cd / && exec bash -c ' 2023-11-26T07:18:11.634 DEBUG:teuthology.orchestra.run.smithi143:> set -e 2023-11-26T07:18:11.634 DEBUG:teuthology.orchestra.run.smithi143:> sudo ip link set brx.1 up 2023-11-26T07:18:11.634 DEBUG:teuthology.orchestra.run.smithi143:> sudo ip link set dev brx.1 master ceph-brx 2023-11-26T07:18:11.634 DEBUG:teuthology.orchestra.run.smithi143:> ') 2023-11-26T07:18:11.673 INFO:tasks.cephfs.fuse_mount:Client client.2 config is {} 2023-11-26T07:18:11.673 INFO:teuthology.orchestra.run:Running command with timeout 60 2023-11-26T07:18:11.673 DEBUG:teuthology.orchestra.run.smithi143:> mkdir -p -v /home/ubuntu/cephtest/mnt.2 2023-11-26T07:18:11.729 INFO:teuthology.orchestra.run.smithi143.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.2' 2023-11-26T07:18:11.729 INFO:teuthology.orchestra.run:Running command with timeout 60 2023-11-26T07:18:11.729 DEBUG:teuthology.orchestra.run.smithi143:> chmod 0000 /home/ubuntu/cephtest/mnt.2 2023-11-26T07:18:11.780 DEBUG:teuthology.orchestra.run.smithi143:> sudo modprobe fuse 2023-11-26T07:18:11.841 DEBUG:teuthology.orchestra.run.smithi143:> cat /proc/self/mounts | awk '{print $2}' 2023-11-26T07:18:11.894 INFO:teuthology.orchestra.run.smithi143.stdout:/sys 2023-11-26T07:18:11.894 INFO:teuthology.orchestra.run.smithi143.stdout:/proc 2023-11-26T07:18:11.894 INFO:teuthology.orchestra.run.smithi143.stdout:/dev 2023-11-26T07:18:11.894 INFO:teuthology.orchestra.run.smithi143.stdout:/dev/pts 2023-11-26T07:18:11.894 INFO:teuthology.orchestra.run.smithi143.stdout:/run 2023-11-26T07:18:11.894 INFO:teuthology.orchestra.run.smithi143.stdout:/ 2023-11-26T07:18:11.894 INFO:teuthology.orchestra.run.smithi143.stdout:/sys/kernel/security 2023-11-26T07:18:11.895 INFO:teuthology.orchestra.run.smithi143.stdout:/dev/shm 2023-11-26T07:18:11.895 INFO:teuthology.orchestra.run.smithi143.stdout:/run/lock 2023-11-26T07:18:11.895 INFO:teuthology.orchestra.run.smithi143.stdout:/sys/fs/cgroup 2023-11-26T07:18:11.895 INFO:teuthology.orchestra.run.smithi143.stdout:/sys/fs/pstore 2023-11-26T07:18:11.895 INFO:teuthology.orchestra.run.smithi143.stdout:/sys/fs/bpf 2023-11-26T07:18:11.895 INFO:teuthology.orchestra.run.smithi143.stdout:/proc/sys/fs/binfmt_misc 2023-11-26T07:18:11.895 INFO:teuthology.orchestra.run.smithi143.stdout:/dev/hugepages 2023-11-26T07:18:11.895 INFO:teuthology.orchestra.run.smithi143.stdout:/dev/mqueue 2023-11-26T07:18:11.895 INFO:teuthology.orchestra.run.smithi143.stdout:/sys/kernel/debug 2023-11-26T07:18:11.895 INFO:teuthology.orchestra.run.smithi143.stdout:/sys/kernel/tracing 2023-11-26T07:18:11.895 INFO:teuthology.orchestra.run.smithi143.stdout:/proc/fs/nfsd 2023-11-26T07:18:11.895 INFO:teuthology.orchestra.run.smithi143.stdout:/sys/fs/fuse/connections 2023-11-26T07:18:11.896 INFO:teuthology.orchestra.run.smithi143.stdout:/sys/kernel/config 2023-11-26T07:18:11.896 INFO:teuthology.orchestra.run.smithi143.stdout:/run/credentials/systemd-sysusers.service 2023-11-26T07:18:11.896 INFO:teuthology.orchestra.run.smithi143.stdout:/run/rpc_pipefs 2023-11-26T07:18:11.896 INFO:teuthology.orchestra.run.smithi143.stdout:/run/user/1000 2023-11-26T07:18:11.896 INFO:teuthology.orchestra.run.smithi143.stdout:/var/lib/ceph 2023-11-26T07:18:11.896 INFO:teuthology.orchestra.run.smithi143.stdout:/var/lib/ceph/osd/ceph-0 2023-11-26T07:18:11.896 INFO:teuthology.orchestra.run.smithi143.stdout:/var/lib/ceph/osd/ceph-1 2023-11-26T07:18:11.896 INFO:teuthology.orchestra.run.smithi143.stdout:/var/lib/ceph/osd/ceph-2 2023-11-26T07:18:11.896 INFO:teuthology.orchestra.run.smithi143.stdout:/var/lib/ceph/osd/ceph-3 2023-11-26T07:18:11.896 INFO:teuthology.orchestra.run.smithi143.stdout:/run/netns 2023-11-26T07:18:11.896 INFO:teuthology.orchestra.run.smithi143.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2023-11-26T07:18:11.896 INFO:teuthology.orchestra.run.smithi143.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2023-11-26T07:18:11.896 INFO:teuthology.orchestra.run.smithi143.stdout:/home/ubuntu/cephtest/mnt.0 2023-11-26T07:18:11.897 INFO:teuthology.orchestra.run.smithi143.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 2023-11-26T07:18:11.897 INFO:teuthology.orchestra.run.smithi143.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 2023-11-26T07:18:11.897 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-26T07:18:11.898 DEBUG:teuthology.orchestra.run.smithi143:> ls /sys/fs/fuse/connections 2023-11-26T07:18:11.948 INFO:teuthology.orchestra.run.smithi143.stdout:43 2023-11-26T07:18:11.948 INFO:tasks.cephfs.fuse_mount:Pre-mount connections: [43] 2023-11-26T07:18:11.949 DEBUG:teuthology.orchestra.run.smithi143:> (cd /home/ubuntu/cephtest && exec sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-fuse -f --admin-socket '/var/run/ceph/$cluster-$name.$pid.asok' /home/ubuntu/cephtest/mnt.2 --id 2) 2023-11-26T07:18:11.995 DEBUG:teuthology.orchestra.run.smithi143:> sudo modprobe fuse 2023-11-26T07:18:12.016 DEBUG:teuthology.orchestra.run.smithi143:> cat /proc/self/mounts | awk '{print $2}' 2023-11-26T07:18:12.072 INFO:teuthology.orchestra.run.smithi143.stdout:/sys 2023-11-26T07:18:12.072 INFO:teuthology.orchestra.run.smithi143.stdout:/proc 2023-11-26T07:18:12.072 INFO:teuthology.orchestra.run.smithi143.stdout:/dev 2023-11-26T07:18:12.072 INFO:teuthology.orchestra.run.smithi143.stdout:/dev/pts 2023-11-26T07:18:12.072 INFO:teuthology.orchestra.run.smithi143.stdout:/run 2023-11-26T07:18:12.072 INFO:teuthology.orchestra.run.smithi143.stdout:/ 2023-11-26T07:18:12.072 INFO:teuthology.orchestra.run.smithi143.stdout:/sys/kernel/security 2023-11-26T07:18:12.072 INFO:teuthology.orchestra.run.smithi143.stdout:/dev/shm 2023-11-26T07:18:12.072 INFO:teuthology.orchestra.run.smithi143.stdout:/run/lock 2023-11-26T07:18:12.073 INFO:teuthology.orchestra.run.smithi143.stdout:/sys/fs/cgroup 2023-11-26T07:18:12.073 INFO:teuthology.orchestra.run.smithi143.stdout:/sys/fs/pstore 2023-11-26T07:18:12.073 INFO:teuthology.orchestra.run.smithi143.stdout:/sys/fs/bpf 2023-11-26T07:18:12.073 INFO:teuthology.orchestra.run.smithi143.stdout:/proc/sys/fs/binfmt_misc 2023-11-26T07:18:12.073 INFO:teuthology.orchestra.run.smithi143.stdout:/dev/hugepages 2023-11-26T07:18:12.073 INFO:teuthology.orchestra.run.smithi143.stdout:/dev/mqueue 2023-11-26T07:18:12.073 INFO:teuthology.orchestra.run.smithi143.stdout:/sys/kernel/debug 2023-11-26T07:18:12.073 INFO:teuthology.orchestra.run.smithi143.stdout:/sys/kernel/tracing 2023-11-26T07:18:12.073 INFO:teuthology.orchestra.run.smithi143.stdout:/proc/fs/nfsd 2023-11-26T07:18:12.073 INFO:teuthology.orchestra.run.smithi143.stdout:/sys/fs/fuse/connections 2023-11-26T07:18:12.073 INFO:teuthology.orchestra.run.smithi143.stdout:/sys/kernel/config 2023-11-26T07:18:12.073 INFO:teuthology.orchestra.run.smithi143.stdout:/run/credentials/systemd-sysusers.service 2023-11-26T07:18:12.073 INFO:teuthology.orchestra.run.smithi143.stdout:/run/rpc_pipefs 2023-11-26T07:18:12.074 INFO:teuthology.orchestra.run.smithi143.stdout:/run/user/1000 2023-11-26T07:18:12.074 INFO:teuthology.orchestra.run.smithi143.stdout:/var/lib/ceph 2023-11-26T07:18:12.074 INFO:teuthology.orchestra.run.smithi143.stdout:/var/lib/ceph/osd/ceph-0 2023-11-26T07:18:12.074 INFO:teuthology.orchestra.run.smithi143.stdout:/var/lib/ceph/osd/ceph-1 2023-11-26T07:18:12.074 INFO:teuthology.orchestra.run.smithi143.stdout:/var/lib/ceph/osd/ceph-2 2023-11-26T07:18:12.074 INFO:teuthology.orchestra.run.smithi143.stdout:/var/lib/ceph/osd/ceph-3 2023-11-26T07:18:12.074 INFO:teuthology.orchestra.run.smithi143.stdout:/run/netns 2023-11-26T07:18:12.074 INFO:teuthology.orchestra.run.smithi143.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2023-11-26T07:18:12.074 INFO:teuthology.orchestra.run.smithi143.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2023-11-26T07:18:12.074 INFO:teuthology.orchestra.run.smithi143.stdout:/home/ubuntu/cephtest/mnt.0 2023-11-26T07:18:12.074 INFO:teuthology.orchestra.run.smithi143.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 2023-11-26T07:18:12.074 INFO:teuthology.orchestra.run.smithi143.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 2023-11-26T07:18:12.075 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-26T07:18:12.075 DEBUG:teuthology.orchestra.run.smithi143:> ls /sys/fs/fuse/connections 2023-11-26T07:18:12.092 INFO:tasks.cephfs.fuse_mount.ceph-fuse.2.smithi143.stderr:2023-11-26T07:18:12.086+0000 7f31454ca580 -1 init, newargv = 0x563891bab140 newargc=15 2023-11-26T07:18:12.092 INFO:tasks.cephfs.fuse_mount.ceph-fuse.2.smithi143.stderr:ceph-fuse[45979]: starting ceph client 2023-11-26T07:18:12.100 INFO:tasks.cephfs.fuse_mount.ceph-fuse.2.smithi143.stderr:ceph-fuse[45979]: starting fuse 2023-11-26T07:18:12.101 INFO:teuthology.orchestra.run.smithi143.stdout:43 2023-11-26T07:18:12.101 INFO:teuthology.orchestra.run.smithi143.stdout:44 2023-11-26T07:18:12.101 INFO:tasks.cephfs.fuse_mount:Post-mount connections: [43, 44] 2023-11-26T07:18:12.101 DEBUG:teuthology.orchestra.run.smithi143:> sudo stdin-killer -- python3 -c ' 2023-11-26T07:18:12.101 DEBUG:teuthology.orchestra.run.smithi143:> import glob 2023-11-26T07:18:12.101 DEBUG:teuthology.orchestra.run.smithi143:> import re 2023-11-26T07:18:12.101 DEBUG:teuthology.orchestra.run.smithi143:> import os 2023-11-26T07:18:12.102 DEBUG:teuthology.orchestra.run.smithi143:> import subprocess 2023-11-26T07:18:12.102 DEBUG:teuthology.orchestra.run.smithi143:> 2023-11-26T07:18:12.102 DEBUG:teuthology.orchestra.run.smithi143:> def _find_admin_socket(client_name): 2023-11-26T07:18:12.102 DEBUG:teuthology.orchestra.run.smithi143:> asok_path = "/var/run/ceph/ceph-client.2.*.asok" 2023-11-26T07:18:12.102 DEBUG:teuthology.orchestra.run.smithi143:> files = glob.glob(asok_path) 2023-11-26T07:18:12.102 DEBUG:teuthology.orchestra.run.smithi143:> mountpoint = "/home/ubuntu/cephtest/mnt.2" 2023-11-26T07:18:12.102 DEBUG:teuthology.orchestra.run.smithi143:> 2023-11-26T07:18:12.102 DEBUG:teuthology.orchestra.run.smithi143:> # Given a non-glob path, it better be there 2023-11-26T07:18:12.102 DEBUG:teuthology.orchestra.run.smithi143:> if "*" not in asok_path: 2023-11-26T07:18:12.102 DEBUG:teuthology.orchestra.run.smithi143:> assert(len(files) == 1) 2023-11-26T07:18:12.102 DEBUG:teuthology.orchestra.run.smithi143:> return files[0] 2023-11-26T07:18:12.102 DEBUG:teuthology.orchestra.run.smithi143:> 2023-11-26T07:18:12.102 DEBUG:teuthology.orchestra.run.smithi143:> for f in files: 2023-11-26T07:18:12.102 DEBUG:teuthology.orchestra.run.smithi143:> pid = re.match(".*\.(\d+)\.asok$", f).group(1) 2023-11-26T07:18:12.103 DEBUG:teuthology.orchestra.run.smithi143:> if os.path.exists("/proc/{0}".format(pid)): 2023-11-26T07:18:12.103 DEBUG:teuthology.orchestra.run.smithi143:> with open("/proc/{0}/cmdline".format(pid), '"'"'r'"'"') as proc_f: 2023-11-26T07:18:12.103 DEBUG:teuthology.orchestra.run.smithi143:> contents = proc_f.read() 2023-11-26T07:18:12.103 DEBUG:teuthology.orchestra.run.smithi143:> if mountpoint in contents: 2023-11-26T07:18:12.103 DEBUG:teuthology.orchestra.run.smithi143:> return f 2023-11-26T07:18:12.103 DEBUG:teuthology.orchestra.run.smithi143:> raise RuntimeError("Client socket {0} not found".format(client_name)) 2023-11-26T07:18:12.103 DEBUG:teuthology.orchestra.run.smithi143:> 2023-11-26T07:18:12.103 DEBUG:teuthology.orchestra.run.smithi143:> print(_find_admin_socket("client.2")) 2023-11-26T07:18:12.103 DEBUG:teuthology.orchestra.run.smithi143:> ' 2023-11-26T07:18:12.230 INFO:teuthology.orchestra.run.smithi143.stdout:/var/run/ceph/ceph-client.2.45979.asok 2023-11-26T07:18:12.233 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:12 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2023-11-26T07:18:12.241 INFO:tasks.cephfs.fuse_mount:Found client admin socket at /var/run/ceph/ceph-client.2.45979.asok 2023-11-26T07:18:12.241 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-26T07:18:12.241 DEBUG:teuthology.orchestra.run.smithi143:> sudo ceph --admin-daemon /var/run/ceph/ceph-client.2.45979.asok status 2023-11-26T07:18:12.415 INFO:teuthology.orchestra.run.smithi143.stdout:{ 2023-11-26T07:18:12.416 INFO:teuthology.orchestra.run.smithi143.stdout: "metadata": { 2023-11-26T07:18:12.416 INFO:teuthology.orchestra.run.smithi143.stdout: "ceph_sha1": "44c2237fe77713214148c1fd622c872f346f1b99", 2023-11-26T07:18:12.416 INFO:teuthology.orchestra.run.smithi143.stdout: "ceph_version": "ceph version 18.0.0-7484-g44c2237f (44c2237fe77713214148c1fd622c872f346f1b99) reef (dev)", 2023-11-26T07:18:12.416 INFO:teuthology.orchestra.run.smithi143.stdout: "entity_id": "2", 2023-11-26T07:18:12.416 INFO:teuthology.orchestra.run.smithi143.stdout: "hostname": "smithi143", 2023-11-26T07:18:12.416 INFO:teuthology.orchestra.run.smithi143.stdout: "mount_point": "/home/ubuntu/cephtest/mnt.2", 2023-11-26T07:18:12.416 INFO:teuthology.orchestra.run.smithi143.stdout: "pid": "45979", 2023-11-26T07:18:12.416 INFO:teuthology.orchestra.run.smithi143.stdout: "root": "/" 2023-11-26T07:18:12.416 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2023-11-26T07:18:12.416 INFO:teuthology.orchestra.run.smithi143.stdout: "dentry_count": 0, 2023-11-26T07:18:12.416 INFO:teuthology.orchestra.run.smithi143.stdout: "dentry_pinned_count": 0, 2023-11-26T07:18:12.416 INFO:teuthology.orchestra.run.smithi143.stdout: "id": 4669, 2023-11-26T07:18:12.417 INFO:teuthology.orchestra.run.smithi143.stdout: "inst": { 2023-11-26T07:18:12.417 INFO:teuthology.orchestra.run.smithi143.stdout: "name": { 2023-11-26T07:18:12.417 INFO:teuthology.orchestra.run.smithi143.stdout: "type": "client", 2023-11-26T07:18:12.417 INFO:teuthology.orchestra.run.smithi143.stdout: "num": 4669 2023-11-26T07:18:12.417 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2023-11-26T07:18:12.417 INFO:teuthology.orchestra.run.smithi143.stdout: "addr": { 2023-11-26T07:18:12.417 INFO:teuthology.orchestra.run.smithi143.stdout: "type": "v1", 2023-11-26T07:18:12.417 INFO:teuthology.orchestra.run.smithi143.stdout: "addr": "172.21.15.143:0", 2023-11-26T07:18:12.417 INFO:teuthology.orchestra.run.smithi143.stdout: "nonce": 1577226588 2023-11-26T07:18:12.417 INFO:teuthology.orchestra.run.smithi143.stdout: } 2023-11-26T07:18:12.417 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2023-11-26T07:18:12.417 INFO:teuthology.orchestra.run.smithi143.stdout: "addr": { 2023-11-26T07:18:12.418 INFO:teuthology.orchestra.run.smithi143.stdout: "type": "v1", 2023-11-26T07:18:12.418 INFO:teuthology.orchestra.run.smithi143.stdout: "addr": "172.21.15.143:0", 2023-11-26T07:18:12.418 INFO:teuthology.orchestra.run.smithi143.stdout: "nonce": 1577226588 2023-11-26T07:18:12.418 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2023-11-26T07:18:12.418 INFO:teuthology.orchestra.run.smithi143.stdout: "inst_str": "client.4669 172.21.15.143:0/1577226588", 2023-11-26T07:18:12.418 INFO:teuthology.orchestra.run.smithi143.stdout: "addr_str": "172.21.15.143:0/1577226588", 2023-11-26T07:18:12.418 INFO:teuthology.orchestra.run.smithi143.stdout: "inode_count": 1, 2023-11-26T07:18:12.418 INFO:teuthology.orchestra.run.smithi143.stdout: "mds_epoch": 8, 2023-11-26T07:18:12.418 INFO:teuthology.orchestra.run.smithi143.stdout: "osd_epoch": 20, 2023-11-26T07:18:12.418 INFO:teuthology.orchestra.run.smithi143.stdout: "osd_epoch_barrier": 0, 2023-11-26T07:18:12.418 INFO:teuthology.orchestra.run.smithi143.stdout: "blocklisted": false, 2023-11-26T07:18:12.418 INFO:teuthology.orchestra.run.smithi143.stdout: "fs_name": "cephfs" 2023-11-26T07:18:12.418 INFO:teuthology.orchestra.run.smithi143.stdout:} 2023-11-26T07:18:12.430 DEBUG:teuthology.orchestra.run.smithi143:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2023-11-26T07:18:12.567 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:12.562+0000 7f29cc72a640 1 Processor -- start 2023-11-26T07:18:12.567 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:12.562+0000 7f29cc72a640 1 -- start start 2023-11-26T07:18:12.569 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:12.562+0000 7f29cc72a640 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7f29c40cf280 0x7f29c40cf710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:18:12.569 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:12.562+0000 7f29cc72a640 1 -- --> v1:172.21.15.143:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f29c40cc730 con 0x7f29c40cd1b0 2023-11-26T07:18:12.569 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:12.562+0000 7f29cc72a640 1 -- --> v1:172.21.15.188:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f29c4070ee0 con 0x7f29c4143ac0 2023-11-26T07:18:12.569 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:12.562+0000 7f29cc72a640 1 -- --> v2:172.21.15.188:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f29c4143ea0 con 0x7f29c40cf280 2023-11-26T07:18:12.569 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:12.566+0000 7f29cacc7640 1 --1- >> v1:172.21.15.143:6789/0 conn(0x7f29c40cd1b0 0x7f29c40cd5c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.143:6789/0 says I am v1:172.21.15.143:52258/0 (socket says 172.21.15.143:52258) 2023-11-26T07:18:12.569 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:12.566+0000 7f29cacc7640 1 -- 172.21.15.143:0/1536384983 learned_addr learned my addr 172.21.15.143:0/1536384983 (peer_addr_for_me v1:172.21.15.143:0/0) 2023-11-26T07:18:12.569 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:12.566+0000 7f29c9cc5640 1 --2- 172.21.15.143:0/1536384983 >> v2:172.21.15.188:3300/0 conn(0x7f29c40cf280 0x7f29c40cf710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:18:12.570 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:12.566+0000 7f29c94c4640 1 -- 172.21.15.143:0/1536384983 <== mon.0 v1:172.21.15.143:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1125906511 0 0) 0x7f29c40cc730 con 0x7f29c40cd1b0 2023-11-26T07:18:12.570 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:12.566+0000 7f29c94c4640 1 -- 172.21.15.143:0/1536384983 --> v1:172.21.15.143:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f29b0003530 con 0x7f29c40cd1b0 2023-11-26T07:18:12.570 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:12.566+0000 7f29c94c4640 1 -- 172.21.15.143:0/1536384983 <== mon.1 v1:172.21.15.188:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3861710630 0 0) 0x7f29c4070ee0 con 0x7f29c4143ac0 2023-11-26T07:18:12.570 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:12.566+0000 7f29c94c4640 1 -- 172.21.15.143:0/1536384983 --> v1:172.21.15.188:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f29c40cc730 con 0x7f29c4143ac0 2023-11-26T07:18:12.570 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:12.566+0000 7f29c9cc5640 1 -- 172.21.15.143:0/1536384983 >> v1:172.21.15.188:6789/0 conn(0x7f29c4143ac0 legacy=0x7f29c4070370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:18:12.570 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:12.566+0000 7f29c9cc5640 1 -- 172.21.15.143:0/1536384983 >> v1:172.21.15.143:6789/0 conn(0x7f29c40cd1b0 legacy=0x7f29c40cd5c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:18:12.570 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:12.566+0000 7f29c9cc5640 1 -- 172.21.15.143:0/1536384983 --> v2:172.21.15.188:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f29c40711c0 con 0x7f29c40cf280 2023-11-26T07:18:12.572 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:12.566+0000 7f29c9cc5640 1 --2- 172.21.15.143:0/1536384983 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f29c40cf280 0x7f29c40cf710 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7f29b40097c0 tx=0x7f29b401c480 comp rx=0 tx=0).ready entity=mon.1 client_cookie=61c28fe3d1e0aec8 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:18:12.572 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:12.566+0000 7f29c94c4640 1 -- 172.21.15.143:0/1536384983 <== mon.1 v2:172.21.15.188:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f29b40a8050 con 0x7f29c40cf280 2023-11-26T07:18:12.572 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:12.566+0000 7f29c94c4640 1 -- 172.21.15.143:0/1536384983 <== mon.1 v2:172.21.15.188:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f29b40ac5f0 con 0x7f29c40cf280 2023-11-26T07:18:12.573 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:12.566+0000 7f29c94c4640 1 -- 172.21.15.143:0/1536384983 <== mon.1 v2:172.21.15.188:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f29b40ac980 con 0x7f29c40cf280 2023-11-26T07:18:12.573 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:12.570+0000 7f29cc72a640 1 -- 172.21.15.143:0/1536384983 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f29c40cf280 msgr2=0x7f29c40cf710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:18:12.573 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:12.570+0000 7f29cc72a640 1 --2- 172.21.15.143:0/1536384983 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f29c40cf280 0x7f29c40cf710 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7f29b40097c0 tx=0x7f29b401c480 comp rx=0 tx=0).stop 2023-11-26T07:18:12.573 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:12.570+0000 7f29cc72a640 1 -- 172.21.15.143:0/1536384983 shutdown_connections 2023-11-26T07:18:12.574 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:12.570+0000 7f29cc72a640 1 --2- 172.21.15.143:0/1536384983 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f29c40cf280 0x7f29c40cf710 unknown :-1 s=CLOSED pgs=107 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:18:12.574 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:12.570+0000 7f29cc72a640 1 -- 172.21.15.143:0/1536384983 >> 172.21.15.143:0/1536384983 conn(0x7f29c406a890 msgr2=0x7f29c406aca0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:18:12.574 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:12.570+0000 7f29cc72a640 1 -- 172.21.15.143:0/1536384983 shutdown_connections 2023-11-26T07:18:12.574 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:12.570+0000 7f29cc72a640 1 -- 172.21.15.143:0/1536384983 wait complete. 2023-11-26T07:18:12.575 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:12.570+0000 7f29cc72a640 1 Processor -- start 2023-11-26T07:18:12.575 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:12.570+0000 7f29cc72a640 1 -- start start 2023-11-26T07:18:12.577 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:12.570+0000 7f29cc72a640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f29c40cd1b0 0x7f29c4156820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:18:12.577 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:12.570+0000 7f29cc72a640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f29c40cf280 0x7f29c41529a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:18:12.577 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:12.570+0000 7f29cc72a640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f29c4143ac0 0x7f29c4152ee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:18:12.578 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:12.570+0000 7f29cc72a640 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f29c4157000 con 0x7f29c4143ac0 2023-11-26T07:18:12.578 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:12.570+0000 7f29cc72a640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f29c40b1df0 con 0x7f29c40cf280 2023-11-26T07:18:12.578 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:12.570+0000 7f29cc72a640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f29c41545c0 con 0x7f29c40cd1b0 2023-11-26T07:18:12.578 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:12.570+0000 7f29cacc7640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f29c4143ac0 0x7f29c4152ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:18:12.578 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:12.570+0000 7f29cacc7640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f29c4143ac0 0x7f29c4152ee0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:48396/0 (socket says 172.21.15.143:48396) 2023-11-26T07:18:12.578 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:12.570+0000 7f29cacc7640 1 -- 172.21.15.143:0/3305562374 learned_addr learned my addr 172.21.15.143:0/3305562374 (peer_addr_for_me v2:172.21.15.143:0/0) 2023-11-26T07:18:12.578 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:12.570+0000 7f29c9cc5640 1 --2- 172.21.15.143:0/3305562374 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f29c40cf280 0x7f29c41529a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:18:12.578 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:12.570+0000 7f29ca4c6640 1 --2- 172.21.15.143:0/3305562374 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f29c40cd1b0 0x7f29c4156820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:18:12.578 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:12.574+0000 7f29cacc7640 1 -- 172.21.15.143:0/3305562374 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f29c40cd1b0 msgr2=0x7f29c4156820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:18:12.578 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:12.574+0000 7f29cacc7640 1 --2- 172.21.15.143:0/3305562374 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f29c40cd1b0 0x7f29c4156820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:18:12.578 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:12.574+0000 7f29cacc7640 1 -- 172.21.15.143:0/3305562374 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f29c40cf280 msgr2=0x7f29c41529a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:18:12.578 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:12.574+0000 7f29cacc7640 1 --2- 172.21.15.143:0/3305562374 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f29c40cf280 0x7f29c41529a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:18:12.579 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:12.574+0000 7f29cacc7640 1 -- 172.21.15.143:0/3305562374 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f29b4009420 con 0x7f29c4143ac0 2023-11-26T07:18:12.579 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:12.574+0000 7f29cacc7640 1 --2- 172.21.15.143:0/3305562374 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f29c4143ac0 0x7f29c4152ee0 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7f29bc00cba0 tx=0x7f29bc00a430 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:18:12.579 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:12.574+0000 7f29baffd640 1 -- 172.21.15.143:0/3305562374 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f29bc00ace0 con 0x7f29c4143ac0 2023-11-26T07:18:12.579 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:12.574+0000 7f29baffd640 1 -- 172.21.15.143:0/3305562374 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f29bc00ae40 con 0x7f29c4143ac0 2023-11-26T07:18:12.579 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:12.574+0000 7f29cc72a640 1 -- 172.21.15.143:0/3305562374 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f29c4156dc0 con 0x7f29c4143ac0 2023-11-26T07:18:12.579 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:12.574+0000 7f29baffd640 1 -- 172.21.15.143:0/3305562374 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f29bc012600 con 0x7f29c4143ac0 2023-11-26T07:18:12.579 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:12.574+0000 7f29cc72a640 1 -- 172.21.15.143:0/3305562374 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f29c4154d80 con 0x7f29c4143ac0 2023-11-26T07:18:12.579 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:12.574+0000 7f29cc72a640 1 -- 172.21.15.143:0/3305562374 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f29c406c4a0 con 0x7f29c4143ac0 2023-11-26T07:18:12.580 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:12.574+0000 7f29baffd640 1 -- 172.21.15.143:0/3305562374 <== mon.0 v2:172.21.15.143:3300/0 4 ==== mgrmap(e 5) v1 ==== 85852+0+0 (secure 0 0 0) 0x7f29bc007070 con 0x7f29c4143ac0 2023-11-26T07:18:12.580 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:12.574+0000 7f29baffd640 1 --2- 172.21.15.143:0/3305562374 >> [v2:172.21.15.143:6832/2150557544,v1:172.21.15.143:6833/2150557544] conn(0x7f29a8066470 0x7f29a8068930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:18:12.580 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:12.574+0000 7f29baffd640 1 -- 172.21.15.143:0/3305562374 <== mon.0 v2:172.21.15.143:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f29bc083c10 con 0x7f29c4143ac0 2023-11-26T07:18:12.580 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:12.574+0000 7f29ca4c6640 1 --2- 172.21.15.143:0/3305562374 >> [v2:172.21.15.143:6832/2150557544,v1:172.21.15.143:6833/2150557544] conn(0x7f29a8066470 0x7f29a8068930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:18:12.581 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:12.578+0000 7f29ca4c6640 1 --2- 172.21.15.143:0/3305562374 >> [v2:172.21.15.143:6832/2150557544,v1:172.21.15.143:6833/2150557544] conn(0x7f29a8066470 0x7f29a8068930 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7f29c00011f0 tx=0x7f29c0009040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:18:12.584 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:12.578+0000 7f29baffd640 1 -- 172.21.15.143:0/3305562374 <== mon.0 v2:172.21.15.143:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172649 (secure 0 0 0) 0x7f29bc051130 con 0x7f29c4143ac0 2023-11-26T07:18:12.771 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:12.766+0000 7f29cc72a640 1 -- 172.21.15.143:0/3305562374 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "fs ls"} v 0) v1 -- 0x7f29c4156d60 con 0x7f29c4143ac0 2023-11-26T07:18:12.771 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:12.766+0000 7f29baffd640 1 -- 172.21.15.143:0/3305562374 <== mon.0 v2:172.21.15.143:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v8) v1 ==== 53+0+73 (secure 0 0 0) 0x7f29bc017070 con 0x7f29c4143ac0 2023-11-26T07:18:12.771 INFO:teuthology.orchestra.run.smithi143.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2023-11-26T07:18:12.773 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:12.770+0000 7f29cc72a640 1 -- 172.21.15.143:0/3305562374 >> [v2:172.21.15.143:6832/2150557544,v1:172.21.15.143:6833/2150557544] conn(0x7f29a8066470 msgr2=0x7f29a8068930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:18:12.773 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:12.770+0000 7f29cc72a640 1 --2- 172.21.15.143:0/3305562374 >> [v2:172.21.15.143:6832/2150557544,v1:172.21.15.143:6833/2150557544] conn(0x7f29a8066470 0x7f29a8068930 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7f29c00011f0 tx=0x7f29c0009040 comp rx=0 tx=0).stop 2023-11-26T07:18:12.774 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:12.770+0000 7f29cc72a640 1 -- 172.21.15.143:0/3305562374 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f29c4143ac0 msgr2=0x7f29c4152ee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:18:12.774 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:12.770+0000 7f29cc72a640 1 --2- 172.21.15.143:0/3305562374 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f29c4143ac0 0x7f29c4152ee0 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7f29bc00cba0 tx=0x7f29bc00a430 comp rx=0 tx=0).stop 2023-11-26T07:18:12.774 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:12.770+0000 7f29cc72a640 1 -- 172.21.15.143:0/3305562374 shutdown_connections 2023-11-26T07:18:12.774 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:12.770+0000 7f29cc72a640 1 --2- 172.21.15.143:0/3305562374 >> [v2:172.21.15.143:6832/2150557544,v1:172.21.15.143:6833/2150557544] conn(0x7f29a8066470 0x7f29a8068930 unknown :-1 s=CLOSED pgs=119 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:18:12.774 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:12.770+0000 7f29cc72a640 1 --2- 172.21.15.143:0/3305562374 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f29c4143ac0 0x7f29c4152ee0 unknown :-1 s=CLOSED pgs=170 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:18:12.774 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:12.770+0000 7f29cc72a640 1 --2- 172.21.15.143:0/3305562374 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f29c40cf280 0x7f29c41529a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:18:12.774 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:12.770+0000 7f29cc72a640 1 --2- 172.21.15.143:0/3305562374 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f29c40cd1b0 0x7f29c4156820 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:18:12.774 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:12.770+0000 7f29cc72a640 1 -- 172.21.15.143:0/3305562374 >> 172.21.15.143:0/3305562374 conn(0x7f29c406a890 msgr2=0x7f29c413ccb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:18:12.774 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:12.770+0000 7f29cc72a640 1 -- 172.21.15.143:0/3305562374 shutdown_connections 2023-11-26T07:18:12.774 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:12.770+0000 7f29cc72a640 1 -- 172.21.15.143:0/3305562374 wait complete. 2023-11-26T07:18:12.792 INFO:tasks.cephfs.mount:Mounting default Ceph FS; just confirmed its presence on cluster 2023-11-26T07:18:12.792 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2023-11-26T07:18:12.792 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi188.front.sepia.ceph.com 2023-11-26T07:18:12.792 INFO:tasks.cephfs.mount:self.client.name = client.1 2023-11-26T07:18:12.792 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.1 2023-11-26T07:18:12.792 INFO:tasks.cephfs.mount:self.cephfs_name = None 2023-11-26T07:18:12.792 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2023-11-26T07:18:12.792 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2023-11-26T07:18:12.793 INFO:tasks.cephfs.mount:Setting the 'None' netns for '/home/ubuntu/cephtest/mnt.1' 2023-11-26T07:18:12.793 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-26T07:18:12.793 DEBUG:teuthology.orchestra.run.smithi188:> ip addr 2023-11-26T07:18:12.803 INFO:teuthology.orchestra.run.smithi188.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2023-11-26T07:18:12.803 INFO:teuthology.orchestra.run.smithi188.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2023-11-26T07:18:12.803 INFO:teuthology.orchestra.run.smithi188.stdout: inet 127.0.0.1/8 scope host lo 2023-11-26T07:18:12.803 INFO:teuthology.orchestra.run.smithi188.stdout: valid_lft forever preferred_lft forever 2023-11-26T07:18:12.803 INFO:teuthology.orchestra.run.smithi188.stdout: inet6 ::1/128 scope host 2023-11-26T07:18:12.804 INFO:teuthology.orchestra.run.smithi188.stdout: valid_lft forever preferred_lft forever 2023-11-26T07:18:12.804 INFO:teuthology.orchestra.run.smithi188.stdout:2: eno1: mtu 1500 qdisc noop state DOWN group default qlen 1000 2023-11-26T07:18:12.804 INFO:teuthology.orchestra.run.smithi188.stdout: link/ether 0c:c4:7a:d9:37:48 brd ff:ff:ff:ff:ff:ff 2023-11-26T07:18:12.804 INFO:teuthology.orchestra.run.smithi188.stdout: altname enp7s0f0 2023-11-26T07:18:12.804 INFO:teuthology.orchestra.run.smithi188.stdout:3: eno2: mtu 1500 qdisc noop state DOWN group default qlen 1000 2023-11-26T07:18:12.804 INFO:teuthology.orchestra.run.smithi188.stdout: link/ether 0c:c4:7a:d9:37:49 brd ff:ff:ff:ff:ff:ff 2023-11-26T07:18:12.804 INFO:teuthology.orchestra.run.smithi188.stdout: altname enp7s0f1 2023-11-26T07:18:12.804 INFO:teuthology.orchestra.run.smithi188.stdout:4: enp3s0f0: mtu 1500 qdisc noop state DOWN group default qlen 1000 2023-11-26T07:18:12.804 INFO:teuthology.orchestra.run.smithi188.stdout: link/ether 0c:c4:7a:88:80:38 brd ff:ff:ff:ff:ff:ff 2023-11-26T07:18:12.804 INFO:teuthology.orchestra.run.smithi188.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2023-11-26T07:18:12.804 INFO:teuthology.orchestra.run.smithi188.stdout: link/ether 0c:c4:7a:88:80:39 brd ff:ff:ff:ff:ff:ff 2023-11-26T07:18:12.804 INFO:teuthology.orchestra.run.smithi188.stdout: inet 172.21.15.188/20 metric 100 brd 172.21.15.255 scope global dynamic enp3s0f1 2023-11-26T07:18:12.804 INFO:teuthology.orchestra.run.smithi188.stdout: valid_lft 42375sec preferred_lft 42375sec 2023-11-26T07:18:12.805 INFO:teuthology.orchestra.run.smithi188.stdout: inet6 fe80::ec4:7aff:fe88:8039/64 scope link 2023-11-26T07:18:12.805 INFO:teuthology.orchestra.run.smithi188.stdout: valid_lft forever preferred_lft forever 2023-11-26T07:18:12.805 INFO:teuthology.orchestra.run.smithi188.stdout:6: docker0: mtu 1500 qdisc noqueue state DOWN group default 2023-11-26T07:18:12.805 INFO:teuthology.orchestra.run.smithi188.stdout: link/ether 02:42:d8:0d:a0:fa brd ff:ff:ff:ff:ff:ff 2023-11-26T07:18:12.805 INFO:teuthology.orchestra.run.smithi188.stdout: inet 172.17.0.1/16 brd 172.17.255.255 scope global docker0 2023-11-26T07:18:12.805 INFO:teuthology.orchestra.run.smithi188.stdout: valid_lft forever preferred_lft forever 2023-11-26T07:18:12.806 INFO:tasks.cephfs.mount:Setuping the 'ceph-brx' with 192.168.255.254/16 2023-11-26T07:18:12.806 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-26T07:18:12.806 DEBUG:teuthology.orchestra.run.smithi188:> (cd / && exec bash -c ' 2023-11-26T07:18:12.806 DEBUG:teuthology.orchestra.run.smithi188:> set -e 2023-11-26T07:18:12.806 DEBUG:teuthology.orchestra.run.smithi188:> sudo ip link add name ceph-brx type bridge 2023-11-26T07:18:12.806 DEBUG:teuthology.orchestra.run.smithi188:> sudo ip addr flush dev ceph-brx 2023-11-26T07:18:12.806 DEBUG:teuthology.orchestra.run.smithi188:> sudo ip link set ceph-brx up 2023-11-26T07:18:12.806 DEBUG:teuthology.orchestra.run.smithi188:> sudo ip addr add 192.168.255.254/16 brd 192.168.255.255 dev ceph-brx 2023-11-26T07:18:12.806 DEBUG:teuthology.orchestra.run.smithi188:> ') 2023-11-26T07:18:12.927 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-26T07:18:12.927 DEBUG:teuthology.orchestra.run.smithi188:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2023-11-26T07:18:12.946 INFO:teuthology.orchestra.run.smithi188.stdout:1 2023-11-26T07:18:12.948 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-26T07:18:12.949 DEBUG:teuthology.orchestra.run.smithi188:> route 2023-11-26T07:18:13.023 INFO:teuthology.orchestra.run.smithi188.stdout:Kernel IP routing table 2023-11-26T07:18:13.024 INFO:teuthology.orchestra.run.smithi188.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2023-11-26T07:18:13.024 INFO:teuthology.orchestra.run.smithi188.stdout:default _gateway 0.0.0.0 UG 0 0 0 enp3s0f1 2023-11-26T07:18:13.024 INFO:teuthology.orchestra.run.smithi188.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2023-11-26T07:18:13.024 INFO:teuthology.orchestra.run.smithi188.stdout:172.17.0.0 0.0.0.0 255.255.0.0 U 0 0 0 docker0 2023-11-26T07:18:13.024 INFO:teuthology.orchestra.run.smithi188.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 enp3s0f1 2023-11-26T07:18:13.024 INFO:teuthology.orchestra.run.smithi188.stdout:ns1.front.sepia 0.0.0.0 255.255.255.255 UH 100 0 0 enp3s0f1 2023-11-26T07:18:13.024 INFO:teuthology.orchestra.run.smithi188.stdout:172.21.0.2 0.0.0.0 255.255.255.255 UH 100 0 0 enp3s0f1 2023-11-26T07:18:13.024 INFO:teuthology.orchestra.run.smithi188.stdout:_gateway 0.0.0.0 255.255.255.255 UH 100 0 0 enp3s0f1 2023-11-26T07:18:13.024 INFO:teuthology.orchestra.run.smithi188.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2023-11-26T07:18:13.024 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-26T07:18:13.025 DEBUG:teuthology.orchestra.run.smithi188:> (cd / && exec bash -c ' 2023-11-26T07:18:13.025 DEBUG:teuthology.orchestra.run.smithi188:> set -e 2023-11-26T07:18:13.025 DEBUG:teuthology.orchestra.run.smithi188:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2023-11-26T07:18:13.025 DEBUG:teuthology.orchestra.run.smithi188:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2023-11-26T07:18:13.025 DEBUG:teuthology.orchestra.run.smithi188:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2023-11-26T07:18:13.025 DEBUG:teuthology.orchestra.run.smithi188:> ') 2023-11-26T07:18:13.121 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-26T07:18:13.122 DEBUG:teuthology.orchestra.run.smithi188:> ip netns list 2023-11-26T07:18:13.131 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-26T07:18:13.131 DEBUG:teuthology.orchestra.run.smithi188:> ip netns list-id 2023-11-26T07:18:13.182 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-26T07:18:13.182 DEBUG:teuthology.orchestra.run.smithi188:> (cd / && exec bash -c ' 2023-11-26T07:18:13.182 DEBUG:teuthology.orchestra.run.smithi188:> set -e 2023-11-26T07:18:13.182 DEBUG:teuthology.orchestra.run.smithi188:> sudo ip netns add ceph-ns--home-ubuntu-cephtest-mnt.1 2023-11-26T07:18:13.182 DEBUG:teuthology.orchestra.run.smithi188:> sudo ip netns set ceph-ns--home-ubuntu-cephtest-mnt.1 0 2023-11-26T07:18:13.183 DEBUG:teuthology.orchestra.run.smithi188:> ') 2023-11-26T07:18:13.265 INFO:tasks.cephfs.mount:Setuping the netns 'ceph-ns--home-ubuntu-cephtest-mnt.1' with 192.168.0.1/16 2023-11-26T07:18:13.265 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-26T07:18:13.265 DEBUG:teuthology.orchestra.run.smithi188:> (cd / && exec bash -c ' 2023-11-26T07:18:13.265 DEBUG:teuthology.orchestra.run.smithi188:> set -e 2023-11-26T07:18:13.266 DEBUG:teuthology.orchestra.run.smithi188:> sudo ip link add veth0 netns ceph-ns--home-ubuntu-cephtest-mnt.1 type veth peer name brx.0 2023-11-26T07:18:13.266 DEBUG:teuthology.orchestra.run.smithi188:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.1 ip addr add 192.168.0.1/16 brd 192.168.255.255 dev veth0 2023-11-26T07:18:13.266 DEBUG:teuthology.orchestra.run.smithi188:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.1 ip link set veth0 up 2023-11-26T07:18:13.266 DEBUG:teuthology.orchestra.run.smithi188:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.1 ip link set lo up 2023-11-26T07:18:13.266 DEBUG:teuthology.orchestra.run.smithi188:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.1 ip route add default via 192.168.255.254 2023-11-26T07:18:13.266 DEBUG:teuthology.orchestra.run.smithi188:> ') 2023-11-26T07:18:13.442 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-26T07:18:13.442 DEBUG:teuthology.orchestra.run.smithi188:> (cd / && exec bash -c ' 2023-11-26T07:18:13.442 DEBUG:teuthology.orchestra.run.smithi188:> set -e 2023-11-26T07:18:13.442 DEBUG:teuthology.orchestra.run.smithi188:> sudo ip link set brx.0 up 2023-11-26T07:18:13.442 DEBUG:teuthology.orchestra.run.smithi188:> sudo ip link set dev brx.0 master ceph-brx 2023-11-26T07:18:13.442 DEBUG:teuthology.orchestra.run.smithi188:> ') 2023-11-26T07:18:13.483 INFO:tasks.cephfs.fuse_mount:Client client.1 config is {} 2023-11-26T07:18:13.483 INFO:teuthology.orchestra.run:Running command with timeout 60 2023-11-26T07:18:13.483 DEBUG:teuthology.orchestra.run.smithi188:> mkdir -p -v /home/ubuntu/cephtest/mnt.1 2023-11-26T07:18:13.533 INFO:teuthology.orchestra.run.smithi188.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.1' 2023-11-26T07:18:13.534 INFO:teuthology.orchestra.run:Running command with timeout 60 2023-11-26T07:18:13.534 DEBUG:teuthology.orchestra.run.smithi188:> chmod 0000 /home/ubuntu/cephtest/mnt.1 2023-11-26T07:18:13.585 DEBUG:teuthology.orchestra.run.smithi188:> sudo modprobe fuse 2023-11-26T07:18:13.646 DEBUG:teuthology.orchestra.run.smithi188:> cat /proc/self/mounts | awk '{print $2}' 2023-11-26T07:18:13.703 INFO:teuthology.orchestra.run.smithi188.stdout:/sys 2023-11-26T07:18:13.703 INFO:teuthology.orchestra.run.smithi188.stdout:/proc 2023-11-26T07:18:13.703 INFO:teuthology.orchestra.run.smithi188.stdout:/dev 2023-11-26T07:18:13.703 INFO:teuthology.orchestra.run.smithi188.stdout:/dev/pts 2023-11-26T07:18:13.703 INFO:teuthology.orchestra.run.smithi188.stdout:/run 2023-11-26T07:18:13.703 INFO:teuthology.orchestra.run.smithi188.stdout:/ 2023-11-26T07:18:13.704 INFO:teuthology.orchestra.run.smithi188.stdout:/sys/kernel/security 2023-11-26T07:18:13.704 INFO:teuthology.orchestra.run.smithi188.stdout:/dev/shm 2023-11-26T07:18:13.704 INFO:teuthology.orchestra.run.smithi188.stdout:/run/lock 2023-11-26T07:18:13.704 INFO:teuthology.orchestra.run.smithi188.stdout:/sys/fs/cgroup 2023-11-26T07:18:13.704 INFO:teuthology.orchestra.run.smithi188.stdout:/sys/fs/pstore 2023-11-26T07:18:13.704 INFO:teuthology.orchestra.run.smithi188.stdout:/sys/fs/bpf 2023-11-26T07:18:13.704 INFO:teuthology.orchestra.run.smithi188.stdout:/proc/sys/fs/binfmt_misc 2023-11-26T07:18:13.704 INFO:teuthology.orchestra.run.smithi188.stdout:/dev/hugepages 2023-11-26T07:18:13.704 INFO:teuthology.orchestra.run.smithi188.stdout:/dev/mqueue 2023-11-26T07:18:13.704 INFO:teuthology.orchestra.run.smithi188.stdout:/sys/kernel/debug 2023-11-26T07:18:13.704 INFO:teuthology.orchestra.run.smithi188.stdout:/sys/kernel/tracing 2023-11-26T07:18:13.704 INFO:teuthology.orchestra.run.smithi188.stdout:/proc/fs/nfsd 2023-11-26T07:18:13.705 INFO:teuthology.orchestra.run.smithi188.stdout:/sys/fs/fuse/connections 2023-11-26T07:18:13.705 INFO:teuthology.orchestra.run.smithi188.stdout:/sys/kernel/config 2023-11-26T07:18:13.705 INFO:teuthology.orchestra.run.smithi188.stdout:/run/credentials/systemd-sysusers.service 2023-11-26T07:18:13.705 INFO:teuthology.orchestra.run.smithi188.stdout:/run/rpc_pipefs 2023-11-26T07:18:13.705 INFO:teuthology.orchestra.run.smithi188.stdout:/run/user/1000 2023-11-26T07:18:13.705 INFO:teuthology.orchestra.run.smithi188.stdout:/var/lib/ceph 2023-11-26T07:18:13.705 INFO:teuthology.orchestra.run.smithi188.stdout:/var/lib/ceph/osd/ceph-4 2023-11-26T07:18:13.705 INFO:teuthology.orchestra.run.smithi188.stdout:/var/lib/ceph/osd/ceph-5 2023-11-26T07:18:13.705 INFO:teuthology.orchestra.run.smithi188.stdout:/var/lib/ceph/osd/ceph-6 2023-11-26T07:18:13.705 INFO:teuthology.orchestra.run.smithi188.stdout:/var/lib/ceph/osd/ceph-7 2023-11-26T07:18:13.705 INFO:teuthology.orchestra.run.smithi188.stdout:/run/netns 2023-11-26T07:18:13.705 INFO:teuthology.orchestra.run.smithi188.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 2023-11-26T07:18:13.705 INFO:teuthology.orchestra.run.smithi188.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 2023-11-26T07:18:13.706 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-26T07:18:13.706 DEBUG:teuthology.orchestra.run.smithi188:> ls /sys/fs/fuse/connections 2023-11-26T07:18:13.758 INFO:tasks.cephfs.fuse_mount:Pre-mount connections: [] 2023-11-26T07:18:13.758 DEBUG:teuthology.orchestra.run.smithi188:> (cd /home/ubuntu/cephtest && exec sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-fuse -f --admin-socket '/var/run/ceph/$cluster-$name.$pid.asok' /home/ubuntu/cephtest/mnt.1 --id 1) 2023-11-26T07:18:13.804 DEBUG:teuthology.orchestra.run.smithi188:> sudo modprobe fuse 2023-11-26T07:18:13.826 DEBUG:teuthology.orchestra.run.smithi188:> cat /proc/self/mounts | awk '{print $2}' 2023-11-26T07:18:13.884 INFO:teuthology.orchestra.run.smithi188.stdout:/sys 2023-11-26T07:18:13.884 INFO:teuthology.orchestra.run.smithi188.stdout:/proc 2023-11-26T07:18:13.884 INFO:teuthology.orchestra.run.smithi188.stdout:/dev 2023-11-26T07:18:13.884 INFO:teuthology.orchestra.run.smithi188.stdout:/dev/pts 2023-11-26T07:18:13.884 INFO:teuthology.orchestra.run.smithi188.stdout:/run 2023-11-26T07:18:13.884 INFO:teuthology.orchestra.run.smithi188.stdout:/ 2023-11-26T07:18:13.884 INFO:teuthology.orchestra.run.smithi188.stdout:/sys/kernel/security 2023-11-26T07:18:13.884 INFO:teuthology.orchestra.run.smithi188.stdout:/dev/shm 2023-11-26T07:18:13.884 INFO:teuthology.orchestra.run.smithi188.stdout:/run/lock 2023-11-26T07:18:13.885 INFO:teuthology.orchestra.run.smithi188.stdout:/sys/fs/cgroup 2023-11-26T07:18:13.885 INFO:teuthology.orchestra.run.smithi188.stdout:/sys/fs/pstore 2023-11-26T07:18:13.885 INFO:teuthology.orchestra.run.smithi188.stdout:/sys/fs/bpf 2023-11-26T07:18:13.885 INFO:teuthology.orchestra.run.smithi188.stdout:/proc/sys/fs/binfmt_misc 2023-11-26T07:18:13.885 INFO:teuthology.orchestra.run.smithi188.stdout:/dev/hugepages 2023-11-26T07:18:13.885 INFO:teuthology.orchestra.run.smithi188.stdout:/dev/mqueue 2023-11-26T07:18:13.885 INFO:teuthology.orchestra.run.smithi188.stdout:/sys/kernel/debug 2023-11-26T07:18:13.885 INFO:teuthology.orchestra.run.smithi188.stdout:/sys/kernel/tracing 2023-11-26T07:18:13.885 INFO:teuthology.orchestra.run.smithi188.stdout:/proc/fs/nfsd 2023-11-26T07:18:13.885 INFO:teuthology.orchestra.run.smithi188.stdout:/sys/fs/fuse/connections 2023-11-26T07:18:13.885 INFO:teuthology.orchestra.run.smithi188.stdout:/sys/kernel/config 2023-11-26T07:18:13.885 INFO:teuthology.orchestra.run.smithi188.stdout:/run/credentials/systemd-sysusers.service 2023-11-26T07:18:13.885 INFO:teuthology.orchestra.run.smithi188.stdout:/run/rpc_pipefs 2023-11-26T07:18:13.886 INFO:teuthology.orchestra.run.smithi188.stdout:/run/user/1000 2023-11-26T07:18:13.886 INFO:teuthology.orchestra.run.smithi188.stdout:/var/lib/ceph 2023-11-26T07:18:13.886 INFO:teuthology.orchestra.run.smithi188.stdout:/var/lib/ceph/osd/ceph-4 2023-11-26T07:18:13.886 INFO:teuthology.orchestra.run.smithi188.stdout:/var/lib/ceph/osd/ceph-5 2023-11-26T07:18:13.886 INFO:teuthology.orchestra.run.smithi188.stdout:/var/lib/ceph/osd/ceph-6 2023-11-26T07:18:13.886 INFO:teuthology.orchestra.run.smithi188.stdout:/var/lib/ceph/osd/ceph-7 2023-11-26T07:18:13.886 INFO:teuthology.orchestra.run.smithi188.stdout:/run/netns 2023-11-26T07:18:13.886 INFO:teuthology.orchestra.run.smithi188.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 2023-11-26T07:18:13.886 INFO:teuthology.orchestra.run.smithi188.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 2023-11-26T07:18:13.887 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-26T07:18:13.887 DEBUG:teuthology.orchestra.run.smithi188:> ls /sys/fs/fuse/connections 2023-11-26T07:18:13.920 INFO:tasks.cephfs.fuse_mount.ceph-fuse.1.smithi188.stderr:2023-11-26T07:18:13.914+0000 7fb16757a580 -1 init, newargv = 0x555dd1896530 newargc=15 2023-11-26T07:18:13.920 INFO:tasks.cephfs.fuse_mount.ceph-fuse.1.smithi188.stderr:ceph-fuse[44114]: starting ceph client 2023-11-26T07:18:13.929 INFO:tasks.cephfs.fuse_mount.ceph-fuse.1.smithi188.stderr:ceph-fuse[44114]: starting fuse 2023-11-26T07:18:13.930 INFO:teuthology.orchestra.run.smithi188.stdout:43 2023-11-26T07:18:13.930 INFO:tasks.cephfs.fuse_mount:Post-mount connections: [43] 2023-11-26T07:18:13.930 DEBUG:teuthology.orchestra.run.smithi188:> sudo stdin-killer -- python3 -c ' 2023-11-26T07:18:13.930 DEBUG:teuthology.orchestra.run.smithi188:> import glob 2023-11-26T07:18:13.930 DEBUG:teuthology.orchestra.run.smithi188:> import re 2023-11-26T07:18:13.930 DEBUG:teuthology.orchestra.run.smithi188:> import os 2023-11-26T07:18:13.930 DEBUG:teuthology.orchestra.run.smithi188:> import subprocess 2023-11-26T07:18:13.931 DEBUG:teuthology.orchestra.run.smithi188:> 2023-11-26T07:18:13.931 DEBUG:teuthology.orchestra.run.smithi188:> def _find_admin_socket(client_name): 2023-11-26T07:18:13.931 DEBUG:teuthology.orchestra.run.smithi188:> asok_path = "/var/run/ceph/ceph-client.1.*.asok" 2023-11-26T07:18:13.931 DEBUG:teuthology.orchestra.run.smithi188:> files = glob.glob(asok_path) 2023-11-26T07:18:13.931 DEBUG:teuthology.orchestra.run.smithi188:> mountpoint = "/home/ubuntu/cephtest/mnt.1" 2023-11-26T07:18:13.931 DEBUG:teuthology.orchestra.run.smithi188:> 2023-11-26T07:18:13.931 DEBUG:teuthology.orchestra.run.smithi188:> # Given a non-glob path, it better be there 2023-11-26T07:18:13.931 DEBUG:teuthology.orchestra.run.smithi188:> if "*" not in asok_path: 2023-11-26T07:18:13.931 DEBUG:teuthology.orchestra.run.smithi188:> assert(len(files) == 1) 2023-11-26T07:18:13.931 DEBUG:teuthology.orchestra.run.smithi188:> return files[0] 2023-11-26T07:18:13.931 DEBUG:teuthology.orchestra.run.smithi188:> 2023-11-26T07:18:13.931 DEBUG:teuthology.orchestra.run.smithi188:> for f in files: 2023-11-26T07:18:13.931 DEBUG:teuthology.orchestra.run.smithi188:> pid = re.match(".*\.(\d+)\.asok$", f).group(1) 2023-11-26T07:18:13.931 DEBUG:teuthology.orchestra.run.smithi188:> if os.path.exists("/proc/{0}".format(pid)): 2023-11-26T07:18:13.932 DEBUG:teuthology.orchestra.run.smithi188:> with open("/proc/{0}/cmdline".format(pid), '"'"'r'"'"') as proc_f: 2023-11-26T07:18:13.932 DEBUG:teuthology.orchestra.run.smithi188:> contents = proc_f.read() 2023-11-26T07:18:13.932 DEBUG:teuthology.orchestra.run.smithi188:> if mountpoint in contents: 2023-11-26T07:18:13.932 DEBUG:teuthology.orchestra.run.smithi188:> return f 2023-11-26T07:18:13.932 DEBUG:teuthology.orchestra.run.smithi188:> raise RuntimeError("Client socket {0} not found".format(client_name)) 2023-11-26T07:18:13.932 DEBUG:teuthology.orchestra.run.smithi188:> 2023-11-26T07:18:13.932 DEBUG:teuthology.orchestra.run.smithi188:> print(_find_admin_socket("client.1")) 2023-11-26T07:18:13.932 DEBUG:teuthology.orchestra.run.smithi188:> ' 2023-11-26T07:18:14.058 INFO:teuthology.orchestra.run.smithi188.stdout:/var/run/ceph/ceph-client.1.44114.asok 2023-11-26T07:18:14.061 INFO:teuthology.orchestra.run.smithi188.stderr:2023-11-26T07:18:14 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2023-11-26T07:18:14.070 INFO:tasks.cephfs.fuse_mount:Found client admin socket at /var/run/ceph/ceph-client.1.44114.asok 2023-11-26T07:18:14.070 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-26T07:18:14.070 DEBUG:teuthology.orchestra.run.smithi188:> sudo ceph --admin-daemon /var/run/ceph/ceph-client.1.44114.asok status 2023-11-26T07:18:14.241 INFO:teuthology.orchestra.run.smithi188.stdout:{ 2023-11-26T07:18:14.241 INFO:teuthology.orchestra.run.smithi188.stdout: "metadata": { 2023-11-26T07:18:14.242 INFO:teuthology.orchestra.run.smithi188.stdout: "ceph_sha1": "44c2237fe77713214148c1fd622c872f346f1b99", 2023-11-26T07:18:14.242 INFO:teuthology.orchestra.run.smithi188.stdout: "ceph_version": "ceph version 18.0.0-7484-g44c2237f (44c2237fe77713214148c1fd622c872f346f1b99) reef (dev)", 2023-11-26T07:18:14.242 INFO:teuthology.orchestra.run.smithi188.stdout: "entity_id": "1", 2023-11-26T07:18:14.242 INFO:teuthology.orchestra.run.smithi188.stdout: "hostname": "smithi188", 2023-11-26T07:18:14.242 INFO:teuthology.orchestra.run.smithi188.stdout: "mount_point": "/home/ubuntu/cephtest/mnt.1", 2023-11-26T07:18:14.242 INFO:teuthology.orchestra.run.smithi188.stdout: "pid": "44114", 2023-11-26T07:18:14.242 INFO:teuthology.orchestra.run.smithi188.stdout: "root": "/" 2023-11-26T07:18:14.242 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2023-11-26T07:18:14.242 INFO:teuthology.orchestra.run.smithi188.stdout: "dentry_count": 0, 2023-11-26T07:18:14.242 INFO:teuthology.orchestra.run.smithi188.stdout: "dentry_pinned_count": 0, 2023-11-26T07:18:14.242 INFO:teuthology.orchestra.run.smithi188.stdout: "id": 4687, 2023-11-26T07:18:14.243 INFO:teuthology.orchestra.run.smithi188.stdout: "inst": { 2023-11-26T07:18:14.243 INFO:teuthology.orchestra.run.smithi188.stdout: "name": { 2023-11-26T07:18:14.243 INFO:teuthology.orchestra.run.smithi188.stdout: "type": "client", 2023-11-26T07:18:14.243 INFO:teuthology.orchestra.run.smithi188.stdout: "num": 4687 2023-11-26T07:18:14.243 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2023-11-26T07:18:14.243 INFO:teuthology.orchestra.run.smithi188.stdout: "addr": { 2023-11-26T07:18:14.243 INFO:teuthology.orchestra.run.smithi188.stdout: "type": "v1", 2023-11-26T07:18:14.243 INFO:teuthology.orchestra.run.smithi188.stdout: "addr": "192.168.0.1:0", 2023-11-26T07:18:14.243 INFO:teuthology.orchestra.run.smithi188.stdout: "nonce": 2773240134 2023-11-26T07:18:14.243 INFO:teuthology.orchestra.run.smithi188.stdout: } 2023-11-26T07:18:14.243 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2023-11-26T07:18:14.243 INFO:teuthology.orchestra.run.smithi188.stdout: "addr": { 2023-11-26T07:18:14.243 INFO:teuthology.orchestra.run.smithi188.stdout: "type": "v1", 2023-11-26T07:18:14.244 INFO:teuthology.orchestra.run.smithi188.stdout: "addr": "192.168.0.1:0", 2023-11-26T07:18:14.244 INFO:teuthology.orchestra.run.smithi188.stdout: "nonce": 2773240134 2023-11-26T07:18:14.244 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2023-11-26T07:18:14.244 INFO:teuthology.orchestra.run.smithi188.stdout: "inst_str": "client.4687 192.168.0.1:0/2773240134", 2023-11-26T07:18:14.244 INFO:teuthology.orchestra.run.smithi188.stdout: "addr_str": "192.168.0.1:0/2773240134", 2023-11-26T07:18:14.244 INFO:teuthology.orchestra.run.smithi188.stdout: "inode_count": 1, 2023-11-26T07:18:14.244 INFO:teuthology.orchestra.run.smithi188.stdout: "mds_epoch": 8, 2023-11-26T07:18:14.244 INFO:teuthology.orchestra.run.smithi188.stdout: "osd_epoch": 20, 2023-11-26T07:18:14.244 INFO:teuthology.orchestra.run.smithi188.stdout: "osd_epoch_barrier": 0, 2023-11-26T07:18:14.244 INFO:teuthology.orchestra.run.smithi188.stdout: "blocklisted": false, 2023-11-26T07:18:14.244 INFO:teuthology.orchestra.run.smithi188.stdout: "fs_name": "cephfs" 2023-11-26T07:18:14.244 INFO:teuthology.orchestra.run.smithi188.stdout:} 2023-11-26T07:18:14.256 DEBUG:teuthology.orchestra.run.smithi143:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2023-11-26T07:18:14.404 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:14.397+0000 7f6391ff8640 1 Processor -- start 2023-11-26T07:18:14.405 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:14.397+0000 7f6391ff8640 1 -- start start 2023-11-26T07:18:14.406 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:14.401+0000 7f6391ff8640 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7f638c0cf280 0x7f638c0cf710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:18:14.406 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:14.401+0000 7f6391ff8640 1 -- --> v1:172.21.15.143:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f638c0cc730 con 0x7f638c0cd1b0 2023-11-26T07:18:14.406 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:14.401+0000 7f6391ff8640 1 -- --> v1:172.21.15.188:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f638c070ee0 con 0x7f638c143ac0 2023-11-26T07:18:14.406 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:14.401+0000 7f6391ff8640 1 -- --> v2:172.21.15.143:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f638c143ea0 con 0x7f638c0cf280 2023-11-26T07:18:14.406 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:14.401+0000 7f638bfff640 1 --1- >> v1:172.21.15.143:6789/0 conn(0x7f638c0cd1b0 0x7f638c0cd5c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.143:6789/0 says I am v1:172.21.15.143:43846/0 (socket says 172.21.15.143:43846) 2023-11-26T07:18:14.406 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:14.401+0000 7f638affd640 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7f638c0cf280 0x7f638c0cf710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:18:14.407 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:14.401+0000 7f638bfff640 1 -- 172.21.15.143:0/3110371944 learned_addr learned my addr 172.21.15.143:0/3110371944 (peer_addr_for_me v1:172.21.15.143:0/0) 2023-11-26T07:18:14.407 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:14.401+0000 7f638affd640 1 -- 172.21.15.143:0/3110371944 >> v1:172.21.15.188:6789/0 conn(0x7f638c143ac0 legacy=0x7f638c070370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:18:14.407 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:14.401+0000 7f638affd640 1 -- 172.21.15.143:0/3110371944 >> v1:172.21.15.143:6789/0 conn(0x7f638c0cd1b0 legacy=0x7f638c0cd5c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:18:14.407 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:14.401+0000 7f638affd640 1 -- 172.21.15.143:0/3110371944 --> v2:172.21.15.143:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f638c0711c0 con 0x7f638c0cf280 2023-11-26T07:18:14.409 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:14.405+0000 7f638affd640 1 --2- 172.21.15.143:0/3110371944 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f638c0cf280 0x7f638c0cf710 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7f63800097c0 tx=0x7f638001c480 comp rx=0 tx=0).ready entity=mon.0 client_cookie=874b24208f376937 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:18:14.409 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:14.405+0000 7f638a7fc640 1 -- 172.21.15.143:0/3110371944 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f63800a8050 con 0x7f638c0cf280 2023-11-26T07:18:14.410 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:14.405+0000 7f638a7fc640 1 -- 172.21.15.143:0/3110371944 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f63800ac5f0 con 0x7f638c0cf280 2023-11-26T07:18:14.410 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:14.405+0000 7f638a7fc640 1 -- 172.21.15.143:0/3110371944 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f63800ac980 con 0x7f638c0cf280 2023-11-26T07:18:14.411 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:14.405+0000 7f6391ff8640 1 -- 172.21.15.143:0/3110371944 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f638c0cf280 msgr2=0x7f638c0cf710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:18:14.411 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:14.405+0000 7f6391ff8640 1 --2- 172.21.15.143:0/3110371944 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f638c0cf280 0x7f638c0cf710 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7f63800097c0 tx=0x7f638001c480 comp rx=0 tx=0).stop 2023-11-26T07:18:14.411 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:14.405+0000 7f6391ff8640 1 -- 172.21.15.143:0/3110371944 shutdown_connections 2023-11-26T07:18:14.411 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:14.405+0000 7f6391ff8640 1 --2- 172.21.15.143:0/3110371944 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f638c0cf280 0x7f638c0cf710 unknown :-1 s=CLOSED pgs=172 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:18:14.411 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:14.405+0000 7f6391ff8640 1 -- 172.21.15.143:0/3110371944 >> 172.21.15.143:0/3110371944 conn(0x7f638c06a890 msgr2=0x7f638c06aca0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:18:14.411 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:14.405+0000 7f6391ff8640 1 -- 172.21.15.143:0/3110371944 shutdown_connections 2023-11-26T07:18:14.411 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:14.405+0000 7f6391ff8640 1 -- 172.21.15.143:0/3110371944 wait complete. 2023-11-26T07:18:14.412 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:14.405+0000 7f6391ff8640 1 Processor -- start 2023-11-26T07:18:14.412 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:14.405+0000 7f6391ff8640 1 -- start start 2023-11-26T07:18:14.414 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:14.409+0000 7f6391ff8640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f638c0cd1b0 0x7f638c157b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:18:14.414 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:14.409+0000 7f6391ff8640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f638c0cf280 0x7f638c0b10c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:18:14.414 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:14.409+0000 7f6391ff8640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f638c143ac0 0x7f638c12d1b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:18:14.414 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:14.409+0000 7f6391ff8640 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f638c158040 con 0x7f638c143ac0 2023-11-26T07:18:14.414 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:14.409+0000 7f6391ff8640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f638c13ebc0 con 0x7f638c0cf280 2023-11-26T07:18:14.414 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:14.409+0000 7f6391ff8640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f638c13dfb0 con 0x7f638c0cd1b0 2023-11-26T07:18:14.414 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:14.409+0000 7f638bfff640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f638c143ac0 0x7f638c12d1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:18:14.414 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:14.409+0000 7f638b7fe640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f638c0cd1b0 0x7f638c157b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:18:14.414 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:14.409+0000 7f638bfff640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f638c143ac0 0x7f638c12d1b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:43778/0 (socket says 172.21.15.143:43778) 2023-11-26T07:18:14.415 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:14.409+0000 7f638affd640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f638c0cf280 0x7f638c0b10c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:18:14.415 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:14.409+0000 7f638bfff640 1 -- 172.21.15.143:0/2918172398 learned_addr learned my addr 172.21.15.143:0/2918172398 (peer_addr_for_me v2:172.21.15.143:0/0) 2023-11-26T07:18:14.415 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:14.409+0000 7f638b7fe640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f638c0cd1b0 0x7f638c157b00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.188:3301/0 says I am v2:172.21.15.143:45356/0 (socket says 172.21.15.143:45356) 2023-11-26T07:18:14.416 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:14.409+0000 7f638bfff640 1 -- 172.21.15.143:0/2918172398 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f638c0cd1b0 msgr2=0x7f638c157b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:18:14.416 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:14.409+0000 7f638bfff640 1 --2- 172.21.15.143:0/2918172398 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f638c0cd1b0 0x7f638c157b00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:18:14.416 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:14.409+0000 7f638bfff640 1 -- 172.21.15.143:0/2918172398 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f638c0cf280 msgr2=0x7f638c0b10c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:18:14.416 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:14.409+0000 7f638bfff640 1 --2- 172.21.15.143:0/2918172398 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f638c0cf280 0x7f638c0b10c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:18:14.416 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:14.409+0000 7f638bfff640 1 -- 172.21.15.143:0/2918172398 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6380009420 con 0x7f638c143ac0 2023-11-26T07:18:14.416 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:14.409+0000 7f638b7fe640 1 --2- 172.21.15.143:0/2918172398 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f638c0cd1b0 0x7f638c157b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-26T07:18:14.416 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:14.409+0000 7f638bfff640 1 --2- 172.21.15.143:0/2918172398 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f638c143ac0 0x7f638c12d1b0 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7f638400cbc0 tx=0x7f638400a430 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:18:14.417 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:14.409+0000 7f6373fff640 1 -- 172.21.15.143:0/2918172398 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f638400ad40 con 0x7f638c143ac0 2023-11-26T07:18:14.417 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:14.409+0000 7f6373fff640 1 -- 172.21.15.143:0/2918172398 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6384012400 con 0x7f638c143ac0 2023-11-26T07:18:14.417 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:14.409+0000 7f6373fff640 1 -- 172.21.15.143:0/2918172398 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f638400aea0 con 0x7f638c143ac0 2023-11-26T07:18:14.417 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:14.409+0000 7f6391ff8640 1 -- 172.21.15.143:0/2918172398 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f638c13e290 con 0x7f638c143ac0 2023-11-26T07:18:14.417 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:14.409+0000 7f6391ff8640 1 -- 172.21.15.143:0/2918172398 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f638c13e850 con 0x7f638c143ac0 2023-11-26T07:18:14.417 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:14.409+0000 7f6391ff8640 1 -- 172.21.15.143:0/2918172398 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f638c13ed30 con 0x7f638c143ac0 2023-11-26T07:18:14.418 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:14.413+0000 7f6373fff640 1 -- 172.21.15.143:0/2918172398 <== mon.0 v2:172.21.15.143:3300/0 4 ==== mgrmap(e 5) v1 ==== 85852+0+0 (secure 0 0 0) 0x7f6384007070 con 0x7f638c143ac0 2023-11-26T07:18:14.418 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:14.413+0000 7f6373fff640 1 --2- 172.21.15.143:0/2918172398 >> [v2:172.21.15.143:6832/2150557544,v1:172.21.15.143:6833/2150557544] conn(0x7f6378066470 0x7f6378068930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:18:14.418 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:14.413+0000 7f6373fff640 1 -- 172.21.15.143:0/2918172398 <== mon.0 v2:172.21.15.143:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f6384083ca0 con 0x7f638c143ac0 2023-11-26T07:18:14.418 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:14.413+0000 7f638b7fe640 1 --2- 172.21.15.143:0/2918172398 >> [v2:172.21.15.143:6832/2150557544,v1:172.21.15.143:6833/2150557544] conn(0x7f6378066470 0x7f6378068930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:18:14.418 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:14.413+0000 7f638b7fe640 1 --2- 172.21.15.143:0/2918172398 >> [v2:172.21.15.143:6832/2150557544,v1:172.21.15.143:6833/2150557544] conn(0x7f6378066470 0x7f6378068930 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7f637c007fd0 tx=0x7f637c009040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:18:14.422 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:14.417+0000 7f6373fff640 1 -- 172.21.15.143:0/2918172398 <== mon.0 v2:172.21.15.143:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172649 (secure 0 0 0) 0x7f63840511c0 con 0x7f638c143ac0 2023-11-26T07:18:14.603 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:14.597+0000 7f6391ff8640 1 -- 172.21.15.143:0/2918172398 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "fs ls"} v 0) v1 -- 0x7f638c067680 con 0x7f638c143ac0 2023-11-26T07:18:14.603 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:14.597+0000 7f6373fff640 1 -- 172.21.15.143:0/2918172398 <== mon.0 v2:172.21.15.143:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v8) v1 ==== 53+0+73 (secure 0 0 0) 0x7f6384017070 con 0x7f638c143ac0 2023-11-26T07:18:14.603 INFO:teuthology.orchestra.run.smithi143.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2023-11-26T07:18:14.605 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:14.597+0000 7f6391ff8640 1 -- 172.21.15.143:0/2918172398 >> [v2:172.21.15.143:6832/2150557544,v1:172.21.15.143:6833/2150557544] conn(0x7f6378066470 msgr2=0x7f6378068930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:18:14.605 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:14.597+0000 7f6391ff8640 1 --2- 172.21.15.143:0/2918172398 >> [v2:172.21.15.143:6832/2150557544,v1:172.21.15.143:6833/2150557544] conn(0x7f6378066470 0x7f6378068930 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7f637c007fd0 tx=0x7f637c009040 comp rx=0 tx=0).stop 2023-11-26T07:18:14.605 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:14.597+0000 7f6391ff8640 1 -- 172.21.15.143:0/2918172398 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f638c143ac0 msgr2=0x7f638c12d1b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:18:14.605 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:14.597+0000 7f6391ff8640 1 --2- 172.21.15.143:0/2918172398 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f638c143ac0 0x7f638c12d1b0 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7f638400cbc0 tx=0x7f638400a430 comp rx=0 tx=0).stop 2023-11-26T07:18:14.605 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:14.597+0000 7f6391ff8640 1 -- 172.21.15.143:0/2918172398 shutdown_connections 2023-11-26T07:18:14.605 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:14.597+0000 7f6391ff8640 1 --2- 172.21.15.143:0/2918172398 >> [v2:172.21.15.143:6832/2150557544,v1:172.21.15.143:6833/2150557544] conn(0x7f6378066470 0x7f6378068930 unknown :-1 s=CLOSED pgs=120 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:18:14.605 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:14.597+0000 7f6391ff8640 1 --2- 172.21.15.143:0/2918172398 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f638c143ac0 0x7f638c12d1b0 unknown :-1 s=CLOSED pgs=173 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:18:14.605 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:14.597+0000 7f6391ff8640 1 --2- 172.21.15.143:0/2918172398 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f638c0cf280 0x7f638c0b10c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:18:14.606 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:14.597+0000 7f6391ff8640 1 --2- 172.21.15.143:0/2918172398 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f638c0cd1b0 0x7f638c157b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:18:14.606 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:14.597+0000 7f6391ff8640 1 -- 172.21.15.143:0/2918172398 >> 172.21.15.143:0/2918172398 conn(0x7f638c06a890 msgr2=0x7f638c0b3850 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:18:14.606 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:14.601+0000 7f6391ff8640 1 -- 172.21.15.143:0/2918172398 shutdown_connections 2023-11-26T07:18:14.606 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:14.601+0000 7f6391ff8640 1 -- 172.21.15.143:0/2918172398 wait complete. 2023-11-26T07:18:14.624 INFO:tasks.cephfs.mount:Mounting default Ceph FS; just confirmed its presence on cluster 2023-11-26T07:18:14.624 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2023-11-26T07:18:14.624 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi188.front.sepia.ceph.com 2023-11-26T07:18:14.624 INFO:tasks.cephfs.mount:self.client.name = client.3 2023-11-26T07:18:14.625 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.3 2023-11-26T07:18:14.625 INFO:tasks.cephfs.mount:self.cephfs_name = None 2023-11-26T07:18:14.625 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2023-11-26T07:18:14.625 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2023-11-26T07:18:14.625 INFO:tasks.cephfs.mount:Setting the 'None' netns for '/home/ubuntu/cephtest/mnt.3' 2023-11-26T07:18:14.625 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-26T07:18:14.625 DEBUG:teuthology.orchestra.run.smithi188:> ip addr 2023-11-26T07:18:14.634 INFO:teuthology.orchestra.run.smithi188.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2023-11-26T07:18:14.635 INFO:teuthology.orchestra.run.smithi188.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2023-11-26T07:18:14.635 INFO:teuthology.orchestra.run.smithi188.stdout: inet 127.0.0.1/8 scope host lo 2023-11-26T07:18:14.635 INFO:teuthology.orchestra.run.smithi188.stdout: valid_lft forever preferred_lft forever 2023-11-26T07:18:14.635 INFO:teuthology.orchestra.run.smithi188.stdout: inet6 ::1/128 scope host 2023-11-26T07:18:14.635 INFO:teuthology.orchestra.run.smithi188.stdout: valid_lft forever preferred_lft forever 2023-11-26T07:18:14.635 INFO:teuthology.orchestra.run.smithi188.stdout:2: eno1: mtu 1500 qdisc noop state DOWN group default qlen 1000 2023-11-26T07:18:14.635 INFO:teuthology.orchestra.run.smithi188.stdout: link/ether 0c:c4:7a:d9:37:48 brd ff:ff:ff:ff:ff:ff 2023-11-26T07:18:14.635 INFO:teuthology.orchestra.run.smithi188.stdout: altname enp7s0f0 2023-11-26T07:18:14.635 INFO:teuthology.orchestra.run.smithi188.stdout:3: eno2: mtu 1500 qdisc noop state DOWN group default qlen 1000 2023-11-26T07:18:14.635 INFO:teuthology.orchestra.run.smithi188.stdout: link/ether 0c:c4:7a:d9:37:49 brd ff:ff:ff:ff:ff:ff 2023-11-26T07:18:14.635 INFO:teuthology.orchestra.run.smithi188.stdout: altname enp7s0f1 2023-11-26T07:18:14.636 INFO:teuthology.orchestra.run.smithi188.stdout:4: enp3s0f0: mtu 1500 qdisc noop state DOWN group default qlen 1000 2023-11-26T07:18:14.636 INFO:teuthology.orchestra.run.smithi188.stdout: link/ether 0c:c4:7a:88:80:38 brd ff:ff:ff:ff:ff:ff 2023-11-26T07:18:14.636 INFO:teuthology.orchestra.run.smithi188.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2023-11-26T07:18:14.636 INFO:teuthology.orchestra.run.smithi188.stdout: link/ether 0c:c4:7a:88:80:39 brd ff:ff:ff:ff:ff:ff 2023-11-26T07:18:14.636 INFO:teuthology.orchestra.run.smithi188.stdout: inet 172.21.15.188/20 metric 100 brd 172.21.15.255 scope global dynamic enp3s0f1 2023-11-26T07:18:14.636 INFO:teuthology.orchestra.run.smithi188.stdout: valid_lft 42373sec preferred_lft 42373sec 2023-11-26T07:18:14.636 INFO:teuthology.orchestra.run.smithi188.stdout: inet6 fe80::ec4:7aff:fe88:8039/64 scope link 2023-11-26T07:18:14.636 INFO:teuthology.orchestra.run.smithi188.stdout: valid_lft forever preferred_lft forever 2023-11-26T07:18:14.636 INFO:teuthology.orchestra.run.smithi188.stdout:6: docker0: mtu 1500 qdisc noqueue state DOWN group default 2023-11-26T07:18:14.636 INFO:teuthology.orchestra.run.smithi188.stdout: link/ether 02:42:d8:0d:a0:fa brd ff:ff:ff:ff:ff:ff 2023-11-26T07:18:14.636 INFO:teuthology.orchestra.run.smithi188.stdout: inet 172.17.0.1/16 brd 172.17.255.255 scope global docker0 2023-11-26T07:18:14.637 INFO:teuthology.orchestra.run.smithi188.stdout: valid_lft forever preferred_lft forever 2023-11-26T07:18:14.637 INFO:teuthology.orchestra.run.smithi188.stdout:7: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2023-11-26T07:18:14.637 INFO:teuthology.orchestra.run.smithi188.stdout: link/ether 5e:1f:84:6d:79:ae brd ff:ff:ff:ff:ff:ff 2023-11-26T07:18:14.637 INFO:teuthology.orchestra.run.smithi188.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2023-11-26T07:18:14.637 INFO:teuthology.orchestra.run.smithi188.stdout: valid_lft forever preferred_lft forever 2023-11-26T07:18:14.637 INFO:teuthology.orchestra.run.smithi188.stdout: inet6 fe80::5c1f:84ff:fe6d:79ae/64 scope link tentative 2023-11-26T07:18:14.637 INFO:teuthology.orchestra.run.smithi188.stdout: valid_lft forever preferred_lft forever 2023-11-26T07:18:14.637 INFO:teuthology.orchestra.run.smithi188.stdout:8: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2023-11-26T07:18:14.637 INFO:teuthology.orchestra.run.smithi188.stdout: link/ether 0e:5c:f6:1f:82:2f brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.1 2023-11-26T07:18:14.637 INFO:teuthology.orchestra.run.smithi188.stdout: inet6 fe80::c5c:f6ff:fe1f:822f/64 scope link tentative 2023-11-26T07:18:14.637 INFO:teuthology.orchestra.run.smithi188.stdout: valid_lft forever preferred_lft forever 2023-11-26T07:18:14.638 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-26T07:18:14.638 DEBUG:teuthology.orchestra.run.smithi188:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2023-11-26T07:18:14.700 INFO:teuthology.orchestra.run.smithi188.stdout:1 2023-11-26T07:18:14.702 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-26T07:18:14.702 DEBUG:teuthology.orchestra.run.smithi188:> route 2023-11-26T07:18:14.756 INFO:teuthology.orchestra.run.smithi188.stdout:Kernel IP routing table 2023-11-26T07:18:14.756 INFO:teuthology.orchestra.run.smithi188.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2023-11-26T07:18:14.756 INFO:teuthology.orchestra.run.smithi188.stdout:default _gateway 0.0.0.0 UG 0 0 0 enp3s0f1 2023-11-26T07:18:14.757 INFO:teuthology.orchestra.run.smithi188.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2023-11-26T07:18:14.757 INFO:teuthology.orchestra.run.smithi188.stdout:172.17.0.0 0.0.0.0 255.255.0.0 U 0 0 0 docker0 2023-11-26T07:18:14.757 INFO:teuthology.orchestra.run.smithi188.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 enp3s0f1 2023-11-26T07:18:14.757 INFO:teuthology.orchestra.run.smithi188.stdout:ns1.front.sepia 0.0.0.0 255.255.255.255 UH 100 0 0 enp3s0f1 2023-11-26T07:18:14.757 INFO:teuthology.orchestra.run.smithi188.stdout:172.21.0.2 0.0.0.0 255.255.255.255 UH 100 0 0 enp3s0f1 2023-11-26T07:18:14.757 INFO:teuthology.orchestra.run.smithi188.stdout:_gateway 0.0.0.0 255.255.255.255 UH 100 0 0 enp3s0f1 2023-11-26T07:18:14.757 INFO:teuthology.orchestra.run.smithi188.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2023-11-26T07:18:14.757 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-26T07:18:14.757 DEBUG:teuthology.orchestra.run.smithi188:> (cd / && exec bash -c ' 2023-11-26T07:18:14.757 DEBUG:teuthology.orchestra.run.smithi188:> set -e 2023-11-26T07:18:14.758 DEBUG:teuthology.orchestra.run.smithi188:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2023-11-26T07:18:14.758 DEBUG:teuthology.orchestra.run.smithi188:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2023-11-26T07:18:14.758 DEBUG:teuthology.orchestra.run.smithi188:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2023-11-26T07:18:14.758 DEBUG:teuthology.orchestra.run.smithi188:> ') 2023-11-26T07:18:14.848 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-26T07:18:14.848 DEBUG:teuthology.orchestra.run.smithi188:> ip netns list 2023-11-26T07:18:14.857 INFO:teuthology.orchestra.run.smithi188.stdout:ceph-ns--home-ubuntu-cephtest-mnt.1 (id: 0) 2023-11-26T07:18:14.858 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-26T07:18:14.858 DEBUG:teuthology.orchestra.run.smithi188:> ip netns list-id 2023-11-26T07:18:14.913 INFO:teuthology.orchestra.run.smithi188.stdout:nsid 0 (iproute2 netns name: ceph-ns--home-ubuntu-cephtest-mnt.1) 2023-11-26T07:18:14.914 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-26T07:18:14.914 DEBUG:teuthology.orchestra.run.smithi188:> (cd / && exec bash -c ' 2023-11-26T07:18:14.914 DEBUG:teuthology.orchestra.run.smithi188:> set -e 2023-11-26T07:18:14.914 DEBUG:teuthology.orchestra.run.smithi188:> sudo ip netns add ceph-ns--home-ubuntu-cephtest-mnt.3 2023-11-26T07:18:14.914 DEBUG:teuthology.orchestra.run.smithi188:> sudo ip netns set ceph-ns--home-ubuntu-cephtest-mnt.3 1 2023-11-26T07:18:14.914 DEBUG:teuthology.orchestra.run.smithi188:> ') 2023-11-26T07:18:14.996 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-26T07:18:14.996 DEBUG:teuthology.orchestra.run.smithi188:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.1 ip addr 2023-11-26T07:18:15.057 INFO:teuthology.orchestra.run.smithi188.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2023-11-26T07:18:15.057 INFO:teuthology.orchestra.run.smithi188.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2023-11-26T07:18:15.058 INFO:teuthology.orchestra.run.smithi188.stdout: inet 127.0.0.1/8 scope host lo 2023-11-26T07:18:15.058 INFO:teuthology.orchestra.run.smithi188.stdout: valid_lft forever preferred_lft forever 2023-11-26T07:18:15.058 INFO:teuthology.orchestra.run.smithi188.stdout: inet6 ::1/128 scope host 2023-11-26T07:18:15.058 INFO:teuthology.orchestra.run.smithi188.stdout: valid_lft forever preferred_lft forever 2023-11-26T07:18:15.058 INFO:teuthology.orchestra.run.smithi188.stdout:2: veth0@if8: mtu 1500 qdisc noqueue state UP group default qlen 1000 2023-11-26T07:18:15.058 INFO:teuthology.orchestra.run.smithi188.stdout: link/ether 62:2d:e9:1c:fa:ce brd ff:ff:ff:ff:ff:ff link-netnsid 0 2023-11-26T07:18:15.058 INFO:teuthology.orchestra.run.smithi188.stdout: inet 192.168.0.1/16 brd 192.168.255.255 scope global veth0 2023-11-26T07:18:15.058 INFO:teuthology.orchestra.run.smithi188.stdout: valid_lft forever preferred_lft forever 2023-11-26T07:18:15.059 INFO:teuthology.orchestra.run.smithi188.stdout: inet6 fe80::602d:e9ff:fe1c:face/64 scope link 2023-11-26T07:18:15.059 INFO:teuthology.orchestra.run.smithi188.stdout: valid_lft forever preferred_lft forever 2023-11-26T07:18:15.060 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-26T07:18:15.060 DEBUG:teuthology.orchestra.run.smithi188:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.1 ip addr 2023-11-26T07:18:15.122 INFO:teuthology.orchestra.run.smithi188.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2023-11-26T07:18:15.122 INFO:teuthology.orchestra.run.smithi188.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2023-11-26T07:18:15.122 INFO:teuthology.orchestra.run.smithi188.stdout: inet 127.0.0.1/8 scope host lo 2023-11-26T07:18:15.122 INFO:teuthology.orchestra.run.smithi188.stdout: valid_lft forever preferred_lft forever 2023-11-26T07:18:15.122 INFO:teuthology.orchestra.run.smithi188.stdout: inet6 ::1/128 scope host 2023-11-26T07:18:15.122 INFO:teuthology.orchestra.run.smithi188.stdout: valid_lft forever preferred_lft forever 2023-11-26T07:18:15.122 INFO:teuthology.orchestra.run.smithi188.stdout:2: veth0@if8: mtu 1500 qdisc noqueue state UP group default qlen 1000 2023-11-26T07:18:15.122 INFO:teuthology.orchestra.run.smithi188.stdout: link/ether 62:2d:e9:1c:fa:ce brd ff:ff:ff:ff:ff:ff link-netnsid 0 2023-11-26T07:18:15.123 INFO:teuthology.orchestra.run.smithi188.stdout: inet 192.168.0.1/16 brd 192.168.255.255 scope global veth0 2023-11-26T07:18:15.123 INFO:teuthology.orchestra.run.smithi188.stdout: valid_lft forever preferred_lft forever 2023-11-26T07:18:15.123 INFO:teuthology.orchestra.run.smithi188.stdout: inet6 fe80::602d:e9ff:fe1c:face/64 scope link 2023-11-26T07:18:15.123 INFO:teuthology.orchestra.run.smithi188.stdout: valid_lft forever preferred_lft forever 2023-11-26T07:18:15.124 INFO:tasks.cephfs.mount:Setuping the netns 'ceph-ns--home-ubuntu-cephtest-mnt.3' with 192.168.0.2/16 2023-11-26T07:18:15.124 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-26T07:18:15.124 DEBUG:teuthology.orchestra.run.smithi188:> (cd / && exec bash -c ' 2023-11-26T07:18:15.124 DEBUG:teuthology.orchestra.run.smithi188:> set -e 2023-11-26T07:18:15.124 DEBUG:teuthology.orchestra.run.smithi188:> sudo ip link add veth0 netns ceph-ns--home-ubuntu-cephtest-mnt.3 type veth peer name brx.1 2023-11-26T07:18:15.124 DEBUG:teuthology.orchestra.run.smithi188:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.3 ip addr add 192.168.0.2/16 brd 192.168.255.255 dev veth0 2023-11-26T07:18:15.124 DEBUG:teuthology.orchestra.run.smithi188:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.3 ip link set veth0 up 2023-11-26T07:18:15.125 DEBUG:teuthology.orchestra.run.smithi188:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.3 ip link set lo up 2023-11-26T07:18:15.125 DEBUG:teuthology.orchestra.run.smithi188:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.3 ip route add default via 192.168.255.254 2023-11-26T07:18:15.125 DEBUG:teuthology.orchestra.run.smithi188:> ') 2023-11-26T07:18:15.261 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-26T07:18:15.261 DEBUG:teuthology.orchestra.run.smithi188:> (cd / && exec bash -c ' 2023-11-26T07:18:15.261 DEBUG:teuthology.orchestra.run.smithi188:> set -e 2023-11-26T07:18:15.261 DEBUG:teuthology.orchestra.run.smithi188:> sudo ip link set brx.1 up 2023-11-26T07:18:15.261 DEBUG:teuthology.orchestra.run.smithi188:> sudo ip link set dev brx.1 master ceph-brx 2023-11-26T07:18:15.261 DEBUG:teuthology.orchestra.run.smithi188:> ') 2023-11-26T07:18:15.301 INFO:tasks.cephfs.fuse_mount:Client client.3 config is {} 2023-11-26T07:18:15.302 INFO:teuthology.orchestra.run:Running command with timeout 60 2023-11-26T07:18:15.302 DEBUG:teuthology.orchestra.run.smithi188:> mkdir -p -v /home/ubuntu/cephtest/mnt.3 2023-11-26T07:18:15.358 INFO:teuthology.orchestra.run.smithi188.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.3' 2023-11-26T07:18:15.358 INFO:teuthology.orchestra.run:Running command with timeout 60 2023-11-26T07:18:15.358 DEBUG:teuthology.orchestra.run.smithi188:> chmod 0000 /home/ubuntu/cephtest/mnt.3 2023-11-26T07:18:15.409 DEBUG:teuthology.orchestra.run.smithi188:> sudo modprobe fuse 2023-11-26T07:18:15.470 DEBUG:teuthology.orchestra.run.smithi188:> cat /proc/self/mounts | awk '{print $2}' 2023-11-26T07:18:15.525 INFO:teuthology.orchestra.run.smithi188.stdout:/sys 2023-11-26T07:18:15.525 INFO:teuthology.orchestra.run.smithi188.stdout:/proc 2023-11-26T07:18:15.525 INFO:teuthology.orchestra.run.smithi188.stdout:/dev 2023-11-26T07:18:15.525 INFO:teuthology.orchestra.run.smithi188.stdout:/dev/pts 2023-11-26T07:18:15.525 INFO:teuthology.orchestra.run.smithi188.stdout:/run 2023-11-26T07:18:15.525 INFO:teuthology.orchestra.run.smithi188.stdout:/ 2023-11-26T07:18:15.525 INFO:teuthology.orchestra.run.smithi188.stdout:/sys/kernel/security 2023-11-26T07:18:15.525 INFO:teuthology.orchestra.run.smithi188.stdout:/dev/shm 2023-11-26T07:18:15.526 INFO:teuthology.orchestra.run.smithi188.stdout:/run/lock 2023-11-26T07:18:15.526 INFO:teuthology.orchestra.run.smithi188.stdout:/sys/fs/cgroup 2023-11-26T07:18:15.526 INFO:teuthology.orchestra.run.smithi188.stdout:/sys/fs/pstore 2023-11-26T07:18:15.526 INFO:teuthology.orchestra.run.smithi188.stdout:/sys/fs/bpf 2023-11-26T07:18:15.526 INFO:teuthology.orchestra.run.smithi188.stdout:/proc/sys/fs/binfmt_misc 2023-11-26T07:18:15.526 INFO:teuthology.orchestra.run.smithi188.stdout:/dev/hugepages 2023-11-26T07:18:15.526 INFO:teuthology.orchestra.run.smithi188.stdout:/dev/mqueue 2023-11-26T07:18:15.526 INFO:teuthology.orchestra.run.smithi188.stdout:/sys/kernel/debug 2023-11-26T07:18:15.526 INFO:teuthology.orchestra.run.smithi188.stdout:/sys/kernel/tracing 2023-11-26T07:18:15.527 INFO:teuthology.orchestra.run.smithi188.stdout:/proc/fs/nfsd 2023-11-26T07:18:15.527 INFO:teuthology.orchestra.run.smithi188.stdout:/sys/fs/fuse/connections 2023-11-26T07:18:15.527 INFO:teuthology.orchestra.run.smithi188.stdout:/sys/kernel/config 2023-11-26T07:18:15.527 INFO:teuthology.orchestra.run.smithi188.stdout:/run/credentials/systemd-sysusers.service 2023-11-26T07:18:15.527 INFO:teuthology.orchestra.run.smithi188.stdout:/run/rpc_pipefs 2023-11-26T07:18:15.527 INFO:teuthology.orchestra.run.smithi188.stdout:/run/user/1000 2023-11-26T07:18:15.527 INFO:teuthology.orchestra.run.smithi188.stdout:/var/lib/ceph 2023-11-26T07:18:15.527 INFO:teuthology.orchestra.run.smithi188.stdout:/var/lib/ceph/osd/ceph-4 2023-11-26T07:18:15.527 INFO:teuthology.orchestra.run.smithi188.stdout:/var/lib/ceph/osd/ceph-5 2023-11-26T07:18:15.527 INFO:teuthology.orchestra.run.smithi188.stdout:/var/lib/ceph/osd/ceph-6 2023-11-26T07:18:15.527 INFO:teuthology.orchestra.run.smithi188.stdout:/var/lib/ceph/osd/ceph-7 2023-11-26T07:18:15.527 INFO:teuthology.orchestra.run.smithi188.stdout:/run/netns 2023-11-26T07:18:15.528 INFO:teuthology.orchestra.run.smithi188.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 2023-11-26T07:18:15.528 INFO:teuthology.orchestra.run.smithi188.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 2023-11-26T07:18:15.528 INFO:teuthology.orchestra.run.smithi188.stdout:/home/ubuntu/cephtest/mnt.1 2023-11-26T07:18:15.528 INFO:teuthology.orchestra.run.smithi188.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 2023-11-26T07:18:15.528 INFO:teuthology.orchestra.run.smithi188.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 2023-11-26T07:18:15.528 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-26T07:18:15.528 DEBUG:teuthology.orchestra.run.smithi188:> ls /sys/fs/fuse/connections 2023-11-26T07:18:15.577 INFO:teuthology.orchestra.run.smithi188.stdout:43 2023-11-26T07:18:15.578 INFO:tasks.cephfs.fuse_mount:Pre-mount connections: [43] 2023-11-26T07:18:15.578 DEBUG:teuthology.orchestra.run.smithi188:> (cd /home/ubuntu/cephtest && exec sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-fuse -f --admin-socket '/var/run/ceph/$cluster-$name.$pid.asok' /home/ubuntu/cephtest/mnt.3 --id 3) 2023-11-26T07:18:15.628 DEBUG:teuthology.orchestra.run.smithi188:> sudo modprobe fuse 2023-11-26T07:18:15.651 DEBUG:teuthology.orchestra.run.smithi188:> cat /proc/self/mounts | awk '{print $2}' 2023-11-26T07:18:15.703 INFO:teuthology.orchestra.run.smithi188.stdout:/sys 2023-11-26T07:18:15.703 INFO:teuthology.orchestra.run.smithi188.stdout:/proc 2023-11-26T07:18:15.703 INFO:teuthology.orchestra.run.smithi188.stdout:/dev 2023-11-26T07:18:15.703 INFO:teuthology.orchestra.run.smithi188.stdout:/dev/pts 2023-11-26T07:18:15.703 INFO:teuthology.orchestra.run.smithi188.stdout:/run 2023-11-26T07:18:15.704 INFO:teuthology.orchestra.run.smithi188.stdout:/ 2023-11-26T07:18:15.704 INFO:teuthology.orchestra.run.smithi188.stdout:/sys/kernel/security 2023-11-26T07:18:15.704 INFO:teuthology.orchestra.run.smithi188.stdout:/dev/shm 2023-11-26T07:18:15.704 INFO:teuthology.orchestra.run.smithi188.stdout:/run/lock 2023-11-26T07:18:15.704 INFO:teuthology.orchestra.run.smithi188.stdout:/sys/fs/cgroup 2023-11-26T07:18:15.704 INFO:teuthology.orchestra.run.smithi188.stdout:/sys/fs/pstore 2023-11-26T07:18:15.704 INFO:teuthology.orchestra.run.smithi188.stdout:/sys/fs/bpf 2023-11-26T07:18:15.704 INFO:teuthology.orchestra.run.smithi188.stdout:/proc/sys/fs/binfmt_misc 2023-11-26T07:18:15.704 INFO:teuthology.orchestra.run.smithi188.stdout:/dev/hugepages 2023-11-26T07:18:15.704 INFO:teuthology.orchestra.run.smithi188.stdout:/dev/mqueue 2023-11-26T07:18:15.704 INFO:teuthology.orchestra.run.smithi188.stdout:/sys/kernel/debug 2023-11-26T07:18:15.704 INFO:teuthology.orchestra.run.smithi188.stdout:/sys/kernel/tracing 2023-11-26T07:18:15.704 INFO:teuthology.orchestra.run.smithi188.stdout:/proc/fs/nfsd 2023-11-26T07:18:15.705 INFO:teuthology.orchestra.run.smithi188.stdout:/sys/fs/fuse/connections 2023-11-26T07:18:15.705 INFO:teuthology.orchestra.run.smithi188.stdout:/sys/kernel/config 2023-11-26T07:18:15.705 INFO:teuthology.orchestra.run.smithi188.stdout:/run/credentials/systemd-sysusers.service 2023-11-26T07:18:15.705 INFO:teuthology.orchestra.run.smithi188.stdout:/run/rpc_pipefs 2023-11-26T07:18:15.705 INFO:teuthology.orchestra.run.smithi188.stdout:/run/user/1000 2023-11-26T07:18:15.705 INFO:teuthology.orchestra.run.smithi188.stdout:/var/lib/ceph 2023-11-26T07:18:15.705 INFO:teuthology.orchestra.run.smithi188.stdout:/var/lib/ceph/osd/ceph-4 2023-11-26T07:18:15.705 INFO:teuthology.orchestra.run.smithi188.stdout:/var/lib/ceph/osd/ceph-5 2023-11-26T07:18:15.705 INFO:teuthology.orchestra.run.smithi188.stdout:/var/lib/ceph/osd/ceph-6 2023-11-26T07:18:15.705 INFO:teuthology.orchestra.run.smithi188.stdout:/var/lib/ceph/osd/ceph-7 2023-11-26T07:18:15.705 INFO:teuthology.orchestra.run.smithi188.stdout:/run/netns 2023-11-26T07:18:15.705 INFO:teuthology.orchestra.run.smithi188.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 2023-11-26T07:18:15.705 INFO:teuthology.orchestra.run.smithi188.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 2023-11-26T07:18:15.706 INFO:teuthology.orchestra.run.smithi188.stdout:/home/ubuntu/cephtest/mnt.1 2023-11-26T07:18:15.706 INFO:teuthology.orchestra.run.smithi188.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 2023-11-26T07:18:15.706 INFO:teuthology.orchestra.run.smithi188.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 2023-11-26T07:18:15.707 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-26T07:18:15.707 DEBUG:teuthology.orchestra.run.smithi188:> ls /sys/fs/fuse/connections 2023-11-26T07:18:15.724 INFO:tasks.cephfs.fuse_mount.ceph-fuse.3.smithi188.stderr:2023-11-26T07:18:15.718+0000 7f39befb4580 -1 init, newargv = 0x564df80dc330 newargc=15 2023-11-26T07:18:15.724 INFO:tasks.cephfs.fuse_mount.ceph-fuse.3.smithi188.stderr:ceph-fuse[44220]: starting ceph client 2023-11-26T07:18:15.731 INFO:tasks.cephfs.fuse_mount.ceph-fuse.3.smithi188.stderr:ceph-fuse[44220]: starting fuse 2023-11-26T07:18:15.734 INFO:teuthology.orchestra.run.smithi188.stdout:43 2023-11-26T07:18:15.734 INFO:teuthology.orchestra.run.smithi188.stdout:44 2023-11-26T07:18:15.734 INFO:tasks.cephfs.fuse_mount:Post-mount connections: [43, 44] 2023-11-26T07:18:15.734 DEBUG:teuthology.orchestra.run.smithi188:> sudo stdin-killer -- python3 -c ' 2023-11-26T07:18:15.735 DEBUG:teuthology.orchestra.run.smithi188:> import glob 2023-11-26T07:18:15.735 DEBUG:teuthology.orchestra.run.smithi188:> import re 2023-11-26T07:18:15.735 DEBUG:teuthology.orchestra.run.smithi188:> import os 2023-11-26T07:18:15.735 DEBUG:teuthology.orchestra.run.smithi188:> import subprocess 2023-11-26T07:18:15.735 DEBUG:teuthology.orchestra.run.smithi188:> 2023-11-26T07:18:15.735 DEBUG:teuthology.orchestra.run.smithi188:> def _find_admin_socket(client_name): 2023-11-26T07:18:15.735 DEBUG:teuthology.orchestra.run.smithi188:> asok_path = "/var/run/ceph/ceph-client.3.*.asok" 2023-11-26T07:18:15.735 DEBUG:teuthology.orchestra.run.smithi188:> files = glob.glob(asok_path) 2023-11-26T07:18:15.735 DEBUG:teuthology.orchestra.run.smithi188:> mountpoint = "/home/ubuntu/cephtest/mnt.3" 2023-11-26T07:18:15.735 DEBUG:teuthology.orchestra.run.smithi188:> 2023-11-26T07:18:15.735 DEBUG:teuthology.orchestra.run.smithi188:> # Given a non-glob path, it better be there 2023-11-26T07:18:15.735 DEBUG:teuthology.orchestra.run.smithi188:> if "*" not in asok_path: 2023-11-26T07:18:15.736 DEBUG:teuthology.orchestra.run.smithi188:> assert(len(files) == 1) 2023-11-26T07:18:15.736 DEBUG:teuthology.orchestra.run.smithi188:> return files[0] 2023-11-26T07:18:15.736 DEBUG:teuthology.orchestra.run.smithi188:> 2023-11-26T07:18:15.736 DEBUG:teuthology.orchestra.run.smithi188:> for f in files: 2023-11-26T07:18:15.736 DEBUG:teuthology.orchestra.run.smithi188:> pid = re.match(".*\.(\d+)\.asok$", f).group(1) 2023-11-26T07:18:15.736 DEBUG:teuthology.orchestra.run.smithi188:> if os.path.exists("/proc/{0}".format(pid)): 2023-11-26T07:18:15.736 DEBUG:teuthology.orchestra.run.smithi188:> with open("/proc/{0}/cmdline".format(pid), '"'"'r'"'"') as proc_f: 2023-11-26T07:18:15.736 DEBUG:teuthology.orchestra.run.smithi188:> contents = proc_f.read() 2023-11-26T07:18:15.736 DEBUG:teuthology.orchestra.run.smithi188:> if mountpoint in contents: 2023-11-26T07:18:15.736 DEBUG:teuthology.orchestra.run.smithi188:> return f 2023-11-26T07:18:15.736 DEBUG:teuthology.orchestra.run.smithi188:> raise RuntimeError("Client socket {0} not found".format(client_name)) 2023-11-26T07:18:15.736 DEBUG:teuthology.orchestra.run.smithi188:> 2023-11-26T07:18:15.737 DEBUG:teuthology.orchestra.run.smithi188:> print(_find_admin_socket("client.3")) 2023-11-26T07:18:15.737 DEBUG:teuthology.orchestra.run.smithi188:> ' 2023-11-26T07:18:15.858 INFO:teuthology.orchestra.run.smithi188.stdout:/var/run/ceph/ceph-client.3.44220.asok 2023-11-26T07:18:15.861 INFO:teuthology.orchestra.run.smithi188.stderr:2023-11-26T07:18:15 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2023-11-26T07:18:15.869 INFO:tasks.cephfs.fuse_mount:Found client admin socket at /var/run/ceph/ceph-client.3.44220.asok 2023-11-26T07:18:15.869 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-26T07:18:15.869 DEBUG:teuthology.orchestra.run.smithi188:> sudo ceph --admin-daemon /var/run/ceph/ceph-client.3.44220.asok status 2023-11-26T07:18:16.042 INFO:teuthology.orchestra.run.smithi188.stdout:{ 2023-11-26T07:18:16.042 INFO:teuthology.orchestra.run.smithi188.stdout: "metadata": { 2023-11-26T07:18:16.042 INFO:teuthology.orchestra.run.smithi188.stdout: "ceph_sha1": "44c2237fe77713214148c1fd622c872f346f1b99", 2023-11-26T07:18:16.042 INFO:teuthology.orchestra.run.smithi188.stdout: "ceph_version": "ceph version 18.0.0-7484-g44c2237f (44c2237fe77713214148c1fd622c872f346f1b99) reef (dev)", 2023-11-26T07:18:16.042 INFO:teuthology.orchestra.run.smithi188.stdout: "entity_id": "3", 2023-11-26T07:18:16.042 INFO:teuthology.orchestra.run.smithi188.stdout: "hostname": "smithi188", 2023-11-26T07:18:16.042 INFO:teuthology.orchestra.run.smithi188.stdout: "mount_point": "/home/ubuntu/cephtest/mnt.3", 2023-11-26T07:18:16.042 INFO:teuthology.orchestra.run.smithi188.stdout: "pid": "44220", 2023-11-26T07:18:16.043 INFO:teuthology.orchestra.run.smithi188.stdout: "root": "/" 2023-11-26T07:18:16.043 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2023-11-26T07:18:16.043 INFO:teuthology.orchestra.run.smithi188.stdout: "dentry_count": 0, 2023-11-26T07:18:16.043 INFO:teuthology.orchestra.run.smithi188.stdout: "dentry_pinned_count": 0, 2023-11-26T07:18:16.043 INFO:teuthology.orchestra.run.smithi188.stdout: "id": 4517, 2023-11-26T07:18:16.043 INFO:teuthology.orchestra.run.smithi188.stdout: "inst": { 2023-11-26T07:18:16.043 INFO:teuthology.orchestra.run.smithi188.stdout: "name": { 2023-11-26T07:18:16.043 INFO:teuthology.orchestra.run.smithi188.stdout: "type": "client", 2023-11-26T07:18:16.043 INFO:teuthology.orchestra.run.smithi188.stdout: "num": 4517 2023-11-26T07:18:16.043 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2023-11-26T07:18:16.043 INFO:teuthology.orchestra.run.smithi188.stdout: "addr": { 2023-11-26T07:18:16.043 INFO:teuthology.orchestra.run.smithi188.stdout: "type": "v1", 2023-11-26T07:18:16.044 INFO:teuthology.orchestra.run.smithi188.stdout: "addr": "192.168.0.2:0", 2023-11-26T07:18:16.044 INFO:teuthology.orchestra.run.smithi188.stdout: "nonce": 634544989 2023-11-26T07:18:16.044 INFO:teuthology.orchestra.run.smithi188.stdout: } 2023-11-26T07:18:16.044 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2023-11-26T07:18:16.044 INFO:teuthology.orchestra.run.smithi188.stdout: "addr": { 2023-11-26T07:18:16.044 INFO:teuthology.orchestra.run.smithi188.stdout: "type": "v1", 2023-11-26T07:18:16.044 INFO:teuthology.orchestra.run.smithi188.stdout: "addr": "192.168.0.2:0", 2023-11-26T07:18:16.044 INFO:teuthology.orchestra.run.smithi188.stdout: "nonce": 634544989 2023-11-26T07:18:16.044 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2023-11-26T07:18:16.044 INFO:teuthology.orchestra.run.smithi188.stdout: "inst_str": "client.4517 192.168.0.2:0/634544989", 2023-11-26T07:18:16.045 INFO:teuthology.orchestra.run.smithi188.stdout: "addr_str": "192.168.0.2:0/634544989", 2023-11-26T07:18:16.045 INFO:teuthology.orchestra.run.smithi188.stdout: "inode_count": 1, 2023-11-26T07:18:16.045 INFO:teuthology.orchestra.run.smithi188.stdout: "mds_epoch": 8, 2023-11-26T07:18:16.045 INFO:teuthology.orchestra.run.smithi188.stdout: "osd_epoch": 20, 2023-11-26T07:18:16.045 INFO:teuthology.orchestra.run.smithi188.stdout: "osd_epoch_barrier": 0, 2023-11-26T07:18:16.045 INFO:teuthology.orchestra.run.smithi188.stdout: "blocklisted": false, 2023-11-26T07:18:16.045 INFO:teuthology.orchestra.run.smithi188.stdout: "fs_name": "cephfs" 2023-11-26T07:18:16.045 INFO:teuthology.orchestra.run.smithi188.stdout:} 2023-11-26T07:18:16.057 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-26T07:18:16.057 DEBUG:teuthology.orchestra.run.smithi143:> stat --file-system '--printf=%T 2023-11-26T07:18:16.057 DEBUG:teuthology.orchestra.run.smithi143:> ' -- /home/ubuntu/cephtest/mnt.0 2023-11-26T07:18:16.067 INFO:teuthology.orchestra.run.smithi143.stdout:fuseblk 2023-11-26T07:18:16.067 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.0 2023-11-26T07:18:16.067 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-26T07:18:16.067 DEBUG:teuthology.orchestra.run.smithi143:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.0 2023-11-26T07:18:16.131 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-26T07:18:16.131 DEBUG:teuthology.orchestra.run.smithi143:> stat --file-system '--printf=%T 2023-11-26T07:18:16.131 DEBUG:teuthology.orchestra.run.smithi143:> ' -- /home/ubuntu/cephtest/mnt.2 2023-11-26T07:18:16.182 INFO:teuthology.orchestra.run.smithi143.stdout:fuseblk 2023-11-26T07:18:16.182 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.2 2023-11-26T07:18:16.182 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-26T07:18:16.182 DEBUG:teuthology.orchestra.run.smithi143:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.2 2023-11-26T07:18:16.248 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-26T07:18:16.248 DEBUG:teuthology.orchestra.run.smithi188:> stat --file-system '--printf=%T 2023-11-26T07:18:16.249 DEBUG:teuthology.orchestra.run.smithi188:> ' -- /home/ubuntu/cephtest/mnt.1 2023-11-26T07:18:16.258 INFO:teuthology.orchestra.run.smithi188.stdout:fuseblk 2023-11-26T07:18:16.258 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.1 2023-11-26T07:18:16.258 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-26T07:18:16.258 DEBUG:teuthology.orchestra.run.smithi188:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.1 2023-11-26T07:18:16.325 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-26T07:18:16.326 DEBUG:teuthology.orchestra.run.smithi188:> stat --file-system '--printf=%T 2023-11-26T07:18:16.326 DEBUG:teuthology.orchestra.run.smithi188:> ' -- /home/ubuntu/cephtest/mnt.3 2023-11-26T07:18:16.378 INFO:teuthology.orchestra.run.smithi188.stdout:fuseblk 2023-11-26T07:18:16.379 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.3 2023-11-26T07:18:16.379 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-26T07:18:16.379 DEBUG:teuthology.orchestra.run.smithi188:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.3 2023-11-26T07:18:16.445 INFO:teuthology.run_tasks:Running task cephfs_test_runner... 2023-11-26T07:18:16.465 DEBUG:teuthology.orchestra.run.smithi143:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-11-26T07:18:16.607 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:16.601+0000 7fc2f7816640 1 Processor -- start 2023-11-26T07:18:16.608 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:16.601+0000 7fc2f7816640 1 -- start start 2023-11-26T07:18:16.610 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:16.605+0000 7fc2f7816640 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7fc2f013e290 0x7fc2f00cf280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:18:16.610 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:16.605+0000 7fc2f7816640 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7fc2f00cf7c0 0x7fc2f015efa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:18:16.610 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:16.605+0000 7fc2f7816640 1 -- --> v1:172.21.15.188:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc2f006c520 con 0x7fc2f015f4e0 2023-11-26T07:18:16.610 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:16.605+0000 7fc2f7816640 1 -- --> v2:172.21.15.143:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc2f00cfc40 con 0x7fc2f013e290 2023-11-26T07:18:16.610 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:16.605+0000 7fc2f7816640 1 -- --> v2:172.21.15.188:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc2f006c800 con 0x7fc2f00cf7c0 2023-11-26T07:18:16.610 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:16.605+0000 7fc2f55b2640 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7fc2f013e290 0x7fc2f00cf280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:18:16.610 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:16.605+0000 7fc2f55b2640 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7fc2f013e290 0x7fc2f00cf280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:43792/0 (socket says 172.21.15.143:43792) 2023-11-26T07:18:16.610 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:16.605+0000 7fc2f55b2640 1 -- 172.21.15.143:0/2216718507 learned_addr learned my addr 172.21.15.143:0/2216718507 (peer_addr_for_me v2:172.21.15.143:0/0) 2023-11-26T07:18:16.610 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:16.605+0000 7fc2f4db1640 1 --2- 172.21.15.143:0/2216718507 >> v2:172.21.15.188:3300/0 conn(0x7fc2f00cf7c0 0x7fc2f015efa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:18:16.611 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:16.605+0000 7fc2f55b2640 1 -- 172.21.15.143:0/2216718507 >> v2:172.21.15.188:3300/0 conn(0x7fc2f00cf7c0 msgr2=0x7fc2f015efa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-26T07:18:16.611 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:16.605+0000 7fc2f55b2640 1 --2- 172.21.15.143:0/2216718507 >> v2:172.21.15.188:3300/0 conn(0x7fc2f00cf7c0 0x7fc2f015efa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:18:16.611 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:16.605+0000 7fc2f55b2640 1 -- 172.21.15.143:0/2216718507 >> v1:172.21.15.188:6789/0 conn(0x7fc2f015f4e0 legacy=0x7fc2f01408e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-26T07:18:16.611 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:16.605+0000 7fc2f55b2640 1 -- 172.21.15.143:0/2216718507 --> v2:172.21.15.143:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc2f006cae0 con 0x7fc2f013e290 2023-11-26T07:18:16.611 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:16.605+0000 7fc2f4db1640 1 --2- 172.21.15.143:0/2216718507 >> v2:172.21.15.188:3300/0 conn(0x7fc2f00cf7c0 0x7fc2f015efa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-26T07:18:16.612 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:16.605+0000 7fc2f55b2640 1 --2- 172.21.15.143:0/2216718507 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fc2f013e290 0x7fc2f00cf280 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7fc2f00725a0 tx=0x7fc2ec01c470 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c3200b334d01ae61 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:18:16.614 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:16.609+0000 7fc2e7fff640 1 -- 172.21.15.143:0/2216718507 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc2ec0ab670 con 0x7fc2f013e290 2023-11-26T07:18:16.614 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:16.609+0000 7fc2e7fff640 1 -- 172.21.15.143:0/2216718507 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc2ec0ab7d0 con 0x7fc2f013e290 2023-11-26T07:18:16.614 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:16.609+0000 7fc2e7fff640 1 -- 172.21.15.143:0/2216718507 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc2ec0abb60 con 0x7fc2f013e290 2023-11-26T07:18:16.614 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:16.609+0000 7fc2f7816640 1 -- 172.21.15.143:0/2216718507 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fc2f013e290 msgr2=0x7fc2f00cf280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:18:16.614 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:16.609+0000 7fc2f7816640 1 --2- 172.21.15.143:0/2216718507 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fc2f013e290 0x7fc2f00cf280 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7fc2f00725a0 tx=0x7fc2ec01c470 comp rx=0 tx=0).stop 2023-11-26T07:18:16.614 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:16.609+0000 7fc2f7816640 1 -- 172.21.15.143:0/2216718507 shutdown_connections 2023-11-26T07:18:16.614 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:16.609+0000 7fc2f7816640 1 --2- 172.21.15.143:0/2216718507 >> v2:172.21.15.188:3300/0 conn(0x7fc2f00cf7c0 0x7fc2f015efa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:18:16.614 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:16.609+0000 7fc2f7816640 1 --2- 172.21.15.143:0/2216718507 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fc2f013e290 0x7fc2f00cf280 unknown :-1 s=CLOSED pgs=175 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:18:16.615 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:16.609+0000 7fc2f7816640 1 -- 172.21.15.143:0/2216718507 >> 172.21.15.143:0/2216718507 conn(0x7fc2f006a890 msgr2=0x7fc2f006aca0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:18:16.615 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:16.609+0000 7fc2f7816640 1 -- 172.21.15.143:0/2216718507 shutdown_connections 2023-11-26T07:18:16.615 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:16.609+0000 7fc2f7816640 1 -- 172.21.15.143:0/2216718507 wait complete. 2023-11-26T07:18:16.615 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:16.609+0000 7fc2f7816640 1 Processor -- start 2023-11-26T07:18:16.615 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:16.609+0000 7fc2f7816640 1 -- start start 2023-11-26T07:18:16.618 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:16.609+0000 7fc2f7816640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fc2f00cf7c0 0x7fc2f00b32d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:18:16.618 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:16.609+0000 7fc2f7816640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fc2f013e290 0x7fc2f015df40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:18:16.618 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:16.609+0000 7fc2f7816640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fc2f015f4e0 0x7fc2f00b0dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:18:16.618 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:16.609+0000 7fc2f7816640 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc2f00ae950 con 0x7fc2f013e290 2023-11-26T07:18:16.618 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:16.609+0000 7fc2f7816640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc2f015f250 con 0x7fc2f015f4e0 2023-11-26T07:18:16.618 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:16.609+0000 7fc2f7816640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc2f015e9b0 con 0x7fc2f00cf7c0 2023-11-26T07:18:16.618 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:16.609+0000 7fc2f4db1640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fc2f013e290 0x7fc2f015df40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:18:16.618 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:16.609+0000 7fc2f4db1640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fc2f013e290 0x7fc2f015df40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:43808/0 (socket says 172.21.15.143:43808) 2023-11-26T07:18:16.619 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:16.609+0000 7fc2f4db1640 1 -- 172.21.15.143:0/3675835815 learned_addr learned my addr 172.21.15.143:0/3675835815 (peer_addr_for_me v2:172.21.15.143:0/0) 2023-11-26T07:18:16.619 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:16.613+0000 7fc2f55b2640 1 --2- 172.21.15.143:0/3675835815 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fc2f00cf7c0 0x7fc2f00b32d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:18:16.619 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:16.613+0000 7fc2f5db3640 1 --2- 172.21.15.143:0/3675835815 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fc2f015f4e0 0x7fc2f00b0dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:18:16.619 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:16.613+0000 7fc2f4db1640 1 -- 172.21.15.143:0/3675835815 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fc2f00cf7c0 msgr2=0x7fc2f00b32d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:18:16.619 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:16.613+0000 7fc2f4db1640 1 --2- 172.21.15.143:0/3675835815 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fc2f00cf7c0 0x7fc2f00b32d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:18:16.619 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:16.613+0000 7fc2f4db1640 1 -- 172.21.15.143:0/3675835815 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fc2f015f4e0 msgr2=0x7fc2f00b0dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:18:16.619 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:16.613+0000 7fc2f4db1640 1 --2- 172.21.15.143:0/3675835815 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fc2f015f4e0 0x7fc2f00b0dc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:18:16.619 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:16.613+0000 7fc2f4db1640 1 -- 172.21.15.143:0/3675835815 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc2ec0094d0 con 0x7fc2f013e290 2023-11-26T07:18:16.619 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:16.613+0000 7fc2f5db3640 1 --2- 172.21.15.143:0/3675835815 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fc2f015f4e0 0x7fc2f00b0dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-26T07:18:16.619 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:16.613+0000 7fc2f4db1640 1 --2- 172.21.15.143:0/3675835815 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fc2f013e290 0x7fc2f015df40 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7fc2e000c7c0 tx=0x7fc2e000cc80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:18:16.619 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:16.613+0000 7fc2e5ffb640 1 -- 172.21.15.143:0/3675835815 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc2e0007a30 con 0x7fc2f013e290 2023-11-26T07:18:16.619 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:16.613+0000 7fc2e5ffb640 1 -- 172.21.15.143:0/3675835815 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc2e0007b90 con 0x7fc2f013e290 2023-11-26T07:18:16.619 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:16.613+0000 7fc2e5ffb640 1 -- 172.21.15.143:0/3675835815 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc2e000ce70 con 0x7fc2f013e290 2023-11-26T07:18:16.620 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:16.613+0000 7fc2f7816640 1 -- 172.21.15.143:0/3675835815 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc2f015ec90 con 0x7fc2f013e290 2023-11-26T07:18:16.620 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:16.613+0000 7fc2f7816640 1 -- 172.21.15.143:0/3675835815 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc2f0070410 con 0x7fc2f013e290 2023-11-26T07:18:16.620 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:16.613+0000 7fc2f7816640 1 -- 172.21.15.143:0/3675835815 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc2f0070630 con 0x7fc2f013e290 2023-11-26T07:18:16.621 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:16.613+0000 7fc2e5ffb640 1 -- 172.21.15.143:0/3675835815 <== mon.0 v2:172.21.15.143:3300/0 4 ==== mgrmap(e 5) v1 ==== 85852+0+0 (secure 0 0 0) 0x7fc2e00113f0 con 0x7fc2f013e290 2023-11-26T07:18:16.621 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:16.613+0000 7fc2e5ffb640 1 --2- 172.21.15.143:0/3675835815 >> [v2:172.21.15.143:6832/2150557544,v1:172.21.15.143:6833/2150557544] conn(0x7fc2d4066470 0x7fc2d4068930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:18:16.621 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:16.613+0000 7fc2e5ffb640 1 -- 172.21.15.143:0/3675835815 <== mon.0 v2:172.21.15.143:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fc2e0084320 con 0x7fc2f013e290 2023-11-26T07:18:16.621 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:16.613+0000 7fc2f55b2640 1 --2- 172.21.15.143:0/3675835815 >> [v2:172.21.15.143:6832/2150557544,v1:172.21.15.143:6833/2150557544] conn(0x7fc2d4066470 0x7fc2d4068930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:18:16.621 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:16.617+0000 7fc2f55b2640 1 --2- 172.21.15.143:0/3675835815 >> [v2:172.21.15.143:6832/2150557544,v1:172.21.15.143:6833/2150557544] conn(0x7fc2d4066470 0x7fc2d4068930 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7fc2ec0af000 tx=0x7fc2ec0a7000 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:18:16.624 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:16.617+0000 7fc2e5ffb640 1 -- 172.21.15.143:0/3675835815 <== mon.0 v2:172.21.15.143:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172649 (secure 0 0 0) 0x7fc2e0051790 con 0x7fc2f013e290 2023-11-26T07:18:16.777 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:16.773+0000 7fc2f7816640 1 -- 172.21.15.143:0/3675835815 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fc2f0067680 con 0x7fc2f013e290 2023-11-26T07:18:16.778 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:16.773+0000 7fc2e5ffb640 1 -- 172.21.15.143:0/3675835815 <== mon.0 v2:172.21.15.143:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v20) v1 ==== 74+0+16541 (secure 0 0 0) 0x7fc2e0054db0 con 0x7fc2f013e290 2023-11-26T07:18:16.779 INFO:teuthology.orchestra.run.smithi143.stdout: 2023-11-26T07:18:16.779 INFO:teuthology.orchestra.run.smithi143.stdout:{"epoch":20,"fsid":"f5033060-2469-4a07-95fe-2b76f5404bc1","created":"2023-11-26T07:17:28.613663+0000","modified":"2023-11-26T07:17:55.505435+0000","last_up_change":"2023-11-26T07:17:41.157813+0000","last_in_change":"2023-11-26T07:17:31.181658+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-11-26T07:17:41.758597+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2023-11-26T07:17:51.878332+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2023-11-26T07:17:52.871396+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"ebbee2c6-023e-44b9-9f1d-6f7bfd309730","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6817","nonce":639807369},{"type":"v1","addr":"172.21.15.143:6819","nonce":639807369}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6821","nonce":639807369},{"type":"v1","addr":"172.21.15.143:6823","nonce":639807369}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6830","nonce":639807369},{"type":"v1","addr":"172.21.15.143:6831","nonce":639807369}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6825","nonce":639807369},{"type":"v1","addr":"172.21.15.143:6828","nonce":639807369}]},"public_addr":"172.21.15.143:6819/639807369","cluster_addr":"172.21.15.143:6823/639807369","heartbeat_back_addr":"172.21.15.143:6831/639807369","heartbeat_front_addr":"172.21.15.143:6828/639807369","state":["exists","up"]},{"osd":1,"uuid":"fec4b8be-7122-49e9-9a19-5850a0700001","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6816","nonce":1144942530},{"type":"v1","addr":"172.21.15.143:6818","nonce":1144942530}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6820","nonce":1144942530},{"type":"v1","addr":"172.21.15.143:6822","nonce":1144942530}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6827","nonce":1144942530},{"type":"v1","addr":"172.21.15.143:6829","nonce":1144942530}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6824","nonce":1144942530},{"type":"v1","addr":"172.21.15.143:6826","nonce":1144942530}]},"public_addr":"172.21.15.143:6818/1144942530","cluster_addr":"172.21.15.143:6822/1144942530","heartbeat_back_addr":"172.21.15.143:6829/1144942530","heartbeat_front_addr":"172.21.15.143:6826/1144942530","state":["exists","up"]},{"osd":2,"uuid":"2888af5b-f933-46ca-8f95-6dd494ff43f6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6808","nonce":3909095186},{"type":"v1","addr":"172.21.15.143:6809","nonce":3909095186}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6810","nonce":3909095186},{"type":"v1","addr":"172.21.15.143:6811","nonce":3909095186}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6814","nonce":3909095186},{"type":"v1","addr":"172.21.15.143:6815","nonce":3909095186}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6812","nonce":3909095186},{"type":"v1","addr":"172.21.15.143:6813","nonce":3909095186}]},"public_addr":"172.21.15.143:6809/3909095186","cluster_addr":"172.21.15.143:6811/3909095186","heartbeat_back_addr":"172.21.15.143:6815/3909095186","heartbeat_front_addr":"172.21.15.143:6813/3909095186","state":["exists","up"]},{"osd":3,"uuid":"d1f52435-7bc1-4081-9550-9cd1ba55f549","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6800","nonce":3854210991},{"type":"v1","addr":"172.21.15.143:6801","nonce":3854210991}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6802","nonce":3854210991},{"type":"v1","addr":"172.21.15.143:6803","nonce":3854210991}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6806","nonce":3854210991},{"type":"v1","addr":"172.21.15.143:6807","nonce":3854210991}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6804","nonce":3854210991},{"type":"v1","addr":"172.21.15.143:6805","nonce":3854210991}]},"public_addr":"172.21.15.143:6801/3854210991","cluster_addr":"172.21.15.143:6803/3854210991","heartbeat_back_addr":"172.21.15.143:6807/3854210991","heartbeat_front_addr":"172.21.15.143:6805/3854210991","state":["exists","up"]},{"osd":4,"uuid":"e2ae8b6e-6e25-49b2-bcbf-aacce68f06a3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6800","nonce":3924606902},{"type":"v1","addr":"172.21.15.188:6801","nonce":3924606902}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6802","nonce":3924606902},{"type":"v1","addr":"172.21.15.188:6803","nonce":3924606902}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6806","nonce":3924606902},{"type":"v1","addr":"172.21.15.188:6807","nonce":3924606902}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6804","nonce":3924606902},{"type":"v1","addr":"172.21.15.188:6805","nonce":3924606902}]},"public_addr":"172.21.15.188:6801/3924606902","cluster_addr":"172.21.15.188:6803/3924606902","heartbeat_back_addr":"172.21.15.188:6807/3924606902","heartbeat_front_addr":"172.21.15.188:6805/3924606902","state":["exists","up"]},{"osd":5,"uuid":"3cec8a32-7698-4c36-8e89-e4ba11934f33","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6813","nonce":186841544},{"type":"v1","addr":"172.21.15.188:6815","nonce":186841544}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6817","nonce":186841544},{"type":"v1","addr":"172.21.15.188:6818","nonce":186841544}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6822","nonce":186841544},{"type":"v1","addr":"172.21.15.188:6823","nonce":186841544}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6820","nonce":186841544},{"type":"v1","addr":"172.21.15.188:6821","nonce":186841544}]},"public_addr":"172.21.15.188:6815/186841544","cluster_addr":"172.21.15.188:6818/186841544","heartbeat_back_addr":"172.21.15.188:6823/186841544","heartbeat_front_addr":"172.21.15.188:6821/186841544","state":["exists","up"]},{"osd":6,"uuid":"58a65d5d-e090-473a-81b4-ad38e85029f9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6808","nonce":2153499405},{"type":"v1","addr":"172.21.15.188:6809","nonce":2153499405}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6810","nonce":2153499405},{"type":"v1","addr":"172.21.15.188:6811","nonce":2153499405}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6816","nonce":2153499405},{"type":"v1","addr":"172.21.15.188:6819","nonce":2153499405}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6812","nonce":2153499405},{"type":"v1","addr":"172.21.15.188:6814","nonce":2153499405}]},"public_addr":"172.21.15.188:6809/2153499405","cluster_addr":"172.21.15.188:6811/2153499405","heartbeat_back_addr":"172.21.15.188:6819/2153499405","heartbeat_front_addr":"172.21.15.188:6814/2153499405","state":["exists","up"]},{"osd":7,"uuid":"ea5b7f70-3554-4cf8-ac30-f7c59b80678d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6824","nonce":2722096729},{"type":"v1","addr":"172.21.15.188:6825","nonce":2722096729}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6826","nonce":2722096729},{"type":"v1","addr":"172.21.15.188:6827","nonce":2722096729}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6830","nonce":2722096729},{"type":"v1","addr":"172.21.15.188:6831","nonce":2722096729}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6828","nonce":2722096729},{"type":"v1","addr":"172.21.15.188:6829","nonce":2722096729}]},"public_addr":"172.21.15.188:6825/2722096729","cluster_addr":"172.21.15.188:6827/2722096729","heartbeat_back_addr":"172.21.15.188:6831/2722096729","heartbeat_front_addr":"172.21.15.188:6829/2722096729","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-26T07:17:36.995032+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-26T07:17:36.932266+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-26T07:17:36.829304+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-26T07:17:36.804297+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-26T07:17:36.667892+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-26T07:17:36.964868+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-26T07:17:36.827332+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-26T07:17:36.893179+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-11-26T07:18:16.781 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:16.773+0000 7fc2f7816640 1 -- 172.21.15.143:0/3675835815 >> [v2:172.21.15.143:6832/2150557544,v1:172.21.15.143:6833/2150557544] conn(0x7fc2d4066470 msgr2=0x7fc2d4068930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:18:16.781 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:16.773+0000 7fc2f7816640 1 --2- 172.21.15.143:0/3675835815 >> [v2:172.21.15.143:6832/2150557544,v1:172.21.15.143:6833/2150557544] conn(0x7fc2d4066470 0x7fc2d4068930 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7fc2ec0af000 tx=0x7fc2ec0a7000 comp rx=0 tx=0).stop 2023-11-26T07:18:16.781 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:16.773+0000 7fc2f7816640 1 -- 172.21.15.143:0/3675835815 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fc2f013e290 msgr2=0x7fc2f015df40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:18:16.781 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:16.773+0000 7fc2f7816640 1 --2- 172.21.15.143:0/3675835815 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fc2f013e290 0x7fc2f015df40 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7fc2e000c7c0 tx=0x7fc2e000cc80 comp rx=0 tx=0).stop 2023-11-26T07:18:16.781 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:16.777+0000 7fc2f7816640 1 -- 172.21.15.143:0/3675835815 shutdown_connections 2023-11-26T07:18:16.781 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:16.777+0000 7fc2f7816640 1 --2- 172.21.15.143:0/3675835815 >> [v2:172.21.15.143:6832/2150557544,v1:172.21.15.143:6833/2150557544] conn(0x7fc2d4066470 0x7fc2d4068930 unknown :-1 s=CLOSED pgs=121 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:18:16.782 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:16.777+0000 7fc2f7816640 1 --2- 172.21.15.143:0/3675835815 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fc2f015f4e0 0x7fc2f00b0dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:18:16.782 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:16.777+0000 7fc2f7816640 1 --2- 172.21.15.143:0/3675835815 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fc2f013e290 0x7fc2f015df40 unknown :-1 s=CLOSED pgs=176 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:18:16.782 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:16.777+0000 7fc2f7816640 1 --2- 172.21.15.143:0/3675835815 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fc2f00cf7c0 0x7fc2f00b32d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:18:16.782 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:16.777+0000 7fc2f7816640 1 -- 172.21.15.143:0/3675835815 >> 172.21.15.143:0/3675835815 conn(0x7fc2f006a890 msgr2=0x7fc2f015b710 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:18:16.782 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:16.777+0000 7fc2f7816640 1 -- 172.21.15.143:0/3675835815 shutdown_connections 2023-11-26T07:18:16.782 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:16.777+0000 7fc2f7816640 1 -- 172.21.15.143:0/3675835815 wait complete. 2023-11-26T07:18:16.800 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-11-26T07:17:41.758597+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2023-11-26T07:17:51.878332+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2023-11-26T07:17:52.871396+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '20', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2023-11-26T07:18:16.800 DEBUG:teuthology.orchestra.run.smithi143:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-11-26T07:18:16.939 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:16.933+0000 7fd2f7c5a640 1 Processor -- start 2023-11-26T07:18:16.939 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:16.933+0000 7fd2f7c5a640 1 -- start start 2023-11-26T07:18:16.941 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:16.937+0000 7fd2f7c5a640 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7fd2f0161cf0 0x7fd2f006bae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:18:16.941 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:16.937+0000 7fd2f7c5a640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fd2f00cdd10 0x7fd2f006d950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:18:16.941 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:16.937+0000 7fd2f7c5a640 1 -- --> v1:172.21.15.188:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd2f00cd610 con 0x7fd2f0070ba0 2023-11-26T07:18:16.941 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:16.937+0000 7fd2f7c5a640 1 -- --> v2:172.21.15.143:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd2f00cd8f0 con 0x7fd2f0161cf0 2023-11-26T07:18:16.941 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:16.937+0000 7fd2f7c5a640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd2f0161210 con 0x7fd2f00cdd10 2023-11-26T07:18:16.942 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:16.937+0000 7fd2f51f5640 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7fd2f0161cf0 0x7fd2f006bae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:18:16.942 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:16.937+0000 7fd2f51f5640 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7fd2f0161cf0 0x7fd2f006bae0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:43810/0 (socket says 172.21.15.143:43810) 2023-11-26T07:18:16.942 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:16.937+0000 7fd2f51f5640 1 -- 172.21.15.143:0/1724669629 learned_addr learned my addr 172.21.15.143:0/1724669629 (peer_addr_for_me v2:172.21.15.143:0/0) 2023-11-26T07:18:16.942 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:16.937+0000 7fd2f61f7640 1 --2- 172.21.15.143:0/1724669629 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fd2f00cdd10 0x7fd2f006d950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:18:16.942 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:16.937+0000 7fd2f51f5640 1 -- 172.21.15.143:0/1724669629 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fd2f00cdd10 msgr2=0x7fd2f006d950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-26T07:18:16.943 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:16.937+0000 7fd2f51f5640 1 --2- 172.21.15.143:0/1724669629 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fd2f00cdd10 0x7fd2f006d950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:18:16.943 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:16.937+0000 7fd2f51f5640 1 -- 172.21.15.143:0/1724669629 >> v1:172.21.15.188:6789/0 conn(0x7fd2f0070ba0 legacy=0x7fd2f0070f80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:18:16.943 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:16.937+0000 7fd2f51f5640 1 -- 172.21.15.143:0/1724669629 --> v2:172.21.15.143:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd2f01614f0 con 0x7fd2f0161cf0 2023-11-26T07:18:16.945 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:16.937+0000 7fd2f51f5640 1 --2- 172.21.15.143:0/1724669629 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fd2f0161cf0 0x7fd2f006bae0 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7fd2e0009870 tx=0x7fd2e001c540 comp rx=0 tx=0).ready entity=mon.0 client_cookie=57c5aca3867e3ab server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:18:16.945 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:16.937+0000 7fd2f49f4640 1 -- 172.21.15.143:0/1724669629 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd2e00a9620 con 0x7fd2f0161cf0 2023-11-26T07:18:16.945 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:16.941+0000 7fd2f49f4640 1 -- 172.21.15.143:0/1724669629 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd2e00a9780 con 0x7fd2f0161cf0 2023-11-26T07:18:16.945 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:16.941+0000 7fd2f49f4640 1 -- 172.21.15.143:0/1724669629 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd2e00a9b10 con 0x7fd2f0161cf0 2023-11-26T07:18:16.946 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:16.941+0000 7fd2f7c5a640 1 -- 172.21.15.143:0/1724669629 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fd2f0161cf0 msgr2=0x7fd2f006bae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:18:16.946 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:16.941+0000 7fd2f7c5a640 1 --2- 172.21.15.143:0/1724669629 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fd2f0161cf0 0x7fd2f006bae0 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7fd2e0009870 tx=0x7fd2e001c540 comp rx=0 tx=0).stop 2023-11-26T07:18:16.946 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:16.941+0000 7fd2f7c5a640 1 -- 172.21.15.143:0/1724669629 shutdown_connections 2023-11-26T07:18:16.946 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:16.941+0000 7fd2f7c5a640 1 --2- 172.21.15.143:0/1724669629 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fd2f0161cf0 0x7fd2f006bae0 unknown :-1 s=CLOSED pgs=177 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:18:16.947 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:16.941+0000 7fd2f7c5a640 1 --2- 172.21.15.143:0/1724669629 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fd2f00cdd10 0x7fd2f006d950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:18:16.947 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:16.941+0000 7fd2f7c5a640 1 -- 172.21.15.143:0/1724669629 >> 172.21.15.143:0/1724669629 conn(0x7fd2f00cf710 msgr2=0x7fd2f00cfb20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:18:16.947 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:16.941+0000 7fd2f7c5a640 1 -- 172.21.15.143:0/1724669629 shutdown_connections 2023-11-26T07:18:16.947 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:16.941+0000 7fd2f7c5a640 1 -- 172.21.15.143:0/1724669629 wait complete. 2023-11-26T07:18:16.952 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:16.941+0000 7fd2f7c5a640 1 Processor -- start 2023-11-26T07:18:16.952 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:16.941+0000 7fd2f7c5a640 1 -- start start 2023-11-26T07:18:16.952 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:16.941+0000 7fd2f7c5a640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fd2f0070ba0 0x7fd2f0150c40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:18:16.952 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:16.941+0000 7fd2f7c5a640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fd2f00cdd10 0x7fd2f014f4b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:18:16.952 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:16.941+0000 7fd2f7c5a640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fd2f0161cf0 0x7fd2f00afa60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:18:16.952 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:16.941+0000 7fd2f7c5a640 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd2f0153e60 con 0x7fd2f0161cf0 2023-11-26T07:18:16.952 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:16.941+0000 7fd2f7c5a640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd2f014fe80 con 0x7fd2f00cdd10 2023-11-26T07:18:16.952 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:16.941+0000 7fd2f7c5a640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd2f0150160 con 0x7fd2f0070ba0 2023-11-26T07:18:16.953 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:16.945+0000 7fd2f61f7640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fd2f0161cf0 0x7fd2f00afa60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:18:16.953 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:16.945+0000 7fd2f59f6640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fd2f0070ba0 0x7fd2f0150c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:18:16.953 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:16.945+0000 7fd2f61f7640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fd2f0161cf0 0x7fd2f00afa60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:43820/0 (socket says 172.21.15.143:43820) 2023-11-26T07:18:16.953 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:16.945+0000 7fd2f61f7640 1 -- 172.21.15.143:0/883251805 learned_addr learned my addr 172.21.15.143:0/883251805 (peer_addr_for_me v2:172.21.15.143:0/0) 2023-11-26T07:18:16.953 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:16.945+0000 7fd2f51f5640 1 --2- 172.21.15.143:0/883251805 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fd2f00cdd10 0x7fd2f014f4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:18:16.953 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:16.945+0000 7fd2f61f7640 1 -- 172.21.15.143:0/883251805 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fd2f0070ba0 msgr2=0x7fd2f0150c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:18:16.953 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:16.945+0000 7fd2f61f7640 1 --2- 172.21.15.143:0/883251805 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fd2f0070ba0 0x7fd2f0150c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:18:16.954 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:16.945+0000 7fd2f61f7640 1 -- 172.21.15.143:0/883251805 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fd2f00cdd10 msgr2=0x7fd2f014f4b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:18:16.954 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:16.945+0000 7fd2f61f7640 1 --2- 172.21.15.143:0/883251805 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fd2f00cdd10 0x7fd2f014f4b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:18:16.954 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:16.945+0000 7fd2f61f7640 1 -- 172.21.15.143:0/883251805 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd2e00094d0 con 0x7fd2f0161cf0 2023-11-26T07:18:16.954 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:16.945+0000 7fd2f61f7640 1 --2- 172.21.15.143:0/883251805 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fd2f0161cf0 0x7fd2f00afa60 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7fd2e800efc0 tx=0x7fd2e800c490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:18:16.954 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:16.945+0000 7fd2e67fc640 1 -- 172.21.15.143:0/883251805 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd2e8014480 con 0x7fd2f0161cf0 2023-11-26T07:18:16.954 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:16.945+0000 7fd2e67fc640 1 -- 172.21.15.143:0/883251805 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd2e800f040 con 0x7fd2f0161cf0 2023-11-26T07:18:16.954 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:16.945+0000 7fd2f7c5a640 1 -- 172.21.15.143:0/883251805 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd2f0150440 con 0x7fd2f0161cf0 2023-11-26T07:18:16.955 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:16.945+0000 7fd2e67fc640 1 -- 172.21.15.143:0/883251805 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd2e8014820 con 0x7fd2f0161cf0 2023-11-26T07:18:16.955 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:16.945+0000 7fd2f7c5a640 1 -- 172.21.15.143:0/883251805 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd2f0151950 con 0x7fd2f0161cf0 2023-11-26T07:18:16.955 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:16.945+0000 7fd2f7c5a640 1 -- 172.21.15.143:0/883251805 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd2f0150960 con 0x7fd2f0161cf0 2023-11-26T07:18:16.955 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:16.945+0000 7fd2e67fc640 1 -- 172.21.15.143:0/883251805 <== mon.0 v2:172.21.15.143:3300/0 4 ==== mgrmap(e 5) v1 ==== 85852+0+0 (secure 0 0 0) 0x7fd2e8014980 con 0x7fd2f0161cf0 2023-11-26T07:18:16.956 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:16.945+0000 7fd2e67fc640 1 --2- 172.21.15.143:0/883251805 >> [v2:172.21.15.143:6832/2150557544,v1:172.21.15.143:6833/2150557544] conn(0x7fd2d4066470 0x7fd2d4068930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:18:16.956 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:16.945+0000 7fd2e67fc640 1 -- 172.21.15.143:0/883251805 <== mon.0 v2:172.21.15.143:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fd2e8084c60 con 0x7fd2f0161cf0 2023-11-26T07:18:16.956 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:16.949+0000 7fd2f59f6640 1 --2- 172.21.15.143:0/883251805 >> [v2:172.21.15.143:6832/2150557544,v1:172.21.15.143:6833/2150557544] conn(0x7fd2d4066470 0x7fd2d4068930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:18:16.956 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:16.949+0000 7fd2f59f6640 1 --2- 172.21.15.143:0/883251805 >> [v2:172.21.15.143:6832/2150557544,v1:172.21.15.143:6833/2150557544] conn(0x7fd2d4066470 0x7fd2d4068930 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7fd2f01541e0 tx=0x7fd2ec009040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:18:16.956 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:16.949+0000 7fd2e67fc640 1 -- 172.21.15.143:0/883251805 <== mon.0 v2:172.21.15.143:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172649 (secure 0 0 0) 0x7fd2f0150960 con 0x7fd2f0161cf0 2023-11-26T07:18:17.107 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:17.101+0000 7fd2f7c5a640 1 -- 172.21.15.143:0/883251805 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7fd2f0150960 con 0x7fd2f0161cf0 2023-11-26T07:18:17.107 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:17.101+0000 7fd2e67fc640 1 -- 172.21.15.143:0/883251805 <== mon.0 v2:172.21.15.143:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v20) v1 ==== 93+0+10 (secure 0 0 0) 0x7fd2e8018020 con 0x7fd2f0161cf0 2023-11-26T07:18:17.107 INFO:teuthology.orchestra.run.smithi143.stdout:pg_num: 1 2023-11-26T07:18:17.109 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:17.105+0000 7fd2f7c5a640 1 -- 172.21.15.143:0/883251805 >> [v2:172.21.15.143:6832/2150557544,v1:172.21.15.143:6833/2150557544] conn(0x7fd2d4066470 msgr2=0x7fd2d4068930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:18:17.110 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:17.105+0000 7fd2f7c5a640 1 --2- 172.21.15.143:0/883251805 >> [v2:172.21.15.143:6832/2150557544,v1:172.21.15.143:6833/2150557544] conn(0x7fd2d4066470 0x7fd2d4068930 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7fd2f01541e0 tx=0x7fd2ec009040 comp rx=0 tx=0).stop 2023-11-26T07:18:17.110 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:17.105+0000 7fd2f7c5a640 1 -- 172.21.15.143:0/883251805 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fd2f0161cf0 msgr2=0x7fd2f00afa60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:18:17.110 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:17.105+0000 7fd2f7c5a640 1 --2- 172.21.15.143:0/883251805 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fd2f0161cf0 0x7fd2f00afa60 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7fd2e800efc0 tx=0x7fd2e800c490 comp rx=0 tx=0).stop 2023-11-26T07:18:17.110 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:17.105+0000 7fd2f7c5a640 1 -- 172.21.15.143:0/883251805 shutdown_connections 2023-11-26T07:18:17.110 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:17.105+0000 7fd2f7c5a640 1 --2- 172.21.15.143:0/883251805 >> [v2:172.21.15.143:6832/2150557544,v1:172.21.15.143:6833/2150557544] conn(0x7fd2d4066470 0x7fd2d4068930 unknown :-1 s=CLOSED pgs=122 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:18:17.110 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:17.105+0000 7fd2f7c5a640 1 --2- 172.21.15.143:0/883251805 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fd2f0161cf0 0x7fd2f00afa60 unknown :-1 s=CLOSED pgs=178 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:18:17.110 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:17.105+0000 7fd2f7c5a640 1 --2- 172.21.15.143:0/883251805 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fd2f00cdd10 0x7fd2f014f4b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:18:17.110 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:17.105+0000 7fd2f7c5a640 1 --2- 172.21.15.143:0/883251805 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fd2f0070ba0 0x7fd2f0150c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:18:17.110 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:17.105+0000 7fd2f7c5a640 1 -- 172.21.15.143:0/883251805 >> 172.21.15.143:0/883251805 conn(0x7fd2f00cf710 msgr2=0x7fd2f015b7f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:18:17.110 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:17.105+0000 7fd2f7c5a640 1 -- 172.21.15.143:0/883251805 shutdown_connections 2023-11-26T07:18:17.110 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:17.105+0000 7fd2f7c5a640 1 -- 172.21.15.143:0/883251805 wait complete. 2023-11-26T07:18:17.128 DEBUG:teuthology.orchestra.run.smithi143:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-11-26T07:18:17.269 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:17.261+0000 7f91e91e7640 1 Processor -- start 2023-11-26T07:18:17.269 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:17.261+0000 7f91e91e7640 1 -- start start 2023-11-26T07:18:17.270 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:17.265+0000 7f91e91e7640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f91e415d6e0 0x7f91e40d0df0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:18:17.270 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:17.265+0000 7f91e91e7640 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7f91e40b2160 0x7f91e40d14b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:18:17.270 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:17.265+0000 7f91e91e7640 1 -- --> v1:172.21.15.143:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f91e40cf0e0 con 0x7f91e40d19f0 2023-11-26T07:18:17.270 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:17.265+0000 7f91e91e7640 1 -- --> v2:172.21.15.143:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f91e415dac0 con 0x7f91e40b2160 2023-11-26T07:18:17.270 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:17.265+0000 7f91e91e7640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f91e40d1e80 con 0x7f91e415d6e0 2023-11-26T07:18:17.271 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:17.265+0000 7f91e359e640 1 --1- >> v1:172.21.15.143:6789/0 conn(0x7f91e40d19f0 0x7f91e40cfbf0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.143:6789/0 says I am v1:172.21.15.143:43862/0 (socket says 172.21.15.143:43862) 2023-11-26T07:18:17.271 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:17.265+0000 7f91e259c640 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7f91e40b2160 0x7f91e40d14b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:18:17.271 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:17.265+0000 7f91e359e640 1 -- 172.21.15.143:0/954397011 learned_addr learned my addr 172.21.15.143:0/954397011 (peer_addr_for_me v1:172.21.15.143:0/0) 2023-11-26T07:18:17.271 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:17.265+0000 7f91e2d9d640 1 --2- 172.21.15.143:0/954397011 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f91e415d6e0 0x7f91e40d0df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:18:17.271 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:17.265+0000 7f91e259c640 1 -- 172.21.15.143:0/954397011 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f91e415d6e0 msgr2=0x7f91e40d0df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-26T07:18:17.271 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:17.265+0000 7f91e259c640 1 --2- 172.21.15.143:0/954397011 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f91e415d6e0 0x7f91e40d0df0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:18:17.271 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:17.265+0000 7f91e259c640 1 -- 172.21.15.143:0/954397011 >> v1:172.21.15.143:6789/0 conn(0x7f91e40d19f0 legacy=0x7f91e40cfbf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:18:17.271 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:17.265+0000 7f91e259c640 1 -- 172.21.15.143:0/954397011 --> v2:172.21.15.143:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f91e40ceb20 con 0x7f91e40b2160 2023-11-26T07:18:17.271 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:17.265+0000 7f91e1d9b640 1 -- 172.21.15.143:0/954397011 <== mon.0 v1:172.21.15.143:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 360008871 0 0) 0x7f91e40cf0e0 con 0x7f91e40d19f0 2023-11-26T07:18:17.273 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:17.269+0000 7f91e259c640 1 --2- 172.21.15.143:0/954397011 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f91e40b2160 0x7f91e40d14b0 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7f91d80097c0 tx=0x7f91d801c480 comp rx=0 tx=0).ready entity=mon.0 client_cookie=54138b58cdf62f91 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:18:17.274 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:17.269+0000 7f91e1d9b640 1 -- 172.21.15.143:0/954397011 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f91d80ab600 con 0x7f91e40b2160 2023-11-26T07:18:17.274 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:17.269+0000 7f91e1d9b640 1 -- 172.21.15.143:0/954397011 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f91d80a7050 con 0x7f91e40b2160 2023-11-26T07:18:17.274 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:17.269+0000 7f91e1d9b640 1 -- 172.21.15.143:0/954397011 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f91d80ab980 con 0x7f91e40b2160 2023-11-26T07:18:17.275 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:17.269+0000 7f91e91e7640 1 -- 172.21.15.143:0/954397011 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f91e40b2160 msgr2=0x7f91e40d14b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:18:17.275 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:17.269+0000 7f91e91e7640 1 --2- 172.21.15.143:0/954397011 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f91e40b2160 0x7f91e40d14b0 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7f91d80097c0 tx=0x7f91d801c480 comp rx=0 tx=0).stop 2023-11-26T07:18:17.275 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:17.269+0000 7f91e91e7640 1 -- 172.21.15.143:0/954397011 shutdown_connections 2023-11-26T07:18:17.275 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:17.269+0000 7f91e91e7640 1 --2- 172.21.15.143:0/954397011 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f91e40b2160 0x7f91e40d14b0 unknown :-1 s=CLOSED pgs=180 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:18:17.275 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:17.269+0000 7f91e91e7640 1 --2- 172.21.15.143:0/954397011 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f91e415d6e0 0x7f91e40d0df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:18:17.275 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:17.269+0000 7f91e91e7640 1 -- 172.21.15.143:0/954397011 >> 172.21.15.143:0/954397011 conn(0x7f91e409c550 msgr2=0x7f91e40d3220 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:18:17.275 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:17.269+0000 7f91e91e7640 1 -- 172.21.15.143:0/954397011 shutdown_connections 2023-11-26T07:18:17.275 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:17.269+0000 7f91e91e7640 1 -- 172.21.15.143:0/954397011 wait complete. 2023-11-26T07:18:17.276 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:17.269+0000 7f91e91e7640 1 Processor -- start 2023-11-26T07:18:17.276 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:17.269+0000 7f91e91e7640 1 -- start start 2023-11-26T07:18:17.279 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:17.273+0000 7f91e91e7640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f91e40b2160 0x7f91e415c230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:18:17.279 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:17.273+0000 7f91e91e7640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f91e40d19f0 0x7f91e41563c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:18:17.279 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:17.273+0000 7f91e91e7640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f91e415d6e0 0x7f91e4156d20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:18:17.279 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:17.273+0000 7f91e91e7640 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f91e4157770 con 0x7f91e40d19f0 2023-11-26T07:18:17.279 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:17.273+0000 7f91e91e7640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f91e4155600 con 0x7f91e40b2160 2023-11-26T07:18:17.279 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:17.273+0000 7f91e91e7640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f91e41558e0 con 0x7f91e415d6e0 2023-11-26T07:18:17.279 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:17.273+0000 7f91e259c640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f91e40d19f0 0x7f91e41563c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:18:17.279 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:17.273+0000 7f91e259c640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f91e40d19f0 0x7f91e41563c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:43836/0 (socket says 172.21.15.143:43836) 2023-11-26T07:18:17.279 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:17.273+0000 7f91e259c640 1 -- 172.21.15.143:0/3654175565 learned_addr learned my addr 172.21.15.143:0/3654175565 (peer_addr_for_me v2:172.21.15.143:0/0) 2023-11-26T07:18:17.279 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:17.273+0000 7f91e2d9d640 1 --2- 172.21.15.143:0/3654175565 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f91e40b2160 0x7f91e415c230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:18:17.279 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:17.273+0000 7f91e359e640 1 --2- 172.21.15.143:0/3654175565 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f91e415d6e0 0x7f91e4156d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:18:17.280 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:17.273+0000 7f91e259c640 1 -- 172.21.15.143:0/3654175565 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f91e415d6e0 msgr2=0x7f91e4156d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:18:17.280 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:17.273+0000 7f91e259c640 1 --2- 172.21.15.143:0/3654175565 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f91e415d6e0 0x7f91e4156d20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:18:17.280 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:17.273+0000 7f91e259c640 1 -- 172.21.15.143:0/3654175565 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f91e40b2160 msgr2=0x7f91e415c230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:18:17.280 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:17.273+0000 7f91e259c640 1 --2- 172.21.15.143:0/3654175565 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f91e40b2160 0x7f91e415c230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:18:17.280 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:17.273+0000 7f91e259c640 1 -- 172.21.15.143:0/3654175565 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f91d8009420 con 0x7f91e40d19f0 2023-11-26T07:18:17.280 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:17.273+0000 7f91e259c640 1 --2- 172.21.15.143:0/3654175565 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f91e40d19f0 0x7f91e41563c0 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7f91d80ae040 tx=0x7f91d8019220 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:18:17.280 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:17.273+0000 7f91cb7fe640 1 -- 172.21.15.143:0/3654175565 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f91d80b4430 con 0x7f91e40d19f0 2023-11-26T07:18:17.280 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:17.273+0000 7f91cb7fe640 1 -- 172.21.15.143:0/3654175565 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f91d80b4590 con 0x7f91e40d19f0 2023-11-26T07:18:17.280 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:17.273+0000 7f91e91e7640 1 -- 172.21.15.143:0/3654175565 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f91e4155bc0 con 0x7f91e40d19f0 2023-11-26T07:18:17.280 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:17.273+0000 7f91cb7fe640 1 -- 172.21.15.143:0/3654175565 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f91d80b4940 con 0x7f91e40d19f0 2023-11-26T07:18:17.280 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:17.273+0000 7f91e91e7640 1 -- 172.21.15.143:0/3654175565 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f91e415c770 con 0x7f91e40d19f0 2023-11-26T07:18:17.281 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:17.273+0000 7f91e91e7640 1 -- 172.21.15.143:0/3654175565 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f91e4067680 con 0x7f91e40d19f0 2023-11-26T07:18:17.281 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:17.277+0000 7f91cb7fe640 1 -- 172.21.15.143:0/3654175565 <== mon.0 v2:172.21.15.143:3300/0 4 ==== mgrmap(e 5) v1 ==== 85852+0+0 (secure 0 0 0) 0x7f91d80b2040 con 0x7f91e40d19f0 2023-11-26T07:18:17.282 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:17.277+0000 7f91cb7fe640 1 --2- 172.21.15.143:0/3654175565 >> [v2:172.21.15.143:6832/2150557544,v1:172.21.15.143:6833/2150557544] conn(0x7f91d0066470 0x7f91d0068930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:18:17.282 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:17.277+0000 7f91cb7fe640 1 -- 172.21.15.143:0/3654175565 <== mon.0 v2:172.21.15.143:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f91d811dde0 con 0x7f91e40d19f0 2023-11-26T07:18:17.282 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:17.277+0000 7f91e2d9d640 1 --2- 172.21.15.143:0/3654175565 >> [v2:172.21.15.143:6832/2150557544,v1:172.21.15.143:6833/2150557544] conn(0x7f91d0066470 0x7f91d0068930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:18:17.282 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:17.277+0000 7f91e2d9d640 1 --2- 172.21.15.143:0/3654175565 >> [v2:172.21.15.143:6832/2150557544,v1:172.21.15.143:6833/2150557544] conn(0x7f91d0066470 0x7f91d0068930 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7f91e40667a0 tx=0x7f91d400b040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:18:17.285 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:17.281+0000 7f91cb7fe640 1 -- 172.21.15.143:0/3654175565 <== mon.0 v2:172.21.15.143:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172649 (secure 0 0 0) 0x7f91d80eb250 con 0x7f91e40d19f0 2023-11-26T07:18:17.435 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:17.429+0000 7f91e91e7640 1 -- 172.21.15.143:0/3654175565 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f91e4156070 con 0x7f91e40d19f0 2023-11-26T07:18:17.435 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:17.429+0000 7f91cb7fe640 1 -- 172.21.15.143:0/3654175565 <== mon.0 v2:172.21.15.143:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v20) v1 ==== 104+0+11 (secure 0 0 0) 0x7f91d80ee870 con 0x7f91e40d19f0 2023-11-26T07:18:17.436 INFO:teuthology.orchestra.run.smithi143.stdout:pg_num: 32 2023-11-26T07:18:17.437 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:17.433+0000 7f91e91e7640 1 -- 172.21.15.143:0/3654175565 >> [v2:172.21.15.143:6832/2150557544,v1:172.21.15.143:6833/2150557544] conn(0x7f91d0066470 msgr2=0x7f91d0068930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:18:17.438 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:17.433+0000 7f91e91e7640 1 --2- 172.21.15.143:0/3654175565 >> [v2:172.21.15.143:6832/2150557544,v1:172.21.15.143:6833/2150557544] conn(0x7f91d0066470 0x7f91d0068930 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7f91e40667a0 tx=0x7f91d400b040 comp rx=0 tx=0).stop 2023-11-26T07:18:17.438 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:17.433+0000 7f91e91e7640 1 -- 172.21.15.143:0/3654175565 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f91e40d19f0 msgr2=0x7f91e41563c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:18:17.438 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:17.433+0000 7f91e91e7640 1 --2- 172.21.15.143:0/3654175565 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f91e40d19f0 0x7f91e41563c0 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7f91d80ae040 tx=0x7f91d8019220 comp rx=0 tx=0).stop 2023-11-26T07:18:17.438 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:17.433+0000 7f91e91e7640 1 -- 172.21.15.143:0/3654175565 shutdown_connections 2023-11-26T07:18:17.438 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:17.433+0000 7f91e91e7640 1 --2- 172.21.15.143:0/3654175565 >> [v2:172.21.15.143:6832/2150557544,v1:172.21.15.143:6833/2150557544] conn(0x7f91d0066470 0x7f91d0068930 unknown :-1 s=CLOSED pgs=123 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:18:17.438 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:17.433+0000 7f91e91e7640 1 --2- 172.21.15.143:0/3654175565 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f91e415d6e0 0x7f91e4156d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:18:17.438 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:17.433+0000 7f91e91e7640 1 --2- 172.21.15.143:0/3654175565 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f91e40d19f0 0x7f91e41563c0 unknown :-1 s=CLOSED pgs=181 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:18:17.438 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:17.433+0000 7f91e91e7640 1 --2- 172.21.15.143:0/3654175565 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f91e40b2160 0x7f91e415c230 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:18:17.439 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:17.433+0000 7f91e91e7640 1 -- 172.21.15.143:0/3654175565 >> 172.21.15.143:0/3654175565 conn(0x7f91e409c550 msgr2=0x7f91e40cfbe0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:18:17.439 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:17.433+0000 7f91e91e7640 1 -- 172.21.15.143:0/3654175565 shutdown_connections 2023-11-26T07:18:17.439 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:17.433+0000 7f91e91e7640 1 -- 172.21.15.143:0/3654175565 wait complete. 2023-11-26T07:18:17.457 DEBUG:teuthology.orchestra.run.smithi143:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-11-26T07:18:17.599 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:17.593+0000 7f1a29d32640 1 Processor -- start 2023-11-26T07:18:17.599 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:17.593+0000 7f1a29d32640 1 -- start start 2023-11-26T07:18:17.600 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:17.593+0000 7f1a29d32640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f1a2415d680 0x7f1a240d00e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:18:17.600 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:17.593+0000 7f1a29d32640 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7f1a240d1d90 0x7f1a240cec30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:18:17.601 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:17.593+0000 7f1a29d32640 1 -- --> v1:172.21.15.188:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1a240cdc40 con 0x7f1a2415d2a0 2023-11-26T07:18:17.601 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:17.593+0000 7f1a29d32640 1 -- --> v2:172.21.15.188:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1a240ce7a0 con 0x7f1a240d1d90 2023-11-26T07:18:17.601 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:17.593+0000 7f1a29d32640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1a240d1440 con 0x7f1a2415d680 2023-11-26T07:18:17.602 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:17.597+0000 7f1a23fff640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f1a2415d680 0x7f1a240d00e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:18:17.602 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:17.597+0000 7f1a23fff640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f1a2415d680 0x7f1a240d00e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.188:3301/0 says I am v2:172.21.15.143:45410/0 (socket says 172.21.15.143:45410) 2023-11-26T07:18:17.602 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:17.597+0000 7f1a23fff640 1 -- 172.21.15.143:0/2835451136 learned_addr learned my addr 172.21.15.143:0/2835451136 (peer_addr_for_me v2:172.21.15.143:0/0) 2023-11-26T07:18:17.602 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:17.597+0000 7f1a29531640 1 --2- 172.21.15.143:0/2835451136 >> v2:172.21.15.188:3300/0 conn(0x7f1a240d1d90 0x7f1a240cec30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:18:17.602 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:17.597+0000 7f1a23fff640 1 -- 172.21.15.143:0/2835451136 >> v2:172.21.15.188:3300/0 conn(0x7f1a240d1d90 msgr2=0x7f1a240cec30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-26T07:18:17.602 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:17.597+0000 7f1a23fff640 1 --2- 172.21.15.143:0/2835451136 >> v2:172.21.15.188:3300/0 conn(0x7f1a240d1d90 0x7f1a240cec30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:18:17.602 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:17.597+0000 7f1a23fff640 1 -- 172.21.15.143:0/2835451136 >> v1:172.21.15.188:6789/0 conn(0x7f1a2415d2a0 legacy=0x7f1a240d1680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:18:17.602 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:17.597+0000 7f1a23fff640 1 -- 172.21.15.143:0/2835451136 --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1a240cde70 con 0x7f1a2415d680 2023-11-26T07:18:17.604 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:17.597+0000 7f1a23fff640 1 --2- 172.21.15.143:0/2835451136 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f1a2415d680 0x7f1a240d00e0 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7f1a10009870 tx=0x7f1a1001c510 comp rx=0 tx=0).ready entity=mon.2 client_cookie=b687a6f32da7a54f server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:18:17.604 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:17.597+0000 7f1a237fe640 1 -- 172.21.15.143:0/2835451136 <== mon.2 v2:172.21.15.188:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1a100ab620 con 0x7f1a2415d680 2023-11-26T07:18:17.605 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:17.597+0000 7f1a237fe640 1 -- 172.21.15.143:0/2835451136 <== mon.2 v2:172.21.15.188:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1a100ab780 con 0x7f1a2415d680 2023-11-26T07:18:17.605 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:17.597+0000 7f1a237fe640 1 -- 172.21.15.143:0/2835451136 <== mon.2 v2:172.21.15.188:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1a100abb10 con 0x7f1a2415d680 2023-11-26T07:18:17.605 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:17.601+0000 7f1a29d32640 1 -- 172.21.15.143:0/2835451136 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f1a2415d680 msgr2=0x7f1a240d00e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:18:17.605 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:17.601+0000 7f1a29d32640 1 --2- 172.21.15.143:0/2835451136 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f1a2415d680 0x7f1a240d00e0 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7f1a10009870 tx=0x7f1a1001c510 comp rx=0 tx=0).stop 2023-11-26T07:18:17.606 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:17.601+0000 7f1a29d32640 1 -- 172.21.15.143:0/2835451136 shutdown_connections 2023-11-26T07:18:17.606 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:17.601+0000 7f1a29d32640 1 --2- 172.21.15.143:0/2835451136 >> v2:172.21.15.188:3300/0 conn(0x7f1a240d1d90 0x7f1a240cec30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:18:17.606 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:17.601+0000 7f1a29d32640 1 --2- 172.21.15.143:0/2835451136 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f1a2415d680 0x7f1a240d00e0 unknown :-1 s=CLOSED pgs=56 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:18:17.606 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:17.601+0000 7f1a29d32640 1 -- 172.21.15.143:0/2835451136 >> 172.21.15.143:0/2835451136 conn(0x7f1a240d30a0 msgr2=0x7f1a240d3480 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:18:17.606 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:17.601+0000 7f1a29d32640 1 -- 172.21.15.143:0/2835451136 shutdown_connections 2023-11-26T07:18:17.606 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:17.601+0000 7f1a29d32640 1 -- 172.21.15.143:0/2835451136 wait complete. 2023-11-26T07:18:17.607 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:17.601+0000 7f1a29d32640 1 Processor -- start 2023-11-26T07:18:17.607 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:17.601+0000 7f1a29d32640 1 -- start start 2023-11-26T07:18:17.608 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:17.601+0000 7f1a29d32640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f1a240d1d90 0x7f1a2414dab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:18:17.608 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:17.601+0000 7f1a29d32640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f1a2415d2a0 0x7f1a240b17e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:18:17.609 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:17.601+0000 7f1a29d32640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f1a2415d680 0x7f1a24160190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:18:17.609 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:17.601+0000 7f1a29d32640 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1a240b1350 con 0x7f1a240d1d90 2023-11-26T07:18:17.609 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:17.601+0000 7f1a29d32640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1a2414dff0 con 0x7f1a2415d2a0 2023-11-26T07:18:17.609 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:17.601+0000 7f1a29d32640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1a240ccf30 con 0x7f1a2415d680 2023-11-26T07:18:17.609 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:17.601+0000 7f1a28d30640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f1a240d1d90 0x7f1a2414dab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:18:17.609 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:17.601+0000 7f1a29531640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f1a2415d680 0x7f1a24160190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:18:17.609 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:17.601+0000 7f1a23fff640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f1a2415d2a0 0x7f1a240b17e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:18:17.609 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:17.601+0000 7f1a28d30640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f1a240d1d90 0x7f1a2414dab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:43848/0 (socket says 172.21.15.143:43848) 2023-11-26T07:18:17.609 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:17.601+0000 7f1a28d30640 1 -- 172.21.15.143:0/1399989784 learned_addr learned my addr 172.21.15.143:0/1399989784 (peer_addr_for_me v2:172.21.15.143:0/0) 2023-11-26T07:18:17.610 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:17.605+0000 7f1a28d30640 1 -- 172.21.15.143:0/1399989784 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f1a2415d680 msgr2=0x7f1a24160190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:18:17.610 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:17.605+0000 7f1a28d30640 1 --2- 172.21.15.143:0/1399989784 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f1a2415d680 0x7f1a24160190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:18:17.610 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:17.605+0000 7f1a28d30640 1 -- 172.21.15.143:0/1399989784 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f1a2415d2a0 msgr2=0x7f1a240b17e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:18:17.610 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:17.605+0000 7f1a28d30640 1 --2- 172.21.15.143:0/1399989784 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f1a2415d2a0 0x7f1a240b17e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:18:17.611 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:17.605+0000 7f1a28d30640 1 -- 172.21.15.143:0/1399989784 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1a100094d0 con 0x7f1a240d1d90 2023-11-26T07:18:17.611 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:17.605+0000 7f1a28d30640 1 --2- 172.21.15.143:0/1399989784 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f1a240d1d90 0x7f1a2414dab0 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7f1a1800c560 tx=0x7f1a1800ca20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:18:17.611 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:17.605+0000 7f1a217fa640 1 -- 172.21.15.143:0/1399989784 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1a18005220 con 0x7f1a240d1d90 2023-11-26T07:18:17.611 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:17.605+0000 7f1a217fa640 1 -- 172.21.15.143:0/1399989784 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1a18005380 con 0x7f1a240d1d90 2023-11-26T07:18:17.611 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:17.605+0000 7f1a29d32640 1 -- 172.21.15.143:0/1399989784 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1a240b04a0 con 0x7f1a240d1d90 2023-11-26T07:18:17.611 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:17.605+0000 7f1a217fa640 1 -- 172.21.15.143:0/1399989784 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1a18007270 con 0x7f1a240d1d90 2023-11-26T07:18:17.611 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:17.605+0000 7f1a29d32640 1 -- 172.21.15.143:0/1399989784 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1a240b0a20 con 0x7f1a240d1d90 2023-11-26T07:18:17.612 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:17.605+0000 7f1a29d32640 1 -- 172.21.15.143:0/1399989784 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1a24067500 con 0x7f1a240d1d90 2023-11-26T07:18:17.612 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:17.605+0000 7f1a217fa640 1 -- 172.21.15.143:0/1399989784 <== mon.0 v2:172.21.15.143:3300/0 4 ==== mgrmap(e 5) v1 ==== 85852+0+0 (secure 0 0 0) 0x7f1a180073d0 con 0x7f1a240d1d90 2023-11-26T07:18:17.612 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:17.605+0000 7f1a217fa640 1 --2- 172.21.15.143:0/1399989784 >> [v2:172.21.15.143:6832/2150557544,v1:172.21.15.143:6833/2150557544] conn(0x7f1a08066470 0x7f1a08068930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:18:17.612 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:17.605+0000 7f1a217fa640 1 -- 172.21.15.143:0/1399989784 <== mon.0 v2:172.21.15.143:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f1a180835d0 con 0x7f1a240d1d90 2023-11-26T07:18:17.613 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:17.605+0000 7f1a23fff640 1 --2- 172.21.15.143:0/1399989784 >> [v2:172.21.15.143:6832/2150557544,v1:172.21.15.143:6833/2150557544] conn(0x7f1a08066470 0x7f1a08068930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:18:17.613 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:17.609+0000 7f1a23fff640 1 --2- 172.21.15.143:0/1399989784 >> [v2:172.21.15.143:6832/2150557544,v1:172.21.15.143:6833/2150557544] conn(0x7f1a08066470 0x7f1a08068930 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7f1a100af000 tx=0x7f1a100b6040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:18:17.616 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:17.609+0000 7f1a217fa640 1 -- 172.21.15.143:0/1399989784 <== mon.0 v2:172.21.15.143:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172649 (secure 0 0 0) 0x7f1a18050af0 con 0x7f1a240d1d90 2023-11-26T07:18:17.771 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:17.765+0000 7f1a29d32640 1 -- 172.21.15.143:0/1399989784 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f1a2408c7f0 con 0x7f1a240d1d90 2023-11-26T07:18:17.771 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:17.765+0000 7f1a217fa640 1 -- 172.21.15.143:0/1399989784 <== mon.0 v2:172.21.15.143:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v20) v1 ==== 100+0+11 (secure 0 0 0) 0x7f1a18054110 con 0x7f1a240d1d90 2023-11-26T07:18:17.772 INFO:teuthology.orchestra.run.smithi143.stdout:pg_num: 64 2023-11-26T07:18:17.773 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:17.769+0000 7f1a29d32640 1 -- 172.21.15.143:0/1399989784 >> [v2:172.21.15.143:6832/2150557544,v1:172.21.15.143:6833/2150557544] conn(0x7f1a08066470 msgr2=0x7f1a08068930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:18:17.773 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:17.769+0000 7f1a29d32640 1 --2- 172.21.15.143:0/1399989784 >> [v2:172.21.15.143:6832/2150557544,v1:172.21.15.143:6833/2150557544] conn(0x7f1a08066470 0x7f1a08068930 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7f1a100af000 tx=0x7f1a100b6040 comp rx=0 tx=0).stop 2023-11-26T07:18:17.774 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:17.769+0000 7f1a29d32640 1 -- 172.21.15.143:0/1399989784 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f1a240d1d90 msgr2=0x7f1a2414dab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:18:17.774 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:17.769+0000 7f1a29d32640 1 --2- 172.21.15.143:0/1399989784 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f1a240d1d90 0x7f1a2414dab0 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7f1a1800c560 tx=0x7f1a1800ca20 comp rx=0 tx=0).stop 2023-11-26T07:18:17.774 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:17.769+0000 7f1a29d32640 1 -- 172.21.15.143:0/1399989784 shutdown_connections 2023-11-26T07:18:17.774 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:17.769+0000 7f1a29d32640 1 --2- 172.21.15.143:0/1399989784 >> [v2:172.21.15.143:6832/2150557544,v1:172.21.15.143:6833/2150557544] conn(0x7f1a08066470 0x7f1a08068930 unknown :-1 s=CLOSED pgs=124 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:18:17.774 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:17.769+0000 7f1a29d32640 1 --2- 172.21.15.143:0/1399989784 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f1a2415d680 0x7f1a24160190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:18:17.774 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:17.769+0000 7f1a29d32640 1 --2- 172.21.15.143:0/1399989784 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f1a2415d2a0 0x7f1a240b17e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:18:17.774 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:17.769+0000 7f1a29d32640 1 --2- 172.21.15.143:0/1399989784 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f1a240d1d90 0x7f1a2414dab0 unknown :-1 s=CLOSED pgs=182 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:18:17.774 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:17.769+0000 7f1a29d32640 1 -- 172.21.15.143:0/1399989784 >> 172.21.15.143:0/1399989784 conn(0x7f1a240d30a0 msgr2=0x7f1a2415a6f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:18:17.774 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:17.769+0000 7f1a29d32640 1 -- 172.21.15.143:0/1399989784 shutdown_connections 2023-11-26T07:18:17.775 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:17.769+0000 7f1a29d32640 1 -- 172.21.15.143:0/1399989784 wait complete. 2023-11-26T07:18:17.793 DEBUG:teuthology.orchestra.run.smithi143:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-11-26T07:18:17.931 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:17.925+0000 7fdb323ec640 1 Processor -- start 2023-11-26T07:18:17.931 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:17.925+0000 7fdb323ec640 1 -- start start 2023-11-26T07:18:17.933 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:17.925+0000 7fdb323ec640 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7fdb2c145fa0 0x7fdb2c072420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:18:17.933 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:17.925+0000 7fdb323ec640 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7fdb2c06b190 0x7fdb2c146530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:18:17.933 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:17.925+0000 7fdb323ec640 1 -- --> v1:172.21.15.143:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fdb2c146380 con 0x7fdb2c06a490 2023-11-26T07:18:17.933 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:17.925+0000 7fdb323ec640 1 -- --> v2:172.21.15.143:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fdb2c071480 con 0x7fdb2c06b190 2023-11-26T07:18:17.933 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:17.925+0000 7fdb323ec640 1 -- --> v2:172.21.15.188:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fdb2c072000 con 0x7fdb2c145fa0 2023-11-26T07:18:17.933 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:17.925+0000 7fdb2b7fe640 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7fdb2c06b190 0x7fdb2c146530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:18:17.934 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:17.925+0000 7fdb30989640 1 --1- >> v1:172.21.15.143:6789/0 conn(0x7fdb2c06a490 0x7fdb2c06e890 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.143:6789/0 says I am v1:172.21.15.143:43878/0 (socket says 172.21.15.143:43878) 2023-11-26T07:18:17.934 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:17.925+0000 7fdb30989640 1 -- 172.21.15.143:0/3002313616 learned_addr learned my addr 172.21.15.143:0/3002313616 (peer_addr_for_me v1:172.21.15.143:0/0) 2023-11-26T07:18:17.934 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:17.929+0000 7fdb2bfff640 1 --2- 172.21.15.143:0/3002313616 >> v2:172.21.15.188:3300/0 conn(0x7fdb2c145fa0 0x7fdb2c072420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:18:17.934 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:17.929+0000 7fdb2b7fe640 1 -- 172.21.15.143:0/3002313616 >> v2:172.21.15.188:3300/0 conn(0x7fdb2c145fa0 msgr2=0x7fdb2c072420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-26T07:18:17.934 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:17.929+0000 7fdb2b7fe640 1 --2- 172.21.15.143:0/3002313616 >> v2:172.21.15.188:3300/0 conn(0x7fdb2c145fa0 0x7fdb2c072420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:18:17.934 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:17.929+0000 7fdb2b7fe640 1 -- 172.21.15.143:0/3002313616 >> v1:172.21.15.143:6789/0 conn(0x7fdb2c06a490 legacy=0x7fdb2c06e890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:18:17.934 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:17.929+0000 7fdb2affd640 1 -- 172.21.15.143:0/3002313616 <== mon.0 v1:172.21.15.143:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 845409495 0 0) 0x7fdb2c146380 con 0x7fdb2c06a490 2023-11-26T07:18:17.934 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:17.929+0000 7fdb2b7fe640 1 -- 172.21.15.143:0/3002313616 --> v2:172.21.15.143:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdb2c072960 con 0x7fdb2c06b190 2023-11-26T07:18:17.934 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:17.929+0000 7fdb2bfff640 1 --2- 172.21.15.143:0/3002313616 >> v2:172.21.15.188:3300/0 conn(0x7fdb2c145fa0 0x7fdb2c072420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-26T07:18:17.936 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:17.929+0000 7fdb2b7fe640 1 --2- 172.21.15.143:0/3002313616 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fdb2c06b190 0x7fdb2c146530 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7fdb2c072b60 tx=0x7fdb2001c430 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5a33d3429819477 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:18:17.936 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:17.929+0000 7fdb2affd640 1 -- 172.21.15.143:0/3002313616 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdb200a7070 con 0x7fdb2c06b190 2023-11-26T07:18:17.936 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:17.929+0000 7fdb2affd640 1 -- 172.21.15.143:0/3002313616 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdb200ab430 con 0x7fdb2c06b190 2023-11-26T07:18:17.936 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:17.929+0000 7fdb2affd640 1 -- 172.21.15.143:0/3002313616 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdb200ab7c0 con 0x7fdb2c06b190 2023-11-26T07:18:17.937 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:17.933+0000 7fdb323ec640 1 -- 172.21.15.143:0/3002313616 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fdb2c06b190 msgr2=0x7fdb2c146530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:18:17.937 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:17.933+0000 7fdb323ec640 1 --2- 172.21.15.143:0/3002313616 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fdb2c06b190 0x7fdb2c146530 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7fdb2c072b60 tx=0x7fdb2001c430 comp rx=0 tx=0).stop 2023-11-26T07:18:17.937 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:17.933+0000 7fdb323ec640 1 -- 172.21.15.143:0/3002313616 shutdown_connections 2023-11-26T07:18:17.937 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:17.933+0000 7fdb323ec640 1 --2- 172.21.15.143:0/3002313616 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fdb2c06b190 0x7fdb2c146530 unknown :-1 s=CLOSED pgs=184 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:18:17.938 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:17.933+0000 7fdb323ec640 1 --2- 172.21.15.143:0/3002313616 >> v2:172.21.15.188:3300/0 conn(0x7fdb2c145fa0 0x7fdb2c072420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:18:17.938 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:17.933+0000 7fdb323ec640 1 -- 172.21.15.143:0/3002313616 >> 172.21.15.143:0/3002313616 conn(0x7fdb2c0cff00 msgr2=0x7fdb2c0d0310 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:18:17.938 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:17.933+0000 7fdb323ec640 1 -- 172.21.15.143:0/3002313616 shutdown_connections 2023-11-26T07:18:17.938 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:17.933+0000 7fdb323ec640 1 -- 172.21.15.143:0/3002313616 wait complete. 2023-11-26T07:18:17.938 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:17.933+0000 7fdb323ec640 1 Processor -- start 2023-11-26T07:18:17.939 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:17.933+0000 7fdb323ec640 1 -- start start 2023-11-26T07:18:17.941 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:17.933+0000 7fdb323ec640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fdb2c06a490 0x7fdb2c164d30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:18:17.941 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:17.933+0000 7fdb323ec640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fdb2c06b190 0x7fdb2c142c40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:18:17.941 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:17.933+0000 7fdb323ec640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fdb2c145fa0 0x7fdb2c1663c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:18:17.941 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:17.933+0000 7fdb323ec640 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdb2c158720 con 0x7fdb2c06a490 2023-11-26T07:18:17.941 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:17.933+0000 7fdb323ec640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdb2c1583f0 con 0x7fdb2c06b190 2023-11-26T07:18:17.941 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:17.933+0000 7fdb323ec640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdb2c158930 con 0x7fdb2c145fa0 2023-11-26T07:18:17.941 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:17.933+0000 7fdb2bfff640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fdb2c06a490 0x7fdb2c164d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:18:17.942 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:17.933+0000 7fdb2bfff640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fdb2c06a490 0x7fdb2c164d30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:43870/0 (socket says 172.21.15.143:43870) 2023-11-26T07:18:17.942 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:17.933+0000 7fdb2bfff640 1 -- 172.21.15.143:0/3992124881 learned_addr learned my addr 172.21.15.143:0/3992124881 (peer_addr_for_me v2:172.21.15.143:0/0) 2023-11-26T07:18:17.942 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:17.933+0000 7fdb2b7fe640 1 --2- 172.21.15.143:0/3992124881 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fdb2c06b190 0x7fdb2c142c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:18:17.942 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:17.933+0000 7fdb30989640 1 --2- 172.21.15.143:0/3992124881 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fdb2c145fa0 0x7fdb2c1663c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:18:17.942 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:17.933+0000 7fdb2bfff640 1 -- 172.21.15.143:0/3992124881 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fdb2c145fa0 msgr2=0x7fdb2c1663c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:18:17.942 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:17.933+0000 7fdb2bfff640 1 --2- 172.21.15.143:0/3992124881 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fdb2c145fa0 0x7fdb2c1663c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:18:17.942 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:17.933+0000 7fdb2bfff640 1 -- 172.21.15.143:0/3992124881 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fdb2c06b190 msgr2=0x7fdb2c142c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:18:17.942 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:17.933+0000 7fdb2bfff640 1 --2- 172.21.15.143:0/3992124881 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fdb2c06b190 0x7fdb2c142c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:18:17.942 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:17.933+0000 7fdb2bfff640 1 -- 172.21.15.143:0/3992124881 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdb1c009420 con 0x7fdb2c06a490 2023-11-26T07:18:17.942 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:17.937+0000 7fdb2bfff640 1 --2- 172.21.15.143:0/3992124881 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fdb2c06a490 0x7fdb2c164d30 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7fdb1c00e980 tx=0x7fdb1c00ee40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:18:17.942 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:17.937+0000 7fdb28ff9640 1 -- 172.21.15.143:0/3992124881 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdb1c00ca00 con 0x7fdb2c06a490 2023-11-26T07:18:17.943 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:17.937+0000 7fdb28ff9640 1 -- 172.21.15.143:0/3992124881 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdb1c00cb60 con 0x7fdb2c06a490 2023-11-26T07:18:17.943 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:17.937+0000 7fdb28ff9640 1 -- 172.21.15.143:0/3992124881 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdb1c013470 con 0x7fdb2c06a490 2023-11-26T07:18:17.943 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:17.937+0000 7fdb323ec640 1 -- 172.21.15.143:0/3992124881 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdb20009420 con 0x7fdb2c06a490 2023-11-26T07:18:17.943 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:17.937+0000 7fdb323ec640 1 -- 172.21.15.143:0/3992124881 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdb2c06ae30 con 0x7fdb2c06a490 2023-11-26T07:18:17.943 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:17.937+0000 7fdb323ec640 1 -- 172.21.15.143:0/3992124881 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fdb2c067680 con 0x7fdb2c06a490 2023-11-26T07:18:17.944 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:17.937+0000 7fdb28ff9640 1 -- 172.21.15.143:0/3992124881 <== mon.0 v2:172.21.15.143:3300/0 4 ==== mgrmap(e 5) v1 ==== 85852+0+0 (secure 0 0 0) 0x7fdb1c0135d0 con 0x7fdb2c06a490 2023-11-26T07:18:17.944 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:17.937+0000 7fdb28ff9640 1 --2- 172.21.15.143:0/3992124881 >> [v2:172.21.15.143:6832/2150557544,v1:172.21.15.143:6833/2150557544] conn(0x7fdb18066470 0x7fdb18068930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:18:17.944 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:17.937+0000 7fdb28ff9640 1 -- 172.21.15.143:0/3992124881 <== mon.0 v2:172.21.15.143:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fdb1c083ff0 con 0x7fdb2c06a490 2023-11-26T07:18:17.944 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:17.937+0000 7fdb2b7fe640 1 --2- 172.21.15.143:0/3992124881 >> [v2:172.21.15.143:6832/2150557544,v1:172.21.15.143:6833/2150557544] conn(0x7fdb18066470 0x7fdb18068930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:18:17.945 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:17.941+0000 7fdb2b7fe640 1 --2- 172.21.15.143:0/3992124881 >> [v2:172.21.15.143:6832/2150557544,v1:172.21.15.143:6833/2150557544] conn(0x7fdb18066470 0x7fdb18068930 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7fdb200af000 tx=0x7fdb200a7000 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:18:17.948 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:17.941+0000 7fdb28ff9640 1 -- 172.21.15.143:0/3992124881 <== mon.0 v2:172.21.15.143:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172649 (secure 0 0 0) 0x7fdb1c051570 con 0x7fdb2c06a490 2023-11-26T07:18:18.097 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:18.093+0000 7fdb323ec640 1 -- 172.21.15.143:0/3992124881 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fdb2c143180 con 0x7fdb2c06a490 2023-11-26T07:18:18.098 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:18.093+0000 7fdb28ff9640 1 -- 172.21.15.143:0/3992124881 <== mon.0 v2:172.21.15.143:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v20) v1 ==== 74+0+16541 (secure 0 0 0) 0x7fdb1c054b90 con 0x7fdb2c06a490 2023-11-26T07:18:18.098 INFO:teuthology.orchestra.run.smithi143.stdout: 2023-11-26T07:18:18.099 INFO:teuthology.orchestra.run.smithi143.stdout:{"epoch":20,"fsid":"f5033060-2469-4a07-95fe-2b76f5404bc1","created":"2023-11-26T07:17:28.613663+0000","modified":"2023-11-26T07:17:55.505435+0000","last_up_change":"2023-11-26T07:17:41.157813+0000","last_in_change":"2023-11-26T07:17:31.181658+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-11-26T07:17:41.758597+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2023-11-26T07:17:51.878332+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2023-11-26T07:17:52.871396+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"ebbee2c6-023e-44b9-9f1d-6f7bfd309730","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6817","nonce":639807369},{"type":"v1","addr":"172.21.15.143:6819","nonce":639807369}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6821","nonce":639807369},{"type":"v1","addr":"172.21.15.143:6823","nonce":639807369}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6830","nonce":639807369},{"type":"v1","addr":"172.21.15.143:6831","nonce":639807369}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6825","nonce":639807369},{"type":"v1","addr":"172.21.15.143:6828","nonce":639807369}]},"public_addr":"172.21.15.143:6819/639807369","cluster_addr":"172.21.15.143:6823/639807369","heartbeat_back_addr":"172.21.15.143:6831/639807369","heartbeat_front_addr":"172.21.15.143:6828/639807369","state":["exists","up"]},{"osd":1,"uuid":"fec4b8be-7122-49e9-9a19-5850a0700001","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6816","nonce":1144942530},{"type":"v1","addr":"172.21.15.143:6818","nonce":1144942530}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6820","nonce":1144942530},{"type":"v1","addr":"172.21.15.143:6822","nonce":1144942530}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6827","nonce":1144942530},{"type":"v1","addr":"172.21.15.143:6829","nonce":1144942530}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6824","nonce":1144942530},{"type":"v1","addr":"172.21.15.143:6826","nonce":1144942530}]},"public_addr":"172.21.15.143:6818/1144942530","cluster_addr":"172.21.15.143:6822/1144942530","heartbeat_back_addr":"172.21.15.143:6829/1144942530","heartbeat_front_addr":"172.21.15.143:6826/1144942530","state":["exists","up"]},{"osd":2,"uuid":"2888af5b-f933-46ca-8f95-6dd494ff43f6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6808","nonce":3909095186},{"type":"v1","addr":"172.21.15.143:6809","nonce":3909095186}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6810","nonce":3909095186},{"type":"v1","addr":"172.21.15.143:6811","nonce":3909095186}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6814","nonce":3909095186},{"type":"v1","addr":"172.21.15.143:6815","nonce":3909095186}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6812","nonce":3909095186},{"type":"v1","addr":"172.21.15.143:6813","nonce":3909095186}]},"public_addr":"172.21.15.143:6809/3909095186","cluster_addr":"172.21.15.143:6811/3909095186","heartbeat_back_addr":"172.21.15.143:6815/3909095186","heartbeat_front_addr":"172.21.15.143:6813/3909095186","state":["exists","up"]},{"osd":3,"uuid":"d1f52435-7bc1-4081-9550-9cd1ba55f549","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6800","nonce":3854210991},{"type":"v1","addr":"172.21.15.143:6801","nonce":3854210991}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6802","nonce":3854210991},{"type":"v1","addr":"172.21.15.143:6803","nonce":3854210991}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6806","nonce":3854210991},{"type":"v1","addr":"172.21.15.143:6807","nonce":3854210991}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6804","nonce":3854210991},{"type":"v1","addr":"172.21.15.143:6805","nonce":3854210991}]},"public_addr":"172.21.15.143:6801/3854210991","cluster_addr":"172.21.15.143:6803/3854210991","heartbeat_back_addr":"172.21.15.143:6807/3854210991","heartbeat_front_addr":"172.21.15.143:6805/3854210991","state":["exists","up"]},{"osd":4,"uuid":"e2ae8b6e-6e25-49b2-bcbf-aacce68f06a3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6800","nonce":3924606902},{"type":"v1","addr":"172.21.15.188:6801","nonce":3924606902}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6802","nonce":3924606902},{"type":"v1","addr":"172.21.15.188:6803","nonce":3924606902}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6806","nonce":3924606902},{"type":"v1","addr":"172.21.15.188:6807","nonce":3924606902}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6804","nonce":3924606902},{"type":"v1","addr":"172.21.15.188:6805","nonce":3924606902}]},"public_addr":"172.21.15.188:6801/3924606902","cluster_addr":"172.21.15.188:6803/3924606902","heartbeat_back_addr":"172.21.15.188:6807/3924606902","heartbeat_front_addr":"172.21.15.188:6805/3924606902","state":["exists","up"]},{"osd":5,"uuid":"3cec8a32-7698-4c36-8e89-e4ba11934f33","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6813","nonce":186841544},{"type":"v1","addr":"172.21.15.188:6815","nonce":186841544}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6817","nonce":186841544},{"type":"v1","addr":"172.21.15.188:6818","nonce":186841544}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6822","nonce":186841544},{"type":"v1","addr":"172.21.15.188:6823","nonce":186841544}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6820","nonce":186841544},{"type":"v1","addr":"172.21.15.188:6821","nonce":186841544}]},"public_addr":"172.21.15.188:6815/186841544","cluster_addr":"172.21.15.188:6818/186841544","heartbeat_back_addr":"172.21.15.188:6823/186841544","heartbeat_front_addr":"172.21.15.188:6821/186841544","state":["exists","up"]},{"osd":6,"uuid":"58a65d5d-e090-473a-81b4-ad38e85029f9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6808","nonce":2153499405},{"type":"v1","addr":"172.21.15.188:6809","nonce":2153499405}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6810","nonce":2153499405},{"type":"v1","addr":"172.21.15.188:6811","nonce":2153499405}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6816","nonce":2153499405},{"type":"v1","addr":"172.21.15.188:6819","nonce":2153499405}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6812","nonce":2153499405},{"type":"v1","addr":"172.21.15.188:6814","nonce":2153499405}]},"public_addr":"172.21.15.188:6809/2153499405","cluster_addr":"172.21.15.188:6811/2153499405","heartbeat_back_addr":"172.21.15.188:6819/2153499405","heartbeat_front_addr":"172.21.15.188:6814/2153499405","state":["exists","up"]},{"osd":7,"uuid":"ea5b7f70-3554-4cf8-ac30-f7c59b80678d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6824","nonce":2722096729},{"type":"v1","addr":"172.21.15.188:6825","nonce":2722096729}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6826","nonce":2722096729},{"type":"v1","addr":"172.21.15.188:6827","nonce":2722096729}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6830","nonce":2722096729},{"type":"v1","addr":"172.21.15.188:6831","nonce":2722096729}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6828","nonce":2722096729},{"type":"v1","addr":"172.21.15.188:6829","nonce":2722096729}]},"public_addr":"172.21.15.188:6825/2722096729","cluster_addr":"172.21.15.188:6827/2722096729","heartbeat_back_addr":"172.21.15.188:6831/2722096729","heartbeat_front_addr":"172.21.15.188:6829/2722096729","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-26T07:17:36.995032+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-26T07:17:36.932266+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-26T07:17:36.829304+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-26T07:17:36.804297+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-26T07:17:36.667892+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-26T07:17:36.964868+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-26T07:17:36.827332+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-26T07:17:36.893179+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-11-26T07:18:18.100 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:18.093+0000 7fdb323ec640 1 -- 172.21.15.143:0/3992124881 >> [v2:172.21.15.143:6832/2150557544,v1:172.21.15.143:6833/2150557544] conn(0x7fdb18066470 msgr2=0x7fdb18068930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:18:18.100 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:18.093+0000 7fdb323ec640 1 --2- 172.21.15.143:0/3992124881 >> [v2:172.21.15.143:6832/2150557544,v1:172.21.15.143:6833/2150557544] conn(0x7fdb18066470 0x7fdb18068930 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7fdb200af000 tx=0x7fdb200a7000 comp rx=0 tx=0).stop 2023-11-26T07:18:18.101 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:18.093+0000 7fdb323ec640 1 -- 172.21.15.143:0/3992124881 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fdb2c06a490 msgr2=0x7fdb2c164d30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:18:18.101 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:18.093+0000 7fdb323ec640 1 --2- 172.21.15.143:0/3992124881 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fdb2c06a490 0x7fdb2c164d30 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7fdb1c00e980 tx=0x7fdb1c00ee40 comp rx=0 tx=0).stop 2023-11-26T07:18:18.101 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:18.093+0000 7fdb323ec640 1 -- 172.21.15.143:0/3992124881 shutdown_connections 2023-11-26T07:18:18.101 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:18.093+0000 7fdb323ec640 1 --2- 172.21.15.143:0/3992124881 >> [v2:172.21.15.143:6832/2150557544,v1:172.21.15.143:6833/2150557544] conn(0x7fdb18066470 0x7fdb18068930 unknown :-1 s=CLOSED pgs=125 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:18:18.101 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:18.093+0000 7fdb323ec640 1 --2- 172.21.15.143:0/3992124881 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fdb2c145fa0 0x7fdb2c1663c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:18:18.101 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:18.093+0000 7fdb323ec640 1 --2- 172.21.15.143:0/3992124881 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fdb2c06b190 0x7fdb2c142c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:18:18.101 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:18.093+0000 7fdb323ec640 1 --2- 172.21.15.143:0/3992124881 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fdb2c06a490 0x7fdb2c164d30 unknown :-1 s=CLOSED pgs=185 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:18:18.101 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:18.093+0000 7fdb323ec640 1 -- 172.21.15.143:0/3992124881 >> 172.21.15.143:0/3992124881 conn(0x7fdb2c0cff00 msgr2=0x7fdb2c0adb70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:18:18.101 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:18.093+0000 7fdb323ec640 1 -- 172.21.15.143:0/3992124881 shutdown_connections 2023-11-26T07:18:18.101 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:18.093+0000 7fdb323ec640 1 -- 172.21.15.143:0/3992124881 wait complete. 2023-11-26T07:18:18.120 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-11-26T07:17:41.758597+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2023-11-26T07:17:51.878332+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2023-11-26T07:17:52.871396+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '20', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2023-11-26T07:18:18.120 DEBUG:teuthology.orchestra.run.smithi143:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-11-26T07:18:18.261 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:18.257+0000 7f1679032640 1 Processor -- start 2023-11-26T07:18:18.262 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:18.257+0000 7f1679032640 1 -- start start 2023-11-26T07:18:18.262 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:18.257+0000 7f1679032640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f1674070ba0 0x7f167415e6f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:18:18.263 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:18.257+0000 7f1679032640 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7f167406bae0 0x7f1674070f80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:18:18.263 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:18.257+0000 7f1679032640 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7f16740cd610 0x7f16741610f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:18:18.263 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:18.257+0000 7f1679032640 1 -- --> v2:172.21.15.143:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f16740714c0 con 0x7f167406bae0 2023-11-26T07:18:18.263 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:18.257+0000 7f1679032640 1 -- --> v2:172.21.15.188:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f16740cde60 con 0x7f16740cd610 2023-11-26T07:18:18.263 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:18.257+0000 7f1679032640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f167415dc10 con 0x7f1674070ba0 2023-11-26T07:18:18.264 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:18.257+0000 7f167259c640 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7f167406bae0 0x7f1674070f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:18:18.264 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:18.257+0000 7f167259c640 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7f167406bae0 0x7f1674070f80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:43880/0 (socket says 172.21.15.143:43880) 2023-11-26T07:18:18.264 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:18.257+0000 7f167259c640 1 -- 172.21.15.143:0/2635584068 learned_addr learned my addr 172.21.15.143:0/2635584068 (peer_addr_for_me v2:172.21.15.143:0/0) 2023-11-26T07:18:18.264 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:18.257+0000 7f1672d9d640 1 --2- 172.21.15.143:0/2635584068 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f1674070ba0 0x7f167415e6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:18:18.264 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:18.257+0000 7f167359e640 1 --2- 172.21.15.143:0/2635584068 >> v2:172.21.15.188:3300/0 conn(0x7f16740cd610 0x7f16741610f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:18:18.264 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:18.257+0000 7f167259c640 1 -- 172.21.15.143:0/2635584068 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f1674070ba0 msgr2=0x7f167415e6f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-26T07:18:18.264 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:18.257+0000 7f167259c640 1 --2- 172.21.15.143:0/2635584068 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f1674070ba0 0x7f167415e6f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:18:18.264 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:18.257+0000 7f167259c640 1 -- 172.21.15.143:0/2635584068 >> v2:172.21.15.188:3300/0 conn(0x7f16740cd610 msgr2=0x7f16741610f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-26T07:18:18.265 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:18.257+0000 7f167259c640 1 --2- 172.21.15.143:0/2635584068 >> v2:172.21.15.188:3300/0 conn(0x7f16740cd610 0x7f16741610f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:18:18.265 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:18.257+0000 7f167259c640 1 -- 172.21.15.143:0/2635584068 --> v2:172.21.15.143:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f167415def0 con 0x7f167406bae0 2023-11-26T07:18:18.266 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:18.261+0000 7f167259c640 1 --2- 172.21.15.143:0/2635584068 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f167406bae0 0x7f1674070f80 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7f16740cbf90 tx=0x7f166801c440 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d0168d3330c94fad server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:18:18.267 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:18.261+0000 7f1671d9b640 1 -- 172.21.15.143:0/2635584068 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f16680ac600 con 0x7f167406bae0 2023-11-26T07:18:18.267 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:18.261+0000 7f1671d9b640 1 -- 172.21.15.143:0/2635584068 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f16680a8050 con 0x7f167406bae0 2023-11-26T07:18:18.267 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:18.261+0000 7f1671d9b640 1 -- 172.21.15.143:0/2635584068 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f16680ac980 con 0x7f167406bae0 2023-11-26T07:18:18.267 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:18.261+0000 7f1679032640 1 -- 172.21.15.143:0/2635584068 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f167406bae0 msgr2=0x7f1674070f80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:18:18.268 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:18.261+0000 7f1679032640 1 --2- 172.21.15.143:0/2635584068 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f167406bae0 0x7f1674070f80 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7f16740cbf90 tx=0x7f166801c440 comp rx=0 tx=0).stop 2023-11-26T07:18:18.268 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:18.261+0000 7f1679032640 1 -- 172.21.15.143:0/2635584068 shutdown_connections 2023-11-26T07:18:18.268 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:18.261+0000 7f1679032640 1 --2- 172.21.15.143:0/2635584068 >> v2:172.21.15.188:3300/0 conn(0x7f16740cd610 0x7f16741610f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:18:18.268 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:18.261+0000 7f1679032640 1 --2- 172.21.15.143:0/2635584068 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f167406bae0 0x7f1674070f80 unknown :-1 s=CLOSED pgs=186 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:18:18.268 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:18.261+0000 7f1679032640 1 --2- 172.21.15.143:0/2635584068 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f1674070ba0 0x7f167415e6f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:18:18.268 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:18.261+0000 7f1679032640 1 -- 172.21.15.143:0/2635584068 >> 172.21.15.143:0/2635584068 conn(0x7f16740cf710 msgr2=0x7f16740cfb20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:18:18.268 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:18.261+0000 7f1679032640 1 -- 172.21.15.143:0/2635584068 shutdown_connections 2023-11-26T07:18:18.268 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:18.261+0000 7f1679032640 1 -- 172.21.15.143:0/2635584068 wait complete. 2023-11-26T07:18:18.269 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:18.265+0000 7f1679032640 1 Processor -- start 2023-11-26T07:18:18.269 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:18.265+0000 7f1679032640 1 -- start start 2023-11-26T07:18:18.271 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:18.265+0000 7f1679032640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f167406bae0 0x7f16741400c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:18:18.271 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:18.265+0000 7f1679032640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f1674070ba0 0x7f1674154710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:18:18.271 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:18.265+0000 7f1679032640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f16740cd610 0x7f1674154c50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:18:18.271 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:18.265+0000 7f1679032640 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1674156600 con 0x7f167406bae0 2023-11-26T07:18:18.271 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:18.265+0000 7f1679032640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1674156810 con 0x7f1674070ba0 2023-11-26T07:18:18.271 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:18.265+0000 7f1679032640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1674156af0 con 0x7f16740cd610 2023-11-26T07:18:18.271 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:18.265+0000 7f1672d9d640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f167406bae0 0x7f16741400c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:18:18.271 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:18.265+0000 7f1672d9d640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f167406bae0 0x7f16741400c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:43888/0 (socket says 172.21.15.143:43888) 2023-11-26T07:18:18.271 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:18.265+0000 7f1672d9d640 1 -- 172.21.15.143:0/1205979196 learned_addr learned my addr 172.21.15.143:0/1205979196 (peer_addr_for_me v2:172.21.15.143:0/0) 2023-11-26T07:18:18.271 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:18.265+0000 7f167259c640 1 --2- 172.21.15.143:0/1205979196 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f1674070ba0 0x7f1674154710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:18:18.272 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:18.265+0000 7f167359e640 1 --2- 172.21.15.143:0/1205979196 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f16740cd610 0x7f1674154c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:18:18.273 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:18.265+0000 7f1672d9d640 1 -- 172.21.15.143:0/1205979196 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f16740cd610 msgr2=0x7f1674154c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:18:18.273 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:18.265+0000 7f1672d9d640 1 --2- 172.21.15.143:0/1205979196 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f16740cd610 0x7f1674154c50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:18:18.273 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:18.265+0000 7f1672d9d640 1 -- 172.21.15.143:0/1205979196 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f1674070ba0 msgr2=0x7f1674154710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:18:18.273 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:18.265+0000 7f1672d9d640 1 --2- 172.21.15.143:0/1205979196 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f1674070ba0 0x7f1674154710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:18:18.273 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:18.265+0000 7f1672d9d640 1 -- 172.21.15.143:0/1205979196 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f16680094d0 con 0x7f167406bae0 2023-11-26T07:18:18.273 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:18.265+0000 7f167259c640 1 --2- 172.21.15.143:0/1205979196 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f1674070ba0 0x7f1674154710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-26T07:18:18.273 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:18.265+0000 7f1672d9d640 1 --2- 172.21.15.143:0/1205979196 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f167406bae0 0x7f16741400c0 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7f166400d800 tx=0x7f166400dcc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:18:18.273 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:18.265+0000 7f165b7fe640 1 -- 172.21.15.143:0/1205979196 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f16640096a0 con 0x7f167406bae0 2023-11-26T07:18:18.273 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:18.265+0000 7f165b7fe640 1 -- 172.21.15.143:0/1205979196 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1664009800 con 0x7f167406bae0 2023-11-26T07:18:18.274 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:18.265+0000 7f1679032640 1 -- 172.21.15.143:0/1205979196 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f16740cdc30 con 0x7f167406bae0 2023-11-26T07:18:18.274 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:18.265+0000 7f165b7fe640 1 -- 172.21.15.143:0/1205979196 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1664009b70 con 0x7f167406bae0 2023-11-26T07:18:18.274 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:18.265+0000 7f1679032640 1 -- 172.21.15.143:0/1205979196 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f16741570b0 con 0x7f167406bae0 2023-11-26T07:18:18.274 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:18.269+0000 7f1679032640 1 -- 172.21.15.143:0/1205979196 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1674067680 con 0x7f167406bae0 2023-11-26T07:18:18.274 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:18.269+0000 7f165b7fe640 1 -- 172.21.15.143:0/1205979196 <== mon.0 v2:172.21.15.143:3300/0 4 ==== mgrmap(e 5) v1 ==== 85852+0+0 (secure 0 0 0) 0x7f1664009cd0 con 0x7f167406bae0 2023-11-26T07:18:18.275 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:18.269+0000 7f165b7fe640 1 --2- 172.21.15.143:0/1205979196 >> [v2:172.21.15.143:6832/2150557544,v1:172.21.15.143:6833/2150557544] conn(0x7f1660066470 0x7f1660068930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:18:18.275 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:18.269+0000 7f165b7fe640 1 -- 172.21.15.143:0/1205979196 <== mon.0 v2:172.21.15.143:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f16640845b0 con 0x7f167406bae0 2023-11-26T07:18:18.275 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:18.269+0000 7f167259c640 1 --2- 172.21.15.143:0/1205979196 >> [v2:172.21.15.143:6832/2150557544,v1:172.21.15.143:6833/2150557544] conn(0x7f1660066470 0x7f1660068930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:18:18.275 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:18.269+0000 7f167259c640 1 --2- 172.21.15.143:0/1205979196 >> [v2:172.21.15.143:6832/2150557544,v1:172.21.15.143:6833/2150557544] conn(0x7f1660066470 0x7f1660068930 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7f16680ae000 tx=0x7f16680a7000 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:18:18.278 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:18.273+0000 7f165b7fe640 1 -- 172.21.15.143:0/1205979196 <== mon.0 v2:172.21.15.143:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172649 (secure 0 0 0) 0x7f16640519b0 con 0x7f167406bae0 2023-11-26T07:18:18.430 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:18.425+0000 7f1679032640 1 -- 172.21.15.143:0/1205979196 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f1674071c40 con 0x7f167406bae0 2023-11-26T07:18:18.431 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:18.425+0000 7f165b7fe640 1 -- 172.21.15.143:0/1205979196 <== mon.0 v2:172.21.15.143:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v20) v1 ==== 93+0+10 (secure 0 0 0) 0x7f1664082e60 con 0x7f167406bae0 2023-11-26T07:18:18.431 INFO:teuthology.orchestra.run.smithi143.stdout:pg_num: 1 2023-11-26T07:18:18.433 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:18.425+0000 7f1679032640 1 -- 172.21.15.143:0/1205979196 >> [v2:172.21.15.143:6832/2150557544,v1:172.21.15.143:6833/2150557544] conn(0x7f1660066470 msgr2=0x7f1660068930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:18:18.433 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:18.425+0000 7f1679032640 1 --2- 172.21.15.143:0/1205979196 >> [v2:172.21.15.143:6832/2150557544,v1:172.21.15.143:6833/2150557544] conn(0x7f1660066470 0x7f1660068930 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7f16680ae000 tx=0x7f16680a7000 comp rx=0 tx=0).stop 2023-11-26T07:18:18.433 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:18.425+0000 7f1679032640 1 -- 172.21.15.143:0/1205979196 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f167406bae0 msgr2=0x7f16741400c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:18:18.433 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:18.425+0000 7f1679032640 1 --2- 172.21.15.143:0/1205979196 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f167406bae0 0x7f16741400c0 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7f166400d800 tx=0x7f166400dcc0 comp rx=0 tx=0).stop 2023-11-26T07:18:18.433 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:18.429+0000 7f1679032640 1 -- 172.21.15.143:0/1205979196 shutdown_connections 2023-11-26T07:18:18.433 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:18.429+0000 7f1679032640 1 --2- 172.21.15.143:0/1205979196 >> [v2:172.21.15.143:6832/2150557544,v1:172.21.15.143:6833/2150557544] conn(0x7f1660066470 0x7f1660068930 unknown :-1 s=CLOSED pgs=126 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:18:18.434 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:18.429+0000 7f1679032640 1 --2- 172.21.15.143:0/1205979196 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f16740cd610 0x7f1674154c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:18:18.434 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:18.429+0000 7f1679032640 1 --2- 172.21.15.143:0/1205979196 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f1674070ba0 0x7f1674154710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:18:18.434 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:18.429+0000 7f1679032640 1 --2- 172.21.15.143:0/1205979196 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f167406bae0 0x7f16741400c0 unknown :-1 s=CLOSED pgs=187 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:18:18.434 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:18.429+0000 7f1679032640 1 -- 172.21.15.143:0/1205979196 >> 172.21.15.143:0/1205979196 conn(0x7f16740cf710 msgr2=0x7f167406a880 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:18:18.434 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:18.429+0000 7f1679032640 1 -- 172.21.15.143:0/1205979196 shutdown_connections 2023-11-26T07:18:18.434 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:18.429+0000 7f1679032640 1 -- 172.21.15.143:0/1205979196 wait complete. 2023-11-26T07:18:18.452 DEBUG:teuthology.orchestra.run.smithi143:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-11-26T07:18:18.596 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:18.589+0000 7fc48989a640 1 Processor -- start 2023-11-26T07:18:18.596 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:18.589+0000 7fc48989a640 1 -- start start 2023-11-26T07:18:18.597 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:18.593+0000 7fc48989a640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fc484143780 0x7fc4840cdfe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:18:18.598 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:18.593+0000 7fc48989a640 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7fc4840722c0 0x7fc48415c5a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:18:18.598 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:18.593+0000 7fc48989a640 1 -- --> v1:172.21.15.143:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc4840ceb50 con 0x7fc4840ce520 2023-11-26T07:18:18.598 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:18.593+0000 7fc48989a640 1 -- --> v2:172.21.15.188:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc48415c110 con 0x7fc4840722c0 2023-11-26T07:18:18.598 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:18.593+0000 7fc48989a640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc48415b7e0 con 0x7fc484143780 2023-11-26T07:18:18.598 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:18.593+0000 7fc47a7fc640 1 --1- >> v1:172.21.15.143:6789/0 conn(0x7fc4840ce520 0x7fc484071bb0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.143:6789/0 says I am v1:172.21.15.143:43894/0 (socket says 172.21.15.143:43894) 2023-11-26T07:18:18.598 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:18.593+0000 7fc47a7fc640 1 -- 172.21.15.143:0/4134122350 learned_addr learned my addr 172.21.15.143:0/4134122350 (peer_addr_for_me v1:172.21.15.143:0/0) 2023-11-26T07:18:18.600 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:18.593+0000 7fc4837fe640 1 --2- 172.21.15.143:0/4134122350 >> v2:172.21.15.188:3300/0 conn(0x7fc4840722c0 0x7fc48415c5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:18:18.600 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:18.593+0000 7fc482ffd640 1 --2- 172.21.15.143:0/4134122350 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fc484143780 0x7fc4840cdfe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:18:18.600 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:18.593+0000 7fc4827fc640 1 -- 172.21.15.143:0/4134122350 <== mon.0 v1:172.21.15.143:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3028057539 0 0) 0x7fc474001120 con 0x7fc4840ce520 2023-11-26T07:18:18.600 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:18.593+0000 7fc4827fc640 1 -- 172.21.15.143:0/4134122350 --> v1:172.21.15.143:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fc470003530 con 0x7fc4840ce520 2023-11-26T07:18:18.600 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:18.593+0000 7fc4827fc640 1 -- 172.21.15.143:0/4134122350 <== mon.0 v1:172.21.15.143:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3862773926 0 0) 0x7fc470003530 con 0x7fc4840ce520 2023-11-26T07:18:18.600 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:18.593+0000 7fc4827fc640 1 -- 172.21.15.143:0/4134122350 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fc484143780 msgr2=0x7fc4840cdfe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-26T07:18:18.600 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:18.593+0000 7fc4827fc640 1 --2- 172.21.15.143:0/4134122350 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fc484143780 0x7fc4840cdfe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:18:18.601 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:18.593+0000 7fc4827fc640 1 -- 172.21.15.143:0/4134122350 >> v2:172.21.15.188:3300/0 conn(0x7fc4840722c0 msgr2=0x7fc48415c5a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-26T07:18:18.601 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:18.593+0000 7fc4827fc640 1 --2- 172.21.15.143:0/4134122350 >> v2:172.21.15.188:3300/0 conn(0x7fc4840722c0 0x7fc48415c5a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:18:18.601 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:18.593+0000 7fc4827fc640 1 -- 172.21.15.143:0/4134122350 --> v1:172.21.15.143:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc48415bac0 con 0x7fc4840ce520 2023-11-26T07:18:18.601 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:18.593+0000 7fc4827fc640 1 -- 172.21.15.143:0/4134122350 <== mon.0 v1:172.21.15.143:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2137882523 0 0) 0x7fc474001810 con 0x7fc4840ce520 2023-11-26T07:18:18.601 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:18.593+0000 7fc4827fc640 1 -- 172.21.15.143:0/4134122350 >> v1:172.21.15.143:6789/0 conn(0x7fc4840ce520 legacy=0x7fc484071bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:18:18.601 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:18.593+0000 7fc4827fc640 1 --2- 172.21.15.143:0/4134122350 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fc470003980 0x7fc470003da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:18:18.601 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:18.593+0000 7fc4827fc640 1 --2- 172.21.15.143:0/4134122350 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fc484143780 0x7fc4700047d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:18:18.601 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:18.593+0000 7fc4827fc640 1 --2- 172.21.15.143:0/4134122350 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fc470004d10 0x7fc470005190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:18:18.601 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:18.593+0000 7fc4827fc640 1 -- 172.21.15.143:0/4134122350 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc48415c110 con 0x7fc470003980 2023-11-26T07:18:18.601 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:18.593+0000 7fc4827fc640 1 -- 172.21.15.143:0/4134122350 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc48415b7e0 con 0x7fc470004d10 2023-11-26T07:18:18.601 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:18.593+0000 7fc4827fc640 1 -- 172.21.15.143:0/4134122350 --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc47000b990 con 0x7fc484143780 2023-11-26T07:18:18.601 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:18.593+0000 7fc482ffd640 1 --2- 172.21.15.143:0/4134122350 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fc470003980 0x7fc470003da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:18:18.601 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:18.593+0000 7fc47a7fc640 1 --2- 172.21.15.143:0/4134122350 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fc484143780 0x7fc4700047d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:18:18.602 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:18.593+0000 7fc4837fe640 1 --2- 172.21.15.143:0/4134122350 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fc470004d10 0x7fc470005190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:18:18.602 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:18.593+0000 7fc482ffd640 1 -- 172.21.15.143:0/4134122350 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fc484143780 msgr2=0x7fc4700047d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-26T07:18:18.602 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:18.593+0000 7fc482ffd640 1 --2- 172.21.15.143:0/4134122350 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fc484143780 0x7fc4700047d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:18:18.602 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:18.593+0000 7fc482ffd640 1 -- 172.21.15.143:0/4134122350 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fc470004d10 msgr2=0x7fc470005190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-26T07:18:18.602 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:18.593+0000 7fc482ffd640 1 --2- 172.21.15.143:0/4134122350 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fc470004d10 0x7fc470005190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:18:18.602 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:18.593+0000 7fc482ffd640 1 -- 172.21.15.143:0/4134122350 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc470003530 con 0x7fc470003980 2023-11-26T07:18:18.602 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:18.593+0000 7fc4837fe640 1 --2- 172.21.15.143:0/4134122350 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fc470004d10 0x7fc470005190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-26T07:18:18.603 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:18.597+0000 7fc482ffd640 1 --2- 172.21.15.143:0/4134122350 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fc470003980 0x7fc470003da0 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7fc46c009ae0 tx=0x7fc46c01ca50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=188d9baa0f9757fa server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:18:18.603 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:18.597+0000 7fc4827fc640 1 -- 172.21.15.143:0/4134122350 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc46c0a5530 con 0x7fc470003980 2023-11-26T07:18:18.603 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:18.597+0000 7fc4827fc640 1 -- 172.21.15.143:0/4134122350 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc46c0a5690 con 0x7fc470003980 2023-11-26T07:18:18.603 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:18.597+0000 7fc4827fc640 1 -- 172.21.15.143:0/4134122350 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc46c0a5a40 con 0x7fc470003980 2023-11-26T07:18:18.604 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:18.597+0000 7fc48989a640 1 -- 172.21.15.143:0/4134122350 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fc470003980 msgr2=0x7fc470003da0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:18:18.604 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:18.597+0000 7fc48989a640 1 --2- 172.21.15.143:0/4134122350 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fc470003980 0x7fc470003da0 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7fc46c009ae0 tx=0x7fc46c01ca50 comp rx=0 tx=0).stop 2023-11-26T07:18:18.604 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:18.597+0000 7fc4837fe640 1 -- 172.21.15.143:0/4134122350 reap_dead start 2023-11-26T07:18:18.604 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:18.597+0000 7fc48989a640 1 -- 172.21.15.143:0/4134122350 shutdown_connections 2023-11-26T07:18:18.604 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:18.597+0000 7fc48989a640 1 -- 172.21.15.143:0/4134122350 >> 172.21.15.143:0/4134122350 conn(0x7fc48406c3f0 msgr2=0x7fc48406c800 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:18:18.604 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:18.597+0000 7fc48989a640 1 -- 172.21.15.143:0/4134122350 shutdown_connections 2023-11-26T07:18:18.604 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:18.597+0000 7fc48989a640 1 -- 172.21.15.143:0/4134122350 wait complete. 2023-11-26T07:18:18.604 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:18.597+0000 7fc48989a640 1 Processor -- start 2023-11-26T07:18:18.605 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:18.601+0000 7fc48989a640 1 -- start start 2023-11-26T07:18:18.607 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:18.601+0000 7fc48989a640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fc470003980 0x7fc48406c800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:18:18.607 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:18.601+0000 7fc48989a640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fc484165d00 0x7fc48413aed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:18:18.607 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:18.601+0000 7fc48989a640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fc48406cd40 0x7fc48413b410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:18:18.607 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:18.601+0000 7fc48989a640 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc48406d2a0 con 0x7fc484165d00 2023-11-26T07:18:18.607 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:18.601+0000 7fc48989a640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc484164f40 con 0x7fc470003980 2023-11-26T07:18:18.607 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:18.601+0000 7fc48989a640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc484165220 con 0x7fc48406cd40 2023-11-26T07:18:18.607 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:18.601+0000 7fc47a7fc640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fc484165d00 0x7fc48413aed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:18:18.607 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:18.601+0000 7fc47a7fc640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fc484165d00 0x7fc48413aed0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:43914/0 (socket says 172.21.15.143:43914) 2023-11-26T07:18:18.607 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:18.601+0000 7fc47a7fc640 1 -- 172.21.15.143:0/2669357899 learned_addr learned my addr 172.21.15.143:0/2669357899 (peer_addr_for_me v2:172.21.15.143:0/0) 2023-11-26T07:18:18.608 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:18.601+0000 7fc482ffd640 1 --2- 172.21.15.143:0/2669357899 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fc470003980 0x7fc48406c800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:18:18.608 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:18.601+0000 7fc4837fe640 1 --2- 172.21.15.143:0/2669357899 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fc48406cd40 0x7fc48413b410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:18:18.608 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:18.601+0000 7fc47a7fc640 1 -- 172.21.15.143:0/2669357899 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fc48406cd40 msgr2=0x7fc48413b410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:18:18.608 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:18.601+0000 7fc47a7fc640 1 --2- 172.21.15.143:0/2669357899 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fc48406cd40 0x7fc48413b410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:18:18.608 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:18.601+0000 7fc47a7fc640 1 -- 172.21.15.143:0/2669357899 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fc470003980 msgr2=0x7fc48406c800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:18:18.608 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:18.601+0000 7fc47a7fc640 1 --2- 172.21.15.143:0/2669357899 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fc470003980 0x7fc48406c800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:18:18.608 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:18.601+0000 7fc47a7fc640 1 -- 172.21.15.143:0/2669357899 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc484165500 con 0x7fc484165d00 2023-11-26T07:18:18.608 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:18.601+0000 7fc47a7fc640 1 --2- 172.21.15.143:0/2669357899 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fc484165d00 0x7fc48413aed0 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7fc474007310 tx=0x7fc4740077d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:18:18.608 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:18.601+0000 7fc480ff9640 1 -- 172.21.15.143:0/2669357899 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc474007e80 con 0x7fc484165d00 2023-11-26T07:18:18.608 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:18.601+0000 7fc480ff9640 1 -- 172.21.15.143:0/2669357899 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc47400f9c0 con 0x7fc484165d00 2023-11-26T07:18:18.608 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:18.601+0000 7fc480ff9640 1 -- 172.21.15.143:0/2669357899 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc47400fdb0 con 0x7fc484165d00 2023-11-26T07:18:18.608 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:18.601+0000 7fc48989a640 1 -- 172.21.15.143:0/2669357899 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc4841657e0 con 0x7fc484165d00 2023-11-26T07:18:18.609 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:18.601+0000 7fc48989a640 1 -- 172.21.15.143:0/2669357899 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc484139900 con 0x7fc484165d00 2023-11-26T07:18:18.609 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:18.601+0000 7fc48989a640 1 -- 172.21.15.143:0/2669357899 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc484067500 con 0x7fc484165d00 2023-11-26T07:18:18.609 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:18.605+0000 7fc480ff9640 1 -- 172.21.15.143:0/2669357899 <== mon.0 v2:172.21.15.143:3300/0 4 ==== mgrmap(e 5) v1 ==== 85852+0+0 (secure 0 0 0) 0x7fc47401d460 con 0x7fc484165d00 2023-11-26T07:18:18.610 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:18.605+0000 7fc480ff9640 1 --2- 172.21.15.143:0/2669357899 >> [v2:172.21.15.143:6832/2150557544,v1:172.21.15.143:6833/2150557544] conn(0x7fc4640664a0 0x7fc464068960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:18:18.610 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:18.605+0000 7fc480ff9640 1 -- 172.21.15.143:0/2669357899 <== mon.0 v2:172.21.15.143:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fc474083f50 con 0x7fc484165d00 2023-11-26T07:18:18.610 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:18.605+0000 7fc482ffd640 1 --2- 172.21.15.143:0/2669357899 >> [v2:172.21.15.143:6832/2150557544,v1:172.21.15.143:6833/2150557544] conn(0x7fc4640664a0 0x7fc464068960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:18:18.611 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:18.605+0000 7fc482ffd640 1 --2- 172.21.15.143:0/2669357899 >> [v2:172.21.15.143:6832/2150557544,v1:172.21.15.143:6833/2150557544] conn(0x7fc4640664a0 0x7fc464068960 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7fc46c0af000 tx=0x7fc46c0046d0 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:18:18.614 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:18.609+0000 7fc480ff9640 1 -- 172.21.15.143:0/2669357899 <== mon.0 v2:172.21.15.143:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172649 (secure 0 0 0) 0x7fc4740512a0 con 0x7fc484165d00 2023-11-26T07:18:18.766 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:18.761+0000 7fc48989a640 1 -- 172.21.15.143:0/2669357899 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7fc4840ceb50 con 0x7fc484165d00 2023-11-26T07:18:18.766 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:18.761+0000 7fc480ff9640 1 -- 172.21.15.143:0/2669357899 <== mon.0 v2:172.21.15.143:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v20) v1 ==== 104+0+11 (secure 0 0 0) 0x7fc4840ceb50 con 0x7fc484165d00 2023-11-26T07:18:18.767 INFO:teuthology.orchestra.run.smithi143.stdout:pg_num: 32 2023-11-26T07:18:18.769 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:18.761+0000 7fc48989a640 1 -- 172.21.15.143:0/2669357899 >> [v2:172.21.15.143:6832/2150557544,v1:172.21.15.143:6833/2150557544] conn(0x7fc4640664a0 msgr2=0x7fc464068960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:18:18.769 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:18.761+0000 7fc48989a640 1 --2- 172.21.15.143:0/2669357899 >> [v2:172.21.15.143:6832/2150557544,v1:172.21.15.143:6833/2150557544] conn(0x7fc4640664a0 0x7fc464068960 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7fc46c0af000 tx=0x7fc46c0046d0 comp rx=0 tx=0).stop 2023-11-26T07:18:18.769 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:18.761+0000 7fc48989a640 1 -- 172.21.15.143:0/2669357899 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fc484165d00 msgr2=0x7fc48413aed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:18:18.769 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:18.761+0000 7fc48989a640 1 --2- 172.21.15.143:0/2669357899 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fc484165d00 0x7fc48413aed0 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7fc474007310 tx=0x7fc4740077d0 comp rx=0 tx=0).stop 2023-11-26T07:18:18.769 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:18.761+0000 7fc48989a640 1 -- 172.21.15.143:0/2669357899 shutdown_connections 2023-11-26T07:18:18.769 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:18.761+0000 7fc48989a640 1 --2- 172.21.15.143:0/2669357899 >> [v2:172.21.15.143:6832/2150557544,v1:172.21.15.143:6833/2150557544] conn(0x7fc4640664a0 0x7fc464068960 unknown :-1 s=CLOSED pgs=127 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:18:18.769 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:18.761+0000 7fc48989a640 1 --2- 172.21.15.143:0/2669357899 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fc48406cd40 0x7fc48413b410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:18:18.769 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:18.761+0000 7fc48989a640 1 --2- 172.21.15.143:0/2669357899 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fc484165d00 0x7fc48413aed0 unknown :-1 s=CLOSED pgs=190 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:18:18.770 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:18.761+0000 7fc48989a640 1 --2- 172.21.15.143:0/2669357899 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fc470003980 0x7fc48406c800 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:18:18.770 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:18.761+0000 7fc48989a640 1 -- 172.21.15.143:0/2669357899 >> 172.21.15.143:0/2669357899 conn(0x7fc48406c3f0 msgr2=0x7fc48413c2a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:18:18.770 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:18.761+0000 7fc48989a640 1 -- 172.21.15.143:0/2669357899 shutdown_connections 2023-11-26T07:18:18.770 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:18.765+0000 7fc48989a640 1 -- 172.21.15.143:0/2669357899 wait complete. 2023-11-26T07:18:18.787 DEBUG:teuthology.orchestra.run.smithi143:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-11-26T07:18:18.927 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:18.921+0000 7fdd702e4640 1 Processor -- start 2023-11-26T07:18:18.927 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:18.921+0000 7fdd702e4640 1 -- start start 2023-11-26T07:18:18.928 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:18.921+0000 7fdd702e4640 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7fdd68143ac0 0x7fdd68070370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:18:18.929 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:18.921+0000 7fdd702e4640 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7fdd680708b0 0x7fdd680cf280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:18:18.929 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:18.921+0000 7fdd702e4640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fdd680cf7c0 0x7fdd680b1ef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:18:18.929 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:18.921+0000 7fdd702e4640 1 -- --> v2:172.21.15.143:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fdd68143ea0 con 0x7fdd680708b0 2023-11-26T07:18:18.929 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:18.921+0000 7fdd702e4640 1 -- --> v2:172.21.15.188:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fdd680cc730 con 0x7fdd68143ac0 2023-11-26T07:18:18.929 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:18.921+0000 7fdd702e4640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdd680cfbd0 con 0x7fdd680cf7c0 2023-11-26T07:18:18.929 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:18.921+0000 7fdd6d87f640 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7fdd680708b0 0x7fdd680cf280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:18:18.929 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:18.921+0000 7fdd6d87f640 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7fdd680708b0 0x7fdd680cf280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:43922/0 (socket says 172.21.15.143:43922) 2023-11-26T07:18:18.929 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:18.921+0000 7fdd6d87f640 1 -- 172.21.15.143:0/3585364821 learned_addr learned my addr 172.21.15.143:0/3585364821 (peer_addr_for_me v2:172.21.15.143:0/0) 2023-11-26T07:18:18.930 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:18.925+0000 7fdd6e881640 1 --2- 172.21.15.143:0/3585364821 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fdd680cf7c0 0x7fdd680b1ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:18:18.930 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:18.925+0000 7fdd6e080640 1 --2- 172.21.15.143:0/3585364821 >> v2:172.21.15.188:3300/0 conn(0x7fdd68143ac0 0x7fdd68070370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:18:18.930 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:18.925+0000 7fdd6d87f640 1 -- 172.21.15.143:0/3585364821 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fdd680cf7c0 msgr2=0x7fdd680b1ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-26T07:18:18.930 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:18.925+0000 7fdd6d87f640 1 --2- 172.21.15.143:0/3585364821 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fdd680cf7c0 0x7fdd680b1ef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:18:18.930 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:18.925+0000 7fdd6d87f640 1 -- 172.21.15.143:0/3585364821 >> v2:172.21.15.188:3300/0 conn(0x7fdd68143ac0 msgr2=0x7fdd68070370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-26T07:18:18.930 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:18.925+0000 7fdd6d87f640 1 --2- 172.21.15.143:0/3585364821 >> v2:172.21.15.188:3300/0 conn(0x7fdd68143ac0 0x7fdd68070370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:18:18.930 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:18.925+0000 7fdd6d87f640 1 -- 172.21.15.143:0/3585364821 --> v2:172.21.15.143:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdd68070ee0 con 0x7fdd680708b0 2023-11-26T07:18:18.932 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:18.925+0000 7fdd6d87f640 1 --2- 172.21.15.143:0/3585364821 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fdd680708b0 0x7fdd680cf280 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7fdd6806dca0 tx=0x7fdd5801c240 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4380280117855f20 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:18:18.932 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:18.925+0000 7fdd6d07e640 1 -- 172.21.15.143:0/3585364821 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdd580ab600 con 0x7fdd680708b0 2023-11-26T07:18:18.932 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:18.925+0000 7fdd6d07e640 1 -- 172.21.15.143:0/3585364821 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdd580a7050 con 0x7fdd680708b0 2023-11-26T07:18:18.932 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:18.925+0000 7fdd6d07e640 1 -- 172.21.15.143:0/3585364821 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdd580ab980 con 0x7fdd680708b0 2023-11-26T07:18:18.933 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:18.929+0000 7fdd702e4640 1 -- 172.21.15.143:0/3585364821 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fdd680708b0 msgr2=0x7fdd680cf280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:18:18.933 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:18.929+0000 7fdd702e4640 1 --2- 172.21.15.143:0/3585364821 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fdd680708b0 0x7fdd680cf280 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7fdd6806dca0 tx=0x7fdd5801c240 comp rx=0 tx=0).stop 2023-11-26T07:18:18.933 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:18.929+0000 7fdd702e4640 1 -- 172.21.15.143:0/3585364821 shutdown_connections 2023-11-26T07:18:18.933 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:18.929+0000 7fdd702e4640 1 --2- 172.21.15.143:0/3585364821 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fdd680cf7c0 0x7fdd680b1ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:18:18.933 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:18.929+0000 7fdd702e4640 1 --2- 172.21.15.143:0/3585364821 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fdd680708b0 0x7fdd680cf280 unknown :-1 s=CLOSED pgs=191 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:18:18.933 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:18.929+0000 7fdd702e4640 1 --2- 172.21.15.143:0/3585364821 >> v2:172.21.15.188:3300/0 conn(0x7fdd68143ac0 0x7fdd68070370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:18:18.934 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:18.929+0000 7fdd702e4640 1 -- 172.21.15.143:0/3585364821 >> 172.21.15.143:0/3585364821 conn(0x7fdd6806a890 msgr2=0x7fdd6806aca0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:18:18.934 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:18.929+0000 7fdd702e4640 1 -- 172.21.15.143:0/3585364821 shutdown_connections 2023-11-26T07:18:18.934 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:18.929+0000 7fdd702e4640 1 -- 172.21.15.143:0/3585364821 wait complete. 2023-11-26T07:18:18.935 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:18.929+0000 7fdd702e4640 1 Processor -- start 2023-11-26T07:18:18.935 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:18.929+0000 7fdd702e4640 1 -- start start 2023-11-26T07:18:18.936 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:18.929+0000 7fdd702e4640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fdd680708b0 0x7fdd6815a7c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:18:18.937 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:18.929+0000 7fdd702e4640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fdd680cf7c0 0x7fdd680ad930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:18:18.937 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:18.929+0000 7fdd702e4640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fdd68143ac0 0x7fdd680ade70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:18:18.937 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:18.929+0000 7fdd702e4640 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdd680af400 con 0x7fdd680708b0 2023-11-26T07:18:18.937 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:18.929+0000 7fdd702e4640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdd6815d1a0 con 0x7fdd68143ac0 2023-11-26T07:18:18.937 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:18.929+0000 7fdd702e4640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdd680af650 con 0x7fdd680cf7c0 2023-11-26T07:18:18.937 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:18.929+0000 7fdd6e080640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fdd680708b0 0x7fdd6815a7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:18:18.937 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:18.929+0000 7fdd6e080640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fdd680708b0 0x7fdd6815a7c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:43930/0 (socket says 172.21.15.143:43930) 2023-11-26T07:18:18.937 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:18.929+0000 7fdd6e080640 1 -- 172.21.15.143:0/3679928109 learned_addr learned my addr 172.21.15.143:0/3679928109 (peer_addr_for_me v2:172.21.15.143:0/0) 2023-11-26T07:18:18.937 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:18.929+0000 7fdd6d87f640 1 --2- 172.21.15.143:0/3679928109 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fdd680cf7c0 0x7fdd680ad930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:18:18.937 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:18.929+0000 7fdd6e881640 1 --2- 172.21.15.143:0/3679928109 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fdd68143ac0 0x7fdd680ade70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:18:18.937 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:18.929+0000 7fdd6e080640 1 -- 172.21.15.143:0/3679928109 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fdd680cf7c0 msgr2=0x7fdd680ad930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:18:18.938 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:18.929+0000 7fdd6e080640 1 --2- 172.21.15.143:0/3679928109 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fdd680cf7c0 0x7fdd680ad930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:18:18.938 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:18.929+0000 7fdd6e080640 1 -- 172.21.15.143:0/3679928109 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fdd68143ac0 msgr2=0x7fdd680ade70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:18:18.938 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:18.929+0000 7fdd6e080640 1 --2- 172.21.15.143:0/3679928109 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fdd68143ac0 0x7fdd680ade70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:18:18.938 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:18.929+0000 7fdd6e080640 1 -- 172.21.15.143:0/3679928109 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdd580094d0 con 0x7fdd680708b0 2023-11-26T07:18:18.938 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:18.929+0000 7fdd6d87f640 1 --2- 172.21.15.143:0/3679928109 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fdd680cf7c0 0x7fdd680ad930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-26T07:18:18.938 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:18.929+0000 7fdd6e881640 1 --2- 172.21.15.143:0/3679928109 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fdd68143ac0 0x7fdd680ade70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-26T07:18:18.938 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:18.933+0000 7fdd6e080640 1 --2- 172.21.15.143:0/3679928109 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fdd680708b0 0x7fdd6815a7c0 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7fdd6400d5f0 tx=0x7fdd6400dab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:18:18.939 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:18.933+0000 7fdd5effd640 1 -- 172.21.15.143:0/3679928109 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdd640125f0 con 0x7fdd680708b0 2023-11-26T07:18:18.939 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:18.933+0000 7fdd5effd640 1 -- 172.21.15.143:0/3679928109 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdd64012750 con 0x7fdd680708b0 2023-11-26T07:18:18.939 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:18.933+0000 7fdd5effd640 1 -- 172.21.15.143:0/3679928109 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdd64012ae0 con 0x7fdd680708b0 2023-11-26T07:18:18.939 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:18.933+0000 7fdd702e4640 1 -- 172.21.15.143:0/3679928109 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdd680af930 con 0x7fdd680708b0 2023-11-26T07:18:18.939 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:18.933+0000 7fdd702e4640 1 -- 172.21.15.143:0/3679928109 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdd6815ad00 con 0x7fdd680708b0 2023-11-26T07:18:18.939 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:18.933+0000 7fdd702e4640 1 -- 172.21.15.143:0/3679928109 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fdd6815af50 con 0x7fdd680708b0 2023-11-26T07:18:18.939 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:18.933+0000 7fdd5effd640 1 -- 172.21.15.143:0/3679928109 <== mon.0 v2:172.21.15.143:3300/0 4 ==== mgrmap(e 5) v1 ==== 85852+0+0 (secure 0 0 0) 0x7fdd64012c40 con 0x7fdd680708b0 2023-11-26T07:18:18.940 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:18.933+0000 7fdd5effd640 1 --2- 172.21.15.143:0/3679928109 >> [v2:172.21.15.143:6832/2150557544,v1:172.21.15.143:6833/2150557544] conn(0x7fdd4c066470 0x7fdd4c068930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:18:18.940 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:18.933+0000 7fdd5effd640 1 -- 172.21.15.143:0/3679928109 <== mon.0 v2:172.21.15.143:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fdd64084390 con 0x7fdd680708b0 2023-11-26T07:18:18.940 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:18.933+0000 7fdd6d87f640 1 --2- 172.21.15.143:0/3679928109 >> [v2:172.21.15.143:6832/2150557544,v1:172.21.15.143:6833/2150557544] conn(0x7fdd4c066470 0x7fdd4c068930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:18:18.940 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:18.933+0000 7fdd6d87f640 1 --2- 172.21.15.143:0/3679928109 >> [v2:172.21.15.143:6832/2150557544,v1:172.21.15.143:6833/2150557544] conn(0x7fdd4c066470 0x7fdd4c068930 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7fdd580ad000 tx=0x7fdd58008040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:18:18.943 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:18.937+0000 7fdd5effd640 1 -- 172.21.15.143:0/3679928109 <== mon.0 v2:172.21.15.143:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172649 (secure 0 0 0) 0x7fdd64051800 con 0x7fdd680708b0 2023-11-26T07:18:19.091 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:19.085+0000 7fdd702e4640 1 -- 172.21.15.143:0/3679928109 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7fdd68067680 con 0x7fdd680708b0 2023-11-26T07:18:19.091 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:19.085+0000 7fdd5effd640 1 -- 172.21.15.143:0/3679928109 <== mon.0 v2:172.21.15.143:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v20) v1 ==== 100+0+11 (secure 0 0 0) 0x7fdd64054e20 con 0x7fdd680708b0 2023-11-26T07:18:19.091 INFO:teuthology.orchestra.run.smithi143.stdout:pg_num: 64 2023-11-26T07:18:19.093 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:19.089+0000 7fdd702e4640 1 -- 172.21.15.143:0/3679928109 >> [v2:172.21.15.143:6832/2150557544,v1:172.21.15.143:6833/2150557544] conn(0x7fdd4c066470 msgr2=0x7fdd4c068930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:18:19.093 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:19.089+0000 7fdd702e4640 1 --2- 172.21.15.143:0/3679928109 >> [v2:172.21.15.143:6832/2150557544,v1:172.21.15.143:6833/2150557544] conn(0x7fdd4c066470 0x7fdd4c068930 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7fdd580ad000 tx=0x7fdd58008040 comp rx=0 tx=0).stop 2023-11-26T07:18:19.093 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:19.089+0000 7fdd702e4640 1 -- 172.21.15.143:0/3679928109 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fdd680708b0 msgr2=0x7fdd6815a7c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:18:19.093 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:19.089+0000 7fdd702e4640 1 --2- 172.21.15.143:0/3679928109 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fdd680708b0 0x7fdd6815a7c0 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7fdd6400d5f0 tx=0x7fdd6400dab0 comp rx=0 tx=0).stop 2023-11-26T07:18:19.094 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:19.089+0000 7fdd702e4640 1 -- 172.21.15.143:0/3679928109 shutdown_connections 2023-11-26T07:18:19.094 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:19.089+0000 7fdd702e4640 1 --2- 172.21.15.143:0/3679928109 >> [v2:172.21.15.143:6832/2150557544,v1:172.21.15.143:6833/2150557544] conn(0x7fdd4c066470 0x7fdd4c068930 unknown :-1 s=CLOSED pgs=128 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:18:19.094 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:19.089+0000 7fdd702e4640 1 --2- 172.21.15.143:0/3679928109 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fdd68143ac0 0x7fdd680ade70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:18:19.094 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:19.089+0000 7fdd702e4640 1 --2- 172.21.15.143:0/3679928109 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fdd680cf7c0 0x7fdd680ad930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:18:19.094 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:19.089+0000 7fdd702e4640 1 --2- 172.21.15.143:0/3679928109 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fdd680708b0 0x7fdd6815a7c0 unknown :-1 s=CLOSED pgs=192 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:18:19.094 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:19.089+0000 7fdd702e4640 1 -- 172.21.15.143:0/3679928109 >> 172.21.15.143:0/3679928109 conn(0x7fdd6806a890 msgr2=0x7fdd680ce660 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:18:19.094 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:19.089+0000 7fdd702e4640 1 -- 172.21.15.143:0/3679928109 shutdown_connections 2023-11-26T07:18:19.094 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:19.089+0000 7fdd702e4640 1 -- 172.21.15.143:0/3679928109 wait complete. 2023-11-26T07:18:19.113 DEBUG:teuthology.orchestra.run.smithi143:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-11-26T07:18:19.250 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:19.245+0000 7f561225a640 1 Processor -- start 2023-11-26T07:18:19.250 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:19.245+0000 7f561225a640 1 -- start start 2023-11-26T07:18:19.252 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:19.245+0000 7f561225a640 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7f560c143ac0 0x7f560c070370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:18:19.252 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:19.245+0000 7f561225a640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f560c0708b0 0x7f560c0cf280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:18:19.253 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:19.245+0000 7f561225a640 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7f560c0cf7c0 0x7f560c0b1ef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:18:19.253 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:19.245+0000 7f561225a640 1 -- --> v2:172.21.15.143:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f560c143ea0 con 0x7f560c143ac0 2023-11-26T07:18:19.253 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:19.245+0000 7f561225a640 1 -- --> v2:172.21.15.188:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f560c0cfbf0 con 0x7f560c0cf7c0 2023-11-26T07:18:19.253 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:19.245+0000 7f561225a640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f560c0cc730 con 0x7f560c0708b0 2023-11-26T07:18:19.253 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:19.245+0000 7f560b7fe640 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7f560c143ac0 0x7f560c070370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:18:19.253 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:19.245+0000 7f560b7fe640 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7f560c143ac0 0x7f560c070370 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:43940/0 (socket says 172.21.15.143:43940) 2023-11-26T07:18:19.253 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:19.245+0000 7f560b7fe640 1 -- 172.21.15.143:0/610293514 learned_addr learned my addr 172.21.15.143:0/610293514 (peer_addr_for_me v2:172.21.15.143:0/0) 2023-11-26T07:18:19.253 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:19.245+0000 7f560affd640 1 --2- 172.21.15.143:0/610293514 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f560c0708b0 0x7f560c0cf280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:18:19.253 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:19.245+0000 7f560bfff640 1 --2- 172.21.15.143:0/610293514 >> v2:172.21.15.188:3300/0 conn(0x7f560c0cf7c0 0x7f560c0b1ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:18:19.254 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:19.245+0000 7f560b7fe640 1 -- 172.21.15.143:0/610293514 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f560c0708b0 msgr2=0x7f560c0cf280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-26T07:18:19.254 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:19.245+0000 7f560b7fe640 1 --2- 172.21.15.143:0/610293514 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f560c0708b0 0x7f560c0cf280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:18:19.254 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:19.245+0000 7f560b7fe640 1 -- 172.21.15.143:0/610293514 >> v2:172.21.15.188:3300/0 conn(0x7f560c0cf7c0 msgr2=0x7f560c0b1ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-26T07:18:19.254 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:19.245+0000 7f560b7fe640 1 --2- 172.21.15.143:0/610293514 >> v2:172.21.15.188:3300/0 conn(0x7f560c0cf7c0 0x7f560c0b1ef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:18:19.254 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:19.245+0000 7f560b7fe640 1 -- 172.21.15.143:0/610293514 --> v2:172.21.15.143:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f560c070ee0 con 0x7f560c143ac0 2023-11-26T07:18:19.254 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:19.245+0000 7f560affd640 1 --2- 172.21.15.143:0/610293514 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f560c0708b0 0x7f560c0cf280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-26T07:18:19.254 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:19.245+0000 7f560bfff640 1 --2- 172.21.15.143:0/610293514 >> v2:172.21.15.188:3300/0 conn(0x7f560c0cf7c0 0x7f560c0b1ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-26T07:18:19.255 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:19.249+0000 7f560b7fe640 1 --2- 172.21.15.143:0/610293514 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f560c143ac0 0x7f560c070370 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7f560c0718d0 tx=0x7f55fc01e910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a3068978783f1abf server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:18:19.255 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:19.249+0000 7f560a7fc640 1 -- 172.21.15.143:0/610293514 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f55fc0a78b0 con 0x7f560c143ac0 2023-11-26T07:18:19.256 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:19.249+0000 7f560a7fc640 1 -- 172.21.15.143:0/610293514 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f55fc0a7a10 con 0x7f560c143ac0 2023-11-26T07:18:19.256 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:19.249+0000 7f560a7fc640 1 -- 172.21.15.143:0/610293514 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f55fc0a7da0 con 0x7f560c143ac0 2023-11-26T07:18:19.257 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:19.249+0000 7f561225a640 1 -- 172.21.15.143:0/610293514 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f560c143ac0 msgr2=0x7f560c070370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:18:19.257 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:19.249+0000 7f561225a640 1 --2- 172.21.15.143:0/610293514 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f560c143ac0 0x7f560c070370 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7f560c0718d0 tx=0x7f55fc01e910 comp rx=0 tx=0).stop 2023-11-26T07:18:19.257 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:19.249+0000 7f561225a640 1 -- 172.21.15.143:0/610293514 shutdown_connections 2023-11-26T07:18:19.257 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:19.249+0000 7f561225a640 1 --2- 172.21.15.143:0/610293514 >> v2:172.21.15.188:3300/0 conn(0x7f560c0cf7c0 0x7f560c0b1ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:18:19.257 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:19.249+0000 7f561225a640 1 --2- 172.21.15.143:0/610293514 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f560c0708b0 0x7f560c0cf280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:18:19.257 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:19.249+0000 7f561225a640 1 --2- 172.21.15.143:0/610293514 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f560c143ac0 0x7f560c070370 unknown :-1 s=CLOSED pgs=193 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:18:19.257 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:19.249+0000 7f561225a640 1 -- 172.21.15.143:0/610293514 >> 172.21.15.143:0/610293514 conn(0x7f560c06a890 msgr2=0x7f560c06aca0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:18:19.257 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:19.249+0000 7f561225a640 1 -- 172.21.15.143:0/610293514 shutdown_connections 2023-11-26T07:18:19.258 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:19.249+0000 7f561225a640 1 -- 172.21.15.143:0/610293514 wait complete. 2023-11-26T07:18:19.258 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:19.253+0000 7f561225a640 1 Processor -- start 2023-11-26T07:18:19.258 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:19.253+0000 7f561225a640 1 -- start start 2023-11-26T07:18:19.260 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:19.253+0000 7f561225a640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f560c0708b0 0x7f560c15a590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:18:19.260 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:19.253+0000 7f561225a640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f560c0cf7c0 0x7f560c1411f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:18:19.261 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:19.253+0000 7f561225a640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f560c143ac0 0x7f560c159600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:18:19.261 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:19.253+0000 7f561225a640 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f560c15bf60 con 0x7f560c0708b0 2023-11-26T07:18:19.261 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:19.253+0000 7f561225a640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f560c15aec0 con 0x7f560c143ac0 2023-11-26T07:18:19.261 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:19.253+0000 7f561225a640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f560c15b1a0 con 0x7f560c0cf7c0 2023-11-26T07:18:19.261 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:19.253+0000 7f560b7fe640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f560c0708b0 0x7f560c15a590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:18:19.261 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:19.253+0000 7f560b7fe640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f560c0708b0 0x7f560c15a590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:43950/0 (socket says 172.21.15.143:43950) 2023-11-26T07:18:19.261 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:19.253+0000 7f560b7fe640 1 -- 172.21.15.143:0/4099754673 learned_addr learned my addr 172.21.15.143:0/4099754673 (peer_addr_for_me v2:172.21.15.143:0/0) 2023-11-26T07:18:19.261 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:19.253+0000 7f560affd640 1 --2- 172.21.15.143:0/4099754673 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f560c0cf7c0 0x7f560c1411f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:18:19.261 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:19.253+0000 7f560bfff640 1 --2- 172.21.15.143:0/4099754673 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f560c143ac0 0x7f560c159600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:18:19.261 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:19.253+0000 7f560b7fe640 1 -- 172.21.15.143:0/4099754673 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f560c0cf7c0 msgr2=0x7f560c1411f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:18:19.262 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:19.253+0000 7f560b7fe640 1 --2- 172.21.15.143:0/4099754673 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f560c0cf7c0 0x7f560c1411f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:18:19.262 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:19.253+0000 7f560b7fe640 1 -- 172.21.15.143:0/4099754673 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f560c143ac0 msgr2=0x7f560c159600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:18:19.262 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:19.253+0000 7f560b7fe640 1 --2- 172.21.15.143:0/4099754673 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f560c143ac0 0x7f560c159600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:18:19.262 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:19.253+0000 7f560b7fe640 1 -- 172.21.15.143:0/4099754673 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f55fc00b770 con 0x7f560c0708b0 2023-11-26T07:18:19.262 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:19.253+0000 7f560b7fe640 1 --2- 172.21.15.143:0/4099754673 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f560c0708b0 0x7f560c15a590 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7f55fc00b2c0 tx=0x7f55fc01b6b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:18:19.262 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:19.253+0000 7f55f3fff640 1 -- 172.21.15.143:0/4099754673 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f55fc0af910 con 0x7f560c0708b0 2023-11-26T07:18:19.262 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:19.253+0000 7f55f3fff640 1 -- 172.21.15.143:0/4099754673 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f55fc0afa70 con 0x7f560c0708b0 2023-11-26T07:18:19.262 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:19.253+0000 7f55f3fff640 1 -- 172.21.15.143:0/4099754673 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f55fc0afe20 con 0x7f560c0708b0 2023-11-26T07:18:19.262 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:19.253+0000 7f561225a640 1 -- 172.21.15.143:0/4099754673 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f560c15b480 con 0x7f560c0708b0 2023-11-26T07:18:19.263 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:19.253+0000 7f561225a640 1 -- 172.21.15.143:0/4099754673 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f560c15ba40 con 0x7f560c0708b0 2023-11-26T07:18:19.264 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:19.257+0000 7f561225a640 1 -- 172.21.15.143:0/4099754673 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f560c15bc30 con 0x7f560c0708b0 2023-11-26T07:18:19.264 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:19.257+0000 7f55f3fff640 1 -- 172.21.15.143:0/4099754673 <== mon.0 v2:172.21.15.143:3300/0 4 ==== mgrmap(e 5) v1 ==== 85852+0+0 (secure 0 0 0) 0x7f55fc0b93e0 con 0x7f560c0708b0 2023-11-26T07:18:19.264 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:19.257+0000 7f55f3fff640 1 --2- 172.21.15.143:0/4099754673 >> [v2:172.21.15.143:6832/2150557544,v1:172.21.15.143:6833/2150557544] conn(0x7f55f80664a0 0x7f55f8068960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:18:19.264 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:19.257+0000 7f55f3fff640 1 -- 172.21.15.143:0/4099754673 <== mon.0 v2:172.21.15.143:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f55fc11e0b0 con 0x7f560c0708b0 2023-11-26T07:18:19.264 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:19.257+0000 7f560affd640 1 --2- 172.21.15.143:0/4099754673 >> [v2:172.21.15.143:6832/2150557544,v1:172.21.15.143:6833/2150557544] conn(0x7f55f80664a0 0x7f55f8068960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:18:19.264 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:19.257+0000 7f560affd640 1 --2- 172.21.15.143:0/4099754673 >> [v2:172.21.15.143:6832/2150557544,v1:172.21.15.143:6833/2150557544] conn(0x7f55f80664a0 0x7f55f8068960 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7f560c0b1ba0 tx=0x7f5600008040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:18:19.267 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:19.261+0000 7f55f3fff640 1 -- 172.21.15.143:0/4099754673 <== mon.0 v2:172.21.15.143:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172649 (secure 0 0 0) 0x7f55fc0eb660 con 0x7f560c0708b0 2023-11-26T07:18:19.416 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:19.409+0000 7f561225a640 1 -- 172.21.15.143:0/4099754673 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f560c067680 con 0x7f560c0708b0 2023-11-26T07:18:19.416 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:19.409+0000 7f55f3fff640 1 -- 172.21.15.143:0/4099754673 <== mon.0 v2:172.21.15.143:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v20) v1 ==== 74+0+16541 (secure 0 0 0) 0x7f55fc0bf090 con 0x7f560c0708b0 2023-11-26T07:18:19.417 INFO:teuthology.orchestra.run.smithi143.stdout: 2023-11-26T07:18:19.417 INFO:teuthology.orchestra.run.smithi143.stdout:{"epoch":20,"fsid":"f5033060-2469-4a07-95fe-2b76f5404bc1","created":"2023-11-26T07:17:28.613663+0000","modified":"2023-11-26T07:17:55.505435+0000","last_up_change":"2023-11-26T07:17:41.157813+0000","last_in_change":"2023-11-26T07:17:31.181658+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-11-26T07:17:41.758597+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2023-11-26T07:17:51.878332+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2023-11-26T07:17:52.871396+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"ebbee2c6-023e-44b9-9f1d-6f7bfd309730","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6817","nonce":639807369},{"type":"v1","addr":"172.21.15.143:6819","nonce":639807369}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6821","nonce":639807369},{"type":"v1","addr":"172.21.15.143:6823","nonce":639807369}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6830","nonce":639807369},{"type":"v1","addr":"172.21.15.143:6831","nonce":639807369}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6825","nonce":639807369},{"type":"v1","addr":"172.21.15.143:6828","nonce":639807369}]},"public_addr":"172.21.15.143:6819/639807369","cluster_addr":"172.21.15.143:6823/639807369","heartbeat_back_addr":"172.21.15.143:6831/639807369","heartbeat_front_addr":"172.21.15.143:6828/639807369","state":["exists","up"]},{"osd":1,"uuid":"fec4b8be-7122-49e9-9a19-5850a0700001","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6816","nonce":1144942530},{"type":"v1","addr":"172.21.15.143:6818","nonce":1144942530}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6820","nonce":1144942530},{"type":"v1","addr":"172.21.15.143:6822","nonce":1144942530}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6827","nonce":1144942530},{"type":"v1","addr":"172.21.15.143:6829","nonce":1144942530}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6824","nonce":1144942530},{"type":"v1","addr":"172.21.15.143:6826","nonce":1144942530}]},"public_addr":"172.21.15.143:6818/1144942530","cluster_addr":"172.21.15.143:6822/1144942530","heartbeat_back_addr":"172.21.15.143:6829/1144942530","heartbeat_front_addr":"172.21.15.143:6826/1144942530","state":["exists","up"]},{"osd":2,"uuid":"2888af5b-f933-46ca-8f95-6dd494ff43f6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6808","nonce":3909095186},{"type":"v1","addr":"172.21.15.143:6809","nonce":3909095186}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6810","nonce":3909095186},{"type":"v1","addr":"172.21.15.143:6811","nonce":3909095186}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6814","nonce":3909095186},{"type":"v1","addr":"172.21.15.143:6815","nonce":3909095186}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6812","nonce":3909095186},{"type":"v1","addr":"172.21.15.143:6813","nonce":3909095186}]},"public_addr":"172.21.15.143:6809/3909095186","cluster_addr":"172.21.15.143:6811/3909095186","heartbeat_back_addr":"172.21.15.143:6815/3909095186","heartbeat_front_addr":"172.21.15.143:6813/3909095186","state":["exists","up"]},{"osd":3,"uuid":"d1f52435-7bc1-4081-9550-9cd1ba55f549","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6800","nonce":3854210991},{"type":"v1","addr":"172.21.15.143:6801","nonce":3854210991}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6802","nonce":3854210991},{"type":"v1","addr":"172.21.15.143:6803","nonce":3854210991}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6806","nonce":3854210991},{"type":"v1","addr":"172.21.15.143:6807","nonce":3854210991}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6804","nonce":3854210991},{"type":"v1","addr":"172.21.15.143:6805","nonce":3854210991}]},"public_addr":"172.21.15.143:6801/3854210991","cluster_addr":"172.21.15.143:6803/3854210991","heartbeat_back_addr":"172.21.15.143:6807/3854210991","heartbeat_front_addr":"172.21.15.143:6805/3854210991","state":["exists","up"]},{"osd":4,"uuid":"e2ae8b6e-6e25-49b2-bcbf-aacce68f06a3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6800","nonce":3924606902},{"type":"v1","addr":"172.21.15.188:6801","nonce":3924606902}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6802","nonce":3924606902},{"type":"v1","addr":"172.21.15.188:6803","nonce":3924606902}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6806","nonce":3924606902},{"type":"v1","addr":"172.21.15.188:6807","nonce":3924606902}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6804","nonce":3924606902},{"type":"v1","addr":"172.21.15.188:6805","nonce":3924606902}]},"public_addr":"172.21.15.188:6801/3924606902","cluster_addr":"172.21.15.188:6803/3924606902","heartbeat_back_addr":"172.21.15.188:6807/3924606902","heartbeat_front_addr":"172.21.15.188:6805/3924606902","state":["exists","up"]},{"osd":5,"uuid":"3cec8a32-7698-4c36-8e89-e4ba11934f33","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6813","nonce":186841544},{"type":"v1","addr":"172.21.15.188:6815","nonce":186841544}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6817","nonce":186841544},{"type":"v1","addr":"172.21.15.188:6818","nonce":186841544}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6822","nonce":186841544},{"type":"v1","addr":"172.21.15.188:6823","nonce":186841544}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6820","nonce":186841544},{"type":"v1","addr":"172.21.15.188:6821","nonce":186841544}]},"public_addr":"172.21.15.188:6815/186841544","cluster_addr":"172.21.15.188:6818/186841544","heartbeat_back_addr":"172.21.15.188:6823/186841544","heartbeat_front_addr":"172.21.15.188:6821/186841544","state":["exists","up"]},{"osd":6,"uuid":"58a65d5d-e090-473a-81b4-ad38e85029f9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6808","nonce":2153499405},{"type":"v1","addr":"172.21.15.188:6809","nonce":2153499405}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6810","nonce":2153499405},{"type":"v1","addr":"172.21.15.188:6811","nonce":2153499405}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6816","nonce":2153499405},{"type":"v1","addr":"172.21.15.188:6819","nonce":2153499405}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6812","nonce":2153499405},{"type":"v1","addr":"172.21.15.188:6814","nonce":2153499405}]},"public_addr":"172.21.15.188:6809/2153499405","cluster_addr":"172.21.15.188:6811/2153499405","heartbeat_back_addr":"172.21.15.188:6819/2153499405","heartbeat_front_addr":"172.21.15.188:6814/2153499405","state":["exists","up"]},{"osd":7,"uuid":"ea5b7f70-3554-4cf8-ac30-f7c59b80678d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6824","nonce":2722096729},{"type":"v1","addr":"172.21.15.188:6825","nonce":2722096729}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6826","nonce":2722096729},{"type":"v1","addr":"172.21.15.188:6827","nonce":2722096729}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6830","nonce":2722096729},{"type":"v1","addr":"172.21.15.188:6831","nonce":2722096729}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6828","nonce":2722096729},{"type":"v1","addr":"172.21.15.188:6829","nonce":2722096729}]},"public_addr":"172.21.15.188:6825/2722096729","cluster_addr":"172.21.15.188:6827/2722096729","heartbeat_back_addr":"172.21.15.188:6831/2722096729","heartbeat_front_addr":"172.21.15.188:6829/2722096729","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-26T07:17:36.995032+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-26T07:17:36.932266+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-26T07:17:36.829304+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-26T07:17:36.804297+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-26T07:17:36.667892+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-26T07:17:36.964868+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-26T07:17:36.827332+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-26T07:17:36.893179+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-11-26T07:18:19.419 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:19.413+0000 7f561225a640 1 -- 172.21.15.143:0/4099754673 >> [v2:172.21.15.143:6832/2150557544,v1:172.21.15.143:6833/2150557544] conn(0x7f55f80664a0 msgr2=0x7f55f8068960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:18:19.419 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:19.413+0000 7f561225a640 1 --2- 172.21.15.143:0/4099754673 >> [v2:172.21.15.143:6832/2150557544,v1:172.21.15.143:6833/2150557544] conn(0x7f55f80664a0 0x7f55f8068960 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7f560c0b1ba0 tx=0x7f5600008040 comp rx=0 tx=0).stop 2023-11-26T07:18:19.419 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:19.413+0000 7f561225a640 1 -- 172.21.15.143:0/4099754673 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f560c0708b0 msgr2=0x7f560c15a590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:18:19.419 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:19.413+0000 7f561225a640 1 --2- 172.21.15.143:0/4099754673 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f560c0708b0 0x7f560c15a590 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7f55fc00b2c0 tx=0x7f55fc01b6b0 comp rx=0 tx=0).stop 2023-11-26T07:18:19.420 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:19.413+0000 7f561225a640 1 -- 172.21.15.143:0/4099754673 shutdown_connections 2023-11-26T07:18:19.420 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:19.413+0000 7f561225a640 1 --2- 172.21.15.143:0/4099754673 >> [v2:172.21.15.143:6832/2150557544,v1:172.21.15.143:6833/2150557544] conn(0x7f55f80664a0 0x7f55f8068960 unknown :-1 s=CLOSED pgs=129 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:18:19.420 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:19.413+0000 7f561225a640 1 --2- 172.21.15.143:0/4099754673 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f560c143ac0 0x7f560c159600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:18:19.420 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:19.413+0000 7f561225a640 1 --2- 172.21.15.143:0/4099754673 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f560c0cf7c0 0x7f560c1411f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:18:19.420 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:19.413+0000 7f561225a640 1 --2- 172.21.15.143:0/4099754673 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f560c0708b0 0x7f560c15a590 unknown :-1 s=CLOSED pgs=194 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:18:19.420 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:19.413+0000 7f561225a640 1 -- 172.21.15.143:0/4099754673 >> 172.21.15.143:0/4099754673 conn(0x7f560c06a890 msgr2=0x7f560c0cb0e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:18:19.420 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:19.413+0000 7f561225a640 1 -- 172.21.15.143:0/4099754673 shutdown_connections 2023-11-26T07:18:19.420 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:19.413+0000 7f561225a640 1 -- 172.21.15.143:0/4099754673 wait complete. 2023-11-26T07:18:19.438 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-11-26T07:17:41.758597+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2023-11-26T07:17:51.878332+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2023-11-26T07:17:52.871396+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '20', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2023-11-26T07:18:19.438 DEBUG:teuthology.orchestra.run.smithi143:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-11-26T07:18:19.578 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:19.573+0000 7fcf3e016640 1 Processor -- start 2023-11-26T07:18:19.579 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:19.573+0000 7fcf3e016640 1 -- start start 2023-11-26T07:18:19.580 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:19.573+0000 7fcf3e016640 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7fcf3806f000 0x7fcf3806d030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:18:19.580 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:19.573+0000 7fcf3e016640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fcf3806f3e0 0x7fcf3806b5a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:18:19.580 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:19.573+0000 7fcf3e016640 1 -- --> v1:172.21.15.188:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fcf380cf260 con 0x7fcf38099a70 2023-11-26T07:18:19.580 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:19.573+0000 7fcf3e016640 1 -- --> v2:172.21.15.188:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fcf3806d570 con 0x7fcf3806f000 2023-11-26T07:18:19.580 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:19.573+0000 7fcf3e016640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcf3809a010 con 0x7fcf3806f3e0 2023-11-26T07:18:19.581 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:19.573+0000 7fcf377fe640 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7fcf3806f000 0x7fcf3806d030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:18:19.581 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:19.573+0000 7fcf36ffd640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fcf3806f3e0 0x7fcf3806b5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:18:19.581 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:19.573+0000 7fcf36ffd640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fcf3806f3e0 0x7fcf3806b5a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.188:3301/0 says I am v2:172.21.15.143:45520/0 (socket says 172.21.15.143:45520) 2023-11-26T07:18:19.581 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:19.573+0000 7fcf36ffd640 1 -- 172.21.15.143:0/191396831 learned_addr learned my addr 172.21.15.143:0/191396831 (peer_addr_for_me v2:172.21.15.143:0/0) 2023-11-26T07:18:19.581 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:19.573+0000 7fcf36ffd640 1 -- 172.21.15.143:0/191396831 >> v2:172.21.15.188:3300/0 conn(0x7fcf3806f000 msgr2=0x7fcf3806d030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-26T07:18:19.581 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:19.573+0000 7fcf36ffd640 1 --2- 172.21.15.143:0/191396831 >> v2:172.21.15.188:3300/0 conn(0x7fcf3806f000 0x7fcf3806d030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:18:19.581 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:19.573+0000 7fcf36ffd640 1 -- 172.21.15.143:0/191396831 >> v1:172.21.15.188:6789/0 conn(0x7fcf38099a70 legacy=0x7fcf3806bae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:18:19.581 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:19.577+0000 7fcf36ffd640 1 -- 172.21.15.143:0/191396831 --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcf3809a2f0 con 0x7fcf3806f3e0 2023-11-26T07:18:19.581 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:19.577+0000 7fcf377fe640 1 --2- 172.21.15.143:0/191396831 >> v2:172.21.15.188:3300/0 conn(0x7fcf3806f000 0x7fcf3806d030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-26T07:18:19.583 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:19.577+0000 7fcf36ffd640 1 --2- 172.21.15.143:0/191396831 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fcf3806f3e0 0x7fcf3806b5a0 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7fcf2c009870 tx=0x7fcf2c01c540 comp rx=0 tx=0).ready entity=mon.2 client_cookie=c27d159eba3bb34d server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:18:19.584 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:19.577+0000 7fcf367fc640 1 -- 172.21.15.143:0/191396831 <== mon.2 v2:172.21.15.188:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcf2c0a7070 con 0x7fcf3806f3e0 2023-11-26T07:18:19.584 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:19.577+0000 7fcf367fc640 1 -- 172.21.15.143:0/191396831 <== mon.2 v2:172.21.15.188:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcf2c0ab430 con 0x7fcf3806f3e0 2023-11-26T07:18:19.584 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:19.577+0000 7fcf367fc640 1 -- 172.21.15.143:0/191396831 <== mon.2 v2:172.21.15.188:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcf2c0ab7c0 con 0x7fcf3806f3e0 2023-11-26T07:18:19.585 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:19.577+0000 7fcf3e016640 1 -- 172.21.15.143:0/191396831 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fcf3806f3e0 msgr2=0x7fcf3806b5a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:18:19.585 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:19.577+0000 7fcf3e016640 1 --2- 172.21.15.143:0/191396831 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fcf3806f3e0 0x7fcf3806b5a0 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7fcf2c009870 tx=0x7fcf2c01c540 comp rx=0 tx=0).stop 2023-11-26T07:18:19.585 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:19.577+0000 7fcf3e016640 1 -- 172.21.15.143:0/191396831 shutdown_connections 2023-11-26T07:18:19.585 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:19.577+0000 7fcf3e016640 1 --2- 172.21.15.143:0/191396831 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fcf3806f3e0 0x7fcf3806b5a0 unknown :-1 s=CLOSED pgs=57 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:18:19.585 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:19.577+0000 7fcf3e016640 1 --2- 172.21.15.143:0/191396831 >> v2:172.21.15.188:3300/0 conn(0x7fcf3806f000 0x7fcf3806d030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:18:19.585 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:19.577+0000 7fcf3e016640 1 -- 172.21.15.143:0/191396831 >> 172.21.15.143:0/191396831 conn(0x7fcf380b2d10 msgr2=0x7fcf380b3120 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:18:19.585 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:19.577+0000 7fcf3e016640 1 -- 172.21.15.143:0/191396831 shutdown_connections 2023-11-26T07:18:19.585 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:19.577+0000 7fcf3e016640 1 -- 172.21.15.143:0/191396831 wait complete. 2023-11-26T07:18:19.586 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:19.581+0000 7fcf3e016640 1 Processor -- start 2023-11-26T07:18:19.586 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:19.581+0000 7fcf3e016640 1 -- start start 2023-11-26T07:18:19.588 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:19.581+0000 7fcf3e016640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fcf3806f000 0x7fcf38160c50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:18:19.588 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:19.581+0000 7fcf3e016640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fcf3806f3e0 0x7fcf38161c60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:18:19.588 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:19.581+0000 7fcf3e016640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fcf38099a70 0x7fcf38157470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:18:19.588 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:19.581+0000 7fcf3e016640 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcf381621a0 con 0x7fcf3806f3e0 2023-11-26T07:18:19.588 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:19.581+0000 7fcf3e016640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcf38168230 con 0x7fcf3806f000 2023-11-26T07:18:19.589 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:19.581+0000 7fcf3e016640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcf38161230 con 0x7fcf38099a70 2023-11-26T07:18:19.589 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:19.581+0000 7fcf36ffd640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fcf3806f3e0 0x7fcf38161c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:18:19.589 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:19.581+0000 7fcf377fe640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fcf3806f000 0x7fcf38160c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:18:19.589 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:19.581+0000 7fcf36ffd640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fcf3806f3e0 0x7fcf38161c60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:43954/0 (socket says 172.21.15.143:43954) 2023-11-26T07:18:19.589 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:19.581+0000 7fcf36ffd640 1 -- 172.21.15.143:0/1907067180 learned_addr learned my addr 172.21.15.143:0/1907067180 (peer_addr_for_me v2:172.21.15.143:0/0) 2023-11-26T07:18:19.589 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:19.581+0000 7fcf37fff640 1 --2- 172.21.15.143:0/1907067180 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fcf38099a70 0x7fcf38157470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:18:19.589 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:19.581+0000 7fcf36ffd640 1 -- 172.21.15.143:0/1907067180 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fcf38099a70 msgr2=0x7fcf38157470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:18:19.589 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:19.581+0000 7fcf36ffd640 1 --2- 172.21.15.143:0/1907067180 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fcf38099a70 0x7fcf38157470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:18:19.589 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:19.581+0000 7fcf36ffd640 1 -- 172.21.15.143:0/1907067180 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fcf3806f000 msgr2=0x7fcf38160c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:18:19.589 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:19.581+0000 7fcf36ffd640 1 --2- 172.21.15.143:0/1907067180 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fcf3806f000 0x7fcf38160c50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:18:19.589 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:19.581+0000 7fcf36ffd640 1 -- 172.21.15.143:0/1907067180 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcf28009420 con 0x7fcf3806f3e0 2023-11-26T07:18:19.589 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:19.581+0000 7fcf377fe640 1 --2- 172.21.15.143:0/1907067180 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fcf3806f000 0x7fcf38160c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-26T07:18:19.590 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:19.581+0000 7fcf36ffd640 1 --2- 172.21.15.143:0/1907067180 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fcf3806f3e0 0x7fcf38161c60 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7fcf2c01a170 tx=0x7fcf2c0192e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:18:19.591 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:19.581+0000 7fcf1ffff640 1 -- 172.21.15.143:0/1907067180 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcf2c0ba070 con 0x7fcf3806f3e0 2023-11-26T07:18:19.591 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:19.585+0000 7fcf1ffff640 1 -- 172.21.15.143:0/1907067180 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcf2c0b4420 con 0x7fcf3806f3e0 2023-11-26T07:18:19.591 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:19.585+0000 7fcf1ffff640 1 -- 172.21.15.143:0/1907067180 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcf2c0b47d0 con 0x7fcf3806f3e0 2023-11-26T07:18:19.591 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:19.585+0000 7fcf3e016640 1 -- 172.21.15.143:0/1907067180 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcf2c0094d0 con 0x7fcf3806f3e0 2023-11-26T07:18:19.591 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:19.585+0000 7fcf3e016640 1 -- 172.21.15.143:0/1907067180 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcf38167ec0 con 0x7fcf3806f3e0 2023-11-26T07:18:19.591 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:19.585+0000 7fcf3e016640 1 -- 172.21.15.143:0/1907067180 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fcf38168370 con 0x7fcf3806f3e0 2023-11-26T07:18:19.591 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:19.585+0000 7fcf1ffff640 1 -- 172.21.15.143:0/1907067180 <== mon.0 v2:172.21.15.143:3300/0 4 ==== mgrmap(e 5) v1 ==== 85852+0+0 (secure 0 0 0) 0x7fcf2c0b4930 con 0x7fcf3806f3e0 2023-11-26T07:18:19.592 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:19.585+0000 7fcf1ffff640 1 --2- 172.21.15.143:0/1907067180 >> [v2:172.21.15.143:6832/2150557544,v1:172.21.15.143:6833/2150557544] conn(0x7fcf24066470 0x7fcf24068930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:18:19.592 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:19.585+0000 7fcf1ffff640 1 -- 172.21.15.143:0/1907067180 <== mon.0 v2:172.21.15.143:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fcf2c11d7b0 con 0x7fcf3806f3e0 2023-11-26T07:18:19.592 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:19.585+0000 7fcf377fe640 1 --2- 172.21.15.143:0/1907067180 >> [v2:172.21.15.143:6832/2150557544,v1:172.21.15.143:6833/2150557544] conn(0x7fcf24066470 0x7fcf24068930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:18:19.592 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:19.585+0000 7fcf377fe640 1 --2- 172.21.15.143:0/1907067180 >> [v2:172.21.15.143:6832/2150557544,v1:172.21.15.143:6833/2150557544] conn(0x7fcf24066470 0x7fcf24068930 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7fcf28009fd0 tx=0x7fcf28009190 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:18:19.595 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:19.589+0000 7fcf1ffff640 1 -- 172.21.15.143:0/1907067180 <== mon.0 v2:172.21.15.143:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172649 (secure 0 0 0) 0x7fcf2c0eac40 con 0x7fcf3806f3e0 2023-11-26T07:18:19.751 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:19.745+0000 7fcf3e016640 1 -- 172.21.15.143:0/1907067180 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7fcf38067680 con 0x7fcf3806f3e0 2023-11-26T07:18:19.751 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:19.745+0000 7fcf1ffff640 1 -- 172.21.15.143:0/1907067180 <== mon.0 v2:172.21.15.143:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v20) v1 ==== 93+0+10 (secure 0 0 0) 0x7fcf2c0ee260 con 0x7fcf3806f3e0 2023-11-26T07:18:19.751 INFO:teuthology.orchestra.run.smithi143.stdout:pg_num: 1 2023-11-26T07:18:19.753 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:19.749+0000 7fcf3e016640 1 -- 172.21.15.143:0/1907067180 >> [v2:172.21.15.143:6832/2150557544,v1:172.21.15.143:6833/2150557544] conn(0x7fcf24066470 msgr2=0x7fcf24068930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:18:19.754 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:19.749+0000 7fcf3e016640 1 --2- 172.21.15.143:0/1907067180 >> [v2:172.21.15.143:6832/2150557544,v1:172.21.15.143:6833/2150557544] conn(0x7fcf24066470 0x7fcf24068930 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7fcf28009fd0 tx=0x7fcf28009190 comp rx=0 tx=0).stop 2023-11-26T07:18:19.754 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:19.749+0000 7fcf3e016640 1 -- 172.21.15.143:0/1907067180 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fcf3806f3e0 msgr2=0x7fcf38161c60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:18:19.754 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:19.749+0000 7fcf3e016640 1 --2- 172.21.15.143:0/1907067180 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fcf3806f3e0 0x7fcf38161c60 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7fcf2c01a170 tx=0x7fcf2c0192e0 comp rx=0 tx=0).stop 2023-11-26T07:18:19.754 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:19.749+0000 7fcf3e016640 1 -- 172.21.15.143:0/1907067180 shutdown_connections 2023-11-26T07:18:19.754 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:19.749+0000 7fcf3e016640 1 --2- 172.21.15.143:0/1907067180 >> [v2:172.21.15.143:6832/2150557544,v1:172.21.15.143:6833/2150557544] conn(0x7fcf24066470 0x7fcf24068930 unknown :-1 s=CLOSED pgs=130 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:18:19.755 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:19.749+0000 7fcf3e016640 1 --2- 172.21.15.143:0/1907067180 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fcf38099a70 0x7fcf38157470 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:18:19.755 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:19.749+0000 7fcf3e016640 1 --2- 172.21.15.143:0/1907067180 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fcf3806f3e0 0x7fcf38161c60 unknown :-1 s=CLOSED pgs=195 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:18:19.755 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:19.749+0000 7fcf3e016640 1 --2- 172.21.15.143:0/1907067180 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fcf3806f000 0x7fcf38160c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:18:19.755 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:19.749+0000 7fcf3e016640 1 -- 172.21.15.143:0/1907067180 >> 172.21.15.143:0/1907067180 conn(0x7fcf380b2d10 msgr2=0x7fcf380cca70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:18:19.756 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:19.749+0000 7fcf3e016640 1 -- 172.21.15.143:0/1907067180 shutdown_connections 2023-11-26T07:18:19.756 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:19.749+0000 7fcf3e016640 1 -- 172.21.15.143:0/1907067180 wait complete. 2023-11-26T07:18:19.773 DEBUG:teuthology.orchestra.run.smithi143:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-11-26T07:18:19.914 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:19.909+0000 7fb3ce11d640 1 Processor -- start 2023-11-26T07:18:19.914 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:19.909+0000 7fb3ce11d640 1 -- start start 2023-11-26T07:18:19.927 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:19.909+0000 7fb3ce11d640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fb3c813e290 0x7fb3c80cf280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:18:19.927 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:19.909+0000 7fb3ce11d640 1 -- --> v1:172.21.15.143:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb3c806c520 con 0x7fb3c815f6b0 2023-11-26T07:18:19.928 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:19.909+0000 7fb3ce11d640 1 -- --> v1:172.21.15.188:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb3c806c800 con 0x7fb3c80cf7c0 2023-11-26T07:18:19.928 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:19.909+0000 7fb3ce11d640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb3c80cfc40 con 0x7fb3c813e290 2023-11-26T07:18:19.928 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:19.909+0000 7fb3c7fff640 1 --1- >> v1:172.21.15.143:6789/0 conn(0x7fb3c815f6b0 0x7fb3c80cd7b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.143:6789/0 says I am v1:172.21.15.143:43910/0 (socket says 172.21.15.143:43910) 2023-11-26T07:18:19.928 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:19.909+0000 7fb3c7fff640 1 -- 172.21.15.143:0/3677551020 learned_addr learned my addr 172.21.15.143:0/3677551020 (peer_addr_for_me v1:172.21.15.143:0/0) 2023-11-26T07:18:19.928 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:19.909+0000 7fb3c77fe640 1 --2- 172.21.15.143:0/3677551020 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fb3c813e290 0x7fb3c80cf280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:18:19.928 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:19.909+0000 7fb3c67fc640 1 -- 172.21.15.143:0/3677551020 <== mon.0 v1:172.21.15.143:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4230792788 0 0) 0x7fb3c806c520 con 0x7fb3c815f6b0 2023-11-26T07:18:19.928 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:19.909+0000 7fb3c67fc640 1 -- 172.21.15.143:0/3677551020 --> v1:172.21.15.143:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fb3b0003530 con 0x7fb3c815f6b0 2023-11-26T07:18:19.928 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:19.909+0000 7fb3c67fc640 1 -- 172.21.15.143:0/3677551020 <== mon.0 v1:172.21.15.143:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 4163632553 0 0) 0x7fb3b0003530 con 0x7fb3c815f6b0 2023-11-26T07:18:19.928 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:19.909+0000 7fb3c67fc640 1 -- 172.21.15.143:0/3677551020 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fb3c813e290 msgr2=0x7fb3c80cf280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-26T07:18:19.928 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:19.909+0000 7fb3c67fc640 1 --2- 172.21.15.143:0/3677551020 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fb3c813e290 0x7fb3c80cf280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:18:19.928 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:19.909+0000 7fb3c67fc640 1 -- 172.21.15.143:0/3677551020 >> v1:172.21.15.188:6789/0 conn(0x7fb3c80cf7c0 legacy=0x7fb3c815efa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:18:19.928 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:19.909+0000 7fb3c67fc640 1 -- 172.21.15.143:0/3677551020 --> v1:172.21.15.143:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb3c806cae0 con 0x7fb3c815f6b0 2023-11-26T07:18:19.929 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:19.909+0000 7fb3c67fc640 1 -- 172.21.15.143:0/3677551020 <== mon.0 v1:172.21.15.143:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2137882523 0 0) 0x7fb3c0001a00 con 0x7fb3c815f6b0 2023-11-26T07:18:19.929 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:19.909+0000 7fb3c67fc640 1 -- 172.21.15.143:0/3677551020 >> v1:172.21.15.143:6789/0 conn(0x7fb3c815f6b0 legacy=0x7fb3c80cd7b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:18:19.929 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:19.909+0000 7fb3c67fc640 1 --2- 172.21.15.143:0/3677551020 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fb3b00039b0 0x7fb3b0003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:18:19.929 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:19.909+0000 7fb3c67fc640 1 --2- 172.21.15.143:0/3677551020 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fb3c813e290 0x7fb3b00047d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:18:19.929 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:19.909+0000 7fb3c67fc640 1 --2- 172.21.15.143:0/3677551020 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fb3b0004d10 0x7fb3b0005190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:18:19.929 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:19.909+0000 7fb3c67fc640 1 -- 172.21.15.143:0/3677551020 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb3c80cfc40 con 0x7fb3b00039b0 2023-11-26T07:18:19.930 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:19.909+0000 7fb3c67fc640 1 -- 172.21.15.143:0/3677551020 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb3b000ba40 con 0x7fb3b0004d10 2023-11-26T07:18:19.930 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:19.909+0000 7fb3c67fc640 1 -- 172.21.15.143:0/3677551020 --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb3b000bc00 con 0x7fb3c813e290 2023-11-26T07:18:19.930 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:19.909+0000 7fb3c77fe640 1 --2- 172.21.15.143:0/3677551020 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fb3b00039b0 0x7fb3b0003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:18:19.930 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:19.909+0000 7fb3c6ffd640 1 --2- 172.21.15.143:0/3677551020 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fb3c813e290 0x7fb3b00047d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:18:19.930 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:19.909+0000 7fb3c7fff640 1 --2- 172.21.15.143:0/3677551020 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fb3b0004d10 0x7fb3b0005190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:18:19.930 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:19.909+0000 7fb3c77fe640 1 -- 172.21.15.143:0/3677551020 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fb3c813e290 msgr2=0x7fb3b00047d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-26T07:18:19.931 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:19.909+0000 7fb3c77fe640 1 --2- 172.21.15.143:0/3677551020 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fb3c813e290 0x7fb3b00047d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:18:19.931 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:19.909+0000 7fb3c77fe640 1 -- 172.21.15.143:0/3677551020 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fb3b0004d10 msgr2=0x7fb3b0005190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-26T07:18:19.931 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:19.909+0000 7fb3c77fe640 1 --2- 172.21.15.143:0/3677551020 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fb3b0004d10 0x7fb3b0005190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:18:19.931 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:19.909+0000 7fb3c77fe640 1 -- 172.21.15.143:0/3677551020 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb3b0003530 con 0x7fb3b00039b0 2023-11-26T07:18:19.931 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:19.909+0000 7fb3c6ffd640 1 --2- 172.21.15.143:0/3677551020 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fb3c813e290 0x7fb3b00047d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-26T07:18:19.931 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:19.909+0000 7fb3c7fff640 1 --2- 172.21.15.143:0/3677551020 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fb3b0004d10 0x7fb3b0005190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-26T07:18:19.931 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:19.913+0000 7fb3c77fe640 1 --2- 172.21.15.143:0/3677551020 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fb3b00039b0 0x7fb3b0003dd0 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7fb3b8009ab0 tx=0x7fb3b801ca30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e96efde04602c454 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:18:19.931 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:19.913+0000 7fb3c67fc640 1 -- 172.21.15.143:0/3677551020 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb3b80a6410 con 0x7fb3b00039b0 2023-11-26T07:18:19.931 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:19.913+0000 7fb3c67fc640 1 -- 172.21.15.143:0/3677551020 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb3b80a6570 con 0x7fb3b00039b0 2023-11-26T07:18:19.931 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:19.913+0000 7fb3c67fc640 1 -- 172.21.15.143:0/3677551020 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb3b80a6920 con 0x7fb3b00039b0 2023-11-26T07:18:19.932 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:19.913+0000 7fb3ce11d640 1 -- 172.21.15.143:0/3677551020 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fb3b00039b0 msgr2=0x7fb3b0003dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:18:19.932 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:19.913+0000 7fb3ce11d640 1 --2- 172.21.15.143:0/3677551020 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fb3b00039b0 0x7fb3b0003dd0 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7fb3b8009ab0 tx=0x7fb3b801ca30 comp rx=0 tx=0).stop 2023-11-26T07:18:19.932 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:19.913+0000 7fb3c7fff640 1 -- 172.21.15.143:0/3677551020 reap_dead start 2023-11-26T07:18:19.932 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:19.913+0000 7fb3ce11d640 1 -- 172.21.15.143:0/3677551020 shutdown_connections 2023-11-26T07:18:19.932 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:19.913+0000 7fb3ce11d640 1 -- 172.21.15.143:0/3677551020 >> 172.21.15.143:0/3677551020 conn(0x7fb3c806a890 msgr2=0x7fb3c806aca0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:18:19.932 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:19.913+0000 7fb3ce11d640 1 -- 172.21.15.143:0/3677551020 shutdown_connections 2023-11-26T07:18:19.932 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:19.913+0000 7fb3ce11d640 1 -- 172.21.15.143:0/3677551020 wait complete. 2023-11-26T07:18:19.932 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:19.917+0000 7fb3ce11d640 1 Processor -- start 2023-11-26T07:18:19.932 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:19.917+0000 7fb3ce11d640 1 -- start start 2023-11-26T07:18:19.932 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:19.917+0000 7fb3ce11d640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fb3b00039b0 0x7fb3c81f1f50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:18:19.933 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:19.917+0000 7fb3ce11d640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fb3c80b3060 0x7fb3c80aeb60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:18:19.933 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:19.917+0000 7fb3ce11d640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fb3c813c8e0 0x7fb3c813cd60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:18:19.933 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:19.917+0000 7fb3ce11d640 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb3c813d2a0 con 0x7fb3b00039b0 2023-11-26T07:18:19.933 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:19.917+0000 7fb3ce11d640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb3c81f2490 con 0x7fb3c80b3060 2023-11-26T07:18:19.933 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:19.917+0000 7fb3ce11d640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb3c80af3b0 con 0x7fb3c813c8e0 2023-11-26T07:18:19.933 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:19.917+0000 7fb3c77fe640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fb3b00039b0 0x7fb3c81f1f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:18:19.933 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:19.917+0000 7fb3c77fe640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fb3b00039b0 0x7fb3c81f1f50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:43972/0 (socket says 172.21.15.143:43972) 2023-11-26T07:18:19.933 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:19.917+0000 7fb3c77fe640 1 -- 172.21.15.143:0/46585914 learned_addr learned my addr 172.21.15.143:0/46585914 (peer_addr_for_me v2:172.21.15.143:0/0) 2023-11-26T07:18:19.933 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:19.917+0000 7fb3c6ffd640 1 --2- 172.21.15.143:0/46585914 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fb3c80b3060 0x7fb3c80aeb60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:18:19.933 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:19.917+0000 7fb3c7fff640 1 --2- 172.21.15.143:0/46585914 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fb3c813c8e0 0x7fb3c813cd60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:18:19.934 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:19.917+0000 7fb3c77fe640 1 -- 172.21.15.143:0/46585914 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fb3c813c8e0 msgr2=0x7fb3c813cd60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:18:19.934 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:19.917+0000 7fb3c77fe640 1 --2- 172.21.15.143:0/46585914 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fb3c813c8e0 0x7fb3c813cd60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:18:19.934 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:19.917+0000 7fb3c77fe640 1 -- 172.21.15.143:0/46585914 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fb3c80b3060 msgr2=0x7fb3c80aeb60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:18:19.934 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:19.917+0000 7fb3c77fe640 1 --2- 172.21.15.143:0/46585914 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fb3c80b3060 0x7fb3c80aeb60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:18:19.934 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:19.917+0000 7fb3c77fe640 1 -- 172.21.15.143:0/46585914 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb3c80af690 con 0x7fb3b00039b0 2023-11-26T07:18:19.934 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:19.917+0000 7fb3c77fe640 1 --2- 172.21.15.143:0/46585914 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fb3b00039b0 0x7fb3c81f1f50 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7fb3b8009ab0 tx=0x7fb3b0005800 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:18:19.934 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:19.917+0000 7fb3affff640 1 -- 172.21.15.143:0/46585914 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb3b800f000 con 0x7fb3b00039b0 2023-11-26T07:18:19.934 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:19.917+0000 7fb3affff640 1 -- 172.21.15.143:0/46585914 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb3b80a6d60 con 0x7fb3b00039b0 2023-11-26T07:18:19.934 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:19.917+0000 7fb3affff640 1 -- 172.21.15.143:0/46585914 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb3b80b9630 con 0x7fb3b00039b0 2023-11-26T07:18:19.934 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:19.917+0000 7fb3ce11d640 1 -- 172.21.15.143:0/46585914 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb3c80af970 con 0x7fb3b00039b0 2023-11-26T07:18:19.935 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:19.917+0000 7fb3ce11d640 1 -- 172.21.15.143:0/46585914 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb3c80aff30 con 0x7fb3b00039b0 2023-11-26T07:18:19.935 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:19.921+0000 7fb3ce11d640 1 -- 172.21.15.143:0/46585914 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb3c8067680 con 0x7fb3b00039b0 2023-11-26T07:18:19.935 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:19.921+0000 7fb3affff640 1 -- 172.21.15.143:0/46585914 <== mon.0 v2:172.21.15.143:3300/0 4 ==== mgrmap(e 5) v1 ==== 85852+0+0 (secure 0 0 0) 0x7fb3b80b03f0 con 0x7fb3b00039b0 2023-11-26T07:18:19.935 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:19.921+0000 7fb3affff640 1 --2- 172.21.15.143:0/46585914 >> [v2:172.21.15.143:6832/2150557544,v1:172.21.15.143:6833/2150557544] conn(0x7fb3b4066470 0x7fb3b4068930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:18:19.935 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:19.921+0000 7fb3affff640 1 -- 172.21.15.143:0/46585914 <== mon.0 v2:172.21.15.143:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fb3b811d810 con 0x7fb3b00039b0 2023-11-26T07:18:19.935 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:19.921+0000 7fb3c6ffd640 1 --2- 172.21.15.143:0/46585914 >> [v2:172.21.15.143:6832/2150557544,v1:172.21.15.143:6833/2150557544] conn(0x7fb3b4066470 0x7fb3b4068930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:18:19.936 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:19.921+0000 7fb3c6ffd640 1 --2- 172.21.15.143:0/46585914 >> [v2:172.21.15.143:6832/2150557544,v1:172.21.15.143:6833/2150557544] conn(0x7fb3b4066470 0x7fb3b4068930 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7fb3bc007fd0 tx=0x7fb3bc009040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:18:19.936 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:19.925+0000 7fb3affff640 1 -- 172.21.15.143:0/46585914 <== mon.0 v2:172.21.15.143:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172649 (secure 0 0 0) 0x7fb3b80eac10 con 0x7fb3b00039b0 2023-11-26T07:18:20.082 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:20.077+0000 7fb3ce11d640 1 -- 172.21.15.143:0/46585914 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7fb3c813d6b0 con 0x7fb3b00039b0 2023-11-26T07:18:20.084 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:20.077+0000 7fb3affff640 1 -- 172.21.15.143:0/46585914 <== mon.0 v2:172.21.15.143:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v20) v1 ==== 104+0+11 (secure 0 0 0) 0x7fb3b80be020 con 0x7fb3b00039b0 2023-11-26T07:18:20.084 INFO:teuthology.orchestra.run.smithi143.stdout:pg_num: 32 2023-11-26T07:18:20.085 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:20.077+0000 7fb3ce11d640 1 -- 172.21.15.143:0/46585914 >> [v2:172.21.15.143:6832/2150557544,v1:172.21.15.143:6833/2150557544] conn(0x7fb3b4066470 msgr2=0x7fb3b4068930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:18:20.085 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:20.077+0000 7fb3ce11d640 1 --2- 172.21.15.143:0/46585914 >> [v2:172.21.15.143:6832/2150557544,v1:172.21.15.143:6833/2150557544] conn(0x7fb3b4066470 0x7fb3b4068930 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7fb3bc007fd0 tx=0x7fb3bc009040 comp rx=0 tx=0).stop 2023-11-26T07:18:20.086 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:20.077+0000 7fb3ce11d640 1 -- 172.21.15.143:0/46585914 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fb3b00039b0 msgr2=0x7fb3c81f1f50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:18:20.086 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:20.077+0000 7fb3ce11d640 1 --2- 172.21.15.143:0/46585914 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fb3b00039b0 0x7fb3c81f1f50 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7fb3b8009ab0 tx=0x7fb3b0005800 comp rx=0 tx=0).stop 2023-11-26T07:18:20.086 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:20.077+0000 7fb3ce11d640 1 -- 172.21.15.143:0/46585914 shutdown_connections 2023-11-26T07:18:20.086 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:20.077+0000 7fb3ce11d640 1 --2- 172.21.15.143:0/46585914 >> [v2:172.21.15.143:6832/2150557544,v1:172.21.15.143:6833/2150557544] conn(0x7fb3b4066470 0x7fb3b4068930 unknown :-1 s=CLOSED pgs=131 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:18:20.086 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:20.077+0000 7fb3ce11d640 1 --2- 172.21.15.143:0/46585914 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fb3c813c8e0 0x7fb3c813cd60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:18:20.086 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:20.077+0000 7fb3ce11d640 1 --2- 172.21.15.143:0/46585914 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fb3c80b3060 0x7fb3c80aeb60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:18:20.086 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:20.077+0000 7fb3ce11d640 1 --2- 172.21.15.143:0/46585914 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fb3b00039b0 0x7fb3c81f1f50 unknown :-1 s=CLOSED pgs=198 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:18:20.086 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:20.077+0000 7fb3ce11d640 1 -- 172.21.15.143:0/46585914 >> 172.21.15.143:0/46585914 conn(0x7fb3c806a890 msgr2=0x7fb3c8138310 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:18:20.086 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:20.081+0000 7fb3ce11d640 1 -- 172.21.15.143:0/46585914 shutdown_connections 2023-11-26T07:18:20.086 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:20.081+0000 7fb3ce11d640 1 -- 172.21.15.143:0/46585914 wait complete. 2023-11-26T07:18:20.104 DEBUG:teuthology.orchestra.run.smithi143:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-11-26T07:18:20.245 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:20.237+0000 7fdc1ea79640 1 Processor -- start 2023-11-26T07:18:20.245 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:20.241+0000 7fdc1ea79640 1 -- start start 2023-11-26T07:18:20.247 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:20.241+0000 7fdc1ea79640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fdc180cf990 0x7fdc1815efa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:18:20.247 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:20.241+0000 7fdc1ea79640 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7fdc1815f4e0 0x7fdc180cd7b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:18:20.247 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:20.241+0000 7fdc1ea79640 1 -- --> v1:172.21.15.143:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fdc1806c520 con 0x7fdc1813e290 2023-11-26T07:18:20.248 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:20.241+0000 7fdc1ea79640 1 -- --> v2:172.21.15.188:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fdc1806c800 con 0x7fdc1815f4e0 2023-11-26T07:18:20.248 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:20.241+0000 7fdc1ea79640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdc1806cae0 con 0x7fdc180cf990 2023-11-26T07:18:20.248 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:20.241+0000 7fdc1c815640 1 --1- >> v1:172.21.15.143:6789/0 conn(0x7fdc1813e290 0x7fdc180cf280 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.143:6789/0 says I am v1:172.21.15.143:43916/0 (socket says 172.21.15.143:43916) 2023-11-26T07:18:20.248 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:20.241+0000 7fdc1c815640 1 -- 172.21.15.143:0/141922430 learned_addr learned my addr 172.21.15.143:0/141922430 (peer_addr_for_me v1:172.21.15.143:0/0) 2023-11-26T07:18:20.248 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:20.241+0000 7fdc17fff640 1 --2- 172.21.15.143:0/141922430 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fdc180cf990 0x7fdc1815efa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:18:20.248 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:20.241+0000 7fdc177fe640 1 -- 172.21.15.143:0/141922430 <== mon.0 v1:172.21.15.143:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1350962974 0 0) 0x7fdc1806c520 con 0x7fdc1813e290 2023-11-26T07:18:20.248 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:20.241+0000 7fdc1d016640 1 --2- 172.21.15.143:0/141922430 >> v2:172.21.15.188:3300/0 conn(0x7fdc1815f4e0 0x7fdc180cd7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:18:20.248 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:20.241+0000 7fdc177fe640 1 -- 172.21.15.143:0/141922430 --> v1:172.21.15.143:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fdc08003530 con 0x7fdc1813e290 2023-11-26T07:18:20.248 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:20.241+0000 7fdc177fe640 1 -- 172.21.15.143:0/141922430 <== mon.0 v1:172.21.15.143:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2861377409 0 0) 0x7fdc08003530 con 0x7fdc1813e290 2023-11-26T07:18:20.249 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:20.241+0000 7fdc177fe640 1 -- 172.21.15.143:0/141922430 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fdc180cf990 msgr2=0x7fdc1815efa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-26T07:18:20.249 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:20.241+0000 7fdc177fe640 1 --2- 172.21.15.143:0/141922430 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fdc180cf990 0x7fdc1815efa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:18:20.249 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:20.241+0000 7fdc177fe640 1 -- 172.21.15.143:0/141922430 >> v2:172.21.15.188:3300/0 conn(0x7fdc1815f4e0 msgr2=0x7fdc180cd7b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-26T07:18:20.249 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:20.241+0000 7fdc177fe640 1 --2- 172.21.15.143:0/141922430 >> v2:172.21.15.188:3300/0 conn(0x7fdc1815f4e0 0x7fdc180cd7b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:18:20.249 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:20.241+0000 7fdc177fe640 1 -- 172.21.15.143:0/141922430 --> v1:172.21.15.143:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdc1806cdc0 con 0x7fdc1813e290 2023-11-26T07:18:20.249 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:20.241+0000 7fdc177fe640 1 -- 172.21.15.143:0/141922430 <== mon.0 v1:172.21.15.143:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2137882523 0 0) 0x7fdc0c001680 con 0x7fdc1813e290 2023-11-26T07:18:20.249 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:20.241+0000 7fdc1d016640 1 --2- 172.21.15.143:0/141922430 >> v2:172.21.15.188:3300/0 conn(0x7fdc1815f4e0 0x7fdc180cd7b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-26T07:18:20.249 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:20.241+0000 7fdc177fe640 1 -- 172.21.15.143:0/141922430 >> v1:172.21.15.143:6789/0 conn(0x7fdc1813e290 legacy=0x7fdc180cf280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:18:20.249 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:20.241+0000 7fdc177fe640 1 --2- 172.21.15.143:0/141922430 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fdc08003980 0x7fdc08003da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:18:20.249 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:20.241+0000 7fdc177fe640 1 --2- 172.21.15.143:0/141922430 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fdc180cf990 0x7fdc080047d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:18:20.249 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:20.241+0000 7fdc177fe640 1 --2- 172.21.15.143:0/141922430 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fdc08004d10 0x7fdc08005190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:18:20.249 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:20.241+0000 7fdc177fe640 1 -- 172.21.15.143:0/141922430 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdc1806c800 con 0x7fdc08003980 2023-11-26T07:18:20.250 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:20.241+0000 7fdc177fe640 1 -- 172.21.15.143:0/141922430 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdc1806cae0 con 0x7fdc08004d10 2023-11-26T07:18:20.250 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:20.241+0000 7fdc177fe640 1 -- 172.21.15.143:0/141922430 --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdc0800b990 con 0x7fdc180cf990 2023-11-26T07:18:20.251 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:20.241+0000 7fdc1c815640 1 --2- 172.21.15.143:0/141922430 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fdc08003980 0x7fdc08003da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:18:20.251 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:20.241+0000 7fdc1d016640 1 --2- 172.21.15.143:0/141922430 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fdc08004d10 0x7fdc08005190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:18:20.251 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:20.241+0000 7fdc17fff640 1 --2- 172.21.15.143:0/141922430 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fdc180cf990 0x7fdc080047d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:18:20.251 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:20.241+0000 7fdc1c815640 1 -- 172.21.15.143:0/141922430 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fdc180cf990 msgr2=0x7fdc080047d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-26T07:18:20.251 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:20.241+0000 7fdc1c815640 1 --2- 172.21.15.143:0/141922430 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fdc180cf990 0x7fdc080047d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:18:20.251 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:20.241+0000 7fdc1c815640 1 -- 172.21.15.143:0/141922430 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fdc08004d10 msgr2=0x7fdc08005190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-26T07:18:20.252 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:20.241+0000 7fdc1c815640 1 --2- 172.21.15.143:0/141922430 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fdc08004d10 0x7fdc08005190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:18:20.252 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:20.241+0000 7fdc1c815640 1 -- 172.21.15.143:0/141922430 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdc08003530 con 0x7fdc08003980 2023-11-26T07:18:20.252 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:20.241+0000 7fdc1d016640 1 --2- 172.21.15.143:0/141922430 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fdc08004d10 0x7fdc08005190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-26T07:18:20.252 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:20.241+0000 7fdc17fff640 1 --2- 172.21.15.143:0/141922430 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fdc180cf990 0x7fdc080047d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-26T07:18:20.252 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:20.245+0000 7fdc1c815640 1 --2- 172.21.15.143:0/141922430 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fdc08003980 0x7fdc08003da0 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7fdc0c00c270 tx=0x7fdc0c01cc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=58fe98d481ce4ccf server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:18:20.252 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:20.245+0000 7fdc177fe640 1 -- 172.21.15.143:0/141922430 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdc0c0058e0 con 0x7fdc08003980 2023-11-26T07:18:20.252 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:20.245+0000 7fdc177fe640 1 -- 172.21.15.143:0/141922430 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdc0c005a40 con 0x7fdc08003980 2023-11-26T07:18:20.252 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:20.245+0000 7fdc177fe640 1 -- 172.21.15.143:0/141922430 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdc0c005e10 con 0x7fdc08003980 2023-11-26T07:18:20.253 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:20.245+0000 7fdc1ea79640 1 -- 172.21.15.143:0/141922430 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fdc08003980 msgr2=0x7fdc08003da0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:18:20.253 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:20.245+0000 7fdc1ea79640 1 --2- 172.21.15.143:0/141922430 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fdc08003980 0x7fdc08003da0 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7fdc0c00c270 tx=0x7fdc0c01cc30 comp rx=0 tx=0).stop 2023-11-26T07:18:20.253 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:20.245+0000 7fdc1d016640 1 -- 172.21.15.143:0/141922430 reap_dead start 2023-11-26T07:18:20.253 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:20.245+0000 7fdc1ea79640 1 -- 172.21.15.143:0/141922430 shutdown_connections 2023-11-26T07:18:20.253 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:20.245+0000 7fdc1ea79640 1 -- 172.21.15.143:0/141922430 >> 172.21.15.143:0/141922430 conn(0x7fdc1806a890 msgr2=0x7fdc1806aca0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:18:20.253 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:20.245+0000 7fdc1ea79640 1 -- 172.21.15.143:0/141922430 shutdown_connections 2023-11-26T07:18:20.253 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:20.245+0000 7fdc1ea79640 1 -- 172.21.15.143:0/141922430 wait complete. 2023-11-26T07:18:20.254 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:20.249+0000 7fdc1ea79640 1 Processor -- start 2023-11-26T07:18:20.254 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:20.249+0000 7fdc1ea79640 1 -- start start 2023-11-26T07:18:20.255 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:20.249+0000 7fdc1ea79640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fdc08003980 0x7fdc18135a90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:18:20.255 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:20.249+0000 7fdc1ea79640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fdc18161190 0x7fdc18161610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:18:20.255 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:20.249+0000 7fdc1ea79640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fdc181f4170 0x7fdc181f45f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:18:20.255 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:20.249+0000 7fdc1ea79640 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdc18161b50 con 0x7fdc18161190 2023-11-26T07:18:20.255 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:20.249+0000 7fdc1ea79640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdc1815d050 con 0x7fdc08003980 2023-11-26T07:18:20.255 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:20.249+0000 7fdc1ea79640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdc1815e820 con 0x7fdc181f4170 2023-11-26T07:18:20.255 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:20.249+0000 7fdc17fff640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fdc18161190 0x7fdc18161610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:18:20.255 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:20.249+0000 7fdc17fff640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fdc18161190 0x7fdc18161610 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:43982/0 (socket says 172.21.15.143:43982) 2023-11-26T07:18:20.255 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:20.249+0000 7fdc17fff640 1 -- 172.21.15.143:0/1634919136 learned_addr learned my addr 172.21.15.143:0/1634919136 (peer_addr_for_me v2:172.21.15.143:0/0) 2023-11-26T07:18:20.257 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:20.249+0000 7fdc1c815640 1 --2- 172.21.15.143:0/1634919136 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fdc08003980 0x7fdc18135a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:18:20.257 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:20.249+0000 7fdc1d016640 1 --2- 172.21.15.143:0/1634919136 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fdc181f4170 0x7fdc181f45f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:18:20.257 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:20.249+0000 7fdc17fff640 1 -- 172.21.15.143:0/1634919136 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fdc181f4170 msgr2=0x7fdc181f45f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:18:20.257 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:20.249+0000 7fdc17fff640 1 --2- 172.21.15.143:0/1634919136 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fdc181f4170 0x7fdc181f45f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:18:20.257 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:20.249+0000 7fdc17fff640 1 -- 172.21.15.143:0/1634919136 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fdc08003980 msgr2=0x7fdc18135a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:18:20.257 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:20.249+0000 7fdc17fff640 1 --2- 172.21.15.143:0/1634919136 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fdc08003980 0x7fdc18135a90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:18:20.257 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:20.249+0000 7fdc17fff640 1 -- 172.21.15.143:0/1634919136 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdc1815b7a0 con 0x7fdc18161190 2023-11-26T07:18:20.257 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:20.249+0000 7fdc17fff640 1 --2- 172.21.15.143:0/1634919136 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fdc18161190 0x7fdc18161610 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7fdc04009fd0 tx=0x7fdc04011b30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:18:20.257 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:20.249+0000 7fdc157fa640 1 -- 172.21.15.143:0/1634919136 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdc04004220 con 0x7fdc18161190 2023-11-26T07:18:20.258 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:20.249+0000 7fdc157fa640 1 -- 172.21.15.143:0/1634919136 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdc04004380 con 0x7fdc18161190 2023-11-26T07:18:20.258 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:20.249+0000 7fdc1ea79640 1 -- 172.21.15.143:0/1634919136 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdc1815c760 con 0x7fdc18161190 2023-11-26T07:18:20.258 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:20.249+0000 7fdc157fa640 1 -- 172.21.15.143:0/1634919136 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdc0400f980 con 0x7fdc18161190 2023-11-26T07:18:20.258 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:20.249+0000 7fdc1ea79640 1 -- 172.21.15.143:0/1634919136 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdc1815cce0 con 0x7fdc18161190 2023-11-26T07:18:20.258 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:20.253+0000 7fdc1ea79640 1 -- 172.21.15.143:0/1634919136 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fdc18067680 con 0x7fdc18161190 2023-11-26T07:18:20.259 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:20.253+0000 7fdc157fa640 1 -- 172.21.15.143:0/1634919136 <== mon.0 v2:172.21.15.143:3300/0 4 ==== mgrmap(e 5) v1 ==== 85852+0+0 (secure 0 0 0) 0x7fdc0400fae0 con 0x7fdc18161190 2023-11-26T07:18:20.259 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:20.253+0000 7fdc157fa640 1 --2- 172.21.15.143:0/1634919136 >> [v2:172.21.15.143:6832/2150557544,v1:172.21.15.143:6833/2150557544] conn(0x7fdbfc066470 0x7fdbfc068930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:18:20.259 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:20.253+0000 7fdc157fa640 1 -- 172.21.15.143:0/1634919136 <== mon.0 v2:172.21.15.143:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fdc04084490 con 0x7fdc18161190 2023-11-26T07:18:20.259 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:20.253+0000 7fdc1c815640 1 --2- 172.21.15.143:0/1634919136 >> [v2:172.21.15.143:6832/2150557544,v1:172.21.15.143:6833/2150557544] conn(0x7fdbfc066470 0x7fdbfc068930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:18:20.259 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:20.253+0000 7fdc1c815640 1 --2- 172.21.15.143:0/1634919136 >> [v2:172.21.15.143:6832/2150557544,v1:172.21.15.143:6833/2150557544] conn(0x7fdbfc066470 0x7fdbfc068930 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7fdc0c0aa000 tx=0x7fdc0c001a80 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:18:20.262 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:20.257+0000 7fdc157fa640 1 -- 172.21.15.143:0/1634919136 <== mon.0 v2:172.21.15.143:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172649 (secure 0 0 0) 0x7fdc04051900 con 0x7fdc18161190 2023-11-26T07:18:20.412 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:20.405+0000 7fdc1ea79640 1 -- 172.21.15.143:0/1634919136 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7fdc1815d190 con 0x7fdc18161190 2023-11-26T07:18:20.412 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:20.405+0000 7fdc157fa640 1 -- 172.21.15.143:0/1634919136 <== mon.0 v2:172.21.15.143:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v20) v1 ==== 100+0+11 (secure 0 0 0) 0x7fdc04022020 con 0x7fdc18161190 2023-11-26T07:18:20.412 INFO:teuthology.orchestra.run.smithi143.stdout:pg_num: 64 2023-11-26T07:18:20.414 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:20.409+0000 7fdc1ea79640 1 -- 172.21.15.143:0/1634919136 >> [v2:172.21.15.143:6832/2150557544,v1:172.21.15.143:6833/2150557544] conn(0x7fdbfc066470 msgr2=0x7fdbfc068930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:18:20.414 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:20.409+0000 7fdc1ea79640 1 --2- 172.21.15.143:0/1634919136 >> [v2:172.21.15.143:6832/2150557544,v1:172.21.15.143:6833/2150557544] conn(0x7fdbfc066470 0x7fdbfc068930 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7fdc0c0aa000 tx=0x7fdc0c001a80 comp rx=0 tx=0).stop 2023-11-26T07:18:20.414 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:20.409+0000 7fdc1ea79640 1 -- 172.21.15.143:0/1634919136 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fdc18161190 msgr2=0x7fdc18161610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:18:20.415 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:20.409+0000 7fdc1ea79640 1 --2- 172.21.15.143:0/1634919136 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fdc18161190 0x7fdc18161610 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7fdc04009fd0 tx=0x7fdc04011b30 comp rx=0 tx=0).stop 2023-11-26T07:18:20.415 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:20.409+0000 7fdc1ea79640 1 -- 172.21.15.143:0/1634919136 shutdown_connections 2023-11-26T07:18:20.415 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:20.409+0000 7fdc1ea79640 1 --2- 172.21.15.143:0/1634919136 >> [v2:172.21.15.143:6832/2150557544,v1:172.21.15.143:6833/2150557544] conn(0x7fdbfc066470 0x7fdbfc068930 unknown :-1 s=CLOSED pgs=132 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:18:20.415 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:20.409+0000 7fdc1ea79640 1 --2- 172.21.15.143:0/1634919136 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fdc181f4170 0x7fdc181f45f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:18:20.415 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:20.409+0000 7fdc1ea79640 1 --2- 172.21.15.143:0/1634919136 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fdc18161190 0x7fdc18161610 unknown :-1 s=CLOSED pgs=201 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:18:20.415 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:20.409+0000 7fdc1ea79640 1 --2- 172.21.15.143:0/1634919136 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fdc08003980 0x7fdc18135a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:18:20.415 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:20.409+0000 7fdc1ea79640 1 -- 172.21.15.143:0/1634919136 >> 172.21.15.143:0/1634919136 conn(0x7fdc1806a890 msgr2=0x7fdc1813b890 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:18:20.415 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:20.409+0000 7fdc1ea79640 1 -- 172.21.15.143:0/1634919136 shutdown_connections 2023-11-26T07:18:20.415 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:20.409+0000 7fdc1ea79640 1 -- 172.21.15.143:0/1634919136 wait complete. 2023-11-26T07:18:20.434 DEBUG:teuthology.orchestra.run.smithi143:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-11-26T07:18:20.573 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:20.569+0000 7f8b39372640 1 Processor -- start 2023-11-26T07:18:20.573 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:20.569+0000 7f8b39372640 1 -- start start 2023-11-26T07:18:20.574 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:20.569+0000 7f8b39372640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f8b340b3610 0x7f8b340cfa40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:18:20.575 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:20.569+0000 7f8b39372640 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7f8b340b45d0 0x7f8b340cff80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:18:20.575 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:20.569+0000 7f8b39372640 1 -- --> v1:172.21.15.143:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8b340cefa0 con 0x7f8b340b4080 2023-11-26T07:18:20.575 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:20.569+0000 7f8b39372640 1 -- --> v2:172.21.15.188:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8b340c9fa0 con 0x7f8b340b45d0 2023-11-26T07:18:20.575 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:20.569+0000 7f8b39372640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8b340b3ef0 con 0x7f8b340b3610 2023-11-26T07:18:20.575 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:20.569+0000 7f8b337fe640 1 --1- >> v1:172.21.15.143:6789/0 conn(0x7f8b340b4080 0x7f8b340cddd0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.143:6789/0 says I am v1:172.21.15.143:43922/0 (socket says 172.21.15.143:43922) 2023-11-26T07:18:20.575 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:20.569+0000 7f8b337fe640 1 -- 172.21.15.143:0/1508828701 learned_addr learned my addr 172.21.15.143:0/1508828701 (peer_addr_for_me v1:172.21.15.143:0/0) 2023-11-26T07:18:20.577 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:20.569+0000 7f8b32ffd640 1 --2- 172.21.15.143:0/1508828701 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f8b340b3610 0x7f8b340cfa40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:18:20.577 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:20.569+0000 7f8b327fc640 1 --2- 172.21.15.143:0/1508828701 >> v2:172.21.15.188:3300/0 conn(0x7f8b340b45d0 0x7f8b340cff80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:18:20.577 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:20.569+0000 7f8b31ffb640 1 -- 172.21.15.143:0/1508828701 <== mon.0 v1:172.21.15.143:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2551171329 0 0) 0x7f8b340cefa0 con 0x7f8b340b4080 2023-11-26T07:18:20.578 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:20.569+0000 7f8b31ffb640 1 -- 172.21.15.143:0/1508828701 --> v1:172.21.15.143:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f8b1c003530 con 0x7f8b340b4080 2023-11-26T07:18:20.578 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:20.569+0000 7f8b31ffb640 1 -- 172.21.15.143:0/1508828701 <== mon.0 v1:172.21.15.143:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3990042327 0 0) 0x7f8b1c003530 con 0x7f8b340b4080 2023-11-26T07:18:20.578 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:20.569+0000 7f8b31ffb640 1 -- 172.21.15.143:0/1508828701 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f8b340b3610 msgr2=0x7f8b340cfa40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-26T07:18:20.578 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:20.569+0000 7f8b31ffb640 1 --2- 172.21.15.143:0/1508828701 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f8b340b3610 0x7f8b340cfa40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:18:20.578 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:20.569+0000 7f8b31ffb640 1 -- 172.21.15.143:0/1508828701 >> v2:172.21.15.188:3300/0 conn(0x7f8b340b45d0 msgr2=0x7f8b340cff80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-26T07:18:20.578 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:20.569+0000 7f8b31ffb640 1 --2- 172.21.15.143:0/1508828701 >> v2:172.21.15.188:3300/0 conn(0x7f8b340b45d0 0x7f8b340cff80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:18:20.578 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:20.569+0000 7f8b31ffb640 1 -- 172.21.15.143:0/1508828701 --> v1:172.21.15.143:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8b340d0f90 con 0x7f8b340b4080 2023-11-26T07:18:20.578 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:20.569+0000 7f8b31ffb640 1 -- 172.21.15.143:0/1508828701 <== mon.0 v1:172.21.15.143:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2137882523 0 0) 0x7f8b2c001a00 con 0x7f8b340b4080 2023-11-26T07:18:20.578 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:20.569+0000 7f8b327fc640 1 --2- 172.21.15.143:0/1508828701 >> v2:172.21.15.188:3300/0 conn(0x7f8b340b45d0 0x7f8b340cff80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-26T07:18:20.578 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:20.569+0000 7f8b31ffb640 1 -- 172.21.15.143:0/1508828701 >> v1:172.21.15.143:6789/0 conn(0x7f8b340b4080 legacy=0x7f8b340cddd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:18:20.579 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:20.569+0000 7f8b31ffb640 1 --2- 172.21.15.143:0/1508828701 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f8b1c003980 0x7f8b1c003da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:18:20.579 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:20.569+0000 7f8b31ffb640 1 --2- 172.21.15.143:0/1508828701 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f8b340b3610 0x7f8b1c0047d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:18:20.579 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:20.569+0000 7f8b31ffb640 1 --2- 172.21.15.143:0/1508828701 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f8b1c004d10 0x7f8b1c005190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:18:20.579 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:20.569+0000 7f8b31ffb640 1 -- 172.21.15.143:0/1508828701 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8b340c9fa0 con 0x7f8b1c003980 2023-11-26T07:18:20.579 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:20.569+0000 7f8b31ffb640 1 -- 172.21.15.143:0/1508828701 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8b340b3ef0 con 0x7f8b1c004d10 2023-11-26T07:18:20.579 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:20.569+0000 7f8b31ffb640 1 -- 172.21.15.143:0/1508828701 --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8b1c00b990 con 0x7f8b340b3610 2023-11-26T07:18:20.579 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:20.569+0000 7f8b32ffd640 1 --2- 172.21.15.143:0/1508828701 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f8b1c003980 0x7f8b1c003da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:18:20.579 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:20.569+0000 7f8b337fe640 1 --2- 172.21.15.143:0/1508828701 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f8b1c004d10 0x7f8b1c005190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:18:20.579 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:20.569+0000 7f8b327fc640 1 --2- 172.21.15.143:0/1508828701 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f8b340b3610 0x7f8b1c0047d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:18:20.579 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:20.569+0000 7f8b32ffd640 1 -- 172.21.15.143:0/1508828701 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f8b340b3610 msgr2=0x7f8b1c0047d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-26T07:18:20.579 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:20.569+0000 7f8b32ffd640 1 --2- 172.21.15.143:0/1508828701 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f8b340b3610 0x7f8b1c0047d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:18:20.580 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:20.569+0000 7f8b32ffd640 1 -- 172.21.15.143:0/1508828701 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f8b1c004d10 msgr2=0x7f8b1c005190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-26T07:18:20.580 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:20.569+0000 7f8b32ffd640 1 --2- 172.21.15.143:0/1508828701 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f8b1c004d10 0x7f8b1c005190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:18:20.580 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:20.569+0000 7f8b32ffd640 1 -- 172.21.15.143:0/1508828701 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8b1c003530 con 0x7f8b1c003980 2023-11-26T07:18:20.581 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:20.573+0000 7f8b32ffd640 1 --2- 172.21.15.143:0/1508828701 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f8b1c003980 0x7f8b1c003da0 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7f8b24009ae0 tx=0x7f8b2401ca50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e7f99fe04afb96d5 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:18:20.581 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:20.573+0000 7f8b31ffb640 1 -- 172.21.15.143:0/1508828701 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8b240a5530 con 0x7f8b1c003980 2023-11-26T07:18:20.581 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:20.573+0000 7f8b31ffb640 1 -- 172.21.15.143:0/1508828701 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8b240a5690 con 0x7f8b1c003980 2023-11-26T07:18:20.581 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:20.573+0000 7f8b31ffb640 1 -- 172.21.15.143:0/1508828701 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8b240a5a40 con 0x7f8b1c003980 2023-11-26T07:18:20.581 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:20.573+0000 7f8b39372640 1 -- 172.21.15.143:0/1508828701 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f8b1c003980 msgr2=0x7f8b1c003da0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:18:20.581 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:20.573+0000 7f8b39372640 1 --2- 172.21.15.143:0/1508828701 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f8b1c003980 0x7f8b1c003da0 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7f8b24009ae0 tx=0x7f8b2401ca50 comp rx=0 tx=0).stop 2023-11-26T07:18:20.581 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:20.573+0000 7f8b337fe640 1 -- 172.21.15.143:0/1508828701 reap_dead start 2023-11-26T07:18:20.581 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:20.573+0000 7f8b39372640 1 -- 172.21.15.143:0/1508828701 shutdown_connections 2023-11-26T07:18:20.581 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:20.573+0000 7f8b39372640 1 -- 172.21.15.143:0/1508828701 >> 172.21.15.143:0/1508828701 conn(0x7f8b340d3030 msgr2=0x7f8b340d3440 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:18:20.582 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:20.573+0000 7f8b39372640 1 -- 172.21.15.143:0/1508828701 shutdown_connections 2023-11-26T07:18:20.582 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:20.573+0000 7f8b39372640 1 -- 172.21.15.143:0/1508828701 wait complete. 2023-11-26T07:18:20.582 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:20.577+0000 7f8b39372640 1 Processor -- start 2023-11-26T07:18:20.582 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:20.577+0000 7f8b39372640 1 -- start start 2023-11-26T07:18:20.587 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:20.577+0000 7f8b39372640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f8b1c003980 0x7f8b340b76b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:18:20.587 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:20.577+0000 7f8b39372640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f8b341564a0 0x7f8b340ca360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:18:20.587 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:20.577+0000 7f8b39372640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f8b3414cdb0 0x7f8b3414d230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:18:20.587 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:20.577+0000 7f8b39372640 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8b340b85c0 con 0x7f8b341564a0 2023-11-26T07:18:20.587 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:20.577+0000 7f8b39372640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8b340b7d20 con 0x7f8b1c003980 2023-11-26T07:18:20.587 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:20.577+0000 7f8b39372640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8b340b8000 con 0x7f8b3414cdb0 2023-11-26T07:18:20.587 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:20.577+0000 7f8b327fc640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f8b341564a0 0x7f8b340ca360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:18:20.588 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:20.577+0000 7f8b327fc640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f8b341564a0 0x7f8b340ca360 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:44010/0 (socket says 172.21.15.143:44010) 2023-11-26T07:18:20.588 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:20.577+0000 7f8b327fc640 1 -- 172.21.15.143:0/2494646231 learned_addr learned my addr 172.21.15.143:0/2494646231 (peer_addr_for_me v2:172.21.15.143:0/0) 2023-11-26T07:18:20.588 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:20.577+0000 7f8b337fe640 1 --2- 172.21.15.143:0/2494646231 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f8b3414cdb0 0x7f8b3414d230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:18:20.588 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:20.577+0000 7f8b32ffd640 1 --2- 172.21.15.143:0/2494646231 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f8b1c003980 0x7f8b340b76b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:18:20.588 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:20.577+0000 7f8b327fc640 1 -- 172.21.15.143:0/2494646231 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f8b3414cdb0 msgr2=0x7f8b3414d230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:18:20.588 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:20.577+0000 7f8b327fc640 1 --2- 172.21.15.143:0/2494646231 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f8b3414cdb0 0x7f8b3414d230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:18:20.588 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:20.577+0000 7f8b327fc640 1 -- 172.21.15.143:0/2494646231 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f8b1c003980 msgr2=0x7f8b340b76b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:18:20.588 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:20.577+0000 7f8b327fc640 1 --2- 172.21.15.143:0/2494646231 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f8b1c003980 0x7f8b340b76b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:18:20.588 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:20.577+0000 7f8b327fc640 1 -- 172.21.15.143:0/2494646231 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8b340b82e0 con 0x7f8b341564a0 2023-11-26T07:18:20.588 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:20.577+0000 7f8b327fc640 1 --2- 172.21.15.143:0/2494646231 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f8b341564a0 0x7f8b340ca360 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7f8b28011cd0 tx=0x7f8b2800f610 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:18:20.588 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:20.577+0000 7f8b1b7fe640 1 -- 172.21.15.143:0/2494646231 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8b2800fd20 con 0x7f8b341564a0 2023-11-26T07:18:20.589 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:20.577+0000 7f8b1b7fe640 1 -- 172.21.15.143:0/2494646231 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8b2800fe80 con 0x7f8b341564a0 2023-11-26T07:18:20.589 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:20.577+0000 7f8b1b7fe640 1 -- 172.21.15.143:0/2494646231 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8b28004250 con 0x7f8b341564a0 2023-11-26T07:18:20.589 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:20.577+0000 7f8b39372640 1 -- 172.21.15.143:0/2494646231 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8b340b09a0 con 0x7f8b341564a0 2023-11-26T07:18:20.589 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:20.581+0000 7f8b39372640 1 -- 172.21.15.143:0/2494646231 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8b340caec0 con 0x7f8b341564a0 2023-11-26T07:18:20.589 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:20.581+0000 7f8b39372640 1 -- 172.21.15.143:0/2494646231 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8b34067500 con 0x7f8b341564a0 2023-11-26T07:18:20.589 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:20.581+0000 7f8b1b7fe640 1 -- 172.21.15.143:0/2494646231 <== mon.0 v2:172.21.15.143:3300/0 4 ==== mgrmap(e 5) v1 ==== 85852+0+0 (secure 0 0 0) 0x7f8b280043b0 con 0x7f8b341564a0 2023-11-26T07:18:20.590 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:20.581+0000 7f8b1b7fe640 1 --2- 172.21.15.143:0/2494646231 >> [v2:172.21.15.143:6832/2150557544,v1:172.21.15.143:6833/2150557544] conn(0x7f8b20066470 0x7f8b20068930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:18:20.590 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:20.581+0000 7f8b1b7fe640 1 -- 172.21.15.143:0/2494646231 <== mon.0 v2:172.21.15.143:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f8b280849c0 con 0x7f8b341564a0 2023-11-26T07:18:20.590 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:20.581+0000 7f8b32ffd640 1 --2- 172.21.15.143:0/2494646231 >> [v2:172.21.15.143:6832/2150557544,v1:172.21.15.143:6833/2150557544] conn(0x7f8b20066470 0x7f8b20068930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:18:20.590 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:20.581+0000 7f8b32ffd640 1 --2- 172.21.15.143:0/2494646231 >> [v2:172.21.15.143:6832/2150557544,v1:172.21.15.143:6833/2150557544] conn(0x7f8b20066470 0x7f8b20068930 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7f8b240af000 tx=0x7f8b240046d0 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:18:20.591 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:20.585+0000 7f8b1b7fe640 1 -- 172.21.15.143:0/2494646231 <== mon.0 v2:172.21.15.143:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172649 (secure 0 0 0) 0x7f8b28051da0 con 0x7f8b341564a0 2023-11-26T07:18:20.750 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:20.745+0000 7f8b39372640 1 -- 172.21.15.143:0/2494646231 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f8b340cf4f0 con 0x7f8b341564a0 2023-11-26T07:18:20.751 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:20.745+0000 7f8b1b7fe640 1 -- 172.21.15.143:0/2494646231 <== mon.0 v2:172.21.15.143:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v20) v1 ==== 74+0+16541 (secure 0 0 0) 0x7f8b28022070 con 0x7f8b341564a0 2023-11-26T07:18:20.751 INFO:teuthology.orchestra.run.smithi143.stdout: 2023-11-26T07:18:20.751 INFO:teuthology.orchestra.run.smithi143.stdout:{"epoch":20,"fsid":"f5033060-2469-4a07-95fe-2b76f5404bc1","created":"2023-11-26T07:17:28.613663+0000","modified":"2023-11-26T07:17:55.505435+0000","last_up_change":"2023-11-26T07:17:41.157813+0000","last_in_change":"2023-11-26T07:17:31.181658+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-11-26T07:17:41.758597+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2023-11-26T07:17:51.878332+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2023-11-26T07:17:52.871396+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"ebbee2c6-023e-44b9-9f1d-6f7bfd309730","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6817","nonce":639807369},{"type":"v1","addr":"172.21.15.143:6819","nonce":639807369}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6821","nonce":639807369},{"type":"v1","addr":"172.21.15.143:6823","nonce":639807369}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6830","nonce":639807369},{"type":"v1","addr":"172.21.15.143:6831","nonce":639807369}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6825","nonce":639807369},{"type":"v1","addr":"172.21.15.143:6828","nonce":639807369}]},"public_addr":"172.21.15.143:6819/639807369","cluster_addr":"172.21.15.143:6823/639807369","heartbeat_back_addr":"172.21.15.143:6831/639807369","heartbeat_front_addr":"172.21.15.143:6828/639807369","state":["exists","up"]},{"osd":1,"uuid":"fec4b8be-7122-49e9-9a19-5850a0700001","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6816","nonce":1144942530},{"type":"v1","addr":"172.21.15.143:6818","nonce":1144942530}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6820","nonce":1144942530},{"type":"v1","addr":"172.21.15.143:6822","nonce":1144942530}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6827","nonce":1144942530},{"type":"v1","addr":"172.21.15.143:6829","nonce":1144942530}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6824","nonce":1144942530},{"type":"v1","addr":"172.21.15.143:6826","nonce":1144942530}]},"public_addr":"172.21.15.143:6818/1144942530","cluster_addr":"172.21.15.143:6822/1144942530","heartbeat_back_addr":"172.21.15.143:6829/1144942530","heartbeat_front_addr":"172.21.15.143:6826/1144942530","state":["exists","up"]},{"osd":2,"uuid":"2888af5b-f933-46ca-8f95-6dd494ff43f6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6808","nonce":3909095186},{"type":"v1","addr":"172.21.15.143:6809","nonce":3909095186}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6810","nonce":3909095186},{"type":"v1","addr":"172.21.15.143:6811","nonce":3909095186}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6814","nonce":3909095186},{"type":"v1","addr":"172.21.15.143:6815","nonce":3909095186}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6812","nonce":3909095186},{"type":"v1","addr":"172.21.15.143:6813","nonce":3909095186}]},"public_addr":"172.21.15.143:6809/3909095186","cluster_addr":"172.21.15.143:6811/3909095186","heartbeat_back_addr":"172.21.15.143:6815/3909095186","heartbeat_front_addr":"172.21.15.143:6813/3909095186","state":["exists","up"]},{"osd":3,"uuid":"d1f52435-7bc1-4081-9550-9cd1ba55f549","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6800","nonce":3854210991},{"type":"v1","addr":"172.21.15.143:6801","nonce":3854210991}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6802","nonce":3854210991},{"type":"v1","addr":"172.21.15.143:6803","nonce":3854210991}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6806","nonce":3854210991},{"type":"v1","addr":"172.21.15.143:6807","nonce":3854210991}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6804","nonce":3854210991},{"type":"v1","addr":"172.21.15.143:6805","nonce":3854210991}]},"public_addr":"172.21.15.143:6801/3854210991","cluster_addr":"172.21.15.143:6803/3854210991","heartbeat_back_addr":"172.21.15.143:6807/3854210991","heartbeat_front_addr":"172.21.15.143:6805/3854210991","state":["exists","up"]},{"osd":4,"uuid":"e2ae8b6e-6e25-49b2-bcbf-aacce68f06a3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6800","nonce":3924606902},{"type":"v1","addr":"172.21.15.188:6801","nonce":3924606902}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6802","nonce":3924606902},{"type":"v1","addr":"172.21.15.188:6803","nonce":3924606902}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6806","nonce":3924606902},{"type":"v1","addr":"172.21.15.188:6807","nonce":3924606902}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6804","nonce":3924606902},{"type":"v1","addr":"172.21.15.188:6805","nonce":3924606902}]},"public_addr":"172.21.15.188:6801/3924606902","cluster_addr":"172.21.15.188:6803/3924606902","heartbeat_back_addr":"172.21.15.188:6807/3924606902","heartbeat_front_addr":"172.21.15.188:6805/3924606902","state":["exists","up"]},{"osd":5,"uuid":"3cec8a32-7698-4c36-8e89-e4ba11934f33","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6813","nonce":186841544},{"type":"v1","addr":"172.21.15.188:6815","nonce":186841544}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6817","nonce":186841544},{"type":"v1","addr":"172.21.15.188:6818","nonce":186841544}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6822","nonce":186841544},{"type":"v1","addr":"172.21.15.188:6823","nonce":186841544}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6820","nonce":186841544},{"type":"v1","addr":"172.21.15.188:6821","nonce":186841544}]},"public_addr":"172.21.15.188:6815/186841544","cluster_addr":"172.21.15.188:6818/186841544","heartbeat_back_addr":"172.21.15.188:6823/186841544","heartbeat_front_addr":"172.21.15.188:6821/186841544","state":["exists","up"]},{"osd":6,"uuid":"58a65d5d-e090-473a-81b4-ad38e85029f9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6808","nonce":2153499405},{"type":"v1","addr":"172.21.15.188:6809","nonce":2153499405}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6810","nonce":2153499405},{"type":"v1","addr":"172.21.15.188:6811","nonce":2153499405}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6816","nonce":2153499405},{"type":"v1","addr":"172.21.15.188:6819","nonce":2153499405}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6812","nonce":2153499405},{"type":"v1","addr":"172.21.15.188:6814","nonce":2153499405}]},"public_addr":"172.21.15.188:6809/2153499405","cluster_addr":"172.21.15.188:6811/2153499405","heartbeat_back_addr":"172.21.15.188:6819/2153499405","heartbeat_front_addr":"172.21.15.188:6814/2153499405","state":["exists","up"]},{"osd":7,"uuid":"ea5b7f70-3554-4cf8-ac30-f7c59b80678d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6824","nonce":2722096729},{"type":"v1","addr":"172.21.15.188:6825","nonce":2722096729}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6826","nonce":2722096729},{"type":"v1","addr":"172.21.15.188:6827","nonce":2722096729}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6830","nonce":2722096729},{"type":"v1","addr":"172.21.15.188:6831","nonce":2722096729}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6828","nonce":2722096729},{"type":"v1","addr":"172.21.15.188:6829","nonce":2722096729}]},"public_addr":"172.21.15.188:6825/2722096729","cluster_addr":"172.21.15.188:6827/2722096729","heartbeat_back_addr":"172.21.15.188:6831/2722096729","heartbeat_front_addr":"172.21.15.188:6829/2722096729","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-26T07:17:36.995032+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-26T07:17:36.932266+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-26T07:17:36.829304+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-26T07:17:36.804297+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-26T07:17:36.667892+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-26T07:17:36.964868+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-26T07:17:36.827332+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-26T07:17:36.893179+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-11-26T07:18:20.753 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:20.749+0000 7f8b39372640 1 -- 172.21.15.143:0/2494646231 >> [v2:172.21.15.143:6832/2150557544,v1:172.21.15.143:6833/2150557544] conn(0x7f8b20066470 msgr2=0x7f8b20068930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:18:20.753 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:20.749+0000 7f8b39372640 1 --2- 172.21.15.143:0/2494646231 >> [v2:172.21.15.143:6832/2150557544,v1:172.21.15.143:6833/2150557544] conn(0x7f8b20066470 0x7f8b20068930 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7f8b240af000 tx=0x7f8b240046d0 comp rx=0 tx=0).stop 2023-11-26T07:18:20.754 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:20.749+0000 7f8b39372640 1 -- 172.21.15.143:0/2494646231 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f8b341564a0 msgr2=0x7f8b340ca360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:18:20.754 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:20.749+0000 7f8b39372640 1 --2- 172.21.15.143:0/2494646231 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f8b341564a0 0x7f8b340ca360 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7f8b28011cd0 tx=0x7f8b2800f610 comp rx=0 tx=0).stop 2023-11-26T07:18:20.754 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:20.749+0000 7f8b39372640 1 -- 172.21.15.143:0/2494646231 shutdown_connections 2023-11-26T07:18:20.754 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:20.749+0000 7f8b39372640 1 --2- 172.21.15.143:0/2494646231 >> [v2:172.21.15.143:6832/2150557544,v1:172.21.15.143:6833/2150557544] conn(0x7f8b20066470 0x7f8b20068930 unknown :-1 s=CLOSED pgs=133 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:18:20.754 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:20.749+0000 7f8b39372640 1 --2- 172.21.15.143:0/2494646231 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f8b3414cdb0 0x7f8b3414d230 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:18:20.754 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:20.749+0000 7f8b39372640 1 --2- 172.21.15.143:0/2494646231 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f8b341564a0 0x7f8b340ca360 unknown :-1 s=CLOSED pgs=204 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:18:20.754 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:20.749+0000 7f8b39372640 1 --2- 172.21.15.143:0/2494646231 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f8b1c003980 0x7f8b340b76b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:18:20.754 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:20.749+0000 7f8b39372640 1 -- 172.21.15.143:0/2494646231 >> 172.21.15.143:0/2494646231 conn(0x7f8b340d3030 msgr2=0x7f8b340d1610 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:18:20.754 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:20.749+0000 7f8b39372640 1 -- 172.21.15.143:0/2494646231 shutdown_connections 2023-11-26T07:18:20.754 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:20.749+0000 7f8b39372640 1 -- 172.21.15.143:0/2494646231 wait complete. 2023-11-26T07:18:20.772 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-11-26T07:17:41.758597+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2023-11-26T07:17:51.878332+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2023-11-26T07:17:52.871396+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '20', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2023-11-26T07:18:20.773 DEBUG:teuthology.orchestra.run.smithi143:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-11-26T07:18:20.914 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:20.909+0000 7fd60505a640 1 Processor -- start 2023-11-26T07:18:20.914 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:20.909+0000 7fd60505a640 1 -- start start 2023-11-26T07:18:20.916 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:20.909+0000 7fd60505a640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fd6000ce680 0x7fd600071850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:18:20.916 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:20.909+0000 7fd60505a640 1 -- --> v1:172.21.15.143:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd60006c750 con 0x7fd6001596d0 2023-11-26T07:18:20.916 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:20.909+0000 7fd60505a640 1 -- --> v1:172.21.15.188:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd6000b3550 con 0x7fd6000b2f80 2023-11-26T07:18:20.916 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:20.909+0000 7fd60505a640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd6000cbe00 con 0x7fd6000ce680 2023-11-26T07:18:20.916 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:20.909+0000 7fd5fed9d640 1 --1- >> v1:172.21.15.143:6789/0 conn(0x7fd6001596d0 0x7fd6000b2870 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.143:6789/0 says I am v1:172.21.15.143:43928/0 (socket says 172.21.15.143:43928) 2023-11-26T07:18:20.916 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:20.909+0000 7fd5fed9d640 1 -- 172.21.15.143:0/3174550056 learned_addr learned my addr 172.21.15.143:0/3174550056 (peer_addr_for_me v1:172.21.15.143:0/0) 2023-11-26T07:18:20.916 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:20.909+0000 7fd5ff59e640 1 --2- 172.21.15.143:0/3174550056 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fd6000ce680 0x7fd600071850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:18:20.917 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:20.909+0000 7fd5fe59c640 1 -- 172.21.15.143:0/3174550056 <== mon.0 v1:172.21.15.143:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 821959379 0 0) 0x7fd60006c750 con 0x7fd6001596d0 2023-11-26T07:18:20.917 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:20.909+0000 7fd5fe59c640 1 -- 172.21.15.143:0/3174550056 --> v1:172.21.15.143:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fd5ec003450 con 0x7fd6001596d0 2023-11-26T07:18:20.917 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:20.909+0000 7fd5fe59c640 1 -- 172.21.15.143:0/3174550056 <== mon.1 v1:172.21.15.188:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 580446604 0 0) 0x7fd6000b3550 con 0x7fd6000b2f80 2023-11-26T07:18:20.917 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:20.909+0000 7fd5ff59e640 1 -- 172.21.15.143:0/3174550056 >> v1:172.21.15.188:6789/0 conn(0x7fd6000b2f80 legacy=0x7fd60006cbe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:18:20.917 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:20.909+0000 7fd5ff59e640 1 -- 172.21.15.143:0/3174550056 >> v1:172.21.15.143:6789/0 conn(0x7fd6001596d0 legacy=0x7fd6000b2870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:18:20.917 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:20.909+0000 7fd5ff59e640 1 -- 172.21.15.143:0/3174550056 --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd600071e80 con 0x7fd6000ce680 2023-11-26T07:18:20.919 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:20.913+0000 7fd5ff59e640 1 --2- 172.21.15.143:0/3174550056 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fd6000ce680 0x7fd600071850 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7fd5f8009dc0 tx=0x7fd5f801c970 comp rx=0 tx=0).ready entity=mon.2 client_cookie=6931fdf07d672d2b server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:18:20.919 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:20.913+0000 7fd5fe59c640 1 -- 172.21.15.143:0/3174550056 <== mon.2 v2:172.21.15.188:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd5f80a58a0 con 0x7fd6000ce680 2023-11-26T07:18:20.920 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:20.913+0000 7fd5fe59c640 1 -- 172.21.15.143:0/3174550056 <== mon.2 v2:172.21.15.188:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd5f80a5a00 con 0x7fd6000ce680 2023-11-26T07:18:20.920 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:20.913+0000 7fd5fe59c640 1 -- 172.21.15.143:0/3174550056 <== mon.2 v2:172.21.15.188:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd5f80a5d90 con 0x7fd6000ce680 2023-11-26T07:18:20.920 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:20.913+0000 7fd60505a640 1 -- 172.21.15.143:0/3174550056 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fd6000ce680 msgr2=0x7fd600071850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:18:20.920 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:20.913+0000 7fd60505a640 1 --2- 172.21.15.143:0/3174550056 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fd6000ce680 0x7fd600071850 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7fd5f8009dc0 tx=0x7fd5f801c970 comp rx=0 tx=0).stop 2023-11-26T07:18:20.921 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:20.913+0000 7fd60505a640 1 -- 172.21.15.143:0/3174550056 shutdown_connections 2023-11-26T07:18:20.921 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:20.913+0000 7fd60505a640 1 --2- 172.21.15.143:0/3174550056 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fd6000ce680 0x7fd600071850 unknown :-1 s=CLOSED pgs=58 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:18:20.921 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:20.913+0000 7fd60505a640 1 -- 172.21.15.143:0/3174550056 >> 172.21.15.143:0/3174550056 conn(0x7fd60006a890 msgr2=0x7fd60006aca0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:18:20.921 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:20.913+0000 7fd60505a640 1 -- 172.21.15.143:0/3174550056 shutdown_connections 2023-11-26T07:18:20.921 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:20.913+0000 7fd60505a640 1 -- 172.21.15.143:0/3174550056 wait complete. 2023-11-26T07:18:20.922 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:20.917+0000 7fd60505a640 1 Processor -- start 2023-11-26T07:18:20.922 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:20.917+0000 7fd60505a640 1 -- start start 2023-11-26T07:18:20.924 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:20.917+0000 7fd60505a640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fd6000b2f80 0x7fd6000ca2b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:18:20.924 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:20.917+0000 7fd60505a640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fd6000ce680 0x7fd60014f4a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:18:20.924 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:20.917+0000 7fd60505a640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fd6001596d0 0x7fd60013c180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:18:20.924 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:20.917+0000 7fd60505a640 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd60014f010 con 0x7fd6001596d0 2023-11-26T07:18:20.924 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:20.917+0000 7fd60505a640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd60014e400 con 0x7fd6000b2f80 2023-11-26T07:18:20.924 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:20.917+0000 7fd60505a640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd60014e6e0 con 0x7fd6000ce680 2023-11-26T07:18:20.925 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:20.917+0000 7fd5ff59e640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fd6001596d0 0x7fd60013c180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:18:20.925 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:20.917+0000 7fd5ff59e640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fd6001596d0 0x7fd60013c180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:44016/0 (socket says 172.21.15.143:44016) 2023-11-26T07:18:20.925 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:20.917+0000 7fd5ff59e640 1 -- 172.21.15.143:0/795124156 learned_addr learned my addr 172.21.15.143:0/795124156 (peer_addr_for_me v2:172.21.15.143:0/0) 2023-11-26T07:18:20.925 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:20.917+0000 7fd5fed9d640 1 --2- 172.21.15.143:0/795124156 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fd6000b2f80 0x7fd6000ca2b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:18:20.925 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:20.917+0000 7fd5f659c640 1 --2- 172.21.15.143:0/795124156 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fd6000ce680 0x7fd60014f4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:18:20.925 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:20.917+0000 7fd5ff59e640 1 -- 172.21.15.143:0/795124156 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fd6000ce680 msgr2=0x7fd60014f4a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:18:20.925 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:20.917+0000 7fd5ff59e640 1 --2- 172.21.15.143:0/795124156 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fd6000ce680 0x7fd60014f4a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:18:20.925 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:20.917+0000 7fd5ff59e640 1 -- 172.21.15.143:0/795124156 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fd6000b2f80 msgr2=0x7fd6000ca2b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:18:20.925 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:20.917+0000 7fd5ff59e640 1 --2- 172.21.15.143:0/795124156 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fd6000b2f80 0x7fd6000ca2b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:18:20.925 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:20.917+0000 7fd5ff59e640 1 -- 172.21.15.143:0/795124156 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd5f8009a70 con 0x7fd6001596d0 2023-11-26T07:18:20.925 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:20.917+0000 7fd5ff59e640 1 --2- 172.21.15.143:0/795124156 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fd6001596d0 0x7fd60013c180 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7fd5f80b0040 tx=0x7fd5f8019710 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:18:20.926 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:20.917+0000 7fd5f7fff640 1 -- 172.21.15.143:0/795124156 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd5f80be070 con 0x7fd6001596d0 2023-11-26T07:18:20.926 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:20.917+0000 7fd5f7fff640 1 -- 172.21.15.143:0/795124156 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd5f80a5860 con 0x7fd6001596d0 2023-11-26T07:18:20.926 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:20.917+0000 7fd5f7fff640 1 -- 172.21.15.143:0/795124156 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd5f80b9860 con 0x7fd6001596d0 2023-11-26T07:18:20.926 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:20.921+0000 7fd60505a640 1 -- 172.21.15.143:0/795124156 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd60014e9c0 con 0x7fd6001596d0 2023-11-26T07:18:20.926 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:20.921+0000 7fd60505a640 1 -- 172.21.15.143:0/795124156 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd6000ca7f0 con 0x7fd6001596d0 2023-11-26T07:18:20.926 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:20.921+0000 7fd60505a640 1 -- 172.21.15.143:0/795124156 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd60014f180 con 0x7fd6001596d0 2023-11-26T07:18:20.927 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:20.921+0000 7fd5f7fff640 1 -- 172.21.15.143:0/795124156 <== mon.0 v2:172.21.15.143:3300/0 4 ==== mgrmap(e 5) v1 ==== 85852+0+0 (secure 0 0 0) 0x7fd5f80b99c0 con 0x7fd6001596d0 2023-11-26T07:18:20.928 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:20.921+0000 7fd5f7fff640 1 --2- 172.21.15.143:0/795124156 >> [v2:172.21.15.143:6832/2150557544,v1:172.21.15.143:6833/2150557544] conn(0x7fd5e00664a0 0x7fd5e0068960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:18:20.928 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:20.921+0000 7fd5f7fff640 1 -- 172.21.15.143:0/795124156 <== mon.0 v2:172.21.15.143:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fd5f811eb40 con 0x7fd6001596d0 2023-11-26T07:18:20.928 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:20.921+0000 7fd5fed9d640 1 --2- 172.21.15.143:0/795124156 >> [v2:172.21.15.143:6832/2150557544,v1:172.21.15.143:6833/2150557544] conn(0x7fd5e00664a0 0x7fd5e0068960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:18:20.929 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:20.925+0000 7fd5fed9d640 1 --2- 172.21.15.143:0/795124156 >> [v2:172.21.15.143:6832/2150557544,v1:172.21.15.143:6833/2150557544] conn(0x7fd5e00664a0 0x7fd5e0068960 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7fd5f0005890 tx=0x7fd5f0009040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:18:20.932 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:20.925+0000 7fd5f7fff640 1 -- 172.21.15.143:0/795124156 <== mon.0 v2:172.21.15.143:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172649 (secure 0 0 0) 0x7fd5f80ec050 con 0x7fd6001596d0 2023-11-26T07:18:21.084 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:21.077+0000 7fd60505a640 1 -- 172.21.15.143:0/795124156 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7fd600067680 con 0x7fd6001596d0 2023-11-26T07:18:21.084 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:21.077+0000 7fd5f7fff640 1 -- 172.21.15.143:0/795124156 <== mon.0 v2:172.21.15.143:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v20) v1 ==== 93+0+10 (secure 0 0 0) 0x7fd5f80ef670 con 0x7fd6001596d0 2023-11-26T07:18:21.084 INFO:teuthology.orchestra.run.smithi143.stdout:pg_num: 1 2023-11-26T07:18:21.087 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:21.081+0000 7fd60505a640 1 -- 172.21.15.143:0/795124156 >> [v2:172.21.15.143:6832/2150557544,v1:172.21.15.143:6833/2150557544] conn(0x7fd5e00664a0 msgr2=0x7fd5e0068960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:18:21.087 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:21.081+0000 7fd60505a640 1 --2- 172.21.15.143:0/795124156 >> [v2:172.21.15.143:6832/2150557544,v1:172.21.15.143:6833/2150557544] conn(0x7fd5e00664a0 0x7fd5e0068960 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7fd5f0005890 tx=0x7fd5f0009040 comp rx=0 tx=0).stop 2023-11-26T07:18:21.087 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:21.081+0000 7fd60505a640 1 -- 172.21.15.143:0/795124156 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fd6001596d0 msgr2=0x7fd60013c180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:18:21.087 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:21.081+0000 7fd60505a640 1 --2- 172.21.15.143:0/795124156 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fd6001596d0 0x7fd60013c180 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7fd5f80b0040 tx=0x7fd5f8019710 comp rx=0 tx=0).stop 2023-11-26T07:18:21.087 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:21.081+0000 7fd60505a640 1 -- 172.21.15.143:0/795124156 shutdown_connections 2023-11-26T07:18:21.087 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:21.081+0000 7fd60505a640 1 --2- 172.21.15.143:0/795124156 >> [v2:172.21.15.143:6832/2150557544,v1:172.21.15.143:6833/2150557544] conn(0x7fd5e00664a0 0x7fd5e0068960 unknown :-1 s=CLOSED pgs=134 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:18:21.088 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:21.081+0000 7fd60505a640 1 --2- 172.21.15.143:0/795124156 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fd6001596d0 0x7fd60013c180 unknown :-1 s=CLOSED pgs=206 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:18:21.088 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:21.081+0000 7fd60505a640 1 --2- 172.21.15.143:0/795124156 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fd6000ce680 0x7fd60014f4a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:18:21.088 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:21.081+0000 7fd60505a640 1 --2- 172.21.15.143:0/795124156 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fd6000b2f80 0x7fd6000ca2b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:18:21.088 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:21.081+0000 7fd60505a640 1 -- 172.21.15.143:0/795124156 >> 172.21.15.143:0/795124156 conn(0x7fd60006a890 msgr2=0x7fd60015cf80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:18:21.088 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:21.081+0000 7fd60505a640 1 -- 172.21.15.143:0/795124156 shutdown_connections 2023-11-26T07:18:21.088 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:21.081+0000 7fd60505a640 1 -- 172.21.15.143:0/795124156 wait complete. 2023-11-26T07:18:21.106 DEBUG:teuthology.orchestra.run.smithi143:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-11-26T07:18:21.246 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:21.241+0000 7f1c72e17640 1 Processor -- start 2023-11-26T07:18:21.246 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:21.241+0000 7f1c72e17640 1 -- start start 2023-11-26T07:18:21.247 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:21.241+0000 7f1c72e17640 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7f1c6c1614f0 0x7f1c6c0cec40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:18:21.247 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:21.241+0000 7f1c72e17640 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7f1c6c0745d0 0x7f1c6c1618d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:18:21.247 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:21.241+0000 7f1c72e17640 1 -- --> v1:172.21.15.143:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1c6c0ccfa0 con 0x7f1c6c161e10 2023-11-26T07:18:21.247 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:21.241+0000 7f1c72e17640 1 -- --> v2:172.21.15.143:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1c6c06e280 con 0x7f1c6c1614f0 2023-11-26T07:18:21.247 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:21.241+0000 7f1c72e17640 1 -- --> v2:172.21.15.188:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1c6c06e8b0 con 0x7f1c6c0745d0 2023-11-26T07:18:21.247 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:21.241+0000 7f1c70bb3640 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7f1c6c1614f0 0x7f1c6c0cec40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:18:21.247 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:21.241+0000 7f1c713b4640 1 --1- >> v1:172.21.15.143:6789/0 conn(0x7f1c6c161e10 0x7f1c6c15d1e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.143:6789/0 says I am v1:172.21.15.143:43944/0 (socket says 172.21.15.143:43944) 2023-11-26T07:18:21.248 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:21.241+0000 7f1c713b4640 1 -- 172.21.15.143:0/3902856871 learned_addr learned my addr 172.21.15.143:0/3902856871 (peer_addr_for_me v1:172.21.15.143:0/0) 2023-11-26T07:18:21.248 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:21.241+0000 7f1c70bb3640 1 -- 172.21.15.143:0/3902856871 >> v2:172.21.15.188:3300/0 conn(0x7f1c6c0745d0 msgr2=0x7f1c6c1618d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-26T07:18:21.248 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:21.241+0000 7f1c63fff640 1 --2- 172.21.15.143:0/3902856871 >> v2:172.21.15.188:3300/0 conn(0x7f1c6c0745d0 0x7f1c6c1618d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:18:21.248 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:21.241+0000 7f1c70bb3640 1 --2- 172.21.15.143:0/3902856871 >> v2:172.21.15.188:3300/0 conn(0x7f1c6c0745d0 0x7f1c6c1618d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:18:21.249 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:21.241+0000 7f1c637fe640 1 -- 172.21.15.143:0/3902856871 <== mon.0 v1:172.21.15.143:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3987592684 0 0) 0x7f1c6c0ccfa0 con 0x7f1c6c161e10 2023-11-26T07:18:21.249 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:21.241+0000 7f1c70bb3640 1 -- 172.21.15.143:0/3902856871 >> v1:172.21.15.143:6789/0 conn(0x7f1c6c161e10 legacy=0x7f1c6c15d1e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:18:21.249 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:21.241+0000 7f1c70bb3640 1 -- 172.21.15.143:0/3902856871 --> v2:172.21.15.143:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1c6c0cd240 con 0x7f1c6c1614f0 2023-11-26T07:18:21.249 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:21.241+0000 7f1c63fff640 1 --2- 172.21.15.143:0/3902856871 >> v2:172.21.15.188:3300/0 conn(0x7f1c6c0745d0 0x7f1c6c1618d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-26T07:18:21.250 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:21.245+0000 7f1c70bb3640 1 --2- 172.21.15.143:0/3902856871 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f1c6c1614f0 0x7f1c6c0cec40 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7f1c6c06fc10 tx=0x7f1c6801c3a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6681bde085ac75c9 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:18:21.250 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:21.245+0000 7f1c637fe640 1 -- 172.21.15.143:0/3902856871 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1c680ad660 con 0x7f1c6c1614f0 2023-11-26T07:18:21.251 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:21.245+0000 7f1c637fe640 1 -- 172.21.15.143:0/3902856871 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1c680ad7c0 con 0x7f1c6c1614f0 2023-11-26T07:18:21.251 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:21.245+0000 7f1c637fe640 1 -- 172.21.15.143:0/3902856871 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1c680adb50 con 0x7f1c6c1614f0 2023-11-26T07:18:21.251 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:21.245+0000 7f1c72e17640 1 -- 172.21.15.143:0/3902856871 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f1c6c1614f0 msgr2=0x7f1c6c0cec40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:18:21.252 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:21.245+0000 7f1c72e17640 1 --2- 172.21.15.143:0/3902856871 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f1c6c1614f0 0x7f1c6c0cec40 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7f1c6c06fc10 tx=0x7f1c6801c3a0 comp rx=0 tx=0).stop 2023-11-26T07:18:21.252 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:21.245+0000 7f1c72e17640 1 -- 172.21.15.143:0/3902856871 shutdown_connections 2023-11-26T07:18:21.252 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:21.245+0000 7f1c72e17640 1 --2- 172.21.15.143:0/3902856871 >> v2:172.21.15.188:3300/0 conn(0x7f1c6c0745d0 0x7f1c6c1618d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:18:21.252 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:21.245+0000 7f1c72e17640 1 --2- 172.21.15.143:0/3902856871 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f1c6c1614f0 0x7f1c6c0cec40 unknown :-1 s=CLOSED pgs=208 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:18:21.252 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:21.245+0000 7f1c72e17640 1 -- 172.21.15.143:0/3902856871 >> 172.21.15.143:0/3902856871 conn(0x7f1c6c0b0d80 msgr2=0x7f1c6c0b1190 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:18:21.252 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:21.245+0000 7f1c72e17640 1 -- 172.21.15.143:0/3902856871 shutdown_connections 2023-11-26T07:18:21.252 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:21.245+0000 7f1c72e17640 1 -- 172.21.15.143:0/3902856871 wait complete. 2023-11-26T07:18:21.253 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:21.249+0000 7f1c72e17640 1 Processor -- start 2023-11-26T07:18:21.253 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:21.249+0000 7f1c72e17640 1 -- start start 2023-11-26T07:18:21.255 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:21.249+0000 7f1c72e17640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f1c6c0745d0 0x7f1c6c152a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:18:21.255 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:21.249+0000 7f1c72e17640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f1c6c1614f0 0x7f1c6c097460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:18:21.255 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:21.249+0000 7f1c72e17640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f1c6c161e10 0x7f1c6c0979a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:18:21.255 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:21.249+0000 7f1c72e17640 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1c6c098c30 con 0x7f1c6c0745d0 2023-11-26T07:18:21.256 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:21.249+0000 7f1c72e17640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1c6c098020 con 0x7f1c6c161e10 2023-11-26T07:18:21.256 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:21.249+0000 7f1c72e17640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1c6c098300 con 0x7f1c6c1614f0 2023-11-26T07:18:21.256 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:21.249+0000 7f1c70bb3640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f1c6c0745d0 0x7f1c6c152a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:18:21.256 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:21.249+0000 7f1c70bb3640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f1c6c0745d0 0x7f1c6c152a00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:44042/0 (socket says 172.21.15.143:44042) 2023-11-26T07:18:21.256 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:21.249+0000 7f1c70bb3640 1 -- 172.21.15.143:0/962527209 learned_addr learned my addr 172.21.15.143:0/962527209 (peer_addr_for_me v2:172.21.15.143:0/0) 2023-11-26T07:18:21.256 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:21.249+0000 7f1c713b4640 1 --2- 172.21.15.143:0/962527209 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f1c6c161e10 0x7f1c6c0979a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:18:21.256 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:21.249+0000 7f1c70bb3640 1 -- 172.21.15.143:0/962527209 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f1c6c1614f0 msgr2=0x7f1c6c097460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:18:21.256 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:21.249+0000 7f1c70bb3640 1 --2- 172.21.15.143:0/962527209 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f1c6c1614f0 0x7f1c6c097460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:18:21.256 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:21.249+0000 7f1c70bb3640 1 -- 172.21.15.143:0/962527209 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f1c6c161e10 msgr2=0x7f1c6c0979a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:18:21.256 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:21.249+0000 7f1c70bb3640 1 --2- 172.21.15.143:0/962527209 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f1c6c161e10 0x7f1c6c0979a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:18:21.256 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:21.249+0000 7f1c70bb3640 1 -- 172.21.15.143:0/962527209 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1c68009420 con 0x7f1c6c0745d0 2023-11-26T07:18:21.257 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:21.249+0000 7f1c713b4640 1 --2- 172.21.15.143:0/962527209 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f1c6c161e10 0x7f1c6c0979a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-26T07:18:21.257 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:21.249+0000 7f1c70bb3640 1 --2- 172.21.15.143:0/962527209 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f1c6c0745d0 0x7f1c6c152a00 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7f1c680b0040 tx=0x7f1c6801abe0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:18:21.257 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:21.249+0000 7f1c617fa640 1 -- 172.21.15.143:0/962527209 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1c680b9660 con 0x7f1c6c0745d0 2023-11-26T07:18:21.257 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:21.249+0000 7f1c617fa640 1 -- 172.21.15.143:0/962527209 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1c680b97e0 con 0x7f1c6c0745d0 2023-11-26T07:18:21.257 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:21.249+0000 7f1c617fa640 1 -- 172.21.15.143:0/962527209 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1c680b9b90 con 0x7f1c6c0745d0 2023-11-26T07:18:21.257 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:21.249+0000 7f1c72e17640 1 -- 172.21.15.143:0/962527209 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1c6c0985e0 con 0x7f1c6c0745d0 2023-11-26T07:18:21.257 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:21.249+0000 7f1c72e17640 1 -- 172.21.15.143:0/962527209 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1c6c152f40 con 0x7f1c6c0745d0 2023-11-26T07:18:21.258 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:21.249+0000 7f1c72e17640 1 -- 172.21.15.143:0/962527209 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1c6c067680 con 0x7f1c6c0745d0 2023-11-26T07:18:21.258 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:21.253+0000 7f1c617fa640 1 -- 172.21.15.143:0/962527209 <== mon.0 v2:172.21.15.143:3300/0 4 ==== mgrmap(e 5) v1 ==== 85852+0+0 (secure 0 0 0) 0x7f1c680b9cf0 con 0x7f1c6c0745d0 2023-11-26T07:18:21.258 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:21.253+0000 7f1c617fa640 1 --2- 172.21.15.143:0/962527209 >> [v2:172.21.15.143:6832/2150557544,v1:172.21.15.143:6833/2150557544] conn(0x7f1c580664a0 0x7f1c58068960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:18:21.258 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:21.253+0000 7f1c617fa640 1 -- 172.21.15.143:0/962527209 <== mon.0 v2:172.21.15.143:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f1c6811eca0 con 0x7f1c6c0745d0 2023-11-26T07:18:21.258 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:21.253+0000 7f1c63fff640 1 --2- 172.21.15.143:0/962527209 >> [v2:172.21.15.143:6832/2150557544,v1:172.21.15.143:6833/2150557544] conn(0x7f1c580664a0 0x7f1c58068960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:18:21.259 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:21.253+0000 7f1c63fff640 1 --2- 172.21.15.143:0/962527209 >> [v2:172.21.15.143:6832/2150557544,v1:172.21.15.143:6833/2150557544] conn(0x7f1c580664a0 0x7f1c58068960 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7f1c6c160630 tx=0x7f1c5c008040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:18:21.262 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:21.257+0000 7f1c617fa640 1 -- 172.21.15.143:0/962527209 <== mon.0 v2:172.21.15.143:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172649 (secure 0 0 0) 0x7f1c680ebff0 con 0x7f1c6c0745d0 2023-11-26T07:18:21.410 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:21.405+0000 7f1c72e17640 1 -- 172.21.15.143:0/962527209 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f1c6c0b51b0 con 0x7f1c6c0745d0 2023-11-26T07:18:21.411 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:21.405+0000 7f1c617fa640 1 -- 172.21.15.143:0/962527209 <== mon.0 v2:172.21.15.143:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v20) v1 ==== 104+0+11 (secure 0 0 0) 0x7f1c680ef610 con 0x7f1c6c0745d0 2023-11-26T07:18:21.411 INFO:teuthology.orchestra.run.smithi143.stdout:pg_num: 32 2023-11-26T07:18:21.413 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:21.405+0000 7f1c72e17640 1 -- 172.21.15.143:0/962527209 >> [v2:172.21.15.143:6832/2150557544,v1:172.21.15.143:6833/2150557544] conn(0x7f1c580664a0 msgr2=0x7f1c58068960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:18:21.413 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:21.405+0000 7f1c72e17640 1 --2- 172.21.15.143:0/962527209 >> [v2:172.21.15.143:6832/2150557544,v1:172.21.15.143:6833/2150557544] conn(0x7f1c580664a0 0x7f1c58068960 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7f1c6c160630 tx=0x7f1c5c008040 comp rx=0 tx=0).stop 2023-11-26T07:18:21.413 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:21.409+0000 7f1c72e17640 1 -- 172.21.15.143:0/962527209 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f1c6c0745d0 msgr2=0x7f1c6c152a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:18:21.413 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:21.409+0000 7f1c72e17640 1 --2- 172.21.15.143:0/962527209 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f1c6c0745d0 0x7f1c6c152a00 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7f1c680b0040 tx=0x7f1c6801abe0 comp rx=0 tx=0).stop 2023-11-26T07:18:21.414 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:21.409+0000 7f1c72e17640 1 -- 172.21.15.143:0/962527209 shutdown_connections 2023-11-26T07:18:21.414 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:21.409+0000 7f1c72e17640 1 --2- 172.21.15.143:0/962527209 >> [v2:172.21.15.143:6832/2150557544,v1:172.21.15.143:6833/2150557544] conn(0x7f1c580664a0 0x7f1c58068960 unknown :-1 s=CLOSED pgs=135 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:18:21.414 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:21.409+0000 7f1c72e17640 1 --2- 172.21.15.143:0/962527209 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f1c6c161e10 0x7f1c6c0979a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:18:21.414 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:21.409+0000 7f1c72e17640 1 --2- 172.21.15.143:0/962527209 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f1c6c1614f0 0x7f1c6c097460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:18:21.414 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:21.409+0000 7f1c72e17640 1 --2- 172.21.15.143:0/962527209 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f1c6c0745d0 0x7f1c6c152a00 unknown :-1 s=CLOSED pgs=209 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:18:21.414 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:21.409+0000 7f1c72e17640 1 -- 172.21.15.143:0/962527209 >> 172.21.15.143:0/962527209 conn(0x7f1c6c0b0d80 msgr2=0x7f1c6c0adf80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:18:21.414 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:21.409+0000 7f1c72e17640 1 -- 172.21.15.143:0/962527209 shutdown_connections 2023-11-26T07:18:21.414 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:21.409+0000 7f1c72e17640 1 -- 172.21.15.143:0/962527209 wait complete. 2023-11-26T07:18:21.433 DEBUG:teuthology.orchestra.run.smithi143:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-11-26T07:18:21.572 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:21.565+0000 7ff94b6ea640 1 Processor -- start 2023-11-26T07:18:21.573 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:21.565+0000 7ff94b6ea640 1 -- start start 2023-11-26T07:18:21.575 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:21.569+0000 7ff94b6ea640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7ff94413e290 0x7ff9440cf280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:18:21.575 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:21.569+0000 7ff94b6ea640 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7ff9440cf7c0 0x7ff94415efa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:18:21.575 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:21.569+0000 7ff94b6ea640 1 -- --> v1:172.21.15.143:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff94406c520 con 0x7ff94415f4e0 2023-11-26T07:18:21.575 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:21.569+0000 7ff94b6ea640 1 -- --> v2:172.21.15.143:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff9440cfc40 con 0x7ff9440cf7c0 2023-11-26T07:18:21.575 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:21.569+0000 7ff94b6ea640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff94406c800 con 0x7ff94413e290 2023-11-26T07:18:21.575 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:21.569+0000 7ff948c85640 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7ff9440cf7c0 0x7ff94415efa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:18:21.575 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:21.569+0000 7ff949c87640 1 --1- >> v1:172.21.15.143:6789/0 conn(0x7ff94415f4e0 0x7ff9441408e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.143:6789/0 says I am v1:172.21.15.143:43954/0 (socket says 172.21.15.143:43954) 2023-11-26T07:18:21.575 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:21.569+0000 7ff949c87640 1 -- 172.21.15.143:0/3967669889 learned_addr learned my addr 172.21.15.143:0/3967669889 (peer_addr_for_me v1:172.21.15.143:0/0) 2023-11-26T07:18:21.575 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:21.569+0000 7ff949486640 1 --2- 172.21.15.143:0/3967669889 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7ff94413e290 0x7ff9440cf280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:18:21.575 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:21.569+0000 7ff93bfff640 1 -- 172.21.15.143:0/3967669889 <== mon.0 v1:172.21.15.143:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 418545853 0 0) 0x7ff94406c520 con 0x7ff94415f4e0 2023-11-26T07:18:21.576 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:21.569+0000 7ff93bfff640 1 -- 172.21.15.143:0/3967669889 --> v1:172.21.15.143:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7ff930003530 con 0x7ff94415f4e0 2023-11-26T07:18:21.576 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:21.569+0000 7ff948c85640 1 -- 172.21.15.143:0/3967669889 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7ff94413e290 msgr2=0x7ff9440cf280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-26T07:18:21.576 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:21.569+0000 7ff948c85640 1 --2- 172.21.15.143:0/3967669889 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7ff94413e290 0x7ff9440cf280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:18:21.576 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:21.569+0000 7ff948c85640 1 -- 172.21.15.143:0/3967669889 >> v1:172.21.15.143:6789/0 conn(0x7ff94415f4e0 legacy=0x7ff9441408e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:18:21.576 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:21.569+0000 7ff948c85640 1 -- 172.21.15.143:0/3967669889 --> v2:172.21.15.143:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff94406cae0 con 0x7ff9440cf7c0 2023-11-26T07:18:21.578 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:21.573+0000 7ff948c85640 1 --2- 172.21.15.143:0/3967669889 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7ff9440cf7c0 0x7ff94415efa0 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7ff9340097c0 tx=0x7ff93401c450 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d703b92d8ed93cb1 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:18:21.578 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:21.573+0000 7ff93bfff640 1 -- 172.21.15.143:0/3967669889 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff9340ab650 con 0x7ff9440cf7c0 2023-11-26T07:18:21.578 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:21.573+0000 7ff93bfff640 1 -- 172.21.15.143:0/3967669889 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff9340ab7b0 con 0x7ff9440cf7c0 2023-11-26T07:18:21.578 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:21.573+0000 7ff93bfff640 1 -- 172.21.15.143:0/3967669889 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff9340abb40 con 0x7ff9440cf7c0 2023-11-26T07:18:21.579 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:21.573+0000 7ff94b6ea640 1 -- 172.21.15.143:0/3967669889 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7ff9440cf7c0 msgr2=0x7ff94415efa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:18:21.579 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:21.573+0000 7ff94b6ea640 1 --2- 172.21.15.143:0/3967669889 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7ff9440cf7c0 0x7ff94415efa0 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7ff9340097c0 tx=0x7ff93401c450 comp rx=0 tx=0).stop 2023-11-26T07:18:21.579 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:21.573+0000 7ff94b6ea640 1 -- 172.21.15.143:0/3967669889 shutdown_connections 2023-11-26T07:18:21.579 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:21.573+0000 7ff94b6ea640 1 --2- 172.21.15.143:0/3967669889 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7ff9440cf7c0 0x7ff94415efa0 unknown :-1 s=CLOSED pgs=211 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:18:21.579 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:21.573+0000 7ff94b6ea640 1 --2- 172.21.15.143:0/3967669889 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7ff94413e290 0x7ff9440cf280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:18:21.579 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:21.573+0000 7ff94b6ea640 1 -- 172.21.15.143:0/3967669889 >> 172.21.15.143:0/3967669889 conn(0x7ff94406a890 msgr2=0x7ff94406aca0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:18:21.579 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:21.573+0000 7ff94b6ea640 1 -- 172.21.15.143:0/3967669889 shutdown_connections 2023-11-26T07:18:21.580 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:21.573+0000 7ff94b6ea640 1 -- 172.21.15.143:0/3967669889 wait complete. 2023-11-26T07:18:21.580 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:21.573+0000 7ff94b6ea640 1 Processor -- start 2023-11-26T07:18:21.580 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:21.573+0000 7ff94b6ea640 1 -- start start 2023-11-26T07:18:21.582 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:21.577+0000 7ff94b6ea640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7ff9440cf7c0 0x7ff94415e220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:18:21.582 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:21.577+0000 7ff94b6ea640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7ff94413e290 0x7ff94415c950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:18:21.582 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:21.577+0000 7ff94b6ea640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7ff94415f4e0 0x7ff9441370d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:18:21.582 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:21.577+0000 7ff94b6ea640 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff9440b0b70 con 0x7ff9440cf7c0 2023-11-26T07:18:21.583 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:21.577+0000 7ff94b6ea640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff9440b0260 con 0x7ff94415f4e0 2023-11-26T07:18:21.583 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:21.577+0000 7ff94b6ea640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff9440b0520 con 0x7ff94413e290 2023-11-26T07:18:21.583 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:21.577+0000 7ff949486640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7ff9440cf7c0 0x7ff94415e220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:18:21.583 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:21.577+0000 7ff949486640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7ff9440cf7c0 0x7ff94415e220 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:44052/0 (socket says 172.21.15.143:44052) 2023-11-26T07:18:21.583 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:21.577+0000 7ff949486640 1 -- 172.21.15.143:0/3906177060 learned_addr learned my addr 172.21.15.143:0/3906177060 (peer_addr_for_me v2:172.21.15.143:0/0) 2023-11-26T07:18:21.583 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:21.577+0000 7ff949486640 1 -- 172.21.15.143:0/3906177060 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7ff94413e290 msgr2=0x7ff94415c950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:18:21.583 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:21.577+0000 7ff949486640 1 --2- 172.21.15.143:0/3906177060 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7ff94413e290 0x7ff94415c950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:18:21.583 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:21.577+0000 7ff949486640 1 -- 172.21.15.143:0/3906177060 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7ff94415f4e0 msgr2=0x7ff9441370d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:18:21.583 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:21.577+0000 7ff949486640 1 --2- 172.21.15.143:0/3906177060 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7ff94415f4e0 0x7ff9441370d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:18:21.583 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:21.577+0000 7ff949486640 1 -- 172.21.15.143:0/3906177060 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff934009420 con 0x7ff9440cf7c0 2023-11-26T07:18:21.583 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:21.577+0000 7ff949486640 1 --2- 172.21.15.143:0/3906177060 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7ff9440cf7c0 0x7ff94415e220 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7ff94000e930 tx=0x7ff94000edf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:18:21.584 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:21.577+0000 7ff939ffb640 1 -- 172.21.15.143:0/3906177060 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff940009690 con 0x7ff9440cf7c0 2023-11-26T07:18:21.584 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:21.577+0000 7ff939ffb640 1 -- 172.21.15.143:0/3906177060 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff94000ca00 con 0x7ff9440cf7c0 2023-11-26T07:18:21.584 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:21.577+0000 7ff939ffb640 1 -- 172.21.15.143:0/3906177060 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff94000cd70 con 0x7ff9440cf7c0 2023-11-26T07:18:21.585 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:21.577+0000 7ff94b6ea640 1 -- 172.21.15.143:0/3906177060 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff9440b0800 con 0x7ff9440cf7c0 2023-11-26T07:18:21.585 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:21.577+0000 7ff94b6ea640 1 -- 172.21.15.143:0/3906177060 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff944137ab0 con 0x7ff9440cf7c0 2023-11-26T07:18:21.585 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:21.577+0000 7ff94b6ea640 1 -- 172.21.15.143:0/3906177060 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff944067680 con 0x7ff9440cf7c0 2023-11-26T07:18:21.585 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:21.581+0000 7ff939ffb640 1 -- 172.21.15.143:0/3906177060 <== mon.0 v2:172.21.15.143:3300/0 4 ==== mgrmap(e 5) v1 ==== 85852+0+0 (secure 0 0 0) 0x7ff940013470 con 0x7ff9440cf7c0 2023-11-26T07:18:21.586 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:21.581+0000 7ff939ffb640 1 --2- 172.21.15.143:0/3906177060 >> [v2:172.21.15.143:6832/2150557544,v1:172.21.15.143:6833/2150557544] conn(0x7ff928066470 0x7ff928068930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:18:21.586 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:21.581+0000 7ff939ffb640 1 -- 172.21.15.143:0/3906177060 <== mon.0 v2:172.21.15.143:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5709+0+0 (secure 0 0 0) 0x7ff940084170 con 0x7ff9440cf7c0 2023-11-26T07:18:21.586 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:21.581+0000 7ff948c85640 1 --2- 172.21.15.143:0/3906177060 >> [v2:172.21.15.143:6832/2150557544,v1:172.21.15.143:6833/2150557544] conn(0x7ff928066470 0x7ff928068930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:18:21.586 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:21.581+0000 7ff948c85640 1 --2- 172.21.15.143:0/3906177060 >> [v2:172.21.15.143:6832/2150557544,v1:172.21.15.143:6833/2150557544] conn(0x7ff928066470 0x7ff928068930 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7ff9340af000 tx=0x7ff934008040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:18:21.589 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:21.585+0000 7ff939ffb640 1 -- 172.21.15.143:0/3906177060 <== mon.0 v2:172.21.15.143:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172649 (secure 0 0 0) 0x7ff940051690 con 0x7ff9440cf7c0 2023-11-26T07:18:21.620 DEBUG:teuthology.orchestra.run.smithi143:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-26T07:18:21.623 DEBUG:teuthology.orchestra.run.smithi188:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-26T07:18:21.743 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:21.737+0000 7ff94b6ea640 1 -- 172.21.15.143:0/3906177060 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7ff94406df50 con 0x7ff9440cf7c0 2023-11-26T07:18:21.744 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:21.737+0000 7ff939ffb640 1 -- 172.21.15.143:0/3906177060 <== mon.0 v2:172.21.15.143:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v20) v1 ==== 100+0+11 (secure 0 0 0) 0x7ff940054cb0 con 0x7ff9440cf7c0 2023-11-26T07:18:21.744 INFO:teuthology.orchestra.run.smithi143.stdout:pg_num: 64 2023-11-26T07:18:21.746 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:21.741+0000 7ff94b6ea640 1 -- 172.21.15.143:0/3906177060 >> [v2:172.21.15.143:6832/2150557544,v1:172.21.15.143:6833/2150557544] conn(0x7ff928066470 msgr2=0x7ff928068930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:18:21.746 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:21.741+0000 7ff94b6ea640 1 --2- 172.21.15.143:0/3906177060 >> [v2:172.21.15.143:6832/2150557544,v1:172.21.15.143:6833/2150557544] conn(0x7ff928066470 0x7ff928068930 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7ff9340af000 tx=0x7ff934008040 comp rx=0 tx=0).stop 2023-11-26T07:18:21.747 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:21.741+0000 7ff94b6ea640 1 -- 172.21.15.143:0/3906177060 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7ff9440cf7c0 msgr2=0x7ff94415e220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:18:21.747 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:21.741+0000 7ff94b6ea640 1 --2- 172.21.15.143:0/3906177060 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7ff9440cf7c0 0x7ff94415e220 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7ff94000e930 tx=0x7ff94000edf0 comp rx=0 tx=0).stop 2023-11-26T07:18:21.747 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:21.741+0000 7ff94b6ea640 1 -- 172.21.15.143:0/3906177060 shutdown_connections 2023-11-26T07:18:21.747 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:21.741+0000 7ff94b6ea640 1 --2- 172.21.15.143:0/3906177060 >> [v2:172.21.15.143:6832/2150557544,v1:172.21.15.143:6833/2150557544] conn(0x7ff928066470 0x7ff928068930 unknown :-1 s=CLOSED pgs=136 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:18:21.747 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:21.741+0000 7ff94b6ea640 1 --2- 172.21.15.143:0/3906177060 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7ff94415f4e0 0x7ff9441370d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:18:21.747 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:21.741+0000 7ff94b6ea640 1 --2- 172.21.15.143:0/3906177060 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7ff94413e290 0x7ff94415c950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:18:21.747 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:21.741+0000 7ff94b6ea640 1 --2- 172.21.15.143:0/3906177060 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7ff9440cf7c0 0x7ff94415e220 unknown :-1 s=CLOSED pgs=212 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:18:21.747 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:21.741+0000 7ff94b6ea640 1 -- 172.21.15.143:0/3906177060 >> 172.21.15.143:0/3906177060 conn(0x7ff94406a890 msgr2=0x7ff944140720 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:18:21.747 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:21.741+0000 7ff94b6ea640 1 -- 172.21.15.143:0/3906177060 shutdown_connections 2023-11-26T07:18:21.747 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:21.741+0000 7ff94b6ea640 1 -- 172.21.15.143:0/3906177060 wait complete. 2023-11-26T07:18:21.835 INFO:tasks.cephfs_test_runner:Starting test: test_binary_metafile_on_legacy_to_v1_upgrade (tasks.cephfs.test_volumes.TestMisc) Validate binary .meta file on legacy subvol root doesn't break the system 2023-11-26T07:18:21.835 INFO:tasks.cephfs_test_runner:test_binary_metafile_on_legacy_to_v1_upgrade (tasks.cephfs.test_volumes.TestMisc) 2023-11-26T07:18:21.835 DEBUG:teuthology.orchestra.run.smithi143:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-11-26T07:18:21.973 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:21.965+0000 7fe97e46b640 1 Processor -- start 2023-11-26T07:18:21.973 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:21.965+0000 7fe97e46b640 1 -- start start 2023-11-26T07:18:21.975 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:21.969+0000 7fe97e46b640 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7fe97813e290 0x7fe9780cf280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:18:21.975 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:21.969+0000 7fe97e46b640 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7fe97815f6b0 0x7fe9780cd7b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:18:21.975 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:21.969+0000 7fe97e46b640 1 -- --> v1:172.21.15.143:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe97815fb20 con 0x7fe9780cf7c0 2023-11-26T07:18:21.975 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:21.969+0000 7fe97e46b640 1 -- --> v2:172.21.15.143:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe9780cfc40 con 0x7fe97813e290 2023-11-26T07:18:21.975 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:21.969+0000 7fe97e46b640 1 -- --> v2:172.21.15.188:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe97806c520 con 0x7fe97815f6b0 2023-11-26T07:18:21.975 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:21.969+0000 7fe9777fe640 1 --1- >> v1:172.21.15.143:6789/0 conn(0x7fe9780cf7c0 0x7fe97815efa0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.143:6789/0 says I am v1:172.21.15.143:43970/0 (socket says 172.21.15.143:43970) 2023-11-26T07:18:21.975 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:21.969+0000 7fe977fff640 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7fe97813e290 0x7fe9780cf280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:18:21.975 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:21.969+0000 7fe9777fe640 1 -- 172.21.15.143:0/3812097183 learned_addr learned my addr 172.21.15.143:0/3812097183 (peer_addr_for_me v1:172.21.15.143:0/0) 2023-11-26T07:18:21.975 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:21.969+0000 7fe97ca08640 1 --2- 172.21.15.143:0/3812097183 >> v2:172.21.15.188:3300/0 conn(0x7fe97815f6b0 0x7fe9780cd7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:18:21.975 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:21.969+0000 7fe976ffd640 1 -- 172.21.15.143:0/3812097183 <== mon.0 v1:172.21.15.143:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3927780944 0 0) 0x7fe96c001120 con 0x7fe9780cf7c0 2023-11-26T07:18:21.975 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:21.969+0000 7fe976ffd640 1 -- 172.21.15.143:0/3812097183 --> v1:172.21.15.143:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fe960003530 con 0x7fe9780cf7c0 2023-11-26T07:18:21.975 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:21.969+0000 7fe977fff640 1 -- 172.21.15.143:0/3812097183 >> v2:172.21.15.188:3300/0 conn(0x7fe97815f6b0 msgr2=0x7fe9780cd7b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-26T07:18:21.975 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:21.969+0000 7fe977fff640 1 --2- 172.21.15.143:0/3812097183 >> v2:172.21.15.188:3300/0 conn(0x7fe97815f6b0 0x7fe9780cd7b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:18:21.976 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:21.969+0000 7fe977fff640 1 -- 172.21.15.143:0/3812097183 >> v1:172.21.15.143:6789/0 conn(0x7fe9780cf7c0 legacy=0x7fe97815efa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:18:21.976 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:21.969+0000 7fe977fff640 1 -- 172.21.15.143:0/3812097183 --> v2:172.21.15.143:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe97806c800 con 0x7fe97813e290 2023-11-26T07:18:21.976 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:21.969+0000 7fe97ca08640 1 --2- 172.21.15.143:0/3812097183 >> v2:172.21.15.188:3300/0 conn(0x7fe97815f6b0 0x7fe9780cd7b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-26T07:18:21.977 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:21.973+0000 7fe977fff640 1 --2- 172.21.15.143:0/3812097183 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fe97813e290 0x7fe9780cf280 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7fe97806e780 tx=0x7fe96801c430 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f3121bb8876f384c server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:18:21.977 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:21.973+0000 7fe976ffd640 1 -- 172.21.15.143:0/3812097183 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe9680a7070 con 0x7fe97813e290 2023-11-26T07:18:21.978 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:21.973+0000 7fe976ffd640 1 -- 172.21.15.143:0/3812097183 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe9680ab430 con 0x7fe97813e290 2023-11-26T07:18:21.978 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:21.973+0000 7fe976ffd640 1 -- 172.21.15.143:0/3812097183 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe9680ab7c0 con 0x7fe97813e290 2023-11-26T07:18:21.978 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:21.973+0000 7fe97e46b640 1 -- 172.21.15.143:0/3812097183 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fe97813e290 msgr2=0x7fe9780cf280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:18:21.979 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:21.973+0000 7fe97e46b640 1 --2- 172.21.15.143:0/3812097183 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fe97813e290 0x7fe9780cf280 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7fe97806e780 tx=0x7fe96801c430 comp rx=0 tx=0).stop 2023-11-26T07:18:21.979 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:21.973+0000 7fe97e46b640 1 -- 172.21.15.143:0/3812097183 shutdown_connections 2023-11-26T07:18:21.979 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:21.973+0000 7fe97e46b640 1 --2- 172.21.15.143:0/3812097183 >> v2:172.21.15.188:3300/0 conn(0x7fe97815f6b0 0x7fe9780cd7b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:18:21.979 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:21.973+0000 7fe97e46b640 1 --2- 172.21.15.143:0/3812097183 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fe97813e290 0x7fe9780cf280 unknown :-1 s=CLOSED pgs=214 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:18:21.979 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:21.973+0000 7fe97e46b640 1 -- 172.21.15.143:0/3812097183 >> 172.21.15.143:0/3812097183 conn(0x7fe97806a890 msgr2=0x7fe97806aca0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:18:21.979 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:21.973+0000 7fe97e46b640 1 -- 172.21.15.143:0/3812097183 shutdown_connections 2023-11-26T07:18:21.979 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:21.973+0000 7fe97e46b640 1 -- 172.21.15.143:0/3812097183 wait complete. 2023-11-26T07:18:21.980 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:21.973+0000 7fe97e46b640 1 Processor -- start 2023-11-26T07:18:21.980 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:21.973+0000 7fe97e46b640 1 -- start start 2023-11-26T07:18:21.982 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:21.977+0000 7fe97e46b640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fe9780cf7c0 0x7fe97813a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:18:21.982 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:21.977+0000 7fe97e46b640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fe97813e290 0x7fe9780adba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:18:21.982 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:21.977+0000 7fe97e46b640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fe97815f6b0 0x7fe9780aaa00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:18:21.982 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:21.977+0000 7fe97e46b640 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe97815df00 con 0x7fe97815f6b0 2023-11-26T07:18:21.982 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:21.977+0000 7fe97e46b640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe97815d2f0 con 0x7fe9780cf7c0 2023-11-26T07:18:21.982 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:21.977+0000 7fe97e46b640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe97815d5d0 con 0x7fe97813e290 2023-11-26T07:18:21.982 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:21.977+0000 7fe97ca08640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fe97815f6b0 0x7fe9780aaa00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:18:21.982 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:21.977+0000 7fe97ca08640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fe97815f6b0 0x7fe9780aaa00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:44074/0 (socket says 172.21.15.143:44074) 2023-11-26T07:18:21.982 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:21.977+0000 7fe97ca08640 1 -- 172.21.15.143:0/235811717 learned_addr learned my addr 172.21.15.143:0/235811717 (peer_addr_for_me v2:172.21.15.143:0/0) 2023-11-26T07:18:21.983 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:21.977+0000 7fe977fff640 1 --2- 172.21.15.143:0/235811717 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fe9780cf7c0 0x7fe97813a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:18:21.983 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:21.977+0000 7fe9777fe640 1 --2- 172.21.15.143:0/235811717 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fe97813e290 0x7fe9780adba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:18:21.983 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:21.977+0000 7fe97ca08640 1 -- 172.21.15.143:0/235811717 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fe97813e290 msgr2=0x7fe9780adba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:18:21.983 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:21.977+0000 7fe97ca08640 1 --2- 172.21.15.143:0/235811717 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fe97813e290 0x7fe9780adba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:18:21.983 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:21.977+0000 7fe97ca08640 1 -- 172.21.15.143:0/235811717 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fe9780cf7c0 msgr2=0x7fe97813a7a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:18:21.983 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:21.977+0000 7fe97ca08640 1 --2- 172.21.15.143:0/235811717 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fe9780cf7c0 0x7fe97813a7a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:18:21.983 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:21.977+0000 7fe97ca08640 1 -- 172.21.15.143:0/235811717 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe968009420 con 0x7fe97815f6b0 2023-11-26T07:18:21.983 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:21.977+0000 7fe97ca08640 1 --2- 172.21.15.143:0/235811717 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fe97815f6b0 0x7fe9780aaa00 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7fe970007df0 tx=0x7fe97000d4a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:18:21.984 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:21.977+0000 7fe974ff9640 1 -- 172.21.15.143:0/235811717 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe9700039a0 con 0x7fe97815f6b0 2023-11-26T07:18:21.984 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:21.977+0000 7fe974ff9640 1 -- 172.21.15.143:0/235811717 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe97000f040 con 0x7fe97815f6b0 2023-11-26T07:18:21.984 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:21.977+0000 7fe974ff9640 1 -- 172.21.15.143:0/235811717 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe970003d40 con 0x7fe97815f6b0 2023-11-26T07:18:21.984 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:21.977+0000 7fe97e46b640 1 -- 172.21.15.143:0/235811717 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe97815d8b0 con 0x7fe97815f6b0 2023-11-26T07:18:21.984 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:21.977+0000 7fe97e46b640 1 -- 172.21.15.143:0/235811717 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe97815cf70 con 0x7fe97815f6b0 2023-11-26T07:18:21.985 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:21.977+0000 7fe97e46b640 1 -- 172.21.15.143:0/235811717 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe978067680 con 0x7fe97815f6b0 2023-11-26T07:18:21.985 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:21.977+0000 7fe974ff9640 1 -- 172.21.15.143:0/235811717 <== mon.0 v2:172.21.15.143:3300/0 4 ==== mgrmap(e 5) v1 ==== 85852+0+0 (secure 0 0 0) 0x7fe970004030 con 0x7fe97815f6b0 2023-11-26T07:18:21.985 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:21.981+0000 7fe974ff9640 1 --2- 172.21.15.143:0/235811717 >> [v2:172.21.15.143:6832/2150557544,v1:172.21.15.143:6833/2150557544] conn(0x7fe964066470 0x7fe964068930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:18:21.985 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:21.981+0000 7fe974ff9640 1 -- 172.21.15.143:0/235811717 <== mon.0 v2:172.21.15.143:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fe9700840d0 con 0x7fe97815f6b0 2023-11-26T07:18:21.985 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:21.981+0000 7fe977fff640 1 --2- 172.21.15.143:0/235811717 >> [v2:172.21.15.143:6832/2150557544,v1:172.21.15.143:6833/2150557544] conn(0x7fe964066470 0x7fe964068930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:18:21.986 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:21.981+0000 7fe977fff640 1 --2- 172.21.15.143:0/235811717 >> [v2:172.21.15.143:6832/2150557544,v1:172.21.15.143:6833/2150557544] conn(0x7fe964066470 0x7fe964068930 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7fe9680af000 tx=0x7fe9680a7000 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:18:21.989 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:21.985+0000 7fe974ff9640 1 -- 172.21.15.143:0/235811717 <== mon.0 v2:172.21.15.143:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172649 (secure 0 0 0) 0x7fe9700515f0 con 0x7fe97815f6b0 2023-11-26T07:18:22.139 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:22.133+0000 7fe97e46b640 1 -- 172.21.15.143:0/235811717 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fe97806df50 con 0x7fe97815f6b0 2023-11-26T07:18:22.140 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:22.133+0000 7fe974ff9640 1 -- 172.21.15.143:0/235811717 <== mon.0 v2:172.21.15.143:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v20) v1 ==== 74+0+16541 (secure 0 0 0) 0x7fe970054c10 con 0x7fe97815f6b0 2023-11-26T07:18:22.140 INFO:teuthology.orchestra.run.smithi143.stdout: 2023-11-26T07:18:22.141 INFO:teuthology.orchestra.run.smithi143.stdout:{"epoch":20,"fsid":"f5033060-2469-4a07-95fe-2b76f5404bc1","created":"2023-11-26T07:17:28.613663+0000","modified":"2023-11-26T07:17:55.505435+0000","last_up_change":"2023-11-26T07:17:41.157813+0000","last_in_change":"2023-11-26T07:17:31.181658+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-11-26T07:17:41.758597+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2023-11-26T07:17:51.878332+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2023-11-26T07:17:52.871396+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"ebbee2c6-023e-44b9-9f1d-6f7bfd309730","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6817","nonce":639807369},{"type":"v1","addr":"172.21.15.143:6819","nonce":639807369}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6821","nonce":639807369},{"type":"v1","addr":"172.21.15.143:6823","nonce":639807369}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6830","nonce":639807369},{"type":"v1","addr":"172.21.15.143:6831","nonce":639807369}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6825","nonce":639807369},{"type":"v1","addr":"172.21.15.143:6828","nonce":639807369}]},"public_addr":"172.21.15.143:6819/639807369","cluster_addr":"172.21.15.143:6823/639807369","heartbeat_back_addr":"172.21.15.143:6831/639807369","heartbeat_front_addr":"172.21.15.143:6828/639807369","state":["exists","up"]},{"osd":1,"uuid":"fec4b8be-7122-49e9-9a19-5850a0700001","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6816","nonce":1144942530},{"type":"v1","addr":"172.21.15.143:6818","nonce":1144942530}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6820","nonce":1144942530},{"type":"v1","addr":"172.21.15.143:6822","nonce":1144942530}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6827","nonce":1144942530},{"type":"v1","addr":"172.21.15.143:6829","nonce":1144942530}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6824","nonce":1144942530},{"type":"v1","addr":"172.21.15.143:6826","nonce":1144942530}]},"public_addr":"172.21.15.143:6818/1144942530","cluster_addr":"172.21.15.143:6822/1144942530","heartbeat_back_addr":"172.21.15.143:6829/1144942530","heartbeat_front_addr":"172.21.15.143:6826/1144942530","state":["exists","up"]},{"osd":2,"uuid":"2888af5b-f933-46ca-8f95-6dd494ff43f6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6808","nonce":3909095186},{"type":"v1","addr":"172.21.15.143:6809","nonce":3909095186}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6810","nonce":3909095186},{"type":"v1","addr":"172.21.15.143:6811","nonce":3909095186}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6814","nonce":3909095186},{"type":"v1","addr":"172.21.15.143:6815","nonce":3909095186}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6812","nonce":3909095186},{"type":"v1","addr":"172.21.15.143:6813","nonce":3909095186}]},"public_addr":"172.21.15.143:6809/3909095186","cluster_addr":"172.21.15.143:6811/3909095186","heartbeat_back_addr":"172.21.15.143:6815/3909095186","heartbeat_front_addr":"172.21.15.143:6813/3909095186","state":["exists","up"]},{"osd":3,"uuid":"d1f52435-7bc1-4081-9550-9cd1ba55f549","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6800","nonce":3854210991},{"type":"v1","addr":"172.21.15.143:6801","nonce":3854210991}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6802","nonce":3854210991},{"type":"v1","addr":"172.21.15.143:6803","nonce":3854210991}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6806","nonce":3854210991},{"type":"v1","addr":"172.21.15.143:6807","nonce":3854210991}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6804","nonce":3854210991},{"type":"v1","addr":"172.21.15.143:6805","nonce":3854210991}]},"public_addr":"172.21.15.143:6801/3854210991","cluster_addr":"172.21.15.143:6803/3854210991","heartbeat_back_addr":"172.21.15.143:6807/3854210991","heartbeat_front_addr":"172.21.15.143:6805/3854210991","state":["exists","up"]},{"osd":4,"uuid":"e2ae8b6e-6e25-49b2-bcbf-aacce68f06a3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6800","nonce":3924606902},{"type":"v1","addr":"172.21.15.188:6801","nonce":3924606902}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6802","nonce":3924606902},{"type":"v1","addr":"172.21.15.188:6803","nonce":3924606902}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6806","nonce":3924606902},{"type":"v1","addr":"172.21.15.188:6807","nonce":3924606902}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6804","nonce":3924606902},{"type":"v1","addr":"172.21.15.188:6805","nonce":3924606902}]},"public_addr":"172.21.15.188:6801/3924606902","cluster_addr":"172.21.15.188:6803/3924606902","heartbeat_back_addr":"172.21.15.188:6807/3924606902","heartbeat_front_addr":"172.21.15.188:6805/3924606902","state":["exists","up"]},{"osd":5,"uuid":"3cec8a32-7698-4c36-8e89-e4ba11934f33","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6813","nonce":186841544},{"type":"v1","addr":"172.21.15.188:6815","nonce":186841544}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6817","nonce":186841544},{"type":"v1","addr":"172.21.15.188:6818","nonce":186841544}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6822","nonce":186841544},{"type":"v1","addr":"172.21.15.188:6823","nonce":186841544}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6820","nonce":186841544},{"type":"v1","addr":"172.21.15.188:6821","nonce":186841544}]},"public_addr":"172.21.15.188:6815/186841544","cluster_addr":"172.21.15.188:6818/186841544","heartbeat_back_addr":"172.21.15.188:6823/186841544","heartbeat_front_addr":"172.21.15.188:6821/186841544","state":["exists","up"]},{"osd":6,"uuid":"58a65d5d-e090-473a-81b4-ad38e85029f9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6808","nonce":2153499405},{"type":"v1","addr":"172.21.15.188:6809","nonce":2153499405}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6810","nonce":2153499405},{"type":"v1","addr":"172.21.15.188:6811","nonce":2153499405}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6816","nonce":2153499405},{"type":"v1","addr":"172.21.15.188:6819","nonce":2153499405}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6812","nonce":2153499405},{"type":"v1","addr":"172.21.15.188:6814","nonce":2153499405}]},"public_addr":"172.21.15.188:6809/2153499405","cluster_addr":"172.21.15.188:6811/2153499405","heartbeat_back_addr":"172.21.15.188:6819/2153499405","heartbeat_front_addr":"172.21.15.188:6814/2153499405","state":["exists","up"]},{"osd":7,"uuid":"ea5b7f70-3554-4cf8-ac30-f7c59b80678d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6824","nonce":2722096729},{"type":"v1","addr":"172.21.15.188:6825","nonce":2722096729}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6826","nonce":2722096729},{"type":"v1","addr":"172.21.15.188:6827","nonce":2722096729}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6830","nonce":2722096729},{"type":"v1","addr":"172.21.15.188:6831","nonce":2722096729}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6828","nonce":2722096729},{"type":"v1","addr":"172.21.15.188:6829","nonce":2722096729}]},"public_addr":"172.21.15.188:6825/2722096729","cluster_addr":"172.21.15.188:6827/2722096729","heartbeat_back_addr":"172.21.15.188:6831/2722096729","heartbeat_front_addr":"172.21.15.188:6829/2722096729","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-26T07:17:36.995032+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-26T07:17:36.932266+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-26T07:17:36.829304+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-26T07:17:36.804297+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-26T07:17:36.667892+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-26T07:17:36.964868+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-26T07:17:36.827332+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-26T07:17:36.893179+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-11-26T07:18:22.142 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:22.137+0000 7fe97e46b640 1 -- 172.21.15.143:0/235811717 >> [v2:172.21.15.143:6832/2150557544,v1:172.21.15.143:6833/2150557544] conn(0x7fe964066470 msgr2=0x7fe964068930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:18:22.142 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:22.137+0000 7fe97e46b640 1 --2- 172.21.15.143:0/235811717 >> [v2:172.21.15.143:6832/2150557544,v1:172.21.15.143:6833/2150557544] conn(0x7fe964066470 0x7fe964068930 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7fe9680af000 tx=0x7fe9680a7000 comp rx=0 tx=0).stop 2023-11-26T07:18:22.142 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:22.137+0000 7fe97e46b640 1 -- 172.21.15.143:0/235811717 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fe97815f6b0 msgr2=0x7fe9780aaa00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:18:22.142 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:22.137+0000 7fe97e46b640 1 --2- 172.21.15.143:0/235811717 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fe97815f6b0 0x7fe9780aaa00 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7fe970007df0 tx=0x7fe97000d4a0 comp rx=0 tx=0).stop 2023-11-26T07:18:22.142 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:22.137+0000 7fe97e46b640 1 -- 172.21.15.143:0/235811717 shutdown_connections 2023-11-26T07:18:22.142 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:22.137+0000 7fe97e46b640 1 --2- 172.21.15.143:0/235811717 >> [v2:172.21.15.143:6832/2150557544,v1:172.21.15.143:6833/2150557544] conn(0x7fe964066470 0x7fe964068930 unknown :-1 s=CLOSED pgs=137 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:18:22.142 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:22.137+0000 7fe97e46b640 1 --2- 172.21.15.143:0/235811717 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fe97815f6b0 0x7fe9780aaa00 unknown :-1 s=CLOSED pgs=215 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:18:22.143 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:22.137+0000 7fe97e46b640 1 --2- 172.21.15.143:0/235811717 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fe97813e290 0x7fe9780adba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:18:22.143 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:22.137+0000 7fe97e46b640 1 --2- 172.21.15.143:0/235811717 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fe9780cf7c0 0x7fe97813a7a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:18:22.143 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:22.137+0000 7fe97e46b640 1 -- 172.21.15.143:0/235811717 >> 172.21.15.143:0/235811717 conn(0x7fe97806a890 msgr2=0x7fe9780af360 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:18:22.143 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:22.137+0000 7fe97e46b640 1 -- 172.21.15.143:0/235811717 shutdown_connections 2023-11-26T07:18:22.143 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:22.137+0000 7fe97e46b640 1 -- 172.21.15.143:0/235811717 wait complete. 2023-11-26T07:18:22.161 INFO:tasks.ceph_test_case.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-11-26T07:17:41.758597+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2023-11-26T07:17:51.878332+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2023-11-26T07:17:52.871396+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '20', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2023-11-26T07:18:22.161 DEBUG:teuthology.orchestra.run.smithi143:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-11-26T07:18:22.302 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:22.297+0000 7fe2a9e4d640 1 Processor -- start 2023-11-26T07:18:22.303 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:22.297+0000 7fe2a9e4d640 1 -- start start 2023-11-26T07:18:22.304 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:22.297+0000 7fe2a9e4d640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fe2a413e290 0x7fe2a40cf280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:18:22.304 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:22.297+0000 7fe2a9e4d640 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7fe2a415f6b0 0x7fe2a40cd7b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:18:22.304 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:22.297+0000 7fe2a9e4d640 1 -- --> v1:172.21.15.188:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe2a406c520 con 0x7fe2a40cf7c0 2023-11-26T07:18:22.304 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:22.297+0000 7fe2a9e4d640 1 -- --> v2:172.21.15.143:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe2a40cfc40 con 0x7fe2a415f6b0 2023-11-26T07:18:22.304 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:22.297+0000 7fe2a9e4d640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe2a415fb40 con 0x7fe2a413e290 2023-11-26T07:18:22.304 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:22.297+0000 7fe2a3fff640 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7fe2a415f6b0 0x7fe2a40cd7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:18:22.304 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:22.297+0000 7fe2a3fff640 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7fe2a415f6b0 0x7fe2a40cd7b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:44078/0 (socket says 172.21.15.143:44078) 2023-11-26T07:18:22.304 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:22.297+0000 7fe2a3fff640 1 -- 172.21.15.143:0/1621957734 learned_addr learned my addr 172.21.15.143:0/1621957734 (peer_addr_for_me v2:172.21.15.143:0/0) 2023-11-26T07:18:22.304 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:22.297+0000 7fe2a37fe640 1 --2- 172.21.15.143:0/1621957734 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fe2a413e290 0x7fe2a40cf280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:18:22.305 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:22.297+0000 7fe2a3fff640 1 -- 172.21.15.143:0/1621957734 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fe2a413e290 msgr2=0x7fe2a40cf280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-26T07:18:22.305 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:22.297+0000 7fe2a3fff640 1 --2- 172.21.15.143:0/1621957734 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fe2a413e290 0x7fe2a40cf280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:18:22.305 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:22.297+0000 7fe2a3fff640 1 -- 172.21.15.143:0/1621957734 >> v1:172.21.15.188:6789/0 conn(0x7fe2a40cf7c0 legacy=0x7fe2a415efa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:18:22.305 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:22.297+0000 7fe2a3fff640 1 -- 172.21.15.143:0/1621957734 --> v2:172.21.15.143:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe2a406c800 con 0x7fe2a415f6b0 2023-11-26T07:18:22.307 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:22.301+0000 7fe2a3fff640 1 --2- 172.21.15.143:0/1621957734 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fe2a415f6b0 0x7fe2a40cd7b0 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7fe29c009e50 tx=0x7fe29c01c9e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=900aba58a0778660 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:18:22.307 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:22.301+0000 7fe2a27fc640 1 -- 172.21.15.143:0/1621957734 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe29c0a58a0 con 0x7fe2a415f6b0 2023-11-26T07:18:22.307 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:22.301+0000 7fe2a27fc640 1 -- 172.21.15.143:0/1621957734 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe29c0a5a00 con 0x7fe2a415f6b0 2023-11-26T07:18:22.308 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:22.301+0000 7fe2a27fc640 1 -- 172.21.15.143:0/1621957734 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe29c0a5d90 con 0x7fe2a415f6b0 2023-11-26T07:18:22.308 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:22.301+0000 7fe2a9e4d640 1 -- 172.21.15.143:0/1621957734 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fe2a415f6b0 msgr2=0x7fe2a40cd7b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:18:22.308 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:22.301+0000 7fe2a9e4d640 1 --2- 172.21.15.143:0/1621957734 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fe2a415f6b0 0x7fe2a40cd7b0 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7fe29c009e50 tx=0x7fe29c01c9e0 comp rx=0 tx=0).stop 2023-11-26T07:18:22.308 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:22.301+0000 7fe2a9e4d640 1 -- 172.21.15.143:0/1621957734 shutdown_connections 2023-11-26T07:18:22.309 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:22.301+0000 7fe2a9e4d640 1 --2- 172.21.15.143:0/1621957734 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fe2a415f6b0 0x7fe2a40cd7b0 unknown :-1 s=CLOSED pgs=216 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:18:22.309 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:22.301+0000 7fe2a9e4d640 1 --2- 172.21.15.143:0/1621957734 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fe2a413e290 0x7fe2a40cf280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:18:22.309 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:22.301+0000 7fe2a9e4d640 1 -- 172.21.15.143:0/1621957734 >> 172.21.15.143:0/1621957734 conn(0x7fe2a406a890 msgr2=0x7fe2a406aca0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:18:22.309 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:22.301+0000 7fe2a9e4d640 1 -- 172.21.15.143:0/1621957734 shutdown_connections 2023-11-26T07:18:22.309 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:22.301+0000 7fe2a9e4d640 1 -- 172.21.15.143:0/1621957734 wait complete. 2023-11-26T07:18:22.310 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:22.305+0000 7fe2a9e4d640 1 Processor -- start 2023-11-26T07:18:22.310 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:22.305+0000 7fe2a9e4d640 1 -- start start 2023-11-26T07:18:22.312 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:22.305+0000 7fe2a9e4d640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fe2a40cf7c0 0x7fe2a4151710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:18:22.312 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:22.305+0000 7fe2a9e4d640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fe2a413e290 0x7fe2a4153fc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:18:22.313 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:22.305+0000 7fe2a9e4d640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fe2a415f6b0 0x7fe2a4135640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:18:22.313 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:22.305+0000 7fe2a9e4d640 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe2a4136000 con 0x7fe2a40cf7c0 2023-11-26T07:18:22.313 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:22.305+0000 7fe2a9e4d640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe2a4151c50 con 0x7fe2a413e290 2023-11-26T07:18:22.313 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:22.305+0000 7fe2a9e4d640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe2a41550b0 con 0x7fe2a415f6b0 2023-11-26T07:18:22.313 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:22.305+0000 7fe2a37fe640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fe2a40cf7c0 0x7fe2a4151710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:18:22.313 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:22.305+0000 7fe2a37fe640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fe2a40cf7c0 0x7fe2a4151710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:44088/0 (socket says 172.21.15.143:44088) 2023-11-26T07:18:22.313 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:22.305+0000 7fe2a37fe640 1 -- 172.21.15.143:0/1829046545 learned_addr learned my addr 172.21.15.143:0/1829046545 (peer_addr_for_me v2:172.21.15.143:0/0) 2023-11-26T07:18:22.313 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:22.305+0000 7fe2a2ffd640 1 --2- 172.21.15.143:0/1829046545 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fe2a413e290 0x7fe2a4153fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:18:22.313 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:22.305+0000 7fe2a3fff640 1 --2- 172.21.15.143:0/1829046545 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fe2a415f6b0 0x7fe2a4135640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:18:22.313 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:22.305+0000 7fe2a37fe640 1 -- 172.21.15.143:0/1829046545 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fe2a415f6b0 msgr2=0x7fe2a4135640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:18:22.313 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:22.305+0000 7fe2a37fe640 1 --2- 172.21.15.143:0/1829046545 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fe2a415f6b0 0x7fe2a4135640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:18:22.313 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:22.305+0000 7fe2a37fe640 1 -- 172.21.15.143:0/1829046545 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fe2a413e290 msgr2=0x7fe2a4153fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:18:22.313 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:22.305+0000 7fe2a37fe640 1 --2- 172.21.15.143:0/1829046545 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fe2a413e290 0x7fe2a4153fc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:18:22.314 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:22.305+0000 7fe2a37fe640 1 -- 172.21.15.143:0/1829046545 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe29c009b30 con 0x7fe2a40cf7c0 2023-11-26T07:18:22.314 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:22.305+0000 7fe2a37fe640 1 --2- 172.21.15.143:0/1829046545 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fe2a40cf7c0 0x7fe2a4151710 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7fe29400d800 tx=0x7fe29400dcc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:18:22.314 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:22.305+0000 7fe28bfff640 1 -- 172.21.15.143:0/1829046545 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe29400b6c0 con 0x7fe2a40cf7c0 2023-11-26T07:18:22.314 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:22.305+0000 7fe28bfff640 1 -- 172.21.15.143:0/1829046545 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe29400b820 con 0x7fe2a40cf7c0 2023-11-26T07:18:22.314 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:22.305+0000 7fe28bfff640 1 -- 172.21.15.143:0/1829046545 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe29400bb90 con 0x7fe2a40cf7c0 2023-11-26T07:18:22.314 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:22.305+0000 7fe2a9e4d640 1 -- 172.21.15.143:0/1829046545 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe2a4154810 con 0x7fe2a40cf7c0 2023-11-26T07:18:22.314 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:22.305+0000 7fe2a9e4d640 1 -- 172.21.15.143:0/1829046545 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe2a4154dd0 con 0x7fe2a40cf7c0 2023-11-26T07:18:22.314 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:22.309+0000 7fe2a9e4d640 1 -- 172.21.15.143:0/1829046545 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe2a40ccba0 con 0x7fe2a40cf7c0 2023-11-26T07:18:22.315 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:22.309+0000 7fe28bfff640 1 -- 172.21.15.143:0/1829046545 <== mon.0 v2:172.21.15.143:3300/0 4 ==== mgrmap(e 5) v1 ==== 85852+0+0 (secure 0 0 0) 0x7fe29400bcf0 con 0x7fe2a40cf7c0 2023-11-26T07:18:22.315 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:22.309+0000 7fe28bfff640 1 --2- 172.21.15.143:0/1829046545 >> [v2:172.21.15.143:6832/2150557544,v1:172.21.15.143:6833/2150557544] conn(0x7fe290066470 0x7fe290068930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:18:22.315 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:22.309+0000 7fe28bfff640 1 -- 172.21.15.143:0/1829046545 <== mon.0 v2:172.21.15.143:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fe294084290 con 0x7fe2a40cf7c0 2023-11-26T07:18:22.315 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:22.309+0000 7fe2a2ffd640 1 --2- 172.21.15.143:0/1829046545 >> [v2:172.21.15.143:6832/2150557544,v1:172.21.15.143:6833/2150557544] conn(0x7fe290066470 0x7fe290068930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:18:22.316 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:22.309+0000 7fe2a2ffd640 1 --2- 172.21.15.143:0/1829046545 >> [v2:172.21.15.143:6832/2150557544,v1:172.21.15.143:6833/2150557544] conn(0x7fe290066470 0x7fe290068930 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7fe2a415e740 tx=0x7fe298009040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:18:22.319 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:22.313+0000 7fe28bfff640 1 -- 172.21.15.143:0/1829046545 <== mon.0 v2:172.21.15.143:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172649 (secure 0 0 0) 0x7fe2940517b0 con 0x7fe2a40cf7c0 2023-11-26T07:18:22.470 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:22.465+0000 7fe2a9e4d640 1 -- 172.21.15.143:0/1829046545 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7fe2a4067680 con 0x7fe2a40cf7c0 2023-11-26T07:18:22.470 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:22.465+0000 7fe28bfff640 1 -- 172.21.15.143:0/1829046545 <== mon.0 v2:172.21.15.143:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v20) v1 ==== 93+0+10 (secure 0 0 0) 0x7fe294054dd0 con 0x7fe2a40cf7c0 2023-11-26T07:18:22.470 INFO:teuthology.orchestra.run.smithi143.stdout:pg_num: 1 2023-11-26T07:18:22.472 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:22.465+0000 7fe2a9e4d640 1 -- 172.21.15.143:0/1829046545 >> [v2:172.21.15.143:6832/2150557544,v1:172.21.15.143:6833/2150557544] conn(0x7fe290066470 msgr2=0x7fe290068930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:18:22.472 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:22.465+0000 7fe2a9e4d640 1 --2- 172.21.15.143:0/1829046545 >> [v2:172.21.15.143:6832/2150557544,v1:172.21.15.143:6833/2150557544] conn(0x7fe290066470 0x7fe290068930 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7fe2a415e740 tx=0x7fe298009040 comp rx=0 tx=0).stop 2023-11-26T07:18:22.472 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:22.465+0000 7fe2a9e4d640 1 -- 172.21.15.143:0/1829046545 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fe2a40cf7c0 msgr2=0x7fe2a4151710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:18:22.472 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:22.465+0000 7fe2a9e4d640 1 --2- 172.21.15.143:0/1829046545 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fe2a40cf7c0 0x7fe2a4151710 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7fe29400d800 tx=0x7fe29400dcc0 comp rx=0 tx=0).stop 2023-11-26T07:18:22.472 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:22.465+0000 7fe2a9e4d640 1 -- 172.21.15.143:0/1829046545 shutdown_connections 2023-11-26T07:18:22.472 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:22.465+0000 7fe2a9e4d640 1 --2- 172.21.15.143:0/1829046545 >> [v2:172.21.15.143:6832/2150557544,v1:172.21.15.143:6833/2150557544] conn(0x7fe290066470 0x7fe290068930 unknown :-1 s=CLOSED pgs=138 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:18:22.472 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:22.465+0000 7fe2a9e4d640 1 --2- 172.21.15.143:0/1829046545 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fe2a415f6b0 0x7fe2a4135640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:18:22.472 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:22.465+0000 7fe2a9e4d640 1 --2- 172.21.15.143:0/1829046545 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fe2a413e290 0x7fe2a4153fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:18:22.472 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:22.465+0000 7fe2a9e4d640 1 --2- 172.21.15.143:0/1829046545 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fe2a40cf7c0 0x7fe2a4151710 unknown :-1 s=CLOSED pgs=217 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:18:22.472 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:22.465+0000 7fe2a9e4d640 1 -- 172.21.15.143:0/1829046545 >> 172.21.15.143:0/1829046545 conn(0x7fe2a406a890 msgr2=0x7fe2a406ac70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:18:22.473 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:22.465+0000 7fe2a9e4d640 1 -- 172.21.15.143:0/1829046545 shutdown_connections 2023-11-26T07:18:22.473 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:22.465+0000 7fe2a9e4d640 1 -- 172.21.15.143:0/1829046545 wait complete. 2023-11-26T07:18:22.491 DEBUG:teuthology.orchestra.run.smithi143:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-11-26T07:18:22.635 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:22.629+0000 7effd6562640 1 Processor -- start 2023-11-26T07:18:22.635 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:22.629+0000 7effd6562640 1 -- start start 2023-11-26T07:18:22.636 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:22.629+0000 7effd6562640 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7effd0072540 0x7effd015e0b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:18:22.636 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:22.629+0000 7effd6562640 1 -- --> v1:172.21.15.143:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7effd0071780 con 0x7effd015aa50 2023-11-26T07:18:22.637 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:22.629+0000 7effd6562640 1 -- --> v1:172.21.15.188:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7effd0071a60 con 0x7effd015ae30 2023-11-26T07:18:22.637 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:22.629+0000 7effd6562640 1 -- --> v2:172.21.15.143:3300/0 -- mon_getmap magic: 0 v1 -- 0x7effd0071d40 con 0x7effd0072540 2023-11-26T07:18:22.637 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:22.629+0000 7effd5560640 1 --1- >> v1:172.21.15.143:6789/0 conn(0x7effd015aa50 0x7effd00add50 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.143:6789/0 says I am v1:172.21.15.143:43976/0 (socket says 172.21.15.143:43976) 2023-11-26T07:18:22.637 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:22.629+0000 7effd5560640 1 -- 172.21.15.143:0/3497197541 learned_addr learned my addr 172.21.15.143:0/3497197541 (peer_addr_for_me v1:172.21.15.143:0/0) 2023-11-26T07:18:22.637 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:22.629+0000 7effd5d61640 1 --2- 172.21.15.143:0/3497197541 >> v2:172.21.15.143:3300/0 conn(0x7effd0072540 0x7effd015e0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:18:22.637 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:22.629+0000 7effc7fff640 1 -- 172.21.15.143:0/3497197541 <== mon.0 v1:172.21.15.143:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2807068016 0 0) 0x7effd0071780 con 0x7effd015aa50 2023-11-26T07:18:22.637 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:22.629+0000 7effc7fff640 1 -- 172.21.15.143:0/3497197541 --> v1:172.21.15.143:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7effbc003530 con 0x7effd015aa50 2023-11-26T07:18:22.637 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:22.629+0000 7effd5d61640 1 -- 172.21.15.143:0/3497197541 >> v1:172.21.15.188:6789/0 conn(0x7effd015ae30 legacy=0x7effd015d9a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:18:22.637 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:22.629+0000 7effd5d61640 1 -- 172.21.15.143:0/3497197541 >> v1:172.21.15.143:6789/0 conn(0x7effd015aa50 legacy=0x7effd00add50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:18:22.637 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:22.629+0000 7effd5d61640 1 -- 172.21.15.143:0/3497197541 --> v2:172.21.15.143:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7effd0072020 con 0x7effd0072540 2023-11-26T07:18:22.639 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:22.633+0000 7effd5d61640 1 --2- 172.21.15.143:0/3497197541 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7effd0072540 0x7effd015e0b0 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7effc8009d90 tx=0x7effc801c8c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c98d1df37921a21e server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:18:22.639 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:22.633+0000 7effc7fff640 1 -- 172.21.15.143:0/3497197541 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7effc80a59a0 con 0x7effd0072540 2023-11-26T07:18:22.639 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:22.633+0000 7effc7fff640 1 -- 172.21.15.143:0/3497197541 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7effc80a5b00 con 0x7effd0072540 2023-11-26T07:18:22.639 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:22.633+0000 7effc7fff640 1 -- 172.21.15.143:0/3497197541 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7effc80a5e70 con 0x7effd0072540 2023-11-26T07:18:22.640 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:22.633+0000 7effd6562640 1 -- 172.21.15.143:0/3497197541 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7effd0072540 msgr2=0x7effd015e0b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:18:22.640 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:22.633+0000 7effd6562640 1 --2- 172.21.15.143:0/3497197541 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7effd0072540 0x7effd015e0b0 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7effc8009d90 tx=0x7effc801c8c0 comp rx=0 tx=0).stop 2023-11-26T07:18:22.640 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:22.633+0000 7effd6562640 1 -- 172.21.15.143:0/3497197541 shutdown_connections 2023-11-26T07:18:22.640 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:22.633+0000 7effd6562640 1 --2- 172.21.15.143:0/3497197541 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7effd0072540 0x7effd015e0b0 unknown :-1 s=CLOSED pgs=219 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:18:22.640 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:22.633+0000 7effd6562640 1 -- 172.21.15.143:0/3497197541 >> 172.21.15.143:0/3497197541 conn(0x7effd006a890 msgr2=0x7effd006aca0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:18:22.641 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:22.633+0000 7effd6562640 1 -- 172.21.15.143:0/3497197541 shutdown_connections 2023-11-26T07:18:22.641 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:22.633+0000 7effd6562640 1 -- 172.21.15.143:0/3497197541 wait complete. 2023-11-26T07:18:22.641 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:22.637+0000 7effd6562640 1 Processor -- start 2023-11-26T07:18:22.641 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:22.637+0000 7effd6562640 1 -- start start 2023-11-26T07:18:22.643 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:22.637+0000 7effd6562640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7effd015aa50 0x7effd015b700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:18:22.643 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:22.637+0000 7effd6562640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7effd015ae30 0x7effd00ae880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:18:22.643 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:22.637+0000 7effd6562640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7effd00aedc0 0x7effd0159530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:18:22.644 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:22.637+0000 7effd6562640 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x7effd013fe90 con 0x7effd00aedc0 2023-11-26T07:18:22.644 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:22.637+0000 7effd6562640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7effd00b04e0 con 0x7effd015ae30 2023-11-26T07:18:22.644 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:22.637+0000 7effd6562640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7effd00af440 con 0x7effd015aa50 2023-11-26T07:18:22.644 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:22.637+0000 7effd5d61640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7effd00aedc0 0x7effd0159530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:18:22.644 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:22.637+0000 7effd5d61640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7effd00aedc0 0x7effd0159530 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:44110/0 (socket says 172.21.15.143:44110) 2023-11-26T07:18:22.644 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:22.637+0000 7effd5d61640 1 -- 172.21.15.143:0/2653800765 learned_addr learned my addr 172.21.15.143:0/2653800765 (peer_addr_for_me v2:172.21.15.143:0/0) 2023-11-26T07:18:22.644 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:22.637+0000 7effd4d5f640 1 --2- 172.21.15.143:0/2653800765 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7effd015ae30 0x7effd00ae880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:18:22.644 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:22.637+0000 7effd5560640 1 --2- 172.21.15.143:0/2653800765 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7effd015aa50 0x7effd015b700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:18:22.644 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:22.637+0000 7effd5d61640 1 -- 172.21.15.143:0/2653800765 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7effd015aa50 msgr2=0x7effd015b700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:18:22.644 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:22.637+0000 7effd5d61640 1 --2- 172.21.15.143:0/2653800765 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7effd015aa50 0x7effd015b700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:18:22.644 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:22.637+0000 7effd5d61640 1 -- 172.21.15.143:0/2653800765 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7effd015ae30 msgr2=0x7effd00ae880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:18:22.644 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:22.637+0000 7effd5d61640 1 --2- 172.21.15.143:0/2653800765 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7effd015ae30 0x7effd00ae880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:18:22.644 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:22.637+0000 7effd5d61640 1 -- 172.21.15.143:0/2653800765 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7effc8009a70 con 0x7effd00aedc0 2023-11-26T07:18:22.645 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:22.637+0000 7effd4d5f640 1 --2- 172.21.15.143:0/2653800765 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7effd015ae30 0x7effd00ae880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-26T07:18:22.645 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:22.637+0000 7effd5560640 1 --2- 172.21.15.143:0/2653800765 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7effd015aa50 0x7effd015b700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-26T07:18:22.645 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:22.637+0000 7effd5d61640 1 --2- 172.21.15.143:0/2653800765 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7effd00aedc0 0x7effd0159530 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7effc80194d0 tx=0x7effc8019660 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:18:22.645 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:22.637+0000 7effc5ffb640 1 -- 172.21.15.143:0/2653800765 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7effc80a59a0 con 0x7effd00aedc0 2023-11-26T07:18:22.646 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:22.637+0000 7effc5ffb640 1 -- 172.21.15.143:0/2653800765 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7effc8009450 con 0x7effd00aedc0 2023-11-26T07:18:22.646 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:22.637+0000 7effc5ffb640 1 -- 172.21.15.143:0/2653800765 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7effc80aed80 con 0x7effd00aedc0 2023-11-26T07:18:22.646 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:22.637+0000 7effd6562640 1 -- 172.21.15.143:0/2653800765 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7effd00af720 con 0x7effd00aedc0 2023-11-26T07:18:22.646 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:22.637+0000 7effd6562640 1 -- 172.21.15.143:0/2653800765 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7effd00afce0 con 0x7effd00aedc0 2023-11-26T07:18:22.646 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:22.641+0000 7effd6562640 1 -- 172.21.15.143:0/2653800765 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7effd0067500 con 0x7effd00aedc0 2023-11-26T07:18:22.646 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:22.641+0000 7effc5ffb640 1 -- 172.21.15.143:0/2653800765 <== mon.0 v2:172.21.15.143:3300/0 4 ==== mgrmap(e 5) v1 ==== 85852+0+0 (secure 0 0 0) 0x7effc80b4050 con 0x7effd00aedc0 2023-11-26T07:18:22.647 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:22.641+0000 7effc5ffb640 1 --2- 172.21.15.143:0/2653800765 >> [v2:172.21.15.143:6832/2150557544,v1:172.21.15.143:6833/2150557544] conn(0x7effb4066470 0x7effb4068930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:18:22.647 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:22.641+0000 7effc5ffb640 1 -- 172.21.15.143:0/2653800765 <== mon.0 v2:172.21.15.143:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5709+0+0 (secure 0 0 0) 0x7effc811f140 con 0x7effd00aedc0 2023-11-26T07:18:22.647 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:22.641+0000 7effd5560640 1 --2- 172.21.15.143:0/2653800765 >> [v2:172.21.15.143:6832/2150557544,v1:172.21.15.143:6833/2150557544] conn(0x7effb4066470 0x7effb4068930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:18:22.647 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:22.641+0000 7effd5560640 1 --2- 172.21.15.143:0/2653800765 >> [v2:172.21.15.143:6832/2150557544,v1:172.21.15.143:6833/2150557544] conn(0x7effb4066470 0x7effb4068930 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7effcc0011f0 tx=0x7effcc009040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:18:22.651 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:22.645+0000 7effc5ffb640 1 -- 172.21.15.143:0/2653800765 <== mon.0 v2:172.21.15.143:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172649 (secure 0 0 0) 0x7effc80ec640 con 0x7effd00aedc0 2023-11-26T07:18:22.801 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:22.793+0000 7effd6562640 1 -- 172.21.15.143:0/2653800765 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7effd00ceaa0 con 0x7effd00aedc0 2023-11-26T07:18:22.801 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:22.797+0000 7effc5ffb640 1 -- 172.21.15.143:0/2653800765 <== mon.0 v2:172.21.15.143:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v20) v1 ==== 104+0+11 (secure 0 0 0) 0x7effc80efc60 con 0x7effd00aedc0 2023-11-26T07:18:22.801 INFO:teuthology.orchestra.run.smithi143.stdout:pg_num: 32 2023-11-26T07:18:22.803 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:22.797+0000 7effd6562640 1 -- 172.21.15.143:0/2653800765 >> [v2:172.21.15.143:6832/2150557544,v1:172.21.15.143:6833/2150557544] conn(0x7effb4066470 msgr2=0x7effb4068930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:18:22.803 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:22.797+0000 7effd6562640 1 --2- 172.21.15.143:0/2653800765 >> [v2:172.21.15.143:6832/2150557544,v1:172.21.15.143:6833/2150557544] conn(0x7effb4066470 0x7effb4068930 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7effcc0011f0 tx=0x7effcc009040 comp rx=0 tx=0).stop 2023-11-26T07:18:22.803 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:22.797+0000 7effd6562640 1 -- 172.21.15.143:0/2653800765 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7effd00aedc0 msgr2=0x7effd0159530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:18:22.803 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:22.797+0000 7effd6562640 1 --2- 172.21.15.143:0/2653800765 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7effd00aedc0 0x7effd0159530 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7effc80194d0 tx=0x7effc8019660 comp rx=0 tx=0).stop 2023-11-26T07:18:22.804 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:22.797+0000 7effd6562640 1 -- 172.21.15.143:0/2653800765 shutdown_connections 2023-11-26T07:18:22.804 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:22.797+0000 7effd6562640 1 --2- 172.21.15.143:0/2653800765 >> [v2:172.21.15.143:6832/2150557544,v1:172.21.15.143:6833/2150557544] conn(0x7effb4066470 0x7effb4068930 unknown :-1 s=CLOSED pgs=139 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:18:22.804 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:22.797+0000 7effd6562640 1 --2- 172.21.15.143:0/2653800765 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7effd00aedc0 0x7effd0159530 unknown :-1 s=CLOSED pgs=220 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:18:22.804 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:22.797+0000 7effd6562640 1 --2- 172.21.15.143:0/2653800765 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7effd015ae30 0x7effd00ae880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:18:22.804 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:22.797+0000 7effd6562640 1 --2- 172.21.15.143:0/2653800765 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7effd015aa50 0x7effd015b700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:18:22.804 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:22.797+0000 7effd6562640 1 -- 172.21.15.143:0/2653800765 >> 172.21.15.143:0/2653800765 conn(0x7effd006a890 msgr2=0x7effd006c850 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:18:22.804 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:22.797+0000 7effd6562640 1 -- 172.21.15.143:0/2653800765 shutdown_connections 2023-11-26T07:18:22.804 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:22.797+0000 7effd6562640 1 -- 172.21.15.143:0/2653800765 wait complete. 2023-11-26T07:18:22.822 DEBUG:teuthology.orchestra.run.smithi143:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-11-26T07:18:22.962 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:22.957+0000 7fe616dd1640 1 Processor -- start 2023-11-26T07:18:22.962 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:22.957+0000 7fe616dd1640 1 -- start start 2023-11-26T07:18:22.963 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:22.957+0000 7fe616dd1640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fe6100cf710 0x7fe6100cd8c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:18:22.964 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:22.957+0000 7fe616dd1640 1 -- --> v1:172.21.15.143:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe6100cc730 con 0x7fe610143ac0 2023-11-26T07:18:22.964 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:22.957+0000 7fe616dd1640 1 -- --> v1:172.21.15.188:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe610070ee0 con 0x7fe6100cf280 2023-11-26T07:18:22.964 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:22.957+0000 7fe616dd1640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe610143ea0 con 0x7fe6100cf710 2023-11-26T07:18:22.964 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:22.957+0000 7fe614b6d640 1 --1- >> v1:172.21.15.143:6789/0 conn(0x7fe610143ac0 0x7fe610070370 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.143:6789/0 says I am v1:172.21.15.143:43984/0 (socket says 172.21.15.143:43984) 2023-11-26T07:18:22.964 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:22.957+0000 7fe614b6d640 1 -- 172.21.15.143:0/3451541955 learned_addr learned my addr 172.21.15.143:0/3451541955 (peer_addr_for_me v1:172.21.15.143:0/0) 2023-11-26T07:18:22.964 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:22.957+0000 7fe61536e640 1 --2- 172.21.15.143:0/3451541955 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fe6100cf710 0x7fe6100cd8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:18:22.964 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:22.957+0000 7fe60f7fe640 1 -- 172.21.15.143:0/3451541955 <== mon.0 v1:172.21.15.143:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3262001199 0 0) 0x7fe6100cc730 con 0x7fe610143ac0 2023-11-26T07:18:22.964 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:22.957+0000 7fe60f7fe640 1 -- 172.21.15.143:0/3451541955 --> v1:172.21.15.143:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fe600005740 con 0x7fe610143ac0 2023-11-26T07:18:22.964 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:22.957+0000 7fe60f7fe640 1 -- 172.21.15.143:0/3451541955 <== mon.1 v1:172.21.15.188:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4020687725 0 0) 0x7fe610070ee0 con 0x7fe6100cf280 2023-11-26T07:18:22.964 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:22.957+0000 7fe60f7fe640 1 -- 172.21.15.143:0/3451541955 --> v1:172.21.15.188:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fe6100cc730 con 0x7fe6100cf280 2023-11-26T07:18:22.964 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:22.957+0000 7fe61536e640 1 -- 172.21.15.143:0/3451541955 >> v1:172.21.15.188:6789/0 conn(0x7fe6100cf280 legacy=0x7fe6100cd1b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:18:22.964 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:22.957+0000 7fe60f7fe640 1 -- 172.21.15.143:0/3451541955 <== mon.0 v1:172.21.15.143:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3309566315 0 0) 0x7fe600005740 con 0x7fe610143ac0 2023-11-26T07:18:22.964 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:22.957+0000 7fe61536e640 1 -- 172.21.15.143:0/3451541955 >> v1:172.21.15.143:6789/0 conn(0x7fe610143ac0 legacy=0x7fe610070370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:18:22.965 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:22.957+0000 7fe61536e640 1 -- 172.21.15.143:0/3451541955 --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe6100711c0 con 0x7fe6100cf710 2023-11-26T07:18:22.966 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:22.961+0000 7fe61536e640 1 --2- 172.21.15.143:0/3451541955 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fe6100cf710 0x7fe6100cd8c0 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7fe608009dc0 tx=0x7fe60801c890 comp rx=0 tx=0).ready entity=mon.2 client_cookie=4b75cfde196e042c server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:18:22.967 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:22.961+0000 7fe60f7fe640 1 -- 172.21.15.143:0/3451541955 <== mon.2 v2:172.21.15.188:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe6040016a0 con 0x7fe6100cf710 2023-11-26T07:18:22.967 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:22.961+0000 7fe60f7fe640 1 -- 172.21.15.143:0/3451541955 <== mon.2 v2:172.21.15.188:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe6080a56d0 con 0x7fe6100cf710 2023-11-26T07:18:22.967 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:22.961+0000 7fe60f7fe640 1 -- 172.21.15.143:0/3451541955 <== mon.2 v2:172.21.15.188:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe6080a5a60 con 0x7fe6100cf710 2023-11-26T07:18:22.968 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:22.961+0000 7fe616dd1640 1 -- 172.21.15.143:0/3451541955 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fe6100cf710 msgr2=0x7fe6100cd8c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:18:22.968 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:22.961+0000 7fe616dd1640 1 --2- 172.21.15.143:0/3451541955 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fe6100cf710 0x7fe6100cd8c0 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7fe608009dc0 tx=0x7fe60801c890 comp rx=0 tx=0).stop 2023-11-26T07:18:22.968 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:22.961+0000 7fe616dd1640 1 -- 172.21.15.143:0/3451541955 shutdown_connections 2023-11-26T07:18:22.968 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:22.961+0000 7fe616dd1640 1 --2- 172.21.15.143:0/3451541955 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fe6100cf710 0x7fe6100cd8c0 unknown :-1 s=CLOSED pgs=59 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:18:22.968 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:22.961+0000 7fe616dd1640 1 -- 172.21.15.143:0/3451541955 >> 172.21.15.143:0/3451541955 conn(0x7fe61006a890 msgr2=0x7fe61006aca0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:18:22.968 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:22.961+0000 7fe616dd1640 1 -- 172.21.15.143:0/3451541955 shutdown_connections 2023-11-26T07:18:22.968 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:22.961+0000 7fe616dd1640 1 -- 172.21.15.143:0/3451541955 wait complete. 2023-11-26T07:18:22.969 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:22.965+0000 7fe616dd1640 1 Processor -- start 2023-11-26T07:18:22.969 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:22.965+0000 7fe616dd1640 1 -- start start 2023-11-26T07:18:22.971 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:22.965+0000 7fe616dd1640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fe6100cf280 0x7fe6101348c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:18:22.971 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:22.965+0000 7fe616dd1640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fe6100cf710 0x7fe610159bb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:18:22.971 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:22.965+0000 7fe616dd1640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fe610143ac0 0x7fe6100aeb10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:18:22.971 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:22.965+0000 7fe616dd1640 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe6101350d0 con 0x7fe610143ac0 2023-11-26T07:18:22.971 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:22.965+0000 7fe616dd1640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe610137640 con 0x7fe6100cf280 2023-11-26T07:18:22.971 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:22.965+0000 7fe616dd1640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe610134e00 con 0x7fe6100cf710 2023-11-26T07:18:22.971 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:22.965+0000 7fe61536e640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fe610143ac0 0x7fe6100aeb10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:18:22.972 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:22.965+0000 7fe61536e640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fe610143ac0 0x7fe6100aeb10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:44122/0 (socket says 172.21.15.143:44122) 2023-11-26T07:18:22.972 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:22.965+0000 7fe61536e640 1 -- 172.21.15.143:0/1477925352 learned_addr learned my addr 172.21.15.143:0/1477925352 (peer_addr_for_me v2:172.21.15.143:0/0) 2023-11-26T07:18:22.972 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:22.965+0000 7fe614b6d640 1 --2- 172.21.15.143:0/1477925352 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fe6100cf280 0x7fe6101348c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:18:22.972 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:22.965+0000 7fe60ffff640 1 --2- 172.21.15.143:0/1477925352 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fe6100cf710 0x7fe610159bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:18:22.972 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:22.965+0000 7fe61536e640 1 -- 172.21.15.143:0/1477925352 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fe6100cf710 msgr2=0x7fe610159bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:18:22.972 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:22.965+0000 7fe61536e640 1 --2- 172.21.15.143:0/1477925352 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fe6100cf710 0x7fe610159bb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:18:22.972 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:22.965+0000 7fe61536e640 1 -- 172.21.15.143:0/1477925352 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fe6100cf280 msgr2=0x7fe6101348c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:18:22.972 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:22.965+0000 7fe61536e640 1 --2- 172.21.15.143:0/1477925352 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fe6100cf280 0x7fe6101348c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:18:22.972 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:22.965+0000 7fe61536e640 1 -- 172.21.15.143:0/1477925352 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe608009a70 con 0x7fe610143ac0 2023-11-26T07:18:22.973 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:22.965+0000 7fe61536e640 1 --2- 172.21.15.143:0/1477925352 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fe610143ac0 0x7fe6100aeb10 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7fe60801af40 tx=0x7fe6100ce610 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:18:22.973 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:22.965+0000 7fe60d7fa640 1 -- 172.21.15.143:0/1477925352 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe6080a5ea0 con 0x7fe610143ac0 2023-11-26T07:18:22.973 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:22.965+0000 7fe60d7fa640 1 -- 172.21.15.143:0/1477925352 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe608003ba0 con 0x7fe610143ac0 2023-11-26T07:18:22.973 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:22.965+0000 7fe616dd1640 1 -- 172.21.15.143:0/1477925352 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe610137850 con 0x7fe610143ac0 2023-11-26T07:18:22.973 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:22.965+0000 7fe60d7fa640 1 -- 172.21.15.143:0/1477925352 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe608009450 con 0x7fe610143ac0 2023-11-26T07:18:22.973 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:22.965+0000 7fe616dd1640 1 -- 172.21.15.143:0/1477925352 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe610137e10 con 0x7fe610143ac0 2023-11-26T07:18:22.973 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:22.965+0000 7fe616dd1640 1 -- 172.21.15.143:0/1477925352 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe610067680 con 0x7fe610143ac0 2023-11-26T07:18:22.974 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:22.969+0000 7fe60d7fa640 1 -- 172.21.15.143:0/1477925352 <== mon.0 v2:172.21.15.143:3300/0 4 ==== mgrmap(e 5) v1 ==== 85852+0+0 (secure 0 0 0) 0x7fe6080b4040 con 0x7fe610143ac0 2023-11-26T07:18:22.974 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:22.969+0000 7fe60d7fa640 1 --2- 172.21.15.143:0/1477925352 >> [v2:172.21.15.143:6832/2150557544,v1:172.21.15.143:6833/2150557544] conn(0x7fe5f4066470 0x7fe5f4068930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:18:22.974 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:22.969+0000 7fe60d7fa640 1 -- 172.21.15.143:0/1477925352 <== mon.0 v2:172.21.15.143:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fe60811d7a0 con 0x7fe610143ac0 2023-11-26T07:18:22.975 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:22.969+0000 7fe614b6d640 1 --2- 172.21.15.143:0/1477925352 >> [v2:172.21.15.143:6832/2150557544,v1:172.21.15.143:6833/2150557544] conn(0x7fe5f4066470 0x7fe5f4068930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:18:22.975 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:22.969+0000 7fe614b6d640 1 --2- 172.21.15.143:0/1477925352 >> [v2:172.21.15.143:6832/2150557544,v1:172.21.15.143:6833/2150557544] conn(0x7fe5f4066470 0x7fe5f4068930 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7fe604000fc0 tx=0x7fe604001b10 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:18:22.978 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:22.973+0000 7fe60d7fa640 1 -- 172.21.15.143:0/1477925352 <== mon.0 v2:172.21.15.143:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172649 (secure 0 0 0) 0x7fe6080eacf0 con 0x7fe610143ac0 2023-11-26T07:18:23.127 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:23.121+0000 7fe616dd1640 1 -- 172.21.15.143:0/1477925352 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7fe610072450 con 0x7fe610143ac0 2023-11-26T07:18:23.128 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:23.121+0000 7fe60d7fa640 1 -- 172.21.15.143:0/1477925352 <== mon.0 v2:172.21.15.143:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v20) v1 ==== 100+0+11 (secure 0 0 0) 0x7fe6080ee310 con 0x7fe610143ac0 2023-11-26T07:18:23.128 INFO:teuthology.orchestra.run.smithi143.stdout:pg_num: 64 2023-11-26T07:18:23.129 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:23.125+0000 7fe616dd1640 1 -- 172.21.15.143:0/1477925352 >> [v2:172.21.15.143:6832/2150557544,v1:172.21.15.143:6833/2150557544] conn(0x7fe5f4066470 msgr2=0x7fe5f4068930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:18:23.129 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:23.125+0000 7fe616dd1640 1 --2- 172.21.15.143:0/1477925352 >> [v2:172.21.15.143:6832/2150557544,v1:172.21.15.143:6833/2150557544] conn(0x7fe5f4066470 0x7fe5f4068930 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7fe604000fc0 tx=0x7fe604001b10 comp rx=0 tx=0).stop 2023-11-26T07:18:23.129 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:23.125+0000 7fe616dd1640 1 -- 172.21.15.143:0/1477925352 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fe610143ac0 msgr2=0x7fe6100aeb10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:18:23.130 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:23.125+0000 7fe616dd1640 1 --2- 172.21.15.143:0/1477925352 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fe610143ac0 0x7fe6100aeb10 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7fe60801af40 tx=0x7fe6100ce610 comp rx=0 tx=0).stop 2023-11-26T07:18:23.130 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:23.125+0000 7fe616dd1640 1 -- 172.21.15.143:0/1477925352 shutdown_connections 2023-11-26T07:18:23.130 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:23.125+0000 7fe616dd1640 1 --2- 172.21.15.143:0/1477925352 >> [v2:172.21.15.143:6832/2150557544,v1:172.21.15.143:6833/2150557544] conn(0x7fe5f4066470 0x7fe5f4068930 unknown :-1 s=CLOSED pgs=140 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:18:23.130 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:23.125+0000 7fe616dd1640 1 --2- 172.21.15.143:0/1477925352 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fe610143ac0 0x7fe6100aeb10 unknown :-1 s=CLOSED pgs=222 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:18:23.130 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:23.125+0000 7fe616dd1640 1 --2- 172.21.15.143:0/1477925352 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fe6100cf710 0x7fe610159bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:18:23.130 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:23.125+0000 7fe616dd1640 1 --2- 172.21.15.143:0/1477925352 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fe6100cf280 0x7fe6101348c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:18:23.130 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:23.125+0000 7fe616dd1640 1 -- 172.21.15.143:0/1477925352 >> 172.21.15.143:0/1477925352 conn(0x7fe61006a890 msgr2=0x7fe61015a590 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:18:23.131 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:23.125+0000 7fe616dd1640 1 -- 172.21.15.143:0/1477925352 shutdown_connections 2023-11-26T07:18:23.131 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:23.125+0000 7fe616dd1640 1 -- 172.21.15.143:0/1477925352 wait complete. 2023-11-26T07:18:23.149 DEBUG:teuthology.orchestra.run.smithi143:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Starting test tasks.cephfs.test_volumes.TestMisc.test_binary_metafile_on_legacy_to_v1_upgrade' 2023-11-26T07:18:23.287 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:23.281+0000 7facba006640 1 Processor -- start 2023-11-26T07:18:23.288 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:23.281+0000 7facba006640 1 -- start start 2023-11-26T07:18:23.289 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:23.281+0000 7facba006640 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7facb40ce7c0 0x7facb415f6b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:18:23.289 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:23.281+0000 7facba006640 1 -- --> v1:172.21.15.143:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7facb406c520 con 0x7facb40cf990 2023-11-26T07:18:23.289 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:23.281+0000 7facba006640 1 -- --> v1:172.21.15.188:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7facb406c800 con 0x7facb413e290 2023-11-26T07:18:23.289 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:23.285+0000 7facba006640 1 -- --> v2:172.21.15.188:3300/0 -- mon_getmap magic: 0 v1 -- 0x7facb406cae0 con 0x7facb40ce7c0 2023-11-26T07:18:23.289 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:23.285+0000 7facb2ffd640 1 --1- >> v1:172.21.15.143:6789/0 conn(0x7facb40cf990 0x7facb415efa0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.143:6789/0 says I am v1:172.21.15.143:43990/0 (socket says 172.21.15.143:43990) 2023-11-26T07:18:23.289 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:23.285+0000 7facb2ffd640 1 -- 172.21.15.143:0/4081434823 learned_addr learned my addr 172.21.15.143:0/4081434823 (peer_addr_for_me v1:172.21.15.143:0/0) 2023-11-26T07:18:23.290 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:23.285+0000 7facb3fff640 1 --2- 172.21.15.143:0/4081434823 >> v2:172.21.15.188:3300/0 conn(0x7facb40ce7c0 0x7facb415f6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:18:23.290 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:23.285+0000 7facb27fc640 1 -- 172.21.15.143:0/4081434823 <== mon.0 v1:172.21.15.143:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1767840019 0 0) 0x7facb406c520 con 0x7facb40cf990 2023-11-26T07:18:23.290 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:23.285+0000 7facb27fc640 1 -- 172.21.15.143:0/4081434823 --> v1:172.21.15.143:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fac9c003530 con 0x7facb40cf990 2023-11-26T07:18:23.291 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:23.285+0000 7facb27fc640 1 -- 172.21.15.143:0/4081434823 <== mon.1 v1:172.21.15.188:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2413964050 0 0) 0x7facb406c800 con 0x7facb413e290 2023-11-26T07:18:23.291 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:23.285+0000 7facb27fc640 1 -- 172.21.15.143:0/4081434823 --> v1:172.21.15.188:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7facb406c520 con 0x7facb413e290 2023-11-26T07:18:23.291 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:23.285+0000 7facb3fff640 1 -- 172.21.15.143:0/4081434823 >> v1:172.21.15.188:6789/0 conn(0x7facb413e290 legacy=0x7facb40cf280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:18:23.291 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:23.285+0000 7facb3fff640 1 -- 172.21.15.143:0/4081434823 >> v1:172.21.15.143:6789/0 conn(0x7facb40cf990 legacy=0x7facb415efa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:18:23.291 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:23.285+0000 7facb3fff640 1 -- 172.21.15.143:0/4081434823 --> v2:172.21.15.188:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7facb406cdc0 con 0x7facb40ce7c0 2023-11-26T07:18:23.292 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:23.285+0000 7facb3fff640 1 --2- 172.21.15.143:0/4081434823 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7facb40ce7c0 0x7facb415f6b0 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7facac009d90 tx=0x7facac01c890 comp rx=0 tx=0).ready entity=mon.1 client_cookie=5954db3bbeb87593 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:18:23.293 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:23.289+0000 7facb27fc640 1 -- 172.21.15.143:0/4081434823 <== mon.1 v2:172.21.15.188:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7facac0a5970 con 0x7facb40ce7c0 2023-11-26T07:18:23.293 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:23.289+0000 7facb27fc640 1 -- 172.21.15.143:0/4081434823 <== mon.1 v2:172.21.15.188:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7facac0a5ad0 con 0x7facb40ce7c0 2023-11-26T07:18:23.293 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:23.289+0000 7facb27fc640 1 -- 172.21.15.143:0/4081434823 <== mon.1 v2:172.21.15.188:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7facac0a5e60 con 0x7facb40ce7c0 2023-11-26T07:18:23.294 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:23.289+0000 7facba006640 1 -- 172.21.15.143:0/4081434823 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7facb40ce7c0 msgr2=0x7facb415f6b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:18:23.294 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:23.289+0000 7facba006640 1 --2- 172.21.15.143:0/4081434823 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7facb40ce7c0 0x7facb415f6b0 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7facac009d90 tx=0x7facac01c890 comp rx=0 tx=0).stop 2023-11-26T07:18:23.294 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:23.289+0000 7facba006640 1 -- 172.21.15.143:0/4081434823 shutdown_connections 2023-11-26T07:18:23.294 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:23.289+0000 7facba006640 1 --2- 172.21.15.143:0/4081434823 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7facb40ce7c0 0x7facb415f6b0 unknown :-1 s=CLOSED pgs=123 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:18:23.294 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:23.289+0000 7facba006640 1 -- 172.21.15.143:0/4081434823 >> 172.21.15.143:0/4081434823 conn(0x7facb406a890 msgr2=0x7facb406aca0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:18:23.295 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:23.289+0000 7facba006640 1 -- 172.21.15.143:0/4081434823 shutdown_connections 2023-11-26T07:18:23.295 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:23.289+0000 7facba006640 1 -- 172.21.15.143:0/4081434823 wait complete. 2023-11-26T07:18:23.295 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:23.289+0000 7facba006640 1 Processor -- start 2023-11-26T07:18:23.295 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:23.289+0000 7facba006640 1 -- start start 2023-11-26T07:18:23.297 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:23.289+0000 7facba006640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7facb40ce7c0 0x7facb415bf20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:18:23.298 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:23.289+0000 7facba006640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7facb40cf990 0x7facb40b0d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:18:23.298 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:23.289+0000 7facba006640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7facb413e290 0x7facb41f53b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:18:23.298 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:23.289+0000 7facba006640 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x7facb40b08f0 con 0x7facb413e290 2023-11-26T07:18:23.298 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:23.289+0000 7facba006640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7facb40afce0 con 0x7facb40ce7c0 2023-11-26T07:18:23.298 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:23.289+0000 7facba006640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7facb40affc0 con 0x7facb40cf990 2023-11-26T07:18:23.298 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:23.289+0000 7facb3fff640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7facb413e290 0x7facb41f53b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:18:23.298 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:23.289+0000 7facb3fff640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7facb413e290 0x7facb41f53b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:44138/0 (socket says 172.21.15.143:44138) 2023-11-26T07:18:23.298 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:23.289+0000 7facb3fff640 1 -- 172.21.15.143:0/872094910 learned_addr learned my addr 172.21.15.143:0/872094910 (peer_addr_for_me v2:172.21.15.143:0/0) 2023-11-26T07:18:23.299 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:23.289+0000 7facb2ffd640 1 --2- 172.21.15.143:0/872094910 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7facb40cf990 0x7facb40b0d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:18:23.299 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:23.289+0000 7facb37fe640 1 --2- 172.21.15.143:0/872094910 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7facb40ce7c0 0x7facb415bf20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:18:23.299 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:23.293+0000 7facb3fff640 1 -- 172.21.15.143:0/872094910 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7facb40cf990 msgr2=0x7facb40b0d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:18:23.299 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:23.293+0000 7facb3fff640 1 --2- 172.21.15.143:0/872094910 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7facb40cf990 0x7facb40b0d80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:18:23.299 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:23.293+0000 7facb3fff640 1 -- 172.21.15.143:0/872094910 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7facb40ce7c0 msgr2=0x7facb415bf20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:18:23.299 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:23.293+0000 7facb3fff640 1 --2- 172.21.15.143:0/872094910 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7facb40ce7c0 0x7facb415bf20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:18:23.299 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:23.293+0000 7facb3fff640 1 -- 172.21.15.143:0/872094910 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7facac009a70 con 0x7facb413e290 2023-11-26T07:18:23.299 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:23.293+0000 7facb2ffd640 1 --2- 172.21.15.143:0/872094910 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7facb40cf990 0x7facb40b0d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-26T07:18:23.300 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:23.293+0000 7facb37fe640 1 --2- 172.21.15.143:0/872094910 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7facb40ce7c0 0x7facb415bf20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-26T07:18:23.300 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:23.293+0000 7facb3fff640 1 --2- 172.21.15.143:0/872094910 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7facb413e290 0x7facb41f53b0 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7facac0b0040 tx=0x7facac019630 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:18:23.300 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:23.293+0000 7fac9bfff640 1 -- 172.21.15.143:0/872094910 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7facac0a5970 con 0x7facb413e290 2023-11-26T07:18:23.300 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:23.293+0000 7fac9bfff640 1 -- 172.21.15.143:0/872094910 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7facac009450 con 0x7facb413e290 2023-11-26T07:18:23.300 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:23.293+0000 7facba006640 1 -- 172.21.15.143:0/872094910 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7facb40b02a0 con 0x7facb413e290 2023-11-26T07:18:23.300 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:23.293+0000 7fac9bfff640 1 -- 172.21.15.143:0/872094910 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7facac0b9690 con 0x7facb413e290 2023-11-26T07:18:23.300 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:23.293+0000 7facba006640 1 -- 172.21.15.143:0/872094910 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7facb406ae40 con 0x7facb413e290 2023-11-26T07:18:23.301 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:23.293+0000 7facba006640 1 -- 172.21.15.143:0/872094910 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7facb40b0a60 con 0x7facb413e290 2023-11-26T07:18:23.301 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:23.293+0000 7fac9bfff640 1 -- 172.21.15.143:0/872094910 <== mon.0 v2:172.21.15.143:3300/0 4 ==== mgrmap(e 5) v1 ==== 85852+0+0 (secure 0 0 0) 0x7facac0b97f0 con 0x7facb413e290 2023-11-26T07:18:23.301 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:23.293+0000 7fac9bfff640 1 --2- 172.21.15.143:0/872094910 >> [v2:172.21.15.143:6832/2150557544,v1:172.21.15.143:6833/2150557544] conn(0x7faca0066470 0x7faca0068930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:18:23.301 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:23.293+0000 7fac9bfff640 1 -- 172.21.15.143:0/872094910 <== mon.0 v2:172.21.15.143:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5709+0+0 (secure 0 0 0) 0x7facac11f790 con 0x7facb413e290 2023-11-26T07:18:23.302 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:23.293+0000 7facb37fe640 1 --2- 172.21.15.143:0/872094910 >> [v2:172.21.15.143:6832/2150557544,v1:172.21.15.143:6833/2150557544] conn(0x7faca0066470 0x7faca0068930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:18:23.302 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:23.297+0000 7facb37fe640 1 --2- 172.21.15.143:0/872094910 >> [v2:172.21.15.143:6832/2150557544,v1:172.21.15.143:6833/2150557544] conn(0x7faca0066470 0x7faca0068930 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7faca40011f0 tx=0x7faca4009040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:18:23.304 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:23.297+0000 7fac9bfff640 1 -- 172.21.15.143:0/872094910 <== mon.0 v2:172.21.15.143:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172649 (secure 0 0 0) 0x7facac0ecc00 con 0x7facb413e290 2023-11-26T07:18:23.507 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:23.501+0000 7facba006640 1 -- 172.21.15.143:0/872094910 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_volumes.TestMisc.test_binary_metafile_on_legacy_to_v1_upgrade"]} v 0) v1 -- 0x7facb40b0100 con 0x7facb413e290 2023-11-26T07:18:23.657 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:23.653+0000 7fac9bfff640 1 -- 172.21.15.143:0/872094910 <== mon.0 v2:172.21.15.143:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_volumes.TestMisc.test_binary_metafile_on_legacy_to_v1_upgrade"]}]=0 v50) v1 ==== 161+0+0 (secure 0 0 0) 0x7facb40b0100 con 0x7facb413e290 2023-11-26T07:18:23.660 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:23.653+0000 7facba006640 1 -- 172.21.15.143:0/872094910 >> [v2:172.21.15.143:6832/2150557544,v1:172.21.15.143:6833/2150557544] conn(0x7faca0066470 msgr2=0x7faca0068930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:18:23.660 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:23.653+0000 7facba006640 1 --2- 172.21.15.143:0/872094910 >> [v2:172.21.15.143:6832/2150557544,v1:172.21.15.143:6833/2150557544] conn(0x7faca0066470 0x7faca0068930 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7faca40011f0 tx=0x7faca4009040 comp rx=0 tx=0).stop 2023-11-26T07:18:23.660 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:23.653+0000 7facba006640 1 -- 172.21.15.143:0/872094910 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7facb413e290 msgr2=0x7facb41f53b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:18:23.660 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:23.653+0000 7facba006640 1 --2- 172.21.15.143:0/872094910 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7facb413e290 0x7facb41f53b0 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7facac0b0040 tx=0x7facac019630 comp rx=0 tx=0).stop 2023-11-26T07:18:23.661 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:23.653+0000 7facba006640 1 -- 172.21.15.143:0/872094910 shutdown_connections 2023-11-26T07:18:23.661 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:23.653+0000 7facba006640 1 --2- 172.21.15.143:0/872094910 >> [v2:172.21.15.143:6832/2150557544,v1:172.21.15.143:6833/2150557544] conn(0x7faca0066470 0x7faca0068930 unknown :-1 s=CLOSED pgs=141 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:18:23.661 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:23.653+0000 7facba006640 1 --2- 172.21.15.143:0/872094910 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7facb413e290 0x7facb41f53b0 unknown :-1 s=CLOSED pgs=224 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:18:23.661 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:23.653+0000 7facba006640 1 --2- 172.21.15.143:0/872094910 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7facb40cf990 0x7facb40b0d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:18:23.661 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:23.653+0000 7facba006640 1 --2- 172.21.15.143:0/872094910 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7facb40ce7c0 0x7facb415bf20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:18:23.661 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:23.653+0000 7facba006640 1 -- 172.21.15.143:0/872094910 >> 172.21.15.143:0/872094910 conn(0x7facb406a890 msgr2=0x7facb40cf280 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:18:23.661 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:23.653+0000 7facba006640 1 -- 172.21.15.143:0/872094910 shutdown_connections 2023-11-26T07:18:23.661 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:23.653+0000 7facba006640 1 -- 172.21.15.143:0/872094910 wait complete. 2023-11-26T07:18:23.682 DEBUG:teuthology.orchestra.run.smithi143:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mon mon_allow_pool_delete True 2023-11-26T07:18:23.824 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:23.817+0000 7f3cc9f57640 1 Processor -- start 2023-11-26T07:18:23.824 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:23.817+0000 7f3cc9f57640 1 -- start start 2023-11-26T07:18:23.825 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:23.821+0000 7f3cc9f57640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f3cc406f000 0x7f3cc406d030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:18:23.825 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:23.821+0000 7f3cc9f57640 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7f3cc4099a70 0x7f3cc406bcb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:18:23.825 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:23.821+0000 7f3cc9f57640 1 -- --> v1:172.21.15.188:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3cc40cf260 con 0x7f3cc406f3e0 2023-11-26T07:18:23.825 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:23.821+0000 7f3cc9f57640 1 -- --> v2:172.21.15.188:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3cc406d570 con 0x7f3cc4099a70 2023-11-26T07:18:23.825 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:23.821+0000 7f3cc9f57640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3cc409a010 con 0x7f3cc406f000 2023-11-26T07:18:23.826 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:23.821+0000 7f3cc3fff640 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7f3cc4099a70 0x7f3cc406bcb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:18:23.826 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:23.821+0000 7f3cc37fe640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f3cc406f000 0x7f3cc406d030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:18:23.826 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:23.821+0000 7f3cc3fff640 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7f3cc4099a70 0x7f3cc406bcb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.188:3300/0 says I am v2:172.21.15.143:35300/0 (socket says 172.21.15.143:35300) 2023-11-26T07:18:23.826 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:23.821+0000 7f3cc3fff640 1 -- 172.21.15.143:0/4053988103 learned_addr learned my addr 172.21.15.143:0/4053988103 (peer_addr_for_me v2:172.21.15.143:0/0) 2023-11-26T07:18:23.826 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:23.821+0000 7f3cc3fff640 1 -- 172.21.15.143:0/4053988103 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f3cc406f000 msgr2=0x7f3cc406d030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-26T07:18:23.826 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:23.821+0000 7f3cc3fff640 1 --2- 172.21.15.143:0/4053988103 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f3cc406f000 0x7f3cc406d030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:18:23.826 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:23.821+0000 7f3cc3fff640 1 -- 172.21.15.143:0/4053988103 >> v1:172.21.15.188:6789/0 conn(0x7f3cc406f3e0 legacy=0x7f3cc406b5a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:18:23.826 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:23.821+0000 7f3cc3fff640 1 -- 172.21.15.143:0/4053988103 --> v2:172.21.15.188:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3cc409a2f0 con 0x7f3cc4099a70 2023-11-26T07:18:23.828 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:23.821+0000 7f3cc3fff640 1 --2- 172.21.15.143:0/4053988103 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f3cc4099a70 0x7f3cc406bcb0 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7f3cbc009e50 tx=0x7f3cbc01ca10 comp rx=0 tx=0).ready entity=mon.1 client_cookie=ab2dd8b771294852 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:18:23.829 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:23.825+0000 7f3cc27fc640 1 -- 172.21.15.143:0/4053988103 <== mon.1 v2:172.21.15.188:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3cbc0a57d0 con 0x7f3cc4099a70 2023-11-26T07:18:23.829 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:23.825+0000 7f3cc27fc640 1 -- 172.21.15.143:0/4053988103 <== mon.1 v2:172.21.15.188:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3cbc0a5930 con 0x7f3cc4099a70 2023-11-26T07:18:23.829 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:23.825+0000 7f3cc27fc640 1 -- 172.21.15.143:0/4053988103 <== mon.1 v2:172.21.15.188:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3cbc0a5cc0 con 0x7f3cc4099a70 2023-11-26T07:18:23.830 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:23.825+0000 7f3cc9f57640 1 -- 172.21.15.143:0/4053988103 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f3cc4099a70 msgr2=0x7f3cc406bcb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:18:23.830 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:23.825+0000 7f3cc9f57640 1 --2- 172.21.15.143:0/4053988103 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f3cc4099a70 0x7f3cc406bcb0 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7f3cbc009e50 tx=0x7f3cbc01ca10 comp rx=0 tx=0).stop 2023-11-26T07:18:23.830 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:23.825+0000 7f3cc9f57640 1 -- 172.21.15.143:0/4053988103 shutdown_connections 2023-11-26T07:18:23.830 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:23.825+0000 7f3cc9f57640 1 --2- 172.21.15.143:0/4053988103 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f3cc4099a70 0x7f3cc406bcb0 unknown :-1 s=CLOSED pgs=125 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:18:23.830 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:23.825+0000 7f3cc9f57640 1 --2- 172.21.15.143:0/4053988103 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f3cc406f000 0x7f3cc406d030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:18:23.830 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:23.825+0000 7f3cc9f57640 1 -- 172.21.15.143:0/4053988103 >> 172.21.15.143:0/4053988103 conn(0x7f3cc40b2d10 msgr2=0x7f3cc40b3120 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:18:23.830 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:23.825+0000 7f3cc9f57640 1 -- 172.21.15.143:0/4053988103 shutdown_connections 2023-11-26T07:18:23.830 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:23.825+0000 7f3cc9f57640 1 -- 172.21.15.143:0/4053988103 wait complete. 2023-11-26T07:18:23.831 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:23.825+0000 7f3cc9f57640 1 Processor -- start 2023-11-26T07:18:23.831 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:23.825+0000 7f3cc9f57640 1 -- start start 2023-11-26T07:18:23.833 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:23.825+0000 7f3cc9f57640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f3cc406f000 0x7f3cc4168060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:18:23.833 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:23.825+0000 7f3cc9f57640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f3cc406f3e0 0x7f3cc41650d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:18:23.833 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:23.825+0000 7f3cc9f57640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f3cc4099a70 0x7f3cc4162270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:18:23.833 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:23.825+0000 7f3cc9f57640 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3cc4167c40 con 0x7f3cc4099a70 2023-11-26T07:18:23.833 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:23.825+0000 7f3cc9f57640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3cc4166fc0 con 0x7f3cc406f000 2023-11-26T07:18:23.833 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:23.825+0000 7f3cc9f57640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3cc41672a0 con 0x7f3cc406f3e0 2023-11-26T07:18:23.833 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:23.825+0000 7f3cc3fff640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f3cc4099a70 0x7f3cc4162270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:18:23.833 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:23.829+0000 7f3cc3fff640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f3cc4099a70 0x7f3cc4162270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:44146/0 (socket says 172.21.15.143:44146) 2023-11-26T07:18:23.833 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:23.829+0000 7f3cc3fff640 1 -- 172.21.15.143:0/1449299534 learned_addr learned my addr 172.21.15.143:0/1449299534 (peer_addr_for_me v2:172.21.15.143:0/0) 2023-11-26T07:18:23.833 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:23.829+0000 7f3cc2ffd640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f3cc406f3e0 0x7f3cc41650d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:18:23.833 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:23.829+0000 7f3cc37fe640 1 --2- 172.21.15.143:0/1449299534 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f3cc406f000 0x7f3cc4168060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:18:23.835 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:23.829+0000 7f3cc3fff640 1 -- 172.21.15.143:0/1449299534 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f3cc406f3e0 msgr2=0x7f3cc41650d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:18:23.835 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:23.829+0000 7f3cc3fff640 1 --2- 172.21.15.143:0/1449299534 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f3cc406f3e0 0x7f3cc41650d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:18:23.835 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:23.829+0000 7f3cc3fff640 1 -- 172.21.15.143:0/1449299534 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f3cc406f000 msgr2=0x7f3cc4168060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:18:23.835 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:23.829+0000 7f3cc3fff640 1 --2- 172.21.15.143:0/1449299534 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f3cc406f000 0x7f3cc4168060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:18:23.835 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:23.829+0000 7f3cc3fff640 1 -- 172.21.15.143:0/1449299534 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3cbc009b30 con 0x7f3cc4099a70 2023-11-26T07:18:23.835 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:23.829+0000 7f3cc37fe640 1 --2- 172.21.15.143:0/1449299534 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f3cc406f000 0x7f3cc4168060 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-26T07:18:23.835 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:23.829+0000 7f3cc3fff640 1 --2- 172.21.15.143:0/1449299534 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f3cc4099a70 0x7f3cc4162270 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7f3cbc0af040 tx=0x7f3cbc0197b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:18:23.835 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:23.829+0000 7f3cabfff640 1 -- 172.21.15.143:0/1449299534 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3cbc0b8670 con 0x7f3cc4099a70 2023-11-26T07:18:23.835 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:23.829+0000 7f3cabfff640 1 -- 172.21.15.143:0/1449299534 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3cbc0b87d0 con 0x7f3cc4099a70 2023-11-26T07:18:23.835 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:23.829+0000 7f3cabfff640 1 -- 172.21.15.143:0/1449299534 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3cbc0b8b80 con 0x7f3cc4099a70 2023-11-26T07:18:23.835 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:23.829+0000 7f3cc9f57640 1 -- 172.21.15.143:0/1449299534 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3cc4167580 con 0x7f3cc4099a70 2023-11-26T07:18:23.835 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:23.829+0000 7f3cc9f57640 1 -- 172.21.15.143:0/1449299534 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3cc4166bc0 con 0x7f3cc4099a70 2023-11-26T07:18:23.836 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:23.829+0000 7f3cc9f57640 1 -- 172.21.15.143:0/1449299534 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3cc4067680 con 0x7f3cc4099a70 2023-11-26T07:18:23.836 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:23.829+0000 7f3cabfff640 1 -- 172.21.15.143:0/1449299534 <== mon.0 v2:172.21.15.143:3300/0 4 ==== mgrmap(e 5) v1 ==== 85852+0+0 (secure 0 0 0) 0x7f3cbc0b8ce0 con 0x7f3cc4099a70 2023-11-26T07:18:23.837 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:23.833+0000 7f3cabfff640 1 --2- 172.21.15.143:0/1449299534 >> [v2:172.21.15.143:6832/2150557544,v1:172.21.15.143:6833/2150557544] conn(0x7f3cb0066470 0x7f3cb0068930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:18:23.837 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:23.833+0000 7f3cabfff640 1 -- 172.21.15.143:0/1449299534 <== mon.0 v2:172.21.15.143:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f3cbc11f300 con 0x7f3cc4099a70 2023-11-26T07:18:23.837 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:23.833+0000 7f3cc37fe640 1 --2- 172.21.15.143:0/1449299534 >> [v2:172.21.15.143:6832/2150557544,v1:172.21.15.143:6833/2150557544] conn(0x7f3cb0066470 0x7f3cb0068930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:18:23.838 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:23.833+0000 7f3cc37fe640 1 --2- 172.21.15.143:0/1449299534 >> [v2:172.21.15.143:6832/2150557544,v1:172.21.15.143:6833/2150557544] conn(0x7f3cb0066470 0x7f3cb0068930 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7f3cb4009f50 tx=0x7f3cb4009190 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:18:23.841 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:23.837+0000 7f3cabfff640 1 -- 172.21.15.143:0/1449299534 <== mon.0 v2:172.21.15.143:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172649 (secure 0 0 0) 0x7f3cbc0ec940 con 0x7f3cc4099a70 2023-11-26T07:18:23.991 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:23.985+0000 7f3cc9f57640 1 -- 172.21.15.143:0/1449299534 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command([{prefix=config set, name=mon_allow_pool_delete}] v 0) v1 -- 0x7f3cc40b4b70 con 0x7f3cc4099a70 2023-11-26T07:18:23.996 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:23.989+0000 7f3cabfff640 1 -- 172.21.15.143:0/1449299534 <== mon.0 v2:172.21.15.143:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mon_allow_pool_delete}]=0 v10)=0 v10) v1 ==== 122+0+0 (secure 0 0 0) 0x7f3cbc0eff60 con 0x7f3cc4099a70 2023-11-26T07:18:23.998 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:23.993+0000 7f3cc9f57640 1 -- 172.21.15.143:0/1449299534 >> [v2:172.21.15.143:6832/2150557544,v1:172.21.15.143:6833/2150557544] conn(0x7f3cb0066470 msgr2=0x7f3cb0068930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:18:23.998 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:23.993+0000 7f3cc9f57640 1 --2- 172.21.15.143:0/1449299534 >> [v2:172.21.15.143:6832/2150557544,v1:172.21.15.143:6833/2150557544] conn(0x7f3cb0066470 0x7f3cb0068930 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7f3cb4009f50 tx=0x7f3cb4009190 comp rx=0 tx=0).stop 2023-11-26T07:18:23.998 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:23.993+0000 7f3cc9f57640 1 -- 172.21.15.143:0/1449299534 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f3cc4099a70 msgr2=0x7f3cc4162270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:18:23.999 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:23.993+0000 7f3cc9f57640 1 --2- 172.21.15.143:0/1449299534 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f3cc4099a70 0x7f3cc4162270 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7f3cbc0af040 tx=0x7f3cbc0197b0 comp rx=0 tx=0).stop 2023-11-26T07:18:23.999 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:23.993+0000 7f3cc9f57640 1 -- 172.21.15.143:0/1449299534 shutdown_connections 2023-11-26T07:18:23.999 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:23.993+0000 7f3cc9f57640 1 --2- 172.21.15.143:0/1449299534 >> [v2:172.21.15.143:6832/2150557544,v1:172.21.15.143:6833/2150557544] conn(0x7f3cb0066470 0x7f3cb0068930 unknown :-1 s=CLOSED pgs=142 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:18:23.999 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:23.993+0000 7f3cc9f57640 1 --2- 172.21.15.143:0/1449299534 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f3cc4099a70 0x7f3cc4162270 unknown :-1 s=CLOSED pgs=225 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:18:23.999 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:23.993+0000 7f3cc9f57640 1 --2- 172.21.15.143:0/1449299534 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f3cc406f3e0 0x7f3cc41650d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:18:23.999 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:23.993+0000 7f3cc9f57640 1 --2- 172.21.15.143:0/1449299534 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f3cc406f000 0x7f3cc4168060 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:18:23.999 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:23.993+0000 7f3cc9f57640 1 -- 172.21.15.143:0/1449299534 >> 172.21.15.143:0/1449299534 conn(0x7f3cc40b2d10 msgr2=0x7f3cc40c4e10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:18:23.999 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:23.993+0000 7f3cc9f57640 1 -- 172.21.15.143:0/1449299534 shutdown_connections 2023-11-26T07:18:23.999 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:23.993+0000 7f3cc9f57640 1 -- 172.21.15.143:0/1449299534 wait complete. 2023-11-26T07:18:24.018 INFO:teuthology.nuke.actions:Clearing teuthology firewall rules... 2023-11-26T07:18:24.018 DEBUG:teuthology.orchestra.run.smithi143:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2023-11-26T07:18:24.060 DEBUG:teuthology.orchestra.run.smithi188:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2023-11-26T07:18:24.164 INFO:teuthology.nuke.actions:Cleared teuthology firewall rules. 2023-11-26T07:18:24.165 DEBUG:teuthology.orchestra.run.smithi143:> set -ex 2023-11-26T07:18:24.165 DEBUG:teuthology.orchestra.run.smithi143:> dd if=/proc/self/mounts of=/dev/stdout 2023-11-26T07:18:24.173 DEBUG:teuthology.orchestra.run.smithi143:> set -ex 2023-11-26T07:18:24.173 DEBUG:teuthology.orchestra.run.smithi143:> dd if=/proc/self/mounts of=/dev/stdout 2023-11-26T07:18:24.228 DEBUG:teuthology.orchestra.run.smithi143:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-11-26T07:18:24.409 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:24.401+0000 7ff8dfcd6640 1 Processor -- start 2023-11-26T07:18:24.410 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:24.405+0000 7ff8dfcd6640 1 -- start start 2023-11-26T07:18:24.411 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:24.405+0000 7ff8dfcd6640 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7ff8d80b2160 0x7ff8d80d0df0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:18:24.411 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:24.405+0000 7ff8dfcd6640 1 -- --> v1:172.21.15.143:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff8d80cf0e0 con 0x7ff8d80d1bc0 2023-11-26T07:18:24.411 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:24.405+0000 7ff8dfcd6640 1 -- --> v1:172.21.15.188:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff8d80ceb20 con 0x7ff8d815d6e0 2023-11-26T07:18:24.411 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:24.405+0000 7ff8dfcd6640 1 -- --> v2:172.21.15.143:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff8d815dac0 con 0x7ff8d80b2160 2023-11-26T07:18:24.411 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:24.405+0000 7ff8de273640 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7ff8d80b2160 0x7ff8d80d0df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:18:24.411 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:24.405+0000 7ff8dd271640 1 --1- >> v1:172.21.15.143:6789/0 conn(0x7ff8d80d1bc0 0x7ff8d80b0750 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.143:6789/0 says I am v1:172.21.15.143:47514/0 (socket says 172.21.15.143:47514) 2023-11-26T07:18:24.411 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:24.405+0000 7ff8dd271640 1 -- 172.21.15.143:0/1901049133 learned_addr learned my addr 172.21.15.143:0/1901049133 (peer_addr_for_me v1:172.21.15.143:0/0) 2023-11-26T07:18:24.412 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:24.405+0000 7ff8de273640 1 -- 172.21.15.143:0/1901049133 >> v1:172.21.15.188:6789/0 conn(0x7ff8d815d6e0 legacy=0x7ff8d80d14b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-26T07:18:24.412 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:24.405+0000 7ff8de273640 1 -- 172.21.15.143:0/1901049133 >> v1:172.21.15.143:6789/0 conn(0x7ff8d80d1bc0 legacy=0x7ff8d80b0750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:18:24.412 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:24.405+0000 7ff8dca70640 1 -- 172.21.15.143:0/1901049133 <== mon.0 v1:172.21.15.143:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3827595312 0 0) 0x7ff8d80cf0e0 con 0x7ff8d80d1bc0 2023-11-26T07:18:24.412 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:24.405+0000 7ff8de273640 1 -- 172.21.15.143:0/1901049133 --> v2:172.21.15.143:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff8d80cee00 con 0x7ff8d80b2160 2023-11-26T07:18:24.414 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:24.409+0000 7ff8de273640 1 --2- 172.21.15.143:0/1901049133 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7ff8d80b2160 0x7ff8d80d0df0 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7ff8d0009d90 tx=0x7ff8d001c950 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7dc480e1e8c2b094 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:18:24.414 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:24.409+0000 7ff8dca70640 1 -- 172.21.15.143:0/1901049133 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff8d00a5900 con 0x7ff8d80b2160 2023-11-26T07:18:24.414 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:24.409+0000 7ff8dca70640 1 -- 172.21.15.143:0/1901049133 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff8d00a5a60 con 0x7ff8d80b2160 2023-11-26T07:18:24.414 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:24.409+0000 7ff8dca70640 1 -- 172.21.15.143:0/1901049133 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff8d00a5df0 con 0x7ff8d80b2160 2023-11-26T07:18:24.415 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:24.409+0000 7ff8dfcd6640 1 -- 172.21.15.143:0/1901049133 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7ff8d80b2160 msgr2=0x7ff8d80d0df0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:18:24.415 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:24.409+0000 7ff8dfcd6640 1 --2- 172.21.15.143:0/1901049133 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7ff8d80b2160 0x7ff8d80d0df0 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7ff8d0009d90 tx=0x7ff8d001c950 comp rx=0 tx=0).stop 2023-11-26T07:18:24.415 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:24.409+0000 7ff8dfcd6640 1 -- 172.21.15.143:0/1901049133 shutdown_connections 2023-11-26T07:18:24.415 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:24.409+0000 7ff8dfcd6640 1 --2- 172.21.15.143:0/1901049133 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7ff8d80b2160 0x7ff8d80d0df0 unknown :-1 s=CLOSED pgs=227 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:18:24.415 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:24.409+0000 7ff8dfcd6640 1 -- 172.21.15.143:0/1901049133 >> 172.21.15.143:0/1901049133 conn(0x7ff8d809c550 msgr2=0x7ff8d80d3220 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:18:24.415 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:24.409+0000 7ff8dfcd6640 1 -- 172.21.15.143:0/1901049133 shutdown_connections 2023-11-26T07:18:24.415 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:24.409+0000 7ff8dfcd6640 1 -- 172.21.15.143:0/1901049133 wait complete. 2023-11-26T07:18:24.416 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:24.409+0000 7ff8dfcd6640 1 Processor -- start 2023-11-26T07:18:24.417 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:24.413+0000 7ff8dfcd6640 1 -- start start 2023-11-26T07:18:24.419 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:24.413+0000 7ff8dfcd6640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7ff8d80b2160 0x7ff8d815caa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:18:24.419 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:24.413+0000 7ff8dfcd6640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7ff8d80d1bc0 0x7ff8d815f410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:18:24.419 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:24.413+0000 7ff8dfcd6640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7ff8d815d6e0 0x7ff8d815f950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:18:24.419 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:24.413+0000 7ff8dfcd6640 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff8d815ff30 con 0x7ff8d80b2160 2023-11-26T07:18:24.419 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:24.413+0000 7ff8dfcd6640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff8d8160210 con 0x7ff8d80d1bc0 2023-11-26T07:18:24.419 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:24.413+0000 7ff8dfcd6640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff8d81604f0 con 0x7ff8d815d6e0 2023-11-26T07:18:24.419 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:24.413+0000 7ff8dda72640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7ff8d80b2160 0x7ff8d815caa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:18:24.420 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:24.413+0000 7ff8dda72640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7ff8d80b2160 0x7ff8d815caa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:33250/0 (socket says 172.21.15.143:33250) 2023-11-26T07:18:24.420 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:24.413+0000 7ff8dda72640 1 -- 172.21.15.143:0/1238252044 learned_addr learned my addr 172.21.15.143:0/1238252044 (peer_addr_for_me v2:172.21.15.143:0/0) 2023-11-26T07:18:24.420 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:24.413+0000 7ff8dd271640 1 --2- 172.21.15.143:0/1238252044 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7ff8d80d1bc0 0x7ff8d815f410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:18:24.420 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:24.413+0000 7ff8de273640 1 --2- 172.21.15.143:0/1238252044 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7ff8d815d6e0 0x7ff8d815f950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:18:24.420 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:24.413+0000 7ff8dda72640 1 -- 172.21.15.143:0/1238252044 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7ff8d815d6e0 msgr2=0x7ff8d815f950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:18:24.420 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:24.413+0000 7ff8dda72640 1 --2- 172.21.15.143:0/1238252044 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7ff8d815d6e0 0x7ff8d815f950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:18:24.420 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:24.413+0000 7ff8dda72640 1 -- 172.21.15.143:0/1238252044 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7ff8d80d1bc0 msgr2=0x7ff8d815f410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:18:24.420 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:24.413+0000 7ff8dda72640 1 --2- 172.21.15.143:0/1238252044 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7ff8d80d1bc0 0x7ff8d815f410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:18:24.420 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:24.413+0000 7ff8dda72640 1 -- 172.21.15.143:0/1238252044 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff8d0009a70 con 0x7ff8d80b2160 2023-11-26T07:18:24.420 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:24.413+0000 7ff8dd271640 1 --2- 172.21.15.143:0/1238252044 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7ff8d80d1bc0 0x7ff8d815f410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-26T07:18:24.420 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:24.413+0000 7ff8dda72640 1 --2- 172.21.15.143:0/1238252044 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7ff8d80b2160 0x7ff8d815caa0 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7ff8d400c3c0 tx=0x7ff8d400c880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:18:24.420 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:24.413+0000 7ff8ce7fc640 1 -- 172.21.15.143:0/1238252044 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff8d40052d0 con 0x7ff8d80b2160 2023-11-26T07:18:24.420 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:24.413+0000 7ff8ce7fc640 1 -- 172.21.15.143:0/1238252044 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff8d4005430 con 0x7ff8d80b2160 2023-11-26T07:18:24.421 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:24.413+0000 7ff8ce7fc640 1 -- 172.21.15.143:0/1238252044 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff8d4003840 con 0x7ff8d80b2160 2023-11-26T07:18:24.421 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:24.413+0000 7ff8dfcd6640 1 -- 172.21.15.143:0/1238252044 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff8d81607d0 con 0x7ff8d80b2160 2023-11-26T07:18:24.421 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:24.413+0000 7ff8dfcd6640 1 -- 172.21.15.143:0/1238252044 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff8d8157e30 con 0x7ff8d80b2160 2023-11-26T07:18:24.421 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:24.413+0000 7ff8dfcd6640 1 -- 172.21.15.143:0/1238252044 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff8d80d2690 con 0x7ff8d80b2160 2023-11-26T07:18:24.421 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:24.417+0000 7ff8ce7fc640 1 -- 172.21.15.143:0/1238252044 <== mon.0 v2:172.21.15.143:3300/0 4 ==== mgrmap(e 5) v1 ==== 85852+0+0 (secure 0 0 0) 0x7ff8d40039a0 con 0x7ff8d80b2160 2023-11-26T07:18:24.422 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:24.417+0000 7ff8ce7fc640 1 --2- 172.21.15.143:0/1238252044 >> [v2:172.21.15.143:6832/2150557544,v1:172.21.15.143:6833/2150557544] conn(0x7ff8bc066470 0x7ff8bc068930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:18:24.422 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:24.417+0000 7ff8ce7fc640 1 -- 172.21.15.143:0/1238252044 <== mon.0 v2:172.21.15.143:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5709+0+0 (secure 0 0 0) 0x7ff8d4083470 con 0x7ff8d80b2160 2023-11-26T07:18:24.422 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:24.417+0000 7ff8dd271640 1 --2- 172.21.15.143:0/1238252044 >> [v2:172.21.15.143:6832/2150557544,v1:172.21.15.143:6833/2150557544] conn(0x7ff8bc066470 0x7ff8bc068930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:18:24.422 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:24.417+0000 7ff8dd271640 1 --2- 172.21.15.143:0/1238252044 >> [v2:172.21.15.143:6832/2150557544,v1:172.21.15.143:6833/2150557544] conn(0x7ff8bc066470 0x7ff8bc068930 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7ff8d80b0ba0 tx=0x7ff8c8007c60 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:18:24.425 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:24.421+0000 7ff8ce7fc640 1 -- 172.21.15.143:0/1238252044 <== mon.0 v2:172.21.15.143:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172649 (secure 0 0 0) 0x7ff8d4050850 con 0x7ff8d80b2160 2023-11-26T07:18:24.575 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:24.569+0000 7ff8dfcd6640 1 -- 172.21.15.143:0/1238252044 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7ff8d8160070 con 0x7ff8d80b2160 2023-11-26T07:18:24.576 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:24.569+0000 7ff8ce7fc640 1 -- 172.21.15.143:0/1238252044 <== mon.0 v2:172.21.15.143:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v20) v1 ==== 74+0+16541 (secure 0 0 0) 0x7ff8d8160070 con 0x7ff8d80b2160 2023-11-26T07:18:24.576 INFO:teuthology.orchestra.run.smithi143.stdout: 2023-11-26T07:18:24.576 INFO:teuthology.orchestra.run.smithi143.stdout:{"epoch":20,"fsid":"f5033060-2469-4a07-95fe-2b76f5404bc1","created":"2023-11-26T07:17:28.613663+0000","modified":"2023-11-26T07:17:55.505435+0000","last_up_change":"2023-11-26T07:17:41.157813+0000","last_in_change":"2023-11-26T07:17:31.181658+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-11-26T07:17:41.758597+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2023-11-26T07:17:51.878332+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2023-11-26T07:17:52.871396+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"ebbee2c6-023e-44b9-9f1d-6f7bfd309730","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6817","nonce":639807369},{"type":"v1","addr":"172.21.15.143:6819","nonce":639807369}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6821","nonce":639807369},{"type":"v1","addr":"172.21.15.143:6823","nonce":639807369}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6830","nonce":639807369},{"type":"v1","addr":"172.21.15.143:6831","nonce":639807369}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6825","nonce":639807369},{"type":"v1","addr":"172.21.15.143:6828","nonce":639807369}]},"public_addr":"172.21.15.143:6819/639807369","cluster_addr":"172.21.15.143:6823/639807369","heartbeat_back_addr":"172.21.15.143:6831/639807369","heartbeat_front_addr":"172.21.15.143:6828/639807369","state":["exists","up"]},{"osd":1,"uuid":"fec4b8be-7122-49e9-9a19-5850a0700001","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6816","nonce":1144942530},{"type":"v1","addr":"172.21.15.143:6818","nonce":1144942530}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6820","nonce":1144942530},{"type":"v1","addr":"172.21.15.143:6822","nonce":1144942530}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6827","nonce":1144942530},{"type":"v1","addr":"172.21.15.143:6829","nonce":1144942530}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6824","nonce":1144942530},{"type":"v1","addr":"172.21.15.143:6826","nonce":1144942530}]},"public_addr":"172.21.15.143:6818/1144942530","cluster_addr":"172.21.15.143:6822/1144942530","heartbeat_back_addr":"172.21.15.143:6829/1144942530","heartbeat_front_addr":"172.21.15.143:6826/1144942530","state":["exists","up"]},{"osd":2,"uuid":"2888af5b-f933-46ca-8f95-6dd494ff43f6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6808","nonce":3909095186},{"type":"v1","addr":"172.21.15.143:6809","nonce":3909095186}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6810","nonce":3909095186},{"type":"v1","addr":"172.21.15.143:6811","nonce":3909095186}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6814","nonce":3909095186},{"type":"v1","addr":"172.21.15.143:6815","nonce":3909095186}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6812","nonce":3909095186},{"type":"v1","addr":"172.21.15.143:6813","nonce":3909095186}]},"public_addr":"172.21.15.143:6809/3909095186","cluster_addr":"172.21.15.143:6811/3909095186","heartbeat_back_addr":"172.21.15.143:6815/3909095186","heartbeat_front_addr":"172.21.15.143:6813/3909095186","state":["exists","up"]},{"osd":3,"uuid":"d1f52435-7bc1-4081-9550-9cd1ba55f549","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6800","nonce":3854210991},{"type":"v1","addr":"172.21.15.143:6801","nonce":3854210991}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6802","nonce":3854210991},{"type":"v1","addr":"172.21.15.143:6803","nonce":3854210991}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6806","nonce":3854210991},{"type":"v1","addr":"172.21.15.143:6807","nonce":3854210991}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6804","nonce":3854210991},{"type":"v1","addr":"172.21.15.143:6805","nonce":3854210991}]},"public_addr":"172.21.15.143:6801/3854210991","cluster_addr":"172.21.15.143:6803/3854210991","heartbeat_back_addr":"172.21.15.143:6807/3854210991","heartbeat_front_addr":"172.21.15.143:6805/3854210991","state":["exists","up"]},{"osd":4,"uuid":"e2ae8b6e-6e25-49b2-bcbf-aacce68f06a3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6800","nonce":3924606902},{"type":"v1","addr":"172.21.15.188:6801","nonce":3924606902}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6802","nonce":3924606902},{"type":"v1","addr":"172.21.15.188:6803","nonce":3924606902}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6806","nonce":3924606902},{"type":"v1","addr":"172.21.15.188:6807","nonce":3924606902}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6804","nonce":3924606902},{"type":"v1","addr":"172.21.15.188:6805","nonce":3924606902}]},"public_addr":"172.21.15.188:6801/3924606902","cluster_addr":"172.21.15.188:6803/3924606902","heartbeat_back_addr":"172.21.15.188:6807/3924606902","heartbeat_front_addr":"172.21.15.188:6805/3924606902","state":["exists","up"]},{"osd":5,"uuid":"3cec8a32-7698-4c36-8e89-e4ba11934f33","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6813","nonce":186841544},{"type":"v1","addr":"172.21.15.188:6815","nonce":186841544}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6817","nonce":186841544},{"type":"v1","addr":"172.21.15.188:6818","nonce":186841544}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6822","nonce":186841544},{"type":"v1","addr":"172.21.15.188:6823","nonce":186841544}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6820","nonce":186841544},{"type":"v1","addr":"172.21.15.188:6821","nonce":186841544}]},"public_addr":"172.21.15.188:6815/186841544","cluster_addr":"172.21.15.188:6818/186841544","heartbeat_back_addr":"172.21.15.188:6823/186841544","heartbeat_front_addr":"172.21.15.188:6821/186841544","state":["exists","up"]},{"osd":6,"uuid":"58a65d5d-e090-473a-81b4-ad38e85029f9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6808","nonce":2153499405},{"type":"v1","addr":"172.21.15.188:6809","nonce":2153499405}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6810","nonce":2153499405},{"type":"v1","addr":"172.21.15.188:6811","nonce":2153499405}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6816","nonce":2153499405},{"type":"v1","addr":"172.21.15.188:6819","nonce":2153499405}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6812","nonce":2153499405},{"type":"v1","addr":"172.21.15.188:6814","nonce":2153499405}]},"public_addr":"172.21.15.188:6809/2153499405","cluster_addr":"172.21.15.188:6811/2153499405","heartbeat_back_addr":"172.21.15.188:6819/2153499405","heartbeat_front_addr":"172.21.15.188:6814/2153499405","state":["exists","up"]},{"osd":7,"uuid":"ea5b7f70-3554-4cf8-ac30-f7c59b80678d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6824","nonce":2722096729},{"type":"v1","addr":"172.21.15.188:6825","nonce":2722096729}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6826","nonce":2722096729},{"type":"v1","addr":"172.21.15.188:6827","nonce":2722096729}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6830","nonce":2722096729},{"type":"v1","addr":"172.21.15.188:6831","nonce":2722096729}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6828","nonce":2722096729},{"type":"v1","addr":"172.21.15.188:6829","nonce":2722096729}]},"public_addr":"172.21.15.188:6825/2722096729","cluster_addr":"172.21.15.188:6827/2722096729","heartbeat_back_addr":"172.21.15.188:6831/2722096729","heartbeat_front_addr":"172.21.15.188:6829/2722096729","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-26T07:17:36.995032+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-26T07:17:36.932266+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-26T07:17:36.829304+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-26T07:17:36.804297+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-26T07:17:36.667892+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-26T07:17:36.964868+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-26T07:17:36.827332+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-26T07:17:36.893179+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-11-26T07:18:24.578 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:24.573+0000 7ff8dfcd6640 1 -- 172.21.15.143:0/1238252044 >> [v2:172.21.15.143:6832/2150557544,v1:172.21.15.143:6833/2150557544] conn(0x7ff8bc066470 msgr2=0x7ff8bc068930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:18:24.578 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:24.573+0000 7ff8dfcd6640 1 --2- 172.21.15.143:0/1238252044 >> [v2:172.21.15.143:6832/2150557544,v1:172.21.15.143:6833/2150557544] conn(0x7ff8bc066470 0x7ff8bc068930 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7ff8d80b0ba0 tx=0x7ff8c8007c60 comp rx=0 tx=0).stop 2023-11-26T07:18:24.578 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:24.573+0000 7ff8dfcd6640 1 -- 172.21.15.143:0/1238252044 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7ff8d80b2160 msgr2=0x7ff8d815caa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:18:24.578 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:24.573+0000 7ff8dfcd6640 1 --2- 172.21.15.143:0/1238252044 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7ff8d80b2160 0x7ff8d815caa0 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7ff8d400c3c0 tx=0x7ff8d400c880 comp rx=0 tx=0).stop 2023-11-26T07:18:24.578 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:24.573+0000 7ff8dfcd6640 1 -- 172.21.15.143:0/1238252044 shutdown_connections 2023-11-26T07:18:24.579 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:24.573+0000 7ff8dfcd6640 1 --2- 172.21.15.143:0/1238252044 >> [v2:172.21.15.143:6832/2150557544,v1:172.21.15.143:6833/2150557544] conn(0x7ff8bc066470 0x7ff8bc068930 unknown :-1 s=CLOSED pgs=143 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:18:24.579 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:24.573+0000 7ff8dfcd6640 1 --2- 172.21.15.143:0/1238252044 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7ff8d815d6e0 0x7ff8d815f950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:18:24.579 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:24.573+0000 7ff8dfcd6640 1 --2- 172.21.15.143:0/1238252044 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7ff8d80d1bc0 0x7ff8d815f410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:18:24.579 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:24.573+0000 7ff8dfcd6640 1 --2- 172.21.15.143:0/1238252044 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7ff8d80b2160 0x7ff8d815caa0 unknown :-1 s=CLOSED pgs=228 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:18:24.579 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:24.573+0000 7ff8dfcd6640 1 -- 172.21.15.143:0/1238252044 >> 172.21.15.143:0/1238252044 conn(0x7ff8d809c550 msgr2=0x7ff8d8158c00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:18:24.579 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:24.573+0000 7ff8dfcd6640 1 -- 172.21.15.143:0/1238252044 shutdown_connections 2023-11-26T07:18:24.579 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:24.573+0000 7ff8dfcd6640 1 -- 172.21.15.143:0/1238252044 wait complete. 2023-11-26T07:18:24.597 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-11-26T07:17:41.758597+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2023-11-26T07:17:51.878332+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2023-11-26T07:17:52.871396+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '20', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2023-11-26T07:18:24.598 DEBUG:teuthology.orchestra.run.smithi143:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-11-26T07:18:24.739 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:24.733+0000 7f90edba1640 1 Processor -- start 2023-11-26T07:18:24.739 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:24.733+0000 7f90edba1640 1 -- start start 2023-11-26T07:18:24.740 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:24.733+0000 7f90edba1640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f90e80cf280 0x7f90e80cf6e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:18:24.740 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:24.733+0000 7f90edba1640 1 -- --> v1:172.21.15.143:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f90e80cfc20 con 0x7f90e8143ac0 2023-11-26T07:18:24.740 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:24.733+0000 7f90edba1640 1 -- --> v1:172.21.15.188:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f90e80cc730 con 0x7f90e80cd1b0 2023-11-26T07:18:24.740 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:24.733+0000 7f90edba1640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f90e8143ea0 con 0x7f90e80cf280 2023-11-26T07:18:24.740 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:24.733+0000 7f90e77fe640 1 --1- >> v1:172.21.15.143:6789/0 conn(0x7f90e8143ac0 0x7f90e8070370 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.143:6789/0 says I am v1:172.21.15.143:47516/0 (socket says 172.21.15.143:47516) 2023-11-26T07:18:24.740 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:24.733+0000 7f90e77fe640 1 -- 172.21.15.143:0/2608905029 learned_addr learned my addr 172.21.15.143:0/2608905029 (peer_addr_for_me v1:172.21.15.143:0/0) 2023-11-26T07:18:24.740 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:24.733+0000 7f90dffff640 1 --2- 172.21.15.143:0/2608905029 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f90e80cf280 0x7f90e80cf6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:18:24.741 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:24.733+0000 7f90e6ffd640 1 -- 172.21.15.143:0/2608905029 <== mon.0 v1:172.21.15.143:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3582772381 0 0) 0x7f90e80cfc20 con 0x7f90e8143ac0 2023-11-26T07:18:24.741 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:24.733+0000 7f90e6ffd640 1 -- 172.21.15.143:0/2608905029 --> v1:172.21.15.143:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f90d4003530 con 0x7f90e8143ac0 2023-11-26T07:18:24.741 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:24.733+0000 7f90e6ffd640 1 -- 172.21.15.143:0/2608905029 <== mon.1 v1:172.21.15.188:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 316223415 0 0) 0x7f90e80cc730 con 0x7f90e80cd1b0 2023-11-26T07:18:24.741 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:24.733+0000 7f90e6ffd640 1 -- 172.21.15.143:0/2608905029 --> v1:172.21.15.188:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f90e80cfc20 con 0x7f90e80cd1b0 2023-11-26T07:18:24.741 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:24.733+0000 7f90dffff640 1 -- 172.21.15.143:0/2608905029 >> v1:172.21.15.188:6789/0 conn(0x7f90e80cd1b0 legacy=0x7f90e80cd5e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:18:24.741 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:24.733+0000 7f90dffff640 1 -- 172.21.15.143:0/2608905029 >> v1:172.21.15.143:6789/0 conn(0x7f90e8143ac0 legacy=0x7f90e8070370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:18:24.742 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:24.733+0000 7f90dffff640 1 -- 172.21.15.143:0/2608905029 --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f90e8070ee0 con 0x7f90e80cf280 2023-11-26T07:18:24.743 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:24.737+0000 7f90dffff640 1 --2- 172.21.15.143:0/2608905029 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f90e80cf280 0x7f90e80cf6e0 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7f90d00097f0 tx=0x7f90d001c4c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=2ad4b49c39e8b853 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:18:24.744 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:24.737+0000 7f90e6ffd640 1 -- 172.21.15.143:0/2608905029 <== mon.2 v2:172.21.15.188:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f90d00aa620 con 0x7f90e80cf280 2023-11-26T07:18:24.744 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:24.737+0000 7f90e6ffd640 1 -- 172.21.15.143:0/2608905029 <== mon.2 v2:172.21.15.188:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f90d00aa780 con 0x7f90e80cf280 2023-11-26T07:18:24.744 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:24.737+0000 7f90e6ffd640 1 -- 172.21.15.143:0/2608905029 <== mon.2 v2:172.21.15.188:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f90d00aaaf0 con 0x7f90e80cf280 2023-11-26T07:18:24.745 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:24.737+0000 7f90edba1640 1 -- 172.21.15.143:0/2608905029 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f90e80cf280 msgr2=0x7f90e80cf6e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:18:24.745 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:24.737+0000 7f90edba1640 1 --2- 172.21.15.143:0/2608905029 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f90e80cf280 0x7f90e80cf6e0 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7f90d00097f0 tx=0x7f90d001c4c0 comp rx=0 tx=0).stop 2023-11-26T07:18:24.745 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:24.737+0000 7f90edba1640 1 -- 172.21.15.143:0/2608905029 shutdown_connections 2023-11-26T07:18:24.745 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:24.737+0000 7f90edba1640 1 --2- 172.21.15.143:0/2608905029 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f90e80cf280 0x7f90e80cf6e0 unknown :-1 s=CLOSED pgs=60 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:18:24.745 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:24.737+0000 7f90edba1640 1 -- 172.21.15.143:0/2608905029 >> 172.21.15.143:0/2608905029 conn(0x7f90e806a890 msgr2=0x7f90e806aca0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:18:24.745 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:24.737+0000 7f90edba1640 1 -- 172.21.15.143:0/2608905029 shutdown_connections 2023-11-26T07:18:24.745 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:24.737+0000 7f90edba1640 1 -- 172.21.15.143:0/2608905029 wait complete. 2023-11-26T07:18:24.746 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:24.741+0000 7f90edba1640 1 Processor -- start 2023-11-26T07:18:24.746 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:24.741+0000 7f90edba1640 1 -- start start 2023-11-26T07:18:24.748 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:24.741+0000 7f90edba1640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f90e80cd1b0 0x7f90e814c830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:18:24.748 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:24.741+0000 7f90edba1640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f90e80cf280 0x7f90e81f5cb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:18:24.748 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:24.741+0000 7f90edba1640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f90e8143ac0 0x7f90e80af460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:18:24.748 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:24.741+0000 7f90edba1640 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f90e8138230 con 0x7f90e8143ac0 2023-11-26T07:18:24.748 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:24.741+0000 7f90edba1640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f90e814cd70 con 0x7f90e80cf280 2023-11-26T07:18:24.748 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:24.741+0000 7f90edba1640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f90e81375b0 con 0x7f90e80cd1b0 2023-11-26T07:18:24.749 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:24.741+0000 7f90e7fff640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f90e8143ac0 0x7f90e80af460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:18:24.749 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:24.741+0000 7f90e77fe640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f90e80cd1b0 0x7f90e814c830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:18:24.749 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:24.741+0000 7f90e7fff640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f90e8143ac0 0x7f90e80af460 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:33252/0 (socket says 172.21.15.143:33252) 2023-11-26T07:18:24.749 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:24.741+0000 7f90dffff640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f90e80cf280 0x7f90e81f5cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:18:24.749 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:24.741+0000 7f90e7fff640 1 -- 172.21.15.143:0/3703443468 learned_addr learned my addr 172.21.15.143:0/3703443468 (peer_addr_for_me v2:172.21.15.143:0/0) 2023-11-26T07:18:24.749 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:24.741+0000 7f90e77fe640 1 --2- 172.21.15.143:0/3703443468 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f90e80cd1b0 0x7f90e814c830 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.188:3301/0 says I am v2:172.21.15.143:45784/0 (socket says 172.21.15.143:45784) 2023-11-26T07:18:24.749 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:24.741+0000 7f90e7fff640 1 -- 172.21.15.143:0/3703443468 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f90e80cd1b0 msgr2=0x7f90e814c830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:18:24.749 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:24.741+0000 7f90e7fff640 1 --2- 172.21.15.143:0/3703443468 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f90e80cd1b0 0x7f90e814c830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:18:24.749 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:24.741+0000 7f90e7fff640 1 -- 172.21.15.143:0/3703443468 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f90e80cf280 msgr2=0x7f90e81f5cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:18:24.749 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:24.741+0000 7f90e7fff640 1 --2- 172.21.15.143:0/3703443468 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f90e80cf280 0x7f90e81f5cb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:18:24.749 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:24.741+0000 7f90e7fff640 1 -- 172.21.15.143:0/3703443468 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f90d0009420 con 0x7f90e8143ac0 2023-11-26T07:18:24.749 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:24.741+0000 7f90e7fff640 1 --2- 172.21.15.143:0/3703443468 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f90e8143ac0 0x7f90e80af460 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7f90e000c760 tx=0x7f90e000cc20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:18:24.750 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:24.741+0000 7f90e4ff9640 1 -- 172.21.15.143:0/3703443468 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f90e0003b80 con 0x7f90e8143ac0 2023-11-26T07:18:24.750 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:24.741+0000 7f90e4ff9640 1 -- 172.21.15.143:0/3703443468 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f90e0003ce0 con 0x7f90e8143ac0 2023-11-26T07:18:24.750 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:24.741+0000 7f90edba1640 1 -- 172.21.15.143:0/3703443468 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f90e8137890 con 0x7f90e8143ac0 2023-11-26T07:18:24.751 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:24.741+0000 7f90e4ff9640 1 -- 172.21.15.143:0/3703443468 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f90e000aa30 con 0x7f90e8143ac0 2023-11-26T07:18:24.751 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:24.745+0000 7f90edba1640 1 -- 172.21.15.143:0/3703443468 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f90e8137e50 con 0x7f90e8143ac0 2023-11-26T07:18:24.751 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:24.745+0000 7f90edba1640 1 -- 172.21.15.143:0/3703443468 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f90e8067680 con 0x7f90e8143ac0 2023-11-26T07:18:24.751 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:24.745+0000 7f90e4ff9640 1 -- 172.21.15.143:0/3703443468 <== mon.0 v2:172.21.15.143:3300/0 4 ==== mgrmap(e 5) v1 ==== 85852+0+0 (secure 0 0 0) 0x7f90e000ab90 con 0x7f90e8143ac0 2023-11-26T07:18:24.752 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:24.745+0000 7f90e4ff9640 1 --2- 172.21.15.143:0/3703443468 >> [v2:172.21.15.143:6832/2150557544,v1:172.21.15.143:6833/2150557544] conn(0x7f90c8066470 0x7f90c8068930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:18:24.752 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:24.745+0000 7f90e4ff9640 1 -- 172.21.15.143:0/3703443468 <== mon.0 v2:172.21.15.143:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f90e0083ae0 con 0x7f90e8143ac0 2023-11-26T07:18:24.752 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:24.745+0000 7f90e77fe640 1 --2- 172.21.15.143:0/3703443468 >> [v2:172.21.15.143:6832/2150557544,v1:172.21.15.143:6833/2150557544] conn(0x7f90c8066470 0x7f90c8068930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:18:24.753 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:24.749+0000 7f90e77fe640 1 --2- 172.21.15.143:0/3703443468 >> [v2:172.21.15.143:6832/2150557544,v1:172.21.15.143:6833/2150557544] conn(0x7f90c8066470 0x7f90c8068930 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7f90d8007fd0 tx=0x7f90d8007d80 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:18:24.756 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:24.749+0000 7f90e4ff9640 1 -- 172.21.15.143:0/3703443468 <== mon.0 v2:172.21.15.143:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172649 (secure 0 0 0) 0x7f90e0051000 con 0x7f90e8143ac0 2023-11-26T07:18:24.907 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:24.901+0000 7f90edba1640 1 -- 172.21.15.143:0/3703443468 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f90e80704f0 con 0x7f90e8143ac0 2023-11-26T07:18:24.908 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:24.901+0000 7f90e4ff9640 1 -- 172.21.15.143:0/3703443468 <== mon.0 v2:172.21.15.143:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v20) v1 ==== 93+0+10 (secure 0 0 0) 0x7f90e0054620 con 0x7f90e8143ac0 2023-11-26T07:18:24.908 INFO:teuthology.orchestra.run.smithi143.stdout:pg_num: 1 2023-11-26T07:18:24.909 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:24.905+0000 7f90edba1640 1 -- 172.21.15.143:0/3703443468 >> [v2:172.21.15.143:6832/2150557544,v1:172.21.15.143:6833/2150557544] conn(0x7f90c8066470 msgr2=0x7f90c8068930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:18:24.909 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:24.905+0000 7f90edba1640 1 --2- 172.21.15.143:0/3703443468 >> [v2:172.21.15.143:6832/2150557544,v1:172.21.15.143:6833/2150557544] conn(0x7f90c8066470 0x7f90c8068930 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7f90d8007fd0 tx=0x7f90d8007d80 comp rx=0 tx=0).stop 2023-11-26T07:18:24.909 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:24.905+0000 7f90edba1640 1 -- 172.21.15.143:0/3703443468 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f90e8143ac0 msgr2=0x7f90e80af460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:18:24.910 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:24.905+0000 7f90edba1640 1 --2- 172.21.15.143:0/3703443468 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f90e8143ac0 0x7f90e80af460 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7f90e000c760 tx=0x7f90e000cc20 comp rx=0 tx=0).stop 2023-11-26T07:18:24.910 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:24.905+0000 7f90edba1640 1 -- 172.21.15.143:0/3703443468 shutdown_connections 2023-11-26T07:18:24.910 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:24.905+0000 7f90edba1640 1 --2- 172.21.15.143:0/3703443468 >> [v2:172.21.15.143:6832/2150557544,v1:172.21.15.143:6833/2150557544] conn(0x7f90c8066470 0x7f90c8068930 unknown :-1 s=CLOSED pgs=144 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:18:24.910 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:24.905+0000 7f90edba1640 1 --2- 172.21.15.143:0/3703443468 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f90e8143ac0 0x7f90e80af460 unknown :-1 s=CLOSED pgs=230 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:18:24.910 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:24.905+0000 7f90edba1640 1 --2- 172.21.15.143:0/3703443468 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f90e80cf280 0x7f90e81f5cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:18:24.910 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:24.905+0000 7f90edba1640 1 --2- 172.21.15.143:0/3703443468 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f90e80cd1b0 0x7f90e814c830 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:18:24.910 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:24.905+0000 7f90edba1640 1 -- 172.21.15.143:0/3703443468 >> 172.21.15.143:0/3703443468 conn(0x7f90e806a890 msgr2=0x7f90e80b2850 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:18:24.910 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:24.905+0000 7f90edba1640 1 -- 172.21.15.143:0/3703443468 shutdown_connections 2023-11-26T07:18:24.910 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:24.905+0000 7f90edba1640 1 -- 172.21.15.143:0/3703443468 wait complete. 2023-11-26T07:18:24.929 DEBUG:teuthology.orchestra.run.smithi143:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-11-26T07:18:25.069 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:25.061+0000 7fea04757640 1 Processor -- start 2023-11-26T07:18:25.069 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:25.061+0000 7fea04757640 1 -- start start 2023-11-26T07:18:25.071 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:25.065+0000 7fea04757640 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7fe9fc13e290 0x7fe9fc0cf280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:18:25.071 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:25.065+0000 7fea04757640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fe9fc0cf7c0 0x7fe9fc15efa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:18:25.071 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:25.065+0000 7fea04757640 1 -- --> v1:172.21.15.143:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe9fc06c520 con 0x7fe9fc15f4e0 2023-11-26T07:18:25.071 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:25.065+0000 7fea04757640 1 -- --> v2:172.21.15.143:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe9fc0cfc40 con 0x7fe9fc13e290 2023-11-26T07:18:25.071 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:25.065+0000 7fea04757640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe9fc06c800 con 0x7fe9fc0cf7c0 2023-11-26T07:18:25.071 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:25.065+0000 7fea024f3640 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7fe9fc13e290 0x7fe9fc0cf280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:18:25.071 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:25.065+0000 7fea024f3640 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7fe9fc13e290 0x7fe9fc0cf280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:33260/0 (socket says 172.21.15.143:33260) 2023-11-26T07:18:25.071 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:25.065+0000 7fea024f3640 1 -- 172.21.15.143:0/952892408 learned_addr learned my addr 172.21.15.143:0/952892408 (peer_addr_for_me v2:172.21.15.143:0/0) 2023-11-26T07:18:25.071 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:25.065+0000 7fea02cf4640 1 --1- 172.21.15.143:0/952892408 >> v1:172.21.15.143:6789/0 conn(0x7fe9fc15f4e0 0x7fe9fc1408e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.143:6789/0 says I am v1:172.21.15.143:47532/0 (socket says 172.21.15.143:47532) 2023-11-26T07:18:25.072 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:25.065+0000 7fea01cf2640 1 --2- 172.21.15.143:0/952892408 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fe9fc0cf7c0 0x7fe9fc15efa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:18:25.072 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:25.065+0000 7fea024f3640 1 -- 172.21.15.143:0/952892408 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fe9fc0cf7c0 msgr2=0x7fe9fc15efa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-26T07:18:25.072 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:25.065+0000 7fea024f3640 1 --2- 172.21.15.143:0/952892408 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fe9fc0cf7c0 0x7fe9fc15efa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:18:25.072 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:25.065+0000 7fea024f3640 1 -- 172.21.15.143:0/952892408 >> v1:172.21.15.143:6789/0 conn(0x7fe9fc15f4e0 legacy=0x7fe9fc1408e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:18:25.072 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:25.065+0000 7fea024f3640 1 -- 172.21.15.143:0/952892408 --> v2:172.21.15.143:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe9fc06cae0 con 0x7fe9fc13e290 2023-11-26T07:18:25.072 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:25.065+0000 7fea01cf2640 1 --2- 172.21.15.143:0/952892408 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fe9fc0cf7c0 0x7fe9fc15efa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-26T07:18:25.074 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:25.069+0000 7fea024f3640 1 --2- 172.21.15.143:0/952892408 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fe9fc13e290 0x7fe9fc0cf280 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7fe9f8009870 tx=0x7fe9f801c3d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a0760e2b4b62cb36 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:18:25.074 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:25.069+0000 7fea014f1640 1 -- 172.21.15.143:0/952892408 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe9f80ab600 con 0x7fe9fc13e290 2023-11-26T07:18:25.074 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:25.069+0000 7fea014f1640 1 -- 172.21.15.143:0/952892408 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe9f80a7050 con 0x7fe9fc13e290 2023-11-26T07:18:25.074 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:25.069+0000 7fea014f1640 1 -- 172.21.15.143:0/952892408 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe9f80ab980 con 0x7fe9fc13e290 2023-11-26T07:18:25.075 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:25.069+0000 7fea04757640 1 -- 172.21.15.143:0/952892408 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fe9fc13e290 msgr2=0x7fe9fc0cf280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:18:25.075 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:25.069+0000 7fea04757640 1 --2- 172.21.15.143:0/952892408 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fe9fc13e290 0x7fe9fc0cf280 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7fe9f8009870 tx=0x7fe9f801c3d0 comp rx=0 tx=0).stop 2023-11-26T07:18:25.075 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:25.069+0000 7fea04757640 1 -- 172.21.15.143:0/952892408 shutdown_connections 2023-11-26T07:18:25.075 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:25.069+0000 7fea04757640 1 --2- 172.21.15.143:0/952892408 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fe9fc0cf7c0 0x7fe9fc15efa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:18:25.075 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:25.069+0000 7fea04757640 1 --2- 172.21.15.143:0/952892408 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fe9fc13e290 0x7fe9fc0cf280 unknown :-1 s=CLOSED pgs=232 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:18:25.076 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:25.069+0000 7fea04757640 1 -- 172.21.15.143:0/952892408 >> 172.21.15.143:0/952892408 conn(0x7fe9fc06a890 msgr2=0x7fe9fc06aca0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:18:25.076 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:25.069+0000 7fea04757640 1 -- 172.21.15.143:0/952892408 shutdown_connections 2023-11-26T07:18:25.076 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:25.069+0000 7fea04757640 1 -- 172.21.15.143:0/952892408 wait complete. 2023-11-26T07:18:25.077 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:25.073+0000 7fea04757640 1 Processor -- start 2023-11-26T07:18:25.077 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:25.073+0000 7fea04757640 1 -- start start 2023-11-26T07:18:25.079 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:25.073+0000 7fea04757640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fe9fc0cf7c0 0x7fe9fc13b6c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:18:25.079 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:25.073+0000 7fea04757640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fe9fc13e290 0x7fe9fc13aff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:18:25.079 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:25.073+0000 7fea04757640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fe9fc15f4e0 0x7fe9fc1f2d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:18:25.079 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:25.073+0000 7fea04757640 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe9fc13c9d0 con 0x7fe9fc0cf7c0 2023-11-26T07:18:25.079 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:25.073+0000 7fea04757640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe9fc1f3280 con 0x7fe9fc13e290 2023-11-26T07:18:25.080 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:25.073+0000 7fea04757640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe9fc13bdc0 con 0x7fe9fc15f4e0 2023-11-26T07:18:25.080 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:25.073+0000 7fea024f3640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fe9fc0cf7c0 0x7fe9fc13b6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:18:25.080 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:25.073+0000 7fea024f3640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fe9fc0cf7c0 0x7fe9fc13b6c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:33270/0 (socket says 172.21.15.143:33270) 2023-11-26T07:18:25.080 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:25.073+0000 7fea024f3640 1 -- 172.21.15.143:0/2638016987 learned_addr learned my addr 172.21.15.143:0/2638016987 (peer_addr_for_me v2:172.21.15.143:0/0) 2023-11-26T07:18:25.080 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:25.073+0000 7fea024f3640 1 -- 172.21.15.143:0/2638016987 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fe9fc15f4e0 msgr2=0x7fe9fc1f2d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:18:25.080 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:25.073+0000 7fea024f3640 1 --2- 172.21.15.143:0/2638016987 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fe9fc15f4e0 0x7fe9fc1f2d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:18:25.080 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:25.073+0000 7fea024f3640 1 -- 172.21.15.143:0/2638016987 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fe9fc13e290 msgr2=0x7fe9fc13aff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:18:25.080 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:25.073+0000 7fea024f3640 1 --2- 172.21.15.143:0/2638016987 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fe9fc13e290 0x7fe9fc13aff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:18:25.080 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:25.073+0000 7fea024f3640 1 -- 172.21.15.143:0/2638016987 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe9f80094d0 con 0x7fe9fc0cf7c0 2023-11-26T07:18:25.080 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:25.073+0000 7fea024f3640 1 --2- 172.21.15.143:0/2638016987 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fe9fc0cf7c0 0x7fe9fc13b6c0 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7fe9f80ae040 tx=0x7fe9f8019170 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:18:25.080 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:25.073+0000 7fe9f2ffd640 1 -- 172.21.15.143:0/2638016987 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe9f80b9070 con 0x7fe9fc0cf7c0 2023-11-26T07:18:25.080 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:25.073+0000 7fe9f2ffd640 1 -- 172.21.15.143:0/2638016987 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe9f80b4710 con 0x7fe9fc0cf7c0 2023-11-26T07:18:25.080 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:25.073+0000 7fea04757640 1 -- 172.21.15.143:0/2638016987 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe9fc13c0a0 con 0x7fe9fc0cf7c0 2023-11-26T07:18:25.081 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:25.073+0000 7fe9f2ffd640 1 -- 172.21.15.143:0/2638016987 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe9f80b4ae0 con 0x7fe9fc0cf7c0 2023-11-26T07:18:25.081 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:25.073+0000 7fea04757640 1 -- 172.21.15.143:0/2638016987 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe9fc13c660 con 0x7fe9fc0cf7c0 2023-11-26T07:18:25.081 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:25.073+0000 7fea04757640 1 -- 172.21.15.143:0/2638016987 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe9fc067680 con 0x7fe9fc0cf7c0 2023-11-26T07:18:25.081 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:25.077+0000 7fe9f2ffd640 1 -- 172.21.15.143:0/2638016987 <== mon.0 v2:172.21.15.143:3300/0 4 ==== mgrmap(e 5) v1 ==== 85852+0+0 (secure 0 0 0) 0x7fe9f80b2070 con 0x7fe9fc0cf7c0 2023-11-26T07:18:25.082 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:25.077+0000 7fe9f2ffd640 1 --2- 172.21.15.143:0/2638016987 >> [v2:172.21.15.143:6832/2150557544,v1:172.21.15.143:6833/2150557544] conn(0x7fe9e0066470 0x7fe9e0068930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:18:25.082 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:25.077+0000 7fe9f2ffd640 1 -- 172.21.15.143:0/2638016987 <== mon.0 v2:172.21.15.143:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fe9f811db00 con 0x7fe9fc0cf7c0 2023-11-26T07:18:25.082 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:25.077+0000 7fea01cf2640 1 --2- 172.21.15.143:0/2638016987 >> [v2:172.21.15.143:6832/2150557544,v1:172.21.15.143:6833/2150557544] conn(0x7fe9e0066470 0x7fe9e0068930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:18:25.082 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:25.077+0000 7fea01cf2640 1 --2- 172.21.15.143:0/2638016987 >> [v2:172.21.15.143:6832/2150557544,v1:172.21.15.143:6833/2150557544] conn(0x7fe9e0066470 0x7fe9e0068930 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7fe9fc0b42c0 tx=0x7fe9ec008040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:18:25.086 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:25.081+0000 7fe9f2ffd640 1 -- 172.21.15.143:0/2638016987 <== mon.0 v2:172.21.15.143:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172649 (secure 0 0 0) 0x7fe9f811fe60 con 0x7fe9fc0cf7c0 2023-11-26T07:18:25.236 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:25.229+0000 7fea04757640 1 -- 172.21.15.143:0/2638016987 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7fe9fc06df50 con 0x7fe9fc0cf7c0 2023-11-26T07:18:25.237 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:25.229+0000 7fe9f2ffd640 1 -- 172.21.15.143:0/2638016987 <== mon.0 v2:172.21.15.143:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v20) v1 ==== 104+0+11 (secure 0 0 0) 0x7fe9f80b00a0 con 0x7fe9fc0cf7c0 2023-11-26T07:18:25.237 INFO:teuthology.orchestra.run.smithi143.stdout:pg_num: 32 2023-11-26T07:18:25.238 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:25.233+0000 7fea04757640 1 -- 172.21.15.143:0/2638016987 >> [v2:172.21.15.143:6832/2150557544,v1:172.21.15.143:6833/2150557544] conn(0x7fe9e0066470 msgr2=0x7fe9e0068930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:18:25.238 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:25.233+0000 7fea04757640 1 --2- 172.21.15.143:0/2638016987 >> [v2:172.21.15.143:6832/2150557544,v1:172.21.15.143:6833/2150557544] conn(0x7fe9e0066470 0x7fe9e0068930 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7fe9fc0b42c0 tx=0x7fe9ec008040 comp rx=0 tx=0).stop 2023-11-26T07:18:25.238 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:25.233+0000 7fea04757640 1 -- 172.21.15.143:0/2638016987 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fe9fc0cf7c0 msgr2=0x7fe9fc13b6c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:18:25.238 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:25.233+0000 7fea04757640 1 --2- 172.21.15.143:0/2638016987 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fe9fc0cf7c0 0x7fe9fc13b6c0 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7fe9f80ae040 tx=0x7fe9f8019170 comp rx=0 tx=0).stop 2023-11-26T07:18:25.238 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:25.233+0000 7fea04757640 1 -- 172.21.15.143:0/2638016987 shutdown_connections 2023-11-26T07:18:25.239 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:25.233+0000 7fea04757640 1 --2- 172.21.15.143:0/2638016987 >> [v2:172.21.15.143:6832/2150557544,v1:172.21.15.143:6833/2150557544] conn(0x7fe9e0066470 0x7fe9e0068930 unknown :-1 s=CLOSED pgs=145 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:18:25.239 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:25.233+0000 7fea04757640 1 --2- 172.21.15.143:0/2638016987 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fe9fc15f4e0 0x7fe9fc1f2d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:18:25.239 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:25.233+0000 7fea04757640 1 --2- 172.21.15.143:0/2638016987 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fe9fc13e290 0x7fe9fc13aff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:18:25.239 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:25.233+0000 7fea04757640 1 --2- 172.21.15.143:0/2638016987 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fe9fc0cf7c0 0x7fe9fc13b6c0 unknown :-1 s=CLOSED pgs=233 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:18:25.239 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:25.233+0000 7fea04757640 1 -- 172.21.15.143:0/2638016987 >> 172.21.15.143:0/2638016987 conn(0x7fe9fc06a890 msgr2=0x7fe9fc0b0ce0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:18:25.239 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:25.233+0000 7fea04757640 1 -- 172.21.15.143:0/2638016987 shutdown_connections 2023-11-26T07:18:25.239 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:25.233+0000 7fea04757640 1 -- 172.21.15.143:0/2638016987 wait complete. 2023-11-26T07:18:25.257 DEBUG:teuthology.orchestra.run.smithi143:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-11-26T07:18:25.397 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:25.393+0000 7facabf0f640 1 Processor -- start 2023-11-26T07:18:25.397 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:25.393+0000 7facabf0f640 1 -- start start 2023-11-26T07:18:25.399 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:25.393+0000 7facabf0f640 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7faca40cf280 0x7faca40cf710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:18:25.399 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:25.393+0000 7facabf0f640 1 -- --> v1:172.21.15.143:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7faca40cc730 con 0x7faca40cd1b0 2023-11-26T07:18:25.399 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:25.393+0000 7facabf0f640 1 -- --> v1:172.21.15.188:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7faca4070ee0 con 0x7faca4143ac0 2023-11-26T07:18:25.399 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:25.393+0000 7facabf0f640 1 -- --> v2:172.21.15.143:3300/0 -- mon_getmap magic: 0 v1 -- 0x7faca4143ea0 con 0x7faca40cf280 2023-11-26T07:18:25.399 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:25.393+0000 7faca94aa640 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7faca40cf280 0x7faca40cf710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:18:25.399 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:25.393+0000 7facaa4ac640 1 --1- >> v1:172.21.15.143:6789/0 conn(0x7faca40cd1b0 0x7faca40cd5c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.143:6789/0 says I am v1:172.21.15.143:47544/0 (socket says 172.21.15.143:47544) 2023-11-26T07:18:25.399 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:25.393+0000 7facaa4ac640 1 -- 172.21.15.143:0/1128097452 learned_addr learned my addr 172.21.15.143:0/1128097452 (peer_addr_for_me v1:172.21.15.143:0/0) 2023-11-26T07:18:25.400 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:25.393+0000 7faca94aa640 1 -- 172.21.15.143:0/1128097452 >> v1:172.21.15.188:6789/0 conn(0x7faca4143ac0 legacy=0x7faca4070370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-26T07:18:25.400 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:25.393+0000 7faca94aa640 1 -- 172.21.15.143:0/1128097452 >> v1:172.21.15.143:6789/0 conn(0x7faca40cd1b0 legacy=0x7faca40cd5c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:18:25.400 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:25.393+0000 7faca94aa640 1 -- 172.21.15.143:0/1128097452 --> v2:172.21.15.143:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7faca40711c0 con 0x7faca40cf280 2023-11-26T07:18:25.402 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:25.397+0000 7faca94aa640 1 --2- 172.21.15.143:0/1128097452 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7faca40cf280 0x7faca40cf710 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7fac940097c0 tx=0x7fac9401c440 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3604bfc48b81b702 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:18:25.402 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:25.397+0000 7faca8ca9640 1 -- 172.21.15.143:0/1128097452 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fac94004230 con 0x7faca40cf280 2023-11-26T07:18:25.402 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:25.397+0000 7faca8ca9640 1 -- 172.21.15.143:0/1128097452 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fac940a7050 con 0x7faca40cf280 2023-11-26T07:18:25.402 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:25.397+0000 7faca8ca9640 1 -- 172.21.15.143:0/1128097452 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fac94003620 con 0x7faca40cf280 2023-11-26T07:18:25.403 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:25.397+0000 7facabf0f640 1 -- 172.21.15.143:0/1128097452 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7faca40cf280 msgr2=0x7faca40cf710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:18:25.403 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:25.397+0000 7facabf0f640 1 --2- 172.21.15.143:0/1128097452 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7faca40cf280 0x7faca40cf710 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7fac940097c0 tx=0x7fac9401c440 comp rx=0 tx=0).stop 2023-11-26T07:18:25.403 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:25.397+0000 7facabf0f640 1 -- 172.21.15.143:0/1128097452 shutdown_connections 2023-11-26T07:18:25.403 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:25.397+0000 7facabf0f640 1 --2- 172.21.15.143:0/1128097452 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7faca40cf280 0x7faca40cf710 unknown :-1 s=CLOSED pgs=235 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:18:25.403 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:25.397+0000 7facabf0f640 1 -- 172.21.15.143:0/1128097452 >> 172.21.15.143:0/1128097452 conn(0x7faca406a890 msgr2=0x7faca406aca0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:18:25.403 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:25.397+0000 7facabf0f640 1 -- 172.21.15.143:0/1128097452 shutdown_connections 2023-11-26T07:18:25.403 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:25.397+0000 7facabf0f640 1 -- 172.21.15.143:0/1128097452 wait complete. 2023-11-26T07:18:25.405 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:25.397+0000 7facabf0f640 1 Processor -- start 2023-11-26T07:18:25.405 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:25.401+0000 7facabf0f640 1 -- start start 2023-11-26T07:18:25.406 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:25.401+0000 7facabf0f640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7faca40cd1b0 0x7faca41564d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:18:25.406 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:25.401+0000 7facabf0f640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7faca40cf280 0x7faca413aa10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:18:25.407 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:25.401+0000 7facabf0f640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7faca4143ac0 0x7faca40a8e70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:18:25.407 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:25.401+0000 7facabf0f640 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faca413af50 con 0x7faca4143ac0 2023-11-26T07:18:25.407 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:25.401+0000 7facabf0f640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faca41572b0 con 0x7faca40cd1b0 2023-11-26T07:18:25.407 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:25.401+0000 7facabf0f640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7faca4157590 con 0x7faca40cf280 2023-11-26T07:18:25.407 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:25.401+0000 7facaa4ac640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7faca4143ac0 0x7faca40a8e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:18:25.407 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:25.401+0000 7facaa4ac640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7faca4143ac0 0x7faca40a8e70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:33280/0 (socket says 172.21.15.143:33280) 2023-11-26T07:18:25.407 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:25.401+0000 7facaa4ac640 1 -- 172.21.15.143:0/851732337 learned_addr learned my addr 172.21.15.143:0/851732337 (peer_addr_for_me v2:172.21.15.143:0/0) 2023-11-26T07:18:25.407 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:25.401+0000 7faca9cab640 1 --2- 172.21.15.143:0/851732337 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7faca40cd1b0 0x7faca41564d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:18:25.407 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:25.401+0000 7faca94aa640 1 --2- 172.21.15.143:0/851732337 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7faca40cf280 0x7faca413aa10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:18:25.407 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:25.401+0000 7facaa4ac640 1 -- 172.21.15.143:0/851732337 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7faca40cf280 msgr2=0x7faca413aa10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:18:25.407 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:25.401+0000 7facaa4ac640 1 --2- 172.21.15.143:0/851732337 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7faca40cf280 0x7faca413aa10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:18:25.407 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:25.401+0000 7facaa4ac640 1 -- 172.21.15.143:0/851732337 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7faca40cd1b0 msgr2=0x7faca41564d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:18:25.408 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:25.401+0000 7facaa4ac640 1 --2- 172.21.15.143:0/851732337 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7faca40cd1b0 0x7faca41564d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:18:25.408 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:25.401+0000 7facaa4ac640 1 -- 172.21.15.143:0/851732337 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fac94009420 con 0x7faca4143ac0 2023-11-26T07:18:25.408 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:25.401+0000 7facaa4ac640 1 --2- 172.21.15.143:0/851732337 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7faca4143ac0 0x7faca40a8e70 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7fac9c00cbc0 tx=0x7fac9c00a430 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:18:25.408 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:25.401+0000 7fac9a7fc640 1 -- 172.21.15.143:0/851732337 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fac9c00ad40 con 0x7faca4143ac0 2023-11-26T07:18:25.409 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:25.401+0000 7fac9a7fc640 1 -- 172.21.15.143:0/851732337 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fac9c012400 con 0x7faca4143ac0 2023-11-26T07:18:25.409 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:25.401+0000 7fac9a7fc640 1 -- 172.21.15.143:0/851732337 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fac9c00aea0 con 0x7faca4143ac0 2023-11-26T07:18:25.409 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:25.401+0000 7facabf0f640 1 -- 172.21.15.143:0/851732337 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7faca4157870 con 0x7faca4143ac0 2023-11-26T07:18:25.409 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:25.401+0000 7facabf0f640 1 -- 172.21.15.143:0/851732337 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7faca4157e30 con 0x7faca4143ac0 2023-11-26T07:18:25.409 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:25.401+0000 7facabf0f640 1 -- 172.21.15.143:0/851732337 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7faca4067680 con 0x7faca4143ac0 2023-11-26T07:18:25.409 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:25.405+0000 7fac9a7fc640 1 -- 172.21.15.143:0/851732337 <== mon.0 v2:172.21.15.143:3300/0 4 ==== mgrmap(e 5) v1 ==== 85852+0+0 (secure 0 0 0) 0x7fac9c007070 con 0x7faca4143ac0 2023-11-26T07:18:25.410 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:25.405+0000 7fac9a7fc640 1 --2- 172.21.15.143:0/851732337 >> [v2:172.21.15.143:6832/2150557544,v1:172.21.15.143:6833/2150557544] conn(0x7fac880664a0 0x7fac88068960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:18:25.410 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:25.405+0000 7fac9a7fc640 1 -- 172.21.15.143:0/851732337 <== mon.0 v2:172.21.15.143:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fac9c083c90 con 0x7faca4143ac0 2023-11-26T07:18:25.410 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:25.405+0000 7faca9cab640 1 --2- 172.21.15.143:0/851732337 >> [v2:172.21.15.143:6832/2150557544,v1:172.21.15.143:6833/2150557544] conn(0x7fac880664a0 0x7fac88068960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:18:25.410 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:25.405+0000 7faca9cab640 1 --2- 172.21.15.143:0/851732337 >> [v2:172.21.15.143:6832/2150557544,v1:172.21.15.143:6833/2150557544] conn(0x7fac880664a0 0x7fac88068960 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7faca00098e0 tx=0x7faca0009060 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:18:25.414 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:25.409+0000 7fac9a7fc640 1 -- 172.21.15.143:0/851732337 <== mon.0 v2:172.21.15.143:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172649 (secure 0 0 0) 0x7fac9c0511c0 con 0x7faca4143ac0 2023-11-26T07:18:25.567 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:25.561+0000 7facabf0f640 1 -- 172.21.15.143:0/851732337 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7faca415a370 con 0x7faca4143ac0 2023-11-26T07:18:25.568 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:25.561+0000 7fac9a7fc640 1 -- 172.21.15.143:0/851732337 <== mon.0 v2:172.21.15.143:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v20) v1 ==== 100+0+11 (secure 0 0 0) 0x7fac9c0170a0 con 0x7faca4143ac0 2023-11-26T07:18:25.568 INFO:teuthology.orchestra.run.smithi143.stdout:pg_num: 64 2023-11-26T07:18:25.569 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:25.561+0000 7facabf0f640 1 -- 172.21.15.143:0/851732337 >> [v2:172.21.15.143:6832/2150557544,v1:172.21.15.143:6833/2150557544] conn(0x7fac880664a0 msgr2=0x7fac88068960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:18:25.569 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:25.561+0000 7facabf0f640 1 --2- 172.21.15.143:0/851732337 >> [v2:172.21.15.143:6832/2150557544,v1:172.21.15.143:6833/2150557544] conn(0x7fac880664a0 0x7fac88068960 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7faca00098e0 tx=0x7faca0009060 comp rx=0 tx=0).stop 2023-11-26T07:18:25.569 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:25.561+0000 7facabf0f640 1 -- 172.21.15.143:0/851732337 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7faca4143ac0 msgr2=0x7faca40a8e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:18:25.570 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:25.561+0000 7facabf0f640 1 --2- 172.21.15.143:0/851732337 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7faca4143ac0 0x7faca40a8e70 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7fac9c00cbc0 tx=0x7fac9c00a430 comp rx=0 tx=0).stop 2023-11-26T07:18:25.570 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:25.561+0000 7facabf0f640 1 -- 172.21.15.143:0/851732337 shutdown_connections 2023-11-26T07:18:25.570 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:25.561+0000 7facabf0f640 1 --2- 172.21.15.143:0/851732337 >> [v2:172.21.15.143:6832/2150557544,v1:172.21.15.143:6833/2150557544] conn(0x7fac880664a0 0x7fac88068960 unknown :-1 s=CLOSED pgs=146 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:18:25.570 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:25.561+0000 7facabf0f640 1 --2- 172.21.15.143:0/851732337 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7faca4143ac0 0x7faca40a8e70 unknown :-1 s=CLOSED pgs=236 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:18:25.571 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:25.561+0000 7facabf0f640 1 --2- 172.21.15.143:0/851732337 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7faca40cf280 0x7faca413aa10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:18:25.571 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:25.561+0000 7facabf0f640 1 --2- 172.21.15.143:0/851732337 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7faca40cd1b0 0x7faca41564d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:18:25.571 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:25.561+0000 7facabf0f640 1 -- 172.21.15.143:0/851732337 >> 172.21.15.143:0/851732337 conn(0x7faca406a890 msgr2=0x7faca4159690 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:18:25.572 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:25.565+0000 7facabf0f640 1 -- 172.21.15.143:0/851732337 shutdown_connections 2023-11-26T07:18:25.572 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:25.565+0000 7facabf0f640 1 -- 172.21.15.143:0/851732337 wait complete. 2023-11-26T07:18:25.588 DEBUG:teuthology.orchestra.run.smithi143:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2023-11-26T07:18:25.730 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:25.721+0000 7f3090f65640 1 Processor -- start 2023-11-26T07:18:25.730 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:25.721+0000 7f3090f65640 1 -- start start 2023-11-26T07:18:25.731 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:25.725+0000 7f3090f65640 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7f308c0ce7c0 0x7f308c15f6b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:18:25.731 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:25.725+0000 7f3090f65640 1 -- --> v1:172.21.15.143:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f308c06c520 con 0x7f308c13e290 2023-11-26T07:18:25.731 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:25.725+0000 7f3090f65640 1 -- --> v1:172.21.15.188:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f308c06c800 con 0x7f308c0cf990 2023-11-26T07:18:25.731 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:25.725+0000 7f3090f65640 1 -- --> v2:172.21.15.143:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f308c06cae0 con 0x7f308c0ce7c0 2023-11-26T07:18:25.731 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:25.725+0000 7f308a59c640 1 --1- >> v1:172.21.15.143:6789/0 conn(0x7f308c13e290 0x7f308c0cf280 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.143:6789/0 says I am v1:172.21.15.143:47552/0 (socket says 172.21.15.143:47552) 2023-11-26T07:18:25.731 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:25.725+0000 7f308a59c640 1 -- 172.21.15.143:0/1504931225 learned_addr learned my addr 172.21.15.143:0/1504931225 (peer_addr_for_me v1:172.21.15.143:0/0) 2023-11-26T07:18:25.731 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:25.725+0000 7f308ad9d640 1 --2- 172.21.15.143:0/1504931225 >> v2:172.21.15.143:3300/0 conn(0x7f308c0ce7c0 0x7f308c15f6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:18:25.731 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:25.725+0000 7f3089d9b640 1 -- 172.21.15.143:0/1504931225 <== mon.0 v1:172.21.15.143:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1246516203 0 0) 0x7f308c06c520 con 0x7f308c13e290 2023-11-26T07:18:25.731 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:25.725+0000 7f3089d9b640 1 -- 172.21.15.143:0/1504931225 --> v1:172.21.15.143:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f3078003530 con 0x7f308c13e290 2023-11-26T07:18:25.732 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:25.725+0000 7f308ad9d640 1 -- 172.21.15.143:0/1504931225 >> v1:172.21.15.188:6789/0 conn(0x7f308c0cf990 legacy=0x7f308c15efa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:18:25.732 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:25.725+0000 7f308ad9d640 1 -- 172.21.15.143:0/1504931225 >> v1:172.21.15.143:6789/0 conn(0x7f308c13e290 legacy=0x7f308c0cf280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:18:25.732 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:25.725+0000 7f308ad9d640 1 -- 172.21.15.143:0/1504931225 --> v2:172.21.15.143:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f308c06cdc0 con 0x7f308c0ce7c0 2023-11-26T07:18:25.733 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:25.725+0000 7f308ad9d640 1 --2- 172.21.15.143:0/1504931225 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f308c0ce7c0 0x7f308c15f6b0 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7f30840097c0 tx=0x7f308401c3c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=66c3138b076578d5 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:18:25.733 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:25.729+0000 7f3089d9b640 1 -- 172.21.15.143:0/1504931225 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f30840ac640 con 0x7f308c0ce7c0 2023-11-26T07:18:25.733 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:25.729+0000 7f3089d9b640 1 -- 172.21.15.143:0/1504931225 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f30840ac7a0 con 0x7f308c0ce7c0 2023-11-26T07:18:25.734 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:25.729+0000 7f3089d9b640 1 -- 172.21.15.143:0/1504931225 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f30840acb30 con 0x7f308c0ce7c0 2023-11-26T07:18:25.735 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:25.729+0000 7f3090f65640 1 -- 172.21.15.143:0/1504931225 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f308c0ce7c0 msgr2=0x7f308c15f6b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:18:25.736 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:25.729+0000 7f3090f65640 1 --2- 172.21.15.143:0/1504931225 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f308c0ce7c0 0x7f308c15f6b0 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7f30840097c0 tx=0x7f308401c3c0 comp rx=0 tx=0).stop 2023-11-26T07:18:25.736 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:25.729+0000 7f3090f65640 1 -- 172.21.15.143:0/1504931225 shutdown_connections 2023-11-26T07:18:25.736 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:25.729+0000 7f3090f65640 1 --2- 172.21.15.143:0/1504931225 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f308c0ce7c0 0x7f308c15f6b0 unknown :-1 s=CLOSED pgs=238 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:18:25.736 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:25.729+0000 7f3090f65640 1 -- 172.21.15.143:0/1504931225 >> 172.21.15.143:0/1504931225 conn(0x7f308c06a890 msgr2=0x7f308c06aca0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:18:25.736 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:25.729+0000 7f3090f65640 1 -- 172.21.15.143:0/1504931225 shutdown_connections 2023-11-26T07:18:25.736 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:25.729+0000 7f3090f65640 1 -- 172.21.15.143:0/1504931225 wait complete. 2023-11-26T07:18:25.736 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:25.729+0000 7f3090f65640 1 Processor -- start 2023-11-26T07:18:25.736 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:25.729+0000 7f3090f65640 1 -- start start 2023-11-26T07:18:25.738 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:25.729+0000 7f3090f65640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f308c0ce7c0 0x7f308c135560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:18:25.738 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:25.729+0000 7f3090f65640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f308c0cf990 0x7f308c137110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:18:25.738 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:25.729+0000 7f3090f65640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f308c13e290 0x7f308c0b0d30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:18:25.739 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:25.729+0000 7f3090f65640 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f308c1520b0 con 0x7f308c0ce7c0 2023-11-26T07:18:25.739 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:25.729+0000 7f3090f65640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f308c152390 con 0x7f308c13e290 2023-11-26T07:18:25.739 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:25.729+0000 7f3090f65640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f308c152670 con 0x7f308c0cf990 2023-11-26T07:18:25.739 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:25.733+0000 7f308a59c640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f308c0ce7c0 0x7f308c135560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:18:25.740 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:25.733+0000 7f308a59c640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f308c0ce7c0 0x7f308c135560 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:33302/0 (socket says 172.21.15.143:33302) 2023-11-26T07:18:25.740 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:25.733+0000 7f308a59c640 1 -- 172.21.15.143:0/1270103065 learned_addr learned my addr 172.21.15.143:0/1270103065 (peer_addr_for_me v2:172.21.15.143:0/0) 2023-11-26T07:18:25.740 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:25.733+0000 7f3081d9b640 1 --2- 172.21.15.143:0/1270103065 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f308c0cf990 0x7f308c137110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:18:25.740 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:25.733+0000 7f308ad9d640 1 --2- 172.21.15.143:0/1270103065 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f308c13e290 0x7f308c0b0d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:18:25.740 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:25.733+0000 7f308a59c640 1 -- 172.21.15.143:0/1270103065 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f308c0cf990 msgr2=0x7f308c137110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:18:25.740 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:25.733+0000 7f308a59c640 1 --2- 172.21.15.143:0/1270103065 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f308c0cf990 0x7f308c137110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:18:25.741 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:25.733+0000 7f308a59c640 1 -- 172.21.15.143:0/1270103065 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f308c13e290 msgr2=0x7f308c0b0d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:18:25.741 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:25.733+0000 7f308a59c640 1 --2- 172.21.15.143:0/1270103065 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f308c13e290 0x7f308c0b0d30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:18:25.741 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:25.733+0000 7f308a59c640 1 -- 172.21.15.143:0/1270103065 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3084009420 con 0x7f308c0ce7c0 2023-11-26T07:18:25.741 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:25.733+0000 7f308a59c640 1 --2- 172.21.15.143:0/1270103065 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f308c0ce7c0 0x7f308c135560 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7f307c00cbc0 tx=0x7f307c00a430 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:18:25.741 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:25.733+0000 7f30837fe640 1 -- 172.21.15.143:0/1270103065 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f307c00ad40 con 0x7f308c0ce7c0 2023-11-26T07:18:25.742 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:25.733+0000 7f30837fe640 1 -- 172.21.15.143:0/1270103065 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f307c012400 con 0x7f308c0ce7c0 2023-11-26T07:18:25.742 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:25.733+0000 7f30837fe640 1 -- 172.21.15.143:0/1270103065 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f307c00aea0 con 0x7f308c0ce7c0 2023-11-26T07:18:25.742 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:25.733+0000 7f3090f65640 1 -- 172.21.15.143:0/1270103065 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f308c152950 con 0x7f308c0ce7c0 2023-11-26T07:18:25.743 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:25.733+0000 7f3090f65640 1 -- 172.21.15.143:0/1270103065 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f308c135aa0 con 0x7f308c0ce7c0 2023-11-26T07:18:25.745 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:25.733+0000 7f3090f65640 1 -- 172.21.15.143:0/1270103065 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f308c067680 con 0x7f308c0ce7c0 2023-11-26T07:18:25.745 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:25.733+0000 7f30837fe640 1 -- 172.21.15.143:0/1270103065 <== mon.0 v2:172.21.15.143:3300/0 4 ==== mgrmap(e 5) v1 ==== 85852+0+0 (secure 0 0 0) 0x7f307c007070 con 0x7f308c0ce7c0 2023-11-26T07:18:25.745 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:25.733+0000 7f30837fe640 1 --2- 172.21.15.143:0/1270103065 >> [v2:172.21.15.143:6832/2150557544,v1:172.21.15.143:6833/2150557544] conn(0x7f306c066470 0x7f306c068930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:18:25.745 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:25.737+0000 7f30837fe640 1 -- 172.21.15.143:0/1270103065 <== mon.0 v2:172.21.15.143:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f307c083e00 con 0x7f308c0ce7c0 2023-11-26T07:18:25.745 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:25.737+0000 7f3081d9b640 1 --2- 172.21.15.143:0/1270103065 >> [v2:172.21.15.143:6832/2150557544,v1:172.21.15.143:6833/2150557544] conn(0x7f306c066470 0x7f306c068930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:18:25.745 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:25.737+0000 7f3081d9b640 1 --2- 172.21.15.143:0/1270103065 >> [v2:172.21.15.143:6832/2150557544,v1:172.21.15.143:6833/2150557544] conn(0x7f306c066470 0x7f306c068930 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7f308c15e6a0 tx=0x7f3074009040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:18:25.746 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:25.737+0000 7f30837fe640 1 -- 172.21.15.143:0/1270103065 <== mon.0 v2:172.21.15.143:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172649 (secure 0 0 0) 0x7f307c051320 con 0x7f308c0ce7c0 2023-11-26T07:18:25.909 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:25.901+0000 7f3090f65640 1 -- 172.21.15.143:0/1270103065 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) v1 -- 0x7f308c067ec0 con 0x7f308c0ce7c0 2023-11-26T07:18:25.909 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:25.905+0000 7f30837fe640 1 -- 172.21.15.143:0/1270103065 <== mon.0 v2:172.21.15.143:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v20) v1 ==== 80+0+0 (secure 0 0 0) 0x7f307c017080 con 0x7f308c0ce7c0 2023-11-26T07:18:25.909 INFO:teuthology.orchestra.run.smithi143.stderr:listed 0 entries 2023-11-26T07:18:25.911 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:25.905+0000 7f3090f65640 1 -- 172.21.15.143:0/1270103065 >> [v2:172.21.15.143:6832/2150557544,v1:172.21.15.143:6833/2150557544] conn(0x7f306c066470 msgr2=0x7f306c068930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:18:25.912 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:25.905+0000 7f3090f65640 1 --2- 172.21.15.143:0/1270103065 >> [v2:172.21.15.143:6832/2150557544,v1:172.21.15.143:6833/2150557544] conn(0x7f306c066470 0x7f306c068930 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7f308c15e6a0 tx=0x7f3074009040 comp rx=0 tx=0).stop 2023-11-26T07:18:25.912 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:25.905+0000 7f3090f65640 1 -- 172.21.15.143:0/1270103065 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f308c0ce7c0 msgr2=0x7f308c135560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:18:25.912 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:25.905+0000 7f3090f65640 1 --2- 172.21.15.143:0/1270103065 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f308c0ce7c0 0x7f308c135560 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7f307c00cbc0 tx=0x7f307c00a430 comp rx=0 tx=0).stop 2023-11-26T07:18:25.912 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:25.905+0000 7f3090f65640 1 -- 172.21.15.143:0/1270103065 shutdown_connections 2023-11-26T07:18:25.913 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:25.905+0000 7f3090f65640 1 --2- 172.21.15.143:0/1270103065 >> [v2:172.21.15.143:6832/2150557544,v1:172.21.15.143:6833/2150557544] conn(0x7f306c066470 0x7f306c068930 unknown :-1 s=CLOSED pgs=147 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:18:25.913 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:25.905+0000 7f3090f65640 1 --2- 172.21.15.143:0/1270103065 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f308c13e290 0x7f308c0b0d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:18:25.913 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:25.905+0000 7f3090f65640 1 --2- 172.21.15.143:0/1270103065 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f308c0cf990 0x7f308c137110 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:18:25.913 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:25.905+0000 7f3090f65640 1 --2- 172.21.15.143:0/1270103065 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f308c0ce7c0 0x7f308c135560 unknown :-1 s=CLOSED pgs=239 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:18:25.913 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:25.905+0000 7f3090f65640 1 -- 172.21.15.143:0/1270103065 >> 172.21.15.143:0/1270103065 conn(0x7f308c06a890 msgr2=0x7f308c13a3e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:18:25.913 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:25.905+0000 7f3090f65640 1 -- 172.21.15.143:0/1270103065 shutdown_connections 2023-11-26T07:18:25.914 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:25.905+0000 7f3090f65640 1 -- 172.21.15.143:0/1270103065 wait complete. 2023-11-26T07:18:25.931 DEBUG:teuthology.orchestra.run.smithi143:> set -ex 2023-11-26T07:18:25.932 DEBUG:teuthology.orchestra.run.smithi143:> dd if=/proc/self/mounts of=/dev/stdout 2023-11-26T07:18:25.942 DEBUG:teuthology.orchestra.run.smithi143:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-11-26T07:18:26.121 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:26.113+0000 7f1ccdd80640 1 Processor -- start 2023-11-26T07:18:26.121 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:26.113+0000 7f1ccdd80640 1 -- start start 2023-11-26T07:18:26.123 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:26.117+0000 7f1ccdd80640 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7f1cc81596d0 0x7f1cc8071850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:18:26.123 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:26.117+0000 7f1ccdd80640 1 -- --> v1:172.21.15.143:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1cc806c750 con 0x7f1cc80b2f80 2023-11-26T07:18:26.123 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:26.117+0000 7f1ccdd80640 1 -- --> v1:172.21.15.188:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1cc80b3550 con 0x7f1cc80ce680 2023-11-26T07:18:26.124 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:26.117+0000 7f1ccdd80640 1 -- --> v2:172.21.15.188:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1cc80cbe00 con 0x7f1cc81596d0 2023-11-26T07:18:26.124 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:26.117+0000 7f1cc7fff640 1 --1- >> v1:172.21.15.143:6789/0 conn(0x7f1cc80b2f80 0x7f1cc806cbe0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.143:6789/0 says I am v1:172.21.15.143:47558/0 (socket says 172.21.15.143:47558) 2023-11-26T07:18:26.125 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:26.117+0000 7f1cc7fff640 1 -- 172.21.15.143:0/1152435010 learned_addr learned my addr 172.21.15.143:0/1152435010 (peer_addr_for_me v1:172.21.15.143:0/0) 2023-11-26T07:18:26.125 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:26.117+0000 7f1cc77fe640 1 --2- 172.21.15.143:0/1152435010 >> v2:172.21.15.188:3300/0 conn(0x7f1cc81596d0 0x7f1cc8071850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:18:26.126 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:26.117+0000 7f1cc67fc640 1 -- 172.21.15.143:0/1152435010 <== mon.0 v1:172.21.15.143:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2253123566 0 0) 0x7f1cc806c750 con 0x7f1cc80b2f80 2023-11-26T07:18:26.126 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:26.117+0000 7f1cc67fc640 1 -- 172.21.15.143:0/1152435010 --> v1:172.21.15.143:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f1cb0003530 con 0x7f1cc80b2f80 2023-11-26T07:18:26.127 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:26.117+0000 7f1cc77fe640 1 -- 172.21.15.143:0/1152435010 >> v1:172.21.15.188:6789/0 conn(0x7f1cc80ce680 legacy=0x7f1cc80b2870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:18:26.127 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:26.117+0000 7f1cc67fc640 1 -- 172.21.15.143:0/1152435010 <== mon.1 v1:172.21.15.188:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4149320861 0 0) 0x7f1cc80b3550 con 0x7f1cc80ce680 2023-11-26T07:18:26.127 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:26.117+0000 7f1cc77fe640 1 -- 172.21.15.143:0/1152435010 >> v1:172.21.15.143:6789/0 conn(0x7f1cc80b2f80 legacy=0x7f1cc806cbe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:18:26.127 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:26.117+0000 7f1cc77fe640 1 -- 172.21.15.143:0/1152435010 --> v2:172.21.15.188:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1cc8071e80 con 0x7f1cc81596d0 2023-11-26T07:18:26.129 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:26.121+0000 7f1cc77fe640 1 --2- 172.21.15.143:0/1152435010 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f1cc81596d0 0x7f1cc8071850 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7f1cb80097c0 tx=0x7f1cb801c2c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=ed828500f1cdbb0d server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:18:26.129 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:26.121+0000 7f1cc67fc640 1 -- 172.21.15.143:0/1152435010 <== mon.1 v2:172.21.15.188:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1cc0001a00 con 0x7f1cc81596d0 2023-11-26T07:18:26.129 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:26.121+0000 7f1cc67fc640 1 -- 172.21.15.143:0/1152435010 <== mon.1 v2:172.21.15.188:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1cb80a7050 con 0x7f1cc81596d0 2023-11-26T07:18:26.129 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:26.121+0000 7f1cc67fc640 1 -- 172.21.15.143:0/1152435010 <== mon.1 v2:172.21.15.188:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1cb80ab400 con 0x7f1cc81596d0 2023-11-26T07:18:26.129 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:26.121+0000 7f1ccdd80640 1 -- 172.21.15.143:0/1152435010 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f1cc81596d0 msgr2=0x7f1cc8071850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:18:26.130 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:26.121+0000 7f1ccdd80640 1 --2- 172.21.15.143:0/1152435010 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f1cc81596d0 0x7f1cc8071850 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7f1cb80097c0 tx=0x7f1cb801c2c0 comp rx=0 tx=0).stop 2023-11-26T07:18:26.130 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:26.121+0000 7f1ccdd80640 1 -- 172.21.15.143:0/1152435010 shutdown_connections 2023-11-26T07:18:26.130 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:26.121+0000 7f1ccdd80640 1 --2- 172.21.15.143:0/1152435010 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f1cc81596d0 0x7f1cc8071850 unknown :-1 s=CLOSED pgs=131 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:18:26.130 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:26.121+0000 7f1ccdd80640 1 -- 172.21.15.143:0/1152435010 >> 172.21.15.143:0/1152435010 conn(0x7f1cc806a890 msgr2=0x7f1cc806aca0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:18:26.130 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:26.121+0000 7f1ccdd80640 1 -- 172.21.15.143:0/1152435010 shutdown_connections 2023-11-26T07:18:26.130 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:26.121+0000 7f1ccdd80640 1 -- 172.21.15.143:0/1152435010 wait complete. 2023-11-26T07:18:26.130 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:26.125+0000 7f1ccdd80640 1 Processor -- start 2023-11-26T07:18:26.131 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:26.125+0000 7f1ccdd80640 1 -- start start 2023-11-26T07:18:26.132 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:26.125+0000 7f1ccdd80640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f1cc80b2f80 0x7f1cc80afc90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:18:26.133 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:26.125+0000 7f1ccdd80640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f1cc80ce680 0x7f1cc815b0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:18:26.133 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:26.125+0000 7f1ccdd80640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f1cc81596d0 0x7f1cc81f6140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:18:26.133 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:26.125+0000 7f1ccdd80640 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1cc8151440 con 0x7f1cc80ce680 2023-11-26T07:18:26.133 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:26.125+0000 7f1ccdd80640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1cc81f9ce0 con 0x7f1cc80b2f80 2023-11-26T07:18:26.133 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:26.125+0000 7f1ccdd80640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1cc81f9fc0 con 0x7f1cc81596d0 2023-11-26T07:18:26.133 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:26.125+0000 7f1cc6ffd640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f1cc80ce680 0x7f1cc815b0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:18:26.133 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:26.125+0000 7f1cc6ffd640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f1cc80ce680 0x7f1cc815b0d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:33310/0 (socket says 172.21.15.143:33310) 2023-11-26T07:18:26.133 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:26.125+0000 7f1cc6ffd640 1 -- 172.21.15.143:0/1135322415 learned_addr learned my addr 172.21.15.143:0/1135322415 (peer_addr_for_me v2:172.21.15.143:0/0) 2023-11-26T07:18:26.133 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:26.125+0000 7f1cc7fff640 1 --2- 172.21.15.143:0/1135322415 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f1cc81596d0 0x7f1cc81f6140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:18:26.133 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:26.125+0000 7f1cc77fe640 1 --2- 172.21.15.143:0/1135322415 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f1cc80b2f80 0x7f1cc80afc90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:18:26.134 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:26.125+0000 7f1cc6ffd640 1 -- 172.21.15.143:0/1135322415 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f1cc81596d0 msgr2=0x7f1cc81f6140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:18:26.134 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:26.125+0000 7f1cc6ffd640 1 --2- 172.21.15.143:0/1135322415 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f1cc81596d0 0x7f1cc81f6140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:18:26.134 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:26.125+0000 7f1cc6ffd640 1 -- 172.21.15.143:0/1135322415 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f1cc80b2f80 msgr2=0x7f1cc80afc90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:18:26.134 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:26.125+0000 7f1cc6ffd640 1 --2- 172.21.15.143:0/1135322415 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f1cc80b2f80 0x7f1cc80afc90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:18:26.134 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:26.125+0000 7f1cc6ffd640 1 -- 172.21.15.143:0/1135322415 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1cb8009420 con 0x7f1cc80ce680 2023-11-26T07:18:26.134 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:26.125+0000 7f1cc6ffd640 1 --2- 172.21.15.143:0/1135322415 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f1cc80ce680 0x7f1cc815b0d0 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7f1cbc00c410 tx=0x7f1cbc00c8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:18:26.134 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:26.125+0000 7f1caffff640 1 -- 172.21.15.143:0/1135322415 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1cbc0054e0 con 0x7f1cc80ce680 2023-11-26T07:18:26.134 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:26.125+0000 7f1caffff640 1 -- 172.21.15.143:0/1135322415 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1cbc003600 con 0x7f1cc80ce680 2023-11-26T07:18:26.134 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:26.125+0000 7f1ccdd80640 1 -- 172.21.15.143:0/1135322415 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1cc81fa2a0 con 0x7f1cc80ce680 2023-11-26T07:18:26.134 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:26.125+0000 7f1caffff640 1 -- 172.21.15.143:0/1135322415 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1cbc0039b0 con 0x7f1cc80ce680 2023-11-26T07:18:26.135 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:26.125+0000 7f1ccdd80640 1 -- 172.21.15.143:0/1135322415 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1cc80b01d0 con 0x7f1cc80ce680 2023-11-26T07:18:26.135 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:26.125+0000 7f1ccdd80640 1 -- 172.21.15.143:0/1135322415 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1cc8151580 con 0x7f1cc80ce680 2023-11-26T07:18:26.135 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:26.129+0000 7f1caffff640 1 -- 172.21.15.143:0/1135322415 <== mon.0 v2:172.21.15.143:3300/0 4 ==== mgrmap(e 5) v1 ==== 85852+0+0 (secure 0 0 0) 0x7f1cbc003b10 con 0x7f1cc80ce680 2023-11-26T07:18:26.136 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:26.129+0000 7f1caffff640 1 --2- 172.21.15.143:0/1135322415 >> [v2:172.21.15.143:6832/2150557544,v1:172.21.15.143:6833/2150557544] conn(0x7f1cb40664a0 0x7f1cb4068960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:18:26.136 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:26.129+0000 7f1caffff640 1 -- 172.21.15.143:0/1135322415 <== mon.0 v2:172.21.15.143:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f1cbc0833a0 con 0x7f1cc80ce680 2023-11-26T07:18:26.136 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:26.129+0000 7f1cc77fe640 1 --2- 172.21.15.143:0/1135322415 >> [v2:172.21.15.143:6832/2150557544,v1:172.21.15.143:6833/2150557544] conn(0x7f1cb40664a0 0x7f1cb4068960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:18:26.136 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:26.129+0000 7f1cc77fe640 1 --2- 172.21.15.143:0/1135322415 >> [v2:172.21.15.143:6832/2150557544,v1:172.21.15.143:6833/2150557544] conn(0x7f1cb40664a0 0x7f1cb4068960 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7f1cb80ad000 tx=0x7f1cb80a6000 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:18:26.139 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:26.133+0000 7f1caffff640 1 -- 172.21.15.143:0/1135322415 <== mon.0 v2:172.21.15.143:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172649 (secure 0 0 0) 0x7f1cc8151580 con 0x7f1cc80ce680 2023-11-26T07:18:26.289 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:26.285+0000 7f1ccdd80640 1 -- 172.21.15.143:0/1135322415 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f1cc8151580 con 0x7f1cc80ce680 2023-11-26T07:18:26.290 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:26.285+0000 7f1caffff640 1 -- 172.21.15.143:0/1135322415 <== mon.0 v2:172.21.15.143:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v20) v1 ==== 74+0+16541 (secure 0 0 0) 0x7f1cc8151580 con 0x7f1cc80ce680 2023-11-26T07:18:26.290 INFO:teuthology.orchestra.run.smithi143.stdout: 2023-11-26T07:18:26.291 INFO:teuthology.orchestra.run.smithi143.stdout:{"epoch":20,"fsid":"f5033060-2469-4a07-95fe-2b76f5404bc1","created":"2023-11-26T07:17:28.613663+0000","modified":"2023-11-26T07:17:55.505435+0000","last_up_change":"2023-11-26T07:17:41.157813+0000","last_in_change":"2023-11-26T07:17:31.181658+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-11-26T07:17:41.758597+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2023-11-26T07:17:51.878332+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2023-11-26T07:17:52.871396+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"ebbee2c6-023e-44b9-9f1d-6f7bfd309730","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6817","nonce":639807369},{"type":"v1","addr":"172.21.15.143:6819","nonce":639807369}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6821","nonce":639807369},{"type":"v1","addr":"172.21.15.143:6823","nonce":639807369}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6830","nonce":639807369},{"type":"v1","addr":"172.21.15.143:6831","nonce":639807369}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6825","nonce":639807369},{"type":"v1","addr":"172.21.15.143:6828","nonce":639807369}]},"public_addr":"172.21.15.143:6819/639807369","cluster_addr":"172.21.15.143:6823/639807369","heartbeat_back_addr":"172.21.15.143:6831/639807369","heartbeat_front_addr":"172.21.15.143:6828/639807369","state":["exists","up"]},{"osd":1,"uuid":"fec4b8be-7122-49e9-9a19-5850a0700001","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6816","nonce":1144942530},{"type":"v1","addr":"172.21.15.143:6818","nonce":1144942530}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6820","nonce":1144942530},{"type":"v1","addr":"172.21.15.143:6822","nonce":1144942530}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6827","nonce":1144942530},{"type":"v1","addr":"172.21.15.143:6829","nonce":1144942530}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6824","nonce":1144942530},{"type":"v1","addr":"172.21.15.143:6826","nonce":1144942530}]},"public_addr":"172.21.15.143:6818/1144942530","cluster_addr":"172.21.15.143:6822/1144942530","heartbeat_back_addr":"172.21.15.143:6829/1144942530","heartbeat_front_addr":"172.21.15.143:6826/1144942530","state":["exists","up"]},{"osd":2,"uuid":"2888af5b-f933-46ca-8f95-6dd494ff43f6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6808","nonce":3909095186},{"type":"v1","addr":"172.21.15.143:6809","nonce":3909095186}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6810","nonce":3909095186},{"type":"v1","addr":"172.21.15.143:6811","nonce":3909095186}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6814","nonce":3909095186},{"type":"v1","addr":"172.21.15.143:6815","nonce":3909095186}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6812","nonce":3909095186},{"type":"v1","addr":"172.21.15.143:6813","nonce":3909095186}]},"public_addr":"172.21.15.143:6809/3909095186","cluster_addr":"172.21.15.143:6811/3909095186","heartbeat_back_addr":"172.21.15.143:6815/3909095186","heartbeat_front_addr":"172.21.15.143:6813/3909095186","state":["exists","up"]},{"osd":3,"uuid":"d1f52435-7bc1-4081-9550-9cd1ba55f549","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6800","nonce":3854210991},{"type":"v1","addr":"172.21.15.143:6801","nonce":3854210991}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6802","nonce":3854210991},{"type":"v1","addr":"172.21.15.143:6803","nonce":3854210991}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6806","nonce":3854210991},{"type":"v1","addr":"172.21.15.143:6807","nonce":3854210991}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6804","nonce":3854210991},{"type":"v1","addr":"172.21.15.143:6805","nonce":3854210991}]},"public_addr":"172.21.15.143:6801/3854210991","cluster_addr":"172.21.15.143:6803/3854210991","heartbeat_back_addr":"172.21.15.143:6807/3854210991","heartbeat_front_addr":"172.21.15.143:6805/3854210991","state":["exists","up"]},{"osd":4,"uuid":"e2ae8b6e-6e25-49b2-bcbf-aacce68f06a3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6800","nonce":3924606902},{"type":"v1","addr":"172.21.15.188:6801","nonce":3924606902}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6802","nonce":3924606902},{"type":"v1","addr":"172.21.15.188:6803","nonce":3924606902}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6806","nonce":3924606902},{"type":"v1","addr":"172.21.15.188:6807","nonce":3924606902}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6804","nonce":3924606902},{"type":"v1","addr":"172.21.15.188:6805","nonce":3924606902}]},"public_addr":"172.21.15.188:6801/3924606902","cluster_addr":"172.21.15.188:6803/3924606902","heartbeat_back_addr":"172.21.15.188:6807/3924606902","heartbeat_front_addr":"172.21.15.188:6805/3924606902","state":["exists","up"]},{"osd":5,"uuid":"3cec8a32-7698-4c36-8e89-e4ba11934f33","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6813","nonce":186841544},{"type":"v1","addr":"172.21.15.188:6815","nonce":186841544}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6817","nonce":186841544},{"type":"v1","addr":"172.21.15.188:6818","nonce":186841544}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6822","nonce":186841544},{"type":"v1","addr":"172.21.15.188:6823","nonce":186841544}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6820","nonce":186841544},{"type":"v1","addr":"172.21.15.188:6821","nonce":186841544}]},"public_addr":"172.21.15.188:6815/186841544","cluster_addr":"172.21.15.188:6818/186841544","heartbeat_back_addr":"172.21.15.188:6823/186841544","heartbeat_front_addr":"172.21.15.188:6821/186841544","state":["exists","up"]},{"osd":6,"uuid":"58a65d5d-e090-473a-81b4-ad38e85029f9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6808","nonce":2153499405},{"type":"v1","addr":"172.21.15.188:6809","nonce":2153499405}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6810","nonce":2153499405},{"type":"v1","addr":"172.21.15.188:6811","nonce":2153499405}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6816","nonce":2153499405},{"type":"v1","addr":"172.21.15.188:6819","nonce":2153499405}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6812","nonce":2153499405},{"type":"v1","addr":"172.21.15.188:6814","nonce":2153499405}]},"public_addr":"172.21.15.188:6809/2153499405","cluster_addr":"172.21.15.188:6811/2153499405","heartbeat_back_addr":"172.21.15.188:6819/2153499405","heartbeat_front_addr":"172.21.15.188:6814/2153499405","state":["exists","up"]},{"osd":7,"uuid":"ea5b7f70-3554-4cf8-ac30-f7c59b80678d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6824","nonce":2722096729},{"type":"v1","addr":"172.21.15.188:6825","nonce":2722096729}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6826","nonce":2722096729},{"type":"v1","addr":"172.21.15.188:6827","nonce":2722096729}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6830","nonce":2722096729},{"type":"v1","addr":"172.21.15.188:6831","nonce":2722096729}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6828","nonce":2722096729},{"type":"v1","addr":"172.21.15.188:6829","nonce":2722096729}]},"public_addr":"172.21.15.188:6825/2722096729","cluster_addr":"172.21.15.188:6827/2722096729","heartbeat_back_addr":"172.21.15.188:6831/2722096729","heartbeat_front_addr":"172.21.15.188:6829/2722096729","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-26T07:17:36.995032+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-26T07:17:36.932266+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-26T07:17:36.829304+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-26T07:17:36.804297+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-26T07:17:36.667892+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-26T07:17:36.964868+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-26T07:17:36.827332+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-26T07:17:36.893179+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-11-26T07:18:26.292 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:26.285+0000 7f1ccdd80640 1 -- 172.21.15.143:0/1135322415 >> [v2:172.21.15.143:6832/2150557544,v1:172.21.15.143:6833/2150557544] conn(0x7f1cb40664a0 msgr2=0x7f1cb4068960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:18:26.292 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:26.285+0000 7f1ccdd80640 1 --2- 172.21.15.143:0/1135322415 >> [v2:172.21.15.143:6832/2150557544,v1:172.21.15.143:6833/2150557544] conn(0x7f1cb40664a0 0x7f1cb4068960 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7f1cb80ad000 tx=0x7f1cb80a6000 comp rx=0 tx=0).stop 2023-11-26T07:18:26.292 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:26.285+0000 7f1ccdd80640 1 -- 172.21.15.143:0/1135322415 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f1cc80ce680 msgr2=0x7f1cc815b0d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:18:26.292 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:26.285+0000 7f1ccdd80640 1 --2- 172.21.15.143:0/1135322415 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f1cc80ce680 0x7f1cc815b0d0 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7f1cbc00c410 tx=0x7f1cbc00c8d0 comp rx=0 tx=0).stop 2023-11-26T07:18:26.293 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:26.285+0000 7f1ccdd80640 1 -- 172.21.15.143:0/1135322415 shutdown_connections 2023-11-26T07:18:26.293 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:26.285+0000 7f1ccdd80640 1 --2- 172.21.15.143:0/1135322415 >> [v2:172.21.15.143:6832/2150557544,v1:172.21.15.143:6833/2150557544] conn(0x7f1cb40664a0 0x7f1cb4068960 unknown :-1 s=CLOSED pgs=148 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:18:26.293 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:26.285+0000 7f1ccdd80640 1 --2- 172.21.15.143:0/1135322415 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f1cc81596d0 0x7f1cc81f6140 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:18:26.293 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:26.285+0000 7f1ccdd80640 1 --2- 172.21.15.143:0/1135322415 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f1cc80ce680 0x7f1cc815b0d0 unknown :-1 s=CLOSED pgs=241 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:18:26.293 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:26.285+0000 7f1ccdd80640 1 --2- 172.21.15.143:0/1135322415 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f1cc80b2f80 0x7f1cc80afc90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:18:26.293 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:26.285+0000 7f1ccdd80640 1 -- 172.21.15.143:0/1135322415 >> 172.21.15.143:0/1135322415 conn(0x7f1cc806a890 msgr2=0x7f1cc814f7f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:18:26.293 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:26.285+0000 7f1ccdd80640 1 -- 172.21.15.143:0/1135322415 shutdown_connections 2023-11-26T07:18:26.293 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:26.285+0000 7f1ccdd80640 1 -- 172.21.15.143:0/1135322415 wait complete. 2023-11-26T07:18:26.311 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-11-26T07:17:41.758597+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2023-11-26T07:17:51.878332+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2023-11-26T07:17:52.871396+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '20', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2023-11-26T07:18:26.312 DEBUG:teuthology.orchestra.run.smithi143:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-11-26T07:18:26.451 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:26.445+0000 7f17fe957640 1 Processor -- start 2023-11-26T07:18:26.452 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:26.445+0000 7f17fe957640 1 -- start start 2023-11-26T07:18:26.454 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:26.445+0000 7f17fe957640 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7f17f80cf990 0x7f17f815efa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:18:26.454 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:26.449+0000 7f17fe957640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f17f815f4e0 0x7f17f80cd7b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:18:26.454 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:26.449+0000 7f17fe957640 1 -- --> v1:172.21.15.143:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f17f806c520 con 0x7f17f813e290 2023-11-26T07:18:26.454 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:26.449+0000 7f17fe957640 1 -- --> v2:172.21.15.143:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f17f806c800 con 0x7f17f80cf990 2023-11-26T07:18:26.454 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:26.449+0000 7f17fe957640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f17f806cae0 con 0x7f17f815f4e0 2023-11-26T07:18:26.454 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:26.449+0000 7f17f77fe640 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7f17f80cf990 0x7f17f815efa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:18:26.454 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:26.449+0000 7f17f77fe640 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7f17f80cf990 0x7f17f815efa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:33322/0 (socket says 172.21.15.143:33322) 2023-11-26T07:18:26.454 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:26.449+0000 7f17f7fff640 1 --1- >> v1:172.21.15.143:6789/0 conn(0x7f17f813e290 0x7f17f80cf280 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.143:6789/0 says I am v1:172.21.15.143:47562/0 (socket says 172.21.15.143:47562) 2023-11-26T07:18:26.454 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:26.449+0000 7f17f77fe640 1 -- 172.21.15.143:0/3974094188 learned_addr learned my addr 172.21.15.143:0/3974094188 (peer_addr_for_me v2:172.21.15.143:0/0) 2023-11-26T07:18:26.454 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:26.449+0000 7f17fcef4640 1 --2- 172.21.15.143:0/3974094188 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f17f815f4e0 0x7f17f80cd7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:18:26.455 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:26.449+0000 7f17f77fe640 1 -- 172.21.15.143:0/3974094188 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f17f815f4e0 msgr2=0x7f17f80cd7b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-26T07:18:26.455 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:26.449+0000 7f17f77fe640 1 --2- 172.21.15.143:0/3974094188 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f17f815f4e0 0x7f17f80cd7b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:18:26.455 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:26.449+0000 7f17f77fe640 1 -- 172.21.15.143:0/3974094188 >> v1:172.21.15.143:6789/0 conn(0x7f17f813e290 legacy=0x7f17f80cf280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:18:26.455 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:26.449+0000 7f17f77fe640 1 -- 172.21.15.143:0/3974094188 --> v2:172.21.15.143:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f17f806cdc0 con 0x7f17f80cf990 2023-11-26T07:18:26.455 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:26.449+0000 7f17fcef4640 1 --2- 172.21.15.143:0/3974094188 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f17f815f4e0 0x7f17f80cd7b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-26T07:18:26.455 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:26.449+0000 7f17f6ffd640 1 -- 172.21.15.143:0/3974094188 <== mon.0 v1:172.21.15.143:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3045730681 0 0) 0x7f17f806c520 con 0x7f17f813e290 2023-11-26T07:18:26.456 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:26.449+0000 7f17f77fe640 1 --2- 172.21.15.143:0/3974094188 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f17f80cf990 0x7f17f815efa0 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x7f17ec009870 tx=0x7f17ec01c510 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3ea8dbdc0551e52a server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:18:26.456 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:26.449+0000 7f17f6ffd640 1 -- 172.21.15.143:0/3974094188 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f17ec0a95f0 con 0x7f17f80cf990 2023-11-26T07:18:26.456 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:26.449+0000 7f17f6ffd640 1 -- 172.21.15.143:0/3974094188 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f17ec0a9750 con 0x7f17f80cf990 2023-11-26T07:18:26.457 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:26.449+0000 7f17f6ffd640 1 -- 172.21.15.143:0/3974094188 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f17ec0a9ae0 con 0x7f17f80cf990 2023-11-26T07:18:26.457 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:26.453+0000 7f17fe957640 1 -- 172.21.15.143:0/3974094188 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f17f80cf990 msgr2=0x7f17f815efa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:18:26.457 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:26.453+0000 7f17fe957640 1 --2- 172.21.15.143:0/3974094188 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f17f80cf990 0x7f17f815efa0 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x7f17ec009870 tx=0x7f17ec01c510 comp rx=0 tx=0).stop 2023-11-26T07:18:26.457 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:26.453+0000 7f17fe957640 1 -- 172.21.15.143:0/3974094188 shutdown_connections 2023-11-26T07:18:26.458 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:26.453+0000 7f17fe957640 1 --2- 172.21.15.143:0/3974094188 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f17f815f4e0 0x7f17f80cd7b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:18:26.458 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:26.453+0000 7f17fe957640 1 --2- 172.21.15.143:0/3974094188 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f17f80cf990 0x7f17f815efa0 unknown :-1 s=CLOSED pgs=243 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:18:26.458 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:26.453+0000 7f17fe957640 1 -- 172.21.15.143:0/3974094188 >> 172.21.15.143:0/3974094188 conn(0x7f17f806a890 msgr2=0x7f17f806aca0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:18:26.458 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:26.453+0000 7f17fe957640 1 -- 172.21.15.143:0/3974094188 shutdown_connections 2023-11-26T07:18:26.458 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:26.453+0000 7f17fe957640 1 -- 172.21.15.143:0/3974094188 wait complete. 2023-11-26T07:18:26.458 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:26.453+0000 7f17fe957640 1 Processor -- start 2023-11-26T07:18:26.459 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:26.453+0000 7f17fe957640 1 -- start start 2023-11-26T07:18:26.461 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:26.453+0000 7f17fe957640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f17f80cf990 0x7f17f81553f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:18:26.461 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:26.453+0000 7f17fe957640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f17f813e290 0x7f17f80aeb60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:18:26.462 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:26.453+0000 7f17fe957640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f17f815f4e0 0x7f17f80b2c40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:18:26.462 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:26.453+0000 7f17fe957640 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f17f80709a0 con 0x7f17f815f4e0 2023-11-26T07:18:26.462 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:26.453+0000 7f17fe957640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f17f8070370 con 0x7f17f813e290 2023-11-26T07:18:26.462 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:26.453+0000 7f17fe957640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f17f80af0a0 con 0x7f17f80cf990 2023-11-26T07:18:26.462 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:26.453+0000 7f17fcef4640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f17f815f4e0 0x7f17f80b2c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:18:26.462 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:26.453+0000 7f17fcef4640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f17f815f4e0 0x7f17f80b2c40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:33330/0 (socket says 172.21.15.143:33330) 2023-11-26T07:18:26.462 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:26.453+0000 7f17fcef4640 1 -- 172.21.15.143:0/2927688114 learned_addr learned my addr 172.21.15.143:0/2927688114 (peer_addr_for_me v2:172.21.15.143:0/0) 2023-11-26T07:18:26.462 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:26.453+0000 7f17f77fe640 1 --2- 172.21.15.143:0/2927688114 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f17f813e290 0x7f17f80aeb60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:18:26.462 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:26.453+0000 7f17f7fff640 1 --2- 172.21.15.143:0/2927688114 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f17f80cf990 0x7f17f81553f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:18:26.462 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:26.453+0000 7f17fcef4640 1 -- 172.21.15.143:0/2927688114 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f17f80cf990 msgr2=0x7f17f81553f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:18:26.462 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:26.453+0000 7f17fcef4640 1 --2- 172.21.15.143:0/2927688114 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f17f80cf990 0x7f17f81553f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:18:26.463 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:26.457+0000 7f17fcef4640 1 -- 172.21.15.143:0/2927688114 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f17f813e290 msgr2=0x7f17f80aeb60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:18:26.463 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:26.457+0000 7f17fcef4640 1 --2- 172.21.15.143:0/2927688114 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f17f813e290 0x7f17f80aeb60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:18:26.463 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:26.457+0000 7f17fcef4640 1 -- 172.21.15.143:0/2927688114 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f17ec0094d0 con 0x7f17f815f4e0 2023-11-26T07:18:26.463 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:26.457+0000 7f17f7fff640 1 --2- 172.21.15.143:0/2927688114 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f17f80cf990 0x7f17f81553f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-26T07:18:26.463 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:26.457+0000 7f17fcef4640 1 --2- 172.21.15.143:0/2927688114 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f17f815f4e0 0x7f17f80b2c40 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x7f17f0007f50 tx=0x7f17f000d560 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:18:26.463 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:26.457+0000 7f17f4ff9640 1 -- 172.21.15.143:0/2927688114 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f17f0003ba0 con 0x7f17f815f4e0 2023-11-26T07:18:26.463 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:26.457+0000 7f17f4ff9640 1 -- 172.21.15.143:0/2927688114 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f17f000f040 con 0x7f17f815f4e0 2023-11-26T07:18:26.463 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:26.457+0000 7f17f4ff9640 1 -- 172.21.15.143:0/2927688114 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f17f000dec0 con 0x7f17f815f4e0 2023-11-26T07:18:26.463 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:26.457+0000 7f17fe957640 1 -- 172.21.15.143:0/2927688114 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f17f813cda0 con 0x7f17f815f4e0 2023-11-26T07:18:26.463 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:26.457+0000 7f17fe957640 1 -- 172.21.15.143:0/2927688114 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f17f80705c0 con 0x7f17f815f4e0 2023-11-26T07:18:26.464 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:26.457+0000 7f17fe957640 1 -- 172.21.15.143:0/2927688114 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f17f8070ae0 con 0x7f17f815f4e0 2023-11-26T07:18:26.464 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:26.457+0000 7f17f4ff9640 1 -- 172.21.15.143:0/2927688114 <== mon.0 v2:172.21.15.143:3300/0 4 ==== mgrmap(e 5) v1 ==== 85852+0+0 (secure 0 0 0) 0x7f17f0004030 con 0x7f17f815f4e0 2023-11-26T07:18:26.464 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:26.457+0000 7f17f4ff9640 1 --2- 172.21.15.143:0/2927688114 >> [v2:172.21.15.143:6832/2150557544,v1:172.21.15.143:6833/2150557544] conn(0x7f17e4066470 0x7f17e4068930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:18:26.464 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:26.457+0000 7f17f4ff9640 1 -- 172.21.15.143:0/2927688114 <== mon.0 v2:172.21.15.143:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f17f0084150 con 0x7f17f815f4e0 2023-11-26T07:18:26.465 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:26.457+0000 7f17f7fff640 1 --2- 172.21.15.143:0/2927688114 >> [v2:172.21.15.143:6832/2150557544,v1:172.21.15.143:6833/2150557544] conn(0x7f17e4066470 0x7f17e4068930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:18:26.465 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:26.461+0000 7f17f7fff640 1 --2- 172.21.15.143:0/2927688114 >> [v2:172.21.15.143:6832/2150557544,v1:172.21.15.143:6833/2150557544] conn(0x7f17e4066470 0x7f17e4068930 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7f17e8007fd0 tx=0x7f17e8007c20 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:18:26.468 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:26.461+0000 7f17f4ff9640 1 -- 172.21.15.143:0/2927688114 <== mon.0 v2:172.21.15.143:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172649 (secure 0 0 0) 0x7f17f8070ae0 con 0x7f17f815f4e0 2023-11-26T07:18:26.622 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:26.617+0000 7f17fe957640 1 -- 172.21.15.143:0/2927688114 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f17f8070ae0 con 0x7f17f815f4e0 2023-11-26T07:18:26.623 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:26.617+0000 7f17f4ff9640 1 -- 172.21.15.143:0/2927688114 <== mon.0 v2:172.21.15.143:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v20) v1 ==== 93+0+10 (secure 0 0 0) 0x7f17f8070ae0 con 0x7f17f815f4e0 2023-11-26T07:18:26.623 INFO:teuthology.orchestra.run.smithi143.stdout:pg_num: 1 2023-11-26T07:18:26.624 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:26.617+0000 7f17fe957640 1 -- 172.21.15.143:0/2927688114 >> [v2:172.21.15.143:6832/2150557544,v1:172.21.15.143:6833/2150557544] conn(0x7f17e4066470 msgr2=0x7f17e4068930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:18:26.624 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:26.617+0000 7f17fe957640 1 --2- 172.21.15.143:0/2927688114 >> [v2:172.21.15.143:6832/2150557544,v1:172.21.15.143:6833/2150557544] conn(0x7f17e4066470 0x7f17e4068930 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7f17e8007fd0 tx=0x7f17e8007c20 comp rx=0 tx=0).stop 2023-11-26T07:18:26.624 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:26.617+0000 7f17fe957640 1 -- 172.21.15.143:0/2927688114 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f17f815f4e0 msgr2=0x7f17f80b2c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:18:26.624 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:26.617+0000 7f17fe957640 1 --2- 172.21.15.143:0/2927688114 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f17f815f4e0 0x7f17f80b2c40 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x7f17f0007f50 tx=0x7f17f000d560 comp rx=0 tx=0).stop 2023-11-26T07:18:26.624 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:26.617+0000 7f17fe957640 1 -- 172.21.15.143:0/2927688114 shutdown_connections 2023-11-26T07:18:26.625 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:26.617+0000 7f17fe957640 1 --2- 172.21.15.143:0/2927688114 >> [v2:172.21.15.143:6832/2150557544,v1:172.21.15.143:6833/2150557544] conn(0x7f17e4066470 0x7f17e4068930 unknown :-1 s=CLOSED pgs=149 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:18:26.625 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:26.617+0000 7f17fe957640 1 --2- 172.21.15.143:0/2927688114 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f17f815f4e0 0x7f17f80b2c40 unknown :-1 s=CLOSED pgs=244 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:18:26.625 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:26.617+0000 7f17fe957640 1 --2- 172.21.15.143:0/2927688114 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f17f813e290 0x7f17f80aeb60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:18:26.625 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:26.617+0000 7f17fe957640 1 --2- 172.21.15.143:0/2927688114 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f17f80cf990 0x7f17f81553f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:18:26.625 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:26.617+0000 7f17fe957640 1 -- 172.21.15.143:0/2927688114 >> 172.21.15.143:0/2927688114 conn(0x7f17f806a890 msgr2=0x7f17f80b1cb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:18:26.625 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:26.617+0000 7f17fe957640 1 -- 172.21.15.143:0/2927688114 shutdown_connections 2023-11-26T07:18:26.625 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:26.617+0000 7f17fe957640 1 -- 172.21.15.143:0/2927688114 wait complete. 2023-11-26T07:18:26.643 DEBUG:teuthology.orchestra.run.smithi143:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-11-26T07:18:26.785 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:26.781+0000 7ff152e5a640 1 Processor -- start 2023-11-26T07:18:26.785 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:26.781+0000 7ff152e5a640 1 -- start start 2023-11-26T07:18:26.786 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:26.781+0000 7ff152e5a640 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7ff14c143ac0 0x7ff14c070370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:18:26.786 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:26.781+0000 7ff152e5a640 1 -- --> v1:172.21.15.143:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff14c0cc730 con 0x7ff14c0708b0 2023-11-26T07:18:26.786 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:26.781+0000 7ff152e5a640 1 -- --> v1:172.21.15.188:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff14c070ee0 con 0x7ff14c0cf990 2023-11-26T07:18:26.786 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:26.781+0000 7ff152e5a640 1 -- --> v2:172.21.15.188:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff14c143ea0 con 0x7ff14c143ac0 2023-11-26T07:18:26.787 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:26.781+0000 7ff143fff640 1 --1- >> v1:172.21.15.143:6789/0 conn(0x7ff14c0708b0 0x7ff14c0cf280 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.143:6789/0 says I am v1:172.21.15.143:47574/0 (socket says 172.21.15.143:47574) 2023-11-26T07:18:26.787 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:26.781+0000 7ff143fff640 1 -- 172.21.15.143:0/2037224396 learned_addr learned my addr 172.21.15.143:0/2037224396 (peer_addr_for_me v1:172.21.15.143:0/0) 2023-11-26T07:18:26.787 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:26.781+0000 7ff150bf6640 1 --2- 172.21.15.143:0/2037224396 >> v2:172.21.15.188:3300/0 conn(0x7ff14c143ac0 0x7ff14c070370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:18:26.787 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:26.781+0000 7ff1437fe640 1 -- 172.21.15.143:0/2037224396 <== mon.0 v1:172.21.15.143:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1770158775 0 0) 0x7ff14c0cc730 con 0x7ff14c0708b0 2023-11-26T07:18:26.788 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:26.781+0000 7ff1437fe640 1 -- 172.21.15.143:0/2037224396 --> v1:172.21.15.143:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7ff134003530 con 0x7ff14c0708b0 2023-11-26T07:18:26.788 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:26.781+0000 7ff1437fe640 1 -- 172.21.15.143:0/2037224396 <== mon.1 v1:172.21.15.188:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3328168264 0 0) 0x7ff14c070ee0 con 0x7ff14c0cf990 2023-11-26T07:18:26.788 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:26.781+0000 7ff1437fe640 1 -- 172.21.15.143:0/2037224396 --> v1:172.21.15.188:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7ff14c0cc730 con 0x7ff14c0cf990 2023-11-26T07:18:26.788 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:26.781+0000 7ff150bf6640 1 -- 172.21.15.143:0/2037224396 >> v1:172.21.15.188:6789/0 conn(0x7ff14c0cf990 legacy=0x7ff14c0cd1b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:18:26.788 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:26.781+0000 7ff150bf6640 1 -- 172.21.15.143:0/2037224396 >> v1:172.21.15.143:6789/0 conn(0x7ff14c0708b0 legacy=0x7ff14c0cf280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:18:26.788 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:26.781+0000 7ff150bf6640 1 -- 172.21.15.143:0/2037224396 --> v2:172.21.15.188:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff14c0711c0 con 0x7ff14c143ac0 2023-11-26T07:18:26.790 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:26.785+0000 7ff150bf6640 1 --2- 172.21.15.143:0/2037224396 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7ff14c143ac0 0x7ff14c070370 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7ff1480097c0 tx=0x7ff14801c480 comp rx=0 tx=0).ready entity=mon.1 client_cookie=97af6e411e2388ec server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:18:26.790 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:26.785+0000 7ff1437fe640 1 -- 172.21.15.143:0/2037224396 <== mon.1 v2:172.21.15.188:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff1480a8050 con 0x7ff14c143ac0 2023-11-26T07:18:26.790 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:26.785+0000 7ff1437fe640 1 -- 172.21.15.143:0/2037224396 <== mon.1 v2:172.21.15.188:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff1480ac5f0 con 0x7ff14c143ac0 2023-11-26T07:18:26.791 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:26.785+0000 7ff1437fe640 1 -- 172.21.15.143:0/2037224396 <== mon.1 v2:172.21.15.188:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff1480ac980 con 0x7ff14c143ac0 2023-11-26T07:18:26.791 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:26.785+0000 7ff152e5a640 1 -- 172.21.15.143:0/2037224396 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7ff14c143ac0 msgr2=0x7ff14c070370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:18:26.791 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:26.785+0000 7ff152e5a640 1 --2- 172.21.15.143:0/2037224396 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7ff14c143ac0 0x7ff14c070370 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7ff1480097c0 tx=0x7ff14801c480 comp rx=0 tx=0).stop 2023-11-26T07:18:26.791 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:26.785+0000 7ff152e5a640 1 -- 172.21.15.143:0/2037224396 shutdown_connections 2023-11-26T07:18:26.792 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:26.785+0000 7ff152e5a640 1 --2- 172.21.15.143:0/2037224396 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7ff14c143ac0 0x7ff14c070370 unknown :-1 s=CLOSED pgs=133 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:18:26.792 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:26.785+0000 7ff152e5a640 1 -- 172.21.15.143:0/2037224396 >> 172.21.15.143:0/2037224396 conn(0x7ff14c06a890 msgr2=0x7ff14c06aca0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:18:26.792 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:26.785+0000 7ff152e5a640 1 -- 172.21.15.143:0/2037224396 shutdown_connections 2023-11-26T07:18:26.792 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:26.785+0000 7ff152e5a640 1 -- 172.21.15.143:0/2037224396 wait complete. 2023-11-26T07:18:26.793 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:26.789+0000 7ff152e5a640 1 Processor -- start 2023-11-26T07:18:26.793 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:26.789+0000 7ff152e5a640 1 -- start start 2023-11-26T07:18:26.795 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:26.789+0000 7ff152e5a640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7ff14c0708b0 0x7ff14c13a640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:18:26.795 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:26.789+0000 7ff152e5a640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7ff14c0cf990 0x7ff14c0b0c70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:18:26.796 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:26.789+0000 7ff152e5a640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7ff14c143ac0 0x7ff14c0b11b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:18:26.796 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:26.789+0000 7ff152e5a640 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff14c0af710 con 0x7ff14c143ac0 2023-11-26T07:18:26.796 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:26.789+0000 7ff152e5a640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff14c0af9f0 con 0x7ff14c0cf990 2023-11-26T07:18:26.796 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:26.789+0000 7ff152e5a640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff14c0afcd0 con 0x7ff14c0708b0 2023-11-26T07:18:26.796 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:26.789+0000 7ff1513f7640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7ff14c143ac0 0x7ff14c0b11b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:18:26.796 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:26.789+0000 7ff1513f7640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7ff14c143ac0 0x7ff14c0b11b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:33340/0 (socket says 172.21.15.143:33340) 2023-11-26T07:18:26.796 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:26.789+0000 7ff1513f7640 1 -- 172.21.15.143:0/1538080158 learned_addr learned my addr 172.21.15.143:0/1538080158 (peer_addr_for_me v2:172.21.15.143:0/0) 2023-11-26T07:18:26.796 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:26.789+0000 7ff143fff640 1 --2- 172.21.15.143:0/1538080158 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7ff14c0cf990 0x7ff14c0b0c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:18:26.796 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:26.789+0000 7ff150bf6640 1 --2- 172.21.15.143:0/1538080158 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7ff14c0708b0 0x7ff14c13a640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:18:26.796 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:26.789+0000 7ff1513f7640 1 -- 172.21.15.143:0/1538080158 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7ff14c0708b0 msgr2=0x7ff14c13a640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:18:26.796 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:26.789+0000 7ff1513f7640 1 --2- 172.21.15.143:0/1538080158 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7ff14c0708b0 0x7ff14c13a640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:18:26.797 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:26.789+0000 7ff1513f7640 1 -- 172.21.15.143:0/1538080158 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7ff14c0cf990 msgr2=0x7ff14c0b0c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:18:26.797 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:26.789+0000 7ff1513f7640 1 --2- 172.21.15.143:0/1538080158 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7ff14c0cf990 0x7ff14c0b0c70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:18:26.797 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:26.789+0000 7ff1513f7640 1 -- 172.21.15.143:0/1538080158 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff148009420 con 0x7ff14c143ac0 2023-11-26T07:18:26.797 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:26.789+0000 7ff1513f7640 1 --2- 172.21.15.143:0/1538080158 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7ff14c143ac0 0x7ff14c0b11b0 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x7ff14400c790 tx=0x7ff14400cc50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:18:26.797 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:26.789+0000 7ff1417fa640 1 -- 172.21.15.143:0/1538080158 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff14400cea0 con 0x7ff14c143ac0 2023-11-26T07:18:26.797 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:26.789+0000 7ff1417fa640 1 -- 172.21.15.143:0/1538080158 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff144003b80 con 0x7ff14c143ac0 2023-11-26T07:18:26.797 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:26.789+0000 7ff1417fa640 1 -- 172.21.15.143:0/1538080158 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff14400a850 con 0x7ff14c143ac0 2023-11-26T07:18:26.797 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:26.789+0000 7ff152e5a640 1 -- 172.21.15.143:0/1538080158 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff14c0affb0 con 0x7ff14c143ac0 2023-11-26T07:18:26.798 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:26.789+0000 7ff152e5a640 1 -- 172.21.15.143:0/1538080158 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff14c0af400 con 0x7ff14c143ac0 2023-11-26T07:18:26.798 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:26.793+0000 7ff152e5a640 1 -- 172.21.15.143:0/1538080158 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff14c0b04d0 con 0x7ff14c143ac0 2023-11-26T07:18:26.798 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:26.793+0000 7ff1417fa640 1 -- 172.21.15.143:0/1538080158 <== mon.0 v2:172.21.15.143:3300/0 4 ==== mgrmap(e 5) v1 ==== 85852+0+0 (secure 0 0 0) 0x7ff14400a9b0 con 0x7ff14c143ac0 2023-11-26T07:18:26.798 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:26.793+0000 7ff1417fa640 1 --2- 172.21.15.143:0/1538080158 >> [v2:172.21.15.143:6832/2150557544,v1:172.21.15.143:6833/2150557544] conn(0x7ff1380664a0 0x7ff138068960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:18:26.799 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:26.793+0000 7ff1417fa640 1 -- 172.21.15.143:0/1538080158 <== mon.0 v2:172.21.15.143:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5709+0+0 (secure 0 0 0) 0x7ff144083bf0 con 0x7ff14c143ac0 2023-11-26T07:18:26.799 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:26.793+0000 7ff150bf6640 1 --2- 172.21.15.143:0/1538080158 >> [v2:172.21.15.143:6832/2150557544,v1:172.21.15.143:6833/2150557544] conn(0x7ff1380664a0 0x7ff138068960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:18:26.799 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:26.793+0000 7ff150bf6640 1 --2- 172.21.15.143:0/1538080158 >> [v2:172.21.15.143:6832/2150557544,v1:172.21.15.143:6833/2150557544] conn(0x7ff1380664a0 0x7ff138068960 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7ff1480ae000 tx=0x7ff1480a7000 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:18:26.802 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:26.797+0000 7ff1417fa640 1 -- 172.21.15.143:0/1538080158 <== mon.0 v2:172.21.15.143:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172649 (secure 0 0 0) 0x7ff14c0b04d0 con 0x7ff14c143ac0 2023-11-26T07:18:26.953 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:26.945+0000 7ff152e5a640 1 -- 172.21.15.143:0/1538080158 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7ff14c0b04d0 con 0x7ff14c143ac0 2023-11-26T07:18:26.953 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:26.945+0000 7ff1417fa640 1 -- 172.21.15.143:0/1538080158 <== mon.0 v2:172.21.15.143:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v20) v1 ==== 104+0+11 (secure 0 0 0) 0x7ff14c0b04d0 con 0x7ff14c143ac0 2023-11-26T07:18:26.953 INFO:teuthology.orchestra.run.smithi143.stdout:pg_num: 32 2023-11-26T07:18:26.954 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:26.949+0000 7ff152e5a640 1 -- 172.21.15.143:0/1538080158 >> [v2:172.21.15.143:6832/2150557544,v1:172.21.15.143:6833/2150557544] conn(0x7ff1380664a0 msgr2=0x7ff138068960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:18:26.955 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:26.949+0000 7ff152e5a640 1 --2- 172.21.15.143:0/1538080158 >> [v2:172.21.15.143:6832/2150557544,v1:172.21.15.143:6833/2150557544] conn(0x7ff1380664a0 0x7ff138068960 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7ff1480ae000 tx=0x7ff1480a7000 comp rx=0 tx=0).stop 2023-11-26T07:18:26.955 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:26.949+0000 7ff152e5a640 1 -- 172.21.15.143:0/1538080158 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7ff14c143ac0 msgr2=0x7ff14c0b11b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:18:26.955 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:26.949+0000 7ff152e5a640 1 --2- 172.21.15.143:0/1538080158 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7ff14c143ac0 0x7ff14c0b11b0 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x7ff14400c790 tx=0x7ff14400cc50 comp rx=0 tx=0).stop 2023-11-26T07:18:26.955 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:26.949+0000 7ff152e5a640 1 -- 172.21.15.143:0/1538080158 shutdown_connections 2023-11-26T07:18:26.955 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:26.949+0000 7ff152e5a640 1 --2- 172.21.15.143:0/1538080158 >> [v2:172.21.15.143:6832/2150557544,v1:172.21.15.143:6833/2150557544] conn(0x7ff1380664a0 0x7ff138068960 unknown :-1 s=CLOSED pgs=150 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:18:26.955 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:26.949+0000 7ff152e5a640 1 --2- 172.21.15.143:0/1538080158 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7ff14c143ac0 0x7ff14c0b11b0 unknown :-1 s=CLOSED pgs=246 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:18:26.955 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:26.949+0000 7ff152e5a640 1 --2- 172.21.15.143:0/1538080158 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7ff14c0cf990 0x7ff14c0b0c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:18:26.955 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:26.949+0000 7ff152e5a640 1 --2- 172.21.15.143:0/1538080158 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7ff14c0708b0 0x7ff14c13a640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:18:26.955 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:26.949+0000 7ff152e5a640 1 -- 172.21.15.143:0/1538080158 >> 172.21.15.143:0/1538080158 conn(0x7ff14c06a890 msgr2=0x7ff14c15cd70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:18:26.955 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:26.949+0000 7ff152e5a640 1 -- 172.21.15.143:0/1538080158 shutdown_connections 2023-11-26T07:18:26.955 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:26.949+0000 7ff152e5a640 1 -- 172.21.15.143:0/1538080158 wait complete. 2023-11-26T07:18:26.973 DEBUG:teuthology.orchestra.run.smithi143:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-11-26T07:18:27.113 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:27.105+0000 7f60f1bb4640 1 Processor -- start 2023-11-26T07:18:27.113 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:27.109+0000 7f60f1bb4640 1 -- start start 2023-11-26T07:18:27.115 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:27.109+0000 7f60f1bb4640 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7f60ec143810 0x7f60ec072140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:18:27.115 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:27.109+0000 7f60f1bb4640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f60ec0b1e10 0x7f60ec15d9e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:18:27.116 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:27.109+0000 7f60f1bb4640 1 -- --> v1:172.21.15.143:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f60ec15e860 con 0x7f60ec15df20 2023-11-26T07:18:27.116 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:27.109+0000 7f60f1bb4640 1 -- --> v2:172.21.15.143:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f60ec15e3b0 con 0x7f60ec143810 2023-11-26T07:18:27.116 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:27.109+0000 7f60f1bb4640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f60ec0cdba0 con 0x7f60ec0b1e10 2023-11-26T07:18:27.116 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:27.109+0000 7f60f0bb2640 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7f60ec143810 0x7f60ec072140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:18:27.116 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:27.109+0000 7f60f13b3640 1 --1- >> v1:172.21.15.143:6789/0 conn(0x7f60ec15df20 0x7f60ec0ce030 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.143:6789/0 says I am v1:172.21.15.143:47582/0 (socket says 172.21.15.143:47582) 2023-11-26T07:18:27.116 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:27.109+0000 7f60f13b3640 1 -- 172.21.15.143:0/1053801616 learned_addr learned my addr 172.21.15.143:0/1053801616 (peer_addr_for_me v1:172.21.15.143:0/0) 2023-11-26T07:18:27.116 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:27.109+0000 7f60f0bb2640 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7f60ec143810 0x7f60ec072140 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:33350/0 (socket says 172.21.15.143:33350) 2023-11-26T07:18:27.116 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:27.109+0000 7f60e3fff640 1 --2- 172.21.15.143:0/1053801616 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f60ec0b1e10 0x7f60ec15d9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:18:27.116 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:27.109+0000 7f60f0bb2640 1 -- 172.21.15.143:0/1053801616 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f60ec0b1e10 msgr2=0x7f60ec15d9e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-26T07:18:27.116 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:27.109+0000 7f60f0bb2640 1 --2- 172.21.15.143:0/1053801616 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f60ec0b1e10 0x7f60ec15d9e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:18:27.116 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:27.109+0000 7f60f0bb2640 1 -- 172.21.15.143:0/1053801616 >> v1:172.21.15.143:6789/0 conn(0x7f60ec15df20 legacy=0x7f60ec0ce030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:18:27.116 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:27.109+0000 7f60f0bb2640 1 -- 172.21.15.143:0/1053801616 --> v2:172.21.15.143:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f60ec143bf0 con 0x7f60ec143810 2023-11-26T07:18:27.117 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:27.109+0000 7f60e37fe640 1 -- 172.21.15.143:0/1053801616 <== mon.0 v1:172.21.15.143:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1268244203 0 0) 0x7f60ec15e860 con 0x7f60ec15df20 2023-11-26T07:18:27.118 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:27.113+0000 7f60f0bb2640 1 --2- 172.21.15.143:0/1053801616 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f60ec143810 0x7f60ec072140 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x7f60e80097c0 tx=0x7f60e801c450 comp rx=0 tx=0).ready entity=mon.0 client_cookie=966317002d6c4d73 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:18:27.118 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:27.113+0000 7f60e37fe640 1 -- 172.21.15.143:0/1053801616 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f60e80ab650 con 0x7f60ec143810 2023-11-26T07:18:27.118 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:27.113+0000 7f60e37fe640 1 -- 172.21.15.143:0/1053801616 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f60e80ab7b0 con 0x7f60ec143810 2023-11-26T07:18:27.118 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:27.113+0000 7f60e37fe640 1 -- 172.21.15.143:0/1053801616 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f60e80abb40 con 0x7f60ec143810 2023-11-26T07:18:27.119 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:27.113+0000 7f60f1bb4640 1 -- 172.21.15.143:0/1053801616 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f60ec143810 msgr2=0x7f60ec072140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:18:27.119 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:27.113+0000 7f60f1bb4640 1 --2- 172.21.15.143:0/1053801616 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f60ec143810 0x7f60ec072140 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x7f60e80097c0 tx=0x7f60e801c450 comp rx=0 tx=0).stop 2023-11-26T07:18:27.119 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:27.113+0000 7f60f1bb4640 1 -- 172.21.15.143:0/1053801616 shutdown_connections 2023-11-26T07:18:27.119 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:27.113+0000 7f60f1bb4640 1 --2- 172.21.15.143:0/1053801616 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f60ec0b1e10 0x7f60ec15d9e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:18:27.119 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:27.113+0000 7f60f1bb4640 1 --2- 172.21.15.143:0/1053801616 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f60ec143810 0x7f60ec072140 unknown :-1 s=CLOSED pgs=248 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:18:27.119 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:27.113+0000 7f60f1bb4640 1 -- 172.21.15.143:0/1053801616 >> 172.21.15.143:0/1053801616 conn(0x7f60ec06a890 msgr2=0x7f60ec06aca0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:18:27.119 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:27.113+0000 7f60f1bb4640 1 -- 172.21.15.143:0/1053801616 shutdown_connections 2023-11-26T07:18:27.119 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:27.113+0000 7f60f1bb4640 1 -- 172.21.15.143:0/1053801616 wait complete. 2023-11-26T07:18:27.120 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:27.113+0000 7f60f1bb4640 1 Processor -- start 2023-11-26T07:18:27.120 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:27.113+0000 7f60f1bb4640 1 -- start start 2023-11-26T07:18:27.123 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:27.117+0000 7f60f1bb4640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f60ec0b1e10 0x7f60ec136be0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:18:27.123 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:27.117+0000 7f60f1bb4640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f60ec143810 0x7f60ec13d110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:18:27.123 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:27.117+0000 7f60f1bb4640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f60ec15df20 0x7f60ec13d650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:18:27.123 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:27.117+0000 7f60f1bb4640 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f60ec140000 con 0x7f60ec143810 2023-11-26T07:18:27.123 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:27.117+0000 7f60f1bb4640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f60ec13ddd0 con 0x7f60ec0b1e10 2023-11-26T07:18:27.123 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:27.117+0000 7f60f1bb4640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f60ec06d240 con 0x7f60ec15df20 2023-11-26T07:18:27.123 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:27.117+0000 7f60e3fff640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f60ec143810 0x7f60ec13d110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:18:27.123 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:27.117+0000 7f60e3fff640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f60ec143810 0x7f60ec13d110 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:33366/0 (socket says 172.21.15.143:33366) 2023-11-26T07:18:27.123 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:27.117+0000 7f60e3fff640 1 -- 172.21.15.143:0/3789143616 learned_addr learned my addr 172.21.15.143:0/3789143616 (peer_addr_for_me v2:172.21.15.143:0/0) 2023-11-26T07:18:27.123 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:27.117+0000 7f60f0bb2640 1 --2- 172.21.15.143:0/3789143616 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f60ec0b1e10 0x7f60ec136be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:18:27.124 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:27.117+0000 7f60f13b3640 1 --2- 172.21.15.143:0/3789143616 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f60ec15df20 0x7f60ec13d650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:18:27.124 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:27.117+0000 7f60e3fff640 1 -- 172.21.15.143:0/3789143616 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f60ec15df20 msgr2=0x7f60ec13d650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:18:27.124 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:27.117+0000 7f60e3fff640 1 --2- 172.21.15.143:0/3789143616 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f60ec15df20 0x7f60ec13d650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:18:27.124 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:27.117+0000 7f60e3fff640 1 -- 172.21.15.143:0/3789143616 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f60ec0b1e10 msgr2=0x7f60ec136be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:18:27.124 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:27.117+0000 7f60e3fff640 1 --2- 172.21.15.143:0/3789143616 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f60ec0b1e10 0x7f60ec136be0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:18:27.124 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:27.117+0000 7f60e3fff640 1 -- 172.21.15.143:0/3789143616 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f60e8009420 con 0x7f60ec143810 2023-11-26T07:18:27.124 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:27.117+0000 7f60f13b3640 1 --2- 172.21.15.143:0/3789143616 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f60ec15df20 0x7f60ec13d650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-26T07:18:27.124 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:27.117+0000 7f60e3fff640 1 --2- 172.21.15.143:0/3789143616 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f60ec143810 0x7f60ec13d110 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0x7f60dc00d800 tx=0x7f60dc00dcc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:18:27.124 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:27.117+0000 7f60e17fa640 1 -- 172.21.15.143:0/3789143616 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f60dc00b6c0 con 0x7f60ec143810 2023-11-26T07:18:27.124 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:27.117+0000 7f60e17fa640 1 -- 172.21.15.143:0/3789143616 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f60dc00b820 con 0x7f60ec143810 2023-11-26T07:18:27.124 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:27.117+0000 7f60e17fa640 1 -- 172.21.15.143:0/3789143616 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f60dc00bb90 con 0x7f60ec143810 2023-11-26T07:18:27.124 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:27.117+0000 7f60f1bb4640 1 -- 172.21.15.143:0/3789143616 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f60ec06d520 con 0x7f60ec143810 2023-11-26T07:18:27.124 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:27.117+0000 7f60f1bb4640 1 -- 172.21.15.143:0/3789143616 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f60ec06dae0 con 0x7f60ec143810 2023-11-26T07:18:27.125 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:27.117+0000 7f60f1bb4640 1 -- 172.21.15.143:0/3789143616 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f60ec067500 con 0x7f60ec143810 2023-11-26T07:18:27.125 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:27.121+0000 7f60e17fa640 1 -- 172.21.15.143:0/3789143616 <== mon.0 v2:172.21.15.143:3300/0 4 ==== mgrmap(e 5) v1 ==== 85852+0+0 (secure 0 0 0) 0x7f60dc00bcf0 con 0x7f60ec143810 2023-11-26T07:18:27.126 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:27.121+0000 7f60e17fa640 1 --2- 172.21.15.143:0/3789143616 >> [v2:172.21.15.143:6832/2150557544,v1:172.21.15.143:6833/2150557544] conn(0x7f60d8066470 0x7f60d8068930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:18:27.126 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:27.121+0000 7f60e17fa640 1 -- 172.21.15.143:0/3789143616 <== mon.0 v2:172.21.15.143:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f60dc084340 con 0x7f60ec143810 2023-11-26T07:18:27.126 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:27.121+0000 7f60f0bb2640 1 --2- 172.21.15.143:0/3789143616 >> [v2:172.21.15.143:6832/2150557544,v1:172.21.15.143:6833/2150557544] conn(0x7f60d8066470 0x7f60d8068930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:18:27.126 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:27.121+0000 7f60f0bb2640 1 --2- 172.21.15.143:0/3789143616 >> [v2:172.21.15.143:6832/2150557544,v1:172.21.15.143:6833/2150557544] conn(0x7f60d8066470 0x7f60d8068930 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7f60e80af000 tx=0x7f60e80a7000 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:18:27.129 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:27.125+0000 7f60e17fa640 1 -- 172.21.15.143:0/3789143616 <== mon.0 v2:172.21.15.143:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172649 (secure 0 0 0) 0x7f60dc0517b0 con 0x7f60ec143810 2023-11-26T07:18:27.280 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:27.273+0000 7f60f1bb4640 1 -- 172.21.15.143:0/3789143616 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f60ec13df40 con 0x7f60ec143810 2023-11-26T07:18:27.281 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:27.273+0000 7f60e17fa640 1 -- 172.21.15.143:0/3789143616 <== mon.0 v2:172.21.15.143:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v20) v1 ==== 100+0+11 (secure 0 0 0) 0x7f60ec13df40 con 0x7f60ec143810 2023-11-26T07:18:27.281 INFO:teuthology.orchestra.run.smithi143.stdout:pg_num: 64 2023-11-26T07:18:27.282 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:27.277+0000 7f60f1bb4640 1 -- 172.21.15.143:0/3789143616 >> [v2:172.21.15.143:6832/2150557544,v1:172.21.15.143:6833/2150557544] conn(0x7f60d8066470 msgr2=0x7f60d8068930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:18:27.283 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:27.277+0000 7f60f1bb4640 1 --2- 172.21.15.143:0/3789143616 >> [v2:172.21.15.143:6832/2150557544,v1:172.21.15.143:6833/2150557544] conn(0x7f60d8066470 0x7f60d8068930 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7f60e80af000 tx=0x7f60e80a7000 comp rx=0 tx=0).stop 2023-11-26T07:18:27.283 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:27.277+0000 7f60f1bb4640 1 -- 172.21.15.143:0/3789143616 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f60ec143810 msgr2=0x7f60ec13d110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:18:27.283 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:27.277+0000 7f60f1bb4640 1 --2- 172.21.15.143:0/3789143616 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f60ec143810 0x7f60ec13d110 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0x7f60dc00d800 tx=0x7f60dc00dcc0 comp rx=0 tx=0).stop 2023-11-26T07:18:27.283 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:27.277+0000 7f60f1bb4640 1 -- 172.21.15.143:0/3789143616 shutdown_connections 2023-11-26T07:18:27.283 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:27.277+0000 7f60f1bb4640 1 --2- 172.21.15.143:0/3789143616 >> [v2:172.21.15.143:6832/2150557544,v1:172.21.15.143:6833/2150557544] conn(0x7f60d8066470 0x7f60d8068930 unknown :-1 s=CLOSED pgs=151 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:18:27.283 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:27.277+0000 7f60f1bb4640 1 --2- 172.21.15.143:0/3789143616 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f60ec15df20 0x7f60ec13d650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:18:27.283 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:27.277+0000 7f60f1bb4640 1 --2- 172.21.15.143:0/3789143616 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f60ec143810 0x7f60ec13d110 unknown :-1 s=CLOSED pgs=249 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:18:27.283 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:27.277+0000 7f60f1bb4640 1 --2- 172.21.15.143:0/3789143616 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f60ec0b1e10 0x7f60ec136be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:18:27.283 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:27.277+0000 7f60f1bb4640 1 -- 172.21.15.143:0/3789143616 >> 172.21.15.143:0/3789143616 conn(0x7f60ec06a890 msgr2=0x7f60ec0b3180 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:18:27.284 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:27.277+0000 7f60f1bb4640 1 -- 172.21.15.143:0/3789143616 shutdown_connections 2023-11-26T07:18:27.284 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:27.277+0000 7f60f1bb4640 1 -- 172.21.15.143:0/3789143616 wait complete. 2023-11-26T07:18:27.302 DEBUG:teuthology.orchestra.run.smithi143:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2023-11-26T07:18:27.441 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:27.433+0000 7f6c3b7c4640 1 Processor -- start 2023-11-26T07:18:27.441 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:27.433+0000 7f6c3b7c4640 1 -- start start 2023-11-26T07:18:27.442 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:27.437+0000 7f6c3b7c4640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f6c340cd610 0x7f6c34071970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:18:27.442 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:27.437+0000 7f6c3b7c4640 1 -- --> v1:172.21.15.143:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6c34071eb0 con 0x7f6c341614b0 2023-11-26T07:18:27.443 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:27.437+0000 7f6c3b7c4640 1 -- --> v1:172.21.15.188:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6c3415d970 con 0x7f6c34070ba0 2023-11-26T07:18:27.443 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:27.437+0000 7f6c3b7c4640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6c3415dc30 con 0x7f6c340cd610 2023-11-26T07:18:27.443 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:27.437+0000 7f6c39560640 1 --1- >> v1:172.21.15.143:6789/0 conn(0x7f6c341614b0 0x7f6c34161890 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.143:6789/0 says I am v1:172.21.15.143:47592/0 (socket says 172.21.15.143:47592) 2023-11-26T07:18:27.443 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:27.437+0000 7f6c39560640 1 -- 172.21.15.143:0/1335062039 learned_addr learned my addr 172.21.15.143:0/1335062039 (peer_addr_for_me v1:172.21.15.143:0/0) 2023-11-26T07:18:27.443 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:27.437+0000 7f6c38d5f640 1 --2- 172.21.15.143:0/1335062039 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f6c340cd610 0x7f6c34071970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:18:27.444 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:27.437+0000 7f6c2bfff640 1 -- 172.21.15.143:0/1335062039 <== mon.0 v1:172.21.15.143:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2798248790 0 0) 0x7f6c34071eb0 con 0x7f6c341614b0 2023-11-26T07:18:27.444 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:27.437+0000 7f6c2bfff640 1 -- 172.21.15.143:0/1335062039 --> v1:172.21.15.143:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f6c20003530 con 0x7f6c341614b0 2023-11-26T07:18:27.444 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:27.437+0000 7f6c38d5f640 1 -- 172.21.15.143:0/1335062039 >> v1:172.21.15.188:6789/0 conn(0x7f6c34070ba0 legacy=0x7f6c3406dd10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:18:27.445 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:27.437+0000 7f6c38d5f640 1 -- 172.21.15.143:0/1335062039 >> v1:172.21.15.143:6789/0 conn(0x7f6c341614b0 legacy=0x7f6c34161890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:18:27.445 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:27.437+0000 7f6c38d5f640 1 -- 172.21.15.143:0/1335062039 --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6c3415df10 con 0x7f6c340cd610 2023-11-26T07:18:27.445 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:27.441+0000 7f6c38d5f640 1 --2- 172.21.15.143:0/1335062039 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f6c340cd610 0x7f6c34071970 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7f6c240097f0 tx=0x7f6c2401c4c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=9d1f79c00668de36 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:18:27.447 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:27.441+0000 7f6c2bfff640 1 -- 172.21.15.143:0/1335062039 <== mon.2 v2:172.21.15.188:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6c240aa5f0 con 0x7f6c340cd610 2023-11-26T07:18:27.447 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:27.441+0000 7f6c2bfff640 1 -- 172.21.15.143:0/1335062039 <== mon.2 v2:172.21.15.188:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6c240aa750 con 0x7f6c340cd610 2023-11-26T07:18:27.447 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:27.441+0000 7f6c2bfff640 1 -- 172.21.15.143:0/1335062039 <== mon.2 v2:172.21.15.188:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6c240aaae0 con 0x7f6c340cd610 2023-11-26T07:18:27.447 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:27.441+0000 7f6c3b7c4640 1 -- 172.21.15.143:0/1335062039 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f6c340cd610 msgr2=0x7f6c34071970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:18:27.447 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:27.441+0000 7f6c3b7c4640 1 --2- 172.21.15.143:0/1335062039 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f6c340cd610 0x7f6c34071970 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7f6c240097f0 tx=0x7f6c2401c4c0 comp rx=0 tx=0).stop 2023-11-26T07:18:27.447 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:27.441+0000 7f6c3b7c4640 1 -- 172.21.15.143:0/1335062039 shutdown_connections 2023-11-26T07:18:27.447 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:27.441+0000 7f6c3b7c4640 1 --2- 172.21.15.143:0/1335062039 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f6c340cd610 0x7f6c34071970 unknown :-1 s=CLOSED pgs=61 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:18:27.448 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:27.441+0000 7f6c3b7c4640 1 -- 172.21.15.143:0/1335062039 >> 172.21.15.143:0/1335062039 conn(0x7f6c340cf710 msgr2=0x7f6c340cfb20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:18:27.448 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:27.441+0000 7f6c3b7c4640 1 -- 172.21.15.143:0/1335062039 shutdown_connections 2023-11-26T07:18:27.448 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:27.441+0000 7f6c3b7c4640 1 -- 172.21.15.143:0/1335062039 wait complete. 2023-11-26T07:18:27.448 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:27.441+0000 7f6c3b7c4640 1 Processor -- start 2023-11-26T07:18:27.449 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:27.441+0000 7f6c3b7c4640 1 -- start start 2023-11-26T07:18:27.451 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:27.445+0000 7f6c3b7c4640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f6c34070ba0 0x7f6c34147080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:18:27.451 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:27.445+0000 7f6c3b7c4640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f6c340cd610 0x7f6c340b2f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:18:27.451 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:27.445+0000 7f6c3b7c4640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f6c341614b0 0x7f6c340b3440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:18:27.451 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:27.445+0000 7f6c3b7c4640 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6c340ae1c0 con 0x7f6c34070ba0 2023-11-26T07:18:27.451 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:27.445+0000 7f6c3b7c4640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6c340ad540 con 0x7f6c340cd610 2023-11-26T07:18:27.451 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:27.445+0000 7f6c3b7c4640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6c340ad820 con 0x7f6c341614b0 2023-11-26T07:18:27.451 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:27.445+0000 7f6c39560640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f6c34070ba0 0x7f6c34147080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:18:27.452 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:27.445+0000 7f6c39560640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f6c34070ba0 0x7f6c34147080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:33372/0 (socket says 172.21.15.143:33372) 2023-11-26T07:18:27.452 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:27.445+0000 7f6c39560640 1 -- 172.21.15.143:0/3902610171 learned_addr learned my addr 172.21.15.143:0/3902610171 (peer_addr_for_me v2:172.21.15.143:0/0) 2023-11-26T07:18:27.452 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:27.445+0000 7f6c38d5f640 1 --2- 172.21.15.143:0/3902610171 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f6c340cd610 0x7f6c340b2f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:18:27.452 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:27.445+0000 7f6c39560640 1 -- 172.21.15.143:0/3902610171 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f6c341614b0 msgr2=0x7f6c340b3440 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2023-11-26T07:18:27.452 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:27.445+0000 7f6c39560640 1 --2- 172.21.15.143:0/3902610171 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f6c341614b0 0x7f6c340b3440 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:18:27.452 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:27.445+0000 7f6c39560640 1 -- 172.21.15.143:0/3902610171 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f6c340cd610 msgr2=0x7f6c340b2f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:18:27.452 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:27.445+0000 7f6c39560640 1 --2- 172.21.15.143:0/3902610171 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f6c340cd610 0x7f6c340b2f00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:18:27.452 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:27.445+0000 7f6c39560640 1 -- 172.21.15.143:0/3902610171 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6c24009420 con 0x7f6c34070ba0 2023-11-26T07:18:27.452 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:27.445+0000 7f6c39560640 1 --2- 172.21.15.143:0/3902610171 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f6c34070ba0 0x7f6c34147080 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 crypto rx=0x7f6c3000c4d0 tx=0x7f6c3000c990 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:18:27.453 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:27.445+0000 7f6c29ffb640 1 -- 172.21.15.143:0/3902610171 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6c30005cf0 con 0x7f6c34070ba0 2023-11-26T07:18:27.453 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:27.445+0000 7f6c29ffb640 1 -- 172.21.15.143:0/3902610171 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6c30005e50 con 0x7f6c34070ba0 2023-11-26T07:18:27.453 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:27.445+0000 7f6c29ffb640 1 -- 172.21.15.143:0/3902610171 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6c30007270 con 0x7f6c34070ba0 2023-11-26T07:18:27.453 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:27.445+0000 7f6c3b7c4640 1 -- 172.21.15.143:0/3902610171 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6c34071540 con 0x7f6c34070ba0 2023-11-26T07:18:27.453 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:27.445+0000 7f6c3b7c4640 1 -- 172.21.15.143:0/3902610171 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6c340adbb0 con 0x7f6c34070ba0 2023-11-26T07:18:27.453 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:27.445+0000 7f6c3b7c4640 1 -- 172.21.15.143:0/3902610171 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6c34147620 con 0x7f6c34070ba0 2023-11-26T07:18:27.454 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:27.445+0000 7f6c29ffb640 1 -- 172.21.15.143:0/3902610171 <== mon.0 v2:172.21.15.143:3300/0 4 ==== mgrmap(e 5) v1 ==== 85852+0+0 (secure 0 0 0) 0x7f6c300073d0 con 0x7f6c34070ba0 2023-11-26T07:18:27.454 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:27.449+0000 7f6c29ffb640 1 --2- 172.21.15.143:0/3902610171 >> [v2:172.21.15.143:6832/2150557544,v1:172.21.15.143:6833/2150557544] conn(0x7f6c18066470 0x7f6c18068930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:18:27.454 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:27.449+0000 7f6c29ffb640 1 -- 172.21.15.143:0/3902610171 <== mon.0 v2:172.21.15.143:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f6c300833d0 con 0x7f6c34070ba0 2023-11-26T07:18:27.454 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:27.449+0000 7f6c38d5f640 1 --2- 172.21.15.143:0/3902610171 >> [v2:172.21.15.143:6832/2150557544,v1:172.21.15.143:6833/2150557544] conn(0x7f6c18066470 0x7f6c18068930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:18:27.455 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:27.449+0000 7f6c38d5f640 1 --2- 172.21.15.143:0/3902610171 >> [v2:172.21.15.143:6832/2150557544,v1:172.21.15.143:6833/2150557544] conn(0x7f6c18066470 0x7f6c18068930 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7f6c240ae000 tx=0x7f6c240a7000 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:18:27.458 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:27.453+0000 7f6c29ffb640 1 -- 172.21.15.143:0/3902610171 <== mon.0 v2:172.21.15.143:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172649 (secure 0 0 0) 0x7f6c30050860 con 0x7f6c34070ba0 2023-11-26T07:18:27.618 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:27.613+0000 7f6c3b7c4640 1 -- 172.21.15.143:0/3902610171 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) v1 -- 0x7f6c34067680 con 0x7f6c34070ba0 2023-11-26T07:18:27.619 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:27.613+0000 7f6c29ffb640 1 -- 172.21.15.143:0/3902610171 <== mon.0 v2:172.21.15.143:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v20) v1 ==== 80+0+0 (secure 0 0 0) 0x7f6c30053e80 con 0x7f6c34070ba0 2023-11-26T07:18:27.619 INFO:teuthology.orchestra.run.smithi143.stderr:listed 0 entries 2023-11-26T07:18:27.621 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:27.613+0000 7f6c3b7c4640 1 -- 172.21.15.143:0/3902610171 >> [v2:172.21.15.143:6832/2150557544,v1:172.21.15.143:6833/2150557544] conn(0x7f6c18066470 msgr2=0x7f6c18068930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:18:27.621 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:27.613+0000 7f6c3b7c4640 1 --2- 172.21.15.143:0/3902610171 >> [v2:172.21.15.143:6832/2150557544,v1:172.21.15.143:6833/2150557544] conn(0x7f6c18066470 0x7f6c18068930 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7f6c240ae000 tx=0x7f6c240a7000 comp rx=0 tx=0).stop 2023-11-26T07:18:27.621 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:27.613+0000 7f6c3b7c4640 1 -- 172.21.15.143:0/3902610171 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f6c34070ba0 msgr2=0x7f6c34147080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:18:27.621 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:27.613+0000 7f6c3b7c4640 1 --2- 172.21.15.143:0/3902610171 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f6c34070ba0 0x7f6c34147080 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 crypto rx=0x7f6c3000c4d0 tx=0x7f6c3000c990 comp rx=0 tx=0).stop 2023-11-26T07:18:27.622 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:27.617+0000 7f6c3b7c4640 1 -- 172.21.15.143:0/3902610171 shutdown_connections 2023-11-26T07:18:27.622 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:27.617+0000 7f6c3b7c4640 1 --2- 172.21.15.143:0/3902610171 >> [v2:172.21.15.143:6832/2150557544,v1:172.21.15.143:6833/2150557544] conn(0x7f6c18066470 0x7f6c18068930 unknown :-1 s=CLOSED pgs=152 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:18:27.622 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:27.617+0000 7f6c3b7c4640 1 --2- 172.21.15.143:0/3902610171 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f6c341614b0 0x7f6c340b3440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:18:27.622 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:27.617+0000 7f6c3b7c4640 1 --2- 172.21.15.143:0/3902610171 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f6c340cd610 0x7f6c340b2f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:18:27.622 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:27.617+0000 7f6c3b7c4640 1 --2- 172.21.15.143:0/3902610171 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f6c34070ba0 0x7f6c34147080 unknown :-1 s=CLOSED pgs=251 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:18:27.622 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:27.617+0000 7f6c3b7c4640 1 -- 172.21.15.143:0/3902610171 >> 172.21.15.143:0/3902610171 conn(0x7f6c340cf710 msgr2=0x7f6c3406aec0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:18:27.622 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:27.617+0000 7f6c3b7c4640 1 -- 172.21.15.143:0/3902610171 shutdown_connections 2023-11-26T07:18:27.622 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:27.617+0000 7f6c3b7c4640 1 -- 172.21.15.143:0/3902610171 wait complete. 2023-11-26T07:18:27.640 INFO:tasks.cephfs.fuse_mount:Running fusermount -u on ubuntu@smithi143.front.sepia.ceph.com... 2023-11-26T07:18:27.640 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-26T07:18:27.640 DEBUG:teuthology.orchestra.run.smithi143:> sudo fusermount -u /home/ubuntu/cephtest/mnt.0 2023-11-26T07:18:27.681 INFO:teuthology.orchestra.run:waiting for 300 2023-11-26T07:18:27.682 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi143.front.sepia.ceph.com 2023-11-26T07:18:27.682 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-26T07:18:27.682 DEBUG:teuthology.orchestra.run.smithi143:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2023-11-26T07:18:27.742 DEBUG:teuthology.orchestra.run.smithi188:> set -ex 2023-11-26T07:18:27.742 DEBUG:teuthology.orchestra.run.smithi188:> dd if=/proc/self/mounts of=/dev/stdout 2023-11-26T07:18:27.753 DEBUG:teuthology.orchestra.run.smithi188:> set -ex 2023-11-26T07:18:27.753 DEBUG:teuthology.orchestra.run.smithi188:> dd if=/proc/self/mounts of=/dev/stdout 2023-11-26T07:18:27.808 DEBUG:teuthology.orchestra.run.smithi143:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-11-26T07:18:27.944 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:27.937+0000 7fc98849c640 1 Processor -- start 2023-11-26T07:18:27.944 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:27.937+0000 7fc98849c640 1 -- start start 2023-11-26T07:18:27.946 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:27.941+0000 7fc98849c640 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7fc98013e290 0x7fc9800cf280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:18:27.946 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:27.941+0000 7fc98849c640 1 -- --> v1:172.21.15.143:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc98015fb20 con 0x7fc9800cf7c0 2023-11-26T07:18:27.946 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:27.941+0000 7fc98849c640 1 -- --> v1:172.21.15.188:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc98006c520 con 0x7fc98015f6b0 2023-11-26T07:18:27.946 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:27.941+0000 7fc98849c640 1 -- --> v2:172.21.15.143:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc9800cfc40 con 0x7fc98013e290 2023-11-26T07:18:27.946 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:27.941+0000 7fc986238640 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7fc98013e290 0x7fc9800cf280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:18:27.946 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:27.941+0000 7fc985a37640 1 --1- >> v1:172.21.15.143:6789/0 conn(0x7fc9800cf7c0 0x7fc98015efa0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.143:6789/0 says I am v1:172.21.15.143:47604/0 (socket says 172.21.15.143:47604) 2023-11-26T07:18:27.946 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:27.941+0000 7fc985a37640 1 -- 172.21.15.143:0/3274801235 learned_addr learned my addr 172.21.15.143:0/3274801235 (peer_addr_for_me v1:172.21.15.143:0/0) 2023-11-26T07:18:27.947 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:27.941+0000 7fc986238640 1 -- 172.21.15.143:0/3274801235 >> v1:172.21.15.188:6789/0 conn(0x7fc98015f6b0 legacy=0x7fc9800cd7b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:18:27.947 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:27.941+0000 7fc986238640 1 -- 172.21.15.143:0/3274801235 >> v1:172.21.15.143:6789/0 conn(0x7fc9800cf7c0 legacy=0x7fc98015efa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:18:27.947 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:27.941+0000 7fc986238640 1 -- 172.21.15.143:0/3274801235 --> v2:172.21.15.143:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc98006c800 con 0x7fc98013e290 2023-11-26T07:18:27.947 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:27.941+0000 7fc985236640 1 -- 172.21.15.143:0/3274801235 <== mon.0 v1:172.21.15.143:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1032321096 0 0) 0x7fc970001120 con 0x7fc9800cf7c0 2023-11-26T07:18:27.949 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:27.945+0000 7fc986238640 1 --2- 172.21.15.143:0/3274801235 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fc98013e290 0x7fc9800cf280 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 crypto rx=0x7fc97c0097c0 tx=0x7fc97c01c450 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e2575e14229dbb server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:18:27.950 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:27.945+0000 7fc985236640 1 -- 172.21.15.143:0/3274801235 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc97c0ab650 con 0x7fc98013e290 2023-11-26T07:18:27.950 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:27.945+0000 7fc985236640 1 -- 172.21.15.143:0/3274801235 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc97c0ab7b0 con 0x7fc98013e290 2023-11-26T07:18:27.950 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:27.945+0000 7fc985236640 1 -- 172.21.15.143:0/3274801235 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc97c0abb40 con 0x7fc98013e290 2023-11-26T07:18:27.950 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:27.945+0000 7fc98849c640 1 -- 172.21.15.143:0/3274801235 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fc98013e290 msgr2=0x7fc9800cf280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:18:27.950 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:27.945+0000 7fc98849c640 1 --2- 172.21.15.143:0/3274801235 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fc98013e290 0x7fc9800cf280 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 crypto rx=0x7fc97c0097c0 tx=0x7fc97c01c450 comp rx=0 tx=0).stop 2023-11-26T07:18:27.951 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:27.945+0000 7fc98849c640 1 -- 172.21.15.143:0/3274801235 shutdown_connections 2023-11-26T07:18:27.951 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:27.945+0000 7fc98849c640 1 --2- 172.21.15.143:0/3274801235 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fc98013e290 0x7fc9800cf280 unknown :-1 s=CLOSED pgs=253 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:18:27.951 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:27.945+0000 7fc98849c640 1 -- 172.21.15.143:0/3274801235 >> 172.21.15.143:0/3274801235 conn(0x7fc98006a890 msgr2=0x7fc98006aca0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:18:27.951 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:27.945+0000 7fc98849c640 1 -- 172.21.15.143:0/3274801235 shutdown_connections 2023-11-26T07:18:27.951 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:27.945+0000 7fc98849c640 1 -- 172.21.15.143:0/3274801235 wait complete. 2023-11-26T07:18:27.952 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:27.945+0000 7fc98849c640 1 Processor -- start 2023-11-26T07:18:27.952 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:27.945+0000 7fc98849c640 1 -- start start 2023-11-26T07:18:27.954 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:27.949+0000 7fc98849c640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fc9800cf7c0 0x7fc98013b270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:18:27.954 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:27.949+0000 7fc98849c640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fc98013e290 0x7fc980137ee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:18:27.954 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:27.949+0000 7fc98849c640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fc98015f6b0 0x7fc980138580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:18:27.954 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:27.949+0000 7fc98849c640 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc980138420 con 0x7fc98015f6b0 2023-11-26T07:18:27.954 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:27.949+0000 7fc98849c640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc98013ade0 con 0x7fc98013e290 2023-11-26T07:18:27.954 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:27.949+0000 7fc98849c640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc98013a1d0 con 0x7fc9800cf7c0 2023-11-26T07:18:27.954 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:27.949+0000 7fc986a39640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fc98015f6b0 0x7fc980138580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:18:27.954 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:27.949+0000 7fc986a39640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fc98015f6b0 0x7fc980138580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:33384/0 (socket says 172.21.15.143:33384) 2023-11-26T07:18:27.954 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:27.949+0000 7fc986a39640 1 -- 172.21.15.143:0/2067571260 learned_addr learned my addr 172.21.15.143:0/2067571260 (peer_addr_for_me v2:172.21.15.143:0/0) 2023-11-26T07:18:27.955 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:27.949+0000 7fc986238640 1 --2- 172.21.15.143:0/2067571260 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fc9800cf7c0 0x7fc98013b270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:18:27.955 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:27.949+0000 7fc985a37640 1 --2- 172.21.15.143:0/2067571260 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fc98013e290 0x7fc980137ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:18:27.955 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:27.949+0000 7fc986a39640 1 -- 172.21.15.143:0/2067571260 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fc9800cf7c0 msgr2=0x7fc98013b270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:18:27.955 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:27.949+0000 7fc986a39640 1 --2- 172.21.15.143:0/2067571260 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fc9800cf7c0 0x7fc98013b270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:18:27.955 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:27.949+0000 7fc986a39640 1 -- 172.21.15.143:0/2067571260 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fc98013e290 msgr2=0x7fc980137ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:18:27.955 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:27.949+0000 7fc986a39640 1 --2- 172.21.15.143:0/2067571260 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fc98013e290 0x7fc980137ee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:18:27.955 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:27.949+0000 7fc986a39640 1 -- 172.21.15.143:0/2067571260 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc97c009420 con 0x7fc98015f6b0 2023-11-26T07:18:27.955 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:27.949+0000 7fc985a37640 1 --2- 172.21.15.143:0/2067571260 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fc98013e290 0x7fc980137ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-26T07:18:27.955 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:27.949+0000 7fc986a39640 1 --2- 172.21.15.143:0/2067571260 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fc98015f6b0 0x7fc980138580 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 crypto rx=0x7fc97800c730 tx=0x7fc97800cbf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:18:27.956 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:27.949+0000 7fc976ffd640 1 -- 172.21.15.143:0/2067571260 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc978003b80 con 0x7fc98015f6b0 2023-11-26T07:18:27.956 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:27.949+0000 7fc976ffd640 1 -- 172.21.15.143:0/2067571260 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc978003ce0 con 0x7fc98015f6b0 2023-11-26T07:18:27.956 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:27.949+0000 7fc976ffd640 1 -- 172.21.15.143:0/2067571260 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc978005310 con 0x7fc98015f6b0 2023-11-26T07:18:27.956 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:27.949+0000 7fc98849c640 1 -- 172.21.15.143:0/2067571260 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc98013a4b0 con 0x7fc98015f6b0 2023-11-26T07:18:27.956 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:27.949+0000 7fc98849c640 1 -- 172.21.15.143:0/2067571260 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc98013aa70 con 0x7fc98015f6b0 2023-11-26T07:18:27.957 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:27.949+0000 7fc98849c640 1 -- 172.21.15.143:0/2067571260 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc98013ac30 con 0x7fc98015f6b0 2023-11-26T07:18:27.957 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:27.949+0000 7fc976ffd640 1 -- 172.21.15.143:0/2067571260 <== mon.0 v2:172.21.15.143:3300/0 4 ==== mgrmap(e 5) v1 ==== 85852+0+0 (secure 0 0 0) 0x7fc978005680 con 0x7fc98015f6b0 2023-11-26T07:18:27.957 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:27.953+0000 7fc976ffd640 1 --2- 172.21.15.143:0/2067571260 >> [v2:172.21.15.143:6832/2150557544,v1:172.21.15.143:6833/2150557544] conn(0x7fc9640664a0 0x7fc964068960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:18:27.957 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:27.953+0000 7fc976ffd640 1 -- 172.21.15.143:0/2067571260 <== mon.0 v2:172.21.15.143:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fc978083da0 con 0x7fc98015f6b0 2023-11-26T07:18:27.957 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:27.953+0000 7fc986238640 1 --2- 172.21.15.143:0/2067571260 >> [v2:172.21.15.143:6832/2150557544,v1:172.21.15.143:6833/2150557544] conn(0x7fc9640664a0 0x7fc964068960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:18:27.958 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:27.953+0000 7fc986238640 1 --2- 172.21.15.143:0/2067571260 >> [v2:172.21.15.143:6832/2150557544,v1:172.21.15.143:6833/2150557544] conn(0x7fc9640664a0 0x7fc964068960 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7fc97c0af000 tx=0x7fc97c0a7000 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:18:27.961 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:27.957+0000 7fc976ffd640 1 -- 172.21.15.143:0/2067571260 <== mon.0 v2:172.21.15.143:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172649 (secure 0 0 0) 0x7fc978051180 con 0x7fc98015f6b0 2023-11-26T07:18:28.111 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:28.105+0000 7fc98849c640 1 -- 172.21.15.143:0/2067571260 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fc980067680 con 0x7fc98015f6b0 2023-11-26T07:18:28.112 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:28.105+0000 7fc976ffd640 1 -- 172.21.15.143:0/2067571260 <== mon.0 v2:172.21.15.143:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v20) v1 ==== 74+0+16541 (secure 0 0 0) 0x7fc9780547a0 con 0x7fc98015f6b0 2023-11-26T07:18:28.112 INFO:teuthology.orchestra.run.smithi143.stdout: 2023-11-26T07:18:28.112 INFO:teuthology.orchestra.run.smithi143.stdout:{"epoch":20,"fsid":"f5033060-2469-4a07-95fe-2b76f5404bc1","created":"2023-11-26T07:17:28.613663+0000","modified":"2023-11-26T07:17:55.505435+0000","last_up_change":"2023-11-26T07:17:41.157813+0000","last_in_change":"2023-11-26T07:17:31.181658+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-11-26T07:17:41.758597+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2023-11-26T07:17:51.878332+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2023-11-26T07:17:52.871396+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"ebbee2c6-023e-44b9-9f1d-6f7bfd309730","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6817","nonce":639807369},{"type":"v1","addr":"172.21.15.143:6819","nonce":639807369}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6821","nonce":639807369},{"type":"v1","addr":"172.21.15.143:6823","nonce":639807369}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6830","nonce":639807369},{"type":"v1","addr":"172.21.15.143:6831","nonce":639807369}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6825","nonce":639807369},{"type":"v1","addr":"172.21.15.143:6828","nonce":639807369}]},"public_addr":"172.21.15.143:6819/639807369","cluster_addr":"172.21.15.143:6823/639807369","heartbeat_back_addr":"172.21.15.143:6831/639807369","heartbeat_front_addr":"172.21.15.143:6828/639807369","state":["exists","up"]},{"osd":1,"uuid":"fec4b8be-7122-49e9-9a19-5850a0700001","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6816","nonce":1144942530},{"type":"v1","addr":"172.21.15.143:6818","nonce":1144942530}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6820","nonce":1144942530},{"type":"v1","addr":"172.21.15.143:6822","nonce":1144942530}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6827","nonce":1144942530},{"type":"v1","addr":"172.21.15.143:6829","nonce":1144942530}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6824","nonce":1144942530},{"type":"v1","addr":"172.21.15.143:6826","nonce":1144942530}]},"public_addr":"172.21.15.143:6818/1144942530","cluster_addr":"172.21.15.143:6822/1144942530","heartbeat_back_addr":"172.21.15.143:6829/1144942530","heartbeat_front_addr":"172.21.15.143:6826/1144942530","state":["exists","up"]},{"osd":2,"uuid":"2888af5b-f933-46ca-8f95-6dd494ff43f6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6808","nonce":3909095186},{"type":"v1","addr":"172.21.15.143:6809","nonce":3909095186}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6810","nonce":3909095186},{"type":"v1","addr":"172.21.15.143:6811","nonce":3909095186}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6814","nonce":3909095186},{"type":"v1","addr":"172.21.15.143:6815","nonce":3909095186}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6812","nonce":3909095186},{"type":"v1","addr":"172.21.15.143:6813","nonce":3909095186}]},"public_addr":"172.21.15.143:6809/3909095186","cluster_addr":"172.21.15.143:6811/3909095186","heartbeat_back_addr":"172.21.15.143:6815/3909095186","heartbeat_front_addr":"172.21.15.143:6813/3909095186","state":["exists","up"]},{"osd":3,"uuid":"d1f52435-7bc1-4081-9550-9cd1ba55f549","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6800","nonce":3854210991},{"type":"v1","addr":"172.21.15.143:6801","nonce":3854210991}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6802","nonce":3854210991},{"type":"v1","addr":"172.21.15.143:6803","nonce":3854210991}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6806","nonce":3854210991},{"type":"v1","addr":"172.21.15.143:6807","nonce":3854210991}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6804","nonce":3854210991},{"type":"v1","addr":"172.21.15.143:6805","nonce":3854210991}]},"public_addr":"172.21.15.143:6801/3854210991","cluster_addr":"172.21.15.143:6803/3854210991","heartbeat_back_addr":"172.21.15.143:6807/3854210991","heartbeat_front_addr":"172.21.15.143:6805/3854210991","state":["exists","up"]},{"osd":4,"uuid":"e2ae8b6e-6e25-49b2-bcbf-aacce68f06a3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6800","nonce":3924606902},{"type":"v1","addr":"172.21.15.188:6801","nonce":3924606902}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6802","nonce":3924606902},{"type":"v1","addr":"172.21.15.188:6803","nonce":3924606902}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6806","nonce":3924606902},{"type":"v1","addr":"172.21.15.188:6807","nonce":3924606902}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6804","nonce":3924606902},{"type":"v1","addr":"172.21.15.188:6805","nonce":3924606902}]},"public_addr":"172.21.15.188:6801/3924606902","cluster_addr":"172.21.15.188:6803/3924606902","heartbeat_back_addr":"172.21.15.188:6807/3924606902","heartbeat_front_addr":"172.21.15.188:6805/3924606902","state":["exists","up"]},{"osd":5,"uuid":"3cec8a32-7698-4c36-8e89-e4ba11934f33","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6813","nonce":186841544},{"type":"v1","addr":"172.21.15.188:6815","nonce":186841544}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6817","nonce":186841544},{"type":"v1","addr":"172.21.15.188:6818","nonce":186841544}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6822","nonce":186841544},{"type":"v1","addr":"172.21.15.188:6823","nonce":186841544}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6820","nonce":186841544},{"type":"v1","addr":"172.21.15.188:6821","nonce":186841544}]},"public_addr":"172.21.15.188:6815/186841544","cluster_addr":"172.21.15.188:6818/186841544","heartbeat_back_addr":"172.21.15.188:6823/186841544","heartbeat_front_addr":"172.21.15.188:6821/186841544","state":["exists","up"]},{"osd":6,"uuid":"58a65d5d-e090-473a-81b4-ad38e85029f9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6808","nonce":2153499405},{"type":"v1","addr":"172.21.15.188:6809","nonce":2153499405}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6810","nonce":2153499405},{"type":"v1","addr":"172.21.15.188:6811","nonce":2153499405}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6816","nonce":2153499405},{"type":"v1","addr":"172.21.15.188:6819","nonce":2153499405}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6812","nonce":2153499405},{"type":"v1","addr":"172.21.15.188:6814","nonce":2153499405}]},"public_addr":"172.21.15.188:6809/2153499405","cluster_addr":"172.21.15.188:6811/2153499405","heartbeat_back_addr":"172.21.15.188:6819/2153499405","heartbeat_front_addr":"172.21.15.188:6814/2153499405","state":["exists","up"]},{"osd":7,"uuid":"ea5b7f70-3554-4cf8-ac30-f7c59b80678d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6824","nonce":2722096729},{"type":"v1","addr":"172.21.15.188:6825","nonce":2722096729}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6826","nonce":2722096729},{"type":"v1","addr":"172.21.15.188:6827","nonce":2722096729}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6830","nonce":2722096729},{"type":"v1","addr":"172.21.15.188:6831","nonce":2722096729}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6828","nonce":2722096729},{"type":"v1","addr":"172.21.15.188:6829","nonce":2722096729}]},"public_addr":"172.21.15.188:6825/2722096729","cluster_addr":"172.21.15.188:6827/2722096729","heartbeat_back_addr":"172.21.15.188:6831/2722096729","heartbeat_front_addr":"172.21.15.188:6829/2722096729","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-26T07:17:36.995032+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-26T07:17:36.932266+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-26T07:17:36.829304+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-26T07:17:36.804297+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-26T07:17:36.667892+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-26T07:17:36.964868+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-26T07:17:36.827332+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-26T07:17:36.893179+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-11-26T07:18:28.114 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:28.109+0000 7fc98849c640 1 -- 172.21.15.143:0/2067571260 >> [v2:172.21.15.143:6832/2150557544,v1:172.21.15.143:6833/2150557544] conn(0x7fc9640664a0 msgr2=0x7fc964068960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:18:28.115 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:28.109+0000 7fc98849c640 1 --2- 172.21.15.143:0/2067571260 >> [v2:172.21.15.143:6832/2150557544,v1:172.21.15.143:6833/2150557544] conn(0x7fc9640664a0 0x7fc964068960 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7fc97c0af000 tx=0x7fc97c0a7000 comp rx=0 tx=0).stop 2023-11-26T07:18:28.115 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:28.109+0000 7fc98849c640 1 -- 172.21.15.143:0/2067571260 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fc98015f6b0 msgr2=0x7fc980138580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:18:28.115 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:28.109+0000 7fc98849c640 1 --2- 172.21.15.143:0/2067571260 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fc98015f6b0 0x7fc980138580 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 crypto rx=0x7fc97800c730 tx=0x7fc97800cbf0 comp rx=0 tx=0).stop 2023-11-26T07:18:28.115 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:28.109+0000 7fc98849c640 1 -- 172.21.15.143:0/2067571260 shutdown_connections 2023-11-26T07:18:28.115 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:28.109+0000 7fc98849c640 1 --2- 172.21.15.143:0/2067571260 >> [v2:172.21.15.143:6832/2150557544,v1:172.21.15.143:6833/2150557544] conn(0x7fc9640664a0 0x7fc964068960 unknown :-1 s=CLOSED pgs=153 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:18:28.115 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:28.109+0000 7fc98849c640 1 --2- 172.21.15.143:0/2067571260 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fc98015f6b0 0x7fc980138580 unknown :-1 s=CLOSED pgs=254 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:18:28.115 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:28.109+0000 7fc98849c640 1 --2- 172.21.15.143:0/2067571260 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fc98013e290 0x7fc980137ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:18:28.115 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:28.109+0000 7fc98849c640 1 --2- 172.21.15.143:0/2067571260 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fc9800cf7c0 0x7fc98013b270 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:18:28.115 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:28.109+0000 7fc98849c640 1 -- 172.21.15.143:0/2067571260 >> 172.21.15.143:0/2067571260 conn(0x7fc98006a890 msgr2=0x7fc980071e50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:18:28.115 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:28.109+0000 7fc98849c640 1 -- 172.21.15.143:0/2067571260 shutdown_connections 2023-11-26T07:18:28.115 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:28.109+0000 7fc98849c640 1 -- 172.21.15.143:0/2067571260 wait complete. 2023-11-26T07:18:28.133 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-11-26T07:17:41.758597+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2023-11-26T07:17:51.878332+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2023-11-26T07:17:52.871396+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '20', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2023-11-26T07:18:28.133 DEBUG:teuthology.orchestra.run.smithi143:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-11-26T07:18:28.274 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:28.269+0000 7f88d110f640 1 Processor -- start 2023-11-26T07:18:28.274 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:28.269+0000 7f88d110f640 1 -- start start 2023-11-26T07:18:28.275 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:28.269+0000 7f88d110f640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f88cc0cf710 0x7f88cc0cd8c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:18:28.275 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:28.269+0000 7f88d110f640 1 -- --> v1:172.21.15.143:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f88cc0cc730 con 0x7f88cc0cf280 2023-11-26T07:18:28.275 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:28.269+0000 7f88d110f640 1 -- --> v1:172.21.15.188:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f88cc070ee0 con 0x7f88cc143ac0 2023-11-26T07:18:28.275 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:28.269+0000 7f88d110f640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f88cc143ea0 con 0x7f88cc0cf710 2023-11-26T07:18:28.275 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:28.269+0000 7f88c3fff640 1 --1- >> v1:172.21.15.143:6789/0 conn(0x7f88cc0cf280 0x7f88cc0cd1b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.143:6789/0 says I am v1:172.21.15.143:47608/0 (socket says 172.21.15.143:47608) 2023-11-26T07:18:28.276 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:28.269+0000 7f88c3fff640 1 -- 172.21.15.143:0/88142355 learned_addr learned my addr 172.21.15.143:0/88142355 (peer_addr_for_me v1:172.21.15.143:0/0) 2023-11-26T07:18:28.276 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:28.269+0000 7f88cb59e640 1 --2- 172.21.15.143:0/88142355 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f88cc0cf710 0x7f88cc0cd8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:18:28.276 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:28.269+0000 7f88ca59c640 1 -- 172.21.15.143:0/88142355 <== mon.0 v1:172.21.15.143:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1138289575 0 0) 0x7f88cc0cc730 con 0x7f88cc0cf280 2023-11-26T07:18:28.276 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:28.269+0000 7f88ca59c640 1 -- 172.21.15.143:0/88142355 --> v1:172.21.15.143:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f88b8003530 con 0x7f88cc0cf280 2023-11-26T07:18:28.277 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:28.269+0000 7f88ca59c640 1 -- 172.21.15.143:0/88142355 <== mon.1 v1:172.21.15.188:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2865345564 0 0) 0x7f88cc070ee0 con 0x7f88cc143ac0 2023-11-26T07:18:28.277 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:28.269+0000 7f88ca59c640 1 -- 172.21.15.143:0/88142355 --> v1:172.21.15.188:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f88cc0cc730 con 0x7f88cc143ac0 2023-11-26T07:18:28.277 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:28.269+0000 7f88cb59e640 1 -- 172.21.15.143:0/88142355 >> v1:172.21.15.188:6789/0 conn(0x7f88cc143ac0 legacy=0x7f88cc070370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:18:28.277 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:28.269+0000 7f88cb59e640 1 -- 172.21.15.143:0/88142355 >> v1:172.21.15.143:6789/0 conn(0x7f88cc0cf280 legacy=0x7f88cc0cd1b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:18:28.277 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:28.269+0000 7f88cb59e640 1 -- 172.21.15.143:0/88142355 --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f88cc0711c0 con 0x7f88cc0cf710 2023-11-26T07:18:28.279 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:28.273+0000 7f88cb59e640 1 --2- 172.21.15.143:0/88142355 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f88cc0cf710 0x7f88cc0cd8c0 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7f88c4009dc0 tx=0x7f88c401c970 comp rx=0 tx=0).ready entity=mon.2 client_cookie=3eeb3a668ee25a9d server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:18:28.279 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:28.273+0000 7f88ca59c640 1 -- 172.21.15.143:0/88142355 <== mon.2 v2:172.21.15.188:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f88c40a5a30 con 0x7f88cc0cf710 2023-11-26T07:18:28.279 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:28.273+0000 7f88ca59c640 1 -- 172.21.15.143:0/88142355 <== mon.2 v2:172.21.15.188:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f88c40a5b90 con 0x7f88cc0cf710 2023-11-26T07:18:28.279 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:28.273+0000 7f88ca59c640 1 -- 172.21.15.143:0/88142355 <== mon.2 v2:172.21.15.188:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f88c40ad440 con 0x7f88cc0cf710 2023-11-26T07:18:28.280 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:28.273+0000 7f88d110f640 1 -- 172.21.15.143:0/88142355 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f88cc0cf710 msgr2=0x7f88cc0cd8c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:18:28.280 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:28.273+0000 7f88d110f640 1 --2- 172.21.15.143:0/88142355 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f88cc0cf710 0x7f88cc0cd8c0 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7f88c4009dc0 tx=0x7f88c401c970 comp rx=0 tx=0).stop 2023-11-26T07:18:28.280 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:28.273+0000 7f88d110f640 1 -- 172.21.15.143:0/88142355 shutdown_connections 2023-11-26T07:18:28.280 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:28.273+0000 7f88d110f640 1 --2- 172.21.15.143:0/88142355 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f88cc0cf710 0x7f88cc0cd8c0 unknown :-1 s=CLOSED pgs=62 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:18:28.280 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:28.273+0000 7f88d110f640 1 -- 172.21.15.143:0/88142355 >> 172.21.15.143:0/88142355 conn(0x7f88cc06a890 msgr2=0x7f88cc06aca0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:18:28.281 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:28.273+0000 7f88d110f640 1 -- 172.21.15.143:0/88142355 shutdown_connections 2023-11-26T07:18:28.281 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:28.273+0000 7f88d110f640 1 -- 172.21.15.143:0/88142355 wait complete. 2023-11-26T07:18:28.282 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:28.277+0000 7f88d110f640 1 Processor -- start 2023-11-26T07:18:28.282 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:28.277+0000 7f88d110f640 1 -- start start 2023-11-26T07:18:28.285 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:28.277+0000 7f88d110f640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f88cc0cf280 0x7f88cc15a590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:18:28.285 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:28.277+0000 7f88d110f640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f88cc0cf710 0x7f88cc159bb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:18:28.285 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:28.277+0000 7f88d110f640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f88cc143ac0 0x7f88cc0aec60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:18:28.285 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:28.277+0000 7f88d110f640 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f88cc1367a0 con 0x7f88cc143ac0 2023-11-26T07:18:28.285 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:28.277+0000 7f88d110f640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f88cc1364c0 con 0x7f88cc0cf280 2023-11-26T07:18:28.285 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:28.277+0000 7f88d110f640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f88cc13e8f0 con 0x7f88cc0cf710 2023-11-26T07:18:28.285 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:28.277+0000 7f88cb59e640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f88cc143ac0 0x7f88cc0aec60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:18:28.285 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:28.277+0000 7f88cb59e640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f88cc143ac0 0x7f88cc0aec60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:33388/0 (socket says 172.21.15.143:33388) 2023-11-26T07:18:28.285 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:28.277+0000 7f88cb59e640 1 -- 172.21.15.143:0/80731113 learned_addr learned my addr 172.21.15.143:0/80731113 (peer_addr_for_me v2:172.21.15.143:0/0) 2023-11-26T07:18:28.285 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:28.277+0000 7f88c3fff640 1 --2- 172.21.15.143:0/80731113 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f88cc0cf710 0x7f88cc159bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:18:28.285 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:28.277+0000 7f88cad9d640 1 --2- 172.21.15.143:0/80731113 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f88cc0cf280 0x7f88cc15a590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:18:28.286 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:28.277+0000 7f88cb59e640 1 -- 172.21.15.143:0/80731113 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f88cc0cf710 msgr2=0x7f88cc159bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:18:28.286 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:28.277+0000 7f88cb59e640 1 --2- 172.21.15.143:0/80731113 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f88cc0cf710 0x7f88cc159bb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:18:28.286 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:28.277+0000 7f88cb59e640 1 -- 172.21.15.143:0/80731113 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f88cc0cf280 msgr2=0x7f88cc15a590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:18:28.286 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:28.277+0000 7f88cb59e640 1 --2- 172.21.15.143:0/80731113 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f88cc0cf280 0x7f88cc15a590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:18:28.286 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:28.277+0000 7f88cb59e640 1 -- 172.21.15.143:0/80731113 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f88c4009a70 con 0x7f88cc143ac0 2023-11-26T07:18:28.286 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:28.277+0000 7f88cad9d640 1 --2- 172.21.15.143:0/80731113 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f88cc0cf280 0x7f88cc15a590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-26T07:18:28.286 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:28.277+0000 7f88cb59e640 1 --2- 172.21.15.143:0/80731113 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f88cc143ac0 0x7f88cc0aec60 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 crypto rx=0x7f88c40b0040 tx=0x7f88cc0ce610 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:18:28.286 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:28.277+0000 7f88c37fe640 1 -- 172.21.15.143:0/80731113 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f88c40adcc0 con 0x7f88cc143ac0 2023-11-26T07:18:28.286 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:28.277+0000 7f88c37fe640 1 -- 172.21.15.143:0/80731113 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f88c40ade20 con 0x7f88cc143ac0 2023-11-26T07:18:28.286 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:28.277+0000 7f88c37fe640 1 -- 172.21.15.143:0/80731113 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f88c40b8660 con 0x7f88cc143ac0 2023-11-26T07:18:28.286 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:28.277+0000 7f88d110f640 1 -- 172.21.15.143:0/80731113 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f88cc13ebd0 con 0x7f88cc143ac0 2023-11-26T07:18:28.286 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:28.277+0000 7f88d110f640 1 -- 172.21.15.143:0/80731113 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f88cc136bb0 con 0x7f88cc143ac0 2023-11-26T07:18:28.287 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:28.281+0000 7f88d110f640 1 -- 172.21.15.143:0/80731113 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f88cc067680 con 0x7f88cc143ac0 2023-11-26T07:18:28.287 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:28.281+0000 7f88c37fe640 1 -- 172.21.15.143:0/80731113 <== mon.0 v2:172.21.15.143:3300/0 4 ==== mgrmap(e 5) v1 ==== 85852+0+0 (secure 0 0 0) 0x7f88c40b4040 con 0x7f88cc143ac0 2023-11-26T07:18:28.288 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:28.281+0000 7f88c37fe640 1 --2- 172.21.15.143:0/80731113 >> [v2:172.21.15.143:6832/2150557544,v1:172.21.15.143:6833/2150557544] conn(0x7f88ac066470 0x7f88ac068930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:18:28.288 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:28.281+0000 7f88c37fe640 1 -- 172.21.15.143:0/80731113 <== mon.0 v2:172.21.15.143:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f88c411ef50 con 0x7f88cc143ac0 2023-11-26T07:18:28.288 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:28.281+0000 7f88cad9d640 1 --2- 172.21.15.143:0/80731113 >> [v2:172.21.15.143:6832/2150557544,v1:172.21.15.143:6833/2150557544] conn(0x7f88ac066470 0x7f88ac068930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:18:28.288 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:28.281+0000 7f88cad9d640 1 --2- 172.21.15.143:0/80731113 >> [v2:172.21.15.143:6832/2150557544,v1:172.21.15.143:6833/2150557544] conn(0x7f88ac066470 0x7f88ac068930 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7f88bc0011f0 tx=0x7f88bc009040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:18:28.291 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:28.285+0000 7f88c37fe640 1 -- 172.21.15.143:0/80731113 <== mon.0 v2:172.21.15.143:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172649 (secure 0 0 0) 0x7f88c40ec470 con 0x7f88cc143ac0 2023-11-26T07:18:28.441 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:28.433+0000 7f88d110f640 1 -- 172.21.15.143:0/80731113 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f88cc13f080 con 0x7f88cc143ac0 2023-11-26T07:18:28.441 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:28.433+0000 7f88c37fe640 1 -- 172.21.15.143:0/80731113 <== mon.0 v2:172.21.15.143:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v20) v1 ==== 93+0+10 (secure 0 0 0) 0x7f88c4009450 con 0x7f88cc143ac0 2023-11-26T07:18:28.441 INFO:teuthology.orchestra.run.smithi143.stdout:pg_num: 1 2023-11-26T07:18:28.443 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:28.437+0000 7f88d110f640 1 -- 172.21.15.143:0/80731113 >> [v2:172.21.15.143:6832/2150557544,v1:172.21.15.143:6833/2150557544] conn(0x7f88ac066470 msgr2=0x7f88ac068930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:18:28.443 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:28.437+0000 7f88d110f640 1 --2- 172.21.15.143:0/80731113 >> [v2:172.21.15.143:6832/2150557544,v1:172.21.15.143:6833/2150557544] conn(0x7f88ac066470 0x7f88ac068930 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7f88bc0011f0 tx=0x7f88bc009040 comp rx=0 tx=0).stop 2023-11-26T07:18:28.443 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:28.437+0000 7f88d110f640 1 -- 172.21.15.143:0/80731113 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f88cc143ac0 msgr2=0x7f88cc0aec60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:18:28.443 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:28.437+0000 7f88d110f640 1 --2- 172.21.15.143:0/80731113 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f88cc143ac0 0x7f88cc0aec60 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 crypto rx=0x7f88c40b0040 tx=0x7f88cc0ce610 comp rx=0 tx=0).stop 2023-11-26T07:18:28.443 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:28.437+0000 7f88d110f640 1 -- 172.21.15.143:0/80731113 shutdown_connections 2023-11-26T07:18:28.443 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:28.437+0000 7f88d110f640 1 --2- 172.21.15.143:0/80731113 >> [v2:172.21.15.143:6832/2150557544,v1:172.21.15.143:6833/2150557544] conn(0x7f88ac066470 0x7f88ac068930 unknown :-1 s=CLOSED pgs=154 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:18:28.444 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:28.437+0000 7f88d110f640 1 --2- 172.21.15.143:0/80731113 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f88cc143ac0 0x7f88cc0aec60 unknown :-1 s=CLOSED pgs=256 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:18:28.444 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:28.437+0000 7f88d110f640 1 --2- 172.21.15.143:0/80731113 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f88cc0cf710 0x7f88cc159bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:18:28.444 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:28.437+0000 7f88d110f640 1 --2- 172.21.15.143:0/80731113 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f88cc0cf280 0x7f88cc15a590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:18:28.444 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:28.437+0000 7f88d110f640 1 -- 172.21.15.143:0/80731113 >> 172.21.15.143:0/80731113 conn(0x7f88cc06a890 msgr2=0x7f88cc0adbd0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:18:28.444 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:28.437+0000 7f88d110f640 1 -- 172.21.15.143:0/80731113 shutdown_connections 2023-11-26T07:18:28.444 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:28.437+0000 7f88d110f640 1 -- 172.21.15.143:0/80731113 wait complete. 2023-11-26T07:18:28.462 DEBUG:teuthology.orchestra.run.smithi143:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-11-26T07:18:28.601 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:28.597+0000 7fb4450e3640 1 Processor -- start 2023-11-26T07:18:28.602 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:28.597+0000 7fb4450e3640 1 -- start start 2023-11-26T07:18:28.603 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:28.597+0000 7fb4450e3640 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7fb4400cd610 0x7fb440071970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:18:28.603 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:28.597+0000 7fb4450e3640 1 -- --> v1:172.21.15.143:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb440071eb0 con 0x7fb440070ba0 2023-11-26T07:18:28.603 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:28.597+0000 7fb4450e3640 1 -- --> v1:172.21.15.188:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb44015d970 con 0x7fb4401614b0 2023-11-26T07:18:28.603 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:28.597+0000 7fb4450e3640 1 -- --> v2:172.21.15.143:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb44015dc30 con 0x7fb4400cd610 2023-11-26T07:18:28.603 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:28.597+0000 7fb43e59c640 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7fb4400cd610 0x7fb440071970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:18:28.603 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:28.597+0000 7fb43e59c640 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7fb4400cd610 0x7fb440071970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:33394/0 (socket says 172.21.15.143:33394) 2023-11-26T07:18:28.603 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:28.597+0000 7fb43e59c640 1 -- 172.21.15.143:0/2053933997 learned_addr learned my addr 172.21.15.143:0/2053933997 (peer_addr_for_me v2:172.21.15.143:0/0) 2023-11-26T07:18:28.603 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:28.597+0000 7fb43f59e640 1 --1- 172.21.15.143:0/2053933997 >> v1:172.21.15.143:6789/0 conn(0x7fb440070ba0 0x7fb440070fa0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.143:6789/0 says I am v1:172.21.15.143:47620/0 (socket says 172.21.15.143:47620) 2023-11-26T07:18:28.604 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:28.597+0000 7fb43e59c640 1 -- 172.21.15.143:0/2053933997 >> v1:172.21.15.188:6789/0 conn(0x7fb4401614b0 legacy=0x7fb440161890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:18:28.604 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:28.597+0000 7fb43e59c640 1 -- 172.21.15.143:0/2053933997 >> v1:172.21.15.143:6789/0 conn(0x7fb440070ba0 legacy=0x7fb440070fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:18:28.604 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:28.597+0000 7fb43dd9b640 1 -- 172.21.15.143:0/2053933997 <== mon.0 v1:172.21.15.143:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2120040635 0 0) 0x7fb440071eb0 con 0x7fb440070ba0 2023-11-26T07:18:28.604 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:28.597+0000 7fb43e59c640 1 -- 172.21.15.143:0/2053933997 --> v2:172.21.15.143:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb44015df10 con 0x7fb4400cd610 2023-11-26T07:18:28.606 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:28.601+0000 7fb43e59c640 1 --2- 172.21.15.143:0/2053933997 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fb4400cd610 0x7fb440071970 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 crypto rx=0x7fb434009870 tx=0x7fb43401c510 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d12c9ec84c29e682 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:18:28.607 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:28.601+0000 7fb43dd9b640 1 -- 172.21.15.143:0/2053933997 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb4340aa670 con 0x7fb4400cd610 2023-11-26T07:18:28.607 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:28.601+0000 7fb43dd9b640 1 -- 172.21.15.143:0/2053933997 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb4340aa7d0 con 0x7fb4400cd610 2023-11-26T07:18:28.607 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:28.601+0000 7fb43dd9b640 1 -- 172.21.15.143:0/2053933997 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb4340aab60 con 0x7fb4400cd610 2023-11-26T07:18:28.607 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:28.601+0000 7fb4450e3640 1 -- 172.21.15.143:0/2053933997 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fb4400cd610 msgr2=0x7fb440071970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:18:28.607 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:28.601+0000 7fb4450e3640 1 --2- 172.21.15.143:0/2053933997 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fb4400cd610 0x7fb440071970 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 crypto rx=0x7fb434009870 tx=0x7fb43401c510 comp rx=0 tx=0).stop 2023-11-26T07:18:28.607 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:28.601+0000 7fb4450e3640 1 -- 172.21.15.143:0/2053933997 shutdown_connections 2023-11-26T07:18:28.608 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:28.601+0000 7fb4450e3640 1 --2- 172.21.15.143:0/2053933997 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fb4400cd610 0x7fb440071970 unknown :-1 s=CLOSED pgs=258 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:18:28.608 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:28.601+0000 7fb4450e3640 1 -- 172.21.15.143:0/2053933997 >> 172.21.15.143:0/2053933997 conn(0x7fb4400cf710 msgr2=0x7fb4400cfb20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:18:28.608 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:28.601+0000 7fb4450e3640 1 -- 172.21.15.143:0/2053933997 shutdown_connections 2023-11-26T07:18:28.608 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:28.601+0000 7fb4450e3640 1 -- 172.21.15.143:0/2053933997 wait complete. 2023-11-26T07:18:28.609 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:28.605+0000 7fb4450e3640 1 Processor -- start 2023-11-26T07:18:28.609 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:28.605+0000 7fb4450e3640 1 -- start start 2023-11-26T07:18:28.611 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:28.605+0000 7fb4450e3640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fb440070ba0 0x7fb440167a40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:18:28.612 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:28.605+0000 7fb4450e3640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fb4400cd610 0x7fb4400b33a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:18:28.612 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:28.605+0000 7fb4450e3640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fb4401614b0 0x7fb4400b38e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:18:28.612 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:28.605+0000 7fb4450e3640 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb440142080 con 0x7fb4400cd610 2023-11-26T07:18:28.612 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:28.605+0000 7fb4450e3640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb440142360 con 0x7fb440070ba0 2023-11-26T07:18:28.612 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:28.605+0000 7fb4450e3640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb440142640 con 0x7fb4401614b0 2023-11-26T07:18:28.612 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:28.605+0000 7fb43e59c640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fb4400cd610 0x7fb4400b33a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:18:28.612 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:28.605+0000 7fb43e59c640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fb4400cd610 0x7fb4400b33a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:33410/0 (socket says 172.21.15.143:33410) 2023-11-26T07:18:28.612 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:28.605+0000 7fb43e59c640 1 -- 172.21.15.143:0/507745831 learned_addr learned my addr 172.21.15.143:0/507745831 (peer_addr_for_me v2:172.21.15.143:0/0) 2023-11-26T07:18:28.612 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:28.605+0000 7fb43f59e640 1 --2- 172.21.15.143:0/507745831 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fb4401614b0 0x7fb4400b38e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:18:28.612 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:28.605+0000 7fb43ed9d640 1 --2- 172.21.15.143:0/507745831 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fb440070ba0 0x7fb440167a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:18:28.612 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:28.605+0000 7fb43e59c640 1 -- 172.21.15.143:0/507745831 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fb4401614b0 msgr2=0x7fb4400b38e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:18:28.612 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:28.605+0000 7fb43e59c640 1 --2- 172.21.15.143:0/507745831 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fb4401614b0 0x7fb4400b38e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:18:28.613 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:28.605+0000 7fb43e59c640 1 -- 172.21.15.143:0/507745831 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fb440070ba0 msgr2=0x7fb440167a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:18:28.613 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:28.605+0000 7fb43e59c640 1 --2- 172.21.15.143:0/507745831 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fb440070ba0 0x7fb440167a40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:18:28.613 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:28.605+0000 7fb43e59c640 1 -- 172.21.15.143:0/507745831 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb4340094d0 con 0x7fb4400cd610 2023-11-26T07:18:28.613 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:28.605+0000 7fb43ed9d640 1 --2- 172.21.15.143:0/507745831 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fb440070ba0 0x7fb440167a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-26T07:18:28.613 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:28.605+0000 7fb43e59c640 1 --2- 172.21.15.143:0/507745831 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fb4400cd610 0x7fb4400b33a0 secure :-1 s=READY pgs=259 cs=0 l=1 rev1=1 crypto rx=0x7fb4340ac040 tx=0x7fb43401ad50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:18:28.613 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:28.605+0000 7fb4277fe640 1 -- 172.21.15.143:0/507745831 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb4340aae90 con 0x7fb4400cd610 2023-11-26T07:18:28.613 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:28.605+0000 7fb4277fe640 1 -- 172.21.15.143:0/507745831 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb4340b7070 con 0x7fb4400cd610 2023-11-26T07:18:28.613 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:28.605+0000 7fb4450e3640 1 -- 172.21.15.143:0/507745831 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb440154a70 con 0x7fb4400cd610 2023-11-26T07:18:28.613 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:28.605+0000 7fb4277fe640 1 -- 172.21.15.143:0/507745831 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb434004220 con 0x7fb4400cd610 2023-11-26T07:18:28.613 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:28.605+0000 7fb4450e3640 1 -- 172.21.15.143:0/507745831 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb440167f80 con 0x7fb4400cd610 2023-11-26T07:18:28.614 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:28.605+0000 7fb4450e3640 1 -- 172.21.15.143:0/507745831 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb440067680 con 0x7fb4400cd610 2023-11-26T07:18:28.614 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:28.609+0000 7fb4277fe640 1 -- 172.21.15.143:0/507745831 <== mon.0 v2:172.21.15.143:3300/0 4 ==== mgrmap(e 5) v1 ==== 85852+0+0 (secure 0 0 0) 0x7fb434004380 con 0x7fb4400cd610 2023-11-26T07:18:28.614 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:28.609+0000 7fb4277fe640 1 --2- 172.21.15.143:0/507745831 >> [v2:172.21.15.143:6832/2150557544,v1:172.21.15.143:6833/2150557544] conn(0x7fb42c066470 0x7fb42c068930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:18:28.614 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:28.609+0000 7fb4277fe640 1 -- 172.21.15.143:0/507745831 <== mon.0 v2:172.21.15.143:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fb43411e250 con 0x7fb4400cd610 2023-11-26T07:18:28.614 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:28.609+0000 7fb43ed9d640 1 --2- 172.21.15.143:0/507745831 >> [v2:172.21.15.143:6832/2150557544,v1:172.21.15.143:6833/2150557544] conn(0x7fb42c066470 0x7fb42c068930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:18:28.615 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:28.609+0000 7fb43ed9d640 1 --2- 172.21.15.143:0/507745831 >> [v2:172.21.15.143:6832/2150557544,v1:172.21.15.143:6833/2150557544] conn(0x7fb42c066470 0x7fb42c068930 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7fb430000f70 tx=0x7fb430009040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:18:28.618 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:28.613+0000 7fb4277fe640 1 -- 172.21.15.143:0/507745831 <== mon.0 v2:172.21.15.143:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172649 (secure 0 0 0) 0x7fb4340eb640 con 0x7fb4400cd610 2023-11-26T07:18:28.770 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:28.765+0000 7fb4450e3640 1 -- 172.21.15.143:0/507745831 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7fb4400cde70 con 0x7fb4400cd610 2023-11-26T07:18:28.771 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:28.765+0000 7fb4277fe640 1 -- 172.21.15.143:0/507745831 <== mon.0 v2:172.21.15.143:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v20) v1 ==== 104+0+11 (secure 0 0 0) 0x7fb4340eec60 con 0x7fb4400cd610 2023-11-26T07:18:28.771 INFO:teuthology.orchestra.run.smithi143.stdout:pg_num: 32 2023-11-26T07:18:28.773 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:28.765+0000 7fb4450e3640 1 -- 172.21.15.143:0/507745831 >> [v2:172.21.15.143:6832/2150557544,v1:172.21.15.143:6833/2150557544] conn(0x7fb42c066470 msgr2=0x7fb42c068930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:18:28.773 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:28.765+0000 7fb4450e3640 1 --2- 172.21.15.143:0/507745831 >> [v2:172.21.15.143:6832/2150557544,v1:172.21.15.143:6833/2150557544] conn(0x7fb42c066470 0x7fb42c068930 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7fb430000f70 tx=0x7fb430009040 comp rx=0 tx=0).stop 2023-11-26T07:18:28.773 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:28.765+0000 7fb4450e3640 1 -- 172.21.15.143:0/507745831 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fb4400cd610 msgr2=0x7fb4400b33a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:18:28.773 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:28.765+0000 7fb4450e3640 1 --2- 172.21.15.143:0/507745831 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fb4400cd610 0x7fb4400b33a0 secure :-1 s=READY pgs=259 cs=0 l=1 rev1=1 crypto rx=0x7fb4340ac040 tx=0x7fb43401ad50 comp rx=0 tx=0).stop 2023-11-26T07:18:28.773 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:28.765+0000 7fb4450e3640 1 -- 172.21.15.143:0/507745831 shutdown_connections 2023-11-26T07:18:28.773 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:28.765+0000 7fb4450e3640 1 --2- 172.21.15.143:0/507745831 >> [v2:172.21.15.143:6832/2150557544,v1:172.21.15.143:6833/2150557544] conn(0x7fb42c066470 0x7fb42c068930 unknown :-1 s=CLOSED pgs=155 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:18:28.773 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:28.765+0000 7fb4450e3640 1 --2- 172.21.15.143:0/507745831 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fb4401614b0 0x7fb4400b38e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:18:28.773 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:28.765+0000 7fb4450e3640 1 --2- 172.21.15.143:0/507745831 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fb4400cd610 0x7fb4400b33a0 unknown :-1 s=CLOSED pgs=259 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:18:28.773 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:28.765+0000 7fb4450e3640 1 --2- 172.21.15.143:0/507745831 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fb440070ba0 0x7fb440167a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:18:28.774 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:28.765+0000 7fb4450e3640 1 -- 172.21.15.143:0/507745831 >> 172.21.15.143:0/507745831 conn(0x7fb4400cf710 msgr2=0x7fb4401683f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:18:28.774 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:28.765+0000 7fb4450e3640 1 -- 172.21.15.143:0/507745831 shutdown_connections 2023-11-26T07:18:28.774 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:28.769+0000 7fb4450e3640 1 -- 172.21.15.143:0/507745831 wait complete. 2023-11-26T07:18:28.792 DEBUG:teuthology.orchestra.run.smithi143:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-11-26T07:18:28.932 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:28.925+0000 7f98139a1640 1 Processor -- start 2023-11-26T07:18:28.932 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:28.925+0000 7f98139a1640 1 -- start start 2023-11-26T07:18:28.935 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:28.929+0000 7f98139a1640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f980c14b1e0 0x7f980c0cf280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:18:28.935 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:28.929+0000 7f98139a1640 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7f980c070fd0 0x7f980c162510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:18:28.935 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:28.929+0000 7f98139a1640 1 -- --> v1:172.21.15.143:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f980c0cfc20 con 0x7f980c0cf7c0 2023-11-26T07:18:28.935 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:28.929+0000 7f98139a1640 1 -- --> v2:172.21.15.143:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f980c162080 con 0x7f980c070fd0 2023-11-26T07:18:28.935 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:28.929+0000 7f98139a1640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f980c161d70 con 0x7f980c14b1e0 2023-11-26T07:18:28.935 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:28.929+0000 7f9811f3e640 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7f980c070fd0 0x7f980c162510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:18:28.935 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:28.929+0000 7f9811f3e640 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7f980c070fd0 0x7f980c162510 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:33426/0 (socket says 172.21.15.143:33426) 2023-11-26T07:18:28.935 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:28.929+0000 7f9811f3e640 1 -- 172.21.15.143:0/3777562183 learned_addr learned my addr 172.21.15.143:0/3777562183 (peer_addr_for_me v2:172.21.15.143:0/0) 2023-11-26T07:18:28.935 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:28.929+0000 7f9810f3c640 1 --1- >> v1:172.21.15.143:6789/0 conn(0x7f980c0cf7c0 0x7f980c0708c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.143:6789/0 says I am v1:172.21.15.143:47626/0 (socket says 172.21.15.143:47626) 2023-11-26T07:18:28.935 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:28.929+0000 7f981173d640 1 --2- 172.21.15.143:0/3777562183 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f980c14b1e0 0x7f980c0cf280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:18:28.936 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:28.929+0000 7f9811f3e640 1 -- 172.21.15.143:0/3777562183 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f980c14b1e0 msgr2=0x7f980c0cf280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-26T07:18:28.936 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:28.929+0000 7f9811f3e640 1 --2- 172.21.15.143:0/3777562183 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f980c14b1e0 0x7f980c0cf280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:18:28.936 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:28.929+0000 7f9811f3e640 1 -- 172.21.15.143:0/3777562183 >> v1:172.21.15.143:6789/0 conn(0x7f980c0cf7c0 legacy=0x7f980c0708c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:18:28.936 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:28.929+0000 7f9803fff640 1 -- 172.21.15.143:0/3777562183 <== mon.0 v1:172.21.15.143:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3707906492 0 0) 0x7f980c0cfc20 con 0x7f980c0cf7c0 2023-11-26T07:18:28.936 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:28.929+0000 7f9811f3e640 1 -- 172.21.15.143:0/3777562183 --> v2:172.21.15.143:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f980c14b5c0 con 0x7f980c070fd0 2023-11-26T07:18:28.937 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:28.933+0000 7f9811f3e640 1 --2- 172.21.15.143:0/3777562183 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f980c070fd0 0x7f980c162510 secure :-1 s=READY pgs=261 cs=0 l=1 rev1=1 crypto rx=0x7f9804009e50 tx=0x7f980401c9e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e581186566d079be server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:18:28.938 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:28.933+0000 7f9803fff640 1 -- 172.21.15.143:0/3777562183 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f98040a58a0 con 0x7f980c070fd0 2023-11-26T07:18:28.938 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:28.933+0000 7f9803fff640 1 -- 172.21.15.143:0/3777562183 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f98040a5a00 con 0x7f980c070fd0 2023-11-26T07:18:28.938 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:28.933+0000 7f9803fff640 1 -- 172.21.15.143:0/3777562183 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f98040a5d90 con 0x7f980c070fd0 2023-11-26T07:18:28.939 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:28.933+0000 7f98139a1640 1 -- 172.21.15.143:0/3777562183 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f980c070fd0 msgr2=0x7f980c162510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:18:28.939 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:28.933+0000 7f98139a1640 1 --2- 172.21.15.143:0/3777562183 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f980c070fd0 0x7f980c162510 secure :-1 s=READY pgs=261 cs=0 l=1 rev1=1 crypto rx=0x7f9804009e50 tx=0x7f980401c9e0 comp rx=0 tx=0).stop 2023-11-26T07:18:28.939 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:28.933+0000 7f98139a1640 1 -- 172.21.15.143:0/3777562183 shutdown_connections 2023-11-26T07:18:28.939 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:28.933+0000 7f98139a1640 1 --2- 172.21.15.143:0/3777562183 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f980c070fd0 0x7f980c162510 unknown :-1 s=CLOSED pgs=261 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:18:28.939 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:28.933+0000 7f98139a1640 1 --2- 172.21.15.143:0/3777562183 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f980c14b1e0 0x7f980c0cf280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:18:28.939 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:28.933+0000 7f98139a1640 1 -- 172.21.15.143:0/3777562183 >> 172.21.15.143:0/3777562183 conn(0x7f980c06a890 msgr2=0x7f980c06aca0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:18:28.939 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:28.933+0000 7f98139a1640 1 -- 172.21.15.143:0/3777562183 shutdown_connections 2023-11-26T07:18:28.939 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:28.933+0000 7f98139a1640 1 -- 172.21.15.143:0/3777562183 wait complete. 2023-11-26T07:18:28.940 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:28.933+0000 7f98139a1640 1 Processor -- start 2023-11-26T07:18:28.940 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:28.933+0000 7f98139a1640 1 -- start start 2023-11-26T07:18:28.942 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:28.937+0000 7f98139a1640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f980c070fd0 0x7f980c143f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:18:28.942 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:28.937+0000 7f98139a1640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f980c0cf7c0 0x7f980c0a9fc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:18:28.942 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:28.937+0000 7f98139a1640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f980c14b1e0 0x7f980c1602f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:18:28.942 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:28.937+0000 7f98139a1640 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f980c0a9b30 con 0x7f980c070fd0 2023-11-26T07:18:28.942 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:28.937+0000 7f98139a1640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f980c0a8c40 con 0x7f980c14b1e0 2023-11-26T07:18:28.942 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:28.937+0000 7f98139a1640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f980c0a8f20 con 0x7f980c0cf7c0 2023-11-26T07:18:28.943 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:28.937+0000 7f981173d640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f980c070fd0 0x7f980c143f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:18:28.943 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:28.937+0000 7f981173d640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f980c070fd0 0x7f980c143f20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:33442/0 (socket says 172.21.15.143:33442) 2023-11-26T07:18:28.943 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:28.937+0000 7f981173d640 1 -- 172.21.15.143:0/1758818234 learned_addr learned my addr 172.21.15.143:0/1758818234 (peer_addr_for_me v2:172.21.15.143:0/0) 2023-11-26T07:18:28.943 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:28.937+0000 7f9810f3c640 1 --2- 172.21.15.143:0/1758818234 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f980c0cf7c0 0x7f980c0a9fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:18:28.943 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:28.937+0000 7f9811f3e640 1 --2- 172.21.15.143:0/1758818234 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f980c14b1e0 0x7f980c1602f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:18:28.943 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:28.937+0000 7f981173d640 1 -- 172.21.15.143:0/1758818234 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f980c0cf7c0 msgr2=0x7f980c0a9fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:18:28.943 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:28.937+0000 7f981173d640 1 --2- 172.21.15.143:0/1758818234 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f980c0cf7c0 0x7f980c0a9fc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:18:28.943 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:28.937+0000 7f981173d640 1 -- 172.21.15.143:0/1758818234 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f980c14b1e0 msgr2=0x7f980c1602f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:18:28.943 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:28.937+0000 7f981173d640 1 --2- 172.21.15.143:0/1758818234 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f980c14b1e0 0x7f980c1602f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:18:28.943 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:28.937+0000 7f981173d640 1 -- 172.21.15.143:0/1758818234 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9804009b30 con 0x7f980c070fd0 2023-11-26T07:18:28.943 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:28.937+0000 7f9811f3e640 1 --2- 172.21.15.143:0/1758818234 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f980c14b1e0 0x7f980c1602f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-26T07:18:28.943 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:28.937+0000 7f9810f3c640 1 --2- 172.21.15.143:0/1758818234 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f980c0cf7c0 0x7f980c0a9fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-26T07:18:28.944 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:28.937+0000 7f981173d640 1 --2- 172.21.15.143:0/1758818234 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f980c070fd0 0x7f980c143f20 secure :-1 s=READY pgs=262 cs=0 l=1 rev1=1 crypto rx=0x7f980800e870 tx=0x7f980800ec70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:18:28.944 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:28.937+0000 7f9801ffb640 1 -- 172.21.15.143:0/1758818234 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9808009510 con 0x7f980c070fd0 2023-11-26T07:18:28.944 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:28.937+0000 7f9801ffb640 1 -- 172.21.15.143:0/1758818234 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f980800eec0 con 0x7f980c070fd0 2023-11-26T07:18:28.944 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:28.937+0000 7f9801ffb640 1 -- 172.21.15.143:0/1758818234 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f980800cb40 con 0x7f980c070fd0 2023-11-26T07:18:28.945 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:28.937+0000 7f98139a1640 1 -- 172.21.15.143:0/1758818234 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f980c0a9200 con 0x7f980c070fd0 2023-11-26T07:18:28.945 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:28.937+0000 7f98139a1640 1 -- 172.21.15.143:0/1758818234 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f980c0a97c0 con 0x7f980c070fd0 2023-11-26T07:18:28.945 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:28.937+0000 7f98139a1640 1 -- 172.21.15.143:0/1758818234 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f980c067680 con 0x7f980c070fd0 2023-11-26T07:18:28.945 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:28.941+0000 7f9801ffb640 1 -- 172.21.15.143:0/1758818234 <== mon.0 v2:172.21.15.143:3300/0 4 ==== mgrmap(e 5) v1 ==== 85852+0+0 (secure 0 0 0) 0x7f980800cca0 con 0x7f980c070fd0 2023-11-26T07:18:28.946 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:28.941+0000 7f9801ffb640 1 --2- 172.21.15.143:0/1758818234 >> [v2:172.21.15.143:6832/2150557544,v1:172.21.15.143:6833/2150557544] conn(0x7f97f00664a0 0x7f97f0068960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:18:28.946 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:28.941+0000 7f9801ffb640 1 -- 172.21.15.143:0/1758818234 <== mon.0 v2:172.21.15.143:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f9808083f60 con 0x7f980c070fd0 2023-11-26T07:18:28.946 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:28.941+0000 7f9810f3c640 1 --2- 172.21.15.143:0/1758818234 >> [v2:172.21.15.143:6832/2150557544,v1:172.21.15.143:6833/2150557544] conn(0x7f97f00664a0 0x7f97f0068960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:18:28.947 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:28.941+0000 7f9810f3c640 1 --2- 172.21.15.143:0/1758818234 >> [v2:172.21.15.143:6832/2150557544,v1:172.21.15.143:6833/2150557544] conn(0x7f97f00664a0 0x7f97f0068960 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7f97fc007fd0 tx=0x7f97fc007c20 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:18:28.950 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:28.945+0000 7f9801ffb640 1 -- 172.21.15.143:0/1758818234 <== mon.0 v2:172.21.15.143:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172649 (secure 0 0 0) 0x7f98080513d0 con 0x7f980c070fd0 2023-11-26T07:18:29.100 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:29.093+0000 7f98139a1640 1 -- 172.21.15.143:0/1758818234 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f980c067db0 con 0x7f980c070fd0 2023-11-26T07:18:29.101 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:29.097+0000 7f9801ffb640 1 -- 172.21.15.143:0/1758818234 <== mon.0 v2:172.21.15.143:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v20) v1 ==== 100+0+11 (secure 0 0 0) 0x7f9808017020 con 0x7f980c070fd0 2023-11-26T07:18:29.101 INFO:teuthology.orchestra.run.smithi143.stdout:pg_num: 64 2023-11-26T07:18:29.103 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:29.097+0000 7f98139a1640 1 -- 172.21.15.143:0/1758818234 >> [v2:172.21.15.143:6832/2150557544,v1:172.21.15.143:6833/2150557544] conn(0x7f97f00664a0 msgr2=0x7f97f0068960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:18:29.103 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:29.097+0000 7f98139a1640 1 --2- 172.21.15.143:0/1758818234 >> [v2:172.21.15.143:6832/2150557544,v1:172.21.15.143:6833/2150557544] conn(0x7f97f00664a0 0x7f97f0068960 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7f97fc007fd0 tx=0x7f97fc007c20 comp rx=0 tx=0).stop 2023-11-26T07:18:29.103 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:29.097+0000 7f98139a1640 1 -- 172.21.15.143:0/1758818234 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f980c070fd0 msgr2=0x7f980c143f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:18:29.104 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:29.097+0000 7f98139a1640 1 --2- 172.21.15.143:0/1758818234 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f980c070fd0 0x7f980c143f20 secure :-1 s=READY pgs=262 cs=0 l=1 rev1=1 crypto rx=0x7f980800e870 tx=0x7f980800ec70 comp rx=0 tx=0).stop 2023-11-26T07:18:29.104 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:29.097+0000 7f98139a1640 1 -- 172.21.15.143:0/1758818234 shutdown_connections 2023-11-26T07:18:29.104 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:29.097+0000 7f98139a1640 1 --2- 172.21.15.143:0/1758818234 >> [v2:172.21.15.143:6832/2150557544,v1:172.21.15.143:6833/2150557544] conn(0x7f97f00664a0 0x7f97f0068960 unknown :-1 s=CLOSED pgs=156 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:18:29.104 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:29.097+0000 7f98139a1640 1 --2- 172.21.15.143:0/1758818234 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f980c14b1e0 0x7f980c1602f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:18:29.104 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:29.097+0000 7f98139a1640 1 --2- 172.21.15.143:0/1758818234 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f980c0cf7c0 0x7f980c0a9fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:18:29.104 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:29.097+0000 7f98139a1640 1 --2- 172.21.15.143:0/1758818234 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f980c070fd0 0x7f980c143f20 unknown :-1 s=CLOSED pgs=262 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:18:29.104 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:29.097+0000 7f98139a1640 1 -- 172.21.15.143:0/1758818234 >> 172.21.15.143:0/1758818234 conn(0x7f980c06a890 msgr2=0x7f980c14a490 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:18:29.104 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:29.097+0000 7f98139a1640 1 -- 172.21.15.143:0/1758818234 shutdown_connections 2023-11-26T07:18:29.104 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:29.097+0000 7f98139a1640 1 -- 172.21.15.143:0/1758818234 wait complete. 2023-11-26T07:18:29.122 DEBUG:teuthology.orchestra.run.smithi143:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2023-11-26T07:18:29.260 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:29.253+0000 7efee1d3e640 1 Processor -- start 2023-11-26T07:18:29.260 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:29.253+0000 7efee1d3e640 1 -- start start 2023-11-26T07:18:29.261 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:29.257+0000 7efee1d3e640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7efedc0cf280 0x7efedc0cf6e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:18:29.262 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:29.257+0000 7efee1d3e640 1 -- --> v1:172.21.15.143:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7efedc0cfc20 con 0x7efedc0cd1b0 2023-11-26T07:18:29.262 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:29.257+0000 7efee1d3e640 1 -- --> v1:172.21.15.188:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7efedc0cc730 con 0x7efedc143ac0 2023-11-26T07:18:29.262 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:29.257+0000 7efee1d3e640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7efedc143ea0 con 0x7efedc0cf280 2023-11-26T07:18:29.262 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:29.257+0000 7efedbfff640 1 --1- >> v1:172.21.15.143:6789/0 conn(0x7efedc0cd1b0 0x7efedc0cd5e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.143:6789/0 says I am v1:172.21.15.143:47636/0 (socket says 172.21.15.143:47636) 2023-11-26T07:18:29.262 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:29.257+0000 7efedbfff640 1 -- 172.21.15.143:0/2249322378 learned_addr learned my addr 172.21.15.143:0/2249322378 (peer_addr_for_me v1:172.21.15.143:0/0) 2023-11-26T07:18:29.262 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:29.257+0000 7efedaffd640 1 --2- 172.21.15.143:0/2249322378 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7efedc0cf280 0x7efedc0cf6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:18:29.263 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:29.257+0000 7efeda7fc640 1 -- 172.21.15.143:0/2249322378 <== mon.0 v1:172.21.15.143:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2331663347 0 0) 0x7efedc0cfc20 con 0x7efedc0cd1b0 2023-11-26T07:18:29.263 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:29.257+0000 7efeda7fc640 1 -- 172.21.15.143:0/2249322378 --> v1:172.21.15.143:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7efec4003530 con 0x7efedc0cd1b0 2023-11-26T07:18:29.263 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:29.257+0000 7efeda7fc640 1 -- 172.21.15.143:0/2249322378 <== mon.1 v1:172.21.15.188:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3991618837 0 0) 0x7efedc0cc730 con 0x7efedc143ac0 2023-11-26T07:18:29.263 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:29.257+0000 7efedaffd640 1 -- 172.21.15.143:0/2249322378 >> v1:172.21.15.188:6789/0 conn(0x7efedc143ac0 legacy=0x7efedc070370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:18:29.263 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:29.257+0000 7efedaffd640 1 -- 172.21.15.143:0/2249322378 >> v1:172.21.15.143:6789/0 conn(0x7efedc0cd1b0 legacy=0x7efedc0cd5e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:18:29.263 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:29.257+0000 7efedaffd640 1 -- 172.21.15.143:0/2249322378 --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efedc070ee0 con 0x7efedc0cf280 2023-11-26T07:18:29.265 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:29.261+0000 7efedaffd640 1 --2- 172.21.15.143:0/2249322378 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7efedc0cf280 0x7efedc0cf6e0 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7efed00097f0 tx=0x7efed001c4c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=2719aa426f8bea21 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:18:29.266 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:29.261+0000 7efeda7fc640 1 -- 172.21.15.143:0/2249322378 <== mon.2 v2:172.21.15.188:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efed00aa620 con 0x7efedc0cf280 2023-11-26T07:18:29.266 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:29.261+0000 7efeda7fc640 1 -- 172.21.15.143:0/2249322378 <== mon.2 v2:172.21.15.188:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7efed00aa780 con 0x7efedc0cf280 2023-11-26T07:18:29.266 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:29.261+0000 7efeda7fc640 1 -- 172.21.15.143:0/2249322378 <== mon.2 v2:172.21.15.188:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efed00aab10 con 0x7efedc0cf280 2023-11-26T07:18:29.267 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:29.261+0000 7efee1d3e640 1 -- 172.21.15.143:0/2249322378 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7efedc0cf280 msgr2=0x7efedc0cf6e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:18:29.267 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:29.261+0000 7efee1d3e640 1 --2- 172.21.15.143:0/2249322378 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7efedc0cf280 0x7efedc0cf6e0 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7efed00097f0 tx=0x7efed001c4c0 comp rx=0 tx=0).stop 2023-11-26T07:18:29.267 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:29.261+0000 7efee1d3e640 1 -- 172.21.15.143:0/2249322378 shutdown_connections 2023-11-26T07:18:29.267 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:29.261+0000 7efee1d3e640 1 --2- 172.21.15.143:0/2249322378 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7efedc0cf280 0x7efedc0cf6e0 unknown :-1 s=CLOSED pgs=63 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:18:29.267 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:29.261+0000 7efee1d3e640 1 -- 172.21.15.143:0/2249322378 >> 172.21.15.143:0/2249322378 conn(0x7efedc06a890 msgr2=0x7efedc06aca0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:18:29.267 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:29.261+0000 7efee1d3e640 1 -- 172.21.15.143:0/2249322378 shutdown_connections 2023-11-26T07:18:29.267 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:29.261+0000 7efee1d3e640 1 -- 172.21.15.143:0/2249322378 wait complete. 2023-11-26T07:18:29.268 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:29.261+0000 7efee1d3e640 1 Processor -- start 2023-11-26T07:18:29.268 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:29.261+0000 7efee1d3e640 1 -- start start 2023-11-26T07:18:29.270 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:29.265+0000 7efee1d3e640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7efedc0cd1b0 0x7efedc159990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:18:29.271 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:29.265+0000 7efee1d3e640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7efedc0cf280 0x7efedc140230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:18:29.271 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:29.265+0000 7efee1d3e640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7efedc143ac0 0x7efedc0b10e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:18:29.271 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:29.265+0000 7efee1d3e640 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efedc13ec60 con 0x7efedc143ac0 2023-11-26T07:18:29.271 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:29.265+0000 7efee1d3e640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efedc13ef40 con 0x7efedc0cf280 2023-11-26T07:18:29.271 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:29.265+0000 7efee1d3e640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7efedc13f220 con 0x7efedc0cd1b0 2023-11-26T07:18:29.271 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:29.265+0000 7efedbfff640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7efedc143ac0 0x7efedc0b10e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:18:29.271 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:29.265+0000 7efedb7fe640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7efedc0cd1b0 0x7efedc159990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:18:29.271 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:29.265+0000 7efedbfff640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7efedc143ac0 0x7efedc0b10e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:33448/0 (socket says 172.21.15.143:33448) 2023-11-26T07:18:29.271 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:29.265+0000 7efedbfff640 1 -- 172.21.15.143:0/3994214503 learned_addr learned my addr 172.21.15.143:0/3994214503 (peer_addr_for_me v2:172.21.15.143:0/0) 2023-11-26T07:18:29.271 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:29.265+0000 7efedaffd640 1 --2- 172.21.15.143:0/3994214503 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7efedc0cf280 0x7efedc140230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:18:29.272 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:29.265+0000 7efedbfff640 1 -- 172.21.15.143:0/3994214503 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7efedc0cd1b0 msgr2=0x7efedc159990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:18:29.272 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:29.265+0000 7efedbfff640 1 --2- 172.21.15.143:0/3994214503 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7efedc0cd1b0 0x7efedc159990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:18:29.272 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:29.265+0000 7efedbfff640 1 -- 172.21.15.143:0/3994214503 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7efedc0cf280 msgr2=0x7efedc140230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:18:29.272 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:29.265+0000 7efedbfff640 1 --2- 172.21.15.143:0/3994214503 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7efedc0cf280 0x7efedc140230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:18:29.272 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:29.265+0000 7efedbfff640 1 -- 172.21.15.143:0/3994214503 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efed0009420 con 0x7efedc143ac0 2023-11-26T07:18:29.272 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:29.265+0000 7efedbfff640 1 --2- 172.21.15.143:0/3994214503 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7efedc143ac0 0x7efedc0b10e0 secure :-1 s=READY pgs=264 cs=0 l=1 rev1=1 crypto rx=0x7efed400ca20 tx=0x7efed400cee0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:18:29.272 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:29.265+0000 7efec3fff640 1 -- 172.21.15.143:0/3994214503 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efed400abd0 con 0x7efedc143ac0 2023-11-26T07:18:29.272 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:29.265+0000 7efec3fff640 1 -- 172.21.15.143:0/3994214503 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7efed400ad30 con 0x7efedc143ac0 2023-11-26T07:18:29.272 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:29.265+0000 7efec3fff640 1 -- 172.21.15.143:0/3994214503 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efed4012620 con 0x7efedc143ac0 2023-11-26T07:18:29.272 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:29.265+0000 7efee1d3e640 1 -- 172.21.15.143:0/3994214503 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7efedc13f500 con 0x7efedc143ac0 2023-11-26T07:18:29.272 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:29.265+0000 7efee1d3e640 1 -- 172.21.15.143:0/3994214503 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7efedc159ed0 con 0x7efedc143ac0 2023-11-26T07:18:29.273 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:29.265+0000 7efee1d3e640 1 -- 172.21.15.143:0/3994214503 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7efedc067680 con 0x7efedc143ac0 2023-11-26T07:18:29.273 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:29.265+0000 7efec3fff640 1 -- 172.21.15.143:0/3994214503 <== mon.0 v2:172.21.15.143:3300/0 4 ==== mgrmap(e 5) v1 ==== 85852+0+0 (secure 0 0 0) 0x7efed4007070 con 0x7efedc143ac0 2023-11-26T07:18:29.273 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:29.269+0000 7efec3fff640 1 --2- 172.21.15.143:0/3994214503 >> [v2:172.21.15.143:6832/2150557544,v1:172.21.15.143:6833/2150557544] conn(0x7efec8066470 0x7efec8068930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:18:29.274 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:29.269+0000 7efec3fff640 1 -- 172.21.15.143:0/3994214503 <== mon.0 v2:172.21.15.143:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5709+0+0 (secure 0 0 0) 0x7efed4083d40 con 0x7efedc143ac0 2023-11-26T07:18:29.274 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:29.269+0000 7efedb7fe640 1 --2- 172.21.15.143:0/3994214503 >> [v2:172.21.15.143:6832/2150557544,v1:172.21.15.143:6833/2150557544] conn(0x7efec8066470 0x7efec8068930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:18:29.274 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:29.269+0000 7efedb7fe640 1 --2- 172.21.15.143:0/3994214503 >> [v2:172.21.15.143:6832/2150557544,v1:172.21.15.143:6833/2150557544] conn(0x7efec8066470 0x7efec8068930 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7efecc000f70 tx=0x7efecc009040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:18:29.277 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:29.273+0000 7efec3fff640 1 -- 172.21.15.143:0/3994214503 <== mon.0 v2:172.21.15.143:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172649 (secure 0 0 0) 0x7efed4051270 con 0x7efedc143ac0 2023-11-26T07:18:29.438 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:29.433+0000 7efee1d3e640 1 -- 172.21.15.143:0/3994214503 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) v1 -- 0x7efedc13eda0 con 0x7efedc143ac0 2023-11-26T07:18:29.438 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:29.433+0000 7efec3fff640 1 -- 172.21.15.143:0/3994214503 <== mon.0 v2:172.21.15.143:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v20) v1 ==== 80+0+0 (secure 0 0 0) 0x7efedc13eda0 con 0x7efedc143ac0 2023-11-26T07:18:29.438 INFO:teuthology.orchestra.run.smithi143.stderr:listed 0 entries 2023-11-26T07:18:29.440 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:29.433+0000 7efee1d3e640 1 -- 172.21.15.143:0/3994214503 >> [v2:172.21.15.143:6832/2150557544,v1:172.21.15.143:6833/2150557544] conn(0x7efec8066470 msgr2=0x7efec8068930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:18:29.440 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:29.433+0000 7efee1d3e640 1 --2- 172.21.15.143:0/3994214503 >> [v2:172.21.15.143:6832/2150557544,v1:172.21.15.143:6833/2150557544] conn(0x7efec8066470 0x7efec8068930 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7efecc000f70 tx=0x7efecc009040 comp rx=0 tx=0).stop 2023-11-26T07:18:29.440 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:29.433+0000 7efee1d3e640 1 -- 172.21.15.143:0/3994214503 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7efedc143ac0 msgr2=0x7efedc0b10e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:18:29.440 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:29.433+0000 7efee1d3e640 1 --2- 172.21.15.143:0/3994214503 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7efedc143ac0 0x7efedc0b10e0 secure :-1 s=READY pgs=264 cs=0 l=1 rev1=1 crypto rx=0x7efed400ca20 tx=0x7efed400cee0 comp rx=0 tx=0).stop 2023-11-26T07:18:29.440 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:29.433+0000 7efee1d3e640 1 -- 172.21.15.143:0/3994214503 shutdown_connections 2023-11-26T07:18:29.440 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:29.433+0000 7efee1d3e640 1 --2- 172.21.15.143:0/3994214503 >> [v2:172.21.15.143:6832/2150557544,v1:172.21.15.143:6833/2150557544] conn(0x7efec8066470 0x7efec8068930 unknown :-1 s=CLOSED pgs=157 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:18:29.441 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:29.433+0000 7efee1d3e640 1 --2- 172.21.15.143:0/3994214503 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7efedc143ac0 0x7efedc0b10e0 unknown :-1 s=CLOSED pgs=264 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:18:29.441 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:29.433+0000 7efee1d3e640 1 --2- 172.21.15.143:0/3994214503 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7efedc0cf280 0x7efedc140230 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:18:29.441 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:29.433+0000 7efee1d3e640 1 --2- 172.21.15.143:0/3994214503 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7efedc0cd1b0 0x7efedc159990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:18:29.441 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:29.433+0000 7efee1d3e640 1 -- 172.21.15.143:0/3994214503 >> 172.21.15.143:0/3994214503 conn(0x7efedc06a890 msgr2=0x7efedc0b1cb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:18:29.441 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:29.433+0000 7efee1d3e640 1 -- 172.21.15.143:0/3994214503 shutdown_connections 2023-11-26T07:18:29.441 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:29.433+0000 7efee1d3e640 1 -- 172.21.15.143:0/3994214503 wait complete. 2023-11-26T07:18:29.459 DEBUG:teuthology.orchestra.run.smithi188:> set -ex 2023-11-26T07:18:29.459 DEBUG:teuthology.orchestra.run.smithi188:> dd if=/proc/self/mounts of=/dev/stdout 2023-11-26T07:18:29.469 DEBUG:teuthology.orchestra.run.smithi143:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-11-26T07:18:29.606 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:29.601+0000 7f4717d0e640 1 Processor -- start 2023-11-26T07:18:29.606 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:29.601+0000 7f4717d0e640 1 -- start start 2023-11-26T07:18:29.608 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:29.601+0000 7f4717d0e640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f471015d6e0 0x7f47100d0df0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:18:29.608 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:29.601+0000 7f4717d0e640 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7f47100b2160 0x7f47100d14b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:18:29.608 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:29.601+0000 7f4717d0e640 1 -- --> v1:172.21.15.143:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f47100cf0e0 con 0x7f47100d19f0 2023-11-26T07:18:29.609 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:29.601+0000 7f4717d0e640 1 -- --> v2:172.21.15.143:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f471015dac0 con 0x7f47100b2160 2023-11-26T07:18:29.609 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:29.601+0000 7f4717d0e640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f47100d1e80 con 0x7f471015d6e0 2023-11-26T07:18:29.609 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:29.601+0000 7f47152a9640 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7f47100b2160 0x7f47100d14b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:18:29.609 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:29.601+0000 7f47152a9640 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7f47100b2160 0x7f47100d14b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:33456/0 (socket says 172.21.15.143:33456) 2023-11-26T07:18:29.609 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:29.601+0000 7f47152a9640 1 -- 172.21.15.143:0/3023788922 learned_addr learned my addr 172.21.15.143:0/3023788922 (peer_addr_for_me v2:172.21.15.143:0/0) 2023-11-26T07:18:29.609 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:29.601+0000 7f47162ab640 1 --1- 172.21.15.143:0/3023788922 >> v1:172.21.15.143:6789/0 conn(0x7f47100d19f0 0x7f47100cfbf0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.143:6789/0 says I am v1:172.21.15.143:47650/0 (socket says 172.21.15.143:47650) 2023-11-26T07:18:29.609 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:29.601+0000 7f4715aaa640 1 --2- 172.21.15.143:0/3023788922 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f471015d6e0 0x7f47100d0df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:18:29.609 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:29.601+0000 7f47152a9640 1 -- 172.21.15.143:0/3023788922 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f471015d6e0 msgr2=0x7f47100d0df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-26T07:18:29.609 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:29.601+0000 7f47152a9640 1 --2- 172.21.15.143:0/3023788922 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f471015d6e0 0x7f47100d0df0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:18:29.609 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:29.601+0000 7f47152a9640 1 -- 172.21.15.143:0/3023788922 >> v1:172.21.15.143:6789/0 conn(0x7f47100d19f0 legacy=0x7f47100cfbf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:18:29.609 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:29.601+0000 7f47152a9640 1 -- 172.21.15.143:0/3023788922 --> v2:172.21.15.143:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f47100ceb20 con 0x7f47100b2160 2023-11-26T07:18:29.611 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:29.605+0000 7f47152a9640 1 --2- 172.21.15.143:0/3023788922 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f47100b2160 0x7f47100d14b0 secure :-1 s=READY pgs=266 cs=0 l=1 rev1=1 crypto rx=0x7f4700009870 tx=0x7f470001c540 comp rx=0 tx=0).ready entity=mon.0 client_cookie=efdbbec54827fdc3 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:18:29.611 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:29.605+0000 7f4714aa8640 1 -- 172.21.15.143:0/3023788922 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f47000a7070 con 0x7f47100b2160 2023-11-26T07:18:29.612 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:29.605+0000 7f4714aa8640 1 -- 172.21.15.143:0/3023788922 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f47000ab430 con 0x7f47100b2160 2023-11-26T07:18:29.612 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:29.605+0000 7f4714aa8640 1 -- 172.21.15.143:0/3023788922 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f47000ab7c0 con 0x7f47100b2160 2023-11-26T07:18:29.612 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:29.605+0000 7f4717d0e640 1 -- 172.21.15.143:0/3023788922 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f47100b2160 msgr2=0x7f47100d14b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:18:29.612 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:29.605+0000 7f4717d0e640 1 --2- 172.21.15.143:0/3023788922 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f47100b2160 0x7f47100d14b0 secure :-1 s=READY pgs=266 cs=0 l=1 rev1=1 crypto rx=0x7f4700009870 tx=0x7f470001c540 comp rx=0 tx=0).stop 2023-11-26T07:18:29.613 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:29.605+0000 7f4717d0e640 1 -- 172.21.15.143:0/3023788922 shutdown_connections 2023-11-26T07:18:29.613 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:29.605+0000 7f4717d0e640 1 --2- 172.21.15.143:0/3023788922 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f47100b2160 0x7f47100d14b0 unknown :-1 s=CLOSED pgs=266 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:18:29.613 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:29.605+0000 7f4717d0e640 1 --2- 172.21.15.143:0/3023788922 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f471015d6e0 0x7f47100d0df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:18:29.613 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:29.605+0000 7f4717d0e640 1 -- 172.21.15.143:0/3023788922 >> 172.21.15.143:0/3023788922 conn(0x7f471009c550 msgr2=0x7f47100d3220 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:18:29.613 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:29.605+0000 7f4717d0e640 1 -- 172.21.15.143:0/3023788922 shutdown_connections 2023-11-26T07:18:29.613 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:29.605+0000 7f4717d0e640 1 -- 172.21.15.143:0/3023788922 wait complete. 2023-11-26T07:18:29.614 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:29.609+0000 7f4717d0e640 1 Processor -- start 2023-11-26T07:18:29.614 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:29.609+0000 7f4717d0e640 1 -- start start 2023-11-26T07:18:29.615 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:29.609+0000 7f4717d0e640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f47100b2160 0x7f471015bd80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:18:29.616 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:29.609+0000 7f4717d0e640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f47100d19f0 0x7f47100b29c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:18:29.616 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:29.609+0000 7f4717d0e640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f471015d6e0 0x7f47100b6e60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:18:29.616 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:29.609+0000 7f4717d0e640 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f471015c2c0 con 0x7f47100d19f0 2023-11-26T07:18:29.616 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:29.609+0000 7f4717d0e640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f47100c8140 con 0x7f47100b2160 2023-11-26T07:18:29.616 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:29.609+0000 7f4717d0e640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f47100b3330 con 0x7f471015d6e0 2023-11-26T07:18:29.616 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:29.609+0000 7f47152a9640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f47100d19f0 0x7f47100b29c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:18:29.616 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:29.609+0000 7f47152a9640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f47100d19f0 0x7f47100b29c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:33458/0 (socket says 172.21.15.143:33458) 2023-11-26T07:18:29.616 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:29.609+0000 7f47152a9640 1 -- 172.21.15.143:0/839164094 learned_addr learned my addr 172.21.15.143:0/839164094 (peer_addr_for_me v2:172.21.15.143:0/0) 2023-11-26T07:18:29.616 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:29.609+0000 7f47152a9640 1 -- 172.21.15.143:0/839164094 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f471015d6e0 msgr2=0x7f47100b6e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:18:29.616 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:29.609+0000 7f47152a9640 1 --2- 172.21.15.143:0/839164094 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f471015d6e0 0x7f47100b6e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:18:29.616 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:29.609+0000 7f47152a9640 1 -- 172.21.15.143:0/839164094 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f47100b2160 msgr2=0x7f471015bd80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:18:29.617 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:29.609+0000 7f47152a9640 1 --2- 172.21.15.143:0/839164094 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f47100b2160 0x7f471015bd80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:18:29.617 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:29.609+0000 7f47152a9640 1 -- 172.21.15.143:0/839164094 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f47000094d0 con 0x7f47100d19f0 2023-11-26T07:18:29.617 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:29.609+0000 7f47152a9640 1 --2- 172.21.15.143:0/839164094 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f47100d19f0 0x7f47100b29c0 secure :-1 s=READY pgs=267 cs=0 l=1 rev1=1 crypto rx=0x7f470001a170 tx=0x7f47000192e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:18:29.618 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:29.609+0000 7f47067fc640 1 -- 172.21.15.143:0/839164094 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f47000ba070 con 0x7f47100d19f0 2023-11-26T07:18:29.618 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:29.609+0000 7f47067fc640 1 -- 172.21.15.143:0/839164094 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f47000b4420 con 0x7f47100d19f0 2023-11-26T07:18:29.618 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:29.609+0000 7f47067fc640 1 -- 172.21.15.143:0/839164094 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f47000b47d0 con 0x7f47100d19f0 2023-11-26T07:18:29.618 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:29.609+0000 7f4717d0e640 1 -- 172.21.15.143:0/839164094 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f47100b3610 con 0x7f47100d19f0 2023-11-26T07:18:29.618 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:29.609+0000 7f4717d0e640 1 -- 172.21.15.143:0/839164094 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f47100c7dd0 con 0x7f47100d19f0 2023-11-26T07:18:29.618 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:29.613+0000 7f4717d0e640 1 -- 172.21.15.143:0/839164094 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f47100b3ac0 con 0x7f47100d19f0 2023-11-26T07:18:29.618 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:29.613+0000 7f47067fc640 1 -- 172.21.15.143:0/839164094 <== mon.0 v2:172.21.15.143:3300/0 4 ==== mgrmap(e 5) v1 ==== 85852+0+0 (secure 0 0 0) 0x7f47000b4930 con 0x7f47100d19f0 2023-11-26T07:18:29.619 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:29.613+0000 7f47067fc640 1 --2- 172.21.15.143:0/839164094 >> [v2:172.21.15.143:6832/2150557544,v1:172.21.15.143:6833/2150557544] conn(0x7f46f4066470 0x7f46f4068930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:18:29.619 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:29.613+0000 7f47067fc640 1 -- 172.21.15.143:0/839164094 <== mon.0 v2:172.21.15.143:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f470011d940 con 0x7f47100d19f0 2023-11-26T07:18:29.619 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:29.613+0000 7f4715aaa640 1 --2- 172.21.15.143:0/839164094 >> [v2:172.21.15.143:6832/2150557544,v1:172.21.15.143:6833/2150557544] conn(0x7f46f4066470 0x7f46f4068930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:18:29.620 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:29.613+0000 7f4715aaa640 1 --2- 172.21.15.143:0/839164094 >> [v2:172.21.15.143:6832/2150557544,v1:172.21.15.143:6833/2150557544] conn(0x7f46f4066470 0x7f46f4068930 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7f470c006aa0 tx=0x7f470c006a30 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:18:29.623 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:29.617+0000 7f47067fc640 1 -- 172.21.15.143:0/839164094 <== mon.0 v2:172.21.15.143:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172649 (secure 0 0 0) 0x7f47000ead20 con 0x7f47100d19f0 2023-11-26T07:18:29.779 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:29.773+0000 7f4717d0e640 1 -- 172.21.15.143:0/839164094 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f4710067680 con 0x7f47100d19f0 2023-11-26T07:18:29.780 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:29.773+0000 7f47067fc640 1 -- 172.21.15.143:0/839164094 <== mon.0 v2:172.21.15.143:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v20) v1 ==== 74+0+16541 (secure 0 0 0) 0x7f47000ee340 con 0x7f47100d19f0 2023-11-26T07:18:29.781 INFO:teuthology.orchestra.run.smithi143.stdout: 2023-11-26T07:18:29.781 INFO:teuthology.orchestra.run.smithi143.stdout:{"epoch":20,"fsid":"f5033060-2469-4a07-95fe-2b76f5404bc1","created":"2023-11-26T07:17:28.613663+0000","modified":"2023-11-26T07:17:55.505435+0000","last_up_change":"2023-11-26T07:17:41.157813+0000","last_in_change":"2023-11-26T07:17:31.181658+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-11-26T07:17:41.758597+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2023-11-26T07:17:51.878332+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2023-11-26T07:17:52.871396+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"ebbee2c6-023e-44b9-9f1d-6f7bfd309730","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6817","nonce":639807369},{"type":"v1","addr":"172.21.15.143:6819","nonce":639807369}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6821","nonce":639807369},{"type":"v1","addr":"172.21.15.143:6823","nonce":639807369}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6830","nonce":639807369},{"type":"v1","addr":"172.21.15.143:6831","nonce":639807369}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6825","nonce":639807369},{"type":"v1","addr":"172.21.15.143:6828","nonce":639807369}]},"public_addr":"172.21.15.143:6819/639807369","cluster_addr":"172.21.15.143:6823/639807369","heartbeat_back_addr":"172.21.15.143:6831/639807369","heartbeat_front_addr":"172.21.15.143:6828/639807369","state":["exists","up"]},{"osd":1,"uuid":"fec4b8be-7122-49e9-9a19-5850a0700001","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6816","nonce":1144942530},{"type":"v1","addr":"172.21.15.143:6818","nonce":1144942530}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6820","nonce":1144942530},{"type":"v1","addr":"172.21.15.143:6822","nonce":1144942530}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6827","nonce":1144942530},{"type":"v1","addr":"172.21.15.143:6829","nonce":1144942530}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6824","nonce":1144942530},{"type":"v1","addr":"172.21.15.143:6826","nonce":1144942530}]},"public_addr":"172.21.15.143:6818/1144942530","cluster_addr":"172.21.15.143:6822/1144942530","heartbeat_back_addr":"172.21.15.143:6829/1144942530","heartbeat_front_addr":"172.21.15.143:6826/1144942530","state":["exists","up"]},{"osd":2,"uuid":"2888af5b-f933-46ca-8f95-6dd494ff43f6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6808","nonce":3909095186},{"type":"v1","addr":"172.21.15.143:6809","nonce":3909095186}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6810","nonce":3909095186},{"type":"v1","addr":"172.21.15.143:6811","nonce":3909095186}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6814","nonce":3909095186},{"type":"v1","addr":"172.21.15.143:6815","nonce":3909095186}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6812","nonce":3909095186},{"type":"v1","addr":"172.21.15.143:6813","nonce":3909095186}]},"public_addr":"172.21.15.143:6809/3909095186","cluster_addr":"172.21.15.143:6811/3909095186","heartbeat_back_addr":"172.21.15.143:6815/3909095186","heartbeat_front_addr":"172.21.15.143:6813/3909095186","state":["exists","up"]},{"osd":3,"uuid":"d1f52435-7bc1-4081-9550-9cd1ba55f549","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6800","nonce":3854210991},{"type":"v1","addr":"172.21.15.143:6801","nonce":3854210991}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6802","nonce":3854210991},{"type":"v1","addr":"172.21.15.143:6803","nonce":3854210991}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6806","nonce":3854210991},{"type":"v1","addr":"172.21.15.143:6807","nonce":3854210991}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6804","nonce":3854210991},{"type":"v1","addr":"172.21.15.143:6805","nonce":3854210991}]},"public_addr":"172.21.15.143:6801/3854210991","cluster_addr":"172.21.15.143:6803/3854210991","heartbeat_back_addr":"172.21.15.143:6807/3854210991","heartbeat_front_addr":"172.21.15.143:6805/3854210991","state":["exists","up"]},{"osd":4,"uuid":"e2ae8b6e-6e25-49b2-bcbf-aacce68f06a3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6800","nonce":3924606902},{"type":"v1","addr":"172.21.15.188:6801","nonce":3924606902}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6802","nonce":3924606902},{"type":"v1","addr":"172.21.15.188:6803","nonce":3924606902}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6806","nonce":3924606902},{"type":"v1","addr":"172.21.15.188:6807","nonce":3924606902}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6804","nonce":3924606902},{"type":"v1","addr":"172.21.15.188:6805","nonce":3924606902}]},"public_addr":"172.21.15.188:6801/3924606902","cluster_addr":"172.21.15.188:6803/3924606902","heartbeat_back_addr":"172.21.15.188:6807/3924606902","heartbeat_front_addr":"172.21.15.188:6805/3924606902","state":["exists","up"]},{"osd":5,"uuid":"3cec8a32-7698-4c36-8e89-e4ba11934f33","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6813","nonce":186841544},{"type":"v1","addr":"172.21.15.188:6815","nonce":186841544}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6817","nonce":186841544},{"type":"v1","addr":"172.21.15.188:6818","nonce":186841544}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6822","nonce":186841544},{"type":"v1","addr":"172.21.15.188:6823","nonce":186841544}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6820","nonce":186841544},{"type":"v1","addr":"172.21.15.188:6821","nonce":186841544}]},"public_addr":"172.21.15.188:6815/186841544","cluster_addr":"172.21.15.188:6818/186841544","heartbeat_back_addr":"172.21.15.188:6823/186841544","heartbeat_front_addr":"172.21.15.188:6821/186841544","state":["exists","up"]},{"osd":6,"uuid":"58a65d5d-e090-473a-81b4-ad38e85029f9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6808","nonce":2153499405},{"type":"v1","addr":"172.21.15.188:6809","nonce":2153499405}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6810","nonce":2153499405},{"type":"v1","addr":"172.21.15.188:6811","nonce":2153499405}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6816","nonce":2153499405},{"type":"v1","addr":"172.21.15.188:6819","nonce":2153499405}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6812","nonce":2153499405},{"type":"v1","addr":"172.21.15.188:6814","nonce":2153499405}]},"public_addr":"172.21.15.188:6809/2153499405","cluster_addr":"172.21.15.188:6811/2153499405","heartbeat_back_addr":"172.21.15.188:6819/2153499405","heartbeat_front_addr":"172.21.15.188:6814/2153499405","state":["exists","up"]},{"osd":7,"uuid":"ea5b7f70-3554-4cf8-ac30-f7c59b80678d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6824","nonce":2722096729},{"type":"v1","addr":"172.21.15.188:6825","nonce":2722096729}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6826","nonce":2722096729},{"type":"v1","addr":"172.21.15.188:6827","nonce":2722096729}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6830","nonce":2722096729},{"type":"v1","addr":"172.21.15.188:6831","nonce":2722096729}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6828","nonce":2722096729},{"type":"v1","addr":"172.21.15.188:6829","nonce":2722096729}]},"public_addr":"172.21.15.188:6825/2722096729","cluster_addr":"172.21.15.188:6827/2722096729","heartbeat_back_addr":"172.21.15.188:6831/2722096729","heartbeat_front_addr":"172.21.15.188:6829/2722096729","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-26T07:17:36.995032+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-26T07:17:36.932266+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-26T07:17:36.829304+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-26T07:17:36.804297+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-26T07:17:36.667892+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-26T07:17:36.964868+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-26T07:17:36.827332+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-26T07:17:36.893179+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-11-26T07:18:29.783 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:29.777+0000 7f4717d0e640 1 -- 172.21.15.143:0/839164094 >> [v2:172.21.15.143:6832/2150557544,v1:172.21.15.143:6833/2150557544] conn(0x7f46f4066470 msgr2=0x7f46f4068930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:18:29.783 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:29.777+0000 7f4717d0e640 1 --2- 172.21.15.143:0/839164094 >> [v2:172.21.15.143:6832/2150557544,v1:172.21.15.143:6833/2150557544] conn(0x7f46f4066470 0x7f46f4068930 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7f470c006aa0 tx=0x7f470c006a30 comp rx=0 tx=0).stop 2023-11-26T07:18:29.783 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:29.777+0000 7f4717d0e640 1 -- 172.21.15.143:0/839164094 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f47100d19f0 msgr2=0x7f47100b29c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:18:29.783 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:29.777+0000 7f4717d0e640 1 --2- 172.21.15.143:0/839164094 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f47100d19f0 0x7f47100b29c0 secure :-1 s=READY pgs=267 cs=0 l=1 rev1=1 crypto rx=0x7f470001a170 tx=0x7f47000192e0 comp rx=0 tx=0).stop 2023-11-26T07:18:29.783 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:29.777+0000 7f4717d0e640 1 -- 172.21.15.143:0/839164094 shutdown_connections 2023-11-26T07:18:29.783 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:29.777+0000 7f4717d0e640 1 --2- 172.21.15.143:0/839164094 >> [v2:172.21.15.143:6832/2150557544,v1:172.21.15.143:6833/2150557544] conn(0x7f46f4066470 0x7f46f4068930 unknown :-1 s=CLOSED pgs=158 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:18:29.783 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:29.777+0000 7f4717d0e640 1 --2- 172.21.15.143:0/839164094 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f471015d6e0 0x7f47100b6e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:18:29.783 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:29.777+0000 7f4717d0e640 1 --2- 172.21.15.143:0/839164094 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f47100d19f0 0x7f47100b29c0 unknown :-1 s=CLOSED pgs=267 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:18:29.783 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:29.777+0000 7f4717d0e640 1 --2- 172.21.15.143:0/839164094 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f47100b2160 0x7f471015bd80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:18:29.783 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:29.777+0000 7f4717d0e640 1 -- 172.21.15.143:0/839164094 >> 172.21.15.143:0/839164094 conn(0x7f471009c550 msgr2=0x7f47100cfbe0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:18:29.783 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:29.777+0000 7f4717d0e640 1 -- 172.21.15.143:0/839164094 shutdown_connections 2023-11-26T07:18:29.784 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:29.777+0000 7f4717d0e640 1 -- 172.21.15.143:0/839164094 wait complete. 2023-11-26T07:18:29.803 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-11-26T07:17:41.758597+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2023-11-26T07:17:51.878332+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2023-11-26T07:17:52.871396+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '20', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2023-11-26T07:18:29.804 DEBUG:teuthology.orchestra.run.smithi143:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-11-26T07:18:29.944 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:29.937+0000 7f97fbda1640 1 Processor -- start 2023-11-26T07:18:29.944 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:29.937+0000 7f97fbda1640 1 -- start start 2023-11-26T07:18:29.946 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:29.941+0000 7f97fbda1640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f97f40ce7c0 0x7f97f415f6b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:18:29.946 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:29.941+0000 7f97fbda1640 1 -- --> v1:172.21.15.143:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f97f406c520 con 0x7f97f413e290 2023-11-26T07:18:29.946 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:29.941+0000 7f97fbda1640 1 -- --> v1:172.21.15.188:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f97f406c800 con 0x7f97f40cf990 2023-11-26T07:18:29.946 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:29.941+0000 7f97fbda1640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f97f406cae0 con 0x7f97f40ce7c0 2023-11-26T07:18:29.946 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:29.941+0000 7f97f9b3d640 1 --1- >> v1:172.21.15.143:6789/0 conn(0x7f97f413e290 0x7f97f40cf280 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.143:6789/0 says I am v1:172.21.15.143:47662/0 (socket says 172.21.15.143:47662) 2023-11-26T07:18:29.946 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:29.941+0000 7f97f9b3d640 1 -- 172.21.15.143:0/353098862 learned_addr learned my addr 172.21.15.143:0/353098862 (peer_addr_for_me v1:172.21.15.143:0/0) 2023-11-26T07:18:29.948 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:29.941+0000 7f97fa33e640 1 --2- 172.21.15.143:0/353098862 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f97f40ce7c0 0x7f97f415f6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:18:29.948 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:29.941+0000 7f97f8b3b640 1 -- 172.21.15.143:0/353098862 <== mon.0 v1:172.21.15.143:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3405005396 0 0) 0x7f97f406c520 con 0x7f97f413e290 2023-11-26T07:18:29.948 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:29.941+0000 7f97f8b3b640 1 -- 172.21.15.143:0/353098862 --> v1:172.21.15.143:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f97e0003530 con 0x7f97f413e290 2023-11-26T07:18:29.948 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:29.941+0000 7f97f8b3b640 1 -- 172.21.15.143:0/353098862 <== mon.1 v1:172.21.15.188:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3895508936 0 0) 0x7f97f406c800 con 0x7f97f40cf990 2023-11-26T07:18:29.948 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:29.941+0000 7f97f8b3b640 1 -- 172.21.15.143:0/353098862 --> v1:172.21.15.188:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f97f406c520 con 0x7f97f40cf990 2023-11-26T07:18:29.948 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:29.941+0000 7f97f8b3b640 1 -- 172.21.15.143:0/353098862 <== mon.0 v1:172.21.15.143:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1279776878 0 0) 0x7f97e0003530 con 0x7f97f413e290 2023-11-26T07:18:29.948 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:29.941+0000 7f97f8b3b640 1 -- 172.21.15.143:0/353098862 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f97f40ce7c0 msgr2=0x7f97f415f6b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-26T07:18:29.949 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:29.941+0000 7f97f8b3b640 1 --2- 172.21.15.143:0/353098862 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f97f40ce7c0 0x7f97f415f6b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:18:29.949 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:29.941+0000 7f97f8b3b640 1 -- 172.21.15.143:0/353098862 >> v1:172.21.15.188:6789/0 conn(0x7f97f40cf990 legacy=0x7f97f415efa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:18:29.949 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:29.941+0000 7f97f8b3b640 1 -- 172.21.15.143:0/353098862 --> v1:172.21.15.143:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f97f406cdc0 con 0x7f97f413e290 2023-11-26T07:18:29.949 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:29.941+0000 7f97f8b3b640 1 -- 172.21.15.143:0/353098862 <== mon.0 v1:172.21.15.143:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2137882523 0 0) 0x7f97f0001680 con 0x7f97f413e290 2023-11-26T07:18:29.949 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:29.941+0000 7f97f8b3b640 1 -- 172.21.15.143:0/353098862 >> v1:172.21.15.143:6789/0 conn(0x7f97f413e290 legacy=0x7f97f40cf280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:18:29.949 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:29.941+0000 7f97f8b3b640 1 --2- 172.21.15.143:0/353098862 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f97f40ce7c0 0x7f97e0003b40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:18:29.949 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:29.941+0000 7f97f8b3b640 1 --2- 172.21.15.143:0/353098862 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f97e00040f0 0x7f97e00047d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:18:29.949 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:29.941+0000 7f97f8b3b640 1 --2- 172.21.15.143:0/353098862 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f97e0004d10 0x7f97e0005190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:18:29.949 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:29.941+0000 7f97f8b3b640 1 -- 172.21.15.143:0/353098862 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f97f406cae0 con 0x7f97e00040f0 2023-11-26T07:18:29.949 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:29.941+0000 7f97f8b3b640 1 -- 172.21.15.143:0/353098862 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f97e000bad0 con 0x7f97e0004d10 2023-11-26T07:18:29.949 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:29.941+0000 7f97f8b3b640 1 -- 172.21.15.143:0/353098862 --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f97e000bc90 con 0x7f97f40ce7c0 2023-11-26T07:18:29.949 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:29.941+0000 7f97f933c640 1 --2- 172.21.15.143:0/353098862 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f97e00040f0 0x7f97e00047d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:18:29.950 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:29.941+0000 7f97f9b3d640 1 --2- 172.21.15.143:0/353098862 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f97f40ce7c0 0x7f97e0003b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:18:29.950 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:29.941+0000 7f97fa33e640 1 --2- 172.21.15.143:0/353098862 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f97e0004d10 0x7f97e0005190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:18:29.950 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:29.941+0000 7f97f933c640 1 -- 172.21.15.143:0/353098862 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f97f40ce7c0 msgr2=0x7f97e0003b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-26T07:18:29.950 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:29.941+0000 7f97f933c640 1 --2- 172.21.15.143:0/353098862 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f97f40ce7c0 0x7f97e0003b40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:18:29.950 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:29.941+0000 7f97f933c640 1 -- 172.21.15.143:0/353098862 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f97e0004d10 msgr2=0x7f97e0005190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-26T07:18:29.950 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:29.941+0000 7f97f933c640 1 --2- 172.21.15.143:0/353098862 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f97e0004d10 0x7f97e0005190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:18:29.950 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:29.941+0000 7f97f933c640 1 -- 172.21.15.143:0/353098862 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f97e0003530 con 0x7f97e00040f0 2023-11-26T07:18:29.951 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:29.945+0000 7f97f933c640 1 --2- 172.21.15.143:0/353098862 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f97e00040f0 0x7f97e00047d0 secure :-1 s=READY pgs=269 cs=0 l=1 rev1=1 crypto rx=0x7f97e40018d0 tx=0x7f97e401ec10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a1a00ad01dce431a server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:18:29.951 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:29.945+0000 7f97f8b3b640 1 -- 172.21.15.143:0/353098862 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f97e40052d0 con 0x7f97e00040f0 2023-11-26T07:18:29.951 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:29.945+0000 7f97f8b3b640 1 -- 172.21.15.143:0/353098862 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f97e4005450 con 0x7f97e00040f0 2023-11-26T07:18:29.951 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:29.945+0000 7f97f8b3b640 1 -- 172.21.15.143:0/353098862 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f97e40a8870 con 0x7f97e00040f0 2023-11-26T07:18:29.951 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:29.945+0000 7f97fbda1640 1 -- 172.21.15.143:0/353098862 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f97e00040f0 msgr2=0x7f97e00047d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:18:29.951 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:29.945+0000 7f97fbda1640 1 --2- 172.21.15.143:0/353098862 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f97e00040f0 0x7f97e00047d0 secure :-1 s=READY pgs=269 cs=0 l=1 rev1=1 crypto rx=0x7f97e40018d0 tx=0x7f97e401ec10 comp rx=0 tx=0).stop 2023-11-26T07:18:29.952 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:29.945+0000 7f97fa33e640 1 -- 172.21.15.143:0/353098862 reap_dead start 2023-11-26T07:18:29.952 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:29.945+0000 7f97fbda1640 1 -- 172.21.15.143:0/353098862 shutdown_connections 2023-11-26T07:18:29.952 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:29.945+0000 7f97fbda1640 1 -- 172.21.15.143:0/353098862 >> 172.21.15.143:0/353098862 conn(0x7f97f406a890 msgr2=0x7f97f406aca0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:18:29.952 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:29.945+0000 7f97fbda1640 1 -- 172.21.15.143:0/353098862 shutdown_connections 2023-11-26T07:18:29.952 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:29.945+0000 7f97fbda1640 1 -- 172.21.15.143:0/353098862 wait complete. 2023-11-26T07:18:29.953 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:29.949+0000 7f97fbda1640 1 Processor -- start 2023-11-26T07:18:29.953 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:29.949+0000 7f97fbda1640 1 -- start start 2023-11-26T07:18:29.955 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:29.949+0000 7f97fbda1640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f97e00040f0 0x7f97f41f7010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:18:29.955 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:29.949+0000 7f97fbda1640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f97f41f2f50 0x7f97f4139790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:18:29.955 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:29.949+0000 7f97fbda1640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f97f40a84d0 0x7f97f40b1080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:18:29.956 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:29.949+0000 7f97fbda1640 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f97f41f7550 con 0x7f97f41f2f50 2023-11-26T07:18:29.956 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:29.949+0000 7f97fbda1640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f97f41f3720 con 0x7f97e00040f0 2023-11-26T07:18:29.956 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:29.949+0000 7f97fbda1640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f97f41f3a00 con 0x7f97f40a84d0 2023-11-26T07:18:29.956 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:29.949+0000 7f97f933c640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f97f41f2f50 0x7f97f4139790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:18:29.956 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:29.949+0000 7f97f933c640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f97f41f2f50 0x7f97f4139790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:33472/0 (socket says 172.21.15.143:33472) 2023-11-26T07:18:29.956 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:29.949+0000 7f97f933c640 1 -- 172.21.15.143:0/1883340875 learned_addr learned my addr 172.21.15.143:0/1883340875 (peer_addr_for_me v2:172.21.15.143:0/0) 2023-11-26T07:18:29.956 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:29.949+0000 7f97fa33e640 1 --2- 172.21.15.143:0/1883340875 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f97f40a84d0 0x7f97f40b1080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:18:29.956 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:29.949+0000 7f97f9b3d640 1 --2- 172.21.15.143:0/1883340875 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f97e00040f0 0x7f97f41f7010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:18:29.956 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:29.949+0000 7f97f933c640 1 -- 172.21.15.143:0/1883340875 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f97f40a84d0 msgr2=0x7f97f40b1080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:18:29.956 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:29.949+0000 7f97f933c640 1 --2- 172.21.15.143:0/1883340875 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f97f40a84d0 0x7f97f40b1080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:18:29.956 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:29.949+0000 7f97f933c640 1 -- 172.21.15.143:0/1883340875 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f97e00040f0 msgr2=0x7f97f41f7010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:18:29.957 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:29.949+0000 7f97f933c640 1 --2- 172.21.15.143:0/1883340875 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f97e00040f0 0x7f97f41f7010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:18:29.957 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:29.949+0000 7f97f933c640 1 -- 172.21.15.143:0/1883340875 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f97f41f3ce0 con 0x7f97f41f2f50 2023-11-26T07:18:29.957 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:29.949+0000 7f97fa33e640 1 --2- 172.21.15.143:0/1883340875 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f97f40a84d0 0x7f97f40b1080 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-26T07:18:29.957 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:29.949+0000 7f97f9b3d640 1 --2- 172.21.15.143:0/1883340875 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f97e00040f0 0x7f97f41f7010 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-26T07:18:29.957 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:29.949+0000 7f97f933c640 1 --2- 172.21.15.143:0/1883340875 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f97f41f2f50 0x7f97f4139790 secure :-1 s=READY pgs=270 cs=0 l=1 rev1=1 crypto rx=0x7f97e40018d0 tx=0x7f97e0005800 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:18:29.957 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:29.949+0000 7f97ea7fc640 1 -- 172.21.15.143:0/1883340875 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f97e40a8d60 con 0x7f97f41f2f50 2023-11-26T07:18:29.957 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:29.949+0000 7f97ea7fc640 1 -- 172.21.15.143:0/1883340875 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f97e40050d0 con 0x7f97f41f2f50 2023-11-26T07:18:29.957 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:29.949+0000 7f97fbda1640 1 -- 172.21.15.143:0/1883340875 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f97f40b0b60 con 0x7f97f41f2f50 2023-11-26T07:18:29.957 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:29.949+0000 7f97ea7fc640 1 -- 172.21.15.143:0/1883340875 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f97e4003600 con 0x7f97f41f2f50 2023-11-26T07:18:29.957 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:29.949+0000 7f97fbda1640 1 -- 172.21.15.143:0/1883340875 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f97f4139cd0 con 0x7f97f41f2f50 2023-11-26T07:18:29.958 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:29.949+0000 7f97fbda1640 1 -- 172.21.15.143:0/1883340875 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f97f41f3f20 con 0x7f97f41f2f50 2023-11-26T07:18:29.958 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:29.953+0000 7f97ea7fc640 1 -- 172.21.15.143:0/1883340875 <== mon.0 v2:172.21.15.143:3300/0 4 ==== mgrmap(e 5) v1 ==== 85852+0+0 (secure 0 0 0) 0x7f97e4003760 con 0x7f97f41f2f50 2023-11-26T07:18:29.958 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:29.953+0000 7f97ea7fc640 1 --2- 172.21.15.143:0/1883340875 >> [v2:172.21.15.143:6832/2150557544,v1:172.21.15.143:6833/2150557544] conn(0x7f97d8066470 0x7f97d8068930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:18:29.958 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:29.953+0000 7f97ea7fc640 1 -- 172.21.15.143:0/1883340875 <== mon.0 v2:172.21.15.143:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f97e4120ed0 con 0x7f97f41f2f50 2023-11-26T07:18:29.958 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:29.953+0000 7f97f9b3d640 1 --2- 172.21.15.143:0/1883340875 >> [v2:172.21.15.143:6832/2150557544,v1:172.21.15.143:6833/2150557544] conn(0x7f97d8066470 0x7f97d8068930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:18:29.959 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:29.953+0000 7f97f9b3d640 1 --2- 172.21.15.143:0/1883340875 >> [v2:172.21.15.143:6832/2150557544,v1:172.21.15.143:6833/2150557544] conn(0x7f97d8066470 0x7f97d8068930 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7f97f0000c10 tx=0x7f97f0009040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:18:29.962 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:29.957+0000 7f97ea7fc640 1 -- 172.21.15.143:0/1883340875 <== mon.0 v2:172.21.15.143:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172649 (secure 0 0 0) 0x7f97f41f3f20 con 0x7f97f41f2f50 2023-11-26T07:18:30.111 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:30.105+0000 7f97fbda1640 1 -- 172.21.15.143:0/1883340875 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f97f40b09c0 con 0x7f97f41f2f50 2023-11-26T07:18:30.112 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:30.105+0000 7f97ea7fc640 1 -- 172.21.15.143:0/1883340875 <== mon.0 v2:172.21.15.143:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v20) v1 ==== 93+0+10 (secure 0 0 0) 0x7f97f40b09c0 con 0x7f97f41f2f50 2023-11-26T07:18:30.112 INFO:teuthology.orchestra.run.smithi143.stdout:pg_num: 1 2023-11-26T07:18:30.114 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:30.109+0000 7f97fbda1640 1 -- 172.21.15.143:0/1883340875 >> [v2:172.21.15.143:6832/2150557544,v1:172.21.15.143:6833/2150557544] conn(0x7f97d8066470 msgr2=0x7f97d8068930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:18:30.114 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:30.109+0000 7f97fbda1640 1 --2- 172.21.15.143:0/1883340875 >> [v2:172.21.15.143:6832/2150557544,v1:172.21.15.143:6833/2150557544] conn(0x7f97d8066470 0x7f97d8068930 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7f97f0000c10 tx=0x7f97f0009040 comp rx=0 tx=0).stop 2023-11-26T07:18:30.115 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:30.109+0000 7f97fbda1640 1 -- 172.21.15.143:0/1883340875 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f97f41f2f50 msgr2=0x7f97f4139790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:18:30.115 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:30.109+0000 7f97fbda1640 1 --2- 172.21.15.143:0/1883340875 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f97f41f2f50 0x7f97f4139790 secure :-1 s=READY pgs=270 cs=0 l=1 rev1=1 crypto rx=0x7f97e40018d0 tx=0x7f97e0005800 comp rx=0 tx=0).stop 2023-11-26T07:18:30.115 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:30.109+0000 7f97fbda1640 1 -- 172.21.15.143:0/1883340875 shutdown_connections 2023-11-26T07:18:30.115 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:30.109+0000 7f97fbda1640 1 --2- 172.21.15.143:0/1883340875 >> [v2:172.21.15.143:6832/2150557544,v1:172.21.15.143:6833/2150557544] conn(0x7f97d8066470 0x7f97d8068930 unknown :-1 s=CLOSED pgs=159 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:18:30.115 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:30.109+0000 7f97fbda1640 1 --2- 172.21.15.143:0/1883340875 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f97f40a84d0 0x7f97f40b1080 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:18:30.115 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:30.109+0000 7f97fbda1640 1 --2- 172.21.15.143:0/1883340875 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f97f41f2f50 0x7f97f4139790 unknown :-1 s=CLOSED pgs=270 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:18:30.115 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:30.109+0000 7f97fbda1640 1 --2- 172.21.15.143:0/1883340875 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f97e00040f0 0x7f97f41f7010 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:18:30.116 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:30.109+0000 7f97fbda1640 1 -- 172.21.15.143:0/1883340875 >> 172.21.15.143:0/1883340875 conn(0x7f97f406a890 msgr2=0x7f97f413bf20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:18:30.116 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:30.109+0000 7f97fbda1640 1 -- 172.21.15.143:0/1883340875 shutdown_connections 2023-11-26T07:18:30.116 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:30.109+0000 7f97fbda1640 1 -- 172.21.15.143:0/1883340875 wait complete. 2023-11-26T07:18:30.133 DEBUG:teuthology.orchestra.run.smithi143:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-11-26T07:18:30.273 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:30.265+0000 7fb9e693e640 1 Processor -- start 2023-11-26T07:18:30.273 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:30.269+0000 7fb9e693e640 1 -- start start 2023-11-26T07:18:30.274 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:30.269+0000 7fb9e693e640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fb9e0143ac0 0x7fb9e0070370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:18:30.274 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:30.269+0000 7fb9e693e640 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7fb9e00cf990 0x7fb9e00cd1b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:18:30.274 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:30.269+0000 7fb9e693e640 1 -- --> v1:172.21.15.143:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb9e00cc730 con 0x7fb9e00708b0 2023-11-26T07:18:30.274 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:30.269+0000 7fb9e693e640 1 -- --> v2:172.21.15.188:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb9e0143ea0 con 0x7fb9e00cf990 2023-11-26T07:18:30.274 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:30.269+0000 7fb9e693e640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb9e0070ee0 con 0x7fb9e0143ac0 2023-11-26T07:18:30.274 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:30.269+0000 7fb9df7fe640 1 --1- >> v1:172.21.15.143:6789/0 conn(0x7fb9e00708b0 0x7fb9e00cf280 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.143:6789/0 says I am v1:172.21.15.143:47674/0 (socket says 172.21.15.143:47674) 2023-11-26T07:18:30.275 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:30.269+0000 7fb9df7fe640 1 -- 172.21.15.143:0/3906048328 learned_addr learned my addr 172.21.15.143:0/3906048328 (peer_addr_for_me v1:172.21.15.143:0/0) 2023-11-26T07:18:30.275 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:30.269+0000 7fb9dffff640 1 --2- 172.21.15.143:0/3906048328 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fb9e0143ac0 0x7fb9e0070370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:18:30.275 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:30.269+0000 7fb9e4edb640 1 --2- 172.21.15.143:0/3906048328 >> v2:172.21.15.188:3300/0 conn(0x7fb9e00cf990 0x7fb9e00cd1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:18:30.276 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:30.269+0000 7fb9deffd640 1 -- 172.21.15.143:0/3906048328 <== mon.0 v1:172.21.15.143:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2166819460 0 0) 0x7fb9e00cc730 con 0x7fb9e00708b0 2023-11-26T07:18:30.276 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:30.269+0000 7fb9deffd640 1 -- 172.21.15.143:0/3906048328 --> v1:172.21.15.143:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fb9c8003530 con 0x7fb9e00708b0 2023-11-26T07:18:30.276 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:30.269+0000 7fb9e4edb640 1 -- 172.21.15.143:0/3906048328 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fb9e0143ac0 msgr2=0x7fb9e0070370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-26T07:18:30.276 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:30.269+0000 7fb9e4edb640 1 --2- 172.21.15.143:0/3906048328 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fb9e0143ac0 0x7fb9e0070370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:18:30.276 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:30.269+0000 7fb9e4edb640 1 -- 172.21.15.143:0/3906048328 >> v1:172.21.15.143:6789/0 conn(0x7fb9e00708b0 legacy=0x7fb9e00cf280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:18:30.276 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:30.269+0000 7fb9e4edb640 1 -- 172.21.15.143:0/3906048328 --> v2:172.21.15.188:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb9e00711c0 con 0x7fb9e00cf990 2023-11-26T07:18:30.278 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:30.273+0000 7fb9e4edb640 1 --2- 172.21.15.143:0/3906048328 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fb9e00cf990 0x7fb9e00cd1b0 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7fb9d8009d90 tx=0x7fb9d801c950 comp rx=0 tx=0).ready entity=mon.1 client_cookie=2872e01d79e48dd1 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:18:30.278 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:30.273+0000 7fb9deffd640 1 -- 172.21.15.143:0/3906048328 <== mon.1 v2:172.21.15.188:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb9d80a5920 con 0x7fb9e00cf990 2023-11-26T07:18:30.278 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:30.273+0000 7fb9deffd640 1 -- 172.21.15.143:0/3906048328 <== mon.1 v2:172.21.15.188:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb9d80a5a80 con 0x7fb9e00cf990 2023-11-26T07:18:30.278 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:30.273+0000 7fb9deffd640 1 -- 172.21.15.143:0/3906048328 <== mon.1 v2:172.21.15.188:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb9d80a5df0 con 0x7fb9e00cf990 2023-11-26T07:18:30.279 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:30.273+0000 7fb9e693e640 1 -- 172.21.15.143:0/3906048328 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fb9e00cf990 msgr2=0x7fb9e00cd1b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:18:30.279 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:30.273+0000 7fb9e693e640 1 --2- 172.21.15.143:0/3906048328 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fb9e00cf990 0x7fb9e00cd1b0 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7fb9d8009d90 tx=0x7fb9d801c950 comp rx=0 tx=0).stop 2023-11-26T07:18:30.279 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:30.273+0000 7fb9e693e640 1 -- 172.21.15.143:0/3906048328 shutdown_connections 2023-11-26T07:18:30.279 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:30.273+0000 7fb9e693e640 1 --2- 172.21.15.143:0/3906048328 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fb9e00cf990 0x7fb9e00cd1b0 unknown :-1 s=CLOSED pgs=140 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:18:30.279 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:30.273+0000 7fb9e693e640 1 --2- 172.21.15.143:0/3906048328 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fb9e0143ac0 0x7fb9e0070370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:18:30.279 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:30.273+0000 7fb9e693e640 1 -- 172.21.15.143:0/3906048328 >> 172.21.15.143:0/3906048328 conn(0x7fb9e006a890 msgr2=0x7fb9e006aca0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:18:30.279 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:30.273+0000 7fb9e693e640 1 -- 172.21.15.143:0/3906048328 shutdown_connections 2023-11-26T07:18:30.280 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:30.273+0000 7fb9e693e640 1 -- 172.21.15.143:0/3906048328 wait complete. 2023-11-26T07:18:30.280 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:30.273+0000 7fb9e693e640 1 Processor -- start 2023-11-26T07:18:30.281 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:30.277+0000 7fb9e693e640 1 -- start start 2023-11-26T07:18:30.282 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:30.277+0000 7fb9e693e640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fb9e00708b0 0x7fb9e006cf70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:18:30.282 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:30.277+0000 7fb9e693e640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fb9e00cf990 0x7fb9e00b16c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:18:30.282 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:30.277+0000 7fb9e693e640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fb9e0143ac0 0x7fb9e00af670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:18:30.283 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:30.277+0000 7fb9e693e640 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb9e00afca0 con 0x7fb9e0143ac0 2023-11-26T07:18:30.283 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:30.277+0000 7fb9e693e640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb9e00aff80 con 0x7fb9e00708b0 2023-11-26T07:18:30.283 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:30.277+0000 7fb9e693e640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb9e00b0260 con 0x7fb9e00cf990 2023-11-26T07:18:30.283 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:30.277+0000 7fb9e4edb640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fb9e0143ac0 0x7fb9e00af670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:18:30.283 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:30.277+0000 7fb9e4edb640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fb9e0143ac0 0x7fb9e00af670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:33482/0 (socket says 172.21.15.143:33482) 2023-11-26T07:18:30.283 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:30.277+0000 7fb9e4edb640 1 -- 172.21.15.143:0/3391542961 learned_addr learned my addr 172.21.15.143:0/3391542961 (peer_addr_for_me v2:172.21.15.143:0/0) 2023-11-26T07:18:30.283 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:30.277+0000 7fb9df7fe640 1 --2- 172.21.15.143:0/3391542961 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fb9e00cf990 0x7fb9e00b16c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:18:30.283 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:30.277+0000 7fb9dffff640 1 --2- 172.21.15.143:0/3391542961 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fb9e00708b0 0x7fb9e006cf70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:18:30.283 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:30.277+0000 7fb9e4edb640 1 -- 172.21.15.143:0/3391542961 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fb9e00cf990 msgr2=0x7fb9e00b16c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:18:30.283 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:30.277+0000 7fb9e4edb640 1 --2- 172.21.15.143:0/3391542961 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fb9e00cf990 0x7fb9e00b16c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:18:30.283 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:30.277+0000 7fb9e4edb640 1 -- 172.21.15.143:0/3391542961 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fb9e00708b0 msgr2=0x7fb9e006cf70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:18:30.283 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:30.277+0000 7fb9e4edb640 1 --2- 172.21.15.143:0/3391542961 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fb9e00708b0 0x7fb9e006cf70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:18:30.283 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:30.277+0000 7fb9e4edb640 1 -- 172.21.15.143:0/3391542961 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb9d8009a70 con 0x7fb9e0143ac0 2023-11-26T07:18:30.284 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:30.277+0000 7fb9e4edb640 1 --2- 172.21.15.143:0/3391542961 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fb9e0143ac0 0x7fb9e00af670 secure :-1 s=READY pgs=272 cs=0 l=1 rev1=1 crypto rx=0x7fb9d8009d90 tx=0x7fb9d80196f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:18:30.284 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:30.277+0000 7fb9dcff9640 1 -- 172.21.15.143:0/3391542961 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb9d80aed10 con 0x7fb9e0143ac0 2023-11-26T07:18:30.285 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:30.277+0000 7fb9dcff9640 1 -- 172.21.15.143:0/3391542961 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb9d80aee70 con 0x7fb9e0143ac0 2023-11-26T07:18:30.285 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:30.277+0000 7fb9dcff9640 1 -- 172.21.15.143:0/3391542961 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb9d80b7610 con 0x7fb9e0143ac0 2023-11-26T07:18:30.285 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:30.277+0000 7fb9e693e640 1 -- 172.21.15.143:0/3391542961 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb9e00b0540 con 0x7fb9e0143ac0 2023-11-26T07:18:30.285 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:30.277+0000 7fb9e693e640 1 -- 172.21.15.143:0/3391542961 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb9e00b1400 con 0x7fb9e0143ac0 2023-11-26T07:18:30.285 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:30.277+0000 7fb9e693e640 1 -- 172.21.15.143:0/3391542961 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb9e00b0a60 con 0x7fb9e0143ac0 2023-11-26T07:18:30.285 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:30.281+0000 7fb9dcff9640 1 -- 172.21.15.143:0/3391542961 <== mon.0 v2:172.21.15.143:3300/0 4 ==== mgrmap(e 5) v1 ==== 85852+0+0 (secure 0 0 0) 0x7fb9d80b4050 con 0x7fb9e0143ac0 2023-11-26T07:18:30.286 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:30.281+0000 7fb9dcff9640 1 --2- 172.21.15.143:0/3391542961 >> [v2:172.21.15.143:6832/2150557544,v1:172.21.15.143:6833/2150557544] conn(0x7fb9cc066470 0x7fb9cc068930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:18:30.286 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:30.281+0000 7fb9dcff9640 1 -- 172.21.15.143:0/3391542961 <== mon.0 v2:172.21.15.143:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fb9d811edc0 con 0x7fb9e0143ac0 2023-11-26T07:18:30.286 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:30.281+0000 7fb9dffff640 1 --2- 172.21.15.143:0/3391542961 >> [v2:172.21.15.143:6832/2150557544,v1:172.21.15.143:6833/2150557544] conn(0x7fb9cc066470 0x7fb9cc068930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:18:30.286 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:30.281+0000 7fb9dffff640 1 --2- 172.21.15.143:0/3391542961 >> [v2:172.21.15.143:6832/2150557544,v1:172.21.15.143:6833/2150557544] conn(0x7fb9cc066470 0x7fb9cc068930 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7fb9d0009ac0 tx=0x7fb9d0009190 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:18:30.290 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:30.285+0000 7fb9dcff9640 1 -- 172.21.15.143:0/3391542961 <== mon.0 v2:172.21.15.143:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172649 (secure 0 0 0) 0x7fb9e00b0a60 con 0x7fb9e0143ac0 2023-11-26T07:18:30.439 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:30.433+0000 7fb9e693e640 1 -- 172.21.15.143:0/3391542961 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7fb9e00b0a60 con 0x7fb9e0143ac0 2023-11-26T07:18:30.439 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:30.433+0000 7fb9dcff9640 1 -- 172.21.15.143:0/3391542961 <== mon.0 v2:172.21.15.143:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v20) v1 ==== 104+0+11 (secure 0 0 0) 0x7fb9e00b0a60 con 0x7fb9e0143ac0 2023-11-26T07:18:30.439 INFO:teuthology.orchestra.run.smithi143.stdout:pg_num: 32 2023-11-26T07:18:30.441 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:30.433+0000 7fb9e693e640 1 -- 172.21.15.143:0/3391542961 >> [v2:172.21.15.143:6832/2150557544,v1:172.21.15.143:6833/2150557544] conn(0x7fb9cc066470 msgr2=0x7fb9cc068930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:18:30.441 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:30.433+0000 7fb9e693e640 1 --2- 172.21.15.143:0/3391542961 >> [v2:172.21.15.143:6832/2150557544,v1:172.21.15.143:6833/2150557544] conn(0x7fb9cc066470 0x7fb9cc068930 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7fb9d0009ac0 tx=0x7fb9d0009190 comp rx=0 tx=0).stop 2023-11-26T07:18:30.441 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:30.433+0000 7fb9e693e640 1 -- 172.21.15.143:0/3391542961 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fb9e0143ac0 msgr2=0x7fb9e00af670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:18:30.441 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:30.433+0000 7fb9e693e640 1 --2- 172.21.15.143:0/3391542961 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fb9e0143ac0 0x7fb9e00af670 secure :-1 s=READY pgs=272 cs=0 l=1 rev1=1 crypto rx=0x7fb9d8009d90 tx=0x7fb9d80196f0 comp rx=0 tx=0).stop 2023-11-26T07:18:30.441 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:30.437+0000 7fb9e693e640 1 -- 172.21.15.143:0/3391542961 shutdown_connections 2023-11-26T07:18:30.441 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:30.437+0000 7fb9e693e640 1 --2- 172.21.15.143:0/3391542961 >> [v2:172.21.15.143:6832/2150557544,v1:172.21.15.143:6833/2150557544] conn(0x7fb9cc066470 0x7fb9cc068930 unknown :-1 s=CLOSED pgs=160 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:18:30.441 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:30.437+0000 7fb9e693e640 1 --2- 172.21.15.143:0/3391542961 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fb9e0143ac0 0x7fb9e00af670 unknown :-1 s=CLOSED pgs=272 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:18:30.442 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:30.437+0000 7fb9e693e640 1 --2- 172.21.15.143:0/3391542961 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fb9e00cf990 0x7fb9e00b16c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:18:30.442 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:30.437+0000 7fb9e693e640 1 --2- 172.21.15.143:0/3391542961 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fb9e00708b0 0x7fb9e006cf70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:18:30.442 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:30.437+0000 7fb9e693e640 1 -- 172.21.15.143:0/3391542961 >> 172.21.15.143:0/3391542961 conn(0x7fb9e006a890 msgr2=0x7fb9e006ac70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:18:30.442 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:30.437+0000 7fb9e693e640 1 -- 172.21.15.143:0/3391542961 shutdown_connections 2023-11-26T07:18:30.442 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:30.437+0000 7fb9e693e640 1 -- 172.21.15.143:0/3391542961 wait complete. 2023-11-26T07:18:30.461 DEBUG:teuthology.orchestra.run.smithi143:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-11-26T07:18:30.600 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:30.593+0000 7f82cfa32640 1 Processor -- start 2023-11-26T07:18:30.600 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:30.593+0000 7f82cfa32640 1 -- start start 2023-11-26T07:18:30.603 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:30.597+0000 7f82cfa32640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f82c813e290 0x7f82c80cf280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:18:30.603 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:30.597+0000 7f82cfa32640 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7f82c80cf7c0 0x7f82c815efa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:18:30.603 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:30.597+0000 7f82cfa32640 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7f82c815f4e0 0x7f82c81408e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:18:30.603 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:30.597+0000 7f82cfa32640 1 -- --> v2:172.21.15.143:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f82c80cfc40 con 0x7f82c815f4e0 2023-11-26T07:18:30.603 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:30.597+0000 7f82cfa32640 1 -- --> v2:172.21.15.188:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f82c806c520 con 0x7f82c80cf7c0 2023-11-26T07:18:30.603 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:30.597+0000 7f82cfa32640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f82c806c800 con 0x7f82c813e290 2023-11-26T07:18:30.603 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:30.597+0000 7f82cdfcf640 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7f82c815f4e0 0x7f82c81408e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:18:30.603 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:30.597+0000 7f82cdfcf640 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7f82c815f4e0 0x7f82c81408e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:33492/0 (socket says 172.21.15.143:33492) 2023-11-26T07:18:30.603 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:30.597+0000 7f82cdfcf640 1 -- 172.21.15.143:0/863720907 learned_addr learned my addr 172.21.15.143:0/863720907 (peer_addr_for_me v2:172.21.15.143:0/0) 2023-11-26T07:18:30.603 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:30.597+0000 7f82ccfcd640 1 --2- 172.21.15.143:0/863720907 >> v2:172.21.15.188:3300/0 conn(0x7f82c80cf7c0 0x7f82c815efa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:18:30.603 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:30.597+0000 7f82cd7ce640 1 --2- 172.21.15.143:0/863720907 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f82c813e290 0x7f82c80cf280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:18:30.604 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:30.597+0000 7f82cdfcf640 1 -- 172.21.15.143:0/863720907 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f82c813e290 msgr2=0x7f82c80cf280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-26T07:18:30.604 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:30.597+0000 7f82cdfcf640 1 --2- 172.21.15.143:0/863720907 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f82c813e290 0x7f82c80cf280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:18:30.604 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:30.597+0000 7f82cdfcf640 1 -- 172.21.15.143:0/863720907 >> v2:172.21.15.188:3300/0 conn(0x7f82c80cf7c0 msgr2=0x7f82c815efa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-26T07:18:30.604 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:30.597+0000 7f82cdfcf640 1 --2- 172.21.15.143:0/863720907 >> v2:172.21.15.188:3300/0 conn(0x7f82c80cf7c0 0x7f82c815efa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:18:30.604 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:30.597+0000 7f82cdfcf640 1 -- 172.21.15.143:0/863720907 --> v2:172.21.15.143:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f82c806cae0 con 0x7f82c815f4e0 2023-11-26T07:18:30.605 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:30.601+0000 7f82cdfcf640 1 --2- 172.21.15.143:0/863720907 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f82c815f4e0 0x7f82c81408e0 secure :-1 s=READY pgs=273 cs=0 l=1 rev1=1 crypto rx=0x7f82c80b45c0 tx=0x7f82c001c6e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e1e08025ac7bf52b server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:18:30.606 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:30.601+0000 7f82bffff640 1 -- 172.21.15.143:0/863720907 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f82c00a5560 con 0x7f82c815f4e0 2023-11-26T07:18:30.606 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:30.601+0000 7f82bffff640 1 -- 172.21.15.143:0/863720907 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f82c00a56c0 con 0x7f82c815f4e0 2023-11-26T07:18:30.606 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:30.601+0000 7f82bffff640 1 -- 172.21.15.143:0/863720907 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f82c00a5a50 con 0x7f82c815f4e0 2023-11-26T07:18:30.606 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:30.601+0000 7f82cfa32640 1 -- 172.21.15.143:0/863720907 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f82c815f4e0 msgr2=0x7f82c81408e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:18:30.607 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:30.601+0000 7f82cfa32640 1 --2- 172.21.15.143:0/863720907 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f82c815f4e0 0x7f82c81408e0 secure :-1 s=READY pgs=273 cs=0 l=1 rev1=1 crypto rx=0x7f82c80b45c0 tx=0x7f82c001c6e0 comp rx=0 tx=0).stop 2023-11-26T07:18:30.607 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:30.601+0000 7f82cfa32640 1 -- 172.21.15.143:0/863720907 shutdown_connections 2023-11-26T07:18:30.607 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:30.601+0000 7f82cfa32640 1 --2- 172.21.15.143:0/863720907 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f82c815f4e0 0x7f82c81408e0 unknown :-1 s=CLOSED pgs=273 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:18:30.607 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:30.601+0000 7f82cfa32640 1 --2- 172.21.15.143:0/863720907 >> v2:172.21.15.188:3300/0 conn(0x7f82c80cf7c0 0x7f82c815efa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:18:30.607 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:30.601+0000 7f82cfa32640 1 --2- 172.21.15.143:0/863720907 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f82c813e290 0x7f82c80cf280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:18:30.607 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:30.601+0000 7f82cfa32640 1 -- 172.21.15.143:0/863720907 >> 172.21.15.143:0/863720907 conn(0x7f82c806a890 msgr2=0x7f82c806aca0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:18:30.607 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:30.601+0000 7f82cfa32640 1 -- 172.21.15.143:0/863720907 shutdown_connections 2023-11-26T07:18:30.607 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:30.601+0000 7f82cfa32640 1 -- 172.21.15.143:0/863720907 wait complete. 2023-11-26T07:18:30.608 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:30.601+0000 7f82cfa32640 1 Processor -- start 2023-11-26T07:18:30.608 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:30.601+0000 7f82cfa32640 1 -- start start 2023-11-26T07:18:30.610 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:30.605+0000 7f82cfa32640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f82c80cf7c0 0x7f82c8137ba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:18:30.610 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:30.605+0000 7f82cfa32640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f82c813e290 0x7f82c813fd00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:18:30.610 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:30.605+0000 7f82cfa32640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f82c815f4e0 0x7f82c8153b40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:18:30.610 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:30.605+0000 7f82cfa32640 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f82c815e570 con 0x7f82c813e290 2023-11-26T07:18:30.610 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:30.605+0000 7f82cfa32640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f82c8154a60 con 0x7f82c815f4e0 2023-11-26T07:18:30.610 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:30.605+0000 7f82cfa32640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f82c81380e0 con 0x7f82c80cf7c0 2023-11-26T07:18:30.610 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:30.605+0000 7f82ccfcd640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f82c813e290 0x7f82c813fd00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:18:30.610 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:30.605+0000 7f82ccfcd640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f82c813e290 0x7f82c813fd00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:33494/0 (socket says 172.21.15.143:33494) 2023-11-26T07:18:30.610 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:30.605+0000 7f82ccfcd640 1 -- 172.21.15.143:0/949820690 learned_addr learned my addr 172.21.15.143:0/949820690 (peer_addr_for_me v2:172.21.15.143:0/0) 2023-11-26T07:18:30.611 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:30.605+0000 7f82cdfcf640 1 --2- 172.21.15.143:0/949820690 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f82c815f4e0 0x7f82c8153b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:18:30.611 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:30.605+0000 7f82cd7ce640 1 --2- 172.21.15.143:0/949820690 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f82c80cf7c0 0x7f82c8137ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:18:30.611 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:30.605+0000 7f82ccfcd640 1 -- 172.21.15.143:0/949820690 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f82c80cf7c0 msgr2=0x7f82c8137ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:18:30.611 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:30.605+0000 7f82ccfcd640 1 --2- 172.21.15.143:0/949820690 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f82c80cf7c0 0x7f82c8137ba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:18:30.611 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:30.605+0000 7f82ccfcd640 1 -- 172.21.15.143:0/949820690 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f82c815f4e0 msgr2=0x7f82c8153b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:18:30.611 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:30.605+0000 7f82ccfcd640 1 --2- 172.21.15.143:0/949820690 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f82c815f4e0 0x7f82c8153b40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:18:30.611 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:30.605+0000 7f82ccfcd640 1 -- 172.21.15.143:0/949820690 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f82c0009b30 con 0x7f82c813e290 2023-11-26T07:18:30.611 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:30.605+0000 7f82cdfcf640 1 --2- 172.21.15.143:0/949820690 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f82c815f4e0 0x7f82c8153b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-26T07:18:30.611 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:30.605+0000 7f82cd7ce640 1 --2- 172.21.15.143:0/949820690 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f82c80cf7c0 0x7f82c8137ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-26T07:18:30.612 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:30.605+0000 7f82ccfcd640 1 --2- 172.21.15.143:0/949820690 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f82c813e290 0x7f82c813fd00 secure :-1 s=READY pgs=274 cs=0 l=1 rev1=1 crypto rx=0x7f82b800d800 tx=0x7f82b800dcc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:18:30.612 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:30.605+0000 7f82bdffb640 1 -- 172.21.15.143:0/949820690 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f82b80096a0 con 0x7f82c813e290 2023-11-26T07:18:30.612 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:30.605+0000 7f82bdffb640 1 -- 172.21.15.143:0/949820690 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f82b8009800 con 0x7f82c813e290 2023-11-26T07:18:30.612 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:30.605+0000 7f82bdffb640 1 -- 172.21.15.143:0/949820690 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f82b8009b70 con 0x7f82c813e290 2023-11-26T07:18:30.612 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:30.605+0000 7f82cfa32640 1 -- 172.21.15.143:0/949820690 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f82c815e290 con 0x7f82c813e290 2023-11-26T07:18:30.612 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:30.605+0000 7f82cfa32640 1 -- 172.21.15.143:0/949820690 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f82c813f500 con 0x7f82c813e290 2023-11-26T07:18:30.613 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:30.605+0000 7f82cfa32640 1 -- 172.21.15.143:0/949820690 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f82c813f080 con 0x7f82c813e290 2023-11-26T07:18:30.613 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:30.605+0000 7f82bdffb640 1 -- 172.21.15.143:0/949820690 <== mon.0 v2:172.21.15.143:3300/0 4 ==== mgrmap(e 5) v1 ==== 85852+0+0 (secure 0 0 0) 0x7f82b8009cd0 con 0x7f82c813e290 2023-11-26T07:18:30.613 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:30.609+0000 7f82bdffb640 1 --2- 172.21.15.143:0/949820690 >> [v2:172.21.15.143:6832/2150557544,v1:172.21.15.143:6833/2150557544] conn(0x7f82ac066470 0x7f82ac068930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:18:30.613 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:30.609+0000 7f82bdffb640 1 -- 172.21.15.143:0/949820690 <== mon.0 v2:172.21.15.143:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f82b8084550 con 0x7f82c813e290 2023-11-26T07:18:30.614 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:30.609+0000 7f82cd7ce640 1 --2- 172.21.15.143:0/949820690 >> [v2:172.21.15.143:6832/2150557544,v1:172.21.15.143:6833/2150557544] conn(0x7f82ac066470 0x7f82ac068930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:18:30.614 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:30.609+0000 7f82cd7ce640 1 --2- 172.21.15.143:0/949820690 >> [v2:172.21.15.143:6832/2150557544,v1:172.21.15.143:6833/2150557544] conn(0x7f82ac066470 0x7f82ac068930 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7f82c80cc100 tx=0x7f82c4008040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:18:30.617 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:30.613+0000 7f82bdffb640 1 -- 172.21.15.143:0/949820690 <== mon.0 v2:172.21.15.143:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172649 (secure 0 0 0) 0x7f82c813f080 con 0x7f82c813e290 2023-11-26T07:18:30.770 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:30.765+0000 7f82cfa32640 1 -- 172.21.15.143:0/949820690 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f82c813f080 con 0x7f82c813e290 2023-11-26T07:18:30.770 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:30.765+0000 7f82bdffb640 1 -- 172.21.15.143:0/949820690 <== mon.0 v2:172.21.15.143:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v20) v1 ==== 100+0+11 (secure 0 0 0) 0x7f82c813f080 con 0x7f82c813e290 2023-11-26T07:18:30.770 INFO:teuthology.orchestra.run.smithi143.stdout:pg_num: 64 2023-11-26T07:18:30.773 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:30.765+0000 7f82cfa32640 1 -- 172.21.15.143:0/949820690 >> [v2:172.21.15.143:6832/2150557544,v1:172.21.15.143:6833/2150557544] conn(0x7f82ac066470 msgr2=0x7f82ac068930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:18:30.773 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:30.765+0000 7f82cfa32640 1 --2- 172.21.15.143:0/949820690 >> [v2:172.21.15.143:6832/2150557544,v1:172.21.15.143:6833/2150557544] conn(0x7f82ac066470 0x7f82ac068930 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7f82c80cc100 tx=0x7f82c4008040 comp rx=0 tx=0).stop 2023-11-26T07:18:30.773 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:30.765+0000 7f82cfa32640 1 -- 172.21.15.143:0/949820690 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f82c813e290 msgr2=0x7f82c813fd00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:18:30.773 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:30.765+0000 7f82cfa32640 1 --2- 172.21.15.143:0/949820690 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f82c813e290 0x7f82c813fd00 secure :-1 s=READY pgs=274 cs=0 l=1 rev1=1 crypto rx=0x7f82b800d800 tx=0x7f82b800dcc0 comp rx=0 tx=0).stop 2023-11-26T07:18:30.773 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:30.765+0000 7f82cfa32640 1 -- 172.21.15.143:0/949820690 shutdown_connections 2023-11-26T07:18:30.773 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:30.765+0000 7f82cfa32640 1 --2- 172.21.15.143:0/949820690 >> [v2:172.21.15.143:6832/2150557544,v1:172.21.15.143:6833/2150557544] conn(0x7f82ac066470 0x7f82ac068930 unknown :-1 s=CLOSED pgs=161 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:18:30.773 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:30.765+0000 7f82cfa32640 1 --2- 172.21.15.143:0/949820690 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f82c815f4e0 0x7f82c8153b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:18:30.773 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:30.765+0000 7f82cfa32640 1 --2- 172.21.15.143:0/949820690 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f82c813e290 0x7f82c813fd00 unknown :-1 s=CLOSED pgs=274 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:18:30.773 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:30.765+0000 7f82cfa32640 1 --2- 172.21.15.143:0/949820690 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f82c80cf7c0 0x7f82c8137ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:18:30.773 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:30.765+0000 7f82cfa32640 1 -- 172.21.15.143:0/949820690 >> 172.21.15.143:0/949820690 conn(0x7f82c806a890 msgr2=0x7f82c813bb90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:18:30.773 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:30.765+0000 7f82cfa32640 1 -- 172.21.15.143:0/949820690 shutdown_connections 2023-11-26T07:18:30.774 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:30.765+0000 7f82cfa32640 1 -- 172.21.15.143:0/949820690 wait complete. 2023-11-26T07:18:30.793 DEBUG:teuthology.orchestra.run.smithi143:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2023-11-26T07:18:30.931 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:30.925+0000 7f032f5a1640 1 Processor -- start 2023-11-26T07:18:30.931 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:30.925+0000 7f032f5a1640 1 -- start start 2023-11-26T07:18:30.932 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:30.925+0000 7f032f5a1640 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7f032813e290 0x7f03280cf280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:18:30.932 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:30.925+0000 7f032f5a1640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f032815f6b0 0x7f03280cd7b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:18:30.932 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:30.925+0000 7f032f5a1640 1 -- --> v1:172.21.15.143:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f032806c520 con 0x7f03280cf7c0 2023-11-26T07:18:30.932 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:30.925+0000 7f032f5a1640 1 -- --> v2:172.21.15.188:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f03280cfc40 con 0x7f032813e290 2023-11-26T07:18:30.932 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:30.925+0000 7f032f5a1640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f032815fb40 con 0x7f032815f6b0 2023-11-26T07:18:30.933 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:30.925+0000 7f032cb3c640 1 --1- >> v1:172.21.15.143:6789/0 conn(0x7f03280cf7c0 0x7f032815efa0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.143:6789/0 says I am v1:172.21.15.143:47676/0 (socket says 172.21.15.143:47676) 2023-11-26T07:18:30.933 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:30.925+0000 7f032cb3c640 1 -- 172.21.15.143:0/731902815 learned_addr learned my addr 172.21.15.143:0/731902815 (peer_addr_for_me v1:172.21.15.143:0/0) 2023-11-26T07:18:30.933 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:30.925+0000 7f032d33d640 1 --2- 172.21.15.143:0/731902815 >> v2:172.21.15.188:3300/0 conn(0x7f032813e290 0x7f03280cf280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:18:30.934 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:30.925+0000 7f032db3e640 1 --2- 172.21.15.143:0/731902815 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f032815f6b0 0x7f03280cd7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:18:30.934 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:30.929+0000 7f031ffff640 1 -- 172.21.15.143:0/731902815 <== mon.0 v1:172.21.15.143:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1081066117 0 0) 0x7f032806c520 con 0x7f03280cf7c0 2023-11-26T07:18:30.934 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:30.929+0000 7f031ffff640 1 -- 172.21.15.143:0/731902815 --> v1:172.21.15.143:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f0314003530 con 0x7f03280cf7c0 2023-11-26T07:18:30.934 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:30.929+0000 7f032d33d640 1 -- 172.21.15.143:0/731902815 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f032815f6b0 msgr2=0x7f03280cd7b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-26T07:18:30.934 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:30.929+0000 7f032d33d640 1 --2- 172.21.15.143:0/731902815 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f032815f6b0 0x7f03280cd7b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:18:30.934 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:30.929+0000 7f032d33d640 1 -- 172.21.15.143:0/731902815 >> v1:172.21.15.143:6789/0 conn(0x7f03280cf7c0 legacy=0x7f032815efa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:18:30.934 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:30.929+0000 7f032d33d640 1 -- 172.21.15.143:0/731902815 --> v2:172.21.15.188:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f032806c800 con 0x7f032813e290 2023-11-26T07:18:30.936 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:30.929+0000 7f032d33d640 1 --2- 172.21.15.143:0/731902815 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f032813e290 0x7f03280cf280 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7f03240097c0 tx=0x7f032401c450 comp rx=0 tx=0).ready entity=mon.1 client_cookie=3a2f58360bd3e097 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:18:30.936 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:30.929+0000 7f031ffff640 1 -- 172.21.15.143:0/731902815 <== mon.1 v2:172.21.15.188:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f03240ab650 con 0x7f032813e290 2023-11-26T07:18:30.936 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:30.929+0000 7f031ffff640 1 -- 172.21.15.143:0/731902815 <== mon.1 v2:172.21.15.188:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f03240ab7b0 con 0x7f032813e290 2023-11-26T07:18:30.936 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:30.929+0000 7f031ffff640 1 -- 172.21.15.143:0/731902815 <== mon.1 v2:172.21.15.188:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f03240abb40 con 0x7f032813e290 2023-11-26T07:18:30.937 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:30.933+0000 7f032f5a1640 1 -- 172.21.15.143:0/731902815 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f032813e290 msgr2=0x7f03280cf280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:18:30.937 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:30.933+0000 7f032f5a1640 1 --2- 172.21.15.143:0/731902815 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f032813e290 0x7f03280cf280 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7f03240097c0 tx=0x7f032401c450 comp rx=0 tx=0).stop 2023-11-26T07:18:30.937 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:30.933+0000 7f032f5a1640 1 -- 172.21.15.143:0/731902815 shutdown_connections 2023-11-26T07:18:30.937 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:30.933+0000 7f032f5a1640 1 --2- 172.21.15.143:0/731902815 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f032815f6b0 0x7f03280cd7b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:18:30.938 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:30.933+0000 7f032f5a1640 1 --2- 172.21.15.143:0/731902815 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f032813e290 0x7f03280cf280 unknown :-1 s=CLOSED pgs=141 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:18:30.938 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:30.933+0000 7f032f5a1640 1 -- 172.21.15.143:0/731902815 >> 172.21.15.143:0/731902815 conn(0x7f032806a890 msgr2=0x7f032806aca0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:18:30.938 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:30.933+0000 7f032f5a1640 1 -- 172.21.15.143:0/731902815 shutdown_connections 2023-11-26T07:18:30.938 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:30.933+0000 7f032f5a1640 1 -- 172.21.15.143:0/731902815 wait complete. 2023-11-26T07:18:30.938 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:30.933+0000 7f032f5a1640 1 Processor -- start 2023-11-26T07:18:30.939 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:30.933+0000 7f032f5a1640 1 -- start start 2023-11-26T07:18:30.940 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:30.933+0000 7f032f5a1640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f03280cf7c0 0x7f032813a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:18:30.940 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:30.933+0000 7f032f5a1640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f032813e290 0x7f03280acdf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:18:30.940 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:30.933+0000 7f032f5a1640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f032815f6b0 0x7f03280ad330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:18:30.941 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:30.933+0000 7f032f5a1640 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f03280a84d0 con 0x7f03280cf7c0 2023-11-26T07:18:30.941 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:30.933+0000 7f032f5a1640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f03280ad940 con 0x7f032813e290 2023-11-26T07:18:30.941 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:30.933+0000 7f032f5a1640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f03280adc20 con 0x7f032815f6b0 2023-11-26T07:18:30.941 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:30.933+0000 7f032d33d640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f03280cf7c0 0x7f032813a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:18:30.941 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:30.933+0000 7f032d33d640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f03280cf7c0 0x7f032813a7a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:33506/0 (socket says 172.21.15.143:33506) 2023-11-26T07:18:30.941 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:30.933+0000 7f032d33d640 1 -- 172.21.15.143:0/2264069563 learned_addr learned my addr 172.21.15.143:0/2264069563 (peer_addr_for_me v2:172.21.15.143:0/0) 2023-11-26T07:18:30.941 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:30.933+0000 7f032d33d640 1 -- 172.21.15.143:0/2264069563 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f032815f6b0 msgr2=0x7f03280ad330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:18:30.941 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:30.933+0000 7f032d33d640 1 --2- 172.21.15.143:0/2264069563 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f032815f6b0 0x7f03280ad330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:18:30.941 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:30.933+0000 7f032d33d640 1 -- 172.21.15.143:0/2264069563 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f032813e290 msgr2=0x7f03280acdf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:18:30.941 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:30.933+0000 7f032d33d640 1 --2- 172.21.15.143:0/2264069563 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f032813e290 0x7f03280acdf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:18:30.941 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:30.933+0000 7f032d33d640 1 -- 172.21.15.143:0/2264069563 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0324009420 con 0x7f03280cf7c0 2023-11-26T07:18:30.941 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:30.933+0000 7f032d33d640 1 --2- 172.21.15.143:0/2264069563 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f03280cf7c0 0x7f032813a7a0 secure :-1 s=READY pgs=276 cs=0 l=1 rev1=1 crypto rx=0x7f03240ad040 tx=0x7f032401ac90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:18:30.942 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:30.937+0000 7f031dffb640 1 -- 172.21.15.143:0/2264069563 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f03240bf610 con 0x7f03280cf7c0 2023-11-26T07:18:30.942 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:30.937+0000 7f031dffb640 1 -- 172.21.15.143:0/2264069563 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f03240bf7b0 con 0x7f03280cf7c0 2023-11-26T07:18:30.942 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:30.937+0000 7f032f5a1640 1 -- 172.21.15.143:0/2264069563 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f03280adf00 con 0x7f03280cf7c0 2023-11-26T07:18:30.943 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:30.937+0000 7f031dffb640 1 -- 172.21.15.143:0/2264069563 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f03240bfb60 con 0x7f03280cf7c0 2023-11-26T07:18:30.943 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:30.937+0000 7f032f5a1640 1 -- 172.21.15.143:0/2264069563 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f03281f9ed0 con 0x7f03280cf7c0 2023-11-26T07:18:30.943 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:30.937+0000 7f032f5a1640 1 -- 172.21.15.143:0/2264069563 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f03280a8640 con 0x7f03280cf7c0 2023-11-26T07:18:30.943 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:30.937+0000 7f031dffb640 1 -- 172.21.15.143:0/2264069563 <== mon.0 v2:172.21.15.143:3300/0 4 ==== mgrmap(e 5) v1 ==== 85852+0+0 (secure 0 0 0) 0x7f03240b5400 con 0x7f03280cf7c0 2023-11-26T07:18:30.944 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:30.937+0000 7f031dffb640 1 --2- 172.21.15.143:0/2264069563 >> [v2:172.21.15.143:6832/2150557544,v1:172.21.15.143:6833/2150557544] conn(0x7f030c066470 0x7f030c068930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:18:30.944 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:30.937+0000 7f031dffb640 1 -- 172.21.15.143:0/2264069563 <== mon.0 v2:172.21.15.143:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f032411ed10 con 0x7f03280cf7c0 2023-11-26T07:18:30.944 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:30.937+0000 7f032cb3c640 1 --2- 172.21.15.143:0/2264069563 >> [v2:172.21.15.143:6832/2150557544,v1:172.21.15.143:6833/2150557544] conn(0x7f030c066470 0x7f030c068930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:18:30.944 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:30.937+0000 7f032cb3c640 1 --2- 172.21.15.143:0/2264069563 >> [v2:172.21.15.143:6832/2150557544,v1:172.21.15.143:6833/2150557544] conn(0x7f030c066470 0x7f030c068930 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7f0318007fd0 tx=0x7f0318007940 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:18:30.947 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:30.941+0000 7f031dffb640 1 -- 172.21.15.143:0/2264069563 <== mon.0 v2:172.21.15.143:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172649 (secure 0 0 0) 0x7f03240ec1a0 con 0x7f03280cf7c0 2023-11-26T07:18:31.109 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:31.105+0000 7f032f5a1640 1 -- 172.21.15.143:0/2264069563 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) v1 -- 0x7f0328067680 con 0x7f03280cf7c0 2023-11-26T07:18:31.110 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:31.105+0000 7f031dffb640 1 -- 172.21.15.143:0/2264069563 <== mon.0 v2:172.21.15.143:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v20) v1 ==== 80+0+0 (secure 0 0 0) 0x7f03240ef7c0 con 0x7f03280cf7c0 2023-11-26T07:18:31.110 INFO:teuthology.orchestra.run.smithi143.stderr:listed 0 entries 2023-11-26T07:18:31.112 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:31.105+0000 7f032f5a1640 1 -- 172.21.15.143:0/2264069563 >> [v2:172.21.15.143:6832/2150557544,v1:172.21.15.143:6833/2150557544] conn(0x7f030c066470 msgr2=0x7f030c068930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:18:31.112 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:31.105+0000 7f032f5a1640 1 --2- 172.21.15.143:0/2264069563 >> [v2:172.21.15.143:6832/2150557544,v1:172.21.15.143:6833/2150557544] conn(0x7f030c066470 0x7f030c068930 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7f0318007fd0 tx=0x7f0318007940 comp rx=0 tx=0).stop 2023-11-26T07:18:31.112 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:31.105+0000 7f032f5a1640 1 -- 172.21.15.143:0/2264069563 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f03280cf7c0 msgr2=0x7f032813a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:18:31.112 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:31.105+0000 7f032f5a1640 1 --2- 172.21.15.143:0/2264069563 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f03280cf7c0 0x7f032813a7a0 secure :-1 s=READY pgs=276 cs=0 l=1 rev1=1 crypto rx=0x7f03240ad040 tx=0x7f032401ac90 comp rx=0 tx=0).stop 2023-11-26T07:18:31.112 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:31.105+0000 7f032f5a1640 1 -- 172.21.15.143:0/2264069563 shutdown_connections 2023-11-26T07:18:31.113 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:31.105+0000 7f032f5a1640 1 --2- 172.21.15.143:0/2264069563 >> [v2:172.21.15.143:6832/2150557544,v1:172.21.15.143:6833/2150557544] conn(0x7f030c066470 0x7f030c068930 unknown :-1 s=CLOSED pgs=162 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:18:31.113 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:31.105+0000 7f032f5a1640 1 --2- 172.21.15.143:0/2264069563 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f032815f6b0 0x7f03280ad330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:18:31.113 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:31.105+0000 7f032f5a1640 1 --2- 172.21.15.143:0/2264069563 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f032813e290 0x7f03280acdf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:18:31.113 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:31.105+0000 7f032f5a1640 1 --2- 172.21.15.143:0/2264069563 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f03280cf7c0 0x7f032813a7a0 unknown :-1 s=CLOSED pgs=276 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:18:31.113 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:31.105+0000 7f032f5a1640 1 -- 172.21.15.143:0/2264069563 >> 172.21.15.143:0/2264069563 conn(0x7f032806a890 msgr2=0x7f0328070da0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:18:31.113 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:31.105+0000 7f032f5a1640 1 -- 172.21.15.143:0/2264069563 shutdown_connections 2023-11-26T07:18:31.113 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:31.105+0000 7f032f5a1640 1 -- 172.21.15.143:0/2264069563 wait complete. 2023-11-26T07:18:31.130 INFO:tasks.cephfs.fuse_mount:Running fusermount -u on ubuntu@smithi188.front.sepia.ceph.com... 2023-11-26T07:18:31.131 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-26T07:18:31.131 DEBUG:teuthology.orchestra.run.smithi188:> sudo fusermount -u /home/ubuntu/cephtest/mnt.1 2023-11-26T07:18:31.174 INFO:teuthology.orchestra.run:waiting for 300 2023-11-26T07:18:31.174 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi188.front.sepia.ceph.com 2023-11-26T07:18:31.174 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-26T07:18:31.174 DEBUG:teuthology.orchestra.run.smithi188:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.1) 2023-11-26T07:18:31.242 DEBUG:teuthology.orchestra.run.smithi143:> set -ex 2023-11-26T07:18:31.242 DEBUG:teuthology.orchestra.run.smithi143:> dd if=/proc/self/mounts of=/dev/stdout 2023-11-26T07:18:31.251 DEBUG:teuthology.orchestra.run.smithi143:> set -ex 2023-11-26T07:18:31.251 DEBUG:teuthology.orchestra.run.smithi143:> dd if=/proc/self/mounts of=/dev/stdout 2023-11-26T07:18:31.304 DEBUG:teuthology.orchestra.run.smithi143:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-11-26T07:18:31.485 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:31.481+0000 7f0267e32640 1 Processor -- start 2023-11-26T07:18:31.485 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:31.481+0000 7f0267e32640 1 -- start start 2023-11-26T07:18:31.487 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:31.481+0000 7f0267e32640 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7f026013e290 0x7f02600cf280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:18:31.487 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:31.481+0000 7f0267e32640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f02600cf7c0 0x7f026015efa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:18:31.487 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:31.481+0000 7f0267e32640 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7f026015f4e0 0x7f02601408e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:18:31.488 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:31.481+0000 7f0267e32640 1 -- --> v2:172.21.15.143:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f02600cfc40 con 0x7f026015f4e0 2023-11-26T07:18:31.488 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:31.481+0000 7f0267e32640 1 -- --> v2:172.21.15.188:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f026006c520 con 0x7f026013e290 2023-11-26T07:18:31.488 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:31.481+0000 7f0267e32640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f026006c800 con 0x7f02600cf7c0 2023-11-26T07:18:31.488 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:31.481+0000 7f02663cf640 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7f026015f4e0 0x7f02601408e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:18:31.488 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:31.481+0000 7f02663cf640 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7f026015f4e0 0x7f02601408e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:33522/0 (socket says 172.21.15.143:33522) 2023-11-26T07:18:31.488 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:31.481+0000 7f02663cf640 1 -- 172.21.15.143:0/4147674929 learned_addr learned my addr 172.21.15.143:0/4147674929 (peer_addr_for_me v2:172.21.15.143:0/0) 2023-11-26T07:18:31.488 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:31.481+0000 7f0265bce640 1 --2- 172.21.15.143:0/4147674929 >> v2:172.21.15.188:3300/0 conn(0x7f026013e290 0x7f02600cf280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:18:31.489 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:31.481+0000 7f02653cd640 1 --2- 172.21.15.143:0/4147674929 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f02600cf7c0 0x7f026015efa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:18:31.489 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:31.481+0000 7f02663cf640 1 -- 172.21.15.143:0/4147674929 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f02600cf7c0 msgr2=0x7f026015efa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-26T07:18:31.489 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:31.481+0000 7f02663cf640 1 --2- 172.21.15.143:0/4147674929 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f02600cf7c0 0x7f026015efa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:18:31.489 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:31.481+0000 7f02663cf640 1 -- 172.21.15.143:0/4147674929 >> v2:172.21.15.188:3300/0 conn(0x7f026013e290 msgr2=0x7f02600cf280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-26T07:18:31.489 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:31.481+0000 7f02663cf640 1 --2- 172.21.15.143:0/4147674929 >> v2:172.21.15.188:3300/0 conn(0x7f026013e290 0x7f02600cf280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:18:31.489 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:31.481+0000 7f02663cf640 1 -- 172.21.15.143:0/4147674929 --> v2:172.21.15.143:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f026006cae0 con 0x7f026015f4e0 2023-11-26T07:18:31.489 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:31.481+0000 7f02653cd640 1 --2- 172.21.15.143:0/4147674929 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f02600cf7c0 0x7f026015efa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-26T07:18:31.489 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:31.481+0000 7f0265bce640 1 --2- 172.21.15.143:0/4147674929 >> v2:172.21.15.188:3300/0 conn(0x7f026013e290 0x7f02600cf280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-26T07:18:31.490 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:31.485+0000 7f02663cf640 1 --2- 172.21.15.143:0/4147674929 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f026015f4e0 0x7f02601408e0 secure :-1 s=READY pgs=277 cs=0 l=1 rev1=1 crypto rx=0x7f02600725a0 tx=0x7f025801c6e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c68445a111e0dfa8 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:18:31.490 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:31.485+0000 7f0264bcc640 1 -- 172.21.15.143:0/4147674929 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f02580a5560 con 0x7f026015f4e0 2023-11-26T07:18:31.490 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:31.485+0000 7f0264bcc640 1 -- 172.21.15.143:0/4147674929 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f02580a56c0 con 0x7f026015f4e0 2023-11-26T07:18:31.490 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:31.485+0000 7f0264bcc640 1 -- 172.21.15.143:0/4147674929 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f02580a5a50 con 0x7f026015f4e0 2023-11-26T07:18:31.491 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:31.485+0000 7f0267e32640 1 -- 172.21.15.143:0/4147674929 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f026015f4e0 msgr2=0x7f02601408e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:18:31.492 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:31.485+0000 7f0267e32640 1 --2- 172.21.15.143:0/4147674929 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f026015f4e0 0x7f02601408e0 secure :-1 s=READY pgs=277 cs=0 l=1 rev1=1 crypto rx=0x7f02600725a0 tx=0x7f025801c6e0 comp rx=0 tx=0).stop 2023-11-26T07:18:31.492 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:31.485+0000 7f0267e32640 1 -- 172.21.15.143:0/4147674929 shutdown_connections 2023-11-26T07:18:31.492 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:31.485+0000 7f0267e32640 1 --2- 172.21.15.143:0/4147674929 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f026015f4e0 0x7f02601408e0 unknown :-1 s=CLOSED pgs=277 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:18:31.492 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:31.485+0000 7f0267e32640 1 --2- 172.21.15.143:0/4147674929 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f02600cf7c0 0x7f026015efa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:18:31.492 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:31.485+0000 7f0267e32640 1 --2- 172.21.15.143:0/4147674929 >> v2:172.21.15.188:3300/0 conn(0x7f026013e290 0x7f02600cf280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:18:31.492 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:31.485+0000 7f0267e32640 1 -- 172.21.15.143:0/4147674929 >> 172.21.15.143:0/4147674929 conn(0x7f026006a890 msgr2=0x7f026006aca0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:18:31.492 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:31.485+0000 7f0267e32640 1 -- 172.21.15.143:0/4147674929 shutdown_connections 2023-11-26T07:18:31.492 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:31.485+0000 7f0267e32640 1 -- 172.21.15.143:0/4147674929 wait complete. 2023-11-26T07:18:31.493 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:31.489+0000 7f0267e32640 1 Processor -- start 2023-11-26T07:18:31.493 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:31.489+0000 7f0267e32640 1 -- start start 2023-11-26T07:18:31.495 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:31.489+0000 7f0267e32640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f02600cf7c0 0x7f0260154880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:18:31.495 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:31.489+0000 7f0267e32640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f026013e290 0x7f02601418f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:18:31.495 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:31.489+0000 7f0267e32640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f026015f4e0 0x7f02600b2d00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:18:31.495 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:31.489+0000 7f0267e32640 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f02601427b0 con 0x7f026013e290 2023-11-26T07:18:31.495 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:31.489+0000 7f0267e32640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f02601421f0 con 0x7f02600cf7c0 2023-11-26T07:18:31.496 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:31.489+0000 7f0267e32640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f02601424d0 con 0x7f026015f4e0 2023-11-26T07:18:31.496 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:31.489+0000 7f02653cd640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f026013e290 0x7f02601418f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:18:31.496 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:31.489+0000 7f02653cd640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f026013e290 0x7f02601418f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:33524/0 (socket says 172.21.15.143:33524) 2023-11-26T07:18:31.496 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:31.489+0000 7f02653cd640 1 -- 172.21.15.143:0/2325229417 learned_addr learned my addr 172.21.15.143:0/2325229417 (peer_addr_for_me v2:172.21.15.143:0/0) 2023-11-26T07:18:31.496 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:31.489+0000 7f02663cf640 1 --2- 172.21.15.143:0/2325229417 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f026015f4e0 0x7f02600b2d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:18:31.496 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:31.489+0000 7f0265bce640 1 --2- 172.21.15.143:0/2325229417 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f02600cf7c0 0x7f0260154880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:18:31.496 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:31.489+0000 7f02653cd640 1 -- 172.21.15.143:0/2325229417 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f026015f4e0 msgr2=0x7f02600b2d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:18:31.496 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:31.489+0000 7f02653cd640 1 --2- 172.21.15.143:0/2325229417 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f026015f4e0 0x7f02600b2d00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:18:31.496 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:31.489+0000 7f02653cd640 1 -- 172.21.15.143:0/2325229417 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f02600cf7c0 msgr2=0x7f0260154880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:18:31.497 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:31.489+0000 7f02653cd640 1 --2- 172.21.15.143:0/2325229417 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f02600cf7c0 0x7f0260154880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:18:31.497 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:31.489+0000 7f02653cd640 1 -- 172.21.15.143:0/2325229417 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0258009b30 con 0x7f026013e290 2023-11-26T07:18:31.497 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:31.489+0000 7f02653cd640 1 --2- 172.21.15.143:0/2325229417 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f026013e290 0x7f02601418f0 secure :-1 s=READY pgs=278 cs=0 l=1 rev1=1 crypto rx=0x7f025000c7c0 tx=0x7f025000cc80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:18:31.497 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:31.489+0000 7f02567fc640 1 -- 172.21.15.143:0/2325229417 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0250007a30 con 0x7f026013e290 2023-11-26T07:18:31.497 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:31.489+0000 7f02567fc640 1 -- 172.21.15.143:0/2325229417 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0250007b90 con 0x7f026013e290 2023-11-26T07:18:31.497 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:31.489+0000 7f02567fc640 1 -- 172.21.15.143:0/2325229417 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f025000ce70 con 0x7f026013e290 2023-11-26T07:18:31.497 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:31.489+0000 7f0267e32640 1 -- 172.21.15.143:0/2325229417 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f02600afd80 con 0x7f026013e290 2023-11-26T07:18:31.497 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:31.489+0000 7f0267e32640 1 -- 172.21.15.143:0/2325229417 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0260154dc0 con 0x7f026013e290 2023-11-26T07:18:31.498 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:31.489+0000 7f0267e32640 1 -- 172.21.15.143:0/2325229417 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0260067680 con 0x7f026013e290 2023-11-26T07:18:31.498 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:31.493+0000 7f02567fc640 1 -- 172.21.15.143:0/2325229417 <== mon.0 v2:172.21.15.143:3300/0 4 ==== mgrmap(e 5) v1 ==== 85852+0+0 (secure 0 0 0) 0x7f02500113f0 con 0x7f026013e290 2023-11-26T07:18:31.498 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:31.493+0000 7f02567fc640 1 --2- 172.21.15.143:0/2325229417 >> [v2:172.21.15.143:6832/2150557544,v1:172.21.15.143:6833/2150557544] conn(0x7f02440664a0 0x7f0244068960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:18:31.499 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:31.493+0000 7f02567fc640 1 -- 172.21.15.143:0/2325229417 <== mon.0 v2:172.21.15.143:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f0250083e60 con 0x7f026013e290 2023-11-26T07:18:31.499 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:31.493+0000 7f0265bce640 1 --2- 172.21.15.143:0/2325229417 >> [v2:172.21.15.143:6832/2150557544,v1:172.21.15.143:6833/2150557544] conn(0x7f02440664a0 0x7f0244068960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:18:31.499 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:31.493+0000 7f0265bce640 1 --2- 172.21.15.143:0/2325229417 >> [v2:172.21.15.143:6832/2150557544,v1:172.21.15.143:6833/2150557544] conn(0x7f02440664a0 0x7f0244068960 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7f025c00ab60 tx=0x7f025c008040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:18:31.502 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:31.497+0000 7f02567fc640 1 -- 172.21.15.143:0/2325229417 <== mon.0 v2:172.21.15.143:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172649 (secure 0 0 0) 0x7f0250051390 con 0x7f026013e290 2023-11-26T07:18:31.653 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:31.645+0000 7f0267e32640 1 -- 172.21.15.143:0/2325229417 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f02600b4340 con 0x7f026013e290 2023-11-26T07:18:31.654 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:31.649+0000 7f02567fc640 1 -- 172.21.15.143:0/2325229417 <== mon.0 v2:172.21.15.143:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v20) v1 ==== 74+0+16541 (secure 0 0 0) 0x7f02500549b0 con 0x7f026013e290 2023-11-26T07:18:31.654 INFO:teuthology.orchestra.run.smithi143.stdout: 2023-11-26T07:18:31.655 INFO:teuthology.orchestra.run.smithi143.stdout:{"epoch":20,"fsid":"f5033060-2469-4a07-95fe-2b76f5404bc1","created":"2023-11-26T07:17:28.613663+0000","modified":"2023-11-26T07:17:55.505435+0000","last_up_change":"2023-11-26T07:17:41.157813+0000","last_in_change":"2023-11-26T07:17:31.181658+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-11-26T07:17:41.758597+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2023-11-26T07:17:51.878332+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2023-11-26T07:17:52.871396+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"ebbee2c6-023e-44b9-9f1d-6f7bfd309730","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6817","nonce":639807369},{"type":"v1","addr":"172.21.15.143:6819","nonce":639807369}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6821","nonce":639807369},{"type":"v1","addr":"172.21.15.143:6823","nonce":639807369}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6830","nonce":639807369},{"type":"v1","addr":"172.21.15.143:6831","nonce":639807369}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6825","nonce":639807369},{"type":"v1","addr":"172.21.15.143:6828","nonce":639807369}]},"public_addr":"172.21.15.143:6819/639807369","cluster_addr":"172.21.15.143:6823/639807369","heartbeat_back_addr":"172.21.15.143:6831/639807369","heartbeat_front_addr":"172.21.15.143:6828/639807369","state":["exists","up"]},{"osd":1,"uuid":"fec4b8be-7122-49e9-9a19-5850a0700001","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6816","nonce":1144942530},{"type":"v1","addr":"172.21.15.143:6818","nonce":1144942530}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6820","nonce":1144942530},{"type":"v1","addr":"172.21.15.143:6822","nonce":1144942530}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6827","nonce":1144942530},{"type":"v1","addr":"172.21.15.143:6829","nonce":1144942530}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6824","nonce":1144942530},{"type":"v1","addr":"172.21.15.143:6826","nonce":1144942530}]},"public_addr":"172.21.15.143:6818/1144942530","cluster_addr":"172.21.15.143:6822/1144942530","heartbeat_back_addr":"172.21.15.143:6829/1144942530","heartbeat_front_addr":"172.21.15.143:6826/1144942530","state":["exists","up"]},{"osd":2,"uuid":"2888af5b-f933-46ca-8f95-6dd494ff43f6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6808","nonce":3909095186},{"type":"v1","addr":"172.21.15.143:6809","nonce":3909095186}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6810","nonce":3909095186},{"type":"v1","addr":"172.21.15.143:6811","nonce":3909095186}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6814","nonce":3909095186},{"type":"v1","addr":"172.21.15.143:6815","nonce":3909095186}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6812","nonce":3909095186},{"type":"v1","addr":"172.21.15.143:6813","nonce":3909095186}]},"public_addr":"172.21.15.143:6809/3909095186","cluster_addr":"172.21.15.143:6811/3909095186","heartbeat_back_addr":"172.21.15.143:6815/3909095186","heartbeat_front_addr":"172.21.15.143:6813/3909095186","state":["exists","up"]},{"osd":3,"uuid":"d1f52435-7bc1-4081-9550-9cd1ba55f549","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6800","nonce":3854210991},{"type":"v1","addr":"172.21.15.143:6801","nonce":3854210991}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6802","nonce":3854210991},{"type":"v1","addr":"172.21.15.143:6803","nonce":3854210991}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6806","nonce":3854210991},{"type":"v1","addr":"172.21.15.143:6807","nonce":3854210991}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6804","nonce":3854210991},{"type":"v1","addr":"172.21.15.143:6805","nonce":3854210991}]},"public_addr":"172.21.15.143:6801/3854210991","cluster_addr":"172.21.15.143:6803/3854210991","heartbeat_back_addr":"172.21.15.143:6807/3854210991","heartbeat_front_addr":"172.21.15.143:6805/3854210991","state":["exists","up"]},{"osd":4,"uuid":"e2ae8b6e-6e25-49b2-bcbf-aacce68f06a3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6800","nonce":3924606902},{"type":"v1","addr":"172.21.15.188:6801","nonce":3924606902}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6802","nonce":3924606902},{"type":"v1","addr":"172.21.15.188:6803","nonce":3924606902}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6806","nonce":3924606902},{"type":"v1","addr":"172.21.15.188:6807","nonce":3924606902}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6804","nonce":3924606902},{"type":"v1","addr":"172.21.15.188:6805","nonce":3924606902}]},"public_addr":"172.21.15.188:6801/3924606902","cluster_addr":"172.21.15.188:6803/3924606902","heartbeat_back_addr":"172.21.15.188:6807/3924606902","heartbeat_front_addr":"172.21.15.188:6805/3924606902","state":["exists","up"]},{"osd":5,"uuid":"3cec8a32-7698-4c36-8e89-e4ba11934f33","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6813","nonce":186841544},{"type":"v1","addr":"172.21.15.188:6815","nonce":186841544}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6817","nonce":186841544},{"type":"v1","addr":"172.21.15.188:6818","nonce":186841544}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6822","nonce":186841544},{"type":"v1","addr":"172.21.15.188:6823","nonce":186841544}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6820","nonce":186841544},{"type":"v1","addr":"172.21.15.188:6821","nonce":186841544}]},"public_addr":"172.21.15.188:6815/186841544","cluster_addr":"172.21.15.188:6818/186841544","heartbeat_back_addr":"172.21.15.188:6823/186841544","heartbeat_front_addr":"172.21.15.188:6821/186841544","state":["exists","up"]},{"osd":6,"uuid":"58a65d5d-e090-473a-81b4-ad38e85029f9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6808","nonce":2153499405},{"type":"v1","addr":"172.21.15.188:6809","nonce":2153499405}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6810","nonce":2153499405},{"type":"v1","addr":"172.21.15.188:6811","nonce":2153499405}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6816","nonce":2153499405},{"type":"v1","addr":"172.21.15.188:6819","nonce":2153499405}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6812","nonce":2153499405},{"type":"v1","addr":"172.21.15.188:6814","nonce":2153499405}]},"public_addr":"172.21.15.188:6809/2153499405","cluster_addr":"172.21.15.188:6811/2153499405","heartbeat_back_addr":"172.21.15.188:6819/2153499405","heartbeat_front_addr":"172.21.15.188:6814/2153499405","state":["exists","up"]},{"osd":7,"uuid":"ea5b7f70-3554-4cf8-ac30-f7c59b80678d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6824","nonce":2722096729},{"type":"v1","addr":"172.21.15.188:6825","nonce":2722096729}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6826","nonce":2722096729},{"type":"v1","addr":"172.21.15.188:6827","nonce":2722096729}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6830","nonce":2722096729},{"type":"v1","addr":"172.21.15.188:6831","nonce":2722096729}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6828","nonce":2722096729},{"type":"v1","addr":"172.21.15.188:6829","nonce":2722096729}]},"public_addr":"172.21.15.188:6825/2722096729","cluster_addr":"172.21.15.188:6827/2722096729","heartbeat_back_addr":"172.21.15.188:6831/2722096729","heartbeat_front_addr":"172.21.15.188:6829/2722096729","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-26T07:17:36.995032+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-26T07:17:36.932266+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-26T07:17:36.829304+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-26T07:17:36.804297+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-26T07:17:36.667892+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-26T07:17:36.964868+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-26T07:17:36.827332+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-26T07:17:36.893179+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-11-26T07:18:31.655 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:31.649+0000 7f0267e32640 1 -- 172.21.15.143:0/2325229417 >> [v2:172.21.15.143:6832/2150557544,v1:172.21.15.143:6833/2150557544] conn(0x7f02440664a0 msgr2=0x7f0244068960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:18:31.656 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:31.649+0000 7f0267e32640 1 --2- 172.21.15.143:0/2325229417 >> [v2:172.21.15.143:6832/2150557544,v1:172.21.15.143:6833/2150557544] conn(0x7f02440664a0 0x7f0244068960 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7f025c00ab60 tx=0x7f025c008040 comp rx=0 tx=0).stop 2023-11-26T07:18:31.656 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:31.649+0000 7f0267e32640 1 -- 172.21.15.143:0/2325229417 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f026013e290 msgr2=0x7f02601418f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:18:31.656 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:31.649+0000 7f0267e32640 1 --2- 172.21.15.143:0/2325229417 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f026013e290 0x7f02601418f0 secure :-1 s=READY pgs=278 cs=0 l=1 rev1=1 crypto rx=0x7f025000c7c0 tx=0x7f025000cc80 comp rx=0 tx=0).stop 2023-11-26T07:18:31.656 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:31.649+0000 7f0267e32640 1 -- 172.21.15.143:0/2325229417 shutdown_connections 2023-11-26T07:18:31.656 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:31.649+0000 7f0267e32640 1 --2- 172.21.15.143:0/2325229417 >> [v2:172.21.15.143:6832/2150557544,v1:172.21.15.143:6833/2150557544] conn(0x7f02440664a0 0x7f0244068960 unknown :-1 s=CLOSED pgs=163 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:18:31.656 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:31.649+0000 7f0267e32640 1 --2- 172.21.15.143:0/2325229417 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f026015f4e0 0x7f02600b2d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:18:31.656 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:31.649+0000 7f0267e32640 1 --2- 172.21.15.143:0/2325229417 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f026013e290 0x7f02601418f0 unknown :-1 s=CLOSED pgs=278 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:18:31.656 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:31.649+0000 7f0267e32640 1 --2- 172.21.15.143:0/2325229417 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f02600cf7c0 0x7f0260154880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:18:31.656 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:31.649+0000 7f0267e32640 1 -- 172.21.15.143:0/2325229417 >> 172.21.15.143:0/2325229417 conn(0x7f026006a890 msgr2=0x7f026013c9d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:18:31.656 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:31.649+0000 7f0267e32640 1 -- 172.21.15.143:0/2325229417 shutdown_connections 2023-11-26T07:18:31.657 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:31.649+0000 7f0267e32640 1 -- 172.21.15.143:0/2325229417 wait complete. 2023-11-26T07:18:31.674 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-11-26T07:17:41.758597+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2023-11-26T07:17:51.878332+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2023-11-26T07:17:52.871396+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '20', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2023-11-26T07:18:31.675 DEBUG:teuthology.orchestra.run.smithi143:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-11-26T07:18:31.819 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:31.813+0000 7f6f1e11d640 1 Processor -- start 2023-11-26T07:18:31.819 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:31.813+0000 7f6f1e11d640 1 -- start start 2023-11-26T07:18:31.821 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:31.813+0000 7f6f1e11d640 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7f6f18143ac0 0x7f6f18070370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:18:31.821 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:31.813+0000 7f6f1e11d640 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7f6f180cf990 0x7f6f180cd1b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:18:31.821 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:31.813+0000 7f6f1e11d640 1 -- --> v1:172.21.15.143:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6f180cc730 con 0x7f6f180708b0 2023-11-26T07:18:31.821 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:31.813+0000 7f6f1e11d640 1 -- --> v2:172.21.15.143:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6f18143ea0 con 0x7f6f18143ac0 2023-11-26T07:18:31.821 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:31.813+0000 7f6f1e11d640 1 -- --> v2:172.21.15.188:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6f18070ee0 con 0x7f6f180cf990 2023-11-26T07:18:31.821 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:31.813+0000 7f6f177fe640 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7f6f18143ac0 0x7f6f18070370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:18:31.822 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:31.813+0000 7f6f0ffff640 1 --1- >> v1:172.21.15.143:6789/0 conn(0x7f6f180708b0 0x7f6f180cf280 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.143:6789/0 says I am v1:172.21.15.143:47688/0 (socket says 172.21.15.143:47688) 2023-11-26T07:18:31.822 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:31.813+0000 7f6f0ffff640 1 -- 172.21.15.143:0/868634917 learned_addr learned my addr 172.21.15.143:0/868634917 (peer_addr_for_me v1:172.21.15.143:0/0) 2023-11-26T07:18:31.822 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:31.813+0000 7f6f177fe640 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7f6f18143ac0 0x7f6f18070370 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:33532/0 (socket says 172.21.15.143:33532) 2023-11-26T07:18:31.822 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:31.817+0000 7f6f17fff640 1 --2- 172.21.15.143:0/868634917 >> v2:172.21.15.188:3300/0 conn(0x7f6f180cf990 0x7f6f180cd1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:18:31.822 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:31.817+0000 7f6f177fe640 1 -- 172.21.15.143:0/868634917 >> v2:172.21.15.188:3300/0 conn(0x7f6f180cf990 msgr2=0x7f6f180cd1b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-26T07:18:31.822 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:31.817+0000 7f6f177fe640 1 --2- 172.21.15.143:0/868634917 >> v2:172.21.15.188:3300/0 conn(0x7f6f180cf990 0x7f6f180cd1b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:18:31.822 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:31.817+0000 7f6f177fe640 1 -- 172.21.15.143:0/868634917 >> v1:172.21.15.143:6789/0 conn(0x7f6f180708b0 legacy=0x7f6f180cf280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:18:31.822 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:31.817+0000 7f6f177fe640 1 -- 172.21.15.143:0/868634917 --> v2:172.21.15.143:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6f180711c0 con 0x7f6f18143ac0 2023-11-26T07:18:31.822 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:31.817+0000 7f6f16ffd640 1 -- 172.21.15.143:0/868634917 <== mon.0 v1:172.21.15.143:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 277164000 0 0) 0x7f6f180cc730 con 0x7f6f180708b0 2023-11-26T07:18:31.824 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:31.817+0000 7f6f177fe640 1 --2- 172.21.15.143:0/868634917 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f6f18143ac0 0x7f6f18070370 secure :-1 s=READY pgs=280 cs=0 l=1 rev1=1 crypto rx=0x7f6f1806e570 tx=0x7f6f0801c460 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3b545a1c507f61e5 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:18:31.824 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:31.817+0000 7f6f16ffd640 1 -- 172.21.15.143:0/868634917 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6f080ab600 con 0x7f6f18143ac0 2023-11-26T07:18:31.824 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:31.817+0000 7f6f16ffd640 1 -- 172.21.15.143:0/868634917 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6f080a7050 con 0x7f6f18143ac0 2023-11-26T07:18:31.824 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:31.817+0000 7f6f16ffd640 1 -- 172.21.15.143:0/868634917 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6f080ab980 con 0x7f6f18143ac0 2023-11-26T07:18:31.825 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:31.817+0000 7f6f1e11d640 1 -- 172.21.15.143:0/868634917 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f6f18143ac0 msgr2=0x7f6f18070370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:18:31.825 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:31.821+0000 7f6f1e11d640 1 --2- 172.21.15.143:0/868634917 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f6f18143ac0 0x7f6f18070370 secure :-1 s=READY pgs=280 cs=0 l=1 rev1=1 crypto rx=0x7f6f1806e570 tx=0x7f6f0801c460 comp rx=0 tx=0).stop 2023-11-26T07:18:31.826 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:31.821+0000 7f6f1e11d640 1 -- 172.21.15.143:0/868634917 shutdown_connections 2023-11-26T07:18:31.826 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:31.821+0000 7f6f1e11d640 1 --2- 172.21.15.143:0/868634917 >> v2:172.21.15.188:3300/0 conn(0x7f6f180cf990 0x7f6f180cd1b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:18:31.826 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:31.821+0000 7f6f1e11d640 1 --2- 172.21.15.143:0/868634917 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f6f18143ac0 0x7f6f18070370 unknown :-1 s=CLOSED pgs=280 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:18:31.826 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:31.821+0000 7f6f1e11d640 1 -- 172.21.15.143:0/868634917 >> 172.21.15.143:0/868634917 conn(0x7f6f1806a890 msgr2=0x7f6f1806aca0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:18:31.826 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:31.821+0000 7f6f1e11d640 1 -- 172.21.15.143:0/868634917 shutdown_connections 2023-11-26T07:18:31.826 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:31.821+0000 7f6f1e11d640 1 -- 172.21.15.143:0/868634917 wait complete. 2023-11-26T07:18:31.827 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:31.821+0000 7f6f1e11d640 1 Processor -- start 2023-11-26T07:18:31.827 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:31.821+0000 7f6f1e11d640 1 -- start start 2023-11-26T07:18:31.828 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:31.821+0000 7f6f1e11d640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f6f180708b0 0x7f6f180a91c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:18:31.828 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:31.821+0000 7f6f1e11d640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f6f180cf990 0x7f6f180b04a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:18:31.828 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:31.821+0000 7f6f1e11d640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f6f18143ac0 0x7f6f181f6b10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:18:31.828 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:31.821+0000 7f6f1e11d640 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6f180a9700 con 0x7f6f180cf990 2023-11-26T07:18:31.829 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:31.821+0000 7f6f1e11d640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6f180b09e0 con 0x7f6f18143ac0 2023-11-26T07:18:31.829 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:31.821+0000 7f6f1e11d640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6f180b0c80 con 0x7f6f180708b0 2023-11-26T07:18:31.829 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:31.821+0000 7f6f0ffff640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f6f180cf990 0x7f6f180b04a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:18:31.829 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:31.821+0000 7f6f0ffff640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f6f180cf990 0x7f6f180b04a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:33548/0 (socket says 172.21.15.143:33548) 2023-11-26T07:18:31.829 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:31.821+0000 7f6f0ffff640 1 -- 172.21.15.143:0/1269372405 learned_addr learned my addr 172.21.15.143:0/1269372405 (peer_addr_for_me v2:172.21.15.143:0/0) 2023-11-26T07:18:31.829 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:31.821+0000 7f6f0ffff640 1 -- 172.21.15.143:0/1269372405 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f6f180708b0 msgr2=0x7f6f180a91c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:18:31.829 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:31.821+0000 7f6f0ffff640 1 --2- 172.21.15.143:0/1269372405 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f6f180708b0 0x7f6f180a91c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:18:31.829 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:31.821+0000 7f6f0ffff640 1 -- 172.21.15.143:0/1269372405 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f6f18143ac0 msgr2=0x7f6f181f6b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:18:31.829 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:31.821+0000 7f6f0ffff640 1 --2- 172.21.15.143:0/1269372405 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f6f18143ac0 0x7f6f181f6b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:18:31.829 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:31.821+0000 7f6f0ffff640 1 -- 172.21.15.143:0/1269372405 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6f08009420 con 0x7f6f180cf990 2023-11-26T07:18:31.830 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:31.821+0000 7f6f0ffff640 1 --2- 172.21.15.143:0/1269372405 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f6f180cf990 0x7f6f180b04a0 secure :-1 s=READY pgs=281 cs=0 l=1 rev1=1 crypto rx=0x7f6f000011f0 tx=0x7f6f0000c9b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:18:31.830 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:31.821+0000 7f6f14ff9640 1 -- 172.21.15.143:0/1269372405 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6f000074e0 con 0x7f6f180cf990 2023-11-26T07:18:31.830 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:31.821+0000 7f6f14ff9640 1 -- 172.21.15.143:0/1269372405 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6f00007640 con 0x7f6f180cf990 2023-11-26T07:18:31.831 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:31.821+0000 7f6f14ff9640 1 -- 172.21.15.143:0/1269372405 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6f00004a10 con 0x7f6f180cf990 2023-11-26T07:18:31.831 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:31.821+0000 7f6f1e11d640 1 -- 172.21.15.143:0/1269372405 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6f180b0f60 con 0x7f6f180cf990 2023-11-26T07:18:31.831 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:31.825+0000 7f6f1e11d640 1 -- 172.21.15.143:0/1269372405 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6f180b1520 con 0x7f6f180cf990 2023-11-26T07:18:31.831 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:31.825+0000 7f6f1e11d640 1 -- 172.21.15.143:0/1269372405 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6f1815d560 con 0x7f6f180cf990 2023-11-26T07:18:31.831 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:31.825+0000 7f6f14ff9640 1 -- 172.21.15.143:0/1269372405 <== mon.0 v2:172.21.15.143:3300/0 4 ==== mgrmap(e 5) v1 ==== 85852+0+0 (secure 0 0 0) 0x7f6f00004b70 con 0x7f6f180cf990 2023-11-26T07:18:31.832 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:31.825+0000 7f6f14ff9640 1 --2- 172.21.15.143:0/1269372405 >> [v2:172.21.15.143:6832/2150557544,v1:172.21.15.143:6833/2150557544] conn(0x7f6ef8066470 0x7f6ef8068930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:18:31.832 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:31.825+0000 7f6f14ff9640 1 -- 172.21.15.143:0/1269372405 <== mon.0 v2:172.21.15.143:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f6f000831e0 con 0x7f6f180cf990 2023-11-26T07:18:31.832 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:31.825+0000 7f6f177fe640 1 --2- 172.21.15.143:0/1269372405 >> [v2:172.21.15.143:6832/2150557544,v1:172.21.15.143:6833/2150557544] conn(0x7f6ef8066470 0x7f6ef8068930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:18:31.832 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:31.825+0000 7f6f177fe640 1 --2- 172.21.15.143:0/1269372405 >> [v2:172.21.15.143:6832/2150557544,v1:172.21.15.143:6833/2150557544] conn(0x7f6ef8066470 0x7f6ef8068930 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7f6f080ad000 tx=0x7f6f08004740 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:18:31.835 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:31.829+0000 7f6f14ff9640 1 -- 172.21.15.143:0/1269372405 <== mon.0 v2:172.21.15.143:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172649 (secure 0 0 0) 0x7f6f000505c0 con 0x7f6f180cf990 2023-11-26T07:18:31.985 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:31.981+0000 7f6f1e11d640 1 -- 172.21.15.143:0/1269372405 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f6f18067680 con 0x7f6f180cf990 2023-11-26T07:18:31.986 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:31.981+0000 7f6f14ff9640 1 -- 172.21.15.143:0/1269372405 <== mon.0 v2:172.21.15.143:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v20) v1 ==== 93+0+10 (secure 0 0 0) 0x7f6f000180a0 con 0x7f6f180cf990 2023-11-26T07:18:31.986 INFO:teuthology.orchestra.run.smithi143.stdout:pg_num: 1 2023-11-26T07:18:31.988 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:31.981+0000 7f6f1e11d640 1 -- 172.21.15.143:0/1269372405 >> [v2:172.21.15.143:6832/2150557544,v1:172.21.15.143:6833/2150557544] conn(0x7f6ef8066470 msgr2=0x7f6ef8068930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:18:31.989 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:31.981+0000 7f6f1e11d640 1 --2- 172.21.15.143:0/1269372405 >> [v2:172.21.15.143:6832/2150557544,v1:172.21.15.143:6833/2150557544] conn(0x7f6ef8066470 0x7f6ef8068930 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7f6f080ad000 tx=0x7f6f08004740 comp rx=0 tx=0).stop 2023-11-26T07:18:31.989 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:31.981+0000 7f6f1e11d640 1 -- 172.21.15.143:0/1269372405 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f6f180cf990 msgr2=0x7f6f180b04a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:18:31.989 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:31.981+0000 7f6f1e11d640 1 --2- 172.21.15.143:0/1269372405 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f6f180cf990 0x7f6f180b04a0 secure :-1 s=READY pgs=281 cs=0 l=1 rev1=1 crypto rx=0x7f6f000011f0 tx=0x7f6f0000c9b0 comp rx=0 tx=0).stop 2023-11-26T07:18:31.989 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:31.981+0000 7f6f1e11d640 1 -- 172.21.15.143:0/1269372405 shutdown_connections 2023-11-26T07:18:31.989 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:31.981+0000 7f6f1e11d640 1 --2- 172.21.15.143:0/1269372405 >> [v2:172.21.15.143:6832/2150557544,v1:172.21.15.143:6833/2150557544] conn(0x7f6ef8066470 0x7f6ef8068930 unknown :-1 s=CLOSED pgs=164 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:18:31.989 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:31.981+0000 7f6f1e11d640 1 --2- 172.21.15.143:0/1269372405 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f6f18143ac0 0x7f6f181f6b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:18:31.989 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:31.981+0000 7f6f1e11d640 1 --2- 172.21.15.143:0/1269372405 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f6f180cf990 0x7f6f180b04a0 unknown :-1 s=CLOSED pgs=281 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:18:31.989 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:31.981+0000 7f6f1e11d640 1 --2- 172.21.15.143:0/1269372405 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f6f180708b0 0x7f6f180a91c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:18:31.989 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:31.981+0000 7f6f1e11d640 1 -- 172.21.15.143:0/1269372405 >> 172.21.15.143:0/1269372405 conn(0x7f6f1806a890 msgr2=0x7f6f1815c5e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:18:31.989 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:31.981+0000 7f6f1e11d640 1 -- 172.21.15.143:0/1269372405 shutdown_connections 2023-11-26T07:18:31.990 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:31.981+0000 7f6f1e11d640 1 -- 172.21.15.143:0/1269372405 wait complete. 2023-11-26T07:18:32.008 DEBUG:teuthology.orchestra.run.smithi143:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-11-26T07:18:32.148 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:32.141+0000 7f40a9965640 1 Processor -- start 2023-11-26T07:18:32.148 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:32.141+0000 7f40a9965640 1 -- start start 2023-11-26T07:18:32.149 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:32.145+0000 7f40a9965640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f40a40d17f0 0x7f40a40d1c30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:18:32.149 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:32.145+0000 7f40a9965640 1 -- --> v1:172.21.15.143:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f40a40d0bc0 con 0x7f40a40b0490 2023-11-26T07:18:32.149 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:32.145+0000 7f40a9965640 1 -- --> v1:172.21.15.188:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f40a40d0320 con 0x7f40a415d580 2023-11-26T07:18:32.149 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:32.145+0000 7f40a9965640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f40a40d0600 con 0x7f40a40d17f0 2023-11-26T07:18:32.150 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:32.145+0000 7f40a37fe640 1 --1- >> v1:172.21.15.143:6789/0 conn(0x7f40a40b0490 0x7f40a40b0920 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.143:6789/0 says I am v1:172.21.15.143:47698/0 (socket says 172.21.15.143:47698) 2023-11-26T07:18:32.150 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:32.145+0000 7f40a37fe640 1 -- 172.21.15.143:0/4208989308 learned_addr learned my addr 172.21.15.143:0/4208989308 (peer_addr_for_me v1:172.21.15.143:0/0) 2023-11-26T07:18:32.152 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:32.145+0000 7f409a7fc640 1 --2- 172.21.15.143:0/4208989308 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f40a40d17f0 0x7f40a40d1c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:18:32.152 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:32.145+0000 7f40a27fc640 1 -- 172.21.15.143:0/4208989308 <== mon.0 v1:172.21.15.143:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4023974716 0 0) 0x7f40a40d0bc0 con 0x7f40a40b0490 2023-11-26T07:18:32.152 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:32.145+0000 7f40a27fc640 1 -- 172.21.15.143:0/4208989308 --> v1:172.21.15.143:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f4090003530 con 0x7f40a40b0490 2023-11-26T07:18:32.152 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:32.145+0000 7f40a27fc640 1 -- 172.21.15.143:0/4208989308 <== mon.1 v1:172.21.15.188:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 455058270 0 0) 0x7f40a40d0320 con 0x7f40a415d580 2023-11-26T07:18:32.152 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:32.145+0000 7f40a27fc640 1 -- 172.21.15.143:0/4208989308 --> v1:172.21.15.188:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f40a40d0bc0 con 0x7f40a415d580 2023-11-26T07:18:32.152 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:32.145+0000 7f40a27fc640 1 -- 172.21.15.143:0/4208989308 <== mon.0 v1:172.21.15.143:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 293282384 0 0) 0x7f4090003530 con 0x7f40a40b0490 2023-11-26T07:18:32.152 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:32.145+0000 7f40a27fc640 1 -- 172.21.15.143:0/4208989308 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f40a40d17f0 msgr2=0x7f40a40d1c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-26T07:18:32.153 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:32.145+0000 7f40a27fc640 1 --2- 172.21.15.143:0/4208989308 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f40a40d17f0 0x7f40a40d1c30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:18:32.153 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:32.145+0000 7f40a27fc640 1 -- 172.21.15.143:0/4208989308 >> v1:172.21.15.188:6789/0 conn(0x7f40a415d580 legacy=0x7f40a40d3920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:18:32.153 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:32.145+0000 7f40a27fc640 1 -- 172.21.15.143:0/4208989308 --> v1:172.21.15.143:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f40a40d08e0 con 0x7f40a40b0490 2023-11-26T07:18:32.153 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:32.145+0000 7f40a27fc640 1 -- 172.21.15.143:0/4208989308 <== mon.0 v1:172.21.15.143:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2137882523 0 0) 0x7f409c001680 con 0x7f40a40b0490 2023-11-26T07:18:32.153 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:32.145+0000 7f40a27fc640 1 -- 172.21.15.143:0/4208989308 >> v1:172.21.15.143:6789/0 conn(0x7f40a40b0490 legacy=0x7f40a40b0920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:18:32.153 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:32.145+0000 7f40a27fc640 1 --2- 172.21.15.143:0/4208989308 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f4090003b40 0x7f4090003f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:18:32.153 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:32.145+0000 7f40a27fc640 1 --2- 172.21.15.143:0/4208989308 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f40900047d0 0x7f4090004c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:18:32.153 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:32.145+0000 7f40a27fc640 1 --2- 172.21.15.143:0/4208989308 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f40a40d17f0 0x7f4090005190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:18:32.153 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:32.145+0000 7f40a27fc640 1 -- 172.21.15.143:0/4208989308 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f40a40d0600 con 0x7f4090003b40 2023-11-26T07:18:32.153 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:32.145+0000 7f40a27fc640 1 -- 172.21.15.143:0/4208989308 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f409000ba40 con 0x7f40900047d0 2023-11-26T07:18:32.153 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:32.145+0000 7f40a27fc640 1 -- 172.21.15.143:0/4208989308 --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f409000bc00 con 0x7f40a40d17f0 2023-11-26T07:18:32.154 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:32.145+0000 7f40a2ffd640 1 --2- 172.21.15.143:0/4208989308 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f4090003b40 0x7f4090003f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:18:32.154 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:32.145+0000 7f409a7fc640 1 --2- 172.21.15.143:0/4208989308 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f40900047d0 0x7f4090004c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:18:32.154 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:32.145+0000 7f40a37fe640 1 --2- 172.21.15.143:0/4208989308 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f40a40d17f0 0x7f4090005190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:18:32.154 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:32.145+0000 7f40a2ffd640 1 -- 172.21.15.143:0/4208989308 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f40a40d17f0 msgr2=0x7f4090005190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-26T07:18:32.154 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:32.145+0000 7f40a2ffd640 1 --2- 172.21.15.143:0/4208989308 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f40a40d17f0 0x7f4090005190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:18:32.154 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:32.145+0000 7f40a2ffd640 1 -- 172.21.15.143:0/4208989308 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f40900047d0 msgr2=0x7f4090004c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-26T07:18:32.154 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:32.145+0000 7f40a2ffd640 1 --2- 172.21.15.143:0/4208989308 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f40900047d0 0x7f4090004c50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:18:32.154 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:32.145+0000 7f40a2ffd640 1 -- 172.21.15.143:0/4208989308 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4090003530 con 0x7f4090003b40 2023-11-26T07:18:32.154 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:32.145+0000 7f40a37fe640 1 --2- 172.21.15.143:0/4208989308 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f40a40d17f0 0x7f4090005190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-26T07:18:32.155 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:32.149+0000 7f40a2ffd640 1 --2- 172.21.15.143:0/4208989308 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f4090003b40 0x7f4090003f40 secure :-1 s=READY pgs=283 cs=0 l=1 rev1=1 crypto rx=0x7f40940018d0 tx=0x7f409401c530 comp rx=0 tx=0).ready entity=mon.0 client_cookie=18eede4970cc38f9 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:18:32.156 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:32.149+0000 7f40a27fc640 1 -- 172.21.15.143:0/4208989308 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f40940052d0 con 0x7f4090003b40 2023-11-26T07:18:32.156 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:32.149+0000 7f40a27fc640 1 -- 172.21.15.143:0/4208989308 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4094005450 con 0x7f4090003b40 2023-11-26T07:18:32.156 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:32.149+0000 7f40a27fc640 1 -- 172.21.15.143:0/4208989308 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4094003840 con 0x7f4090003b40 2023-11-26T07:18:32.156 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:32.149+0000 7f40a9965640 1 -- 172.21.15.143:0/4208989308 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f4090003b40 msgr2=0x7f4090003f40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:18:32.156 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:32.149+0000 7f40a9965640 1 --2- 172.21.15.143:0/4208989308 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f4090003b40 0x7f4090003f40 secure :-1 s=READY pgs=283 cs=0 l=1 rev1=1 crypto rx=0x7f40940018d0 tx=0x7f409401c530 comp rx=0 tx=0).stop 2023-11-26T07:18:32.156 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:32.149+0000 7f40a37fe640 1 -- 172.21.15.143:0/4208989308 reap_dead start 2023-11-26T07:18:32.156 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:32.149+0000 7f40a9965640 1 -- 172.21.15.143:0/4208989308 shutdown_connections 2023-11-26T07:18:32.156 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:32.149+0000 7f40a9965640 1 -- 172.21.15.143:0/4208989308 >> 172.21.15.143:0/4208989308 conn(0x7f40a409c550 msgr2=0x7f40a40d3210 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:18:32.157 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:32.149+0000 7f40a9965640 1 -- 172.21.15.143:0/4208989308 shutdown_connections 2023-11-26T07:18:32.157 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:32.149+0000 7f40a9965640 1 -- 172.21.15.143:0/4208989308 wait complete. 2023-11-26T07:18:32.158 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:32.149+0000 7f40a9965640 1 Processor -- start 2023-11-26T07:18:32.158 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:32.149+0000 7f40a9965640 1 -- start start 2023-11-26T07:18:32.158 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:32.153+0000 7f40a9965640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f4090003b40 0x7f40a4150440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:18:32.159 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:32.153+0000 7f40a9965640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f40a41f6430 0x7f40a406cb40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:18:32.159 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:32.153+0000 7f40a9965640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f40a4162a00 0x7f40a4162eb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:18:32.159 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:32.153+0000 7f40a9965640 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f40a41633f0 con 0x7f40a41f6430 2023-11-26T07:18:32.159 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:32.153+0000 7f40a9965640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f40a4150980 con 0x7f40a4162a00 2023-11-26T07:18:32.159 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:32.153+0000 7f40a9965640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f40a406d080 con 0x7f4090003b40 2023-11-26T07:18:32.159 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:32.153+0000 7f409a7fc640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f40a41f6430 0x7f40a406cb40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:18:32.159 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:32.153+0000 7f409a7fc640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f40a41f6430 0x7f40a406cb40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:33564/0 (socket says 172.21.15.143:33564) 2023-11-26T07:18:32.159 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:32.153+0000 7f409a7fc640 1 -- 172.21.15.143:0/2557969098 learned_addr learned my addr 172.21.15.143:0/2557969098 (peer_addr_for_me v2:172.21.15.143:0/0) 2023-11-26T07:18:32.159 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:32.153+0000 7f40a37fe640 1 --2- 172.21.15.143:0/2557969098 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f40a4162a00 0x7f40a4162eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:18:32.159 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:32.153+0000 7f40a2ffd640 1 --2- 172.21.15.143:0/2557969098 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f4090003b40 0x7f40a4150440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:18:32.160 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:32.153+0000 7f409a7fc640 1 -- 172.21.15.143:0/2557969098 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f4090003b40 msgr2=0x7f40a4150440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:18:32.160 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:32.153+0000 7f409a7fc640 1 --2- 172.21.15.143:0/2557969098 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f4090003b40 0x7f40a4150440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:18:32.160 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:32.153+0000 7f409a7fc640 1 -- 172.21.15.143:0/2557969098 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f40a4162a00 msgr2=0x7f40a4162eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:18:32.160 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:32.153+0000 7f409a7fc640 1 --2- 172.21.15.143:0/2557969098 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f40a4162a00 0x7f40a4162eb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:18:32.160 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:32.153+0000 7f409a7fc640 1 -- 172.21.15.143:0/2557969098 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f40a41f7420 con 0x7f40a41f6430 2023-11-26T07:18:32.160 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:32.153+0000 7f409a7fc640 1 --2- 172.21.15.143:0/2557969098 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f40a41f6430 0x7f40a406cb40 secure :-1 s=READY pgs=284 cs=0 l=1 rev1=1 crypto rx=0x7f408c011e30 tx=0x7f408c00f580 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:18:32.161 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:32.153+0000 7f40a0ff9640 1 -- 172.21.15.143:0/2557969098 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f408c00e040 con 0x7f40a41f6430 2023-11-26T07:18:32.161 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:32.153+0000 7f40a0ff9640 1 -- 172.21.15.143:0/2557969098 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f408c00f040 con 0x7f40a41f6430 2023-11-26T07:18:32.161 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:32.153+0000 7f40a0ff9640 1 -- 172.21.15.143:0/2557969098 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f408c009c00 con 0x7f40a41f6430 2023-11-26T07:18:32.161 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:32.153+0000 7f40a9965640 1 -- 172.21.15.143:0/2557969098 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f40a41f68b0 con 0x7f40a41f6430 2023-11-26T07:18:32.161 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:32.153+0000 7f40a9965640 1 -- 172.21.15.143:0/2557969098 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f40a41f6e60 con 0x7f40a41f6430 2023-11-26T07:18:32.162 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:32.153+0000 7f40a9965640 1 -- 172.21.15.143:0/2557969098 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f40a4067680 con 0x7f40a41f6430 2023-11-26T07:18:32.162 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:32.157+0000 7f40a0ff9640 1 -- 172.21.15.143:0/2557969098 <== mon.0 v2:172.21.15.143:3300/0 4 ==== mgrmap(e 5) v1 ==== 85852+0+0 (secure 0 0 0) 0x7f408c015850 con 0x7f40a41f6430 2023-11-26T07:18:32.162 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:32.157+0000 7f40a0ff9640 1 --2- 172.21.15.143:0/2557969098 >> [v2:172.21.15.143:6832/2150557544,v1:172.21.15.143:6833/2150557544] conn(0x7f40840664a0 0x7f4084068960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:18:32.163 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:32.157+0000 7f40a0ff9640 1 -- 172.21.15.143:0/2557969098 <== mon.0 v2:172.21.15.143:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f408c084b50 con 0x7f40a41f6430 2023-11-26T07:18:32.163 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:32.157+0000 7f40a2ffd640 1 --2- 172.21.15.143:0/2557969098 >> [v2:172.21.15.143:6832/2150557544,v1:172.21.15.143:6833/2150557544] conn(0x7f40840664a0 0x7f4084068960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:18:32.163 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:32.157+0000 7f40a2ffd640 1 --2- 172.21.15.143:0/2557969098 >> [v2:172.21.15.143:6832/2150557544,v1:172.21.15.143:6833/2150557544] conn(0x7f40840664a0 0x7f4084068960 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7f40940ad000 tx=0x7f40940a7000 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:18:32.165 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:32.161+0000 7f40a0ff9640 1 -- 172.21.15.143:0/2557969098 <== mon.0 v2:172.21.15.143:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172649 (secure 0 0 0) 0x7f408c052110 con 0x7f40a41f6430 2023-11-26T07:18:32.316 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:32.309+0000 7f40a9965640 1 -- 172.21.15.143:0/2557969098 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f40a40b4ab0 con 0x7f40a41f6430 2023-11-26T07:18:32.316 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:32.309+0000 7f40a0ff9640 1 -- 172.21.15.143:0/2557969098 <== mon.0 v2:172.21.15.143:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v20) v1 ==== 104+0+11 (secure 0 0 0) 0x7f408c055730 con 0x7f40a41f6430 2023-11-26T07:18:32.317 INFO:teuthology.orchestra.run.smithi143.stdout:pg_num: 32 2023-11-26T07:18:32.319 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:32.313+0000 7f40a9965640 1 -- 172.21.15.143:0/2557969098 >> [v2:172.21.15.143:6832/2150557544,v1:172.21.15.143:6833/2150557544] conn(0x7f40840664a0 msgr2=0x7f4084068960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:18:32.319 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:32.313+0000 7f40a9965640 1 --2- 172.21.15.143:0/2557969098 >> [v2:172.21.15.143:6832/2150557544,v1:172.21.15.143:6833/2150557544] conn(0x7f40840664a0 0x7f4084068960 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7f40940ad000 tx=0x7f40940a7000 comp rx=0 tx=0).stop 2023-11-26T07:18:32.319 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:32.313+0000 7f40a9965640 1 -- 172.21.15.143:0/2557969098 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f40a41f6430 msgr2=0x7f40a406cb40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:18:32.319 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:32.313+0000 7f40a9965640 1 --2- 172.21.15.143:0/2557969098 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f40a41f6430 0x7f40a406cb40 secure :-1 s=READY pgs=284 cs=0 l=1 rev1=1 crypto rx=0x7f408c011e30 tx=0x7f408c00f580 comp rx=0 tx=0).stop 2023-11-26T07:18:32.319 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:32.313+0000 7f40a9965640 1 -- 172.21.15.143:0/2557969098 shutdown_connections 2023-11-26T07:18:32.319 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:32.313+0000 7f40a9965640 1 --2- 172.21.15.143:0/2557969098 >> [v2:172.21.15.143:6832/2150557544,v1:172.21.15.143:6833/2150557544] conn(0x7f40840664a0 0x7f4084068960 unknown :-1 s=CLOSED pgs=165 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:18:32.319 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:32.313+0000 7f40a9965640 1 --2- 172.21.15.143:0/2557969098 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f40a4162a00 0x7f40a4162eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:18:32.319 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:32.313+0000 7f40a9965640 1 --2- 172.21.15.143:0/2557969098 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f40a41f6430 0x7f40a406cb40 unknown :-1 s=CLOSED pgs=284 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:18:32.319 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:32.313+0000 7f40a9965640 1 --2- 172.21.15.143:0/2557969098 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f4090003b40 0x7f40a4150440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:18:32.319 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:32.313+0000 7f40a9965640 1 -- 172.21.15.143:0/2557969098 >> 172.21.15.143:0/2557969098 conn(0x7f40a409c550 msgr2=0x7f40a40b0920 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:18:32.320 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:32.313+0000 7f40a9965640 1 -- 172.21.15.143:0/2557969098 shutdown_connections 2023-11-26T07:18:32.320 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:32.313+0000 7f40a9965640 1 -- 172.21.15.143:0/2557969098 wait complete. 2023-11-26T07:18:32.337 DEBUG:teuthology.orchestra.run.smithi143:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-11-26T07:18:32.477 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:32.469+0000 7f9c41a79640 1 Processor -- start 2023-11-26T07:18:32.477 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:32.469+0000 7f9c41a79640 1 -- start start 2023-11-26T07:18:32.478 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:32.473+0000 7f9c41a79640 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7f9c3c1614b0 0x7f9c3c0cd610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:18:32.478 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:32.473+0000 7f9c41a79640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f9c3c070ba0 0x7f9c3c071010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:18:32.478 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:32.473+0000 7f9c41a79640 1 -- --> v1:172.21.15.188:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9c3c15d970 con 0x7f9c3c071970 2023-11-26T07:18:32.479 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:32.473+0000 7f9c41a79640 1 -- --> v2:172.21.15.188:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9c3c071550 con 0x7f9c3c1614b0 2023-11-26T07:18:32.479 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:32.473+0000 7f9c41a79640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9c3c15dc30 con 0x7f9c3c070ba0 2023-11-26T07:18:32.479 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:32.473+0000 7f9c3b7fe640 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7f9c3c1614b0 0x7f9c3c0cd610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:18:32.479 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:32.473+0000 7f9c3bfff640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f9c3c070ba0 0x7f9c3c071010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:18:32.479 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:32.473+0000 7f9c3b7fe640 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7f9c3c1614b0 0x7f9c3c0cd610 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.188:3300/0 says I am v2:172.21.15.143:33996/0 (socket says 172.21.15.143:33996) 2023-11-26T07:18:32.479 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:32.473+0000 7f9c3bfff640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f9c3c070ba0 0x7f9c3c071010 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.188:3301/0 says I am v2:172.21.15.143:46078/0 (socket says 172.21.15.143:46078) 2023-11-26T07:18:32.479 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:32.473+0000 7f9c3b7fe640 1 -- 172.21.15.143:0/3431518703 learned_addr learned my addr 172.21.15.143:0/3431518703 (peer_addr_for_me v2:172.21.15.143:0/0) 2023-11-26T07:18:32.480 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:32.473+0000 7f9c3b7fe640 1 -- 172.21.15.143:0/3431518703 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f9c3c070ba0 msgr2=0x7f9c3c071010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-26T07:18:32.480 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:32.473+0000 7f9c3b7fe640 1 --2- 172.21.15.143:0/3431518703 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f9c3c070ba0 0x7f9c3c071010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:18:32.480 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:32.473+0000 7f9c3b7fe640 1 -- 172.21.15.143:0/3431518703 >> v1:172.21.15.188:6789/0 conn(0x7f9c3c071970 legacy=0x7f9c3c161890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:18:32.480 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:32.473+0000 7f9c3b7fe640 1 -- 172.21.15.143:0/3431518703 --> v2:172.21.15.188:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9c3c15df10 con 0x7f9c3c1614b0 2023-11-26T07:18:32.482 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:32.477+0000 7f9c3b7fe640 1 --2- 172.21.15.143:0/3431518703 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f9c3c1614b0 0x7f9c3c0cd610 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7f9c2c009870 tx=0x7f9c2c01c510 comp rx=0 tx=0).ready entity=mon.1 client_cookie=327fbeaf7781a454 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:18:32.482 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:32.477+0000 7f9c3a7fc640 1 -- 172.21.15.143:0/3431518703 <== mon.1 v2:172.21.15.188:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9c2c0aa670 con 0x7f9c3c1614b0 2023-11-26T07:18:32.482 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:32.477+0000 7f9c3a7fc640 1 -- 172.21.15.143:0/3431518703 <== mon.1 v2:172.21.15.188:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9c2c0aa7d0 con 0x7f9c3c1614b0 2023-11-26T07:18:32.482 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:32.477+0000 7f9c3a7fc640 1 -- 172.21.15.143:0/3431518703 <== mon.1 v2:172.21.15.188:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9c2c0aab60 con 0x7f9c3c1614b0 2023-11-26T07:18:32.483 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:32.477+0000 7f9c41a79640 1 -- 172.21.15.143:0/3431518703 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f9c3c1614b0 msgr2=0x7f9c3c0cd610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:18:32.483 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:32.477+0000 7f9c41a79640 1 --2- 172.21.15.143:0/3431518703 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f9c3c1614b0 0x7f9c3c0cd610 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7f9c2c009870 tx=0x7f9c2c01c510 comp rx=0 tx=0).stop 2023-11-26T07:18:32.483 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:32.477+0000 7f9c41a79640 1 -- 172.21.15.143:0/3431518703 shutdown_connections 2023-11-26T07:18:32.483 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:32.477+0000 7f9c41a79640 1 --2- 172.21.15.143:0/3431518703 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f9c3c070ba0 0x7f9c3c071010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:18:32.483 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:32.477+0000 7f9c41a79640 1 --2- 172.21.15.143:0/3431518703 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f9c3c1614b0 0x7f9c3c0cd610 unknown :-1 s=CLOSED pgs=144 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:18:32.483 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:32.477+0000 7f9c41a79640 1 -- 172.21.15.143:0/3431518703 >> 172.21.15.143:0/3431518703 conn(0x7f9c3c0cf710 msgr2=0x7f9c3c0cfb20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:18:32.483 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:32.477+0000 7f9c41a79640 1 -- 172.21.15.143:0/3431518703 shutdown_connections 2023-11-26T07:18:32.484 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:32.477+0000 7f9c41a79640 1 -- 172.21.15.143:0/3431518703 wait complete. 2023-11-26T07:18:32.484 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:32.477+0000 7f9c41a79640 1 Processor -- start 2023-11-26T07:18:32.485 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:32.481+0000 7f9c41a79640 1 -- start start 2023-11-26T07:18:32.487 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:32.481+0000 7f9c41a79640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f9c3c070ba0 0x7f9c3c144390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:18:32.487 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:32.481+0000 7f9c41a79640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f9c3c071970 0x7f9c3c157f60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:18:32.487 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:32.481+0000 7f9c41a79640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f9c3c1614b0 0x7f9c3c0ad930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:18:32.487 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:32.481+0000 7f9c41a79640 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9c3c155f80 con 0x7f9c3c1614b0 2023-11-26T07:18:32.487 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:32.481+0000 7f9c41a79640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9c3c0b4990 con 0x7f9c3c070ba0 2023-11-26T07:18:32.487 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:32.481+0000 7f9c41a79640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9c3c0b4410 con 0x7f9c3c071970 2023-11-26T07:18:32.487 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:32.481+0000 7f9c3bfff640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f9c3c1614b0 0x7f9c3c0ad930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:18:32.487 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:32.481+0000 7f9c3bfff640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f9c3c1614b0 0x7f9c3c0ad930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:33576/0 (socket says 172.21.15.143:33576) 2023-11-26T07:18:32.488 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:32.481+0000 7f9c3bfff640 1 -- 172.21.15.143:0/4052170628 learned_addr learned my addr 172.21.15.143:0/4052170628 (peer_addr_for_me v2:172.21.15.143:0/0) 2023-11-26T07:18:32.488 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:32.481+0000 7f9c3b7fe640 1 --2- 172.21.15.143:0/4052170628 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f9c3c070ba0 0x7f9c3c144390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:18:32.488 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:32.481+0000 7f9c3affd640 1 --2- 172.21.15.143:0/4052170628 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f9c3c071970 0x7f9c3c157f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:18:32.488 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:32.481+0000 7f9c3bfff640 1 -- 172.21.15.143:0/4052170628 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f9c3c071970 msgr2=0x7f9c3c157f60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:18:32.488 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:32.481+0000 7f9c3bfff640 1 --2- 172.21.15.143:0/4052170628 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f9c3c071970 0x7f9c3c157f60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:18:32.488 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:32.481+0000 7f9c3bfff640 1 -- 172.21.15.143:0/4052170628 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f9c3c070ba0 msgr2=0x7f9c3c144390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:18:32.488 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:32.481+0000 7f9c3bfff640 1 --2- 172.21.15.143:0/4052170628 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f9c3c070ba0 0x7f9c3c144390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:18:32.488 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:32.481+0000 7f9c3bfff640 1 -- 172.21.15.143:0/4052170628 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9c2c0094d0 con 0x7f9c3c1614b0 2023-11-26T07:18:32.488 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:32.481+0000 7f9c3b7fe640 1 --2- 172.21.15.143:0/4052170628 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f9c3c070ba0 0x7f9c3c144390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-26T07:18:32.488 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:32.481+0000 7f9c3affd640 1 --2- 172.21.15.143:0/4052170628 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f9c3c071970 0x7f9c3c157f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-26T07:18:32.488 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:32.481+0000 7f9c3bfff640 1 --2- 172.21.15.143:0/4052170628 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f9c3c1614b0 0x7f9c3c0ad930 secure :-1 s=READY pgs=285 cs=0 l=1 rev1=1 crypto rx=0x7f9c3400efc0 tx=0x7f9c3400c490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:18:32.488 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:32.481+0000 7f9c23fff640 1 -- 172.21.15.143:0/4052170628 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9c34009ba0 con 0x7f9c3c1614b0 2023-11-26T07:18:32.489 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:32.481+0000 7f9c23fff640 1 -- 172.21.15.143:0/4052170628 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9c3400f040 con 0x7f9c3c1614b0 2023-11-26T07:18:32.489 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:32.481+0000 7f9c41a79640 1 -- 172.21.15.143:0/4052170628 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9c3c1584a0 con 0x7f9c3c1614b0 2023-11-26T07:18:32.489 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:32.481+0000 7f9c23fff640 1 -- 172.21.15.143:0/4052170628 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9c340146c0 con 0x7f9c3c1614b0 2023-11-26T07:18:32.489 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:32.481+0000 7f9c41a79640 1 -- 172.21.15.143:0/4052170628 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9c3c1559c0 con 0x7f9c3c1614b0 2023-11-26T07:18:32.490 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:32.481+0000 7f9c41a79640 1 -- 172.21.15.143:0/4052170628 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9c3c161300 con 0x7f9c3c1614b0 2023-11-26T07:18:32.490 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:32.485+0000 7f9c23fff640 1 -- 172.21.15.143:0/4052170628 <== mon.0 v2:172.21.15.143:3300/0 4 ==== mgrmap(e 5) v1 ==== 85852+0+0 (secure 0 0 0) 0x7f9c34014820 con 0x7f9c3c1614b0 2023-11-26T07:18:32.490 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:32.485+0000 7f9c23fff640 1 --2- 172.21.15.143:0/4052170628 >> [v2:172.21.15.143:6832/2150557544,v1:172.21.15.143:6833/2150557544] conn(0x7f9c28066470 0x7f9c28068930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:18:32.490 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:32.485+0000 7f9c23fff640 1 -- 172.21.15.143:0/4052170628 <== mon.0 v2:172.21.15.143:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f9c34084c90 con 0x7f9c3c1614b0 2023-11-26T07:18:32.490 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:32.485+0000 7f9c3b7fe640 1 --2- 172.21.15.143:0/4052170628 >> [v2:172.21.15.143:6832/2150557544,v1:172.21.15.143:6833/2150557544] conn(0x7f9c28066470 0x7f9c28068930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:18:32.490 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:32.485+0000 7f9c3b7fe640 1 --2- 172.21.15.143:0/4052170628 >> [v2:172.21.15.143:6832/2150557544,v1:172.21.15.143:6833/2150557544] conn(0x7f9c28066470 0x7f9c28068930 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7f9c2c0ae000 tx=0x7f9c2c008040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:18:32.493 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:32.489+0000 7f9c23fff640 1 -- 172.21.15.143:0/4052170628 <== mon.0 v2:172.21.15.143:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172649 (secure 0 0 0) 0x7f9c34052100 con 0x7f9c3c1614b0 2023-11-26T07:18:32.643 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:32.637+0000 7f9c41a79640 1 -- 172.21.15.143:0/4052170628 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f9c3c067680 con 0x7f9c3c1614b0 2023-11-26T07:18:32.643 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:32.637+0000 7f9c23fff640 1 -- 172.21.15.143:0/4052170628 <== mon.0 v2:172.21.15.143:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v20) v1 ==== 100+0+11 (secure 0 0 0) 0x7f9c34018020 con 0x7f9c3c1614b0 2023-11-26T07:18:32.643 INFO:teuthology.orchestra.run.smithi143.stdout:pg_num: 64 2023-11-26T07:18:32.645 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:32.641+0000 7f9c41a79640 1 -- 172.21.15.143:0/4052170628 >> [v2:172.21.15.143:6832/2150557544,v1:172.21.15.143:6833/2150557544] conn(0x7f9c28066470 msgr2=0x7f9c28068930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:18:32.645 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:32.641+0000 7f9c41a79640 1 --2- 172.21.15.143:0/4052170628 >> [v2:172.21.15.143:6832/2150557544,v1:172.21.15.143:6833/2150557544] conn(0x7f9c28066470 0x7f9c28068930 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7f9c2c0ae000 tx=0x7f9c2c008040 comp rx=0 tx=0).stop 2023-11-26T07:18:32.645 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:32.641+0000 7f9c41a79640 1 -- 172.21.15.143:0/4052170628 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f9c3c1614b0 msgr2=0x7f9c3c0ad930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:18:32.645 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:32.641+0000 7f9c41a79640 1 --2- 172.21.15.143:0/4052170628 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f9c3c1614b0 0x7f9c3c0ad930 secure :-1 s=READY pgs=285 cs=0 l=1 rev1=1 crypto rx=0x7f9c3400efc0 tx=0x7f9c3400c490 comp rx=0 tx=0).stop 2023-11-26T07:18:32.646 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:32.641+0000 7f9c41a79640 1 -- 172.21.15.143:0/4052170628 shutdown_connections 2023-11-26T07:18:32.646 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:32.641+0000 7f9c41a79640 1 --2- 172.21.15.143:0/4052170628 >> [v2:172.21.15.143:6832/2150557544,v1:172.21.15.143:6833/2150557544] conn(0x7f9c28066470 0x7f9c28068930 unknown :-1 s=CLOSED pgs=166 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:18:32.646 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:32.641+0000 7f9c41a79640 1 --2- 172.21.15.143:0/4052170628 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f9c3c1614b0 0x7f9c3c0ad930 unknown :-1 s=CLOSED pgs=285 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:18:32.646 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:32.641+0000 7f9c41a79640 1 --2- 172.21.15.143:0/4052170628 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f9c3c071970 0x7f9c3c157f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:18:32.646 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:32.641+0000 7f9c41a79640 1 --2- 172.21.15.143:0/4052170628 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f9c3c070ba0 0x7f9c3c144390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:18:32.646 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:32.641+0000 7f9c41a79640 1 -- 172.21.15.143:0/4052170628 >> 172.21.15.143:0/4052170628 conn(0x7f9c3c0cf710 msgr2=0x7f9c3c0cfaf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:18:32.646 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:32.641+0000 7f9c41a79640 1 -- 172.21.15.143:0/4052170628 shutdown_connections 2023-11-26T07:18:32.646 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:32.641+0000 7f9c41a79640 1 -- 172.21.15.143:0/4052170628 wait complete. 2023-11-26T07:18:32.666 DEBUG:teuthology.orchestra.run.smithi143:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2023-11-26T07:18:32.809 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:32.805+0000 7f2cbbc5a640 1 Processor -- start 2023-11-26T07:18:32.809 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:32.805+0000 7f2cbbc5a640 1 -- start start 2023-11-26T07:18:32.811 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:32.805+0000 7f2cbbc5a640 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7f2cb41596d0 0x7f2cb4071850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:18:32.811 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:32.805+0000 7f2cbbc5a640 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7f2cb40b2f80 0x7f2cb406cbe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:18:32.811 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:32.805+0000 7f2cbbc5a640 1 -- --> v1:172.21.15.188:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2cb406c750 con 0x7f2cb40ce680 2023-11-26T07:18:32.811 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:32.805+0000 7f2cbbc5a640 1 -- --> v2:172.21.15.143:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2cb40cbe00 con 0x7f2cb41596d0 2023-11-26T07:18:32.811 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:32.805+0000 7f2cbbc5a640 1 -- --> v2:172.21.15.188:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2cb40b3550 con 0x7f2cb40b2f80 2023-11-26T07:18:32.812 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:32.805+0000 7f2cb99f6640 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7f2cb41596d0 0x7f2cb4071850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:18:32.812 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:32.805+0000 7f2cb99f6640 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7f2cb41596d0 0x7f2cb4071850 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:33584/0 (socket says 172.21.15.143:33584) 2023-11-26T07:18:32.812 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:32.805+0000 7f2cb99f6640 1 -- 172.21.15.143:0/2912500059 learned_addr learned my addr 172.21.15.143:0/2912500059 (peer_addr_for_me v2:172.21.15.143:0/0) 2023-11-26T07:18:32.812 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:32.805+0000 7f2cba1f7640 1 --2- 172.21.15.143:0/2912500059 >> v2:172.21.15.188:3300/0 conn(0x7f2cb40b2f80 0x7f2cb406cbe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:18:32.813 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:32.805+0000 7f2cb99f6640 1 -- 172.21.15.143:0/2912500059 >> v2:172.21.15.188:3300/0 conn(0x7f2cb40b2f80 msgr2=0x7f2cb406cbe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-26T07:18:32.813 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:32.805+0000 7f2cb99f6640 1 --2- 172.21.15.143:0/2912500059 >> v2:172.21.15.188:3300/0 conn(0x7f2cb40b2f80 0x7f2cb406cbe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:18:32.813 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:32.805+0000 7f2cb99f6640 1 -- 172.21.15.143:0/2912500059 >> v1:172.21.15.188:6789/0 conn(0x7f2cb40ce680 legacy=0x7f2cb40b2870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:18:32.813 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:32.805+0000 7f2cb99f6640 1 -- 172.21.15.143:0/2912500059 --> v2:172.21.15.143:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2cb4071e80 con 0x7f2cb41596d0 2023-11-26T07:18:32.814 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:32.809+0000 7f2cb99f6640 1 --2- 172.21.15.143:0/2912500059 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f2cb41596d0 0x7f2cb4071850 secure :-1 s=READY pgs=286 cs=0 l=1 rev1=1 crypto rx=0x7f2cb406e860 tx=0x7f2cb001c500 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8aa5b880e153b20d server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:18:32.814 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:32.809+0000 7f2cb89f4640 1 -- 172.21.15.143:0/2912500059 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2cb00ab600 con 0x7f2cb41596d0 2023-11-26T07:18:32.814 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:32.809+0000 7f2cb89f4640 1 -- 172.21.15.143:0/2912500059 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2cb00a7050 con 0x7f2cb41596d0 2023-11-26T07:18:32.815 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:32.809+0000 7f2cb89f4640 1 -- 172.21.15.143:0/2912500059 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2cb00ab980 con 0x7f2cb41596d0 2023-11-26T07:18:32.815 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:32.809+0000 7f2cbbc5a640 1 -- 172.21.15.143:0/2912500059 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f2cb41596d0 msgr2=0x7f2cb4071850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:18:32.815 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:32.809+0000 7f2cbbc5a640 1 --2- 172.21.15.143:0/2912500059 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f2cb41596d0 0x7f2cb4071850 secure :-1 s=READY pgs=286 cs=0 l=1 rev1=1 crypto rx=0x7f2cb406e860 tx=0x7f2cb001c500 comp rx=0 tx=0).stop 2023-11-26T07:18:32.815 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:32.809+0000 7f2cbbc5a640 1 -- 172.21.15.143:0/2912500059 shutdown_connections 2023-11-26T07:18:32.816 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:32.809+0000 7f2cbbc5a640 1 --2- 172.21.15.143:0/2912500059 >> v2:172.21.15.188:3300/0 conn(0x7f2cb40b2f80 0x7f2cb406cbe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:18:32.816 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:32.809+0000 7f2cbbc5a640 1 --2- 172.21.15.143:0/2912500059 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f2cb41596d0 0x7f2cb4071850 unknown :-1 s=CLOSED pgs=286 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:18:32.816 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:32.809+0000 7f2cbbc5a640 1 -- 172.21.15.143:0/2912500059 >> 172.21.15.143:0/2912500059 conn(0x7f2cb406a890 msgr2=0x7f2cb406aca0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:18:32.816 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:32.809+0000 7f2cbbc5a640 1 -- 172.21.15.143:0/2912500059 shutdown_connections 2023-11-26T07:18:32.816 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:32.809+0000 7f2cbbc5a640 1 -- 172.21.15.143:0/2912500059 wait complete. 2023-11-26T07:18:32.817 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:32.813+0000 7f2cbbc5a640 1 Processor -- start 2023-11-26T07:18:32.817 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:32.813+0000 7f2cbbc5a640 1 -- start start 2023-11-26T07:18:32.819 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:32.813+0000 7f2cbbc5a640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f2cb40b2f80 0x7f2cb4141a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:18:32.819 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:32.813+0000 7f2cbbc5a640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f2cb40ce680 0x7f2cb41447f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:18:32.819 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:32.813+0000 7f2cbbc5a640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f2cb41596d0 0x7f2cb4167d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:18:32.819 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:32.813+0000 7f2cbbc5a640 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2cb414d040 con 0x7f2cb40b2f80 2023-11-26T07:18:32.819 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:32.813+0000 7f2cbbc5a640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2cb414db60 con 0x7f2cb40ce680 2023-11-26T07:18:32.819 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:32.813+0000 7f2cbbc5a640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2cb414d2c0 con 0x7f2cb41596d0 2023-11-26T07:18:32.819 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:32.813+0000 7f2cb99f6640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f2cb40b2f80 0x7f2cb4141a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:18:32.819 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:32.813+0000 7f2cb99f6640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f2cb40b2f80 0x7f2cb4141a30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:33590/0 (socket says 172.21.15.143:33590) 2023-11-26T07:18:32.819 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:32.813+0000 7f2cb99f6640 1 -- 172.21.15.143:0/2018884491 learned_addr learned my addr 172.21.15.143:0/2018884491 (peer_addr_for_me v2:172.21.15.143:0/0) 2023-11-26T07:18:32.820 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:32.813+0000 7f2cba1f7640 1 --2- 172.21.15.143:0/2018884491 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f2cb41596d0 0x7f2cb4167d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:18:32.820 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:32.813+0000 7f2cb91f5640 1 --2- 172.21.15.143:0/2018884491 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f2cb40ce680 0x7f2cb41447f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:18:32.820 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:32.813+0000 7f2cb99f6640 1 -- 172.21.15.143:0/2018884491 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f2cb41596d0 msgr2=0x7f2cb4167d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:18:32.820 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:32.813+0000 7f2cb99f6640 1 --2- 172.21.15.143:0/2018884491 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f2cb41596d0 0x7f2cb4167d50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:18:32.820 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:32.813+0000 7f2cb99f6640 1 -- 172.21.15.143:0/2018884491 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f2cb40ce680 msgr2=0x7f2cb41447f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:18:32.820 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:32.813+0000 7f2cb99f6640 1 --2- 172.21.15.143:0/2018884491 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f2cb40ce680 0x7f2cb41447f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:18:32.820 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:32.813+0000 7f2cb99f6640 1 -- 172.21.15.143:0/2018884491 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2cb00094d0 con 0x7f2cb40b2f80 2023-11-26T07:18:32.820 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:32.813+0000 7f2cba1f7640 1 --2- 172.21.15.143:0/2018884491 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f2cb41596d0 0x7f2cb4167d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-26T07:18:32.820 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:32.813+0000 7f2cb99f6640 1 --2- 172.21.15.143:0/2018884491 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f2cb40b2f80 0x7f2cb4141a30 secure :-1 s=READY pgs=287 cs=0 l=1 rev1=1 crypto rx=0x7f2cb0019110 tx=0x7f2cb00192a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:18:32.821 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:32.813+0000 7f2caa7fc640 1 -- 172.21.15.143:0/2018884491 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2cb00b4420 con 0x7f2cb40b2f80 2023-11-26T07:18:32.821 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:32.813+0000 7f2caa7fc640 1 -- 172.21.15.143:0/2018884491 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2cb00b4580 con 0x7f2cb40b2f80 2023-11-26T07:18:32.821 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:32.813+0000 7f2cbbc5a640 1 -- 172.21.15.143:0/2018884491 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2cb414d5a0 con 0x7f2cb40b2f80 2023-11-26T07:18:32.821 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:32.813+0000 7f2caa7fc640 1 -- 172.21.15.143:0/2018884491 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2cb00b4930 con 0x7f2cb40b2f80 2023-11-26T07:18:32.821 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:32.813+0000 7f2cbbc5a640 1 -- 172.21.15.143:0/2018884491 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2cb40ae870 con 0x7f2cb40b2f80 2023-11-26T07:18:32.822 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:32.813+0000 7f2cbbc5a640 1 -- 172.21.15.143:0/2018884491 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2cb4067680 con 0x7f2cb40b2f80 2023-11-26T07:18:32.822 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:32.817+0000 7f2caa7fc640 1 -- 172.21.15.143:0/2018884491 <== mon.0 v2:172.21.15.143:3300/0 4 ==== mgrmap(e 5) v1 ==== 85852+0+0 (secure 0 0 0) 0x7f2cb00b2070 con 0x7f2cb40b2f80 2023-11-26T07:18:32.822 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:32.817+0000 7f2caa7fc640 1 --2- 172.21.15.143:0/2018884491 >> [v2:172.21.15.143:6832/2150557544,v1:172.21.15.143:6833/2150557544] conn(0x7f2c98066470 0x7f2c98068930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:18:32.822 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:32.817+0000 7f2caa7fc640 1 -- 172.21.15.143:0/2018884491 <== mon.0 v2:172.21.15.143:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f2cb011d140 con 0x7f2cb40b2f80 2023-11-26T07:18:32.822 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:32.817+0000 7f2cb91f5640 1 --2- 172.21.15.143:0/2018884491 >> [v2:172.21.15.143:6832/2150557544,v1:172.21.15.143:6833/2150557544] conn(0x7f2c98066470 0x7f2c98068930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:18:32.823 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:32.817+0000 7f2cb91f5640 1 --2- 172.21.15.143:0/2018884491 >> [v2:172.21.15.143:6832/2150557544,v1:172.21.15.143:6833/2150557544] conn(0x7f2c98066470 0x7f2c98068930 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7f2cb40b4000 tx=0x7f2ca4009040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:18:32.826 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:32.821+0000 7f2caa7fc640 1 -- 172.21.15.143:0/2018884491 <== mon.0 v2:172.21.15.143:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172649 (secure 0 0 0) 0x7f2cb00ea520 con 0x7f2cb40b2f80 2023-11-26T07:18:32.987 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:32.981+0000 7f2cbbc5a640 1 -- 172.21.15.143:0/2018884491 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) v1 -- 0x7f2cb40709f0 con 0x7f2cb40b2f80 2023-11-26T07:18:32.988 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:32.981+0000 7f2caa7fc640 1 -- 172.21.15.143:0/2018884491 <== mon.0 v2:172.21.15.143:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v20) v1 ==== 80+0+0 (secure 0 0 0) 0x7f2cb00edb40 con 0x7f2cb40b2f80 2023-11-26T07:18:32.988 INFO:teuthology.orchestra.run.smithi143.stderr:listed 0 entries 2023-11-26T07:18:32.990 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:32.985+0000 7f2cbbc5a640 1 -- 172.21.15.143:0/2018884491 >> [v2:172.21.15.143:6832/2150557544,v1:172.21.15.143:6833/2150557544] conn(0x7f2c98066470 msgr2=0x7f2c98068930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:18:32.990 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:32.985+0000 7f2cbbc5a640 1 --2- 172.21.15.143:0/2018884491 >> [v2:172.21.15.143:6832/2150557544,v1:172.21.15.143:6833/2150557544] conn(0x7f2c98066470 0x7f2c98068930 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7f2cb40b4000 tx=0x7f2ca4009040 comp rx=0 tx=0).stop 2023-11-26T07:18:32.990 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:32.985+0000 7f2cbbc5a640 1 -- 172.21.15.143:0/2018884491 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f2cb40b2f80 msgr2=0x7f2cb4141a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:18:32.990 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:32.985+0000 7f2cbbc5a640 1 --2- 172.21.15.143:0/2018884491 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f2cb40b2f80 0x7f2cb4141a30 secure :-1 s=READY pgs=287 cs=0 l=1 rev1=1 crypto rx=0x7f2cb0019110 tx=0x7f2cb00192a0 comp rx=0 tx=0).stop 2023-11-26T07:18:32.990 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:32.985+0000 7f2cbbc5a640 1 -- 172.21.15.143:0/2018884491 shutdown_connections 2023-11-26T07:18:32.990 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:32.985+0000 7f2cbbc5a640 1 --2- 172.21.15.143:0/2018884491 >> [v2:172.21.15.143:6832/2150557544,v1:172.21.15.143:6833/2150557544] conn(0x7f2c98066470 0x7f2c98068930 unknown :-1 s=CLOSED pgs=167 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:18:32.991 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:32.985+0000 7f2cbbc5a640 1 --2- 172.21.15.143:0/2018884491 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f2cb41596d0 0x7f2cb4167d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:18:32.991 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:32.985+0000 7f2cbbc5a640 1 --2- 172.21.15.143:0/2018884491 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f2cb40ce680 0x7f2cb41447f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:18:32.991 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:32.985+0000 7f2cbbc5a640 1 --2- 172.21.15.143:0/2018884491 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f2cb40b2f80 0x7f2cb4141a30 unknown :-1 s=CLOSED pgs=287 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:18:32.991 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:32.985+0000 7f2cbbc5a640 1 -- 172.21.15.143:0/2018884491 >> 172.21.15.143:0/2018884491 conn(0x7f2cb406a890 msgr2=0x7f2cb4070c10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:18:32.991 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:32.985+0000 7f2cbbc5a640 1 -- 172.21.15.143:0/2018884491 shutdown_connections 2023-11-26T07:18:32.991 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:32.985+0000 7f2cbbc5a640 1 -- 172.21.15.143:0/2018884491 wait complete. 2023-11-26T07:18:33.009 DEBUG:teuthology.orchestra.run.smithi143:> set -ex 2023-11-26T07:18:33.009 DEBUG:teuthology.orchestra.run.smithi143:> dd if=/proc/self/mounts of=/dev/stdout 2023-11-26T07:18:33.019 DEBUG:teuthology.orchestra.run.smithi143:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-11-26T07:18:33.196 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:33.189+0000 7fe963c32640 1 Processor -- start 2023-11-26T07:18:33.196 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:33.189+0000 7fe963c32640 1 -- start start 2023-11-26T07:18:33.199 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:33.193+0000 7fe963c32640 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7fe95c0cf990 0x7fe95c0708c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:18:33.199 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:33.193+0000 7fe963c32640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fe95c070e00 0x7fe95c162510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:18:33.199 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:33.193+0000 7fe963c32640 1 -- --> v1:172.21.15.143:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe95c162080 con 0x7fe95c14b1e0 2023-11-26T07:18:33.199 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:33.193+0000 7fe963c32640 1 -- --> v2:172.21.15.143:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe95c161d70 con 0x7fe95c0cf990 2023-11-26T07:18:33.199 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:33.193+0000 7fe963c32640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe95c14b5c0 con 0x7fe95c070e00 2023-11-26T07:18:33.199 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:33.193+0000 7fe9611cd640 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7fe95c0cf990 0x7fe95c0708c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:18:33.199 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:33.193+0000 7fe9619ce640 1 --1- >> v1:172.21.15.143:6789/0 conn(0x7fe95c14b1e0 0x7fe95c0cf280 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.143:6789/0 says I am v1:172.21.15.143:47700/0 (socket says 172.21.15.143:47700) 2023-11-26T07:18:33.199 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:33.193+0000 7fe9619ce640 1 -- 172.21.15.143:0/3381961265 learned_addr learned my addr 172.21.15.143:0/3381961265 (peer_addr_for_me v1:172.21.15.143:0/0) 2023-11-26T07:18:33.199 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:33.193+0000 7fe9621cf640 1 --2- 172.21.15.143:0/3381961265 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fe95c070e00 0x7fe95c162510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:18:33.199 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:33.193+0000 7fe9609cc640 1 -- 172.21.15.143:0/3381961265 <== mon.0 v1:172.21.15.143:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2182015371 0 0) 0x7fe95c162080 con 0x7fe95c14b1e0 2023-11-26T07:18:33.199 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:33.193+0000 7fe9609cc640 1 -- 172.21.15.143:0/3381961265 --> v1:172.21.15.143:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fe948003530 con 0x7fe95c14b1e0 2023-11-26T07:18:33.200 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:33.193+0000 7fe9611cd640 1 -- 172.21.15.143:0/3381961265 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fe95c070e00 msgr2=0x7fe95c162510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-26T07:18:33.200 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:33.193+0000 7fe9611cd640 1 --2- 172.21.15.143:0/3381961265 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fe95c070e00 0x7fe95c162510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:18:33.200 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:33.193+0000 7fe9611cd640 1 -- 172.21.15.143:0/3381961265 >> v1:172.21.15.143:6789/0 conn(0x7fe95c14b1e0 legacy=0x7fe95c0cf280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:18:33.200 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:33.193+0000 7fe9611cd640 1 -- 172.21.15.143:0/3381961265 --> v2:172.21.15.143:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe95c071580 con 0x7fe95c0cf990 2023-11-26T07:18:33.200 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:33.193+0000 7fe9621cf640 1 --2- 172.21.15.143:0/3381961265 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fe95c070e00 0x7fe95c162510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-26T07:18:33.202 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:33.197+0000 7fe9611cd640 1 --2- 172.21.15.143:0/3381961265 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fe95c0cf990 0x7fe95c0708c0 secure :-1 s=READY pgs=289 cs=0 l=1 rev1=1 crypto rx=0x7fe94c0097c0 tx=0x7fe94c01c450 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fad5bc7ee1414d24 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:18:33.202 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:33.197+0000 7fe9609cc640 1 -- 172.21.15.143:0/3381961265 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe94c0ab650 con 0x7fe95c0cf990 2023-11-26T07:18:33.202 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:33.197+0000 7fe9609cc640 1 -- 172.21.15.143:0/3381961265 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe94c0ab7b0 con 0x7fe95c0cf990 2023-11-26T07:18:33.202 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:33.197+0000 7fe9609cc640 1 -- 172.21.15.143:0/3381961265 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe94c0abb40 con 0x7fe95c0cf990 2023-11-26T07:18:33.203 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:33.197+0000 7fe963c32640 1 -- 172.21.15.143:0/3381961265 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fe95c0cf990 msgr2=0x7fe95c0708c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:18:33.203 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:33.197+0000 7fe963c32640 1 --2- 172.21.15.143:0/3381961265 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fe95c0cf990 0x7fe95c0708c0 secure :-1 s=READY pgs=289 cs=0 l=1 rev1=1 crypto rx=0x7fe94c0097c0 tx=0x7fe94c01c450 comp rx=0 tx=0).stop 2023-11-26T07:18:33.203 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:33.197+0000 7fe963c32640 1 -- 172.21.15.143:0/3381961265 shutdown_connections 2023-11-26T07:18:33.203 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:33.197+0000 7fe963c32640 1 --2- 172.21.15.143:0/3381961265 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fe95c070e00 0x7fe95c162510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:18:33.203 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:33.197+0000 7fe963c32640 1 --2- 172.21.15.143:0/3381961265 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fe95c0cf990 0x7fe95c0708c0 unknown :-1 s=CLOSED pgs=289 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:18:33.203 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:33.197+0000 7fe963c32640 1 -- 172.21.15.143:0/3381961265 >> 172.21.15.143:0/3381961265 conn(0x7fe95c06a890 msgr2=0x7fe95c06aca0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:18:33.203 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:33.197+0000 7fe963c32640 1 -- 172.21.15.143:0/3381961265 shutdown_connections 2023-11-26T07:18:33.203 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:33.197+0000 7fe963c32640 1 -- 172.21.15.143:0/3381961265 wait complete. 2023-11-26T07:18:33.204 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:33.197+0000 7fe963c32640 1 Processor -- start 2023-11-26T07:18:33.204 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:33.197+0000 7fe963c32640 1 -- start start 2023-11-26T07:18:33.207 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:33.201+0000 7fe963c32640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fe95c070e00 0x7fe95c1472a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:18:33.207 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:33.201+0000 7fe963c32640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fe95c0cf990 0x7fe95c1430d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:18:33.207 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:33.201+0000 7fe963c32640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fe95c14b1e0 0x7fe95c1460f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:18:33.208 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:33.201+0000 7fe963c32640 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe95c161630 con 0x7fe95c0cf990 2023-11-26T07:18:33.208 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:33.201+0000 7fe963c32640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe95c160a20 con 0x7fe95c070e00 2023-11-26T07:18:33.208 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:33.201+0000 7fe963c32640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe95c160d00 con 0x7fe95c14b1e0 2023-11-26T07:18:33.208 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:33.201+0000 7fe9611cd640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fe95c0cf990 0x7fe95c1430d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:18:33.208 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:33.201+0000 7fe9611cd640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fe95c0cf990 0x7fe95c1430d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:33614/0 (socket says 172.21.15.143:33614) 2023-11-26T07:18:33.208 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:33.201+0000 7fe9611cd640 1 -- 172.21.15.143:0/577728651 learned_addr learned my addr 172.21.15.143:0/577728651 (peer_addr_for_me v2:172.21.15.143:0/0) 2023-11-26T07:18:33.208 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:33.201+0000 7fe9619ce640 1 --2- 172.21.15.143:0/577728651 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fe95c070e00 0x7fe95c1472a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:18:33.208 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:33.201+0000 7fe9621cf640 1 --2- 172.21.15.143:0/577728651 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fe95c14b1e0 0x7fe95c1460f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:18:33.208 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:33.201+0000 7fe9611cd640 1 -- 172.21.15.143:0/577728651 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fe95c14b1e0 msgr2=0x7fe95c1460f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:18:33.208 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:33.201+0000 7fe9611cd640 1 --2- 172.21.15.143:0/577728651 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fe95c14b1e0 0x7fe95c1460f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:18:33.209 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:33.201+0000 7fe9611cd640 1 -- 172.21.15.143:0/577728651 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fe95c070e00 msgr2=0x7fe95c1472a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:18:33.209 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:33.201+0000 7fe9611cd640 1 --2- 172.21.15.143:0/577728651 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fe95c070e00 0x7fe95c1472a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:18:33.209 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:33.201+0000 7fe9611cd640 1 -- 172.21.15.143:0/577728651 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe94c009420 con 0x7fe95c0cf990 2023-11-26T07:18:33.209 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:33.201+0000 7fe9621cf640 1 --2- 172.21.15.143:0/577728651 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fe95c14b1e0 0x7fe95c1460f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-26T07:18:33.209 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:33.201+0000 7fe9611cd640 1 --2- 172.21.15.143:0/577728651 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fe95c0cf990 0x7fe95c1430d0 secure :-1 s=READY pgs=290 cs=0 l=1 rev1=1 crypto rx=0x7fe94c01a080 tx=0x7fe94c01ac90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:18:33.209 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:33.201+0000 7fe9527fc640 1 -- 172.21.15.143:0/577728651 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe94c0b5810 con 0x7fe95c0cf990 2023-11-26T07:18:33.209 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:33.201+0000 7fe9527fc640 1 -- 172.21.15.143:0/577728651 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe94c0b5990 con 0x7fe95c0cf990 2023-11-26T07:18:33.209 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:33.201+0000 7fe9527fc640 1 -- 172.21.15.143:0/577728651 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe94c0b5d40 con 0x7fe95c0cf990 2023-11-26T07:18:33.209 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:33.201+0000 7fe963c32640 1 -- 172.21.15.143:0/577728651 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe95c160fe0 con 0x7fe95c0cf990 2023-11-26T07:18:33.210 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:33.201+0000 7fe963c32640 1 -- 172.21.15.143:0/577728651 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe95c1477e0 con 0x7fe95c0cf990 2023-11-26T07:18:33.211 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:33.201+0000 7fe963c32640 1 -- 172.21.15.143:0/577728651 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe95c1617a0 con 0x7fe95c0cf990 2023-11-26T07:18:33.211 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:33.201+0000 7fe9527fc640 1 -- 172.21.15.143:0/577728651 <== mon.0 v2:172.21.15.143:3300/0 4 ==== mgrmap(e 5) v1 ==== 85852+0+0 (secure 0 0 0) 0x7fe94c0b4390 con 0x7fe95c0cf990 2023-11-26T07:18:33.211 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:33.205+0000 7fe9527fc640 1 --2- 172.21.15.143:0/577728651 >> [v2:172.21.15.143:6832/2150557544,v1:172.21.15.143:6833/2150557544] conn(0x7fe940066470 0x7fe940068930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:18:33.211 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:33.205+0000 7fe9527fc640 1 -- 172.21.15.143:0/577728651 <== mon.0 v2:172.21.15.143:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fe94c11e960 con 0x7fe95c0cf990 2023-11-26T07:18:33.211 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:33.205+0000 7fe9619ce640 1 --2- 172.21.15.143:0/577728651 >> [v2:172.21.15.143:6832/2150557544,v1:172.21.15.143:6833/2150557544] conn(0x7fe940066470 0x7fe940068930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:18:33.211 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:33.205+0000 7fe9619ce640 1 --2- 172.21.15.143:0/577728651 >> [v2:172.21.15.143:6832/2150557544,v1:172.21.15.143:6833/2150557544] conn(0x7fe940066470 0x7fe940068930 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7fe9580011f0 tx=0x7fe958007d40 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:18:33.213 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:33.209+0000 7fe9527fc640 1 -- 172.21.15.143:0/577728651 <== mon.0 v2:172.21.15.143:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172649 (secure 0 0 0) 0x7fe94c0ebdf0 con 0x7fe95c0cf990 2023-11-26T07:18:33.363 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:33.357+0000 7fe963c32640 1 -- 172.21.15.143:0/577728651 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fe95c067680 con 0x7fe95c0cf990 2023-11-26T07:18:33.364 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:33.357+0000 7fe9527fc640 1 -- 172.21.15.143:0/577728651 <== mon.0 v2:172.21.15.143:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v20) v1 ==== 74+0+16541 (secure 0 0 0) 0x7fe94c0ef410 con 0x7fe95c0cf990 2023-11-26T07:18:33.365 INFO:teuthology.orchestra.run.smithi143.stdout: 2023-11-26T07:18:33.365 INFO:teuthology.orchestra.run.smithi143.stdout:{"epoch":20,"fsid":"f5033060-2469-4a07-95fe-2b76f5404bc1","created":"2023-11-26T07:17:28.613663+0000","modified":"2023-11-26T07:17:55.505435+0000","last_up_change":"2023-11-26T07:17:41.157813+0000","last_in_change":"2023-11-26T07:17:31.181658+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-11-26T07:17:41.758597+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2023-11-26T07:17:51.878332+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2023-11-26T07:17:52.871396+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"ebbee2c6-023e-44b9-9f1d-6f7bfd309730","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6817","nonce":639807369},{"type":"v1","addr":"172.21.15.143:6819","nonce":639807369}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6821","nonce":639807369},{"type":"v1","addr":"172.21.15.143:6823","nonce":639807369}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6830","nonce":639807369},{"type":"v1","addr":"172.21.15.143:6831","nonce":639807369}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6825","nonce":639807369},{"type":"v1","addr":"172.21.15.143:6828","nonce":639807369}]},"public_addr":"172.21.15.143:6819/639807369","cluster_addr":"172.21.15.143:6823/639807369","heartbeat_back_addr":"172.21.15.143:6831/639807369","heartbeat_front_addr":"172.21.15.143:6828/639807369","state":["exists","up"]},{"osd":1,"uuid":"fec4b8be-7122-49e9-9a19-5850a0700001","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6816","nonce":1144942530},{"type":"v1","addr":"172.21.15.143:6818","nonce":1144942530}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6820","nonce":1144942530},{"type":"v1","addr":"172.21.15.143:6822","nonce":1144942530}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6827","nonce":1144942530},{"type":"v1","addr":"172.21.15.143:6829","nonce":1144942530}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6824","nonce":1144942530},{"type":"v1","addr":"172.21.15.143:6826","nonce":1144942530}]},"public_addr":"172.21.15.143:6818/1144942530","cluster_addr":"172.21.15.143:6822/1144942530","heartbeat_back_addr":"172.21.15.143:6829/1144942530","heartbeat_front_addr":"172.21.15.143:6826/1144942530","state":["exists","up"]},{"osd":2,"uuid":"2888af5b-f933-46ca-8f95-6dd494ff43f6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6808","nonce":3909095186},{"type":"v1","addr":"172.21.15.143:6809","nonce":3909095186}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6810","nonce":3909095186},{"type":"v1","addr":"172.21.15.143:6811","nonce":3909095186}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6814","nonce":3909095186},{"type":"v1","addr":"172.21.15.143:6815","nonce":3909095186}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6812","nonce":3909095186},{"type":"v1","addr":"172.21.15.143:6813","nonce":3909095186}]},"public_addr":"172.21.15.143:6809/3909095186","cluster_addr":"172.21.15.143:6811/3909095186","heartbeat_back_addr":"172.21.15.143:6815/3909095186","heartbeat_front_addr":"172.21.15.143:6813/3909095186","state":["exists","up"]},{"osd":3,"uuid":"d1f52435-7bc1-4081-9550-9cd1ba55f549","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6800","nonce":3854210991},{"type":"v1","addr":"172.21.15.143:6801","nonce":3854210991}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6802","nonce":3854210991},{"type":"v1","addr":"172.21.15.143:6803","nonce":3854210991}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6806","nonce":3854210991},{"type":"v1","addr":"172.21.15.143:6807","nonce":3854210991}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6804","nonce":3854210991},{"type":"v1","addr":"172.21.15.143:6805","nonce":3854210991}]},"public_addr":"172.21.15.143:6801/3854210991","cluster_addr":"172.21.15.143:6803/3854210991","heartbeat_back_addr":"172.21.15.143:6807/3854210991","heartbeat_front_addr":"172.21.15.143:6805/3854210991","state":["exists","up"]},{"osd":4,"uuid":"e2ae8b6e-6e25-49b2-bcbf-aacce68f06a3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6800","nonce":3924606902},{"type":"v1","addr":"172.21.15.188:6801","nonce":3924606902}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6802","nonce":3924606902},{"type":"v1","addr":"172.21.15.188:6803","nonce":3924606902}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6806","nonce":3924606902},{"type":"v1","addr":"172.21.15.188:6807","nonce":3924606902}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6804","nonce":3924606902},{"type":"v1","addr":"172.21.15.188:6805","nonce":3924606902}]},"public_addr":"172.21.15.188:6801/3924606902","cluster_addr":"172.21.15.188:6803/3924606902","heartbeat_back_addr":"172.21.15.188:6807/3924606902","heartbeat_front_addr":"172.21.15.188:6805/3924606902","state":["exists","up"]},{"osd":5,"uuid":"3cec8a32-7698-4c36-8e89-e4ba11934f33","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6813","nonce":186841544},{"type":"v1","addr":"172.21.15.188:6815","nonce":186841544}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6817","nonce":186841544},{"type":"v1","addr":"172.21.15.188:6818","nonce":186841544}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6822","nonce":186841544},{"type":"v1","addr":"172.21.15.188:6823","nonce":186841544}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6820","nonce":186841544},{"type":"v1","addr":"172.21.15.188:6821","nonce":186841544}]},"public_addr":"172.21.15.188:6815/186841544","cluster_addr":"172.21.15.188:6818/186841544","heartbeat_back_addr":"172.21.15.188:6823/186841544","heartbeat_front_addr":"172.21.15.188:6821/186841544","state":["exists","up"]},{"osd":6,"uuid":"58a65d5d-e090-473a-81b4-ad38e85029f9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6808","nonce":2153499405},{"type":"v1","addr":"172.21.15.188:6809","nonce":2153499405}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6810","nonce":2153499405},{"type":"v1","addr":"172.21.15.188:6811","nonce":2153499405}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6816","nonce":2153499405},{"type":"v1","addr":"172.21.15.188:6819","nonce":2153499405}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6812","nonce":2153499405},{"type":"v1","addr":"172.21.15.188:6814","nonce":2153499405}]},"public_addr":"172.21.15.188:6809/2153499405","cluster_addr":"172.21.15.188:6811/2153499405","heartbeat_back_addr":"172.21.15.188:6819/2153499405","heartbeat_front_addr":"172.21.15.188:6814/2153499405","state":["exists","up"]},{"osd":7,"uuid":"ea5b7f70-3554-4cf8-ac30-f7c59b80678d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6824","nonce":2722096729},{"type":"v1","addr":"172.21.15.188:6825","nonce":2722096729}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6826","nonce":2722096729},{"type":"v1","addr":"172.21.15.188:6827","nonce":2722096729}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6830","nonce":2722096729},{"type":"v1","addr":"172.21.15.188:6831","nonce":2722096729}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6828","nonce":2722096729},{"type":"v1","addr":"172.21.15.188:6829","nonce":2722096729}]},"public_addr":"172.21.15.188:6825/2722096729","cluster_addr":"172.21.15.188:6827/2722096729","heartbeat_back_addr":"172.21.15.188:6831/2722096729","heartbeat_front_addr":"172.21.15.188:6829/2722096729","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-26T07:17:36.995032+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-26T07:17:36.932266+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-26T07:17:36.829304+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-26T07:17:36.804297+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-26T07:17:36.667892+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-26T07:17:36.964868+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-26T07:17:36.827332+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-26T07:17:36.893179+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-11-26T07:18:33.367 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:33.361+0000 7fe963c32640 1 -- 172.21.15.143:0/577728651 >> [v2:172.21.15.143:6832/2150557544,v1:172.21.15.143:6833/2150557544] conn(0x7fe940066470 msgr2=0x7fe940068930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:18:33.367 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:33.361+0000 7fe963c32640 1 --2- 172.21.15.143:0/577728651 >> [v2:172.21.15.143:6832/2150557544,v1:172.21.15.143:6833/2150557544] conn(0x7fe940066470 0x7fe940068930 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7fe9580011f0 tx=0x7fe958007d40 comp rx=0 tx=0).stop 2023-11-26T07:18:33.367 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:33.361+0000 7fe963c32640 1 -- 172.21.15.143:0/577728651 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fe95c0cf990 msgr2=0x7fe95c1430d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:18:33.367 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:33.361+0000 7fe963c32640 1 --2- 172.21.15.143:0/577728651 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fe95c0cf990 0x7fe95c1430d0 secure :-1 s=READY pgs=290 cs=0 l=1 rev1=1 crypto rx=0x7fe94c01a080 tx=0x7fe94c01ac90 comp rx=0 tx=0).stop 2023-11-26T07:18:33.367 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:33.361+0000 7fe963c32640 1 -- 172.21.15.143:0/577728651 shutdown_connections 2023-11-26T07:18:33.368 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:33.361+0000 7fe963c32640 1 --2- 172.21.15.143:0/577728651 >> [v2:172.21.15.143:6832/2150557544,v1:172.21.15.143:6833/2150557544] conn(0x7fe940066470 0x7fe940068930 unknown :-1 s=CLOSED pgs=168 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:18:33.368 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:33.361+0000 7fe963c32640 1 --2- 172.21.15.143:0/577728651 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fe95c14b1e0 0x7fe95c1460f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:18:33.368 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:33.361+0000 7fe963c32640 1 --2- 172.21.15.143:0/577728651 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fe95c0cf990 0x7fe95c1430d0 unknown :-1 s=CLOSED pgs=290 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:18:33.368 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:33.361+0000 7fe963c32640 1 --2- 172.21.15.143:0/577728651 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fe95c070e00 0x7fe95c1472a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:18:33.368 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:33.361+0000 7fe963c32640 1 -- 172.21.15.143:0/577728651 >> 172.21.15.143:0/577728651 conn(0x7fe95c06a890 msgr2=0x7fe95c0b0e40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:18:33.368 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:33.361+0000 7fe963c32640 1 -- 172.21.15.143:0/577728651 shutdown_connections 2023-11-26T07:18:33.368 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:33.361+0000 7fe963c32640 1 -- 172.21.15.143:0/577728651 wait complete. 2023-11-26T07:18:33.386 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-11-26T07:17:41.758597+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2023-11-26T07:17:51.878332+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2023-11-26T07:17:52.871396+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '20', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2023-11-26T07:18:33.386 DEBUG:teuthology.orchestra.run.smithi143:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-11-26T07:18:33.529 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:33.521+0000 7f4c1d6ea640 1 Processor -- start 2023-11-26T07:18:33.529 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:33.521+0000 7f4c1d6ea640 1 -- start start 2023-11-26T07:18:33.530 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:33.525+0000 7f4c1d6ea640 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7f4c180d0150 0x7f4c180c6b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:18:33.530 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:33.525+0000 7f4c1d6ea640 1 -- --> v1:172.21.15.143:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4c180cefa0 con 0x7f4c180b45d0 2023-11-26T07:18:33.530 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:33.525+0000 7f4c1d6ea640 1 -- --> v1:172.21.15.188:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4c180d0f90 con 0x7f4c180b3610 2023-11-26T07:18:33.530 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:33.525+0000 7f4c1d6ea640 1 -- --> v2:172.21.15.143:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4c180c9fa0 con 0x7f4c180d0150 2023-11-26T07:18:33.530 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:33.525+0000 7f4c0ffff640 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7f4c180d0150 0x7f4c180c6b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:18:33.531 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:33.525+0000 7f4c177fe640 1 --1- >> v1:172.21.15.143:6789/0 conn(0x7f4c180b45d0 0x7f4c180c70c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.143:6789/0 says I am v1:172.21.15.143:47706/0 (socket says 172.21.15.143:47706) 2023-11-26T07:18:33.531 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:33.525+0000 7f4c0ffff640 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7f4c180d0150 0x7f4c180c6b80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:33630/0 (socket says 172.21.15.143:33630) 2023-11-26T07:18:33.531 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:33.525+0000 7f4c177fe640 1 -- 172.21.15.143:0/2423223630 learned_addr learned my addr 172.21.15.143:0/2423223630 (peer_addr_for_me v1:172.21.15.143:0/0) 2023-11-26T07:18:33.531 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:33.525+0000 7f4c0ffff640 1 -- 172.21.15.143:0/2423223630 >> v1:172.21.15.188:6789/0 conn(0x7f4c180b3610 legacy=0x7f4c180cfa40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:18:33.531 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:33.525+0000 7f4c0ffff640 1 -- 172.21.15.143:0/2423223630 >> v1:172.21.15.143:6789/0 conn(0x7f4c180b45d0 legacy=0x7f4c180c70c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:18:33.531 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:33.525+0000 7f4c0ffff640 1 -- 172.21.15.143:0/2423223630 --> v2:172.21.15.143:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4c180d3ef0 con 0x7f4c180d0150 2023-11-26T07:18:33.532 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:33.525+0000 7f4c167fc640 1 -- 172.21.15.143:0/2423223630 <== mon.0 v1:172.21.15.143:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 767643763 0 0) 0x7f4c180cefa0 con 0x7f4c180b45d0 2023-11-26T07:18:33.533 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:33.529+0000 7f4c0ffff640 1 --2- 172.21.15.143:0/2423223630 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f4c180d0150 0x7f4c180c6b80 secure :-1 s=READY pgs=292 cs=0 l=1 rev1=1 crypto rx=0x7f4c000097c0 tx=0x7f4c0001ec70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2a5850576ea4ae87 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:18:33.534 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:33.529+0000 7f4c167fc640 1 -- 172.21.15.143:0/2423223630 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4c000a8970 con 0x7f4c180d0150 2023-11-26T07:18:33.534 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:33.529+0000 7f4c167fc640 1 -- 172.21.15.143:0/2423223630 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4c000a8ad0 con 0x7f4c180d0150 2023-11-26T07:18:33.534 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:33.529+0000 7f4c167fc640 1 -- 172.21.15.143:0/2423223630 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4c000a8e60 con 0x7f4c180d0150 2023-11-26T07:18:33.534 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:33.529+0000 7f4c1d6ea640 1 -- 172.21.15.143:0/2423223630 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f4c180d0150 msgr2=0x7f4c180c6b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:18:33.535 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:33.529+0000 7f4c1d6ea640 1 --2- 172.21.15.143:0/2423223630 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f4c180d0150 0x7f4c180c6b80 secure :-1 s=READY pgs=292 cs=0 l=1 rev1=1 crypto rx=0x7f4c000097c0 tx=0x7f4c0001ec70 comp rx=0 tx=0).stop 2023-11-26T07:18:33.535 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:33.529+0000 7f4c1d6ea640 1 -- 172.21.15.143:0/2423223630 shutdown_connections 2023-11-26T07:18:33.535 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:33.529+0000 7f4c1d6ea640 1 --2- 172.21.15.143:0/2423223630 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f4c180d0150 0x7f4c180c6b80 unknown :-1 s=CLOSED pgs=292 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:18:33.535 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:33.529+0000 7f4c1d6ea640 1 -- 172.21.15.143:0/2423223630 >> 172.21.15.143:0/2423223630 conn(0x7f4c180d3030 msgr2=0x7f4c180d3440 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:18:33.535 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:33.529+0000 7f4c1d6ea640 1 -- 172.21.15.143:0/2423223630 shutdown_connections 2023-11-26T07:18:33.535 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:33.529+0000 7f4c1d6ea640 1 -- 172.21.15.143:0/2423223630 wait complete. 2023-11-26T07:18:33.536 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:33.529+0000 7f4c1d6ea640 1 Processor -- start 2023-11-26T07:18:33.536 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:33.529+0000 7f4c1d6ea640 1 -- start start 2023-11-26T07:18:33.538 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:33.533+0000 7f4c1d6ea640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f4c180b3610 0x7f4c180b76d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:18:33.538 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:33.533+0000 7f4c1d6ea640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f4c180b45d0 0x7f4c1815eee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:18:33.538 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:33.533+0000 7f4c1d6ea640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f4c180d0150 0x7f4c180b5c50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:18:33.538 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:33.533+0000 7f4c1d6ea640 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4c180b3ef0 con 0x7f4c180b45d0 2023-11-26T07:18:33.538 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:33.533+0000 7f4c1d6ea640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4c180b7240 con 0x7f4c180d0150 2023-11-26T07:18:33.538 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:33.533+0000 7f4c1d6ea640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4c180b6630 con 0x7f4c180b3610 2023-11-26T07:18:33.539 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:33.533+0000 7f4c0ffff640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f4c180b45d0 0x7f4c1815eee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:18:33.539 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:33.533+0000 7f4c0ffff640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f4c180b45d0 0x7f4c1815eee0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:33642/0 (socket says 172.21.15.143:33642) 2023-11-26T07:18:33.539 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:33.533+0000 7f4c0ffff640 1 -- 172.21.15.143:0/3070798585 learned_addr learned my addr 172.21.15.143:0/3070798585 (peer_addr_for_me v2:172.21.15.143:0/0) 2023-11-26T07:18:33.539 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:33.533+0000 7f4c16ffd640 1 --2- 172.21.15.143:0/3070798585 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f4c180b3610 0x7f4c180b76d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:18:33.539 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:33.533+0000 7f4c177fe640 1 --2- 172.21.15.143:0/3070798585 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f4c180d0150 0x7f4c180b5c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:18:33.539 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:33.533+0000 7f4c0ffff640 1 -- 172.21.15.143:0/3070798585 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f4c180b3610 msgr2=0x7f4c180b76d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:18:33.539 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:33.533+0000 7f4c0ffff640 1 --2- 172.21.15.143:0/3070798585 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f4c180b3610 0x7f4c180b76d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:18:33.539 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:33.533+0000 7f4c0ffff640 1 -- 172.21.15.143:0/3070798585 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f4c180d0150 msgr2=0x7f4c180b5c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:18:33.539 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:33.533+0000 7f4c0ffff640 1 --2- 172.21.15.143:0/3070798585 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f4c180d0150 0x7f4c180b5c50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:18:33.539 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:33.533+0000 7f4c0ffff640 1 -- 172.21.15.143:0/3070798585 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4c00009420 con 0x7f4c180b45d0 2023-11-26T07:18:33.539 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:33.533+0000 7f4c0ffff640 1 --2- 172.21.15.143:0/3070798585 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f4c180b45d0 0x7f4c1815eee0 secure :-1 s=READY pgs=293 cs=0 l=1 rev1=1 crypto rx=0x7f4c000097c0 tx=0x7f4c0001ba10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:18:33.540 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:33.533+0000 7f4c0f7fe640 1 -- 172.21.15.143:0/3070798585 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4c000b9630 con 0x7f4c180b45d0 2023-11-26T07:18:33.540 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:33.533+0000 7f4c0f7fe640 1 -- 172.21.15.143:0/3070798585 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4c000b97b0 con 0x7f4c180b45d0 2023-11-26T07:18:33.540 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:33.533+0000 7f4c1d6ea640 1 -- 172.21.15.143:0/3070798585 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4c180b6910 con 0x7f4c180b45d0 2023-11-26T07:18:33.540 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:33.533+0000 7f4c0f7fe640 1 -- 172.21.15.143:0/3070798585 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4c000b9b60 con 0x7f4c180b45d0 2023-11-26T07:18:33.540 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:33.533+0000 7f4c1d6ea640 1 -- 172.21.15.143:0/3070798585 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4c180b6ed0 con 0x7f4c180b45d0 2023-11-26T07:18:33.541 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:33.533+0000 7f4c1d6ea640 1 -- 172.21.15.143:0/3070798585 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4c18067500 con 0x7f4c180b45d0 2023-11-26T07:18:33.541 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:33.533+0000 7f4c0f7fe640 1 -- 172.21.15.143:0/3070798585 <== mon.0 v2:172.21.15.143:3300/0 4 ==== mgrmap(e 5) v1 ==== 85852+0+0 (secure 0 0 0) 0x7f4c000b7040 con 0x7f4c180b45d0 2023-11-26T07:18:33.542 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:33.537+0000 7f4c0f7fe640 1 --2- 172.21.15.143:0/3070798585 >> [v2:172.21.15.143:6832/2150557544,v1:172.21.15.143:6833/2150557544] conn(0x7f4bf8066470 0x7f4bf8068930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:18:33.542 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:33.537+0000 7f4c0f7fe640 1 -- 172.21.15.143:0/3070798585 <== mon.0 v2:172.21.15.143:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f4c00121d60 con 0x7f4c180b45d0 2023-11-26T07:18:33.542 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:33.537+0000 7f4c16ffd640 1 --2- 172.21.15.143:0/3070798585 >> [v2:172.21.15.143:6832/2150557544,v1:172.21.15.143:6833/2150557544] conn(0x7f4bf8066470 0x7f4bf8068930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:18:33.542 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:33.537+0000 7f4c16ffd640 1 --2- 172.21.15.143:0/3070798585 >> [v2:172.21.15.143:6832/2150557544,v1:172.21.15.143:6833/2150557544] conn(0x7f4bf8066470 0x7f4bf8068930 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7f4c180a0490 tx=0x7f4c08009040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:18:33.546 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:33.541+0000 7f4c0f7fe640 1 -- 172.21.15.143:0/3070798585 <== mon.0 v2:172.21.15.143:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172649 (secure 0 0 0) 0x7f4c000ef280 con 0x7f4c180b45d0 2023-11-26T07:18:33.697 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:33.693+0000 7f4c1d6ea640 1 -- 172.21.15.143:0/3070798585 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f4c180b8eb0 con 0x7f4c180b45d0 2023-11-26T07:18:33.697 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:33.693+0000 7f4c0f7fe640 1 -- 172.21.15.143:0/3070798585 <== mon.0 v2:172.21.15.143:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v20) v1 ==== 93+0+10 (secure 0 0 0) 0x7f4c000b5020 con 0x7f4c180b45d0 2023-11-26T07:18:33.697 INFO:teuthology.orchestra.run.smithi143.stdout:pg_num: 1 2023-11-26T07:18:33.700 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:33.693+0000 7f4c1d6ea640 1 -- 172.21.15.143:0/3070798585 >> [v2:172.21.15.143:6832/2150557544,v1:172.21.15.143:6833/2150557544] conn(0x7f4bf8066470 msgr2=0x7f4bf8068930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:18:33.700 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:33.693+0000 7f4c1d6ea640 1 --2- 172.21.15.143:0/3070798585 >> [v2:172.21.15.143:6832/2150557544,v1:172.21.15.143:6833/2150557544] conn(0x7f4bf8066470 0x7f4bf8068930 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7f4c180a0490 tx=0x7f4c08009040 comp rx=0 tx=0).stop 2023-11-26T07:18:33.700 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:33.693+0000 7f4c1d6ea640 1 -- 172.21.15.143:0/3070798585 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f4c180b45d0 msgr2=0x7f4c1815eee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:18:33.700 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:33.693+0000 7f4c1d6ea640 1 --2- 172.21.15.143:0/3070798585 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f4c180b45d0 0x7f4c1815eee0 secure :-1 s=READY pgs=293 cs=0 l=1 rev1=1 crypto rx=0x7f4c000097c0 tx=0x7f4c0001ba10 comp rx=0 tx=0).stop 2023-11-26T07:18:33.700 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:33.693+0000 7f4c1d6ea640 1 -- 172.21.15.143:0/3070798585 shutdown_connections 2023-11-26T07:18:33.700 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:33.693+0000 7f4c1d6ea640 1 --2- 172.21.15.143:0/3070798585 >> [v2:172.21.15.143:6832/2150557544,v1:172.21.15.143:6833/2150557544] conn(0x7f4bf8066470 0x7f4bf8068930 unknown :-1 s=CLOSED pgs=169 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:18:33.700 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:33.693+0000 7f4c1d6ea640 1 --2- 172.21.15.143:0/3070798585 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f4c180d0150 0x7f4c180b5c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:18:33.700 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:33.693+0000 7f4c1d6ea640 1 --2- 172.21.15.143:0/3070798585 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f4c180b45d0 0x7f4c1815eee0 unknown :-1 s=CLOSED pgs=293 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:18:33.700 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:33.693+0000 7f4c1d6ea640 1 --2- 172.21.15.143:0/3070798585 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f4c180b3610 0x7f4c180b76d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:18:33.700 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:33.693+0000 7f4c1d6ea640 1 -- 172.21.15.143:0/3070798585 >> 172.21.15.143:0/3070798585 conn(0x7f4c180d3030 msgr2=0x7f4c1806b970 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:18:33.700 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:33.693+0000 7f4c1d6ea640 1 -- 172.21.15.143:0/3070798585 shutdown_connections 2023-11-26T07:18:33.700 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:33.693+0000 7f4c1d6ea640 1 -- 172.21.15.143:0/3070798585 wait complete. 2023-11-26T07:18:33.719 DEBUG:teuthology.orchestra.run.smithi143:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-11-26T07:18:33.859 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:33.853+0000 7f573b3a1640 1 Processor -- start 2023-11-26T07:18:33.859 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:33.853+0000 7f573b3a1640 1 -- start start 2023-11-26T07:18:33.861 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:33.853+0000 7f573b3a1640 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7f573413e290 0x7f57340cf280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:18:33.861 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:33.853+0000 7f573b3a1640 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7f57340cf7c0 0x7f573415efa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:18:33.861 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:33.853+0000 7f573b3a1640 1 -- --> v1:172.21.15.143:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f573406c520 con 0x7f573415f4e0 2023-11-26T07:18:33.861 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:33.853+0000 7f573b3a1640 1 -- --> v2:172.21.15.143:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f57340cfc40 con 0x7f57340cf7c0 2023-11-26T07:18:33.861 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:33.853+0000 7f573b3a1640 1 -- --> v2:172.21.15.188:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f573406c800 con 0x7f573413e290 2023-11-26T07:18:33.861 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:33.857+0000 7f573893c640 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7f57340cf7c0 0x7f573415efa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:18:33.861 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:33.857+0000 7f573993e640 1 --1- >> v1:172.21.15.143:6789/0 conn(0x7f573415f4e0 0x7f57341408e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.143:6789/0 says I am v1:172.21.15.143:47714/0 (socket says 172.21.15.143:47714) 2023-11-26T07:18:33.861 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:33.857+0000 7f573993e640 1 -- 172.21.15.143:0/2113251664 learned_addr learned my addr 172.21.15.143:0/2113251664 (peer_addr_for_me v1:172.21.15.143:0/0) 2023-11-26T07:18:33.862 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:33.857+0000 7f573913d640 1 --2- 172.21.15.143:0/2113251664 >> v2:172.21.15.188:3300/0 conn(0x7f573413e290 0x7f57340cf280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:18:33.862 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:33.857+0000 7f573893c640 1 -- 172.21.15.143:0/2113251664 >> v2:172.21.15.188:3300/0 conn(0x7f573413e290 msgr2=0x7f57340cf280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-26T07:18:33.862 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:33.857+0000 7f573893c640 1 --2- 172.21.15.143:0/2113251664 >> v2:172.21.15.188:3300/0 conn(0x7f573413e290 0x7f57340cf280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:18:33.862 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:33.857+0000 7f573893c640 1 -- 172.21.15.143:0/2113251664 >> v1:172.21.15.143:6789/0 conn(0x7f573415f4e0 legacy=0x7f57341408e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:18:33.863 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:33.857+0000 7f573893c640 1 -- 172.21.15.143:0/2113251664 --> v2:172.21.15.143:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f573406cae0 con 0x7f57340cf7c0 2023-11-26T07:18:33.863 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:33.857+0000 7f572bfff640 1 -- 172.21.15.143:0/2113251664 <== mon.0 v1:172.21.15.143:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2071929134 0 0) 0x7f573406c520 con 0x7f573415f4e0 2023-11-26T07:18:33.864 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:33.857+0000 7f573893c640 1 --2- 172.21.15.143:0/2113251664 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f57340cf7c0 0x7f573415efa0 secure :-1 s=READY pgs=295 cs=0 l=1 rev1=1 crypto rx=0x7f57340725a0 tx=0x7f572401c430 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e0ca44087b4b7b06 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:18:33.864 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:33.857+0000 7f572bfff640 1 -- 172.21.15.143:0/2113251664 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f57240a7070 con 0x7f57340cf7c0 2023-11-26T07:18:33.865 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:33.857+0000 7f572bfff640 1 -- 172.21.15.143:0/2113251664 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f57240ab430 con 0x7f57340cf7c0 2023-11-26T07:18:33.865 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:33.857+0000 7f572bfff640 1 -- 172.21.15.143:0/2113251664 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f57240ab7c0 con 0x7f57340cf7c0 2023-11-26T07:18:33.865 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:33.861+0000 7f573b3a1640 1 -- 172.21.15.143:0/2113251664 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f57340cf7c0 msgr2=0x7f573415efa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:18:33.866 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:33.861+0000 7f573b3a1640 1 --2- 172.21.15.143:0/2113251664 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f57340cf7c0 0x7f573415efa0 secure :-1 s=READY pgs=295 cs=0 l=1 rev1=1 crypto rx=0x7f57340725a0 tx=0x7f572401c430 comp rx=0 tx=0).stop 2023-11-26T07:18:33.866 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:33.861+0000 7f573b3a1640 1 -- 172.21.15.143:0/2113251664 shutdown_connections 2023-11-26T07:18:33.866 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:33.861+0000 7f573b3a1640 1 --2- 172.21.15.143:0/2113251664 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f57340cf7c0 0x7f573415efa0 unknown :-1 s=CLOSED pgs=295 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:18:33.866 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:33.861+0000 7f573b3a1640 1 --2- 172.21.15.143:0/2113251664 >> v2:172.21.15.188:3300/0 conn(0x7f573413e290 0x7f57340cf280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:18:33.866 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:33.861+0000 7f573b3a1640 1 -- 172.21.15.143:0/2113251664 >> 172.21.15.143:0/2113251664 conn(0x7f573406a890 msgr2=0x7f573406aca0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:18:33.866 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:33.861+0000 7f573b3a1640 1 -- 172.21.15.143:0/2113251664 shutdown_connections 2023-11-26T07:18:33.866 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:33.861+0000 7f573b3a1640 1 -- 172.21.15.143:0/2113251664 wait complete. 2023-11-26T07:18:33.867 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:33.861+0000 7f573b3a1640 1 Processor -- start 2023-11-26T07:18:33.867 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:33.861+0000 7f573b3a1640 1 -- start start 2023-11-26T07:18:33.868 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:33.861+0000 7f573b3a1640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f57340cf7c0 0x7f5734154d60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:18:33.869 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:33.861+0000 7f573b3a1640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f573413e290 0x7f5734155d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:18:33.869 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:33.861+0000 7f573b3a1640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f573415f4e0 0x7f57340aeb60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:18:33.869 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:33.861+0000 7f573b3a1640 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f573415c700 con 0x7f573413e290 2023-11-26T07:18:33.869 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:33.861+0000 7f573b3a1640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f573415c9e0 con 0x7f573415f4e0 2023-11-26T07:18:33.869 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:33.861+0000 7f573b3a1640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f573415ccc0 con 0x7f57340cf7c0 2023-11-26T07:18:33.869 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:33.861+0000 7f573893c640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f573413e290 0x7f5734155d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:18:33.869 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:33.861+0000 7f573893c640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f573413e290 0x7f5734155d10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:33666/0 (socket says 172.21.15.143:33666) 2023-11-26T07:18:33.869 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:33.861+0000 7f573893c640 1 -- 172.21.15.143:0/1884278585 learned_addr learned my addr 172.21.15.143:0/1884278585 (peer_addr_for_me v2:172.21.15.143:0/0) 2023-11-26T07:18:33.869 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:33.861+0000 7f573893c640 1 -- 172.21.15.143:0/1884278585 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f57340cf7c0 msgr2=0x7f5734154d60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:18:33.869 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:33.861+0000 7f573893c640 1 --2- 172.21.15.143:0/1884278585 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f57340cf7c0 0x7f5734154d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:18:33.869 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:33.861+0000 7f573893c640 1 -- 172.21.15.143:0/1884278585 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f573415f4e0 msgr2=0x7f57340aeb60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:18:33.870 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:33.861+0000 7f573893c640 1 --2- 172.21.15.143:0/1884278585 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f573415f4e0 0x7f57340aeb60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:18:33.870 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:33.861+0000 7f573893c640 1 -- 172.21.15.143:0/1884278585 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5724009420 con 0x7f573413e290 2023-11-26T07:18:33.870 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:33.865+0000 7f573893c640 1 --2- 172.21.15.143:0/1884278585 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f573413e290 0x7f5734155d10 secure :-1 s=READY pgs=296 cs=0 l=1 rev1=1 crypto rx=0x7f57240ad040 tx=0x7f572401ac70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:18:33.871 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:33.865+0000 7f5729ffb640 1 -- 172.21.15.143:0/1884278585 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f57240bf610 con 0x7f573413e290 2023-11-26T07:18:33.871 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:33.865+0000 7f5729ffb640 1 -- 172.21.15.143:0/1884278585 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f57240bf790 con 0x7f573413e290 2023-11-26T07:18:33.871 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:33.865+0000 7f5729ffb640 1 -- 172.21.15.143:0/1884278585 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f57240bfb40 con 0x7f573413e290 2023-11-26T07:18:33.871 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:33.865+0000 7f573b3a1640 1 -- 172.21.15.143:0/1884278585 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f573415cfa0 con 0x7f573413e290 2023-11-26T07:18:33.871 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:33.865+0000 7f573b3a1640 1 -- 172.21.15.143:0/1884278585 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5734156250 con 0x7f573413e290 2023-11-26T07:18:33.871 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:33.865+0000 7f573b3a1640 1 -- 172.21.15.143:0/1884278585 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f573415fb20 con 0x7f573413e290 2023-11-26T07:18:33.871 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:33.865+0000 7f5729ffb640 1 -- 172.21.15.143:0/1884278585 <== mon.0 v2:172.21.15.143:3300/0 4 ==== mgrmap(e 5) v1 ==== 85852+0+0 (secure 0 0 0) 0x7f57240b5400 con 0x7f573413e290 2023-11-26T07:18:33.872 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:33.865+0000 7f5729ffb640 1 --2- 172.21.15.143:0/1884278585 >> [v2:172.21.15.143:6832/2150557544,v1:172.21.15.143:6833/2150557544] conn(0x7f5718066470 0x7f5718068930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:18:33.872 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:33.865+0000 7f5729ffb640 1 -- 172.21.15.143:0/1884278585 <== mon.0 v2:172.21.15.143:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f572411e820 con 0x7f573413e290 2023-11-26T07:18:33.872 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:33.865+0000 7f573913d640 1 --2- 172.21.15.143:0/1884278585 >> [v2:172.21.15.143:6832/2150557544,v1:172.21.15.143:6833/2150557544] conn(0x7f5718066470 0x7f5718068930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:18:33.873 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:33.869+0000 7f573913d640 1 --2- 172.21.15.143:0/1884278585 >> [v2:172.21.15.143:6832/2150557544,v1:172.21.15.143:6833/2150557544] conn(0x7f5718066470 0x7f5718068930 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7f5730006aa0 tx=0x7f5730006a30 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:18:33.876 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:33.869+0000 7f5729ffb640 1 -- 172.21.15.143:0/1884278585 <== mon.0 v2:172.21.15.143:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172649 (secure 0 0 0) 0x7f573415fb20 con 0x7f573413e290 2023-11-26T07:18:34.028 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:34.021+0000 7f573b3a1640 1 -- 172.21.15.143:0/1884278585 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f573415fb20 con 0x7f573413e290 2023-11-26T07:18:34.028 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:34.021+0000 7f5729ffb640 1 -- 172.21.15.143:0/1884278585 <== mon.0 v2:172.21.15.143:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v20) v1 ==== 104+0+11 (secure 0 0 0) 0x7f57240c4020 con 0x7f573413e290 2023-11-26T07:18:34.029 INFO:teuthology.orchestra.run.smithi143.stdout:pg_num: 32 2023-11-26T07:18:34.031 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:34.025+0000 7f573b3a1640 1 -- 172.21.15.143:0/1884278585 >> [v2:172.21.15.143:6832/2150557544,v1:172.21.15.143:6833/2150557544] conn(0x7f5718066470 msgr2=0x7f5718068930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:18:34.031 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:34.025+0000 7f573b3a1640 1 --2- 172.21.15.143:0/1884278585 >> [v2:172.21.15.143:6832/2150557544,v1:172.21.15.143:6833/2150557544] conn(0x7f5718066470 0x7f5718068930 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7f5730006aa0 tx=0x7f5730006a30 comp rx=0 tx=0).stop 2023-11-26T07:18:34.031 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:34.025+0000 7f573b3a1640 1 -- 172.21.15.143:0/1884278585 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f573413e290 msgr2=0x7f5734155d10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:18:34.031 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:34.025+0000 7f573b3a1640 1 --2- 172.21.15.143:0/1884278585 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f573413e290 0x7f5734155d10 secure :-1 s=READY pgs=296 cs=0 l=1 rev1=1 crypto rx=0x7f57240ad040 tx=0x7f572401ac70 comp rx=0 tx=0).stop 2023-11-26T07:18:34.031 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:34.025+0000 7f573b3a1640 1 -- 172.21.15.143:0/1884278585 shutdown_connections 2023-11-26T07:18:34.031 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:34.025+0000 7f573b3a1640 1 --2- 172.21.15.143:0/1884278585 >> [v2:172.21.15.143:6832/2150557544,v1:172.21.15.143:6833/2150557544] conn(0x7f5718066470 0x7f5718068930 unknown :-1 s=CLOSED pgs=170 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:18:34.031 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:34.025+0000 7f573b3a1640 1 --2- 172.21.15.143:0/1884278585 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f573415f4e0 0x7f57340aeb60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:18:34.031 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:34.025+0000 7f573b3a1640 1 --2- 172.21.15.143:0/1884278585 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f573413e290 0x7f5734155d10 unknown :-1 s=CLOSED pgs=296 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:18:34.031 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:34.025+0000 7f573b3a1640 1 --2- 172.21.15.143:0/1884278585 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f57340cf7c0 0x7f5734154d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:18:34.032 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:34.025+0000 7f573b3a1640 1 -- 172.21.15.143:0/1884278585 >> 172.21.15.143:0/1884278585 conn(0x7f573406a890 msgr2=0x7f5734138e90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:18:34.032 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:34.025+0000 7f573b3a1640 1 -- 172.21.15.143:0/1884278585 shutdown_connections 2023-11-26T07:18:34.032 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:34.025+0000 7f573b3a1640 1 -- 172.21.15.143:0/1884278585 wait complete. 2023-11-26T07:18:34.050 DEBUG:teuthology.orchestra.run.smithi143:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-11-26T07:18:34.189 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:34.185+0000 7f1aa82c9640 1 Processor -- start 2023-11-26T07:18:34.189 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:34.185+0000 7f1aa82c9640 1 -- start start 2023-11-26T07:18:34.191 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:34.185+0000 7f1aa82c9640 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7f1aa0143ac0 0x7f1aa0070370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:18:34.191 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:34.185+0000 7f1aa82c9640 1 -- --> v1:172.21.15.143:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1aa00cc730 con 0x7f1aa00708b0 2023-11-26T07:18:34.191 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:34.185+0000 7f1aa82c9640 1 -- --> v1:172.21.15.188:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1aa0070ee0 con 0x7f1aa00cf990 2023-11-26T07:18:34.191 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:34.185+0000 7f1aa82c9640 1 -- --> v2:172.21.15.188:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1aa0143ea0 con 0x7f1aa0143ac0 2023-11-26T07:18:34.191 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:34.185+0000 7f1aa5864640 1 --1- >> v1:172.21.15.143:6789/0 conn(0x7f1aa00708b0 0x7f1aa00cf280 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.143:6789/0 says I am v1:172.21.15.143:47728/0 (socket says 172.21.15.143:47728) 2023-11-26T07:18:34.191 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:34.185+0000 7f1aa5864640 1 -- 172.21.15.143:0/2437592232 learned_addr learned my addr 172.21.15.143:0/2437592232 (peer_addr_for_me v1:172.21.15.143:0/0) 2023-11-26T07:18:34.193 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:34.185+0000 7f1aa6065640 1 --2- 172.21.15.143:0/2437592232 >> v2:172.21.15.188:3300/0 conn(0x7f1aa0143ac0 0x7f1aa0070370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:18:34.193 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:34.185+0000 7f1aa5063640 1 -- 172.21.15.143:0/2437592232 <== mon.0 v1:172.21.15.143:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 402210700 0 0) 0x7f1aa00cc730 con 0x7f1aa00708b0 2023-11-26T07:18:34.193 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:34.185+0000 7f1aa5063640 1 -- 172.21.15.143:0/2437592232 --> v1:172.21.15.143:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f1a8c005740 con 0x7f1aa00708b0 2023-11-26T07:18:34.193 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:34.185+0000 7f1aa5063640 1 -- 172.21.15.143:0/2437592232 <== mon.0 v1:172.21.15.143:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1071523555 0 0) 0x7f1a8c005740 con 0x7f1aa00708b0 2023-11-26T07:18:34.193 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:34.185+0000 7f1aa5063640 1 -- 172.21.15.143:0/2437592232 >> v2:172.21.15.188:3300/0 conn(0x7f1aa0143ac0 msgr2=0x7f1aa0070370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-26T07:18:34.193 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:34.185+0000 7f1aa5063640 1 --2- 172.21.15.143:0/2437592232 >> v2:172.21.15.188:3300/0 conn(0x7f1aa0143ac0 0x7f1aa0070370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:18:34.193 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:34.185+0000 7f1aa5063640 1 -- 172.21.15.143:0/2437592232 >> v1:172.21.15.188:6789/0 conn(0x7f1aa00cf990 legacy=0x7f1aa00cd1b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:18:34.194 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:34.185+0000 7f1aa5063640 1 -- 172.21.15.143:0/2437592232 --> v1:172.21.15.143:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1aa00711c0 con 0x7f1aa00708b0 2023-11-26T07:18:34.194 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:34.185+0000 7f1aa5063640 1 -- 172.21.15.143:0/2437592232 <== mon.0 v1:172.21.15.143:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2137882523 0 0) 0x7f1a90001680 con 0x7f1aa00708b0 2023-11-26T07:18:34.194 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:34.185+0000 7f1aa5063640 1 -- 172.21.15.143:0/2437592232 >> v1:172.21.15.143:6789/0 conn(0x7f1aa00708b0 legacy=0x7f1aa00cf280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:18:34.194 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:34.185+0000 7f1aa5063640 1 --2- 172.21.15.143:0/2437592232 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f1a8c005b90 0x7f1a8c005f70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:18:34.194 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:34.185+0000 7f1aa5063640 1 --2- 172.21.15.143:0/2437592232 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f1a8c0069e0 0x7f1a8c006e60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:18:34.194 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:34.185+0000 7f1aa5063640 1 --2- 172.21.15.143:0/2437592232 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f1a8c0073a0 0x7f1a8c00dba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:18:34.194 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:34.185+0000 7f1aa5063640 1 -- 172.21.15.143:0/2437592232 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1a8c00e1c0 con 0x7f1a8c0073a0 2023-11-26T07:18:34.194 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:34.185+0000 7f1aa5063640 1 -- 172.21.15.143:0/2437592232 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1a8c00e330 con 0x7f1a8c0069e0 2023-11-26T07:18:34.194 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:34.185+0000 7f1aa5063640 1 -- 172.21.15.143:0/2437592232 --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1a8c00e4f0 con 0x7f1a8c005b90 2023-11-26T07:18:34.194 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:34.185+0000 7f1aa6866640 1 --2- 172.21.15.143:0/2437592232 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f1a8c0073a0 0x7f1a8c00dba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:18:34.194 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:34.185+0000 7f1aa5864640 1 --2- 172.21.15.143:0/2437592232 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f1a8c0069e0 0x7f1a8c006e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:18:34.194 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:34.185+0000 7f1aa6065640 1 --2- 172.21.15.143:0/2437592232 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f1a8c005b90 0x7f1a8c005f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:18:34.195 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:34.185+0000 7f1aa6866640 1 -- 172.21.15.143:0/2437592232 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f1a8c005b90 msgr2=0x7f1a8c005f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-26T07:18:34.195 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:34.185+0000 7f1aa6866640 1 --2- 172.21.15.143:0/2437592232 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f1a8c005b90 0x7f1a8c005f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:18:34.195 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:34.185+0000 7f1aa6866640 1 -- 172.21.15.143:0/2437592232 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f1a8c0069e0 msgr2=0x7f1a8c006e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-26T07:18:34.195 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:34.185+0000 7f1aa6866640 1 --2- 172.21.15.143:0/2437592232 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f1a8c0069e0 0x7f1a8c006e60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:18:34.195 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:34.185+0000 7f1aa6866640 1 -- 172.21.15.143:0/2437592232 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1a8c005740 con 0x7f1a8c0073a0 2023-11-26T07:18:34.195 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:34.185+0000 7f1aa6065640 1 --2- 172.21.15.143:0/2437592232 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f1a8c005b90 0x7f1a8c005f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-26T07:18:34.196 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:34.189+0000 7f1aa6866640 1 -- 172.21.15.143:0/2437592232 reap_dead start 2023-11-26T07:18:34.196 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:34.189+0000 7f1aa6866640 1 --2- 172.21.15.143:0/2437592232 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f1a8c0073a0 0x7f1a8c00dba0 secure :-1 s=READY pgs=298 cs=0 l=1 rev1=1 crypto rx=0x7f1a98001f60 tx=0x7f1a9801c860 comp rx=0 tx=0).ready entity=mon.0 client_cookie=be27335d86a081c4 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:18:34.196 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:34.189+0000 7f1aa5063640 1 -- 172.21.15.143:0/2437592232 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1a98003b80 con 0x7f1a8c0073a0 2023-11-26T07:18:34.196 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:34.189+0000 7f1aa5063640 1 -- 172.21.15.143:0/2437592232 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1a98003ce0 con 0x7f1a8c0073a0 2023-11-26T07:18:34.197 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:34.189+0000 7f1aa5063640 1 -- 172.21.15.143:0/2437592232 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1a980052f0 con 0x7f1a8c0073a0 2023-11-26T07:18:34.197 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:34.189+0000 7f1aa82c9640 1 -- 172.21.15.143:0/2437592232 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f1a8c0073a0 msgr2=0x7f1a8c00dba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:18:34.197 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:34.189+0000 7f1aa82c9640 1 --2- 172.21.15.143:0/2437592232 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f1a8c0073a0 0x7f1a8c00dba0 secure :-1 s=READY pgs=298 cs=0 l=1 rev1=1 crypto rx=0x7f1a98001f60 tx=0x7f1a9801c860 comp rx=0 tx=0).stop 2023-11-26T07:18:34.197 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:34.189+0000 7f1aa82c9640 1 -- 172.21.15.143:0/2437592232 shutdown_connections 2023-11-26T07:18:34.197 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:34.189+0000 7f1aa82c9640 1 --2- 172.21.15.143:0/2437592232 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f1a8c0073a0 0x7f1a8c00dba0 unknown :-1 s=CLOSED pgs=298 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:18:34.197 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:34.189+0000 7f1aa82c9640 1 -- 172.21.15.143:0/2437592232 >> 172.21.15.143:0/2437592232 conn(0x7f1aa006a890 msgr2=0x7f1aa006aca0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:18:34.197 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:34.189+0000 7f1aa82c9640 1 -- 172.21.15.143:0/2437592232 shutdown_connections 2023-11-26T07:18:34.197 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:34.189+0000 7f1aa82c9640 1 -- 172.21.15.143:0/2437592232 wait complete. 2023-11-26T07:18:34.198 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:34.193+0000 7f1aa82c9640 1 Processor -- start 2023-11-26T07:18:34.198 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:34.193+0000 7f1aa82c9640 1 -- start start 2023-11-26T07:18:34.199 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:34.193+0000 7f1aa82c9640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f1a8c0073a0 0x7f1aa01edd20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:18:34.200 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:34.193+0000 7f1aa82c9640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f1aa00b31e0 0x7f1aa00b3660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:18:34.200 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:34.193+0000 7f1aa82c9640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f1aa00ad330 0x7f1aa00ad7b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:18:34.200 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:34.193+0000 7f1aa82c9640 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1aa00b3ba0 con 0x7f1aa00ad330 2023-11-26T07:18:34.200 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:34.193+0000 7f1aa82c9640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1aa00a8700 con 0x7f1a8c0073a0 2023-11-26T07:18:34.200 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:34.193+0000 7f1aa82c9640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1aa00ae060 con 0x7f1aa00b31e0 2023-11-26T07:18:34.200 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:34.193+0000 7f1aa6866640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f1aa00ad330 0x7f1aa00ad7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:18:34.200 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:34.193+0000 7f1aa6866640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f1aa00ad330 0x7f1aa00ad7b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:33690/0 (socket says 172.21.15.143:33690) 2023-11-26T07:18:34.200 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:34.193+0000 7f1aa6866640 1 -- 172.21.15.143:0/2981770122 learned_addr learned my addr 172.21.15.143:0/2981770122 (peer_addr_for_me v2:172.21.15.143:0/0) 2023-11-26T07:18:34.200 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:34.193+0000 7f1aa5864640 1 --2- 172.21.15.143:0/2981770122 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f1aa00b31e0 0x7f1aa00b3660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:18:34.200 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:34.193+0000 7f1aa6065640 1 --2- 172.21.15.143:0/2981770122 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f1a8c0073a0 0x7f1aa01edd20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:18:34.200 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:34.193+0000 7f1aa6866640 1 -- 172.21.15.143:0/2981770122 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f1aa00b31e0 msgr2=0x7f1aa00b3660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:18:34.200 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:34.193+0000 7f1aa6866640 1 --2- 172.21.15.143:0/2981770122 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f1aa00b31e0 0x7f1aa00b3660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:18:34.201 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:34.193+0000 7f1aa6866640 1 -- 172.21.15.143:0/2981770122 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f1a8c0073a0 msgr2=0x7f1aa01edd20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:18:34.201 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:34.193+0000 7f1aa6866640 1 --2- 172.21.15.143:0/2981770122 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f1a8c0073a0 0x7f1aa01edd20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:18:34.201 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:34.193+0000 7f1aa6866640 1 -- 172.21.15.143:0/2981770122 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1aa00a89a0 con 0x7f1aa00ad330 2023-11-26T07:18:34.201 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:34.193+0000 7f1aa6065640 1 --2- 172.21.15.143:0/2981770122 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f1a8c0073a0 0x7f1aa01edd20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-26T07:18:34.202 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:34.193+0000 7f1aa6866640 1 --2- 172.21.15.143:0/2981770122 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f1aa00ad330 0x7f1aa00ad7b0 secure :-1 s=READY pgs=299 cs=0 l=1 rev1=1 crypto rx=0x7f1a9800a030 tx=0x7f1a8c007a10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:18:34.202 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:34.193+0000 7f1a96ffd640 1 -- 172.21.15.143:0/2981770122 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1a98003b80 con 0x7f1aa00ad330 2023-11-26T07:18:34.202 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:34.193+0000 7f1a96ffd640 1 -- 172.21.15.143:0/2981770122 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1a98006b60 con 0x7f1aa00ad330 2023-11-26T07:18:34.202 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:34.193+0000 7f1aa82c9640 1 -- 172.21.15.143:0/2981770122 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1aa00a8c80 con 0x7f1aa00ad330 2023-11-26T07:18:34.202 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:34.193+0000 7f1a96ffd640 1 -- 172.21.15.143:0/2981770122 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1a980a5550 con 0x7f1aa00ad330 2023-11-26T07:18:34.202 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:34.193+0000 7f1aa82c9640 1 -- 172.21.15.143:0/2981770122 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1aa01ee260 con 0x7f1aa00ad330 2023-11-26T07:18:34.202 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:34.197+0000 7f1aa82c9640 1 -- 172.21.15.143:0/2981770122 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1aa0067680 con 0x7f1aa00ad330 2023-11-26T07:18:34.203 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:34.197+0000 7f1a96ffd640 1 -- 172.21.15.143:0/2981770122 <== mon.0 v2:172.21.15.143:3300/0 4 ==== mgrmap(e 5) v1 ==== 85852+0+0 (secure 0 0 0) 0x7f1a980a56b0 con 0x7f1aa00ad330 2023-11-26T07:18:34.203 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:34.197+0000 7f1a96ffd640 1 --2- 172.21.15.143:0/2981770122 >> [v2:172.21.15.143:6832/2150557544,v1:172.21.15.143:6833/2150557544] conn(0x7f1a84066470 0x7f1a84068930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:18:34.203 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:34.197+0000 7f1a96ffd640 1 -- 172.21.15.143:0/2981770122 <== mon.0 v2:172.21.15.143:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f1a9811d0d0 con 0x7f1aa00ad330 2023-11-26T07:18:34.203 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:34.197+0000 7f1aa6065640 1 --2- 172.21.15.143:0/2981770122 >> [v2:172.21.15.143:6832/2150557544,v1:172.21.15.143:6833/2150557544] conn(0x7f1a84066470 0x7f1a84068930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:18:34.204 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:34.197+0000 7f1aa6065640 1 --2- 172.21.15.143:0/2981770122 >> [v2:172.21.15.143:6832/2150557544,v1:172.21.15.143:6833/2150557544] conn(0x7f1a84066470 0x7f1a84068930 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7f1a9c009a70 tx=0x7f1a9c008040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:18:34.207 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:34.201+0000 7f1a96ffd640 1 -- 172.21.15.143:0/2981770122 <== mon.0 v2:172.21.15.143:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172649 (secure 0 0 0) 0x7f1a980ea4b0 con 0x7f1aa00ad330 2023-11-26T07:18:34.355 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:34.349+0000 7f1aa82c9640 1 -- 172.21.15.143:0/2981770122 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f1aa015d410 con 0x7f1aa00ad330 2023-11-26T07:18:34.355 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:34.349+0000 7f1a96ffd640 1 -- 172.21.15.143:0/2981770122 <== mon.0 v2:172.21.15.143:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v20) v1 ==== 100+0+11 (secure 0 0 0) 0x7f1a980edad0 con 0x7f1aa00ad330 2023-11-26T07:18:34.356 INFO:teuthology.orchestra.run.smithi143.stdout:pg_num: 64 2023-11-26T07:18:34.357 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:34.353+0000 7f1aa82c9640 1 -- 172.21.15.143:0/2981770122 >> [v2:172.21.15.143:6832/2150557544,v1:172.21.15.143:6833/2150557544] conn(0x7f1a84066470 msgr2=0x7f1a84068930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:18:34.357 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:34.353+0000 7f1aa82c9640 1 --2- 172.21.15.143:0/2981770122 >> [v2:172.21.15.143:6832/2150557544,v1:172.21.15.143:6833/2150557544] conn(0x7f1a84066470 0x7f1a84068930 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7f1a9c009a70 tx=0x7f1a9c008040 comp rx=0 tx=0).stop 2023-11-26T07:18:34.357 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:34.353+0000 7f1aa82c9640 1 -- 172.21.15.143:0/2981770122 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f1aa00ad330 msgr2=0x7f1aa00ad7b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:18:34.358 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:34.353+0000 7f1aa82c9640 1 --2- 172.21.15.143:0/2981770122 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f1aa00ad330 0x7f1aa00ad7b0 secure :-1 s=READY pgs=299 cs=0 l=1 rev1=1 crypto rx=0x7f1a9800a030 tx=0x7f1a8c007a10 comp rx=0 tx=0).stop 2023-11-26T07:18:34.358 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:34.353+0000 7f1aa82c9640 1 -- 172.21.15.143:0/2981770122 shutdown_connections 2023-11-26T07:18:34.358 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:34.353+0000 7f1aa82c9640 1 --2- 172.21.15.143:0/2981770122 >> [v2:172.21.15.143:6832/2150557544,v1:172.21.15.143:6833/2150557544] conn(0x7f1a84066470 0x7f1a84068930 unknown :-1 s=CLOSED pgs=171 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:18:34.358 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:34.353+0000 7f1aa82c9640 1 --2- 172.21.15.143:0/2981770122 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f1aa00ad330 0x7f1aa00ad7b0 unknown :-1 s=CLOSED pgs=299 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:18:34.358 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:34.353+0000 7f1aa82c9640 1 --2- 172.21.15.143:0/2981770122 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f1aa00b31e0 0x7f1aa00b3660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:18:34.358 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:34.353+0000 7f1aa82c9640 1 --2- 172.21.15.143:0/2981770122 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f1a8c0073a0 0x7f1aa01edd20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:18:34.358 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:34.353+0000 7f1aa82c9640 1 -- 172.21.15.143:0/2981770122 >> 172.21.15.143:0/2981770122 conn(0x7f1aa006a890 msgr2=0x7f1aa015c8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:18:34.358 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:34.353+0000 7f1aa82c9640 1 -- 172.21.15.143:0/2981770122 shutdown_connections 2023-11-26T07:18:34.358 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:34.353+0000 7f1aa82c9640 1 -- 172.21.15.143:0/2981770122 wait complete. 2023-11-26T07:18:34.376 DEBUG:teuthology.orchestra.run.smithi143:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2023-11-26T07:18:34.516 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:34.509+0000 7fe415eba640 1 Processor -- start 2023-11-26T07:18:34.517 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:34.509+0000 7fe415eba640 1 -- start start 2023-11-26T07:18:34.517 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:34.513+0000 7fe415eba640 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7fe41015d230 0x7fe4100d3be0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:18:34.518 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:34.513+0000 7fe415eba640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fe4100d01b0 0x7fe4100cdc40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:18:34.518 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:34.513+0000 7fe415eba640 1 -- --> v1:172.21.15.143:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe4100ce180 con 0x7fe41015d610 2023-11-26T07:18:34.518 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:34.513+0000 7fe415eba640 1 -- --> v2:172.21.15.188:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe4100ce3b0 con 0x7fe41015d230 2023-11-26T07:18:34.518 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:34.513+0000 7fe415eba640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe4100ce690 con 0x7fe4100d01b0 2023-11-26T07:18:34.518 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:34.513+0000 7fe40ffff640 1 --1- >> v1:172.21.15.143:6789/0 conn(0x7fe41015d610 0x7fe4100cfaa0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.143:6789/0 says I am v1:172.21.15.143:50860/0 (socket says 172.21.15.143:50860) 2023-11-26T07:18:34.518 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:34.513+0000 7fe40ffff640 1 -- 172.21.15.143:0/1723988006 learned_addr learned my addr 172.21.15.143:0/1723988006 (peer_addr_for_me v1:172.21.15.143:0/0) 2023-11-26T07:18:34.518 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:34.513+0000 7fe414eb8640 1 --2- 172.21.15.143:0/1723988006 >> v2:172.21.15.188:3300/0 conn(0x7fe41015d230 0x7fe4100d3be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:18:34.518 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:34.513+0000 7fe4156b9640 1 --2- 172.21.15.143:0/1723988006 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fe4100d01b0 0x7fe4100cdc40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:18:34.519 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:34.513+0000 7fe40f7fe640 1 -- 172.21.15.143:0/1723988006 <== mon.0 v1:172.21.15.143:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 299586620 0 0) 0x7fe4100ce180 con 0x7fe41015d610 2023-11-26T07:18:34.519 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:34.513+0000 7fe414eb8640 1 -- 172.21.15.143:0/1723988006 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fe4100d01b0 msgr2=0x7fe4100cdc40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-26T07:18:34.519 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:34.513+0000 7fe414eb8640 1 --2- 172.21.15.143:0/1723988006 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fe4100d01b0 0x7fe4100cdc40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:18:34.519 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:34.513+0000 7fe414eb8640 1 -- 172.21.15.143:0/1723988006 >> v1:172.21.15.143:6789/0 conn(0x7fe41015d610 legacy=0x7fe4100cfaa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:18:34.519 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:34.513+0000 7fe414eb8640 1 -- 172.21.15.143:0/1723988006 --> v2:172.21.15.188:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe4100ce970 con 0x7fe41015d230 2023-11-26T07:18:34.519 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:34.513+0000 7fe4156b9640 1 --2- 172.21.15.143:0/1723988006 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fe4100d01b0 0x7fe4100cdc40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-26T07:18:34.521 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:34.517+0000 7fe414eb8640 1 --2- 172.21.15.143:0/1723988006 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fe41015d230 0x7fe4100d3be0 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7fe4040097c0 tx=0x7fe40401c360 comp rx=0 tx=0).ready entity=mon.1 client_cookie=65f86f1d1aada639 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:18:34.522 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:34.517+0000 7fe40f7fe640 1 -- 172.21.15.143:0/1723988006 <== mon.1 v2:172.21.15.188:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe4040ad660 con 0x7fe41015d230 2023-11-26T07:18:34.522 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:34.517+0000 7fe40f7fe640 1 -- 172.21.15.143:0/1723988006 <== mon.1 v2:172.21.15.188:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe4040ad7c0 con 0x7fe41015d230 2023-11-26T07:18:34.522 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:34.517+0000 7fe40f7fe640 1 -- 172.21.15.143:0/1723988006 <== mon.1 v2:172.21.15.188:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe4040adb50 con 0x7fe41015d230 2023-11-26T07:18:34.523 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:34.517+0000 7fe415eba640 1 -- 172.21.15.143:0/1723988006 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fe41015d230 msgr2=0x7fe4100d3be0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:18:34.523 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:34.517+0000 7fe415eba640 1 --2- 172.21.15.143:0/1723988006 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fe41015d230 0x7fe4100d3be0 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7fe4040097c0 tx=0x7fe40401c360 comp rx=0 tx=0).stop 2023-11-26T07:18:34.523 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:34.517+0000 7fe415eba640 1 -- 172.21.15.143:0/1723988006 shutdown_connections 2023-11-26T07:18:34.523 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:34.517+0000 7fe415eba640 1 --2- 172.21.15.143:0/1723988006 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fe4100d01b0 0x7fe4100cdc40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:18:34.523 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:34.517+0000 7fe415eba640 1 --2- 172.21.15.143:0/1723988006 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fe41015d230 0x7fe4100d3be0 unknown :-1 s=CLOSED pgs=148 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:18:34.523 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:34.517+0000 7fe415eba640 1 -- 172.21.15.143:0/1723988006 >> 172.21.15.143:0/1723988006 conn(0x7fe4100d30c0 msgr2=0x7fe4100d34d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:18:34.524 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:34.517+0000 7fe415eba640 1 -- 172.21.15.143:0/1723988006 shutdown_connections 2023-11-26T07:18:34.524 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:34.517+0000 7fe415eba640 1 -- 172.21.15.143:0/1723988006 wait complete. 2023-11-26T07:18:34.524 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:34.517+0000 7fe415eba640 1 Processor -- start 2023-11-26T07:18:34.524 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:34.517+0000 7fe415eba640 1 -- start start 2023-11-26T07:18:34.527 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:34.521+0000 7fe415eba640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fe4100d01b0 0x7fe410160140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:18:34.527 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:34.521+0000 7fe415eba640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fe41015d230 0x7fe410157810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:18:34.527 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:34.521+0000 7fe415eba640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fe41015d610 0x7fe41014df20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:18:34.527 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:34.521+0000 7fe415eba640 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe4100ca330 con 0x7fe41015d230 2023-11-26T07:18:34.527 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:34.521+0000 7fe415eba640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe4100ca610 con 0x7fe4100d01b0 2023-11-26T07:18:34.527 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:34.521+0000 7fe415eba640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe4100ca8f0 con 0x7fe41015d610 2023-11-26T07:18:34.527 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:34.521+0000 7fe40ffff640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fe41015d230 0x7fe410157810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:18:34.527 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:34.521+0000 7fe40ffff640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fe41015d230 0x7fe410157810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:38778/0 (socket says 172.21.15.143:38778) 2023-11-26T07:18:34.528 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:34.521+0000 7fe40ffff640 1 -- 172.21.15.143:0/382143986 learned_addr learned my addr 172.21.15.143:0/382143986 (peer_addr_for_me v2:172.21.15.143:0/0) 2023-11-26T07:18:34.528 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:34.521+0000 7fe414eb8640 1 --2- 172.21.15.143:0/382143986 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fe4100d01b0 0x7fe410160140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:18:34.528 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:34.521+0000 7fe4156b9640 1 --2- 172.21.15.143:0/382143986 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fe41015d610 0x7fe41014df20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:18:34.528 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:34.521+0000 7fe40ffff640 1 -- 172.21.15.143:0/382143986 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fe41015d610 msgr2=0x7fe41014df20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:18:34.528 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:34.521+0000 7fe40ffff640 1 --2- 172.21.15.143:0/382143986 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fe41015d610 0x7fe41014df20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:18:34.528 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:34.521+0000 7fe40ffff640 1 -- 172.21.15.143:0/382143986 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fe4100d01b0 msgr2=0x7fe410160140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:18:34.528 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:34.521+0000 7fe40ffff640 1 --2- 172.21.15.143:0/382143986 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fe4100d01b0 0x7fe410160140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:18:34.528 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:34.521+0000 7fe40ffff640 1 -- 172.21.15.143:0/382143986 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe408009a70 con 0x7fe41015d230 2023-11-26T07:18:34.529 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:34.521+0000 7fe40ffff640 1 --2- 172.21.15.143:0/382143986 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fe41015d230 0x7fe410157810 secure :-1 s=READY pgs=301 cs=0 l=1 rev1=1 crypto rx=0x7fe3fc00c480 tx=0x7fe3fc00c880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:18:34.529 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:34.521+0000 7fe40d7fa640 1 -- 172.21.15.143:0/382143986 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe3fc0074e0 con 0x7fe41015d230 2023-11-26T07:18:34.529 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:34.521+0000 7fe40d7fa640 1 -- 172.21.15.143:0/382143986 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe3fc007660 con 0x7fe41015d230 2023-11-26T07:18:34.529 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:34.521+0000 7fe40d7fa640 1 -- 172.21.15.143:0/382143986 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe3fc004a30 con 0x7fe41015d230 2023-11-26T07:18:34.529 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:34.521+0000 7fe415eba640 1 -- 172.21.15.143:0/382143986 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe404009420 con 0x7fe41015d230 2023-11-26T07:18:34.529 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:34.521+0000 7fe415eba640 1 -- 172.21.15.143:0/382143986 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe410160680 con 0x7fe41015d230 2023-11-26T07:18:34.529 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:34.521+0000 7fe415eba640 1 -- 172.21.15.143:0/382143986 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe410067c10 con 0x7fe41015d230 2023-11-26T07:18:34.530 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:34.525+0000 7fe40d7fa640 1 -- 172.21.15.143:0/382143986 <== mon.0 v2:172.21.15.143:3300/0 4 ==== mgrmap(e 5) v1 ==== 85852+0+0 (secure 0 0 0) 0x7fe3fc004b90 con 0x7fe41015d230 2023-11-26T07:18:34.530 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:34.525+0000 7fe40d7fa640 1 --2- 172.21.15.143:0/382143986 >> [v2:172.21.15.143:6832/2150557544,v1:172.21.15.143:6833/2150557544] conn(0x7fe3f4066470 0x7fe3f4068930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:18:34.530 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:34.525+0000 7fe40d7fa640 1 -- 172.21.15.143:0/382143986 <== mon.0 v2:172.21.15.143:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fe3fc083b90 con 0x7fe41015d230 2023-11-26T07:18:34.531 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:34.525+0000 7fe414eb8640 1 --2- 172.21.15.143:0/382143986 >> [v2:172.21.15.143:6832/2150557544,v1:172.21.15.143:6833/2150557544] conn(0x7fe3f4066470 0x7fe3f4068930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:18:34.531 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:34.525+0000 7fe414eb8640 1 --2- 172.21.15.143:0/382143986 >> [v2:172.21.15.143:6832/2150557544,v1:172.21.15.143:6833/2150557544] conn(0x7fe3f4066470 0x7fe3f4068930 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7fe4040af000 tx=0x7fe4040a7000 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:18:34.534 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:34.529+0000 7fe40d7fa640 1 -- 172.21.15.143:0/382143986 <== mon.0 v2:172.21.15.143:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172649 (secure 0 0 0) 0x7fe3fc051010 con 0x7fe41015d230 2023-11-26T07:18:34.696 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:34.689+0000 7fe415eba640 1 -- 172.21.15.143:0/382143986 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) v1 -- 0x7fe4100ca470 con 0x7fe41015d230 2023-11-26T07:18:34.696 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:34.689+0000 7fe40d7fa640 1 -- 172.21.15.143:0/382143986 <== mon.0 v2:172.21.15.143:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v20) v1 ==== 80+0+0 (secure 0 0 0) 0x7fe4100ca470 con 0x7fe41015d230 2023-11-26T07:18:34.696 INFO:teuthology.orchestra.run.smithi143.stderr:listed 0 entries 2023-11-26T07:18:34.698 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:34.693+0000 7fe415eba640 1 -- 172.21.15.143:0/382143986 >> [v2:172.21.15.143:6832/2150557544,v1:172.21.15.143:6833/2150557544] conn(0x7fe3f4066470 msgr2=0x7fe3f4068930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:18:34.699 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:34.693+0000 7fe415eba640 1 --2- 172.21.15.143:0/382143986 >> [v2:172.21.15.143:6832/2150557544,v1:172.21.15.143:6833/2150557544] conn(0x7fe3f4066470 0x7fe3f4068930 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7fe4040af000 tx=0x7fe4040a7000 comp rx=0 tx=0).stop 2023-11-26T07:18:34.699 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:34.693+0000 7fe415eba640 1 -- 172.21.15.143:0/382143986 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fe41015d230 msgr2=0x7fe410157810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:18:34.699 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:34.693+0000 7fe415eba640 1 --2- 172.21.15.143:0/382143986 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fe41015d230 0x7fe410157810 secure :-1 s=READY pgs=301 cs=0 l=1 rev1=1 crypto rx=0x7fe3fc00c480 tx=0x7fe3fc00c880 comp rx=0 tx=0).stop 2023-11-26T07:18:34.699 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:34.693+0000 7fe415eba640 1 -- 172.21.15.143:0/382143986 shutdown_connections 2023-11-26T07:18:34.700 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:34.693+0000 7fe415eba640 1 --2- 172.21.15.143:0/382143986 >> [v2:172.21.15.143:6832/2150557544,v1:172.21.15.143:6833/2150557544] conn(0x7fe3f4066470 0x7fe3f4068930 unknown :-1 s=CLOSED pgs=172 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:18:34.700 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:34.693+0000 7fe415eba640 1 --2- 172.21.15.143:0/382143986 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fe41015d610 0x7fe41014df20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:18:34.700 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:34.693+0000 7fe415eba640 1 --2- 172.21.15.143:0/382143986 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fe41015d230 0x7fe410157810 unknown :-1 s=CLOSED pgs=301 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:18:34.700 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:34.693+0000 7fe415eba640 1 --2- 172.21.15.143:0/382143986 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fe4100d01b0 0x7fe410160140 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:18:34.700 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:34.693+0000 7fe415eba640 1 -- 172.21.15.143:0/382143986 >> 172.21.15.143:0/382143986 conn(0x7fe4100d30c0 msgr2=0x7fe4100d16a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:18:34.700 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:34.693+0000 7fe415eba640 1 -- 172.21.15.143:0/382143986 shutdown_connections 2023-11-26T07:18:34.701 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:34.693+0000 7fe415eba640 1 -- 172.21.15.143:0/382143986 wait complete. 2023-11-26T07:18:34.721 INFO:tasks.cephfs.fuse_mount:Running fusermount -u on ubuntu@smithi143.front.sepia.ceph.com... 2023-11-26T07:18:34.721 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-26T07:18:34.721 DEBUG:teuthology.orchestra.run.smithi143:> sudo fusermount -u /home/ubuntu/cephtest/mnt.2 2023-11-26T07:18:34.742 INFO:teuthology.orchestra.run:waiting for 300 2023-11-26T07:18:34.743 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi143.front.sepia.ceph.com 2023-11-26T07:18:34.743 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-26T07:18:34.743 DEBUG:teuthology.orchestra.run.smithi143:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.2) 2023-11-26T07:18:34.793 DEBUG:teuthology.orchestra.run.smithi188:> set -ex 2023-11-26T07:18:34.793 DEBUG:teuthology.orchestra.run.smithi188:> dd if=/proc/self/mounts of=/dev/stdout 2023-11-26T07:18:34.803 DEBUG:teuthology.orchestra.run.smithi188:> set -ex 2023-11-26T07:18:34.803 DEBUG:teuthology.orchestra.run.smithi188:> dd if=/proc/self/mounts of=/dev/stdout 2023-11-26T07:18:34.855 DEBUG:teuthology.orchestra.run.smithi143:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-11-26T07:18:34.992 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:34.985+0000 7f9f5872a640 1 Processor -- start 2023-11-26T07:18:34.992 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:34.985+0000 7f9f5872a640 1 -- start start 2023-11-26T07:18:34.993 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:34.989+0000 7f9f5872a640 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7f9f500cf990 0x7f9f5015efa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:18:34.993 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:34.989+0000 7f9f5872a640 1 -- --> v1:172.21.15.143:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9f5006c520 con 0x7f9f5015f4e0 2023-11-26T07:18:34.993 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:34.989+0000 7f9f5872a640 1 -- --> v1:172.21.15.188:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9f5006c800 con 0x7f9f5013e290 2023-11-26T07:18:34.994 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:34.989+0000 7f9f5872a640 1 -- --> v2:172.21.15.188:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9f5006cae0 con 0x7f9f500cf990 2023-11-26T07:18:34.996 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:34.989+0000 7f9f56cc7640 1 --1- >> v1:172.21.15.143:6789/0 conn(0x7f9f5015f4e0 0x7f9f500cd7b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.143:6789/0 says I am v1:172.21.15.143:50866/0 (socket says 172.21.15.143:50866) 2023-11-26T07:18:34.996 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:34.989+0000 7f9f56cc7640 1 -- 172.21.15.143:0/2182227769 learned_addr learned my addr 172.21.15.143:0/2182227769 (peer_addr_for_me v1:172.21.15.143:0/0) 2023-11-26T07:18:34.996 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:34.989+0000 7f9f554c4640 1 -- 172.21.15.143:0/2182227769 <== mon.0 v1:172.21.15.143:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2699121839 0 0) 0x7f9f5006c520 con 0x7f9f5015f4e0 2023-11-26T07:18:34.996 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:34.989+0000 7f9f55cc5640 1 --2- 172.21.15.143:0/2182227769 >> v2:172.21.15.188:3300/0 conn(0x7f9f500cf990 0x7f9f5015efa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:18:34.996 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:34.989+0000 7f9f554c4640 1 -- 172.21.15.143:0/2182227769 --> v1:172.21.15.143:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f9f3c003530 con 0x7f9f5015f4e0 2023-11-26T07:18:34.996 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:34.989+0000 7f9f554c4640 1 -- 172.21.15.143:0/2182227769 <== mon.0 v1:172.21.15.143:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3966534025 0 0) 0x7f9f3c003530 con 0x7f9f5015f4e0 2023-11-26T07:18:34.997 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:34.989+0000 7f9f554c4640 1 -- 172.21.15.143:0/2182227769 >> v2:172.21.15.188:3300/0 conn(0x7f9f500cf990 msgr2=0x7f9f5015efa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-26T07:18:34.997 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:34.989+0000 7f9f554c4640 1 --2- 172.21.15.143:0/2182227769 >> v2:172.21.15.188:3300/0 conn(0x7f9f500cf990 0x7f9f5015efa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:18:34.997 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:34.989+0000 7f9f554c4640 1 -- 172.21.15.143:0/2182227769 >> v1:172.21.15.188:6789/0 conn(0x7f9f5013e290 legacy=0x7f9f500cf280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:18:34.997 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:34.989+0000 7f9f554c4640 1 -- 172.21.15.143:0/2182227769 --> v1:172.21.15.143:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9f5006cdc0 con 0x7f9f5015f4e0 2023-11-26T07:18:34.997 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:34.989+0000 7f9f554c4640 1 -- 172.21.15.143:0/2182227769 <== mon.0 v1:172.21.15.143:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2137882523 0 0) 0x7f9f48001a00 con 0x7f9f5015f4e0 2023-11-26T07:18:34.997 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:34.989+0000 7f9f554c4640 1 -- 172.21.15.143:0/2182227769 >> v1:172.21.15.143:6789/0 conn(0x7f9f5015f4e0 legacy=0x7f9f500cd7b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:18:34.997 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:34.989+0000 7f9f554c4640 1 --2- 172.21.15.143:0/2182227769 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f9f3c0039d0 0x7f9f3c003df0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:18:34.997 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:34.989+0000 7f9f554c4640 1 --2- 172.21.15.143:0/2182227769 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f9f3c0047d0 0x7f9f3c004c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:18:34.997 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:34.989+0000 7f9f554c4640 1 --2- 172.21.15.143:0/2182227769 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f9f3c005190 0x7f9f3c00b990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:18:34.997 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:34.989+0000 7f9f554c4640 1 -- 172.21.15.143:0/2182227769 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9f5006cae0 con 0x7f9f3c0039d0 2023-11-26T07:18:34.997 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:34.989+0000 7f9f554c4640 1 -- 172.21.15.143:0/2182227769 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9f3c00c090 con 0x7f9f3c0047d0 2023-11-26T07:18:34.998 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:34.989+0000 7f9f554c4640 1 -- 172.21.15.143:0/2182227769 --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9f3c00c250 con 0x7f9f3c005190 2023-11-26T07:18:34.998 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:34.989+0000 7f9f564c6640 1 --2- 172.21.15.143:0/2182227769 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f9f3c0039d0 0x7f9f3c003df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:18:34.998 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:34.989+0000 7f9f55cc5640 1 --2- 172.21.15.143:0/2182227769 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f9f3c0047d0 0x7f9f3c004c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:18:34.998 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:34.989+0000 7f9f56cc7640 1 --2- 172.21.15.143:0/2182227769 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f9f3c005190 0x7f9f3c00b990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:18:34.998 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:34.989+0000 7f9f564c6640 1 -- 172.21.15.143:0/2182227769 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f9f3c005190 msgr2=0x7f9f3c00b990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-26T07:18:34.998 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:34.989+0000 7f9f564c6640 1 --2- 172.21.15.143:0/2182227769 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f9f3c005190 0x7f9f3c00b990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:18:34.998 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:34.989+0000 7f9f564c6640 1 -- 172.21.15.143:0/2182227769 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f9f3c0047d0 msgr2=0x7f9f3c004c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-26T07:18:34.999 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:34.989+0000 7f9f564c6640 1 --2- 172.21.15.143:0/2182227769 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f9f3c0047d0 0x7f9f3c004c50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:18:34.999 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:34.989+0000 7f9f564c6640 1 -- 172.21.15.143:0/2182227769 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9f3c003530 con 0x7f9f3c0039d0 2023-11-26T07:18:34.999 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:34.989+0000 7f9f56cc7640 1 --2- 172.21.15.143:0/2182227769 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f9f3c005190 0x7f9f3c00b990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-26T07:18:34.999 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:34.989+0000 7f9f56cc7640 1 -- 172.21.15.143:0/2182227769 reap_dead start 2023-11-26T07:18:35.000 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:34.993+0000 7f9f564c6640 1 --2- 172.21.15.143:0/2182227769 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f9f3c0039d0 0x7f9f3c003df0 secure :-1 s=READY pgs=303 cs=0 l=1 rev1=1 crypto rx=0x7f9f4c001780 tx=0x7f9f4c01c3b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a29a0f53d6631e5 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:18:35.000 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:34.993+0000 7f9f554c4640 1 -- 172.21.15.143:0/2182227769 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9f4c0052d0 con 0x7f9f3c0039d0 2023-11-26T07:18:35.000 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:34.993+0000 7f9f554c4640 1 -- 172.21.15.143:0/2182227769 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9f4c005450 con 0x7f9f3c0039d0 2023-11-26T07:18:35.000 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:34.993+0000 7f9f554c4640 1 -- 172.21.15.143:0/2182227769 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9f4c003840 con 0x7f9f3c0039d0 2023-11-26T07:18:35.000 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:34.993+0000 7f9f5872a640 1 -- 172.21.15.143:0/2182227769 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f9f3c0039d0 msgr2=0x7f9f3c003df0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:18:35.001 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:34.993+0000 7f9f5872a640 1 --2- 172.21.15.143:0/2182227769 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f9f3c0039d0 0x7f9f3c003df0 secure :-1 s=READY pgs=303 cs=0 l=1 rev1=1 crypto rx=0x7f9f4c001780 tx=0x7f9f4c01c3b0 comp rx=0 tx=0).stop 2023-11-26T07:18:35.001 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:34.993+0000 7f9f5872a640 1 -- 172.21.15.143:0/2182227769 shutdown_connections 2023-11-26T07:18:35.001 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:34.993+0000 7f9f5872a640 1 --2- 172.21.15.143:0/2182227769 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f9f3c0039d0 0x7f9f3c003df0 unknown :-1 s=CLOSED pgs=303 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:18:35.001 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:34.993+0000 7f9f5872a640 1 -- 172.21.15.143:0/2182227769 >> 172.21.15.143:0/2182227769 conn(0x7f9f5006a890 msgr2=0x7f9f5006aca0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:18:35.001 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:34.993+0000 7f9f5872a640 1 -- 172.21.15.143:0/2182227769 shutdown_connections 2023-11-26T07:18:35.001 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:34.993+0000 7f9f5872a640 1 -- 172.21.15.143:0/2182227769 wait complete. 2023-11-26T07:18:35.002 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:34.997+0000 7f9f5872a640 1 Processor -- start 2023-11-26T07:18:35.002 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:34.997+0000 7f9f5872a640 1 -- start start 2023-11-26T07:18:35.003 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:34.997+0000 7f9f5872a640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f9f3c0039d0 0x7f9f500af570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:18:35.003 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:34.997+0000 7f9f5872a640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f9f5015e280 0x7f9f500afab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:18:35.003 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:34.997+0000 7f9f5872a640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f9f5015a150 0x7f9f5015a5d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:18:35.003 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:34.997+0000 7f9f5872a640 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9f5015ab10 con 0x7f9f5015a150 2023-11-26T07:18:35.004 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:34.997+0000 7f9f5872a640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9f5014d870 con 0x7f9f3c0039d0 2023-11-26T07:18:35.004 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:34.997+0000 7f9f5872a640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9f500b00e0 con 0x7f9f5015e280 2023-11-26T07:18:35.004 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:34.997+0000 7f9f56cc7640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f9f5015a150 0x7f9f5015a5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:18:35.004 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:34.997+0000 7f9f56cc7640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f9f5015a150 0x7f9f5015a5d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:38792/0 (socket says 172.21.15.143:38792) 2023-11-26T07:18:35.004 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:34.997+0000 7f9f56cc7640 1 -- 172.21.15.143:0/3645671208 learned_addr learned my addr 172.21.15.143:0/3645671208 (peer_addr_for_me v2:172.21.15.143:0/0) 2023-11-26T07:18:35.004 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:34.997+0000 7f9f564c6640 1 --2- 172.21.15.143:0/3645671208 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f9f3c0039d0 0x7f9f500af570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:18:35.004 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:34.997+0000 7f9f55cc5640 1 --2- 172.21.15.143:0/3645671208 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f9f5015e280 0x7f9f500afab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:18:35.004 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:34.997+0000 7f9f56cc7640 1 -- 172.21.15.143:0/3645671208 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f9f5015e280 msgr2=0x7f9f500afab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:18:35.004 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:34.997+0000 7f9f56cc7640 1 --2- 172.21.15.143:0/3645671208 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f9f5015e280 0x7f9f500afab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:18:35.005 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:34.997+0000 7f9f56cc7640 1 -- 172.21.15.143:0/3645671208 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f9f3c0039d0 msgr2=0x7f9f500af570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:18:35.005 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:34.997+0000 7f9f56cc7640 1 --2- 172.21.15.143:0/3645671208 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f9f3c0039d0 0x7f9f500af570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:18:35.005 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:34.997+0000 7f9f56cc7640 1 -- 172.21.15.143:0/3645671208 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9f500b03c0 con 0x7f9f5015a150 2023-11-26T07:18:35.005 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:34.997+0000 7f9f564c6640 1 --2- 172.21.15.143:0/3645671208 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f9f3c0039d0 0x7f9f500af570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-26T07:18:35.005 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:34.997+0000 7f9f56cc7640 1 --2- 172.21.15.143:0/3645671208 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f9f5015a150 0x7f9f5015a5d0 secure :-1 s=READY pgs=304 cs=0 l=1 rev1=1 crypto rx=0x7f9f5015f8c0 tx=0x7f9f4800b4a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:18:35.006 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:34.997+0000 7f9f46ffd640 1 -- 172.21.15.143:0/3645671208 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9f48004d70 con 0x7f9f5015a150 2023-11-26T07:18:35.006 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:34.997+0000 7f9f46ffd640 1 -- 172.21.15.143:0/3645671208 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9f4800bb80 con 0x7f9f5015a150 2023-11-26T07:18:35.006 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:34.997+0000 7f9f46ffd640 1 -- 172.21.15.143:0/3645671208 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9f48007070 con 0x7f9f5015a150 2023-11-26T07:18:35.006 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:34.997+0000 7f9f5872a640 1 -- 172.21.15.143:0/3645671208 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9f50147470 con 0x7f9f5015a150 2023-11-26T07:18:35.006 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:34.997+0000 7f9f5872a640 1 -- 172.21.15.143:0/3645671208 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9f500b06a0 con 0x7f9f5015a150 2023-11-26T07:18:35.006 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:35.001+0000 7f9f5872a640 1 -- 172.21.15.143:0/3645671208 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9f50067680 con 0x7f9f5015a150 2023-11-26T07:18:35.007 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:35.001+0000 7f9f46ffd640 1 -- 172.21.15.143:0/3645671208 <== mon.0 v2:172.21.15.143:3300/0 4 ==== mgrmap(e 5) v1 ==== 85852+0+0 (secure 0 0 0) 0x7f9f480071d0 con 0x7f9f5015a150 2023-11-26T07:18:35.007 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:35.001+0000 7f9f46ffd640 1 --2- 172.21.15.143:0/3645671208 >> [v2:172.21.15.143:6832/2150557544,v1:172.21.15.143:6833/2150557544] conn(0x7f9f34066470 0x7f9f34068930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:18:35.007 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:35.001+0000 7f9f46ffd640 1 -- 172.21.15.143:0/3645671208 <== mon.0 v2:172.21.15.143:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 5825+0+0 (secure 0 0 0) 0x7f9f48084300 con 0x7f9f5015a150 2023-11-26T07:18:35.007 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:35.001+0000 7f9f564c6640 1 --2- 172.21.15.143:0/3645671208 >> [v2:172.21.15.143:6832/2150557544,v1:172.21.15.143:6833/2150557544] conn(0x7f9f34066470 0x7f9f34068930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:18:35.008 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:35.001+0000 7f9f564c6640 1 --2- 172.21.15.143:0/3645671208 >> [v2:172.21.15.143:6832/2150557544,v1:172.21.15.143:6833/2150557544] conn(0x7f9f34066470 0x7f9f34068930 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7f9f4c001750 tx=0x7f9f4c0011a0 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:18:35.010 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:35.005+0000 7f9f46ffd640 1 -- 172.21.15.143:0/3645671208 <== mon.0 v2:172.21.15.143:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172649 (secure 0 0 0) 0x7f9f48051660 con 0x7f9f5015a150 2023-11-26T07:18:35.162 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:35.157+0000 7f9f5872a640 1 -- 172.21.15.143:0/3645671208 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f9f50147920 con 0x7f9f5015a150 2023-11-26T07:18:35.163 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:35.157+0000 7f9f46ffd640 1 -- 172.21.15.143:0/3645671208 <== mon.0 v2:172.21.15.143:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v21) v1 ==== 74+0+16731 (secure 0 0 0) 0x7f9f48054c80 con 0x7f9f5015a150 2023-11-26T07:18:35.163 INFO:teuthology.orchestra.run.smithi143.stdout: 2023-11-26T07:18:35.163 INFO:teuthology.orchestra.run.smithi143.stdout:{"epoch":21,"fsid":"f5033060-2469-4a07-95fe-2b76f5404bc1","created":"2023-11-26T07:17:28.613663+0000","modified":"2023-11-26T07:18:34.698858+0000","last_up_change":"2023-11-26T07:17:41.157813+0000","last_in_change":"2023-11-26T07:17:31.181658+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-11-26T07:17:41.758597+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2023-11-26T07:17:51.878332+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2023-11-26T07:17:52.871396+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"ebbee2c6-023e-44b9-9f1d-6f7bfd309730","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6817","nonce":639807369},{"type":"v1","addr":"172.21.15.143:6819","nonce":639807369}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6821","nonce":639807369},{"type":"v1","addr":"172.21.15.143:6823","nonce":639807369}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6830","nonce":639807369},{"type":"v1","addr":"172.21.15.143:6831","nonce":639807369}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6825","nonce":639807369},{"type":"v1","addr":"172.21.15.143:6828","nonce":639807369}]},"public_addr":"172.21.15.143:6819/639807369","cluster_addr":"172.21.15.143:6823/639807369","heartbeat_back_addr":"172.21.15.143:6831/639807369","heartbeat_front_addr":"172.21.15.143:6828/639807369","state":["exists","up"]},{"osd":1,"uuid":"fec4b8be-7122-49e9-9a19-5850a0700001","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6816","nonce":1144942530},{"type":"v1","addr":"172.21.15.143:6818","nonce":1144942530}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6820","nonce":1144942530},{"type":"v1","addr":"172.21.15.143:6822","nonce":1144942530}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6827","nonce":1144942530},{"type":"v1","addr":"172.21.15.143:6829","nonce":1144942530}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6824","nonce":1144942530},{"type":"v1","addr":"172.21.15.143:6826","nonce":1144942530}]},"public_addr":"172.21.15.143:6818/1144942530","cluster_addr":"172.21.15.143:6822/1144942530","heartbeat_back_addr":"172.21.15.143:6829/1144942530","heartbeat_front_addr":"172.21.15.143:6826/1144942530","state":["exists","up"]},{"osd":2,"uuid":"2888af5b-f933-46ca-8f95-6dd494ff43f6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6808","nonce":3909095186},{"type":"v1","addr":"172.21.15.143:6809","nonce":3909095186}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6810","nonce":3909095186},{"type":"v1","addr":"172.21.15.143:6811","nonce":3909095186}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6814","nonce":3909095186},{"type":"v1","addr":"172.21.15.143:6815","nonce":3909095186}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6812","nonce":3909095186},{"type":"v1","addr":"172.21.15.143:6813","nonce":3909095186}]},"public_addr":"172.21.15.143:6809/3909095186","cluster_addr":"172.21.15.143:6811/3909095186","heartbeat_back_addr":"172.21.15.143:6815/3909095186","heartbeat_front_addr":"172.21.15.143:6813/3909095186","state":["exists","up"]},{"osd":3,"uuid":"d1f52435-7bc1-4081-9550-9cd1ba55f549","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6800","nonce":3854210991},{"type":"v1","addr":"172.21.15.143:6801","nonce":3854210991}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6802","nonce":3854210991},{"type":"v1","addr":"172.21.15.143:6803","nonce":3854210991}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6806","nonce":3854210991},{"type":"v1","addr":"172.21.15.143:6807","nonce":3854210991}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6804","nonce":3854210991},{"type":"v1","addr":"172.21.15.143:6805","nonce":3854210991}]},"public_addr":"172.21.15.143:6801/3854210991","cluster_addr":"172.21.15.143:6803/3854210991","heartbeat_back_addr":"172.21.15.143:6807/3854210991","heartbeat_front_addr":"172.21.15.143:6805/3854210991","state":["exists","up"]},{"osd":4,"uuid":"e2ae8b6e-6e25-49b2-bcbf-aacce68f06a3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6800","nonce":3924606902},{"type":"v1","addr":"172.21.15.188:6801","nonce":3924606902}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6802","nonce":3924606902},{"type":"v1","addr":"172.21.15.188:6803","nonce":3924606902}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6806","nonce":3924606902},{"type":"v1","addr":"172.21.15.188:6807","nonce":3924606902}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6804","nonce":3924606902},{"type":"v1","addr":"172.21.15.188:6805","nonce":3924606902}]},"public_addr":"172.21.15.188:6801/3924606902","cluster_addr":"172.21.15.188:6803/3924606902","heartbeat_back_addr":"172.21.15.188:6807/3924606902","heartbeat_front_addr":"172.21.15.188:6805/3924606902","state":["exists","up"]},{"osd":5,"uuid":"3cec8a32-7698-4c36-8e89-e4ba11934f33","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6813","nonce":186841544},{"type":"v1","addr":"172.21.15.188:6815","nonce":186841544}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6817","nonce":186841544},{"type":"v1","addr":"172.21.15.188:6818","nonce":186841544}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6822","nonce":186841544},{"type":"v1","addr":"172.21.15.188:6823","nonce":186841544}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6820","nonce":186841544},{"type":"v1","addr":"172.21.15.188:6821","nonce":186841544}]},"public_addr":"172.21.15.188:6815/186841544","cluster_addr":"172.21.15.188:6818/186841544","heartbeat_back_addr":"172.21.15.188:6823/186841544","heartbeat_front_addr":"172.21.15.188:6821/186841544","state":["exists","up"]},{"osd":6,"uuid":"58a65d5d-e090-473a-81b4-ad38e85029f9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6808","nonce":2153499405},{"type":"v1","addr":"172.21.15.188:6809","nonce":2153499405}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6810","nonce":2153499405},{"type":"v1","addr":"172.21.15.188:6811","nonce":2153499405}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6816","nonce":2153499405},{"type":"v1","addr":"172.21.15.188:6819","nonce":2153499405}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6812","nonce":2153499405},{"type":"v1","addr":"172.21.15.188:6814","nonce":2153499405}]},"public_addr":"172.21.15.188:6809/2153499405","cluster_addr":"172.21.15.188:6811/2153499405","heartbeat_back_addr":"172.21.15.188:6819/2153499405","heartbeat_front_addr":"172.21.15.188:6814/2153499405","state":["exists","up"]},{"osd":7,"uuid":"ea5b7f70-3554-4cf8-ac30-f7c59b80678d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6824","nonce":2722096729},{"type":"v1","addr":"172.21.15.188:6825","nonce":2722096729}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6826","nonce":2722096729},{"type":"v1","addr":"172.21.15.188:6827","nonce":2722096729}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6830","nonce":2722096729},{"type":"v1","addr":"172.21.15.188:6831","nonce":2722096729}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6828","nonce":2722096729},{"type":"v1","addr":"172.21.15.188:6829","nonce":2722096729}]},"public_addr":"172.21.15.188:6825/2722096729","cluster_addr":"172.21.15.188:6827/2722096729","heartbeat_back_addr":"172.21.15.188:6831/2722096729","heartbeat_front_addr":"172.21.15.188:6829/2722096729","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-26T07:17:36.995032+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-26T07:17:36.932266+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-26T07:17:36.829304+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-26T07:17:36.804297+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-26T07:17:36.667892+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-26T07:17:36.964868+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-26T07:17:36.827332+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-26T07:17:36.893179+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"3.1f","mappings":[{"from":5,"to":1}]},{"pgid":"3.24","mappings":[{"from":3,"to":2}]},{"pgid":"3.26","mappings":[{"from":7,"to":2}]},{"pgid":"3.36","mappings":[{"from":5,"to":2}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-11-26T07:18:35.165 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:35.157+0000 7f9f5872a640 1 -- 172.21.15.143:0/3645671208 >> [v2:172.21.15.143:6832/2150557544,v1:172.21.15.143:6833/2150557544] conn(0x7f9f34066470 msgr2=0x7f9f34068930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:18:35.165 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:35.157+0000 7f9f5872a640 1 --2- 172.21.15.143:0/3645671208 >> [v2:172.21.15.143:6832/2150557544,v1:172.21.15.143:6833/2150557544] conn(0x7f9f34066470 0x7f9f34068930 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7f9f4c001750 tx=0x7f9f4c0011a0 comp rx=0 tx=0).stop 2023-11-26T07:18:35.165 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:35.157+0000 7f9f5872a640 1 -- 172.21.15.143:0/3645671208 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f9f5015a150 msgr2=0x7f9f5015a5d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:18:35.165 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:35.157+0000 7f9f5872a640 1 --2- 172.21.15.143:0/3645671208 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f9f5015a150 0x7f9f5015a5d0 secure :-1 s=READY pgs=304 cs=0 l=1 rev1=1 crypto rx=0x7f9f5015f8c0 tx=0x7f9f4800b4a0 comp rx=0 tx=0).stop 2023-11-26T07:18:35.165 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:35.157+0000 7f9f5872a640 1 -- 172.21.15.143:0/3645671208 shutdown_connections 2023-11-26T07:18:35.165 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:35.157+0000 7f9f5872a640 1 --2- 172.21.15.143:0/3645671208 >> [v2:172.21.15.143:6832/2150557544,v1:172.21.15.143:6833/2150557544] conn(0x7f9f34066470 0x7f9f34068930 unknown :-1 s=CLOSED pgs=173 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:18:35.166 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:35.157+0000 7f9f5872a640 1 --2- 172.21.15.143:0/3645671208 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f9f5015a150 0x7f9f5015a5d0 unknown :-1 s=CLOSED pgs=304 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:18:35.166 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:35.157+0000 7f9f5872a640 1 --2- 172.21.15.143:0/3645671208 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f9f5015e280 0x7f9f500afab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:18:35.166 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:35.157+0000 7f9f5872a640 1 --2- 172.21.15.143:0/3645671208 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f9f3c0039d0 0x7f9f500af570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:18:35.166 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:35.157+0000 7f9f5872a640 1 -- 172.21.15.143:0/3645671208 >> 172.21.15.143:0/3645671208 conn(0x7f9f5006a890 msgr2=0x7f9f50137b50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:18:35.166 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:35.157+0000 7f9f5872a640 1 -- 172.21.15.143:0/3645671208 shutdown_connections 2023-11-26T07:18:35.166 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:35.157+0000 7f9f5872a640 1 -- 172.21.15.143:0/3645671208 wait complete. 2023-11-26T07:18:35.184 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-11-26T07:17:41.758597+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2023-11-26T07:17:51.878332+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2023-11-26T07:17:52.871396+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '20', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2023-11-26T07:18:35.185 DEBUG:teuthology.orchestra.run.smithi143:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-11-26T07:18:35.326 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:35.321+0000 7fc466b1c640 1 Processor -- start 2023-11-26T07:18:35.326 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:35.321+0000 7fc466b1c640 1 -- start start 2023-11-26T07:18:35.328 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:35.321+0000 7fc466b1c640 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7fc460143ac0 0x7fc460070370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:18:35.328 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:35.321+0000 7fc466b1c640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fc4600708b0 0x7fc4600cf280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:18:35.328 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:35.321+0000 7fc466b1c640 1 -- --> v1:172.21.15.143:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc4600cfbf0 con 0x7fc4600cf7c0 2023-11-26T07:18:35.328 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:35.321+0000 7fc466b1c640 1 -- --> v2:172.21.15.143:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc460143ea0 con 0x7fc460143ac0 2023-11-26T07:18:35.328 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:35.321+0000 7fc466b1c640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc4600cc730 con 0x7fc4600708b0 2023-11-26T07:18:35.329 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:35.321+0000 7fc4648b8640 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7fc460143ac0 0x7fc460070370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:18:35.329 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:35.321+0000 7fc4650b9640 1 --1- >> v1:172.21.15.143:6789/0 conn(0x7fc4600cf7c0 0x7fc4600b1ef0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.143:6789/0 says I am v1:172.21.15.143:50878/0 (socket says 172.21.15.143:50878) 2023-11-26T07:18:35.329 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:35.321+0000 7fc4650b9640 1 -- 172.21.15.143:0/369421644 learned_addr learned my addr 172.21.15.143:0/369421644 (peer_addr_for_me v1:172.21.15.143:0/0) 2023-11-26T07:18:35.329 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:35.321+0000 7fc45ffff640 1 --2- 172.21.15.143:0/369421644 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fc4600708b0 0x7fc4600cf280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:18:35.329 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:35.321+0000 7fc4648b8640 1 -- 172.21.15.143:0/369421644 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fc4600708b0 msgr2=0x7fc4600cf280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-26T07:18:35.330 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:35.321+0000 7fc4648b8640 1 --2- 172.21.15.143:0/369421644 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fc4600708b0 0x7fc4600cf280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:18:35.330 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:35.321+0000 7fc4648b8640 1 -- 172.21.15.143:0/369421644 >> v1:172.21.15.143:6789/0 conn(0x7fc4600cf7c0 legacy=0x7fc4600b1ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:18:35.330 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:35.321+0000 7fc4648b8640 1 -- 172.21.15.143:0/369421644 --> v2:172.21.15.143:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc460070ee0 con 0x7fc460143ac0 2023-11-26T07:18:35.331 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:35.325+0000 7fc4648b8640 1 --2- 172.21.15.143:0/369421644 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fc460143ac0 0x7fc460070370 secure :-1 s=READY pgs=306 cs=0 l=1 rev1=1 crypto rx=0x7fc4540097c0 tx=0x7fc45401c480 comp rx=0 tx=0).ready entity=mon.0 client_cookie=27aeeb4e7af9dc32 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:18:35.331 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:35.325+0000 7fc45f7fe640 1 -- 172.21.15.143:0/369421644 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc4540ad640 con 0x7fc460143ac0 2023-11-26T07:18:35.332 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:35.325+0000 7fc45f7fe640 1 -- 172.21.15.143:0/369421644 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc4540ad7a0 con 0x7fc460143ac0 2023-11-26T07:18:35.332 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:35.325+0000 7fc45f7fe640 1 -- 172.21.15.143:0/369421644 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc4540adb30 con 0x7fc460143ac0 2023-11-26T07:18:35.332 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:35.325+0000 7fc466b1c640 1 -- 172.21.15.143:0/369421644 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fc460143ac0 msgr2=0x7fc460070370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:18:35.333 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:35.325+0000 7fc466b1c640 1 --2- 172.21.15.143:0/369421644 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fc460143ac0 0x7fc460070370 secure :-1 s=READY pgs=306 cs=0 l=1 rev1=1 crypto rx=0x7fc4540097c0 tx=0x7fc45401c480 comp rx=0 tx=0).stop 2023-11-26T07:18:35.333 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:35.325+0000 7fc466b1c640 1 -- 172.21.15.143:0/369421644 shutdown_connections 2023-11-26T07:18:35.333 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:35.325+0000 7fc466b1c640 1 --2- 172.21.15.143:0/369421644 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fc4600708b0 0x7fc4600cf280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:18:35.333 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:35.325+0000 7fc466b1c640 1 --2- 172.21.15.143:0/369421644 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fc460143ac0 0x7fc460070370 unknown :-1 s=CLOSED pgs=306 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:18:35.333 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:35.325+0000 7fc466b1c640 1 -- 172.21.15.143:0/369421644 >> 172.21.15.143:0/369421644 conn(0x7fc46006a890 msgr2=0x7fc46006aca0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:18:35.333 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:35.325+0000 7fc466b1c640 1 -- 172.21.15.143:0/369421644 shutdown_connections 2023-11-26T07:18:35.333 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:35.325+0000 7fc466b1c640 1 -- 172.21.15.143:0/369421644 wait complete. 2023-11-26T07:18:35.334 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:35.329+0000 7fc466b1c640 1 Processor -- start 2023-11-26T07:18:35.334 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:35.329+0000 7fc466b1c640 1 -- start start 2023-11-26T07:18:35.336 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:35.329+0000 7fc466b1c640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fc4600708b0 0x7fc4600b1290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:18:35.336 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:35.329+0000 7fc466b1c640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fc4600cf7c0 0x7fc4601510c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:18:35.336 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:35.329+0000 7fc466b1c640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fc460143ac0 0x7fc460135120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:18:35.336 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:35.329+0000 7fc466b1c640 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc460138fb0 con 0x7fc460143ac0 2023-11-26T07:18:35.336 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:35.329+0000 7fc466b1c640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc4600b1870 con 0x7fc4600708b0 2023-11-26T07:18:35.336 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:35.329+0000 7fc466b1c640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc460139200 con 0x7fc4600cf7c0 2023-11-26T07:18:35.336 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:35.329+0000 7fc4650b9640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fc460143ac0 0x7fc460135120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:18:35.336 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:35.329+0000 7fc4650b9640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fc460143ac0 0x7fc460135120 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:38800/0 (socket says 172.21.15.143:38800) 2023-11-26T07:18:35.336 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:35.329+0000 7fc4650b9640 1 -- 172.21.15.143:0/2695952127 learned_addr learned my addr 172.21.15.143:0/2695952127 (peer_addr_for_me v2:172.21.15.143:0/0) 2023-11-26T07:18:35.336 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:35.329+0000 7fc4648b8640 1 --2- 172.21.15.143:0/2695952127 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fc4600708b0 0x7fc4600b1290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:18:35.337 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:35.329+0000 7fc45ffff640 1 --2- 172.21.15.143:0/2695952127 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fc4600cf7c0 0x7fc4601510c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:18:35.337 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:35.329+0000 7fc4650b9640 1 -- 172.21.15.143:0/2695952127 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fc4600cf7c0 msgr2=0x7fc4601510c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:18:35.337 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:35.329+0000 7fc4650b9640 1 --2- 172.21.15.143:0/2695952127 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fc4600cf7c0 0x7fc4601510c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:18:35.337 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:35.329+0000 7fc4650b9640 1 -- 172.21.15.143:0/2695952127 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fc4600708b0 msgr2=0x7fc4600b1290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:18:35.337 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:35.329+0000 7fc4650b9640 1 --2- 172.21.15.143:0/2695952127 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fc4600708b0 0x7fc4600b1290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:18:35.337 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:35.329+0000 7fc4650b9640 1 -- 172.21.15.143:0/2695952127 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc454009420 con 0x7fc460143ac0 2023-11-26T07:18:35.337 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:35.329+0000 7fc4648b8640 1 --2- 172.21.15.143:0/2695952127 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fc4600708b0 0x7fc4600b1290 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-26T07:18:35.338 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:35.329+0000 7fc4650b9640 1 --2- 172.21.15.143:0/2695952127 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fc460143ac0 0x7fc460135120 secure :-1 s=READY pgs=307 cs=0 l=1 rev1=1 crypto rx=0x7fc45800cae0 tx=0x7fc45800a060 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:18:35.338 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:35.329+0000 7fc45d7fa640 1 -- 172.21.15.143:0/2695952127 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc45800ac40 con 0x7fc460143ac0 2023-11-26T07:18:35.338 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:35.329+0000 7fc45d7fa640 1 -- 172.21.15.143:0/2695952127 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc45800ada0 con 0x7fc460143ac0 2023-11-26T07:18:35.338 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:35.329+0000 7fc45d7fa640 1 -- 172.21.15.143:0/2695952127 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc458012620 con 0x7fc460143ac0 2023-11-26T07:18:35.338 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:35.329+0000 7fc466b1c640 1 -- 172.21.15.143:0/2695952127 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc4601394e0 con 0x7fc460143ac0 2023-11-26T07:18:35.338 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:35.329+0000 7fc466b1c640 1 -- 172.21.15.143:0/2695952127 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc460151600 con 0x7fc460143ac0 2023-11-26T07:18:35.338 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:35.333+0000 7fc466b1c640 1 -- 172.21.15.143:0/2695952127 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc460067680 con 0x7fc460143ac0 2023-11-26T07:18:35.339 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:35.333+0000 7fc45d7fa640 1 -- 172.21.15.143:0/2695952127 <== mon.0 v2:172.21.15.143:3300/0 4 ==== mgrmap(e 5) v1 ==== 85852+0+0 (secure 0 0 0) 0x7fc458007070 con 0x7fc460143ac0 2023-11-26T07:18:35.339 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:35.333+0000 7fc45d7fa640 1 --2- 172.21.15.143:0/2695952127 >> [v2:172.21.15.143:6832/2150557544,v1:172.21.15.143:6833/2150557544] conn(0x7fc444066470 0x7fc444068930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:18:35.339 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:35.333+0000 7fc45d7fa640 1 -- 172.21.15.143:0/2695952127 <== mon.0 v2:172.21.15.143:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 5825+0+0 (secure 0 0 0) 0x7fc458083e00 con 0x7fc460143ac0 2023-11-26T07:18:35.340 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:35.333+0000 7fc4648b8640 1 --2- 172.21.15.143:0/2695952127 >> [v2:172.21.15.143:6832/2150557544,v1:172.21.15.143:6833/2150557544] conn(0x7fc444066470 0x7fc444068930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:18:35.340 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:35.333+0000 7fc4648b8640 1 --2- 172.21.15.143:0/2695952127 >> [v2:172.21.15.143:6832/2150557544,v1:172.21.15.143:6833/2150557544] conn(0x7fc444066470 0x7fc444068930 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7fc4540af000 tx=0x7fc454008040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:18:35.343 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:35.337+0000 7fc45d7fa640 1 -- 172.21.15.143:0/2695952127 <== mon.0 v2:172.21.15.143:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172649 (secure 0 0 0) 0x7fc458051270 con 0x7fc460143ac0 2023-11-26T07:18:35.493 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:35.489+0000 7fc466b1c640 1 -- 172.21.15.143:0/2695952127 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7fc4600b21e0 con 0x7fc460143ac0 2023-11-26T07:18:35.493 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:35.489+0000 7fc45d7fa640 1 -- 172.21.15.143:0/2695952127 <== mon.0 v2:172.21.15.143:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v21) v1 ==== 93+0+10 (secure 0 0 0) 0x7fc4580170a0 con 0x7fc460143ac0 2023-11-26T07:18:35.494 INFO:teuthology.orchestra.run.smithi143.stdout:pg_num: 1 2023-11-26T07:18:35.496 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:35.489+0000 7fc466b1c640 1 -- 172.21.15.143:0/2695952127 >> [v2:172.21.15.143:6832/2150557544,v1:172.21.15.143:6833/2150557544] conn(0x7fc444066470 msgr2=0x7fc444068930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:18:35.496 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:35.489+0000 7fc466b1c640 1 --2- 172.21.15.143:0/2695952127 >> [v2:172.21.15.143:6832/2150557544,v1:172.21.15.143:6833/2150557544] conn(0x7fc444066470 0x7fc444068930 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7fc4540af000 tx=0x7fc454008040 comp rx=0 tx=0).stop 2023-11-26T07:18:35.496 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:35.489+0000 7fc466b1c640 1 -- 172.21.15.143:0/2695952127 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fc460143ac0 msgr2=0x7fc460135120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:18:35.496 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:35.489+0000 7fc466b1c640 1 --2- 172.21.15.143:0/2695952127 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fc460143ac0 0x7fc460135120 secure :-1 s=READY pgs=307 cs=0 l=1 rev1=1 crypto rx=0x7fc45800cae0 tx=0x7fc45800a060 comp rx=0 tx=0).stop 2023-11-26T07:18:35.496 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:35.489+0000 7fc466b1c640 1 -- 172.21.15.143:0/2695952127 shutdown_connections 2023-11-26T07:18:35.496 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:35.489+0000 7fc466b1c640 1 --2- 172.21.15.143:0/2695952127 >> [v2:172.21.15.143:6832/2150557544,v1:172.21.15.143:6833/2150557544] conn(0x7fc444066470 0x7fc444068930 unknown :-1 s=CLOSED pgs=174 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:18:35.496 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:35.489+0000 7fc466b1c640 1 --2- 172.21.15.143:0/2695952127 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fc460143ac0 0x7fc460135120 unknown :-1 s=CLOSED pgs=307 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:18:35.496 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:35.489+0000 7fc466b1c640 1 --2- 172.21.15.143:0/2695952127 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fc4600cf7c0 0x7fc4601510c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:18:35.496 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:35.489+0000 7fc466b1c640 1 --2- 172.21.15.143:0/2695952127 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fc4600708b0 0x7fc4600b1290 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:18:35.496 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:35.489+0000 7fc466b1c640 1 -- 172.21.15.143:0/2695952127 >> 172.21.15.143:0/2695952127 conn(0x7fc46006a890 msgr2=0x7fc46006ac70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:18:35.497 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:35.489+0000 7fc466b1c640 1 -- 172.21.15.143:0/2695952127 shutdown_connections 2023-11-26T07:18:35.497 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:35.489+0000 7fc466b1c640 1 -- 172.21.15.143:0/2695952127 wait complete. 2023-11-26T07:18:35.515 DEBUG:teuthology.orchestra.run.smithi143:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-11-26T07:18:35.656 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:35.649+0000 7fbf0f4ad640 1 Processor -- start 2023-11-26T07:18:35.656 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:35.649+0000 7fbf0f4ad640 1 -- start start 2023-11-26T07:18:35.658 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:35.653+0000 7fbf0f4ad640 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7fbf08144720 0x7fbf0806d300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:18:35.658 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:35.653+0000 7fbf0f4ad640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fbf0806d840 0x7fbf080714e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:18:35.658 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:35.653+0000 7fbf0f4ad640 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7fbf08071a20 0x7fbf080cc170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:18:35.658 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:35.653+0000 7fbf0f4ad640 1 -- --> v2:172.21.15.143:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fbf08071e90 con 0x7fbf08144720 2023-11-26T07:18:35.658 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:35.653+0000 7fbf0f4ad640 1 -- --> v2:172.21.15.188:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fbf0806e3f0 con 0x7fbf08071a20 2023-11-26T07:18:35.658 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:35.653+0000 7fbf0f4ad640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbf080722f0 con 0x7fbf0806d840 2023-11-26T07:18:35.658 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:35.653+0000 7fbf0e4ab640 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7fbf08144720 0x7fbf0806d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:18:35.659 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:35.653+0000 7fbf0e4ab640 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7fbf08144720 0x7fbf0806d300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:38816/0 (socket says 172.21.15.143:38816) 2023-11-26T07:18:35.659 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:35.653+0000 7fbf0e4ab640 1 -- 172.21.15.143:0/1663195395 learned_addr learned my addr 172.21.15.143:0/1663195395 (peer_addr_for_me v2:172.21.15.143:0/0) 2023-11-26T07:18:35.659 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:35.653+0000 7fbf0ecac640 1 --2- 172.21.15.143:0/1663195395 >> v2:172.21.15.188:3300/0 conn(0x7fbf08071a20 0x7fbf080cc170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:18:35.659 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:35.653+0000 7fbf0dcaa640 1 --2- 172.21.15.143:0/1663195395 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fbf0806d840 0x7fbf080714e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:18:35.659 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:35.653+0000 7fbf0e4ab640 1 -- 172.21.15.143:0/1663195395 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fbf0806d840 msgr2=0x7fbf080714e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-26T07:18:35.659 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:35.653+0000 7fbf0e4ab640 1 --2- 172.21.15.143:0/1663195395 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fbf0806d840 0x7fbf080714e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:18:35.659 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:35.653+0000 7fbf0e4ab640 1 -- 172.21.15.143:0/1663195395 >> v2:172.21.15.188:3300/0 conn(0x7fbf08071a20 msgr2=0x7fbf080cc170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-26T07:18:35.659 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:35.653+0000 7fbf0e4ab640 1 --2- 172.21.15.143:0/1663195395 >> v2:172.21.15.188:3300/0 conn(0x7fbf08071a20 0x7fbf080cc170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:18:35.659 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:35.653+0000 7fbf0e4ab640 1 -- 172.21.15.143:0/1663195395 --> v2:172.21.15.143:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbf0806de30 con 0x7fbf08144720 2023-11-26T07:18:35.660 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:35.657+0000 7fbf0e4ab640 1 --2- 172.21.15.143:0/1663195395 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fbf08144720 0x7fbf0806d300 secure :-1 s=READY pgs=308 cs=0 l=1 rev1=1 crypto rx=0x7fbf080b1ec0 tx=0x7fbf0401c210 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e585192d745f4d38 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:18:35.661 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:35.657+0000 7fbf0d4a9640 1 -- 172.21.15.143:0/1663195395 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbf040ab600 con 0x7fbf08144720 2023-11-26T07:18:35.661 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:35.657+0000 7fbf0d4a9640 1 -- 172.21.15.143:0/1663195395 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbf040a7050 con 0x7fbf08144720 2023-11-26T07:18:35.661 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:35.657+0000 7fbf0d4a9640 1 -- 172.21.15.143:0/1663195395 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbf040ab980 con 0x7fbf08144720 2023-11-26T07:18:35.662 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:35.657+0000 7fbf0f4ad640 1 -- 172.21.15.143:0/1663195395 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fbf08144720 msgr2=0x7fbf0806d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:18:35.662 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:35.657+0000 7fbf0f4ad640 1 --2- 172.21.15.143:0/1663195395 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fbf08144720 0x7fbf0806d300 secure :-1 s=READY pgs=308 cs=0 l=1 rev1=1 crypto rx=0x7fbf080b1ec0 tx=0x7fbf0401c210 comp rx=0 tx=0).stop 2023-11-26T07:18:35.662 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:35.657+0000 7fbf0f4ad640 1 -- 172.21.15.143:0/1663195395 shutdown_connections 2023-11-26T07:18:35.662 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:35.657+0000 7fbf0f4ad640 1 --2- 172.21.15.143:0/1663195395 >> v2:172.21.15.188:3300/0 conn(0x7fbf08071a20 0x7fbf080cc170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:18:35.662 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:35.657+0000 7fbf0f4ad640 1 --2- 172.21.15.143:0/1663195395 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fbf0806d840 0x7fbf080714e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:18:35.662 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:35.657+0000 7fbf0f4ad640 1 --2- 172.21.15.143:0/1663195395 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fbf08144720 0x7fbf0806d300 unknown :-1 s=CLOSED pgs=308 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:18:35.662 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:35.657+0000 7fbf0f4ad640 1 -- 172.21.15.143:0/1663195395 >> 172.21.15.143:0/1663195395 conn(0x7fbf0806a890 msgr2=0x7fbf0806aca0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:18:35.663 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:35.657+0000 7fbf0f4ad640 1 -- 172.21.15.143:0/1663195395 shutdown_connections 2023-11-26T07:18:35.663 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:35.657+0000 7fbf0f4ad640 1 -- 172.21.15.143:0/1663195395 wait complete. 2023-11-26T07:18:35.663 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:35.657+0000 7fbf0f4ad640 1 Processor -- start 2023-11-26T07:18:35.663 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:35.657+0000 7fbf0f4ad640 1 -- start start 2023-11-26T07:18:35.665 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:35.657+0000 7fbf0f4ad640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fbf0806d840 0x7fbf0813a5d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:18:35.665 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:35.657+0000 7fbf0f4ad640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fbf08071a20 0x7fbf080b3330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:18:35.665 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:35.657+0000 7fbf0f4ad640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fbf08144720 0x7fbf0813c000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:18:35.665 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:35.657+0000 7fbf0f4ad640 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbf08143340 con 0x7fbf08144720 2023-11-26T07:18:35.665 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:35.657+0000 7fbf0f4ad640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbf081426c0 con 0x7fbf08071a20 2023-11-26T07:18:35.665 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:35.657+0000 7fbf0f4ad640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbf081429a0 con 0x7fbf0806d840 2023-11-26T07:18:35.666 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:35.661+0000 7fbf0ecac640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fbf08144720 0x7fbf0813c000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:18:35.666 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:35.661+0000 7fbf0ecac640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fbf08144720 0x7fbf0813c000 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:38826/0 (socket says 172.21.15.143:38826) 2023-11-26T07:18:35.666 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:35.661+0000 7fbf0ecac640 1 -- 172.21.15.143:0/3342913142 learned_addr learned my addr 172.21.15.143:0/3342913142 (peer_addr_for_me v2:172.21.15.143:0/0) 2023-11-26T07:18:35.666 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:35.661+0000 7fbf0e4ab640 1 --2- 172.21.15.143:0/3342913142 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fbf0806d840 0x7fbf0813a5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:18:35.666 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:35.661+0000 7fbf0dcaa640 1 --2- 172.21.15.143:0/3342913142 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fbf08071a20 0x7fbf080b3330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:18:35.666 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:35.661+0000 7fbf0ecac640 1 -- 172.21.15.143:0/3342913142 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fbf0806d840 msgr2=0x7fbf0813a5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:18:35.666 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:35.661+0000 7fbf0ecac640 1 --2- 172.21.15.143:0/3342913142 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fbf0806d840 0x7fbf0813a5d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:18:35.666 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:35.661+0000 7fbf0ecac640 1 -- 172.21.15.143:0/3342913142 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fbf08071a20 msgr2=0x7fbf080b3330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:18:35.666 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:35.661+0000 7fbf0ecac640 1 --2- 172.21.15.143:0/3342913142 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fbf08071a20 0x7fbf080b3330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:18:35.666 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:35.661+0000 7fbf0ecac640 1 -- 172.21.15.143:0/3342913142 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbf040094d0 con 0x7fbf08144720 2023-11-26T07:18:35.667 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:35.661+0000 7fbf0ecac640 1 --2- 172.21.15.143:0/3342913142 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fbf08144720 0x7fbf0813c000 secure :-1 s=READY pgs=309 cs=0 l=1 rev1=1 crypto rx=0x7fbf00007e40 tx=0x7fbf0000e4b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:18:35.667 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:35.661+0000 7fbefeffd640 1 -- 172.21.15.143:0/3342913142 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbf00014620 con 0x7fbf08144720 2023-11-26T07:18:35.667 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:35.661+0000 7fbefeffd640 1 -- 172.21.15.143:0/3342913142 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbf0000ee90 con 0x7fbf08144720 2023-11-26T07:18:35.667 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:35.661+0000 7fbefeffd640 1 -- 172.21.15.143:0/3342913142 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbf000149e0 con 0x7fbf08144720 2023-11-26T07:18:35.667 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:35.661+0000 7fbf0f4ad640 1 -- 172.21.15.143:0/3342913142 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbf08142c80 con 0x7fbf08144720 2023-11-26T07:18:35.668 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:35.661+0000 7fbf0f4ad640 1 -- 172.21.15.143:0/3342913142 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbf08142350 con 0x7fbf08144720 2023-11-26T07:18:35.668 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:35.661+0000 7fbf0f4ad640 1 -- 172.21.15.143:0/3342913142 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbf08143480 con 0x7fbf08144720 2023-11-26T07:18:35.668 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:35.661+0000 7fbefeffd640 1 -- 172.21.15.143:0/3342913142 <== mon.0 v2:172.21.15.143:3300/0 4 ==== mgrmap(e 5) v1 ==== 85852+0+0 (secure 0 0 0) 0x7fbf00014b40 con 0x7fbf08144720 2023-11-26T07:18:35.669 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:35.665+0000 7fbefeffd640 1 --2- 172.21.15.143:0/3342913142 >> [v2:172.21.15.143:6832/2150557544,v1:172.21.15.143:6833/2150557544] conn(0x7fbeec066470 0x7fbeec068930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:18:35.669 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:35.665+0000 7fbefeffd640 1 -- 172.21.15.143:0/3342913142 <== mon.0 v2:172.21.15.143:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 5825+0+0 (secure 0 0 0) 0x7fbf00084700 con 0x7fbf08144720 2023-11-26T07:18:35.669 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:35.665+0000 7fbf0e4ab640 1 --2- 172.21.15.143:0/3342913142 >> [v2:172.21.15.143:6832/2150557544,v1:172.21.15.143:6833/2150557544] conn(0x7fbeec066470 0x7fbeec068930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:18:35.670 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:35.665+0000 7fbf0e4ab640 1 --2- 172.21.15.143:0/3342913142 >> [v2:172.21.15.143:6832/2150557544,v1:172.21.15.143:6833/2150557544] conn(0x7fbeec066470 0x7fbeec068930 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7fbf040ad000 tx=0x7fbf040a6000 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:18:35.673 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:35.669+0000 7fbefeffd640 1 -- 172.21.15.143:0/3342913142 <== mon.0 v2:172.21.15.143:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172649 (secure 0 0 0) 0x7fbf08143480 con 0x7fbf08144720 2023-11-26T07:18:35.829 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:35.825+0000 7fbf0f4ad640 1 -- 172.21.15.143:0/3342913142 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7fbf08143480 con 0x7fbf08144720 2023-11-26T07:18:35.830 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:35.825+0000 7fbefeffd640 1 -- 172.21.15.143:0/3342913142 <== mon.0 v2:172.21.15.143:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v22) v1 ==== 104+0+11 (secure 0 0 0) 0x7fbf08143480 con 0x7fbf08144720 2023-11-26T07:18:35.830 INFO:teuthology.orchestra.run.smithi143.stdout:pg_num: 32 2023-11-26T07:18:35.832 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:35.825+0000 7fbf0f4ad640 1 -- 172.21.15.143:0/3342913142 >> [v2:172.21.15.143:6832/2150557544,v1:172.21.15.143:6833/2150557544] conn(0x7fbeec066470 msgr2=0x7fbeec068930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:18:35.832 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:35.825+0000 7fbf0f4ad640 1 --2- 172.21.15.143:0/3342913142 >> [v2:172.21.15.143:6832/2150557544,v1:172.21.15.143:6833/2150557544] conn(0x7fbeec066470 0x7fbeec068930 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7fbf040ad000 tx=0x7fbf040a6000 comp rx=0 tx=0).stop 2023-11-26T07:18:35.832 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:35.825+0000 7fbf0f4ad640 1 -- 172.21.15.143:0/3342913142 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fbf08144720 msgr2=0x7fbf0813c000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:18:35.832 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:35.825+0000 7fbf0f4ad640 1 --2- 172.21.15.143:0/3342913142 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fbf08144720 0x7fbf0813c000 secure :-1 s=READY pgs=309 cs=0 l=1 rev1=1 crypto rx=0x7fbf00007e40 tx=0x7fbf0000e4b0 comp rx=0 tx=0).stop 2023-11-26T07:18:35.832 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:35.825+0000 7fbf0f4ad640 1 -- 172.21.15.143:0/3342913142 shutdown_connections 2023-11-26T07:18:35.832 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:35.825+0000 7fbf0f4ad640 1 --2- 172.21.15.143:0/3342913142 >> [v2:172.21.15.143:6832/2150557544,v1:172.21.15.143:6833/2150557544] conn(0x7fbeec066470 0x7fbeec068930 unknown :-1 s=CLOSED pgs=175 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:18:35.833 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:35.825+0000 7fbf0f4ad640 1 --2- 172.21.15.143:0/3342913142 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fbf08144720 0x7fbf0813c000 unknown :-1 s=CLOSED pgs=309 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:18:35.833 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:35.825+0000 7fbf0f4ad640 1 --2- 172.21.15.143:0/3342913142 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fbf08071a20 0x7fbf080b3330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:18:35.833 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:35.825+0000 7fbf0f4ad640 1 --2- 172.21.15.143:0/3342913142 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fbf0806d840 0x7fbf0813a5d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:18:35.833 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:35.825+0000 7fbf0f4ad640 1 -- 172.21.15.143:0/3342913142 >> 172.21.15.143:0/3342913142 conn(0x7fbf0806a890 msgr2=0x7fbf080afe50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:18:35.833 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:35.825+0000 7fbf0f4ad640 1 -- 172.21.15.143:0/3342913142 shutdown_connections 2023-11-26T07:18:35.833 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:35.825+0000 7fbf0f4ad640 1 -- 172.21.15.143:0/3342913142 wait complete. 2023-11-26T07:18:35.851 DEBUG:teuthology.orchestra.run.smithi143:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-11-26T07:18:35.991 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:35.985+0000 7f212ba6b640 1 Processor -- start 2023-11-26T07:18:35.991 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:35.985+0000 7f212ba6b640 1 -- start start 2023-11-26T07:18:35.992 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:35.985+0000 7f212ba6b640 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7f21240d1bc0 0x7f21240d0df0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:18:35.992 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:35.985+0000 7f212ba6b640 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7f21240b2160 0x7f21240b0750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:18:35.992 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:35.985+0000 7f212ba6b640 1 -- --> v1:172.21.15.143:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f21240cf0e0 con 0x7f212415d6e0 2023-11-26T07:18:35.992 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:35.985+0000 7f212ba6b640 1 -- --> v2:172.21.15.143:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f212415dac0 con 0x7f21240d1bc0 2023-11-26T07:18:35.992 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:35.985+0000 7f212ba6b640 1 -- --> v2:172.21.15.188:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f21240ceb20 con 0x7f21240b2160 2023-11-26T07:18:35.992 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:35.985+0000 7f2129807640 1 --1- >> v1:172.21.15.143:6789/0 conn(0x7f212415d6e0 0x7f21240d14b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.143:6789/0 says I am v1:172.21.15.143:50894/0 (socket says 172.21.15.143:50894) 2023-11-26T07:18:35.992 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:35.985+0000 7f2129807640 1 -- 172.21.15.143:0/1241246246 learned_addr learned my addr 172.21.15.143:0/1241246246 (peer_addr_for_me v1:172.21.15.143:0/0) 2023-11-26T07:18:35.992 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:35.985+0000 7f2129006640 1 --2- 172.21.15.143:0/1241246246 >> v2:172.21.15.143:3300/0 conn(0x7f21240d1bc0 0x7f21240d0df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:18:35.993 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:35.985+0000 7f212a008640 1 --2- 172.21.15.143:0/1241246246 >> v2:172.21.15.188:3300/0 conn(0x7f21240b2160 0x7f21240b0750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:18:35.993 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:35.985+0000 7f2129006640 1 -- 172.21.15.143:0/1241246246 >> v2:172.21.15.188:3300/0 conn(0x7f21240b2160 msgr2=0x7f21240b0750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-26T07:18:35.993 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:35.985+0000 7f2129006640 1 --2- 172.21.15.143:0/1241246246 >> v2:172.21.15.188:3300/0 conn(0x7f21240b2160 0x7f21240b0750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:18:35.993 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:35.985+0000 7f2129006640 1 -- 172.21.15.143:0/1241246246 >> v1:172.21.15.143:6789/0 conn(0x7f212415d6e0 legacy=0x7f21240d14b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:18:35.993 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:35.985+0000 7f2129006640 1 -- 172.21.15.143:0/1241246246 --> v2:172.21.15.143:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f21240cee00 con 0x7f21240d1bc0 2023-11-26T07:18:35.993 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:35.985+0000 7f212a008640 1 --2- 172.21.15.143:0/1241246246 >> v2:172.21.15.188:3300/0 conn(0x7f21240b2160 0x7f21240b0750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-26T07:18:35.994 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:35.985+0000 7f2128805640 1 -- 172.21.15.143:0/1241246246 <== mon.0 v1:172.21.15.143:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4030728879 0 0) 0x7f21240cf0e0 con 0x7f212415d6e0 2023-11-26T07:18:35.995 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:35.989+0000 7f2129006640 1 --2- 172.21.15.143:0/1241246246 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f21240d1bc0 0x7f21240d0df0 secure :-1 s=READY pgs=311 cs=0 l=1 rev1=1 crypto rx=0x7f212408d820 tx=0x7f211401c460 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fbd255386c6c6046 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:18:35.995 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:35.989+0000 7f2128805640 1 -- 172.21.15.143:0/1241246246 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2114004230 con 0x7f21240d1bc0 2023-11-26T07:18:35.996 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:35.989+0000 7f2128805640 1 -- 172.21.15.143:0/1241246246 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f21140a7050 con 0x7f21240d1bc0 2023-11-26T07:18:35.996 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:35.989+0000 7f2128805640 1 -- 172.21.15.143:0/1241246246 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2114003620 con 0x7f21240d1bc0 2023-11-26T07:18:35.996 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:35.989+0000 7f212ba6b640 1 -- 172.21.15.143:0/1241246246 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f21240d1bc0 msgr2=0x7f21240d0df0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:18:35.996 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:35.989+0000 7f212ba6b640 1 --2- 172.21.15.143:0/1241246246 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f21240d1bc0 0x7f21240d0df0 secure :-1 s=READY pgs=311 cs=0 l=1 rev1=1 crypto rx=0x7f212408d820 tx=0x7f211401c460 comp rx=0 tx=0).stop 2023-11-26T07:18:35.997 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:35.989+0000 7f212ba6b640 1 -- 172.21.15.143:0/1241246246 shutdown_connections 2023-11-26T07:18:35.997 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:35.989+0000 7f212ba6b640 1 --2- 172.21.15.143:0/1241246246 >> v2:172.21.15.188:3300/0 conn(0x7f21240b2160 0x7f21240b0750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:18:35.997 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:35.989+0000 7f212ba6b640 1 --2- 172.21.15.143:0/1241246246 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f21240d1bc0 0x7f21240d0df0 unknown :-1 s=CLOSED pgs=311 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:18:35.997 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:35.989+0000 7f212ba6b640 1 -- 172.21.15.143:0/1241246246 >> 172.21.15.143:0/1241246246 conn(0x7f212409c550 msgr2=0x7f21240d3220 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:18:35.997 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:35.989+0000 7f212ba6b640 1 -- 172.21.15.143:0/1241246246 shutdown_connections 2023-11-26T07:18:35.997 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:35.989+0000 7f212ba6b640 1 -- 172.21.15.143:0/1241246246 wait complete. 2023-11-26T07:18:35.998 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:35.993+0000 7f212ba6b640 1 Processor -- start 2023-11-26T07:18:35.998 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:35.993+0000 7f212ba6b640 1 -- start start 2023-11-26T07:18:36.000 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:35.993+0000 7f212ba6b640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f21240b2160 0x7f2124155760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:18:36.000 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:35.993+0000 7f212ba6b640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f21240d1bc0 0x7f212415c130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:18:36.000 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:35.993+0000 7f212ba6b640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f212415d6e0 0x7f21240b7ee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:18:36.000 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:35.993+0000 7f212ba6b640 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f21240b6930 con 0x7f212415d6e0 2023-11-26T07:18:36.000 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:35.993+0000 7f212ba6b640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2124155ca0 con 0x7f21240b2160 2023-11-26T07:18:36.000 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:35.993+0000 7f212ba6b640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f21240b7140 con 0x7f21240d1bc0 2023-11-26T07:18:36.000 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:35.993+0000 7f212a008640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f212415d6e0 0x7f21240b7ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:18:36.001 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:35.993+0000 7f212a008640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f212415d6e0 0x7f21240b7ee0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:38848/0 (socket says 172.21.15.143:38848) 2023-11-26T07:18:36.001 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:35.993+0000 7f212a008640 1 -- 172.21.15.143:0/2884002298 learned_addr learned my addr 172.21.15.143:0/2884002298 (peer_addr_for_me v2:172.21.15.143:0/0) 2023-11-26T07:18:36.001 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:35.993+0000 7f2129807640 1 --2- 172.21.15.143:0/2884002298 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f21240b2160 0x7f2124155760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:18:36.001 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:35.993+0000 7f2129006640 1 --2- 172.21.15.143:0/2884002298 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f21240d1bc0 0x7f212415c130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:18:36.001 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:35.993+0000 7f212a008640 1 -- 172.21.15.143:0/2884002298 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f21240d1bc0 msgr2=0x7f212415c130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:18:36.001 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:35.993+0000 7f212a008640 1 --2- 172.21.15.143:0/2884002298 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f21240d1bc0 0x7f212415c130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:18:36.001 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:35.993+0000 7f212a008640 1 -- 172.21.15.143:0/2884002298 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f21240b2160 msgr2=0x7f2124155760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:18:36.001 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:35.993+0000 7f212a008640 1 --2- 172.21.15.143:0/2884002298 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f21240b2160 0x7f2124155760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:18:36.001 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:35.993+0000 7f212a008640 1 -- 172.21.15.143:0/2884002298 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2114009420 con 0x7f212415d6e0 2023-11-26T07:18:36.001 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:35.993+0000 7f212a008640 1 --2- 172.21.15.143:0/2884002298 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f212415d6e0 0x7f21240b7ee0 secure :-1 s=READY pgs=312 cs=0 l=1 rev1=1 crypto rx=0x7f211c00cea0 tx=0x7f211c00cf90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:18:36.002 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:35.993+0000 7f211a7fc640 1 -- 172.21.15.143:0/2884002298 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f211c003ba0 con 0x7f212415d6e0 2023-11-26T07:18:36.002 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:35.993+0000 7f211a7fc640 1 -- 172.21.15.143:0/2884002298 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f211c003d20 con 0x7f212415d6e0 2023-11-26T07:18:36.002 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:35.993+0000 7f211a7fc640 1 -- 172.21.15.143:0/2884002298 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f211c00d880 con 0x7f212415d6e0 2023-11-26T07:18:36.002 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:35.993+0000 7f212ba6b640 1 -- 172.21.15.143:0/2884002298 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f21240b6b80 con 0x7f212415d6e0 2023-11-26T07:18:36.003 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:35.997+0000 7f212ba6b640 1 -- 172.21.15.143:0/2884002298 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f212415ede0 con 0x7f212415d6e0 2023-11-26T07:18:36.003 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:35.997+0000 7f212ba6b640 1 -- 172.21.15.143:0/2884002298 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f212415ec40 con 0x7f212415d6e0 2023-11-26T07:18:36.003 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:35.997+0000 7f211a7fc640 1 -- 172.21.15.143:0/2884002298 <== mon.0 v2:172.21.15.143:3300/0 4 ==== mgrmap(e 5) v1 ==== 85852+0+0 (secure 0 0 0) 0x7f211c00d9e0 con 0x7f212415d6e0 2023-11-26T07:18:36.003 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:35.997+0000 7f211a7fc640 1 --2- 172.21.15.143:0/2884002298 >> [v2:172.21.15.143:6832/2150557544,v1:172.21.15.143:6833/2150557544] conn(0x7f2108066470 0x7f2108068930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:18:36.003 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:35.997+0000 7f211a7fc640 1 -- 172.21.15.143:0/2884002298 <== mon.0 v2:172.21.15.143:3300/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 5825+0+0 (secure 0 0 0) 0x7f211c084820 con 0x7f212415d6e0 2023-11-26T07:18:36.003 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:35.997+0000 7f2129807640 1 --2- 172.21.15.143:0/2884002298 >> [v2:172.21.15.143:6832/2150557544,v1:172.21.15.143:6833/2150557544] conn(0x7f2108066470 0x7f2108068930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:18:36.004 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:35.997+0000 7f2129807640 1 --2- 172.21.15.143:0/2884002298 >> [v2:172.21.15.143:6832/2150557544,v1:172.21.15.143:6833/2150557544] conn(0x7f2108066470 0x7f2108068930 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7f2120000f70 tx=0x7f2120007df0 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:18:36.007 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:36.001+0000 7f211a7fc640 1 -- 172.21.15.143:0/2884002298 <== mon.0 v2:172.21.15.143:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172649 (secure 0 0 0) 0x7f212415ec40 con 0x7f212415d6e0 2023-11-26T07:18:36.156 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:36.149+0000 7f212ba6b640 1 -- 172.21.15.143:0/2884002298 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f212415ec40 con 0x7f212415d6e0 2023-11-26T07:18:36.156 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:36.149+0000 7f211a7fc640 1 -- 172.21.15.143:0/2884002298 <== mon.0 v2:172.21.15.143:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v22) v1 ==== 100+0+11 (secure 0 0 0) 0x7f212415ec40 con 0x7f212415d6e0 2023-11-26T07:18:36.156 INFO:teuthology.orchestra.run.smithi143.stdout:pg_num: 64 2023-11-26T07:18:36.158 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:36.153+0000 7f212ba6b640 1 -- 172.21.15.143:0/2884002298 >> [v2:172.21.15.143:6832/2150557544,v1:172.21.15.143:6833/2150557544] conn(0x7f2108066470 msgr2=0x7f2108068930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:18:36.159 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:36.153+0000 7f212ba6b640 1 --2- 172.21.15.143:0/2884002298 >> [v2:172.21.15.143:6832/2150557544,v1:172.21.15.143:6833/2150557544] conn(0x7f2108066470 0x7f2108068930 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7f2120000f70 tx=0x7f2120007df0 comp rx=0 tx=0).stop 2023-11-26T07:18:36.159 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:36.153+0000 7f212ba6b640 1 -- 172.21.15.143:0/2884002298 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f212415d6e0 msgr2=0x7f21240b7ee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:18:36.159 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:36.153+0000 7f212ba6b640 1 --2- 172.21.15.143:0/2884002298 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f212415d6e0 0x7f21240b7ee0 secure :-1 s=READY pgs=312 cs=0 l=1 rev1=1 crypto rx=0x7f211c00cea0 tx=0x7f211c00cf90 comp rx=0 tx=0).stop 2023-11-26T07:18:36.159 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:36.153+0000 7f212ba6b640 1 -- 172.21.15.143:0/2884002298 shutdown_connections 2023-11-26T07:18:36.159 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:36.153+0000 7f212ba6b640 1 --2- 172.21.15.143:0/2884002298 >> [v2:172.21.15.143:6832/2150557544,v1:172.21.15.143:6833/2150557544] conn(0x7f2108066470 0x7f2108068930 unknown :-1 s=CLOSED pgs=176 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:18:36.159 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:36.153+0000 7f212ba6b640 1 --2- 172.21.15.143:0/2884002298 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f212415d6e0 0x7f21240b7ee0 unknown :-1 s=CLOSED pgs=312 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:18:36.159 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:36.153+0000 7f212ba6b640 1 --2- 172.21.15.143:0/2884002298 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f21240d1bc0 0x7f212415c130 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:18:36.159 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:36.153+0000 7f212ba6b640 1 --2- 172.21.15.143:0/2884002298 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f21240b2160 0x7f2124155760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:18:36.159 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:36.153+0000 7f212ba6b640 1 -- 172.21.15.143:0/2884002298 >> 172.21.15.143:0/2884002298 conn(0x7f212409c550 msgr2=0x7f21240734a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:18:36.159 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:36.153+0000 7f212ba6b640 1 -- 172.21.15.143:0/2884002298 shutdown_connections 2023-11-26T07:18:36.159 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:36.153+0000 7f212ba6b640 1 -- 172.21.15.143:0/2884002298 wait complete. 2023-11-26T07:18:36.178 DEBUG:teuthology.orchestra.run.smithi143:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2023-11-26T07:18:36.316 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:36.309+0000 7f4a8d1df640 1 Processor -- start 2023-11-26T07:18:36.316 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:36.309+0000 7f4a8d1df640 1 -- start start 2023-11-26T07:18:36.317 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:36.313+0000 7f4a8d1df640 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7f4a8813e290 0x7f4a880cf280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:18:36.317 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:36.313+0000 7f4a8d1df640 1 -- --> v1:172.21.15.143:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4a8815fb20 con 0x7f4a880cf7c0 2023-11-26T07:18:36.318 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:36.313+0000 7f4a8d1df640 1 -- --> v1:172.21.15.188:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4a8806c520 con 0x7f4a8815f6b0 2023-11-26T07:18:36.318 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:36.313+0000 7f4a8d1df640 1 -- --> v2:172.21.15.188:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4a880cfc40 con 0x7f4a8813e290 2023-11-26T07:18:36.318 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:36.313+0000 7f4a8659c640 1 --1- >> v1:172.21.15.143:6789/0 conn(0x7f4a880cf7c0 0x7f4a8815efa0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.143:6789/0 says I am v1:172.21.15.143:50898/0 (socket says 172.21.15.143:50898) 2023-11-26T07:18:36.318 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:36.313+0000 7f4a8659c640 1 -- 172.21.15.143:0/2930183589 learned_addr learned my addr 172.21.15.143:0/2930183589 (peer_addr_for_me v1:172.21.15.143:0/0) 2023-11-26T07:18:36.320 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:36.313+0000 7f4a85d9b640 1 -- 172.21.15.143:0/2930183589 <== mon.0 v1:172.21.15.143:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1241048233 0 0) 0x7f4a7c001120 con 0x7f4a880cf7c0 2023-11-26T07:18:36.320 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:36.313+0000 7f4a85d9b640 1 -- 172.21.15.143:0/2930183589 --> v1:172.21.15.143:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f4a70003530 con 0x7f4a880cf7c0 2023-11-26T07:18:36.320 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:36.313+0000 7f4a86d9d640 1 --2- 172.21.15.143:0/2930183589 >> v2:172.21.15.188:3300/0 conn(0x7f4a8813e290 0x7f4a880cf280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:18:36.320 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:36.313+0000 7f4a85d9b640 1 -- 172.21.15.143:0/2930183589 <== mon.0 v1:172.21.15.143:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2900133 0 0) 0x7f4a70003530 con 0x7f4a880cf7c0 2023-11-26T07:18:36.320 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:36.313+0000 7f4a85d9b640 1 -- 172.21.15.143:0/2930183589 >> v2:172.21.15.188:3300/0 conn(0x7f4a8813e290 msgr2=0x7f4a880cf280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-26T07:18:36.320 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:36.313+0000 7f4a85d9b640 1 --2- 172.21.15.143:0/2930183589 >> v2:172.21.15.188:3300/0 conn(0x7f4a8813e290 0x7f4a880cf280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:18:36.320 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:36.313+0000 7f4a85d9b640 1 -- 172.21.15.143:0/2930183589 >> v1:172.21.15.188:6789/0 conn(0x7f4a8815f6b0 legacy=0x7f4a880cd7b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:18:36.320 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:36.313+0000 7f4a85d9b640 1 -- 172.21.15.143:0/2930183589 --> v1:172.21.15.143:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4a8806c800 con 0x7f4a880cf7c0 2023-11-26T07:18:36.320 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:36.313+0000 7f4a85d9b640 1 -- 172.21.15.143:0/2930183589 <== mon.0 v1:172.21.15.143:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2137882523 0 0) 0x7f4a7c001830 con 0x7f4a880cf7c0 2023-11-26T07:18:36.320 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:36.313+0000 7f4a85d9b640 1 -- 172.21.15.143:0/2930183589 >> v1:172.21.15.143:6789/0 conn(0x7f4a880cf7c0 legacy=0x7f4a8815efa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:18:36.320 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:36.313+0000 7f4a85d9b640 1 --2- 172.21.15.143:0/2930183589 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f4a70003980 0x7f4a70003da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:18:36.320 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:36.313+0000 7f4a85d9b640 1 --2- 172.21.15.143:0/2930183589 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f4a700047d0 0x7f4a70004c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:18:36.321 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:36.313+0000 7f4a85d9b640 1 --2- 172.21.15.143:0/2930183589 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f4a70005190 0x7f4a7000b990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:18:36.321 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:36.313+0000 7f4a85d9b640 1 -- 172.21.15.143:0/2930183589 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4a880cfc40 con 0x7f4a70005190 2023-11-26T07:18:36.321 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:36.313+0000 7f4a85d9b640 1 -- 172.21.15.143:0/2930183589 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4a7000c030 con 0x7f4a700047d0 2023-11-26T07:18:36.321 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:36.313+0000 7f4a85d9b640 1 -- 172.21.15.143:0/2930183589 --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4a70010600 con 0x7f4a70003980 2023-11-26T07:18:36.321 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:36.313+0000 7f4a86d9d640 1 --2- 172.21.15.143:0/2930183589 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f4a70003980 0x7f4a70003da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:18:36.321 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:36.313+0000 7f4a8659c640 1 --2- 172.21.15.143:0/2930183589 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f4a700047d0 0x7f4a70004c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:18:36.321 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:36.313+0000 7f4a8759e640 1 --2- 172.21.15.143:0/2930183589 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f4a70005190 0x7f4a7000b990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:18:36.321 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:36.313+0000 7f4a86d9d640 1 -- 172.21.15.143:0/2930183589 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f4a700047d0 msgr2=0x7f4a70004c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-26T07:18:36.321 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:36.313+0000 7f4a86d9d640 1 --2- 172.21.15.143:0/2930183589 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f4a700047d0 0x7f4a70004c50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:18:36.321 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:36.313+0000 7f4a86d9d640 1 -- 172.21.15.143:0/2930183589 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f4a70005190 msgr2=0x7f4a7000b990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-26T07:18:36.321 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:36.313+0000 7f4a86d9d640 1 --2- 172.21.15.143:0/2930183589 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f4a70005190 0x7f4a7000b990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:18:36.321 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:36.313+0000 7f4a86d9d640 1 -- 172.21.15.143:0/2930183589 --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4a70003530 con 0x7f4a70003980 2023-11-26T07:18:36.322 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:36.313+0000 7f4a8759e640 1 --2- 172.21.15.143:0/2930183589 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f4a70005190 0x7f4a7000b990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-26T07:18:36.322 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:36.313+0000 7f4a8759e640 1 -- 172.21.15.143:0/2930183589 reap_dead start 2023-11-26T07:18:36.322 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:36.317+0000 7f4a86d9d640 1 --2- 172.21.15.143:0/2930183589 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f4a70003980 0x7f4a70003da0 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7f4a78009ae0 tx=0x7f4a7801ca50 comp rx=0 tx=0).ready entity=mon.2 client_cookie=38d3744fa1a54cb7 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:18:36.322 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:36.317+0000 7f4a85d9b640 1 -- 172.21.15.143:0/2930183589 <== mon.2 v2:172.21.15.188:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4a780a5640 con 0x7f4a70003980 2023-11-26T07:18:36.322 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:36.317+0000 7f4a85d9b640 1 -- 172.21.15.143:0/2930183589 <== mon.2 v2:172.21.15.188:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4a780a57a0 con 0x7f4a70003980 2023-11-26T07:18:36.323 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:36.317+0000 7f4a85d9b640 1 -- 172.21.15.143:0/2930183589 <== mon.2 v2:172.21.15.188:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4a780a5b50 con 0x7f4a70003980 2023-11-26T07:18:36.323 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:36.317+0000 7f4a8d1df640 1 -- 172.21.15.143:0/2930183589 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f4a70003980 msgr2=0x7f4a70003da0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:18:36.323 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:36.317+0000 7f4a8d1df640 1 --2- 172.21.15.143:0/2930183589 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f4a70003980 0x7f4a70003da0 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7f4a78009ae0 tx=0x7f4a7801ca50 comp rx=0 tx=0).stop 2023-11-26T07:18:36.323 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:36.317+0000 7f4a8d1df640 1 -- 172.21.15.143:0/2930183589 shutdown_connections 2023-11-26T07:18:36.324 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:36.317+0000 7f4a8d1df640 1 --2- 172.21.15.143:0/2930183589 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f4a70003980 0x7f4a70003da0 unknown :-1 s=CLOSED pgs=64 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:18:36.324 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:36.317+0000 7f4a8d1df640 1 -- 172.21.15.143:0/2930183589 >> 172.21.15.143:0/2930183589 conn(0x7f4a8806a890 msgr2=0x7f4a8806aca0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:18:36.324 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:36.317+0000 7f4a8d1df640 1 -- 172.21.15.143:0/2930183589 shutdown_connections 2023-11-26T07:18:36.324 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:36.317+0000 7f4a8d1df640 1 -- 172.21.15.143:0/2930183589 wait complete. 2023-11-26T07:18:36.325 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:36.321+0000 7f4a8d1df640 1 Processor -- start 2023-11-26T07:18:36.325 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:36.321+0000 7f4a8d1df640 1 -- start start 2023-11-26T07:18:36.327 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:36.321+0000 7f4a8d1df640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f4a70003980 0x7f4a880b3b60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:18:36.327 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:36.321+0000 7f4a8d1df640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f4a880b2550 0x7f4a8813a8a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:18:36.327 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:36.321+0000 7f4a8d1df640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f4a881eefa0 0x7f4a881ef420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:18:36.327 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:36.321+0000 7f4a8d1df640 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4a880b36d0 con 0x7f4a880b2550 2023-11-26T07:18:36.328 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:36.321+0000 7f4a8d1df640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4a880b2ac0 con 0x7f4a70003980 2023-11-26T07:18:36.328 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:36.321+0000 7f4a8d1df640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4a880b2da0 con 0x7f4a881eefa0 2023-11-26T07:18:36.328 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:36.321+0000 7f4a8659c640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f4a880b2550 0x7f4a8813a8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:18:36.328 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:36.321+0000 7f4a8659c640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f4a880b2550 0x7f4a8813a8a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:38874/0 (socket says 172.21.15.143:38874) 2023-11-26T07:18:36.328 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:36.321+0000 7f4a8659c640 1 -- 172.21.15.143:0/1202305619 learned_addr learned my addr 172.21.15.143:0/1202305619 (peer_addr_for_me v2:172.21.15.143:0/0) 2023-11-26T07:18:36.328 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:36.321+0000 7f4a8759e640 1 --2- 172.21.15.143:0/1202305619 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f4a881eefa0 0x7f4a881ef420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:18:36.328 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:36.321+0000 7f4a86d9d640 1 --2- 172.21.15.143:0/1202305619 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f4a70003980 0x7f4a880b3b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:18:36.328 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:36.321+0000 7f4a8659c640 1 -- 172.21.15.143:0/1202305619 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f4a881eefa0 msgr2=0x7f4a881ef420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:18:36.328 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:36.321+0000 7f4a8659c640 1 --2- 172.21.15.143:0/1202305619 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f4a881eefa0 0x7f4a881ef420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:18:36.328 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:36.321+0000 7f4a8659c640 1 -- 172.21.15.143:0/1202305619 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f4a70003980 msgr2=0x7f4a880b3b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:18:36.328 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:36.321+0000 7f4a8659c640 1 --2- 172.21.15.143:0/1202305619 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f4a70003980 0x7f4a880b3b60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:18:36.328 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:36.321+0000 7f4a8659c640 1 -- 172.21.15.143:0/1202305619 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4a880b3080 con 0x7f4a880b2550 2023-11-26T07:18:36.329 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:36.321+0000 7f4a8759e640 1 --2- 172.21.15.143:0/1202305619 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f4a881eefa0 0x7f4a881ef420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-26T07:18:36.329 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:36.321+0000 7f4a86d9d640 1 --2- 172.21.15.143:0/1202305619 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f4a70003980 0x7f4a880b3b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-26T07:18:36.329 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:36.321+0000 7f4a8659c640 1 --2- 172.21.15.143:0/1202305619 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f4a880b2550 0x7f4a8813a8a0 secure :-1 s=READY pgs=314 cs=0 l=1 rev1=1 crypto rx=0x7f4a7c0072c0 tx=0x7f4a7c007780 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:18:36.329 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:36.321+0000 7f4a6f7fe640 1 -- 172.21.15.143:0/1202305619 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4a7c00f960 con 0x7f4a880b2550 2023-11-26T07:18:36.329 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:36.321+0000 7f4a6f7fe640 1 -- 172.21.15.143:0/1202305619 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4a7c00fac0 con 0x7f4a880b2550 2023-11-26T07:18:36.329 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:36.321+0000 7f4a6f7fe640 1 -- 172.21.15.143:0/1202305619 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4a7c00fe70 con 0x7f4a880b2550 2023-11-26T07:18:36.329 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:36.321+0000 7f4a8d1df640 1 -- 172.21.15.143:0/1202305619 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4a880b3360 con 0x7f4a880b2550 2023-11-26T07:18:36.329 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:36.321+0000 7f4a8d1df640 1 -- 172.21.15.143:0/1202305619 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4a8813afb0 con 0x7f4a880b2550 2023-11-26T07:18:36.330 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:36.321+0000 7f4a8d1df640 1 -- 172.21.15.143:0/1202305619 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4a880b3880 con 0x7f4a880b2550 2023-11-26T07:18:36.330 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:36.325+0000 7f4a6f7fe640 1 -- 172.21.15.143:0/1202305619 <== mon.0 v2:172.21.15.143:3300/0 4 ==== mgrmap(e 5) v1 ==== 85852+0+0 (secure 0 0 0) 0x7f4a7c01d460 con 0x7f4a880b2550 2023-11-26T07:18:36.330 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:36.325+0000 7f4a6f7fe640 1 --2- 172.21.15.143:0/1202305619 >> [v2:172.21.15.143:6832/2150557544,v1:172.21.15.143:6833/2150557544] conn(0x7f4a740664a0 0x7f4a74068960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:18:36.331 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:36.325+0000 7f4a6f7fe640 1 -- 172.21.15.143:0/1202305619 <== mon.0 v2:172.21.15.143:3300/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 5825+0+0 (secure 0 0 0) 0x7f4a7c083ff0 con 0x7f4a880b2550 2023-11-26T07:18:36.331 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:36.325+0000 7f4a86d9d640 1 --2- 172.21.15.143:0/1202305619 >> [v2:172.21.15.143:6832/2150557544,v1:172.21.15.143:6833/2150557544] conn(0x7f4a740664a0 0x7f4a74068960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:18:36.331 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:36.325+0000 7f4a86d9d640 1 --2- 172.21.15.143:0/1202305619 >> [v2:172.21.15.143:6832/2150557544,v1:172.21.15.143:6833/2150557544] conn(0x7f4a740664a0 0x7f4a74068960 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7f4a780af000 tx=0x7f4a78008040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:18:36.334 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:36.329+0000 7f4a6f7fe640 1 -- 172.21.15.143:0/1202305619 <== mon.0 v2:172.21.15.143:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172649 (secure 0 0 0) 0x7f4a7c0b5280 con 0x7f4a880b2550 2023-11-26T07:18:36.498 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:36.493+0000 7f4a8d1df640 1 -- 172.21.15.143:0/1202305619 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) v1 -- 0x7f4a8815fb20 con 0x7f4a880b2550 2023-11-26T07:18:36.500 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:36.493+0000 7f4a6f7fe640 1 -- 172.21.15.143:0/1202305619 <== mon.0 v2:172.21.15.143:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v22) v1 ==== 80+0+0 (secure 0 0 0) 0x7f4a8815fb20 con 0x7f4a880b2550 2023-11-26T07:18:36.500 INFO:teuthology.orchestra.run.smithi143.stderr:listed 0 entries 2023-11-26T07:18:36.502 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:36.493+0000 7f4a8d1df640 1 -- 172.21.15.143:0/1202305619 >> [v2:172.21.15.143:6832/2150557544,v1:172.21.15.143:6833/2150557544] conn(0x7f4a740664a0 msgr2=0x7f4a74068960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:18:36.502 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:36.493+0000 7f4a8d1df640 1 --2- 172.21.15.143:0/1202305619 >> [v2:172.21.15.143:6832/2150557544,v1:172.21.15.143:6833/2150557544] conn(0x7f4a740664a0 0x7f4a74068960 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7f4a780af000 tx=0x7f4a78008040 comp rx=0 tx=0).stop 2023-11-26T07:18:36.502 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:36.493+0000 7f4a8d1df640 1 -- 172.21.15.143:0/1202305619 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f4a880b2550 msgr2=0x7f4a8813a8a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:18:36.503 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:36.493+0000 7f4a8d1df640 1 --2- 172.21.15.143:0/1202305619 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f4a880b2550 0x7f4a8813a8a0 secure :-1 s=READY pgs=314 cs=0 l=1 rev1=1 crypto rx=0x7f4a7c0072c0 tx=0x7f4a7c007780 comp rx=0 tx=0).stop 2023-11-26T07:18:36.503 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:36.497+0000 7f4a8d1df640 1 -- 172.21.15.143:0/1202305619 shutdown_connections 2023-11-26T07:18:36.503 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:36.497+0000 7f4a8d1df640 1 --2- 172.21.15.143:0/1202305619 >> [v2:172.21.15.143:6832/2150557544,v1:172.21.15.143:6833/2150557544] conn(0x7f4a740664a0 0x7f4a74068960 unknown :-1 s=CLOSED pgs=177 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:18:36.503 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:36.497+0000 7f4a8d1df640 1 --2- 172.21.15.143:0/1202305619 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f4a881eefa0 0x7f4a881ef420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:18:36.503 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:36.497+0000 7f4a8d1df640 1 --2- 172.21.15.143:0/1202305619 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f4a880b2550 0x7f4a8813a8a0 unknown :-1 s=CLOSED pgs=314 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:18:36.503 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:36.497+0000 7f4a8d1df640 1 --2- 172.21.15.143:0/1202305619 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f4a70003980 0x7f4a880b3b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:18:36.503 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:36.497+0000 7f4a8d1df640 1 -- 172.21.15.143:0/1202305619 >> 172.21.15.143:0/1202305619 conn(0x7f4a8806a890 msgr2=0x7f4a88070c00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:18:36.503 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:36.497+0000 7f4a8d1df640 1 -- 172.21.15.143:0/1202305619 shutdown_connections 2023-11-26T07:18:36.504 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:36.497+0000 7f4a8d1df640 1 -- 172.21.15.143:0/1202305619 wait complete. 2023-11-26T07:18:36.530 DEBUG:teuthology.orchestra.run.smithi188:> set -ex 2023-11-26T07:18:36.531 DEBUG:teuthology.orchestra.run.smithi188:> dd if=/proc/self/mounts of=/dev/stdout 2023-11-26T07:18:36.544 DEBUG:teuthology.orchestra.run.smithi143:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-11-26T07:18:36.689 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:36.681+0000 7f1ff90c9640 1 Processor -- start 2023-11-26T07:18:36.689 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:36.685+0000 7f1ff90c9640 1 -- start start 2023-11-26T07:18:36.691 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:36.685+0000 7f1ff90c9640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f1ff40cf990 0x7f1ff415efa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:18:36.691 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:36.685+0000 7f1ff90c9640 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7f1ff415f4e0 0x7f1ff40cd7b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:18:36.691 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:36.685+0000 7f1ff90c9640 1 -- --> v1:172.21.15.143:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1ff406c520 con 0x7f1ff413e290 2023-11-26T07:18:36.691 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:36.685+0000 7f1ff90c9640 1 -- --> v2:172.21.15.143:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1ff406c800 con 0x7f1ff415f4e0 2023-11-26T07:18:36.691 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:36.685+0000 7f1ff90c9640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1ff406cae0 con 0x7f1ff40cf990 2023-11-26T07:18:36.691 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:36.685+0000 7f1ff359e640 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7f1ff415f4e0 0x7f1ff40cd7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:18:36.691 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:36.685+0000 7f1ff2d9d640 1 --1- >> v1:172.21.15.143:6789/0 conn(0x7f1ff413e290 0x7f1ff40cf280 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.143:6789/0 says I am v1:172.21.15.143:50900/0 (socket says 172.21.15.143:50900) 2023-11-26T07:18:36.691 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:36.685+0000 7f1ff359e640 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7f1ff415f4e0 0x7f1ff40cd7b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:38882/0 (socket says 172.21.15.143:38882) 2023-11-26T07:18:36.692 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:36.685+0000 7f1ff2d9d640 1 -- 172.21.15.143:0/1625945899 learned_addr learned my addr 172.21.15.143:0/1625945899 (peer_addr_for_me v1:172.21.15.143:0/0) 2023-11-26T07:18:36.692 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:36.685+0000 7f1febfff640 1 --2- 172.21.15.143:0/1625945899 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f1ff40cf990 0x7f1ff415efa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:18:36.692 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:36.685+0000 7f1ff359e640 1 -- 172.21.15.143:0/1625945899 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f1ff40cf990 msgr2=0x7f1ff415efa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-26T07:18:36.693 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:36.685+0000 7f1ff359e640 1 --2- 172.21.15.143:0/1625945899 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f1ff40cf990 0x7f1ff415efa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:18:36.693 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:36.685+0000 7f1ff359e640 1 -- 172.21.15.143:0/1625945899 >> v1:172.21.15.143:6789/0 conn(0x7f1ff413e290 legacy=0x7f1ff40cf280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:18:36.693 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:36.685+0000 7f1ff359e640 1 -- 172.21.15.143:0/1625945899 --> v2:172.21.15.143:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1ff406cdc0 con 0x7f1ff415f4e0 2023-11-26T07:18:36.693 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:36.685+0000 7f1ff259c640 1 -- 172.21.15.143:0/1625945899 <== mon.0 v1:172.21.15.143:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2116844758 0 0) 0x7f1ff406c520 con 0x7f1ff413e290 2023-11-26T07:18:36.693 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:36.685+0000 7f1febfff640 1 --2- 172.21.15.143:0/1625945899 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f1ff40cf990 0x7f1ff415efa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-26T07:18:36.694 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:36.689+0000 7f1ff359e640 1 --2- 172.21.15.143:0/1625945899 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f1ff415f4e0 0x7f1ff40cd7b0 secure :-1 s=READY pgs=316 cs=0 l=1 rev1=1 crypto rx=0x7f1fec009d90 tx=0x7f1fec01c920 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8c6c6dba13804159 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:18:36.694 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:36.689+0000 7f1ff259c640 1 -- 172.21.15.143:0/1625945899 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1fec0a5a00 con 0x7f1ff415f4e0 2023-11-26T07:18:36.694 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:36.689+0000 7f1ff259c640 1 -- 172.21.15.143:0/1625945899 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1fec0a5b60 con 0x7f1ff415f4e0 2023-11-26T07:18:36.695 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:36.689+0000 7f1ff259c640 1 -- 172.21.15.143:0/1625945899 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1fec0ad440 con 0x7f1ff415f4e0 2023-11-26T07:18:36.696 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:36.689+0000 7f1ff90c9640 1 -- 172.21.15.143:0/1625945899 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f1ff415f4e0 msgr2=0x7f1ff40cd7b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:18:36.696 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:36.689+0000 7f1ff90c9640 1 --2- 172.21.15.143:0/1625945899 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f1ff415f4e0 0x7f1ff40cd7b0 secure :-1 s=READY pgs=316 cs=0 l=1 rev1=1 crypto rx=0x7f1fec009d90 tx=0x7f1fec01c920 comp rx=0 tx=0).stop 2023-11-26T07:18:36.696 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:36.689+0000 7f1ff90c9640 1 -- 172.21.15.143:0/1625945899 shutdown_connections 2023-11-26T07:18:36.696 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:36.689+0000 7f1ff90c9640 1 --2- 172.21.15.143:0/1625945899 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f1ff415f4e0 0x7f1ff40cd7b0 unknown :-1 s=CLOSED pgs=316 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:18:36.697 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:36.689+0000 7f1ff90c9640 1 --2- 172.21.15.143:0/1625945899 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f1ff40cf990 0x7f1ff415efa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:18:36.697 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:36.689+0000 7f1ff90c9640 1 -- 172.21.15.143:0/1625945899 >> 172.21.15.143:0/1625945899 conn(0x7f1ff406a890 msgr2=0x7f1ff406aca0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:18:36.697 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:36.689+0000 7f1ff90c9640 1 -- 172.21.15.143:0/1625945899 shutdown_connections 2023-11-26T07:18:36.697 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:36.689+0000 7f1ff90c9640 1 -- 172.21.15.143:0/1625945899 wait complete. 2023-11-26T07:18:36.697 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:36.689+0000 7f1ff90c9640 1 Processor -- start 2023-11-26T07:18:36.698 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:36.689+0000 7f1ff90c9640 1 -- start start 2023-11-26T07:18:36.698 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:36.693+0000 7f1ff90c9640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f1ff40cf990 0x7f1ff414d310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:18:36.698 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:36.693+0000 7f1ff90c9640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f1ff413e290 0x7f1ff4136b10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:18:36.701 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:36.693+0000 7f1ff90c9640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f1ff415f4e0 0x7f1ff415a150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:18:36.701 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:36.693+0000 7f1ff90c9640 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1ff4139490 con 0x7f1ff40cf990 2023-11-26T07:18:36.702 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:36.693+0000 7f1ff90c9640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1ff414d850 con 0x7f1ff413e290 2023-11-26T07:18:36.702 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:36.693+0000 7f1ff90c9640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1ff4072170 con 0x7f1ff415f4e0 2023-11-26T07:18:36.702 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:36.693+0000 7f1ff2d9d640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f1ff40cf990 0x7f1ff414d310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:18:36.702 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:36.693+0000 7f1ff2d9d640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f1ff40cf990 0x7f1ff414d310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:38886/0 (socket says 172.21.15.143:38886) 2023-11-26T07:18:36.702 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:36.693+0000 7f1ff2d9d640 1 -- 172.21.15.143:0/3476286382 learned_addr learned my addr 172.21.15.143:0/3476286382 (peer_addr_for_me v2:172.21.15.143:0/0) 2023-11-26T07:18:36.702 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:36.693+0000 7f1febfff640 1 --2- 172.21.15.143:0/3476286382 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f1ff413e290 0x7f1ff4136b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:18:36.703 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:36.693+0000 7f1ff359e640 1 --2- 172.21.15.143:0/3476286382 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f1ff415f4e0 0x7f1ff415a150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:18:36.703 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:36.693+0000 7f1ff2d9d640 1 -- 172.21.15.143:0/3476286382 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f1ff415f4e0 msgr2=0x7f1ff415a150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:18:36.703 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:36.693+0000 7f1ff2d9d640 1 --2- 172.21.15.143:0/3476286382 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f1ff415f4e0 0x7f1ff415a150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:18:36.703 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:36.693+0000 7f1ff2d9d640 1 -- 172.21.15.143:0/3476286382 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f1ff413e290 msgr2=0x7f1ff4136b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:18:36.703 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:36.693+0000 7f1ff2d9d640 1 --2- 172.21.15.143:0/3476286382 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f1ff413e290 0x7f1ff4136b10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:18:36.703 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:36.693+0000 7f1ff2d9d640 1 -- 172.21.15.143:0/3476286382 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1fec009a70 con 0x7f1ff40cf990 2023-11-26T07:18:36.704 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:36.693+0000 7f1ff359e640 1 --2- 172.21.15.143:0/3476286382 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f1ff415f4e0 0x7f1ff415a150 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-26T07:18:36.704 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:36.693+0000 7f1ff2d9d640 1 --2- 172.21.15.143:0/3476286382 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f1ff40cf990 0x7f1ff414d310 secure :-1 s=READY pgs=317 cs=0 l=1 rev1=1 crypto rx=0x7f1fe400c520 tx=0x7f1fe400c9e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:18:36.704 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:36.693+0000 7f1feb7fe640 1 -- 172.21.15.143:0/3476286382 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1fe40074e0 con 0x7f1ff40cf990 2023-11-26T07:18:36.704 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:36.693+0000 7f1feb7fe640 1 -- 172.21.15.143:0/3476286382 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1fe4007640 con 0x7f1ff40cf990 2023-11-26T07:18:36.704 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:36.693+0000 7f1feb7fe640 1 -- 172.21.15.143:0/3476286382 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1fe40049f0 con 0x7f1ff40cf990 2023-11-26T07:18:36.704 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:36.693+0000 7f1ff90c9640 1 -- 172.21.15.143:0/3476286382 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1ff4138dd0 con 0x7f1ff40cf990 2023-11-26T07:18:36.704 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:36.693+0000 7f1ff90c9640 1 -- 172.21.15.143:0/3476286382 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1ff41398b0 con 0x7f1ff40cf990 2023-11-26T07:18:36.706 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:36.693+0000 7f1ff90c9640 1 -- 172.21.15.143:0/3476286382 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1ff41395d0 con 0x7f1ff40cf990 2023-11-26T07:18:36.706 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:36.697+0000 7f1feb7fe640 1 -- 172.21.15.143:0/3476286382 <== mon.0 v2:172.21.15.143:3300/0 4 ==== mgrmap(e 5) v1 ==== 85852+0+0 (secure 0 0 0) 0x7f1fe4004b50 con 0x7f1ff40cf990 2023-11-26T07:18:36.706 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:36.697+0000 7f1feb7fe640 1 --2- 172.21.15.143:0/3476286382 >> [v2:172.21.15.143:6832/2150557544,v1:172.21.15.143:6833/2150557544] conn(0x7f1fd4066470 0x7f1fd4068930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:18:36.711 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:36.697+0000 7f1feb7fe640 1 -- 172.21.15.143:0/3476286382 <== mon.0 v2:172.21.15.143:3300/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 5825+0+0 (secure 0 0 0) 0x7f1fe40830e0 con 0x7f1ff40cf990 2023-11-26T07:18:36.711 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:36.697+0000 7f1febfff640 1 --2- 172.21.15.143:0/3476286382 >> [v2:172.21.15.143:6832/2150557544,v1:172.21.15.143:6833/2150557544] conn(0x7f1fd4066470 0x7f1fd4068930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:18:36.711 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:36.697+0000 7f1febfff640 1 --2- 172.21.15.143:0/3476286382 >> [v2:172.21.15.143:6832/2150557544,v1:172.21.15.143:6833/2150557544] conn(0x7f1fd4066470 0x7f1fd4068930 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7f1ff40719f0 tx=0x7f1fdc008040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:18:36.711 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:36.701+0000 7f1feb7fe640 1 -- 172.21.15.143:0/3476286382 <== mon.0 v2:172.21.15.143:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172649 (secure 0 0 0) 0x7f1ff41395d0 con 0x7f1ff40cf990 2023-11-26T07:18:36.859 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:36.853+0000 7f1ff90c9640 1 -- 172.21.15.143:0/3476286382 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f1ff41395d0 con 0x7f1ff40cf990 2023-11-26T07:18:36.859 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:36.853+0000 7f1feb7fe640 1 -- 172.21.15.143:0/3476286382 <== mon.0 v2:172.21.15.143:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v22) v1 ==== 74+0+16731 (secure 0 0 0) 0x7f1ff41395d0 con 0x7f1ff40cf990 2023-11-26T07:18:36.859 INFO:teuthology.orchestra.run.smithi143.stdout: 2023-11-26T07:18:36.859 INFO:teuthology.orchestra.run.smithi143.stdout:{"epoch":22,"fsid":"f5033060-2469-4a07-95fe-2b76f5404bc1","created":"2023-11-26T07:17:28.613663+0000","modified":"2023-11-26T07:18:35.705228+0000","last_up_change":"2023-11-26T07:17:41.157813+0000","last_in_change":"2023-11-26T07:17:31.181658+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-11-26T07:17:41.758597+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2023-11-26T07:17:51.878332+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2023-11-26T07:17:52.871396+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"ebbee2c6-023e-44b9-9f1d-6f7bfd309730","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6817","nonce":639807369},{"type":"v1","addr":"172.21.15.143:6819","nonce":639807369}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6821","nonce":639807369},{"type":"v1","addr":"172.21.15.143:6823","nonce":639807369}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6830","nonce":639807369},{"type":"v1","addr":"172.21.15.143:6831","nonce":639807369}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6825","nonce":639807369},{"type":"v1","addr":"172.21.15.143:6828","nonce":639807369}]},"public_addr":"172.21.15.143:6819/639807369","cluster_addr":"172.21.15.143:6823/639807369","heartbeat_back_addr":"172.21.15.143:6831/639807369","heartbeat_front_addr":"172.21.15.143:6828/639807369","state":["exists","up"]},{"osd":1,"uuid":"fec4b8be-7122-49e9-9a19-5850a0700001","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6816","nonce":1144942530},{"type":"v1","addr":"172.21.15.143:6818","nonce":1144942530}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6820","nonce":1144942530},{"type":"v1","addr":"172.21.15.143:6822","nonce":1144942530}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6827","nonce":1144942530},{"type":"v1","addr":"172.21.15.143:6829","nonce":1144942530}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6824","nonce":1144942530},{"type":"v1","addr":"172.21.15.143:6826","nonce":1144942530}]},"public_addr":"172.21.15.143:6818/1144942530","cluster_addr":"172.21.15.143:6822/1144942530","heartbeat_back_addr":"172.21.15.143:6829/1144942530","heartbeat_front_addr":"172.21.15.143:6826/1144942530","state":["exists","up"]},{"osd":2,"uuid":"2888af5b-f933-46ca-8f95-6dd494ff43f6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6808","nonce":3909095186},{"type":"v1","addr":"172.21.15.143:6809","nonce":3909095186}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6810","nonce":3909095186},{"type":"v1","addr":"172.21.15.143:6811","nonce":3909095186}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6814","nonce":3909095186},{"type":"v1","addr":"172.21.15.143:6815","nonce":3909095186}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6812","nonce":3909095186},{"type":"v1","addr":"172.21.15.143:6813","nonce":3909095186}]},"public_addr":"172.21.15.143:6809/3909095186","cluster_addr":"172.21.15.143:6811/3909095186","heartbeat_back_addr":"172.21.15.143:6815/3909095186","heartbeat_front_addr":"172.21.15.143:6813/3909095186","state":["exists","up"]},{"osd":3,"uuid":"d1f52435-7bc1-4081-9550-9cd1ba55f549","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6800","nonce":3854210991},{"type":"v1","addr":"172.21.15.143:6801","nonce":3854210991}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6802","nonce":3854210991},{"type":"v1","addr":"172.21.15.143:6803","nonce":3854210991}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6806","nonce":3854210991},{"type":"v1","addr":"172.21.15.143:6807","nonce":3854210991}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6804","nonce":3854210991},{"type":"v1","addr":"172.21.15.143:6805","nonce":3854210991}]},"public_addr":"172.21.15.143:6801/3854210991","cluster_addr":"172.21.15.143:6803/3854210991","heartbeat_back_addr":"172.21.15.143:6807/3854210991","heartbeat_front_addr":"172.21.15.143:6805/3854210991","state":["exists","up"]},{"osd":4,"uuid":"e2ae8b6e-6e25-49b2-bcbf-aacce68f06a3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6800","nonce":3924606902},{"type":"v1","addr":"172.21.15.188:6801","nonce":3924606902}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6802","nonce":3924606902},{"type":"v1","addr":"172.21.15.188:6803","nonce":3924606902}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6806","nonce":3924606902},{"type":"v1","addr":"172.21.15.188:6807","nonce":3924606902}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6804","nonce":3924606902},{"type":"v1","addr":"172.21.15.188:6805","nonce":3924606902}]},"public_addr":"172.21.15.188:6801/3924606902","cluster_addr":"172.21.15.188:6803/3924606902","heartbeat_back_addr":"172.21.15.188:6807/3924606902","heartbeat_front_addr":"172.21.15.188:6805/3924606902","state":["exists","up"]},{"osd":5,"uuid":"3cec8a32-7698-4c36-8e89-e4ba11934f33","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6813","nonce":186841544},{"type":"v1","addr":"172.21.15.188:6815","nonce":186841544}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6817","nonce":186841544},{"type":"v1","addr":"172.21.15.188:6818","nonce":186841544}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6822","nonce":186841544},{"type":"v1","addr":"172.21.15.188:6823","nonce":186841544}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6820","nonce":186841544},{"type":"v1","addr":"172.21.15.188:6821","nonce":186841544}]},"public_addr":"172.21.15.188:6815/186841544","cluster_addr":"172.21.15.188:6818/186841544","heartbeat_back_addr":"172.21.15.188:6823/186841544","heartbeat_front_addr":"172.21.15.188:6821/186841544","state":["exists","up"]},{"osd":6,"uuid":"58a65d5d-e090-473a-81b4-ad38e85029f9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6808","nonce":2153499405},{"type":"v1","addr":"172.21.15.188:6809","nonce":2153499405}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6810","nonce":2153499405},{"type":"v1","addr":"172.21.15.188:6811","nonce":2153499405}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6816","nonce":2153499405},{"type":"v1","addr":"172.21.15.188:6819","nonce":2153499405}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6812","nonce":2153499405},{"type":"v1","addr":"172.21.15.188:6814","nonce":2153499405}]},"public_addr":"172.21.15.188:6809/2153499405","cluster_addr":"172.21.15.188:6811/2153499405","heartbeat_back_addr":"172.21.15.188:6819/2153499405","heartbeat_front_addr":"172.21.15.188:6814/2153499405","state":["exists","up"]},{"osd":7,"uuid":"ea5b7f70-3554-4cf8-ac30-f7c59b80678d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6824","nonce":2722096729},{"type":"v1","addr":"172.21.15.188:6825","nonce":2722096729}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6826","nonce":2722096729},{"type":"v1","addr":"172.21.15.188:6827","nonce":2722096729}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6830","nonce":2722096729},{"type":"v1","addr":"172.21.15.188:6831","nonce":2722096729}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6828","nonce":2722096729},{"type":"v1","addr":"172.21.15.188:6829","nonce":2722096729}]},"public_addr":"172.21.15.188:6825/2722096729","cluster_addr":"172.21.15.188:6827/2722096729","heartbeat_back_addr":"172.21.15.188:6831/2722096729","heartbeat_front_addr":"172.21.15.188:6829/2722096729","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-26T07:17:36.995032+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-26T07:17:36.932266+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-26T07:17:36.829304+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-26T07:17:36.804297+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-26T07:17:36.667892+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-26T07:17:36.964868+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-26T07:17:36.827332+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-26T07:17:36.893179+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"3.1f","mappings":[{"from":5,"to":1}]},{"pgid":"3.24","mappings":[{"from":3,"to":2}]},{"pgid":"3.26","mappings":[{"from":7,"to":2}]},{"pgid":"3.36","mappings":[{"from":5,"to":2}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-11-26T07:18:36.861 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:36.853+0000 7f1ff90c9640 1 -- 172.21.15.143:0/3476286382 >> [v2:172.21.15.143:6832/2150557544,v1:172.21.15.143:6833/2150557544] conn(0x7f1fd4066470 msgr2=0x7f1fd4068930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:18:36.861 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:36.853+0000 7f1ff90c9640 1 --2- 172.21.15.143:0/3476286382 >> [v2:172.21.15.143:6832/2150557544,v1:172.21.15.143:6833/2150557544] conn(0x7f1fd4066470 0x7f1fd4068930 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7f1ff40719f0 tx=0x7f1fdc008040 comp rx=0 tx=0).stop 2023-11-26T07:18:36.861 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:36.853+0000 7f1ff90c9640 1 -- 172.21.15.143:0/3476286382 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f1ff40cf990 msgr2=0x7f1ff414d310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:18:36.861 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:36.853+0000 7f1ff90c9640 1 --2- 172.21.15.143:0/3476286382 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f1ff40cf990 0x7f1ff414d310 secure :-1 s=READY pgs=317 cs=0 l=1 rev1=1 crypto rx=0x7f1fe400c520 tx=0x7f1fe400c9e0 comp rx=0 tx=0).stop 2023-11-26T07:18:36.861 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:36.853+0000 7f1ff90c9640 1 -- 172.21.15.143:0/3476286382 shutdown_connections 2023-11-26T07:18:36.861 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:36.853+0000 7f1ff90c9640 1 --2- 172.21.15.143:0/3476286382 >> [v2:172.21.15.143:6832/2150557544,v1:172.21.15.143:6833/2150557544] conn(0x7f1fd4066470 0x7f1fd4068930 unknown :-1 s=CLOSED pgs=178 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:18:36.862 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:36.853+0000 7f1ff90c9640 1 --2- 172.21.15.143:0/3476286382 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f1ff415f4e0 0x7f1ff415a150 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:18:36.862 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:36.853+0000 7f1ff90c9640 1 --2- 172.21.15.143:0/3476286382 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f1ff413e290 0x7f1ff4136b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:18:36.862 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:36.853+0000 7f1ff90c9640 1 --2- 172.21.15.143:0/3476286382 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f1ff40cf990 0x7f1ff414d310 unknown :-1 s=CLOSED pgs=317 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:18:36.862 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:36.853+0000 7f1ff90c9640 1 -- 172.21.15.143:0/3476286382 >> 172.21.15.143:0/3476286382 conn(0x7f1ff406a890 msgr2=0x7f1ff40b25e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:18:36.862 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:36.853+0000 7f1ff90c9640 1 -- 172.21.15.143:0/3476286382 shutdown_connections 2023-11-26T07:18:36.862 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:36.853+0000 7f1ff90c9640 1 -- 172.21.15.143:0/3476286382 wait complete. 2023-11-26T07:18:36.880 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-11-26T07:17:41.758597+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2023-11-26T07:17:51.878332+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2023-11-26T07:17:52.871396+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '20', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2023-11-26T07:18:36.881 DEBUG:teuthology.orchestra.run.smithi143:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-11-26T07:18:37.020 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:37.013+0000 7f8a1759e640 1 Processor -- start 2023-11-26T07:18:37.020 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:37.013+0000 7f8a1759e640 1 -- start start 2023-11-26T07:18:37.021 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:37.017+0000 7f8a1759e640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f8a180cfa70 0x7f8a180d0e40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:18:37.021 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:37.017+0000 7f8a1759e640 1 -- --> v1:172.21.15.143:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8a180d3d20 con 0x7f8a180ced30 2023-11-26T07:18:37.021 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:37.017+0000 7f8a1759e640 1 -- --> v1:172.21.15.188:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8a180afd70 con 0x7f8a180cfe50 2023-11-26T07:18:37.021 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:37.017+0000 7f8a1759e640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8a180d4000 con 0x7f8a180cfa70 2023-11-26T07:18:37.021 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:37.017+0000 7f8a16d9d640 1 --1- >> v1:172.21.15.143:6789/0 conn(0x7f8a180ced30 0x7f8a180b0f60 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.143:6789/0 says I am v1:172.21.15.143:50914/0 (socket says 172.21.15.143:50914) 2023-11-26T07:18:37.021 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:37.017+0000 7f8a16d9d640 1 -- 172.21.15.143:0/1988424274 learned_addr learned my addr 172.21.15.143:0/1988424274 (peer_addr_for_me v1:172.21.15.143:0/0) 2023-11-26T07:18:37.022 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:37.017+0000 7f8a1659c640 1 --2- 172.21.15.143:0/1988424274 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f8a180cfa70 0x7f8a180d0e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:18:37.022 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:37.017+0000 7f8a15d9b640 1 -- 172.21.15.143:0/1988424274 <== mon.0 v1:172.21.15.143:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1508262909 0 0) 0x7f8a180d3d20 con 0x7f8a180ced30 2023-11-26T07:18:37.023 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:37.017+0000 7f8a15d9b640 1 -- 172.21.15.143:0/1988424274 --> v1:172.21.15.143:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f8a04003530 con 0x7f8a180ced30 2023-11-26T07:18:37.023 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:37.017+0000 7f8a15d9b640 1 -- 172.21.15.143:0/1988424274 <== mon.1 v1:172.21.15.188:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4165492683 0 0) 0x7f8a180afd70 con 0x7f8a180cfe50 2023-11-26T07:18:37.023 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:37.017+0000 7f8a15d9b640 1 -- 172.21.15.143:0/1988424274 --> v1:172.21.15.188:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f8a180d3d20 con 0x7f8a180cfe50 2023-11-26T07:18:37.023 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:37.017+0000 7f8a1659c640 1 -- 172.21.15.143:0/1988424274 >> v1:172.21.15.188:6789/0 conn(0x7f8a180cfe50 legacy=0x7f8a180b0850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:18:37.023 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:37.017+0000 7f8a1659c640 1 -- 172.21.15.143:0/1988424274 >> v1:172.21.15.143:6789/0 conn(0x7f8a180ced30 legacy=0x7f8a180b0f60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:18:37.023 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:37.017+0000 7f8a15d9b640 1 -- 172.21.15.143:0/1988424274 <== mon.0 v1:172.21.15.143:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3399247002 0 0) 0x7f8a04003530 con 0x7f8a180ced30 2023-11-26T07:18:37.023 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:37.017+0000 7f8a1659c640 1 -- 172.21.15.143:0/1988424274 --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8a180b0050 con 0x7f8a180cfa70 2023-11-26T07:18:37.025 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:37.021+0000 7f8a1659c640 1 --2- 172.21.15.143:0/1988424274 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f8a180cfa70 0x7f8a180d0e40 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7f8a080097f0 tx=0x7f8a0801c3b0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=b4842f43d2683c03 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:18:37.025 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:37.021+0000 7f8a15d9b640 1 -- 172.21.15.143:0/1988424274 <== mon.2 v2:172.21.15.188:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8a10001a00 con 0x7f8a180cfa70 2023-11-26T07:18:37.025 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:37.021+0000 7f8a15d9b640 1 -- 172.21.15.143:0/1988424274 <== mon.2 v2:172.21.15.188:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8a080a7050 con 0x7f8a180cfa70 2023-11-26T07:18:37.025 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:37.021+0000 7f8a15d9b640 1 -- 172.21.15.143:0/1988424274 <== mon.2 v2:172.21.15.188:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8a080ab600 con 0x7f8a180cfa70 2023-11-26T07:18:37.026 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:37.021+0000 7f8a1759e640 1 -- 172.21.15.143:0/1988424274 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f8a180cfa70 msgr2=0x7f8a180d0e40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:18:37.026 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:37.021+0000 7f8a1759e640 1 --2- 172.21.15.143:0/1988424274 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f8a180cfa70 0x7f8a180d0e40 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7f8a080097f0 tx=0x7f8a0801c3b0 comp rx=0 tx=0).stop 2023-11-26T07:18:37.026 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:37.021+0000 7f8a1759e640 1 -- 172.21.15.143:0/1988424274 shutdown_connections 2023-11-26T07:18:37.026 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:37.021+0000 7f8a1759e640 1 --2- 172.21.15.143:0/1988424274 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f8a180cfa70 0x7f8a180d0e40 unknown :-1 s=CLOSED pgs=65 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:18:37.027 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:37.021+0000 7f8a1759e640 1 -- 172.21.15.143:0/1988424274 >> 172.21.15.143:0/1988424274 conn(0x7f8a180d3060 msgr2=0x7f8a180d3470 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:18:37.027 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:37.021+0000 7f8a1759e640 1 -- 172.21.15.143:0/1988424274 shutdown_connections 2023-11-26T07:18:37.027 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:37.021+0000 7f8a1759e640 1 -- 172.21.15.143:0/1988424274 wait complete. 2023-11-26T07:18:37.028 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:37.021+0000 7f8a1759e640 1 Processor -- start 2023-11-26T07:18:37.028 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:37.021+0000 7f8a1759e640 1 -- start start 2023-11-26T07:18:37.029 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:37.025+0000 7f8a1759e640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f8a180ced30 0x7f8a180b0ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:18:37.029 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:37.025+0000 7f8a1759e640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f8a180cfa70 0x7f8a181535b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:18:37.030 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:37.025+0000 7f8a1759e640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f8a180cfe50 0x7f8a18153af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:18:37.030 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:37.025+0000 7f8a1759e640 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8a18154030 con 0x7f8a180cfa70 2023-11-26T07:18:37.030 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:37.025+0000 7f8a1759e640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8a18156470 con 0x7f8a180ced30 2023-11-26T07:18:37.030 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:37.025+0000 7f8a1759e640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8a18160040 con 0x7f8a180cfe50 2023-11-26T07:18:37.030 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:37.025+0000 7f8a0ffff640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f8a180cfa70 0x7f8a181535b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:18:37.030 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:37.025+0000 7f8a0ffff640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f8a180cfa70 0x7f8a181535b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:38894/0 (socket says 172.21.15.143:38894) 2023-11-26T07:18:37.030 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:37.025+0000 7f8a0ffff640 1 -- 172.21.15.143:0/1725582354 learned_addr learned my addr 172.21.15.143:0/1725582354 (peer_addr_for_me v2:172.21.15.143:0/0) 2023-11-26T07:18:37.030 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:37.025+0000 7f8a1659c640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f8a180ced30 0x7f8a180b0ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:18:37.030 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:37.025+0000 7f8a16d9d640 1 --2- 172.21.15.143:0/1725582354 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f8a180cfe50 0x7f8a18153af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:18:37.030 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:37.025+0000 7f8a0ffff640 1 -- 172.21.15.143:0/1725582354 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f8a180cfe50 msgr2=0x7f8a18153af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:18:37.031 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:37.025+0000 7f8a0ffff640 1 --2- 172.21.15.143:0/1725582354 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f8a180cfe50 0x7f8a18153af0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:18:37.031 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:37.025+0000 7f8a0ffff640 1 -- 172.21.15.143:0/1725582354 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f8a180ced30 msgr2=0x7f8a180b0ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:18:37.031 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:37.025+0000 7f8a0ffff640 1 --2- 172.21.15.143:0/1725582354 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f8a180ced30 0x7f8a180b0ab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:18:37.031 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:37.025+0000 7f8a0ffff640 1 -- 172.21.15.143:0/1725582354 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8a08009420 con 0x7f8a180cfa70 2023-11-26T07:18:37.032 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:37.025+0000 7f8a0ffff640 1 --2- 172.21.15.143:0/1725582354 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f8a180cfa70 0x7f8a181535b0 secure :-1 s=READY pgs=319 cs=0 l=1 rev1=1 crypto rx=0x7f8a0000c430 tx=0x7f8a0000c8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:18:37.032 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:37.025+0000 7f8a0effd640 1 -- 172.21.15.143:0/1725582354 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8a000052d0 con 0x7f8a180cfa70 2023-11-26T07:18:37.032 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:37.025+0000 7f8a0effd640 1 -- 172.21.15.143:0/1725582354 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8a00005450 con 0x7f8a180cfa70 2023-11-26T07:18:37.032 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:37.025+0000 7f8a1759e640 1 -- 172.21.15.143:0/1725582354 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8a18160320 con 0x7f8a180cfa70 2023-11-26T07:18:37.032 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:37.025+0000 7f8a0effd640 1 -- 172.21.15.143:0/1725582354 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8a00003840 con 0x7f8a180cfa70 2023-11-26T07:18:37.032 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:37.025+0000 7f8a1759e640 1 -- 172.21.15.143:0/1725582354 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8a181608e0 con 0x7f8a180cfa70 2023-11-26T07:18:37.032 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:37.025+0000 7f8a1759e640 1 -- 172.21.15.143:0/1725582354 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8a18067500 con 0x7f8a180cfa70 2023-11-26T07:18:37.033 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:37.025+0000 7f8a0effd640 1 -- 172.21.15.143:0/1725582354 <== mon.0 v2:172.21.15.143:3300/0 4 ==== mgrmap(e 5) v1 ==== 85852+0+0 (secure 0 0 0) 0x7f8a000039a0 con 0x7f8a180cfa70 2023-11-26T07:18:37.033 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:37.029+0000 7f8a0effd640 1 --2- 172.21.15.143:0/1725582354 >> [v2:172.21.15.143:6832/2150557544,v1:172.21.15.143:6833/2150557544] conn(0x7f89f8066470 0x7f89f8068930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:18:37.033 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:37.029+0000 7f8a0effd640 1 -- 172.21.15.143:0/1725582354 <== mon.0 v2:172.21.15.143:3300/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 5825+0+0 (secure 0 0 0) 0x7f8a00083480 con 0x7f8a180cfa70 2023-11-26T07:18:37.033 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:37.029+0000 7f8a1659c640 1 --2- 172.21.15.143:0/1725582354 >> [v2:172.21.15.143:6832/2150557544,v1:172.21.15.143:6833/2150557544] conn(0x7f89f8066470 0x7f89f8068930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:18:37.034 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:37.029+0000 7f8a1659c640 1 --2- 172.21.15.143:0/1725582354 >> [v2:172.21.15.143:6832/2150557544,v1:172.21.15.143:6833/2150557544] conn(0x7f89f8066470 0x7f89f8068930 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7f8a080ad000 tx=0x7f8a08008040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:18:37.037 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:37.033+0000 7f8a0effd640 1 -- 172.21.15.143:0/1725582354 <== mon.0 v2:172.21.15.143:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172649 (secure 0 0 0) 0x7f8a00050930 con 0x7f8a180cfa70 2023-11-26T07:18:37.186 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:37.181+0000 7f8a1759e640 1 -- 172.21.15.143:0/1725582354 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f8a1808b0b0 con 0x7f8a180cfa70 2023-11-26T07:18:37.186 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:37.181+0000 7f8a0effd640 1 -- 172.21.15.143:0/1725582354 <== mon.0 v2:172.21.15.143:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v22) v1 ==== 93+0+10 (secure 0 0 0) 0x7f8a00053f50 con 0x7f8a180cfa70 2023-11-26T07:18:37.186 INFO:teuthology.orchestra.run.smithi143.stdout:pg_num: 1 2023-11-26T07:18:37.188 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:37.181+0000 7f8a1759e640 1 -- 172.21.15.143:0/1725582354 >> [v2:172.21.15.143:6832/2150557544,v1:172.21.15.143:6833/2150557544] conn(0x7f89f8066470 msgr2=0x7f89f8068930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:18:37.188 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:37.181+0000 7f8a1759e640 1 --2- 172.21.15.143:0/1725582354 >> [v2:172.21.15.143:6832/2150557544,v1:172.21.15.143:6833/2150557544] conn(0x7f89f8066470 0x7f89f8068930 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7f8a080ad000 tx=0x7f8a08008040 comp rx=0 tx=0).stop 2023-11-26T07:18:37.188 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:37.181+0000 7f8a1759e640 1 -- 172.21.15.143:0/1725582354 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f8a180cfa70 msgr2=0x7f8a181535b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:18:37.188 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:37.181+0000 7f8a1759e640 1 --2- 172.21.15.143:0/1725582354 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f8a180cfa70 0x7f8a181535b0 secure :-1 s=READY pgs=319 cs=0 l=1 rev1=1 crypto rx=0x7f8a0000c430 tx=0x7f8a0000c8f0 comp rx=0 tx=0).stop 2023-11-26T07:18:37.188 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:37.181+0000 7f8a1759e640 1 -- 172.21.15.143:0/1725582354 shutdown_connections 2023-11-26T07:18:37.188 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:37.181+0000 7f8a1759e640 1 --2- 172.21.15.143:0/1725582354 >> [v2:172.21.15.143:6832/2150557544,v1:172.21.15.143:6833/2150557544] conn(0x7f89f8066470 0x7f89f8068930 unknown :-1 s=CLOSED pgs=179 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:18:37.188 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:37.181+0000 7f8a1759e640 1 --2- 172.21.15.143:0/1725582354 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f8a180cfe50 0x7f8a18153af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:18:37.189 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:37.181+0000 7f8a1759e640 1 --2- 172.21.15.143:0/1725582354 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f8a180cfa70 0x7f8a181535b0 unknown :-1 s=CLOSED pgs=319 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:18:37.189 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:37.181+0000 7f8a1759e640 1 --2- 172.21.15.143:0/1725582354 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f8a180ced30 0x7f8a180b0ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:18:37.189 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:37.181+0000 7f8a1759e640 1 -- 172.21.15.143:0/1725582354 >> 172.21.15.143:0/1725582354 conn(0x7f8a180d3060 msgr2=0x7f8a180b2440 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:18:37.189 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:37.181+0000 7f8a1759e640 1 -- 172.21.15.143:0/1725582354 shutdown_connections 2023-11-26T07:18:37.189 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:37.181+0000 7f8a1759e640 1 -- 172.21.15.143:0/1725582354 wait complete. 2023-11-26T07:18:37.207 DEBUG:teuthology.orchestra.run.smithi143:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-11-26T07:18:37.347 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:37.341+0000 7fe64c3c1640 1 Processor -- start 2023-11-26T07:18:37.348 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:37.341+0000 7fe64c3c1640 1 -- start start 2023-11-26T07:18:37.349 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:37.345+0000 7fe64c3c1640 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7fe6440cf990 0x7fe64415efa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:18:37.349 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:37.345+0000 7fe64c3c1640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fe64415f4e0 0x7fe6440cd7b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:18:37.349 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:37.345+0000 7fe64c3c1640 1 -- --> v1:172.21.15.143:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe64406c520 con 0x7fe64413e290 2023-11-26T07:18:37.349 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:37.345+0000 7fe64c3c1640 1 -- --> v2:172.21.15.143:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe64406c800 con 0x7fe6440cf990 2023-11-26T07:18:37.349 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:37.345+0000 7fe64c3c1640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe64406cae0 con 0x7fe64415f4e0 2023-11-26T07:18:37.350 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:37.345+0000 7fe64995c640 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7fe6440cf990 0x7fe64415efa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:18:37.350 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:37.345+0000 7fe64a15d640 1 --1- >> v1:172.21.15.143:6789/0 conn(0x7fe64413e290 0x7fe6440cf280 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.143:6789/0 says I am v1:172.21.15.143:50920/0 (socket says 172.21.15.143:50920) 2023-11-26T07:18:37.350 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:37.345+0000 7fe64a15d640 1 -- 172.21.15.143:0/3485197227 learned_addr learned my addr 172.21.15.143:0/3485197227 (peer_addr_for_me v1:172.21.15.143:0/0) 2023-11-26T07:18:37.350 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:37.345+0000 7fe64995c640 1 -- 172.21.15.143:0/3485197227 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fe64415f4e0 msgr2=0x7fe6440cd7b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-26T07:18:37.350 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:37.345+0000 7fe64a95e640 1 --2- 172.21.15.143:0/3485197227 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fe64415f4e0 0x7fe6440cd7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:18:37.351 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:37.345+0000 7fe64995c640 1 --2- 172.21.15.143:0/3485197227 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fe64415f4e0 0x7fe6440cd7b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:18:37.351 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:37.345+0000 7fe64995c640 1 -- 172.21.15.143:0/3485197227 >> v1:172.21.15.143:6789/0 conn(0x7fe64413e290 legacy=0x7fe6440cf280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:18:37.351 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:37.345+0000 7fe64995c640 1 -- 172.21.15.143:0/3485197227 --> v2:172.21.15.143:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe64406cdc0 con 0x7fe6440cf990 2023-11-26T07:18:37.351 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:37.345+0000 7fe64a95e640 1 --2- 172.21.15.143:0/3485197227 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fe64415f4e0 0x7fe6440cd7b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-26T07:18:37.352 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:37.345+0000 7fe64995c640 1 --2- 172.21.15.143:0/3485197227 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fe6440cf990 0x7fe64415efa0 secure :-1 s=READY pgs=321 cs=0 l=1 rev1=1 crypto rx=0x7fe6340097c0 tx=0x7fe63401c450 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a4939fb91d6a8c7b server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:18:37.353 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:37.345+0000 7fe64915b640 1 -- 172.21.15.143:0/3485197227 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe6340ab670 con 0x7fe6440cf990 2023-11-26T07:18:37.353 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:37.345+0000 7fe64915b640 1 -- 172.21.15.143:0/3485197227 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe6340ab7d0 con 0x7fe6440cf990 2023-11-26T07:18:37.353 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:37.345+0000 7fe64915b640 1 -- 172.21.15.143:0/3485197227 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe6340abb60 con 0x7fe6440cf990 2023-11-26T07:18:37.354 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:37.349+0000 7fe64c3c1640 1 -- 172.21.15.143:0/3485197227 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fe6440cf990 msgr2=0x7fe64415efa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:18:37.354 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:37.349+0000 7fe64c3c1640 1 --2- 172.21.15.143:0/3485197227 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fe6440cf990 0x7fe64415efa0 secure :-1 s=READY pgs=321 cs=0 l=1 rev1=1 crypto rx=0x7fe6340097c0 tx=0x7fe63401c450 comp rx=0 tx=0).stop 2023-11-26T07:18:37.354 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:37.349+0000 7fe64c3c1640 1 -- 172.21.15.143:0/3485197227 shutdown_connections 2023-11-26T07:18:37.354 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:37.349+0000 7fe64c3c1640 1 --2- 172.21.15.143:0/3485197227 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fe64415f4e0 0x7fe6440cd7b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:18:37.354 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:37.349+0000 7fe64c3c1640 1 --2- 172.21.15.143:0/3485197227 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fe6440cf990 0x7fe64415efa0 unknown :-1 s=CLOSED pgs=321 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:18:37.354 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:37.349+0000 7fe64c3c1640 1 -- 172.21.15.143:0/3485197227 >> 172.21.15.143:0/3485197227 conn(0x7fe64406a890 msgr2=0x7fe64406aca0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:18:37.354 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:37.349+0000 7fe64c3c1640 1 -- 172.21.15.143:0/3485197227 shutdown_connections 2023-11-26T07:18:37.354 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:37.349+0000 7fe64c3c1640 1 -- 172.21.15.143:0/3485197227 wait complete. 2023-11-26T07:18:37.355 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:37.349+0000 7fe64c3c1640 1 Processor -- start 2023-11-26T07:18:37.355 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:37.349+0000 7fe64c3c1640 1 -- start start 2023-11-26T07:18:37.357 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:37.349+0000 7fe64c3c1640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fe6440cf990 0x7fe6440b0d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:18:37.358 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:37.349+0000 7fe64c3c1640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fe64413e290 0x7fe64415a3b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:18:37.358 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:37.349+0000 7fe64c3c1640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fe64415f4e0 0x7fe644135a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:18:37.358 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:37.349+0000 7fe64c3c1640 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe6441549a0 con 0x7fe64413e290 2023-11-26T07:18:37.358 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:37.349+0000 7fe64c3c1640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe6441365d0 con 0x7fe64415f4e0 2023-11-26T07:18:37.358 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:37.349+0000 7fe64c3c1640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe6441368b0 con 0x7fe6440cf990 2023-11-26T07:18:37.358 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:37.349+0000 7fe64995c640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fe64413e290 0x7fe64415a3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:18:37.358 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:37.349+0000 7fe64995c640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fe64413e290 0x7fe64415a3b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:38904/0 (socket says 172.21.15.143:38904) 2023-11-26T07:18:37.358 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:37.349+0000 7fe64995c640 1 -- 172.21.15.143:0/759683921 learned_addr learned my addr 172.21.15.143:0/759683921 (peer_addr_for_me v2:172.21.15.143:0/0) 2023-11-26T07:18:37.358 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:37.349+0000 7fe64a15d640 1 --2- 172.21.15.143:0/759683921 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fe6440cf990 0x7fe6440b0d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:18:37.358 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:37.353+0000 7fe64a95e640 1 --2- 172.21.15.143:0/759683921 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fe64415f4e0 0x7fe644135a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:18:37.358 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:37.353+0000 7fe64995c640 1 -- 172.21.15.143:0/759683921 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fe6440cf990 msgr2=0x7fe6440b0d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:18:37.358 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:37.353+0000 7fe64995c640 1 --2- 172.21.15.143:0/759683921 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fe6440cf990 0x7fe6440b0d40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:18:37.359 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:37.353+0000 7fe64995c640 1 -- 172.21.15.143:0/759683921 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fe64415f4e0 msgr2=0x7fe644135a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:18:37.359 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:37.353+0000 7fe64995c640 1 --2- 172.21.15.143:0/759683921 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fe64415f4e0 0x7fe644135a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:18:37.359 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:37.353+0000 7fe64995c640 1 -- 172.21.15.143:0/759683921 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe634009420 con 0x7fe64413e290 2023-11-26T07:18:37.359 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:37.353+0000 7fe64995c640 1 --2- 172.21.15.143:0/759683921 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fe64413e290 0x7fe64415a3b0 secure :-1 s=READY pgs=322 cs=0 l=1 rev1=1 crypto rx=0x7fe6340ad040 tx=0x7fe63401ac90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:18:37.359 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:37.353+0000 7fe63affd640 1 -- 172.21.15.143:0/759683921 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe6340b5810 con 0x7fe64413e290 2023-11-26T07:18:37.359 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:37.353+0000 7fe63affd640 1 -- 172.21.15.143:0/759683921 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe6340b5990 con 0x7fe64413e290 2023-11-26T07:18:37.359 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:37.353+0000 7fe64c3c1640 1 -- 172.21.15.143:0/759683921 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe6440b0260 con 0x7fe64413e290 2023-11-26T07:18:37.359 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:37.353+0000 7fe63affd640 1 -- 172.21.15.143:0/759683921 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe6340b5d40 con 0x7fe64413e290 2023-11-26T07:18:37.359 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:37.353+0000 7fe64c3c1640 1 -- 172.21.15.143:0/759683921 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe6440b0820 con 0x7fe64413e290 2023-11-26T07:18:37.360 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:37.353+0000 7fe64c3c1640 1 -- 172.21.15.143:0/759683921 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe6440b0a10 con 0x7fe64413e290 2023-11-26T07:18:37.360 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:37.353+0000 7fe63affd640 1 -- 172.21.15.143:0/759683921 <== mon.0 v2:172.21.15.143:3300/0 4 ==== mgrmap(e 5) v1 ==== 85852+0+0 (secure 0 0 0) 0x7fe6340bf410 con 0x7fe64413e290 2023-11-26T07:18:37.360 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:37.353+0000 7fe63affd640 1 --2- 172.21.15.143:0/759683921 >> [v2:172.21.15.143:6832/2150557544,v1:172.21.15.143:6833/2150557544] conn(0x7fe628066470 0x7fe628068930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:18:37.360 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:37.353+0000 7fe63affd640 1 -- 172.21.15.143:0/759683921 <== mon.0 v2:172.21.15.143:3300/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 5825+0+0 (secure 0 0 0) 0x7fe64415fb00 con 0x7fe64413e290 2023-11-26T07:18:37.360 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:37.353+0000 7fe64a15d640 1 --2- 172.21.15.143:0/759683921 >> [v2:172.21.15.143:6832/2150557544,v1:172.21.15.143:6833/2150557544] conn(0x7fe628066470 0x7fe628068930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:18:37.361 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:37.357+0000 7fe64a15d640 1 --2- 172.21.15.143:0/759683921 >> [v2:172.21.15.143:6832/2150557544,v1:172.21.15.143:6833/2150557544] conn(0x7fe628066470 0x7fe628068930 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7fe640005890 tx=0x7fe640009040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:18:37.364 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:37.357+0000 7fe63affd640 1 -- 172.21.15.143:0/759683921 <== mon.0 v2:172.21.15.143:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172649 (secure 0 0 0) 0x7fe6340ec2b0 con 0x7fe64413e290 2023-11-26T07:18:37.520 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:37.513+0000 7fe64c3c1640 1 -- 172.21.15.143:0/759683921 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7fe6441369f0 con 0x7fe64413e290 2023-11-26T07:18:37.521 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:37.517+0000 7fe63affd640 1 -- 172.21.15.143:0/759683921 <== mon.0 v2:172.21.15.143:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v22) v1 ==== 104+0+11 (secure 0 0 0) 0x7fe6441369f0 con 0x7fe64413e290 2023-11-26T07:18:37.521 INFO:teuthology.orchestra.run.smithi143.stdout:pg_num: 32 2023-11-26T07:18:37.523 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:37.517+0000 7fe64c3c1640 1 -- 172.21.15.143:0/759683921 >> [v2:172.21.15.143:6832/2150557544,v1:172.21.15.143:6833/2150557544] conn(0x7fe628066470 msgr2=0x7fe628068930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:18:37.523 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:37.517+0000 7fe64c3c1640 1 --2- 172.21.15.143:0/759683921 >> [v2:172.21.15.143:6832/2150557544,v1:172.21.15.143:6833/2150557544] conn(0x7fe628066470 0x7fe628068930 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7fe640005890 tx=0x7fe640009040 comp rx=0 tx=0).stop 2023-11-26T07:18:37.523 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:37.517+0000 7fe64c3c1640 1 -- 172.21.15.143:0/759683921 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fe64413e290 msgr2=0x7fe64415a3b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:18:37.523 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:37.517+0000 7fe64c3c1640 1 --2- 172.21.15.143:0/759683921 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fe64413e290 0x7fe64415a3b0 secure :-1 s=READY pgs=322 cs=0 l=1 rev1=1 crypto rx=0x7fe6340ad040 tx=0x7fe63401ac90 comp rx=0 tx=0).stop 2023-11-26T07:18:37.523 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:37.517+0000 7fe64c3c1640 1 -- 172.21.15.143:0/759683921 shutdown_connections 2023-11-26T07:18:37.523 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:37.517+0000 7fe64c3c1640 1 --2- 172.21.15.143:0/759683921 >> [v2:172.21.15.143:6832/2150557544,v1:172.21.15.143:6833/2150557544] conn(0x7fe628066470 0x7fe628068930 unknown :-1 s=CLOSED pgs=180 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:18:37.524 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:37.517+0000 7fe64c3c1640 1 --2- 172.21.15.143:0/759683921 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fe64415f4e0 0x7fe644135a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:18:37.524 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:37.517+0000 7fe64c3c1640 1 --2- 172.21.15.143:0/759683921 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fe64413e290 0x7fe64415a3b0 unknown :-1 s=CLOSED pgs=322 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:18:37.524 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:37.517+0000 7fe64c3c1640 1 --2- 172.21.15.143:0/759683921 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fe6440cf990 0x7fe6440b0d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:18:37.524 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:37.517+0000 7fe64c3c1640 1 -- 172.21.15.143:0/759683921 >> 172.21.15.143:0/759683921 conn(0x7fe64406a890 msgr2=0x7fe6440cf250 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:18:37.524 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:37.517+0000 7fe64c3c1640 1 -- 172.21.15.143:0/759683921 shutdown_connections 2023-11-26T07:18:37.524 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:37.517+0000 7fe64c3c1640 1 -- 172.21.15.143:0/759683921 wait complete. 2023-11-26T07:18:37.542 DEBUG:teuthology.orchestra.run.smithi143:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-11-26T07:18:37.683 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:37.677+0000 7fb8cbbb8640 1 Processor -- start 2023-11-26T07:18:37.683 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:37.677+0000 7fb8cbbb8640 1 -- start start 2023-11-26T07:18:37.685 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:37.677+0000 7fb8cbbb8640 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7fb8c40cd610 0x7fb8c4071970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:18:37.685 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:37.677+0000 7fb8cbbb8640 1 -- --> v1:172.21.15.143:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb8c4071eb0 con 0x7fb8c41614b0 2023-11-26T07:18:37.685 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:37.677+0000 7fb8cbbb8640 1 -- --> v1:172.21.15.188:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb8c415d970 con 0x7fb8c4070ba0 2023-11-26T07:18:37.685 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:37.677+0000 7fb8cbbb8640 1 -- --> v2:172.21.15.188:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb8c415dc30 con 0x7fb8c40cd610 2023-11-26T07:18:37.685 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:37.677+0000 7fb8c9954640 1 --1- >> v1:172.21.15.143:6789/0 conn(0x7fb8c41614b0 0x7fb8c4161890 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.143:6789/0 says I am v1:172.21.15.143:50934/0 (socket says 172.21.15.143:50934) 2023-11-26T07:18:37.685 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:37.677+0000 7fb8c9954640 1 -- 172.21.15.143:0/401654000 learned_addr learned my addr 172.21.15.143:0/401654000 (peer_addr_for_me v1:172.21.15.143:0/0) 2023-11-26T07:18:37.685 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:37.681+0000 7fb8c9153640 1 --2- 172.21.15.143:0/401654000 >> v2:172.21.15.188:3300/0 conn(0x7fb8c40cd610 0x7fb8c4071970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:18:37.686 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:37.681+0000 7fb8c8952640 1 -- 172.21.15.143:0/401654000 <== mon.0 v1:172.21.15.143:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 294663722 0 0) 0x7fb8c4071eb0 con 0x7fb8c41614b0 2023-11-26T07:18:37.686 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:37.681+0000 7fb8c8952640 1 -- 172.21.15.143:0/401654000 --> v1:172.21.15.143:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fb8b0003530 con 0x7fb8c41614b0 2023-11-26T07:18:37.686 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:37.681+0000 7fb8c9153640 1 -- 172.21.15.143:0/401654000 >> v1:172.21.15.188:6789/0 conn(0x7fb8c4070ba0 legacy=0x7fb8c4070fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:18:37.686 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:37.681+0000 7fb8c9153640 1 -- 172.21.15.143:0/401654000 >> v1:172.21.15.143:6789/0 conn(0x7fb8c41614b0 legacy=0x7fb8c4161890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:18:37.686 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:37.681+0000 7fb8c9153640 1 -- 172.21.15.143:0/401654000 --> v2:172.21.15.188:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb8c415df10 con 0x7fb8c40cd610 2023-11-26T07:18:37.688 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:37.681+0000 7fb8c9153640 1 --2- 172.21.15.143:0/401654000 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fb8c40cd610 0x7fb8c4071970 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7fb8b40097c0 tx=0x7fb8b401c450 comp rx=0 tx=0).ready entity=mon.1 client_cookie=6ce8168abea11b7d server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:18:37.689 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:37.685+0000 7fb8c8952640 1 -- 172.21.15.143:0/401654000 <== mon.1 v2:172.21.15.188:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb8b40ab650 con 0x7fb8c40cd610 2023-11-26T07:18:37.689 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:37.685+0000 7fb8c8952640 1 -- 172.21.15.143:0/401654000 <== mon.1 v2:172.21.15.188:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb8b40ab7b0 con 0x7fb8c40cd610 2023-11-26T07:18:37.689 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:37.685+0000 7fb8c8952640 1 -- 172.21.15.143:0/401654000 <== mon.1 v2:172.21.15.188:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb8b40abb40 con 0x7fb8c40cd610 2023-11-26T07:18:37.690 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:37.685+0000 7fb8cbbb8640 1 -- 172.21.15.143:0/401654000 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fb8c40cd610 msgr2=0x7fb8c4071970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:18:37.690 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:37.685+0000 7fb8cbbb8640 1 --2- 172.21.15.143:0/401654000 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fb8c40cd610 0x7fb8c4071970 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7fb8b40097c0 tx=0x7fb8b401c450 comp rx=0 tx=0).stop 2023-11-26T07:18:37.690 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:37.685+0000 7fb8cbbb8640 1 -- 172.21.15.143:0/401654000 shutdown_connections 2023-11-26T07:18:37.690 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:37.685+0000 7fb8cbbb8640 1 --2- 172.21.15.143:0/401654000 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fb8c40cd610 0x7fb8c4071970 unknown :-1 s=CLOSED pgs=153 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:18:37.690 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:37.685+0000 7fb8cbbb8640 1 -- 172.21.15.143:0/401654000 >> 172.21.15.143:0/401654000 conn(0x7fb8c40cf710 msgr2=0x7fb8c40cfb20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:18:37.690 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:37.685+0000 7fb8cbbb8640 1 -- 172.21.15.143:0/401654000 shutdown_connections 2023-11-26T07:18:37.690 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:37.685+0000 7fb8cbbb8640 1 -- 172.21.15.143:0/401654000 wait complete. 2023-11-26T07:18:37.691 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:37.685+0000 7fb8cbbb8640 1 Processor -- start 2023-11-26T07:18:37.691 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:37.685+0000 7fb8cbbb8640 1 -- start start 2023-11-26T07:18:37.693 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:37.685+0000 7fb8cbbb8640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fb8c4070ba0 0x7fb8c40b06f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:18:37.693 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:37.685+0000 7fb8cbbb8640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fb8c40cd610 0x7fb8c40af2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:18:37.693 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:37.685+0000 7fb8cbbb8640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fb8c41614b0 0x7fb8c40af830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:18:37.693 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:37.685+0000 7fb8cbbb8640 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb8c4153510 con 0x7fb8c41614b0 2023-11-26T07:18:37.693 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:37.685+0000 7fb8cbbb8640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb8c4150700 con 0x7fb8c4070ba0 2023-11-26T07:18:37.693 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:37.685+0000 7fb8cbbb8640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb8c41540e0 con 0x7fb8c40cd610 2023-11-26T07:18:37.693 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:37.685+0000 7fb8ca155640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fb8c41614b0 0x7fb8c40af830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:18:37.694 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:37.685+0000 7fb8ca155640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fb8c41614b0 0x7fb8c40af830 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:38918/0 (socket says 172.21.15.143:38918) 2023-11-26T07:18:37.694 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:37.685+0000 7fb8ca155640 1 -- 172.21.15.143:0/2797950580 learned_addr learned my addr 172.21.15.143:0/2797950580 (peer_addr_for_me v2:172.21.15.143:0/0) 2023-11-26T07:18:37.694 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:37.689+0000 7fb8c9954640 1 --2- 172.21.15.143:0/2797950580 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fb8c4070ba0 0x7fb8c40b06f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:18:37.694 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:37.689+0000 7fb8c9153640 1 --2- 172.21.15.143:0/2797950580 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fb8c40cd610 0x7fb8c40af2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:18:37.694 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:37.689+0000 7fb8ca155640 1 -- 172.21.15.143:0/2797950580 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fb8c40cd610 msgr2=0x7fb8c40af2f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:18:37.694 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:37.689+0000 7fb8ca155640 1 --2- 172.21.15.143:0/2797950580 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fb8c40cd610 0x7fb8c40af2f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:18:37.694 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:37.689+0000 7fb8ca155640 1 -- 172.21.15.143:0/2797950580 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fb8c4070ba0 msgr2=0x7fb8c40b06f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:18:37.694 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:37.689+0000 7fb8ca155640 1 --2- 172.21.15.143:0/2797950580 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fb8c4070ba0 0x7fb8c40b06f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:18:37.694 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:37.689+0000 7fb8ca155640 1 -- 172.21.15.143:0/2797950580 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb8b4009420 con 0x7fb8c41614b0 2023-11-26T07:18:37.694 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:37.689+0000 7fb8c9153640 1 --2- 172.21.15.143:0/2797950580 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fb8c40cd610 0x7fb8c40af2f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-26T07:18:37.694 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:37.689+0000 7fb8c9954640 1 --2- 172.21.15.143:0/2797950580 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fb8c4070ba0 0x7fb8c40b06f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-26T07:18:37.695 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:37.689+0000 7fb8ca155640 1 --2- 172.21.15.143:0/2797950580 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fb8c41614b0 0x7fb8c40af830 secure :-1 s=READY pgs=324 cs=0 l=1 rev1=1 crypto rx=0x7fb8bc00c730 tx=0x7fb8bc00cbf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:18:37.696 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:37.689+0000 7fb8ba7fc640 1 -- 172.21.15.143:0/2797950580 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb8bc003b80 con 0x7fb8c41614b0 2023-11-26T07:18:37.696 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:37.689+0000 7fb8ba7fc640 1 -- 172.21.15.143:0/2797950580 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb8bc003ce0 con 0x7fb8c41614b0 2023-11-26T07:18:37.696 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:37.689+0000 7fb8ba7fc640 1 -- 172.21.15.143:0/2797950580 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb8bc00aa30 con 0x7fb8c41614b0 2023-11-26T07:18:37.696 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:37.689+0000 7fb8cbbb8640 1 -- 172.21.15.143:0/2797950580 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb8c4153b20 con 0x7fb8c41614b0 2023-11-26T07:18:37.696 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:37.689+0000 7fb8cbbb8640 1 -- 172.21.15.143:0/2797950580 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb8c4152d10 con 0x7fb8c41614b0 2023-11-26T07:18:37.696 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:37.689+0000 7fb8cbbb8640 1 -- 172.21.15.143:0/2797950580 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb8c4161320 con 0x7fb8c41614b0 2023-11-26T07:18:37.696 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:37.689+0000 7fb8ba7fc640 1 -- 172.21.15.143:0/2797950580 <== mon.0 v2:172.21.15.143:3300/0 4 ==== mgrmap(e 5) v1 ==== 85852+0+0 (secure 0 0 0) 0x7fb8bc00ab90 con 0x7fb8c41614b0 2023-11-26T07:18:37.697 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:37.689+0000 7fb8ba7fc640 1 --2- 172.21.15.143:0/2797950580 >> [v2:172.21.15.143:6832/2150557544,v1:172.21.15.143:6833/2150557544] conn(0x7fb8a8066470 0x7fb8a8068930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:18:37.697 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:37.689+0000 7fb8ba7fc640 1 -- 172.21.15.143:0/2797950580 <== mon.0 v2:172.21.15.143:3300/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 5825+0+0 (secure 0 0 0) 0x7fb8bc083dd0 con 0x7fb8c41614b0 2023-11-26T07:18:37.697 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:37.689+0000 7fb8c9954640 1 --2- 172.21.15.143:0/2797950580 >> [v2:172.21.15.143:6832/2150557544,v1:172.21.15.143:6833/2150557544] conn(0x7fb8a8066470 0x7fb8a8068930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:18:37.698 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:37.693+0000 7fb8c9954640 1 --2- 172.21.15.143:0/2797950580 >> [v2:172.21.15.143:6832/2150557544,v1:172.21.15.143:6833/2150557544] conn(0x7fb8a8066470 0x7fb8a8068930 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7fb8c0005890 tx=0x7fb8c0009040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:18:37.700 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:37.693+0000 7fb8ba7fc640 1 -- 172.21.15.143:0/2797950580 <== mon.0 v2:172.21.15.143:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172649 (secure 0 0 0) 0x7fb8bc051130 con 0x7fb8c41614b0 2023-11-26T07:18:37.856 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:37.849+0000 7fb8cbbb8640 1 -- 172.21.15.143:0/2797950580 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7fb8c4152b70 con 0x7fb8c41614b0 2023-11-26T07:18:37.856 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:37.849+0000 7fb8ba7fc640 1 -- 172.21.15.143:0/2797950580 <== mon.0 v2:172.21.15.143:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v22) v1 ==== 100+0+11 (secure 0 0 0) 0x7fb8c4152b70 con 0x7fb8c41614b0 2023-11-26T07:18:37.857 INFO:teuthology.orchestra.run.smithi143.stdout:pg_num: 64 2023-11-26T07:18:37.859 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:37.853+0000 7fb8cbbb8640 1 -- 172.21.15.143:0/2797950580 >> [v2:172.21.15.143:6832/2150557544,v1:172.21.15.143:6833/2150557544] conn(0x7fb8a8066470 msgr2=0x7fb8a8068930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:18:37.859 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:37.853+0000 7fb8cbbb8640 1 --2- 172.21.15.143:0/2797950580 >> [v2:172.21.15.143:6832/2150557544,v1:172.21.15.143:6833/2150557544] conn(0x7fb8a8066470 0x7fb8a8068930 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7fb8c0005890 tx=0x7fb8c0009040 comp rx=0 tx=0).stop 2023-11-26T07:18:37.859 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:37.853+0000 7fb8cbbb8640 1 -- 172.21.15.143:0/2797950580 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fb8c41614b0 msgr2=0x7fb8c40af830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:18:37.859 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:37.853+0000 7fb8cbbb8640 1 --2- 172.21.15.143:0/2797950580 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fb8c41614b0 0x7fb8c40af830 secure :-1 s=READY pgs=324 cs=0 l=1 rev1=1 crypto rx=0x7fb8bc00c730 tx=0x7fb8bc00cbf0 comp rx=0 tx=0).stop 2023-11-26T07:18:37.859 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:37.853+0000 7fb8cbbb8640 1 -- 172.21.15.143:0/2797950580 shutdown_connections 2023-11-26T07:18:37.859 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:37.853+0000 7fb8cbbb8640 1 --2- 172.21.15.143:0/2797950580 >> [v2:172.21.15.143:6832/2150557544,v1:172.21.15.143:6833/2150557544] conn(0x7fb8a8066470 0x7fb8a8068930 unknown :-1 s=CLOSED pgs=181 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:18:37.860 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:37.853+0000 7fb8cbbb8640 1 --2- 172.21.15.143:0/2797950580 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fb8c41614b0 0x7fb8c40af830 unknown :-1 s=CLOSED pgs=324 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:18:37.860 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:37.853+0000 7fb8cbbb8640 1 --2- 172.21.15.143:0/2797950580 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fb8c40cd610 0x7fb8c40af2f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:18:37.860 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:37.853+0000 7fb8cbbb8640 1 --2- 172.21.15.143:0/2797950580 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fb8c4070ba0 0x7fb8c40b06f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:18:37.860 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:37.853+0000 7fb8cbbb8640 1 -- 172.21.15.143:0/2797950580 >> 172.21.15.143:0/2797950580 conn(0x7fb8c40cf710 msgr2=0x7fb8c40cfaf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:18:37.860 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:37.853+0000 7fb8cbbb8640 1 -- 172.21.15.143:0/2797950580 shutdown_connections 2023-11-26T07:18:37.860 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:37.853+0000 7fb8cbbb8640 1 -- 172.21.15.143:0/2797950580 wait complete. 2023-11-26T07:18:37.878 DEBUG:teuthology.orchestra.run.smithi143:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2023-11-26T07:18:38.018 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:38.013+0000 7f76a273e640 1 Processor -- start 2023-11-26T07:18:38.018 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:38.013+0000 7f76a273e640 1 -- start start 2023-11-26T07:18:38.019 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:38.013+0000 7f76a273e640 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7f769c143ac0 0x7f769c070370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:18:38.019 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:38.013+0000 7f76a273e640 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7f769c0cf990 0x7f769c0cd1b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:18:38.019 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:38.013+0000 7f76a273e640 1 -- --> v1:172.21.15.188:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f769c0cc730 con 0x7f769c0708b0 2023-11-26T07:18:38.020 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:38.013+0000 7f76a273e640 1 -- --> v2:172.21.15.143:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f769c143ea0 con 0x7f769c143ac0 2023-11-26T07:18:38.020 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:38.013+0000 7f76a273e640 1 -- --> v2:172.21.15.188:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f769c070ee0 con 0x7f769c0cf990 2023-11-26T07:18:38.020 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:38.013+0000 7f769bfff640 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7f769c143ac0 0x7f769c070370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:18:38.020 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:38.013+0000 7f769bfff640 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7f769c143ac0 0x7f769c070370 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:38934/0 (socket says 172.21.15.143:38934) 2023-11-26T07:18:38.020 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:38.013+0000 7f769bfff640 1 -- 172.21.15.143:0/3588211186 learned_addr learned my addr 172.21.15.143:0/3588211186 (peer_addr_for_me v2:172.21.15.143:0/0) 2023-11-26T07:18:38.021 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:38.013+0000 7f769bfff640 1 -- 172.21.15.143:0/3588211186 >> v2:172.21.15.188:3300/0 conn(0x7f769c0cf990 msgr2=0x7f769c0cd1b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-26T07:18:38.021 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:38.013+0000 7f76a0cdb640 1 --2- 172.21.15.143:0/3588211186 >> v2:172.21.15.188:3300/0 conn(0x7f769c0cf990 0x7f769c0cd1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:18:38.021 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:38.013+0000 7f769bfff640 1 --2- 172.21.15.143:0/3588211186 >> v2:172.21.15.188:3300/0 conn(0x7f769c0cf990 0x7f769c0cd1b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:18:38.021 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:38.013+0000 7f769bfff640 1 -- 172.21.15.143:0/3588211186 >> v1:172.21.15.188:6789/0 conn(0x7f769c0708b0 legacy=0x7f769c0cf280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-26T07:18:38.021 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:38.013+0000 7f769bfff640 1 -- 172.21.15.143:0/3588211186 --> v2:172.21.15.143:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f769c0711c0 con 0x7f769c143ac0 2023-11-26T07:18:38.022 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:38.013+0000 7f76a0cdb640 1 --2- 172.21.15.143:0/3588211186 >> v2:172.21.15.188:3300/0 conn(0x7f769c0cf990 0x7f769c0cd1b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-26T07:18:38.022 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:38.017+0000 7f769bfff640 1 --2- 172.21.15.143:0/3588211186 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f769c143ac0 0x7f769c070370 secure :-1 s=READY pgs=325 cs=0 l=1 rev1=1 crypto rx=0x7f769c06e570 tx=0x7f768c01c4c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9c5fa507a51b3842 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:18:38.023 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:38.017+0000 7f769affd640 1 -- 172.21.15.143:0/3588211186 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f768c0aa620 con 0x7f769c143ac0 2023-11-26T07:18:38.023 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:38.017+0000 7f769affd640 1 -- 172.21.15.143:0/3588211186 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f768c0aa780 con 0x7f769c143ac0 2023-11-26T07:18:38.023 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:38.017+0000 7f769affd640 1 -- 172.21.15.143:0/3588211186 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f768c0aab10 con 0x7f769c143ac0 2023-11-26T07:18:38.024 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:38.017+0000 7f76a273e640 1 -- 172.21.15.143:0/3588211186 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f769c143ac0 msgr2=0x7f769c070370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:18:38.024 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:38.017+0000 7f76a273e640 1 --2- 172.21.15.143:0/3588211186 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f769c143ac0 0x7f769c070370 secure :-1 s=READY pgs=325 cs=0 l=1 rev1=1 crypto rx=0x7f769c06e570 tx=0x7f768c01c4c0 comp rx=0 tx=0).stop 2023-11-26T07:18:38.024 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:38.017+0000 7f76a273e640 1 -- 172.21.15.143:0/3588211186 shutdown_connections 2023-11-26T07:18:38.024 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:38.017+0000 7f76a273e640 1 --2- 172.21.15.143:0/3588211186 >> v2:172.21.15.188:3300/0 conn(0x7f769c0cf990 0x7f769c0cd1b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:18:38.025 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:38.017+0000 7f76a273e640 1 --2- 172.21.15.143:0/3588211186 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f769c143ac0 0x7f769c070370 unknown :-1 s=CLOSED pgs=325 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:18:38.025 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:38.017+0000 7f76a273e640 1 -- 172.21.15.143:0/3588211186 >> 172.21.15.143:0/3588211186 conn(0x7f769c06a890 msgr2=0x7f769c06aca0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:18:38.025 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:38.017+0000 7f76a273e640 1 -- 172.21.15.143:0/3588211186 shutdown_connections 2023-11-26T07:18:38.025 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:38.017+0000 7f76a273e640 1 -- 172.21.15.143:0/3588211186 wait complete. 2023-11-26T07:18:38.026 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:38.021+0000 7f76a273e640 1 Processor -- start 2023-11-26T07:18:38.026 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:38.021+0000 7f76a273e640 1 -- start start 2023-11-26T07:18:38.028 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:38.021+0000 7f76a273e640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f769c0708b0 0x7f769c1387a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:18:38.028 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:38.021+0000 7f76a273e640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f769c0cf990 0x7f769c1f9610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:18:38.028 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:38.021+0000 7f76a273e640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f769c143ac0 0x7f769c1f9b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:18:38.028 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:38.021+0000 7f76a273e640 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f769c0af900 con 0x7f769c0708b0 2023-11-26T07:18:38.029 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:38.021+0000 7f76a273e640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f769c0afbe0 con 0x7f769c0cf990 2023-11-26T07:18:38.029 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:38.021+0000 7f76a273e640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f769c0afec0 con 0x7f769c143ac0 2023-11-26T07:18:38.029 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:38.021+0000 7f769bfff640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f769c0708b0 0x7f769c1387a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:18:38.029 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:38.021+0000 7f769bfff640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f769c0708b0 0x7f769c1387a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:38936/0 (socket says 172.21.15.143:38936) 2023-11-26T07:18:38.029 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:38.021+0000 7f769bfff640 1 -- 172.21.15.143:0/3056849440 learned_addr learned my addr 172.21.15.143:0/3056849440 (peer_addr_for_me v2:172.21.15.143:0/0) 2023-11-26T07:18:38.029 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:38.021+0000 7f76a0cdb640 1 --2- 172.21.15.143:0/3056849440 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f769c143ac0 0x7f769c1f9b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:18:38.029 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:38.021+0000 7f769b7fe640 1 --2- 172.21.15.143:0/3056849440 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f769c0cf990 0x7f769c1f9610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:18:38.029 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:38.021+0000 7f769bfff640 1 -- 172.21.15.143:0/3056849440 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f769c143ac0 msgr2=0x7f769c1f9b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:18:38.029 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:38.021+0000 7f769bfff640 1 --2- 172.21.15.143:0/3056849440 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f769c143ac0 0x7f769c1f9b50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:18:38.030 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:38.021+0000 7f769bfff640 1 -- 172.21.15.143:0/3056849440 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f769c0cf990 msgr2=0x7f769c1f9610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:18:38.030 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:38.021+0000 7f769bfff640 1 --2- 172.21.15.143:0/3056849440 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f769c0cf990 0x7f769c1f9610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:18:38.030 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:38.021+0000 7f769bfff640 1 -- 172.21.15.143:0/3056849440 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f768c0094d0 con 0x7f769c0708b0 2023-11-26T07:18:38.030 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:38.021+0000 7f769bfff640 1 --2- 172.21.15.143:0/3056849440 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f769c0708b0 0x7f769c1387a0 secure :-1 s=READY pgs=326 cs=0 l=1 rev1=1 crypto rx=0x7f768c01a0f0 tx=0x7f768c01ad00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:18:38.030 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:38.021+0000 7f7698ff9640 1 -- 172.21.15.143:0/3056849440 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f768c0bd3f0 con 0x7f769c0708b0 2023-11-26T07:18:38.030 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:38.021+0000 7f7698ff9640 1 -- 172.21.15.143:0/3056849440 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f768c0bd570 con 0x7f769c0708b0 2023-11-26T07:18:38.030 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:38.021+0000 7f7698ff9640 1 -- 172.21.15.143:0/3056849440 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f768c0bd920 con 0x7f769c0708b0 2023-11-26T07:18:38.030 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:38.021+0000 7f76a273e640 1 -- 172.21.15.143:0/3056849440 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f769c0b01a0 con 0x7f769c0708b0 2023-11-26T07:18:38.031 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:38.021+0000 7f76a273e640 1 -- 172.21.15.143:0/3056849440 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f769c138ce0 con 0x7f769c0708b0 2023-11-26T07:18:38.032 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:38.025+0000 7f76a273e640 1 -- 172.21.15.143:0/3056849440 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f769c067680 con 0x7f769c0708b0 2023-11-26T07:18:38.032 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:38.025+0000 7f7698ff9640 1 -- 172.21.15.143:0/3056849440 <== mon.0 v2:172.21.15.143:3300/0 4 ==== mgrmap(e 5) v1 ==== 85852+0+0 (secure 0 0 0) 0x7f768c0c6440 con 0x7f769c0708b0 2023-11-26T07:18:38.032 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:38.025+0000 7f7698ff9640 1 --2- 172.21.15.143:0/3056849440 >> [v2:172.21.15.143:6832/2150557544,v1:172.21.15.143:6833/2150557544] conn(0x7f7688066470 0x7f7688068930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:18:38.032 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:38.025+0000 7f7698ff9640 1 -- 172.21.15.143:0/3056849440 <== mon.0 v2:172.21.15.143:3300/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 5825+0+0 (secure 0 0 0) 0x7f768c11e700 con 0x7f769c0708b0 2023-11-26T07:18:38.032 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:38.025+0000 7f769b7fe640 1 --2- 172.21.15.143:0/3056849440 >> [v2:172.21.15.143:6832/2150557544,v1:172.21.15.143:6833/2150557544] conn(0x7f7688066470 0x7f7688068930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:18:38.032 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:38.025+0000 7f769b7fe640 1 --2- 172.21.15.143:0/3056849440 >> [v2:172.21.15.143:6832/2150557544,v1:172.21.15.143:6833/2150557544] conn(0x7f7688066470 0x7f7688068930 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7f769c0b1bd0 tx=0x7f7690009040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:18:38.034 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:38.029+0000 7f7698ff9640 1 -- 172.21.15.143:0/3056849440 <== mon.0 v2:172.21.15.143:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172649 (secure 0 0 0) 0x7f768c0ebc10 con 0x7f769c0708b0 2023-11-26T07:18:38.197 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:38.193+0000 7f76a273e640 1 -- 172.21.15.143:0/3056849440 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) v1 -- 0x7f769c13fbf0 con 0x7f769c0708b0 2023-11-26T07:18:38.197 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:38.193+0000 7f7698ff9640 1 -- 172.21.15.143:0/3056849440 <== mon.0 v2:172.21.15.143:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v22) v1 ==== 80+0+0 (secure 0 0 0) 0x7f768c0ef230 con 0x7f769c0708b0 2023-11-26T07:18:38.198 INFO:teuthology.orchestra.run.smithi143.stderr:listed 0 entries 2023-11-26T07:18:38.199 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:38.193+0000 7f76a273e640 1 -- 172.21.15.143:0/3056849440 >> [v2:172.21.15.143:6832/2150557544,v1:172.21.15.143:6833/2150557544] conn(0x7f7688066470 msgr2=0x7f7688068930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:18:38.200 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:38.193+0000 7f76a273e640 1 --2- 172.21.15.143:0/3056849440 >> [v2:172.21.15.143:6832/2150557544,v1:172.21.15.143:6833/2150557544] conn(0x7f7688066470 0x7f7688068930 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7f769c0b1bd0 tx=0x7f7690009040 comp rx=0 tx=0).stop 2023-11-26T07:18:38.200 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:38.193+0000 7f76a273e640 1 -- 172.21.15.143:0/3056849440 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f769c0708b0 msgr2=0x7f769c1387a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:18:38.200 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:38.193+0000 7f76a273e640 1 --2- 172.21.15.143:0/3056849440 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f769c0708b0 0x7f769c1387a0 secure :-1 s=READY pgs=326 cs=0 l=1 rev1=1 crypto rx=0x7f768c01a0f0 tx=0x7f768c01ad00 comp rx=0 tx=0).stop 2023-11-26T07:18:38.200 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:38.193+0000 7f76a273e640 1 -- 172.21.15.143:0/3056849440 shutdown_connections 2023-11-26T07:18:38.200 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:38.193+0000 7f76a273e640 1 --2- 172.21.15.143:0/3056849440 >> [v2:172.21.15.143:6832/2150557544,v1:172.21.15.143:6833/2150557544] conn(0x7f7688066470 0x7f7688068930 unknown :-1 s=CLOSED pgs=182 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:18:38.200 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:38.193+0000 7f76a273e640 1 --2- 172.21.15.143:0/3056849440 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f769c143ac0 0x7f769c1f9b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:18:38.200 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:38.193+0000 7f76a273e640 1 --2- 172.21.15.143:0/3056849440 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f769c0cf990 0x7f769c1f9610 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:18:38.200 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:38.193+0000 7f76a273e640 1 --2- 172.21.15.143:0/3056849440 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f769c0708b0 0x7f769c1387a0 unknown :-1 s=CLOSED pgs=326 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:18:38.201 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:38.193+0000 7f76a273e640 1 -- 172.21.15.143:0/3056849440 >> 172.21.15.143:0/3056849440 conn(0x7f769c06a890 msgr2=0x7f769c13f2c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:18:38.201 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:38.193+0000 7f76a273e640 1 -- 172.21.15.143:0/3056849440 shutdown_connections 2023-11-26T07:18:38.201 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:38.193+0000 7f76a273e640 1 -- 172.21.15.143:0/3056849440 wait complete. 2023-11-26T07:18:38.218 INFO:tasks.cephfs.fuse_mount:Running fusermount -u on ubuntu@smithi188.front.sepia.ceph.com... 2023-11-26T07:18:38.219 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-26T07:18:38.219 DEBUG:teuthology.orchestra.run.smithi188:> sudo fusermount -u /home/ubuntu/cephtest/mnt.3 2023-11-26T07:18:38.243 INFO:teuthology.orchestra.run:waiting for 300 2023-11-26T07:18:38.243 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi188.front.sepia.ceph.com 2023-11-26T07:18:38.243 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-26T07:18:38.244 DEBUG:teuthology.orchestra.run.smithi188:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.3) 2023-11-26T07:18:38.295 INFO:tasks.cephfs.cephfs_test_case:[, , , ] 2023-11-26T07:18:38.296 DEBUG:teuthology.orchestra.run.smithi143:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-11-26T07:18:38.434 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:38.429+0000 7f47752f7640 1 Processor -- start 2023-11-26T07:18:38.434 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:38.429+0000 7f47752f7640 1 -- start start 2023-11-26T07:18:38.435 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:38.429+0000 7f47752f7640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f47700b2160 0x7f47700d0df0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:18:38.435 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:38.429+0000 7f47752f7640 1 -- --> v1:172.21.15.143:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f47700cf0e0 con 0x7f47700d1bc0 2023-11-26T07:18:38.435 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:38.429+0000 7f47752f7640 1 -- --> v1:172.21.15.188:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f47700ceb20 con 0x7f477015d6e0 2023-11-26T07:18:38.435 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:38.429+0000 7f47752f7640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f477015dac0 con 0x7f47700b2160 2023-11-26T07:18:38.435 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:38.429+0000 7f47667fc640 1 --1- >> v1:172.21.15.143:6789/0 conn(0x7f47700d1bc0 0x7f47700b0750 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.143:6789/0 says I am v1:172.21.15.143:50940/0 (socket says 172.21.15.143:50940) 2023-11-26T07:18:38.435 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:38.429+0000 7f47667fc640 1 -- 172.21.15.143:0/54479314 learned_addr learned my addr 172.21.15.143:0/54479314 (peer_addr_for_me v1:172.21.15.143:0/0) 2023-11-26T07:18:38.437 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:38.429+0000 7f476f7fe640 1 --2- 172.21.15.143:0/54479314 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f47700b2160 0x7f47700d0df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:18:38.437 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:38.429+0000 7f476e7fc640 1 -- 172.21.15.143:0/54479314 <== mon.0 v1:172.21.15.143:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 332441266 0 0) 0x7f47700cf0e0 con 0x7f47700d1bc0 2023-11-26T07:18:38.437 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:38.429+0000 7f476e7fc640 1 -- 172.21.15.143:0/54479314 --> v1:172.21.15.143:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f475c003530 con 0x7f47700d1bc0 2023-11-26T07:18:38.437 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:38.429+0000 7f476e7fc640 1 -- 172.21.15.143:0/54479314 <== mon.1 v1:172.21.15.188:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3285471644 0 0) 0x7f47700ceb20 con 0x7f477015d6e0 2023-11-26T07:18:38.437 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:38.429+0000 7f476e7fc640 1 -- 172.21.15.143:0/54479314 --> v1:172.21.15.188:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f47700cf0e0 con 0x7f477015d6e0 2023-11-26T07:18:38.438 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:38.429+0000 7f476e7fc640 1 -- 172.21.15.143:0/54479314 <== mon.0 v1:172.21.15.143:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1572354013 0 0) 0x7f475c003530 con 0x7f47700d1bc0 2023-11-26T07:18:38.438 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:38.429+0000 7f476e7fc640 1 -- 172.21.15.143:0/54479314 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f47700b2160 msgr2=0x7f47700d0df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-26T07:18:38.438 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:38.429+0000 7f476e7fc640 1 --2- 172.21.15.143:0/54479314 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f47700b2160 0x7f47700d0df0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:18:38.438 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:38.429+0000 7f476e7fc640 1 -- 172.21.15.143:0/54479314 >> v1:172.21.15.188:6789/0 conn(0x7f477015d6e0 legacy=0x7f47700d14b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:18:38.438 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:38.429+0000 7f476e7fc640 1 -- 172.21.15.143:0/54479314 --> v1:172.21.15.143:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f47700cee00 con 0x7f47700d1bc0 2023-11-26T07:18:38.438 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:38.429+0000 7f476e7fc640 1 -- 172.21.15.143:0/54479314 <== mon.0 v1:172.21.15.143:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2137882523 0 0) 0x7f4758001a00 con 0x7f47700d1bc0 2023-11-26T07:18:38.438 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:38.429+0000 7f476e7fc640 1 -- 172.21.15.143:0/54479314 >> v1:172.21.15.143:6789/0 conn(0x7f47700d1bc0 legacy=0x7f47700b0750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:18:38.438 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:38.429+0000 7f476e7fc640 1 --2- 172.21.15.143:0/54479314 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f47700b2160 0x7f475c003b40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:18:38.438 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:38.429+0000 7f476e7fc640 1 --2- 172.21.15.143:0/54479314 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f475c0040f0 0x7f475c0047d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:18:38.438 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:38.429+0000 7f476e7fc640 1 --2- 172.21.15.143:0/54479314 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f475c004d10 0x7f475c005190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:18:38.438 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:38.429+0000 7f476e7fc640 1 -- 172.21.15.143:0/54479314 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f477015dac0 con 0x7f475c0040f0 2023-11-26T07:18:38.439 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:38.429+0000 7f476e7fc640 1 -- 172.21.15.143:0/54479314 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f475c00bad0 con 0x7f475c004d10 2023-11-26T07:18:38.439 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:38.429+0000 7f476e7fc640 1 -- 172.21.15.143:0/54479314 --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f475c00bc90 con 0x7f47700b2160 2023-11-26T07:18:38.439 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:38.429+0000 7f47667fc640 1 --2- 172.21.15.143:0/54479314 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f475c0040f0 0x7f475c0047d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:18:38.439 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:38.429+0000 7f476effd640 1 --2- 172.21.15.143:0/54479314 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f47700b2160 0x7f475c003b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:18:38.439 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:38.429+0000 7f476f7fe640 1 --2- 172.21.15.143:0/54479314 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f475c004d10 0x7f475c005190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:18:38.439 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:38.433+0000 7f47667fc640 1 -- 172.21.15.143:0/54479314 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f47700b2160 msgr2=0x7f475c003b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-26T07:18:38.439 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:38.433+0000 7f47667fc640 1 --2- 172.21.15.143:0/54479314 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f47700b2160 0x7f475c003b40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:18:38.439 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:38.433+0000 7f47667fc640 1 -- 172.21.15.143:0/54479314 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f475c004d10 msgr2=0x7f475c005190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-26T07:18:38.439 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:38.433+0000 7f47667fc640 1 --2- 172.21.15.143:0/54479314 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f475c004d10 0x7f475c005190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:18:38.439 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:38.433+0000 7f47667fc640 1 -- 172.21.15.143:0/54479314 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f475c003530 con 0x7f475c0040f0 2023-11-26T07:18:38.440 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:38.433+0000 7f47667fc640 1 --2- 172.21.15.143:0/54479314 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f475c0040f0 0x7f475c0047d0 secure :-1 s=READY pgs=328 cs=0 l=1 rev1=1 crypto rx=0x7f475800a710 tx=0x7f475801d140 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e14a2d121485382 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:18:38.440 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:38.433+0000 7f476e7fc640 1 -- 172.21.15.143:0/54479314 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f47580ad630 con 0x7f475c0040f0 2023-11-26T07:18:38.440 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:38.433+0000 7f476e7fc640 1 -- 172.21.15.143:0/54479314 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f47580a8070 con 0x7f475c0040f0 2023-11-26T07:18:38.440 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:38.433+0000 7f476e7fc640 1 -- 172.21.15.143:0/54479314 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f47580ada50 con 0x7f475c0040f0 2023-11-26T07:18:38.441 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:38.433+0000 7f47752f7640 1 -- 172.21.15.143:0/54479314 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f475c0040f0 msgr2=0x7f475c0047d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:18:38.441 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:38.433+0000 7f47752f7640 1 --2- 172.21.15.143:0/54479314 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f475c0040f0 0x7f475c0047d0 secure :-1 s=READY pgs=328 cs=0 l=1 rev1=1 crypto rx=0x7f475800a710 tx=0x7f475801d140 comp rx=0 tx=0).stop 2023-11-26T07:18:38.441 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:38.433+0000 7f476f7fe640 1 -- 172.21.15.143:0/54479314 reap_dead start 2023-11-26T07:18:38.441 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:38.433+0000 7f47752f7640 1 -- 172.21.15.143:0/54479314 shutdown_connections 2023-11-26T07:18:38.441 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:38.433+0000 7f47752f7640 1 -- 172.21.15.143:0/54479314 >> 172.21.15.143:0/54479314 conn(0x7f477009c550 msgr2=0x7f47700d3220 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:18:38.441 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:38.437+0000 7f47752f7640 1 -- 172.21.15.143:0/54479314 shutdown_connections 2023-11-26T07:18:38.441 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:38.437+0000 7f47752f7640 1 -- 172.21.15.143:0/54479314 wait complete. 2023-11-26T07:18:38.442 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:38.437+0000 7f47752f7640 1 Processor -- start 2023-11-26T07:18:38.443 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:38.437+0000 7f47752f7640 1 -- start start 2023-11-26T07:18:38.444 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:38.437+0000 7f47752f7640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f475c0040f0 0x7f47700b8810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:18:38.444 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:38.437+0000 7f47752f7640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f47700cb830 0x7f47700cbcb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:18:38.445 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:38.437+0000 7f47752f7640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f4770159740 0x7f4770159bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:18:38.445 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:38.437+0000 7f47752f7640 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f477015a130 con 0x7f47700cb830 2023-11-26T07:18:38.445 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:38.437+0000 7f47752f7640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f47700b5fe0 con 0x7f4770159740 2023-11-26T07:18:38.445 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:38.437+0000 7f47752f7640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f47700b62c0 con 0x7f475c0040f0 2023-11-26T07:18:38.445 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:38.437+0000 7f47667fc640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f47700cb830 0x7f47700cbcb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:18:38.445 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:38.437+0000 7f47667fc640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f47700cb830 0x7f47700cbcb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:38962/0 (socket says 172.21.15.143:38962) 2023-11-26T07:18:38.445 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:38.437+0000 7f47667fc640 1 -- 172.21.15.143:0/1782213383 learned_addr learned my addr 172.21.15.143:0/1782213383 (peer_addr_for_me v2:172.21.15.143:0/0) 2023-11-26T07:18:38.445 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:38.437+0000 7f476effd640 1 --2- 172.21.15.143:0/1782213383 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f475c0040f0 0x7f47700b8810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:18:38.445 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:38.437+0000 7f476f7fe640 1 --2- 172.21.15.143:0/1782213383 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f4770159740 0x7f4770159bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:18:38.445 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:38.437+0000 7f47667fc640 1 -- 172.21.15.143:0/1782213383 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f475c0040f0 msgr2=0x7f47700b8810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:18:38.445 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:38.437+0000 7f47667fc640 1 --2- 172.21.15.143:0/1782213383 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f475c0040f0 0x7f47700b8810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:18:38.445 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:38.437+0000 7f47667fc640 1 -- 172.21.15.143:0/1782213383 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f4770159740 msgr2=0x7f4770159bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:18:38.445 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:38.437+0000 7f47667fc640 1 --2- 172.21.15.143:0/1782213383 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f4770159740 0x7f4770159bf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:18:38.446 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:38.437+0000 7f47667fc640 1 -- 172.21.15.143:0/1782213383 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f47700b65a0 con 0x7f47700cb830 2023-11-26T07:18:38.446 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:38.437+0000 7f476effd640 1 --2- 172.21.15.143:0/1782213383 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f475c0040f0 0x7f47700b8810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-26T07:18:38.446 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:38.437+0000 7f476f7fe640 1 --2- 172.21.15.143:0/1782213383 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f4770159740 0x7f4770159bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-26T07:18:38.447 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:38.441+0000 7f47667fc640 1 --2- 172.21.15.143:0/1782213383 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f47700cb830 0x7f47700cbcb0 secure :-1 s=READY pgs=329 cs=0 l=1 rev1=1 crypto rx=0x7f475800a710 tx=0x7f475c005800 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:18:38.447 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:38.441+0000 7f4767fff640 1 -- 172.21.15.143:0/1782213383 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f47580ad430 con 0x7f47700cb830 2023-11-26T07:18:38.447 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:38.441+0000 7f4767fff640 1 -- 172.21.15.143:0/1782213383 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f47580ad590 con 0x7f47700cb830 2023-11-26T07:18:38.447 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:38.441+0000 7f47752f7640 1 -- 172.21.15.143:0/1782213383 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f47700b6880 con 0x7f47700cb830 2023-11-26T07:18:38.447 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:38.441+0000 7f4767fff640 1 -- 172.21.15.143:0/1782213383 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f47580ad900 con 0x7f47700cb830 2023-11-26T07:18:38.447 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:38.441+0000 7f47752f7640 1 -- 172.21.15.143:0/1782213383 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f47700b8d50 con 0x7f47700cb830 2023-11-26T07:18:38.447 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:38.441+0000 7f47752f7640 1 -- 172.21.15.143:0/1782213383 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4770067680 con 0x7f47700cb830 2023-11-26T07:18:38.447 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:38.441+0000 7f4767fff640 1 -- 172.21.15.143:0/1782213383 <== mon.0 v2:172.21.15.143:3300/0 4 ==== mgrmap(e 5) v1 ==== 85852+0+0 (secure 0 0 0) 0x7f47580aa030 con 0x7f47700cb830 2023-11-26T07:18:38.448 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:38.441+0000 7f4767fff640 1 --2- 172.21.15.143:0/1782213383 >> [v2:172.21.15.143:6832/2150557544,v1:172.21.15.143:6833/2150557544] conn(0x7f4750066470 0x7f4750068930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:18:38.448 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:38.441+0000 7f4767fff640 1 -- 172.21.15.143:0/1782213383 <== mon.0 v2:172.21.15.143:3300/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 5825+0+0 (secure 0 0 0) 0x7f4760001d30 con 0x7f47700cb830 2023-11-26T07:18:38.448 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:38.441+0000 7f476effd640 1 --2- 172.21.15.143:0/1782213383 >> [v2:172.21.15.143:6832/2150557544,v1:172.21.15.143:6833/2150557544] conn(0x7f4750066470 0x7f4750068930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:18:38.449 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:38.445+0000 7f476effd640 1 --2- 172.21.15.143:0/1782213383 >> [v2:172.21.15.143:6832/2150557544,v1:172.21.15.143:6833/2150557544] conn(0x7f4750066470 0x7f4750068930 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7f4760000f70 tx=0x7f4760000f00 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:18:38.452 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:38.445+0000 7f4767fff640 1 -- 172.21.15.143:0/1782213383 <== mon.0 v2:172.21.15.143:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172649 (secure 0 0 0) 0x7f47580ebee0 con 0x7f47700cb830 2023-11-26T07:18:38.632 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:38.625+0000 7f47752f7640 1 -- 172.21.15.143:0/1782213383 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f47700d3560 con 0x7f47700cb830 2023-11-26T07:18:38.633 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:38.625+0000 7f4767fff640 1 -- 172.21.15.143:0/1782213383 <== mon.0 v2:172.21.15.143:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 8 v8) v1 ==== 93+0+4913 (secure 0 0 0) 0x7f47580ef500 con 0x7f47700cb830 2023-11-26T07:18:38.633 INFO:teuthology.orchestra.run.smithi143.stdout: 2023-11-26T07:18:38.633 INFO:teuthology.orchestra.run.smithi143.stdout:{"epoch":8,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4241,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.188:6834/652687322","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6832","nonce":652687322},{"type":"v1","addr":"172.21.15.188:6834","nonce":652687322}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":4266,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.143:6835/3850184704","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6834","nonce":3850184704},{"type":"v1","addr":"172.21.15.143:6835","nonce":3850184704}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":2},{"gid":4273,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.143:6837/1695478651","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6836","nonce":1695478651},{"type":"v1","addr":"172.21.15.143:6837","nonce":1695478651}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":8,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-11-26T07:17:53.876989+0000","modified":"2023-11-26T07:17:56.885608+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":4285},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4285":{"gid":4285,"name":"b","rank":0,"incarnation":4,"state":"up:active","state_seq":3,"addr":"172.21.15.188:6835/994617268","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6833","nonce":994617268},{"type":"v1","addr":"172.21.15.188:6835","nonce":994617268}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2023-11-26T07:18:38.633 INFO:teuthology.orchestra.run.smithi143.stderr:dumped fsmap epoch 8 2023-11-26T07:18:38.635 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:38.629+0000 7f47752f7640 1 -- 172.21.15.143:0/1782213383 >> [v2:172.21.15.143:6832/2150557544,v1:172.21.15.143:6833/2150557544] conn(0x7f4750066470 msgr2=0x7f4750068930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:18:38.635 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:38.629+0000 7f47752f7640 1 --2- 172.21.15.143:0/1782213383 >> [v2:172.21.15.143:6832/2150557544,v1:172.21.15.143:6833/2150557544] conn(0x7f4750066470 0x7f4750068930 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7f4760000f70 tx=0x7f4760000f00 comp rx=0 tx=0).stop 2023-11-26T07:18:38.635 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:38.629+0000 7f47752f7640 1 -- 172.21.15.143:0/1782213383 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f47700cb830 msgr2=0x7f47700cbcb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:18:38.635 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:38.629+0000 7f47752f7640 1 --2- 172.21.15.143:0/1782213383 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f47700cb830 0x7f47700cbcb0 secure :-1 s=READY pgs=329 cs=0 l=1 rev1=1 crypto rx=0x7f475800a710 tx=0x7f475c005800 comp rx=0 tx=0).stop 2023-11-26T07:18:38.635 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:38.629+0000 7f47752f7640 1 -- 172.21.15.143:0/1782213383 shutdown_connections 2023-11-26T07:18:38.635 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:38.629+0000 7f47752f7640 1 --2- 172.21.15.143:0/1782213383 >> [v2:172.21.15.143:6832/2150557544,v1:172.21.15.143:6833/2150557544] conn(0x7f4750066470 0x7f4750068930 unknown :-1 s=CLOSED pgs=183 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:18:38.635 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:38.629+0000 7f47752f7640 1 --2- 172.21.15.143:0/1782213383 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f4770159740 0x7f4770159bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:18:38.635 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:38.629+0000 7f47752f7640 1 --2- 172.21.15.143:0/1782213383 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f47700cb830 0x7f47700cbcb0 unknown :-1 s=CLOSED pgs=329 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:18:38.635 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:38.629+0000 7f47752f7640 1 --2- 172.21.15.143:0/1782213383 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f475c0040f0 0x7f47700b8810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:18:38.636 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:38.629+0000 7f47752f7640 1 -- 172.21.15.143:0/1782213383 >> 172.21.15.143:0/1782213383 conn(0x7f477009c550 msgr2=0x7f47700d0330 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:18:38.636 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:38.629+0000 7f47752f7640 1 -- 172.21.15.143:0/1782213383 shutdown_connections 2023-11-26T07:18:38.636 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:38.629+0000 7f47752f7640 1 -- 172.21.15.143:0/1782213383 wait complete. 2023-11-26T07:18:38.654 DEBUG:teuthology.orchestra.run.smithi143:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-11-26T07:18:38.794 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:38.789+0000 7ff3a48ea640 1 Processor -- start 2023-11-26T07:18:38.795 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:38.789+0000 7ff3a48ea640 1 -- start start 2023-11-26T07:18:38.796 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:38.789+0000 7ff3a48ea640 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7ff3a00cf990 0x7ff3a015efa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:18:38.796 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:38.789+0000 7ff3a48ea640 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7ff3a015f4e0 0x7ff3a00cd7b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:18:38.796 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:38.789+0000 7ff3a48ea640 1 -- --> v1:172.21.15.188:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff3a006c520 con 0x7ff3a013e290 2023-11-26T07:18:38.796 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:38.789+0000 7ff3a48ea640 1 -- --> v2:172.21.15.143:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff3a006c800 con 0x7ff3a015f4e0 2023-11-26T07:18:38.796 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:38.789+0000 7ff3a48ea640 1 -- --> v2:172.21.15.188:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff3a006cae0 con 0x7ff3a00cf990 2023-11-26T07:18:38.797 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:38.789+0000 7ff39ed9d640 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7ff3a015f4e0 0x7ff3a00cd7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:18:38.797 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:38.789+0000 7ff39ed9d640 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7ff3a015f4e0 0x7ff3a00cd7b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:38976/0 (socket says 172.21.15.143:38976) 2023-11-26T07:18:38.797 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:38.789+0000 7ff39ed9d640 1 -- 172.21.15.143:0/1175173443 learned_addr learned my addr 172.21.15.143:0/1175173443 (peer_addr_for_me v2:172.21.15.143:0/0) 2023-11-26T07:18:38.797 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:38.789+0000 7ff397fff640 1 --2- 172.21.15.143:0/1175173443 >> v2:172.21.15.188:3300/0 conn(0x7ff3a00cf990 0x7ff3a015efa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:18:38.797 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:38.789+0000 7ff39ed9d640 1 -- 172.21.15.143:0/1175173443 >> v2:172.21.15.188:3300/0 conn(0x7ff3a00cf990 msgr2=0x7ff3a015efa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-26T07:18:38.797 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:38.789+0000 7ff39ed9d640 1 --2- 172.21.15.143:0/1175173443 >> v2:172.21.15.188:3300/0 conn(0x7ff3a00cf990 0x7ff3a015efa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:18:38.798 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:38.793+0000 7ff39ed9d640 1 -- 172.21.15.143:0/1175173443 >> v1:172.21.15.188:6789/0 conn(0x7ff3a013e290 legacy=0x7ff3a00cf280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:18:38.798 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:38.793+0000 7ff39ed9d640 1 -- 172.21.15.143:0/1175173443 --> v2:172.21.15.143:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff3a006cdc0 con 0x7ff3a015f4e0 2023-11-26T07:18:38.799 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:38.793+0000 7ff39ed9d640 1 --2- 172.21.15.143:0/1175173443 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7ff3a015f4e0 0x7ff3a00cd7b0 secure :-1 s=READY pgs=330 cs=0 l=1 rev1=1 crypto rx=0x7ff3a00cfe40 tx=0x7ff39801c9a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c8c665228f1d41e8 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:18:38.800 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:38.793+0000 7ff39dd9b640 1 -- 172.21.15.143:0/1175173443 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff3a015fb60 con 0x7ff3a015f4e0 2023-11-26T07:18:38.800 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:38.793+0000 7ff39dd9b640 1 -- 172.21.15.143:0/1175173443 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff3980a5880 con 0x7ff3a015f4e0 2023-11-26T07:18:38.800 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:38.793+0000 7ff39dd9b640 1 -- 172.21.15.143:0/1175173443 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff3980a5c10 con 0x7ff3a015f4e0 2023-11-26T07:18:38.801 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:38.793+0000 7ff3a48ea640 1 -- 172.21.15.143:0/1175173443 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7ff3a015f4e0 msgr2=0x7ff3a00cd7b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:18:38.801 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:38.793+0000 7ff3a48ea640 1 --2- 172.21.15.143:0/1175173443 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7ff3a015f4e0 0x7ff3a00cd7b0 secure :-1 s=READY pgs=330 cs=0 l=1 rev1=1 crypto rx=0x7ff3a00cfe40 tx=0x7ff39801c9a0 comp rx=0 tx=0).stop 2023-11-26T07:18:38.801 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:38.793+0000 7ff3a48ea640 1 -- 172.21.15.143:0/1175173443 shutdown_connections 2023-11-26T07:18:38.801 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:38.793+0000 7ff3a48ea640 1 --2- 172.21.15.143:0/1175173443 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7ff3a015f4e0 0x7ff3a00cd7b0 unknown :-1 s=CLOSED pgs=330 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:18:38.801 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:38.793+0000 7ff3a48ea640 1 --2- 172.21.15.143:0/1175173443 >> v2:172.21.15.188:3300/0 conn(0x7ff3a00cf990 0x7ff3a015efa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:18:38.801 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:38.793+0000 7ff3a48ea640 1 -- 172.21.15.143:0/1175173443 >> 172.21.15.143:0/1175173443 conn(0x7ff3a006a890 msgr2=0x7ff3a006aca0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:18:38.801 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:38.793+0000 7ff3a48ea640 1 -- 172.21.15.143:0/1175173443 shutdown_connections 2023-11-26T07:18:38.801 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:38.797+0000 7ff3a48ea640 1 -- 172.21.15.143:0/1175173443 wait complete. 2023-11-26T07:18:38.802 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:38.797+0000 7ff3a48ea640 1 Processor -- start 2023-11-26T07:18:38.803 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:38.797+0000 7ff3a48ea640 1 -- start start 2023-11-26T07:18:38.804 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:38.797+0000 7ff3a48ea640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7ff3a00cf990 0x7ff3a00b1070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:18:38.805 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:38.797+0000 7ff3a48ea640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7ff3a013e290 0x7ff3a015bef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:18:38.805 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:38.797+0000 7ff3a48ea640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7ff3a015f4e0 0x7ff3a01f88e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:18:38.805 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:38.797+0000 7ff3a48ea640 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff3a015d2a0 con 0x7ff3a013e290 2023-11-26T07:18:38.805 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:38.797+0000 7ff3a48ea640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff3a00b15b0 con 0x7ff3a00cf990 2023-11-26T07:18:38.805 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:38.797+0000 7ff3a48ea640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff3a015c430 con 0x7ff3a015f4e0 2023-11-26T07:18:38.805 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:38.797+0000 7ff397fff640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7ff3a013e290 0x7ff3a015bef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:18:38.805 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:38.797+0000 7ff397fff640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7ff3a013e290 0x7ff3a015bef0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:38992/0 (socket says 172.21.15.143:38992) 2023-11-26T07:18:38.805 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:38.797+0000 7ff397fff640 1 -- 172.21.15.143:0/3171851166 learned_addr learned my addr 172.21.15.143:0/3171851166 (peer_addr_for_me v2:172.21.15.143:0/0) 2023-11-26T07:18:38.805 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:38.797+0000 7ff39e59c640 1 --2- 172.21.15.143:0/3171851166 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7ff3a00cf990 0x7ff3a00b1070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:18:38.805 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:38.797+0000 7ff39ed9d640 1 --2- 172.21.15.143:0/3171851166 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7ff3a015f4e0 0x7ff3a01f88e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:18:38.805 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:38.797+0000 7ff397fff640 1 -- 172.21.15.143:0/3171851166 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7ff3a015f4e0 msgr2=0x7ff3a01f88e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:18:38.806 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:38.797+0000 7ff397fff640 1 --2- 172.21.15.143:0/3171851166 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7ff3a015f4e0 0x7ff3a01f88e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:18:38.806 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:38.797+0000 7ff397fff640 1 -- 172.21.15.143:0/3171851166 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7ff3a00cf990 msgr2=0x7ff3a00b1070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:18:38.806 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:38.797+0000 7ff397fff640 1 --2- 172.21.15.143:0/3171851166 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7ff3a00cf990 0x7ff3a00b1070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:18:38.806 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:38.797+0000 7ff397fff640 1 -- 172.21.15.143:0/3171851166 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff398009b30 con 0x7ff3a013e290 2023-11-26T07:18:38.806 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:38.801+0000 7ff397fff640 1 --2- 172.21.15.143:0/3171851166 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7ff3a013e290 0x7ff3a015bef0 secure :-1 s=READY pgs=331 cs=0 l=1 rev1=1 crypto rx=0x7ff38800d5f0 tx=0x7ff38800dab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:18:38.807 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:38.801+0000 7ff396ffd640 1 -- 172.21.15.143:0/3171851166 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff3880125f0 con 0x7ff3a013e290 2023-11-26T07:18:38.807 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:38.801+0000 7ff396ffd640 1 -- 172.21.15.143:0/3171851166 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff388012750 con 0x7ff3a013e290 2023-11-26T07:18:38.807 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:38.801+0000 7ff396ffd640 1 -- 172.21.15.143:0/3171851166 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff388012b00 con 0x7ff3a013e290 2023-11-26T07:18:38.807 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:38.801+0000 7ff3a48ea640 1 -- 172.21.15.143:0/3171851166 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff3a01f8e20 con 0x7ff3a013e290 2023-11-26T07:18:38.807 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:38.801+0000 7ff3a48ea640 1 -- 172.21.15.143:0/3171851166 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff3a015c900 con 0x7ff3a013e290 2023-11-26T07:18:38.807 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:38.801+0000 7ff3a48ea640 1 -- 172.21.15.143:0/3171851166 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff3a0067680 con 0x7ff3a013e290 2023-11-26T07:18:38.807 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:38.801+0000 7ff396ffd640 1 -- 172.21.15.143:0/3171851166 <== mon.0 v2:172.21.15.143:3300/0 4 ==== mgrmap(e 5) v1 ==== 85852+0+0 (secure 0 0 0) 0x7ff388012c60 con 0x7ff3a013e290 2023-11-26T07:18:38.808 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:38.801+0000 7ff396ffd640 1 --2- 172.21.15.143:0/3171851166 >> [v2:172.21.15.143:6832/2150557544,v1:172.21.15.143:6833/2150557544] conn(0x7ff380066470 0x7ff380068930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:18:38.808 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:38.801+0000 7ff396ffd640 1 -- 172.21.15.143:0/3171851166 <== mon.0 v2:172.21.15.143:3300/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 5825+0+0 (secure 0 0 0) 0x7ff3880843a0 con 0x7ff3a013e290 2023-11-26T07:18:38.808 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:38.801+0000 7ff39e59c640 1 --2- 172.21.15.143:0/3171851166 >> [v2:172.21.15.143:6832/2150557544,v1:172.21.15.143:6833/2150557544] conn(0x7ff380066470 0x7ff380068930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:18:38.809 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:38.805+0000 7ff39e59c640 1 --2- 172.21.15.143:0/3171851166 >> [v2:172.21.15.143:6832/2150557544,v1:172.21.15.143:6833/2150557544] conn(0x7ff380066470 0x7ff380068930 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7ff3a0155420 tx=0x7ff390009040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:18:38.812 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:38.805+0000 7ff396ffd640 1 -- 172.21.15.143:0/3171851166 <== mon.0 v2:172.21.15.143:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172649 (secure 0 0 0) 0x7ff3880517b0 con 0x7ff3a013e290 2023-11-26T07:18:38.961 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:38.957+0000 7ff3a48ea640 1 -- 172.21.15.143:0/3171851166 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7ff3a013a4d0 con 0x7ff3a013e290 2023-11-26T07:18:38.962 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:38.957+0000 7ff396ffd640 1 -- 172.21.15.143:0/3171851166 <== mon.0 v2:172.21.15.143:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v22) v1 ==== 74+0+16731 (secure 0 0 0) 0x7ff388054dd0 con 0x7ff3a013e290 2023-11-26T07:18:38.963 INFO:teuthology.orchestra.run.smithi143.stdout: 2023-11-26T07:18:38.963 INFO:teuthology.orchestra.run.smithi143.stdout:{"epoch":22,"fsid":"f5033060-2469-4a07-95fe-2b76f5404bc1","created":"2023-11-26T07:17:28.613663+0000","modified":"2023-11-26T07:18:35.705228+0000","last_up_change":"2023-11-26T07:17:41.157813+0000","last_in_change":"2023-11-26T07:17:31.181658+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-11-26T07:17:41.758597+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2023-11-26T07:17:51.878332+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2023-11-26T07:17:52.871396+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"ebbee2c6-023e-44b9-9f1d-6f7bfd309730","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6817","nonce":639807369},{"type":"v1","addr":"172.21.15.143:6819","nonce":639807369}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6821","nonce":639807369},{"type":"v1","addr":"172.21.15.143:6823","nonce":639807369}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6830","nonce":639807369},{"type":"v1","addr":"172.21.15.143:6831","nonce":639807369}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6825","nonce":639807369},{"type":"v1","addr":"172.21.15.143:6828","nonce":639807369}]},"public_addr":"172.21.15.143:6819/639807369","cluster_addr":"172.21.15.143:6823/639807369","heartbeat_back_addr":"172.21.15.143:6831/639807369","heartbeat_front_addr":"172.21.15.143:6828/639807369","state":["exists","up"]},{"osd":1,"uuid":"fec4b8be-7122-49e9-9a19-5850a0700001","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6816","nonce":1144942530},{"type":"v1","addr":"172.21.15.143:6818","nonce":1144942530}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6820","nonce":1144942530},{"type":"v1","addr":"172.21.15.143:6822","nonce":1144942530}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6827","nonce":1144942530},{"type":"v1","addr":"172.21.15.143:6829","nonce":1144942530}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6824","nonce":1144942530},{"type":"v1","addr":"172.21.15.143:6826","nonce":1144942530}]},"public_addr":"172.21.15.143:6818/1144942530","cluster_addr":"172.21.15.143:6822/1144942530","heartbeat_back_addr":"172.21.15.143:6829/1144942530","heartbeat_front_addr":"172.21.15.143:6826/1144942530","state":["exists","up"]},{"osd":2,"uuid":"2888af5b-f933-46ca-8f95-6dd494ff43f6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6808","nonce":3909095186},{"type":"v1","addr":"172.21.15.143:6809","nonce":3909095186}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6810","nonce":3909095186},{"type":"v1","addr":"172.21.15.143:6811","nonce":3909095186}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6814","nonce":3909095186},{"type":"v1","addr":"172.21.15.143:6815","nonce":3909095186}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6812","nonce":3909095186},{"type":"v1","addr":"172.21.15.143:6813","nonce":3909095186}]},"public_addr":"172.21.15.143:6809/3909095186","cluster_addr":"172.21.15.143:6811/3909095186","heartbeat_back_addr":"172.21.15.143:6815/3909095186","heartbeat_front_addr":"172.21.15.143:6813/3909095186","state":["exists","up"]},{"osd":3,"uuid":"d1f52435-7bc1-4081-9550-9cd1ba55f549","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6800","nonce":3854210991},{"type":"v1","addr":"172.21.15.143:6801","nonce":3854210991}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6802","nonce":3854210991},{"type":"v1","addr":"172.21.15.143:6803","nonce":3854210991}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6806","nonce":3854210991},{"type":"v1","addr":"172.21.15.143:6807","nonce":3854210991}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6804","nonce":3854210991},{"type":"v1","addr":"172.21.15.143:6805","nonce":3854210991}]},"public_addr":"172.21.15.143:6801/3854210991","cluster_addr":"172.21.15.143:6803/3854210991","heartbeat_back_addr":"172.21.15.143:6807/3854210991","heartbeat_front_addr":"172.21.15.143:6805/3854210991","state":["exists","up"]},{"osd":4,"uuid":"e2ae8b6e-6e25-49b2-bcbf-aacce68f06a3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6800","nonce":3924606902},{"type":"v1","addr":"172.21.15.188:6801","nonce":3924606902}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6802","nonce":3924606902},{"type":"v1","addr":"172.21.15.188:6803","nonce":3924606902}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6806","nonce":3924606902},{"type":"v1","addr":"172.21.15.188:6807","nonce":3924606902}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6804","nonce":3924606902},{"type":"v1","addr":"172.21.15.188:6805","nonce":3924606902}]},"public_addr":"172.21.15.188:6801/3924606902","cluster_addr":"172.21.15.188:6803/3924606902","heartbeat_back_addr":"172.21.15.188:6807/3924606902","heartbeat_front_addr":"172.21.15.188:6805/3924606902","state":["exists","up"]},{"osd":5,"uuid":"3cec8a32-7698-4c36-8e89-e4ba11934f33","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6813","nonce":186841544},{"type":"v1","addr":"172.21.15.188:6815","nonce":186841544}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6817","nonce":186841544},{"type":"v1","addr":"172.21.15.188:6818","nonce":186841544}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6822","nonce":186841544},{"type":"v1","addr":"172.21.15.188:6823","nonce":186841544}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6820","nonce":186841544},{"type":"v1","addr":"172.21.15.188:6821","nonce":186841544}]},"public_addr":"172.21.15.188:6815/186841544","cluster_addr":"172.21.15.188:6818/186841544","heartbeat_back_addr":"172.21.15.188:6823/186841544","heartbeat_front_addr":"172.21.15.188:6821/186841544","state":["exists","up"]},{"osd":6,"uuid":"58a65d5d-e090-473a-81b4-ad38e85029f9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6808","nonce":2153499405},{"type":"v1","addr":"172.21.15.188:6809","nonce":2153499405}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6810","nonce":2153499405},{"type":"v1","addr":"172.21.15.188:6811","nonce":2153499405}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6816","nonce":2153499405},{"type":"v1","addr":"172.21.15.188:6819","nonce":2153499405}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6812","nonce":2153499405},{"type":"v1","addr":"172.21.15.188:6814","nonce":2153499405}]},"public_addr":"172.21.15.188:6809/2153499405","cluster_addr":"172.21.15.188:6811/2153499405","heartbeat_back_addr":"172.21.15.188:6819/2153499405","heartbeat_front_addr":"172.21.15.188:6814/2153499405","state":["exists","up"]},{"osd":7,"uuid":"ea5b7f70-3554-4cf8-ac30-f7c59b80678d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6824","nonce":2722096729},{"type":"v1","addr":"172.21.15.188:6825","nonce":2722096729}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6826","nonce":2722096729},{"type":"v1","addr":"172.21.15.188:6827","nonce":2722096729}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6830","nonce":2722096729},{"type":"v1","addr":"172.21.15.188:6831","nonce":2722096729}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6828","nonce":2722096729},{"type":"v1","addr":"172.21.15.188:6829","nonce":2722096729}]},"public_addr":"172.21.15.188:6825/2722096729","cluster_addr":"172.21.15.188:6827/2722096729","heartbeat_back_addr":"172.21.15.188:6831/2722096729","heartbeat_front_addr":"172.21.15.188:6829/2722096729","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-26T07:17:36.995032+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-26T07:17:36.932266+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-26T07:17:36.829304+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-26T07:17:36.804297+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-26T07:17:36.667892+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-26T07:17:36.964868+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-26T07:17:36.827332+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-26T07:17:36.893179+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"3.1f","mappings":[{"from":5,"to":1}]},{"pgid":"3.24","mappings":[{"from":3,"to":2}]},{"pgid":"3.26","mappings":[{"from":7,"to":2}]},{"pgid":"3.36","mappings":[{"from":5,"to":2}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-11-26T07:18:38.964 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:38.957+0000 7ff3a48ea640 1 -- 172.21.15.143:0/3171851166 >> [v2:172.21.15.143:6832/2150557544,v1:172.21.15.143:6833/2150557544] conn(0x7ff380066470 msgr2=0x7ff380068930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:18:38.965 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:38.957+0000 7ff3a48ea640 1 --2- 172.21.15.143:0/3171851166 >> [v2:172.21.15.143:6832/2150557544,v1:172.21.15.143:6833/2150557544] conn(0x7ff380066470 0x7ff380068930 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7ff3a0155420 tx=0x7ff390009040 comp rx=0 tx=0).stop 2023-11-26T07:18:38.965 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:38.957+0000 7ff3a48ea640 1 -- 172.21.15.143:0/3171851166 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7ff3a013e290 msgr2=0x7ff3a015bef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:18:38.965 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:38.957+0000 7ff3a48ea640 1 --2- 172.21.15.143:0/3171851166 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7ff3a013e290 0x7ff3a015bef0 secure :-1 s=READY pgs=331 cs=0 l=1 rev1=1 crypto rx=0x7ff38800d5f0 tx=0x7ff38800dab0 comp rx=0 tx=0).stop 2023-11-26T07:18:38.965 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:38.957+0000 7ff3a48ea640 1 -- 172.21.15.143:0/3171851166 shutdown_connections 2023-11-26T07:18:38.965 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:38.957+0000 7ff3a48ea640 1 --2- 172.21.15.143:0/3171851166 >> [v2:172.21.15.143:6832/2150557544,v1:172.21.15.143:6833/2150557544] conn(0x7ff380066470 0x7ff380068930 unknown :-1 s=CLOSED pgs=184 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:18:38.965 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:38.957+0000 7ff3a48ea640 1 --2- 172.21.15.143:0/3171851166 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7ff3a015f4e0 0x7ff3a01f88e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:18:38.965 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:38.957+0000 7ff3a48ea640 1 --2- 172.21.15.143:0/3171851166 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7ff3a013e290 0x7ff3a015bef0 unknown :-1 s=CLOSED pgs=331 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:18:38.965 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:38.957+0000 7ff3a48ea640 1 --2- 172.21.15.143:0/3171851166 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7ff3a00cf990 0x7ff3a00b1070 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:18:38.965 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:38.957+0000 7ff3a48ea640 1 -- 172.21.15.143:0/3171851166 >> 172.21.15.143:0/3171851166 conn(0x7ff3a006a890 msgr2=0x7ff3a006ac70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:18:38.965 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:38.957+0000 7ff3a48ea640 1 -- 172.21.15.143:0/3171851166 shutdown_connections 2023-11-26T07:18:38.965 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:38.957+0000 7ff3a48ea640 1 -- 172.21.15.143:0/3171851166 wait complete. 2023-11-26T07:18:38.984 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-11-26T07:17:41.758597+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2023-11-26T07:17:51.878332+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2023-11-26T07:17:52.871396+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '20', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2023-11-26T07:18:38.984 DEBUG:teuthology.orchestra.run.smithi143:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-11-26T07:18:39.125 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:39.121+0000 7f500fd65640 1 Processor -- start 2023-11-26T07:18:39.125 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:39.121+0000 7f500fd65640 1 -- start start 2023-11-26T07:18:39.127 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:39.121+0000 7f500fd65640 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7f50080cf990 0x7f500815efa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:18:39.127 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:39.121+0000 7f500fd65640 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7f500815f4e0 0x7f50080cd7b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:18:39.127 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:39.121+0000 7f500fd65640 1 -- --> v1:172.21.15.188:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f500806c520 con 0x7f500813e290 2023-11-26T07:18:39.127 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:39.121+0000 7f500fd65640 1 -- --> v2:172.21.15.143:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f500806c800 con 0x7f500815f4e0 2023-11-26T07:18:39.127 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:39.121+0000 7f500fd65640 1 -- --> v2:172.21.15.188:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f500806cae0 con 0x7f50080cf990 2023-11-26T07:18:39.127 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:39.121+0000 7f500e302640 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7f500815f4e0 0x7f50080cd7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:18:39.127 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:39.121+0000 7f500e302640 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7f500815f4e0 0x7f50080cd7b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:39006/0 (socket says 172.21.15.143:39006) 2023-11-26T07:18:39.127 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:39.121+0000 7f500e302640 1 -- 172.21.15.143:0/1240175783 learned_addr learned my addr 172.21.15.143:0/1240175783 (peer_addr_for_me v2:172.21.15.143:0/0) 2023-11-26T07:18:39.128 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:39.121+0000 7f500d300640 1 --2- 172.21.15.143:0/1240175783 >> v2:172.21.15.188:3300/0 conn(0x7f50080cf990 0x7f500815efa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:18:39.128 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:39.121+0000 7f500e302640 1 -- 172.21.15.143:0/1240175783 >> v2:172.21.15.188:3300/0 conn(0x7f50080cf990 msgr2=0x7f500815efa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-26T07:18:39.128 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:39.121+0000 7f500e302640 1 --2- 172.21.15.143:0/1240175783 >> v2:172.21.15.188:3300/0 conn(0x7f50080cf990 0x7f500815efa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:18:39.128 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:39.121+0000 7f500e302640 1 -- 172.21.15.143:0/1240175783 >> v1:172.21.15.188:6789/0 conn(0x7f500813e290 legacy=0x7f50080cf280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-26T07:18:39.128 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:39.121+0000 7f500e302640 1 -- 172.21.15.143:0/1240175783 --> v2:172.21.15.143:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f500806cdc0 con 0x7f500815f4e0 2023-11-26T07:18:39.130 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:39.125+0000 7f500e302640 1 --2- 172.21.15.143:0/1240175783 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f500815f4e0 0x7f50080cd7b0 secure :-1 s=READY pgs=332 cs=0 l=1 rev1=1 crypto rx=0x7f50080cfe40 tx=0x7f500001c9a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=44e0553a34769e03 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:18:39.130 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:39.125+0000 7f500caff640 1 -- 172.21.15.143:0/1240175783 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f500815fb60 con 0x7f500815f4e0 2023-11-26T07:18:39.130 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:39.125+0000 7f500caff640 1 -- 172.21.15.143:0/1240175783 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f50000a5880 con 0x7f500815f4e0 2023-11-26T07:18:39.130 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:39.125+0000 7f500caff640 1 -- 172.21.15.143:0/1240175783 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f50000a5c10 con 0x7f500815f4e0 2023-11-26T07:18:39.131 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:39.125+0000 7f500fd65640 1 -- 172.21.15.143:0/1240175783 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f500815f4e0 msgr2=0x7f50080cd7b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:18:39.131 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:39.125+0000 7f500fd65640 1 --2- 172.21.15.143:0/1240175783 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f500815f4e0 0x7f50080cd7b0 secure :-1 s=READY pgs=332 cs=0 l=1 rev1=1 crypto rx=0x7f50080cfe40 tx=0x7f500001c9a0 comp rx=0 tx=0).stop 2023-11-26T07:18:39.131 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:39.125+0000 7f500fd65640 1 -- 172.21.15.143:0/1240175783 shutdown_connections 2023-11-26T07:18:39.131 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:39.125+0000 7f500fd65640 1 --2- 172.21.15.143:0/1240175783 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f500815f4e0 0x7f50080cd7b0 unknown :-1 s=CLOSED pgs=332 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:18:39.132 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:39.125+0000 7f500fd65640 1 --2- 172.21.15.143:0/1240175783 >> v2:172.21.15.188:3300/0 conn(0x7f50080cf990 0x7f500815efa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:18:39.132 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:39.125+0000 7f500fd65640 1 -- 172.21.15.143:0/1240175783 >> 172.21.15.143:0/1240175783 conn(0x7f500806a890 msgr2=0x7f500806aca0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:18:39.132 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:39.125+0000 7f500fd65640 1 -- 172.21.15.143:0/1240175783 shutdown_connections 2023-11-26T07:18:39.132 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:39.125+0000 7f500fd65640 1 -- 172.21.15.143:0/1240175783 wait complete. 2023-11-26T07:18:39.133 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:39.129+0000 7f500fd65640 1 Processor -- start 2023-11-26T07:18:39.133 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:39.129+0000 7f500fd65640 1 -- start start 2023-11-26T07:18:39.135 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:39.129+0000 7f500fd65640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f50080cf990 0x7f500815a1f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:18:39.135 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:39.129+0000 7f500fd65640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f500813e290 0x7f50080aeb60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:18:39.135 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:39.129+0000 7f500fd65640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f500815f4e0 0x7f50080af0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:18:39.135 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:39.129+0000 7f500fd65640 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5008137fc0 con 0x7f500813e290 2023-11-26T07:18:39.135 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:39.129+0000 7f500fd65640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5008070b20 con 0x7f500815f4e0 2023-11-26T07:18:39.135 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:39.129+0000 7f500fd65640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f50081f5910 con 0x7f50080cf990 2023-11-26T07:18:39.135 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:39.129+0000 7f500d300640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f500813e290 0x7f50080aeb60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:18:39.135 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:39.129+0000 7f500d300640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f500813e290 0x7f50080aeb60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:39022/0 (socket says 172.21.15.143:39022) 2023-11-26T07:18:39.135 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:39.129+0000 7f500d300640 1 -- 172.21.15.143:0/4055329109 learned_addr learned my addr 172.21.15.143:0/4055329109 (peer_addr_for_me v2:172.21.15.143:0/0) 2023-11-26T07:18:39.135 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:39.129+0000 7f500db01640 1 --2- 172.21.15.143:0/4055329109 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f50080cf990 0x7f500815a1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:18:39.135 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:39.129+0000 7f500e302640 1 --2- 172.21.15.143:0/4055329109 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f500815f4e0 0x7f50080af0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:18:39.136 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:39.129+0000 7f500d300640 1 -- 172.21.15.143:0/4055329109 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f50080cf990 msgr2=0x7f500815a1f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:18:39.136 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:39.129+0000 7f500d300640 1 --2- 172.21.15.143:0/4055329109 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f50080cf990 0x7f500815a1f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:18:39.136 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:39.129+0000 7f500d300640 1 -- 172.21.15.143:0/4055329109 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f500815f4e0 msgr2=0x7f50080af0a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:18:39.136 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:39.129+0000 7f500d300640 1 --2- 172.21.15.143:0/4055329109 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f500815f4e0 0x7f50080af0a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:18:39.136 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:39.129+0000 7f500d300640 1 -- 172.21.15.143:0/4055329109 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5000009b30 con 0x7f500813e290 2023-11-26T07:18:39.136 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:39.129+0000 7f500d300640 1 --2- 172.21.15.143:0/4055329109 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f500813e290 0x7f50080aeb60 secure :-1 s=READY pgs=333 cs=0 l=1 rev1=1 crypto rx=0x7f4ff800d800 tx=0x7f4ff800dcc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:18:39.137 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:39.129+0000 7f4ffe7fc640 1 -- 172.21.15.143:0/4055329109 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4ff800b6c0 con 0x7f500813e290 2023-11-26T07:18:39.137 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:39.129+0000 7f4ffe7fc640 1 -- 172.21.15.143:0/4055329109 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4ff800b820 con 0x7f500813e290 2023-11-26T07:18:39.137 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:39.129+0000 7f4ffe7fc640 1 -- 172.21.15.143:0/4055329109 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4ff800bb90 con 0x7f500813e290 2023-11-26T07:18:39.137 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:39.129+0000 7f500fd65640 1 -- 172.21.15.143:0/4055329109 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f500815e130 con 0x7f500813e290 2023-11-26T07:18:39.137 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:39.129+0000 7f500fd65640 1 -- 172.21.15.143:0/4055329109 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f50081f5bb0 con 0x7f500813e290 2023-11-26T07:18:39.137 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:39.129+0000 7f500fd65640 1 -- 172.21.15.143:0/4055329109 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5008067680 con 0x7f500813e290 2023-11-26T07:18:39.138 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:39.133+0000 7f4ffe7fc640 1 -- 172.21.15.143:0/4055329109 <== mon.0 v2:172.21.15.143:3300/0 4 ==== mgrmap(e 5) v1 ==== 85852+0+0 (secure 0 0 0) 0x7f4ff800bcf0 con 0x7f500813e290 2023-11-26T07:18:39.138 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:39.133+0000 7f4ffe7fc640 1 --2- 172.21.15.143:0/4055329109 >> [v2:172.21.15.143:6832/2150557544,v1:172.21.15.143:6833/2150557544] conn(0x7f4fec066470 0x7f4fec068930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:18:39.138 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:39.133+0000 7f4ffe7fc640 1 -- 172.21.15.143:0/4055329109 <== mon.0 v2:172.21.15.143:3300/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 5825+0+0 (secure 0 0 0) 0x7f4ff80843a0 con 0x7f500813e290 2023-11-26T07:18:39.139 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:39.133+0000 7f500db01640 1 --2- 172.21.15.143:0/4055329109 >> [v2:172.21.15.143:6832/2150557544,v1:172.21.15.143:6833/2150557544] conn(0x7f4fec066470 0x7f4fec068930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:18:39.139 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:39.133+0000 7f500db01640 1 --2- 172.21.15.143:0/4055329109 >> [v2:172.21.15.143:6832/2150557544,v1:172.21.15.143:6833/2150557544] conn(0x7f4fec066470 0x7f4fec068930 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7f5004007fd0 tx=0x7f5004009040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:18:39.142 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:39.137+0000 7f4ffe7fc640 1 -- 172.21.15.143:0/4055329109 <== mon.0 v2:172.21.15.143:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172649 (secure 0 0 0) 0x7f4ff80517b0 con 0x7f500813e290 2023-11-26T07:18:39.294 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:39.289+0000 7f500fd65640 1 -- 172.21.15.143:0/4055329109 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f500806df50 con 0x7f500813e290 2023-11-26T07:18:39.294 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:39.289+0000 7f4ffe7fc640 1 -- 172.21.15.143:0/4055329109 <== mon.0 v2:172.21.15.143:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v22) v1 ==== 93+0+10 (secure 0 0 0) 0x7f4ff8054dd0 con 0x7f500813e290 2023-11-26T07:18:39.295 INFO:teuthology.orchestra.run.smithi143.stdout:pg_num: 1 2023-11-26T07:18:39.296 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:39.289+0000 7f500fd65640 1 -- 172.21.15.143:0/4055329109 >> [v2:172.21.15.143:6832/2150557544,v1:172.21.15.143:6833/2150557544] conn(0x7f4fec066470 msgr2=0x7f4fec068930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:18:39.297 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:39.289+0000 7f500fd65640 1 --2- 172.21.15.143:0/4055329109 >> [v2:172.21.15.143:6832/2150557544,v1:172.21.15.143:6833/2150557544] conn(0x7f4fec066470 0x7f4fec068930 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7f5004007fd0 tx=0x7f5004009040 comp rx=0 tx=0).stop 2023-11-26T07:18:39.297 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:39.289+0000 7f500fd65640 1 -- 172.21.15.143:0/4055329109 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f500813e290 msgr2=0x7f50080aeb60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:18:39.297 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:39.289+0000 7f500fd65640 1 --2- 172.21.15.143:0/4055329109 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f500813e290 0x7f50080aeb60 secure :-1 s=READY pgs=333 cs=0 l=1 rev1=1 crypto rx=0x7f4ff800d800 tx=0x7f4ff800dcc0 comp rx=0 tx=0).stop 2023-11-26T07:18:39.297 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:39.289+0000 7f500fd65640 1 -- 172.21.15.143:0/4055329109 shutdown_connections 2023-11-26T07:18:39.297 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:39.289+0000 7f500fd65640 1 --2- 172.21.15.143:0/4055329109 >> [v2:172.21.15.143:6832/2150557544,v1:172.21.15.143:6833/2150557544] conn(0x7f4fec066470 0x7f4fec068930 unknown :-1 s=CLOSED pgs=185 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:18:39.297 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:39.289+0000 7f500fd65640 1 --2- 172.21.15.143:0/4055329109 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f500815f4e0 0x7f50080af0a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:18:39.297 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:39.289+0000 7f500fd65640 1 --2- 172.21.15.143:0/4055329109 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f500813e290 0x7f50080aeb60 unknown :-1 s=CLOSED pgs=333 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:18:39.297 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:39.289+0000 7f500fd65640 1 --2- 172.21.15.143:0/4055329109 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f50080cf990 0x7f500815a1f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:18:39.297 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:39.289+0000 7f500fd65640 1 -- 172.21.15.143:0/4055329109 >> 172.21.15.143:0/4055329109 conn(0x7f500806a890 msgr2=0x7f50080b25e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:18:39.297 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:39.289+0000 7f500fd65640 1 -- 172.21.15.143:0/4055329109 shutdown_connections 2023-11-26T07:18:39.298 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:39.289+0000 7f500fd65640 1 -- 172.21.15.143:0/4055329109 wait complete. 2023-11-26T07:18:39.315 DEBUG:teuthology.orchestra.run.smithi143:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-11-26T07:18:39.456 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:39.449+0000 7f25e5cc9640 1 Processor -- start 2023-11-26T07:18:39.456 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:39.449+0000 7f25e5cc9640 1 -- start start 2023-11-26T07:18:39.458 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:39.453+0000 7f25e5cc9640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f25e013e290 0x7f25e00cf280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:18:39.459 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:39.453+0000 7f25e5cc9640 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7f25e00cf7c0 0x7f25e015efa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:18:39.459 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:39.453+0000 7f25e5cc9640 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7f25e015f4e0 0x7f25e01408e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:18:39.459 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:39.453+0000 7f25e5cc9640 1 -- --> v2:172.21.15.143:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f25e00cfc40 con 0x7f25e015f4e0 2023-11-26T07:18:39.459 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:39.453+0000 7f25e5cc9640 1 -- --> v2:172.21.15.188:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f25e006c520 con 0x7f25e00cf7c0 2023-11-26T07:18:39.459 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:39.453+0000 7f25e5cc9640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f25e006c800 con 0x7f25e013e290 2023-11-26T07:18:39.459 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:39.453+0000 7f25dffff640 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7f25e015f4e0 0x7f25e01408e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:18:39.459 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:39.453+0000 7f25dffff640 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7f25e015f4e0 0x7f25e01408e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:39034/0 (socket says 172.21.15.143:39034) 2023-11-26T07:18:39.459 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:39.453+0000 7f25dffff640 1 -- 172.21.15.143:0/145225865 learned_addr learned my addr 172.21.15.143:0/145225865 (peer_addr_for_me v2:172.21.15.143:0/0) 2023-11-26T07:18:39.460 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:39.453+0000 7f25deffd640 1 --2- 172.21.15.143:0/145225865 >> v2:172.21.15.188:3300/0 conn(0x7f25e00cf7c0 0x7f25e015efa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:18:39.460 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:39.453+0000 7f25df7fe640 1 --2- 172.21.15.143:0/145225865 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f25e013e290 0x7f25e00cf280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:18:39.460 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:39.453+0000 7f25dffff640 1 -- 172.21.15.143:0/145225865 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f25e013e290 msgr2=0x7f25e00cf280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-26T07:18:39.460 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:39.453+0000 7f25dffff640 1 --2- 172.21.15.143:0/145225865 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f25e013e290 0x7f25e00cf280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:18:39.460 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:39.453+0000 7f25dffff640 1 -- 172.21.15.143:0/145225865 >> v2:172.21.15.188:3300/0 conn(0x7f25e00cf7c0 msgr2=0x7f25e015efa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-26T07:18:39.460 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:39.453+0000 7f25dffff640 1 --2- 172.21.15.143:0/145225865 >> v2:172.21.15.188:3300/0 conn(0x7f25e00cf7c0 0x7f25e015efa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:18:39.460 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:39.453+0000 7f25dffff640 1 -- 172.21.15.143:0/145225865 --> v2:172.21.15.143:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f25e006cae0 con 0x7f25e015f4e0 2023-11-26T07:18:39.460 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:39.453+0000 7f25deffd640 1 --2- 172.21.15.143:0/145225865 >> v2:172.21.15.188:3300/0 conn(0x7f25e00cf7c0 0x7f25e015efa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-26T07:18:39.461 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:39.457+0000 7f25dffff640 1 --2- 172.21.15.143:0/145225865 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f25e015f4e0 0x7f25e01408e0 secure :-1 s=READY pgs=334 cs=0 l=1 rev1=1 crypto rx=0x7f25e00b45c0 tx=0x7f25d801c6e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=14d6b4ce64964638 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:18:39.461 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:39.457+0000 7f25de7fc640 1 -- 172.21.15.143:0/145225865 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f25d80a5560 con 0x7f25e015f4e0 2023-11-26T07:18:39.462 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:39.457+0000 7f25de7fc640 1 -- 172.21.15.143:0/145225865 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f25d80a56c0 con 0x7f25e015f4e0 2023-11-26T07:18:39.462 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:39.457+0000 7f25de7fc640 1 -- 172.21.15.143:0/145225865 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f25d80a5a50 con 0x7f25e015f4e0 2023-11-26T07:18:39.463 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:39.457+0000 7f25e5cc9640 1 -- 172.21.15.143:0/145225865 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f25e015f4e0 msgr2=0x7f25e01408e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:18:39.463 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:39.457+0000 7f25e5cc9640 1 --2- 172.21.15.143:0/145225865 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f25e015f4e0 0x7f25e01408e0 secure :-1 s=READY pgs=334 cs=0 l=1 rev1=1 crypto rx=0x7f25e00b45c0 tx=0x7f25d801c6e0 comp rx=0 tx=0).stop 2023-11-26T07:18:39.463 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:39.457+0000 7f25e5cc9640 1 -- 172.21.15.143:0/145225865 shutdown_connections 2023-11-26T07:18:39.463 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:39.457+0000 7f25e5cc9640 1 --2- 172.21.15.143:0/145225865 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f25e015f4e0 0x7f25e01408e0 unknown :-1 s=CLOSED pgs=334 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:18:39.463 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:39.457+0000 7f25e5cc9640 1 --2- 172.21.15.143:0/145225865 >> v2:172.21.15.188:3300/0 conn(0x7f25e00cf7c0 0x7f25e015efa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:18:39.463 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:39.457+0000 7f25e5cc9640 1 --2- 172.21.15.143:0/145225865 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f25e013e290 0x7f25e00cf280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:18:39.463 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:39.457+0000 7f25e5cc9640 1 -- 172.21.15.143:0/145225865 >> 172.21.15.143:0/145225865 conn(0x7f25e006a890 msgr2=0x7f25e006aca0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:18:39.463 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:39.457+0000 7f25e5cc9640 1 -- 172.21.15.143:0/145225865 shutdown_connections 2023-11-26T07:18:39.464 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:39.457+0000 7f25e5cc9640 1 -- 172.21.15.143:0/145225865 wait complete. 2023-11-26T07:18:39.464 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:39.457+0000 7f25e5cc9640 1 Processor -- start 2023-11-26T07:18:39.464 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:39.457+0000 7f25e5cc9640 1 -- start start 2023-11-26T07:18:39.466 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:39.461+0000 7f25e5cc9640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f25e00cf7c0 0x7f25e013b890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:18:39.466 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:39.461+0000 7f25e5cc9640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f25e013e290 0x7f25e0154d30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:18:39.467 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:39.461+0000 7f25e5cc9640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f25e015f4e0 0x7f25e0151f50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:18:39.467 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:39.461+0000 7f25e5cc9640 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f25e013cb10 con 0x7f25e013e290 2023-11-26T07:18:39.467 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:39.461+0000 7f25e5cc9640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f25e0155270 con 0x7f25e00cf7c0 2023-11-26T07:18:39.467 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:39.461+0000 7f25e5cc9640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f25e013bf00 con 0x7f25e015f4e0 2023-11-26T07:18:39.467 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:39.461+0000 7f25deffd640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f25e013e290 0x7f25e0154d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:18:39.467 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:39.461+0000 7f25deffd640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f25e013e290 0x7f25e0154d30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:39046/0 (socket says 172.21.15.143:39046) 2023-11-26T07:18:39.467 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:39.461+0000 7f25deffd640 1 -- 172.21.15.143:0/2154284204 learned_addr learned my addr 172.21.15.143:0/2154284204 (peer_addr_for_me v2:172.21.15.143:0/0) 2023-11-26T07:18:39.467 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:39.461+0000 7f25df7fe640 1 --2- 172.21.15.143:0/2154284204 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f25e00cf7c0 0x7f25e013b890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:18:39.467 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:39.461+0000 7f25dffff640 1 --2- 172.21.15.143:0/2154284204 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f25e015f4e0 0x7f25e0151f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:18:39.467 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:39.461+0000 7f25deffd640 1 -- 172.21.15.143:0/2154284204 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f25e015f4e0 msgr2=0x7f25e0151f50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:18:39.468 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:39.461+0000 7f25deffd640 1 --2- 172.21.15.143:0/2154284204 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f25e015f4e0 0x7f25e0151f50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:18:39.468 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:39.461+0000 7f25deffd640 1 -- 172.21.15.143:0/2154284204 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f25e00cf7c0 msgr2=0x7f25e013b890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:18:39.468 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:39.461+0000 7f25deffd640 1 --2- 172.21.15.143:0/2154284204 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f25e00cf7c0 0x7f25e013b890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:18:39.468 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:39.461+0000 7f25deffd640 1 -- 172.21.15.143:0/2154284204 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f25d8009b30 con 0x7f25e013e290 2023-11-26T07:18:39.468 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:39.461+0000 7f25deffd640 1 --2- 172.21.15.143:0/2154284204 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f25e013e290 0x7f25e0154d30 secure :-1 s=READY pgs=335 cs=0 l=1 rev1=1 crypto rx=0x7f25d400e840 tx=0x7f25d400ed00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:18:39.468 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:39.461+0000 7f25c7fff640 1 -- 172.21.15.143:0/2154284204 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f25d4004030 con 0x7f25e013e290 2023-11-26T07:18:39.468 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:39.461+0000 7f25c7fff640 1 -- 172.21.15.143:0/2154284204 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f25d4004190 con 0x7f25e013e290 2023-11-26T07:18:39.468 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:39.461+0000 7f25c7fff640 1 -- 172.21.15.143:0/2154284204 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f25d4004520 con 0x7f25e013e290 2023-11-26T07:18:39.468 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:39.461+0000 7f25e5cc9640 1 -- 172.21.15.143:0/2154284204 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f25e013c1e0 con 0x7f25e013e290 2023-11-26T07:18:39.468 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:39.461+0000 7f25e5cc9640 1 -- 172.21.15.143:0/2154284204 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f25e013c7a0 con 0x7f25e013e290 2023-11-26T07:18:39.469 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:39.461+0000 7f25e5cc9640 1 -- 172.21.15.143:0/2154284204 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f25e00cf310 con 0x7f25e013e290 2023-11-26T07:18:39.469 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:39.461+0000 7f25c7fff640 1 -- 172.21.15.143:0/2154284204 <== mon.0 v2:172.21.15.143:3300/0 4 ==== mgrmap(e 5) v1 ==== 85852+0+0 (secure 0 0 0) 0x7f25d400c6d0 con 0x7f25e013e290 2023-11-26T07:18:39.470 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:39.465+0000 7f25c7fff640 1 --2- 172.21.15.143:0/2154284204 >> [v2:172.21.15.143:6832/2150557544,v1:172.21.15.143:6833/2150557544] conn(0x7f25cc066470 0x7f25cc068930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:18:39.470 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:39.465+0000 7f25c7fff640 1 -- 172.21.15.143:0/2154284204 <== mon.0 v2:172.21.15.143:3300/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 5825+0+0 (secure 0 0 0) 0x7f25d40839e0 con 0x7f25e013e290 2023-11-26T07:18:39.470 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:39.465+0000 7f25df7fe640 1 --2- 172.21.15.143:0/2154284204 >> [v2:172.21.15.143:6832/2150557544,v1:172.21.15.143:6833/2150557544] conn(0x7f25cc066470 0x7f25cc068930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:18:39.470 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:39.465+0000 7f25df7fe640 1 --2- 172.21.15.143:0/2154284204 >> [v2:172.21.15.143:6832/2150557544,v1:172.21.15.143:6833/2150557544] conn(0x7f25cc066470 0x7f25cc068930 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7f25d0007a60 tx=0x7f25d0008040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:18:39.473 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:39.469+0000 7f25c7fff640 1 -- 172.21.15.143:0/2154284204 <== mon.0 v2:172.21.15.143:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172649 (secure 0 0 0) 0x7f25d4050e80 con 0x7f25e013e290 2023-11-26T07:18:39.622 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:39.617+0000 7f25e5cc9640 1 -- 172.21.15.143:0/2154284204 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f25e0067680 con 0x7f25e013e290 2023-11-26T07:18:39.622 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:39.617+0000 7f25c7fff640 1 -- 172.21.15.143:0/2154284204 <== mon.0 v2:172.21.15.143:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v22) v1 ==== 104+0+11 (secure 0 0 0) 0x7f25d40544a0 con 0x7f25e013e290 2023-11-26T07:18:39.623 INFO:teuthology.orchestra.run.smithi143.stdout:pg_num: 32 2023-11-26T07:18:39.625 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:39.617+0000 7f25e5cc9640 1 -- 172.21.15.143:0/2154284204 >> [v2:172.21.15.143:6832/2150557544,v1:172.21.15.143:6833/2150557544] conn(0x7f25cc066470 msgr2=0x7f25cc068930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:18:39.625 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:39.617+0000 7f25e5cc9640 1 --2- 172.21.15.143:0/2154284204 >> [v2:172.21.15.143:6832/2150557544,v1:172.21.15.143:6833/2150557544] conn(0x7f25cc066470 0x7f25cc068930 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7f25d0007a60 tx=0x7f25d0008040 comp rx=0 tx=0).stop 2023-11-26T07:18:39.625 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:39.617+0000 7f25e5cc9640 1 -- 172.21.15.143:0/2154284204 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f25e013e290 msgr2=0x7f25e0154d30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:18:39.625 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:39.617+0000 7f25e5cc9640 1 --2- 172.21.15.143:0/2154284204 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f25e013e290 0x7f25e0154d30 secure :-1 s=READY pgs=335 cs=0 l=1 rev1=1 crypto rx=0x7f25d400e840 tx=0x7f25d400ed00 comp rx=0 tx=0).stop 2023-11-26T07:18:39.625 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:39.617+0000 7f25e5cc9640 1 -- 172.21.15.143:0/2154284204 shutdown_connections 2023-11-26T07:18:39.625 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:39.617+0000 7f25e5cc9640 1 --2- 172.21.15.143:0/2154284204 >> [v2:172.21.15.143:6832/2150557544,v1:172.21.15.143:6833/2150557544] conn(0x7f25cc066470 0x7f25cc068930 unknown :-1 s=CLOSED pgs=186 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:18:39.626 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:39.617+0000 7f25e5cc9640 1 --2- 172.21.15.143:0/2154284204 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f25e015f4e0 0x7f25e0151f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:18:39.626 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:39.617+0000 7f25e5cc9640 1 --2- 172.21.15.143:0/2154284204 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f25e013e290 0x7f25e0154d30 unknown :-1 s=CLOSED pgs=335 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:18:39.626 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:39.617+0000 7f25e5cc9640 1 --2- 172.21.15.143:0/2154284204 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f25e00cf7c0 0x7f25e013b890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:18:39.626 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:39.617+0000 7f25e5cc9640 1 -- 172.21.15.143:0/2154284204 >> 172.21.15.143:0/2154284204 conn(0x7f25e006a890 msgr2=0x7f25e00cc570 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:18:39.626 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:39.621+0000 7f25e5cc9640 1 -- 172.21.15.143:0/2154284204 shutdown_connections 2023-11-26T07:18:39.626 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:39.621+0000 7f25e5cc9640 1 -- 172.21.15.143:0/2154284204 wait complete. 2023-11-26T07:18:39.644 DEBUG:teuthology.orchestra.run.smithi143:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-11-26T07:18:39.788 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:39.781+0000 7f4118ec9640 1 Processor -- start 2023-11-26T07:18:39.788 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:39.781+0000 7f4118ec9640 1 -- start start 2023-11-26T07:18:39.790 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:39.785+0000 7f4118ec9640 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7f4114143ac0 0x7f4114070370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:18:39.790 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:39.785+0000 7f4118ec9640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f41140708b0 0x7f41140cf280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:18:39.790 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:39.785+0000 7f4118ec9640 1 -- --> v1:172.21.15.143:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f41140cfbf0 con 0x7f41140cf7c0 2023-11-26T07:18:39.790 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:39.785+0000 7f4118ec9640 1 -- --> v2:172.21.15.143:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4114143ea0 con 0x7f4114143ac0 2023-11-26T07:18:39.790 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:39.785+0000 7f4118ec9640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f41140cc730 con 0x7f41140708b0 2023-11-26T07:18:39.790 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:39.785+0000 7f411259c640 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7f4114143ac0 0x7f4114070370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:18:39.790 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:39.785+0000 7f4112d9d640 1 --1- >> v1:172.21.15.143:6789/0 conn(0x7f41140cf7c0 0x7f41140b1ef0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.143:6789/0 says I am v1:172.21.15.143:50942/0 (socket says 172.21.15.143:50942) 2023-11-26T07:18:39.790 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:39.785+0000 7f4112d9d640 1 -- 172.21.15.143:0/2105050788 learned_addr learned my addr 172.21.15.143:0/2105050788 (peer_addr_for_me v1:172.21.15.143:0/0) 2023-11-26T07:18:39.790 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:39.785+0000 7f4111d9b640 1 --2- 172.21.15.143:0/2105050788 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f41140708b0 0x7f41140cf280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:18:39.791 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:39.785+0000 7f411259c640 1 -- 172.21.15.143:0/2105050788 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f41140708b0 msgr2=0x7f41140cf280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-26T07:18:39.791 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:39.785+0000 7f411259c640 1 --2- 172.21.15.143:0/2105050788 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f41140708b0 0x7f41140cf280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:18:39.791 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:39.785+0000 7f411259c640 1 -- 172.21.15.143:0/2105050788 >> v1:172.21.15.143:6789/0 conn(0x7f41140cf7c0 legacy=0x7f41140b1ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:18:39.791 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:39.785+0000 7f411259c640 1 -- 172.21.15.143:0/2105050788 --> v2:172.21.15.143:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4114070ee0 con 0x7f4114143ac0 2023-11-26T07:18:39.794 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:39.789+0000 7f411259c640 1 --2- 172.21.15.143:0/2105050788 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f4114143ac0 0x7f4114070370 secure :-1 s=READY pgs=337 cs=0 l=1 rev1=1 crypto rx=0x7f4104009d90 tx=0x7f410401c950 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b049bd763146e820 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:18:39.794 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:39.789+0000 7f411159a640 1 -- 172.21.15.143:0/2105050788 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f41040a5920 con 0x7f4114143ac0 2023-11-26T07:18:39.794 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:39.789+0000 7f411159a640 1 -- 172.21.15.143:0/2105050788 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f41040a5a80 con 0x7f4114143ac0 2023-11-26T07:18:39.795 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:39.789+0000 7f411159a640 1 -- 172.21.15.143:0/2105050788 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f41040a5e10 con 0x7f4114143ac0 2023-11-26T07:18:39.795 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:39.789+0000 7f4118ec9640 1 -- 172.21.15.143:0/2105050788 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f4114143ac0 msgr2=0x7f4114070370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:18:39.795 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:39.789+0000 7f4118ec9640 1 --2- 172.21.15.143:0/2105050788 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f4114143ac0 0x7f4114070370 secure :-1 s=READY pgs=337 cs=0 l=1 rev1=1 crypto rx=0x7f4104009d90 tx=0x7f410401c950 comp rx=0 tx=0).stop 2023-11-26T07:18:39.795 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:39.789+0000 7f4118ec9640 1 -- 172.21.15.143:0/2105050788 shutdown_connections 2023-11-26T07:18:39.795 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:39.789+0000 7f4118ec9640 1 --2- 172.21.15.143:0/2105050788 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f41140708b0 0x7f41140cf280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:18:39.795 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:39.789+0000 7f4118ec9640 1 --2- 172.21.15.143:0/2105050788 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f4114143ac0 0x7f4114070370 unknown :-1 s=CLOSED pgs=337 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:18:39.795 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:39.789+0000 7f4118ec9640 1 -- 172.21.15.143:0/2105050788 >> 172.21.15.143:0/2105050788 conn(0x7f411406a890 msgr2=0x7f411406aca0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:18:39.795 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:39.789+0000 7f4118ec9640 1 -- 172.21.15.143:0/2105050788 shutdown_connections 2023-11-26T07:18:39.796 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:39.789+0000 7f4118ec9640 1 -- 172.21.15.143:0/2105050788 wait complete. 2023-11-26T07:18:39.796 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:39.789+0000 7f4118ec9640 1 Processor -- start 2023-11-26T07:18:39.796 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:39.789+0000 7f4118ec9640 1 -- start start 2023-11-26T07:18:39.799 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:39.793+0000 7f4118ec9640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f41140708b0 0x7f411413c060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:18:39.799 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:39.793+0000 7f4118ec9640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f41140cf7c0 0x7f4114159140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:18:39.799 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:39.793+0000 7f4118ec9640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f4114143ac0 0x7f411413af30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:18:39.799 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:39.793+0000 7f4118ec9640 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f41140add50 con 0x7f41140708b0 2023-11-26T07:18:39.799 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:39.793+0000 7f4118ec9640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f411415c8b0 con 0x7f4114143ac0 2023-11-26T07:18:39.799 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:39.793+0000 7f4118ec9640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f41140ae1e0 con 0x7f41140cf7c0 2023-11-26T07:18:39.799 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:39.793+0000 7f411259c640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f41140708b0 0x7f411413c060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:18:39.800 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:39.793+0000 7f411259c640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f41140708b0 0x7f411413c060 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:39068/0 (socket says 172.21.15.143:39068) 2023-11-26T07:18:39.800 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:39.793+0000 7f411259c640 1 -- 172.21.15.143:0/3466921933 learned_addr learned my addr 172.21.15.143:0/3466921933 (peer_addr_for_me v2:172.21.15.143:0/0) 2023-11-26T07:18:39.800 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:39.793+0000 7f4111d9b640 1 --2- 172.21.15.143:0/3466921933 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f41140cf7c0 0x7f4114159140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:18:39.800 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:39.793+0000 7f4112d9d640 1 --2- 172.21.15.143:0/3466921933 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f4114143ac0 0x7f411413af30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:18:39.800 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:39.793+0000 7f411259c640 1 -- 172.21.15.143:0/3466921933 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f41140cf7c0 msgr2=0x7f4114159140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:18:39.800 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:39.793+0000 7f411259c640 1 --2- 172.21.15.143:0/3466921933 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f41140cf7c0 0x7f4114159140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:18:39.800 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:39.793+0000 7f411259c640 1 -- 172.21.15.143:0/3466921933 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f4114143ac0 msgr2=0x7f411413af30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:18:39.800 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:39.793+0000 7f411259c640 1 --2- 172.21.15.143:0/3466921933 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f4114143ac0 0x7f411413af30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:18:39.801 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:39.793+0000 7f411259c640 1 -- 172.21.15.143:0/3466921933 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4104009a70 con 0x7f41140708b0 2023-11-26T07:18:39.801 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:39.793+0000 7f411259c640 1 --2- 172.21.15.143:0/3466921933 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f41140708b0 0x7f411413c060 secure :-1 s=READY pgs=338 cs=0 l=1 rev1=1 crypto rx=0x7f410401a580 tx=0x7f41040196f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:18:39.801 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:39.793+0000 7f4102ffd640 1 -- 172.21.15.143:0/3466921933 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f41040aee00 con 0x7f41140708b0 2023-11-26T07:18:39.801 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:39.793+0000 7f4102ffd640 1 -- 172.21.15.143:0/3466921933 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4104009450 con 0x7f41140708b0 2023-11-26T07:18:39.801 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:39.793+0000 7f4102ffd640 1 -- 172.21.15.143:0/3466921933 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f41040b7650 con 0x7f41140708b0 2023-11-26T07:18:39.801 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:39.793+0000 7f4118ec9640 1 -- 172.21.15.143:0/3466921933 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f41140ad420 con 0x7f41140708b0 2023-11-26T07:18:39.801 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:39.793+0000 7f4118ec9640 1 -- 172.21.15.143:0/3466921933 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f41140ad9e0 con 0x7f41140708b0 2023-11-26T07:18:39.802 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:39.793+0000 7f4118ec9640 1 -- 172.21.15.143:0/3466921933 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4114067680 con 0x7f41140708b0 2023-11-26T07:18:39.802 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:39.793+0000 7f4102ffd640 1 -- 172.21.15.143:0/3466921933 <== mon.0 v2:172.21.15.143:3300/0 4 ==== mgrmap(e 5) v1 ==== 85852+0+0 (secure 0 0 0) 0x7f41040b4050 con 0x7f41140708b0 2023-11-26T07:18:39.803 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:39.797+0000 7f4102ffd640 1 --2- 172.21.15.143:0/3466921933 >> [v2:172.21.15.143:6832/2150557544,v1:172.21.15.143:6833/2150557544] conn(0x7f40f4066470 0x7f40f4068930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:18:39.803 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:39.797+0000 7f4102ffd640 1 -- 172.21.15.143:0/3466921933 <== mon.0 v2:172.21.15.143:3300/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 5825+0+0 (secure 0 0 0) 0x7f410411f4a0 con 0x7f41140708b0 2023-11-26T07:18:39.803 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:39.797+0000 7f4111d9b640 1 --2- 172.21.15.143:0/3466921933 >> [v2:172.21.15.143:6832/2150557544,v1:172.21.15.143:6833/2150557544] conn(0x7f40f4066470 0x7f40f4068930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:18:39.803 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:39.797+0000 7f4111d9b640 1 --2- 172.21.15.143:0/3466921933 >> [v2:172.21.15.143:6832/2150557544,v1:172.21.15.143:6833/2150557544] conn(0x7f40f4066470 0x7f40f4068930 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7f41140cb430 tx=0x7f4108009190 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:18:39.805 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:39.801+0000 7f4102ffd640 1 -- 172.21.15.143:0/3466921933 <== mon.0 v2:172.21.15.143:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172649 (secure 0 0 0) 0x7f41040ec940 con 0x7f41140708b0 2023-11-26T07:18:39.955 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:39.949+0000 7f4118ec9640 1 -- 172.21.15.143:0/3466921933 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f41140cb180 con 0x7f41140708b0 2023-11-26T07:18:39.955 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:39.949+0000 7f4102ffd640 1 -- 172.21.15.143:0/3466921933 <== mon.0 v2:172.21.15.143:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v22) v1 ==== 100+0+11 (secure 0 0 0) 0x7f41040b2020 con 0x7f41140708b0 2023-11-26T07:18:39.956 INFO:teuthology.orchestra.run.smithi143.stdout:pg_num: 64 2023-11-26T07:18:39.957 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:39.953+0000 7f4118ec9640 1 -- 172.21.15.143:0/3466921933 >> [v2:172.21.15.143:6832/2150557544,v1:172.21.15.143:6833/2150557544] conn(0x7f40f4066470 msgr2=0x7f40f4068930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:18:39.958 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:39.953+0000 7f4118ec9640 1 --2- 172.21.15.143:0/3466921933 >> [v2:172.21.15.143:6832/2150557544,v1:172.21.15.143:6833/2150557544] conn(0x7f40f4066470 0x7f40f4068930 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7f41140cb430 tx=0x7f4108009190 comp rx=0 tx=0).stop 2023-11-26T07:18:39.958 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:39.953+0000 7f4118ec9640 1 -- 172.21.15.143:0/3466921933 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f41140708b0 msgr2=0x7f411413c060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:18:39.958 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:39.953+0000 7f4118ec9640 1 --2- 172.21.15.143:0/3466921933 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f41140708b0 0x7f411413c060 secure :-1 s=READY pgs=338 cs=0 l=1 rev1=1 crypto rx=0x7f410401a580 tx=0x7f41040196f0 comp rx=0 tx=0).stop 2023-11-26T07:18:39.958 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:39.953+0000 7f4118ec9640 1 -- 172.21.15.143:0/3466921933 shutdown_connections 2023-11-26T07:18:39.958 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:39.953+0000 7f4118ec9640 1 --2- 172.21.15.143:0/3466921933 >> [v2:172.21.15.143:6832/2150557544,v1:172.21.15.143:6833/2150557544] conn(0x7f40f4066470 0x7f40f4068930 unknown :-1 s=CLOSED pgs=187 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:18:39.958 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:39.953+0000 7f4118ec9640 1 --2- 172.21.15.143:0/3466921933 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f4114143ac0 0x7f411413af30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:18:39.958 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:39.953+0000 7f4118ec9640 1 --2- 172.21.15.143:0/3466921933 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f41140cf7c0 0x7f4114159140 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:18:39.958 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:39.953+0000 7f4118ec9640 1 --2- 172.21.15.143:0/3466921933 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f41140708b0 0x7f411413c060 unknown :-1 s=CLOSED pgs=338 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:18:39.958 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:39.953+0000 7f4118ec9640 1 -- 172.21.15.143:0/3466921933 >> 172.21.15.143:0/3466921933 conn(0x7f411406a890 msgr2=0x7f411413d300 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:18:39.958 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:39.953+0000 7f4118ec9640 1 -- 172.21.15.143:0/3466921933 shutdown_connections 2023-11-26T07:18:39.959 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:39.953+0000 7f4118ec9640 1 -- 172.21.15.143:0/3466921933 wait complete. 2023-11-26T07:18:39.977 DEBUG:teuthology.orchestra.run.smithi143:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-11-26T07:18:40.114 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:40.109+0000 7f9330679640 1 Processor -- start 2023-11-26T07:18:40.114 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:40.109+0000 7f9330679640 1 -- start start 2023-11-26T07:18:40.117 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:40.109+0000 7f9330679640 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7f932815d6e0 0x7f93280d0df0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:18:40.117 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:40.109+0000 7f9330679640 1 -- --> v1:172.21.15.143:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f93280cf0e0 con 0x7f93280d1bc0 2023-11-26T07:18:40.117 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:40.109+0000 7f9330679640 1 -- --> v1:172.21.15.188:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f93280ceb20 con 0x7f93280b2160 2023-11-26T07:18:40.117 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:40.109+0000 7f9330679640 1 -- --> v2:172.21.15.188:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f932815dac0 con 0x7f932815d6e0 2023-11-26T07:18:40.117 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:40.109+0000 7f932ec16640 1 --1- >> v1:172.21.15.143:6789/0 conn(0x7f93280d1bc0 0x7f93280b0750 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.143:6789/0 says I am v1:172.21.15.143:50958/0 (socket says 172.21.15.143:50958) 2023-11-26T07:18:40.117 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:40.109+0000 7f932ec16640 1 -- 172.21.15.143:0/4294320371 learned_addr learned my addr 172.21.15.143:0/4294320371 (peer_addr_for_me v1:172.21.15.143:0/0) 2023-11-26T07:18:40.117 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:40.109+0000 7f932e415640 1 --2- 172.21.15.143:0/4294320371 >> v2:172.21.15.188:3300/0 conn(0x7f932815d6e0 0x7f93280d0df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:18:40.117 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:40.109+0000 7f932d413640 1 -- 172.21.15.143:0/4294320371 <== mon.0 v1:172.21.15.143:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 652277103 0 0) 0x7f93280cf0e0 con 0x7f93280d1bc0 2023-11-26T07:18:40.118 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:40.109+0000 7f932d413640 1 -- 172.21.15.143:0/4294320371 --> v1:172.21.15.143:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f9314003530 con 0x7f93280d1bc0 2023-11-26T07:18:40.118 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:40.109+0000 7f932d413640 1 -- 172.21.15.143:0/4294320371 <== mon.1 v1:172.21.15.188:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3845809277 0 0) 0x7f93280ceb20 con 0x7f93280b2160 2023-11-26T07:18:40.118 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:40.109+0000 7f932d413640 1 -- 172.21.15.143:0/4294320371 --> v1:172.21.15.188:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f93280cf0e0 con 0x7f93280b2160 2023-11-26T07:18:40.118 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:40.109+0000 7f932e415640 1 -- 172.21.15.143:0/4294320371 >> v1:172.21.15.188:6789/0 conn(0x7f93280b2160 legacy=0x7f93280d14b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:18:40.118 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:40.109+0000 7f932e415640 1 -- 172.21.15.143:0/4294320371 >> v1:172.21.15.143:6789/0 conn(0x7f93280d1bc0 legacy=0x7f93280b0750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:18:40.118 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:40.109+0000 7f932e415640 1 -- 172.21.15.143:0/4294320371 --> v2:172.21.15.188:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f93280cee00 con 0x7f932815d6e0 2023-11-26T07:18:40.119 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:40.113+0000 7f932e415640 1 --2- 172.21.15.143:0/4294320371 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f932815d6e0 0x7f93280d0df0 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7f93240097c0 tx=0x7f932401c480 comp rx=0 tx=0).ready entity=mon.1 client_cookie=963351302d63af22 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:18:40.119 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:40.113+0000 7f932d413640 1 -- 172.21.15.143:0/4294320371 <== mon.1 v2:172.21.15.188:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f93240ab600 con 0x7f932815d6e0 2023-11-26T07:18:40.119 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:40.113+0000 7f932d413640 1 -- 172.21.15.143:0/4294320371 <== mon.1 v2:172.21.15.188:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f93240a7050 con 0x7f932815d6e0 2023-11-26T07:18:40.119 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:40.113+0000 7f932d413640 1 -- 172.21.15.143:0/4294320371 <== mon.1 v2:172.21.15.188:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f93240ab980 con 0x7f932815d6e0 2023-11-26T07:18:40.120 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:40.113+0000 7f9330679640 1 -- 172.21.15.143:0/4294320371 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f932815d6e0 msgr2=0x7f93280d0df0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:18:40.120 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:40.113+0000 7f9330679640 1 --2- 172.21.15.143:0/4294320371 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f932815d6e0 0x7f93280d0df0 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7f93240097c0 tx=0x7f932401c480 comp rx=0 tx=0).stop 2023-11-26T07:18:40.120 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:40.113+0000 7f9330679640 1 -- 172.21.15.143:0/4294320371 shutdown_connections 2023-11-26T07:18:40.120 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:40.113+0000 7f9330679640 1 --2- 172.21.15.143:0/4294320371 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f932815d6e0 0x7f93280d0df0 unknown :-1 s=CLOSED pgs=159 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:18:40.120 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:40.113+0000 7f9330679640 1 -- 172.21.15.143:0/4294320371 >> 172.21.15.143:0/4294320371 conn(0x7f932809c550 msgr2=0x7f93280d3220 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:18:40.121 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:40.113+0000 7f9330679640 1 -- 172.21.15.143:0/4294320371 shutdown_connections 2023-11-26T07:18:40.121 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:40.113+0000 7f9330679640 1 -- 172.21.15.143:0/4294320371 wait complete. 2023-11-26T07:18:40.121 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:40.117+0000 7f9330679640 1 Processor -- start 2023-11-26T07:18:40.122 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:40.117+0000 7f9330679640 1 -- start start 2023-11-26T07:18:40.124 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:40.117+0000 7f9330679640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f93280b2160 0x7f9328151c10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:18:40.124 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:40.117+0000 7f9330679640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f93280d1bc0 0x7f9328151240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:18:40.124 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:40.117+0000 7f9330679640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f932815d6e0 0x7f93280c8370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:18:40.124 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:40.117+0000 7f9330679640 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f93280c9b80 con 0x7f93280b2160 2023-11-26T07:18:40.124 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:40.117+0000 7f9330679640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f93280c7ee0 con 0x7f93280d1bc0 2023-11-26T07:18:40.124 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:40.117+0000 7f9330679640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f93280c98a0 con 0x7f932815d6e0 2023-11-26T07:18:40.125 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:40.117+0000 7f932e415640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f93280b2160 0x7f9328151c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:18:40.125 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:40.117+0000 7f932e415640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f93280b2160 0x7f9328151c10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:39076/0 (socket says 172.21.15.143:39076) 2023-11-26T07:18:40.125 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:40.117+0000 7f932e415640 1 -- 172.21.15.143:0/2635531009 learned_addr learned my addr 172.21.15.143:0/2635531009 (peer_addr_for_me v2:172.21.15.143:0/0) 2023-11-26T07:18:40.125 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:40.117+0000 7f932ec16640 1 --2- 172.21.15.143:0/2635531009 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f932815d6e0 0x7f93280c8370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:18:40.125 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:40.117+0000 7f932dc14640 1 --2- 172.21.15.143:0/2635531009 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f93280d1bc0 0x7f9328151240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:18:40.125 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:40.117+0000 7f932e415640 1 -- 172.21.15.143:0/2635531009 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f932815d6e0 msgr2=0x7f93280c8370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:18:40.125 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:40.117+0000 7f932e415640 1 --2- 172.21.15.143:0/2635531009 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f932815d6e0 0x7f93280c8370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:18:40.125 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:40.117+0000 7f932e415640 1 -- 172.21.15.143:0/2635531009 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f93280d1bc0 msgr2=0x7f9328151240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:18:40.125 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:40.117+0000 7f932e415640 1 --2- 172.21.15.143:0/2635531009 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f93280d1bc0 0x7f9328151240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:18:40.126 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:40.117+0000 7f932e415640 1 -- 172.21.15.143:0/2635531009 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9324009420 con 0x7f93280b2160 2023-11-26T07:18:40.126 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:40.117+0000 7f932e415640 1 --2- 172.21.15.143:0/2635531009 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f93280b2160 0x7f9328151c10 secure :-1 s=READY pgs=340 cs=0 l=1 rev1=1 crypto rx=0x7f93240ae040 tx=0x7f932401acc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:18:40.126 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:40.117+0000 7f931effd640 1 -- 172.21.15.143:0/2635531009 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f93240b7450 con 0x7f93280b2160 2023-11-26T07:18:40.126 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:40.117+0000 7f931effd640 1 -- 172.21.15.143:0/2635531009 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f93240b75d0 con 0x7f93280b2160 2023-11-26T07:18:40.126 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:40.117+0000 7f931effd640 1 -- 172.21.15.143:0/2635531009 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f93240b7980 con 0x7f93280b2160 2023-11-26T07:18:40.126 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:40.117+0000 7f9330679640 1 -- 172.21.15.143:0/2635531009 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f93280c7890 con 0x7f93280b2160 2023-11-26T07:18:40.126 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:40.117+0000 7f9330679640 1 -- 172.21.15.143:0/2635531009 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9328152150 con 0x7f93280b2160 2023-11-26T07:18:40.127 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:40.121+0000 7f9330679640 1 -- 172.21.15.143:0/2635531009 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9328067680 con 0x7f93280b2160 2023-11-26T07:18:40.127 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:40.121+0000 7f931effd640 1 -- 172.21.15.143:0/2635531009 <== mon.0 v2:172.21.15.143:3300/0 4 ==== mgrmap(e 5) v1 ==== 85852+0+0 (secure 0 0 0) 0x7f93240b7ae0 con 0x7f93280b2160 2023-11-26T07:18:40.127 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:40.121+0000 7f931effd640 1 --2- 172.21.15.143:0/2635531009 >> [v2:172.21.15.143:6832/2150557544,v1:172.21.15.143:6833/2150557544] conn(0x7f930c066470 0x7f930c068930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:18:40.128 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:40.121+0000 7f931effd640 1 -- 172.21.15.143:0/2635531009 <== mon.0 v2:172.21.15.143:3300/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 5825+0+0 (secure 0 0 0) 0x7f932411e600 con 0x7f93280b2160 2023-11-26T07:18:40.128 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:40.121+0000 7f932dc14640 1 --2- 172.21.15.143:0/2635531009 >> [v2:172.21.15.143:6832/2150557544,v1:172.21.15.143:6833/2150557544] conn(0x7f930c066470 0x7f930c068930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:18:40.128 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:40.121+0000 7f932dc14640 1 --2- 172.21.15.143:0/2635531009 >> [v2:172.21.15.143:6832/2150557544,v1:172.21.15.143:6833/2150557544] conn(0x7f930c066470 0x7f930c068930 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7f9328159060 tx=0x7f9318009040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:18:40.131 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:40.125+0000 7f931effd640 1 -- 172.21.15.143:0/2635531009 <== mon.0 v2:172.21.15.143:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172649 (secure 0 0 0) 0x7f93240eb9f0 con 0x7f93280b2160 2023-11-26T07:18:40.310 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:40.305+0000 7f9330679640 1 -- 172.21.15.143:0/2635531009 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f9328159160 con 0x7f93280b2160 2023-11-26T07:18:40.311 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:40.305+0000 7f931effd640 1 -- 172.21.15.143:0/2635531009 <== mon.0 v2:172.21.15.143:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 8 v8) v1 ==== 93+0+4913 (secure 0 0 0) 0x7f93240ef010 con 0x7f93280b2160 2023-11-26T07:18:40.311 INFO:teuthology.orchestra.run.smithi143.stdout: 2023-11-26T07:18:40.312 INFO:teuthology.orchestra.run.smithi143.stdout:{"epoch":8,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4241,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.188:6834/652687322","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6832","nonce":652687322},{"type":"v1","addr":"172.21.15.188:6834","nonce":652687322}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":4266,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.143:6835/3850184704","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6834","nonce":3850184704},{"type":"v1","addr":"172.21.15.143:6835","nonce":3850184704}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":2},{"gid":4273,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.143:6837/1695478651","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6836","nonce":1695478651},{"type":"v1","addr":"172.21.15.143:6837","nonce":1695478651}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":8,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-11-26T07:17:53.876989+0000","modified":"2023-11-26T07:17:56.885608+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":4285},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4285":{"gid":4285,"name":"b","rank":0,"incarnation":4,"state":"up:active","state_seq":3,"addr":"172.21.15.188:6835/994617268","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6833","nonce":994617268},{"type":"v1","addr":"172.21.15.188:6835","nonce":994617268}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2023-11-26T07:18:40.312 INFO:teuthology.orchestra.run.smithi143.stderr:dumped fsmap epoch 8 2023-11-26T07:18:40.313 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:40.309+0000 7f9330679640 1 -- 172.21.15.143:0/2635531009 >> [v2:172.21.15.143:6832/2150557544,v1:172.21.15.143:6833/2150557544] conn(0x7f930c066470 msgr2=0x7f930c068930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:18:40.313 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:40.309+0000 7f9330679640 1 --2- 172.21.15.143:0/2635531009 >> [v2:172.21.15.143:6832/2150557544,v1:172.21.15.143:6833/2150557544] conn(0x7f930c066470 0x7f930c068930 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7f9328159060 tx=0x7f9318009040 comp rx=0 tx=0).stop 2023-11-26T07:18:40.313 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:40.309+0000 7f9330679640 1 -- 172.21.15.143:0/2635531009 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f93280b2160 msgr2=0x7f9328151c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:18:40.314 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:40.309+0000 7f9330679640 1 --2- 172.21.15.143:0/2635531009 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f93280b2160 0x7f9328151c10 secure :-1 s=READY pgs=340 cs=0 l=1 rev1=1 crypto rx=0x7f93240ae040 tx=0x7f932401acc0 comp rx=0 tx=0).stop 2023-11-26T07:18:40.314 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:40.309+0000 7f9330679640 1 -- 172.21.15.143:0/2635531009 shutdown_connections 2023-11-26T07:18:40.314 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:40.309+0000 7f9330679640 1 --2- 172.21.15.143:0/2635531009 >> [v2:172.21.15.143:6832/2150557544,v1:172.21.15.143:6833/2150557544] conn(0x7f930c066470 0x7f930c068930 unknown :-1 s=CLOSED pgs=188 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:18:40.314 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:40.309+0000 7f9330679640 1 --2- 172.21.15.143:0/2635531009 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f932815d6e0 0x7f93280c8370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:18:40.314 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:40.309+0000 7f9330679640 1 --2- 172.21.15.143:0/2635531009 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f93280d1bc0 0x7f9328151240 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:18:40.314 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:40.309+0000 7f9330679640 1 --2- 172.21.15.143:0/2635531009 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f93280b2160 0x7f9328151c10 unknown :-1 s=CLOSED pgs=340 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:18:40.314 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:40.309+0000 7f9330679640 1 -- 172.21.15.143:0/2635531009 >> 172.21.15.143:0/2635531009 conn(0x7f932809c550 msgr2=0x7f93280734a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:18:40.314 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:40.309+0000 7f9330679640 1 -- 172.21.15.143:0/2635531009 shutdown_connections 2023-11-26T07:18:40.314 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:40.309+0000 7f9330679640 1 -- 172.21.15.143:0/2635531009 wait complete. 2023-11-26T07:18:40.332 DEBUG:teuthology.orchestra.run.smithi143:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-11-26T07:18:40.469 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:40.461+0000 7f9eeb357640 1 Processor -- start 2023-11-26T07:18:40.469 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:40.461+0000 7f9eeb357640 1 -- start start 2023-11-26T07:18:40.470 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:40.465+0000 7f9eeb357640 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7f9ee40cf280 0x7f9ee40cf710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:18:40.470 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:40.465+0000 7f9eeb357640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f9ee40cd1b0 0x7f9ee40cd5c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:18:40.470 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:40.465+0000 7f9eeb357640 1 -- --> v1:172.21.15.188:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9ee40cc730 con 0x7f9ee4143ac0 2023-11-26T07:18:40.471 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:40.465+0000 7f9eeb357640 1 -- --> v2:172.21.15.143:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9ee4143ea0 con 0x7f9ee40cf280 2023-11-26T07:18:40.471 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:40.465+0000 7f9eeb357640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9ee40cfc50 con 0x7f9ee40cd1b0 2023-11-26T07:18:40.471 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:40.465+0000 7f9ee88f2640 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7f9ee40cf280 0x7f9ee40cf710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:18:40.471 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:40.465+0000 7f9ee88f2640 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7f9ee40cf280 0x7f9ee40cf710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:39078/0 (socket says 172.21.15.143:39078) 2023-11-26T07:18:40.471 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:40.465+0000 7f9ee88f2640 1 -- 172.21.15.143:0/1021176038 learned_addr learned my addr 172.21.15.143:0/1021176038 (peer_addr_for_me v2:172.21.15.143:0/0) 2023-11-26T07:18:40.471 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:40.465+0000 7f9ee98f4640 1 --2- 172.21.15.143:0/1021176038 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f9ee40cd1b0 0x7f9ee40cd5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:18:40.471 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:40.465+0000 7f9ee88f2640 1 -- 172.21.15.143:0/1021176038 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f9ee40cd1b0 msgr2=0x7f9ee40cd5c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-26T07:18:40.471 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:40.465+0000 7f9ee88f2640 1 --2- 172.21.15.143:0/1021176038 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f9ee40cd1b0 0x7f9ee40cd5c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:18:40.471 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:40.465+0000 7f9ee88f2640 1 -- 172.21.15.143:0/1021176038 >> v1:172.21.15.188:6789/0 conn(0x7f9ee4143ac0 legacy=0x7f9ee4070370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-26T07:18:40.471 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:40.465+0000 7f9ee88f2640 1 -- 172.21.15.143:0/1021176038 --> v2:172.21.15.143:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9ee4070ee0 con 0x7f9ee40cf280 2023-11-26T07:18:40.471 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:40.465+0000 7f9ee98f4640 1 --2- 172.21.15.143:0/1021176038 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f9ee40cd1b0 0x7f9ee40cd5c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-26T07:18:40.473 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:40.469+0000 7f9ee88f2640 1 --2- 172.21.15.143:0/1021176038 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f9ee40cf280 0x7f9ee40cf710 secure :-1 s=READY pgs=341 cs=0 l=1 rev1=1 crypto rx=0x7f9ee40aeb10 tx=0x7f9ed401eb90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f9802aa1b4e10ae4 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:18:40.474 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:40.469+0000 7f9edbfff640 1 -- 172.21.15.143:0/1021176038 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9ed40a7c20 con 0x7f9ee40cf280 2023-11-26T07:18:40.474 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:40.469+0000 7f9edbfff640 1 -- 172.21.15.143:0/1021176038 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9ed40a7d80 con 0x7f9ee40cf280 2023-11-26T07:18:40.474 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:40.469+0000 7f9edbfff640 1 -- 172.21.15.143:0/1021176038 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9ed40af660 con 0x7f9ee40cf280 2023-11-26T07:18:40.475 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:40.469+0000 7f9eeb357640 1 -- 172.21.15.143:0/1021176038 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f9ee40cf280 msgr2=0x7f9ee40cf710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:18:40.475 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:40.469+0000 7f9eeb357640 1 --2- 172.21.15.143:0/1021176038 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f9ee40cf280 0x7f9ee40cf710 secure :-1 s=READY pgs=341 cs=0 l=1 rev1=1 crypto rx=0x7f9ee40aeb10 tx=0x7f9ed401eb90 comp rx=0 tx=0).stop 2023-11-26T07:18:40.475 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:40.469+0000 7f9eeb357640 1 -- 172.21.15.143:0/1021176038 shutdown_connections 2023-11-26T07:18:40.475 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:40.469+0000 7f9eeb357640 1 --2- 172.21.15.143:0/1021176038 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f9ee40cf280 0x7f9ee40cf710 unknown :-1 s=CLOSED pgs=341 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:18:40.475 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:40.469+0000 7f9eeb357640 1 --2- 172.21.15.143:0/1021176038 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f9ee40cd1b0 0x7f9ee40cd5c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:18:40.475 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:40.469+0000 7f9eeb357640 1 -- 172.21.15.143:0/1021176038 >> 172.21.15.143:0/1021176038 conn(0x7f9ee406a890 msgr2=0x7f9ee406aca0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:18:40.475 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:40.469+0000 7f9eeb357640 1 -- 172.21.15.143:0/1021176038 shutdown_connections 2023-11-26T07:18:40.475 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:40.469+0000 7f9eeb357640 1 -- 172.21.15.143:0/1021176038 wait complete. 2023-11-26T07:18:40.476 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:40.469+0000 7f9eeb357640 1 Processor -- start 2023-11-26T07:18:40.476 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:40.469+0000 7f9eeb357640 1 -- start start 2023-11-26T07:18:40.478 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:40.473+0000 7f9eeb357640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f9ee40cd1b0 0x7f9ee41593a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:18:40.478 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:40.473+0000 7f9eeb357640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f9ee40cf280 0x7f9ee41f73c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:18:40.478 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:40.473+0000 7f9eeb357640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f9ee4143ac0 0x7f9ee415ab00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:18:40.478 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:40.473+0000 7f9eeb357640 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9ee413ee70 con 0x7f9ee4143ac0 2023-11-26T07:18:40.478 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:40.473+0000 7f9eeb357640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9ee413f110 con 0x7f9ee40cf280 2023-11-26T07:18:40.478 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:40.473+0000 7f9eeb357640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9ee413f3f0 con 0x7f9ee40cd1b0 2023-11-26T07:18:40.479 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:40.473+0000 7f9ee98f4640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f9ee4143ac0 0x7f9ee415ab00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:18:40.479 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:40.473+0000 7f9ee98f4640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f9ee4143ac0 0x7f9ee415ab00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:39092/0 (socket says 172.21.15.143:39092) 2023-11-26T07:18:40.479 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:40.473+0000 7f9ee98f4640 1 -- 172.21.15.143:0/309231424 learned_addr learned my addr 172.21.15.143:0/309231424 (peer_addr_for_me v2:172.21.15.143:0/0) 2023-11-26T07:18:40.479 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:40.473+0000 7f9ee90f3640 1 --2- 172.21.15.143:0/309231424 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f9ee40cd1b0 0x7f9ee41593a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:18:40.479 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:40.473+0000 7f9ee88f2640 1 --2- 172.21.15.143:0/309231424 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f9ee40cf280 0x7f9ee41f73c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:18:40.479 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:40.473+0000 7f9ee98f4640 1 -- 172.21.15.143:0/309231424 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f9ee40cd1b0 msgr2=0x7f9ee41593a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:18:40.479 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:40.473+0000 7f9ee98f4640 1 --2- 172.21.15.143:0/309231424 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f9ee40cd1b0 0x7f9ee41593a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:18:40.479 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:40.473+0000 7f9ee98f4640 1 -- 172.21.15.143:0/309231424 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f9ee40cf280 msgr2=0x7f9ee41f73c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:18:40.479 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:40.473+0000 7f9ee98f4640 1 --2- 172.21.15.143:0/309231424 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f9ee40cf280 0x7f9ee41f73c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:18:40.479 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:40.473+0000 7f9ee98f4640 1 -- 172.21.15.143:0/309231424 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9ed400b770 con 0x7f9ee4143ac0 2023-11-26T07:18:40.479 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:40.473+0000 7f9ee98f4640 1 --2- 172.21.15.143:0/309231424 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f9ee4143ac0 0x7f9ee415ab00 secure :-1 s=READY pgs=342 cs=0 l=1 rev1=1 crypto rx=0x7f9edc007df0 tx=0x7f9edc00d4a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:18:40.480 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:40.473+0000 7f9ed9ffb640 1 -- 172.21.15.143:0/309231424 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9edc0039a0 con 0x7f9ee4143ac0 2023-11-26T07:18:40.480 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:40.473+0000 7f9ed9ffb640 1 -- 172.21.15.143:0/309231424 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9edc00f040 con 0x7f9ee4143ac0 2023-11-26T07:18:40.481 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:40.473+0000 7f9eeb357640 1 -- 172.21.15.143:0/309231424 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9ee413f6d0 con 0x7f9ee4143ac0 2023-11-26T07:18:40.481 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:40.473+0000 7f9ed9ffb640 1 -- 172.21.15.143:0/309231424 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9edc003d40 con 0x7f9ee4143ac0 2023-11-26T07:18:40.481 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:40.473+0000 7f9eeb357640 1 -- 172.21.15.143:0/309231424 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9ee415b040 con 0x7f9ee4143ac0 2023-11-26T07:18:40.481 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:40.473+0000 7f9eeb357640 1 -- 172.21.15.143:0/309231424 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9ee4067680 con 0x7f9ee4143ac0 2023-11-26T07:18:40.481 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:40.473+0000 7f9ed9ffb640 1 -- 172.21.15.143:0/309231424 <== mon.0 v2:172.21.15.143:3300/0 4 ==== mgrmap(e 5) v1 ==== 85852+0+0 (secure 0 0 0) 0x7f9edc004030 con 0x7f9ee4143ac0 2023-11-26T07:18:40.481 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:40.477+0000 7f9ed9ffb640 1 --2- 172.21.15.143:0/309231424 >> [v2:172.21.15.143:6832/2150557544,v1:172.21.15.143:6833/2150557544] conn(0x7f9ec8066470 0x7f9ec8068930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:18:40.482 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:40.477+0000 7f9ed9ffb640 1 -- 172.21.15.143:0/309231424 <== mon.0 v2:172.21.15.143:3300/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 5825+0+0 (secure 0 0 0) 0x7f9edc084150 con 0x7f9ee4143ac0 2023-11-26T07:18:40.482 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:40.477+0000 7f9ee90f3640 1 --2- 172.21.15.143:0/309231424 >> [v2:172.21.15.143:6832/2150557544,v1:172.21.15.143:6833/2150557544] conn(0x7f9ec8066470 0x7f9ec8068930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:18:40.482 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:40.477+0000 7f9ee90f3640 1 --2- 172.21.15.143:0/309231424 >> [v2:172.21.15.143:6832/2150557544,v1:172.21.15.143:6833/2150557544] conn(0x7f9ec8066470 0x7f9ec8068930 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7f9ee0007fd0 tx=0x7f9ee0007e40 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:18:40.485 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:40.481+0000 7f9ed9ffb640 1 -- 172.21.15.143:0/309231424 <== mon.0 v2:172.21.15.143:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172649 (secure 0 0 0) 0x7f9edc0515f0 con 0x7f9ee4143ac0 2023-11-26T07:18:40.637 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:40.633+0000 7f9eeb357640 1 -- 172.21.15.143:0/309231424 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f9ee413fb80 con 0x7f9ee4143ac0 2023-11-26T07:18:40.638 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:40.633+0000 7f9ed9ffb640 1 -- 172.21.15.143:0/309231424 <== mon.0 v2:172.21.15.143:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v22) v1 ==== 74+0+16731 (secure 0 0 0) 0x7f9edc054c10 con 0x7f9ee4143ac0 2023-11-26T07:18:40.638 INFO:teuthology.orchestra.run.smithi143.stdout: 2023-11-26T07:18:40.639 INFO:teuthology.orchestra.run.smithi143.stdout:{"epoch":22,"fsid":"f5033060-2469-4a07-95fe-2b76f5404bc1","created":"2023-11-26T07:17:28.613663+0000","modified":"2023-11-26T07:18:35.705228+0000","last_up_change":"2023-11-26T07:17:41.157813+0000","last_in_change":"2023-11-26T07:17:31.181658+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-11-26T07:17:41.758597+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2023-11-26T07:17:51.878332+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2023-11-26T07:17:52.871396+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"ebbee2c6-023e-44b9-9f1d-6f7bfd309730","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6817","nonce":639807369},{"type":"v1","addr":"172.21.15.143:6819","nonce":639807369}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6821","nonce":639807369},{"type":"v1","addr":"172.21.15.143:6823","nonce":639807369}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6830","nonce":639807369},{"type":"v1","addr":"172.21.15.143:6831","nonce":639807369}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6825","nonce":639807369},{"type":"v1","addr":"172.21.15.143:6828","nonce":639807369}]},"public_addr":"172.21.15.143:6819/639807369","cluster_addr":"172.21.15.143:6823/639807369","heartbeat_back_addr":"172.21.15.143:6831/639807369","heartbeat_front_addr":"172.21.15.143:6828/639807369","state":["exists","up"]},{"osd":1,"uuid":"fec4b8be-7122-49e9-9a19-5850a0700001","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6816","nonce":1144942530},{"type":"v1","addr":"172.21.15.143:6818","nonce":1144942530}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6820","nonce":1144942530},{"type":"v1","addr":"172.21.15.143:6822","nonce":1144942530}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6827","nonce":1144942530},{"type":"v1","addr":"172.21.15.143:6829","nonce":1144942530}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6824","nonce":1144942530},{"type":"v1","addr":"172.21.15.143:6826","nonce":1144942530}]},"public_addr":"172.21.15.143:6818/1144942530","cluster_addr":"172.21.15.143:6822/1144942530","heartbeat_back_addr":"172.21.15.143:6829/1144942530","heartbeat_front_addr":"172.21.15.143:6826/1144942530","state":["exists","up"]},{"osd":2,"uuid":"2888af5b-f933-46ca-8f95-6dd494ff43f6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6808","nonce":3909095186},{"type":"v1","addr":"172.21.15.143:6809","nonce":3909095186}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6810","nonce":3909095186},{"type":"v1","addr":"172.21.15.143:6811","nonce":3909095186}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6814","nonce":3909095186},{"type":"v1","addr":"172.21.15.143:6815","nonce":3909095186}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6812","nonce":3909095186},{"type":"v1","addr":"172.21.15.143:6813","nonce":3909095186}]},"public_addr":"172.21.15.143:6809/3909095186","cluster_addr":"172.21.15.143:6811/3909095186","heartbeat_back_addr":"172.21.15.143:6815/3909095186","heartbeat_front_addr":"172.21.15.143:6813/3909095186","state":["exists","up"]},{"osd":3,"uuid":"d1f52435-7bc1-4081-9550-9cd1ba55f549","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6800","nonce":3854210991},{"type":"v1","addr":"172.21.15.143:6801","nonce":3854210991}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6802","nonce":3854210991},{"type":"v1","addr":"172.21.15.143:6803","nonce":3854210991}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6806","nonce":3854210991},{"type":"v1","addr":"172.21.15.143:6807","nonce":3854210991}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6804","nonce":3854210991},{"type":"v1","addr":"172.21.15.143:6805","nonce":3854210991}]},"public_addr":"172.21.15.143:6801/3854210991","cluster_addr":"172.21.15.143:6803/3854210991","heartbeat_back_addr":"172.21.15.143:6807/3854210991","heartbeat_front_addr":"172.21.15.143:6805/3854210991","state":["exists","up"]},{"osd":4,"uuid":"e2ae8b6e-6e25-49b2-bcbf-aacce68f06a3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6800","nonce":3924606902},{"type":"v1","addr":"172.21.15.188:6801","nonce":3924606902}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6802","nonce":3924606902},{"type":"v1","addr":"172.21.15.188:6803","nonce":3924606902}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6806","nonce":3924606902},{"type":"v1","addr":"172.21.15.188:6807","nonce":3924606902}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6804","nonce":3924606902},{"type":"v1","addr":"172.21.15.188:6805","nonce":3924606902}]},"public_addr":"172.21.15.188:6801/3924606902","cluster_addr":"172.21.15.188:6803/3924606902","heartbeat_back_addr":"172.21.15.188:6807/3924606902","heartbeat_front_addr":"172.21.15.188:6805/3924606902","state":["exists","up"]},{"osd":5,"uuid":"3cec8a32-7698-4c36-8e89-e4ba11934f33","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6813","nonce":186841544},{"type":"v1","addr":"172.21.15.188:6815","nonce":186841544}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6817","nonce":186841544},{"type":"v1","addr":"172.21.15.188:6818","nonce":186841544}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6822","nonce":186841544},{"type":"v1","addr":"172.21.15.188:6823","nonce":186841544}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6820","nonce":186841544},{"type":"v1","addr":"172.21.15.188:6821","nonce":186841544}]},"public_addr":"172.21.15.188:6815/186841544","cluster_addr":"172.21.15.188:6818/186841544","heartbeat_back_addr":"172.21.15.188:6823/186841544","heartbeat_front_addr":"172.21.15.188:6821/186841544","state":["exists","up"]},{"osd":6,"uuid":"58a65d5d-e090-473a-81b4-ad38e85029f9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6808","nonce":2153499405},{"type":"v1","addr":"172.21.15.188:6809","nonce":2153499405}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6810","nonce":2153499405},{"type":"v1","addr":"172.21.15.188:6811","nonce":2153499405}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6816","nonce":2153499405},{"type":"v1","addr":"172.21.15.188:6819","nonce":2153499405}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6812","nonce":2153499405},{"type":"v1","addr":"172.21.15.188:6814","nonce":2153499405}]},"public_addr":"172.21.15.188:6809/2153499405","cluster_addr":"172.21.15.188:6811/2153499405","heartbeat_back_addr":"172.21.15.188:6819/2153499405","heartbeat_front_addr":"172.21.15.188:6814/2153499405","state":["exists","up"]},{"osd":7,"uuid":"ea5b7f70-3554-4cf8-ac30-f7c59b80678d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6824","nonce":2722096729},{"type":"v1","addr":"172.21.15.188:6825","nonce":2722096729}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6826","nonce":2722096729},{"type":"v1","addr":"172.21.15.188:6827","nonce":2722096729}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6830","nonce":2722096729},{"type":"v1","addr":"172.21.15.188:6831","nonce":2722096729}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6828","nonce":2722096729},{"type":"v1","addr":"172.21.15.188:6829","nonce":2722096729}]},"public_addr":"172.21.15.188:6825/2722096729","cluster_addr":"172.21.15.188:6827/2722096729","heartbeat_back_addr":"172.21.15.188:6831/2722096729","heartbeat_front_addr":"172.21.15.188:6829/2722096729","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-26T07:17:36.995032+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-26T07:17:36.932266+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-26T07:17:36.829304+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-26T07:17:36.804297+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-26T07:17:36.667892+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-26T07:17:36.964868+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-26T07:17:36.827332+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-26T07:17:36.893179+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"3.1f","mappings":[{"from":5,"to":1}]},{"pgid":"3.24","mappings":[{"from":3,"to":2}]},{"pgid":"3.26","mappings":[{"from":7,"to":2}]},{"pgid":"3.36","mappings":[{"from":5,"to":2}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-11-26T07:18:40.641 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:40.633+0000 7f9eeb357640 1 -- 172.21.15.143:0/309231424 >> [v2:172.21.15.143:6832/2150557544,v1:172.21.15.143:6833/2150557544] conn(0x7f9ec8066470 msgr2=0x7f9ec8068930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:18:40.641 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:40.633+0000 7f9eeb357640 1 --2- 172.21.15.143:0/309231424 >> [v2:172.21.15.143:6832/2150557544,v1:172.21.15.143:6833/2150557544] conn(0x7f9ec8066470 0x7f9ec8068930 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7f9ee0007fd0 tx=0x7f9ee0007e40 comp rx=0 tx=0).stop 2023-11-26T07:18:40.642 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:40.633+0000 7f9eeb357640 1 -- 172.21.15.143:0/309231424 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f9ee4143ac0 msgr2=0x7f9ee415ab00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:18:40.642 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:40.633+0000 7f9eeb357640 1 --2- 172.21.15.143:0/309231424 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f9ee4143ac0 0x7f9ee415ab00 secure :-1 s=READY pgs=342 cs=0 l=1 rev1=1 crypto rx=0x7f9edc007df0 tx=0x7f9edc00d4a0 comp rx=0 tx=0).stop 2023-11-26T07:18:40.642 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:40.633+0000 7f9eeb357640 1 -- 172.21.15.143:0/309231424 shutdown_connections 2023-11-26T07:18:40.642 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:40.633+0000 7f9eeb357640 1 --2- 172.21.15.143:0/309231424 >> [v2:172.21.15.143:6832/2150557544,v1:172.21.15.143:6833/2150557544] conn(0x7f9ec8066470 0x7f9ec8068930 unknown :-1 s=CLOSED pgs=189 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:18:40.642 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:40.633+0000 7f9eeb357640 1 --2- 172.21.15.143:0/309231424 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f9ee4143ac0 0x7f9ee415ab00 unknown :-1 s=CLOSED pgs=342 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:18:40.642 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:40.633+0000 7f9eeb357640 1 --2- 172.21.15.143:0/309231424 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f9ee40cf280 0x7f9ee41f73c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:18:40.642 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:40.633+0000 7f9eeb357640 1 --2- 172.21.15.143:0/309231424 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f9ee40cd1b0 0x7f9ee41593a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:18:40.642 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:40.633+0000 7f9eeb357640 1 -- 172.21.15.143:0/309231424 >> 172.21.15.143:0/309231424 conn(0x7f9ee406a890 msgr2=0x7f9ee406ac70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:18:40.642 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:40.633+0000 7f9eeb357640 1 -- 172.21.15.143:0/309231424 shutdown_connections 2023-11-26T07:18:40.642 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:40.633+0000 7f9eeb357640 1 -- 172.21.15.143:0/309231424 wait complete. 2023-11-26T07:18:40.660 INFO:tasks.cephfs.filesystem:Destroying file system cephfs and related pools 2023-11-26T07:18:40.660 DEBUG:teuthology.orchestra.run.smithi143:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-11-26T07:18:40.799 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:40.793+0000 7f5846606640 1 Processor -- start 2023-11-26T07:18:40.799 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:40.793+0000 7f5846606640 1 -- start start 2023-11-26T07:18:40.801 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:40.793+0000 7f5846606640 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7f58400cf280 0x7f58400cf710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:18:40.801 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:40.793+0000 7f5846606640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f58400cd1b0 0x7f58400cd5c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:18:40.801 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:40.793+0000 7f5846606640 1 -- --> v1:172.21.15.188:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f58400cc730 con 0x7f5840143ac0 2023-11-26T07:18:40.801 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:40.793+0000 7f5846606640 1 -- --> v2:172.21.15.143:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5840143ea0 con 0x7f58400cf280 2023-11-26T07:18:40.801 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:40.793+0000 7f5846606640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f58400cfc50 con 0x7f58400cd1b0 2023-11-26T07:18:40.801 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:40.797+0000 7f583f7fe640 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7f58400cf280 0x7f58400cf710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:18:40.801 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:40.797+0000 7f583f7fe640 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7f58400cf280 0x7f58400cf710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:39106/0 (socket says 172.21.15.143:39106) 2023-11-26T07:18:40.802 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:40.797+0000 7f583f7fe640 1 -- 172.21.15.143:0/1117656953 learned_addr learned my addr 172.21.15.143:0/1117656953 (peer_addr_for_me v2:172.21.15.143:0/0) 2023-11-26T07:18:40.802 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:40.797+0000 7f5844ba3640 1 --2- 172.21.15.143:0/1117656953 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f58400cd1b0 0x7f58400cd5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:18:40.802 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:40.797+0000 7f583f7fe640 1 -- 172.21.15.143:0/1117656953 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f58400cd1b0 msgr2=0x7f58400cd5c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-26T07:18:40.802 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:40.797+0000 7f583f7fe640 1 --2- 172.21.15.143:0/1117656953 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f58400cd1b0 0x7f58400cd5c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:18:40.802 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:40.797+0000 7f583f7fe640 1 -- 172.21.15.143:0/1117656953 >> v1:172.21.15.188:6789/0 conn(0x7f5840143ac0 legacy=0x7f5840070370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:18:40.802 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:40.797+0000 7f583f7fe640 1 -- 172.21.15.143:0/1117656953 --> v2:172.21.15.143:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5840070ee0 con 0x7f58400cf280 2023-11-26T07:18:40.804 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:40.797+0000 7f583f7fe640 1 --2- 172.21.15.143:0/1117656953 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f58400cf280 0x7f58400cf710 secure :-1 s=READY pgs=343 cs=0 l=1 rev1=1 crypto rx=0x7f5834009870 tx=0x7f583401c540 comp rx=0 tx=0).ready entity=mon.0 client_cookie=56da8e60f20a2aba server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:18:40.804 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:40.797+0000 7f583effd640 1 -- 172.21.15.143:0/1117656953 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f58340a9400 con 0x7f58400cf280 2023-11-26T07:18:40.804 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:40.797+0000 7f583effd640 1 -- 172.21.15.143:0/1117656953 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f58340a9560 con 0x7f58400cf280 2023-11-26T07:18:40.804 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:40.797+0000 7f583effd640 1 -- 172.21.15.143:0/1117656953 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f58340a98f0 con 0x7f58400cf280 2023-11-26T07:18:40.805 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:40.801+0000 7f5846606640 1 -- 172.21.15.143:0/1117656953 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f58400cf280 msgr2=0x7f58400cf710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:18:40.805 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:40.801+0000 7f5846606640 1 --2- 172.21.15.143:0/1117656953 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f58400cf280 0x7f58400cf710 secure :-1 s=READY pgs=343 cs=0 l=1 rev1=1 crypto rx=0x7f5834009870 tx=0x7f583401c540 comp rx=0 tx=0).stop 2023-11-26T07:18:40.805 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:40.801+0000 7f5846606640 1 -- 172.21.15.143:0/1117656953 shutdown_connections 2023-11-26T07:18:40.805 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:40.801+0000 7f5846606640 1 --2- 172.21.15.143:0/1117656953 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f58400cf280 0x7f58400cf710 unknown :-1 s=CLOSED pgs=343 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:18:40.805 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:40.801+0000 7f5846606640 1 --2- 172.21.15.143:0/1117656953 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f58400cd1b0 0x7f58400cd5c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:18:40.806 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:40.801+0000 7f5846606640 1 -- 172.21.15.143:0/1117656953 >> 172.21.15.143:0/1117656953 conn(0x7f584006a890 msgr2=0x7f584006aca0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:18:40.806 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:40.801+0000 7f5846606640 1 -- 172.21.15.143:0/1117656953 shutdown_connections 2023-11-26T07:18:40.806 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:40.801+0000 7f5846606640 1 -- 172.21.15.143:0/1117656953 wait complete. 2023-11-26T07:18:40.806 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:40.801+0000 7f5846606640 1 Processor -- start 2023-11-26T07:18:40.807 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:40.801+0000 7f5846606640 1 -- start start 2023-11-26T07:18:40.808 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:40.801+0000 7f5846606640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f58400cd1b0 0x7f58400adf10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:18:40.808 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:40.801+0000 7f5846606640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f58400cf280 0x7f58401588e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:18:40.808 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:40.801+0000 7f5846606640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f5840143ac0 0x7f58400aad10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:18:40.809 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:40.801+0000 7f5846606640 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5840158e20 con 0x7f58400cf280 2023-11-26T07:18:40.809 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:40.801+0000 7f5846606640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f58400ace70 con 0x7f58400cd1b0 2023-11-26T07:18:40.809 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:40.801+0000 7f5846606640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f58400ad150 con 0x7f5840143ac0 2023-11-26T07:18:40.809 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:40.801+0000 7f583f7fe640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f58400cf280 0x7f58401588e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:18:40.809 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:40.801+0000 7f583f7fe640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f58400cf280 0x7f58401588e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:39114/0 (socket says 172.21.15.143:39114) 2023-11-26T07:18:40.809 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:40.801+0000 7f583f7fe640 1 -- 172.21.15.143:0/1162759698 learned_addr learned my addr 172.21.15.143:0/1162759698 (peer_addr_for_me v2:172.21.15.143:0/0) 2023-11-26T07:18:40.809 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:40.801+0000 7f583ffff640 1 --2- 172.21.15.143:0/1162759698 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f58400cd1b0 0x7f58400adf10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:18:40.809 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:40.801+0000 7f5844ba3640 1 --2- 172.21.15.143:0/1162759698 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f5840143ac0 0x7f58400aad10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:18:40.809 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:40.801+0000 7f583f7fe640 1 -- 172.21.15.143:0/1162759698 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f5840143ac0 msgr2=0x7f58400aad10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:18:40.809 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:40.801+0000 7f583f7fe640 1 --2- 172.21.15.143:0/1162759698 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f5840143ac0 0x7f58400aad10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:18:40.809 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:40.801+0000 7f583f7fe640 1 -- 172.21.15.143:0/1162759698 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f58400cd1b0 msgr2=0x7f58400adf10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:18:40.809 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:40.801+0000 7f583f7fe640 1 --2- 172.21.15.143:0/1162759698 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f58400cd1b0 0x7f58400adf10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:18:40.810 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:40.801+0000 7f583f7fe640 1 -- 172.21.15.143:0/1162759698 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f58340094d0 con 0x7f58400cf280 2023-11-26T07:18:40.810 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:40.801+0000 7f5844ba3640 1 --2- 172.21.15.143:0/1162759698 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f5840143ac0 0x7f58400aad10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-26T07:18:40.810 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:40.801+0000 7f583ffff640 1 --2- 172.21.15.143:0/1162759698 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f58400cd1b0 0x7f58400adf10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-26T07:18:40.811 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:40.801+0000 7f583f7fe640 1 --2- 172.21.15.143:0/1162759698 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f58400cf280 0x7f58401588e0 secure :-1 s=READY pgs=344 cs=0 l=1 rev1=1 crypto rx=0x7f583401abf0 tx=0x7f58340192e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:18:40.811 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:40.805+0000 7f583cff9640 1 -- 172.21.15.143:0/1162759698 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f58340a9400 con 0x7f58400cf280 2023-11-26T07:18:40.811 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:40.805+0000 7f583cff9640 1 -- 172.21.15.143:0/1162759698 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f58340a9e90 con 0x7f58400cf280 2023-11-26T07:18:40.811 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:40.805+0000 7f583cff9640 1 -- 172.21.15.143:0/1162759698 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f58340b1a00 con 0x7f58400cf280 2023-11-26T07:18:40.811 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:40.805+0000 7f5846606640 1 -- 172.21.15.143:0/1162759698 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f58400ad430 con 0x7f58400cf280 2023-11-26T07:18:40.811 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:40.805+0000 7f5846606640 1 -- 172.21.15.143:0/1162759698 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f58400ad9f0 con 0x7f58400cf280 2023-11-26T07:18:40.811 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:40.805+0000 7f5846606640 1 -- 172.21.15.143:0/1162759698 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f58400b1b20 con 0x7f58400cf280 2023-11-26T07:18:40.811 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:40.805+0000 7f583cff9640 1 -- 172.21.15.143:0/1162759698 <== mon.0 v2:172.21.15.143:3300/0 4 ==== mgrmap(e 5) v1 ==== 85852+0+0 (secure 0 0 0) 0x7f58340b1b60 con 0x7f58400cf280 2023-11-26T07:18:40.812 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:40.805+0000 7f583cff9640 1 --2- 172.21.15.143:0/1162759698 >> [v2:172.21.15.143:6832/2150557544,v1:172.21.15.143:6833/2150557544] conn(0x7f582c066470 0x7f582c068930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:18:40.812 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:40.805+0000 7f583cff9640 1 -- 172.21.15.143:0/1162759698 <== mon.0 v2:172.21.15.143:3300/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 5825+0+0 (secure 0 0 0) 0x7f583411e260 con 0x7f58400cf280 2023-11-26T07:18:40.812 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:40.805+0000 7f583ffff640 1 --2- 172.21.15.143:0/1162759698 >> [v2:172.21.15.143:6832/2150557544,v1:172.21.15.143:6833/2150557544] conn(0x7f582c066470 0x7f582c068930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:18:40.812 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:40.809+0000 7f583ffff640 1 --2- 172.21.15.143:0/1162759698 >> [v2:172.21.15.143:6832/2150557544,v1:172.21.15.143:6833/2150557544] conn(0x7f582c066470 0x7f582c068930 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7f5830000f70 tx=0x7f5830009040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:18:40.816 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:40.809+0000 7f583cff9640 1 -- 172.21.15.143:0/1162759698 <== mon.0 v2:172.21.15.143:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172649 (secure 0 0 0) 0x7f58340eb770 con 0x7f58400cf280 2023-11-26T07:18:41.035 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:41.029+0000 7f5846606640 1 -- 172.21.15.143:0/1162759698 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f58400adbe0 con 0x7f58400cf280 2023-11-26T07:18:41.035 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:41.029+0000 7f583cff9640 1 -- 172.21.15.143:0/1162759698 <== mon.0 v2:172.21.15.143:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 8 v8) v1 ==== 93+0+4913 (secure 0 0 0) 0x7f58340a6030 con 0x7f58400cf280 2023-11-26T07:18:41.036 INFO:teuthology.orchestra.run.smithi143.stdout: 2023-11-26T07:18:41.036 INFO:teuthology.orchestra.run.smithi143.stdout:{"epoch":8,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4241,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.188:6834/652687322","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6832","nonce":652687322},{"type":"v1","addr":"172.21.15.188:6834","nonce":652687322}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":4266,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.143:6835/3850184704","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6834","nonce":3850184704},{"type":"v1","addr":"172.21.15.143:6835","nonce":3850184704}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":2},{"gid":4273,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.143:6837/1695478651","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6836","nonce":1695478651},{"type":"v1","addr":"172.21.15.143:6837","nonce":1695478651}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":8,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-11-26T07:17:53.876989+0000","modified":"2023-11-26T07:17:56.885608+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":4285},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4285":{"gid":4285,"name":"b","rank":0,"incarnation":4,"state":"up:active","state_seq":3,"addr":"172.21.15.188:6835/994617268","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6833","nonce":994617268},{"type":"v1","addr":"172.21.15.188:6835","nonce":994617268}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2023-11-26T07:18:41.036 INFO:teuthology.orchestra.run.smithi143.stderr:dumped fsmap epoch 8 2023-11-26T07:18:41.037 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:41.033+0000 7f5846606640 1 -- 172.21.15.143:0/1162759698 >> [v2:172.21.15.143:6832/2150557544,v1:172.21.15.143:6833/2150557544] conn(0x7f582c066470 msgr2=0x7f582c068930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:18:41.038 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:41.033+0000 7f5846606640 1 --2- 172.21.15.143:0/1162759698 >> [v2:172.21.15.143:6832/2150557544,v1:172.21.15.143:6833/2150557544] conn(0x7f582c066470 0x7f582c068930 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7f5830000f70 tx=0x7f5830009040 comp rx=0 tx=0).stop 2023-11-26T07:18:41.038 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:41.033+0000 7f5846606640 1 -- 172.21.15.143:0/1162759698 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f58400cf280 msgr2=0x7f58401588e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:18:41.038 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:41.033+0000 7f5846606640 1 --2- 172.21.15.143:0/1162759698 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f58400cf280 0x7f58401588e0 secure :-1 s=READY pgs=344 cs=0 l=1 rev1=1 crypto rx=0x7f583401abf0 tx=0x7f58340192e0 comp rx=0 tx=0).stop 2023-11-26T07:18:41.038 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:41.033+0000 7f5846606640 1 -- 172.21.15.143:0/1162759698 shutdown_connections 2023-11-26T07:18:41.038 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:41.033+0000 7f5846606640 1 --2- 172.21.15.143:0/1162759698 >> [v2:172.21.15.143:6832/2150557544,v1:172.21.15.143:6833/2150557544] conn(0x7f582c066470 0x7f582c068930 unknown :-1 s=CLOSED pgs=190 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:18:41.038 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:41.033+0000 7f5846606640 1 --2- 172.21.15.143:0/1162759698 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f5840143ac0 0x7f58400aad10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:18:41.038 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:41.033+0000 7f5846606640 1 --2- 172.21.15.143:0/1162759698 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f58400cf280 0x7f58401588e0 unknown :-1 s=CLOSED pgs=344 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:18:41.038 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:41.033+0000 7f5846606640 1 --2- 172.21.15.143:0/1162759698 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f58400cd1b0 0x7f58400adf10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:18:41.038 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:41.033+0000 7f5846606640 1 -- 172.21.15.143:0/1162759698 >> 172.21.15.143:0/1162759698 conn(0x7f584006a890 msgr2=0x7f58401596b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:18:41.038 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:41.033+0000 7f5846606640 1 -- 172.21.15.143:0/1162759698 shutdown_connections 2023-11-26T07:18:41.039 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:41.033+0000 7f5846606640 1 -- 172.21.15.143:0/1162759698 wait complete. 2023-11-26T07:18:41.057 DEBUG:teuthology.orchestra.run.smithi143:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-11-26T07:18:41.194 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:41.189+0000 7faaf2d8d640 1 Processor -- start 2023-11-26T07:18:41.194 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:41.189+0000 7faaf2d8d640 1 -- start start 2023-11-26T07:18:41.195 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:41.189+0000 7faaf2d8d640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7faaec143ac0 0x7faaec070370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:18:41.195 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:41.189+0000 7faaf2d8d640 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7faaec0cf990 0x7faaec0cd1b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:18:41.195 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:41.189+0000 7faaf2d8d640 1 -- --> v1:172.21.15.188:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7faaec0cc730 con 0x7faaec0708b0 2023-11-26T07:18:41.196 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:41.189+0000 7faaf2d8d640 1 -- --> v2:172.21.15.188:3300/0 -- mon_getmap magic: 0 v1 -- 0x7faaec143ea0 con 0x7faaec0cf990 2023-11-26T07:18:41.196 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:41.189+0000 7faaf2d8d640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7faaec070ee0 con 0x7faaec143ac0 2023-11-26T07:18:41.196 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:41.189+0000 7faaf132a640 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7faaec0cf990 0x7faaec0cd1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:18:41.196 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:41.189+0000 7faaf0b29640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7faaec143ac0 0x7faaec070370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:18:41.196 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:41.189+0000 7faaf0b29640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7faaec143ac0 0x7faaec070370 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.188:3301/0 says I am v2:172.21.15.143:43128/0 (socket says 172.21.15.143:43128) 2023-11-26T07:18:41.196 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:41.189+0000 7faaf0b29640 1 -- 172.21.15.143:0/3571116509 learned_addr learned my addr 172.21.15.143:0/3571116509 (peer_addr_for_me v2:172.21.15.143:0/0) 2023-11-26T07:18:41.196 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:41.189+0000 7faaf132a640 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7faaec0cf990 0x7faaec0cd1b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.188:3300/0 says I am v2:172.21.15.143:33370/0 (socket says 172.21.15.143:33370) 2023-11-26T07:18:41.197 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:41.189+0000 7faaf0b29640 1 -- 172.21.15.143:0/3571116509 >> v2:172.21.15.188:3300/0 conn(0x7faaec0cf990 msgr2=0x7faaec0cd1b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-26T07:18:41.197 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:41.189+0000 7faaf0b29640 1 --2- 172.21.15.143:0/3571116509 >> v2:172.21.15.188:3300/0 conn(0x7faaec0cf990 0x7faaec0cd1b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:18:41.197 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:41.189+0000 7faaf0b29640 1 -- 172.21.15.143:0/3571116509 >> v1:172.21.15.188:6789/0 conn(0x7faaec0708b0 legacy=0x7faaec0cf280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:18:41.197 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:41.189+0000 7faaf0b29640 1 -- 172.21.15.143:0/3571116509 --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7faaec0711c0 con 0x7faaec143ac0 2023-11-26T07:18:41.197 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:41.189+0000 7faaf132a640 1 --2- 172.21.15.143:0/3571116509 >> v2:172.21.15.188:3300/0 conn(0x7faaec0cf990 0x7faaec0cd1b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-26T07:18:41.199 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:41.193+0000 7faaf0b29640 1 --2- 172.21.15.143:0/3571116509 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7faaec143ac0 0x7faaec070370 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7faae800bb10 tx=0x7faae801ec10 comp rx=0 tx=0).ready entity=mon.2 client_cookie=e252ed525b6d49fd server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:18:41.199 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:41.193+0000 7faae37fe640 1 -- 172.21.15.143:0/3571116509 <== mon.2 v2:172.21.15.188:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faae80a7bf0 con 0x7faaec143ac0 2023-11-26T07:18:41.199 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:41.193+0000 7faae37fe640 1 -- 172.21.15.143:0/3571116509 <== mon.2 v2:172.21.15.188:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7faae80a7d50 con 0x7faaec143ac0 2023-11-26T07:18:41.199 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:41.193+0000 7faae37fe640 1 -- 172.21.15.143:0/3571116509 <== mon.2 v2:172.21.15.188:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faae80af660 con 0x7faaec143ac0 2023-11-26T07:18:41.200 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:41.193+0000 7faaf2d8d640 1 -- 172.21.15.143:0/3571116509 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7faaec143ac0 msgr2=0x7faaec070370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:18:41.200 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:41.193+0000 7faaf2d8d640 1 --2- 172.21.15.143:0/3571116509 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7faaec143ac0 0x7faaec070370 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7faae800bb10 tx=0x7faae801ec10 comp rx=0 tx=0).stop 2023-11-26T07:18:41.200 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:41.193+0000 7faaf2d8d640 1 -- 172.21.15.143:0/3571116509 shutdown_connections 2023-11-26T07:18:41.200 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:41.193+0000 7faaf2d8d640 1 --2- 172.21.15.143:0/3571116509 >> v2:172.21.15.188:3300/0 conn(0x7faaec0cf990 0x7faaec0cd1b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:18:41.200 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:41.193+0000 7faaf2d8d640 1 --2- 172.21.15.143:0/3571116509 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7faaec143ac0 0x7faaec070370 unknown :-1 s=CLOSED pgs=66 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:18:41.201 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:41.193+0000 7faaf2d8d640 1 -- 172.21.15.143:0/3571116509 >> 172.21.15.143:0/3571116509 conn(0x7faaec06a890 msgr2=0x7faaec06aca0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:18:41.201 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:41.193+0000 7faaf2d8d640 1 -- 172.21.15.143:0/3571116509 shutdown_connections 2023-11-26T07:18:41.201 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:41.193+0000 7faaf2d8d640 1 -- 172.21.15.143:0/3571116509 wait complete. 2023-11-26T07:18:41.201 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:41.197+0000 7faaf2d8d640 1 Processor -- start 2023-11-26T07:18:41.201 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:41.197+0000 7faaf2d8d640 1 -- start start 2023-11-26T07:18:41.203 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:41.197+0000 7faaf2d8d640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7faaec0708b0 0x7faaec141050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:18:41.204 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:41.197+0000 7faaf2d8d640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7faaec0cf990 0x7faaec0af8e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:18:41.204 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:41.197+0000 7faaf2d8d640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7faaec143ac0 0x7faaec0b0ff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:18:41.204 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:41.197+0000 7faaf2d8d640 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faaec0aeb60 con 0x7faaec0708b0 2023-11-26T07:18:41.204 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:41.197+0000 7faaf2d8d640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faaec141950 con 0x7faaec0cf990 2023-11-26T07:18:41.204 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:41.197+0000 7faaf2d8d640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7faaec0aee00 con 0x7faaec143ac0 2023-11-26T07:18:41.204 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:41.197+0000 7faaf0b29640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7faaec0708b0 0x7faaec141050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:18:41.204 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:41.197+0000 7faaf0b29640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7faaec0708b0 0x7faaec141050 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:39120/0 (socket says 172.21.15.143:39120) 2023-11-26T07:18:41.204 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:41.197+0000 7faaf0b29640 1 -- 172.21.15.143:0/2679091687 learned_addr learned my addr 172.21.15.143:0/2679091687 (peer_addr_for_me v2:172.21.15.143:0/0) 2023-11-26T07:18:41.204 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:41.197+0000 7faaf132a640 1 --2- 172.21.15.143:0/2679091687 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7faaec143ac0 0x7faaec0b0ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:18:41.204 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:41.197+0000 7faae3fff640 1 --2- 172.21.15.143:0/2679091687 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7faaec0cf990 0x7faaec0af8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:18:41.204 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:41.197+0000 7faaf0b29640 1 -- 172.21.15.143:0/2679091687 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7faaec143ac0 msgr2=0x7faaec0b0ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:18:41.205 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:41.197+0000 7faaf0b29640 1 --2- 172.21.15.143:0/2679091687 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7faaec143ac0 0x7faaec0b0ff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:18:41.205 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:41.197+0000 7faaf0b29640 1 -- 172.21.15.143:0/2679091687 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7faaec0cf990 msgr2=0x7faaec0af8e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:18:41.205 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:41.197+0000 7faaf0b29640 1 --2- 172.21.15.143:0/2679091687 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7faaec0cf990 0x7faaec0af8e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:18:41.205 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:41.197+0000 7faaf0b29640 1 -- 172.21.15.143:0/2679091687 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7faae4009a70 con 0x7faaec0708b0 2023-11-26T07:18:41.205 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:41.197+0000 7faaf0b29640 1 --2- 172.21.15.143:0/2679091687 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7faaec0708b0 0x7faaec141050 secure :-1 s=READY pgs=345 cs=0 l=1 rev1=1 crypto rx=0x7faae800b2c0 tx=0x7faae801b9b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:18:41.206 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:41.197+0000 7faae17fa640 1 -- 172.21.15.143:0/2679091687 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faae80afe30 con 0x7faaec0708b0 2023-11-26T07:18:41.206 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:41.197+0000 7faae17fa640 1 -- 172.21.15.143:0/2679091687 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7faae80ba440 con 0x7faaec0708b0 2023-11-26T07:18:41.206 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:41.197+0000 7faae17fa640 1 -- 172.21.15.143:0/2679091687 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faae80ba7f0 con 0x7faaec0708b0 2023-11-26T07:18:41.206 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:41.197+0000 7faaf2d8d640 1 -- 172.21.15.143:0/2679091687 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7faae800b770 con 0x7faaec0708b0 2023-11-26T07:18:41.206 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:41.197+0000 7faaf2d8d640 1 -- 172.21.15.143:0/2679091687 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7faaec0af3c0 con 0x7faaec0708b0 2023-11-26T07:18:41.206 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:41.201+0000 7faaf2d8d640 1 -- 172.21.15.143:0/2679091687 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7faaec0af5b0 con 0x7faaec0708b0 2023-11-26T07:18:41.206 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:41.201+0000 7faae17fa640 1 -- 172.21.15.143:0/2679091687 <== mon.0 v2:172.21.15.143:3300/0 4 ==== mgrmap(e 5) v1 ==== 85852+0+0 (secure 0 0 0) 0x7faae80b93e0 con 0x7faaec0708b0 2023-11-26T07:18:41.207 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:41.201+0000 7faae17fa640 1 --2- 172.21.15.143:0/2679091687 >> [v2:172.21.15.143:6832/2150557544,v1:172.21.15.143:6833/2150557544] conn(0x7faad8066470 0x7faad8068930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:18:41.207 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:41.201+0000 7faae17fa640 1 -- 172.21.15.143:0/2679091687 <== mon.0 v2:172.21.15.143:3300/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 5825+0+0 (secure 0 0 0) 0x7faae811e410 con 0x7faaec0708b0 2023-11-26T07:18:41.207 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:41.201+0000 7faae3fff640 1 --2- 172.21.15.143:0/2679091687 >> [v2:172.21.15.143:6832/2150557544,v1:172.21.15.143:6833/2150557544] conn(0x7faad8066470 0x7faad8068930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:18:41.207 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:41.201+0000 7faae3fff640 1 --2- 172.21.15.143:0/2679091687 >> [v2:172.21.15.143:6832/2150557544,v1:172.21.15.143:6833/2150557544] conn(0x7faad8066470 0x7faad8068930 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7faaec1593e0 tx=0x7faadc009040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:18:41.211 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:41.205+0000 7faae17fa640 1 -- 172.21.15.143:0/2679091687 <== mon.0 v2:172.21.15.143:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172649 (secure 0 0 0) 0x7faae80eb9d0 con 0x7faaec0708b0 2023-11-26T07:18:41.390 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:41.385+0000 7faaf2d8d640 1 -- 172.21.15.143:0/2679091687 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7faaec067680 con 0x7faaec0708b0 2023-11-26T07:18:41.391 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:41.385+0000 7faae17fa640 1 -- 172.21.15.143:0/2679091687 <== mon.0 v2:172.21.15.143:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 8 v8) v1 ==== 93+0+4913 (secure 0 0 0) 0x7faae80b4090 con 0x7faaec0708b0 2023-11-26T07:18:41.391 INFO:teuthology.orchestra.run.smithi143.stdout: 2023-11-26T07:18:41.391 INFO:teuthology.orchestra.run.smithi143.stdout:{"epoch":8,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4241,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.188:6834/652687322","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6832","nonce":652687322},{"type":"v1","addr":"172.21.15.188:6834","nonce":652687322}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":4266,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.143:6835/3850184704","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6834","nonce":3850184704},{"type":"v1","addr":"172.21.15.143:6835","nonce":3850184704}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":2},{"gid":4273,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.143:6837/1695478651","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6836","nonce":1695478651},{"type":"v1","addr":"172.21.15.143:6837","nonce":1695478651}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":8,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-11-26T07:17:53.876989+0000","modified":"2023-11-26T07:17:56.885608+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":4285},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4285":{"gid":4285,"name":"b","rank":0,"incarnation":4,"state":"up:active","state_seq":3,"addr":"172.21.15.188:6835/994617268","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6833","nonce":994617268},{"type":"v1","addr":"172.21.15.188:6835","nonce":994617268}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2023-11-26T07:18:41.391 INFO:teuthology.orchestra.run.smithi143.stderr:dumped fsmap epoch 8 2023-11-26T07:18:41.393 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:41.385+0000 7faaf2d8d640 1 -- 172.21.15.143:0/2679091687 >> [v2:172.21.15.143:6832/2150557544,v1:172.21.15.143:6833/2150557544] conn(0x7faad8066470 msgr2=0x7faad8068930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:18:41.393 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:41.385+0000 7faaf2d8d640 1 --2- 172.21.15.143:0/2679091687 >> [v2:172.21.15.143:6832/2150557544,v1:172.21.15.143:6833/2150557544] conn(0x7faad8066470 0x7faad8068930 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7faaec1593e0 tx=0x7faadc009040 comp rx=0 tx=0).stop 2023-11-26T07:18:41.393 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:41.385+0000 7faaf2d8d640 1 -- 172.21.15.143:0/2679091687 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7faaec0708b0 msgr2=0x7faaec141050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:18:41.393 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:41.385+0000 7faaf2d8d640 1 --2- 172.21.15.143:0/2679091687 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7faaec0708b0 0x7faaec141050 secure :-1 s=READY pgs=345 cs=0 l=1 rev1=1 crypto rx=0x7faae800b2c0 tx=0x7faae801b9b0 comp rx=0 tx=0).stop 2023-11-26T07:18:41.393 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:41.389+0000 7faaf2d8d640 1 -- 172.21.15.143:0/2679091687 shutdown_connections 2023-11-26T07:18:41.393 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:41.389+0000 7faaf2d8d640 1 --2- 172.21.15.143:0/2679091687 >> [v2:172.21.15.143:6832/2150557544,v1:172.21.15.143:6833/2150557544] conn(0x7faad8066470 0x7faad8068930 unknown :-1 s=CLOSED pgs=191 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:18:41.394 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:41.389+0000 7faaf2d8d640 1 --2- 172.21.15.143:0/2679091687 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7faaec143ac0 0x7faaec0b0ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:18:41.394 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:41.389+0000 7faaf2d8d640 1 --2- 172.21.15.143:0/2679091687 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7faaec0cf990 0x7faaec0af8e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:18:41.394 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:41.389+0000 7faaf2d8d640 1 --2- 172.21.15.143:0/2679091687 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7faaec0708b0 0x7faaec141050 unknown :-1 s=CLOSED pgs=345 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:18:41.394 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:41.389+0000 7faaf2d8d640 1 -- 172.21.15.143:0/2679091687 >> 172.21.15.143:0/2679091687 conn(0x7faaec06a890 msgr2=0x7faaec0ad380 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:18:41.394 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:41.389+0000 7faaf2d8d640 1 -- 172.21.15.143:0/2679091687 shutdown_connections 2023-11-26T07:18:41.394 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:41.389+0000 7faaf2d8d640 1 -- 172.21.15.143:0/2679091687 wait complete. 2023-11-26T07:18:41.412 DEBUG:teuthology.orchestra.run.smithi143:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-11-26T07:18:41.552 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:41.545+0000 7f94ace91640 1 Processor -- start 2023-11-26T07:18:41.553 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:41.549+0000 7f94ace91640 1 -- start start 2023-11-26T07:18:41.555 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:41.549+0000 7f94ace91640 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7f94a8143ac0 0x7f94a8070370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:18:41.555 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:41.549+0000 7f94ace91640 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7f94a80708b0 0x7f94a80cf280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:18:41.555 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:41.549+0000 7f94ace91640 1 -- --> v1:172.21.15.188:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f94a80cc730 con 0x7f94a80cf7c0 2023-11-26T07:18:41.555 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:41.549+0000 7f94ace91640 1 -- --> v2:172.21.15.143:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f94a8143ea0 con 0x7f94a8143ac0 2023-11-26T07:18:41.555 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:41.549+0000 7f94ace91640 1 -- --> v2:172.21.15.188:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f94a80cfbd0 con 0x7f94a80708b0 2023-11-26T07:18:41.555 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:41.549+0000 7f94a659c640 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7f94a8143ac0 0x7f94a8070370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:18:41.555 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:41.549+0000 7f94a659c640 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7f94a8143ac0 0x7f94a8070370 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:39134/0 (socket says 172.21.15.143:39134) 2023-11-26T07:18:41.555 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:41.549+0000 7f94a659c640 1 -- 172.21.15.143:0/3366036886 learned_addr learned my addr 172.21.15.143:0/3366036886 (peer_addr_for_me v2:172.21.15.143:0/0) 2023-11-26T07:18:41.555 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:41.549+0000 7f949ffff640 1 --2- 172.21.15.143:0/3366036886 >> v2:172.21.15.188:3300/0 conn(0x7f94a80708b0 0x7f94a80cf280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:18:41.556 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:41.549+0000 7f94a659c640 1 -- 172.21.15.143:0/3366036886 >> v2:172.21.15.188:3300/0 conn(0x7f94a80708b0 msgr2=0x7f94a80cf280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-26T07:18:41.556 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:41.549+0000 7f94a659c640 1 --2- 172.21.15.143:0/3366036886 >> v2:172.21.15.188:3300/0 conn(0x7f94a80708b0 0x7f94a80cf280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:18:41.556 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:41.549+0000 7f94a659c640 1 -- 172.21.15.143:0/3366036886 >> v1:172.21.15.188:6789/0 conn(0x7f94a80cf7c0 legacy=0x7f94a80b1ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-26T07:18:41.556 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:41.549+0000 7f94a659c640 1 -- 172.21.15.143:0/3366036886 --> v2:172.21.15.143:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f94a8070ee0 con 0x7f94a8143ac0 2023-11-26T07:18:41.558 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:41.553+0000 7f94a659c640 1 --2- 172.21.15.143:0/3366036886 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f94a8143ac0 0x7f94a8070370 secure :-1 s=READY pgs=346 cs=0 l=1 rev1=1 crypto rx=0x7f94a80aeb10 tx=0x7f949801c4c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9c282fe5b4264bb server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:18:41.558 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:41.553+0000 7f94a5d9b640 1 -- 172.21.15.143:0/3366036886 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f94980ac630 con 0x7f94a8143ac0 2023-11-26T07:18:41.558 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:41.553+0000 7f94a5d9b640 1 -- 172.21.15.143:0/3366036886 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f94980ac790 con 0x7f94a8143ac0 2023-11-26T07:18:41.558 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:41.553+0000 7f94a5d9b640 1 -- 172.21.15.143:0/3366036886 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f94980acb20 con 0x7f94a8143ac0 2023-11-26T07:18:41.559 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:41.553+0000 7f94ace91640 1 -- 172.21.15.143:0/3366036886 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f94a8143ac0 msgr2=0x7f94a8070370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:18:41.559 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:41.553+0000 7f94ace91640 1 --2- 172.21.15.143:0/3366036886 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f94a8143ac0 0x7f94a8070370 secure :-1 s=READY pgs=346 cs=0 l=1 rev1=1 crypto rx=0x7f94a80aeb10 tx=0x7f949801c4c0 comp rx=0 tx=0).stop 2023-11-26T07:18:41.560 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:41.553+0000 7f94ace91640 1 -- 172.21.15.143:0/3366036886 shutdown_connections 2023-11-26T07:18:41.560 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:41.553+0000 7f94ace91640 1 --2- 172.21.15.143:0/3366036886 >> v2:172.21.15.188:3300/0 conn(0x7f94a80708b0 0x7f94a80cf280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:18:41.560 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:41.553+0000 7f94ace91640 1 --2- 172.21.15.143:0/3366036886 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f94a8143ac0 0x7f94a8070370 unknown :-1 s=CLOSED pgs=346 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:18:41.560 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:41.553+0000 7f94ace91640 1 -- 172.21.15.143:0/3366036886 >> 172.21.15.143:0/3366036886 conn(0x7f94a806a890 msgr2=0x7f94a806aca0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:18:41.560 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:41.553+0000 7f94ace91640 1 -- 172.21.15.143:0/3366036886 shutdown_connections 2023-11-26T07:18:41.560 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:41.553+0000 7f94ace91640 1 -- 172.21.15.143:0/3366036886 wait complete. 2023-11-26T07:18:41.561 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:41.557+0000 7f94ace91640 1 Processor -- start 2023-11-26T07:18:41.561 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:41.557+0000 7f94ace91640 1 -- start start 2023-11-26T07:18:41.563 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:41.557+0000 7f94ace91640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f94a80708b0 0x7f94a81529b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:18:41.563 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:41.557+0000 7f94ace91640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f94a80cf7c0 0x7f94a8155050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:18:41.563 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:41.557+0000 7f94ace91640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f94a8143ac0 0x7f94a8155590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:18:41.563 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:41.557+0000 7f94ace91640 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f94a81f7b40 con 0x7f94a80cf7c0 2023-11-26T07:18:41.563 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:41.557+0000 7f94ace91640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f94a81f7e20 con 0x7f94a8143ac0 2023-11-26T07:18:41.563 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:41.557+0000 7f94ace91640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f94a81f8100 con 0x7f94a80708b0 2023-11-26T07:18:41.563 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:41.557+0000 7f949ffff640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f94a80cf7c0 0x7f94a8155050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:18:41.564 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:41.557+0000 7f949ffff640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f94a80cf7c0 0x7f94a8155050 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:39148/0 (socket says 172.21.15.143:39148) 2023-11-26T07:18:41.564 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:41.557+0000 7f949ffff640 1 -- 172.21.15.143:0/2700256653 learned_addr learned my addr 172.21.15.143:0/2700256653 (peer_addr_for_me v2:172.21.15.143:0/0) 2023-11-26T07:18:41.564 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:41.557+0000 7f94a659c640 1 --2- 172.21.15.143:0/2700256653 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f94a80708b0 0x7f94a81529b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:18:41.564 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:41.557+0000 7f94a6d9d640 1 --2- 172.21.15.143:0/2700256653 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f94a8143ac0 0x7f94a8155590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:18:41.564 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:41.557+0000 7f949ffff640 1 -- 172.21.15.143:0/2700256653 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f94a80708b0 msgr2=0x7f94a81529b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:18:41.564 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:41.557+0000 7f949ffff640 1 --2- 172.21.15.143:0/2700256653 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f94a80708b0 0x7f94a81529b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:18:41.564 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:41.557+0000 7f949ffff640 1 -- 172.21.15.143:0/2700256653 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f94a8143ac0 msgr2=0x7f94a8155590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:18:41.564 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:41.557+0000 7f949ffff640 1 --2- 172.21.15.143:0/2700256653 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f94a8143ac0 0x7f94a8155590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:18:41.564 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:41.557+0000 7f949ffff640 1 -- 172.21.15.143:0/2700256653 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f94980094d0 con 0x7f94a80cf7c0 2023-11-26T07:18:41.564 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:41.557+0000 7f949ffff640 1 --2- 172.21.15.143:0/2700256653 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f94a80cf7c0 0x7f94a8155050 secure :-1 s=READY pgs=347 cs=0 l=1 rev1=1 crypto rx=0x7f949000d800 tx=0x7f949000dcc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:18:41.564 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:41.557+0000 7f949effd640 1 -- 172.21.15.143:0/2700256653 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f949000b6c0 con 0x7f94a80cf7c0 2023-11-26T07:18:41.564 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:41.557+0000 7f949effd640 1 -- 172.21.15.143:0/2700256653 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f949000b820 con 0x7f94a80cf7c0 2023-11-26T07:18:41.565 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:41.557+0000 7f949effd640 1 -- 172.21.15.143:0/2700256653 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f949000bb90 con 0x7f94a80cf7c0 2023-11-26T07:18:41.565 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:41.557+0000 7f94ace91640 1 -- 172.21.15.143:0/2700256653 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f94a81f83e0 con 0x7f94a80cf7c0 2023-11-26T07:18:41.565 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:41.557+0000 7f94ace91640 1 -- 172.21.15.143:0/2700256653 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f94a8152ef0 con 0x7f94a80cf7c0 2023-11-26T07:18:41.565 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:41.557+0000 7f94ace91640 1 -- 172.21.15.143:0/2700256653 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f94a81530b0 con 0x7f94a80cf7c0 2023-11-26T07:18:41.565 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:41.561+0000 7f949effd640 1 -- 172.21.15.143:0/2700256653 <== mon.0 v2:172.21.15.143:3300/0 4 ==== mgrmap(e 5) v1 ==== 85852+0+0 (secure 0 0 0) 0x7f949000bcf0 con 0x7f94a80cf7c0 2023-11-26T07:18:41.566 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:41.561+0000 7f949effd640 1 --2- 172.21.15.143:0/2700256653 >> [v2:172.21.15.143:6832/2150557544,v1:172.21.15.143:6833/2150557544] conn(0x7f9488066470 0x7f9488068930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:18:41.566 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:41.561+0000 7f949effd640 1 -- 172.21.15.143:0/2700256653 <== mon.0 v2:172.21.15.143:3300/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 5825+0+0 (secure 0 0 0) 0x7f94900843a0 con 0x7f94a80cf7c0 2023-11-26T07:18:41.566 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:41.561+0000 7f94a659c640 1 --2- 172.21.15.143:0/2700256653 >> [v2:172.21.15.143:6832/2150557544,v1:172.21.15.143:6833/2150557544] conn(0x7f9488066470 0x7f9488068930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:18:41.567 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:41.561+0000 7f94a659c640 1 --2- 172.21.15.143:0/2700256653 >> [v2:172.21.15.143:6832/2150557544,v1:172.21.15.143:6833/2150557544] conn(0x7f9488066470 0x7f9488068930 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7f94980ae000 tx=0x7f94980a7000 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:18:41.570 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:41.565+0000 7f949effd640 1 -- 172.21.15.143:0/2700256653 <== mon.0 v2:172.21.15.143:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172649 (secure 0 0 0) 0x7f94900517b0 con 0x7f94a80cf7c0 2023-11-26T07:18:41.722 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:41.717+0000 7f94ace91640 1 -- 172.21.15.143:0/2700256653 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f94a8067680 con 0x7f94a80cf7c0 2023-11-26T07:18:41.723 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:41.717+0000 7f949effd640 1 -- 172.21.15.143:0/2700256653 <== mon.0 v2:172.21.15.143:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v22) v1 ==== 74+0+16731 (secure 0 0 0) 0x7f9490054dd0 con 0x7f94a80cf7c0 2023-11-26T07:18:41.723 INFO:teuthology.orchestra.run.smithi143.stdout: 2023-11-26T07:18:41.724 INFO:teuthology.orchestra.run.smithi143.stdout:{"epoch":22,"fsid":"f5033060-2469-4a07-95fe-2b76f5404bc1","created":"2023-11-26T07:17:28.613663+0000","modified":"2023-11-26T07:18:35.705228+0000","last_up_change":"2023-11-26T07:17:41.157813+0000","last_in_change":"2023-11-26T07:17:31.181658+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-11-26T07:17:41.758597+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2023-11-26T07:17:51.878332+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2023-11-26T07:17:52.871396+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"ebbee2c6-023e-44b9-9f1d-6f7bfd309730","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6817","nonce":639807369},{"type":"v1","addr":"172.21.15.143:6819","nonce":639807369}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6821","nonce":639807369},{"type":"v1","addr":"172.21.15.143:6823","nonce":639807369}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6830","nonce":639807369},{"type":"v1","addr":"172.21.15.143:6831","nonce":639807369}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6825","nonce":639807369},{"type":"v1","addr":"172.21.15.143:6828","nonce":639807369}]},"public_addr":"172.21.15.143:6819/639807369","cluster_addr":"172.21.15.143:6823/639807369","heartbeat_back_addr":"172.21.15.143:6831/639807369","heartbeat_front_addr":"172.21.15.143:6828/639807369","state":["exists","up"]},{"osd":1,"uuid":"fec4b8be-7122-49e9-9a19-5850a0700001","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6816","nonce":1144942530},{"type":"v1","addr":"172.21.15.143:6818","nonce":1144942530}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6820","nonce":1144942530},{"type":"v1","addr":"172.21.15.143:6822","nonce":1144942530}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6827","nonce":1144942530},{"type":"v1","addr":"172.21.15.143:6829","nonce":1144942530}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6824","nonce":1144942530},{"type":"v1","addr":"172.21.15.143:6826","nonce":1144942530}]},"public_addr":"172.21.15.143:6818/1144942530","cluster_addr":"172.21.15.143:6822/1144942530","heartbeat_back_addr":"172.21.15.143:6829/1144942530","heartbeat_front_addr":"172.21.15.143:6826/1144942530","state":["exists","up"]},{"osd":2,"uuid":"2888af5b-f933-46ca-8f95-6dd494ff43f6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6808","nonce":3909095186},{"type":"v1","addr":"172.21.15.143:6809","nonce":3909095186}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6810","nonce":3909095186},{"type":"v1","addr":"172.21.15.143:6811","nonce":3909095186}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6814","nonce":3909095186},{"type":"v1","addr":"172.21.15.143:6815","nonce":3909095186}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6812","nonce":3909095186},{"type":"v1","addr":"172.21.15.143:6813","nonce":3909095186}]},"public_addr":"172.21.15.143:6809/3909095186","cluster_addr":"172.21.15.143:6811/3909095186","heartbeat_back_addr":"172.21.15.143:6815/3909095186","heartbeat_front_addr":"172.21.15.143:6813/3909095186","state":["exists","up"]},{"osd":3,"uuid":"d1f52435-7bc1-4081-9550-9cd1ba55f549","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6800","nonce":3854210991},{"type":"v1","addr":"172.21.15.143:6801","nonce":3854210991}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6802","nonce":3854210991},{"type":"v1","addr":"172.21.15.143:6803","nonce":3854210991}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6806","nonce":3854210991},{"type":"v1","addr":"172.21.15.143:6807","nonce":3854210991}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6804","nonce":3854210991},{"type":"v1","addr":"172.21.15.143:6805","nonce":3854210991}]},"public_addr":"172.21.15.143:6801/3854210991","cluster_addr":"172.21.15.143:6803/3854210991","heartbeat_back_addr":"172.21.15.143:6807/3854210991","heartbeat_front_addr":"172.21.15.143:6805/3854210991","state":["exists","up"]},{"osd":4,"uuid":"e2ae8b6e-6e25-49b2-bcbf-aacce68f06a3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6800","nonce":3924606902},{"type":"v1","addr":"172.21.15.188:6801","nonce":3924606902}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6802","nonce":3924606902},{"type":"v1","addr":"172.21.15.188:6803","nonce":3924606902}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6806","nonce":3924606902},{"type":"v1","addr":"172.21.15.188:6807","nonce":3924606902}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6804","nonce":3924606902},{"type":"v1","addr":"172.21.15.188:6805","nonce":3924606902}]},"public_addr":"172.21.15.188:6801/3924606902","cluster_addr":"172.21.15.188:6803/3924606902","heartbeat_back_addr":"172.21.15.188:6807/3924606902","heartbeat_front_addr":"172.21.15.188:6805/3924606902","state":["exists","up"]},{"osd":5,"uuid":"3cec8a32-7698-4c36-8e89-e4ba11934f33","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6813","nonce":186841544},{"type":"v1","addr":"172.21.15.188:6815","nonce":186841544}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6817","nonce":186841544},{"type":"v1","addr":"172.21.15.188:6818","nonce":186841544}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6822","nonce":186841544},{"type":"v1","addr":"172.21.15.188:6823","nonce":186841544}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6820","nonce":186841544},{"type":"v1","addr":"172.21.15.188:6821","nonce":186841544}]},"public_addr":"172.21.15.188:6815/186841544","cluster_addr":"172.21.15.188:6818/186841544","heartbeat_back_addr":"172.21.15.188:6823/186841544","heartbeat_front_addr":"172.21.15.188:6821/186841544","state":["exists","up"]},{"osd":6,"uuid":"58a65d5d-e090-473a-81b4-ad38e85029f9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6808","nonce":2153499405},{"type":"v1","addr":"172.21.15.188:6809","nonce":2153499405}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6810","nonce":2153499405},{"type":"v1","addr":"172.21.15.188:6811","nonce":2153499405}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6816","nonce":2153499405},{"type":"v1","addr":"172.21.15.188:6819","nonce":2153499405}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6812","nonce":2153499405},{"type":"v1","addr":"172.21.15.188:6814","nonce":2153499405}]},"public_addr":"172.21.15.188:6809/2153499405","cluster_addr":"172.21.15.188:6811/2153499405","heartbeat_back_addr":"172.21.15.188:6819/2153499405","heartbeat_front_addr":"172.21.15.188:6814/2153499405","state":["exists","up"]},{"osd":7,"uuid":"ea5b7f70-3554-4cf8-ac30-f7c59b80678d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6824","nonce":2722096729},{"type":"v1","addr":"172.21.15.188:6825","nonce":2722096729}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6826","nonce":2722096729},{"type":"v1","addr":"172.21.15.188:6827","nonce":2722096729}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6830","nonce":2722096729},{"type":"v1","addr":"172.21.15.188:6831","nonce":2722096729}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6828","nonce":2722096729},{"type":"v1","addr":"172.21.15.188:6829","nonce":2722096729}]},"public_addr":"172.21.15.188:6825/2722096729","cluster_addr":"172.21.15.188:6827/2722096729","heartbeat_back_addr":"172.21.15.188:6831/2722096729","heartbeat_front_addr":"172.21.15.188:6829/2722096729","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-26T07:17:36.995032+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-26T07:17:36.932266+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-26T07:17:36.829304+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-26T07:17:36.804297+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-26T07:17:36.667892+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-26T07:17:36.964868+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-26T07:17:36.827332+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-26T07:17:36.893179+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"3.1f","mappings":[{"from":5,"to":1}]},{"pgid":"3.24","mappings":[{"from":3,"to":2}]},{"pgid":"3.26","mappings":[{"from":7,"to":2}]},{"pgid":"3.36","mappings":[{"from":5,"to":2}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-11-26T07:18:41.726 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:41.721+0000 7f94ace91640 1 -- 172.21.15.143:0/2700256653 >> [v2:172.21.15.143:6832/2150557544,v1:172.21.15.143:6833/2150557544] conn(0x7f9488066470 msgr2=0x7f9488068930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:18:41.726 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:41.721+0000 7f94ace91640 1 --2- 172.21.15.143:0/2700256653 >> [v2:172.21.15.143:6832/2150557544,v1:172.21.15.143:6833/2150557544] conn(0x7f9488066470 0x7f9488068930 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7f94980ae000 tx=0x7f94980a7000 comp rx=0 tx=0).stop 2023-11-26T07:18:41.726 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:41.721+0000 7f94ace91640 1 -- 172.21.15.143:0/2700256653 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f94a80cf7c0 msgr2=0x7f94a8155050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:18:41.726 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:41.721+0000 7f94ace91640 1 --2- 172.21.15.143:0/2700256653 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f94a80cf7c0 0x7f94a8155050 secure :-1 s=READY pgs=347 cs=0 l=1 rev1=1 crypto rx=0x7f949000d800 tx=0x7f949000dcc0 comp rx=0 tx=0).stop 2023-11-26T07:18:41.726 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:41.721+0000 7f94ace91640 1 -- 172.21.15.143:0/2700256653 shutdown_connections 2023-11-26T07:18:41.726 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:41.721+0000 7f94ace91640 1 --2- 172.21.15.143:0/2700256653 >> [v2:172.21.15.143:6832/2150557544,v1:172.21.15.143:6833/2150557544] conn(0x7f9488066470 0x7f9488068930 unknown :-1 s=CLOSED pgs=192 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:18:41.726 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:41.721+0000 7f94ace91640 1 --2- 172.21.15.143:0/2700256653 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f94a8143ac0 0x7f94a8155590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:18:41.726 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:41.721+0000 7f94ace91640 1 --2- 172.21.15.143:0/2700256653 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f94a80cf7c0 0x7f94a8155050 unknown :-1 s=CLOSED pgs=347 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:18:41.726 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:41.721+0000 7f94ace91640 1 --2- 172.21.15.143:0/2700256653 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f94a80708b0 0x7f94a81529b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:18:41.727 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:41.721+0000 7f94ace91640 1 -- 172.21.15.143:0/2700256653 >> 172.21.15.143:0/2700256653 conn(0x7f94a806a890 msgr2=0x7f94a80b1cb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:18:41.727 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:41.721+0000 7f94ace91640 1 -- 172.21.15.143:0/2700256653 shutdown_connections 2023-11-26T07:18:41.727 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:41.721+0000 7f94ace91640 1 -- 172.21.15.143:0/2700256653 wait complete. 2023-11-26T07:18:41.747 DEBUG:teuthology.orchestra.run.smithi143:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs fail cephfs 2023-11-26T07:18:41.885 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:41.881+0000 7fe657f0e640 1 Processor -- start 2023-11-26T07:18:41.885 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:41.881+0000 7fe657f0e640 1 -- start start 2023-11-26T07:18:41.886 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:41.881+0000 7fe657f0e640 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7fe6500ce680 0x7fe650071850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:18:41.886 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:41.881+0000 7fe657f0e640 1 -- --> v1:172.21.15.143:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe65006c750 con 0x7fe6501596d0 2023-11-26T07:18:41.887 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:41.881+0000 7fe657f0e640 1 -- --> v1:172.21.15.188:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe6500b3550 con 0x7fe6500b2f80 2023-11-26T07:18:41.887 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:41.881+0000 7fe657f0e640 1 -- --> v2:172.21.15.143:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe6500cbe00 con 0x7fe6500ce680 2023-11-26T07:18:41.887 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:41.881+0000 7fe655caa640 1 --1- >> v1:172.21.15.143:6789/0 conn(0x7fe6501596d0 0x7fe6500b2870 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.143:6789/0 says I am v1:172.21.15.143:50960/0 (socket says 172.21.15.143:50960) 2023-11-26T07:18:41.887 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:41.881+0000 7fe655caa640 1 -- 172.21.15.143:0/3418389389 learned_addr learned my addr 172.21.15.143:0/3418389389 (peer_addr_for_me v1:172.21.15.143:0/0) 2023-11-26T07:18:41.887 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:41.881+0000 7fe6564ab640 1 --2- 172.21.15.143:0/3418389389 >> v2:172.21.15.143:3300/0 conn(0x7fe6500ce680 0x7fe650071850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:18:41.887 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:41.881+0000 7fe654ca8640 1 -- 172.21.15.143:0/3418389389 <== mon.0 v1:172.21.15.143:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3351744039 0 0) 0x7fe65006c750 con 0x7fe6501596d0 2023-11-26T07:18:41.888 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:41.881+0000 7fe654ca8640 1 -- 172.21.15.143:0/3418389389 --> v1:172.21.15.143:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fe63c003450 con 0x7fe6501596d0 2023-11-26T07:18:41.888 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:41.881+0000 7fe6564ab640 1 -- 172.21.15.143:0/3418389389 >> v1:172.21.15.188:6789/0 conn(0x7fe6500b2f80 legacy=0x7fe65006cbe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:18:41.888 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:41.881+0000 7fe6564ab640 1 -- 172.21.15.143:0/3418389389 >> v1:172.21.15.143:6789/0 conn(0x7fe6501596d0 legacy=0x7fe6500b2870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:18:41.888 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:41.881+0000 7fe6564ab640 1 -- 172.21.15.143:0/3418389389 --> v2:172.21.15.143:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe650071e80 con 0x7fe6500ce680 2023-11-26T07:18:41.890 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:41.885+0000 7fe6564ab640 1 --2- 172.21.15.143:0/3418389389 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fe6500ce680 0x7fe650071850 secure :-1 s=READY pgs=349 cs=0 l=1 rev1=1 crypto rx=0x7fe6500cbf40 tx=0x7fe64801c8a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=32bb26746ec77283 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:18:41.890 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:41.885+0000 7fe654ca8640 1 -- 172.21.15.143:0/3418389389 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe6480a57d0 con 0x7fe6500ce680 2023-11-26T07:18:41.890 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:41.885+0000 7fe654ca8640 1 -- 172.21.15.143:0/3418389389 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe6480a5930 con 0x7fe6500ce680 2023-11-26T07:18:41.890 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:41.885+0000 7fe654ca8640 1 -- 172.21.15.143:0/3418389389 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe6480a5ca0 con 0x7fe6500ce680 2023-11-26T07:18:41.891 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:41.885+0000 7fe657f0e640 1 -- 172.21.15.143:0/3418389389 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fe6500ce680 msgr2=0x7fe650071850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:18:41.891 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:41.885+0000 7fe657f0e640 1 --2- 172.21.15.143:0/3418389389 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fe6500ce680 0x7fe650071850 secure :-1 s=READY pgs=349 cs=0 l=1 rev1=1 crypto rx=0x7fe6500cbf40 tx=0x7fe64801c8a0 comp rx=0 tx=0).stop 2023-11-26T07:18:41.891 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:41.885+0000 7fe657f0e640 1 -- 172.21.15.143:0/3418389389 shutdown_connections 2023-11-26T07:18:41.891 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:41.885+0000 7fe657f0e640 1 --2- 172.21.15.143:0/3418389389 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fe6500ce680 0x7fe650071850 unknown :-1 s=CLOSED pgs=349 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:18:41.891 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:41.885+0000 7fe657f0e640 1 -- 172.21.15.143:0/3418389389 >> 172.21.15.143:0/3418389389 conn(0x7fe65006a890 msgr2=0x7fe65006aca0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:18:41.892 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:41.885+0000 7fe657f0e640 1 -- 172.21.15.143:0/3418389389 shutdown_connections 2023-11-26T07:18:41.892 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:41.885+0000 7fe657f0e640 1 -- 172.21.15.143:0/3418389389 wait complete. 2023-11-26T07:18:41.892 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:41.885+0000 7fe657f0e640 1 Processor -- start 2023-11-26T07:18:41.893 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:41.889+0000 7fe657f0e640 1 -- start start 2023-11-26T07:18:41.894 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:41.889+0000 7fe657f0e640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fe6500b2f80 0x7fe650142840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:18:41.894 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:41.889+0000 7fe657f0e640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fe6500ce680 0x7fe65015a210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:18:41.894 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:41.889+0000 7fe657f0e640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fe6501596d0 0x7fe65015a750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:18:41.894 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:41.889+0000 7fe657f0e640 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe650145e40 con 0x7fe6500b2f80 2023-11-26T07:18:41.895 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:41.889+0000 7fe657f0e640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe650145b60 con 0x7fe6500ce680 2023-11-26T07:18:41.895 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:41.889+0000 7fe657f0e640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe65015bfb0 con 0x7fe6501596d0 2023-11-26T07:18:41.895 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:41.889+0000 7fe655caa640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fe6500b2f80 0x7fe650142840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:18:41.895 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:41.889+0000 7fe655caa640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fe6500b2f80 0x7fe650142840 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:39160/0 (socket says 172.21.15.143:39160) 2023-11-26T07:18:41.895 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:41.889+0000 7fe655caa640 1 -- 172.21.15.143:0/1376550947 learned_addr learned my addr 172.21.15.143:0/1376550947 (peer_addr_for_me v2:172.21.15.143:0/0) 2023-11-26T07:18:41.895 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:41.889+0000 7fe6564ab640 1 --2- 172.21.15.143:0/1376550947 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fe6501596d0 0x7fe65015a750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:18:41.895 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:41.889+0000 7fe6554a9640 1 --2- 172.21.15.143:0/1376550947 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fe6500ce680 0x7fe65015a210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:18:41.895 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:41.889+0000 7fe655caa640 1 -- 172.21.15.143:0/1376550947 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fe6501596d0 msgr2=0x7fe65015a750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:18:41.895 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:41.889+0000 7fe655caa640 1 --2- 172.21.15.143:0/1376550947 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fe6501596d0 0x7fe65015a750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:18:41.895 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:41.889+0000 7fe655caa640 1 -- 172.21.15.143:0/1376550947 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fe6500ce680 msgr2=0x7fe65015a210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:18:41.895 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:41.889+0000 7fe655caa640 1 --2- 172.21.15.143:0/1376550947 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fe6500ce680 0x7fe65015a210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:18:41.895 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:41.889+0000 7fe655caa640 1 -- 172.21.15.143:0/1376550947 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe648009a70 con 0x7fe6500b2f80 2023-11-26T07:18:41.896 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:41.889+0000 7fe655caa640 1 --2- 172.21.15.143:0/1376550947 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fe6500b2f80 0x7fe650142840 secure :-1 s=READY pgs=350 cs=0 l=1 rev1=1 crypto rx=0x7fe64c00c420 tx=0x7fe64c00c8e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:18:41.896 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:41.889+0000 7fe6467fc640 1 -- 172.21.15.143:0/1376550947 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe64c005cf0 con 0x7fe6500b2f80 2023-11-26T07:18:41.896 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:41.889+0000 7fe6467fc640 1 -- 172.21.15.143:0/1376550947 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe64c005e50 con 0x7fe6500b2f80 2023-11-26T07:18:41.897 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:41.889+0000 7fe6467fc640 1 -- 172.21.15.143:0/1376550947 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe64c0072d0 con 0x7fe6500b2f80 2023-11-26T07:18:41.897 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:41.889+0000 7fe657f0e640 1 -- 172.21.15.143:0/1376550947 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe65015c290 con 0x7fe6500b2f80 2023-11-26T07:18:41.897 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:41.889+0000 7fe657f0e640 1 -- 172.21.15.143:0/1376550947 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe65015ca90 con 0x7fe6500b2f80 2023-11-26T07:18:41.897 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:41.889+0000 7fe657f0e640 1 -- 172.21.15.143:0/1376550947 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe650067680 con 0x7fe6500b2f80 2023-11-26T07:18:41.897 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:41.893+0000 7fe6467fc640 1 -- 172.21.15.143:0/1376550947 <== mon.0 v2:172.21.15.143:3300/0 4 ==== mgrmap(e 5) v1 ==== 85852+0+0 (secure 0 0 0) 0x7fe64c007430 con 0x7fe6500b2f80 2023-11-26T07:18:41.897 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:41.893+0000 7fe6467fc640 1 --2- 172.21.15.143:0/1376550947 >> [v2:172.21.15.143:6832/2150557544,v1:172.21.15.143:6833/2150557544] conn(0x7fe634066470 0x7fe634068930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:18:41.898 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:41.893+0000 7fe6467fc640 1 -- 172.21.15.143:0/1376550947 <== mon.0 v2:172.21.15.143:3300/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 5825+0+0 (secure 0 0 0) 0x7fe64c0833a0 con 0x7fe6500b2f80 2023-11-26T07:18:41.898 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:41.893+0000 7fe6554a9640 1 --2- 172.21.15.143:0/1376550947 >> [v2:172.21.15.143:6832/2150557544,v1:172.21.15.143:6833/2150557544] conn(0x7fe634066470 0x7fe634068930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:18:41.898 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:41.893+0000 7fe6554a9640 1 --2- 172.21.15.143:0/1376550947 >> [v2:172.21.15.143:6832/2150557544,v1:172.21.15.143:6833/2150557544] conn(0x7fe634066470 0x7fe634068930 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7fe65015a0d0 tx=0x7fe640009040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:18:41.901 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:41.897+0000 7fe6467fc640 1 -- 172.21.15.143:0/1376550947 <== mon.0 v2:172.21.15.143:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172649 (secure 0 0 0) 0x7fe64c0508c0 con 0x7fe6500b2f80 2023-11-26T07:18:42.092 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:42.085+0000 7fe657f0e640 1 -- 172.21.15.143:0/1376550947 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs"} v 0) v1 -- 0x7fe65015c740 con 0x7fe6500b2f80 2023-11-26T07:18:42.092 INFO:tasks.ceph.mon.a.smithi143.stderr:2023-11-26T07:18:42.085+0000 7f6946f73640 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2023-11-26T07:18:42.095 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:42.089+0000 7fe6467fc640 1 -- 172.21.15.143:0/1376550947 <== mon.0 v2:172.21.15.143:3300/0 7 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs"}]=0 cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v9) v1 ==== 161+0+0 (secure 0 0 0) 0x7fe64c053ee0 con 0x7fe6500b2f80 2023-11-26T07:18:42.095 INFO:teuthology.orchestra.run.smithi143.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2023-11-26T07:18:42.097 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:42.093+0000 7fe657f0e640 1 -- 172.21.15.143:0/1376550947 >> [v2:172.21.15.143:6832/2150557544,v1:172.21.15.143:6833/2150557544] conn(0x7fe634066470 msgr2=0x7fe634068930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:18:42.097 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:42.093+0000 7fe657f0e640 1 --2- 172.21.15.143:0/1376550947 >> [v2:172.21.15.143:6832/2150557544,v1:172.21.15.143:6833/2150557544] conn(0x7fe634066470 0x7fe634068930 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7fe65015a0d0 tx=0x7fe640009040 comp rx=0 tx=0).stop 2023-11-26T07:18:42.098 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:42.093+0000 7fe657f0e640 1 -- 172.21.15.143:0/1376550947 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fe6500b2f80 msgr2=0x7fe650142840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:18:42.098 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:42.093+0000 7fe657f0e640 1 --2- 172.21.15.143:0/1376550947 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fe6500b2f80 0x7fe650142840 secure :-1 s=READY pgs=350 cs=0 l=1 rev1=1 crypto rx=0x7fe64c00c420 tx=0x7fe64c00c8e0 comp rx=0 tx=0).stop 2023-11-26T07:18:42.098 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:42.093+0000 7fe657f0e640 1 -- 172.21.15.143:0/1376550947 shutdown_connections 2023-11-26T07:18:42.098 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:42.093+0000 7fe657f0e640 1 --2- 172.21.15.143:0/1376550947 >> [v2:172.21.15.143:6832/2150557544,v1:172.21.15.143:6833/2150557544] conn(0x7fe634066470 0x7fe634068930 unknown :-1 s=CLOSED pgs=193 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:18:42.098 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:42.093+0000 7fe657f0e640 1 --2- 172.21.15.143:0/1376550947 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fe6501596d0 0x7fe65015a750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:18:42.098 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:42.093+0000 7fe657f0e640 1 --2- 172.21.15.143:0/1376550947 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fe6500ce680 0x7fe65015a210 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:18:42.098 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:42.093+0000 7fe657f0e640 1 --2- 172.21.15.143:0/1376550947 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fe6500b2f80 0x7fe650142840 unknown :-1 s=CLOSED pgs=350 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:18:42.098 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:42.093+0000 7fe657f0e640 1 -- 172.21.15.143:0/1376550947 >> 172.21.15.143:0/1376550947 conn(0x7fe65006a890 msgr2=0x7fe65015cf80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:18:42.098 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:42.093+0000 7fe657f0e640 1 -- 172.21.15.143:0/1376550947 shutdown_connections 2023-11-26T07:18:42.098 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:42.093+0000 7fe657f0e640 1 -- 172.21.15.143:0/1376550947 wait complete. 2023-11-26T07:18:42.106 INFO:tasks.ceph.mds.b.smithi188.stderr: -2740> 2023-11-26T07:17:58.238+0000 7fde7bce6640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2023-11-26T07:18:42.117 DEBUG:teuthology.orchestra.run.smithi143:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs rm cephfs --yes-i-really-mean-it 2023-11-26T07:18:42.155 INFO:tasks.ceph.mds.b.smithi188.stdout:starting mds.b at 2023-11-26T07:18:42.256 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:42.249+0000 7fb925a68640 1 Processor -- start 2023-11-26T07:18:42.256 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:42.249+0000 7fb925a68640 1 -- start start 2023-11-26T07:18:42.258 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:42.253+0000 7fb925a68640 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7fb9201596d0 0x7fb920071850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:18:42.258 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:42.253+0000 7fb925a68640 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7fb9200ce680 0x7fb9200b2870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:18:42.258 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:42.253+0000 7fb925a68640 1 -- --> v1:172.21.15.143:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb9200b3200 con 0x7fb9200b2db0 2023-11-26T07:18:42.259 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:42.253+0000 7fb925a68640 1 -- --> v2:172.21.15.143:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb9200cbe00 con 0x7fb9200ce680 2023-11-26T07:18:42.259 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:42.253+0000 7fb925a68640 1 -- --> v2:172.21.15.188:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb92006c750 con 0x7fb9201596d0 2023-11-26T07:18:42.259 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:42.253+0000 7fb91effd640 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7fb9200ce680 0x7fb9200b2870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:18:42.259 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:42.253+0000 7fb91ffff640 1 --1- >> v1:172.21.15.143:6789/0 conn(0x7fb9200b2db0 0x7fb92015d2c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.143:6789/0 says I am v1:172.21.15.143:50974/0 (socket says 172.21.15.143:50974) 2023-11-26T07:18:42.259 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:42.253+0000 7fb91ffff640 1 -- 172.21.15.143:0/4053029193 learned_addr learned my addr 172.21.15.143:0/4053029193 (peer_addr_for_me v1:172.21.15.143:0/0) 2023-11-26T07:18:42.259 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:42.253+0000 7fb91f7fe640 1 --2- 172.21.15.143:0/4053029193 >> v2:172.21.15.188:3300/0 conn(0x7fb9201596d0 0x7fb920071850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:18:42.259 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:42.253+0000 7fb91effd640 1 -- 172.21.15.143:0/4053029193 >> v2:172.21.15.188:3300/0 conn(0x7fb9201596d0 msgr2=0x7fb920071850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-26T07:18:42.259 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:42.253+0000 7fb91effd640 1 --2- 172.21.15.143:0/4053029193 >> v2:172.21.15.188:3300/0 conn(0x7fb9201596d0 0x7fb920071850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:18:42.259 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:42.253+0000 7fb91effd640 1 -- 172.21.15.143:0/4053029193 >> v1:172.21.15.143:6789/0 conn(0x7fb9200b2db0 legacy=0x7fb92015d2c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:18:42.259 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:42.253+0000 7fb91effd640 1 -- 172.21.15.143:0/4053029193 --> v2:172.21.15.143:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb9200b3550 con 0x7fb9200ce680 2023-11-26T07:18:42.261 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:42.257+0000 7fb91effd640 1 --2- 172.21.15.143:0/4053029193 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fb9200ce680 0x7fb9200b2870 secure :-1 s=READY pgs=353 cs=0 l=1 rev1=1 crypto rx=0x7fb9200cbf40 tx=0x7fb91401c440 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1904b7f7a5817101 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:18:42.261 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:42.257+0000 7fb91e7fc640 1 -- 172.21.15.143:0/4053029193 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb9140ab600 con 0x7fb9200ce680 2023-11-26T07:18:42.261 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:42.257+0000 7fb91e7fc640 1 -- 172.21.15.143:0/4053029193 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb9140a7050 con 0x7fb9200ce680 2023-11-26T07:18:42.262 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:42.257+0000 7fb91e7fc640 1 -- 172.21.15.143:0/4053029193 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb9140ab980 con 0x7fb9200ce680 2023-11-26T07:18:42.262 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:42.257+0000 7fb925a68640 1 -- 172.21.15.143:0/4053029193 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fb9200ce680 msgr2=0x7fb9200b2870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:18:42.262 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:42.257+0000 7fb925a68640 1 --2- 172.21.15.143:0/4053029193 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fb9200ce680 0x7fb9200b2870 secure :-1 s=READY pgs=353 cs=0 l=1 rev1=1 crypto rx=0x7fb9200cbf40 tx=0x7fb91401c440 comp rx=0 tx=0).stop 2023-11-26T07:18:42.262 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:42.257+0000 7fb925a68640 1 -- 172.21.15.143:0/4053029193 shutdown_connections 2023-11-26T07:18:42.263 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:42.257+0000 7fb925a68640 1 --2- 172.21.15.143:0/4053029193 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fb9200ce680 0x7fb9200b2870 unknown :-1 s=CLOSED pgs=353 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:18:42.263 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:42.257+0000 7fb925a68640 1 --2- 172.21.15.143:0/4053029193 >> v2:172.21.15.188:3300/0 conn(0x7fb9201596d0 0x7fb920071850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:18:42.263 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:42.257+0000 7fb925a68640 1 -- 172.21.15.143:0/4053029193 >> 172.21.15.143:0/4053029193 conn(0x7fb92006a890 msgr2=0x7fb92006aca0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:18:42.263 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:42.257+0000 7fb925a68640 1 -- 172.21.15.143:0/4053029193 shutdown_connections 2023-11-26T07:18:42.263 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:42.257+0000 7fb925a68640 1 -- 172.21.15.143:0/4053029193 wait complete. 2023-11-26T07:18:42.264 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:42.257+0000 7fb925a68640 1 Processor -- start 2023-11-26T07:18:42.264 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:42.257+0000 7fb925a68640 1 -- start start 2023-11-26T07:18:42.266 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:42.261+0000 7fb925a68640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fb9200b2db0 0x7fb920148ba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:18:42.266 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:42.261+0000 7fb925a68640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fb9200ce680 0x7fb92014c810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:18:42.267 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:42.261+0000 7fb925a68640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fb9201596d0 0x7fb92014ed40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:18:42.267 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:42.261+0000 7fb925a68640 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb92014c3f0 con 0x7fb9200b2db0 2023-11-26T07:18:42.267 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:42.261+0000 7fb925a68640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb92014bae0 con 0x7fb9201596d0 2023-11-26T07:18:42.267 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:42.261+0000 7fb925a68640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb92014e770 con 0x7fb9200ce680 2023-11-26T07:18:42.267 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:42.261+0000 7fb91f7fe640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fb9200b2db0 0x7fb920148ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:18:42.267 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:42.261+0000 7fb91f7fe640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fb9200b2db0 0x7fb920148ba0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:39176/0 (socket says 172.21.15.143:39176) 2023-11-26T07:18:42.267 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:42.261+0000 7fb91f7fe640 1 -- 172.21.15.143:0/1999268548 learned_addr learned my addr 172.21.15.143:0/1999268548 (peer_addr_for_me v2:172.21.15.143:0/0) 2023-11-26T07:18:42.267 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:42.261+0000 7fb91effd640 1 --2- 172.21.15.143:0/1999268548 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fb9200ce680 0x7fb92014c810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:18:42.267 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:42.261+0000 7fb91ffff640 1 --2- 172.21.15.143:0/1999268548 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fb9201596d0 0x7fb92014ed40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:18:42.267 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:42.261+0000 7fb91f7fe640 1 -- 172.21.15.143:0/1999268548 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fb9200ce680 msgr2=0x7fb92014c810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:18:42.267 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:42.261+0000 7fb91f7fe640 1 --2- 172.21.15.143:0/1999268548 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fb9200ce680 0x7fb92014c810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:18:42.267 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:42.261+0000 7fb91f7fe640 1 -- 172.21.15.143:0/1999268548 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fb9201596d0 msgr2=0x7fb92014ed40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:18:42.268 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:42.261+0000 7fb91f7fe640 1 --2- 172.21.15.143:0/1999268548 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fb9201596d0 0x7fb92014ed40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:18:42.268 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:42.261+0000 7fb91f7fe640 1 -- 172.21.15.143:0/1999268548 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb914009420 con 0x7fb9200b2db0 2023-11-26T07:18:42.268 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:42.261+0000 7fb91effd640 1 --2- 172.21.15.143:0/1999268548 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fb9200ce680 0x7fb92014c810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-26T07:18:42.268 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:42.261+0000 7fb91f7fe640 1 --2- 172.21.15.143:0/1999268548 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fb9200b2db0 0x7fb920148ba0 secure :-1 s=READY pgs=354 cs=0 l=1 rev1=1 crypto rx=0x7fb91000e870 tx=0x7fb91000ec70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:18:42.268 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:42.261+0000 7fb907fff640 1 -- 172.21.15.143:0/1999268548 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb910009510 con 0x7fb9200b2db0 2023-11-26T07:18:42.268 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:42.261+0000 7fb907fff640 1 -- 172.21.15.143:0/1999268548 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb91000eec0 con 0x7fb9200b2db0 2023-11-26T07:18:42.268 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:42.261+0000 7fb925a68640 1 -- 172.21.15.143:0/1999268548 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb92014bd30 con 0x7fb9200b2db0 2023-11-26T07:18:42.268 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:42.261+0000 7fb907fff640 1 -- 172.21.15.143:0/1999268548 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb91000cb40 con 0x7fb9200b2db0 2023-11-26T07:18:42.268 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:42.261+0000 7fb925a68640 1 -- 172.21.15.143:0/1999268548 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb9201490e0 con 0x7fb9200b2db0 2023-11-26T07:18:42.269 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:42.261+0000 7fb925a68640 1 -- 172.21.15.143:0/1999268548 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb920067680 con 0x7fb9200b2db0 2023-11-26T07:18:42.269 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:42.261+0000 7fb907fff640 1 -- 172.21.15.143:0/1999268548 <== mon.0 v2:172.21.15.143:3300/0 4 ==== mgrmap(e 5) v1 ==== 85852+0+0 (secure 0 0 0) 0x7fb91000cca0 con 0x7fb9200b2db0 2023-11-26T07:18:42.269 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:42.265+0000 7fb907fff640 1 --2- 172.21.15.143:0/1999268548 >> [v2:172.21.15.143:6832/2150557544,v1:172.21.15.143:6833/2150557544] conn(0x7fb90c0664a0 0x7fb90c068960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:18:42.269 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:42.265+0000 7fb907fff640 1 -- 172.21.15.143:0/1999268548 <== mon.0 v2:172.21.15.143:3300/0 5 ==== osd_map(23..23 src has 1..23) v4 ==== 5911+0+0 (secure 0 0 0) 0x7fb910083f90 con 0x7fb9200b2db0 2023-11-26T07:18:42.269 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:42.265+0000 7fb91effd640 1 --2- 172.21.15.143:0/1999268548 >> [v2:172.21.15.143:6832/2150557544,v1:172.21.15.143:6833/2150557544] conn(0x7fb90c0664a0 0x7fb90c068960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:18:42.270 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:42.265+0000 7fb91effd640 1 --2- 172.21.15.143:0/1999268548 >> [v2:172.21.15.143:6832/2150557544,v1:172.21.15.143:6833/2150557544] conn(0x7fb90c0664a0 0x7fb90c068960 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7fb9140ad000 tx=0x7fb914008040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:18:42.273 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:42.269+0000 7fb907fff640 1 -- 172.21.15.143:0/1999268548 <== mon.0 v2:172.21.15.143:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172649 (secure 0 0 0) 0x7fb9100b5280 con 0x7fb9200b2db0 2023-11-26T07:18:42.454 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:42.449+0000 7fb925a68640 1 -- 172.21.15.143:0/1999268548 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true} v 0) v1 -- 0x7fb9200709f0 con 0x7fb9200b2db0 2023-11-26T07:18:43.097 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:43.093+0000 7fb907fff640 1 -- 172.21.15.143:0/1999268548 <== mon.0 v2:172.21.15.143:3300/0 7 ==== mon_command_ack([{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]=0 v10) v1 ==== 104+0+0 (secure 0 0 0) 0x7fb9100513d0 con 0x7fb9200b2db0 2023-11-26T07:18:43.100 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:43.093+0000 7fb925a68640 1 -- 172.21.15.143:0/1999268548 >> [v2:172.21.15.143:6832/2150557544,v1:172.21.15.143:6833/2150557544] conn(0x7fb90c0664a0 msgr2=0x7fb90c068960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:18:43.101 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:43.093+0000 7fb925a68640 1 --2- 172.21.15.143:0/1999268548 >> [v2:172.21.15.143:6832/2150557544,v1:172.21.15.143:6833/2150557544] conn(0x7fb90c0664a0 0x7fb90c068960 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7fb9140ad000 tx=0x7fb914008040 comp rx=0 tx=0).stop 2023-11-26T07:18:43.101 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:43.093+0000 7fb925a68640 1 -- 172.21.15.143:0/1999268548 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fb9200b2db0 msgr2=0x7fb920148ba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:18:43.101 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:43.093+0000 7fb925a68640 1 --2- 172.21.15.143:0/1999268548 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fb9200b2db0 0x7fb920148ba0 secure :-1 s=READY pgs=354 cs=0 l=1 rev1=1 crypto rx=0x7fb91000e870 tx=0x7fb91000ec70 comp rx=0 tx=0).stop 2023-11-26T07:18:43.101 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:43.093+0000 7fb925a68640 1 -- 172.21.15.143:0/1999268548 shutdown_connections 2023-11-26T07:18:43.101 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:43.093+0000 7fb925a68640 1 --2- 172.21.15.143:0/1999268548 >> [v2:172.21.15.143:6832/2150557544,v1:172.21.15.143:6833/2150557544] conn(0x7fb90c0664a0 0x7fb90c068960 unknown :-1 s=CLOSED pgs=194 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:18:43.101 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:43.093+0000 7fb925a68640 1 --2- 172.21.15.143:0/1999268548 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fb9201596d0 0x7fb92014ed40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:18:43.101 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:43.093+0000 7fb925a68640 1 --2- 172.21.15.143:0/1999268548 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fb9200ce680 0x7fb92014c810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:18:43.101 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:43.093+0000 7fb925a68640 1 --2- 172.21.15.143:0/1999268548 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fb9200b2db0 0x7fb920148ba0 unknown :-1 s=CLOSED pgs=354 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:18:43.101 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:43.093+0000 7fb925a68640 1 -- 172.21.15.143:0/1999268548 >> 172.21.15.143:0/1999268548 conn(0x7fb92006a890 msgr2=0x7fb92014fdd0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:18:43.101 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:43.097+0000 7fb925a68640 1 -- 172.21.15.143:0/1999268548 shutdown_connections 2023-11-26T07:18:43.101 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:43.097+0000 7fb925a68640 1 -- 172.21.15.143:0/1999268548 wait complete. 2023-11-26T07:18:43.122 DEBUG:teuthology.orchestra.run.smithi143:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_metadata cephfs_metadata --yes-i-really-really-mean-it 2023-11-26T07:18:43.265 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:43.261+0000 7f2cdd12a640 1 Processor -- start 2023-11-26T07:18:43.265 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:43.261+0000 7f2cdd12a640 1 -- start start 2023-11-26T07:18:43.267 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:43.261+0000 7f2cdd12a640 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7f2cd80cf280 0x7f2cd80cf710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:18:43.267 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:43.261+0000 7f2cdd12a640 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7f2cd80cd1b0 0x7f2cd80cd5c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:18:43.267 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:43.261+0000 7f2cdd12a640 1 -- --> v1:172.21.15.188:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2cd80cc730 con 0x7f2cd8143ac0 2023-11-26T07:18:43.267 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:43.261+0000 7f2cdd12a640 1 -- --> v2:172.21.15.143:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2cd8143ea0 con 0x7f2cd80cf280 2023-11-26T07:18:43.267 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:43.261+0000 7f2cdd12a640 1 -- --> v2:172.21.15.188:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2cd80cfc50 con 0x7f2cd80cd1b0 2023-11-26T07:18:43.267 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:43.261+0000 7f2cd659c640 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7f2cd80cf280 0x7f2cd80cf710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:18:43.267 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:43.261+0000 7f2cd659c640 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7f2cd80cf280 0x7f2cd80cf710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:39192/0 (socket says 172.21.15.143:39192) 2023-11-26T07:18:43.267 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:43.261+0000 7f2cd659c640 1 -- 172.21.15.143:0/1828722100 learned_addr learned my addr 172.21.15.143:0/1828722100 (peer_addr_for_me v2:172.21.15.143:0/0) 2023-11-26T07:18:43.267 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:43.261+0000 7f2cd759e640 1 --2- 172.21.15.143:0/1828722100 >> v2:172.21.15.188:3300/0 conn(0x7f2cd80cd1b0 0x7f2cd80cd5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:18:43.268 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:43.261+0000 7f2cd659c640 1 -- 172.21.15.143:0/1828722100 >> v2:172.21.15.188:3300/0 conn(0x7f2cd80cd1b0 msgr2=0x7f2cd80cd5c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-26T07:18:43.268 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:43.261+0000 7f2cd659c640 1 --2- 172.21.15.143:0/1828722100 >> v2:172.21.15.188:3300/0 conn(0x7f2cd80cd1b0 0x7f2cd80cd5c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:18:43.268 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:43.261+0000 7f2cd659c640 1 -- 172.21.15.143:0/1828722100 >> v1:172.21.15.188:6789/0 conn(0x7f2cd8143ac0 legacy=0x7f2cd8070370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-26T07:18:43.268 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:43.261+0000 7f2cd659c640 1 -- 172.21.15.143:0/1828722100 --> v2:172.21.15.143:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2cd8070ee0 con 0x7f2cd80cf280 2023-11-26T07:18:43.270 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:43.265+0000 7f2cd659c640 1 --2- 172.21.15.143:0/1828722100 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f2cd80cf280 0x7f2cd80cf710 secure :-1 s=READY pgs=355 cs=0 l=1 rev1=1 crypto rx=0x7f2cd806e570 tx=0x7f2ccc01c4c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2bdc8a8108a0bc94 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:18:43.270 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:43.265+0000 7f2cd5d9b640 1 -- 172.21.15.143:0/1828722100 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2ccc0aa620 con 0x7f2cd80cf280 2023-11-26T07:18:43.270 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:43.265+0000 7f2cd5d9b640 1 -- 172.21.15.143:0/1828722100 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2ccc0aa780 con 0x7f2cd80cf280 2023-11-26T07:18:43.271 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:43.265+0000 7f2cd5d9b640 1 -- 172.21.15.143:0/1828722100 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2ccc0aab10 con 0x7f2cd80cf280 2023-11-26T07:18:43.271 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:43.265+0000 7f2cdd12a640 1 -- 172.21.15.143:0/1828722100 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f2cd80cf280 msgr2=0x7f2cd80cf710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:18:43.271 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:43.265+0000 7f2cdd12a640 1 --2- 172.21.15.143:0/1828722100 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f2cd80cf280 0x7f2cd80cf710 secure :-1 s=READY pgs=355 cs=0 l=1 rev1=1 crypto rx=0x7f2cd806e570 tx=0x7f2ccc01c4c0 comp rx=0 tx=0).stop 2023-11-26T07:18:43.271 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:43.265+0000 7f2cdd12a640 1 -- 172.21.15.143:0/1828722100 shutdown_connections 2023-11-26T07:18:43.272 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:43.265+0000 7f2cdd12a640 1 --2- 172.21.15.143:0/1828722100 >> v2:172.21.15.188:3300/0 conn(0x7f2cd80cd1b0 0x7f2cd80cd5c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:18:43.272 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:43.265+0000 7f2cdd12a640 1 --2- 172.21.15.143:0/1828722100 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f2cd80cf280 0x7f2cd80cf710 unknown :-1 s=CLOSED pgs=355 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:18:43.272 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:43.265+0000 7f2cdd12a640 1 -- 172.21.15.143:0/1828722100 >> 172.21.15.143:0/1828722100 conn(0x7f2cd806a890 msgr2=0x7f2cd806aca0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:18:43.272 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:43.265+0000 7f2cdd12a640 1 -- 172.21.15.143:0/1828722100 shutdown_connections 2023-11-26T07:18:43.272 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:43.265+0000 7f2cdd12a640 1 -- 172.21.15.143:0/1828722100 wait complete. 2023-11-26T07:18:43.273 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:43.269+0000 7f2cdd12a640 1 Processor -- start 2023-11-26T07:18:43.273 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:43.269+0000 7f2cdd12a640 1 -- start start 2023-11-26T07:18:43.275 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:43.269+0000 7f2cdd12a640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f2cd80cd1b0 0x7f2cd8158130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:18:43.276 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:43.269+0000 7f2cdd12a640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f2cd80cf280 0x7f2cd8142420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:18:43.276 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:43.269+0000 7f2cdd12a640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f2cd8143ac0 0x7f2cd8142960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:18:43.276 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:43.269+0000 7f2cdd12a640 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2cd813fed0 con 0x7f2cd80cd1b0 2023-11-26T07:18:43.276 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:43.269+0000 7f2cdd12a640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2cd815c490 con 0x7f2cd80cf280 2023-11-26T07:18:43.276 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:43.269+0000 7f2cdd12a640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2cd8158e20 con 0x7f2cd8143ac0 2023-11-26T07:18:43.276 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:43.269+0000 7f2cd6d9d640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f2cd80cd1b0 0x7f2cd8158130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:18:43.276 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:43.269+0000 7f2cd6d9d640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f2cd80cd1b0 0x7f2cd8158130 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:39204/0 (socket says 172.21.15.143:39204) 2023-11-26T07:18:43.276 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:43.269+0000 7f2cd6d9d640 1 -- 172.21.15.143:0/4288961353 learned_addr learned my addr 172.21.15.143:0/4288961353 (peer_addr_for_me v2:172.21.15.143:0/0) 2023-11-26T07:18:43.276 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:43.269+0000 7f2cd759e640 1 --2- 172.21.15.143:0/4288961353 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f2cd8143ac0 0x7f2cd8142960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:18:43.276 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:43.269+0000 7f2cd659c640 1 --2- 172.21.15.143:0/4288961353 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f2cd80cf280 0x7f2cd8142420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:18:43.276 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:43.269+0000 7f2cd6d9d640 1 -- 172.21.15.143:0/4288961353 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f2cd8143ac0 msgr2=0x7f2cd8142960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:18:43.277 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:43.269+0000 7f2cd6d9d640 1 --2- 172.21.15.143:0/4288961353 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f2cd8143ac0 0x7f2cd8142960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:18:43.277 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:43.269+0000 7f2cd6d9d640 1 -- 172.21.15.143:0/4288961353 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f2cd80cf280 msgr2=0x7f2cd8142420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:18:43.277 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:43.269+0000 7f2cd6d9d640 1 --2- 172.21.15.143:0/4288961353 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f2cd80cf280 0x7f2cd8142420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:18:43.277 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:43.269+0000 7f2cd6d9d640 1 -- 172.21.15.143:0/4288961353 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2ccc0094d0 con 0x7f2cd80cd1b0 2023-11-26T07:18:43.277 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:43.269+0000 7f2cd759e640 1 --2- 172.21.15.143:0/4288961353 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f2cd8143ac0 0x7f2cd8142960 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-26T07:18:43.277 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:43.269+0000 7f2cd659c640 1 --2- 172.21.15.143:0/4288961353 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f2cd80cf280 0x7f2cd8142420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-26T07:18:43.277 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:43.269+0000 7f2cd6d9d640 1 --2- 172.21.15.143:0/4288961353 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f2cd80cd1b0 0x7f2cd8158130 secure :-1 s=READY pgs=356 cs=0 l=1 rev1=1 crypto rx=0x7f2cc800c450 tx=0x7f2cc800c910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:18:43.277 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:43.269+0000 7f2cbf7fe640 1 -- 172.21.15.143:0/4288961353 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2cc8012830 con 0x7f2cd80cd1b0 2023-11-26T07:18:43.277 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:43.269+0000 7f2cbf7fe640 1 -- 172.21.15.143:0/4288961353 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2cc80129b0 con 0x7f2cd80cd1b0 2023-11-26T07:18:43.277 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:43.269+0000 7f2cbf7fe640 1 -- 172.21.15.143:0/4288961353 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2cc8012d60 con 0x7f2cd80cd1b0 2023-11-26T07:18:43.277 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:43.269+0000 7f2cdd12a640 1 -- 172.21.15.143:0/4288961353 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2cd813f910 con 0x7f2cd80cd1b0 2023-11-26T07:18:43.278 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:43.269+0000 7f2cdd12a640 1 -- 172.21.15.143:0/4288961353 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2cd8158ab0 con 0x7f2cd80cd1b0 2023-11-26T07:18:43.278 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:43.269+0000 7f2cdd12a640 1 -- 172.21.15.143:0/4288961353 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2cd8158910 con 0x7f2cd80cd1b0 2023-11-26T07:18:43.278 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:43.273+0000 7f2cbf7fe640 1 -- 172.21.15.143:0/4288961353 <== mon.0 v2:172.21.15.143:3300/0 4 ==== mgrmap(e 5) v1 ==== 85852+0+0 (secure 0 0 0) 0x7f2cc801c460 con 0x7f2cd80cd1b0 2023-11-26T07:18:43.279 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:43.273+0000 7f2cbf7fe640 1 --2- 172.21.15.143:0/4288961353 >> [v2:172.21.15.143:6832/2150557544,v1:172.21.15.143:6833/2150557544] conn(0x7f2cc4066470 0x7f2cc4068930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:18:43.279 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:43.273+0000 7f2cbf7fe640 1 -- 172.21.15.143:0/4288961353 <== mon.0 v2:172.21.15.143:3300/0 5 ==== osd_map(23..23 src has 1..23) v4 ==== 5911+0+0 (secure 0 0 0) 0x7f2cc8083e40 con 0x7f2cd80cd1b0 2023-11-26T07:18:43.279 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:43.273+0000 7f2cd659c640 1 --2- 172.21.15.143:0/4288961353 >> [v2:172.21.15.143:6832/2150557544,v1:172.21.15.143:6833/2150557544] conn(0x7f2cc4066470 0x7f2cc4068930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:18:43.280 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:43.273+0000 7f2cd659c640 1 --2- 172.21.15.143:0/4288961353 >> [v2:172.21.15.143:6832/2150557544,v1:172.21.15.143:6833/2150557544] conn(0x7f2cc4066470 0x7f2cc4068930 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7f2ccc0ae000 tx=0x7f2ccc008040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:18:43.283 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:43.277+0000 7f2cbf7fe640 1 -- 172.21.15.143:0/4288961353 <== mon.0 v2:172.21.15.143:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172649 (secure 0 0 0) 0x7f2cd8158910 con 0x7f2cd80cd1b0 2023-11-26T07:18:43.434 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:43.429+0000 7f2cdd12a640 1 -- 172.21.15.143:0/4288961353 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7f2cd8158910 con 0x7f2cd80cd1b0 2023-11-26T07:18:44.434 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:44.429+0000 7f2cbf7fe640 1 -- 172.21.15.143:0/4288961353 <== mon.0 v2:172.21.15.143:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_metadata' removed v24) v1 ==== 181+0+0 (secure 0 0 0) 0x7f2cd8158910 con 0x7f2cd80cd1b0 2023-11-26T07:18:44.434 INFO:teuthology.orchestra.run.smithi143.stderr:pool 'cephfs_metadata' removed 2023-11-26T07:18:44.438 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:44.429+0000 7f2cdd12a640 1 -- 172.21.15.143:0/4288961353 >> [v2:172.21.15.143:6832/2150557544,v1:172.21.15.143:6833/2150557544] conn(0x7f2cc4066470 msgr2=0x7f2cc4068930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:18:44.438 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:44.433+0000 7f2cdd12a640 1 --2- 172.21.15.143:0/4288961353 >> [v2:172.21.15.143:6832/2150557544,v1:172.21.15.143:6833/2150557544] conn(0x7f2cc4066470 0x7f2cc4068930 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7f2ccc0ae000 tx=0x7f2ccc008040 comp rx=0 tx=0).stop 2023-11-26T07:18:44.438 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:44.433+0000 7f2cdd12a640 1 -- 172.21.15.143:0/4288961353 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f2cd80cd1b0 msgr2=0x7f2cd8158130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:18:44.438 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:44.433+0000 7f2cdd12a640 1 --2- 172.21.15.143:0/4288961353 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f2cd80cd1b0 0x7f2cd8158130 secure :-1 s=READY pgs=356 cs=0 l=1 rev1=1 crypto rx=0x7f2cc800c450 tx=0x7f2cc800c910 comp rx=0 tx=0).stop 2023-11-26T07:18:44.438 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:44.433+0000 7f2cdd12a640 1 -- 172.21.15.143:0/4288961353 shutdown_connections 2023-11-26T07:18:44.438 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:44.433+0000 7f2cdd12a640 1 --2- 172.21.15.143:0/4288961353 >> [v2:172.21.15.143:6832/2150557544,v1:172.21.15.143:6833/2150557544] conn(0x7f2cc4066470 0x7f2cc4068930 unknown :-1 s=CLOSED pgs=196 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:18:44.438 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:44.433+0000 7f2cdd12a640 1 --2- 172.21.15.143:0/4288961353 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f2cd8143ac0 0x7f2cd8142960 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:18:44.439 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:44.433+0000 7f2cdd12a640 1 --2- 172.21.15.143:0/4288961353 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f2cd80cf280 0x7f2cd8142420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:18:44.439 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:44.433+0000 7f2cdd12a640 1 --2- 172.21.15.143:0/4288961353 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f2cd80cd1b0 0x7f2cd8158130 secure :-1 s=CLOSED pgs=356 cs=0 l=1 rev1=1 crypto rx=0x7f2cc800c450 tx=0x7f2cc800c910 comp rx=0 tx=0).stop 2023-11-26T07:18:44.439 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:44.433+0000 7f2cdd12a640 1 -- 172.21.15.143:0/4288961353 >> 172.21.15.143:0/4288961353 conn(0x7f2cd806a890 msgr2=0x7f2cd80cb0e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:18:44.439 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:44.433+0000 7f2cdd12a640 1 -- 172.21.15.143:0/4288961353 shutdown_connections 2023-11-26T07:18:44.439 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:44.433+0000 7f2cdd12a640 1 -- 172.21.15.143:0/4288961353 wait complete. 2023-11-26T07:18:44.463 DEBUG:teuthology.orchestra.run.smithi143:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_data cephfs_data --yes-i-really-really-mean-it 2023-11-26T07:18:44.608 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:44.601+0000 7f31171e7640 1 Processor -- start 2023-11-26T07:18:44.608 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:44.601+0000 7f31171e7640 1 -- start start 2023-11-26T07:18:44.610 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:44.605+0000 7f31171e7640 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7f31100cd610 0x7f3110071970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:18:44.610 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:44.605+0000 7f31171e7640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f3110070ba0 0x7f3110070ff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:18:44.610 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:44.605+0000 7f31171e7640 1 -- --> v1:172.21.15.188:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3110071530 con 0x7f31101614b0 2023-11-26T07:18:44.610 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:44.605+0000 7f31171e7640 1 -- --> v2:172.21.15.143:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3110071eb0 con 0x7f31100cd610 2023-11-26T07:18:44.610 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:44.605+0000 7f31171e7640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f311015d970 con 0x7f3110070ba0 2023-11-26T07:18:44.610 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:44.605+0000 7f310ffff640 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7f31100cd610 0x7f3110071970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:18:44.610 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:44.605+0000 7f310ffff640 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7f31100cd610 0x7f3110071970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:47346/0 (socket says 172.21.15.143:47346) 2023-11-26T07:18:44.610 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:44.605+0000 7f310ffff640 1 -- 172.21.15.143:0/1685297292 learned_addr learned my addr 172.21.15.143:0/1685297292 (peer_addr_for_me v2:172.21.15.143:0/0) 2023-11-26T07:18:44.611 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:44.605+0000 7f3115784640 1 --2- 172.21.15.143:0/1685297292 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f3110070ba0 0x7f3110070ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:18:44.611 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:44.605+0000 7f310ffff640 1 -- 172.21.15.143:0/1685297292 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f3110070ba0 msgr2=0x7f3110070ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-26T07:18:44.611 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:44.605+0000 7f310ffff640 1 --2- 172.21.15.143:0/1685297292 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f3110070ba0 0x7f3110070ff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:18:44.611 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:44.605+0000 7f310ffff640 1 -- 172.21.15.143:0/1685297292 >> v1:172.21.15.188:6789/0 conn(0x7f31101614b0 legacy=0x7f3110161890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:18:44.611 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:44.605+0000 7f310ffff640 1 -- 172.21.15.143:0/1685297292 --> v2:172.21.15.143:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f311015dc30 con 0x7f31100cd610 2023-11-26T07:18:44.613 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:44.609+0000 7f310ffff640 1 --2- 172.21.15.143:0/1685297292 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f31100cd610 0x7f3110071970 secure :-1 s=READY pgs=357 cs=0 l=1 rev1=1 crypto rx=0x7f30fc009870 tx=0x7f30fc01c510 comp rx=0 tx=0).ready entity=mon.0 client_cookie=200bc5fac9b70e14 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:18:44.613 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:44.609+0000 7f310f7fe640 1 -- 172.21.15.143:0/1685297292 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f30fc0aa670 con 0x7f31100cd610 2023-11-26T07:18:44.614 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:44.609+0000 7f310f7fe640 1 -- 172.21.15.143:0/1685297292 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f30fc0aa7d0 con 0x7f31100cd610 2023-11-26T07:18:44.614 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:44.609+0000 7f310f7fe640 1 -- 172.21.15.143:0/1685297292 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f30fc0aab60 con 0x7f31100cd610 2023-11-26T07:18:44.614 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:44.609+0000 7f31171e7640 1 -- 172.21.15.143:0/1685297292 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f31100cd610 msgr2=0x7f3110071970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:18:44.615 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:44.609+0000 7f31171e7640 1 --2- 172.21.15.143:0/1685297292 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f31100cd610 0x7f3110071970 secure :-1 s=READY pgs=357 cs=0 l=1 rev1=1 crypto rx=0x7f30fc009870 tx=0x7f30fc01c510 comp rx=0 tx=0).stop 2023-11-26T07:18:44.615 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:44.609+0000 7f31171e7640 1 -- 172.21.15.143:0/1685297292 shutdown_connections 2023-11-26T07:18:44.615 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:44.609+0000 7f31171e7640 1 --2- 172.21.15.143:0/1685297292 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f31100cd610 0x7f3110071970 unknown :-1 s=CLOSED pgs=357 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:18:44.615 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:44.609+0000 7f31171e7640 1 --2- 172.21.15.143:0/1685297292 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f3110070ba0 0x7f3110070ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:18:44.615 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:44.609+0000 7f31171e7640 1 -- 172.21.15.143:0/1685297292 >> 172.21.15.143:0/1685297292 conn(0x7f31100cf710 msgr2=0x7f31100cfb20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:18:44.615 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:44.609+0000 7f31171e7640 1 -- 172.21.15.143:0/1685297292 shutdown_connections 2023-11-26T07:18:44.615 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:44.609+0000 7f31171e7640 1 -- 172.21.15.143:0/1685297292 wait complete. 2023-11-26T07:18:44.616 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:44.609+0000 7f31171e7640 1 Processor -- start 2023-11-26T07:18:44.616 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:44.609+0000 7f31171e7640 1 -- start start 2023-11-26T07:18:44.618 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:44.613+0000 7f31171e7640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f3110070ba0 0x7f3110159dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:18:44.619 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:44.613+0000 7f31171e7640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f31100cd610 0x7f3110157380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:18:44.619 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:44.613+0000 7f31171e7640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f31101614b0 0x7f31100b37e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:18:44.619 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:44.613+0000 7f31171e7640 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f31101578c0 con 0x7f31101614b0 2023-11-26T07:18:44.619 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:44.613+0000 7f31171e7640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3110159940 con 0x7f31100cd610 2023-11-26T07:18:44.619 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:44.613+0000 7f31171e7640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f31100b3d20 con 0x7f3110070ba0 2023-11-26T07:18:44.619 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:44.613+0000 7f3115784640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f31101614b0 0x7f31100b37e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:18:44.619 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:44.613+0000 7f3115784640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f31101614b0 0x7f31100b37e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:47354/0 (socket says 172.21.15.143:47354) 2023-11-26T07:18:44.619 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:44.613+0000 7f3115784640 1 -- 172.21.15.143:0/3779622275 learned_addr learned my addr 172.21.15.143:0/3779622275 (peer_addr_for_me v2:172.21.15.143:0/0) 2023-11-26T07:18:44.619 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:44.613+0000 7f3114f83640 1 --2- 172.21.15.143:0/3779622275 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f3110070ba0 0x7f3110159dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:18:44.619 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:44.613+0000 7f310ffff640 1 --2- 172.21.15.143:0/3779622275 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f31100cd610 0x7f3110157380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:18:44.619 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:44.613+0000 7f3115784640 1 -- 172.21.15.143:0/3779622275 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f3110070ba0 msgr2=0x7f3110159dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:18:44.620 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:44.613+0000 7f3115784640 1 --2- 172.21.15.143:0/3779622275 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f3110070ba0 0x7f3110159dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:18:44.620 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:44.613+0000 7f3115784640 1 -- 172.21.15.143:0/3779622275 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f31100cd610 msgr2=0x7f3110157380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:18:44.620 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:44.613+0000 7f3115784640 1 --2- 172.21.15.143:0/3779622275 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f31100cd610 0x7f3110157380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:18:44.620 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:44.613+0000 7f3115784640 1 -- 172.21.15.143:0/3779622275 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f30fc0094d0 con 0x7f31101614b0 2023-11-26T07:18:44.620 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:44.613+0000 7f310ffff640 1 --2- 172.21.15.143:0/3779622275 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f31100cd610 0x7f3110157380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-26T07:18:44.620 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:44.613+0000 7f3115784640 1 --2- 172.21.15.143:0/3779622275 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f31101614b0 0x7f31100b37e0 secure :-1 s=READY pgs=358 cs=0 l=1 rev1=1 crypto rx=0x7f3108007e60 tx=0x7f310800e4b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:18:44.620 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:44.613+0000 7f310d7fa640 1 -- 172.21.15.143:0/3779622275 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3108014420 con 0x7f31101614b0 2023-11-26T07:18:44.620 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:44.613+0000 7f310d7fa640 1 -- 172.21.15.143:0/3779622275 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f31080145a0 con 0x7f31101614b0 2023-11-26T07:18:44.620 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:44.613+0000 7f31171e7640 1 -- 172.21.15.143:0/3779622275 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3110158d30 con 0x7f31101614b0 2023-11-26T07:18:44.620 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:44.613+0000 7f310d7fa640 1 -- 172.21.15.143:0/3779622275 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3108014950 con 0x7f31101614b0 2023-11-26T07:18:44.620 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:44.613+0000 7f31171e7640 1 -- 172.21.15.143:0/3779622275 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f31101592f0 con 0x7f31101614b0 2023-11-26T07:18:44.621 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:44.613+0000 7f31171e7640 1 -- 172.21.15.143:0/3779622275 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3110159ab0 con 0x7f31101614b0 2023-11-26T07:18:44.621 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:44.613+0000 7f310d7fa640 1 -- 172.21.15.143:0/3779622275 <== mon.0 v2:172.21.15.143:3300/0 4 ==== mgrmap(e 5) v1 ==== 85852+0+0 (secure 0 0 0) 0x7f3108014ab0 con 0x7f31101614b0 2023-11-26T07:18:44.621 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:44.617+0000 7f310d7fa640 1 --2- 172.21.15.143:0/3779622275 >> [v2:172.21.15.143:6832/2150557544,v1:172.21.15.143:6833/2150557544] conn(0x7f30f40664a0 0x7f30f4068960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:18:44.621 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:44.617+0000 7f310d7fa640 1 -- 172.21.15.143:0/3779622275 <== mon.0 v2:172.21.15.143:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 5473+0+0 (secure 0 0 0) 0x7f31080845c0 con 0x7f31101614b0 2023-11-26T07:18:44.622 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:44.617+0000 7f3114f83640 1 --2- 172.21.15.143:0/3779622275 >> [v2:172.21.15.143:6832/2150557544,v1:172.21.15.143:6833/2150557544] conn(0x7f30f40664a0 0x7f30f4068960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:18:44.622 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:44.617+0000 7f3114f83640 1 --2- 172.21.15.143:0/3779622275 >> [v2:172.21.15.143:6832/2150557544,v1:172.21.15.143:6833/2150557544] conn(0x7f30f40664a0 0x7f30f4068960 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7f3110167100 tx=0x7f3104009040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:18:44.625 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:44.621+0000 7f310d7fa640 1 -- 172.21.15.143:0/3779622275 <== mon.0 v2:172.21.15.143:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172649 (secure 0 0 0) 0x7f3108051b10 con 0x7f31101614b0 2023-11-26T07:18:44.779 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:44.773+0000 7f31171e7640 1 -- 172.21.15.143:0/3779622275 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7f3110067680 con 0x7f31101614b0 2023-11-26T07:18:45.436 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:45.433+0000 7f310d7fa640 1 -- 172.21.15.143:0/3779622275 <== mon.0 v2:172.21.15.143:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_data' removed v25) v1 ==== 169+0+0 (secure 0 0 0) 0x7f3108055130 con 0x7f31101614b0 2023-11-26T07:18:45.437 INFO:teuthology.orchestra.run.smithi143.stderr:pool 'cephfs_data' removed 2023-11-26T07:18:45.439 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:45.433+0000 7f31171e7640 1 -- 172.21.15.143:0/3779622275 >> [v2:172.21.15.143:6832/2150557544,v1:172.21.15.143:6833/2150557544] conn(0x7f30f40664a0 msgr2=0x7f30f4068960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:18:45.439 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:45.433+0000 7f31171e7640 1 --2- 172.21.15.143:0/3779622275 >> [v2:172.21.15.143:6832/2150557544,v1:172.21.15.143:6833/2150557544] conn(0x7f30f40664a0 0x7f30f4068960 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7f3110167100 tx=0x7f3104009040 comp rx=0 tx=0).stop 2023-11-26T07:18:45.439 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:45.433+0000 7f31171e7640 1 -- 172.21.15.143:0/3779622275 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f31101614b0 msgr2=0x7f31100b37e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:18:45.440 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:45.433+0000 7f31171e7640 1 --2- 172.21.15.143:0/3779622275 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f31101614b0 0x7f31100b37e0 secure :-1 s=READY pgs=358 cs=0 l=1 rev1=1 crypto rx=0x7f3108007e60 tx=0x7f310800e4b0 comp rx=0 tx=0).stop 2023-11-26T07:18:45.440 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:45.433+0000 7f31171e7640 1 -- 172.21.15.143:0/3779622275 shutdown_connections 2023-11-26T07:18:45.440 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:45.433+0000 7f31171e7640 1 --2- 172.21.15.143:0/3779622275 >> [v2:172.21.15.143:6832/2150557544,v1:172.21.15.143:6833/2150557544] conn(0x7f30f40664a0 0x7f30f4068960 unknown :-1 s=CLOSED pgs=197 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:18:45.440 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:45.433+0000 7f31171e7640 1 --2- 172.21.15.143:0/3779622275 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f31101614b0 0x7f31100b37e0 unknown :-1 s=CLOSED pgs=358 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:18:45.440 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:45.433+0000 7f31171e7640 1 --2- 172.21.15.143:0/3779622275 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f31100cd610 0x7f3110157380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:18:45.441 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:45.433+0000 7f31171e7640 1 --2- 172.21.15.143:0/3779622275 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f3110070ba0 0x7f3110159dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:18:45.441 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:45.433+0000 7f31171e7640 1 -- 172.21.15.143:0/3779622275 >> 172.21.15.143:0/3779622275 conn(0x7f31100cf710 msgr2=0x7f311015fb80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:18:45.441 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:45.433+0000 7f31171e7640 1 -- 172.21.15.143:0/3779622275 shutdown_connections 2023-11-26T07:18:45.441 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:45.437+0000 7f31171e7640 1 -- 172.21.15.143:0/3779622275 wait complete. 2023-11-26T07:18:45.475 INFO:tasks.ceph.mds.a:Restarting daemon 2023-11-26T07:18:45.475 INFO:tasks.ceph.mds.a:Stopping old one... 2023-11-26T07:18:45.476 DEBUG:tasks.ceph.mds.a:waiting for process to exit 2023-11-26T07:18:45.476 INFO:teuthology.orchestra.run:waiting for 300 2023-11-26T07:18:45.476 INFO:tasks.ceph.mds.c:Restarting daemon 2023-11-26T07:18:45.476 INFO:tasks.ceph.mds.c:Stopping old one... 2023-11-26T07:18:45.476 DEBUG:tasks.ceph.mds.c:waiting for process to exit 2023-11-26T07:18:45.476 INFO:teuthology.orchestra.run:waiting for 300 2023-11-26T07:18:45.477 INFO:tasks.ceph.mds.b:Restarting daemon 2023-11-26T07:18:45.477 INFO:tasks.ceph.mds.b:Stopping old one... 2023-11-26T07:18:45.477 DEBUG:tasks.ceph.mds.b:waiting for process to exit 2023-11-26T07:18:45.477 INFO:teuthology.orchestra.run:waiting for 300 2023-11-26T07:18:45.477 INFO:tasks.ceph.mds.d:Restarting daemon 2023-11-26T07:18:45.477 INFO:tasks.ceph.mds.d:Stopping old one... 2023-11-26T07:18:45.477 DEBUG:tasks.ceph.mds.d:waiting for process to exit 2023-11-26T07:18:45.478 INFO:teuthology.orchestra.run:waiting for 300 2023-11-26T07:18:45.492 INFO:tasks.ceph.mds.a:Stopped 2023-11-26T07:18:45.492 DEBUG:teuthology.orchestra.run.smithi143:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i a 2023-11-26T07:18:45.494 INFO:tasks.ceph.mds.c:Stopped 2023-11-26T07:18:45.494 DEBUG:teuthology.orchestra.run.smithi143:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i c 2023-11-26T07:18:45.495 INFO:tasks.ceph.mds.d:Stopped 2023-11-26T07:18:45.495 DEBUG:teuthology.orchestra.run.smithi188:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i d 2023-11-26T07:18:45.496 INFO:tasks.ceph.mds.b:Stopped 2023-11-26T07:18:45.496 DEBUG:teuthology.orchestra.run.smithi188:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i b 2023-11-26T07:18:45.539 INFO:tasks.ceph.mds.a:Started 2023-11-26T07:18:45.540 DEBUG:teuthology.parallel:result is None 2023-11-26T07:18:45.541 INFO:tasks.ceph.mds.c:Started 2023-11-26T07:18:45.542 DEBUG:teuthology.parallel:result is None 2023-11-26T07:18:45.545 INFO:tasks.ceph.mds.d:Started 2023-11-26T07:18:45.545 DEBUG:teuthology.parallel:result is None 2023-11-26T07:18:45.547 INFO:tasks.ceph.mds.b:Started 2023-11-26T07:18:45.547 DEBUG:teuthology.parallel:result is None 2023-11-26T07:18:45.547 DEBUG:teuthology.orchestra.run.smithi143:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist clear 2023-11-26T07:18:45.628 INFO:tasks.ceph.mds.a.smithi143.stdout:starting mds.a at 2023-11-26T07:18:45.630 INFO:tasks.ceph.mds.c.smithi143.stdout:starting mds.c at 2023-11-26T07:18:45.642 INFO:tasks.ceph.mds.d.smithi188.stdout:starting mds.d at 2023-11-26T07:18:45.643 INFO:tasks.ceph.mds.b.smithi188.stdout:starting mds.b at 2023-11-26T07:18:45.696 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:45.689+0000 7fc9b4d65640 1 Processor -- start 2023-11-26T07:18:45.696 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:45.689+0000 7fc9b4d65640 1 -- start start 2023-11-26T07:18:45.698 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:45.693+0000 7fc9b4d65640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fc9b00cf990 0x7fc9b015efa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:18:45.698 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:45.693+0000 7fc9b4d65640 1 -- --> v1:172.21.15.143:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc9b006c520 con 0x7fc9b013e290 2023-11-26T07:18:45.698 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:45.693+0000 7fc9b4d65640 1 -- --> v1:172.21.15.188:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc9b006c800 con 0x7fc9b015f4e0 2023-11-26T07:18:45.698 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:45.693+0000 7fc9b4d65640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc9b006cae0 con 0x7fc9b00cf990 2023-11-26T07:18:45.698 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:45.693+0000 7fc9ae59c640 1 --1- >> v1:172.21.15.143:6789/0 conn(0x7fc9b013e290 0x7fc9b00cf280 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.143:6789/0 says I am v1:172.21.15.143:55122/0 (socket says 172.21.15.143:55122) 2023-11-26T07:18:45.698 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:45.693+0000 7fc9ae59c640 1 -- 172.21.15.143:0/1655214199 learned_addr learned my addr 172.21.15.143:0/1655214199 (peer_addr_for_me v1:172.21.15.143:0/0) 2023-11-26T07:18:45.698 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:45.693+0000 7fc9a5d9b640 1 --2- 172.21.15.143:0/1655214199 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fc9b00cf990 0x7fc9b015efa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:18:45.698 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:45.693+0000 7fc9add9b640 1 -- 172.21.15.143:0/1655214199 <== mon.0 v1:172.21.15.143:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2938874880 0 0) 0x7fc9b006c520 con 0x7fc9b013e290 2023-11-26T07:18:45.698 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:45.693+0000 7fc9add9b640 1 -- 172.21.15.143:0/1655214199 --> v1:172.21.15.143:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fc99c003530 con 0x7fc9b013e290 2023-11-26T07:18:45.698 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:45.693+0000 7fc9add9b640 1 -- 172.21.15.143:0/1655214199 <== mon.1 v1:172.21.15.188:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1892887685 0 0) 0x7fc9b006c800 con 0x7fc9b015f4e0 2023-11-26T07:18:45.698 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:45.693+0000 7fc9add9b640 1 -- 172.21.15.143:0/1655214199 --> v1:172.21.15.188:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fc9b006c520 con 0x7fc9b015f4e0 2023-11-26T07:18:45.698 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:45.693+0000 7fc9add9b640 1 -- 172.21.15.143:0/1655214199 <== mon.0 v1:172.21.15.143:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 714091437 0 0) 0x7fc99c003530 con 0x7fc9b013e290 2023-11-26T07:18:45.699 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:45.693+0000 7fc9a5d9b640 1 -- 172.21.15.143:0/1655214199 >> v1:172.21.15.188:6789/0 conn(0x7fc9b015f4e0 legacy=0x7fc9b00cd7b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:18:45.699 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:45.693+0000 7fc9a5d9b640 1 -- 172.21.15.143:0/1655214199 >> v1:172.21.15.143:6789/0 conn(0x7fc9b013e290 legacy=0x7fc9b00cf280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:18:45.699 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:45.693+0000 7fc9a5d9b640 1 -- 172.21.15.143:0/1655214199 --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc9b006cdc0 con 0x7fc9b00cf990 2023-11-26T07:18:45.701 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:45.697+0000 7fc9a5d9b640 1 --2- 172.21.15.143:0/1655214199 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fc9b00cf990 0x7fc9b015efa0 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7fc998009dc0 tx=0x7fc99801c860 comp rx=0 tx=0).ready entity=mon.2 client_cookie=cbac6637982753f3 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:18:45.701 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:45.697+0000 7fc9add9b640 1 -- 172.21.15.143:0/1655214199 <== mon.2 v2:172.21.15.188:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc9a0001680 con 0x7fc9b00cf990 2023-11-26T07:18:45.701 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:45.697+0000 7fc9add9b640 1 -- 172.21.15.143:0/1655214199 <== mon.2 v2:172.21.15.188:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc9980a56c0 con 0x7fc9b00cf990 2023-11-26T07:18:45.701 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:45.697+0000 7fc9add9b640 1 -- 172.21.15.143:0/1655214199 <== mon.2 v2:172.21.15.188:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc9980a5a50 con 0x7fc9b00cf990 2023-11-26T07:18:45.702 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:45.697+0000 7fc9b4d65640 1 -- 172.21.15.143:0/1655214199 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fc9b00cf990 msgr2=0x7fc9b015efa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:18:45.702 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:45.697+0000 7fc9b4d65640 1 --2- 172.21.15.143:0/1655214199 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fc9b00cf990 0x7fc9b015efa0 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7fc998009dc0 tx=0x7fc99801c860 comp rx=0 tx=0).stop 2023-11-26T07:18:45.702 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:45.697+0000 7fc9b4d65640 1 -- 172.21.15.143:0/1655214199 shutdown_connections 2023-11-26T07:18:45.702 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:45.697+0000 7fc9b4d65640 1 --2- 172.21.15.143:0/1655214199 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fc9b00cf990 0x7fc9b015efa0 unknown :-1 s=CLOSED pgs=72 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:18:45.702 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:45.697+0000 7fc9b4d65640 1 -- 172.21.15.143:0/1655214199 >> 172.21.15.143:0/1655214199 conn(0x7fc9b006a890 msgr2=0x7fc9b006aca0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:18:45.703 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:45.697+0000 7fc9b4d65640 1 -- 172.21.15.143:0/1655214199 shutdown_connections 2023-11-26T07:18:45.703 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:45.697+0000 7fc9b4d65640 1 -- 172.21.15.143:0/1655214199 wait complete. 2023-11-26T07:18:45.704 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:45.697+0000 7fc9b4d65640 1 Processor -- start 2023-11-26T07:18:45.704 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:45.697+0000 7fc9b4d65640 1 -- start start 2023-11-26T07:18:45.706 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:45.701+0000 7fc9b4d65640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fc9b00cf990 0x7fc9b0136b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:18:45.707 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:45.701+0000 7fc9b4d65640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fc9b013e290 0x7fc9b01f5f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:18:45.707 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:45.701+0000 7fc9b4d65640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fc9b015f4e0 0x7fc9b00b3750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:18:45.707 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:45.701+0000 7fc9b4d65640 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc9b00b0c10 con 0x7fc9b00cf990 2023-11-26T07:18:45.707 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:45.701+0000 7fc9b4d65640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc9b015ba80 con 0x7fc9b013e290 2023-11-26T07:18:45.707 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:45.701+0000 7fc9b4d65640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc9b015bd60 con 0x7fc9b015f4e0 2023-11-26T07:18:45.707 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:45.701+0000 7fc9ae59c640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fc9b00cf990 0x7fc9b0136b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:18:45.707 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:45.701+0000 7fc9ae59c640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fc9b00cf990 0x7fc9b0136b80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:47394/0 (socket says 172.21.15.143:47394) 2023-11-26T07:18:45.707 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:45.701+0000 7fc9ae59c640 1 -- 172.21.15.143:0/3395462902 learned_addr learned my addr 172.21.15.143:0/3395462902 (peer_addr_for_me v2:172.21.15.143:0/0) 2023-11-26T07:18:45.707 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:45.701+0000 7fc9aed9d640 1 --2- 172.21.15.143:0/3395462902 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fc9b015f4e0 0x7fc9b00b3750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:18:45.707 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:45.701+0000 7fc9a5d9b640 1 --2- 172.21.15.143:0/3395462902 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fc9b013e290 0x7fc9b01f5f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:18:45.707 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:45.701+0000 7fc9ae59c640 1 -- 172.21.15.143:0/3395462902 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fc9b015f4e0 msgr2=0x7fc9b00b3750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:18:45.708 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:45.701+0000 7fc9ae59c640 1 --2- 172.21.15.143:0/3395462902 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fc9b015f4e0 0x7fc9b00b3750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:18:45.708 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:45.701+0000 7fc9ae59c640 1 -- 172.21.15.143:0/3395462902 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fc9b013e290 msgr2=0x7fc9b01f5f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:18:45.708 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:45.701+0000 7fc9ae59c640 1 --2- 172.21.15.143:0/3395462902 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fc9b013e290 0x7fc9b01f5f30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:18:45.708 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:45.701+0000 7fc9ae59c640 1 -- 172.21.15.143:0/3395462902 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc998009a70 con 0x7fc9b00cf990 2023-11-26T07:18:45.708 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:45.701+0000 7fc9aed9d640 1 --2- 172.21.15.143:0/3395462902 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fc9b015f4e0 0x7fc9b00b3750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-26T07:18:45.708 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:45.701+0000 7fc9ae59c640 1 --2- 172.21.15.143:0/3395462902 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fc9b00cf990 0x7fc9b0136b80 secure :-1 s=READY pgs=366 cs=0 l=1 rev1=1 crypto rx=0x7fc9a000c780 tx=0x7fc9a000cc40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:18:45.708 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:45.701+0000 7fc9a77fe640 1 -- 172.21.15.143:0/3395462902 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc9a0005cf0 con 0x7fc9b00cf990 2023-11-26T07:18:45.708 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:45.701+0000 7fc9a77fe640 1 -- 172.21.15.143:0/3395462902 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc9a0005e70 con 0x7fc9b00cf990 2023-11-26T07:18:45.708 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:45.701+0000 7fc9a77fe640 1 -- 172.21.15.143:0/3395462902 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc9a0007290 con 0x7fc9b00cf990 2023-11-26T07:18:45.708 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:45.701+0000 7fc9b4d65640 1 -- 172.21.15.143:0/3395462902 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc9b015c040 con 0x7fc9b00cf990 2023-11-26T07:18:45.708 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:45.701+0000 7fc9b4d65640 1 -- 172.21.15.143:0/3395462902 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc9b01370c0 con 0x7fc9b00cf990 2023-11-26T07:18:45.709 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:45.701+0000 7fc9b4d65640 1 -- 172.21.15.143:0/3395462902 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc9b00710f0 con 0x7fc9b00cf990 2023-11-26T07:18:45.709 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:45.701+0000 7fc9a77fe640 1 -- 172.21.15.143:0/3395462902 <== mon.0 v2:172.21.15.143:3300/0 4 ==== mgrmap(e 5) v1 ==== 85852+0+0 (secure 0 0 0) 0x7fc9a00073f0 con 0x7fc9b00cf990 2023-11-26T07:18:45.709 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:45.705+0000 7fc9a77fe640 1 --2- 172.21.15.143:0/3395462902 >> [v2:172.21.15.143:6832/2150557544,v1:172.21.15.143:6833/2150557544] conn(0x7fc9900664a0 0x7fc990068960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:18:45.710 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:45.705+0000 7fc9a77fe640 1 -- 172.21.15.143:0/3395462902 <== mon.0 v2:172.21.15.143:3300/0 5 ==== osd_map(25..25 src has 1..25) v4 ==== 4943+0+0 (secure 0 0 0) 0x7fc9a0083290 con 0x7fc9b00cf990 2023-11-26T07:18:45.710 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:45.705+0000 7fc9a5d9b640 1 --2- 172.21.15.143:0/3395462902 >> [v2:172.21.15.143:6832/2150557544,v1:172.21.15.143:6833/2150557544] conn(0x7fc9900664a0 0x7fc990068960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:18:45.710 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:45.705+0000 7fc9a5d9b640 1 --2- 172.21.15.143:0/3395462902 >> [v2:172.21.15.143:6832/2150557544,v1:172.21.15.143:6833/2150557544] conn(0x7fc9900664a0 0x7fc990068960 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7fc9980af000 tx=0x7fc9980a7070 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:18:45.715 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:45.709+0000 7fc9a77fe640 1 -- 172.21.15.143:0/3395462902 <== mon.0 v2:172.21.15.143:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172649 (secure 0 0 0) 0x7fc9a0050ab0 con 0x7fc9b00cf990 2023-11-26T07:18:45.875 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:45.869+0000 7fc9b4d65640 1 -- 172.21.15.143:0/3395462902 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "osd blocklist clear"} v 0) v1 -- 0x7fc9b0067680 con 0x7fc9b00cf990 2023-11-26T07:18:46.463 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:46.457+0000 7fc9a77fe640 1 -- 172.21.15.143:0/3395462902 <== mon.0 v2:172.21.15.143:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist clear"}]=0 removed all blocklist entries v27) v1 ==== 97+0+0 (secure 0 0 0) 0x7fc9a00540d0 con 0x7fc9b00cf990 2023-11-26T07:18:46.463 INFO:teuthology.orchestra.run.smithi143.stderr: removed all blocklist entries 2023-11-26T07:18:46.465 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:46.461+0000 7fc9b4d65640 1 -- 172.21.15.143:0/3395462902 >> [v2:172.21.15.143:6832/2150557544,v1:172.21.15.143:6833/2150557544] conn(0x7fc9900664a0 msgr2=0x7fc990068960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:18:46.465 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:46.461+0000 7fc9b4d65640 1 --2- 172.21.15.143:0/3395462902 >> [v2:172.21.15.143:6832/2150557544,v1:172.21.15.143:6833/2150557544] conn(0x7fc9900664a0 0x7fc990068960 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7fc9980af000 tx=0x7fc9980a7070 comp rx=0 tx=0).stop 2023-11-26T07:18:46.466 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:46.461+0000 7fc9b4d65640 1 -- 172.21.15.143:0/3395462902 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fc9b00cf990 msgr2=0x7fc9b0136b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:18:46.466 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:46.461+0000 7fc9b4d65640 1 --2- 172.21.15.143:0/3395462902 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fc9b00cf990 0x7fc9b0136b80 secure :-1 s=READY pgs=366 cs=0 l=1 rev1=1 crypto rx=0x7fc9a000c780 tx=0x7fc9a000cc40 comp rx=0 tx=0).stop 2023-11-26T07:18:46.466 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:46.461+0000 7fc9b4d65640 1 -- 172.21.15.143:0/3395462902 shutdown_connections 2023-11-26T07:18:46.466 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:46.461+0000 7fc9b4d65640 1 --2- 172.21.15.143:0/3395462902 >> [v2:172.21.15.143:6832/2150557544,v1:172.21.15.143:6833/2150557544] conn(0x7fc9900664a0 0x7fc990068960 unknown :-1 s=CLOSED pgs=198 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:18:46.466 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:46.461+0000 7fc9b4d65640 1 --2- 172.21.15.143:0/3395462902 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fc9b015f4e0 0x7fc9b00b3750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:18:46.467 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:46.461+0000 7fc9b4d65640 1 --2- 172.21.15.143:0/3395462902 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fc9b013e290 0x7fc9b01f5f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:18:46.467 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:46.461+0000 7fc9b4d65640 1 --2- 172.21.15.143:0/3395462902 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fc9b00cf990 0x7fc9b0136b80 unknown :-1 s=CLOSED pgs=366 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:18:46.467 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:46.461+0000 7fc9b4d65640 1 -- 172.21.15.143:0/3395462902 >> 172.21.15.143:0/3395462902 conn(0x7fc9b006a890 msgr2=0x7fc9b006ac70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:18:46.467 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:46.461+0000 7fc9b4d65640 1 -- 172.21.15.143:0/3395462902 shutdown_connections 2023-11-26T07:18:46.467 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:46.461+0000 7fc9b4d65640 1 -- 172.21.15.143:0/3395462902 wait complete. 2023-11-26T07:18:46.488 DEBUG:teuthology.orchestra.run.smithi143:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth ls --format=json-pretty 2023-11-26T07:18:46.625 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:46.621+0000 7fe8ffe9c640 1 Processor -- start 2023-11-26T07:18:46.626 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:46.621+0000 7fe8ffe9c640 1 -- start start 2023-11-26T07:18:46.627 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:46.621+0000 7fe8ffe9c640 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7fe8f815d6e0 0x7fe8f80d0df0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:18:46.627 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:46.621+0000 7fe8ffe9c640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fe8f80d1bc0 0x7fe8f80b0750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:18:46.628 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:46.621+0000 7fe8ffe9c640 1 -- --> v1:172.21.15.143:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe8f80cf0e0 con 0x7fe8f80b2160 2023-11-26T07:18:46.628 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:46.621+0000 7fe8ffe9c640 1 -- --> v2:172.21.15.188:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe8f815dac0 con 0x7fe8f815d6e0 2023-11-26T07:18:46.628 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:46.621+0000 7fe8ffe9c640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe8f80ceb20 con 0x7fe8f80d1bc0 2023-11-26T07:18:46.628 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:46.621+0000 7fe8fd437640 1 --1- >> v1:172.21.15.143:6789/0 conn(0x7fe8f80b2160 0x7fe8f80d14b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.143:6789/0 says I am v1:172.21.15.143:55128/0 (socket says 172.21.15.143:55128) 2023-11-26T07:18:46.628 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:46.621+0000 7fe8fd437640 1 -- 172.21.15.143:0/1299888752 learned_addr learned my addr 172.21.15.143:0/1299888752 (peer_addr_for_me v1:172.21.15.143:0/0) 2023-11-26T07:18:46.628 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:46.621+0000 7fe8fe439640 1 --2- 172.21.15.143:0/1299888752 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fe8f80d1bc0 0x7fe8f80b0750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:18:46.628 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:46.621+0000 7fe8fdc38640 1 --2- 172.21.15.143:0/1299888752 >> v2:172.21.15.188:3300/0 conn(0x7fe8f815d6e0 0x7fe8f80d0df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:18:46.629 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:46.621+0000 7fe8fcc36640 1 -- 172.21.15.143:0/1299888752 <== mon.0 v1:172.21.15.143:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 536347934 0 0) 0x7fe8f80cf0e0 con 0x7fe8f80b2160 2023-11-26T07:18:46.629 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:46.621+0000 7fe8fcc36640 1 -- 172.21.15.143:0/1299888752 --> v1:172.21.15.143:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fe8e4003530 con 0x7fe8f80b2160 2023-11-26T07:18:46.629 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:46.621+0000 7fe8fe439640 1 -- 172.21.15.143:0/1299888752 >> v2:172.21.15.188:3300/0 conn(0x7fe8f815d6e0 msgr2=0x7fe8f80d0df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-26T07:18:46.629 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:46.621+0000 7fe8fe439640 1 --2- 172.21.15.143:0/1299888752 >> v2:172.21.15.188:3300/0 conn(0x7fe8f815d6e0 0x7fe8f80d0df0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:18:46.629 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:46.621+0000 7fe8fe439640 1 -- 172.21.15.143:0/1299888752 >> v1:172.21.15.143:6789/0 conn(0x7fe8f80b2160 legacy=0x7fe8f80d14b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:18:46.629 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:46.621+0000 7fe8fe439640 1 -- 172.21.15.143:0/1299888752 --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe8f80cee00 con 0x7fe8f80d1bc0 2023-11-26T07:18:46.629 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:46.621+0000 7fe8fdc38640 1 --2- 172.21.15.143:0/1299888752 >> v2:172.21.15.188:3300/0 conn(0x7fe8f815d6e0 0x7fe8f80d0df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-26T07:18:46.631 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:46.625+0000 7fe8fe439640 1 --2- 172.21.15.143:0/1299888752 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fe8f80d1bc0 0x7fe8f80b0750 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7fe8f0009d90 tx=0x7fe8f001c950 comp rx=0 tx=0).ready entity=mon.2 client_cookie=ccc83daba973f89e server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:18:46.631 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:46.625+0000 7fe8fcc36640 1 -- 172.21.15.143:0/1299888752 <== mon.2 v2:172.21.15.188:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe8f00a5900 con 0x7fe8f80d1bc0 2023-11-26T07:18:46.631 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:46.625+0000 7fe8fcc36640 1 -- 172.21.15.143:0/1299888752 <== mon.2 v2:172.21.15.188:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe8f00a5a60 con 0x7fe8f80d1bc0 2023-11-26T07:18:46.631 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:46.625+0000 7fe8fcc36640 1 -- 172.21.15.143:0/1299888752 <== mon.2 v2:172.21.15.188:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe8f00a5df0 con 0x7fe8f80d1bc0 2023-11-26T07:18:46.632 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:46.625+0000 7fe8ffe9c640 1 -- 172.21.15.143:0/1299888752 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fe8f80d1bc0 msgr2=0x7fe8f80b0750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:18:46.632 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:46.625+0000 7fe8ffe9c640 1 --2- 172.21.15.143:0/1299888752 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fe8f80d1bc0 0x7fe8f80b0750 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7fe8f0009d90 tx=0x7fe8f001c950 comp rx=0 tx=0).stop 2023-11-26T07:18:46.632 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:46.625+0000 7fe8ffe9c640 1 -- 172.21.15.143:0/1299888752 shutdown_connections 2023-11-26T07:18:46.632 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:46.625+0000 7fe8ffe9c640 1 --2- 172.21.15.143:0/1299888752 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fe8f80d1bc0 0x7fe8f80b0750 unknown :-1 s=CLOSED pgs=73 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:18:46.632 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:46.625+0000 7fe8ffe9c640 1 --2- 172.21.15.143:0/1299888752 >> v2:172.21.15.188:3300/0 conn(0x7fe8f815d6e0 0x7fe8f80d0df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:18:46.632 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:46.625+0000 7fe8ffe9c640 1 -- 172.21.15.143:0/1299888752 >> 172.21.15.143:0/1299888752 conn(0x7fe8f809c550 msgr2=0x7fe8f80d3220 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:18:46.632 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:46.625+0000 7fe8ffe9c640 1 -- 172.21.15.143:0/1299888752 shutdown_connections 2023-11-26T07:18:46.633 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:46.625+0000 7fe8ffe9c640 1 -- 172.21.15.143:0/1299888752 wait complete. 2023-11-26T07:18:46.634 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:46.629+0000 7fe8ffe9c640 1 Processor -- start 2023-11-26T07:18:46.634 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:46.629+0000 7fe8ffe9c640 1 -- start start 2023-11-26T07:18:46.636 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:46.629+0000 7fe8ffe9c640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fe8f80b2160 0x7fe8f80cbab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:18:46.636 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:46.629+0000 7fe8ffe9c640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fe8f80d1bc0 0x7fe8f8159660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:18:46.636 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:46.629+0000 7fe8ffe9c640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fe8f815d6e0 0x7fe8f8153260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:18:46.636 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:46.629+0000 7fe8ffe9c640 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe8f80cd040 con 0x7fe8f815d6e0 2023-11-26T07:18:46.636 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:46.629+0000 7fe8ffe9c640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe8f80cbff0 con 0x7fe8f80b2160 2023-11-26T07:18:46.636 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:46.629+0000 7fe8ffe9c640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe8f8154500 con 0x7fe8f80d1bc0 2023-11-26T07:18:46.637 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:46.629+0000 7fe8fe439640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fe8f815d6e0 0x7fe8f8153260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:18:46.637 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:46.629+0000 7fe8fdc38640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fe8f80b2160 0x7fe8f80cbab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:18:46.637 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:46.629+0000 7fe8fdc38640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fe8f80b2160 0x7fe8f80cbab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.188:3300/0 says I am v2:172.21.15.143:59824/0 (socket says 172.21.15.143:59824) 2023-11-26T07:18:46.637 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:46.629+0000 7fe8fdc38640 1 -- 172.21.15.143:0/1490751228 learned_addr learned my addr 172.21.15.143:0/1490751228 (peer_addr_for_me v2:172.21.15.143:0/0) 2023-11-26T07:18:46.637 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:46.629+0000 7fe8fd437640 1 --2- 172.21.15.143:0/1490751228 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fe8f80d1bc0 0x7fe8f8159660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:18:46.637 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:46.629+0000 7fe8fe439640 1 -- 172.21.15.143:0/1490751228 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fe8f80d1bc0 msgr2=0x7fe8f8159660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:18:46.637 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:46.629+0000 7fe8fe439640 1 --2- 172.21.15.143:0/1490751228 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fe8f80d1bc0 0x7fe8f8159660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:18:46.637 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:46.629+0000 7fe8fe439640 1 -- 172.21.15.143:0/1490751228 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fe8f80b2160 msgr2=0x7fe8f80cbab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:18:46.637 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:46.629+0000 7fe8fe439640 1 --2- 172.21.15.143:0/1490751228 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fe8f80b2160 0x7fe8f80cbab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:18:46.637 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:46.629+0000 7fe8fe439640 1 -- 172.21.15.143:0/1490751228 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe8f4009420 con 0x7fe8f815d6e0 2023-11-26T07:18:46.637 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:46.629+0000 7fe8fe439640 1 --2- 172.21.15.143:0/1490751228 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fe8f815d6e0 0x7fe8f8153260 secure :-1 s=READY pgs=368 cs=0 l=1 rev1=1 crypto rx=0x7fe8f80cec60 tx=0x7fe8f00196f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:18:46.637 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:46.629+0000 7fe8ee7fc640 1 -- 172.21.15.143:0/1490751228 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe8f00bc070 con 0x7fe8f815d6e0 2023-11-26T07:18:46.637 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:46.629+0000 7fe8ee7fc640 1 -- 172.21.15.143:0/1490751228 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe8f00b7440 con 0x7fe8f815d6e0 2023-11-26T07:18:46.638 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:46.629+0000 7fe8ffe9c640 1 -- 172.21.15.143:0/1490751228 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe8f0009a70 con 0x7fe8f815d6e0 2023-11-26T07:18:46.638 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:46.629+0000 7fe8ee7fc640 1 -- 172.21.15.143:0/1490751228 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe8f00b77d0 con 0x7fe8f815d6e0 2023-11-26T07:18:46.638 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:46.629+0000 7fe8ffe9c640 1 -- 172.21.15.143:0/1490751228 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe8f8154780 con 0x7fe8f815d6e0 2023-11-26T07:18:46.638 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:46.633+0000 7fe8ffe9c640 1 -- 172.21.15.143:0/1490751228 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe8f8067680 con 0x7fe8f815d6e0 2023-11-26T07:18:46.638 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:46.633+0000 7fe8ee7fc640 1 -- 172.21.15.143:0/1490751228 <== mon.0 v2:172.21.15.143:3300/0 4 ==== mgrmap(e 5) v1 ==== 85852+0+0 (secure 0 0 0) 0x7fe8f00b7930 con 0x7fe8f815d6e0 2023-11-26T07:18:46.639 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:46.633+0000 7fe8ee7fc640 1 --2- 172.21.15.143:0/1490751228 >> [v2:172.21.15.143:6832/2150557544,v1:172.21.15.143:6833/2150557544] conn(0x7fe8dc066470 0x7fe8dc068930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:18:46.639 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:46.633+0000 7fe8ee7fc640 1 -- 172.21.15.143:0/1490751228 <== mon.0 v2:172.21.15.143:3300/0 5 ==== osd_map(29..29 src has 1..29) v4 ==== 4857+0+0 (secure 0 0 0) 0x7fe8f011dfb0 con 0x7fe8f815d6e0 2023-11-26T07:18:46.639 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:46.633+0000 7fe8fdc38640 1 --2- 172.21.15.143:0/1490751228 >> [v2:172.21.15.143:6832/2150557544,v1:172.21.15.143:6833/2150557544] conn(0x7fe8dc066470 0x7fe8dc068930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:18:46.640 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:46.633+0000 7fe8fdc38640 1 --2- 172.21.15.143:0/1490751228 >> [v2:172.21.15.143:6832/2150557544,v1:172.21.15.143:6833/2150557544] conn(0x7fe8dc066470 0x7fe8dc068930 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7fe8f81607b0 tx=0x7fe8f4009190 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:18:46.643 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:46.637+0000 7fe8ee7fc640 1 -- 172.21.15.143:0/1490751228 <== mon.0 v2:172.21.15.143:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172649 (secure 0 0 0) 0x7fe8f00eb790 con 0x7fe8f815d6e0 2023-11-26T07:18:46.844 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:46.837+0000 7fe8ffe9c640 1 -- 172.21.15.143:0/1490751228 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "auth ls", "format": "json-pretty"} v 0) v1 -- 0x7fe8f80b1c70 con 0x7fe8f815d6e0 2023-11-26T07:18:46.844 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:46.837+0000 7fe8ee7fc640 1 -- 172.21.15.143:0/1490751228 <== mon.0 v2:172.21.15.143:3300/0 7 ==== mon_command_ack([{"prefix": "auth ls", "format": "json-pretty"}]=0 v6) v1 ==== 80+0+6645 (secure 0 0 0) 0x7fe8f00b2090 con 0x7fe8f815d6e0 2023-11-26T07:18:46.845 INFO:teuthology.orchestra.run.smithi143.stdout: 2023-11-26T07:18:46.845 INFO:teuthology.orchestra.run.smithi143.stdout:{ 2023-11-26T07:18:46.845 INFO:teuthology.orchestra.run.smithi143.stdout: "auth_dump": [ 2023-11-26T07:18:46.845 INFO:teuthology.orchestra.run.smithi143.stdout: { 2023-11-26T07:18:46.845 INFO:teuthology.orchestra.run.smithi143.stdout: "entity": "mds.a", 2023-11-26T07:18:46.845 INFO:teuthology.orchestra.run.smithi143.stdout: "key": "AQDl8GJldeBsOBAAhk75XZnQRURKcr0MrW17Lg==", 2023-11-26T07:18:46.845 INFO:teuthology.orchestra.run.smithi143.stdout: "caps": { 2023-11-26T07:18:46.845 INFO:teuthology.orchestra.run.smithi143.stdout: "mds": "allow", 2023-11-26T07:18:46.845 INFO:teuthology.orchestra.run.smithi143.stdout: "mgr": "allow *", 2023-11-26T07:18:46.845 INFO:teuthology.orchestra.run.smithi143.stdout: "mon": "allow *", 2023-11-26T07:18:46.845 INFO:teuthology.orchestra.run.smithi143.stdout: "osd": "allow *" 2023-11-26T07:18:46.846 INFO:teuthology.orchestra.run.smithi143.stdout: } 2023-11-26T07:18:46.846 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2023-11-26T07:18:46.846 INFO:teuthology.orchestra.run.smithi143.stdout: { 2023-11-26T07:18:46.846 INFO:teuthology.orchestra.run.smithi143.stdout: "entity": "mds.b", 2023-11-26T07:18:46.846 INFO:teuthology.orchestra.run.smithi143.stdout: "key": "AQDm8GJl0lvhCBAAXuvHKoB8TAhp0bJ4DcV94g==", 2023-11-26T07:18:46.846 INFO:teuthology.orchestra.run.smithi143.stdout: "caps": { 2023-11-26T07:18:46.846 INFO:teuthology.orchestra.run.smithi143.stdout: "mds": "allow", 2023-11-26T07:18:46.846 INFO:teuthology.orchestra.run.smithi143.stdout: "mgr": "allow *", 2023-11-26T07:18:46.846 INFO:teuthology.orchestra.run.smithi143.stdout: "mon": "allow *", 2023-11-26T07:18:46.846 INFO:teuthology.orchestra.run.smithi143.stdout: "osd": "allow *" 2023-11-26T07:18:46.846 INFO:teuthology.orchestra.run.smithi143.stdout: } 2023-11-26T07:18:46.846 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2023-11-26T07:18:46.847 INFO:teuthology.orchestra.run.smithi143.stdout: { 2023-11-26T07:18:46.847 INFO:teuthology.orchestra.run.smithi143.stdout: "entity": "mds.c", 2023-11-26T07:18:46.847 INFO:teuthology.orchestra.run.smithi143.stdout: "key": "AQDm8GJluQ0PBBAA23EmVdD3bVk52h6izUXDRQ==", 2023-11-26T07:18:46.847 INFO:teuthology.orchestra.run.smithi143.stdout: "caps": { 2023-11-26T07:18:46.847 INFO:teuthology.orchestra.run.smithi143.stdout: "mds": "allow", 2023-11-26T07:18:46.847 INFO:teuthology.orchestra.run.smithi143.stdout: "mgr": "allow *", 2023-11-26T07:18:46.847 INFO:teuthology.orchestra.run.smithi143.stdout: "mon": "allow *", 2023-11-26T07:18:46.847 INFO:teuthology.orchestra.run.smithi143.stdout: "osd": "allow *" 2023-11-26T07:18:46.847 INFO:teuthology.orchestra.run.smithi143.stdout: } 2023-11-26T07:18:46.847 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2023-11-26T07:18:46.847 INFO:teuthology.orchestra.run.smithi143.stdout: { 2023-11-26T07:18:46.847 INFO:teuthology.orchestra.run.smithi143.stdout: "entity": "mds.d", 2023-11-26T07:18:46.847 INFO:teuthology.orchestra.run.smithi143.stdout: "key": "AQDm8GJlkt9nEBAAg+LYtKVNHAN/fwYjEcNJHQ==", 2023-11-26T07:18:46.847 INFO:teuthology.orchestra.run.smithi143.stdout: "caps": { 2023-11-26T07:18:46.848 INFO:teuthology.orchestra.run.smithi143.stdout: "mds": "allow", 2023-11-26T07:18:46.848 INFO:teuthology.orchestra.run.smithi143.stdout: "mgr": "allow *", 2023-11-26T07:18:46.848 INFO:teuthology.orchestra.run.smithi143.stdout: "mon": "allow *", 2023-11-26T07:18:46.848 INFO:teuthology.orchestra.run.smithi143.stdout: "osd": "allow *" 2023-11-26T07:18:46.848 INFO:teuthology.orchestra.run.smithi143.stdout: } 2023-11-26T07:18:46.848 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2023-11-26T07:18:46.848 INFO:teuthology.orchestra.run.smithi143.stdout: { 2023-11-26T07:18:46.848 INFO:teuthology.orchestra.run.smithi143.stdout: "entity": "osd.0", 2023-11-26T07:18:46.848 INFO:teuthology.orchestra.run.smithi143.stdout: "key": "AQDp8GJlNscZLhAA31zK4vTcnxwiRIRgxzewpA==", 2023-11-26T07:18:46.848 INFO:teuthology.orchestra.run.smithi143.stdout: "caps": { 2023-11-26T07:18:46.848 INFO:teuthology.orchestra.run.smithi143.stdout: "mgr": "allow profile osd", 2023-11-26T07:18:46.848 INFO:teuthology.orchestra.run.smithi143.stdout: "mon": "allow profile osd", 2023-11-26T07:18:46.848 INFO:teuthology.orchestra.run.smithi143.stdout: "osd": "allow *" 2023-11-26T07:18:46.849 INFO:teuthology.orchestra.run.smithi143.stdout: } 2023-11-26T07:18:46.849 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2023-11-26T07:18:46.849 INFO:teuthology.orchestra.run.smithi143.stdout: { 2023-11-26T07:18:46.849 INFO:teuthology.orchestra.run.smithi143.stdout: "entity": "osd.1", 2023-11-26T07:18:46.849 INFO:teuthology.orchestra.run.smithi143.stdout: "key": "AQDs8GJlsyKUNBAA+dQ1ElGPrdP1GV7lk4JQjQ==", 2023-11-26T07:18:46.849 INFO:teuthology.orchestra.run.smithi143.stdout: "caps": { 2023-11-26T07:18:46.849 INFO:teuthology.orchestra.run.smithi143.stdout: "mgr": "allow profile osd", 2023-11-26T07:18:46.849 INFO:teuthology.orchestra.run.smithi143.stdout: "mon": "allow profile osd", 2023-11-26T07:18:46.849 INFO:teuthology.orchestra.run.smithi143.stdout: "osd": "allow *" 2023-11-26T07:18:46.849 INFO:teuthology.orchestra.run.smithi143.stdout: } 2023-11-26T07:18:46.849 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2023-11-26T07:18:46.849 INFO:teuthology.orchestra.run.smithi143.stdout: { 2023-11-26T07:18:46.849 INFO:teuthology.orchestra.run.smithi143.stdout: "entity": "osd.2", 2023-11-26T07:18:46.849 INFO:teuthology.orchestra.run.smithi143.stdout: "key": "AQDv8GJlXsy7OhAAJJdldsodgGZVtyV2HuyC4A==", 2023-11-26T07:18:46.850 INFO:teuthology.orchestra.run.smithi143.stdout: "caps": { 2023-11-26T07:18:46.850 INFO:teuthology.orchestra.run.smithi143.stdout: "mgr": "allow profile osd", 2023-11-26T07:18:46.850 INFO:teuthology.orchestra.run.smithi143.stdout: "mon": "allow profile osd", 2023-11-26T07:18:46.850 INFO:teuthology.orchestra.run.smithi143.stdout: "osd": "allow *" 2023-11-26T07:18:46.850 INFO:teuthology.orchestra.run.smithi143.stdout: } 2023-11-26T07:18:46.850 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2023-11-26T07:18:46.850 INFO:teuthology.orchestra.run.smithi143.stdout: { 2023-11-26T07:18:46.850 INFO:teuthology.orchestra.run.smithi143.stdout: "entity": "osd.3", 2023-11-26T07:18:46.850 INFO:teuthology.orchestra.run.smithi143.stdout: "key": "AQDz8GJloLhMBxAAOWD9ec2y4c8nkdJmQnne8Q==", 2023-11-26T07:18:46.850 INFO:teuthology.orchestra.run.smithi143.stdout: "caps": { 2023-11-26T07:18:46.850 INFO:teuthology.orchestra.run.smithi143.stdout: "mgr": "allow profile osd", 2023-11-26T07:18:46.850 INFO:teuthology.orchestra.run.smithi143.stdout: "mon": "allow profile osd", 2023-11-26T07:18:46.850 INFO:teuthology.orchestra.run.smithi143.stdout: "osd": "allow *" 2023-11-26T07:18:46.851 INFO:teuthology.orchestra.run.smithi143.stdout: } 2023-11-26T07:18:46.851 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2023-11-26T07:18:46.851 INFO:teuthology.orchestra.run.smithi143.stdout: { 2023-11-26T07:18:46.851 INFO:teuthology.orchestra.run.smithi143.stdout: "entity": "osd.4", 2023-11-26T07:18:46.851 INFO:teuthology.orchestra.run.smithi143.stdout: "key": "AQD58GJlDlZsDBAAZNauAPPPfAZ+KIqSmgSQhg==", 2023-11-26T07:18:46.851 INFO:teuthology.orchestra.run.smithi143.stdout: "caps": { 2023-11-26T07:18:46.851 INFO:teuthology.orchestra.run.smithi143.stdout: "mgr": "allow profile osd", 2023-11-26T07:18:46.851 INFO:teuthology.orchestra.run.smithi143.stdout: "mon": "allow profile osd", 2023-11-26T07:18:46.851 INFO:teuthology.orchestra.run.smithi143.stdout: "osd": "allow *" 2023-11-26T07:18:46.851 INFO:teuthology.orchestra.run.smithi143.stdout: } 2023-11-26T07:18:46.851 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2023-11-26T07:18:46.851 INFO:teuthology.orchestra.run.smithi143.stdout: { 2023-11-26T07:18:46.851 INFO:teuthology.orchestra.run.smithi143.stdout: "entity": "osd.5", 2023-11-26T07:18:46.851 INFO:teuthology.orchestra.run.smithi143.stdout: "key": "AQD88GJlNE4mExAA5jYkCa0tE3oR6rgK1Dt5cA==", 2023-11-26T07:18:46.852 INFO:teuthology.orchestra.run.smithi143.stdout: "caps": { 2023-11-26T07:18:46.852 INFO:teuthology.orchestra.run.smithi143.stdout: "mgr": "allow profile osd", 2023-11-26T07:18:46.852 INFO:teuthology.orchestra.run.smithi143.stdout: "mon": "allow profile osd", 2023-11-26T07:18:46.852 INFO:teuthology.orchestra.run.smithi143.stdout: "osd": "allow *" 2023-11-26T07:18:46.852 INFO:teuthology.orchestra.run.smithi143.stdout: } 2023-11-26T07:18:46.852 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2023-11-26T07:18:46.852 INFO:teuthology.orchestra.run.smithi143.stdout: { 2023-11-26T07:18:46.852 INFO:teuthology.orchestra.run.smithi143.stdout: "entity": "osd.6", 2023-11-26T07:18:46.852 INFO:teuthology.orchestra.run.smithi143.stdout: "key": "AQD/8GJlh0lYGRAArHalCyZ36UvBOVMZor+N/g==", 2023-11-26T07:18:46.852 INFO:teuthology.orchestra.run.smithi143.stdout: "caps": { 2023-11-26T07:18:46.852 INFO:teuthology.orchestra.run.smithi143.stdout: "mgr": "allow profile osd", 2023-11-26T07:18:46.852 INFO:teuthology.orchestra.run.smithi143.stdout: "mon": "allow profile osd", 2023-11-26T07:18:46.853 INFO:teuthology.orchestra.run.smithi143.stdout: "osd": "allow *" 2023-11-26T07:18:46.853 INFO:teuthology.orchestra.run.smithi143.stdout: } 2023-11-26T07:18:46.853 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2023-11-26T07:18:46.853 INFO:teuthology.orchestra.run.smithi143.stdout: { 2023-11-26T07:18:46.853 INFO:teuthology.orchestra.run.smithi143.stdout: "entity": "osd.7", 2023-11-26T07:18:46.853 INFO:teuthology.orchestra.run.smithi143.stdout: "key": "AQAC8WJlJavtIBAAKMvnR4AnnrDVOXlzN7j05w==", 2023-11-26T07:18:46.853 INFO:teuthology.orchestra.run.smithi143.stdout: "caps": { 2023-11-26T07:18:46.853 INFO:teuthology.orchestra.run.smithi143.stdout: "mgr": "allow profile osd", 2023-11-26T07:18:46.853 INFO:teuthology.orchestra.run.smithi143.stdout: "mon": "allow profile osd", 2023-11-26T07:18:46.853 INFO:teuthology.orchestra.run.smithi143.stdout: "osd": "allow *" 2023-11-26T07:18:46.853 INFO:teuthology.orchestra.run.smithi143.stdout: } 2023-11-26T07:18:46.853 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2023-11-26T07:18:46.853 INFO:teuthology.orchestra.run.smithi143.stdout: { 2023-11-26T07:18:46.854 INFO:teuthology.orchestra.run.smithi143.stdout: "entity": "client.0", 2023-11-26T07:18:46.854 INFO:teuthology.orchestra.run.smithi143.stdout: "key": "AQDm8GJlWMh9FBAAslVzi6fm4/wt7vr/rHCHHQ==", 2023-11-26T07:18:46.854 INFO:teuthology.orchestra.run.smithi143.stdout: "caps": { 2023-11-26T07:18:46.854 INFO:teuthology.orchestra.run.smithi143.stdout: "mds": "allow", 2023-11-26T07:18:46.854 INFO:teuthology.orchestra.run.smithi143.stdout: "mgr": "allow r", 2023-11-26T07:18:46.854 INFO:teuthology.orchestra.run.smithi143.stdout: "mon": "allow rw", 2023-11-26T07:18:46.854 INFO:teuthology.orchestra.run.smithi143.stdout: "osd": "allow rwx" 2023-11-26T07:18:46.854 INFO:teuthology.orchestra.run.smithi143.stdout: } 2023-11-26T07:18:46.854 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2023-11-26T07:18:46.854 INFO:teuthology.orchestra.run.smithi143.stdout: { 2023-11-26T07:18:46.854 INFO:teuthology.orchestra.run.smithi143.stdout: "entity": "client.1", 2023-11-26T07:18:46.854 INFO:teuthology.orchestra.run.smithi143.stdout: "key": "AQDm8GJl5vkYHhAAgP+gKH2+lpl3aTlgqZNIXg==", 2023-11-26T07:18:46.854 INFO:teuthology.orchestra.run.smithi143.stdout: "caps": { 2023-11-26T07:18:46.855 INFO:teuthology.orchestra.run.smithi143.stdout: "mds": "allow", 2023-11-26T07:18:46.855 INFO:teuthology.orchestra.run.smithi143.stdout: "mgr": "allow r", 2023-11-26T07:18:46.855 INFO:teuthology.orchestra.run.smithi143.stdout: "mon": "allow rw", 2023-11-26T07:18:46.855 INFO:teuthology.orchestra.run.smithi143.stdout: "osd": "allow rwx" 2023-11-26T07:18:46.855 INFO:teuthology.orchestra.run.smithi143.stdout: } 2023-11-26T07:18:46.855 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2023-11-26T07:18:46.855 INFO:teuthology.orchestra.run.smithi143.stdout: { 2023-11-26T07:18:46.855 INFO:teuthology.orchestra.run.smithi143.stdout: "entity": "client.2", 2023-11-26T07:18:46.855 INFO:teuthology.orchestra.run.smithi143.stdout: "key": "AQDm8GJlW4KlGhAANFKsyVJems0ZoE6bo3WmfQ==", 2023-11-26T07:18:46.855 INFO:teuthology.orchestra.run.smithi143.stdout: "caps": { 2023-11-26T07:18:46.855 INFO:teuthology.orchestra.run.smithi143.stdout: "mds": "allow", 2023-11-26T07:18:46.855 INFO:teuthology.orchestra.run.smithi143.stdout: "mgr": "allow r", 2023-11-26T07:18:46.855 INFO:teuthology.orchestra.run.smithi143.stdout: "mon": "allow rw", 2023-11-26T07:18:46.856 INFO:teuthology.orchestra.run.smithi143.stdout: "osd": "allow rwx" 2023-11-26T07:18:46.856 INFO:teuthology.orchestra.run.smithi143.stdout: } 2023-11-26T07:18:46.856 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2023-11-26T07:18:46.856 INFO:teuthology.orchestra.run.smithi143.stdout: { 2023-11-26T07:18:46.856 INFO:teuthology.orchestra.run.smithi143.stdout: "entity": "client.3", 2023-11-26T07:18:46.856 INFO:teuthology.orchestra.run.smithi143.stdout: "key": "AQDm8GJlAJZSJBAAUSUNcwRBY01hOuilBJjlFQ==", 2023-11-26T07:18:46.856 INFO:teuthology.orchestra.run.smithi143.stdout: "caps": { 2023-11-26T07:18:46.856 INFO:teuthology.orchestra.run.smithi143.stdout: "mds": "allow", 2023-11-26T07:18:46.856 INFO:teuthology.orchestra.run.smithi143.stdout: "mgr": "allow r", 2023-11-26T07:18:46.856 INFO:teuthology.orchestra.run.smithi143.stdout: "mon": "allow rw", 2023-11-26T07:18:46.856 INFO:teuthology.orchestra.run.smithi143.stdout: "osd": "allow rwx" 2023-11-26T07:18:46.856 INFO:teuthology.orchestra.run.smithi143.stdout: } 2023-11-26T07:18:46.856 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2023-11-26T07:18:46.857 INFO:teuthology.orchestra.run.smithi143.stdout: { 2023-11-26T07:18:46.857 INFO:teuthology.orchestra.run.smithi143.stdout: "entity": "client.admin", 2023-11-26T07:18:46.857 INFO:teuthology.orchestra.run.smithi143.stdout: "key": "AQDl8GJlbQv7GxAA0Midh8tAO2KO+dH10OV8Xg==", 2023-11-26T07:18:46.857 INFO:teuthology.orchestra.run.smithi143.stdout: "caps": { 2023-11-26T07:18:46.857 INFO:teuthology.orchestra.run.smithi143.stdout: "mds": "allow *", 2023-11-26T07:18:46.857 INFO:teuthology.orchestra.run.smithi143.stdout: "mgr": "allow *", 2023-11-26T07:18:46.857 INFO:teuthology.orchestra.run.smithi143.stdout: "mon": "allow *", 2023-11-26T07:18:46.857 INFO:teuthology.orchestra.run.smithi143.stdout: "osd": "allow *" 2023-11-26T07:18:46.857 INFO:teuthology.orchestra.run.smithi143.stdout: } 2023-11-26T07:18:46.857 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2023-11-26T07:18:46.857 INFO:teuthology.orchestra.run.smithi143.stdout: { 2023-11-26T07:18:46.857 INFO:teuthology.orchestra.run.smithi143.stdout: "entity": "client.bootstrap-mds", 2023-11-26T07:18:46.857 INFO:teuthology.orchestra.run.smithi143.stdout: "key": "AQAI8WJlluKeJBAAl2NXECj47Mddcd10uvHFVg==", 2023-11-26T07:18:46.858 INFO:teuthology.orchestra.run.smithi143.stdout: "caps": { 2023-11-26T07:18:46.858 INFO:teuthology.orchestra.run.smithi143.stdout: "mon": "allow profile bootstrap-mds" 2023-11-26T07:18:46.858 INFO:teuthology.orchestra.run.smithi143.stdout: } 2023-11-26T07:18:46.858 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2023-11-26T07:18:46.858 INFO:teuthology.orchestra.run.smithi143.stdout: { 2023-11-26T07:18:46.858 INFO:teuthology.orchestra.run.smithi143.stdout: "entity": "client.bootstrap-mgr", 2023-11-26T07:18:46.858 INFO:teuthology.orchestra.run.smithi143.stdout: "key": "AQAI8WJl2OyeJBAA9tidBTKmH+A1k4ZKguJfgg==", 2023-11-26T07:18:46.858 INFO:teuthology.orchestra.run.smithi143.stdout: "caps": { 2023-11-26T07:18:46.858 INFO:teuthology.orchestra.run.smithi143.stdout: "mon": "allow profile bootstrap-mgr" 2023-11-26T07:18:46.858 INFO:teuthology.orchestra.run.smithi143.stdout: } 2023-11-26T07:18:46.858 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2023-11-26T07:18:46.858 INFO:teuthology.orchestra.run.smithi143.stdout: { 2023-11-26T07:18:46.858 INFO:teuthology.orchestra.run.smithi143.stdout: "entity": "client.bootstrap-osd", 2023-11-26T07:18:46.859 INFO:teuthology.orchestra.run.smithi143.stdout: "key": "AQAI8WJlnfWeJBAA+/TiMJpiQ0h2KSloaDYjPQ==", 2023-11-26T07:18:46.859 INFO:teuthology.orchestra.run.smithi143.stdout: "caps": { 2023-11-26T07:18:46.859 INFO:teuthology.orchestra.run.smithi143.stdout: "mon": "allow profile bootstrap-osd" 2023-11-26T07:18:46.859 INFO:teuthology.orchestra.run.smithi143.stdout: } 2023-11-26T07:18:46.859 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2023-11-26T07:18:46.859 INFO:teuthology.orchestra.run.smithi143.stdout: { 2023-11-26T07:18:46.859 INFO:teuthology.orchestra.run.smithi143.stdout: "entity": "client.bootstrap-rbd", 2023-11-26T07:18:46.859 INFO:teuthology.orchestra.run.smithi143.stdout: "key": "AQAI8WJlM/6eJBAAvMaMyIEo++Q88XCYICBfaQ==", 2023-11-26T07:18:46.859 INFO:teuthology.orchestra.run.smithi143.stdout: "caps": { 2023-11-26T07:18:46.859 INFO:teuthology.orchestra.run.smithi143.stdout: "mon": "allow profile bootstrap-rbd" 2023-11-26T07:18:46.859 INFO:teuthology.orchestra.run.smithi143.stdout: } 2023-11-26T07:18:46.859 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2023-11-26T07:18:46.859 INFO:teuthology.orchestra.run.smithi143.stdout: { 2023-11-26T07:18:46.860 INFO:teuthology.orchestra.run.smithi143.stdout: "entity": "client.bootstrap-rbd-mirror", 2023-11-26T07:18:46.860 INFO:teuthology.orchestra.run.smithi143.stdout: "key": "AQAI8WJlMgefJBAAIVsVOBochgjSGMoSQlDSIA==", 2023-11-26T07:18:46.860 INFO:teuthology.orchestra.run.smithi143.stdout: "caps": { 2023-11-26T07:18:46.860 INFO:teuthology.orchestra.run.smithi143.stdout: "mon": "allow profile bootstrap-rbd-mirror" 2023-11-26T07:18:46.860 INFO:teuthology.orchestra.run.smithi143.stdout: } 2023-11-26T07:18:46.860 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2023-11-26T07:18:46.860 INFO:teuthology.orchestra.run.smithi143.stdout: { 2023-11-26T07:18:46.860 INFO:teuthology.orchestra.run.smithi143.stdout: "entity": "client.bootstrap-rgw", 2023-11-26T07:18:46.860 INFO:teuthology.orchestra.run.smithi143.stdout: "key": "AQAI8WJlARGfJBAAvWYx8UmBVk7pet7NIdU+YA==", 2023-11-26T07:18:46.860 INFO:teuthology.orchestra.run.smithi143.stdout: "caps": { 2023-11-26T07:18:46.860 INFO:teuthology.orchestra.run.smithi143.stdout: "mon": "allow profile bootstrap-rgw" 2023-11-26T07:18:46.860 INFO:teuthology.orchestra.run.smithi143.stdout: } 2023-11-26T07:18:46.860 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2023-11-26T07:18:46.860 INFO:teuthology.orchestra.run.smithi143.stdout: { 2023-11-26T07:18:46.861 INFO:teuthology.orchestra.run.smithi143.stdout: "entity": "mgr.x", 2023-11-26T07:18:46.861 INFO:teuthology.orchestra.run.smithi143.stdout: "key": "AQDl8GJlXy8cNBAA4f5RDC1MASCqZPcJQvnUDA==", 2023-11-26T07:18:46.861 INFO:teuthology.orchestra.run.smithi143.stdout: "caps": { 2023-11-26T07:18:46.861 INFO:teuthology.orchestra.run.smithi143.stdout: "mds": "allow *", 2023-11-26T07:18:46.861 INFO:teuthology.orchestra.run.smithi143.stdout: "mon": "allow profile mgr", 2023-11-26T07:18:46.861 INFO:teuthology.orchestra.run.smithi143.stdout: "osd": "allow *" 2023-11-26T07:18:46.861 INFO:teuthology.orchestra.run.smithi143.stdout: } 2023-11-26T07:18:46.861 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2023-11-26T07:18:46.861 INFO:teuthology.orchestra.run.smithi143.stdout: { 2023-11-26T07:18:46.861 INFO:teuthology.orchestra.run.smithi143.stdout: "entity": "mgr.y", 2023-11-26T07:18:46.862 INFO:teuthology.orchestra.run.smithi143.stdout: "key": "AQDl8GJlfuFPMBAA3TYEFvdfk4DyGsBkbpv0hg==", 2023-11-26T07:18:46.862 INFO:teuthology.orchestra.run.smithi143.stdout: "caps": { 2023-11-26T07:18:46.862 INFO:teuthology.orchestra.run.smithi143.stdout: "mds": "allow *", 2023-11-26T07:18:46.862 INFO:teuthology.orchestra.run.smithi143.stdout: "mon": "allow profile mgr", 2023-11-26T07:18:46.862 INFO:teuthology.orchestra.run.smithi143.stdout: "osd": "allow *" 2023-11-26T07:18:46.862 INFO:teuthology.orchestra.run.smithi143.stdout: } 2023-11-26T07:18:46.862 INFO:teuthology.orchestra.run.smithi143.stdout: } 2023-11-26T07:18:46.862 INFO:teuthology.orchestra.run.smithi143.stdout: ] 2023-11-26T07:18:46.862 INFO:teuthology.orchestra.run.smithi143.stdout:} 2023-11-26T07:18:46.863 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:46.841+0000 7fe8ffe9c640 1 -- 172.21.15.143:0/1490751228 >> [v2:172.21.15.143:6832/2150557544,v1:172.21.15.143:6833/2150557544] conn(0x7fe8dc066470 msgr2=0x7fe8dc068930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:18:46.863 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:46.841+0000 7fe8ffe9c640 1 --2- 172.21.15.143:0/1490751228 >> [v2:172.21.15.143:6832/2150557544,v1:172.21.15.143:6833/2150557544] conn(0x7fe8dc066470 0x7fe8dc068930 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7fe8f81607b0 tx=0x7fe8f4009190 comp rx=0 tx=0).stop 2023-11-26T07:18:46.863 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:46.841+0000 7fe8ffe9c640 1 -- 172.21.15.143:0/1490751228 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fe8f815d6e0 msgr2=0x7fe8f8153260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:18:46.863 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:46.841+0000 7fe8ffe9c640 1 --2- 172.21.15.143:0/1490751228 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fe8f815d6e0 0x7fe8f8153260 secure :-1 s=READY pgs=368 cs=0 l=1 rev1=1 crypto rx=0x7fe8f80cec60 tx=0x7fe8f00196f0 comp rx=0 tx=0).stop 2023-11-26T07:18:46.863 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:46.841+0000 7fe8ffe9c640 1 -- 172.21.15.143:0/1490751228 shutdown_connections 2023-11-26T07:18:46.863 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:46.841+0000 7fe8ffe9c640 1 --2- 172.21.15.143:0/1490751228 >> [v2:172.21.15.143:6832/2150557544,v1:172.21.15.143:6833/2150557544] conn(0x7fe8dc066470 0x7fe8dc068930 unknown :-1 s=CLOSED pgs=200 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:18:46.864 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:46.841+0000 7fe8ffe9c640 1 --2- 172.21.15.143:0/1490751228 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fe8f815d6e0 0x7fe8f8153260 unknown :-1 s=CLOSED pgs=368 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:18:46.864 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:46.841+0000 7fe8ffe9c640 1 --2- 172.21.15.143:0/1490751228 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fe8f80d1bc0 0x7fe8f8159660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:18:46.864 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:46.841+0000 7fe8ffe9c640 1 --2- 172.21.15.143:0/1490751228 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fe8f80b2160 0x7fe8f80cbab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:18:46.864 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:46.841+0000 7fe8ffe9c640 1 -- 172.21.15.143:0/1490751228 >> 172.21.15.143:0/1490751228 conn(0x7fe8f809c550 msgr2=0x7fe8f80b3170 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:18:46.864 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:46.841+0000 7fe8ffe9c640 1 -- 172.21.15.143:0/1490751228 shutdown_connections 2023-11-26T07:18:46.864 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:46.841+0000 7fe8ffe9c640 1 -- 172.21.15.143:0/1490751228 wait complete. 2023-11-26T07:18:46.866 DEBUG:teuthology.orchestra.run.smithi143:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth del client.bootstrap-mds 2023-11-26T07:18:47.005 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:46.997+0000 7fabc2aea640 1 Processor -- start 2023-11-26T07:18:47.005 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:47.001+0000 7fabc2aea640 1 -- start start 2023-11-26T07:18:47.006 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:47.001+0000 7fabc2aea640 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7fabbc0ce7c0 0x7fabbc15f6b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:18:47.006 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:47.001+0000 7fabc2aea640 1 -- --> v1:172.21.15.143:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fabbc06c520 con 0x7fabbc0cf990 2023-11-26T07:18:47.006 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:47.001+0000 7fabc2aea640 1 -- --> v1:172.21.15.188:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fabbc06c800 con 0x7fabbc13e290 2023-11-26T07:18:47.006 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:47.001+0000 7fabc2aea640 1 -- --> v2:172.21.15.188:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fabbc06cae0 con 0x7fabbc0ce7c0 2023-11-26T07:18:47.007 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:47.001+0000 7fabbbfff640 1 --1- >> v1:172.21.15.143:6789/0 conn(0x7fabbc0cf990 0x7fabbc15efa0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.143:6789/0 says I am v1:172.21.15.143:55142/0 (socket says 172.21.15.143:55142) 2023-11-26T07:18:47.007 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:47.001+0000 7fabbbfff640 1 -- 172.21.15.143:0/422940763 learned_addr learned my addr 172.21.15.143:0/422940763 (peer_addr_for_me v1:172.21.15.143:0/0) 2023-11-26T07:18:47.007 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:47.001+0000 7fabc1087640 1 --2- 172.21.15.143:0/422940763 >> v2:172.21.15.188:3300/0 conn(0x7fabbc0ce7c0 0x7fabbc15f6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:18:47.007 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:47.001+0000 7fabbb7fe640 1 -- 172.21.15.143:0/422940763 <== mon.0 v1:172.21.15.143:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1673611314 0 0) 0x7fabbc06c520 con 0x7fabbc0cf990 2023-11-26T07:18:47.008 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:47.001+0000 7fabbb7fe640 1 -- 172.21.15.143:0/422940763 --> v1:172.21.15.143:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fabac003530 con 0x7fabbc0cf990 2023-11-26T07:18:47.008 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:47.001+0000 7fabc1087640 1 -- 172.21.15.143:0/422940763 >> v1:172.21.15.188:6789/0 conn(0x7fabbc13e290 legacy=0x7fabbc0cf280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:18:47.008 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:47.001+0000 7fabbb7fe640 1 -- 172.21.15.143:0/422940763 <== mon.1 v1:172.21.15.188:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3724680046 0 0) 0x7fabbc06c800 con 0x7fabbc13e290 2023-11-26T07:18:47.008 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:47.001+0000 7fabc1087640 1 -- 172.21.15.143:0/422940763 >> v1:172.21.15.143:6789/0 conn(0x7fabbc0cf990 legacy=0x7fabbc15efa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:18:47.008 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:47.001+0000 7fabc1087640 1 -- 172.21.15.143:0/422940763 --> v2:172.21.15.188:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fabbc06cdc0 con 0x7fabbc0ce7c0 2023-11-26T07:18:47.010 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:47.005+0000 7fabc1087640 1 --2- 172.21.15.143:0/422940763 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fabbc0ce7c0 0x7fabbc15f6b0 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7fabb4009d90 tx=0x7fabb401c890 comp rx=0 tx=0).ready entity=mon.1 client_cookie=5bc503c2d7e14419 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:18:47.010 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:47.005+0000 7fabbb7fe640 1 -- 172.21.15.143:0/422940763 <== mon.1 v2:172.21.15.188:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fabb40a5970 con 0x7fabbc0ce7c0 2023-11-26T07:18:47.010 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:47.005+0000 7fabbb7fe640 1 -- 172.21.15.143:0/422940763 <== mon.1 v2:172.21.15.188:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fabb40a5ad0 con 0x7fabbc0ce7c0 2023-11-26T07:18:47.010 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:47.005+0000 7fabbb7fe640 1 -- 172.21.15.143:0/422940763 <== mon.1 v2:172.21.15.188:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fabb40a5e60 con 0x7fabbc0ce7c0 2023-11-26T07:18:47.011 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:47.005+0000 7fabc2aea640 1 -- 172.21.15.143:0/422940763 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fabbc0ce7c0 msgr2=0x7fabbc15f6b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:18:47.011 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:47.005+0000 7fabc2aea640 1 --2- 172.21.15.143:0/422940763 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fabbc0ce7c0 0x7fabbc15f6b0 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7fabb4009d90 tx=0x7fabb401c890 comp rx=0 tx=0).stop 2023-11-26T07:18:47.011 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:47.005+0000 7fabc2aea640 1 -- 172.21.15.143:0/422940763 shutdown_connections 2023-11-26T07:18:47.011 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:47.005+0000 7fabc2aea640 1 --2- 172.21.15.143:0/422940763 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fabbc0ce7c0 0x7fabbc15f6b0 unknown :-1 s=CLOSED pgs=174 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:18:47.011 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:47.005+0000 7fabc2aea640 1 -- 172.21.15.143:0/422940763 >> 172.21.15.143:0/422940763 conn(0x7fabbc06a890 msgr2=0x7fabbc06aca0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:18:47.011 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:47.005+0000 7fabc2aea640 1 -- 172.21.15.143:0/422940763 shutdown_connections 2023-11-26T07:18:47.011 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:47.005+0000 7fabc2aea640 1 -- 172.21.15.143:0/422940763 wait complete. 2023-11-26T07:18:47.012 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:47.005+0000 7fabc2aea640 1 Processor -- start 2023-11-26T07:18:47.012 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:47.005+0000 7fabc2aea640 1 -- start start 2023-11-26T07:18:47.014 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:47.009+0000 7fabc2aea640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fabbc0ce7c0 0x7fabbc128620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:18:47.014 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:47.009+0000 7fabc2aea640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fabbc0cf990 0x7fabbc14dd20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:18:47.015 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:47.009+0000 7fabc2aea640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fabbc13e290 0x7fabbc159f90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:18:47.015 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:47.009+0000 7fabc2aea640 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fabbc128b60 con 0x7fabbc0cf990 2023-11-26T07:18:47.015 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:47.009+0000 7fabc2aea640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fabbc14c7b0 con 0x7fabbc13e290 2023-11-26T07:18:47.015 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:47.009+0000 7fabc2aea640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fabbc14c4d0 con 0x7fabbc0ce7c0 2023-11-26T07:18:47.015 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:47.009+0000 7fabbbfff640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fabbc0cf990 0x7fabbc14dd20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:18:47.015 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:47.009+0000 7fabbbfff640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fabbc0cf990 0x7fabbc14dd20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:47404/0 (socket says 172.21.15.143:47404) 2023-11-26T07:18:47.015 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:47.009+0000 7fabbbfff640 1 -- 172.21.15.143:0/772530955 learned_addr learned my addr 172.21.15.143:0/772530955 (peer_addr_for_me v2:172.21.15.143:0/0) 2023-11-26T07:18:47.015 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:47.009+0000 7fabc1087640 1 --2- 172.21.15.143:0/772530955 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fabbc13e290 0x7fabbc159f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:18:47.015 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:47.009+0000 7fabc0886640 1 --2- 172.21.15.143:0/772530955 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fabbc0ce7c0 0x7fabbc128620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:18:47.015 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:47.009+0000 7fabbbfff640 1 -- 172.21.15.143:0/772530955 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fabbc0ce7c0 msgr2=0x7fabbc128620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:18:47.015 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:47.009+0000 7fabbbfff640 1 --2- 172.21.15.143:0/772530955 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fabbc0ce7c0 0x7fabbc128620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:18:47.015 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:47.009+0000 7fabbbfff640 1 -- 172.21.15.143:0/772530955 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fabbc13e290 msgr2=0x7fabbc159f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:18:47.016 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:47.009+0000 7fabbbfff640 1 --2- 172.21.15.143:0/772530955 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fabbc13e290 0x7fabbc159f90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:18:47.016 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:47.009+0000 7fabbbfff640 1 -- 172.21.15.143:0/772530955 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fabb4009a70 con 0x7fabbc0cf990 2023-11-26T07:18:47.016 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:47.009+0000 7fabbbfff640 1 --2- 172.21.15.143:0/772530955 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fabbc0cf990 0x7fabbc14dd20 secure :-1 s=READY pgs=370 cs=0 l=1 rev1=1 crypto rx=0x7faba800c450 tx=0x7faba800c910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:18:47.016 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:47.009+0000 7fabb97fa640 1 -- 172.21.15.143:0/772530955 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faba8005cf0 con 0x7fabbc0cf990 2023-11-26T07:18:47.017 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:47.009+0000 7fabb97fa640 1 -- 172.21.15.143:0/772530955 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7faba8005e50 con 0x7fabbc0cf990 2023-11-26T07:18:47.017 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:47.009+0000 7fabc2aea640 1 -- 172.21.15.143:0/772530955 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fabbc0b1d50 con 0x7fabbc0cf990 2023-11-26T07:18:47.017 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:47.009+0000 7fabb97fa640 1 -- 172.21.15.143:0/772530955 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faba8007290 con 0x7fabbc0cf990 2023-11-26T07:18:47.017 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:47.009+0000 7fabc2aea640 1 -- 172.21.15.143:0/772530955 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fabbc0b2310 con 0x7fabbc0cf990 2023-11-26T07:18:47.017 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:47.009+0000 7fabc2aea640 1 -- 172.21.15.143:0/772530955 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fabbc06df50 con 0x7fabbc0cf990 2023-11-26T07:18:47.017 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:47.013+0000 7fabb97fa640 1 -- 172.21.15.143:0/772530955 <== mon.0 v2:172.21.15.143:3300/0 4 ==== mgrmap(e 5) v1 ==== 85852+0+0 (secure 0 0 0) 0x7faba80073f0 con 0x7fabbc0cf990 2023-11-26T07:18:47.018 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:47.013+0000 7fabb97fa640 1 --2- 172.21.15.143:0/772530955 >> [v2:172.21.15.143:6832/2150557544,v1:172.21.15.143:6833/2150557544] conn(0x7faba0066470 0x7faba0068930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:18:47.018 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:47.013+0000 7fabb97fa640 1 -- 172.21.15.143:0/772530955 <== mon.0 v2:172.21.15.143:3300/0 5 ==== osd_map(29..29 src has 1..29) v4 ==== 4857+0+0 (secure 0 0 0) 0x7faba80830f0 con 0x7fabbc0cf990 2023-11-26T07:18:47.018 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:47.013+0000 7fabc0886640 1 --2- 172.21.15.143:0/772530955 >> [v2:172.21.15.143:6832/2150557544,v1:172.21.15.143:6833/2150557544] conn(0x7faba0066470 0x7faba0068930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:18:47.018 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:47.013+0000 7fabc0886640 1 --2- 172.21.15.143:0/772530955 >> [v2:172.21.15.143:6832/2150557544,v1:172.21.15.143:6833/2150557544] conn(0x7faba0066470 0x7faba0068930 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7fabb0000f70 tx=0x7fabb0009040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:18:47.022 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:47.017+0000 7fabb97fa640 1 -- 172.21.15.143:0/772530955 <== mon.0 v2:172.21.15.143:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172649 (secure 0 0 0) 0x7faba80508d0 con 0x7fabbc0cf990 2023-11-26T07:18:47.223 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:47.217+0000 7fabc2aea640 1 -- 172.21.15.143:0/772530955 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "auth del", "entity": "client.bootstrap-mds"} v 0) v1 -- 0x7fabbc067660 con 0x7fabbc0cf990 2023-11-26T07:18:47.225 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:47.221+0000 7fabb97fa640 1 -- 172.21.15.143:0/772530955 <== mon.0 v2:172.21.15.143:3300/0 7 ==== mon_command_ack([{"prefix": "auth del", "entity": "client.bootstrap-mds"}]=0 v7) v1 ==== 90+0+0 (secure 0 0 0) 0x7faba80170a0 con 0x7fabbc0cf990 2023-11-26T07:18:47.228 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:47.221+0000 7fabc2aea640 1 -- 172.21.15.143:0/772530955 >> [v2:172.21.15.143:6832/2150557544,v1:172.21.15.143:6833/2150557544] conn(0x7faba0066470 msgr2=0x7faba0068930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:18:47.228 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:47.221+0000 7fabc2aea640 1 --2- 172.21.15.143:0/772530955 >> [v2:172.21.15.143:6832/2150557544,v1:172.21.15.143:6833/2150557544] conn(0x7faba0066470 0x7faba0068930 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7fabb0000f70 tx=0x7fabb0009040 comp rx=0 tx=0).stop 2023-11-26T07:18:47.228 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:47.221+0000 7fabc2aea640 1 -- 172.21.15.143:0/772530955 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fabbc0cf990 msgr2=0x7fabbc14dd20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:18:47.228 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:47.221+0000 7fabc2aea640 1 --2- 172.21.15.143:0/772530955 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fabbc0cf990 0x7fabbc14dd20 secure :-1 s=READY pgs=370 cs=0 l=1 rev1=1 crypto rx=0x7faba800c450 tx=0x7faba800c910 comp rx=0 tx=0).stop 2023-11-26T07:18:47.228 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:47.221+0000 7fabc2aea640 1 -- 172.21.15.143:0/772530955 shutdown_connections 2023-11-26T07:18:47.228 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:47.221+0000 7fabc2aea640 1 --2- 172.21.15.143:0/772530955 >> [v2:172.21.15.143:6832/2150557544,v1:172.21.15.143:6833/2150557544] conn(0x7faba0066470 0x7faba0068930 unknown :-1 s=CLOSED pgs=201 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:18:47.228 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:47.221+0000 7fabc2aea640 1 --2- 172.21.15.143:0/772530955 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fabbc13e290 0x7fabbc159f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:18:47.229 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:47.221+0000 7fabc2aea640 1 --2- 172.21.15.143:0/772530955 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fabbc0cf990 0x7fabbc14dd20 unknown :-1 s=CLOSED pgs=370 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:18:47.229 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:47.221+0000 7fabc2aea640 1 --2- 172.21.15.143:0/772530955 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fabbc0ce7c0 0x7fabbc128620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:18:47.229 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:47.221+0000 7fabc2aea640 1 -- 172.21.15.143:0/772530955 >> 172.21.15.143:0/772530955 conn(0x7fabbc06a890 msgr2=0x7fabbc06ac70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:18:47.229 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:47.221+0000 7fabc2aea640 1 -- 172.21.15.143:0/772530955 shutdown_connections 2023-11-26T07:18:47.229 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:47.221+0000 7fabc2aea640 1 -- 172.21.15.143:0/772530955 wait complete. 2023-11-26T07:18:47.247 DEBUG:teuthology.orchestra.run.smithi143:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth del client.bootstrap-mgr 2023-11-26T07:18:47.385 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:47.381+0000 7fabd445a640 1 Processor -- start 2023-11-26T07:18:47.385 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:47.381+0000 7fabd445a640 1 -- start start 2023-11-26T07:18:47.387 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:47.381+0000 7fabd445a640 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7fabcc0cf280 0x7fabcc0cf710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:18:47.387 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:47.381+0000 7fabd445a640 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7fabcc0cd1b0 0x7fabcc0cd5c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:18:47.387 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:47.381+0000 7fabd445a640 1 -- --> v1:172.21.15.188:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fabcc0cc730 con 0x7fabcc143ac0 2023-11-26T07:18:47.387 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:47.381+0000 7fabd445a640 1 -- --> v2:172.21.15.143:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fabcc143ea0 con 0x7fabcc0cd1b0 2023-11-26T07:18:47.387 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:47.381+0000 7fabd445a640 1 -- --> v2:172.21.15.188:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fabcc0cfc50 con 0x7fabcc0cf280 2023-11-26T07:18:47.387 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:47.381+0000 7fabd29f7640 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7fabcc0cd1b0 0x7fabcc0cd5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:18:47.387 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:47.381+0000 7fabd29f7640 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7fabcc0cd1b0 0x7fabcc0cd5c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:47406/0 (socket says 172.21.15.143:47406) 2023-11-26T07:18:47.387 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:47.381+0000 7fabd29f7640 1 -- 172.21.15.143:0/921360289 learned_addr learned my addr 172.21.15.143:0/921360289 (peer_addr_for_me v2:172.21.15.143:0/0) 2023-11-26T07:18:47.388 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:47.381+0000 7fabd19f5640 1 --2- 172.21.15.143:0/921360289 >> v2:172.21.15.188:3300/0 conn(0x7fabcc0cf280 0x7fabcc0cf710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:18:47.388 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:47.381+0000 7fabd29f7640 1 -- 172.21.15.143:0/921360289 >> v2:172.21.15.188:3300/0 conn(0x7fabcc0cf280 msgr2=0x7fabcc0cf710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-26T07:18:47.388 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:47.381+0000 7fabd29f7640 1 --2- 172.21.15.143:0/921360289 >> v2:172.21.15.188:3300/0 conn(0x7fabcc0cf280 0x7fabcc0cf710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:18:47.388 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:47.381+0000 7fabd29f7640 1 -- 172.21.15.143:0/921360289 >> v1:172.21.15.188:6789/0 conn(0x7fabcc143ac0 legacy=0x7fabcc070370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:18:47.388 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:47.381+0000 7fabd29f7640 1 -- 172.21.15.143:0/921360289 --> v2:172.21.15.143:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fabcc070ee0 con 0x7fabcc0cd1b0 2023-11-26T07:18:47.390 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:47.385+0000 7fabd29f7640 1 --2- 172.21.15.143:0/921360289 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fabcc0cd1b0 0x7fabcc0cd5c0 secure :-1 s=READY pgs=371 cs=0 l=1 rev1=1 crypto rx=0x7fabcc0aeb10 tx=0x7fabc401ebd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=23d24cb7f80cfd3b server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:18:47.390 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:47.385+0000 7fabd11f4640 1 -- 172.21.15.143:0/921360289 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fabc400b2d0 con 0x7fabcc0cd1b0 2023-11-26T07:18:47.390 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:47.385+0000 7fabd11f4640 1 -- 172.21.15.143:0/921360289 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fabc400b430 con 0x7fabcc0cd1b0 2023-11-26T07:18:47.390 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:47.385+0000 7fabd11f4640 1 -- 172.21.15.143:0/921360289 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fabc40a7cc0 con 0x7fabcc0cd1b0 2023-11-26T07:18:47.391 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:47.385+0000 7fabd445a640 1 -- 172.21.15.143:0/921360289 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fabcc0cd1b0 msgr2=0x7fabcc0cd5c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:18:47.391 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:47.385+0000 7fabd445a640 1 --2- 172.21.15.143:0/921360289 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fabcc0cd1b0 0x7fabcc0cd5c0 secure :-1 s=READY pgs=371 cs=0 l=1 rev1=1 crypto rx=0x7fabcc0aeb10 tx=0x7fabc401ebd0 comp rx=0 tx=0).stop 2023-11-26T07:18:47.391 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:47.385+0000 7fabd445a640 1 -- 172.21.15.143:0/921360289 shutdown_connections 2023-11-26T07:18:47.391 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:47.385+0000 7fabd445a640 1 --2- 172.21.15.143:0/921360289 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fabcc0cd1b0 0x7fabcc0cd5c0 unknown :-1 s=CLOSED pgs=371 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:18:47.391 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:47.385+0000 7fabd445a640 1 --2- 172.21.15.143:0/921360289 >> v2:172.21.15.188:3300/0 conn(0x7fabcc0cf280 0x7fabcc0cf710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:18:47.392 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:47.385+0000 7fabd445a640 1 -- 172.21.15.143:0/921360289 >> 172.21.15.143:0/921360289 conn(0x7fabcc06a890 msgr2=0x7fabcc06aca0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:18:47.392 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:47.385+0000 7fabd445a640 1 -- 172.21.15.143:0/921360289 shutdown_connections 2023-11-26T07:18:47.392 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:47.385+0000 7fabd445a640 1 -- 172.21.15.143:0/921360289 wait complete. 2023-11-26T07:18:47.393 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:47.389+0000 7fabd445a640 1 Processor -- start 2023-11-26T07:18:47.393 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:47.389+0000 7fabd445a640 1 -- start start 2023-11-26T07:18:47.395 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:47.389+0000 7fabd445a640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fabcc0cd1b0 0x7fabcc154730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:18:47.395 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:47.389+0000 7fabd445a640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fabcc0cf280 0x7fabcc13d9a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:18:47.395 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:47.389+0000 7fabd445a640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fabcc143ac0 0x7fabcc134f90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:18:47.395 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:47.389+0000 7fabd445a640 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fabcc154c70 con 0x7fabcc0cf280 2023-11-26T07:18:47.395 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:47.389+0000 7fabd445a640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fabcc158af0 con 0x7fabcc0cd1b0 2023-11-26T07:18:47.395 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:47.389+0000 7fabd445a640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fabcc158dd0 con 0x7fabcc143ac0 2023-11-26T07:18:47.395 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:47.389+0000 7fabd19f5640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fabcc0cf280 0x7fabcc13d9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:18:47.395 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:47.389+0000 7fabd19f5640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fabcc0cf280 0x7fabcc13d9a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:47414/0 (socket says 172.21.15.143:47414) 2023-11-26T07:18:47.396 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:47.389+0000 7fabd19f5640 1 -- 172.21.15.143:0/1060689419 learned_addr learned my addr 172.21.15.143:0/1060689419 (peer_addr_for_me v2:172.21.15.143:0/0) 2023-11-26T07:18:47.396 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:47.389+0000 7fabd29f7640 1 --2- 172.21.15.143:0/1060689419 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fabcc143ac0 0x7fabcc134f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:18:47.396 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:47.389+0000 7fabd21f6640 1 --2- 172.21.15.143:0/1060689419 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fabcc0cd1b0 0x7fabcc154730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:18:47.396 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:47.389+0000 7fabd19f5640 1 -- 172.21.15.143:0/1060689419 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fabcc143ac0 msgr2=0x7fabcc134f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:18:47.396 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:47.389+0000 7fabd19f5640 1 --2- 172.21.15.143:0/1060689419 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fabcc143ac0 0x7fabcc134f90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:18:47.396 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:47.389+0000 7fabd19f5640 1 -- 172.21.15.143:0/1060689419 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fabcc0cd1b0 msgr2=0x7fabcc154730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:18:47.396 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:47.389+0000 7fabd19f5640 1 --2- 172.21.15.143:0/1060689419 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fabcc0cd1b0 0x7fabcc154730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:18:47.396 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:47.389+0000 7fabd19f5640 1 -- 172.21.15.143:0/1060689419 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fabc400bd40 con 0x7fabcc0cf280 2023-11-26T07:18:47.396 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:47.389+0000 7fabd29f7640 1 --2- 172.21.15.143:0/1060689419 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fabcc143ac0 0x7fabcc134f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-26T07:18:47.396 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:47.389+0000 7fabd21f6640 1 --2- 172.21.15.143:0/1060689419 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fabcc0cd1b0 0x7fabcc154730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-26T07:18:47.396 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:47.389+0000 7fabd19f5640 1 --2- 172.21.15.143:0/1060689419 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fabcc0cf280 0x7fabcc13d9a0 secure :-1 s=READY pgs=372 cs=0 l=1 rev1=1 crypto rx=0x7fabbc00d5f0 tx=0x7fabbc00dab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:18:47.396 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:47.389+0000 7fabc2ffd640 1 -- 172.21.15.143:0/1060689419 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fabbc0125f0 con 0x7fabcc0cf280 2023-11-26T07:18:47.397 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:47.389+0000 7fabc2ffd640 1 -- 172.21.15.143:0/1060689419 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fabbc012750 con 0x7fabcc0cf280 2023-11-26T07:18:47.397 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:47.389+0000 7fabc2ffd640 1 -- 172.21.15.143:0/1060689419 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fabbc012ae0 con 0x7fabcc0cf280 2023-11-26T07:18:47.397 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:47.389+0000 7fabd445a640 1 -- 172.21.15.143:0/1060689419 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fabcc1590b0 con 0x7fabcc0cf280 2023-11-26T07:18:47.397 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:47.389+0000 7fabd445a640 1 -- 172.21.15.143:0/1060689419 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fabcc13dee0 con 0x7fabcc0cf280 2023-11-26T07:18:47.397 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:47.389+0000 7fabd445a640 1 -- 172.21.15.143:0/1060689419 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fabcc067680 con 0x7fabcc0cf280 2023-11-26T07:18:47.397 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:47.393+0000 7fabc2ffd640 1 -- 172.21.15.143:0/1060689419 <== mon.0 v2:172.21.15.143:3300/0 4 ==== mgrmap(e 5) v1 ==== 85852+0+0 (secure 0 0 0) 0x7fabbc012c40 con 0x7fabcc0cf280 2023-11-26T07:18:47.398 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:47.393+0000 7fabc2ffd640 1 --2- 172.21.15.143:0/1060689419 >> [v2:172.21.15.143:6832/2150557544,v1:172.21.15.143:6833/2150557544] conn(0x7fabb0066470 0x7fabb0068930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:18:47.398 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:47.393+0000 7fabc2ffd640 1 -- 172.21.15.143:0/1060689419 <== mon.0 v2:172.21.15.143:3300/0 5 ==== osd_map(29..29 src has 1..29) v4 ==== 4857+0+0 (secure 0 0 0) 0x7fabbc0840e0 con 0x7fabcc0cf280 2023-11-26T07:18:47.398 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:47.393+0000 7fabd21f6640 1 --2- 172.21.15.143:0/1060689419 >> [v2:172.21.15.143:6832/2150557544,v1:172.21.15.143:6833/2150557544] conn(0x7fabb0066470 0x7fabb0068930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:18:47.398 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:47.393+0000 7fabd21f6640 1 --2- 172.21.15.143:0/1060689419 >> [v2:172.21.15.143:6832/2150557544,v1:172.21.15.143:6833/2150557544] conn(0x7fabb0066470 0x7fabb0068930 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7fabc8000f70 tx=0x7fabc8009040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:18:47.401 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:47.397+0000 7fabc2ffd640 1 -- 172.21.15.143:0/1060689419 <== mon.0 v2:172.21.15.143:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172649 (secure 0 0 0) 0x7fabbc051800 con 0x7fabcc0cf280 2023-11-26T07:18:47.609 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:47.605+0000 7fabd445a640 1 -- 172.21.15.143:0/1060689419 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "auth del", "entity": "client.bootstrap-mgr"} v 0) v1 -- 0x7fabcc06e1f0 con 0x7fabcc0cf280 2023-11-26T07:18:47.611 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:47.605+0000 7fabc2ffd640 1 -- 172.21.15.143:0/1060689419 <== mon.0 v2:172.21.15.143:3300/0 7 ==== mon_command_ack([{"prefix": "auth del", "entity": "client.bootstrap-mgr"}]=0 v8) v1 ==== 90+0+0 (secure 0 0 0) 0x7fabbc054e20 con 0x7fabcc0cf280 2023-11-26T07:18:47.615 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:47.609+0000 7fabd445a640 1 -- 172.21.15.143:0/1060689419 >> [v2:172.21.15.143:6832/2150557544,v1:172.21.15.143:6833/2150557544] conn(0x7fabb0066470 msgr2=0x7fabb0068930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:18:47.615 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:47.609+0000 7fabd445a640 1 --2- 172.21.15.143:0/1060689419 >> [v2:172.21.15.143:6832/2150557544,v1:172.21.15.143:6833/2150557544] conn(0x7fabb0066470 0x7fabb0068930 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7fabc8000f70 tx=0x7fabc8009040 comp rx=0 tx=0).stop 2023-11-26T07:18:47.615 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:47.609+0000 7fabd445a640 1 -- 172.21.15.143:0/1060689419 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fabcc0cf280 msgr2=0x7fabcc13d9a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:18:47.615 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:47.609+0000 7fabd445a640 1 --2- 172.21.15.143:0/1060689419 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fabcc0cf280 0x7fabcc13d9a0 secure :-1 s=READY pgs=372 cs=0 l=1 rev1=1 crypto rx=0x7fabbc00d5f0 tx=0x7fabbc00dab0 comp rx=0 tx=0).stop 2023-11-26T07:18:47.615 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:47.609+0000 7fabd445a640 1 -- 172.21.15.143:0/1060689419 shutdown_connections 2023-11-26T07:18:47.615 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:47.609+0000 7fabd445a640 1 --2- 172.21.15.143:0/1060689419 >> [v2:172.21.15.143:6832/2150557544,v1:172.21.15.143:6833/2150557544] conn(0x7fabb0066470 0x7fabb0068930 unknown :-1 s=CLOSED pgs=202 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:18:47.615 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:47.609+0000 7fabd445a640 1 --2- 172.21.15.143:0/1060689419 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fabcc143ac0 0x7fabcc134f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:18:47.616 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:47.609+0000 7fabd445a640 1 --2- 172.21.15.143:0/1060689419 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fabcc0cf280 0x7fabcc13d9a0 unknown :-1 s=CLOSED pgs=372 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:18:47.616 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:47.609+0000 7fabd445a640 1 --2- 172.21.15.143:0/1060689419 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fabcc0cd1b0 0x7fabcc154730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:18:47.616 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:47.609+0000 7fabd445a640 1 -- 172.21.15.143:0/1060689419 >> 172.21.15.143:0/1060689419 conn(0x7fabcc06a890 msgr2=0x7fabcc15a600 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:18:47.616 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:47.609+0000 7fabd445a640 1 -- 172.21.15.143:0/1060689419 shutdown_connections 2023-11-26T07:18:47.616 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:47.609+0000 7fabd445a640 1 -- 172.21.15.143:0/1060689419 wait complete. 2023-11-26T07:18:47.634 DEBUG:teuthology.orchestra.run.smithi143:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth del client.bootstrap-osd 2023-11-26T07:18:47.772 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:47.765+0000 7faa45365640 1 Processor -- start 2023-11-26T07:18:47.772 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:47.765+0000 7faa45365640 1 -- start start 2023-11-26T07:18:47.773 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:47.769+0000 7faa45365640 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7faa4015d6e0 0x7faa400d0df0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:18:47.774 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:47.769+0000 7faa45365640 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7faa400d1bc0 0x7faa400b0750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:18:47.774 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:47.769+0000 7faa45365640 1 -- --> v1:172.21.15.188:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7faa400cf0e0 con 0x7faa400b2160 2023-11-26T07:18:47.774 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:47.769+0000 7faa45365640 1 -- --> v2:172.21.15.143:3300/0 -- mon_getmap magic: 0 v1 -- 0x7faa4015dac0 con 0x7faa400d1bc0 2023-11-26T07:18:47.774 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:47.769+0000 7faa45365640 1 -- --> v2:172.21.15.188:3300/0 -- mon_getmap magic: 0 v1 -- 0x7faa400ceb20 con 0x7faa4015d6e0 2023-11-26T07:18:47.774 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:47.769+0000 7faa3f7fe640 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7faa400d1bc0 0x7faa400b0750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:18:47.774 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:47.769+0000 7faa3f7fe640 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7faa400d1bc0 0x7faa400b0750 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:47416/0 (socket says 172.21.15.143:47416) 2023-11-26T07:18:47.774 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:47.769+0000 7faa3f7fe640 1 -- 172.21.15.143:0/3278903768 learned_addr learned my addr 172.21.15.143:0/3278903768 (peer_addr_for_me v2:172.21.15.143:0/0) 2023-11-26T07:18:47.775 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:47.769+0000 7faa3effd640 1 --2- 172.21.15.143:0/3278903768 >> v2:172.21.15.188:3300/0 conn(0x7faa4015d6e0 0x7faa400d0df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:18:47.775 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:47.769+0000 7faa3f7fe640 1 -- 172.21.15.143:0/3278903768 >> v2:172.21.15.188:3300/0 conn(0x7faa4015d6e0 msgr2=0x7faa400d0df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-26T07:18:47.775 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:47.769+0000 7faa3f7fe640 1 --2- 172.21.15.143:0/3278903768 >> v2:172.21.15.188:3300/0 conn(0x7faa4015d6e0 0x7faa400d0df0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:18:47.775 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:47.769+0000 7faa3f7fe640 1 -- 172.21.15.143:0/3278903768 >> v1:172.21.15.188:6789/0 conn(0x7faa400b2160 legacy=0x7faa400d14b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-26T07:18:47.775 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:47.769+0000 7faa3f7fe640 1 -- 172.21.15.143:0/3278903768 --> v2:172.21.15.143:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7faa400cee00 con 0x7faa400d1bc0 2023-11-26T07:18:47.775 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:47.769+0000 7faa3effd640 1 --2- 172.21.15.143:0/3278903768 >> v2:172.21.15.188:3300/0 conn(0x7faa4015d6e0 0x7faa400d0df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-26T07:18:47.777 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:47.773+0000 7faa3f7fe640 1 --2- 172.21.15.143:0/3278903768 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7faa400d1bc0 0x7faa400b0750 secure :-1 s=READY pgs=373 cs=0 l=1 rev1=1 crypto rx=0x7faa400b5db0 tx=0x7faa3801c990 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2b664bb1644e65a0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:18:47.777 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:47.773+0000 7faa3dffb640 1 -- 172.21.15.143:0/3278903768 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faa380a5740 con 0x7faa400d1bc0 2023-11-26T07:18:47.777 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:47.773+0000 7faa3dffb640 1 -- 172.21.15.143:0/3278903768 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7faa380a58a0 con 0x7faa400d1bc0 2023-11-26T07:18:47.778 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:47.773+0000 7faa3dffb640 1 -- 172.21.15.143:0/3278903768 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faa380a5c30 con 0x7faa400d1bc0 2023-11-26T07:18:47.778 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:47.773+0000 7faa45365640 1 -- 172.21.15.143:0/3278903768 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7faa400d1bc0 msgr2=0x7faa400b0750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:18:47.778 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:47.773+0000 7faa45365640 1 --2- 172.21.15.143:0/3278903768 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7faa400d1bc0 0x7faa400b0750 secure :-1 s=READY pgs=373 cs=0 l=1 rev1=1 crypto rx=0x7faa400b5db0 tx=0x7faa3801c990 comp rx=0 tx=0).stop 2023-11-26T07:18:47.779 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:47.773+0000 7faa45365640 1 -- 172.21.15.143:0/3278903768 shutdown_connections 2023-11-26T07:18:47.779 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:47.773+0000 7faa45365640 1 --2- 172.21.15.143:0/3278903768 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7faa400d1bc0 0x7faa400b0750 unknown :-1 s=CLOSED pgs=373 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:18:47.779 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:47.773+0000 7faa45365640 1 --2- 172.21.15.143:0/3278903768 >> v2:172.21.15.188:3300/0 conn(0x7faa4015d6e0 0x7faa400d0df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:18:47.779 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:47.773+0000 7faa45365640 1 -- 172.21.15.143:0/3278903768 >> 172.21.15.143:0/3278903768 conn(0x7faa4009c550 msgr2=0x7faa400d3220 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:18:47.779 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:47.773+0000 7faa45365640 1 -- 172.21.15.143:0/3278903768 shutdown_connections 2023-11-26T07:18:47.779 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:47.773+0000 7faa45365640 1 -- 172.21.15.143:0/3278903768 wait complete. 2023-11-26T07:18:47.780 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:47.773+0000 7faa45365640 1 Processor -- start 2023-11-26T07:18:47.780 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:47.773+0000 7faa45365640 1 -- start start 2023-11-26T07:18:47.783 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:47.777+0000 7faa45365640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7faa400b2160 0x7faa40158c80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:18:47.783 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:47.777+0000 7faa45365640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7faa400d1bc0 0x7faa400b3bb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:18:47.783 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:47.777+0000 7faa45365640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7faa4015d6e0 0x7faa400ca2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:18:47.783 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:47.777+0000 7faa45365640 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faa401578e0 con 0x7faa400d1bc0 2023-11-26T07:18:47.783 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:47.777+0000 7faa45365640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faa40155a20 con 0x7faa400b2160 2023-11-26T07:18:47.783 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:47.777+0000 7faa45365640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7faa40157b30 con 0x7faa4015d6e0 2023-11-26T07:18:47.783 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:47.777+0000 7faa3e7fc640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7faa400d1bc0 0x7faa400b3bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:18:47.783 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:47.777+0000 7faa3e7fc640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7faa400d1bc0 0x7faa400b3bb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:47424/0 (socket says 172.21.15.143:47424) 2023-11-26T07:18:47.783 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:47.777+0000 7faa3e7fc640 1 -- 172.21.15.143:0/3977787570 learned_addr learned my addr 172.21.15.143:0/3977787570 (peer_addr_for_me v2:172.21.15.143:0/0) 2023-11-26T07:18:47.783 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:47.777+0000 7faa3effd640 1 --2- 172.21.15.143:0/3977787570 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7faa400b2160 0x7faa40158c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:18:47.784 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:47.777+0000 7faa3f7fe640 1 --2- 172.21.15.143:0/3977787570 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7faa4015d6e0 0x7faa400ca2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:18:47.784 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:47.777+0000 7faa3e7fc640 1 -- 172.21.15.143:0/3977787570 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7faa4015d6e0 msgr2=0x7faa400ca2f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:18:47.784 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:47.777+0000 7faa3e7fc640 1 --2- 172.21.15.143:0/3977787570 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7faa4015d6e0 0x7faa400ca2f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:18:47.784 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:47.777+0000 7faa3e7fc640 1 -- 172.21.15.143:0/3977787570 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7faa400b2160 msgr2=0x7faa40158c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:18:47.784 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:47.777+0000 7faa3e7fc640 1 --2- 172.21.15.143:0/3977787570 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7faa400b2160 0x7faa40158c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:18:47.784 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:47.777+0000 7faa3e7fc640 1 -- 172.21.15.143:0/3977787570 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7faa38009b30 con 0x7faa400d1bc0 2023-11-26T07:18:47.784 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:47.777+0000 7faa3f7fe640 1 --2- 172.21.15.143:0/3977787570 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7faa4015d6e0 0x7faa400ca2f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-26T07:18:47.784 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:47.777+0000 7faa3e7fc640 1 --2- 172.21.15.143:0/3977787570 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7faa400d1bc0 0x7faa400b3bb0 secure :-1 s=READY pgs=374 cs=0 l=1 rev1=1 crypto rx=0x7faa3400c450 tx=0x7faa3400c910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:18:47.784 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:47.777+0000 7faa277fe640 1 -- 172.21.15.143:0/3977787570 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faa34012830 con 0x7faa400d1bc0 2023-11-26T07:18:47.784 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:47.777+0000 7faa277fe640 1 -- 172.21.15.143:0/3977787570 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7faa340129b0 con 0x7faa400d1bc0 2023-11-26T07:18:47.784 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:47.777+0000 7faa45365640 1 -- 172.21.15.143:0/3977787570 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7faa40157e10 con 0x7faa400d1bc0 2023-11-26T07:18:47.784 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:47.777+0000 7faa277fe640 1 -- 172.21.15.143:0/3977787570 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faa34012d60 con 0x7faa400d1bc0 2023-11-26T07:18:47.785 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:47.777+0000 7faa45365640 1 -- 172.21.15.143:0/3977787570 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7faa40154640 con 0x7faa400d1bc0 2023-11-26T07:18:47.785 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:47.777+0000 7faa45365640 1 -- 172.21.15.143:0/3977787570 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7faa40158050 con 0x7faa400d1bc0 2023-11-26T07:18:47.785 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:47.777+0000 7faa277fe640 1 -- 172.21.15.143:0/3977787570 <== mon.0 v2:172.21.15.143:3300/0 4 ==== mgrmap(e 5) v1 ==== 85852+0+0 (secure 0 0 0) 0x7faa3401c460 con 0x7faa400d1bc0 2023-11-26T07:18:47.785 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:47.781+0000 7faa277fe640 1 --2- 172.21.15.143:0/3977787570 >> [v2:172.21.15.143:6832/2150557544,v1:172.21.15.143:6833/2150557544] conn(0x7faa2c066470 0x7faa2c068930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:18:47.786 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:47.781+0000 7faa277fe640 1 -- 172.21.15.143:0/3977787570 <== mon.0 v2:172.21.15.143:3300/0 5 ==== osd_map(29..29 src has 1..29) v4 ==== 4857+0+0 (secure 0 0 0) 0x7faa34083930 con 0x7faa400d1bc0 2023-11-26T07:18:47.786 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:47.781+0000 7faa3effd640 1 --2- 172.21.15.143:0/3977787570 >> [v2:172.21.15.143:6832/2150557544,v1:172.21.15.143:6833/2150557544] conn(0x7faa2c066470 0x7faa2c068930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:18:47.786 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:47.781+0000 7faa3effd640 1 --2- 172.21.15.143:0/3977787570 >> [v2:172.21.15.143:6832/2150557544,v1:172.21.15.143:6833/2150557544] conn(0x7faa2c066470 0x7faa2c068930 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7faa4015e630 tx=0x7faa30008040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:18:47.789 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:47.785+0000 7faa277fe640 1 -- 172.21.15.143:0/3977787570 <== mon.0 v2:172.21.15.143:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172649 (secure 0 0 0) 0x7faa40158050 con 0x7faa400d1bc0 2023-11-26T07:18:47.994 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:47.989+0000 7faa45365640 1 -- 172.21.15.143:0/3977787570 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "auth del", "entity": "client.bootstrap-osd"} v 0) v1 -- 0x7faa401541c0 con 0x7faa400d1bc0 2023-11-26T07:18:47.997 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:47.993+0000 7faa277fe640 1 -- 172.21.15.143:0/3977787570 <== mon.0 v2:172.21.15.143:3300/0 7 ==== mon_command_ack([{"prefix": "auth del", "entity": "client.bootstrap-osd"}]=0 v9) v1 ==== 90+0+0 (secure 0 0 0) 0x7faa401541c0 con 0x7faa400d1bc0 2023-11-26T07:18:48.000 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:47.993+0000 7faa45365640 1 -- 172.21.15.143:0/3977787570 >> [v2:172.21.15.143:6832/2150557544,v1:172.21.15.143:6833/2150557544] conn(0x7faa2c066470 msgr2=0x7faa2c068930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:18:48.000 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:47.993+0000 7faa45365640 1 --2- 172.21.15.143:0/3977787570 >> [v2:172.21.15.143:6832/2150557544,v1:172.21.15.143:6833/2150557544] conn(0x7faa2c066470 0x7faa2c068930 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7faa4015e630 tx=0x7faa30008040 comp rx=0 tx=0).stop 2023-11-26T07:18:48.000 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:47.993+0000 7faa45365640 1 -- 172.21.15.143:0/3977787570 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7faa400d1bc0 msgr2=0x7faa400b3bb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:18:48.000 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:47.993+0000 7faa45365640 1 --2- 172.21.15.143:0/3977787570 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7faa400d1bc0 0x7faa400b3bb0 secure :-1 s=READY pgs=374 cs=0 l=1 rev1=1 crypto rx=0x7faa3400c450 tx=0x7faa3400c910 comp rx=0 tx=0).stop 2023-11-26T07:18:48.000 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:47.993+0000 7faa45365640 1 -- 172.21.15.143:0/3977787570 shutdown_connections 2023-11-26T07:18:48.000 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:47.993+0000 7faa45365640 1 --2- 172.21.15.143:0/3977787570 >> [v2:172.21.15.143:6832/2150557544,v1:172.21.15.143:6833/2150557544] conn(0x7faa2c066470 0x7faa2c068930 unknown :-1 s=CLOSED pgs=206 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:18:48.000 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:47.993+0000 7faa45365640 1 --2- 172.21.15.143:0/3977787570 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7faa4015d6e0 0x7faa400ca2f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:18:48.000 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:47.993+0000 7faa45365640 1 --2- 172.21.15.143:0/3977787570 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7faa400d1bc0 0x7faa400b3bb0 unknown :-1 s=CLOSED pgs=374 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:18:48.000 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:47.993+0000 7faa45365640 1 --2- 172.21.15.143:0/3977787570 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7faa400b2160 0x7faa40158c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:18:48.001 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:47.993+0000 7faa45365640 1 -- 172.21.15.143:0/3977787570 >> 172.21.15.143:0/3977787570 conn(0x7faa4009c550 msgr2=0x7faa40160220 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:18:48.001 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:47.993+0000 7faa45365640 1 -- 172.21.15.143:0/3977787570 shutdown_connections 2023-11-26T07:18:48.001 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:47.993+0000 7faa45365640 1 -- 172.21.15.143:0/3977787570 wait complete. 2023-11-26T07:18:48.019 DEBUG:teuthology.orchestra.run.smithi143:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth del client.bootstrap-rbd 2023-11-26T07:18:48.157 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:48.153+0000 7f50b8d1d640 1 Processor -- start 2023-11-26T07:18:48.158 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:48.153+0000 7f50b8d1d640 1 -- start start 2023-11-26T07:18:48.159 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:48.153+0000 7f50b8d1d640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f50b4143ac0 0x7f50b4070370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:18:48.159 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:48.153+0000 7f50b8d1d640 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7f50b40708b0 0x7f50b40cf280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:18:48.159 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:48.153+0000 7f50b8d1d640 1 -- --> v1:172.21.15.188:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f50b40cfbf0 con 0x7f50b40cf7c0 2023-11-26T07:18:48.159 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:48.153+0000 7f50b8d1d640 1 -- --> v2:172.21.15.188:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f50b4143ea0 con 0x7f50b40708b0 2023-11-26T07:18:48.159 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:48.153+0000 7f50b8d1d640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f50b40cc730 con 0x7f50b4143ac0 2023-11-26T07:18:48.160 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:48.153+0000 7f50a9d9b640 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7f50b40708b0 0x7f50b40cf280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:18:48.160 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:48.153+0000 7f50b259c640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f50b4143ac0 0x7f50b4070370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:18:48.160 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:48.153+0000 7f50a9d9b640 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7f50b40708b0 0x7f50b40cf280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.188:3300/0 says I am v2:172.21.15.143:59880/0 (socket says 172.21.15.143:59880) 2023-11-26T07:18:48.160 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:48.153+0000 7f50a9d9b640 1 -- 172.21.15.143:0/2234826399 learned_addr learned my addr 172.21.15.143:0/2234826399 (peer_addr_for_me v2:172.21.15.143:0/0) 2023-11-26T07:18:48.160 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:48.153+0000 7f50b2d9d640 1 --1- 172.21.15.143:0/2234826399 >> v1:172.21.15.188:6789/0 conn(0x7f50b40cf7c0 0x7f50b40b1ef0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.188:6789/0 says I am v1:172.21.15.143:52176/0 (socket says 172.21.15.143:52176) 2023-11-26T07:18:48.160 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:48.153+0000 7f50a9d9b640 1 -- 172.21.15.143:0/2234826399 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f50b4143ac0 msgr2=0x7f50b4070370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-26T07:18:48.161 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:48.153+0000 7f50a9d9b640 1 --2- 172.21.15.143:0/2234826399 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f50b4143ac0 0x7f50b4070370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:18:48.161 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:48.153+0000 7f50a9d9b640 1 -- 172.21.15.143:0/2234826399 >> v1:172.21.15.188:6789/0 conn(0x7f50b40cf7c0 legacy=0x7f50b40b1ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:18:48.161 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:48.153+0000 7f50a9d9b640 1 -- 172.21.15.143:0/2234826399 --> v2:172.21.15.188:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f50b4070ee0 con 0x7f50b40708b0 2023-11-26T07:18:48.162 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:48.157+0000 7f50a9d9b640 1 --2- 172.21.15.143:0/2234826399 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f50b40708b0 0x7f50b40cf280 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7f509c009e50 tx=0x7f509c01ca10 comp rx=0 tx=0).ready entity=mon.1 client_cookie=3aac958546674893 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:18:48.163 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:48.157+0000 7f50b1d9b640 1 -- 172.21.15.143:0/2234826399 <== mon.1 v2:172.21.15.188:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f509c0a57f0 con 0x7f50b40708b0 2023-11-26T07:18:48.163 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:48.157+0000 7f50b1d9b640 1 -- 172.21.15.143:0/2234826399 <== mon.1 v2:172.21.15.188:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f509c0a5950 con 0x7f50b40708b0 2023-11-26T07:18:48.163 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:48.157+0000 7f50b1d9b640 1 -- 172.21.15.143:0/2234826399 <== mon.1 v2:172.21.15.188:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f509c0a5ce0 con 0x7f50b40708b0 2023-11-26T07:18:48.164 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:48.157+0000 7f50b8d1d640 1 -- 172.21.15.143:0/2234826399 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f50b40708b0 msgr2=0x7f50b40cf280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:18:48.164 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:48.157+0000 7f50b8d1d640 1 --2- 172.21.15.143:0/2234826399 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f50b40708b0 0x7f50b40cf280 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7f509c009e50 tx=0x7f509c01ca10 comp rx=0 tx=0).stop 2023-11-26T07:18:48.164 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:48.157+0000 7f50b8d1d640 1 -- 172.21.15.143:0/2234826399 shutdown_connections 2023-11-26T07:18:48.164 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:48.157+0000 7f50b8d1d640 1 --2- 172.21.15.143:0/2234826399 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f50b40708b0 0x7f50b40cf280 unknown :-1 s=CLOSED pgs=178 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:18:48.164 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:48.157+0000 7f50b8d1d640 1 --2- 172.21.15.143:0/2234826399 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f50b4143ac0 0x7f50b4070370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:18:48.164 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:48.157+0000 7f50b8d1d640 1 -- 172.21.15.143:0/2234826399 >> 172.21.15.143:0/2234826399 conn(0x7f50b406a890 msgr2=0x7f50b406aca0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:18:48.164 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:48.157+0000 7f50b8d1d640 1 -- 172.21.15.143:0/2234826399 shutdown_connections 2023-11-26T07:18:48.164 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:48.157+0000 7f50b8d1d640 1 -- 172.21.15.143:0/2234826399 wait complete. 2023-11-26T07:18:48.166 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:48.161+0000 7f50b8d1d640 1 Processor -- start 2023-11-26T07:18:48.166 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:48.161+0000 7f50b8d1d640 1 -- start start 2023-11-26T07:18:48.167 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:48.161+0000 7f50b8d1d640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f50b40708b0 0x7f50b41320e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:18:48.167 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:48.161+0000 7f50b8d1d640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f50b40cf7c0 0x7f50b413bb90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:18:48.167 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:48.161+0000 7f50b8d1d640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f50b4143ac0 0x7f50b413c0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:18:48.167 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:48.161+0000 7f50b8d1d640 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f50b4157010 con 0x7f50b40708b0 2023-11-26T07:18:48.168 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:48.161+0000 7f50b8d1d640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f50b4132620 con 0x7f50b4143ac0 2023-11-26T07:18:48.168 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:48.161+0000 7f50b8d1d640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f50b4134110 con 0x7f50b40cf7c0 2023-11-26T07:18:48.168 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:48.161+0000 7f50b259c640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f50b40708b0 0x7f50b41320e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:18:48.168 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:48.161+0000 7f50b259c640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f50b40708b0 0x7f50b41320e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:47428/0 (socket says 172.21.15.143:47428) 2023-11-26T07:18:48.168 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:48.161+0000 7f50b259c640 1 -- 172.21.15.143:0/2146506376 learned_addr learned my addr 172.21.15.143:0/2146506376 (peer_addr_for_me v2:172.21.15.143:0/0) 2023-11-26T07:18:48.168 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:48.161+0000 7f50a9d9b640 1 --2- 172.21.15.143:0/2146506376 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f50b40cf7c0 0x7f50b413bb90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:18:48.168 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:48.161+0000 7f50b2d9d640 1 --2- 172.21.15.143:0/2146506376 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f50b4143ac0 0x7f50b413c0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:18:48.168 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:48.161+0000 7f50b259c640 1 -- 172.21.15.143:0/2146506376 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f50b40cf7c0 msgr2=0x7f50b413bb90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:18:48.168 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:48.161+0000 7f50b259c640 1 --2- 172.21.15.143:0/2146506376 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f50b40cf7c0 0x7f50b413bb90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:18:48.168 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:48.161+0000 7f50b259c640 1 -- 172.21.15.143:0/2146506376 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f50b4143ac0 msgr2=0x7f50b413c0d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:18:48.168 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:48.161+0000 7f50b259c640 1 --2- 172.21.15.143:0/2146506376 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f50b4143ac0 0x7f50b413c0d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:18:48.168 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:48.161+0000 7f50b259c640 1 -- 172.21.15.143:0/2146506376 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f509c009b30 con 0x7f50b40708b0 2023-11-26T07:18:48.168 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:48.161+0000 7f50b2d9d640 1 --2- 172.21.15.143:0/2146506376 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f50b4143ac0 0x7f50b413c0d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-26T07:18:48.169 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:48.161+0000 7f50b259c640 1 --2- 172.21.15.143:0/2146506376 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f50b40708b0 0x7f50b41320e0 secure :-1 s=READY pgs=375 cs=0 l=1 rev1=1 crypto rx=0x7f50a400e870 tx=0x7f50a400ec70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:18:48.169 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:48.161+0000 7f50ab7fe640 1 -- 172.21.15.143:0/2146506376 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f50a4009510 con 0x7f50b40708b0 2023-11-26T07:18:48.170 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:48.161+0000 7f50ab7fe640 1 -- 172.21.15.143:0/2146506376 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f50a400eec0 con 0x7f50b40708b0 2023-11-26T07:18:48.170 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:48.161+0000 7f50b8d1d640 1 -- 172.21.15.143:0/2146506376 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f50b4156110 con 0x7f50b40708b0 2023-11-26T07:18:48.170 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:48.161+0000 7f50ab7fe640 1 -- 172.21.15.143:0/2146506376 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f50a400cb40 con 0x7f50b40708b0 2023-11-26T07:18:48.170 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:48.161+0000 7f50b8d1d640 1 -- 172.21.15.143:0/2146506376 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f50b4156670 con 0x7f50b40708b0 2023-11-26T07:18:48.170 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:48.165+0000 7f50b8d1d640 1 -- 172.21.15.143:0/2146506376 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f50b4067680 con 0x7f50b40708b0 2023-11-26T07:18:48.170 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:48.165+0000 7f50ab7fe640 1 -- 172.21.15.143:0/2146506376 <== mon.0 v2:172.21.15.143:3300/0 4 ==== mgrmap(e 5) v1 ==== 85852+0+0 (secure 0 0 0) 0x7f50a400cca0 con 0x7f50b40708b0 2023-11-26T07:18:48.171 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:48.165+0000 7f50ab7fe640 1 --2- 172.21.15.143:0/2146506376 >> [v2:172.21.15.143:6832/2150557544,v1:172.21.15.143:6833/2150557544] conn(0x7f5094066470 0x7f5094068930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:18:48.171 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:48.165+0000 7f50ab7fe640 1 -- 172.21.15.143:0/2146506376 <== mon.0 v2:172.21.15.143:3300/0 5 ==== osd_map(29..29 src has 1..29) v4 ==== 4857+0+0 (secure 0 0 0) 0x7f50a4083c10 con 0x7f50b40708b0 2023-11-26T07:18:48.171 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:48.165+0000 7f50a9d9b640 1 --2- 172.21.15.143:0/2146506376 >> [v2:172.21.15.143:6832/2150557544,v1:172.21.15.143:6833/2150557544] conn(0x7f5094066470 0x7f5094068930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:18:48.171 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:48.165+0000 7f50a9d9b640 1 --2- 172.21.15.143:0/2146506376 >> [v2:172.21.15.143:6832/2150557544,v1:172.21.15.143:6833/2150557544] conn(0x7f5094066470 0x7f5094068930 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7f509c0ae000 tx=0x7f509c0a7070 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:18:48.174 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:48.169+0000 7f50ab7fe640 1 -- 172.21.15.143:0/2146506376 <== mon.0 v2:172.21.15.143:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172649 (secure 0 0 0) 0x7f50a4051480 con 0x7f50b40708b0 2023-11-26T07:18:48.380 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:48.373+0000 7f50b8d1d640 1 -- 172.21.15.143:0/2146506376 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "auth del", "entity": "client.bootstrap-rbd"} v 0) v1 -- 0x7f50b4067f30 con 0x7f50b40708b0 2023-11-26T07:18:48.383 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:48.377+0000 7f50ab7fe640 1 -- 172.21.15.143:0/2146506376 <== mon.0 v2:172.21.15.143:3300/0 7 ==== mon_command_ack([{"prefix": "auth del", "entity": "client.bootstrap-rbd"}]=0 v10) v1 ==== 90+0+0 (secure 0 0 0) 0x7f50a4054aa0 con 0x7f50b40708b0 2023-11-26T07:18:48.385 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:48.381+0000 7f50b8d1d640 1 -- 172.21.15.143:0/2146506376 >> [v2:172.21.15.143:6832/2150557544,v1:172.21.15.143:6833/2150557544] conn(0x7f5094066470 msgr2=0x7f5094068930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:18:48.386 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:48.381+0000 7f50b8d1d640 1 --2- 172.21.15.143:0/2146506376 >> [v2:172.21.15.143:6832/2150557544,v1:172.21.15.143:6833/2150557544] conn(0x7f5094066470 0x7f5094068930 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7f509c0ae000 tx=0x7f509c0a7070 comp rx=0 tx=0).stop 2023-11-26T07:18:48.386 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:48.381+0000 7f50b8d1d640 1 -- 172.21.15.143:0/2146506376 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f50b40708b0 msgr2=0x7f50b41320e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:18:48.386 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:48.381+0000 7f50b8d1d640 1 --2- 172.21.15.143:0/2146506376 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f50b40708b0 0x7f50b41320e0 secure :-1 s=READY pgs=375 cs=0 l=1 rev1=1 crypto rx=0x7f50a400e870 tx=0x7f50a400ec70 comp rx=0 tx=0).stop 2023-11-26T07:18:48.386 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:48.381+0000 7f50b8d1d640 1 -- 172.21.15.143:0/2146506376 shutdown_connections 2023-11-26T07:18:48.386 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:48.381+0000 7f50b8d1d640 1 --2- 172.21.15.143:0/2146506376 >> [v2:172.21.15.143:6832/2150557544,v1:172.21.15.143:6833/2150557544] conn(0x7f5094066470 0x7f5094068930 unknown :-1 s=CLOSED pgs=207 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:18:48.386 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:48.381+0000 7f50b8d1d640 1 --2- 172.21.15.143:0/2146506376 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f50b4143ac0 0x7f50b413c0d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:18:48.386 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:48.381+0000 7f50b8d1d640 1 --2- 172.21.15.143:0/2146506376 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f50b40cf7c0 0x7f50b413bb90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:18:48.386 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:48.381+0000 7f50b8d1d640 1 --2- 172.21.15.143:0/2146506376 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f50b40708b0 0x7f50b41320e0 unknown :-1 s=CLOSED pgs=375 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:18:48.386 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:48.381+0000 7f50b8d1d640 1 -- 172.21.15.143:0/2146506376 >> 172.21.15.143:0/2146506376 conn(0x7f50b406a890 msgr2=0x7f50b413dee0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:18:48.386 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:48.381+0000 7f50b8d1d640 1 -- 172.21.15.143:0/2146506376 shutdown_connections 2023-11-26T07:18:48.386 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:48.381+0000 7f50b8d1d640 1 -- 172.21.15.143:0/2146506376 wait complete. 2023-11-26T07:18:48.404 DEBUG:teuthology.orchestra.run.smithi143:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth del client.bootstrap-rbd-mirror 2023-11-26T07:18:48.544 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:48.537+0000 7f6bcf59e640 1 Processor -- start 2023-11-26T07:18:48.544 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:48.537+0000 7f6bcf59e640 1 -- start start 2023-11-26T07:18:48.546 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:48.541+0000 7f6bcf59e640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f6bd0145eb0 0x7f6bd006a490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:18:48.546 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:48.541+0000 7f6bcf59e640 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7f6bd006a9d0 0x7f6bd006e870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:18:48.546 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:48.541+0000 7f6bcf59e640 1 -- --> v1:172.21.15.143:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6bd006ae30 con 0x7f6bd006edb0 2023-11-26T07:18:48.547 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:48.541+0000 7f6bcf59e640 1 -- --> v2:172.21.15.143:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6bd00b3ff0 con 0x7f6bd006a9d0 2023-11-26T07:18:48.547 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:48.541+0000 7f6bcf59e640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6bd006f580 con 0x7f6bd0145eb0 2023-11-26T07:18:48.547 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:48.541+0000 7f6bc7fff640 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7f6bd006a9d0 0x7f6bd006e870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:18:48.547 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:48.541+0000 7f6bced9d640 1 --1- >> v1:172.21.15.143:6789/0 conn(0x7f6bd006edb0 0x7f6bd006d650 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.143:6789/0 says I am v1:172.21.15.143:55150/0 (socket says 172.21.15.143:55150) 2023-11-26T07:18:48.547 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:48.541+0000 7f6bced9d640 1 -- 172.21.15.143:0/812676872 learned_addr learned my addr 172.21.15.143:0/812676872 (peer_addr_for_me v1:172.21.15.143:0/0) 2023-11-26T07:18:48.547 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:48.541+0000 7f6bce59c640 1 --2- 172.21.15.143:0/812676872 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f6bd0145eb0 0x7f6bd006a490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:18:48.547 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:48.541+0000 7f6bc7fff640 1 -- 172.21.15.143:0/812676872 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f6bd0145eb0 msgr2=0x7f6bd006a490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-26T07:18:48.547 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:48.541+0000 7f6bc7fff640 1 --2- 172.21.15.143:0/812676872 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f6bd0145eb0 0x7f6bd006a490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:18:48.547 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:48.541+0000 7f6bc7fff640 1 -- 172.21.15.143:0/812676872 >> v1:172.21.15.143:6789/0 conn(0x7f6bd006edb0 legacy=0x7f6bd006d650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:18:48.547 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:48.541+0000 7f6bc7fff640 1 -- 172.21.15.143:0/812676872 --> v2:172.21.15.143:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6bd0161850 con 0x7f6bd006a9d0 2023-11-26T07:18:48.547 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:48.541+0000 7f6bcdd9b640 1 -- 172.21.15.143:0/812676872 <== mon.0 v1:172.21.15.143:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 647739499 0 0) 0x7f6bd006ae30 con 0x7f6bd006edb0 2023-11-26T07:18:48.549 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:48.545+0000 7f6bc7fff640 1 --2- 172.21.15.143:0/812676872 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f6bd006a9d0 0x7f6bd006e870 secure :-1 s=READY pgs=377 cs=0 l=1 rev1=1 crypto rx=0x7f6bb80097c0 tx=0x7f6bb801c3f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d63903e3197bb4d8 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:18:48.549 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:48.545+0000 7f6bcdd9b640 1 -- 172.21.15.143:0/812676872 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6bb80ab600 con 0x7f6bd006a9d0 2023-11-26T07:18:48.549 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:48.545+0000 7f6bcdd9b640 1 -- 172.21.15.143:0/812676872 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6bb80a7050 con 0x7f6bd006a9d0 2023-11-26T07:18:48.549 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:48.545+0000 7f6bcdd9b640 1 -- 172.21.15.143:0/812676872 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6bb80ab980 con 0x7f6bd006a9d0 2023-11-26T07:18:48.550 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:48.545+0000 7f6bcf59e640 1 -- 172.21.15.143:0/812676872 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f6bd006a9d0 msgr2=0x7f6bd006e870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:18:48.550 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:48.545+0000 7f6bcf59e640 1 --2- 172.21.15.143:0/812676872 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f6bd006a9d0 0x7f6bd006e870 secure :-1 s=READY pgs=377 cs=0 l=1 rev1=1 crypto rx=0x7f6bb80097c0 tx=0x7f6bb801c3f0 comp rx=0 tx=0).stop 2023-11-26T07:18:48.550 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:48.545+0000 7f6bcf59e640 1 -- 172.21.15.143:0/812676872 shutdown_connections 2023-11-26T07:18:48.550 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:48.545+0000 7f6bcf59e640 1 --2- 172.21.15.143:0/812676872 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f6bd006a9d0 0x7f6bd006e870 unknown :-1 s=CLOSED pgs=377 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:18:48.551 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:48.545+0000 7f6bcf59e640 1 --2- 172.21.15.143:0/812676872 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f6bd0145eb0 0x7f6bd006a490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:18:48.551 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:48.545+0000 7f6bcf59e640 1 -- 172.21.15.143:0/812676872 >> 172.21.15.143:0/812676872 conn(0x7f6bd00b1ce0 msgr2=0x7f6bd00b20f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:18:48.551 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:48.545+0000 7f6bcf59e640 1 -- 172.21.15.143:0/812676872 shutdown_connections 2023-11-26T07:18:48.551 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:48.545+0000 7f6bcf59e640 1 -- 172.21.15.143:0/812676872 wait complete. 2023-11-26T07:18:48.552 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:48.545+0000 7f6bcf59e640 1 Processor -- start 2023-11-26T07:18:48.552 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:48.545+0000 7f6bcf59e640 1 -- start start 2023-11-26T07:18:48.554 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:48.549+0000 7f6bcf59e640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f6bd006a9d0 0x7f6bd015ab70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:18:48.554 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:48.549+0000 7f6bcf59e640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f6bd006edb0 0x7f6bd013d050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:18:48.554 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:48.549+0000 7f6bcf59e640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f6bd0145eb0 0x7f6bd013daf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:18:48.554 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:48.549+0000 7f6bcf59e640 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6bd015fdb0 con 0x7f6bd0145eb0 2023-11-26T07:18:48.555 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:48.549+0000 7f6bcf59e640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6bd0160050 con 0x7f6bd006edb0 2023-11-26T07:18:48.555 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:48.549+0000 7f6bcf59e640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6bd0160330 con 0x7f6bd006a9d0 2023-11-26T07:18:48.555 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:48.549+0000 7f6bced9d640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f6bd0145eb0 0x7f6bd013daf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:18:48.555 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:48.549+0000 7f6bced9d640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f6bd0145eb0 0x7f6bd013daf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:47446/0 (socket says 172.21.15.143:47446) 2023-11-26T07:18:48.555 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:48.549+0000 7f6bced9d640 1 -- 172.21.15.143:0/4144611590 learned_addr learned my addr 172.21.15.143:0/4144611590 (peer_addr_for_me v2:172.21.15.143:0/0) 2023-11-26T07:18:48.555 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:48.549+0000 7f6bce59c640 1 --2- 172.21.15.143:0/4144611590 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f6bd006a9d0 0x7f6bd015ab70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:18:48.555 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:48.549+0000 7f6bc7fff640 1 --2- 172.21.15.143:0/4144611590 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f6bd006edb0 0x7f6bd013d050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:18:48.555 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:48.549+0000 7f6bced9d640 1 -- 172.21.15.143:0/4144611590 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f6bd006a9d0 msgr2=0x7f6bd015ab70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:18:48.555 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:48.549+0000 7f6bced9d640 1 --2- 172.21.15.143:0/4144611590 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f6bd006a9d0 0x7f6bd015ab70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:18:48.555 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:48.549+0000 7f6bced9d640 1 -- 172.21.15.143:0/4144611590 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f6bd006edb0 msgr2=0x7f6bd013d050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:18:48.555 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:48.549+0000 7f6bced9d640 1 --2- 172.21.15.143:0/4144611590 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f6bd006edb0 0x7f6bd013d050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:18:48.555 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:48.549+0000 7f6bced9d640 1 -- 172.21.15.143:0/4144611590 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6bb8009420 con 0x7f6bd0145eb0 2023-11-26T07:18:48.555 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:48.549+0000 7f6bced9d640 1 --2- 172.21.15.143:0/4144611590 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f6bd0145eb0 0x7f6bd013daf0 secure :-1 s=READY pgs=378 cs=0 l=1 rev1=1 crypto rx=0x7f6bc8001570 tx=0x7f6bc800cca0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:18:48.556 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:48.549+0000 7f6bc6ffd640 1 -- 172.21.15.143:0/4144611590 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6bc800a640 con 0x7f6bd0145eb0 2023-11-26T07:18:48.556 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:48.549+0000 7f6bc6ffd640 1 -- 172.21.15.143:0/4144611590 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6bc800a7c0 con 0x7f6bd0145eb0 2023-11-26T07:18:48.556 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:48.549+0000 7f6bc6ffd640 1 -- 172.21.15.143:0/4144611590 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6bc800ab70 con 0x7f6bd0145eb0 2023-11-26T07:18:48.556 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:48.549+0000 7f6bcf59e640 1 -- 172.21.15.143:0/4144611590 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6bd015b220 con 0x7f6bd0145eb0 2023-11-26T07:18:48.556 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:48.549+0000 7f6bcf59e640 1 -- 172.21.15.143:0/4144611590 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6bd013d590 con 0x7f6bd0145eb0 2023-11-26T07:18:48.556 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:48.549+0000 7f6bcf59e640 1 -- 172.21.15.143:0/4144611590 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6bd00b3e20 con 0x7f6bd0145eb0 2023-11-26T07:18:48.556 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:48.549+0000 7f6bc6ffd640 1 -- 172.21.15.143:0/4144611590 <== mon.0 v2:172.21.15.143:3300/0 4 ==== mgrmap(e 5) v1 ==== 85852+0+0 (secure 0 0 0) 0x7f6bc800acd0 con 0x7f6bd0145eb0 2023-11-26T07:18:48.557 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:48.553+0000 7f6bc6ffd640 1 --2- 172.21.15.143:0/4144611590 >> [v2:172.21.15.143:6832/2150557544,v1:172.21.15.143:6833/2150557544] conn(0x7f6bb00664a0 0x7f6bb0068960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:18:48.557 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:48.553+0000 7f6bc6ffd640 1 -- 172.21.15.143:0/4144611590 <== mon.0 v2:172.21.15.143:3300/0 5 ==== osd_map(29..29 src has 1..29) v4 ==== 4857+0+0 (secure 0 0 0) 0x7f6bc8082f90 con 0x7f6bd0145eb0 2023-11-26T07:18:48.557 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:48.553+0000 7f6bce59c640 1 --2- 172.21.15.143:0/4144611590 >> [v2:172.21.15.143:6832/2150557544,v1:172.21.15.143:6833/2150557544] conn(0x7f6bb00664a0 0x7f6bb0068960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:18:48.558 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:48.553+0000 7f6bce59c640 1 --2- 172.21.15.143:0/4144611590 >> [v2:172.21.15.143:6832/2150557544,v1:172.21.15.143:6833/2150557544] conn(0x7f6bb00664a0 0x7f6bb0068960 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7f6bd015d110 tx=0x7f6bc0008040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:18:48.561 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:48.557+0000 7f6bc6ffd640 1 -- 172.21.15.143:0/4144611590 <== mon.0 v2:172.21.15.143:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172649 (secure 0 0 0) 0x7f6bc80b4280 con 0x7f6bd0145eb0 2023-11-26T07:18:48.767 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:48.761+0000 7f6bcf59e640 1 -- 172.21.15.143:0/4144611590 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "auth del", "entity": "client.bootstrap-rbd-mirror"} v 0) v1 -- 0x7f6bd0067500 con 0x7f6bd0145eb0 2023-11-26T07:18:48.770 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:48.765+0000 7f6bc6ffd640 1 -- 172.21.15.143:0/4144611590 <== mon.0 v2:172.21.15.143:3300/0 7 ==== mon_command_ack([{"prefix": "auth del", "entity": "client.bootstrap-rbd-mirror"}]=0 v11) v1 ==== 97+0+0 (secure 0 0 0) 0x7f6bc8018020 con 0x7f6bd0145eb0 2023-11-26T07:18:48.772 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:48.765+0000 7f6bcf59e640 1 -- 172.21.15.143:0/4144611590 >> [v2:172.21.15.143:6832/2150557544,v1:172.21.15.143:6833/2150557544] conn(0x7f6bb00664a0 msgr2=0x7f6bb0068960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:18:48.772 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:48.765+0000 7f6bcf59e640 1 --2- 172.21.15.143:0/4144611590 >> [v2:172.21.15.143:6832/2150557544,v1:172.21.15.143:6833/2150557544] conn(0x7f6bb00664a0 0x7f6bb0068960 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7f6bd015d110 tx=0x7f6bc0008040 comp rx=0 tx=0).stop 2023-11-26T07:18:48.772 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:48.765+0000 7f6bcf59e640 1 -- 172.21.15.143:0/4144611590 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f6bd0145eb0 msgr2=0x7f6bd013daf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:18:48.772 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:48.765+0000 7f6bcf59e640 1 --2- 172.21.15.143:0/4144611590 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f6bd0145eb0 0x7f6bd013daf0 secure :-1 s=READY pgs=378 cs=0 l=1 rev1=1 crypto rx=0x7f6bc8001570 tx=0x7f6bc800cca0 comp rx=0 tx=0).stop 2023-11-26T07:18:48.773 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:48.765+0000 7f6bcf59e640 1 -- 172.21.15.143:0/4144611590 shutdown_connections 2023-11-26T07:18:48.773 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:48.765+0000 7f6bcf59e640 1 --2- 172.21.15.143:0/4144611590 >> [v2:172.21.15.143:6832/2150557544,v1:172.21.15.143:6833/2150557544] conn(0x7f6bb00664a0 0x7f6bb0068960 unknown :-1 s=CLOSED pgs=208 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:18:48.773 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:48.765+0000 7f6bcf59e640 1 --2- 172.21.15.143:0/4144611590 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f6bd0145eb0 0x7f6bd013daf0 unknown :-1 s=CLOSED pgs=378 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:18:48.773 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:48.765+0000 7f6bcf59e640 1 --2- 172.21.15.143:0/4144611590 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f6bd006edb0 0x7f6bd013d050 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:18:48.773 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:48.765+0000 7f6bcf59e640 1 --2- 172.21.15.143:0/4144611590 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f6bd006a9d0 0x7f6bd015ab70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:18:48.773 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:48.765+0000 7f6bcf59e640 1 -- 172.21.15.143:0/4144611590 >> 172.21.15.143:0/4144611590 conn(0x7f6bd00b1ce0 msgr2=0x7f6bd00aeef0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:18:48.774 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:48.765+0000 7f6bcf59e640 1 -- 172.21.15.143:0/4144611590 shutdown_connections 2023-11-26T07:18:48.774 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:48.765+0000 7f6bcf59e640 1 -- 172.21.15.143:0/4144611590 wait complete. 2023-11-26T07:18:48.791 DEBUG:teuthology.orchestra.run.smithi143:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth del client.bootstrap-rgw 2023-11-26T07:18:48.931 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:48.925+0000 7f2aa3b1d640 1 Processor -- start 2023-11-26T07:18:48.932 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:48.925+0000 7f2aa3b1d640 1 -- start start 2023-11-26T07:18:48.934 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:48.929+0000 7f2aa3b1d640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f2a9c13e290 0x7f2a9c0cf280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:18:48.934 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:48.929+0000 7f2aa3b1d640 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7f2a9c0cf7c0 0x7f2a9c15efa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:18:48.934 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:48.929+0000 7f2aa3b1d640 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7f2a9c15f4e0 0x7f2a9c1408e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:18:48.934 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:48.929+0000 7f2aa3b1d640 1 -- --> v2:172.21.15.143:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2a9c0cfc40 con 0x7f2a9c15f4e0 2023-11-26T07:18:48.935 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:48.929+0000 7f2aa3b1d640 1 -- --> v2:172.21.15.188:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2a9c06c520 con 0x7f2a9c0cf7c0 2023-11-26T07:18:48.935 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:48.929+0000 7f2aa3b1d640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2a9c06c800 con 0x7f2a9c13e290 2023-11-26T07:18:48.935 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:48.929+0000 7f2aa20ba640 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7f2a9c15f4e0 0x7f2a9c1408e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:18:48.935 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:48.929+0000 7f2aa20ba640 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7f2a9c15f4e0 0x7f2a9c1408e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:47452/0 (socket says 172.21.15.143:47452) 2023-11-26T07:18:48.935 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:48.929+0000 7f2aa20ba640 1 -- 172.21.15.143:0/1383160191 learned_addr learned my addr 172.21.15.143:0/1383160191 (peer_addr_for_me v2:172.21.15.143:0/0) 2023-11-26T07:18:48.936 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:48.929+0000 7f2aa10b8640 1 --2- 172.21.15.143:0/1383160191 >> v2:172.21.15.188:3300/0 conn(0x7f2a9c0cf7c0 0x7f2a9c15efa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:18:48.936 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:48.929+0000 7f2aa18b9640 1 --2- 172.21.15.143:0/1383160191 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f2a9c13e290 0x7f2a9c0cf280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:18:48.936 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:48.929+0000 7f2aa20ba640 1 -- 172.21.15.143:0/1383160191 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f2a9c13e290 msgr2=0x7f2a9c0cf280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-26T07:18:48.936 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:48.929+0000 7f2aa20ba640 1 --2- 172.21.15.143:0/1383160191 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f2a9c13e290 0x7f2a9c0cf280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:18:48.936 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:48.929+0000 7f2aa20ba640 1 -- 172.21.15.143:0/1383160191 >> v2:172.21.15.188:3300/0 conn(0x7f2a9c0cf7c0 msgr2=0x7f2a9c15efa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-26T07:18:48.937 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:48.929+0000 7f2aa20ba640 1 --2- 172.21.15.143:0/1383160191 >> v2:172.21.15.188:3300/0 conn(0x7f2a9c0cf7c0 0x7f2a9c15efa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:18:48.937 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:48.929+0000 7f2aa20ba640 1 -- 172.21.15.143:0/1383160191 --> v2:172.21.15.143:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2a9c06cae0 con 0x7f2a9c15f4e0 2023-11-26T07:18:48.937 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:48.929+0000 7f2aa10b8640 1 --2- 172.21.15.143:0/1383160191 >> v2:172.21.15.188:3300/0 conn(0x7f2a9c0cf7c0 0x7f2a9c15efa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-26T07:18:48.937 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:48.929+0000 7f2aa18b9640 1 --2- 172.21.15.143:0/1383160191 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f2a9c13e290 0x7f2a9c0cf280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-26T07:18:48.938 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:48.929+0000 7f2aa20ba640 1 --2- 172.21.15.143:0/1383160191 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f2a9c15f4e0 0x7f2a9c1408e0 secure :-1 s=READY pgs=379 cs=0 l=1 rev1=1 crypto rx=0x7f2a9c0b45c0 tx=0x7f2a9401c6e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8cfd3364527fa4b7 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:18:48.939 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:48.933+0000 7f2aa08b7640 1 -- 172.21.15.143:0/1383160191 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2a940a5560 con 0x7f2a9c15f4e0 2023-11-26T07:18:48.939 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:48.933+0000 7f2aa08b7640 1 -- 172.21.15.143:0/1383160191 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2a940a56c0 con 0x7f2a9c15f4e0 2023-11-26T07:18:48.939 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:48.933+0000 7f2aa08b7640 1 -- 172.21.15.143:0/1383160191 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2a940a5a50 con 0x7f2a9c15f4e0 2023-11-26T07:18:48.939 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:48.933+0000 7f2aa3b1d640 1 -- 172.21.15.143:0/1383160191 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f2a9c15f4e0 msgr2=0x7f2a9c1408e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:18:48.939 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:48.933+0000 7f2aa3b1d640 1 --2- 172.21.15.143:0/1383160191 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f2a9c15f4e0 0x7f2a9c1408e0 secure :-1 s=READY pgs=379 cs=0 l=1 rev1=1 crypto rx=0x7f2a9c0b45c0 tx=0x7f2a9401c6e0 comp rx=0 tx=0).stop 2023-11-26T07:18:48.940 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:48.933+0000 7f2aa3b1d640 1 -- 172.21.15.143:0/1383160191 shutdown_connections 2023-11-26T07:18:48.940 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:48.933+0000 7f2aa3b1d640 1 --2- 172.21.15.143:0/1383160191 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f2a9c15f4e0 0x7f2a9c1408e0 unknown :-1 s=CLOSED pgs=379 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:18:48.940 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:48.933+0000 7f2aa3b1d640 1 --2- 172.21.15.143:0/1383160191 >> v2:172.21.15.188:3300/0 conn(0x7f2a9c0cf7c0 0x7f2a9c15efa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:18:48.940 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:48.933+0000 7f2aa3b1d640 1 --2- 172.21.15.143:0/1383160191 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f2a9c13e290 0x7f2a9c0cf280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:18:48.940 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:48.933+0000 7f2aa3b1d640 1 -- 172.21.15.143:0/1383160191 >> 172.21.15.143:0/1383160191 conn(0x7f2a9c06a890 msgr2=0x7f2a9c06aca0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:18:48.940 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:48.933+0000 7f2aa3b1d640 1 -- 172.21.15.143:0/1383160191 shutdown_connections 2023-11-26T07:18:48.941 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:48.933+0000 7f2aa3b1d640 1 -- 172.21.15.143:0/1383160191 wait complete. 2023-11-26T07:18:48.942 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:48.933+0000 7f2aa3b1d640 1 Processor -- start 2023-11-26T07:18:48.943 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:48.933+0000 7f2aa3b1d640 1 -- start start 2023-11-26T07:18:48.943 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:48.933+0000 7f2aa3b1d640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f2a9c0cf7c0 0x7f2a9c0af540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:18:48.943 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:48.933+0000 7f2aa3b1d640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f2a9c13e290 0x7f2a9c137890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:18:48.943 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:48.933+0000 7f2aa3b1d640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f2a9c15f4e0 0x7f2a9c15d1e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:18:48.943 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:48.933+0000 7f2aa3b1d640 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2a9c0b0660 con 0x7f2a9c13e290 2023-11-26T07:18:48.943 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:48.933+0000 7f2aa3b1d640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2a9c142370 con 0x7f2a9c0cf7c0 2023-11-26T07:18:48.944 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:48.933+0000 7f2aa3b1d640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2a9c0afdc0 con 0x7f2a9c15f4e0 2023-11-26T07:18:48.944 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:48.933+0000 7f2aa10b8640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f2a9c13e290 0x7f2a9c137890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:18:48.944 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:48.933+0000 7f2aa10b8640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f2a9c13e290 0x7f2a9c137890 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:47466/0 (socket says 172.21.15.143:47466) 2023-11-26T07:18:48.944 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:48.933+0000 7f2aa10b8640 1 -- 172.21.15.143:0/1419389850 learned_addr learned my addr 172.21.15.143:0/1419389850 (peer_addr_for_me v2:172.21.15.143:0/0) 2023-11-26T07:18:48.944 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:48.933+0000 7f2aa18b9640 1 --2- 172.21.15.143:0/1419389850 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f2a9c0cf7c0 0x7f2a9c0af540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:18:48.944 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:48.933+0000 7f2aa20ba640 1 --2- 172.21.15.143:0/1419389850 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f2a9c15f4e0 0x7f2a9c15d1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:18:48.944 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:48.937+0000 7f2aa10b8640 1 -- 172.21.15.143:0/1419389850 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f2a9c15f4e0 msgr2=0x7f2a9c15d1e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:18:48.945 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:48.937+0000 7f2aa10b8640 1 --2- 172.21.15.143:0/1419389850 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f2a9c15f4e0 0x7f2a9c15d1e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:18:48.945 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:48.937+0000 7f2aa10b8640 1 -- 172.21.15.143:0/1419389850 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f2a9c0cf7c0 msgr2=0x7f2a9c0af540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:18:48.945 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:48.937+0000 7f2aa10b8640 1 --2- 172.21.15.143:0/1419389850 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f2a9c0cf7c0 0x7f2a9c0af540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:18:48.945 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:48.937+0000 7f2aa10b8640 1 -- 172.21.15.143:0/1419389850 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2a94009b30 con 0x7f2a9c13e290 2023-11-26T07:18:48.945 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:48.937+0000 7f2aa10b8640 1 --2- 172.21.15.143:0/1419389850 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f2a9c13e290 0x7f2a9c137890 secure :-1 s=READY pgs=380 cs=0 l=1 rev1=1 crypto rx=0x7f2a8c00c7c0 tx=0x7f2a8c00cc80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:18:48.945 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:48.937+0000 7f2a927fc640 1 -- 172.21.15.143:0/1419389850 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2a8c007a30 con 0x7f2a9c13e290 2023-11-26T07:18:48.945 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:48.937+0000 7f2a927fc640 1 -- 172.21.15.143:0/1419389850 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2a8c007b90 con 0x7f2a9c13e290 2023-11-26T07:18:48.946 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:48.937+0000 7f2a927fc640 1 -- 172.21.15.143:0/1419389850 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2a8c00ce70 con 0x7f2a9c13e290 2023-11-26T07:18:48.946 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:48.937+0000 7f2aa3b1d640 1 -- 172.21.15.143:0/1419389850 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2a9c0b00a0 con 0x7f2a9c13e290 2023-11-26T07:18:48.946 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:48.937+0000 7f2aa3b1d640 1 -- 172.21.15.143:0/1419389850 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2a9c142610 con 0x7f2a9c13e290 2023-11-26T07:18:48.946 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:48.937+0000 7f2aa3b1d640 1 -- 172.21.15.143:0/1419389850 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2a9c142830 con 0x7f2a9c13e290 2023-11-26T07:18:48.947 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:48.937+0000 7f2a927fc640 1 -- 172.21.15.143:0/1419389850 <== mon.0 v2:172.21.15.143:3300/0 4 ==== mgrmap(e 5) v1 ==== 85852+0+0 (secure 0 0 0) 0x7f2a8c0113f0 con 0x7f2a9c13e290 2023-11-26T07:18:48.947 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:48.937+0000 7f2a927fc640 1 --2- 172.21.15.143:0/1419389850 >> [v2:172.21.15.143:6832/2150557544,v1:172.21.15.143:6833/2150557544] conn(0x7f2a800664a0 0x7f2a80068960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:18:48.947 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:48.937+0000 7f2a927fc640 1 -- 172.21.15.143:0/1419389850 <== mon.0 v2:172.21.15.143:3300/0 5 ==== osd_map(29..29 src has 1..29) v4 ==== 4857+0+0 (secure 0 0 0) 0x7f2a8c083c50 con 0x7f2a9c13e290 2023-11-26T07:18:48.947 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:48.937+0000 7f2aa18b9640 1 --2- 172.21.15.143:0/1419389850 >> [v2:172.21.15.143:6832/2150557544,v1:172.21.15.143:6833/2150557544] conn(0x7f2a800664a0 0x7f2a80068960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:18:48.948 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:48.941+0000 7f2aa18b9640 1 --2- 172.21.15.143:0/1419389850 >> [v2:172.21.15.143:6832/2150557544,v1:172.21.15.143:6833/2150557544] conn(0x7f2a800664a0 0x7f2a80068960 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7f2a9800ab60 tx=0x7f2a98008040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:18:48.948 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:48.941+0000 7f2a927fc640 1 -- 172.21.15.143:0/1419389850 <== mon.0 v2:172.21.15.143:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172649 (secure 0 0 0) 0x7f2a8c051430 con 0x7f2a9c13e290 2023-11-26T07:18:49.150 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:49.145+0000 7f2aa3b1d640 1 -- 172.21.15.143:0/1419389850 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "auth del", "entity": "client.bootstrap-rgw"} v 0) v1 -- 0x7f2a9c073ce0 con 0x7f2a9c13e290 2023-11-26T07:18:49.153 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:49.149+0000 7f2a927fc640 1 -- 172.21.15.143:0/1419389850 <== mon.0 v2:172.21.15.143:3300/0 7 ==== mon_command_ack([{"prefix": "auth del", "entity": "client.bootstrap-rgw"}]=0 v12) v1 ==== 90+0+0 (secure 0 0 0) 0x7f2a9c073ce0 con 0x7f2a9c13e290 2023-11-26T07:18:49.155 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:49.149+0000 7f2aa3b1d640 1 -- 172.21.15.143:0/1419389850 >> [v2:172.21.15.143:6832/2150557544,v1:172.21.15.143:6833/2150557544] conn(0x7f2a800664a0 msgr2=0x7f2a80068960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:18:49.155 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:49.149+0000 7f2aa3b1d640 1 --2- 172.21.15.143:0/1419389850 >> [v2:172.21.15.143:6832/2150557544,v1:172.21.15.143:6833/2150557544] conn(0x7f2a800664a0 0x7f2a80068960 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7f2a9800ab60 tx=0x7f2a98008040 comp rx=0 tx=0).stop 2023-11-26T07:18:49.155 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:49.149+0000 7f2aa3b1d640 1 -- 172.21.15.143:0/1419389850 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f2a9c13e290 msgr2=0x7f2a9c137890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:18:49.156 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:49.149+0000 7f2aa3b1d640 1 --2- 172.21.15.143:0/1419389850 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f2a9c13e290 0x7f2a9c137890 secure :-1 s=READY pgs=380 cs=0 l=1 rev1=1 crypto rx=0x7f2a8c00c7c0 tx=0x7f2a8c00cc80 comp rx=0 tx=0).stop 2023-11-26T07:18:49.156 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:49.149+0000 7f2aa3b1d640 1 -- 172.21.15.143:0/1419389850 shutdown_connections 2023-11-26T07:18:49.156 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:49.149+0000 7f2aa3b1d640 1 --2- 172.21.15.143:0/1419389850 >> [v2:172.21.15.143:6832/2150557544,v1:172.21.15.143:6833/2150557544] conn(0x7f2a800664a0 0x7f2a80068960 unknown :-1 s=CLOSED pgs=209 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:18:49.156 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:49.149+0000 7f2aa3b1d640 1 --2- 172.21.15.143:0/1419389850 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f2a9c15f4e0 0x7f2a9c15d1e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:18:49.156 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:49.149+0000 7f2aa3b1d640 1 --2- 172.21.15.143:0/1419389850 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f2a9c13e290 0x7f2a9c137890 unknown :-1 s=CLOSED pgs=380 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:18:49.156 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:49.149+0000 7f2aa3b1d640 1 --2- 172.21.15.143:0/1419389850 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f2a9c0cf7c0 0x7f2a9c0af540 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:18:49.157 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:49.149+0000 7f2aa3b1d640 1 -- 172.21.15.143:0/1419389850 >> 172.21.15.143:0/1419389850 conn(0x7f2a9c06a890 msgr2=0x7f2a9c0ce5f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:18:49.157 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:49.149+0000 7f2aa3b1d640 1 -- 172.21.15.143:0/1419389850 shutdown_connections 2023-11-26T07:18:49.157 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:49.149+0000 7f2aa3b1d640 1 -- 172.21.15.143:0/1419389850 wait complete. 2023-11-26T07:18:49.175 DEBUG:teuthology.orchestra.run.smithi143:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-11-26T07:18:49.314 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:49.305+0000 7f1498735640 1 Processor -- start 2023-11-26T07:18:49.315 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:49.305+0000 7f1498735640 1 -- start start 2023-11-26T07:18:49.315 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:49.309+0000 7f1498735640 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7f14900cf280 0x7f14900cf710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:18:49.315 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:49.309+0000 7f1498735640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f14900cd1b0 0x7f14900cd5c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:18:49.315 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:49.309+0000 7f1498735640 1 -- --> v1:172.21.15.143:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f14900cc730 con 0x7f1490143ac0 2023-11-26T07:18:49.315 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:49.309+0000 7f1498735640 1 -- --> v2:172.21.15.188:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1490143ea0 con 0x7f14900cf280 2023-11-26T07:18:49.315 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:49.309+0000 7f1498735640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f14900cfc50 con 0x7f14900cd1b0 2023-11-26T07:18:49.316 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:49.309+0000 7f14964d1640 1 --1- >> v1:172.21.15.143:6789/0 conn(0x7f1490143ac0 0x7f1490070370 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.143:6789/0 says I am v1:172.21.15.143:55152/0 (socket says 172.21.15.143:55152) 2023-11-26T07:18:49.316 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:49.309+0000 7f14964d1640 1 -- 172.21.15.143:0/2098630568 learned_addr learned my addr 172.21.15.143:0/2098630568 (peer_addr_for_me v1:172.21.15.143:0/0) 2023-11-26T07:18:49.316 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:49.309+0000 7f1496cd2640 1 --2- 172.21.15.143:0/2098630568 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f14900cd1b0 0x7f14900cd5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:18:49.316 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:49.309+0000 7f1495cd0640 1 --2- 172.21.15.143:0/2098630568 >> v2:172.21.15.188:3300/0 conn(0x7f14900cf280 0x7f14900cf710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:18:49.316 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:49.309+0000 7f14954cf640 1 -- 172.21.15.143:0/2098630568 <== mon.0 v1:172.21.15.143:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3787605378 0 0) 0x7f14900cc730 con 0x7f1490143ac0 2023-11-26T07:18:49.316 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:49.309+0000 7f14954cf640 1 -- 172.21.15.143:0/2098630568 --> v1:172.21.15.143:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f147c003530 con 0x7f1490143ac0 2023-11-26T07:18:49.317 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:49.309+0000 7f1496cd2640 1 -- 172.21.15.143:0/2098630568 >> v2:172.21.15.188:3300/0 conn(0x7f14900cf280 msgr2=0x7f14900cf710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-26T07:18:49.317 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:49.309+0000 7f1496cd2640 1 --2- 172.21.15.143:0/2098630568 >> v2:172.21.15.188:3300/0 conn(0x7f14900cf280 0x7f14900cf710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:18:49.317 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:49.309+0000 7f1496cd2640 1 -- 172.21.15.143:0/2098630568 >> v1:172.21.15.143:6789/0 conn(0x7f1490143ac0 legacy=0x7f1490070370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:18:49.317 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:49.309+0000 7f1496cd2640 1 -- 172.21.15.143:0/2098630568 --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1490070ee0 con 0x7f14900cd1b0 2023-11-26T07:18:49.317 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:49.309+0000 7f1495cd0640 1 --2- 172.21.15.143:0/2098630568 >> v2:172.21.15.188:3300/0 conn(0x7f14900cf280 0x7f14900cf710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-26T07:18:49.318 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:49.313+0000 7f1496cd2640 1 --2- 172.21.15.143:0/2098630568 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f14900cd1b0 0x7f14900cd5c0 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7f1488009d90 tx=0x7f148801c950 comp rx=0 tx=0).ready entity=mon.2 client_cookie=b2115a67da4a1de8 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:18:49.318 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:49.313+0000 7f14954cf640 1 -- 172.21.15.143:0/2098630568 <== mon.2 v2:172.21.15.188:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f14880a5a10 con 0x7f14900cd1b0 2023-11-26T07:18:49.319 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:49.313+0000 7f14954cf640 1 -- 172.21.15.143:0/2098630568 <== mon.2 v2:172.21.15.188:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f14880a5b70 con 0x7f14900cd1b0 2023-11-26T07:18:49.319 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:49.313+0000 7f14954cf640 1 -- 172.21.15.143:0/2098630568 <== mon.2 v2:172.21.15.188:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f14880ad440 con 0x7f14900cd1b0 2023-11-26T07:18:49.319 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:49.313+0000 7f1498735640 1 -- 172.21.15.143:0/2098630568 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f14900cd1b0 msgr2=0x7f14900cd5c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:18:49.319 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:49.313+0000 7f1498735640 1 --2- 172.21.15.143:0/2098630568 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f14900cd1b0 0x7f14900cd5c0 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7f1488009d90 tx=0x7f148801c950 comp rx=0 tx=0).stop 2023-11-26T07:18:49.319 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:49.313+0000 7f1498735640 1 -- 172.21.15.143:0/2098630568 shutdown_connections 2023-11-26T07:18:49.319 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:49.313+0000 7f1498735640 1 --2- 172.21.15.143:0/2098630568 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f14900cd1b0 0x7f14900cd5c0 unknown :-1 s=CLOSED pgs=74 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:18:49.320 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:49.313+0000 7f1498735640 1 --2- 172.21.15.143:0/2098630568 >> v2:172.21.15.188:3300/0 conn(0x7f14900cf280 0x7f14900cf710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:18:49.320 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:49.313+0000 7f1498735640 1 -- 172.21.15.143:0/2098630568 >> 172.21.15.143:0/2098630568 conn(0x7f149006a890 msgr2=0x7f149006aca0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:18:49.320 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:49.313+0000 7f1498735640 1 -- 172.21.15.143:0/2098630568 shutdown_connections 2023-11-26T07:18:49.320 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:49.313+0000 7f1498735640 1 -- 172.21.15.143:0/2098630568 wait complete. 2023-11-26T07:18:49.322 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:49.313+0000 7f1498735640 1 Processor -- start 2023-11-26T07:18:49.322 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:49.313+0000 7f1498735640 1 -- start start 2023-11-26T07:18:49.322 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:49.313+0000 7f1498735640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f14900cd1b0 0x7f149013db40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:18:49.322 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:49.313+0000 7f1498735640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f14900cf280 0x7f1490163f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:18:49.322 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:49.313+0000 7f1498735640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f1490143ac0 0x7f1490159ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:18:49.323 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:49.313+0000 7f1498735640 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f14900ac8e0 con 0x7f14900cd1b0 2023-11-26T07:18:49.323 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:49.313+0000 7f1498735640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1490164460 con 0x7f1490143ac0 2023-11-26T07:18:49.323 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:49.313+0000 7f1498735640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1490159ff0 con 0x7f14900cf280 2023-11-26T07:18:49.323 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:49.317+0000 7f14964d1640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f14900cd1b0 0x7f149013db40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:18:49.323 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:49.317+0000 7f14964d1640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f14900cd1b0 0x7f149013db40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:47472/0 (socket says 172.21.15.143:47472) 2023-11-26T07:18:49.323 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:49.317+0000 7f14964d1640 1 -- 172.21.15.143:0/4006105233 learned_addr learned my addr 172.21.15.143:0/4006105233 (peer_addr_for_me v2:172.21.15.143:0/0) 2023-11-26T07:18:49.324 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:49.317+0000 7f1495cd0640 1 --2- 172.21.15.143:0/4006105233 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f14900cf280 0x7f1490163f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:18:49.324 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:49.317+0000 7f1496cd2640 1 --2- 172.21.15.143:0/4006105233 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f1490143ac0 0x7f1490159ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:18:49.324 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:49.317+0000 7f14964d1640 1 -- 172.21.15.143:0/4006105233 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f14900cf280 msgr2=0x7f1490163f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:18:49.324 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:49.317+0000 7f14964d1640 1 --2- 172.21.15.143:0/4006105233 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f14900cf280 0x7f1490163f20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:18:49.324 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:49.317+0000 7f14964d1640 1 -- 172.21.15.143:0/4006105233 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f1490143ac0 msgr2=0x7f1490159ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:18:49.324 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:49.317+0000 7f14964d1640 1 --2- 172.21.15.143:0/4006105233 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f1490143ac0 0x7f1490159ab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:18:49.325 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:49.317+0000 7f14964d1640 1 -- 172.21.15.143:0/4006105233 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1480009420 con 0x7f14900cd1b0 2023-11-26T07:18:49.325 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:49.317+0000 7f14964d1640 1 --2- 172.21.15.143:0/4006105233 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f14900cd1b0 0x7f149013db40 secure :-1 s=READY pgs=382 cs=0 l=1 rev1=1 crypto rx=0x7f148c00c530 tx=0x7f148c00c9f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:18:49.325 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:49.317+0000 7f1486ffd640 1 -- 172.21.15.143:0/4006105233 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f148c005cf0 con 0x7f14900cd1b0 2023-11-26T07:18:49.325 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:49.317+0000 7f1486ffd640 1 -- 172.21.15.143:0/4006105233 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f148c005e50 con 0x7f14900cd1b0 2023-11-26T07:18:49.325 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:49.317+0000 7f1498735640 1 -- 172.21.15.143:0/4006105233 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1488009a70 con 0x7f14900cd1b0 2023-11-26T07:18:49.325 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:49.317+0000 7f1486ffd640 1 -- 172.21.15.143:0/4006105233 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f148c007270 con 0x7f14900cd1b0 2023-11-26T07:18:49.325 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:49.317+0000 7f1498735640 1 -- 172.21.15.143:0/4006105233 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f14900acaf0 con 0x7f14900cd1b0 2023-11-26T07:18:49.326 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:49.317+0000 7f1498735640 1 -- 172.21.15.143:0/4006105233 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1490067680 con 0x7f14900cd1b0 2023-11-26T07:18:49.327 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:49.317+0000 7f1486ffd640 1 -- 172.21.15.143:0/4006105233 <== mon.0 v2:172.21.15.143:3300/0 4 ==== mgrmap(e 5) v1 ==== 85852+0+0 (secure 0 0 0) 0x7f148c0073d0 con 0x7f14900cd1b0 2023-11-26T07:18:49.327 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:49.321+0000 7f1486ffd640 1 --2- 172.21.15.143:0/4006105233 >> [v2:172.21.15.143:6832/2150557544,v1:172.21.15.143:6833/2150557544] conn(0x7f1474066470 0x7f1474068930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:18:49.327 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:49.321+0000 7f1486ffd640 1 -- 172.21.15.143:0/4006105233 <== mon.0 v2:172.21.15.143:3300/0 5 ==== osd_map(29..29 src has 1..29) v4 ==== 4857+0+0 (secure 0 0 0) 0x7f148c083100 con 0x7f14900cd1b0 2023-11-26T07:18:49.327 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:49.321+0000 7f1495cd0640 1 --2- 172.21.15.143:0/4006105233 >> [v2:172.21.15.143:6832/2150557544,v1:172.21.15.143:6833/2150557544] conn(0x7f1474066470 0x7f1474068930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:18:49.327 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:49.321+0000 7f1495cd0640 1 --2- 172.21.15.143:0/4006105233 >> [v2:172.21.15.143:6832/2150557544,v1:172.21.15.143:6833/2150557544] conn(0x7f1474066470 0x7f1474068930 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7f1480009c50 tx=0x7f1480009190 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:18:49.328 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:49.325+0000 7f1486ffd640 1 -- 172.21.15.143:0/4006105233 <== mon.0 v2:172.21.15.143:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172649 (secure 0 0 0) 0x7f148c0509a0 con 0x7f14900cd1b0 2023-11-26T07:18:49.479 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:49.473+0000 7f1498735640 1 -- 172.21.15.143:0/4006105233 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f14900ae980 con 0x7f14900cd1b0 2023-11-26T07:18:49.480 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:49.473+0000 7f1486ffd640 1 -- 172.21.15.143:0/4006105233 <== mon.0 v2:172.21.15.143:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v29) v1 ==== 74+0+12893 (secure 0 0 0) 0x7f148c053fc0 con 0x7f14900cd1b0 2023-11-26T07:18:49.480 INFO:teuthology.orchestra.run.smithi143.stdout: 2023-11-26T07:18:49.480 INFO:teuthology.orchestra.run.smithi143.stdout:{"epoch":29,"fsid":"f5033060-2469-4a07-95fe-2b76f5404bc1","created":"2023-11-26T07:17:28.613663+0000","modified":"2023-11-26T07:18:46.475695+0000","last_up_change":"2023-11-26T07:17:41.157813+0000","last_in_change":"2023-11-26T07:17:31.181658+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-11-26T07:17:41.758597+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"ebbee2c6-023e-44b9-9f1d-6f7bfd309730","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6817","nonce":639807369},{"type":"v1","addr":"172.21.15.143:6819","nonce":639807369}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6821","nonce":639807369},{"type":"v1","addr":"172.21.15.143:6823","nonce":639807369}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6830","nonce":639807369},{"type":"v1","addr":"172.21.15.143:6831","nonce":639807369}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6825","nonce":639807369},{"type":"v1","addr":"172.21.15.143:6828","nonce":639807369}]},"public_addr":"172.21.15.143:6819/639807369","cluster_addr":"172.21.15.143:6823/639807369","heartbeat_back_addr":"172.21.15.143:6831/639807369","heartbeat_front_addr":"172.21.15.143:6828/639807369","state":["exists","up"]},{"osd":1,"uuid":"fec4b8be-7122-49e9-9a19-5850a0700001","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6816","nonce":1144942530},{"type":"v1","addr":"172.21.15.143:6818","nonce":1144942530}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6820","nonce":1144942530},{"type":"v1","addr":"172.21.15.143:6822","nonce":1144942530}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6827","nonce":1144942530},{"type":"v1","addr":"172.21.15.143:6829","nonce":1144942530}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6824","nonce":1144942530},{"type":"v1","addr":"172.21.15.143:6826","nonce":1144942530}]},"public_addr":"172.21.15.143:6818/1144942530","cluster_addr":"172.21.15.143:6822/1144942530","heartbeat_back_addr":"172.21.15.143:6829/1144942530","heartbeat_front_addr":"172.21.15.143:6826/1144942530","state":["exists","up"]},{"osd":2,"uuid":"2888af5b-f933-46ca-8f95-6dd494ff43f6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6808","nonce":3909095186},{"type":"v1","addr":"172.21.15.143:6809","nonce":3909095186}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6810","nonce":3909095186},{"type":"v1","addr":"172.21.15.143:6811","nonce":3909095186}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6814","nonce":3909095186},{"type":"v1","addr":"172.21.15.143:6815","nonce":3909095186}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6812","nonce":3909095186},{"type":"v1","addr":"172.21.15.143:6813","nonce":3909095186}]},"public_addr":"172.21.15.143:6809/3909095186","cluster_addr":"172.21.15.143:6811/3909095186","heartbeat_back_addr":"172.21.15.143:6815/3909095186","heartbeat_front_addr":"172.21.15.143:6813/3909095186","state":["exists","up"]},{"osd":3,"uuid":"d1f52435-7bc1-4081-9550-9cd1ba55f549","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6800","nonce":3854210991},{"type":"v1","addr":"172.21.15.143:6801","nonce":3854210991}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6802","nonce":3854210991},{"type":"v1","addr":"172.21.15.143:6803","nonce":3854210991}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6806","nonce":3854210991},{"type":"v1","addr":"172.21.15.143:6807","nonce":3854210991}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6804","nonce":3854210991},{"type":"v1","addr":"172.21.15.143:6805","nonce":3854210991}]},"public_addr":"172.21.15.143:6801/3854210991","cluster_addr":"172.21.15.143:6803/3854210991","heartbeat_back_addr":"172.21.15.143:6807/3854210991","heartbeat_front_addr":"172.21.15.143:6805/3854210991","state":["exists","up"]},{"osd":4,"uuid":"e2ae8b6e-6e25-49b2-bcbf-aacce68f06a3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6800","nonce":3924606902},{"type":"v1","addr":"172.21.15.188:6801","nonce":3924606902}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6802","nonce":3924606902},{"type":"v1","addr":"172.21.15.188:6803","nonce":3924606902}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6806","nonce":3924606902},{"type":"v1","addr":"172.21.15.188:6807","nonce":3924606902}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6804","nonce":3924606902},{"type":"v1","addr":"172.21.15.188:6805","nonce":3924606902}]},"public_addr":"172.21.15.188:6801/3924606902","cluster_addr":"172.21.15.188:6803/3924606902","heartbeat_back_addr":"172.21.15.188:6807/3924606902","heartbeat_front_addr":"172.21.15.188:6805/3924606902","state":["exists","up"]},{"osd":5,"uuid":"3cec8a32-7698-4c36-8e89-e4ba11934f33","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6813","nonce":186841544},{"type":"v1","addr":"172.21.15.188:6815","nonce":186841544}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6817","nonce":186841544},{"type":"v1","addr":"172.21.15.188:6818","nonce":186841544}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6822","nonce":186841544},{"type":"v1","addr":"172.21.15.188:6823","nonce":186841544}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6820","nonce":186841544},{"type":"v1","addr":"172.21.15.188:6821","nonce":186841544}]},"public_addr":"172.21.15.188:6815/186841544","cluster_addr":"172.21.15.188:6818/186841544","heartbeat_back_addr":"172.21.15.188:6823/186841544","heartbeat_front_addr":"172.21.15.188:6821/186841544","state":["exists","up"]},{"osd":6,"uuid":"58a65d5d-e090-473a-81b4-ad38e85029f9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6808","nonce":2153499405},{"type":"v1","addr":"172.21.15.188:6809","nonce":2153499405}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6810","nonce":2153499405},{"type":"v1","addr":"172.21.15.188:6811","nonce":2153499405}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6816","nonce":2153499405},{"type":"v1","addr":"172.21.15.188:6819","nonce":2153499405}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6812","nonce":2153499405},{"type":"v1","addr":"172.21.15.188:6814","nonce":2153499405}]},"public_addr":"172.21.15.188:6809/2153499405","cluster_addr":"172.21.15.188:6811/2153499405","heartbeat_back_addr":"172.21.15.188:6819/2153499405","heartbeat_front_addr":"172.21.15.188:6814/2153499405","state":["exists","up"]},{"osd":7,"uuid":"ea5b7f70-3554-4cf8-ac30-f7c59b80678d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6824","nonce":2722096729},{"type":"v1","addr":"172.21.15.188:6825","nonce":2722096729}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6826","nonce":2722096729},{"type":"v1","addr":"172.21.15.188:6827","nonce":2722096729}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6830","nonce":2722096729},{"type":"v1","addr":"172.21.15.188:6831","nonce":2722096729}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6828","nonce":2722096729},{"type":"v1","addr":"172.21.15.188:6829","nonce":2722096729}]},"public_addr":"172.21.15.188:6825/2722096729","cluster_addr":"172.21.15.188:6827/2722096729","heartbeat_back_addr":"172.21.15.188:6831/2722096729","heartbeat_front_addr":"172.21.15.188:6829/2722096729","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-26T07:17:36.995032+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-26T07:17:36.932266+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-26T07:17:36.829304+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-26T07:17:36.804297+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-26T07:17:36.667892+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-26T07:17:36.964868+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-26T07:17:36.827332+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-26T07:17:36.893179+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-11-26T07:18:49.482 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:49.477+0000 7f1498735640 1 -- 172.21.15.143:0/4006105233 >> [v2:172.21.15.143:6832/2150557544,v1:172.21.15.143:6833/2150557544] conn(0x7f1474066470 msgr2=0x7f1474068930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:18:49.482 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:49.477+0000 7f1498735640 1 --2- 172.21.15.143:0/4006105233 >> [v2:172.21.15.143:6832/2150557544,v1:172.21.15.143:6833/2150557544] conn(0x7f1474066470 0x7f1474068930 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7f1480009c50 tx=0x7f1480009190 comp rx=0 tx=0).stop 2023-11-26T07:18:49.482 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:49.477+0000 7f1498735640 1 -- 172.21.15.143:0/4006105233 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f14900cd1b0 msgr2=0x7f149013db40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:18:49.483 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:49.477+0000 7f1498735640 1 --2- 172.21.15.143:0/4006105233 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f14900cd1b0 0x7f149013db40 secure :-1 s=READY pgs=382 cs=0 l=1 rev1=1 crypto rx=0x7f148c00c530 tx=0x7f148c00c9f0 comp rx=0 tx=0).stop 2023-11-26T07:18:49.483 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:49.477+0000 7f1498735640 1 -- 172.21.15.143:0/4006105233 shutdown_connections 2023-11-26T07:18:49.483 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:49.477+0000 7f1498735640 1 --2- 172.21.15.143:0/4006105233 >> [v2:172.21.15.143:6832/2150557544,v1:172.21.15.143:6833/2150557544] conn(0x7f1474066470 0x7f1474068930 unknown :-1 s=CLOSED pgs=210 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:18:49.483 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:49.477+0000 7f1498735640 1 --2- 172.21.15.143:0/4006105233 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f1490143ac0 0x7f1490159ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:18:49.483 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:49.477+0000 7f1498735640 1 --2- 172.21.15.143:0/4006105233 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f14900cf280 0x7f1490163f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:18:49.483 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:49.477+0000 7f1498735640 1 --2- 172.21.15.143:0/4006105233 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f14900cd1b0 0x7f149013db40 unknown :-1 s=CLOSED pgs=382 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:18:49.483 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:49.477+0000 7f1498735640 1 -- 172.21.15.143:0/4006105233 >> 172.21.15.143:0/4006105233 conn(0x7f149006a890 msgr2=0x7f149015c7a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:18:49.483 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:49.477+0000 7f1498735640 1 -- 172.21.15.143:0/4006105233 shutdown_connections 2023-11-26T07:18:49.483 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:49.477+0000 7f1498735640 1 -- 172.21.15.143:0/4006105233 wait complete. 2023-11-26T07:18:49.501 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-11-26T07:17:41.758597+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2023-11-26T07:18:49.501 DEBUG:teuthology.orchestra.run.smithi143:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-11-26T07:18:49.643 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:49.637+0000 7fb0471c4640 1 Processor -- start 2023-11-26T07:18:49.643 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:49.637+0000 7fb0471c4640 1 -- start start 2023-11-26T07:18:49.645 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:49.637+0000 7fb0471c4640 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7fb0400d1bc0 0x7fb0400d0df0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:18:49.645 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:49.637+0000 7fb0471c4640 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7fb0400b2160 0x7fb0400b0750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:18:49.645 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:49.637+0000 7fb0471c4640 1 -- --> v1:172.21.15.188:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb0400cf0e0 con 0x7fb04015d6e0 2023-11-26T07:18:49.645 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:49.637+0000 7fb0471c4640 1 -- --> v2:172.21.15.143:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb04015dac0 con 0x7fb0400d1bc0 2023-11-26T07:18:49.645 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:49.637+0000 7fb0471c4640 1 -- --> v2:172.21.15.188:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb0400ceb20 con 0x7fb0400b2160 2023-11-26T07:18:49.645 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:49.641+0000 7fb03ffff640 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7fb0400d1bc0 0x7fb0400d0df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:18:49.645 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:49.641+0000 7fb03ffff640 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7fb0400d1bc0 0x7fb0400d0df0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:47476/0 (socket says 172.21.15.143:47476) 2023-11-26T07:18:49.645 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:49.641+0000 7fb03ffff640 1 -- 172.21.15.143:0/3300557478 learned_addr learned my addr 172.21.15.143:0/3300557478 (peer_addr_for_me v2:172.21.15.143:0/0) 2023-11-26T07:18:49.645 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:49.641+0000 7fb045761640 1 --2- 172.21.15.143:0/3300557478 >> v2:172.21.15.188:3300/0 conn(0x7fb0400b2160 0x7fb0400b0750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:18:49.646 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:49.641+0000 7fb03ffff640 1 -- 172.21.15.143:0/3300557478 >> v2:172.21.15.188:3300/0 conn(0x7fb0400b2160 msgr2=0x7fb0400b0750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-26T07:18:49.646 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:49.641+0000 7fb03ffff640 1 --2- 172.21.15.143:0/3300557478 >> v2:172.21.15.188:3300/0 conn(0x7fb0400b2160 0x7fb0400b0750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:18:49.646 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:49.641+0000 7fb03ffff640 1 -- 172.21.15.143:0/3300557478 >> v1:172.21.15.188:6789/0 conn(0x7fb04015d6e0 legacy=0x7fb0400d14b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:18:49.646 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:49.641+0000 7fb03ffff640 1 -- 172.21.15.143:0/3300557478 --> v2:172.21.15.143:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb0400cee00 con 0x7fb0400d1bc0 2023-11-26T07:18:49.648 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:49.641+0000 7fb03ffff640 1 --2- 172.21.15.143:0/3300557478 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fb0400d1bc0 0x7fb0400d0df0 secure :-1 s=READY pgs=383 cs=0 l=1 rev1=1 crypto rx=0x7fb04008d820 tx=0x7fb02c01c500 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d8b7cbedb27f2b96 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:18:49.648 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:49.641+0000 7fb03f7fe640 1 -- 172.21.15.143:0/3300557478 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb02c0ab5f0 con 0x7fb0400d1bc0 2023-11-26T07:18:49.648 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:49.641+0000 7fb03f7fe640 1 -- 172.21.15.143:0/3300557478 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb02c0ab750 con 0x7fb0400d1bc0 2023-11-26T07:18:49.648 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:49.641+0000 7fb03f7fe640 1 -- 172.21.15.143:0/3300557478 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb02c0abae0 con 0x7fb0400d1bc0 2023-11-26T07:18:49.649 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:49.645+0000 7fb0471c4640 1 -- 172.21.15.143:0/3300557478 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fb0400d1bc0 msgr2=0x7fb0400d0df0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:18:49.649 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:49.645+0000 7fb0471c4640 1 --2- 172.21.15.143:0/3300557478 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fb0400d1bc0 0x7fb0400d0df0 secure :-1 s=READY pgs=383 cs=0 l=1 rev1=1 crypto rx=0x7fb04008d820 tx=0x7fb02c01c500 comp rx=0 tx=0).stop 2023-11-26T07:18:49.649 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:49.645+0000 7fb0471c4640 1 -- 172.21.15.143:0/3300557478 shutdown_connections 2023-11-26T07:18:49.649 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:49.645+0000 7fb0471c4640 1 --2- 172.21.15.143:0/3300557478 >> v2:172.21.15.188:3300/0 conn(0x7fb0400b2160 0x7fb0400b0750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:18:49.649 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:49.645+0000 7fb0471c4640 1 --2- 172.21.15.143:0/3300557478 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fb0400d1bc0 0x7fb0400d0df0 unknown :-1 s=CLOSED pgs=383 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:18:49.649 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:49.645+0000 7fb0471c4640 1 -- 172.21.15.143:0/3300557478 >> 172.21.15.143:0/3300557478 conn(0x7fb04009c550 msgr2=0x7fb0400d3220 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:18:49.650 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:49.645+0000 7fb0471c4640 1 -- 172.21.15.143:0/3300557478 shutdown_connections 2023-11-26T07:18:49.650 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:49.645+0000 7fb0471c4640 1 -- 172.21.15.143:0/3300557478 wait complete. 2023-11-26T07:18:49.651 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:49.645+0000 7fb0471c4640 1 Processor -- start 2023-11-26T07:18:49.651 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:49.645+0000 7fb0471c4640 1 -- start start 2023-11-26T07:18:49.653 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:49.645+0000 7fb0471c4640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fb0400b2160 0x7fb0400b8630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:18:49.653 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:49.645+0000 7fb0471c4640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fb0400d1bc0 0x7fb0400b6e90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:18:49.653 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:49.645+0000 7fb0471c4640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fb04015d6e0 0x7fb0400c6d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:18:49.653 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:49.645+0000 7fb0471c4640 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb040154270 con 0x7fb0400b2160 2023-11-26T07:18:49.653 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:49.645+0000 7fb0471c4640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb0400b7590 con 0x7fb04015d6e0 2023-11-26T07:18:49.653 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:49.645+0000 7fb0471c4640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb0400b7870 con 0x7fb0400d1bc0 2023-11-26T07:18:49.653 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:49.645+0000 7fb044f60640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fb0400b2160 0x7fb0400b8630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:18:49.653 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:49.645+0000 7fb044f60640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fb0400b2160 0x7fb0400b8630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:47488/0 (socket says 172.21.15.143:47488) 2023-11-26T07:18:49.653 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:49.645+0000 7fb044f60640 1 -- 172.21.15.143:0/3360801180 learned_addr learned my addr 172.21.15.143:0/3360801180 (peer_addr_for_me v2:172.21.15.143:0/0) 2023-11-26T07:18:49.654 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:49.645+0000 7fb03ffff640 1 --2- 172.21.15.143:0/3360801180 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fb0400d1bc0 0x7fb0400b6e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:18:49.654 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:49.645+0000 7fb045761640 1 --2- 172.21.15.143:0/3360801180 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fb04015d6e0 0x7fb0400c6d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:18:49.654 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:49.649+0000 7fb044f60640 1 -- 172.21.15.143:0/3360801180 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fb0400d1bc0 msgr2=0x7fb0400b6e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:18:49.654 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:49.649+0000 7fb044f60640 1 --2- 172.21.15.143:0/3360801180 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fb0400d1bc0 0x7fb0400b6e90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:18:49.654 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:49.649+0000 7fb044f60640 1 -- 172.21.15.143:0/3360801180 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fb04015d6e0 msgr2=0x7fb0400c6d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:18:49.654 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:49.649+0000 7fb044f60640 1 --2- 172.21.15.143:0/3360801180 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fb04015d6e0 0x7fb0400c6d50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:18:49.654 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:49.649+0000 7fb044f60640 1 -- 172.21.15.143:0/3360801180 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb02c0094d0 con 0x7fb0400b2160 2023-11-26T07:18:49.654 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:49.649+0000 7fb044f60640 1 --2- 172.21.15.143:0/3360801180 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fb0400b2160 0x7fb0400b8630 secure :-1 s=READY pgs=384 cs=0 l=1 rev1=1 crypto rx=0x7fb03400c3c0 tx=0x7fb03400c880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:18:49.655 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:49.649+0000 7fb03d7fa640 1 -- 172.21.15.143:0/3360801180 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb0340052d0 con 0x7fb0400b2160 2023-11-26T07:18:49.655 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:49.649+0000 7fb03d7fa640 1 -- 172.21.15.143:0/3360801180 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb034005430 con 0x7fb0400b2160 2023-11-26T07:18:49.655 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:49.649+0000 7fb0471c4640 1 -- 172.21.15.143:0/3360801180 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb0400b7b50 con 0x7fb0400b2160 2023-11-26T07:18:49.655 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:49.649+0000 7fb03d7fa640 1 -- 172.21.15.143:0/3360801180 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb034003840 con 0x7fb0400b2160 2023-11-26T07:18:49.655 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:49.649+0000 7fb0471c4640 1 -- 172.21.15.143:0/3360801180 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb0400b8110 con 0x7fb0400b2160 2023-11-26T07:18:49.655 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:49.649+0000 7fb0471c4640 1 -- 172.21.15.143:0/3360801180 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb040067680 con 0x7fb0400b2160 2023-11-26T07:18:49.656 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:49.649+0000 7fb03d7fa640 1 -- 172.21.15.143:0/3360801180 <== mon.0 v2:172.21.15.143:3300/0 4 ==== mgrmap(e 5) v1 ==== 85852+0+0 (secure 0 0 0) 0x7fb0340039a0 con 0x7fb0400b2160 2023-11-26T07:18:49.656 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:49.649+0000 7fb03d7fa640 1 --2- 172.21.15.143:0/3360801180 >> [v2:172.21.15.143:6832/2150557544,v1:172.21.15.143:6833/2150557544] conn(0x7fb024066470 0x7fb024068930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:18:49.657 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:49.653+0000 7fb03d7fa640 1 -- 172.21.15.143:0/3360801180 <== mon.0 v2:172.21.15.143:3300/0 5 ==== osd_map(29..29 src has 1..29) v4 ==== 4857+0+0 (secure 0 0 0) 0x7fb034082fe0 con 0x7fb0400b2160 2023-11-26T07:18:49.657 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:49.653+0000 7fb03ffff640 1 --2- 172.21.15.143:0/3360801180 >> [v2:172.21.15.143:6832/2150557544,v1:172.21.15.143:6833/2150557544] conn(0x7fb024066470 0x7fb024068930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:18:49.657 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:49.653+0000 7fb03ffff640 1 --2- 172.21.15.143:0/3360801180 >> [v2:172.21.15.143:6832/2150557544,v1:172.21.15.143:6833/2150557544] conn(0x7fb024066470 0x7fb024068930 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7fb02c0af000 tx=0x7fb02c0b6040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:18:49.660 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:49.653+0000 7fb03d7fa640 1 -- 172.21.15.143:0/3360801180 <== mon.0 v2:172.21.15.143:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172649 (secure 0 0 0) 0x7fb034050820 con 0x7fb0400b2160 2023-11-26T07:18:49.812 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:49.805+0000 7fb0471c4640 1 -- 172.21.15.143:0/3360801180 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7fb0400cdcf0 con 0x7fb0400b2160 2023-11-26T07:18:49.812 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:49.805+0000 7fb03d7fa640 1 -- 172.21.15.143:0/3360801180 <== mon.0 v2:172.21.15.143:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v29) v1 ==== 93+0+10 (secure 0 0 0) 0x7fb034017020 con 0x7fb0400b2160 2023-11-26T07:18:49.813 INFO:teuthology.orchestra.run.smithi143.stdout:pg_num: 1 2023-11-26T07:18:49.814 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:49.809+0000 7fb0471c4640 1 -- 172.21.15.143:0/3360801180 >> [v2:172.21.15.143:6832/2150557544,v1:172.21.15.143:6833/2150557544] conn(0x7fb024066470 msgr2=0x7fb024068930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:18:49.815 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:49.809+0000 7fb0471c4640 1 --2- 172.21.15.143:0/3360801180 >> [v2:172.21.15.143:6832/2150557544,v1:172.21.15.143:6833/2150557544] conn(0x7fb024066470 0x7fb024068930 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7fb02c0af000 tx=0x7fb02c0b6040 comp rx=0 tx=0).stop 2023-11-26T07:18:49.815 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:49.809+0000 7fb0471c4640 1 -- 172.21.15.143:0/3360801180 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fb0400b2160 msgr2=0x7fb0400b8630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:18:49.815 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:49.809+0000 7fb0471c4640 1 --2- 172.21.15.143:0/3360801180 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fb0400b2160 0x7fb0400b8630 secure :-1 s=READY pgs=384 cs=0 l=1 rev1=1 crypto rx=0x7fb03400c3c0 tx=0x7fb03400c880 comp rx=0 tx=0).stop 2023-11-26T07:18:49.815 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:49.809+0000 7fb0471c4640 1 -- 172.21.15.143:0/3360801180 shutdown_connections 2023-11-26T07:18:49.815 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:49.809+0000 7fb0471c4640 1 --2- 172.21.15.143:0/3360801180 >> [v2:172.21.15.143:6832/2150557544,v1:172.21.15.143:6833/2150557544] conn(0x7fb024066470 0x7fb024068930 unknown :-1 s=CLOSED pgs=211 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:18:49.815 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:49.809+0000 7fb0471c4640 1 --2- 172.21.15.143:0/3360801180 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fb04015d6e0 0x7fb0400c6d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:18:49.815 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:49.809+0000 7fb0471c4640 1 --2- 172.21.15.143:0/3360801180 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fb0400d1bc0 0x7fb0400b6e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:18:49.815 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:49.809+0000 7fb0471c4640 1 --2- 172.21.15.143:0/3360801180 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fb0400b2160 0x7fb0400b8630 unknown :-1 s=CLOSED pgs=384 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:18:49.815 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:49.809+0000 7fb0471c4640 1 -- 172.21.15.143:0/3360801180 >> 172.21.15.143:0/3360801180 conn(0x7fb04009c550 msgr2=0x7fb0400cf490 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:18:49.815 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:49.809+0000 7fb0471c4640 1 -- 172.21.15.143:0/3360801180 shutdown_connections 2023-11-26T07:18:49.815 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:49.809+0000 7fb0471c4640 1 -- 172.21.15.143:0/3360801180 wait complete. 2023-11-26T07:18:49.834 DEBUG:teuthology.orchestra.run.smithi143:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --format=json-pretty osd lspools 2023-11-26T07:18:49.971 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:49.965+0000 7fdb5b5b8640 1 Processor -- start 2023-11-26T07:18:49.972 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:49.965+0000 7fdb5b5b8640 1 -- start start 2023-11-26T07:18:49.974 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:49.969+0000 7fdb5b5b8640 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7fdb540cf990 0x7fdb540708c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:18:49.974 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:49.969+0000 7fdb5b5b8640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fdb54070e00 0x7fdb54162510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:18:49.974 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:49.969+0000 7fdb5b5b8640 1 -- --> v1:172.21.15.143:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fdb54162080 con 0x7fdb5414b1e0 2023-11-26T07:18:49.974 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:49.969+0000 7fdb5b5b8640 1 -- --> v2:172.21.15.143:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fdb54161d70 con 0x7fdb540cf990 2023-11-26T07:18:49.974 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:49.969+0000 7fdb5b5b8640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdb5414b5c0 con 0x7fdb54070e00 2023-11-26T07:18:49.974 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:49.969+0000 7fdb59354640 1 --1- >> v1:172.21.15.143:6789/0 conn(0x7fdb5414b1e0 0x7fdb540cf280 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.143:6789/0 says I am v1:172.21.15.143:55168/0 (socket says 172.21.15.143:55168) 2023-11-26T07:18:49.975 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:49.969+0000 7fdb59354640 1 -- 172.21.15.143:0/290935355 learned_addr learned my addr 172.21.15.143:0/290935355 (peer_addr_for_me v1:172.21.15.143:0/0) 2023-11-26T07:18:49.975 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:49.969+0000 7fdb58b53640 1 --2- 172.21.15.143:0/290935355 >> v2:172.21.15.143:3300/0 conn(0x7fdb540cf990 0x7fdb540708c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:18:49.975 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:49.969+0000 7fdb59b55640 1 --2- 172.21.15.143:0/290935355 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fdb54070e00 0x7fdb54162510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:18:49.975 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:49.969+0000 7fdb4bfff640 1 -- 172.21.15.143:0/290935355 <== mon.0 v1:172.21.15.143:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2501003657 0 0) 0x7fdb54162080 con 0x7fdb5414b1e0 2023-11-26T07:18:49.975 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:49.969+0000 7fdb4bfff640 1 -- 172.21.15.143:0/290935355 --> v1:172.21.15.143:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fdb40003530 con 0x7fdb5414b1e0 2023-11-26T07:18:49.975 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:49.969+0000 7fdb58b53640 1 -- 172.21.15.143:0/290935355 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fdb54070e00 msgr2=0x7fdb54162510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-26T07:18:49.975 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:49.969+0000 7fdb58b53640 1 --2- 172.21.15.143:0/290935355 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fdb54070e00 0x7fdb54162510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:18:49.975 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:49.969+0000 7fdb58b53640 1 -- 172.21.15.143:0/290935355 >> v1:172.21.15.143:6789/0 conn(0x7fdb5414b1e0 legacy=0x7fdb540cf280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:18:49.975 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:49.969+0000 7fdb58b53640 1 -- 172.21.15.143:0/290935355 --> v2:172.21.15.143:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdb54071580 con 0x7fdb540cf990 2023-11-26T07:18:49.977 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:49.973+0000 7fdb58b53640 1 --2- 172.21.15.143:0/290935355 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fdb540cf990 0x7fdb540708c0 secure :-1 s=READY pgs=386 cs=0 l=1 rev1=1 crypto rx=0x7fdb440097c0 tx=0x7fdb4401c450 comp rx=0 tx=0).ready entity=mon.0 client_cookie=228db005fe089a7a server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:18:49.977 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:49.973+0000 7fdb4bfff640 1 -- 172.21.15.143:0/290935355 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdb440ab650 con 0x7fdb540cf990 2023-11-26T07:18:49.977 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:49.973+0000 7fdb4bfff640 1 -- 172.21.15.143:0/290935355 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdb440ab7b0 con 0x7fdb540cf990 2023-11-26T07:18:49.977 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:49.973+0000 7fdb4bfff640 1 -- 172.21.15.143:0/290935355 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdb440abb40 con 0x7fdb540cf990 2023-11-26T07:18:49.978 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:49.973+0000 7fdb5b5b8640 1 -- 172.21.15.143:0/290935355 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fdb540cf990 msgr2=0x7fdb540708c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:18:49.978 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:49.973+0000 7fdb5b5b8640 1 --2- 172.21.15.143:0/290935355 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fdb540cf990 0x7fdb540708c0 secure :-1 s=READY pgs=386 cs=0 l=1 rev1=1 crypto rx=0x7fdb440097c0 tx=0x7fdb4401c450 comp rx=0 tx=0).stop 2023-11-26T07:18:49.978 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:49.973+0000 7fdb5b5b8640 1 -- 172.21.15.143:0/290935355 shutdown_connections 2023-11-26T07:18:49.978 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:49.973+0000 7fdb5b5b8640 1 --2- 172.21.15.143:0/290935355 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fdb54070e00 0x7fdb54162510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:18:49.978 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:49.973+0000 7fdb5b5b8640 1 --2- 172.21.15.143:0/290935355 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fdb540cf990 0x7fdb540708c0 unknown :-1 s=CLOSED pgs=386 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:18:49.979 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:49.973+0000 7fdb5b5b8640 1 -- 172.21.15.143:0/290935355 >> 172.21.15.143:0/290935355 conn(0x7fdb5406a890 msgr2=0x7fdb5406aca0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:18:49.979 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:49.973+0000 7fdb5b5b8640 1 -- 172.21.15.143:0/290935355 shutdown_connections 2023-11-26T07:18:49.979 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:49.973+0000 7fdb5b5b8640 1 -- 172.21.15.143:0/290935355 wait complete. 2023-11-26T07:18:49.979 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:49.973+0000 7fdb5b5b8640 1 Processor -- start 2023-11-26T07:18:49.980 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:49.973+0000 7fdb5b5b8640 1 -- start start 2023-11-26T07:18:49.981 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:49.977+0000 7fdb5b5b8640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fdb54070e00 0x7fdb54142fd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:18:49.981 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:49.977+0000 7fdb5b5b8640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fdb540cf990 0x7fdb540afa20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:18:49.981 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:49.977+0000 7fdb5b5b8640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fdb5414b1e0 0x7fdb540aff60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:18:49.982 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:49.977+0000 7fdb5b5b8640 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdb5414bba0 con 0x7fdb5414b1e0 2023-11-26T07:18:49.982 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:49.977+0000 7fdb5b5b8640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdb54161980 con 0x7fdb54070e00 2023-11-26T07:18:49.982 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:49.977+0000 7fdb5b5b8640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdb541613c0 con 0x7fdb540cf990 2023-11-26T07:18:49.982 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:49.977+0000 7fdb59b55640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fdb5414b1e0 0x7fdb540aff60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:18:49.982 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:49.977+0000 7fdb59b55640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fdb5414b1e0 0x7fdb540aff60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:47506/0 (socket says 172.21.15.143:47506) 2023-11-26T07:18:49.982 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:49.977+0000 7fdb59b55640 1 -- 172.21.15.143:0/2403183325 learned_addr learned my addr 172.21.15.143:0/2403183325 (peer_addr_for_me v2:172.21.15.143:0/0) 2023-11-26T07:18:49.982 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:49.977+0000 7fdb58b53640 1 --2- 172.21.15.143:0/2403183325 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fdb540cf990 0x7fdb540afa20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:18:49.982 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:49.977+0000 7fdb59354640 1 --2- 172.21.15.143:0/2403183325 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fdb54070e00 0x7fdb54142fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:18:49.982 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:49.977+0000 7fdb59b55640 1 -- 172.21.15.143:0/2403183325 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fdb540cf990 msgr2=0x7fdb540afa20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:18:49.982 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:49.977+0000 7fdb59b55640 1 --2- 172.21.15.143:0/2403183325 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fdb540cf990 0x7fdb540afa20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:18:49.982 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:49.977+0000 7fdb59b55640 1 -- 172.21.15.143:0/2403183325 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fdb54070e00 msgr2=0x7fdb54142fd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:18:49.983 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:49.977+0000 7fdb59b55640 1 --2- 172.21.15.143:0/2403183325 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fdb54070e00 0x7fdb54142fd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:18:49.983 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:49.977+0000 7fdb59b55640 1 -- 172.21.15.143:0/2403183325 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdb44009420 con 0x7fdb5414b1e0 2023-11-26T07:18:49.983 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:49.977+0000 7fdb59354640 1 --2- 172.21.15.143:0/2403183325 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fdb54070e00 0x7fdb54142fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-26T07:18:49.984 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:49.977+0000 7fdb59b55640 1 --2- 172.21.15.143:0/2403183325 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fdb5414b1e0 0x7fdb540aff60 secure :-1 s=READY pgs=387 cs=0 l=1 rev1=1 crypto rx=0x7fdb4c00efc0 tx=0x7fdb4c00c490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:18:49.984 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:49.977+0000 7fdb49ffb640 1 -- 172.21.15.143:0/2403183325 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdb4c014480 con 0x7fdb5414b1e0 2023-11-26T07:18:49.984 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:49.977+0000 7fdb49ffb640 1 -- 172.21.15.143:0/2403183325 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdb4c00f040 con 0x7fdb5414b1e0 2023-11-26T07:18:49.984 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:49.977+0000 7fdb5b5b8640 1 -- 172.21.15.143:0/2403183325 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdb541616a0 con 0x7fdb5414b1e0 2023-11-26T07:18:49.984 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:49.977+0000 7fdb49ffb640 1 -- 172.21.15.143:0/2403183325 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdb4c014820 con 0x7fdb5414b1e0 2023-11-26T07:18:49.984 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:49.977+0000 7fdb5b5b8640 1 -- 172.21.15.143:0/2403183325 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdb54143510 con 0x7fdb5414b1e0 2023-11-26T07:18:49.984 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:49.977+0000 7fdb5b5b8640 1 -- 172.21.15.143:0/2403183325 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fdb5414bd10 con 0x7fdb5414b1e0 2023-11-26T07:18:49.984 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:49.977+0000 7fdb49ffb640 1 -- 172.21.15.143:0/2403183325 <== mon.0 v2:172.21.15.143:3300/0 4 ==== mgrmap(e 5) v1 ==== 85852+0+0 (secure 0 0 0) 0x7fdb4c014980 con 0x7fdb5414b1e0 2023-11-26T07:18:49.985 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:49.981+0000 7fdb49ffb640 1 --2- 172.21.15.143:0/2403183325 >> [v2:172.21.15.143:6832/2150557544,v1:172.21.15.143:6833/2150557544] conn(0x7fdb38066470 0x7fdb38068930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:18:49.985 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:49.981+0000 7fdb49ffb640 1 -- 172.21.15.143:0/2403183325 <== mon.0 v2:172.21.15.143:3300/0 5 ==== osd_map(29..29 src has 1..29) v4 ==== 4857+0+0 (secure 0 0 0) 0x7fdb4c084680 con 0x7fdb5414b1e0 2023-11-26T07:18:49.985 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:49.981+0000 7fdb59354640 1 --2- 172.21.15.143:0/2403183325 >> [v2:172.21.15.143:6832/2150557544,v1:172.21.15.143:6833/2150557544] conn(0x7fdb38066470 0x7fdb38068930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:18:49.986 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:49.981+0000 7fdb59354640 1 --2- 172.21.15.143:0/2403183325 >> [v2:172.21.15.143:6832/2150557544,v1:172.21.15.143:6833/2150557544] conn(0x7fdb38066470 0x7fdb38068930 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7fdb500011f0 tx=0x7fdb50009040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:18:49.989 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:49.985+0000 7fdb49ffb640 1 -- 172.21.15.143:0/2403183325 <== mon.0 v2:172.21.15.143:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172649 (secure 0 0 0) 0x7fdb4c051e40 con 0x7fdb5414b1e0 2023-11-26T07:18:50.155 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:50.149+0000 7fdb5b5b8640 1 -- 172.21.15.143:0/2403183325 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "osd lspools", "format": "json-pretty"} v 0) v1 -- 0x7fdb54067680 con 0x7fdb5414b1e0 2023-11-26T07:18:50.155 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:50.149+0000 7fdb49ffb640 1 -- 172.21.15.143:0/2403183325 <== mon.0 v2:172.21.15.143:3300/0 7 ==== mon_command_ack([{"prefix": "osd lspools", "format": "json-pretty"}]=0 v29) v1 ==== 84+0+65 (secure 0 0 0) 0x7fdb4c018060 con 0x7fdb5414b1e0 2023-11-26T07:18:50.155 INFO:teuthology.orchestra.run.smithi143.stdout: 2023-11-26T07:18:50.156 INFO:teuthology.orchestra.run.smithi143.stdout:[ 2023-11-26T07:18:50.156 INFO:teuthology.orchestra.run.smithi143.stdout: { 2023-11-26T07:18:50.156 INFO:teuthology.orchestra.run.smithi143.stdout: "poolnum": 1, 2023-11-26T07:18:50.156 INFO:teuthology.orchestra.run.smithi143.stdout: "poolname": ".mgr" 2023-11-26T07:18:50.156 INFO:teuthology.orchestra.run.smithi143.stdout: } 2023-11-26T07:18:50.156 INFO:teuthology.orchestra.run.smithi143.stdout:] 2023-11-26T07:18:50.158 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:50.153+0000 7fdb5b5b8640 1 -- 172.21.15.143:0/2403183325 >> [v2:172.21.15.143:6832/2150557544,v1:172.21.15.143:6833/2150557544] conn(0x7fdb38066470 msgr2=0x7fdb38068930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:18:50.158 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:50.153+0000 7fdb5b5b8640 1 --2- 172.21.15.143:0/2403183325 >> [v2:172.21.15.143:6832/2150557544,v1:172.21.15.143:6833/2150557544] conn(0x7fdb38066470 0x7fdb38068930 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7fdb500011f0 tx=0x7fdb50009040 comp rx=0 tx=0).stop 2023-11-26T07:18:50.158 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:50.153+0000 7fdb5b5b8640 1 -- 172.21.15.143:0/2403183325 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fdb5414b1e0 msgr2=0x7fdb540aff60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:18:50.158 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:50.153+0000 7fdb5b5b8640 1 --2- 172.21.15.143:0/2403183325 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fdb5414b1e0 0x7fdb540aff60 secure :-1 s=READY pgs=387 cs=0 l=1 rev1=1 crypto rx=0x7fdb4c00efc0 tx=0x7fdb4c00c490 comp rx=0 tx=0).stop 2023-11-26T07:18:50.158 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:50.153+0000 7fdb5b5b8640 1 -- 172.21.15.143:0/2403183325 shutdown_connections 2023-11-26T07:18:50.158 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:50.153+0000 7fdb5b5b8640 1 --2- 172.21.15.143:0/2403183325 >> [v2:172.21.15.143:6832/2150557544,v1:172.21.15.143:6833/2150557544] conn(0x7fdb38066470 0x7fdb38068930 unknown :-1 s=CLOSED pgs=212 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:18:50.158 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:50.153+0000 7fdb5b5b8640 1 --2- 172.21.15.143:0/2403183325 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fdb5414b1e0 0x7fdb540aff60 unknown :-1 s=CLOSED pgs=387 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:18:50.158 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:50.153+0000 7fdb5b5b8640 1 --2- 172.21.15.143:0/2403183325 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fdb540cf990 0x7fdb540afa20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:18:50.158 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:50.153+0000 7fdb5b5b8640 1 --2- 172.21.15.143:0/2403183325 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fdb54070e00 0x7fdb54142fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:18:50.159 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:50.153+0000 7fdb5b5b8640 1 -- 172.21.15.143:0/2403183325 >> 172.21.15.143:0/2403183325 conn(0x7fdb5406a890 msgr2=0x7fdb540cf250 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:18:50.159 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:50.153+0000 7fdb5b5b8640 1 -- 172.21.15.143:0/2403183325 shutdown_connections 2023-11-26T07:18:50.159 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:50.153+0000 7fdb5b5b8640 1 -- 172.21.15.143:0/2403183325 wait complete. 2023-11-26T07:18:50.177 DEBUG:tasks.cephfs.filesystem:Creating filesystem 'cephfs' 2023-11-26T07:18:50.177 DEBUG:teuthology.orchestra.run.smithi143:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_metadata --pg_num_min 64 2023-11-26T07:18:50.320 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:50.313+0000 7f3fbcf8d640 1 Processor -- start 2023-11-26T07:18:50.320 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:50.313+0000 7f3fbcf8d640 1 -- start start 2023-11-26T07:18:50.322 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:50.317+0000 7f3fbcf8d640 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7f3fb815d6e0 0x7f3fb80d0df0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:18:50.322 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:50.317+0000 7f3fbcf8d640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f3fb80b2160 0x7f3fb80d14b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:18:50.322 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:50.317+0000 7f3fbcf8d640 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7f3fb80d19f0 0x7f3fb80cfbf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:18:50.322 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:50.317+0000 7f3fbcf8d640 1 -- --> v2:172.21.15.143:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3fb815dac0 con 0x7f3fb80d19f0 2023-11-26T07:18:50.322 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:50.317+0000 7f3fbcf8d640 1 -- --> v2:172.21.15.188:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3fb80d1e80 con 0x7f3fb815d6e0 2023-11-26T07:18:50.322 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:50.317+0000 7f3fbcf8d640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3fb80cf0e0 con 0x7f3fb80b2160 2023-11-26T07:18:50.323 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:50.317+0000 7f3fb6d9d640 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7f3fb80d19f0 0x7f3fb80cfbf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:18:50.323 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:50.317+0000 7f3fb6d9d640 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7f3fb80d19f0 0x7f3fb80cfbf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:47518/0 (socket says 172.21.15.143:47518) 2023-11-26T07:18:50.323 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:50.317+0000 7f3fb6d9d640 1 -- 172.21.15.143:0/3287050808 learned_addr learned my addr 172.21.15.143:0/3287050808 (peer_addr_for_me v2:172.21.15.143:0/0) 2023-11-26T07:18:50.323 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:50.317+0000 7f3faffff640 1 --2- 172.21.15.143:0/3287050808 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f3fb80b2160 0x7f3fb80d14b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:18:50.323 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:50.317+0000 7f3fb659c640 1 --2- 172.21.15.143:0/3287050808 >> v2:172.21.15.188:3300/0 conn(0x7f3fb815d6e0 0x7f3fb80d0df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:18:50.323 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:50.317+0000 7f3fb6d9d640 1 -- 172.21.15.143:0/3287050808 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f3fb80b2160 msgr2=0x7f3fb80d14b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-26T07:18:50.324 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:50.317+0000 7f3fb6d9d640 1 --2- 172.21.15.143:0/3287050808 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f3fb80b2160 0x7f3fb80d14b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:18:50.324 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:50.317+0000 7f3fb6d9d640 1 -- 172.21.15.143:0/3287050808 >> v2:172.21.15.188:3300/0 conn(0x7f3fb815d6e0 msgr2=0x7f3fb80d0df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-26T07:18:50.324 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:50.317+0000 7f3fb6d9d640 1 --2- 172.21.15.143:0/3287050808 >> v2:172.21.15.188:3300/0 conn(0x7f3fb815d6e0 0x7f3fb80d0df0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:18:50.324 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:50.317+0000 7f3fb6d9d640 1 -- 172.21.15.143:0/3287050808 --> v2:172.21.15.143:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3fb80ceb20 con 0x7f3fb80d19f0 2023-11-26T07:18:50.324 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:50.317+0000 7f3fb659c640 1 --2- 172.21.15.143:0/3287050808 >> v2:172.21.15.188:3300/0 conn(0x7f3fb815d6e0 0x7f3fb80d0df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-26T07:18:50.327 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:50.321+0000 7f3fb6d9d640 1 --2- 172.21.15.143:0/3287050808 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f3fb80d19f0 0x7f3fb80cfbf0 secure :-1 s=READY pgs=388 cs=0 l=1 rev1=1 crypto rx=0x7f3fb80b5db0 tx=0x7f3fb001c710 comp rx=0 tx=0).ready entity=mon.0 client_cookie=726e5aed0c6f4d21 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:18:50.327 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:50.321+0000 7f3fb5d9b640 1 -- 172.21.15.143:0/3287050808 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3fb00a6490 con 0x7f3fb80d19f0 2023-11-26T07:18:50.327 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:50.321+0000 7f3fb5d9b640 1 -- 172.21.15.143:0/3287050808 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3fb00a65f0 con 0x7f3fb80d19f0 2023-11-26T07:18:50.328 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:50.321+0000 7f3fb5d9b640 1 -- 172.21.15.143:0/3287050808 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3fb00a6980 con 0x7f3fb80d19f0 2023-11-26T07:18:50.328 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:50.321+0000 7f3fbcf8d640 1 -- 172.21.15.143:0/3287050808 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f3fb80d19f0 msgr2=0x7f3fb80cfbf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:18:50.328 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:50.321+0000 7f3fbcf8d640 1 --2- 172.21.15.143:0/3287050808 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f3fb80d19f0 0x7f3fb80cfbf0 secure :-1 s=READY pgs=388 cs=0 l=1 rev1=1 crypto rx=0x7f3fb80b5db0 tx=0x7f3fb001c710 comp rx=0 tx=0).stop 2023-11-26T07:18:50.329 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:50.321+0000 7f3fbcf8d640 1 -- 172.21.15.143:0/3287050808 shutdown_connections 2023-11-26T07:18:50.329 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:50.321+0000 7f3fbcf8d640 1 --2- 172.21.15.143:0/3287050808 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f3fb80d19f0 0x7f3fb80cfbf0 unknown :-1 s=CLOSED pgs=388 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:18:50.329 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:50.321+0000 7f3fbcf8d640 1 --2- 172.21.15.143:0/3287050808 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f3fb80b2160 0x7f3fb80d14b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:18:50.329 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:50.321+0000 7f3fbcf8d640 1 --2- 172.21.15.143:0/3287050808 >> v2:172.21.15.188:3300/0 conn(0x7f3fb815d6e0 0x7f3fb80d0df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:18:50.329 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:50.321+0000 7f3fbcf8d640 1 -- 172.21.15.143:0/3287050808 >> 172.21.15.143:0/3287050808 conn(0x7f3fb809c550 msgr2=0x7f3fb80d3220 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:18:50.329 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:50.321+0000 7f3fbcf8d640 1 -- 172.21.15.143:0/3287050808 shutdown_connections 2023-11-26T07:18:50.329 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:50.321+0000 7f3fbcf8d640 1 -- 172.21.15.143:0/3287050808 wait complete. 2023-11-26T07:18:50.330 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:50.325+0000 7f3fbcf8d640 1 Processor -- start 2023-11-26T07:18:50.330 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:50.325+0000 7f3fbcf8d640 1 -- start start 2023-11-26T07:18:50.331 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:50.325+0000 7f3fbcf8d640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f3fb80b2160 0x7f3fb8151230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:18:50.332 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:50.325+0000 7f3fbcf8d640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f3fb80d19f0 0x7f3fb80b6800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:18:50.332 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:50.325+0000 7f3fbcf8d640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f3fb815d6e0 0x7f3fb80b6d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:18:50.332 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:50.325+0000 7f3fbcf8d640 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3fb806c4d0 con 0x7f3fb80b2160 2023-11-26T07:18:50.332 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:50.325+0000 7f3fbcf8d640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3fb81510b0 con 0x7f3fb815d6e0 2023-11-26T07:18:50.332 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:50.325+0000 7f3fbcf8d640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3fb806bb70 con 0x7f3fb80d19f0 2023-11-26T07:18:50.332 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:50.325+0000 7f3fb659c640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f3fb80b2160 0x7f3fb8151230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:18:50.332 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:50.325+0000 7f3fb659c640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f3fb80b2160 0x7f3fb8151230 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:47530/0 (socket says 172.21.15.143:47530) 2023-11-26T07:18:50.332 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:50.325+0000 7f3fb659c640 1 -- 172.21.15.143:0/3304173664 learned_addr learned my addr 172.21.15.143:0/3304173664 (peer_addr_for_me v2:172.21.15.143:0/0) 2023-11-26T07:18:50.332 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:50.325+0000 7f3fb659c640 1 -- 172.21.15.143:0/3304173664 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f3fb80d19f0 msgr2=0x7f3fb80b6800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:18:50.332 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:50.325+0000 7f3fb659c640 1 --2- 172.21.15.143:0/3304173664 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f3fb80d19f0 0x7f3fb80b6800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:18:50.332 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:50.325+0000 7f3fb659c640 1 -- 172.21.15.143:0/3304173664 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f3fb815d6e0 msgr2=0x7f3fb80b6d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:18:50.333 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:50.325+0000 7f3fb659c640 1 --2- 172.21.15.143:0/3304173664 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f3fb815d6e0 0x7f3fb80b6d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:18:50.333 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:50.325+0000 7f3fb659c640 1 -- 172.21.15.143:0/3304173664 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3fb0009b30 con 0x7f3fb80b2160 2023-11-26T07:18:50.334 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:50.325+0000 7f3fb659c640 1 --2- 172.21.15.143:0/3304173664 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f3fb80b2160 0x7f3fb8151230 secure :-1 s=READY pgs=389 cs=0 l=1 rev1=1 crypto rx=0x7f3fa800d800 tx=0x7f3fa800dcc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:18:50.334 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:50.325+0000 7f3faeffd640 1 -- 172.21.15.143:0/3304173664 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3fa80096c0 con 0x7f3fb80b2160 2023-11-26T07:18:50.334 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:50.325+0000 7f3faeffd640 1 -- 172.21.15.143:0/3304173664 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3fa8009820 con 0x7f3fb80b2160 2023-11-26T07:18:50.334 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:50.325+0000 7f3faeffd640 1 -- 172.21.15.143:0/3304173664 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3fa8009bd0 con 0x7f3fb80b2160 2023-11-26T07:18:50.334 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:50.325+0000 7f3fbcf8d640 1 -- 172.21.15.143:0/3304173664 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3fb806be10 con 0x7f3fb80b2160 2023-11-26T07:18:50.334 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:50.325+0000 7f3fbcf8d640 1 -- 172.21.15.143:0/3304173664 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3fb806c8f0 con 0x7f3fb80b2160 2023-11-26T07:18:50.334 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:50.329+0000 7f3fbcf8d640 1 -- 172.21.15.143:0/3304173664 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3fb8067680 con 0x7f3fb80b2160 2023-11-26T07:18:50.334 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:50.329+0000 7f3faeffd640 1 -- 172.21.15.143:0/3304173664 <== mon.0 v2:172.21.15.143:3300/0 4 ==== mgrmap(e 5) v1 ==== 85852+0+0 (secure 0 0 0) 0x7f3fa8009d30 con 0x7f3fb80b2160 2023-11-26T07:18:50.336 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:50.329+0000 7f3faeffd640 1 --2- 172.21.15.143:0/3304173664 >> [v2:172.21.15.143:6832/2150557544,v1:172.21.15.143:6833/2150557544] conn(0x7f3f98066470 0x7f3f98068930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:18:50.336 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:50.329+0000 7f3faeffd640 1 -- 172.21.15.143:0/3304173664 <== mon.0 v2:172.21.15.143:3300/0 5 ==== osd_map(29..29 src has 1..29) v4 ==== 4857+0+0 (secure 0 0 0) 0x7f3fa8084240 con 0x7f3fb80b2160 2023-11-26T07:18:50.336 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:50.329+0000 7f3faffff640 1 --2- 172.21.15.143:0/3304173664 >> [v2:172.21.15.143:6832/2150557544,v1:172.21.15.143:6833/2150557544] conn(0x7f3f98066470 0x7f3f98068930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:18:50.336 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:50.333+0000 7f3faffff640 1 --2- 172.21.15.143:0/3304173664 >> [v2:172.21.15.143:6832/2150557544,v1:172.21.15.143:6833/2150557544] conn(0x7f3f98066470 0x7f3f98068930 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7f3fb80b5cf0 tx=0x7f3fa0008040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:18:50.339 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:50.333+0000 7f3faeffd640 1 -- 172.21.15.143:0/3304173664 <== mon.0 v2:172.21.15.143:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172649 (secure 0 0 0) 0x7f3fa8051a00 con 0x7f3fb80b2160 2023-11-26T07:18:50.491 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:50.485+0000 7f3fbcf8d640 1 -- 172.21.15.143:0/3304173664 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64} v 0) v1 -- 0x7f3fb8067d40 con 0x7f3fb80b2160 2023-11-26T07:18:50.626 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:50.621+0000 7f3faeffd640 1 -- 172.21.15.143:0/3304173664 <== mon.0 v2:172.21.15.143:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]=0 pool 'cephfs_metadata' created v30) v1 ==== 138+0+0 (secure 0 0 0) 0x7f3fa8055020 con 0x7f3fb80b2160 2023-11-26T07:18:50.626 INFO:teuthology.orchestra.run.smithi143.stderr:pool 'cephfs_metadata' created 2023-11-26T07:18:50.628 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:50.621+0000 7f3fbcf8d640 1 -- 172.21.15.143:0/3304173664 >> [v2:172.21.15.143:6832/2150557544,v1:172.21.15.143:6833/2150557544] conn(0x7f3f98066470 msgr2=0x7f3f98068930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:18:50.628 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:50.621+0000 7f3fbcf8d640 1 --2- 172.21.15.143:0/3304173664 >> [v2:172.21.15.143:6832/2150557544,v1:172.21.15.143:6833/2150557544] conn(0x7f3f98066470 0x7f3f98068930 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7f3fb80b5cf0 tx=0x7f3fa0008040 comp rx=0 tx=0).stop 2023-11-26T07:18:50.628 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:50.621+0000 7f3fbcf8d640 1 -- 172.21.15.143:0/3304173664 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f3fb80b2160 msgr2=0x7f3fb8151230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:18:50.628 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:50.621+0000 7f3fbcf8d640 1 --2- 172.21.15.143:0/3304173664 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f3fb80b2160 0x7f3fb8151230 secure :-1 s=READY pgs=389 cs=0 l=1 rev1=1 crypto rx=0x7f3fa800d800 tx=0x7f3fa800dcc0 comp rx=0 tx=0).stop 2023-11-26T07:18:50.628 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:50.621+0000 7f3fbcf8d640 1 -- 172.21.15.143:0/3304173664 shutdown_connections 2023-11-26T07:18:50.628 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:50.621+0000 7f3fbcf8d640 1 --2- 172.21.15.143:0/3304173664 >> [v2:172.21.15.143:6832/2150557544,v1:172.21.15.143:6833/2150557544] conn(0x7f3f98066470 0x7f3f98068930 unknown :-1 s=CLOSED pgs=213 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:18:50.629 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:50.621+0000 7f3fbcf8d640 1 --2- 172.21.15.143:0/3304173664 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f3fb815d6e0 0x7f3fb80b6d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:18:50.629 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:50.621+0000 7f3fbcf8d640 1 --2- 172.21.15.143:0/3304173664 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f3fb80d19f0 0x7f3fb80b6800 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:18:50.629 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:50.621+0000 7f3fbcf8d640 1 --2- 172.21.15.143:0/3304173664 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f3fb80b2160 0x7f3fb8151230 unknown :-1 s=CLOSED pgs=389 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:18:50.629 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:50.621+0000 7f3fbcf8d640 1 -- 172.21.15.143:0/3304173664 >> 172.21.15.143:0/3304173664 conn(0x7f3fb809c550 msgr2=0x7f3fb80d3220 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:18:50.629 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:50.621+0000 7f3fbcf8d640 1 -- 172.21.15.143:0/3304173664 shutdown_connections 2023-11-26T07:18:50.629 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:50.621+0000 7f3fbcf8d640 1 -- 172.21.15.143:0/3304173664 wait complete. 2023-11-26T07:18:50.653 DEBUG:teuthology.orchestra.run.smithi143:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_data 64 --pg_num_min 64 --target_size_ratio 0.9 2023-11-26T07:18:50.798 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:50.793+0000 7f988a4c9640 1 Processor -- start 2023-11-26T07:18:50.798 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:50.793+0000 7f988a4c9640 1 -- start start 2023-11-26T07:18:50.800 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:50.793+0000 7f988a4c9640 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7f98840d1c20 0x7f98840d0720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:18:50.800 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:50.793+0000 7f988a4c9640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f98841608f0 0x7f9884160d60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:18:50.800 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:50.793+0000 7f988a4c9640 1 -- --> v1:172.21.15.188:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f98840b2510 con 0x7f98840d1840 2023-11-26T07:18:50.800 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:50.793+0000 7f988a4c9640 1 -- --> v2:172.21.15.143:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9884161f80 con 0x7f98840d1c20 2023-11-26T07:18:50.800 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:50.793+0000 7f988a4c9640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f98840cfc40 con 0x7f98841608f0 2023-11-26T07:18:50.801 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:50.793+0000 7f98837fe640 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7f98840d1c20 0x7f98840d0720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:18:50.801 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:50.793+0000 7f98837fe640 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7f98840d1c20 0x7f98840d0720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:47538/0 (socket says 172.21.15.143:47538) 2023-11-26T07:18:50.801 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:50.793+0000 7f98837fe640 1 -- 172.21.15.143:0/42507114 learned_addr learned my addr 172.21.15.143:0/42507114 (peer_addr_for_me v2:172.21.15.143:0/0) 2023-11-26T07:18:50.801 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:50.793+0000 7f98837fe640 1 -- 172.21.15.143:0/42507114 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f98841608f0 msgr2=0x7f9884160d60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-26T07:18:50.801 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:50.793+0000 7f9888a66640 1 --2- 172.21.15.143:0/42507114 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f98841608f0 0x7f9884160d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:18:50.801 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:50.793+0000 7f98837fe640 1 --2- 172.21.15.143:0/42507114 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f98841608f0 0x7f9884160d60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:18:50.801 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:50.793+0000 7f98837fe640 1 -- 172.21.15.143:0/42507114 >> v1:172.21.15.188:6789/0 conn(0x7f98840d1840 legacy=0x7f9884161410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-26T07:18:50.801 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:50.793+0000 7f98837fe640 1 -- 172.21.15.143:0/42507114 --> v2:172.21.15.143:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f98840cff20 con 0x7f98840d1c20 2023-11-26T07:18:50.801 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:50.793+0000 7f9888a66640 1 --2- 172.21.15.143:0/42507114 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f98841608f0 0x7f9884160d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-26T07:18:50.803 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:50.797+0000 7f98837fe640 1 --2- 172.21.15.143:0/42507114 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f98840d1c20 0x7f98840d0720 secure :-1 s=READY pgs=390 cs=0 l=1 rev1=1 crypto rx=0x7f98780098a0 tx=0x7f987801c450 comp rx=0 tx=0).ready entity=mon.0 client_cookie=78e5f1039cce7b6a server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:18:50.803 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:50.797+0000 7f9882ffd640 1 -- 172.21.15.143:0/42507114 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f98780ab600 con 0x7f98840d1c20 2023-11-26T07:18:50.803 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:50.797+0000 7f9882ffd640 1 -- 172.21.15.143:0/42507114 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f98780a7050 con 0x7f98840d1c20 2023-11-26T07:18:50.803 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:50.797+0000 7f9882ffd640 1 -- 172.21.15.143:0/42507114 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f98780ab980 con 0x7f98840d1c20 2023-11-26T07:18:50.804 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:50.797+0000 7f988a4c9640 1 -- 172.21.15.143:0/42507114 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f98840d1c20 msgr2=0x7f98840d0720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:18:50.804 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:50.797+0000 7f988a4c9640 1 --2- 172.21.15.143:0/42507114 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f98840d1c20 0x7f98840d0720 secure :-1 s=READY pgs=390 cs=0 l=1 rev1=1 crypto rx=0x7f98780098a0 tx=0x7f987801c450 comp rx=0 tx=0).stop 2023-11-26T07:18:50.804 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:50.797+0000 7f988a4c9640 1 -- 172.21.15.143:0/42507114 shutdown_connections 2023-11-26T07:18:50.804 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:50.797+0000 7f988a4c9640 1 --2- 172.21.15.143:0/42507114 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f98840d1c20 0x7f98840d0720 unknown :-1 s=CLOSED pgs=390 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:18:50.805 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:50.797+0000 7f988a4c9640 1 --2- 172.21.15.143:0/42507114 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f98841608f0 0x7f9884160d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:18:50.805 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:50.797+0000 7f988a4c9640 1 -- 172.21.15.143:0/42507114 >> 172.21.15.143:0/42507114 conn(0x7f988409c570 msgr2=0x7f98840d3240 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:18:50.805 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:50.797+0000 7f988a4c9640 1 -- 172.21.15.143:0/42507114 shutdown_connections 2023-11-26T07:18:50.805 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:50.797+0000 7f988a4c9640 1 -- 172.21.15.143:0/42507114 wait complete. 2023-11-26T07:18:50.806 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:50.801+0000 7f988a4c9640 1 Processor -- start 2023-11-26T07:18:50.806 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:50.801+0000 7f988a4c9640 1 -- start start 2023-11-26T07:18:50.808 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:50.801+0000 7f988a4c9640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f98840d1840 0x7f98841553a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:18:50.808 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:50.801+0000 7f988a4c9640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f98840d1c20 0x7f9884157df0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:18:50.808 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:50.801+0000 7f988a4c9640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f98841608f0 0x7f988415e4f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:18:50.808 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:50.801+0000 7f988a4c9640 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9884157320 con 0x7f98840d1c20 2023-11-26T07:18:50.808 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:50.801+0000 7f988a4c9640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9884156280 con 0x7f98841608f0 2023-11-26T07:18:50.808 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:50.801+0000 7f988a4c9640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9884156560 con 0x7f98840d1840 2023-11-26T07:18:50.808 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:50.801+0000 7f98837fe640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f98840d1c20 0x7f9884157df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:18:50.808 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:50.801+0000 7f98837fe640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f98840d1c20 0x7f9884157df0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:47548/0 (socket says 172.21.15.143:47548) 2023-11-26T07:18:50.808 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:50.801+0000 7f98837fe640 1 -- 172.21.15.143:0/3750970646 learned_addr learned my addr 172.21.15.143:0/3750970646 (peer_addr_for_me v2:172.21.15.143:0/0) 2023-11-26T07:18:50.808 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:50.801+0000 7f9888a66640 1 --2- 172.21.15.143:0/3750970646 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f98841608f0 0x7f988415e4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:18:50.808 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:50.801+0000 7f9883fff640 1 --2- 172.21.15.143:0/3750970646 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f98840d1840 0x7f98841553a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:18:50.809 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:50.801+0000 7f98837fe640 1 -- 172.21.15.143:0/3750970646 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f98840d1840 msgr2=0x7f98841553a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:18:50.809 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:50.801+0000 7f98837fe640 1 --2- 172.21.15.143:0/3750970646 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f98840d1840 0x7f98841553a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:18:50.809 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:50.801+0000 7f98837fe640 1 -- 172.21.15.143:0/3750970646 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f98841608f0 msgr2=0x7f988415e4f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:18:50.809 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:50.801+0000 7f98837fe640 1 --2- 172.21.15.143:0/3750970646 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f98841608f0 0x7f988415e4f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:18:50.809 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:50.801+0000 7f98837fe640 1 -- 172.21.15.143:0/3750970646 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f98780094d0 con 0x7f98840d1c20 2023-11-26T07:18:50.809 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:50.801+0000 7f98837fe640 1 --2- 172.21.15.143:0/3750970646 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f98840d1c20 0x7f9884157df0 secure :-1 s=READY pgs=391 cs=0 l=1 rev1=1 crypto rx=0x7f98780ae040 tx=0x7f987801ac90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:18:50.810 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:50.801+0000 7f9880ff9640 1 -- 172.21.15.143:0/3750970646 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f98780b6620 con 0x7f98840d1c20 2023-11-26T07:18:50.810 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:50.801+0000 7f9880ff9640 1 -- 172.21.15.143:0/3750970646 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f98780b6780 con 0x7f98840d1c20 2023-11-26T07:18:50.810 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:50.801+0000 7f9880ff9640 1 -- 172.21.15.143:0/3750970646 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f98780b6b30 con 0x7f98840d1c20 2023-11-26T07:18:50.810 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:50.801+0000 7f988a4c9640 1 -- 172.21.15.143:0/3750970646 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9884156840 con 0x7f98840d1c20 2023-11-26T07:18:50.810 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:50.801+0000 7f988a4c9640 1 -- 172.21.15.143:0/3750970646 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9884156e00 con 0x7f98840d1c20 2023-11-26T07:18:50.810 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:50.805+0000 7f988a4c9640 1 -- 172.21.15.143:0/3750970646 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9884067720 con 0x7f98840d1c20 2023-11-26T07:18:50.810 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:50.805+0000 7f9880ff9640 1 -- 172.21.15.143:0/3750970646 <== mon.0 v2:172.21.15.143:3300/0 4 ==== mgrmap(e 5) v1 ==== 85852+0+0 (secure 0 0 0) 0x7f98780b2040 con 0x7f98840d1c20 2023-11-26T07:18:50.811 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:50.805+0000 7f9880ff9640 1 --2- 172.21.15.143:0/3750970646 >> [v2:172.21.15.143:6832/2150557544,v1:172.21.15.143:6833/2150557544] conn(0x7f9870066470 0x7f9870068930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:18:50.811 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:50.805+0000 7f9880ff9640 1 -- 172.21.15.143:0/3750970646 <== mon.0 v2:172.21.15.143:3300/0 5 ==== osd_map(30..30 src has 1..30) v4 ==== 5227+0+0 (secure 0 0 0) 0x7f987811e570 con 0x7f98840d1c20 2023-11-26T07:18:50.812 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:50.805+0000 7f9883fff640 1 --2- 172.21.15.143:0/3750970646 >> [v2:172.21.15.143:6832/2150557544,v1:172.21.15.143:6833/2150557544] conn(0x7f9870066470 0x7f9870068930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:18:50.812 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:50.805+0000 7f9883fff640 1 --2- 172.21.15.143:0/3750970646 >> [v2:172.21.15.143:6832/2150557544,v1:172.21.15.143:6833/2150557544] conn(0x7f9870066470 0x7f9870068930 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7f9874004e30 tx=0x7f9874009040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:18:50.815 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:50.809+0000 7f9880ff9640 1 -- 172.21.15.143:0/3750970646 <== mon.0 v2:172.21.15.143:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172649 (secure 0 0 0) 0x7f98780ebc40 con 0x7f98840d1c20 2023-11-26T07:18:50.965 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:50.961+0000 7f988a4c9640 1 -- 172.21.15.143:0/3750970646 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9} v 0) v1 -- 0x7f9884156ff0 con 0x7f98840d1c20 2023-11-26T07:18:51.629 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:51.625+0000 7f9880ff9640 1 -- 172.21.15.143:0/3750970646 <== mon.0 v2:172.21.15.143:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]=0 pool 'cephfs_data' created v31) v1 ==== 170+0+0 (secure 0 0 0) 0x7f98780ef260 con 0x7f98840d1c20 2023-11-26T07:18:51.629 INFO:teuthology.orchestra.run.smithi143.stderr:pool 'cephfs_data' created 2023-11-26T07:18:51.631 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:51.625+0000 7f988a4c9640 1 -- 172.21.15.143:0/3750970646 >> [v2:172.21.15.143:6832/2150557544,v1:172.21.15.143:6833/2150557544] conn(0x7f9870066470 msgr2=0x7f9870068930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:18:51.631 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:51.625+0000 7f988a4c9640 1 --2- 172.21.15.143:0/3750970646 >> [v2:172.21.15.143:6832/2150557544,v1:172.21.15.143:6833/2150557544] conn(0x7f9870066470 0x7f9870068930 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7f9874004e30 tx=0x7f9874009040 comp rx=0 tx=0).stop 2023-11-26T07:18:51.631 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:51.625+0000 7f988a4c9640 1 -- 172.21.15.143:0/3750970646 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f98840d1c20 msgr2=0x7f9884157df0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:18:51.631 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:51.625+0000 7f988a4c9640 1 --2- 172.21.15.143:0/3750970646 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f98840d1c20 0x7f9884157df0 secure :-1 s=READY pgs=391 cs=0 l=1 rev1=1 crypto rx=0x7f98780ae040 tx=0x7f987801ac90 comp rx=0 tx=0).stop 2023-11-26T07:18:51.632 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:51.625+0000 7f988a4c9640 1 -- 172.21.15.143:0/3750970646 shutdown_connections 2023-11-26T07:18:51.632 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:51.625+0000 7f988a4c9640 1 --2- 172.21.15.143:0/3750970646 >> [v2:172.21.15.143:6832/2150557544,v1:172.21.15.143:6833/2150557544] conn(0x7f9870066470 0x7f9870068930 unknown :-1 s=CLOSED pgs=214 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:18:51.632 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:51.625+0000 7f988a4c9640 1 --2- 172.21.15.143:0/3750970646 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f98841608f0 0x7f988415e4f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:18:51.632 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:51.625+0000 7f988a4c9640 1 --2- 172.21.15.143:0/3750970646 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f98840d1c20 0x7f9884157df0 unknown :-1 s=CLOSED pgs=391 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:18:51.632 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:51.625+0000 7f988a4c9640 1 --2- 172.21.15.143:0/3750970646 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f98840d1840 0x7f98841553a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:18:51.632 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:51.625+0000 7f988a4c9640 1 -- 172.21.15.143:0/3750970646 >> 172.21.15.143:0/3750970646 conn(0x7f988409c570 msgr2=0x7f988415ff40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:18:51.633 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:51.625+0000 7f988a4c9640 1 -- 172.21.15.143:0/3750970646 shutdown_connections 2023-11-26T07:18:51.633 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:51.625+0000 7f988a4c9640 1 -- 172.21.15.143:0/3750970646 wait complete. 2023-11-26T07:18:51.672 DEBUG:teuthology.orchestra.run.smithi143:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs new cephfs cephfs_metadata cephfs_data 2023-11-26T07:18:51.681 DEBUG:teuthology.orchestra.run.smithi143:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-26T07:18:51.723 DEBUG:teuthology.orchestra.run.smithi188:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-26T07:18:51.816 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:51.809+0000 7fc7edcd3640 1 Processor -- start 2023-11-26T07:18:51.816 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:51.809+0000 7fc7edcd3640 1 -- start start 2023-11-26T07:18:51.817 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:51.813+0000 7fc7edcd3640 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7fc7e80b1e10 0x7fc7e815d9e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:18:51.817 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:51.813+0000 7fc7edcd3640 1 -- --> v1:172.21.15.143:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc7e815e860 con 0x7fc7e815df20 2023-11-26T07:18:51.817 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:51.813+0000 7fc7edcd3640 1 -- --> v1:172.21.15.188:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc7e80cdba0 con 0x7fc7e8143810 2023-11-26T07:18:51.817 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:51.813+0000 7fc7edcd3640 1 -- --> v2:172.21.15.143:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc7e815e3b0 con 0x7fc7e80b1e10 2023-11-26T07:18:51.818 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:51.813+0000 7fc7e7fff640 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7fc7e80b1e10 0x7fc7e815d9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:18:51.818 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:51.813+0000 7fc7ed4d2640 1 --1- >> v1:172.21.15.143:6789/0 conn(0x7fc7e815df20 0x7fc7e80ce030 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.143:6789/0 says I am v1:172.21.15.143:55178/0 (socket says 172.21.15.143:55178) 2023-11-26T07:18:51.818 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:51.813+0000 7fc7ed4d2640 1 -- 172.21.15.143:0/2758779122 learned_addr learned my addr 172.21.15.143:0/2758779122 (peer_addr_for_me v1:172.21.15.143:0/0) 2023-11-26T07:18:51.818 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:51.813+0000 7fc7e7fff640 1 -- 172.21.15.143:0/2758779122 >> v1:172.21.15.188:6789/0 conn(0x7fc7e8143810 legacy=0x7fc7e8072140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:18:51.819 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:51.813+0000 7fc7e7fff640 1 -- 172.21.15.143:0/2758779122 >> v1:172.21.15.143:6789/0 conn(0x7fc7e815df20 legacy=0x7fc7e80ce030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:18:51.819 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:51.813+0000 7fc7e7fff640 1 -- 172.21.15.143:0/2758779122 --> v2:172.21.15.143:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc7e8143bf0 con 0x7fc7e80b1e10 2023-11-26T07:18:51.820 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:51.817+0000 7fc7e7fff640 1 --2- 172.21.15.143:0/2758779122 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fc7e80b1e10 0x7fc7e815d9e0 secure :-1 s=READY pgs=393 cs=0 l=1 rev1=1 crypto rx=0x7fc7d40097c0 tx=0x7fc7d401c310 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ec7af668c7a67869 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:18:51.821 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:51.817+0000 7fc7e77fe640 1 -- 172.21.15.143:0/2758779122 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc7d40ab600 con 0x7fc7e80b1e10 2023-11-26T07:18:51.821 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:51.817+0000 7fc7e77fe640 1 -- 172.21.15.143:0/2758779122 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc7d40a7050 con 0x7fc7e80b1e10 2023-11-26T07:18:51.821 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:51.817+0000 7fc7e77fe640 1 -- 172.21.15.143:0/2758779122 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc7d40ab980 con 0x7fc7e80b1e10 2023-11-26T07:18:51.822 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:51.817+0000 7fc7edcd3640 1 -- 172.21.15.143:0/2758779122 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fc7e80b1e10 msgr2=0x7fc7e815d9e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:18:51.822 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:51.817+0000 7fc7edcd3640 1 --2- 172.21.15.143:0/2758779122 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fc7e80b1e10 0x7fc7e815d9e0 secure :-1 s=READY pgs=393 cs=0 l=1 rev1=1 crypto rx=0x7fc7d40097c0 tx=0x7fc7d401c310 comp rx=0 tx=0).stop 2023-11-26T07:18:51.822 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:51.817+0000 7fc7edcd3640 1 -- 172.21.15.143:0/2758779122 shutdown_connections 2023-11-26T07:18:51.822 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:51.817+0000 7fc7edcd3640 1 --2- 172.21.15.143:0/2758779122 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fc7e80b1e10 0x7fc7e815d9e0 unknown :-1 s=CLOSED pgs=393 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:18:51.822 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:51.817+0000 7fc7edcd3640 1 -- 172.21.15.143:0/2758779122 >> 172.21.15.143:0/2758779122 conn(0x7fc7e806a890 msgr2=0x7fc7e806aca0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:18:51.823 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:51.817+0000 7fc7edcd3640 1 -- 172.21.15.143:0/2758779122 shutdown_connections 2023-11-26T07:18:51.823 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:51.817+0000 7fc7edcd3640 1 -- 172.21.15.143:0/2758779122 wait complete. 2023-11-26T07:18:51.823 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:51.817+0000 7fc7edcd3640 1 Processor -- start 2023-11-26T07:18:51.823 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:51.817+0000 7fc7edcd3640 1 -- start start 2023-11-26T07:18:51.825 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:51.821+0000 7fc7edcd3640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fc7e80b1e10 0x7fc7e8157520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:18:51.825 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:51.821+0000 7fc7edcd3640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fc7e8143810 0x7fc7e80ce290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:18:51.825 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:51.821+0000 7fc7edcd3640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fc7e815df20 0x7fc7e8131fc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:18:51.825 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:51.821+0000 7fc7edcd3640 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc7e8157e70 con 0x7fc7e815df20 2023-11-26T07:18:51.825 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:51.821+0000 7fc7edcd3640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc7e8158150 con 0x7fc7e8143810 2023-11-26T07:18:51.825 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:51.821+0000 7fc7edcd3640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc7e8158430 con 0x7fc7e80b1e10 2023-11-26T07:18:51.825 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:51.821+0000 7fc7ed4d2640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fc7e815df20 0x7fc7e8131fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:18:51.826 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:51.821+0000 7fc7e7fff640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fc7e8143810 0x7fc7e80ce290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:18:51.826 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:51.821+0000 7fc7ed4d2640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fc7e815df20 0x7fc7e8131fc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:47580/0 (socket says 172.21.15.143:47580) 2023-11-26T07:18:51.826 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:51.821+0000 7fc7ed4d2640 1 -- 172.21.15.143:0/2168230597 learned_addr learned my addr 172.21.15.143:0/2168230597 (peer_addr_for_me v2:172.21.15.143:0/0) 2023-11-26T07:18:51.826 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:51.821+0000 7fc7eccd1640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fc7e80b1e10 0x7fc7e8157520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:18:51.827 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:51.821+0000 7fc7ed4d2640 1 -- 172.21.15.143:0/2168230597 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fc7e80b1e10 msgr2=0x7fc7e8157520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:18:51.827 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:51.821+0000 7fc7ed4d2640 1 --2- 172.21.15.143:0/2168230597 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fc7e80b1e10 0x7fc7e8157520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:18:51.827 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:51.821+0000 7fc7ed4d2640 1 -- 172.21.15.143:0/2168230597 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fc7e8143810 msgr2=0x7fc7e80ce290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:18:51.827 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:51.821+0000 7fc7ed4d2640 1 --2- 172.21.15.143:0/2168230597 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fc7e8143810 0x7fc7e80ce290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:18:51.827 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:51.821+0000 7fc7ed4d2640 1 -- 172.21.15.143:0/2168230597 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc7d4009420 con 0x7fc7e815df20 2023-11-26T07:18:51.827 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:51.821+0000 7fc7ed4d2640 1 --2- 172.21.15.143:0/2168230597 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fc7e815df20 0x7fc7e8131fc0 secure :-1 s=READY pgs=394 cs=0 l=1 rev1=1 crypto rx=0x7fc7e000e550 tx=0x7fc7e000ea10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:18:51.827 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:51.821+0000 7fc7e57fa640 1 -- 172.21.15.143:0/2168230597 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc7e0012810 con 0x7fc7e815df20 2023-11-26T07:18:51.827 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:51.821+0000 7fc7e57fa640 1 -- 172.21.15.143:0/2168230597 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc7e0012970 con 0x7fc7e815df20 2023-11-26T07:18:51.827 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:51.821+0000 7fc7e57fa640 1 -- 172.21.15.143:0/2168230597 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc7e0012d20 con 0x7fc7e815df20 2023-11-26T07:18:51.827 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:51.821+0000 7fc7edcd3640 1 -- 172.21.15.143:0/2168230597 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc7e8158710 con 0x7fc7e815df20 2023-11-26T07:18:51.828 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:51.821+0000 7fc7edcd3640 1 -- 172.21.15.143:0/2168230597 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc7e8157a60 con 0x7fc7e815df20 2023-11-26T07:18:51.828 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:51.821+0000 7fc7edcd3640 1 -- 172.21.15.143:0/2168230597 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc7e8067500 con 0x7fc7e815df20 2023-11-26T07:18:51.828 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:51.821+0000 7fc7e57fa640 1 -- 172.21.15.143:0/2168230597 <== mon.0 v2:172.21.15.143:3300/0 4 ==== mgrmap(e 5) v1 ==== 85852+0+0 (secure 0 0 0) 0x7fc7e0009070 con 0x7fc7e815df20 2023-11-26T07:18:51.830 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:51.825+0000 7fc7e57fa640 1 --2- 172.21.15.143:0/2168230597 >> [v2:172.21.15.143:6832/2150557544,v1:172.21.15.143:6833/2150557544] conn(0x7fc7cc066470 0x7fc7cc068930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:18:51.830 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:51.825+0000 7fc7e57fa640 1 -- 172.21.15.143:0/2168230597 <== mon.0 v2:172.21.15.143:3300/0 5 ==== osd_map(31..31 src has 1..31) v4 ==== 5609+0+0 (secure 0 0 0) 0x7fc7e0086b50 con 0x7fc7e815df20 2023-11-26T07:18:51.830 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:51.825+0000 7fc7eccd1640 1 --2- 172.21.15.143:0/2168230597 >> [v2:172.21.15.143:6832/2150557544,v1:172.21.15.143:6833/2150557544] conn(0x7fc7cc066470 0x7fc7cc068930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:18:51.830 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:51.825+0000 7fc7eccd1640 1 --2- 172.21.15.143:0/2168230597 >> [v2:172.21.15.143:6832/2150557544,v1:172.21.15.143:6833/2150557544] conn(0x7fc7cc066470 0x7fc7cc068930 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7fc7dc007fd0 tx=0x7fc7dc009040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:18:51.834 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:51.829+0000 7fc7e57fa640 1 -- 172.21.15.143:0/2168230597 <== mon.0 v2:172.21.15.143:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172649 (secure 0 0 0) 0x7fc7e00540d0 con 0x7fc7e815df20 2023-11-26T07:18:52.014 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:52.009+0000 7fc7edcd3640 1 -- 172.21.15.143:0/2168230597 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"} v 0) v1 -- 0x7fc7e8158bc0 con 0x7fc7e815df20 2023-11-26T07:18:52.645 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:52.641+0000 7fc7e57fa640 1 -- 172.21.15.143:0/2168230597 <== mon.0 v2:172.21.15.143:3300/0 7 ==== mon_command_ack([{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]=0 new fs with metadata pool 4 and data pool 5 v13) v1 ==== 172+0+0 (secure 0 0 0) 0x7fc7e00576f0 con 0x7fc7e815df20 2023-11-26T07:18:52.646 INFO:teuthology.orchestra.run.smithi143.stderr:new fs with metadata pool 4 and data pool 5 2023-11-26T07:18:52.649 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:52.645+0000 7fc7edcd3640 1 -- 172.21.15.143:0/2168230597 >> [v2:172.21.15.143:6832/2150557544,v1:172.21.15.143:6833/2150557544] conn(0x7fc7cc066470 msgr2=0x7fc7cc068930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:18:52.649 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:52.645+0000 7fc7edcd3640 1 --2- 172.21.15.143:0/2168230597 >> [v2:172.21.15.143:6832/2150557544,v1:172.21.15.143:6833/2150557544] conn(0x7fc7cc066470 0x7fc7cc068930 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7fc7dc007fd0 tx=0x7fc7dc009040 comp rx=0 tx=0).stop 2023-11-26T07:18:52.649 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:52.645+0000 7fc7edcd3640 1 -- 172.21.15.143:0/2168230597 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fc7e815df20 msgr2=0x7fc7e8131fc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:18:52.649 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:52.645+0000 7fc7edcd3640 1 --2- 172.21.15.143:0/2168230597 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fc7e815df20 0x7fc7e8131fc0 secure :-1 s=READY pgs=394 cs=0 l=1 rev1=1 crypto rx=0x7fc7e000e550 tx=0x7fc7e000ea10 comp rx=0 tx=0).stop 2023-11-26T07:18:52.649 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:52.645+0000 7fc7edcd3640 1 -- 172.21.15.143:0/2168230597 shutdown_connections 2023-11-26T07:18:52.650 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:52.645+0000 7fc7edcd3640 1 --2- 172.21.15.143:0/2168230597 >> [v2:172.21.15.143:6832/2150557544,v1:172.21.15.143:6833/2150557544] conn(0x7fc7cc066470 0x7fc7cc068930 unknown :-1 s=CLOSED pgs=215 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:18:52.650 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:52.645+0000 7fc7edcd3640 1 --2- 172.21.15.143:0/2168230597 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fc7e815df20 0x7fc7e8131fc0 unknown :-1 s=CLOSED pgs=394 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:18:52.650 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:52.645+0000 7fc7edcd3640 1 --2- 172.21.15.143:0/2168230597 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fc7e8143810 0x7fc7e80ce290 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:18:52.650 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:52.645+0000 7fc7edcd3640 1 --2- 172.21.15.143:0/2168230597 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fc7e80b1e10 0x7fc7e8157520 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:18:52.650 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:52.645+0000 7fc7edcd3640 1 -- 172.21.15.143:0/2168230597 >> 172.21.15.143:0/2168230597 conn(0x7fc7e806a890 msgr2=0x7fc7e815d800 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:18:52.651 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:52.645+0000 7fc7edcd3640 1 -- 172.21.15.143:0/2168230597 shutdown_connections 2023-11-26T07:18:52.651 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:52.645+0000 7fc7edcd3640 1 -- 172.21.15.143:0/2168230597 wait complete. 2023-11-26T07:18:52.698 DEBUG:teuthology.orchestra.run.smithi143:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-11-26T07:18:52.878 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:52.873+0000 7f2eec11c640 1 Processor -- start 2023-11-26T07:18:52.879 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:52.873+0000 7f2eec11c640 1 -- start start 2023-11-26T07:18:52.880 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:52.873+0000 7f2eec11c640 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7f2ee40cf990 0x7f2ee40708c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:18:52.880 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:52.873+0000 7f2eec11c640 1 -- --> v1:172.21.15.143:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2ee4162080 con 0x7f2ee4070e00 2023-11-26T07:18:52.880 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:52.873+0000 7f2eec11c640 1 -- --> v1:172.21.15.188:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2ee4161d70 con 0x7f2ee414b1e0 2023-11-26T07:18:52.880 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:52.873+0000 7f2eec11c640 1 -- --> v2:172.21.15.143:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2ee414b5c0 con 0x7f2ee40cf990 2023-11-26T07:18:52.880 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:52.873+0000 7f2eea6b9640 1 --1- >> v1:172.21.15.143:6789/0 conn(0x7f2ee4070e00 0x7f2ee4162510 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.143:6789/0 says I am v1:172.21.15.143:55182/0 (socket says 172.21.15.143:55182) 2023-11-26T07:18:52.880 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:52.873+0000 7f2eea6b9640 1 -- 172.21.15.143:0/2373532847 learned_addr learned my addr 172.21.15.143:0/2373532847 (peer_addr_for_me v1:172.21.15.143:0/0) 2023-11-26T07:18:52.881 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:52.873+0000 7f2ee96b7640 1 --2- 172.21.15.143:0/2373532847 >> v2:172.21.15.143:3300/0 conn(0x7f2ee40cf990 0x7f2ee40708c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:18:52.881 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:52.877+0000 7f2ee8eb6640 1 -- 172.21.15.143:0/2373532847 <== mon.0 v1:172.21.15.143:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2944551361 0 0) 0x7f2ee4162080 con 0x7f2ee4070e00 2023-11-26T07:18:52.881 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:52.877+0000 7f2ee8eb6640 1 -- 172.21.15.143:0/2373532847 --> v1:172.21.15.143:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f2ed0003530 con 0x7f2ee4070e00 2023-11-26T07:18:52.881 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:52.877+0000 7f2ee96b7640 1 -- 172.21.15.143:0/2373532847 >> v1:172.21.15.188:6789/0 conn(0x7f2ee414b1e0 legacy=0x7f2ee40cf280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:18:52.881 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:52.877+0000 7f2ee96b7640 1 -- 172.21.15.143:0/2373532847 >> v1:172.21.15.143:6789/0 conn(0x7f2ee4070e00 legacy=0x7f2ee4162510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:18:52.882 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:52.877+0000 7f2ee96b7640 1 -- 172.21.15.143:0/2373532847 --> v2:172.21.15.143:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2ee4071580 con 0x7f2ee40cf990 2023-11-26T07:18:52.884 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:52.877+0000 7f2ee96b7640 1 --2- 172.21.15.143:0/2373532847 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f2ee40cf990 0x7f2ee40708c0 secure :-1 s=READY pgs=396 cs=0 l=1 rev1=1 crypto rx=0x7f2ed40097c0 tx=0x7f2ed401c450 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2da8d5e79843fd8d server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:18:52.884 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:52.877+0000 7f2ee8eb6640 1 -- 172.21.15.143:0/2373532847 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2ed40ab650 con 0x7f2ee40cf990 2023-11-26T07:18:52.884 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:52.877+0000 7f2ee8eb6640 1 -- 172.21.15.143:0/2373532847 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2ed40ab7b0 con 0x7f2ee40cf990 2023-11-26T07:18:52.884 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:52.877+0000 7f2ee8eb6640 1 -- 172.21.15.143:0/2373532847 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2ed40abb40 con 0x7f2ee40cf990 2023-11-26T07:18:52.885 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:52.877+0000 7f2eec11c640 1 -- 172.21.15.143:0/2373532847 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f2ee40cf990 msgr2=0x7f2ee40708c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:18:52.885 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:52.877+0000 7f2eec11c640 1 --2- 172.21.15.143:0/2373532847 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f2ee40cf990 0x7f2ee40708c0 secure :-1 s=READY pgs=396 cs=0 l=1 rev1=1 crypto rx=0x7f2ed40097c0 tx=0x7f2ed401c450 comp rx=0 tx=0).stop 2023-11-26T07:18:52.885 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:52.881+0000 7f2eec11c640 1 -- 172.21.15.143:0/2373532847 shutdown_connections 2023-11-26T07:18:52.885 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:52.881+0000 7f2eec11c640 1 --2- 172.21.15.143:0/2373532847 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f2ee40cf990 0x7f2ee40708c0 unknown :-1 s=CLOSED pgs=396 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:18:52.885 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:52.881+0000 7f2eec11c640 1 -- 172.21.15.143:0/2373532847 >> 172.21.15.143:0/2373532847 conn(0x7f2ee406a890 msgr2=0x7f2ee406aca0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:18:52.885 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:52.881+0000 7f2eec11c640 1 -- 172.21.15.143:0/2373532847 shutdown_connections 2023-11-26T07:18:52.885 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:52.881+0000 7f2eec11c640 1 -- 172.21.15.143:0/2373532847 wait complete. 2023-11-26T07:18:52.886 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:52.881+0000 7f2eec11c640 1 Processor -- start 2023-11-26T07:18:52.886 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:52.881+0000 7f2eec11c640 1 -- start start 2023-11-26T07:18:52.887 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:52.881+0000 7f2eec11c640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f2ee4070e00 0x7f2ee4160f50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:18:52.888 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:52.881+0000 7f2eec11c640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f2ee40cf990 0x7f2ee40ad990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:18:52.888 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:52.881+0000 7f2eec11c640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f2ee414b1e0 0x7f2ee414dbe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:18:52.888 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:52.881+0000 7f2eec11c640 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2ee414c940 con 0x7f2ee4070e00 2023-11-26T07:18:52.888 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:52.881+0000 7f2eec11c640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2ee414bd30 con 0x7f2ee40cf990 2023-11-26T07:18:52.888 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:52.881+0000 7f2eec11c640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2ee414c010 con 0x7f2ee414b1e0 2023-11-26T07:18:52.888 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:52.881+0000 7f2ee9eb8640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f2ee4070e00 0x7f2ee4160f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:18:52.888 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:52.881+0000 7f2ee9eb8640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f2ee4070e00 0x7f2ee4160f50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:47594/0 (socket says 172.21.15.143:47594) 2023-11-26T07:18:52.888 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:52.881+0000 7f2ee9eb8640 1 -- 172.21.15.143:0/2858544564 learned_addr learned my addr 172.21.15.143:0/2858544564 (peer_addr_for_me v2:172.21.15.143:0/0) 2023-11-26T07:18:52.888 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:52.881+0000 7f2eea6b9640 1 --2- 172.21.15.143:0/2858544564 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f2ee414b1e0 0x7f2ee414dbe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:18:52.888 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:52.881+0000 7f2ee96b7640 1 --2- 172.21.15.143:0/2858544564 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f2ee40cf990 0x7f2ee40ad990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:18:52.889 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:52.881+0000 7f2ee9eb8640 1 -- 172.21.15.143:0/2858544564 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f2ee414b1e0 msgr2=0x7f2ee414dbe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:18:52.890 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:52.881+0000 7f2ee9eb8640 1 --2- 172.21.15.143:0/2858544564 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f2ee414b1e0 0x7f2ee414dbe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:18:52.890 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:52.881+0000 7f2ee9eb8640 1 -- 172.21.15.143:0/2858544564 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f2ee40cf990 msgr2=0x7f2ee40ad990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:18:52.890 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:52.881+0000 7f2ee9eb8640 1 --2- 172.21.15.143:0/2858544564 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f2ee40cf990 0x7f2ee40ad990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:18:52.890 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:52.881+0000 7f2ee9eb8640 1 -- 172.21.15.143:0/2858544564 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2ed4009420 con 0x7f2ee4070e00 2023-11-26T07:18:52.890 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:52.881+0000 7f2eea6b9640 1 --2- 172.21.15.143:0/2858544564 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f2ee414b1e0 0x7f2ee414dbe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-26T07:18:52.890 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:52.881+0000 7f2ee9eb8640 1 --2- 172.21.15.143:0/2858544564 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f2ee4070e00 0x7f2ee4160f50 secure :-1 s=READY pgs=397 cs=0 l=1 rev1=1 crypto rx=0x7f2ee000c500 tx=0x7f2ee000c9c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:18:52.890 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:52.885+0000 7f2eda7fc640 1 -- 172.21.15.143:0/2858544564 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2ee0005220 con 0x7f2ee4070e00 2023-11-26T07:18:52.890 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:52.885+0000 7f2eda7fc640 1 -- 172.21.15.143:0/2858544564 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2ee0005380 con 0x7f2ee4070e00 2023-11-26T07:18:52.890 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:52.885+0000 7f2eda7fc640 1 -- 172.21.15.143:0/2858544564 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2ee0007270 con 0x7f2ee4070e00 2023-11-26T07:18:52.890 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:52.885+0000 7f2eec11c640 1 -- 172.21.15.143:0/2858544564 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2ee414c2f0 con 0x7f2ee4070e00 2023-11-26T07:18:52.890 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:52.885+0000 7f2eec11c640 1 -- 172.21.15.143:0/2858544564 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2ee4161490 con 0x7f2ee4070e00 2023-11-26T07:18:52.890 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:52.885+0000 7f2eec11c640 1 -- 172.21.15.143:0/2858544564 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2ee4067680 con 0x7f2ee4070e00 2023-11-26T07:18:52.891 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:52.885+0000 7f2eda7fc640 1 -- 172.21.15.143:0/2858544564 <== mon.0 v2:172.21.15.143:3300/0 4 ==== mgrmap(e 5) v1 ==== 85852+0+0 (secure 0 0 0) 0x7f2ee00073d0 con 0x7f2ee4070e00 2023-11-26T07:18:52.891 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:52.885+0000 7f2eda7fc640 1 --2- 172.21.15.143:0/2858544564 >> [v2:172.21.15.143:6832/2150557544,v1:172.21.15.143:6833/2150557544] conn(0x7f2ec8066470 0x7f2ec8068930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:18:52.891 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:52.885+0000 7f2eda7fc640 1 -- 172.21.15.143:0/2858544564 <== mon.0 v2:172.21.15.143:3300/0 5 ==== osd_map(32..32 src has 1..32) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f2ee00837c0 con 0x7f2ee4070e00 2023-11-26T07:18:52.892 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:52.885+0000 7f2ee96b7640 1 --2- 172.21.15.143:0/2858544564 >> [v2:172.21.15.143:6832/2150557544,v1:172.21.15.143:6833/2150557544] conn(0x7f2ec8066470 0x7f2ec8068930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:18:52.892 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:52.885+0000 7f2ee96b7640 1 --2- 172.21.15.143:0/2858544564 >> [v2:172.21.15.143:6832/2150557544,v1:172.21.15.143:6833/2150557544] conn(0x7f2ec8066470 0x7f2ec8068930 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7f2ed40af000 tx=0x7f2ed40a7000 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:18:52.896 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:52.889+0000 7f2eda7fc640 1 -- 172.21.15.143:0/2858544564 <== mon.0 v2:172.21.15.143:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172649 (secure 0 0 0) 0x7f2ee0050bb0 con 0x7f2ee4070e00 2023-11-26T07:18:53.045 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:53.041+0000 7f2eec11c640 1 -- 172.21.15.143:0/2858544564 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f2ee40cf220 con 0x7f2ee4070e00 2023-11-26T07:18:53.046 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:53.041+0000 7f2eda7fc640 1 -- 172.21.15.143:0/2858544564 <== mon.0 v2:172.21.15.143:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v32) v1 ==== 74+0+16618 (secure 0 0 0) 0x7f2ee40cf220 con 0x7f2ee4070e00 2023-11-26T07:18:53.046 INFO:teuthology.orchestra.run.smithi143.stdout: 2023-11-26T07:18:53.046 INFO:teuthology.orchestra.run.smithi143.stdout:{"epoch":32,"fsid":"f5033060-2469-4a07-95fe-2b76f5404bc1","created":"2023-11-26T07:17:28.613663+0000","modified":"2023-11-26T07:18:52.014230+0000","last_up_change":"2023-11-26T07:17:41.157813+0000","last_in_change":"2023-11-26T07:17:31.181658+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":5,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-11-26T07:17:41.758597+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_metadata","create_time":"2023-11-26T07:18:50.535934+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"32","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":5,"pool_name":"cephfs_data","create_time":"2023-11-26T07:18:51.008494+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"32","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"ebbee2c6-023e-44b9-9f1d-6f7bfd309730","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6817","nonce":639807369},{"type":"v1","addr":"172.21.15.143:6819","nonce":639807369}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6821","nonce":639807369},{"type":"v1","addr":"172.21.15.143:6823","nonce":639807369}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6830","nonce":639807369},{"type":"v1","addr":"172.21.15.143:6831","nonce":639807369}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6825","nonce":639807369},{"type":"v1","addr":"172.21.15.143:6828","nonce":639807369}]},"public_addr":"172.21.15.143:6819/639807369","cluster_addr":"172.21.15.143:6823/639807369","heartbeat_back_addr":"172.21.15.143:6831/639807369","heartbeat_front_addr":"172.21.15.143:6828/639807369","state":["exists","up"]},{"osd":1,"uuid":"fec4b8be-7122-49e9-9a19-5850a0700001","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6816","nonce":1144942530},{"type":"v1","addr":"172.21.15.143:6818","nonce":1144942530}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6820","nonce":1144942530},{"type":"v1","addr":"172.21.15.143:6822","nonce":1144942530}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6827","nonce":1144942530},{"type":"v1","addr":"172.21.15.143:6829","nonce":1144942530}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6824","nonce":1144942530},{"type":"v1","addr":"172.21.15.143:6826","nonce":1144942530}]},"public_addr":"172.21.15.143:6818/1144942530","cluster_addr":"172.21.15.143:6822/1144942530","heartbeat_back_addr":"172.21.15.143:6829/1144942530","heartbeat_front_addr":"172.21.15.143:6826/1144942530","state":["exists","up"]},{"osd":2,"uuid":"2888af5b-f933-46ca-8f95-6dd494ff43f6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6808","nonce":3909095186},{"type":"v1","addr":"172.21.15.143:6809","nonce":3909095186}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6810","nonce":3909095186},{"type":"v1","addr":"172.21.15.143:6811","nonce":3909095186}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6814","nonce":3909095186},{"type":"v1","addr":"172.21.15.143:6815","nonce":3909095186}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6812","nonce":3909095186},{"type":"v1","addr":"172.21.15.143:6813","nonce":3909095186}]},"public_addr":"172.21.15.143:6809/3909095186","cluster_addr":"172.21.15.143:6811/3909095186","heartbeat_back_addr":"172.21.15.143:6815/3909095186","heartbeat_front_addr":"172.21.15.143:6813/3909095186","state":["exists","up"]},{"osd":3,"uuid":"d1f52435-7bc1-4081-9550-9cd1ba55f549","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6800","nonce":3854210991},{"type":"v1","addr":"172.21.15.143:6801","nonce":3854210991}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6802","nonce":3854210991},{"type":"v1","addr":"172.21.15.143:6803","nonce":3854210991}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6806","nonce":3854210991},{"type":"v1","addr":"172.21.15.143:6807","nonce":3854210991}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6804","nonce":3854210991},{"type":"v1","addr":"172.21.15.143:6805","nonce":3854210991}]},"public_addr":"172.21.15.143:6801/3854210991","cluster_addr":"172.21.15.143:6803/3854210991","heartbeat_back_addr":"172.21.15.143:6807/3854210991","heartbeat_front_addr":"172.21.15.143:6805/3854210991","state":["exists","up"]},{"osd":4,"uuid":"e2ae8b6e-6e25-49b2-bcbf-aacce68f06a3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6800","nonce":3924606902},{"type":"v1","addr":"172.21.15.188:6801","nonce":3924606902}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6802","nonce":3924606902},{"type":"v1","addr":"172.21.15.188:6803","nonce":3924606902}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6806","nonce":3924606902},{"type":"v1","addr":"172.21.15.188:6807","nonce":3924606902}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6804","nonce":3924606902},{"type":"v1","addr":"172.21.15.188:6805","nonce":3924606902}]},"public_addr":"172.21.15.188:6801/3924606902","cluster_addr":"172.21.15.188:6803/3924606902","heartbeat_back_addr":"172.21.15.188:6807/3924606902","heartbeat_front_addr":"172.21.15.188:6805/3924606902","state":["exists","up"]},{"osd":5,"uuid":"3cec8a32-7698-4c36-8e89-e4ba11934f33","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6813","nonce":186841544},{"type":"v1","addr":"172.21.15.188:6815","nonce":186841544}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6817","nonce":186841544},{"type":"v1","addr":"172.21.15.188:6818","nonce":186841544}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6822","nonce":186841544},{"type":"v1","addr":"172.21.15.188:6823","nonce":186841544}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6820","nonce":186841544},{"type":"v1","addr":"172.21.15.188:6821","nonce":186841544}]},"public_addr":"172.21.15.188:6815/186841544","cluster_addr":"172.21.15.188:6818/186841544","heartbeat_back_addr":"172.21.15.188:6823/186841544","heartbeat_front_addr":"172.21.15.188:6821/186841544","state":["exists","up"]},{"osd":6,"uuid":"58a65d5d-e090-473a-81b4-ad38e85029f9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6808","nonce":2153499405},{"type":"v1","addr":"172.21.15.188:6809","nonce":2153499405}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6810","nonce":2153499405},{"type":"v1","addr":"172.21.15.188:6811","nonce":2153499405}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6816","nonce":2153499405},{"type":"v1","addr":"172.21.15.188:6819","nonce":2153499405}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6812","nonce":2153499405},{"type":"v1","addr":"172.21.15.188:6814","nonce":2153499405}]},"public_addr":"172.21.15.188:6809/2153499405","cluster_addr":"172.21.15.188:6811/2153499405","heartbeat_back_addr":"172.21.15.188:6819/2153499405","heartbeat_front_addr":"172.21.15.188:6814/2153499405","state":["exists","up"]},{"osd":7,"uuid":"ea5b7f70-3554-4cf8-ac30-f7c59b80678d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6824","nonce":2722096729},{"type":"v1","addr":"172.21.15.188:6825","nonce":2722096729}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6826","nonce":2722096729},{"type":"v1","addr":"172.21.15.188:6827","nonce":2722096729}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6830","nonce":2722096729},{"type":"v1","addr":"172.21.15.188:6831","nonce":2722096729}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6828","nonce":2722096729},{"type":"v1","addr":"172.21.15.188:6829","nonce":2722096729}]},"public_addr":"172.21.15.188:6825/2722096729","cluster_addr":"172.21.15.188:6827/2722096729","heartbeat_back_addr":"172.21.15.188:6831/2722096729","heartbeat_front_addr":"172.21.15.188:6829/2722096729","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-26T07:17:36.995032+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-26T07:17:36.932266+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-26T07:17:36.829304+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-26T07:17:36.804297+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-26T07:17:36.667892+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-26T07:17:36.964868+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-26T07:17:36.827332+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-26T07:17:36.893179+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-11-26T07:18:53.048 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:53.041+0000 7f2eec11c640 1 -- 172.21.15.143:0/2858544564 >> [v2:172.21.15.143:6832/2150557544,v1:172.21.15.143:6833/2150557544] conn(0x7f2ec8066470 msgr2=0x7f2ec8068930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:18:53.048 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:53.041+0000 7f2eec11c640 1 --2- 172.21.15.143:0/2858544564 >> [v2:172.21.15.143:6832/2150557544,v1:172.21.15.143:6833/2150557544] conn(0x7f2ec8066470 0x7f2ec8068930 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7f2ed40af000 tx=0x7f2ed40a7000 comp rx=0 tx=0).stop 2023-11-26T07:18:53.048 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:53.041+0000 7f2eec11c640 1 -- 172.21.15.143:0/2858544564 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f2ee4070e00 msgr2=0x7f2ee4160f50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:18:53.049 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:53.041+0000 7f2eec11c640 1 --2- 172.21.15.143:0/2858544564 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f2ee4070e00 0x7f2ee4160f50 secure :-1 s=READY pgs=397 cs=0 l=1 rev1=1 crypto rx=0x7f2ee000c500 tx=0x7f2ee000c9c0 comp rx=0 tx=0).stop 2023-11-26T07:18:53.049 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:53.041+0000 7f2eec11c640 1 -- 172.21.15.143:0/2858544564 shutdown_connections 2023-11-26T07:18:53.049 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:53.041+0000 7f2eec11c640 1 --2- 172.21.15.143:0/2858544564 >> [v2:172.21.15.143:6832/2150557544,v1:172.21.15.143:6833/2150557544] conn(0x7f2ec8066470 0x7f2ec8068930 unknown :-1 s=CLOSED pgs=216 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:18:53.049 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:53.041+0000 7f2eec11c640 1 --2- 172.21.15.143:0/2858544564 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f2ee414b1e0 0x7f2ee414dbe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:18:53.049 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:53.041+0000 7f2eec11c640 1 --2- 172.21.15.143:0/2858544564 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f2ee40cf990 0x7f2ee40ad990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:18:53.049 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:53.041+0000 7f2eec11c640 1 --2- 172.21.15.143:0/2858544564 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f2ee4070e00 0x7f2ee4160f50 unknown :-1 s=CLOSED pgs=397 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:18:53.049 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:53.041+0000 7f2eec11c640 1 -- 172.21.15.143:0/2858544564 >> 172.21.15.143:0/2858544564 conn(0x7f2ee406a890 msgr2=0x7f2ee40a84d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:18:53.049 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:53.041+0000 7f2eec11c640 1 -- 172.21.15.143:0/2858544564 shutdown_connections 2023-11-26T07:18:53.049 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:53.041+0000 7f2eec11c640 1 -- 172.21.15.143:0/2858544564 wait complete. 2023-11-26T07:18:53.067 DEBUG:teuthology.orchestra.run.smithi143:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-11-26T07:18:53.204 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:53.197+0000 7fda32372640 1 Processor -- start 2023-11-26T07:18:53.204 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:53.197+0000 7fda32372640 1 -- start start 2023-11-26T07:18:53.206 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:53.201+0000 7fda32372640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fda2c0cf280 0x7fda2c0cf6e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:18:53.206 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:53.201+0000 7fda32372640 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7fda2c0cd1b0 0x7fda2c0cd5e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:18:53.206 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:53.201+0000 7fda32372640 1 -- --> v1:172.21.15.143:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fda2c0cfc20 con 0x7fda2c143ac0 2023-11-26T07:18:53.207 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:53.201+0000 7fda32372640 1 -- --> v2:172.21.15.143:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fda2c143ea0 con 0x7fda2c0cd1b0 2023-11-26T07:18:53.207 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:53.201+0000 7fda32372640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fda2c0cc730 con 0x7fda2c0cf280 2023-11-26T07:18:53.207 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:53.201+0000 7fda3090f640 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7fda2c0cd1b0 0x7fda2c0cd5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:18:53.207 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:53.201+0000 7fda2bfff640 1 --1- >> v1:172.21.15.143:6789/0 conn(0x7fda2c143ac0 0x7fda2c070370 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.143:6789/0 says I am v1:172.21.15.143:55198/0 (socket says 172.21.15.143:55198) 2023-11-26T07:18:53.207 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:53.201+0000 7fda2bfff640 1 -- 172.21.15.143:0/106079261 learned_addr learned my addr 172.21.15.143:0/106079261 (peer_addr_for_me v1:172.21.15.143:0/0) 2023-11-26T07:18:53.207 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:53.201+0000 7fda2b7fe640 1 --2- 172.21.15.143:0/106079261 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fda2c0cf280 0x7fda2c0cf6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:18:53.207 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:53.201+0000 7fda3090f640 1 -- 172.21.15.143:0/106079261 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fda2c0cf280 msgr2=0x7fda2c0cf6e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-26T07:18:53.207 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:53.201+0000 7fda3090f640 1 --2- 172.21.15.143:0/106079261 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fda2c0cf280 0x7fda2c0cf6e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:18:53.207 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:53.201+0000 7fda3090f640 1 -- 172.21.15.143:0/106079261 >> v1:172.21.15.143:6789/0 conn(0x7fda2c143ac0 legacy=0x7fda2c070370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:18:53.207 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:53.201+0000 7fda3090f640 1 -- 172.21.15.143:0/106079261 --> v2:172.21.15.143:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fda2c070ee0 con 0x7fda2c0cd1b0 2023-11-26T07:18:53.209 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:53.205+0000 7fda3090f640 1 --2- 172.21.15.143:0/106079261 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fda2c0cd1b0 0x7fda2c0cd5e0 secure :-1 s=READY pgs=399 cs=0 l=1 rev1=1 crypto rx=0x7fda24009d90 tx=0x7fda2401c950 comp rx=0 tx=0).ready entity=mon.0 client_cookie=75f9e4457a0d4dd9 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:18:53.209 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:53.205+0000 7fda2affd640 1 -- 172.21.15.143:0/106079261 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fda240a5a10 con 0x7fda2c0cd1b0 2023-11-26T07:18:53.210 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:53.205+0000 7fda2affd640 1 -- 172.21.15.143:0/106079261 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fda240a5b70 con 0x7fda2c0cd1b0 2023-11-26T07:18:53.210 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:53.205+0000 7fda2affd640 1 -- 172.21.15.143:0/106079261 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fda240ad440 con 0x7fda2c0cd1b0 2023-11-26T07:18:53.210 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:53.205+0000 7fda32372640 1 -- 172.21.15.143:0/106079261 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fda2c0cd1b0 msgr2=0x7fda2c0cd5e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:18:53.211 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:53.205+0000 7fda32372640 1 --2- 172.21.15.143:0/106079261 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fda2c0cd1b0 0x7fda2c0cd5e0 secure :-1 s=READY pgs=399 cs=0 l=1 rev1=1 crypto rx=0x7fda24009d90 tx=0x7fda2401c950 comp rx=0 tx=0).stop 2023-11-26T07:18:53.211 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:53.205+0000 7fda32372640 1 -- 172.21.15.143:0/106079261 shutdown_connections 2023-11-26T07:18:53.211 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:53.205+0000 7fda32372640 1 --2- 172.21.15.143:0/106079261 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fda2c0cd1b0 0x7fda2c0cd5e0 unknown :-1 s=CLOSED pgs=399 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:18:53.211 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:53.205+0000 7fda32372640 1 --2- 172.21.15.143:0/106079261 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fda2c0cf280 0x7fda2c0cf6e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:18:53.211 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:53.205+0000 7fda32372640 1 -- 172.21.15.143:0/106079261 >> 172.21.15.143:0/106079261 conn(0x7fda2c06a890 msgr2=0x7fda2c06aca0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:18:53.211 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:53.205+0000 7fda32372640 1 -- 172.21.15.143:0/106079261 shutdown_connections 2023-11-26T07:18:53.211 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:53.205+0000 7fda32372640 1 -- 172.21.15.143:0/106079261 wait complete. 2023-11-26T07:18:53.212 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:53.205+0000 7fda32372640 1 Processor -- start 2023-11-26T07:18:53.212 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:53.205+0000 7fda32372640 1 -- start start 2023-11-26T07:18:53.214 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:53.209+0000 7fda32372640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fda2c0cd1b0 0x7fda2c158450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:18:53.214 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:53.209+0000 7fda32372640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fda2c0cf280 0x7fda2c157d20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:18:53.215 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:53.209+0000 7fda32372640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fda2c143ac0 0x7fda2c06c520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:18:53.215 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:53.209+0000 7fda32372640 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fda2c158260 con 0x7fda2c143ac0 2023-11-26T07:18:53.215 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:53.209+0000 7fda32372640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fda2c158a50 con 0x7fda2c0cd1b0 2023-11-26T07:18:53.215 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:53.209+0000 7fda32372640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fda2c158d30 con 0x7fda2c0cf280 2023-11-26T07:18:53.215 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:53.209+0000 7fda3090f640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fda2c143ac0 0x7fda2c06c520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:18:53.215 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:53.209+0000 7fda3090f640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fda2c143ac0 0x7fda2c06c520 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:47608/0 (socket says 172.21.15.143:47608) 2023-11-26T07:18:53.215 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:53.209+0000 7fda3090f640 1 -- 172.21.15.143:0/1940724738 learned_addr learned my addr 172.21.15.143:0/1940724738 (peer_addr_for_me v2:172.21.15.143:0/0) 2023-11-26T07:18:53.215 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:53.209+0000 7fda2b7fe640 1 --2- 172.21.15.143:0/1940724738 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fda2c0cf280 0x7fda2c157d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:18:53.215 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:53.209+0000 7fda2bfff640 1 --2- 172.21.15.143:0/1940724738 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fda2c0cd1b0 0x7fda2c158450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:18:53.215 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:53.209+0000 7fda3090f640 1 -- 172.21.15.143:0/1940724738 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fda2c0cf280 msgr2=0x7fda2c157d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:18:53.215 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:53.209+0000 7fda3090f640 1 --2- 172.21.15.143:0/1940724738 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fda2c0cf280 0x7fda2c157d20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:18:53.216 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:53.209+0000 7fda3090f640 1 -- 172.21.15.143:0/1940724738 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fda2c0cd1b0 msgr2=0x7fda2c158450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:18:53.216 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:53.209+0000 7fda3090f640 1 --2- 172.21.15.143:0/1940724738 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fda2c0cd1b0 0x7fda2c158450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:18:53.216 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:53.209+0000 7fda3090f640 1 -- 172.21.15.143:0/1940724738 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fda24009a70 con 0x7fda2c143ac0 2023-11-26T07:18:53.216 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:53.209+0000 7fda2b7fe640 1 --2- 172.21.15.143:0/1940724738 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fda2c0cf280 0x7fda2c157d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-26T07:18:53.216 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:53.209+0000 7fda3090f640 1 --2- 172.21.15.143:0/1940724738 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fda2c143ac0 0x7fda2c06c520 secure :-1 s=READY pgs=400 cs=0 l=1 rev1=1 crypto rx=0x7fda240b0040 tx=0x7fda240196f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:18:53.216 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:53.209+0000 7fda28ff9640 1 -- 172.21.15.143:0/1940724738 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fda240ade40 con 0x7fda2c143ac0 2023-11-26T07:18:53.216 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:53.209+0000 7fda28ff9640 1 -- 172.21.15.143:0/1940724738 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fda24009450 con 0x7fda2c143ac0 2023-11-26T07:18:53.216 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:53.209+0000 7fda28ff9640 1 -- 172.21.15.143:0/1940724738 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fda240b9690 con 0x7fda2c143ac0 2023-11-26T07:18:53.216 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:53.209+0000 7fda32372640 1 -- 172.21.15.143:0/1940724738 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fda2c159010 con 0x7fda2c143ac0 2023-11-26T07:18:53.216 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:53.209+0000 7fda32372640 1 -- 172.21.15.143:0/1940724738 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fda2c1595d0 con 0x7fda2c143ac0 2023-11-26T07:18:53.217 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:53.209+0000 7fda32372640 1 -- 172.21.15.143:0/1940724738 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fda2c067680 con 0x7fda2c143ac0 2023-11-26T07:18:53.217 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:53.209+0000 7fda28ff9640 1 -- 172.21.15.143:0/1940724738 <== mon.0 v2:172.21.15.143:3300/0 4 ==== mgrmap(e 5) v1 ==== 85852+0+0 (secure 0 0 0) 0x7fda240b97f0 con 0x7fda2c143ac0 2023-11-26T07:18:53.217 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:53.213+0000 7fda28ff9640 1 --2- 172.21.15.143:0/1940724738 >> [v2:172.21.15.143:6832/2150557544,v1:172.21.15.143:6833/2150557544] conn(0x7fda18066470 0x7fda18068930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:18:53.218 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:53.213+0000 7fda28ff9640 1 -- 172.21.15.143:0/1940724738 <== mon.0 v2:172.21.15.143:3300/0 5 ==== osd_map(32..32 src has 1..32) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fda2411f660 con 0x7fda2c143ac0 2023-11-26T07:18:53.218 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:53.213+0000 7fda2bfff640 1 --2- 172.21.15.143:0/1940724738 >> [v2:172.21.15.143:6832/2150557544,v1:172.21.15.143:6833/2150557544] conn(0x7fda18066470 0x7fda18068930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:18:53.218 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:53.213+0000 7fda2bfff640 1 --2- 172.21.15.143:0/1940724738 >> [v2:172.21.15.143:6832/2150557544,v1:172.21.15.143:6833/2150557544] conn(0x7fda18066470 0x7fda18068930 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7fda1c005890 tx=0x7fda1c009040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:18:53.221 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:53.217+0000 7fda28ff9640 1 -- 172.21.15.143:0/1940724738 <== mon.0 v2:172.21.15.143:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172649 (secure 0 0 0) 0x7fda240eca30 con 0x7fda2c143ac0 2023-11-26T07:18:53.372 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:53.365+0000 7fda32372640 1 -- 172.21.15.143:0/1940724738 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fda2c140c70 con 0x7fda2c143ac0 2023-11-26T07:18:53.373 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:53.369+0000 7fda28ff9640 1 -- 172.21.15.143:0/1940724738 <== mon.0 v2:172.21.15.143:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v32) v1 ==== 74+0+16618 (secure 0 0 0) 0x7fda240f0050 con 0x7fda2c143ac0 2023-11-26T07:18:53.373 INFO:teuthology.orchestra.run.smithi143.stdout: 2023-11-26T07:18:53.374 INFO:teuthology.orchestra.run.smithi143.stdout:{"epoch":32,"fsid":"f5033060-2469-4a07-95fe-2b76f5404bc1","created":"2023-11-26T07:17:28.613663+0000","modified":"2023-11-26T07:18:52.014230+0000","last_up_change":"2023-11-26T07:17:41.157813+0000","last_in_change":"2023-11-26T07:17:31.181658+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":5,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-11-26T07:17:41.758597+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_metadata","create_time":"2023-11-26T07:18:50.535934+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"32","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":5,"pool_name":"cephfs_data","create_time":"2023-11-26T07:18:51.008494+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"32","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"ebbee2c6-023e-44b9-9f1d-6f7bfd309730","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6817","nonce":639807369},{"type":"v1","addr":"172.21.15.143:6819","nonce":639807369}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6821","nonce":639807369},{"type":"v1","addr":"172.21.15.143:6823","nonce":639807369}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6830","nonce":639807369},{"type":"v1","addr":"172.21.15.143:6831","nonce":639807369}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6825","nonce":639807369},{"type":"v1","addr":"172.21.15.143:6828","nonce":639807369}]},"public_addr":"172.21.15.143:6819/639807369","cluster_addr":"172.21.15.143:6823/639807369","heartbeat_back_addr":"172.21.15.143:6831/639807369","heartbeat_front_addr":"172.21.15.143:6828/639807369","state":["exists","up"]},{"osd":1,"uuid":"fec4b8be-7122-49e9-9a19-5850a0700001","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6816","nonce":1144942530},{"type":"v1","addr":"172.21.15.143:6818","nonce":1144942530}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6820","nonce":1144942530},{"type":"v1","addr":"172.21.15.143:6822","nonce":1144942530}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6827","nonce":1144942530},{"type":"v1","addr":"172.21.15.143:6829","nonce":1144942530}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6824","nonce":1144942530},{"type":"v1","addr":"172.21.15.143:6826","nonce":1144942530}]},"public_addr":"172.21.15.143:6818/1144942530","cluster_addr":"172.21.15.143:6822/1144942530","heartbeat_back_addr":"172.21.15.143:6829/1144942530","heartbeat_front_addr":"172.21.15.143:6826/1144942530","state":["exists","up"]},{"osd":2,"uuid":"2888af5b-f933-46ca-8f95-6dd494ff43f6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6808","nonce":3909095186},{"type":"v1","addr":"172.21.15.143:6809","nonce":3909095186}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6810","nonce":3909095186},{"type":"v1","addr":"172.21.15.143:6811","nonce":3909095186}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6814","nonce":3909095186},{"type":"v1","addr":"172.21.15.143:6815","nonce":3909095186}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6812","nonce":3909095186},{"type":"v1","addr":"172.21.15.143:6813","nonce":3909095186}]},"public_addr":"172.21.15.143:6809/3909095186","cluster_addr":"172.21.15.143:6811/3909095186","heartbeat_back_addr":"172.21.15.143:6815/3909095186","heartbeat_front_addr":"172.21.15.143:6813/3909095186","state":["exists","up"]},{"osd":3,"uuid":"d1f52435-7bc1-4081-9550-9cd1ba55f549","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6800","nonce":3854210991},{"type":"v1","addr":"172.21.15.143:6801","nonce":3854210991}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6802","nonce":3854210991},{"type":"v1","addr":"172.21.15.143:6803","nonce":3854210991}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6806","nonce":3854210991},{"type":"v1","addr":"172.21.15.143:6807","nonce":3854210991}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6804","nonce":3854210991},{"type":"v1","addr":"172.21.15.143:6805","nonce":3854210991}]},"public_addr":"172.21.15.143:6801/3854210991","cluster_addr":"172.21.15.143:6803/3854210991","heartbeat_back_addr":"172.21.15.143:6807/3854210991","heartbeat_front_addr":"172.21.15.143:6805/3854210991","state":["exists","up"]},{"osd":4,"uuid":"e2ae8b6e-6e25-49b2-bcbf-aacce68f06a3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6800","nonce":3924606902},{"type":"v1","addr":"172.21.15.188:6801","nonce":3924606902}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6802","nonce":3924606902},{"type":"v1","addr":"172.21.15.188:6803","nonce":3924606902}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6806","nonce":3924606902},{"type":"v1","addr":"172.21.15.188:6807","nonce":3924606902}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6804","nonce":3924606902},{"type":"v1","addr":"172.21.15.188:6805","nonce":3924606902}]},"public_addr":"172.21.15.188:6801/3924606902","cluster_addr":"172.21.15.188:6803/3924606902","heartbeat_back_addr":"172.21.15.188:6807/3924606902","heartbeat_front_addr":"172.21.15.188:6805/3924606902","state":["exists","up"]},{"osd":5,"uuid":"3cec8a32-7698-4c36-8e89-e4ba11934f33","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6813","nonce":186841544},{"type":"v1","addr":"172.21.15.188:6815","nonce":186841544}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6817","nonce":186841544},{"type":"v1","addr":"172.21.15.188:6818","nonce":186841544}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6822","nonce":186841544},{"type":"v1","addr":"172.21.15.188:6823","nonce":186841544}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6820","nonce":186841544},{"type":"v1","addr":"172.21.15.188:6821","nonce":186841544}]},"public_addr":"172.21.15.188:6815/186841544","cluster_addr":"172.21.15.188:6818/186841544","heartbeat_back_addr":"172.21.15.188:6823/186841544","heartbeat_front_addr":"172.21.15.188:6821/186841544","state":["exists","up"]},{"osd":6,"uuid":"58a65d5d-e090-473a-81b4-ad38e85029f9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6808","nonce":2153499405},{"type":"v1","addr":"172.21.15.188:6809","nonce":2153499405}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6810","nonce":2153499405},{"type":"v1","addr":"172.21.15.188:6811","nonce":2153499405}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6816","nonce":2153499405},{"type":"v1","addr":"172.21.15.188:6819","nonce":2153499405}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6812","nonce":2153499405},{"type":"v1","addr":"172.21.15.188:6814","nonce":2153499405}]},"public_addr":"172.21.15.188:6809/2153499405","cluster_addr":"172.21.15.188:6811/2153499405","heartbeat_back_addr":"172.21.15.188:6819/2153499405","heartbeat_front_addr":"172.21.15.188:6814/2153499405","state":["exists","up"]},{"osd":7,"uuid":"ea5b7f70-3554-4cf8-ac30-f7c59b80678d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6824","nonce":2722096729},{"type":"v1","addr":"172.21.15.188:6825","nonce":2722096729}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6826","nonce":2722096729},{"type":"v1","addr":"172.21.15.188:6827","nonce":2722096729}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6830","nonce":2722096729},{"type":"v1","addr":"172.21.15.188:6831","nonce":2722096729}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6828","nonce":2722096729},{"type":"v1","addr":"172.21.15.188:6829","nonce":2722096729}]},"public_addr":"172.21.15.188:6825/2722096729","cluster_addr":"172.21.15.188:6827/2722096729","heartbeat_back_addr":"172.21.15.188:6831/2722096729","heartbeat_front_addr":"172.21.15.188:6829/2722096729","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-26T07:17:36.995032+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-26T07:17:36.932266+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-26T07:17:36.829304+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-26T07:17:36.804297+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-26T07:17:36.667892+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-26T07:17:36.964868+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-26T07:17:36.827332+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-26T07:17:36.893179+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-11-26T07:18:53.376 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:53.369+0000 7fda32372640 1 -- 172.21.15.143:0/1940724738 >> [v2:172.21.15.143:6832/2150557544,v1:172.21.15.143:6833/2150557544] conn(0x7fda18066470 msgr2=0x7fda18068930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:18:53.376 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:53.369+0000 7fda32372640 1 --2- 172.21.15.143:0/1940724738 >> [v2:172.21.15.143:6832/2150557544,v1:172.21.15.143:6833/2150557544] conn(0x7fda18066470 0x7fda18068930 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7fda1c005890 tx=0x7fda1c009040 comp rx=0 tx=0).stop 2023-11-26T07:18:53.376 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:53.369+0000 7fda32372640 1 -- 172.21.15.143:0/1940724738 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fda2c143ac0 msgr2=0x7fda2c06c520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:18:53.376 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:53.369+0000 7fda32372640 1 --2- 172.21.15.143:0/1940724738 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fda2c143ac0 0x7fda2c06c520 secure :-1 s=READY pgs=400 cs=0 l=1 rev1=1 crypto rx=0x7fda240b0040 tx=0x7fda240196f0 comp rx=0 tx=0).stop 2023-11-26T07:18:53.376 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:53.369+0000 7fda32372640 1 -- 172.21.15.143:0/1940724738 shutdown_connections 2023-11-26T07:18:53.376 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:53.369+0000 7fda32372640 1 --2- 172.21.15.143:0/1940724738 >> [v2:172.21.15.143:6832/2150557544,v1:172.21.15.143:6833/2150557544] conn(0x7fda18066470 0x7fda18068930 unknown :-1 s=CLOSED pgs=217 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:18:53.376 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:53.369+0000 7fda32372640 1 --2- 172.21.15.143:0/1940724738 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fda2c143ac0 0x7fda2c06c520 unknown :-1 s=CLOSED pgs=400 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:18:53.376 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:53.369+0000 7fda32372640 1 --2- 172.21.15.143:0/1940724738 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fda2c0cf280 0x7fda2c157d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:18:53.376 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:53.369+0000 7fda32372640 1 --2- 172.21.15.143:0/1940724738 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fda2c0cd1b0 0x7fda2c158450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:18:53.376 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:53.369+0000 7fda32372640 1 -- 172.21.15.143:0/1940724738 >> 172.21.15.143:0/1940724738 conn(0x7fda2c06a890 msgr2=0x7fda2c0cb0e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:18:53.376 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:53.369+0000 7fda32372640 1 -- 172.21.15.143:0/1940724738 shutdown_connections 2023-11-26T07:18:53.376 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:53.369+0000 7fda32372640 1 -- 172.21.15.143:0/1940724738 wait complete. 2023-11-26T07:18:53.394 DEBUG:teuthology.orchestra.run.smithi143:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs standby_count_wanted 0 2023-11-26T07:18:53.535 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:53.529+0000 7f964ce79640 1 Processor -- start 2023-11-26T07:18:53.536 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:53.529+0000 7f964ce79640 1 -- start start 2023-11-26T07:18:53.537 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:53.533+0000 7f964ce79640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f96480ce680 0x7f9648071850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:18:53.537 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:53.533+0000 7f964ce79640 1 -- --> v1:172.21.15.143:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f964806c750 con 0x7f96481596d0 2023-11-26T07:18:53.537 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:53.533+0000 7f964ce79640 1 -- --> v1:172.21.15.188:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f96480b3550 con 0x7f96480b2f80 2023-11-26T07:18:53.537 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:53.533+0000 7f964ce79640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f96480cbe00 con 0x7f96480ce680 2023-11-26T07:18:53.537 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:53.533+0000 7f964659c640 1 --1- >> v1:172.21.15.143:6789/0 conn(0x7f96481596d0 0x7f96480b2870 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.143:6789/0 says I am v1:172.21.15.143:55214/0 (socket says 172.21.15.143:55214) 2023-11-26T07:18:53.537 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:53.533+0000 7f964659c640 1 -- 172.21.15.143:0/757716073 learned_addr learned my addr 172.21.15.143:0/757716073 (peer_addr_for_me v1:172.21.15.143:0/0) 2023-11-26T07:18:53.538 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:53.533+0000 7f9646d9d640 1 --2- 172.21.15.143:0/757716073 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f96480ce680 0x7f9648071850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:18:53.538 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:53.533+0000 7f9645d9b640 1 -- 172.21.15.143:0/757716073 <== mon.0 v1:172.21.15.143:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2369671649 0 0) 0x7f964806c750 con 0x7f96481596d0 2023-11-26T07:18:53.538 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:53.533+0000 7f9645d9b640 1 -- 172.21.15.143:0/757716073 --> v1:172.21.15.143:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f9634003450 con 0x7f96481596d0 2023-11-26T07:18:53.538 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:53.533+0000 7f9645d9b640 1 -- 172.21.15.143:0/757716073 <== mon.1 v1:172.21.15.188:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1316150565 0 0) 0x7f96480b3550 con 0x7f96480b2f80 2023-11-26T07:18:53.538 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:53.533+0000 7f9646d9d640 1 -- 172.21.15.143:0/757716073 >> v1:172.21.15.188:6789/0 conn(0x7f96480b2f80 legacy=0x7f964806cbe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:18:53.538 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:53.533+0000 7f9646d9d640 1 -- 172.21.15.143:0/757716073 >> v1:172.21.15.143:6789/0 conn(0x7f96481596d0 legacy=0x7f96480b2870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:18:53.538 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:53.533+0000 7f9646d9d640 1 -- 172.21.15.143:0/757716073 --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9648071e80 con 0x7f96480ce680 2023-11-26T07:18:53.541 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:53.537+0000 7f9646d9d640 1 --2- 172.21.15.143:0/757716073 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f96480ce680 0x7f9648071850 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7f9640009dc0 tx=0x7f964001c970 comp rx=0 tx=0).ready entity=mon.2 client_cookie=80273d899fd7753e server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:18:53.541 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:53.537+0000 7f9645d9b640 1 -- 172.21.15.143:0/757716073 <== mon.2 v2:172.21.15.188:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f96400a58a0 con 0x7f96480ce680 2023-11-26T07:18:53.541 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:53.537+0000 7f9645d9b640 1 -- 172.21.15.143:0/757716073 <== mon.2 v2:172.21.15.188:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f96400a5a00 con 0x7f96480ce680 2023-11-26T07:18:53.541 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:53.537+0000 7f9645d9b640 1 -- 172.21.15.143:0/757716073 <== mon.2 v2:172.21.15.188:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f96400a5d90 con 0x7f96480ce680 2023-11-26T07:18:53.542 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:53.537+0000 7f964ce79640 1 -- 172.21.15.143:0/757716073 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f96480ce680 msgr2=0x7f9648071850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:18:53.542 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:53.537+0000 7f964ce79640 1 --2- 172.21.15.143:0/757716073 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f96480ce680 0x7f9648071850 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7f9640009dc0 tx=0x7f964001c970 comp rx=0 tx=0).stop 2023-11-26T07:18:53.542 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:53.537+0000 7f964ce79640 1 -- 172.21.15.143:0/757716073 shutdown_connections 2023-11-26T07:18:53.542 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:53.537+0000 7f964ce79640 1 --2- 172.21.15.143:0/757716073 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f96480ce680 0x7f9648071850 unknown :-1 s=CLOSED pgs=75 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:18:53.542 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:53.537+0000 7f964ce79640 1 -- 172.21.15.143:0/757716073 >> 172.21.15.143:0/757716073 conn(0x7f964806a890 msgr2=0x7f964806aca0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:18:53.543 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:53.537+0000 7f964ce79640 1 -- 172.21.15.143:0/757716073 shutdown_connections 2023-11-26T07:18:53.543 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:53.537+0000 7f964ce79640 1 -- 172.21.15.143:0/757716073 wait complete. 2023-11-26T07:18:53.544 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:53.537+0000 7f964ce79640 1 Processor -- start 2023-11-26T07:18:53.544 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:53.537+0000 7f964ce79640 1 -- start start 2023-11-26T07:18:53.546 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:53.541+0000 7f964ce79640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f96480b2f80 0x7f96480ca2b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:18:53.546 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:53.541+0000 7f964ce79640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f96480ce680 0x7f964814f4a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:18:53.546 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:53.541+0000 7f964ce79640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f96481596d0 0x7f964813c180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:18:53.546 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:53.541+0000 7f964ce79640 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f964814f010 con 0x7f96480b2f80 2023-11-26T07:18:53.546 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:53.541+0000 7f964ce79640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f964814e400 con 0x7f96481596d0 2023-11-26T07:18:53.546 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:53.541+0000 7f964ce79640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f964814e6e0 con 0x7f96480ce680 2023-11-26T07:18:53.546 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:53.541+0000 7f964659c640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f96480b2f80 0x7f96480ca2b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:18:53.546 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:53.541+0000 7f964659c640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f96480b2f80 0x7f96480ca2b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:47620/0 (socket says 172.21.15.143:47620) 2023-11-26T07:18:53.546 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:53.541+0000 7f964659c640 1 -- 172.21.15.143:0/1917801275 learned_addr learned my addr 172.21.15.143:0/1917801275 (peer_addr_for_me v2:172.21.15.143:0/0) 2023-11-26T07:18:53.546 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:53.541+0000 7f963ffff640 1 --2- 172.21.15.143:0/1917801275 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f96480ce680 0x7f964814f4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:18:53.546 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:53.541+0000 7f9646d9d640 1 --2- 172.21.15.143:0/1917801275 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f96481596d0 0x7f964813c180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:18:53.547 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:53.541+0000 7f964659c640 1 -- 172.21.15.143:0/1917801275 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f96480ce680 msgr2=0x7f964814f4a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:18:53.547 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:53.541+0000 7f964659c640 1 --2- 172.21.15.143:0/1917801275 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f96480ce680 0x7f964814f4a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:18:53.547 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:53.541+0000 7f964659c640 1 -- 172.21.15.143:0/1917801275 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f96481596d0 msgr2=0x7f964813c180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:18:53.547 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:53.541+0000 7f964659c640 1 --2- 172.21.15.143:0/1917801275 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f96481596d0 0x7f964813c180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:18:53.547 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:53.541+0000 7f964659c640 1 -- 172.21.15.143:0/1917801275 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9640009a70 con 0x7f96480b2f80 2023-11-26T07:18:53.547 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:53.541+0000 7f964659c640 1 --2- 172.21.15.143:0/1917801275 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f96480b2f80 0x7f96480ca2b0 secure :-1 s=READY pgs=402 cs=0 l=1 rev1=1 crypto rx=0x7f963800c3c0 tx=0x7f963800c880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:18:53.548 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:53.541+0000 7f963effd640 1 -- 172.21.15.143:0/1917801275 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9638005cf0 con 0x7f96480b2f80 2023-11-26T07:18:53.548 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:53.541+0000 7f963effd640 1 -- 172.21.15.143:0/1917801275 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9638005e50 con 0x7f96480b2f80 2023-11-26T07:18:53.548 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:53.541+0000 7f964ce79640 1 -- 172.21.15.143:0/1917801275 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f964814e9c0 con 0x7f96480b2f80 2023-11-26T07:18:53.548 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:53.541+0000 7f963effd640 1 -- 172.21.15.143:0/1917801275 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f96380072f0 con 0x7f96480b2f80 2023-11-26T07:18:53.548 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:53.541+0000 7f964ce79640 1 -- 172.21.15.143:0/1917801275 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f96480ca7f0 con 0x7f96480b2f80 2023-11-26T07:18:53.548 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:53.541+0000 7f964ce79640 1 -- 172.21.15.143:0/1917801275 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9648067680 con 0x7f96480b2f80 2023-11-26T07:18:53.548 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:53.541+0000 7f963effd640 1 -- 172.21.15.143:0/1917801275 <== mon.0 v2:172.21.15.143:3300/0 4 ==== mgrmap(e 5) v1 ==== 85852+0+0 (secure 0 0 0) 0x7f9638007450 con 0x7f96480b2f80 2023-11-26T07:18:53.549 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:53.545+0000 7f963effd640 1 --2- 172.21.15.143:0/1917801275 >> [v2:172.21.15.143:6832/2150557544,v1:172.21.15.143:6833/2150557544] conn(0x7f96280664a0 0x7f9628068960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:18:53.550 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:53.545+0000 7f963effd640 1 -- 172.21.15.143:0/1917801275 <== mon.0 v2:172.21.15.143:3300/0 5 ==== osd_map(32..32 src has 1..32) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f9638083380 con 0x7f96480b2f80 2023-11-26T07:18:53.550 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:53.545+0000 7f963ffff640 1 --2- 172.21.15.143:0/1917801275 >> [v2:172.21.15.143:6832/2150557544,v1:172.21.15.143:6833/2150557544] conn(0x7f96280664a0 0x7f9628068960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:18:53.550 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:53.545+0000 7f963ffff640 1 --2- 172.21.15.143:0/1917801275 >> [v2:172.21.15.143:6832/2150557544,v1:172.21.15.143:6833/2150557544] conn(0x7f96280664a0 0x7f9628068960 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7f964813f330 tx=0x7f9630009040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:18:53.554 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:53.549+0000 7f963effd640 1 -- 172.21.15.143:0/1917801275 <== mon.0 v2:172.21.15.143:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172649 (secure 0 0 0) 0x7f96380508b0 con 0x7f96480b2f80 2023-11-26T07:18:53.740 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:53.733+0000 7f964ce79640 1 -- 172.21.15.143:0/1917801275 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"} v 0) v1 -- 0x7f964815dc70 con 0x7f96480b2f80 2023-11-26T07:18:54.674 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:54.669+0000 7f963effd640 1 -- 172.21.15.143:0/1917801275 <== mon.0 v2:172.21.15.143:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]=0 v15) v1 ==== 118+0+0 (secure 0 0 0) 0x7f96380170b0 con 0x7f96480b2f80 2023-11-26T07:18:54.676 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:54.669+0000 7f964ce79640 1 -- 172.21.15.143:0/1917801275 >> [v2:172.21.15.143:6832/2150557544,v1:172.21.15.143:6833/2150557544] conn(0x7f96280664a0 msgr2=0x7f9628068960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:18:54.676 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:54.669+0000 7f964ce79640 1 --2- 172.21.15.143:0/1917801275 >> [v2:172.21.15.143:6832/2150557544,v1:172.21.15.143:6833/2150557544] conn(0x7f96280664a0 0x7f9628068960 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7f964813f330 tx=0x7f9630009040 comp rx=0 tx=0).stop 2023-11-26T07:18:54.677 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:54.669+0000 7f964ce79640 1 -- 172.21.15.143:0/1917801275 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f96480b2f80 msgr2=0x7f96480ca2b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:18:54.677 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:54.669+0000 7f964ce79640 1 --2- 172.21.15.143:0/1917801275 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f96480b2f80 0x7f96480ca2b0 secure :-1 s=READY pgs=402 cs=0 l=1 rev1=1 crypto rx=0x7f963800c3c0 tx=0x7f963800c880 comp rx=0 tx=0).stop 2023-11-26T07:18:54.677 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:54.673+0000 7f964ce79640 1 -- 172.21.15.143:0/1917801275 shutdown_connections 2023-11-26T07:18:54.677 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:54.673+0000 7f964ce79640 1 --2- 172.21.15.143:0/1917801275 >> [v2:172.21.15.143:6832/2150557544,v1:172.21.15.143:6833/2150557544] conn(0x7f96280664a0 0x7f9628068960 unknown :-1 s=CLOSED pgs=218 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:18:54.677 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:54.673+0000 7f964ce79640 1 --2- 172.21.15.143:0/1917801275 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f96481596d0 0x7f964813c180 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:18:54.677 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:54.673+0000 7f964ce79640 1 --2- 172.21.15.143:0/1917801275 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f96480ce680 0x7f964814f4a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:18:54.678 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:54.673+0000 7f964ce79640 1 --2- 172.21.15.143:0/1917801275 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f96480b2f80 0x7f96480ca2b0 unknown :-1 s=CLOSED pgs=402 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:18:54.678 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:54.673+0000 7f964ce79640 1 -- 172.21.15.143:0/1917801275 >> 172.21.15.143:0/1917801275 conn(0x7f964806a890 msgr2=0x7f964815cf80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:18:54.678 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:54.673+0000 7f964ce79640 1 -- 172.21.15.143:0/1917801275 shutdown_connections 2023-11-26T07:18:54.678 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:54.673+0000 7f964ce79640 1 -- 172.21.15.143:0/1917801275 wait complete. 2023-11-26T07:18:54.699 DEBUG:tasks.cephfs.filesystem:fs_config: {} 2023-11-26T07:18:54.700 DEBUG:teuthology.orchestra.run.smithi143:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs allow_standby_replay false 2023-11-26T07:18:54.846 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:54.841+0000 7fdc1d557640 1 Processor -- start 2023-11-26T07:18:54.846 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:54.841+0000 7fdc1d557640 1 -- start start 2023-11-26T07:18:54.848 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:54.841+0000 7fdc1d557640 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7fdc18143ac0 0x7fdc18070370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:18:54.848 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:54.841+0000 7fdc1d557640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fdc180cf990 0x7fdc180cd1b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:18:54.848 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:54.841+0000 7fdc1d557640 1 -- --> v1:172.21.15.188:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fdc180cc730 con 0x7fdc180708b0 2023-11-26T07:18:54.848 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:54.841+0000 7fdc1d557640 1 -- --> v2:172.21.15.143:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fdc18143ea0 con 0x7fdc18143ac0 2023-11-26T07:18:54.849 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:54.841+0000 7fdc1d557640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdc18070ee0 con 0x7fdc180cf990 2023-11-26T07:18:54.849 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:54.841+0000 7fdc16ffd640 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7fdc18143ac0 0x7fdc18070370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:18:54.849 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:54.841+0000 7fdc16ffd640 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7fdc18143ac0 0x7fdc18070370 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:51652/0 (socket says 172.21.15.143:51652) 2023-11-26T07:18:54.849 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:54.841+0000 7fdc16ffd640 1 -- 172.21.15.143:0/3499308366 learned_addr learned my addr 172.21.15.143:0/3499308366 (peer_addr_for_me v2:172.21.15.143:0/0) 2023-11-26T07:18:54.849 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:54.841+0000 7fdc177fe640 1 --2- 172.21.15.143:0/3499308366 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fdc180cf990 0x7fdc180cd1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:18:54.849 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:54.841+0000 7fdc16ffd640 1 -- 172.21.15.143:0/3499308366 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fdc180cf990 msgr2=0x7fdc180cd1b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-26T07:18:54.849 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:54.841+0000 7fdc16ffd640 1 --2- 172.21.15.143:0/3499308366 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fdc180cf990 0x7fdc180cd1b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:18:54.849 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:54.841+0000 7fdc16ffd640 1 -- 172.21.15.143:0/3499308366 >> v1:172.21.15.188:6789/0 conn(0x7fdc180708b0 legacy=0x7fdc180cf280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-26T07:18:54.849 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:54.841+0000 7fdc16ffd640 1 -- 172.21.15.143:0/3499308366 --> v2:172.21.15.143:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdc180711c0 con 0x7fdc18143ac0 2023-11-26T07:18:54.851 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:54.845+0000 7fdc16ffd640 1 --2- 172.21.15.143:0/3499308366 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fdc18143ac0 0x7fdc18070370 secure :-1 s=READY pgs=403 cs=0 l=1 rev1=1 crypto rx=0x7fdc1806dca0 tx=0x7fdc0001eb90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=28db244f72eddab0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:18:54.851 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:54.845+0000 7fdc167fc640 1 -- 172.21.15.143:0/3499308366 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdc000a7c20 con 0x7fdc18143ac0 2023-11-26T07:18:54.851 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:54.845+0000 7fdc167fc640 1 -- 172.21.15.143:0/3499308366 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdc000a7d80 con 0x7fdc18143ac0 2023-11-26T07:18:54.852 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:54.845+0000 7fdc167fc640 1 -- 172.21.15.143:0/3499308366 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdc000af660 con 0x7fdc18143ac0 2023-11-26T07:18:54.852 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:54.845+0000 7fdc1d557640 1 -- 172.21.15.143:0/3499308366 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fdc18143ac0 msgr2=0x7fdc18070370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:18:54.852 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:54.845+0000 7fdc1d557640 1 --2- 172.21.15.143:0/3499308366 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fdc18143ac0 0x7fdc18070370 secure :-1 s=READY pgs=403 cs=0 l=1 rev1=1 crypto rx=0x7fdc1806dca0 tx=0x7fdc0001eb90 comp rx=0 tx=0).stop 2023-11-26T07:18:54.852 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:54.845+0000 7fdc1d557640 1 -- 172.21.15.143:0/3499308366 shutdown_connections 2023-11-26T07:18:54.853 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:54.845+0000 7fdc1d557640 1 --2- 172.21.15.143:0/3499308366 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fdc180cf990 0x7fdc180cd1b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:18:54.853 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:54.845+0000 7fdc1d557640 1 --2- 172.21.15.143:0/3499308366 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fdc18143ac0 0x7fdc18070370 unknown :-1 s=CLOSED pgs=403 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:18:54.853 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:54.845+0000 7fdc1d557640 1 -- 172.21.15.143:0/3499308366 >> 172.21.15.143:0/3499308366 conn(0x7fdc1806a890 msgr2=0x7fdc1806aca0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:18:54.853 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:54.845+0000 7fdc1d557640 1 -- 172.21.15.143:0/3499308366 shutdown_connections 2023-11-26T07:18:54.853 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:54.845+0000 7fdc1d557640 1 -- 172.21.15.143:0/3499308366 wait complete. 2023-11-26T07:18:54.854 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:54.849+0000 7fdc1d557640 1 Processor -- start 2023-11-26T07:18:54.854 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:54.849+0000 7fdc1d557640 1 -- start start 2023-11-26T07:18:54.856 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:54.849+0000 7fdc1d557640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fdc180708b0 0x7fdc180b2000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:18:54.856 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:54.849+0000 7fdc1d557640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fdc180cf990 0x7fdc180b3160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:18:54.856 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:54.849+0000 7fdc1d557640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fdc18143ac0 0x7fdc18136f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:18:54.856 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:54.849+0000 7fdc1d557640 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdc1813bf30 con 0x7fdc180708b0 2023-11-26T07:18:54.856 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:54.849+0000 7fdc1d557640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdc180b36a0 con 0x7fdc180cf990 2023-11-26T07:18:54.856 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:54.849+0000 7fdc1d557640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdc180b3920 con 0x7fdc18143ac0 2023-11-26T07:18:54.856 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:54.849+0000 7fdc16ffd640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fdc180708b0 0x7fdc180b2000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:18:54.857 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:54.849+0000 7fdc16ffd640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fdc180708b0 0x7fdc180b2000 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:51666/0 (socket says 172.21.15.143:51666) 2023-11-26T07:18:54.857 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:54.849+0000 7fdc16ffd640 1 -- 172.21.15.143:0/36776579 learned_addr learned my addr 172.21.15.143:0/36776579 (peer_addr_for_me v2:172.21.15.143:0/0) 2023-11-26T07:18:54.857 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:54.849+0000 7fdc0e7fc640 1 --2- 172.21.15.143:0/36776579 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fdc180cf990 0x7fdc180b3160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:18:54.857 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:54.849+0000 7fdc177fe640 1 --2- 172.21.15.143:0/36776579 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fdc18143ac0 0x7fdc18136f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:18:54.857 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:54.849+0000 7fdc16ffd640 1 -- 172.21.15.143:0/36776579 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fdc18143ac0 msgr2=0x7fdc18136f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:18:54.857 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:54.849+0000 7fdc16ffd640 1 --2- 172.21.15.143:0/36776579 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fdc18143ac0 0x7fdc18136f20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:18:54.857 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:54.849+0000 7fdc16ffd640 1 -- 172.21.15.143:0/36776579 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fdc180cf990 msgr2=0x7fdc180b3160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:18:54.857 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:54.849+0000 7fdc16ffd640 1 --2- 172.21.15.143:0/36776579 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fdc180cf990 0x7fdc180b3160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:18:54.857 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:54.849+0000 7fdc16ffd640 1 -- 172.21.15.143:0/36776579 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdc0000b770 con 0x7fdc180708b0 2023-11-26T07:18:54.857 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:54.849+0000 7fdc177fe640 1 --2- 172.21.15.143:0/36776579 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fdc18143ac0 0x7fdc18136f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-26T07:18:54.857 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:54.849+0000 7fdc16ffd640 1 --2- 172.21.15.143:0/36776579 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fdc180708b0 0x7fdc180b2000 secure :-1 s=READY pgs=404 cs=0 l=1 rev1=1 crypto rx=0x7fdc0001f430 tx=0x7fdc0001b930 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:18:54.858 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:54.849+0000 7fdc0ffff640 1 -- 172.21.15.143:0/36776579 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdc000ba660 con 0x7fdc180708b0 2023-11-26T07:18:54.858 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:54.849+0000 7fdc0ffff640 1 -- 172.21.15.143:0/36776579 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdc000ba7c0 con 0x7fdc180708b0 2023-11-26T07:18:54.858 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:54.849+0000 7fdc1d557640 1 -- 172.21.15.143:0/36776579 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdc1813b8e0 con 0x7fdc180708b0 2023-11-26T07:18:54.859 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:54.849+0000 7fdc0ffff640 1 -- 172.21.15.143:0/36776579 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdc000bab70 con 0x7fdc180708b0 2023-11-26T07:18:54.859 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:54.853+0000 7fdc1d557640 1 -- 172.21.15.143:0/36776579 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdc18137460 con 0x7fdc180708b0 2023-11-26T07:18:54.859 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:54.853+0000 7fdc1d557640 1 -- 172.21.15.143:0/36776579 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fdc18137710 con 0x7fdc180708b0 2023-11-26T07:18:54.859 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:54.853+0000 7fdc0ffff640 1 -- 172.21.15.143:0/36776579 <== mon.0 v2:172.21.15.143:3300/0 4 ==== mgrmap(e 5) v1 ==== 85852+0+0 (secure 0 0 0) 0x7fdc000c33b0 con 0x7fdc180708b0 2023-11-26T07:18:54.859 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:54.853+0000 7fdc0ffff640 1 --2- 172.21.15.143:0/36776579 >> [v2:172.21.15.143:6832/2150557544,v1:172.21.15.143:6833/2150557544] conn(0x7fdbf80664a0 0x7fdbf8068960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:18:54.860 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:54.853+0000 7fdc0ffff640 1 -- 172.21.15.143:0/36776579 <== mon.0 v2:172.21.15.143:3300/0 5 ==== osd_map(33..33 src has 1..33) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fdc0011ed40 con 0x7fdc180708b0 2023-11-26T07:18:54.860 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:54.853+0000 7fdc0e7fc640 1 --2- 172.21.15.143:0/36776579 >> [v2:172.21.15.143:6832/2150557544,v1:172.21.15.143:6833/2150557544] conn(0x7fdbf80664a0 0x7fdbf8068960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:18:54.860 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:54.853+0000 7fdc0e7fc640 1 --2- 172.21.15.143:0/36776579 >> [v2:172.21.15.143:6832/2150557544,v1:172.21.15.143:6833/2150557544] conn(0x7fdbf80664a0 0x7fdbf8068960 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7fdc181f9390 tx=0x7fdc08007e40 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:18:54.863 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:54.857+0000 7fdc0ffff640 1 -- 172.21.15.143:0/36776579 <== mon.0 v2:172.21.15.143:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172649 (secure 0 0 0) 0x7fdc000ec2f0 con 0x7fdc180708b0 2023-11-26T07:18:55.043 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:55.037+0000 7fdc1d557640 1 -- 172.21.15.143:0/36776579 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"} v 0) v1 -- 0x7fdc1813b740 con 0x7fdc180708b0 2023-11-26T07:18:55.676 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:55.669+0000 7fdc0ffff640 1 -- 172.21.15.143:0/36776579 <== mon.0 v2:172.21.15.143:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"}]=0 v16) v1 ==== 122+0+0 (secure 0 0 0) 0x7fdc1813b740 con 0x7fdc180708b0 2023-11-26T07:18:55.679 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:55.673+0000 7fdc1d557640 1 -- 172.21.15.143:0/36776579 >> [v2:172.21.15.143:6832/2150557544,v1:172.21.15.143:6833/2150557544] conn(0x7fdbf80664a0 msgr2=0x7fdbf8068960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:18:55.679 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:55.673+0000 7fdc1d557640 1 --2- 172.21.15.143:0/36776579 >> [v2:172.21.15.143:6832/2150557544,v1:172.21.15.143:6833/2150557544] conn(0x7fdbf80664a0 0x7fdbf8068960 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7fdc181f9390 tx=0x7fdc08007e40 comp rx=0 tx=0).stop 2023-11-26T07:18:55.679 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:55.673+0000 7fdc1d557640 1 -- 172.21.15.143:0/36776579 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fdc180708b0 msgr2=0x7fdc180b2000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:18:55.679 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:55.673+0000 7fdc1d557640 1 --2- 172.21.15.143:0/36776579 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fdc180708b0 0x7fdc180b2000 secure :-1 s=READY pgs=404 cs=0 l=1 rev1=1 crypto rx=0x7fdc0001f430 tx=0x7fdc0001b930 comp rx=0 tx=0).stop 2023-11-26T07:18:55.679 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:55.673+0000 7fdc1d557640 1 -- 172.21.15.143:0/36776579 shutdown_connections 2023-11-26T07:18:55.679 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:55.673+0000 7fdc1d557640 1 --2- 172.21.15.143:0/36776579 >> [v2:172.21.15.143:6832/2150557544,v1:172.21.15.143:6833/2150557544] conn(0x7fdbf80664a0 0x7fdbf8068960 unknown :-1 s=CLOSED pgs=219 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:18:55.679 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:55.673+0000 7fdc1d557640 1 --2- 172.21.15.143:0/36776579 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fdc18143ac0 0x7fdc18136f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:18:55.679 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:55.673+0000 7fdc1d557640 1 --2- 172.21.15.143:0/36776579 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fdc180cf990 0x7fdc180b3160 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:18:55.679 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:55.673+0000 7fdc1d557640 1 --2- 172.21.15.143:0/36776579 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fdc180708b0 0x7fdc180b2000 unknown :-1 s=CLOSED pgs=404 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:18:55.679 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:55.673+0000 7fdc1d557640 1 -- 172.21.15.143:0/36776579 >> 172.21.15.143:0/36776579 conn(0x7fdc1806a890 msgr2=0x7fdc1815a590 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:18:55.680 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:55.673+0000 7fdc1d557640 1 -- 172.21.15.143:0/36776579 shutdown_connections 2023-11-26T07:18:55.680 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:55.673+0000 7fdc1d557640 1 -- 172.21.15.143:0/36776579 wait complete. 2023-11-26T07:18:55.701 DEBUG:tasks.cephfs.filesystem:Not Creating any subvols for filesystem 'cephfs' 2023-11-26T07:18:55.701 DEBUG:teuthology.orchestra.run.smithi143:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-11-26T07:18:55.841 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:55.836+0000 7f15db580640 1 Processor -- start 2023-11-26T07:18:55.841 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:55.836+0000 7f15db580640 1 -- start start 2023-11-26T07:18:55.843 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:55.836+0000 7f15db580640 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7f15d406f000 0x7f15d406d030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:18:55.843 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:55.836+0000 7f15db580640 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7f15d4099a70 0x7f15d406bcb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:18:55.844 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:55.836+0000 7f15db580640 1 -- --> v1:172.21.15.143:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f15d40cf260 con 0x7f15d406f3e0 2023-11-26T07:18:55.844 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:55.836+0000 7f15db580640 1 -- --> v2:172.21.15.143:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f15d406d570 con 0x7f15d4099a70 2023-11-26T07:18:55.844 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:55.836+0000 7f15db580640 1 -- --> v2:172.21.15.188:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f15d409a010 con 0x7f15d406f000 2023-11-26T07:18:55.844 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:55.836+0000 7f15d8b1b640 1 --1- >> v1:172.21.15.143:6789/0 conn(0x7f15d406f3e0 0x7f15d406b5a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.143:6789/0 says I am v1:172.21.15.143:35262/0 (socket says 172.21.15.143:35262) 2023-11-26T07:18:55.844 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:55.836+0000 7f15d8b1b640 1 -- 172.21.15.143:0/3026898380 learned_addr learned my addr 172.21.15.143:0/3026898380 (peer_addr_for_me v1:172.21.15.143:0/0) 2023-11-26T07:18:55.844 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:55.836+0000 7f15d9b1d640 1 --2- 172.21.15.143:0/3026898380 >> v2:172.21.15.143:3300/0 conn(0x7f15d4099a70 0x7f15d406bcb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:18:55.844 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:55.836+0000 7f15d931c640 1 --2- 172.21.15.143:0/3026898380 >> v2:172.21.15.188:3300/0 conn(0x7f15d406f000 0x7f15d406d030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:18:55.844 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:55.836+0000 7f15d9b1d640 1 -- 172.21.15.143:0/3026898380 >> v2:172.21.15.188:3300/0 conn(0x7f15d406f000 msgr2=0x7f15d406d030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-26T07:18:55.844 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:55.836+0000 7f15d9b1d640 1 --2- 172.21.15.143:0/3026898380 >> v2:172.21.15.188:3300/0 conn(0x7f15d406f000 0x7f15d406d030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:18:55.844 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:55.836+0000 7f15d9b1d640 1 -- 172.21.15.143:0/3026898380 >> v1:172.21.15.143:6789/0 conn(0x7f15d406f3e0 legacy=0x7f15d406b5a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:18:55.844 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:55.836+0000 7f15d9b1d640 1 -- 172.21.15.143:0/3026898380 --> v2:172.21.15.143:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f15d409a2f0 con 0x7f15d4099a70 2023-11-26T07:18:55.844 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:55.836+0000 7f15cbfff640 1 -- 172.21.15.143:0/3026898380 <== mon.0 v1:172.21.15.143:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3638925875 0 0) 0x7f15d40cf260 con 0x7f15d406f3e0 2023-11-26T07:18:55.846 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:55.840+0000 7f15d9b1d640 1 --2- 172.21.15.143:0/3026898380 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f15d4099a70 0x7f15d406bcb0 secure :-1 s=READY pgs=406 cs=0 l=1 rev1=1 crypto rx=0x7f15d4071b70 tx=0x7f15cc01f0f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3c5c8605cb5490f3 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:18:55.846 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:55.840+0000 7f15cbfff640 1 -- 172.21.15.143:0/3026898380 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f15cc0b0440 con 0x7f15d4099a70 2023-11-26T07:18:55.846 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:55.840+0000 7f15cbfff640 1 -- 172.21.15.143:0/3026898380 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f15cc0b05a0 con 0x7f15d4099a70 2023-11-26T07:18:55.846 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:55.840+0000 7f15cbfff640 1 -- 172.21.15.143:0/3026898380 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f15cc0b0930 con 0x7f15d4099a70 2023-11-26T07:18:55.847 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:55.840+0000 7f15db580640 1 -- 172.21.15.143:0/3026898380 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f15d4099a70 msgr2=0x7f15d406bcb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:18:55.847 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:55.840+0000 7f15db580640 1 --2- 172.21.15.143:0/3026898380 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f15d4099a70 0x7f15d406bcb0 secure :-1 s=READY pgs=406 cs=0 l=1 rev1=1 crypto rx=0x7f15d4071b70 tx=0x7f15cc01f0f0 comp rx=0 tx=0).stop 2023-11-26T07:18:55.847 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:55.840+0000 7f15db580640 1 -- 172.21.15.143:0/3026898380 shutdown_connections 2023-11-26T07:18:55.847 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:55.840+0000 7f15db580640 1 --2- 172.21.15.143:0/3026898380 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f15d4099a70 0x7f15d406bcb0 unknown :-1 s=CLOSED pgs=406 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:18:55.848 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:55.840+0000 7f15db580640 1 --2- 172.21.15.143:0/3026898380 >> v2:172.21.15.188:3300/0 conn(0x7f15d406f000 0x7f15d406d030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:18:55.848 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:55.840+0000 7f15db580640 1 -- 172.21.15.143:0/3026898380 >> 172.21.15.143:0/3026898380 conn(0x7f15d40b2d10 msgr2=0x7f15d40b3120 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:18:55.848 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:55.840+0000 7f15db580640 1 -- 172.21.15.143:0/3026898380 shutdown_connections 2023-11-26T07:18:55.848 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:55.840+0000 7f15db580640 1 -- 172.21.15.143:0/3026898380 wait complete. 2023-11-26T07:18:55.849 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:55.844+0000 7f15db580640 1 Processor -- start 2023-11-26T07:18:55.849 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:55.844+0000 7f15db580640 1 -- start start 2023-11-26T07:18:55.851 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:55.844+0000 7f15db580640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f15d406f000 0x7f15d409baa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:18:55.851 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:55.844+0000 7f15db580640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f15d406f3e0 0x7f15d409df30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:18:55.851 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:55.844+0000 7f15db580640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f15d4099a70 0x7f15d41f3190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:18:55.851 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:55.844+0000 7f15db580640 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f15d409eef0 con 0x7f15d406f000 2023-11-26T07:18:55.851 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:55.844+0000 7f15db580640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f15d409e470 con 0x7f15d4099a70 2023-11-26T07:18:55.851 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:55.844+0000 7f15db580640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f15d409e650 con 0x7f15d406f3e0 2023-11-26T07:18:55.851 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:55.844+0000 7f15d931c640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f15d406f000 0x7f15d409baa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:18:55.851 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:55.844+0000 7f15d931c640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f15d406f000 0x7f15d409baa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:51690/0 (socket says 172.21.15.143:51690) 2023-11-26T07:18:55.852 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:55.844+0000 7f15d931c640 1 -- 172.21.15.143:0/1102218654 learned_addr learned my addr 172.21.15.143:0/1102218654 (peer_addr_for_me v2:172.21.15.143:0/0) 2023-11-26T07:18:55.852 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:55.844+0000 7f15d8b1b640 1 --2- 172.21.15.143:0/1102218654 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f15d406f3e0 0x7f15d409df30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:18:55.852 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:55.844+0000 7f15d9b1d640 1 --2- 172.21.15.143:0/1102218654 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f15d4099a70 0x7f15d41f3190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:18:55.852 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:55.844+0000 7f15d931c640 1 -- 172.21.15.143:0/1102218654 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f15d406f3e0 msgr2=0x7f15d409df30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:18:55.852 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:55.844+0000 7f15d931c640 1 --2- 172.21.15.143:0/1102218654 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f15d406f3e0 0x7f15d409df30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:18:55.852 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:55.844+0000 7f15d931c640 1 -- 172.21.15.143:0/1102218654 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f15d4099a70 msgr2=0x7f15d41f3190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:18:55.852 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:55.844+0000 7f15d931c640 1 --2- 172.21.15.143:0/1102218654 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f15d4099a70 0x7f15d41f3190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:18:55.852 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:55.844+0000 7f15d931c640 1 -- 172.21.15.143:0/1102218654 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f15cc009a70 con 0x7f15d406f000 2023-11-26T07:18:55.852 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:55.844+0000 7f15d9b1d640 1 --2- 172.21.15.143:0/1102218654 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f15d4099a70 0x7f15d41f3190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-26T07:18:55.852 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:55.844+0000 7f15d931c640 1 --2- 172.21.15.143:0/1102218654 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f15d406f000 0x7f15d409baa0 secure :-1 s=READY pgs=407 cs=0 l=1 rev1=1 crypto rx=0x7f15d000d800 tx=0x7f15d000dcc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:18:55.852 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:55.844+0000 7f15c9ffb640 1 -- 172.21.15.143:0/1102218654 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f15d000b6c0 con 0x7f15d406f000 2023-11-26T07:18:55.852 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:55.844+0000 7f15c9ffb640 1 -- 172.21.15.143:0/1102218654 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f15d000b820 con 0x7f15d406f000 2023-11-26T07:18:55.852 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:55.844+0000 7f15c9ffb640 1 -- 172.21.15.143:0/1102218654 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f15d000bb90 con 0x7f15d406f000 2023-11-26T07:18:55.853 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:55.844+0000 7f15db580640 1 -- 172.21.15.143:0/1102218654 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f15d409e930 con 0x7f15d406f000 2023-11-26T07:18:55.853 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:55.844+0000 7f15db580640 1 -- 172.21.15.143:0/1102218654 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f15d415f370 con 0x7f15d406f000 2023-11-26T07:18:55.853 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:55.848+0000 7f15db580640 1 -- 172.21.15.143:0/1102218654 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f15d415f530 con 0x7f15d406f000 2023-11-26T07:18:55.853 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:55.848+0000 7f15c9ffb640 1 -- 172.21.15.143:0/1102218654 <== mon.0 v2:172.21.15.143:3300/0 4 ==== mgrmap(e 5) v1 ==== 85852+0+0 (secure 0 0 0) 0x7f15d000bcf0 con 0x7f15d406f000 2023-11-26T07:18:55.854 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:55.848+0000 7f15c9ffb640 1 --2- 172.21.15.143:0/1102218654 >> [v2:172.21.15.143:6832/2150557544,v1:172.21.15.143:6833/2150557544] conn(0x7f15b8066470 0x7f15b8068930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:18:55.854 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:55.848+0000 7f15c9ffb640 1 -- 172.21.15.143:0/1102218654 <== mon.0 v2:172.21.15.143:3300/0 5 ==== osd_map(33..33 src has 1..33) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f15d00843a0 con 0x7f15d406f000 2023-11-26T07:18:55.854 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:55.848+0000 7f15d8b1b640 1 --2- 172.21.15.143:0/1102218654 >> [v2:172.21.15.143:6832/2150557544,v1:172.21.15.143:6833/2150557544] conn(0x7f15b8066470 0x7f15b8068930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:18:55.855 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:55.848+0000 7f15d8b1b640 1 --2- 172.21.15.143:0/1102218654 >> [v2:172.21.15.143:6832/2150557544,v1:172.21.15.143:6833/2150557544] conn(0x7f15b8066470 0x7f15b8068930 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7f15c4000f70 tx=0x7f15c4007b80 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:18:55.858 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:55.852+0000 7f15c9ffb640 1 -- 172.21.15.143:0/1102218654 <== mon.0 v2:172.21.15.143:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172649 (secure 0 0 0) 0x7f15d0051820 con 0x7f15d406f000 2023-11-26T07:18:56.039 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:56.032+0000 7f15db580640 1 -- 172.21.15.143:0/1102218654 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f15d415f1d0 con 0x7f15d406f000 2023-11-26T07:18:56.039 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:56.032+0000 7f15c9ffb640 1 -- 172.21.15.143:0/1102218654 <== mon.0 v2:172.21.15.143:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 16 v16) v1 ==== 94+0+4925 (secure 0 0 0) 0x7f15d415f1d0 con 0x7f15d406f000 2023-11-26T07:18:56.039 INFO:teuthology.orchestra.run.smithi143.stdout: 2023-11-26T07:18:56.040 INFO:teuthology.orchestra.run.smithi143.stdout:{"epoch":16,"default_fscid":2,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4847,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.188:6834/1547410397","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6832","nonce":1547410397},{"type":"v1","addr":"172.21.15.188:6834","nonce":1547410397}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":12},{"gid":4850,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.188:6835/3247640329","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6833","nonce":3247640329},{"type":"v1","addr":"172.21.15.188:6835","nonce":3247640329}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":12},{"gid":5217,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.143:6835/2123707650","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6834","nonce":2123707650},{"type":"v1","addr":"172.21.15.143:6835","nonce":2123707650}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":11}],"filesystems":[{"mdsmap":{"epoch":16,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-11-26T07:18:52.014584+0000","modified":"2023-11-26T07:18:55.043646+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":5220},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5220":{"gid":5220,"name":"c","rank":0,"incarnation":13,"state":"up:active","state_seq":3,"addr":"172.21.15.143:6837/2658667363","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6836","nonce":2658667363},{"type":"v1","addr":"172.21.15.143:6837","nonce":2658667363}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[5],"metadata_pool":4,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":2}]} 2023-11-26T07:18:56.040 INFO:teuthology.orchestra.run.smithi143.stderr:dumped fsmap epoch 16 2023-11-26T07:18:56.041 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:56.036+0000 7f15db580640 1 -- 172.21.15.143:0/1102218654 >> [v2:172.21.15.143:6832/2150557544,v1:172.21.15.143:6833/2150557544] conn(0x7f15b8066470 msgr2=0x7f15b8068930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:18:56.042 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:56.036+0000 7f15db580640 1 --2- 172.21.15.143:0/1102218654 >> [v2:172.21.15.143:6832/2150557544,v1:172.21.15.143:6833/2150557544] conn(0x7f15b8066470 0x7f15b8068930 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7f15c4000f70 tx=0x7f15c4007b80 comp rx=0 tx=0).stop 2023-11-26T07:18:56.042 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:56.036+0000 7f15db580640 1 -- 172.21.15.143:0/1102218654 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f15d406f000 msgr2=0x7f15d409baa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:18:56.042 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:56.036+0000 7f15db580640 1 --2- 172.21.15.143:0/1102218654 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f15d406f000 0x7f15d409baa0 secure :-1 s=READY pgs=407 cs=0 l=1 rev1=1 crypto rx=0x7f15d000d800 tx=0x7f15d000dcc0 comp rx=0 tx=0).stop 2023-11-26T07:18:56.042 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:56.036+0000 7f15db580640 1 -- 172.21.15.143:0/1102218654 shutdown_connections 2023-11-26T07:18:56.042 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:56.036+0000 7f15db580640 1 --2- 172.21.15.143:0/1102218654 >> [v2:172.21.15.143:6832/2150557544,v1:172.21.15.143:6833/2150557544] conn(0x7f15b8066470 0x7f15b8068930 unknown :-1 s=CLOSED pgs=220 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:18:56.042 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:56.036+0000 7f15db580640 1 --2- 172.21.15.143:0/1102218654 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f15d4099a70 0x7f15d41f3190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:18:56.042 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:56.036+0000 7f15db580640 1 --2- 172.21.15.143:0/1102218654 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f15d406f3e0 0x7f15d409df30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:18:56.042 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:56.036+0000 7f15db580640 1 --2- 172.21.15.143:0/1102218654 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f15d406f000 0x7f15d409baa0 unknown :-1 s=CLOSED pgs=407 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:18:56.042 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:56.036+0000 7f15db580640 1 -- 172.21.15.143:0/1102218654 >> 172.21.15.143:0/1102218654 conn(0x7f15d40b2d10 msgr2=0x7f15d406c580 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:18:56.042 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:56.036+0000 7f15db580640 1 -- 172.21.15.143:0/1102218654 shutdown_connections 2023-11-26T07:18:56.042 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:56.036+0000 7f15db580640 1 -- 172.21.15.143:0/1102218654 wait complete. 2023-11-26T07:18:56.061 DEBUG:teuthology.orchestra.run.smithi143:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-11-26T07:18:56.198 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:56.192+0000 7f38bd7f5640 1 Processor -- start 2023-11-26T07:18:56.198 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:56.192+0000 7f38bd7f5640 1 -- start start 2023-11-26T07:18:56.199 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:56.192+0000 7f38bd7f5640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f38b815d6e0 0x7f38b80d0df0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:18:56.199 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:56.192+0000 7f38bd7f5640 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7f38b80b2160 0x7f38b80d14b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:18:56.199 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:56.192+0000 7f38bd7f5640 1 -- --> v1:172.21.15.143:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f38b80cf0e0 con 0x7f38b80d19f0 2023-11-26T07:18:56.199 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:56.192+0000 7f38bd7f5640 1 -- --> v2:172.21.15.188:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f38b815dac0 con 0x7f38b80b2160 2023-11-26T07:18:56.199 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:56.192+0000 7f38bd7f5640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f38b80d1e80 con 0x7f38b815d6e0 2023-11-26T07:18:56.200 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:56.192+0000 7f38b6ffd640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f38b815d6e0 0x7f38b80d0df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:18:56.200 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:56.192+0000 7f38b77fe640 1 --1- >> v1:172.21.15.143:6789/0 conn(0x7f38b80d19f0 0x7f38b80cfbf0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.143:6789/0 says I am v1:172.21.15.143:35276/0 (socket says 172.21.15.143:35276) 2023-11-26T07:18:56.201 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:56.192+0000 7f38b6ffd640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f38b815d6e0 0x7f38b80d0df0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.188:3301/0 says I am v2:172.21.15.143:40646/0 (socket says 172.21.15.143:40646) 2023-11-26T07:18:56.201 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:56.192+0000 7f38b77fe640 1 -- 172.21.15.143:0/309919707 learned_addr learned my addr 172.21.15.143:0/309919707 (peer_addr_for_me v1:172.21.15.143:0/0) 2023-11-26T07:18:56.201 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:56.192+0000 7f38ae7fc640 1 --2- 172.21.15.143:0/309919707 >> v2:172.21.15.188:3300/0 conn(0x7f38b80b2160 0x7f38b80d14b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:18:56.201 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:56.192+0000 7f38b67fc640 1 -- 172.21.15.143:0/309919707 <== mon.0 v1:172.21.15.143:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3811253634 0 0) 0x7f38b80cf0e0 con 0x7f38b80d19f0 2023-11-26T07:18:56.201 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:56.192+0000 7f38b6ffd640 1 -- 172.21.15.143:0/309919707 >> v2:172.21.15.188:3300/0 conn(0x7f38b80b2160 msgr2=0x7f38b80d14b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-26T07:18:56.201 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:56.192+0000 7f38b6ffd640 1 --2- 172.21.15.143:0/309919707 >> v2:172.21.15.188:3300/0 conn(0x7f38b80b2160 0x7f38b80d14b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:18:56.201 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:56.192+0000 7f38b6ffd640 1 -- 172.21.15.143:0/309919707 >> v1:172.21.15.143:6789/0 conn(0x7f38b80d19f0 legacy=0x7f38b80cfbf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:18:56.201 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:56.192+0000 7f38b6ffd640 1 -- 172.21.15.143:0/309919707 --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f38b80ceb20 con 0x7f38b815d6e0 2023-11-26T07:18:56.203 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:56.196+0000 7f38b6ffd640 1 --2- 172.21.15.143:0/309919707 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f38b815d6e0 0x7f38b80d0df0 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7f38a80097c0 tx=0x7f38a801c480 comp rx=0 tx=0).ready entity=mon.2 client_cookie=1d253412766974f server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:18:56.203 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:56.196+0000 7f38b67fc640 1 -- 172.21.15.143:0/309919707 <== mon.2 v2:172.21.15.188:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f38a80ab600 con 0x7f38b815d6e0 2023-11-26T07:18:56.203 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:56.196+0000 7f38b67fc640 1 -- 172.21.15.143:0/309919707 <== mon.2 v2:172.21.15.188:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f38a80a7050 con 0x7f38b815d6e0 2023-11-26T07:18:56.203 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:56.196+0000 7f38b67fc640 1 -- 172.21.15.143:0/309919707 <== mon.2 v2:172.21.15.188:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f38a80ab980 con 0x7f38b815d6e0 2023-11-26T07:18:56.204 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:56.196+0000 7f38bd7f5640 1 -- 172.21.15.143:0/309919707 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f38b815d6e0 msgr2=0x7f38b80d0df0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:18:56.204 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:56.196+0000 7f38bd7f5640 1 --2- 172.21.15.143:0/309919707 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f38b815d6e0 0x7f38b80d0df0 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7f38a80097c0 tx=0x7f38a801c480 comp rx=0 tx=0).stop 2023-11-26T07:18:56.204 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:56.196+0000 7f38bd7f5640 1 -- 172.21.15.143:0/309919707 shutdown_connections 2023-11-26T07:18:56.204 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:56.196+0000 7f38bd7f5640 1 --2- 172.21.15.143:0/309919707 >> v2:172.21.15.188:3300/0 conn(0x7f38b80b2160 0x7f38b80d14b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:18:56.204 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:56.196+0000 7f38bd7f5640 1 --2- 172.21.15.143:0/309919707 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f38b815d6e0 0x7f38b80d0df0 unknown :-1 s=CLOSED pgs=76 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:18:56.204 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:56.196+0000 7f38bd7f5640 1 -- 172.21.15.143:0/309919707 >> 172.21.15.143:0/309919707 conn(0x7f38b809c550 msgr2=0x7f38b80d3220 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:18:56.205 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:56.196+0000 7f38bd7f5640 1 -- 172.21.15.143:0/309919707 shutdown_connections 2023-11-26T07:18:56.205 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:56.196+0000 7f38bd7f5640 1 -- 172.21.15.143:0/309919707 wait complete. 2023-11-26T07:18:56.205 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:56.200+0000 7f38bd7f5640 1 Processor -- start 2023-11-26T07:18:56.206 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:56.200+0000 7f38bd7f5640 1 -- start start 2023-11-26T07:18:56.207 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:56.200+0000 7f38bd7f5640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f38b80b2160 0x7f38b815dc00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:18:56.208 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:56.200+0000 7f38bd7f5640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f38b80d19f0 0x7f38b814ab00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:18:56.208 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:56.200+0000 7f38bd7f5640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f38b815d6e0 0x7f38b81576a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:18:56.208 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:56.200+0000 7f38bd7f5640 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f38b81592e0 con 0x7f38b80b2160 2023-11-26T07:18:56.208 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:56.200+0000 7f38bd7f5640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f38b8158a40 con 0x7f38b80d19f0 2023-11-26T07:18:56.208 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:56.200+0000 7f38bd7f5640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f38b8158d20 con 0x7f38b815d6e0 2023-11-26T07:18:56.208 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:56.200+0000 7f38b6ffd640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f38b80b2160 0x7f38b815dc00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:18:56.208 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:56.200+0000 7f38b6ffd640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f38b80b2160 0x7f38b815dc00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:51702/0 (socket says 172.21.15.143:51702) 2023-11-26T07:18:56.208 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:56.200+0000 7f38b6ffd640 1 -- 172.21.15.143:0/1978638822 learned_addr learned my addr 172.21.15.143:0/1978638822 (peer_addr_for_me v2:172.21.15.143:0/0) 2023-11-26T07:18:56.208 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:56.200+0000 7f38ae7fc640 1 --2- 172.21.15.143:0/1978638822 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f38b80d19f0 0x7f38b814ab00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:18:56.208 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:56.200+0000 7f38b77fe640 1 --2- 172.21.15.143:0/1978638822 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f38b815d6e0 0x7f38b81576a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:18:56.208 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:56.200+0000 7f38b6ffd640 1 -- 172.21.15.143:0/1978638822 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f38b815d6e0 msgr2=0x7f38b81576a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:18:56.208 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:56.200+0000 7f38b6ffd640 1 --2- 172.21.15.143:0/1978638822 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f38b815d6e0 0x7f38b81576a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:18:56.208 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:56.200+0000 7f38b6ffd640 1 -- 172.21.15.143:0/1978638822 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f38b80d19f0 msgr2=0x7f38b814ab00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:18:56.209 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:56.200+0000 7f38b6ffd640 1 --2- 172.21.15.143:0/1978638822 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f38b80d19f0 0x7f38b814ab00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:18:56.209 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:56.200+0000 7f38b6ffd640 1 -- 172.21.15.143:0/1978638822 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f38a8009420 con 0x7f38b80b2160 2023-11-26T07:18:56.209 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:56.200+0000 7f38b77fe640 1 --2- 172.21.15.143:0/1978638822 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f38b815d6e0 0x7f38b81576a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-26T07:18:56.209 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:56.200+0000 7f38b6ffd640 1 --2- 172.21.15.143:0/1978638822 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f38b80b2160 0x7f38b815dc00 secure :-1 s=READY pgs=409 cs=0 l=1 rev1=1 crypto rx=0x7f38a801a0b0 tx=0x7f38a8019220 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:18:56.210 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:56.200+0000 7f38affff640 1 -- 172.21.15.143:0/1978638822 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f38a80b4400 con 0x7f38b80b2160 2023-11-26T07:18:56.210 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:56.200+0000 7f38affff640 1 -- 172.21.15.143:0/1978638822 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f38a80b4580 con 0x7f38b80b2160 2023-11-26T07:18:56.210 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:56.200+0000 7f38affff640 1 -- 172.21.15.143:0/1978638822 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f38a80b4930 con 0x7f38b80b2160 2023-11-26T07:18:56.210 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:56.200+0000 7f38bd7f5640 1 -- 172.21.15.143:0/1978638822 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f38b8159000 con 0x7f38b80b2160 2023-11-26T07:18:56.210 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:56.200+0000 7f38bd7f5640 1 -- 172.21.15.143:0/1978638822 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f38b814b040 con 0x7f38b80b2160 2023-11-26T07:18:56.210 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:56.204+0000 7f38bd7f5640 1 -- 172.21.15.143:0/1978638822 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f38b8067680 con 0x7f38b80b2160 2023-11-26T07:18:56.210 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:56.204+0000 7f38affff640 1 -- 172.21.15.143:0/1978638822 <== mon.0 v2:172.21.15.143:3300/0 4 ==== mgrmap(e 5) v1 ==== 85852+0+0 (secure 0 0 0) 0x7f38a80b2040 con 0x7f38b80b2160 2023-11-26T07:18:56.211 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:56.204+0000 7f38affff640 1 --2- 172.21.15.143:0/1978638822 >> [v2:172.21.15.143:6832/2150557544,v1:172.21.15.143:6833/2150557544] conn(0x7f389c066470 0x7f389c068930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:18:56.211 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:56.204+0000 7f38affff640 1 -- 172.21.15.143:0/1978638822 <== mon.0 v2:172.21.15.143:3300/0 5 ==== osd_map(33..33 src has 1..33) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f38a811de30 con 0x7f38b80b2160 2023-11-26T07:18:56.211 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:56.204+0000 7f38ae7fc640 1 --2- 172.21.15.143:0/1978638822 >> [v2:172.21.15.143:6832/2150557544,v1:172.21.15.143:6833/2150557544] conn(0x7f389c066470 0x7f389c068930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:18:56.212 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:56.204+0000 7f38ae7fc640 1 --2- 172.21.15.143:0/1978638822 >> [v2:172.21.15.143:6832/2150557544,v1:172.21.15.143:6833/2150557544] conn(0x7f389c066470 0x7f389c068930 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7f38b81537a0 tx=0x7f38a0009190 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:18:56.215 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:56.208+0000 7f38affff640 1 -- 172.21.15.143:0/1978638822 <== mon.0 v2:172.21.15.143:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172649 (secure 0 0 0) 0x7f38a80eb2a0 con 0x7f38b80b2160 2023-11-26T07:18:56.364 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:56.356+0000 7f38bd7f5640 1 -- 172.21.15.143:0/1978638822 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f38b8159b50 con 0x7f38b80b2160 2023-11-26T07:18:56.365 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:56.360+0000 7f38affff640 1 -- 172.21.15.143:0/1978638822 <== mon.0 v2:172.21.15.143:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v33) v1 ==== 74+0+16605 (secure 0 0 0) 0x7f38a80b8090 con 0x7f38b80b2160 2023-11-26T07:18:56.365 INFO:teuthology.orchestra.run.smithi143.stdout: 2023-11-26T07:18:56.365 INFO:teuthology.orchestra.run.smithi143.stdout:{"epoch":33,"fsid":"f5033060-2469-4a07-95fe-2b76f5404bc1","created":"2023-11-26T07:17:28.613663+0000","modified":"2023-11-26T07:18:53.659425+0000","last_up_change":"2023-11-26T07:17:41.157813+0000","last_in_change":"2023-11-26T07:17:31.181658+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":5,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-11-26T07:17:41.758597+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_metadata","create_time":"2023-11-26T07:18:50.535934+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"32","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":5,"pool_name":"cephfs_data","create_time":"2023-11-26T07:18:51.008494+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"33","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"ebbee2c6-023e-44b9-9f1d-6f7bfd309730","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6817","nonce":639807369},{"type":"v1","addr":"172.21.15.143:6819","nonce":639807369}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6821","nonce":639807369},{"type":"v1","addr":"172.21.15.143:6823","nonce":639807369}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6830","nonce":639807369},{"type":"v1","addr":"172.21.15.143:6831","nonce":639807369}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6825","nonce":639807369},{"type":"v1","addr":"172.21.15.143:6828","nonce":639807369}]},"public_addr":"172.21.15.143:6819/639807369","cluster_addr":"172.21.15.143:6823/639807369","heartbeat_back_addr":"172.21.15.143:6831/639807369","heartbeat_front_addr":"172.21.15.143:6828/639807369","state":["exists","up"]},{"osd":1,"uuid":"fec4b8be-7122-49e9-9a19-5850a0700001","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6816","nonce":1144942530},{"type":"v1","addr":"172.21.15.143:6818","nonce":1144942530}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6820","nonce":1144942530},{"type":"v1","addr":"172.21.15.143:6822","nonce":1144942530}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6827","nonce":1144942530},{"type":"v1","addr":"172.21.15.143:6829","nonce":1144942530}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6824","nonce":1144942530},{"type":"v1","addr":"172.21.15.143:6826","nonce":1144942530}]},"public_addr":"172.21.15.143:6818/1144942530","cluster_addr":"172.21.15.143:6822/1144942530","heartbeat_back_addr":"172.21.15.143:6829/1144942530","heartbeat_front_addr":"172.21.15.143:6826/1144942530","state":["exists","up"]},{"osd":2,"uuid":"2888af5b-f933-46ca-8f95-6dd494ff43f6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6808","nonce":3909095186},{"type":"v1","addr":"172.21.15.143:6809","nonce":3909095186}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6810","nonce":3909095186},{"type":"v1","addr":"172.21.15.143:6811","nonce":3909095186}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6814","nonce":3909095186},{"type":"v1","addr":"172.21.15.143:6815","nonce":3909095186}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6812","nonce":3909095186},{"type":"v1","addr":"172.21.15.143:6813","nonce":3909095186}]},"public_addr":"172.21.15.143:6809/3909095186","cluster_addr":"172.21.15.143:6811/3909095186","heartbeat_back_addr":"172.21.15.143:6815/3909095186","heartbeat_front_addr":"172.21.15.143:6813/3909095186","state":["exists","up"]},{"osd":3,"uuid":"d1f52435-7bc1-4081-9550-9cd1ba55f549","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6800","nonce":3854210991},{"type":"v1","addr":"172.21.15.143:6801","nonce":3854210991}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6802","nonce":3854210991},{"type":"v1","addr":"172.21.15.143:6803","nonce":3854210991}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6806","nonce":3854210991},{"type":"v1","addr":"172.21.15.143:6807","nonce":3854210991}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6804","nonce":3854210991},{"type":"v1","addr":"172.21.15.143:6805","nonce":3854210991}]},"public_addr":"172.21.15.143:6801/3854210991","cluster_addr":"172.21.15.143:6803/3854210991","heartbeat_back_addr":"172.21.15.143:6807/3854210991","heartbeat_front_addr":"172.21.15.143:6805/3854210991","state":["exists","up"]},{"osd":4,"uuid":"e2ae8b6e-6e25-49b2-bcbf-aacce68f06a3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6800","nonce":3924606902},{"type":"v1","addr":"172.21.15.188:6801","nonce":3924606902}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6802","nonce":3924606902},{"type":"v1","addr":"172.21.15.188:6803","nonce":3924606902}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6806","nonce":3924606902},{"type":"v1","addr":"172.21.15.188:6807","nonce":3924606902}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6804","nonce":3924606902},{"type":"v1","addr":"172.21.15.188:6805","nonce":3924606902}]},"public_addr":"172.21.15.188:6801/3924606902","cluster_addr":"172.21.15.188:6803/3924606902","heartbeat_back_addr":"172.21.15.188:6807/3924606902","heartbeat_front_addr":"172.21.15.188:6805/3924606902","state":["exists","up"]},{"osd":5,"uuid":"3cec8a32-7698-4c36-8e89-e4ba11934f33","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6813","nonce":186841544},{"type":"v1","addr":"172.21.15.188:6815","nonce":186841544}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6817","nonce":186841544},{"type":"v1","addr":"172.21.15.188:6818","nonce":186841544}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6822","nonce":186841544},{"type":"v1","addr":"172.21.15.188:6823","nonce":186841544}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6820","nonce":186841544},{"type":"v1","addr":"172.21.15.188:6821","nonce":186841544}]},"public_addr":"172.21.15.188:6815/186841544","cluster_addr":"172.21.15.188:6818/186841544","heartbeat_back_addr":"172.21.15.188:6823/186841544","heartbeat_front_addr":"172.21.15.188:6821/186841544","state":["exists","up"]},{"osd":6,"uuid":"58a65d5d-e090-473a-81b4-ad38e85029f9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6808","nonce":2153499405},{"type":"v1","addr":"172.21.15.188:6809","nonce":2153499405}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6810","nonce":2153499405},{"type":"v1","addr":"172.21.15.188:6811","nonce":2153499405}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6816","nonce":2153499405},{"type":"v1","addr":"172.21.15.188:6819","nonce":2153499405}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6812","nonce":2153499405},{"type":"v1","addr":"172.21.15.188:6814","nonce":2153499405}]},"public_addr":"172.21.15.188:6809/2153499405","cluster_addr":"172.21.15.188:6811/2153499405","heartbeat_back_addr":"172.21.15.188:6819/2153499405","heartbeat_front_addr":"172.21.15.188:6814/2153499405","state":["exists","up"]},{"osd":7,"uuid":"ea5b7f70-3554-4cf8-ac30-f7c59b80678d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6824","nonce":2722096729},{"type":"v1","addr":"172.21.15.188:6825","nonce":2722096729}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6826","nonce":2722096729},{"type":"v1","addr":"172.21.15.188:6827","nonce":2722096729}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6830","nonce":2722096729},{"type":"v1","addr":"172.21.15.188:6831","nonce":2722096729}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6828","nonce":2722096729},{"type":"v1","addr":"172.21.15.188:6829","nonce":2722096729}]},"public_addr":"172.21.15.188:6825/2722096729","cluster_addr":"172.21.15.188:6827/2722096729","heartbeat_back_addr":"172.21.15.188:6831/2722096729","heartbeat_front_addr":"172.21.15.188:6829/2722096729","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-26T07:17:36.995032+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-26T07:17:36.932266+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-26T07:17:36.829304+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-26T07:17:36.804297+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-26T07:17:36.667892+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-26T07:17:36.964868+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-26T07:17:36.827332+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-26T07:17:36.893179+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-11-26T07:18:56.367 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:56.360+0000 7f38bd7f5640 1 -- 172.21.15.143:0/1978638822 >> [v2:172.21.15.143:6832/2150557544,v1:172.21.15.143:6833/2150557544] conn(0x7f389c066470 msgr2=0x7f389c068930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:18:56.367 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:56.360+0000 7f38bd7f5640 1 --2- 172.21.15.143:0/1978638822 >> [v2:172.21.15.143:6832/2150557544,v1:172.21.15.143:6833/2150557544] conn(0x7f389c066470 0x7f389c068930 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7f38b81537a0 tx=0x7f38a0009190 comp rx=0 tx=0).stop 2023-11-26T07:18:56.367 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:56.360+0000 7f38bd7f5640 1 -- 172.21.15.143:0/1978638822 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f38b80b2160 msgr2=0x7f38b815dc00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:18:56.367 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:56.360+0000 7f38bd7f5640 1 --2- 172.21.15.143:0/1978638822 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f38b80b2160 0x7f38b815dc00 secure :-1 s=READY pgs=409 cs=0 l=1 rev1=1 crypto rx=0x7f38a801a0b0 tx=0x7f38a8019220 comp rx=0 tx=0).stop 2023-11-26T07:18:56.367 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:56.360+0000 7f38bd7f5640 1 -- 172.21.15.143:0/1978638822 shutdown_connections 2023-11-26T07:18:56.367 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:56.360+0000 7f38bd7f5640 1 --2- 172.21.15.143:0/1978638822 >> [v2:172.21.15.143:6832/2150557544,v1:172.21.15.143:6833/2150557544] conn(0x7f389c066470 0x7f389c068930 unknown :-1 s=CLOSED pgs=221 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:18:56.367 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:56.360+0000 7f38bd7f5640 1 --2- 172.21.15.143:0/1978638822 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f38b815d6e0 0x7f38b81576a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:18:56.368 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:56.360+0000 7f38bd7f5640 1 --2- 172.21.15.143:0/1978638822 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f38b80d19f0 0x7f38b814ab00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:18:56.368 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:56.360+0000 7f38bd7f5640 1 --2- 172.21.15.143:0/1978638822 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f38b80b2160 0x7f38b815dc00 unknown :-1 s=CLOSED pgs=409 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:18:56.368 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:56.360+0000 7f38bd7f5640 1 -- 172.21.15.143:0/1978638822 >> 172.21.15.143:0/1978638822 conn(0x7f38b809c550 msgr2=0x7f38b80d3220 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:18:56.368 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:56.360+0000 7f38bd7f5640 1 -- 172.21.15.143:0/1978638822 shutdown_connections 2023-11-26T07:18:56.368 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:56.360+0000 7f38bd7f5640 1 -- 172.21.15.143:0/1978638822 wait complete. 2023-11-26T07:18:56.386 INFO:tasks.cephfs.filesystem.ceph_manager:waiting for clean 2023-11-26T07:18:56.386 DEBUG:teuthology.orchestra.run.smithi143:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-11-26T07:18:56.524 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:56.516+0000 7fe97238d640 1 Processor -- start 2023-11-26T07:18:56.524 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:56.516+0000 7fe97238d640 1 -- start start 2023-11-26T07:18:56.525 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:56.520+0000 7fe97238d640 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7fe96c06bae0 0x7fe96c15e6f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:18:56.526 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:56.520+0000 7fe97238d640 1 -- --> v1:172.21.15.143:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe96c0cde60 con 0x7fe96c0cd610 2023-11-26T07:18:56.526 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:56.520+0000 7fe97238d640 1 -- --> v1:172.21.15.188:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe96c15dc10 con 0x7fe96c070ba0 2023-11-26T07:18:56.526 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:56.520+0000 7fe97238d640 1 -- --> v2:172.21.15.188:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe96c15def0 con 0x7fe96c06bae0 2023-11-26T07:18:56.526 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:56.520+0000 7fe97092a640 1 --1- >> v1:172.21.15.143:6789/0 conn(0x7fe96c0cd610 0x7fe96c1610f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.143:6789/0 says I am v1:172.21.15.143:35288/0 (socket says 172.21.15.143:35288) 2023-11-26T07:18:56.526 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:56.520+0000 7fe97092a640 1 -- 172.21.15.143:0/51587699 learned_addr learned my addr 172.21.15.143:0/51587699 (peer_addr_for_me v1:172.21.15.143:0/0) 2023-11-26T07:18:56.527 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:56.520+0000 7fe96b7fe640 1 --2- 172.21.15.143:0/51587699 >> v2:172.21.15.188:3300/0 conn(0x7fe96c06bae0 0x7fe96c15e6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:18:56.527 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:56.520+0000 7fe96affd640 1 -- 172.21.15.143:0/51587699 <== mon.0 v1:172.21.15.143:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1008767235 0 0) 0x7fe96c0cde60 con 0x7fe96c0cd610 2023-11-26T07:18:56.527 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:56.520+0000 7fe96affd640 1 -- 172.21.15.143:0/51587699 --> v1:172.21.15.143:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fe954003530 con 0x7fe96c0cd610 2023-11-26T07:18:56.527 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:56.520+0000 7fe96affd640 1 -- 172.21.15.143:0/51587699 <== mon.1 v1:172.21.15.188:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 959671353 0 0) 0x7fe96c15dc10 con 0x7fe96c070ba0 2023-11-26T07:18:56.527 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:56.520+0000 7fe96affd640 1 -- 172.21.15.143:0/51587699 --> v1:172.21.15.188:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fe96c0cde60 con 0x7fe96c070ba0 2023-11-26T07:18:56.527 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:56.520+0000 7fe96b7fe640 1 -- 172.21.15.143:0/51587699 >> v1:172.21.15.188:6789/0 conn(0x7fe96c070ba0 legacy=0x7fe96c070f80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:18:56.527 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:56.520+0000 7fe96affd640 1 -- 172.21.15.143:0/51587699 <== mon.0 v1:172.21.15.143:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 872712598 0 0) 0x7fe954003530 con 0x7fe96c0cd610 2023-11-26T07:18:56.527 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:56.520+0000 7fe96b7fe640 1 -- 172.21.15.143:0/51587699 >> v1:172.21.15.143:6789/0 conn(0x7fe96c0cd610 legacy=0x7fe96c1610f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:18:56.527 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:56.520+0000 7fe96b7fe640 1 -- 172.21.15.143:0/51587699 --> v2:172.21.15.188:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe96c15e1d0 con 0x7fe96c06bae0 2023-11-26T07:18:56.529 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:56.524+0000 7fe96b7fe640 1 --2- 172.21.15.143:0/51587699 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fe96c06bae0 0x7fe96c15e6f0 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7fe9600097c0 tx=0x7fe96001c310 comp rx=0 tx=0).ready entity=mon.1 client_cookie=1a253c111c80ae08 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:18:56.530 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:56.524+0000 7fe96affd640 1 -- 172.21.15.143:0/51587699 <== mon.1 v2:172.21.15.188:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe964001a00 con 0x7fe96c06bae0 2023-11-26T07:18:56.530 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:56.524+0000 7fe96affd640 1 -- 172.21.15.143:0/51587699 <== mon.1 v2:172.21.15.188:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe9600a7050 con 0x7fe96c06bae0 2023-11-26T07:18:56.530 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:56.524+0000 7fe96affd640 1 -- 172.21.15.143:0/51587699 <== mon.1 v2:172.21.15.188:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe9600ab620 con 0x7fe96c06bae0 2023-11-26T07:18:56.530 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:56.524+0000 7fe97238d640 1 -- 172.21.15.143:0/51587699 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fe96c06bae0 msgr2=0x7fe96c15e6f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:18:56.531 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:56.524+0000 7fe97238d640 1 --2- 172.21.15.143:0/51587699 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fe96c06bae0 0x7fe96c15e6f0 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7fe9600097c0 tx=0x7fe96001c310 comp rx=0 tx=0).stop 2023-11-26T07:18:56.531 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:56.524+0000 7fe97238d640 1 -- 172.21.15.143:0/51587699 shutdown_connections 2023-11-26T07:18:56.531 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:56.524+0000 7fe97238d640 1 --2- 172.21.15.143:0/51587699 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fe96c06bae0 0x7fe96c15e6f0 unknown :-1 s=CLOSED pgs=186 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:18:56.531 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:56.524+0000 7fe97238d640 1 -- 172.21.15.143:0/51587699 >> 172.21.15.143:0/51587699 conn(0x7fe96c0cf710 msgr2=0x7fe96c0cfb20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:18:56.531 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:56.524+0000 7fe97238d640 1 -- 172.21.15.143:0/51587699 shutdown_connections 2023-11-26T07:18:56.531 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:56.524+0000 7fe97238d640 1 -- 172.21.15.143:0/51587699 wait complete. 2023-11-26T07:18:56.532 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:56.524+0000 7fe97238d640 1 Processor -- start 2023-11-26T07:18:56.532 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:56.528+0000 7fe97238d640 1 -- start start 2023-11-26T07:18:56.534 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:56.528+0000 7fe97238d640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fe96c06bae0 0x7fe96c158500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:18:56.534 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:56.528+0000 7fe97238d640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fe96c070ba0 0x7fe96c1513d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:18:56.534 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:56.528+0000 7fe97238d640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fe96c0cd610 0x7fe96c156990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:18:56.534 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:56.528+0000 7fe97238d640 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe96c0aca80 con 0x7fe96c070ba0 2023-11-26T07:18:56.535 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:56.528+0000 7fe97238d640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe96c0acd60 con 0x7fe96c06bae0 2023-11-26T07:18:56.535 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:56.528+0000 7fe97238d640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe96c0ad040 con 0x7fe96c0cd610 2023-11-26T07:18:56.537 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:56.528+0000 7fe96b7fe640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fe96c070ba0 0x7fe96c1513d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:18:56.537 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:56.528+0000 7fe96b7fe640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fe96c070ba0 0x7fe96c1513d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:51708/0 (socket says 172.21.15.143:51708) 2023-11-26T07:18:56.537 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:56.528+0000 7fe96b7fe640 1 -- 172.21.15.143:0/3897996471 learned_addr learned my addr 172.21.15.143:0/3897996471 (peer_addr_for_me v2:172.21.15.143:0/0) 2023-11-26T07:18:56.537 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:56.528+0000 7fe96bfff640 1 --2- 172.21.15.143:0/3897996471 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fe96c06bae0 0x7fe96c158500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:18:56.537 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:56.528+0000 7fe97092a640 1 --2- 172.21.15.143:0/3897996471 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fe96c0cd610 0x7fe96c156990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:18:56.537 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:56.528+0000 7fe96b7fe640 1 -- 172.21.15.143:0/3897996471 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fe96c0cd610 msgr2=0x7fe96c156990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:18:56.537 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:56.528+0000 7fe96b7fe640 1 --2- 172.21.15.143:0/3897996471 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fe96c0cd610 0x7fe96c156990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:18:56.537 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:56.528+0000 7fe96b7fe640 1 -- 172.21.15.143:0/3897996471 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fe96c06bae0 msgr2=0x7fe96c158500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:18:56.537 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:56.528+0000 7fe96b7fe640 1 --2- 172.21.15.143:0/3897996471 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fe96c06bae0 0x7fe96c158500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:18:56.537 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:56.528+0000 7fe96b7fe640 1 -- 172.21.15.143:0/3897996471 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe960009420 con 0x7fe96c070ba0 2023-11-26T07:18:56.537 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:56.528+0000 7fe97092a640 1 --2- 172.21.15.143:0/3897996471 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fe96c0cd610 0x7fe96c156990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-26T07:18:56.537 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:56.528+0000 7fe96bfff640 1 --2- 172.21.15.143:0/3897996471 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fe96c06bae0 0x7fe96c158500 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-26T07:18:56.538 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:56.528+0000 7fe96b7fe640 1 --2- 172.21.15.143:0/3897996471 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fe96c070ba0 0x7fe96c1513d0 secure :-1 s=READY pgs=411 cs=0 l=1 rev1=1 crypto rx=0x7fe9600097c0 tx=0x7fe9600190b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:18:56.539 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:56.528+0000 7fe968ff9640 1 -- 172.21.15.143:0/3897996471 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe960004020 con 0x7fe96c070ba0 2023-11-26T07:18:56.539 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:56.528+0000 7fe968ff9640 1 -- 172.21.15.143:0/3897996471 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe960004180 con 0x7fe96c070ba0 2023-11-26T07:18:56.539 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:56.528+0000 7fe968ff9640 1 -- 172.21.15.143:0/3897996471 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe960004510 con 0x7fe96c070ba0 2023-11-26T07:18:56.539 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:56.528+0000 7fe97238d640 1 -- 172.21.15.143:0/3897996471 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe96c0ad320 con 0x7fe96c070ba0 2023-11-26T07:18:56.539 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:56.528+0000 7fe97238d640 1 -- 172.21.15.143:0/3897996471 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe96c155130 con 0x7fe96c070ba0 2023-11-26T07:18:56.539 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:56.528+0000 7fe97238d640 1 -- 172.21.15.143:0/3897996471 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe96c067680 con 0x7fe96c070ba0 2023-11-26T07:18:56.539 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:56.532+0000 7fe968ff9640 1 -- 172.21.15.143:0/3897996471 <== mon.0 v2:172.21.15.143:3300/0 4 ==== mgrmap(e 5) v1 ==== 85852+0+0 (secure 0 0 0) 0x7fe9600b2040 con 0x7fe96c070ba0 2023-11-26T07:18:56.539 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:56.532+0000 7fe968ff9640 1 --2- 172.21.15.143:0/3897996471 >> [v2:172.21.15.143:6832/2150557544,v1:172.21.15.143:6833/2150557544] conn(0x7fe958066470 0x7fe958068930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:18:56.539 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:56.532+0000 7fe968ff9640 1 -- 172.21.15.143:0/3897996471 <== mon.0 v2:172.21.15.143:3300/0 5 ==== osd_map(33..33 src has 1..33) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fe96011dd50 con 0x7fe96c070ba0 2023-11-26T07:18:56.539 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:56.532+0000 7fe96bfff640 1 --2- 172.21.15.143:0/3897996471 >> [v2:172.21.15.143:6832/2150557544,v1:172.21.15.143:6833/2150557544] conn(0x7fe958066470 0x7fe958068930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:18:56.540 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:56.532+0000 7fe96bfff640 1 --2- 172.21.15.143:0/3897996471 >> [v2:172.21.15.143:6832/2150557544,v1:172.21.15.143:6833/2150557544] conn(0x7fe958066470 0x7fe958068930 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7fe95c0011f0 tx=0x7fe95c009040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:18:56.541 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:56.536+0000 7fe968ff9640 1 -- 172.21.15.143:0/3897996471 <== mon.0 v2:172.21.15.143:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172649 (secure 0 0 0) 0x7fe9600eb1b0 con 0x7fe96c070ba0 2023-11-26T07:18:56.689 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:56.684+0000 7fe97238d640 1 -- 172.21.15.143:0/3897996471 --> [v2:172.21.15.143:6832/2150557544,v1:172.21.15.143:6833/2150557544] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7fe96c0ad1b0 con 0x7fe958066470 2023-11-26T07:18:56.697 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:56.692+0000 7fe968ff9640 1 -- 172.21.15.143:0/3897996471 <== mgr.4100 v2:172.21.15.143:6832/2150557544 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+284432 (secure 0 0 0) 0x7fe96c0ad1b0 con 0x7fe958066470 2023-11-26T07:18:56.700 INFO:teuthology.orchestra.run.smithi143.stdout: 2023-11-26T07:18:56.702 INFO:teuthology.orchestra.run.smithi143.stderr:dumped all 2023-11-26T07:18:56.703 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:56.696+0000 7fe97238d640 1 -- 172.21.15.143:0/3897996471 >> [v2:172.21.15.143:6832/2150557544,v1:172.21.15.143:6833/2150557544] conn(0x7fe958066470 msgr2=0x7fe958068930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:18:56.703 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:56.696+0000 7fe97238d640 1 --2- 172.21.15.143:0/3897996471 >> [v2:172.21.15.143:6832/2150557544,v1:172.21.15.143:6833/2150557544] conn(0x7fe958066470 0x7fe958068930 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7fe95c0011f0 tx=0x7fe95c009040 comp rx=0 tx=0).stop 2023-11-26T07:18:56.703 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:56.696+0000 7fe97238d640 1 -- 172.21.15.143:0/3897996471 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fe96c070ba0 msgr2=0x7fe96c1513d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:18:56.703 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:56.696+0000 7fe97238d640 1 --2- 172.21.15.143:0/3897996471 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fe96c070ba0 0x7fe96c1513d0 secure :-1 s=READY pgs=411 cs=0 l=1 rev1=1 crypto rx=0x7fe9600097c0 tx=0x7fe9600190b0 comp rx=0 tx=0).stop 2023-11-26T07:18:56.703 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:56.696+0000 7fe97238d640 1 -- 172.21.15.143:0/3897996471 shutdown_connections 2023-11-26T07:18:56.703 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:56.696+0000 7fe97238d640 1 --2- 172.21.15.143:0/3897996471 >> [v2:172.21.15.143:6832/2150557544,v1:172.21.15.143:6833/2150557544] conn(0x7fe958066470 0x7fe958068930 unknown :-1 s=CLOSED pgs=222 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:18:56.704 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:56.696+0000 7fe97238d640 1 --2- 172.21.15.143:0/3897996471 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fe96c0cd610 0x7fe96c156990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:18:56.704 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:56.696+0000 7fe97238d640 1 --2- 172.21.15.143:0/3897996471 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fe96c070ba0 0x7fe96c1513d0 unknown :-1 s=CLOSED pgs=411 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:18:56.704 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:56.696+0000 7fe97238d640 1 --2- 172.21.15.143:0/3897996471 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fe96c06bae0 0x7fe96c158500 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:18:56.704 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:56.696+0000 7fe97238d640 1 -- 172.21.15.143:0/3897996471 >> 172.21.15.143:0/3897996471 conn(0x7fe96c0cf710 msgr2=0x7fe96c161040 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:18:56.704 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:56.696+0000 7fe97238d640 1 -- 172.21.15.143:0/3897996471 shutdown_connections 2023-11-26T07:18:56.704 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:56.696+0000 7fe97238d640 1 -- 172.21.15.143:0/3897996471 wait complete. 2023-11-26T07:18:56.720 INFO:teuthology.orchestra.run.smithi143.stdout:{"pg_ready":true,"pg_map":{"version":63,"stamp":"2023-11-26T07:18:55.705391+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":20,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":219996,"kb_used_data":3820,"kb_used_omap":12,"kb_used_meta":216051,"kb_avail":754754724,"statfs":{"total":773094113280,"available":772868837376,"internally_reserved":0,"allocated":3911680,"data_stored":2217537,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12749,"internal_metadata":221236787},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":36,"apply_latency_ms":36,"commit_latency_ns":36000000,"apply_latency_ns":36000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"5.076804"},"pg_stats":[{"pgid":"5.3f","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2023-11-26T07:18:51.645063+0000","last_change":"2023-11-26T07:18:51.645063+0000","last_active":"2023-11-26T07:18:51.625829+0000","last_peered":"2023-11-26T07:18:51.625829+0000","last_clean":"2023-11-26T07:18:51.625829+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-26T07:18:51.645063+0000","last_undegraded":"2023-11-26T07:18:51.645063+0000","last_fullsized":"2023-11-26T07:18:51.645063+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:18:51.625829+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:18:51.625829+0000","last_clean_scrub_stamp":"2023-11-26T07:18:51.625829+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.3e","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2023-11-26T07:18:51.643947+0000","last_change":"2023-11-26T07:18:51.643947+0000","last_active":"2023-11-26T07:18:51.625829+0000","last_peered":"2023-11-26T07:18:51.625829+0000","last_clean":"2023-11-26T07:18:51.625829+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-26T07:18:51.643947+0000","last_undegraded":"2023-11-26T07:18:51.643947+0000","last_fullsized":"2023-11-26T07:18:51.643947+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:18:51.625829+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:18:51.625829+0000","last_clean_scrub_stamp":"2023-11-26T07:18:51.625829+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.3d","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2023-11-26T07:18:51.634980+0000","last_change":"2023-11-26T07:18:51.634980+0000","last_active":"2023-11-26T07:18:51.625829+0000","last_peered":"2023-11-26T07:18:51.625829+0000","last_clean":"2023-11-26T07:18:51.625829+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-26T07:18:51.634980+0000","last_undegraded":"2023-11-26T07:18:51.634980+0000","last_fullsized":"2023-11-26T07:18:51.634980+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:18:51.625829+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:18:51.625829+0000","last_clean_scrub_stamp":"2023-11-26T07:18:51.625829+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.3c","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2023-11-26T07:18:51.634028+0000","last_change":"2023-11-26T07:18:51.634028+0000","last_active":"2023-11-26T07:18:51.625829+0000","last_peered":"2023-11-26T07:18:51.625829+0000","last_clean":"2023-11-26T07:18:51.625829+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-26T07:18:51.634028+0000","last_undegraded":"2023-11-26T07:18:51.634028+0000","last_fullsized":"2023-11-26T07:18:51.634028+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:18:51.625829+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:18:51.625829+0000","last_clean_scrub_stamp":"2023-11-26T07:18:51.625829+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.3b","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2023-11-26T07:18:51.651017+0000","last_change":"2023-11-26T07:18:51.651017+0000","last_active":"2023-11-26T07:18:51.625829+0000","last_peered":"2023-11-26T07:18:51.625829+0000","last_clean":"2023-11-26T07:18:51.625829+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-26T07:18:51.651017+0000","last_undegraded":"2023-11-26T07:18:51.651017+0000","last_fullsized":"2023-11-26T07:18:51.651017+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:18:51.625829+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:18:51.625829+0000","last_clean_scrub_stamp":"2023-11-26T07:18:51.625829+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.3a","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2023-11-26T07:18:51.644135+0000","last_change":"2023-11-26T07:18:51.644135+0000","last_active":"2023-11-26T07:18:51.625829+0000","last_peered":"2023-11-26T07:18:51.625829+0000","last_clean":"2023-11-26T07:18:51.625829+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-26T07:18:51.644135+0000","last_undegraded":"2023-11-26T07:18:51.644135+0000","last_fullsized":"2023-11-26T07:18:51.644135+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:18:51.625829+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:18:51.625829+0000","last_clean_scrub_stamp":"2023-11-26T07:18:51.625829+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.39","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2023-11-26T07:18:51.641562+0000","last_change":"2023-11-26T07:18:51.641562+0000","last_active":"2023-11-26T07:18:51.625829+0000","last_peered":"2023-11-26T07:18:51.625829+0000","last_clean":"2023-11-26T07:18:51.625829+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-26T07:18:51.641562+0000","last_undegraded":"2023-11-26T07:18:51.641562+0000","last_fullsized":"2023-11-26T07:18:51.641562+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:18:51.625829+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:18:51.625829+0000","last_clean_scrub_stamp":"2023-11-26T07:18:51.625829+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.38","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2023-11-26T07:18:51.646503+0000","last_change":"2023-11-26T07:18:51.646503+0000","last_active":"2023-11-26T07:18:51.625829+0000","last_peered":"2023-11-26T07:18:51.625829+0000","last_clean":"2023-11-26T07:18:51.625829+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-26T07:18:51.646503+0000","last_undegraded":"2023-11-26T07:18:51.646503+0000","last_fullsized":"2023-11-26T07:18:51.646503+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:18:51.625829+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:18:51.625829+0000","last_clean_scrub_stamp":"2023-11-26T07:18:51.625829+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.37","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2023-11-26T07:18:51.636576+0000","last_change":"2023-11-26T07:18:51.636576+0000","last_active":"2023-11-26T07:18:51.625829+0000","last_peered":"2023-11-26T07:18:51.625829+0000","last_clean":"2023-11-26T07:18:51.625829+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-26T07:18:51.636576+0000","last_undegraded":"2023-11-26T07:18:51.636576+0000","last_fullsized":"2023-11-26T07:18:51.636576+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:18:51.625829+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:18:51.625829+0000","last_clean_scrub_stamp":"2023-11-26T07:18:51.625829+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.36","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2023-11-26T07:18:51.633151+0000","last_change":"2023-11-26T07:18:51.633151+0000","last_active":"2023-11-26T07:18:51.625829+0000","last_peered":"2023-11-26T07:18:51.625829+0000","last_clean":"2023-11-26T07:18:51.625829+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-26T07:18:51.633151+0000","last_undegraded":"2023-11-26T07:18:51.633151+0000","last_fullsized":"2023-11-26T07:18:51.633151+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:18:51.625829+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:18:51.625829+0000","last_clean_scrub_stamp":"2023-11-26T07:18:51.625829+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.35","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2023-11-26T07:18:51.640729+0000","last_change":"2023-11-26T07:18:51.640729+0000","last_active":"2023-11-26T07:18:51.625829+0000","last_peered":"2023-11-26T07:18:51.625829+0000","last_clean":"2023-11-26T07:18:51.625829+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-26T07:18:51.640729+0000","last_undegraded":"2023-11-26T07:18:51.640729+0000","last_fullsized":"2023-11-26T07:18:51.640729+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:18:51.625829+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:18:51.625829+0000","last_clean_scrub_stamp":"2023-11-26T07:18:51.625829+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.34","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2023-11-26T07:18:51.641193+0000","last_change":"2023-11-26T07:18:51.641193+0000","last_active":"2023-11-26T07:18:51.625829+0000","last_peered":"2023-11-26T07:18:51.625829+0000","last_clean":"2023-11-26T07:18:51.625829+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-26T07:18:51.641193+0000","last_undegraded":"2023-11-26T07:18:51.641193+0000","last_fullsized":"2023-11-26T07:18:51.641193+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:18:51.625829+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:18:51.625829+0000","last_clean_scrub_stamp":"2023-11-26T07:18:51.625829+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.33","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2023-11-26T07:18:51.649682+0000","last_change":"2023-11-26T07:18:51.649682+0000","last_active":"2023-11-26T07:18:51.625829+0000","last_peered":"2023-11-26T07:18:51.625829+0000","last_clean":"2023-11-26T07:18:51.625829+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-26T07:18:51.649682+0000","last_undegraded":"2023-11-26T07:18:51.649682+0000","last_fullsized":"2023-11-26T07:18:51.649682+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:18:51.625829+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:18:51.625829+0000","last_clean_scrub_stamp":"2023-11-26T07:18:51.625829+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.32","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2023-11-26T07:18:51.643138+0000","last_change":"2023-11-26T07:18:51.643138+0000","last_active":"2023-11-26T07:18:51.625829+0000","last_peered":"2023-11-26T07:18:51.625829+0000","last_clean":"2023-11-26T07:18:51.625829+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-26T07:18:51.643138+0000","last_undegraded":"2023-11-26T07:18:51.643138+0000","last_fullsized":"2023-11-26T07:18:51.643138+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:18:51.625829+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:18:51.625829+0000","last_clean_scrub_stamp":"2023-11-26T07:18:51.625829+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.31","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2023-11-26T07:18:51.634598+0000","last_change":"2023-11-26T07:18:51.634598+0000","last_active":"2023-11-26T07:18:51.625829+0000","last_peered":"2023-11-26T07:18:51.625829+0000","last_clean":"2023-11-26T07:18:51.625829+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-26T07:18:51.634598+0000","last_undegraded":"2023-11-26T07:18:51.634598+0000","last_fullsized":"2023-11-26T07:18:51.634598+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:18:51.625829+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:18:51.625829+0000","last_clean_scrub_stamp":"2023-11-26T07:18:51.625829+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.30","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2023-11-26T07:18:51.640197+0000","last_change":"2023-11-26T07:18:51.640197+0000","last_active":"2023-11-26T07:18:51.625829+0000","last_peered":"2023-11-26T07:18:51.625829+0000","last_clean":"2023-11-26T07:18:51.625829+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-26T07:18:51.640197+0000","last_undegraded":"2023-11-26T07:18:51.640197+0000","last_fullsized":"2023-11-26T07:18:51.640197+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:18:51.625829+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:18:51.625829+0000","last_clean_scrub_stamp":"2023-11-26T07:18:51.625829+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.2f","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2023-11-26T07:18:51.641801+0000","last_change":"2023-11-26T07:18:51.641801+0000","last_active":"2023-11-26T07:18:51.625829+0000","last_peered":"2023-11-26T07:18:51.625829+0000","last_clean":"2023-11-26T07:18:51.625829+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-26T07:18:51.641801+0000","last_undegraded":"2023-11-26T07:18:51.641801+0000","last_fullsized":"2023-11-26T07:18:51.641801+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:18:51.625829+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:18:51.625829+0000","last_clean_scrub_stamp":"2023-11-26T07:18:51.625829+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.2e","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2023-11-26T07:18:51.643556+0000","last_change":"2023-11-26T07:18:51.643556+0000","last_active":"2023-11-26T07:18:51.625829+0000","last_peered":"2023-11-26T07:18:51.625829+0000","last_clean":"2023-11-26T07:18:51.625829+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-26T07:18:51.643556+0000","last_undegraded":"2023-11-26T07:18:51.643556+0000","last_fullsized":"2023-11-26T07:18:51.643556+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:18:51.625829+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:18:51.625829+0000","last_clean_scrub_stamp":"2023-11-26T07:18:51.625829+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.2d","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2023-11-26T07:18:51.642785+0000","last_change":"2023-11-26T07:18:51.642785+0000","last_active":"2023-11-26T07:18:51.625829+0000","last_peered":"2023-11-26T07:18:51.625829+0000","last_clean":"2023-11-26T07:18:51.625829+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-26T07:18:51.642785+0000","last_undegraded":"2023-11-26T07:18:51.642785+0000","last_fullsized":"2023-11-26T07:18:51.642785+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:18:51.625829+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:18:51.625829+0000","last_clean_scrub_stamp":"2023-11-26T07:18:51.625829+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.2c","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2023-11-26T07:18:51.651641+0000","last_change":"2023-11-26T07:18:51.651641+0000","last_active":"2023-11-26T07:18:51.625829+0000","last_peered":"2023-11-26T07:18:51.625829+0000","last_clean":"2023-11-26T07:18:51.625829+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-26T07:18:51.651641+0000","last_undegraded":"2023-11-26T07:18:51.651641+0000","last_fullsized":"2023-11-26T07:18:51.651641+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:18:51.625829+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:18:51.625829+0000","last_clean_scrub_stamp":"2023-11-26T07:18:51.625829+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.2b","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2023-11-26T07:18:51.637758+0000","last_change":"2023-11-26T07:18:51.637758+0000","last_active":"2023-11-26T07:18:51.625829+0000","last_peered":"2023-11-26T07:18:51.625829+0000","last_clean":"2023-11-26T07:18:51.625829+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-26T07:18:51.637758+0000","last_undegraded":"2023-11-26T07:18:51.637758+0000","last_fullsized":"2023-11-26T07:18:51.637758+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:18:51.625829+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:18:51.625829+0000","last_clean_scrub_stamp":"2023-11-26T07:18:51.625829+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.2a","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2023-11-26T07:18:51.636183+0000","last_change":"2023-11-26T07:18:51.636183+0000","last_active":"2023-11-26T07:18:51.625829+0000","last_peered":"2023-11-26T07:18:51.625829+0000","last_clean":"2023-11-26T07:18:51.625829+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-26T07:18:51.636183+0000","last_undegraded":"2023-11-26T07:18:51.636183+0000","last_fullsized":"2023-11-26T07:18:51.636183+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:18:51.625829+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:18:51.625829+0000","last_clean_scrub_stamp":"2023-11-26T07:18:51.625829+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.29","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2023-11-26T07:18:51.641884+0000","last_change":"2023-11-26T07:18:51.641884+0000","last_active":"2023-11-26T07:18:51.625829+0000","last_peered":"2023-11-26T07:18:51.625829+0000","last_clean":"2023-11-26T07:18:51.625829+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-26T07:18:51.641884+0000","last_undegraded":"2023-11-26T07:18:51.641884+0000","last_fullsized":"2023-11-26T07:18:51.641884+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:18:51.625829+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:18:51.625829+0000","last_clean_scrub_stamp":"2023-11-26T07:18:51.625829+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.28","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2023-11-26T07:18:51.635231+0000","last_change":"2023-11-26T07:18:51.635231+0000","last_active":"2023-11-26T07:18:51.625829+0000","last_peered":"2023-11-26T07:18:51.625829+0000","last_clean":"2023-11-26T07:18:51.625829+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-26T07:18:51.635231+0000","last_undegraded":"2023-11-26T07:18:51.635231+0000","last_fullsized":"2023-11-26T07:18:51.635231+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:18:51.625829+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:18:51.625829+0000","last_clean_scrub_stamp":"2023-11-26T07:18:51.625829+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.27","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2023-11-26T07:18:51.635689+0000","last_change":"2023-11-26T07:18:51.635689+0000","last_active":"2023-11-26T07:18:51.625829+0000","last_peered":"2023-11-26T07:18:51.625829+0000","last_clean":"2023-11-26T07:18:51.625829+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-26T07:18:51.635689+0000","last_undegraded":"2023-11-26T07:18:51.635689+0000","last_fullsized":"2023-11-26T07:18:51.635689+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:18:51.625829+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:18:51.625829+0000","last_clean_scrub_stamp":"2023-11-26T07:18:51.625829+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.26","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2023-11-26T07:18:51.643037+0000","last_change":"2023-11-26T07:18:51.643037+0000","last_active":"2023-11-26T07:18:51.625829+0000","last_peered":"2023-11-26T07:18:51.625829+0000","last_clean":"2023-11-26T07:18:51.625829+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-26T07:18:51.643037+0000","last_undegraded":"2023-11-26T07:18:51.643037+0000","last_fullsized":"2023-11-26T07:18:51.643037+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:18:51.625829+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:18:51.625829+0000","last_clean_scrub_stamp":"2023-11-26T07:18:51.625829+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.25","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2023-11-26T07:18:51.634261+0000","last_change":"2023-11-26T07:18:51.634261+0000","last_active":"2023-11-26T07:18:51.625829+0000","last_peered":"2023-11-26T07:18:51.625829+0000","last_clean":"2023-11-26T07:18:51.625829+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-26T07:18:51.634261+0000","last_undegraded":"2023-11-26T07:18:51.634261+0000","last_fullsized":"2023-11-26T07:18:51.634261+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:18:51.625829+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:18:51.625829+0000","last_clean_scrub_stamp":"2023-11-26T07:18:51.625829+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.24","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2023-11-26T07:18:51.637792+0000","last_change":"2023-11-26T07:18:51.637792+0000","last_active":"2023-11-26T07:18:51.625829+0000","last_peered":"2023-11-26T07:18:51.625829+0000","last_clean":"2023-11-26T07:18:51.625829+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-26T07:18:51.637792+0000","last_undegraded":"2023-11-26T07:18:51.637792+0000","last_fullsized":"2023-11-26T07:18:51.637792+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:18:51.625829+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:18:51.625829+0000","last_clean_scrub_stamp":"2023-11-26T07:18:51.625829+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.23","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2023-11-26T07:18:51.637010+0000","last_change":"2023-11-26T07:18:51.637010+0000","last_active":"2023-11-26T07:18:51.625829+0000","last_peered":"2023-11-26T07:18:51.625829+0000","last_clean":"2023-11-26T07:18:51.625829+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-26T07:18:51.637010+0000","last_undegraded":"2023-11-26T07:18:51.637010+0000","last_fullsized":"2023-11-26T07:18:51.637010+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:18:51.625829+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:18:51.625829+0000","last_clean_scrub_stamp":"2023-11-26T07:18:51.625829+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.22","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2023-11-26T07:18:51.644576+0000","last_change":"2023-11-26T07:18:51.644576+0000","last_active":"2023-11-26T07:18:51.625829+0000","last_peered":"2023-11-26T07:18:51.625829+0000","last_clean":"2023-11-26T07:18:51.625829+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-26T07:18:51.644576+0000","last_undegraded":"2023-11-26T07:18:51.644576+0000","last_fullsized":"2023-11-26T07:18:51.644576+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:18:51.625829+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:18:51.625829+0000","last_clean_scrub_stamp":"2023-11-26T07:18:51.625829+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.21","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2023-11-26T07:18:51.644802+0000","last_change":"2023-11-26T07:18:51.644802+0000","last_active":"2023-11-26T07:18:51.625829+0000","last_peered":"2023-11-26T07:18:51.625829+0000","last_clean":"2023-11-26T07:18:51.625829+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-26T07:18:51.644802+0000","last_undegraded":"2023-11-26T07:18:51.644802+0000","last_fullsized":"2023-11-26T07:18:51.644802+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:18:51.625829+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:18:51.625829+0000","last_clean_scrub_stamp":"2023-11-26T07:18:51.625829+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.20","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2023-11-26T07:18:51.646001+0000","last_change":"2023-11-26T07:18:51.646001+0000","last_active":"2023-11-26T07:18:51.625829+0000","last_peered":"2023-11-26T07:18:51.625829+0000","last_clean":"2023-11-26T07:18:51.625829+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-26T07:18:51.646001+0000","last_undegraded":"2023-11-26T07:18:51.646001+0000","last_fullsized":"2023-11-26T07:18:51.646001+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:18:51.625829+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:18:51.625829+0000","last_clean_scrub_stamp":"2023-11-26T07:18:51.625829+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.1e","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2023-11-26T07:18:51.641565+0000","last_change":"2023-11-26T07:18:51.641565+0000","last_active":"2023-11-26T07:18:51.625829+0000","last_peered":"2023-11-26T07:18:51.625829+0000","last_clean":"2023-11-26T07:18:51.625829+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-26T07:18:51.641565+0000","last_undegraded":"2023-11-26T07:18:51.641565+0000","last_fullsized":"2023-11-26T07:18:51.641565+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:18:51.625829+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:18:51.625829+0000","last_clean_scrub_stamp":"2023-11-26T07:18:51.625829+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1f","version":"0'0","reported_seq":9,"reported_epoch":31,"state":"active+clean","last_fresh":"2023-11-26T07:18:51.664628+0000","last_change":"2023-11-26T07:18:51.661754+0000","last_active":"2023-11-26T07:18:51.664628+0000","last_peered":"2023-11-26T07:18:51.664628+0000","last_clean":"2023-11-26T07:18:51.664628+0000","last_became_active":"2023-11-26T07:18:51.661410+0000","last_became_peered":"2023-11-26T07:18:51.661410+0000","last_unstale":"2023-11-26T07:18:51.664628+0000","last_undegraded":"2023-11-26T07:18:51.664628+0000","last_fullsized":"2023-11-26T07:18:51.664628+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:18:50.617605+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:18:50.617605+0000","last_clean_scrub_stamp":"2023-11-26T07:18:50.617605+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T12:04:39.473159+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.1f","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2023-11-26T07:18:51.642728+0000","last_change":"2023-11-26T07:18:51.642728+0000","last_active":"2023-11-26T07:18:51.625829+0000","last_peered":"2023-11-26T07:18:51.625829+0000","last_clean":"2023-11-26T07:18:51.625829+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-26T07:18:51.642728+0000","last_undegraded":"2023-11-26T07:18:51.642728+0000","last_fullsized":"2023-11-26T07:18:51.642728+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:18:51.625829+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:18:51.625829+0000","last_clean_scrub_stamp":"2023-11-26T07:18:51.625829+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.1e","version":"0'0","reported_seq":9,"reported_epoch":31,"state":"active+clean","last_fresh":"2023-11-26T07:18:51.656785+0000","last_change":"2023-11-26T07:18:51.652999+0000","last_active":"2023-11-26T07:18:51.656785+0000","last_peered":"2023-11-26T07:18:51.656785+0000","last_clean":"2023-11-26T07:18:51.656785+0000","last_became_active":"2023-11-26T07:18:51.652268+0000","last_became_peered":"2023-11-26T07:18:51.652268+0000","last_unstale":"2023-11-26T07:18:51.656785+0000","last_undegraded":"2023-11-26T07:18:51.656785+0000","last_fullsized":"2023-11-26T07:18:51.656785+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:18:50.617605+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:18:50.617605+0000","last_clean_scrub_stamp":"2023-11-26T07:18:50.617605+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T17:27:24.322165+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.1c","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2023-11-26T07:18:51.638001+0000","last_change":"2023-11-26T07:18:51.638001+0000","last_active":"2023-11-26T07:18:51.625829+0000","last_peered":"2023-11-26T07:18:51.625829+0000","last_clean":"2023-11-26T07:18:51.625829+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-26T07:18:51.638001+0000","last_undegraded":"2023-11-26T07:18:51.638001+0000","last_fullsized":"2023-11-26T07:18:51.638001+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:18:51.625829+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:18:51.625829+0000","last_clean_scrub_stamp":"2023-11-26T07:18:51.625829+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.1d","version":"0'0","reported_seq":9,"reported_epoch":31,"state":"active+clean","last_fresh":"2023-11-26T07:18:51.661518+0000","last_change":"2023-11-26T07:18:51.653241+0000","last_active":"2023-11-26T07:18:51.661518+0000","last_peered":"2023-11-26T07:18:51.661518+0000","last_clean":"2023-11-26T07:18:51.661518+0000","last_became_active":"2023-11-26T07:18:51.652971+0000","last_became_peered":"2023-11-26T07:18:51.652971+0000","last_unstale":"2023-11-26T07:18:51.661518+0000","last_undegraded":"2023-11-26T07:18:51.661518+0000","last_fullsized":"2023-11-26T07:18:51.661518+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:18:50.617605+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:18:50.617605+0000","last_clean_scrub_stamp":"2023-11-26T07:18:50.617605+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T10:40:39.059176+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.1d","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2023-11-26T07:18:51.633747+0000","last_change":"2023-11-26T07:18:51.633747+0000","last_active":"2023-11-26T07:18:51.625829+0000","last_peered":"2023-11-26T07:18:51.625829+0000","last_clean":"2023-11-26T07:18:51.625829+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-26T07:18:51.633747+0000","last_undegraded":"2023-11-26T07:18:51.633747+0000","last_fullsized":"2023-11-26T07:18:51.633747+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:18:51.625829+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:18:51.625829+0000","last_clean_scrub_stamp":"2023-11-26T07:18:51.625829+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1c","version":"0'0","reported_seq":9,"reported_epoch":31,"state":"active+clean","last_fresh":"2023-11-26T07:18:51.664404+0000","last_change":"2023-11-26T07:18:51.662450+0000","last_active":"2023-11-26T07:18:51.664404+0000","last_peered":"2023-11-26T07:18:51.664404+0000","last_clean":"2023-11-26T07:18:51.664404+0000","last_became_active":"2023-11-26T07:18:51.662140+0000","last_became_peered":"2023-11-26T07:18:51.662140+0000","last_unstale":"2023-11-26T07:18:51.664404+0000","last_undegraded":"2023-11-26T07:18:51.664404+0000","last_fullsized":"2023-11-26T07:18:51.664404+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:18:50.617605+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:18:50.617605+0000","last_clean_scrub_stamp":"2023-11-26T07:18:50.617605+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T17:15:12.064097+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.1a","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2023-11-26T07:18:51.640301+0000","last_change":"2023-11-26T07:18:51.640301+0000","last_active":"2023-11-26T07:18:51.625829+0000","last_peered":"2023-11-26T07:18:51.625829+0000","last_clean":"2023-11-26T07:18:51.625829+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-26T07:18:51.640301+0000","last_undegraded":"2023-11-26T07:18:51.640301+0000","last_fullsized":"2023-11-26T07:18:51.640301+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:18:51.625829+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:18:51.625829+0000","last_clean_scrub_stamp":"2023-11-26T07:18:51.625829+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.1b","version":"0'0","reported_seq":9,"reported_epoch":31,"state":"active+clean","last_fresh":"2023-11-26T07:18:51.658420+0000","last_change":"2023-11-26T07:18:51.654689+0000","last_active":"2023-11-26T07:18:51.658420+0000","last_peered":"2023-11-26T07:18:51.658420+0000","last_clean":"2023-11-26T07:18:51.658420+0000","last_became_active":"2023-11-26T07:18:51.653354+0000","last_became_peered":"2023-11-26T07:18:51.653354+0000","last_unstale":"2023-11-26T07:18:51.658420+0000","last_undegraded":"2023-11-26T07:18:51.658420+0000","last_fullsized":"2023-11-26T07:18:51.658420+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:18:50.617605+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:18:50.617605+0000","last_clean_scrub_stamp":"2023-11-26T07:18:50.617605+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T17:28:21.481079+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.1b","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2023-11-26T07:18:51.635218+0000","last_change":"2023-11-26T07:18:51.635218+0000","last_active":"2023-11-26T07:18:51.625829+0000","last_peered":"2023-11-26T07:18:51.625829+0000","last_clean":"2023-11-26T07:18:51.625829+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-26T07:18:51.635218+0000","last_undegraded":"2023-11-26T07:18:51.635218+0000","last_fullsized":"2023-11-26T07:18:51.635218+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:18:51.625829+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:18:51.625829+0000","last_clean_scrub_stamp":"2023-11-26T07:18:51.625829+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1a","version":"0'0","reported_seq":9,"reported_epoch":31,"state":"active+clean","last_fresh":"2023-11-26T07:18:51.656991+0000","last_change":"2023-11-26T07:18:51.653682+0000","last_active":"2023-11-26T07:18:51.656991+0000","last_peered":"2023-11-26T07:18:51.656991+0000","last_clean":"2023-11-26T07:18:51.656991+0000","last_became_active":"2023-11-26T07:18:51.653366+0000","last_became_peered":"2023-11-26T07:18:51.653366+0000","last_unstale":"2023-11-26T07:18:51.656991+0000","last_undegraded":"2023-11-26T07:18:51.656991+0000","last_fullsized":"2023-11-26T07:18:51.656991+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:18:50.617605+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:18:50.617605+0000","last_clean_scrub_stamp":"2023-11-26T07:18:50.617605+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T18:27:57.438972+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.18","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2023-11-26T07:18:51.634554+0000","last_change":"2023-11-26T07:18:51.634554+0000","last_active":"2023-11-26T07:18:51.625829+0000","last_peered":"2023-11-26T07:18:51.625829+0000","last_clean":"2023-11-26T07:18:51.625829+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-26T07:18:51.634554+0000","last_undegraded":"2023-11-26T07:18:51.634554+0000","last_fullsized":"2023-11-26T07:18:51.634554+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:18:51.625829+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:18:51.625829+0000","last_clean_scrub_stamp":"2023-11-26T07:18:51.625829+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.19","version":"0'0","reported_seq":9,"reported_epoch":31,"state":"active+clean","last_fresh":"2023-11-26T07:18:51.664030+0000","last_change":"2023-11-26T07:18:51.657763+0000","last_active":"2023-11-26T07:18:51.664030+0000","last_peered":"2023-11-26T07:18:51.664030+0000","last_clean":"2023-11-26T07:18:51.664030+0000","last_became_active":"2023-11-26T07:18:51.656677+0000","last_became_peered":"2023-11-26T07:18:51.656677+0000","last_unstale":"2023-11-26T07:18:51.664030+0000","last_undegraded":"2023-11-26T07:18:51.664030+0000","last_fullsized":"2023-11-26T07:18:51.664030+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:18:50.617605+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:18:50.617605+0000","last_clean_scrub_stamp":"2023-11-26T07:18:50.617605+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T18:26:18.407283+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.19","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2023-11-26T07:18:51.633130+0000","last_change":"2023-11-26T07:18:51.633130+0000","last_active":"2023-11-26T07:18:51.625829+0000","last_peered":"2023-11-26T07:18:51.625829+0000","last_clean":"2023-11-26T07:18:51.625829+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-26T07:18:51.633130+0000","last_undegraded":"2023-11-26T07:18:51.633130+0000","last_fullsized":"2023-11-26T07:18:51.633130+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:18:51.625829+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:18:51.625829+0000","last_clean_scrub_stamp":"2023-11-26T07:18:51.625829+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.18","version":"0'0","reported_seq":9,"reported_epoch":31,"state":"active+clean","last_fresh":"2023-11-26T07:18:51.658262+0000","last_change":"2023-11-26T07:18:51.656204+0000","last_active":"2023-11-26T07:18:51.658262+0000","last_peered":"2023-11-26T07:18:51.658262+0000","last_clean":"2023-11-26T07:18:51.658262+0000","last_became_active":"2023-11-26T07:18:51.655809+0000","last_became_peered":"2023-11-26T07:18:51.655809+0000","last_unstale":"2023-11-26T07:18:51.658262+0000","last_undegraded":"2023-11-26T07:18:51.658262+0000","last_fullsized":"2023-11-26T07:18:51.658262+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:18:50.617605+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:18:50.617605+0000","last_clean_scrub_stamp":"2023-11-26T07:18:50.617605+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T13:58:04.420017+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.16","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2023-11-26T07:18:51.635635+0000","last_change":"2023-11-26T07:18:51.635635+0000","last_active":"2023-11-26T07:18:51.625829+0000","last_peered":"2023-11-26T07:18:51.625829+0000","last_clean":"2023-11-26T07:18:51.625829+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-26T07:18:51.635635+0000","last_undegraded":"2023-11-26T07:18:51.635635+0000","last_fullsized":"2023-11-26T07:18:51.635635+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:18:51.625829+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:18:51.625829+0000","last_clean_scrub_stamp":"2023-11-26T07:18:51.625829+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.17","version":"0'0","reported_seq":9,"reported_epoch":31,"state":"active+clean","last_fresh":"2023-11-26T07:18:51.661481+0000","last_change":"2023-11-26T07:18:51.653322+0000","last_active":"2023-11-26T07:18:51.661481+0000","last_peered":"2023-11-26T07:18:51.661481+0000","last_clean":"2023-11-26T07:18:51.661481+0000","last_became_active":"2023-11-26T07:18:51.652388+0000","last_became_peered":"2023-11-26T07:18:51.652388+0000","last_unstale":"2023-11-26T07:18:51.661481+0000","last_undegraded":"2023-11-26T07:18:51.661481+0000","last_fullsized":"2023-11-26T07:18:51.661481+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:18:50.617605+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:18:50.617605+0000","last_clean_scrub_stamp":"2023-11-26T07:18:50.617605+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T07:50:54.298421+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.17","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2023-11-26T07:18:51.642485+0000","last_change":"2023-11-26T07:18:51.642485+0000","last_active":"2023-11-26T07:18:51.625829+0000","last_peered":"2023-11-26T07:18:51.625829+0000","last_clean":"2023-11-26T07:18:51.625829+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-26T07:18:51.642485+0000","last_undegraded":"2023-11-26T07:18:51.642485+0000","last_fullsized":"2023-11-26T07:18:51.642485+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:18:51.625829+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:18:51.625829+0000","last_clean_scrub_stamp":"2023-11-26T07:18:51.625829+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.16","version":"0'0","reported_seq":9,"reported_epoch":31,"state":"active+clean","last_fresh":"2023-11-26T07:18:51.650667+0000","last_change":"2023-11-26T07:18:51.644845+0000","last_active":"2023-11-26T07:18:51.650667+0000","last_peered":"2023-11-26T07:18:51.650667+0000","last_clean":"2023-11-26T07:18:51.650667+0000","last_became_active":"2023-11-26T07:18:51.644261+0000","last_became_peered":"2023-11-26T07:18:51.644261+0000","last_unstale":"2023-11-26T07:18:51.650667+0000","last_undegraded":"2023-11-26T07:18:51.650667+0000","last_fullsized":"2023-11-26T07:18:51.650667+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:18:50.617605+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:18:50.617605+0000","last_clean_scrub_stamp":"2023-11-26T07:18:50.617605+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T18:26:18.407283+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.14","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2023-11-26T07:18:51.643692+0000","last_change":"2023-11-26T07:18:51.643692+0000","last_active":"2023-11-26T07:18:51.625829+0000","last_peered":"2023-11-26T07:18:51.625829+0000","last_clean":"2023-11-26T07:18:51.625829+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-26T07:18:51.643692+0000","last_undegraded":"2023-11-26T07:18:51.643692+0000","last_fullsized":"2023-11-26T07:18:51.643692+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:18:51.625829+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:18:51.625829+0000","last_clean_scrub_stamp":"2023-11-26T07:18:51.625829+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.15","version":"0'0","reported_seq":9,"reported_epoch":31,"state":"active+clean","last_fresh":"2023-11-26T07:18:51.660677+0000","last_change":"2023-11-26T07:18:51.657428+0000","last_active":"2023-11-26T07:18:51.660677+0000","last_peered":"2023-11-26T07:18:51.660677+0000","last_clean":"2023-11-26T07:18:51.660677+0000","last_became_active":"2023-11-26T07:18:51.657075+0000","last_became_peered":"2023-11-26T07:18:51.657075+0000","last_unstale":"2023-11-26T07:18:51.660677+0000","last_undegraded":"2023-11-26T07:18:51.660677+0000","last_fullsized":"2023-11-26T07:18:51.660677+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:18:50.617605+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:18:50.617605+0000","last_clean_scrub_stamp":"2023-11-26T07:18:50.617605+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T07:50:54.298421+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.15","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2023-11-26T07:18:51.638905+0000","last_change":"2023-11-26T07:18:51.638905+0000","last_active":"2023-11-26T07:18:51.625829+0000","last_peered":"2023-11-26T07:18:51.625829+0000","last_clean":"2023-11-26T07:18:51.625829+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-26T07:18:51.638905+0000","last_undegraded":"2023-11-26T07:18:51.638905+0000","last_fullsized":"2023-11-26T07:18:51.638905+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:18:51.625829+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:18:51.625829+0000","last_clean_scrub_stamp":"2023-11-26T07:18:51.625829+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.14","version":"0'0","reported_seq":9,"reported_epoch":31,"state":"active+clean","last_fresh":"2023-11-26T07:18:51.651808+0000","last_change":"2023-11-26T07:18:51.647209+0000","last_active":"2023-11-26T07:18:51.651808+0000","last_peered":"2023-11-26T07:18:51.651808+0000","last_clean":"2023-11-26T07:18:51.651808+0000","last_became_active":"2023-11-26T07:18:51.647051+0000","last_became_peered":"2023-11-26T07:18:51.647051+0000","last_unstale":"2023-11-26T07:18:51.651808+0000","last_undegraded":"2023-11-26T07:18:51.651808+0000","last_fullsized":"2023-11-26T07:18:51.651808+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:18:50.617605+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:18:50.617605+0000","last_clean_scrub_stamp":"2023-11-26T07:18:50.617605+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T14:40:15.233288+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.12","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2023-11-26T07:18:51.633529+0000","last_change":"2023-11-26T07:18:51.633529+0000","last_active":"2023-11-26T07:18:51.625829+0000","last_peered":"2023-11-26T07:18:51.625829+0000","last_clean":"2023-11-26T07:18:51.625829+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-26T07:18:51.633529+0000","last_undegraded":"2023-11-26T07:18:51.633529+0000","last_fullsized":"2023-11-26T07:18:51.633529+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:18:51.625829+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:18:51.625829+0000","last_clean_scrub_stamp":"2023-11-26T07:18:51.625829+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.13","version":"0'0","reported_seq":9,"reported_epoch":31,"state":"active+clean","last_fresh":"2023-11-26T07:18:51.659054+0000","last_change":"2023-11-26T07:18:51.656339+0000","last_active":"2023-11-26T07:18:51.659054+0000","last_peered":"2023-11-26T07:18:51.659054+0000","last_clean":"2023-11-26T07:18:51.659054+0000","last_became_active":"2023-11-26T07:18:51.655998+0000","last_became_peered":"2023-11-26T07:18:51.655998+0000","last_unstale":"2023-11-26T07:18:51.659054+0000","last_undegraded":"2023-11-26T07:18:51.659054+0000","last_fullsized":"2023-11-26T07:18:51.659054+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:18:50.617605+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:18:50.617605+0000","last_clean_scrub_stamp":"2023-11-26T07:18:50.617605+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T11:29:07.700412+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.13","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2023-11-26T07:18:51.642640+0000","last_change":"2023-11-26T07:18:51.642640+0000","last_active":"2023-11-26T07:18:51.625829+0000","last_peered":"2023-11-26T07:18:51.625829+0000","last_clean":"2023-11-26T07:18:51.625829+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-26T07:18:51.642640+0000","last_undegraded":"2023-11-26T07:18:51.642640+0000","last_fullsized":"2023-11-26T07:18:51.642640+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:18:51.625829+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:18:51.625829+0000","last_clean_scrub_stamp":"2023-11-26T07:18:51.625829+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.12","version":"0'0","reported_seq":9,"reported_epoch":31,"state":"active+clean","last_fresh":"2023-11-26T07:18:51.663995+0000","last_change":"2023-11-26T07:18:51.658016+0000","last_active":"2023-11-26T07:18:51.663995+0000","last_peered":"2023-11-26T07:18:51.663995+0000","last_clean":"2023-11-26T07:18:51.663995+0000","last_became_active":"2023-11-26T07:18:51.657646+0000","last_became_peered":"2023-11-26T07:18:51.657646+0000","last_unstale":"2023-11-26T07:18:51.663995+0000","last_undegraded":"2023-11-26T07:18:51.663995+0000","last_fullsized":"2023-11-26T07:18:51.663995+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:18:50.617605+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:18:50.617605+0000","last_clean_scrub_stamp":"2023-11-26T07:18:50.617605+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T19:10:39.237269+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.10","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2023-11-26T07:18:51.645524+0000","last_change":"2023-11-26T07:18:51.645524+0000","last_active":"2023-11-26T07:18:51.625829+0000","last_peered":"2023-11-26T07:18:51.625829+0000","last_clean":"2023-11-26T07:18:51.625829+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-26T07:18:51.645524+0000","last_undegraded":"2023-11-26T07:18:51.645524+0000","last_fullsized":"2023-11-26T07:18:51.645524+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:18:51.625829+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:18:51.625829+0000","last_clean_scrub_stamp":"2023-11-26T07:18:51.625829+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.11","version":"0'0","reported_seq":9,"reported_epoch":31,"state":"active+clean","last_fresh":"2023-11-26T07:18:51.661545+0000","last_change":"2023-11-26T07:18:51.654685+0000","last_active":"2023-11-26T07:18:51.661545+0000","last_peered":"2023-11-26T07:18:51.661545+0000","last_clean":"2023-11-26T07:18:51.661545+0000","last_became_active":"2023-11-26T07:18:51.653542+0000","last_became_peered":"2023-11-26T07:18:51.653542+0000","last_unstale":"2023-11-26T07:18:51.661545+0000","last_undegraded":"2023-11-26T07:18:51.661545+0000","last_fullsized":"2023-11-26T07:18:51.661545+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:18:50.617605+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:18:50.617605+0000","last_clean_scrub_stamp":"2023-11-26T07:18:50.617605+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T14:17:17.432332+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.11","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2023-11-26T07:18:51.642391+0000","last_change":"2023-11-26T07:18:51.642391+0000","last_active":"2023-11-26T07:18:51.625829+0000","last_peered":"2023-11-26T07:18:51.625829+0000","last_clean":"2023-11-26T07:18:51.625829+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-26T07:18:51.642391+0000","last_undegraded":"2023-11-26T07:18:51.642391+0000","last_fullsized":"2023-11-26T07:18:51.642391+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:18:51.625829+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:18:51.625829+0000","last_clean_scrub_stamp":"2023-11-26T07:18:51.625829+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.10","version":"0'0","reported_seq":9,"reported_epoch":31,"state":"active+clean","last_fresh":"2023-11-26T07:18:51.661372+0000","last_change":"2023-11-26T07:18:51.652858+0000","last_active":"2023-11-26T07:18:51.661372+0000","last_peered":"2023-11-26T07:18:51.661372+0000","last_clean":"2023-11-26T07:18:51.661372+0000","last_became_active":"2023-11-26T07:18:51.652583+0000","last_became_peered":"2023-11-26T07:18:51.652583+0000","last_unstale":"2023-11-26T07:18:51.661372+0000","last_undegraded":"2023-11-26T07:18:51.661372+0000","last_fullsized":"2023-11-26T07:18:51.661372+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:18:50.617605+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:18:50.617605+0000","last_clean_scrub_stamp":"2023-11-26T07:18:50.617605+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T14:49:28.420672+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.e","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2023-11-26T07:18:51.635911+0000","last_change":"2023-11-26T07:18:51.635911+0000","last_active":"2023-11-26T07:18:51.625829+0000","last_peered":"2023-11-26T07:18:51.625829+0000","last_clean":"2023-11-26T07:18:51.625829+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-26T07:18:51.635911+0000","last_undegraded":"2023-11-26T07:18:51.635911+0000","last_fullsized":"2023-11-26T07:18:51.635911+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:18:51.625829+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:18:51.625829+0000","last_clean_scrub_stamp":"2023-11-26T07:18:51.625829+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.f","version":"0'0","reported_seq":9,"reported_epoch":31,"state":"active+clean","last_fresh":"2023-11-26T07:18:51.661667+0000","last_change":"2023-11-26T07:18:51.656212+0000","last_active":"2023-11-26T07:18:51.661667+0000","last_peered":"2023-11-26T07:18:51.661667+0000","last_clean":"2023-11-26T07:18:51.661667+0000","last_became_active":"2023-11-26T07:18:51.650720+0000","last_became_peered":"2023-11-26T07:18:51.650720+0000","last_unstale":"2023-11-26T07:18:51.661667+0000","last_undegraded":"2023-11-26T07:18:51.661667+0000","last_fullsized":"2023-11-26T07:18:51.661667+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:18:50.617605+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:18:50.617605+0000","last_clean_scrub_stamp":"2023-11-26T07:18:50.617605+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T14:40:15.233288+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.f","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2023-11-26T07:18:51.641273+0000","last_change":"2023-11-26T07:18:51.641273+0000","last_active":"2023-11-26T07:18:51.625829+0000","last_peered":"2023-11-26T07:18:51.625829+0000","last_clean":"2023-11-26T07:18:51.625829+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-26T07:18:51.641273+0000","last_undegraded":"2023-11-26T07:18:51.641273+0000","last_fullsized":"2023-11-26T07:18:51.641273+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:18:51.625829+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:18:51.625829+0000","last_clean_scrub_stamp":"2023-11-26T07:18:51.625829+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.e","version":"0'0","reported_seq":9,"reported_epoch":31,"state":"active+clean","last_fresh":"2023-11-26T07:18:51.664526+0000","last_change":"2023-11-26T07:18:51.660325+0000","last_active":"2023-11-26T07:18:51.664526+0000","last_peered":"2023-11-26T07:18:51.664526+0000","last_clean":"2023-11-26T07:18:51.664526+0000","last_became_active":"2023-11-26T07:18:51.659918+0000","last_became_peered":"2023-11-26T07:18:51.659918+0000","last_unstale":"2023-11-26T07:18:51.664526+0000","last_undegraded":"2023-11-26T07:18:51.664526+0000","last_fullsized":"2023-11-26T07:18:51.664526+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:18:50.617605+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:18:50.617605+0000","last_clean_scrub_stamp":"2023-11-26T07:18:50.617605+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T15:20:53.245654+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.c","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2023-11-26T07:18:51.633505+0000","last_change":"2023-11-26T07:18:51.633505+0000","last_active":"2023-11-26T07:18:51.625829+0000","last_peered":"2023-11-26T07:18:51.625829+0000","last_clean":"2023-11-26T07:18:51.625829+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-26T07:18:51.633505+0000","last_undegraded":"2023-11-26T07:18:51.633505+0000","last_fullsized":"2023-11-26T07:18:51.633505+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:18:51.625829+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:18:51.625829+0000","last_clean_scrub_stamp":"2023-11-26T07:18:51.625829+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.d","version":"0'0","reported_seq":9,"reported_epoch":31,"state":"active+clean","last_fresh":"2023-11-26T07:18:51.662230+0000","last_change":"2023-11-26T07:18:51.658440+0000","last_active":"2023-11-26T07:18:51.662230+0000","last_peered":"2023-11-26T07:18:51.662230+0000","last_clean":"2023-11-26T07:18:51.662230+0000","last_became_active":"2023-11-26T07:18:51.658161+0000","last_became_peered":"2023-11-26T07:18:51.658161+0000","last_unstale":"2023-11-26T07:18:51.662230+0000","last_undegraded":"2023-11-26T07:18:51.662230+0000","last_fullsized":"2023-11-26T07:18:51.662230+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:18:50.617605+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:18:50.617605+0000","last_clean_scrub_stamp":"2023-11-26T07:18:50.617605+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T08:54:02.907365+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.d","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2023-11-26T07:18:51.653086+0000","last_change":"2023-11-26T07:18:51.653086+0000","last_active":"2023-11-26T07:18:51.625829+0000","last_peered":"2023-11-26T07:18:51.625829+0000","last_clean":"2023-11-26T07:18:51.625829+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-26T07:18:51.653086+0000","last_undegraded":"2023-11-26T07:18:51.653086+0000","last_fullsized":"2023-11-26T07:18:51.653086+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:18:51.625829+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:18:51.625829+0000","last_clean_scrub_stamp":"2023-11-26T07:18:51.625829+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.c","version":"0'0","reported_seq":9,"reported_epoch":31,"state":"active+clean","last_fresh":"2023-11-26T07:18:51.658594+0000","last_change":"2023-11-26T07:18:51.654164+0000","last_active":"2023-11-26T07:18:51.658594+0000","last_peered":"2023-11-26T07:18:51.658594+0000","last_clean":"2023-11-26T07:18:51.658594+0000","last_became_active":"2023-11-26T07:18:51.653823+0000","last_became_peered":"2023-11-26T07:18:51.653823+0000","last_unstale":"2023-11-26T07:18:51.658594+0000","last_undegraded":"2023-11-26T07:18:51.658594+0000","last_fullsized":"2023-11-26T07:18:51.658594+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:18:50.617605+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:18:50.617605+0000","last_clean_scrub_stamp":"2023-11-26T07:18:50.617605+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T12:22:07.968357+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.a","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2023-11-26T07:18:51.649002+0000","last_change":"2023-11-26T07:18:51.649002+0000","last_active":"2023-11-26T07:18:51.625829+0000","last_peered":"2023-11-26T07:18:51.625829+0000","last_clean":"2023-11-26T07:18:51.625829+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-26T07:18:51.649002+0000","last_undegraded":"2023-11-26T07:18:51.649002+0000","last_fullsized":"2023-11-26T07:18:51.649002+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:18:51.625829+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:18:51.625829+0000","last_clean_scrub_stamp":"2023-11-26T07:18:51.625829+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.b","version":"0'0","reported_seq":9,"reported_epoch":31,"state":"active+clean","last_fresh":"2023-11-26T07:18:51.655810+0000","last_change":"2023-11-26T07:18:51.646694+0000","last_active":"2023-11-26T07:18:51.655810+0000","last_peered":"2023-11-26T07:18:51.655810+0000","last_clean":"2023-11-26T07:18:51.655810+0000","last_became_active":"2023-11-26T07:18:51.644567+0000","last_became_peered":"2023-11-26T07:18:51.644567+0000","last_unstale":"2023-11-26T07:18:51.655810+0000","last_undegraded":"2023-11-26T07:18:51.655810+0000","last_fullsized":"2023-11-26T07:18:51.655810+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:18:50.617605+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:18:50.617605+0000","last_clean_scrub_stamp":"2023-11-26T07:18:50.617605+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T12:04:39.473159+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.b","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2023-11-26T07:18:51.649020+0000","last_change":"2023-11-26T07:18:51.649020+0000","last_active":"2023-11-26T07:18:51.625829+0000","last_peered":"2023-11-26T07:18:51.625829+0000","last_clean":"2023-11-26T07:18:51.625829+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-26T07:18:51.649020+0000","last_undegraded":"2023-11-26T07:18:51.649020+0000","last_fullsized":"2023-11-26T07:18:51.649020+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:18:51.625829+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:18:51.625829+0000","last_clean_scrub_stamp":"2023-11-26T07:18:51.625829+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.a","version":"0'0","reported_seq":9,"reported_epoch":31,"state":"active+clean","last_fresh":"2023-11-26T07:18:51.664562+0000","last_change":"2023-11-26T07:18:51.660988+0000","last_active":"2023-11-26T07:18:51.664562+0000","last_peered":"2023-11-26T07:18:51.664562+0000","last_clean":"2023-11-26T07:18:51.664562+0000","last_became_active":"2023-11-26T07:18:51.660676+0000","last_became_peered":"2023-11-26T07:18:51.660676+0000","last_unstale":"2023-11-26T07:18:51.664562+0000","last_undegraded":"2023-11-26T07:18:51.664562+0000","last_fullsized":"2023-11-26T07:18:51.664562+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:18:50.617605+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:18:50.617605+0000","last_clean_scrub_stamp":"2023-11-26T07:18:50.617605+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T19:10:39.237269+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.8","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2023-11-26T07:18:51.649368+0000","last_change":"2023-11-26T07:18:51.649368+0000","last_active":"2023-11-26T07:18:51.625829+0000","last_peered":"2023-11-26T07:18:51.625829+0000","last_clean":"2023-11-26T07:18:51.625829+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-26T07:18:51.649368+0000","last_undegraded":"2023-11-26T07:18:51.649368+0000","last_fullsized":"2023-11-26T07:18:51.649368+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:18:51.625829+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:18:51.625829+0000","last_clean_scrub_stamp":"2023-11-26T07:18:51.625829+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.9","version":"0'0","reported_seq":9,"reported_epoch":31,"state":"active+clean","last_fresh":"2023-11-26T07:18:51.657644+0000","last_change":"2023-11-26T07:18:51.652977+0000","last_active":"2023-11-26T07:18:51.657644+0000","last_peered":"2023-11-26T07:18:51.657644+0000","last_clean":"2023-11-26T07:18:51.657644+0000","last_became_active":"2023-11-26T07:18:51.652670+0000","last_became_peered":"2023-11-26T07:18:51.652670+0000","last_unstale":"2023-11-26T07:18:51.657644+0000","last_undegraded":"2023-11-26T07:18:51.657644+0000","last_fullsized":"2023-11-26T07:18:51.657644+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:18:50.617605+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:18:50.617605+0000","last_clean_scrub_stamp":"2023-11-26T07:18:50.617605+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T15:39:09.212431+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.9","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2023-11-26T07:18:51.642082+0000","last_change":"2023-11-26T07:18:51.642082+0000","last_active":"2023-11-26T07:18:51.625829+0000","last_peered":"2023-11-26T07:18:51.625829+0000","last_clean":"2023-11-26T07:18:51.625829+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-26T07:18:51.642082+0000","last_undegraded":"2023-11-26T07:18:51.642082+0000","last_fullsized":"2023-11-26T07:18:51.642082+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:18:51.625829+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:18:51.625829+0000","last_clean_scrub_stamp":"2023-11-26T07:18:51.625829+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.8","version":"0'0","reported_seq":9,"reported_epoch":31,"state":"active+clean","last_fresh":"2023-11-26T07:18:51.662484+0000","last_change":"2023-11-26T07:18:51.659037+0000","last_active":"2023-11-26T07:18:51.662484+0000","last_peered":"2023-11-26T07:18:51.662484+0000","last_clean":"2023-11-26T07:18:51.662484+0000","last_became_active":"2023-11-26T07:18:51.658596+0000","last_became_peered":"2023-11-26T07:18:51.658596+0000","last_unstale":"2023-11-26T07:18:51.662484+0000","last_undegraded":"2023-11-26T07:18:51.662484+0000","last_fullsized":"2023-11-26T07:18:51.662484+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:18:50.617605+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:18:50.617605+0000","last_clean_scrub_stamp":"2023-11-26T07:18:50.617605+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T10:40:39.059176+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.6","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2023-11-26T07:18:51.651544+0000","last_change":"2023-11-26T07:18:51.651544+0000","last_active":"2023-11-26T07:18:51.625829+0000","last_peered":"2023-11-26T07:18:51.625829+0000","last_clean":"2023-11-26T07:18:51.625829+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-26T07:18:51.651544+0000","last_undegraded":"2023-11-26T07:18:51.651544+0000","last_fullsized":"2023-11-26T07:18:51.651544+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:18:51.625829+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:18:51.625829+0000","last_clean_scrub_stamp":"2023-11-26T07:18:51.625829+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.7","version":"0'0","reported_seq":9,"reported_epoch":31,"state":"active+clean","last_fresh":"2023-11-26T07:18:51.661322+0000","last_change":"2023-11-26T07:18:51.653789+0000","last_active":"2023-11-26T07:18:51.661322+0000","last_peered":"2023-11-26T07:18:51.661322+0000","last_clean":"2023-11-26T07:18:51.661322+0000","last_became_active":"2023-11-26T07:18:51.652136+0000","last_became_peered":"2023-11-26T07:18:51.652136+0000","last_unstale":"2023-11-26T07:18:51.661322+0000","last_undegraded":"2023-11-26T07:18:51.661322+0000","last_fullsized":"2023-11-26T07:18:51.661322+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:18:50.617605+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:18:50.617605+0000","last_clean_scrub_stamp":"2023-11-26T07:18:50.617605+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T18:26:18.407283+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.7","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2023-11-26T07:18:51.637655+0000","last_change":"2023-11-26T07:18:51.637655+0000","last_active":"2023-11-26T07:18:51.625829+0000","last_peered":"2023-11-26T07:18:51.625829+0000","last_clean":"2023-11-26T07:18:51.625829+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-26T07:18:51.637655+0000","last_undegraded":"2023-11-26T07:18:51.637655+0000","last_fullsized":"2023-11-26T07:18:51.637655+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:18:51.625829+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:18:51.625829+0000","last_clean_scrub_stamp":"2023-11-26T07:18:51.625829+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.6","version":"0'0","reported_seq":9,"reported_epoch":31,"state":"active+clean","last_fresh":"2023-11-26T07:18:51.648265+0000","last_change":"2023-11-26T07:18:51.644857+0000","last_active":"2023-11-26T07:18:51.648265+0000","last_peered":"2023-11-26T07:18:51.648265+0000","last_clean":"2023-11-26T07:18:51.648265+0000","last_became_active":"2023-11-26T07:18:51.644616+0000","last_became_peered":"2023-11-26T07:18:51.644616+0000","last_unstale":"2023-11-26T07:18:51.648265+0000","last_undegraded":"2023-11-26T07:18:51.648265+0000","last_fullsized":"2023-11-26T07:18:51.648265+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:18:50.617605+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:18:50.617605+0000","last_clean_scrub_stamp":"2023-11-26T07:18:50.617605+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T14:40:15.233288+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.5","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2023-11-26T07:18:51.641452+0000","last_change":"2023-11-26T07:18:51.641452+0000","last_active":"2023-11-26T07:18:51.625829+0000","last_peered":"2023-11-26T07:18:51.625829+0000","last_clean":"2023-11-26T07:18:51.625829+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-26T07:18:51.641452+0000","last_undegraded":"2023-11-26T07:18:51.641452+0000","last_fullsized":"2023-11-26T07:18:51.641452+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:18:51.625829+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:18:51.625829+0000","last_clean_scrub_stamp":"2023-11-26T07:18:51.625829+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.4","version":"0'0","reported_seq":9,"reported_epoch":31,"state":"active+clean","last_fresh":"2023-11-26T07:18:51.664083+0000","last_change":"2023-11-26T07:18:51.660079+0000","last_active":"2023-11-26T07:18:51.664083+0000","last_peered":"2023-11-26T07:18:51.664083+0000","last_clean":"2023-11-26T07:18:51.664083+0000","last_became_active":"2023-11-26T07:18:51.659661+0000","last_became_peered":"2023-11-26T07:18:51.659661+0000","last_unstale":"2023-11-26T07:18:51.664083+0000","last_undegraded":"2023-11-26T07:18:51.664083+0000","last_fullsized":"2023-11-26T07:18:51.664083+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:18:50.617605+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:18:50.617605+0000","last_clean_scrub_stamp":"2023-11-26T07:18:50.617605+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T12:04:39.473159+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.2","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2023-11-26T07:18:51.642362+0000","last_change":"2023-11-26T07:18:51.642362+0000","last_active":"2023-11-26T07:18:51.625829+0000","last_peered":"2023-11-26T07:18:51.625829+0000","last_clean":"2023-11-26T07:18:51.625829+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-26T07:18:51.642362+0000","last_undegraded":"2023-11-26T07:18:51.642362+0000","last_fullsized":"2023-11-26T07:18:51.642362+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:18:51.625829+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:18:51.625829+0000","last_clean_scrub_stamp":"2023-11-26T07:18:51.625829+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.3","version":"0'0","reported_seq":9,"reported_epoch":31,"state":"active+clean","last_fresh":"2023-11-26T07:18:51.655830+0000","last_change":"2023-11-26T07:18:51.647332+0000","last_active":"2023-11-26T07:18:51.655830+0000","last_peered":"2023-11-26T07:18:51.655830+0000","last_clean":"2023-11-26T07:18:51.655830+0000","last_became_active":"2023-11-26T07:18:51.647054+0000","last_became_peered":"2023-11-26T07:18:51.647054+0000","last_unstale":"2023-11-26T07:18:51.655830+0000","last_undegraded":"2023-11-26T07:18:51.655830+0000","last_fullsized":"2023-11-26T07:18:51.655830+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:18:50.617605+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:18:50.617605+0000","last_clean_scrub_stamp":"2023-11-26T07:18:50.617605+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T19:10:39.237269+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.3","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2023-11-26T07:18:51.642003+0000","last_change":"2023-11-26T07:18:51.642003+0000","last_active":"2023-11-26T07:18:51.625829+0000","last_peered":"2023-11-26T07:18:51.625829+0000","last_clean":"2023-11-26T07:18:51.625829+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-26T07:18:51.642003+0000","last_undegraded":"2023-11-26T07:18:51.642003+0000","last_fullsized":"2023-11-26T07:18:51.642003+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:18:51.625829+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:18:51.625829+0000","last_clean_scrub_stamp":"2023-11-26T07:18:51.625829+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.2","version":"0'0","reported_seq":9,"reported_epoch":31,"state":"active+clean","last_fresh":"2023-11-26T07:18:51.658369+0000","last_change":"2023-11-26T07:18:51.651994+0000","last_active":"2023-11-26T07:18:51.658369+0000","last_peered":"2023-11-26T07:18:51.658369+0000","last_clean":"2023-11-26T07:18:51.658369+0000","last_became_active":"2023-11-26T07:18:51.650255+0000","last_became_peered":"2023-11-26T07:18:51.650255+0000","last_unstale":"2023-11-26T07:18:51.658369+0000","last_undegraded":"2023-11-26T07:18:51.658369+0000","last_fullsized":"2023-11-26T07:18:51.658369+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:18:50.617605+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:18:50.617605+0000","last_clean_scrub_stamp":"2023-11-26T07:18:50.617605+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T17:27:24.322165+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.0","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2023-11-26T07:18:51.642260+0000","last_change":"2023-11-26T07:18:51.642260+0000","last_active":"2023-11-26T07:18:51.625829+0000","last_peered":"2023-11-26T07:18:51.625829+0000","last_clean":"2023-11-26T07:18:51.625829+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-26T07:18:51.642260+0000","last_undegraded":"2023-11-26T07:18:51.642260+0000","last_fullsized":"2023-11-26T07:18:51.642260+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:18:51.625829+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:18:51.625829+0000","last_clean_scrub_stamp":"2023-11-26T07:18:51.625829+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1","version":"0'0","reported_seq":9,"reported_epoch":31,"state":"active+clean","last_fresh":"2023-11-26T07:18:51.658380+0000","last_change":"2023-11-26T07:18:51.654435+0000","last_active":"2023-11-26T07:18:51.658380+0000","last_peered":"2023-11-26T07:18:51.658380+0000","last_clean":"2023-11-26T07:18:51.658380+0000","last_became_active":"2023-11-26T07:18:51.654107+0000","last_became_peered":"2023-11-26T07:18:51.654107+0000","last_unstale":"2023-11-26T07:18:51.658380+0000","last_undegraded":"2023-11-26T07:18:51.658380+0000","last_fullsized":"2023-11-26T07:18:51.658380+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:18:50.617605+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:18:50.617605+0000","last_clean_scrub_stamp":"2023-11-26T07:18:50.617605+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T15:50:37.023527+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.1","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2023-11-26T07:18:51.635350+0000","last_change":"2023-11-26T07:18:51.635350+0000","last_active":"2023-11-26T07:18:51.625829+0000","last_peered":"2023-11-26T07:18:51.625829+0000","last_clean":"2023-11-26T07:18:51.625829+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-26T07:18:51.635350+0000","last_undegraded":"2023-11-26T07:18:51.635350+0000","last_fullsized":"2023-11-26T07:18:51.635350+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:18:51.625829+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:18:51.625829+0000","last_clean_scrub_stamp":"2023-11-26T07:18:51.625829+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.0","version":"0'0","reported_seq":9,"reported_epoch":31,"state":"active+clean","last_fresh":"2023-11-26T07:18:51.661597+0000","last_change":"2023-11-26T07:18:51.655721+0000","last_active":"2023-11-26T07:18:51.661597+0000","last_peered":"2023-11-26T07:18:51.661597+0000","last_clean":"2023-11-26T07:18:51.661597+0000","last_became_active":"2023-11-26T07:18:51.655334+0000","last_became_peered":"2023-11-26T07:18:51.655334+0000","last_unstale":"2023-11-26T07:18:51.661597+0000","last_undegraded":"2023-11-26T07:18:51.661597+0000","last_fullsized":"2023-11-26T07:18:51.661597+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:18:50.617605+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:18:50.617605+0000","last_clean_scrub_stamp":"2023-11-26T07:18:50.617605+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T11:49:26.615849+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.4","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2023-11-26T07:18:51.643304+0000","last_change":"2023-11-26T07:18:51.643304+0000","last_active":"2023-11-26T07:18:51.625829+0000","last_peered":"2023-11-26T07:18:51.625829+0000","last_clean":"2023-11-26T07:18:51.625829+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-26T07:18:51.643304+0000","last_undegraded":"2023-11-26T07:18:51.643304+0000","last_fullsized":"2023-11-26T07:18:51.643304+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:18:51.625829+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:18:51.625829+0000","last_clean_scrub_stamp":"2023-11-26T07:18:51.625829+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.5","version":"0'0","reported_seq":9,"reported_epoch":31,"state":"active+clean","last_fresh":"2023-11-26T07:18:51.664594+0000","last_change":"2023-11-26T07:18:51.661135+0000","last_active":"2023-11-26T07:18:51.664594+0000","last_peered":"2023-11-26T07:18:51.664594+0000","last_clean":"2023-11-26T07:18:51.664594+0000","last_became_active":"2023-11-26T07:18:51.659814+0000","last_became_peered":"2023-11-26T07:18:51.659814+0000","last_unstale":"2023-11-26T07:18:51.664594+0000","last_undegraded":"2023-11-26T07:18:51.664594+0000","last_fullsized":"2023-11-26T07:18:51.664594+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:18:50.617605+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:18:50.617605+0000","last_clean_scrub_stamp":"2023-11-26T07:18:50.617605+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T17:37:35.256892+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"1.0","version":"15'117","reported_seq":173,"reported_epoch":30,"state":"active+clean","last_fresh":"2023-11-26T07:18:50.636811+0000","last_change":"2023-11-26T07:17:43.827802+0000","last_active":"2023-11-26T07:18:50.636811+0000","last_peered":"2023-11-26T07:18:50.636811+0000","last_clean":"2023-11-26T07:18:50.636811+0000","last_became_active":"2023-11-26T07:17:43.827308+0000","last_became_peered":"2023-11-26T07:17:43.827308+0000","last_unstale":"2023-11-26T07:18:50.636811+0000","last_undegraded":"2023-11-26T07:18:50.636811+0000","last_fullsized":"2023-11-26T07:18:50.636811+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:17:42.160945+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:17:42.160945+0000","last_clean_scrub_stamp":"2023-11-26T07:17:42.160945+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T08:17:54.167366+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":5,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":4,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574864,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27932,"kb_used_data":908,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343908,"statfs":{"total":96636764160,"available":96608161792,"internally_reserved":0,"allocated":929792,"data_stored":717618,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Nov 26 07:18:42 2023","interfaces":[{"interface":"back","average":{"1min":0.71499999999999997,"5min":0.71499999999999997,"15min":0.71499999999999997},"min":{"1min":0.54300000000000004,"5min":0.54300000000000004,"15min":0.54300000000000004},"max":{"1min":0.91500000000000004,"5min":0.91500000000000004,"15min":0.91500000000000004},"last":0.60099999999999998},{"interface":"front","average":{"1min":0.70899999999999996,"5min":0.70899999999999996,"15min":0.70899999999999996},"min":{"1min":0.52900000000000003,"5min":0.52900000000000003,"15min":0.52900000000000003},"max":{"1min":1.0980000000000001,"5min":1.0980000000000001,"15min":1.0980000000000001},"last":0.69199999999999995}]},{"osd":1,"last update":"Sun Nov 26 07:18:42 2023","interfaces":[{"interface":"back","average":{"1min":0.68200000000000005,"5min":0.68200000000000005,"15min":0.68200000000000005},"min":{"1min":0.47999999999999998,"5min":0.47999999999999998,"15min":0.47999999999999998},"max":{"1min":1.0669999999999999,"5min":1.0669999999999999,"15min":1.0669999999999999},"last":0.67300000000000004},{"interface":"front","average":{"1min":0.83899999999999997,"5min":0.83899999999999997,"15min":0.83899999999999997},"min":{"1min":0.47599999999999998,"5min":0.47599999999999998,"15min":0.47599999999999998},"max":{"1min":4.5309999999999997,"5min":4.5309999999999997,"15min":4.5309999999999997},"last":0.627}]},{"osd":2,"last update":"Sun Nov 26 07:18:42 2023","interfaces":[{"interface":"back","average":{"1min":0.67600000000000005,"5min":0.67600000000000005,"15min":0.67600000000000005},"min":{"1min":0.52000000000000002,"5min":0.52000000000000002,"15min":0.52000000000000002},"max":{"1min":0.873,"5min":0.873,"15min":0.873},"last":0.54400000000000004},{"interface":"front","average":{"1min":0.71299999999999997,"5min":0.71299999999999997,"15min":0.71299999999999997},"min":{"1min":0.54700000000000004,"5min":0.54700000000000004,"15min":0.54700000000000004},"max":{"1min":1.1339999999999999,"5min":1.1339999999999999,"15min":1.1339999999999999},"last":0.499}]},{"osd":3,"last update":"Sun Nov 26 07:18:42 2023","interfaces":[{"interface":"back","average":{"1min":0.69699999999999995,"5min":0.69699999999999995,"15min":0.69699999999999995},"min":{"1min":0.51800000000000002,"5min":0.51800000000000002,"15min":0.51800000000000002},"max":{"1min":1.026,"5min":1.026,"15min":1.026},"last":0.64300000000000002},{"interface":"front","average":{"1min":0.73299999999999998,"5min":0.73299999999999998,"15min":0.73299999999999998},"min":{"1min":0.50600000000000001,"5min":0.50600000000000001,"15min":0.50600000000000001},"max":{"1min":1.4490000000000001,"5min":1.4490000000000001,"15min":1.4490000000000001},"last":0.755}]},{"osd":4,"last update":"Sun Nov 26 07:18:42 2023","interfaces":[{"interface":"back","average":{"1min":0.59999999999999998,"5min":0.59999999999999998,"15min":0.59999999999999998},"min":{"1min":0.39400000000000002,"5min":0.39400000000000002,"15min":0.39400000000000002},"max":{"1min":0.94799999999999995,"5min":0.94799999999999995,"15min":0.94799999999999995},"last":0.47099999999999997},{"interface":"front","average":{"1min":0.67300000000000004,"5min":0.67300000000000004,"15min":0.67300000000000004},"min":{"1min":0.436,"5min":0.436,"15min":0.436},"max":{"1min":1.5509999999999999,"5min":1.5509999999999999,"15min":1.5509999999999999},"last":0.70899999999999996}]},{"osd":5,"last update":"Sun Nov 26 07:18:42 2023","interfaces":[{"interface":"back","average":{"1min":0.66600000000000004,"5min":0.66600000000000004,"15min":0.66600000000000004},"min":{"1min":0.443,"5min":0.443,"15min":0.443},"max":{"1min":1.415,"5min":1.415,"15min":1.415},"last":0.72599999999999998},{"interface":"front","average":{"1min":0.71399999999999997,"5min":0.71399999999999997,"15min":0.71399999999999997},"min":{"1min":0.53100000000000003,"5min":0.53100000000000003,"15min":0.53100000000000003},"max":{"1min":1.3859999999999999,"5min":1.3859999999999999,"15min":1.3859999999999999},"last":0.52800000000000002}]},{"osd":6,"last update":"Sun Nov 26 07:18:42 2023","interfaces":[{"interface":"back","average":{"1min":0.77100000000000002,"5min":0.77100000000000002,"15min":0.77100000000000002},"min":{"1min":0.64100000000000001,"5min":0.64100000000000001,"15min":0.64100000000000001},"max":{"1min":1.3069999999999999,"5min":1.3069999999999999,"15min":1.3069999999999999},"last":0.58399999999999996},{"interface":"front","average":{"1min":0.77100000000000002,"5min":0.77100000000000002,"15min":0.77100000000000002},"min":{"1min":0.57399999999999995,"5min":0.57399999999999995,"15min":0.57399999999999995},"max":{"1min":1.6759999999999999,"5min":1.6759999999999999,"15min":1.6759999999999999},"last":0.74299999999999999}]}]},{"osd":6,"up_from":13,"seq":55834574866,"num_pgs":3,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27348,"kb_used_data":328,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344492,"statfs":{"total":96636764160,"available":96608759808,"internally_reserved":0,"allocated":335872,"data_stored":127250,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Nov 26 07:18:45 2023","interfaces":[{"interface":"back","average":{"1min":0.68400000000000005,"5min":0.68400000000000005,"15min":0.68400000000000005},"min":{"1min":0.36699999999999999,"5min":0.36699999999999999,"15min":0.36699999999999999},"max":{"1min":1.7549999999999999,"5min":1.7549999999999999,"15min":1.7549999999999999},"last":0.64900000000000002},{"interface":"front","average":{"1min":0.95799999999999996,"5min":0.95799999999999996,"15min":0.95799999999999996},"min":{"1min":0.38300000000000001,"5min":0.38300000000000001,"15min":0.38300000000000001},"max":{"1min":5.7489999999999997,"5min":5.7489999999999997,"15min":5.7489999999999997},"last":0.56000000000000005}]},{"osd":1,"last update":"Sun Nov 26 07:18:45 2023","interfaces":[{"interface":"back","average":{"1min":0.91400000000000003,"5min":0.91400000000000003,"15min":0.91400000000000003},"min":{"1min":0.42099999999999999,"5min":0.42099999999999999,"15min":0.42099999999999999},"max":{"1min":5.4240000000000004,"5min":5.4240000000000004,"15min":5.4240000000000004},"last":0.58199999999999996},{"interface":"front","average":{"1min":0.89700000000000002,"5min":0.89700000000000002,"15min":0.89700000000000002},"min":{"1min":0.499,"5min":0.499,"15min":0.499},"max":{"1min":5.6020000000000003,"5min":5.6020000000000003,"15min":5.6020000000000003},"last":0.75}]},{"osd":2,"last update":"Sun Nov 26 07:18:45 2023","interfaces":[{"interface":"back","average":{"1min":0.69699999999999995,"5min":0.69699999999999995,"15min":0.69699999999999995},"min":{"1min":0.47599999999999998,"5min":0.47599999999999998,"15min":0.47599999999999998},"max":{"1min":1.8520000000000001,"5min":1.8520000000000001,"15min":1.8520000000000001},"last":0.60599999999999998},{"interface":"front","average":{"1min":0.73399999999999999,"5min":0.73399999999999999,"15min":0.73399999999999999},"min":{"1min":0.46700000000000003,"5min":0.46700000000000003,"15min":0.46700000000000003},"max":{"1min":1.9159999999999999,"5min":1.9159999999999999,"15min":1.9159999999999999},"last":0.51700000000000002}]},{"osd":3,"last update":"Sun Nov 26 07:18:45 2023","interfaces":[{"interface":"back","average":{"1min":0.90200000000000002,"5min":0.90200000000000002,"15min":0.90200000000000002},"min":{"1min":0.502,"5min":0.502,"15min":0.502},"max":{"1min":5.4390000000000001,"5min":5.4390000000000001,"15min":5.4390000000000001},"last":0.63400000000000001},{"interface":"front","average":{"1min":0.94499999999999995,"5min":0.94499999999999995,"15min":0.94499999999999995},"min":{"1min":0.54400000000000004,"5min":0.54400000000000004,"15min":0.54400000000000004},"max":{"1min":5.5490000000000004,"5min":5.5490000000000004,"15min":5.5490000000000004},"last":0.72799999999999998}]},{"osd":4,"last update":"Sun Nov 26 07:18:45 2023","interfaces":[{"interface":"back","average":{"1min":0.64200000000000002,"5min":0.64200000000000002,"15min":0.64200000000000002},"min":{"1min":0.46899999999999997,"5min":0.46899999999999997,"15min":0.46899999999999997},"max":{"1min":0.85099999999999998,"5min":0.85099999999999998,"15min":0.85099999999999998},"last":0.45500000000000002},{"interface":"front","average":{"1min":0.67500000000000004,"5min":0.67500000000000004,"15min":0.67500000000000004},"min":{"1min":0.40699999999999997,"5min":0.40699999999999997,"15min":0.40699999999999997},"max":{"1min":0.93300000000000005,"5min":0.93300000000000005,"15min":0.93300000000000005},"last":0.68700000000000006}]},{"osd":5,"last update":"Sun Nov 26 07:18:45 2023","interfaces":[{"interface":"back","average":{"1min":0.65900000000000003,"5min":0.65900000000000003,"15min":0.65900000000000003},"min":{"1min":0.42499999999999999,"5min":0.42499999999999999,"15min":0.42499999999999999},"max":{"1min":0.79300000000000004,"5min":0.79300000000000004,"15min":0.79300000000000004},"last":0.53400000000000003},{"interface":"front","average":{"1min":0.71299999999999997,"5min":0.71299999999999997,"15min":0.71299999999999997},"min":{"1min":0.436,"5min":0.436,"15min":0.436},"max":{"1min":0.93000000000000005,"5min":0.93000000000000005,"15min":0.93000000000000005},"last":0.70599999999999996}]},{"osd":7,"last update":"Sun Nov 26 07:18:45 2023","interfaces":[{"interface":"back","average":{"1min":0.74199999999999999,"5min":0.74199999999999999,"15min":0.74199999999999999},"min":{"1min":0.46600000000000003,"5min":0.46600000000000003,"15min":0.46600000000000003},"max":{"1min":0.998,"5min":0.998,"15min":0.998},"last":0.78200000000000003},{"interface":"front","average":{"1min":0.71599999999999997,"5min":0.71599999999999997,"15min":0.71599999999999997},"min":{"1min":0.498,"5min":0.498,"15min":0.498},"max":{"1min":0.86499999999999999,"5min":0.86499999999999999,"15min":0.86499999999999999},"last":0.66600000000000004}]}]},{"osd":5,"up_from":13,"seq":55834574864,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27360,"kb_used_data":340,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344480,"statfs":{"total":96636764160,"available":96608747520,"internally_reserved":0,"allocated":348160,"data_stored":133517,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Nov 26 07:18:45 2023","interfaces":[{"interface":"back","average":{"1min":0.66600000000000004,"5min":0.66600000000000004,"15min":0.66600000000000004},"min":{"1min":0.38100000000000001,"5min":0.38100000000000001,"15min":0.38100000000000001},"max":{"1min":1.0289999999999999,"5min":1.0289999999999999,"15min":1.0289999999999999},"last":0.59099999999999997},{"interface":"front","average":{"1min":0.66000000000000003,"5min":0.66000000000000003,"15min":0.66000000000000003},"min":{"1min":0.438,"5min":0.438,"15min":0.438},"max":{"1min":1.05,"5min":1.05,"15min":1.05},"last":0.54700000000000004}]},{"osd":1,"last update":"Sun Nov 26 07:18:45 2023","interfaces":[{"interface":"back","average":{"1min":0.66800000000000004,"5min":0.66800000000000004,"15min":0.66800000000000004},"min":{"1min":0.41599999999999998,"5min":0.41599999999999998,"15min":0.41599999999999998},"max":{"1min":0.86699999999999999,"5min":0.86699999999999999,"15min":0.86699999999999999},"last":0.79400000000000004},{"interface":"front","average":{"1min":0.68600000000000005,"5min":0.68600000000000005,"15min":0.68600000000000005},"min":{"1min":0.49199999999999999,"5min":0.49199999999999999,"15min":0.49199999999999999},"max":{"1min":0.98899999999999999,"5min":0.98899999999999999,"15min":0.98899999999999999},"last":0.52900000000000003}]},{"osd":2,"last update":"Sun Nov 26 07:18:45 2023","interfaces":[{"interface":"back","average":{"1min":0.67500000000000004,"5min":0.67500000000000004,"15min":0.67500000000000004},"min":{"1min":0.45900000000000002,"5min":0.45900000000000002,"15min":0.45900000000000002},"max":{"1min":0.89800000000000002,"5min":0.89800000000000002,"15min":0.89800000000000002},"last":0.67500000000000004},{"interface":"front","average":{"1min":0.67000000000000004,"5min":0.67000000000000004,"15min":0.67000000000000004},"min":{"1min":0.51200000000000001,"5min":0.51200000000000001,"15min":0.51200000000000001},"max":{"1min":0.89700000000000002,"5min":0.89700000000000002,"15min":0.89700000000000002},"last":0.73399999999999999}]},{"osd":3,"last update":"Sun Nov 26 07:18:45 2023","interfaces":[{"interface":"back","average":{"1min":0.71999999999999997,"5min":0.71999999999999997,"15min":0.71999999999999997},"min":{"1min":0.43099999999999999,"5min":0.43099999999999999,"15min":0.43099999999999999},"max":{"1min":1.0029999999999999,"5min":1.0029999999999999,"15min":1.0029999999999999},"last":0.629},{"interface":"front","average":{"1min":0.67500000000000004,"5min":0.67500000000000004,"15min":0.67500000000000004},"min":{"1min":0.44600000000000001,"5min":0.44600000000000001,"15min":0.44600000000000001},"max":{"1min":1.0209999999999999,"5min":1.0209999999999999,"15min":1.0209999999999999},"last":0.48199999999999998}]},{"osd":4,"last update":"Sun Nov 26 07:18:45 2023","interfaces":[{"interface":"back","average":{"1min":0.63500000000000001,"5min":0.63500000000000001,"15min":0.63500000000000001},"min":{"1min":0.39200000000000002,"5min":0.39200000000000002,"15min":0.39200000000000002},"max":{"1min":0.95099999999999996,"5min":0.95099999999999996,"15min":0.95099999999999996},"last":0.70199999999999996},{"interface":"front","average":{"1min":0.70899999999999996,"5min":0.70899999999999996,"15min":0.70899999999999996},"min":{"1min":0.41299999999999998,"5min":0.41299999999999998,"15min":0.41299999999999998},"max":{"1min":1.254,"5min":1.254,"15min":1.254},"last":0.75900000000000001}]},{"osd":6,"last update":"Sun Nov 26 07:18:45 2023","interfaces":[{"interface":"back","average":{"1min":0.67200000000000004,"5min":0.67200000000000004,"15min":0.67200000000000004},"min":{"1min":0.45600000000000002,"5min":0.45600000000000002,"15min":0.45600000000000002},"max":{"1min":0.94499999999999995,"5min":0.94499999999999995,"15min":0.94499999999999995},"last":0.80900000000000005},{"interface":"front","average":{"1min":0.72699999999999998,"5min":0.72699999999999998,"15min":0.72699999999999998},"min":{"1min":0.44,"5min":0.44,"15min":0.44},"max":{"1min":1.0649999999999999,"5min":1.0649999999999999,"15min":1.0649999999999999},"last":0.77700000000000002}]},{"osd":7,"last update":"Sun Nov 26 07:18:45 2023","interfaces":[{"interface":"back","average":{"1min":0.80400000000000005,"5min":0.80400000000000005,"15min":0.80400000000000005},"min":{"1min":0.48199999999999998,"5min":0.48199999999999998,"15min":0.48199999999999998},"max":{"1min":1.4079999999999999,"5min":1.4079999999999999,"15min":1.4079999999999999},"last":0.85399999999999998},{"interface":"front","average":{"1min":0.77500000000000002,"5min":0.77500000000000002,"15min":0.77500000000000002},"min":{"1min":0.51600000000000001,"5min":0.51600000000000001,"15min":0.51600000000000001},"max":{"1min":1.0169999999999999,"5min":1.0169999999999999,"15min":1.0169999999999999},"last":0.82499999999999996}]}]},{"osd":4,"up_from":13,"seq":55834574866,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27348,"kb_used_data":328,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344492,"statfs":{"total":96636764160,"available":96608759808,"internally_reserved":0,"allocated":335872,"data_stored":127250,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Nov 26 07:18:46 2023","interfaces":[{"interface":"back","average":{"1min":0.79400000000000004,"5min":0.79400000000000004,"15min":0.79400000000000004},"min":{"1min":0.45600000000000002,"5min":0.45600000000000002,"15min":0.45600000000000002},"max":{"1min":4.1689999999999996,"5min":4.1689999999999996,"15min":4.1689999999999996},"last":0.58599999999999997},{"interface":"front","average":{"1min":1.002,"5min":1.002,"15min":1.002},"min":{"1min":0.47499999999999998,"5min":0.47499999999999998,"15min":0.47499999999999998},"max":{"1min":7.7889999999999997,"5min":7.7889999999999997,"15min":7.7889999999999997},"last":0.54200000000000004}]},{"osd":1,"last update":"Sun Nov 26 07:18:46 2023","interfaces":[{"interface":"back","average":{"1min":0.64700000000000002,"5min":0.64700000000000002,"15min":0.64700000000000002},"min":{"1min":0.52300000000000002,"5min":0.52300000000000002,"15min":0.52300000000000002},"max":{"1min":0.78400000000000003,"5min":0.78400000000000003,"15min":0.78400000000000003},"last":0.63400000000000001},{"interface":"front","average":{"1min":0.81999999999999995,"5min":0.81999999999999995,"15min":0.81999999999999995},"min":{"1min":0.44500000000000001,"5min":0.44500000000000001,"15min":0.44500000000000001},"max":{"1min":4.3140000000000001,"5min":4.3140000000000001,"15min":4.3140000000000001},"last":0.52400000000000002}]},{"osd":2,"last update":"Sun Nov 26 07:18:46 2023","interfaces":[{"interface":"back","average":{"1min":0.80700000000000005,"5min":0.80700000000000005,"15min":0.80700000000000005},"min":{"1min":0.57799999999999996,"5min":0.57799999999999996,"15min":0.57799999999999996},"max":{"1min":2.7850000000000001,"5min":2.7850000000000001,"15min":2.7850000000000001},"last":0.76100000000000001},{"interface":"front","average":{"1min":1.0660000000000001,"5min":1.0660000000000001,"15min":1.0660000000000001},"min":{"1min":0.53400000000000003,"5min":0.53400000000000003,"15min":0.53400000000000003},"max":{"1min":7.6470000000000002,"5min":7.6470000000000002,"15min":7.6470000000000002},"last":0.68899999999999995}]},{"osd":3,"last update":"Sun Nov 26 07:18:46 2023","interfaces":[{"interface":"back","average":{"1min":0.72999999999999998,"5min":0.72999999999999998,"15min":0.72999999999999998},"min":{"1min":0.48599999999999999,"5min":0.48599999999999999,"15min":0.48599999999999999},"max":{"1min":1.591,"5min":1.591,"15min":1.591},"last":0.45700000000000002},{"interface":"front","average":{"1min":0.879,"5min":0.879,"15min":0.879},"min":{"1min":0.56899999999999995,"5min":0.56899999999999995,"15min":0.56899999999999995},"max":{"1min":3.452,"5min":3.452,"15min":3.452},"last":0.80400000000000005}]},{"osd":5,"last update":"Sun Nov 26 07:18:46 2023","interfaces":[{"interface":"back","average":{"1min":0.61099999999999999,"5min":0.61099999999999999,"15min":0.61099999999999999},"min":{"1min":0.43099999999999999,"5min":0.43099999999999999,"15min":0.43099999999999999},"max":{"1min":0.96199999999999997,"5min":0.96199999999999997,"15min":0.96199999999999997},"last":0.73999999999999999},{"interface":"front","average":{"1min":0.64100000000000001,"5min":0.64100000000000001,"15min":0.64100000000000001},"min":{"1min":0.41499999999999998,"5min":0.41499999999999998,"15min":0.41499999999999998},"max":{"1min":1.538,"5min":1.538,"15min":1.538},"last":0.56799999999999995}]},{"osd":6,"last update":"Sun Nov 26 07:18:46 2023","interfaces":[{"interface":"back","average":{"1min":0.79400000000000004,"5min":0.79400000000000004,"15min":0.79400000000000004},"min":{"1min":0.39400000000000002,"5min":0.39400000000000002,"15min":0.39400000000000002},"max":{"1min":2.8330000000000002,"5min":2.8330000000000002,"15min":2.8330000000000002},"last":0.70999999999999996},{"interface":"front","average":{"1min":1.0089999999999999,"5min":1.0089999999999999,"15min":1.0089999999999999},"min":{"1min":0.42899999999999999,"5min":0.42899999999999999,"15min":0.42899999999999999},"max":{"1min":7.6050000000000004,"5min":7.6050000000000004,"15min":7.6050000000000004},"last":0.78500000000000003}]},{"osd":7,"last update":"Sun Nov 26 07:18:46 2023","interfaces":[{"interface":"back","average":{"1min":0.86099999999999999,"5min":0.86099999999999999,"15min":0.86099999999999999},"min":{"1min":0.44700000000000001,"5min":0.44700000000000001,"15min":0.44700000000000001},"max":{"1min":4.3689999999999998,"5min":4.3689999999999998,"15min":4.3689999999999998},"last":0.61299999999999999},{"interface":"front","average":{"1min":0.89200000000000002,"5min":0.89200000000000002,"15min":0.89200000000000002},"min":{"1min":0.505,"5min":0.505,"15min":0.505},"max":{"1min":3.403,"5min":3.403,"15min":3.403},"last":0.83799999999999997}]}]},{"osd":3,"up_from":13,"seq":55834574867,"num_pgs":7,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27352,"kb_used_data":328,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344488,"statfs":{"total":96636764160,"available":96608755712,"internally_reserved":0,"allocated":335872,"data_stored":127250,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1592,"internal_metadata":27654600},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Nov 26 07:18:43 2023","interfaces":[{"interface":"back","average":{"1min":0.379,"5min":0.379,"15min":0.379},"min":{"1min":0.26100000000000001,"5min":0.26100000000000001,"15min":0.26100000000000001},"max":{"1min":0.58599999999999997,"5min":0.58599999999999997,"15min":0.58599999999999997},"last":0.40000000000000002},{"interface":"front","average":{"1min":0.433,"5min":0.433,"15min":0.433},"min":{"1min":0.30199999999999999,"5min":0.30199999999999999,"15min":0.30199999999999999},"max":{"1min":0.63,"5min":0.63,"15min":0.63},"last":0.26900000000000002}]},{"osd":1,"last update":"Sun Nov 26 07:18:43 2023","interfaces":[{"interface":"back","average":{"1min":0.42099999999999999,"5min":0.42099999999999999,"15min":0.42099999999999999},"min":{"1min":0.312,"5min":0.312,"15min":0.312},"max":{"1min":0.752,"5min":0.752,"15min":0.752},"last":0.28799999999999998},{"interface":"front","average":{"1min":0.48399999999999999,"5min":0.48399999999999999,"15min":0.48399999999999999},"min":{"1min":0.38,"5min":0.38,"15min":0.38},"max":{"1min":0.628,"5min":0.628,"15min":0.628},"last":0.35099999999999998}]},{"osd":2,"last update":"Sun Nov 26 07:18:43 2023","interfaces":[{"interface":"back","average":{"1min":0.53300000000000003,"5min":0.53300000000000003,"15min":0.53300000000000003},"min":{"1min":0.40300000000000002,"5min":0.40300000000000002,"15min":0.40300000000000002},"max":{"1min":0.82199999999999995,"5min":0.82199999999999995,"15min":0.82199999999999995},"last":0.38200000000000001},{"interface":"front","average":{"1min":0.505,"5min":0.505,"15min":0.505},"min":{"1min":0.35999999999999999,"5min":0.35999999999999999,"15min":0.35999999999999999},"max":{"1min":0.72199999999999998,"5min":0.72199999999999998,"15min":0.72199999999999998},"last":0.33000000000000002}]},{"osd":4,"last update":"Sun Nov 26 07:18:43 2023","interfaces":[{"interface":"back","average":{"1min":0.70599999999999996,"5min":0.70599999999999996,"15min":0.70599999999999996},"min":{"1min":0.48199999999999998,"5min":0.48199999999999998,"15min":0.48199999999999998},"max":{"1min":0.96199999999999997,"5min":0.96199999999999997,"15min":0.96199999999999997},"last":0.52200000000000002},{"interface":"front","average":{"1min":0.72799999999999998,"5min":0.72799999999999998,"15min":0.72799999999999998},"min":{"1min":0.441,"5min":0.441,"15min":0.441},"max":{"1min":1.171,"5min":1.171,"15min":1.171},"last":0.59699999999999998}]},{"osd":5,"last update":"Sun Nov 26 07:18:43 2023","interfaces":[{"interface":"back","average":{"1min":0.73899999999999999,"5min":0.73899999999999999,"15min":0.73899999999999999},"min":{"1min":0.53700000000000003,"5min":0.53700000000000003,"15min":0.53700000000000003},"max":{"1min":0.94099999999999995,"5min":0.94099999999999995,"15min":0.94099999999999995},"last":0.53800000000000003},{"interface":"front","average":{"1min":0.73799999999999999,"5min":0.73799999999999999,"15min":0.73799999999999999},"min":{"1min":0.504,"5min":0.504,"15min":0.504},"max":{"1min":1.0820000000000001,"5min":1.0820000000000001,"15min":1.0820000000000001},"last":0.61199999999999999}]},{"osd":6,"last update":"Sun Nov 26 07:18:43 2023","interfaces":[{"interface":"back","average":{"1min":0.81399999999999995,"5min":0.81399999999999995,"15min":0.81399999999999995},"min":{"1min":0.55200000000000005,"5min":0.55200000000000005,"15min":0.55200000000000005},"max":{"1min":1.2729999999999999,"5min":1.2729999999999999,"15min":1.2729999999999999},"last":0.64500000000000002},{"interface":"front","average":{"1min":0.79300000000000004,"5min":0.79300000000000004,"15min":0.79300000000000004},"min":{"1min":0.47299999999999998,"5min":0.47299999999999998,"15min":0.47299999999999998},"max":{"1min":1.204,"5min":1.204,"15min":1.204},"last":0.63200000000000001}]},{"osd":7,"last update":"Sun Nov 26 07:18:43 2023","interfaces":[{"interface":"back","average":{"1min":0.80100000000000005,"5min":0.80100000000000005,"15min":0.80100000000000005},"min":{"1min":0.63300000000000001,"5min":0.63300000000000001,"15min":0.63300000000000001},"max":{"1min":1.3600000000000001,"5min":1.3600000000000001,"15min":1.3600000000000001},"last":0.65500000000000003},{"interface":"front","average":{"1min":0.82599999999999996,"5min":0.82599999999999996,"15min":0.82599999999999996},"min":{"1min":0.50600000000000001,"5min":0.50600000000000001,"15min":0.50600000000000001},"max":{"1min":1.617,"5min":1.617,"15min":1.617},"last":0.67000000000000004}]}]},{"osd":2,"up_from":13,"seq":55834574867,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27360,"kb_used_data":340,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344480,"statfs":{"total":96636764160,"available":96608747520,"internally_reserved":0,"allocated":348160,"data_stored":133517,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Nov 26 07:18:43 2023","interfaces":[{"interface":"back","average":{"1min":0.71399999999999997,"5min":0.71399999999999997,"15min":0.71399999999999997},"min":{"1min":0.28699999999999998,"5min":0.28699999999999998,"15min":0.28699999999999998},"max":{"1min":4.7030000000000003,"5min":4.7030000000000003,"15min":4.7030000000000003},"last":0.32500000000000001},{"interface":"front","average":{"1min":0.56200000000000006,"5min":0.56200000000000006,"15min":0.56200000000000006},"min":{"1min":0.32300000000000001,"5min":0.32300000000000001,"15min":0.32300000000000001},"max":{"1min":2.4340000000000002,"5min":2.4340000000000002,"15min":2.4340000000000002},"last":0.49099999999999999}]},{"osd":1,"last update":"Sun Nov 26 07:18:43 2023","interfaces":[{"interface":"back","average":{"1min":0.86399999999999999,"5min":0.86399999999999999,"15min":0.86399999999999999},"min":{"1min":0.34599999999999997,"5min":0.34599999999999997,"15min":0.34599999999999997},"max":{"1min":4.2690000000000001,"5min":4.2690000000000001,"15min":4.2690000000000001},"last":0.42499999999999999},{"interface":"front","average":{"1min":0.74199999999999999,"5min":0.74199999999999999,"15min":0.74199999999999999},"min":{"1min":0.31900000000000001,"5min":0.31900000000000001,"15min":0.31900000000000001},"max":{"1min":3.9940000000000002,"5min":3.9940000000000002,"15min":3.9940000000000002},"last":0.35099999999999998}]},{"osd":3,"last update":"Sun Nov 26 07:18:43 2023","interfaces":[{"interface":"back","average":{"1min":0.64300000000000002,"5min":0.64300000000000002,"15min":0.64300000000000002},"min":{"1min":0.34399999999999997,"5min":0.34399999999999997,"15min":0.34399999999999997},"max":{"1min":4.0300000000000002,"5min":4.0300000000000002,"15min":4.0300000000000002},"last":0.38700000000000001},{"interface":"front","average":{"1min":0.91400000000000003,"5min":0.91400000000000003,"15min":0.91400000000000003},"min":{"1min":0.41699999999999998,"5min":0.41699999999999998,"15min":0.41699999999999998},"max":{"1min":4.5810000000000004,"5min":4.5810000000000004,"15min":4.5810000000000004},"last":0.46500000000000002}]},{"osd":4,"last update":"Sun Nov 26 07:18:43 2023","interfaces":[{"interface":"back","average":{"1min":0.98499999999999999,"5min":0.98499999999999999,"15min":0.98499999999999999},"min":{"1min":0.55100000000000005,"5min":0.55100000000000005,"15min":0.55100000000000005},"max":{"1min":4.4269999999999996,"5min":4.4269999999999996,"15min":4.4269999999999996},"last":1.01},{"interface":"front","average":{"1min":1.0149999999999999,"5min":1.0149999999999999,"15min":1.0149999999999999},"min":{"1min":0.60299999999999998,"5min":0.60299999999999998,"15min":0.60299999999999998},"max":{"1min":4.085,"5min":4.085,"15min":4.085},"last":1.0600000000000001}]},{"osd":5,"last update":"Sun Nov 26 07:18:43 2023","interfaces":[{"interface":"back","average":{"1min":0.874,"5min":0.874,"15min":0.874},"min":{"1min":0.52900000000000003,"5min":0.52900000000000003,"15min":0.52900000000000003},"max":{"1min":3.9590000000000001,"5min":3.9590000000000001,"15min":3.9590000000000001},"last":0.81000000000000005},{"interface":"front","average":{"1min":0.98799999999999999,"5min":0.98799999999999999,"15min":0.98799999999999999},"min":{"1min":0.58799999999999997,"5min":0.58799999999999997,"15min":0.58799999999999997},"max":{"1min":4.3849999999999998,"5min":4.3849999999999998,"15min":4.3849999999999998},"last":0.86299999999999999}]},{"osd":6,"last update":"Sun Nov 26 07:18:43 2023","interfaces":[{"interface":"back","average":{"1min":0.996,"5min":0.996,"15min":0.996},"min":{"1min":0.56599999999999995,"5min":0.56599999999999995,"15min":0.56599999999999995},"max":{"1min":4.0449999999999999,"5min":4.0449999999999999,"15min":4.0449999999999999},"last":0.91000000000000003},{"interface":"front","average":{"1min":0.91800000000000004,"5min":0.91800000000000004,"15min":0.91800000000000004},"min":{"1min":0.57099999999999995,"5min":0.57099999999999995,"15min":0.57099999999999995},"max":{"1min":3.992,"5min":3.992,"15min":3.992},"last":1.073}]},{"osd":7,"last update":"Sun Nov 26 07:18:43 2023","interfaces":[{"interface":"back","average":{"1min":1.0349999999999999,"5min":1.0349999999999999,"15min":1.0349999999999999},"min":{"1min":0.63100000000000001,"5min":0.63100000000000001,"15min":0.63100000000000001},"max":{"1min":4.1219999999999999,"5min":4.1219999999999999,"15min":4.1219999999999999},"last":0.69699999999999995},{"interface":"front","average":{"1min":0.93799999999999994,"5min":0.93799999999999994,"15min":0.93799999999999994},"min":{"1min":0.57699999999999996,"5min":0.57699999999999996,"15min":0.57699999999999996},"max":{"1min":3.9159999999999999,"5min":3.9159999999999999,"15min":3.9159999999999999},"last":0.95199999999999996}]}]},{"osd":1,"up_from":13,"seq":55834574866,"num_pgs":5,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27352,"kb_used_data":328,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344488,"statfs":{"total":96636764160,"available":96608755712,"internally_reserved":0,"allocated":335872,"data_stored":127250,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Nov 26 07:18:41 2023","interfaces":[{"interface":"back","average":{"1min":0.41699999999999998,"5min":0.41699999999999998,"15min":0.41699999999999998},"min":{"1min":0.26300000000000001,"5min":0.26300000000000001,"15min":0.26300000000000001},"max":{"1min":0.61199999999999999,"5min":0.61199999999999999,"15min":0.61199999999999999},"last":0.29499999999999998},{"interface":"front","average":{"1min":0.48599999999999999,"5min":0.48599999999999999,"15min":0.48599999999999999},"min":{"1min":0.30499999999999999,"5min":0.30499999999999999,"15min":0.30499999999999999},"max":{"1min":0.78700000000000003,"5min":0.78700000000000003,"15min":0.78700000000000003},"last":0.34699999999999998}]},{"osd":2,"last update":"Sun Nov 26 07:18:41 2023","interfaces":[{"interface":"back","average":{"1min":0.45500000000000002,"5min":0.45500000000000002,"15min":0.45500000000000002},"min":{"1min":0.33400000000000002,"5min":0.33400000000000002,"15min":0.33400000000000002},"max":{"1min":0.67600000000000005,"5min":0.67600000000000005,"15min":0.67600000000000005},"last":0.39200000000000002},{"interface":"front","average":{"1min":0.50800000000000001,"5min":0.50800000000000001,"15min":0.50800000000000001},"min":{"1min":0.32300000000000001,"5min":0.32300000000000001,"15min":0.32300000000000001},"max":{"1min":0.66500000000000004,"5min":0.66500000000000004,"15min":0.66500000000000004},"last":0.40699999999999997}]},{"osd":3,"last update":"Sun Nov 26 07:18:41 2023","interfaces":[{"interface":"back","average":{"1min":0.53000000000000003,"5min":0.53000000000000003,"15min":0.53000000000000003},"min":{"1min":0.38400000000000001,"5min":0.38400000000000001,"15min":0.38400000000000001},"max":{"1min":0.73699999999999999,"5min":0.73699999999999999,"15min":0.73699999999999999},"last":0.48499999999999999},{"interface":"front","average":{"1min":0.497,"5min":0.497,"15min":0.497},"min":{"1min":0.36199999999999999,"5min":0.36199999999999999,"15min":0.36199999999999999},"max":{"1min":0.80700000000000005,"5min":0.80700000000000005,"15min":0.80700000000000005},"last":0.432}]},{"osd":4,"last update":"Sun Nov 26 07:18:41 2023","interfaces":[{"interface":"back","average":{"1min":0.69399999999999995,"5min":0.69399999999999995,"15min":0.69399999999999995},"min":{"1min":0.48599999999999999,"5min":0.48599999999999999,"15min":0.48599999999999999},"max":{"1min":1.0229999999999999,"5min":1.0229999999999999,"15min":1.0229999999999999},"last":0.54400000000000004},{"interface":"front","average":{"1min":0.68000000000000005,"5min":0.68000000000000005,"15min":0.68000000000000005},"min":{"1min":0.49199999999999999,"5min":0.49199999999999999,"15min":0.49199999999999999},"max":{"1min":1.0629999999999999,"5min":1.0629999999999999,"15min":1.0629999999999999},"last":0.58399999999999996}]},{"osd":5,"last update":"Sun Nov 26 07:18:41 2023","interfaces":[{"interface":"back","average":{"1min":0.71899999999999997,"5min":0.71899999999999997,"15min":0.71899999999999997},"min":{"1min":0.49299999999999999,"5min":0.49299999999999999,"15min":0.49299999999999999},"max":{"1min":1.0880000000000001,"5min":1.0880000000000001,"15min":1.0880000000000001},"last":0.60399999999999998},{"interface":"front","average":{"1min":0.71699999999999997,"5min":0.71699999999999997,"15min":0.71699999999999997},"min":{"1min":0.53500000000000003,"5min":0.53500000000000003,"15min":0.53500000000000003},"max":{"1min":1.014,"5min":1.014,"15min":1.014},"last":0.68500000000000005}]},{"osd":6,"last update":"Sun Nov 26 07:18:41 2023","interfaces":[{"interface":"back","average":{"1min":0.70399999999999996,"5min":0.70399999999999996,"15min":0.70399999999999996},"min":{"1min":0.50600000000000001,"5min":0.50600000000000001,"15min":0.50600000000000001},"max":{"1min":0.97599999999999998,"5min":0.97599999999999998,"15min":0.97599999999999998},"last":0.61199999999999999},{"interface":"front","average":{"1min":0.70099999999999996,"5min":0.70099999999999996,"15min":0.70099999999999996},"min":{"1min":0.55000000000000004,"5min":0.55000000000000004,"15min":0.55000000000000004},"max":{"1min":1.1339999999999999,"5min":1.1339999999999999,"15min":1.1339999999999999},"last":0.56100000000000005}]},{"osd":7,"last update":"Sun Nov 26 07:18:41 2023","interfaces":[{"interface":"back","average":{"1min":0.71499999999999997,"5min":0.71499999999999997,"15min":0.71499999999999997},"min":{"1min":0.55100000000000005,"5min":0.55100000000000005,"15min":0.55100000000000005},"max":{"1min":0.92900000000000005,"5min":0.92900000000000005,"15min":0.92900000000000005},"last":0.59299999999999997},{"interface":"front","average":{"1min":0.753,"5min":0.753,"15min":0.753},"min":{"1min":0.56100000000000005,"5min":0.56100000000000005,"15min":0.56100000000000005},"max":{"1min":1.024,"5min":1.024,"15min":1.024},"last":0.71499999999999997}]}]},{"osd":0,"up_from":13,"seq":55834574866,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27944,"kb_used_data":920,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343896,"statfs":{"total":96636764160,"available":96608149504,"internally_reserved":0,"allocated":942080,"data_stored":723885,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Sun Nov 26 07:18:41 2023","interfaces":[{"interface":"back","average":{"1min":0.43099999999999999,"5min":0.43099999999999999,"15min":0.43099999999999999},"min":{"1min":0.28000000000000003,"5min":0.28000000000000003,"15min":0.28000000000000003},"max":{"1min":0.63100000000000001,"5min":0.63100000000000001,"15min":0.63100000000000001},"last":0.38200000000000001},{"interface":"front","average":{"1min":0.48299999999999998,"5min":0.48299999999999998,"15min":0.48299999999999998},"min":{"1min":0.29899999999999999,"5min":0.29899999999999999,"15min":0.29899999999999999},"max":{"1min":1.133,"5min":1.133,"15min":1.133},"last":0.30199999999999999}]},{"osd":2,"last update":"Sun Nov 26 07:18:41 2023","interfaces":[{"interface":"back","average":{"1min":0.53700000000000003,"5min":0.53700000000000003,"15min":0.53700000000000003},"min":{"1min":0.40899999999999997,"5min":0.40899999999999997,"15min":0.40899999999999997},"max":{"1min":0.67400000000000004,"5min":0.67400000000000004,"15min":0.67400000000000004},"last":0.48099999999999998},{"interface":"front","average":{"1min":0.56399999999999995,"5min":0.56399999999999995,"15min":0.56399999999999995},"min":{"1min":0.36399999999999999,"5min":0.36399999999999999,"15min":0.36399999999999999},"max":{"1min":0.72499999999999998,"5min":0.72499999999999998,"15min":0.72499999999999998},"last":0.45300000000000001}]},{"osd":3,"last update":"Sun Nov 26 07:18:41 2023","interfaces":[{"interface":"back","average":{"1min":0.53100000000000003,"5min":0.53100000000000003,"15min":0.53100000000000003},"min":{"1min":0.35799999999999998,"5min":0.35799999999999998,"15min":0.35799999999999998},"max":{"1min":0.97399999999999998,"5min":0.97399999999999998,"15min":0.97399999999999998},"last":0.436},{"interface":"front","average":{"1min":0.56200000000000006,"5min":0.56200000000000006,"15min":0.56200000000000006},"min":{"1min":0.40600000000000003,"5min":0.40600000000000003,"15min":0.40600000000000003},"max":{"1min":1.2509999999999999,"5min":1.2509999999999999,"15min":1.2509999999999999},"last":0.498}]},{"osd":4,"last update":"Sun Nov 26 07:18:41 2023","interfaces":[{"interface":"back","average":{"1min":0.75900000000000001,"5min":0.75900000000000001,"15min":0.75900000000000001},"min":{"1min":0.60999999999999999,"5min":0.60999999999999999,"15min":0.60999999999999999},"max":{"1min":1.2010000000000001,"5min":1.2010000000000001,"15min":1.2010000000000001},"last":0.69599999999999995},{"interface":"front","average":{"1min":0.78900000000000003,"5min":0.78900000000000003,"15min":0.78900000000000003},"min":{"1min":0.59999999999999998,"5min":0.59999999999999998,"15min":0.59999999999999998},"max":{"1min":0.99399999999999999,"5min":0.99399999999999999,"15min":0.99399999999999999},"last":0.71899999999999997}]},{"osd":5,"last update":"Sun Nov 26 07:18:41 2023","interfaces":[{"interface":"back","average":{"1min":0.76000000000000001,"5min":0.76000000000000001,"15min":0.76000000000000001},"min":{"1min":0.53000000000000003,"5min":0.53000000000000003,"15min":0.53000000000000003},"max":{"1min":1.1579999999999999,"5min":1.1579999999999999,"15min":1.1579999999999999},"last":0.73599999999999999},{"interface":"front","average":{"1min":0.79700000000000004,"5min":0.79700000000000004,"15min":0.79700000000000004},"min":{"1min":0.59699999999999998,"5min":0.59699999999999998,"15min":0.59699999999999998},"max":{"1min":1.0409999999999999,"5min":1.0409999999999999,"15min":1.0409999999999999},"last":0.67000000000000004}]},{"osd":6,"last update":"Sun Nov 26 07:18:41 2023","interfaces":[{"interface":"back","average":{"1min":0.76400000000000001,"5min":0.76400000000000001,"15min":0.76400000000000001},"min":{"1min":0.56599999999999995,"5min":0.56599999999999995,"15min":0.56599999999999995},"max":{"1min":1.3029999999999999,"5min":1.3029999999999999,"15min":1.3029999999999999},"last":0.55300000000000005},{"interface":"front","average":{"1min":0.76900000000000002,"5min":0.76900000000000002,"15min":0.76900000000000002},"min":{"1min":0.58599999999999997,"5min":0.58599999999999997,"15min":0.58599999999999997},"max":{"1min":1.2210000000000001,"5min":1.2210000000000001,"15min":1.2210000000000001},"last":0.76000000000000001}]},{"osd":7,"last update":"Sun Nov 26 07:18:41 2023","interfaces":[{"interface":"back","average":{"1min":0.81100000000000005,"5min":0.81100000000000005,"15min":0.81100000000000005},"min":{"1min":0.59999999999999998,"5min":0.59999999999999998,"15min":0.59999999999999998},"max":{"1min":1.177,"5min":1.177,"15min":1.177},"last":0.624},{"interface":"front","average":{"1min":0.79900000000000004,"5min":0.79900000000000004,"15min":0.79900000000000004},"min":{"1min":0.60899999999999999,"5min":0.60899999999999999,"15min":0.60899999999999999},"max":{"1min":1.2390000000000001,"5min":1.2390000000000001,"15min":1.2390000000000001},"last":0.65100000000000002}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-11-26T07:18:56.725 DEBUG:teuthology.orchestra.run.smithi143:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-11-26T07:18:56.906 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:56.900+0000 7fc575ed6640 1 Processor -- start 2023-11-26T07:18:56.906 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:56.900+0000 7fc575ed6640 1 -- start start 2023-11-26T07:18:56.908 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:56.900+0000 7fc575ed6640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fc570143ac0 0x7fc570070370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:18:56.908 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:56.900+0000 7fc575ed6640 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7fc5700cf990 0x7fc5700cd1b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:18:56.908 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:56.900+0000 7fc575ed6640 1 -- --> v1:172.21.15.143:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc5700cc730 con 0x7fc5700708b0 2023-11-26T07:18:56.908 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:56.900+0000 7fc575ed6640 1 -- --> v2:172.21.15.188:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc570143ea0 con 0x7fc5700cf990 2023-11-26T07:18:56.908 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:56.900+0000 7fc575ed6640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc570070ee0 con 0x7fc570143ac0 2023-11-26T07:18:56.908 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:56.900+0000 7fc567fff640 1 --1- >> v1:172.21.15.143:6789/0 conn(0x7fc5700708b0 0x7fc5700cf280 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.143:6789/0 says I am v1:172.21.15.143:35290/0 (socket says 172.21.15.143:35290) 2023-11-26T07:18:56.908 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:56.900+0000 7fc567fff640 1 -- 172.21.15.143:0/599773134 learned_addr learned my addr 172.21.15.143:0/599773134 (peer_addr_for_me v1:172.21.15.143:0/0) 2023-11-26T07:18:56.910 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:56.900+0000 7fc56ffff640 1 --2- 172.21.15.143:0/599773134 >> v2:172.21.15.188:3300/0 conn(0x7fc5700cf990 0x7fc5700cd1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:18:56.910 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:56.900+0000 7fc56f7fe640 1 --2- 172.21.15.143:0/599773134 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fc570143ac0 0x7fc570070370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:18:56.911 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:56.900+0000 7fc56effd640 1 -- 172.21.15.143:0/599773134 <== mon.0 v1:172.21.15.143:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3207142054 0 0) 0x7fc5700cc730 con 0x7fc5700708b0 2023-11-26T07:18:56.911 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:56.900+0000 7fc56effd640 1 -- 172.21.15.143:0/599773134 --> v1:172.21.15.143:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fc55c003530 con 0x7fc5700708b0 2023-11-26T07:18:56.911 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:56.900+0000 7fc56effd640 1 -- 172.21.15.143:0/599773134 <== mon.0 v1:172.21.15.143:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 538676288 0 0) 0x7fc55c003530 con 0x7fc5700708b0 2023-11-26T07:18:56.911 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:56.900+0000 7fc56effd640 1 -- 172.21.15.143:0/599773134 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fc570143ac0 msgr2=0x7fc570070370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-26T07:18:56.911 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:56.900+0000 7fc56effd640 1 --2- 172.21.15.143:0/599773134 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fc570143ac0 0x7fc570070370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:18:56.911 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:56.900+0000 7fc56effd640 1 -- 172.21.15.143:0/599773134 >> v2:172.21.15.188:3300/0 conn(0x7fc5700cf990 msgr2=0x7fc5700cd1b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-26T07:18:56.911 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:56.900+0000 7fc56effd640 1 --2- 172.21.15.143:0/599773134 >> v2:172.21.15.188:3300/0 conn(0x7fc5700cf990 0x7fc5700cd1b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:18:56.911 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:56.900+0000 7fc56effd640 1 -- 172.21.15.143:0/599773134 --> v1:172.21.15.143:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc5700711c0 con 0x7fc5700708b0 2023-11-26T07:18:56.911 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:56.900+0000 7fc56effd640 1 -- 172.21.15.143:0/599773134 <== mon.0 v1:172.21.15.143:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2137882523 0 0) 0x7fc5580016a0 con 0x7fc5700708b0 2023-11-26T07:18:56.912 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:56.900+0000 7fc56effd640 1 -- 172.21.15.143:0/599773134 >> v1:172.21.15.143:6789/0 conn(0x7fc5700708b0 legacy=0x7fc5700cf280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:18:56.912 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:56.904+0000 7fc56effd640 1 --2- 172.21.15.143:0/599773134 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fc55c003980 0x7fc55c003da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:18:56.912 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:56.904+0000 7fc56effd640 1 --2- 172.21.15.143:0/599773134 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fc55c0047d0 0x7fc55c004c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:18:56.912 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:56.904+0000 7fc56effd640 1 --2- 172.21.15.143:0/599773134 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fc570143ac0 0x7fc55c005190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:18:56.912 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:56.904+0000 7fc56effd640 1 -- 172.21.15.143:0/599773134 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc570070ee0 con 0x7fc55c003980 2023-11-26T07:18:56.912 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:56.904+0000 7fc56effd640 1 -- 172.21.15.143:0/599773134 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc55c00b990 con 0x7fc55c0047d0 2023-11-26T07:18:56.912 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:56.904+0000 7fc56effd640 1 -- 172.21.15.143:0/599773134 --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc55c00bad0 con 0x7fc570143ac0 2023-11-26T07:18:56.912 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:56.904+0000 7fc56f7fe640 1 --2- 172.21.15.143:0/599773134 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fc55c003980 0x7fc55c003da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:18:56.912 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:56.904+0000 7fc567fff640 1 --2- 172.21.15.143:0/599773134 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fc55c0047d0 0x7fc55c004c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:18:56.913 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:56.904+0000 7fc56ffff640 1 --2- 172.21.15.143:0/599773134 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fc570143ac0 0x7fc55c005190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:18:56.913 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:56.904+0000 7fc56f7fe640 1 -- 172.21.15.143:0/599773134 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fc570143ac0 msgr2=0x7fc55c005190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-26T07:18:56.913 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:56.904+0000 7fc56f7fe640 1 --2- 172.21.15.143:0/599773134 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fc570143ac0 0x7fc55c005190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:18:56.913 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:56.904+0000 7fc56f7fe640 1 -- 172.21.15.143:0/599773134 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fc55c0047d0 msgr2=0x7fc55c004c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-26T07:18:56.913 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:56.904+0000 7fc56f7fe640 1 --2- 172.21.15.143:0/599773134 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fc55c0047d0 0x7fc55c004c50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:18:56.913 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:56.904+0000 7fc56f7fe640 1 -- 172.21.15.143:0/599773134 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc55c003530 con 0x7fc55c003980 2023-11-26T07:18:56.915 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:56.904+0000 7fc56f7fe640 1 --2- 172.21.15.143:0/599773134 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fc55c003980 0x7fc55c003da0 secure :-1 s=READY pgs=413 cs=0 l=1 rev1=1 crypto rx=0x7fc560009ae0 tx=0x7fc56001c9f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5bc1a7e81ee7e76d server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:18:56.915 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:56.904+0000 7fc56effd640 1 -- 172.21.15.143:0/599773134 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc5600a6460 con 0x7fc55c003980 2023-11-26T07:18:56.915 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:56.904+0000 7fc56effd640 1 -- 172.21.15.143:0/599773134 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc5600a65c0 con 0x7fc55c003980 2023-11-26T07:18:56.915 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:56.904+0000 7fc56effd640 1 -- 172.21.15.143:0/599773134 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc5600a6970 con 0x7fc55c003980 2023-11-26T07:18:56.915 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:56.908+0000 7fc575ed6640 1 -- 172.21.15.143:0/599773134 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fc55c003980 msgr2=0x7fc55c003da0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:18:56.915 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:56.908+0000 7fc575ed6640 1 --2- 172.21.15.143:0/599773134 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fc55c003980 0x7fc55c003da0 secure :-1 s=READY pgs=413 cs=0 l=1 rev1=1 crypto rx=0x7fc560009ae0 tx=0x7fc56001c9f0 comp rx=0 tx=0).stop 2023-11-26T07:18:56.916 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:56.908+0000 7fc56ffff640 1 -- 172.21.15.143:0/599773134 reap_dead start 2023-11-26T07:18:56.916 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:56.908+0000 7fc575ed6640 1 -- 172.21.15.143:0/599773134 shutdown_connections 2023-11-26T07:18:56.916 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:56.908+0000 7fc575ed6640 1 -- 172.21.15.143:0/599773134 >> 172.21.15.143:0/599773134 conn(0x7fc57006a890 msgr2=0x7fc57006aca0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:18:56.916 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:56.908+0000 7fc575ed6640 1 -- 172.21.15.143:0/599773134 shutdown_connections 2023-11-26T07:18:56.916 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:56.908+0000 7fc575ed6640 1 -- 172.21.15.143:0/599773134 wait complete. 2023-11-26T07:18:56.916 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:56.908+0000 7fc575ed6640 1 Processor -- start 2023-11-26T07:18:56.916 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:56.908+0000 7fc575ed6640 1 -- start start 2023-11-26T07:18:56.918 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:56.908+0000 7fc575ed6640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fc55c003980 0x7fc570159b40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:18:56.918 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:56.908+0000 7fc575ed6640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fc5700af2c0 0x7fc57015cb20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:18:56.918 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:56.908+0000 7fc575ed6640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fc57015d060 0x7fc5700b1020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:18:56.918 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:56.908+0000 7fc575ed6640 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc57015b160 con 0x7fc5700af2c0 2023-11-26T07:18:56.918 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:56.908+0000 7fc575ed6640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc57015b7a0 con 0x7fc55c003980 2023-11-26T07:18:56.918 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:56.908+0000 7fc575ed6640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc57015ba80 con 0x7fc57015d060 2023-11-26T07:18:56.919 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:56.908+0000 7fc567fff640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fc5700af2c0 0x7fc57015cb20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:18:56.919 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:56.908+0000 7fc567fff640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fc5700af2c0 0x7fc57015cb20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:51730/0 (socket says 172.21.15.143:51730) 2023-11-26T07:18:56.919 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:56.908+0000 7fc567fff640 1 -- 172.21.15.143:0/1393253460 learned_addr learned my addr 172.21.15.143:0/1393253460 (peer_addr_for_me v2:172.21.15.143:0/0) 2023-11-26T07:18:56.919 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:56.908+0000 7fc56f7fe640 1 --2- 172.21.15.143:0/1393253460 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fc55c003980 0x7fc570159b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:18:56.919 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:56.908+0000 7fc56ffff640 1 --2- 172.21.15.143:0/1393253460 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fc57015d060 0x7fc5700b1020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:18:56.919 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:56.908+0000 7fc567fff640 1 -- 172.21.15.143:0/1393253460 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fc57015d060 msgr2=0x7fc5700b1020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:18:56.919 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:56.908+0000 7fc567fff640 1 --2- 172.21.15.143:0/1393253460 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fc57015d060 0x7fc5700b1020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:18:56.919 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:56.908+0000 7fc567fff640 1 -- 172.21.15.143:0/1393253460 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fc55c003980 msgr2=0x7fc570159b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:18:56.919 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:56.908+0000 7fc567fff640 1 --2- 172.21.15.143:0/1393253460 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fc55c003980 0x7fc570159b40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:18:56.919 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:56.908+0000 7fc567fff640 1 -- 172.21.15.143:0/1393253460 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc57015bd60 con 0x7fc5700af2c0 2023-11-26T07:18:56.919 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:56.908+0000 7fc567fff640 1 --2- 172.21.15.143:0/1393253460 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fc5700af2c0 0x7fc57015cb20 secure :-1 s=READY pgs=414 cs=0 l=1 rev1=1 crypto rx=0x7fc558001890 tx=0x7fc558007430 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:18:56.919 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:56.912+0000 7fc56cff9640 1 -- 172.21.15.143:0/1393253460 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc55800f680 con 0x7fc5700af2c0 2023-11-26T07:18:56.920 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:56.912+0000 7fc56cff9640 1 -- 172.21.15.143:0/1393253460 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc55800f800 con 0x7fc5700af2c0 2023-11-26T07:18:56.920 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:56.912+0000 7fc56cff9640 1 -- 172.21.15.143:0/1393253460 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc55800fbb0 con 0x7fc5700af2c0 2023-11-26T07:18:56.920 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:56.912+0000 7fc575ed6640 1 -- 172.21.15.143:0/1393253460 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc57015c040 con 0x7fc5700af2c0 2023-11-26T07:18:56.920 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:56.912+0000 7fc575ed6640 1 -- 172.21.15.143:0/1393253460 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc57015c600 con 0x7fc5700af2c0 2023-11-26T07:18:56.920 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:56.912+0000 7fc575ed6640 1 -- 172.21.15.143:0/1393253460 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc570067680 con 0x7fc5700af2c0 2023-11-26T07:18:56.921 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:56.912+0000 7fc56cff9640 1 -- 172.21.15.143:0/1393253460 <== mon.0 v2:172.21.15.143:3300/0 4 ==== mgrmap(e 5) v1 ==== 85852+0+0 (secure 0 0 0) 0x7fc55800fd10 con 0x7fc5700af2c0 2023-11-26T07:18:56.921 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:56.912+0000 7fc56cff9640 1 --2- 172.21.15.143:0/1393253460 >> [v2:172.21.15.143:6832/2150557544,v1:172.21.15.143:6833/2150557544] conn(0x7fc5500664a0 0x7fc550068960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:18:56.921 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:56.912+0000 7fc56cff9640 1 -- 172.21.15.143:0/1393253460 <== mon.0 v2:172.21.15.143:3300/0 5 ==== osd_map(33..33 src has 1..33) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fc558083eb0 con 0x7fc5700af2c0 2023-11-26T07:18:56.921 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:56.912+0000 7fc56f7fe640 1 --2- 172.21.15.143:0/1393253460 >> [v2:172.21.15.143:6832/2150557544,v1:172.21.15.143:6833/2150557544] conn(0x7fc5500664a0 0x7fc550068960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:18:56.921 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:56.916+0000 7fc56f7fe640 1 --2- 172.21.15.143:0/1393253460 >> [v2:172.21.15.143:6832/2150557544,v1:172.21.15.143:6833/2150557544] conn(0x7fc5500664a0 0x7fc550068960 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7fc5600ad000 tx=0x7fc5600b3040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:18:56.924 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:56.916+0000 7fc56cff9640 1 -- 172.21.15.143:0/1393253460 <== mon.0 v2:172.21.15.143:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172649 (secure 0 0 0) 0x7fc558051290 con 0x7fc5700af2c0 2023-11-26T07:18:57.071 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:57.064+0000 7fc575ed6640 1 -- 172.21.15.143:0/1393253460 --> [v2:172.21.15.143:6832/2150557544,v1:172.21.15.143:6833/2150557544] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7fc570070c90 con 0x7fc5500664a0 2023-11-26T07:18:57.078 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:57.072+0000 7fc56cff9640 1 -- 172.21.15.143:0/1393253460 <== mgr.4100 v2:172.21.15.143:6832/2150557544 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+284432 (secure 0 0 0) 0x7fc56000f0a0 con 0x7fc5500664a0 2023-11-26T07:18:57.080 INFO:teuthology.orchestra.run.smithi143.stdout: 2023-11-26T07:18:57.082 INFO:teuthology.orchestra.run.smithi143.stderr:dumped all 2023-11-26T07:18:57.083 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:57.076+0000 7fc575ed6640 1 -- 172.21.15.143:0/1393253460 >> [v2:172.21.15.143:6832/2150557544,v1:172.21.15.143:6833/2150557544] conn(0x7fc5500664a0 msgr2=0x7fc550068960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:18:57.083 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:57.076+0000 7fc575ed6640 1 --2- 172.21.15.143:0/1393253460 >> [v2:172.21.15.143:6832/2150557544,v1:172.21.15.143:6833/2150557544] conn(0x7fc5500664a0 0x7fc550068960 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7fc5600ad000 tx=0x7fc5600b3040 comp rx=0 tx=0).stop 2023-11-26T07:18:57.083 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:57.076+0000 7fc575ed6640 1 -- 172.21.15.143:0/1393253460 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fc5700af2c0 msgr2=0x7fc57015cb20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:18:57.083 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:57.076+0000 7fc575ed6640 1 --2- 172.21.15.143:0/1393253460 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fc5700af2c0 0x7fc57015cb20 secure :-1 s=READY pgs=414 cs=0 l=1 rev1=1 crypto rx=0x7fc558001890 tx=0x7fc558007430 comp rx=0 tx=0).stop 2023-11-26T07:18:57.083 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:57.076+0000 7fc575ed6640 1 -- 172.21.15.143:0/1393253460 shutdown_connections 2023-11-26T07:18:57.083 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:57.076+0000 7fc575ed6640 1 --2- 172.21.15.143:0/1393253460 >> [v2:172.21.15.143:6832/2150557544,v1:172.21.15.143:6833/2150557544] conn(0x7fc5500664a0 0x7fc550068960 unknown :-1 s=CLOSED pgs=223 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:18:57.084 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:57.076+0000 7fc575ed6640 1 --2- 172.21.15.143:0/1393253460 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fc57015d060 0x7fc5700b1020 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:18:57.084 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:57.076+0000 7fc575ed6640 1 --2- 172.21.15.143:0/1393253460 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fc5700af2c0 0x7fc57015cb20 unknown :-1 s=CLOSED pgs=414 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:18:57.084 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:57.076+0000 7fc575ed6640 1 --2- 172.21.15.143:0/1393253460 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fc55c003980 0x7fc570159b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:18:57.084 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:57.076+0000 7fc575ed6640 1 -- 172.21.15.143:0/1393253460 >> 172.21.15.143:0/1393253460 conn(0x7fc57006a890 msgr2=0x7fc570072130 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:18:57.084 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:57.076+0000 7fc575ed6640 1 -- 172.21.15.143:0/1393253460 shutdown_connections 2023-11-26T07:18:57.084 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:57.076+0000 7fc575ed6640 1 -- 172.21.15.143:0/1393253460 wait complete. 2023-11-26T07:18:57.101 INFO:teuthology.orchestra.run.smithi143.stdout:{"pg_ready":true,"pg_map":{"version":63,"stamp":"2023-11-26T07:18:55.705391+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":20,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":219996,"kb_used_data":3820,"kb_used_omap":12,"kb_used_meta":216051,"kb_avail":754754724,"statfs":{"total":773094113280,"available":772868837376,"internally_reserved":0,"allocated":3911680,"data_stored":2217537,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12749,"internal_metadata":221236787},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":36,"apply_latency_ms":36,"commit_latency_ns":36000000,"apply_latency_ns":36000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"5.076804"},"pg_stats":[{"pgid":"5.3f","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2023-11-26T07:18:51.645063+0000","last_change":"2023-11-26T07:18:51.645063+0000","last_active":"2023-11-26T07:18:51.625829+0000","last_peered":"2023-11-26T07:18:51.625829+0000","last_clean":"2023-11-26T07:18:51.625829+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-26T07:18:51.645063+0000","last_undegraded":"2023-11-26T07:18:51.645063+0000","last_fullsized":"2023-11-26T07:18:51.645063+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:18:51.625829+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:18:51.625829+0000","last_clean_scrub_stamp":"2023-11-26T07:18:51.625829+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.3e","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2023-11-26T07:18:51.643947+0000","last_change":"2023-11-26T07:18:51.643947+0000","last_active":"2023-11-26T07:18:51.625829+0000","last_peered":"2023-11-26T07:18:51.625829+0000","last_clean":"2023-11-26T07:18:51.625829+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-26T07:18:51.643947+0000","last_undegraded":"2023-11-26T07:18:51.643947+0000","last_fullsized":"2023-11-26T07:18:51.643947+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:18:51.625829+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:18:51.625829+0000","last_clean_scrub_stamp":"2023-11-26T07:18:51.625829+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.3d","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2023-11-26T07:18:51.634980+0000","last_change":"2023-11-26T07:18:51.634980+0000","last_active":"2023-11-26T07:18:51.625829+0000","last_peered":"2023-11-26T07:18:51.625829+0000","last_clean":"2023-11-26T07:18:51.625829+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-26T07:18:51.634980+0000","last_undegraded":"2023-11-26T07:18:51.634980+0000","last_fullsized":"2023-11-26T07:18:51.634980+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:18:51.625829+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:18:51.625829+0000","last_clean_scrub_stamp":"2023-11-26T07:18:51.625829+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.3c","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2023-11-26T07:18:51.634028+0000","last_change":"2023-11-26T07:18:51.634028+0000","last_active":"2023-11-26T07:18:51.625829+0000","last_peered":"2023-11-26T07:18:51.625829+0000","last_clean":"2023-11-26T07:18:51.625829+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-26T07:18:51.634028+0000","last_undegraded":"2023-11-26T07:18:51.634028+0000","last_fullsized":"2023-11-26T07:18:51.634028+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:18:51.625829+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:18:51.625829+0000","last_clean_scrub_stamp":"2023-11-26T07:18:51.625829+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.3b","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2023-11-26T07:18:51.651017+0000","last_change":"2023-11-26T07:18:51.651017+0000","last_active":"2023-11-26T07:18:51.625829+0000","last_peered":"2023-11-26T07:18:51.625829+0000","last_clean":"2023-11-26T07:18:51.625829+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-26T07:18:51.651017+0000","last_undegraded":"2023-11-26T07:18:51.651017+0000","last_fullsized":"2023-11-26T07:18:51.651017+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:18:51.625829+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:18:51.625829+0000","last_clean_scrub_stamp":"2023-11-26T07:18:51.625829+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.3a","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2023-11-26T07:18:51.644135+0000","last_change":"2023-11-26T07:18:51.644135+0000","last_active":"2023-11-26T07:18:51.625829+0000","last_peered":"2023-11-26T07:18:51.625829+0000","last_clean":"2023-11-26T07:18:51.625829+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-26T07:18:51.644135+0000","last_undegraded":"2023-11-26T07:18:51.644135+0000","last_fullsized":"2023-11-26T07:18:51.644135+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:18:51.625829+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:18:51.625829+0000","last_clean_scrub_stamp":"2023-11-26T07:18:51.625829+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.39","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2023-11-26T07:18:51.641562+0000","last_change":"2023-11-26T07:18:51.641562+0000","last_active":"2023-11-26T07:18:51.625829+0000","last_peered":"2023-11-26T07:18:51.625829+0000","last_clean":"2023-11-26T07:18:51.625829+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-26T07:18:51.641562+0000","last_undegraded":"2023-11-26T07:18:51.641562+0000","last_fullsized":"2023-11-26T07:18:51.641562+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:18:51.625829+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:18:51.625829+0000","last_clean_scrub_stamp":"2023-11-26T07:18:51.625829+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.38","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2023-11-26T07:18:51.646503+0000","last_change":"2023-11-26T07:18:51.646503+0000","last_active":"2023-11-26T07:18:51.625829+0000","last_peered":"2023-11-26T07:18:51.625829+0000","last_clean":"2023-11-26T07:18:51.625829+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-26T07:18:51.646503+0000","last_undegraded":"2023-11-26T07:18:51.646503+0000","last_fullsized":"2023-11-26T07:18:51.646503+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:18:51.625829+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:18:51.625829+0000","last_clean_scrub_stamp":"2023-11-26T07:18:51.625829+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.37","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2023-11-26T07:18:51.636576+0000","last_change":"2023-11-26T07:18:51.636576+0000","last_active":"2023-11-26T07:18:51.625829+0000","last_peered":"2023-11-26T07:18:51.625829+0000","last_clean":"2023-11-26T07:18:51.625829+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-26T07:18:51.636576+0000","last_undegraded":"2023-11-26T07:18:51.636576+0000","last_fullsized":"2023-11-26T07:18:51.636576+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:18:51.625829+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:18:51.625829+0000","last_clean_scrub_stamp":"2023-11-26T07:18:51.625829+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.36","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2023-11-26T07:18:51.633151+0000","last_change":"2023-11-26T07:18:51.633151+0000","last_active":"2023-11-26T07:18:51.625829+0000","last_peered":"2023-11-26T07:18:51.625829+0000","last_clean":"2023-11-26T07:18:51.625829+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-26T07:18:51.633151+0000","last_undegraded":"2023-11-26T07:18:51.633151+0000","last_fullsized":"2023-11-26T07:18:51.633151+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:18:51.625829+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:18:51.625829+0000","last_clean_scrub_stamp":"2023-11-26T07:18:51.625829+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.35","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2023-11-26T07:18:51.640729+0000","last_change":"2023-11-26T07:18:51.640729+0000","last_active":"2023-11-26T07:18:51.625829+0000","last_peered":"2023-11-26T07:18:51.625829+0000","last_clean":"2023-11-26T07:18:51.625829+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-26T07:18:51.640729+0000","last_undegraded":"2023-11-26T07:18:51.640729+0000","last_fullsized":"2023-11-26T07:18:51.640729+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:18:51.625829+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:18:51.625829+0000","last_clean_scrub_stamp":"2023-11-26T07:18:51.625829+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.34","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2023-11-26T07:18:51.641193+0000","last_change":"2023-11-26T07:18:51.641193+0000","last_active":"2023-11-26T07:18:51.625829+0000","last_peered":"2023-11-26T07:18:51.625829+0000","last_clean":"2023-11-26T07:18:51.625829+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-26T07:18:51.641193+0000","last_undegraded":"2023-11-26T07:18:51.641193+0000","last_fullsized":"2023-11-26T07:18:51.641193+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:18:51.625829+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:18:51.625829+0000","last_clean_scrub_stamp":"2023-11-26T07:18:51.625829+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.33","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2023-11-26T07:18:51.649682+0000","last_change":"2023-11-26T07:18:51.649682+0000","last_active":"2023-11-26T07:18:51.625829+0000","last_peered":"2023-11-26T07:18:51.625829+0000","last_clean":"2023-11-26T07:18:51.625829+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-26T07:18:51.649682+0000","last_undegraded":"2023-11-26T07:18:51.649682+0000","last_fullsized":"2023-11-26T07:18:51.649682+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:18:51.625829+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:18:51.625829+0000","last_clean_scrub_stamp":"2023-11-26T07:18:51.625829+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.32","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2023-11-26T07:18:51.643138+0000","last_change":"2023-11-26T07:18:51.643138+0000","last_active":"2023-11-26T07:18:51.625829+0000","last_peered":"2023-11-26T07:18:51.625829+0000","last_clean":"2023-11-26T07:18:51.625829+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-26T07:18:51.643138+0000","last_undegraded":"2023-11-26T07:18:51.643138+0000","last_fullsized":"2023-11-26T07:18:51.643138+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:18:51.625829+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:18:51.625829+0000","last_clean_scrub_stamp":"2023-11-26T07:18:51.625829+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.31","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2023-11-26T07:18:51.634598+0000","last_change":"2023-11-26T07:18:51.634598+0000","last_active":"2023-11-26T07:18:51.625829+0000","last_peered":"2023-11-26T07:18:51.625829+0000","last_clean":"2023-11-26T07:18:51.625829+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-26T07:18:51.634598+0000","last_undegraded":"2023-11-26T07:18:51.634598+0000","last_fullsized":"2023-11-26T07:18:51.634598+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:18:51.625829+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:18:51.625829+0000","last_clean_scrub_stamp":"2023-11-26T07:18:51.625829+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.30","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2023-11-26T07:18:51.640197+0000","last_change":"2023-11-26T07:18:51.640197+0000","last_active":"2023-11-26T07:18:51.625829+0000","last_peered":"2023-11-26T07:18:51.625829+0000","last_clean":"2023-11-26T07:18:51.625829+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-26T07:18:51.640197+0000","last_undegraded":"2023-11-26T07:18:51.640197+0000","last_fullsized":"2023-11-26T07:18:51.640197+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:18:51.625829+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:18:51.625829+0000","last_clean_scrub_stamp":"2023-11-26T07:18:51.625829+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.2f","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2023-11-26T07:18:51.641801+0000","last_change":"2023-11-26T07:18:51.641801+0000","last_active":"2023-11-26T07:18:51.625829+0000","last_peered":"2023-11-26T07:18:51.625829+0000","last_clean":"2023-11-26T07:18:51.625829+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-26T07:18:51.641801+0000","last_undegraded":"2023-11-26T07:18:51.641801+0000","last_fullsized":"2023-11-26T07:18:51.641801+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:18:51.625829+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:18:51.625829+0000","last_clean_scrub_stamp":"2023-11-26T07:18:51.625829+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.2e","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2023-11-26T07:18:51.643556+0000","last_change":"2023-11-26T07:18:51.643556+0000","last_active":"2023-11-26T07:18:51.625829+0000","last_peered":"2023-11-26T07:18:51.625829+0000","last_clean":"2023-11-26T07:18:51.625829+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-26T07:18:51.643556+0000","last_undegraded":"2023-11-26T07:18:51.643556+0000","last_fullsized":"2023-11-26T07:18:51.643556+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:18:51.625829+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:18:51.625829+0000","last_clean_scrub_stamp":"2023-11-26T07:18:51.625829+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.2d","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2023-11-26T07:18:51.642785+0000","last_change":"2023-11-26T07:18:51.642785+0000","last_active":"2023-11-26T07:18:51.625829+0000","last_peered":"2023-11-26T07:18:51.625829+0000","last_clean":"2023-11-26T07:18:51.625829+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-26T07:18:51.642785+0000","last_undegraded":"2023-11-26T07:18:51.642785+0000","last_fullsized":"2023-11-26T07:18:51.642785+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:18:51.625829+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:18:51.625829+0000","last_clean_scrub_stamp":"2023-11-26T07:18:51.625829+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.2c","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2023-11-26T07:18:51.651641+0000","last_change":"2023-11-26T07:18:51.651641+0000","last_active":"2023-11-26T07:18:51.625829+0000","last_peered":"2023-11-26T07:18:51.625829+0000","last_clean":"2023-11-26T07:18:51.625829+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-26T07:18:51.651641+0000","last_undegraded":"2023-11-26T07:18:51.651641+0000","last_fullsized":"2023-11-26T07:18:51.651641+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:18:51.625829+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:18:51.625829+0000","last_clean_scrub_stamp":"2023-11-26T07:18:51.625829+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.2b","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2023-11-26T07:18:51.637758+0000","last_change":"2023-11-26T07:18:51.637758+0000","last_active":"2023-11-26T07:18:51.625829+0000","last_peered":"2023-11-26T07:18:51.625829+0000","last_clean":"2023-11-26T07:18:51.625829+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-26T07:18:51.637758+0000","last_undegraded":"2023-11-26T07:18:51.637758+0000","last_fullsized":"2023-11-26T07:18:51.637758+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:18:51.625829+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:18:51.625829+0000","last_clean_scrub_stamp":"2023-11-26T07:18:51.625829+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.2a","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2023-11-26T07:18:51.636183+0000","last_change":"2023-11-26T07:18:51.636183+0000","last_active":"2023-11-26T07:18:51.625829+0000","last_peered":"2023-11-26T07:18:51.625829+0000","last_clean":"2023-11-26T07:18:51.625829+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-26T07:18:51.636183+0000","last_undegraded":"2023-11-26T07:18:51.636183+0000","last_fullsized":"2023-11-26T07:18:51.636183+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:18:51.625829+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:18:51.625829+0000","last_clean_scrub_stamp":"2023-11-26T07:18:51.625829+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.29","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2023-11-26T07:18:51.641884+0000","last_change":"2023-11-26T07:18:51.641884+0000","last_active":"2023-11-26T07:18:51.625829+0000","last_peered":"2023-11-26T07:18:51.625829+0000","last_clean":"2023-11-26T07:18:51.625829+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-26T07:18:51.641884+0000","last_undegraded":"2023-11-26T07:18:51.641884+0000","last_fullsized":"2023-11-26T07:18:51.641884+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:18:51.625829+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:18:51.625829+0000","last_clean_scrub_stamp":"2023-11-26T07:18:51.625829+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.28","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2023-11-26T07:18:51.635231+0000","last_change":"2023-11-26T07:18:51.635231+0000","last_active":"2023-11-26T07:18:51.625829+0000","last_peered":"2023-11-26T07:18:51.625829+0000","last_clean":"2023-11-26T07:18:51.625829+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-26T07:18:51.635231+0000","last_undegraded":"2023-11-26T07:18:51.635231+0000","last_fullsized":"2023-11-26T07:18:51.635231+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:18:51.625829+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:18:51.625829+0000","last_clean_scrub_stamp":"2023-11-26T07:18:51.625829+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.27","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2023-11-26T07:18:51.635689+0000","last_change":"2023-11-26T07:18:51.635689+0000","last_active":"2023-11-26T07:18:51.625829+0000","last_peered":"2023-11-26T07:18:51.625829+0000","last_clean":"2023-11-26T07:18:51.625829+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-26T07:18:51.635689+0000","last_undegraded":"2023-11-26T07:18:51.635689+0000","last_fullsized":"2023-11-26T07:18:51.635689+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:18:51.625829+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:18:51.625829+0000","last_clean_scrub_stamp":"2023-11-26T07:18:51.625829+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.26","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2023-11-26T07:18:51.643037+0000","last_change":"2023-11-26T07:18:51.643037+0000","last_active":"2023-11-26T07:18:51.625829+0000","last_peered":"2023-11-26T07:18:51.625829+0000","last_clean":"2023-11-26T07:18:51.625829+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-26T07:18:51.643037+0000","last_undegraded":"2023-11-26T07:18:51.643037+0000","last_fullsized":"2023-11-26T07:18:51.643037+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:18:51.625829+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:18:51.625829+0000","last_clean_scrub_stamp":"2023-11-26T07:18:51.625829+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.25","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2023-11-26T07:18:51.634261+0000","last_change":"2023-11-26T07:18:51.634261+0000","last_active":"2023-11-26T07:18:51.625829+0000","last_peered":"2023-11-26T07:18:51.625829+0000","last_clean":"2023-11-26T07:18:51.625829+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-26T07:18:51.634261+0000","last_undegraded":"2023-11-26T07:18:51.634261+0000","last_fullsized":"2023-11-26T07:18:51.634261+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:18:51.625829+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:18:51.625829+0000","last_clean_scrub_stamp":"2023-11-26T07:18:51.625829+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.24","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2023-11-26T07:18:51.637792+0000","last_change":"2023-11-26T07:18:51.637792+0000","last_active":"2023-11-26T07:18:51.625829+0000","last_peered":"2023-11-26T07:18:51.625829+0000","last_clean":"2023-11-26T07:18:51.625829+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-26T07:18:51.637792+0000","last_undegraded":"2023-11-26T07:18:51.637792+0000","last_fullsized":"2023-11-26T07:18:51.637792+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:18:51.625829+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:18:51.625829+0000","last_clean_scrub_stamp":"2023-11-26T07:18:51.625829+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.23","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2023-11-26T07:18:51.637010+0000","last_change":"2023-11-26T07:18:51.637010+0000","last_active":"2023-11-26T07:18:51.625829+0000","last_peered":"2023-11-26T07:18:51.625829+0000","last_clean":"2023-11-26T07:18:51.625829+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-26T07:18:51.637010+0000","last_undegraded":"2023-11-26T07:18:51.637010+0000","last_fullsized":"2023-11-26T07:18:51.637010+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:18:51.625829+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:18:51.625829+0000","last_clean_scrub_stamp":"2023-11-26T07:18:51.625829+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.22","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2023-11-26T07:18:51.644576+0000","last_change":"2023-11-26T07:18:51.644576+0000","last_active":"2023-11-26T07:18:51.625829+0000","last_peered":"2023-11-26T07:18:51.625829+0000","last_clean":"2023-11-26T07:18:51.625829+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-26T07:18:51.644576+0000","last_undegraded":"2023-11-26T07:18:51.644576+0000","last_fullsized":"2023-11-26T07:18:51.644576+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:18:51.625829+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:18:51.625829+0000","last_clean_scrub_stamp":"2023-11-26T07:18:51.625829+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.21","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2023-11-26T07:18:51.644802+0000","last_change":"2023-11-26T07:18:51.644802+0000","last_active":"2023-11-26T07:18:51.625829+0000","last_peered":"2023-11-26T07:18:51.625829+0000","last_clean":"2023-11-26T07:18:51.625829+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-26T07:18:51.644802+0000","last_undegraded":"2023-11-26T07:18:51.644802+0000","last_fullsized":"2023-11-26T07:18:51.644802+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:18:51.625829+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:18:51.625829+0000","last_clean_scrub_stamp":"2023-11-26T07:18:51.625829+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.20","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2023-11-26T07:18:51.646001+0000","last_change":"2023-11-26T07:18:51.646001+0000","last_active":"2023-11-26T07:18:51.625829+0000","last_peered":"2023-11-26T07:18:51.625829+0000","last_clean":"2023-11-26T07:18:51.625829+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-26T07:18:51.646001+0000","last_undegraded":"2023-11-26T07:18:51.646001+0000","last_fullsized":"2023-11-26T07:18:51.646001+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:18:51.625829+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:18:51.625829+0000","last_clean_scrub_stamp":"2023-11-26T07:18:51.625829+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.1e","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2023-11-26T07:18:51.641565+0000","last_change":"2023-11-26T07:18:51.641565+0000","last_active":"2023-11-26T07:18:51.625829+0000","last_peered":"2023-11-26T07:18:51.625829+0000","last_clean":"2023-11-26T07:18:51.625829+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-26T07:18:51.641565+0000","last_undegraded":"2023-11-26T07:18:51.641565+0000","last_fullsized":"2023-11-26T07:18:51.641565+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:18:51.625829+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:18:51.625829+0000","last_clean_scrub_stamp":"2023-11-26T07:18:51.625829+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1f","version":"0'0","reported_seq":9,"reported_epoch":31,"state":"active+clean","last_fresh":"2023-11-26T07:18:51.664628+0000","last_change":"2023-11-26T07:18:51.661754+0000","last_active":"2023-11-26T07:18:51.664628+0000","last_peered":"2023-11-26T07:18:51.664628+0000","last_clean":"2023-11-26T07:18:51.664628+0000","last_became_active":"2023-11-26T07:18:51.661410+0000","last_became_peered":"2023-11-26T07:18:51.661410+0000","last_unstale":"2023-11-26T07:18:51.664628+0000","last_undegraded":"2023-11-26T07:18:51.664628+0000","last_fullsized":"2023-11-26T07:18:51.664628+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:18:50.617605+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:18:50.617605+0000","last_clean_scrub_stamp":"2023-11-26T07:18:50.617605+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T12:04:39.473159+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.1f","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2023-11-26T07:18:51.642728+0000","last_change":"2023-11-26T07:18:51.642728+0000","last_active":"2023-11-26T07:18:51.625829+0000","last_peered":"2023-11-26T07:18:51.625829+0000","last_clean":"2023-11-26T07:18:51.625829+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-26T07:18:51.642728+0000","last_undegraded":"2023-11-26T07:18:51.642728+0000","last_fullsized":"2023-11-26T07:18:51.642728+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:18:51.625829+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:18:51.625829+0000","last_clean_scrub_stamp":"2023-11-26T07:18:51.625829+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.1e","version":"0'0","reported_seq":9,"reported_epoch":31,"state":"active+clean","last_fresh":"2023-11-26T07:18:51.656785+0000","last_change":"2023-11-26T07:18:51.652999+0000","last_active":"2023-11-26T07:18:51.656785+0000","last_peered":"2023-11-26T07:18:51.656785+0000","last_clean":"2023-11-26T07:18:51.656785+0000","last_became_active":"2023-11-26T07:18:51.652268+0000","last_became_peered":"2023-11-26T07:18:51.652268+0000","last_unstale":"2023-11-26T07:18:51.656785+0000","last_undegraded":"2023-11-26T07:18:51.656785+0000","last_fullsized":"2023-11-26T07:18:51.656785+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:18:50.617605+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:18:50.617605+0000","last_clean_scrub_stamp":"2023-11-26T07:18:50.617605+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T17:27:24.322165+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.1c","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2023-11-26T07:18:51.638001+0000","last_change":"2023-11-26T07:18:51.638001+0000","last_active":"2023-11-26T07:18:51.625829+0000","last_peered":"2023-11-26T07:18:51.625829+0000","last_clean":"2023-11-26T07:18:51.625829+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-26T07:18:51.638001+0000","last_undegraded":"2023-11-26T07:18:51.638001+0000","last_fullsized":"2023-11-26T07:18:51.638001+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:18:51.625829+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:18:51.625829+0000","last_clean_scrub_stamp":"2023-11-26T07:18:51.625829+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.1d","version":"0'0","reported_seq":9,"reported_epoch":31,"state":"active+clean","last_fresh":"2023-11-26T07:18:51.661518+0000","last_change":"2023-11-26T07:18:51.653241+0000","last_active":"2023-11-26T07:18:51.661518+0000","last_peered":"2023-11-26T07:18:51.661518+0000","last_clean":"2023-11-26T07:18:51.661518+0000","last_became_active":"2023-11-26T07:18:51.652971+0000","last_became_peered":"2023-11-26T07:18:51.652971+0000","last_unstale":"2023-11-26T07:18:51.661518+0000","last_undegraded":"2023-11-26T07:18:51.661518+0000","last_fullsized":"2023-11-26T07:18:51.661518+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:18:50.617605+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:18:50.617605+0000","last_clean_scrub_stamp":"2023-11-26T07:18:50.617605+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T10:40:39.059176+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.1d","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2023-11-26T07:18:51.633747+0000","last_change":"2023-11-26T07:18:51.633747+0000","last_active":"2023-11-26T07:18:51.625829+0000","last_peered":"2023-11-26T07:18:51.625829+0000","last_clean":"2023-11-26T07:18:51.625829+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-26T07:18:51.633747+0000","last_undegraded":"2023-11-26T07:18:51.633747+0000","last_fullsized":"2023-11-26T07:18:51.633747+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:18:51.625829+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:18:51.625829+0000","last_clean_scrub_stamp":"2023-11-26T07:18:51.625829+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1c","version":"0'0","reported_seq":9,"reported_epoch":31,"state":"active+clean","last_fresh":"2023-11-26T07:18:51.664404+0000","last_change":"2023-11-26T07:18:51.662450+0000","last_active":"2023-11-26T07:18:51.664404+0000","last_peered":"2023-11-26T07:18:51.664404+0000","last_clean":"2023-11-26T07:18:51.664404+0000","last_became_active":"2023-11-26T07:18:51.662140+0000","last_became_peered":"2023-11-26T07:18:51.662140+0000","last_unstale":"2023-11-26T07:18:51.664404+0000","last_undegraded":"2023-11-26T07:18:51.664404+0000","last_fullsized":"2023-11-26T07:18:51.664404+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:18:50.617605+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:18:50.617605+0000","last_clean_scrub_stamp":"2023-11-26T07:18:50.617605+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T17:15:12.064097+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.1a","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2023-11-26T07:18:51.640301+0000","last_change":"2023-11-26T07:18:51.640301+0000","last_active":"2023-11-26T07:18:51.625829+0000","last_peered":"2023-11-26T07:18:51.625829+0000","last_clean":"2023-11-26T07:18:51.625829+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-26T07:18:51.640301+0000","last_undegraded":"2023-11-26T07:18:51.640301+0000","last_fullsized":"2023-11-26T07:18:51.640301+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:18:51.625829+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:18:51.625829+0000","last_clean_scrub_stamp":"2023-11-26T07:18:51.625829+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.1b","version":"0'0","reported_seq":9,"reported_epoch":31,"state":"active+clean","last_fresh":"2023-11-26T07:18:51.658420+0000","last_change":"2023-11-26T07:18:51.654689+0000","last_active":"2023-11-26T07:18:51.658420+0000","last_peered":"2023-11-26T07:18:51.658420+0000","last_clean":"2023-11-26T07:18:51.658420+0000","last_became_active":"2023-11-26T07:18:51.653354+0000","last_became_peered":"2023-11-26T07:18:51.653354+0000","last_unstale":"2023-11-26T07:18:51.658420+0000","last_undegraded":"2023-11-26T07:18:51.658420+0000","last_fullsized":"2023-11-26T07:18:51.658420+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:18:50.617605+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:18:50.617605+0000","last_clean_scrub_stamp":"2023-11-26T07:18:50.617605+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T17:28:21.481079+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.1b","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2023-11-26T07:18:51.635218+0000","last_change":"2023-11-26T07:18:51.635218+0000","last_active":"2023-11-26T07:18:51.625829+0000","last_peered":"2023-11-26T07:18:51.625829+0000","last_clean":"2023-11-26T07:18:51.625829+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-26T07:18:51.635218+0000","last_undegraded":"2023-11-26T07:18:51.635218+0000","last_fullsized":"2023-11-26T07:18:51.635218+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:18:51.625829+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:18:51.625829+0000","last_clean_scrub_stamp":"2023-11-26T07:18:51.625829+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1a","version":"0'0","reported_seq":9,"reported_epoch":31,"state":"active+clean","last_fresh":"2023-11-26T07:18:51.656991+0000","last_change":"2023-11-26T07:18:51.653682+0000","last_active":"2023-11-26T07:18:51.656991+0000","last_peered":"2023-11-26T07:18:51.656991+0000","last_clean":"2023-11-26T07:18:51.656991+0000","last_became_active":"2023-11-26T07:18:51.653366+0000","last_became_peered":"2023-11-26T07:18:51.653366+0000","last_unstale":"2023-11-26T07:18:51.656991+0000","last_undegraded":"2023-11-26T07:18:51.656991+0000","last_fullsized":"2023-11-26T07:18:51.656991+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:18:50.617605+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:18:50.617605+0000","last_clean_scrub_stamp":"2023-11-26T07:18:50.617605+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T18:27:57.438972+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.18","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2023-11-26T07:18:51.634554+0000","last_change":"2023-11-26T07:18:51.634554+0000","last_active":"2023-11-26T07:18:51.625829+0000","last_peered":"2023-11-26T07:18:51.625829+0000","last_clean":"2023-11-26T07:18:51.625829+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-26T07:18:51.634554+0000","last_undegraded":"2023-11-26T07:18:51.634554+0000","last_fullsized":"2023-11-26T07:18:51.634554+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:18:51.625829+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:18:51.625829+0000","last_clean_scrub_stamp":"2023-11-26T07:18:51.625829+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.19","version":"0'0","reported_seq":9,"reported_epoch":31,"state":"active+clean","last_fresh":"2023-11-26T07:18:51.664030+0000","last_change":"2023-11-26T07:18:51.657763+0000","last_active":"2023-11-26T07:18:51.664030+0000","last_peered":"2023-11-26T07:18:51.664030+0000","last_clean":"2023-11-26T07:18:51.664030+0000","last_became_active":"2023-11-26T07:18:51.656677+0000","last_became_peered":"2023-11-26T07:18:51.656677+0000","last_unstale":"2023-11-26T07:18:51.664030+0000","last_undegraded":"2023-11-26T07:18:51.664030+0000","last_fullsized":"2023-11-26T07:18:51.664030+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:18:50.617605+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:18:50.617605+0000","last_clean_scrub_stamp":"2023-11-26T07:18:50.617605+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T18:26:18.407283+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.19","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2023-11-26T07:18:51.633130+0000","last_change":"2023-11-26T07:18:51.633130+0000","last_active":"2023-11-26T07:18:51.625829+0000","last_peered":"2023-11-26T07:18:51.625829+0000","last_clean":"2023-11-26T07:18:51.625829+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-26T07:18:51.633130+0000","last_undegraded":"2023-11-26T07:18:51.633130+0000","last_fullsized":"2023-11-26T07:18:51.633130+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:18:51.625829+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:18:51.625829+0000","last_clean_scrub_stamp":"2023-11-26T07:18:51.625829+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.18","version":"0'0","reported_seq":9,"reported_epoch":31,"state":"active+clean","last_fresh":"2023-11-26T07:18:51.658262+0000","last_change":"2023-11-26T07:18:51.656204+0000","last_active":"2023-11-26T07:18:51.658262+0000","last_peered":"2023-11-26T07:18:51.658262+0000","last_clean":"2023-11-26T07:18:51.658262+0000","last_became_active":"2023-11-26T07:18:51.655809+0000","last_became_peered":"2023-11-26T07:18:51.655809+0000","last_unstale":"2023-11-26T07:18:51.658262+0000","last_undegraded":"2023-11-26T07:18:51.658262+0000","last_fullsized":"2023-11-26T07:18:51.658262+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:18:50.617605+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:18:50.617605+0000","last_clean_scrub_stamp":"2023-11-26T07:18:50.617605+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T13:58:04.420017+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.16","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2023-11-26T07:18:51.635635+0000","last_change":"2023-11-26T07:18:51.635635+0000","last_active":"2023-11-26T07:18:51.625829+0000","last_peered":"2023-11-26T07:18:51.625829+0000","last_clean":"2023-11-26T07:18:51.625829+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-26T07:18:51.635635+0000","last_undegraded":"2023-11-26T07:18:51.635635+0000","last_fullsized":"2023-11-26T07:18:51.635635+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:18:51.625829+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:18:51.625829+0000","last_clean_scrub_stamp":"2023-11-26T07:18:51.625829+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.17","version":"0'0","reported_seq":9,"reported_epoch":31,"state":"active+clean","last_fresh":"2023-11-26T07:18:51.661481+0000","last_change":"2023-11-26T07:18:51.653322+0000","last_active":"2023-11-26T07:18:51.661481+0000","last_peered":"2023-11-26T07:18:51.661481+0000","last_clean":"2023-11-26T07:18:51.661481+0000","last_became_active":"2023-11-26T07:18:51.652388+0000","last_became_peered":"2023-11-26T07:18:51.652388+0000","last_unstale":"2023-11-26T07:18:51.661481+0000","last_undegraded":"2023-11-26T07:18:51.661481+0000","last_fullsized":"2023-11-26T07:18:51.661481+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:18:50.617605+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:18:50.617605+0000","last_clean_scrub_stamp":"2023-11-26T07:18:50.617605+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T07:50:54.298421+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.17","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2023-11-26T07:18:51.642485+0000","last_change":"2023-11-26T07:18:51.642485+0000","last_active":"2023-11-26T07:18:51.625829+0000","last_peered":"2023-11-26T07:18:51.625829+0000","last_clean":"2023-11-26T07:18:51.625829+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-26T07:18:51.642485+0000","last_undegraded":"2023-11-26T07:18:51.642485+0000","last_fullsized":"2023-11-26T07:18:51.642485+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:18:51.625829+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:18:51.625829+0000","last_clean_scrub_stamp":"2023-11-26T07:18:51.625829+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.16","version":"0'0","reported_seq":9,"reported_epoch":31,"state":"active+clean","last_fresh":"2023-11-26T07:18:51.650667+0000","last_change":"2023-11-26T07:18:51.644845+0000","last_active":"2023-11-26T07:18:51.650667+0000","last_peered":"2023-11-26T07:18:51.650667+0000","last_clean":"2023-11-26T07:18:51.650667+0000","last_became_active":"2023-11-26T07:18:51.644261+0000","last_became_peered":"2023-11-26T07:18:51.644261+0000","last_unstale":"2023-11-26T07:18:51.650667+0000","last_undegraded":"2023-11-26T07:18:51.650667+0000","last_fullsized":"2023-11-26T07:18:51.650667+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:18:50.617605+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:18:50.617605+0000","last_clean_scrub_stamp":"2023-11-26T07:18:50.617605+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T18:26:18.407283+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.14","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2023-11-26T07:18:51.643692+0000","last_change":"2023-11-26T07:18:51.643692+0000","last_active":"2023-11-26T07:18:51.625829+0000","last_peered":"2023-11-26T07:18:51.625829+0000","last_clean":"2023-11-26T07:18:51.625829+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-26T07:18:51.643692+0000","last_undegraded":"2023-11-26T07:18:51.643692+0000","last_fullsized":"2023-11-26T07:18:51.643692+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:18:51.625829+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:18:51.625829+0000","last_clean_scrub_stamp":"2023-11-26T07:18:51.625829+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.15","version":"0'0","reported_seq":9,"reported_epoch":31,"state":"active+clean","last_fresh":"2023-11-26T07:18:51.660677+0000","last_change":"2023-11-26T07:18:51.657428+0000","last_active":"2023-11-26T07:18:51.660677+0000","last_peered":"2023-11-26T07:18:51.660677+0000","last_clean":"2023-11-26T07:18:51.660677+0000","last_became_active":"2023-11-26T07:18:51.657075+0000","last_became_peered":"2023-11-26T07:18:51.657075+0000","last_unstale":"2023-11-26T07:18:51.660677+0000","last_undegraded":"2023-11-26T07:18:51.660677+0000","last_fullsized":"2023-11-26T07:18:51.660677+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:18:50.617605+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:18:50.617605+0000","last_clean_scrub_stamp":"2023-11-26T07:18:50.617605+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T07:50:54.298421+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.15","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2023-11-26T07:18:51.638905+0000","last_change":"2023-11-26T07:18:51.638905+0000","last_active":"2023-11-26T07:18:51.625829+0000","last_peered":"2023-11-26T07:18:51.625829+0000","last_clean":"2023-11-26T07:18:51.625829+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-26T07:18:51.638905+0000","last_undegraded":"2023-11-26T07:18:51.638905+0000","last_fullsized":"2023-11-26T07:18:51.638905+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:18:51.625829+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:18:51.625829+0000","last_clean_scrub_stamp":"2023-11-26T07:18:51.625829+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.14","version":"0'0","reported_seq":9,"reported_epoch":31,"state":"active+clean","last_fresh":"2023-11-26T07:18:51.651808+0000","last_change":"2023-11-26T07:18:51.647209+0000","last_active":"2023-11-26T07:18:51.651808+0000","last_peered":"2023-11-26T07:18:51.651808+0000","last_clean":"2023-11-26T07:18:51.651808+0000","last_became_active":"2023-11-26T07:18:51.647051+0000","last_became_peered":"2023-11-26T07:18:51.647051+0000","last_unstale":"2023-11-26T07:18:51.651808+0000","last_undegraded":"2023-11-26T07:18:51.651808+0000","last_fullsized":"2023-11-26T07:18:51.651808+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:18:50.617605+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:18:50.617605+0000","last_clean_scrub_stamp":"2023-11-26T07:18:50.617605+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T14:40:15.233288+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.12","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2023-11-26T07:18:51.633529+0000","last_change":"2023-11-26T07:18:51.633529+0000","last_active":"2023-11-26T07:18:51.625829+0000","last_peered":"2023-11-26T07:18:51.625829+0000","last_clean":"2023-11-26T07:18:51.625829+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-26T07:18:51.633529+0000","last_undegraded":"2023-11-26T07:18:51.633529+0000","last_fullsized":"2023-11-26T07:18:51.633529+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:18:51.625829+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:18:51.625829+0000","last_clean_scrub_stamp":"2023-11-26T07:18:51.625829+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.13","version":"0'0","reported_seq":9,"reported_epoch":31,"state":"active+clean","last_fresh":"2023-11-26T07:18:51.659054+0000","last_change":"2023-11-26T07:18:51.656339+0000","last_active":"2023-11-26T07:18:51.659054+0000","last_peered":"2023-11-26T07:18:51.659054+0000","last_clean":"2023-11-26T07:18:51.659054+0000","last_became_active":"2023-11-26T07:18:51.655998+0000","last_became_peered":"2023-11-26T07:18:51.655998+0000","last_unstale":"2023-11-26T07:18:51.659054+0000","last_undegraded":"2023-11-26T07:18:51.659054+0000","last_fullsized":"2023-11-26T07:18:51.659054+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:18:50.617605+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:18:50.617605+0000","last_clean_scrub_stamp":"2023-11-26T07:18:50.617605+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T11:29:07.700412+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.13","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2023-11-26T07:18:51.642640+0000","last_change":"2023-11-26T07:18:51.642640+0000","last_active":"2023-11-26T07:18:51.625829+0000","last_peered":"2023-11-26T07:18:51.625829+0000","last_clean":"2023-11-26T07:18:51.625829+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-26T07:18:51.642640+0000","last_undegraded":"2023-11-26T07:18:51.642640+0000","last_fullsized":"2023-11-26T07:18:51.642640+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:18:51.625829+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:18:51.625829+0000","last_clean_scrub_stamp":"2023-11-26T07:18:51.625829+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.12","version":"0'0","reported_seq":9,"reported_epoch":31,"state":"active+clean","last_fresh":"2023-11-26T07:18:51.663995+0000","last_change":"2023-11-26T07:18:51.658016+0000","last_active":"2023-11-26T07:18:51.663995+0000","last_peered":"2023-11-26T07:18:51.663995+0000","last_clean":"2023-11-26T07:18:51.663995+0000","last_became_active":"2023-11-26T07:18:51.657646+0000","last_became_peered":"2023-11-26T07:18:51.657646+0000","last_unstale":"2023-11-26T07:18:51.663995+0000","last_undegraded":"2023-11-26T07:18:51.663995+0000","last_fullsized":"2023-11-26T07:18:51.663995+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:18:50.617605+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:18:50.617605+0000","last_clean_scrub_stamp":"2023-11-26T07:18:50.617605+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T19:10:39.237269+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.10","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2023-11-26T07:18:51.645524+0000","last_change":"2023-11-26T07:18:51.645524+0000","last_active":"2023-11-26T07:18:51.625829+0000","last_peered":"2023-11-26T07:18:51.625829+0000","last_clean":"2023-11-26T07:18:51.625829+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-26T07:18:51.645524+0000","last_undegraded":"2023-11-26T07:18:51.645524+0000","last_fullsized":"2023-11-26T07:18:51.645524+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:18:51.625829+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:18:51.625829+0000","last_clean_scrub_stamp":"2023-11-26T07:18:51.625829+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.11","version":"0'0","reported_seq":9,"reported_epoch":31,"state":"active+clean","last_fresh":"2023-11-26T07:18:51.661545+0000","last_change":"2023-11-26T07:18:51.654685+0000","last_active":"2023-11-26T07:18:51.661545+0000","last_peered":"2023-11-26T07:18:51.661545+0000","last_clean":"2023-11-26T07:18:51.661545+0000","last_became_active":"2023-11-26T07:18:51.653542+0000","last_became_peered":"2023-11-26T07:18:51.653542+0000","last_unstale":"2023-11-26T07:18:51.661545+0000","last_undegraded":"2023-11-26T07:18:51.661545+0000","last_fullsized":"2023-11-26T07:18:51.661545+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:18:50.617605+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:18:50.617605+0000","last_clean_scrub_stamp":"2023-11-26T07:18:50.617605+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T14:17:17.432332+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.11","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2023-11-26T07:18:51.642391+0000","last_change":"2023-11-26T07:18:51.642391+0000","last_active":"2023-11-26T07:18:51.625829+0000","last_peered":"2023-11-26T07:18:51.625829+0000","last_clean":"2023-11-26T07:18:51.625829+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-26T07:18:51.642391+0000","last_undegraded":"2023-11-26T07:18:51.642391+0000","last_fullsized":"2023-11-26T07:18:51.642391+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:18:51.625829+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:18:51.625829+0000","last_clean_scrub_stamp":"2023-11-26T07:18:51.625829+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.10","version":"0'0","reported_seq":9,"reported_epoch":31,"state":"active+clean","last_fresh":"2023-11-26T07:18:51.661372+0000","last_change":"2023-11-26T07:18:51.652858+0000","last_active":"2023-11-26T07:18:51.661372+0000","last_peered":"2023-11-26T07:18:51.661372+0000","last_clean":"2023-11-26T07:18:51.661372+0000","last_became_active":"2023-11-26T07:18:51.652583+0000","last_became_peered":"2023-11-26T07:18:51.652583+0000","last_unstale":"2023-11-26T07:18:51.661372+0000","last_undegraded":"2023-11-26T07:18:51.661372+0000","last_fullsized":"2023-11-26T07:18:51.661372+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:18:50.617605+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:18:50.617605+0000","last_clean_scrub_stamp":"2023-11-26T07:18:50.617605+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T14:49:28.420672+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.e","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2023-11-26T07:18:51.635911+0000","last_change":"2023-11-26T07:18:51.635911+0000","last_active":"2023-11-26T07:18:51.625829+0000","last_peered":"2023-11-26T07:18:51.625829+0000","last_clean":"2023-11-26T07:18:51.625829+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-26T07:18:51.635911+0000","last_undegraded":"2023-11-26T07:18:51.635911+0000","last_fullsized":"2023-11-26T07:18:51.635911+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:18:51.625829+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:18:51.625829+0000","last_clean_scrub_stamp":"2023-11-26T07:18:51.625829+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.f","version":"0'0","reported_seq":9,"reported_epoch":31,"state":"active+clean","last_fresh":"2023-11-26T07:18:51.661667+0000","last_change":"2023-11-26T07:18:51.656212+0000","last_active":"2023-11-26T07:18:51.661667+0000","last_peered":"2023-11-26T07:18:51.661667+0000","last_clean":"2023-11-26T07:18:51.661667+0000","last_became_active":"2023-11-26T07:18:51.650720+0000","last_became_peered":"2023-11-26T07:18:51.650720+0000","last_unstale":"2023-11-26T07:18:51.661667+0000","last_undegraded":"2023-11-26T07:18:51.661667+0000","last_fullsized":"2023-11-26T07:18:51.661667+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:18:50.617605+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:18:50.617605+0000","last_clean_scrub_stamp":"2023-11-26T07:18:50.617605+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T14:40:15.233288+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.f","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2023-11-26T07:18:51.641273+0000","last_change":"2023-11-26T07:18:51.641273+0000","last_active":"2023-11-26T07:18:51.625829+0000","last_peered":"2023-11-26T07:18:51.625829+0000","last_clean":"2023-11-26T07:18:51.625829+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-26T07:18:51.641273+0000","last_undegraded":"2023-11-26T07:18:51.641273+0000","last_fullsized":"2023-11-26T07:18:51.641273+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:18:51.625829+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:18:51.625829+0000","last_clean_scrub_stamp":"2023-11-26T07:18:51.625829+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.e","version":"0'0","reported_seq":9,"reported_epoch":31,"state":"active+clean","last_fresh":"2023-11-26T07:18:51.664526+0000","last_change":"2023-11-26T07:18:51.660325+0000","last_active":"2023-11-26T07:18:51.664526+0000","last_peered":"2023-11-26T07:18:51.664526+0000","last_clean":"2023-11-26T07:18:51.664526+0000","last_became_active":"2023-11-26T07:18:51.659918+0000","last_became_peered":"2023-11-26T07:18:51.659918+0000","last_unstale":"2023-11-26T07:18:51.664526+0000","last_undegraded":"2023-11-26T07:18:51.664526+0000","last_fullsized":"2023-11-26T07:18:51.664526+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:18:50.617605+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:18:50.617605+0000","last_clean_scrub_stamp":"2023-11-26T07:18:50.617605+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T15:20:53.245654+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.c","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2023-11-26T07:18:51.633505+0000","last_change":"2023-11-26T07:18:51.633505+0000","last_active":"2023-11-26T07:18:51.625829+0000","last_peered":"2023-11-26T07:18:51.625829+0000","last_clean":"2023-11-26T07:18:51.625829+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-26T07:18:51.633505+0000","last_undegraded":"2023-11-26T07:18:51.633505+0000","last_fullsized":"2023-11-26T07:18:51.633505+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:18:51.625829+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:18:51.625829+0000","last_clean_scrub_stamp":"2023-11-26T07:18:51.625829+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.d","version":"0'0","reported_seq":9,"reported_epoch":31,"state":"active+clean","last_fresh":"2023-11-26T07:18:51.662230+0000","last_change":"2023-11-26T07:18:51.658440+0000","last_active":"2023-11-26T07:18:51.662230+0000","last_peered":"2023-11-26T07:18:51.662230+0000","last_clean":"2023-11-26T07:18:51.662230+0000","last_became_active":"2023-11-26T07:18:51.658161+0000","last_became_peered":"2023-11-26T07:18:51.658161+0000","last_unstale":"2023-11-26T07:18:51.662230+0000","last_undegraded":"2023-11-26T07:18:51.662230+0000","last_fullsized":"2023-11-26T07:18:51.662230+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:18:50.617605+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:18:50.617605+0000","last_clean_scrub_stamp":"2023-11-26T07:18:50.617605+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T08:54:02.907365+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.d","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2023-11-26T07:18:51.653086+0000","last_change":"2023-11-26T07:18:51.653086+0000","last_active":"2023-11-26T07:18:51.625829+0000","last_peered":"2023-11-26T07:18:51.625829+0000","last_clean":"2023-11-26T07:18:51.625829+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-26T07:18:51.653086+0000","last_undegraded":"2023-11-26T07:18:51.653086+0000","last_fullsized":"2023-11-26T07:18:51.653086+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:18:51.625829+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:18:51.625829+0000","last_clean_scrub_stamp":"2023-11-26T07:18:51.625829+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.c","version":"0'0","reported_seq":9,"reported_epoch":31,"state":"active+clean","last_fresh":"2023-11-26T07:18:51.658594+0000","last_change":"2023-11-26T07:18:51.654164+0000","last_active":"2023-11-26T07:18:51.658594+0000","last_peered":"2023-11-26T07:18:51.658594+0000","last_clean":"2023-11-26T07:18:51.658594+0000","last_became_active":"2023-11-26T07:18:51.653823+0000","last_became_peered":"2023-11-26T07:18:51.653823+0000","last_unstale":"2023-11-26T07:18:51.658594+0000","last_undegraded":"2023-11-26T07:18:51.658594+0000","last_fullsized":"2023-11-26T07:18:51.658594+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:18:50.617605+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:18:50.617605+0000","last_clean_scrub_stamp":"2023-11-26T07:18:50.617605+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T12:22:07.968357+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.a","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2023-11-26T07:18:51.649002+0000","last_change":"2023-11-26T07:18:51.649002+0000","last_active":"2023-11-26T07:18:51.625829+0000","last_peered":"2023-11-26T07:18:51.625829+0000","last_clean":"2023-11-26T07:18:51.625829+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-26T07:18:51.649002+0000","last_undegraded":"2023-11-26T07:18:51.649002+0000","last_fullsized":"2023-11-26T07:18:51.649002+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:18:51.625829+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:18:51.625829+0000","last_clean_scrub_stamp":"2023-11-26T07:18:51.625829+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.b","version":"0'0","reported_seq":9,"reported_epoch":31,"state":"active+clean","last_fresh":"2023-11-26T07:18:51.655810+0000","last_change":"2023-11-26T07:18:51.646694+0000","last_active":"2023-11-26T07:18:51.655810+0000","last_peered":"2023-11-26T07:18:51.655810+0000","last_clean":"2023-11-26T07:18:51.655810+0000","last_became_active":"2023-11-26T07:18:51.644567+0000","last_became_peered":"2023-11-26T07:18:51.644567+0000","last_unstale":"2023-11-26T07:18:51.655810+0000","last_undegraded":"2023-11-26T07:18:51.655810+0000","last_fullsized":"2023-11-26T07:18:51.655810+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:18:50.617605+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:18:50.617605+0000","last_clean_scrub_stamp":"2023-11-26T07:18:50.617605+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T12:04:39.473159+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.b","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2023-11-26T07:18:51.649020+0000","last_change":"2023-11-26T07:18:51.649020+0000","last_active":"2023-11-26T07:18:51.625829+0000","last_peered":"2023-11-26T07:18:51.625829+0000","last_clean":"2023-11-26T07:18:51.625829+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-26T07:18:51.649020+0000","last_undegraded":"2023-11-26T07:18:51.649020+0000","last_fullsized":"2023-11-26T07:18:51.649020+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:18:51.625829+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:18:51.625829+0000","last_clean_scrub_stamp":"2023-11-26T07:18:51.625829+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.a","version":"0'0","reported_seq":9,"reported_epoch":31,"state":"active+clean","last_fresh":"2023-11-26T07:18:51.664562+0000","last_change":"2023-11-26T07:18:51.660988+0000","last_active":"2023-11-26T07:18:51.664562+0000","last_peered":"2023-11-26T07:18:51.664562+0000","last_clean":"2023-11-26T07:18:51.664562+0000","last_became_active":"2023-11-26T07:18:51.660676+0000","last_became_peered":"2023-11-26T07:18:51.660676+0000","last_unstale":"2023-11-26T07:18:51.664562+0000","last_undegraded":"2023-11-26T07:18:51.664562+0000","last_fullsized":"2023-11-26T07:18:51.664562+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:18:50.617605+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:18:50.617605+0000","last_clean_scrub_stamp":"2023-11-26T07:18:50.617605+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T19:10:39.237269+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.8","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2023-11-26T07:18:51.649368+0000","last_change":"2023-11-26T07:18:51.649368+0000","last_active":"2023-11-26T07:18:51.625829+0000","last_peered":"2023-11-26T07:18:51.625829+0000","last_clean":"2023-11-26T07:18:51.625829+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-26T07:18:51.649368+0000","last_undegraded":"2023-11-26T07:18:51.649368+0000","last_fullsized":"2023-11-26T07:18:51.649368+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:18:51.625829+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:18:51.625829+0000","last_clean_scrub_stamp":"2023-11-26T07:18:51.625829+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.9","version":"0'0","reported_seq":9,"reported_epoch":31,"state":"active+clean","last_fresh":"2023-11-26T07:18:51.657644+0000","last_change":"2023-11-26T07:18:51.652977+0000","last_active":"2023-11-26T07:18:51.657644+0000","last_peered":"2023-11-26T07:18:51.657644+0000","last_clean":"2023-11-26T07:18:51.657644+0000","last_became_active":"2023-11-26T07:18:51.652670+0000","last_became_peered":"2023-11-26T07:18:51.652670+0000","last_unstale":"2023-11-26T07:18:51.657644+0000","last_undegraded":"2023-11-26T07:18:51.657644+0000","last_fullsized":"2023-11-26T07:18:51.657644+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:18:50.617605+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:18:50.617605+0000","last_clean_scrub_stamp":"2023-11-26T07:18:50.617605+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T15:39:09.212431+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.9","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2023-11-26T07:18:51.642082+0000","last_change":"2023-11-26T07:18:51.642082+0000","last_active":"2023-11-26T07:18:51.625829+0000","last_peered":"2023-11-26T07:18:51.625829+0000","last_clean":"2023-11-26T07:18:51.625829+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-26T07:18:51.642082+0000","last_undegraded":"2023-11-26T07:18:51.642082+0000","last_fullsized":"2023-11-26T07:18:51.642082+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:18:51.625829+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:18:51.625829+0000","last_clean_scrub_stamp":"2023-11-26T07:18:51.625829+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.8","version":"0'0","reported_seq":9,"reported_epoch":31,"state":"active+clean","last_fresh":"2023-11-26T07:18:51.662484+0000","last_change":"2023-11-26T07:18:51.659037+0000","last_active":"2023-11-26T07:18:51.662484+0000","last_peered":"2023-11-26T07:18:51.662484+0000","last_clean":"2023-11-26T07:18:51.662484+0000","last_became_active":"2023-11-26T07:18:51.658596+0000","last_became_peered":"2023-11-26T07:18:51.658596+0000","last_unstale":"2023-11-26T07:18:51.662484+0000","last_undegraded":"2023-11-26T07:18:51.662484+0000","last_fullsized":"2023-11-26T07:18:51.662484+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:18:50.617605+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:18:50.617605+0000","last_clean_scrub_stamp":"2023-11-26T07:18:50.617605+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T10:40:39.059176+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.6","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2023-11-26T07:18:51.651544+0000","last_change":"2023-11-26T07:18:51.651544+0000","last_active":"2023-11-26T07:18:51.625829+0000","last_peered":"2023-11-26T07:18:51.625829+0000","last_clean":"2023-11-26T07:18:51.625829+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-26T07:18:51.651544+0000","last_undegraded":"2023-11-26T07:18:51.651544+0000","last_fullsized":"2023-11-26T07:18:51.651544+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:18:51.625829+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:18:51.625829+0000","last_clean_scrub_stamp":"2023-11-26T07:18:51.625829+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.7","version":"0'0","reported_seq":9,"reported_epoch":31,"state":"active+clean","last_fresh":"2023-11-26T07:18:51.661322+0000","last_change":"2023-11-26T07:18:51.653789+0000","last_active":"2023-11-26T07:18:51.661322+0000","last_peered":"2023-11-26T07:18:51.661322+0000","last_clean":"2023-11-26T07:18:51.661322+0000","last_became_active":"2023-11-26T07:18:51.652136+0000","last_became_peered":"2023-11-26T07:18:51.652136+0000","last_unstale":"2023-11-26T07:18:51.661322+0000","last_undegraded":"2023-11-26T07:18:51.661322+0000","last_fullsized":"2023-11-26T07:18:51.661322+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:18:50.617605+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:18:50.617605+0000","last_clean_scrub_stamp":"2023-11-26T07:18:50.617605+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T18:26:18.407283+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.7","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2023-11-26T07:18:51.637655+0000","last_change":"2023-11-26T07:18:51.637655+0000","last_active":"2023-11-26T07:18:51.625829+0000","last_peered":"2023-11-26T07:18:51.625829+0000","last_clean":"2023-11-26T07:18:51.625829+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-26T07:18:51.637655+0000","last_undegraded":"2023-11-26T07:18:51.637655+0000","last_fullsized":"2023-11-26T07:18:51.637655+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:18:51.625829+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:18:51.625829+0000","last_clean_scrub_stamp":"2023-11-26T07:18:51.625829+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.6","version":"0'0","reported_seq":9,"reported_epoch":31,"state":"active+clean","last_fresh":"2023-11-26T07:18:51.648265+0000","last_change":"2023-11-26T07:18:51.644857+0000","last_active":"2023-11-26T07:18:51.648265+0000","last_peered":"2023-11-26T07:18:51.648265+0000","last_clean":"2023-11-26T07:18:51.648265+0000","last_became_active":"2023-11-26T07:18:51.644616+0000","last_became_peered":"2023-11-26T07:18:51.644616+0000","last_unstale":"2023-11-26T07:18:51.648265+0000","last_undegraded":"2023-11-26T07:18:51.648265+0000","last_fullsized":"2023-11-26T07:18:51.648265+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:18:50.617605+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:18:50.617605+0000","last_clean_scrub_stamp":"2023-11-26T07:18:50.617605+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T14:40:15.233288+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.5","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2023-11-26T07:18:51.641452+0000","last_change":"2023-11-26T07:18:51.641452+0000","last_active":"2023-11-26T07:18:51.625829+0000","last_peered":"2023-11-26T07:18:51.625829+0000","last_clean":"2023-11-26T07:18:51.625829+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-26T07:18:51.641452+0000","last_undegraded":"2023-11-26T07:18:51.641452+0000","last_fullsized":"2023-11-26T07:18:51.641452+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:18:51.625829+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:18:51.625829+0000","last_clean_scrub_stamp":"2023-11-26T07:18:51.625829+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.4","version":"0'0","reported_seq":9,"reported_epoch":31,"state":"active+clean","last_fresh":"2023-11-26T07:18:51.664083+0000","last_change":"2023-11-26T07:18:51.660079+0000","last_active":"2023-11-26T07:18:51.664083+0000","last_peered":"2023-11-26T07:18:51.664083+0000","last_clean":"2023-11-26T07:18:51.664083+0000","last_became_active":"2023-11-26T07:18:51.659661+0000","last_became_peered":"2023-11-26T07:18:51.659661+0000","last_unstale":"2023-11-26T07:18:51.664083+0000","last_undegraded":"2023-11-26T07:18:51.664083+0000","last_fullsized":"2023-11-26T07:18:51.664083+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:18:50.617605+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:18:50.617605+0000","last_clean_scrub_stamp":"2023-11-26T07:18:50.617605+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T12:04:39.473159+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.2","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2023-11-26T07:18:51.642362+0000","last_change":"2023-11-26T07:18:51.642362+0000","last_active":"2023-11-26T07:18:51.625829+0000","last_peered":"2023-11-26T07:18:51.625829+0000","last_clean":"2023-11-26T07:18:51.625829+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-26T07:18:51.642362+0000","last_undegraded":"2023-11-26T07:18:51.642362+0000","last_fullsized":"2023-11-26T07:18:51.642362+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:18:51.625829+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:18:51.625829+0000","last_clean_scrub_stamp":"2023-11-26T07:18:51.625829+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.3","version":"0'0","reported_seq":9,"reported_epoch":31,"state":"active+clean","last_fresh":"2023-11-26T07:18:51.655830+0000","last_change":"2023-11-26T07:18:51.647332+0000","last_active":"2023-11-26T07:18:51.655830+0000","last_peered":"2023-11-26T07:18:51.655830+0000","last_clean":"2023-11-26T07:18:51.655830+0000","last_became_active":"2023-11-26T07:18:51.647054+0000","last_became_peered":"2023-11-26T07:18:51.647054+0000","last_unstale":"2023-11-26T07:18:51.655830+0000","last_undegraded":"2023-11-26T07:18:51.655830+0000","last_fullsized":"2023-11-26T07:18:51.655830+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:18:50.617605+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:18:50.617605+0000","last_clean_scrub_stamp":"2023-11-26T07:18:50.617605+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T19:10:39.237269+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.3","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2023-11-26T07:18:51.642003+0000","last_change":"2023-11-26T07:18:51.642003+0000","last_active":"2023-11-26T07:18:51.625829+0000","last_peered":"2023-11-26T07:18:51.625829+0000","last_clean":"2023-11-26T07:18:51.625829+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-26T07:18:51.642003+0000","last_undegraded":"2023-11-26T07:18:51.642003+0000","last_fullsized":"2023-11-26T07:18:51.642003+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:18:51.625829+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:18:51.625829+0000","last_clean_scrub_stamp":"2023-11-26T07:18:51.625829+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.2","version":"0'0","reported_seq":9,"reported_epoch":31,"state":"active+clean","last_fresh":"2023-11-26T07:18:51.658369+0000","last_change":"2023-11-26T07:18:51.651994+0000","last_active":"2023-11-26T07:18:51.658369+0000","last_peered":"2023-11-26T07:18:51.658369+0000","last_clean":"2023-11-26T07:18:51.658369+0000","last_became_active":"2023-11-26T07:18:51.650255+0000","last_became_peered":"2023-11-26T07:18:51.650255+0000","last_unstale":"2023-11-26T07:18:51.658369+0000","last_undegraded":"2023-11-26T07:18:51.658369+0000","last_fullsized":"2023-11-26T07:18:51.658369+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:18:50.617605+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:18:50.617605+0000","last_clean_scrub_stamp":"2023-11-26T07:18:50.617605+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T17:27:24.322165+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.0","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2023-11-26T07:18:51.642260+0000","last_change":"2023-11-26T07:18:51.642260+0000","last_active":"2023-11-26T07:18:51.625829+0000","last_peered":"2023-11-26T07:18:51.625829+0000","last_clean":"2023-11-26T07:18:51.625829+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-26T07:18:51.642260+0000","last_undegraded":"2023-11-26T07:18:51.642260+0000","last_fullsized":"2023-11-26T07:18:51.642260+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:18:51.625829+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:18:51.625829+0000","last_clean_scrub_stamp":"2023-11-26T07:18:51.625829+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1","version":"0'0","reported_seq":9,"reported_epoch":31,"state":"active+clean","last_fresh":"2023-11-26T07:18:51.658380+0000","last_change":"2023-11-26T07:18:51.654435+0000","last_active":"2023-11-26T07:18:51.658380+0000","last_peered":"2023-11-26T07:18:51.658380+0000","last_clean":"2023-11-26T07:18:51.658380+0000","last_became_active":"2023-11-26T07:18:51.654107+0000","last_became_peered":"2023-11-26T07:18:51.654107+0000","last_unstale":"2023-11-26T07:18:51.658380+0000","last_undegraded":"2023-11-26T07:18:51.658380+0000","last_fullsized":"2023-11-26T07:18:51.658380+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:18:50.617605+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:18:50.617605+0000","last_clean_scrub_stamp":"2023-11-26T07:18:50.617605+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T15:50:37.023527+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.1","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2023-11-26T07:18:51.635350+0000","last_change":"2023-11-26T07:18:51.635350+0000","last_active":"2023-11-26T07:18:51.625829+0000","last_peered":"2023-11-26T07:18:51.625829+0000","last_clean":"2023-11-26T07:18:51.625829+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-26T07:18:51.635350+0000","last_undegraded":"2023-11-26T07:18:51.635350+0000","last_fullsized":"2023-11-26T07:18:51.635350+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:18:51.625829+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:18:51.625829+0000","last_clean_scrub_stamp":"2023-11-26T07:18:51.625829+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.0","version":"0'0","reported_seq":9,"reported_epoch":31,"state":"active+clean","last_fresh":"2023-11-26T07:18:51.661597+0000","last_change":"2023-11-26T07:18:51.655721+0000","last_active":"2023-11-26T07:18:51.661597+0000","last_peered":"2023-11-26T07:18:51.661597+0000","last_clean":"2023-11-26T07:18:51.661597+0000","last_became_active":"2023-11-26T07:18:51.655334+0000","last_became_peered":"2023-11-26T07:18:51.655334+0000","last_unstale":"2023-11-26T07:18:51.661597+0000","last_undegraded":"2023-11-26T07:18:51.661597+0000","last_fullsized":"2023-11-26T07:18:51.661597+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:18:50.617605+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:18:50.617605+0000","last_clean_scrub_stamp":"2023-11-26T07:18:50.617605+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T11:49:26.615849+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.4","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2023-11-26T07:18:51.643304+0000","last_change":"2023-11-26T07:18:51.643304+0000","last_active":"2023-11-26T07:18:51.625829+0000","last_peered":"2023-11-26T07:18:51.625829+0000","last_clean":"2023-11-26T07:18:51.625829+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-26T07:18:51.643304+0000","last_undegraded":"2023-11-26T07:18:51.643304+0000","last_fullsized":"2023-11-26T07:18:51.643304+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:18:51.625829+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:18:51.625829+0000","last_clean_scrub_stamp":"2023-11-26T07:18:51.625829+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.5","version":"0'0","reported_seq":9,"reported_epoch":31,"state":"active+clean","last_fresh":"2023-11-26T07:18:51.664594+0000","last_change":"2023-11-26T07:18:51.661135+0000","last_active":"2023-11-26T07:18:51.664594+0000","last_peered":"2023-11-26T07:18:51.664594+0000","last_clean":"2023-11-26T07:18:51.664594+0000","last_became_active":"2023-11-26T07:18:51.659814+0000","last_became_peered":"2023-11-26T07:18:51.659814+0000","last_unstale":"2023-11-26T07:18:51.664594+0000","last_undegraded":"2023-11-26T07:18:51.664594+0000","last_fullsized":"2023-11-26T07:18:51.664594+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:18:50.617605+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:18:50.617605+0000","last_clean_scrub_stamp":"2023-11-26T07:18:50.617605+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T17:37:35.256892+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"1.0","version":"15'117","reported_seq":173,"reported_epoch":30,"state":"active+clean","last_fresh":"2023-11-26T07:18:50.636811+0000","last_change":"2023-11-26T07:17:43.827802+0000","last_active":"2023-11-26T07:18:50.636811+0000","last_peered":"2023-11-26T07:18:50.636811+0000","last_clean":"2023-11-26T07:18:50.636811+0000","last_became_active":"2023-11-26T07:17:43.827308+0000","last_became_peered":"2023-11-26T07:17:43.827308+0000","last_unstale":"2023-11-26T07:18:50.636811+0000","last_undegraded":"2023-11-26T07:18:50.636811+0000","last_fullsized":"2023-11-26T07:18:50.636811+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:17:42.160945+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:17:42.160945+0000","last_clean_scrub_stamp":"2023-11-26T07:17:42.160945+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T08:17:54.167366+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":5,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":4,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574864,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27932,"kb_used_data":908,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343908,"statfs":{"total":96636764160,"available":96608161792,"internally_reserved":0,"allocated":929792,"data_stored":717618,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Nov 26 07:18:42 2023","interfaces":[{"interface":"back","average":{"1min":0.71499999999999997,"5min":0.71499999999999997,"15min":0.71499999999999997},"min":{"1min":0.54300000000000004,"5min":0.54300000000000004,"15min":0.54300000000000004},"max":{"1min":0.91500000000000004,"5min":0.91500000000000004,"15min":0.91500000000000004},"last":0.60099999999999998},{"interface":"front","average":{"1min":0.70899999999999996,"5min":0.70899999999999996,"15min":0.70899999999999996},"min":{"1min":0.52900000000000003,"5min":0.52900000000000003,"15min":0.52900000000000003},"max":{"1min":1.0980000000000001,"5min":1.0980000000000001,"15min":1.0980000000000001},"last":0.69199999999999995}]},{"osd":1,"last update":"Sun Nov 26 07:18:42 2023","interfaces":[{"interface":"back","average":{"1min":0.68200000000000005,"5min":0.68200000000000005,"15min":0.68200000000000005},"min":{"1min":0.47999999999999998,"5min":0.47999999999999998,"15min":0.47999999999999998},"max":{"1min":1.0669999999999999,"5min":1.0669999999999999,"15min":1.0669999999999999},"last":0.67300000000000004},{"interface":"front","average":{"1min":0.83899999999999997,"5min":0.83899999999999997,"15min":0.83899999999999997},"min":{"1min":0.47599999999999998,"5min":0.47599999999999998,"15min":0.47599999999999998},"max":{"1min":4.5309999999999997,"5min":4.5309999999999997,"15min":4.5309999999999997},"last":0.627}]},{"osd":2,"last update":"Sun Nov 26 07:18:42 2023","interfaces":[{"interface":"back","average":{"1min":0.67600000000000005,"5min":0.67600000000000005,"15min":0.67600000000000005},"min":{"1min":0.52000000000000002,"5min":0.52000000000000002,"15min":0.52000000000000002},"max":{"1min":0.873,"5min":0.873,"15min":0.873},"last":0.54400000000000004},{"interface":"front","average":{"1min":0.71299999999999997,"5min":0.71299999999999997,"15min":0.71299999999999997},"min":{"1min":0.54700000000000004,"5min":0.54700000000000004,"15min":0.54700000000000004},"max":{"1min":1.1339999999999999,"5min":1.1339999999999999,"15min":1.1339999999999999},"last":0.499}]},{"osd":3,"last update":"Sun Nov 26 07:18:42 2023","interfaces":[{"interface":"back","average":{"1min":0.69699999999999995,"5min":0.69699999999999995,"15min":0.69699999999999995},"min":{"1min":0.51800000000000002,"5min":0.51800000000000002,"15min":0.51800000000000002},"max":{"1min":1.026,"5min":1.026,"15min":1.026},"last":0.64300000000000002},{"interface":"front","average":{"1min":0.73299999999999998,"5min":0.73299999999999998,"15min":0.73299999999999998},"min":{"1min":0.50600000000000001,"5min":0.50600000000000001,"15min":0.50600000000000001},"max":{"1min":1.4490000000000001,"5min":1.4490000000000001,"15min":1.4490000000000001},"last":0.755}]},{"osd":4,"last update":"Sun Nov 26 07:18:42 2023","interfaces":[{"interface":"back","average":{"1min":0.59999999999999998,"5min":0.59999999999999998,"15min":0.59999999999999998},"min":{"1min":0.39400000000000002,"5min":0.39400000000000002,"15min":0.39400000000000002},"max":{"1min":0.94799999999999995,"5min":0.94799999999999995,"15min":0.94799999999999995},"last":0.47099999999999997},{"interface":"front","average":{"1min":0.67300000000000004,"5min":0.67300000000000004,"15min":0.67300000000000004},"min":{"1min":0.436,"5min":0.436,"15min":0.436},"max":{"1min":1.5509999999999999,"5min":1.5509999999999999,"15min":1.5509999999999999},"last":0.70899999999999996}]},{"osd":5,"last update":"Sun Nov 26 07:18:42 2023","interfaces":[{"interface":"back","average":{"1min":0.66600000000000004,"5min":0.66600000000000004,"15min":0.66600000000000004},"min":{"1min":0.443,"5min":0.443,"15min":0.443},"max":{"1min":1.415,"5min":1.415,"15min":1.415},"last":0.72599999999999998},{"interface":"front","average":{"1min":0.71399999999999997,"5min":0.71399999999999997,"15min":0.71399999999999997},"min":{"1min":0.53100000000000003,"5min":0.53100000000000003,"15min":0.53100000000000003},"max":{"1min":1.3859999999999999,"5min":1.3859999999999999,"15min":1.3859999999999999},"last":0.52800000000000002}]},{"osd":6,"last update":"Sun Nov 26 07:18:42 2023","interfaces":[{"interface":"back","average":{"1min":0.77100000000000002,"5min":0.77100000000000002,"15min":0.77100000000000002},"min":{"1min":0.64100000000000001,"5min":0.64100000000000001,"15min":0.64100000000000001},"max":{"1min":1.3069999999999999,"5min":1.3069999999999999,"15min":1.3069999999999999},"last":0.58399999999999996},{"interface":"front","average":{"1min":0.77100000000000002,"5min":0.77100000000000002,"15min":0.77100000000000002},"min":{"1min":0.57399999999999995,"5min":0.57399999999999995,"15min":0.57399999999999995},"max":{"1min":1.6759999999999999,"5min":1.6759999999999999,"15min":1.6759999999999999},"last":0.74299999999999999}]}]},{"osd":6,"up_from":13,"seq":55834574866,"num_pgs":3,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27348,"kb_used_data":328,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344492,"statfs":{"total":96636764160,"available":96608759808,"internally_reserved":0,"allocated":335872,"data_stored":127250,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Nov 26 07:18:45 2023","interfaces":[{"interface":"back","average":{"1min":0.68400000000000005,"5min":0.68400000000000005,"15min":0.68400000000000005},"min":{"1min":0.36699999999999999,"5min":0.36699999999999999,"15min":0.36699999999999999},"max":{"1min":1.7549999999999999,"5min":1.7549999999999999,"15min":1.7549999999999999},"last":0.64900000000000002},{"interface":"front","average":{"1min":0.95799999999999996,"5min":0.95799999999999996,"15min":0.95799999999999996},"min":{"1min":0.38300000000000001,"5min":0.38300000000000001,"15min":0.38300000000000001},"max":{"1min":5.7489999999999997,"5min":5.7489999999999997,"15min":5.7489999999999997},"last":0.56000000000000005}]},{"osd":1,"last update":"Sun Nov 26 07:18:45 2023","interfaces":[{"interface":"back","average":{"1min":0.91400000000000003,"5min":0.91400000000000003,"15min":0.91400000000000003},"min":{"1min":0.42099999999999999,"5min":0.42099999999999999,"15min":0.42099999999999999},"max":{"1min":5.4240000000000004,"5min":5.4240000000000004,"15min":5.4240000000000004},"last":0.58199999999999996},{"interface":"front","average":{"1min":0.89700000000000002,"5min":0.89700000000000002,"15min":0.89700000000000002},"min":{"1min":0.499,"5min":0.499,"15min":0.499},"max":{"1min":5.6020000000000003,"5min":5.6020000000000003,"15min":5.6020000000000003},"last":0.75}]},{"osd":2,"last update":"Sun Nov 26 07:18:45 2023","interfaces":[{"interface":"back","average":{"1min":0.69699999999999995,"5min":0.69699999999999995,"15min":0.69699999999999995},"min":{"1min":0.47599999999999998,"5min":0.47599999999999998,"15min":0.47599999999999998},"max":{"1min":1.8520000000000001,"5min":1.8520000000000001,"15min":1.8520000000000001},"last":0.60599999999999998},{"interface":"front","average":{"1min":0.73399999999999999,"5min":0.73399999999999999,"15min":0.73399999999999999},"min":{"1min":0.46700000000000003,"5min":0.46700000000000003,"15min":0.46700000000000003},"max":{"1min":1.9159999999999999,"5min":1.9159999999999999,"15min":1.9159999999999999},"last":0.51700000000000002}]},{"osd":3,"last update":"Sun Nov 26 07:18:45 2023","interfaces":[{"interface":"back","average":{"1min":0.90200000000000002,"5min":0.90200000000000002,"15min":0.90200000000000002},"min":{"1min":0.502,"5min":0.502,"15min":0.502},"max":{"1min":5.4390000000000001,"5min":5.4390000000000001,"15min":5.4390000000000001},"last":0.63400000000000001},{"interface":"front","average":{"1min":0.94499999999999995,"5min":0.94499999999999995,"15min":0.94499999999999995},"min":{"1min":0.54400000000000004,"5min":0.54400000000000004,"15min":0.54400000000000004},"max":{"1min":5.5490000000000004,"5min":5.5490000000000004,"15min":5.5490000000000004},"last":0.72799999999999998}]},{"osd":4,"last update":"Sun Nov 26 07:18:45 2023","interfaces":[{"interface":"back","average":{"1min":0.64200000000000002,"5min":0.64200000000000002,"15min":0.64200000000000002},"min":{"1min":0.46899999999999997,"5min":0.46899999999999997,"15min":0.46899999999999997},"max":{"1min":0.85099999999999998,"5min":0.85099999999999998,"15min":0.85099999999999998},"last":0.45500000000000002},{"interface":"front","average":{"1min":0.67500000000000004,"5min":0.67500000000000004,"15min":0.67500000000000004},"min":{"1min":0.40699999999999997,"5min":0.40699999999999997,"15min":0.40699999999999997},"max":{"1min":0.93300000000000005,"5min":0.93300000000000005,"15min":0.93300000000000005},"last":0.68700000000000006}]},{"osd":5,"last update":"Sun Nov 26 07:18:45 2023","interfaces":[{"interface":"back","average":{"1min":0.65900000000000003,"5min":0.65900000000000003,"15min":0.65900000000000003},"min":{"1min":0.42499999999999999,"5min":0.42499999999999999,"15min":0.42499999999999999},"max":{"1min":0.79300000000000004,"5min":0.79300000000000004,"15min":0.79300000000000004},"last":0.53400000000000003},{"interface":"front","average":{"1min":0.71299999999999997,"5min":0.71299999999999997,"15min":0.71299999999999997},"min":{"1min":0.436,"5min":0.436,"15min":0.436},"max":{"1min":0.93000000000000005,"5min":0.93000000000000005,"15min":0.93000000000000005},"last":0.70599999999999996}]},{"osd":7,"last update":"Sun Nov 26 07:18:45 2023","interfaces":[{"interface":"back","average":{"1min":0.74199999999999999,"5min":0.74199999999999999,"15min":0.74199999999999999},"min":{"1min":0.46600000000000003,"5min":0.46600000000000003,"15min":0.46600000000000003},"max":{"1min":0.998,"5min":0.998,"15min":0.998},"last":0.78200000000000003},{"interface":"front","average":{"1min":0.71599999999999997,"5min":0.71599999999999997,"15min":0.71599999999999997},"min":{"1min":0.498,"5min":0.498,"15min":0.498},"max":{"1min":0.86499999999999999,"5min":0.86499999999999999,"15min":0.86499999999999999},"last":0.66600000000000004}]}]},{"osd":5,"up_from":13,"seq":55834574864,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27360,"kb_used_data":340,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344480,"statfs":{"total":96636764160,"available":96608747520,"internally_reserved":0,"allocated":348160,"data_stored":133517,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Nov 26 07:18:45 2023","interfaces":[{"interface":"back","average":{"1min":0.66600000000000004,"5min":0.66600000000000004,"15min":0.66600000000000004},"min":{"1min":0.38100000000000001,"5min":0.38100000000000001,"15min":0.38100000000000001},"max":{"1min":1.0289999999999999,"5min":1.0289999999999999,"15min":1.0289999999999999},"last":0.59099999999999997},{"interface":"front","average":{"1min":0.66000000000000003,"5min":0.66000000000000003,"15min":0.66000000000000003},"min":{"1min":0.438,"5min":0.438,"15min":0.438},"max":{"1min":1.05,"5min":1.05,"15min":1.05},"last":0.54700000000000004}]},{"osd":1,"last update":"Sun Nov 26 07:18:45 2023","interfaces":[{"interface":"back","average":{"1min":0.66800000000000004,"5min":0.66800000000000004,"15min":0.66800000000000004},"min":{"1min":0.41599999999999998,"5min":0.41599999999999998,"15min":0.41599999999999998},"max":{"1min":0.86699999999999999,"5min":0.86699999999999999,"15min":0.86699999999999999},"last":0.79400000000000004},{"interface":"front","average":{"1min":0.68600000000000005,"5min":0.68600000000000005,"15min":0.68600000000000005},"min":{"1min":0.49199999999999999,"5min":0.49199999999999999,"15min":0.49199999999999999},"max":{"1min":0.98899999999999999,"5min":0.98899999999999999,"15min":0.98899999999999999},"last":0.52900000000000003}]},{"osd":2,"last update":"Sun Nov 26 07:18:45 2023","interfaces":[{"interface":"back","average":{"1min":0.67500000000000004,"5min":0.67500000000000004,"15min":0.67500000000000004},"min":{"1min":0.45900000000000002,"5min":0.45900000000000002,"15min":0.45900000000000002},"max":{"1min":0.89800000000000002,"5min":0.89800000000000002,"15min":0.89800000000000002},"last":0.67500000000000004},{"interface":"front","average":{"1min":0.67000000000000004,"5min":0.67000000000000004,"15min":0.67000000000000004},"min":{"1min":0.51200000000000001,"5min":0.51200000000000001,"15min":0.51200000000000001},"max":{"1min":0.89700000000000002,"5min":0.89700000000000002,"15min":0.89700000000000002},"last":0.73399999999999999}]},{"osd":3,"last update":"Sun Nov 26 07:18:45 2023","interfaces":[{"interface":"back","average":{"1min":0.71999999999999997,"5min":0.71999999999999997,"15min":0.71999999999999997},"min":{"1min":0.43099999999999999,"5min":0.43099999999999999,"15min":0.43099999999999999},"max":{"1min":1.0029999999999999,"5min":1.0029999999999999,"15min":1.0029999999999999},"last":0.629},{"interface":"front","average":{"1min":0.67500000000000004,"5min":0.67500000000000004,"15min":0.67500000000000004},"min":{"1min":0.44600000000000001,"5min":0.44600000000000001,"15min":0.44600000000000001},"max":{"1min":1.0209999999999999,"5min":1.0209999999999999,"15min":1.0209999999999999},"last":0.48199999999999998}]},{"osd":4,"last update":"Sun Nov 26 07:18:45 2023","interfaces":[{"interface":"back","average":{"1min":0.63500000000000001,"5min":0.63500000000000001,"15min":0.63500000000000001},"min":{"1min":0.39200000000000002,"5min":0.39200000000000002,"15min":0.39200000000000002},"max":{"1min":0.95099999999999996,"5min":0.95099999999999996,"15min":0.95099999999999996},"last":0.70199999999999996},{"interface":"front","average":{"1min":0.70899999999999996,"5min":0.70899999999999996,"15min":0.70899999999999996},"min":{"1min":0.41299999999999998,"5min":0.41299999999999998,"15min":0.41299999999999998},"max":{"1min":1.254,"5min":1.254,"15min":1.254},"last":0.75900000000000001}]},{"osd":6,"last update":"Sun Nov 26 07:18:45 2023","interfaces":[{"interface":"back","average":{"1min":0.67200000000000004,"5min":0.67200000000000004,"15min":0.67200000000000004},"min":{"1min":0.45600000000000002,"5min":0.45600000000000002,"15min":0.45600000000000002},"max":{"1min":0.94499999999999995,"5min":0.94499999999999995,"15min":0.94499999999999995},"last":0.80900000000000005},{"interface":"front","average":{"1min":0.72699999999999998,"5min":0.72699999999999998,"15min":0.72699999999999998},"min":{"1min":0.44,"5min":0.44,"15min":0.44},"max":{"1min":1.0649999999999999,"5min":1.0649999999999999,"15min":1.0649999999999999},"last":0.77700000000000002}]},{"osd":7,"last update":"Sun Nov 26 07:18:45 2023","interfaces":[{"interface":"back","average":{"1min":0.80400000000000005,"5min":0.80400000000000005,"15min":0.80400000000000005},"min":{"1min":0.48199999999999998,"5min":0.48199999999999998,"15min":0.48199999999999998},"max":{"1min":1.4079999999999999,"5min":1.4079999999999999,"15min":1.4079999999999999},"last":0.85399999999999998},{"interface":"front","average":{"1min":0.77500000000000002,"5min":0.77500000000000002,"15min":0.77500000000000002},"min":{"1min":0.51600000000000001,"5min":0.51600000000000001,"15min":0.51600000000000001},"max":{"1min":1.0169999999999999,"5min":1.0169999999999999,"15min":1.0169999999999999},"last":0.82499999999999996}]}]},{"osd":4,"up_from":13,"seq":55834574866,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27348,"kb_used_data":328,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344492,"statfs":{"total":96636764160,"available":96608759808,"internally_reserved":0,"allocated":335872,"data_stored":127250,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Nov 26 07:18:46 2023","interfaces":[{"interface":"back","average":{"1min":0.79400000000000004,"5min":0.79400000000000004,"15min":0.79400000000000004},"min":{"1min":0.45600000000000002,"5min":0.45600000000000002,"15min":0.45600000000000002},"max":{"1min":4.1689999999999996,"5min":4.1689999999999996,"15min":4.1689999999999996},"last":0.58599999999999997},{"interface":"front","average":{"1min":1.002,"5min":1.002,"15min":1.002},"min":{"1min":0.47499999999999998,"5min":0.47499999999999998,"15min":0.47499999999999998},"max":{"1min":7.7889999999999997,"5min":7.7889999999999997,"15min":7.7889999999999997},"last":0.54200000000000004}]},{"osd":1,"last update":"Sun Nov 26 07:18:46 2023","interfaces":[{"interface":"back","average":{"1min":0.64700000000000002,"5min":0.64700000000000002,"15min":0.64700000000000002},"min":{"1min":0.52300000000000002,"5min":0.52300000000000002,"15min":0.52300000000000002},"max":{"1min":0.78400000000000003,"5min":0.78400000000000003,"15min":0.78400000000000003},"last":0.63400000000000001},{"interface":"front","average":{"1min":0.81999999999999995,"5min":0.81999999999999995,"15min":0.81999999999999995},"min":{"1min":0.44500000000000001,"5min":0.44500000000000001,"15min":0.44500000000000001},"max":{"1min":4.3140000000000001,"5min":4.3140000000000001,"15min":4.3140000000000001},"last":0.52400000000000002}]},{"osd":2,"last update":"Sun Nov 26 07:18:46 2023","interfaces":[{"interface":"back","average":{"1min":0.80700000000000005,"5min":0.80700000000000005,"15min":0.80700000000000005},"min":{"1min":0.57799999999999996,"5min":0.57799999999999996,"15min":0.57799999999999996},"max":{"1min":2.7850000000000001,"5min":2.7850000000000001,"15min":2.7850000000000001},"last":0.76100000000000001},{"interface":"front","average":{"1min":1.0660000000000001,"5min":1.0660000000000001,"15min":1.0660000000000001},"min":{"1min":0.53400000000000003,"5min":0.53400000000000003,"15min":0.53400000000000003},"max":{"1min":7.6470000000000002,"5min":7.6470000000000002,"15min":7.6470000000000002},"last":0.68899999999999995}]},{"osd":3,"last update":"Sun Nov 26 07:18:46 2023","interfaces":[{"interface":"back","average":{"1min":0.72999999999999998,"5min":0.72999999999999998,"15min":0.72999999999999998},"min":{"1min":0.48599999999999999,"5min":0.48599999999999999,"15min":0.48599999999999999},"max":{"1min":1.591,"5min":1.591,"15min":1.591},"last":0.45700000000000002},{"interface":"front","average":{"1min":0.879,"5min":0.879,"15min":0.879},"min":{"1min":0.56899999999999995,"5min":0.56899999999999995,"15min":0.56899999999999995},"max":{"1min":3.452,"5min":3.452,"15min":3.452},"last":0.80400000000000005}]},{"osd":5,"last update":"Sun Nov 26 07:18:46 2023","interfaces":[{"interface":"back","average":{"1min":0.61099999999999999,"5min":0.61099999999999999,"15min":0.61099999999999999},"min":{"1min":0.43099999999999999,"5min":0.43099999999999999,"15min":0.43099999999999999},"max":{"1min":0.96199999999999997,"5min":0.96199999999999997,"15min":0.96199999999999997},"last":0.73999999999999999},{"interface":"front","average":{"1min":0.64100000000000001,"5min":0.64100000000000001,"15min":0.64100000000000001},"min":{"1min":0.41499999999999998,"5min":0.41499999999999998,"15min":0.41499999999999998},"max":{"1min":1.538,"5min":1.538,"15min":1.538},"last":0.56799999999999995}]},{"osd":6,"last update":"Sun Nov 26 07:18:46 2023","interfaces":[{"interface":"back","average":{"1min":0.79400000000000004,"5min":0.79400000000000004,"15min":0.79400000000000004},"min":{"1min":0.39400000000000002,"5min":0.39400000000000002,"15min":0.39400000000000002},"max":{"1min":2.8330000000000002,"5min":2.8330000000000002,"15min":2.8330000000000002},"last":0.70999999999999996},{"interface":"front","average":{"1min":1.0089999999999999,"5min":1.0089999999999999,"15min":1.0089999999999999},"min":{"1min":0.42899999999999999,"5min":0.42899999999999999,"15min":0.42899999999999999},"max":{"1min":7.6050000000000004,"5min":7.6050000000000004,"15min":7.6050000000000004},"last":0.78500000000000003}]},{"osd":7,"last update":"Sun Nov 26 07:18:46 2023","interfaces":[{"interface":"back","average":{"1min":0.86099999999999999,"5min":0.86099999999999999,"15min":0.86099999999999999},"min":{"1min":0.44700000000000001,"5min":0.44700000000000001,"15min":0.44700000000000001},"max":{"1min":4.3689999999999998,"5min":4.3689999999999998,"15min":4.3689999999999998},"last":0.61299999999999999},{"interface":"front","average":{"1min":0.89200000000000002,"5min":0.89200000000000002,"15min":0.89200000000000002},"min":{"1min":0.505,"5min":0.505,"15min":0.505},"max":{"1min":3.403,"5min":3.403,"15min":3.403},"last":0.83799999999999997}]}]},{"osd":3,"up_from":13,"seq":55834574867,"num_pgs":7,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27352,"kb_used_data":328,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344488,"statfs":{"total":96636764160,"available":96608755712,"internally_reserved":0,"allocated":335872,"data_stored":127250,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1592,"internal_metadata":27654600},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Nov 26 07:18:43 2023","interfaces":[{"interface":"back","average":{"1min":0.379,"5min":0.379,"15min":0.379},"min":{"1min":0.26100000000000001,"5min":0.26100000000000001,"15min":0.26100000000000001},"max":{"1min":0.58599999999999997,"5min":0.58599999999999997,"15min":0.58599999999999997},"last":0.40000000000000002},{"interface":"front","average":{"1min":0.433,"5min":0.433,"15min":0.433},"min":{"1min":0.30199999999999999,"5min":0.30199999999999999,"15min":0.30199999999999999},"max":{"1min":0.63,"5min":0.63,"15min":0.63},"last":0.26900000000000002}]},{"osd":1,"last update":"Sun Nov 26 07:18:43 2023","interfaces":[{"interface":"back","average":{"1min":0.42099999999999999,"5min":0.42099999999999999,"15min":0.42099999999999999},"min":{"1min":0.312,"5min":0.312,"15min":0.312},"max":{"1min":0.752,"5min":0.752,"15min":0.752},"last":0.28799999999999998},{"interface":"front","average":{"1min":0.48399999999999999,"5min":0.48399999999999999,"15min":0.48399999999999999},"min":{"1min":0.38,"5min":0.38,"15min":0.38},"max":{"1min":0.628,"5min":0.628,"15min":0.628},"last":0.35099999999999998}]},{"osd":2,"last update":"Sun Nov 26 07:18:43 2023","interfaces":[{"interface":"back","average":{"1min":0.53300000000000003,"5min":0.53300000000000003,"15min":0.53300000000000003},"min":{"1min":0.40300000000000002,"5min":0.40300000000000002,"15min":0.40300000000000002},"max":{"1min":0.82199999999999995,"5min":0.82199999999999995,"15min":0.82199999999999995},"last":0.38200000000000001},{"interface":"front","average":{"1min":0.505,"5min":0.505,"15min":0.505},"min":{"1min":0.35999999999999999,"5min":0.35999999999999999,"15min":0.35999999999999999},"max":{"1min":0.72199999999999998,"5min":0.72199999999999998,"15min":0.72199999999999998},"last":0.33000000000000002}]},{"osd":4,"last update":"Sun Nov 26 07:18:43 2023","interfaces":[{"interface":"back","average":{"1min":0.70599999999999996,"5min":0.70599999999999996,"15min":0.70599999999999996},"min":{"1min":0.48199999999999998,"5min":0.48199999999999998,"15min":0.48199999999999998},"max":{"1min":0.96199999999999997,"5min":0.96199999999999997,"15min":0.96199999999999997},"last":0.52200000000000002},{"interface":"front","average":{"1min":0.72799999999999998,"5min":0.72799999999999998,"15min":0.72799999999999998},"min":{"1min":0.441,"5min":0.441,"15min":0.441},"max":{"1min":1.171,"5min":1.171,"15min":1.171},"last":0.59699999999999998}]},{"osd":5,"last update":"Sun Nov 26 07:18:43 2023","interfaces":[{"interface":"back","average":{"1min":0.73899999999999999,"5min":0.73899999999999999,"15min":0.73899999999999999},"min":{"1min":0.53700000000000003,"5min":0.53700000000000003,"15min":0.53700000000000003},"max":{"1min":0.94099999999999995,"5min":0.94099999999999995,"15min":0.94099999999999995},"last":0.53800000000000003},{"interface":"front","average":{"1min":0.73799999999999999,"5min":0.73799999999999999,"15min":0.73799999999999999},"min":{"1min":0.504,"5min":0.504,"15min":0.504},"max":{"1min":1.0820000000000001,"5min":1.0820000000000001,"15min":1.0820000000000001},"last":0.61199999999999999}]},{"osd":6,"last update":"Sun Nov 26 07:18:43 2023","interfaces":[{"interface":"back","average":{"1min":0.81399999999999995,"5min":0.81399999999999995,"15min":0.81399999999999995},"min":{"1min":0.55200000000000005,"5min":0.55200000000000005,"15min":0.55200000000000005},"max":{"1min":1.2729999999999999,"5min":1.2729999999999999,"15min":1.2729999999999999},"last":0.64500000000000002},{"interface":"front","average":{"1min":0.79300000000000004,"5min":0.79300000000000004,"15min":0.79300000000000004},"min":{"1min":0.47299999999999998,"5min":0.47299999999999998,"15min":0.47299999999999998},"max":{"1min":1.204,"5min":1.204,"15min":1.204},"last":0.63200000000000001}]},{"osd":7,"last update":"Sun Nov 26 07:18:43 2023","interfaces":[{"interface":"back","average":{"1min":0.80100000000000005,"5min":0.80100000000000005,"15min":0.80100000000000005},"min":{"1min":0.63300000000000001,"5min":0.63300000000000001,"15min":0.63300000000000001},"max":{"1min":1.3600000000000001,"5min":1.3600000000000001,"15min":1.3600000000000001},"last":0.65500000000000003},{"interface":"front","average":{"1min":0.82599999999999996,"5min":0.82599999999999996,"15min":0.82599999999999996},"min":{"1min":0.50600000000000001,"5min":0.50600000000000001,"15min":0.50600000000000001},"max":{"1min":1.617,"5min":1.617,"15min":1.617},"last":0.67000000000000004}]}]},{"osd":2,"up_from":13,"seq":55834574867,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27360,"kb_used_data":340,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344480,"statfs":{"total":96636764160,"available":96608747520,"internally_reserved":0,"allocated":348160,"data_stored":133517,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Nov 26 07:18:43 2023","interfaces":[{"interface":"back","average":{"1min":0.71399999999999997,"5min":0.71399999999999997,"15min":0.71399999999999997},"min":{"1min":0.28699999999999998,"5min":0.28699999999999998,"15min":0.28699999999999998},"max":{"1min":4.7030000000000003,"5min":4.7030000000000003,"15min":4.7030000000000003},"last":0.32500000000000001},{"interface":"front","average":{"1min":0.56200000000000006,"5min":0.56200000000000006,"15min":0.56200000000000006},"min":{"1min":0.32300000000000001,"5min":0.32300000000000001,"15min":0.32300000000000001},"max":{"1min":2.4340000000000002,"5min":2.4340000000000002,"15min":2.4340000000000002},"last":0.49099999999999999}]},{"osd":1,"last update":"Sun Nov 26 07:18:43 2023","interfaces":[{"interface":"back","average":{"1min":0.86399999999999999,"5min":0.86399999999999999,"15min":0.86399999999999999},"min":{"1min":0.34599999999999997,"5min":0.34599999999999997,"15min":0.34599999999999997},"max":{"1min":4.2690000000000001,"5min":4.2690000000000001,"15min":4.2690000000000001},"last":0.42499999999999999},{"interface":"front","average":{"1min":0.74199999999999999,"5min":0.74199999999999999,"15min":0.74199999999999999},"min":{"1min":0.31900000000000001,"5min":0.31900000000000001,"15min":0.31900000000000001},"max":{"1min":3.9940000000000002,"5min":3.9940000000000002,"15min":3.9940000000000002},"last":0.35099999999999998}]},{"osd":3,"last update":"Sun Nov 26 07:18:43 2023","interfaces":[{"interface":"back","average":{"1min":0.64300000000000002,"5min":0.64300000000000002,"15min":0.64300000000000002},"min":{"1min":0.34399999999999997,"5min":0.34399999999999997,"15min":0.34399999999999997},"max":{"1min":4.0300000000000002,"5min":4.0300000000000002,"15min":4.0300000000000002},"last":0.38700000000000001},{"interface":"front","average":{"1min":0.91400000000000003,"5min":0.91400000000000003,"15min":0.91400000000000003},"min":{"1min":0.41699999999999998,"5min":0.41699999999999998,"15min":0.41699999999999998},"max":{"1min":4.5810000000000004,"5min":4.5810000000000004,"15min":4.5810000000000004},"last":0.46500000000000002}]},{"osd":4,"last update":"Sun Nov 26 07:18:43 2023","interfaces":[{"interface":"back","average":{"1min":0.98499999999999999,"5min":0.98499999999999999,"15min":0.98499999999999999},"min":{"1min":0.55100000000000005,"5min":0.55100000000000005,"15min":0.55100000000000005},"max":{"1min":4.4269999999999996,"5min":4.4269999999999996,"15min":4.4269999999999996},"last":1.01},{"interface":"front","average":{"1min":1.0149999999999999,"5min":1.0149999999999999,"15min":1.0149999999999999},"min":{"1min":0.60299999999999998,"5min":0.60299999999999998,"15min":0.60299999999999998},"max":{"1min":4.085,"5min":4.085,"15min":4.085},"last":1.0600000000000001}]},{"osd":5,"last update":"Sun Nov 26 07:18:43 2023","interfaces":[{"interface":"back","average":{"1min":0.874,"5min":0.874,"15min":0.874},"min":{"1min":0.52900000000000003,"5min":0.52900000000000003,"15min":0.52900000000000003},"max":{"1min":3.9590000000000001,"5min":3.9590000000000001,"15min":3.9590000000000001},"last":0.81000000000000005},{"interface":"front","average":{"1min":0.98799999999999999,"5min":0.98799999999999999,"15min":0.98799999999999999},"min":{"1min":0.58799999999999997,"5min":0.58799999999999997,"15min":0.58799999999999997},"max":{"1min":4.3849999999999998,"5min":4.3849999999999998,"15min":4.3849999999999998},"last":0.86299999999999999}]},{"osd":6,"last update":"Sun Nov 26 07:18:43 2023","interfaces":[{"interface":"back","average":{"1min":0.996,"5min":0.996,"15min":0.996},"min":{"1min":0.56599999999999995,"5min":0.56599999999999995,"15min":0.56599999999999995},"max":{"1min":4.0449999999999999,"5min":4.0449999999999999,"15min":4.0449999999999999},"last":0.91000000000000003},{"interface":"front","average":{"1min":0.91800000000000004,"5min":0.91800000000000004,"15min":0.91800000000000004},"min":{"1min":0.57099999999999995,"5min":0.57099999999999995,"15min":0.57099999999999995},"max":{"1min":3.992,"5min":3.992,"15min":3.992},"last":1.073}]},{"osd":7,"last update":"Sun Nov 26 07:18:43 2023","interfaces":[{"interface":"back","average":{"1min":1.0349999999999999,"5min":1.0349999999999999,"15min":1.0349999999999999},"min":{"1min":0.63100000000000001,"5min":0.63100000000000001,"15min":0.63100000000000001},"max":{"1min":4.1219999999999999,"5min":4.1219999999999999,"15min":4.1219999999999999},"last":0.69699999999999995},{"interface":"front","average":{"1min":0.93799999999999994,"5min":0.93799999999999994,"15min":0.93799999999999994},"min":{"1min":0.57699999999999996,"5min":0.57699999999999996,"15min":0.57699999999999996},"max":{"1min":3.9159999999999999,"5min":3.9159999999999999,"15min":3.9159999999999999},"last":0.95199999999999996}]}]},{"osd":1,"up_from":13,"seq":55834574866,"num_pgs":5,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27352,"kb_used_data":328,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344488,"statfs":{"total":96636764160,"available":96608755712,"internally_reserved":0,"allocated":335872,"data_stored":127250,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Nov 26 07:18:41 2023","interfaces":[{"interface":"back","average":{"1min":0.41699999999999998,"5min":0.41699999999999998,"15min":0.41699999999999998},"min":{"1min":0.26300000000000001,"5min":0.26300000000000001,"15min":0.26300000000000001},"max":{"1min":0.61199999999999999,"5min":0.61199999999999999,"15min":0.61199999999999999},"last":0.29499999999999998},{"interface":"front","average":{"1min":0.48599999999999999,"5min":0.48599999999999999,"15min":0.48599999999999999},"min":{"1min":0.30499999999999999,"5min":0.30499999999999999,"15min":0.30499999999999999},"max":{"1min":0.78700000000000003,"5min":0.78700000000000003,"15min":0.78700000000000003},"last":0.34699999999999998}]},{"osd":2,"last update":"Sun Nov 26 07:18:41 2023","interfaces":[{"interface":"back","average":{"1min":0.45500000000000002,"5min":0.45500000000000002,"15min":0.45500000000000002},"min":{"1min":0.33400000000000002,"5min":0.33400000000000002,"15min":0.33400000000000002},"max":{"1min":0.67600000000000005,"5min":0.67600000000000005,"15min":0.67600000000000005},"last":0.39200000000000002},{"interface":"front","average":{"1min":0.50800000000000001,"5min":0.50800000000000001,"15min":0.50800000000000001},"min":{"1min":0.32300000000000001,"5min":0.32300000000000001,"15min":0.32300000000000001},"max":{"1min":0.66500000000000004,"5min":0.66500000000000004,"15min":0.66500000000000004},"last":0.40699999999999997}]},{"osd":3,"last update":"Sun Nov 26 07:18:41 2023","interfaces":[{"interface":"back","average":{"1min":0.53000000000000003,"5min":0.53000000000000003,"15min":0.53000000000000003},"min":{"1min":0.38400000000000001,"5min":0.38400000000000001,"15min":0.38400000000000001},"max":{"1min":0.73699999999999999,"5min":0.73699999999999999,"15min":0.73699999999999999},"last":0.48499999999999999},{"interface":"front","average":{"1min":0.497,"5min":0.497,"15min":0.497},"min":{"1min":0.36199999999999999,"5min":0.36199999999999999,"15min":0.36199999999999999},"max":{"1min":0.80700000000000005,"5min":0.80700000000000005,"15min":0.80700000000000005},"last":0.432}]},{"osd":4,"last update":"Sun Nov 26 07:18:41 2023","interfaces":[{"interface":"back","average":{"1min":0.69399999999999995,"5min":0.69399999999999995,"15min":0.69399999999999995},"min":{"1min":0.48599999999999999,"5min":0.48599999999999999,"15min":0.48599999999999999},"max":{"1min":1.0229999999999999,"5min":1.0229999999999999,"15min":1.0229999999999999},"last":0.54400000000000004},{"interface":"front","average":{"1min":0.68000000000000005,"5min":0.68000000000000005,"15min":0.68000000000000005},"min":{"1min":0.49199999999999999,"5min":0.49199999999999999,"15min":0.49199999999999999},"max":{"1min":1.0629999999999999,"5min":1.0629999999999999,"15min":1.0629999999999999},"last":0.58399999999999996}]},{"osd":5,"last update":"Sun Nov 26 07:18:41 2023","interfaces":[{"interface":"back","average":{"1min":0.71899999999999997,"5min":0.71899999999999997,"15min":0.71899999999999997},"min":{"1min":0.49299999999999999,"5min":0.49299999999999999,"15min":0.49299999999999999},"max":{"1min":1.0880000000000001,"5min":1.0880000000000001,"15min":1.0880000000000001},"last":0.60399999999999998},{"interface":"front","average":{"1min":0.71699999999999997,"5min":0.71699999999999997,"15min":0.71699999999999997},"min":{"1min":0.53500000000000003,"5min":0.53500000000000003,"15min":0.53500000000000003},"max":{"1min":1.014,"5min":1.014,"15min":1.014},"last":0.68500000000000005}]},{"osd":6,"last update":"Sun Nov 26 07:18:41 2023","interfaces":[{"interface":"back","average":{"1min":0.70399999999999996,"5min":0.70399999999999996,"15min":0.70399999999999996},"min":{"1min":0.50600000000000001,"5min":0.50600000000000001,"15min":0.50600000000000001},"max":{"1min":0.97599999999999998,"5min":0.97599999999999998,"15min":0.97599999999999998},"last":0.61199999999999999},{"interface":"front","average":{"1min":0.70099999999999996,"5min":0.70099999999999996,"15min":0.70099999999999996},"min":{"1min":0.55000000000000004,"5min":0.55000000000000004,"15min":0.55000000000000004},"max":{"1min":1.1339999999999999,"5min":1.1339999999999999,"15min":1.1339999999999999},"last":0.56100000000000005}]},{"osd":7,"last update":"Sun Nov 26 07:18:41 2023","interfaces":[{"interface":"back","average":{"1min":0.71499999999999997,"5min":0.71499999999999997,"15min":0.71499999999999997},"min":{"1min":0.55100000000000005,"5min":0.55100000000000005,"15min":0.55100000000000005},"max":{"1min":0.92900000000000005,"5min":0.92900000000000005,"15min":0.92900000000000005},"last":0.59299999999999997},{"interface":"front","average":{"1min":0.753,"5min":0.753,"15min":0.753},"min":{"1min":0.56100000000000005,"5min":0.56100000000000005,"15min":0.56100000000000005},"max":{"1min":1.024,"5min":1.024,"15min":1.024},"last":0.71499999999999997}]}]},{"osd":0,"up_from":13,"seq":55834574866,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27944,"kb_used_data":920,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343896,"statfs":{"total":96636764160,"available":96608149504,"internally_reserved":0,"allocated":942080,"data_stored":723885,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Sun Nov 26 07:18:41 2023","interfaces":[{"interface":"back","average":{"1min":0.43099999999999999,"5min":0.43099999999999999,"15min":0.43099999999999999},"min":{"1min":0.28000000000000003,"5min":0.28000000000000003,"15min":0.28000000000000003},"max":{"1min":0.63100000000000001,"5min":0.63100000000000001,"15min":0.63100000000000001},"last":0.38200000000000001},{"interface":"front","average":{"1min":0.48299999999999998,"5min":0.48299999999999998,"15min":0.48299999999999998},"min":{"1min":0.29899999999999999,"5min":0.29899999999999999,"15min":0.29899999999999999},"max":{"1min":1.133,"5min":1.133,"15min":1.133},"last":0.30199999999999999}]},{"osd":2,"last update":"Sun Nov 26 07:18:41 2023","interfaces":[{"interface":"back","average":{"1min":0.53700000000000003,"5min":0.53700000000000003,"15min":0.53700000000000003},"min":{"1min":0.40899999999999997,"5min":0.40899999999999997,"15min":0.40899999999999997},"max":{"1min":0.67400000000000004,"5min":0.67400000000000004,"15min":0.67400000000000004},"last":0.48099999999999998},{"interface":"front","average":{"1min":0.56399999999999995,"5min":0.56399999999999995,"15min":0.56399999999999995},"min":{"1min":0.36399999999999999,"5min":0.36399999999999999,"15min":0.36399999999999999},"max":{"1min":0.72499999999999998,"5min":0.72499999999999998,"15min":0.72499999999999998},"last":0.45300000000000001}]},{"osd":3,"last update":"Sun Nov 26 07:18:41 2023","interfaces":[{"interface":"back","average":{"1min":0.53100000000000003,"5min":0.53100000000000003,"15min":0.53100000000000003},"min":{"1min":0.35799999999999998,"5min":0.35799999999999998,"15min":0.35799999999999998},"max":{"1min":0.97399999999999998,"5min":0.97399999999999998,"15min":0.97399999999999998},"last":0.436},{"interface":"front","average":{"1min":0.56200000000000006,"5min":0.56200000000000006,"15min":0.56200000000000006},"min":{"1min":0.40600000000000003,"5min":0.40600000000000003,"15min":0.40600000000000003},"max":{"1min":1.2509999999999999,"5min":1.2509999999999999,"15min":1.2509999999999999},"last":0.498}]},{"osd":4,"last update":"Sun Nov 26 07:18:41 2023","interfaces":[{"interface":"back","average":{"1min":0.75900000000000001,"5min":0.75900000000000001,"15min":0.75900000000000001},"min":{"1min":0.60999999999999999,"5min":0.60999999999999999,"15min":0.60999999999999999},"max":{"1min":1.2010000000000001,"5min":1.2010000000000001,"15min":1.2010000000000001},"last":0.69599999999999995},{"interface":"front","average":{"1min":0.78900000000000003,"5min":0.78900000000000003,"15min":0.78900000000000003},"min":{"1min":0.59999999999999998,"5min":0.59999999999999998,"15min":0.59999999999999998},"max":{"1min":0.99399999999999999,"5min":0.99399999999999999,"15min":0.99399999999999999},"last":0.71899999999999997}]},{"osd":5,"last update":"Sun Nov 26 07:18:41 2023","interfaces":[{"interface":"back","average":{"1min":0.76000000000000001,"5min":0.76000000000000001,"15min":0.76000000000000001},"min":{"1min":0.53000000000000003,"5min":0.53000000000000003,"15min":0.53000000000000003},"max":{"1min":1.1579999999999999,"5min":1.1579999999999999,"15min":1.1579999999999999},"last":0.73599999999999999},{"interface":"front","average":{"1min":0.79700000000000004,"5min":0.79700000000000004,"15min":0.79700000000000004},"min":{"1min":0.59699999999999998,"5min":0.59699999999999998,"15min":0.59699999999999998},"max":{"1min":1.0409999999999999,"5min":1.0409999999999999,"15min":1.0409999999999999},"last":0.67000000000000004}]},{"osd":6,"last update":"Sun Nov 26 07:18:41 2023","interfaces":[{"interface":"back","average":{"1min":0.76400000000000001,"5min":0.76400000000000001,"15min":0.76400000000000001},"min":{"1min":0.56599999999999995,"5min":0.56599999999999995,"15min":0.56599999999999995},"max":{"1min":1.3029999999999999,"5min":1.3029999999999999,"15min":1.3029999999999999},"last":0.55300000000000005},{"interface":"front","average":{"1min":0.76900000000000002,"5min":0.76900000000000002,"15min":0.76900000000000002},"min":{"1min":0.58599999999999997,"5min":0.58599999999999997,"15min":0.58599999999999997},"max":{"1min":1.2210000000000001,"5min":1.2210000000000001,"15min":1.2210000000000001},"last":0.76000000000000001}]},{"osd":7,"last update":"Sun Nov 26 07:18:41 2023","interfaces":[{"interface":"back","average":{"1min":0.81100000000000005,"5min":0.81100000000000005,"15min":0.81100000000000005},"min":{"1min":0.59999999999999998,"5min":0.59999999999999998,"15min":0.59999999999999998},"max":{"1min":1.177,"5min":1.177,"15min":1.177},"last":0.624},{"interface":"front","average":{"1min":0.79900000000000004,"5min":0.79900000000000004,"15min":0.79900000000000004},"min":{"1min":0.60899999999999999,"5min":0.60899999999999999,"15min":0.60899999999999999},"max":{"1min":1.2390000000000001,"5min":1.2390000000000001,"15min":1.2390000000000001},"last":0.65100000000000002}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-11-26T07:18:57.106 DEBUG:teuthology.orchestra.run.smithi143:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-11-26T07:18:57.285 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:57.280+0000 7f7f5969c640 1 Processor -- start 2023-11-26T07:18:57.286 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:57.280+0000 7f7f5969c640 1 -- start start 2023-11-26T07:18:57.287 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:57.280+0000 7f7f5969c640 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7f7f54145fa0 0x7f7f54072420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:18:57.287 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:57.280+0000 7f7f5969c640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f7f5406a490 0x7f7f5406a8e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:18:57.287 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:57.280+0000 7f7f5969c640 1 -- --> v1:172.21.15.143:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7f54146380 con 0x7f7f5406b190 2023-11-26T07:18:57.287 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:57.280+0000 7f7f5969c640 1 -- --> v2:172.21.15.188:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7f54071480 con 0x7f7f54145fa0 2023-11-26T07:18:57.287 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:57.280+0000 7f7f5969c640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7f54072000 con 0x7f7f5406a490 2023-11-26T07:18:57.287 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:57.280+0000 7f7f4a7fc640 1 --1- >> v1:172.21.15.143:6789/0 conn(0x7f7f5406b190 0x7f7f54146530 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.143:6789/0 says I am v1:172.21.15.143:35292/0 (socket says 172.21.15.143:35292) 2023-11-26T07:18:57.287 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:57.280+0000 7f7f4a7fc640 1 -- 172.21.15.143:0/2691096232 learned_addr learned my addr 172.21.15.143:0/2691096232 (peer_addr_for_me v1:172.21.15.143:0/0) 2023-11-26T07:18:57.289 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:57.280+0000 7f7f52ffd640 1 --2- 172.21.15.143:0/2691096232 >> v2:172.21.15.188:3300/0 conn(0x7f7f54145fa0 0x7f7f54072420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:18:57.289 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:57.280+0000 7f7f537fe640 1 --2- 172.21.15.143:0/2691096232 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f7f5406a490 0x7f7f5406a8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:18:57.289 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:57.280+0000 7f7f527fc640 1 -- 172.21.15.143:0/2691096232 <== mon.0 v1:172.21.15.143:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 109382866 0 0) 0x7f7f54146380 con 0x7f7f5406b190 2023-11-26T07:18:57.289 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:57.280+0000 7f7f527fc640 1 -- 172.21.15.143:0/2691096232 --> v1:172.21.15.143:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f7f40003530 con 0x7f7f5406b190 2023-11-26T07:18:57.290 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:57.280+0000 7f7f527fc640 1 -- 172.21.15.143:0/2691096232 <== mon.0 v1:172.21.15.143:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 811633431 0 0) 0x7f7f40003530 con 0x7f7f5406b190 2023-11-26T07:18:57.290 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:57.280+0000 7f7f527fc640 1 -- 172.21.15.143:0/2691096232 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f7f5406a490 msgr2=0x7f7f5406a8e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-26T07:18:57.290 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:57.280+0000 7f7f527fc640 1 --2- 172.21.15.143:0/2691096232 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f7f5406a490 0x7f7f5406a8e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:18:57.290 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:57.280+0000 7f7f527fc640 1 -- 172.21.15.143:0/2691096232 >> v2:172.21.15.188:3300/0 conn(0x7f7f54145fa0 msgr2=0x7f7f54072420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-26T07:18:57.290 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:57.280+0000 7f7f527fc640 1 --2- 172.21.15.143:0/2691096232 >> v2:172.21.15.188:3300/0 conn(0x7f7f54145fa0 0x7f7f54072420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:18:57.290 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:57.280+0000 7f7f527fc640 1 -- 172.21.15.143:0/2691096232 --> v1:172.21.15.143:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7f54146c40 con 0x7f7f5406b190 2023-11-26T07:18:57.290 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:57.280+0000 7f7f527fc640 1 -- 172.21.15.143:0/2691096232 <== mon.0 v1:172.21.15.143:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2137882523 0 0) 0x7f7f3c001680 con 0x7f7f5406b190 2023-11-26T07:18:57.290 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:57.280+0000 7f7f527fc640 1 -- 172.21.15.143:0/2691096232 >> v1:172.21.15.143:6789/0 conn(0x7f7f5406b190 legacy=0x7f7f54146530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:18:57.290 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:57.280+0000 7f7f527fc640 1 --2- 172.21.15.143:0/2691096232 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f7f40003980 0x7f7f40003da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:18:57.290 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:57.280+0000 7f7f527fc640 1 --2- 172.21.15.143:0/2691096232 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f7f400047d0 0x7f7f40004c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:18:57.290 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:57.280+0000 7f7f527fc640 1 --2- 172.21.15.143:0/2691096232 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f7f5406a490 0x7f7f40005190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:18:57.290 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:57.280+0000 7f7f527fc640 1 -- 172.21.15.143:0/2691096232 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7f54071480 con 0x7f7f40003980 2023-11-26T07:18:57.291 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:57.280+0000 7f7f527fc640 1 -- 172.21.15.143:0/2691096232 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7f54072000 con 0x7f7f400047d0 2023-11-26T07:18:57.291 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:57.280+0000 7f7f527fc640 1 -- 172.21.15.143:0/2691096232 --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7f4000b990 con 0x7f7f5406a490 2023-11-26T07:18:57.291 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:57.280+0000 7f7f52ffd640 1 --2- 172.21.15.143:0/2691096232 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f7f40003980 0x7f7f40003da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:18:57.291 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:57.280+0000 7f7f4a7fc640 1 --2- 172.21.15.143:0/2691096232 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f7f400047d0 0x7f7f40004c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:18:57.291 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:57.280+0000 7f7f537fe640 1 --2- 172.21.15.143:0/2691096232 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f7f5406a490 0x7f7f40005190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:18:57.291 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:57.280+0000 7f7f52ffd640 1 -- 172.21.15.143:0/2691096232 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f7f5406a490 msgr2=0x7f7f40005190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-26T07:18:57.291 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:57.280+0000 7f7f52ffd640 1 --2- 172.21.15.143:0/2691096232 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f7f5406a490 0x7f7f40005190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:18:57.291 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:57.284+0000 7f7f52ffd640 1 -- 172.21.15.143:0/2691096232 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f7f400047d0 msgr2=0x7f7f40004c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-26T07:18:57.291 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:57.284+0000 7f7f52ffd640 1 --2- 172.21.15.143:0/2691096232 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f7f400047d0 0x7f7f40004c50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:18:57.291 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:57.284+0000 7f7f52ffd640 1 -- 172.21.15.143:0/2691096232 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7f40003530 con 0x7f7f40003980 2023-11-26T07:18:57.291 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:57.284+0000 7f7f4a7fc640 1 --2- 172.21.15.143:0/2691096232 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f7f400047d0 0x7f7f40004c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-26T07:18:57.293 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:57.284+0000 7f7f52ffd640 1 --2- 172.21.15.143:0/2691096232 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f7f40003980 0x7f7f40003da0 secure :-1 s=READY pgs=416 cs=0 l=1 rev1=1 crypto rx=0x7f7f44009ae0 tx=0x7f7f4401ca30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b5c3fdb5d57e403f server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:18:57.293 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:57.284+0000 7f7f527fc640 1 -- 172.21.15.143:0/2691096232 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7f440a6480 con 0x7f7f40003980 2023-11-26T07:18:57.293 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:57.284+0000 7f7f527fc640 1 -- 172.21.15.143:0/2691096232 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7f440a65e0 con 0x7f7f40003980 2023-11-26T07:18:57.293 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:57.284+0000 7f7f527fc640 1 -- 172.21.15.143:0/2691096232 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7f440a6990 con 0x7f7f40003980 2023-11-26T07:18:57.293 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:57.284+0000 7f7f5969c640 1 -- 172.21.15.143:0/2691096232 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f7f40003980 msgr2=0x7f7f40003da0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:18:57.293 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:57.284+0000 7f7f5969c640 1 --2- 172.21.15.143:0/2691096232 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f7f40003980 0x7f7f40003da0 secure :-1 s=READY pgs=416 cs=0 l=1 rev1=1 crypto rx=0x7f7f44009ae0 tx=0x7f7f4401ca30 comp rx=0 tx=0).stop 2023-11-26T07:18:57.293 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:57.284+0000 7f7f537fe640 1 -- 172.21.15.143:0/2691096232 reap_dead start 2023-11-26T07:18:57.293 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:57.284+0000 7f7f5969c640 1 -- 172.21.15.143:0/2691096232 shutdown_connections 2023-11-26T07:18:57.293 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:57.284+0000 7f7f5969c640 1 -- 172.21.15.143:0/2691096232 >> 172.21.15.143:0/2691096232 conn(0x7f7f540cff00 msgr2=0x7f7f540d0310 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:18:57.294 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:57.284+0000 7f7f5969c640 1 -- 172.21.15.143:0/2691096232 shutdown_connections 2023-11-26T07:18:57.294 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:57.284+0000 7f7f5969c640 1 -- 172.21.15.143:0/2691096232 wait complete. 2023-11-26T07:18:57.294 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:57.288+0000 7f7f5969c640 1 Processor -- start 2023-11-26T07:18:57.294 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:57.288+0000 7f7f5969c640 1 -- start start 2023-11-26T07:18:57.296 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:57.288+0000 7f7f5969c640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f7f40003980 0x7f7f5413aa30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:18:57.297 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:57.288+0000 7f7f5969c640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f7f54141030 0x7f7f54141680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:18:57.297 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:57.288+0000 7f7f5969c640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f7f54159ce0 0x7f7f5415a160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:18:57.297 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:57.288+0000 7f7f5969c640 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7f5415a6a0 con 0x7f7f54141030 2023-11-26T07:18:57.297 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:57.288+0000 7f7f5969c640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7f54159690 con 0x7f7f54159ce0 2023-11-26T07:18:57.297 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:57.288+0000 7f7f5969c640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7f5413d560 con 0x7f7f40003980 2023-11-26T07:18:57.297 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:57.288+0000 7f7f4a7fc640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f7f54141030 0x7f7f54141680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:18:57.297 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:57.288+0000 7f7f4a7fc640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f7f54141030 0x7f7f54141680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:51750/0 (socket says 172.21.15.143:51750) 2023-11-26T07:18:57.297 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:57.288+0000 7f7f4a7fc640 1 -- 172.21.15.143:0/1942544898 learned_addr learned my addr 172.21.15.143:0/1942544898 (peer_addr_for_me v2:172.21.15.143:0/0) 2023-11-26T07:18:57.297 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:57.288+0000 7f7f52ffd640 1 --2- 172.21.15.143:0/1942544898 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f7f40003980 0x7f7f5413aa30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:18:57.297 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:57.288+0000 7f7f537fe640 1 --2- 172.21.15.143:0/1942544898 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f7f54159ce0 0x7f7f5415a160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:18:57.297 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:57.288+0000 7f7f4a7fc640 1 -- 172.21.15.143:0/1942544898 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f7f40003980 msgr2=0x7f7f5413aa30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:18:57.298 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:57.288+0000 7f7f4a7fc640 1 --2- 172.21.15.143:0/1942544898 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f7f40003980 0x7f7f5413aa30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:18:57.298 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:57.288+0000 7f7f4a7fc640 1 -- 172.21.15.143:0/1942544898 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f7f54159ce0 msgr2=0x7f7f5415a160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:18:57.298 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:57.288+0000 7f7f4a7fc640 1 --2- 172.21.15.143:0/1942544898 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f7f54159ce0 0x7f7f5415a160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:18:57.298 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:57.288+0000 7f7f4a7fc640 1 -- 172.21.15.143:0/1942544898 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7f5406ae30 con 0x7f7f54141030 2023-11-26T07:18:57.298 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:57.288+0000 7f7f537fe640 1 --2- 172.21.15.143:0/1942544898 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f7f54159ce0 0x7f7f5415a160 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-26T07:18:57.298 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:57.288+0000 7f7f4a7fc640 1 --2- 172.21.15.143:0/1942544898 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f7f54141030 0x7f7f54141680 secure :-1 s=READY pgs=417 cs=0 l=1 rev1=1 crypto rx=0x7f7f3c001000 tx=0x7f7f3c00eb50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:18:57.298 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:57.288+0000 7f7f4bfff640 1 -- 172.21.15.143:0/1942544898 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7f3c00c690 con 0x7f7f54141030 2023-11-26T07:18:57.298 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:57.288+0000 7f7f4bfff640 1 -- 172.21.15.143:0/1942544898 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7f3c00c7f0 con 0x7f7f54141030 2023-11-26T07:18:57.298 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:57.288+0000 7f7f5969c640 1 -- 172.21.15.143:0/1942544898 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7f5413d840 con 0x7f7f54141030 2023-11-26T07:18:57.298 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:57.288+0000 7f7f4bfff640 1 -- 172.21.15.143:0/1942544898 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7f3c00cb60 con 0x7f7f54141030 2023-11-26T07:18:57.299 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:57.292+0000 7f7f5969c640 1 -- 172.21.15.143:0/1942544898 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7f5413de00 con 0x7f7f54141030 2023-11-26T07:18:57.299 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:57.292+0000 7f7f5969c640 1 -- 172.21.15.143:0/1942544898 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7f54157460 con 0x7f7f54141030 2023-11-26T07:18:57.299 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:57.292+0000 7f7f4bfff640 1 -- 172.21.15.143:0/1942544898 <== mon.0 v2:172.21.15.143:3300/0 4 ==== mgrmap(e 5) v1 ==== 85852+0+0 (secure 0 0 0) 0x7f7f3c00ccc0 con 0x7f7f54141030 2023-11-26T07:18:57.300 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:57.292+0000 7f7f4bfff640 1 --2- 172.21.15.143:0/1942544898 >> [v2:172.21.15.143:6832/2150557544,v1:172.21.15.143:6833/2150557544] conn(0x7f7f340664a0 0x7f7f34068960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:18:57.300 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:57.292+0000 7f7f4bfff640 1 -- 172.21.15.143:0/1942544898 <== mon.0 v2:172.21.15.143:3300/0 5 ==== osd_map(33..33 src has 1..33) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f7f3c083af0 con 0x7f7f54141030 2023-11-26T07:18:57.300 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:57.292+0000 7f7f52ffd640 1 --2- 172.21.15.143:0/1942544898 >> [v2:172.21.15.143:6832/2150557544,v1:172.21.15.143:6833/2150557544] conn(0x7f7f340664a0 0x7f7f34068960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:18:57.300 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:57.292+0000 7f7f52ffd640 1 --2- 172.21.15.143:0/1942544898 >> [v2:172.21.15.143:6832/2150557544,v1:172.21.15.143:6833/2150557544] conn(0x7f7f340664a0 0x7f7f34068960 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7f7f440ad000 tx=0x7f7f440b3040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:18:57.303 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:57.296+0000 7f7f4bfff640 1 -- 172.21.15.143:0/1942544898 <== mon.0 v2:172.21.15.143:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172649 (secure 0 0 0) 0x7f7f3c050f70 con 0x7f7f54141030 2023-11-26T07:18:57.451 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:57.444+0000 7f7f5969c640 1 -- 172.21.15.143:0/1942544898 --> [v2:172.21.15.143:6832/2150557544,v1:172.21.15.143:6833/2150557544] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f7f5406acb0 con 0x7f7f340664a0 2023-11-26T07:18:57.458 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:57.452+0000 7f7f4bfff640 1 -- 172.21.15.143:0/1942544898 <== mgr.4100 v2:172.21.15.143:6832/2150557544 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+284432 (secure 0 0 0) 0x7f7f440a6750 con 0x7f7f340664a0 2023-11-26T07:18:57.461 INFO:teuthology.orchestra.run.smithi143.stdout: 2023-11-26T07:18:57.463 INFO:teuthology.orchestra.run.smithi143.stderr:dumped all 2023-11-26T07:18:57.464 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:57.456+0000 7f7f5969c640 1 -- 172.21.15.143:0/1942544898 >> [v2:172.21.15.143:6832/2150557544,v1:172.21.15.143:6833/2150557544] conn(0x7f7f340664a0 msgr2=0x7f7f34068960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:18:57.464 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:57.456+0000 7f7f5969c640 1 --2- 172.21.15.143:0/1942544898 >> [v2:172.21.15.143:6832/2150557544,v1:172.21.15.143:6833/2150557544] conn(0x7f7f340664a0 0x7f7f34068960 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7f7f440ad000 tx=0x7f7f440b3040 comp rx=0 tx=0).stop 2023-11-26T07:18:57.464 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:57.456+0000 7f7f5969c640 1 -- 172.21.15.143:0/1942544898 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f7f54141030 msgr2=0x7f7f54141680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:18:57.464 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:57.456+0000 7f7f5969c640 1 --2- 172.21.15.143:0/1942544898 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f7f54141030 0x7f7f54141680 secure :-1 s=READY pgs=417 cs=0 l=1 rev1=1 crypto rx=0x7f7f3c001000 tx=0x7f7f3c00eb50 comp rx=0 tx=0).stop 2023-11-26T07:18:57.464 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:57.456+0000 7f7f5969c640 1 -- 172.21.15.143:0/1942544898 shutdown_connections 2023-11-26T07:18:57.464 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:57.456+0000 7f7f5969c640 1 --2- 172.21.15.143:0/1942544898 >> [v2:172.21.15.143:6832/2150557544,v1:172.21.15.143:6833/2150557544] conn(0x7f7f340664a0 0x7f7f34068960 unknown :-1 s=CLOSED pgs=224 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:18:57.464 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:57.456+0000 7f7f5969c640 1 --2- 172.21.15.143:0/1942544898 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f7f54159ce0 0x7f7f5415a160 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:18:57.464 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:57.456+0000 7f7f5969c640 1 --2- 172.21.15.143:0/1942544898 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f7f54141030 0x7f7f54141680 unknown :-1 s=CLOSED pgs=417 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:18:57.464 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:57.456+0000 7f7f5969c640 1 --2- 172.21.15.143:0/1942544898 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f7f40003980 0x7f7f5413aa30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:18:57.464 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:57.456+0000 7f7f5969c640 1 -- 172.21.15.143:0/1942544898 >> 172.21.15.143:0/1942544898 conn(0x7f7f540cff00 msgr2=0x7f7f54145270 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:18:57.464 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:57.456+0000 7f7f5969c640 1 -- 172.21.15.143:0/1942544898 shutdown_connections 2023-11-26T07:18:57.465 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:57.456+0000 7f7f5969c640 1 -- 172.21.15.143:0/1942544898 wait complete. 2023-11-26T07:18:57.471 INFO:tasks.ceph.mds.c.smithi143.stderr:2023-11-26T07:18:57.464+0000 7f240e3d0640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2023-11-26T07:18:57.481 INFO:teuthology.orchestra.run.smithi143.stdout:{"pg_ready":true,"pg_map":{"version":63,"stamp":"2023-11-26T07:18:55.705391+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":20,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":219996,"kb_used_data":3820,"kb_used_omap":12,"kb_used_meta":216051,"kb_avail":754754724,"statfs":{"total":773094113280,"available":772868837376,"internally_reserved":0,"allocated":3911680,"data_stored":2217537,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12749,"internal_metadata":221236787},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":36,"apply_latency_ms":36,"commit_latency_ns":36000000,"apply_latency_ns":36000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"5.076804"},"pg_stats":[{"pgid":"5.3f","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2023-11-26T07:18:51.645063+0000","last_change":"2023-11-26T07:18:51.645063+0000","last_active":"2023-11-26T07:18:51.625829+0000","last_peered":"2023-11-26T07:18:51.625829+0000","last_clean":"2023-11-26T07:18:51.625829+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-26T07:18:51.645063+0000","last_undegraded":"2023-11-26T07:18:51.645063+0000","last_fullsized":"2023-11-26T07:18:51.645063+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:18:51.625829+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:18:51.625829+0000","last_clean_scrub_stamp":"2023-11-26T07:18:51.625829+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.3e","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2023-11-26T07:18:51.643947+0000","last_change":"2023-11-26T07:18:51.643947+0000","last_active":"2023-11-26T07:18:51.625829+0000","last_peered":"2023-11-26T07:18:51.625829+0000","last_clean":"2023-11-26T07:18:51.625829+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-26T07:18:51.643947+0000","last_undegraded":"2023-11-26T07:18:51.643947+0000","last_fullsized":"2023-11-26T07:18:51.643947+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:18:51.625829+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:18:51.625829+0000","last_clean_scrub_stamp":"2023-11-26T07:18:51.625829+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.3d","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2023-11-26T07:18:51.634980+0000","last_change":"2023-11-26T07:18:51.634980+0000","last_active":"2023-11-26T07:18:51.625829+0000","last_peered":"2023-11-26T07:18:51.625829+0000","last_clean":"2023-11-26T07:18:51.625829+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-26T07:18:51.634980+0000","last_undegraded":"2023-11-26T07:18:51.634980+0000","last_fullsized":"2023-11-26T07:18:51.634980+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:18:51.625829+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:18:51.625829+0000","last_clean_scrub_stamp":"2023-11-26T07:18:51.625829+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.3c","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2023-11-26T07:18:51.634028+0000","last_change":"2023-11-26T07:18:51.634028+0000","last_active":"2023-11-26T07:18:51.625829+0000","last_peered":"2023-11-26T07:18:51.625829+0000","last_clean":"2023-11-26T07:18:51.625829+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-26T07:18:51.634028+0000","last_undegraded":"2023-11-26T07:18:51.634028+0000","last_fullsized":"2023-11-26T07:18:51.634028+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:18:51.625829+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:18:51.625829+0000","last_clean_scrub_stamp":"2023-11-26T07:18:51.625829+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.3b","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2023-11-26T07:18:51.651017+0000","last_change":"2023-11-26T07:18:51.651017+0000","last_active":"2023-11-26T07:18:51.625829+0000","last_peered":"2023-11-26T07:18:51.625829+0000","last_clean":"2023-11-26T07:18:51.625829+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-26T07:18:51.651017+0000","last_undegraded":"2023-11-26T07:18:51.651017+0000","last_fullsized":"2023-11-26T07:18:51.651017+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:18:51.625829+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:18:51.625829+0000","last_clean_scrub_stamp":"2023-11-26T07:18:51.625829+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.3a","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2023-11-26T07:18:51.644135+0000","last_change":"2023-11-26T07:18:51.644135+0000","last_active":"2023-11-26T07:18:51.625829+0000","last_peered":"2023-11-26T07:18:51.625829+0000","last_clean":"2023-11-26T07:18:51.625829+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-26T07:18:51.644135+0000","last_undegraded":"2023-11-26T07:18:51.644135+0000","last_fullsized":"2023-11-26T07:18:51.644135+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:18:51.625829+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:18:51.625829+0000","last_clean_scrub_stamp":"2023-11-26T07:18:51.625829+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.39","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2023-11-26T07:18:51.641562+0000","last_change":"2023-11-26T07:18:51.641562+0000","last_active":"2023-11-26T07:18:51.625829+0000","last_peered":"2023-11-26T07:18:51.625829+0000","last_clean":"2023-11-26T07:18:51.625829+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-26T07:18:51.641562+0000","last_undegraded":"2023-11-26T07:18:51.641562+0000","last_fullsized":"2023-11-26T07:18:51.641562+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:18:51.625829+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:18:51.625829+0000","last_clean_scrub_stamp":"2023-11-26T07:18:51.625829+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.38","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2023-11-26T07:18:51.646503+0000","last_change":"2023-11-26T07:18:51.646503+0000","last_active":"2023-11-26T07:18:51.625829+0000","last_peered":"2023-11-26T07:18:51.625829+0000","last_clean":"2023-11-26T07:18:51.625829+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-26T07:18:51.646503+0000","last_undegraded":"2023-11-26T07:18:51.646503+0000","last_fullsized":"2023-11-26T07:18:51.646503+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:18:51.625829+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:18:51.625829+0000","last_clean_scrub_stamp":"2023-11-26T07:18:51.625829+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.37","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2023-11-26T07:18:51.636576+0000","last_change":"2023-11-26T07:18:51.636576+0000","last_active":"2023-11-26T07:18:51.625829+0000","last_peered":"2023-11-26T07:18:51.625829+0000","last_clean":"2023-11-26T07:18:51.625829+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-26T07:18:51.636576+0000","last_undegraded":"2023-11-26T07:18:51.636576+0000","last_fullsized":"2023-11-26T07:18:51.636576+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:18:51.625829+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:18:51.625829+0000","last_clean_scrub_stamp":"2023-11-26T07:18:51.625829+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.36","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2023-11-26T07:18:51.633151+0000","last_change":"2023-11-26T07:18:51.633151+0000","last_active":"2023-11-26T07:18:51.625829+0000","last_peered":"2023-11-26T07:18:51.625829+0000","last_clean":"2023-11-26T07:18:51.625829+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-26T07:18:51.633151+0000","last_undegraded":"2023-11-26T07:18:51.633151+0000","last_fullsized":"2023-11-26T07:18:51.633151+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:18:51.625829+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:18:51.625829+0000","last_clean_scrub_stamp":"2023-11-26T07:18:51.625829+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.35","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2023-11-26T07:18:51.640729+0000","last_change":"2023-11-26T07:18:51.640729+0000","last_active":"2023-11-26T07:18:51.625829+0000","last_peered":"2023-11-26T07:18:51.625829+0000","last_clean":"2023-11-26T07:18:51.625829+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-26T07:18:51.640729+0000","last_undegraded":"2023-11-26T07:18:51.640729+0000","last_fullsized":"2023-11-26T07:18:51.640729+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:18:51.625829+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:18:51.625829+0000","last_clean_scrub_stamp":"2023-11-26T07:18:51.625829+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.34","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2023-11-26T07:18:51.641193+0000","last_change":"2023-11-26T07:18:51.641193+0000","last_active":"2023-11-26T07:18:51.625829+0000","last_peered":"2023-11-26T07:18:51.625829+0000","last_clean":"2023-11-26T07:18:51.625829+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-26T07:18:51.641193+0000","last_undegraded":"2023-11-26T07:18:51.641193+0000","last_fullsized":"2023-11-26T07:18:51.641193+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:18:51.625829+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:18:51.625829+0000","last_clean_scrub_stamp":"2023-11-26T07:18:51.625829+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.33","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2023-11-26T07:18:51.649682+0000","last_change":"2023-11-26T07:18:51.649682+0000","last_active":"2023-11-26T07:18:51.625829+0000","last_peered":"2023-11-26T07:18:51.625829+0000","last_clean":"2023-11-26T07:18:51.625829+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-26T07:18:51.649682+0000","last_undegraded":"2023-11-26T07:18:51.649682+0000","last_fullsized":"2023-11-26T07:18:51.649682+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:18:51.625829+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:18:51.625829+0000","last_clean_scrub_stamp":"2023-11-26T07:18:51.625829+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.32","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2023-11-26T07:18:51.643138+0000","last_change":"2023-11-26T07:18:51.643138+0000","last_active":"2023-11-26T07:18:51.625829+0000","last_peered":"2023-11-26T07:18:51.625829+0000","last_clean":"2023-11-26T07:18:51.625829+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-26T07:18:51.643138+0000","last_undegraded":"2023-11-26T07:18:51.643138+0000","last_fullsized":"2023-11-26T07:18:51.643138+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:18:51.625829+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:18:51.625829+0000","last_clean_scrub_stamp":"2023-11-26T07:18:51.625829+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.31","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2023-11-26T07:18:51.634598+0000","last_change":"2023-11-26T07:18:51.634598+0000","last_active":"2023-11-26T07:18:51.625829+0000","last_peered":"2023-11-26T07:18:51.625829+0000","last_clean":"2023-11-26T07:18:51.625829+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-26T07:18:51.634598+0000","last_undegraded":"2023-11-26T07:18:51.634598+0000","last_fullsized":"2023-11-26T07:18:51.634598+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:18:51.625829+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:18:51.625829+0000","last_clean_scrub_stamp":"2023-11-26T07:18:51.625829+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.30","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2023-11-26T07:18:51.640197+0000","last_change":"2023-11-26T07:18:51.640197+0000","last_active":"2023-11-26T07:18:51.625829+0000","last_peered":"2023-11-26T07:18:51.625829+0000","last_clean":"2023-11-26T07:18:51.625829+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-26T07:18:51.640197+0000","last_undegraded":"2023-11-26T07:18:51.640197+0000","last_fullsized":"2023-11-26T07:18:51.640197+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:18:51.625829+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:18:51.625829+0000","last_clean_scrub_stamp":"2023-11-26T07:18:51.625829+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.2f","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2023-11-26T07:18:51.641801+0000","last_change":"2023-11-26T07:18:51.641801+0000","last_active":"2023-11-26T07:18:51.625829+0000","last_peered":"2023-11-26T07:18:51.625829+0000","last_clean":"2023-11-26T07:18:51.625829+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-26T07:18:51.641801+0000","last_undegraded":"2023-11-26T07:18:51.641801+0000","last_fullsized":"2023-11-26T07:18:51.641801+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:18:51.625829+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:18:51.625829+0000","last_clean_scrub_stamp":"2023-11-26T07:18:51.625829+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.2e","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2023-11-26T07:18:51.643556+0000","last_change":"2023-11-26T07:18:51.643556+0000","last_active":"2023-11-26T07:18:51.625829+0000","last_peered":"2023-11-26T07:18:51.625829+0000","last_clean":"2023-11-26T07:18:51.625829+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-26T07:18:51.643556+0000","last_undegraded":"2023-11-26T07:18:51.643556+0000","last_fullsized":"2023-11-26T07:18:51.643556+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:18:51.625829+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:18:51.625829+0000","last_clean_scrub_stamp":"2023-11-26T07:18:51.625829+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.2d","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2023-11-26T07:18:51.642785+0000","last_change":"2023-11-26T07:18:51.642785+0000","last_active":"2023-11-26T07:18:51.625829+0000","last_peered":"2023-11-26T07:18:51.625829+0000","last_clean":"2023-11-26T07:18:51.625829+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-26T07:18:51.642785+0000","last_undegraded":"2023-11-26T07:18:51.642785+0000","last_fullsized":"2023-11-26T07:18:51.642785+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:18:51.625829+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:18:51.625829+0000","last_clean_scrub_stamp":"2023-11-26T07:18:51.625829+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.2c","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2023-11-26T07:18:51.651641+0000","last_change":"2023-11-26T07:18:51.651641+0000","last_active":"2023-11-26T07:18:51.625829+0000","last_peered":"2023-11-26T07:18:51.625829+0000","last_clean":"2023-11-26T07:18:51.625829+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-26T07:18:51.651641+0000","last_undegraded":"2023-11-26T07:18:51.651641+0000","last_fullsized":"2023-11-26T07:18:51.651641+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:18:51.625829+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:18:51.625829+0000","last_clean_scrub_stamp":"2023-11-26T07:18:51.625829+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.2b","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2023-11-26T07:18:51.637758+0000","last_change":"2023-11-26T07:18:51.637758+0000","last_active":"2023-11-26T07:18:51.625829+0000","last_peered":"2023-11-26T07:18:51.625829+0000","last_clean":"2023-11-26T07:18:51.625829+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-26T07:18:51.637758+0000","last_undegraded":"2023-11-26T07:18:51.637758+0000","last_fullsized":"2023-11-26T07:18:51.637758+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:18:51.625829+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:18:51.625829+0000","last_clean_scrub_stamp":"2023-11-26T07:18:51.625829+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.2a","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2023-11-26T07:18:51.636183+0000","last_change":"2023-11-26T07:18:51.636183+0000","last_active":"2023-11-26T07:18:51.625829+0000","last_peered":"2023-11-26T07:18:51.625829+0000","last_clean":"2023-11-26T07:18:51.625829+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-26T07:18:51.636183+0000","last_undegraded":"2023-11-26T07:18:51.636183+0000","last_fullsized":"2023-11-26T07:18:51.636183+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:18:51.625829+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:18:51.625829+0000","last_clean_scrub_stamp":"2023-11-26T07:18:51.625829+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.29","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2023-11-26T07:18:51.641884+0000","last_change":"2023-11-26T07:18:51.641884+0000","last_active":"2023-11-26T07:18:51.625829+0000","last_peered":"2023-11-26T07:18:51.625829+0000","last_clean":"2023-11-26T07:18:51.625829+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-26T07:18:51.641884+0000","last_undegraded":"2023-11-26T07:18:51.641884+0000","last_fullsized":"2023-11-26T07:18:51.641884+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:18:51.625829+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:18:51.625829+0000","last_clean_scrub_stamp":"2023-11-26T07:18:51.625829+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.28","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2023-11-26T07:18:51.635231+0000","last_change":"2023-11-26T07:18:51.635231+0000","last_active":"2023-11-26T07:18:51.625829+0000","last_peered":"2023-11-26T07:18:51.625829+0000","last_clean":"2023-11-26T07:18:51.625829+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-26T07:18:51.635231+0000","last_undegraded":"2023-11-26T07:18:51.635231+0000","last_fullsized":"2023-11-26T07:18:51.635231+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:18:51.625829+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:18:51.625829+0000","last_clean_scrub_stamp":"2023-11-26T07:18:51.625829+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.27","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2023-11-26T07:18:51.635689+0000","last_change":"2023-11-26T07:18:51.635689+0000","last_active":"2023-11-26T07:18:51.625829+0000","last_peered":"2023-11-26T07:18:51.625829+0000","last_clean":"2023-11-26T07:18:51.625829+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-26T07:18:51.635689+0000","last_undegraded":"2023-11-26T07:18:51.635689+0000","last_fullsized":"2023-11-26T07:18:51.635689+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:18:51.625829+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:18:51.625829+0000","last_clean_scrub_stamp":"2023-11-26T07:18:51.625829+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.26","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2023-11-26T07:18:51.643037+0000","last_change":"2023-11-26T07:18:51.643037+0000","last_active":"2023-11-26T07:18:51.625829+0000","last_peered":"2023-11-26T07:18:51.625829+0000","last_clean":"2023-11-26T07:18:51.625829+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-26T07:18:51.643037+0000","last_undegraded":"2023-11-26T07:18:51.643037+0000","last_fullsized":"2023-11-26T07:18:51.643037+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:18:51.625829+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:18:51.625829+0000","last_clean_scrub_stamp":"2023-11-26T07:18:51.625829+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.25","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2023-11-26T07:18:51.634261+0000","last_change":"2023-11-26T07:18:51.634261+0000","last_active":"2023-11-26T07:18:51.625829+0000","last_peered":"2023-11-26T07:18:51.625829+0000","last_clean":"2023-11-26T07:18:51.625829+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-26T07:18:51.634261+0000","last_undegraded":"2023-11-26T07:18:51.634261+0000","last_fullsized":"2023-11-26T07:18:51.634261+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:18:51.625829+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:18:51.625829+0000","last_clean_scrub_stamp":"2023-11-26T07:18:51.625829+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.24","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2023-11-26T07:18:51.637792+0000","last_change":"2023-11-26T07:18:51.637792+0000","last_active":"2023-11-26T07:18:51.625829+0000","last_peered":"2023-11-26T07:18:51.625829+0000","last_clean":"2023-11-26T07:18:51.625829+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-26T07:18:51.637792+0000","last_undegraded":"2023-11-26T07:18:51.637792+0000","last_fullsized":"2023-11-26T07:18:51.637792+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:18:51.625829+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:18:51.625829+0000","last_clean_scrub_stamp":"2023-11-26T07:18:51.625829+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.23","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2023-11-26T07:18:51.637010+0000","last_change":"2023-11-26T07:18:51.637010+0000","last_active":"2023-11-26T07:18:51.625829+0000","last_peered":"2023-11-26T07:18:51.625829+0000","last_clean":"2023-11-26T07:18:51.625829+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-26T07:18:51.637010+0000","last_undegraded":"2023-11-26T07:18:51.637010+0000","last_fullsized":"2023-11-26T07:18:51.637010+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:18:51.625829+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:18:51.625829+0000","last_clean_scrub_stamp":"2023-11-26T07:18:51.625829+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.22","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2023-11-26T07:18:51.644576+0000","last_change":"2023-11-26T07:18:51.644576+0000","last_active":"2023-11-26T07:18:51.625829+0000","last_peered":"2023-11-26T07:18:51.625829+0000","last_clean":"2023-11-26T07:18:51.625829+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-26T07:18:51.644576+0000","last_undegraded":"2023-11-26T07:18:51.644576+0000","last_fullsized":"2023-11-26T07:18:51.644576+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:18:51.625829+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:18:51.625829+0000","last_clean_scrub_stamp":"2023-11-26T07:18:51.625829+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.21","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2023-11-26T07:18:51.644802+0000","last_change":"2023-11-26T07:18:51.644802+0000","last_active":"2023-11-26T07:18:51.625829+0000","last_peered":"2023-11-26T07:18:51.625829+0000","last_clean":"2023-11-26T07:18:51.625829+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-26T07:18:51.644802+0000","last_undegraded":"2023-11-26T07:18:51.644802+0000","last_fullsized":"2023-11-26T07:18:51.644802+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:18:51.625829+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:18:51.625829+0000","last_clean_scrub_stamp":"2023-11-26T07:18:51.625829+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.20","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2023-11-26T07:18:51.646001+0000","last_change":"2023-11-26T07:18:51.646001+0000","last_active":"2023-11-26T07:18:51.625829+0000","last_peered":"2023-11-26T07:18:51.625829+0000","last_clean":"2023-11-26T07:18:51.625829+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-26T07:18:51.646001+0000","last_undegraded":"2023-11-26T07:18:51.646001+0000","last_fullsized":"2023-11-26T07:18:51.646001+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:18:51.625829+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:18:51.625829+0000","last_clean_scrub_stamp":"2023-11-26T07:18:51.625829+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.1e","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2023-11-26T07:18:51.641565+0000","last_change":"2023-11-26T07:18:51.641565+0000","last_active":"2023-11-26T07:18:51.625829+0000","last_peered":"2023-11-26T07:18:51.625829+0000","last_clean":"2023-11-26T07:18:51.625829+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-26T07:18:51.641565+0000","last_undegraded":"2023-11-26T07:18:51.641565+0000","last_fullsized":"2023-11-26T07:18:51.641565+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:18:51.625829+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:18:51.625829+0000","last_clean_scrub_stamp":"2023-11-26T07:18:51.625829+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1f","version":"0'0","reported_seq":9,"reported_epoch":31,"state":"active+clean","last_fresh":"2023-11-26T07:18:51.664628+0000","last_change":"2023-11-26T07:18:51.661754+0000","last_active":"2023-11-26T07:18:51.664628+0000","last_peered":"2023-11-26T07:18:51.664628+0000","last_clean":"2023-11-26T07:18:51.664628+0000","last_became_active":"2023-11-26T07:18:51.661410+0000","last_became_peered":"2023-11-26T07:18:51.661410+0000","last_unstale":"2023-11-26T07:18:51.664628+0000","last_undegraded":"2023-11-26T07:18:51.664628+0000","last_fullsized":"2023-11-26T07:18:51.664628+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:18:50.617605+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:18:50.617605+0000","last_clean_scrub_stamp":"2023-11-26T07:18:50.617605+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T12:04:39.473159+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.1f","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2023-11-26T07:18:51.642728+0000","last_change":"2023-11-26T07:18:51.642728+0000","last_active":"2023-11-26T07:18:51.625829+0000","last_peered":"2023-11-26T07:18:51.625829+0000","last_clean":"2023-11-26T07:18:51.625829+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-26T07:18:51.642728+0000","last_undegraded":"2023-11-26T07:18:51.642728+0000","last_fullsized":"2023-11-26T07:18:51.642728+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:18:51.625829+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:18:51.625829+0000","last_clean_scrub_stamp":"2023-11-26T07:18:51.625829+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.1e","version":"0'0","reported_seq":9,"reported_epoch":31,"state":"active+clean","last_fresh":"2023-11-26T07:18:51.656785+0000","last_change":"2023-11-26T07:18:51.652999+0000","last_active":"2023-11-26T07:18:51.656785+0000","last_peered":"2023-11-26T07:18:51.656785+0000","last_clean":"2023-11-26T07:18:51.656785+0000","last_became_active":"2023-11-26T07:18:51.652268+0000","last_became_peered":"2023-11-26T07:18:51.652268+0000","last_unstale":"2023-11-26T07:18:51.656785+0000","last_undegraded":"2023-11-26T07:18:51.656785+0000","last_fullsized":"2023-11-26T07:18:51.656785+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:18:50.617605+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:18:50.617605+0000","last_clean_scrub_stamp":"2023-11-26T07:18:50.617605+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T17:27:24.322165+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.1c","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2023-11-26T07:18:51.638001+0000","last_change":"2023-11-26T07:18:51.638001+0000","last_active":"2023-11-26T07:18:51.625829+0000","last_peered":"2023-11-26T07:18:51.625829+0000","last_clean":"2023-11-26T07:18:51.625829+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-26T07:18:51.638001+0000","last_undegraded":"2023-11-26T07:18:51.638001+0000","last_fullsized":"2023-11-26T07:18:51.638001+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:18:51.625829+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:18:51.625829+0000","last_clean_scrub_stamp":"2023-11-26T07:18:51.625829+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.1d","version":"0'0","reported_seq":9,"reported_epoch":31,"state":"active+clean","last_fresh":"2023-11-26T07:18:51.661518+0000","last_change":"2023-11-26T07:18:51.653241+0000","last_active":"2023-11-26T07:18:51.661518+0000","last_peered":"2023-11-26T07:18:51.661518+0000","last_clean":"2023-11-26T07:18:51.661518+0000","last_became_active":"2023-11-26T07:18:51.652971+0000","last_became_peered":"2023-11-26T07:18:51.652971+0000","last_unstale":"2023-11-26T07:18:51.661518+0000","last_undegraded":"2023-11-26T07:18:51.661518+0000","last_fullsized":"2023-11-26T07:18:51.661518+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:18:50.617605+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:18:50.617605+0000","last_clean_scrub_stamp":"2023-11-26T07:18:50.617605+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T10:40:39.059176+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.1d","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2023-11-26T07:18:51.633747+0000","last_change":"2023-11-26T07:18:51.633747+0000","last_active":"2023-11-26T07:18:51.625829+0000","last_peered":"2023-11-26T07:18:51.625829+0000","last_clean":"2023-11-26T07:18:51.625829+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-26T07:18:51.633747+0000","last_undegraded":"2023-11-26T07:18:51.633747+0000","last_fullsized":"2023-11-26T07:18:51.633747+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:18:51.625829+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:18:51.625829+0000","last_clean_scrub_stamp":"2023-11-26T07:18:51.625829+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1c","version":"0'0","reported_seq":9,"reported_epoch":31,"state":"active+clean","last_fresh":"2023-11-26T07:18:51.664404+0000","last_change":"2023-11-26T07:18:51.662450+0000","last_active":"2023-11-26T07:18:51.664404+0000","last_peered":"2023-11-26T07:18:51.664404+0000","last_clean":"2023-11-26T07:18:51.664404+0000","last_became_active":"2023-11-26T07:18:51.662140+0000","last_became_peered":"2023-11-26T07:18:51.662140+0000","last_unstale":"2023-11-26T07:18:51.664404+0000","last_undegraded":"2023-11-26T07:18:51.664404+0000","last_fullsized":"2023-11-26T07:18:51.664404+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:18:50.617605+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:18:50.617605+0000","last_clean_scrub_stamp":"2023-11-26T07:18:50.617605+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T17:15:12.064097+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.1a","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2023-11-26T07:18:51.640301+0000","last_change":"2023-11-26T07:18:51.640301+0000","last_active":"2023-11-26T07:18:51.625829+0000","last_peered":"2023-11-26T07:18:51.625829+0000","last_clean":"2023-11-26T07:18:51.625829+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-26T07:18:51.640301+0000","last_undegraded":"2023-11-26T07:18:51.640301+0000","last_fullsized":"2023-11-26T07:18:51.640301+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:18:51.625829+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:18:51.625829+0000","last_clean_scrub_stamp":"2023-11-26T07:18:51.625829+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.1b","version":"0'0","reported_seq":9,"reported_epoch":31,"state":"active+clean","last_fresh":"2023-11-26T07:18:51.658420+0000","last_change":"2023-11-26T07:18:51.654689+0000","last_active":"2023-11-26T07:18:51.658420+0000","last_peered":"2023-11-26T07:18:51.658420+0000","last_clean":"2023-11-26T07:18:51.658420+0000","last_became_active":"2023-11-26T07:18:51.653354+0000","last_became_peered":"2023-11-26T07:18:51.653354+0000","last_unstale":"2023-11-26T07:18:51.658420+0000","last_undegraded":"2023-11-26T07:18:51.658420+0000","last_fullsized":"2023-11-26T07:18:51.658420+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:18:50.617605+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:18:50.617605+0000","last_clean_scrub_stamp":"2023-11-26T07:18:50.617605+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T17:28:21.481079+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.1b","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2023-11-26T07:18:51.635218+0000","last_change":"2023-11-26T07:18:51.635218+0000","last_active":"2023-11-26T07:18:51.625829+0000","last_peered":"2023-11-26T07:18:51.625829+0000","last_clean":"2023-11-26T07:18:51.625829+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-26T07:18:51.635218+0000","last_undegraded":"2023-11-26T07:18:51.635218+0000","last_fullsized":"2023-11-26T07:18:51.635218+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:18:51.625829+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:18:51.625829+0000","last_clean_scrub_stamp":"2023-11-26T07:18:51.625829+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1a","version":"0'0","reported_seq":9,"reported_epoch":31,"state":"active+clean","last_fresh":"2023-11-26T07:18:51.656991+0000","last_change":"2023-11-26T07:18:51.653682+0000","last_active":"2023-11-26T07:18:51.656991+0000","last_peered":"2023-11-26T07:18:51.656991+0000","last_clean":"2023-11-26T07:18:51.656991+0000","last_became_active":"2023-11-26T07:18:51.653366+0000","last_became_peered":"2023-11-26T07:18:51.653366+0000","last_unstale":"2023-11-26T07:18:51.656991+0000","last_undegraded":"2023-11-26T07:18:51.656991+0000","last_fullsized":"2023-11-26T07:18:51.656991+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:18:50.617605+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:18:50.617605+0000","last_clean_scrub_stamp":"2023-11-26T07:18:50.617605+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T18:27:57.438972+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.18","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2023-11-26T07:18:51.634554+0000","last_change":"2023-11-26T07:18:51.634554+0000","last_active":"2023-11-26T07:18:51.625829+0000","last_peered":"2023-11-26T07:18:51.625829+0000","last_clean":"2023-11-26T07:18:51.625829+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-26T07:18:51.634554+0000","last_undegraded":"2023-11-26T07:18:51.634554+0000","last_fullsized":"2023-11-26T07:18:51.634554+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:18:51.625829+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:18:51.625829+0000","last_clean_scrub_stamp":"2023-11-26T07:18:51.625829+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.19","version":"0'0","reported_seq":9,"reported_epoch":31,"state":"active+clean","last_fresh":"2023-11-26T07:18:51.664030+0000","last_change":"2023-11-26T07:18:51.657763+0000","last_active":"2023-11-26T07:18:51.664030+0000","last_peered":"2023-11-26T07:18:51.664030+0000","last_clean":"2023-11-26T07:18:51.664030+0000","last_became_active":"2023-11-26T07:18:51.656677+0000","last_became_peered":"2023-11-26T07:18:51.656677+0000","last_unstale":"2023-11-26T07:18:51.664030+0000","last_undegraded":"2023-11-26T07:18:51.664030+0000","last_fullsized":"2023-11-26T07:18:51.664030+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:18:50.617605+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:18:50.617605+0000","last_clean_scrub_stamp":"2023-11-26T07:18:50.617605+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T18:26:18.407283+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.19","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2023-11-26T07:18:51.633130+0000","last_change":"2023-11-26T07:18:51.633130+0000","last_active":"2023-11-26T07:18:51.625829+0000","last_peered":"2023-11-26T07:18:51.625829+0000","last_clean":"2023-11-26T07:18:51.625829+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-26T07:18:51.633130+0000","last_undegraded":"2023-11-26T07:18:51.633130+0000","last_fullsized":"2023-11-26T07:18:51.633130+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:18:51.625829+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:18:51.625829+0000","last_clean_scrub_stamp":"2023-11-26T07:18:51.625829+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.18","version":"0'0","reported_seq":9,"reported_epoch":31,"state":"active+clean","last_fresh":"2023-11-26T07:18:51.658262+0000","last_change":"2023-11-26T07:18:51.656204+0000","last_active":"2023-11-26T07:18:51.658262+0000","last_peered":"2023-11-26T07:18:51.658262+0000","last_clean":"2023-11-26T07:18:51.658262+0000","last_became_active":"2023-11-26T07:18:51.655809+0000","last_became_peered":"2023-11-26T07:18:51.655809+0000","last_unstale":"2023-11-26T07:18:51.658262+0000","last_undegraded":"2023-11-26T07:18:51.658262+0000","last_fullsized":"2023-11-26T07:18:51.658262+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:18:50.617605+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:18:50.617605+0000","last_clean_scrub_stamp":"2023-11-26T07:18:50.617605+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T13:58:04.420017+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.16","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2023-11-26T07:18:51.635635+0000","last_change":"2023-11-26T07:18:51.635635+0000","last_active":"2023-11-26T07:18:51.625829+0000","last_peered":"2023-11-26T07:18:51.625829+0000","last_clean":"2023-11-26T07:18:51.625829+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-26T07:18:51.635635+0000","last_undegraded":"2023-11-26T07:18:51.635635+0000","last_fullsized":"2023-11-26T07:18:51.635635+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:18:51.625829+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:18:51.625829+0000","last_clean_scrub_stamp":"2023-11-26T07:18:51.625829+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.17","version":"0'0","reported_seq":9,"reported_epoch":31,"state":"active+clean","last_fresh":"2023-11-26T07:18:51.661481+0000","last_change":"2023-11-26T07:18:51.653322+0000","last_active":"2023-11-26T07:18:51.661481+0000","last_peered":"2023-11-26T07:18:51.661481+0000","last_clean":"2023-11-26T07:18:51.661481+0000","last_became_active":"2023-11-26T07:18:51.652388+0000","last_became_peered":"2023-11-26T07:18:51.652388+0000","last_unstale":"2023-11-26T07:18:51.661481+0000","last_undegraded":"2023-11-26T07:18:51.661481+0000","last_fullsized":"2023-11-26T07:18:51.661481+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:18:50.617605+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:18:50.617605+0000","last_clean_scrub_stamp":"2023-11-26T07:18:50.617605+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T07:50:54.298421+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.17","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2023-11-26T07:18:51.642485+0000","last_change":"2023-11-26T07:18:51.642485+0000","last_active":"2023-11-26T07:18:51.625829+0000","last_peered":"2023-11-26T07:18:51.625829+0000","last_clean":"2023-11-26T07:18:51.625829+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-26T07:18:51.642485+0000","last_undegraded":"2023-11-26T07:18:51.642485+0000","last_fullsized":"2023-11-26T07:18:51.642485+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:18:51.625829+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:18:51.625829+0000","last_clean_scrub_stamp":"2023-11-26T07:18:51.625829+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.16","version":"0'0","reported_seq":9,"reported_epoch":31,"state":"active+clean","last_fresh":"2023-11-26T07:18:51.650667+0000","last_change":"2023-11-26T07:18:51.644845+0000","last_active":"2023-11-26T07:18:51.650667+0000","last_peered":"2023-11-26T07:18:51.650667+0000","last_clean":"2023-11-26T07:18:51.650667+0000","last_became_active":"2023-11-26T07:18:51.644261+0000","last_became_peered":"2023-11-26T07:18:51.644261+0000","last_unstale":"2023-11-26T07:18:51.650667+0000","last_undegraded":"2023-11-26T07:18:51.650667+0000","last_fullsized":"2023-11-26T07:18:51.650667+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:18:50.617605+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:18:50.617605+0000","last_clean_scrub_stamp":"2023-11-26T07:18:50.617605+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T18:26:18.407283+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.14","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2023-11-26T07:18:51.643692+0000","last_change":"2023-11-26T07:18:51.643692+0000","last_active":"2023-11-26T07:18:51.625829+0000","last_peered":"2023-11-26T07:18:51.625829+0000","last_clean":"2023-11-26T07:18:51.625829+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-26T07:18:51.643692+0000","last_undegraded":"2023-11-26T07:18:51.643692+0000","last_fullsized":"2023-11-26T07:18:51.643692+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:18:51.625829+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:18:51.625829+0000","last_clean_scrub_stamp":"2023-11-26T07:18:51.625829+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.15","version":"0'0","reported_seq":9,"reported_epoch":31,"state":"active+clean","last_fresh":"2023-11-26T07:18:51.660677+0000","last_change":"2023-11-26T07:18:51.657428+0000","last_active":"2023-11-26T07:18:51.660677+0000","last_peered":"2023-11-26T07:18:51.660677+0000","last_clean":"2023-11-26T07:18:51.660677+0000","last_became_active":"2023-11-26T07:18:51.657075+0000","last_became_peered":"2023-11-26T07:18:51.657075+0000","last_unstale":"2023-11-26T07:18:51.660677+0000","last_undegraded":"2023-11-26T07:18:51.660677+0000","last_fullsized":"2023-11-26T07:18:51.660677+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:18:50.617605+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:18:50.617605+0000","last_clean_scrub_stamp":"2023-11-26T07:18:50.617605+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T07:50:54.298421+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.15","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2023-11-26T07:18:51.638905+0000","last_change":"2023-11-26T07:18:51.638905+0000","last_active":"2023-11-26T07:18:51.625829+0000","last_peered":"2023-11-26T07:18:51.625829+0000","last_clean":"2023-11-26T07:18:51.625829+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-26T07:18:51.638905+0000","last_undegraded":"2023-11-26T07:18:51.638905+0000","last_fullsized":"2023-11-26T07:18:51.638905+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:18:51.625829+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:18:51.625829+0000","last_clean_scrub_stamp":"2023-11-26T07:18:51.625829+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.14","version":"0'0","reported_seq":9,"reported_epoch":31,"state":"active+clean","last_fresh":"2023-11-26T07:18:51.651808+0000","last_change":"2023-11-26T07:18:51.647209+0000","last_active":"2023-11-26T07:18:51.651808+0000","last_peered":"2023-11-26T07:18:51.651808+0000","last_clean":"2023-11-26T07:18:51.651808+0000","last_became_active":"2023-11-26T07:18:51.647051+0000","last_became_peered":"2023-11-26T07:18:51.647051+0000","last_unstale":"2023-11-26T07:18:51.651808+0000","last_undegraded":"2023-11-26T07:18:51.651808+0000","last_fullsized":"2023-11-26T07:18:51.651808+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:18:50.617605+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:18:50.617605+0000","last_clean_scrub_stamp":"2023-11-26T07:18:50.617605+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T14:40:15.233288+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.12","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2023-11-26T07:18:51.633529+0000","last_change":"2023-11-26T07:18:51.633529+0000","last_active":"2023-11-26T07:18:51.625829+0000","last_peered":"2023-11-26T07:18:51.625829+0000","last_clean":"2023-11-26T07:18:51.625829+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-26T07:18:51.633529+0000","last_undegraded":"2023-11-26T07:18:51.633529+0000","last_fullsized":"2023-11-26T07:18:51.633529+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:18:51.625829+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:18:51.625829+0000","last_clean_scrub_stamp":"2023-11-26T07:18:51.625829+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.13","version":"0'0","reported_seq":9,"reported_epoch":31,"state":"active+clean","last_fresh":"2023-11-26T07:18:51.659054+0000","last_change":"2023-11-26T07:18:51.656339+0000","last_active":"2023-11-26T07:18:51.659054+0000","last_peered":"2023-11-26T07:18:51.659054+0000","last_clean":"2023-11-26T07:18:51.659054+0000","last_became_active":"2023-11-26T07:18:51.655998+0000","last_became_peered":"2023-11-26T07:18:51.655998+0000","last_unstale":"2023-11-26T07:18:51.659054+0000","last_undegraded":"2023-11-26T07:18:51.659054+0000","last_fullsized":"2023-11-26T07:18:51.659054+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:18:50.617605+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:18:50.617605+0000","last_clean_scrub_stamp":"2023-11-26T07:18:50.617605+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T11:29:07.700412+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.13","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2023-11-26T07:18:51.642640+0000","last_change":"2023-11-26T07:18:51.642640+0000","last_active":"2023-11-26T07:18:51.625829+0000","last_peered":"2023-11-26T07:18:51.625829+0000","last_clean":"2023-11-26T07:18:51.625829+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-26T07:18:51.642640+0000","last_undegraded":"2023-11-26T07:18:51.642640+0000","last_fullsized":"2023-11-26T07:18:51.642640+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:18:51.625829+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:18:51.625829+0000","last_clean_scrub_stamp":"2023-11-26T07:18:51.625829+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.12","version":"0'0","reported_seq":9,"reported_epoch":31,"state":"active+clean","last_fresh":"2023-11-26T07:18:51.663995+0000","last_change":"2023-11-26T07:18:51.658016+0000","last_active":"2023-11-26T07:18:51.663995+0000","last_peered":"2023-11-26T07:18:51.663995+0000","last_clean":"2023-11-26T07:18:51.663995+0000","last_became_active":"2023-11-26T07:18:51.657646+0000","last_became_peered":"2023-11-26T07:18:51.657646+0000","last_unstale":"2023-11-26T07:18:51.663995+0000","last_undegraded":"2023-11-26T07:18:51.663995+0000","last_fullsized":"2023-11-26T07:18:51.663995+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:18:50.617605+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:18:50.617605+0000","last_clean_scrub_stamp":"2023-11-26T07:18:50.617605+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T19:10:39.237269+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.10","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2023-11-26T07:18:51.645524+0000","last_change":"2023-11-26T07:18:51.645524+0000","last_active":"2023-11-26T07:18:51.625829+0000","last_peered":"2023-11-26T07:18:51.625829+0000","last_clean":"2023-11-26T07:18:51.625829+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-26T07:18:51.645524+0000","last_undegraded":"2023-11-26T07:18:51.645524+0000","last_fullsized":"2023-11-26T07:18:51.645524+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:18:51.625829+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:18:51.625829+0000","last_clean_scrub_stamp":"2023-11-26T07:18:51.625829+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.11","version":"0'0","reported_seq":9,"reported_epoch":31,"state":"active+clean","last_fresh":"2023-11-26T07:18:51.661545+0000","last_change":"2023-11-26T07:18:51.654685+0000","last_active":"2023-11-26T07:18:51.661545+0000","last_peered":"2023-11-26T07:18:51.661545+0000","last_clean":"2023-11-26T07:18:51.661545+0000","last_became_active":"2023-11-26T07:18:51.653542+0000","last_became_peered":"2023-11-26T07:18:51.653542+0000","last_unstale":"2023-11-26T07:18:51.661545+0000","last_undegraded":"2023-11-26T07:18:51.661545+0000","last_fullsized":"2023-11-26T07:18:51.661545+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:18:50.617605+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:18:50.617605+0000","last_clean_scrub_stamp":"2023-11-26T07:18:50.617605+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T14:17:17.432332+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.11","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2023-11-26T07:18:51.642391+0000","last_change":"2023-11-26T07:18:51.642391+0000","last_active":"2023-11-26T07:18:51.625829+0000","last_peered":"2023-11-26T07:18:51.625829+0000","last_clean":"2023-11-26T07:18:51.625829+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-26T07:18:51.642391+0000","last_undegraded":"2023-11-26T07:18:51.642391+0000","last_fullsized":"2023-11-26T07:18:51.642391+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:18:51.625829+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:18:51.625829+0000","last_clean_scrub_stamp":"2023-11-26T07:18:51.625829+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.10","version":"0'0","reported_seq":9,"reported_epoch":31,"state":"active+clean","last_fresh":"2023-11-26T07:18:51.661372+0000","last_change":"2023-11-26T07:18:51.652858+0000","last_active":"2023-11-26T07:18:51.661372+0000","last_peered":"2023-11-26T07:18:51.661372+0000","last_clean":"2023-11-26T07:18:51.661372+0000","last_became_active":"2023-11-26T07:18:51.652583+0000","last_became_peered":"2023-11-26T07:18:51.652583+0000","last_unstale":"2023-11-26T07:18:51.661372+0000","last_undegraded":"2023-11-26T07:18:51.661372+0000","last_fullsized":"2023-11-26T07:18:51.661372+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:18:50.617605+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:18:50.617605+0000","last_clean_scrub_stamp":"2023-11-26T07:18:50.617605+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T14:49:28.420672+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.e","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2023-11-26T07:18:51.635911+0000","last_change":"2023-11-26T07:18:51.635911+0000","last_active":"2023-11-26T07:18:51.625829+0000","last_peered":"2023-11-26T07:18:51.625829+0000","last_clean":"2023-11-26T07:18:51.625829+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-26T07:18:51.635911+0000","last_undegraded":"2023-11-26T07:18:51.635911+0000","last_fullsized":"2023-11-26T07:18:51.635911+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:18:51.625829+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:18:51.625829+0000","last_clean_scrub_stamp":"2023-11-26T07:18:51.625829+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.f","version":"0'0","reported_seq":9,"reported_epoch":31,"state":"active+clean","last_fresh":"2023-11-26T07:18:51.661667+0000","last_change":"2023-11-26T07:18:51.656212+0000","last_active":"2023-11-26T07:18:51.661667+0000","last_peered":"2023-11-26T07:18:51.661667+0000","last_clean":"2023-11-26T07:18:51.661667+0000","last_became_active":"2023-11-26T07:18:51.650720+0000","last_became_peered":"2023-11-26T07:18:51.650720+0000","last_unstale":"2023-11-26T07:18:51.661667+0000","last_undegraded":"2023-11-26T07:18:51.661667+0000","last_fullsized":"2023-11-26T07:18:51.661667+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:18:50.617605+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:18:50.617605+0000","last_clean_scrub_stamp":"2023-11-26T07:18:50.617605+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T14:40:15.233288+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.f","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2023-11-26T07:18:51.641273+0000","last_change":"2023-11-26T07:18:51.641273+0000","last_active":"2023-11-26T07:18:51.625829+0000","last_peered":"2023-11-26T07:18:51.625829+0000","last_clean":"2023-11-26T07:18:51.625829+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-26T07:18:51.641273+0000","last_undegraded":"2023-11-26T07:18:51.641273+0000","last_fullsized":"2023-11-26T07:18:51.641273+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:18:51.625829+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:18:51.625829+0000","last_clean_scrub_stamp":"2023-11-26T07:18:51.625829+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.e","version":"0'0","reported_seq":9,"reported_epoch":31,"state":"active+clean","last_fresh":"2023-11-26T07:18:51.664526+0000","last_change":"2023-11-26T07:18:51.660325+0000","last_active":"2023-11-26T07:18:51.664526+0000","last_peered":"2023-11-26T07:18:51.664526+0000","last_clean":"2023-11-26T07:18:51.664526+0000","last_became_active":"2023-11-26T07:18:51.659918+0000","last_became_peered":"2023-11-26T07:18:51.659918+0000","last_unstale":"2023-11-26T07:18:51.664526+0000","last_undegraded":"2023-11-26T07:18:51.664526+0000","last_fullsized":"2023-11-26T07:18:51.664526+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:18:50.617605+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:18:50.617605+0000","last_clean_scrub_stamp":"2023-11-26T07:18:50.617605+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T15:20:53.245654+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.c","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2023-11-26T07:18:51.633505+0000","last_change":"2023-11-26T07:18:51.633505+0000","last_active":"2023-11-26T07:18:51.625829+0000","last_peered":"2023-11-26T07:18:51.625829+0000","last_clean":"2023-11-26T07:18:51.625829+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-26T07:18:51.633505+0000","last_undegraded":"2023-11-26T07:18:51.633505+0000","last_fullsized":"2023-11-26T07:18:51.633505+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:18:51.625829+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:18:51.625829+0000","last_clean_scrub_stamp":"2023-11-26T07:18:51.625829+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.d","version":"0'0","reported_seq":9,"reported_epoch":31,"state":"active+clean","last_fresh":"2023-11-26T07:18:51.662230+0000","last_change":"2023-11-26T07:18:51.658440+0000","last_active":"2023-11-26T07:18:51.662230+0000","last_peered":"2023-11-26T07:18:51.662230+0000","last_clean":"2023-11-26T07:18:51.662230+0000","last_became_active":"2023-11-26T07:18:51.658161+0000","last_became_peered":"2023-11-26T07:18:51.658161+0000","last_unstale":"2023-11-26T07:18:51.662230+0000","last_undegraded":"2023-11-26T07:18:51.662230+0000","last_fullsized":"2023-11-26T07:18:51.662230+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:18:50.617605+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:18:50.617605+0000","last_clean_scrub_stamp":"2023-11-26T07:18:50.617605+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T08:54:02.907365+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.d","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2023-11-26T07:18:51.653086+0000","last_change":"2023-11-26T07:18:51.653086+0000","last_active":"2023-11-26T07:18:51.625829+0000","last_peered":"2023-11-26T07:18:51.625829+0000","last_clean":"2023-11-26T07:18:51.625829+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-26T07:18:51.653086+0000","last_undegraded":"2023-11-26T07:18:51.653086+0000","last_fullsized":"2023-11-26T07:18:51.653086+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:18:51.625829+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:18:51.625829+0000","last_clean_scrub_stamp":"2023-11-26T07:18:51.625829+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.c","version":"0'0","reported_seq":9,"reported_epoch":31,"state":"active+clean","last_fresh":"2023-11-26T07:18:51.658594+0000","last_change":"2023-11-26T07:18:51.654164+0000","last_active":"2023-11-26T07:18:51.658594+0000","last_peered":"2023-11-26T07:18:51.658594+0000","last_clean":"2023-11-26T07:18:51.658594+0000","last_became_active":"2023-11-26T07:18:51.653823+0000","last_became_peered":"2023-11-26T07:18:51.653823+0000","last_unstale":"2023-11-26T07:18:51.658594+0000","last_undegraded":"2023-11-26T07:18:51.658594+0000","last_fullsized":"2023-11-26T07:18:51.658594+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:18:50.617605+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:18:50.617605+0000","last_clean_scrub_stamp":"2023-11-26T07:18:50.617605+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T12:22:07.968357+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.a","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2023-11-26T07:18:51.649002+0000","last_change":"2023-11-26T07:18:51.649002+0000","last_active":"2023-11-26T07:18:51.625829+0000","last_peered":"2023-11-26T07:18:51.625829+0000","last_clean":"2023-11-26T07:18:51.625829+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-26T07:18:51.649002+0000","last_undegraded":"2023-11-26T07:18:51.649002+0000","last_fullsized":"2023-11-26T07:18:51.649002+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:18:51.625829+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:18:51.625829+0000","last_clean_scrub_stamp":"2023-11-26T07:18:51.625829+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.b","version":"0'0","reported_seq":9,"reported_epoch":31,"state":"active+clean","last_fresh":"2023-11-26T07:18:51.655810+0000","last_change":"2023-11-26T07:18:51.646694+0000","last_active":"2023-11-26T07:18:51.655810+0000","last_peered":"2023-11-26T07:18:51.655810+0000","last_clean":"2023-11-26T07:18:51.655810+0000","last_became_active":"2023-11-26T07:18:51.644567+0000","last_became_peered":"2023-11-26T07:18:51.644567+0000","last_unstale":"2023-11-26T07:18:51.655810+0000","last_undegraded":"2023-11-26T07:18:51.655810+0000","last_fullsized":"2023-11-26T07:18:51.655810+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:18:50.617605+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:18:50.617605+0000","last_clean_scrub_stamp":"2023-11-26T07:18:50.617605+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T12:04:39.473159+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.b","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2023-11-26T07:18:51.649020+0000","last_change":"2023-11-26T07:18:51.649020+0000","last_active":"2023-11-26T07:18:51.625829+0000","last_peered":"2023-11-26T07:18:51.625829+0000","last_clean":"2023-11-26T07:18:51.625829+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-26T07:18:51.649020+0000","last_undegraded":"2023-11-26T07:18:51.649020+0000","last_fullsized":"2023-11-26T07:18:51.649020+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:18:51.625829+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:18:51.625829+0000","last_clean_scrub_stamp":"2023-11-26T07:18:51.625829+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.a","version":"0'0","reported_seq":9,"reported_epoch":31,"state":"active+clean","last_fresh":"2023-11-26T07:18:51.664562+0000","last_change":"2023-11-26T07:18:51.660988+0000","last_active":"2023-11-26T07:18:51.664562+0000","last_peered":"2023-11-26T07:18:51.664562+0000","last_clean":"2023-11-26T07:18:51.664562+0000","last_became_active":"2023-11-26T07:18:51.660676+0000","last_became_peered":"2023-11-26T07:18:51.660676+0000","last_unstale":"2023-11-26T07:18:51.664562+0000","last_undegraded":"2023-11-26T07:18:51.664562+0000","last_fullsized":"2023-11-26T07:18:51.664562+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:18:50.617605+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:18:50.617605+0000","last_clean_scrub_stamp":"2023-11-26T07:18:50.617605+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T19:10:39.237269+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.8","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2023-11-26T07:18:51.649368+0000","last_change":"2023-11-26T07:18:51.649368+0000","last_active":"2023-11-26T07:18:51.625829+0000","last_peered":"2023-11-26T07:18:51.625829+0000","last_clean":"2023-11-26T07:18:51.625829+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-26T07:18:51.649368+0000","last_undegraded":"2023-11-26T07:18:51.649368+0000","last_fullsized":"2023-11-26T07:18:51.649368+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:18:51.625829+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:18:51.625829+0000","last_clean_scrub_stamp":"2023-11-26T07:18:51.625829+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.9","version":"0'0","reported_seq":9,"reported_epoch":31,"state":"active+clean","last_fresh":"2023-11-26T07:18:51.657644+0000","last_change":"2023-11-26T07:18:51.652977+0000","last_active":"2023-11-26T07:18:51.657644+0000","last_peered":"2023-11-26T07:18:51.657644+0000","last_clean":"2023-11-26T07:18:51.657644+0000","last_became_active":"2023-11-26T07:18:51.652670+0000","last_became_peered":"2023-11-26T07:18:51.652670+0000","last_unstale":"2023-11-26T07:18:51.657644+0000","last_undegraded":"2023-11-26T07:18:51.657644+0000","last_fullsized":"2023-11-26T07:18:51.657644+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:18:50.617605+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:18:50.617605+0000","last_clean_scrub_stamp":"2023-11-26T07:18:50.617605+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T15:39:09.212431+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.9","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2023-11-26T07:18:51.642082+0000","last_change":"2023-11-26T07:18:51.642082+0000","last_active":"2023-11-26T07:18:51.625829+0000","last_peered":"2023-11-26T07:18:51.625829+0000","last_clean":"2023-11-26T07:18:51.625829+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-26T07:18:51.642082+0000","last_undegraded":"2023-11-26T07:18:51.642082+0000","last_fullsized":"2023-11-26T07:18:51.642082+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:18:51.625829+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:18:51.625829+0000","last_clean_scrub_stamp":"2023-11-26T07:18:51.625829+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.8","version":"0'0","reported_seq":9,"reported_epoch":31,"state":"active+clean","last_fresh":"2023-11-26T07:18:51.662484+0000","last_change":"2023-11-26T07:18:51.659037+0000","last_active":"2023-11-26T07:18:51.662484+0000","last_peered":"2023-11-26T07:18:51.662484+0000","last_clean":"2023-11-26T07:18:51.662484+0000","last_became_active":"2023-11-26T07:18:51.658596+0000","last_became_peered":"2023-11-26T07:18:51.658596+0000","last_unstale":"2023-11-26T07:18:51.662484+0000","last_undegraded":"2023-11-26T07:18:51.662484+0000","last_fullsized":"2023-11-26T07:18:51.662484+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:18:50.617605+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:18:50.617605+0000","last_clean_scrub_stamp":"2023-11-26T07:18:50.617605+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T10:40:39.059176+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.6","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2023-11-26T07:18:51.651544+0000","last_change":"2023-11-26T07:18:51.651544+0000","last_active":"2023-11-26T07:18:51.625829+0000","last_peered":"2023-11-26T07:18:51.625829+0000","last_clean":"2023-11-26T07:18:51.625829+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-26T07:18:51.651544+0000","last_undegraded":"2023-11-26T07:18:51.651544+0000","last_fullsized":"2023-11-26T07:18:51.651544+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:18:51.625829+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:18:51.625829+0000","last_clean_scrub_stamp":"2023-11-26T07:18:51.625829+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.7","version":"0'0","reported_seq":9,"reported_epoch":31,"state":"active+clean","last_fresh":"2023-11-26T07:18:51.661322+0000","last_change":"2023-11-26T07:18:51.653789+0000","last_active":"2023-11-26T07:18:51.661322+0000","last_peered":"2023-11-26T07:18:51.661322+0000","last_clean":"2023-11-26T07:18:51.661322+0000","last_became_active":"2023-11-26T07:18:51.652136+0000","last_became_peered":"2023-11-26T07:18:51.652136+0000","last_unstale":"2023-11-26T07:18:51.661322+0000","last_undegraded":"2023-11-26T07:18:51.661322+0000","last_fullsized":"2023-11-26T07:18:51.661322+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:18:50.617605+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:18:50.617605+0000","last_clean_scrub_stamp":"2023-11-26T07:18:50.617605+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T18:26:18.407283+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.7","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2023-11-26T07:18:51.637655+0000","last_change":"2023-11-26T07:18:51.637655+0000","last_active":"2023-11-26T07:18:51.625829+0000","last_peered":"2023-11-26T07:18:51.625829+0000","last_clean":"2023-11-26T07:18:51.625829+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-26T07:18:51.637655+0000","last_undegraded":"2023-11-26T07:18:51.637655+0000","last_fullsized":"2023-11-26T07:18:51.637655+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:18:51.625829+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:18:51.625829+0000","last_clean_scrub_stamp":"2023-11-26T07:18:51.625829+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.6","version":"0'0","reported_seq":9,"reported_epoch":31,"state":"active+clean","last_fresh":"2023-11-26T07:18:51.648265+0000","last_change":"2023-11-26T07:18:51.644857+0000","last_active":"2023-11-26T07:18:51.648265+0000","last_peered":"2023-11-26T07:18:51.648265+0000","last_clean":"2023-11-26T07:18:51.648265+0000","last_became_active":"2023-11-26T07:18:51.644616+0000","last_became_peered":"2023-11-26T07:18:51.644616+0000","last_unstale":"2023-11-26T07:18:51.648265+0000","last_undegraded":"2023-11-26T07:18:51.648265+0000","last_fullsized":"2023-11-26T07:18:51.648265+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:18:50.617605+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:18:50.617605+0000","last_clean_scrub_stamp":"2023-11-26T07:18:50.617605+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T14:40:15.233288+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.5","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2023-11-26T07:18:51.641452+0000","last_change":"2023-11-26T07:18:51.641452+0000","last_active":"2023-11-26T07:18:51.625829+0000","last_peered":"2023-11-26T07:18:51.625829+0000","last_clean":"2023-11-26T07:18:51.625829+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-26T07:18:51.641452+0000","last_undegraded":"2023-11-26T07:18:51.641452+0000","last_fullsized":"2023-11-26T07:18:51.641452+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:18:51.625829+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:18:51.625829+0000","last_clean_scrub_stamp":"2023-11-26T07:18:51.625829+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.4","version":"0'0","reported_seq":9,"reported_epoch":31,"state":"active+clean","last_fresh":"2023-11-26T07:18:51.664083+0000","last_change":"2023-11-26T07:18:51.660079+0000","last_active":"2023-11-26T07:18:51.664083+0000","last_peered":"2023-11-26T07:18:51.664083+0000","last_clean":"2023-11-26T07:18:51.664083+0000","last_became_active":"2023-11-26T07:18:51.659661+0000","last_became_peered":"2023-11-26T07:18:51.659661+0000","last_unstale":"2023-11-26T07:18:51.664083+0000","last_undegraded":"2023-11-26T07:18:51.664083+0000","last_fullsized":"2023-11-26T07:18:51.664083+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:18:50.617605+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:18:50.617605+0000","last_clean_scrub_stamp":"2023-11-26T07:18:50.617605+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T12:04:39.473159+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.2","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2023-11-26T07:18:51.642362+0000","last_change":"2023-11-26T07:18:51.642362+0000","last_active":"2023-11-26T07:18:51.625829+0000","last_peered":"2023-11-26T07:18:51.625829+0000","last_clean":"2023-11-26T07:18:51.625829+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-26T07:18:51.642362+0000","last_undegraded":"2023-11-26T07:18:51.642362+0000","last_fullsized":"2023-11-26T07:18:51.642362+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:18:51.625829+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:18:51.625829+0000","last_clean_scrub_stamp":"2023-11-26T07:18:51.625829+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.3","version":"0'0","reported_seq":9,"reported_epoch":31,"state":"active+clean","last_fresh":"2023-11-26T07:18:51.655830+0000","last_change":"2023-11-26T07:18:51.647332+0000","last_active":"2023-11-26T07:18:51.655830+0000","last_peered":"2023-11-26T07:18:51.655830+0000","last_clean":"2023-11-26T07:18:51.655830+0000","last_became_active":"2023-11-26T07:18:51.647054+0000","last_became_peered":"2023-11-26T07:18:51.647054+0000","last_unstale":"2023-11-26T07:18:51.655830+0000","last_undegraded":"2023-11-26T07:18:51.655830+0000","last_fullsized":"2023-11-26T07:18:51.655830+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:18:50.617605+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:18:50.617605+0000","last_clean_scrub_stamp":"2023-11-26T07:18:50.617605+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T19:10:39.237269+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.3","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2023-11-26T07:18:51.642003+0000","last_change":"2023-11-26T07:18:51.642003+0000","last_active":"2023-11-26T07:18:51.625829+0000","last_peered":"2023-11-26T07:18:51.625829+0000","last_clean":"2023-11-26T07:18:51.625829+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-26T07:18:51.642003+0000","last_undegraded":"2023-11-26T07:18:51.642003+0000","last_fullsized":"2023-11-26T07:18:51.642003+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:18:51.625829+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:18:51.625829+0000","last_clean_scrub_stamp":"2023-11-26T07:18:51.625829+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.2","version":"0'0","reported_seq":9,"reported_epoch":31,"state":"active+clean","last_fresh":"2023-11-26T07:18:51.658369+0000","last_change":"2023-11-26T07:18:51.651994+0000","last_active":"2023-11-26T07:18:51.658369+0000","last_peered":"2023-11-26T07:18:51.658369+0000","last_clean":"2023-11-26T07:18:51.658369+0000","last_became_active":"2023-11-26T07:18:51.650255+0000","last_became_peered":"2023-11-26T07:18:51.650255+0000","last_unstale":"2023-11-26T07:18:51.658369+0000","last_undegraded":"2023-11-26T07:18:51.658369+0000","last_fullsized":"2023-11-26T07:18:51.658369+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:18:50.617605+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:18:50.617605+0000","last_clean_scrub_stamp":"2023-11-26T07:18:50.617605+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T17:27:24.322165+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.0","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2023-11-26T07:18:51.642260+0000","last_change":"2023-11-26T07:18:51.642260+0000","last_active":"2023-11-26T07:18:51.625829+0000","last_peered":"2023-11-26T07:18:51.625829+0000","last_clean":"2023-11-26T07:18:51.625829+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-26T07:18:51.642260+0000","last_undegraded":"2023-11-26T07:18:51.642260+0000","last_fullsized":"2023-11-26T07:18:51.642260+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:18:51.625829+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:18:51.625829+0000","last_clean_scrub_stamp":"2023-11-26T07:18:51.625829+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1","version":"0'0","reported_seq":9,"reported_epoch":31,"state":"active+clean","last_fresh":"2023-11-26T07:18:51.658380+0000","last_change":"2023-11-26T07:18:51.654435+0000","last_active":"2023-11-26T07:18:51.658380+0000","last_peered":"2023-11-26T07:18:51.658380+0000","last_clean":"2023-11-26T07:18:51.658380+0000","last_became_active":"2023-11-26T07:18:51.654107+0000","last_became_peered":"2023-11-26T07:18:51.654107+0000","last_unstale":"2023-11-26T07:18:51.658380+0000","last_undegraded":"2023-11-26T07:18:51.658380+0000","last_fullsized":"2023-11-26T07:18:51.658380+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:18:50.617605+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:18:50.617605+0000","last_clean_scrub_stamp":"2023-11-26T07:18:50.617605+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T15:50:37.023527+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.1","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2023-11-26T07:18:51.635350+0000","last_change":"2023-11-26T07:18:51.635350+0000","last_active":"2023-11-26T07:18:51.625829+0000","last_peered":"2023-11-26T07:18:51.625829+0000","last_clean":"2023-11-26T07:18:51.625829+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-26T07:18:51.635350+0000","last_undegraded":"2023-11-26T07:18:51.635350+0000","last_fullsized":"2023-11-26T07:18:51.635350+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:18:51.625829+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:18:51.625829+0000","last_clean_scrub_stamp":"2023-11-26T07:18:51.625829+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.0","version":"0'0","reported_seq":9,"reported_epoch":31,"state":"active+clean","last_fresh":"2023-11-26T07:18:51.661597+0000","last_change":"2023-11-26T07:18:51.655721+0000","last_active":"2023-11-26T07:18:51.661597+0000","last_peered":"2023-11-26T07:18:51.661597+0000","last_clean":"2023-11-26T07:18:51.661597+0000","last_became_active":"2023-11-26T07:18:51.655334+0000","last_became_peered":"2023-11-26T07:18:51.655334+0000","last_unstale":"2023-11-26T07:18:51.661597+0000","last_undegraded":"2023-11-26T07:18:51.661597+0000","last_fullsized":"2023-11-26T07:18:51.661597+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:18:50.617605+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:18:50.617605+0000","last_clean_scrub_stamp":"2023-11-26T07:18:50.617605+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T11:49:26.615849+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.4","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2023-11-26T07:18:51.643304+0000","last_change":"2023-11-26T07:18:51.643304+0000","last_active":"2023-11-26T07:18:51.625829+0000","last_peered":"2023-11-26T07:18:51.625829+0000","last_clean":"2023-11-26T07:18:51.625829+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-26T07:18:51.643304+0000","last_undegraded":"2023-11-26T07:18:51.643304+0000","last_fullsized":"2023-11-26T07:18:51.643304+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:18:51.625829+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:18:51.625829+0000","last_clean_scrub_stamp":"2023-11-26T07:18:51.625829+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.5","version":"0'0","reported_seq":9,"reported_epoch":31,"state":"active+clean","last_fresh":"2023-11-26T07:18:51.664594+0000","last_change":"2023-11-26T07:18:51.661135+0000","last_active":"2023-11-26T07:18:51.664594+0000","last_peered":"2023-11-26T07:18:51.664594+0000","last_clean":"2023-11-26T07:18:51.664594+0000","last_became_active":"2023-11-26T07:18:51.659814+0000","last_became_peered":"2023-11-26T07:18:51.659814+0000","last_unstale":"2023-11-26T07:18:51.664594+0000","last_undegraded":"2023-11-26T07:18:51.664594+0000","last_fullsized":"2023-11-26T07:18:51.664594+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:18:50.617605+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:18:50.617605+0000","last_clean_scrub_stamp":"2023-11-26T07:18:50.617605+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T17:37:35.256892+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"1.0","version":"15'117","reported_seq":173,"reported_epoch":30,"state":"active+clean","last_fresh":"2023-11-26T07:18:50.636811+0000","last_change":"2023-11-26T07:17:43.827802+0000","last_active":"2023-11-26T07:18:50.636811+0000","last_peered":"2023-11-26T07:18:50.636811+0000","last_clean":"2023-11-26T07:18:50.636811+0000","last_became_active":"2023-11-26T07:17:43.827308+0000","last_became_peered":"2023-11-26T07:17:43.827308+0000","last_unstale":"2023-11-26T07:18:50.636811+0000","last_undegraded":"2023-11-26T07:18:50.636811+0000","last_fullsized":"2023-11-26T07:18:50.636811+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:17:42.160945+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:17:42.160945+0000","last_clean_scrub_stamp":"2023-11-26T07:17:42.160945+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T08:17:54.167366+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":5,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":4,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574864,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27932,"kb_used_data":908,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343908,"statfs":{"total":96636764160,"available":96608161792,"internally_reserved":0,"allocated":929792,"data_stored":717618,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Nov 26 07:18:42 2023","interfaces":[{"interface":"back","average":{"1min":0.71499999999999997,"5min":0.71499999999999997,"15min":0.71499999999999997},"min":{"1min":0.54300000000000004,"5min":0.54300000000000004,"15min":0.54300000000000004},"max":{"1min":0.91500000000000004,"5min":0.91500000000000004,"15min":0.91500000000000004},"last":0.60099999999999998},{"interface":"front","average":{"1min":0.70899999999999996,"5min":0.70899999999999996,"15min":0.70899999999999996},"min":{"1min":0.52900000000000003,"5min":0.52900000000000003,"15min":0.52900000000000003},"max":{"1min":1.0980000000000001,"5min":1.0980000000000001,"15min":1.0980000000000001},"last":0.69199999999999995}]},{"osd":1,"last update":"Sun Nov 26 07:18:42 2023","interfaces":[{"interface":"back","average":{"1min":0.68200000000000005,"5min":0.68200000000000005,"15min":0.68200000000000005},"min":{"1min":0.47999999999999998,"5min":0.47999999999999998,"15min":0.47999999999999998},"max":{"1min":1.0669999999999999,"5min":1.0669999999999999,"15min":1.0669999999999999},"last":0.67300000000000004},{"interface":"front","average":{"1min":0.83899999999999997,"5min":0.83899999999999997,"15min":0.83899999999999997},"min":{"1min":0.47599999999999998,"5min":0.47599999999999998,"15min":0.47599999999999998},"max":{"1min":4.5309999999999997,"5min":4.5309999999999997,"15min":4.5309999999999997},"last":0.627}]},{"osd":2,"last update":"Sun Nov 26 07:18:42 2023","interfaces":[{"interface":"back","average":{"1min":0.67600000000000005,"5min":0.67600000000000005,"15min":0.67600000000000005},"min":{"1min":0.52000000000000002,"5min":0.52000000000000002,"15min":0.52000000000000002},"max":{"1min":0.873,"5min":0.873,"15min":0.873},"last":0.54400000000000004},{"interface":"front","average":{"1min":0.71299999999999997,"5min":0.71299999999999997,"15min":0.71299999999999997},"min":{"1min":0.54700000000000004,"5min":0.54700000000000004,"15min":0.54700000000000004},"max":{"1min":1.1339999999999999,"5min":1.1339999999999999,"15min":1.1339999999999999},"last":0.499}]},{"osd":3,"last update":"Sun Nov 26 07:18:42 2023","interfaces":[{"interface":"back","average":{"1min":0.69699999999999995,"5min":0.69699999999999995,"15min":0.69699999999999995},"min":{"1min":0.51800000000000002,"5min":0.51800000000000002,"15min":0.51800000000000002},"max":{"1min":1.026,"5min":1.026,"15min":1.026},"last":0.64300000000000002},{"interface":"front","average":{"1min":0.73299999999999998,"5min":0.73299999999999998,"15min":0.73299999999999998},"min":{"1min":0.50600000000000001,"5min":0.50600000000000001,"15min":0.50600000000000001},"max":{"1min":1.4490000000000001,"5min":1.4490000000000001,"15min":1.4490000000000001},"last":0.755}]},{"osd":4,"last update":"Sun Nov 26 07:18:42 2023","interfaces":[{"interface":"back","average":{"1min":0.59999999999999998,"5min":0.59999999999999998,"15min":0.59999999999999998},"min":{"1min":0.39400000000000002,"5min":0.39400000000000002,"15min":0.39400000000000002},"max":{"1min":0.94799999999999995,"5min":0.94799999999999995,"15min":0.94799999999999995},"last":0.47099999999999997},{"interface":"front","average":{"1min":0.67300000000000004,"5min":0.67300000000000004,"15min":0.67300000000000004},"min":{"1min":0.436,"5min":0.436,"15min":0.436},"max":{"1min":1.5509999999999999,"5min":1.5509999999999999,"15min":1.5509999999999999},"last":0.70899999999999996}]},{"osd":5,"last update":"Sun Nov 26 07:18:42 2023","interfaces":[{"interface":"back","average":{"1min":0.66600000000000004,"5min":0.66600000000000004,"15min":0.66600000000000004},"min":{"1min":0.443,"5min":0.443,"15min":0.443},"max":{"1min":1.415,"5min":1.415,"15min":1.415},"last":0.72599999999999998},{"interface":"front","average":{"1min":0.71399999999999997,"5min":0.71399999999999997,"15min":0.71399999999999997},"min":{"1min":0.53100000000000003,"5min":0.53100000000000003,"15min":0.53100000000000003},"max":{"1min":1.3859999999999999,"5min":1.3859999999999999,"15min":1.3859999999999999},"last":0.52800000000000002}]},{"osd":6,"last update":"Sun Nov 26 07:18:42 2023","interfaces":[{"interface":"back","average":{"1min":0.77100000000000002,"5min":0.77100000000000002,"15min":0.77100000000000002},"min":{"1min":0.64100000000000001,"5min":0.64100000000000001,"15min":0.64100000000000001},"max":{"1min":1.3069999999999999,"5min":1.3069999999999999,"15min":1.3069999999999999},"last":0.58399999999999996},{"interface":"front","average":{"1min":0.77100000000000002,"5min":0.77100000000000002,"15min":0.77100000000000002},"min":{"1min":0.57399999999999995,"5min":0.57399999999999995,"15min":0.57399999999999995},"max":{"1min":1.6759999999999999,"5min":1.6759999999999999,"15min":1.6759999999999999},"last":0.74299999999999999}]}]},{"osd":6,"up_from":13,"seq":55834574866,"num_pgs":3,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27348,"kb_used_data":328,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344492,"statfs":{"total":96636764160,"available":96608759808,"internally_reserved":0,"allocated":335872,"data_stored":127250,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Nov 26 07:18:45 2023","interfaces":[{"interface":"back","average":{"1min":0.68400000000000005,"5min":0.68400000000000005,"15min":0.68400000000000005},"min":{"1min":0.36699999999999999,"5min":0.36699999999999999,"15min":0.36699999999999999},"max":{"1min":1.7549999999999999,"5min":1.7549999999999999,"15min":1.7549999999999999},"last":0.64900000000000002},{"interface":"front","average":{"1min":0.95799999999999996,"5min":0.95799999999999996,"15min":0.95799999999999996},"min":{"1min":0.38300000000000001,"5min":0.38300000000000001,"15min":0.38300000000000001},"max":{"1min":5.7489999999999997,"5min":5.7489999999999997,"15min":5.7489999999999997},"last":0.56000000000000005}]},{"osd":1,"last update":"Sun Nov 26 07:18:45 2023","interfaces":[{"interface":"back","average":{"1min":0.91400000000000003,"5min":0.91400000000000003,"15min":0.91400000000000003},"min":{"1min":0.42099999999999999,"5min":0.42099999999999999,"15min":0.42099999999999999},"max":{"1min":5.4240000000000004,"5min":5.4240000000000004,"15min":5.4240000000000004},"last":0.58199999999999996},{"interface":"front","average":{"1min":0.89700000000000002,"5min":0.89700000000000002,"15min":0.89700000000000002},"min":{"1min":0.499,"5min":0.499,"15min":0.499},"max":{"1min":5.6020000000000003,"5min":5.6020000000000003,"15min":5.6020000000000003},"last":0.75}]},{"osd":2,"last update":"Sun Nov 26 07:18:45 2023","interfaces":[{"interface":"back","average":{"1min":0.69699999999999995,"5min":0.69699999999999995,"15min":0.69699999999999995},"min":{"1min":0.47599999999999998,"5min":0.47599999999999998,"15min":0.47599999999999998},"max":{"1min":1.8520000000000001,"5min":1.8520000000000001,"15min":1.8520000000000001},"last":0.60599999999999998},{"interface":"front","average":{"1min":0.73399999999999999,"5min":0.73399999999999999,"15min":0.73399999999999999},"min":{"1min":0.46700000000000003,"5min":0.46700000000000003,"15min":0.46700000000000003},"max":{"1min":1.9159999999999999,"5min":1.9159999999999999,"15min":1.9159999999999999},"last":0.51700000000000002}]},{"osd":3,"last update":"Sun Nov 26 07:18:45 2023","interfaces":[{"interface":"back","average":{"1min":0.90200000000000002,"5min":0.90200000000000002,"15min":0.90200000000000002},"min":{"1min":0.502,"5min":0.502,"15min":0.502},"max":{"1min":5.4390000000000001,"5min":5.4390000000000001,"15min":5.4390000000000001},"last":0.63400000000000001},{"interface":"front","average":{"1min":0.94499999999999995,"5min":0.94499999999999995,"15min":0.94499999999999995},"min":{"1min":0.54400000000000004,"5min":0.54400000000000004,"15min":0.54400000000000004},"max":{"1min":5.5490000000000004,"5min":5.5490000000000004,"15min":5.5490000000000004},"last":0.72799999999999998}]},{"osd":4,"last update":"Sun Nov 26 07:18:45 2023","interfaces":[{"interface":"back","average":{"1min":0.64200000000000002,"5min":0.64200000000000002,"15min":0.64200000000000002},"min":{"1min":0.46899999999999997,"5min":0.46899999999999997,"15min":0.46899999999999997},"max":{"1min":0.85099999999999998,"5min":0.85099999999999998,"15min":0.85099999999999998},"last":0.45500000000000002},{"interface":"front","average":{"1min":0.67500000000000004,"5min":0.67500000000000004,"15min":0.67500000000000004},"min":{"1min":0.40699999999999997,"5min":0.40699999999999997,"15min":0.40699999999999997},"max":{"1min":0.93300000000000005,"5min":0.93300000000000005,"15min":0.93300000000000005},"last":0.68700000000000006}]},{"osd":5,"last update":"Sun Nov 26 07:18:45 2023","interfaces":[{"interface":"back","average":{"1min":0.65900000000000003,"5min":0.65900000000000003,"15min":0.65900000000000003},"min":{"1min":0.42499999999999999,"5min":0.42499999999999999,"15min":0.42499999999999999},"max":{"1min":0.79300000000000004,"5min":0.79300000000000004,"15min":0.79300000000000004},"last":0.53400000000000003},{"interface":"front","average":{"1min":0.71299999999999997,"5min":0.71299999999999997,"15min":0.71299999999999997},"min":{"1min":0.436,"5min":0.436,"15min":0.436},"max":{"1min":0.93000000000000005,"5min":0.93000000000000005,"15min":0.93000000000000005},"last":0.70599999999999996}]},{"osd":7,"last update":"Sun Nov 26 07:18:45 2023","interfaces":[{"interface":"back","average":{"1min":0.74199999999999999,"5min":0.74199999999999999,"15min":0.74199999999999999},"min":{"1min":0.46600000000000003,"5min":0.46600000000000003,"15min":0.46600000000000003},"max":{"1min":0.998,"5min":0.998,"15min":0.998},"last":0.78200000000000003},{"interface":"front","average":{"1min":0.71599999999999997,"5min":0.71599999999999997,"15min":0.71599999999999997},"min":{"1min":0.498,"5min":0.498,"15min":0.498},"max":{"1min":0.86499999999999999,"5min":0.86499999999999999,"15min":0.86499999999999999},"last":0.66600000000000004}]}]},{"osd":5,"up_from":13,"seq":55834574864,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27360,"kb_used_data":340,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344480,"statfs":{"total":96636764160,"available":96608747520,"internally_reserved":0,"allocated":348160,"data_stored":133517,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Nov 26 07:18:45 2023","interfaces":[{"interface":"back","average":{"1min":0.66600000000000004,"5min":0.66600000000000004,"15min":0.66600000000000004},"min":{"1min":0.38100000000000001,"5min":0.38100000000000001,"15min":0.38100000000000001},"max":{"1min":1.0289999999999999,"5min":1.0289999999999999,"15min":1.0289999999999999},"last":0.59099999999999997},{"interface":"front","average":{"1min":0.66000000000000003,"5min":0.66000000000000003,"15min":0.66000000000000003},"min":{"1min":0.438,"5min":0.438,"15min":0.438},"max":{"1min":1.05,"5min":1.05,"15min":1.05},"last":0.54700000000000004}]},{"osd":1,"last update":"Sun Nov 26 07:18:45 2023","interfaces":[{"interface":"back","average":{"1min":0.66800000000000004,"5min":0.66800000000000004,"15min":0.66800000000000004},"min":{"1min":0.41599999999999998,"5min":0.41599999999999998,"15min":0.41599999999999998},"max":{"1min":0.86699999999999999,"5min":0.86699999999999999,"15min":0.86699999999999999},"last":0.79400000000000004},{"interface":"front","average":{"1min":0.68600000000000005,"5min":0.68600000000000005,"15min":0.68600000000000005},"min":{"1min":0.49199999999999999,"5min":0.49199999999999999,"15min":0.49199999999999999},"max":{"1min":0.98899999999999999,"5min":0.98899999999999999,"15min":0.98899999999999999},"last":0.52900000000000003}]},{"osd":2,"last update":"Sun Nov 26 07:18:45 2023","interfaces":[{"interface":"back","average":{"1min":0.67500000000000004,"5min":0.67500000000000004,"15min":0.67500000000000004},"min":{"1min":0.45900000000000002,"5min":0.45900000000000002,"15min":0.45900000000000002},"max":{"1min":0.89800000000000002,"5min":0.89800000000000002,"15min":0.89800000000000002},"last":0.67500000000000004},{"interface":"front","average":{"1min":0.67000000000000004,"5min":0.67000000000000004,"15min":0.67000000000000004},"min":{"1min":0.51200000000000001,"5min":0.51200000000000001,"15min":0.51200000000000001},"max":{"1min":0.89700000000000002,"5min":0.89700000000000002,"15min":0.89700000000000002},"last":0.73399999999999999}]},{"osd":3,"last update":"Sun Nov 26 07:18:45 2023","interfaces":[{"interface":"back","average":{"1min":0.71999999999999997,"5min":0.71999999999999997,"15min":0.71999999999999997},"min":{"1min":0.43099999999999999,"5min":0.43099999999999999,"15min":0.43099999999999999},"max":{"1min":1.0029999999999999,"5min":1.0029999999999999,"15min":1.0029999999999999},"last":0.629},{"interface":"front","average":{"1min":0.67500000000000004,"5min":0.67500000000000004,"15min":0.67500000000000004},"min":{"1min":0.44600000000000001,"5min":0.44600000000000001,"15min":0.44600000000000001},"max":{"1min":1.0209999999999999,"5min":1.0209999999999999,"15min":1.0209999999999999},"last":0.48199999999999998}]},{"osd":4,"last update":"Sun Nov 26 07:18:45 2023","interfaces":[{"interface":"back","average":{"1min":0.63500000000000001,"5min":0.63500000000000001,"15min":0.63500000000000001},"min":{"1min":0.39200000000000002,"5min":0.39200000000000002,"15min":0.39200000000000002},"max":{"1min":0.95099999999999996,"5min":0.95099999999999996,"15min":0.95099999999999996},"last":0.70199999999999996},{"interface":"front","average":{"1min":0.70899999999999996,"5min":0.70899999999999996,"15min":0.70899999999999996},"min":{"1min":0.41299999999999998,"5min":0.41299999999999998,"15min":0.41299999999999998},"max":{"1min":1.254,"5min":1.254,"15min":1.254},"last":0.75900000000000001}]},{"osd":6,"last update":"Sun Nov 26 07:18:45 2023","interfaces":[{"interface":"back","average":{"1min":0.67200000000000004,"5min":0.67200000000000004,"15min":0.67200000000000004},"min":{"1min":0.45600000000000002,"5min":0.45600000000000002,"15min":0.45600000000000002},"max":{"1min":0.94499999999999995,"5min":0.94499999999999995,"15min":0.94499999999999995},"last":0.80900000000000005},{"interface":"front","average":{"1min":0.72699999999999998,"5min":0.72699999999999998,"15min":0.72699999999999998},"min":{"1min":0.44,"5min":0.44,"15min":0.44},"max":{"1min":1.0649999999999999,"5min":1.0649999999999999,"15min":1.0649999999999999},"last":0.77700000000000002}]},{"osd":7,"last update":"Sun Nov 26 07:18:45 2023","interfaces":[{"interface":"back","average":{"1min":0.80400000000000005,"5min":0.80400000000000005,"15min":0.80400000000000005},"min":{"1min":0.48199999999999998,"5min":0.48199999999999998,"15min":0.48199999999999998},"max":{"1min":1.4079999999999999,"5min":1.4079999999999999,"15min":1.4079999999999999},"last":0.85399999999999998},{"interface":"front","average":{"1min":0.77500000000000002,"5min":0.77500000000000002,"15min":0.77500000000000002},"min":{"1min":0.51600000000000001,"5min":0.51600000000000001,"15min":0.51600000000000001},"max":{"1min":1.0169999999999999,"5min":1.0169999999999999,"15min":1.0169999999999999},"last":0.82499999999999996}]}]},{"osd":4,"up_from":13,"seq":55834574866,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27348,"kb_used_data":328,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344492,"statfs":{"total":96636764160,"available":96608759808,"internally_reserved":0,"allocated":335872,"data_stored":127250,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Nov 26 07:18:46 2023","interfaces":[{"interface":"back","average":{"1min":0.79400000000000004,"5min":0.79400000000000004,"15min":0.79400000000000004},"min":{"1min":0.45600000000000002,"5min":0.45600000000000002,"15min":0.45600000000000002},"max":{"1min":4.1689999999999996,"5min":4.1689999999999996,"15min":4.1689999999999996},"last":0.58599999999999997},{"interface":"front","average":{"1min":1.002,"5min":1.002,"15min":1.002},"min":{"1min":0.47499999999999998,"5min":0.47499999999999998,"15min":0.47499999999999998},"max":{"1min":7.7889999999999997,"5min":7.7889999999999997,"15min":7.7889999999999997},"last":0.54200000000000004}]},{"osd":1,"last update":"Sun Nov 26 07:18:46 2023","interfaces":[{"interface":"back","average":{"1min":0.64700000000000002,"5min":0.64700000000000002,"15min":0.64700000000000002},"min":{"1min":0.52300000000000002,"5min":0.52300000000000002,"15min":0.52300000000000002},"max":{"1min":0.78400000000000003,"5min":0.78400000000000003,"15min":0.78400000000000003},"last":0.63400000000000001},{"interface":"front","average":{"1min":0.81999999999999995,"5min":0.81999999999999995,"15min":0.81999999999999995},"min":{"1min":0.44500000000000001,"5min":0.44500000000000001,"15min":0.44500000000000001},"max":{"1min":4.3140000000000001,"5min":4.3140000000000001,"15min":4.3140000000000001},"last":0.52400000000000002}]},{"osd":2,"last update":"Sun Nov 26 07:18:46 2023","interfaces":[{"interface":"back","average":{"1min":0.80700000000000005,"5min":0.80700000000000005,"15min":0.80700000000000005},"min":{"1min":0.57799999999999996,"5min":0.57799999999999996,"15min":0.57799999999999996},"max":{"1min":2.7850000000000001,"5min":2.7850000000000001,"15min":2.7850000000000001},"last":0.76100000000000001},{"interface":"front","average":{"1min":1.0660000000000001,"5min":1.0660000000000001,"15min":1.0660000000000001},"min":{"1min":0.53400000000000003,"5min":0.53400000000000003,"15min":0.53400000000000003},"max":{"1min":7.6470000000000002,"5min":7.6470000000000002,"15min":7.6470000000000002},"last":0.68899999999999995}]},{"osd":3,"last update":"Sun Nov 26 07:18:46 2023","interfaces":[{"interface":"back","average":{"1min":0.72999999999999998,"5min":0.72999999999999998,"15min":0.72999999999999998},"min":{"1min":0.48599999999999999,"5min":0.48599999999999999,"15min":0.48599999999999999},"max":{"1min":1.591,"5min":1.591,"15min":1.591},"last":0.45700000000000002},{"interface":"front","average":{"1min":0.879,"5min":0.879,"15min":0.879},"min":{"1min":0.56899999999999995,"5min":0.56899999999999995,"15min":0.56899999999999995},"max":{"1min":3.452,"5min":3.452,"15min":3.452},"last":0.80400000000000005}]},{"osd":5,"last update":"Sun Nov 26 07:18:46 2023","interfaces":[{"interface":"back","average":{"1min":0.61099999999999999,"5min":0.61099999999999999,"15min":0.61099999999999999},"min":{"1min":0.43099999999999999,"5min":0.43099999999999999,"15min":0.43099999999999999},"max":{"1min":0.96199999999999997,"5min":0.96199999999999997,"15min":0.96199999999999997},"last":0.73999999999999999},{"interface":"front","average":{"1min":0.64100000000000001,"5min":0.64100000000000001,"15min":0.64100000000000001},"min":{"1min":0.41499999999999998,"5min":0.41499999999999998,"15min":0.41499999999999998},"max":{"1min":1.538,"5min":1.538,"15min":1.538},"last":0.56799999999999995}]},{"osd":6,"last update":"Sun Nov 26 07:18:46 2023","interfaces":[{"interface":"back","average":{"1min":0.79400000000000004,"5min":0.79400000000000004,"15min":0.79400000000000004},"min":{"1min":0.39400000000000002,"5min":0.39400000000000002,"15min":0.39400000000000002},"max":{"1min":2.8330000000000002,"5min":2.8330000000000002,"15min":2.8330000000000002},"last":0.70999999999999996},{"interface":"front","average":{"1min":1.0089999999999999,"5min":1.0089999999999999,"15min":1.0089999999999999},"min":{"1min":0.42899999999999999,"5min":0.42899999999999999,"15min":0.42899999999999999},"max":{"1min":7.6050000000000004,"5min":7.6050000000000004,"15min":7.6050000000000004},"last":0.78500000000000003}]},{"osd":7,"last update":"Sun Nov 26 07:18:46 2023","interfaces":[{"interface":"back","average":{"1min":0.86099999999999999,"5min":0.86099999999999999,"15min":0.86099999999999999},"min":{"1min":0.44700000000000001,"5min":0.44700000000000001,"15min":0.44700000000000001},"max":{"1min":4.3689999999999998,"5min":4.3689999999999998,"15min":4.3689999999999998},"last":0.61299999999999999},{"interface":"front","average":{"1min":0.89200000000000002,"5min":0.89200000000000002,"15min":0.89200000000000002},"min":{"1min":0.505,"5min":0.505,"15min":0.505},"max":{"1min":3.403,"5min":3.403,"15min":3.403},"last":0.83799999999999997}]}]},{"osd":3,"up_from":13,"seq":55834574867,"num_pgs":7,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27352,"kb_used_data":328,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344488,"statfs":{"total":96636764160,"available":96608755712,"internally_reserved":0,"allocated":335872,"data_stored":127250,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1592,"internal_metadata":27654600},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Nov 26 07:18:43 2023","interfaces":[{"interface":"back","average":{"1min":0.379,"5min":0.379,"15min":0.379},"min":{"1min":0.26100000000000001,"5min":0.26100000000000001,"15min":0.26100000000000001},"max":{"1min":0.58599999999999997,"5min":0.58599999999999997,"15min":0.58599999999999997},"last":0.40000000000000002},{"interface":"front","average":{"1min":0.433,"5min":0.433,"15min":0.433},"min":{"1min":0.30199999999999999,"5min":0.30199999999999999,"15min":0.30199999999999999},"max":{"1min":0.63,"5min":0.63,"15min":0.63},"last":0.26900000000000002}]},{"osd":1,"last update":"Sun Nov 26 07:18:43 2023","interfaces":[{"interface":"back","average":{"1min":0.42099999999999999,"5min":0.42099999999999999,"15min":0.42099999999999999},"min":{"1min":0.312,"5min":0.312,"15min":0.312},"max":{"1min":0.752,"5min":0.752,"15min":0.752},"last":0.28799999999999998},{"interface":"front","average":{"1min":0.48399999999999999,"5min":0.48399999999999999,"15min":0.48399999999999999},"min":{"1min":0.38,"5min":0.38,"15min":0.38},"max":{"1min":0.628,"5min":0.628,"15min":0.628},"last":0.35099999999999998}]},{"osd":2,"last update":"Sun Nov 26 07:18:43 2023","interfaces":[{"interface":"back","average":{"1min":0.53300000000000003,"5min":0.53300000000000003,"15min":0.53300000000000003},"min":{"1min":0.40300000000000002,"5min":0.40300000000000002,"15min":0.40300000000000002},"max":{"1min":0.82199999999999995,"5min":0.82199999999999995,"15min":0.82199999999999995},"last":0.38200000000000001},{"interface":"front","average":{"1min":0.505,"5min":0.505,"15min":0.505},"min":{"1min":0.35999999999999999,"5min":0.35999999999999999,"15min":0.35999999999999999},"max":{"1min":0.72199999999999998,"5min":0.72199999999999998,"15min":0.72199999999999998},"last":0.33000000000000002}]},{"osd":4,"last update":"Sun Nov 26 07:18:43 2023","interfaces":[{"interface":"back","average":{"1min":0.70599999999999996,"5min":0.70599999999999996,"15min":0.70599999999999996},"min":{"1min":0.48199999999999998,"5min":0.48199999999999998,"15min":0.48199999999999998},"max":{"1min":0.96199999999999997,"5min":0.96199999999999997,"15min":0.96199999999999997},"last":0.52200000000000002},{"interface":"front","average":{"1min":0.72799999999999998,"5min":0.72799999999999998,"15min":0.72799999999999998},"min":{"1min":0.441,"5min":0.441,"15min":0.441},"max":{"1min":1.171,"5min":1.171,"15min":1.171},"last":0.59699999999999998}]},{"osd":5,"last update":"Sun Nov 26 07:18:43 2023","interfaces":[{"interface":"back","average":{"1min":0.73899999999999999,"5min":0.73899999999999999,"15min":0.73899999999999999},"min":{"1min":0.53700000000000003,"5min":0.53700000000000003,"15min":0.53700000000000003},"max":{"1min":0.94099999999999995,"5min":0.94099999999999995,"15min":0.94099999999999995},"last":0.53800000000000003},{"interface":"front","average":{"1min":0.73799999999999999,"5min":0.73799999999999999,"15min":0.73799999999999999},"min":{"1min":0.504,"5min":0.504,"15min":0.504},"max":{"1min":1.0820000000000001,"5min":1.0820000000000001,"15min":1.0820000000000001},"last":0.61199999999999999}]},{"osd":6,"last update":"Sun Nov 26 07:18:43 2023","interfaces":[{"interface":"back","average":{"1min":0.81399999999999995,"5min":0.81399999999999995,"15min":0.81399999999999995},"min":{"1min":0.55200000000000005,"5min":0.55200000000000005,"15min":0.55200000000000005},"max":{"1min":1.2729999999999999,"5min":1.2729999999999999,"15min":1.2729999999999999},"last":0.64500000000000002},{"interface":"front","average":{"1min":0.79300000000000004,"5min":0.79300000000000004,"15min":0.79300000000000004},"min":{"1min":0.47299999999999998,"5min":0.47299999999999998,"15min":0.47299999999999998},"max":{"1min":1.204,"5min":1.204,"15min":1.204},"last":0.63200000000000001}]},{"osd":7,"last update":"Sun Nov 26 07:18:43 2023","interfaces":[{"interface":"back","average":{"1min":0.80100000000000005,"5min":0.80100000000000005,"15min":0.80100000000000005},"min":{"1min":0.63300000000000001,"5min":0.63300000000000001,"15min":0.63300000000000001},"max":{"1min":1.3600000000000001,"5min":1.3600000000000001,"15min":1.3600000000000001},"last":0.65500000000000003},{"interface":"front","average":{"1min":0.82599999999999996,"5min":0.82599999999999996,"15min":0.82599999999999996},"min":{"1min":0.50600000000000001,"5min":0.50600000000000001,"15min":0.50600000000000001},"max":{"1min":1.617,"5min":1.617,"15min":1.617},"last":0.67000000000000004}]}]},{"osd":2,"up_from":13,"seq":55834574867,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27360,"kb_used_data":340,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344480,"statfs":{"total":96636764160,"available":96608747520,"internally_reserved":0,"allocated":348160,"data_stored":133517,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Nov 26 07:18:43 2023","interfaces":[{"interface":"back","average":{"1min":0.71399999999999997,"5min":0.71399999999999997,"15min":0.71399999999999997},"min":{"1min":0.28699999999999998,"5min":0.28699999999999998,"15min":0.28699999999999998},"max":{"1min":4.7030000000000003,"5min":4.7030000000000003,"15min":4.7030000000000003},"last":0.32500000000000001},{"interface":"front","average":{"1min":0.56200000000000006,"5min":0.56200000000000006,"15min":0.56200000000000006},"min":{"1min":0.32300000000000001,"5min":0.32300000000000001,"15min":0.32300000000000001},"max":{"1min":2.4340000000000002,"5min":2.4340000000000002,"15min":2.4340000000000002},"last":0.49099999999999999}]},{"osd":1,"last update":"Sun Nov 26 07:18:43 2023","interfaces":[{"interface":"back","average":{"1min":0.86399999999999999,"5min":0.86399999999999999,"15min":0.86399999999999999},"min":{"1min":0.34599999999999997,"5min":0.34599999999999997,"15min":0.34599999999999997},"max":{"1min":4.2690000000000001,"5min":4.2690000000000001,"15min":4.2690000000000001},"last":0.42499999999999999},{"interface":"front","average":{"1min":0.74199999999999999,"5min":0.74199999999999999,"15min":0.74199999999999999},"min":{"1min":0.31900000000000001,"5min":0.31900000000000001,"15min":0.31900000000000001},"max":{"1min":3.9940000000000002,"5min":3.9940000000000002,"15min":3.9940000000000002},"last":0.35099999999999998}]},{"osd":3,"last update":"Sun Nov 26 07:18:43 2023","interfaces":[{"interface":"back","average":{"1min":0.64300000000000002,"5min":0.64300000000000002,"15min":0.64300000000000002},"min":{"1min":0.34399999999999997,"5min":0.34399999999999997,"15min":0.34399999999999997},"max":{"1min":4.0300000000000002,"5min":4.0300000000000002,"15min":4.0300000000000002},"last":0.38700000000000001},{"interface":"front","average":{"1min":0.91400000000000003,"5min":0.91400000000000003,"15min":0.91400000000000003},"min":{"1min":0.41699999999999998,"5min":0.41699999999999998,"15min":0.41699999999999998},"max":{"1min":4.5810000000000004,"5min":4.5810000000000004,"15min":4.5810000000000004},"last":0.46500000000000002}]},{"osd":4,"last update":"Sun Nov 26 07:18:43 2023","interfaces":[{"interface":"back","average":{"1min":0.98499999999999999,"5min":0.98499999999999999,"15min":0.98499999999999999},"min":{"1min":0.55100000000000005,"5min":0.55100000000000005,"15min":0.55100000000000005},"max":{"1min":4.4269999999999996,"5min":4.4269999999999996,"15min":4.4269999999999996},"last":1.01},{"interface":"front","average":{"1min":1.0149999999999999,"5min":1.0149999999999999,"15min":1.0149999999999999},"min":{"1min":0.60299999999999998,"5min":0.60299999999999998,"15min":0.60299999999999998},"max":{"1min":4.085,"5min":4.085,"15min":4.085},"last":1.0600000000000001}]},{"osd":5,"last update":"Sun Nov 26 07:18:43 2023","interfaces":[{"interface":"back","average":{"1min":0.874,"5min":0.874,"15min":0.874},"min":{"1min":0.52900000000000003,"5min":0.52900000000000003,"15min":0.52900000000000003},"max":{"1min":3.9590000000000001,"5min":3.9590000000000001,"15min":3.9590000000000001},"last":0.81000000000000005},{"interface":"front","average":{"1min":0.98799999999999999,"5min":0.98799999999999999,"15min":0.98799999999999999},"min":{"1min":0.58799999999999997,"5min":0.58799999999999997,"15min":0.58799999999999997},"max":{"1min":4.3849999999999998,"5min":4.3849999999999998,"15min":4.3849999999999998},"last":0.86299999999999999}]},{"osd":6,"last update":"Sun Nov 26 07:18:43 2023","interfaces":[{"interface":"back","average":{"1min":0.996,"5min":0.996,"15min":0.996},"min":{"1min":0.56599999999999995,"5min":0.56599999999999995,"15min":0.56599999999999995},"max":{"1min":4.0449999999999999,"5min":4.0449999999999999,"15min":4.0449999999999999},"last":0.91000000000000003},{"interface":"front","average":{"1min":0.91800000000000004,"5min":0.91800000000000004,"15min":0.91800000000000004},"min":{"1min":0.57099999999999995,"5min":0.57099999999999995,"15min":0.57099999999999995},"max":{"1min":3.992,"5min":3.992,"15min":3.992},"last":1.073}]},{"osd":7,"last update":"Sun Nov 26 07:18:43 2023","interfaces":[{"interface":"back","average":{"1min":1.0349999999999999,"5min":1.0349999999999999,"15min":1.0349999999999999},"min":{"1min":0.63100000000000001,"5min":0.63100000000000001,"15min":0.63100000000000001},"max":{"1min":4.1219999999999999,"5min":4.1219999999999999,"15min":4.1219999999999999},"last":0.69699999999999995},{"interface":"front","average":{"1min":0.93799999999999994,"5min":0.93799999999999994,"15min":0.93799999999999994},"min":{"1min":0.57699999999999996,"5min":0.57699999999999996,"15min":0.57699999999999996},"max":{"1min":3.9159999999999999,"5min":3.9159999999999999,"15min":3.9159999999999999},"last":0.95199999999999996}]}]},{"osd":1,"up_from":13,"seq":55834574866,"num_pgs":5,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27352,"kb_used_data":328,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344488,"statfs":{"total":96636764160,"available":96608755712,"internally_reserved":0,"allocated":335872,"data_stored":127250,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Nov 26 07:18:41 2023","interfaces":[{"interface":"back","average":{"1min":0.41699999999999998,"5min":0.41699999999999998,"15min":0.41699999999999998},"min":{"1min":0.26300000000000001,"5min":0.26300000000000001,"15min":0.26300000000000001},"max":{"1min":0.61199999999999999,"5min":0.61199999999999999,"15min":0.61199999999999999},"last":0.29499999999999998},{"interface":"front","average":{"1min":0.48599999999999999,"5min":0.48599999999999999,"15min":0.48599999999999999},"min":{"1min":0.30499999999999999,"5min":0.30499999999999999,"15min":0.30499999999999999},"max":{"1min":0.78700000000000003,"5min":0.78700000000000003,"15min":0.78700000000000003},"last":0.34699999999999998}]},{"osd":2,"last update":"Sun Nov 26 07:18:41 2023","interfaces":[{"interface":"back","average":{"1min":0.45500000000000002,"5min":0.45500000000000002,"15min":0.45500000000000002},"min":{"1min":0.33400000000000002,"5min":0.33400000000000002,"15min":0.33400000000000002},"max":{"1min":0.67600000000000005,"5min":0.67600000000000005,"15min":0.67600000000000005},"last":0.39200000000000002},{"interface":"front","average":{"1min":0.50800000000000001,"5min":0.50800000000000001,"15min":0.50800000000000001},"min":{"1min":0.32300000000000001,"5min":0.32300000000000001,"15min":0.32300000000000001},"max":{"1min":0.66500000000000004,"5min":0.66500000000000004,"15min":0.66500000000000004},"last":0.40699999999999997}]},{"osd":3,"last update":"Sun Nov 26 07:18:41 2023","interfaces":[{"interface":"back","average":{"1min":0.53000000000000003,"5min":0.53000000000000003,"15min":0.53000000000000003},"min":{"1min":0.38400000000000001,"5min":0.38400000000000001,"15min":0.38400000000000001},"max":{"1min":0.73699999999999999,"5min":0.73699999999999999,"15min":0.73699999999999999},"last":0.48499999999999999},{"interface":"front","average":{"1min":0.497,"5min":0.497,"15min":0.497},"min":{"1min":0.36199999999999999,"5min":0.36199999999999999,"15min":0.36199999999999999},"max":{"1min":0.80700000000000005,"5min":0.80700000000000005,"15min":0.80700000000000005},"last":0.432}]},{"osd":4,"last update":"Sun Nov 26 07:18:41 2023","interfaces":[{"interface":"back","average":{"1min":0.69399999999999995,"5min":0.69399999999999995,"15min":0.69399999999999995},"min":{"1min":0.48599999999999999,"5min":0.48599999999999999,"15min":0.48599999999999999},"max":{"1min":1.0229999999999999,"5min":1.0229999999999999,"15min":1.0229999999999999},"last":0.54400000000000004},{"interface":"front","average":{"1min":0.68000000000000005,"5min":0.68000000000000005,"15min":0.68000000000000005},"min":{"1min":0.49199999999999999,"5min":0.49199999999999999,"15min":0.49199999999999999},"max":{"1min":1.0629999999999999,"5min":1.0629999999999999,"15min":1.0629999999999999},"last":0.58399999999999996}]},{"osd":5,"last update":"Sun Nov 26 07:18:41 2023","interfaces":[{"interface":"back","average":{"1min":0.71899999999999997,"5min":0.71899999999999997,"15min":0.71899999999999997},"min":{"1min":0.49299999999999999,"5min":0.49299999999999999,"15min":0.49299999999999999},"max":{"1min":1.0880000000000001,"5min":1.0880000000000001,"15min":1.0880000000000001},"last":0.60399999999999998},{"interface":"front","average":{"1min":0.71699999999999997,"5min":0.71699999999999997,"15min":0.71699999999999997},"min":{"1min":0.53500000000000003,"5min":0.53500000000000003,"15min":0.53500000000000003},"max":{"1min":1.014,"5min":1.014,"15min":1.014},"last":0.68500000000000005}]},{"osd":6,"last update":"Sun Nov 26 07:18:41 2023","interfaces":[{"interface":"back","average":{"1min":0.70399999999999996,"5min":0.70399999999999996,"15min":0.70399999999999996},"min":{"1min":0.50600000000000001,"5min":0.50600000000000001,"15min":0.50600000000000001},"max":{"1min":0.97599999999999998,"5min":0.97599999999999998,"15min":0.97599999999999998},"last":0.61199999999999999},{"interface":"front","average":{"1min":0.70099999999999996,"5min":0.70099999999999996,"15min":0.70099999999999996},"min":{"1min":0.55000000000000004,"5min":0.55000000000000004,"15min":0.55000000000000004},"max":{"1min":1.1339999999999999,"5min":1.1339999999999999,"15min":1.1339999999999999},"last":0.56100000000000005}]},{"osd":7,"last update":"Sun Nov 26 07:18:41 2023","interfaces":[{"interface":"back","average":{"1min":0.71499999999999997,"5min":0.71499999999999997,"15min":0.71499999999999997},"min":{"1min":0.55100000000000005,"5min":0.55100000000000005,"15min":0.55100000000000005},"max":{"1min":0.92900000000000005,"5min":0.92900000000000005,"15min":0.92900000000000005},"last":0.59299999999999997},{"interface":"front","average":{"1min":0.753,"5min":0.753,"15min":0.753},"min":{"1min":0.56100000000000005,"5min":0.56100000000000005,"15min":0.56100000000000005},"max":{"1min":1.024,"5min":1.024,"15min":1.024},"last":0.71499999999999997}]}]},{"osd":0,"up_from":13,"seq":55834574866,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27944,"kb_used_data":920,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343896,"statfs":{"total":96636764160,"available":96608149504,"internally_reserved":0,"allocated":942080,"data_stored":723885,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Sun Nov 26 07:18:41 2023","interfaces":[{"interface":"back","average":{"1min":0.43099999999999999,"5min":0.43099999999999999,"15min":0.43099999999999999},"min":{"1min":0.28000000000000003,"5min":0.28000000000000003,"15min":0.28000000000000003},"max":{"1min":0.63100000000000001,"5min":0.63100000000000001,"15min":0.63100000000000001},"last":0.38200000000000001},{"interface":"front","average":{"1min":0.48299999999999998,"5min":0.48299999999999998,"15min":0.48299999999999998},"min":{"1min":0.29899999999999999,"5min":0.29899999999999999,"15min":0.29899999999999999},"max":{"1min":1.133,"5min":1.133,"15min":1.133},"last":0.30199999999999999}]},{"osd":2,"last update":"Sun Nov 26 07:18:41 2023","interfaces":[{"interface":"back","average":{"1min":0.53700000000000003,"5min":0.53700000000000003,"15min":0.53700000000000003},"min":{"1min":0.40899999999999997,"5min":0.40899999999999997,"15min":0.40899999999999997},"max":{"1min":0.67400000000000004,"5min":0.67400000000000004,"15min":0.67400000000000004},"last":0.48099999999999998},{"interface":"front","average":{"1min":0.56399999999999995,"5min":0.56399999999999995,"15min":0.56399999999999995},"min":{"1min":0.36399999999999999,"5min":0.36399999999999999,"15min":0.36399999999999999},"max":{"1min":0.72499999999999998,"5min":0.72499999999999998,"15min":0.72499999999999998},"last":0.45300000000000001}]},{"osd":3,"last update":"Sun Nov 26 07:18:41 2023","interfaces":[{"interface":"back","average":{"1min":0.53100000000000003,"5min":0.53100000000000003,"15min":0.53100000000000003},"min":{"1min":0.35799999999999998,"5min":0.35799999999999998,"15min":0.35799999999999998},"max":{"1min":0.97399999999999998,"5min":0.97399999999999998,"15min":0.97399999999999998},"last":0.436},{"interface":"front","average":{"1min":0.56200000000000006,"5min":0.56200000000000006,"15min":0.56200000000000006},"min":{"1min":0.40600000000000003,"5min":0.40600000000000003,"15min":0.40600000000000003},"max":{"1min":1.2509999999999999,"5min":1.2509999999999999,"15min":1.2509999999999999},"last":0.498}]},{"osd":4,"last update":"Sun Nov 26 07:18:41 2023","interfaces":[{"interface":"back","average":{"1min":0.75900000000000001,"5min":0.75900000000000001,"15min":0.75900000000000001},"min":{"1min":0.60999999999999999,"5min":0.60999999999999999,"15min":0.60999999999999999},"max":{"1min":1.2010000000000001,"5min":1.2010000000000001,"15min":1.2010000000000001},"last":0.69599999999999995},{"interface":"front","average":{"1min":0.78900000000000003,"5min":0.78900000000000003,"15min":0.78900000000000003},"min":{"1min":0.59999999999999998,"5min":0.59999999999999998,"15min":0.59999999999999998},"max":{"1min":0.99399999999999999,"5min":0.99399999999999999,"15min":0.99399999999999999},"last":0.71899999999999997}]},{"osd":5,"last update":"Sun Nov 26 07:18:41 2023","interfaces":[{"interface":"back","average":{"1min":0.76000000000000001,"5min":0.76000000000000001,"15min":0.76000000000000001},"min":{"1min":0.53000000000000003,"5min":0.53000000000000003,"15min":0.53000000000000003},"max":{"1min":1.1579999999999999,"5min":1.1579999999999999,"15min":1.1579999999999999},"last":0.73599999999999999},{"interface":"front","average":{"1min":0.79700000000000004,"5min":0.79700000000000004,"15min":0.79700000000000004},"min":{"1min":0.59699999999999998,"5min":0.59699999999999998,"15min":0.59699999999999998},"max":{"1min":1.0409999999999999,"5min":1.0409999999999999,"15min":1.0409999999999999},"last":0.67000000000000004}]},{"osd":6,"last update":"Sun Nov 26 07:18:41 2023","interfaces":[{"interface":"back","average":{"1min":0.76400000000000001,"5min":0.76400000000000001,"15min":0.76400000000000001},"min":{"1min":0.56599999999999995,"5min":0.56599999999999995,"15min":0.56599999999999995},"max":{"1min":1.3029999999999999,"5min":1.3029999999999999,"15min":1.3029999999999999},"last":0.55300000000000005},{"interface":"front","average":{"1min":0.76900000000000002,"5min":0.76900000000000002,"15min":0.76900000000000002},"min":{"1min":0.58599999999999997,"5min":0.58599999999999997,"15min":0.58599999999999997},"max":{"1min":1.2210000000000001,"5min":1.2210000000000001,"15min":1.2210000000000001},"last":0.76000000000000001}]},{"osd":7,"last update":"Sun Nov 26 07:18:41 2023","interfaces":[{"interface":"back","average":{"1min":0.81100000000000005,"5min":0.81100000000000005,"15min":0.81100000000000005},"min":{"1min":0.59999999999999998,"5min":0.59999999999999998,"15min":0.59999999999999998},"max":{"1min":1.177,"5min":1.177,"15min":1.177},"last":0.624},{"interface":"front","average":{"1min":0.79900000000000004,"5min":0.79900000000000004,"15min":0.79900000000000004},"min":{"1min":0.60899999999999999,"5min":0.60899999999999999,"15min":0.60899999999999999},"max":{"1min":1.2390000000000001,"5min":1.2390000000000001,"15min":1.2390000000000001},"last":0.65100000000000002}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-11-26T07:18:57.486 INFO:tasks.cephfs.filesystem.ceph_manager:PG 5.3f is not active+clean 2023-11-26T07:18:57.486 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '5.3f', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 31, 'state': 'creating+peering', 'last_fresh': '2023-11-26T07:18:51.645063+0000', 'last_change': '2023-11-26T07:18:51.645063+0000', 'last_active': '2023-11-26T07:18:51.625829+0000', 'last_peered': '2023-11-26T07:18:51.625829+0000', 'last_clean': '2023-11-26T07:18:51.625829+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-26T07:18:51.645063+0000', 'last_undegraded': '2023-11-26T07:18:51.645063+0000', 'last_fullsized': '2023-11-26T07:18:51.645063+0000', 'mapping_epoch': 31, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 31, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-26T07:18:51.625829+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-26T07:18:51.625829+0000', 'last_clean_scrub_stamp': '2023-11-26T07:18:51.625829+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [6, 5], 'acting': [6, 5], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 6, 'acting_primary': 6, 'purged_snaps': []} 2023-11-26T07:18:57.486 INFO:tasks.cephfs.filesystem.ceph_manager:PG 5.3e is not active+clean 2023-11-26T07:18:57.486 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '5.3e', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 31, 'state': 'creating+peering', 'last_fresh': '2023-11-26T07:18:51.643947+0000', 'last_change': '2023-11-26T07:18:51.643947+0000', 'last_active': '2023-11-26T07:18:51.625829+0000', 'last_peered': '2023-11-26T07:18:51.625829+0000', 'last_clean': '2023-11-26T07:18:51.625829+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-26T07:18:51.643947+0000', 'last_undegraded': '2023-11-26T07:18:51.643947+0000', 'last_fullsized': '2023-11-26T07:18:51.643947+0000', 'mapping_epoch': 31, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 31, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-26T07:18:51.625829+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-26T07:18:51.625829+0000', 'last_clean_scrub_stamp': '2023-11-26T07:18:51.625829+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 0], 'acting': [3, 0], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2023-11-26T07:18:57.486 INFO:tasks.cephfs.filesystem.ceph_manager:PG 5.3d is not active+clean 2023-11-26T07:18:57.486 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '5.3d', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 31, 'state': 'creating+peering', 'last_fresh': '2023-11-26T07:18:51.634980+0000', 'last_change': '2023-11-26T07:18:51.634980+0000', 'last_active': '2023-11-26T07:18:51.625829+0000', 'last_peered': '2023-11-26T07:18:51.625829+0000', 'last_clean': '2023-11-26T07:18:51.625829+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-26T07:18:51.634980+0000', 'last_undegraded': '2023-11-26T07:18:51.634980+0000', 'last_fullsized': '2023-11-26T07:18:51.634980+0000', 'mapping_epoch': 31, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 31, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-26T07:18:51.625829+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-26T07:18:51.625829+0000', 'last_clean_scrub_stamp': '2023-11-26T07:18:51.625829+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [5, 1], 'acting': [5, 1], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 5, 'acting_primary': 5, 'purged_snaps': []} 2023-11-26T07:18:57.486 INFO:tasks.cephfs.filesystem.ceph_manager:PG 5.3c is not active+clean 2023-11-26T07:18:57.487 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '5.3c', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 31, 'state': 'creating+peering', 'last_fresh': '2023-11-26T07:18:51.634028+0000', 'last_change': '2023-11-26T07:18:51.634028+0000', 'last_active': '2023-11-26T07:18:51.625829+0000', 'last_peered': '2023-11-26T07:18:51.625829+0000', 'last_clean': '2023-11-26T07:18:51.625829+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-26T07:18:51.634028+0000', 'last_undegraded': '2023-11-26T07:18:51.634028+0000', 'last_fullsized': '2023-11-26T07:18:51.634028+0000', 'mapping_epoch': 31, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 31, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-26T07:18:51.625829+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-26T07:18:51.625829+0000', 'last_clean_scrub_stamp': '2023-11-26T07:18:51.625829+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 4], 'acting': [1, 4], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 1, 'purged_snaps': []} 2023-11-26T07:18:57.487 INFO:tasks.cephfs.filesystem.ceph_manager:PG 5.3b is not active+clean 2023-11-26T07:18:57.487 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '5.3b', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 31, 'state': 'creating+peering', 'last_fresh': '2023-11-26T07:18:51.651017+0000', 'last_change': '2023-11-26T07:18:51.651017+0000', 'last_active': '2023-11-26T07:18:51.625829+0000', 'last_peered': '2023-11-26T07:18:51.625829+0000', 'last_clean': '2023-11-26T07:18:51.625829+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-26T07:18:51.651017+0000', 'last_undegraded': '2023-11-26T07:18:51.651017+0000', 'last_fullsized': '2023-11-26T07:18:51.651017+0000', 'mapping_epoch': 31, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 31, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-26T07:18:51.625829+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-26T07:18:51.625829+0000', 'last_clean_scrub_stamp': '2023-11-26T07:18:51.625829+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 3], 'acting': [2, 3], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2023-11-26T07:18:57.487 INFO:tasks.cephfs.filesystem.ceph_manager:PG 5.3a is not active+clean 2023-11-26T07:18:57.487 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '5.3a', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 31, 'state': 'creating+peering', 'last_fresh': '2023-11-26T07:18:51.644135+0000', 'last_change': '2023-11-26T07:18:51.644135+0000', 'last_active': '2023-11-26T07:18:51.625829+0000', 'last_peered': '2023-11-26T07:18:51.625829+0000', 'last_clean': '2023-11-26T07:18:51.625829+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-26T07:18:51.644135+0000', 'last_undegraded': '2023-11-26T07:18:51.644135+0000', 'last_fullsized': '2023-11-26T07:18:51.644135+0000', 'mapping_epoch': 31, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 31, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-26T07:18:51.625829+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-26T07:18:51.625829+0000', 'last_clean_scrub_stamp': '2023-11-26T07:18:51.625829+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 5], 'acting': [3, 5], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2023-11-26T07:18:57.487 INFO:tasks.cephfs.filesystem.ceph_manager:PG 5.39 is not active+clean 2023-11-26T07:18:57.487 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '5.39', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 31, 'state': 'creating+peering', 'last_fresh': '2023-11-26T07:18:51.641562+0000', 'last_change': '2023-11-26T07:18:51.641562+0000', 'last_active': '2023-11-26T07:18:51.625829+0000', 'last_peered': '2023-11-26T07:18:51.625829+0000', 'last_clean': '2023-11-26T07:18:51.625829+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-26T07:18:51.641562+0000', 'last_undegraded': '2023-11-26T07:18:51.641562+0000', 'last_fullsized': '2023-11-26T07:18:51.641562+0000', 'mapping_epoch': 31, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 31, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-26T07:18:51.625829+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-26T07:18:51.625829+0000', 'last_clean_scrub_stamp': '2023-11-26T07:18:51.625829+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [7, 6], 'acting': [7, 6], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 7, 'acting_primary': 7, 'purged_snaps': []} 2023-11-26T07:18:57.487 INFO:tasks.cephfs.filesystem.ceph_manager:PG 5.38 is not active+clean 2023-11-26T07:18:57.487 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '5.38', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 31, 'state': 'creating+peering', 'last_fresh': '2023-11-26T07:18:51.646503+0000', 'last_change': '2023-11-26T07:18:51.646503+0000', 'last_active': '2023-11-26T07:18:51.625829+0000', 'last_peered': '2023-11-26T07:18:51.625829+0000', 'last_clean': '2023-11-26T07:18:51.625829+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-26T07:18:51.646503+0000', 'last_undegraded': '2023-11-26T07:18:51.646503+0000', 'last_fullsized': '2023-11-26T07:18:51.646503+0000', 'mapping_epoch': 31, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 31, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-26T07:18:51.625829+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-26T07:18:51.625829+0000', 'last_clean_scrub_stamp': '2023-11-26T07:18:51.625829+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [7, 2], 'acting': [7, 2], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 7, 'acting_primary': 7, 'purged_snaps': []} 2023-11-26T07:18:57.487 INFO:tasks.cephfs.filesystem.ceph_manager:PG 5.37 is not active+clean 2023-11-26T07:18:57.488 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '5.37', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 31, 'state': 'creating+peering', 'last_fresh': '2023-11-26T07:18:51.636576+0000', 'last_change': '2023-11-26T07:18:51.636576+0000', 'last_active': '2023-11-26T07:18:51.625829+0000', 'last_peered': '2023-11-26T07:18:51.625829+0000', 'last_clean': '2023-11-26T07:18:51.625829+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-26T07:18:51.636576+0000', 'last_undegraded': '2023-11-26T07:18:51.636576+0000', 'last_fullsized': '2023-11-26T07:18:51.636576+0000', 'mapping_epoch': 31, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 31, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-26T07:18:51.625829+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-26T07:18:51.625829+0000', 'last_clean_scrub_stamp': '2023-11-26T07:18:51.625829+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [4, 1], 'acting': [4, 1], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 4, 'acting_primary': 4, 'purged_snaps': []} 2023-11-26T07:18:57.488 INFO:tasks.cephfs.filesystem.ceph_manager:PG 5.36 is not active+clean 2023-11-26T07:18:57.488 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '5.36', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 31, 'state': 'creating+peering', 'last_fresh': '2023-11-26T07:18:51.633151+0000', 'last_change': '2023-11-26T07:18:51.633151+0000', 'last_active': '2023-11-26T07:18:51.625829+0000', 'last_peered': '2023-11-26T07:18:51.625829+0000', 'last_clean': '2023-11-26T07:18:51.625829+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-26T07:18:51.633151+0000', 'last_undegraded': '2023-11-26T07:18:51.633151+0000', 'last_fullsized': '2023-11-26T07:18:51.633151+0000', 'mapping_epoch': 31, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 31, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-26T07:18:51.625829+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-26T07:18:51.625829+0000', 'last_clean_scrub_stamp': '2023-11-26T07:18:51.625829+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 3], 'acting': [1, 3], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 1, 'purged_snaps': []} 2023-11-26T07:18:57.488 INFO:tasks.cephfs.filesystem.ceph_manager:PG 5.35 is not active+clean 2023-11-26T07:18:57.488 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '5.35', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 31, 'state': 'creating+peering', 'last_fresh': '2023-11-26T07:18:51.640729+0000', 'last_change': '2023-11-26T07:18:51.640729+0000', 'last_active': '2023-11-26T07:18:51.625829+0000', 'last_peered': '2023-11-26T07:18:51.625829+0000', 'last_clean': '2023-11-26T07:18:51.625829+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-26T07:18:51.640729+0000', 'last_undegraded': '2023-11-26T07:18:51.640729+0000', 'last_fullsized': '2023-11-26T07:18:51.640729+0000', 'mapping_epoch': 31, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 31, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-26T07:18:51.625829+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-26T07:18:51.625829+0000', 'last_clean_scrub_stamp': '2023-11-26T07:18:51.625829+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [7, 1], 'acting': [7, 1], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 7, 'acting_primary': 7, 'purged_snaps': []} 2023-11-26T07:18:57.488 INFO:tasks.cephfs.filesystem.ceph_manager:PG 5.34 is not active+clean 2023-11-26T07:18:57.488 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '5.34', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 31, 'state': 'creating+peering', 'last_fresh': '2023-11-26T07:18:51.641193+0000', 'last_change': '2023-11-26T07:18:51.641193+0000', 'last_active': '2023-11-26T07:18:51.625829+0000', 'last_peered': '2023-11-26T07:18:51.625829+0000', 'last_clean': '2023-11-26T07:18:51.625829+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-26T07:18:51.641193+0000', 'last_undegraded': '2023-11-26T07:18:51.641193+0000', 'last_fullsized': '2023-11-26T07:18:51.641193+0000', 'mapping_epoch': 31, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 31, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-26T07:18:51.625829+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-26T07:18:51.625829+0000', 'last_clean_scrub_stamp': '2023-11-26T07:18:51.625829+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [5, 3], 'acting': [5, 3], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 5, 'acting_primary': 5, 'purged_snaps': []} 2023-11-26T07:18:57.488 INFO:tasks.cephfs.filesystem.ceph_manager:PG 5.33 is not active+clean 2023-11-26T07:18:57.488 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '5.33', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 31, 'state': 'creating+peering', 'last_fresh': '2023-11-26T07:18:51.649682+0000', 'last_change': '2023-11-26T07:18:51.649682+0000', 'last_active': '2023-11-26T07:18:51.625829+0000', 'last_peered': '2023-11-26T07:18:51.625829+0000', 'last_clean': '2023-11-26T07:18:51.625829+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-26T07:18:51.649682+0000', 'last_undegraded': '2023-11-26T07:18:51.649682+0000', 'last_fullsized': '2023-11-26T07:18:51.649682+0000', 'mapping_epoch': 31, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 31, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-26T07:18:51.625829+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-26T07:18:51.625829+0000', 'last_clean_scrub_stamp': '2023-11-26T07:18:51.625829+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 1], 'acting': [2, 1], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2023-11-26T07:18:57.488 INFO:tasks.cephfs.filesystem.ceph_manager:PG 5.32 is not active+clean 2023-11-26T07:18:57.489 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '5.32', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 31, 'state': 'creating+peering', 'last_fresh': '2023-11-26T07:18:51.643138+0000', 'last_change': '2023-11-26T07:18:51.643138+0000', 'last_active': '2023-11-26T07:18:51.625829+0000', 'last_peered': '2023-11-26T07:18:51.625829+0000', 'last_clean': '2023-11-26T07:18:51.625829+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-26T07:18:51.643138+0000', 'last_undegraded': '2023-11-26T07:18:51.643138+0000', 'last_fullsized': '2023-11-26T07:18:51.643138+0000', 'mapping_epoch': 31, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 31, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-26T07:18:51.625829+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-26T07:18:51.625829+0000', 'last_clean_scrub_stamp': '2023-11-26T07:18:51.625829+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 2], 'acting': [3, 2], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2023-11-26T07:18:57.489 INFO:tasks.cephfs.filesystem.ceph_manager:PG 5.31 is not active+clean 2023-11-26T07:18:57.489 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '5.31', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 31, 'state': 'creating+peering', 'last_fresh': '2023-11-26T07:18:51.634598+0000', 'last_change': '2023-11-26T07:18:51.634598+0000', 'last_active': '2023-11-26T07:18:51.625829+0000', 'last_peered': '2023-11-26T07:18:51.625829+0000', 'last_clean': '2023-11-26T07:18:51.625829+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-26T07:18:51.634598+0000', 'last_undegraded': '2023-11-26T07:18:51.634598+0000', 'last_fullsized': '2023-11-26T07:18:51.634598+0000', 'mapping_epoch': 31, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 31, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-26T07:18:51.625829+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-26T07:18:51.625829+0000', 'last_clean_scrub_stamp': '2023-11-26T07:18:51.625829+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [5, 6], 'acting': [5, 6], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 5, 'acting_primary': 5, 'purged_snaps': []} 2023-11-26T07:18:57.489 INFO:tasks.cephfs.filesystem.ceph_manager:PG 5.30 is not active+clean 2023-11-26T07:18:57.489 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '5.30', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 31, 'state': 'creating+peering', 'last_fresh': '2023-11-26T07:18:51.640197+0000', 'last_change': '2023-11-26T07:18:51.640197+0000', 'last_active': '2023-11-26T07:18:51.625829+0000', 'last_peered': '2023-11-26T07:18:51.625829+0000', 'last_clean': '2023-11-26T07:18:51.625829+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-26T07:18:51.640197+0000', 'last_undegraded': '2023-11-26T07:18:51.640197+0000', 'last_fullsized': '2023-11-26T07:18:51.640197+0000', 'mapping_epoch': 31, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 31, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-26T07:18:51.625829+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-26T07:18:51.625829+0000', 'last_clean_scrub_stamp': '2023-11-26T07:18:51.625829+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [7, 3], 'acting': [7, 3], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 7, 'acting_primary': 7, 'purged_snaps': []} 2023-11-26T07:18:57.489 INFO:tasks.cephfs.filesystem.ceph_manager:PG 5.2f is not active+clean 2023-11-26T07:18:57.489 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '5.2f', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 31, 'state': 'creating+peering', 'last_fresh': '2023-11-26T07:18:51.641801+0000', 'last_change': '2023-11-26T07:18:51.641801+0000', 'last_active': '2023-11-26T07:18:51.625829+0000', 'last_peered': '2023-11-26T07:18:51.625829+0000', 'last_clean': '2023-11-26T07:18:51.625829+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-26T07:18:51.641801+0000', 'last_undegraded': '2023-11-26T07:18:51.641801+0000', 'last_fullsized': '2023-11-26T07:18:51.641801+0000', 'mapping_epoch': 31, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 31, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-26T07:18:51.625829+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-26T07:18:51.625829+0000', 'last_clean_scrub_stamp': '2023-11-26T07:18:51.625829+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [7, 6], 'acting': [7, 6], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 7, 'acting_primary': 7, 'purged_snaps': []} 2023-11-26T07:18:57.489 INFO:tasks.cephfs.filesystem.ceph_manager:PG 5.2e is not active+clean 2023-11-26T07:18:57.489 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '5.2e', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 31, 'state': 'creating+peering', 'last_fresh': '2023-11-26T07:18:51.643556+0000', 'last_change': '2023-11-26T07:18:51.643556+0000', 'last_active': '2023-11-26T07:18:51.625829+0000', 'last_peered': '2023-11-26T07:18:51.625829+0000', 'last_clean': '2023-11-26T07:18:51.625829+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-26T07:18:51.643556+0000', 'last_undegraded': '2023-11-26T07:18:51.643556+0000', 'last_fullsized': '2023-11-26T07:18:51.643556+0000', 'mapping_epoch': 31, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 31, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-26T07:18:51.625829+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-26T07:18:51.625829+0000', 'last_clean_scrub_stamp': '2023-11-26T07:18:51.625829+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 1], 'acting': [3, 1], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2023-11-26T07:18:57.489 INFO:tasks.cephfs.filesystem.ceph_manager:PG 5.2d is not active+clean 2023-11-26T07:18:57.489 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '5.2d', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 31, 'state': 'creating+peering', 'last_fresh': '2023-11-26T07:18:51.642785+0000', 'last_change': '2023-11-26T07:18:51.642785+0000', 'last_active': '2023-11-26T07:18:51.625829+0000', 'last_peered': '2023-11-26T07:18:51.625829+0000', 'last_clean': '2023-11-26T07:18:51.625829+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-26T07:18:51.642785+0000', 'last_undegraded': '2023-11-26T07:18:51.642785+0000', 'last_fullsized': '2023-11-26T07:18:51.642785+0000', 'mapping_epoch': 31, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 31, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-26T07:18:51.625829+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-26T07:18:51.625829+0000', 'last_clean_scrub_stamp': '2023-11-26T07:18:51.625829+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 1], 'acting': [3, 1], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2023-11-26T07:18:57.490 INFO:tasks.cephfs.filesystem.ceph_manager:PG 5.2c is not active+clean 2023-11-26T07:18:57.490 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '5.2c', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 31, 'state': 'creating+peering', 'last_fresh': '2023-11-26T07:18:51.651641+0000', 'last_change': '2023-11-26T07:18:51.651641+0000', 'last_active': '2023-11-26T07:18:51.625829+0000', 'last_peered': '2023-11-26T07:18:51.625829+0000', 'last_clean': '2023-11-26T07:18:51.625829+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-26T07:18:51.651641+0000', 'last_undegraded': '2023-11-26T07:18:51.651641+0000', 'last_fullsized': '2023-11-26T07:18:51.651641+0000', 'mapping_epoch': 31, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 31, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-26T07:18:51.625829+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-26T07:18:51.625829+0000', 'last_clean_scrub_stamp': '2023-11-26T07:18:51.625829+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 5], 'acting': [2, 5], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2023-11-26T07:18:57.490 INFO:tasks.cephfs.filesystem.ceph_manager:PG 5.2b is not active+clean 2023-11-26T07:18:57.490 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '5.2b', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 31, 'state': 'creating+peering', 'last_fresh': '2023-11-26T07:18:51.637758+0000', 'last_change': '2023-11-26T07:18:51.637758+0000', 'last_active': '2023-11-26T07:18:51.625829+0000', 'last_peered': '2023-11-26T07:18:51.625829+0000', 'last_clean': '2023-11-26T07:18:51.625829+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-26T07:18:51.637758+0000', 'last_undegraded': '2023-11-26T07:18:51.637758+0000', 'last_fullsized': '2023-11-26T07:18:51.637758+0000', 'mapping_epoch': 31, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 31, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-26T07:18:51.625829+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-26T07:18:51.625829+0000', 'last_clean_scrub_stamp': '2023-11-26T07:18:51.625829+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [4, 0], 'acting': [4, 0], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 4, 'acting_primary': 4, 'purged_snaps': []} 2023-11-26T07:18:57.490 INFO:tasks.cephfs.filesystem.ceph_manager:PG 5.2a is not active+clean 2023-11-26T07:18:57.490 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '5.2a', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 31, 'state': 'creating+peering', 'last_fresh': '2023-11-26T07:18:51.636183+0000', 'last_change': '2023-11-26T07:18:51.636183+0000', 'last_active': '2023-11-26T07:18:51.625829+0000', 'last_peered': '2023-11-26T07:18:51.625829+0000', 'last_clean': '2023-11-26T07:18:51.625829+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-26T07:18:51.636183+0000', 'last_undegraded': '2023-11-26T07:18:51.636183+0000', 'last_fullsized': '2023-11-26T07:18:51.636183+0000', 'mapping_epoch': 31, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 31, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-26T07:18:51.625829+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-26T07:18:51.625829+0000', 'last_clean_scrub_stamp': '2023-11-26T07:18:51.625829+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [4, 3], 'acting': [4, 3], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 4, 'acting_primary': 4, 'purged_snaps': []} 2023-11-26T07:18:57.490 INFO:tasks.cephfs.filesystem.ceph_manager:PG 5.29 is not active+clean 2023-11-26T07:18:57.490 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '5.29', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 31, 'state': 'creating+peering', 'last_fresh': '2023-11-26T07:18:51.641884+0000', 'last_change': '2023-11-26T07:18:51.641884+0000', 'last_active': '2023-11-26T07:18:51.625829+0000', 'last_peered': '2023-11-26T07:18:51.625829+0000', 'last_clean': '2023-11-26T07:18:51.625829+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-26T07:18:51.641884+0000', 'last_undegraded': '2023-11-26T07:18:51.641884+0000', 'last_fullsized': '2023-11-26T07:18:51.641884+0000', 'mapping_epoch': 31, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 31, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-26T07:18:51.625829+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-26T07:18:51.625829+0000', 'last_clean_scrub_stamp': '2023-11-26T07:18:51.625829+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 7], 'acting': [0, 7], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2023-11-26T07:18:57.490 INFO:tasks.cephfs.filesystem.ceph_manager:PG 5.28 is not active+clean 2023-11-26T07:18:57.490 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '5.28', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 31, 'state': 'creating+peering', 'last_fresh': '2023-11-26T07:18:51.635231+0000', 'last_change': '2023-11-26T07:18:51.635231+0000', 'last_active': '2023-11-26T07:18:51.625829+0000', 'last_peered': '2023-11-26T07:18:51.625829+0000', 'last_clean': '2023-11-26T07:18:51.625829+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-26T07:18:51.635231+0000', 'last_undegraded': '2023-11-26T07:18:51.635231+0000', 'last_fullsized': '2023-11-26T07:18:51.635231+0000', 'mapping_epoch': 31, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 31, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-26T07:18:51.625829+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-26T07:18:51.625829+0000', 'last_clean_scrub_stamp': '2023-11-26T07:18:51.625829+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [5, 2], 'acting': [5, 2], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 5, 'acting_primary': 5, 'purged_snaps': []} 2023-11-26T07:18:57.491 INFO:tasks.cephfs.filesystem.ceph_manager:PG 5.27 is not active+clean 2023-11-26T07:18:57.491 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '5.27', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 31, 'state': 'creating+peering', 'last_fresh': '2023-11-26T07:18:51.635689+0000', 'last_change': '2023-11-26T07:18:51.635689+0000', 'last_active': '2023-11-26T07:18:51.625829+0000', 'last_peered': '2023-11-26T07:18:51.625829+0000', 'last_clean': '2023-11-26T07:18:51.625829+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-26T07:18:51.635689+0000', 'last_undegraded': '2023-11-26T07:18:51.635689+0000', 'last_fullsized': '2023-11-26T07:18:51.635689+0000', 'mapping_epoch': 31, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 31, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-26T07:18:51.625829+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-26T07:18:51.625829+0000', 'last_clean_scrub_stamp': '2023-11-26T07:18:51.625829+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 7], 'acting': [1, 7], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 1, 'purged_snaps': []} 2023-11-26T07:18:57.491 INFO:tasks.cephfs.filesystem.ceph_manager:PG 5.26 is not active+clean 2023-11-26T07:18:57.491 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '5.26', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 31, 'state': 'creating+peering', 'last_fresh': '2023-11-26T07:18:51.643037+0000', 'last_change': '2023-11-26T07:18:51.643037+0000', 'last_active': '2023-11-26T07:18:51.625829+0000', 'last_peered': '2023-11-26T07:18:51.625829+0000', 'last_clean': '2023-11-26T07:18:51.625829+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-26T07:18:51.643037+0000', 'last_undegraded': '2023-11-26T07:18:51.643037+0000', 'last_fullsized': '2023-11-26T07:18:51.643037+0000', 'mapping_epoch': 31, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 31, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-26T07:18:51.625829+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-26T07:18:51.625829+0000', 'last_clean_scrub_stamp': '2023-11-26T07:18:51.625829+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 7], 'acting': [3, 7], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2023-11-26T07:18:57.491 INFO:tasks.cephfs.filesystem.ceph_manager:PG 5.25 is not active+clean 2023-11-26T07:18:57.491 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '5.25', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 31, 'state': 'creating+peering', 'last_fresh': '2023-11-26T07:18:51.634261+0000', 'last_change': '2023-11-26T07:18:51.634261+0000', 'last_active': '2023-11-26T07:18:51.625829+0000', 'last_peered': '2023-11-26T07:18:51.625829+0000', 'last_clean': '2023-11-26T07:18:51.625829+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-26T07:18:51.634261+0000', 'last_undegraded': '2023-11-26T07:18:51.634261+0000', 'last_fullsized': '2023-11-26T07:18:51.634261+0000', 'mapping_epoch': 31, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 31, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-26T07:18:51.625829+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-26T07:18:51.625829+0000', 'last_clean_scrub_stamp': '2023-11-26T07:18:51.625829+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 4], 'acting': [1, 4], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 1, 'purged_snaps': []} 2023-11-26T07:18:57.491 INFO:tasks.cephfs.filesystem.ceph_manager:PG 5.24 is not active+clean 2023-11-26T07:18:57.491 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '5.24', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 31, 'state': 'creating+peering', 'last_fresh': '2023-11-26T07:18:51.637792+0000', 'last_change': '2023-11-26T07:18:51.637792+0000', 'last_active': '2023-11-26T07:18:51.625829+0000', 'last_peered': '2023-11-26T07:18:51.625829+0000', 'last_clean': '2023-11-26T07:18:51.625829+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-26T07:18:51.637792+0000', 'last_undegraded': '2023-11-26T07:18:51.637792+0000', 'last_fullsized': '2023-11-26T07:18:51.637792+0000', 'mapping_epoch': 31, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 31, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-26T07:18:51.625829+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-26T07:18:51.625829+0000', 'last_clean_scrub_stamp': '2023-11-26T07:18:51.625829+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [5, 0], 'acting': [5, 0], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 5, 'acting_primary': 5, 'purged_snaps': []} 2023-11-26T07:18:57.491 INFO:tasks.cephfs.filesystem.ceph_manager:PG 5.23 is not active+clean 2023-11-26T07:18:57.491 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '5.23', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 31, 'state': 'creating+peering', 'last_fresh': '2023-11-26T07:18:51.637010+0000', 'last_change': '2023-11-26T07:18:51.637010+0000', 'last_active': '2023-11-26T07:18:51.625829+0000', 'last_peered': '2023-11-26T07:18:51.625829+0000', 'last_clean': '2023-11-26T07:18:51.625829+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-26T07:18:51.637010+0000', 'last_undegraded': '2023-11-26T07:18:51.637010+0000', 'last_fullsized': '2023-11-26T07:18:51.637010+0000', 'mapping_epoch': 31, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 31, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-26T07:18:51.625829+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-26T07:18:51.625829+0000', 'last_clean_scrub_stamp': '2023-11-26T07:18:51.625829+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [4, 3], 'acting': [4, 3], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 4, 'acting_primary': 4, 'purged_snaps': []} 2023-11-26T07:18:57.492 INFO:tasks.cephfs.filesystem.ceph_manager:PG 5.22 is not active+clean 2023-11-26T07:18:57.492 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '5.22', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 31, 'state': 'creating+peering', 'last_fresh': '2023-11-26T07:18:51.644576+0000', 'last_change': '2023-11-26T07:18:51.644576+0000', 'last_active': '2023-11-26T07:18:51.625829+0000', 'last_peered': '2023-11-26T07:18:51.625829+0000', 'last_clean': '2023-11-26T07:18:51.625829+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-26T07:18:51.644576+0000', 'last_undegraded': '2023-11-26T07:18:51.644576+0000', 'last_fullsized': '2023-11-26T07:18:51.644576+0000', 'mapping_epoch': 31, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 31, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-26T07:18:51.625829+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-26T07:18:51.625829+0000', 'last_clean_scrub_stamp': '2023-11-26T07:18:51.625829+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [7, 6], 'acting': [7, 6], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 7, 'acting_primary': 7, 'purged_snaps': []} 2023-11-26T07:18:57.492 INFO:tasks.cephfs.filesystem.ceph_manager:PG 5.21 is not active+clean 2023-11-26T07:18:57.492 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '5.21', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 31, 'state': 'creating+peering', 'last_fresh': '2023-11-26T07:18:51.644802+0000', 'last_change': '2023-11-26T07:18:51.644802+0000', 'last_active': '2023-11-26T07:18:51.625829+0000', 'last_peered': '2023-11-26T07:18:51.625829+0000', 'last_clean': '2023-11-26T07:18:51.625829+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-26T07:18:51.644802+0000', 'last_undegraded': '2023-11-26T07:18:51.644802+0000', 'last_fullsized': '2023-11-26T07:18:51.644802+0000', 'mapping_epoch': 31, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 31, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-26T07:18:51.625829+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-26T07:18:51.625829+0000', 'last_clean_scrub_stamp': '2023-11-26T07:18:51.625829+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [6, 2], 'acting': [6, 2], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 6, 'acting_primary': 6, 'purged_snaps': []} 2023-11-26T07:18:57.492 INFO:tasks.cephfs.filesystem.ceph_manager:PG 5.20 is not active+clean 2023-11-26T07:18:57.492 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '5.20', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 31, 'state': 'creating+peering', 'last_fresh': '2023-11-26T07:18:51.646001+0000', 'last_change': '2023-11-26T07:18:51.646001+0000', 'last_active': '2023-11-26T07:18:51.625829+0000', 'last_peered': '2023-11-26T07:18:51.625829+0000', 'last_clean': '2023-11-26T07:18:51.625829+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-26T07:18:51.646001+0000', 'last_undegraded': '2023-11-26T07:18:51.646001+0000', 'last_fullsized': '2023-11-26T07:18:51.646001+0000', 'mapping_epoch': 31, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 31, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-26T07:18:51.625829+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-26T07:18:51.625829+0000', 'last_clean_scrub_stamp': '2023-11-26T07:18:51.625829+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [7, 4], 'acting': [7, 4], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 7, 'acting_primary': 7, 'purged_snaps': []} 2023-11-26T07:18:57.492 INFO:tasks.cephfs.filesystem.ceph_manager:PG 5.1e is not active+clean 2023-11-26T07:18:57.492 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '5.1e', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 31, 'state': 'creating+peering', 'last_fresh': '2023-11-26T07:18:51.641565+0000', 'last_change': '2023-11-26T07:18:51.641565+0000', 'last_active': '2023-11-26T07:18:51.625829+0000', 'last_peered': '2023-11-26T07:18:51.625829+0000', 'last_clean': '2023-11-26T07:18:51.625829+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-26T07:18:51.641565+0000', 'last_undegraded': '2023-11-26T07:18:51.641565+0000', 'last_fullsized': '2023-11-26T07:18:51.641565+0000', 'mapping_epoch': 31, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 31, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-26T07:18:51.625829+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-26T07:18:51.625829+0000', 'last_clean_scrub_stamp': '2023-11-26T07:18:51.625829+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 7], 'acting': [0, 7], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2023-11-26T07:18:57.492 INFO:tasks.cephfs.filesystem.ceph_manager:PG 5.1f is not active+clean 2023-11-26T07:18:57.492 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '5.1f', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 31, 'state': 'creating+peering', 'last_fresh': '2023-11-26T07:18:51.642728+0000', 'last_change': '2023-11-26T07:18:51.642728+0000', 'last_active': '2023-11-26T07:18:51.625829+0000', 'last_peered': '2023-11-26T07:18:51.625829+0000', 'last_clean': '2023-11-26T07:18:51.625829+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-26T07:18:51.642728+0000', 'last_undegraded': '2023-11-26T07:18:51.642728+0000', 'last_fullsized': '2023-11-26T07:18:51.642728+0000', 'mapping_epoch': 31, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 31, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-26T07:18:51.625829+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-26T07:18:51.625829+0000', 'last_clean_scrub_stamp': '2023-11-26T07:18:51.625829+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [6, 4], 'acting': [6, 4], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 6, 'acting_primary': 6, 'purged_snaps': []} 2023-11-26T07:18:57.493 INFO:tasks.cephfs.filesystem.ceph_manager:PG 5.1c is not active+clean 2023-11-26T07:18:57.493 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '5.1c', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 31, 'state': 'creating+peering', 'last_fresh': '2023-11-26T07:18:51.638001+0000', 'last_change': '2023-11-26T07:18:51.638001+0000', 'last_active': '2023-11-26T07:18:51.625829+0000', 'last_peered': '2023-11-26T07:18:51.625829+0000', 'last_clean': '2023-11-26T07:18:51.625829+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-26T07:18:51.638001+0000', 'last_undegraded': '2023-11-26T07:18:51.638001+0000', 'last_fullsized': '2023-11-26T07:18:51.638001+0000', 'mapping_epoch': 31, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 31, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-26T07:18:51.625829+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-26T07:18:51.625829+0000', 'last_clean_scrub_stamp': '2023-11-26T07:18:51.625829+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [4, 3], 'acting': [4, 3], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 4, 'acting_primary': 4, 'purged_snaps': []} 2023-11-26T07:18:57.493 INFO:tasks.cephfs.filesystem.ceph_manager:PG 5.1d is not active+clean 2023-11-26T07:18:57.493 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '5.1d', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 31, 'state': 'creating+peering', 'last_fresh': '2023-11-26T07:18:51.633747+0000', 'last_change': '2023-11-26T07:18:51.633747+0000', 'last_active': '2023-11-26T07:18:51.625829+0000', 'last_peered': '2023-11-26T07:18:51.625829+0000', 'last_clean': '2023-11-26T07:18:51.625829+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-26T07:18:51.633747+0000', 'last_undegraded': '2023-11-26T07:18:51.633747+0000', 'last_fullsized': '2023-11-26T07:18:51.633747+0000', 'mapping_epoch': 31, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 31, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-26T07:18:51.625829+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-26T07:18:51.625829+0000', 'last_clean_scrub_stamp': '2023-11-26T07:18:51.625829+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 4], 'acting': [1, 4], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 1, 'purged_snaps': []} 2023-11-26T07:18:57.493 INFO:tasks.cephfs.filesystem.ceph_manager:PG 5.1a is not active+clean 2023-11-26T07:18:57.493 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '5.1a', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 31, 'state': 'creating+peering', 'last_fresh': '2023-11-26T07:18:51.640301+0000', 'last_change': '2023-11-26T07:18:51.640301+0000', 'last_active': '2023-11-26T07:18:51.625829+0000', 'last_peered': '2023-11-26T07:18:51.625829+0000', 'last_clean': '2023-11-26T07:18:51.625829+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-26T07:18:51.640301+0000', 'last_undegraded': '2023-11-26T07:18:51.640301+0000', 'last_fullsized': '2023-11-26T07:18:51.640301+0000', 'mapping_epoch': 31, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 31, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-26T07:18:51.625829+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-26T07:18:51.625829+0000', 'last_clean_scrub_stamp': '2023-11-26T07:18:51.625829+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [7, 4], 'acting': [7, 4], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 7, 'acting_primary': 7, 'purged_snaps': []} 2023-11-26T07:18:57.493 INFO:tasks.cephfs.filesystem.ceph_manager:PG 5.1b is not active+clean 2023-11-26T07:18:57.493 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '5.1b', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 31, 'state': 'creating+peering', 'last_fresh': '2023-11-26T07:18:51.635218+0000', 'last_change': '2023-11-26T07:18:51.635218+0000', 'last_active': '2023-11-26T07:18:51.625829+0000', 'last_peered': '2023-11-26T07:18:51.625829+0000', 'last_clean': '2023-11-26T07:18:51.625829+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-26T07:18:51.635218+0000', 'last_undegraded': '2023-11-26T07:18:51.635218+0000', 'last_fullsized': '2023-11-26T07:18:51.635218+0000', 'mapping_epoch': 31, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 31, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-26T07:18:51.625829+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-26T07:18:51.625829+0000', 'last_clean_scrub_stamp': '2023-11-26T07:18:51.625829+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [5, 0], 'acting': [5, 0], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 5, 'acting_primary': 5, 'purged_snaps': []} 2023-11-26T07:18:57.493 INFO:tasks.cephfs.filesystem.ceph_manager:PG 5.18 is not active+clean 2023-11-26T07:18:57.493 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '5.18', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 31, 'state': 'creating+peering', 'last_fresh': '2023-11-26T07:18:51.634554+0000', 'last_change': '2023-11-26T07:18:51.634554+0000', 'last_active': '2023-11-26T07:18:51.625829+0000', 'last_peered': '2023-11-26T07:18:51.625829+0000', 'last_clean': '2023-11-26T07:18:51.625829+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-26T07:18:51.634554+0000', 'last_undegraded': '2023-11-26T07:18:51.634554+0000', 'last_fullsized': '2023-11-26T07:18:51.634554+0000', 'mapping_epoch': 31, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 31, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-26T07:18:51.625829+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-26T07:18:51.625829+0000', 'last_clean_scrub_stamp': '2023-11-26T07:18:51.625829+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [4, 6], 'acting': [4, 6], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 4, 'acting_primary': 4, 'purged_snaps': []} 2023-11-26T07:18:57.494 INFO:tasks.cephfs.filesystem.ceph_manager:PG 5.19 is not active+clean 2023-11-26T07:18:57.494 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '5.19', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 31, 'state': 'creating+peering', 'last_fresh': '2023-11-26T07:18:51.633130+0000', 'last_change': '2023-11-26T07:18:51.633130+0000', 'last_active': '2023-11-26T07:18:51.625829+0000', 'last_peered': '2023-11-26T07:18:51.625829+0000', 'last_clean': '2023-11-26T07:18:51.625829+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-26T07:18:51.633130+0000', 'last_undegraded': '2023-11-26T07:18:51.633130+0000', 'last_fullsized': '2023-11-26T07:18:51.633130+0000', 'mapping_epoch': 31, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 31, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-26T07:18:51.625829+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-26T07:18:51.625829+0000', 'last_clean_scrub_stamp': '2023-11-26T07:18:51.625829+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 5], 'acting': [1, 5], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 1, 'purged_snaps': []} 2023-11-26T07:18:57.494 INFO:tasks.cephfs.filesystem.ceph_manager:PG 5.16 is not active+clean 2023-11-26T07:18:57.494 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '5.16', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 31, 'state': 'creating+peering', 'last_fresh': '2023-11-26T07:18:51.635635+0000', 'last_change': '2023-11-26T07:18:51.635635+0000', 'last_active': '2023-11-26T07:18:51.625829+0000', 'last_peered': '2023-11-26T07:18:51.625829+0000', 'last_clean': '2023-11-26T07:18:51.625829+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-26T07:18:51.635635+0000', 'last_undegraded': '2023-11-26T07:18:51.635635+0000', 'last_fullsized': '2023-11-26T07:18:51.635635+0000', 'mapping_epoch': 31, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 31, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-26T07:18:51.625829+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-26T07:18:51.625829+0000', 'last_clean_scrub_stamp': '2023-11-26T07:18:51.625829+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [5, 3], 'acting': [5, 3], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 5, 'acting_primary': 5, 'purged_snaps': []} 2023-11-26T07:18:57.494 INFO:tasks.cephfs.filesystem.ceph_manager:PG 5.17 is not active+clean 2023-11-26T07:18:57.494 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '5.17', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 31, 'state': 'creating+peering', 'last_fresh': '2023-11-26T07:18:51.642485+0000', 'last_change': '2023-11-26T07:18:51.642485+0000', 'last_active': '2023-11-26T07:18:51.625829+0000', 'last_peered': '2023-11-26T07:18:51.625829+0000', 'last_clean': '2023-11-26T07:18:51.625829+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-26T07:18:51.642485+0000', 'last_undegraded': '2023-11-26T07:18:51.642485+0000', 'last_fullsized': '2023-11-26T07:18:51.642485+0000', 'mapping_epoch': 31, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 31, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-26T07:18:51.625829+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-26T07:18:51.625829+0000', 'last_clean_scrub_stamp': '2023-11-26T07:18:51.625829+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 1], 'acting': [3, 1], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2023-11-26T07:18:57.494 INFO:tasks.cephfs.filesystem.ceph_manager:PG 5.14 is not active+clean 2023-11-26T07:18:57.494 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '5.14', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 31, 'state': 'creating+peering', 'last_fresh': '2023-11-26T07:18:51.643692+0000', 'last_change': '2023-11-26T07:18:51.643692+0000', 'last_active': '2023-11-26T07:18:51.625829+0000', 'last_peered': '2023-11-26T07:18:51.625829+0000', 'last_clean': '2023-11-26T07:18:51.625829+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-26T07:18:51.643692+0000', 'last_undegraded': '2023-11-26T07:18:51.643692+0000', 'last_fullsized': '2023-11-26T07:18:51.643692+0000', 'mapping_epoch': 31, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 31, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-26T07:18:51.625829+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-26T07:18:51.625829+0000', 'last_clean_scrub_stamp': '2023-11-26T07:18:51.625829+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 7], 'acting': [3, 7], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2023-11-26T07:18:57.494 INFO:tasks.cephfs.filesystem.ceph_manager:PG 5.15 is not active+clean 2023-11-26T07:18:57.494 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '5.15', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 31, 'state': 'creating+peering', 'last_fresh': '2023-11-26T07:18:51.638905+0000', 'last_change': '2023-11-26T07:18:51.638905+0000', 'last_active': '2023-11-26T07:18:51.625829+0000', 'last_peered': '2023-11-26T07:18:51.625829+0000', 'last_clean': '2023-11-26T07:18:51.625829+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-26T07:18:51.638905+0000', 'last_undegraded': '2023-11-26T07:18:51.638905+0000', 'last_fullsized': '2023-11-26T07:18:51.638905+0000', 'mapping_epoch': 31, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 31, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-26T07:18:51.625829+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-26T07:18:51.625829+0000', 'last_clean_scrub_stamp': '2023-11-26T07:18:51.625829+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [5, 1], 'acting': [5, 1], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 5, 'acting_primary': 5, 'purged_snaps': []} 2023-11-26T07:18:57.495 INFO:tasks.cephfs.filesystem.ceph_manager:PG 5.12 is not active+clean 2023-11-26T07:18:57.495 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '5.12', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 31, 'state': 'creating+peering', 'last_fresh': '2023-11-26T07:18:51.633529+0000', 'last_change': '2023-11-26T07:18:51.633529+0000', 'last_active': '2023-11-26T07:18:51.625829+0000', 'last_peered': '2023-11-26T07:18:51.625829+0000', 'last_clean': '2023-11-26T07:18:51.625829+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-26T07:18:51.633529+0000', 'last_undegraded': '2023-11-26T07:18:51.633529+0000', 'last_fullsized': '2023-11-26T07:18:51.633529+0000', 'mapping_epoch': 31, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 31, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-26T07:18:51.625829+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-26T07:18:51.625829+0000', 'last_clean_scrub_stamp': '2023-11-26T07:18:51.625829+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 5], 'acting': [1, 5], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 1, 'purged_snaps': []} 2023-11-26T07:18:57.495 INFO:tasks.cephfs.filesystem.ceph_manager:PG 5.13 is not active+clean 2023-11-26T07:18:57.495 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '5.13', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 31, 'state': 'creating+peering', 'last_fresh': '2023-11-26T07:18:51.642640+0000', 'last_change': '2023-11-26T07:18:51.642640+0000', 'last_active': '2023-11-26T07:18:51.625829+0000', 'last_peered': '2023-11-26T07:18:51.625829+0000', 'last_clean': '2023-11-26T07:18:51.625829+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-26T07:18:51.642640+0000', 'last_undegraded': '2023-11-26T07:18:51.642640+0000', 'last_fullsized': '2023-11-26T07:18:51.642640+0000', 'mapping_epoch': 31, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 31, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-26T07:18:51.625829+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-26T07:18:51.625829+0000', 'last_clean_scrub_stamp': '2023-11-26T07:18:51.625829+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 0], 'acting': [3, 0], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2023-11-26T07:18:57.495 INFO:tasks.cephfs.filesystem.ceph_manager:PG 5.10 is not active+clean 2023-11-26T07:18:57.495 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '5.10', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 31, 'state': 'creating+peering', 'last_fresh': '2023-11-26T07:18:51.645524+0000', 'last_change': '2023-11-26T07:18:51.645524+0000', 'last_active': '2023-11-26T07:18:51.625829+0000', 'last_peered': '2023-11-26T07:18:51.625829+0000', 'last_clean': '2023-11-26T07:18:51.625829+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-26T07:18:51.645524+0000', 'last_undegraded': '2023-11-26T07:18:51.645524+0000', 'last_fullsized': '2023-11-26T07:18:51.645524+0000', 'mapping_epoch': 31, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 31, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-26T07:18:51.625829+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-26T07:18:51.625829+0000', 'last_clean_scrub_stamp': '2023-11-26T07:18:51.625829+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [7, 4], 'acting': [7, 4], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 7, 'acting_primary': 7, 'purged_snaps': []} 2023-11-26T07:18:57.495 INFO:tasks.cephfs.filesystem.ceph_manager:PG 5.11 is not active+clean 2023-11-26T07:18:57.495 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '5.11', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 31, 'state': 'creating+peering', 'last_fresh': '2023-11-26T07:18:51.642391+0000', 'last_change': '2023-11-26T07:18:51.642391+0000', 'last_active': '2023-11-26T07:18:51.625829+0000', 'last_peered': '2023-11-26T07:18:51.625829+0000', 'last_clean': '2023-11-26T07:18:51.625829+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-26T07:18:51.642391+0000', 'last_undegraded': '2023-11-26T07:18:51.642391+0000', 'last_fullsized': '2023-11-26T07:18:51.642391+0000', 'mapping_epoch': 31, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 31, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-26T07:18:51.625829+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-26T07:18:51.625829+0000', 'last_clean_scrub_stamp': '2023-11-26T07:18:51.625829+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [6, 4], 'acting': [6, 4], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 6, 'acting_primary': 6, 'purged_snaps': []} 2023-11-26T07:18:57.495 INFO:tasks.cephfs.filesystem.ceph_manager:PG 5.e is not active+clean 2023-11-26T07:18:57.495 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '5.e', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 31, 'state': 'creating+peering', 'last_fresh': '2023-11-26T07:18:51.635911+0000', 'last_change': '2023-11-26T07:18:51.635911+0000', 'last_active': '2023-11-26T07:18:51.625829+0000', 'last_peered': '2023-11-26T07:18:51.625829+0000', 'last_clean': '2023-11-26T07:18:51.625829+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-26T07:18:51.635911+0000', 'last_undegraded': '2023-11-26T07:18:51.635911+0000', 'last_fullsized': '2023-11-26T07:18:51.635911+0000', 'mapping_epoch': 31, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 31, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-26T07:18:51.625829+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-26T07:18:51.625829+0000', 'last_clean_scrub_stamp': '2023-11-26T07:18:51.625829+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [4, 5], 'acting': [4, 5], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 4, 'acting_primary': 4, 'purged_snaps': []} 2023-11-26T07:18:57.495 INFO:tasks.cephfs.filesystem.ceph_manager:PG 5.f is not active+clean 2023-11-26T07:18:57.496 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '5.f', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 31, 'state': 'creating+peering', 'last_fresh': '2023-11-26T07:18:51.641273+0000', 'last_change': '2023-11-26T07:18:51.641273+0000', 'last_active': '2023-11-26T07:18:51.625829+0000', 'last_peered': '2023-11-26T07:18:51.625829+0000', 'last_clean': '2023-11-26T07:18:51.625829+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-26T07:18:51.641273+0000', 'last_undegraded': '2023-11-26T07:18:51.641273+0000', 'last_fullsized': '2023-11-26T07:18:51.641273+0000', 'mapping_epoch': 31, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 31, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-26T07:18:51.625829+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-26T07:18:51.625829+0000', 'last_clean_scrub_stamp': '2023-11-26T07:18:51.625829+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [5, 4], 'acting': [5, 4], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 5, 'acting_primary': 5, 'purged_snaps': []} 2023-11-26T07:18:57.496 INFO:tasks.cephfs.filesystem.ceph_manager:PG 5.c is not active+clean 2023-11-26T07:18:57.496 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '5.c', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 31, 'state': 'creating+peering', 'last_fresh': '2023-11-26T07:18:51.633505+0000', 'last_change': '2023-11-26T07:18:51.633505+0000', 'last_active': '2023-11-26T07:18:51.625829+0000', 'last_peered': '2023-11-26T07:18:51.625829+0000', 'last_clean': '2023-11-26T07:18:51.625829+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-26T07:18:51.633505+0000', 'last_undegraded': '2023-11-26T07:18:51.633505+0000', 'last_fullsized': '2023-11-26T07:18:51.633505+0000', 'mapping_epoch': 31, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 31, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-26T07:18:51.625829+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-26T07:18:51.625829+0000', 'last_clean_scrub_stamp': '2023-11-26T07:18:51.625829+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 4], 'acting': [1, 4], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 1, 'purged_snaps': []} 2023-11-26T07:18:57.496 INFO:tasks.cephfs.filesystem.ceph_manager:PG 5.d is not active+clean 2023-11-26T07:18:57.496 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '5.d', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 31, 'state': 'creating+peering', 'last_fresh': '2023-11-26T07:18:51.653086+0000', 'last_change': '2023-11-26T07:18:51.653086+0000', 'last_active': '2023-11-26T07:18:51.625829+0000', 'last_peered': '2023-11-26T07:18:51.625829+0000', 'last_clean': '2023-11-26T07:18:51.625829+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-26T07:18:51.653086+0000', 'last_undegraded': '2023-11-26T07:18:51.653086+0000', 'last_fullsized': '2023-11-26T07:18:51.653086+0000', 'mapping_epoch': 31, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 31, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-26T07:18:51.625829+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-26T07:18:51.625829+0000', 'last_clean_scrub_stamp': '2023-11-26T07:18:51.625829+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 7], 'acting': [2, 7], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2023-11-26T07:18:57.496 INFO:tasks.cephfs.filesystem.ceph_manager:PG 5.a is not active+clean 2023-11-26T07:18:57.496 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '5.a', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 31, 'state': 'creating+peering', 'last_fresh': '2023-11-26T07:18:51.649002+0000', 'last_change': '2023-11-26T07:18:51.649002+0000', 'last_active': '2023-11-26T07:18:51.625829+0000', 'last_peered': '2023-11-26T07:18:51.625829+0000', 'last_clean': '2023-11-26T07:18:51.625829+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-26T07:18:51.649002+0000', 'last_undegraded': '2023-11-26T07:18:51.649002+0000', 'last_fullsized': '2023-11-26T07:18:51.649002+0000', 'mapping_epoch': 31, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 31, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-26T07:18:51.625829+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-26T07:18:51.625829+0000', 'last_clean_scrub_stamp': '2023-11-26T07:18:51.625829+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 4], 'acting': [2, 4], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2023-11-26T07:18:57.496 INFO:tasks.cephfs.filesystem.ceph_manager:PG 5.b is not active+clean 2023-11-26T07:18:57.496 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '5.b', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 31, 'state': 'creating+peering', 'last_fresh': '2023-11-26T07:18:51.649020+0000', 'last_change': '2023-11-26T07:18:51.649020+0000', 'last_active': '2023-11-26T07:18:51.625829+0000', 'last_peered': '2023-11-26T07:18:51.625829+0000', 'last_clean': '2023-11-26T07:18:51.625829+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-26T07:18:51.649020+0000', 'last_undegraded': '2023-11-26T07:18:51.649020+0000', 'last_fullsized': '2023-11-26T07:18:51.649020+0000', 'mapping_epoch': 31, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 31, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-26T07:18:51.625829+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-26T07:18:51.625829+0000', 'last_clean_scrub_stamp': '2023-11-26T07:18:51.625829+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 0], 'acting': [2, 0], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2023-11-26T07:18:57.496 INFO:tasks.cephfs.filesystem.ceph_manager:PG 5.8 is not active+clean 2023-11-26T07:18:57.497 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '5.8', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 31, 'state': 'creating+peering', 'last_fresh': '2023-11-26T07:18:51.649368+0000', 'last_change': '2023-11-26T07:18:51.649368+0000', 'last_active': '2023-11-26T07:18:51.625829+0000', 'last_peered': '2023-11-26T07:18:51.625829+0000', 'last_clean': '2023-11-26T07:18:51.625829+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-26T07:18:51.649368+0000', 'last_undegraded': '2023-11-26T07:18:51.649368+0000', 'last_fullsized': '2023-11-26T07:18:51.649368+0000', 'mapping_epoch': 31, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 31, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-26T07:18:51.625829+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-26T07:18:51.625829+0000', 'last_clean_scrub_stamp': '2023-11-26T07:18:51.625829+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 0], 'acting': [2, 0], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2023-11-26T07:18:57.497 INFO:tasks.cephfs.filesystem.ceph_manager:PG 5.9 is not active+clean 2023-11-26T07:18:57.497 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '5.9', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 31, 'state': 'creating+peering', 'last_fresh': '2023-11-26T07:18:51.642082+0000', 'last_change': '2023-11-26T07:18:51.642082+0000', 'last_active': '2023-11-26T07:18:51.625829+0000', 'last_peered': '2023-11-26T07:18:51.625829+0000', 'last_clean': '2023-11-26T07:18:51.625829+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-26T07:18:51.642082+0000', 'last_undegraded': '2023-11-26T07:18:51.642082+0000', 'last_fullsized': '2023-11-26T07:18:51.642082+0000', 'mapping_epoch': 31, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 31, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-26T07:18:51.625829+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-26T07:18:51.625829+0000', 'last_clean_scrub_stamp': '2023-11-26T07:18:51.625829+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [7, 6], 'acting': [7, 6], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 7, 'acting_primary': 7, 'purged_snaps': []} 2023-11-26T07:18:57.497 INFO:tasks.cephfs.filesystem.ceph_manager:PG 5.6 is not active+clean 2023-11-26T07:18:57.497 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '5.6', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 31, 'state': 'creating+peering', 'last_fresh': '2023-11-26T07:18:51.651544+0000', 'last_change': '2023-11-26T07:18:51.651544+0000', 'last_active': '2023-11-26T07:18:51.625829+0000', 'last_peered': '2023-11-26T07:18:51.625829+0000', 'last_clean': '2023-11-26T07:18:51.625829+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-26T07:18:51.651544+0000', 'last_undegraded': '2023-11-26T07:18:51.651544+0000', 'last_fullsized': '2023-11-26T07:18:51.651544+0000', 'mapping_epoch': 31, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 31, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-26T07:18:51.625829+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-26T07:18:51.625829+0000', 'last_clean_scrub_stamp': '2023-11-26T07:18:51.625829+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 5], 'acting': [2, 5], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2023-11-26T07:18:57.497 INFO:tasks.cephfs.filesystem.ceph_manager:PG 5.7 is not active+clean 2023-11-26T07:18:57.497 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '5.7', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 31, 'state': 'creating+peering', 'last_fresh': '2023-11-26T07:18:51.637655+0000', 'last_change': '2023-11-26T07:18:51.637655+0000', 'last_active': '2023-11-26T07:18:51.625829+0000', 'last_peered': '2023-11-26T07:18:51.625829+0000', 'last_clean': '2023-11-26T07:18:51.625829+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-26T07:18:51.637655+0000', 'last_undegraded': '2023-11-26T07:18:51.637655+0000', 'last_fullsized': '2023-11-26T07:18:51.637655+0000', 'mapping_epoch': 31, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 31, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-26T07:18:51.625829+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-26T07:18:51.625829+0000', 'last_clean_scrub_stamp': '2023-11-26T07:18:51.625829+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [5, 1], 'acting': [5, 1], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 5, 'acting_primary': 5, 'purged_snaps': []} 2023-11-26T07:18:57.497 INFO:tasks.cephfs.filesystem.ceph_manager:PG 5.5 is not active+clean 2023-11-26T07:18:57.497 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '5.5', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 31, 'state': 'creating+peering', 'last_fresh': '2023-11-26T07:18:51.641452+0000', 'last_change': '2023-11-26T07:18:51.641452+0000', 'last_active': '2023-11-26T07:18:51.625829+0000', 'last_peered': '2023-11-26T07:18:51.625829+0000', 'last_clean': '2023-11-26T07:18:51.625829+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-26T07:18:51.641452+0000', 'last_undegraded': '2023-11-26T07:18:51.641452+0000', 'last_fullsized': '2023-11-26T07:18:51.641452+0000', 'mapping_epoch': 31, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 31, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-26T07:18:51.625829+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-26T07:18:51.625829+0000', 'last_clean_scrub_stamp': '2023-11-26T07:18:51.625829+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0, 1], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2023-11-26T07:18:57.497 INFO:tasks.cephfs.filesystem.ceph_manager:PG 5.2 is not active+clean 2023-11-26T07:18:57.497 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '5.2', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 31, 'state': 'creating+peering', 'last_fresh': '2023-11-26T07:18:51.642362+0000', 'last_change': '2023-11-26T07:18:51.642362+0000', 'last_active': '2023-11-26T07:18:51.625829+0000', 'last_peered': '2023-11-26T07:18:51.625829+0000', 'last_clean': '2023-11-26T07:18:51.625829+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-26T07:18:51.642362+0000', 'last_undegraded': '2023-11-26T07:18:51.642362+0000', 'last_fullsized': '2023-11-26T07:18:51.642362+0000', 'mapping_epoch': 31, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 31, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-26T07:18:51.625829+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-26T07:18:51.625829+0000', 'last_clean_scrub_stamp': '2023-11-26T07:18:51.625829+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [6, 0], 'acting': [6, 0], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 6, 'acting_primary': 6, 'purged_snaps': []} 2023-11-26T07:18:57.498 INFO:tasks.cephfs.filesystem.ceph_manager:PG 5.3 is not active+clean 2023-11-26T07:18:57.498 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '5.3', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 31, 'state': 'creating+peering', 'last_fresh': '2023-11-26T07:18:51.642003+0000', 'last_change': '2023-11-26T07:18:51.642003+0000', 'last_active': '2023-11-26T07:18:51.625829+0000', 'last_peered': '2023-11-26T07:18:51.625829+0000', 'last_clean': '2023-11-26T07:18:51.625829+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-26T07:18:51.642003+0000', 'last_undegraded': '2023-11-26T07:18:51.642003+0000', 'last_fullsized': '2023-11-26T07:18:51.642003+0000', 'mapping_epoch': 31, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 31, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-26T07:18:51.625829+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-26T07:18:51.625829+0000', 'last_clean_scrub_stamp': '2023-11-26T07:18:51.625829+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 6], 'acting': [0, 6], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2023-11-26T07:18:57.498 INFO:tasks.cephfs.filesystem.ceph_manager:PG 5.0 is not active+clean 2023-11-26T07:18:57.498 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '5.0', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 31, 'state': 'creating+peering', 'last_fresh': '2023-11-26T07:18:51.642260+0000', 'last_change': '2023-11-26T07:18:51.642260+0000', 'last_active': '2023-11-26T07:18:51.625829+0000', 'last_peered': '2023-11-26T07:18:51.625829+0000', 'last_clean': '2023-11-26T07:18:51.625829+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-26T07:18:51.642260+0000', 'last_undegraded': '2023-11-26T07:18:51.642260+0000', 'last_fullsized': '2023-11-26T07:18:51.642260+0000', 'mapping_epoch': 31, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 31, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-26T07:18:51.625829+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-26T07:18:51.625829+0000', 'last_clean_scrub_stamp': '2023-11-26T07:18:51.625829+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 1], 'acting': [3, 1], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2023-11-26T07:18:57.498 INFO:tasks.cephfs.filesystem.ceph_manager:PG 5.1 is not active+clean 2023-11-26T07:18:57.498 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '5.1', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 31, 'state': 'creating+peering', 'last_fresh': '2023-11-26T07:18:51.635350+0000', 'last_change': '2023-11-26T07:18:51.635350+0000', 'last_active': '2023-11-26T07:18:51.625829+0000', 'last_peered': '2023-11-26T07:18:51.625829+0000', 'last_clean': '2023-11-26T07:18:51.625829+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-26T07:18:51.635350+0000', 'last_undegraded': '2023-11-26T07:18:51.635350+0000', 'last_fullsized': '2023-11-26T07:18:51.635350+0000', 'mapping_epoch': 31, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 31, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-26T07:18:51.625829+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-26T07:18:51.625829+0000', 'last_clean_scrub_stamp': '2023-11-26T07:18:51.625829+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [4, 3], 'acting': [4, 3], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 4, 'acting_primary': 4, 'purged_snaps': []} 2023-11-26T07:18:57.498 INFO:tasks.cephfs.filesystem.ceph_manager:PG 5.4 is not active+clean 2023-11-26T07:18:57.498 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '5.4', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 31, 'state': 'creating+peering', 'last_fresh': '2023-11-26T07:18:51.643304+0000', 'last_change': '2023-11-26T07:18:51.643304+0000', 'last_active': '2023-11-26T07:18:51.625829+0000', 'last_peered': '2023-11-26T07:18:51.625829+0000', 'last_clean': '2023-11-26T07:18:51.625829+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-26T07:18:51.643304+0000', 'last_undegraded': '2023-11-26T07:18:51.643304+0000', 'last_fullsized': '2023-11-26T07:18:51.643304+0000', 'mapping_epoch': 31, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 31, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-26T07:18:51.625829+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-26T07:18:51.625829+0000', 'last_clean_scrub_stamp': '2023-11-26T07:18:51.625829+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [7, 2], 'acting': [7, 2], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 7, 'acting_primary': 7, 'purged_snaps': []} 2023-11-26T07:18:57.499 DEBUG:teuthology.orchestra.run.smithi143:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2023-11-26T07:18:57.666 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:57.660+0000 7f36256e4640 1 Processor -- start 2023-11-26T07:18:57.666 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:57.660+0000 7f36256e4640 1 -- start start 2023-11-26T07:18:57.667 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:57.660+0000 7f36256e4640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f362015d6e0 0x7f36200d0df0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:18:57.667 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:57.660+0000 7f36256e4640 1 -- --> v1:172.21.15.143:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f36200cf0e0 con 0x7f36200b2160 2023-11-26T07:18:57.667 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:57.660+0000 7f36256e4640 1 -- --> v1:172.21.15.188:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f36200ceb20 con 0x7f36200d1bc0 2023-11-26T07:18:57.667 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:57.660+0000 7f36256e4640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f362015dac0 con 0x7f362015d6e0 2023-11-26T07:18:57.667 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:57.660+0000 7f361e7fc640 1 --1- >> v1:172.21.15.143:6789/0 conn(0x7f36200b2160 0x7f36200d14b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.143:6789/0 says I am v1:172.21.15.143:35296/0 (socket says 172.21.15.143:35296) 2023-11-26T07:18:57.667 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:57.660+0000 7f361e7fc640 1 -- 172.21.15.143:0/810643633 learned_addr learned my addr 172.21.15.143:0/810643633 (peer_addr_for_me v1:172.21.15.143:0/0) 2023-11-26T07:18:57.669 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:57.660+0000 7f361dffb640 1 -- 172.21.15.143:0/810643633 <== mon.0 v1:172.21.15.143:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1628038282 0 0) 0x7f36200cf0e0 con 0x7f36200b2160 2023-11-26T07:18:57.669 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:57.660+0000 7f361dffb640 1 -- 172.21.15.143:0/810643633 --> v1:172.21.15.143:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f3608003530 con 0x7f36200b2160 2023-11-26T07:18:57.669 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:57.660+0000 7f361effd640 1 --2- 172.21.15.143:0/810643633 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f362015d6e0 0x7f36200d0df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:18:57.669 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:57.660+0000 7f361dffb640 1 -- 172.21.15.143:0/810643633 <== mon.0 v1:172.21.15.143:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1512816505 0 0) 0x7f3608003530 con 0x7f36200b2160 2023-11-26T07:18:57.670 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:57.660+0000 7f361dffb640 1 -- 172.21.15.143:0/810643633 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f362015d6e0 msgr2=0x7f36200d0df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-26T07:18:57.670 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:57.660+0000 7f361dffb640 1 --2- 172.21.15.143:0/810643633 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f362015d6e0 0x7f36200d0df0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:18:57.670 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:57.660+0000 7f361dffb640 1 -- 172.21.15.143:0/810643633 >> v1:172.21.15.188:6789/0 conn(0x7f36200d1bc0 legacy=0x7f36200b0750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:18:57.670 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:57.660+0000 7f361dffb640 1 -- 172.21.15.143:0/810643633 --> v1:172.21.15.143:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f36200cee00 con 0x7f36200b2160 2023-11-26T07:18:57.670 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:57.660+0000 7f361dffb640 1 -- 172.21.15.143:0/810643633 <== mon.0 v1:172.21.15.143:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2137882523 0 0) 0x7f3614001680 con 0x7f36200b2160 2023-11-26T07:18:57.670 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:57.660+0000 7f361dffb640 1 -- 172.21.15.143:0/810643633 >> v1:172.21.15.143:6789/0 conn(0x7f36200b2160 legacy=0x7f36200d14b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:18:57.670 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:57.660+0000 7f361dffb640 1 --2- 172.21.15.143:0/810643633 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f362015d6e0 0x7f36080039b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:18:57.670 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:57.660+0000 7f361dffb640 1 --2- 172.21.15.143:0/810643633 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f3608004030 0x7f36080047d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:18:57.670 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:57.660+0000 7f361dffb640 1 --2- 172.21.15.143:0/810643633 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f3608004d10 0x7f3608005190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:18:57.670 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:57.660+0000 7f361dffb640 1 -- 172.21.15.143:0/810643633 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f362015dac0 con 0x7f3608004030 2023-11-26T07:18:57.670 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:57.660+0000 7f361dffb640 1 -- 172.21.15.143:0/810643633 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f360800bbd0 con 0x7f3608004d10 2023-11-26T07:18:57.670 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:57.660+0000 7f361dffb640 1 -- 172.21.15.143:0/810643633 --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f360800bd90 con 0x7f362015d6e0 2023-11-26T07:18:57.671 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:57.660+0000 7f361e7fc640 1 --2- 172.21.15.143:0/810643633 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f3608004030 0x7f36080047d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:18:57.671 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:57.660+0000 7f361effd640 1 --2- 172.21.15.143:0/810643633 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f362015d6e0 0x7f36080039b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:18:57.671 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:57.660+0000 7f361f7fe640 1 --2- 172.21.15.143:0/810643633 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f3608004d10 0x7f3608005190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:18:57.671 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:57.660+0000 7f361e7fc640 1 -- 172.21.15.143:0/810643633 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f362015d6e0 msgr2=0x7f36080039b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-26T07:18:57.671 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:57.660+0000 7f361e7fc640 1 --2- 172.21.15.143:0/810643633 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f362015d6e0 0x7f36080039b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:18:57.671 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:57.660+0000 7f361e7fc640 1 -- 172.21.15.143:0/810643633 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f3608004d10 msgr2=0x7f3608005190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-26T07:18:57.671 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:57.660+0000 7f361e7fc640 1 --2- 172.21.15.143:0/810643633 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f3608004d10 0x7f3608005190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:18:57.671 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:57.660+0000 7f361e7fc640 1 -- 172.21.15.143:0/810643633 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3608003530 con 0x7f3608004030 2023-11-26T07:18:57.672 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:57.664+0000 7f361e7fc640 1 --2- 172.21.15.143:0/810643633 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f3608004030 0x7f36080047d0 secure :-1 s=READY pgs=419 cs=0 l=1 rev1=1 crypto rx=0x7f361400c0e0 tx=0x7f361401f2d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7950301f99cfcc49 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:18:57.672 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:57.664+0000 7f361dffb640 1 -- 172.21.15.143:0/810643633 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f36140058e0 con 0x7f3608004030 2023-11-26T07:18:57.673 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:57.664+0000 7f361dffb640 1 -- 172.21.15.143:0/810643633 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3614005a40 con 0x7f3608004030 2023-11-26T07:18:57.673 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:57.664+0000 7f361dffb640 1 -- 172.21.15.143:0/810643633 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3614005e70 con 0x7f3608004030 2023-11-26T07:18:57.673 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:57.664+0000 7f36256e4640 1 -- 172.21.15.143:0/810643633 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f3608004030 msgr2=0x7f36080047d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:18:57.673 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:57.664+0000 7f36256e4640 1 --2- 172.21.15.143:0/810643633 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f3608004030 0x7f36080047d0 secure :-1 s=READY pgs=419 cs=0 l=1 rev1=1 crypto rx=0x7f361400c0e0 tx=0x7f361401f2d0 comp rx=0 tx=0).stop 2023-11-26T07:18:57.673 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:57.664+0000 7f361f7fe640 1 -- 172.21.15.143:0/810643633 reap_dead start 2023-11-26T07:18:57.673 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:57.664+0000 7f36256e4640 1 -- 172.21.15.143:0/810643633 shutdown_connections 2023-11-26T07:18:57.673 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:57.664+0000 7f36256e4640 1 -- 172.21.15.143:0/810643633 >> 172.21.15.143:0/810643633 conn(0x7f362009c550 msgr2=0x7f36200d3220 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:18:57.673 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:57.664+0000 7f36256e4640 1 -- 172.21.15.143:0/810643633 shutdown_connections 2023-11-26T07:18:57.673 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:57.664+0000 7f36256e4640 1 -- 172.21.15.143:0/810643633 wait complete. 2023-11-26T07:18:57.674 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:57.668+0000 7f36256e4640 1 Processor -- start 2023-11-26T07:18:57.674 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:57.668+0000 7f36256e4640 1 -- start start 2023-11-26T07:18:57.676 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:57.668+0000 7f36256e4640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f3608004030 0x7f36200b5020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:18:57.677 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:57.668+0000 7f36256e4640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f36200b6820 0x7f36200caee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:18:57.677 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:57.668+0000 7f36256e4640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f36200b7d30 0x7f36200ccac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:18:57.677 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:57.668+0000 7f36256e4640 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f36200b2600 con 0x7f3608004030 2023-11-26T07:18:57.677 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:57.668+0000 7f36256e4640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f36200b6f90 con 0x7f36200b7d30 2023-11-26T07:18:57.677 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:57.668+0000 7f36256e4640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f36200b28c0 con 0x7f36200b6820 2023-11-26T07:18:57.677 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:57.668+0000 7f361effd640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f3608004030 0x7f36200b5020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:18:57.677 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:57.668+0000 7f361effd640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f3608004030 0x7f36200b5020 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:51766/0 (socket says 172.21.15.143:51766) 2023-11-26T07:18:57.677 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:57.668+0000 7f361effd640 1 -- 172.21.15.143:0/2071645996 learned_addr learned my addr 172.21.15.143:0/2071645996 (peer_addr_for_me v2:172.21.15.143:0/0) 2023-11-26T07:18:57.677 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:57.668+0000 7f361e7fc640 1 --2- 172.21.15.143:0/2071645996 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f36200b6820 0x7f36200caee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:18:57.677 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:57.668+0000 7f361f7fe640 1 --2- 172.21.15.143:0/2071645996 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f36200b7d30 0x7f36200ccac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:18:57.678 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:57.668+0000 7f361effd640 1 -- 172.21.15.143:0/2071645996 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f36200b6820 msgr2=0x7f36200caee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:18:57.678 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:57.668+0000 7f361effd640 1 --2- 172.21.15.143:0/2071645996 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f36200b6820 0x7f36200caee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:18:57.678 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:57.668+0000 7f361effd640 1 -- 172.21.15.143:0/2071645996 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f36200b7d30 msgr2=0x7f36200ccac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:18:57.678 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:57.668+0000 7f361effd640 1 --2- 172.21.15.143:0/2071645996 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f36200b7d30 0x7f36200ccac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:18:57.678 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:57.668+0000 7f361effd640 1 -- 172.21.15.143:0/2071645996 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f36200b2ba0 con 0x7f3608004030 2023-11-26T07:18:57.678 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:57.668+0000 7f361effd640 1 --2- 172.21.15.143:0/2071645996 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f3608004030 0x7f36200b5020 secure :-1 s=READY pgs=420 cs=0 l=1 rev1=1 crypto rx=0x7f361000b7f0 tx=0x7f361000bcb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:18:57.678 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:57.668+0000 7f36077fe640 1 -- 172.21.15.143:0/2071645996 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3610004170 con 0x7f3608004030 2023-11-26T07:18:57.678 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:57.668+0000 7f36077fe640 1 -- 172.21.15.143:0/2071645996 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3610010830 con 0x7f3608004030 2023-11-26T07:18:57.678 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:57.668+0000 7f36256e4640 1 -- 172.21.15.143:0/2071645996 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f36200b2e80 con 0x7f3608004030 2023-11-26T07:18:57.678 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:57.668+0000 7f36077fe640 1 -- 172.21.15.143:0/2071645996 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3610010be0 con 0x7f3608004030 2023-11-26T07:18:57.678 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:57.672+0000 7f36256e4640 1 -- 172.21.15.143:0/2071645996 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f36200cb420 con 0x7f3608004030 2023-11-26T07:18:57.679 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:57.672+0000 7f36256e4640 1 -- 172.21.15.143:0/2071645996 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3620067680 con 0x7f3608004030 2023-11-26T07:18:57.679 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:57.672+0000 7f36077fe640 1 -- 172.21.15.143:0/2071645996 <== mon.0 v2:172.21.15.143:3300/0 4 ==== mgrmap(e 5) v1 ==== 85852+0+0 (secure 0 0 0) 0x7f3610014430 con 0x7f3608004030 2023-11-26T07:18:57.679 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:57.672+0000 7f36077fe640 1 --2- 172.21.15.143:0/2071645996 >> [v2:172.21.15.143:6832/2150557544,v1:172.21.15.143:6833/2150557544] conn(0x7f360c066470 0x7f360c068930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:18:57.680 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:57.672+0000 7f36077fe640 1 -- 172.21.15.143:0/2071645996 <== mon.0 v2:172.21.15.143:3300/0 5 ==== osd_map(33..33 src has 1..33) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f3610084600 con 0x7f3608004030 2023-11-26T07:18:57.680 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:57.672+0000 7f361e7fc640 1 --2- 172.21.15.143:0/2071645996 >> [v2:172.21.15.143:6832/2150557544,v1:172.21.15.143:6833/2150557544] conn(0x7f360c066470 0x7f360c068930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:18:57.680 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:57.676+0000 7f361e7fc640 1 --2- 172.21.15.143:0/2071645996 >> [v2:172.21.15.143:6832/2150557544,v1:172.21.15.143:6833/2150557544] conn(0x7f360c066470 0x7f360c068930 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7f36140ae000 tx=0x7f36140aa070 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:18:57.684 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:57.676+0000 7f36077fe640 1 -- 172.21.15.143:0/2071645996 <== mon.0 v2:172.21.15.143:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172649 (secure 0 0 0) 0x7f3610051b20 con 0x7f3608004030 2023-11-26T07:18:57.888 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:57.880+0000 7f36256e4640 1 -- 172.21.15.143:0/2071645996 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "status", "format": "json"} v 0) v1 -- 0x7f36200740c0 con 0x7f3608004030 2023-11-26T07:18:57.888 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:57.880+0000 7f36077fe640 1 -- 172.21.15.143:0/2071645996 <== mon.0 v2:172.21.15.143:3300/0 7 ==== mon_command_ack([{"prefix": "status", "format": "json"}]=0 v0) v1 ==== 72+0+1580 (secure 0 0 0) 0x7f3610055140 con 0x7f3608004030 2023-11-26T07:18:57.888 INFO:teuthology.orchestra.run.smithi143.stdout: 2023-11-26T07:18:57.889 INFO:teuthology.orchestra.run.smithi143.stdout:{"fsid":"f5033060-2469-4a07-95fe-2b76f5404bc1","health":{"status":"HEALTH_OK","checks":{},"mutes":[]},"election_epoch":4,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":89,"monmap":{"epoch":1,"min_mon_release_name":"reef","num_mons":3},"osdmap":{"epoch":33,"num_osds":8,"num_up_osds":8,"osd_up_since":1700983061,"num_in_osds":8,"osd_in_since":1700983051,"num_remapped_pgs":0},"pgmap":{"pgs_by_state":[{"state_name":"creating+peering","count":64},{"state_name":"active+clean","count":33}],"num_pgs":97,"num_pools":3,"num_objects":2,"data_bytes":590368,"bytes_used":225275904,"bytes_avail":772868837376,"bytes_total":773094113280,"inactive_pgs_ratio":0.65979379415512085},"fsmap":{"epoch":16,"id":2,"up":1,"in":1,"max":1,"by_rank":[{"filesystem_id":2,"rank":0,"name":"c","status":"up:active","gid":5220}],"up:standby":3},"mgrmap":{"available":true,"num_standbys":1,"modules":["iostat","nfs","restful"],"services":{}},"servicemap":{"epoch":3,"modified":"2023-11-26T07:18:43.703638+0000","services":{"mds":{"daemons":{"summary":"","a":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}},"b":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}},"c":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}},"d":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}}}}}},"progress_events":{}} 2023-11-26T07:18:57.890 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:57.884+0000 7f36256e4640 1 -- 172.21.15.143:0/2071645996 >> [v2:172.21.15.143:6832/2150557544,v1:172.21.15.143:6833/2150557544] conn(0x7f360c066470 msgr2=0x7f360c068930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:18:57.890 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:57.884+0000 7f36256e4640 1 --2- 172.21.15.143:0/2071645996 >> [v2:172.21.15.143:6832/2150557544,v1:172.21.15.143:6833/2150557544] conn(0x7f360c066470 0x7f360c068930 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7f36140ae000 tx=0x7f36140aa070 comp rx=0 tx=0).stop 2023-11-26T07:18:57.891 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:57.884+0000 7f36256e4640 1 -- 172.21.15.143:0/2071645996 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f3608004030 msgr2=0x7f36200b5020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:18:57.891 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:57.884+0000 7f36256e4640 1 --2- 172.21.15.143:0/2071645996 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f3608004030 0x7f36200b5020 secure :-1 s=READY pgs=420 cs=0 l=1 rev1=1 crypto rx=0x7f361000b7f0 tx=0x7f361000bcb0 comp rx=0 tx=0).stop 2023-11-26T07:18:57.891 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:57.884+0000 7f36256e4640 1 -- 172.21.15.143:0/2071645996 shutdown_connections 2023-11-26T07:18:57.891 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:57.884+0000 7f36256e4640 1 --2- 172.21.15.143:0/2071645996 >> [v2:172.21.15.143:6832/2150557544,v1:172.21.15.143:6833/2150557544] conn(0x7f360c066470 0x7f360c068930 unknown :-1 s=CLOSED pgs=225 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:18:57.891 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:57.884+0000 7f36256e4640 1 --2- 172.21.15.143:0/2071645996 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f36200b7d30 0x7f36200ccac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:18:57.891 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:57.884+0000 7f36256e4640 1 --2- 172.21.15.143:0/2071645996 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f36200b6820 0x7f36200caee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:18:57.891 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:57.884+0000 7f36256e4640 1 --2- 172.21.15.143:0/2071645996 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f3608004030 0x7f36200b5020 unknown :-1 s=CLOSED pgs=420 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:18:57.891 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:57.884+0000 7f36256e4640 1 -- 172.21.15.143:0/2071645996 >> 172.21.15.143:0/2071645996 conn(0x7f362009c550 msgr2=0x7f36200d0320 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:18:57.891 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:57.884+0000 7f36256e4640 1 -- 172.21.15.143:0/2071645996 shutdown_connections 2023-11-26T07:18:57.891 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:57.884+0000 7f36256e4640 1 -- 172.21.15.143:0/2071645996 wait complete. 2023-11-26T07:18:57.909 INFO:tasks.cephfs.filesystem.ceph_manager:no progress seen, keeping timeout for now 2023-11-26T07:18:57.910 DEBUG:teuthology.orchestra.run.smithi143:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-11-26T07:18:58.047 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:58.040+0000 7f302f757640 1 Processor -- start 2023-11-26T07:18:58.047 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:58.040+0000 7f302f757640 1 -- start start 2023-11-26T07:18:58.048 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:58.040+0000 7f302f757640 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7f302806ec30 0x7f3028161010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:18:58.049 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:58.040+0000 7f302f757640 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7f3028161550 0x7f302806b4a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:18:58.049 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:58.040+0000 7f302f757640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f302806b9e0 0x7f30280cd880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:18:58.049 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:58.040+0000 7f302f757640 1 -- --> v2:172.21.15.143:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f302806be50 con 0x7f3028161550 2023-11-26T07:18:58.049 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:58.040+0000 7f302f757640 1 -- --> v2:172.21.15.188:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3028161ab0 con 0x7f302806ec30 2023-11-26T07:18:58.049 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:58.040+0000 7f302f757640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3028161d90 con 0x7f302806b9e0 2023-11-26T07:18:58.049 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:58.044+0000 7f302ccf2640 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7f3028161550 0x7f302806b4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:18:58.049 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:58.044+0000 7f302ccf2640 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7f3028161550 0x7f302806b4a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:51770/0 (socket says 172.21.15.143:51770) 2023-11-26T07:18:58.049 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:58.044+0000 7f302ccf2640 1 -- 172.21.15.143:0/2995109239 learned_addr learned my addr 172.21.15.143:0/2995109239 (peer_addr_for_me v2:172.21.15.143:0/0) 2023-11-26T07:18:58.050 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:58.044+0000 7f302d4f3640 1 --2- 172.21.15.143:0/2995109239 >> v2:172.21.15.188:3300/0 conn(0x7f302806ec30 0x7f3028161010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:18:58.050 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:58.044+0000 7f302dcf4640 1 --2- 172.21.15.143:0/2995109239 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f302806b9e0 0x7f30280cd880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:18:58.050 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:58.044+0000 7f302ccf2640 1 -- 172.21.15.143:0/2995109239 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f302806b9e0 msgr2=0x7f30280cd880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-26T07:18:58.050 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:58.044+0000 7f302ccf2640 1 --2- 172.21.15.143:0/2995109239 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f302806b9e0 0x7f30280cd880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:18:58.050 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:58.044+0000 7f302ccf2640 1 -- 172.21.15.143:0/2995109239 >> v2:172.21.15.188:3300/0 conn(0x7f302806ec30 msgr2=0x7f3028161010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-26T07:18:58.050 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:58.044+0000 7f302ccf2640 1 --2- 172.21.15.143:0/2995109239 >> v2:172.21.15.188:3300/0 conn(0x7f302806ec30 0x7f3028161010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:18:58.050 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:58.044+0000 7f302ccf2640 1 -- 172.21.15.143:0/2995109239 --> v2:172.21.15.143:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f30280ad820 con 0x7f3028161550 2023-11-26T07:18:58.050 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:58.044+0000 7f302d4f3640 1 --2- 172.21.15.143:0/2995109239 >> v2:172.21.15.188:3300/0 conn(0x7f302806ec30 0x7f3028161010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-26T07:18:58.051 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:58.044+0000 7f302dcf4640 1 --2- 172.21.15.143:0/2995109239 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f302806b9e0 0x7f30280cd880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-26T07:18:58.052 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:58.044+0000 7f302ccf2640 1 --2- 172.21.15.143:0/2995109239 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f3028161550 0x7f302806b4a0 secure :-1 s=READY pgs=421 cs=0 l=1 rev1=1 crypto rx=0x7f3028066340 tx=0x7f301801c180 comp rx=0 tx=0).ready entity=mon.0 client_cookie=24d59595f149a871 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:18:58.052 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:58.044+0000 7f301ffff640 1 -- 172.21.15.143:0/2995109239 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f30180ab600 con 0x7f3028161550 2023-11-26T07:18:58.052 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:58.044+0000 7f301ffff640 1 -- 172.21.15.143:0/2995109239 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f30180a7050 con 0x7f3028161550 2023-11-26T07:18:58.052 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:58.044+0000 7f301ffff640 1 -- 172.21.15.143:0/2995109239 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f30180ab980 con 0x7f3028161550 2023-11-26T07:18:58.053 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:58.048+0000 7f302f757640 1 -- 172.21.15.143:0/2995109239 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f3028161550 msgr2=0x7f302806b4a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:18:58.053 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:58.048+0000 7f302f757640 1 --2- 172.21.15.143:0/2995109239 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f3028161550 0x7f302806b4a0 secure :-1 s=READY pgs=421 cs=0 l=1 rev1=1 crypto rx=0x7f3028066340 tx=0x7f301801c180 comp rx=0 tx=0).stop 2023-11-26T07:18:58.053 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:58.048+0000 7f302f757640 1 -- 172.21.15.143:0/2995109239 shutdown_connections 2023-11-26T07:18:58.053 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:58.048+0000 7f302f757640 1 --2- 172.21.15.143:0/2995109239 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f302806b9e0 0x7f30280cd880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:18:58.053 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:58.048+0000 7f302f757640 1 --2- 172.21.15.143:0/2995109239 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f3028161550 0x7f302806b4a0 unknown :-1 s=CLOSED pgs=421 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:18:58.053 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:58.048+0000 7f302f757640 1 --2- 172.21.15.143:0/2995109239 >> v2:172.21.15.188:3300/0 conn(0x7f302806ec30 0x7f3028161010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:18:58.054 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:58.048+0000 7f302f757640 1 -- 172.21.15.143:0/2995109239 >> 172.21.15.143:0/2995109239 conn(0x7f3028070db0 msgr2=0x7f30280711c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:18:58.054 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:58.048+0000 7f302f757640 1 -- 172.21.15.143:0/2995109239 shutdown_connections 2023-11-26T07:18:58.054 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:58.048+0000 7f302f757640 1 -- 172.21.15.143:0/2995109239 wait complete. 2023-11-26T07:18:58.054 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:58.048+0000 7f302f757640 1 Processor -- start 2023-11-26T07:18:58.054 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:58.048+0000 7f302f757640 1 -- start start 2023-11-26T07:18:58.057 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:58.048+0000 7f302f757640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f302806b9e0 0x7f30280b2820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:18:58.057 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:58.048+0000 7f302f757640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f302806ec30 0x7f3028146aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:18:58.057 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:58.048+0000 7f302f757640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f3028161550 0x7f302814fcf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:18:58.057 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:58.048+0000 7f302f757640 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f30280b58f0 con 0x7f302806ec30 2023-11-26T07:18:58.057 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:58.048+0000 7f302f757640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f30280b2d60 con 0x7f302806b9e0 2023-11-26T07:18:58.057 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:58.048+0000 7f302f757640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3028147290 con 0x7f3028161550 2023-11-26T07:18:58.057 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:58.048+0000 7f302ccf2640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f302806ec30 0x7f3028146aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:18:58.057 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:58.048+0000 7f302ccf2640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f302806ec30 0x7f3028146aa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:51782/0 (socket says 172.21.15.143:51782) 2023-11-26T07:18:58.057 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:58.048+0000 7f302ccf2640 1 -- 172.21.15.143:0/2694401617 learned_addr learned my addr 172.21.15.143:0/2694401617 (peer_addr_for_me v2:172.21.15.143:0/0) 2023-11-26T07:18:58.058 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:58.048+0000 7f302d4f3640 1 --2- 172.21.15.143:0/2694401617 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f302806b9e0 0x7f30280b2820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:18:58.058 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:58.048+0000 7f302dcf4640 1 --2- 172.21.15.143:0/2694401617 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f3028161550 0x7f302814fcf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:18:58.058 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:58.048+0000 7f302ccf2640 1 -- 172.21.15.143:0/2694401617 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f3028161550 msgr2=0x7f302814fcf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:18:58.058 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:58.048+0000 7f302ccf2640 1 --2- 172.21.15.143:0/2694401617 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f3028161550 0x7f302814fcf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:18:58.058 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:58.048+0000 7f302ccf2640 1 -- 172.21.15.143:0/2694401617 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f302806b9e0 msgr2=0x7f30280b2820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:18:58.058 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:58.048+0000 7f302ccf2640 1 --2- 172.21.15.143:0/2694401617 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f302806b9e0 0x7f30280b2820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:18:58.058 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:58.048+0000 7f302ccf2640 1 -- 172.21.15.143:0/2694401617 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f30180094d0 con 0x7f302806ec30 2023-11-26T07:18:58.058 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:58.048+0000 7f302d4f3640 1 --2- 172.21.15.143:0/2694401617 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f302806b9e0 0x7f30280b2820 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-26T07:18:58.058 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:58.048+0000 7f302dcf4640 1 --2- 172.21.15.143:0/2694401617 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f3028161550 0x7f302814fcf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-26T07:18:58.058 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:58.052+0000 7f302ccf2640 1 --2- 172.21.15.143:0/2694401617 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f302806ec30 0x7f3028146aa0 secure :-1 s=READY pgs=422 cs=0 l=1 rev1=1 crypto rx=0x7f301801a830 tx=0x7f3018018f20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:18:58.058 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:58.052+0000 7f301dffb640 1 -- 172.21.15.143:0/2694401617 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f30180b4660 con 0x7f302806ec30 2023-11-26T07:18:58.058 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:58.052+0000 7f301dffb640 1 -- 172.21.15.143:0/2694401617 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f30180b47c0 con 0x7f302806ec30 2023-11-26T07:18:58.059 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:58.052+0000 7f302f757640 1 -- 172.21.15.143:0/2694401617 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3028146fe0 con 0x7f302806ec30 2023-11-26T07:18:58.059 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:58.052+0000 7f301dffb640 1 -- 172.21.15.143:0/2694401617 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f30180b4b70 con 0x7f302806ec30 2023-11-26T07:18:58.059 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:58.052+0000 7f302f757640 1 -- 172.21.15.143:0/2694401617 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f30280b3270 con 0x7f302806ec30 2023-11-26T07:18:58.059 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:58.052+0000 7f302f757640 1 -- 172.21.15.143:0/2694401617 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f30280676f0 con 0x7f302806ec30 2023-11-26T07:18:58.059 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:58.052+0000 7f301dffb640 1 -- 172.21.15.143:0/2694401617 <== mon.0 v2:172.21.15.143:3300/0 4 ==== mgrmap(e 5) v1 ==== 85852+0+0 (secure 0 0 0) 0x7f30180b2070 con 0x7f302806ec30 2023-11-26T07:18:58.059 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:58.052+0000 7f301dffb640 1 --2- 172.21.15.143:0/2694401617 >> [v2:172.21.15.143:6832/2150557544,v1:172.21.15.143:6833/2150557544] conn(0x7f300c066470 0x7f300c068930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:18:58.060 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:58.052+0000 7f301dffb640 1 -- 172.21.15.143:0/2694401617 <== mon.0 v2:172.21.15.143:3300/0 5 ==== osd_map(33..33 src has 1..33) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f301811e0f0 con 0x7f302806ec30 2023-11-26T07:18:58.060 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:58.052+0000 7f302d4f3640 1 --2- 172.21.15.143:0/2694401617 >> [v2:172.21.15.143:6832/2150557544,v1:172.21.15.143:6833/2150557544] conn(0x7f300c066470 0x7f300c068930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:18:58.060 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:58.056+0000 7f302d4f3640 1 --2- 172.21.15.143:0/2694401617 >> [v2:172.21.15.143:6832/2150557544,v1:172.21.15.143:6833/2150557544] conn(0x7f300c066470 0x7f300c068930 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7f3024008930 tx=0x7f3024008040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:18:58.063 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:58.056+0000 7f301dffb640 1 -- 172.21.15.143:0/2694401617 <== mon.0 v2:172.21.15.143:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172649 (secure 0 0 0) 0x7f30180eb4d0 con 0x7f302806ec30 2023-11-26T07:18:58.212 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:58.204+0000 7f302f757640 1 -- 172.21.15.143:0/2694401617 --> [v2:172.21.15.143:6832/2150557544,v1:172.21.15.143:6833/2150557544] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f302806f010 con 0x7f300c066470 2023-11-26T07:18:58.219 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:58.212+0000 7f301dffb640 1 -- 172.21.15.143:0/2694401617 <== mgr.4100 v2:172.21.15.143:6832/2150557544 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+289780 (secure 0 0 0) 0x7f302400c040 con 0x7f300c066470 2023-11-26T07:18:58.222 INFO:teuthology.orchestra.run.smithi143.stdout: 2023-11-26T07:18:58.224 INFO:teuthology.orchestra.run.smithi143.stderr:dumped all 2023-11-26T07:18:58.225 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:58.216+0000 7f302f757640 1 -- 172.21.15.143:0/2694401617 >> [v2:172.21.15.143:6832/2150557544,v1:172.21.15.143:6833/2150557544] conn(0x7f300c066470 msgr2=0x7f300c068930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:18:58.225 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:58.216+0000 7f302f757640 1 --2- 172.21.15.143:0/2694401617 >> [v2:172.21.15.143:6832/2150557544,v1:172.21.15.143:6833/2150557544] conn(0x7f300c066470 0x7f300c068930 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7f3024008930 tx=0x7f3024008040 comp rx=0 tx=0).stop 2023-11-26T07:18:58.225 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:58.216+0000 7f302f757640 1 -- 172.21.15.143:0/2694401617 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f302806ec30 msgr2=0x7f3028146aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:18:58.225 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:58.216+0000 7f302f757640 1 --2- 172.21.15.143:0/2694401617 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f302806ec30 0x7f3028146aa0 secure :-1 s=READY pgs=422 cs=0 l=1 rev1=1 crypto rx=0x7f301801a830 tx=0x7f3018018f20 comp rx=0 tx=0).stop 2023-11-26T07:18:58.226 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:58.216+0000 7f302f757640 1 -- 172.21.15.143:0/2694401617 shutdown_connections 2023-11-26T07:18:58.226 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:58.216+0000 7f302f757640 1 --2- 172.21.15.143:0/2694401617 >> [v2:172.21.15.143:6832/2150557544,v1:172.21.15.143:6833/2150557544] conn(0x7f300c066470 0x7f300c068930 unknown :-1 s=CLOSED pgs=226 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:18:58.226 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:58.216+0000 7f302f757640 1 --2- 172.21.15.143:0/2694401617 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f3028161550 0x7f302814fcf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:18:58.226 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:58.216+0000 7f302f757640 1 --2- 172.21.15.143:0/2694401617 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f302806ec30 0x7f3028146aa0 unknown :-1 s=CLOSED pgs=422 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:18:58.226 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:58.216+0000 7f302f757640 1 --2- 172.21.15.143:0/2694401617 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f302806b9e0 0x7f30280b2820 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:18:58.226 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:58.216+0000 7f302f757640 1 -- 172.21.15.143:0/2694401617 >> 172.21.15.143:0/2694401617 conn(0x7f3028070db0 msgr2=0x7f302815d4c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:18:58.226 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:58.216+0000 7f302f757640 1 -- 172.21.15.143:0/2694401617 shutdown_connections 2023-11-26T07:18:58.226 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:18:58.216+0000 7f302f757640 1 -- 172.21.15.143:0/2694401617 wait complete. 2023-11-26T07:18:58.243 INFO:teuthology.orchestra.run.smithi143.stdout:{"pg_ready":true,"pg_map":{"version":64,"stamp":"2023-11-26T07:18:57.705676+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592778,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":204,"num_write_kb":2430,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":156,"ondisk_log_size":156,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":161,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":220308,"kb_used_data":4136,"kb_used_omap":12,"kb_used_meta":216051,"kb_avail":754754412,"statfs":{"total":773094113280,"available":772868517888,"internally_reserved":0,"allocated":4235264,"data_stored":2358292,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12749,"internal_metadata":221236787},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":50,"apply_latency_ms":50,"commit_latency_ns":50000000,"apply_latency_ns":50000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"6.073556"},"pg_stats":[{"pgid":"5.3f","version":"0'0","reported_seq":11,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-11-26T07:18:54.776405+0000","last_change":"2023-11-26T07:18:52.685323+0000","last_active":"2023-11-26T07:18:54.776405+0000","last_peered":"2023-11-26T07:18:54.776405+0000","last_clean":"2023-11-26T07:18:54.776405+0000","last_became_active":"2023-11-26T07:18:52.685015+0000","last_became_peered":"2023-11-26T07:18:52.685015+0000","last_unstale":"2023-11-26T07:18:54.776405+0000","last_undegraded":"2023-11-26T07:18:54.776405+0000","last_fullsized":"2023-11-26T07:18:54.776405+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:18:51.625829+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:18:51.625829+0000","last_clean_scrub_stamp":"2023-11-26T07:18:51.625829+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T10:33:50.109555+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.3e","version":"0'0","reported_seq":11,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-11-26T07:18:54.777777+0000","last_change":"2023-11-26T07:18:52.661591+0000","last_active":"2023-11-26T07:18:54.777777+0000","last_peered":"2023-11-26T07:18:54.777777+0000","last_clean":"2023-11-26T07:18:54.777777+0000","last_became_active":"2023-11-26T07:18:52.661241+0000","last_became_peered":"2023-11-26T07:18:52.661241+0000","last_unstale":"2023-11-26T07:18:54.777777+0000","last_undegraded":"2023-11-26T07:18:54.777777+0000","last_fullsized":"2023-11-26T07:18:54.777777+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:18:51.625829+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:18:51.625829+0000","last_clean_scrub_stamp":"2023-11-26T07:18:51.625829+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T07:35:48.362218+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.3d","version":"0'0","reported_seq":11,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-11-26T07:18:54.776648+0000","last_change":"2023-11-26T07:18:52.674764+0000","last_active":"2023-11-26T07:18:54.776648+0000","last_peered":"2023-11-26T07:18:54.776648+0000","last_clean":"2023-11-26T07:18:54.776648+0000","last_became_active":"2023-11-26T07:18:52.674530+0000","last_became_peered":"2023-11-26T07:18:52.674530+0000","last_unstale":"2023-11-26T07:18:54.776648+0000","last_undegraded":"2023-11-26T07:18:54.776648+0000","last_fullsized":"2023-11-26T07:18:54.776648+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:18:51.625829+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:18:51.625829+0000","last_clean_scrub_stamp":"2023-11-26T07:18:51.625829+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T13:59:11.780648+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.3c","version":"0'0","reported_seq":11,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-11-26T07:18:53.665727+0000","last_change":"2023-11-26T07:18:52.685609+0000","last_active":"2023-11-26T07:18:53.665727+0000","last_peered":"2023-11-26T07:18:53.665727+0000","last_clean":"2023-11-26T07:18:53.665727+0000","last_became_active":"2023-11-26T07:18:52.685320+0000","last_became_peered":"2023-11-26T07:18:52.685320+0000","last_unstale":"2023-11-26T07:18:53.665727+0000","last_undegraded":"2023-11-26T07:18:53.665727+0000","last_fullsized":"2023-11-26T07:18:53.665727+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:18:51.625829+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:18:51.625829+0000","last_clean_scrub_stamp":"2023-11-26T07:18:51.625829+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T08:47:52.417322+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.3b","version":"0'0","reported_seq":11,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-11-26T07:18:54.776752+0000","last_change":"2023-11-26T07:18:52.666834+0000","last_active":"2023-11-26T07:18:54.776752+0000","last_peered":"2023-11-26T07:18:54.776752+0000","last_clean":"2023-11-26T07:18:54.776752+0000","last_became_active":"2023-11-26T07:18:52.666516+0000","last_became_peered":"2023-11-26T07:18:52.666516+0000","last_unstale":"2023-11-26T07:18:54.776752+0000","last_undegraded":"2023-11-26T07:18:54.776752+0000","last_fullsized":"2023-11-26T07:18:54.776752+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:18:51.625829+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:18:51.625829+0000","last_clean_scrub_stamp":"2023-11-26T07:18:51.625829+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T19:13:00.194342+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.3a","version":"0'0","reported_seq":11,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-11-26T07:18:54.777241+0000","last_change":"2023-11-26T07:18:52.669715+0000","last_active":"2023-11-26T07:18:54.777241+0000","last_peered":"2023-11-26T07:18:54.777241+0000","last_clean":"2023-11-26T07:18:54.777241+0000","last_became_active":"2023-11-26T07:18:52.669328+0000","last_became_peered":"2023-11-26T07:18:52.669328+0000","last_unstale":"2023-11-26T07:18:54.777241+0000","last_undegraded":"2023-11-26T07:18:54.777241+0000","last_fullsized":"2023-11-26T07:18:54.777241+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:18:51.625829+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:18:51.625829+0000","last_clean_scrub_stamp":"2023-11-26T07:18:51.625829+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T08:36:31.952917+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.39","version":"0'0","reported_seq":11,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-11-26T07:18:54.776755+0000","last_change":"2023-11-26T07:18:52.686894+0000","last_active":"2023-11-26T07:18:54.776755+0000","last_peered":"2023-11-26T07:18:54.776755+0000","last_clean":"2023-11-26T07:18:54.776755+0000","last_became_active":"2023-11-26T07:18:52.686521+0000","last_became_peered":"2023-11-26T07:18:52.686521+0000","last_unstale":"2023-11-26T07:18:54.776755+0000","last_undegraded":"2023-11-26T07:18:54.776755+0000","last_fullsized":"2023-11-26T07:18:54.776755+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:18:51.625829+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:18:51.625829+0000","last_clean_scrub_stamp":"2023-11-26T07:18:51.625829+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T12:03:00.136302+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.38","version":"0'0","reported_seq":11,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-11-26T07:18:54.777511+0000","last_change":"2023-11-26T07:18:52.690560+0000","last_active":"2023-11-26T07:18:54.777511+0000","last_peered":"2023-11-26T07:18:54.777511+0000","last_clean":"2023-11-26T07:18:54.777511+0000","last_became_active":"2023-11-26T07:18:52.690283+0000","last_became_peered":"2023-11-26T07:18:52.690283+0000","last_unstale":"2023-11-26T07:18:54.777511+0000","last_undegraded":"2023-11-26T07:18:54.777511+0000","last_fullsized":"2023-11-26T07:18:54.777511+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:18:51.625829+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:18:51.625829+0000","last_clean_scrub_stamp":"2023-11-26T07:18:51.625829+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T08:28:20.572561+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.37","version":"0'0","reported_seq":11,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-11-26T07:18:54.777644+0000","last_change":"2023-11-26T07:18:52.682711+0000","last_active":"2023-11-26T07:18:54.777644+0000","last_peered":"2023-11-26T07:18:54.777644+0000","last_clean":"2023-11-26T07:18:54.777644+0000","last_became_active":"2023-11-26T07:18:52.682295+0000","last_became_peered":"2023-11-26T07:18:52.682295+0000","last_unstale":"2023-11-26T07:18:54.777644+0000","last_undegraded":"2023-11-26T07:18:54.777644+0000","last_fullsized":"2023-11-26T07:18:54.777644+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:18:51.625829+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:18:51.625829+0000","last_clean_scrub_stamp":"2023-11-26T07:18:51.625829+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T09:38:45.595955+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.36","version":"0'0","reported_seq":11,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-11-26T07:18:53.666016+0000","last_change":"2023-11-26T07:18:52.681413+0000","last_active":"2023-11-26T07:18:53.666016+0000","last_peered":"2023-11-26T07:18:53.666016+0000","last_clean":"2023-11-26T07:18:53.666016+0000","last_became_active":"2023-11-26T07:18:52.680984+0000","last_became_peered":"2023-11-26T07:18:52.680984+0000","last_unstale":"2023-11-26T07:18:53.666016+0000","last_undegraded":"2023-11-26T07:18:53.666016+0000","last_fullsized":"2023-11-26T07:18:53.666016+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:18:51.625829+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:18:51.625829+0000","last_clean_scrub_stamp":"2023-11-26T07:18:51.625829+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T17:29:13.304698+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.35","version":"0'0","reported_seq":11,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-11-26T07:18:54.777930+0000","last_change":"2023-11-26T07:18:52.676152+0000","last_active":"2023-11-26T07:18:54.777930+0000","last_peered":"2023-11-26T07:18:54.777930+0000","last_clean":"2023-11-26T07:18:54.777930+0000","last_became_active":"2023-11-26T07:18:52.675857+0000","last_became_peered":"2023-11-26T07:18:52.675857+0000","last_unstale":"2023-11-26T07:18:54.777930+0000","last_undegraded":"2023-11-26T07:18:54.777930+0000","last_fullsized":"2023-11-26T07:18:54.777930+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:18:51.625829+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:18:51.625829+0000","last_clean_scrub_stamp":"2023-11-26T07:18:51.625829+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T12:10:13.607538+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.34","version":"0'0","reported_seq":11,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-11-26T07:18:54.777008+0000","last_change":"2023-11-26T07:18:52.689871+0000","last_active":"2023-11-26T07:18:54.777008+0000","last_peered":"2023-11-26T07:18:54.777008+0000","last_clean":"2023-11-26T07:18:54.777008+0000","last_became_active":"2023-11-26T07:18:52.689612+0000","last_became_peered":"2023-11-26T07:18:52.689612+0000","last_unstale":"2023-11-26T07:18:54.777008+0000","last_undegraded":"2023-11-26T07:18:54.777008+0000","last_fullsized":"2023-11-26T07:18:54.777008+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:18:51.625829+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:18:51.625829+0000","last_clean_scrub_stamp":"2023-11-26T07:18:51.625829+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T17:23:21.873210+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.33","version":"0'0","reported_seq":11,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-11-26T07:18:54.776822+0000","last_change":"2023-11-26T07:18:52.685975+0000","last_active":"2023-11-26T07:18:54.776822+0000","last_peered":"2023-11-26T07:18:54.776822+0000","last_clean":"2023-11-26T07:18:54.776822+0000","last_became_active":"2023-11-26T07:18:52.685643+0000","last_became_peered":"2023-11-26T07:18:52.685643+0000","last_unstale":"2023-11-26T07:18:54.776822+0000","last_undegraded":"2023-11-26T07:18:54.776822+0000","last_fullsized":"2023-11-26T07:18:54.776822+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:18:51.625829+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:18:51.625829+0000","last_clean_scrub_stamp":"2023-11-26T07:18:51.625829+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T13:37:16.389834+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.32","version":"0'0","reported_seq":11,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-11-26T07:18:54.777317+0000","last_change":"2023-11-26T07:18:52.665807+0000","last_active":"2023-11-26T07:18:54.777317+0000","last_peered":"2023-11-26T07:18:54.777317+0000","last_clean":"2023-11-26T07:18:54.777317+0000","last_became_active":"2023-11-26T07:18:52.665536+0000","last_became_peered":"2023-11-26T07:18:52.665536+0000","last_unstale":"2023-11-26T07:18:54.777317+0000","last_undegraded":"2023-11-26T07:18:54.777317+0000","last_fullsized":"2023-11-26T07:18:54.777317+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:18:51.625829+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:18:51.625829+0000","last_clean_scrub_stamp":"2023-11-26T07:18:51.625829+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T08:28:20.572561+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.31","version":"0'0","reported_seq":11,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-11-26T07:18:54.776584+0000","last_change":"2023-11-26T07:18:52.685374+0000","last_active":"2023-11-26T07:18:54.776584+0000","last_peered":"2023-11-26T07:18:54.776584+0000","last_clean":"2023-11-26T07:18:54.776584+0000","last_became_active":"2023-11-26T07:18:52.685003+0000","last_became_peered":"2023-11-26T07:18:52.685003+0000","last_unstale":"2023-11-26T07:18:54.776584+0000","last_undegraded":"2023-11-26T07:18:54.776584+0000","last_fullsized":"2023-11-26T07:18:54.776584+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:18:51.625829+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:18:51.625829+0000","last_clean_scrub_stamp":"2023-11-26T07:18:51.625829+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T14:18:07.188933+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.30","version":"0'0","reported_seq":11,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-11-26T07:18:54.777803+0000","last_change":"2023-11-26T07:18:52.682405+0000","last_active":"2023-11-26T07:18:54.777803+0000","last_peered":"2023-11-26T07:18:54.777803+0000","last_clean":"2023-11-26T07:18:54.777803+0000","last_became_active":"2023-11-26T07:18:52.682113+0000","last_became_peered":"2023-11-26T07:18:52.682113+0000","last_unstale":"2023-11-26T07:18:54.777803+0000","last_undegraded":"2023-11-26T07:18:54.777803+0000","last_fullsized":"2023-11-26T07:18:54.777803+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:18:51.625829+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:18:51.625829+0000","last_clean_scrub_stamp":"2023-11-26T07:18:51.625829+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T07:35:48.362218+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.2f","version":"0'0","reported_seq":11,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-11-26T07:18:54.778013+0000","last_change":"2023-11-26T07:18:52.688334+0000","last_active":"2023-11-26T07:18:54.778013+0000","last_peered":"2023-11-26T07:18:54.778013+0000","last_clean":"2023-11-26T07:18:54.778013+0000","last_became_active":"2023-11-26T07:18:52.688029+0000","last_became_peered":"2023-11-26T07:18:52.688029+0000","last_unstale":"2023-11-26T07:18:54.778013+0000","last_undegraded":"2023-11-26T07:18:54.778013+0000","last_fullsized":"2023-11-26T07:18:54.778013+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:18:51.625829+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:18:51.625829+0000","last_clean_scrub_stamp":"2023-11-26T07:18:51.625829+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T08:49:07.031743+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.2e","version":"0'0","reported_seq":11,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-11-26T07:18:54.777813+0000","last_change":"2023-11-26T07:18:52.680334+0000","last_active":"2023-11-26T07:18:54.777813+0000","last_peered":"2023-11-26T07:18:54.777813+0000","last_clean":"2023-11-26T07:18:54.777813+0000","last_became_active":"2023-11-26T07:18:52.679769+0000","last_became_peered":"2023-11-26T07:18:52.679769+0000","last_unstale":"2023-11-26T07:18:54.777813+0000","last_undegraded":"2023-11-26T07:18:54.777813+0000","last_fullsized":"2023-11-26T07:18:54.777813+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:18:51.625829+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:18:51.625829+0000","last_clean_scrub_stamp":"2023-11-26T07:18:51.625829+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T12:10:13.607538+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.2d","version":"0'0","reported_seq":11,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-11-26T07:18:54.777458+0000","last_change":"2023-11-26T07:18:52.680533+0000","last_active":"2023-11-26T07:18:54.777458+0000","last_peered":"2023-11-26T07:18:54.777458+0000","last_clean":"2023-11-26T07:18:54.777458+0000","last_became_active":"2023-11-26T07:18:52.680250+0000","last_became_peered":"2023-11-26T07:18:52.680250+0000","last_unstale":"2023-11-26T07:18:54.777458+0000","last_undegraded":"2023-11-26T07:18:54.777458+0000","last_fullsized":"2023-11-26T07:18:54.777458+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:18:51.625829+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:18:51.625829+0000","last_clean_scrub_stamp":"2023-11-26T07:18:51.625829+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T12:03:00.136302+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.2c","version":"0'0","reported_seq":11,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-11-26T07:18:54.777021+0000","last_change":"2023-11-26T07:18:52.679421+0000","last_active":"2023-11-26T07:18:54.777021+0000","last_peered":"2023-11-26T07:18:54.777021+0000","last_clean":"2023-11-26T07:18:54.777021+0000","last_became_active":"2023-11-26T07:18:52.677732+0000","last_became_peered":"2023-11-26T07:18:52.677732+0000","last_unstale":"2023-11-26T07:18:54.777021+0000","last_undegraded":"2023-11-26T07:18:54.777021+0000","last_fullsized":"2023-11-26T07:18:54.777021+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:18:51.625829+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:18:51.625829+0000","last_clean_scrub_stamp":"2023-11-26T07:18:51.625829+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T12:04:40.481383+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.2b","version":"0'0","reported_seq":11,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-11-26T07:18:54.777011+0000","last_change":"2023-11-26T07:18:52.666128+0000","last_active":"2023-11-26T07:18:54.777011+0000","last_peered":"2023-11-26T07:18:54.777011+0000","last_clean":"2023-11-26T07:18:54.777011+0000","last_became_active":"2023-11-26T07:18:52.665802+0000","last_became_peered":"2023-11-26T07:18:52.665802+0000","last_unstale":"2023-11-26T07:18:54.777011+0000","last_undegraded":"2023-11-26T07:18:54.777011+0000","last_fullsized":"2023-11-26T07:18:54.777011+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:18:51.625829+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:18:51.625829+0000","last_clean_scrub_stamp":"2023-11-26T07:18:51.625829+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T08:30:11.556027+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.2a","version":"0'0","reported_seq":11,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-11-26T07:18:54.777828+0000","last_change":"2023-11-26T07:18:52.672815+0000","last_active":"2023-11-26T07:18:54.777828+0000","last_peered":"2023-11-26T07:18:54.777828+0000","last_clean":"2023-11-26T07:18:54.777828+0000","last_became_active":"2023-11-26T07:18:52.667693+0000","last_became_peered":"2023-11-26T07:18:52.667693+0000","last_unstale":"2023-11-26T07:18:54.777828+0000","last_undegraded":"2023-11-26T07:18:54.777828+0000","last_fullsized":"2023-11-26T07:18:54.777828+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:18:51.625829+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:18:51.625829+0000","last_clean_scrub_stamp":"2023-11-26T07:18:51.625829+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T12:40:28.638833+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.29","version":"0'0","reported_seq":11,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-11-26T07:18:53.663848+0000","last_change":"2023-11-26T07:18:52.672298+0000","last_active":"2023-11-26T07:18:53.663848+0000","last_peered":"2023-11-26T07:18:53.663848+0000","last_clean":"2023-11-26T07:18:53.663848+0000","last_became_active":"2023-11-26T07:18:52.670876+0000","last_became_peered":"2023-11-26T07:18:52.670876+0000","last_unstale":"2023-11-26T07:18:53.663848+0000","last_undegraded":"2023-11-26T07:18:53.663848+0000","last_fullsized":"2023-11-26T07:18:53.663848+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:18:51.625829+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:18:51.625829+0000","last_clean_scrub_stamp":"2023-11-26T07:18:51.625829+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T13:18:27.001149+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.28","version":"0'0","reported_seq":11,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-11-26T07:18:54.775585+0000","last_change":"2023-11-26T07:18:52.691983+0000","last_active":"2023-11-26T07:18:54.775585+0000","last_peered":"2023-11-26T07:18:54.775585+0000","last_clean":"2023-11-26T07:18:54.775585+0000","last_became_active":"2023-11-26T07:18:52.691661+0000","last_became_peered":"2023-11-26T07:18:52.691661+0000","last_unstale":"2023-11-26T07:18:54.775585+0000","last_undegraded":"2023-11-26T07:18:54.775585+0000","last_fullsized":"2023-11-26T07:18:54.775585+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:18:51.625829+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:18:51.625829+0000","last_clean_scrub_stamp":"2023-11-26T07:18:51.625829+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T08:47:52.417322+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.27","version":"0'0","reported_seq":11,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-11-26T07:18:53.666090+0000","last_change":"2023-11-26T07:18:52.676572+0000","last_active":"2023-11-26T07:18:53.666090+0000","last_peered":"2023-11-26T07:18:53.666090+0000","last_clean":"2023-11-26T07:18:53.666090+0000","last_became_active":"2023-11-26T07:18:52.676288+0000","last_became_peered":"2023-11-26T07:18:52.676288+0000","last_unstale":"2023-11-26T07:18:53.666090+0000","last_undegraded":"2023-11-26T07:18:53.666090+0000","last_fullsized":"2023-11-26T07:18:53.666090+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:18:51.625829+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:18:51.625829+0000","last_clean_scrub_stamp":"2023-11-26T07:18:51.625829+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T14:18:07.188933+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.26","version":"0'0","reported_seq":11,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-11-26T07:18:54.777903+0000","last_change":"2023-11-26T07:18:52.665634+0000","last_active":"2023-11-26T07:18:54.777903+0000","last_peered":"2023-11-26T07:18:54.777903+0000","last_clean":"2023-11-26T07:18:54.777903+0000","last_became_active":"2023-11-26T07:18:52.665348+0000","last_became_peered":"2023-11-26T07:18:52.665348+0000","last_unstale":"2023-11-26T07:18:54.777903+0000","last_undegraded":"2023-11-26T07:18:54.777903+0000","last_fullsized":"2023-11-26T07:18:54.777903+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:18:51.625829+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:18:51.625829+0000","last_clean_scrub_stamp":"2023-11-26T07:18:51.625829+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T08:49:07.031743+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.25","version":"0'0","reported_seq":11,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-11-26T07:18:53.665947+0000","last_change":"2023-11-26T07:18:52.687999+0000","last_active":"2023-11-26T07:18:53.665947+0000","last_peered":"2023-11-26T07:18:53.665947+0000","last_clean":"2023-11-26T07:18:53.665947+0000","last_became_active":"2023-11-26T07:18:52.687720+0000","last_became_peered":"2023-11-26T07:18:52.687720+0000","last_unstale":"2023-11-26T07:18:53.665947+0000","last_undegraded":"2023-11-26T07:18:53.665947+0000","last_fullsized":"2023-11-26T07:18:53.665947+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:18:51.625829+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:18:51.625829+0000","last_clean_scrub_stamp":"2023-11-26T07:18:51.625829+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T09:49:42.889030+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.24","version":"0'0","reported_seq":11,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-11-26T07:18:54.777156+0000","last_change":"2023-11-26T07:18:52.675688+0000","last_active":"2023-11-26T07:18:54.777156+0000","last_peered":"2023-11-26T07:18:54.777156+0000","last_clean":"2023-11-26T07:18:54.777156+0000","last_became_active":"2023-11-26T07:18:52.675385+0000","last_became_peered":"2023-11-26T07:18:52.675385+0000","last_unstale":"2023-11-26T07:18:54.777156+0000","last_undegraded":"2023-11-26T07:18:54.777156+0000","last_fullsized":"2023-11-26T07:18:54.777156+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:18:51.625829+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:18:51.625829+0000","last_clean_scrub_stamp":"2023-11-26T07:18:51.625829+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T08:36:31.952917+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.23","version":"0'0","reported_seq":11,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-11-26T07:18:54.777176+0000","last_change":"2023-11-26T07:18:52.667885+0000","last_active":"2023-11-26T07:18:54.777176+0000","last_peered":"2023-11-26T07:18:54.777176+0000","last_clean":"2023-11-26T07:18:54.777176+0000","last_became_active":"2023-11-26T07:18:52.667553+0000","last_became_peered":"2023-11-26T07:18:52.667553+0000","last_unstale":"2023-11-26T07:18:54.777176+0000","last_undegraded":"2023-11-26T07:18:54.777176+0000","last_fullsized":"2023-11-26T07:18:54.777176+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:18:51.625829+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:18:51.625829+0000","last_clean_scrub_stamp":"2023-11-26T07:18:51.625829+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T08:44:22.910250+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.22","version":"0'0","reported_seq":11,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-11-26T07:18:54.776076+0000","last_change":"2023-11-26T07:18:52.686734+0000","last_active":"2023-11-26T07:18:54.776076+0000","last_peered":"2023-11-26T07:18:54.776076+0000","last_clean":"2023-11-26T07:18:54.776076+0000","last_became_active":"2023-11-26T07:18:52.686491+0000","last_became_peered":"2023-11-26T07:18:52.686491+0000","last_unstale":"2023-11-26T07:18:54.776076+0000","last_undegraded":"2023-11-26T07:18:54.776076+0000","last_fullsized":"2023-11-26T07:18:54.776076+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:18:51.625829+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:18:51.625829+0000","last_clean_scrub_stamp":"2023-11-26T07:18:51.625829+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T13:59:11.780648+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.21","version":"0'0","reported_seq":11,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-11-26T07:18:54.775987+0000","last_change":"2023-11-26T07:18:52.683056+0000","last_active":"2023-11-26T07:18:54.775987+0000","last_peered":"2023-11-26T07:18:54.775987+0000","last_clean":"2023-11-26T07:18:54.775987+0000","last_became_active":"2023-11-26T07:18:52.682753+0000","last_became_peered":"2023-11-26T07:18:52.682753+0000","last_unstale":"2023-11-26T07:18:54.775987+0000","last_undegraded":"2023-11-26T07:18:54.775987+0000","last_fullsized":"2023-11-26T07:18:54.775987+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:18:51.625829+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:18:51.625829+0000","last_clean_scrub_stamp":"2023-11-26T07:18:51.625829+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T16:41:02.104869+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.20","version":"0'0","reported_seq":11,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-11-26T07:18:54.777588+0000","last_change":"2023-11-26T07:18:52.681823+0000","last_active":"2023-11-26T07:18:54.777588+0000","last_peered":"2023-11-26T07:18:54.777588+0000","last_clean":"2023-11-26T07:18:54.777588+0000","last_became_active":"2023-11-26T07:18:52.681425+0000","last_became_peered":"2023-11-26T07:18:52.681425+0000","last_unstale":"2023-11-26T07:18:54.777588+0000","last_undegraded":"2023-11-26T07:18:54.777588+0000","last_fullsized":"2023-11-26T07:18:54.777588+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:18:51.625829+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:18:51.625829+0000","last_clean_scrub_stamp":"2023-11-26T07:18:51.625829+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T16:48:48.991856+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.1e","version":"0'0","reported_seq":11,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-11-26T07:18:53.664049+0000","last_change":"2023-11-26T07:18:52.669618+0000","last_active":"2023-11-26T07:18:53.664049+0000","last_peered":"2023-11-26T07:18:53.664049+0000","last_clean":"2023-11-26T07:18:53.664049+0000","last_became_active":"2023-11-26T07:18:52.667803+0000","last_became_peered":"2023-11-26T07:18:52.667803+0000","last_unstale":"2023-11-26T07:18:53.664049+0000","last_undegraded":"2023-11-26T07:18:53.664049+0000","last_fullsized":"2023-11-26T07:18:53.664049+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:18:51.625829+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:18:51.625829+0000","last_clean_scrub_stamp":"2023-11-26T07:18:51.625829+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T08:44:22.910250+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1f","version":"32'4","reported_seq":16,"reported_epoch":32,"state":"active+clean","last_fresh":"2023-11-26T07:18:52.692661+0000","last_change":"2023-11-26T07:18:51.661754+0000","last_active":"2023-11-26T07:18:52.692661+0000","last_peered":"2023-11-26T07:18:52.692661+0000","last_clean":"2023-11-26T07:18:52.692661+0000","last_became_active":"2023-11-26T07:18:51.661410+0000","last_became_peered":"2023-11-26T07:18:51.661410+0000","last_unstale":"2023-11-26T07:18:52.692661+0000","last_undegraded":"2023-11-26T07:18:52.692661+0000","last_fullsized":"2023-11-26T07:18:52.692661+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:18:50.617605+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:18:50.617605+0000","last_clean_scrub_stamp":"2023-11-26T07:18:50.617605+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T11:49:26.615849+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.1f","version":"0'0","reported_seq":11,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-11-26T07:18:54.776547+0000","last_change":"2023-11-26T07:18:52.675236+0000","last_active":"2023-11-26T07:18:54.776547+0000","last_peered":"2023-11-26T07:18:54.776547+0000","last_clean":"2023-11-26T07:18:54.776547+0000","last_became_active":"2023-11-26T07:18:52.674546+0000","last_became_peered":"2023-11-26T07:18:52.674546+0000","last_unstale":"2023-11-26T07:18:54.776547+0000","last_undegraded":"2023-11-26T07:18:54.776547+0000","last_fullsized":"2023-11-26T07:18:54.776547+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:18:51.625829+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:18:51.625829+0000","last_clean_scrub_stamp":"2023-11-26T07:18:51.625829+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T12:12:36.504349+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.1e","version":"32'1","reported_seq":13,"reported_epoch":32,"state":"active+clean","last_fresh":"2023-11-26T07:18:52.688044+0000","last_change":"2023-11-26T07:18:51.652999+0000","last_active":"2023-11-26T07:18:52.688044+0000","last_peered":"2023-11-26T07:18:52.688044+0000","last_clean":"2023-11-26T07:18:52.688044+0000","last_became_active":"2023-11-26T07:18:51.652268+0000","last_became_peered":"2023-11-26T07:18:51.652268+0000","last_unstale":"2023-11-26T07:18:52.688044+0000","last_undegraded":"2023-11-26T07:18:52.688044+0000","last_fullsized":"2023-11-26T07:18:52.688044+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:18:50.617605+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:18:50.617605+0000","last_clean_scrub_stamp":"2023-11-26T07:18:50.617605+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T10:17:26.493927+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.1c","version":"0'0","reported_seq":11,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-11-26T07:18:54.775954+0000","last_change":"2023-11-26T07:18:52.689076+0000","last_active":"2023-11-26T07:18:54.775954+0000","last_peered":"2023-11-26T07:18:54.775954+0000","last_clean":"2023-11-26T07:18:54.775954+0000","last_became_active":"2023-11-26T07:18:52.688877+0000","last_became_peered":"2023-11-26T07:18:52.688877+0000","last_unstale":"2023-11-26T07:18:54.775954+0000","last_undegraded":"2023-11-26T07:18:54.775954+0000","last_fullsized":"2023-11-26T07:18:54.775954+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:18:51.625829+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:18:51.625829+0000","last_clean_scrub_stamp":"2023-11-26T07:18:51.625829+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T13:47:40.261002+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.1d","version":"32'2","reported_seq":14,"reported_epoch":32,"state":"active+clean","last_fresh":"2023-11-26T07:18:52.689886+0000","last_change":"2023-11-26T07:18:51.653241+0000","last_active":"2023-11-26T07:18:52.689886+0000","last_peered":"2023-11-26T07:18:52.689886+0000","last_clean":"2023-11-26T07:18:52.689886+0000","last_became_active":"2023-11-26T07:18:51.652971+0000","last_became_peered":"2023-11-26T07:18:51.652971+0000","last_unstale":"2023-11-26T07:18:52.689886+0000","last_undegraded":"2023-11-26T07:18:52.689886+0000","last_fullsized":"2023-11-26T07:18:52.689886+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:18:50.617605+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:18:50.617605+0000","last_clean_scrub_stamp":"2023-11-26T07:18:50.617605+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T10:49:58.494062+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.1d","version":"0'0","reported_seq":11,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-11-26T07:18:53.665999+0000","last_change":"2023-11-26T07:18:52.687986+0000","last_active":"2023-11-26T07:18:53.665999+0000","last_peered":"2023-11-26T07:18:53.665999+0000","last_clean":"2023-11-26T07:18:53.665999+0000","last_became_active":"2023-11-26T07:18:52.687689+0000","last_became_peered":"2023-11-26T07:18:52.687689+0000","last_unstale":"2023-11-26T07:18:53.665999+0000","last_undegraded":"2023-11-26T07:18:53.665999+0000","last_fullsized":"2023-11-26T07:18:53.665999+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:18:51.625829+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:18:51.625829+0000","last_clean_scrub_stamp":"2023-11-26T07:18:51.625829+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T10:23:04.707081+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1c","version":"32'2","reported_seq":14,"reported_epoch":32,"state":"active+clean","last_fresh":"2023-11-26T07:18:52.690837+0000","last_change":"2023-11-26T07:18:51.662450+0000","last_active":"2023-11-26T07:18:52.690837+0000","last_peered":"2023-11-26T07:18:52.690837+0000","last_clean":"2023-11-26T07:18:52.690837+0000","last_became_active":"2023-11-26T07:18:51.662140+0000","last_became_peered":"2023-11-26T07:18:51.662140+0000","last_unstale":"2023-11-26T07:18:52.690837+0000","last_undegraded":"2023-11-26T07:18:52.690837+0000","last_fullsized":"2023-11-26T07:18:52.690837+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:18:50.617605+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:18:50.617605+0000","last_clean_scrub_stamp":"2023-11-26T07:18:50.617605+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T18:27:57.438972+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.1a","version":"0'0","reported_seq":11,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-11-26T07:18:54.776226+0000","last_change":"2023-11-26T07:18:52.684355+0000","last_active":"2023-11-26T07:18:54.776226+0000","last_peered":"2023-11-26T07:18:54.776226+0000","last_clean":"2023-11-26T07:18:54.776226+0000","last_became_active":"2023-11-26T07:18:52.683892+0000","last_became_peered":"2023-11-26T07:18:52.683892+0000","last_unstale":"2023-11-26T07:18:54.776226+0000","last_undegraded":"2023-11-26T07:18:54.776226+0000","last_fullsized":"2023-11-26T07:18:54.776226+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:18:51.625829+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:18:51.625829+0000","last_clean_scrub_stamp":"2023-11-26T07:18:51.625829+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T17:23:21.873210+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.1b","version":"32'1","reported_seq":13,"reported_epoch":32,"state":"active+clean","last_fresh":"2023-11-26T07:18:52.694034+0000","last_change":"2023-11-26T07:18:51.654689+0000","last_active":"2023-11-26T07:18:52.694034+0000","last_peered":"2023-11-26T07:18:52.694034+0000","last_clean":"2023-11-26T07:18:52.694034+0000","last_became_active":"2023-11-26T07:18:51.653354+0000","last_became_peered":"2023-11-26T07:18:51.653354+0000","last_unstale":"2023-11-26T07:18:52.694034+0000","last_undegraded":"2023-11-26T07:18:52.694034+0000","last_fullsized":"2023-11-26T07:18:52.694034+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:18:50.617605+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:18:50.617605+0000","last_clean_scrub_stamp":"2023-11-26T07:18:50.617605+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T11:49:26.615849+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.1b","version":"0'0","reported_seq":11,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-11-26T07:18:54.775959+0000","last_change":"2023-11-26T07:18:52.677308+0000","last_active":"2023-11-26T07:18:54.775959+0000","last_peered":"2023-11-26T07:18:54.775959+0000","last_clean":"2023-11-26T07:18:54.775959+0000","last_became_active":"2023-11-26T07:18:52.676288+0000","last_became_peered":"2023-11-26T07:18:52.676288+0000","last_unstale":"2023-11-26T07:18:54.775959+0000","last_undegraded":"2023-11-26T07:18:54.775959+0000","last_fullsized":"2023-11-26T07:18:54.775959+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:18:51.625829+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:18:51.625829+0000","last_clean_scrub_stamp":"2023-11-26T07:18:51.625829+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T10:49:59.502286+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1a","version":"32'1","reported_seq":13,"reported_epoch":32,"state":"active+clean","last_fresh":"2023-11-26T07:18:52.695040+0000","last_change":"2023-11-26T07:18:51.653682+0000","last_active":"2023-11-26T07:18:52.695040+0000","last_peered":"2023-11-26T07:18:52.695040+0000","last_clean":"2023-11-26T07:18:52.695040+0000","last_became_active":"2023-11-26T07:18:51.653366+0000","last_became_peered":"2023-11-26T07:18:51.653366+0000","last_unstale":"2023-11-26T07:18:52.695040+0000","last_undegraded":"2023-11-26T07:18:52.695040+0000","last_fullsized":"2023-11-26T07:18:52.695040+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:18:50.617605+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:18:50.617605+0000","last_clean_scrub_stamp":"2023-11-26T07:18:50.617605+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T18:26:18.407283+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.18","version":"0'0","reported_seq":11,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-11-26T07:18:54.775568+0000","last_change":"2023-11-26T07:18:52.684533+0000","last_active":"2023-11-26T07:18:54.775568+0000","last_peered":"2023-11-26T07:18:54.775568+0000","last_clean":"2023-11-26T07:18:54.775568+0000","last_became_active":"2023-11-26T07:18:52.684193+0000","last_became_peered":"2023-11-26T07:18:52.684193+0000","last_unstale":"2023-11-26T07:18:54.775568+0000","last_undegraded":"2023-11-26T07:18:54.775568+0000","last_fullsized":"2023-11-26T07:18:54.775568+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:18:51.625829+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:18:51.625829+0000","last_clean_scrub_stamp":"2023-11-26T07:18:51.625829+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T12:12:36.504349+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.19","version":"0'0","reported_seq":12,"reported_epoch":32,"state":"active+clean","last_fresh":"2023-11-26T07:18:52.649183+0000","last_change":"2023-11-26T07:18:51.657763+0000","last_active":"2023-11-26T07:18:52.649183+0000","last_peered":"2023-11-26T07:18:52.649183+0000","last_clean":"2023-11-26T07:18:52.649183+0000","last_became_active":"2023-11-26T07:18:51.656677+0000","last_became_peered":"2023-11-26T07:18:51.656677+0000","last_unstale":"2023-11-26T07:18:52.649183+0000","last_undegraded":"2023-11-26T07:18:52.649183+0000","last_fullsized":"2023-11-26T07:18:52.649183+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:18:50.617605+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:18:50.617605+0000","last_clean_scrub_stamp":"2023-11-26T07:18:50.617605+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T16:41:01.096645+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.19","version":"0'0","reported_seq":11,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-11-26T07:18:53.665328+0000","last_change":"2023-11-26T07:18:52.676586+0000","last_active":"2023-11-26T07:18:53.665328+0000","last_peered":"2023-11-26T07:18:53.665328+0000","last_clean":"2023-11-26T07:18:53.665328+0000","last_became_active":"2023-11-26T07:18:52.676328+0000","last_became_peered":"2023-11-26T07:18:52.676328+0000","last_unstale":"2023-11-26T07:18:53.665328+0000","last_undegraded":"2023-11-26T07:18:53.665328+0000","last_fullsized":"2023-11-26T07:18:53.665328+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:18:51.625829+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:18:51.625829+0000","last_clean_scrub_stamp":"2023-11-26T07:18:51.625829+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T12:40:28.638833+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.18","version":"0'0","reported_seq":12,"reported_epoch":32,"state":"active+clean","last_fresh":"2023-11-26T07:18:52.653501+0000","last_change":"2023-11-26T07:18:51.656204+0000","last_active":"2023-11-26T07:18:52.653501+0000","last_peered":"2023-11-26T07:18:52.653501+0000","last_clean":"2023-11-26T07:18:52.653501+0000","last_became_active":"2023-11-26T07:18:51.655809+0000","last_became_peered":"2023-11-26T07:18:51.655809+0000","last_unstale":"2023-11-26T07:18:52.653501+0000","last_undegraded":"2023-11-26T07:18:52.653501+0000","last_fullsized":"2023-11-26T07:18:52.653501+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:18:50.617605+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:18:50.617605+0000","last_clean_scrub_stamp":"2023-11-26T07:18:50.617605+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T07:50:54.298421+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.16","version":"0'0","reported_seq":11,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-11-26T07:18:54.777323+0000","last_change":"2023-11-26T07:18:52.690469+0000","last_active":"2023-11-26T07:18:54.777323+0000","last_peered":"2023-11-26T07:18:54.777323+0000","last_clean":"2023-11-26T07:18:54.777323+0000","last_became_active":"2023-11-26T07:18:52.690199+0000","last_became_peered":"2023-11-26T07:18:52.690199+0000","last_unstale":"2023-11-26T07:18:54.777323+0000","last_undegraded":"2023-11-26T07:18:54.777323+0000","last_fullsized":"2023-11-26T07:18:54.777323+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:18:51.625829+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:18:51.625829+0000","last_clean_scrub_stamp":"2023-11-26T07:18:51.625829+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T12:03:00.136302+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.17","version":"0'0","reported_seq":12,"reported_epoch":32,"state":"active+clean","last_fresh":"2023-11-26T07:18:52.651648+0000","last_change":"2023-11-26T07:18:51.653322+0000","last_active":"2023-11-26T07:18:52.651648+0000","last_peered":"2023-11-26T07:18:52.651648+0000","last_clean":"2023-11-26T07:18:52.651648+0000","last_became_active":"2023-11-26T07:18:51.652388+0000","last_became_peered":"2023-11-26T07:18:51.652388+0000","last_unstale":"2023-11-26T07:18:52.651648+0000","last_undegraded":"2023-11-26T07:18:52.651648+0000","last_fullsized":"2023-11-26T07:18:52.651648+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:18:50.617605+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:18:50.617605+0000","last_clean_scrub_stamp":"2023-11-26T07:18:50.617605+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T13:59:10.772424+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.17","version":"0'0","reported_seq":11,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-11-26T07:18:54.777705+0000","last_change":"2023-11-26T07:18:52.680505+0000","last_active":"2023-11-26T07:18:54.777705+0000","last_peered":"2023-11-26T07:18:54.777705+0000","last_clean":"2023-11-26T07:18:54.777705+0000","last_became_active":"2023-11-26T07:18:52.680214+0000","last_became_peered":"2023-11-26T07:18:52.680214+0000","last_unstale":"2023-11-26T07:18:54.777705+0000","last_undegraded":"2023-11-26T07:18:54.777705+0000","last_fullsized":"2023-11-26T07:18:54.777705+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:18:51.625829+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:18:51.625829+0000","last_clean_scrub_stamp":"2023-11-26T07:18:51.625829+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T13:50:40.217037+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.16","version":"0'0","reported_seq":12,"reported_epoch":32,"state":"active+clean","last_fresh":"2023-11-26T07:18:52.655213+0000","last_change":"2023-11-26T07:18:51.644845+0000","last_active":"2023-11-26T07:18:52.655213+0000","last_peered":"2023-11-26T07:18:52.655213+0000","last_clean":"2023-11-26T07:18:52.655213+0000","last_became_active":"2023-11-26T07:18:51.644261+0000","last_became_peered":"2023-11-26T07:18:51.644261+0000","last_unstale":"2023-11-26T07:18:52.655213+0000","last_undegraded":"2023-11-26T07:18:52.655213+0000","last_fullsized":"2023-11-26T07:18:52.655213+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:18:50.617605+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:18:50.617605+0000","last_clean_scrub_stamp":"2023-11-26T07:18:50.617605+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T10:33:49.101331+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.14","version":"0'0","reported_seq":11,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-11-26T07:18:54.777560+0000","last_change":"2023-11-26T07:18:52.669834+0000","last_active":"2023-11-26T07:18:54.777560+0000","last_peered":"2023-11-26T07:18:54.777560+0000","last_clean":"2023-11-26T07:18:54.777560+0000","last_became_active":"2023-11-26T07:18:52.669564+0000","last_became_peered":"2023-11-26T07:18:52.669564+0000","last_unstale":"2023-11-26T07:18:54.777560+0000","last_undegraded":"2023-11-26T07:18:54.777560+0000","last_fullsized":"2023-11-26T07:18:54.777560+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:18:51.625829+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:18:51.625829+0000","last_clean_scrub_stamp":"2023-11-26T07:18:51.625829+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T16:48:48.991856+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.15","version":"32'3","reported_seq":15,"reported_epoch":32,"state":"active+clean","last_fresh":"2023-11-26T07:18:52.682254+0000","last_change":"2023-11-26T07:18:51.657428+0000","last_active":"2023-11-26T07:18:52.682254+0000","last_peered":"2023-11-26T07:18:52.682254+0000","last_clean":"2023-11-26T07:18:52.682254+0000","last_became_active":"2023-11-26T07:18:51.657075+0000","last_became_peered":"2023-11-26T07:18:51.657075+0000","last_unstale":"2023-11-26T07:18:52.682254+0000","last_undegraded":"2023-11-26T07:18:52.682254+0000","last_fullsized":"2023-11-26T07:18:52.682254+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:18:50.617605+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:18:50.617605+0000","last_clean_scrub_stamp":"2023-11-26T07:18:50.617605+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T13:47:39.252778+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.15","version":"0'0","reported_seq":11,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-11-26T07:18:54.776501+0000","last_change":"2023-11-26T07:18:52.677313+0000","last_active":"2023-11-26T07:18:54.776501+0000","last_peered":"2023-11-26T07:18:54.776501+0000","last_clean":"2023-11-26T07:18:54.776501+0000","last_became_active":"2023-11-26T07:18:52.676992+0000","last_became_peered":"2023-11-26T07:18:52.676992+0000","last_unstale":"2023-11-26T07:18:54.776501+0000","last_undegraded":"2023-11-26T07:18:54.776501+0000","last_fullsized":"2023-11-26T07:18:54.776501+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:18:51.625829+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:18:51.625829+0000","last_clean_scrub_stamp":"2023-11-26T07:18:51.625829+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T17:29:13.304698+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.14","version":"32'6","reported_seq":18,"reported_epoch":32,"state":"active+clean","last_fresh":"2023-11-26T07:18:52.698836+0000","last_change":"2023-11-26T07:18:51.647209+0000","last_active":"2023-11-26T07:18:52.698836+0000","last_peered":"2023-11-26T07:18:52.698836+0000","last_clean":"2023-11-26T07:18:52.698836+0000","last_became_active":"2023-11-26T07:18:51.647051+0000","last_became_peered":"2023-11-26T07:18:51.647051+0000","last_unstale":"2023-11-26T07:18:52.698836+0000","last_undegraded":"2023-11-26T07:18:52.698836+0000","last_fullsized":"2023-11-26T07:18:52.698836+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:18:50.617605+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:18:50.617605+0000","last_clean_scrub_stamp":"2023-11-26T07:18:50.617605+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T12:40:27.630609+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.12","version":"0'0","reported_seq":11,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-11-26T07:18:53.665370+0000","last_change":"2023-11-26T07:18:52.677367+0000","last_active":"2023-11-26T07:18:53.665370+0000","last_peered":"2023-11-26T07:18:53.665370+0000","last_clean":"2023-11-26T07:18:53.665370+0000","last_became_active":"2023-11-26T07:18:52.677098+0000","last_became_peered":"2023-11-26T07:18:52.677098+0000","last_unstale":"2023-11-26T07:18:53.665370+0000","last_undegraded":"2023-11-26T07:18:53.665370+0000","last_fullsized":"2023-11-26T07:18:53.665370+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:18:51.625829+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:18:51.625829+0000","last_clean_scrub_stamp":"2023-11-26T07:18:51.625829+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T13:47:40.261002+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.13","version":"32'2","reported_seq":14,"reported_epoch":32,"state":"active+clean","last_fresh":"2023-11-26T07:18:52.691338+0000","last_change":"2023-11-26T07:18:51.656339+0000","last_active":"2023-11-26T07:18:52.691338+0000","last_peered":"2023-11-26T07:18:52.691338+0000","last_clean":"2023-11-26T07:18:52.691338+0000","last_became_active":"2023-11-26T07:18:51.655998+0000","last_became_peered":"2023-11-26T07:18:51.655998+0000","last_unstale":"2023-11-26T07:18:52.691338+0000","last_undegraded":"2023-11-26T07:18:52.691338+0000","last_fullsized":"2023-11-26T07:18:52.691338+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:18:50.617605+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:18:50.617605+0000","last_clean_scrub_stamp":"2023-11-26T07:18:50.617605+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T16:41:01.096645+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.13","version":"0'0","reported_seq":11,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-11-26T07:18:54.777486+0000","last_change":"2023-11-26T07:18:52.661410+0000","last_active":"2023-11-26T07:18:54.777486+0000","last_peered":"2023-11-26T07:18:54.777486+0000","last_clean":"2023-11-26T07:18:54.777486+0000","last_became_active":"2023-11-26T07:18:52.661149+0000","last_became_peered":"2023-11-26T07:18:52.661149+0000","last_unstale":"2023-11-26T07:18:54.777486+0000","last_undegraded":"2023-11-26T07:18:54.777486+0000","last_fullsized":"2023-11-26T07:18:54.777486+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:18:51.625829+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:18:51.625829+0000","last_clean_scrub_stamp":"2023-11-26T07:18:51.625829+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T15:42:06.512477+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.12","version":"0'0","reported_seq":12,"reported_epoch":32,"state":"active+clean","last_fresh":"2023-11-26T07:18:52.671303+0000","last_change":"2023-11-26T07:18:51.658016+0000","last_active":"2023-11-26T07:18:52.671303+0000","last_peered":"2023-11-26T07:18:52.671303+0000","last_clean":"2023-11-26T07:18:52.671303+0000","last_became_active":"2023-11-26T07:18:51.657646+0000","last_became_peered":"2023-11-26T07:18:51.657646+0000","last_unstale":"2023-11-26T07:18:52.671303+0000","last_undegraded":"2023-11-26T07:18:52.671303+0000","last_fullsized":"2023-11-26T07:18:52.671303+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:18:50.617605+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:18:50.617605+0000","last_clean_scrub_stamp":"2023-11-26T07:18:50.617605+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T12:12:35.496125+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.10","version":"0'0","reported_seq":11,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-11-26T07:18:54.777611+0000","last_change":"2023-11-26T07:18:52.672193+0000","last_active":"2023-11-26T07:18:54.777611+0000","last_peered":"2023-11-26T07:18:54.777611+0000","last_clean":"2023-11-26T07:18:54.777611+0000","last_became_active":"2023-11-26T07:18:52.671857+0000","last_became_peered":"2023-11-26T07:18:52.671857+0000","last_unstale":"2023-11-26T07:18:54.777611+0000","last_undegraded":"2023-11-26T07:18:54.777611+0000","last_fullsized":"2023-11-26T07:18:54.777611+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:18:51.625829+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:18:51.625829+0000","last_clean_scrub_stamp":"2023-11-26T07:18:51.625829+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T13:50:40.217037+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.11","version":"0'0","reported_seq":12,"reported_epoch":32,"state":"active+clean","last_fresh":"2023-11-26T07:18:52.649437+0000","last_change":"2023-11-26T07:18:51.654685+0000","last_active":"2023-11-26T07:18:52.649437+0000","last_peered":"2023-11-26T07:18:52.649437+0000","last_clean":"2023-11-26T07:18:52.649437+0000","last_became_active":"2023-11-26T07:18:51.653542+0000","last_became_peered":"2023-11-26T07:18:51.653542+0000","last_unstale":"2023-11-26T07:18:52.649437+0000","last_undegraded":"2023-11-26T07:18:52.649437+0000","last_fullsized":"2023-11-26T07:18:52.649437+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:18:50.617605+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:18:50.617605+0000","last_clean_scrub_stamp":"2023-11-26T07:18:50.617605+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T12:12:35.496125+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.11","version":"0'0","reported_seq":11,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-11-26T07:18:54.776160+0000","last_change":"2023-11-26T07:18:52.673902+0000","last_active":"2023-11-26T07:18:54.776160+0000","last_peered":"2023-11-26T07:18:54.776160+0000","last_clean":"2023-11-26T07:18:54.776160+0000","last_became_active":"2023-11-26T07:18:52.673049+0000","last_became_peered":"2023-11-26T07:18:52.673049+0000","last_unstale":"2023-11-26T07:18:54.776160+0000","last_undegraded":"2023-11-26T07:18:54.776160+0000","last_fullsized":"2023-11-26T07:18:54.776160+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:18:51.625829+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:18:51.625829+0000","last_clean_scrub_stamp":"2023-11-26T07:18:51.625829+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T10:17:27.502151+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.10","version":"32'8","reported_seq":20,"reported_epoch":32,"state":"active+clean","last_fresh":"2023-11-26T07:18:52.693910+0000","last_change":"2023-11-26T07:18:51.652858+0000","last_active":"2023-11-26T07:18:52.693910+0000","last_peered":"2023-11-26T07:18:52.693910+0000","last_clean":"2023-11-26T07:18:52.693910+0000","last_became_active":"2023-11-26T07:18:51.652583+0000","last_became_peered":"2023-11-26T07:18:51.652583+0000","last_unstale":"2023-11-26T07:18:52.693910+0000","last_undegraded":"2023-11-26T07:18:52.693910+0000","last_fullsized":"2023-11-26T07:18:52.693910+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:18:50.617605+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:18:50.617605+0000","last_clean_scrub_stamp":"2023-11-26T07:18:50.617605+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T10:33:49.101331+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.e","version":"0'0","reported_seq":11,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-11-26T07:18:54.776989+0000","last_change":"2023-11-26T07:18:52.680594+0000","last_active":"2023-11-26T07:18:54.776989+0000","last_peered":"2023-11-26T07:18:54.776989+0000","last_clean":"2023-11-26T07:18:54.776989+0000","last_became_active":"2023-11-26T07:18:52.673803+0000","last_became_peered":"2023-11-26T07:18:52.673803+0000","last_unstale":"2023-11-26T07:18:54.776989+0000","last_undegraded":"2023-11-26T07:18:54.776989+0000","last_fullsized":"2023-11-26T07:18:54.776989+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:18:51.625829+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:18:51.625829+0000","last_clean_scrub_stamp":"2023-11-26T07:18:51.625829+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T13:18:27.001149+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.f","version":"32'1","reported_seq":13,"reported_epoch":32,"state":"active+clean","last_fresh":"2023-11-26T07:18:52.687288+0000","last_change":"2023-11-26T07:18:51.656212+0000","last_active":"2023-11-26T07:18:52.687288+0000","last_peered":"2023-11-26T07:18:52.687288+0000","last_clean":"2023-11-26T07:18:52.687288+0000","last_became_active":"2023-11-26T07:18:51.650720+0000","last_became_peered":"2023-11-26T07:18:51.650720+0000","last_unstale":"2023-11-26T07:18:52.687288+0000","last_undegraded":"2023-11-26T07:18:52.687288+0000","last_fullsized":"2023-11-26T07:18:52.687288+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:18:50.617605+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:18:50.617605+0000","last_clean_scrub_stamp":"2023-11-26T07:18:50.617605+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T08:44:21.902026+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.f","version":"0'0","reported_seq":11,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-11-26T07:18:54.776149+0000","last_change":"2023-11-26T07:18:52.671950+0000","last_active":"2023-11-26T07:18:54.776149+0000","last_peered":"2023-11-26T07:18:54.776149+0000","last_clean":"2023-11-26T07:18:54.776149+0000","last_became_active":"2023-11-26T07:18:52.671645+0000","last_became_peered":"2023-11-26T07:18:52.671645+0000","last_unstale":"2023-11-26T07:18:54.776149+0000","last_undegraded":"2023-11-26T07:18:54.776149+0000","last_fullsized":"2023-11-26T07:18:54.776149+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:18:51.625829+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:18:51.625829+0000","last_clean_scrub_stamp":"2023-11-26T07:18:51.625829+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T09:49:42.889030+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.e","version":"32'2","reported_seq":14,"reported_epoch":32,"state":"active+clean","last_fresh":"2023-11-26T07:18:52.688955+0000","last_change":"2023-11-26T07:18:51.660325+0000","last_active":"2023-11-26T07:18:52.688955+0000","last_peered":"2023-11-26T07:18:52.688955+0000","last_clean":"2023-11-26T07:18:52.688955+0000","last_became_active":"2023-11-26T07:18:51.659918+0000","last_became_peered":"2023-11-26T07:18:51.659918+0000","last_unstale":"2023-11-26T07:18:52.688955+0000","last_undegraded":"2023-11-26T07:18:52.688955+0000","last_fullsized":"2023-11-26T07:18:52.688955+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:18:50.617605+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:18:50.617605+0000","last_clean_scrub_stamp":"2023-11-26T07:18:50.617605+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T10:17:26.493927+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.c","version":"0'0","reported_seq":11,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-11-26T07:18:53.665748+0000","last_change":"2023-11-26T07:18:52.685610+0000","last_active":"2023-11-26T07:18:53.665748+0000","last_peered":"2023-11-26T07:18:53.665748+0000","last_clean":"2023-11-26T07:18:53.665748+0000","last_became_active":"2023-11-26T07:18:52.685310+0000","last_became_peered":"2023-11-26T07:18:52.685310+0000","last_unstale":"2023-11-26T07:18:53.665748+0000","last_undegraded":"2023-11-26T07:18:53.665748+0000","last_fullsized":"2023-11-26T07:18:53.665748+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:18:51.625829+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:18:51.625829+0000","last_clean_scrub_stamp":"2023-11-26T07:18:51.625829+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T10:49:59.502286+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.d","version":"32'1","reported_seq":13,"reported_epoch":32,"state":"active+clean","last_fresh":"2023-11-26T07:18:52.691763+0000","last_change":"2023-11-26T07:18:51.658440+0000","last_active":"2023-11-26T07:18:52.691763+0000","last_peered":"2023-11-26T07:18:52.691763+0000","last_clean":"2023-11-26T07:18:52.691763+0000","last_became_active":"2023-11-26T07:18:51.658161+0000","last_became_peered":"2023-11-26T07:18:51.658161+0000","last_unstale":"2023-11-26T07:18:52.691763+0000","last_undegraded":"2023-11-26T07:18:52.691763+0000","last_fullsized":"2023-11-26T07:18:52.691763+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:18:50.617605+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:18:50.617605+0000","last_clean_scrub_stamp":"2023-11-26T07:18:50.617605+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T14:17:17.432332+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.d","version":"0'0","reported_seq":11,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-11-26T07:18:54.777019+0000","last_change":"2023-11-26T07:18:52.674278+0000","last_active":"2023-11-26T07:18:54.777019+0000","last_peered":"2023-11-26T07:18:54.777019+0000","last_clean":"2023-11-26T07:18:54.777019+0000","last_became_active":"2023-11-26T07:18:52.674012+0000","last_became_peered":"2023-11-26T07:18:52.674012+0000","last_unstale":"2023-11-26T07:18:54.777019+0000","last_undegraded":"2023-11-26T07:18:54.777019+0000","last_fullsized":"2023-11-26T07:18:54.777019+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:18:51.625829+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:18:51.625829+0000","last_clean_scrub_stamp":"2023-11-26T07:18:51.625829+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T19:10:40.245493+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.c","version":"0'0","reported_seq":12,"reported_epoch":32,"state":"active+clean","last_fresh":"2023-11-26T07:18:52.652890+0000","last_change":"2023-11-26T07:18:51.654164+0000","last_active":"2023-11-26T07:18:52.652890+0000","last_peered":"2023-11-26T07:18:52.652890+0000","last_clean":"2023-11-26T07:18:52.652890+0000","last_became_active":"2023-11-26T07:18:51.653823+0000","last_became_peered":"2023-11-26T07:18:51.653823+0000","last_unstale":"2023-11-26T07:18:52.652890+0000","last_undegraded":"2023-11-26T07:18:52.652890+0000","last_fullsized":"2023-11-26T07:18:52.652890+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:18:50.617605+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:18:50.617605+0000","last_clean_scrub_stamp":"2023-11-26T07:18:50.617605+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T17:37:35.256892+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.a","version":"0'0","reported_seq":11,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-11-26T07:18:54.776991+0000","last_change":"2023-11-26T07:18:52.667222+0000","last_active":"2023-11-26T07:18:54.776991+0000","last_peered":"2023-11-26T07:18:54.776991+0000","last_clean":"2023-11-26T07:18:54.776991+0000","last_became_active":"2023-11-26T07:18:52.665846+0000","last_became_peered":"2023-11-26T07:18:52.665846+0000","last_unstale":"2023-11-26T07:18:54.776991+0000","last_undegraded":"2023-11-26T07:18:54.776991+0000","last_fullsized":"2023-11-26T07:18:54.776991+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:18:51.625829+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:18:51.625829+0000","last_clean_scrub_stamp":"2023-11-26T07:18:51.625829+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T17:37:36.265116+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.b","version":"32'2","reported_seq":14,"reported_epoch":32,"state":"active+clean","last_fresh":"2023-11-26T07:18:52.683434+0000","last_change":"2023-11-26T07:18:51.646694+0000","last_active":"2023-11-26T07:18:52.683434+0000","last_peered":"2023-11-26T07:18:52.683434+0000","last_clean":"2023-11-26T07:18:52.683434+0000","last_became_active":"2023-11-26T07:18:51.644567+0000","last_became_peered":"2023-11-26T07:18:51.644567+0000","last_unstale":"2023-11-26T07:18:52.683434+0000","last_undegraded":"2023-11-26T07:18:52.683434+0000","last_fullsized":"2023-11-26T07:18:52.683434+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:18:50.617605+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:18:50.617605+0000","last_clean_scrub_stamp":"2023-11-26T07:18:50.617605+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T11:49:26.615849+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.b","version":"0'0","reported_seq":11,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-11-26T07:18:54.776902+0000","last_change":"2023-11-26T07:18:52.670716+0000","last_active":"2023-11-26T07:18:54.776902+0000","last_peered":"2023-11-26T07:18:54.776902+0000","last_clean":"2023-11-26T07:18:54.776902+0000","last_became_active":"2023-11-26T07:18:52.670370+0000","last_became_peered":"2023-11-26T07:18:52.670370+0000","last_unstale":"2023-11-26T07:18:54.776902+0000","last_undegraded":"2023-11-26T07:18:54.776902+0000","last_fullsized":"2023-11-26T07:18:54.776902+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:18:51.625829+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:18:51.625829+0000","last_clean_scrub_stamp":"2023-11-26T07:18:51.625829+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T17:22:39.981484+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.a","version":"0'0","reported_seq":12,"reported_epoch":32,"state":"active+clean","last_fresh":"2023-11-26T07:18:52.658364+0000","last_change":"2023-11-26T07:18:51.660988+0000","last_active":"2023-11-26T07:18:52.658364+0000","last_peered":"2023-11-26T07:18:52.658364+0000","last_clean":"2023-11-26T07:18:52.658364+0000","last_became_active":"2023-11-26T07:18:51.660676+0000","last_became_peered":"2023-11-26T07:18:51.660676+0000","last_unstale":"2023-11-26T07:18:52.658364+0000","last_undegraded":"2023-11-26T07:18:52.658364+0000","last_fullsized":"2023-11-26T07:18:52.658364+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:18:50.617605+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:18:50.617605+0000","last_clean_scrub_stamp":"2023-11-26T07:18:50.617605+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T10:40:39.059176+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.8","version":"0'0","reported_seq":11,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-11-26T07:18:54.776593+0000","last_change":"2023-11-26T07:18:52.670731+0000","last_active":"2023-11-26T07:18:54.776593+0000","last_peered":"2023-11-26T07:18:54.776593+0000","last_clean":"2023-11-26T07:18:54.776593+0000","last_became_active":"2023-11-26T07:18:52.670500+0000","last_became_peered":"2023-11-26T07:18:52.670500+0000","last_unstale":"2023-11-26T07:18:54.776593+0000","last_undegraded":"2023-11-26T07:18:54.776593+0000","last_fullsized":"2023-11-26T07:18:54.776593+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:18:51.625829+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:18:51.625829+0000","last_clean_scrub_stamp":"2023-11-26T07:18:51.625829+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T10:54:58.699396+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.9","version":"0'0","reported_seq":12,"reported_epoch":32,"state":"active+clean","last_fresh":"2023-11-26T07:18:52.653043+0000","last_change":"2023-11-26T07:18:51.652977+0000","last_active":"2023-11-26T07:18:52.653043+0000","last_peered":"2023-11-26T07:18:52.653043+0000","last_clean":"2023-11-26T07:18:52.653043+0000","last_became_active":"2023-11-26T07:18:51.652670+0000","last_became_peered":"2023-11-26T07:18:51.652670+0000","last_unstale":"2023-11-26T07:18:52.653043+0000","last_undegraded":"2023-11-26T07:18:52.653043+0000","last_fullsized":"2023-11-26T07:18:52.653043+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:18:50.617605+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:18:50.617605+0000","last_clean_scrub_stamp":"2023-11-26T07:18:50.617605+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T12:04:39.473159+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.9","version":"0'0","reported_seq":11,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-11-26T07:18:54.776619+0000","last_change":"2023-11-26T07:18:52.686470+0000","last_active":"2023-11-26T07:18:54.776619+0000","last_peered":"2023-11-26T07:18:54.776619+0000","last_clean":"2023-11-26T07:18:54.776619+0000","last_became_active":"2023-11-26T07:18:52.686061+0000","last_became_peered":"2023-11-26T07:18:52.686061+0000","last_unstale":"2023-11-26T07:18:54.776619+0000","last_undegraded":"2023-11-26T07:18:54.776619+0000","last_fullsized":"2023-11-26T07:18:54.776619+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:18:51.625829+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:18:51.625829+0000","last_clean_scrub_stamp":"2023-11-26T07:18:51.625829+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T08:36:31.952917+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.8","version":"0'0","reported_seq":12,"reported_epoch":32,"state":"active+clean","last_fresh":"2023-11-26T07:18:52.664183+0000","last_change":"2023-11-26T07:18:51.659037+0000","last_active":"2023-11-26T07:18:52.664183+0000","last_peered":"2023-11-26T07:18:52.664183+0000","last_clean":"2023-11-26T07:18:52.664183+0000","last_became_active":"2023-11-26T07:18:51.658596+0000","last_became_peered":"2023-11-26T07:18:51.658596+0000","last_unstale":"2023-11-26T07:18:52.664183+0000","last_undegraded":"2023-11-26T07:18:52.664183+0000","last_fullsized":"2023-11-26T07:18:52.664183+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:18:50.617605+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:18:50.617605+0000","last_clean_scrub_stamp":"2023-11-26T07:18:50.617605+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T12:40:27.630609+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.6","version":"0'0","reported_seq":11,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-11-26T07:18:54.777159+0000","last_change":"2023-11-26T07:18:52.678430+0000","last_active":"2023-11-26T07:18:54.777159+0000","last_peered":"2023-11-26T07:18:54.777159+0000","last_clean":"2023-11-26T07:18:54.777159+0000","last_became_active":"2023-11-26T07:18:52.678167+0000","last_became_peered":"2023-11-26T07:18:52.678167+0000","last_unstale":"2023-11-26T07:18:54.777159+0000","last_undegraded":"2023-11-26T07:18:54.777159+0000","last_fullsized":"2023-11-26T07:18:54.777159+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:18:51.625829+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:18:51.625829+0000","last_clean_scrub_stamp":"2023-11-26T07:18:51.625829+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T17:27:25.330389+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.7","version":"32'2","reported_seq":14,"reported_epoch":32,"state":"active+clean","last_fresh":"2023-11-26T07:18:52.681021+0000","last_change":"2023-11-26T07:18:51.653789+0000","last_active":"2023-11-26T07:18:52.681021+0000","last_peered":"2023-11-26T07:18:52.681021+0000","last_clean":"2023-11-26T07:18:52.681021+0000","last_became_active":"2023-11-26T07:18:51.652136+0000","last_became_peered":"2023-11-26T07:18:51.652136+0000","last_unstale":"2023-11-26T07:18:52.681021+0000","last_undegraded":"2023-11-26T07:18:52.681021+0000","last_fullsized":"2023-11-26T07:18:52.681021+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:18:50.617605+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:18:50.617605+0000","last_clean_scrub_stamp":"2023-11-26T07:18:50.617605+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T09:38:44.587731+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.7","version":"0'0","reported_seq":11,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-11-26T07:18:54.776302+0000","last_change":"2023-11-26T07:18:52.677480+0000","last_active":"2023-11-26T07:18:54.776302+0000","last_peered":"2023-11-26T07:18:54.776302+0000","last_clean":"2023-11-26T07:18:54.776302+0000","last_became_active":"2023-11-26T07:18:52.677195+0000","last_became_peered":"2023-11-26T07:18:52.677195+0000","last_unstale":"2023-11-26T07:18:54.776302+0000","last_undegraded":"2023-11-26T07:18:54.776302+0000","last_fullsized":"2023-11-26T07:18:54.776302+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:18:51.625829+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:18:51.625829+0000","last_clean_scrub_stamp":"2023-11-26T07:18:51.625829+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T10:23:04.707081+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.6","version":"33'1","reported_seq":13,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-11-26T07:18:53.664307+0000","last_change":"2023-11-26T07:18:51.644857+0000","last_active":"2023-11-26T07:18:53.664307+0000","last_peered":"2023-11-26T07:18:53.664307+0000","last_clean":"2023-11-26T07:18:53.664307+0000","last_became_active":"2023-11-26T07:18:51.644616+0000","last_became_peered":"2023-11-26T07:18:51.644616+0000","last_unstale":"2023-11-26T07:18:53.664307+0000","last_undegraded":"2023-11-26T07:18:53.664307+0000","last_fullsized":"2023-11-26T07:18:53.664307+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:18:50.617605+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:18:50.617605+0000","last_clean_scrub_stamp":"2023-11-26T07:18:50.617605+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T12:12:35.496125+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.5","version":"0'0","reported_seq":11,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-11-26T07:18:53.663956+0000","last_change":"2023-11-26T07:18:52.676788+0000","last_active":"2023-11-26T07:18:53.663956+0000","last_peered":"2023-11-26T07:18:53.663956+0000","last_clean":"2023-11-26T07:18:53.663956+0000","last_became_active":"2023-11-26T07:18:52.676532+0000","last_became_peered":"2023-11-26T07:18:52.676532+0000","last_unstale":"2023-11-26T07:18:53.663956+0000","last_undegraded":"2023-11-26T07:18:53.663956+0000","last_fullsized":"2023-11-26T07:18:53.663956+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:18:51.625829+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:18:51.625829+0000","last_clean_scrub_stamp":"2023-11-26T07:18:51.625829+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T08:30:11.556027+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.4","version":"0'0","reported_seq":12,"reported_epoch":32,"state":"active+clean","last_fresh":"2023-11-26T07:18:52.671376+0000","last_change":"2023-11-26T07:18:51.660079+0000","last_active":"2023-11-26T07:18:52.671376+0000","last_peered":"2023-11-26T07:18:52.671376+0000","last_clean":"2023-11-26T07:18:52.671376+0000","last_became_active":"2023-11-26T07:18:51.659661+0000","last_became_peered":"2023-11-26T07:18:51.659661+0000","last_unstale":"2023-11-26T07:18:52.671376+0000","last_undegraded":"2023-11-26T07:18:52.671376+0000","last_fullsized":"2023-11-26T07:18:52.671376+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:18:50.617605+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:18:50.617605+0000","last_clean_scrub_stamp":"2023-11-26T07:18:50.617605+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T13:18:25.992925+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.2","version":"0'0","reported_seq":11,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-11-26T07:18:54.776634+0000","last_change":"2023-11-26T07:18:52.682229+0000","last_active":"2023-11-26T07:18:54.776634+0000","last_peered":"2023-11-26T07:18:54.776634+0000","last_clean":"2023-11-26T07:18:54.776634+0000","last_became_active":"2023-11-26T07:18:52.681612+0000","last_became_peered":"2023-11-26T07:18:52.681612+0000","last_unstale":"2023-11-26T07:18:54.776634+0000","last_undegraded":"2023-11-26T07:18:54.776634+0000","last_fullsized":"2023-11-26T07:18:54.776634+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:18:51.625829+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:18:51.625829+0000","last_clean_scrub_stamp":"2023-11-26T07:18:51.625829+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T13:18:27.001149+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.3","version":"0'0","reported_seq":12,"reported_epoch":32,"state":"active+clean","last_fresh":"2023-11-26T07:18:52.654786+0000","last_change":"2023-11-26T07:18:51.647332+0000","last_active":"2023-11-26T07:18:52.654786+0000","last_peered":"2023-11-26T07:18:52.654786+0000","last_clean":"2023-11-26T07:18:52.654786+0000","last_became_active":"2023-11-26T07:18:51.647054+0000","last_became_peered":"2023-11-26T07:18:51.647054+0000","last_unstale":"2023-11-26T07:18:52.654786+0000","last_undegraded":"2023-11-26T07:18:52.654786+0000","last_fullsized":"2023-11-26T07:18:52.654786+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:18:50.617605+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:18:50.617605+0000","last_clean_scrub_stamp":"2023-11-26T07:18:50.617605+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T16:41:01.096645+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.3","version":"0'0","reported_seq":11,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-11-26T07:18:53.664412+0000","last_change":"2023-11-26T07:18:52.683961+0000","last_active":"2023-11-26T07:18:53.664412+0000","last_peered":"2023-11-26T07:18:53.664412+0000","last_clean":"2023-11-26T07:18:53.664412+0000","last_became_active":"2023-11-26T07:18:52.683685+0000","last_became_peered":"2023-11-26T07:18:52.683685+0000","last_unstale":"2023-11-26T07:18:53.664412+0000","last_undegraded":"2023-11-26T07:18:53.664412+0000","last_fullsized":"2023-11-26T07:18:53.664412+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:18:51.625829+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:18:51.625829+0000","last_clean_scrub_stamp":"2023-11-26T07:18:51.625829+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T09:38:45.595955+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.2","version":"0'0","reported_seq":12,"reported_epoch":32,"state":"active+clean","last_fresh":"2023-11-26T07:18:52.671429+0000","last_change":"2023-11-26T07:18:51.651994+0000","last_active":"2023-11-26T07:18:52.671429+0000","last_peered":"2023-11-26T07:18:52.671429+0000","last_clean":"2023-11-26T07:18:52.671429+0000","last_became_active":"2023-11-26T07:18:51.650255+0000","last_became_peered":"2023-11-26T07:18:51.650255+0000","last_unstale":"2023-11-26T07:18:52.671429+0000","last_undegraded":"2023-11-26T07:18:52.671429+0000","last_fullsized":"2023-11-26T07:18:52.671429+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:18:50.617605+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:18:50.617605+0000","last_clean_scrub_stamp":"2023-11-26T07:18:50.617605+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T08:30:10.547803+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.0","version":"0'0","reported_seq":11,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-11-26T07:18:54.776518+0000","last_change":"2023-11-26T07:18:52.681139+0000","last_active":"2023-11-26T07:18:54.776518+0000","last_peered":"2023-11-26T07:18:54.776518+0000","last_clean":"2023-11-26T07:18:54.776518+0000","last_became_active":"2023-11-26T07:18:52.679760+0000","last_became_peered":"2023-11-26T07:18:52.679760+0000","last_unstale":"2023-11-26T07:18:54.776518+0000","last_undegraded":"2023-11-26T07:18:54.776518+0000","last_fullsized":"2023-11-26T07:18:54.776518+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:18:51.625829+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:18:51.625829+0000","last_clean_scrub_stamp":"2023-11-26T07:18:51.625829+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T17:23:21.873210+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1","version":"0'0","reported_seq":12,"reported_epoch":32,"state":"active+clean","last_fresh":"2023-11-26T07:18:52.653205+0000","last_change":"2023-11-26T07:18:51.654435+0000","last_active":"2023-11-26T07:18:52.653205+0000","last_peered":"2023-11-26T07:18:52.653205+0000","last_clean":"2023-11-26T07:18:52.653205+0000","last_became_active":"2023-11-26T07:18:51.654107+0000","last_became_peered":"2023-11-26T07:18:51.654107+0000","last_unstale":"2023-11-26T07:18:52.653205+0000","last_undegraded":"2023-11-26T07:18:52.653205+0000","last_fullsized":"2023-11-26T07:18:52.653205+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:18:50.617605+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:18:50.617605+0000","last_clean_scrub_stamp":"2023-11-26T07:18:50.617605+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T14:40:15.233288+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.1","version":"0'0","reported_seq":11,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-11-26T07:18:54.775507+0000","last_change":"2023-11-26T07:18:52.668080+0000","last_active":"2023-11-26T07:18:54.775507+0000","last_peered":"2023-11-26T07:18:54.775507+0000","last_clean":"2023-11-26T07:18:54.775507+0000","last_became_active":"2023-11-26T07:18:52.667573+0000","last_became_peered":"2023-11-26T07:18:52.667573+0000","last_unstale":"2023-11-26T07:18:54.775507+0000","last_undegraded":"2023-11-26T07:18:54.775507+0000","last_fullsized":"2023-11-26T07:18:54.775507+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:18:51.625829+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:18:51.625829+0000","last_clean_scrub_stamp":"2023-11-26T07:18:51.625829+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T10:33:50.109555+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.0","version":"0'0","reported_seq":12,"reported_epoch":32,"state":"active+clean","last_fresh":"2023-11-26T07:18:52.649482+0000","last_change":"2023-11-26T07:18:51.655721+0000","last_active":"2023-11-26T07:18:52.649482+0000","last_peered":"2023-11-26T07:18:52.649482+0000","last_clean":"2023-11-26T07:18:52.649482+0000","last_became_active":"2023-11-26T07:18:51.655334+0000","last_became_peered":"2023-11-26T07:18:51.655334+0000","last_unstale":"2023-11-26T07:18:52.649482+0000","last_undegraded":"2023-11-26T07:18:52.649482+0000","last_fullsized":"2023-11-26T07:18:52.649482+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:18:50.617605+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:18:50.617605+0000","last_clean_scrub_stamp":"2023-11-26T07:18:50.617605+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T13:18:25.992925+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.4","version":"0'0","reported_seq":11,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-11-26T07:18:54.776949+0000","last_change":"2023-11-26T07:18:52.678680+0000","last_active":"2023-11-26T07:18:54.776949+0000","last_peered":"2023-11-26T07:18:54.776949+0000","last_clean":"2023-11-26T07:18:54.776949+0000","last_became_active":"2023-11-26T07:18:52.678376+0000","last_became_peered":"2023-11-26T07:18:52.678376+0000","last_unstale":"2023-11-26T07:18:54.776949+0000","last_undegraded":"2023-11-26T07:18:54.776949+0000","last_fullsized":"2023-11-26T07:18:54.776949+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:18:51.625829+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:18:51.625829+0000","last_clean_scrub_stamp":"2023-11-26T07:18:51.625829+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T15:42:06.512477+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.5","version":"0'0","reported_seq":12,"reported_epoch":32,"state":"active+clean","last_fresh":"2023-11-26T07:18:52.658814+0000","last_change":"2023-11-26T07:18:51.661135+0000","last_active":"2023-11-26T07:18:52.658814+0000","last_peered":"2023-11-26T07:18:52.658814+0000","last_clean":"2023-11-26T07:18:52.658814+0000","last_became_active":"2023-11-26T07:18:51.659814+0000","last_became_peered":"2023-11-26T07:18:51.659814+0000","last_unstale":"2023-11-26T07:18:52.658814+0000","last_undegraded":"2023-11-26T07:18:52.658814+0000","last_fullsized":"2023-11-26T07:18:52.658814+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:18:50.617605+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:18:50.617605+0000","last_clean_scrub_stamp":"2023-11-26T07:18:50.617605+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T14:49:28.420672+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"1.0","version":"15'117","reported_seq":175,"reported_epoch":32,"state":"active+clean","last_fresh":"2023-11-26T07:18:52.665022+0000","last_change":"2023-11-26T07:17:43.827802+0000","last_active":"2023-11-26T07:18:52.665022+0000","last_peered":"2023-11-26T07:18:52.665022+0000","last_clean":"2023-11-26T07:18:52.665022+0000","last_became_active":"2023-11-26T07:17:43.827308+0000","last_became_peered":"2023-11-26T07:17:43.827308+0000","last_unstale":"2023-11-26T07:18:52.665022+0000","last_undegraded":"2023-11-26T07:18:52.665022+0000","last_fullsized":"2023-11-26T07:18:52.665022+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:17:42.160945+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:17:42.160945+0000","last_clean_scrub_stamp":"2023-11-26T07:17:42.160945+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T08:17:54.167366+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":5,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":4,"num_pg":32,"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":39,"ondisk_log_size":39,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574865,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27968,"kb_used_data":948,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343872,"statfs":{"total":96636764160,"available":96608124928,"internally_reserved":0,"allocated":970752,"data_stored":737050,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Nov 26 07:18:42 2023","interfaces":[{"interface":"back","average":{"1min":0.71499999999999997,"5min":0.71499999999999997,"15min":0.71499999999999997},"min":{"1min":0.54300000000000004,"5min":0.54300000000000004,"15min":0.54300000000000004},"max":{"1min":0.91500000000000004,"5min":0.91500000000000004,"15min":0.91500000000000004},"last":0.50600000000000001},{"interface":"front","average":{"1min":0.70899999999999996,"5min":0.70899999999999996,"15min":0.70899999999999996},"min":{"1min":0.52900000000000003,"5min":0.52900000000000003,"15min":0.52900000000000003},"max":{"1min":1.0980000000000001,"5min":1.0980000000000001,"15min":1.0980000000000001},"last":0.75700000000000001}]},{"osd":1,"last update":"Sun Nov 26 07:18:42 2023","interfaces":[{"interface":"back","average":{"1min":0.68200000000000005,"5min":0.68200000000000005,"15min":0.68200000000000005},"min":{"1min":0.47999999999999998,"5min":0.47999999999999998,"15min":0.47999999999999998},"max":{"1min":1.0669999999999999,"5min":1.0669999999999999,"15min":1.0669999999999999},"last":0.63700000000000001},{"interface":"front","average":{"1min":0.83899999999999997,"5min":0.83899999999999997,"15min":0.83899999999999997},"min":{"1min":0.47599999999999998,"5min":0.47599999999999998,"15min":0.47599999999999998},"max":{"1min":4.5309999999999997,"5min":4.5309999999999997,"15min":4.5309999999999997},"last":0.53200000000000003}]},{"osd":2,"last update":"Sun Nov 26 07:18:42 2023","interfaces":[{"interface":"back","average":{"1min":0.67600000000000005,"5min":0.67600000000000005,"15min":0.67600000000000005},"min":{"1min":0.52000000000000002,"5min":0.52000000000000002,"15min":0.52000000000000002},"max":{"1min":0.873,"5min":0.873,"15min":0.873},"last":0.65700000000000003},{"interface":"front","average":{"1min":0.71299999999999997,"5min":0.71299999999999997,"15min":0.71299999999999997},"min":{"1min":0.54700000000000004,"5min":0.54700000000000004,"15min":0.54700000000000004},"max":{"1min":1.1339999999999999,"5min":1.1339999999999999,"15min":1.1339999999999999},"last":0.59099999999999997}]},{"osd":3,"last update":"Sun Nov 26 07:18:42 2023","interfaces":[{"interface":"back","average":{"1min":0.69699999999999995,"5min":0.69699999999999995,"15min":0.69699999999999995},"min":{"1min":0.51800000000000002,"5min":0.51800000000000002,"15min":0.51800000000000002},"max":{"1min":1.026,"5min":1.026,"15min":1.026},"last":0.71099999999999997},{"interface":"front","average":{"1min":0.73299999999999998,"5min":0.73299999999999998,"15min":0.73299999999999998},"min":{"1min":0.50600000000000001,"5min":0.50600000000000001,"15min":0.50600000000000001},"max":{"1min":1.4490000000000001,"5min":1.4490000000000001,"15min":1.4490000000000001},"last":0.67400000000000004}]},{"osd":4,"last update":"Sun Nov 26 07:18:42 2023","interfaces":[{"interface":"back","average":{"1min":0.59999999999999998,"5min":0.59999999999999998,"15min":0.59999999999999998},"min":{"1min":0.39400000000000002,"5min":0.39400000000000002,"15min":0.39400000000000002},"max":{"1min":0.94799999999999995,"5min":0.94799999999999995,"15min":0.94799999999999995},"last":0.68999999999999995},{"interface":"front","average":{"1min":0.67300000000000004,"5min":0.67300000000000004,"15min":0.67300000000000004},"min":{"1min":0.436,"5min":0.436,"15min":0.436},"max":{"1min":1.5509999999999999,"5min":1.5509999999999999,"15min":1.5509999999999999},"last":0.56499999999999995}]},{"osd":5,"last update":"Sun Nov 26 07:18:42 2023","interfaces":[{"interface":"back","average":{"1min":0.66600000000000004,"5min":0.66600000000000004,"15min":0.66600000000000004},"min":{"1min":0.443,"5min":0.443,"15min":0.443},"max":{"1min":1.415,"5min":1.415,"15min":1.415},"last":0.55400000000000005},{"interface":"front","average":{"1min":0.71399999999999997,"5min":0.71399999999999997,"15min":0.71399999999999997},"min":{"1min":0.53100000000000003,"5min":0.53100000000000003,"15min":0.53100000000000003},"max":{"1min":1.3859999999999999,"5min":1.3859999999999999,"15min":1.3859999999999999},"last":0.60799999999999998}]},{"osd":6,"last update":"Sun Nov 26 07:18:42 2023","interfaces":[{"interface":"back","average":{"1min":0.77100000000000002,"5min":0.77100000000000002,"15min":0.77100000000000002},"min":{"1min":0.64100000000000001,"5min":0.64100000000000001,"15min":0.64100000000000001},"max":{"1min":1.3069999999999999,"5min":1.3069999999999999,"15min":1.3069999999999999},"last":0.80600000000000005},{"interface":"front","average":{"1min":0.77100000000000002,"5min":0.77100000000000002,"15min":0.77100000000000002},"min":{"1min":0.57399999999999995,"5min":0.57399999999999995,"15min":0.57399999999999995},"max":{"1min":1.6759999999999999,"5min":1.6759999999999999,"15min":1.6759999999999999},"last":0.74099999999999999}]}]},{"osd":6,"up_from":13,"seq":55834574867,"num_pgs":19,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27392,"kb_used_data":372,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344448,"statfs":{"total":96636764160,"available":96608714752,"internally_reserved":0,"allocated":380928,"data_stored":147208,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":9,"apply_latency_ms":9,"commit_latency_ns":9000000,"apply_latency_ns":9000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Nov 26 07:18:45 2023","interfaces":[{"interface":"back","average":{"1min":0.68400000000000005,"5min":0.68400000000000005,"15min":0.68400000000000005},"min":{"1min":0.36699999999999999,"5min":0.36699999999999999,"15min":0.36699999999999999},"max":{"1min":1.7549999999999999,"5min":1.7549999999999999,"15min":1.7549999999999999},"last":0.72399999999999998},{"interface":"front","average":{"1min":0.95799999999999996,"5min":0.95799999999999996,"15min":0.95799999999999996},"min":{"1min":0.38300000000000001,"5min":0.38300000000000001,"15min":0.38300000000000001},"max":{"1min":5.7489999999999997,"5min":5.7489999999999997,"15min":5.7489999999999997},"last":0.86099999999999999}]},{"osd":1,"last update":"Sun Nov 26 07:18:45 2023","interfaces":[{"interface":"back","average":{"1min":0.91400000000000003,"5min":0.91400000000000003,"15min":0.91400000000000003},"min":{"1min":0.42099999999999999,"5min":0.42099999999999999,"15min":0.42099999999999999},"max":{"1min":5.4240000000000004,"5min":5.4240000000000004,"15min":5.4240000000000004},"last":0.78000000000000003},{"interface":"front","average":{"1min":0.89700000000000002,"5min":0.89700000000000002,"15min":0.89700000000000002},"min":{"1min":0.499,"5min":0.499,"15min":0.499},"max":{"1min":5.6020000000000003,"5min":5.6020000000000003,"15min":5.6020000000000003},"last":0.90700000000000003}]},{"osd":2,"last update":"Sun Nov 26 07:18:45 2023","interfaces":[{"interface":"back","average":{"1min":0.69699999999999995,"5min":0.69699999999999995,"15min":0.69699999999999995},"min":{"1min":0.47599999999999998,"5min":0.47599999999999998,"15min":0.47599999999999998},"max":{"1min":1.8520000000000001,"5min":1.8520000000000001,"15min":1.8520000000000001},"last":0.76000000000000001},{"interface":"front","average":{"1min":0.73399999999999999,"5min":0.73399999999999999,"15min":0.73399999999999999},"min":{"1min":0.46700000000000003,"5min":0.46700000000000003,"15min":0.46700000000000003},"max":{"1min":1.9159999999999999,"5min":1.9159999999999999,"15min":1.9159999999999999},"last":0.82799999999999996}]},{"osd":3,"last update":"Sun Nov 26 07:18:45 2023","interfaces":[{"interface":"back","average":{"1min":0.90200000000000002,"5min":0.90200000000000002,"15min":0.90200000000000002},"min":{"1min":0.502,"5min":0.502,"15min":0.502},"max":{"1min":5.4390000000000001,"5min":5.4390000000000001,"15min":5.4390000000000001},"last":0.80100000000000005},{"interface":"front","average":{"1min":0.94499999999999995,"5min":0.94499999999999995,"15min":0.94499999999999995},"min":{"1min":0.54400000000000004,"5min":0.54400000000000004,"15min":0.54400000000000004},"max":{"1min":5.5490000000000004,"5min":5.5490000000000004,"15min":5.5490000000000004},"last":0.84499999999999997}]},{"osd":4,"last update":"Sun Nov 26 07:18:45 2023","interfaces":[{"interface":"back","average":{"1min":0.64200000000000002,"5min":0.64200000000000002,"15min":0.64200000000000002},"min":{"1min":0.46899999999999997,"5min":0.46899999999999997,"15min":0.46899999999999997},"max":{"1min":0.85099999999999998,"5min":0.85099999999999998,"15min":0.85099999999999998},"last":0.51500000000000001},{"interface":"front","average":{"1min":0.67500000000000004,"5min":0.67500000000000004,"15min":0.67500000000000004},"min":{"1min":0.40699999999999997,"5min":0.40699999999999997,"15min":0.40699999999999997},"max":{"1min":0.93300000000000005,"5min":0.93300000000000005,"15min":0.93300000000000005},"last":0.47799999999999998}]},{"osd":5,"last update":"Sun Nov 26 07:18:45 2023","interfaces":[{"interface":"back","average":{"1min":0.65900000000000003,"5min":0.65900000000000003,"15min":0.65900000000000003},"min":{"1min":0.42499999999999999,"5min":0.42499999999999999,"15min":0.42499999999999999},"max":{"1min":0.79300000000000004,"5min":0.79300000000000004,"15min":0.79300000000000004},"last":0.57399999999999995},{"interface":"front","average":{"1min":0.71299999999999997,"5min":0.71299999999999997,"15min":0.71299999999999997},"min":{"1min":0.436,"5min":0.436,"15min":0.436},"max":{"1min":0.93000000000000005,"5min":0.93000000000000005,"15min":0.93000000000000005},"last":0.628}]},{"osd":7,"last update":"Sun Nov 26 07:18:45 2023","interfaces":[{"interface":"back","average":{"1min":0.74199999999999999,"5min":0.74199999999999999,"15min":0.74199999999999999},"min":{"1min":0.46600000000000003,"5min":0.46600000000000003,"15min":0.46600000000000003},"max":{"1min":0.998,"5min":0.998,"15min":0.998},"last":0.55200000000000005},{"interface":"front","average":{"1min":0.71599999999999997,"5min":0.71599999999999997,"15min":0.71599999999999997},"min":{"1min":0.498,"5min":0.498,"15min":0.498},"max":{"1min":0.86499999999999999,"5min":0.86499999999999999,"15min":0.86499999999999999},"last":0.61399999999999999}]}]},{"osd":5,"up_from":13,"seq":55834574865,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27392,"kb_used_data":372,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344448,"statfs":{"total":96636764160,"available":96608714752,"internally_reserved":0,"allocated":380928,"data_stored":147264,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Nov 26 07:18:45 2023","interfaces":[{"interface":"back","average":{"1min":0.66600000000000004,"5min":0.66600000000000004,"15min":0.66600000000000004},"min":{"1min":0.38100000000000001,"5min":0.38100000000000001,"15min":0.38100000000000001},"max":{"1min":1.0289999999999999,"5min":1.0289999999999999,"15min":1.0289999999999999},"last":0.59099999999999997},{"interface":"front","average":{"1min":0.66000000000000003,"5min":0.66000000000000003,"15min":0.66000000000000003},"min":{"1min":0.438,"5min":0.438,"15min":0.438},"max":{"1min":1.05,"5min":1.05,"15min":1.05},"last":0.54700000000000004}]},{"osd":1,"last update":"Sun Nov 26 07:18:45 2023","interfaces":[{"interface":"back","average":{"1min":0.66800000000000004,"5min":0.66800000000000004,"15min":0.66800000000000004},"min":{"1min":0.41599999999999998,"5min":0.41599999999999998,"15min":0.41599999999999998},"max":{"1min":0.86699999999999999,"5min":0.86699999999999999,"15min":0.86699999999999999},"last":0.79400000000000004},{"interface":"front","average":{"1min":0.68600000000000005,"5min":0.68600000000000005,"15min":0.68600000000000005},"min":{"1min":0.49199999999999999,"5min":0.49199999999999999,"15min":0.49199999999999999},"max":{"1min":0.98899999999999999,"5min":0.98899999999999999,"15min":0.98899999999999999},"last":0.52900000000000003}]},{"osd":2,"last update":"Sun Nov 26 07:18:45 2023","interfaces":[{"interface":"back","average":{"1min":0.67500000000000004,"5min":0.67500000000000004,"15min":0.67500000000000004},"min":{"1min":0.45900000000000002,"5min":0.45900000000000002,"15min":0.45900000000000002},"max":{"1min":0.89800000000000002,"5min":0.89800000000000002,"15min":0.89800000000000002},"last":0.67500000000000004},{"interface":"front","average":{"1min":0.67000000000000004,"5min":0.67000000000000004,"15min":0.67000000000000004},"min":{"1min":0.51200000000000001,"5min":0.51200000000000001,"15min":0.51200000000000001},"max":{"1min":0.89700000000000002,"5min":0.89700000000000002,"15min":0.89700000000000002},"last":0.73399999999999999}]},{"osd":3,"last update":"Sun Nov 26 07:18:45 2023","interfaces":[{"interface":"back","average":{"1min":0.71999999999999997,"5min":0.71999999999999997,"15min":0.71999999999999997},"min":{"1min":0.43099999999999999,"5min":0.43099999999999999,"15min":0.43099999999999999},"max":{"1min":1.0029999999999999,"5min":1.0029999999999999,"15min":1.0029999999999999},"last":0.629},{"interface":"front","average":{"1min":0.67500000000000004,"5min":0.67500000000000004,"15min":0.67500000000000004},"min":{"1min":0.44600000000000001,"5min":0.44600000000000001,"15min":0.44600000000000001},"max":{"1min":1.0209999999999999,"5min":1.0209999999999999,"15min":1.0209999999999999},"last":0.48199999999999998}]},{"osd":4,"last update":"Sun Nov 26 07:18:45 2023","interfaces":[{"interface":"back","average":{"1min":0.63500000000000001,"5min":0.63500000000000001,"15min":0.63500000000000001},"min":{"1min":0.39200000000000002,"5min":0.39200000000000002,"15min":0.39200000000000002},"max":{"1min":0.95099999999999996,"5min":0.95099999999999996,"15min":0.95099999999999996},"last":0.70199999999999996},{"interface":"front","average":{"1min":0.70899999999999996,"5min":0.70899999999999996,"15min":0.70899999999999996},"min":{"1min":0.41299999999999998,"5min":0.41299999999999998,"15min":0.41299999999999998},"max":{"1min":1.254,"5min":1.254,"15min":1.254},"last":0.75900000000000001}]},{"osd":6,"last update":"Sun Nov 26 07:18:45 2023","interfaces":[{"interface":"back","average":{"1min":0.67200000000000004,"5min":0.67200000000000004,"15min":0.67200000000000004},"min":{"1min":0.45600000000000002,"5min":0.45600000000000002,"15min":0.45600000000000002},"max":{"1min":0.94499999999999995,"5min":0.94499999999999995,"15min":0.94499999999999995},"last":0.80900000000000005},{"interface":"front","average":{"1min":0.72699999999999998,"5min":0.72699999999999998,"15min":0.72699999999999998},"min":{"1min":0.44,"5min":0.44,"15min":0.44},"max":{"1min":1.0649999999999999,"5min":1.0649999999999999,"15min":1.0649999999999999},"last":0.77700000000000002}]},{"osd":7,"last update":"Sun Nov 26 07:18:45 2023","interfaces":[{"interface":"back","average":{"1min":0.80400000000000005,"5min":0.80400000000000005,"15min":0.80400000000000005},"min":{"1min":0.48199999999999998,"5min":0.48199999999999998,"15min":0.48199999999999998},"max":{"1min":1.4079999999999999,"5min":1.4079999999999999,"15min":1.4079999999999999},"last":0.85399999999999998},{"interface":"front","average":{"1min":0.77500000000000002,"5min":0.77500000000000002,"15min":0.77500000000000002},"min":{"1min":0.51600000000000001,"5min":0.51600000000000001,"15min":0.51600000000000001},"max":{"1min":1.0169999999999999,"5min":1.0169999999999999,"15min":1.0169999999999999},"last":0.82499999999999996}]}]},{"osd":4,"up_from":13,"seq":55834574867,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27392,"kb_used_data":372,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344448,"statfs":{"total":96636764160,"available":96608714752,"internally_reserved":0,"allocated":380928,"data_stored":146696,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Nov 26 07:18:46 2023","interfaces":[{"interface":"back","average":{"1min":0.79400000000000004,"5min":0.79400000000000004,"15min":0.79400000000000004},"min":{"1min":0.45600000000000002,"5min":0.45600000000000002,"15min":0.45600000000000002},"max":{"1min":4.1689999999999996,"5min":4.1689999999999996,"15min":4.1689999999999996},"last":0.60299999999999998},{"interface":"front","average":{"1min":1.002,"5min":1.002,"15min":1.002},"min":{"1min":0.47499999999999998,"5min":0.47499999999999998,"15min":0.47499999999999998},"max":{"1min":7.7889999999999997,"5min":7.7889999999999997,"15min":7.7889999999999997},"last":0.64300000000000002}]},{"osd":1,"last update":"Sun Nov 26 07:18:46 2023","interfaces":[{"interface":"back","average":{"1min":0.64700000000000002,"5min":0.64700000000000002,"15min":0.64700000000000002},"min":{"1min":0.52300000000000002,"5min":0.52300000000000002,"15min":0.52300000000000002},"max":{"1min":0.78400000000000003,"5min":0.78400000000000003,"15min":0.78400000000000003},"last":0.72499999999999998},{"interface":"front","average":{"1min":0.81999999999999995,"5min":0.81999999999999995,"15min":0.81999999999999995},"min":{"1min":0.44500000000000001,"5min":0.44500000000000001,"15min":0.44500000000000001},"max":{"1min":4.3140000000000001,"5min":4.3140000000000001,"15min":4.3140000000000001},"last":0.66700000000000004}]},{"osd":2,"last update":"Sun Nov 26 07:18:46 2023","interfaces":[{"interface":"back","average":{"1min":0.80700000000000005,"5min":0.80700000000000005,"15min":0.80700000000000005},"min":{"1min":0.57799999999999996,"5min":0.57799999999999996,"15min":0.57799999999999996},"max":{"1min":2.7850000000000001,"5min":2.7850000000000001,"15min":2.7850000000000001},"last":0.56000000000000005},{"interface":"front","average":{"1min":1.0660000000000001,"5min":1.0660000000000001,"15min":1.0660000000000001},"min":{"1min":0.53400000000000003,"5min":0.53400000000000003,"15min":0.53400000000000003},"max":{"1min":7.6470000000000002,"5min":7.6470000000000002,"15min":7.6470000000000002},"last":0.76100000000000001}]},{"osd":3,"last update":"Sun Nov 26 07:18:46 2023","interfaces":[{"interface":"back","average":{"1min":0.72999999999999998,"5min":0.72999999999999998,"15min":0.72999999999999998},"min":{"1min":0.48599999999999999,"5min":0.48599999999999999,"15min":0.48599999999999999},"max":{"1min":1.591,"5min":1.591,"15min":1.591},"last":0.70699999999999996},{"interface":"front","average":{"1min":0.879,"5min":0.879,"15min":0.879},"min":{"1min":0.56899999999999995,"5min":0.56899999999999995,"15min":0.56899999999999995},"max":{"1min":3.452,"5min":3.452,"15min":3.452},"last":0.68500000000000005}]},{"osd":5,"last update":"Sun Nov 26 07:18:46 2023","interfaces":[{"interface":"back","average":{"1min":0.61099999999999999,"5min":0.61099999999999999,"15min":0.61099999999999999},"min":{"1min":0.43099999999999999,"5min":0.43099999999999999,"15min":0.43099999999999999},"max":{"1min":0.96199999999999997,"5min":0.96199999999999997,"15min":0.96199999999999997},"last":0.46100000000000002},{"interface":"front","average":{"1min":0.64100000000000001,"5min":0.64100000000000001,"15min":0.64100000000000001},"min":{"1min":0.41499999999999998,"5min":0.41499999999999998,"15min":0.41499999999999998},"max":{"1min":1.538,"5min":1.538,"15min":1.538},"last":0.52100000000000002}]},{"osd":6,"last update":"Sun Nov 26 07:18:46 2023","interfaces":[{"interface":"back","average":{"1min":0.79400000000000004,"5min":0.79400000000000004,"15min":0.79400000000000004},"min":{"1min":0.39400000000000002,"5min":0.39400000000000002,"15min":0.39400000000000002},"max":{"1min":2.8330000000000002,"5min":2.8330000000000002,"15min":2.8330000000000002},"last":0.432},{"interface":"front","average":{"1min":1.0089999999999999,"5min":1.0089999999999999,"15min":1.0089999999999999},"min":{"1min":0.42899999999999999,"5min":0.42899999999999999,"15min":0.42899999999999999},"max":{"1min":7.6050000000000004,"5min":7.6050000000000004,"15min":7.6050000000000004},"last":0.58299999999999996}]},{"osd":7,"last update":"Sun Nov 26 07:18:46 2023","interfaces":[{"interface":"back","average":{"1min":0.86099999999999999,"5min":0.86099999999999999,"15min":0.86099999999999999},"min":{"1min":0.44700000000000001,"5min":0.44700000000000001,"15min":0.44700000000000001},"max":{"1min":4.3689999999999998,"5min":4.3689999999999998,"15min":4.3689999999999998},"last":0.62},{"interface":"front","average":{"1min":0.89200000000000002,"5min":0.89200000000000002,"15min":0.89200000000000002},"min":{"1min":0.505,"5min":0.505,"15min":0.505},"max":{"1min":3.403,"5min":3.403,"15min":3.403},"last":0.48399999999999999}]}]},{"osd":3,"up_from":13,"seq":55834574868,"num_pgs":31,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27400,"kb_used_data":380,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344440,"statfs":{"total":96636764160,"available":96608706560,"internally_reserved":0,"allocated":389120,"data_stored":148226,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1592,"internal_metadata":27654600},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":7,"apply_latency_ms":7,"commit_latency_ns":7000000,"apply_latency_ns":7000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Nov 26 07:18:43 2023","interfaces":[{"interface":"back","average":{"1min":0.379,"5min":0.379,"15min":0.379},"min":{"1min":0.26100000000000001,"5min":0.26100000000000001,"15min":0.26100000000000001},"max":{"1min":0.58599999999999997,"5min":0.58599999999999997,"15min":0.58599999999999997},"last":0.40500000000000003},{"interface":"front","average":{"1min":0.433,"5min":0.433,"15min":0.433},"min":{"1min":0.30199999999999999,"5min":0.30199999999999999,"15min":0.30199999999999999},"max":{"1min":0.63,"5min":0.63,"15min":0.63},"last":0.34899999999999998}]},{"osd":1,"last update":"Sun Nov 26 07:18:43 2023","interfaces":[{"interface":"back","average":{"1min":0.42099999999999999,"5min":0.42099999999999999,"15min":0.42099999999999999},"min":{"1min":0.312,"5min":0.312,"15min":0.312},"max":{"1min":0.752,"5min":0.752,"15min":0.752},"last":0.379},{"interface":"front","average":{"1min":0.48399999999999999,"5min":0.48399999999999999,"15min":0.48399999999999999},"min":{"1min":0.38,"5min":0.38,"15min":0.38},"max":{"1min":0.628,"5min":0.628,"15min":0.628},"last":0.46300000000000002}]},{"osd":2,"last update":"Sun Nov 26 07:18:43 2023","interfaces":[{"interface":"back","average":{"1min":0.53300000000000003,"5min":0.53300000000000003,"15min":0.53300000000000003},"min":{"1min":0.40300000000000002,"5min":0.40300000000000002,"15min":0.40300000000000002},"max":{"1min":0.82199999999999995,"5min":0.82199999999999995,"15min":0.82199999999999995},"last":0.50900000000000001},{"interface":"front","average":{"1min":0.505,"5min":0.505,"15min":0.505},"min":{"1min":0.35999999999999999,"5min":0.35999999999999999,"15min":0.35999999999999999},"max":{"1min":0.72199999999999998,"5min":0.72199999999999998,"15min":0.72199999999999998},"last":0.49399999999999999}]},{"osd":4,"last update":"Sun Nov 26 07:18:43 2023","interfaces":[{"interface":"back","average":{"1min":0.70599999999999996,"5min":0.70599999999999996,"15min":0.70599999999999996},"min":{"1min":0.48199999999999998,"5min":0.48199999999999998,"15min":0.48199999999999998},"max":{"1min":0.96199999999999997,"5min":0.96199999999999997,"15min":0.96199999999999997},"last":0.58499999999999996},{"interface":"front","average":{"1min":0.72799999999999998,"5min":0.72799999999999998,"15min":0.72799999999999998},"min":{"1min":0.441,"5min":0.441,"15min":0.441},"max":{"1min":1.171,"5min":1.171,"15min":1.171},"last":0.67400000000000004}]},{"osd":5,"last update":"Sun Nov 26 07:18:43 2023","interfaces":[{"interface":"back","average":{"1min":0.73899999999999999,"5min":0.73899999999999999,"15min":0.73899999999999999},"min":{"1min":0.53700000000000003,"5min":0.53700000000000003,"15min":0.53700000000000003},"max":{"1min":0.94099999999999995,"5min":0.94099999999999995,"15min":0.94099999999999995},"last":0.57099999999999995},{"interface":"front","average":{"1min":0.73799999999999999,"5min":0.73799999999999999,"15min":0.73799999999999999},"min":{"1min":0.504,"5min":0.504,"15min":0.504},"max":{"1min":1.0820000000000001,"5min":1.0820000000000001,"15min":1.0820000000000001},"last":0.54400000000000004}]},{"osd":6,"last update":"Sun Nov 26 07:18:43 2023","interfaces":[{"interface":"back","average":{"1min":0.81399999999999995,"5min":0.81399999999999995,"15min":0.81399999999999995},"min":{"1min":0.55200000000000005,"5min":0.55200000000000005,"15min":0.55200000000000005},"max":{"1min":1.2729999999999999,"5min":1.2729999999999999,"15min":1.2729999999999999},"last":0.55700000000000005},{"interface":"front","average":{"1min":0.79300000000000004,"5min":0.79300000000000004,"15min":0.79300000000000004},"min":{"1min":0.47299999999999998,"5min":0.47299999999999998,"15min":0.47299999999999998},"max":{"1min":1.204,"5min":1.204,"15min":1.204},"last":0.68799999999999994}]},{"osd":7,"last update":"Sun Nov 26 07:18:43 2023","interfaces":[{"interface":"back","average":{"1min":0.80100000000000005,"5min":0.80100000000000005,"15min":0.80100000000000005},"min":{"1min":0.63300000000000001,"5min":0.63300000000000001,"15min":0.63300000000000001},"max":{"1min":1.3600000000000001,"5min":1.3600000000000001,"15min":1.3600000000000001},"last":0.56599999999999995},{"interface":"front","average":{"1min":0.82599999999999996,"5min":0.82599999999999996,"15min":0.82599999999999996},"min":{"1min":0.50600000000000001,"5min":0.50600000000000001,"15min":0.50600000000000001},"max":{"1min":1.617,"5min":1.617,"15min":1.617},"last":0.71099999999999997}]}]},{"osd":2,"up_from":13,"seq":55834574868,"num_pgs":17,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27388,"kb_used_data":368,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344452,"statfs":{"total":96636764160,"available":96608718848,"internally_reserved":0,"allocated":376832,"data_stored":146662,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Nov 26 07:18:43 2023","interfaces":[{"interface":"back","average":{"1min":0.71399999999999997,"5min":0.71399999999999997,"15min":0.71399999999999997},"min":{"1min":0.28699999999999998,"5min":0.28699999999999998,"15min":0.28699999999999998},"max":{"1min":4.7030000000000003,"5min":4.7030000000000003,"15min":4.7030000000000003},"last":0.38300000000000001},{"interface":"front","average":{"1min":0.56200000000000006,"5min":0.56200000000000006,"15min":0.56200000000000006},"min":{"1min":0.32300000000000001,"5min":0.32300000000000001,"15min":0.32300000000000001},"max":{"1min":2.4340000000000002,"5min":2.4340000000000002,"15min":2.4340000000000002},"last":0.41399999999999998}]},{"osd":1,"last update":"Sun Nov 26 07:18:43 2023","interfaces":[{"interface":"back","average":{"1min":0.86399999999999999,"5min":0.86399999999999999,"15min":0.86399999999999999},"min":{"1min":0.34599999999999997,"5min":0.34599999999999997,"15min":0.34599999999999997},"max":{"1min":4.2690000000000001,"5min":4.2690000000000001,"15min":4.2690000000000001},"last":0.435},{"interface":"front","average":{"1min":0.74199999999999999,"5min":0.74199999999999999,"15min":0.74199999999999999},"min":{"1min":0.31900000000000001,"5min":0.31900000000000001,"15min":0.31900000000000001},"max":{"1min":3.9940000000000002,"5min":3.9940000000000002,"15min":3.9940000000000002},"last":0.45000000000000001}]},{"osd":3,"last update":"Sun Nov 26 07:18:43 2023","interfaces":[{"interface":"back","average":{"1min":0.64300000000000002,"5min":0.64300000000000002,"15min":0.64300000000000002},"min":{"1min":0.34399999999999997,"5min":0.34399999999999997,"15min":0.34399999999999997},"max":{"1min":4.0300000000000002,"5min":4.0300000000000002,"15min":4.0300000000000002},"last":0.36699999999999999},{"interface":"front","average":{"1min":0.91400000000000003,"5min":0.91400000000000003,"15min":0.91400000000000003},"min":{"1min":0.41699999999999998,"5min":0.41699999999999998,"15min":0.41699999999999998},"max":{"1min":4.5810000000000004,"5min":4.5810000000000004,"15min":4.5810000000000004},"last":0.61199999999999999}]},{"osd":4,"last update":"Sun Nov 26 07:18:43 2023","interfaces":[{"interface":"back","average":{"1min":0.98499999999999999,"5min":0.98499999999999999,"15min":0.98499999999999999},"min":{"1min":0.55100000000000005,"5min":0.55100000000000005,"15min":0.55100000000000005},"max":{"1min":4.4269999999999996,"5min":4.4269999999999996,"15min":4.4269999999999996},"last":0.84299999999999997},{"interface":"front","average":{"1min":1.0149999999999999,"5min":1.0149999999999999,"15min":1.0149999999999999},"min":{"1min":0.60299999999999998,"5min":0.60299999999999998,"15min":0.60299999999999998},"max":{"1min":4.085,"5min":4.085,"15min":4.085},"last":0.746}]},{"osd":5,"last update":"Sun Nov 26 07:18:43 2023","interfaces":[{"interface":"back","average":{"1min":0.874,"5min":0.874,"15min":0.874},"min":{"1min":0.52900000000000003,"5min":0.52900000000000003,"15min":0.52900000000000003},"max":{"1min":3.9590000000000001,"5min":3.9590000000000001,"15min":3.9590000000000001},"last":0.86299999999999999},{"interface":"front","average":{"1min":0.98799999999999999,"5min":0.98799999999999999,"15min":0.98799999999999999},"min":{"1min":0.58799999999999997,"5min":0.58799999999999997,"15min":0.58799999999999997},"max":{"1min":4.3849999999999998,"5min":4.3849999999999998,"15min":4.3849999999999998},"last":0.78800000000000003}]},{"osd":6,"last update":"Sun Nov 26 07:18:43 2023","interfaces":[{"interface":"back","average":{"1min":0.996,"5min":0.996,"15min":0.996},"min":{"1min":0.56599999999999995,"5min":0.56599999999999995,"15min":0.56599999999999995},"max":{"1min":4.0449999999999999,"5min":4.0449999999999999,"15min":4.0449999999999999},"last":0.70099999999999996},{"interface":"front","average":{"1min":0.91800000000000004,"5min":0.91800000000000004,"15min":0.91800000000000004},"min":{"1min":0.57099999999999995,"5min":0.57099999999999995,"15min":0.57099999999999995},"max":{"1min":3.992,"5min":3.992,"15min":3.992},"last":0.80500000000000005}]},{"osd":7,"last update":"Sun Nov 26 07:18:43 2023","interfaces":[{"interface":"back","average":{"1min":1.0349999999999999,"5min":1.0349999999999999,"15min":1.0349999999999999},"min":{"1min":0.63100000000000001,"5min":0.63100000000000001,"15min":0.63100000000000001},"max":{"1min":4.1219999999999999,"5min":4.1219999999999999,"15min":4.1219999999999999},"last":0.82599999999999996},{"interface":"front","average":{"1min":0.93799999999999994,"5min":0.93799999999999994,"15min":0.93799999999999994},"min":{"1min":0.57699999999999996,"5min":0.57699999999999996,"15min":0.57699999999999996},"max":{"1min":3.9159999999999999,"5min":3.9159999999999999,"15min":3.9159999999999999},"last":0.73099999999999998}]}]},{"osd":1,"up_from":13,"seq":55834574867,"num_pgs":32,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27408,"kb_used_data":380,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344432,"statfs":{"total":96636764160,"available":96608698368,"internally_reserved":0,"allocated":389120,"data_stored":148226,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Nov 26 07:18:41 2023","interfaces":[{"interface":"back","average":{"1min":0.41699999999999998,"5min":0.41699999999999998,"15min":0.41699999999999998},"min":{"1min":0.26300000000000001,"5min":0.26300000000000001,"15min":0.26300000000000001},"max":{"1min":0.61199999999999999,"5min":0.61199999999999999,"15min":0.61199999999999999},"last":0.33400000000000002},{"interface":"front","average":{"1min":0.48599999999999999,"5min":0.48599999999999999,"15min":0.48599999999999999},"min":{"1min":0.30499999999999999,"5min":0.30499999999999999,"15min":0.30499999999999999},"max":{"1min":0.78700000000000003,"5min":0.78700000000000003,"15min":0.78700000000000003},"last":0.46400000000000002}]},{"osd":2,"last update":"Sun Nov 26 07:18:41 2023","interfaces":[{"interface":"back","average":{"1min":0.45500000000000002,"5min":0.45500000000000002,"15min":0.45500000000000002},"min":{"1min":0.33400000000000002,"5min":0.33400000000000002,"15min":0.33400000000000002},"max":{"1min":0.67600000000000005,"5min":0.67600000000000005,"15min":0.67600000000000005},"last":0.40100000000000002},{"interface":"front","average":{"1min":0.50800000000000001,"5min":0.50800000000000001,"15min":0.50800000000000001},"min":{"1min":0.32300000000000001,"5min":0.32300000000000001,"15min":0.32300000000000001},"max":{"1min":0.66500000000000004,"5min":0.66500000000000004,"15min":0.66500000000000004},"last":0.56999999999999995}]},{"osd":3,"last update":"Sun Nov 26 07:18:41 2023","interfaces":[{"interface":"back","average":{"1min":0.53000000000000003,"5min":0.53000000000000003,"15min":0.53000000000000003},"min":{"1min":0.38400000000000001,"5min":0.38400000000000001,"15min":0.38400000000000001},"max":{"1min":0.73699999999999999,"5min":0.73699999999999999,"15min":0.73699999999999999},"last":0.54200000000000004},{"interface":"front","average":{"1min":0.497,"5min":0.497,"15min":0.497},"min":{"1min":0.36199999999999999,"5min":0.36199999999999999,"15min":0.36199999999999999},"max":{"1min":0.80700000000000005,"5min":0.80700000000000005,"15min":0.80700000000000005},"last":0.52000000000000002}]},{"osd":4,"last update":"Sun Nov 26 07:18:41 2023","interfaces":[{"interface":"back","average":{"1min":0.69399999999999995,"5min":0.69399999999999995,"15min":0.69399999999999995},"min":{"1min":0.48599999999999999,"5min":0.48599999999999999,"15min":0.48599999999999999},"max":{"1min":1.0229999999999999,"5min":1.0229999999999999,"15min":1.0229999999999999},"last":0.73099999999999998},{"interface":"front","average":{"1min":0.68000000000000005,"5min":0.68000000000000005,"15min":0.68000000000000005},"min":{"1min":0.49199999999999999,"5min":0.49199999999999999,"15min":0.49199999999999999},"max":{"1min":1.0629999999999999,"5min":1.0629999999999999,"15min":1.0629999999999999},"last":0.66100000000000003}]},{"osd":5,"last update":"Sun Nov 26 07:18:41 2023","interfaces":[{"interface":"back","average":{"1min":0.71899999999999997,"5min":0.71899999999999997,"15min":0.71899999999999997},"min":{"1min":0.49299999999999999,"5min":0.49299999999999999,"15min":0.49299999999999999},"max":{"1min":1.0880000000000001,"5min":1.0880000000000001,"15min":1.0880000000000001},"last":0.75800000000000001},{"interface":"front","average":{"1min":0.71699999999999997,"5min":0.71699999999999997,"15min":0.71699999999999997},"min":{"1min":0.53500000000000003,"5min":0.53500000000000003,"15min":0.53500000000000003},"max":{"1min":1.014,"5min":1.014,"15min":1.014},"last":0.745}]},{"osd":6,"last update":"Sun Nov 26 07:18:41 2023","interfaces":[{"interface":"back","average":{"1min":0.70399999999999996,"5min":0.70399999999999996,"15min":0.70399999999999996},"min":{"1min":0.50600000000000001,"5min":0.50600000000000001,"15min":0.50600000000000001},"max":{"1min":0.97599999999999998,"5min":0.97599999999999998,"15min":0.97599999999999998},"last":0.69099999999999995},{"interface":"front","average":{"1min":0.70099999999999996,"5min":0.70099999999999996,"15min":0.70099999999999996},"min":{"1min":0.55000000000000004,"5min":0.55000000000000004,"15min":0.55000000000000004},"max":{"1min":1.1339999999999999,"5min":1.1339999999999999,"15min":1.1339999999999999},"last":0.78200000000000003}]},{"osd":7,"last update":"Sun Nov 26 07:18:41 2023","interfaces":[{"interface":"back","average":{"1min":0.71499999999999997,"5min":0.71499999999999997,"15min":0.71499999999999997},"min":{"1min":0.55100000000000005,"5min":0.55100000000000005,"15min":0.55100000000000005},"max":{"1min":0.92900000000000005,"5min":0.92900000000000005,"15min":0.92900000000000005},"last":0.71699999999999997},{"interface":"front","average":{"1min":0.753,"5min":0.753,"15min":0.753},"min":{"1min":0.56100000000000005,"5min":0.56100000000000005,"15min":0.56100000000000005},"max":{"1min":1.024,"5min":1.024,"15min":1.024},"last":0.84599999999999997}]}]},{"osd":0,"up_from":13,"seq":55834574867,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27968,"kb_used_data":944,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343872,"statfs":{"total":96636764160,"available":96608124928,"internally_reserved":0,"allocated":966656,"data_stored":736960,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":7,"apply_latency_ms":7,"commit_latency_ns":7000000,"apply_latency_ns":7000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Sun Nov 26 07:18:41 2023","interfaces":[{"interface":"back","average":{"1min":0.43099999999999999,"5min":0.43099999999999999,"15min":0.43099999999999999},"min":{"1min":0.28000000000000003,"5min":0.28000000000000003,"15min":0.28000000000000003},"max":{"1min":0.63100000000000001,"5min":0.63100000000000001,"15min":0.63100000000000001},"last":0.379},{"interface":"front","average":{"1min":0.48299999999999998,"5min":0.48299999999999998,"15min":0.48299999999999998},"min":{"1min":0.29899999999999999,"5min":0.29899999999999999,"15min":0.29899999999999999},"max":{"1min":1.133,"5min":1.133,"15min":1.133},"last":0.48699999999999999}]},{"osd":2,"last update":"Sun Nov 26 07:18:41 2023","interfaces":[{"interface":"back","average":{"1min":0.53700000000000003,"5min":0.53700000000000003,"15min":0.53700000000000003},"min":{"1min":0.40899999999999997,"5min":0.40899999999999997,"15min":0.40899999999999997},"max":{"1min":0.67400000000000004,"5min":0.67400000000000004,"15min":0.67400000000000004},"last":0.46100000000000002},{"interface":"front","average":{"1min":0.56399999999999995,"5min":0.56399999999999995,"15min":0.56399999999999995},"min":{"1min":0.36399999999999999,"5min":0.36399999999999999,"15min":0.36399999999999999},"max":{"1min":0.72499999999999998,"5min":0.72499999999999998,"15min":0.72499999999999998},"last":0.39800000000000002}]},{"osd":3,"last update":"Sun Nov 26 07:18:41 2023","interfaces":[{"interface":"back","average":{"1min":0.53100000000000003,"5min":0.53100000000000003,"15min":0.53100000000000003},"min":{"1min":0.35799999999999998,"5min":0.35799999999999998,"15min":0.35799999999999998},"max":{"1min":0.97399999999999998,"5min":0.97399999999999998,"15min":0.97399999999999998},"last":0.54400000000000004},{"interface":"front","average":{"1min":0.56200000000000006,"5min":0.56200000000000006,"15min":0.56200000000000006},"min":{"1min":0.40600000000000003,"5min":0.40600000000000003,"15min":0.40600000000000003},"max":{"1min":1.2509999999999999,"5min":1.2509999999999999,"15min":1.2509999999999999},"last":0.52100000000000002}]},{"osd":4,"last update":"Sun Nov 26 07:18:41 2023","interfaces":[{"interface":"back","average":{"1min":0.75900000000000001,"5min":0.75900000000000001,"15min":0.75900000000000001},"min":{"1min":0.60999999999999999,"5min":0.60999999999999999,"15min":0.60999999999999999},"max":{"1min":1.2010000000000001,"5min":1.2010000000000001,"15min":1.2010000000000001},"last":0.78900000000000003},{"interface":"front","average":{"1min":0.78900000000000003,"5min":0.78900000000000003,"15min":0.78900000000000003},"min":{"1min":0.59999999999999998,"5min":0.59999999999999998,"15min":0.59999999999999998},"max":{"1min":0.99399999999999999,"5min":0.99399999999999999,"15min":0.99399999999999999},"last":0.91800000000000004}]},{"osd":5,"last update":"Sun Nov 26 07:18:41 2023","interfaces":[{"interface":"back","average":{"1min":0.76000000000000001,"5min":0.76000000000000001,"15min":0.76000000000000001},"min":{"1min":0.53000000000000003,"5min":0.53000000000000003,"15min":0.53000000000000003},"max":{"1min":1.1579999999999999,"5min":1.1579999999999999,"15min":1.1579999999999999},"last":0.88800000000000001},{"interface":"front","average":{"1min":0.79700000000000004,"5min":0.79700000000000004,"15min":0.79700000000000004},"min":{"1min":0.59699999999999998,"5min":0.59699999999999998,"15min":0.59699999999999998},"max":{"1min":1.0409999999999999,"5min":1.0409999999999999,"15min":1.0409999999999999},"last":0.82299999999999995}]},{"osd":6,"last update":"Sun Nov 26 07:18:41 2023","interfaces":[{"interface":"back","average":{"1min":0.76400000000000001,"5min":0.76400000000000001,"15min":0.76400000000000001},"min":{"1min":0.56599999999999995,"5min":0.56599999999999995,"15min":0.56599999999999995},"max":{"1min":1.3029999999999999,"5min":1.3029999999999999,"15min":1.3029999999999999},"last":0.76300000000000001},{"interface":"front","average":{"1min":0.76900000000000002,"5min":0.76900000000000002,"15min":0.76900000000000002},"min":{"1min":0.58599999999999997,"5min":0.58599999999999997,"15min":0.58599999999999997},"max":{"1min":1.2210000000000001,"5min":1.2210000000000001,"15min":1.2210000000000001},"last":0.66400000000000003}]},{"osd":7,"last update":"Sun Nov 26 07:18:41 2023","interfaces":[{"interface":"back","average":{"1min":0.81100000000000005,"5min":0.81100000000000005,"15min":0.81100000000000005},"min":{"1min":0.59999999999999998,"5min":0.59999999999999998,"15min":0.59999999999999998},"max":{"1min":1.177,"5min":1.177,"15min":1.177},"last":0.81200000000000006},{"interface":"front","average":{"1min":0.79900000000000004,"5min":0.79900000000000004,"15min":0.79900000000000004},"min":{"1min":0.60899999999999999,"5min":0.60899999999999999,"15min":0.60899999999999999},"max":{"1min":1.2390000000000001,"5min":1.2390000000000001,"15min":1.2390000000000001},"last":0.71299999999999997}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1634,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":70,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1634,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":104,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":672,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":616,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-11-26T07:19:01.248 DEBUG:teuthology.orchestra.run.smithi143:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-11-26T07:19:01.389 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:01.384+0000 7f78fca5a640 1 Processor -- start 2023-11-26T07:19:01.389 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:01.384+0000 7f78fca5a640 1 -- start start 2023-11-26T07:19:01.391 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:01.384+0000 7f78fca5a640 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7f78f815d6e0 0x7f78f80d0df0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:19:01.391 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:01.384+0000 7f78fca5a640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f78f80d1bc0 0x7f78f80b0750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:19:01.391 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:01.384+0000 7f78fca5a640 1 -- --> v1:172.21.15.188:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f78f80cf0e0 con 0x7f78f80b2160 2023-11-26T07:19:01.391 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:01.384+0000 7f78fca5a640 1 -- --> v2:172.21.15.143:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f78f815dac0 con 0x7f78f815d6e0 2023-11-26T07:19:01.391 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:01.384+0000 7f78fca5a640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f78f80ceb20 con 0x7f78f80d1bc0 2023-11-26T07:19:01.391 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:01.384+0000 7f78f659c640 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7f78f815d6e0 0x7f78f80d0df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:19:01.391 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:01.384+0000 7f78f659c640 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7f78f815d6e0 0x7f78f80d0df0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:51788/0 (socket says 172.21.15.143:51788) 2023-11-26T07:19:01.392 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:01.384+0000 7f78f659c640 1 -- 172.21.15.143:0/4108478212 learned_addr learned my addr 172.21.15.143:0/4108478212 (peer_addr_for_me v2:172.21.15.143:0/0) 2023-11-26T07:19:01.392 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:01.384+0000 7f78f6d9d640 1 --2- 172.21.15.143:0/4108478212 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f78f80d1bc0 0x7f78f80b0750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:19:01.392 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:01.384+0000 7f78f659c640 1 -- 172.21.15.143:0/4108478212 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f78f80d1bc0 msgr2=0x7f78f80b0750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-26T07:19:01.392 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:01.384+0000 7f78f659c640 1 --2- 172.21.15.143:0/4108478212 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f78f80d1bc0 0x7f78f80b0750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:19:01.392 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:01.384+0000 7f78f659c640 1 -- 172.21.15.143:0/4108478212 >> v1:172.21.15.188:6789/0 conn(0x7f78f80b2160 legacy=0x7f78f80d14b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:19:01.392 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:01.384+0000 7f78f659c640 1 -- 172.21.15.143:0/4108478212 --> v2:172.21.15.143:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f78f80cee00 con 0x7f78f815d6e0 2023-11-26T07:19:01.394 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:01.388+0000 7f78f659c640 1 --2- 172.21.15.143:0/4108478212 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f78f815d6e0 0x7f78f80d0df0 secure :-1 s=READY pgs=423 cs=0 l=1 rev1=1 crypto rx=0x7f78e8009870 tx=0x7f78e801c540 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1ca855226c39a285 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:19:01.394 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:01.388+0000 7f78f559a640 1 -- 172.21.15.143:0/4108478212 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f78e80a7070 con 0x7f78f815d6e0 2023-11-26T07:19:01.394 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:01.388+0000 7f78f559a640 1 -- 172.21.15.143:0/4108478212 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f78e80ab430 con 0x7f78f815d6e0 2023-11-26T07:19:01.394 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:01.388+0000 7f78f559a640 1 -- 172.21.15.143:0/4108478212 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f78e80ab7c0 con 0x7f78f815d6e0 2023-11-26T07:19:01.395 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:01.388+0000 7f78fca5a640 1 -- 172.21.15.143:0/4108478212 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f78f815d6e0 msgr2=0x7f78f80d0df0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:19:01.395 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:01.388+0000 7f78fca5a640 1 --2- 172.21.15.143:0/4108478212 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f78f815d6e0 0x7f78f80d0df0 secure :-1 s=READY pgs=423 cs=0 l=1 rev1=1 crypto rx=0x7f78e8009870 tx=0x7f78e801c540 comp rx=0 tx=0).stop 2023-11-26T07:19:01.395 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:01.388+0000 7f78fca5a640 1 -- 172.21.15.143:0/4108478212 shutdown_connections 2023-11-26T07:19:01.395 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:01.388+0000 7f78fca5a640 1 --2- 172.21.15.143:0/4108478212 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f78f80d1bc0 0x7f78f80b0750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:19:01.395 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:01.388+0000 7f78fca5a640 1 --2- 172.21.15.143:0/4108478212 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f78f815d6e0 0x7f78f80d0df0 unknown :-1 s=CLOSED pgs=423 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:19:01.395 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:01.388+0000 7f78fca5a640 1 -- 172.21.15.143:0/4108478212 >> 172.21.15.143:0/4108478212 conn(0x7f78f809c550 msgr2=0x7f78f80d3220 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:19:01.396 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:01.388+0000 7f78fca5a640 1 -- 172.21.15.143:0/4108478212 shutdown_connections 2023-11-26T07:19:01.396 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:01.388+0000 7f78fca5a640 1 -- 172.21.15.143:0/4108478212 wait complete. 2023-11-26T07:19:01.397 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:01.392+0000 7f78fca5a640 1 Processor -- start 2023-11-26T07:19:01.397 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:01.392+0000 7f78fca5a640 1 -- start start 2023-11-26T07:19:01.399 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:01.392+0000 7f78fca5a640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f78f80b2160 0x7f78f815bbd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:19:01.399 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:01.392+0000 7f78fca5a640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f78f80d1bc0 0x7f78f815dc00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:19:01.399 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:01.392+0000 7f78fca5a640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f78f815d6e0 0x7f78f815e140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:19:01.399 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:01.392+0000 7f78fca5a640 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f78f815c110 con 0x7f78f80d1bc0 2023-11-26T07:19:01.399 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:01.392+0000 7f78fca5a640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f78f815ab30 con 0x7f78f80b2160 2023-11-26T07:19:01.399 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:01.392+0000 7f78fca5a640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f78f815ae10 con 0x7f78f815d6e0 2023-11-26T07:19:01.400 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:01.392+0000 7f78f5d9b640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f78f80d1bc0 0x7f78f815dc00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:19:01.400 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:01.392+0000 7f78f5d9b640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f78f80d1bc0 0x7f78f815dc00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:51798/0 (socket says 172.21.15.143:51798) 2023-11-26T07:19:01.400 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:01.392+0000 7f78f5d9b640 1 -- 172.21.15.143:0/187073197 learned_addr learned my addr 172.21.15.143:0/187073197 (peer_addr_for_me v2:172.21.15.143:0/0) 2023-11-26T07:19:01.400 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:01.392+0000 7f78f659c640 1 --2- 172.21.15.143:0/187073197 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f78f80b2160 0x7f78f815bbd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:19:01.400 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:01.392+0000 7f78f6d9d640 1 --2- 172.21.15.143:0/187073197 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f78f815d6e0 0x7f78f815e140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:19:01.400 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:01.392+0000 7f78f5d9b640 1 -- 172.21.15.143:0/187073197 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f78f815d6e0 msgr2=0x7f78f815e140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:19:01.400 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:01.392+0000 7f78f5d9b640 1 --2- 172.21.15.143:0/187073197 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f78f815d6e0 0x7f78f815e140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:19:01.400 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:01.392+0000 7f78f5d9b640 1 -- 172.21.15.143:0/187073197 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f78f80b2160 msgr2=0x7f78f815bbd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:19:01.400 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:01.392+0000 7f78f5d9b640 1 --2- 172.21.15.143:0/187073197 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f78f80b2160 0x7f78f815bbd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:19:01.400 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:01.392+0000 7f78f5d9b640 1 -- 172.21.15.143:0/187073197 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f78e80094d0 con 0x7f78f80d1bc0 2023-11-26T07:19:01.400 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:01.392+0000 7f78f5d9b640 1 --2- 172.21.15.143:0/187073197 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f78f80d1bc0 0x7f78f815dc00 secure :-1 s=READY pgs=424 cs=0 l=1 rev1=1 crypto rx=0x7f78ec00c500 tx=0x7f78ec00c9c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:19:01.400 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:01.392+0000 7f78e6ffd640 1 -- 172.21.15.143:0/187073197 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f78ec005220 con 0x7f78f80d1bc0 2023-11-26T07:19:01.400 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:01.392+0000 7f78e6ffd640 1 -- 172.21.15.143:0/187073197 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f78ec005380 con 0x7f78f80d1bc0 2023-11-26T07:19:01.401 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:01.392+0000 7f78e6ffd640 1 -- 172.21.15.143:0/187073197 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f78ec007270 con 0x7f78f80d1bc0 2023-11-26T07:19:01.401 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:01.392+0000 7f78fca5a640 1 -- 172.21.15.143:0/187073197 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f78f815b0f0 con 0x7f78f80d1bc0 2023-11-26T07:19:01.401 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:01.392+0000 7f78fca5a640 1 -- 172.21.15.143:0/187073197 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f78f815b6b0 con 0x7f78f80d1bc0 2023-11-26T07:19:01.401 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:01.392+0000 7f78fca5a640 1 -- 172.21.15.143:0/187073197 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f78f8067680 con 0x7f78f80d1bc0 2023-11-26T07:19:01.401 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:01.396+0000 7f78e6ffd640 1 -- 172.21.15.143:0/187073197 <== mon.0 v2:172.21.15.143:3300/0 4 ==== mgrmap(e 5) v1 ==== 85852+0+0 (secure 0 0 0) 0x7f78ec0073d0 con 0x7f78f80d1bc0 2023-11-26T07:19:01.402 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:01.396+0000 7f78e6ffd640 1 --2- 172.21.15.143:0/187073197 >> [v2:172.21.15.143:6832/2150557544,v1:172.21.15.143:6833/2150557544] conn(0x7f78d8066470 0x7f78d8068930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:19:01.402 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:01.396+0000 7f78e6ffd640 1 -- 172.21.15.143:0/187073197 <== mon.0 v2:172.21.15.143:3300/0 5 ==== osd_map(33..33 src has 1..33) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f78ec083680 con 0x7f78f80d1bc0 2023-11-26T07:19:01.402 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:01.396+0000 7f78f659c640 1 --2- 172.21.15.143:0/187073197 >> [v2:172.21.15.143:6832/2150557544,v1:172.21.15.143:6833/2150557544] conn(0x7f78d8066470 0x7f78d8068930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:19:01.403 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:01.396+0000 7f78f659c640 1 --2- 172.21.15.143:0/187073197 >> [v2:172.21.15.143:6832/2150557544,v1:172.21.15.143:6833/2150557544] conn(0x7f78d8066470 0x7f78d8068930 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7f78e80af000 tx=0x7f78e8008040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:19:01.406 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:01.400+0000 7f78e6ffd640 1 -- 172.21.15.143:0/187073197 <== mon.0 v2:172.21.15.143:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172649 (secure 0 0 0) 0x7f78ec050b20 con 0x7f78f80d1bc0 2023-11-26T07:19:01.553 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:01.548+0000 7f78fca5a640 1 -- 172.21.15.143:0/187073197 --> [v2:172.21.15.143:6832/2150557544,v1:172.21.15.143:6833/2150557544] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f78f80d1130 con 0x7f78d8066470 2023-11-26T07:19:01.560 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:01.552+0000 7f78e6ffd640 1 -- 172.21.15.143:0/187073197 <== mgr.4100 v2:172.21.15.143:6832/2150557544 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+289780 (secure 0 0 0) 0x7f78f80d1130 con 0x7f78d8066470 2023-11-26T07:19:01.563 INFO:teuthology.orchestra.run.smithi143.stdout: 2023-11-26T07:19:01.565 INFO:teuthology.orchestra.run.smithi143.stderr:dumped all 2023-11-26T07:19:01.566 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:01.556+0000 7f78fca5a640 1 -- 172.21.15.143:0/187073197 >> [v2:172.21.15.143:6832/2150557544,v1:172.21.15.143:6833/2150557544] conn(0x7f78d8066470 msgr2=0x7f78d8068930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:19:01.566 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:01.556+0000 7f78fca5a640 1 --2- 172.21.15.143:0/187073197 >> [v2:172.21.15.143:6832/2150557544,v1:172.21.15.143:6833/2150557544] conn(0x7f78d8066470 0x7f78d8068930 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7f78e80af000 tx=0x7f78e8008040 comp rx=0 tx=0).stop 2023-11-26T07:19:01.566 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:01.556+0000 7f78fca5a640 1 -- 172.21.15.143:0/187073197 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f78f80d1bc0 msgr2=0x7f78f815dc00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:19:01.566 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:01.556+0000 7f78fca5a640 1 --2- 172.21.15.143:0/187073197 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f78f80d1bc0 0x7f78f815dc00 secure :-1 s=READY pgs=424 cs=0 l=1 rev1=1 crypto rx=0x7f78ec00c500 tx=0x7f78ec00c9c0 comp rx=0 tx=0).stop 2023-11-26T07:19:01.566 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:01.556+0000 7f78fca5a640 1 -- 172.21.15.143:0/187073197 shutdown_connections 2023-11-26T07:19:01.566 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:01.556+0000 7f78fca5a640 1 --2- 172.21.15.143:0/187073197 >> [v2:172.21.15.143:6832/2150557544,v1:172.21.15.143:6833/2150557544] conn(0x7f78d8066470 0x7f78d8068930 unknown :-1 s=CLOSED pgs=227 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:19:01.566 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:01.556+0000 7f78fca5a640 1 --2- 172.21.15.143:0/187073197 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f78f815d6e0 0x7f78f815e140 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:19:01.567 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:01.556+0000 7f78fca5a640 1 --2- 172.21.15.143:0/187073197 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f78f80d1bc0 0x7f78f815dc00 unknown :-1 s=CLOSED pgs=424 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:19:01.567 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:01.556+0000 7f78fca5a640 1 --2- 172.21.15.143:0/187073197 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f78f80b2160 0x7f78f815bbd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:19:01.567 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:01.556+0000 7f78fca5a640 1 -- 172.21.15.143:0/187073197 >> 172.21.15.143:0/187073197 conn(0x7f78f809c550 msgr2=0x7f78f806c690 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:19:01.567 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:01.556+0000 7f78fca5a640 1 -- 172.21.15.143:0/187073197 shutdown_connections 2023-11-26T07:19:01.567 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:01.556+0000 7f78fca5a640 1 -- 172.21.15.143:0/187073197 wait complete. 2023-11-26T07:19:01.583 INFO:teuthology.orchestra.run.smithi143.stdout:{"pg_ready":true,"pg_map":{"version":65,"stamp":"2023-11-26T07:18:59.706168+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592778,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":204,"num_write_kb":2430,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":156,"ondisk_log_size":156,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":161,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":220308,"kb_used_data":4136,"kb_used_omap":12,"kb_used_meta":216051,"kb_avail":754754412,"statfs":{"total":773094113280,"available":772868517888,"internally_reserved":0,"allocated":4235264,"data_stored":2358292,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12749,"internal_metadata":221236787},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":50,"apply_latency_ms":50,"commit_latency_ns":50000000,"apply_latency_ns":50000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"8.001364"},"pg_stats":[{"pgid":"5.3f","version":"0'0","reported_seq":11,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-11-26T07:18:54.776405+0000","last_change":"2023-11-26T07:18:52.685323+0000","last_active":"2023-11-26T07:18:54.776405+0000","last_peered":"2023-11-26T07:18:54.776405+0000","last_clean":"2023-11-26T07:18:54.776405+0000","last_became_active":"2023-11-26T07:18:52.685015+0000","last_became_peered":"2023-11-26T07:18:52.685015+0000","last_unstale":"2023-11-26T07:18:54.776405+0000","last_undegraded":"2023-11-26T07:18:54.776405+0000","last_fullsized":"2023-11-26T07:18:54.776405+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:18:51.625829+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:18:51.625829+0000","last_clean_scrub_stamp":"2023-11-26T07:18:51.625829+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T10:33:50.109555+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.3e","version":"0'0","reported_seq":11,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-11-26T07:18:54.777777+0000","last_change":"2023-11-26T07:18:52.661591+0000","last_active":"2023-11-26T07:18:54.777777+0000","last_peered":"2023-11-26T07:18:54.777777+0000","last_clean":"2023-11-26T07:18:54.777777+0000","last_became_active":"2023-11-26T07:18:52.661241+0000","last_became_peered":"2023-11-26T07:18:52.661241+0000","last_unstale":"2023-11-26T07:18:54.777777+0000","last_undegraded":"2023-11-26T07:18:54.777777+0000","last_fullsized":"2023-11-26T07:18:54.777777+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:18:51.625829+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:18:51.625829+0000","last_clean_scrub_stamp":"2023-11-26T07:18:51.625829+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T07:35:48.362218+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.3d","version":"0'0","reported_seq":11,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-11-26T07:18:54.776648+0000","last_change":"2023-11-26T07:18:52.674764+0000","last_active":"2023-11-26T07:18:54.776648+0000","last_peered":"2023-11-26T07:18:54.776648+0000","last_clean":"2023-11-26T07:18:54.776648+0000","last_became_active":"2023-11-26T07:18:52.674530+0000","last_became_peered":"2023-11-26T07:18:52.674530+0000","last_unstale":"2023-11-26T07:18:54.776648+0000","last_undegraded":"2023-11-26T07:18:54.776648+0000","last_fullsized":"2023-11-26T07:18:54.776648+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:18:51.625829+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:18:51.625829+0000","last_clean_scrub_stamp":"2023-11-26T07:18:51.625829+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T13:59:11.780648+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.3c","version":"0'0","reported_seq":11,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-11-26T07:18:53.665727+0000","last_change":"2023-11-26T07:18:52.685609+0000","last_active":"2023-11-26T07:18:53.665727+0000","last_peered":"2023-11-26T07:18:53.665727+0000","last_clean":"2023-11-26T07:18:53.665727+0000","last_became_active":"2023-11-26T07:18:52.685320+0000","last_became_peered":"2023-11-26T07:18:52.685320+0000","last_unstale":"2023-11-26T07:18:53.665727+0000","last_undegraded":"2023-11-26T07:18:53.665727+0000","last_fullsized":"2023-11-26T07:18:53.665727+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:18:51.625829+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:18:51.625829+0000","last_clean_scrub_stamp":"2023-11-26T07:18:51.625829+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T08:47:52.417322+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.3b","version":"0'0","reported_seq":11,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-11-26T07:18:54.776752+0000","last_change":"2023-11-26T07:18:52.666834+0000","last_active":"2023-11-26T07:18:54.776752+0000","last_peered":"2023-11-26T07:18:54.776752+0000","last_clean":"2023-11-26T07:18:54.776752+0000","last_became_active":"2023-11-26T07:18:52.666516+0000","last_became_peered":"2023-11-26T07:18:52.666516+0000","last_unstale":"2023-11-26T07:18:54.776752+0000","last_undegraded":"2023-11-26T07:18:54.776752+0000","last_fullsized":"2023-11-26T07:18:54.776752+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:18:51.625829+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:18:51.625829+0000","last_clean_scrub_stamp":"2023-11-26T07:18:51.625829+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T19:13:00.194342+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.3a","version":"0'0","reported_seq":11,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-11-26T07:18:54.777241+0000","last_change":"2023-11-26T07:18:52.669715+0000","last_active":"2023-11-26T07:18:54.777241+0000","last_peered":"2023-11-26T07:18:54.777241+0000","last_clean":"2023-11-26T07:18:54.777241+0000","last_became_active":"2023-11-26T07:18:52.669328+0000","last_became_peered":"2023-11-26T07:18:52.669328+0000","last_unstale":"2023-11-26T07:18:54.777241+0000","last_undegraded":"2023-11-26T07:18:54.777241+0000","last_fullsized":"2023-11-26T07:18:54.777241+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:18:51.625829+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:18:51.625829+0000","last_clean_scrub_stamp":"2023-11-26T07:18:51.625829+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T08:36:31.952917+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.39","version":"0'0","reported_seq":11,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-11-26T07:18:54.776755+0000","last_change":"2023-11-26T07:18:52.686894+0000","last_active":"2023-11-26T07:18:54.776755+0000","last_peered":"2023-11-26T07:18:54.776755+0000","last_clean":"2023-11-26T07:18:54.776755+0000","last_became_active":"2023-11-26T07:18:52.686521+0000","last_became_peered":"2023-11-26T07:18:52.686521+0000","last_unstale":"2023-11-26T07:18:54.776755+0000","last_undegraded":"2023-11-26T07:18:54.776755+0000","last_fullsized":"2023-11-26T07:18:54.776755+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:18:51.625829+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:18:51.625829+0000","last_clean_scrub_stamp":"2023-11-26T07:18:51.625829+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T12:03:00.136302+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.38","version":"0'0","reported_seq":11,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-11-26T07:18:54.777511+0000","last_change":"2023-11-26T07:18:52.690560+0000","last_active":"2023-11-26T07:18:54.777511+0000","last_peered":"2023-11-26T07:18:54.777511+0000","last_clean":"2023-11-26T07:18:54.777511+0000","last_became_active":"2023-11-26T07:18:52.690283+0000","last_became_peered":"2023-11-26T07:18:52.690283+0000","last_unstale":"2023-11-26T07:18:54.777511+0000","last_undegraded":"2023-11-26T07:18:54.777511+0000","last_fullsized":"2023-11-26T07:18:54.777511+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:18:51.625829+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:18:51.625829+0000","last_clean_scrub_stamp":"2023-11-26T07:18:51.625829+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T08:28:20.572561+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.37","version":"0'0","reported_seq":11,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-11-26T07:18:54.777644+0000","last_change":"2023-11-26T07:18:52.682711+0000","last_active":"2023-11-26T07:18:54.777644+0000","last_peered":"2023-11-26T07:18:54.777644+0000","last_clean":"2023-11-26T07:18:54.777644+0000","last_became_active":"2023-11-26T07:18:52.682295+0000","last_became_peered":"2023-11-26T07:18:52.682295+0000","last_unstale":"2023-11-26T07:18:54.777644+0000","last_undegraded":"2023-11-26T07:18:54.777644+0000","last_fullsized":"2023-11-26T07:18:54.777644+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:18:51.625829+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:18:51.625829+0000","last_clean_scrub_stamp":"2023-11-26T07:18:51.625829+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T09:38:45.595955+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.36","version":"0'0","reported_seq":11,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-11-26T07:18:53.666016+0000","last_change":"2023-11-26T07:18:52.681413+0000","last_active":"2023-11-26T07:18:53.666016+0000","last_peered":"2023-11-26T07:18:53.666016+0000","last_clean":"2023-11-26T07:18:53.666016+0000","last_became_active":"2023-11-26T07:18:52.680984+0000","last_became_peered":"2023-11-26T07:18:52.680984+0000","last_unstale":"2023-11-26T07:18:53.666016+0000","last_undegraded":"2023-11-26T07:18:53.666016+0000","last_fullsized":"2023-11-26T07:18:53.666016+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:18:51.625829+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:18:51.625829+0000","last_clean_scrub_stamp":"2023-11-26T07:18:51.625829+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T17:29:13.304698+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.35","version":"0'0","reported_seq":11,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-11-26T07:18:54.777930+0000","last_change":"2023-11-26T07:18:52.676152+0000","last_active":"2023-11-26T07:18:54.777930+0000","last_peered":"2023-11-26T07:18:54.777930+0000","last_clean":"2023-11-26T07:18:54.777930+0000","last_became_active":"2023-11-26T07:18:52.675857+0000","last_became_peered":"2023-11-26T07:18:52.675857+0000","last_unstale":"2023-11-26T07:18:54.777930+0000","last_undegraded":"2023-11-26T07:18:54.777930+0000","last_fullsized":"2023-11-26T07:18:54.777930+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:18:51.625829+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:18:51.625829+0000","last_clean_scrub_stamp":"2023-11-26T07:18:51.625829+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T12:10:13.607538+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.34","version":"0'0","reported_seq":11,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-11-26T07:18:54.777008+0000","last_change":"2023-11-26T07:18:52.689871+0000","last_active":"2023-11-26T07:18:54.777008+0000","last_peered":"2023-11-26T07:18:54.777008+0000","last_clean":"2023-11-26T07:18:54.777008+0000","last_became_active":"2023-11-26T07:18:52.689612+0000","last_became_peered":"2023-11-26T07:18:52.689612+0000","last_unstale":"2023-11-26T07:18:54.777008+0000","last_undegraded":"2023-11-26T07:18:54.777008+0000","last_fullsized":"2023-11-26T07:18:54.777008+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:18:51.625829+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:18:51.625829+0000","last_clean_scrub_stamp":"2023-11-26T07:18:51.625829+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T17:23:21.873210+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.33","version":"0'0","reported_seq":11,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-11-26T07:18:54.776822+0000","last_change":"2023-11-26T07:18:52.685975+0000","last_active":"2023-11-26T07:18:54.776822+0000","last_peered":"2023-11-26T07:18:54.776822+0000","last_clean":"2023-11-26T07:18:54.776822+0000","last_became_active":"2023-11-26T07:18:52.685643+0000","last_became_peered":"2023-11-26T07:18:52.685643+0000","last_unstale":"2023-11-26T07:18:54.776822+0000","last_undegraded":"2023-11-26T07:18:54.776822+0000","last_fullsized":"2023-11-26T07:18:54.776822+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:18:51.625829+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:18:51.625829+0000","last_clean_scrub_stamp":"2023-11-26T07:18:51.625829+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T13:37:16.389834+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.32","version":"0'0","reported_seq":11,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-11-26T07:18:54.777317+0000","last_change":"2023-11-26T07:18:52.665807+0000","last_active":"2023-11-26T07:18:54.777317+0000","last_peered":"2023-11-26T07:18:54.777317+0000","last_clean":"2023-11-26T07:18:54.777317+0000","last_became_active":"2023-11-26T07:18:52.665536+0000","last_became_peered":"2023-11-26T07:18:52.665536+0000","last_unstale":"2023-11-26T07:18:54.777317+0000","last_undegraded":"2023-11-26T07:18:54.777317+0000","last_fullsized":"2023-11-26T07:18:54.777317+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:18:51.625829+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:18:51.625829+0000","last_clean_scrub_stamp":"2023-11-26T07:18:51.625829+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T08:28:20.572561+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.31","version":"0'0","reported_seq":11,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-11-26T07:18:54.776584+0000","last_change":"2023-11-26T07:18:52.685374+0000","last_active":"2023-11-26T07:18:54.776584+0000","last_peered":"2023-11-26T07:18:54.776584+0000","last_clean":"2023-11-26T07:18:54.776584+0000","last_became_active":"2023-11-26T07:18:52.685003+0000","last_became_peered":"2023-11-26T07:18:52.685003+0000","last_unstale":"2023-11-26T07:18:54.776584+0000","last_undegraded":"2023-11-26T07:18:54.776584+0000","last_fullsized":"2023-11-26T07:18:54.776584+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:18:51.625829+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:18:51.625829+0000","last_clean_scrub_stamp":"2023-11-26T07:18:51.625829+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T14:18:07.188933+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.30","version":"0'0","reported_seq":11,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-11-26T07:18:54.777803+0000","last_change":"2023-11-26T07:18:52.682405+0000","last_active":"2023-11-26T07:18:54.777803+0000","last_peered":"2023-11-26T07:18:54.777803+0000","last_clean":"2023-11-26T07:18:54.777803+0000","last_became_active":"2023-11-26T07:18:52.682113+0000","last_became_peered":"2023-11-26T07:18:52.682113+0000","last_unstale":"2023-11-26T07:18:54.777803+0000","last_undegraded":"2023-11-26T07:18:54.777803+0000","last_fullsized":"2023-11-26T07:18:54.777803+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:18:51.625829+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:18:51.625829+0000","last_clean_scrub_stamp":"2023-11-26T07:18:51.625829+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T07:35:48.362218+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.2f","version":"0'0","reported_seq":11,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-11-26T07:18:54.778013+0000","last_change":"2023-11-26T07:18:52.688334+0000","last_active":"2023-11-26T07:18:54.778013+0000","last_peered":"2023-11-26T07:18:54.778013+0000","last_clean":"2023-11-26T07:18:54.778013+0000","last_became_active":"2023-11-26T07:18:52.688029+0000","last_became_peered":"2023-11-26T07:18:52.688029+0000","last_unstale":"2023-11-26T07:18:54.778013+0000","last_undegraded":"2023-11-26T07:18:54.778013+0000","last_fullsized":"2023-11-26T07:18:54.778013+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:18:51.625829+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:18:51.625829+0000","last_clean_scrub_stamp":"2023-11-26T07:18:51.625829+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T08:49:07.031743+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.2e","version":"0'0","reported_seq":11,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-11-26T07:18:54.777813+0000","last_change":"2023-11-26T07:18:52.680334+0000","last_active":"2023-11-26T07:18:54.777813+0000","last_peered":"2023-11-26T07:18:54.777813+0000","last_clean":"2023-11-26T07:18:54.777813+0000","last_became_active":"2023-11-26T07:18:52.679769+0000","last_became_peered":"2023-11-26T07:18:52.679769+0000","last_unstale":"2023-11-26T07:18:54.777813+0000","last_undegraded":"2023-11-26T07:18:54.777813+0000","last_fullsized":"2023-11-26T07:18:54.777813+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:18:51.625829+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:18:51.625829+0000","last_clean_scrub_stamp":"2023-11-26T07:18:51.625829+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T12:10:13.607538+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.2d","version":"0'0","reported_seq":11,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-11-26T07:18:54.777458+0000","last_change":"2023-11-26T07:18:52.680533+0000","last_active":"2023-11-26T07:18:54.777458+0000","last_peered":"2023-11-26T07:18:54.777458+0000","last_clean":"2023-11-26T07:18:54.777458+0000","last_became_active":"2023-11-26T07:18:52.680250+0000","last_became_peered":"2023-11-26T07:18:52.680250+0000","last_unstale":"2023-11-26T07:18:54.777458+0000","last_undegraded":"2023-11-26T07:18:54.777458+0000","last_fullsized":"2023-11-26T07:18:54.777458+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:18:51.625829+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:18:51.625829+0000","last_clean_scrub_stamp":"2023-11-26T07:18:51.625829+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T12:03:00.136302+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.2c","version":"0'0","reported_seq":11,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-11-26T07:18:54.777021+0000","last_change":"2023-11-26T07:18:52.679421+0000","last_active":"2023-11-26T07:18:54.777021+0000","last_peered":"2023-11-26T07:18:54.777021+0000","last_clean":"2023-11-26T07:18:54.777021+0000","last_became_active":"2023-11-26T07:18:52.677732+0000","last_became_peered":"2023-11-26T07:18:52.677732+0000","last_unstale":"2023-11-26T07:18:54.777021+0000","last_undegraded":"2023-11-26T07:18:54.777021+0000","last_fullsized":"2023-11-26T07:18:54.777021+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:18:51.625829+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:18:51.625829+0000","last_clean_scrub_stamp":"2023-11-26T07:18:51.625829+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T12:04:40.481383+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.2b","version":"0'0","reported_seq":11,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-11-26T07:18:54.777011+0000","last_change":"2023-11-26T07:18:52.666128+0000","last_active":"2023-11-26T07:18:54.777011+0000","last_peered":"2023-11-26T07:18:54.777011+0000","last_clean":"2023-11-26T07:18:54.777011+0000","last_became_active":"2023-11-26T07:18:52.665802+0000","last_became_peered":"2023-11-26T07:18:52.665802+0000","last_unstale":"2023-11-26T07:18:54.777011+0000","last_undegraded":"2023-11-26T07:18:54.777011+0000","last_fullsized":"2023-11-26T07:18:54.777011+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:18:51.625829+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:18:51.625829+0000","last_clean_scrub_stamp":"2023-11-26T07:18:51.625829+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T08:30:11.556027+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.2a","version":"0'0","reported_seq":11,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-11-26T07:18:54.777828+0000","last_change":"2023-11-26T07:18:52.672815+0000","last_active":"2023-11-26T07:18:54.777828+0000","last_peered":"2023-11-26T07:18:54.777828+0000","last_clean":"2023-11-26T07:18:54.777828+0000","last_became_active":"2023-11-26T07:18:52.667693+0000","last_became_peered":"2023-11-26T07:18:52.667693+0000","last_unstale":"2023-11-26T07:18:54.777828+0000","last_undegraded":"2023-11-26T07:18:54.777828+0000","last_fullsized":"2023-11-26T07:18:54.777828+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:18:51.625829+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:18:51.625829+0000","last_clean_scrub_stamp":"2023-11-26T07:18:51.625829+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T12:40:28.638833+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.29","version":"0'0","reported_seq":11,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-11-26T07:18:53.663848+0000","last_change":"2023-11-26T07:18:52.672298+0000","last_active":"2023-11-26T07:18:53.663848+0000","last_peered":"2023-11-26T07:18:53.663848+0000","last_clean":"2023-11-26T07:18:53.663848+0000","last_became_active":"2023-11-26T07:18:52.670876+0000","last_became_peered":"2023-11-26T07:18:52.670876+0000","last_unstale":"2023-11-26T07:18:53.663848+0000","last_undegraded":"2023-11-26T07:18:53.663848+0000","last_fullsized":"2023-11-26T07:18:53.663848+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:18:51.625829+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:18:51.625829+0000","last_clean_scrub_stamp":"2023-11-26T07:18:51.625829+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T13:18:27.001149+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.28","version":"0'0","reported_seq":11,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-11-26T07:18:54.775585+0000","last_change":"2023-11-26T07:18:52.691983+0000","last_active":"2023-11-26T07:18:54.775585+0000","last_peered":"2023-11-26T07:18:54.775585+0000","last_clean":"2023-11-26T07:18:54.775585+0000","last_became_active":"2023-11-26T07:18:52.691661+0000","last_became_peered":"2023-11-26T07:18:52.691661+0000","last_unstale":"2023-11-26T07:18:54.775585+0000","last_undegraded":"2023-11-26T07:18:54.775585+0000","last_fullsized":"2023-11-26T07:18:54.775585+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:18:51.625829+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:18:51.625829+0000","last_clean_scrub_stamp":"2023-11-26T07:18:51.625829+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T08:47:52.417322+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.27","version":"0'0","reported_seq":11,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-11-26T07:18:53.666090+0000","last_change":"2023-11-26T07:18:52.676572+0000","last_active":"2023-11-26T07:18:53.666090+0000","last_peered":"2023-11-26T07:18:53.666090+0000","last_clean":"2023-11-26T07:18:53.666090+0000","last_became_active":"2023-11-26T07:18:52.676288+0000","last_became_peered":"2023-11-26T07:18:52.676288+0000","last_unstale":"2023-11-26T07:18:53.666090+0000","last_undegraded":"2023-11-26T07:18:53.666090+0000","last_fullsized":"2023-11-26T07:18:53.666090+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:18:51.625829+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:18:51.625829+0000","last_clean_scrub_stamp":"2023-11-26T07:18:51.625829+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T14:18:07.188933+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.26","version":"0'0","reported_seq":11,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-11-26T07:18:54.777903+0000","last_change":"2023-11-26T07:18:52.665634+0000","last_active":"2023-11-26T07:18:54.777903+0000","last_peered":"2023-11-26T07:18:54.777903+0000","last_clean":"2023-11-26T07:18:54.777903+0000","last_became_active":"2023-11-26T07:18:52.665348+0000","last_became_peered":"2023-11-26T07:18:52.665348+0000","last_unstale":"2023-11-26T07:18:54.777903+0000","last_undegraded":"2023-11-26T07:18:54.777903+0000","last_fullsized":"2023-11-26T07:18:54.777903+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:18:51.625829+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:18:51.625829+0000","last_clean_scrub_stamp":"2023-11-26T07:18:51.625829+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T08:49:07.031743+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.25","version":"0'0","reported_seq":11,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-11-26T07:18:53.665947+0000","last_change":"2023-11-26T07:18:52.687999+0000","last_active":"2023-11-26T07:18:53.665947+0000","last_peered":"2023-11-26T07:18:53.665947+0000","last_clean":"2023-11-26T07:18:53.665947+0000","last_became_active":"2023-11-26T07:18:52.687720+0000","last_became_peered":"2023-11-26T07:18:52.687720+0000","last_unstale":"2023-11-26T07:18:53.665947+0000","last_undegraded":"2023-11-26T07:18:53.665947+0000","last_fullsized":"2023-11-26T07:18:53.665947+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:18:51.625829+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:18:51.625829+0000","last_clean_scrub_stamp":"2023-11-26T07:18:51.625829+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T09:49:42.889030+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.24","version":"0'0","reported_seq":11,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-11-26T07:18:54.777156+0000","last_change":"2023-11-26T07:18:52.675688+0000","last_active":"2023-11-26T07:18:54.777156+0000","last_peered":"2023-11-26T07:18:54.777156+0000","last_clean":"2023-11-26T07:18:54.777156+0000","last_became_active":"2023-11-26T07:18:52.675385+0000","last_became_peered":"2023-11-26T07:18:52.675385+0000","last_unstale":"2023-11-26T07:18:54.777156+0000","last_undegraded":"2023-11-26T07:18:54.777156+0000","last_fullsized":"2023-11-26T07:18:54.777156+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:18:51.625829+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:18:51.625829+0000","last_clean_scrub_stamp":"2023-11-26T07:18:51.625829+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T08:36:31.952917+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.23","version":"0'0","reported_seq":11,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-11-26T07:18:54.777176+0000","last_change":"2023-11-26T07:18:52.667885+0000","last_active":"2023-11-26T07:18:54.777176+0000","last_peered":"2023-11-26T07:18:54.777176+0000","last_clean":"2023-11-26T07:18:54.777176+0000","last_became_active":"2023-11-26T07:18:52.667553+0000","last_became_peered":"2023-11-26T07:18:52.667553+0000","last_unstale":"2023-11-26T07:18:54.777176+0000","last_undegraded":"2023-11-26T07:18:54.777176+0000","last_fullsized":"2023-11-26T07:18:54.777176+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:18:51.625829+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:18:51.625829+0000","last_clean_scrub_stamp":"2023-11-26T07:18:51.625829+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T08:44:22.910250+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.22","version":"0'0","reported_seq":11,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-11-26T07:18:54.776076+0000","last_change":"2023-11-26T07:18:52.686734+0000","last_active":"2023-11-26T07:18:54.776076+0000","last_peered":"2023-11-26T07:18:54.776076+0000","last_clean":"2023-11-26T07:18:54.776076+0000","last_became_active":"2023-11-26T07:18:52.686491+0000","last_became_peered":"2023-11-26T07:18:52.686491+0000","last_unstale":"2023-11-26T07:18:54.776076+0000","last_undegraded":"2023-11-26T07:18:54.776076+0000","last_fullsized":"2023-11-26T07:18:54.776076+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:18:51.625829+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:18:51.625829+0000","last_clean_scrub_stamp":"2023-11-26T07:18:51.625829+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T13:59:11.780648+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.21","version":"0'0","reported_seq":11,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-11-26T07:18:54.775987+0000","last_change":"2023-11-26T07:18:52.683056+0000","last_active":"2023-11-26T07:18:54.775987+0000","last_peered":"2023-11-26T07:18:54.775987+0000","last_clean":"2023-11-26T07:18:54.775987+0000","last_became_active":"2023-11-26T07:18:52.682753+0000","last_became_peered":"2023-11-26T07:18:52.682753+0000","last_unstale":"2023-11-26T07:18:54.775987+0000","last_undegraded":"2023-11-26T07:18:54.775987+0000","last_fullsized":"2023-11-26T07:18:54.775987+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:18:51.625829+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:18:51.625829+0000","last_clean_scrub_stamp":"2023-11-26T07:18:51.625829+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T16:41:02.104869+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.20","version":"0'0","reported_seq":11,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-11-26T07:18:54.777588+0000","last_change":"2023-11-26T07:18:52.681823+0000","last_active":"2023-11-26T07:18:54.777588+0000","last_peered":"2023-11-26T07:18:54.777588+0000","last_clean":"2023-11-26T07:18:54.777588+0000","last_became_active":"2023-11-26T07:18:52.681425+0000","last_became_peered":"2023-11-26T07:18:52.681425+0000","last_unstale":"2023-11-26T07:18:54.777588+0000","last_undegraded":"2023-11-26T07:18:54.777588+0000","last_fullsized":"2023-11-26T07:18:54.777588+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:18:51.625829+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:18:51.625829+0000","last_clean_scrub_stamp":"2023-11-26T07:18:51.625829+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T16:48:48.991856+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.1e","version":"0'0","reported_seq":11,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-11-26T07:18:53.664049+0000","last_change":"2023-11-26T07:18:52.669618+0000","last_active":"2023-11-26T07:18:53.664049+0000","last_peered":"2023-11-26T07:18:53.664049+0000","last_clean":"2023-11-26T07:18:53.664049+0000","last_became_active":"2023-11-26T07:18:52.667803+0000","last_became_peered":"2023-11-26T07:18:52.667803+0000","last_unstale":"2023-11-26T07:18:53.664049+0000","last_undegraded":"2023-11-26T07:18:53.664049+0000","last_fullsized":"2023-11-26T07:18:53.664049+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:18:51.625829+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:18:51.625829+0000","last_clean_scrub_stamp":"2023-11-26T07:18:51.625829+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T08:44:22.910250+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1f","version":"32'4","reported_seq":16,"reported_epoch":32,"state":"active+clean","last_fresh":"2023-11-26T07:18:52.692661+0000","last_change":"2023-11-26T07:18:51.661754+0000","last_active":"2023-11-26T07:18:52.692661+0000","last_peered":"2023-11-26T07:18:52.692661+0000","last_clean":"2023-11-26T07:18:52.692661+0000","last_became_active":"2023-11-26T07:18:51.661410+0000","last_became_peered":"2023-11-26T07:18:51.661410+0000","last_unstale":"2023-11-26T07:18:52.692661+0000","last_undegraded":"2023-11-26T07:18:52.692661+0000","last_fullsized":"2023-11-26T07:18:52.692661+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:18:50.617605+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:18:50.617605+0000","last_clean_scrub_stamp":"2023-11-26T07:18:50.617605+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T11:49:26.615849+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.1f","version":"0'0","reported_seq":11,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-11-26T07:18:54.776547+0000","last_change":"2023-11-26T07:18:52.675236+0000","last_active":"2023-11-26T07:18:54.776547+0000","last_peered":"2023-11-26T07:18:54.776547+0000","last_clean":"2023-11-26T07:18:54.776547+0000","last_became_active":"2023-11-26T07:18:52.674546+0000","last_became_peered":"2023-11-26T07:18:52.674546+0000","last_unstale":"2023-11-26T07:18:54.776547+0000","last_undegraded":"2023-11-26T07:18:54.776547+0000","last_fullsized":"2023-11-26T07:18:54.776547+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:18:51.625829+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:18:51.625829+0000","last_clean_scrub_stamp":"2023-11-26T07:18:51.625829+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T12:12:36.504349+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.1e","version":"32'1","reported_seq":13,"reported_epoch":32,"state":"active+clean","last_fresh":"2023-11-26T07:18:52.688044+0000","last_change":"2023-11-26T07:18:51.652999+0000","last_active":"2023-11-26T07:18:52.688044+0000","last_peered":"2023-11-26T07:18:52.688044+0000","last_clean":"2023-11-26T07:18:52.688044+0000","last_became_active":"2023-11-26T07:18:51.652268+0000","last_became_peered":"2023-11-26T07:18:51.652268+0000","last_unstale":"2023-11-26T07:18:52.688044+0000","last_undegraded":"2023-11-26T07:18:52.688044+0000","last_fullsized":"2023-11-26T07:18:52.688044+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:18:50.617605+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:18:50.617605+0000","last_clean_scrub_stamp":"2023-11-26T07:18:50.617605+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T10:17:26.493927+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.1c","version":"0'0","reported_seq":11,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-11-26T07:18:54.775954+0000","last_change":"2023-11-26T07:18:52.689076+0000","last_active":"2023-11-26T07:18:54.775954+0000","last_peered":"2023-11-26T07:18:54.775954+0000","last_clean":"2023-11-26T07:18:54.775954+0000","last_became_active":"2023-11-26T07:18:52.688877+0000","last_became_peered":"2023-11-26T07:18:52.688877+0000","last_unstale":"2023-11-26T07:18:54.775954+0000","last_undegraded":"2023-11-26T07:18:54.775954+0000","last_fullsized":"2023-11-26T07:18:54.775954+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:18:51.625829+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:18:51.625829+0000","last_clean_scrub_stamp":"2023-11-26T07:18:51.625829+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T13:47:40.261002+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.1d","version":"32'2","reported_seq":14,"reported_epoch":32,"state":"active+clean","last_fresh":"2023-11-26T07:18:52.689886+0000","last_change":"2023-11-26T07:18:51.653241+0000","last_active":"2023-11-26T07:18:52.689886+0000","last_peered":"2023-11-26T07:18:52.689886+0000","last_clean":"2023-11-26T07:18:52.689886+0000","last_became_active":"2023-11-26T07:18:51.652971+0000","last_became_peered":"2023-11-26T07:18:51.652971+0000","last_unstale":"2023-11-26T07:18:52.689886+0000","last_undegraded":"2023-11-26T07:18:52.689886+0000","last_fullsized":"2023-11-26T07:18:52.689886+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:18:50.617605+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:18:50.617605+0000","last_clean_scrub_stamp":"2023-11-26T07:18:50.617605+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T10:49:58.494062+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.1d","version":"0'0","reported_seq":11,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-11-26T07:18:53.665999+0000","last_change":"2023-11-26T07:18:52.687986+0000","last_active":"2023-11-26T07:18:53.665999+0000","last_peered":"2023-11-26T07:18:53.665999+0000","last_clean":"2023-11-26T07:18:53.665999+0000","last_became_active":"2023-11-26T07:18:52.687689+0000","last_became_peered":"2023-11-26T07:18:52.687689+0000","last_unstale":"2023-11-26T07:18:53.665999+0000","last_undegraded":"2023-11-26T07:18:53.665999+0000","last_fullsized":"2023-11-26T07:18:53.665999+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:18:51.625829+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:18:51.625829+0000","last_clean_scrub_stamp":"2023-11-26T07:18:51.625829+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T10:23:04.707081+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1c","version":"32'2","reported_seq":14,"reported_epoch":32,"state":"active+clean","last_fresh":"2023-11-26T07:18:52.690837+0000","last_change":"2023-11-26T07:18:51.662450+0000","last_active":"2023-11-26T07:18:52.690837+0000","last_peered":"2023-11-26T07:18:52.690837+0000","last_clean":"2023-11-26T07:18:52.690837+0000","last_became_active":"2023-11-26T07:18:51.662140+0000","last_became_peered":"2023-11-26T07:18:51.662140+0000","last_unstale":"2023-11-26T07:18:52.690837+0000","last_undegraded":"2023-11-26T07:18:52.690837+0000","last_fullsized":"2023-11-26T07:18:52.690837+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:18:50.617605+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:18:50.617605+0000","last_clean_scrub_stamp":"2023-11-26T07:18:50.617605+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T18:27:57.438972+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.1a","version":"0'0","reported_seq":11,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-11-26T07:18:54.776226+0000","last_change":"2023-11-26T07:18:52.684355+0000","last_active":"2023-11-26T07:18:54.776226+0000","last_peered":"2023-11-26T07:18:54.776226+0000","last_clean":"2023-11-26T07:18:54.776226+0000","last_became_active":"2023-11-26T07:18:52.683892+0000","last_became_peered":"2023-11-26T07:18:52.683892+0000","last_unstale":"2023-11-26T07:18:54.776226+0000","last_undegraded":"2023-11-26T07:18:54.776226+0000","last_fullsized":"2023-11-26T07:18:54.776226+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:18:51.625829+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:18:51.625829+0000","last_clean_scrub_stamp":"2023-11-26T07:18:51.625829+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T17:23:21.873210+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.1b","version":"32'1","reported_seq":13,"reported_epoch":32,"state":"active+clean","last_fresh":"2023-11-26T07:18:52.694034+0000","last_change":"2023-11-26T07:18:51.654689+0000","last_active":"2023-11-26T07:18:52.694034+0000","last_peered":"2023-11-26T07:18:52.694034+0000","last_clean":"2023-11-26T07:18:52.694034+0000","last_became_active":"2023-11-26T07:18:51.653354+0000","last_became_peered":"2023-11-26T07:18:51.653354+0000","last_unstale":"2023-11-26T07:18:52.694034+0000","last_undegraded":"2023-11-26T07:18:52.694034+0000","last_fullsized":"2023-11-26T07:18:52.694034+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:18:50.617605+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:18:50.617605+0000","last_clean_scrub_stamp":"2023-11-26T07:18:50.617605+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T11:49:26.615849+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.1b","version":"0'0","reported_seq":11,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-11-26T07:18:54.775959+0000","last_change":"2023-11-26T07:18:52.677308+0000","last_active":"2023-11-26T07:18:54.775959+0000","last_peered":"2023-11-26T07:18:54.775959+0000","last_clean":"2023-11-26T07:18:54.775959+0000","last_became_active":"2023-11-26T07:18:52.676288+0000","last_became_peered":"2023-11-26T07:18:52.676288+0000","last_unstale":"2023-11-26T07:18:54.775959+0000","last_undegraded":"2023-11-26T07:18:54.775959+0000","last_fullsized":"2023-11-26T07:18:54.775959+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:18:51.625829+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:18:51.625829+0000","last_clean_scrub_stamp":"2023-11-26T07:18:51.625829+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T10:49:59.502286+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1a","version":"32'1","reported_seq":13,"reported_epoch":32,"state":"active+clean","last_fresh":"2023-11-26T07:18:52.695040+0000","last_change":"2023-11-26T07:18:51.653682+0000","last_active":"2023-11-26T07:18:52.695040+0000","last_peered":"2023-11-26T07:18:52.695040+0000","last_clean":"2023-11-26T07:18:52.695040+0000","last_became_active":"2023-11-26T07:18:51.653366+0000","last_became_peered":"2023-11-26T07:18:51.653366+0000","last_unstale":"2023-11-26T07:18:52.695040+0000","last_undegraded":"2023-11-26T07:18:52.695040+0000","last_fullsized":"2023-11-26T07:18:52.695040+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:18:50.617605+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:18:50.617605+0000","last_clean_scrub_stamp":"2023-11-26T07:18:50.617605+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T18:26:18.407283+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.18","version":"0'0","reported_seq":11,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-11-26T07:18:54.775568+0000","last_change":"2023-11-26T07:18:52.684533+0000","last_active":"2023-11-26T07:18:54.775568+0000","last_peered":"2023-11-26T07:18:54.775568+0000","last_clean":"2023-11-26T07:18:54.775568+0000","last_became_active":"2023-11-26T07:18:52.684193+0000","last_became_peered":"2023-11-26T07:18:52.684193+0000","last_unstale":"2023-11-26T07:18:54.775568+0000","last_undegraded":"2023-11-26T07:18:54.775568+0000","last_fullsized":"2023-11-26T07:18:54.775568+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:18:51.625829+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:18:51.625829+0000","last_clean_scrub_stamp":"2023-11-26T07:18:51.625829+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T12:12:36.504349+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.19","version":"0'0","reported_seq":12,"reported_epoch":32,"state":"active+clean","last_fresh":"2023-11-26T07:18:52.649183+0000","last_change":"2023-11-26T07:18:51.657763+0000","last_active":"2023-11-26T07:18:52.649183+0000","last_peered":"2023-11-26T07:18:52.649183+0000","last_clean":"2023-11-26T07:18:52.649183+0000","last_became_active":"2023-11-26T07:18:51.656677+0000","last_became_peered":"2023-11-26T07:18:51.656677+0000","last_unstale":"2023-11-26T07:18:52.649183+0000","last_undegraded":"2023-11-26T07:18:52.649183+0000","last_fullsized":"2023-11-26T07:18:52.649183+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:18:50.617605+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:18:50.617605+0000","last_clean_scrub_stamp":"2023-11-26T07:18:50.617605+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T16:41:01.096645+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.19","version":"0'0","reported_seq":11,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-11-26T07:18:53.665328+0000","last_change":"2023-11-26T07:18:52.676586+0000","last_active":"2023-11-26T07:18:53.665328+0000","last_peered":"2023-11-26T07:18:53.665328+0000","last_clean":"2023-11-26T07:18:53.665328+0000","last_became_active":"2023-11-26T07:18:52.676328+0000","last_became_peered":"2023-11-26T07:18:52.676328+0000","last_unstale":"2023-11-26T07:18:53.665328+0000","last_undegraded":"2023-11-26T07:18:53.665328+0000","last_fullsized":"2023-11-26T07:18:53.665328+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:18:51.625829+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:18:51.625829+0000","last_clean_scrub_stamp":"2023-11-26T07:18:51.625829+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T12:40:28.638833+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.18","version":"0'0","reported_seq":12,"reported_epoch":32,"state":"active+clean","last_fresh":"2023-11-26T07:18:52.653501+0000","last_change":"2023-11-26T07:18:51.656204+0000","last_active":"2023-11-26T07:18:52.653501+0000","last_peered":"2023-11-26T07:18:52.653501+0000","last_clean":"2023-11-26T07:18:52.653501+0000","last_became_active":"2023-11-26T07:18:51.655809+0000","last_became_peered":"2023-11-26T07:18:51.655809+0000","last_unstale":"2023-11-26T07:18:52.653501+0000","last_undegraded":"2023-11-26T07:18:52.653501+0000","last_fullsized":"2023-11-26T07:18:52.653501+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:18:50.617605+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:18:50.617605+0000","last_clean_scrub_stamp":"2023-11-26T07:18:50.617605+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T07:50:54.298421+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.16","version":"0'0","reported_seq":11,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-11-26T07:18:54.777323+0000","last_change":"2023-11-26T07:18:52.690469+0000","last_active":"2023-11-26T07:18:54.777323+0000","last_peered":"2023-11-26T07:18:54.777323+0000","last_clean":"2023-11-26T07:18:54.777323+0000","last_became_active":"2023-11-26T07:18:52.690199+0000","last_became_peered":"2023-11-26T07:18:52.690199+0000","last_unstale":"2023-11-26T07:18:54.777323+0000","last_undegraded":"2023-11-26T07:18:54.777323+0000","last_fullsized":"2023-11-26T07:18:54.777323+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:18:51.625829+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:18:51.625829+0000","last_clean_scrub_stamp":"2023-11-26T07:18:51.625829+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T12:03:00.136302+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.17","version":"0'0","reported_seq":12,"reported_epoch":32,"state":"active+clean","last_fresh":"2023-11-26T07:18:52.651648+0000","last_change":"2023-11-26T07:18:51.653322+0000","last_active":"2023-11-26T07:18:52.651648+0000","last_peered":"2023-11-26T07:18:52.651648+0000","last_clean":"2023-11-26T07:18:52.651648+0000","last_became_active":"2023-11-26T07:18:51.652388+0000","last_became_peered":"2023-11-26T07:18:51.652388+0000","last_unstale":"2023-11-26T07:18:52.651648+0000","last_undegraded":"2023-11-26T07:18:52.651648+0000","last_fullsized":"2023-11-26T07:18:52.651648+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:18:50.617605+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:18:50.617605+0000","last_clean_scrub_stamp":"2023-11-26T07:18:50.617605+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T13:59:10.772424+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.17","version":"0'0","reported_seq":11,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-11-26T07:18:54.777705+0000","last_change":"2023-11-26T07:18:52.680505+0000","last_active":"2023-11-26T07:18:54.777705+0000","last_peered":"2023-11-26T07:18:54.777705+0000","last_clean":"2023-11-26T07:18:54.777705+0000","last_became_active":"2023-11-26T07:18:52.680214+0000","last_became_peered":"2023-11-26T07:18:52.680214+0000","last_unstale":"2023-11-26T07:18:54.777705+0000","last_undegraded":"2023-11-26T07:18:54.777705+0000","last_fullsized":"2023-11-26T07:18:54.777705+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:18:51.625829+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:18:51.625829+0000","last_clean_scrub_stamp":"2023-11-26T07:18:51.625829+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T13:50:40.217037+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.16","version":"0'0","reported_seq":12,"reported_epoch":32,"state":"active+clean","last_fresh":"2023-11-26T07:18:52.655213+0000","last_change":"2023-11-26T07:18:51.644845+0000","last_active":"2023-11-26T07:18:52.655213+0000","last_peered":"2023-11-26T07:18:52.655213+0000","last_clean":"2023-11-26T07:18:52.655213+0000","last_became_active":"2023-11-26T07:18:51.644261+0000","last_became_peered":"2023-11-26T07:18:51.644261+0000","last_unstale":"2023-11-26T07:18:52.655213+0000","last_undegraded":"2023-11-26T07:18:52.655213+0000","last_fullsized":"2023-11-26T07:18:52.655213+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:18:50.617605+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:18:50.617605+0000","last_clean_scrub_stamp":"2023-11-26T07:18:50.617605+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T10:33:49.101331+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.14","version":"0'0","reported_seq":11,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-11-26T07:18:54.777560+0000","last_change":"2023-11-26T07:18:52.669834+0000","last_active":"2023-11-26T07:18:54.777560+0000","last_peered":"2023-11-26T07:18:54.777560+0000","last_clean":"2023-11-26T07:18:54.777560+0000","last_became_active":"2023-11-26T07:18:52.669564+0000","last_became_peered":"2023-11-26T07:18:52.669564+0000","last_unstale":"2023-11-26T07:18:54.777560+0000","last_undegraded":"2023-11-26T07:18:54.777560+0000","last_fullsized":"2023-11-26T07:18:54.777560+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:18:51.625829+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:18:51.625829+0000","last_clean_scrub_stamp":"2023-11-26T07:18:51.625829+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T16:48:48.991856+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.15","version":"32'3","reported_seq":15,"reported_epoch":32,"state":"active+clean","last_fresh":"2023-11-26T07:18:52.682254+0000","last_change":"2023-11-26T07:18:51.657428+0000","last_active":"2023-11-26T07:18:52.682254+0000","last_peered":"2023-11-26T07:18:52.682254+0000","last_clean":"2023-11-26T07:18:52.682254+0000","last_became_active":"2023-11-26T07:18:51.657075+0000","last_became_peered":"2023-11-26T07:18:51.657075+0000","last_unstale":"2023-11-26T07:18:52.682254+0000","last_undegraded":"2023-11-26T07:18:52.682254+0000","last_fullsized":"2023-11-26T07:18:52.682254+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:18:50.617605+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:18:50.617605+0000","last_clean_scrub_stamp":"2023-11-26T07:18:50.617605+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T13:47:39.252778+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.15","version":"0'0","reported_seq":11,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-11-26T07:18:54.776501+0000","last_change":"2023-11-26T07:18:52.677313+0000","last_active":"2023-11-26T07:18:54.776501+0000","last_peered":"2023-11-26T07:18:54.776501+0000","last_clean":"2023-11-26T07:18:54.776501+0000","last_became_active":"2023-11-26T07:18:52.676992+0000","last_became_peered":"2023-11-26T07:18:52.676992+0000","last_unstale":"2023-11-26T07:18:54.776501+0000","last_undegraded":"2023-11-26T07:18:54.776501+0000","last_fullsized":"2023-11-26T07:18:54.776501+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:18:51.625829+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:18:51.625829+0000","last_clean_scrub_stamp":"2023-11-26T07:18:51.625829+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T17:29:13.304698+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.14","version":"32'6","reported_seq":18,"reported_epoch":32,"state":"active+clean","last_fresh":"2023-11-26T07:18:52.698836+0000","last_change":"2023-11-26T07:18:51.647209+0000","last_active":"2023-11-26T07:18:52.698836+0000","last_peered":"2023-11-26T07:18:52.698836+0000","last_clean":"2023-11-26T07:18:52.698836+0000","last_became_active":"2023-11-26T07:18:51.647051+0000","last_became_peered":"2023-11-26T07:18:51.647051+0000","last_unstale":"2023-11-26T07:18:52.698836+0000","last_undegraded":"2023-11-26T07:18:52.698836+0000","last_fullsized":"2023-11-26T07:18:52.698836+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:18:50.617605+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:18:50.617605+0000","last_clean_scrub_stamp":"2023-11-26T07:18:50.617605+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T12:40:27.630609+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.12","version":"0'0","reported_seq":11,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-11-26T07:18:53.665370+0000","last_change":"2023-11-26T07:18:52.677367+0000","last_active":"2023-11-26T07:18:53.665370+0000","last_peered":"2023-11-26T07:18:53.665370+0000","last_clean":"2023-11-26T07:18:53.665370+0000","last_became_active":"2023-11-26T07:18:52.677098+0000","last_became_peered":"2023-11-26T07:18:52.677098+0000","last_unstale":"2023-11-26T07:18:53.665370+0000","last_undegraded":"2023-11-26T07:18:53.665370+0000","last_fullsized":"2023-11-26T07:18:53.665370+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:18:51.625829+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:18:51.625829+0000","last_clean_scrub_stamp":"2023-11-26T07:18:51.625829+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T13:47:40.261002+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.13","version":"32'2","reported_seq":14,"reported_epoch":32,"state":"active+clean","last_fresh":"2023-11-26T07:18:52.691338+0000","last_change":"2023-11-26T07:18:51.656339+0000","last_active":"2023-11-26T07:18:52.691338+0000","last_peered":"2023-11-26T07:18:52.691338+0000","last_clean":"2023-11-26T07:18:52.691338+0000","last_became_active":"2023-11-26T07:18:51.655998+0000","last_became_peered":"2023-11-26T07:18:51.655998+0000","last_unstale":"2023-11-26T07:18:52.691338+0000","last_undegraded":"2023-11-26T07:18:52.691338+0000","last_fullsized":"2023-11-26T07:18:52.691338+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:18:50.617605+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:18:50.617605+0000","last_clean_scrub_stamp":"2023-11-26T07:18:50.617605+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T16:41:01.096645+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.13","version":"0'0","reported_seq":11,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-11-26T07:18:54.777486+0000","last_change":"2023-11-26T07:18:52.661410+0000","last_active":"2023-11-26T07:18:54.777486+0000","last_peered":"2023-11-26T07:18:54.777486+0000","last_clean":"2023-11-26T07:18:54.777486+0000","last_became_active":"2023-11-26T07:18:52.661149+0000","last_became_peered":"2023-11-26T07:18:52.661149+0000","last_unstale":"2023-11-26T07:18:54.777486+0000","last_undegraded":"2023-11-26T07:18:54.777486+0000","last_fullsized":"2023-11-26T07:18:54.777486+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:18:51.625829+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:18:51.625829+0000","last_clean_scrub_stamp":"2023-11-26T07:18:51.625829+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T15:42:06.512477+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.12","version":"0'0","reported_seq":12,"reported_epoch":32,"state":"active+clean","last_fresh":"2023-11-26T07:18:52.671303+0000","last_change":"2023-11-26T07:18:51.658016+0000","last_active":"2023-11-26T07:18:52.671303+0000","last_peered":"2023-11-26T07:18:52.671303+0000","last_clean":"2023-11-26T07:18:52.671303+0000","last_became_active":"2023-11-26T07:18:51.657646+0000","last_became_peered":"2023-11-26T07:18:51.657646+0000","last_unstale":"2023-11-26T07:18:52.671303+0000","last_undegraded":"2023-11-26T07:18:52.671303+0000","last_fullsized":"2023-11-26T07:18:52.671303+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:18:50.617605+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:18:50.617605+0000","last_clean_scrub_stamp":"2023-11-26T07:18:50.617605+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T12:12:35.496125+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.10","version":"0'0","reported_seq":11,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-11-26T07:18:54.777611+0000","last_change":"2023-11-26T07:18:52.672193+0000","last_active":"2023-11-26T07:18:54.777611+0000","last_peered":"2023-11-26T07:18:54.777611+0000","last_clean":"2023-11-26T07:18:54.777611+0000","last_became_active":"2023-11-26T07:18:52.671857+0000","last_became_peered":"2023-11-26T07:18:52.671857+0000","last_unstale":"2023-11-26T07:18:54.777611+0000","last_undegraded":"2023-11-26T07:18:54.777611+0000","last_fullsized":"2023-11-26T07:18:54.777611+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:18:51.625829+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:18:51.625829+0000","last_clean_scrub_stamp":"2023-11-26T07:18:51.625829+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T13:50:40.217037+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.11","version":"0'0","reported_seq":12,"reported_epoch":32,"state":"active+clean","last_fresh":"2023-11-26T07:18:52.649437+0000","last_change":"2023-11-26T07:18:51.654685+0000","last_active":"2023-11-26T07:18:52.649437+0000","last_peered":"2023-11-26T07:18:52.649437+0000","last_clean":"2023-11-26T07:18:52.649437+0000","last_became_active":"2023-11-26T07:18:51.653542+0000","last_became_peered":"2023-11-26T07:18:51.653542+0000","last_unstale":"2023-11-26T07:18:52.649437+0000","last_undegraded":"2023-11-26T07:18:52.649437+0000","last_fullsized":"2023-11-26T07:18:52.649437+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:18:50.617605+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:18:50.617605+0000","last_clean_scrub_stamp":"2023-11-26T07:18:50.617605+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T12:12:35.496125+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.11","version":"0'0","reported_seq":11,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-11-26T07:18:54.776160+0000","last_change":"2023-11-26T07:18:52.673902+0000","last_active":"2023-11-26T07:18:54.776160+0000","last_peered":"2023-11-26T07:18:54.776160+0000","last_clean":"2023-11-26T07:18:54.776160+0000","last_became_active":"2023-11-26T07:18:52.673049+0000","last_became_peered":"2023-11-26T07:18:52.673049+0000","last_unstale":"2023-11-26T07:18:54.776160+0000","last_undegraded":"2023-11-26T07:18:54.776160+0000","last_fullsized":"2023-11-26T07:18:54.776160+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:18:51.625829+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:18:51.625829+0000","last_clean_scrub_stamp":"2023-11-26T07:18:51.625829+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T10:17:27.502151+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.10","version":"32'8","reported_seq":20,"reported_epoch":32,"state":"active+clean","last_fresh":"2023-11-26T07:18:52.693910+0000","last_change":"2023-11-26T07:18:51.652858+0000","last_active":"2023-11-26T07:18:52.693910+0000","last_peered":"2023-11-26T07:18:52.693910+0000","last_clean":"2023-11-26T07:18:52.693910+0000","last_became_active":"2023-11-26T07:18:51.652583+0000","last_became_peered":"2023-11-26T07:18:51.652583+0000","last_unstale":"2023-11-26T07:18:52.693910+0000","last_undegraded":"2023-11-26T07:18:52.693910+0000","last_fullsized":"2023-11-26T07:18:52.693910+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:18:50.617605+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:18:50.617605+0000","last_clean_scrub_stamp":"2023-11-26T07:18:50.617605+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T10:33:49.101331+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.e","version":"0'0","reported_seq":11,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-11-26T07:18:54.776989+0000","last_change":"2023-11-26T07:18:52.680594+0000","last_active":"2023-11-26T07:18:54.776989+0000","last_peered":"2023-11-26T07:18:54.776989+0000","last_clean":"2023-11-26T07:18:54.776989+0000","last_became_active":"2023-11-26T07:18:52.673803+0000","last_became_peered":"2023-11-26T07:18:52.673803+0000","last_unstale":"2023-11-26T07:18:54.776989+0000","last_undegraded":"2023-11-26T07:18:54.776989+0000","last_fullsized":"2023-11-26T07:18:54.776989+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:18:51.625829+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:18:51.625829+0000","last_clean_scrub_stamp":"2023-11-26T07:18:51.625829+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T13:18:27.001149+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.f","version":"32'1","reported_seq":13,"reported_epoch":32,"state":"active+clean","last_fresh":"2023-11-26T07:18:52.687288+0000","last_change":"2023-11-26T07:18:51.656212+0000","last_active":"2023-11-26T07:18:52.687288+0000","last_peered":"2023-11-26T07:18:52.687288+0000","last_clean":"2023-11-26T07:18:52.687288+0000","last_became_active":"2023-11-26T07:18:51.650720+0000","last_became_peered":"2023-11-26T07:18:51.650720+0000","last_unstale":"2023-11-26T07:18:52.687288+0000","last_undegraded":"2023-11-26T07:18:52.687288+0000","last_fullsized":"2023-11-26T07:18:52.687288+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:18:50.617605+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:18:50.617605+0000","last_clean_scrub_stamp":"2023-11-26T07:18:50.617605+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T08:44:21.902026+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.f","version":"0'0","reported_seq":11,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-11-26T07:18:54.776149+0000","last_change":"2023-11-26T07:18:52.671950+0000","last_active":"2023-11-26T07:18:54.776149+0000","last_peered":"2023-11-26T07:18:54.776149+0000","last_clean":"2023-11-26T07:18:54.776149+0000","last_became_active":"2023-11-26T07:18:52.671645+0000","last_became_peered":"2023-11-26T07:18:52.671645+0000","last_unstale":"2023-11-26T07:18:54.776149+0000","last_undegraded":"2023-11-26T07:18:54.776149+0000","last_fullsized":"2023-11-26T07:18:54.776149+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:18:51.625829+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:18:51.625829+0000","last_clean_scrub_stamp":"2023-11-26T07:18:51.625829+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T09:49:42.889030+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.e","version":"32'2","reported_seq":14,"reported_epoch":32,"state":"active+clean","last_fresh":"2023-11-26T07:18:52.688955+0000","last_change":"2023-11-26T07:18:51.660325+0000","last_active":"2023-11-26T07:18:52.688955+0000","last_peered":"2023-11-26T07:18:52.688955+0000","last_clean":"2023-11-26T07:18:52.688955+0000","last_became_active":"2023-11-26T07:18:51.659918+0000","last_became_peered":"2023-11-26T07:18:51.659918+0000","last_unstale":"2023-11-26T07:18:52.688955+0000","last_undegraded":"2023-11-26T07:18:52.688955+0000","last_fullsized":"2023-11-26T07:18:52.688955+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:18:50.617605+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:18:50.617605+0000","last_clean_scrub_stamp":"2023-11-26T07:18:50.617605+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T10:17:26.493927+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.c","version":"0'0","reported_seq":11,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-11-26T07:18:53.665748+0000","last_change":"2023-11-26T07:18:52.685610+0000","last_active":"2023-11-26T07:18:53.665748+0000","last_peered":"2023-11-26T07:18:53.665748+0000","last_clean":"2023-11-26T07:18:53.665748+0000","last_became_active":"2023-11-26T07:18:52.685310+0000","last_became_peered":"2023-11-26T07:18:52.685310+0000","last_unstale":"2023-11-26T07:18:53.665748+0000","last_undegraded":"2023-11-26T07:18:53.665748+0000","last_fullsized":"2023-11-26T07:18:53.665748+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:18:51.625829+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:18:51.625829+0000","last_clean_scrub_stamp":"2023-11-26T07:18:51.625829+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T10:49:59.502286+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.d","version":"32'1","reported_seq":13,"reported_epoch":32,"state":"active+clean","last_fresh":"2023-11-26T07:18:52.691763+0000","last_change":"2023-11-26T07:18:51.658440+0000","last_active":"2023-11-26T07:18:52.691763+0000","last_peered":"2023-11-26T07:18:52.691763+0000","last_clean":"2023-11-26T07:18:52.691763+0000","last_became_active":"2023-11-26T07:18:51.658161+0000","last_became_peered":"2023-11-26T07:18:51.658161+0000","last_unstale":"2023-11-26T07:18:52.691763+0000","last_undegraded":"2023-11-26T07:18:52.691763+0000","last_fullsized":"2023-11-26T07:18:52.691763+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:18:50.617605+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:18:50.617605+0000","last_clean_scrub_stamp":"2023-11-26T07:18:50.617605+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T14:17:17.432332+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.d","version":"0'0","reported_seq":11,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-11-26T07:18:54.777019+0000","last_change":"2023-11-26T07:18:52.674278+0000","last_active":"2023-11-26T07:18:54.777019+0000","last_peered":"2023-11-26T07:18:54.777019+0000","last_clean":"2023-11-26T07:18:54.777019+0000","last_became_active":"2023-11-26T07:18:52.674012+0000","last_became_peered":"2023-11-26T07:18:52.674012+0000","last_unstale":"2023-11-26T07:18:54.777019+0000","last_undegraded":"2023-11-26T07:18:54.777019+0000","last_fullsized":"2023-11-26T07:18:54.777019+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:18:51.625829+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:18:51.625829+0000","last_clean_scrub_stamp":"2023-11-26T07:18:51.625829+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T19:10:40.245493+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.c","version":"0'0","reported_seq":12,"reported_epoch":32,"state":"active+clean","last_fresh":"2023-11-26T07:18:52.652890+0000","last_change":"2023-11-26T07:18:51.654164+0000","last_active":"2023-11-26T07:18:52.652890+0000","last_peered":"2023-11-26T07:18:52.652890+0000","last_clean":"2023-11-26T07:18:52.652890+0000","last_became_active":"2023-11-26T07:18:51.653823+0000","last_became_peered":"2023-11-26T07:18:51.653823+0000","last_unstale":"2023-11-26T07:18:52.652890+0000","last_undegraded":"2023-11-26T07:18:52.652890+0000","last_fullsized":"2023-11-26T07:18:52.652890+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:18:50.617605+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:18:50.617605+0000","last_clean_scrub_stamp":"2023-11-26T07:18:50.617605+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T17:37:35.256892+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.a","version":"0'0","reported_seq":11,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-11-26T07:18:54.776991+0000","last_change":"2023-11-26T07:18:52.667222+0000","last_active":"2023-11-26T07:18:54.776991+0000","last_peered":"2023-11-26T07:18:54.776991+0000","last_clean":"2023-11-26T07:18:54.776991+0000","last_became_active":"2023-11-26T07:18:52.665846+0000","last_became_peered":"2023-11-26T07:18:52.665846+0000","last_unstale":"2023-11-26T07:18:54.776991+0000","last_undegraded":"2023-11-26T07:18:54.776991+0000","last_fullsized":"2023-11-26T07:18:54.776991+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:18:51.625829+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:18:51.625829+0000","last_clean_scrub_stamp":"2023-11-26T07:18:51.625829+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T17:37:36.265116+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.b","version":"32'2","reported_seq":14,"reported_epoch":32,"state":"active+clean","last_fresh":"2023-11-26T07:18:52.683434+0000","last_change":"2023-11-26T07:18:51.646694+0000","last_active":"2023-11-26T07:18:52.683434+0000","last_peered":"2023-11-26T07:18:52.683434+0000","last_clean":"2023-11-26T07:18:52.683434+0000","last_became_active":"2023-11-26T07:18:51.644567+0000","last_became_peered":"2023-11-26T07:18:51.644567+0000","last_unstale":"2023-11-26T07:18:52.683434+0000","last_undegraded":"2023-11-26T07:18:52.683434+0000","last_fullsized":"2023-11-26T07:18:52.683434+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:18:50.617605+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:18:50.617605+0000","last_clean_scrub_stamp":"2023-11-26T07:18:50.617605+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T11:49:26.615849+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.b","version":"0'0","reported_seq":11,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-11-26T07:18:54.776902+0000","last_change":"2023-11-26T07:18:52.670716+0000","last_active":"2023-11-26T07:18:54.776902+0000","last_peered":"2023-11-26T07:18:54.776902+0000","last_clean":"2023-11-26T07:18:54.776902+0000","last_became_active":"2023-11-26T07:18:52.670370+0000","last_became_peered":"2023-11-26T07:18:52.670370+0000","last_unstale":"2023-11-26T07:18:54.776902+0000","last_undegraded":"2023-11-26T07:18:54.776902+0000","last_fullsized":"2023-11-26T07:18:54.776902+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:18:51.625829+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:18:51.625829+0000","last_clean_scrub_stamp":"2023-11-26T07:18:51.625829+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T17:22:39.981484+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.a","version":"0'0","reported_seq":12,"reported_epoch":32,"state":"active+clean","last_fresh":"2023-11-26T07:18:52.658364+0000","last_change":"2023-11-26T07:18:51.660988+0000","last_active":"2023-11-26T07:18:52.658364+0000","last_peered":"2023-11-26T07:18:52.658364+0000","last_clean":"2023-11-26T07:18:52.658364+0000","last_became_active":"2023-11-26T07:18:51.660676+0000","last_became_peered":"2023-11-26T07:18:51.660676+0000","last_unstale":"2023-11-26T07:18:52.658364+0000","last_undegraded":"2023-11-26T07:18:52.658364+0000","last_fullsized":"2023-11-26T07:18:52.658364+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:18:50.617605+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:18:50.617605+0000","last_clean_scrub_stamp":"2023-11-26T07:18:50.617605+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T10:40:39.059176+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.8","version":"0'0","reported_seq":11,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-11-26T07:18:54.776593+0000","last_change":"2023-11-26T07:18:52.670731+0000","last_active":"2023-11-26T07:18:54.776593+0000","last_peered":"2023-11-26T07:18:54.776593+0000","last_clean":"2023-11-26T07:18:54.776593+0000","last_became_active":"2023-11-26T07:18:52.670500+0000","last_became_peered":"2023-11-26T07:18:52.670500+0000","last_unstale":"2023-11-26T07:18:54.776593+0000","last_undegraded":"2023-11-26T07:18:54.776593+0000","last_fullsized":"2023-11-26T07:18:54.776593+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:18:51.625829+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:18:51.625829+0000","last_clean_scrub_stamp":"2023-11-26T07:18:51.625829+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T10:54:58.699396+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.9","version":"0'0","reported_seq":12,"reported_epoch":32,"state":"active+clean","last_fresh":"2023-11-26T07:18:52.653043+0000","last_change":"2023-11-26T07:18:51.652977+0000","last_active":"2023-11-26T07:18:52.653043+0000","last_peered":"2023-11-26T07:18:52.653043+0000","last_clean":"2023-11-26T07:18:52.653043+0000","last_became_active":"2023-11-26T07:18:51.652670+0000","last_became_peered":"2023-11-26T07:18:51.652670+0000","last_unstale":"2023-11-26T07:18:52.653043+0000","last_undegraded":"2023-11-26T07:18:52.653043+0000","last_fullsized":"2023-11-26T07:18:52.653043+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:18:50.617605+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:18:50.617605+0000","last_clean_scrub_stamp":"2023-11-26T07:18:50.617605+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T12:04:39.473159+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.9","version":"0'0","reported_seq":11,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-11-26T07:18:54.776619+0000","last_change":"2023-11-26T07:18:52.686470+0000","last_active":"2023-11-26T07:18:54.776619+0000","last_peered":"2023-11-26T07:18:54.776619+0000","last_clean":"2023-11-26T07:18:54.776619+0000","last_became_active":"2023-11-26T07:18:52.686061+0000","last_became_peered":"2023-11-26T07:18:52.686061+0000","last_unstale":"2023-11-26T07:18:54.776619+0000","last_undegraded":"2023-11-26T07:18:54.776619+0000","last_fullsized":"2023-11-26T07:18:54.776619+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:18:51.625829+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:18:51.625829+0000","last_clean_scrub_stamp":"2023-11-26T07:18:51.625829+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T08:36:31.952917+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.8","version":"0'0","reported_seq":12,"reported_epoch":32,"state":"active+clean","last_fresh":"2023-11-26T07:18:52.664183+0000","last_change":"2023-11-26T07:18:51.659037+0000","last_active":"2023-11-26T07:18:52.664183+0000","last_peered":"2023-11-26T07:18:52.664183+0000","last_clean":"2023-11-26T07:18:52.664183+0000","last_became_active":"2023-11-26T07:18:51.658596+0000","last_became_peered":"2023-11-26T07:18:51.658596+0000","last_unstale":"2023-11-26T07:18:52.664183+0000","last_undegraded":"2023-11-26T07:18:52.664183+0000","last_fullsized":"2023-11-26T07:18:52.664183+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:18:50.617605+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:18:50.617605+0000","last_clean_scrub_stamp":"2023-11-26T07:18:50.617605+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T12:40:27.630609+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.6","version":"0'0","reported_seq":11,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-11-26T07:18:54.777159+0000","last_change":"2023-11-26T07:18:52.678430+0000","last_active":"2023-11-26T07:18:54.777159+0000","last_peered":"2023-11-26T07:18:54.777159+0000","last_clean":"2023-11-26T07:18:54.777159+0000","last_became_active":"2023-11-26T07:18:52.678167+0000","last_became_peered":"2023-11-26T07:18:52.678167+0000","last_unstale":"2023-11-26T07:18:54.777159+0000","last_undegraded":"2023-11-26T07:18:54.777159+0000","last_fullsized":"2023-11-26T07:18:54.777159+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:18:51.625829+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:18:51.625829+0000","last_clean_scrub_stamp":"2023-11-26T07:18:51.625829+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T17:27:25.330389+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.7","version":"32'2","reported_seq":14,"reported_epoch":32,"state":"active+clean","last_fresh":"2023-11-26T07:18:52.681021+0000","last_change":"2023-11-26T07:18:51.653789+0000","last_active":"2023-11-26T07:18:52.681021+0000","last_peered":"2023-11-26T07:18:52.681021+0000","last_clean":"2023-11-26T07:18:52.681021+0000","last_became_active":"2023-11-26T07:18:51.652136+0000","last_became_peered":"2023-11-26T07:18:51.652136+0000","last_unstale":"2023-11-26T07:18:52.681021+0000","last_undegraded":"2023-11-26T07:18:52.681021+0000","last_fullsized":"2023-11-26T07:18:52.681021+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:18:50.617605+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:18:50.617605+0000","last_clean_scrub_stamp":"2023-11-26T07:18:50.617605+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T09:38:44.587731+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.7","version":"0'0","reported_seq":11,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-11-26T07:18:54.776302+0000","last_change":"2023-11-26T07:18:52.677480+0000","last_active":"2023-11-26T07:18:54.776302+0000","last_peered":"2023-11-26T07:18:54.776302+0000","last_clean":"2023-11-26T07:18:54.776302+0000","last_became_active":"2023-11-26T07:18:52.677195+0000","last_became_peered":"2023-11-26T07:18:52.677195+0000","last_unstale":"2023-11-26T07:18:54.776302+0000","last_undegraded":"2023-11-26T07:18:54.776302+0000","last_fullsized":"2023-11-26T07:18:54.776302+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:18:51.625829+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:18:51.625829+0000","last_clean_scrub_stamp":"2023-11-26T07:18:51.625829+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T10:23:04.707081+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.6","version":"33'1","reported_seq":13,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-11-26T07:18:53.664307+0000","last_change":"2023-11-26T07:18:51.644857+0000","last_active":"2023-11-26T07:18:53.664307+0000","last_peered":"2023-11-26T07:18:53.664307+0000","last_clean":"2023-11-26T07:18:53.664307+0000","last_became_active":"2023-11-26T07:18:51.644616+0000","last_became_peered":"2023-11-26T07:18:51.644616+0000","last_unstale":"2023-11-26T07:18:53.664307+0000","last_undegraded":"2023-11-26T07:18:53.664307+0000","last_fullsized":"2023-11-26T07:18:53.664307+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:18:50.617605+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:18:50.617605+0000","last_clean_scrub_stamp":"2023-11-26T07:18:50.617605+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T12:12:35.496125+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.5","version":"0'0","reported_seq":11,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-11-26T07:18:53.663956+0000","last_change":"2023-11-26T07:18:52.676788+0000","last_active":"2023-11-26T07:18:53.663956+0000","last_peered":"2023-11-26T07:18:53.663956+0000","last_clean":"2023-11-26T07:18:53.663956+0000","last_became_active":"2023-11-26T07:18:52.676532+0000","last_became_peered":"2023-11-26T07:18:52.676532+0000","last_unstale":"2023-11-26T07:18:53.663956+0000","last_undegraded":"2023-11-26T07:18:53.663956+0000","last_fullsized":"2023-11-26T07:18:53.663956+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:18:51.625829+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:18:51.625829+0000","last_clean_scrub_stamp":"2023-11-26T07:18:51.625829+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T08:30:11.556027+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.4","version":"0'0","reported_seq":12,"reported_epoch":32,"state":"active+clean","last_fresh":"2023-11-26T07:18:52.671376+0000","last_change":"2023-11-26T07:18:51.660079+0000","last_active":"2023-11-26T07:18:52.671376+0000","last_peered":"2023-11-26T07:18:52.671376+0000","last_clean":"2023-11-26T07:18:52.671376+0000","last_became_active":"2023-11-26T07:18:51.659661+0000","last_became_peered":"2023-11-26T07:18:51.659661+0000","last_unstale":"2023-11-26T07:18:52.671376+0000","last_undegraded":"2023-11-26T07:18:52.671376+0000","last_fullsized":"2023-11-26T07:18:52.671376+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:18:50.617605+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:18:50.617605+0000","last_clean_scrub_stamp":"2023-11-26T07:18:50.617605+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T13:18:25.992925+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.2","version":"0'0","reported_seq":11,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-11-26T07:18:54.776634+0000","last_change":"2023-11-26T07:18:52.682229+0000","last_active":"2023-11-26T07:18:54.776634+0000","last_peered":"2023-11-26T07:18:54.776634+0000","last_clean":"2023-11-26T07:18:54.776634+0000","last_became_active":"2023-11-26T07:18:52.681612+0000","last_became_peered":"2023-11-26T07:18:52.681612+0000","last_unstale":"2023-11-26T07:18:54.776634+0000","last_undegraded":"2023-11-26T07:18:54.776634+0000","last_fullsized":"2023-11-26T07:18:54.776634+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:18:51.625829+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:18:51.625829+0000","last_clean_scrub_stamp":"2023-11-26T07:18:51.625829+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T13:18:27.001149+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.3","version":"0'0","reported_seq":12,"reported_epoch":32,"state":"active+clean","last_fresh":"2023-11-26T07:18:52.654786+0000","last_change":"2023-11-26T07:18:51.647332+0000","last_active":"2023-11-26T07:18:52.654786+0000","last_peered":"2023-11-26T07:18:52.654786+0000","last_clean":"2023-11-26T07:18:52.654786+0000","last_became_active":"2023-11-26T07:18:51.647054+0000","last_became_peered":"2023-11-26T07:18:51.647054+0000","last_unstale":"2023-11-26T07:18:52.654786+0000","last_undegraded":"2023-11-26T07:18:52.654786+0000","last_fullsized":"2023-11-26T07:18:52.654786+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:18:50.617605+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:18:50.617605+0000","last_clean_scrub_stamp":"2023-11-26T07:18:50.617605+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T16:41:01.096645+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.3","version":"0'0","reported_seq":11,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-11-26T07:18:53.664412+0000","last_change":"2023-11-26T07:18:52.683961+0000","last_active":"2023-11-26T07:18:53.664412+0000","last_peered":"2023-11-26T07:18:53.664412+0000","last_clean":"2023-11-26T07:18:53.664412+0000","last_became_active":"2023-11-26T07:18:52.683685+0000","last_became_peered":"2023-11-26T07:18:52.683685+0000","last_unstale":"2023-11-26T07:18:53.664412+0000","last_undegraded":"2023-11-26T07:18:53.664412+0000","last_fullsized":"2023-11-26T07:18:53.664412+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:18:51.625829+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:18:51.625829+0000","last_clean_scrub_stamp":"2023-11-26T07:18:51.625829+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T09:38:45.595955+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.2","version":"0'0","reported_seq":12,"reported_epoch":32,"state":"active+clean","last_fresh":"2023-11-26T07:18:52.671429+0000","last_change":"2023-11-26T07:18:51.651994+0000","last_active":"2023-11-26T07:18:52.671429+0000","last_peered":"2023-11-26T07:18:52.671429+0000","last_clean":"2023-11-26T07:18:52.671429+0000","last_became_active":"2023-11-26T07:18:51.650255+0000","last_became_peered":"2023-11-26T07:18:51.650255+0000","last_unstale":"2023-11-26T07:18:52.671429+0000","last_undegraded":"2023-11-26T07:18:52.671429+0000","last_fullsized":"2023-11-26T07:18:52.671429+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:18:50.617605+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:18:50.617605+0000","last_clean_scrub_stamp":"2023-11-26T07:18:50.617605+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T08:30:10.547803+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.0","version":"0'0","reported_seq":11,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-11-26T07:18:54.776518+0000","last_change":"2023-11-26T07:18:52.681139+0000","last_active":"2023-11-26T07:18:54.776518+0000","last_peered":"2023-11-26T07:18:54.776518+0000","last_clean":"2023-11-26T07:18:54.776518+0000","last_became_active":"2023-11-26T07:18:52.679760+0000","last_became_peered":"2023-11-26T07:18:52.679760+0000","last_unstale":"2023-11-26T07:18:54.776518+0000","last_undegraded":"2023-11-26T07:18:54.776518+0000","last_fullsized":"2023-11-26T07:18:54.776518+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:18:51.625829+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:18:51.625829+0000","last_clean_scrub_stamp":"2023-11-26T07:18:51.625829+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T17:23:21.873210+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1","version":"0'0","reported_seq":12,"reported_epoch":32,"state":"active+clean","last_fresh":"2023-11-26T07:18:52.653205+0000","last_change":"2023-11-26T07:18:51.654435+0000","last_active":"2023-11-26T07:18:52.653205+0000","last_peered":"2023-11-26T07:18:52.653205+0000","last_clean":"2023-11-26T07:18:52.653205+0000","last_became_active":"2023-11-26T07:18:51.654107+0000","last_became_peered":"2023-11-26T07:18:51.654107+0000","last_unstale":"2023-11-26T07:18:52.653205+0000","last_undegraded":"2023-11-26T07:18:52.653205+0000","last_fullsized":"2023-11-26T07:18:52.653205+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:18:50.617605+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:18:50.617605+0000","last_clean_scrub_stamp":"2023-11-26T07:18:50.617605+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T14:40:15.233288+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.1","version":"0'0","reported_seq":11,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-11-26T07:18:54.775507+0000","last_change":"2023-11-26T07:18:52.668080+0000","last_active":"2023-11-26T07:18:54.775507+0000","last_peered":"2023-11-26T07:18:54.775507+0000","last_clean":"2023-11-26T07:18:54.775507+0000","last_became_active":"2023-11-26T07:18:52.667573+0000","last_became_peered":"2023-11-26T07:18:52.667573+0000","last_unstale":"2023-11-26T07:18:54.775507+0000","last_undegraded":"2023-11-26T07:18:54.775507+0000","last_fullsized":"2023-11-26T07:18:54.775507+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:18:51.625829+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:18:51.625829+0000","last_clean_scrub_stamp":"2023-11-26T07:18:51.625829+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T10:33:50.109555+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.0","version":"0'0","reported_seq":12,"reported_epoch":32,"state":"active+clean","last_fresh":"2023-11-26T07:18:52.649482+0000","last_change":"2023-11-26T07:18:51.655721+0000","last_active":"2023-11-26T07:18:52.649482+0000","last_peered":"2023-11-26T07:18:52.649482+0000","last_clean":"2023-11-26T07:18:52.649482+0000","last_became_active":"2023-11-26T07:18:51.655334+0000","last_became_peered":"2023-11-26T07:18:51.655334+0000","last_unstale":"2023-11-26T07:18:52.649482+0000","last_undegraded":"2023-11-26T07:18:52.649482+0000","last_fullsized":"2023-11-26T07:18:52.649482+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:18:50.617605+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:18:50.617605+0000","last_clean_scrub_stamp":"2023-11-26T07:18:50.617605+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T13:18:25.992925+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.4","version":"0'0","reported_seq":11,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-11-26T07:18:54.776949+0000","last_change":"2023-11-26T07:18:52.678680+0000","last_active":"2023-11-26T07:18:54.776949+0000","last_peered":"2023-11-26T07:18:54.776949+0000","last_clean":"2023-11-26T07:18:54.776949+0000","last_became_active":"2023-11-26T07:18:52.678376+0000","last_became_peered":"2023-11-26T07:18:52.678376+0000","last_unstale":"2023-11-26T07:18:54.776949+0000","last_undegraded":"2023-11-26T07:18:54.776949+0000","last_fullsized":"2023-11-26T07:18:54.776949+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:18:51.625829+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:18:51.625829+0000","last_clean_scrub_stamp":"2023-11-26T07:18:51.625829+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T15:42:06.512477+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.5","version":"0'0","reported_seq":12,"reported_epoch":32,"state":"active+clean","last_fresh":"2023-11-26T07:18:52.658814+0000","last_change":"2023-11-26T07:18:51.661135+0000","last_active":"2023-11-26T07:18:52.658814+0000","last_peered":"2023-11-26T07:18:52.658814+0000","last_clean":"2023-11-26T07:18:52.658814+0000","last_became_active":"2023-11-26T07:18:51.659814+0000","last_became_peered":"2023-11-26T07:18:51.659814+0000","last_unstale":"2023-11-26T07:18:52.658814+0000","last_undegraded":"2023-11-26T07:18:52.658814+0000","last_fullsized":"2023-11-26T07:18:52.658814+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:18:50.617605+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:18:50.617605+0000","last_clean_scrub_stamp":"2023-11-26T07:18:50.617605+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T14:49:28.420672+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"1.0","version":"15'117","reported_seq":175,"reported_epoch":32,"state":"active+clean","last_fresh":"2023-11-26T07:18:52.665022+0000","last_change":"2023-11-26T07:17:43.827802+0000","last_active":"2023-11-26T07:18:52.665022+0000","last_peered":"2023-11-26T07:18:52.665022+0000","last_clean":"2023-11-26T07:18:52.665022+0000","last_became_active":"2023-11-26T07:17:43.827308+0000","last_became_peered":"2023-11-26T07:17:43.827308+0000","last_unstale":"2023-11-26T07:18:52.665022+0000","last_undegraded":"2023-11-26T07:18:52.665022+0000","last_fullsized":"2023-11-26T07:18:52.665022+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:17:42.160945+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:17:42.160945+0000","last_clean_scrub_stamp":"2023-11-26T07:17:42.160945+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T08:17:54.167366+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":5,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":4,"num_pg":32,"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":39,"ondisk_log_size":39,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574865,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27968,"kb_used_data":948,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343872,"statfs":{"total":96636764160,"available":96608124928,"internally_reserved":0,"allocated":970752,"data_stored":737050,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Nov 26 07:18:42 2023","interfaces":[{"interface":"back","average":{"1min":0.71499999999999997,"5min":0.71499999999999997,"15min":0.71499999999999997},"min":{"1min":0.54300000000000004,"5min":0.54300000000000004,"15min":0.54300000000000004},"max":{"1min":0.91500000000000004,"5min":0.91500000000000004,"15min":0.91500000000000004},"last":0.50600000000000001},{"interface":"front","average":{"1min":0.70899999999999996,"5min":0.70899999999999996,"15min":0.70899999999999996},"min":{"1min":0.52900000000000003,"5min":0.52900000000000003,"15min":0.52900000000000003},"max":{"1min":1.0980000000000001,"5min":1.0980000000000001,"15min":1.0980000000000001},"last":0.75700000000000001}]},{"osd":1,"last update":"Sun Nov 26 07:18:42 2023","interfaces":[{"interface":"back","average":{"1min":0.68200000000000005,"5min":0.68200000000000005,"15min":0.68200000000000005},"min":{"1min":0.47999999999999998,"5min":0.47999999999999998,"15min":0.47999999999999998},"max":{"1min":1.0669999999999999,"5min":1.0669999999999999,"15min":1.0669999999999999},"last":0.63700000000000001},{"interface":"front","average":{"1min":0.83899999999999997,"5min":0.83899999999999997,"15min":0.83899999999999997},"min":{"1min":0.47599999999999998,"5min":0.47599999999999998,"15min":0.47599999999999998},"max":{"1min":4.5309999999999997,"5min":4.5309999999999997,"15min":4.5309999999999997},"last":0.53200000000000003}]},{"osd":2,"last update":"Sun Nov 26 07:18:42 2023","interfaces":[{"interface":"back","average":{"1min":0.67600000000000005,"5min":0.67600000000000005,"15min":0.67600000000000005},"min":{"1min":0.52000000000000002,"5min":0.52000000000000002,"15min":0.52000000000000002},"max":{"1min":0.873,"5min":0.873,"15min":0.873},"last":0.65700000000000003},{"interface":"front","average":{"1min":0.71299999999999997,"5min":0.71299999999999997,"15min":0.71299999999999997},"min":{"1min":0.54700000000000004,"5min":0.54700000000000004,"15min":0.54700000000000004},"max":{"1min":1.1339999999999999,"5min":1.1339999999999999,"15min":1.1339999999999999},"last":0.59099999999999997}]},{"osd":3,"last update":"Sun Nov 26 07:18:42 2023","interfaces":[{"interface":"back","average":{"1min":0.69699999999999995,"5min":0.69699999999999995,"15min":0.69699999999999995},"min":{"1min":0.51800000000000002,"5min":0.51800000000000002,"15min":0.51800000000000002},"max":{"1min":1.026,"5min":1.026,"15min":1.026},"last":0.71099999999999997},{"interface":"front","average":{"1min":0.73299999999999998,"5min":0.73299999999999998,"15min":0.73299999999999998},"min":{"1min":0.50600000000000001,"5min":0.50600000000000001,"15min":0.50600000000000001},"max":{"1min":1.4490000000000001,"5min":1.4490000000000001,"15min":1.4490000000000001},"last":0.67400000000000004}]},{"osd":4,"last update":"Sun Nov 26 07:18:42 2023","interfaces":[{"interface":"back","average":{"1min":0.59999999999999998,"5min":0.59999999999999998,"15min":0.59999999999999998},"min":{"1min":0.39400000000000002,"5min":0.39400000000000002,"15min":0.39400000000000002},"max":{"1min":0.94799999999999995,"5min":0.94799999999999995,"15min":0.94799999999999995},"last":0.68999999999999995},{"interface":"front","average":{"1min":0.67300000000000004,"5min":0.67300000000000004,"15min":0.67300000000000004},"min":{"1min":0.436,"5min":0.436,"15min":0.436},"max":{"1min":1.5509999999999999,"5min":1.5509999999999999,"15min":1.5509999999999999},"last":0.56499999999999995}]},{"osd":5,"last update":"Sun Nov 26 07:18:42 2023","interfaces":[{"interface":"back","average":{"1min":0.66600000000000004,"5min":0.66600000000000004,"15min":0.66600000000000004},"min":{"1min":0.443,"5min":0.443,"15min":0.443},"max":{"1min":1.415,"5min":1.415,"15min":1.415},"last":0.55400000000000005},{"interface":"front","average":{"1min":0.71399999999999997,"5min":0.71399999999999997,"15min":0.71399999999999997},"min":{"1min":0.53100000000000003,"5min":0.53100000000000003,"15min":0.53100000000000003},"max":{"1min":1.3859999999999999,"5min":1.3859999999999999,"15min":1.3859999999999999},"last":0.60799999999999998}]},{"osd":6,"last update":"Sun Nov 26 07:18:42 2023","interfaces":[{"interface":"back","average":{"1min":0.77100000000000002,"5min":0.77100000000000002,"15min":0.77100000000000002},"min":{"1min":0.64100000000000001,"5min":0.64100000000000001,"15min":0.64100000000000001},"max":{"1min":1.3069999999999999,"5min":1.3069999999999999,"15min":1.3069999999999999},"last":0.80600000000000005},{"interface":"front","average":{"1min":0.77100000000000002,"5min":0.77100000000000002,"15min":0.77100000000000002},"min":{"1min":0.57399999999999995,"5min":0.57399999999999995,"15min":0.57399999999999995},"max":{"1min":1.6759999999999999,"5min":1.6759999999999999,"15min":1.6759999999999999},"last":0.74099999999999999}]}]},{"osd":6,"up_from":13,"seq":55834574867,"num_pgs":19,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27392,"kb_used_data":372,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344448,"statfs":{"total":96636764160,"available":96608714752,"internally_reserved":0,"allocated":380928,"data_stored":147208,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":9,"apply_latency_ms":9,"commit_latency_ns":9000000,"apply_latency_ns":9000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Nov 26 07:18:45 2023","interfaces":[{"interface":"back","average":{"1min":0.68400000000000005,"5min":0.68400000000000005,"15min":0.68400000000000005},"min":{"1min":0.36699999999999999,"5min":0.36699999999999999,"15min":0.36699999999999999},"max":{"1min":1.7549999999999999,"5min":1.7549999999999999,"15min":1.7549999999999999},"last":0.72399999999999998},{"interface":"front","average":{"1min":0.95799999999999996,"5min":0.95799999999999996,"15min":0.95799999999999996},"min":{"1min":0.38300000000000001,"5min":0.38300000000000001,"15min":0.38300000000000001},"max":{"1min":5.7489999999999997,"5min":5.7489999999999997,"15min":5.7489999999999997},"last":0.86099999999999999}]},{"osd":1,"last update":"Sun Nov 26 07:18:45 2023","interfaces":[{"interface":"back","average":{"1min":0.91400000000000003,"5min":0.91400000000000003,"15min":0.91400000000000003},"min":{"1min":0.42099999999999999,"5min":0.42099999999999999,"15min":0.42099999999999999},"max":{"1min":5.4240000000000004,"5min":5.4240000000000004,"15min":5.4240000000000004},"last":0.78000000000000003},{"interface":"front","average":{"1min":0.89700000000000002,"5min":0.89700000000000002,"15min":0.89700000000000002},"min":{"1min":0.499,"5min":0.499,"15min":0.499},"max":{"1min":5.6020000000000003,"5min":5.6020000000000003,"15min":5.6020000000000003},"last":0.90700000000000003}]},{"osd":2,"last update":"Sun Nov 26 07:18:45 2023","interfaces":[{"interface":"back","average":{"1min":0.69699999999999995,"5min":0.69699999999999995,"15min":0.69699999999999995},"min":{"1min":0.47599999999999998,"5min":0.47599999999999998,"15min":0.47599999999999998},"max":{"1min":1.8520000000000001,"5min":1.8520000000000001,"15min":1.8520000000000001},"last":0.76000000000000001},{"interface":"front","average":{"1min":0.73399999999999999,"5min":0.73399999999999999,"15min":0.73399999999999999},"min":{"1min":0.46700000000000003,"5min":0.46700000000000003,"15min":0.46700000000000003},"max":{"1min":1.9159999999999999,"5min":1.9159999999999999,"15min":1.9159999999999999},"last":0.82799999999999996}]},{"osd":3,"last update":"Sun Nov 26 07:18:45 2023","interfaces":[{"interface":"back","average":{"1min":0.90200000000000002,"5min":0.90200000000000002,"15min":0.90200000000000002},"min":{"1min":0.502,"5min":0.502,"15min":0.502},"max":{"1min":5.4390000000000001,"5min":5.4390000000000001,"15min":5.4390000000000001},"last":0.80100000000000005},{"interface":"front","average":{"1min":0.94499999999999995,"5min":0.94499999999999995,"15min":0.94499999999999995},"min":{"1min":0.54400000000000004,"5min":0.54400000000000004,"15min":0.54400000000000004},"max":{"1min":5.5490000000000004,"5min":5.5490000000000004,"15min":5.5490000000000004},"last":0.84499999999999997}]},{"osd":4,"last update":"Sun Nov 26 07:18:45 2023","interfaces":[{"interface":"back","average":{"1min":0.64200000000000002,"5min":0.64200000000000002,"15min":0.64200000000000002},"min":{"1min":0.46899999999999997,"5min":0.46899999999999997,"15min":0.46899999999999997},"max":{"1min":0.85099999999999998,"5min":0.85099999999999998,"15min":0.85099999999999998},"last":0.51500000000000001},{"interface":"front","average":{"1min":0.67500000000000004,"5min":0.67500000000000004,"15min":0.67500000000000004},"min":{"1min":0.40699999999999997,"5min":0.40699999999999997,"15min":0.40699999999999997},"max":{"1min":0.93300000000000005,"5min":0.93300000000000005,"15min":0.93300000000000005},"last":0.47799999999999998}]},{"osd":5,"last update":"Sun Nov 26 07:18:45 2023","interfaces":[{"interface":"back","average":{"1min":0.65900000000000003,"5min":0.65900000000000003,"15min":0.65900000000000003},"min":{"1min":0.42499999999999999,"5min":0.42499999999999999,"15min":0.42499999999999999},"max":{"1min":0.79300000000000004,"5min":0.79300000000000004,"15min":0.79300000000000004},"last":0.57399999999999995},{"interface":"front","average":{"1min":0.71299999999999997,"5min":0.71299999999999997,"15min":0.71299999999999997},"min":{"1min":0.436,"5min":0.436,"15min":0.436},"max":{"1min":0.93000000000000005,"5min":0.93000000000000005,"15min":0.93000000000000005},"last":0.628}]},{"osd":7,"last update":"Sun Nov 26 07:18:45 2023","interfaces":[{"interface":"back","average":{"1min":0.74199999999999999,"5min":0.74199999999999999,"15min":0.74199999999999999},"min":{"1min":0.46600000000000003,"5min":0.46600000000000003,"15min":0.46600000000000003},"max":{"1min":0.998,"5min":0.998,"15min":0.998},"last":0.55200000000000005},{"interface":"front","average":{"1min":0.71599999999999997,"5min":0.71599999999999997,"15min":0.71599999999999997},"min":{"1min":0.498,"5min":0.498,"15min":0.498},"max":{"1min":0.86499999999999999,"5min":0.86499999999999999,"15min":0.86499999999999999},"last":0.61399999999999999}]}]},{"osd":5,"up_from":13,"seq":55834574865,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27392,"kb_used_data":372,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344448,"statfs":{"total":96636764160,"available":96608714752,"internally_reserved":0,"allocated":380928,"data_stored":147264,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Nov 26 07:18:45 2023","interfaces":[{"interface":"back","average":{"1min":0.66600000000000004,"5min":0.66600000000000004,"15min":0.66600000000000004},"min":{"1min":0.38100000000000001,"5min":0.38100000000000001,"15min":0.38100000000000001},"max":{"1min":1.0289999999999999,"5min":1.0289999999999999,"15min":1.0289999999999999},"last":0.59099999999999997},{"interface":"front","average":{"1min":0.66000000000000003,"5min":0.66000000000000003,"15min":0.66000000000000003},"min":{"1min":0.438,"5min":0.438,"15min":0.438},"max":{"1min":1.05,"5min":1.05,"15min":1.05},"last":0.54700000000000004}]},{"osd":1,"last update":"Sun Nov 26 07:18:45 2023","interfaces":[{"interface":"back","average":{"1min":0.66800000000000004,"5min":0.66800000000000004,"15min":0.66800000000000004},"min":{"1min":0.41599999999999998,"5min":0.41599999999999998,"15min":0.41599999999999998},"max":{"1min":0.86699999999999999,"5min":0.86699999999999999,"15min":0.86699999999999999},"last":0.79400000000000004},{"interface":"front","average":{"1min":0.68600000000000005,"5min":0.68600000000000005,"15min":0.68600000000000005},"min":{"1min":0.49199999999999999,"5min":0.49199999999999999,"15min":0.49199999999999999},"max":{"1min":0.98899999999999999,"5min":0.98899999999999999,"15min":0.98899999999999999},"last":0.52900000000000003}]},{"osd":2,"last update":"Sun Nov 26 07:18:45 2023","interfaces":[{"interface":"back","average":{"1min":0.67500000000000004,"5min":0.67500000000000004,"15min":0.67500000000000004},"min":{"1min":0.45900000000000002,"5min":0.45900000000000002,"15min":0.45900000000000002},"max":{"1min":0.89800000000000002,"5min":0.89800000000000002,"15min":0.89800000000000002},"last":0.67500000000000004},{"interface":"front","average":{"1min":0.67000000000000004,"5min":0.67000000000000004,"15min":0.67000000000000004},"min":{"1min":0.51200000000000001,"5min":0.51200000000000001,"15min":0.51200000000000001},"max":{"1min":0.89700000000000002,"5min":0.89700000000000002,"15min":0.89700000000000002},"last":0.73399999999999999}]},{"osd":3,"last update":"Sun Nov 26 07:18:45 2023","interfaces":[{"interface":"back","average":{"1min":0.71999999999999997,"5min":0.71999999999999997,"15min":0.71999999999999997},"min":{"1min":0.43099999999999999,"5min":0.43099999999999999,"15min":0.43099999999999999},"max":{"1min":1.0029999999999999,"5min":1.0029999999999999,"15min":1.0029999999999999},"last":0.629},{"interface":"front","average":{"1min":0.67500000000000004,"5min":0.67500000000000004,"15min":0.67500000000000004},"min":{"1min":0.44600000000000001,"5min":0.44600000000000001,"15min":0.44600000000000001},"max":{"1min":1.0209999999999999,"5min":1.0209999999999999,"15min":1.0209999999999999},"last":0.48199999999999998}]},{"osd":4,"last update":"Sun Nov 26 07:18:45 2023","interfaces":[{"interface":"back","average":{"1min":0.63500000000000001,"5min":0.63500000000000001,"15min":0.63500000000000001},"min":{"1min":0.39200000000000002,"5min":0.39200000000000002,"15min":0.39200000000000002},"max":{"1min":0.95099999999999996,"5min":0.95099999999999996,"15min":0.95099999999999996},"last":0.70199999999999996},{"interface":"front","average":{"1min":0.70899999999999996,"5min":0.70899999999999996,"15min":0.70899999999999996},"min":{"1min":0.41299999999999998,"5min":0.41299999999999998,"15min":0.41299999999999998},"max":{"1min":1.254,"5min":1.254,"15min":1.254},"last":0.75900000000000001}]},{"osd":6,"last update":"Sun Nov 26 07:18:45 2023","interfaces":[{"interface":"back","average":{"1min":0.67200000000000004,"5min":0.67200000000000004,"15min":0.67200000000000004},"min":{"1min":0.45600000000000002,"5min":0.45600000000000002,"15min":0.45600000000000002},"max":{"1min":0.94499999999999995,"5min":0.94499999999999995,"15min":0.94499999999999995},"last":0.80900000000000005},{"interface":"front","average":{"1min":0.72699999999999998,"5min":0.72699999999999998,"15min":0.72699999999999998},"min":{"1min":0.44,"5min":0.44,"15min":0.44},"max":{"1min":1.0649999999999999,"5min":1.0649999999999999,"15min":1.0649999999999999},"last":0.77700000000000002}]},{"osd":7,"last update":"Sun Nov 26 07:18:45 2023","interfaces":[{"interface":"back","average":{"1min":0.80400000000000005,"5min":0.80400000000000005,"15min":0.80400000000000005},"min":{"1min":0.48199999999999998,"5min":0.48199999999999998,"15min":0.48199999999999998},"max":{"1min":1.4079999999999999,"5min":1.4079999999999999,"15min":1.4079999999999999},"last":0.85399999999999998},{"interface":"front","average":{"1min":0.77500000000000002,"5min":0.77500000000000002,"15min":0.77500000000000002},"min":{"1min":0.51600000000000001,"5min":0.51600000000000001,"15min":0.51600000000000001},"max":{"1min":1.0169999999999999,"5min":1.0169999999999999,"15min":1.0169999999999999},"last":0.82499999999999996}]}]},{"osd":4,"up_from":13,"seq":55834574867,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27392,"kb_used_data":372,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344448,"statfs":{"total":96636764160,"available":96608714752,"internally_reserved":0,"allocated":380928,"data_stored":146696,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Nov 26 07:18:46 2023","interfaces":[{"interface":"back","average":{"1min":0.79400000000000004,"5min":0.79400000000000004,"15min":0.79400000000000004},"min":{"1min":0.45600000000000002,"5min":0.45600000000000002,"15min":0.45600000000000002},"max":{"1min":4.1689999999999996,"5min":4.1689999999999996,"15min":4.1689999999999996},"last":0.60299999999999998},{"interface":"front","average":{"1min":1.002,"5min":1.002,"15min":1.002},"min":{"1min":0.47499999999999998,"5min":0.47499999999999998,"15min":0.47499999999999998},"max":{"1min":7.7889999999999997,"5min":7.7889999999999997,"15min":7.7889999999999997},"last":0.64300000000000002}]},{"osd":1,"last update":"Sun Nov 26 07:18:46 2023","interfaces":[{"interface":"back","average":{"1min":0.64700000000000002,"5min":0.64700000000000002,"15min":0.64700000000000002},"min":{"1min":0.52300000000000002,"5min":0.52300000000000002,"15min":0.52300000000000002},"max":{"1min":0.78400000000000003,"5min":0.78400000000000003,"15min":0.78400000000000003},"last":0.72499999999999998},{"interface":"front","average":{"1min":0.81999999999999995,"5min":0.81999999999999995,"15min":0.81999999999999995},"min":{"1min":0.44500000000000001,"5min":0.44500000000000001,"15min":0.44500000000000001},"max":{"1min":4.3140000000000001,"5min":4.3140000000000001,"15min":4.3140000000000001},"last":0.66700000000000004}]},{"osd":2,"last update":"Sun Nov 26 07:18:46 2023","interfaces":[{"interface":"back","average":{"1min":0.80700000000000005,"5min":0.80700000000000005,"15min":0.80700000000000005},"min":{"1min":0.57799999999999996,"5min":0.57799999999999996,"15min":0.57799999999999996},"max":{"1min":2.7850000000000001,"5min":2.7850000000000001,"15min":2.7850000000000001},"last":0.56000000000000005},{"interface":"front","average":{"1min":1.0660000000000001,"5min":1.0660000000000001,"15min":1.0660000000000001},"min":{"1min":0.53400000000000003,"5min":0.53400000000000003,"15min":0.53400000000000003},"max":{"1min":7.6470000000000002,"5min":7.6470000000000002,"15min":7.6470000000000002},"last":0.76100000000000001}]},{"osd":3,"last update":"Sun Nov 26 07:18:46 2023","interfaces":[{"interface":"back","average":{"1min":0.72999999999999998,"5min":0.72999999999999998,"15min":0.72999999999999998},"min":{"1min":0.48599999999999999,"5min":0.48599999999999999,"15min":0.48599999999999999},"max":{"1min":1.591,"5min":1.591,"15min":1.591},"last":0.70699999999999996},{"interface":"front","average":{"1min":0.879,"5min":0.879,"15min":0.879},"min":{"1min":0.56899999999999995,"5min":0.56899999999999995,"15min":0.56899999999999995},"max":{"1min":3.452,"5min":3.452,"15min":3.452},"last":0.68500000000000005}]},{"osd":5,"last update":"Sun Nov 26 07:18:46 2023","interfaces":[{"interface":"back","average":{"1min":0.61099999999999999,"5min":0.61099999999999999,"15min":0.61099999999999999},"min":{"1min":0.43099999999999999,"5min":0.43099999999999999,"15min":0.43099999999999999},"max":{"1min":0.96199999999999997,"5min":0.96199999999999997,"15min":0.96199999999999997},"last":0.46100000000000002},{"interface":"front","average":{"1min":0.64100000000000001,"5min":0.64100000000000001,"15min":0.64100000000000001},"min":{"1min":0.41499999999999998,"5min":0.41499999999999998,"15min":0.41499999999999998},"max":{"1min":1.538,"5min":1.538,"15min":1.538},"last":0.52100000000000002}]},{"osd":6,"last update":"Sun Nov 26 07:18:46 2023","interfaces":[{"interface":"back","average":{"1min":0.79400000000000004,"5min":0.79400000000000004,"15min":0.79400000000000004},"min":{"1min":0.39400000000000002,"5min":0.39400000000000002,"15min":0.39400000000000002},"max":{"1min":2.8330000000000002,"5min":2.8330000000000002,"15min":2.8330000000000002},"last":0.432},{"interface":"front","average":{"1min":1.0089999999999999,"5min":1.0089999999999999,"15min":1.0089999999999999},"min":{"1min":0.42899999999999999,"5min":0.42899999999999999,"15min":0.42899999999999999},"max":{"1min":7.6050000000000004,"5min":7.6050000000000004,"15min":7.6050000000000004},"last":0.58299999999999996}]},{"osd":7,"last update":"Sun Nov 26 07:18:46 2023","interfaces":[{"interface":"back","average":{"1min":0.86099999999999999,"5min":0.86099999999999999,"15min":0.86099999999999999},"min":{"1min":0.44700000000000001,"5min":0.44700000000000001,"15min":0.44700000000000001},"max":{"1min":4.3689999999999998,"5min":4.3689999999999998,"15min":4.3689999999999998},"last":0.62},{"interface":"front","average":{"1min":0.89200000000000002,"5min":0.89200000000000002,"15min":0.89200000000000002},"min":{"1min":0.505,"5min":0.505,"15min":0.505},"max":{"1min":3.403,"5min":3.403,"15min":3.403},"last":0.48399999999999999}]}]},{"osd":3,"up_from":13,"seq":55834574868,"num_pgs":31,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27400,"kb_used_data":380,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344440,"statfs":{"total":96636764160,"available":96608706560,"internally_reserved":0,"allocated":389120,"data_stored":148226,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1592,"internal_metadata":27654600},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":7,"apply_latency_ms":7,"commit_latency_ns":7000000,"apply_latency_ns":7000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Nov 26 07:18:43 2023","interfaces":[{"interface":"back","average":{"1min":0.379,"5min":0.379,"15min":0.379},"min":{"1min":0.26100000000000001,"5min":0.26100000000000001,"15min":0.26100000000000001},"max":{"1min":0.58599999999999997,"5min":0.58599999999999997,"15min":0.58599999999999997},"last":0.40500000000000003},{"interface":"front","average":{"1min":0.433,"5min":0.433,"15min":0.433},"min":{"1min":0.30199999999999999,"5min":0.30199999999999999,"15min":0.30199999999999999},"max":{"1min":0.63,"5min":0.63,"15min":0.63},"last":0.34899999999999998}]},{"osd":1,"last update":"Sun Nov 26 07:18:43 2023","interfaces":[{"interface":"back","average":{"1min":0.42099999999999999,"5min":0.42099999999999999,"15min":0.42099999999999999},"min":{"1min":0.312,"5min":0.312,"15min":0.312},"max":{"1min":0.752,"5min":0.752,"15min":0.752},"last":0.379},{"interface":"front","average":{"1min":0.48399999999999999,"5min":0.48399999999999999,"15min":0.48399999999999999},"min":{"1min":0.38,"5min":0.38,"15min":0.38},"max":{"1min":0.628,"5min":0.628,"15min":0.628},"last":0.46300000000000002}]},{"osd":2,"last update":"Sun Nov 26 07:18:43 2023","interfaces":[{"interface":"back","average":{"1min":0.53300000000000003,"5min":0.53300000000000003,"15min":0.53300000000000003},"min":{"1min":0.40300000000000002,"5min":0.40300000000000002,"15min":0.40300000000000002},"max":{"1min":0.82199999999999995,"5min":0.82199999999999995,"15min":0.82199999999999995},"last":0.50900000000000001},{"interface":"front","average":{"1min":0.505,"5min":0.505,"15min":0.505},"min":{"1min":0.35999999999999999,"5min":0.35999999999999999,"15min":0.35999999999999999},"max":{"1min":0.72199999999999998,"5min":0.72199999999999998,"15min":0.72199999999999998},"last":0.49399999999999999}]},{"osd":4,"last update":"Sun Nov 26 07:18:43 2023","interfaces":[{"interface":"back","average":{"1min":0.70599999999999996,"5min":0.70599999999999996,"15min":0.70599999999999996},"min":{"1min":0.48199999999999998,"5min":0.48199999999999998,"15min":0.48199999999999998},"max":{"1min":0.96199999999999997,"5min":0.96199999999999997,"15min":0.96199999999999997},"last":0.58499999999999996},{"interface":"front","average":{"1min":0.72799999999999998,"5min":0.72799999999999998,"15min":0.72799999999999998},"min":{"1min":0.441,"5min":0.441,"15min":0.441},"max":{"1min":1.171,"5min":1.171,"15min":1.171},"last":0.67400000000000004}]},{"osd":5,"last update":"Sun Nov 26 07:18:43 2023","interfaces":[{"interface":"back","average":{"1min":0.73899999999999999,"5min":0.73899999999999999,"15min":0.73899999999999999},"min":{"1min":0.53700000000000003,"5min":0.53700000000000003,"15min":0.53700000000000003},"max":{"1min":0.94099999999999995,"5min":0.94099999999999995,"15min":0.94099999999999995},"last":0.57099999999999995},{"interface":"front","average":{"1min":0.73799999999999999,"5min":0.73799999999999999,"15min":0.73799999999999999},"min":{"1min":0.504,"5min":0.504,"15min":0.504},"max":{"1min":1.0820000000000001,"5min":1.0820000000000001,"15min":1.0820000000000001},"last":0.54400000000000004}]},{"osd":6,"last update":"Sun Nov 26 07:18:43 2023","interfaces":[{"interface":"back","average":{"1min":0.81399999999999995,"5min":0.81399999999999995,"15min":0.81399999999999995},"min":{"1min":0.55200000000000005,"5min":0.55200000000000005,"15min":0.55200000000000005},"max":{"1min":1.2729999999999999,"5min":1.2729999999999999,"15min":1.2729999999999999},"last":0.55700000000000005},{"interface":"front","average":{"1min":0.79300000000000004,"5min":0.79300000000000004,"15min":0.79300000000000004},"min":{"1min":0.47299999999999998,"5min":0.47299999999999998,"15min":0.47299999999999998},"max":{"1min":1.204,"5min":1.204,"15min":1.204},"last":0.68799999999999994}]},{"osd":7,"last update":"Sun Nov 26 07:18:43 2023","interfaces":[{"interface":"back","average":{"1min":0.80100000000000005,"5min":0.80100000000000005,"15min":0.80100000000000005},"min":{"1min":0.63300000000000001,"5min":0.63300000000000001,"15min":0.63300000000000001},"max":{"1min":1.3600000000000001,"5min":1.3600000000000001,"15min":1.3600000000000001},"last":0.56599999999999995},{"interface":"front","average":{"1min":0.82599999999999996,"5min":0.82599999999999996,"15min":0.82599999999999996},"min":{"1min":0.50600000000000001,"5min":0.50600000000000001,"15min":0.50600000000000001},"max":{"1min":1.617,"5min":1.617,"15min":1.617},"last":0.71099999999999997}]}]},{"osd":2,"up_from":13,"seq":55834574868,"num_pgs":17,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27388,"kb_used_data":368,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344452,"statfs":{"total":96636764160,"available":96608718848,"internally_reserved":0,"allocated":376832,"data_stored":146662,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Nov 26 07:18:43 2023","interfaces":[{"interface":"back","average":{"1min":0.71399999999999997,"5min":0.71399999999999997,"15min":0.71399999999999997},"min":{"1min":0.28699999999999998,"5min":0.28699999999999998,"15min":0.28699999999999998},"max":{"1min":4.7030000000000003,"5min":4.7030000000000003,"15min":4.7030000000000003},"last":0.38300000000000001},{"interface":"front","average":{"1min":0.56200000000000006,"5min":0.56200000000000006,"15min":0.56200000000000006},"min":{"1min":0.32300000000000001,"5min":0.32300000000000001,"15min":0.32300000000000001},"max":{"1min":2.4340000000000002,"5min":2.4340000000000002,"15min":2.4340000000000002},"last":0.41399999999999998}]},{"osd":1,"last update":"Sun Nov 26 07:18:43 2023","interfaces":[{"interface":"back","average":{"1min":0.86399999999999999,"5min":0.86399999999999999,"15min":0.86399999999999999},"min":{"1min":0.34599999999999997,"5min":0.34599999999999997,"15min":0.34599999999999997},"max":{"1min":4.2690000000000001,"5min":4.2690000000000001,"15min":4.2690000000000001},"last":0.435},{"interface":"front","average":{"1min":0.74199999999999999,"5min":0.74199999999999999,"15min":0.74199999999999999},"min":{"1min":0.31900000000000001,"5min":0.31900000000000001,"15min":0.31900000000000001},"max":{"1min":3.9940000000000002,"5min":3.9940000000000002,"15min":3.9940000000000002},"last":0.45000000000000001}]},{"osd":3,"last update":"Sun Nov 26 07:18:43 2023","interfaces":[{"interface":"back","average":{"1min":0.64300000000000002,"5min":0.64300000000000002,"15min":0.64300000000000002},"min":{"1min":0.34399999999999997,"5min":0.34399999999999997,"15min":0.34399999999999997},"max":{"1min":4.0300000000000002,"5min":4.0300000000000002,"15min":4.0300000000000002},"last":0.36699999999999999},{"interface":"front","average":{"1min":0.91400000000000003,"5min":0.91400000000000003,"15min":0.91400000000000003},"min":{"1min":0.41699999999999998,"5min":0.41699999999999998,"15min":0.41699999999999998},"max":{"1min":4.5810000000000004,"5min":4.5810000000000004,"15min":4.5810000000000004},"last":0.61199999999999999}]},{"osd":4,"last update":"Sun Nov 26 07:18:43 2023","interfaces":[{"interface":"back","average":{"1min":0.98499999999999999,"5min":0.98499999999999999,"15min":0.98499999999999999},"min":{"1min":0.55100000000000005,"5min":0.55100000000000005,"15min":0.55100000000000005},"max":{"1min":4.4269999999999996,"5min":4.4269999999999996,"15min":4.4269999999999996},"last":0.84299999999999997},{"interface":"front","average":{"1min":1.0149999999999999,"5min":1.0149999999999999,"15min":1.0149999999999999},"min":{"1min":0.60299999999999998,"5min":0.60299999999999998,"15min":0.60299999999999998},"max":{"1min":4.085,"5min":4.085,"15min":4.085},"last":0.746}]},{"osd":5,"last update":"Sun Nov 26 07:18:43 2023","interfaces":[{"interface":"back","average":{"1min":0.874,"5min":0.874,"15min":0.874},"min":{"1min":0.52900000000000003,"5min":0.52900000000000003,"15min":0.52900000000000003},"max":{"1min":3.9590000000000001,"5min":3.9590000000000001,"15min":3.9590000000000001},"last":0.86299999999999999},{"interface":"front","average":{"1min":0.98799999999999999,"5min":0.98799999999999999,"15min":0.98799999999999999},"min":{"1min":0.58799999999999997,"5min":0.58799999999999997,"15min":0.58799999999999997},"max":{"1min":4.3849999999999998,"5min":4.3849999999999998,"15min":4.3849999999999998},"last":0.78800000000000003}]},{"osd":6,"last update":"Sun Nov 26 07:18:43 2023","interfaces":[{"interface":"back","average":{"1min":0.996,"5min":0.996,"15min":0.996},"min":{"1min":0.56599999999999995,"5min":0.56599999999999995,"15min":0.56599999999999995},"max":{"1min":4.0449999999999999,"5min":4.0449999999999999,"15min":4.0449999999999999},"last":0.70099999999999996},{"interface":"front","average":{"1min":0.91800000000000004,"5min":0.91800000000000004,"15min":0.91800000000000004},"min":{"1min":0.57099999999999995,"5min":0.57099999999999995,"15min":0.57099999999999995},"max":{"1min":3.992,"5min":3.992,"15min":3.992},"last":0.80500000000000005}]},{"osd":7,"last update":"Sun Nov 26 07:18:43 2023","interfaces":[{"interface":"back","average":{"1min":1.0349999999999999,"5min":1.0349999999999999,"15min":1.0349999999999999},"min":{"1min":0.63100000000000001,"5min":0.63100000000000001,"15min":0.63100000000000001},"max":{"1min":4.1219999999999999,"5min":4.1219999999999999,"15min":4.1219999999999999},"last":0.82599999999999996},{"interface":"front","average":{"1min":0.93799999999999994,"5min":0.93799999999999994,"15min":0.93799999999999994},"min":{"1min":0.57699999999999996,"5min":0.57699999999999996,"15min":0.57699999999999996},"max":{"1min":3.9159999999999999,"5min":3.9159999999999999,"15min":3.9159999999999999},"last":0.73099999999999998}]}]},{"osd":1,"up_from":13,"seq":55834574867,"num_pgs":32,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27408,"kb_used_data":380,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344432,"statfs":{"total":96636764160,"available":96608698368,"internally_reserved":0,"allocated":389120,"data_stored":148226,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Nov 26 07:18:41 2023","interfaces":[{"interface":"back","average":{"1min":0.41699999999999998,"5min":0.41699999999999998,"15min":0.41699999999999998},"min":{"1min":0.26300000000000001,"5min":0.26300000000000001,"15min":0.26300000000000001},"max":{"1min":0.61199999999999999,"5min":0.61199999999999999,"15min":0.61199999999999999},"last":0.33400000000000002},{"interface":"front","average":{"1min":0.48599999999999999,"5min":0.48599999999999999,"15min":0.48599999999999999},"min":{"1min":0.30499999999999999,"5min":0.30499999999999999,"15min":0.30499999999999999},"max":{"1min":0.78700000000000003,"5min":0.78700000000000003,"15min":0.78700000000000003},"last":0.46400000000000002}]},{"osd":2,"last update":"Sun Nov 26 07:18:41 2023","interfaces":[{"interface":"back","average":{"1min":0.45500000000000002,"5min":0.45500000000000002,"15min":0.45500000000000002},"min":{"1min":0.33400000000000002,"5min":0.33400000000000002,"15min":0.33400000000000002},"max":{"1min":0.67600000000000005,"5min":0.67600000000000005,"15min":0.67600000000000005},"last":0.40100000000000002},{"interface":"front","average":{"1min":0.50800000000000001,"5min":0.50800000000000001,"15min":0.50800000000000001},"min":{"1min":0.32300000000000001,"5min":0.32300000000000001,"15min":0.32300000000000001},"max":{"1min":0.66500000000000004,"5min":0.66500000000000004,"15min":0.66500000000000004},"last":0.56999999999999995}]},{"osd":3,"last update":"Sun Nov 26 07:18:41 2023","interfaces":[{"interface":"back","average":{"1min":0.53000000000000003,"5min":0.53000000000000003,"15min":0.53000000000000003},"min":{"1min":0.38400000000000001,"5min":0.38400000000000001,"15min":0.38400000000000001},"max":{"1min":0.73699999999999999,"5min":0.73699999999999999,"15min":0.73699999999999999},"last":0.54200000000000004},{"interface":"front","average":{"1min":0.497,"5min":0.497,"15min":0.497},"min":{"1min":0.36199999999999999,"5min":0.36199999999999999,"15min":0.36199999999999999},"max":{"1min":0.80700000000000005,"5min":0.80700000000000005,"15min":0.80700000000000005},"last":0.52000000000000002}]},{"osd":4,"last update":"Sun Nov 26 07:18:41 2023","interfaces":[{"interface":"back","average":{"1min":0.69399999999999995,"5min":0.69399999999999995,"15min":0.69399999999999995},"min":{"1min":0.48599999999999999,"5min":0.48599999999999999,"15min":0.48599999999999999},"max":{"1min":1.0229999999999999,"5min":1.0229999999999999,"15min":1.0229999999999999},"last":0.73099999999999998},{"interface":"front","average":{"1min":0.68000000000000005,"5min":0.68000000000000005,"15min":0.68000000000000005},"min":{"1min":0.49199999999999999,"5min":0.49199999999999999,"15min":0.49199999999999999},"max":{"1min":1.0629999999999999,"5min":1.0629999999999999,"15min":1.0629999999999999},"last":0.66100000000000003}]},{"osd":5,"last update":"Sun Nov 26 07:18:41 2023","interfaces":[{"interface":"back","average":{"1min":0.71899999999999997,"5min":0.71899999999999997,"15min":0.71899999999999997},"min":{"1min":0.49299999999999999,"5min":0.49299999999999999,"15min":0.49299999999999999},"max":{"1min":1.0880000000000001,"5min":1.0880000000000001,"15min":1.0880000000000001},"last":0.75800000000000001},{"interface":"front","average":{"1min":0.71699999999999997,"5min":0.71699999999999997,"15min":0.71699999999999997},"min":{"1min":0.53500000000000003,"5min":0.53500000000000003,"15min":0.53500000000000003},"max":{"1min":1.014,"5min":1.014,"15min":1.014},"last":0.745}]},{"osd":6,"last update":"Sun Nov 26 07:18:41 2023","interfaces":[{"interface":"back","average":{"1min":0.70399999999999996,"5min":0.70399999999999996,"15min":0.70399999999999996},"min":{"1min":0.50600000000000001,"5min":0.50600000000000001,"15min":0.50600000000000001},"max":{"1min":0.97599999999999998,"5min":0.97599999999999998,"15min":0.97599999999999998},"last":0.69099999999999995},{"interface":"front","average":{"1min":0.70099999999999996,"5min":0.70099999999999996,"15min":0.70099999999999996},"min":{"1min":0.55000000000000004,"5min":0.55000000000000004,"15min":0.55000000000000004},"max":{"1min":1.1339999999999999,"5min":1.1339999999999999,"15min":1.1339999999999999},"last":0.78200000000000003}]},{"osd":7,"last update":"Sun Nov 26 07:18:41 2023","interfaces":[{"interface":"back","average":{"1min":0.71499999999999997,"5min":0.71499999999999997,"15min":0.71499999999999997},"min":{"1min":0.55100000000000005,"5min":0.55100000000000005,"15min":0.55100000000000005},"max":{"1min":0.92900000000000005,"5min":0.92900000000000005,"15min":0.92900000000000005},"last":0.71699999999999997},{"interface":"front","average":{"1min":0.753,"5min":0.753,"15min":0.753},"min":{"1min":0.56100000000000005,"5min":0.56100000000000005,"15min":0.56100000000000005},"max":{"1min":1.024,"5min":1.024,"15min":1.024},"last":0.84599999999999997}]}]},{"osd":0,"up_from":13,"seq":55834574867,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27968,"kb_used_data":944,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343872,"statfs":{"total":96636764160,"available":96608124928,"internally_reserved":0,"allocated":966656,"data_stored":736960,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":7,"apply_latency_ms":7,"commit_latency_ns":7000000,"apply_latency_ns":7000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Sun Nov 26 07:18:41 2023","interfaces":[{"interface":"back","average":{"1min":0.43099999999999999,"5min":0.43099999999999999,"15min":0.43099999999999999},"min":{"1min":0.28000000000000003,"5min":0.28000000000000003,"15min":0.28000000000000003},"max":{"1min":0.63100000000000001,"5min":0.63100000000000001,"15min":0.63100000000000001},"last":0.379},{"interface":"front","average":{"1min":0.48299999999999998,"5min":0.48299999999999998,"15min":0.48299999999999998},"min":{"1min":0.29899999999999999,"5min":0.29899999999999999,"15min":0.29899999999999999},"max":{"1min":1.133,"5min":1.133,"15min":1.133},"last":0.48699999999999999}]},{"osd":2,"last update":"Sun Nov 26 07:18:41 2023","interfaces":[{"interface":"back","average":{"1min":0.53700000000000003,"5min":0.53700000000000003,"15min":0.53700000000000003},"min":{"1min":0.40899999999999997,"5min":0.40899999999999997,"15min":0.40899999999999997},"max":{"1min":0.67400000000000004,"5min":0.67400000000000004,"15min":0.67400000000000004},"last":0.46100000000000002},{"interface":"front","average":{"1min":0.56399999999999995,"5min":0.56399999999999995,"15min":0.56399999999999995},"min":{"1min":0.36399999999999999,"5min":0.36399999999999999,"15min":0.36399999999999999},"max":{"1min":0.72499999999999998,"5min":0.72499999999999998,"15min":0.72499999999999998},"last":0.39800000000000002}]},{"osd":3,"last update":"Sun Nov 26 07:18:41 2023","interfaces":[{"interface":"back","average":{"1min":0.53100000000000003,"5min":0.53100000000000003,"15min":0.53100000000000003},"min":{"1min":0.35799999999999998,"5min":0.35799999999999998,"15min":0.35799999999999998},"max":{"1min":0.97399999999999998,"5min":0.97399999999999998,"15min":0.97399999999999998},"last":0.54400000000000004},{"interface":"front","average":{"1min":0.56200000000000006,"5min":0.56200000000000006,"15min":0.56200000000000006},"min":{"1min":0.40600000000000003,"5min":0.40600000000000003,"15min":0.40600000000000003},"max":{"1min":1.2509999999999999,"5min":1.2509999999999999,"15min":1.2509999999999999},"last":0.52100000000000002}]},{"osd":4,"last update":"Sun Nov 26 07:18:41 2023","interfaces":[{"interface":"back","average":{"1min":0.75900000000000001,"5min":0.75900000000000001,"15min":0.75900000000000001},"min":{"1min":0.60999999999999999,"5min":0.60999999999999999,"15min":0.60999999999999999},"max":{"1min":1.2010000000000001,"5min":1.2010000000000001,"15min":1.2010000000000001},"last":0.78900000000000003},{"interface":"front","average":{"1min":0.78900000000000003,"5min":0.78900000000000003,"15min":0.78900000000000003},"min":{"1min":0.59999999999999998,"5min":0.59999999999999998,"15min":0.59999999999999998},"max":{"1min":0.99399999999999999,"5min":0.99399999999999999,"15min":0.99399999999999999},"last":0.91800000000000004}]},{"osd":5,"last update":"Sun Nov 26 07:18:41 2023","interfaces":[{"interface":"back","average":{"1min":0.76000000000000001,"5min":0.76000000000000001,"15min":0.76000000000000001},"min":{"1min":0.53000000000000003,"5min":0.53000000000000003,"15min":0.53000000000000003},"max":{"1min":1.1579999999999999,"5min":1.1579999999999999,"15min":1.1579999999999999},"last":0.88800000000000001},{"interface":"front","average":{"1min":0.79700000000000004,"5min":0.79700000000000004,"15min":0.79700000000000004},"min":{"1min":0.59699999999999998,"5min":0.59699999999999998,"15min":0.59699999999999998},"max":{"1min":1.0409999999999999,"5min":1.0409999999999999,"15min":1.0409999999999999},"last":0.82299999999999995}]},{"osd":6,"last update":"Sun Nov 26 07:18:41 2023","interfaces":[{"interface":"back","average":{"1min":0.76400000000000001,"5min":0.76400000000000001,"15min":0.76400000000000001},"min":{"1min":0.56599999999999995,"5min":0.56599999999999995,"15min":0.56599999999999995},"max":{"1min":1.3029999999999999,"5min":1.3029999999999999,"15min":1.3029999999999999},"last":0.76300000000000001},{"interface":"front","average":{"1min":0.76900000000000002,"5min":0.76900000000000002,"15min":0.76900000000000002},"min":{"1min":0.58599999999999997,"5min":0.58599999999999997,"15min":0.58599999999999997},"max":{"1min":1.2210000000000001,"5min":1.2210000000000001,"15min":1.2210000000000001},"last":0.66400000000000003}]},{"osd":7,"last update":"Sun Nov 26 07:18:41 2023","interfaces":[{"interface":"back","average":{"1min":0.81100000000000005,"5min":0.81100000000000005,"15min":0.81100000000000005},"min":{"1min":0.59999999999999998,"5min":0.59999999999999998,"15min":0.59999999999999998},"max":{"1min":1.177,"5min":1.177,"15min":1.177},"last":0.81200000000000006},{"interface":"front","average":{"1min":0.79900000000000004,"5min":0.79900000000000004,"15min":0.79900000000000004},"min":{"1min":0.60899999999999999,"5min":0.60899999999999999,"15min":0.60899999999999999},"max":{"1min":1.2390000000000001,"5min":1.2390000000000001,"15min":1.2390000000000001},"last":0.71299999999999997}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1634,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":70,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1634,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":104,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":672,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":616,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-11-26T07:19:01.588 INFO:tasks.cephfs.filesystem.ceph_manager:clean! 2023-11-26T07:19:01.589 DEBUG:teuthology.orchestra.run.smithi143:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth caps client.0 mon 'allow r' osd 'allow rw tag cephfs data=cephfs' mds allow 2023-11-26T07:19:01.772 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:01.764+0000 7fa43569a640 1 Processor -- start 2023-11-26T07:19:01.772 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:01.764+0000 7fa43569a640 1 -- start start 2023-11-26T07:19:01.773 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:01.768+0000 7fa43569a640 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7fa4300d1c20 0x7fa4300d0720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:19:01.774 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:01.768+0000 7fa43569a640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fa4301608f0 0x7fa430160d60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:19:01.774 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:01.768+0000 7fa43569a640 1 -- --> v1:172.21.15.188:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa4300b2510 con 0x7fa4300d1840 2023-11-26T07:19:01.774 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:01.768+0000 7fa43569a640 1 -- --> v2:172.21.15.188:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa430161f80 con 0x7fa4300d1c20 2023-11-26T07:19:01.774 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:01.768+0000 7fa43569a640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa4300cfc40 con 0x7fa4301608f0 2023-11-26T07:19:01.774 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:01.768+0000 7fa427fff640 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7fa4300d1c20 0x7fa4300d0720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:19:01.774 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:01.768+0000 7fa42f7fe640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fa4301608f0 0x7fa430160d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:19:01.774 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:01.768+0000 7fa42effd640 1 --1- >> v1:172.21.15.188:6789/0 conn(0x7fa4300d1840 0x7fa430161410 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.188:6789/0 says I am v1:172.21.15.143:41516/0 (socket says 172.21.15.143:41516) 2023-11-26T07:19:01.774 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:01.768+0000 7fa42f7fe640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fa4301608f0 0x7fa430160d60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.188:3301/0 says I am v2:172.21.15.143:40786/0 (socket says 172.21.15.143:40786) 2023-11-26T07:19:01.774 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:01.768+0000 7fa42effd640 1 -- 172.21.15.143:0/3722830289 learned_addr learned my addr 172.21.15.143:0/3722830289 (peer_addr_for_me v1:172.21.15.143:0/0) 2023-11-26T07:19:01.775 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:01.768+0000 7fa427fff640 1 -- 172.21.15.143:0/3722830289 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fa4301608f0 msgr2=0x7fa430160d60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-26T07:19:01.775 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:01.768+0000 7fa427fff640 1 --2- 172.21.15.143:0/3722830289 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fa4301608f0 0x7fa430160d60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:19:01.775 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:01.768+0000 7fa427fff640 1 -- 172.21.15.143:0/3722830289 >> v1:172.21.15.188:6789/0 conn(0x7fa4300d1840 legacy=0x7fa430161410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:19:01.775 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:01.768+0000 7fa427fff640 1 -- 172.21.15.143:0/3722830289 --> v2:172.21.15.188:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa4300cff20 con 0x7fa4300d1c20 2023-11-26T07:19:01.775 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:01.768+0000 7fa42e7fc640 1 -- 172.21.15.143:0/3722830289 <== mon.1 v1:172.21.15.188:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4017170552 0 0) 0x7fa4300b2510 con 0x7fa4300d1840 2023-11-26T07:19:01.775 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:01.768+0000 7fa42f7fe640 1 --2- 172.21.15.143:0/3722830289 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fa4301608f0 0x7fa430160d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-26T07:19:01.777 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:01.772+0000 7fa427fff640 1 --2- 172.21.15.143:0/3722830289 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fa4300d1c20 0x7fa4300d0720 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7fa4180097f0 tx=0x7fa41801c3f0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=9b264d04f27e3d22 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:19:01.777 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:01.772+0000 7fa42e7fc640 1 -- 172.21.15.143:0/3722830289 <== mon.1 v2:172.21.15.188:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa4180ab600 con 0x7fa4300d1c20 2023-11-26T07:19:01.778 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:01.772+0000 7fa42e7fc640 1 -- 172.21.15.143:0/3722830289 <== mon.1 v2:172.21.15.188:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa4180a7050 con 0x7fa4300d1c20 2023-11-26T07:19:01.778 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:01.772+0000 7fa42e7fc640 1 -- 172.21.15.143:0/3722830289 <== mon.1 v2:172.21.15.188:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa4180ab980 con 0x7fa4300d1c20 2023-11-26T07:19:01.778 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:01.772+0000 7fa43569a640 1 -- 172.21.15.143:0/3722830289 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fa4300d1c20 msgr2=0x7fa4300d0720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:19:01.779 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:01.772+0000 7fa43569a640 1 --2- 172.21.15.143:0/3722830289 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fa4300d1c20 0x7fa4300d0720 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7fa4180097f0 tx=0x7fa41801c3f0 comp rx=0 tx=0).stop 2023-11-26T07:19:01.779 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:01.772+0000 7fa43569a640 1 -- 172.21.15.143:0/3722830289 shutdown_connections 2023-11-26T07:19:01.779 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:01.772+0000 7fa43569a640 1 --2- 172.21.15.143:0/3722830289 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fa4300d1c20 0x7fa4300d0720 unknown :-1 s=CLOSED pgs=190 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:19:01.779 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:01.772+0000 7fa43569a640 1 --2- 172.21.15.143:0/3722830289 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fa4301608f0 0x7fa430160d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:19:01.779 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:01.772+0000 7fa43569a640 1 -- 172.21.15.143:0/3722830289 >> 172.21.15.143:0/3722830289 conn(0x7fa43009c570 msgr2=0x7fa4300d3240 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:19:01.779 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:01.772+0000 7fa43569a640 1 -- 172.21.15.143:0/3722830289 shutdown_connections 2023-11-26T07:19:01.779 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:01.772+0000 7fa43569a640 1 -- 172.21.15.143:0/3722830289 wait complete. 2023-11-26T07:19:01.780 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:01.772+0000 7fa43569a640 1 Processor -- start 2023-11-26T07:19:01.780 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:01.772+0000 7fa43569a640 1 -- start start 2023-11-26T07:19:01.781 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:01.776+0000 7fa43569a640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fa4300d1840 0x7fa430156400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:19:01.781 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:01.776+0000 7fa43569a640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fa4300d1c20 0x7fa430150020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:19:01.781 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:01.776+0000 7fa43569a640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fa4301608f0 0x7fa4300b3d20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:19:01.781 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:01.776+0000 7fa43569a640 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa4301578d0 con 0x7fa4300d1840 2023-11-26T07:19:01.781 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:01.776+0000 7fa43569a640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa430156c50 con 0x7fa4300d1c20 2023-11-26T07:19:01.782 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:01.776+0000 7fa43569a640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa430156f30 con 0x7fa4301608f0 2023-11-26T07:19:01.782 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:01.776+0000 7fa42effd640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fa4300d1840 0x7fa430156400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:19:01.782 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:01.776+0000 7fa42effd640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fa4300d1840 0x7fa430156400 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:51808/0 (socket says 172.21.15.143:51808) 2023-11-26T07:19:01.782 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:01.776+0000 7fa42effd640 1 -- 172.21.15.143:0/1424197912 learned_addr learned my addr 172.21.15.143:0/1424197912 (peer_addr_for_me v2:172.21.15.143:0/0) 2023-11-26T07:19:01.782 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:01.776+0000 7fa42f7fe640 1 --2- 172.21.15.143:0/1424197912 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fa4301608f0 0x7fa4300b3d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:19:01.782 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:01.776+0000 7fa427fff640 1 --2- 172.21.15.143:0/1424197912 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fa4300d1c20 0x7fa430150020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:19:01.783 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:01.776+0000 7fa42effd640 1 -- 172.21.15.143:0/1424197912 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fa4301608f0 msgr2=0x7fa4300b3d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:19:01.783 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:01.776+0000 7fa42effd640 1 --2- 172.21.15.143:0/1424197912 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fa4301608f0 0x7fa4300b3d20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:19:01.783 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:01.776+0000 7fa42effd640 1 -- 172.21.15.143:0/1424197912 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fa4300d1c20 msgr2=0x7fa430150020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:19:01.783 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:01.776+0000 7fa42effd640 1 --2- 172.21.15.143:0/1424197912 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fa4300d1c20 0x7fa430150020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:19:01.783 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:01.776+0000 7fa42effd640 1 -- 172.21.15.143:0/1424197912 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa428009a70 con 0x7fa4300d1840 2023-11-26T07:19:01.784 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:01.776+0000 7fa42effd640 1 --2- 172.21.15.143:0/1424197912 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fa4300d1840 0x7fa430156400 secure :-1 s=READY pgs=425 cs=0 l=1 rev1=1 crypto rx=0x7fa4200011f0 tx=0x7fa42000c8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:19:01.784 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:01.776+0000 7fa4277fe640 1 -- 172.21.15.143:0/1424197912 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa4200074e0 con 0x7fa4300d1840 2023-11-26T07:19:01.784 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:01.776+0000 7fa4277fe640 1 -- 172.21.15.143:0/1424197912 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa420007640 con 0x7fa4300d1840 2023-11-26T07:19:01.784 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:01.776+0000 7fa43569a640 1 -- 172.21.15.143:0/1424197912 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa418009420 con 0x7fa4300d1840 2023-11-26T07:19:01.784 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:01.776+0000 7fa4277fe640 1 -- 172.21.15.143:0/1424197912 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa4200049f0 con 0x7fa4300d1840 2023-11-26T07:19:01.784 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:01.776+0000 7fa43569a640 1 -- 172.21.15.143:0/1424197912 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa4301574f0 con 0x7fa4300d1840 2023-11-26T07:19:01.784 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:01.776+0000 7fa43569a640 1 -- 172.21.15.143:0/1424197912 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa430157a10 con 0x7fa4300d1840 2023-11-26T07:19:01.784 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:01.776+0000 7fa4277fe640 1 -- 172.21.15.143:0/1424197912 <== mon.0 v2:172.21.15.143:3300/0 4 ==== mgrmap(e 5) v1 ==== 85852+0+0 (secure 0 0 0) 0x7fa420004b50 con 0x7fa4300d1840 2023-11-26T07:19:01.785 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:01.780+0000 7fa4277fe640 1 --2- 172.21.15.143:0/1424197912 >> [v2:172.21.15.143:6832/2150557544,v1:172.21.15.143:6833/2150557544] conn(0x7fa410066470 0x7fa410068930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:19:01.785 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:01.780+0000 7fa4277fe640 1 -- 172.21.15.143:0/1424197912 <== mon.0 v2:172.21.15.143:3300/0 5 ==== osd_map(33..33 src has 1..33) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fa420083b80 con 0x7fa4300d1840 2023-11-26T07:19:01.785 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:01.780+0000 7fa427fff640 1 --2- 172.21.15.143:0/1424197912 >> [v2:172.21.15.143:6832/2150557544,v1:172.21.15.143:6833/2150557544] conn(0x7fa410066470 0x7fa410068930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:19:01.786 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:01.780+0000 7fa427fff640 1 --2- 172.21.15.143:0/1424197912 >> [v2:172.21.15.143:6832/2150557544,v1:172.21.15.143:6833/2150557544] conn(0x7fa410066470 0x7fa410068930 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7fa4180ad000 tx=0x7fa4180a6000 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:19:01.789 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:01.784+0000 7fa4277fe640 1 -- 172.21.15.143:0/1424197912 <== mon.0 v2:172.21.15.143:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172649 (secure 0 0 0) 0x7fa420051070 con 0x7fa4300d1840 2023-11-26T07:19:01.993 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:01.988+0000 7fa43569a640 1 -- 172.21.15.143:0/1424197912 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]} v 0) v1 -- 0x7fa430067720 con 0x7fa4300d1840 2023-11-26T07:19:01.996 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:01.988+0000 7fa4277fe640 1 -- 172.21.15.143:0/1424197912 <== mon.0 v2:172.21.15.143:3300/0 7 ==== mon_command_ack([{"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]}]=0 updated caps for client.0 v13) v1 ==== 190+0+147 (secure 0 0 0) 0x7fa420054690 con 0x7fa4300d1840 2023-11-26T07:19:01.996 INFO:teuthology.orchestra.run.smithi143.stdout:[client.0] 2023-11-26T07:19:01.996 INFO:teuthology.orchestra.run.smithi143.stdout: key = AQDm8GJlWMh9FBAAslVzi6fm4/wt7vr/rHCHHQ== 2023-11-26T07:19:01.996 INFO:teuthology.orchestra.run.smithi143.stdout: caps mds = "allow" 2023-11-26T07:19:01.996 INFO:teuthology.orchestra.run.smithi143.stdout: caps mon = "allow r" 2023-11-26T07:19:01.997 INFO:teuthology.orchestra.run.smithi143.stdout: caps osd = "allow rw tag cephfs data=cephfs" 2023-11-26T07:19:01.997 INFO:teuthology.orchestra.run.smithi143.stderr:updated caps for client.0 2023-11-26T07:19:01.998 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:01.992+0000 7fa43569a640 1 -- 172.21.15.143:0/1424197912 >> [v2:172.21.15.143:6832/2150557544,v1:172.21.15.143:6833/2150557544] conn(0x7fa410066470 msgr2=0x7fa410068930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:19:01.998 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:01.992+0000 7fa43569a640 1 --2- 172.21.15.143:0/1424197912 >> [v2:172.21.15.143:6832/2150557544,v1:172.21.15.143:6833/2150557544] conn(0x7fa410066470 0x7fa410068930 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7fa4180ad000 tx=0x7fa4180a6000 comp rx=0 tx=0).stop 2023-11-26T07:19:01.999 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:01.992+0000 7fa43569a640 1 -- 172.21.15.143:0/1424197912 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fa4300d1840 msgr2=0x7fa430156400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:19:01.999 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:01.992+0000 7fa43569a640 1 --2- 172.21.15.143:0/1424197912 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fa4300d1840 0x7fa430156400 secure :-1 s=READY pgs=425 cs=0 l=1 rev1=1 crypto rx=0x7fa4200011f0 tx=0x7fa42000c8f0 comp rx=0 tx=0).stop 2023-11-26T07:19:01.999 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:01.992+0000 7fa43569a640 1 -- 172.21.15.143:0/1424197912 shutdown_connections 2023-11-26T07:19:01.999 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:01.992+0000 7fa43569a640 1 --2- 172.21.15.143:0/1424197912 >> [v2:172.21.15.143:6832/2150557544,v1:172.21.15.143:6833/2150557544] conn(0x7fa410066470 0x7fa410068930 unknown :-1 s=CLOSED pgs=228 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:19:01.999 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:01.992+0000 7fa43569a640 1 --2- 172.21.15.143:0/1424197912 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fa4301608f0 0x7fa4300b3d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:19:01.999 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:01.992+0000 7fa43569a640 1 --2- 172.21.15.143:0/1424197912 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fa4300d1c20 0x7fa430150020 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:19:01.999 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:01.992+0000 7fa43569a640 1 --2- 172.21.15.143:0/1424197912 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fa4300d1840 0x7fa430156400 unknown :-1 s=CLOSED pgs=425 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:19:01.999 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:01.992+0000 7fa43569a640 1 -- 172.21.15.143:0/1424197912 >> 172.21.15.143:0/1424197912 conn(0x7fa43009c570 msgr2=0x7fa4300cec30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:19:01.999 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:01.992+0000 7fa43569a640 1 -- 172.21.15.143:0/1424197912 shutdown_connections 2023-11-26T07:19:01.999 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:01.992+0000 7fa43569a640 1 -- 172.21.15.143:0/1424197912 wait complete. 2023-11-26T07:19:02.017 DEBUG:teuthology.orchestra.run.smithi143:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-11-26T07:19:02.154 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:02.148+0000 7f625df65640 1 Processor -- start 2023-11-26T07:19:02.154 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:02.148+0000 7f625df65640 1 -- start start 2023-11-26T07:19:02.155 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:02.148+0000 7f625df65640 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7f625815d6e0 0x7f62580d0df0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:19:02.155 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:02.148+0000 7f625df65640 1 -- --> v1:172.21.15.143:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f62580cf0e0 con 0x7f62580b2160 2023-11-26T07:19:02.155 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:02.148+0000 7f625df65640 1 -- --> v1:172.21.15.188:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f62580ceb20 con 0x7f62580d1bc0 2023-11-26T07:19:02.156 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:02.148+0000 7f625df65640 1 -- --> v2:172.21.15.188:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f625815dac0 con 0x7f625815d6e0 2023-11-26T07:19:02.156 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:02.148+0000 7f624ffff640 1 --1- >> v1:172.21.15.143:6789/0 conn(0x7f62580b2160 0x7f62580d14b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.143:6789/0 says I am v1:172.21.15.143:35306/0 (socket says 172.21.15.143:35306) 2023-11-26T07:19:02.156 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:02.148+0000 7f624ffff640 1 -- 172.21.15.143:0/2856563468 learned_addr learned my addr 172.21.15.143:0/2856563468 (peer_addr_for_me v1:172.21.15.143:0/0) 2023-11-26T07:19:02.156 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:02.148+0000 7f62577fe640 1 --2- 172.21.15.143:0/2856563468 >> v2:172.21.15.188:3300/0 conn(0x7f625815d6e0 0x7f62580d0df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:19:02.157 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:02.148+0000 7f6256ffd640 1 -- 172.21.15.143:0/2856563468 <== mon.0 v1:172.21.15.143:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1429855056 0 0) 0x7f62580cf0e0 con 0x7f62580b2160 2023-11-26T07:19:02.157 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:02.148+0000 7f6256ffd640 1 -- 172.21.15.143:0/2856563468 --> v1:172.21.15.143:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f6244003530 con 0x7f62580b2160 2023-11-26T07:19:02.157 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:02.148+0000 7f62577fe640 1 -- 172.21.15.143:0/2856563468 >> v1:172.21.15.188:6789/0 conn(0x7f62580d1bc0 legacy=0x7f62580b0750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:19:02.157 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:02.148+0000 7f6256ffd640 1 -- 172.21.15.143:0/2856563468 <== mon.1 v1:172.21.15.188:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3492789175 0 0) 0x7f62580ceb20 con 0x7f62580d1bc0 2023-11-26T07:19:02.157 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:02.148+0000 7f62577fe640 1 -- 172.21.15.143:0/2856563468 >> v1:172.21.15.143:6789/0 conn(0x7f62580b2160 legacy=0x7f62580d14b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:19:02.157 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:02.148+0000 7f62577fe640 1 -- 172.21.15.143:0/2856563468 --> v2:172.21.15.188:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f62580cee00 con 0x7f625815d6e0 2023-11-26T07:19:02.159 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:02.156+0000 7f62577fe640 1 --2- 172.21.15.143:0/2856563468 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f625815d6e0 0x7f62580d0df0 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7f62480097c0 tx=0x7f624801c480 comp rx=0 tx=0).ready entity=mon.1 client_cookie=feef152908f43222 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:19:02.159 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:02.156+0000 7f6256ffd640 1 -- 172.21.15.143:0/2856563468 <== mon.1 v2:172.21.15.188:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f62480ab600 con 0x7f625815d6e0 2023-11-26T07:19:02.159 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:02.156+0000 7f6256ffd640 1 -- 172.21.15.143:0/2856563468 <== mon.1 v2:172.21.15.188:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f62480a7050 con 0x7f625815d6e0 2023-11-26T07:19:02.159 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:02.156+0000 7f6256ffd640 1 -- 172.21.15.143:0/2856563468 <== mon.1 v2:172.21.15.188:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f62480ab980 con 0x7f625815d6e0 2023-11-26T07:19:02.160 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:02.156+0000 7f625df65640 1 -- 172.21.15.143:0/2856563468 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f625815d6e0 msgr2=0x7f62580d0df0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:19:02.160 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:02.156+0000 7f625df65640 1 --2- 172.21.15.143:0/2856563468 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f625815d6e0 0x7f62580d0df0 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7f62480097c0 tx=0x7f624801c480 comp rx=0 tx=0).stop 2023-11-26T07:19:02.160 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:02.156+0000 7f625df65640 1 -- 172.21.15.143:0/2856563468 shutdown_connections 2023-11-26T07:19:02.160 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:02.156+0000 7f625df65640 1 --2- 172.21.15.143:0/2856563468 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f625815d6e0 0x7f62580d0df0 unknown :-1 s=CLOSED pgs=192 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:19:02.160 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:02.156+0000 7f625df65640 1 -- 172.21.15.143:0/2856563468 >> 172.21.15.143:0/2856563468 conn(0x7f625809c550 msgr2=0x7f62580d3220 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:19:02.161 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:02.156+0000 7f625df65640 1 -- 172.21.15.143:0/2856563468 shutdown_connections 2023-11-26T07:19:02.161 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:02.156+0000 7f625df65640 1 -- 172.21.15.143:0/2856563468 wait complete. 2023-11-26T07:19:02.162 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:02.156+0000 7f625df65640 1 Processor -- start 2023-11-26T07:19:02.162 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:02.156+0000 7f625df65640 1 -- start start 2023-11-26T07:19:02.164 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:02.156+0000 7f625df65640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f62580b2160 0x7f62581510e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:19:02.164 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:02.156+0000 7f625df65640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f62580d1bc0 0x7f625815dc00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:19:02.164 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:02.156+0000 7f625df65640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f625815d6e0 0x7f625815e140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:19:02.164 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:02.156+0000 7f625df65640 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f62581f5610 con 0x7f625815d6e0 2023-11-26T07:19:02.164 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:02.156+0000 7f625df65640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f62580c9880 con 0x7f62580d1bc0 2023-11-26T07:19:02.164 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:02.156+0000 7f625df65640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6258151620 con 0x7f62580b2160 2023-11-26T07:19:02.165 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:02.156+0000 7f62577fe640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f62580b2160 0x7f62581510e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:19:02.165 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:02.156+0000 7f6257fff640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f625815d6e0 0x7f625815e140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:19:02.165 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:02.156+0000 7f624ffff640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f62580d1bc0 0x7f625815dc00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:19:02.165 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:02.156+0000 7f6257fff640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f625815d6e0 0x7f625815e140 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:51822/0 (socket says 172.21.15.143:51822) 2023-11-26T07:19:02.165 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:02.156+0000 7f6257fff640 1 -- 172.21.15.143:0/1027454638 learned_addr learned my addr 172.21.15.143:0/1027454638 (peer_addr_for_me v2:172.21.15.143:0/0) 2023-11-26T07:19:02.165 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:02.156+0000 7f62577fe640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f62580b2160 0x7f62581510e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.188:3301/0 says I am v2:172.21.15.143:40800/0 (socket says 172.21.15.143:40800) 2023-11-26T07:19:02.165 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:02.156+0000 7f6257fff640 1 -- 172.21.15.143:0/1027454638 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f62580b2160 msgr2=0x7f62581510e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:19:02.165 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:02.156+0000 7f6257fff640 1 --2- 172.21.15.143:0/1027454638 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f62580b2160 0x7f62581510e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:19:02.165 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:02.156+0000 7f6257fff640 1 -- 172.21.15.143:0/1027454638 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f62580d1bc0 msgr2=0x7f625815dc00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:19:02.165 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:02.156+0000 7f6257fff640 1 --2- 172.21.15.143:0/1027454638 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f62580d1bc0 0x7f625815dc00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:19:02.165 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:02.156+0000 7f6257fff640 1 -- 172.21.15.143:0/1027454638 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6248009420 con 0x7f625815d6e0 2023-11-26T07:19:02.166 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:02.156+0000 7f6257fff640 1 --2- 172.21.15.143:0/1027454638 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f625815d6e0 0x7f625815e140 secure :-1 s=READY pgs=427 cs=0 l=1 rev1=1 crypto rx=0x7f625000c7c0 tx=0x7f625000cc80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:19:02.166 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:02.156+0000 7f6254ff9640 1 -- 172.21.15.143:0/1027454638 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6250003db0 con 0x7f625815d6e0 2023-11-26T07:19:02.166 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:02.156+0000 7f6254ff9640 1 -- 172.21.15.143:0/1027454638 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6250005070 con 0x7f625815d6e0 2023-11-26T07:19:02.166 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:02.156+0000 7f6254ff9640 1 -- 172.21.15.143:0/1027454638 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6250005420 con 0x7f625815d6e0 2023-11-26T07:19:02.166 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:02.156+0000 7f625df65640 1 -- 172.21.15.143:0/1027454638 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f62580c95a0 con 0x7f625815d6e0 2023-11-26T07:19:02.166 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:02.156+0000 7f625df65640 1 -- 172.21.15.143:0/1027454638 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f62580b7320 con 0x7f625815d6e0 2023-11-26T07:19:02.166 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:02.164+0000 7f625df65640 1 -- 172.21.15.143:0/1027454638 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f62580b6ea0 con 0x7f625815d6e0 2023-11-26T07:19:02.167 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:02.164+0000 7f6254ff9640 1 -- 172.21.15.143:0/1027454638 <== mon.0 v2:172.21.15.143:3300/0 4 ==== mgrmap(e 5) v1 ==== 85852+0+0 (secure 0 0 0) 0x7f6250005580 con 0x7f625815d6e0 2023-11-26T07:19:02.167 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:02.164+0000 7f6254ff9640 1 --2- 172.21.15.143:0/1027454638 >> [v2:172.21.15.143:6832/2150557544,v1:172.21.15.143:6833/2150557544] conn(0x7f6238066470 0x7f6238068930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:19:02.167 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:02.164+0000 7f6254ff9640 1 -- 172.21.15.143:0/1027454638 <== mon.0 v2:172.21.15.143:3300/0 5 ==== osd_map(33..33 src has 1..33) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f6250083740 con 0x7f625815d6e0 2023-11-26T07:19:02.167 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:02.164+0000 7f62577fe640 1 --2- 172.21.15.143:0/1027454638 >> [v2:172.21.15.143:6832/2150557544,v1:172.21.15.143:6833/2150557544] conn(0x7f6238066470 0x7f6238068930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:19:02.168 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:02.164+0000 7f62577fe640 1 --2- 172.21.15.143:0/1027454638 >> [v2:172.21.15.143:6832/2150557544,v1:172.21.15.143:6833/2150557544] conn(0x7f6238066470 0x7f6238068930 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7f62480ad000 tx=0x7f62480a6000 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:19:02.171 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:02.168+0000 7f6254ff9640 1 -- 172.21.15.143:0/1027454638 <== mon.0 v2:172.21.15.143:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172649 (secure 0 0 0) 0x7f62580b6ea0 con 0x7f625815d6e0 2023-11-26T07:19:02.355 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:02.352+0000 7f625df65640 1 -- 172.21.15.143:0/1027454638 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f62580b6ea0 con 0x7f625815d6e0 2023-11-26T07:19:02.356 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:02.352+0000 7f6254ff9640 1 -- 172.21.15.143:0/1027454638 <== mon.0 v2:172.21.15.143:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 16 v16) v1 ==== 94+0+4925 (secure 0 0 0) 0x7f62580b6ea0 con 0x7f625815d6e0 2023-11-26T07:19:02.356 INFO:teuthology.orchestra.run.smithi143.stdout: 2023-11-26T07:19:02.356 INFO:teuthology.orchestra.run.smithi143.stdout:{"epoch":16,"default_fscid":2,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4847,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.188:6834/1547410397","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6832","nonce":1547410397},{"type":"v1","addr":"172.21.15.188:6834","nonce":1547410397}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":12},{"gid":4850,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.188:6835/3247640329","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6833","nonce":3247640329},{"type":"v1","addr":"172.21.15.188:6835","nonce":3247640329}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":12},{"gid":5217,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.143:6835/2123707650","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6834","nonce":2123707650},{"type":"v1","addr":"172.21.15.143:6835","nonce":2123707650}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":11}],"filesystems":[{"mdsmap":{"epoch":16,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-11-26T07:18:52.014584+0000","modified":"2023-11-26T07:18:55.043646+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":5220},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5220":{"gid":5220,"name":"c","rank":0,"incarnation":13,"state":"up:active","state_seq":3,"addr":"172.21.15.143:6837/2658667363","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6836","nonce":2658667363},{"type":"v1","addr":"172.21.15.143:6837","nonce":2658667363}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[5],"metadata_pool":4,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":2}]} 2023-11-26T07:19:02.357 INFO:teuthology.orchestra.run.smithi143.stderr:dumped fsmap epoch 16 2023-11-26T07:19:02.358 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:02.356+0000 7f625df65640 1 -- 172.21.15.143:0/1027454638 >> [v2:172.21.15.143:6832/2150557544,v1:172.21.15.143:6833/2150557544] conn(0x7f6238066470 msgr2=0x7f6238068930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:19:02.358 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:02.356+0000 7f625df65640 1 --2- 172.21.15.143:0/1027454638 >> [v2:172.21.15.143:6832/2150557544,v1:172.21.15.143:6833/2150557544] conn(0x7f6238066470 0x7f6238068930 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7f62480ad000 tx=0x7f62480a6000 comp rx=0 tx=0).stop 2023-11-26T07:19:02.358 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:02.356+0000 7f625df65640 1 -- 172.21.15.143:0/1027454638 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f625815d6e0 msgr2=0x7f625815e140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:19:02.358 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:02.356+0000 7f625df65640 1 --2- 172.21.15.143:0/1027454638 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f625815d6e0 0x7f625815e140 secure :-1 s=READY pgs=427 cs=0 l=1 rev1=1 crypto rx=0x7f625000c7c0 tx=0x7f625000cc80 comp rx=0 tx=0).stop 2023-11-26T07:19:02.359 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:02.356+0000 7f625df65640 1 -- 172.21.15.143:0/1027454638 shutdown_connections 2023-11-26T07:19:02.359 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:02.356+0000 7f625df65640 1 --2- 172.21.15.143:0/1027454638 >> [v2:172.21.15.143:6832/2150557544,v1:172.21.15.143:6833/2150557544] conn(0x7f6238066470 0x7f6238068930 unknown :-1 s=CLOSED pgs=229 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:19:02.359 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:02.356+0000 7f625df65640 1 --2- 172.21.15.143:0/1027454638 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f625815d6e0 0x7f625815e140 unknown :-1 s=CLOSED pgs=427 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:19:02.359 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:02.356+0000 7f625df65640 1 --2- 172.21.15.143:0/1027454638 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f62580d1bc0 0x7f625815dc00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:19:02.359 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:02.356+0000 7f625df65640 1 --2- 172.21.15.143:0/1027454638 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f62580b2160 0x7f62581510e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:19:02.359 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:02.356+0000 7f625df65640 1 -- 172.21.15.143:0/1027454638 >> 172.21.15.143:0/1027454638 conn(0x7f625809c550 msgr2=0x7f62580b2630 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:19:02.359 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:02.356+0000 7f625df65640 1 -- 172.21.15.143:0/1027454638 shutdown_connections 2023-11-26T07:19:02.359 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:02.356+0000 7f625df65640 1 -- 172.21.15.143:0/1027454638 wait complete. 2023-11-26T07:19:02.779 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 16, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2023-11-26T07:18:52.014584+0000', 'modified': '2023-11-26T07:18:55.043646+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 5220}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_5220': {'gid': 5220, 'name': 'c', 'rank': 0, 'incarnation': 13, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.143:6837/2658667363', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.143:6836', 'nonce': 2658667363}, {'type': 'v1', 'addr': '172.21.15.143:6837', 'nonce': 2658667363}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138322906710015, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}}}, 'data_pools': [5], 'metadata_pool': 4, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0} 2023-11-26T07:19:02.779 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2023-11-26T07:19:02.780 DEBUG:teuthology.orchestra.run.smithi143:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.c status 2023-11-26T07:19:02.919 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:02.916+0000 7feb301e7640 1 Processor -- start 2023-11-26T07:19:02.919 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:02.916+0000 7feb301e7640 1 -- start start 2023-11-26T07:19:02.921 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:02.920+0000 7feb301e7640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7feb280cf280 0x7feb280cf6e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:19:02.921 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:02.920+0000 7feb301e7640 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7feb280cd1b0 0x7feb280cd5e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:19:02.921 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:02.920+0000 7feb301e7640 1 -- --> v1:172.21.15.143:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7feb280cfc20 con 0x7feb28143ac0 2023-11-26T07:19:02.921 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:02.920+0000 7feb301e7640 1 -- --> v2:172.21.15.188:3300/0 -- mon_getmap magic: 0 v1 -- 0x7feb28143ea0 con 0x7feb280cd1b0 2023-11-26T07:19:02.921 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:02.920+0000 7feb301e7640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7feb280cc730 con 0x7feb280cf280 2023-11-26T07:19:02.921 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:02.920+0000 7feb2df83640 1 --1- >> v1:172.21.15.143:6789/0 conn(0x7feb28143ac0 0x7feb28070370 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.143:6789/0 says I am v1:172.21.15.143:35318/0 (socket says 172.21.15.143:35318) 2023-11-26T07:19:02.922 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:02.920+0000 7feb2df83640 1 -- 172.21.15.143:0/4272993098 learned_addr learned my addr 172.21.15.143:0/4272993098 (peer_addr_for_me v1:172.21.15.143:0/0) 2023-11-26T07:19:02.922 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:02.920+0000 7feb2e784640 1 --2- 172.21.15.143:0/4272993098 >> v2:172.21.15.188:3300/0 conn(0x7feb280cd1b0 0x7feb280cd5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:19:02.922 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:02.920+0000 7feb2d782640 1 --2- 172.21.15.143:0/4272993098 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7feb280cf280 0x7feb280cf6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:19:02.922 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:02.920+0000 7feb2cf81640 1 -- 172.21.15.143:0/4272993098 <== mon.0 v1:172.21.15.143:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3765652060 0 0) 0x7feb280cfc20 con 0x7feb28143ac0 2023-11-26T07:19:02.922 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:02.920+0000 7feb2cf81640 1 -- 172.21.15.143:0/4272993098 --> v1:172.21.15.143:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7feb14003530 con 0x7feb28143ac0 2023-11-26T07:19:02.923 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:02.920+0000 7feb2cf81640 1 -- 172.21.15.143:0/4272993098 <== mon.0 v1:172.21.15.143:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3080493164 0 0) 0x7feb14003530 con 0x7feb28143ac0 2023-11-26T07:19:02.924 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:02.920+0000 7feb2cf81640 1 -- 172.21.15.143:0/4272993098 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7feb280cf280 msgr2=0x7feb280cf6e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-26T07:19:02.924 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:02.920+0000 7feb2cf81640 1 --2- 172.21.15.143:0/4272993098 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7feb280cf280 0x7feb280cf6e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:19:02.924 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:02.920+0000 7feb2cf81640 1 -- 172.21.15.143:0/4272993098 >> v2:172.21.15.188:3300/0 conn(0x7feb280cd1b0 msgr2=0x7feb280cd5e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-26T07:19:02.924 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:02.920+0000 7feb2cf81640 1 --2- 172.21.15.143:0/4272993098 >> v2:172.21.15.188:3300/0 conn(0x7feb280cd1b0 0x7feb280cd5e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:19:02.924 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:02.920+0000 7feb2cf81640 1 -- 172.21.15.143:0/4272993098 --> v1:172.21.15.143:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7feb28070ee0 con 0x7feb28143ac0 2023-11-26T07:19:02.924 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:02.920+0000 7feb2cf81640 1 -- 172.21.15.143:0/4272993098 <== mon.0 v1:172.21.15.143:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2137882523 0 0) 0x7feb24001680 con 0x7feb28143ac0 2023-11-26T07:19:02.924 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:02.920+0000 7feb2cf81640 1 -- 172.21.15.143:0/4272993098 >> v1:172.21.15.143:6789/0 conn(0x7feb28143ac0 legacy=0x7feb28070370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:19:02.924 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:02.920+0000 7feb2cf81640 1 --2- 172.21.15.143:0/4272993098 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7feb14003980 0x7feb14003da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:19:02.924 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:02.920+0000 7feb2cf81640 1 --2- 172.21.15.143:0/4272993098 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7feb280cf280 0x7feb140047d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:19:02.924 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:02.920+0000 7feb2cf81640 1 --2- 172.21.15.143:0/4272993098 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7feb14004d10 0x7feb14005190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:19:02.925 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:02.920+0000 7feb2cf81640 1 -- 172.21.15.143:0/4272993098 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x7feb280cc730 con 0x7feb14004d10 2023-11-26T07:19:02.925 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:02.920+0000 7feb2cf81640 1 -- 172.21.15.143:0/4272993098 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7feb1400b990 con 0x7feb14003980 2023-11-26T07:19:02.925 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:02.920+0000 7feb2cf81640 1 -- 172.21.15.143:0/4272993098 --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7feb1400bad0 con 0x7feb280cf280 2023-11-26T07:19:02.925 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:02.920+0000 7feb2e784640 1 --2- 172.21.15.143:0/4272993098 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7feb14004d10 0x7feb14005190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:19:02.925 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:02.920+0000 7feb2d782640 1 --2- 172.21.15.143:0/4272993098 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7feb280cf280 0x7feb140047d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:19:02.925 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:02.920+0000 7feb2df83640 1 --2- 172.21.15.143:0/4272993098 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7feb14003980 0x7feb14003da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:19:02.925 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:02.920+0000 7feb2e784640 1 -- 172.21.15.143:0/4272993098 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7feb280cf280 msgr2=0x7feb140047d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-26T07:19:02.925 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:02.920+0000 7feb2e784640 1 --2- 172.21.15.143:0/4272993098 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7feb280cf280 0x7feb140047d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:19:02.925 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:02.920+0000 7feb2e784640 1 -- 172.21.15.143:0/4272993098 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7feb14003980 msgr2=0x7feb14003da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-26T07:19:02.925 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:02.920+0000 7feb2e784640 1 --2- 172.21.15.143:0/4272993098 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7feb14003980 0x7feb14003da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:19:02.925 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:02.920+0000 7feb2e784640 1 -- 172.21.15.143:0/4272993098 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7feb14003530 con 0x7feb14004d10 2023-11-26T07:19:02.926 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:02.924+0000 7feb2e784640 1 --2- 172.21.15.143:0/4272993098 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7feb14004d10 0x7feb14005190 secure :-1 s=READY pgs=429 cs=0 l=1 rev1=1 crypto rx=0x7feb20009bd0 tx=0x7feb2001ee90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=55b56b941c7f334b server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:19:02.927 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:02.924+0000 7feb2cf81640 1 -- 172.21.15.143:0/4272993098 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feb200a7ea0 con 0x7feb14004d10 2023-11-26T07:19:02.927 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:02.924+0000 7feb2cf81640 1 -- 172.21.15.143:0/4272993098 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7feb2000f040 con 0x7feb14004d10 2023-11-26T07:19:02.927 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:02.924+0000 7feb2cf81640 1 -- 172.21.15.143:0/4272993098 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feb200af680 con 0x7feb14004d10 2023-11-26T07:19:02.927 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:02.924+0000 7feb301e7640 1 -- 172.21.15.143:0/4272993098 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7feb14004d10 msgr2=0x7feb14005190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:19:02.927 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:02.924+0000 7feb301e7640 1 --2- 172.21.15.143:0/4272993098 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7feb14004d10 0x7feb14005190 secure :-1 s=READY pgs=429 cs=0 l=1 rev1=1 crypto rx=0x7feb20009bd0 tx=0x7feb2001ee90 comp rx=0 tx=0).stop 2023-11-26T07:19:02.927 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:02.924+0000 7feb2e784640 1 -- 172.21.15.143:0/4272993098 reap_dead start 2023-11-26T07:19:02.927 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:02.924+0000 7feb301e7640 1 -- 172.21.15.143:0/4272993098 shutdown_connections 2023-11-26T07:19:02.927 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:02.924+0000 7feb301e7640 1 -- 172.21.15.143:0/4272993098 >> 172.21.15.143:0/4272993098 conn(0x7feb2806a890 msgr2=0x7feb2806aca0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:19:02.927 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:02.924+0000 7feb301e7640 1 -- 172.21.15.143:0/4272993098 shutdown_connections 2023-11-26T07:19:02.928 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:02.924+0000 7feb301e7640 1 -- 172.21.15.143:0/4272993098 wait complete. 2023-11-26T07:19:02.928 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:02.924+0000 7feb301e7640 1 Processor -- start 2023-11-26T07:19:02.928 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:02.924+0000 7feb301e7640 1 -- start start 2023-11-26T07:19:02.930 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:02.928+0000 7feb301e7640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7feb14004d10 0x7feb2813aa60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:19:02.930 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:02.928+0000 7feb301e7640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7feb281596c0 0x7feb28139790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:19:02.930 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:02.928+0000 7feb301e7640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7feb2815ba70 0x7feb2815bef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:19:02.931 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:02.928+0000 7feb301e7640 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x7feb2815c430 con 0x7feb281596c0 2023-11-26T07:19:02.931 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:02.928+0000 7feb301e7640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7feb2813afa0 con 0x7feb2815ba70 2023-11-26T07:19:02.931 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:02.928+0000 7feb301e7640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7feb2813b1d0 con 0x7feb14004d10 2023-11-26T07:19:02.931 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:02.928+0000 7feb2d782640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7feb281596c0 0x7feb28139790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:19:02.931 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:02.928+0000 7feb2d782640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7feb281596c0 0x7feb28139790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:51846/0 (socket says 172.21.15.143:51846) 2023-11-26T07:19:02.931 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:02.928+0000 7feb2d782640 1 -- 172.21.15.143:0/1087916543 learned_addr learned my addr 172.21.15.143:0/1087916543 (peer_addr_for_me v2:172.21.15.143:0/0) 2023-11-26T07:19:02.931 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:02.928+0000 7feb2df83640 1 --2- 172.21.15.143:0/1087916543 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7feb14004d10 0x7feb2813aa60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:19:02.931 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:02.928+0000 7feb2e784640 1 --2- 172.21.15.143:0/1087916543 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7feb2815ba70 0x7feb2815bef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:19:02.931 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:02.928+0000 7feb2d782640 1 -- 172.21.15.143:0/1087916543 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7feb14004d10 msgr2=0x7feb2813aa60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:19:02.931 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:02.928+0000 7feb2d782640 1 --2- 172.21.15.143:0/1087916543 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7feb14004d10 0x7feb2813aa60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:19:02.931 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:02.928+0000 7feb2d782640 1 -- 172.21.15.143:0/1087916543 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7feb2815ba70 msgr2=0x7feb2815bef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:19:02.932 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:02.928+0000 7feb2d782640 1 --2- 172.21.15.143:0/1087916543 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7feb2815ba70 0x7feb2815bef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:19:02.932 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:02.928+0000 7feb2d782640 1 -- 172.21.15.143:0/1087916543 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7feb280afbc0 con 0x7feb281596c0 2023-11-26T07:19:02.932 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:02.928+0000 7feb2e784640 1 --2- 172.21.15.143:0/1087916543 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7feb2815ba70 0x7feb2815bef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-26T07:19:02.932 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:02.928+0000 7feb2d782640 1 --2- 172.21.15.143:0/1087916543 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7feb281596c0 0x7feb28139790 secure :-1 s=READY pgs=430 cs=0 l=1 rev1=1 crypto rx=0x7feb180119d0 tx=0x7feb18011e90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:19:02.932 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:02.928+0000 7feb1e7fc640 1 -- 172.21.15.143:0/1087916543 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feb1800fb50 con 0x7feb281596c0 2023-11-26T07:19:02.932 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:02.928+0000 7feb1e7fc640 1 -- 172.21.15.143:0/1087916543 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7feb1800fcb0 con 0x7feb281596c0 2023-11-26T07:19:02.932 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:02.928+0000 7feb1e7fc640 1 -- 172.21.15.143:0/1087916543 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feb18014440 con 0x7feb281596c0 2023-11-26T07:19:02.932 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:02.928+0000 7feb301e7640 1 -- 172.21.15.143:0/1087916543 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7feb280afea0 con 0x7feb281596c0 2023-11-26T07:19:02.932 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:02.928+0000 7feb301e7640 1 -- 172.21.15.143:0/1087916543 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7feb280b0460 con 0x7feb281596c0 2023-11-26T07:19:02.933 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:02.928+0000 7feb1e7fc640 1 -- 172.21.15.143:0/1087916543 <== mon.0 v2:172.21.15.143:3300/0 4 ==== mgrmap(e 5) v1 ==== 85852+0+0 (secure 0 0 0) 0x7feb180145a0 con 0x7feb281596c0 2023-11-26T07:19:02.933 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:02.932+0000 7feb1e7fc640 1 --2- 172.21.15.143:0/1087916543 >> [v2:172.21.15.143:6832/2150557544,v1:172.21.15.143:6833/2150557544] conn(0x7feb0c066470 0x7feb0c068930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:19:02.933 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:02.932+0000 7feb1e7fc640 1 -- 172.21.15.143:0/1087916543 <== mon.0 v2:172.21.15.143:3300/0 5 ==== osd_map(33..33 src has 1..33) v4 ==== 5709+0+0 (secure 0 0 0) 0x7feb180847e0 con 0x7feb281596c0 2023-11-26T07:19:02.933 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:02.932+0000 7feb2df83640 1 --2- 172.21.15.143:0/1087916543 >> [v2:172.21.15.143:6832/2150557544,v1:172.21.15.143:6833/2150557544] conn(0x7feb0c066470 0x7feb0c068930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:19:02.934 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:02.932+0000 7feb33556000 1 Processor -- start 2023-11-26T07:19:02.934 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:02.932+0000 7feb33556000 1 -- start start 2023-11-26T07:19:02.934 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:02.932+0000 7feb2df83640 1 --2- 172.21.15.143:0/1087916543 >> [v2:172.21.15.143:6832/2150557544,v1:172.21.15.143:6833/2150557544] conn(0x7feb0c066470 0x7feb0c068930 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7feb24000c10 tx=0x7feb24009040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:19:02.935 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:02.932+0000 7feb33556000 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x557bd6353c90 0x557bd6354070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:19:02.936 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:02.932+0000 7feb33556000 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x557bd63545b0 0x557bd6309de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:19:02.936 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:02.932+0000 7feb33556000 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x557bd630a320 0x557bd630a700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:19:02.936 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:02.932+0000 7feb33556000 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x557bd6174130 con 0x557bd630a320 2023-11-26T07:19:02.936 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:02.932+0000 7feb33556000 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x557bd61619a0 con 0x557bd6353c90 2023-11-26T07:19:02.936 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:02.932+0000 7feb33556000 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x557bd615eed0 con 0x557bd63545b0 2023-11-26T07:19:02.936 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:02.932+0000 7feb2e784640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x557bd630a320 0x557bd630a700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:19:02.936 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:02.932+0000 7feb2e784640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x557bd630a320 0x557bd630a700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:51856/0 (socket says 172.21.15.143:51856) 2023-11-26T07:19:02.936 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:02.932+0000 7feb2e784640 1 -- 172.21.15.143:0/2747021996 learned_addr learned my addr 172.21.15.143:0/2747021996 (peer_addr_for_me v2:172.21.15.143:0/0) 2023-11-26T07:19:02.936 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:02.932+0000 7feb2df83640 1 --2- 172.21.15.143:0/2747021996 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x557bd6353c90 0x557bd6354070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:19:02.936 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:02.932+0000 7feb2d782640 1 --2- 172.21.15.143:0/2747021996 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x557bd63545b0 0x557bd6309de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:19:02.936 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:02.932+0000 7feb2e784640 1 -- 172.21.15.143:0/2747021996 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x557bd63545b0 msgr2=0x557bd6309de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-26T07:19:02.937 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:02.932+0000 7feb2e784640 1 --2- 172.21.15.143:0/2747021996 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x557bd63545b0 0x557bd6309de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:19:02.937 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:02.932+0000 7feb2e784640 1 -- 172.21.15.143:0/2747021996 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x557bd6353c90 msgr2=0x557bd6354070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-26T07:19:02.937 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:02.932+0000 7feb2e784640 1 --2- 172.21.15.143:0/2747021996 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x557bd6353c90 0x557bd6354070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:19:02.937 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:02.932+0000 7feb2e784640 1 -- 172.21.15.143:0/2747021996 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x557bd5e52890 con 0x557bd630a320 2023-11-26T07:19:02.937 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:02.932+0000 7feb2df83640 1 --2- 172.21.15.143:0/2747021996 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x557bd6353c90 0x557bd6354070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-26T07:19:02.938 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:02.932+0000 7feb2e784640 1 --2- 172.21.15.143:0/2747021996 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x557bd630a320 0x557bd630a700 secure :-1 s=READY pgs=431 cs=0 l=1 rev1=1 crypto rx=0x7feb20009f90 tx=0x7feb2001d6d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=73a20fb55ae429d2 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:19:02.938 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:02.932+0000 7feb0b7fe640 1 -- 172.21.15.143:0/2747021996 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feb200039a0 con 0x557bd630a320 2023-11-26T07:19:02.938 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:02.932+0000 7feb0b7fe640 1 -- 172.21.15.143:0/2747021996 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7feb20003b00 con 0x557bd630a320 2023-11-26T07:19:02.938 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:02.932+0000 7feb0b7fe640 1 -- 172.21.15.143:0/2747021996 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feb20003e70 con 0x557bd630a320 2023-11-26T07:19:02.939 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:02.936+0000 7feb33556000 1 -- 172.21.15.143:0/2747021996 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x557bd630a320 msgr2=0x557bd630a700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:19:02.939 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:02.936+0000 7feb33556000 1 --2- 172.21.15.143:0/2747021996 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x557bd630a320 0x557bd630a700 secure :-1 s=READY pgs=431 cs=0 l=1 rev1=1 crypto rx=0x7feb20009f90 tx=0x7feb2001d6d0 comp rx=0 tx=0).stop 2023-11-26T07:19:02.939 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:02.936+0000 7feb33556000 1 -- 172.21.15.143:0/2747021996 shutdown_connections 2023-11-26T07:19:02.939 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:02.936+0000 7feb33556000 1 --2- 172.21.15.143:0/2747021996 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x557bd630a320 0x557bd630a700 unknown :-1 s=CLOSED pgs=431 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:19:02.939 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:02.936+0000 7feb33556000 1 --2- 172.21.15.143:0/2747021996 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x557bd63545b0 0x557bd6309de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:19:02.939 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:02.936+0000 7feb33556000 1 --2- 172.21.15.143:0/2747021996 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x557bd6353c90 0x557bd6354070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:19:02.939 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:02.936+0000 7feb33556000 1 -- 172.21.15.143:0/2747021996 >> 172.21.15.143:0/2747021996 conn(0x557bd63062d0 msgr2=0x557bd63086c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:19:02.939 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:02.936+0000 7feb33556000 1 -- 172.21.15.143:0/2747021996 shutdown_connections 2023-11-26T07:19:02.939 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:02.936+0000 7feb33556000 1 -- 172.21.15.143:0/2747021996 wait complete. 2023-11-26T07:19:02.939 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:02.936+0000 7feb33556000 1 -- start start 2023-11-26T07:19:02.939 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:02.936+0000 7feb33556000 1 Processor -- start 2023-11-26T07:19:02.942 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:02.936+0000 7feb33556000 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x557bd630a320 0x557bd630ed40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:19:02.942 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:02.936+0000 7feb33556000 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x557bd6353c90 0x557bd630f280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:19:02.942 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:02.936+0000 7feb33556000 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x557bd63545b0 0x557bd6322910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:19:02.943 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:02.936+0000 7feb33556000 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x557bd615a7c0 con 0x557bd6353c90 2023-11-26T07:19:02.943 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:02.936+0000 7feb33556000 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x557bd6112bf0 con 0x557bd63545b0 2023-11-26T07:19:02.943 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:02.936+0000 7feb33556000 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x557bd6118a90 con 0x557bd630a320 2023-11-26T07:19:02.943 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:02.936+0000 7feb33556000 10 client.0 _ll_register_callbacks cb 0x557bd62f53c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-11-26T07:19:02.943 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:02.936+0000 7feb2d782640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x557bd6353c90 0x557bd630f280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:19:02.943 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:02.936+0000 7feb2d782640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x557bd6353c90 0x557bd630f280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:51858/0 (socket says 172.21.15.143:51858) 2023-11-26T07:19:02.943 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:02.936+0000 7feb2d782640 1 -- 172.21.15.143:0/2785628638 learned_addr learned my addr 172.21.15.143:0/2785628638 (peer_addr_for_me v2:172.21.15.143:0/0) 2023-11-26T07:19:02.943 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:02.936+0000 7feb2d782640 1 -- 172.21.15.143:0/2785628638 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x557bd630a320 msgr2=0x557bd630ed40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-26T07:19:02.943 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:02.936+0000 7feb2e784640 1 --2- 172.21.15.143:0/2785628638 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x557bd63545b0 0x557bd6322910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:19:02.943 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:02.936+0000 7feb2df83640 1 --2- 172.21.15.143:0/2785628638 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x557bd630a320 0x557bd630ed40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:19:02.943 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:02.936+0000 7feb2d782640 1 --2- 172.21.15.143:0/2785628638 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x557bd630a320 0x557bd630ed40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:19:02.943 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:02.936+0000 7feb2d782640 1 -- 172.21.15.143:0/2785628638 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x557bd63545b0 msgr2=0x557bd6322910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-26T07:19:02.944 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:02.936+0000 7feb2d782640 1 --2- 172.21.15.143:0/2785628638 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x557bd63545b0 0x557bd6322910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:19:02.944 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:02.936+0000 7feb2d782640 1 -- 172.21.15.143:0/2785628638 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7feb14003530 con 0x557bd6353c90 2023-11-26T07:19:02.944 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:02.936+0000 7feb2e784640 1 --2- 172.21.15.143:0/2785628638 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x557bd63545b0 0x557bd6322910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-26T07:19:02.944 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:02.936+0000 7feb2df83640 1 --2- 172.21.15.143:0/2785628638 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x557bd630a320 0x557bd630ed40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-26T07:19:02.944 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:02.936+0000 7feb2d782640 1 --2- 172.21.15.143:0/2785628638 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x557bd6353c90 0x557bd630f280 secure :-1 s=READY pgs=432 cs=0 l=1 rev1=1 crypto rx=0x7feb18051e40 tx=0x7feb18055a80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d1ad11e4ac32811d server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:19:02.944 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:02.936+0000 7feb08ff9640 10 client.0 ms_handle_connect on v2:172.21.15.143:3300/0 2023-11-26T07:19:02.944 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:02.936+0000 7feb08ff9640 1 -- 172.21.15.143:0/2785628638 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feb18055da0 con 0x557bd6353c90 2023-11-26T07:19:02.944 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:02.936+0000 7feb08ff9640 1 -- 172.21.15.143:0/2785628638 <== mon.0 v2:172.21.15.143:3300/0 2 ==== osd_map(33..33 src has 1..33) v4 ==== 5709+0+0 (secure 0 0 0) 0x7feb1801b510 con 0x557bd6353c90 2023-11-26T07:19:02.944 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:02.936+0000 7feb33556000 1 -- 172.21.15.143:0/2785628638 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x557bd611c470 con 0x557bd6353c90 2023-11-26T07:19:02.944 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:02.936+0000 7feb08ff9640 1 -- 172.21.15.143:0/2785628638 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_get_version_reply(handle=1 version=16) v2 ==== 24+0+0 (secure 0 0 0) 0x7feb18054e30 con 0x557bd6353c90 2023-11-26T07:19:02.944 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:02.936+0000 7feb33556000 10 client.5427 fetch_fsmap learned FSMap version 16 2023-11-26T07:19:02.944 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:02.936+0000 7feb33556000 1 -- 172.21.15.143:0/2785628638 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,fsmap=16,monmap=2+}) v3 -- 0x557bd611b190 con 0x557bd6353c90 2023-11-26T07:19:02.945 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:02.940+0000 7feb08ff9640 1 -- 172.21.15.143:0/2785628638 <== mon.0 v2:172.21.15.143:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7feb18077020 con 0x557bd6353c90 2023-11-26T07:19:02.945 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:02.940+0000 7feb08ff9640 1 -- 172.21.15.143:0/2785628638 <== mon.0 v2:172.21.15.143:3300/0 5 ==== fsmap(e 16) v1 ==== 3294+0+0 (secure 0 0 0) 0x7feb1801b020 con 0x557bd6353c90 2023-11-26T07:19:02.945 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:02.940+0000 7feb33556000 10 client.5427 fetch_fsmap finished waiting for FSMap version 16 2023-11-26T07:19:02.945 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:02.940+0000 7feb33556000 10 client.5427 resolve_mds: resolved name 'c' to daemon mds.c 2023-11-26T07:19:02.945 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:02.940+0000 7feb33556000 20 client.5427 populate_metadata read hostname 'smithi143' 2023-11-26T07:19:02.945 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:02.940+0000 7feb33556000 1 --2- 172.21.15.143:0/2785628638 >> [v2:172.21.15.143:6836/2658667363,v1:172.21.15.143:6837/2658667363] conn(0x557bd6323320 0x557bd6365f50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:19:02.945 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:02.940+0000 7feb33556000 4 client.5427 mds_command: new command op to 5220 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2023-11-26T07:19:02.945 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:02.940+0000 7feb33556000 1 -- 172.21.15.143:0/2785628638 --> [v2:172.21.15.143:6836/2658667363,v1:172.21.15.143:6837/2658667363] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x557bd611fb50 con 0x557bd6323320 2023-11-26T07:19:02.945 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:02.940+0000 7feb2df83640 1 --2- 172.21.15.143:0/2785628638 >> [v2:172.21.15.143:6836/2658667363,v1:172.21.15.143:6837/2658667363] conn(0x557bd6323320 0x557bd6365f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:19:02.945 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:02.940+0000 7feb2df83640 1 --2- 172.21.15.143:0/2785628638 >> [v2:172.21.15.143:6836/2658667363,v1:172.21.15.143:6837/2658667363] conn(0x557bd6323320 0x557bd6365f50 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=9aa305981eff11bf server_cookie=935bf4272686eaf8 in_seq=0 out_seq=0 2023-11-26T07:19:02.945 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:02.940+0000 7feb08ff9640 10 client.5427 ms_handle_connect on v2:172.21.15.143:6836/2658667363 2023-11-26T07:19:02.945 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:02.940+0000 7feb08ff9640 1 -- 172.21.15.143:0/2785628638 <== mds.0 v2:172.21.15.143:6836/2658667363 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x557bd611fb50 con 0x557bd6323320 2023-11-26T07:19:02.946 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:02.940+0000 7feb08ff9640 10 client.5427 handle_command_reply: tid=0 2023-11-26T07:19:02.946 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:02.940+0000 7feb33556000 1 client.5427 shutdown 2023-11-26T07:19:02.946 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:02.940+0000 7feb33556000 1 -- 172.21.15.143:0/2785628638 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x557bd6353c90 msgr2=0x557bd630f280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:19:02.946 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:02.940+0000 7feb33556000 1 --2- 172.21.15.143:0/2785628638 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x557bd6353c90 0x557bd630f280 secure :-1 s=READY pgs=432 cs=0 l=1 rev1=1 crypto rx=0x7feb18051e40 tx=0x7feb18055a80 comp rx=0 tx=0).stop 2023-11-26T07:19:02.946 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:02.940+0000 7feb33556000 1 -- 172.21.15.143:0/2785628638 shutdown_connections 2023-11-26T07:19:02.946 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:02.940+0000 7feb33556000 1 --2- 172.21.15.143:0/2785628638 >> [v2:172.21.15.143:6836/2658667363,v1:172.21.15.143:6837/2658667363] conn(0x557bd6323320 0x557bd6365f50 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:19:02.946 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:02.940+0000 7feb33556000 1 --2- 172.21.15.143:0/2785628638 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x557bd63545b0 0x557bd6322910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:19:02.946 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:02.940+0000 7feb33556000 1 --2- 172.21.15.143:0/2785628638 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x557bd6353c90 0x557bd630f280 unknown :-1 s=CLOSED pgs=432 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:19:02.946 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:02.940+0000 7feb33556000 1 --2- 172.21.15.143:0/2785628638 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x557bd630a320 0x557bd630ed40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:19:02.946 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:02.940+0000 7feb08ff9640 0 client.5427 ms_handle_reset on v2:172.21.15.143:6836/2658667363 2023-11-26T07:19:02.946 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:02.940+0000 7feb33556000 1 -- 172.21.15.143:0/2785628638 >> 172.21.15.143:0/2785628638 conn(0x557bd63062d0 msgr2=0x557bd63086c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:19:02.947 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:02.940+0000 7feb33556000 1 -- 172.21.15.143:0/2785628638 shutdown_connections 2023-11-26T07:19:02.947 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:02.940+0000 7feb33556000 1 -- 172.21.15.143:0/2785628638 wait complete. 2023-11-26T07:19:02.947 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:02.940+0000 7feb33556000 20 client.5427 trim_cache size 0 max 16384 2023-11-26T07:19:02.958 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:02.956+0000 7feb33556000 1 Processor -- start 2023-11-26T07:19:02.958 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:02.956+0000 7feb33556000 1 -- start start 2023-11-26T07:19:02.959 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:02.956+0000 7feb33556000 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x557bd63062d0 0x557bd6351b30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:19:02.959 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:02.956+0000 7feb33556000 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x557bd630a320 0x557bd6352070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:19:02.960 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:02.956+0000 7feb33556000 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x557bd6353c90 0x557bd630a700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:19:02.960 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:02.956+0000 7feb33556000 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x557bd5e13940 con 0x557bd6353c90 2023-11-26T07:19:02.960 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:02.956+0000 7feb33556000 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x557bd619c140 con 0x557bd630a320 2023-11-26T07:19:02.960 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:02.956+0000 7feb33556000 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x557bd6156130 con 0x557bd63062d0 2023-11-26T07:19:02.960 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:02.956+0000 7feb2e784640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x557bd6353c90 0x557bd630a700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:19:02.960 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:02.956+0000 7feb2e784640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x557bd6353c90 0x557bd630a700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:51874/0 (socket says 172.21.15.143:51874) 2023-11-26T07:19:02.960 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:02.956+0000 7feb2e784640 1 -- 172.21.15.143:0/3004775859 learned_addr learned my addr 172.21.15.143:0/3004775859 (peer_addr_for_me v2:172.21.15.143:0/0) 2023-11-26T07:19:02.960 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:02.956+0000 7feb2df83640 1 --2- 172.21.15.143:0/3004775859 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x557bd63062d0 0x557bd6351b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:19:02.960 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:02.956+0000 7feb2d782640 1 --2- 172.21.15.143:0/3004775859 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x557bd630a320 0x557bd6352070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:19:02.960 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:02.956+0000 7feb2e784640 1 -- 172.21.15.143:0/3004775859 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x557bd63062d0 msgr2=0x557bd6351b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-26T07:19:02.960 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:02.956+0000 7feb2e784640 1 --2- 172.21.15.143:0/3004775859 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x557bd63062d0 0x557bd6351b30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:19:02.961 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:02.956+0000 7feb2e784640 1 -- 172.21.15.143:0/3004775859 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x557bd630a320 msgr2=0x557bd6352070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-26T07:19:02.961 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:02.956+0000 7feb2e784640 1 --2- 172.21.15.143:0/3004775859 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x557bd630a320 0x557bd6352070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:19:02.961 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:02.956+0000 7feb2e784640 1 -- 172.21.15.143:0/3004775859 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x557bd6101490 con 0x557bd6353c90 2023-11-26T07:19:02.961 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:02.956+0000 7feb2e784640 1 --2- 172.21.15.143:0/3004775859 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x557bd6353c90 0x557bd630a700 secure :-1 s=READY pgs=433 cs=0 l=1 rev1=1 crypto rx=0x7feb2001ecc0 tx=0x7feb20004a50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=efa09c6674797110 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:19:02.962 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:02.956+0000 7feb0bfff640 1 -- 172.21.15.143:0/3004775859 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feb20003a60 con 0x557bd6353c90 2023-11-26T07:19:02.962 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:02.956+0000 7feb0bfff640 1 -- 172.21.15.143:0/3004775859 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7feb20003be0 con 0x557bd6353c90 2023-11-26T07:19:02.962 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:02.956+0000 7feb0bfff640 1 -- 172.21.15.143:0/3004775859 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feb20004030 con 0x557bd6353c90 2023-11-26T07:19:02.962 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:02.956+0000 7feb33556000 1 -- 172.21.15.143:0/3004775859 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x557bd6353c90 msgr2=0x557bd630a700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:19:02.962 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:02.956+0000 7feb33556000 1 --2- 172.21.15.143:0/3004775859 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x557bd6353c90 0x557bd630a700 secure :-1 s=READY pgs=433 cs=0 l=1 rev1=1 crypto rx=0x7feb2001ecc0 tx=0x7feb20004a50 comp rx=0 tx=0).stop 2023-11-26T07:19:02.962 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:02.960+0000 7feb33556000 1 -- 172.21.15.143:0/3004775859 shutdown_connections 2023-11-26T07:19:02.963 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:02.960+0000 7feb33556000 1 --2- 172.21.15.143:0/3004775859 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x557bd6353c90 0x557bd630a700 unknown :-1 s=CLOSED pgs=433 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:19:02.963 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:02.960+0000 7feb33556000 1 --2- 172.21.15.143:0/3004775859 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x557bd630a320 0x557bd6352070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:19:02.963 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:02.960+0000 7feb33556000 1 --2- 172.21.15.143:0/3004775859 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x557bd63062d0 0x557bd6351b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:19:02.963 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:02.960+0000 7feb33556000 1 -- 172.21.15.143:0/3004775859 >> 172.21.15.143:0/3004775859 conn(0x557bd630ef10 msgr2=0x557bd63086c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:19:02.963 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:02.960+0000 7feb33556000 1 -- 172.21.15.143:0/3004775859 shutdown_connections 2023-11-26T07:19:02.963 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:02.960+0000 7feb33556000 1 -- 172.21.15.143:0/3004775859 wait complete. 2023-11-26T07:19:02.963 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:02.960+0000 7feb33556000 1 -- start start 2023-11-26T07:19:02.963 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:02.960+0000 7feb33556000 1 Processor -- start 2023-11-26T07:19:02.967 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:02.960+0000 7feb33556000 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x557bd63062d0 0x557bd6366fa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:19:02.967 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:02.960+0000 7feb33556000 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x557bd630a320 0x557bd63674e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:19:02.967 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:02.960+0000 7feb33556000 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x557bd6353c90 0x557bd6375df0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:19:02.967 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:02.960+0000 7feb33556000 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x557bd6157bc0 con 0x557bd63062d0 2023-11-26T07:19:02.967 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:02.960+0000 7feb33556000 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x557bd619c140 con 0x557bd630a320 2023-11-26T07:19:02.967 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:02.960+0000 7feb33556000 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x557bd61e1fa0 con 0x557bd6353c90 2023-11-26T07:19:02.967 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:02.960+0000 7feb33556000 10 client.0 _ll_register_callbacks cb 0x557bd61a8c00 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-11-26T07:19:02.967 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:02.960+0000 7feb2df83640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x557bd63062d0 0x557bd6366fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:19:02.967 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:02.960+0000 7feb2df83640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x557bd63062d0 0x557bd6366fa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:51882/0 (socket says 172.21.15.143:51882) 2023-11-26T07:19:02.967 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:02.960+0000 7feb2df83640 1 -- 172.21.15.143:0/1662829824 learned_addr learned my addr 172.21.15.143:0/1662829824 (peer_addr_for_me v2:172.21.15.143:0/0) 2023-11-26T07:19:02.967 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:02.960+0000 7feb2e784640 1 --2- 172.21.15.143:0/1662829824 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x557bd6353c90 0x557bd6375df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:19:02.967 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:02.960+0000 7feb2d782640 1 --2- 172.21.15.143:0/1662829824 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x557bd630a320 0x557bd63674e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:19:02.968 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:02.960+0000 7feb2df83640 1 -- 172.21.15.143:0/1662829824 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x557bd6353c90 msgr2=0x557bd6375df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-26T07:19:02.968 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:02.960+0000 7feb2df83640 1 --2- 172.21.15.143:0/1662829824 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x557bd6353c90 0x557bd6375df0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:19:02.968 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:02.960+0000 7feb2df83640 1 -- 172.21.15.143:0/1662829824 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x557bd630a320 msgr2=0x557bd63674e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-26T07:19:02.968 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:02.960+0000 7feb2df83640 1 --2- 172.21.15.143:0/1662829824 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x557bd630a320 0x557bd63674e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:19:02.968 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:02.960+0000 7feb2df83640 1 -- 172.21.15.143:0/1662829824 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x557bd5e52890 con 0x557bd63062d0 2023-11-26T07:19:02.968 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:02.960+0000 7feb2e784640 1 --2- 172.21.15.143:0/1662829824 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x557bd6353c90 0x557bd6375df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-26T07:19:02.968 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:02.960+0000 7feb2df83640 1 --2- 172.21.15.143:0/1662829824 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x557bd63062d0 0x557bd6366fa0 secure :-1 s=READY pgs=434 cs=0 l=1 rev1=1 crypto rx=0x7feb24001cf0 tx=0x7feb2815a590 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b5a72b088d5a917a server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:19:02.968 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:02.960+0000 7feb08ff9640 10 client.0 ms_handle_connect on v2:172.21.15.143:3300/0 2023-11-26T07:19:02.968 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:02.960+0000 7feb08ff9640 1 -- 172.21.15.143:0/1662829824 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feb2402b070 con 0x557bd63062d0 2023-11-26T07:19:02.968 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:02.960+0000 7feb08ff9640 1 -- 172.21.15.143:0/1662829824 <== mon.0 v2:172.21.15.143:3300/0 2 ==== osd_map(33..33 src has 1..33) v4 ==== 5709+0+0 (secure 0 0 0) 0x7feb2401a040 con 0x557bd63062d0 2023-11-26T07:19:02.968 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:02.960+0000 7feb33556000 1 -- 172.21.15.143:0/1662829824 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x557bd6159110 con 0x557bd63062d0 2023-11-26T07:19:02.968 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:02.960+0000 7feb08ff9640 1 -- 172.21.15.143:0/1662829824 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_get_version_reply(handle=1 version=16) v2 ==== 24+0+0 (secure 0 0 0) 0x7feb24015640 con 0x557bd63062d0 2023-11-26T07:19:02.968 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:02.960+0000 7feb33556000 10 client.5433 fetch_fsmap learned FSMap version 16 2023-11-26T07:19:02.969 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:02.960+0000 7feb33556000 1 -- 172.21.15.143:0/1662829824 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,fsmap=16,monmap=2+}) v3 -- 0x557bd5e52cc0 con 0x557bd63062d0 2023-11-26T07:19:02.969 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:02.960+0000 7feb08ff9640 1 -- 172.21.15.143:0/1662829824 <== mon.0 v2:172.21.15.143:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x557bd6159110 con 0x557bd63062d0 2023-11-26T07:19:02.969 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:02.964+0000 7feb08ff9640 1 -- 172.21.15.143:0/1662829824 <== mon.0 v2:172.21.15.143:3300/0 5 ==== fsmap(e 16) v1 ==== 3294+0+0 (secure 0 0 0) 0x7feb24020040 con 0x557bd63062d0 2023-11-26T07:19:02.969 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:02.964+0000 7feb33556000 10 client.5433 fetch_fsmap finished waiting for FSMap version 16 2023-11-26T07:19:02.969 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:02.964+0000 7feb33556000 10 client.5433 resolve_mds: resolved name 'c' to daemon mds.c 2023-11-26T07:19:02.969 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:02.964+0000 7feb33556000 20 client.5433 populate_metadata read hostname 'smithi143' 2023-11-26T07:19:02.969 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:02.964+0000 7feb33556000 1 --2- 172.21.15.143:0/1662829824 >> [v2:172.21.15.143:6836/2658667363,v1:172.21.15.143:6837/2658667363] conn(0x557bd63545b0 0x557bd63c1260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:19:02.969 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:02.964+0000 7feb33556000 4 client.5433 mds_command: new command op to 5220 tid=0 multi_id=0 [{"prefix": "status"}] 2023-11-26T07:19:02.969 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:02.964+0000 7feb33556000 1 -- 172.21.15.143:0/1662829824 --> [v2:172.21.15.143:6836/2658667363,v1:172.21.15.143:6837/2658667363] -- command(tid 0: {"prefix": "status"}) v1 -- 0x557bd61889a0 con 0x557bd63545b0 2023-11-26T07:19:02.969 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:02.964+0000 7feb2d782640 1 --2- 172.21.15.143:0/1662829824 >> [v2:172.21.15.143:6836/2658667363,v1:172.21.15.143:6837/2658667363] conn(0x557bd63545b0 0x557bd63c1260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:19:02.969 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:02.964+0000 7feb2d782640 1 --2- 172.21.15.143:0/1662829824 >> [v2:172.21.15.143:6836/2658667363,v1:172.21.15.143:6837/2658667363] conn(0x557bd63545b0 0x557bd63c1260 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=53d8ebd3bd6303e3 server_cookie=9ab871969b7384e6 in_seq=0 out_seq=0 2023-11-26T07:19:02.969 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:02.964+0000 7feb08ff9640 10 client.5433 ms_handle_connect on v2:172.21.15.143:6836/2658667363 2023-11-26T07:19:02.969 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:02.964+0000 7feb08ff9640 1 -- 172.21.15.143:0/1662829824 <== mds.0 v2:172.21.15.143:6836/2658667363 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+331 (crc 0 0 0) 0x557bd61889a0 con 0x557bd63545b0 2023-11-26T07:19:02.969 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:02.964+0000 7feb08ff9640 10 client.5433 handle_command_reply: tid=0 2023-11-26T07:19:02.970 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:02.964+0000 7feb33556000 1 client.5433 shutdown 2023-11-26T07:19:02.970 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:02.964+0000 7feb33556000 1 -- 172.21.15.143:0/1662829824 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x557bd63062d0 msgr2=0x557bd6366fa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:19:02.970 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:02.964+0000 7feb33556000 1 --2- 172.21.15.143:0/1662829824 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x557bd63062d0 0x557bd6366fa0 secure :-1 s=READY pgs=434 cs=0 l=1 rev1=1 crypto rx=0x7feb24001cf0 tx=0x7feb2815a590 comp rx=0 tx=0).stop 2023-11-26T07:19:02.970 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:02.964+0000 7feb33556000 1 -- 172.21.15.143:0/1662829824 shutdown_connections 2023-11-26T07:19:02.970 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:02.964+0000 7feb33556000 1 --2- 172.21.15.143:0/1662829824 >> [v2:172.21.15.143:6836/2658667363,v1:172.21.15.143:6837/2658667363] conn(0x557bd63545b0 0x557bd63c1260 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:19:02.970 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:02.964+0000 7feb33556000 1 --2- 172.21.15.143:0/1662829824 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x557bd6353c90 0x557bd6375df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:19:02.970 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:02.964+0000 7feb33556000 1 --2- 172.21.15.143:0/1662829824 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x557bd630a320 0x557bd63674e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:19:02.970 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:02.964+0000 7feb08ff9640 0 client.5433 ms_handle_reset on v2:172.21.15.143:6836/2658667363 2023-11-26T07:19:02.970 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:02.964+0000 7feb33556000 1 --2- 172.21.15.143:0/1662829824 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x557bd63062d0 0x557bd6366fa0 unknown :-1 s=CLOSED pgs=434 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:19:02.970 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:02.964+0000 7feb33556000 1 -- 172.21.15.143:0/1662829824 >> 172.21.15.143:0/1662829824 conn(0x557bd630ef10 msgr2=0x557bd6308f50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:19:02.970 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:02.964+0000 7feb33556000 1 -- 172.21.15.143:0/1662829824 shutdown_connections 2023-11-26T07:19:02.970 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:02.964+0000 7feb33556000 1 -- 172.21.15.143:0/1662829824 wait complete. 2023-11-26T07:19:02.970 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:02.964+0000 7feb33556000 20 client.5433 trim_cache size 0 max 16384 2023-11-26T07:19:02.971 INFO:teuthology.orchestra.run.smithi143.stdout:{ 2023-11-26T07:19:02.971 INFO:teuthology.orchestra.run.smithi143.stdout: "cluster_fsid": "f5033060-2469-4a07-95fe-2b76f5404bc1", 2023-11-26T07:19:02.971 INFO:teuthology.orchestra.run.smithi143.stdout: "whoami": 0, 2023-11-26T07:19:02.971 INFO:teuthology.orchestra.run.smithi143.stdout: "id": 5220, 2023-11-26T07:19:02.971 INFO:teuthology.orchestra.run.smithi143.stdout: "want_state": "up:active", 2023-11-26T07:19:02.971 INFO:teuthology.orchestra.run.smithi143.stdout: "state": "up:active", 2023-11-26T07:19:02.971 INFO:teuthology.orchestra.run.smithi143.stdout: "fs_name": "cephfs", 2023-11-26T07:19:02.971 INFO:teuthology.orchestra.run.smithi143.stdout: "rank_uptime": 10.319438462999999, 2023-11-26T07:19:02.971 INFO:teuthology.orchestra.run.smithi143.stdout: "mdsmap_epoch": 16, 2023-11-26T07:19:02.972 INFO:teuthology.orchestra.run.smithi143.stdout: "osdmap_epoch": 33, 2023-11-26T07:19:02.972 INFO:teuthology.orchestra.run.smithi143.stdout: "osdmap_epoch_barrier": 33, 2023-11-26T07:19:02.972 INFO:teuthology.orchestra.run.smithi143.stdout: "uptime": 17.331193333000002 2023-11-26T07:19:02.972 INFO:teuthology.orchestra.run.smithi143.stdout:} 2023-11-26T07:19:02.972 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:02.964+0000 7feb301e7640 1 -- 172.21.15.143:0/1087916543 >> [v2:172.21.15.143:6832/2150557544,v1:172.21.15.143:6833/2150557544] conn(0x7feb0c066470 msgr2=0x7feb0c068930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:19:02.972 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:02.964+0000 7feb301e7640 1 --2- 172.21.15.143:0/1087916543 >> [v2:172.21.15.143:6832/2150557544,v1:172.21.15.143:6833/2150557544] conn(0x7feb0c066470 0x7feb0c068930 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7feb24000c10 tx=0x7feb24009040 comp rx=0 tx=0).stop 2023-11-26T07:19:02.973 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:02.964+0000 7feb301e7640 1 -- 172.21.15.143:0/1087916543 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7feb281596c0 msgr2=0x7feb28139790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:19:02.973 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:02.964+0000 7feb301e7640 1 --2- 172.21.15.143:0/1087916543 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7feb281596c0 0x7feb28139790 secure :-1 s=READY pgs=430 cs=0 l=1 rev1=1 crypto rx=0x7feb180119d0 tx=0x7feb18011e90 comp rx=0 tx=0).stop 2023-11-26T07:19:02.973 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:02.964+0000 7feb301e7640 1 -- 172.21.15.143:0/1087916543 shutdown_connections 2023-11-26T07:19:02.973 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:02.964+0000 7feb301e7640 1 --2- 172.21.15.143:0/1087916543 >> [v2:172.21.15.143:6832/2150557544,v1:172.21.15.143:6833/2150557544] conn(0x7feb0c066470 0x7feb0c068930 unknown :-1 s=CLOSED pgs=230 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:19:02.973 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:02.964+0000 7feb301e7640 1 --2- 172.21.15.143:0/1087916543 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7feb2815ba70 0x7feb2815bef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:19:02.973 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:02.964+0000 7feb301e7640 1 --2- 172.21.15.143:0/1087916543 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7feb281596c0 0x7feb28139790 unknown :-1 s=CLOSED pgs=430 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:19:02.973 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:02.964+0000 7feb301e7640 1 --2- 172.21.15.143:0/1087916543 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7feb14004d10 0x7feb2813aa60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:19:02.973 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:02.964+0000 7feb301e7640 1 -- 172.21.15.143:0/1087916543 >> 172.21.15.143:0/1087916543 conn(0x7feb2806a890 msgr2=0x7feb280ce3d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:19:02.973 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:02.964+0000 7feb301e7640 1 -- 172.21.15.143:0/1087916543 shutdown_connections 2023-11-26T07:19:02.973 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:02.964+0000 7feb301e7640 1 -- 172.21.15.143:0/1087916543 wait complete. 2023-11-26T07:19:02.985 DEBUG:teuthology.orchestra.run.smithi143:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2023-11-26T07:19:03.123 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:03.120+0000 7f8648e5a640 1 Processor -- start 2023-11-26T07:19:03.123 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:03.120+0000 7f8648e5a640 1 -- start start 2023-11-26T07:19:03.125 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:03.120+0000 7f8648e5a640 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7f8644143ac0 0x7f8644070370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:19:03.125 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:03.120+0000 7f8648e5a640 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7f86440cf990 0x7f86440cd1b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:19:03.125 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:03.120+0000 7f8648e5a640 1 -- --> v1:172.21.15.188:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f86440cc730 con 0x7f86440708b0 2023-11-26T07:19:03.125 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:03.120+0000 7f8648e5a640 1 -- --> v2:172.21.15.143:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8644143ea0 con 0x7f86440cf990 2023-11-26T07:19:03.125 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:03.120+0000 7f8648e5a640 1 -- --> v2:172.21.15.188:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8644070ee0 con 0x7f8644143ac0 2023-11-26T07:19:03.125 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:03.124+0000 7f8642d9d640 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7f86440cf990 0x7f86440cd1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:19:03.125 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:03.124+0000 7f8642d9d640 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7f86440cf990 0x7f86440cd1b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:51894/0 (socket says 172.21.15.143:51894) 2023-11-26T07:19:03.125 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:03.124+0000 7f8642d9d640 1 -- 172.21.15.143:0/3219397300 learned_addr learned my addr 172.21.15.143:0/3219397300 (peer_addr_for_me v2:172.21.15.143:0/0) 2023-11-26T07:19:03.125 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:03.124+0000 7f864259c640 1 --2- 172.21.15.143:0/3219397300 >> v2:172.21.15.188:3300/0 conn(0x7f8644143ac0 0x7f8644070370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:19:03.126 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:03.124+0000 7f8642d9d640 1 -- 172.21.15.143:0/3219397300 >> v2:172.21.15.188:3300/0 conn(0x7f8644143ac0 msgr2=0x7f8644070370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-26T07:19:03.126 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:03.124+0000 7f8642d9d640 1 --2- 172.21.15.143:0/3219397300 >> v2:172.21.15.188:3300/0 conn(0x7f8644143ac0 0x7f8644070370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:19:03.126 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:03.124+0000 7f8642d9d640 1 -- 172.21.15.143:0/3219397300 >> v1:172.21.15.188:6789/0 conn(0x7f86440708b0 legacy=0x7f86440cf280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-26T07:19:03.126 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:03.124+0000 7f8642d9d640 1 -- 172.21.15.143:0/3219397300 --> v2:172.21.15.143:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f86440711c0 con 0x7f86440cf990 2023-11-26T07:19:03.126 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:03.124+0000 7f864259c640 1 --2- 172.21.15.143:0/3219397300 >> v2:172.21.15.188:3300/0 conn(0x7f8644143ac0 0x7f8644070370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-26T07:19:03.128 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:03.124+0000 7f8642d9d640 1 --2- 172.21.15.143:0/3219397300 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f86440cf990 0x7f86440cd1b0 secure :-1 s=READY pgs=435 cs=0 l=1 rev1=1 crypto rx=0x7f864406dca0 tx=0x7f863c01c9d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9eb23a029336bc32 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:19:03.128 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:03.124+0000 7f8641d9b640 1 -- 172.21.15.143:0/3219397300 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f863c0a57a0 con 0x7f86440cf990 2023-11-26T07:19:03.128 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:03.124+0000 7f8641d9b640 1 -- 172.21.15.143:0/3219397300 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f863c0a5900 con 0x7f86440cf990 2023-11-26T07:19:03.128 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:03.124+0000 7f8641d9b640 1 -- 172.21.15.143:0/3219397300 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f863c0a5c90 con 0x7f86440cf990 2023-11-26T07:19:03.129 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:03.128+0000 7f8648e5a640 1 -- 172.21.15.143:0/3219397300 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f86440cf990 msgr2=0x7f86440cd1b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:19:03.129 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:03.128+0000 7f8648e5a640 1 --2- 172.21.15.143:0/3219397300 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f86440cf990 0x7f86440cd1b0 secure :-1 s=READY pgs=435 cs=0 l=1 rev1=1 crypto rx=0x7f864406dca0 tx=0x7f863c01c9d0 comp rx=0 tx=0).stop 2023-11-26T07:19:03.129 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:03.128+0000 7f8648e5a640 1 -- 172.21.15.143:0/3219397300 shutdown_connections 2023-11-26T07:19:03.129 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:03.128+0000 7f8648e5a640 1 --2- 172.21.15.143:0/3219397300 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f86440cf990 0x7f86440cd1b0 unknown :-1 s=CLOSED pgs=435 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:19:03.130 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:03.128+0000 7f8648e5a640 1 --2- 172.21.15.143:0/3219397300 >> v2:172.21.15.188:3300/0 conn(0x7f8644143ac0 0x7f8644070370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:19:03.130 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:03.128+0000 7f8648e5a640 1 -- 172.21.15.143:0/3219397300 >> 172.21.15.143:0/3219397300 conn(0x7f864406a890 msgr2=0x7f864406aca0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:19:03.130 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:03.128+0000 7f8648e5a640 1 -- 172.21.15.143:0/3219397300 shutdown_connections 2023-11-26T07:19:03.130 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:03.128+0000 7f8648e5a640 1 -- 172.21.15.143:0/3219397300 wait complete. 2023-11-26T07:19:03.131 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:03.128+0000 7f8648e5a640 1 Processor -- start 2023-11-26T07:19:03.131 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:03.128+0000 7f8648e5a640 1 -- start start 2023-11-26T07:19:03.133 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:03.128+0000 7f8648e5a640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f86440708b0 0x7f864415a600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:19:03.133 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:03.128+0000 7f8648e5a640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f86440cf990 0x7f86440b1050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:19:03.133 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:03.128+0000 7f8648e5a640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f8644143ac0 0x7f86440b1590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:19:03.133 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:03.128+0000 7f8648e5a640 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f86440af3c0 con 0x7f86440cf990 2023-11-26T07:19:03.133 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:03.128+0000 7f8648e5a640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f86441410c0 con 0x7f86440708b0 2023-11-26T07:19:03.133 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:03.128+0000 7f8648e5a640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f86441413a0 con 0x7f8644143ac0 2023-11-26T07:19:03.133 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:03.128+0000 7f8639d9b640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f86440cf990 0x7f86440b1050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:19:03.133 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:03.128+0000 7f8639d9b640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f86440cf990 0x7f86440b1050 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:51900/0 (socket says 172.21.15.143:51900) 2023-11-26T07:19:03.133 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:03.128+0000 7f8639d9b640 1 -- 172.21.15.143:0/1351002156 learned_addr learned my addr 172.21.15.143:0/1351002156 (peer_addr_for_me v2:172.21.15.143:0/0) 2023-11-26T07:19:03.133 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:03.132+0000 7f864259c640 1 --2- 172.21.15.143:0/1351002156 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f86440708b0 0x7f864415a600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:19:03.133 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:03.132+0000 7f8642d9d640 1 --2- 172.21.15.143:0/1351002156 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f8644143ac0 0x7f86440b1590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:19:03.134 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:03.132+0000 7f8639d9b640 1 -- 172.21.15.143:0/1351002156 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f8644143ac0 msgr2=0x7f86440b1590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:19:03.134 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:03.132+0000 7f8639d9b640 1 --2- 172.21.15.143:0/1351002156 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f8644143ac0 0x7f86440b1590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:19:03.134 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:03.132+0000 7f8639d9b640 1 -- 172.21.15.143:0/1351002156 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f86440708b0 msgr2=0x7f864415a600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:19:03.134 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:03.132+0000 7f8639d9b640 1 --2- 172.21.15.143:0/1351002156 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f86440708b0 0x7f864415a600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:19:03.134 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:03.132+0000 7f8639d9b640 1 -- 172.21.15.143:0/1351002156 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f863c009b30 con 0x7f86440cf990 2023-11-26T07:19:03.135 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:03.132+0000 7f8639d9b640 1 --2- 172.21.15.143:0/1351002156 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f86440cf990 0x7f86440b1050 secure :-1 s=READY pgs=436 cs=0 l=1 rev1=1 crypto rx=0x7f863400c500 tx=0x7f863400c9c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:19:03.135 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:03.132+0000 7f863b7fe640 1 -- 172.21.15.143:0/1351002156 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8634005220 con 0x7f86440cf990 2023-11-26T07:19:03.135 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:03.132+0000 7f863b7fe640 1 -- 172.21.15.143:0/1351002156 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8634005380 con 0x7f86440cf990 2023-11-26T07:19:03.135 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:03.132+0000 7f863b7fe640 1 -- 172.21.15.143:0/1351002156 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8634007270 con 0x7f86440cf990 2023-11-26T07:19:03.135 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:03.132+0000 7f8648e5a640 1 -- 172.21.15.143:0/1351002156 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f86440aed00 con 0x7f86440cf990 2023-11-26T07:19:03.135 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:03.132+0000 7f8648e5a640 1 -- 172.21.15.143:0/1351002156 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f864415ab40 con 0x7f86440cf990 2023-11-26T07:19:03.135 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:03.132+0000 7f8648e5a640 1 -- 172.21.15.143:0/1351002156 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8644067680 con 0x7f86440cf990 2023-11-26T07:19:03.135 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:03.132+0000 7f863b7fe640 1 -- 172.21.15.143:0/1351002156 <== mon.0 v2:172.21.15.143:3300/0 4 ==== mgrmap(e 5) v1 ==== 85852+0+0 (secure 0 0 0) 0x7f86340073d0 con 0x7f86440cf990 2023-11-26T07:19:03.136 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:03.132+0000 7f863b7fe640 1 --2- 172.21.15.143:0/1351002156 >> [v2:172.21.15.143:6832/2150557544,v1:172.21.15.143:6833/2150557544] conn(0x7f8624066470 0x7f8624068930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:19:03.136 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:03.136+0000 7f863b7fe640 1 -- 172.21.15.143:0/1351002156 <== mon.0 v2:172.21.15.143:3300/0 5 ==== osd_map(33..33 src has 1..33) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f8634083710 con 0x7f86440cf990 2023-11-26T07:19:03.137 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:03.136+0000 7f864259c640 1 --2- 172.21.15.143:0/1351002156 >> [v2:172.21.15.143:6832/2150557544,v1:172.21.15.143:6833/2150557544] conn(0x7f8624066470 0x7f8624068930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:19:03.137 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:03.136+0000 7f864259c640 1 --2- 172.21.15.143:0/1351002156 >> [v2:172.21.15.143:6832/2150557544,v1:172.21.15.143:6833/2150557544] conn(0x7f8624066470 0x7f8624068930 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7f862c0045f0 tx=0x7f862c008040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:19:03.140 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:03.136+0000 7f863b7fe640 1 -- 172.21.15.143:0/1351002156 <== mon.0 v2:172.21.15.143:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172649 (secure 0 0 0) 0x7f8634050bb0 con 0x7f86440cf990 2023-11-26T07:19:03.321 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:03.320+0000 7f8648e5a640 1 -- 172.21.15.143:0/1351002156 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "fs ls"} v 0) v1 -- 0x7f864415a370 con 0x7f86440cf990 2023-11-26T07:19:03.322 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:03.320+0000 7f863b7fe640 1 -- 172.21.15.143:0/1351002156 <== mon.0 v2:172.21.15.143:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v16) v1 ==== 53+0+73 (secure 0 0 0) 0x7f86340541d0 con 0x7f86440cf990 2023-11-26T07:19:03.322 INFO:teuthology.orchestra.run.smithi143.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2023-11-26T07:19:03.324 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:03.320+0000 7f8648e5a640 1 -- 172.21.15.143:0/1351002156 >> [v2:172.21.15.143:6832/2150557544,v1:172.21.15.143:6833/2150557544] conn(0x7f8624066470 msgr2=0x7f8624068930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:19:03.324 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:03.320+0000 7f8648e5a640 1 --2- 172.21.15.143:0/1351002156 >> [v2:172.21.15.143:6832/2150557544,v1:172.21.15.143:6833/2150557544] conn(0x7f8624066470 0x7f8624068930 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7f862c0045f0 tx=0x7f862c008040 comp rx=0 tx=0).stop 2023-11-26T07:19:03.324 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:03.320+0000 7f8648e5a640 1 -- 172.21.15.143:0/1351002156 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f86440cf990 msgr2=0x7f86440b1050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:19:03.324 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:03.320+0000 7f8648e5a640 1 --2- 172.21.15.143:0/1351002156 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f86440cf990 0x7f86440b1050 secure :-1 s=READY pgs=436 cs=0 l=1 rev1=1 crypto rx=0x7f863400c500 tx=0x7f863400c9c0 comp rx=0 tx=0).stop 2023-11-26T07:19:03.324 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:03.320+0000 7f8648e5a640 1 -- 172.21.15.143:0/1351002156 shutdown_connections 2023-11-26T07:19:03.324 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:03.320+0000 7f8648e5a640 1 --2- 172.21.15.143:0/1351002156 >> [v2:172.21.15.143:6832/2150557544,v1:172.21.15.143:6833/2150557544] conn(0x7f8624066470 0x7f8624068930 unknown :-1 s=CLOSED pgs=231 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:19:03.324 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:03.320+0000 7f8648e5a640 1 --2- 172.21.15.143:0/1351002156 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f8644143ac0 0x7f86440b1590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:19:03.324 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:03.320+0000 7f8648e5a640 1 --2- 172.21.15.143:0/1351002156 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f86440cf990 0x7f86440b1050 unknown :-1 s=CLOSED pgs=436 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:19:03.325 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:03.320+0000 7f8648e5a640 1 --2- 172.21.15.143:0/1351002156 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f86440708b0 0x7f864415a600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:19:03.325 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:03.320+0000 7f8648e5a640 1 -- 172.21.15.143:0/1351002156 >> 172.21.15.143:0/1351002156 conn(0x7f864406a890 msgr2=0x7f86440add90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:19:03.325 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:03.320+0000 7f8648e5a640 1 -- 172.21.15.143:0/1351002156 shutdown_connections 2023-11-26T07:19:03.325 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:03.320+0000 7f8648e5a640 1 -- 172.21.15.143:0/1351002156 wait complete. 2023-11-26T07:19:03.343 INFO:tasks.cephfs.mount:Mounting default Ceph FS; just confirmed its presence on cluster 2023-11-26T07:19:03.343 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2023-11-26T07:19:03.343 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi143.front.sepia.ceph.com 2023-11-26T07:19:03.343 INFO:tasks.cephfs.mount:self.client.name = client.0 2023-11-26T07:19:03.343 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2023-11-26T07:19:03.343 INFO:tasks.cephfs.mount:self.cephfs_name = None 2023-11-26T07:19:03.344 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2023-11-26T07:19:03.344 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2023-11-26T07:19:03.344 INFO:tasks.cephfs.mount:Setting the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2023-11-26T07:19:03.344 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-26T07:19:03.344 DEBUG:teuthology.orchestra.run.smithi143:> ip addr 2023-11-26T07:19:03.353 INFO:teuthology.orchestra.run.smithi143.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2023-11-26T07:19:03.354 INFO:teuthology.orchestra.run.smithi143.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2023-11-26T07:19:03.354 INFO:teuthology.orchestra.run.smithi143.stdout: inet 127.0.0.1/8 scope host lo 2023-11-26T07:19:03.354 INFO:teuthology.orchestra.run.smithi143.stdout: valid_lft forever preferred_lft forever 2023-11-26T07:19:03.354 INFO:teuthology.orchestra.run.smithi143.stdout: inet6 ::1/128 scope host 2023-11-26T07:19:03.354 INFO:teuthology.orchestra.run.smithi143.stdout: valid_lft forever preferred_lft forever 2023-11-26T07:19:03.354 INFO:teuthology.orchestra.run.smithi143.stdout:2: eno1: mtu 1500 qdisc noop state DOWN group default qlen 1000 2023-11-26T07:19:03.354 INFO:teuthology.orchestra.run.smithi143.stdout: link/ether 0c:c4:7a:d9:3b:94 brd ff:ff:ff:ff:ff:ff 2023-11-26T07:19:03.354 INFO:teuthology.orchestra.run.smithi143.stdout: altname enp7s0f0 2023-11-26T07:19:03.354 INFO:teuthology.orchestra.run.smithi143.stdout:3: eno2: mtu 1500 qdisc noop state DOWN group default qlen 1000 2023-11-26T07:19:03.354 INFO:teuthology.orchestra.run.smithi143.stdout: link/ether 0c:c4:7a:d9:3b:95 brd ff:ff:ff:ff:ff:ff 2023-11-26T07:19:03.354 INFO:teuthology.orchestra.run.smithi143.stdout: altname enp7s0f1 2023-11-26T07:19:03.355 INFO:teuthology.orchestra.run.smithi143.stdout:4: enp3s0f0: mtu 1500 qdisc noop state DOWN group default qlen 1000 2023-11-26T07:19:03.355 INFO:teuthology.orchestra.run.smithi143.stdout: link/ether 0c:c4:7a:88:77:82 brd ff:ff:ff:ff:ff:ff 2023-11-26T07:19:03.355 INFO:teuthology.orchestra.run.smithi143.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2023-11-26T07:19:03.355 INFO:teuthology.orchestra.run.smithi143.stdout: link/ether 0c:c4:7a:88:77:83 brd ff:ff:ff:ff:ff:ff 2023-11-26T07:19:03.355 INFO:teuthology.orchestra.run.smithi143.stdout: inet 172.21.15.143/20 metric 100 brd 172.21.15.255 scope global dynamic enp3s0f1 2023-11-26T07:19:03.355 INFO:teuthology.orchestra.run.smithi143.stdout: valid_lft 42330sec preferred_lft 42330sec 2023-11-26T07:19:03.355 INFO:teuthology.orchestra.run.smithi143.stdout: inet6 fe80::ec4:7aff:fe88:7783/64 scope link 2023-11-26T07:19:03.355 INFO:teuthology.orchestra.run.smithi143.stdout: valid_lft forever preferred_lft forever 2023-11-26T07:19:03.355 INFO:teuthology.orchestra.run.smithi143.stdout:6: docker0: mtu 1500 qdisc noqueue state DOWN group default 2023-11-26T07:19:03.355 INFO:teuthology.orchestra.run.smithi143.stdout: link/ether 02:42:c4:b4:2f:eb brd ff:ff:ff:ff:ff:ff 2023-11-26T07:19:03.355 INFO:teuthology.orchestra.run.smithi143.stdout: inet 172.17.0.1/16 brd 172.17.255.255 scope global docker0 2023-11-26T07:19:03.355 INFO:teuthology.orchestra.run.smithi143.stdout: valid_lft forever preferred_lft forever 2023-11-26T07:19:03.355 INFO:teuthology.orchestra.run.smithi143.stdout:7: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2023-11-26T07:19:03.356 INFO:teuthology.orchestra.run.smithi143.stdout: link/ether 5e:1f:84:6d:79:ae brd ff:ff:ff:ff:ff:ff 2023-11-26T07:19:03.356 INFO:teuthology.orchestra.run.smithi143.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2023-11-26T07:19:03.356 INFO:teuthology.orchestra.run.smithi143.stdout: valid_lft forever preferred_lft forever 2023-11-26T07:19:03.356 INFO:teuthology.orchestra.run.smithi143.stdout: inet6 fe80::5c1f:84ff:fe6d:79ae/64 scope link 2023-11-26T07:19:03.356 INFO:teuthology.orchestra.run.smithi143.stdout: valid_lft forever preferred_lft forever 2023-11-26T07:19:03.356 INFO:teuthology.orchestra.run.smithi143.stdout:8: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2023-11-26T07:19:03.356 INFO:teuthology.orchestra.run.smithi143.stdout: link/ether 0e:5c:f6:1f:82:2f brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.0 2023-11-26T07:19:03.356 INFO:teuthology.orchestra.run.smithi143.stdout: inet6 fe80::c5c:f6ff:fe1f:822f/64 scope link 2023-11-26T07:19:03.356 INFO:teuthology.orchestra.run.smithi143.stdout: valid_lft forever preferred_lft forever 2023-11-26T07:19:03.356 INFO:teuthology.orchestra.run.smithi143.stdout:9: brx.1@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2023-11-26T07:19:03.356 INFO:teuthology.orchestra.run.smithi143.stdout: link/ether c6:a6:3d:69:d0:db brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.2 2023-11-26T07:19:03.356 INFO:teuthology.orchestra.run.smithi143.stdout: inet6 fe80::c4a6:3dff:fe69:d0db/64 scope link 2023-11-26T07:19:03.356 INFO:teuthology.orchestra.run.smithi143.stdout: valid_lft forever preferred_lft forever 2023-11-26T07:19:03.357 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-26T07:19:03.357 DEBUG:teuthology.orchestra.run.smithi143:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2023-11-26T07:19:03.415 INFO:teuthology.orchestra.run.smithi143.stdout:1 2023-11-26T07:19:03.417 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-26T07:19:03.417 DEBUG:teuthology.orchestra.run.smithi143:> route 2023-11-26T07:19:03.472 INFO:teuthology.orchestra.run.smithi143.stdout:Kernel IP routing table 2023-11-26T07:19:03.472 INFO:teuthology.orchestra.run.smithi143.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2023-11-26T07:19:03.472 INFO:teuthology.orchestra.run.smithi143.stdout:default _gateway 0.0.0.0 UG 0 0 0 enp3s0f1 2023-11-26T07:19:03.472 INFO:teuthology.orchestra.run.smithi143.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2023-11-26T07:19:03.472 INFO:teuthology.orchestra.run.smithi143.stdout:172.17.0.0 0.0.0.0 255.255.0.0 U 0 0 0 docker0 2023-11-26T07:19:03.472 INFO:teuthology.orchestra.run.smithi143.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 enp3s0f1 2023-11-26T07:19:03.472 INFO:teuthology.orchestra.run.smithi143.stdout:ns1.front.sepia 0.0.0.0 255.255.255.255 UH 100 0 0 enp3s0f1 2023-11-26T07:19:03.472 INFO:teuthology.orchestra.run.smithi143.stdout:172.21.0.2 0.0.0.0 255.255.255.255 UH 100 0 0 enp3s0f1 2023-11-26T07:19:03.472 INFO:teuthology.orchestra.run.smithi143.stdout:_gateway 0.0.0.0 255.255.255.255 UH 100 0 0 enp3s0f1 2023-11-26T07:19:03.472 INFO:teuthology.orchestra.run.smithi143.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2023-11-26T07:19:03.473 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-26T07:19:03.473 DEBUG:teuthology.orchestra.run.smithi143:> (cd / && exec bash -c ' 2023-11-26T07:19:03.473 DEBUG:teuthology.orchestra.run.smithi143:> set -e 2023-11-26T07:19:03.473 DEBUG:teuthology.orchestra.run.smithi143:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2023-11-26T07:19:03.473 DEBUG:teuthology.orchestra.run.smithi143:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2023-11-26T07:19:03.473 DEBUG:teuthology.orchestra.run.smithi143:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2023-11-26T07:19:03.473 DEBUG:teuthology.orchestra.run.smithi143:> ') 2023-11-26T07:19:03.562 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-26T07:19:03.563 DEBUG:teuthology.orchestra.run.smithi143:> ip netns list 2023-11-26T07:19:03.571 INFO:teuthology.orchestra.run.smithi143.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2023-11-26T07:19:03.571 INFO:teuthology.orchestra.run.smithi143.stdout:ceph-ns--home-ubuntu-cephtest-mnt.2 (id: 1) 2023-11-26T07:19:03.572 INFO:tasks.cephfs.mount:Resuming the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2023-11-26T07:19:03.572 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-26T07:19:03.572 DEBUG:teuthology.orchestra.run.smithi143:> sudo ip link set brx.0 up 2023-11-26T07:19:03.636 INFO:tasks.cephfs.fuse_mount:Client client.0 config is {} 2023-11-26T07:19:03.636 INFO:teuthology.orchestra.run:Running command with timeout 60 2023-11-26T07:19:03.637 DEBUG:teuthology.orchestra.run.smithi143:> mkdir -p -v /home/ubuntu/cephtest/mnt.0 2023-11-26T07:19:03.688 INFO:teuthology.orchestra.run.smithi143.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.0' 2023-11-26T07:19:03.689 INFO:teuthology.orchestra.run:Running command with timeout 60 2023-11-26T07:19:03.689 DEBUG:teuthology.orchestra.run.smithi143:> chmod 0000 /home/ubuntu/cephtest/mnt.0 2023-11-26T07:19:03.740 DEBUG:teuthology.orchestra.run.smithi143:> sudo modprobe fuse 2023-11-26T07:19:03.805 DEBUG:teuthology.orchestra.run.smithi143:> cat /proc/self/mounts | awk '{print $2}' 2023-11-26T07:19:03.858 INFO:teuthology.orchestra.run.smithi143.stdout:/sys 2023-11-26T07:19:03.858 INFO:teuthology.orchestra.run.smithi143.stdout:/proc 2023-11-26T07:19:03.858 INFO:teuthology.orchestra.run.smithi143.stdout:/dev 2023-11-26T07:19:03.858 INFO:teuthology.orchestra.run.smithi143.stdout:/dev/pts 2023-11-26T07:19:03.858 INFO:teuthology.orchestra.run.smithi143.stdout:/run 2023-11-26T07:19:03.858 INFO:teuthology.orchestra.run.smithi143.stdout:/ 2023-11-26T07:19:03.858 INFO:teuthology.orchestra.run.smithi143.stdout:/sys/kernel/security 2023-11-26T07:19:03.858 INFO:teuthology.orchestra.run.smithi143.stdout:/dev/shm 2023-11-26T07:19:03.858 INFO:teuthology.orchestra.run.smithi143.stdout:/run/lock 2023-11-26T07:19:03.859 INFO:teuthology.orchestra.run.smithi143.stdout:/sys/fs/cgroup 2023-11-26T07:19:03.859 INFO:teuthology.orchestra.run.smithi143.stdout:/sys/fs/pstore 2023-11-26T07:19:03.859 INFO:teuthology.orchestra.run.smithi143.stdout:/sys/fs/bpf 2023-11-26T07:19:03.859 INFO:teuthology.orchestra.run.smithi143.stdout:/proc/sys/fs/binfmt_misc 2023-11-26T07:19:03.859 INFO:teuthology.orchestra.run.smithi143.stdout:/dev/hugepages 2023-11-26T07:19:03.859 INFO:teuthology.orchestra.run.smithi143.stdout:/dev/mqueue 2023-11-26T07:19:03.859 INFO:teuthology.orchestra.run.smithi143.stdout:/sys/kernel/debug 2023-11-26T07:19:03.859 INFO:teuthology.orchestra.run.smithi143.stdout:/sys/kernel/tracing 2023-11-26T07:19:03.859 INFO:teuthology.orchestra.run.smithi143.stdout:/proc/fs/nfsd 2023-11-26T07:19:03.859 INFO:teuthology.orchestra.run.smithi143.stdout:/sys/fs/fuse/connections 2023-11-26T07:19:03.859 INFO:teuthology.orchestra.run.smithi143.stdout:/sys/kernel/config 2023-11-26T07:19:03.860 INFO:teuthology.orchestra.run.smithi143.stdout:/run/credentials/systemd-sysusers.service 2023-11-26T07:19:03.860 INFO:teuthology.orchestra.run.smithi143.stdout:/run/rpc_pipefs 2023-11-26T07:19:03.860 INFO:teuthology.orchestra.run.smithi143.stdout:/run/user/1000 2023-11-26T07:19:03.860 INFO:teuthology.orchestra.run.smithi143.stdout:/var/lib/ceph 2023-11-26T07:19:03.860 INFO:teuthology.orchestra.run.smithi143.stdout:/var/lib/ceph/osd/ceph-0 2023-11-26T07:19:03.860 INFO:teuthology.orchestra.run.smithi143.stdout:/var/lib/ceph/osd/ceph-1 2023-11-26T07:19:03.860 INFO:teuthology.orchestra.run.smithi143.stdout:/var/lib/ceph/osd/ceph-2 2023-11-26T07:19:03.860 INFO:teuthology.orchestra.run.smithi143.stdout:/var/lib/ceph/osd/ceph-3 2023-11-26T07:19:03.860 INFO:teuthology.orchestra.run.smithi143.stdout:/run/netns 2023-11-26T07:19:03.860 INFO:teuthology.orchestra.run.smithi143.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2023-11-26T07:19:03.860 INFO:teuthology.orchestra.run.smithi143.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2023-11-26T07:19:03.861 INFO:teuthology.orchestra.run.smithi143.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 2023-11-26T07:19:03.861 INFO:teuthology.orchestra.run.smithi143.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 2023-11-26T07:19:03.861 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-26T07:19:03.862 DEBUG:teuthology.orchestra.run.smithi143:> ls /sys/fs/fuse/connections 2023-11-26T07:19:03.912 INFO:tasks.cephfs.fuse_mount:Pre-mount connections: [] 2023-11-26T07:19:03.912 DEBUG:teuthology.orchestra.run.smithi143:> (cd /home/ubuntu/cephtest && exec sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-fuse -f --admin-socket '/var/run/ceph/$cluster-$name.$pid.asok' /home/ubuntu/cephtest/mnt.0 --id 0) 2023-11-26T07:19:03.959 DEBUG:teuthology.orchestra.run.smithi143:> sudo modprobe fuse 2023-11-26T07:19:03.980 DEBUG:teuthology.orchestra.run.smithi143:> cat /proc/self/mounts | awk '{print $2}' 2023-11-26T07:19:04.034 INFO:teuthology.orchestra.run.smithi143.stdout:/sys 2023-11-26T07:19:04.034 INFO:teuthology.orchestra.run.smithi143.stdout:/proc 2023-11-26T07:19:04.034 INFO:teuthology.orchestra.run.smithi143.stdout:/dev 2023-11-26T07:19:04.034 INFO:teuthology.orchestra.run.smithi143.stdout:/dev/pts 2023-11-26T07:19:04.034 INFO:teuthology.orchestra.run.smithi143.stdout:/run 2023-11-26T07:19:04.035 INFO:teuthology.orchestra.run.smithi143.stdout:/ 2023-11-26T07:19:04.035 INFO:teuthology.orchestra.run.smithi143.stdout:/sys/kernel/security 2023-11-26T07:19:04.035 INFO:teuthology.orchestra.run.smithi143.stdout:/dev/shm 2023-11-26T07:19:04.035 INFO:teuthology.orchestra.run.smithi143.stdout:/run/lock 2023-11-26T07:19:04.035 INFO:teuthology.orchestra.run.smithi143.stdout:/sys/fs/cgroup 2023-11-26T07:19:04.035 INFO:teuthology.orchestra.run.smithi143.stdout:/sys/fs/pstore 2023-11-26T07:19:04.035 INFO:teuthology.orchestra.run.smithi143.stdout:/sys/fs/bpf 2023-11-26T07:19:04.035 INFO:teuthology.orchestra.run.smithi143.stdout:/proc/sys/fs/binfmt_misc 2023-11-26T07:19:04.035 INFO:teuthology.orchestra.run.smithi143.stdout:/dev/hugepages 2023-11-26T07:19:04.035 INFO:teuthology.orchestra.run.smithi143.stdout:/dev/mqueue 2023-11-26T07:19:04.035 INFO:teuthology.orchestra.run.smithi143.stdout:/sys/kernel/debug 2023-11-26T07:19:04.035 INFO:teuthology.orchestra.run.smithi143.stdout:/sys/kernel/tracing 2023-11-26T07:19:04.035 INFO:teuthology.orchestra.run.smithi143.stdout:/proc/fs/nfsd 2023-11-26T07:19:04.036 INFO:teuthology.orchestra.run.smithi143.stdout:/sys/fs/fuse/connections 2023-11-26T07:19:04.036 INFO:teuthology.orchestra.run.smithi143.stdout:/sys/kernel/config 2023-11-26T07:19:04.036 INFO:teuthology.orchestra.run.smithi143.stdout:/run/credentials/systemd-sysusers.service 2023-11-26T07:19:04.036 INFO:teuthology.orchestra.run.smithi143.stdout:/run/rpc_pipefs 2023-11-26T07:19:04.036 INFO:teuthology.orchestra.run.smithi143.stdout:/run/user/1000 2023-11-26T07:19:04.036 INFO:teuthology.orchestra.run.smithi143.stdout:/var/lib/ceph 2023-11-26T07:19:04.036 INFO:teuthology.orchestra.run.smithi143.stdout:/var/lib/ceph/osd/ceph-0 2023-11-26T07:19:04.036 INFO:teuthology.orchestra.run.smithi143.stdout:/var/lib/ceph/osd/ceph-1 2023-11-26T07:19:04.036 INFO:teuthology.orchestra.run.smithi143.stdout:/var/lib/ceph/osd/ceph-2 2023-11-26T07:19:04.036 INFO:teuthology.orchestra.run.smithi143.stdout:/var/lib/ceph/osd/ceph-3 2023-11-26T07:19:04.036 INFO:teuthology.orchestra.run.smithi143.stdout:/run/netns 2023-11-26T07:19:04.036 INFO:teuthology.orchestra.run.smithi143.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2023-11-26T07:19:04.037 INFO:teuthology.orchestra.run.smithi143.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2023-11-26T07:19:04.037 INFO:teuthology.orchestra.run.smithi143.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 2023-11-26T07:19:04.037 INFO:teuthology.orchestra.run.smithi143.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 2023-11-26T07:19:04.037 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-26T07:19:04.038 DEBUG:teuthology.orchestra.run.smithi143:> ls /sys/fs/fuse/connections 2023-11-26T07:19:04.054 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi143.stderr:2023-11-26T07:19:04.052+0000 7f7ea2255580 -1 init, newargv = 0x55dcc19ff3c0 newargc=15 2023-11-26T07:19:04.054 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi143.stderr:ceph-fuse[48976]: starting ceph client 2023-11-26T07:19:04.061 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi143.stderr:ceph-fuse[48976]: starting fuse 2023-11-26T07:19:04.063 INFO:teuthology.orchestra.run.smithi143.stdout:43 2023-11-26T07:19:04.064 INFO:tasks.cephfs.fuse_mount:Post-mount connections: [43] 2023-11-26T07:19:04.064 DEBUG:teuthology.orchestra.run.smithi143:> sudo stdin-killer -- python3 -c ' 2023-11-26T07:19:04.064 DEBUG:teuthology.orchestra.run.smithi143:> import glob 2023-11-26T07:19:04.064 DEBUG:teuthology.orchestra.run.smithi143:> import re 2023-11-26T07:19:04.064 DEBUG:teuthology.orchestra.run.smithi143:> import os 2023-11-26T07:19:04.064 DEBUG:teuthology.orchestra.run.smithi143:> import subprocess 2023-11-26T07:19:04.064 DEBUG:teuthology.orchestra.run.smithi143:> 2023-11-26T07:19:04.064 DEBUG:teuthology.orchestra.run.smithi143:> def _find_admin_socket(client_name): 2023-11-26T07:19:04.065 DEBUG:teuthology.orchestra.run.smithi143:> asok_path = "/var/run/ceph/ceph-client.0.*.asok" 2023-11-26T07:19:04.065 DEBUG:teuthology.orchestra.run.smithi143:> files = glob.glob(asok_path) 2023-11-26T07:19:04.065 DEBUG:teuthology.orchestra.run.smithi143:> mountpoint = "/home/ubuntu/cephtest/mnt.0" 2023-11-26T07:19:04.065 DEBUG:teuthology.orchestra.run.smithi143:> 2023-11-26T07:19:04.065 DEBUG:teuthology.orchestra.run.smithi143:> # Given a non-glob path, it better be there 2023-11-26T07:19:04.065 DEBUG:teuthology.orchestra.run.smithi143:> if "*" not in asok_path: 2023-11-26T07:19:04.065 DEBUG:teuthology.orchestra.run.smithi143:> assert(len(files) == 1) 2023-11-26T07:19:04.065 DEBUG:teuthology.orchestra.run.smithi143:> return files[0] 2023-11-26T07:19:04.065 DEBUG:teuthology.orchestra.run.smithi143:> 2023-11-26T07:19:04.065 DEBUG:teuthology.orchestra.run.smithi143:> for f in files: 2023-11-26T07:19:04.065 DEBUG:teuthology.orchestra.run.smithi143:> pid = re.match(".*\.(\d+)\.asok$", f).group(1) 2023-11-26T07:19:04.065 DEBUG:teuthology.orchestra.run.smithi143:> if os.path.exists("/proc/{0}".format(pid)): 2023-11-26T07:19:04.065 DEBUG:teuthology.orchestra.run.smithi143:> with open("/proc/{0}/cmdline".format(pid), '"'"'r'"'"') as proc_f: 2023-11-26T07:19:04.065 DEBUG:teuthology.orchestra.run.smithi143:> contents = proc_f.read() 2023-11-26T07:19:04.066 DEBUG:teuthology.orchestra.run.smithi143:> if mountpoint in contents: 2023-11-26T07:19:04.066 DEBUG:teuthology.orchestra.run.smithi143:> return f 2023-11-26T07:19:04.066 DEBUG:teuthology.orchestra.run.smithi143:> raise RuntimeError("Client socket {0} not found".format(client_name)) 2023-11-26T07:19:04.066 DEBUG:teuthology.orchestra.run.smithi143:> 2023-11-26T07:19:04.066 DEBUG:teuthology.orchestra.run.smithi143:> print(_find_admin_socket("client.0")) 2023-11-26T07:19:04.066 DEBUG:teuthology.orchestra.run.smithi143:> ' 2023-11-26T07:19:04.184 INFO:teuthology.orchestra.run.smithi143.stdout:/var/run/ceph/ceph-client.0.48976.asok 2023-11-26T07:19:04.188 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:04 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2023-11-26T07:19:04.195 INFO:tasks.cephfs.fuse_mount:Found client admin socket at /var/run/ceph/ceph-client.0.48976.asok 2023-11-26T07:19:04.195 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-26T07:19:04.196 DEBUG:teuthology.orchestra.run.smithi143:> sudo ceph --admin-daemon /var/run/ceph/ceph-client.0.48976.asok status 2023-11-26T07:19:04.364 INFO:teuthology.orchestra.run.smithi143.stdout:{ 2023-11-26T07:19:04.364 INFO:teuthology.orchestra.run.smithi143.stdout: "metadata": { 2023-11-26T07:19:04.364 INFO:teuthology.orchestra.run.smithi143.stdout: "ceph_sha1": "44c2237fe77713214148c1fd622c872f346f1b99", 2023-11-26T07:19:04.364 INFO:teuthology.orchestra.run.smithi143.stdout: "ceph_version": "ceph version 18.0.0-7484-g44c2237f (44c2237fe77713214148c1fd622c872f346f1b99) reef (dev)", 2023-11-26T07:19:04.364 INFO:teuthology.orchestra.run.smithi143.stdout: "entity_id": "0", 2023-11-26T07:19:04.364 INFO:teuthology.orchestra.run.smithi143.stdout: "hostname": "smithi143", 2023-11-26T07:19:04.364 INFO:teuthology.orchestra.run.smithi143.stdout: "mount_point": "/home/ubuntu/cephtest/mnt.0", 2023-11-26T07:19:04.364 INFO:teuthology.orchestra.run.smithi143.stdout: "pid": "48976", 2023-11-26T07:19:04.364 INFO:teuthology.orchestra.run.smithi143.stdout: "root": "/" 2023-11-26T07:19:04.364 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2023-11-26T07:19:04.364 INFO:teuthology.orchestra.run.smithi143.stdout: "dentry_count": 0, 2023-11-26T07:19:04.365 INFO:teuthology.orchestra.run.smithi143.stdout: "dentry_pinned_count": 0, 2023-11-26T07:19:04.365 INFO:teuthology.orchestra.run.smithi143.stdout: "id": 5448, 2023-11-26T07:19:04.365 INFO:teuthology.orchestra.run.smithi143.stdout: "inst": { 2023-11-26T07:19:04.365 INFO:teuthology.orchestra.run.smithi143.stdout: "name": { 2023-11-26T07:19:04.365 INFO:teuthology.orchestra.run.smithi143.stdout: "type": "client", 2023-11-26T07:19:04.365 INFO:teuthology.orchestra.run.smithi143.stdout: "num": 5448 2023-11-26T07:19:04.365 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2023-11-26T07:19:04.365 INFO:teuthology.orchestra.run.smithi143.stdout: "addr": { 2023-11-26T07:19:04.365 INFO:teuthology.orchestra.run.smithi143.stdout: "type": "v1", 2023-11-26T07:19:04.365 INFO:teuthology.orchestra.run.smithi143.stdout: "addr": "192.168.0.1:0", 2023-11-26T07:19:04.365 INFO:teuthology.orchestra.run.smithi143.stdout: "nonce": 2175779416 2023-11-26T07:19:04.365 INFO:teuthology.orchestra.run.smithi143.stdout: } 2023-11-26T07:19:04.366 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2023-11-26T07:19:04.366 INFO:teuthology.orchestra.run.smithi143.stdout: "addr": { 2023-11-26T07:19:04.366 INFO:teuthology.orchestra.run.smithi143.stdout: "type": "v1", 2023-11-26T07:19:04.366 INFO:teuthology.orchestra.run.smithi143.stdout: "addr": "192.168.0.1:0", 2023-11-26T07:19:04.366 INFO:teuthology.orchestra.run.smithi143.stdout: "nonce": 2175779416 2023-11-26T07:19:04.366 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2023-11-26T07:19:04.366 INFO:teuthology.orchestra.run.smithi143.stdout: "inst_str": "client.5448 192.168.0.1:0/2175779416", 2023-11-26T07:19:04.366 INFO:teuthology.orchestra.run.smithi143.stdout: "addr_str": "192.168.0.1:0/2175779416", 2023-11-26T07:19:04.366 INFO:teuthology.orchestra.run.smithi143.stdout: "inode_count": 1, 2023-11-26T07:19:04.366 INFO:teuthology.orchestra.run.smithi143.stdout: "mds_epoch": 16, 2023-11-26T07:19:04.366 INFO:teuthology.orchestra.run.smithi143.stdout: "osd_epoch": 33, 2023-11-26T07:19:04.366 INFO:teuthology.orchestra.run.smithi143.stdout: "osd_epoch_barrier": 0, 2023-11-26T07:19:04.367 INFO:teuthology.orchestra.run.smithi143.stdout: "blocklisted": false, 2023-11-26T07:19:04.367 INFO:teuthology.orchestra.run.smithi143.stdout: "fs_name": "cephfs" 2023-11-26T07:19:04.367 INFO:teuthology.orchestra.run.smithi143.stdout:} 2023-11-26T07:19:04.380 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-26T07:19:04.380 DEBUG:teuthology.orchestra.run.smithi143:> stat --file-system '--printf=%T 2023-11-26T07:19:04.380 DEBUG:teuthology.orchestra.run.smithi143:> ' -- /home/ubuntu/cephtest/mnt.0 2023-11-26T07:19:04.389 INFO:teuthology.orchestra.run.smithi143.stdout:fuseblk 2023-11-26T07:19:04.390 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.0 2023-11-26T07:19:04.390 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-26T07:19:04.390 DEBUG:teuthology.orchestra.run.smithi143:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.0 2023-11-26T07:19:04.454 DEBUG:teuthology.orchestra.run.smithi143:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2023-11-26T07:19:04.634 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:04.632+0000 7f5a126bb640 1 Processor -- start 2023-11-26T07:19:04.634 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:04.632+0000 7f5a126bb640 1 -- start start 2023-11-26T07:19:04.635 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:04.632+0000 7f5a126bb640 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7f5a0c15d6e0 0x7f5a0c0d0df0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:19:04.635 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:04.632+0000 7f5a126bb640 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7f5a0c0d1bc0 0x7f5a0c0b0750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:19:04.636 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:04.632+0000 7f5a126bb640 1 -- --> v1:172.21.15.143:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5a0c0cf0e0 con 0x7f5a0c0b2160 2023-11-26T07:19:04.636 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:04.632+0000 7f5a126bb640 1 -- --> v2:172.21.15.143:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5a0c15dac0 con 0x7f5a0c15d6e0 2023-11-26T07:19:04.636 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:04.632+0000 7f5a126bb640 1 -- --> v2:172.21.15.188:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5a0c0ceb20 con 0x7f5a0c0d1bc0 2023-11-26T07:19:04.636 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:04.632+0000 7f5a0bfff640 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7f5a0c15d6e0 0x7f5a0c0d0df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:19:04.636 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:04.632+0000 7f5a0bfff640 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7f5a0c15d6e0 0x7f5a0c0d0df0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:48432/0 (socket says 172.21.15.143:48432) 2023-11-26T07:19:04.636 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:04.632+0000 7f5a0b7fe640 1 --1- >> v1:172.21.15.143:6789/0 conn(0x7f5a0c0b2160 0x7f5a0c0d14b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.143:6789/0 says I am v1:172.21.15.143:38216/0 (socket says 172.21.15.143:38216) 2023-11-26T07:19:04.636 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:04.632+0000 7f5a0bfff640 1 -- 172.21.15.143:0/3044073091 learned_addr learned my addr 172.21.15.143:0/3044073091 (peer_addr_for_me v2:172.21.15.143:0/0) 2023-11-26T07:19:04.636 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:04.632+0000 7f5a10c58640 1 --2- 172.21.15.143:0/3044073091 >> v2:172.21.15.188:3300/0 conn(0x7f5a0c0d1bc0 0x7f5a0c0b0750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:19:04.636 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:04.632+0000 7f5a0bfff640 1 -- 172.21.15.143:0/3044073091 >> v2:172.21.15.188:3300/0 conn(0x7f5a0c0d1bc0 msgr2=0x7f5a0c0b0750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-26T07:19:04.636 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:04.632+0000 7f5a0bfff640 1 --2- 172.21.15.143:0/3044073091 >> v2:172.21.15.188:3300/0 conn(0x7f5a0c0d1bc0 0x7f5a0c0b0750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:19:04.636 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:04.632+0000 7f5a0bfff640 1 -- 172.21.15.143:0/3044073091 >> v1:172.21.15.143:6789/0 conn(0x7f5a0c0b2160 legacy=0x7f5a0c0d14b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:19:04.636 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:04.632+0000 7f5a0bfff640 1 -- 172.21.15.143:0/3044073091 --> v2:172.21.15.143:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5a0c0cee00 con 0x7f5a0c15d6e0 2023-11-26T07:19:04.637 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:04.632+0000 7f5a10c58640 1 --2- 172.21.15.143:0/3044073091 >> v2:172.21.15.188:3300/0 conn(0x7f5a0c0d1bc0 0x7f5a0c0b0750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-26T07:19:04.639 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:04.636+0000 7f5a0bfff640 1 --2- 172.21.15.143:0/3044073091 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f5a0c15d6e0 0x7f5a0c0d0df0 secure :-1 s=READY pgs=441 cs=0 l=1 rev1=1 crypto rx=0x7f5a0c0b92d0 tx=0x7f59fc01c500 comp rx=0 tx=0).ready entity=mon.0 client_cookie=10d30287d74aa639 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:19:04.639 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:04.636+0000 7f5a0affd640 1 -- 172.21.15.143:0/3044073091 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f59fc0ab5f0 con 0x7f5a0c15d6e0 2023-11-26T07:19:04.639 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:04.636+0000 7f5a0affd640 1 -- 172.21.15.143:0/3044073091 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f59fc0ab750 con 0x7f5a0c15d6e0 2023-11-26T07:19:04.639 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:04.636+0000 7f5a0affd640 1 -- 172.21.15.143:0/3044073091 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f59fc0abae0 con 0x7f5a0c15d6e0 2023-11-26T07:19:04.640 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:04.636+0000 7f5a126bb640 1 -- 172.21.15.143:0/3044073091 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f5a0c15d6e0 msgr2=0x7f5a0c0d0df0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:19:04.640 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:04.636+0000 7f5a126bb640 1 --2- 172.21.15.143:0/3044073091 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f5a0c15d6e0 0x7f5a0c0d0df0 secure :-1 s=READY pgs=441 cs=0 l=1 rev1=1 crypto rx=0x7f5a0c0b92d0 tx=0x7f59fc01c500 comp rx=0 tx=0).stop 2023-11-26T07:19:04.640 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:04.636+0000 7f5a126bb640 1 -- 172.21.15.143:0/3044073091 shutdown_connections 2023-11-26T07:19:04.640 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:04.636+0000 7f5a126bb640 1 --2- 172.21.15.143:0/3044073091 >> v2:172.21.15.188:3300/0 conn(0x7f5a0c0d1bc0 0x7f5a0c0b0750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:19:04.640 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:04.636+0000 7f5a126bb640 1 --2- 172.21.15.143:0/3044073091 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f5a0c15d6e0 0x7f5a0c0d0df0 unknown :-1 s=CLOSED pgs=441 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:19:04.640 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:04.636+0000 7f5a126bb640 1 -- 172.21.15.143:0/3044073091 >> 172.21.15.143:0/3044073091 conn(0x7f5a0c09c550 msgr2=0x7f5a0c0d3220 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:19:04.640 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:04.636+0000 7f5a126bb640 1 -- 172.21.15.143:0/3044073091 shutdown_connections 2023-11-26T07:19:04.640 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:04.636+0000 7f5a126bb640 1 -- 172.21.15.143:0/3044073091 wait complete. 2023-11-26T07:19:04.641 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:04.640+0000 7f5a126bb640 1 Processor -- start 2023-11-26T07:19:04.642 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:04.640+0000 7f5a126bb640 1 -- start start 2023-11-26T07:19:04.643 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:04.640+0000 7f5a126bb640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f5a0c0b2160 0x7f5a0c1552b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:19:04.643 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:04.640+0000 7f5a126bb640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f5a0c0d1bc0 0x7f5a0c0cca80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:19:04.643 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:04.640+0000 7f5a126bb640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f5a0c15d6e0 0x7f5a0c06c110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:19:04.643 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:04.640+0000 7f5a126bb640 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5a0c0c8510 con 0x7f5a0c15d6e0 2023-11-26T07:19:04.643 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:04.640+0000 7f5a126bb640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5a0c0c86a0 con 0x7f5a0c0d1bc0 2023-11-26T07:19:04.643 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:04.640+0000 7f5a126bb640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5a0c0ca2f0 con 0x7f5a0c0b2160 2023-11-26T07:19:04.643 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:04.640+0000 7f5a10c58640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f5a0c15d6e0 0x7f5a0c06c110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:19:04.643 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:04.640+0000 7f5a10c58640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f5a0c15d6e0 0x7f5a0c06c110 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:48438/0 (socket says 172.21.15.143:48438) 2023-11-26T07:19:04.643 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:04.640+0000 7f5a10c58640 1 -- 172.21.15.143:0/856045306 learned_addr learned my addr 172.21.15.143:0/856045306 (peer_addr_for_me v2:172.21.15.143:0/0) 2023-11-26T07:19:04.644 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:04.640+0000 7f5a0bfff640 1 --2- 172.21.15.143:0/856045306 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f5a0c0b2160 0x7f5a0c1552b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:19:04.645 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:04.640+0000 7f5a0b7fe640 1 --2- 172.21.15.143:0/856045306 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f5a0c0d1bc0 0x7f5a0c0cca80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:19:04.645 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:04.640+0000 7f5a10c58640 1 -- 172.21.15.143:0/856045306 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f5a0c0b2160 msgr2=0x7f5a0c1552b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:19:04.645 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:04.640+0000 7f5a10c58640 1 --2- 172.21.15.143:0/856045306 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f5a0c0b2160 0x7f5a0c1552b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:19:04.645 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:04.640+0000 7f5a10c58640 1 -- 172.21.15.143:0/856045306 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f5a0c0d1bc0 msgr2=0x7f5a0c0cca80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:19:04.645 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:04.640+0000 7f5a10c58640 1 --2- 172.21.15.143:0/856045306 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f5a0c0d1bc0 0x7f5a0c0cca80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:19:04.645 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:04.640+0000 7f5a10c58640 1 -- 172.21.15.143:0/856045306 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f59fc0094d0 con 0x7f5a0c15d6e0 2023-11-26T07:19:04.645 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:04.640+0000 7f5a0b7fe640 1 --2- 172.21.15.143:0/856045306 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f5a0c0d1bc0 0x7f5a0c0cca80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-26T07:19:04.645 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:04.640+0000 7f5a10c58640 1 --2- 172.21.15.143:0/856045306 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f5a0c15d6e0 0x7f5a0c06c110 secure :-1 s=READY pgs=442 cs=0 l=1 rev1=1 crypto rx=0x7f5a04007ec0 tx=0x7f5a0400d4a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:19:04.645 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:04.640+0000 7f5a08ff9640 1 -- 172.21.15.143:0/856045306 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5a04003ba0 con 0x7f5a0c15d6e0 2023-11-26T07:19:04.645 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:04.640+0000 7f5a08ff9640 1 -- 172.21.15.143:0/856045306 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5a0400f040 con 0x7f5a0c15d6e0 2023-11-26T07:19:04.645 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:04.640+0000 7f5a08ff9640 1 -- 172.21.15.143:0/856045306 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5a0400de80 con 0x7f5a0c15d6e0 2023-11-26T07:19:04.646 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:04.640+0000 7f5a126bb640 1 -- 172.21.15.143:0/856045306 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5a0c1547c0 con 0x7f5a0c15d6e0 2023-11-26T07:19:04.646 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:04.640+0000 7f5a126bb640 1 -- 172.21.15.143:0/856045306 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5a0c0c7080 con 0x7f5a0c15d6e0 2023-11-26T07:19:04.646 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:04.644+0000 7f5a126bb640 1 -- 172.21.15.143:0/856045306 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5a0c0c8810 con 0x7f5a0c15d6e0 2023-11-26T07:19:04.646 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:04.644+0000 7f5a08ff9640 1 -- 172.21.15.143:0/856045306 <== mon.0 v2:172.21.15.143:3300/0 4 ==== mgrmap(e 5) v1 ==== 85852+0+0 (secure 0 0 0) 0x7f5a04004030 con 0x7f5a0c15d6e0 2023-11-26T07:19:04.647 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:04.644+0000 7f5a08ff9640 1 --2- 172.21.15.143:0/856045306 >> [v2:172.21.15.143:6832/2150557544,v1:172.21.15.143:6833/2150557544] conn(0x7f59f8066470 0x7f59f8068930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:19:04.647 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:04.644+0000 7f5a08ff9640 1 -- 172.21.15.143:0/856045306 <== mon.0 v2:172.21.15.143:3300/0 5 ==== osd_map(33..33 src has 1..33) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f5a04084130 con 0x7f5a0c15d6e0 2023-11-26T07:19:04.647 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:04.644+0000 7f5a0bfff640 1 --2- 172.21.15.143:0/856045306 >> [v2:172.21.15.143:6832/2150557544,v1:172.21.15.143:6833/2150557544] conn(0x7f59f8066470 0x7f59f8068930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:19:04.647 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:04.644+0000 7f5a0bfff640 1 --2- 172.21.15.143:0/856045306 >> [v2:172.21.15.143:6832/2150557544,v1:172.21.15.143:6833/2150557544] conn(0x7f59f8066470 0x7f59f8068930 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7f59fc0af000 tx=0x7f59fc0a7000 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:19:04.651 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:04.648+0000 7f5a08ff9640 1 -- 172.21.15.143:0/856045306 <== mon.0 v2:172.21.15.143:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172649 (secure 0 0 0) 0x7f5a0c0c8810 con 0x7f5a0c15d6e0 2023-11-26T07:19:04.835 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:04.832+0000 7f5a126bb640 1 -- 172.21.15.143:0/856045306 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "fs ls"} v 0) v1 -- 0x7f5a0c0c8810 con 0x7f5a0c15d6e0 2023-11-26T07:19:04.835 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:04.832+0000 7f5a08ff9640 1 -- 172.21.15.143:0/856045306 <== mon.0 v2:172.21.15.143:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v16) v1 ==== 53+0+73 (secure 0 0 0) 0x7f5a0c0c8810 con 0x7f5a0c15d6e0 2023-11-26T07:19:04.836 INFO:teuthology.orchestra.run.smithi143.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2023-11-26T07:19:04.838 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:04.836+0000 7f5a126bb640 1 -- 172.21.15.143:0/856045306 >> [v2:172.21.15.143:6832/2150557544,v1:172.21.15.143:6833/2150557544] conn(0x7f59f8066470 msgr2=0x7f59f8068930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:19:04.838 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:04.836+0000 7f5a126bb640 1 --2- 172.21.15.143:0/856045306 >> [v2:172.21.15.143:6832/2150557544,v1:172.21.15.143:6833/2150557544] conn(0x7f59f8066470 0x7f59f8068930 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7f59fc0af000 tx=0x7f59fc0a7000 comp rx=0 tx=0).stop 2023-11-26T07:19:04.838 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:04.836+0000 7f5a126bb640 1 -- 172.21.15.143:0/856045306 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f5a0c15d6e0 msgr2=0x7f5a0c06c110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:19:04.838 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:04.836+0000 7f5a126bb640 1 --2- 172.21.15.143:0/856045306 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f5a0c15d6e0 0x7f5a0c06c110 secure :-1 s=READY pgs=442 cs=0 l=1 rev1=1 crypto rx=0x7f5a04007ec0 tx=0x7f5a0400d4a0 comp rx=0 tx=0).stop 2023-11-26T07:19:04.838 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:04.836+0000 7f5a126bb640 1 -- 172.21.15.143:0/856045306 shutdown_connections 2023-11-26T07:19:04.838 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:04.836+0000 7f5a126bb640 1 --2- 172.21.15.143:0/856045306 >> [v2:172.21.15.143:6832/2150557544,v1:172.21.15.143:6833/2150557544] conn(0x7f59f8066470 0x7f59f8068930 unknown :-1 s=CLOSED pgs=232 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:19:04.838 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:04.836+0000 7f5a126bb640 1 --2- 172.21.15.143:0/856045306 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f5a0c15d6e0 0x7f5a0c06c110 unknown :-1 s=CLOSED pgs=442 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:19:04.838 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:04.836+0000 7f5a126bb640 1 --2- 172.21.15.143:0/856045306 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f5a0c0d1bc0 0x7f5a0c0cca80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:19:04.838 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:04.836+0000 7f5a126bb640 1 --2- 172.21.15.143:0/856045306 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f5a0c0b2160 0x7f5a0c1552b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:19:04.839 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:04.836+0000 7f5a126bb640 1 -- 172.21.15.143:0/856045306 >> 172.21.15.143:0/856045306 conn(0x7f5a0c09c550 msgr2=0x7f5a0c155f70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:19:04.839 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:04.836+0000 7f5a126bb640 1 -- 172.21.15.143:0/856045306 shutdown_connections 2023-11-26T07:19:04.839 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:04.836+0000 7f5a126bb640 1 -- 172.21.15.143:0/856045306 wait complete. 2023-11-26T07:19:04.856 INFO:tasks.cephfs.mount:Mounting default Ceph FS; just confirmed its presence on cluster 2023-11-26T07:19:04.856 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2023-11-26T07:19:04.856 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi188.front.sepia.ceph.com 2023-11-26T07:19:04.857 INFO:tasks.cephfs.mount:self.client.name = client.1 2023-11-26T07:19:04.857 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.1 2023-11-26T07:19:04.857 INFO:tasks.cephfs.mount:self.cephfs_name = None 2023-11-26T07:19:04.857 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2023-11-26T07:19:04.857 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2023-11-26T07:19:04.857 INFO:tasks.cephfs.mount:Setting the 'ceph-ns--home-ubuntu-cephtest-mnt.1' netns for '/home/ubuntu/cephtest/mnt.1' 2023-11-26T07:19:04.857 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-26T07:19:04.857 DEBUG:teuthology.orchestra.run.smithi188:> ip addr 2023-11-26T07:19:04.868 INFO:teuthology.orchestra.run.smithi188.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2023-11-26T07:19:04.868 INFO:teuthology.orchestra.run.smithi188.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2023-11-26T07:19:04.868 INFO:teuthology.orchestra.run.smithi188.stdout: inet 127.0.0.1/8 scope host lo 2023-11-26T07:19:04.868 INFO:teuthology.orchestra.run.smithi188.stdout: valid_lft forever preferred_lft forever 2023-11-26T07:19:04.868 INFO:teuthology.orchestra.run.smithi188.stdout: inet6 ::1/128 scope host 2023-11-26T07:19:04.868 INFO:teuthology.orchestra.run.smithi188.stdout: valid_lft forever preferred_lft forever 2023-11-26T07:19:04.868 INFO:teuthology.orchestra.run.smithi188.stdout:2: eno1: mtu 1500 qdisc noop state DOWN group default qlen 1000 2023-11-26T07:19:04.868 INFO:teuthology.orchestra.run.smithi188.stdout: link/ether 0c:c4:7a:d9:37:48 brd ff:ff:ff:ff:ff:ff 2023-11-26T07:19:04.868 INFO:teuthology.orchestra.run.smithi188.stdout: altname enp7s0f0 2023-11-26T07:19:04.869 INFO:teuthology.orchestra.run.smithi188.stdout:3: eno2: mtu 1500 qdisc noop state DOWN group default qlen 1000 2023-11-26T07:19:04.869 INFO:teuthology.orchestra.run.smithi188.stdout: link/ether 0c:c4:7a:d9:37:49 brd ff:ff:ff:ff:ff:ff 2023-11-26T07:19:04.869 INFO:teuthology.orchestra.run.smithi188.stdout: altname enp7s0f1 2023-11-26T07:19:04.869 INFO:teuthology.orchestra.run.smithi188.stdout:4: enp3s0f0: mtu 1500 qdisc noop state DOWN group default qlen 1000 2023-11-26T07:19:04.869 INFO:teuthology.orchestra.run.smithi188.stdout: link/ether 0c:c4:7a:88:80:38 brd ff:ff:ff:ff:ff:ff 2023-11-26T07:19:04.869 INFO:teuthology.orchestra.run.smithi188.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2023-11-26T07:19:04.869 INFO:teuthology.orchestra.run.smithi188.stdout: link/ether 0c:c4:7a:88:80:39 brd ff:ff:ff:ff:ff:ff 2023-11-26T07:19:04.869 INFO:teuthology.orchestra.run.smithi188.stdout: inet 172.21.15.188/20 metric 100 brd 172.21.15.255 scope global dynamic enp3s0f1 2023-11-26T07:19:04.869 INFO:teuthology.orchestra.run.smithi188.stdout: valid_lft 42323sec preferred_lft 42323sec 2023-11-26T07:19:04.869 INFO:teuthology.orchestra.run.smithi188.stdout: inet6 fe80::ec4:7aff:fe88:8039/64 scope link 2023-11-26T07:19:04.869 INFO:teuthology.orchestra.run.smithi188.stdout: valid_lft forever preferred_lft forever 2023-11-26T07:19:04.869 INFO:teuthology.orchestra.run.smithi188.stdout:6: docker0: mtu 1500 qdisc noqueue state DOWN group default 2023-11-26T07:19:04.870 INFO:teuthology.orchestra.run.smithi188.stdout: link/ether 02:42:d8:0d:a0:fa brd ff:ff:ff:ff:ff:ff 2023-11-26T07:19:04.870 INFO:teuthology.orchestra.run.smithi188.stdout: inet 172.17.0.1/16 brd 172.17.255.255 scope global docker0 2023-11-26T07:19:04.870 INFO:teuthology.orchestra.run.smithi188.stdout: valid_lft forever preferred_lft forever 2023-11-26T07:19:04.870 INFO:teuthology.orchestra.run.smithi188.stdout:7: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2023-11-26T07:19:04.870 INFO:teuthology.orchestra.run.smithi188.stdout: link/ether 5e:1f:84:6d:79:ae brd ff:ff:ff:ff:ff:ff 2023-11-26T07:19:04.870 INFO:teuthology.orchestra.run.smithi188.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2023-11-26T07:19:04.870 INFO:teuthology.orchestra.run.smithi188.stdout: valid_lft forever preferred_lft forever 2023-11-26T07:19:04.870 INFO:teuthology.orchestra.run.smithi188.stdout: inet6 fe80::5c1f:84ff:fe6d:79ae/64 scope link 2023-11-26T07:19:04.870 INFO:teuthology.orchestra.run.smithi188.stdout: valid_lft forever preferred_lft forever 2023-11-26T07:19:04.870 INFO:teuthology.orchestra.run.smithi188.stdout:8: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2023-11-26T07:19:04.870 INFO:teuthology.orchestra.run.smithi188.stdout: link/ether 0e:5c:f6:1f:82:2f brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.1 2023-11-26T07:19:04.871 INFO:teuthology.orchestra.run.smithi188.stdout: inet6 fe80::c5c:f6ff:fe1f:822f/64 scope link 2023-11-26T07:19:04.871 INFO:teuthology.orchestra.run.smithi188.stdout: valid_lft forever preferred_lft forever 2023-11-26T07:19:04.871 INFO:teuthology.orchestra.run.smithi188.stdout:9: brx.1@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2023-11-26T07:19:04.871 INFO:teuthology.orchestra.run.smithi188.stdout: link/ether c6:a6:3d:69:d0:db brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.3 2023-11-26T07:19:04.871 INFO:teuthology.orchestra.run.smithi188.stdout: inet6 fe80::c4a6:3dff:fe69:d0db/64 scope link 2023-11-26T07:19:04.871 INFO:teuthology.orchestra.run.smithi188.stdout: valid_lft forever preferred_lft forever 2023-11-26T07:19:04.872 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-26T07:19:04.872 DEBUG:teuthology.orchestra.run.smithi188:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2023-11-26T07:19:04.934 INFO:teuthology.orchestra.run.smithi188.stdout:1 2023-11-26T07:19:04.936 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-26T07:19:04.936 DEBUG:teuthology.orchestra.run.smithi188:> route 2023-11-26T07:19:04.993 INFO:teuthology.orchestra.run.smithi188.stdout:Kernel IP routing table 2023-11-26T07:19:04.993 INFO:teuthology.orchestra.run.smithi188.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2023-11-26T07:19:04.993 INFO:teuthology.orchestra.run.smithi188.stdout:default _gateway 0.0.0.0 UG 0 0 0 enp3s0f1 2023-11-26T07:19:04.993 INFO:teuthology.orchestra.run.smithi188.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2023-11-26T07:19:04.993 INFO:teuthology.orchestra.run.smithi188.stdout:172.17.0.0 0.0.0.0 255.255.0.0 U 0 0 0 docker0 2023-11-26T07:19:04.993 INFO:teuthology.orchestra.run.smithi188.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 enp3s0f1 2023-11-26T07:19:04.993 INFO:teuthology.orchestra.run.smithi188.stdout:ns1.front.sepia 0.0.0.0 255.255.255.255 UH 100 0 0 enp3s0f1 2023-11-26T07:19:04.993 INFO:teuthology.orchestra.run.smithi188.stdout:172.21.0.2 0.0.0.0 255.255.255.255 UH 100 0 0 enp3s0f1 2023-11-26T07:19:04.993 INFO:teuthology.orchestra.run.smithi188.stdout:_gateway 0.0.0.0 255.255.255.255 UH 100 0 0 enp3s0f1 2023-11-26T07:19:04.993 INFO:teuthology.orchestra.run.smithi188.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2023-11-26T07:19:04.994 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-26T07:19:04.994 DEBUG:teuthology.orchestra.run.smithi188:> (cd / && exec bash -c ' 2023-11-26T07:19:04.994 DEBUG:teuthology.orchestra.run.smithi188:> set -e 2023-11-26T07:19:04.994 DEBUG:teuthology.orchestra.run.smithi188:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2023-11-26T07:19:04.994 DEBUG:teuthology.orchestra.run.smithi188:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2023-11-26T07:19:04.994 DEBUG:teuthology.orchestra.run.smithi188:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2023-11-26T07:19:04.994 DEBUG:teuthology.orchestra.run.smithi188:> ') 2023-11-26T07:19:05.092 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-26T07:19:05.092 DEBUG:teuthology.orchestra.run.smithi188:> ip netns list 2023-11-26T07:19:05.101 INFO:teuthology.orchestra.run.smithi188.stdout:ceph-ns--home-ubuntu-cephtest-mnt.1 (id: 0) 2023-11-26T07:19:05.101 INFO:teuthology.orchestra.run.smithi188.stdout:ceph-ns--home-ubuntu-cephtest-mnt.3 (id: 1) 2023-11-26T07:19:05.102 INFO:tasks.cephfs.mount:Resuming the 'ceph-ns--home-ubuntu-cephtest-mnt.1' netns for '/home/ubuntu/cephtest/mnt.1' 2023-11-26T07:19:05.102 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-26T07:19:05.102 DEBUG:teuthology.orchestra.run.smithi188:> sudo ip link set brx.0 up 2023-11-26T07:19:05.170 INFO:tasks.cephfs.fuse_mount:Client client.1 config is {} 2023-11-26T07:19:05.170 INFO:teuthology.orchestra.run:Running command with timeout 60 2023-11-26T07:19:05.170 DEBUG:teuthology.orchestra.run.smithi188:> mkdir -p -v /home/ubuntu/cephtest/mnt.1 2023-11-26T07:19:05.222 INFO:teuthology.orchestra.run.smithi188.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.1' 2023-11-26T07:19:05.222 INFO:teuthology.orchestra.run:Running command with timeout 60 2023-11-26T07:19:05.222 DEBUG:teuthology.orchestra.run.smithi188:> chmod 0000 /home/ubuntu/cephtest/mnt.1 2023-11-26T07:19:05.277 DEBUG:teuthology.orchestra.run.smithi188:> sudo modprobe fuse 2023-11-26T07:19:05.343 DEBUG:teuthology.orchestra.run.smithi188:> cat /proc/self/mounts | awk '{print $2}' 2023-11-26T07:19:05.395 INFO:teuthology.orchestra.run.smithi188.stdout:/sys 2023-11-26T07:19:05.395 INFO:teuthology.orchestra.run.smithi188.stdout:/proc 2023-11-26T07:19:05.395 INFO:teuthology.orchestra.run.smithi188.stdout:/dev 2023-11-26T07:19:05.395 INFO:teuthology.orchestra.run.smithi188.stdout:/dev/pts 2023-11-26T07:19:05.396 INFO:teuthology.orchestra.run.smithi188.stdout:/run 2023-11-26T07:19:05.396 INFO:teuthology.orchestra.run.smithi188.stdout:/ 2023-11-26T07:19:05.396 INFO:teuthology.orchestra.run.smithi188.stdout:/sys/kernel/security 2023-11-26T07:19:05.396 INFO:teuthology.orchestra.run.smithi188.stdout:/dev/shm 2023-11-26T07:19:05.396 INFO:teuthology.orchestra.run.smithi188.stdout:/run/lock 2023-11-26T07:19:05.396 INFO:teuthology.orchestra.run.smithi188.stdout:/sys/fs/cgroup 2023-11-26T07:19:05.396 INFO:teuthology.orchestra.run.smithi188.stdout:/sys/fs/pstore 2023-11-26T07:19:05.396 INFO:teuthology.orchestra.run.smithi188.stdout:/sys/fs/bpf 2023-11-26T07:19:05.396 INFO:teuthology.orchestra.run.smithi188.stdout:/proc/sys/fs/binfmt_misc 2023-11-26T07:19:05.396 INFO:teuthology.orchestra.run.smithi188.stdout:/dev/hugepages 2023-11-26T07:19:05.396 INFO:teuthology.orchestra.run.smithi188.stdout:/dev/mqueue 2023-11-26T07:19:05.396 INFO:teuthology.orchestra.run.smithi188.stdout:/sys/kernel/debug 2023-11-26T07:19:05.397 INFO:teuthology.orchestra.run.smithi188.stdout:/sys/kernel/tracing 2023-11-26T07:19:05.397 INFO:teuthology.orchestra.run.smithi188.stdout:/proc/fs/nfsd 2023-11-26T07:19:05.397 INFO:teuthology.orchestra.run.smithi188.stdout:/sys/fs/fuse/connections 2023-11-26T07:19:05.397 INFO:teuthology.orchestra.run.smithi188.stdout:/sys/kernel/config 2023-11-26T07:19:05.397 INFO:teuthology.orchestra.run.smithi188.stdout:/run/credentials/systemd-sysusers.service 2023-11-26T07:19:05.397 INFO:teuthology.orchestra.run.smithi188.stdout:/run/rpc_pipefs 2023-11-26T07:19:05.397 INFO:teuthology.orchestra.run.smithi188.stdout:/run/user/1000 2023-11-26T07:19:05.397 INFO:teuthology.orchestra.run.smithi188.stdout:/var/lib/ceph 2023-11-26T07:19:05.397 INFO:teuthology.orchestra.run.smithi188.stdout:/var/lib/ceph/osd/ceph-4 2023-11-26T07:19:05.397 INFO:teuthology.orchestra.run.smithi188.stdout:/var/lib/ceph/osd/ceph-5 2023-11-26T07:19:05.397 INFO:teuthology.orchestra.run.smithi188.stdout:/var/lib/ceph/osd/ceph-6 2023-11-26T07:19:05.397 INFO:teuthology.orchestra.run.smithi188.stdout:/var/lib/ceph/osd/ceph-7 2023-11-26T07:19:05.397 INFO:teuthology.orchestra.run.smithi188.stdout:/run/netns 2023-11-26T07:19:05.398 INFO:teuthology.orchestra.run.smithi188.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 2023-11-26T07:19:05.398 INFO:teuthology.orchestra.run.smithi188.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 2023-11-26T07:19:05.398 INFO:teuthology.orchestra.run.smithi188.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 2023-11-26T07:19:05.398 INFO:teuthology.orchestra.run.smithi188.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 2023-11-26T07:19:05.399 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-26T07:19:05.399 DEBUG:teuthology.orchestra.run.smithi188:> ls /sys/fs/fuse/connections 2023-11-26T07:19:05.449 INFO:tasks.cephfs.fuse_mount:Pre-mount connections: [] 2023-11-26T07:19:05.449 DEBUG:teuthology.orchestra.run.smithi188:> (cd /home/ubuntu/cephtest && exec sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-fuse -f --admin-socket '/var/run/ceph/$cluster-$name.$pid.asok' /home/ubuntu/cephtest/mnt.1 --id 1) 2023-11-26T07:19:05.496 DEBUG:teuthology.orchestra.run.smithi188:> sudo modprobe fuse 2023-11-26T07:19:05.518 DEBUG:teuthology.orchestra.run.smithi188:> cat /proc/self/mounts | awk '{print $2}' 2023-11-26T07:19:05.571 INFO:teuthology.orchestra.run.smithi188.stdout:/sys 2023-11-26T07:19:05.571 INFO:teuthology.orchestra.run.smithi188.stdout:/proc 2023-11-26T07:19:05.571 INFO:teuthology.orchestra.run.smithi188.stdout:/dev 2023-11-26T07:19:05.572 INFO:teuthology.orchestra.run.smithi188.stdout:/dev/pts 2023-11-26T07:19:05.572 INFO:teuthology.orchestra.run.smithi188.stdout:/run 2023-11-26T07:19:05.572 INFO:teuthology.orchestra.run.smithi188.stdout:/ 2023-11-26T07:19:05.572 INFO:teuthology.orchestra.run.smithi188.stdout:/sys/kernel/security 2023-11-26T07:19:05.572 INFO:teuthology.orchestra.run.smithi188.stdout:/dev/shm 2023-11-26T07:19:05.572 INFO:teuthology.orchestra.run.smithi188.stdout:/run/lock 2023-11-26T07:19:05.572 INFO:teuthology.orchestra.run.smithi188.stdout:/sys/fs/cgroup 2023-11-26T07:19:05.572 INFO:teuthology.orchestra.run.smithi188.stdout:/sys/fs/pstore 2023-11-26T07:19:05.572 INFO:teuthology.orchestra.run.smithi188.stdout:/sys/fs/bpf 2023-11-26T07:19:05.572 INFO:teuthology.orchestra.run.smithi188.stdout:/proc/sys/fs/binfmt_misc 2023-11-26T07:19:05.572 INFO:teuthology.orchestra.run.smithi188.stdout:/dev/hugepages 2023-11-26T07:19:05.572 INFO:teuthology.orchestra.run.smithi188.stdout:/dev/mqueue 2023-11-26T07:19:05.573 INFO:teuthology.orchestra.run.smithi188.stdout:/sys/kernel/debug 2023-11-26T07:19:05.573 INFO:teuthology.orchestra.run.smithi188.stdout:/sys/kernel/tracing 2023-11-26T07:19:05.573 INFO:teuthology.orchestra.run.smithi188.stdout:/proc/fs/nfsd 2023-11-26T07:19:05.573 INFO:teuthology.orchestra.run.smithi188.stdout:/sys/fs/fuse/connections 2023-11-26T07:19:05.573 INFO:teuthology.orchestra.run.smithi188.stdout:/sys/kernel/config 2023-11-26T07:19:05.573 INFO:teuthology.orchestra.run.smithi188.stdout:/run/credentials/systemd-sysusers.service 2023-11-26T07:19:05.573 INFO:teuthology.orchestra.run.smithi188.stdout:/run/rpc_pipefs 2023-11-26T07:19:05.573 INFO:teuthology.orchestra.run.smithi188.stdout:/run/user/1000 2023-11-26T07:19:05.573 INFO:teuthology.orchestra.run.smithi188.stdout:/var/lib/ceph 2023-11-26T07:19:05.573 INFO:teuthology.orchestra.run.smithi188.stdout:/var/lib/ceph/osd/ceph-4 2023-11-26T07:19:05.573 INFO:teuthology.orchestra.run.smithi188.stdout:/var/lib/ceph/osd/ceph-5 2023-11-26T07:19:05.573 INFO:teuthology.orchestra.run.smithi188.stdout:/var/lib/ceph/osd/ceph-6 2023-11-26T07:19:05.573 INFO:teuthology.orchestra.run.smithi188.stdout:/var/lib/ceph/osd/ceph-7 2023-11-26T07:19:05.574 INFO:teuthology.orchestra.run.smithi188.stdout:/run/netns 2023-11-26T07:19:05.574 INFO:teuthology.orchestra.run.smithi188.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 2023-11-26T07:19:05.574 INFO:teuthology.orchestra.run.smithi188.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 2023-11-26T07:19:05.574 INFO:teuthology.orchestra.run.smithi188.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 2023-11-26T07:19:05.574 INFO:teuthology.orchestra.run.smithi188.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 2023-11-26T07:19:05.575 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-26T07:19:05.575 DEBUG:teuthology.orchestra.run.smithi188:> ls /sys/fs/fuse/connections 2023-11-26T07:19:05.595 INFO:tasks.cephfs.fuse_mount.ceph-fuse.1.smithi188.stderr:2023-11-26T07:19:05.589+0000 7f8dcf517580 -1 init, newargv = 0x56265eb917d0 newargc=15 2023-11-26T07:19:05.596 INFO:tasks.cephfs.fuse_mount.ceph-fuse.1.smithi188.stderr:ceph-fuse[44390]: starting ceph client 2023-11-26T07:19:05.603 INFO:tasks.cephfs.fuse_mount.ceph-fuse.1.smithi188.stderr:ceph-fuse[44390]: starting fuse 2023-11-26T07:19:05.604 INFO:teuthology.orchestra.run.smithi188.stdout:43 2023-11-26T07:19:05.604 INFO:tasks.cephfs.fuse_mount:Post-mount connections: [43] 2023-11-26T07:19:05.604 DEBUG:teuthology.orchestra.run.smithi188:> sudo stdin-killer -- python3 -c ' 2023-11-26T07:19:05.604 DEBUG:teuthology.orchestra.run.smithi188:> import glob 2023-11-26T07:19:05.604 DEBUG:teuthology.orchestra.run.smithi188:> import re 2023-11-26T07:19:05.604 DEBUG:teuthology.orchestra.run.smithi188:> import os 2023-11-26T07:19:05.604 DEBUG:teuthology.orchestra.run.smithi188:> import subprocess 2023-11-26T07:19:05.605 DEBUG:teuthology.orchestra.run.smithi188:> 2023-11-26T07:19:05.605 DEBUG:teuthology.orchestra.run.smithi188:> def _find_admin_socket(client_name): 2023-11-26T07:19:05.605 DEBUG:teuthology.orchestra.run.smithi188:> asok_path = "/var/run/ceph/ceph-client.1.*.asok" 2023-11-26T07:19:05.605 DEBUG:teuthology.orchestra.run.smithi188:> files = glob.glob(asok_path) 2023-11-26T07:19:05.605 DEBUG:teuthology.orchestra.run.smithi188:> mountpoint = "/home/ubuntu/cephtest/mnt.1" 2023-11-26T07:19:05.605 DEBUG:teuthology.orchestra.run.smithi188:> 2023-11-26T07:19:05.605 DEBUG:teuthology.orchestra.run.smithi188:> # Given a non-glob path, it better be there 2023-11-26T07:19:05.605 DEBUG:teuthology.orchestra.run.smithi188:> if "*" not in asok_path: 2023-11-26T07:19:05.605 DEBUG:teuthology.orchestra.run.smithi188:> assert(len(files) == 1) 2023-11-26T07:19:05.605 DEBUG:teuthology.orchestra.run.smithi188:> return files[0] 2023-11-26T07:19:05.605 DEBUG:teuthology.orchestra.run.smithi188:> 2023-11-26T07:19:05.605 DEBUG:teuthology.orchestra.run.smithi188:> for f in files: 2023-11-26T07:19:05.605 DEBUG:teuthology.orchestra.run.smithi188:> pid = re.match(".*\.(\d+)\.asok$", f).group(1) 2023-11-26T07:19:05.605 DEBUG:teuthology.orchestra.run.smithi188:> if os.path.exists("/proc/{0}".format(pid)): 2023-11-26T07:19:05.606 DEBUG:teuthology.orchestra.run.smithi188:> with open("/proc/{0}/cmdline".format(pid), '"'"'r'"'"') as proc_f: 2023-11-26T07:19:05.606 DEBUG:teuthology.orchestra.run.smithi188:> contents = proc_f.read() 2023-11-26T07:19:05.606 DEBUG:teuthology.orchestra.run.smithi188:> if mountpoint in contents: 2023-11-26T07:19:05.606 DEBUG:teuthology.orchestra.run.smithi188:> return f 2023-11-26T07:19:05.606 DEBUG:teuthology.orchestra.run.smithi188:> raise RuntimeError("Client socket {0} not found".format(client_name)) 2023-11-26T07:19:05.606 DEBUG:teuthology.orchestra.run.smithi188:> 2023-11-26T07:19:05.606 DEBUG:teuthology.orchestra.run.smithi188:> print(_find_admin_socket("client.1")) 2023-11-26T07:19:05.606 DEBUG:teuthology.orchestra.run.smithi188:> ' 2023-11-26T07:19:05.730 INFO:teuthology.orchestra.run.smithi188.stdout:/var/run/ceph/ceph-client.1.44390.asok 2023-11-26T07:19:05.733 INFO:teuthology.orchestra.run.smithi188.stderr:2023-11-26T07:19:05 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2023-11-26T07:19:05.741 INFO:tasks.cephfs.fuse_mount:Found client admin socket at /var/run/ceph/ceph-client.1.44390.asok 2023-11-26T07:19:05.741 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-26T07:19:05.741 DEBUG:teuthology.orchestra.run.smithi188:> sudo ceph --admin-daemon /var/run/ceph/ceph-client.1.44390.asok status 2023-11-26T07:19:05.913 INFO:teuthology.orchestra.run.smithi188.stdout:{ 2023-11-26T07:19:05.913 INFO:teuthology.orchestra.run.smithi188.stdout: "metadata": { 2023-11-26T07:19:05.914 INFO:teuthology.orchestra.run.smithi188.stdout: "ceph_sha1": "44c2237fe77713214148c1fd622c872f346f1b99", 2023-11-26T07:19:05.914 INFO:teuthology.orchestra.run.smithi188.stdout: "ceph_version": "ceph version 18.0.0-7484-g44c2237f (44c2237fe77713214148c1fd622c872f346f1b99) reef (dev)", 2023-11-26T07:19:05.914 INFO:teuthology.orchestra.run.smithi188.stdout: "entity_id": "1", 2023-11-26T07:19:05.914 INFO:teuthology.orchestra.run.smithi188.stdout: "hostname": "smithi188", 2023-11-26T07:19:05.914 INFO:teuthology.orchestra.run.smithi188.stdout: "mount_point": "/home/ubuntu/cephtest/mnt.1", 2023-11-26T07:19:05.914 INFO:teuthology.orchestra.run.smithi188.stdout: "pid": "44390", 2023-11-26T07:19:05.914 INFO:teuthology.orchestra.run.smithi188.stdout: "root": "/" 2023-11-26T07:19:05.914 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2023-11-26T07:19:05.914 INFO:teuthology.orchestra.run.smithi188.stdout: "dentry_count": 0, 2023-11-26T07:19:05.914 INFO:teuthology.orchestra.run.smithi188.stdout: "dentry_pinned_count": 0, 2023-11-26T07:19:05.915 INFO:teuthology.orchestra.run.smithi188.stdout: "id": 5308, 2023-11-26T07:19:05.915 INFO:teuthology.orchestra.run.smithi188.stdout: "inst": { 2023-11-26T07:19:05.915 INFO:teuthology.orchestra.run.smithi188.stdout: "name": { 2023-11-26T07:19:05.915 INFO:teuthology.orchestra.run.smithi188.stdout: "type": "client", 2023-11-26T07:19:05.915 INFO:teuthology.orchestra.run.smithi188.stdout: "num": 5308 2023-11-26T07:19:05.915 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2023-11-26T07:19:05.915 INFO:teuthology.orchestra.run.smithi188.stdout: "addr": { 2023-11-26T07:19:05.915 INFO:teuthology.orchestra.run.smithi188.stdout: "type": "v1", 2023-11-26T07:19:05.915 INFO:teuthology.orchestra.run.smithi188.stdout: "addr": "192.168.0.1:0", 2023-11-26T07:19:05.915 INFO:teuthology.orchestra.run.smithi188.stdout: "nonce": 612455784 2023-11-26T07:19:05.915 INFO:teuthology.orchestra.run.smithi188.stdout: } 2023-11-26T07:19:05.915 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2023-11-26T07:19:05.915 INFO:teuthology.orchestra.run.smithi188.stdout: "addr": { 2023-11-26T07:19:05.916 INFO:teuthology.orchestra.run.smithi188.stdout: "type": "v1", 2023-11-26T07:19:05.916 INFO:teuthology.orchestra.run.smithi188.stdout: "addr": "192.168.0.1:0", 2023-11-26T07:19:05.916 INFO:teuthology.orchestra.run.smithi188.stdout: "nonce": 612455784 2023-11-26T07:19:05.916 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2023-11-26T07:19:05.916 INFO:teuthology.orchestra.run.smithi188.stdout: "inst_str": "client.5308 192.168.0.1:0/612455784", 2023-11-26T07:19:05.916 INFO:teuthology.orchestra.run.smithi188.stdout: "addr_str": "192.168.0.1:0/612455784", 2023-11-26T07:19:05.916 INFO:teuthology.orchestra.run.smithi188.stdout: "inode_count": 1, 2023-11-26T07:19:05.916 INFO:teuthology.orchestra.run.smithi188.stdout: "mds_epoch": 16, 2023-11-26T07:19:05.916 INFO:teuthology.orchestra.run.smithi188.stdout: "osd_epoch": 33, 2023-11-26T07:19:05.916 INFO:teuthology.orchestra.run.smithi188.stdout: "osd_epoch_barrier": 0, 2023-11-26T07:19:05.916 INFO:teuthology.orchestra.run.smithi188.stdout: "blocklisted": false, 2023-11-26T07:19:05.917 INFO:teuthology.orchestra.run.smithi188.stdout: "fs_name": "cephfs" 2023-11-26T07:19:05.917 INFO:teuthology.orchestra.run.smithi188.stdout:} 2023-11-26T07:19:05.928 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-26T07:19:05.928 DEBUG:teuthology.orchestra.run.smithi188:> stat --file-system '--printf=%T 2023-11-26T07:19:05.928 DEBUG:teuthology.orchestra.run.smithi188:> ' -- /home/ubuntu/cephtest/mnt.1 2023-11-26T07:19:05.938 INFO:teuthology.orchestra.run.smithi188.stdout:fuseblk 2023-11-26T07:19:05.938 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.1 2023-11-26T07:19:05.938 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-26T07:19:05.938 DEBUG:teuthology.orchestra.run.smithi188:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.1 2023-11-26T07:19:06.005 DEBUG:teuthology.orchestra.run.smithi143:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs flag set enable_multiple true --yes-i-really-mean-it 2023-11-26T07:19:06.157 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:06.156+0000 7fe48b91c640 1 Processor -- start 2023-11-26T07:19:06.158 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:06.156+0000 7fe48b91c640 1 -- start start 2023-11-26T07:19:06.160 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:06.156+0000 7fe48b91c640 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7fe48414b1e0 0x7fe4840cf280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:19:06.160 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:06.156+0000 7fe48b91c640 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7fe484070fd0 0x7fe484162510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:19:06.161 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:06.156+0000 7fe48b91c640 1 -- --> v1:172.21.15.188:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe4840cfc20 con 0x7fe4840cf7c0 2023-11-26T07:19:06.161 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:06.156+0000 7fe48b91c640 1 -- --> v2:172.21.15.143:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe484162080 con 0x7fe48414b1e0 2023-11-26T07:19:06.161 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:06.156+0000 7fe48b91c640 1 -- --> v2:172.21.15.188:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe484161d70 con 0x7fe484070fd0 2023-11-26T07:19:06.161 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:06.156+0000 7fe4896b8640 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7fe48414b1e0 0x7fe4840cf280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:19:06.161 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:06.156+0000 7fe4896b8640 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7fe48414b1e0 0x7fe4840cf280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:48452/0 (socket says 172.21.15.143:48452) 2023-11-26T07:19:06.161 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:06.156+0000 7fe4896b8640 1 -- 172.21.15.143:0/3142426885 learned_addr learned my addr 172.21.15.143:0/3142426885 (peer_addr_for_me v2:172.21.15.143:0/0) 2023-11-26T07:19:06.161 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:06.156+0000 7fe489eb9640 1 --2- 172.21.15.143:0/3142426885 >> v2:172.21.15.188:3300/0 conn(0x7fe484070fd0 0x7fe484162510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:19:06.161 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:06.156+0000 7fe4896b8640 1 -- 172.21.15.143:0/3142426885 >> v2:172.21.15.188:3300/0 conn(0x7fe484070fd0 msgr2=0x7fe484162510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-26T07:19:06.161 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:06.156+0000 7fe4896b8640 1 --2- 172.21.15.143:0/3142426885 >> v2:172.21.15.188:3300/0 conn(0x7fe484070fd0 0x7fe484162510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:19:06.161 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:06.156+0000 7fe4896b8640 1 -- 172.21.15.143:0/3142426885 >> v1:172.21.15.188:6789/0 conn(0x7fe4840cf7c0 legacy=0x7fe4840708c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:19:06.162 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:06.156+0000 7fe4896b8640 1 -- 172.21.15.143:0/3142426885 --> v2:172.21.15.143:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe48414b5c0 con 0x7fe48414b1e0 2023-11-26T07:19:06.162 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:06.156+0000 7fe489eb9640 1 --2- 172.21.15.143:0/3142426885 >> v2:172.21.15.188:3300/0 conn(0x7fe484070fd0 0x7fe484162510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-26T07:19:06.163 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:06.160+0000 7fe4896b8640 1 --2- 172.21.15.143:0/3142426885 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fe48414b1e0 0x7fe4840cf280 secure :-1 s=READY pgs=444 cs=0 l=1 rev1=1 crypto rx=0x7fe484072110 tx=0x7fe48001c4d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a1b8d40053ee085 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:19:06.163 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:06.160+0000 7fe47bfff640 1 -- 172.21.15.143:0/3142426885 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe4800aa430 con 0x7fe48414b1e0 2023-11-26T07:19:06.163 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:06.160+0000 7fe47bfff640 1 -- 172.21.15.143:0/3142426885 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe4800aa590 con 0x7fe48414b1e0 2023-11-26T07:19:06.163 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:06.160+0000 7fe47bfff640 1 -- 172.21.15.143:0/3142426885 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe4800aa920 con 0x7fe48414b1e0 2023-11-26T07:19:06.164 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:06.160+0000 7fe48b91c640 1 -- 172.21.15.143:0/3142426885 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fe48414b1e0 msgr2=0x7fe4840cf280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:19:06.164 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:06.160+0000 7fe48b91c640 1 --2- 172.21.15.143:0/3142426885 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fe48414b1e0 0x7fe4840cf280 secure :-1 s=READY pgs=444 cs=0 l=1 rev1=1 crypto rx=0x7fe484072110 tx=0x7fe48001c4d0 comp rx=0 tx=0).stop 2023-11-26T07:19:06.164 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:06.160+0000 7fe48b91c640 1 -- 172.21.15.143:0/3142426885 shutdown_connections 2023-11-26T07:19:06.164 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:06.160+0000 7fe48b91c640 1 --2- 172.21.15.143:0/3142426885 >> v2:172.21.15.188:3300/0 conn(0x7fe484070fd0 0x7fe484162510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:19:06.164 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:06.160+0000 7fe48b91c640 1 --2- 172.21.15.143:0/3142426885 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fe48414b1e0 0x7fe4840cf280 unknown :-1 s=CLOSED pgs=444 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:19:06.164 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:06.160+0000 7fe48b91c640 1 -- 172.21.15.143:0/3142426885 >> 172.21.15.143:0/3142426885 conn(0x7fe48406a890 msgr2=0x7fe48406aca0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:19:06.164 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:06.160+0000 7fe48b91c640 1 -- 172.21.15.143:0/3142426885 shutdown_connections 2023-11-26T07:19:06.164 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:06.160+0000 7fe48b91c640 1 -- 172.21.15.143:0/3142426885 wait complete. 2023-11-26T07:19:06.165 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:06.164+0000 7fe48b91c640 1 Processor -- start 2023-11-26T07:19:06.165 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:06.164+0000 7fe48b91c640 1 -- start start 2023-11-26T07:19:06.168 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:06.164+0000 7fe48b91c640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fe484070fd0 0x7fe484161300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:19:06.168 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:06.164+0000 7fe48b91c640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fe4840cf7c0 0x7fe4840ac3a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:19:06.168 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:06.164+0000 7fe48b91c640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fe48414b1e0 0x7fe4840b0250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:19:06.168 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:06.164+0000 7fe48b91c640 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe484161840 con 0x7fe484070fd0 2023-11-26T07:19:06.168 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:06.164+0000 7fe48b91c640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe4840acc80 con 0x7fe4840cf7c0 2023-11-26T07:19:06.168 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:06.164+0000 7fe48b91c640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe4840acf60 con 0x7fe48414b1e0 2023-11-26T07:19:06.168 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:06.164+0000 7fe4896b8640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fe484070fd0 0x7fe484161300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:19:06.168 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:06.164+0000 7fe4896b8640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fe484070fd0 0x7fe484161300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:48466/0 (socket says 172.21.15.143:48466) 2023-11-26T07:19:06.168 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:06.164+0000 7fe4896b8640 1 -- 172.21.15.143:0/2965949151 learned_addr learned my addr 172.21.15.143:0/2965949151 (peer_addr_for_me v2:172.21.15.143:0/0) 2023-11-26T07:19:06.168 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:06.164+0000 7fe4896b8640 1 -- 172.21.15.143:0/2965949151 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fe48414b1e0 msgr2=0x7fe4840b0250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:19:06.169 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:06.164+0000 7fe4896b8640 1 --2- 172.21.15.143:0/2965949151 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fe48414b1e0 0x7fe4840b0250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:19:06.169 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:06.164+0000 7fe4896b8640 1 -- 172.21.15.143:0/2965949151 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fe4840cf7c0 msgr2=0x7fe4840ac3a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:19:06.169 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:06.164+0000 7fe4896b8640 1 --2- 172.21.15.143:0/2965949151 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fe4840cf7c0 0x7fe4840ac3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:19:06.169 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:06.164+0000 7fe4896b8640 1 -- 172.21.15.143:0/2965949151 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe4800094d0 con 0x7fe484070fd0 2023-11-26T07:19:06.169 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:06.164+0000 7fe4896b8640 1 --2- 172.21.15.143:0/2965949151 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fe484070fd0 0x7fe484161300 secure :-1 s=READY pgs=445 cs=0 l=1 rev1=1 crypto rx=0x7fe4800ac040 tx=0x7fe48001ad10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:19:06.169 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:06.164+0000 7fe479ffb640 1 -- 172.21.15.143:0/2965949151 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe4800b3800 con 0x7fe484070fd0 2023-11-26T07:19:06.169 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:06.164+0000 7fe479ffb640 1 -- 172.21.15.143:0/2965949151 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe4800b3980 con 0x7fe484070fd0 2023-11-26T07:19:06.169 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:06.164+0000 7fe479ffb640 1 -- 172.21.15.143:0/2965949151 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe4800b3d30 con 0x7fe484070fd0 2023-11-26T07:19:06.169 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:06.164+0000 7fe48b91c640 1 -- 172.21.15.143:0/2965949151 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe4840ad240 con 0x7fe484070fd0 2023-11-26T07:19:06.169 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:06.164+0000 7fe48b91c640 1 -- 172.21.15.143:0/2965949151 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe4840ac8e0 con 0x7fe484070fd0 2023-11-26T07:19:06.170 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:06.168+0000 7fe48b91c640 1 -- 172.21.15.143:0/2965949151 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe484067680 con 0x7fe484070fd0 2023-11-26T07:19:06.170 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:06.168+0000 7fe479ffb640 1 -- 172.21.15.143:0/2965949151 <== mon.0 v2:172.21.15.143:3300/0 4 ==== mgrmap(e 5) v1 ==== 85852+0+0 (secure 0 0 0) 0x7fe4800b2380 con 0x7fe484070fd0 2023-11-26T07:19:06.171 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:06.168+0000 7fe479ffb640 1 --2- 172.21.15.143:0/2965949151 >> [v2:172.21.15.143:6832/2150557544,v1:172.21.15.143:6833/2150557544] conn(0x7fe468066470 0x7fe468068930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:19:06.171 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:06.168+0000 7fe479ffb640 1 -- 172.21.15.143:0/2965949151 <== mon.0 v2:172.21.15.143:3300/0 5 ==== osd_map(33..33 src has 1..33) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fe48011e3d0 con 0x7fe484070fd0 2023-11-26T07:19:06.171 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:06.168+0000 7fe488eb7640 1 --2- 172.21.15.143:0/2965949151 >> [v2:172.21.15.143:6832/2150557544,v1:172.21.15.143:6833/2150557544] conn(0x7fe468066470 0x7fe468068930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:19:06.171 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:06.168+0000 7fe488eb7640 1 --2- 172.21.15.143:0/2965949151 >> [v2:172.21.15.143:6832/2150557544,v1:172.21.15.143:6833/2150557544] conn(0x7fe468066470 0x7fe468068930 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7fe474000f70 tx=0x7fe4740079c0 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:19:06.175 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:06.172+0000 7fe479ffb640 1 -- 172.21.15.143:0/2965949151 <== mon.0 v2:172.21.15.143:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172649 (secure 0 0 0) 0x7fe4800eb8f0 con 0x7fe484070fd0 2023-11-26T07:19:06.356 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:06.352+0000 7fe48b91c640 1 -- 172.21.15.143:0/2965949151 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "fs flag set", "flag_name": "enable_multiple", "val": "true", "yes_i_really_mean_it": true} v 0) v1 -- 0x7fe4840a8fb0 con 0x7fe484070fd0 2023-11-26T07:19:06.441 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:06.440+0000 7fe479ffb640 1 -- 172.21.15.143:0/2965949151 <== mon.0 v2:172.21.15.143:3300/0 7 ==== mon_command_ack([{"prefix": "fs flag set", "flag_name": "enable_multiple", "val": "true", "yes_i_really_mean_it": true}]=0 v17) v1 ==== 136+0+0 (secure 0 0 0) 0x7fe4800eef10 con 0x7fe484070fd0 2023-11-26T07:19:06.444 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:06.440+0000 7fe48b91c640 1 -- 172.21.15.143:0/2965949151 >> [v2:172.21.15.143:6832/2150557544,v1:172.21.15.143:6833/2150557544] conn(0x7fe468066470 msgr2=0x7fe468068930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:19:06.445 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:06.440+0000 7fe48b91c640 1 --2- 172.21.15.143:0/2965949151 >> [v2:172.21.15.143:6832/2150557544,v1:172.21.15.143:6833/2150557544] conn(0x7fe468066470 0x7fe468068930 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7fe474000f70 tx=0x7fe4740079c0 comp rx=0 tx=0).stop 2023-11-26T07:19:06.445 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:06.440+0000 7fe48b91c640 1 -- 172.21.15.143:0/2965949151 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fe484070fd0 msgr2=0x7fe484161300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:19:06.445 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:06.440+0000 7fe48b91c640 1 --2- 172.21.15.143:0/2965949151 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fe484070fd0 0x7fe484161300 secure :-1 s=READY pgs=445 cs=0 l=1 rev1=1 crypto rx=0x7fe4800ac040 tx=0x7fe48001ad10 comp rx=0 tx=0).stop 2023-11-26T07:19:06.445 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:06.440+0000 7fe48b91c640 1 -- 172.21.15.143:0/2965949151 shutdown_connections 2023-11-26T07:19:06.445 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:06.440+0000 7fe48b91c640 1 --2- 172.21.15.143:0/2965949151 >> [v2:172.21.15.143:6832/2150557544,v1:172.21.15.143:6833/2150557544] conn(0x7fe468066470 0x7fe468068930 unknown :-1 s=CLOSED pgs=233 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:19:06.445 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:06.440+0000 7fe48b91c640 1 --2- 172.21.15.143:0/2965949151 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fe48414b1e0 0x7fe4840b0250 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:19:06.445 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:06.440+0000 7fe48b91c640 1 --2- 172.21.15.143:0/2965949151 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fe4840cf7c0 0x7fe4840ac3a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:19:06.445 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:06.440+0000 7fe48b91c640 1 --2- 172.21.15.143:0/2965949151 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fe484070fd0 0x7fe484161300 unknown :-1 s=CLOSED pgs=445 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:19:06.445 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:06.440+0000 7fe48b91c640 1 -- 172.21.15.143:0/2965949151 >> 172.21.15.143:0/2965949151 conn(0x7fe48406a890 msgr2=0x7fe48406ac70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:19:06.445 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:06.440+0000 7fe48b91c640 1 -- 172.21.15.143:0/2965949151 shutdown_connections 2023-11-26T07:19:06.445 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:06.440+0000 7fe48b91c640 1 -- 172.21.15.143:0/2965949151 wait complete. 2023-11-26T07:19:06.463 DEBUG:teuthology.orchestra.run.smithi143:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs volume ls 2023-11-26T07:19:06.600 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:06.596+0000 7f588ea79640 1 Processor -- start 2023-11-26T07:19:06.601 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:06.600+0000 7f588ea79640 1 -- start start 2023-11-26T07:19:06.603 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:06.600+0000 7f588ea79640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f5888143ac0 0x7f5888070370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:19:06.603 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:06.600+0000 7f588ea79640 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7f58880708b0 0x7f58880cf280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:19:06.603 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:06.600+0000 7f588ea79640 1 -- --> v1:172.21.15.143:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f58880cfbf0 con 0x7f58880cf7c0 2023-11-26T07:19:06.603 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:06.600+0000 7f588ea79640 1 -- --> v2:172.21.15.143:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5888143ea0 con 0x7f58880708b0 2023-11-26T07:19:06.603 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:06.600+0000 7f588ea79640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f58880cc730 con 0x7f5888143ac0 2023-11-26T07:19:06.603 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:06.600+0000 7f5887fff640 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7f58880708b0 0x7f58880cf280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:19:06.603 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:06.600+0000 7f5887fff640 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7f58880708b0 0x7f58880cf280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:48468/0 (socket says 172.21.15.143:48468) 2023-11-26T07:19:06.603 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:06.600+0000 7f588d016640 1 --1- >> v1:172.21.15.143:6789/0 conn(0x7f58880cf7c0 0x7f58880b1ef0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.143:6789/0 says I am v1:172.21.15.143:38226/0 (socket says 172.21.15.143:38226) 2023-11-26T07:19:06.603 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:06.600+0000 7f5887fff640 1 -- 172.21.15.143:0/2794294923 learned_addr learned my addr 172.21.15.143:0/2794294923 (peer_addr_for_me v2:172.21.15.143:0/0) 2023-11-26T07:19:06.603 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:06.600+0000 7f588c815640 1 --2- 172.21.15.143:0/2794294923 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f5888143ac0 0x7f5888070370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:19:06.604 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:06.600+0000 7f5887fff640 1 -- 172.21.15.143:0/2794294923 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f5888143ac0 msgr2=0x7f5888070370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-26T07:19:06.604 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:06.600+0000 7f5887fff640 1 --2- 172.21.15.143:0/2794294923 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f5888143ac0 0x7f5888070370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:19:06.604 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:06.600+0000 7f5887fff640 1 -- 172.21.15.143:0/2794294923 >> v1:172.21.15.143:6789/0 conn(0x7f58880cf7c0 legacy=0x7f58880b1ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:19:06.604 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:06.600+0000 7f5887fff640 1 -- 172.21.15.143:0/2794294923 --> v2:172.21.15.143:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5888070ee0 con 0x7f58880708b0 2023-11-26T07:19:06.604 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:06.600+0000 7f58877fe640 1 -- 172.21.15.143:0/2794294923 <== mon.0 v1:172.21.15.143:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1277350885 0 0) 0x7f58880cfbf0 con 0x7f58880cf7c0 2023-11-26T07:19:06.606 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:06.604+0000 7f5887fff640 1 --2- 172.21.15.143:0/2794294923 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f58880708b0 0x7f58880cf280 secure :-1 s=READY pgs=447 cs=0 l=1 rev1=1 crypto rx=0x7f5874009870 tx=0x7f587401c540 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1deeafffcb0f405b server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:19:06.606 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:06.604+0000 7f58877fe640 1 -- 172.21.15.143:0/2794294923 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f58740ab650 con 0x7f58880708b0 2023-11-26T07:19:06.606 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:06.604+0000 7f58877fe640 1 -- 172.21.15.143:0/2794294923 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f58740ab7b0 con 0x7f58880708b0 2023-11-26T07:19:06.606 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:06.604+0000 7f58877fe640 1 -- 172.21.15.143:0/2794294923 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f58740abb40 con 0x7f58880708b0 2023-11-26T07:19:06.607 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:06.604+0000 7f588ea79640 1 -- 172.21.15.143:0/2794294923 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f58880708b0 msgr2=0x7f58880cf280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:19:06.607 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:06.604+0000 7f588ea79640 1 --2- 172.21.15.143:0/2794294923 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f58880708b0 0x7f58880cf280 secure :-1 s=READY pgs=447 cs=0 l=1 rev1=1 crypto rx=0x7f5874009870 tx=0x7f587401c540 comp rx=0 tx=0).stop 2023-11-26T07:19:06.607 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:06.604+0000 7f588ea79640 1 -- 172.21.15.143:0/2794294923 shutdown_connections 2023-11-26T07:19:06.607 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:06.604+0000 7f588ea79640 1 --2- 172.21.15.143:0/2794294923 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f58880708b0 0x7f58880cf280 unknown :-1 s=CLOSED pgs=447 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:19:06.607 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:06.604+0000 7f588ea79640 1 --2- 172.21.15.143:0/2794294923 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f5888143ac0 0x7f5888070370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:19:06.607 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:06.604+0000 7f588ea79640 1 -- 172.21.15.143:0/2794294923 >> 172.21.15.143:0/2794294923 conn(0x7f588806a890 msgr2=0x7f588806aca0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:19:06.607 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:06.604+0000 7f588ea79640 1 -- 172.21.15.143:0/2794294923 shutdown_connections 2023-11-26T07:19:06.607 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:06.604+0000 7f588ea79640 1 -- 172.21.15.143:0/2794294923 wait complete. 2023-11-26T07:19:06.608 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:06.604+0000 7f588ea79640 1 Processor -- start 2023-11-26T07:19:06.608 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:06.604+0000 7f588ea79640 1 -- start start 2023-11-26T07:19:06.610 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:06.608+0000 7f588ea79640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f58880708b0 0x7f58880cd5a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:19:06.611 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:06.608+0000 7f588ea79640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f58880cf7c0 0x7f58881592f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:19:06.611 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:06.608+0000 7f588ea79640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f5888143ac0 0x7f58880a8cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:19:06.611 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:06.608+0000 7f588ea79640 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f58880cddb0 con 0x7f58880708b0 2023-11-26T07:19:06.611 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:06.608+0000 7f588ea79640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f588813b5c0 con 0x7f5888143ac0 2023-11-26T07:19:06.611 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:06.608+0000 7f588ea79640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f58880cdae0 con 0x7f58880cf7c0 2023-11-26T07:19:06.611 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:06.608+0000 7f588c815640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f58880708b0 0x7f58880cd5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:19:06.611 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:06.608+0000 7f588c815640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f58880708b0 0x7f58880cd5a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:48484/0 (socket says 172.21.15.143:48484) 2023-11-26T07:19:06.611 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:06.608+0000 7f588c815640 1 -- 172.21.15.143:0/573981006 learned_addr learned my addr 172.21.15.143:0/573981006 (peer_addr_for_me v2:172.21.15.143:0/0) 2023-11-26T07:19:06.611 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:06.608+0000 7f5887fff640 1 --2- 172.21.15.143:0/573981006 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f58880cf7c0 0x7f58881592f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:19:06.611 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:06.608+0000 7f588d016640 1 --2- 172.21.15.143:0/573981006 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f5888143ac0 0x7f58880a8cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:19:06.612 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:06.608+0000 7f588c815640 1 -- 172.21.15.143:0/573981006 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f58880cf7c0 msgr2=0x7f58881592f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:19:06.612 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:06.608+0000 7f588c815640 1 --2- 172.21.15.143:0/573981006 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f58880cf7c0 0x7f58881592f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:19:06.612 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:06.608+0000 7f588c815640 1 -- 172.21.15.143:0/573981006 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f5888143ac0 msgr2=0x7f58880a8cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:19:06.612 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:06.608+0000 7f588c815640 1 --2- 172.21.15.143:0/573981006 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f5888143ac0 0x7f58880a8cc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:19:06.612 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:06.608+0000 7f588c815640 1 -- 172.21.15.143:0/573981006 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f58740094d0 con 0x7f58880708b0 2023-11-26T07:19:06.612 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:06.608+0000 7f588d016640 1 --2- 172.21.15.143:0/573981006 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f5888143ac0 0x7f58880a8cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-26T07:19:06.612 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:06.608+0000 7f5887fff640 1 --2- 172.21.15.143:0/573981006 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f58880cf7c0 0x7f58881592f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-26T07:19:06.612 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:06.608+0000 7f588c815640 1 --2- 172.21.15.143:0/573981006 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f58880708b0 0x7f58880cd5a0 secure :-1 s=READY pgs=448 cs=0 l=1 rev1=1 crypto rx=0x7f587c00e930 tx=0x7f587c00edf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:19:06.612 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:06.608+0000 7f58857fa640 1 -- 172.21.15.143:0/573981006 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f587c009690 con 0x7f58880708b0 2023-11-26T07:19:06.612 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:06.608+0000 7f58857fa640 1 -- 172.21.15.143:0/573981006 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f587c00ca00 con 0x7f58880708b0 2023-11-26T07:19:06.612 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:06.608+0000 7f58857fa640 1 -- 172.21.15.143:0/573981006 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f587c00cd70 con 0x7f58880708b0 2023-11-26T07:19:06.612 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:06.608+0000 7f588ea79640 1 -- 172.21.15.143:0/573981006 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f588813b860 con 0x7f58880708b0 2023-11-26T07:19:06.613 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:06.608+0000 7f588ea79640 1 -- 172.21.15.143:0/573981006 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f588813be20 con 0x7f58880708b0 2023-11-26T07:19:06.613 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:06.608+0000 7f588ea79640 1 -- 172.21.15.143:0/573981006 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5888067680 con 0x7f58880708b0 2023-11-26T07:19:06.613 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:06.608+0000 7f58857fa640 1 -- 172.21.15.143:0/573981006 <== mon.0 v2:172.21.15.143:3300/0 4 ==== mgrmap(e 5) v1 ==== 85852+0+0 (secure 0 0 0) 0x7f587c013470 con 0x7f58880708b0 2023-11-26T07:19:06.614 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:06.612+0000 7f58857fa640 1 --2- 172.21.15.143:0/573981006 >> [v2:172.21.15.143:6832/2150557544,v1:172.21.15.143:6833/2150557544] conn(0x7f586c066470 0x7f586c068930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:19:06.614 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:06.612+0000 7f58857fa640 1 -- 172.21.15.143:0/573981006 <== mon.0 v2:172.21.15.143:3300/0 5 ==== osd_map(33..33 src has 1..33) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f587c084010 con 0x7f58880708b0 2023-11-26T07:19:06.614 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:06.612+0000 7f5887fff640 1 --2- 172.21.15.143:0/573981006 >> [v2:172.21.15.143:6832/2150557544,v1:172.21.15.143:6833/2150557544] conn(0x7f586c066470 0x7f586c068930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:19:06.614 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:06.612+0000 7f5887fff640 1 --2- 172.21.15.143:0/573981006 >> [v2:172.21.15.143:6832/2150557544,v1:172.21.15.143:6833/2150557544] conn(0x7f586c066470 0x7f586c068930 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7f58740af000 tx=0x7f58740b6040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:19:06.617 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:06.616+0000 7f58857fa640 1 -- 172.21.15.143:0/573981006 <== mon.0 v2:172.21.15.143:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172649 (secure 0 0 0) 0x7f587c051690 con 0x7f58880708b0 2023-11-26T07:19:06.788 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:06.784+0000 7f588ea79640 1 -- 172.21.15.143:0/573981006 --> [v2:172.21.15.143:6832/2150557544,v1:172.21.15.143:6833/2150557544] -- mgr_command(tid 0: {"prefix": "fs volume ls", "target": ["mon-mgr", ""]}) v1 -- 0x7f5888071c30 con 0x7f586c066470 2023-11-26T07:19:06.790 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:06.788+0000 7f58857fa640 1 -- 172.21.15.143:0/573981006 <== mgr.4100 v2:172.21.15.143:6832/2150557544 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+41 (secure 0 0 0) 0x7f58740ab450 con 0x7f586c066470 2023-11-26T07:19:06.790 INFO:teuthology.orchestra.run.smithi143.stdout:[ 2023-11-26T07:19:06.790 INFO:teuthology.orchestra.run.smithi143.stdout: { 2023-11-26T07:19:06.790 INFO:teuthology.orchestra.run.smithi143.stdout: "name": "cephfs" 2023-11-26T07:19:06.790 INFO:teuthology.orchestra.run.smithi143.stdout: } 2023-11-26T07:19:06.790 INFO:teuthology.orchestra.run.smithi143.stdout:] 2023-11-26T07:19:06.792 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:06.788+0000 7f588ea79640 1 -- 172.21.15.143:0/573981006 >> [v2:172.21.15.143:6832/2150557544,v1:172.21.15.143:6833/2150557544] conn(0x7f586c066470 msgr2=0x7f586c068930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:19:06.792 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:06.788+0000 7f588ea79640 1 --2- 172.21.15.143:0/573981006 >> [v2:172.21.15.143:6832/2150557544,v1:172.21.15.143:6833/2150557544] conn(0x7f586c066470 0x7f586c068930 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7f58740af000 tx=0x7f58740b6040 comp rx=0 tx=0).stop 2023-11-26T07:19:06.792 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:06.788+0000 7f588ea79640 1 -- 172.21.15.143:0/573981006 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f58880708b0 msgr2=0x7f58880cd5a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:19:06.792 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:06.788+0000 7f588ea79640 1 --2- 172.21.15.143:0/573981006 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f58880708b0 0x7f58880cd5a0 secure :-1 s=READY pgs=448 cs=0 l=1 rev1=1 crypto rx=0x7f587c00e930 tx=0x7f587c00edf0 comp rx=0 tx=0).stop 2023-11-26T07:19:06.792 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:06.788+0000 7f588ea79640 1 -- 172.21.15.143:0/573981006 shutdown_connections 2023-11-26T07:19:06.792 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:06.788+0000 7f588ea79640 1 --2- 172.21.15.143:0/573981006 >> [v2:172.21.15.143:6832/2150557544,v1:172.21.15.143:6833/2150557544] conn(0x7f586c066470 0x7f586c068930 unknown :-1 s=CLOSED pgs=234 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:19:06.792 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:06.788+0000 7f588ea79640 1 --2- 172.21.15.143:0/573981006 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f5888143ac0 0x7f58880a8cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:19:06.793 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:06.788+0000 7f588ea79640 1 --2- 172.21.15.143:0/573981006 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f58880cf7c0 0x7f58881592f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:19:06.793 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:06.788+0000 7f588ea79640 1 --2- 172.21.15.143:0/573981006 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f58880708b0 0x7f58880cd5a0 unknown :-1 s=CLOSED pgs=448 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:19:06.793 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:06.788+0000 7f588ea79640 1 -- 172.21.15.143:0/573981006 >> 172.21.15.143:0/573981006 conn(0x7f588806a890 msgr2=0x7f588806c9d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:19:06.793 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:06.788+0000 7f588ea79640 1 -- 172.21.15.143:0/573981006 shutdown_connections 2023-11-26T07:19:06.793 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:06.788+0000 7f588ea79640 1 -- 172.21.15.143:0/573981006 wait complete. 2023-11-26T07:19:06.811 DEBUG:teuthology.orchestra.run.smithi143:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mon mon_allow_pool_delete True 2023-11-26T07:19:06.951 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:06.948+0000 7f7b7cafe640 1 Processor -- start 2023-11-26T07:19:06.951 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:06.948+0000 7f7b7cafe640 1 -- start start 2023-11-26T07:19:06.953 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:06.948+0000 7f7b7cafe640 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7f7b7813e850 0x7f7b78140bc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:19:06.953 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:06.948+0000 7f7b7cafe640 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7f7b780cdb90 0x7f7b780ce000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:19:06.953 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:06.948+0000 7f7b7cafe640 1 -- --> v1:172.21.15.143:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7b780ce900 con 0x7f7b781407e0 2023-11-26T07:19:06.953 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:06.948+0000 7f7b7cafe640 1 -- --> v2:172.21.15.143:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7b78070410 con 0x7f7b780cdb90 2023-11-26T07:19:06.953 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:06.948+0000 7f7b7cafe640 1 -- --> v2:172.21.15.188:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7b780706f0 con 0x7f7b7813e850 2023-11-26T07:19:06.953 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:06.952+0000 7f7b76d9d640 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7f7b780cdb90 0x7f7b780ce000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:19:06.953 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:06.952+0000 7f7b7659c640 1 --1- >> v1:172.21.15.143:6789/0 conn(0x7f7b781407e0 0x7f7b7813df90 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.143:6789/0 says I am v1:172.21.15.143:38230/0 (socket says 172.21.15.143:38230) 2023-11-26T07:19:06.954 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:06.952+0000 7f7b7659c640 1 -- 172.21.15.143:0/3256094398 learned_addr learned my addr 172.21.15.143:0/3256094398 (peer_addr_for_me v1:172.21.15.143:0/0) 2023-11-26T07:19:06.954 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:06.952+0000 7f7b6ffff640 1 --2- 172.21.15.143:0/3256094398 >> v2:172.21.15.188:3300/0 conn(0x7f7b7813e850 0x7f7b78140bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:19:06.954 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:06.952+0000 7f7b76d9d640 1 -- 172.21.15.143:0/3256094398 >> v2:172.21.15.188:3300/0 conn(0x7f7b7813e850 msgr2=0x7f7b78140bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-26T07:19:06.954 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:06.952+0000 7f7b76d9d640 1 --2- 172.21.15.143:0/3256094398 >> v2:172.21.15.188:3300/0 conn(0x7f7b7813e850 0x7f7b78140bc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:19:06.954 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:06.952+0000 7f7b76d9d640 1 -- 172.21.15.143:0/3256094398 >> v1:172.21.15.143:6789/0 conn(0x7f7b781407e0 legacy=0x7f7b7813df90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:19:06.954 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:06.952+0000 7f7b76d9d640 1 -- 172.21.15.143:0/3256094398 --> v2:172.21.15.143:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7b780709d0 con 0x7f7b780cdb90 2023-11-26T07:19:06.954 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:06.952+0000 7f7b6ffff640 1 --2- 172.21.15.143:0/3256094398 >> v2:172.21.15.188:3300/0 conn(0x7f7b7813e850 0x7f7b78140bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-26T07:19:06.954 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:06.952+0000 7f7b75d9b640 1 -- 172.21.15.143:0/3256094398 <== mon.0 v1:172.21.15.143:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1508729550 0 0) 0x7f7b780ce900 con 0x7f7b781407e0 2023-11-26T07:19:06.956 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:06.952+0000 7f7b76d9d640 1 --2- 172.21.15.143:0/3256094398 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f7b780cdb90 0x7f7b780ce000 secure :-1 s=READY pgs=450 cs=0 l=1 rev1=1 crypto rx=0x7f7b780cd2b0 tx=0x7f7b7001c880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=172ddfd72cce5439 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:19:06.956 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:06.952+0000 7f7b75d9b640 1 -- 172.21.15.143:0/3256094398 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7b700a5860 con 0x7f7b780cdb90 2023-11-26T07:19:06.956 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:06.952+0000 7f7b75d9b640 1 -- 172.21.15.143:0/3256094398 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7b700a59c0 con 0x7f7b780cdb90 2023-11-26T07:19:06.956 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:06.952+0000 7f7b75d9b640 1 -- 172.21.15.143:0/3256094398 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7b700a5d50 con 0x7f7b780cdb90 2023-11-26T07:19:06.957 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:06.956+0000 7f7b7cafe640 1 -- 172.21.15.143:0/3256094398 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f7b780cdb90 msgr2=0x7f7b780ce000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:19:06.957 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:06.956+0000 7f7b7cafe640 1 --2- 172.21.15.143:0/3256094398 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f7b780cdb90 0x7f7b780ce000 secure :-1 s=READY pgs=450 cs=0 l=1 rev1=1 crypto rx=0x7f7b780cd2b0 tx=0x7f7b7001c880 comp rx=0 tx=0).stop 2023-11-26T07:19:06.957 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:06.956+0000 7f7b7cafe640 1 -- 172.21.15.143:0/3256094398 shutdown_connections 2023-11-26T07:19:06.957 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:06.956+0000 7f7b7cafe640 1 --2- 172.21.15.143:0/3256094398 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f7b780cdb90 0x7f7b780ce000 unknown :-1 s=CLOSED pgs=450 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:19:06.957 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:06.956+0000 7f7b7cafe640 1 --2- 172.21.15.143:0/3256094398 >> v2:172.21.15.188:3300/0 conn(0x7f7b7813e850 0x7f7b78140bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:19:06.958 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:06.956+0000 7f7b7cafe640 1 -- 172.21.15.143:0/3256094398 >> 172.21.15.143:0/3256094398 conn(0x7f7b780b0bf0 msgr2=0x7f7b780b1000 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:19:06.958 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:06.956+0000 7f7b7cafe640 1 -- 172.21.15.143:0/3256094398 shutdown_connections 2023-11-26T07:19:06.958 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:06.956+0000 7f7b7cafe640 1 -- 172.21.15.143:0/3256094398 wait complete. 2023-11-26T07:19:06.959 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:06.956+0000 7f7b7cafe640 1 Processor -- start 2023-11-26T07:19:06.959 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:06.956+0000 7f7b7cafe640 1 -- start start 2023-11-26T07:19:06.960 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:06.956+0000 7f7b7cafe640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f7b780cdb90 0x7f7b7815c2d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:19:06.961 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:06.956+0000 7f7b7cafe640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f7b7813e850 0x7f7b7815a790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:19:06.961 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:06.956+0000 7f7b7cafe640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f7b781407e0 0x7f7b781426c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:19:06.961 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:06.956+0000 7f7b7cafe640 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7b78157210 con 0x7f7b780cdb90 2023-11-26T07:19:06.961 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:06.956+0000 7f7b7cafe640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7b781574f0 con 0x7f7b7813e850 2023-11-26T07:19:06.961 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:06.956+0000 7f7b7cafe640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7b7815acd0 con 0x7f7b781407e0 2023-11-26T07:19:06.961 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:06.956+0000 7f7b7659c640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f7b780cdb90 0x7f7b7815c2d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:19:06.961 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:06.956+0000 7f7b7659c640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f7b780cdb90 0x7f7b7815c2d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:48500/0 (socket says 172.21.15.143:48500) 2023-11-26T07:19:06.961 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:06.956+0000 7f7b7659c640 1 -- 172.21.15.143:0/257178871 learned_addr learned my addr 172.21.15.143:0/257178871 (peer_addr_for_me v2:172.21.15.143:0/0) 2023-11-26T07:19:06.961 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:06.956+0000 7f7b76d9d640 1 --2- 172.21.15.143:0/257178871 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f7b781407e0 0x7f7b781426c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:19:06.961 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:06.956+0000 7f7b6ffff640 1 --2- 172.21.15.143:0/257178871 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f7b7813e850 0x7f7b7815a790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:19:06.961 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:06.956+0000 7f7b7659c640 1 -- 172.21.15.143:0/257178871 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f7b781407e0 msgr2=0x7f7b781426c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:19:06.961 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:06.960+0000 7f7b7659c640 1 --2- 172.21.15.143:0/257178871 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f7b781407e0 0x7f7b781426c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:19:06.962 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:06.960+0000 7f7b7659c640 1 -- 172.21.15.143:0/257178871 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f7b7813e850 msgr2=0x7f7b7815a790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:19:06.962 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:06.960+0000 7f7b7659c640 1 --2- 172.21.15.143:0/257178871 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f7b7813e850 0x7f7b7815a790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:19:06.962 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:06.960+0000 7f7b7659c640 1 -- 172.21.15.143:0/257178871 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7b70009a70 con 0x7f7b780cdb90 2023-11-26T07:19:06.962 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:06.960+0000 7f7b6ffff640 1 --2- 172.21.15.143:0/257178871 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f7b7813e850 0x7f7b7815a790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-26T07:19:06.962 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:06.960+0000 7f7b7659c640 1 --2- 172.21.15.143:0/257178871 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f7b780cdb90 0x7f7b7815c2d0 secure :-1 s=READY pgs=451 cs=0 l=1 rev1=1 crypto rx=0x7f7b6800c4b0 tx=0x7f7b6800c970 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:19:06.963 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:06.960+0000 7f7b6effd640 1 -- 172.21.15.143:0/257178871 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7b680074e0 con 0x7f7b780cdb90 2023-11-26T07:19:06.963 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:06.960+0000 7f7b6effd640 1 -- 172.21.15.143:0/257178871 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7b68007660 con 0x7f7b780cdb90 2023-11-26T07:19:06.963 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:06.960+0000 7f7b7cafe640 1 -- 172.21.15.143:0/257178871 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7b7815af70 con 0x7f7b780cdb90 2023-11-26T07:19:06.963 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:06.960+0000 7f7b6effd640 1 -- 172.21.15.143:0/257178871 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7b68004a30 con 0x7f7b780cdb90 2023-11-26T07:19:06.963 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:06.960+0000 7f7b7cafe640 1 -- 172.21.15.143:0/257178871 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7b7815b530 con 0x7f7b780cdb90 2023-11-26T07:19:06.963 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:06.960+0000 7f7b7cafe640 1 -- 172.21.15.143:0/257178871 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7b78067680 con 0x7f7b780cdb90 2023-11-26T07:19:06.963 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:06.960+0000 7f7b6effd640 1 -- 172.21.15.143:0/257178871 <== mon.0 v2:172.21.15.143:3300/0 4 ==== mgrmap(e 5) v1 ==== 85852+0+0 (secure 0 0 0) 0x7f7b68004b90 con 0x7f7b780cdb90 2023-11-26T07:19:06.964 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:06.960+0000 7f7b6effd640 1 --2- 172.21.15.143:0/257178871 >> [v2:172.21.15.143:6832/2150557544,v1:172.21.15.143:6833/2150557544] conn(0x7f7b580664a0 0x7f7b58068960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:19:06.964 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:06.960+0000 7f7b6effd640 1 -- 172.21.15.143:0/257178871 <== mon.0 v2:172.21.15.143:3300/0 5 ==== osd_map(33..33 src has 1..33) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f7b68083290 con 0x7f7b780cdb90 2023-11-26T07:19:06.964 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:06.960+0000 7f7b6ffff640 1 --2- 172.21.15.143:0/257178871 >> [v2:172.21.15.143:6832/2150557544,v1:172.21.15.143:6833/2150557544] conn(0x7f7b580664a0 0x7f7b58068960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:19:06.964 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:06.964+0000 7f7b6ffff640 1 --2- 172.21.15.143:0/257178871 >> [v2:172.21.15.143:6832/2150557544,v1:172.21.15.143:6833/2150557544] conn(0x7f7b580664a0 0x7f7b58068960 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7f7b78143600 tx=0x7f7b60008040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:19:06.967 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:06.964+0000 7f7b6effd640 1 -- 172.21.15.143:0/257178871 <== mon.0 v2:172.21.15.143:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172649 (secure 0 0 0) 0x7f7b68050720 con 0x7f7b780cdb90 2023-11-26T07:19:07.120 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:07.116+0000 7f7b7cafe640 1 -- 172.21.15.143:0/257178871 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command([{prefix=config set, name=mon_allow_pool_delete}] v 0) v1 -- 0x7f7b780cf9f0 con 0x7f7b780cdb90 2023-11-26T07:19:07.120 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:07.116+0000 7f7b6effd640 1 -- 172.21.15.143:0/257178871 <== mon.0 v2:172.21.15.143:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mon_allow_pool_delete}]=0 v10)=0 v10) v1 ==== 122+0+0 (secure 0 0 0) 0x7f7b68053d40 con 0x7f7b780cdb90 2023-11-26T07:19:07.122 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:07.120+0000 7f7b7cafe640 1 -- 172.21.15.143:0/257178871 >> [v2:172.21.15.143:6832/2150557544,v1:172.21.15.143:6833/2150557544] conn(0x7f7b580664a0 msgr2=0x7f7b58068960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:19:07.122 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:07.120+0000 7f7b7cafe640 1 --2- 172.21.15.143:0/257178871 >> [v2:172.21.15.143:6832/2150557544,v1:172.21.15.143:6833/2150557544] conn(0x7f7b580664a0 0x7f7b58068960 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7f7b78143600 tx=0x7f7b60008040 comp rx=0 tx=0).stop 2023-11-26T07:19:07.122 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:07.120+0000 7f7b7cafe640 1 -- 172.21.15.143:0/257178871 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f7b780cdb90 msgr2=0x7f7b7815c2d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:19:07.122 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:07.120+0000 7f7b7cafe640 1 --2- 172.21.15.143:0/257178871 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f7b780cdb90 0x7f7b7815c2d0 secure :-1 s=READY pgs=451 cs=0 l=1 rev1=1 crypto rx=0x7f7b6800c4b0 tx=0x7f7b6800c970 comp rx=0 tx=0).stop 2023-11-26T07:19:07.122 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:07.120+0000 7f7b7cafe640 1 -- 172.21.15.143:0/257178871 shutdown_connections 2023-11-26T07:19:07.122 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:07.120+0000 7f7b7cafe640 1 --2- 172.21.15.143:0/257178871 >> [v2:172.21.15.143:6832/2150557544,v1:172.21.15.143:6833/2150557544] conn(0x7f7b580664a0 0x7f7b58068960 unknown :-1 s=CLOSED pgs=235 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:19:07.122 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:07.120+0000 7f7b7cafe640 1 --2- 172.21.15.143:0/257178871 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f7b781407e0 0x7f7b781426c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:19:07.122 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:07.120+0000 7f7b7cafe640 1 --2- 172.21.15.143:0/257178871 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f7b7813e850 0x7f7b7815a790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:19:07.123 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:07.120+0000 7f7b7cafe640 1 --2- 172.21.15.143:0/257178871 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f7b780cdb90 0x7f7b7815c2d0 unknown :-1 s=CLOSED pgs=451 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:19:07.123 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:07.120+0000 7f7b7cafe640 1 -- 172.21.15.143:0/257178871 >> 172.21.15.143:0/257178871 conn(0x7f7b780b0bf0 msgr2=0x7f7b780b0fd0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:19:07.123 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:07.120+0000 7f7b7cafe640 1 -- 172.21.15.143:0/257178871 shutdown_connections 2023-11-26T07:19:07.123 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:07.120+0000 7f7b7cafe640 1 -- 172.21.15.143:0/257178871 wait complete. 2023-11-26T07:19:07.141 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-26T07:19:07.141 DEBUG:teuthology.orchestra.run.smithi143:> (cd /home/ubuntu/cephtest/mnt.0 && exec sudo mkdir -p ./volumes/subvol_grp78310/subvol12410) 2023-11-26T07:19:07.172 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-26T07:19:07.172 DEBUG:teuthology.orchestra.run.smithi143:> (cd /home/ubuntu/cephtest/mnt.0 && exec getfattr --only-values -n ceph.dir.layout.pool .) 2023-11-26T07:19:07.254 INFO:teuthology.orchestra.run.smithi143.stdout:cephfs_data 2023-11-26T07:19:07.255 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-26T07:19:07.255 DEBUG:teuthology.orchestra.run.smithi143:> (cd /home/ubuntu/cephtest/mnt.0 && exec sudo setfattr -n ceph.dir.layout.pool -v cephfs_data ./volumes/subvol_grp78310/subvol12410) 2023-11-26T07:19:07.322 DEBUG:teuthology.orchestra.run.smithi143:> set -ex 2023-11-26T07:19:07.322 DEBUG:teuthology.orchestra.run.smithi143:> sudo dd of=/home/ubuntu/cephtest/mnt.0/./volumes/subvol_grp78310/subvol12410/.meta 2023-11-26T07:19:07.392 DEBUG:teuthology.orchestra.run.smithi143:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume getpath cephfs subvol12410 subvol_grp78310 2023-11-26T07:19:07.584 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:07.580+0000 7f6895d57640 1 Processor -- start 2023-11-26T07:19:07.584 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:07.580+0000 7f6895d57640 1 -- start start 2023-11-26T07:19:07.585 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:07.584+0000 7f6895d57640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f689013e290 0x7f68900cf280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:19:07.585 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:07.584+0000 7f6895d57640 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7f689015f6b0 0x7f68900cd7b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:19:07.585 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:07.584+0000 7f6895d57640 1 -- --> v1:172.21.15.143:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f689006c520 con 0x7f68900cf7c0 2023-11-26T07:19:07.585 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:07.584+0000 7f6895d57640 1 -- --> v2:172.21.15.188:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f68900cfc40 con 0x7f689015f6b0 2023-11-26T07:19:07.586 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:07.584+0000 7f6895d57640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f689015fb40 con 0x7f689013e290 2023-11-26T07:19:07.586 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:07.584+0000 7f688effd640 1 --1- >> v1:172.21.15.143:6789/0 conn(0x7f68900cf7c0 0x7f689015efa0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.143:6789/0 says I am v1:172.21.15.143:38246/0 (socket says 172.21.15.143:38246) 2023-11-26T07:19:07.587 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:07.584+0000 7f688effd640 1 -- 172.21.15.143:0/473047006 learned_addr learned my addr 172.21.15.143:0/473047006 (peer_addr_for_me v1:172.21.15.143:0/0) 2023-11-26T07:19:07.587 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:07.584+0000 7f688ffff640 1 --2- 172.21.15.143:0/473047006 >> v2:172.21.15.188:3300/0 conn(0x7f689015f6b0 0x7f68900cd7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:19:07.587 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:07.584+0000 7f688f7fe640 1 --2- 172.21.15.143:0/473047006 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f689013e290 0x7f68900cf280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:19:07.587 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:07.584+0000 7f688e7fc640 1 -- 172.21.15.143:0/473047006 <== mon.0 v1:172.21.15.143:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1570115965 0 0) 0x7f689006c520 con 0x7f68900cf7c0 2023-11-26T07:19:07.587 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:07.584+0000 7f688e7fc640 1 -- 172.21.15.143:0/473047006 --> v1:172.21.15.143:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f6878003530 con 0x7f68900cf7c0 2023-11-26T07:19:07.587 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:07.584+0000 7f688ffff640 1 -- 172.21.15.143:0/473047006 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f689013e290 msgr2=0x7f68900cf280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-26T07:19:07.587 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:07.584+0000 7f688ffff640 1 --2- 172.21.15.143:0/473047006 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f689013e290 0x7f68900cf280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:19:07.587 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:07.584+0000 7f688ffff640 1 -- 172.21.15.143:0/473047006 >> v1:172.21.15.143:6789/0 conn(0x7f68900cf7c0 legacy=0x7f689015efa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:19:07.587 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:07.584+0000 7f688ffff640 1 -- 172.21.15.143:0/473047006 --> v2:172.21.15.188:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f689006c800 con 0x7f689015f6b0 2023-11-26T07:19:07.589 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:07.588+0000 7f688ffff640 1 --2- 172.21.15.143:0/473047006 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f689015f6b0 0x7f68900cd7b0 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7f6888009d90 tx=0x7f688801c920 comp rx=0 tx=0).ready entity=mon.1 client_cookie=29a833531ce2fd05 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:19:07.590 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:07.588+0000 7f688e7fc640 1 -- 172.21.15.143:0/473047006 <== mon.1 v2:172.21.15.188:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f68880a59e0 con 0x7f689015f6b0 2023-11-26T07:19:07.590 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:07.588+0000 7f688e7fc640 1 -- 172.21.15.143:0/473047006 <== mon.1 v2:172.21.15.188:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f68880a5b40 con 0x7f689015f6b0 2023-11-26T07:19:07.590 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:07.588+0000 7f688e7fc640 1 -- 172.21.15.143:0/473047006 <== mon.1 v2:172.21.15.188:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f68880ad440 con 0x7f689015f6b0 2023-11-26T07:19:07.590 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:07.588+0000 7f6895d57640 1 -- 172.21.15.143:0/473047006 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f689015f6b0 msgr2=0x7f68900cd7b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:19:07.591 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:07.588+0000 7f6895d57640 1 --2- 172.21.15.143:0/473047006 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f689015f6b0 0x7f68900cd7b0 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7f6888009d90 tx=0x7f688801c920 comp rx=0 tx=0).stop 2023-11-26T07:19:07.591 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:07.588+0000 7f6895d57640 1 -- 172.21.15.143:0/473047006 shutdown_connections 2023-11-26T07:19:07.591 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:07.588+0000 7f6895d57640 1 --2- 172.21.15.143:0/473047006 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f689015f6b0 0x7f68900cd7b0 unknown :-1 s=CLOSED pgs=198 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:19:07.591 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:07.588+0000 7f6895d57640 1 --2- 172.21.15.143:0/473047006 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f689013e290 0x7f68900cf280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:19:07.591 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:07.588+0000 7f6895d57640 1 -- 172.21.15.143:0/473047006 >> 172.21.15.143:0/473047006 conn(0x7f689006a890 msgr2=0x7f689006aca0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:19:07.591 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:07.588+0000 7f6895d57640 1 -- 172.21.15.143:0/473047006 shutdown_connections 2023-11-26T07:19:07.591 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:07.588+0000 7f6895d57640 1 -- 172.21.15.143:0/473047006 wait complete. 2023-11-26T07:19:07.591 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:07.588+0000 7f6895d57640 1 Processor -- start 2023-11-26T07:19:07.592 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:07.588+0000 7f6895d57640 1 -- start start 2023-11-26T07:19:07.593 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:07.592+0000 7f6895d57640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f68900cf7c0 0x7f68900a84d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:19:07.593 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:07.592+0000 7f6895d57640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f689013e290 0x7f689012a630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:19:07.593 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:07.592+0000 7f6895d57640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f689015f6b0 0x7f68900b0590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:19:07.593 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:07.592+0000 7f6895d57640 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f68900b0170 con 0x7f68900cf7c0 2023-11-26T07:19:07.594 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:07.592+0000 7f6895d57640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f68900af7f0 con 0x7f689015f6b0 2023-11-26T07:19:07.594 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:07.592+0000 7f6895d57640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f68900a8ab0 con 0x7f689013e290 2023-11-26T07:19:07.594 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:07.592+0000 7f688f7fe640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f68900cf7c0 0x7f68900a84d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:19:07.594 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:07.592+0000 7f688f7fe640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f68900cf7c0 0x7f68900a84d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:48510/0 (socket says 172.21.15.143:48510) 2023-11-26T07:19:07.594 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:07.592+0000 7f688f7fe640 1 -- 172.21.15.143:0/3921246302 learned_addr learned my addr 172.21.15.143:0/3921246302 (peer_addr_for_me v2:172.21.15.143:0/0) 2023-11-26T07:19:07.594 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:07.592+0000 7f688ffff640 1 --2- 172.21.15.143:0/3921246302 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f689015f6b0 0x7f68900b0590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:19:07.594 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:07.592+0000 7f688effd640 1 --2- 172.21.15.143:0/3921246302 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f689013e290 0x7f689012a630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:19:07.597 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:07.592+0000 7f688f7fe640 1 -- 172.21.15.143:0/3921246302 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f689013e290 msgr2=0x7f689012a630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:19:07.597 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:07.592+0000 7f688f7fe640 1 --2- 172.21.15.143:0/3921246302 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f689013e290 0x7f689012a630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:19:07.597 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:07.592+0000 7f688f7fe640 1 -- 172.21.15.143:0/3921246302 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f689015f6b0 msgr2=0x7f68900b0590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:19:07.597 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:07.592+0000 7f688f7fe640 1 --2- 172.21.15.143:0/3921246302 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f689015f6b0 0x7f68900b0590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:19:07.597 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:07.592+0000 7f688f7fe640 1 -- 172.21.15.143:0/3921246302 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6888009a70 con 0x7f68900cf7c0 2023-11-26T07:19:07.597 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:07.592+0000 7f688ffff640 1 --2- 172.21.15.143:0/3921246302 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f689015f6b0 0x7f68900b0590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-26T07:19:07.597 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:07.592+0000 7f688f7fe640 1 --2- 172.21.15.143:0/3921246302 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f68900cf7c0 0x7f68900a84d0 secure :-1 s=READY pgs=453 cs=0 l=1 rev1=1 crypto rx=0x7f688000e930 tx=0x7f688000edf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:19:07.597 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:07.592+0000 7f6877fff640 1 -- 172.21.15.143:0/3921246302 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6880009690 con 0x7f68900cf7c0 2023-11-26T07:19:07.598 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:07.592+0000 7f6877fff640 1 -- 172.21.15.143:0/3921246302 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f688000ca00 con 0x7f68900cf7c0 2023-11-26T07:19:07.598 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:07.592+0000 7f6877fff640 1 -- 172.21.15.143:0/3921246302 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f688000cd70 con 0x7f68900cf7c0 2023-11-26T07:19:07.598 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:07.592+0000 7f6895d57640 1 -- 172.21.15.143:0/3921246302 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f68900afab0 con 0x7f68900cf7c0 2023-11-26T07:19:07.598 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:07.592+0000 7f6895d57640 1 -- 172.21.15.143:0/3921246302 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6890137dc0 con 0x7f68900cf7c0 2023-11-26T07:19:07.598 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:07.592+0000 7f6895d57640 1 -- 172.21.15.143:0/3921246302 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6890067680 con 0x7f68900cf7c0 2023-11-26T07:19:07.598 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:07.596+0000 7f6877fff640 1 -- 172.21.15.143:0/3921246302 <== mon.0 v2:172.21.15.143:3300/0 4 ==== mgrmap(e 5) v1 ==== 85852+0+0 (secure 0 0 0) 0x7f6880013470 con 0x7f68900cf7c0 2023-11-26T07:19:07.600 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:07.596+0000 7f6877fff640 1 --2- 172.21.15.143:0/3921246302 >> [v2:172.21.15.143:6832/2150557544,v1:172.21.15.143:6833/2150557544] conn(0x7f687c066470 0x7f687c068930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:19:07.600 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:07.596+0000 7f6877fff640 1 -- 172.21.15.143:0/3921246302 <== mon.0 v2:172.21.15.143:3300/0 5 ==== osd_map(33..33 src has 1..33) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f68800840a0 con 0x7f68900cf7c0 2023-11-26T07:19:07.600 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:07.596+0000 7f688effd640 1 --2- 172.21.15.143:0/3921246302 >> [v2:172.21.15.143:6832/2150557544,v1:172.21.15.143:6833/2150557544] conn(0x7f687c066470 0x7f687c068930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:19:07.600 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:07.596+0000 7f688effd640 1 --2- 172.21.15.143:0/3921246302 >> [v2:172.21.15.143:6832/2150557544,v1:172.21.15.143:6833/2150557544] conn(0x7f687c066470 0x7f687c068930 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7f68900b17d0 tx=0x7f6884009040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:19:07.602 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:07.600+0000 7f6877fff640 1 -- 172.21.15.143:0/3921246302 <== mon.0 v2:172.21.15.143:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172649 (secure 0 0 0) 0x7f68800515f0 con 0x7f68900cf7c0 2023-11-26T07:19:07.780 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:07.776+0000 7f6895d57640 1 -- 172.21.15.143:0/3921246302 --> [v2:172.21.15.143:6832/2150557544,v1:172.21.15.143:6833/2150557544] -- mgr_command(tid 0: {"prefix": "fs subvolume getpath", "vol_name": "cephfs", "sub_name": "subvol12410", "group_name": "subvol_grp78310", "target": ["mon-mgr", ""]}) v1 -- 0x7f68900cc3c0 con 0x7f687c066470 2023-11-26T07:19:07.785 INFO:tasks.ceph.mgr.y.smithi143.stderr:2023-11-26T07:19:07.784+0000 7f5048ac6640 -1 client.0 error registering admin socket command: (17) File exists 2023-11-26T07:19:07.785 INFO:tasks.ceph.mgr.y.smithi143.stderr:2023-11-26T07:19:07.784+0000 7f5048ac6640 -1 client.0 error registering admin socket command: (17) File exists 2023-11-26T07:19:07.785 INFO:tasks.ceph.mgr.y.smithi143.stderr:2023-11-26T07:19:07.784+0000 7f5048ac6640 -1 client.0 error registering admin socket command: (17) File exists 2023-11-26T07:19:07.785 INFO:tasks.ceph.mgr.y.smithi143.stderr:2023-11-26T07:19:07.784+0000 7f5048ac6640 -1 client.0 error registering admin socket command: (17) File exists 2023-11-26T07:19:07.786 INFO:tasks.ceph.mgr.y.smithi143.stderr:2023-11-26T07:19:07.784+0000 7f5048ac6640 -1 client.0 error registering admin socket command: (17) File exists 2023-11-26T07:19:07.966 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:07.964+0000 7f6877fff640 1 -- 172.21.15.143:0/3921246302 <== mgr.4100 v2:172.21.15.143:6832/2150557544 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+37 (secure 0 0 0) 0x7f68900cc3c0 con 0x7f687c066470 2023-11-26T07:19:07.967 INFO:teuthology.orchestra.run.smithi143.stdout:/volumes/subvol_grp78310/subvol12410 2023-11-26T07:19:07.968 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:07.964+0000 7f6895d57640 1 -- 172.21.15.143:0/3921246302 >> [v2:172.21.15.143:6832/2150557544,v1:172.21.15.143:6833/2150557544] conn(0x7f687c066470 msgr2=0x7f687c068930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:19:07.968 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:07.964+0000 7f6895d57640 1 --2- 172.21.15.143:0/3921246302 >> [v2:172.21.15.143:6832/2150557544,v1:172.21.15.143:6833/2150557544] conn(0x7f687c066470 0x7f687c068930 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7f68900b17d0 tx=0x7f6884009040 comp rx=0 tx=0).stop 2023-11-26T07:19:07.968 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:07.964+0000 7f6895d57640 1 -- 172.21.15.143:0/3921246302 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f68900cf7c0 msgr2=0x7f68900a84d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:19:07.968 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:07.964+0000 7f6895d57640 1 --2- 172.21.15.143:0/3921246302 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f68900cf7c0 0x7f68900a84d0 secure :-1 s=READY pgs=453 cs=0 l=1 rev1=1 crypto rx=0x7f688000e930 tx=0x7f688000edf0 comp rx=0 tx=0).stop 2023-11-26T07:19:07.968 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:07.964+0000 7f6895d57640 1 -- 172.21.15.143:0/3921246302 shutdown_connections 2023-11-26T07:19:07.969 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:07.964+0000 7f6895d57640 1 --2- 172.21.15.143:0/3921246302 >> [v2:172.21.15.143:6832/2150557544,v1:172.21.15.143:6833/2150557544] conn(0x7f687c066470 0x7f687c068930 unknown :-1 s=CLOSED pgs=236 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:19:07.969 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:07.964+0000 7f6895d57640 1 --2- 172.21.15.143:0/3921246302 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f689015f6b0 0x7f68900b0590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:19:07.969 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:07.964+0000 7f6895d57640 1 --2- 172.21.15.143:0/3921246302 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f689013e290 0x7f689012a630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:19:07.969 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:07.964+0000 7f6895d57640 1 --2- 172.21.15.143:0/3921246302 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f68900cf7c0 0x7f68900a84d0 unknown :-1 s=CLOSED pgs=453 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:19:07.969 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:07.964+0000 7f6895d57640 1 -- 172.21.15.143:0/3921246302 >> 172.21.15.143:0/3921246302 conn(0x7f689006a890 msgr2=0x7f68900b1cb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:19:07.969 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:07.964+0000 7f6895d57640 1 -- 172.21.15.143:0/3921246302 shutdown_connections 2023-11-26T07:19:07.969 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:07.964+0000 7f6895d57640 1 -- 172.21.15.143:0/3921246302 wait complete. 2023-11-26T07:19:07.987 DEBUG:teuthology.orchestra.run.smithi143:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume getpath cephfs subvol12410 subvol_grp78310 2023-11-26T07:19:08.132 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:08.128+0000 7fe3db85a640 1 Processor -- start 2023-11-26T07:19:08.132 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:08.128+0000 7fe3db85a640 1 -- start start 2023-11-26T07:19:08.134 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:08.132+0000 7fe3db85a640 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7fe3d40cf280 0x7fe3d40cf710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:19:08.134 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:08.132+0000 7fe3db85a640 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7fe3d40cd1b0 0x7fe3d40cd5c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:19:08.134 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:08.132+0000 7fe3db85a640 1 -- --> v1:172.21.15.188:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe3d40cc730 con 0x7fe3d4143ac0 2023-11-26T07:19:08.134 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:08.132+0000 7fe3db85a640 1 -- --> v2:172.21.15.143:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe3d4143ea0 con 0x7fe3d40cd1b0 2023-11-26T07:19:08.135 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:08.132+0000 7fe3db85a640 1 -- --> v2:172.21.15.188:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe3d40cfc50 con 0x7fe3d40cf280 2023-11-26T07:19:08.135 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:08.132+0000 7fe3d9df7640 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7fe3d40cd1b0 0x7fe3d40cd5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:19:08.135 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:08.132+0000 7fe3d9df7640 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7fe3d40cd1b0 0x7fe3d40cd5c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:48530/0 (socket says 172.21.15.143:48530) 2023-11-26T07:19:08.135 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:08.132+0000 7fe3d9df7640 1 -- 172.21.15.143:0/2142581824 learned_addr learned my addr 172.21.15.143:0/2142581824 (peer_addr_for_me v2:172.21.15.143:0/0) 2023-11-26T07:19:08.135 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:08.132+0000 7fe3d8df5640 1 --2- 172.21.15.143:0/2142581824 >> v2:172.21.15.188:3300/0 conn(0x7fe3d40cf280 0x7fe3d40cf710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:19:08.135 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:08.132+0000 7fe3d9df7640 1 -- 172.21.15.143:0/2142581824 >> v2:172.21.15.188:3300/0 conn(0x7fe3d40cf280 msgr2=0x7fe3d40cf710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-26T07:19:08.135 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:08.132+0000 7fe3d9df7640 1 --2- 172.21.15.143:0/2142581824 >> v2:172.21.15.188:3300/0 conn(0x7fe3d40cf280 0x7fe3d40cf710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:19:08.135 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:08.132+0000 7fe3d9df7640 1 -- 172.21.15.143:0/2142581824 >> v1:172.21.15.188:6789/0 conn(0x7fe3d4143ac0 legacy=0x7fe3d4070370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:19:08.135 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:08.132+0000 7fe3d9df7640 1 -- 172.21.15.143:0/2142581824 --> v2:172.21.15.143:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe3d4070ee0 con 0x7fe3d40cd1b0 2023-11-26T07:19:08.137 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:08.136+0000 7fe3d9df7640 1 --2- 172.21.15.143:0/2142581824 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fe3d40cd1b0 0x7fe3d40cd5c0 secure :-1 s=READY pgs=456 cs=0 l=1 rev1=1 crypto rx=0x7fe3d40aeb10 tx=0x7fe3cc01c9d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1c322dc453ed26e0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:19:08.137 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:08.136+0000 7fe3cbfff640 1 -- 172.21.15.143:0/2142581824 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe3cc0a5890 con 0x7fe3d40cd1b0 2023-11-26T07:19:08.138 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:08.136+0000 7fe3cbfff640 1 -- 172.21.15.143:0/2142581824 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe3cc0a59f0 con 0x7fe3d40cd1b0 2023-11-26T07:19:08.138 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:08.136+0000 7fe3cbfff640 1 -- 172.21.15.143:0/2142581824 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe3cc0a5d80 con 0x7fe3d40cd1b0 2023-11-26T07:19:08.138 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:08.136+0000 7fe3db85a640 1 -- 172.21.15.143:0/2142581824 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fe3d40cd1b0 msgr2=0x7fe3d40cd5c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:19:08.139 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:08.136+0000 7fe3db85a640 1 --2- 172.21.15.143:0/2142581824 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fe3d40cd1b0 0x7fe3d40cd5c0 secure :-1 s=READY pgs=456 cs=0 l=1 rev1=1 crypto rx=0x7fe3d40aeb10 tx=0x7fe3cc01c9d0 comp rx=0 tx=0).stop 2023-11-26T07:19:08.139 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:08.136+0000 7fe3db85a640 1 -- 172.21.15.143:0/2142581824 shutdown_connections 2023-11-26T07:19:08.139 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:08.136+0000 7fe3db85a640 1 --2- 172.21.15.143:0/2142581824 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fe3d40cd1b0 0x7fe3d40cd5c0 unknown :-1 s=CLOSED pgs=456 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:19:08.139 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:08.136+0000 7fe3db85a640 1 --2- 172.21.15.143:0/2142581824 >> v2:172.21.15.188:3300/0 conn(0x7fe3d40cf280 0x7fe3d40cf710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:19:08.139 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:08.136+0000 7fe3db85a640 1 -- 172.21.15.143:0/2142581824 >> 172.21.15.143:0/2142581824 conn(0x7fe3d406a890 msgr2=0x7fe3d406aca0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:19:08.139 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:08.136+0000 7fe3db85a640 1 -- 172.21.15.143:0/2142581824 shutdown_connections 2023-11-26T07:19:08.139 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:08.136+0000 7fe3db85a640 1 -- 172.21.15.143:0/2142581824 wait complete. 2023-11-26T07:19:08.140 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:08.136+0000 7fe3db85a640 1 Processor -- start 2023-11-26T07:19:08.140 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:08.136+0000 7fe3db85a640 1 -- start start 2023-11-26T07:19:08.142 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:08.140+0000 7fe3db85a640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fe3d40cd1b0 0x7fe3d4158ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:19:08.142 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:08.140+0000 7fe3db85a640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fe3d40cf280 0x7fe3d4166500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:19:08.143 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:08.140+0000 7fe3db85a640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fe3d4143ac0 0x7fe3d406c520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:19:08.143 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:08.140+0000 7fe3db85a640 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe3d4159ed0 con 0x7fe3d40cd1b0 2023-11-26T07:19:08.143 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:08.140+0000 7fe3db85a640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe3d4159010 con 0x7fe3d4143ac0 2023-11-26T07:19:08.143 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:08.140+0000 7fe3db85a640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe3d4166a40 con 0x7fe3d40cf280 2023-11-26T07:19:08.143 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:08.140+0000 7fe3d95f6640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fe3d40cd1b0 0x7fe3d4158ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:19:08.143 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:08.140+0000 7fe3d95f6640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fe3d40cd1b0 0x7fe3d4158ad0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:48538/0 (socket says 172.21.15.143:48538) 2023-11-26T07:19:08.143 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:08.140+0000 7fe3d95f6640 1 -- 172.21.15.143:0/4073108089 learned_addr learned my addr 172.21.15.143:0/4073108089 (peer_addr_for_me v2:172.21.15.143:0/0) 2023-11-26T07:19:08.143 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:08.140+0000 7fe3d9df7640 1 --2- 172.21.15.143:0/4073108089 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fe3d4143ac0 0x7fe3d406c520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:19:08.143 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:08.140+0000 7fe3d8df5640 1 --2- 172.21.15.143:0/4073108089 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fe3d40cf280 0x7fe3d4166500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:19:08.143 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:08.140+0000 7fe3d95f6640 1 -- 172.21.15.143:0/4073108089 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fe3d40cf280 msgr2=0x7fe3d4166500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:19:08.143 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:08.140+0000 7fe3d95f6640 1 --2- 172.21.15.143:0/4073108089 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fe3d40cf280 0x7fe3d4166500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:19:08.143 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:08.140+0000 7fe3d95f6640 1 -- 172.21.15.143:0/4073108089 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fe3d4143ac0 msgr2=0x7fe3d406c520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:19:08.143 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:08.140+0000 7fe3d95f6640 1 --2- 172.21.15.143:0/4073108089 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fe3d4143ac0 0x7fe3d406c520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:19:08.144 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:08.140+0000 7fe3d95f6640 1 -- 172.21.15.143:0/4073108089 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe3cc009b30 con 0x7fe3d40cd1b0 2023-11-26T07:19:08.144 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:08.140+0000 7fe3d95f6640 1 --2- 172.21.15.143:0/4073108089 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fe3d40cd1b0 0x7fe3d4158ad0 secure :-1 s=READY pgs=457 cs=0 l=1 rev1=1 crypto rx=0x7fe3d000c3c0 tx=0x7fe3d000c880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:19:08.144 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:08.140+0000 7fe3c9ffb640 1 -- 172.21.15.143:0/4073108089 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe3d00052d0 con 0x7fe3d40cd1b0 2023-11-26T07:19:08.144 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:08.140+0000 7fe3c9ffb640 1 -- 172.21.15.143:0/4073108089 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe3d0005430 con 0x7fe3d40cd1b0 2023-11-26T07:19:08.144 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:08.140+0000 7fe3c9ffb640 1 -- 172.21.15.143:0/4073108089 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe3d0003840 con 0x7fe3d40cd1b0 2023-11-26T07:19:08.144 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:08.140+0000 7fe3db85a640 1 -- 172.21.15.143:0/4073108089 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe3d41592c0 con 0x7fe3d40cd1b0 2023-11-26T07:19:08.144 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:08.140+0000 7fe3db85a640 1 -- 172.21.15.143:0/4073108089 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe3d4159880 con 0x7fe3d40cd1b0 2023-11-26T07:19:08.144 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:08.140+0000 7fe3db85a640 1 -- 172.21.15.143:0/4073108089 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe3d4073ce0 con 0x7fe3d40cd1b0 2023-11-26T07:19:08.145 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:08.140+0000 7fe3c9ffb640 1 -- 172.21.15.143:0/4073108089 <== mon.0 v2:172.21.15.143:3300/0 4 ==== mgrmap(e 5) v1 ==== 85852+0+0 (secure 0 0 0) 0x7fe3d00039a0 con 0x7fe3d40cd1b0 2023-11-26T07:19:08.145 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:08.144+0000 7fe3c9ffb640 1 --2- 172.21.15.143:0/4073108089 >> [v2:172.21.15.143:6832/2150557544,v1:172.21.15.143:6833/2150557544] conn(0x7fe3b8066470 0x7fe3b8068930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:19:08.145 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:08.144+0000 7fe3c9ffb640 1 -- 172.21.15.143:0/4073108089 <== mon.0 v2:172.21.15.143:3300/0 5 ==== osd_map(33..33 src has 1..33) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fe3d0083310 con 0x7fe3d40cd1b0 2023-11-26T07:19:08.145 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:08.144+0000 7fe3d8df5640 1 --2- 172.21.15.143:0/4073108089 >> [v2:172.21.15.143:6832/2150557544,v1:172.21.15.143:6833/2150557544] conn(0x7fe3b8066470 0x7fe3b8068930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:19:08.146 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:08.144+0000 7fe3d8df5640 1 --2- 172.21.15.143:0/4073108089 >> [v2:172.21.15.143:6832/2150557544,v1:172.21.15.143:6833/2150557544] conn(0x7fe3b8066470 0x7fe3b8068930 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7fe3d4135f50 tx=0x7fe3c4009190 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:19:08.149 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:08.148+0000 7fe3c9ffb640 1 -- 172.21.15.143:0/4073108089 <== mon.0 v2:172.21.15.143:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172649 (secure 0 0 0) 0x7fe3d4073ce0 con 0x7fe3d40cd1b0 2023-11-26T07:19:08.322 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:08.320+0000 7fe3db85a640 1 -- 172.21.15.143:0/4073108089 --> [v2:172.21.15.143:6832/2150557544,v1:172.21.15.143:6833/2150557544] -- mgr_command(tid 0: {"prefix": "fs subvolume getpath", "vol_name": "cephfs", "sub_name": "subvol12410", "group_name": "subvol_grp78310", "target": ["mon-mgr", ""]}) v1 -- 0x7fe3d4140220 con 0x7fe3b8066470 2023-11-26T07:19:08.326 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:08.324+0000 7fe3c9ffb640 1 -- 172.21.15.143:0/4073108089 <== mgr.4100 v2:172.21.15.143:6832/2150557544 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+37 (secure 0 0 0) 0x7fe3c4004040 con 0x7fe3b8066470 2023-11-26T07:19:08.326 INFO:teuthology.orchestra.run.smithi143.stdout:/volumes/subvol_grp78310/subvol12410 2023-11-26T07:19:08.328 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:08.324+0000 7fe3db85a640 1 -- 172.21.15.143:0/4073108089 >> [v2:172.21.15.143:6832/2150557544,v1:172.21.15.143:6833/2150557544] conn(0x7fe3b8066470 msgr2=0x7fe3b8068930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:19:08.328 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:08.324+0000 7fe3db85a640 1 --2- 172.21.15.143:0/4073108089 >> [v2:172.21.15.143:6832/2150557544,v1:172.21.15.143:6833/2150557544] conn(0x7fe3b8066470 0x7fe3b8068930 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7fe3d4135f50 tx=0x7fe3c4009190 comp rx=0 tx=0).stop 2023-11-26T07:19:08.328 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:08.324+0000 7fe3db85a640 1 -- 172.21.15.143:0/4073108089 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fe3d40cd1b0 msgr2=0x7fe3d4158ad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:19:08.329 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:08.324+0000 7fe3db85a640 1 --2- 172.21.15.143:0/4073108089 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fe3d40cd1b0 0x7fe3d4158ad0 secure :-1 s=READY pgs=457 cs=0 l=1 rev1=1 crypto rx=0x7fe3d000c3c0 tx=0x7fe3d000c880 comp rx=0 tx=0).stop 2023-11-26T07:19:08.329 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:08.324+0000 7fe3db85a640 1 -- 172.21.15.143:0/4073108089 shutdown_connections 2023-11-26T07:19:08.329 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:08.324+0000 7fe3db85a640 1 --2- 172.21.15.143:0/4073108089 >> [v2:172.21.15.143:6832/2150557544,v1:172.21.15.143:6833/2150557544] conn(0x7fe3b8066470 0x7fe3b8068930 unknown :-1 s=CLOSED pgs=237 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:19:08.329 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:08.324+0000 7fe3db85a640 1 --2- 172.21.15.143:0/4073108089 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fe3d4143ac0 0x7fe3d406c520 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:19:08.329 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:08.324+0000 7fe3db85a640 1 --2- 172.21.15.143:0/4073108089 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fe3d40cf280 0x7fe3d4166500 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:19:08.329 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:08.324+0000 7fe3db85a640 1 --2- 172.21.15.143:0/4073108089 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fe3d40cd1b0 0x7fe3d4158ad0 unknown :-1 s=CLOSED pgs=457 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:19:08.329 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:08.324+0000 7fe3db85a640 1 -- 172.21.15.143:0/4073108089 >> 172.21.15.143:0/4073108089 conn(0x7fe3d406a890 msgr2=0x7fe3d413c460 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:19:08.329 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:08.324+0000 7fe3db85a640 1 -- 172.21.15.143:0/4073108089 shutdown_connections 2023-11-26T07:19:08.329 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:08.324+0000 7fe3db85a640 1 -- 172.21.15.143:0/4073108089 wait complete. 2023-11-26T07:19:08.347 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-26T07:19:08.347 DEBUG:teuthology.orchestra.run.smithi143:> (cd /home/ubuntu/cephtest/mnt.0 && exec sudo cat ./volumes/_legacy/1f4a70a2d39ea5823967fe13028e868b.meta) 2023-11-26T07:19:10.644 INFO:teuthology.orchestra.run.smithi143.stdout:[GLOBAL] 2023-11-26T07:19:10.644 INFO:teuthology.orchestra.run.smithi143.stdout:version = 1 2023-11-26T07:19:10.644 INFO:teuthology.orchestra.run.smithi143.stdout:type = subvolume 2023-11-26T07:19:10.644 INFO:teuthology.orchestra.run.smithi143.stdout:path = /volumes/subvol_grp78310/subvol12410 2023-11-26T07:19:10.644 INFO:teuthology.orchestra.run.smithi143.stdout:state = complete 2023-11-26T07:19:10.644 INFO:teuthology.orchestra.run.smithi143.stdout: 2023-11-26T07:19:10.647 DEBUG:teuthology.orchestra.run.smithi143:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume rm cephfs subvol12410 subvol_grp78310 2023-11-26T07:19:10.796 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:10.792+0000 7fe35eb65640 1 Processor -- start 2023-11-26T07:19:10.796 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:10.792+0000 7fe35eb65640 1 -- start start 2023-11-26T07:19:10.798 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:10.796+0000 7fe35eb65640 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7fe3580cf280 0x7fe3580cf710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:19:10.798 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:10.796+0000 7fe35eb65640 1 -- --> v1:172.21.15.143:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe3580cc730 con 0x7fe358143ac0 2023-11-26T07:19:10.798 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:10.796+0000 7fe35eb65640 1 -- --> v1:172.21.15.188:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe358070ee0 con 0x7fe3580cd1b0 2023-11-26T07:19:10.798 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:10.796+0000 7fe35eb65640 1 -- --> v2:172.21.15.188:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe358143ea0 con 0x7fe3580cf280 2023-11-26T07:19:10.798 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:10.796+0000 7fe35c901640 1 --1- >> v1:172.21.15.143:6789/0 conn(0x7fe358143ac0 0x7fe358070370 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.143:6789/0 says I am v1:172.21.15.143:38258/0 (socket says 172.21.15.143:38258) 2023-11-26T07:19:10.798 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:10.796+0000 7fe35c901640 1 -- 172.21.15.143:0/530774308 learned_addr learned my addr 172.21.15.143:0/530774308 (peer_addr_for_me v1:172.21.15.143:0/0) 2023-11-26T07:19:10.800 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:10.796+0000 7fe34ffff640 1 --2- 172.21.15.143:0/530774308 >> v2:172.21.15.188:3300/0 conn(0x7fe3580cf280 0x7fe3580cf710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:19:10.800 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:10.796+0000 7fe34f7fe640 1 -- 172.21.15.143:0/530774308 <== mon.0 v1:172.21.15.143:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2074491853 0 0) 0x7fe3580cc730 con 0x7fe358143ac0 2023-11-26T07:19:10.800 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:10.796+0000 7fe34f7fe640 1 -- 172.21.15.143:0/530774308 --> v1:172.21.15.143:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fe340005740 con 0x7fe358143ac0 2023-11-26T07:19:10.801 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:10.796+0000 7fe34f7fe640 1 -- 172.21.15.143:0/530774308 <== mon.1 v1:172.21.15.188:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2735449574 0 0) 0x7fe358070ee0 con 0x7fe3580cd1b0 2023-11-26T07:19:10.801 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:10.796+0000 7fe34f7fe640 1 -- 172.21.15.143:0/530774308 --> v1:172.21.15.188:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fe3580cc730 con 0x7fe3580cd1b0 2023-11-26T07:19:10.801 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:10.796+0000 7fe34f7fe640 1 -- 172.21.15.143:0/530774308 <== mon.0 v1:172.21.15.143:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 861516888 0 0) 0x7fe340005740 con 0x7fe358143ac0 2023-11-26T07:19:10.801 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:10.796+0000 7fe34f7fe640 1 -- 172.21.15.143:0/530774308 >> v2:172.21.15.188:3300/0 conn(0x7fe3580cf280 msgr2=0x7fe3580cf710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-26T07:19:10.801 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:10.796+0000 7fe34f7fe640 1 --2- 172.21.15.143:0/530774308 >> v2:172.21.15.188:3300/0 conn(0x7fe3580cf280 0x7fe3580cf710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:19:10.801 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:10.796+0000 7fe34f7fe640 1 -- 172.21.15.143:0/530774308 >> v1:172.21.15.188:6789/0 conn(0x7fe3580cd1b0 legacy=0x7fe3580cd5c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:19:10.801 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:10.796+0000 7fe34f7fe640 1 -- 172.21.15.143:0/530774308 --> v1:172.21.15.143:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe3580711c0 con 0x7fe358143ac0 2023-11-26T07:19:10.801 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:10.796+0000 7fe34f7fe640 1 -- 172.21.15.143:0/530774308 <== mon.0 v1:172.21.15.143:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2137882523 0 0) 0x7fe354001680 con 0x7fe358143ac0 2023-11-26T07:19:10.801 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:10.796+0000 7fe34f7fe640 1 -- 172.21.15.143:0/530774308 >> v1:172.21.15.143:6789/0 conn(0x7fe358143ac0 legacy=0x7fe358070370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:19:10.801 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:10.796+0000 7fe34f7fe640 1 --2- 172.21.15.143:0/530774308 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fe340005d20 0x7fe340006120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:19:10.801 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:10.796+0000 7fe34f7fe640 1 --2- 172.21.15.143:0/530774308 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fe3400069e0 0x7fe340006e60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:19:10.802 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:10.796+0000 7fe34f7fe640 1 --2- 172.21.15.143:0/530774308 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fe3400073a0 0x7fe34000dba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:19:10.802 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:10.796+0000 7fe34f7fe640 1 -- 172.21.15.143:0/530774308 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe34000e1f0 con 0x7fe3400073a0 2023-11-26T07:19:10.802 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:10.796+0000 7fe34f7fe640 1 -- 172.21.15.143:0/530774308 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe34000e360 con 0x7fe340005d20 2023-11-26T07:19:10.802 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:10.796+0000 7fe34f7fe640 1 -- 172.21.15.143:0/530774308 --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe34000e520 con 0x7fe3400069e0 2023-11-26T07:19:10.802 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:10.796+0000 7fe35d102640 1 --2- 172.21.15.143:0/530774308 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fe3400073a0 0x7fe34000dba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:19:10.802 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:10.796+0000 7fe35c901640 1 --2- 172.21.15.143:0/530774308 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fe340005d20 0x7fe340006120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:19:10.802 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:10.796+0000 7fe34ffff640 1 --2- 172.21.15.143:0/530774308 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fe3400069e0 0x7fe340006e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:19:10.802 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:10.796+0000 7fe35c901640 1 -- 172.21.15.143:0/530774308 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fe3400069e0 msgr2=0x7fe340006e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-26T07:19:10.802 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:10.796+0000 7fe35c901640 1 --2- 172.21.15.143:0/530774308 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fe3400069e0 0x7fe340006e60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:19:10.802 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:10.796+0000 7fe35c901640 1 -- 172.21.15.143:0/530774308 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fe3400073a0 msgr2=0x7fe34000dba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-26T07:19:10.803 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:10.796+0000 7fe35c901640 1 --2- 172.21.15.143:0/530774308 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fe3400073a0 0x7fe34000dba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:19:10.803 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:10.796+0000 7fe35c901640 1 -- 172.21.15.143:0/530774308 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe340005740 con 0x7fe340005d20 2023-11-26T07:19:10.803 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:10.796+0000 7fe35d102640 1 --2- 172.21.15.143:0/530774308 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fe3400073a0 0x7fe34000dba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-26T07:19:10.803 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:10.796+0000 7fe35d102640 1 -- 172.21.15.143:0/530774308 reap_dead start 2023-11-26T07:19:10.804 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:10.800+0000 7fe35c901640 1 --2- 172.21.15.143:0/530774308 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fe340005d20 0x7fe340006120 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7fe35400c160 tx=0x7fe35401f330 comp rx=0 tx=0).ready entity=mon.1 client_cookie=6dfe95e8924f07c7 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:19:10.804 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:10.800+0000 7fe34f7fe640 1 -- 172.21.15.143:0/530774308 <== mon.1 v2:172.21.15.188:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe354005ae0 con 0x7fe340005d20 2023-11-26T07:19:10.804 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:10.800+0000 7fe34f7fe640 1 -- 172.21.15.143:0/530774308 <== mon.1 v2:172.21.15.188:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe3540ab070 con 0x7fe340005d20 2023-11-26T07:19:10.804 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:10.800+0000 7fe34f7fe640 1 -- 172.21.15.143:0/530774308 <== mon.1 v2:172.21.15.188:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe354007070 con 0x7fe340005d20 2023-11-26T07:19:10.804 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:10.800+0000 7fe35eb65640 1 -- 172.21.15.143:0/530774308 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fe340005d20 msgr2=0x7fe340006120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:19:10.804 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:10.800+0000 7fe35eb65640 1 --2- 172.21.15.143:0/530774308 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fe340005d20 0x7fe340006120 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7fe35400c160 tx=0x7fe35401f330 comp rx=0 tx=0).stop 2023-11-26T07:19:10.804 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:10.800+0000 7fe35eb65640 1 -- 172.21.15.143:0/530774308 shutdown_connections 2023-11-26T07:19:10.804 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:10.800+0000 7fe35eb65640 1 --2- 172.21.15.143:0/530774308 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fe340005d20 0x7fe340006120 unknown :-1 s=CLOSED pgs=201 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:19:10.805 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:10.800+0000 7fe35eb65640 1 -- 172.21.15.143:0/530774308 >> 172.21.15.143:0/530774308 conn(0x7fe35806a890 msgr2=0x7fe35806aca0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:19:10.805 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:10.800+0000 7fe35eb65640 1 -- 172.21.15.143:0/530774308 shutdown_connections 2023-11-26T07:19:10.805 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:10.800+0000 7fe35eb65640 1 -- 172.21.15.143:0/530774308 wait complete. 2023-11-26T07:19:10.805 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:10.804+0000 7fe35eb65640 1 Processor -- start 2023-11-26T07:19:10.805 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:10.804+0000 7fe35eb65640 1 -- start start 2023-11-26T07:19:10.807 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:10.804+0000 7fe35eb65640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fe340005d20 0x7fe3581593d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:19:10.807 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:10.804+0000 7fe35eb65640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fe3581398a0 0x7fe3580add30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:19:10.807 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:10.804+0000 7fe35eb65640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fe35815bbd0 0x7fe35815c050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:19:10.807 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:10.804+0000 7fe35eb65640 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe35815c590 con 0x7fe3581398a0 2023-11-26T07:19:10.807 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:10.804+0000 7fe35eb65640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe358139d20 con 0x7fe35815bbd0 2023-11-26T07:19:10.807 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:10.804+0000 7fe35eb65640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe3581385a0 con 0x7fe340005d20 2023-11-26T07:19:10.807 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:10.804+0000 7fe34ffff640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fe3581398a0 0x7fe3580add30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:19:10.808 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:10.804+0000 7fe34ffff640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fe3581398a0 0x7fe3580add30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:48558/0 (socket says 172.21.15.143:48558) 2023-11-26T07:19:10.808 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:10.804+0000 7fe34ffff640 1 -- 172.21.15.143:0/3120578460 learned_addr learned my addr 172.21.15.143:0/3120578460 (peer_addr_for_me v2:172.21.15.143:0/0) 2023-11-26T07:19:10.808 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:10.804+0000 7fe35c901640 1 --2- 172.21.15.143:0/3120578460 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fe340005d20 0x7fe3581593d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:19:10.808 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:10.804+0000 7fe35d102640 1 --2- 172.21.15.143:0/3120578460 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fe35815bbd0 0x7fe35815c050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:19:10.808 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:10.804+0000 7fe34ffff640 1 -- 172.21.15.143:0/3120578460 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fe340005d20 msgr2=0x7fe3581593d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:19:10.808 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:10.804+0000 7fe34ffff640 1 --2- 172.21.15.143:0/3120578460 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fe340005d20 0x7fe3581593d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:19:10.808 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:10.804+0000 7fe34ffff640 1 -- 172.21.15.143:0/3120578460 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fe35815bbd0 msgr2=0x7fe35815c050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:19:10.808 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:10.804+0000 7fe34ffff640 1 --2- 172.21.15.143:0/3120578460 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fe35815bbd0 0x7fe35815c050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:19:10.808 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:10.804+0000 7fe34ffff640 1 -- 172.21.15.143:0/3120578460 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe358138820 con 0x7fe3581398a0 2023-11-26T07:19:10.808 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:10.804+0000 7fe35d102640 1 --2- 172.21.15.143:0/3120578460 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fe35815bbd0 0x7fe35815c050 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-26T07:19:10.808 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:10.804+0000 7fe34ffff640 1 --2- 172.21.15.143:0/3120578460 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fe3581398a0 0x7fe3580add30 secure :-1 s=READY pgs=459 cs=0 l=1 rev1=1 crypto rx=0x7fe348011a30 tx=0x7fe348011ef0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:19:10.809 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:10.804+0000 7fe34d7fa640 1 -- 172.21.15.143:0/3120578460 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe34800fb50 con 0x7fe3581398a0 2023-11-26T07:19:10.809 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:10.804+0000 7fe34d7fa640 1 -- 172.21.15.143:0/3120578460 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe34800fcb0 con 0x7fe3581398a0 2023-11-26T07:19:10.809 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:10.804+0000 7fe34d7fa640 1 -- 172.21.15.143:0/3120578460 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe348014440 con 0x7fe3581398a0 2023-11-26T07:19:10.810 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:10.804+0000 7fe35eb65640 1 -- 172.21.15.143:0/3120578460 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe358138ae0 con 0x7fe3581398a0 2023-11-26T07:19:10.810 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:10.804+0000 7fe35eb65640 1 -- 172.21.15.143:0/3120578460 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe3581390a0 con 0x7fe3581398a0 2023-11-26T07:19:10.810 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:10.804+0000 7fe35eb65640 1 -- 172.21.15.143:0/3120578460 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe358067680 con 0x7fe3581398a0 2023-11-26T07:19:10.810 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:10.808+0000 7fe34d7fa640 1 -- 172.21.15.143:0/3120578460 <== mon.0 v2:172.21.15.143:3300/0 4 ==== mgrmap(e 6) v1 ==== 85903+0+0 (secure 0 0 0) 0x7fe3480145a0 con 0x7fe3581398a0 2023-11-26T07:19:10.810 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:10.808+0000 7fe34d7fa640 1 --2- 172.21.15.143:0/3120578460 >> [v2:172.21.15.143:6832/2150557544,v1:172.21.15.143:6833/2150557544] conn(0x7fe344066530 0x7fe3440689f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:19:10.811 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:10.808+0000 7fe34d7fa640 1 -- 172.21.15.143:0/3120578460 <== mon.0 v2:172.21.15.143:3300/0 5 ==== osd_map(33..33 src has 1..33) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fe348084710 con 0x7fe3581398a0 2023-11-26T07:19:10.811 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:10.808+0000 7fe35c901640 1 --2- 172.21.15.143:0/3120578460 >> [v2:172.21.15.143:6832/2150557544,v1:172.21.15.143:6833/2150557544] conn(0x7fe344066530 0x7fe3440689f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:19:10.811 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:10.808+0000 7fe35c901640 1 --2- 172.21.15.143:0/3120578460 >> [v2:172.21.15.143:6832/2150557544,v1:172.21.15.143:6833/2150557544] conn(0x7fe344066530 0x7fe3440689f0 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7fe3540ae000 tx=0x7fe354007040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:19:10.814 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:10.812+0000 7fe34d7fa640 1 -- 172.21.15.143:0/3120578460 <== mon.0 v2:172.21.15.143:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172649 (secure 0 0 0) 0x7fe348051b60 con 0x7fe3581398a0 2023-11-26T07:19:10.986 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:10.984+0000 7fe35eb65640 1 -- 172.21.15.143:0/3120578460 --> [v2:172.21.15.143:6832/2150557544,v1:172.21.15.143:6833/2150557544] -- mgr_command(tid 0: {"prefix": "fs subvolume rm", "vol_name": "cephfs", "sub_name": "subvol12410", "group_name": "subvol_grp78310", "target": ["mon-mgr", ""]}) v1 -- 0x7fe358071c30 con 0x7fe344066530 2023-11-26T07:19:10.999 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:10.996+0000 7fe34d7fa640 1 -- 172.21.15.143:0/3120578460 <== mgr.4100 v2:172.21.15.143:6832/2150557544 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+0 (secure 0 0 0) 0x7fe3540b7040 con 0x7fe344066530 2023-11-26T07:19:11.002 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:11.000+0000 7fe35eb65640 1 -- 172.21.15.143:0/3120578460 >> [v2:172.21.15.143:6832/2150557544,v1:172.21.15.143:6833/2150557544] conn(0x7fe344066530 msgr2=0x7fe3440689f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:19:11.002 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:11.000+0000 7fe35eb65640 1 --2- 172.21.15.143:0/3120578460 >> [v2:172.21.15.143:6832/2150557544,v1:172.21.15.143:6833/2150557544] conn(0x7fe344066530 0x7fe3440689f0 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7fe3540ae000 tx=0x7fe354007040 comp rx=0 tx=0).stop 2023-11-26T07:19:11.002 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:11.000+0000 7fe35eb65640 1 -- 172.21.15.143:0/3120578460 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fe3581398a0 msgr2=0x7fe3580add30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:19:11.002 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:11.000+0000 7fe35eb65640 1 --2- 172.21.15.143:0/3120578460 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fe3581398a0 0x7fe3580add30 secure :-1 s=READY pgs=459 cs=0 l=1 rev1=1 crypto rx=0x7fe348011a30 tx=0x7fe348011ef0 comp rx=0 tx=0).stop 2023-11-26T07:19:11.002 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:11.000+0000 7fe35eb65640 1 -- 172.21.15.143:0/3120578460 shutdown_connections 2023-11-26T07:19:11.002 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:11.000+0000 7fe35eb65640 1 --2- 172.21.15.143:0/3120578460 >> [v2:172.21.15.143:6832/2150557544,v1:172.21.15.143:6833/2150557544] conn(0x7fe344066530 0x7fe3440689f0 unknown :-1 s=CLOSED pgs=238 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:19:11.002 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:11.000+0000 7fe35eb65640 1 --2- 172.21.15.143:0/3120578460 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fe35815bbd0 0x7fe35815c050 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:19:11.002 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:11.000+0000 7fe35eb65640 1 --2- 172.21.15.143:0/3120578460 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fe3581398a0 0x7fe3580add30 unknown :-1 s=CLOSED pgs=459 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:19:11.002 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:11.000+0000 7fe35eb65640 1 --2- 172.21.15.143:0/3120578460 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fe340005d20 0x7fe3581593d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:19:11.003 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:11.000+0000 7fe35eb65640 1 -- 172.21.15.143:0/3120578460 >> 172.21.15.143:0/3120578460 conn(0x7fe35806a890 msgr2=0x7fe358140dc0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:19:11.003 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:11.000+0000 7fe35eb65640 1 -- 172.21.15.143:0/3120578460 shutdown_connections 2023-11-26T07:19:11.003 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:11.000+0000 7fe35eb65640 1 -- 172.21.15.143:0/3120578460 wait complete. 2023-11-26T07:19:11.003 INFO:tasks.ceph.mgr.y.smithi143.stderr:2023-11-26T07:19:11.000+0000 7f504bacc640 -1 client.0 error registering admin socket command: (17) File exists 2023-11-26T07:19:11.003 INFO:tasks.ceph.mgr.y.smithi143.stderr:2023-11-26T07:19:11.000+0000 7f504bacc640 -1 client.0 error registering admin socket command: (17) File exists 2023-11-26T07:19:11.003 INFO:tasks.ceph.mgr.y.smithi143.stderr:2023-11-26T07:19:11.000+0000 7f504bacc640 -1 client.0 error registering admin socket command: (17) File exists 2023-11-26T07:19:11.004 INFO:tasks.ceph.mgr.y.smithi143.stderr:2023-11-26T07:19:11.000+0000 7f504bacc640 -1 client.0 error registering admin socket command: (17) File exists 2023-11-26T07:19:11.004 INFO:tasks.ceph.mgr.y.smithi143.stderr:2023-11-26T07:19:11.000+0000 7f504bacc640 -1 client.0 error registering admin socket command: (17) File exists 2023-11-26T07:19:11.020 INFO:tasks.ceph.mgr.y.smithi143.stderr:2023-11-26T07:19:11.016+0000 7f504a2c9640 -1 client.0 error registering admin socket command: (17) File exists 2023-11-26T07:19:11.020 INFO:tasks.ceph.mgr.y.smithi143.stderr:2023-11-26T07:19:11.016+0000 7f504a2c9640 -1 client.0 error registering admin socket command: (17) File exists 2023-11-26T07:19:11.020 INFO:tasks.ceph.mgr.y.smithi143.stderr:2023-11-26T07:19:11.016+0000 7f504a2c9640 -1 client.0 error registering admin socket command: (17) File exists 2023-11-26T07:19:11.020 INFO:tasks.ceph.mgr.y.smithi143.stderr:2023-11-26T07:19:11.016+0000 7f504a2c9640 -1 client.0 error registering admin socket command: (17) File exists 2023-11-26T07:19:11.020 INFO:tasks.ceph.mgr.y.smithi143.stderr:2023-11-26T07:19:11.016+0000 7f504a2c9640 -1 client.0 error registering admin socket command: (17) File exists 2023-11-26T07:19:11.026 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-26T07:19:11.026 DEBUG:teuthology.orchestra.run.smithi143:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c 'stat -c %h /home/ubuntu/cephtest/mnt.0/./volumes/_deleting') 2023-11-26T07:19:11.041 INFO:teuthology.orchestra.run.smithi143.stdout:2 2023-11-26T07:19:11.042 DEBUG:teuthology.orchestra.run.smithi143:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolumegroup rm cephfs subvol_grp78310 2023-11-26T07:19:11.227 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:11.224+0000 7f24dbc5a640 1 Processor -- start 2023-11-26T07:19:11.227 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:11.224+0000 7f24dbc5a640 1 -- start start 2023-11-26T07:19:11.229 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:11.228+0000 7f24dbc5a640 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7f24d415d6e0 0x7f24d40d0df0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:19:11.229 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:11.228+0000 7f24dbc5a640 1 -- --> v1:172.21.15.143:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f24d40cf0e0 con 0x7f24d40d1bc0 2023-11-26T07:19:11.229 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:11.228+0000 7f24dbc5a640 1 -- --> v1:172.21.15.188:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f24d40ceb20 con 0x7f24d40b2160 2023-11-26T07:19:11.229 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:11.228+0000 7f24dbc5a640 1 -- --> v2:172.21.15.143:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f24d415dac0 con 0x7f24d415d6e0 2023-11-26T07:19:11.229 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:11.228+0000 7f24d99f6640 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7f24d415d6e0 0x7f24d40d0df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:19:11.229 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:11.228+0000 7f24da1f7640 1 --1- >> v1:172.21.15.143:6789/0 conn(0x7f24d40d1bc0 0x7f24d40b0750 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.143:6789/0 says I am v1:172.21.15.143:38272/0 (socket says 172.21.15.143:38272) 2023-11-26T07:19:11.229 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:11.228+0000 7f24da1f7640 1 -- 172.21.15.143:0/2221648930 learned_addr learned my addr 172.21.15.143:0/2221648930 (peer_addr_for_me v1:172.21.15.143:0/0) 2023-11-26T07:19:11.230 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:11.228+0000 7f24d99f6640 1 -- 172.21.15.143:0/2221648930 >> v1:172.21.15.188:6789/0 conn(0x7f24d40b2160 legacy=0x7f24d40d14b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:19:11.230 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:11.228+0000 7f24d99f6640 1 -- 172.21.15.143:0/2221648930 >> v1:172.21.15.143:6789/0 conn(0x7f24d40d1bc0 legacy=0x7f24d40b0750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:19:11.230 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:11.228+0000 7f24d99f6640 1 -- 172.21.15.143:0/2221648930 --> v2:172.21.15.143:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f24d40cee00 con 0x7f24d415d6e0 2023-11-26T07:19:11.232 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:11.228+0000 7f24d99f6640 1 --2- 172.21.15.143:0/2221648930 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f24d415d6e0 0x7f24d40d0df0 secure :-1 s=READY pgs=464 cs=0 l=1 rev1=1 crypto rx=0x7f24d00097c0 tx=0x7f24d001c480 comp rx=0 tx=0).ready entity=mon.0 client_cookie=daeb155293e0d264 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:19:11.233 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:11.228+0000 7f24d89f4640 1 -- 172.21.15.143:0/2221648930 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f24d00ab600 con 0x7f24d415d6e0 2023-11-26T07:19:11.233 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:11.232+0000 7f24d89f4640 1 -- 172.21.15.143:0/2221648930 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f24d00a7050 con 0x7f24d415d6e0 2023-11-26T07:19:11.233 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:11.232+0000 7f24d89f4640 1 -- 172.21.15.143:0/2221648930 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f24d00ab980 con 0x7f24d415d6e0 2023-11-26T07:19:11.233 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:11.232+0000 7f24dbc5a640 1 -- 172.21.15.143:0/2221648930 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f24d415d6e0 msgr2=0x7f24d40d0df0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:19:11.233 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:11.232+0000 7f24dbc5a640 1 --2- 172.21.15.143:0/2221648930 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f24d415d6e0 0x7f24d40d0df0 secure :-1 s=READY pgs=464 cs=0 l=1 rev1=1 crypto rx=0x7f24d00097c0 tx=0x7f24d001c480 comp rx=0 tx=0).stop 2023-11-26T07:19:11.233 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:11.232+0000 7f24dbc5a640 1 -- 172.21.15.143:0/2221648930 shutdown_connections 2023-11-26T07:19:11.234 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:11.232+0000 7f24dbc5a640 1 --2- 172.21.15.143:0/2221648930 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f24d415d6e0 0x7f24d40d0df0 unknown :-1 s=CLOSED pgs=464 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:19:11.234 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:11.232+0000 7f24dbc5a640 1 -- 172.21.15.143:0/2221648930 >> 172.21.15.143:0/2221648930 conn(0x7f24d409c550 msgr2=0x7f24d40d3220 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:19:11.234 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:11.232+0000 7f24dbc5a640 1 -- 172.21.15.143:0/2221648930 shutdown_connections 2023-11-26T07:19:11.234 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:11.232+0000 7f24dbc5a640 1 -- 172.21.15.143:0/2221648930 wait complete. 2023-11-26T07:19:11.235 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:11.232+0000 7f24dbc5a640 1 Processor -- start 2023-11-26T07:19:11.235 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:11.232+0000 7f24dbc5a640 1 -- start start 2023-11-26T07:19:11.237 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:11.232+0000 7f24dbc5a640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f24d40b2160 0x7f24d4154310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:19:11.237 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:11.232+0000 7f24dbc5a640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f24d40d1bc0 0x7f24d4153b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:19:11.237 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:11.232+0000 7f24dbc5a640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f24d415d6e0 0x7f24d406b0c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:19:11.237 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:11.232+0000 7f24dbc5a640 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f24d415e2f0 con 0x7f24d415d6e0 2023-11-26T07:19:11.237 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:11.232+0000 7f24dbc5a640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f24d4154090 con 0x7f24d40b2160 2023-11-26T07:19:11.237 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:11.232+0000 7f24dbc5a640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f24d415e940 con 0x7f24d40d1bc0 2023-11-26T07:19:11.237 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:11.232+0000 7f24da1f7640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f24d415d6e0 0x7f24d406b0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:19:11.238 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:11.232+0000 7f24da1f7640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f24d415d6e0 0x7f24d406b0c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:48622/0 (socket says 172.21.15.143:48622) 2023-11-26T07:19:11.238 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:11.232+0000 7f24da1f7640 1 -- 172.21.15.143:0/3198971010 learned_addr learned my addr 172.21.15.143:0/3198971010 (peer_addr_for_me v2:172.21.15.143:0/0) 2023-11-26T07:19:11.238 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:11.232+0000 7f24d99f6640 1 --2- 172.21.15.143:0/3198971010 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f24d40b2160 0x7f24d4154310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:19:11.238 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:11.232+0000 7f24d91f5640 1 --2- 172.21.15.143:0/3198971010 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f24d40d1bc0 0x7f24d4153b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:19:11.238 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:11.236+0000 7f24da1f7640 1 -- 172.21.15.143:0/3198971010 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f24d40d1bc0 msgr2=0x7f24d4153b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:19:11.238 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:11.236+0000 7f24da1f7640 1 --2- 172.21.15.143:0/3198971010 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f24d40d1bc0 0x7f24d4153b50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:19:11.238 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:11.236+0000 7f24da1f7640 1 -- 172.21.15.143:0/3198971010 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f24d40b2160 msgr2=0x7f24d4154310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:19:11.238 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:11.236+0000 7f24da1f7640 1 --2- 172.21.15.143:0/3198971010 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f24d40b2160 0x7f24d4154310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:19:11.238 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:11.236+0000 7f24da1f7640 1 -- 172.21.15.143:0/3198971010 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f24d0009420 con 0x7f24d415d6e0 2023-11-26T07:19:11.238 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:11.236+0000 7f24da1f7640 1 --2- 172.21.15.143:0/3198971010 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f24d415d6e0 0x7f24d406b0c0 secure :-1 s=READY pgs=465 cs=0 l=1 rev1=1 crypto rx=0x7f24cc00caf0 tx=0x7f24cc00a060 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:19:11.239 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:11.236+0000 7f24ca7fc640 1 -- 172.21.15.143:0/3198971010 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f24cc00aca0 con 0x7f24d415d6e0 2023-11-26T07:19:11.239 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:11.236+0000 7f24ca7fc640 1 -- 172.21.15.143:0/3198971010 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f24cc00ae00 con 0x7f24d415d6e0 2023-11-26T07:19:11.240 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:11.236+0000 7f24ca7fc640 1 -- 172.21.15.143:0/3198971010 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f24cc012600 con 0x7f24d415d6e0 2023-11-26T07:19:11.240 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:11.236+0000 7f24dbc5a640 1 -- 172.21.15.143:0/3198971010 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f24d415dca0 con 0x7f24d415d6e0 2023-11-26T07:19:11.240 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:11.236+0000 7f24dbc5a640 1 -- 172.21.15.143:0/3198971010 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f24d415e5d0 con 0x7f24d415d6e0 2023-11-26T07:19:11.240 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:11.236+0000 7f24dbc5a640 1 -- 172.21.15.143:0/3198971010 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f24d4067680 con 0x7f24d415d6e0 2023-11-26T07:19:11.240 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:11.236+0000 7f24ca7fc640 1 -- 172.21.15.143:0/3198971010 <== mon.0 v2:172.21.15.143:3300/0 4 ==== mgrmap(e 6) v1 ==== 85903+0+0 (secure 0 0 0) 0x7f24cc007070 con 0x7f24d415d6e0 2023-11-26T07:19:11.241 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:11.236+0000 7f24ca7fc640 1 --2- 172.21.15.143:0/3198971010 >> [v2:172.21.15.143:6832/2150557544,v1:172.21.15.143:6833/2150557544] conn(0x7f24b8066530 0x7f24b80689f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:19:11.241 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:11.236+0000 7f24ca7fc640 1 -- 172.21.15.143:0/3198971010 <== mon.0 v2:172.21.15.143:3300/0 5 ==== osd_map(33..33 src has 1..33) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f24cc083cf0 con 0x7f24d415d6e0 2023-11-26T07:19:11.241 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:11.236+0000 7f24d99f6640 1 --2- 172.21.15.143:0/3198971010 >> [v2:172.21.15.143:6832/2150557544,v1:172.21.15.143:6833/2150557544] conn(0x7f24b8066530 0x7f24b80689f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:19:11.241 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:11.240+0000 7f24d99f6640 1 --2- 172.21.15.143:0/3198971010 >> [v2:172.21.15.143:6832/2150557544,v1:172.21.15.143:6833/2150557544] conn(0x7f24b8066530 0x7f24b80689f0 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7f24d00ad000 tx=0x7f24d00a6000 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:19:11.244 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:11.240+0000 7f24ca7fc640 1 -- 172.21.15.143:0/3198971010 <== mon.0 v2:172.21.15.143:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172649 (secure 0 0 0) 0x7f24cc0511c0 con 0x7f24d415d6e0 2023-11-26T07:19:11.414 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:11.412+0000 7f24dbc5a640 1 -- 172.21.15.143:0/3198971010 --> [v2:172.21.15.143:6832/2150557544,v1:172.21.15.143:6833/2150557544] -- mgr_command(tid 0: {"prefix": "fs subvolumegroup rm", "vol_name": "cephfs", "group_name": "subvol_grp78310", "target": ["mon-mgr", ""]}) v1 -- 0x7f24d40d3630 con 0x7f24b8066530 2023-11-26T07:19:11.418 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:11.416+0000 7f24ca7fc640 1 -- 172.21.15.143:0/3198971010 <== mgr.4100 v2:172.21.15.143:6832/2150557544 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+0 (secure 0 0 0) 0x7f24d40d3630 con 0x7f24b8066530 2023-11-26T07:19:11.420 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:11.416+0000 7f24dbc5a640 1 -- 172.21.15.143:0/3198971010 >> [v2:172.21.15.143:6832/2150557544,v1:172.21.15.143:6833/2150557544] conn(0x7f24b8066530 msgr2=0x7f24b80689f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:19:11.420 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:11.416+0000 7f24dbc5a640 1 --2- 172.21.15.143:0/3198971010 >> [v2:172.21.15.143:6832/2150557544,v1:172.21.15.143:6833/2150557544] conn(0x7f24b8066530 0x7f24b80689f0 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7f24d00ad000 tx=0x7f24d00a6000 comp rx=0 tx=0).stop 2023-11-26T07:19:11.420 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:11.416+0000 7f24dbc5a640 1 -- 172.21.15.143:0/3198971010 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f24d415d6e0 msgr2=0x7f24d406b0c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:19:11.421 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:11.416+0000 7f24dbc5a640 1 --2- 172.21.15.143:0/3198971010 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f24d415d6e0 0x7f24d406b0c0 secure :-1 s=READY pgs=465 cs=0 l=1 rev1=1 crypto rx=0x7f24cc00caf0 tx=0x7f24cc00a060 comp rx=0 tx=0).stop 2023-11-26T07:19:11.421 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:11.416+0000 7f24dbc5a640 1 -- 172.21.15.143:0/3198971010 shutdown_connections 2023-11-26T07:19:11.421 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:11.416+0000 7f24dbc5a640 1 --2- 172.21.15.143:0/3198971010 >> [v2:172.21.15.143:6832/2150557544,v1:172.21.15.143:6833/2150557544] conn(0x7f24b8066530 0x7f24b80689f0 unknown :-1 s=CLOSED pgs=239 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:19:11.421 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:11.416+0000 7f24dbc5a640 1 --2- 172.21.15.143:0/3198971010 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f24d415d6e0 0x7f24d406b0c0 unknown :-1 s=CLOSED pgs=465 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:19:11.421 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:11.416+0000 7f24dbc5a640 1 --2- 172.21.15.143:0/3198971010 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f24d40d1bc0 0x7f24d4153b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:19:11.421 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:11.416+0000 7f24dbc5a640 1 --2- 172.21.15.143:0/3198971010 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f24d40b2160 0x7f24d4154310 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:19:11.421 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:11.416+0000 7f24dbc5a640 1 -- 172.21.15.143:0/3198971010 >> 172.21.15.143:0/3198971010 conn(0x7f24d409c550 msgr2=0x7f24d40cf490 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:19:11.421 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:11.416+0000 7f24dbc5a640 1 -- 172.21.15.143:0/3198971010 shutdown_connections 2023-11-26T07:19:11.421 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:11.416+0000 7f24dbc5a640 1 -- 172.21.15.143:0/3198971010 wait complete. 2023-11-26T07:19:11.439 INFO:teuthology.nuke.actions:Clearing teuthology firewall rules... 2023-11-26T07:19:11.439 DEBUG:teuthology.orchestra.run.smithi143:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2023-11-26T07:19:11.527 DEBUG:teuthology.orchestra.run.smithi188:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2023-11-26T07:19:11.612 INFO:teuthology.nuke.actions:Cleared teuthology firewall rules. 2023-11-26T07:19:11.612 DEBUG:teuthology.orchestra.run.smithi143:> set -ex 2023-11-26T07:19:11.612 DEBUG:teuthology.orchestra.run.smithi143:> dd if=/proc/self/mounts of=/dev/stdout 2023-11-26T07:19:11.621 DEBUG:teuthology.orchestra.run.smithi143:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-11-26T07:19:11.802 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:11.800+0000 7f5fb125a640 1 Processor -- start 2023-11-26T07:19:11.802 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:11.800+0000 7f5fb125a640 1 -- start start 2023-11-26T07:19:11.804 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:11.800+0000 7f5fb125a640 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7f5fac0d1bc0 0x7f5fac0d0df0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:19:11.804 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:11.800+0000 7f5fb125a640 1 -- --> v1:172.21.15.143:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5fac0cf0e0 con 0x7f5fac15d6e0 2023-11-26T07:19:11.804 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:11.800+0000 7f5fb125a640 1 -- --> v1:172.21.15.188:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5fac0ceb20 con 0x7f5fac0b2160 2023-11-26T07:19:11.804 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:11.800+0000 7f5fb125a640 1 -- --> v2:172.21.15.143:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5fac15dac0 con 0x7f5fac0d1bc0 2023-11-26T07:19:11.804 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:11.800+0000 7f5fa259c640 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7f5fac0d1bc0 0x7f5fac0d0df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:19:11.804 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:11.800+0000 7f5faad9d640 1 --1- >> v1:172.21.15.143:6789/0 conn(0x7f5fac15d6e0 0x7f5fac0d14b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.143:6789/0 says I am v1:172.21.15.143:38288/0 (socket says 172.21.15.143:38288) 2023-11-26T07:19:11.804 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:11.800+0000 7f5faad9d640 1 -- 172.21.15.143:0/846897154 learned_addr learned my addr 172.21.15.143:0/846897154 (peer_addr_for_me v1:172.21.15.143:0/0) 2023-11-26T07:19:11.805 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:11.800+0000 7f5fa259c640 1 -- 172.21.15.143:0/846897154 >> v1:172.21.15.188:6789/0 conn(0x7f5fac0b2160 legacy=0x7f5fac0b0750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-26T07:19:11.805 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:11.800+0000 7f5fa259c640 1 -- 172.21.15.143:0/846897154 >> v1:172.21.15.143:6789/0 conn(0x7f5fac15d6e0 legacy=0x7f5fac0d14b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:19:11.805 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:11.800+0000 7f5fa259c640 1 -- 172.21.15.143:0/846897154 --> v2:172.21.15.143:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5fac0cee00 con 0x7f5fac0d1bc0 2023-11-26T07:19:11.805 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:11.800+0000 7f5faa59c640 1 -- 172.21.15.143:0/846897154 <== mon.0 v1:172.21.15.143:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3245423402 0 0) 0x7f5fac0cf0e0 con 0x7f5fac15d6e0 2023-11-26T07:19:11.807 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:11.804+0000 7f5fa259c640 1 --2- 172.21.15.143:0/846897154 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f5fac0d1bc0 0x7f5fac0d0df0 secure :-1 s=READY pgs=467 cs=0 l=1 rev1=1 crypto rx=0x7f5f94009d90 tx=0x7f5f9401c910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9d7da6a0d298faa1 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:19:11.807 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:11.804+0000 7f5faa59c640 1 -- 172.21.15.143:0/846897154 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5f940a58c0 con 0x7f5fac0d1bc0 2023-11-26T07:19:11.808 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:11.804+0000 7f5faa59c640 1 -- 172.21.15.143:0/846897154 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5f940a5a20 con 0x7f5fac0d1bc0 2023-11-26T07:19:11.808 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:11.804+0000 7f5faa59c640 1 -- 172.21.15.143:0/846897154 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5f940a5db0 con 0x7f5fac0d1bc0 2023-11-26T07:19:11.808 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:11.804+0000 7f5fb125a640 1 -- 172.21.15.143:0/846897154 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f5fac0d1bc0 msgr2=0x7f5fac0d0df0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:19:11.808 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:11.804+0000 7f5fb125a640 1 --2- 172.21.15.143:0/846897154 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f5fac0d1bc0 0x7f5fac0d0df0 secure :-1 s=READY pgs=467 cs=0 l=1 rev1=1 crypto rx=0x7f5f94009d90 tx=0x7f5f9401c910 comp rx=0 tx=0).stop 2023-11-26T07:19:11.808 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:11.804+0000 7f5fb125a640 1 -- 172.21.15.143:0/846897154 shutdown_connections 2023-11-26T07:19:11.809 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:11.804+0000 7f5fb125a640 1 --2- 172.21.15.143:0/846897154 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f5fac0d1bc0 0x7f5fac0d0df0 unknown :-1 s=CLOSED pgs=467 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:19:11.809 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:11.804+0000 7f5fb125a640 1 -- 172.21.15.143:0/846897154 >> 172.21.15.143:0/846897154 conn(0x7f5fac09c550 msgr2=0x7f5fac0d3220 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:19:11.809 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:11.804+0000 7f5fb125a640 1 -- 172.21.15.143:0/846897154 shutdown_connections 2023-11-26T07:19:11.809 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:11.804+0000 7f5fb125a640 1 -- 172.21.15.143:0/846897154 wait complete. 2023-11-26T07:19:11.809 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:11.808+0000 7f5fb125a640 1 Processor -- start 2023-11-26T07:19:11.810 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:11.808+0000 7f5fb125a640 1 -- start start 2023-11-26T07:19:11.812 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:11.808+0000 7f5fb125a640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f5fac0b2160 0x7f5fac154590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:19:11.812 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:11.808+0000 7f5fb125a640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f5fac0d1bc0 0x7f5fac0c8dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:19:11.812 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:11.808+0000 7f5fb125a640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f5fac15d6e0 0x7f5fac06c110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:19:11.812 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:11.808+0000 7f5fb125a640 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5fac154ad0 con 0x7f5fac0b2160 2023-11-26T07:19:11.812 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:11.808+0000 7f5fb125a640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5fac0cab20 con 0x7f5fac0d1bc0 2023-11-26T07:19:11.813 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:11.808+0000 7f5fb125a640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5fac0c70c0 con 0x7f5fac15d6e0 2023-11-26T07:19:11.813 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:11.808+0000 7f5faad9d640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f5fac0b2160 0x7f5fac154590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:19:11.813 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:11.808+0000 7f5faad9d640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f5fac0b2160 0x7f5fac154590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:48644/0 (socket says 172.21.15.143:48644) 2023-11-26T07:19:11.813 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:11.808+0000 7f5faad9d640 1 -- 172.21.15.143:0/645969230 learned_addr learned my addr 172.21.15.143:0/645969230 (peer_addr_for_me v2:172.21.15.143:0/0) 2023-11-26T07:19:11.813 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:11.808+0000 7f5fa259c640 1 --2- 172.21.15.143:0/645969230 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f5fac0d1bc0 0x7f5fac0c8dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:19:11.813 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:11.808+0000 7f5fab59e640 1 --2- 172.21.15.143:0/645969230 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f5fac15d6e0 0x7f5fac06c110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:19:11.813 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:11.808+0000 7f5faad9d640 1 -- 172.21.15.143:0/645969230 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f5fac15d6e0 msgr2=0x7f5fac06c110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:19:11.813 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:11.808+0000 7f5faad9d640 1 --2- 172.21.15.143:0/645969230 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f5fac15d6e0 0x7f5fac06c110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:19:11.813 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:11.808+0000 7f5faad9d640 1 -- 172.21.15.143:0/645969230 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f5fac0d1bc0 msgr2=0x7f5fac0c8dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:19:11.813 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:11.808+0000 7f5faad9d640 1 --2- 172.21.15.143:0/645969230 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f5fac0d1bc0 0x7f5fac0c8dc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:19:11.813 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:11.808+0000 7f5faad9d640 1 -- 172.21.15.143:0/645969230 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5f94009a70 con 0x7f5fac0b2160 2023-11-26T07:19:11.814 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:11.808+0000 7f5faad9d640 1 --2- 172.21.15.143:0/645969230 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f5fac0b2160 0x7f5fac154590 secure :-1 s=READY pgs=468 cs=0 l=1 rev1=1 crypto rx=0x7f5f9c00c480 tx=0x7f5f9c00c940 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:19:11.814 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:11.808+0000 7f5fa3fff640 1 -- 172.21.15.143:0/645969230 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5f9c0074e0 con 0x7f5fac0b2160 2023-11-26T07:19:11.814 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:11.808+0000 7f5fa3fff640 1 -- 172.21.15.143:0/645969230 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5f9c007660 con 0x7f5fac0b2160 2023-11-26T07:19:11.814 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:11.808+0000 7f5fa3fff640 1 -- 172.21.15.143:0/645969230 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5f9c004a30 con 0x7f5fac0b2160 2023-11-26T07:19:11.814 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:11.808+0000 7f5fb125a640 1 -- 172.21.15.143:0/645969230 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5fac0c73a0 con 0x7f5fac0b2160 2023-11-26T07:19:11.814 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:11.808+0000 7f5fb125a640 1 -- 172.21.15.143:0/645969230 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5fac0c7960 con 0x7f5fac0b2160 2023-11-26T07:19:11.815 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:11.812+0000 7f5fb125a640 1 -- 172.21.15.143:0/645969230 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5fac0c7b80 con 0x7f5fac0b2160 2023-11-26T07:19:11.815 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:11.812+0000 7f5fa3fff640 1 -- 172.21.15.143:0/645969230 <== mon.0 v2:172.21.15.143:3300/0 4 ==== mgrmap(e 6) v1 ==== 85903+0+0 (secure 0 0 0) 0x7f5f9c004b90 con 0x7f5fac0b2160 2023-11-26T07:19:11.815 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:11.812+0000 7f5fa3fff640 1 --2- 172.21.15.143:0/645969230 >> [v2:172.21.15.143:6832/2150557544,v1:172.21.15.143:6833/2150557544] conn(0x7f5f8c066500 0x7f5f8c0689c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:19:11.815 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:11.812+0000 7f5fa3fff640 1 -- 172.21.15.143:0/645969230 <== mon.0 v2:172.21.15.143:3300/0 5 ==== osd_map(33..33 src has 1..33) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f5f9c0830c0 con 0x7f5fac0b2160 2023-11-26T07:19:11.815 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:11.812+0000 7f5fa259c640 1 --2- 172.21.15.143:0/645969230 >> [v2:172.21.15.143:6832/2150557544,v1:172.21.15.143:6833/2150557544] conn(0x7f5f8c066500 0x7f5f8c0689c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:19:11.816 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:11.812+0000 7f5fa259c640 1 --2- 172.21.15.143:0/645969230 >> [v2:172.21.15.143:6832/2150557544,v1:172.21.15.143:6833/2150557544] conn(0x7f5f8c066500 0x7f5f8c0689c0 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7f5f940af000 tx=0x7f5f940a7070 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:19:11.819 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:11.816+0000 7f5fa3fff640 1 -- 172.21.15.143:0/645969230 <== mon.0 v2:172.21.15.143:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172649 (secure 0 0 0) 0x7f5f9c053f70 con 0x7f5fac0b2160 2023-11-26T07:19:11.968 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:11.964+0000 7f5fb125a640 1 -- 172.21.15.143:0/645969230 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f5fac067680 con 0x7f5fac0b2160 2023-11-26T07:19:11.969 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:11.968+0000 7f5fa3fff640 1 -- 172.21.15.143:0/645969230 <== mon.0 v2:172.21.15.143:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v33) v1 ==== 74+0+16605 (secure 0 0 0) 0x7f5f9c053b00 con 0x7f5fac0b2160 2023-11-26T07:19:11.970 INFO:teuthology.orchestra.run.smithi143.stdout: 2023-11-26T07:19:11.970 INFO:teuthology.orchestra.run.smithi143.stdout:{"epoch":33,"fsid":"f5033060-2469-4a07-95fe-2b76f5404bc1","created":"2023-11-26T07:17:28.613663+0000","modified":"2023-11-26T07:18:53.659425+0000","last_up_change":"2023-11-26T07:17:41.157813+0000","last_in_change":"2023-11-26T07:17:31.181658+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":5,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-11-26T07:17:41.758597+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_metadata","create_time":"2023-11-26T07:18:50.535934+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"32","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":5,"pool_name":"cephfs_data","create_time":"2023-11-26T07:18:51.008494+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"33","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"ebbee2c6-023e-44b9-9f1d-6f7bfd309730","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6817","nonce":639807369},{"type":"v1","addr":"172.21.15.143:6819","nonce":639807369}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6821","nonce":639807369},{"type":"v1","addr":"172.21.15.143:6823","nonce":639807369}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6830","nonce":639807369},{"type":"v1","addr":"172.21.15.143:6831","nonce":639807369}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6825","nonce":639807369},{"type":"v1","addr":"172.21.15.143:6828","nonce":639807369}]},"public_addr":"172.21.15.143:6819/639807369","cluster_addr":"172.21.15.143:6823/639807369","heartbeat_back_addr":"172.21.15.143:6831/639807369","heartbeat_front_addr":"172.21.15.143:6828/639807369","state":["exists","up"]},{"osd":1,"uuid":"fec4b8be-7122-49e9-9a19-5850a0700001","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6816","nonce":1144942530},{"type":"v1","addr":"172.21.15.143:6818","nonce":1144942530}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6820","nonce":1144942530},{"type":"v1","addr":"172.21.15.143:6822","nonce":1144942530}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6827","nonce":1144942530},{"type":"v1","addr":"172.21.15.143:6829","nonce":1144942530}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6824","nonce":1144942530},{"type":"v1","addr":"172.21.15.143:6826","nonce":1144942530}]},"public_addr":"172.21.15.143:6818/1144942530","cluster_addr":"172.21.15.143:6822/1144942530","heartbeat_back_addr":"172.21.15.143:6829/1144942530","heartbeat_front_addr":"172.21.15.143:6826/1144942530","state":["exists","up"]},{"osd":2,"uuid":"2888af5b-f933-46ca-8f95-6dd494ff43f6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6808","nonce":3909095186},{"type":"v1","addr":"172.21.15.143:6809","nonce":3909095186}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6810","nonce":3909095186},{"type":"v1","addr":"172.21.15.143:6811","nonce":3909095186}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6814","nonce":3909095186},{"type":"v1","addr":"172.21.15.143:6815","nonce":3909095186}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6812","nonce":3909095186},{"type":"v1","addr":"172.21.15.143:6813","nonce":3909095186}]},"public_addr":"172.21.15.143:6809/3909095186","cluster_addr":"172.21.15.143:6811/3909095186","heartbeat_back_addr":"172.21.15.143:6815/3909095186","heartbeat_front_addr":"172.21.15.143:6813/3909095186","state":["exists","up"]},{"osd":3,"uuid":"d1f52435-7bc1-4081-9550-9cd1ba55f549","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6800","nonce":3854210991},{"type":"v1","addr":"172.21.15.143:6801","nonce":3854210991}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6802","nonce":3854210991},{"type":"v1","addr":"172.21.15.143:6803","nonce":3854210991}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6806","nonce":3854210991},{"type":"v1","addr":"172.21.15.143:6807","nonce":3854210991}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6804","nonce":3854210991},{"type":"v1","addr":"172.21.15.143:6805","nonce":3854210991}]},"public_addr":"172.21.15.143:6801/3854210991","cluster_addr":"172.21.15.143:6803/3854210991","heartbeat_back_addr":"172.21.15.143:6807/3854210991","heartbeat_front_addr":"172.21.15.143:6805/3854210991","state":["exists","up"]},{"osd":4,"uuid":"e2ae8b6e-6e25-49b2-bcbf-aacce68f06a3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6800","nonce":3924606902},{"type":"v1","addr":"172.21.15.188:6801","nonce":3924606902}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6802","nonce":3924606902},{"type":"v1","addr":"172.21.15.188:6803","nonce":3924606902}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6806","nonce":3924606902},{"type":"v1","addr":"172.21.15.188:6807","nonce":3924606902}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6804","nonce":3924606902},{"type":"v1","addr":"172.21.15.188:6805","nonce":3924606902}]},"public_addr":"172.21.15.188:6801/3924606902","cluster_addr":"172.21.15.188:6803/3924606902","heartbeat_back_addr":"172.21.15.188:6807/3924606902","heartbeat_front_addr":"172.21.15.188:6805/3924606902","state":["exists","up"]},{"osd":5,"uuid":"3cec8a32-7698-4c36-8e89-e4ba11934f33","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6813","nonce":186841544},{"type":"v1","addr":"172.21.15.188:6815","nonce":186841544}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6817","nonce":186841544},{"type":"v1","addr":"172.21.15.188:6818","nonce":186841544}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6822","nonce":186841544},{"type":"v1","addr":"172.21.15.188:6823","nonce":186841544}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6820","nonce":186841544},{"type":"v1","addr":"172.21.15.188:6821","nonce":186841544}]},"public_addr":"172.21.15.188:6815/186841544","cluster_addr":"172.21.15.188:6818/186841544","heartbeat_back_addr":"172.21.15.188:6823/186841544","heartbeat_front_addr":"172.21.15.188:6821/186841544","state":["exists","up"]},{"osd":6,"uuid":"58a65d5d-e090-473a-81b4-ad38e85029f9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6808","nonce":2153499405},{"type":"v1","addr":"172.21.15.188:6809","nonce":2153499405}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6810","nonce":2153499405},{"type":"v1","addr":"172.21.15.188:6811","nonce":2153499405}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6816","nonce":2153499405},{"type":"v1","addr":"172.21.15.188:6819","nonce":2153499405}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6812","nonce":2153499405},{"type":"v1","addr":"172.21.15.188:6814","nonce":2153499405}]},"public_addr":"172.21.15.188:6809/2153499405","cluster_addr":"172.21.15.188:6811/2153499405","heartbeat_back_addr":"172.21.15.188:6819/2153499405","heartbeat_front_addr":"172.21.15.188:6814/2153499405","state":["exists","up"]},{"osd":7,"uuid":"ea5b7f70-3554-4cf8-ac30-f7c59b80678d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6824","nonce":2722096729},{"type":"v1","addr":"172.21.15.188:6825","nonce":2722096729}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6826","nonce":2722096729},{"type":"v1","addr":"172.21.15.188:6827","nonce":2722096729}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6830","nonce":2722096729},{"type":"v1","addr":"172.21.15.188:6831","nonce":2722096729}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6828","nonce":2722096729},{"type":"v1","addr":"172.21.15.188:6829","nonce":2722096729}]},"public_addr":"172.21.15.188:6825/2722096729","cluster_addr":"172.21.15.188:6827/2722096729","heartbeat_back_addr":"172.21.15.188:6831/2722096729","heartbeat_front_addr":"172.21.15.188:6829/2722096729","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-26T07:17:36.995032+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-26T07:17:36.932266+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-26T07:17:36.829304+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-26T07:17:36.804297+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-26T07:17:36.667892+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-26T07:17:36.964868+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-26T07:17:36.827332+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-26T07:17:36.893179+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-11-26T07:19:11.972 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:11.968+0000 7f5fb125a640 1 -- 172.21.15.143:0/645969230 >> [v2:172.21.15.143:6832/2150557544,v1:172.21.15.143:6833/2150557544] conn(0x7f5f8c066500 msgr2=0x7f5f8c0689c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:19:11.972 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:11.968+0000 7f5fb125a640 1 --2- 172.21.15.143:0/645969230 >> [v2:172.21.15.143:6832/2150557544,v1:172.21.15.143:6833/2150557544] conn(0x7f5f8c066500 0x7f5f8c0689c0 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7f5f940af000 tx=0x7f5f940a7070 comp rx=0 tx=0).stop 2023-11-26T07:19:11.972 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:11.968+0000 7f5fb125a640 1 -- 172.21.15.143:0/645969230 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f5fac0b2160 msgr2=0x7f5fac154590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:19:11.972 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:11.968+0000 7f5fb125a640 1 --2- 172.21.15.143:0/645969230 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f5fac0b2160 0x7f5fac154590 secure :-1 s=READY pgs=468 cs=0 l=1 rev1=1 crypto rx=0x7f5f9c00c480 tx=0x7f5f9c00c940 comp rx=0 tx=0).stop 2023-11-26T07:19:11.972 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:11.968+0000 7f5fb125a640 1 -- 172.21.15.143:0/645969230 shutdown_connections 2023-11-26T07:19:11.972 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:11.968+0000 7f5fb125a640 1 --2- 172.21.15.143:0/645969230 >> [v2:172.21.15.143:6832/2150557544,v1:172.21.15.143:6833/2150557544] conn(0x7f5f8c066500 0x7f5f8c0689c0 unknown :-1 s=CLOSED pgs=240 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:19:11.972 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:11.968+0000 7f5fb125a640 1 --2- 172.21.15.143:0/645969230 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f5fac15d6e0 0x7f5fac06c110 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:19:11.972 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:11.968+0000 7f5fb125a640 1 --2- 172.21.15.143:0/645969230 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f5fac0d1bc0 0x7f5fac0c8dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:19:11.973 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:11.968+0000 7f5fb125a640 1 --2- 172.21.15.143:0/645969230 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f5fac0b2160 0x7f5fac154590 unknown :-1 s=CLOSED pgs=468 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:19:11.973 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:11.968+0000 7f5fb125a640 1 -- 172.21.15.143:0/645969230 >> 172.21.15.143:0/645969230 conn(0x7f5fac09c550 msgr2=0x7f5fac0cfd20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:19:11.973 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:11.968+0000 7f5fb125a640 1 -- 172.21.15.143:0/645969230 shutdown_connections 2023-11-26T07:19:11.973 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:11.968+0000 7f5fb125a640 1 -- 172.21.15.143:0/645969230 wait complete. 2023-11-26T07:19:11.990 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-11-26T07:17:41.758597+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_metadata', 'create_time': '2023-11-26T07:18:50.535934+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '32', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 2.25, 'score_stable': 2.25, 'optimal_score': 1, 'raw_score_acting': 2.25, 'raw_score_stable': 2.25, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 5, 'pool_name': 'cephfs_data', 'create_time': '2023-11-26T07:18:51.008494+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '33', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.3799999952316284, 'score_stable': 1.3799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.3799999952316284, 'raw_score_stable': 1.3799999952316284, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2023-11-26T07:19:11.991 DEBUG:teuthology.orchestra.run.smithi143:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-11-26T07:19:12.131 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:12.128+0000 7f7d2c89a640 1 Processor -- start 2023-11-26T07:19:12.131 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:12.128+0000 7f7d2c89a640 1 -- start start 2023-11-26T07:19:12.133 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:12.132+0000 7f7d2c89a640 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7f7d28143ac0 0x7f7d28070370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:19:12.133 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:12.132+0000 7f7d2c89a640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f7d280cf990 0x7f7d280cd1b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:19:12.133 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:12.132+0000 7f7d2c89a640 1 -- --> v1:172.21.15.143:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7d280cc730 con 0x7f7d280708b0 2023-11-26T07:19:12.133 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:12.132+0000 7f7d2c89a640 1 -- --> v2:172.21.15.143:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7d28143ea0 con 0x7f7d28143ac0 2023-11-26T07:19:12.133 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:12.132+0000 7f7d2c89a640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7d28070ee0 con 0x7f7d280cf990 2023-11-26T07:19:12.133 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:12.132+0000 7f7d2659c640 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7f7d28143ac0 0x7f7d28070370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:19:12.134 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:12.132+0000 7f7d25d9b640 1 --1- >> v1:172.21.15.143:6789/0 conn(0x7f7d280708b0 0x7f7d280cf280 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.143:6789/0 says I am v1:172.21.15.143:38294/0 (socket says 172.21.15.143:38294) 2023-11-26T07:19:12.134 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:12.132+0000 7f7d25d9b640 1 -- 172.21.15.143:0/1171114954 learned_addr learned my addr 172.21.15.143:0/1171114954 (peer_addr_for_me v1:172.21.15.143:0/0) 2023-11-26T07:19:12.134 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:12.132+0000 7f7d26d9d640 1 --2- 172.21.15.143:0/1171114954 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f7d280cf990 0x7f7d280cd1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:19:12.134 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:12.132+0000 7f7d2659c640 1 -- 172.21.15.143:0/1171114954 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f7d280cf990 msgr2=0x7f7d280cd1b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-26T07:19:12.134 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:12.132+0000 7f7d2659c640 1 --2- 172.21.15.143:0/1171114954 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f7d280cf990 0x7f7d280cd1b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:19:12.135 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:12.132+0000 7f7d2659c640 1 -- 172.21.15.143:0/1171114954 >> v1:172.21.15.143:6789/0 conn(0x7f7d280708b0 legacy=0x7f7d280cf280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:19:12.135 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:12.132+0000 7f7d2659c640 1 -- 172.21.15.143:0/1171114954 --> v2:172.21.15.143:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7d280711c0 con 0x7f7d28143ac0 2023-11-26T07:19:12.136 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:12.132+0000 7f7d2659c640 1 --2- 172.21.15.143:0/1171114954 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f7d28143ac0 0x7f7d28070370 secure :-1 s=READY pgs=470 cs=0 l=1 rev1=1 crypto rx=0x7f7d180097c0 tx=0x7f7d1801c480 comp rx=0 tx=0).ready entity=mon.0 client_cookie=708164163b9fec01 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:19:12.137 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:12.136+0000 7f7d2559a640 1 -- 172.21.15.143:0/1171114954 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7d180ad640 con 0x7f7d28143ac0 2023-11-26T07:19:12.137 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:12.136+0000 7f7d2559a640 1 -- 172.21.15.143:0/1171114954 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7d180ad7a0 con 0x7f7d28143ac0 2023-11-26T07:19:12.137 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:12.136+0000 7f7d2559a640 1 -- 172.21.15.143:0/1171114954 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7d180adb30 con 0x7f7d28143ac0 2023-11-26T07:19:12.138 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:12.136+0000 7f7d2c89a640 1 -- 172.21.15.143:0/1171114954 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f7d28143ac0 msgr2=0x7f7d28070370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:19:12.138 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:12.136+0000 7f7d2c89a640 1 --2- 172.21.15.143:0/1171114954 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f7d28143ac0 0x7f7d28070370 secure :-1 s=READY pgs=470 cs=0 l=1 rev1=1 crypto rx=0x7f7d180097c0 tx=0x7f7d1801c480 comp rx=0 tx=0).stop 2023-11-26T07:19:12.138 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:12.136+0000 7f7d2c89a640 1 -- 172.21.15.143:0/1171114954 shutdown_connections 2023-11-26T07:19:12.138 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:12.136+0000 7f7d2c89a640 1 --2- 172.21.15.143:0/1171114954 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f7d280cf990 0x7f7d280cd1b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:19:12.138 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:12.136+0000 7f7d2c89a640 1 --2- 172.21.15.143:0/1171114954 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f7d28143ac0 0x7f7d28070370 unknown :-1 s=CLOSED pgs=470 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:19:12.138 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:12.136+0000 7f7d2c89a640 1 -- 172.21.15.143:0/1171114954 >> 172.21.15.143:0/1171114954 conn(0x7f7d2806a890 msgr2=0x7f7d2806aca0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:19:12.138 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:12.136+0000 7f7d2c89a640 1 -- 172.21.15.143:0/1171114954 shutdown_connections 2023-11-26T07:19:12.138 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:12.136+0000 7f7d2c89a640 1 -- 172.21.15.143:0/1171114954 wait complete. 2023-11-26T07:19:12.139 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:12.136+0000 7f7d2c89a640 1 Processor -- start 2023-11-26T07:19:12.139 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:12.136+0000 7f7d2c89a640 1 -- start start 2023-11-26T07:19:12.141 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:12.136+0000 7f7d2c89a640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f7d280708b0 0x7f7d2813fd20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:19:12.141 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:12.136+0000 7f7d2c89a640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f7d280cf990 0x7f7d28141130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:19:12.141 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:12.136+0000 7f7d2c89a640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f7d28143ac0 0x7f7d28159140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:19:12.141 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:12.136+0000 7f7d2c89a640 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7d2813e9a0 con 0x7f7d28143ac0 2023-11-26T07:19:12.141 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:12.136+0000 7f7d2c89a640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7d2813ec80 con 0x7f7d280708b0 2023-11-26T07:19:12.141 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:12.136+0000 7f7d2c89a640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7d2813ef60 con 0x7f7d280cf990 2023-11-26T07:19:12.141 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:12.136+0000 7f7d26d9d640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f7d28143ac0 0x7f7d28159140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:19:12.141 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:12.140+0000 7f7d25d9b640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f7d280cf990 0x7f7d28141130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:19:12.141 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:12.140+0000 7f7d26d9d640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f7d28143ac0 0x7f7d28159140 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:48668/0 (socket says 172.21.15.143:48668) 2023-11-26T07:19:12.142 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:12.140+0000 7f7d26d9d640 1 -- 172.21.15.143:0/1885008650 learned_addr learned my addr 172.21.15.143:0/1885008650 (peer_addr_for_me v2:172.21.15.143:0/0) 2023-11-26T07:19:12.142 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:12.140+0000 7f7d2659c640 1 --2- 172.21.15.143:0/1885008650 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f7d280708b0 0x7f7d2813fd20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:19:12.142 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:12.140+0000 7f7d26d9d640 1 -- 172.21.15.143:0/1885008650 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f7d280cf990 msgr2=0x7f7d28141130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:19:12.142 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:12.140+0000 7f7d26d9d640 1 --2- 172.21.15.143:0/1885008650 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f7d280cf990 0x7f7d28141130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:19:12.142 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:12.140+0000 7f7d26d9d640 1 -- 172.21.15.143:0/1885008650 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f7d280708b0 msgr2=0x7f7d2813fd20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:19:12.142 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:12.140+0000 7f7d26d9d640 1 --2- 172.21.15.143:0/1885008650 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f7d280708b0 0x7f7d2813fd20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:19:12.142 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:12.140+0000 7f7d26d9d640 1 -- 172.21.15.143:0/1885008650 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7d18009420 con 0x7f7d28143ac0 2023-11-26T07:19:12.143 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:12.140+0000 7f7d26d9d640 1 --2- 172.21.15.143:0/1885008650 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f7d28143ac0 0x7f7d28159140 secure :-1 s=READY pgs=471 cs=0 l=1 rev1=1 crypto rx=0x7f7d20007e40 tx=0x7f7d2000e4b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:19:12.143 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:12.140+0000 7f7d16ffd640 1 -- 172.21.15.143:0/1885008650 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7d20014620 con 0x7f7d28143ac0 2023-11-26T07:19:12.143 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:12.140+0000 7f7d16ffd640 1 -- 172.21.15.143:0/1885008650 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7d2000ee90 con 0x7f7d28143ac0 2023-11-26T07:19:12.143 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:12.140+0000 7f7d2c89a640 1 -- 172.21.15.143:0/1885008650 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7d2813f240 con 0x7f7d28143ac0 2023-11-26T07:19:12.143 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:12.140+0000 7f7d16ffd640 1 -- 172.21.15.143:0/1885008650 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7d200149e0 con 0x7f7d28143ac0 2023-11-26T07:19:12.144 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:12.140+0000 7f7d2c89a640 1 -- 172.21.15.143:0/1885008650 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7d2813f800 con 0x7f7d28143ac0 2023-11-26T07:19:12.144 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:12.140+0000 7f7d2c89a640 1 -- 172.21.15.143:0/1885008650 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7d28067680 con 0x7f7d28143ac0 2023-11-26T07:19:12.144 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:12.140+0000 7f7d16ffd640 1 -- 172.21.15.143:0/1885008650 <== mon.0 v2:172.21.15.143:3300/0 4 ==== mgrmap(e 6) v1 ==== 85903+0+0 (secure 0 0 0) 0x7f7d20014b40 con 0x7f7d28143ac0 2023-11-26T07:19:12.144 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:12.140+0000 7f7d16ffd640 1 --2- 172.21.15.143:0/1885008650 >> [v2:172.21.15.143:6832/2150557544,v1:172.21.15.143:6833/2150557544] conn(0x7f7d08066500 0x7f7d080689c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:19:12.144 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:12.140+0000 7f7d16ffd640 1 -- 172.21.15.143:0/1885008650 <== mon.0 v2:172.21.15.143:3300/0 5 ==== osd_map(33..33 src has 1..33) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f7d20084650 con 0x7f7d28143ac0 2023-11-26T07:19:12.144 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:12.140+0000 7f7d2659c640 1 --2- 172.21.15.143:0/1885008650 >> [v2:172.21.15.143:6832/2150557544,v1:172.21.15.143:6833/2150557544] conn(0x7f7d08066500 0x7f7d080689c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:19:12.145 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:12.144+0000 7f7d2659c640 1 --2- 172.21.15.143:0/1885008650 >> [v2:172.21.15.143:6832/2150557544,v1:172.21.15.143:6833/2150557544] conn(0x7f7d08066500 0x7f7d080689c0 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7f7d180af000 tx=0x7f7d18008040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:19:12.148 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:12.144+0000 7f7d16ffd640 1 -- 172.21.15.143:0/1885008650 <== mon.0 v2:172.21.15.143:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172649 (secure 0 0 0) 0x7f7d20051c20 con 0x7f7d28143ac0 2023-11-26T07:19:12.298 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:12.296+0000 7f7d2c89a640 1 -- 172.21.15.143:0/1885008650 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f7d2806af30 con 0x7f7d28143ac0 2023-11-26T07:19:12.298 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:12.296+0000 7f7d16ffd640 1 -- 172.21.15.143:0/1885008650 <== mon.0 v2:172.21.15.143:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v33) v1 ==== 93+0+10 (secure 0 0 0) 0x7f7d20055240 con 0x7f7d28143ac0 2023-11-26T07:19:12.299 INFO:teuthology.orchestra.run.smithi143.stdout:pg_num: 1 2023-11-26T07:19:12.300 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:12.296+0000 7f7d2c89a640 1 -- 172.21.15.143:0/1885008650 >> [v2:172.21.15.143:6832/2150557544,v1:172.21.15.143:6833/2150557544] conn(0x7f7d08066500 msgr2=0x7f7d080689c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:19:12.300 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:12.296+0000 7f7d2c89a640 1 --2- 172.21.15.143:0/1885008650 >> [v2:172.21.15.143:6832/2150557544,v1:172.21.15.143:6833/2150557544] conn(0x7f7d08066500 0x7f7d080689c0 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7f7d180af000 tx=0x7f7d18008040 comp rx=0 tx=0).stop 2023-11-26T07:19:12.301 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:12.296+0000 7f7d2c89a640 1 -- 172.21.15.143:0/1885008650 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f7d28143ac0 msgr2=0x7f7d28159140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:19:12.301 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:12.296+0000 7f7d2c89a640 1 --2- 172.21.15.143:0/1885008650 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f7d28143ac0 0x7f7d28159140 secure :-1 s=READY pgs=471 cs=0 l=1 rev1=1 crypto rx=0x7f7d20007e40 tx=0x7f7d2000e4b0 comp rx=0 tx=0).stop 2023-11-26T07:19:12.301 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:12.296+0000 7f7d2c89a640 1 -- 172.21.15.143:0/1885008650 shutdown_connections 2023-11-26T07:19:12.301 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:12.296+0000 7f7d2c89a640 1 --2- 172.21.15.143:0/1885008650 >> [v2:172.21.15.143:6832/2150557544,v1:172.21.15.143:6833/2150557544] conn(0x7f7d08066500 0x7f7d080689c0 unknown :-1 s=CLOSED pgs=241 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:19:12.301 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:12.296+0000 7f7d2c89a640 1 --2- 172.21.15.143:0/1885008650 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f7d28143ac0 0x7f7d28159140 unknown :-1 s=CLOSED pgs=471 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:19:12.301 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:12.296+0000 7f7d2c89a640 1 --2- 172.21.15.143:0/1885008650 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f7d280cf990 0x7f7d28141130 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:19:12.301 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:12.296+0000 7f7d2c89a640 1 --2- 172.21.15.143:0/1885008650 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f7d280708b0 0x7f7d2813fd20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:19:12.301 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:12.296+0000 7f7d2c89a640 1 -- 172.21.15.143:0/1885008650 >> 172.21.15.143:0/1885008650 conn(0x7f7d2806a890 msgr2=0x7f7d280ad390 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:19:12.301 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:12.296+0000 7f7d2c89a640 1 -- 172.21.15.143:0/1885008650 shutdown_connections 2023-11-26T07:19:12.301 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:12.296+0000 7f7d2c89a640 1 -- 172.21.15.143:0/1885008650 wait complete. 2023-11-26T07:19:12.319 DEBUG:teuthology.orchestra.run.smithi143:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-11-26T07:19:12.459 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:12.456+0000 7fd4a4279640 1 Processor -- start 2023-11-26T07:19:12.459 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:12.456+0000 7fd4a4279640 1 -- start start 2023-11-26T07:19:12.462 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:12.456+0000 7fd4a4279640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fd49c13e290 0x7fd49c0cf280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:19:12.462 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:12.456+0000 7fd4a4279640 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7fd49c0cf7c0 0x7fd49c15efa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:19:12.462 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:12.456+0000 7fd4a4279640 1 -- --> v1:172.21.15.143:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd49c06c520 con 0x7fd49c15f4e0 2023-11-26T07:19:12.462 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:12.456+0000 7fd4a4279640 1 -- --> v2:172.21.15.188:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd49c0cfc40 con 0x7fd49c0cf7c0 2023-11-26T07:19:12.462 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:12.456+0000 7fd4a4279640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd49c06c800 con 0x7fd49c13e290 2023-11-26T07:19:12.463 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:12.456+0000 7fd4a2816640 1 --1- >> v1:172.21.15.143:6789/0 conn(0x7fd49c15f4e0 0x7fd49c1408e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.143:6789/0 says I am v1:172.21.15.143:38304/0 (socket says 172.21.15.143:38304) 2023-11-26T07:19:12.463 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:12.456+0000 7fd4a2816640 1 -- 172.21.15.143:0/925882472 learned_addr learned my addr 172.21.15.143:0/925882472 (peer_addr_for_me v1:172.21.15.143:0/0) 2023-11-26T07:19:12.463 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:12.460+0000 7fd4a1814640 1 --2- 172.21.15.143:0/925882472 >> v2:172.21.15.188:3300/0 conn(0x7fd49c0cf7c0 0x7fd49c15efa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:19:12.463 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:12.460+0000 7fd4a2015640 1 --2- 172.21.15.143:0/925882472 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fd49c13e290 0x7fd49c0cf280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:19:12.463 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:12.460+0000 7fd4a1013640 1 -- 172.21.15.143:0/925882472 <== mon.0 v1:172.21.15.143:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1544943946 0 0) 0x7fd49c06c520 con 0x7fd49c15f4e0 2023-11-26T07:19:12.463 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:12.460+0000 7fd4a1013640 1 -- 172.21.15.143:0/925882472 --> v1:172.21.15.143:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fd488003530 con 0x7fd49c15f4e0 2023-11-26T07:19:12.463 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:12.460+0000 7fd4a1013640 1 -- 172.21.15.143:0/925882472 <== mon.0 v1:172.21.15.143:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1497423975 0 0) 0x7fd488003530 con 0x7fd49c15f4e0 2023-11-26T07:19:12.463 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:12.460+0000 7fd4a1013640 1 -- 172.21.15.143:0/925882472 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fd49c13e290 msgr2=0x7fd49c0cf280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-26T07:19:12.463 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:12.460+0000 7fd4a1013640 1 --2- 172.21.15.143:0/925882472 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fd49c13e290 0x7fd49c0cf280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:19:12.463 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:12.460+0000 7fd4a1013640 1 -- 172.21.15.143:0/925882472 >> v2:172.21.15.188:3300/0 conn(0x7fd49c0cf7c0 msgr2=0x7fd49c15efa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-26T07:19:12.463 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:12.460+0000 7fd4a1013640 1 --2- 172.21.15.143:0/925882472 >> v2:172.21.15.188:3300/0 conn(0x7fd49c0cf7c0 0x7fd49c15efa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:19:12.463 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:12.460+0000 7fd4a1013640 1 -- 172.21.15.143:0/925882472 --> v1:172.21.15.143:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd49c06cae0 con 0x7fd49c15f4e0 2023-11-26T07:19:12.464 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:12.460+0000 7fd4a1013640 1 -- 172.21.15.143:0/925882472 <== mon.0 v1:172.21.15.143:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2137882523 0 0) 0x7fd494001a00 con 0x7fd49c15f4e0 2023-11-26T07:19:12.464 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:12.460+0000 7fd4a1013640 1 -- 172.21.15.143:0/925882472 >> v1:172.21.15.143:6789/0 conn(0x7fd49c15f4e0 legacy=0x7fd49c1408e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:19:12.464 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:12.460+0000 7fd4a1013640 1 --2- 172.21.15.143:0/925882472 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fd488003980 0x7fd488003da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:19:12.464 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:12.460+0000 7fd4a1013640 1 --2- 172.21.15.143:0/925882472 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fd49c13e290 0x7fd4880047d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:19:12.464 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:12.460+0000 7fd4a1013640 1 --2- 172.21.15.143:0/925882472 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fd488004d10 0x7fd488005190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:19:12.464 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:12.460+0000 7fd4a1013640 1 -- 172.21.15.143:0/925882472 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd49c0cfc40 con 0x7fd488003980 2023-11-26T07:19:12.464 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:12.460+0000 7fd4a1013640 1 -- 172.21.15.143:0/925882472 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd49c06c800 con 0x7fd488004d10 2023-11-26T07:19:12.464 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:12.460+0000 7fd4a1013640 1 -- 172.21.15.143:0/925882472 --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd48800b990 con 0x7fd49c13e290 2023-11-26T07:19:12.464 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:12.460+0000 7fd4a2015640 1 --2- 172.21.15.143:0/925882472 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fd488003980 0x7fd488003da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:19:12.464 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:12.460+0000 7fd4a1814640 1 --2- 172.21.15.143:0/925882472 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fd49c13e290 0x7fd4880047d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:19:12.464 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:12.460+0000 7fd4a2816640 1 --2- 172.21.15.143:0/925882472 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fd488004d10 0x7fd488005190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:19:12.464 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:12.460+0000 7fd4a2015640 1 -- 172.21.15.143:0/925882472 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fd49c13e290 msgr2=0x7fd4880047d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-26T07:19:12.464 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:12.460+0000 7fd4a2015640 1 --2- 172.21.15.143:0/925882472 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fd49c13e290 0x7fd4880047d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:19:12.465 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:12.460+0000 7fd4a2015640 1 -- 172.21.15.143:0/925882472 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fd488004d10 msgr2=0x7fd488005190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-26T07:19:12.465 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:12.460+0000 7fd4a2015640 1 --2- 172.21.15.143:0/925882472 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fd488004d10 0x7fd488005190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:19:12.465 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:12.460+0000 7fd4a2015640 1 -- 172.21.15.143:0/925882472 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd488003530 con 0x7fd488003980 2023-11-26T07:19:12.465 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:12.460+0000 7fd4a2816640 1 --2- 172.21.15.143:0/925882472 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fd488004d10 0x7fd488005190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-26T07:19:12.465 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:12.464+0000 7fd4a2015640 1 --2- 172.21.15.143:0/925882472 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fd488003980 0x7fd488003da0 secure :-1 s=READY pgs=473 cs=0 l=1 rev1=1 crypto rx=0x7fd498009ae0 tx=0x7fd49801ca20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1c5a8b1287570995 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:19:12.468 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:12.464+0000 7fd4a1013640 1 -- 172.21.15.143:0/925882472 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd4980a55f0 con 0x7fd488003980 2023-11-26T07:19:12.468 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:12.464+0000 7fd4a1013640 1 -- 172.21.15.143:0/925882472 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd4980a5750 con 0x7fd488003980 2023-11-26T07:19:12.468 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:12.464+0000 7fd4a1013640 1 -- 172.21.15.143:0/925882472 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd4980a5b00 con 0x7fd488003980 2023-11-26T07:19:12.469 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:12.464+0000 7fd4a4279640 1 -- 172.21.15.143:0/925882472 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fd488003980 msgr2=0x7fd488003da0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:19:12.469 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:12.464+0000 7fd4a4279640 1 --2- 172.21.15.143:0/925882472 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fd488003980 0x7fd488003da0 secure :-1 s=READY pgs=473 cs=0 l=1 rev1=1 crypto rx=0x7fd498009ae0 tx=0x7fd49801ca20 comp rx=0 tx=0).stop 2023-11-26T07:19:12.469 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:12.464+0000 7fd4a2816640 1 -- 172.21.15.143:0/925882472 reap_dead start 2023-11-26T07:19:12.469 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:12.468+0000 7fd4a4279640 1 -- 172.21.15.143:0/925882472 shutdown_connections 2023-11-26T07:19:12.469 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:12.468+0000 7fd4a4279640 1 -- 172.21.15.143:0/925882472 >> 172.21.15.143:0/925882472 conn(0x7fd49c06a890 msgr2=0x7fd49c06aca0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:19:12.469 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:12.468+0000 7fd4a4279640 1 -- 172.21.15.143:0/925882472 shutdown_connections 2023-11-26T07:19:12.469 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:12.468+0000 7fd4a4279640 1 -- 172.21.15.143:0/925882472 wait complete. 2023-11-26T07:19:12.470 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:12.468+0000 7fd4a4279640 1 Processor -- start 2023-11-26T07:19:12.471 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:12.468+0000 7fd4a4279640 1 -- start start 2023-11-26T07:19:12.472 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:12.468+0000 7fd4a4279640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fd488003980 0x7fd49c0b3710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:19:12.472 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:12.468+0000 7fd4a4279640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fd49c15b580 0x7fd49c0b0de0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:19:12.472 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:12.468+0000 7fd4a4279640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fd49c15dff0 0x7fd49c13cd90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:19:12.472 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:12.468+0000 7fd4a4279640 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd49c15cf90 con 0x7fd49c15b580 2023-11-26T07:19:12.472 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:12.468+0000 7fd4a4279640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd49c15ba00 con 0x7fd49c15dff0 2023-11-26T07:19:12.472 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:12.468+0000 7fd4a4279640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd49c15d230 con 0x7fd488003980 2023-11-26T07:19:12.473 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:12.468+0000 7fd4a2816640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fd49c15dff0 0x7fd49c13cd90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:19:12.473 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:12.468+0000 7fd4a2816640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fd49c15dff0 0x7fd49c13cd90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.188:3300/0 says I am v2:172.21.15.143:56234/0 (socket says 172.21.15.143:56234) 2023-11-26T07:19:12.474 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:12.468+0000 7fd4a2816640 1 -- 172.21.15.143:0/2178065132 learned_addr learned my addr 172.21.15.143:0/2178065132 (peer_addr_for_me v2:172.21.15.143:0/0) 2023-11-26T07:19:12.474 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:12.468+0000 7fd4a1814640 1 --2- 172.21.15.143:0/2178065132 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fd49c15b580 0x7fd49c0b0de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:19:12.474 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:12.472+0000 7fd4a2015640 1 --2- 172.21.15.143:0/2178065132 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fd488003980 0x7fd49c0b3710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:19:12.474 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:12.472+0000 7fd4a2816640 1 -- 172.21.15.143:0/2178065132 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fd488003980 msgr2=0x7fd49c0b3710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:19:12.474 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:12.472+0000 7fd4a2816640 1 --2- 172.21.15.143:0/2178065132 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fd488003980 0x7fd49c0b3710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:19:12.474 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:12.472+0000 7fd4a2816640 1 -- 172.21.15.143:0/2178065132 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fd49c15b580 msgr2=0x7fd49c0b0de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:19:12.474 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:12.472+0000 7fd4a2816640 1 --2- 172.21.15.143:0/2178065132 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fd49c15b580 0x7fd49c0b0de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:19:12.474 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:12.472+0000 7fd4a2816640 1 -- 172.21.15.143:0/2178065132 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd49c15d510 con 0x7fd49c15dff0 2023-11-26T07:19:12.474 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:12.472+0000 7fd4a2015640 1 --2- 172.21.15.143:0/2178065132 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fd488003980 0x7fd49c0b3710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-26T07:19:12.475 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:12.472+0000 7fd4a2816640 1 --2- 172.21.15.143:0/2178065132 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fd49c15dff0 0x7fd49c13cd90 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7fd494001f80 tx=0x7fd49400e530 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:19:12.476 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:12.472+0000 7fd492ffd640 1 -- 172.21.15.143:0/2178065132 <== mon.1 v2:172.21.15.188:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd49400eb90 con 0x7fd49c15dff0 2023-11-26T07:19:12.476 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:12.472+0000 7fd492ffd640 1 -- 172.21.15.143:0/2178065132 <== mon.1 v2:172.21.15.188:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd49400ecf0 con 0x7fd49c15dff0 2023-11-26T07:19:12.477 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:12.472+0000 7fd492ffd640 1 -- 172.21.15.143:0/2178065132 <== mon.1 v2:172.21.15.188:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd494013620 con 0x7fd49c15dff0 2023-11-26T07:19:12.477 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:12.472+0000 7fd4a4279640 1 -- 172.21.15.143:0/2178065132 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd49c15d7f0 con 0x7fd49c15dff0 2023-11-26T07:19:12.477 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:12.472+0000 7fd4a4279640 1 -- 172.21.15.143:0/2178065132 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd49c0b1320 con 0x7fd49c15dff0 2023-11-26T07:19:12.477 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:12.476+0000 7fd4a4279640 1 -- 172.21.15.143:0/2178065132 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd49c067680 con 0x7fd49c15dff0 2023-11-26T07:19:12.479 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:12.476+0000 7fd492ffd640 1 -- 172.21.15.143:0/2178065132 <== mon.1 v2:172.21.15.188:3300/0 4 ==== mgrmap(e 7) v1 ==== 86005+0+0 (secure 0 0 0) 0x7fd494013780 con 0x7fd49c15dff0 2023-11-26T07:19:12.480 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:12.476+0000 7fd492ffd640 1 --2- 172.21.15.143:0/2178065132 >> [v2:172.21.15.143:6832/2150557544,v1:172.21.15.143:6833/2150557544] conn(0x7fd480066650 0x7fd480068b10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:19:12.480 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:12.476+0000 7fd492ffd640 1 -- 172.21.15.143:0/2178065132 <== mon.1 v2:172.21.15.188:3300/0 5 ==== osd_map(33..33 src has 1..33) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fd494085030 con 0x7fd49c15dff0 2023-11-26T07:19:12.480 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:12.476+0000 7fd4a2015640 1 --2- 172.21.15.143:0/2178065132 >> [v2:172.21.15.143:6832/2150557544,v1:172.21.15.143:6833/2150557544] conn(0x7fd480066650 0x7fd480068b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:19:12.481 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:12.480+0000 7fd4a2015640 1 --2- 172.21.15.143:0/2178065132 >> [v2:172.21.15.143:6832/2150557544,v1:172.21.15.143:6833/2150557544] conn(0x7fd480066650 0x7fd480068b10 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7fd4980af000 tx=0x7fd4980046d0 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:19:12.483 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:12.480+0000 7fd492ffd640 1 -- 172.21.15.143:0/2178065132 <== mon.1 v2:172.21.15.188:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172649 (secure 0 0 0) 0x7fd494052290 con 0x7fd49c15dff0 2023-11-26T07:19:12.631 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:12.628+0000 7fd4a4279640 1 -- 172.21.15.143:0/2178065132 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7fd49c0cc310 con 0x7fd49c15dff0 2023-11-26T07:19:12.632 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:12.628+0000 7fd492ffd640 1 -- 172.21.15.143:0/2178065132 <== mon.1 v2:172.21.15.188:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v33) v1 ==== 104+0+11 (secure 0 0 0) 0x7fd4940558b0 con 0x7fd49c15dff0 2023-11-26T07:19:12.632 INFO:teuthology.orchestra.run.smithi143.stdout:pg_num: 32 2023-11-26T07:19:12.634 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:12.632+0000 7fd4a4279640 1 -- 172.21.15.143:0/2178065132 >> [v2:172.21.15.143:6832/2150557544,v1:172.21.15.143:6833/2150557544] conn(0x7fd480066650 msgr2=0x7fd480068b10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:19:12.634 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:12.632+0000 7fd4a4279640 1 --2- 172.21.15.143:0/2178065132 >> [v2:172.21.15.143:6832/2150557544,v1:172.21.15.143:6833/2150557544] conn(0x7fd480066650 0x7fd480068b10 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7fd4980af000 tx=0x7fd4980046d0 comp rx=0 tx=0).stop 2023-11-26T07:19:12.634 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:12.632+0000 7fd4a4279640 1 -- 172.21.15.143:0/2178065132 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fd49c15dff0 msgr2=0x7fd49c13cd90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:19:12.634 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:12.632+0000 7fd4a4279640 1 --2- 172.21.15.143:0/2178065132 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fd49c15dff0 0x7fd49c13cd90 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7fd494001f80 tx=0x7fd49400e530 comp rx=0 tx=0).stop 2023-11-26T07:19:12.634 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:12.632+0000 7fd4a4279640 1 -- 172.21.15.143:0/2178065132 shutdown_connections 2023-11-26T07:19:12.635 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:12.632+0000 7fd4a4279640 1 --2- 172.21.15.143:0/2178065132 >> [v2:172.21.15.143:6832/2150557544,v1:172.21.15.143:6833/2150557544] conn(0x7fd480066650 0x7fd480068b10 unknown :-1 s=CLOSED pgs=242 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:19:12.635 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:12.632+0000 7fd4a4279640 1 --2- 172.21.15.143:0/2178065132 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fd49c15dff0 0x7fd49c13cd90 unknown :-1 s=CLOSED pgs=204 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:19:12.635 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:12.632+0000 7fd4a4279640 1 --2- 172.21.15.143:0/2178065132 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fd49c15b580 0x7fd49c0b0de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:19:12.635 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:12.632+0000 7fd4a4279640 1 --2- 172.21.15.143:0/2178065132 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fd488003980 0x7fd49c0b3710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:19:12.635 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:12.632+0000 7fd4a4279640 1 -- 172.21.15.143:0/2178065132 >> 172.21.15.143:0/2178065132 conn(0x7fd49c06a890 msgr2=0x7fd49c070d10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:19:12.635 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:12.632+0000 7fd4a4279640 1 -- 172.21.15.143:0/2178065132 shutdown_connections 2023-11-26T07:19:12.635 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:12.632+0000 7fd4a4279640 1 -- 172.21.15.143:0/2178065132 wait complete. 2023-11-26T07:19:12.653 DEBUG:teuthology.orchestra.run.smithi143:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-11-26T07:19:12.793 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:12.792+0000 7f0de25e4640 1 Processor -- start 2023-11-26T07:19:12.793 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:12.792+0000 7f0de25e4640 1 -- start start 2023-11-26T07:19:12.794 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:12.792+0000 7f0de25e4640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f0ddc144720 0x7f0ddc06d300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:19:12.794 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:12.792+0000 7f0de25e4640 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7f0ddc071bf0 0x7f0ddc15bde0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:19:12.795 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:12.792+0000 7f0de25e4640 1 -- --> v1:172.21.15.188:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0ddc06e3f0 con 0x7f0ddc06d840 2023-11-26T07:19:12.795 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:12.792+0000 7f0de25e4640 1 -- --> v2:172.21.15.143:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0ddc0722f0 con 0x7f0ddc071bf0 2023-11-26T07:19:12.795 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:12.792+0000 7f0de25e4640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0ddc06de30 con 0x7f0ddc144720 2023-11-26T07:19:12.795 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:12.792+0000 7f0de1de3640 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7f0ddc071bf0 0x7f0ddc15bde0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:19:12.795 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:12.792+0000 7f0de1de3640 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7f0ddc071bf0 0x7f0ddc15bde0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:48692/0 (socket says 172.21.15.143:48692) 2023-11-26T07:19:12.795 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:12.792+0000 7f0de1de3640 1 -- 172.21.15.143:0/603077735 learned_addr learned my addr 172.21.15.143:0/603077735 (peer_addr_for_me v2:172.21.15.143:0/0) 2023-11-26T07:19:12.796 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:12.792+0000 7f0de15e2640 1 --2- 172.21.15.143:0/603077735 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f0ddc144720 0x7f0ddc06d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:19:12.796 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:12.792+0000 7f0de1de3640 1 -- 172.21.15.143:0/603077735 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f0ddc144720 msgr2=0x7f0ddc06d300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-26T07:19:12.796 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:12.792+0000 7f0de1de3640 1 --2- 172.21.15.143:0/603077735 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f0ddc144720 0x7f0ddc06d300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:19:12.796 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:12.792+0000 7f0de1de3640 1 -- 172.21.15.143:0/603077735 >> v1:172.21.15.188:6789/0 conn(0x7f0ddc06d840 legacy=0x7f0ddc0714e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-26T07:19:12.796 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:12.792+0000 7f0de1de3640 1 -- 172.21.15.143:0/603077735 --> v2:172.21.15.143:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0ddc06e110 con 0x7f0ddc071bf0 2023-11-26T07:19:12.796 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:12.792+0000 7f0de15e2640 1 --2- 172.21.15.143:0/603077735 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f0ddc144720 0x7f0ddc06d300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-26T07:19:12.798 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:12.796+0000 7f0de1de3640 1 --2- 172.21.15.143:0/603077735 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f0ddc071bf0 0x7f0ddc15bde0 secure :-1 s=READY pgs=474 cs=0 l=1 rev1=1 crypto rx=0x7f0dd4009e50 tx=0x7f0dd401c9a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2cb7ae916b6d9bba server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:19:12.798 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:12.796+0000 7f0dd3fff640 1 -- 172.21.15.143:0/603077735 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0dd40a5820 con 0x7f0ddc071bf0 2023-11-26T07:19:12.798 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:12.796+0000 7f0dd3fff640 1 -- 172.21.15.143:0/603077735 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0dd40a5980 con 0x7f0ddc071bf0 2023-11-26T07:19:12.798 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:12.796+0000 7f0dd3fff640 1 -- 172.21.15.143:0/603077735 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0dd40a5d10 con 0x7f0ddc071bf0 2023-11-26T07:19:12.799 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:12.796+0000 7f0de25e4640 1 -- 172.21.15.143:0/603077735 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f0ddc071bf0 msgr2=0x7f0ddc15bde0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:19:12.799 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:12.796+0000 7f0de25e4640 1 --2- 172.21.15.143:0/603077735 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f0ddc071bf0 0x7f0ddc15bde0 secure :-1 s=READY pgs=474 cs=0 l=1 rev1=1 crypto rx=0x7f0dd4009e50 tx=0x7f0dd401c9a0 comp rx=0 tx=0).stop 2023-11-26T07:19:12.799 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:12.796+0000 7f0de25e4640 1 -- 172.21.15.143:0/603077735 shutdown_connections 2023-11-26T07:19:12.799 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:12.796+0000 7f0de25e4640 1 --2- 172.21.15.143:0/603077735 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f0ddc071bf0 0x7f0ddc15bde0 unknown :-1 s=CLOSED pgs=474 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:19:12.800 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:12.796+0000 7f0de25e4640 1 --2- 172.21.15.143:0/603077735 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f0ddc144720 0x7f0ddc06d300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:19:12.800 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:12.796+0000 7f0de25e4640 1 -- 172.21.15.143:0/603077735 >> 172.21.15.143:0/603077735 conn(0x7f0ddc06a890 msgr2=0x7f0ddc06aca0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:19:12.800 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:12.796+0000 7f0de25e4640 1 -- 172.21.15.143:0/603077735 shutdown_connections 2023-11-26T07:19:12.800 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:12.796+0000 7f0de25e4640 1 -- 172.21.15.143:0/603077735 wait complete. 2023-11-26T07:19:12.800 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:12.800+0000 7f0de25e4640 1 Processor -- start 2023-11-26T07:19:12.801 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:12.800+0000 7f0de25e4640 1 -- start start 2023-11-26T07:19:12.802 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:12.800+0000 7f0de25e4640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f0ddc06d840 0x7f0ddc13d790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:19:12.802 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:12.800+0000 7f0de25e4640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f0ddc071bf0 0x7f0ddc0b2f90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:19:12.802 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:12.800+0000 7f0de25e4640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f0ddc144720 0x7f0ddc0c82d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:19:12.802 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:12.800+0000 7f0de25e4640 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0ddc0c5330 con 0x7f0ddc144720 2023-11-26T07:19:12.803 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:12.800+0000 7f0de25e4640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0ddc0c5050 con 0x7f0ddc06d840 2023-11-26T07:19:12.803 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:12.800+0000 7f0de25e4640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0ddc0b2790 con 0x7f0ddc071bf0 2023-11-26T07:19:12.803 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:12.800+0000 7f0de1de3640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f0ddc144720 0x7f0ddc0c82d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:19:12.803 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:12.800+0000 7f0de1de3640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f0ddc144720 0x7f0ddc0c82d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:48698/0 (socket says 172.21.15.143:48698) 2023-11-26T07:19:12.803 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:12.800+0000 7f0de1de3640 1 -- 172.21.15.143:0/1615155147 learned_addr learned my addr 172.21.15.143:0/1615155147 (peer_addr_for_me v2:172.21.15.143:0/0) 2023-11-26T07:19:12.803 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:12.800+0000 7f0de0de1640 1 --2- 172.21.15.143:0/1615155147 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f0ddc071bf0 0x7f0ddc0b2f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:19:12.803 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:12.800+0000 7f0de15e2640 1 --2- 172.21.15.143:0/1615155147 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f0ddc06d840 0x7f0ddc13d790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:19:12.803 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:12.800+0000 7f0de1de3640 1 -- 172.21.15.143:0/1615155147 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f0ddc071bf0 msgr2=0x7f0ddc0b2f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:19:12.803 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:12.800+0000 7f0de1de3640 1 --2- 172.21.15.143:0/1615155147 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f0ddc071bf0 0x7f0ddc0b2f90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:19:12.803 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:12.800+0000 7f0de1de3640 1 -- 172.21.15.143:0/1615155147 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f0ddc06d840 msgr2=0x7f0ddc13d790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:19:12.803 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:12.800+0000 7f0de1de3640 1 --2- 172.21.15.143:0/1615155147 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f0ddc06d840 0x7f0ddc13d790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:19:12.803 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:12.800+0000 7f0de1de3640 1 -- 172.21.15.143:0/1615155147 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0dd4009b30 con 0x7f0ddc144720 2023-11-26T07:19:12.803 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:12.800+0000 7f0de15e2640 1 --2- 172.21.15.143:0/1615155147 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f0ddc06d840 0x7f0ddc13d790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-26T07:19:12.804 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:12.800+0000 7f0de1de3640 1 --2- 172.21.15.143:0/1615155147 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f0ddc144720 0x7f0ddc0c82d0 secure :-1 s=READY pgs=475 cs=0 l=1 rev1=1 crypto rx=0x7f0dd40af040 tx=0x7f0dd4019740 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:19:12.804 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:12.800+0000 7f0dd1ffb640 1 -- 172.21.15.143:0/1615155147 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0dd40b8670 con 0x7f0ddc144720 2023-11-26T07:19:12.804 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:12.800+0000 7f0dd1ffb640 1 -- 172.21.15.143:0/1615155147 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0dd40b87d0 con 0x7f0ddc144720 2023-11-26T07:19:12.805 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:12.800+0000 7f0de25e4640 1 -- 172.21.15.143:0/1615155147 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0ddc0b2a70 con 0x7f0ddc144720 2023-11-26T07:19:12.805 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:12.800+0000 7f0dd1ffb640 1 -- 172.21.15.143:0/1615155147 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0dd40b8b80 con 0x7f0ddc144720 2023-11-26T07:19:12.805 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:12.800+0000 7f0de25e4640 1 -- 172.21.15.143:0/1615155147 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0ddc13dcd0 con 0x7f0ddc144720 2023-11-26T07:19:12.805 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:12.800+0000 7f0de25e4640 1 -- 172.21.15.143:0/1615155147 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0ddc0b25f0 con 0x7f0ddc144720 2023-11-26T07:19:12.805 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:12.804+0000 7f0dd1ffb640 1 -- 172.21.15.143:0/1615155147 <== mon.0 v2:172.21.15.143:3300/0 4 ==== mgrmap(e 7) v1 ==== 86005+0+0 (secure 0 0 0) 0x7f0dd40b8ce0 con 0x7f0ddc144720 2023-11-26T07:19:12.806 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:12.804+0000 7f0dd1ffb640 1 --2- 172.21.15.143:0/1615155147 >> [v2:172.21.15.143:6832/2150557544,v1:172.21.15.143:6833/2150557544] conn(0x7f0dc0066620 0x7f0dc0068ae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:19:12.806 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:12.804+0000 7f0dd1ffb640 1 -- 172.21.15.143:0/1615155147 <== mon.0 v2:172.21.15.143:3300/0 5 ==== osd_map(33..33 src has 1..33) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f0dd411f980 con 0x7f0ddc144720 2023-11-26T07:19:12.806 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:12.804+0000 7f0de15e2640 1 --2- 172.21.15.143:0/1615155147 >> [v2:172.21.15.143:6832/2150557544,v1:172.21.15.143:6833/2150557544] conn(0x7f0dc0066620 0x7f0dc0068ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:19:12.807 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:12.804+0000 7f0de15e2640 1 --2- 172.21.15.143:0/1615155147 >> [v2:172.21.15.143:6832/2150557544,v1:172.21.15.143:6833/2150557544] conn(0x7f0dc0066620 0x7f0dc0068ae0 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x7f0dd800a9b0 tx=0x7f0dd8008040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:19:12.810 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:12.808+0000 7f0dd1ffb640 1 -- 172.21.15.143:0/1615155147 <== mon.0 v2:172.21.15.143:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172649 (secure 0 0 0) 0x7f0ddc0b25f0 con 0x7f0ddc144720 2023-11-26T07:19:12.959 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:12.956+0000 7f0de25e4640 1 -- 172.21.15.143:0/1615155147 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f0ddc0b2cb0 con 0x7f0ddc144720 2023-11-26T07:19:12.960 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:12.956+0000 7f0dd1ffb640 1 -- 172.21.15.143:0/1615155147 <== mon.0 v2:172.21.15.143:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v33) v1 ==== 100+0+11 (secure 0 0 0) 0x7f0ddc0b2cb0 con 0x7f0ddc144720 2023-11-26T07:19:12.960 INFO:teuthology.orchestra.run.smithi143.stdout:pg_num: 64 2023-11-26T07:19:12.962 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:12.960+0000 7f0de25e4640 1 -- 172.21.15.143:0/1615155147 >> [v2:172.21.15.143:6832/2150557544,v1:172.21.15.143:6833/2150557544] conn(0x7f0dc0066620 msgr2=0x7f0dc0068ae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:19:12.962 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:12.960+0000 7f0de25e4640 1 --2- 172.21.15.143:0/1615155147 >> [v2:172.21.15.143:6832/2150557544,v1:172.21.15.143:6833/2150557544] conn(0x7f0dc0066620 0x7f0dc0068ae0 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x7f0dd800a9b0 tx=0x7f0dd8008040 comp rx=0 tx=0).stop 2023-11-26T07:19:12.962 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:12.960+0000 7f0de25e4640 1 -- 172.21.15.143:0/1615155147 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f0ddc144720 msgr2=0x7f0ddc0c82d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:19:12.962 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:12.960+0000 7f0de25e4640 1 --2- 172.21.15.143:0/1615155147 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f0ddc144720 0x7f0ddc0c82d0 secure :-1 s=READY pgs=475 cs=0 l=1 rev1=1 crypto rx=0x7f0dd40af040 tx=0x7f0dd4019740 comp rx=0 tx=0).stop 2023-11-26T07:19:12.963 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:12.960+0000 7f0de25e4640 1 -- 172.21.15.143:0/1615155147 shutdown_connections 2023-11-26T07:19:12.963 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:12.960+0000 7f0de25e4640 1 --2- 172.21.15.143:0/1615155147 >> [v2:172.21.15.143:6832/2150557544,v1:172.21.15.143:6833/2150557544] conn(0x7f0dc0066620 0x7f0dc0068ae0 unknown :-1 s=CLOSED pgs=243 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:19:12.963 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:12.960+0000 7f0de25e4640 1 --2- 172.21.15.143:0/1615155147 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f0ddc144720 0x7f0ddc0c82d0 unknown :-1 s=CLOSED pgs=475 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:19:12.963 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:12.960+0000 7f0de25e4640 1 --2- 172.21.15.143:0/1615155147 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f0ddc071bf0 0x7f0ddc0b2f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:19:12.963 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:12.960+0000 7f0de25e4640 1 --2- 172.21.15.143:0/1615155147 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f0ddc06d840 0x7f0ddc13d790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:19:12.963 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:12.960+0000 7f0de25e4640 1 -- 172.21.15.143:0/1615155147 >> 172.21.15.143:0/1615155147 conn(0x7f0ddc06a890 msgr2=0x7f0ddc13f9a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:19:12.963 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:12.960+0000 7f0de25e4640 1 -- 172.21.15.143:0/1615155147 shutdown_connections 2023-11-26T07:19:12.963 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:12.960+0000 7f0de25e4640 1 -- 172.21.15.143:0/1615155147 wait complete. 2023-11-26T07:19:12.981 DEBUG:teuthology.orchestra.run.smithi143:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2023-11-26T07:19:13.120 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:13.116+0000 7f5bbfc9c640 1 Processor -- start 2023-11-26T07:19:13.120 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:13.116+0000 7f5bbfc9c640 1 -- start start 2023-11-26T07:19:13.121 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:13.120+0000 7f5bbfc9c640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f5bb813e290 0x7f5bb80cf280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:19:13.121 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:13.120+0000 7f5bbfc9c640 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7f5bb80cf7c0 0x7f5bb815efa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:19:13.122 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:13.120+0000 7f5bbfc9c640 1 -- --> v1:172.21.15.188:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5bb806c520 con 0x7f5bb815f4e0 2023-11-26T07:19:13.122 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:13.120+0000 7f5bbfc9c640 1 -- --> v2:172.21.15.188:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5bb80cfc40 con 0x7f5bb80cf7c0 2023-11-26T07:19:13.122 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:13.120+0000 7f5bbfc9c640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5bb806c800 con 0x7f5bb813e290 2023-11-26T07:19:13.123 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:13.120+0000 7f5bbd237640 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7f5bb80cf7c0 0x7f5bb815efa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:19:13.123 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:13.120+0000 7f5bbda38640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f5bb813e290 0x7f5bb80cf280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:19:13.123 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:13.120+0000 7f5bbe239640 1 --1- >> v1:172.21.15.188:6789/0 conn(0x7f5bb815f4e0 0x7f5bb81408e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.188:6789/0 says I am v1:172.21.15.143:55496/0 (socket says 172.21.15.143:55496) 2023-11-26T07:19:13.123 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:13.120+0000 7f5bbd237640 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7f5bb80cf7c0 0x7f5bb815efa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.188:3300/0 says I am v2:172.21.15.143:56254/0 (socket says 172.21.15.143:56254) 2023-11-26T07:19:13.123 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:13.120+0000 7f5bbe239640 1 -- 172.21.15.143:0/2857063931 learned_addr learned my addr 172.21.15.143:0/2857063931 (peer_addr_for_me v1:172.21.15.143:0/0) 2023-11-26T07:19:13.123 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:13.120+0000 7f5bbd237640 1 -- 172.21.15.143:0/2857063931 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f5bb813e290 msgr2=0x7f5bb80cf280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-26T07:19:13.124 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:13.120+0000 7f5bbd237640 1 --2- 172.21.15.143:0/2857063931 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f5bb813e290 0x7f5bb80cf280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:19:13.124 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:13.120+0000 7f5bbd237640 1 -- 172.21.15.143:0/2857063931 >> v1:172.21.15.188:6789/0 conn(0x7f5bb815f4e0 legacy=0x7f5bb81408e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:19:13.124 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:13.120+0000 7f5bbd237640 1 -- 172.21.15.143:0/2857063931 --> v2:172.21.15.188:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5bb806cae0 con 0x7f5bb80cf7c0 2023-11-26T07:19:13.124 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:13.120+0000 7f5bbca36640 1 -- 172.21.15.143:0/2857063931 <== mon.1 v1:172.21.15.188:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3352879601 0 0) 0x7f5bb806c520 con 0x7f5bb815f4e0 2023-11-26T07:19:13.125 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:13.124+0000 7f5bbd237640 1 --2- 172.21.15.143:0/2857063931 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f5bb80cf7c0 0x7f5bb815efa0 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7f5ba80097c0 tx=0x7f5ba801c450 comp rx=0 tx=0).ready entity=mon.1 client_cookie=461b597fc2b56bdd server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:19:13.126 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:13.124+0000 7f5bbca36640 1 -- 172.21.15.143:0/2857063931 <== mon.1 v2:172.21.15.188:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5ba80ab650 con 0x7f5bb80cf7c0 2023-11-26T07:19:13.126 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:13.124+0000 7f5bbca36640 1 -- 172.21.15.143:0/2857063931 <== mon.1 v2:172.21.15.188:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5ba80ab7b0 con 0x7f5bb80cf7c0 2023-11-26T07:19:13.126 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:13.124+0000 7f5bbca36640 1 -- 172.21.15.143:0/2857063931 <== mon.1 v2:172.21.15.188:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5ba80abb40 con 0x7f5bb80cf7c0 2023-11-26T07:19:13.127 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:13.124+0000 7f5bbfc9c640 1 -- 172.21.15.143:0/2857063931 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f5bb80cf7c0 msgr2=0x7f5bb815efa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:19:13.127 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:13.124+0000 7f5bbfc9c640 1 --2- 172.21.15.143:0/2857063931 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f5bb80cf7c0 0x7f5bb815efa0 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7f5ba80097c0 tx=0x7f5ba801c450 comp rx=0 tx=0).stop 2023-11-26T07:19:13.127 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:13.124+0000 7f5bbfc9c640 1 -- 172.21.15.143:0/2857063931 shutdown_connections 2023-11-26T07:19:13.127 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:13.124+0000 7f5bbfc9c640 1 --2- 172.21.15.143:0/2857063931 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f5bb80cf7c0 0x7f5bb815efa0 unknown :-1 s=CLOSED pgs=207 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:19:13.127 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:13.124+0000 7f5bbfc9c640 1 --2- 172.21.15.143:0/2857063931 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f5bb813e290 0x7f5bb80cf280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:19:13.128 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:13.124+0000 7f5bbfc9c640 1 -- 172.21.15.143:0/2857063931 >> 172.21.15.143:0/2857063931 conn(0x7f5bb806a890 msgr2=0x7f5bb806aca0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:19:13.128 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:13.124+0000 7f5bbfc9c640 1 -- 172.21.15.143:0/2857063931 shutdown_connections 2023-11-26T07:19:13.128 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:13.124+0000 7f5bbfc9c640 1 -- 172.21.15.143:0/2857063931 wait complete. 2023-11-26T07:19:13.128 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:13.124+0000 7f5bbfc9c640 1 Processor -- start 2023-11-26T07:19:13.128 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:13.124+0000 7f5bbfc9c640 1 -- start start 2023-11-26T07:19:13.131 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:13.128+0000 7f5bbfc9c640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f5bb80cf7c0 0x7f5bb8152a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:19:13.131 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:13.128+0000 7f5bbfc9c640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f5bb813e290 0x7f5bb813a040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:19:13.131 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:13.128+0000 7f5bbfc9c640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f5bb815f4e0 0x7f5bb80aeb60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:19:13.131 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:13.128+0000 7f5bbfc9c640 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5bb8139bb0 con 0x7f5bb813e290 2023-11-26T07:19:13.131 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:13.128+0000 7f5bbfc9c640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5bb813a580 con 0x7f5bb815f4e0 2023-11-26T07:19:13.131 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:13.128+0000 7f5bbfc9c640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5bb8138fa0 con 0x7f5bb80cf7c0 2023-11-26T07:19:13.131 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:13.128+0000 7f5bbd237640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f5bb813e290 0x7f5bb813a040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:19:13.131 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:13.128+0000 7f5bbd237640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f5bb813e290 0x7f5bb813a040 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:48710/0 (socket says 172.21.15.143:48710) 2023-11-26T07:19:13.131 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:13.128+0000 7f5bbd237640 1 -- 172.21.15.143:0/1725298333 learned_addr learned my addr 172.21.15.143:0/1725298333 (peer_addr_for_me v2:172.21.15.143:0/0) 2023-11-26T07:19:13.132 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:13.128+0000 7f5bbda38640 1 --2- 172.21.15.143:0/1725298333 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f5bb80cf7c0 0x7f5bb8152a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:19:13.132 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:13.128+0000 7f5bbe239640 1 --2- 172.21.15.143:0/1725298333 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f5bb815f4e0 0x7f5bb80aeb60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:19:13.132 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:13.128+0000 7f5bbd237640 1 -- 172.21.15.143:0/1725298333 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f5bb80cf7c0 msgr2=0x7f5bb8152a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:19:13.132 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:13.128+0000 7f5bbd237640 1 --2- 172.21.15.143:0/1725298333 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f5bb80cf7c0 0x7f5bb8152a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:19:13.132 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:13.128+0000 7f5bbd237640 1 -- 172.21.15.143:0/1725298333 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f5bb815f4e0 msgr2=0x7f5bb80aeb60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:19:13.132 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:13.128+0000 7f5bbd237640 1 --2- 172.21.15.143:0/1725298333 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f5bb815f4e0 0x7f5bb80aeb60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:19:13.132 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:13.128+0000 7f5bbd237640 1 -- 172.21.15.143:0/1725298333 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5ba8009420 con 0x7f5bb813e290 2023-11-26T07:19:13.132 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:13.128+0000 7f5bbd237640 1 --2- 172.21.15.143:0/1725298333 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f5bb813e290 0x7f5bb813a040 secure :-1 s=READY pgs=476 cs=0 l=1 rev1=1 crypto rx=0x7f5ba80ad040 tx=0x7f5ba801ac90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:19:13.132 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:13.128+0000 7f5bae7fc640 1 -- 172.21.15.143:0/1725298333 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5ba80bf610 con 0x7f5bb813e290 2023-11-26T07:19:13.132 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:13.128+0000 7f5bae7fc640 1 -- 172.21.15.143:0/1725298333 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5ba80bf790 con 0x7f5bb813e290 2023-11-26T07:19:13.132 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:13.128+0000 7f5bae7fc640 1 -- 172.21.15.143:0/1725298333 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5ba80bfb40 con 0x7f5bb813e290 2023-11-26T07:19:13.133 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:13.128+0000 7f5bbfc9c640 1 -- 172.21.15.143:0/1725298333 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5bb8139280 con 0x7f5bb813e290 2023-11-26T07:19:13.133 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:13.128+0000 7f5bbfc9c640 1 -- 172.21.15.143:0/1725298333 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5bb8139840 con 0x7f5bb813e290 2023-11-26T07:19:13.133 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:13.128+0000 7f5bbfc9c640 1 -- 172.21.15.143:0/1725298333 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5bb8067680 con 0x7f5bb813e290 2023-11-26T07:19:13.134 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:13.128+0000 7f5bae7fc640 1 -- 172.21.15.143:0/1725298333 <== mon.0 v2:172.21.15.143:3300/0 4 ==== mgrmap(e 7) v1 ==== 86005+0+0 (secure 0 0 0) 0x7f5ba80bfca0 con 0x7f5bb813e290 2023-11-26T07:19:13.134 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:13.132+0000 7f5bae7fc640 1 --2- 172.21.15.143:0/1725298333 >> [v2:172.21.15.143:6832/2150557544,v1:172.21.15.143:6833/2150557544] conn(0x7f5b9c066620 0x7f5b9c068ae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:19:13.134 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:13.132+0000 7f5bae7fc640 1 -- 172.21.15.143:0/1725298333 <== mon.0 v2:172.21.15.143:3300/0 5 ==== osd_map(33..33 src has 1..33) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f5ba811ec10 con 0x7f5bb813e290 2023-11-26T07:19:13.134 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:13.132+0000 7f5bbda38640 1 --2- 172.21.15.143:0/1725298333 >> [v2:172.21.15.143:6832/2150557544,v1:172.21.15.143:6833/2150557544] conn(0x7f5b9c066620 0x7f5b9c068ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:19:13.134 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:13.132+0000 7f5bbda38640 1 --2- 172.21.15.143:0/1725298333 >> [v2:172.21.15.143:6832/2150557544,v1:172.21.15.143:6833/2150557544] conn(0x7f5b9c066620 0x7f5b9c068ae0 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x7f5bb4009b60 tx=0x7f5bb4009190 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:19:13.137 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:13.136+0000 7f5bae7fc640 1 -- 172.21.15.143:0/1725298333 <== mon.0 v2:172.21.15.143:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172649 (secure 0 0 0) 0x7f5ba80ebf60 con 0x7f5bb813e290 2023-11-26T07:19:13.300 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:13.296+0000 7f5bbfc9c640 1 -- 172.21.15.143:0/1725298333 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) v1 -- 0x7f5bb806df50 con 0x7f5bb813e290 2023-11-26T07:19:13.300 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:13.296+0000 7f5bae7fc640 1 -- 172.21.15.143:0/1725298333 <== mon.0 v2:172.21.15.143:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v33) v1 ==== 80+0+0 (secure 0 0 0) 0x7f5ba80ef580 con 0x7f5bb813e290 2023-11-26T07:19:13.300 INFO:teuthology.orchestra.run.smithi143.stderr:listed 0 entries 2023-11-26T07:19:13.302 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:13.300+0000 7f5bbfc9c640 1 -- 172.21.15.143:0/1725298333 >> [v2:172.21.15.143:6832/2150557544,v1:172.21.15.143:6833/2150557544] conn(0x7f5b9c066620 msgr2=0x7f5b9c068ae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:19:13.303 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:13.300+0000 7f5bbfc9c640 1 --2- 172.21.15.143:0/1725298333 >> [v2:172.21.15.143:6832/2150557544,v1:172.21.15.143:6833/2150557544] conn(0x7f5b9c066620 0x7f5b9c068ae0 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x7f5bb4009b60 tx=0x7f5bb4009190 comp rx=0 tx=0).stop 2023-11-26T07:19:13.303 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:13.300+0000 7f5bbfc9c640 1 -- 172.21.15.143:0/1725298333 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f5bb813e290 msgr2=0x7f5bb813a040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:19:13.303 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:13.300+0000 7f5bbfc9c640 1 --2- 172.21.15.143:0/1725298333 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f5bb813e290 0x7f5bb813a040 secure :-1 s=READY pgs=476 cs=0 l=1 rev1=1 crypto rx=0x7f5ba80ad040 tx=0x7f5ba801ac90 comp rx=0 tx=0).stop 2023-11-26T07:19:13.303 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:13.300+0000 7f5bbfc9c640 1 -- 172.21.15.143:0/1725298333 shutdown_connections 2023-11-26T07:19:13.303 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:13.300+0000 7f5bbfc9c640 1 --2- 172.21.15.143:0/1725298333 >> [v2:172.21.15.143:6832/2150557544,v1:172.21.15.143:6833/2150557544] conn(0x7f5b9c066620 0x7f5b9c068ae0 unknown :-1 s=CLOSED pgs=244 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:19:13.303 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:13.300+0000 7f5bbfc9c640 1 --2- 172.21.15.143:0/1725298333 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f5bb815f4e0 0x7f5bb80aeb60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:19:13.303 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:13.300+0000 7f5bbfc9c640 1 --2- 172.21.15.143:0/1725298333 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f5bb813e290 0x7f5bb813a040 unknown :-1 s=CLOSED pgs=476 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:19:13.303 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:13.300+0000 7f5bbfc9c640 1 --2- 172.21.15.143:0/1725298333 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f5bb80cf7c0 0x7f5bb8152a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:19:13.303 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:13.300+0000 7f5bbfc9c640 1 -- 172.21.15.143:0/1725298333 >> 172.21.15.143:0/1725298333 conn(0x7f5bb806a890 msgr2=0x7f5bb815a880 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:19:13.303 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:13.300+0000 7f5bbfc9c640 1 -- 172.21.15.143:0/1725298333 shutdown_connections 2023-11-26T07:19:13.303 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:13.300+0000 7f5bbfc9c640 1 -- 172.21.15.143:0/1725298333 wait complete. 2023-11-26T07:19:13.321 INFO:tasks.cephfs.fuse_mount:Running fusermount -u on ubuntu@smithi143.front.sepia.ceph.com... 2023-11-26T07:19:13.321 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-26T07:19:13.322 DEBUG:teuthology.orchestra.run.smithi143:> sudo fusermount -u /home/ubuntu/cephtest/mnt.0 2023-11-26T07:19:13.340 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi143.stderr:ceph-fuse[48976]: fuse finished with error 0 and tester_r 0 2023-11-26T07:19:13.343 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi143.front.sepia.ceph.com 2023-11-26T07:19:13.344 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-26T07:19:13.344 DEBUG:teuthology.orchestra.run.smithi143:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2023-11-26T07:19:13.447 DEBUG:teuthology.orchestra.run.smithi188:> set -ex 2023-11-26T07:19:13.447 DEBUG:teuthology.orchestra.run.smithi188:> dd if=/proc/self/mounts of=/dev/stdout 2023-11-26T07:19:13.457 DEBUG:teuthology.orchestra.run.smithi143:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-11-26T07:19:13.595 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:13.592+0000 7f832789c640 1 Processor -- start 2023-11-26T07:19:13.595 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:13.592+0000 7f832789c640 1 -- start start 2023-11-26T07:19:13.596 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:13.592+0000 7f832789c640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f83200cf990 0x7f83200708c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:19:13.596 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:13.592+0000 7f832789c640 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7f8320070e00 0x7f8320162510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:19:13.597 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:13.592+0000 7f832789c640 1 -- --> v1:172.21.15.143:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8320162080 con 0x7f832014b1e0 2023-11-26T07:19:13.597 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:13.592+0000 7f832789c640 1 -- --> v2:172.21.15.188:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8320161d70 con 0x7f8320070e00 2023-11-26T07:19:13.597 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:13.592+0000 7f832789c640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f832014b5c0 con 0x7f83200cf990 2023-11-26T07:19:13.597 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:13.596+0000 7f8325638640 1 --1- >> v1:172.21.15.143:6789/0 conn(0x7f832014b1e0 0x7f83200cf280 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.143:6789/0 says I am v1:172.21.15.143:38308/0 (socket says 172.21.15.143:38308) 2023-11-26T07:19:13.597 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:13.596+0000 7f8325638640 1 -- 172.21.15.143:0/2957665766 learned_addr learned my addr 172.21.15.143:0/2957665766 (peer_addr_for_me v1:172.21.15.143:0/0) 2023-11-26T07:19:13.598 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:13.596+0000 7f8324e37640 1 --2- 172.21.15.143:0/2957665766 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f83200cf990 0x7f83200708c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:19:13.598 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:13.596+0000 7f8325e39640 1 --2- 172.21.15.143:0/2957665766 >> v2:172.21.15.188:3300/0 conn(0x7f8320070e00 0x7f8320162510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:19:13.598 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:13.596+0000 7f8317fff640 1 -- 172.21.15.143:0/2957665766 <== mon.0 v1:172.21.15.143:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3764468782 0 0) 0x7f8320162080 con 0x7f832014b1e0 2023-11-26T07:19:13.598 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:13.596+0000 7f8317fff640 1 -- 172.21.15.143:0/2957665766 --> v1:172.21.15.143:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f830c003530 con 0x7f832014b1e0 2023-11-26T07:19:13.598 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:13.596+0000 7f8325e39640 1 -- 172.21.15.143:0/2957665766 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f83200cf990 msgr2=0x7f83200708c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-26T07:19:13.598 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:13.596+0000 7f8325e39640 1 --2- 172.21.15.143:0/2957665766 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f83200cf990 0x7f83200708c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:19:13.598 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:13.596+0000 7f8325e39640 1 -- 172.21.15.143:0/2957665766 >> v1:172.21.15.143:6789/0 conn(0x7f832014b1e0 legacy=0x7f83200cf280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:19:13.598 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:13.596+0000 7f8325e39640 1 -- 172.21.15.143:0/2957665766 --> v2:172.21.15.188:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8320071580 con 0x7f8320070e00 2023-11-26T07:19:13.598 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:13.596+0000 7f8324e37640 1 --2- 172.21.15.143:0/2957665766 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f83200cf990 0x7f83200708c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-26T07:19:13.600 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:13.596+0000 7f8325e39640 1 --2- 172.21.15.143:0/2957665766 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f8320070e00 0x7f8320162510 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7f8318009d90 tx=0x7f831801c920 comp rx=0 tx=0).ready entity=mon.1 client_cookie=5d9eea7461eac9ce server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:19:13.600 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:13.596+0000 7f8317fff640 1 -- 172.21.15.143:0/2957665766 <== mon.1 v2:172.21.15.188:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f83180a59e0 con 0x7f8320070e00 2023-11-26T07:19:13.601 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:13.600+0000 7f8317fff640 1 -- 172.21.15.143:0/2957665766 <== mon.1 v2:172.21.15.188:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f83180a5b40 con 0x7f8320070e00 2023-11-26T07:19:13.601 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:13.600+0000 7f8317fff640 1 -- 172.21.15.143:0/2957665766 <== mon.1 v2:172.21.15.188:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f83180ad440 con 0x7f8320070e00 2023-11-26T07:19:13.601 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:13.600+0000 7f832789c640 1 -- 172.21.15.143:0/2957665766 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f8320070e00 msgr2=0x7f8320162510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:19:13.601 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:13.600+0000 7f832789c640 1 --2- 172.21.15.143:0/2957665766 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f8320070e00 0x7f8320162510 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7f8318009d90 tx=0x7f831801c920 comp rx=0 tx=0).stop 2023-11-26T07:19:13.602 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:13.600+0000 7f832789c640 1 -- 172.21.15.143:0/2957665766 shutdown_connections 2023-11-26T07:19:13.602 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:13.600+0000 7f832789c640 1 --2- 172.21.15.143:0/2957665766 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f8320070e00 0x7f8320162510 unknown :-1 s=CLOSED pgs=208 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:19:13.602 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:13.600+0000 7f832789c640 1 --2- 172.21.15.143:0/2957665766 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f83200cf990 0x7f83200708c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:19:13.602 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:13.600+0000 7f832789c640 1 -- 172.21.15.143:0/2957665766 >> 172.21.15.143:0/2957665766 conn(0x7f832006a890 msgr2=0x7f832006aca0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:19:13.602 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:13.600+0000 7f832789c640 1 -- 172.21.15.143:0/2957665766 shutdown_connections 2023-11-26T07:19:13.602 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:13.600+0000 7f832789c640 1 -- 172.21.15.143:0/2957665766 wait complete. 2023-11-26T07:19:13.603 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:13.600+0000 7f832789c640 1 Processor -- start 2023-11-26T07:19:13.603 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:13.600+0000 7f832789c640 1 -- start start 2023-11-26T07:19:13.604 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:13.600+0000 7f832789c640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f8320070e00 0x7f8320160b40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:19:13.605 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:13.600+0000 7f832789c640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f83200cf990 0x7f8320163990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:19:13.605 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:13.600+0000 7f832789c640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f832014b1e0 0x7f8320163ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:19:13.605 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:13.600+0000 7f832789c640 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f83200aa930 con 0x7f83200cf990 2023-11-26T07:19:13.605 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:13.600+0000 7f832789c640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f83200ab320 con 0x7f8320070e00 2023-11-26T07:19:13.605 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:13.600+0000 7f832789c640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f83200ab600 con 0x7f832014b1e0 2023-11-26T07:19:13.605 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:13.600+0000 7f8324e37640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f83200cf990 0x7f8320163990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:19:13.605 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:13.600+0000 7f8324e37640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f83200cf990 0x7f8320163990 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:48712/0 (socket says 172.21.15.143:48712) 2023-11-26T07:19:13.605 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:13.600+0000 7f8324e37640 1 -- 172.21.15.143:0/88537458 learned_addr learned my addr 172.21.15.143:0/88537458 (peer_addr_for_me v2:172.21.15.143:0/0) 2023-11-26T07:19:13.605 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:13.600+0000 7f8325e39640 1 --2- 172.21.15.143:0/88537458 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f832014b1e0 0x7f8320163ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:19:13.605 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:13.600+0000 7f8325638640 1 --2- 172.21.15.143:0/88537458 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f8320070e00 0x7f8320160b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:19:13.605 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:13.604+0000 7f8324e37640 1 -- 172.21.15.143:0/88537458 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f832014b1e0 msgr2=0x7f8320163ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:19:13.606 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:13.604+0000 7f8324e37640 1 --2- 172.21.15.143:0/88537458 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f832014b1e0 0x7f8320163ed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:19:13.606 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:13.604+0000 7f8324e37640 1 -- 172.21.15.143:0/88537458 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f8320070e00 msgr2=0x7f8320160b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:19:13.606 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:13.604+0000 7f8324e37640 1 --2- 172.21.15.143:0/88537458 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f8320070e00 0x7f8320160b40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:19:13.606 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:13.604+0000 7f8324e37640 1 -- 172.21.15.143:0/88537458 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8310009420 con 0x7f83200cf990 2023-11-26T07:19:13.607 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:13.604+0000 7f8324e37640 1 --2- 172.21.15.143:0/88537458 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f83200cf990 0x7f8320163990 secure :-1 s=READY pgs=478 cs=0 l=1 rev1=1 crypto rx=0x7f831000ebe0 tx=0x7f831000c520 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:19:13.607 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:13.604+0000 7f8315ffb640 1 -- 172.21.15.143:0/88537458 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f831000eee0 con 0x7f83200cf990 2023-11-26T07:19:13.607 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:13.604+0000 7f8315ffb640 1 -- 172.21.15.143:0/88537458 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f831000cc30 con 0x7f83200cf990 2023-11-26T07:19:13.607 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:13.604+0000 7f8315ffb640 1 -- 172.21.15.143:0/88537458 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8310013470 con 0x7f83200cf990 2023-11-26T07:19:13.607 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:13.604+0000 7f832789c640 1 -- 172.21.15.143:0/88537458 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8318009a70 con 0x7f83200cf990 2023-11-26T07:19:13.607 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:13.604+0000 7f832789c640 1 -- 172.21.15.143:0/88537458 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f83200abbc0 con 0x7f83200cf990 2023-11-26T07:19:13.607 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:13.604+0000 7f832789c640 1 -- 172.21.15.143:0/88537458 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f83200abd80 con 0x7f83200cf990 2023-11-26T07:19:13.608 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:13.604+0000 7f8315ffb640 1 -- 172.21.15.143:0/88537458 <== mon.0 v2:172.21.15.143:3300/0 4 ==== mgrmap(e 7) v1 ==== 86005+0+0 (secure 0 0 0) 0x7f83100135d0 con 0x7f83200cf990 2023-11-26T07:19:13.608 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:13.604+0000 7f8315ffb640 1 --2- 172.21.15.143:0/88537458 >> [v2:172.21.15.143:6832/2150557544,v1:172.21.15.143:6833/2150557544] conn(0x7f8304066620 0x7f8304068ae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:19:13.608 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:13.604+0000 7f8315ffb640 1 -- 172.21.15.143:0/88537458 <== mon.0 v2:172.21.15.143:3300/0 5 ==== osd_map(33..33 src has 1..33) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f8310084360 con 0x7f83200cf990 2023-11-26T07:19:13.608 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:13.604+0000 7f8325638640 1 --2- 172.21.15.143:0/88537458 >> [v2:172.21.15.143:6832/2150557544,v1:172.21.15.143:6833/2150557544] conn(0x7f8304066620 0x7f8304068ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:19:13.609 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:13.608+0000 7f8325638640 1 --2- 172.21.15.143:0/88537458 >> [v2:172.21.15.143:6832/2150557544,v1:172.21.15.143:6833/2150557544] conn(0x7f8304066620 0x7f8304068ae0 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x7f831c0011f0 tx=0x7f831c007d40 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:19:13.612 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:13.608+0000 7f8315ffb640 1 -- 172.21.15.143:0/88537458 <== mon.0 v2:172.21.15.143:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172649 (secure 0 0 0) 0x7f8310051620 con 0x7f83200cf990 2023-11-26T07:19:13.768 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:13.764+0000 7f832789c640 1 -- 172.21.15.143:0/88537458 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f8320067680 con 0x7f83200cf990 2023-11-26T07:19:13.769 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:13.768+0000 7f8315ffb640 1 -- 172.21.15.143:0/88537458 <== mon.0 v2:172.21.15.143:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v33) v1 ==== 74+0+16605 (secure 0 0 0) 0x7f8310054c40 con 0x7f83200cf990 2023-11-26T07:19:13.770 INFO:teuthology.orchestra.run.smithi143.stdout: 2023-11-26T07:19:13.770 INFO:teuthology.orchestra.run.smithi143.stdout:{"epoch":33,"fsid":"f5033060-2469-4a07-95fe-2b76f5404bc1","created":"2023-11-26T07:17:28.613663+0000","modified":"2023-11-26T07:18:53.659425+0000","last_up_change":"2023-11-26T07:17:41.157813+0000","last_in_change":"2023-11-26T07:17:31.181658+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":5,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-11-26T07:17:41.758597+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_metadata","create_time":"2023-11-26T07:18:50.535934+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"32","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":5,"pool_name":"cephfs_data","create_time":"2023-11-26T07:18:51.008494+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"33","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"ebbee2c6-023e-44b9-9f1d-6f7bfd309730","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6817","nonce":639807369},{"type":"v1","addr":"172.21.15.143:6819","nonce":639807369}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6821","nonce":639807369},{"type":"v1","addr":"172.21.15.143:6823","nonce":639807369}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6830","nonce":639807369},{"type":"v1","addr":"172.21.15.143:6831","nonce":639807369}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6825","nonce":639807369},{"type":"v1","addr":"172.21.15.143:6828","nonce":639807369}]},"public_addr":"172.21.15.143:6819/639807369","cluster_addr":"172.21.15.143:6823/639807369","heartbeat_back_addr":"172.21.15.143:6831/639807369","heartbeat_front_addr":"172.21.15.143:6828/639807369","state":["exists","up"]},{"osd":1,"uuid":"fec4b8be-7122-49e9-9a19-5850a0700001","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6816","nonce":1144942530},{"type":"v1","addr":"172.21.15.143:6818","nonce":1144942530}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6820","nonce":1144942530},{"type":"v1","addr":"172.21.15.143:6822","nonce":1144942530}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6827","nonce":1144942530},{"type":"v1","addr":"172.21.15.143:6829","nonce":1144942530}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6824","nonce":1144942530},{"type":"v1","addr":"172.21.15.143:6826","nonce":1144942530}]},"public_addr":"172.21.15.143:6818/1144942530","cluster_addr":"172.21.15.143:6822/1144942530","heartbeat_back_addr":"172.21.15.143:6829/1144942530","heartbeat_front_addr":"172.21.15.143:6826/1144942530","state":["exists","up"]},{"osd":2,"uuid":"2888af5b-f933-46ca-8f95-6dd494ff43f6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6808","nonce":3909095186},{"type":"v1","addr":"172.21.15.143:6809","nonce":3909095186}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6810","nonce":3909095186},{"type":"v1","addr":"172.21.15.143:6811","nonce":3909095186}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6814","nonce":3909095186},{"type":"v1","addr":"172.21.15.143:6815","nonce":3909095186}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6812","nonce":3909095186},{"type":"v1","addr":"172.21.15.143:6813","nonce":3909095186}]},"public_addr":"172.21.15.143:6809/3909095186","cluster_addr":"172.21.15.143:6811/3909095186","heartbeat_back_addr":"172.21.15.143:6815/3909095186","heartbeat_front_addr":"172.21.15.143:6813/3909095186","state":["exists","up"]},{"osd":3,"uuid":"d1f52435-7bc1-4081-9550-9cd1ba55f549","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6800","nonce":3854210991},{"type":"v1","addr":"172.21.15.143:6801","nonce":3854210991}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6802","nonce":3854210991},{"type":"v1","addr":"172.21.15.143:6803","nonce":3854210991}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6806","nonce":3854210991},{"type":"v1","addr":"172.21.15.143:6807","nonce":3854210991}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6804","nonce":3854210991},{"type":"v1","addr":"172.21.15.143:6805","nonce":3854210991}]},"public_addr":"172.21.15.143:6801/3854210991","cluster_addr":"172.21.15.143:6803/3854210991","heartbeat_back_addr":"172.21.15.143:6807/3854210991","heartbeat_front_addr":"172.21.15.143:6805/3854210991","state":["exists","up"]},{"osd":4,"uuid":"e2ae8b6e-6e25-49b2-bcbf-aacce68f06a3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6800","nonce":3924606902},{"type":"v1","addr":"172.21.15.188:6801","nonce":3924606902}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6802","nonce":3924606902},{"type":"v1","addr":"172.21.15.188:6803","nonce":3924606902}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6806","nonce":3924606902},{"type":"v1","addr":"172.21.15.188:6807","nonce":3924606902}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6804","nonce":3924606902},{"type":"v1","addr":"172.21.15.188:6805","nonce":3924606902}]},"public_addr":"172.21.15.188:6801/3924606902","cluster_addr":"172.21.15.188:6803/3924606902","heartbeat_back_addr":"172.21.15.188:6807/3924606902","heartbeat_front_addr":"172.21.15.188:6805/3924606902","state":["exists","up"]},{"osd":5,"uuid":"3cec8a32-7698-4c36-8e89-e4ba11934f33","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6813","nonce":186841544},{"type":"v1","addr":"172.21.15.188:6815","nonce":186841544}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6817","nonce":186841544},{"type":"v1","addr":"172.21.15.188:6818","nonce":186841544}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6822","nonce":186841544},{"type":"v1","addr":"172.21.15.188:6823","nonce":186841544}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6820","nonce":186841544},{"type":"v1","addr":"172.21.15.188:6821","nonce":186841544}]},"public_addr":"172.21.15.188:6815/186841544","cluster_addr":"172.21.15.188:6818/186841544","heartbeat_back_addr":"172.21.15.188:6823/186841544","heartbeat_front_addr":"172.21.15.188:6821/186841544","state":["exists","up"]},{"osd":6,"uuid":"58a65d5d-e090-473a-81b4-ad38e85029f9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6808","nonce":2153499405},{"type":"v1","addr":"172.21.15.188:6809","nonce":2153499405}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6810","nonce":2153499405},{"type":"v1","addr":"172.21.15.188:6811","nonce":2153499405}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6816","nonce":2153499405},{"type":"v1","addr":"172.21.15.188:6819","nonce":2153499405}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6812","nonce":2153499405},{"type":"v1","addr":"172.21.15.188:6814","nonce":2153499405}]},"public_addr":"172.21.15.188:6809/2153499405","cluster_addr":"172.21.15.188:6811/2153499405","heartbeat_back_addr":"172.21.15.188:6819/2153499405","heartbeat_front_addr":"172.21.15.188:6814/2153499405","state":["exists","up"]},{"osd":7,"uuid":"ea5b7f70-3554-4cf8-ac30-f7c59b80678d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6824","nonce":2722096729},{"type":"v1","addr":"172.21.15.188:6825","nonce":2722096729}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6826","nonce":2722096729},{"type":"v1","addr":"172.21.15.188:6827","nonce":2722096729}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6830","nonce":2722096729},{"type":"v1","addr":"172.21.15.188:6831","nonce":2722096729}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6828","nonce":2722096729},{"type":"v1","addr":"172.21.15.188:6829","nonce":2722096729}]},"public_addr":"172.21.15.188:6825/2722096729","cluster_addr":"172.21.15.188:6827/2722096729","heartbeat_back_addr":"172.21.15.188:6831/2722096729","heartbeat_front_addr":"172.21.15.188:6829/2722096729","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-26T07:17:36.995032+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-26T07:17:36.932266+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-26T07:17:36.829304+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-26T07:17:36.804297+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-26T07:17:36.667892+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-26T07:17:36.964868+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-26T07:17:36.827332+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-26T07:17:36.893179+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-11-26T07:19:13.772 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:13.768+0000 7f832789c640 1 -- 172.21.15.143:0/88537458 >> [v2:172.21.15.143:6832/2150557544,v1:172.21.15.143:6833/2150557544] conn(0x7f8304066620 msgr2=0x7f8304068ae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:19:13.772 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:13.768+0000 7f832789c640 1 --2- 172.21.15.143:0/88537458 >> [v2:172.21.15.143:6832/2150557544,v1:172.21.15.143:6833/2150557544] conn(0x7f8304066620 0x7f8304068ae0 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x7f831c0011f0 tx=0x7f831c007d40 comp rx=0 tx=0).stop 2023-11-26T07:19:13.772 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:13.768+0000 7f832789c640 1 -- 172.21.15.143:0/88537458 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f83200cf990 msgr2=0x7f8320163990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:19:13.772 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:13.768+0000 7f832789c640 1 --2- 172.21.15.143:0/88537458 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f83200cf990 0x7f8320163990 secure :-1 s=READY pgs=478 cs=0 l=1 rev1=1 crypto rx=0x7f831000ebe0 tx=0x7f831000c520 comp rx=0 tx=0).stop 2023-11-26T07:19:13.772 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:13.768+0000 7f832789c640 1 -- 172.21.15.143:0/88537458 shutdown_connections 2023-11-26T07:19:13.772 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:13.768+0000 7f832789c640 1 --2- 172.21.15.143:0/88537458 >> [v2:172.21.15.143:6832/2150557544,v1:172.21.15.143:6833/2150557544] conn(0x7f8304066620 0x7f8304068ae0 unknown :-1 s=CLOSED pgs=245 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:19:13.772 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:13.768+0000 7f832789c640 1 --2- 172.21.15.143:0/88537458 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f832014b1e0 0x7f8320163ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:19:13.772 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:13.768+0000 7f832789c640 1 --2- 172.21.15.143:0/88537458 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f83200cf990 0x7f8320163990 unknown :-1 s=CLOSED pgs=478 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:19:13.772 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:13.768+0000 7f832789c640 1 --2- 172.21.15.143:0/88537458 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f8320070e00 0x7f8320160b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:19:13.772 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:13.768+0000 7f832789c640 1 -- 172.21.15.143:0/88537458 >> 172.21.15.143:0/88537458 conn(0x7f832006a890 msgr2=0x7f83200b1120 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:19:13.772 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:13.768+0000 7f832789c640 1 -- 172.21.15.143:0/88537458 shutdown_connections 2023-11-26T07:19:13.773 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:13.768+0000 7f832789c640 1 -- 172.21.15.143:0/88537458 wait complete. 2023-11-26T07:19:13.791 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-11-26T07:17:41.758597+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_metadata', 'create_time': '2023-11-26T07:18:50.535934+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '32', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 2.25, 'score_stable': 2.25, 'optimal_score': 1, 'raw_score_acting': 2.25, 'raw_score_stable': 2.25, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 5, 'pool_name': 'cephfs_data', 'create_time': '2023-11-26T07:18:51.008494+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '33', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.3799999952316284, 'score_stable': 1.3799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.3799999952316284, 'raw_score_stable': 1.3799999952316284, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2023-11-26T07:19:13.791 DEBUG:teuthology.orchestra.run.smithi143:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-11-26T07:19:13.932 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:13.928+0000 7f083b59e640 1 Processor -- start 2023-11-26T07:19:13.932 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:13.928+0000 7f083b59e640 1 -- start start 2023-11-26T07:19:13.933 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:13.932+0000 7f083b59e640 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7f083c143810 0x7f083c072140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:19:13.934 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:13.932+0000 7f083b59e640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f083c15e0f0 0x7f083c0ce030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:19:13.934 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:13.932+0000 7f083b59e640 1 -- --> v1:172.21.15.143:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f083c15e860 con 0x7f083c0b1e10 2023-11-26T07:19:13.934 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:13.932+0000 7f083b59e640 1 -- --> v2:172.21.15.188:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f083c0cdba0 con 0x7f083c143810 2023-11-26T07:19:13.934 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:13.932+0000 7f083b59e640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f083c143bf0 con 0x7f083c15e0f0 2023-11-26T07:19:13.934 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:13.932+0000 7f0833fff640 1 --1- >> v1:172.21.15.143:6789/0 conn(0x7f083c0b1e10 0x7f083c15d9e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.143:6789/0 says I am v1:172.21.15.143:38316/0 (socket says 172.21.15.143:38316) 2023-11-26T07:19:13.934 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:13.932+0000 7f0833fff640 1 -- 172.21.15.143:0/2391602171 learned_addr learned my addr 172.21.15.143:0/2391602171 (peer_addr_for_me v1:172.21.15.143:0/0) 2023-11-26T07:19:13.934 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:13.932+0000 7f083ad9d640 1 --2- 172.21.15.143:0/2391602171 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f083c15e0f0 0x7f083c0ce030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:19:13.934 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:13.932+0000 7f083a59c640 1 --2- 172.21.15.143:0/2391602171 >> v2:172.21.15.188:3300/0 conn(0x7f083c143810 0x7f083c072140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:19:13.935 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:13.932+0000 7f0839d9b640 1 -- 172.21.15.143:0/2391602171 <== mon.0 v1:172.21.15.143:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 484825399 0 0) 0x7f083c15e860 con 0x7f083c0b1e10 2023-11-26T07:19:13.935 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:13.932+0000 7f0839d9b640 1 -- 172.21.15.143:0/2391602171 --> v1:172.21.15.143:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f0828003530 con 0x7f083c0b1e10 2023-11-26T07:19:13.935 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:13.932+0000 7f083ad9d640 1 -- 172.21.15.143:0/2391602171 >> v2:172.21.15.188:3300/0 conn(0x7f083c143810 msgr2=0x7f083c072140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-26T07:19:13.935 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:13.932+0000 7f083ad9d640 1 --2- 172.21.15.143:0/2391602171 >> v2:172.21.15.188:3300/0 conn(0x7f083c143810 0x7f083c072140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:19:13.935 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:13.932+0000 7f083ad9d640 1 -- 172.21.15.143:0/2391602171 >> v1:172.21.15.143:6789/0 conn(0x7f083c0b1e10 legacy=0x7f083c15d9e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:19:13.935 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:13.932+0000 7f083ad9d640 1 -- 172.21.15.143:0/2391602171 --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f083c0714f0 con 0x7f083c15e0f0 2023-11-26T07:19:13.936 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:13.932+0000 7f083a59c640 1 --2- 172.21.15.143:0/2391602171 >> v2:172.21.15.188:3300/0 conn(0x7f083c143810 0x7f083c072140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-26T07:19:13.937 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:13.936+0000 7f083ad9d640 1 --2- 172.21.15.143:0/2391602171 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f083c15e0f0 0x7f083c0ce030 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7f0834009d90 tx=0x7f083401c920 comp rx=0 tx=0).ready entity=mon.2 client_cookie=69a446cd19b912dd server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:19:13.937 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:13.936+0000 7f0839d9b640 1 -- 172.21.15.143:0/2391602171 <== mon.2 v2:172.21.15.188:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f08340a5a00 con 0x7f083c15e0f0 2023-11-26T07:19:13.938 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:13.936+0000 7f0839d9b640 1 -- 172.21.15.143:0/2391602171 <== mon.2 v2:172.21.15.188:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f08340a5b60 con 0x7f083c15e0f0 2023-11-26T07:19:13.938 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:13.936+0000 7f0839d9b640 1 -- 172.21.15.143:0/2391602171 <== mon.2 v2:172.21.15.188:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f08340ad440 con 0x7f083c15e0f0 2023-11-26T07:19:13.938 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:13.936+0000 7f083b59e640 1 -- 172.21.15.143:0/2391602171 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f083c15e0f0 msgr2=0x7f083c0ce030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:19:13.938 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:13.936+0000 7f083b59e640 1 --2- 172.21.15.143:0/2391602171 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f083c15e0f0 0x7f083c0ce030 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7f0834009d90 tx=0x7f083401c920 comp rx=0 tx=0).stop 2023-11-26T07:19:13.939 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:13.936+0000 7f083b59e640 1 -- 172.21.15.143:0/2391602171 shutdown_connections 2023-11-26T07:19:13.939 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:13.936+0000 7f083b59e640 1 --2- 172.21.15.143:0/2391602171 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f083c15e0f0 0x7f083c0ce030 unknown :-1 s=CLOSED pgs=77 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:19:13.939 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:13.936+0000 7f083b59e640 1 --2- 172.21.15.143:0/2391602171 >> v2:172.21.15.188:3300/0 conn(0x7f083c143810 0x7f083c072140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:19:13.939 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:13.936+0000 7f083b59e640 1 -- 172.21.15.143:0/2391602171 >> 172.21.15.143:0/2391602171 conn(0x7f083c06a890 msgr2=0x7f083c06aca0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:19:13.939 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:13.936+0000 7f083b59e640 1 -- 172.21.15.143:0/2391602171 shutdown_connections 2023-11-26T07:19:13.939 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:13.936+0000 7f083b59e640 1 -- 172.21.15.143:0/2391602171 wait complete. 2023-11-26T07:19:13.940 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:13.936+0000 7f083b59e640 1 Processor -- start 2023-11-26T07:19:13.940 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:13.936+0000 7f083b59e640 1 -- start start 2023-11-26T07:19:13.942 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:13.940+0000 7f083b59e640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f083c0b1e10 0x7f083c141d00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:19:13.942 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:13.940+0000 7f083b59e640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f083c143810 0x7f083c12f290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:19:13.942 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:13.940+0000 7f083b59e640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f083c15e0f0 0x7f083c157710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:19:13.942 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:13.940+0000 7f083b59e640 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f083c12f7d0 con 0x7f083c143810 2023-11-26T07:19:13.942 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:13.940+0000 7f083b59e640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f083c130270 con 0x7f083c0b1e10 2023-11-26T07:19:13.942 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:13.940+0000 7f083b59e640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f083c157c50 con 0x7f083c15e0f0 2023-11-26T07:19:13.942 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:13.940+0000 7f0833fff640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f083c143810 0x7f083c12f290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:19:13.942 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:13.940+0000 7f0833fff640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f083c143810 0x7f083c12f290 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:48722/0 (socket says 172.21.15.143:48722) 2023-11-26T07:19:13.942 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:13.940+0000 7f0833fff640 1 -- 172.21.15.143:0/936330586 learned_addr learned my addr 172.21.15.143:0/936330586 (peer_addr_for_me v2:172.21.15.143:0/0) 2023-11-26T07:19:13.942 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:13.940+0000 7f083a59c640 1 --2- 172.21.15.143:0/936330586 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f083c0b1e10 0x7f083c141d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:19:13.943 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:13.940+0000 7f083ad9d640 1 --2- 172.21.15.143:0/936330586 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f083c15e0f0 0x7f083c157710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:19:13.943 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:13.940+0000 7f0833fff640 1 -- 172.21.15.143:0/936330586 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f083c15e0f0 msgr2=0x7f083c157710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:19:13.943 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:13.940+0000 7f0833fff640 1 --2- 172.21.15.143:0/936330586 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f083c15e0f0 0x7f083c157710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:19:13.943 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:13.940+0000 7f0833fff640 1 -- 172.21.15.143:0/936330586 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f083c0b1e10 msgr2=0x7f083c141d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:19:13.943 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:13.940+0000 7f0833fff640 1 --2- 172.21.15.143:0/936330586 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f083c0b1e10 0x7f083c141d00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:19:13.943 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:13.940+0000 7f0833fff640 1 -- 172.21.15.143:0/936330586 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f082c009420 con 0x7f083c143810 2023-11-26T07:19:13.943 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:13.940+0000 7f0833fff640 1 --2- 172.21.15.143:0/936330586 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f083c143810 0x7f083c12f290 secure :-1 s=READY pgs=480 cs=0 l=1 rev1=1 crypto rx=0x7f082400c3f0 tx=0x7f082400c8b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:19:13.944 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:13.940+0000 7f0832ffd640 1 -- 172.21.15.143:0/936330586 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0824005cf0 con 0x7f083c143810 2023-11-26T07:19:13.944 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:13.940+0000 7f0832ffd640 1 -- 172.21.15.143:0/936330586 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0824005e50 con 0x7f083c143810 2023-11-26T07:19:13.944 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:13.940+0000 7f083b59e640 1 -- 172.21.15.143:0/936330586 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0834009a70 con 0x7f083c143810 2023-11-26T07:19:13.944 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:13.940+0000 7f0832ffd640 1 -- 172.21.15.143:0/936330586 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f08240072f0 con 0x7f083c143810 2023-11-26T07:19:13.944 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:13.940+0000 7f083b59e640 1 -- 172.21.15.143:0/936330586 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f083c12d470 con 0x7f083c143810 2023-11-26T07:19:13.944 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:13.940+0000 7f083b59e640 1 -- 172.21.15.143:0/936330586 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f083c12d2a0 con 0x7f083c143810 2023-11-26T07:19:13.945 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:13.944+0000 7f0832ffd640 1 -- 172.21.15.143:0/936330586 <== mon.0 v2:172.21.15.143:3300/0 4 ==== mgrmap(e 7) v1 ==== 86005+0+0 (secure 0 0 0) 0x7f0824007450 con 0x7f083c143810 2023-11-26T07:19:13.945 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:13.944+0000 7f0832ffd640 1 --2- 172.21.15.143:0/936330586 >> [v2:172.21.15.143:6832/2150557544,v1:172.21.15.143:6833/2150557544] conn(0x7f081c066620 0x7f081c068ae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:19:13.946 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:13.944+0000 7f0832ffd640 1 -- 172.21.15.143:0/936330586 <== mon.0 v2:172.21.15.143:3300/0 5 ==== osd_map(33..33 src has 1..33) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f0824083890 con 0x7f083c143810 2023-11-26T07:19:13.946 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:13.944+0000 7f083a59c640 1 --2- 172.21.15.143:0/936330586 >> [v2:172.21.15.143:6832/2150557544,v1:172.21.15.143:6833/2150557544] conn(0x7f081c066620 0x7f081c068ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:19:13.946 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:13.944+0000 7f083a59c640 1 --2- 172.21.15.143:0/936330586 >> [v2:172.21.15.143:6832/2150557544,v1:172.21.15.143:6833/2150557544] conn(0x7f081c066620 0x7f081c068ae0 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x7f082c009c80 tx=0x7f082c008040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:19:13.950 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:13.948+0000 7f0832ffd640 1 -- 172.21.15.143:0/936330586 <== mon.0 v2:172.21.15.143:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172649 (secure 0 0 0) 0x7f0824050960 con 0x7f083c143810 2023-11-26T07:19:14.099 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:14.096+0000 7f083b59e640 1 -- 172.21.15.143:0/936330586 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f083c067500 con 0x7f083c143810 2023-11-26T07:19:14.100 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:14.096+0000 7f0832ffd640 1 -- 172.21.15.143:0/936330586 <== mon.0 v2:172.21.15.143:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v33) v1 ==== 93+0+10 (secure 0 0 0) 0x7f0824017070 con 0x7f083c143810 2023-11-26T07:19:14.100 INFO:teuthology.orchestra.run.smithi143.stdout:pg_num: 1 2023-11-26T07:19:14.102 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:14.100+0000 7f083b59e640 1 -- 172.21.15.143:0/936330586 >> [v2:172.21.15.143:6832/2150557544,v1:172.21.15.143:6833/2150557544] conn(0x7f081c066620 msgr2=0x7f081c068ae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:19:14.102 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:14.100+0000 7f083b59e640 1 --2- 172.21.15.143:0/936330586 >> [v2:172.21.15.143:6832/2150557544,v1:172.21.15.143:6833/2150557544] conn(0x7f081c066620 0x7f081c068ae0 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x7f082c009c80 tx=0x7f082c008040 comp rx=0 tx=0).stop 2023-11-26T07:19:14.102 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:14.100+0000 7f083b59e640 1 -- 172.21.15.143:0/936330586 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f083c143810 msgr2=0x7f083c12f290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:19:14.102 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:14.100+0000 7f083b59e640 1 --2- 172.21.15.143:0/936330586 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f083c143810 0x7f083c12f290 secure :-1 s=READY pgs=480 cs=0 l=1 rev1=1 crypto rx=0x7f082400c3f0 tx=0x7f082400c8b0 comp rx=0 tx=0).stop 2023-11-26T07:19:14.102 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:14.100+0000 7f083b59e640 1 -- 172.21.15.143:0/936330586 shutdown_connections 2023-11-26T07:19:14.102 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:14.100+0000 7f083b59e640 1 --2- 172.21.15.143:0/936330586 >> [v2:172.21.15.143:6832/2150557544,v1:172.21.15.143:6833/2150557544] conn(0x7f081c066620 0x7f081c068ae0 unknown :-1 s=CLOSED pgs=246 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:19:14.102 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:14.100+0000 7f083b59e640 1 --2- 172.21.15.143:0/936330586 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f083c15e0f0 0x7f083c157710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:19:14.102 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:14.100+0000 7f083b59e640 1 --2- 172.21.15.143:0/936330586 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f083c143810 0x7f083c12f290 unknown :-1 s=CLOSED pgs=480 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:19:14.103 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:14.100+0000 7f083b59e640 1 --2- 172.21.15.143:0/936330586 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f083c0b1e10 0x7f083c141d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:19:14.103 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:14.100+0000 7f083b59e640 1 -- 172.21.15.143:0/936330586 >> 172.21.15.143:0/936330586 conn(0x7f083c06a890 msgr2=0x7f083c142b10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:19:14.103 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:14.100+0000 7f083b59e640 1 -- 172.21.15.143:0/936330586 shutdown_connections 2023-11-26T07:19:14.103 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:14.100+0000 7f083b59e640 1 -- 172.21.15.143:0/936330586 wait complete. 2023-11-26T07:19:14.121 DEBUG:teuthology.orchestra.run.smithi143:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-11-26T07:19:14.261 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:14.260+0000 7f1bd03a1640 1 Processor -- start 2023-11-26T07:19:14.261 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:14.260+0000 7f1bd03a1640 1 -- start start 2023-11-26T07:19:14.263 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:14.260+0000 7f1bd03a1640 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7f1bc80ce7c0 0x7f1bc815f6b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:19:14.263 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:14.260+0000 7f1bd03a1640 1 -- --> v1:172.21.15.143:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1bc806c520 con 0x7f1bc813e290 2023-11-26T07:19:14.263 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:14.260+0000 7f1bd03a1640 1 -- --> v1:172.21.15.188:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1bc806c800 con 0x7f1bc80cf990 2023-11-26T07:19:14.263 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:14.260+0000 7f1bd03a1640 1 -- --> v2:172.21.15.143:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1bc806cae0 con 0x7f1bc80ce7c0 2023-11-26T07:19:14.263 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:14.260+0000 7f1bce13d640 1 --1- >> v1:172.21.15.143:6789/0 conn(0x7f1bc813e290 0x7f1bc80cf280 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.143:6789/0 says I am v1:172.21.15.143:38324/0 (socket says 172.21.15.143:38324) 2023-11-26T07:19:14.263 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:14.260+0000 7f1bce13d640 1 -- 172.21.15.143:0/3468458624 learned_addr learned my addr 172.21.15.143:0/3468458624 (peer_addr_for_me v1:172.21.15.143:0/0) 2023-11-26T07:19:14.263 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:14.260+0000 7f1bce93e640 1 --2- 172.21.15.143:0/3468458624 >> v2:172.21.15.143:3300/0 conn(0x7f1bc80ce7c0 0x7f1bc815f6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:19:14.264 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:14.260+0000 7f1bce93e640 1 -- 172.21.15.143:0/3468458624 >> v1:172.21.15.188:6789/0 conn(0x7f1bc80cf990 legacy=0x7f1bc815efa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-26T07:19:14.264 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:14.260+0000 7f1bce93e640 1 -- 172.21.15.143:0/3468458624 >> v1:172.21.15.143:6789/0 conn(0x7f1bc813e290 legacy=0x7f1bc80cf280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:19:14.264 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:14.260+0000 7f1bce93e640 1 -- 172.21.15.143:0/3468458624 --> v2:172.21.15.143:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1bc806cdc0 con 0x7f1bc80ce7c0 2023-11-26T07:19:14.266 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:14.264+0000 7f1bce93e640 1 --2- 172.21.15.143:0/3468458624 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f1bc80ce7c0 0x7f1bc815f6b0 secure :-1 s=READY pgs=482 cs=0 l=1 rev1=1 crypto rx=0x7f1bc0009d90 tx=0x7f1bc001c850 comp rx=0 tx=0).ready entity=mon.0 client_cookie=81d96801c91325ab server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:19:14.266 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:14.264+0000 7f1bcd13b640 1 -- 172.21.15.143:0/3468458624 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1bc00a5930 con 0x7f1bc80ce7c0 2023-11-26T07:19:14.267 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:14.264+0000 7f1bcd13b640 1 -- 172.21.15.143:0/3468458624 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1bc00a5a90 con 0x7f1bc80ce7c0 2023-11-26T07:19:14.267 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:14.264+0000 7f1bcd13b640 1 -- 172.21.15.143:0/3468458624 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1bc00a5e20 con 0x7f1bc80ce7c0 2023-11-26T07:19:14.267 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:14.264+0000 7f1bd03a1640 1 -- 172.21.15.143:0/3468458624 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f1bc80ce7c0 msgr2=0x7f1bc815f6b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:19:14.267 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:14.264+0000 7f1bd03a1640 1 --2- 172.21.15.143:0/3468458624 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f1bc80ce7c0 0x7f1bc815f6b0 secure :-1 s=READY pgs=482 cs=0 l=1 rev1=1 crypto rx=0x7f1bc0009d90 tx=0x7f1bc001c850 comp rx=0 tx=0).stop 2023-11-26T07:19:14.268 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:14.264+0000 7f1bd03a1640 1 -- 172.21.15.143:0/3468458624 shutdown_connections 2023-11-26T07:19:14.268 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:14.264+0000 7f1bd03a1640 1 --2- 172.21.15.143:0/3468458624 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f1bc80ce7c0 0x7f1bc815f6b0 unknown :-1 s=CLOSED pgs=482 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:19:14.268 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:14.264+0000 7f1bd03a1640 1 -- 172.21.15.143:0/3468458624 >> 172.21.15.143:0/3468458624 conn(0x7f1bc806a890 msgr2=0x7f1bc806aca0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:19:14.268 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:14.264+0000 7f1bd03a1640 1 -- 172.21.15.143:0/3468458624 shutdown_connections 2023-11-26T07:19:14.268 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:14.264+0000 7f1bd03a1640 1 -- 172.21.15.143:0/3468458624 wait complete. 2023-11-26T07:19:14.269 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:14.268+0000 7f1bd03a1640 1 Processor -- start 2023-11-26T07:19:14.269 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:14.268+0000 7f1bd03a1640 1 -- start start 2023-11-26T07:19:14.271 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:14.268+0000 7f1bd03a1640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f1bc80ce7c0 0x7f1bc80b3870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:19:14.271 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:14.268+0000 7f1bd03a1640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f1bc80cf990 0x7f1bc8137990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:19:14.271 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:14.268+0000 7f1bd03a1640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f1bc813e290 0x7f1bc80aeb60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:19:14.272 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:14.268+0000 7f1bd03a1640 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1bc80b2580 con 0x7f1bc80ce7c0 2023-11-26T07:19:14.272 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:14.268+0000 7f1bd03a1640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1bc80b33e0 con 0x7f1bc80cf990 2023-11-26T07:19:14.272 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:14.268+0000 7f1bd03a1640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1bc80b27d0 con 0x7f1bc813e290 2023-11-26T07:19:14.272 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:14.268+0000 7f1bce13d640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f1bc80ce7c0 0x7f1bc80b3870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:19:14.272 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:14.268+0000 7f1bce13d640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f1bc80ce7c0 0x7f1bc80b3870 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:48740/0 (socket says 172.21.15.143:48740) 2023-11-26T07:19:14.272 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:14.268+0000 7f1bce13d640 1 -- 172.21.15.143:0/3306345158 learned_addr learned my addr 172.21.15.143:0/3306345158 (peer_addr_for_me v2:172.21.15.143:0/0) 2023-11-26T07:19:14.272 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:14.268+0000 7f1bce93e640 1 --2- 172.21.15.143:0/3306345158 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f1bc813e290 0x7f1bc80aeb60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:19:14.272 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:14.268+0000 7f1bcd93c640 1 --2- 172.21.15.143:0/3306345158 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f1bc80cf990 0x7f1bc8137990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:19:14.272 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:14.268+0000 7f1bce13d640 1 -- 172.21.15.143:0/3306345158 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f1bc813e290 msgr2=0x7f1bc80aeb60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:19:14.272 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:14.268+0000 7f1bce13d640 1 --2- 172.21.15.143:0/3306345158 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f1bc813e290 0x7f1bc80aeb60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:19:14.272 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:14.268+0000 7f1bce13d640 1 -- 172.21.15.143:0/3306345158 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f1bc80cf990 msgr2=0x7f1bc8137990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:19:14.273 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:14.268+0000 7f1bce13d640 1 --2- 172.21.15.143:0/3306345158 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f1bc80cf990 0x7f1bc8137990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:19:14.273 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:14.268+0000 7f1bce13d640 1 -- 172.21.15.143:0/3306345158 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1bc0009a70 con 0x7f1bc80ce7c0 2023-11-26T07:19:14.273 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:14.268+0000 7f1bcd93c640 1 --2- 172.21.15.143:0/3306345158 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f1bc80cf990 0x7f1bc8137990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-26T07:19:14.273 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:14.268+0000 7f1bce13d640 1 --2- 172.21.15.143:0/3306345158 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f1bc80ce7c0 0x7f1bc80b3870 secure :-1 s=READY pgs=483 cs=0 l=1 rev1=1 crypto rx=0x7f1bc400c400 tx=0x7f1bc400c8c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:19:14.273 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:14.268+0000 7f1bbeffd640 1 -- 172.21.15.143:0/3306345158 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1bc4005cf0 con 0x7f1bc80ce7c0 2023-11-26T07:19:14.273 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:14.268+0000 7f1bbeffd640 1 -- 172.21.15.143:0/3306345158 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1bc4005e50 con 0x7f1bc80ce7c0 2023-11-26T07:19:14.273 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:14.268+0000 7f1bd03a1640 1 -- 172.21.15.143:0/3306345158 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1bc80b2ab0 con 0x7f1bc80ce7c0 2023-11-26T07:19:14.273 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:14.268+0000 7f1bbeffd640 1 -- 172.21.15.143:0/3306345158 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1bc40072d0 con 0x7f1bc80ce7c0 2023-11-26T07:19:14.273 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:14.268+0000 7f1bd03a1640 1 -- 172.21.15.143:0/3306345158 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1bc80b3070 con 0x7f1bc80ce7c0 2023-11-26T07:19:14.274 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:14.268+0000 7f1bd03a1640 1 -- 172.21.15.143:0/3306345158 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1bc8067680 con 0x7f1bc80ce7c0 2023-11-26T07:19:14.274 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:14.272+0000 7f1bbeffd640 1 -- 172.21.15.143:0/3306345158 <== mon.0 v2:172.21.15.143:3300/0 4 ==== mgrmap(e 7) v1 ==== 86005+0+0 (secure 0 0 0) 0x7f1bc4007430 con 0x7f1bc80ce7c0 2023-11-26T07:19:14.274 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:14.272+0000 7f1bbeffd640 1 --2- 172.21.15.143:0/3306345158 >> [v2:172.21.15.143:6832/2150557544,v1:172.21.15.143:6833/2150557544] conn(0x7f1bac066650 0x7f1bac068b10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:19:14.274 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:14.272+0000 7f1bbeffd640 1 -- 172.21.15.143:0/3306345158 <== mon.0 v2:172.21.15.143:3300/0 5 ==== osd_map(33..33 src has 1..33) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f1bc40835f0 con 0x7f1bc80ce7c0 2023-11-26T07:19:14.274 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:14.272+0000 7f1bcd93c640 1 --2- 172.21.15.143:0/3306345158 >> [v2:172.21.15.143:6832/2150557544,v1:172.21.15.143:6833/2150557544] conn(0x7f1bac066650 0x7f1bac068b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:19:14.275 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:14.272+0000 7f1bcd93c640 1 --2- 172.21.15.143:0/3306345158 >> [v2:172.21.15.143:6832/2150557544,v1:172.21.15.143:6833/2150557544] conn(0x7f1bac066650 0x7f1bac068b10 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0x7f1bc81f9570 tx=0x7f1bb8009040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:19:14.278 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:14.276+0000 7f1bbeffd640 1 -- 172.21.15.143:0/3306345158 <== mon.0 v2:172.21.15.143:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172649 (secure 0 0 0) 0x7f1bc4050890 con 0x7f1bc80ce7c0 2023-11-26T07:19:14.428 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:14.424+0000 7f1bd03a1640 1 -- 172.21.15.143:0/3306345158 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f1bc813bbd0 con 0x7f1bc80ce7c0 2023-11-26T07:19:14.429 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:14.424+0000 7f1bbeffd640 1 -- 172.21.15.143:0/3306345158 <== mon.0 v2:172.21.15.143:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v33) v1 ==== 104+0+11 (secure 0 0 0) 0x7f1bc4053eb0 con 0x7f1bc80ce7c0 2023-11-26T07:19:14.429 INFO:teuthology.orchestra.run.smithi143.stdout:pg_num: 32 2023-11-26T07:19:14.431 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:14.428+0000 7f1bd03a1640 1 -- 172.21.15.143:0/3306345158 >> [v2:172.21.15.143:6832/2150557544,v1:172.21.15.143:6833/2150557544] conn(0x7f1bac066650 msgr2=0x7f1bac068b10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:19:14.431 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:14.428+0000 7f1bd03a1640 1 --2- 172.21.15.143:0/3306345158 >> [v2:172.21.15.143:6832/2150557544,v1:172.21.15.143:6833/2150557544] conn(0x7f1bac066650 0x7f1bac068b10 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0x7f1bc81f9570 tx=0x7f1bb8009040 comp rx=0 tx=0).stop 2023-11-26T07:19:14.431 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:14.428+0000 7f1bd03a1640 1 -- 172.21.15.143:0/3306345158 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f1bc80ce7c0 msgr2=0x7f1bc80b3870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:19:14.431 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:14.428+0000 7f1bd03a1640 1 --2- 172.21.15.143:0/3306345158 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f1bc80ce7c0 0x7f1bc80b3870 secure :-1 s=READY pgs=483 cs=0 l=1 rev1=1 crypto rx=0x7f1bc400c400 tx=0x7f1bc400c8c0 comp rx=0 tx=0).stop 2023-11-26T07:19:14.431 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:14.428+0000 7f1bd03a1640 1 -- 172.21.15.143:0/3306345158 shutdown_connections 2023-11-26T07:19:14.431 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:14.428+0000 7f1bd03a1640 1 --2- 172.21.15.143:0/3306345158 >> [v2:172.21.15.143:6832/2150557544,v1:172.21.15.143:6833/2150557544] conn(0x7f1bac066650 0x7f1bac068b10 unknown :-1 s=CLOSED pgs=247 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:19:14.432 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:14.428+0000 7f1bd03a1640 1 --2- 172.21.15.143:0/3306345158 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f1bc813e290 0x7f1bc80aeb60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:19:14.432 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:14.428+0000 7f1bd03a1640 1 --2- 172.21.15.143:0/3306345158 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f1bc80cf990 0x7f1bc8137990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:19:14.432 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:14.428+0000 7f1bd03a1640 1 --2- 172.21.15.143:0/3306345158 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f1bc80ce7c0 0x7f1bc80b3870 unknown :-1 s=CLOSED pgs=483 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:19:14.432 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:14.428+0000 7f1bd03a1640 1 -- 172.21.15.143:0/3306345158 >> 172.21.15.143:0/3306345158 conn(0x7f1bc806a890 msgr2=0x7f1bc814fdd0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:19:14.432 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:14.428+0000 7f1bd03a1640 1 -- 172.21.15.143:0/3306345158 shutdown_connections 2023-11-26T07:19:14.432 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:14.428+0000 7f1bd03a1640 1 -- 172.21.15.143:0/3306345158 wait complete. 2023-11-26T07:19:14.449 DEBUG:teuthology.orchestra.run.smithi143:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-11-26T07:19:14.590 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:14.588+0000 7f3175957640 1 Processor -- start 2023-11-26T07:19:14.590 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:14.588+0000 7f3175957640 1 -- start start 2023-11-26T07:19:14.592 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:14.588+0000 7f3175957640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f317013e290 0x7f31700cf280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:19:14.592 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:14.588+0000 7f3175957640 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7f31700cf7c0 0x7f317015efa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:19:14.592 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:14.588+0000 7f3175957640 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7f317015f4e0 0x7f31701408e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:19:14.592 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:14.588+0000 7f3175957640 1 -- --> v2:172.21.15.143:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f31700cfc40 con 0x7f317015f4e0 2023-11-26T07:19:14.592 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:14.588+0000 7f3175957640 1 -- --> v2:172.21.15.188:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f317006c520 con 0x7f31700cf7c0 2023-11-26T07:19:14.592 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:14.588+0000 7f3175957640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f317006c800 con 0x7f317013e290 2023-11-26T07:19:14.592 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:14.588+0000 7f316f7fe640 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7f317015f4e0 0x7f31701408e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:19:14.592 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:14.588+0000 7f316f7fe640 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7f317015f4e0 0x7f31701408e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:47912/0 (socket says 172.21.15.143:47912) 2023-11-26T07:19:14.592 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:14.588+0000 7f316f7fe640 1 -- 172.21.15.143:0/1893028418 learned_addr learned my addr 172.21.15.143:0/1893028418 (peer_addr_for_me v2:172.21.15.143:0/0) 2023-11-26T07:19:14.593 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:14.588+0000 7f31667fc640 1 --2- 172.21.15.143:0/1893028418 >> v2:172.21.15.188:3300/0 conn(0x7f31700cf7c0 0x7f317015efa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:19:14.593 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:14.588+0000 7f316effd640 1 --2- 172.21.15.143:0/1893028418 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f317013e290 0x7f31700cf280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:19:14.593 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:14.588+0000 7f316f7fe640 1 -- 172.21.15.143:0/1893028418 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f317013e290 msgr2=0x7f31700cf280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-26T07:19:14.593 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:14.588+0000 7f316f7fe640 1 --2- 172.21.15.143:0/1893028418 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f317013e290 0x7f31700cf280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:19:14.594 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:14.588+0000 7f316f7fe640 1 -- 172.21.15.143:0/1893028418 >> v2:172.21.15.188:3300/0 conn(0x7f31700cf7c0 msgr2=0x7f317015efa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-26T07:19:14.594 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:14.588+0000 7f316f7fe640 1 --2- 172.21.15.143:0/1893028418 >> v2:172.21.15.188:3300/0 conn(0x7f31700cf7c0 0x7f317015efa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:19:14.594 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:14.588+0000 7f316f7fe640 1 -- 172.21.15.143:0/1893028418 --> v2:172.21.15.143:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f317006cae0 con 0x7f317015f4e0 2023-11-26T07:19:14.594 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:14.588+0000 7f31667fc640 1 --2- 172.21.15.143:0/1893028418 >> v2:172.21.15.188:3300/0 conn(0x7f31700cf7c0 0x7f317015efa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-26T07:19:14.594 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:14.588+0000 7f316effd640 1 --2- 172.21.15.143:0/1893028418 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f317013e290 0x7f31700cf280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-26T07:19:14.595 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:14.592+0000 7f316f7fe640 1 --2- 172.21.15.143:0/1893028418 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f317015f4e0 0x7f31701408e0 secure :-1 s=READY pgs=484 cs=0 l=1 rev1=1 crypto rx=0x7f31700b45c0 tx=0x7f316801c210 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c571c8c0cca1e429 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:19:14.595 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:14.592+0000 7f316e7fc640 1 -- 172.21.15.143:0/1893028418 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f31680ab600 con 0x7f317015f4e0 2023-11-26T07:19:14.595 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:14.592+0000 7f316e7fc640 1 -- 172.21.15.143:0/1893028418 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f31680a7050 con 0x7f317015f4e0 2023-11-26T07:19:14.596 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:14.592+0000 7f316e7fc640 1 -- 172.21.15.143:0/1893028418 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f31680ab980 con 0x7f317015f4e0 2023-11-26T07:19:14.596 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:14.592+0000 7f3175957640 1 -- 172.21.15.143:0/1893028418 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f317015f4e0 msgr2=0x7f31701408e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:19:14.596 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:14.592+0000 7f3175957640 1 --2- 172.21.15.143:0/1893028418 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f317015f4e0 0x7f31701408e0 secure :-1 s=READY pgs=484 cs=0 l=1 rev1=1 crypto rx=0x7f31700b45c0 tx=0x7f316801c210 comp rx=0 tx=0).stop 2023-11-26T07:19:14.597 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:14.592+0000 7f3175957640 1 -- 172.21.15.143:0/1893028418 shutdown_connections 2023-11-26T07:19:14.597 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:14.592+0000 7f3175957640 1 --2- 172.21.15.143:0/1893028418 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f317015f4e0 0x7f31701408e0 unknown :-1 s=CLOSED pgs=484 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:19:14.597 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:14.592+0000 7f3175957640 1 --2- 172.21.15.143:0/1893028418 >> v2:172.21.15.188:3300/0 conn(0x7f31700cf7c0 0x7f317015efa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:19:14.597 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:14.592+0000 7f3175957640 1 --2- 172.21.15.143:0/1893028418 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f317013e290 0x7f31700cf280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:19:14.597 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:14.592+0000 7f3175957640 1 -- 172.21.15.143:0/1893028418 >> 172.21.15.143:0/1893028418 conn(0x7f317006a890 msgr2=0x7f317006aca0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:19:14.597 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:14.592+0000 7f3175957640 1 -- 172.21.15.143:0/1893028418 shutdown_connections 2023-11-26T07:19:14.597 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:14.592+0000 7f3175957640 1 -- 172.21.15.143:0/1893028418 wait complete. 2023-11-26T07:19:14.597 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:14.596+0000 7f3175957640 1 Processor -- start 2023-11-26T07:19:14.598 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:14.596+0000 7f3175957640 1 -- start start 2023-11-26T07:19:14.599 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:14.596+0000 7f3175957640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f31700cf7c0 0x7f3170071da0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:19:14.599 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:14.596+0000 7f3175957640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f317013e290 0x7f3170143310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:19:14.599 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:14.596+0000 7f3175957640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f317015f4e0 0x7f317015a130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:19:14.599 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:14.596+0000 7f3175957640 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3170143850 con 0x7f317013e290 2023-11-26T07:19:14.600 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:14.596+0000 7f3175957640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3170143a80 con 0x7f317015f4e0 2023-11-26T07:19:14.600 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:14.596+0000 7f3175957640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3170143d60 con 0x7f31700cf7c0 2023-11-26T07:19:14.600 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:14.596+0000 7f31667fc640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f317013e290 0x7f3170143310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:19:14.600 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:14.596+0000 7f31667fc640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f317013e290 0x7f3170143310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:47922/0 (socket says 172.21.15.143:47922) 2023-11-26T07:19:14.600 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:14.596+0000 7f31667fc640 1 -- 172.21.15.143:0/2568462167 learned_addr learned my addr 172.21.15.143:0/2568462167 (peer_addr_for_me v2:172.21.15.143:0/0) 2023-11-26T07:19:14.600 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:14.596+0000 7f316f7fe640 1 --2- 172.21.15.143:0/2568462167 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f317015f4e0 0x7f317015a130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:19:14.600 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:14.596+0000 7f316effd640 1 --2- 172.21.15.143:0/2568462167 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f31700cf7c0 0x7f3170071da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:19:14.601 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:14.596+0000 7f31667fc640 1 -- 172.21.15.143:0/2568462167 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f31700cf7c0 msgr2=0x7f3170071da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:19:14.601 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:14.596+0000 7f31667fc640 1 --2- 172.21.15.143:0/2568462167 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f31700cf7c0 0x7f3170071da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:19:14.601 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:14.596+0000 7f31667fc640 1 -- 172.21.15.143:0/2568462167 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f317015f4e0 msgr2=0x7f317015a130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:19:14.602 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:14.596+0000 7f31667fc640 1 --2- 172.21.15.143:0/2568462167 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f317015f4e0 0x7f317015a130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:19:14.602 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:14.596+0000 7f31667fc640 1 -- 172.21.15.143:0/2568462167 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f31680094d0 con 0x7f317013e290 2023-11-26T07:19:14.602 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:14.596+0000 7f316effd640 1 --2- 172.21.15.143:0/2568462167 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f31700cf7c0 0x7f3170071da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-26T07:19:14.602 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:14.596+0000 7f31667fc640 1 --2- 172.21.15.143:0/2568462167 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f317013e290 0x7f3170143310 secure :-1 s=READY pgs=485 cs=0 l=1 rev1=1 crypto rx=0x7f315800c7c0 tx=0x7f315800cc80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:19:14.602 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:14.596+0000 7f316cff9640 1 -- 172.21.15.143:0/2568462167 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3158007a30 con 0x7f317013e290 2023-11-26T07:19:14.602 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:14.596+0000 7f316cff9640 1 -- 172.21.15.143:0/2568462167 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3158007b90 con 0x7f317013e290 2023-11-26T07:19:14.602 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:14.596+0000 7f316cff9640 1 -- 172.21.15.143:0/2568462167 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f315800ce70 con 0x7f317013e290 2023-11-26T07:19:14.602 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:14.596+0000 7f3175957640 1 -- 172.21.15.143:0/2568462167 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f31700aeb60 con 0x7f317013e290 2023-11-26T07:19:14.602 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:14.596+0000 7f3175957640 1 -- 172.21.15.143:0/2568462167 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f31700af120 con 0x7f317013e290 2023-11-26T07:19:14.602 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:14.600+0000 7f3175957640 1 -- 172.21.15.143:0/2568462167 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3170067680 con 0x7f317013e290 2023-11-26T07:19:14.603 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:14.600+0000 7f316cff9640 1 -- 172.21.15.143:0/2568462167 <== mon.0 v2:172.21.15.143:3300/0 4 ==== mgrmap(e 7) v1 ==== 86005+0+0 (secure 0 0 0) 0x7f31580113f0 con 0x7f317013e290 2023-11-26T07:19:14.603 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:14.600+0000 7f316cff9640 1 --2- 172.21.15.143:0/2568462167 >> [v2:172.21.15.143:6832/2150557544,v1:172.21.15.143:6833/2150557544] conn(0x7f3150066620 0x7f3150068ae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:19:14.603 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:14.600+0000 7f316cff9640 1 -- 172.21.15.143:0/2568462167 <== mon.0 v2:172.21.15.143:3300/0 5 ==== osd_map(33..33 src has 1..33) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f3158084410 con 0x7f317013e290 2023-11-26T07:19:14.603 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:14.600+0000 7f316effd640 1 --2- 172.21.15.143:0/2568462167 >> [v2:172.21.15.143:6832/2150557544,v1:172.21.15.143:6833/2150557544] conn(0x7f3150066620 0x7f3150068ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:19:14.604 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:14.600+0000 7f316effd640 1 --2- 172.21.15.143:0/2568462167 >> [v2:172.21.15.143:6832/2150557544,v1:172.21.15.143:6833/2150557544] conn(0x7f3150066620 0x7f3150068ae0 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x7f316000e550 tx=0x7f3160007ab0 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:19:14.607 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:14.604+0000 7f316cff9640 1 -- 172.21.15.143:0/2568462167 <== mon.0 v2:172.21.15.143:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172649 (secure 0 0 0) 0x7f3158051510 con 0x7f317013e290 2023-11-26T07:19:14.759 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:14.756+0000 7f3175957640 1 -- 172.21.15.143:0/2568462167 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f3170139340 con 0x7f317013e290 2023-11-26T07:19:14.759 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:14.756+0000 7f316cff9640 1 -- 172.21.15.143:0/2568462167 <== mon.0 v2:172.21.15.143:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v33) v1 ==== 100+0+11 (secure 0 0 0) 0x7f3158054b30 con 0x7f317013e290 2023-11-26T07:19:14.759 INFO:teuthology.orchestra.run.smithi143.stdout:pg_num: 64 2023-11-26T07:19:14.761 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:14.760+0000 7f3175957640 1 -- 172.21.15.143:0/2568462167 >> [v2:172.21.15.143:6832/2150557544,v1:172.21.15.143:6833/2150557544] conn(0x7f3150066620 msgr2=0x7f3150068ae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:19:14.761 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:14.760+0000 7f3175957640 1 --2- 172.21.15.143:0/2568462167 >> [v2:172.21.15.143:6832/2150557544,v1:172.21.15.143:6833/2150557544] conn(0x7f3150066620 0x7f3150068ae0 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x7f316000e550 tx=0x7f3160007ab0 comp rx=0 tx=0).stop 2023-11-26T07:19:14.761 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:14.760+0000 7f3175957640 1 -- 172.21.15.143:0/2568462167 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f317013e290 msgr2=0x7f3170143310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:19:14.762 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:14.760+0000 7f3175957640 1 --2- 172.21.15.143:0/2568462167 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f317013e290 0x7f3170143310 secure :-1 s=READY pgs=485 cs=0 l=1 rev1=1 crypto rx=0x7f315800c7c0 tx=0x7f315800cc80 comp rx=0 tx=0).stop 2023-11-26T07:19:14.762 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:14.760+0000 7f3175957640 1 -- 172.21.15.143:0/2568462167 shutdown_connections 2023-11-26T07:19:14.762 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:14.760+0000 7f3175957640 1 --2- 172.21.15.143:0/2568462167 >> [v2:172.21.15.143:6832/2150557544,v1:172.21.15.143:6833/2150557544] conn(0x7f3150066620 0x7f3150068ae0 unknown :-1 s=CLOSED pgs=248 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:19:14.762 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:14.760+0000 7f3175957640 1 --2- 172.21.15.143:0/2568462167 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f317015f4e0 0x7f317015a130 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:19:14.762 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:14.760+0000 7f3175957640 1 --2- 172.21.15.143:0/2568462167 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f317013e290 0x7f3170143310 unknown :-1 s=CLOSED pgs=485 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:19:14.762 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:14.760+0000 7f3175957640 1 --2- 172.21.15.143:0/2568462167 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f31700cf7c0 0x7f3170071da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:19:14.762 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:14.760+0000 7f3175957640 1 -- 172.21.15.143:0/2568462167 >> 172.21.15.143:0/2568462167 conn(0x7f317006a890 msgr2=0x7f3170139560 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:19:14.762 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:14.760+0000 7f3175957640 1 -- 172.21.15.143:0/2568462167 shutdown_connections 2023-11-26T07:19:14.762 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:14.760+0000 7f3175957640 1 -- 172.21.15.143:0/2568462167 wait complete. 2023-11-26T07:19:14.781 DEBUG:teuthology.orchestra.run.smithi143:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2023-11-26T07:19:14.919 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:14.916+0000 7fc4eb232640 1 Processor -- start 2023-11-26T07:19:14.920 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:14.916+0000 7fc4eb232640 1 -- start start 2023-11-26T07:19:14.921 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:14.920+0000 7fc4eb232640 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7fc4e4070ba0 0x7fc4e406bae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:19:14.921 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:14.920+0000 7fc4eb232640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fc4e40cdd10 0x7fc4e406d950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:19:14.922 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:14.920+0000 7fc4eb232640 1 -- --> v1:172.21.15.188:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc4e40cd610 con 0x7fc4e4161cf0 2023-11-26T07:19:14.922 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:14.920+0000 7fc4eb232640 1 -- --> v2:172.21.15.188:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc4e40cd8f0 con 0x7fc4e4070ba0 2023-11-26T07:19:14.922 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:14.920+0000 7fc4eb232640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc4e4161210 con 0x7fc4e40cdd10 2023-11-26T07:19:14.922 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:14.920+0000 7fc4e3fff640 1 --1- >> v1:172.21.15.188:6789/0 conn(0x7fc4e4161cf0 0x7fc4e4070f80 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.188:6789/0 says I am v1:172.21.15.143:58436/0 (socket says 172.21.15.143:58436) 2023-11-26T07:19:14.922 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:14.920+0000 7fc4e97cf640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fc4e40cdd10 0x7fc4e406d950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:19:14.922 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:14.920+0000 7fc4e8fce640 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7fc4e4070ba0 0x7fc4e406bae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:19:14.922 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:14.920+0000 7fc4e3fff640 1 -- 172.21.15.143:0/2940425937 learned_addr learned my addr 172.21.15.143:0/2940425937 (peer_addr_for_me v1:172.21.15.143:0/0) 2023-11-26T07:19:14.923 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:14.920+0000 7fc4e97cf640 1 -- 172.21.15.143:0/2940425937 >> v2:172.21.15.188:3300/0 conn(0x7fc4e4070ba0 msgr2=0x7fc4e406bae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-26T07:19:14.923 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:14.920+0000 7fc4e97cf640 1 --2- 172.21.15.143:0/2940425937 >> v2:172.21.15.188:3300/0 conn(0x7fc4e4070ba0 0x7fc4e406bae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:19:14.923 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:14.920+0000 7fc4e97cf640 1 -- 172.21.15.143:0/2940425937 >> v1:172.21.15.188:6789/0 conn(0x7fc4e4161cf0 legacy=0x7fc4e4070f80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:19:14.923 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:14.920+0000 7fc4e37fe640 1 -- 172.21.15.143:0/2940425937 <== mon.1 v1:172.21.15.188:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2188112990 0 0) 0x7fc4e40cd610 con 0x7fc4e4161cf0 2023-11-26T07:19:14.923 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:14.920+0000 7fc4e97cf640 1 -- 172.21.15.143:0/2940425937 --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc4e41614f0 con 0x7fc4e40cdd10 2023-11-26T07:19:14.925 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:14.924+0000 7fc4e97cf640 1 --2- 172.21.15.143:0/2940425937 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fc4e40cdd10 0x7fc4e406d950 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7fc4dc009d90 tx=0x7fc4dc01c950 comp rx=0 tx=0).ready entity=mon.2 client_cookie=3620275e197135fd server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:19:14.925 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:14.924+0000 7fc4e37fe640 1 -- 172.21.15.143:0/2940425937 <== mon.2 v2:172.21.15.188:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc4dc0a59f0 con 0x7fc4e40cdd10 2023-11-26T07:19:14.925 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:14.924+0000 7fc4e37fe640 1 -- 172.21.15.143:0/2940425937 <== mon.2 v2:172.21.15.188:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc4dc0a5b50 con 0x7fc4e40cdd10 2023-11-26T07:19:14.925 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:14.924+0000 7fc4e37fe640 1 -- 172.21.15.143:0/2940425937 <== mon.2 v2:172.21.15.188:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc4dc0ad440 con 0x7fc4e40cdd10 2023-11-26T07:19:14.926 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:14.924+0000 7fc4eb232640 1 -- 172.21.15.143:0/2940425937 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fc4e40cdd10 msgr2=0x7fc4e406d950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:19:14.926 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:14.924+0000 7fc4eb232640 1 --2- 172.21.15.143:0/2940425937 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fc4e40cdd10 0x7fc4e406d950 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7fc4dc009d90 tx=0x7fc4dc01c950 comp rx=0 tx=0).stop 2023-11-26T07:19:14.926 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:14.924+0000 7fc4eb232640 1 -- 172.21.15.143:0/2940425937 shutdown_connections 2023-11-26T07:19:14.927 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:14.924+0000 7fc4eb232640 1 --2- 172.21.15.143:0/2940425937 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fc4e40cdd10 0x7fc4e406d950 unknown :-1 s=CLOSED pgs=78 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:19:14.927 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:14.924+0000 7fc4eb232640 1 --2- 172.21.15.143:0/2940425937 >> v2:172.21.15.188:3300/0 conn(0x7fc4e4070ba0 0x7fc4e406bae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:19:14.927 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:14.924+0000 7fc4eb232640 1 -- 172.21.15.143:0/2940425937 >> 172.21.15.143:0/2940425937 conn(0x7fc4e40cf710 msgr2=0x7fc4e40cfb20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:19:14.927 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:14.924+0000 7fc4eb232640 1 -- 172.21.15.143:0/2940425937 shutdown_connections 2023-11-26T07:19:14.927 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:14.924+0000 7fc4eb232640 1 -- 172.21.15.143:0/2940425937 wait complete. 2023-11-26T07:19:14.928 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:14.924+0000 7fc4eb232640 1 Processor -- start 2023-11-26T07:19:14.928 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:14.924+0000 7fc4eb232640 1 -- start start 2023-11-26T07:19:14.930 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:14.928+0000 7fc4eb232640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fc4e4070ba0 0x7fc4e4143360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:19:14.930 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:14.928+0000 7fc4eb232640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fc4e40cdd10 0x7fc4e40af000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:19:14.930 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:14.928+0000 7fc4eb232640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fc4e4161cf0 0x7fc4e40af540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:19:14.930 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:14.928+0000 7fc4eb232640 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc4e415f680 con 0x7fc4e4161cf0 2023-11-26T07:19:14.931 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:14.928+0000 7fc4eb232640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc4e41567e0 con 0x7fc4e4070ba0 2023-11-26T07:19:14.931 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:14.928+0000 7fc4eb232640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc4e415e780 con 0x7fc4e40cdd10 2023-11-26T07:19:14.931 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:14.928+0000 7fc4e97cf640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fc4e4161cf0 0x7fc4e40af540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:19:14.931 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:14.928+0000 7fc4e97cf640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fc4e4161cf0 0x7fc4e40af540 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:47930/0 (socket says 172.21.15.143:47930) 2023-11-26T07:19:14.931 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:14.928+0000 7fc4e8fce640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fc4e4070ba0 0x7fc4e4143360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:19:14.931 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:14.928+0000 7fc4e97cf640 1 -- 172.21.15.143:0/393929340 learned_addr learned my addr 172.21.15.143:0/393929340 (peer_addr_for_me v2:172.21.15.143:0/0) 2023-11-26T07:19:14.931 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:14.928+0000 7fc4e3fff640 1 --2- 172.21.15.143:0/393929340 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fc4e40cdd10 0x7fc4e40af000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:19:14.931 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:14.928+0000 7fc4e97cf640 1 -- 172.21.15.143:0/393929340 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fc4e40cdd10 msgr2=0x7fc4e40af000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:19:14.931 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:14.928+0000 7fc4e97cf640 1 --2- 172.21.15.143:0/393929340 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fc4e40cdd10 0x7fc4e40af000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:19:14.931 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:14.928+0000 7fc4e97cf640 1 -- 172.21.15.143:0/393929340 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fc4e4070ba0 msgr2=0x7fc4e4143360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:19:14.932 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:14.928+0000 7fc4e97cf640 1 --2- 172.21.15.143:0/393929340 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fc4e4070ba0 0x7fc4e4143360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:19:14.932 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:14.928+0000 7fc4e97cf640 1 -- 172.21.15.143:0/393929340 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc4dc009a70 con 0x7fc4e4161cf0 2023-11-26T07:19:14.932 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:14.928+0000 7fc4e97cf640 1 --2- 172.21.15.143:0/393929340 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fc4e4161cf0 0x7fc4e40af540 secure :-1 s=READY pgs=486 cs=0 l=1 rev1=1 crypto rx=0x7fc4dc01a580 tx=0x7fc4dc0196f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:19:14.932 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:14.928+0000 7fc4e17fa640 1 -- 172.21.15.143:0/393929340 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc4dc0add70 con 0x7fc4e4161cf0 2023-11-26T07:19:14.932 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:14.928+0000 7fc4e17fa640 1 -- 172.21.15.143:0/393929340 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc4dc009450 con 0x7fc4e4161cf0 2023-11-26T07:19:14.932 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:14.928+0000 7fc4eb232640 1 -- 172.21.15.143:0/393929340 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc4e415ea00 con 0x7fc4e4161cf0 2023-11-26T07:19:14.932 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:14.928+0000 7fc4e17fa640 1 -- 172.21.15.143:0/393929340 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc4dc0b9690 con 0x7fc4e4161cf0 2023-11-26T07:19:14.932 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:14.928+0000 7fc4eb232640 1 -- 172.21.15.143:0/393929340 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc4e415efc0 con 0x7fc4e4161cf0 2023-11-26T07:19:14.933 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:14.928+0000 7fc4eb232640 1 -- 172.21.15.143:0/393929340 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc4e415f7c0 con 0x7fc4e4161cf0 2023-11-26T07:19:14.933 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:14.928+0000 7fc4e17fa640 1 -- 172.21.15.143:0/393929340 <== mon.0 v2:172.21.15.143:3300/0 4 ==== mgrmap(e 7) v1 ==== 86005+0+0 (secure 0 0 0) 0x7fc4dc0b97f0 con 0x7fc4e4161cf0 2023-11-26T07:19:14.933 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:14.932+0000 7fc4e17fa640 1 --2- 172.21.15.143:0/393929340 >> [v2:172.21.15.143:6832/2150557544,v1:172.21.15.143:6833/2150557544] conn(0x7fc4c8066620 0x7fc4c8068ae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:19:14.933 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:14.932+0000 7fc4e17fa640 1 -- 172.21.15.143:0/393929340 <== mon.0 v2:172.21.15.143:3300/0 5 ==== osd_map(33..33 src has 1..33) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fc4dc11f700 con 0x7fc4e4161cf0 2023-11-26T07:19:14.934 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:14.932+0000 7fc4e8fce640 1 --2- 172.21.15.143:0/393929340 >> [v2:172.21.15.143:6832/2150557544,v1:172.21.15.143:6833/2150557544] conn(0x7fc4c8066620 0x7fc4c8068ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:19:14.934 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:14.932+0000 7fc4e8fce640 1 --2- 172.21.15.143:0/393929340 >> [v2:172.21.15.143:6832/2150557544,v1:172.21.15.143:6833/2150557544] conn(0x7fc4c8066620 0x7fc4c8068ae0 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0x7fc4d8009b60 tx=0x7fc4d8009190 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:19:14.937 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:14.936+0000 7fc4e17fa640 1 -- 172.21.15.143:0/393929340 <== mon.0 v2:172.21.15.143:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172649 (secure 0 0 0) 0x7fc4e415f7c0 con 0x7fc4e4161cf0 2023-11-26T07:19:15.101 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:15.100+0000 7fc4eb232640 1 -- 172.21.15.143:0/393929340 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) v1 -- 0x7fc4e415f7c0 con 0x7fc4e4161cf0 2023-11-26T07:19:15.102 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:15.100+0000 7fc4e17fa640 1 -- 172.21.15.143:0/393929340 <== mon.0 v2:172.21.15.143:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v33) v1 ==== 80+0+0 (secure 0 0 0) 0x7fc4e415f7c0 con 0x7fc4e4161cf0 2023-11-26T07:19:15.102 INFO:teuthology.orchestra.run.smithi143.stderr:listed 0 entries 2023-11-26T07:19:15.104 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:15.100+0000 7fc4eb232640 1 -- 172.21.15.143:0/393929340 >> [v2:172.21.15.143:6832/2150557544,v1:172.21.15.143:6833/2150557544] conn(0x7fc4c8066620 msgr2=0x7fc4c8068ae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:19:15.104 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:15.100+0000 7fc4eb232640 1 --2- 172.21.15.143:0/393929340 >> [v2:172.21.15.143:6832/2150557544,v1:172.21.15.143:6833/2150557544] conn(0x7fc4c8066620 0x7fc4c8068ae0 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0x7fc4d8009b60 tx=0x7fc4d8009190 comp rx=0 tx=0).stop 2023-11-26T07:19:15.104 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:15.100+0000 7fc4eb232640 1 -- 172.21.15.143:0/393929340 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fc4e4161cf0 msgr2=0x7fc4e40af540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:19:15.104 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:15.100+0000 7fc4eb232640 1 --2- 172.21.15.143:0/393929340 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fc4e4161cf0 0x7fc4e40af540 secure :-1 s=READY pgs=486 cs=0 l=1 rev1=1 crypto rx=0x7fc4dc01a580 tx=0x7fc4dc0196f0 comp rx=0 tx=0).stop 2023-11-26T07:19:15.104 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:15.100+0000 7fc4eb232640 1 -- 172.21.15.143:0/393929340 shutdown_connections 2023-11-26T07:19:15.105 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:15.100+0000 7fc4eb232640 1 --2- 172.21.15.143:0/393929340 >> [v2:172.21.15.143:6832/2150557544,v1:172.21.15.143:6833/2150557544] conn(0x7fc4c8066620 0x7fc4c8068ae0 unknown :-1 s=CLOSED pgs=249 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:19:15.105 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:15.100+0000 7fc4eb232640 1 --2- 172.21.15.143:0/393929340 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fc4e4161cf0 0x7fc4e40af540 unknown :-1 s=CLOSED pgs=486 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:19:15.105 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:15.100+0000 7fc4eb232640 1 --2- 172.21.15.143:0/393929340 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fc4e40cdd10 0x7fc4e40af000 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:19:15.105 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:15.100+0000 7fc4eb232640 1 --2- 172.21.15.143:0/393929340 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fc4e4070ba0 0x7fc4e4143360 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:19:15.105 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:15.100+0000 7fc4eb232640 1 -- 172.21.15.143:0/393929340 >> 172.21.15.143:0/393929340 conn(0x7fc4e40cf710 msgr2=0x7fc4e406b2f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:19:15.105 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:15.100+0000 7fc4eb232640 1 -- 172.21.15.143:0/393929340 shutdown_connections 2023-11-26T07:19:15.105 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:15.100+0000 7fc4eb232640 1 -- 172.21.15.143:0/393929340 wait complete. 2023-11-26T07:19:15.123 INFO:tasks.cephfs.fuse_mount:Running fusermount -u on ubuntu@smithi188.front.sepia.ceph.com... 2023-11-26T07:19:15.123 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-26T07:19:15.124 DEBUG:teuthology.orchestra.run.smithi188:> sudo fusermount -u /home/ubuntu/cephtest/mnt.1 2023-11-26T07:19:15.145 INFO:tasks.cephfs.fuse_mount.ceph-fuse.1.smithi188.stderr:ceph-fuse[44390]: fuse finished with error 0 and tester_r 0 2023-11-26T07:19:15.148 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi188.front.sepia.ceph.com 2023-11-26T07:19:15.148 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-26T07:19:15.148 DEBUG:teuthology.orchestra.run.smithi188:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.1) 2023-11-26T07:19:15.199 DEBUG:teuthology.orchestra.run.smithi143:> set -ex 2023-11-26T07:19:15.200 DEBUG:teuthology.orchestra.run.smithi143:> dd if=/proc/self/mounts of=/dev/stdout 2023-11-26T07:19:15.209 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,relatime,size=16354672k,nr_inodes=4088668,mode=755 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3275384k,mode=755 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=3455 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysusers.service ramfs ro,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3275380k,nr_inodes=818845,mode=700,uid=1000,gid=1271 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3275384k,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2023-11-26T07:19:15.209 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi143.front.sepia.ceph.com 2023-11-26T07:19:15.209 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-26T07:19:15.209 DEBUG:teuthology.orchestra.run.smithi143:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.2) 2023-11-26T07:19:15.260 INFO:teuthology.orchestra.run.smithi143.stderr:rmdir: failed to remove '/home/ubuntu/cephtest/mnt.2': No such file or directory 2023-11-26T07:19:15.261 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-11-26T07:19:15.261 DEBUG:teuthology.orchestra.run.smithi188:> set -ex 2023-11-26T07:19:15.261 DEBUG:teuthology.orchestra.run.smithi188:> dd if=/proc/self/mounts of=/dev/stdout 2023-11-26T07:19:15.271 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,relatime,size=16354628k,nr_inodes=4088657,mode=755 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3275376k,mode=755 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=787 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysusers.service ramfs ro,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3275372k,nr_inodes=818843,mode=700,uid=1000,gid=1271 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3275376k,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2023-11-26T07:19:15.271 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi188.front.sepia.ceph.com 2023-11-26T07:19:15.271 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-26T07:19:15.271 DEBUG:teuthology.orchestra.run.smithi188:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.3) 2023-11-26T07:19:15.323 INFO:teuthology.orchestra.run.smithi188.stderr:rmdir: failed to remove '/home/ubuntu/cephtest/mnt.3': No such file or directory 2023-11-26T07:19:15.323 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-11-26T07:19:15.324 DEBUG:teuthology.orchestra.run.smithi143:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-11-26T07:19:15.462 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:15.460+0000 7f2195b8d640 1 Processor -- start 2023-11-26T07:19:15.462 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:15.460+0000 7f2195b8d640 1 -- start start 2023-11-26T07:19:15.463 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:15.460+0000 7f2195b8d640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f219015d6e0 0x7f21900d0df0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:19:15.463 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:15.460+0000 7f2195b8d640 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7f21900b2160 0x7f21900d14b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:19:15.463 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:15.460+0000 7f2195b8d640 1 -- --> v1:172.21.15.188:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f21900cf0e0 con 0x7f21900d19f0 2023-11-26T07:19:15.463 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:15.460+0000 7f2195b8d640 1 -- --> v2:172.21.15.188:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f219015dac0 con 0x7f21900b2160 2023-11-26T07:19:15.463 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:15.460+0000 7f2195b8d640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f21900d1e80 con 0x7f219015d6e0 2023-11-26T07:19:15.464 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:15.460+0000 7f2187fff640 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7f21900b2160 0x7f21900d14b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:19:15.464 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:15.460+0000 7f218f7fe640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f219015d6e0 0x7f21900d0df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:19:15.464 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:15.460+0000 7f2187fff640 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7f21900b2160 0x7f21900d14b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.188:3300/0 says I am v2:172.21.15.143:37506/0 (socket says 172.21.15.143:37506) 2023-11-26T07:19:15.465 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:15.460+0000 7f2187fff640 1 -- 172.21.15.143:0/1880137146 learned_addr learned my addr 172.21.15.143:0/1880137146 (peer_addr_for_me v2:172.21.15.143:0/0) 2023-11-26T07:19:15.465 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:15.460+0000 7f2187fff640 1 -- 172.21.15.143:0/1880137146 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f219015d6e0 msgr2=0x7f21900d0df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-26T07:19:15.465 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:15.460+0000 7f2187fff640 1 --2- 172.21.15.143:0/1880137146 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f219015d6e0 0x7f21900d0df0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:19:15.465 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:15.460+0000 7f2187fff640 1 -- 172.21.15.143:0/1880137146 >> v1:172.21.15.188:6789/0 conn(0x7f21900d19f0 legacy=0x7f21900cfbf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:19:15.465 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:15.460+0000 7f2187fff640 1 -- 172.21.15.143:0/1880137146 --> v2:172.21.15.188:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f21900ceb20 con 0x7f21900b2160 2023-11-26T07:19:15.465 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:15.460+0000 7f218f7fe640 1 --2- 172.21.15.143:0/1880137146 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f219015d6e0 0x7f21900d0df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-26T07:19:15.467 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:15.464+0000 7f2187fff640 1 --2- 172.21.15.143:0/1880137146 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f21900b2160 0x7f21900d14b0 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7f2178009870 tx=0x7f217801c540 comp rx=0 tx=0).ready entity=mon.1 client_cookie=ccb51b745969efa5 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:19:15.467 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:15.464+0000 7f218effd640 1 -- 172.21.15.143:0/1880137146 <== mon.1 v2:172.21.15.188:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f21780a7070 con 0x7f21900b2160 2023-11-26T07:19:15.467 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:15.464+0000 7f218effd640 1 -- 172.21.15.143:0/1880137146 <== mon.1 v2:172.21.15.188:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f21780ab430 con 0x7f21900b2160 2023-11-26T07:19:15.467 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:15.464+0000 7f218effd640 1 -- 172.21.15.143:0/1880137146 <== mon.1 v2:172.21.15.188:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f21780ab7c0 con 0x7f21900b2160 2023-11-26T07:19:15.468 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:15.464+0000 7f2195b8d640 1 -- 172.21.15.143:0/1880137146 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f21900b2160 msgr2=0x7f21900d14b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:19:15.468 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:15.464+0000 7f2195b8d640 1 --2- 172.21.15.143:0/1880137146 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f21900b2160 0x7f21900d14b0 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7f2178009870 tx=0x7f217801c540 comp rx=0 tx=0).stop 2023-11-26T07:19:15.468 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:15.464+0000 7f2195b8d640 1 -- 172.21.15.143:0/1880137146 shutdown_connections 2023-11-26T07:19:15.468 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:15.464+0000 7f2195b8d640 1 --2- 172.21.15.143:0/1880137146 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f21900b2160 0x7f21900d14b0 unknown :-1 s=CLOSED pgs=212 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:19:15.468 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:15.464+0000 7f2195b8d640 1 --2- 172.21.15.143:0/1880137146 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f219015d6e0 0x7f21900d0df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:19:15.468 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:15.464+0000 7f2195b8d640 1 -- 172.21.15.143:0/1880137146 >> 172.21.15.143:0/1880137146 conn(0x7f219009c550 msgr2=0x7f21900d3220 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:19:15.468 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:15.464+0000 7f2195b8d640 1 -- 172.21.15.143:0/1880137146 shutdown_connections 2023-11-26T07:19:15.468 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:15.464+0000 7f2195b8d640 1 -- 172.21.15.143:0/1880137146 wait complete. 2023-11-26T07:19:15.469 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:15.468+0000 7f2195b8d640 1 Processor -- start 2023-11-26T07:19:15.469 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:15.468+0000 7f2195b8d640 1 -- start start 2023-11-26T07:19:15.471 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:15.468+0000 7f2195b8d640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f21900b2160 0x7f2190154ae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:19:15.471 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:15.468+0000 7f2195b8d640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f21900d19f0 0x7f219015c160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:19:15.472 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:15.468+0000 7f2195b8d640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f219015d6e0 0x7f219015c6a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:19:15.472 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:15.468+0000 7f2195b8d640 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f21900cd380 con 0x7f219015d6e0 2023-11-26T07:19:15.472 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:15.468+0000 7f2195b8d640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f21900b6600 con 0x7f21900d19f0 2023-11-26T07:19:15.472 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:15.468+0000 7f2195b8d640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2190155020 con 0x7f21900b2160 2023-11-26T07:19:15.472 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:15.468+0000 7f218ffff640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f219015d6e0 0x7f219015c6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:19:15.472 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:15.468+0000 7f218ffff640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f219015d6e0 0x7f219015c6a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:47934/0 (socket says 172.21.15.143:47934) 2023-11-26T07:19:15.472 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:15.468+0000 7f218ffff640 1 -- 172.21.15.143:0/1938636341 learned_addr learned my addr 172.21.15.143:0/1938636341 (peer_addr_for_me v2:172.21.15.143:0/0) 2023-11-26T07:19:15.472 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:15.468+0000 7f218f7fe640 1 --2- 172.21.15.143:0/1938636341 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f21900b2160 0x7f2190154ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:19:15.472 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:15.468+0000 7f2187fff640 1 --2- 172.21.15.143:0/1938636341 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f21900d19f0 0x7f219015c160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:19:15.472 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:15.468+0000 7f218ffff640 1 -- 172.21.15.143:0/1938636341 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f21900b2160 msgr2=0x7f2190154ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:19:15.472 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:15.468+0000 7f218ffff640 1 --2- 172.21.15.143:0/1938636341 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f21900b2160 0x7f2190154ae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:19:15.473 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:15.468+0000 7f218ffff640 1 -- 172.21.15.143:0/1938636341 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f21900d19f0 msgr2=0x7f219015c160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:19:15.473 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:15.468+0000 7f218ffff640 1 --2- 172.21.15.143:0/1938636341 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f21900d19f0 0x7f219015c160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:19:15.473 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:15.468+0000 7f218ffff640 1 -- 172.21.15.143:0/1938636341 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2180009420 con 0x7f219015d6e0 2023-11-26T07:19:15.473 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:15.468+0000 7f2187fff640 1 --2- 172.21.15.143:0/1938636341 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f21900d19f0 0x7f219015c160 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-26T07:19:15.473 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:15.468+0000 7f218f7fe640 1 --2- 172.21.15.143:0/1938636341 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f21900b2160 0x7f2190154ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-26T07:19:15.473 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:15.468+0000 7f218ffff640 1 --2- 172.21.15.143:0/1938636341 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f219015d6e0 0x7f219015c6a0 secure :-1 s=READY pgs=487 cs=0 l=1 rev1=1 crypto rx=0x7f218800c770 tx=0x7f218800cc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:19:15.474 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:15.468+0000 7f218cff9640 1 -- 172.21.15.143:0/1938636341 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2188003b80 con 0x7f219015d6e0 2023-11-26T07:19:15.474 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:15.468+0000 7f218cff9640 1 -- 172.21.15.143:0/1938636341 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2188003ce0 con 0x7f219015d6e0 2023-11-26T07:19:15.474 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:15.468+0000 7f2195b8d640 1 -- 172.21.15.143:0/1938636341 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f21780094d0 con 0x7f219015d6e0 2023-11-26T07:19:15.474 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:15.468+0000 7f218cff9640 1 -- 172.21.15.143:0/1938636341 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f218800aa90 con 0x7f219015d6e0 2023-11-26T07:19:15.474 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:15.468+0000 7f2195b8d640 1 -- 172.21.15.143:0/1938636341 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f21900b5fb0 con 0x7f219015d6e0 2023-11-26T07:19:15.474 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:15.472+0000 7f2195b8d640 1 -- 172.21.15.143:0/1938636341 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f21900b5e10 con 0x7f219015d6e0 2023-11-26T07:19:15.474 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:15.472+0000 7f218cff9640 1 -- 172.21.15.143:0/1938636341 <== mon.0 v2:172.21.15.143:3300/0 4 ==== mgrmap(e 7) v1 ==== 86005+0+0 (secure 0 0 0) 0x7f218800abf0 con 0x7f219015d6e0 2023-11-26T07:19:15.475 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:15.472+0000 7f218cff9640 1 --2- 172.21.15.143:0/1938636341 >> [v2:172.21.15.143:6832/2150557544,v1:172.21.15.143:6833/2150557544] conn(0x7f2170066650 0x7f2170068b10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:19:15.475 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:15.472+0000 7f218cff9640 1 -- 172.21.15.143:0/1938636341 <== mon.0 v2:172.21.15.143:3300/0 5 ==== osd_map(33..33 src has 1..33) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f2188084040 con 0x7f219015d6e0 2023-11-26T07:19:15.475 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:15.472+0000 7f218f7fe640 1 --2- 172.21.15.143:0/1938636341 >> [v2:172.21.15.143:6832/2150557544,v1:172.21.15.143:6833/2150557544] conn(0x7f2170066650 0x7f2170068b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:19:15.476 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:15.472+0000 7f218f7fe640 1 --2- 172.21.15.143:0/1938636341 >> [v2:172.21.15.143:6832/2150557544,v1:172.21.15.143:6833/2150557544] conn(0x7f2170066650 0x7f2170068b10 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 crypto rx=0x7f2180009ca0 tx=0x7f2180008040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:19:15.479 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:15.476+0000 7f218cff9640 1 -- 172.21.15.143:0/1938636341 <== mon.0 v2:172.21.15.143:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172649 (secure 0 0 0) 0x7f21880b5280 con 0x7f219015d6e0 2023-11-26T07:19:15.663 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:15.660+0000 7f2195b8d640 1 -- 172.21.15.143:0/1938636341 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f2190067680 con 0x7f219015d6e0 2023-11-26T07:19:15.663 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:15.660+0000 7f218cff9640 1 -- 172.21.15.143:0/1938636341 <== mon.0 v2:172.21.15.143:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 17 v17) v1 ==== 94+0+4925 (secure 0 0 0) 0x7f21900b5e10 con 0x7f219015d6e0 2023-11-26T07:19:15.664 INFO:teuthology.orchestra.run.smithi143.stdout: 2023-11-26T07:19:15.664 INFO:teuthology.orchestra.run.smithi143.stdout:{"epoch":17,"default_fscid":2,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4847,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.188:6834/1547410397","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6832","nonce":1547410397},{"type":"v1","addr":"172.21.15.188:6834","nonce":1547410397}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":12},{"gid":4850,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.188:6835/3247640329","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6833","nonce":3247640329},{"type":"v1","addr":"172.21.15.188:6835","nonce":3247640329}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":12},{"gid":5217,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.143:6835/2123707650","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6834","nonce":2123707650},{"type":"v1","addr":"172.21.15.143:6835","nonce":2123707650}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":11}],"filesystems":[{"mdsmap":{"epoch":16,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-11-26T07:18:52.014584+0000","modified":"2023-11-26T07:18:55.043646+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":5220},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5220":{"gid":5220,"name":"c","rank":0,"incarnation":13,"state":"up:active","state_seq":3,"addr":"172.21.15.143:6837/2658667363","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6836","nonce":2658667363},{"type":"v1","addr":"172.21.15.143:6837","nonce":2658667363}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[5],"metadata_pool":4,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":2}]} 2023-11-26T07:19:15.664 INFO:teuthology.orchestra.run.smithi143.stderr:dumped fsmap epoch 17 2023-11-26T07:19:15.666 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:15.664+0000 7f2195b8d640 1 -- 172.21.15.143:0/1938636341 >> [v2:172.21.15.143:6832/2150557544,v1:172.21.15.143:6833/2150557544] conn(0x7f2170066650 msgr2=0x7f2170068b10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:19:15.666 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:15.664+0000 7f2195b8d640 1 --2- 172.21.15.143:0/1938636341 >> [v2:172.21.15.143:6832/2150557544,v1:172.21.15.143:6833/2150557544] conn(0x7f2170066650 0x7f2170068b10 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 crypto rx=0x7f2180009ca0 tx=0x7f2180008040 comp rx=0 tx=0).stop 2023-11-26T07:19:15.666 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:15.664+0000 7f2195b8d640 1 -- 172.21.15.143:0/1938636341 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f219015d6e0 msgr2=0x7f219015c6a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:19:15.666 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:15.664+0000 7f2195b8d640 1 --2- 172.21.15.143:0/1938636341 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f219015d6e0 0x7f219015c6a0 secure :-1 s=READY pgs=487 cs=0 l=1 rev1=1 crypto rx=0x7f218800c770 tx=0x7f218800cc30 comp rx=0 tx=0).stop 2023-11-26T07:19:15.666 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:15.664+0000 7f2195b8d640 1 -- 172.21.15.143:0/1938636341 shutdown_connections 2023-11-26T07:19:15.666 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:15.664+0000 7f2195b8d640 1 --2- 172.21.15.143:0/1938636341 >> [v2:172.21.15.143:6832/2150557544,v1:172.21.15.143:6833/2150557544] conn(0x7f2170066650 0x7f2170068b10 unknown :-1 s=CLOSED pgs=250 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:19:15.666 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:15.664+0000 7f2195b8d640 1 --2- 172.21.15.143:0/1938636341 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f219015d6e0 0x7f219015c6a0 unknown :-1 s=CLOSED pgs=487 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:19:15.666 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:15.664+0000 7f2195b8d640 1 --2- 172.21.15.143:0/1938636341 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f21900d19f0 0x7f219015c160 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:19:15.666 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:15.664+0000 7f2195b8d640 1 --2- 172.21.15.143:0/1938636341 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f21900b2160 0x7f2190154ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:19:15.666 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:15.664+0000 7f2195b8d640 1 -- 172.21.15.143:0/1938636341 >> 172.21.15.143:0/1938636341 conn(0x7f219009c550 msgr2=0x7f21900cfbe0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:19:15.667 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:15.664+0000 7f2195b8d640 1 -- 172.21.15.143:0/1938636341 shutdown_connections 2023-11-26T07:19:15.667 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:15.664+0000 7f2195b8d640 1 -- 172.21.15.143:0/1938636341 wait complete. 2023-11-26T07:19:15.685 DEBUG:teuthology.orchestra.run.smithi143:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-11-26T07:19:15.823 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:15.820+0000 7fcf502d6640 1 Processor -- start 2023-11-26T07:19:15.823 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:15.820+0000 7fcf502d6640 1 -- start start 2023-11-26T07:19:15.824 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:15.820+0000 7fcf502d6640 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7fcf480ce7c0 0x7fcf4815f6b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:19:15.824 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:15.820+0000 7fcf502d6640 1 -- --> v1:172.21.15.143:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fcf4806c520 con 0x7fcf480cf990 2023-11-26T07:19:15.825 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:15.820+0000 7fcf502d6640 1 -- --> v1:172.21.15.188:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fcf4806c800 con 0x7fcf4813e290 2023-11-26T07:19:15.825 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:15.820+0000 7fcf502d6640 1 -- --> v2:172.21.15.188:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fcf4806cae0 con 0x7fcf480ce7c0 2023-11-26T07:19:15.825 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:15.824+0000 7fcf4d871640 1 --1- >> v1:172.21.15.143:6789/0 conn(0x7fcf480cf990 0x7fcf4815efa0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.143:6789/0 says I am v1:172.21.15.143:50088/0 (socket says 172.21.15.143:50088) 2023-11-26T07:19:15.825 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:15.824+0000 7fcf4d871640 1 -- 172.21.15.143:0/630362527 learned_addr learned my addr 172.21.15.143:0/630362527 (peer_addr_for_me v1:172.21.15.143:0/0) 2023-11-26T07:19:15.826 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:15.824+0000 7fcf4e873640 1 --2- 172.21.15.143:0/630362527 >> v2:172.21.15.188:3300/0 conn(0x7fcf480ce7c0 0x7fcf4815f6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:19:15.826 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:15.824+0000 7fcf4d070640 1 -- 172.21.15.143:0/630362527 <== mon.0 v1:172.21.15.143:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3861562214 0 0) 0x7fcf4806c520 con 0x7fcf480cf990 2023-11-26T07:19:15.826 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:15.824+0000 7fcf4d070640 1 -- 172.21.15.143:0/630362527 --> v1:172.21.15.143:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fcf34003530 con 0x7fcf480cf990 2023-11-26T07:19:15.826 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:15.824+0000 7fcf4e873640 1 -- 172.21.15.143:0/630362527 >> v1:172.21.15.188:6789/0 conn(0x7fcf4813e290 legacy=0x7fcf480cf280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:19:15.826 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:15.824+0000 7fcf4d070640 1 -- 172.21.15.143:0/630362527 <== mon.0 v1:172.21.15.143:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2759197746 0 0) 0x7fcf34003530 con 0x7fcf480cf990 2023-11-26T07:19:15.826 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:15.824+0000 7fcf4e873640 1 -- 172.21.15.143:0/630362527 >> v1:172.21.15.143:6789/0 conn(0x7fcf480cf990 legacy=0x7fcf4815efa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:19:15.826 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:15.824+0000 7fcf4e873640 1 -- 172.21.15.143:0/630362527 --> v2:172.21.15.188:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcf4806cdc0 con 0x7fcf480ce7c0 2023-11-26T07:19:15.828 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:15.824+0000 7fcf4e873640 1 --2- 172.21.15.143:0/630362527 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fcf480ce7c0 0x7fcf4815f6b0 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7fcf40009d90 tx=0x7fcf4001c7b0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=a6a62985b34a0938 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:19:15.828 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:15.824+0000 7fcf4d070640 1 -- 172.21.15.143:0/630362527 <== mon.1 v2:172.21.15.188:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcf38001680 con 0x7fcf480ce7c0 2023-11-26T07:19:15.829 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:15.824+0000 7fcf4d070640 1 -- 172.21.15.143:0/630362527 <== mon.1 v2:172.21.15.188:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcf400a5610 con 0x7fcf480ce7c0 2023-11-26T07:19:15.829 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:15.824+0000 7fcf4d070640 1 -- 172.21.15.143:0/630362527 <== mon.1 v2:172.21.15.188:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcf400a59a0 con 0x7fcf480ce7c0 2023-11-26T07:19:15.829 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:15.828+0000 7fcf502d6640 1 -- 172.21.15.143:0/630362527 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fcf480ce7c0 msgr2=0x7fcf4815f6b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:19:15.829 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:15.828+0000 7fcf502d6640 1 --2- 172.21.15.143:0/630362527 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fcf480ce7c0 0x7fcf4815f6b0 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7fcf40009d90 tx=0x7fcf4001c7b0 comp rx=0 tx=0).stop 2023-11-26T07:19:15.830 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:15.828+0000 7fcf502d6640 1 -- 172.21.15.143:0/630362527 shutdown_connections 2023-11-26T07:19:15.830 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:15.828+0000 7fcf502d6640 1 --2- 172.21.15.143:0/630362527 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fcf480ce7c0 0x7fcf4815f6b0 unknown :-1 s=CLOSED pgs=214 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:19:15.830 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:15.828+0000 7fcf502d6640 1 -- 172.21.15.143:0/630362527 >> 172.21.15.143:0/630362527 conn(0x7fcf4806a890 msgr2=0x7fcf4806aca0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:19:15.830 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:15.828+0000 7fcf502d6640 1 -- 172.21.15.143:0/630362527 shutdown_connections 2023-11-26T07:19:15.830 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:15.828+0000 7fcf502d6640 1 -- 172.21.15.143:0/630362527 wait complete. 2023-11-26T07:19:15.831 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:15.828+0000 7fcf502d6640 1 Processor -- start 2023-11-26T07:19:15.831 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:15.828+0000 7fcf502d6640 1 -- start start 2023-11-26T07:19:15.833 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:15.828+0000 7fcf502d6640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fcf480ce7c0 0x7fcf4815d940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:19:15.833 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:15.828+0000 7fcf502d6640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fcf480cf990 0x7fcf481367b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:19:15.833 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:15.828+0000 7fcf502d6640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fcf4813e290 0x7fcf48136cf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:19:15.833 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:15.828+0000 7fcf502d6640 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcf48070ad0 con 0x7fcf480cf990 2023-11-26T07:19:15.833 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:15.828+0000 7fcf502d6640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcf48137230 con 0x7fcf480ce7c0 2023-11-26T07:19:15.833 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:15.828+0000 7fcf502d6640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcf480b1650 con 0x7fcf4813e290 2023-11-26T07:19:15.833 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:15.828+0000 7fcf4d871640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fcf480cf990 0x7fcf481367b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:19:15.833 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:15.828+0000 7fcf4d871640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fcf480cf990 0x7fcf481367b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:47940/0 (socket says 172.21.15.143:47940) 2023-11-26T07:19:15.833 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:15.828+0000 7fcf4d871640 1 -- 172.21.15.143:0/1892664327 learned_addr learned my addr 172.21.15.143:0/1892664327 (peer_addr_for_me v2:172.21.15.143:0/0) 2023-11-26T07:19:15.833 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:15.828+0000 7fcf4e873640 1 --2- 172.21.15.143:0/1892664327 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fcf4813e290 0x7fcf48136cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:19:15.834 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:15.828+0000 7fcf4e072640 1 --2- 172.21.15.143:0/1892664327 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fcf480ce7c0 0x7fcf4815d940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:19:15.834 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:15.832+0000 7fcf4d871640 1 -- 172.21.15.143:0/1892664327 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fcf4813e290 msgr2=0x7fcf48136cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:19:15.834 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:15.832+0000 7fcf4d871640 1 --2- 172.21.15.143:0/1892664327 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fcf4813e290 0x7fcf48136cf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:19:15.834 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:15.832+0000 7fcf4d871640 1 -- 172.21.15.143:0/1892664327 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fcf480ce7c0 msgr2=0x7fcf4815d940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:19:15.834 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:15.832+0000 7fcf4d871640 1 --2- 172.21.15.143:0/1892664327 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fcf480ce7c0 0x7fcf4815d940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:19:15.834 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:15.832+0000 7fcf4d871640 1 -- 172.21.15.143:0/1892664327 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcf40009a70 con 0x7fcf480cf990 2023-11-26T07:19:15.834 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:15.832+0000 7fcf4d871640 1 --2- 172.21.15.143:0/1892664327 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fcf480cf990 0x7fcf481367b0 secure :-1 s=READY pgs=489 cs=0 l=1 rev1=1 crypto rx=0x7fcf3800c730 tx=0x7fcf3800cbf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:19:15.835 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:15.832+0000 7fcf3effd640 1 -- 172.21.15.143:0/1892664327 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcf38005ae0 con 0x7fcf480cf990 2023-11-26T07:19:15.835 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:15.832+0000 7fcf3effd640 1 -- 172.21.15.143:0/1892664327 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcf38005c40 con 0x7fcf480cf990 2023-11-26T07:19:15.835 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:15.832+0000 7fcf3effd640 1 -- 172.21.15.143:0/1892664327 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcf38007070 con 0x7fcf480cf990 2023-11-26T07:19:15.835 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:15.832+0000 7fcf502d6640 1 -- 172.21.15.143:0/1892664327 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcf480b1930 con 0x7fcf480cf990 2023-11-26T07:19:15.835 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:15.832+0000 7fcf502d6640 1 -- 172.21.15.143:0/1892664327 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcf480706f0 con 0x7fcf480cf990 2023-11-26T07:19:15.835 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:15.832+0000 7fcf502d6640 1 -- 172.21.15.143:0/1892664327 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fcf48067680 con 0x7fcf480cf990 2023-11-26T07:19:15.836 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:15.832+0000 7fcf3effd640 1 -- 172.21.15.143:0/1892664327 <== mon.0 v2:172.21.15.143:3300/0 4 ==== mgrmap(e 7) v1 ==== 86005+0+0 (secure 0 0 0) 0x7fcf380071d0 con 0x7fcf480cf990 2023-11-26T07:19:15.836 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:15.832+0000 7fcf3effd640 1 --2- 172.21.15.143:0/1892664327 >> [v2:172.21.15.143:6832/2150557544,v1:172.21.15.143:6833/2150557544] conn(0x7fcf2c066620 0x7fcf2c068ae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:19:15.836 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:15.832+0000 7fcf3effd640 1 -- 172.21.15.143:0/1892664327 <== mon.0 v2:172.21.15.143:3300/0 5 ==== osd_map(33..33 src has 1..33) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fcf380838f0 con 0x7fcf480cf990 2023-11-26T07:19:15.836 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:15.832+0000 7fcf4e072640 1 --2- 172.21.15.143:0/1892664327 >> [v2:172.21.15.143:6832/2150557544,v1:172.21.15.143:6833/2150557544] conn(0x7fcf2c066620 0x7fcf2c068ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:19:15.837 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:15.836+0000 7fcf4e072640 1 --2- 172.21.15.143:0/1892664327 >> [v2:172.21.15.143:6832/2150557544,v1:172.21.15.143:6833/2150557544] conn(0x7fcf2c066620 0x7fcf2c068ae0 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 crypto rx=0x7fcf44007fd0 tx=0x7fcf44009040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:19:15.840 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:15.836+0000 7fcf3effd640 1 -- 172.21.15.143:0/1892664327 <== mon.0 v2:172.21.15.143:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172649 (secure 0 0 0) 0x7fcf38050bb0 con 0x7fcf480cf990 2023-11-26T07:19:15.993 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:15.988+0000 7fcf502d6640 1 -- 172.21.15.143:0/1892664327 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fcf4806df50 con 0x7fcf480cf990 2023-11-26T07:19:15.993 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:15.992+0000 7fcf3effd640 1 -- 172.21.15.143:0/1892664327 <== mon.0 v2:172.21.15.143:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v33) v1 ==== 74+0+16605 (secure 0 0 0) 0x7fcf38017020 con 0x7fcf480cf990 2023-11-26T07:19:15.994 INFO:teuthology.orchestra.run.smithi143.stdout: 2023-11-26T07:19:15.994 INFO:teuthology.orchestra.run.smithi143.stdout:{"epoch":33,"fsid":"f5033060-2469-4a07-95fe-2b76f5404bc1","created":"2023-11-26T07:17:28.613663+0000","modified":"2023-11-26T07:18:53.659425+0000","last_up_change":"2023-11-26T07:17:41.157813+0000","last_in_change":"2023-11-26T07:17:31.181658+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":5,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-11-26T07:17:41.758597+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_metadata","create_time":"2023-11-26T07:18:50.535934+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"32","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":5,"pool_name":"cephfs_data","create_time":"2023-11-26T07:18:51.008494+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"33","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"ebbee2c6-023e-44b9-9f1d-6f7bfd309730","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6817","nonce":639807369},{"type":"v1","addr":"172.21.15.143:6819","nonce":639807369}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6821","nonce":639807369},{"type":"v1","addr":"172.21.15.143:6823","nonce":639807369}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6830","nonce":639807369},{"type":"v1","addr":"172.21.15.143:6831","nonce":639807369}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6825","nonce":639807369},{"type":"v1","addr":"172.21.15.143:6828","nonce":639807369}]},"public_addr":"172.21.15.143:6819/639807369","cluster_addr":"172.21.15.143:6823/639807369","heartbeat_back_addr":"172.21.15.143:6831/639807369","heartbeat_front_addr":"172.21.15.143:6828/639807369","state":["exists","up"]},{"osd":1,"uuid":"fec4b8be-7122-49e9-9a19-5850a0700001","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6816","nonce":1144942530},{"type":"v1","addr":"172.21.15.143:6818","nonce":1144942530}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6820","nonce":1144942530},{"type":"v1","addr":"172.21.15.143:6822","nonce":1144942530}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6827","nonce":1144942530},{"type":"v1","addr":"172.21.15.143:6829","nonce":1144942530}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6824","nonce":1144942530},{"type":"v1","addr":"172.21.15.143:6826","nonce":1144942530}]},"public_addr":"172.21.15.143:6818/1144942530","cluster_addr":"172.21.15.143:6822/1144942530","heartbeat_back_addr":"172.21.15.143:6829/1144942530","heartbeat_front_addr":"172.21.15.143:6826/1144942530","state":["exists","up"]},{"osd":2,"uuid":"2888af5b-f933-46ca-8f95-6dd494ff43f6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6808","nonce":3909095186},{"type":"v1","addr":"172.21.15.143:6809","nonce":3909095186}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6810","nonce":3909095186},{"type":"v1","addr":"172.21.15.143:6811","nonce":3909095186}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6814","nonce":3909095186},{"type":"v1","addr":"172.21.15.143:6815","nonce":3909095186}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6812","nonce":3909095186},{"type":"v1","addr":"172.21.15.143:6813","nonce":3909095186}]},"public_addr":"172.21.15.143:6809/3909095186","cluster_addr":"172.21.15.143:6811/3909095186","heartbeat_back_addr":"172.21.15.143:6815/3909095186","heartbeat_front_addr":"172.21.15.143:6813/3909095186","state":["exists","up"]},{"osd":3,"uuid":"d1f52435-7bc1-4081-9550-9cd1ba55f549","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6800","nonce":3854210991},{"type":"v1","addr":"172.21.15.143:6801","nonce":3854210991}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6802","nonce":3854210991},{"type":"v1","addr":"172.21.15.143:6803","nonce":3854210991}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6806","nonce":3854210991},{"type":"v1","addr":"172.21.15.143:6807","nonce":3854210991}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6804","nonce":3854210991},{"type":"v1","addr":"172.21.15.143:6805","nonce":3854210991}]},"public_addr":"172.21.15.143:6801/3854210991","cluster_addr":"172.21.15.143:6803/3854210991","heartbeat_back_addr":"172.21.15.143:6807/3854210991","heartbeat_front_addr":"172.21.15.143:6805/3854210991","state":["exists","up"]},{"osd":4,"uuid":"e2ae8b6e-6e25-49b2-bcbf-aacce68f06a3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6800","nonce":3924606902},{"type":"v1","addr":"172.21.15.188:6801","nonce":3924606902}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6802","nonce":3924606902},{"type":"v1","addr":"172.21.15.188:6803","nonce":3924606902}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6806","nonce":3924606902},{"type":"v1","addr":"172.21.15.188:6807","nonce":3924606902}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6804","nonce":3924606902},{"type":"v1","addr":"172.21.15.188:6805","nonce":3924606902}]},"public_addr":"172.21.15.188:6801/3924606902","cluster_addr":"172.21.15.188:6803/3924606902","heartbeat_back_addr":"172.21.15.188:6807/3924606902","heartbeat_front_addr":"172.21.15.188:6805/3924606902","state":["exists","up"]},{"osd":5,"uuid":"3cec8a32-7698-4c36-8e89-e4ba11934f33","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6813","nonce":186841544},{"type":"v1","addr":"172.21.15.188:6815","nonce":186841544}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6817","nonce":186841544},{"type":"v1","addr":"172.21.15.188:6818","nonce":186841544}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6822","nonce":186841544},{"type":"v1","addr":"172.21.15.188:6823","nonce":186841544}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6820","nonce":186841544},{"type":"v1","addr":"172.21.15.188:6821","nonce":186841544}]},"public_addr":"172.21.15.188:6815/186841544","cluster_addr":"172.21.15.188:6818/186841544","heartbeat_back_addr":"172.21.15.188:6823/186841544","heartbeat_front_addr":"172.21.15.188:6821/186841544","state":["exists","up"]},{"osd":6,"uuid":"58a65d5d-e090-473a-81b4-ad38e85029f9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6808","nonce":2153499405},{"type":"v1","addr":"172.21.15.188:6809","nonce":2153499405}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6810","nonce":2153499405},{"type":"v1","addr":"172.21.15.188:6811","nonce":2153499405}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6816","nonce":2153499405},{"type":"v1","addr":"172.21.15.188:6819","nonce":2153499405}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6812","nonce":2153499405},{"type":"v1","addr":"172.21.15.188:6814","nonce":2153499405}]},"public_addr":"172.21.15.188:6809/2153499405","cluster_addr":"172.21.15.188:6811/2153499405","heartbeat_back_addr":"172.21.15.188:6819/2153499405","heartbeat_front_addr":"172.21.15.188:6814/2153499405","state":["exists","up"]},{"osd":7,"uuid":"ea5b7f70-3554-4cf8-ac30-f7c59b80678d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6824","nonce":2722096729},{"type":"v1","addr":"172.21.15.188:6825","nonce":2722096729}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6826","nonce":2722096729},{"type":"v1","addr":"172.21.15.188:6827","nonce":2722096729}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6830","nonce":2722096729},{"type":"v1","addr":"172.21.15.188:6831","nonce":2722096729}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6828","nonce":2722096729},{"type":"v1","addr":"172.21.15.188:6829","nonce":2722096729}]},"public_addr":"172.21.15.188:6825/2722096729","cluster_addr":"172.21.15.188:6827/2722096729","heartbeat_back_addr":"172.21.15.188:6831/2722096729","heartbeat_front_addr":"172.21.15.188:6829/2722096729","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-26T07:17:36.995032+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-26T07:17:36.932266+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-26T07:17:36.829304+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-26T07:17:36.804297+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-26T07:17:36.667892+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-26T07:17:36.964868+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-26T07:17:36.827332+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-26T07:17:36.893179+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-11-26T07:19:15.996 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:15.992+0000 7fcf502d6640 1 -- 172.21.15.143:0/1892664327 >> [v2:172.21.15.143:6832/2150557544,v1:172.21.15.143:6833/2150557544] conn(0x7fcf2c066620 msgr2=0x7fcf2c068ae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:19:15.996 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:15.992+0000 7fcf502d6640 1 --2- 172.21.15.143:0/1892664327 >> [v2:172.21.15.143:6832/2150557544,v1:172.21.15.143:6833/2150557544] conn(0x7fcf2c066620 0x7fcf2c068ae0 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 crypto rx=0x7fcf44007fd0 tx=0x7fcf44009040 comp rx=0 tx=0).stop 2023-11-26T07:19:15.996 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:15.992+0000 7fcf502d6640 1 -- 172.21.15.143:0/1892664327 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fcf480cf990 msgr2=0x7fcf481367b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:19:15.996 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:15.992+0000 7fcf502d6640 1 --2- 172.21.15.143:0/1892664327 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fcf480cf990 0x7fcf481367b0 secure :-1 s=READY pgs=489 cs=0 l=1 rev1=1 crypto rx=0x7fcf3800c730 tx=0x7fcf3800cbf0 comp rx=0 tx=0).stop 2023-11-26T07:19:15.996 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:15.992+0000 7fcf502d6640 1 -- 172.21.15.143:0/1892664327 shutdown_connections 2023-11-26T07:19:15.997 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:15.992+0000 7fcf502d6640 1 --2- 172.21.15.143:0/1892664327 >> [v2:172.21.15.143:6832/2150557544,v1:172.21.15.143:6833/2150557544] conn(0x7fcf2c066620 0x7fcf2c068ae0 unknown :-1 s=CLOSED pgs=251 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:19:15.997 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:15.992+0000 7fcf502d6640 1 --2- 172.21.15.143:0/1892664327 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fcf4813e290 0x7fcf48136cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:19:15.997 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:15.992+0000 7fcf502d6640 1 --2- 172.21.15.143:0/1892664327 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fcf480cf990 0x7fcf481367b0 unknown :-1 s=CLOSED pgs=489 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:19:15.997 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:15.992+0000 7fcf502d6640 1 --2- 172.21.15.143:0/1892664327 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fcf480ce7c0 0x7fcf4815d940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:19:15.997 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:15.992+0000 7fcf502d6640 1 -- 172.21.15.143:0/1892664327 >> 172.21.15.143:0/1892664327 conn(0x7fcf4806a890 msgr2=0x7fcf480b1cb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:19:15.997 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:15.992+0000 7fcf502d6640 1 -- 172.21.15.143:0/1892664327 shutdown_connections 2023-11-26T07:19:15.997 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:15.992+0000 7fcf502d6640 1 -- 172.21.15.143:0/1892664327 wait complete. 2023-11-26T07:19:16.015 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-11-26T07:17:41.758597+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_metadata', 'create_time': '2023-11-26T07:18:50.535934+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '32', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 2.25, 'score_stable': 2.25, 'optimal_score': 1, 'raw_score_acting': 2.25, 'raw_score_stable': 2.25, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 5, 'pool_name': 'cephfs_data', 'create_time': '2023-11-26T07:18:51.008494+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '33', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.3799999952316284, 'score_stable': 1.3799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.3799999952316284, 'raw_score_stable': 1.3799999952316284, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2023-11-26T07:19:16.015 DEBUG:teuthology.orchestra.run.smithi143:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-11-26T07:19:16.156 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:16.152+0000 7f22014c9640 1 Processor -- start 2023-11-26T07:19:16.157 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:16.156+0000 7f22014c9640 1 -- start start 2023-11-26T07:19:16.158 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:16.156+0000 7f22014c9640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f21fc15d580 0x7f21fc0d3920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:19:16.158 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:16.156+0000 7f22014c9640 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7f21fc0d3e60 0x7f21fc0d17f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:19:16.158 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:16.156+0000 7f22014c9640 1 -- --> v1:172.21.15.143:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f21fc0d21e0 con 0x7f21fc0d1d30 2023-11-26T07:19:16.158 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:16.156+0000 7f22014c9640 1 -- --> v2:172.21.15.188:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f21fc0d0bc0 con 0x7f21fc0d3e60 2023-11-26T07:19:16.158 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:16.156+0000 7f22014c9640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f21fc0d0320 con 0x7f21fc15d580 2023-11-26T07:19:16.158 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:16.156+0000 7f21fb7fe640 1 --1- >> v1:172.21.15.143:6789/0 conn(0x7f21fc0d1d30 0x7f21fc0cdce0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.143:6789/0 says I am v1:172.21.15.143:50096/0 (socket says 172.21.15.143:50096) 2023-11-26T07:19:16.158 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:16.156+0000 7f21fb7fe640 1 -- 172.21.15.143:0/376542838 learned_addr learned my addr 172.21.15.143:0/376542838 (peer_addr_for_me v1:172.21.15.143:0/0) 2023-11-26T07:19:16.159 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:16.156+0000 7f21f3fff640 1 --2- 172.21.15.143:0/376542838 >> v2:172.21.15.188:3300/0 conn(0x7f21fc0d3e60 0x7f21fc0d17f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:19:16.159 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:16.156+0000 7f21faffd640 1 --2- 172.21.15.143:0/376542838 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f21fc15d580 0x7f21fc0d3920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:19:16.159 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:16.156+0000 7f21fa7fc640 1 -- 172.21.15.143:0/376542838 <== mon.0 v1:172.21.15.143:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1328550981 0 0) 0x7f21fc0d21e0 con 0x7f21fc0d1d30 2023-11-26T07:19:16.159 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:16.156+0000 7f21fa7fc640 1 -- 172.21.15.143:0/376542838 --> v1:172.21.15.143:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f21e8003530 con 0x7f21fc0d1d30 2023-11-26T07:19:16.160 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:16.156+0000 7f21f3fff640 1 -- 172.21.15.143:0/376542838 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f21fc15d580 msgr2=0x7f21fc0d3920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-26T07:19:16.160 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:16.156+0000 7f21f3fff640 1 --2- 172.21.15.143:0/376542838 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f21fc15d580 0x7f21fc0d3920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:19:16.160 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:16.156+0000 7f21f3fff640 1 -- 172.21.15.143:0/376542838 >> v1:172.21.15.143:6789/0 conn(0x7f21fc0d1d30 legacy=0x7f21fc0cdce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:19:16.160 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:16.156+0000 7f21f3fff640 1 -- 172.21.15.143:0/376542838 --> v2:172.21.15.188:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f21fc0d0600 con 0x7f21fc0d3e60 2023-11-26T07:19:16.160 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:16.156+0000 7f21faffd640 1 --2- 172.21.15.143:0/376542838 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f21fc15d580 0x7f21fc0d3920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-26T07:19:16.162 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:16.160+0000 7f21f3fff640 1 --2- 172.21.15.143:0/376542838 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f21fc0d3e60 0x7f21fc0d17f0 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7f21e40097c0 tx=0x7f21e401c450 comp rx=0 tx=0).ready entity=mon.1 client_cookie=81a36aaf98d6f5c0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:19:16.162 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:16.160+0000 7f21fa7fc640 1 -- 172.21.15.143:0/376542838 <== mon.1 v2:172.21.15.188:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f21e40ab650 con 0x7f21fc0d3e60 2023-11-26T07:19:16.162 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:16.160+0000 7f21fa7fc640 1 -- 172.21.15.143:0/376542838 <== mon.1 v2:172.21.15.188:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f21e40ab7b0 con 0x7f21fc0d3e60 2023-11-26T07:19:16.162 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:16.160+0000 7f21fa7fc640 1 -- 172.21.15.143:0/376542838 <== mon.1 v2:172.21.15.188:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f21e40abb40 con 0x7f21fc0d3e60 2023-11-26T07:19:16.163 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:16.160+0000 7f22014c9640 1 -- 172.21.15.143:0/376542838 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f21fc0d3e60 msgr2=0x7f21fc0d17f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:19:16.163 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:16.160+0000 7f22014c9640 1 --2- 172.21.15.143:0/376542838 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f21fc0d3e60 0x7f21fc0d17f0 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7f21e40097c0 tx=0x7f21e401c450 comp rx=0 tx=0).stop 2023-11-26T07:19:16.163 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:16.160+0000 7f22014c9640 1 -- 172.21.15.143:0/376542838 shutdown_connections 2023-11-26T07:19:16.163 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:16.160+0000 7f22014c9640 1 --2- 172.21.15.143:0/376542838 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f21fc0d3e60 0x7f21fc0d17f0 unknown :-1 s=CLOSED pgs=215 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:19:16.163 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:16.160+0000 7f22014c9640 1 --2- 172.21.15.143:0/376542838 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f21fc15d580 0x7f21fc0d3920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:19:16.163 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:16.160+0000 7f22014c9640 1 -- 172.21.15.143:0/376542838 >> 172.21.15.143:0/376542838 conn(0x7f21fc09c550 msgr2=0x7f21fc0d3210 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:19:16.163 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:16.160+0000 7f22014c9640 1 -- 172.21.15.143:0/376542838 shutdown_connections 2023-11-26T07:19:16.163 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:16.160+0000 7f22014c9640 1 -- 172.21.15.143:0/376542838 wait complete. 2023-11-26T07:19:16.164 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:16.160+0000 7f22014c9640 1 Processor -- start 2023-11-26T07:19:16.164 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:16.160+0000 7f22014c9640 1 -- start start 2023-11-26T07:19:16.166 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:16.164+0000 7f22014c9640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f21fc0d1d30 0x7f21fc14c850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:19:16.166 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:16.164+0000 7f22014c9640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f21fc0d3e60 0x7f21fc15fed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:19:16.166 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:16.164+0000 7f22014c9640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f21fc15d580 0x7f21fc0b8ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:19:16.166 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:16.164+0000 7f22014c9640 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f21fc0b8ff0 con 0x7f21fc0d3e60 2023-11-26T07:19:16.166 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:16.164+0000 7f22014c9640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f21fc160b20 con 0x7f21fc0d1d30 2023-11-26T07:19:16.166 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:16.164+0000 7f22014c9640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f21fc15aa90 con 0x7f21fc15d580 2023-11-26T07:19:16.166 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:16.164+0000 7f21f3fff640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f21fc0d3e60 0x7f21fc15fed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:19:16.166 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:16.164+0000 7f21f3fff640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f21fc0d3e60 0x7f21fc15fed0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:47952/0 (socket says 172.21.15.143:47952) 2023-11-26T07:19:16.167 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:16.164+0000 7f21f3fff640 1 -- 172.21.15.143:0/1345987575 learned_addr learned my addr 172.21.15.143:0/1345987575 (peer_addr_for_me v2:172.21.15.143:0/0) 2023-11-26T07:19:16.167 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:16.164+0000 7f21faffd640 1 --2- 172.21.15.143:0/1345987575 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f21fc0d1d30 0x7f21fc14c850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:19:16.167 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:16.164+0000 7f21fb7fe640 1 --2- 172.21.15.143:0/1345987575 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f21fc15d580 0x7f21fc0b8ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:19:16.167 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:16.164+0000 7f21f3fff640 1 -- 172.21.15.143:0/1345987575 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f21fc15d580 msgr2=0x7f21fc0b8ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:19:16.167 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:16.164+0000 7f21f3fff640 1 --2- 172.21.15.143:0/1345987575 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f21fc15d580 0x7f21fc0b8ab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:19:16.167 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:16.164+0000 7f21f3fff640 1 -- 172.21.15.143:0/1345987575 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f21fc0d1d30 msgr2=0x7f21fc14c850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:19:16.167 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:16.164+0000 7f21f3fff640 1 --2- 172.21.15.143:0/1345987575 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f21fc0d1d30 0x7f21fc14c850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:19:16.167 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:16.164+0000 7f21f3fff640 1 -- 172.21.15.143:0/1345987575 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f21ec009420 con 0x7f21fc0d3e60 2023-11-26T07:19:16.168 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:16.164+0000 7f21f3fff640 1 --2- 172.21.15.143:0/1345987575 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f21fc0d3e60 0x7f21fc15fed0 secure :-1 s=READY pgs=491 cs=0 l=1 rev1=1 crypto rx=0x7f21e40ad040 tx=0x7f21e401ac90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:19:16.168 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:16.164+0000 7f21f37fe640 1 -- 172.21.15.143:0/1345987575 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f21e40bf610 con 0x7f21fc0d3e60 2023-11-26T07:19:16.168 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:16.164+0000 7f21f37fe640 1 -- 172.21.15.143:0/1345987575 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f21e40bf7b0 con 0x7f21fc0d3e60 2023-11-26T07:19:16.168 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:16.164+0000 7f22014c9640 1 -- 172.21.15.143:0/1345987575 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f21e4009420 con 0x7f21fc0d3e60 2023-11-26T07:19:16.169 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:16.164+0000 7f21f37fe640 1 -- 172.21.15.143:0/1345987575 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f21e40bfb60 con 0x7f21fc0d3e60 2023-11-26T07:19:16.169 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:16.164+0000 7f22014c9640 1 -- 172.21.15.143:0/1345987575 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f21fc160840 con 0x7f21fc0d3e60 2023-11-26T07:19:16.169 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:16.164+0000 7f22014c9640 1 -- 172.21.15.143:0/1345987575 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f21fc0b4b00 con 0x7f21fc0d3e60 2023-11-26T07:19:16.169 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:16.164+0000 7f21f37fe640 1 -- 172.21.15.143:0/1345987575 <== mon.0 v2:172.21.15.143:3300/0 4 ==== mgrmap(e 7) v1 ==== 86005+0+0 (secure 0 0 0) 0x7f21e40b5400 con 0x7f21fc0d3e60 2023-11-26T07:19:16.169 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:16.168+0000 7f21f37fe640 1 --2- 172.21.15.143:0/1345987575 >> [v2:172.21.15.143:6832/2150557544,v1:172.21.15.143:6833/2150557544] conn(0x7f21dc066650 0x7f21dc068b10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:19:16.169 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:16.168+0000 7f21f37fe640 1 -- 172.21.15.143:0/1345987575 <== mon.0 v2:172.21.15.143:3300/0 5 ==== osd_map(33..33 src has 1..33) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f21e411eec0 con 0x7f21fc0d3e60 2023-11-26T07:19:16.170 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:16.168+0000 7f21faffd640 1 --2- 172.21.15.143:0/1345987575 >> [v2:172.21.15.143:6832/2150557544,v1:172.21.15.143:6833/2150557544] conn(0x7f21dc066650 0x7f21dc068b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:19:16.170 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:16.168+0000 7f21faffd640 1 --2- 172.21.15.143:0/1345987575 >> [v2:172.21.15.143:6832/2150557544,v1:172.21.15.143:6833/2150557544] conn(0x7f21dc066650 0x7f21dc068b10 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x7f21ec009cf0 tx=0x7f21ec009190 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:19:16.173 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:16.172+0000 7f21f37fe640 1 -- 172.21.15.143:0/1345987575 <== mon.0 v2:172.21.15.143:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172649 (secure 0 0 0) 0x7f21e40ec280 con 0x7f21fc0d3e60 2023-11-26T07:19:16.325 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:16.320+0000 7f22014c9640 1 -- 172.21.15.143:0/1345987575 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f21fc067680 con 0x7f21fc0d3e60 2023-11-26T07:19:16.325 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:16.324+0000 7f21f37fe640 1 -- 172.21.15.143:0/1345987575 <== mon.0 v2:172.21.15.143:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v33) v1 ==== 93+0+10 (secure 0 0 0) 0x7f21e40ef8a0 con 0x7f21fc0d3e60 2023-11-26T07:19:16.326 INFO:teuthology.orchestra.run.smithi143.stdout:pg_num: 1 2023-11-26T07:19:16.327 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:16.324+0000 7f22014c9640 1 -- 172.21.15.143:0/1345987575 >> [v2:172.21.15.143:6832/2150557544,v1:172.21.15.143:6833/2150557544] conn(0x7f21dc066650 msgr2=0x7f21dc068b10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:19:16.327 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:16.324+0000 7f22014c9640 1 --2- 172.21.15.143:0/1345987575 >> [v2:172.21.15.143:6832/2150557544,v1:172.21.15.143:6833/2150557544] conn(0x7f21dc066650 0x7f21dc068b10 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x7f21ec009cf0 tx=0x7f21ec009190 comp rx=0 tx=0).stop 2023-11-26T07:19:16.328 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:16.324+0000 7f22014c9640 1 -- 172.21.15.143:0/1345987575 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f21fc0d3e60 msgr2=0x7f21fc15fed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:19:16.328 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:16.324+0000 7f22014c9640 1 --2- 172.21.15.143:0/1345987575 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f21fc0d3e60 0x7f21fc15fed0 secure :-1 s=READY pgs=491 cs=0 l=1 rev1=1 crypto rx=0x7f21e40ad040 tx=0x7f21e401ac90 comp rx=0 tx=0).stop 2023-11-26T07:19:16.328 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:16.324+0000 7f22014c9640 1 -- 172.21.15.143:0/1345987575 shutdown_connections 2023-11-26T07:19:16.328 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:16.324+0000 7f22014c9640 1 --2- 172.21.15.143:0/1345987575 >> [v2:172.21.15.143:6832/2150557544,v1:172.21.15.143:6833/2150557544] conn(0x7f21dc066650 0x7f21dc068b10 unknown :-1 s=CLOSED pgs=252 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:19:16.328 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:16.324+0000 7f22014c9640 1 --2- 172.21.15.143:0/1345987575 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f21fc15d580 0x7f21fc0b8ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:19:16.328 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:16.324+0000 7f22014c9640 1 --2- 172.21.15.143:0/1345987575 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f21fc0d3e60 0x7f21fc15fed0 unknown :-1 s=CLOSED pgs=491 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:19:16.328 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:16.324+0000 7f22014c9640 1 --2- 172.21.15.143:0/1345987575 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f21fc0d1d30 0x7f21fc14c850 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:19:16.329 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:16.324+0000 7f22014c9640 1 -- 172.21.15.143:0/1345987575 >> 172.21.15.143:0/1345987575 conn(0x7f21fc09c550 msgr2=0x7f21fc1562e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:19:16.329 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:16.324+0000 7f22014c9640 1 -- 172.21.15.143:0/1345987575 shutdown_connections 2023-11-26T07:19:16.329 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:16.324+0000 7f22014c9640 1 -- 172.21.15.143:0/1345987575 wait complete. 2023-11-26T07:19:16.346 DEBUG:teuthology.orchestra.run.smithi143:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-11-26T07:19:16.486 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:16.484+0000 7ffb55006640 1 Processor -- start 2023-11-26T07:19:16.487 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:16.484+0000 7ffb55006640 1 -- start start 2023-11-26T07:19:16.488 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:16.484+0000 7ffb55006640 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7ffb5015d6e0 0x7ffb500d0df0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:19:16.488 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:16.484+0000 7ffb55006640 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7ffb500d1bc0 0x7ffb500b0750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:19:16.488 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:16.484+0000 7ffb55006640 1 -- --> v1:172.21.15.143:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ffb500cf0e0 con 0x7ffb500b2160 2023-11-26T07:19:16.488 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:16.484+0000 7ffb55006640 1 -- --> v2:172.21.15.143:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ffb5015dac0 con 0x7ffb5015d6e0 2023-11-26T07:19:16.488 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:16.484+0000 7ffb55006640 1 -- --> v2:172.21.15.188:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ffb500ceb20 con 0x7ffb500d1bc0 2023-11-26T07:19:16.489 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:16.484+0000 7ffb4ed9d640 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7ffb5015d6e0 0x7ffb500d0df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:19:16.489 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:16.484+0000 7ffb4ed9d640 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7ffb5015d6e0 0x7ffb500d0df0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:47954/0 (socket says 172.21.15.143:47954) 2023-11-26T07:19:16.489 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:16.484+0000 7ffb4ed9d640 1 -- 172.21.15.143:0/3130837714 learned_addr learned my addr 172.21.15.143:0/3130837714 (peer_addr_for_me v2:172.21.15.143:0/0) 2023-11-26T07:19:16.489 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:16.484+0000 7ffb4e59c640 1 --1- 172.21.15.143:0/3130837714 >> v1:172.21.15.143:6789/0 conn(0x7ffb500b2160 0x7ffb500d14b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.143:6789/0 says I am v1:172.21.15.143:50110/0 (socket says 172.21.15.143:50110) 2023-11-26T07:19:16.489 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:16.484+0000 7ffb4f59e640 1 --2- 172.21.15.143:0/3130837714 >> v2:172.21.15.188:3300/0 conn(0x7ffb500d1bc0 0x7ffb500b0750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:19:16.490 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:16.484+0000 7ffb4dd9b640 1 -- 172.21.15.143:0/3130837714 <== mon.0 v1:172.21.15.143:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3734764665 0 0) 0x7ffb500cf0e0 con 0x7ffb500b2160 2023-11-26T07:19:16.490 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:16.488+0000 7ffb4dd9b640 1 -- 172.21.15.143:0/3130837714 --> v1:172.21.15.143:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7ffb38003530 con 0x7ffb500b2160 2023-11-26T07:19:16.490 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:16.488+0000 7ffb4f59e640 1 -- 172.21.15.143:0/3130837714 >> v2:172.21.15.143:3300/0 conn(0x7ffb5015d6e0 msgr2=0x7ffb500d0df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-26T07:19:16.490 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:16.488+0000 7ffb4f59e640 1 --2- 172.21.15.143:0/3130837714 >> v2:172.21.15.143:3300/0 conn(0x7ffb5015d6e0 0x7ffb500d0df0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:19:16.490 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:16.488+0000 7ffb4f59e640 1 -- 172.21.15.143:0/3130837714 >> v1:172.21.15.143:6789/0 conn(0x7ffb500b2160 legacy=0x7ffb500d14b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:19:16.490 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:16.488+0000 7ffb4f59e640 1 -- 172.21.15.143:0/3130837714 --> v2:172.21.15.188:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ffb500cee00 con 0x7ffb500d1bc0 2023-11-26T07:19:16.490 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:16.488+0000 7ffb4ed9d640 1 --2- 172.21.15.143:0/3130837714 >> v2:172.21.15.143:3300/0 conn(0x7ffb5015d6e0 0x7ffb500d0df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-26T07:19:16.491 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:16.488+0000 7ffb4f59e640 1 --2- 172.21.15.143:0/3130837714 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7ffb500d1bc0 0x7ffb500b0750 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7ffb500b92d0 tx=0x7ffb4801c910 comp rx=0 tx=0).ready entity=mon.1 client_cookie=79a6fb6a1cedf151 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:19:16.492 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:16.488+0000 7ffb4dd9b640 1 -- 172.21.15.143:0/3130837714 <== mon.1 v2:172.21.15.188:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffb480a58c0 con 0x7ffb500d1bc0 2023-11-26T07:19:16.492 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:16.488+0000 7ffb4dd9b640 1 -- 172.21.15.143:0/3130837714 <== mon.1 v2:172.21.15.188:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ffb480a5a20 con 0x7ffb500d1bc0 2023-11-26T07:19:16.492 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:16.488+0000 7ffb4dd9b640 1 -- 172.21.15.143:0/3130837714 <== mon.1 v2:172.21.15.188:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffb480a5db0 con 0x7ffb500d1bc0 2023-11-26T07:19:16.493 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:16.488+0000 7ffb55006640 1 -- 172.21.15.143:0/3130837714 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7ffb500d1bc0 msgr2=0x7ffb500b0750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:19:16.493 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:16.488+0000 7ffb55006640 1 --2- 172.21.15.143:0/3130837714 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7ffb500d1bc0 0x7ffb500b0750 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7ffb500b92d0 tx=0x7ffb4801c910 comp rx=0 tx=0).stop 2023-11-26T07:19:16.493 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:16.492+0000 7ffb55006640 1 -- 172.21.15.143:0/3130837714 shutdown_connections 2023-11-26T07:19:16.493 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:16.492+0000 7ffb55006640 1 --2- 172.21.15.143:0/3130837714 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7ffb500d1bc0 0x7ffb500b0750 unknown :-1 s=CLOSED pgs=216 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:19:16.493 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:16.492+0000 7ffb55006640 1 --2- 172.21.15.143:0/3130837714 >> v2:172.21.15.143:3300/0 conn(0x7ffb5015d6e0 0x7ffb500d0df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:19:16.494 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:16.492+0000 7ffb55006640 1 -- 172.21.15.143:0/3130837714 >> 172.21.15.143:0/3130837714 conn(0x7ffb5009c550 msgr2=0x7ffb500d3220 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:19:16.494 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:16.492+0000 7ffb55006640 1 -- 172.21.15.143:0/3130837714 shutdown_connections 2023-11-26T07:19:16.494 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:16.492+0000 7ffb55006640 1 -- 172.21.15.143:0/3130837714 wait complete. 2023-11-26T07:19:16.494 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:16.492+0000 7ffb55006640 1 Processor -- start 2023-11-26T07:19:16.495 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:16.492+0000 7ffb55006640 1 -- start start 2023-11-26T07:19:16.497 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:16.492+0000 7ffb55006640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7ffb500b2160 0x7ffb501605c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:19:16.497 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:16.492+0000 7ffb55006640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7ffb500d1bc0 0x7ffb50154170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:19:16.497 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:16.492+0000 7ffb55006640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7ffb5015d6e0 0x7ffb501420f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:19:16.497 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:16.492+0000 7ffb55006640 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffb50160b00 con 0x7ffb500d1bc0 2023-11-26T07:19:16.497 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:16.492+0000 7ffb55006640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffb50154a10 con 0x7ffb5015d6e0 2023-11-26T07:19:16.498 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:16.492+0000 7ffb55006640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ffb50154cf0 con 0x7ffb500b2160 2023-11-26T07:19:16.498 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:16.492+0000 7ffb4e59c640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7ffb500d1bc0 0x7ffb50154170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:19:16.498 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:16.492+0000 7ffb4e59c640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7ffb500d1bc0 0x7ffb50154170 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:47968/0 (socket says 172.21.15.143:47968) 2023-11-26T07:19:16.498 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:16.492+0000 7ffb4e59c640 1 -- 172.21.15.143:0/2027741032 learned_addr learned my addr 172.21.15.143:0/2027741032 (peer_addr_for_me v2:172.21.15.143:0/0) 2023-11-26T07:19:16.498 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:16.492+0000 7ffb4ed9d640 1 --2- 172.21.15.143:0/2027741032 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7ffb500b2160 0x7ffb501605c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:19:16.498 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:16.492+0000 7ffb4f59e640 1 --2- 172.21.15.143:0/2027741032 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7ffb5015d6e0 0x7ffb501420f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:19:16.498 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:16.492+0000 7ffb4e59c640 1 -- 172.21.15.143:0/2027741032 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7ffb500b2160 msgr2=0x7ffb501605c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:19:16.498 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:16.492+0000 7ffb4e59c640 1 --2- 172.21.15.143:0/2027741032 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7ffb500b2160 0x7ffb501605c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:19:16.498 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:16.492+0000 7ffb4e59c640 1 -- 172.21.15.143:0/2027741032 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7ffb5015d6e0 msgr2=0x7ffb501420f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:19:16.498 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:16.492+0000 7ffb4e59c640 1 --2- 172.21.15.143:0/2027741032 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7ffb5015d6e0 0x7ffb501420f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:19:16.498 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:16.492+0000 7ffb4e59c640 1 -- 172.21.15.143:0/2027741032 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ffb400094d0 con 0x7ffb500d1bc0 2023-11-26T07:19:16.499 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:16.496+0000 7ffb4e59c640 1 --2- 172.21.15.143:0/2027741032 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7ffb500d1bc0 0x7ffb50154170 secure :-1 s=READY pgs=493 cs=0 l=1 rev1=1 crypto rx=0x7ffb4400c4d0 tx=0x7ffb4400c990 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:19:16.499 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:16.496+0000 7ffb377fe640 1 -- 172.21.15.143:0/2027741032 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffb44007480 con 0x7ffb500d1bc0 2023-11-26T07:19:16.499 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:16.496+0000 7ffb377fe640 1 -- 172.21.15.143:0/2027741032 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ffb440075e0 con 0x7ffb500d1bc0 2023-11-26T07:19:16.499 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:16.496+0000 7ffb377fe640 1 -- 172.21.15.143:0/2027741032 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffb44004a30 con 0x7ffb500d1bc0 2023-11-26T07:19:16.499 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:16.496+0000 7ffb55006640 1 -- 172.21.15.143:0/2027741032 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ffb48009a70 con 0x7ffb500d1bc0 2023-11-26T07:19:16.499 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:16.496+0000 7ffb55006640 1 -- 172.21.15.143:0/2027741032 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ffb501552b0 con 0x7ffb500d1bc0 2023-11-26T07:19:16.500 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:16.496+0000 7ffb55006640 1 -- 172.21.15.143:0/2027741032 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ffb50067680 con 0x7ffb500d1bc0 2023-11-26T07:19:16.500 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:16.496+0000 7ffb377fe640 1 -- 172.21.15.143:0/2027741032 <== mon.0 v2:172.21.15.143:3300/0 4 ==== mgrmap(e 7) v1 ==== 86005+0+0 (secure 0 0 0) 0x7ffb44004b90 con 0x7ffb500d1bc0 2023-11-26T07:19:16.500 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:16.496+0000 7ffb377fe640 1 --2- 172.21.15.143:0/2027741032 >> [v2:172.21.15.143:6832/2150557544,v1:172.21.15.143:6833/2150557544] conn(0x7ffb3c066620 0x7ffb3c068ae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:19:16.500 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:16.496+0000 7ffb377fe640 1 -- 172.21.15.143:0/2027741032 <== mon.0 v2:172.21.15.143:3300/0 5 ==== osd_map(33..33 src has 1..33) v4 ==== 5709+0+0 (secure 0 0 0) 0x7ffb44083290 con 0x7ffb500d1bc0 2023-11-26T07:19:16.500 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:16.496+0000 7ffb4ed9d640 1 --2- 172.21.15.143:0/2027741032 >> [v2:172.21.15.143:6832/2150557544,v1:172.21.15.143:6833/2150557544] conn(0x7ffb3c066620 0x7ffb3c068ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:19:16.500 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:16.496+0000 7ffb4ed9d640 1 --2- 172.21.15.143:0/2027741032 >> [v2:172.21.15.143:6832/2150557544,v1:172.21.15.143:6833/2150557544] conn(0x7ffb3c066620 0x7ffb3c068ae0 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 crypto rx=0x7ffb50168110 tx=0x7ffb40009240 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:19:16.503 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:16.500+0000 7ffb377fe640 1 -- 172.21.15.143:0/2027741032 <== mon.0 v2:172.21.15.143:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172649 (secure 0 0 0) 0x7ffb44050550 con 0x7ffb500d1bc0 2023-11-26T07:19:16.654 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:16.652+0000 7ffb55006640 1 -- 172.21.15.143:0/2027741032 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7ffb5006c910 con 0x7ffb500d1bc0 2023-11-26T07:19:16.654 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:16.652+0000 7ffb377fe640 1 -- 172.21.15.143:0/2027741032 <== mon.0 v2:172.21.15.143:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v33) v1 ==== 104+0+11 (secure 0 0 0) 0x7ffb44053b70 con 0x7ffb500d1bc0 2023-11-26T07:19:16.655 INFO:teuthology.orchestra.run.smithi143.stdout:pg_num: 32 2023-11-26T07:19:16.657 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:16.652+0000 7ffb55006640 1 -- 172.21.15.143:0/2027741032 >> [v2:172.21.15.143:6832/2150557544,v1:172.21.15.143:6833/2150557544] conn(0x7ffb3c066620 msgr2=0x7ffb3c068ae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:19:16.657 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:16.652+0000 7ffb55006640 1 --2- 172.21.15.143:0/2027741032 >> [v2:172.21.15.143:6832/2150557544,v1:172.21.15.143:6833/2150557544] conn(0x7ffb3c066620 0x7ffb3c068ae0 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 crypto rx=0x7ffb50168110 tx=0x7ffb40009240 comp rx=0 tx=0).stop 2023-11-26T07:19:16.657 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:16.652+0000 7ffb55006640 1 -- 172.21.15.143:0/2027741032 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7ffb500d1bc0 msgr2=0x7ffb50154170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:19:16.657 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:16.652+0000 7ffb55006640 1 --2- 172.21.15.143:0/2027741032 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7ffb500d1bc0 0x7ffb50154170 secure :-1 s=READY pgs=493 cs=0 l=1 rev1=1 crypto rx=0x7ffb4400c4d0 tx=0x7ffb4400c990 comp rx=0 tx=0).stop 2023-11-26T07:19:16.657 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:16.652+0000 7ffb55006640 1 -- 172.21.15.143:0/2027741032 shutdown_connections 2023-11-26T07:19:16.657 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:16.652+0000 7ffb55006640 1 --2- 172.21.15.143:0/2027741032 >> [v2:172.21.15.143:6832/2150557544,v1:172.21.15.143:6833/2150557544] conn(0x7ffb3c066620 0x7ffb3c068ae0 unknown :-1 s=CLOSED pgs=253 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:19:16.657 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:16.652+0000 7ffb55006640 1 --2- 172.21.15.143:0/2027741032 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7ffb5015d6e0 0x7ffb501420f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:19:16.657 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:16.652+0000 7ffb55006640 1 --2- 172.21.15.143:0/2027741032 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7ffb500d1bc0 0x7ffb50154170 unknown :-1 s=CLOSED pgs=493 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:19:16.657 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:16.652+0000 7ffb55006640 1 --2- 172.21.15.143:0/2027741032 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7ffb500b2160 0x7ffb501605c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:19:16.658 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:16.652+0000 7ffb55006640 1 -- 172.21.15.143:0/2027741032 >> 172.21.15.143:0/2027741032 conn(0x7ffb5009c550 msgr2=0x7ffb5006b9d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:19:16.658 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:16.652+0000 7ffb55006640 1 -- 172.21.15.143:0/2027741032 shutdown_connections 2023-11-26T07:19:16.658 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:16.652+0000 7ffb55006640 1 -- 172.21.15.143:0/2027741032 wait complete. 2023-11-26T07:19:16.677 DEBUG:teuthology.orchestra.run.smithi143:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-11-26T07:19:16.820 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:16.816+0000 7f126c6fe640 1 Processor -- start 2023-11-26T07:19:16.821 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:16.816+0000 7f126c6fe640 1 -- start start 2023-11-26T07:19:16.822 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:16.820+0000 7f126c6fe640 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7f12640cd610 0x7f1264071970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:19:16.822 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:16.820+0000 7f126c6fe640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f1264070ba0 0x7f1264070ff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:19:16.823 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:16.820+0000 7f126c6fe640 1 -- --> v1:172.21.15.188:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1264071530 con 0x7f12641614b0 2023-11-26T07:19:16.823 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:16.820+0000 7f126c6fe640 1 -- --> v2:172.21.15.188:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1264071eb0 con 0x7f12640cd610 2023-11-26T07:19:16.823 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:16.820+0000 7f126c6fe640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f126415d970 con 0x7f1264070ba0 2023-11-26T07:19:16.823 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:16.820+0000 7f126ac9b640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f1264070ba0 0x7f1264070ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:19:16.823 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:16.820+0000 7f1269c99640 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7f12640cd610 0x7f1264071970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:19:16.823 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:16.820+0000 7f126ac9b640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f1264070ba0 0x7f1264070ff0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.188:3301/0 says I am v2:172.21.15.143:56510/0 (socket says 172.21.15.143:56510) 2023-11-26T07:19:16.823 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:16.820+0000 7f126ac9b640 1 -- 172.21.15.143:0/3667971184 learned_addr learned my addr 172.21.15.143:0/3667971184 (peer_addr_for_me v2:172.21.15.143:0/0) 2023-11-26T07:19:16.823 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:16.820+0000 7f1269c99640 1 -- 172.21.15.143:0/3667971184 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f1264070ba0 msgr2=0x7f1264070ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-26T07:19:16.823 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:16.820+0000 7f1269c99640 1 --2- 172.21.15.143:0/3667971184 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f1264070ba0 0x7f1264070ff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:19:16.823 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:16.820+0000 7f1269c99640 1 -- 172.21.15.143:0/3667971184 >> v1:172.21.15.188:6789/0 conn(0x7f12641614b0 legacy=0x7f1264161890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:19:16.824 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:16.820+0000 7f1269c99640 1 -- 172.21.15.143:0/3667971184 --> v2:172.21.15.188:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f126415dc30 con 0x7f12640cd610 2023-11-26T07:19:16.824 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:16.820+0000 7f126ac9b640 1 --2- 172.21.15.143:0/3667971184 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f1264070ba0 0x7f1264070ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-26T07:19:16.825 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:16.824+0000 7f1269c99640 1 --2- 172.21.15.143:0/3667971184 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f12640cd610 0x7f1264071970 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7f12540097c0 tx=0x7f125401c450 comp rx=0 tx=0).ready entity=mon.1 client_cookie=e012d2a6673f8df6 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:19:16.826 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:16.824+0000 7f1269498640 1 -- 172.21.15.143:0/3667971184 <== mon.1 v2:172.21.15.188:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f12540ab650 con 0x7f12640cd610 2023-11-26T07:19:16.827 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:16.824+0000 7f1269498640 1 -- 172.21.15.143:0/3667971184 <== mon.1 v2:172.21.15.188:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f12540ab7b0 con 0x7f12640cd610 2023-11-26T07:19:16.827 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:16.824+0000 7f1269498640 1 -- 172.21.15.143:0/3667971184 <== mon.1 v2:172.21.15.188:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f12540abb40 con 0x7f12640cd610 2023-11-26T07:19:16.827 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:16.824+0000 7f126c6fe640 1 -- 172.21.15.143:0/3667971184 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f12640cd610 msgr2=0x7f1264071970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:19:16.827 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:16.824+0000 7f126c6fe640 1 --2- 172.21.15.143:0/3667971184 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f12640cd610 0x7f1264071970 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7f12540097c0 tx=0x7f125401c450 comp rx=0 tx=0).stop 2023-11-26T07:19:16.827 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:16.824+0000 7f126c6fe640 1 -- 172.21.15.143:0/3667971184 shutdown_connections 2023-11-26T07:19:16.827 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:16.824+0000 7f126c6fe640 1 --2- 172.21.15.143:0/3667971184 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f12640cd610 0x7f1264071970 unknown :-1 s=CLOSED pgs=218 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:19:16.827 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:16.824+0000 7f126c6fe640 1 --2- 172.21.15.143:0/3667971184 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f1264070ba0 0x7f1264070ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:19:16.827 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:16.824+0000 7f126c6fe640 1 -- 172.21.15.143:0/3667971184 >> 172.21.15.143:0/3667971184 conn(0x7f12640cf710 msgr2=0x7f12640cfb20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:19:16.827 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:16.824+0000 7f126c6fe640 1 -- 172.21.15.143:0/3667971184 shutdown_connections 2023-11-26T07:19:16.827 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:16.824+0000 7f126c6fe640 1 -- 172.21.15.143:0/3667971184 wait complete. 2023-11-26T07:19:16.828 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:16.824+0000 7f126c6fe640 1 Processor -- start 2023-11-26T07:19:16.828 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:16.824+0000 7f126c6fe640 1 -- start start 2023-11-26T07:19:16.830 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:16.828+0000 7f126c6fe640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f1264070ba0 0x7f1264156830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:19:16.830 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:16.828+0000 7f126c6fe640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f12640cd610 0x7f12640b2d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:19:16.831 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:16.828+0000 7f126c6fe640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f12641614b0 0x7f126415c550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:19:16.831 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:16.828+0000 7f126c6fe640 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1264158610 con 0x7f1264070ba0 2023-11-26T07:19:16.831 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:16.828+0000 7f126c6fe640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1264158000 con 0x7f12641614b0 2023-11-26T07:19:16.831 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:16.828+0000 7f126c6fe640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1264166df0 con 0x7f12640cd610 2023-11-26T07:19:16.831 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:16.828+0000 7f126a49a640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f1264070ba0 0x7f1264156830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:19:16.831 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:16.828+0000 7f126a49a640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f1264070ba0 0x7f1264156830 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:47976/0 (socket says 172.21.15.143:47976) 2023-11-26T07:19:16.831 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:16.828+0000 7f126a49a640 1 -- 172.21.15.143:0/46388952 learned_addr learned my addr 172.21.15.143:0/46388952 (peer_addr_for_me v2:172.21.15.143:0/0) 2023-11-26T07:19:16.831 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:16.828+0000 7f126ac9b640 1 --2- 172.21.15.143:0/46388952 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f12641614b0 0x7f126415c550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:19:16.831 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:16.828+0000 7f1269c99640 1 --2- 172.21.15.143:0/46388952 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f12640cd610 0x7f12640b2d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:19:16.832 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:16.828+0000 7f126a49a640 1 -- 172.21.15.143:0/46388952 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f12640cd610 msgr2=0x7f12640b2d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:19:16.832 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:16.828+0000 7f126a49a640 1 --2- 172.21.15.143:0/46388952 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f12640cd610 0x7f12640b2d10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:19:16.832 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:16.828+0000 7f126a49a640 1 -- 172.21.15.143:0/46388952 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f12641614b0 msgr2=0x7f126415c550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:19:16.832 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:16.828+0000 7f126a49a640 1 --2- 172.21.15.143:0/46388952 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f12641614b0 0x7f126415c550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:19:16.832 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:16.828+0000 7f126a49a640 1 -- 172.21.15.143:0/46388952 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f125c009b30 con 0x7f1264070ba0 2023-11-26T07:19:16.832 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:16.828+0000 7f126a49a640 1 --2- 172.21.15.143:0/46388952 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f1264070ba0 0x7f1264156830 secure :-1 s=READY pgs=494 cs=0 l=1 rev1=1 crypto rx=0x7f126000c450 tx=0x7f126000c910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:19:16.832 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:16.828+0000 7f125affd640 1 -- 172.21.15.143:0/46388952 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f12600052d0 con 0x7f1264070ba0 2023-11-26T07:19:16.832 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:16.828+0000 7f125affd640 1 -- 172.21.15.143:0/46388952 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1260005450 con 0x7f1264070ba0 2023-11-26T07:19:16.832 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:16.828+0000 7f126c6fe640 1 -- 172.21.15.143:0/46388952 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1254009420 con 0x7f1264070ba0 2023-11-26T07:19:16.832 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:16.828+0000 7f125affd640 1 -- 172.21.15.143:0/46388952 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1260012870 con 0x7f1264070ba0 2023-11-26T07:19:16.833 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:16.828+0000 7f126c6fe640 1 -- 172.21.15.143:0/46388952 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f12641582a0 con 0x7f1264070ba0 2023-11-26T07:19:16.833 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:16.828+0000 7f126c6fe640 1 -- 172.21.15.143:0/46388952 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1264067680 con 0x7f1264070ba0 2023-11-26T07:19:16.834 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:16.828+0000 7f125affd640 1 -- 172.21.15.143:0/46388952 <== mon.0 v2:172.21.15.143:3300/0 4 ==== mgrmap(e 7) v1 ==== 86005+0+0 (secure 0 0 0) 0x7f12600129d0 con 0x7f1264070ba0 2023-11-26T07:19:16.834 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:16.832+0000 7f125affd640 1 --2- 172.21.15.143:0/46388952 >> [v2:172.21.15.143:6832/2150557544,v1:172.21.15.143:6833/2150557544] conn(0x7f1248066620 0x7f1248068ae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:19:16.834 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:16.832+0000 7f125affd640 1 -- 172.21.15.143:0/46388952 <== mon.0 v2:172.21.15.143:3300/0 5 ==== osd_map(33..33 src has 1..33) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f1260083a40 con 0x7f1264070ba0 2023-11-26T07:19:16.834 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:16.832+0000 7f1269c99640 1 --2- 172.21.15.143:0/46388952 >> [v2:172.21.15.143:6832/2150557544,v1:172.21.15.143:6833/2150557544] conn(0x7f1248066620 0x7f1248068ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:19:16.834 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:16.832+0000 7f1269c99640 1 --2- 172.21.15.143:0/46388952 >> [v2:172.21.15.143:6832/2150557544,v1:172.21.15.143:6833/2150557544] conn(0x7f1248066620 0x7f1248068ae0 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 crypto rx=0x7f12540af000 tx=0x7f12540a7000 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:19:16.837 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:16.836+0000 7f125affd640 1 -- 172.21.15.143:0/46388952 <== mon.0 v2:172.21.15.143:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172649 (secure 0 0 0) 0x7f1260085e60 con 0x7f1264070ba0 2023-11-26T07:19:16.987 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:16.984+0000 7f126c6fe640 1 -- 172.21.15.143:0/46388952 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f1264071530 con 0x7f1264070ba0 2023-11-26T07:19:16.988 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:16.984+0000 7f125affd640 1 -- 172.21.15.143:0/46388952 <== mon.0 v2:172.21.15.143:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v33) v1 ==== 100+0+11 (secure 0 0 0) 0x7f1264071530 con 0x7f1264070ba0 2023-11-26T07:19:16.988 INFO:teuthology.orchestra.run.smithi143.stdout:pg_num: 64 2023-11-26T07:19:16.990 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:16.988+0000 7f126c6fe640 1 -- 172.21.15.143:0/46388952 >> [v2:172.21.15.143:6832/2150557544,v1:172.21.15.143:6833/2150557544] conn(0x7f1248066620 msgr2=0x7f1248068ae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:19:16.990 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:16.988+0000 7f126c6fe640 1 --2- 172.21.15.143:0/46388952 >> [v2:172.21.15.143:6832/2150557544,v1:172.21.15.143:6833/2150557544] conn(0x7f1248066620 0x7f1248068ae0 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 crypto rx=0x7f12540af000 tx=0x7f12540a7000 comp rx=0 tx=0).stop 2023-11-26T07:19:16.990 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:16.988+0000 7f126c6fe640 1 -- 172.21.15.143:0/46388952 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f1264070ba0 msgr2=0x7f1264156830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:19:16.990 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:16.988+0000 7f126c6fe640 1 --2- 172.21.15.143:0/46388952 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f1264070ba0 0x7f1264156830 secure :-1 s=READY pgs=494 cs=0 l=1 rev1=1 crypto rx=0x7f126000c450 tx=0x7f126000c910 comp rx=0 tx=0).stop 2023-11-26T07:19:16.990 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:16.988+0000 7f126c6fe640 1 -- 172.21.15.143:0/46388952 shutdown_connections 2023-11-26T07:19:16.991 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:16.988+0000 7f126c6fe640 1 --2- 172.21.15.143:0/46388952 >> [v2:172.21.15.143:6832/2150557544,v1:172.21.15.143:6833/2150557544] conn(0x7f1248066620 0x7f1248068ae0 unknown :-1 s=CLOSED pgs=254 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:19:16.991 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:16.988+0000 7f126c6fe640 1 --2- 172.21.15.143:0/46388952 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f12641614b0 0x7f126415c550 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:19:16.991 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:16.988+0000 7f126c6fe640 1 --2- 172.21.15.143:0/46388952 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f12640cd610 0x7f12640b2d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:19:16.991 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:16.988+0000 7f126c6fe640 1 --2- 172.21.15.143:0/46388952 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f1264070ba0 0x7f1264156830 unknown :-1 s=CLOSED pgs=494 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:19:16.991 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:16.988+0000 7f126c6fe640 1 -- 172.21.15.143:0/46388952 >> 172.21.15.143:0/46388952 conn(0x7f12640cf710 msgr2=0x7f126415efd0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:19:16.991 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:16.988+0000 7f126c6fe640 1 -- 172.21.15.143:0/46388952 shutdown_connections 2023-11-26T07:19:16.992 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:16.988+0000 7f126c6fe640 1 -- 172.21.15.143:0/46388952 wait complete. 2023-11-26T07:19:17.008 DEBUG:teuthology.orchestra.run.smithi143:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-11-26T07:19:17.146 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:17.144+0000 7f79daa9c640 1 Processor -- start 2023-11-26T07:19:17.147 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:17.144+0000 7f79daa9c640 1 -- start start 2023-11-26T07:19:17.148 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:17.144+0000 7f79daa9c640 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7f79d4143ac0 0x7f79d4070370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:19:17.148 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:17.144+0000 7f79daa9c640 1 -- --> v1:172.21.15.143:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f79d40cc730 con 0x7f79d40cf990 2023-11-26T07:19:17.148 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:17.144+0000 7f79daa9c640 1 -- --> v1:172.21.15.188:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f79d4070ee0 con 0x7f79d40708b0 2023-11-26T07:19:17.149 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:17.144+0000 7f79daa9c640 1 -- --> v2:172.21.15.143:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f79d4143ea0 con 0x7f79d4143ac0 2023-11-26T07:19:17.149 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:17.144+0000 7f79d8838640 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7f79d4143ac0 0x7f79d4070370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:19:17.149 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:17.144+0000 7f79d8838640 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7f79d4143ac0 0x7f79d4070370 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:47982/0 (socket says 172.21.15.143:47982) 2023-11-26T07:19:17.149 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:17.144+0000 7f79d8838640 1 -- 172.21.15.143:0/860592642 learned_addr learned my addr 172.21.15.143:0/860592642 (peer_addr_for_me v2:172.21.15.143:0/0) 2023-11-26T07:19:17.149 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:17.144+0000 7f79d8838640 1 -- 172.21.15.143:0/860592642 >> v1:172.21.15.188:6789/0 conn(0x7f79d40708b0 legacy=0x7f79d40cf280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:19:17.149 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:17.144+0000 7f79d8838640 1 -- 172.21.15.143:0/860592642 >> v1:172.21.15.143:6789/0 conn(0x7f79d40cf990 legacy=0x7f79d40cd1b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:19:17.149 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:17.144+0000 7f79d8838640 1 -- 172.21.15.143:0/860592642 --> v2:172.21.15.143:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f79d40711c0 con 0x7f79d4143ac0 2023-11-26T07:19:17.151 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:17.148+0000 7f79d8838640 1 --2- 172.21.15.143:0/860592642 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f79d4143ac0 0x7f79d4070370 secure :-1 s=READY pgs=496 cs=0 l=1 rev1=1 crypto rx=0x7f79d000bb10 tx=0x7f79d001ec10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a7418b46e35580 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:19:17.151 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:17.148+0000 7f79cb7fe640 1 -- 172.21.15.143:0/860592642 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f79d00a7bf0 con 0x7f79d4143ac0 2023-11-26T07:19:17.151 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:17.148+0000 7f79cb7fe640 1 -- 172.21.15.143:0/860592642 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f79d00a7d50 con 0x7f79d4143ac0 2023-11-26T07:19:17.152 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:17.148+0000 7f79cb7fe640 1 -- 172.21.15.143:0/860592642 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f79d00af660 con 0x7f79d4143ac0 2023-11-26T07:19:17.153 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:17.148+0000 7f79daa9c640 1 -- 172.21.15.143:0/860592642 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f79d4143ac0 msgr2=0x7f79d4070370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:19:17.153 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:17.148+0000 7f79daa9c640 1 --2- 172.21.15.143:0/860592642 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f79d4143ac0 0x7f79d4070370 secure :-1 s=READY pgs=496 cs=0 l=1 rev1=1 crypto rx=0x7f79d000bb10 tx=0x7f79d001ec10 comp rx=0 tx=0).stop 2023-11-26T07:19:17.153 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:17.148+0000 7f79daa9c640 1 -- 172.21.15.143:0/860592642 shutdown_connections 2023-11-26T07:19:17.153 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:17.148+0000 7f79daa9c640 1 --2- 172.21.15.143:0/860592642 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f79d4143ac0 0x7f79d4070370 unknown :-1 s=CLOSED pgs=496 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:19:17.153 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:17.148+0000 7f79daa9c640 1 -- 172.21.15.143:0/860592642 >> 172.21.15.143:0/860592642 conn(0x7f79d406a890 msgr2=0x7f79d406aca0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:19:17.154 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:17.148+0000 7f79daa9c640 1 -- 172.21.15.143:0/860592642 shutdown_connections 2023-11-26T07:19:17.154 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:17.148+0000 7f79daa9c640 1 -- 172.21.15.143:0/860592642 wait complete. 2023-11-26T07:19:17.156 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:17.152+0000 7f79daa9c640 1 Processor -- start 2023-11-26T07:19:17.157 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:17.152+0000 7f79daa9c640 1 -- start start 2023-11-26T07:19:17.157 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:17.152+0000 7f79daa9c640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f79d40708b0 0x7f79d415c480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:19:17.157 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:17.152+0000 7f79daa9c640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f79d40cf990 0x7f79d4138520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:19:17.157 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:17.152+0000 7f79daa9c640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f79d4143ac0 0x7f79d4138a60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:19:17.157 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:17.152+0000 7f79daa9c640 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f79d4138fa0 con 0x7f79d40708b0 2023-11-26T07:19:17.157 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:17.152+0000 7f79daa9c640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f79d41572b0 con 0x7f79d40cf990 2023-11-26T07:19:17.157 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:17.152+0000 7f79daa9c640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f79d40b1030 con 0x7f79d4143ac0 2023-11-26T07:19:17.158 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:17.152+0000 7f79d8838640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f79d40708b0 0x7f79d415c480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:19:17.158 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:17.152+0000 7f79d8838640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f79d40708b0 0x7f79d415c480 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:47988/0 (socket says 172.21.15.143:47988) 2023-11-26T07:19:17.158 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:17.152+0000 7f79d8838640 1 -- 172.21.15.143:0/532248745 learned_addr learned my addr 172.21.15.143:0/532248745 (peer_addr_for_me v2:172.21.15.143:0/0) 2023-11-26T07:19:17.158 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:17.152+0000 7f79cbfff640 1 --2- 172.21.15.143:0/532248745 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f79d40cf990 0x7f79d4138520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:19:17.158 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:17.152+0000 7f79d9039640 1 --2- 172.21.15.143:0/532248745 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f79d4143ac0 0x7f79d4138a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:19:17.158 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:17.152+0000 7f79d8838640 1 -- 172.21.15.143:0/532248745 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f79d4143ac0 msgr2=0x7f79d4138a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:19:17.159 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:17.152+0000 7f79d8838640 1 --2- 172.21.15.143:0/532248745 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f79d4143ac0 0x7f79d4138a60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:19:17.159 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:17.152+0000 7f79d8838640 1 -- 172.21.15.143:0/532248745 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f79d40cf990 msgr2=0x7f79d4138520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:19:17.159 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:17.152+0000 7f79d8838640 1 --2- 172.21.15.143:0/532248745 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f79d40cf990 0x7f79d4138520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:19:17.159 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:17.152+0000 7f79d8838640 1 -- 172.21.15.143:0/532248745 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f79d000b770 con 0x7f79d40708b0 2023-11-26T07:19:17.159 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:17.152+0000 7f79d8838640 1 --2- 172.21.15.143:0/532248745 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f79d40708b0 0x7f79d415c480 secure :-1 s=READY pgs=497 cs=0 l=1 rev1=1 crypto rx=0x7f79d001b820 tx=0x7f79d001b9b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:19:17.159 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:17.152+0000 7f79c97fa640 1 -- 172.21.15.143:0/532248745 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f79d00a7bf0 con 0x7f79d40708b0 2023-11-26T07:19:17.159 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:17.152+0000 7f79c97fa640 1 -- 172.21.15.143:0/532248745 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f79d00b8400 con 0x7f79d40708b0 2023-11-26T07:19:17.160 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:17.152+0000 7f79daa9c640 1 -- 172.21.15.143:0/532248745 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f79d40b1310 con 0x7f79d40708b0 2023-11-26T07:19:17.160 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:17.152+0000 7f79c97fa640 1 -- 172.21.15.143:0/532248745 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f79d00b8790 con 0x7f79d40708b0 2023-11-26T07:19:17.160 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:17.152+0000 7f79daa9c640 1 -- 172.21.15.143:0/532248745 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f79d41577e0 con 0x7f79d40708b0 2023-11-26T07:19:17.161 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:17.156+0000 7f79daa9c640 1 -- 172.21.15.143:0/532248745 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f79d4067680 con 0x7f79d40708b0 2023-11-26T07:19:17.161 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:17.156+0000 7f79c97fa640 1 -- 172.21.15.143:0/532248745 <== mon.0 v2:172.21.15.143:3300/0 4 ==== mgrmap(e 7) v1 ==== 86005+0+0 (secure 0 0 0) 0x7f79d00b6040 con 0x7f79d40708b0 2023-11-26T07:19:17.161 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:17.156+0000 7f79c97fa640 1 --2- 172.21.15.143:0/532248745 >> [v2:172.21.15.143:6832/2150557544,v1:172.21.15.143:6833/2150557544] conn(0x7f79c0066620 0x7f79c0068ae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:19:17.161 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:17.156+0000 7f79c97fa640 1 -- 172.21.15.143:0/532248745 <== mon.0 v2:172.21.15.143:3300/0 5 ==== osd_map(33..33 src has 1..33) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f79d0120f10 con 0x7f79d40708b0 2023-11-26T07:19:17.162 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:17.156+0000 7f79cbfff640 1 --2- 172.21.15.143:0/532248745 >> [v2:172.21.15.143:6832/2150557544,v1:172.21.15.143:6833/2150557544] conn(0x7f79c0066620 0x7f79c0068ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:19:17.162 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:17.156+0000 7f79cbfff640 1 --2- 172.21.15.143:0/532248745 >> [v2:172.21.15.143:6832/2150557544,v1:172.21.15.143:6833/2150557544] conn(0x7f79c0066620 0x7f79c0068ae0 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 crypto rx=0x7f79d4142f60 tx=0x7f79c4009040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:19:17.163 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:17.160+0000 7f79c97fa640 1 -- 172.21.15.143:0/532248745 <== mon.0 v2:172.21.15.143:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172649 (secure 0 0 0) 0x7f79d00ee1c0 con 0x7f79d40708b0 2023-11-26T07:19:17.342 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:17.340+0000 7f79daa9c640 1 -- 172.21.15.143:0/532248745 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f79d40cd330 con 0x7f79d40708b0 2023-11-26T07:19:17.342 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:17.340+0000 7f79c97fa640 1 -- 172.21.15.143:0/532248745 <== mon.0 v2:172.21.15.143:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 17 v17) v1 ==== 94+0+4925 (secure 0 0 0) 0x7f79d00b4020 con 0x7f79d40708b0 2023-11-26T07:19:17.342 INFO:teuthology.orchestra.run.smithi143.stdout: 2023-11-26T07:19:17.343 INFO:teuthology.orchestra.run.smithi143.stdout:{"epoch":17,"default_fscid":2,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4847,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.188:6834/1547410397","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6832","nonce":1547410397},{"type":"v1","addr":"172.21.15.188:6834","nonce":1547410397}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":12},{"gid":4850,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.188:6835/3247640329","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6833","nonce":3247640329},{"type":"v1","addr":"172.21.15.188:6835","nonce":3247640329}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":12},{"gid":5217,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.143:6835/2123707650","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6834","nonce":2123707650},{"type":"v1","addr":"172.21.15.143:6835","nonce":2123707650}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":11}],"filesystems":[{"mdsmap":{"epoch":16,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-11-26T07:18:52.014584+0000","modified":"2023-11-26T07:18:55.043646+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":5220},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5220":{"gid":5220,"name":"c","rank":0,"incarnation":13,"state":"up:active","state_seq":3,"addr":"172.21.15.143:6837/2658667363","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6836","nonce":2658667363},{"type":"v1","addr":"172.21.15.143:6837","nonce":2658667363}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[5],"metadata_pool":4,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":2}]} 2023-11-26T07:19:17.343 INFO:teuthology.orchestra.run.smithi143.stderr:dumped fsmap epoch 17 2023-11-26T07:19:17.345 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:17.340+0000 7f79daa9c640 1 -- 172.21.15.143:0/532248745 >> [v2:172.21.15.143:6832/2150557544,v1:172.21.15.143:6833/2150557544] conn(0x7f79c0066620 msgr2=0x7f79c0068ae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:19:17.345 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:17.340+0000 7f79daa9c640 1 --2- 172.21.15.143:0/532248745 >> [v2:172.21.15.143:6832/2150557544,v1:172.21.15.143:6833/2150557544] conn(0x7f79c0066620 0x7f79c0068ae0 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 crypto rx=0x7f79d4142f60 tx=0x7f79c4009040 comp rx=0 tx=0).stop 2023-11-26T07:19:17.345 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:17.340+0000 7f79daa9c640 1 -- 172.21.15.143:0/532248745 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f79d40708b0 msgr2=0x7f79d415c480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:19:17.345 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:17.340+0000 7f79daa9c640 1 --2- 172.21.15.143:0/532248745 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f79d40708b0 0x7f79d415c480 secure :-1 s=READY pgs=497 cs=0 l=1 rev1=1 crypto rx=0x7f79d001b820 tx=0x7f79d001b9b0 comp rx=0 tx=0).stop 2023-11-26T07:19:17.345 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:17.340+0000 7f79daa9c640 1 -- 172.21.15.143:0/532248745 shutdown_connections 2023-11-26T07:19:17.345 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:17.340+0000 7f79daa9c640 1 --2- 172.21.15.143:0/532248745 >> [v2:172.21.15.143:6832/2150557544,v1:172.21.15.143:6833/2150557544] conn(0x7f79c0066620 0x7f79c0068ae0 unknown :-1 s=CLOSED pgs=255 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:19:17.345 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:17.340+0000 7f79daa9c640 1 --2- 172.21.15.143:0/532248745 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f79d4143ac0 0x7f79d4138a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:19:17.345 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:17.340+0000 7f79daa9c640 1 --2- 172.21.15.143:0/532248745 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f79d40cf990 0x7f79d4138520 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:19:17.345 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:17.340+0000 7f79daa9c640 1 --2- 172.21.15.143:0/532248745 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f79d40708b0 0x7f79d415c480 unknown :-1 s=CLOSED pgs=497 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:19:17.345 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:17.340+0000 7f79daa9c640 1 -- 172.21.15.143:0/532248745 >> 172.21.15.143:0/532248745 conn(0x7f79d406a890 msgr2=0x7f79d40b1cb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:19:17.345 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:17.340+0000 7f79daa9c640 1 -- 172.21.15.143:0/532248745 shutdown_connections 2023-11-26T07:19:17.346 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:17.344+0000 7f79daa9c640 1 -- 172.21.15.143:0/532248745 wait complete. 2023-11-26T07:19:17.363 DEBUG:teuthology.orchestra.run.smithi143:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-11-26T07:19:17.501 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:17.496+0000 7f081e22f640 1 Processor -- start 2023-11-26T07:19:17.501 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:17.500+0000 7f081e22f640 1 -- start start 2023-11-26T07:19:17.505 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:17.500+0000 7f081e22f640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f08180cfa70 0x7f08180d0e40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:19:17.505 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:17.500+0000 7f081e22f640 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7f08180ced30 0x7f08180b0f60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:19:17.505 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:17.500+0000 7f081e22f640 1 -- --> v1:172.21.15.143:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f08180d3d20 con 0x7f08180cfe50 2023-11-26T07:19:17.506 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:17.500+0000 7f081e22f640 1 -- --> v2:172.21.15.188:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f08180d4000 con 0x7f08180ced30 2023-11-26T07:19:17.506 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:17.500+0000 7f081e22f640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f08180afd70 con 0x7f08180cfa70 2023-11-26T07:19:17.506 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:17.500+0000 7f081ca2c640 1 --1- >> v1:172.21.15.143:6789/0 conn(0x7f08180cfe50 0x7f08180b0850 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.143:6789/0 says I am v1:172.21.15.143:50118/0 (socket says 172.21.15.143:50118) 2023-11-26T07:19:17.506 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:17.500+0000 7f081ca2c640 1 -- 172.21.15.143:0/4290536720 learned_addr learned my addr 172.21.15.143:0/4290536720 (peer_addr_for_me v1:172.21.15.143:0/0) 2023-11-26T07:19:17.506 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:17.500+0000 7f081d22d640 1 --2- 172.21.15.143:0/4290536720 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f08180cfa70 0x7f08180d0e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:19:17.506 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:17.500+0000 7f081da2e640 1 --2- 172.21.15.143:0/4290536720 >> v2:172.21.15.188:3300/0 conn(0x7f08180ced30 0x7f08180b0f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:19:17.507 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:17.500+0000 7f080ffff640 1 -- 172.21.15.143:0/4290536720 <== mon.0 v1:172.21.15.143:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2930510306 0 0) 0x7f08180d3d20 con 0x7f08180cfe50 2023-11-26T07:19:17.507 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:17.500+0000 7f080ffff640 1 -- 172.21.15.143:0/4290536720 --> v1:172.21.15.143:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f0804003530 con 0x7f08180cfe50 2023-11-26T07:19:17.507 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:17.500+0000 7f080ffff640 1 -- 172.21.15.143:0/4290536720 <== mon.0 v1:172.21.15.143:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1549078981 0 0) 0x7f0804003530 con 0x7f08180cfe50 2023-11-26T07:19:17.507 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:17.500+0000 7f080ffff640 1 -- 172.21.15.143:0/4290536720 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f08180cfa70 msgr2=0x7f08180d0e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-26T07:19:17.507 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:17.500+0000 7f080ffff640 1 --2- 172.21.15.143:0/4290536720 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f08180cfa70 0x7f08180d0e40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:19:17.507 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:17.500+0000 7f080ffff640 1 -- 172.21.15.143:0/4290536720 >> v2:172.21.15.188:3300/0 conn(0x7f08180ced30 msgr2=0x7f08180b0f60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-26T07:19:17.507 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:17.500+0000 7f080ffff640 1 --2- 172.21.15.143:0/4290536720 >> v2:172.21.15.188:3300/0 conn(0x7f08180ced30 0x7f08180b0f60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:19:17.508 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:17.500+0000 7f080ffff640 1 -- 172.21.15.143:0/4290536720 --> v1:172.21.15.143:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f08180b0050 con 0x7f08180cfe50 2023-11-26T07:19:17.508 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:17.500+0000 7f080ffff640 1 -- 172.21.15.143:0/4290536720 <== mon.0 v1:172.21.15.143:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2137882523 0 0) 0x7f0808001680 con 0x7f08180cfe50 2023-11-26T07:19:17.508 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:17.500+0000 7f081da2e640 1 --2- 172.21.15.143:0/4290536720 >> v2:172.21.15.188:3300/0 conn(0x7f08180ced30 0x7f08180b0f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-26T07:19:17.508 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:17.500+0000 7f080ffff640 1 -- 172.21.15.143:0/4290536720 >> v1:172.21.15.143:6789/0 conn(0x7f08180cfe50 legacy=0x7f08180b0850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:19:17.508 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:17.500+0000 7f080ffff640 1 --2- 172.21.15.143:0/4290536720 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f0804003980 0x7f0804003da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:19:17.508 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:17.500+0000 7f080ffff640 1 --2- 172.21.15.143:0/4290536720 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f08180cfa70 0x7f08040047d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:19:17.508 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:17.500+0000 7f080ffff640 1 --2- 172.21.15.143:0/4290536720 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f0804004d10 0x7f0804005190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:19:17.509 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:17.500+0000 7f080ffff640 1 -- 172.21.15.143:0/4290536720 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f08180d4000 con 0x7f0804003980 2023-11-26T07:19:17.509 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:17.500+0000 7f080ffff640 1 -- 172.21.15.143:0/4290536720 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f08180afd70 con 0x7f0804004d10 2023-11-26T07:19:17.509 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:17.500+0000 7f080ffff640 1 -- 172.21.15.143:0/4290536720 --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f080400b990 con 0x7f08180cfa70 2023-11-26T07:19:17.509 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:17.500+0000 7f081d22d640 1 --2- 172.21.15.143:0/4290536720 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f0804003980 0x7f0804003da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:19:17.509 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:17.500+0000 7f081da2e640 1 --2- 172.21.15.143:0/4290536720 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f0804004d10 0x7f0804005190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:19:17.509 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:17.500+0000 7f081ca2c640 1 --2- 172.21.15.143:0/4290536720 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f08180cfa70 0x7f08040047d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:19:17.510 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:17.500+0000 7f081d22d640 1 -- 172.21.15.143:0/4290536720 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f08180cfa70 msgr2=0x7f08040047d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-26T07:19:17.510 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:17.500+0000 7f081d22d640 1 --2- 172.21.15.143:0/4290536720 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f08180cfa70 0x7f08040047d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:19:17.510 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:17.500+0000 7f081d22d640 1 -- 172.21.15.143:0/4290536720 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f0804004d10 msgr2=0x7f0804005190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-26T07:19:17.510 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:17.500+0000 7f081d22d640 1 --2- 172.21.15.143:0/4290536720 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f0804004d10 0x7f0804005190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:19:17.510 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:17.500+0000 7f081d22d640 1 -- 172.21.15.143:0/4290536720 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0804003530 con 0x7f0804003980 2023-11-26T07:19:17.510 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:17.500+0000 7f081ca2c640 1 --2- 172.21.15.143:0/4290536720 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f08180cfa70 0x7f08040047d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-26T07:19:17.511 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:17.500+0000 7f081da2e640 1 --2- 172.21.15.143:0/4290536720 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f0804004d10 0x7f0804005190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-26T07:19:17.516 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:17.504+0000 7f081d22d640 1 --2- 172.21.15.143:0/4290536720 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f0804003980 0x7f0804003da0 secure :-1 s=READY pgs=499 cs=0 l=1 rev1=1 crypto rx=0x7f0814009ae0 tx=0x7f08180b1cb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3c6998d77fbd360f server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:19:17.516 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:17.504+0000 7f080ffff640 1 -- 172.21.15.143:0/4290536720 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f08140a5610 con 0x7f0804003980 2023-11-26T07:19:17.516 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:17.504+0000 7f080ffff640 1 -- 172.21.15.143:0/4290536720 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f08140a5770 con 0x7f0804003980 2023-11-26T07:19:17.516 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:17.504+0000 7f080ffff640 1 -- 172.21.15.143:0/4290536720 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f08140a5b20 con 0x7f0804003980 2023-11-26T07:19:17.516 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:17.504+0000 7f081e22f640 1 -- 172.21.15.143:0/4290536720 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f0804003980 msgr2=0x7f0804003da0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:19:17.516 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:17.504+0000 7f081e22f640 1 --2- 172.21.15.143:0/4290536720 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f0804003980 0x7f0804003da0 secure :-1 s=READY pgs=499 cs=0 l=1 rev1=1 crypto rx=0x7f0814009ae0 tx=0x7f08180b1cb0 comp rx=0 tx=0).stop 2023-11-26T07:19:17.516 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:17.504+0000 7f081da2e640 1 -- 172.21.15.143:0/4290536720 reap_dead start 2023-11-26T07:19:17.517 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:17.504+0000 7f081e22f640 1 -- 172.21.15.143:0/4290536720 shutdown_connections 2023-11-26T07:19:17.517 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:17.504+0000 7f081e22f640 1 -- 172.21.15.143:0/4290536720 >> 172.21.15.143:0/4290536720 conn(0x7f08180d3060 msgr2=0x7f08180d3470 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:19:17.517 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:17.504+0000 7f081e22f640 1 -- 172.21.15.143:0/4290536720 shutdown_connections 2023-11-26T07:19:17.517 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:17.504+0000 7f081e22f640 1 -- 172.21.15.143:0/4290536720 wait complete. 2023-11-26T07:19:17.517 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:17.508+0000 7f081e22f640 1 Processor -- start 2023-11-26T07:19:17.517 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:17.508+0000 7f081e22f640 1 -- start start 2023-11-26T07:19:17.517 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:17.508+0000 7f081e22f640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f0804003980 0x7f081815c790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:19:17.517 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:17.508+0000 7f081e22f640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f081815e940 0x7f0818161550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:19:17.517 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:17.508+0000 7f081e22f640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f081815ba30 0x7f08180c8130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:19:17.517 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:17.508+0000 7f081e22f640 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0818161a90 con 0x7f0804003980 2023-11-26T07:19:17.518 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:17.508+0000 7f081e22f640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f081815db80 con 0x7f081815ba30 2023-11-26T07:19:17.518 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:17.508+0000 7f081e22f640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f081815de60 con 0x7f081815e940 2023-11-26T07:19:17.518 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:17.508+0000 7f081d22d640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f0804003980 0x7f081815c790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:19:17.518 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:17.508+0000 7f081d22d640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f0804003980 0x7f081815c790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:48006/0 (socket says 172.21.15.143:48006) 2023-11-26T07:19:17.518 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:17.508+0000 7f081d22d640 1 -- 172.21.15.143:0/3469336951 learned_addr learned my addr 172.21.15.143:0/3469336951 (peer_addr_for_me v2:172.21.15.143:0/0) 2023-11-26T07:19:17.518 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:17.508+0000 7f081ca2c640 1 --2- 172.21.15.143:0/3469336951 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f081815e940 0x7f0818161550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:19:17.518 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:17.508+0000 7f081da2e640 1 --2- 172.21.15.143:0/3469336951 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f081815ba30 0x7f08180c8130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:19:17.518 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:17.508+0000 7f081d22d640 1 -- 172.21.15.143:0/3469336951 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f081815e940 msgr2=0x7f0818161550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:19:17.518 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:17.508+0000 7f081d22d640 1 --2- 172.21.15.143:0/3469336951 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f081815e940 0x7f0818161550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:19:17.518 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:17.508+0000 7f081d22d640 1 -- 172.21.15.143:0/3469336951 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f081815ba30 msgr2=0x7f08180c8130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:19:17.519 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:17.508+0000 7f081d22d640 1 --2- 172.21.15.143:0/3469336951 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f081815ba30 0x7f08180c8130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:19:17.519 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:17.508+0000 7f081d22d640 1 -- 172.21.15.143:0/3469336951 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f081815e140 con 0x7f0804003980 2023-11-26T07:19:17.519 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:17.508+0000 7f081d22d640 1 --2- 172.21.15.143:0/3469336951 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f0804003980 0x7f081815c790 secure :-1 s=READY pgs=500 cs=0 l=1 rev1=1 crypto rx=0x7f0814009ae0 tx=0x7f08140197c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:19:17.519 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:17.508+0000 7f080dffb640 1 -- 172.21.15.143:0/3469336951 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f08140a5610 con 0x7f0804003980 2023-11-26T07:19:17.519 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:17.508+0000 7f080dffb640 1 -- 172.21.15.143:0/3469336951 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0814004020 con 0x7f0804003980 2023-11-26T07:19:17.519 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:17.508+0000 7f081e22f640 1 -- 172.21.15.143:0/3469336951 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f081815e420 con 0x7f0804003980 2023-11-26T07:19:17.519 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:17.508+0000 7f080dffb640 1 -- 172.21.15.143:0/3469336951 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f08140043d0 con 0x7f0804003980 2023-11-26T07:19:17.519 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:17.508+0000 7f081e22f640 1 -- 172.21.15.143:0/3469336951 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f081815c080 con 0x7f0804003980 2023-11-26T07:19:17.519 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:17.512+0000 7f081e22f640 1 -- 172.21.15.143:0/3469336951 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0818067500 con 0x7f0804003980 2023-11-26T07:19:17.519 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:17.512+0000 7f080dffb640 1 -- 172.21.15.143:0/3469336951 <== mon.0 v2:172.21.15.143:3300/0 4 ==== mgrmap(e 7) v1 ==== 86005+0+0 (secure 0 0 0) 0x7f08140b4070 con 0x7f0804003980 2023-11-26T07:19:17.520 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:17.512+0000 7f080dffb640 1 --2- 172.21.15.143:0/3469336951 >> [v2:172.21.15.143:6832/2150557544,v1:172.21.15.143:6833/2150557544] conn(0x7f07fc066650 0x7f07fc068b10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:19:17.520 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:17.512+0000 7f080dffb640 1 -- 172.21.15.143:0/3469336951 <== mon.0 v2:172.21.15.143:3300/0 5 ==== osd_map(33..33 src has 1..33) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f081411d8e0 con 0x7f0804003980 2023-11-26T07:19:17.520 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:17.512+0000 7f081ca2c640 1 --2- 172.21.15.143:0/3469336951 >> [v2:172.21.15.143:6832/2150557544,v1:172.21.15.143:6833/2150557544] conn(0x7f07fc066650 0x7f07fc068b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:19:17.520 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:17.512+0000 7f081ca2c640 1 --2- 172.21.15.143:0/3469336951 >> [v2:172.21.15.143:6832/2150557544,v1:172.21.15.143:6833/2150557544] conn(0x7f07fc066650 0x7f07fc068b10 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 crypto rx=0x7f08180a0e60 tx=0x7f0808009040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:19:17.520 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:17.516+0000 7f080dffb640 1 -- 172.21.15.143:0/3469336951 <== mon.0 v2:172.21.15.143:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172649 (secure 0 0 0) 0x7f08140ea9b0 con 0x7f0804003980 2023-11-26T07:19:17.671 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:17.668+0000 7f081e22f640 1 -- 172.21.15.143:0/3469336951 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f081815c080 con 0x7f0804003980 2023-11-26T07:19:17.672 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:17.668+0000 7f080dffb640 1 -- 172.21.15.143:0/3469336951 <== mon.0 v2:172.21.15.143:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v33) v1 ==== 74+0+16605 (secure 0 0 0) 0x7f081815c080 con 0x7f0804003980 2023-11-26T07:19:17.673 INFO:teuthology.orchestra.run.smithi143.stdout: 2023-11-26T07:19:17.673 INFO:teuthology.orchestra.run.smithi143.stdout:{"epoch":33,"fsid":"f5033060-2469-4a07-95fe-2b76f5404bc1","created":"2023-11-26T07:17:28.613663+0000","modified":"2023-11-26T07:18:53.659425+0000","last_up_change":"2023-11-26T07:17:41.157813+0000","last_in_change":"2023-11-26T07:17:31.181658+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":5,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-11-26T07:17:41.758597+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_metadata","create_time":"2023-11-26T07:18:50.535934+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"32","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":5,"pool_name":"cephfs_data","create_time":"2023-11-26T07:18:51.008494+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"33","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"ebbee2c6-023e-44b9-9f1d-6f7bfd309730","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6817","nonce":639807369},{"type":"v1","addr":"172.21.15.143:6819","nonce":639807369}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6821","nonce":639807369},{"type":"v1","addr":"172.21.15.143:6823","nonce":639807369}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6830","nonce":639807369},{"type":"v1","addr":"172.21.15.143:6831","nonce":639807369}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6825","nonce":639807369},{"type":"v1","addr":"172.21.15.143:6828","nonce":639807369}]},"public_addr":"172.21.15.143:6819/639807369","cluster_addr":"172.21.15.143:6823/639807369","heartbeat_back_addr":"172.21.15.143:6831/639807369","heartbeat_front_addr":"172.21.15.143:6828/639807369","state":["exists","up"]},{"osd":1,"uuid":"fec4b8be-7122-49e9-9a19-5850a0700001","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6816","nonce":1144942530},{"type":"v1","addr":"172.21.15.143:6818","nonce":1144942530}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6820","nonce":1144942530},{"type":"v1","addr":"172.21.15.143:6822","nonce":1144942530}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6827","nonce":1144942530},{"type":"v1","addr":"172.21.15.143:6829","nonce":1144942530}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6824","nonce":1144942530},{"type":"v1","addr":"172.21.15.143:6826","nonce":1144942530}]},"public_addr":"172.21.15.143:6818/1144942530","cluster_addr":"172.21.15.143:6822/1144942530","heartbeat_back_addr":"172.21.15.143:6829/1144942530","heartbeat_front_addr":"172.21.15.143:6826/1144942530","state":["exists","up"]},{"osd":2,"uuid":"2888af5b-f933-46ca-8f95-6dd494ff43f6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6808","nonce":3909095186},{"type":"v1","addr":"172.21.15.143:6809","nonce":3909095186}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6810","nonce":3909095186},{"type":"v1","addr":"172.21.15.143:6811","nonce":3909095186}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6814","nonce":3909095186},{"type":"v1","addr":"172.21.15.143:6815","nonce":3909095186}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6812","nonce":3909095186},{"type":"v1","addr":"172.21.15.143:6813","nonce":3909095186}]},"public_addr":"172.21.15.143:6809/3909095186","cluster_addr":"172.21.15.143:6811/3909095186","heartbeat_back_addr":"172.21.15.143:6815/3909095186","heartbeat_front_addr":"172.21.15.143:6813/3909095186","state":["exists","up"]},{"osd":3,"uuid":"d1f52435-7bc1-4081-9550-9cd1ba55f549","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6800","nonce":3854210991},{"type":"v1","addr":"172.21.15.143:6801","nonce":3854210991}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6802","nonce":3854210991},{"type":"v1","addr":"172.21.15.143:6803","nonce":3854210991}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6806","nonce":3854210991},{"type":"v1","addr":"172.21.15.143:6807","nonce":3854210991}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6804","nonce":3854210991},{"type":"v1","addr":"172.21.15.143:6805","nonce":3854210991}]},"public_addr":"172.21.15.143:6801/3854210991","cluster_addr":"172.21.15.143:6803/3854210991","heartbeat_back_addr":"172.21.15.143:6807/3854210991","heartbeat_front_addr":"172.21.15.143:6805/3854210991","state":["exists","up"]},{"osd":4,"uuid":"e2ae8b6e-6e25-49b2-bcbf-aacce68f06a3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6800","nonce":3924606902},{"type":"v1","addr":"172.21.15.188:6801","nonce":3924606902}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6802","nonce":3924606902},{"type":"v1","addr":"172.21.15.188:6803","nonce":3924606902}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6806","nonce":3924606902},{"type":"v1","addr":"172.21.15.188:6807","nonce":3924606902}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6804","nonce":3924606902},{"type":"v1","addr":"172.21.15.188:6805","nonce":3924606902}]},"public_addr":"172.21.15.188:6801/3924606902","cluster_addr":"172.21.15.188:6803/3924606902","heartbeat_back_addr":"172.21.15.188:6807/3924606902","heartbeat_front_addr":"172.21.15.188:6805/3924606902","state":["exists","up"]},{"osd":5,"uuid":"3cec8a32-7698-4c36-8e89-e4ba11934f33","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6813","nonce":186841544},{"type":"v1","addr":"172.21.15.188:6815","nonce":186841544}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6817","nonce":186841544},{"type":"v1","addr":"172.21.15.188:6818","nonce":186841544}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6822","nonce":186841544},{"type":"v1","addr":"172.21.15.188:6823","nonce":186841544}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6820","nonce":186841544},{"type":"v1","addr":"172.21.15.188:6821","nonce":186841544}]},"public_addr":"172.21.15.188:6815/186841544","cluster_addr":"172.21.15.188:6818/186841544","heartbeat_back_addr":"172.21.15.188:6823/186841544","heartbeat_front_addr":"172.21.15.188:6821/186841544","state":["exists","up"]},{"osd":6,"uuid":"58a65d5d-e090-473a-81b4-ad38e85029f9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6808","nonce":2153499405},{"type":"v1","addr":"172.21.15.188:6809","nonce":2153499405}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6810","nonce":2153499405},{"type":"v1","addr":"172.21.15.188:6811","nonce":2153499405}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6816","nonce":2153499405},{"type":"v1","addr":"172.21.15.188:6819","nonce":2153499405}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6812","nonce":2153499405},{"type":"v1","addr":"172.21.15.188:6814","nonce":2153499405}]},"public_addr":"172.21.15.188:6809/2153499405","cluster_addr":"172.21.15.188:6811/2153499405","heartbeat_back_addr":"172.21.15.188:6819/2153499405","heartbeat_front_addr":"172.21.15.188:6814/2153499405","state":["exists","up"]},{"osd":7,"uuid":"ea5b7f70-3554-4cf8-ac30-f7c59b80678d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6824","nonce":2722096729},{"type":"v1","addr":"172.21.15.188:6825","nonce":2722096729}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6826","nonce":2722096729},{"type":"v1","addr":"172.21.15.188:6827","nonce":2722096729}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6830","nonce":2722096729},{"type":"v1","addr":"172.21.15.188:6831","nonce":2722096729}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6828","nonce":2722096729},{"type":"v1","addr":"172.21.15.188:6829","nonce":2722096729}]},"public_addr":"172.21.15.188:6825/2722096729","cluster_addr":"172.21.15.188:6827/2722096729","heartbeat_back_addr":"172.21.15.188:6831/2722096729","heartbeat_front_addr":"172.21.15.188:6829/2722096729","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-26T07:17:36.995032+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-26T07:17:36.932266+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-26T07:17:36.829304+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-26T07:17:36.804297+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-26T07:17:36.667892+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-26T07:17:36.964868+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-26T07:17:36.827332+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-26T07:17:36.893179+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-11-26T07:19:17.675 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:17.672+0000 7f081e22f640 1 -- 172.21.15.143:0/3469336951 >> [v2:172.21.15.143:6832/2150557544,v1:172.21.15.143:6833/2150557544] conn(0x7f07fc066650 msgr2=0x7f07fc068b10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:19:17.675 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:17.672+0000 7f081e22f640 1 --2- 172.21.15.143:0/3469336951 >> [v2:172.21.15.143:6832/2150557544,v1:172.21.15.143:6833/2150557544] conn(0x7f07fc066650 0x7f07fc068b10 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 crypto rx=0x7f08180a0e60 tx=0x7f0808009040 comp rx=0 tx=0).stop 2023-11-26T07:19:17.675 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:17.672+0000 7f081e22f640 1 -- 172.21.15.143:0/3469336951 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f0804003980 msgr2=0x7f081815c790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:19:17.675 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:17.672+0000 7f081e22f640 1 --2- 172.21.15.143:0/3469336951 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f0804003980 0x7f081815c790 secure :-1 s=READY pgs=500 cs=0 l=1 rev1=1 crypto rx=0x7f0814009ae0 tx=0x7f08140197c0 comp rx=0 tx=0).stop 2023-11-26T07:19:17.675 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:17.672+0000 7f081e22f640 1 -- 172.21.15.143:0/3469336951 shutdown_connections 2023-11-26T07:19:17.676 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:17.672+0000 7f081e22f640 1 --2- 172.21.15.143:0/3469336951 >> [v2:172.21.15.143:6832/2150557544,v1:172.21.15.143:6833/2150557544] conn(0x7f07fc066650 0x7f07fc068b10 unknown :-1 s=CLOSED pgs=256 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:19:17.676 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:17.672+0000 7f081e22f640 1 --2- 172.21.15.143:0/3469336951 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f081815ba30 0x7f08180c8130 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:19:17.676 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:17.672+0000 7f081e22f640 1 --2- 172.21.15.143:0/3469336951 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f081815e940 0x7f0818161550 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:19:17.676 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:17.672+0000 7f081e22f640 1 --2- 172.21.15.143:0/3469336951 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f0804003980 0x7f081815c790 unknown :-1 s=CLOSED pgs=500 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:19:17.676 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:17.672+0000 7f081e22f640 1 -- 172.21.15.143:0/3469336951 >> 172.21.15.143:0/3469336951 conn(0x7f08180d3060 msgr2=0x7f08180b0850 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:19:17.676 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:17.672+0000 7f081e22f640 1 -- 172.21.15.143:0/3469336951 shutdown_connections 2023-11-26T07:19:17.676 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:17.672+0000 7f081e22f640 1 -- 172.21.15.143:0/3469336951 wait complete. 2023-11-26T07:19:17.694 INFO:tasks.cephfs.filesystem:Destroying file system cephfs and related pools 2023-11-26T07:19:17.694 DEBUG:teuthology.orchestra.run.smithi143:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-11-26T07:19:17.832 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:17.828+0000 7fed17ee4640 1 Processor -- start 2023-11-26T07:19:17.833 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:17.828+0000 7fed17ee4640 1 -- start start 2023-11-26T07:19:17.834 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:17.832+0000 7fed17ee4640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fed1013e290 0x7fed100cf280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:19:17.834 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:17.832+0000 7fed17ee4640 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7fed1015f6b0 0x7fed100cd7b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:19:17.834 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:17.832+0000 7fed17ee4640 1 -- --> v1:172.21.15.188:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fed1006c520 con 0x7fed100cf7c0 2023-11-26T07:19:17.834 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:17.832+0000 7fed17ee4640 1 -- --> v2:172.21.15.188:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fed100cfc40 con 0x7fed1015f6b0 2023-11-26T07:19:17.834 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:17.832+0000 7fed17ee4640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fed1015fb40 con 0x7fed1013e290 2023-11-26T07:19:17.835 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:17.832+0000 7fed16481640 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7fed1015f6b0 0x7fed100cd7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:19:17.835 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:17.832+0000 7fed15c80640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fed1013e290 0x7fed100cf280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:19:17.835 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:17.832+0000 7fed16481640 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7fed1015f6b0 0x7fed100cd7b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.188:3300/0 says I am v2:172.21.15.143:37624/0 (socket says 172.21.15.143:37624) 2023-11-26T07:19:17.835 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:17.832+0000 7fed16481640 1 -- 172.21.15.143:0/2544657359 learned_addr learned my addr 172.21.15.143:0/2544657359 (peer_addr_for_me v2:172.21.15.143:0/0) 2023-11-26T07:19:17.836 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:17.832+0000 7fed16481640 1 -- 172.21.15.143:0/2544657359 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fed1013e290 msgr2=0x7fed100cf280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-26T07:19:17.836 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:17.832+0000 7fed16481640 1 --2- 172.21.15.143:0/2544657359 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fed1013e290 0x7fed100cf280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:19:17.836 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:17.832+0000 7fed16481640 1 -- 172.21.15.143:0/2544657359 >> v1:172.21.15.188:6789/0 conn(0x7fed100cf7c0 legacy=0x7fed1015efa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:19:17.836 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:17.832+0000 7fed16481640 1 -- 172.21.15.143:0/2544657359 --> v2:172.21.15.188:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fed1006c800 con 0x7fed1015f6b0 2023-11-26T07:19:17.837 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:17.836+0000 7fed16481640 1 --2- 172.21.15.143:0/2544657359 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fed1015f6b0 0x7fed100cd7b0 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7fed08009e50 tx=0x7fed0801c9e0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=ede7d2a8e5d32605 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:19:17.838 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:17.836+0000 7fed14c7e640 1 -- 172.21.15.143:0/2544657359 <== mon.1 v2:172.21.15.188:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fed080a58a0 con 0x7fed1015f6b0 2023-11-26T07:19:17.838 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:17.836+0000 7fed14c7e640 1 -- 172.21.15.143:0/2544657359 <== mon.1 v2:172.21.15.188:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fed080a5a00 con 0x7fed1015f6b0 2023-11-26T07:19:17.838 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:17.836+0000 7fed14c7e640 1 -- 172.21.15.143:0/2544657359 <== mon.1 v2:172.21.15.188:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fed080a5d90 con 0x7fed1015f6b0 2023-11-26T07:19:17.839 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:17.836+0000 7fed17ee4640 1 -- 172.21.15.143:0/2544657359 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fed1015f6b0 msgr2=0x7fed100cd7b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:19:17.839 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:17.836+0000 7fed17ee4640 1 --2- 172.21.15.143:0/2544657359 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fed1015f6b0 0x7fed100cd7b0 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7fed08009e50 tx=0x7fed0801c9e0 comp rx=0 tx=0).stop 2023-11-26T07:19:17.840 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:17.836+0000 7fed17ee4640 1 -- 172.21.15.143:0/2544657359 shutdown_connections 2023-11-26T07:19:17.840 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:17.836+0000 7fed17ee4640 1 --2- 172.21.15.143:0/2544657359 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fed1015f6b0 0x7fed100cd7b0 unknown :-1 s=CLOSED pgs=221 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:19:17.840 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:17.836+0000 7fed17ee4640 1 --2- 172.21.15.143:0/2544657359 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fed1013e290 0x7fed100cf280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:19:17.840 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:17.836+0000 7fed17ee4640 1 -- 172.21.15.143:0/2544657359 >> 172.21.15.143:0/2544657359 conn(0x7fed1006a890 msgr2=0x7fed1006aca0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:19:17.840 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:17.836+0000 7fed17ee4640 1 -- 172.21.15.143:0/2544657359 shutdown_connections 2023-11-26T07:19:17.840 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:17.836+0000 7fed17ee4640 1 -- 172.21.15.143:0/2544657359 wait complete. 2023-11-26T07:19:17.841 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:17.836+0000 7fed17ee4640 1 Processor -- start 2023-11-26T07:19:17.841 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:17.840+0000 7fed17ee4640 1 -- start start 2023-11-26T07:19:17.843 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:17.840+0000 7fed17ee4640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fed100cf7c0 0x7fed1014e5c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:19:17.843 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:17.840+0000 7fed17ee4640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fed1013e290 0x7fed10139850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:19:17.844 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:17.840+0000 7fed17ee4640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fed1015f6b0 0x7fed101372e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:19:17.844 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:17.840+0000 7fed17ee4640 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fed10150470 con 0x7fed1015f6b0 2023-11-26T07:19:17.844 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:17.840+0000 7fed17ee4640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fed1014e300 con 0x7fed1013e290 2023-11-26T07:19:17.844 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:17.840+0000 7fed17ee4640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fed1014f7f0 con 0x7fed100cf7c0 2023-11-26T07:19:17.844 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:17.840+0000 7fed16481640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fed1015f6b0 0x7fed101372e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:19:17.844 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:17.840+0000 7fed16481640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fed1015f6b0 0x7fed101372e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:48012/0 (socket says 172.21.15.143:48012) 2023-11-26T07:19:17.844 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:17.840+0000 7fed16481640 1 -- 172.21.15.143:0/3341255795 learned_addr learned my addr 172.21.15.143:0/3341255795 (peer_addr_for_me v2:172.21.15.143:0/0) 2023-11-26T07:19:17.844 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:17.840+0000 7fed1547f640 1 --2- 172.21.15.143:0/3341255795 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fed1013e290 0x7fed10139850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:19:17.845 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:17.840+0000 7fed16481640 1 -- 172.21.15.143:0/3341255795 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fed100cf7c0 msgr2=0x7fed1014e5c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:19:17.845 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:17.840+0000 7fed15c80640 1 --2- 172.21.15.143:0/3341255795 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fed100cf7c0 0x7fed1014e5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:19:17.845 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:17.840+0000 7fed16481640 1 --2- 172.21.15.143:0/3341255795 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fed100cf7c0 0x7fed1014e5c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:19:17.845 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:17.840+0000 7fed16481640 1 -- 172.21.15.143:0/3341255795 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fed1013e290 msgr2=0x7fed10139850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:19:17.845 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:17.840+0000 7fed16481640 1 --2- 172.21.15.143:0/3341255795 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fed1013e290 0x7fed10139850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:19:17.845 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:17.840+0000 7fed16481640 1 -- 172.21.15.143:0/3341255795 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fed08009b30 con 0x7fed1015f6b0 2023-11-26T07:19:17.845 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:17.840+0000 7fed15c80640 1 --2- 172.21.15.143:0/3341255795 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fed100cf7c0 0x7fed1014e5c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-26T07:19:17.845 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:17.840+0000 7fed16481640 1 --2- 172.21.15.143:0/3341255795 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fed1015f6b0 0x7fed101372e0 secure :-1 s=READY pgs=501 cs=0 l=1 rev1=1 crypto rx=0x7fed080af040 tx=0x7fed08019780 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:19:17.845 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:17.840+0000 7fed067fc640 1 -- 172.21.15.143:0/3341255795 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fed080bd070 con 0x7fed1015f6b0 2023-11-26T07:19:17.845 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:17.840+0000 7fed067fc640 1 -- 172.21.15.143:0/3341255795 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fed080b8470 con 0x7fed1015f6b0 2023-11-26T07:19:17.845 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:17.840+0000 7fed17ee4640 1 -- 172.21.15.143:0/3341255795 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fed1014fad0 con 0x7fed1015f6b0 2023-11-26T07:19:17.845 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:17.840+0000 7fed067fc640 1 -- 172.21.15.143:0/3341255795 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fed080b8840 con 0x7fed1015f6b0 2023-11-26T07:19:17.846 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:17.840+0000 7fed17ee4640 1 -- 172.21.15.143:0/3341255795 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fed10150090 con 0x7fed1015f6b0 2023-11-26T07:19:17.846 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:17.840+0000 7fed17ee4640 1 -- 172.21.15.143:0/3341255795 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fed10067680 con 0x7fed1015f6b0 2023-11-26T07:19:17.846 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:17.844+0000 7fed067fc640 1 -- 172.21.15.143:0/3341255795 <== mon.0 v2:172.21.15.143:3300/0 4 ==== mgrmap(e 7) v1 ==== 86005+0+0 (secure 0 0 0) 0x7fed080b89a0 con 0x7fed1015f6b0 2023-11-26T07:19:17.847 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:17.844+0000 7fed067fc640 1 --2- 172.21.15.143:0/3341255795 >> [v2:172.21.15.143:6832/2150557544,v1:172.21.15.143:6833/2150557544] conn(0x7fecf4066620 0x7fecf4068ae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:19:17.847 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:17.844+0000 7fed067fc640 1 -- 172.21.15.143:0/3341255795 <== mon.0 v2:172.21.15.143:3300/0 5 ==== osd_map(33..33 src has 1..33) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fed0811f2b0 con 0x7fed1015f6b0 2023-11-26T07:19:17.847 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:17.844+0000 7fed15c80640 1 --2- 172.21.15.143:0/3341255795 >> [v2:172.21.15.143:6832/2150557544,v1:172.21.15.143:6833/2150557544] conn(0x7fecf4066620 0x7fecf4068ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:19:17.847 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:17.844+0000 7fed15c80640 1 --2- 172.21.15.143:0/3341255795 >> [v2:172.21.15.143:6832/2150557544,v1:172.21.15.143:6833/2150557544] conn(0x7fecf4066620 0x7fecf4068ae0 secure :-1 s=READY pgs=257 cs=0 l=1 rev1=1 crypto rx=0x7fed0c009ae0 tx=0x7fed0c008040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:19:17.850 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:17.848+0000 7fed067fc640 1 -- 172.21.15.143:0/3341255795 <== mon.0 v2:172.21.15.143:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172649 (secure 0 0 0) 0x7fed080ec5d0 con 0x7fed1015f6b0 2023-11-26T07:19:18.032 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:18.028+0000 7fed17ee4640 1 -- 172.21.15.143:0/3341255795 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fed100b2af0 con 0x7fed1015f6b0 2023-11-26T07:19:18.032 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:18.028+0000 7fed067fc640 1 -- 172.21.15.143:0/3341255795 <== mon.0 v2:172.21.15.143:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 17 v17) v1 ==== 94+0+4925 (secure 0 0 0) 0x7fed080ace80 con 0x7fed1015f6b0 2023-11-26T07:19:18.033 INFO:teuthology.orchestra.run.smithi143.stdout: 2023-11-26T07:19:18.033 INFO:teuthology.orchestra.run.smithi143.stdout:{"epoch":17,"default_fscid":2,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4847,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.188:6834/1547410397","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6832","nonce":1547410397},{"type":"v1","addr":"172.21.15.188:6834","nonce":1547410397}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":12},{"gid":4850,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.188:6835/3247640329","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6833","nonce":3247640329},{"type":"v1","addr":"172.21.15.188:6835","nonce":3247640329}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":12},{"gid":5217,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.143:6835/2123707650","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6834","nonce":2123707650},{"type":"v1","addr":"172.21.15.143:6835","nonce":2123707650}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":11}],"filesystems":[{"mdsmap":{"epoch":16,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-11-26T07:18:52.014584+0000","modified":"2023-11-26T07:18:55.043646+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":5220},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5220":{"gid":5220,"name":"c","rank":0,"incarnation":13,"state":"up:active","state_seq":3,"addr":"172.21.15.143:6837/2658667363","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6836","nonce":2658667363},{"type":"v1","addr":"172.21.15.143:6837","nonce":2658667363}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[5],"metadata_pool":4,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":2}]} 2023-11-26T07:19:18.033 INFO:teuthology.orchestra.run.smithi143.stderr:dumped fsmap epoch 17 2023-11-26T07:19:18.034 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:18.032+0000 7fed17ee4640 1 -- 172.21.15.143:0/3341255795 >> [v2:172.21.15.143:6832/2150557544,v1:172.21.15.143:6833/2150557544] conn(0x7fecf4066620 msgr2=0x7fecf4068ae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:19:18.035 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:18.032+0000 7fed17ee4640 1 --2- 172.21.15.143:0/3341255795 >> [v2:172.21.15.143:6832/2150557544,v1:172.21.15.143:6833/2150557544] conn(0x7fecf4066620 0x7fecf4068ae0 secure :-1 s=READY pgs=257 cs=0 l=1 rev1=1 crypto rx=0x7fed0c009ae0 tx=0x7fed0c008040 comp rx=0 tx=0).stop 2023-11-26T07:19:18.035 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:18.032+0000 7fed17ee4640 1 -- 172.21.15.143:0/3341255795 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fed1015f6b0 msgr2=0x7fed101372e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:19:18.035 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:18.032+0000 7fed17ee4640 1 --2- 172.21.15.143:0/3341255795 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fed1015f6b0 0x7fed101372e0 secure :-1 s=READY pgs=501 cs=0 l=1 rev1=1 crypto rx=0x7fed080af040 tx=0x7fed08019780 comp rx=0 tx=0).stop 2023-11-26T07:19:18.035 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:18.032+0000 7fed17ee4640 1 -- 172.21.15.143:0/3341255795 shutdown_connections 2023-11-26T07:19:18.035 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:18.032+0000 7fed17ee4640 1 --2- 172.21.15.143:0/3341255795 >> [v2:172.21.15.143:6832/2150557544,v1:172.21.15.143:6833/2150557544] conn(0x7fecf4066620 0x7fecf4068ae0 unknown :-1 s=CLOSED pgs=257 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:19:18.035 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:18.032+0000 7fed17ee4640 1 --2- 172.21.15.143:0/3341255795 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fed1015f6b0 0x7fed101372e0 unknown :-1 s=CLOSED pgs=501 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:19:18.035 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:18.032+0000 7fed17ee4640 1 --2- 172.21.15.143:0/3341255795 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fed1013e290 0x7fed10139850 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:19:18.035 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:18.032+0000 7fed17ee4640 1 --2- 172.21.15.143:0/3341255795 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fed100cf7c0 0x7fed1014e5c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:19:18.035 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:18.032+0000 7fed17ee4640 1 -- 172.21.15.143:0/3341255795 >> 172.21.15.143:0/3341255795 conn(0x7fed1006a890 msgr2=0x7fed1006ac70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:19:18.035 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:18.032+0000 7fed17ee4640 1 -- 172.21.15.143:0/3341255795 shutdown_connections 2023-11-26T07:19:18.035 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:18.032+0000 7fed17ee4640 1 -- 172.21.15.143:0/3341255795 wait complete. 2023-11-26T07:19:18.053 DEBUG:teuthology.orchestra.run.smithi143:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-11-26T07:19:18.190 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:18.188+0000 7f98ee765640 1 Processor -- start 2023-11-26T07:19:18.190 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:18.188+0000 7f98ee765640 1 -- start start 2023-11-26T07:19:18.192 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:18.188+0000 7f98ee765640 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7f98e81596d0 0x7f98e8071850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:19:18.192 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:18.188+0000 7f98ee765640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f98e80ce680 0x7f98e80b2870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:19:18.192 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:18.188+0000 7f98ee765640 1 -- --> v1:172.21.15.188:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f98e806c750 con 0x7f98e80b2db0 2023-11-26T07:19:18.192 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:18.188+0000 7f98ee765640 1 -- --> v2:172.21.15.143:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f98e80b3240 con 0x7f98e81596d0 2023-11-26T07:19:18.192 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:18.188+0000 7f98ee765640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f98e80cbe00 con 0x7f98e80ce680 2023-11-26T07:19:18.193 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:18.188+0000 7f98e7fff640 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7f98e81596d0 0x7f98e8071850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:19:18.193 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:18.188+0000 7f98e7fff640 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7f98e81596d0 0x7f98e8071850 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:48020/0 (socket says 172.21.15.143:48020) 2023-11-26T07:19:18.193 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:18.188+0000 7f98e7fff640 1 -- 172.21.15.143:0/2574160018 learned_addr learned my addr 172.21.15.143:0/2574160018 (peer_addr_for_me v2:172.21.15.143:0/0) 2023-11-26T07:19:18.193 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:18.188+0000 7f98e77fe640 1 --2- 172.21.15.143:0/2574160018 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f98e80ce680 0x7f98e80b2870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:19:18.193 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:18.188+0000 7f98e7fff640 1 -- 172.21.15.143:0/2574160018 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f98e80ce680 msgr2=0x7f98e80b2870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-26T07:19:18.193 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:18.188+0000 7f98e7fff640 1 --2- 172.21.15.143:0/2574160018 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f98e80ce680 0x7f98e80b2870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:19:18.193 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:18.188+0000 7f98e7fff640 1 -- 172.21.15.143:0/2574160018 >> v1:172.21.15.188:6789/0 conn(0x7f98e80b2db0 legacy=0x7f98e815d2c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-26T07:19:18.193 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:18.188+0000 7f98e7fff640 1 -- 172.21.15.143:0/2574160018 --> v2:172.21.15.143:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f98e80b3550 con 0x7f98e81596d0 2023-11-26T07:19:18.195 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:18.192+0000 7f98e7fff640 1 --2- 172.21.15.143:0/2574160018 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f98e81596d0 0x7f98e8071850 secure :-1 s=READY pgs=502 cs=0 l=1 rev1=1 crypto rx=0x7f98d800aa10 tx=0x7f98d801d910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bacbdb8a50482255 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:19:18.195 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:18.192+0000 7f98e6ffd640 1 -- 172.21.15.143:0/2574160018 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f98d80a6830 con 0x7f98e81596d0 2023-11-26T07:19:18.195 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:18.192+0000 7f98e6ffd640 1 -- 172.21.15.143:0/2574160018 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f98d80a6990 con 0x7f98e81596d0 2023-11-26T07:19:18.195 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:18.192+0000 7f98e6ffd640 1 -- 172.21.15.143:0/2574160018 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f98d80a6d20 con 0x7f98e81596d0 2023-11-26T07:19:18.196 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:18.192+0000 7f98ee765640 1 -- 172.21.15.143:0/2574160018 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f98e81596d0 msgr2=0x7f98e8071850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:19:18.196 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:18.192+0000 7f98ee765640 1 --2- 172.21.15.143:0/2574160018 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f98e81596d0 0x7f98e8071850 secure :-1 s=READY pgs=502 cs=0 l=1 rev1=1 crypto rx=0x7f98d800aa10 tx=0x7f98d801d910 comp rx=0 tx=0).stop 2023-11-26T07:19:18.196 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:18.192+0000 7f98ee765640 1 -- 172.21.15.143:0/2574160018 shutdown_connections 2023-11-26T07:19:18.197 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:18.192+0000 7f98ee765640 1 --2- 172.21.15.143:0/2574160018 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f98e80ce680 0x7f98e80b2870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:19:18.197 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:18.192+0000 7f98ee765640 1 --2- 172.21.15.143:0/2574160018 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f98e81596d0 0x7f98e8071850 unknown :-1 s=CLOSED pgs=502 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:19:18.197 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:18.192+0000 7f98ee765640 1 -- 172.21.15.143:0/2574160018 >> 172.21.15.143:0/2574160018 conn(0x7f98e806a890 msgr2=0x7f98e806aca0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:19:18.197 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:18.192+0000 7f98ee765640 1 -- 172.21.15.143:0/2574160018 shutdown_connections 2023-11-26T07:19:18.197 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:18.192+0000 7f98ee765640 1 -- 172.21.15.143:0/2574160018 wait complete. 2023-11-26T07:19:18.198 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:18.196+0000 7f98ee765640 1 Processor -- start 2023-11-26T07:19:18.198 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:18.196+0000 7f98ee765640 1 -- start start 2023-11-26T07:19:18.200 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:18.196+0000 7f98ee765640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f98e80b2db0 0x7f98e8146270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:19:18.200 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:18.196+0000 7f98ee765640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f98e80ce680 0x7f98e8158970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:19:18.200 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:18.196+0000 7f98ee765640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f98e81596d0 0x7f98e814c600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:19:18.200 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:18.196+0000 7f98ee765640 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f98e815c520 con 0x7f98e80b2db0 2023-11-26T07:19:18.201 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:18.196+0000 7f98ee765640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f98e815b450 con 0x7f98e80ce680 2023-11-26T07:19:18.201 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:18.196+0000 7f98ee765640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f98e815a8d0 con 0x7f98e81596d0 2023-11-26T07:19:18.201 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:18.196+0000 7f98e7fff640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f98e80b2db0 0x7f98e8146270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:19:18.201 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:18.196+0000 7f98e7fff640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f98e80b2db0 0x7f98e8146270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:48032/0 (socket says 172.21.15.143:48032) 2023-11-26T07:19:18.201 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:18.196+0000 7f98e7fff640 1 -- 172.21.15.143:0/2729992646 learned_addr learned my addr 172.21.15.143:0/2729992646 (peer_addr_for_me v2:172.21.15.143:0/0) 2023-11-26T07:19:18.201 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:18.196+0000 7f98ecd02640 1 --2- 172.21.15.143:0/2729992646 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f98e81596d0 0x7f98e814c600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:19:18.201 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:18.196+0000 7f98e77fe640 1 --2- 172.21.15.143:0/2729992646 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f98e80ce680 0x7f98e8158970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:19:18.201 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:18.196+0000 7f98e7fff640 1 -- 172.21.15.143:0/2729992646 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f98e81596d0 msgr2=0x7f98e814c600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:19:18.202 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:18.196+0000 7f98e7fff640 1 --2- 172.21.15.143:0/2729992646 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f98e81596d0 0x7f98e814c600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:19:18.202 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:18.196+0000 7f98e7fff640 1 -- 172.21.15.143:0/2729992646 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f98e80ce680 msgr2=0x7f98e8158970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:19:18.202 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:18.196+0000 7f98e7fff640 1 --2- 172.21.15.143:0/2729992646 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f98e80ce680 0x7f98e8158970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:19:18.202 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:18.196+0000 7f98e7fff640 1 -- 172.21.15.143:0/2729992646 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f98d800a670 con 0x7f98e80b2db0 2023-11-26T07:19:18.202 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:18.196+0000 7f98e77fe640 1 --2- 172.21.15.143:0/2729992646 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f98e80ce680 0x7f98e8158970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-26T07:19:18.202 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:18.196+0000 7f98e7fff640 1 --2- 172.21.15.143:0/2729992646 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f98e80b2db0 0x7f98e8146270 secure :-1 s=READY pgs=503 cs=0 l=1 rev1=1 crypto rx=0x7f98d801b540 tx=0x7f98d801c150 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:19:18.202 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:18.196+0000 7f98e4ff9640 1 -- 172.21.15.143:0/2729992646 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f98d80b9920 con 0x7f98e80b2db0 2023-11-26T07:19:18.202 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:18.196+0000 7f98e4ff9640 1 -- 172.21.15.143:0/2729992646 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f98d80b9a80 con 0x7f98e80b2db0 2023-11-26T07:19:18.202 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:18.196+0000 7f98ee765640 1 -- 172.21.15.143:0/2729992646 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f98e815abb0 con 0x7f98e80b2db0 2023-11-26T07:19:18.203 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:18.196+0000 7f98e4ff9640 1 -- 172.21.15.143:0/2729992646 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f98d80b9e30 con 0x7f98e80b2db0 2023-11-26T07:19:18.203 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:18.196+0000 7f98ee765640 1 -- 172.21.15.143:0/2729992646 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f98e815b170 con 0x7f98e80b2db0 2023-11-26T07:19:18.203 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:18.200+0000 7f98ee765640 1 -- 172.21.15.143:0/2729992646 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f98e8067680 con 0x7f98e80b2db0 2023-11-26T07:19:18.204 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:18.200+0000 7f98e4ff9640 1 -- 172.21.15.143:0/2729992646 <== mon.0 v2:172.21.15.143:3300/0 4 ==== mgrmap(e 7) v1 ==== 86005+0+0 (secure 0 0 0) 0x7f98d80b5070 con 0x7f98e80b2db0 2023-11-26T07:19:18.204 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:18.200+0000 7f98e4ff9640 1 --2- 172.21.15.143:0/2729992646 >> [v2:172.21.15.143:6832/2150557544,v1:172.21.15.143:6833/2150557544] conn(0x7f98d4066620 0x7f98d4068ae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:19:18.204 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:18.200+0000 7f98e4ff9640 1 -- 172.21.15.143:0/2729992646 <== mon.0 v2:172.21.15.143:3300/0 5 ==== osd_map(33..33 src has 1..33) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f98d811edb0 con 0x7f98e80b2db0 2023-11-26T07:19:18.204 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:18.200+0000 7f98e77fe640 1 --2- 172.21.15.143:0/2729992646 >> [v2:172.21.15.143:6832/2150557544,v1:172.21.15.143:6833/2150557544] conn(0x7f98d4066620 0x7f98d4068ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:19:18.204 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:18.200+0000 7f98e77fe640 1 --2- 172.21.15.143:0/2729992646 >> [v2:172.21.15.143:6832/2150557544,v1:172.21.15.143:6833/2150557544] conn(0x7f98d4066620 0x7f98d4068ae0 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 crypto rx=0x7f98e80b3c80 tx=0x7f98dc008040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:19:18.206 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:18.204+0000 7f98e4ff9640 1 -- 172.21.15.143:0/2729992646 <== mon.0 v2:172.21.15.143:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172649 (secure 0 0 0) 0x7f98d8123070 con 0x7f98e80b2db0 2023-11-26T07:19:18.385 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:18.380+0000 7f98ee765640 1 -- 172.21.15.143:0/2729992646 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f98e80709f0 con 0x7f98e80b2db0 2023-11-26T07:19:18.385 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:18.384+0000 7f98e4ff9640 1 -- 172.21.15.143:0/2729992646 <== mon.0 v2:172.21.15.143:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 17 v17) v1 ==== 94+0+4925 (secure 0 0 0) 0x7f98d80b3070 con 0x7f98e80b2db0 2023-11-26T07:19:18.386 INFO:teuthology.orchestra.run.smithi143.stdout: 2023-11-26T07:19:18.386 INFO:teuthology.orchestra.run.smithi143.stdout:{"epoch":17,"default_fscid":2,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4847,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.188:6834/1547410397","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6832","nonce":1547410397},{"type":"v1","addr":"172.21.15.188:6834","nonce":1547410397}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":12},{"gid":4850,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.188:6835/3247640329","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6833","nonce":3247640329},{"type":"v1","addr":"172.21.15.188:6835","nonce":3247640329}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":12},{"gid":5217,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.143:6835/2123707650","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6834","nonce":2123707650},{"type":"v1","addr":"172.21.15.143:6835","nonce":2123707650}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":11}],"filesystems":[{"mdsmap":{"epoch":16,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-11-26T07:18:52.014584+0000","modified":"2023-11-26T07:18:55.043646+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":5220},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5220":{"gid":5220,"name":"c","rank":0,"incarnation":13,"state":"up:active","state_seq":3,"addr":"172.21.15.143:6837/2658667363","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6836","nonce":2658667363},{"type":"v1","addr":"172.21.15.143:6837","nonce":2658667363}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[5],"metadata_pool":4,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":2}]} 2023-11-26T07:19:18.386 INFO:teuthology.orchestra.run.smithi143.stderr:dumped fsmap epoch 17 2023-11-26T07:19:18.387 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:18.384+0000 7f98ee765640 1 -- 172.21.15.143:0/2729992646 >> [v2:172.21.15.143:6832/2150557544,v1:172.21.15.143:6833/2150557544] conn(0x7f98d4066620 msgr2=0x7f98d4068ae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:19:18.387 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:18.384+0000 7f98ee765640 1 --2- 172.21.15.143:0/2729992646 >> [v2:172.21.15.143:6832/2150557544,v1:172.21.15.143:6833/2150557544] conn(0x7f98d4066620 0x7f98d4068ae0 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 crypto rx=0x7f98e80b3c80 tx=0x7f98dc008040 comp rx=0 tx=0).stop 2023-11-26T07:19:18.387 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:18.384+0000 7f98ee765640 1 -- 172.21.15.143:0/2729992646 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f98e80b2db0 msgr2=0x7f98e8146270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:19:18.388 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:18.384+0000 7f98ee765640 1 --2- 172.21.15.143:0/2729992646 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f98e80b2db0 0x7f98e8146270 secure :-1 s=READY pgs=503 cs=0 l=1 rev1=1 crypto rx=0x7f98d801b540 tx=0x7f98d801c150 comp rx=0 tx=0).stop 2023-11-26T07:19:18.388 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:18.384+0000 7f98ee765640 1 -- 172.21.15.143:0/2729992646 shutdown_connections 2023-11-26T07:19:18.388 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:18.384+0000 7f98ee765640 1 --2- 172.21.15.143:0/2729992646 >> [v2:172.21.15.143:6832/2150557544,v1:172.21.15.143:6833/2150557544] conn(0x7f98d4066620 0x7f98d4068ae0 unknown :-1 s=CLOSED pgs=258 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:19:18.388 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:18.384+0000 7f98ee765640 1 --2- 172.21.15.143:0/2729992646 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f98e81596d0 0x7f98e814c600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:19:18.388 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:18.384+0000 7f98ee765640 1 --2- 172.21.15.143:0/2729992646 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f98e80ce680 0x7f98e8158970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:19:18.388 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:18.384+0000 7f98ee765640 1 --2- 172.21.15.143:0/2729992646 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f98e80b2db0 0x7f98e8146270 unknown :-1 s=CLOSED pgs=503 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:19:18.388 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:18.384+0000 7f98ee765640 1 -- 172.21.15.143:0/2729992646 >> 172.21.15.143:0/2729992646 conn(0x7f98e806a890 msgr2=0x7f98e814db20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:19:18.388 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:18.384+0000 7f98ee765640 1 -- 172.21.15.143:0/2729992646 shutdown_connections 2023-11-26T07:19:18.388 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:18.384+0000 7f98ee765640 1 -- 172.21.15.143:0/2729992646 wait complete. 2023-11-26T07:19:18.406 DEBUG:teuthology.orchestra.run.smithi143:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-11-26T07:19:18.545 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:18.544+0000 7f5c599d0640 1 Processor -- start 2023-11-26T07:19:18.545 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:18.544+0000 7f5c599d0640 1 -- start start 2023-11-26T07:19:18.546 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:18.544+0000 7f5c599d0640 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7f5c54143810 0x7f5c54072140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:19:18.546 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:18.544+0000 7f5c599d0640 1 -- --> v1:172.21.15.143:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5c5415e860 con 0x7f5c540b1e10 2023-11-26T07:19:18.547 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:18.544+0000 7f5c599d0640 1 -- --> v1:172.21.15.188:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5c540cdba0 con 0x7f5c5415e0f0 2023-11-26T07:19:18.547 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:18.544+0000 7f5c599d0640 1 -- --> v2:172.21.15.188:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5c54143bf0 con 0x7f5c54143810 2023-11-26T07:19:18.547 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:18.544+0000 7f5c53fff640 1 --1- >> v1:172.21.15.143:6789/0 conn(0x7f5c540b1e10 0x7f5c5415d9e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.143:6789/0 says I am v1:172.21.15.143:50134/0 (socket says 172.21.15.143:50134) 2023-11-26T07:19:18.547 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:18.544+0000 7f5c53fff640 1 -- 172.21.15.143:0/1569124022 learned_addr learned my addr 172.21.15.143:0/1569124022 (peer_addr_for_me v1:172.21.15.143:0/0) 2023-11-26T07:19:18.547 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:18.544+0000 7f5c589ce640 1 --2- 172.21.15.143:0/1569124022 >> v2:172.21.15.188:3300/0 conn(0x7f5c54143810 0x7f5c54072140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:19:18.548 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:18.544+0000 7f5c537fe640 1 -- 172.21.15.143:0/1569124022 <== mon.0 v1:172.21.15.143:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 572072171 0 0) 0x7f5c5415e860 con 0x7f5c540b1e10 2023-11-26T07:19:18.548 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:18.544+0000 7f5c537fe640 1 -- 172.21.15.143:0/1569124022 --> v1:172.21.15.143:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f5c44003530 con 0x7f5c540b1e10 2023-11-26T07:19:18.548 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:18.544+0000 7f5c537fe640 1 -- 172.21.15.143:0/1569124022 <== mon.0 v1:172.21.15.143:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1481774228 0 0) 0x7f5c44003530 con 0x7f5c540b1e10 2023-11-26T07:19:18.548 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:18.544+0000 7f5c589ce640 1 -- 172.21.15.143:0/1569124022 >> v1:172.21.15.188:6789/0 conn(0x7f5c5415e0f0 legacy=0x7f5c540ce030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:19:18.548 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:18.544+0000 7f5c589ce640 1 -- 172.21.15.143:0/1569124022 >> v1:172.21.15.143:6789/0 conn(0x7f5c540b1e10 legacy=0x7f5c5415d9e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:19:18.548 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:18.544+0000 7f5c589ce640 1 -- 172.21.15.143:0/1569124022 --> v2:172.21.15.188:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5c540714f0 con 0x7f5c54143810 2023-11-26T07:19:18.551 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:18.548+0000 7f5c589ce640 1 --2- 172.21.15.143:0/1569124022 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f5c54143810 0x7f5c54072140 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7f5c480097c0 tx=0x7f5c4801c370 comp rx=0 tx=0).ready entity=mon.1 client_cookie=3be5fb187ff8df52 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:19:18.551 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:18.548+0000 7f5c537fe640 1 -- 172.21.15.143:0/1569124022 <== mon.1 v2:172.21.15.188:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5c40001680 con 0x7f5c54143810 2023-11-26T07:19:18.551 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:18.548+0000 7f5c537fe640 1 -- 172.21.15.143:0/1569124022 <== mon.1 v2:172.21.15.188:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5c480a7050 con 0x7f5c54143810 2023-11-26T07:19:18.551 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:18.548+0000 7f5c537fe640 1 -- 172.21.15.143:0/1569124022 <== mon.1 v2:172.21.15.188:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5c480ab620 con 0x7f5c54143810 2023-11-26T07:19:18.552 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:18.548+0000 7f5c599d0640 1 -- 172.21.15.143:0/1569124022 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f5c54143810 msgr2=0x7f5c54072140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:19:18.552 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:18.548+0000 7f5c599d0640 1 --2- 172.21.15.143:0/1569124022 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f5c54143810 0x7f5c54072140 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7f5c480097c0 tx=0x7f5c4801c370 comp rx=0 tx=0).stop 2023-11-26T07:19:18.552 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:18.548+0000 7f5c599d0640 1 -- 172.21.15.143:0/1569124022 shutdown_connections 2023-11-26T07:19:18.552 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:18.548+0000 7f5c599d0640 1 --2- 172.21.15.143:0/1569124022 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f5c54143810 0x7f5c54072140 unknown :-1 s=CLOSED pgs=224 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:19:18.552 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:18.548+0000 7f5c599d0640 1 -- 172.21.15.143:0/1569124022 >> 172.21.15.143:0/1569124022 conn(0x7f5c5406a890 msgr2=0x7f5c5406aca0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:19:18.552 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:18.548+0000 7f5c599d0640 1 -- 172.21.15.143:0/1569124022 shutdown_connections 2023-11-26T07:19:18.552 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:18.548+0000 7f5c599d0640 1 -- 172.21.15.143:0/1569124022 wait complete. 2023-11-26T07:19:18.553 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:18.552+0000 7f5c599d0640 1 Processor -- start 2023-11-26T07:19:18.553 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:18.552+0000 7f5c599d0640 1 -- start start 2023-11-26T07:19:18.554 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:18.552+0000 7f5c599d0640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f5c540b1e10 0x7f5c541f6000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:19:18.554 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:18.552+0000 7f5c599d0640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f5c54143810 0x7f5c5415cdd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:19:18.555 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:18.552+0000 7f5c599d0640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f5c5415e0f0 0x7f5c540b0db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:19:18.555 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:18.552+0000 7f5c599d0640 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5c5413f1e0 con 0x7f5c54143810 2023-11-26T07:19:18.555 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:18.552+0000 7f5c599d0640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5c541f6540 con 0x7f5c5415e0f0 2023-11-26T07:19:18.555 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:18.552+0000 7f5c599d0640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5c5413e560 con 0x7f5c540b1e10 2023-11-26T07:19:18.555 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:18.552+0000 7f5c53fff640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f5c54143810 0x7f5c5415cdd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:19:18.555 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:18.552+0000 7f5c53fff640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f5c54143810 0x7f5c5415cdd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:48040/0 (socket says 172.21.15.143:48040) 2023-11-26T07:19:18.555 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:18.552+0000 7f5c53fff640 1 -- 172.21.15.143:0/1398491151 learned_addr learned my addr 172.21.15.143:0/1398491151 (peer_addr_for_me v2:172.21.15.143:0/0) 2023-11-26T07:19:18.557 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:18.552+0000 7f5c589ce640 1 --2- 172.21.15.143:0/1398491151 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f5c540b1e10 0x7f5c541f6000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:19:18.557 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:18.552+0000 7f5c53fff640 1 -- 172.21.15.143:0/1398491151 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f5c540b1e10 msgr2=0x7f5c541f6000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:19:18.557 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:18.552+0000 7f5c53fff640 1 --2- 172.21.15.143:0/1398491151 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f5c540b1e10 0x7f5c541f6000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:19:18.557 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:18.552+0000 7f5c53fff640 1 -- 172.21.15.143:0/1398491151 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f5c5415e0f0 msgr2=0x7f5c540b0db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:19:18.557 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:18.552+0000 7f5c53fff640 1 --2- 172.21.15.143:0/1398491151 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f5c5415e0f0 0x7f5c540b0db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:19:18.557 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:18.552+0000 7f5c53fff640 1 -- 172.21.15.143:0/1398491151 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5c48009420 con 0x7f5c54143810 2023-11-26T07:19:18.557 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:18.552+0000 7f5c589ce640 1 --2- 172.21.15.143:0/1398491151 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f5c540b1e10 0x7f5c541f6000 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-26T07:19:18.557 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:18.552+0000 7f5c53fff640 1 --2- 172.21.15.143:0/1398491151 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f5c54143810 0x7f5c5415cdd0 secure :-1 s=READY pgs=505 cs=0 l=1 rev1=1 crypto rx=0x7f5c4000c730 tx=0x7f5c4000cbf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:19:18.558 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:18.552+0000 7f5c517fa640 1 -- 172.21.15.143:0/1398491151 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5c40005ae0 con 0x7f5c54143810 2023-11-26T07:19:18.558 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:18.552+0000 7f5c517fa640 1 -- 172.21.15.143:0/1398491151 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5c40005c40 con 0x7f5c54143810 2023-11-26T07:19:18.558 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:18.552+0000 7f5c517fa640 1 -- 172.21.15.143:0/1398491151 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5c400072b0 con 0x7f5c54143810 2023-11-26T07:19:18.558 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:18.552+0000 7f5c599d0640 1 -- 172.21.15.143:0/1398491151 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5c5413e840 con 0x7f5c54143810 2023-11-26T07:19:18.558 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:18.552+0000 7f5c599d0640 1 -- 172.21.15.143:0/1398491151 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5c5413ee00 con 0x7f5c54143810 2023-11-26T07:19:18.558 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:18.552+0000 7f5c599d0640 1 -- 172.21.15.143:0/1398491151 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5c5413f320 con 0x7f5c54143810 2023-11-26T07:19:18.559 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:18.556+0000 7f5c517fa640 1 -- 172.21.15.143:0/1398491151 <== mon.0 v2:172.21.15.143:3300/0 4 ==== mgrmap(e 7) v1 ==== 86005+0+0 (secure 0 0 0) 0x7f5c40007410 con 0x7f5c54143810 2023-11-26T07:19:18.559 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:18.556+0000 7f5c517fa640 1 --2- 172.21.15.143:0/1398491151 >> [v2:172.21.15.143:6832/2150557544,v1:172.21.15.143:6833/2150557544] conn(0x7f5c38066620 0x7f5c38068ae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:19:18.559 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:18.556+0000 7f5c517fa640 1 -- 172.21.15.143:0/1398491151 <== mon.0 v2:172.21.15.143:3300/0 5 ==== osd_map(33..33 src has 1..33) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f5c40083a10 con 0x7f5c54143810 2023-11-26T07:19:18.559 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:18.556+0000 7f5c589ce640 1 --2- 172.21.15.143:0/1398491151 >> [v2:172.21.15.143:6832/2150557544,v1:172.21.15.143:6833/2150557544] conn(0x7f5c38066620 0x7f5c38068ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:19:18.560 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:18.556+0000 7f5c589ce640 1 --2- 172.21.15.143:0/1398491151 >> [v2:172.21.15.143:6832/2150557544,v1:172.21.15.143:6833/2150557544] conn(0x7f5c38066620 0x7f5c38068ae0 secure :-1 s=READY pgs=259 cs=0 l=1 rev1=1 crypto rx=0x7f5c480ad000 tx=0x7f5c48008040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:19:18.562 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:18.560+0000 7f5c517fa640 1 -- 172.21.15.143:0/1398491151 <== mon.0 v2:172.21.15.143:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172649 (secure 0 0 0) 0x7f5c5413f320 con 0x7f5c54143810 2023-11-26T07:19:18.712 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:18.708+0000 7f5c599d0640 1 -- 172.21.15.143:0/1398491151 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f5c5413f320 con 0x7f5c54143810 2023-11-26T07:19:18.713 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:18.712+0000 7f5c517fa640 1 -- 172.21.15.143:0/1398491151 <== mon.0 v2:172.21.15.143:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v33) v1 ==== 74+0+16605 (secure 0 0 0) 0x7f5c5413f320 con 0x7f5c54143810 2023-11-26T07:19:18.714 INFO:teuthology.orchestra.run.smithi143.stdout: 2023-11-26T07:19:18.714 INFO:teuthology.orchestra.run.smithi143.stdout:{"epoch":33,"fsid":"f5033060-2469-4a07-95fe-2b76f5404bc1","created":"2023-11-26T07:17:28.613663+0000","modified":"2023-11-26T07:18:53.659425+0000","last_up_change":"2023-11-26T07:17:41.157813+0000","last_in_change":"2023-11-26T07:17:31.181658+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":5,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-11-26T07:17:41.758597+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_metadata","create_time":"2023-11-26T07:18:50.535934+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"32","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":5,"pool_name":"cephfs_data","create_time":"2023-11-26T07:18:51.008494+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"33","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"ebbee2c6-023e-44b9-9f1d-6f7bfd309730","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6817","nonce":639807369},{"type":"v1","addr":"172.21.15.143:6819","nonce":639807369}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6821","nonce":639807369},{"type":"v1","addr":"172.21.15.143:6823","nonce":639807369}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6830","nonce":639807369},{"type":"v1","addr":"172.21.15.143:6831","nonce":639807369}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6825","nonce":639807369},{"type":"v1","addr":"172.21.15.143:6828","nonce":639807369}]},"public_addr":"172.21.15.143:6819/639807369","cluster_addr":"172.21.15.143:6823/639807369","heartbeat_back_addr":"172.21.15.143:6831/639807369","heartbeat_front_addr":"172.21.15.143:6828/639807369","state":["exists","up"]},{"osd":1,"uuid":"fec4b8be-7122-49e9-9a19-5850a0700001","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6816","nonce":1144942530},{"type":"v1","addr":"172.21.15.143:6818","nonce":1144942530}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6820","nonce":1144942530},{"type":"v1","addr":"172.21.15.143:6822","nonce":1144942530}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6827","nonce":1144942530},{"type":"v1","addr":"172.21.15.143:6829","nonce":1144942530}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6824","nonce":1144942530},{"type":"v1","addr":"172.21.15.143:6826","nonce":1144942530}]},"public_addr":"172.21.15.143:6818/1144942530","cluster_addr":"172.21.15.143:6822/1144942530","heartbeat_back_addr":"172.21.15.143:6829/1144942530","heartbeat_front_addr":"172.21.15.143:6826/1144942530","state":["exists","up"]},{"osd":2,"uuid":"2888af5b-f933-46ca-8f95-6dd494ff43f6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6808","nonce":3909095186},{"type":"v1","addr":"172.21.15.143:6809","nonce":3909095186}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6810","nonce":3909095186},{"type":"v1","addr":"172.21.15.143:6811","nonce":3909095186}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6814","nonce":3909095186},{"type":"v1","addr":"172.21.15.143:6815","nonce":3909095186}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6812","nonce":3909095186},{"type":"v1","addr":"172.21.15.143:6813","nonce":3909095186}]},"public_addr":"172.21.15.143:6809/3909095186","cluster_addr":"172.21.15.143:6811/3909095186","heartbeat_back_addr":"172.21.15.143:6815/3909095186","heartbeat_front_addr":"172.21.15.143:6813/3909095186","state":["exists","up"]},{"osd":3,"uuid":"d1f52435-7bc1-4081-9550-9cd1ba55f549","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6800","nonce":3854210991},{"type":"v1","addr":"172.21.15.143:6801","nonce":3854210991}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6802","nonce":3854210991},{"type":"v1","addr":"172.21.15.143:6803","nonce":3854210991}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6806","nonce":3854210991},{"type":"v1","addr":"172.21.15.143:6807","nonce":3854210991}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6804","nonce":3854210991},{"type":"v1","addr":"172.21.15.143:6805","nonce":3854210991}]},"public_addr":"172.21.15.143:6801/3854210991","cluster_addr":"172.21.15.143:6803/3854210991","heartbeat_back_addr":"172.21.15.143:6807/3854210991","heartbeat_front_addr":"172.21.15.143:6805/3854210991","state":["exists","up"]},{"osd":4,"uuid":"e2ae8b6e-6e25-49b2-bcbf-aacce68f06a3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6800","nonce":3924606902},{"type":"v1","addr":"172.21.15.188:6801","nonce":3924606902}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6802","nonce":3924606902},{"type":"v1","addr":"172.21.15.188:6803","nonce":3924606902}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6806","nonce":3924606902},{"type":"v1","addr":"172.21.15.188:6807","nonce":3924606902}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6804","nonce":3924606902},{"type":"v1","addr":"172.21.15.188:6805","nonce":3924606902}]},"public_addr":"172.21.15.188:6801/3924606902","cluster_addr":"172.21.15.188:6803/3924606902","heartbeat_back_addr":"172.21.15.188:6807/3924606902","heartbeat_front_addr":"172.21.15.188:6805/3924606902","state":["exists","up"]},{"osd":5,"uuid":"3cec8a32-7698-4c36-8e89-e4ba11934f33","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6813","nonce":186841544},{"type":"v1","addr":"172.21.15.188:6815","nonce":186841544}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6817","nonce":186841544},{"type":"v1","addr":"172.21.15.188:6818","nonce":186841544}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6822","nonce":186841544},{"type":"v1","addr":"172.21.15.188:6823","nonce":186841544}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6820","nonce":186841544},{"type":"v1","addr":"172.21.15.188:6821","nonce":186841544}]},"public_addr":"172.21.15.188:6815/186841544","cluster_addr":"172.21.15.188:6818/186841544","heartbeat_back_addr":"172.21.15.188:6823/186841544","heartbeat_front_addr":"172.21.15.188:6821/186841544","state":["exists","up"]},{"osd":6,"uuid":"58a65d5d-e090-473a-81b4-ad38e85029f9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6808","nonce":2153499405},{"type":"v1","addr":"172.21.15.188:6809","nonce":2153499405}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6810","nonce":2153499405},{"type":"v1","addr":"172.21.15.188:6811","nonce":2153499405}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6816","nonce":2153499405},{"type":"v1","addr":"172.21.15.188:6819","nonce":2153499405}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6812","nonce":2153499405},{"type":"v1","addr":"172.21.15.188:6814","nonce":2153499405}]},"public_addr":"172.21.15.188:6809/2153499405","cluster_addr":"172.21.15.188:6811/2153499405","heartbeat_back_addr":"172.21.15.188:6819/2153499405","heartbeat_front_addr":"172.21.15.188:6814/2153499405","state":["exists","up"]},{"osd":7,"uuid":"ea5b7f70-3554-4cf8-ac30-f7c59b80678d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6824","nonce":2722096729},{"type":"v1","addr":"172.21.15.188:6825","nonce":2722096729}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6826","nonce":2722096729},{"type":"v1","addr":"172.21.15.188:6827","nonce":2722096729}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6830","nonce":2722096729},{"type":"v1","addr":"172.21.15.188:6831","nonce":2722096729}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6828","nonce":2722096729},{"type":"v1","addr":"172.21.15.188:6829","nonce":2722096729}]},"public_addr":"172.21.15.188:6825/2722096729","cluster_addr":"172.21.15.188:6827/2722096729","heartbeat_back_addr":"172.21.15.188:6831/2722096729","heartbeat_front_addr":"172.21.15.188:6829/2722096729","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-26T07:17:36.995032+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-26T07:17:36.932266+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-26T07:17:36.829304+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-26T07:17:36.804297+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-26T07:17:36.667892+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-26T07:17:36.964868+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-26T07:17:36.827332+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-26T07:17:36.893179+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-11-26T07:19:18.716 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:18.712+0000 7f5c599d0640 1 -- 172.21.15.143:0/1398491151 >> [v2:172.21.15.143:6832/2150557544,v1:172.21.15.143:6833/2150557544] conn(0x7f5c38066620 msgr2=0x7f5c38068ae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:19:18.716 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:18.712+0000 7f5c599d0640 1 --2- 172.21.15.143:0/1398491151 >> [v2:172.21.15.143:6832/2150557544,v1:172.21.15.143:6833/2150557544] conn(0x7f5c38066620 0x7f5c38068ae0 secure :-1 s=READY pgs=259 cs=0 l=1 rev1=1 crypto rx=0x7f5c480ad000 tx=0x7f5c48008040 comp rx=0 tx=0).stop 2023-11-26T07:19:18.716 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:18.712+0000 7f5c599d0640 1 -- 172.21.15.143:0/1398491151 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f5c54143810 msgr2=0x7f5c5415cdd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:19:18.716 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:18.712+0000 7f5c599d0640 1 --2- 172.21.15.143:0/1398491151 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f5c54143810 0x7f5c5415cdd0 secure :-1 s=READY pgs=505 cs=0 l=1 rev1=1 crypto rx=0x7f5c4000c730 tx=0x7f5c4000cbf0 comp rx=0 tx=0).stop 2023-11-26T07:19:18.716 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:18.712+0000 7f5c599d0640 1 -- 172.21.15.143:0/1398491151 shutdown_connections 2023-11-26T07:19:18.716 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:18.712+0000 7f5c599d0640 1 --2- 172.21.15.143:0/1398491151 >> [v2:172.21.15.143:6832/2150557544,v1:172.21.15.143:6833/2150557544] conn(0x7f5c38066620 0x7f5c38068ae0 unknown :-1 s=CLOSED pgs=259 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:19:18.716 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:18.712+0000 7f5c599d0640 1 --2- 172.21.15.143:0/1398491151 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f5c5415e0f0 0x7f5c540b0db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:19:18.716 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:18.712+0000 7f5c599d0640 1 --2- 172.21.15.143:0/1398491151 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f5c54143810 0x7f5c5415cdd0 unknown :-1 s=CLOSED pgs=505 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:19:18.716 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:18.712+0000 7f5c599d0640 1 --2- 172.21.15.143:0/1398491151 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f5c540b1e10 0x7f5c541f6000 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:19:18.717 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:18.712+0000 7f5c599d0640 1 -- 172.21.15.143:0/1398491151 >> 172.21.15.143:0/1398491151 conn(0x7f5c5406a890 msgr2=0x7f5c540b2d40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:19:18.717 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:18.712+0000 7f5c599d0640 1 -- 172.21.15.143:0/1398491151 shutdown_connections 2023-11-26T07:19:18.717 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:18.712+0000 7f5c599d0640 1 -- 172.21.15.143:0/1398491151 wait complete. 2023-11-26T07:19:18.736 DEBUG:teuthology.orchestra.run.smithi143:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs fail cephfs 2023-11-26T07:19:18.875 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:18.872+0000 7f31970aa640 1 Processor -- start 2023-11-26T07:19:18.875 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:18.872+0000 7f31970aa640 1 -- start start 2023-11-26T07:19:18.876 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:18.872+0000 7f31970aa640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f31900cf990 0x7f319015efa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:19:18.877 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:18.872+0000 7f31970aa640 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7f319015f4e0 0x7f31900cd7b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:19:18.877 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:18.872+0000 7f31970aa640 1 -- --> v1:172.21.15.188:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f319006c520 con 0x7f319013e290 2023-11-26T07:19:18.877 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:18.872+0000 7f31970aa640 1 -- --> v2:172.21.15.143:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f319006c800 con 0x7f319015f4e0 2023-11-26T07:19:18.877 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:18.872+0000 7f31970aa640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f319006cae0 con 0x7f31900cf990 2023-11-26T07:19:18.877 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:18.872+0000 7f3195647640 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7f319015f4e0 0x7f31900cd7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:19:18.877 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:18.876+0000 7f3195647640 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7f319015f4e0 0x7f31900cd7b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:48050/0 (socket says 172.21.15.143:48050) 2023-11-26T07:19:18.877 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:18.876+0000 7f3195647640 1 -- 172.21.15.143:0/2459679656 learned_addr learned my addr 172.21.15.143:0/2459679656 (peer_addr_for_me v2:172.21.15.143:0/0) 2023-11-26T07:19:18.877 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:18.876+0000 7f318ffff640 1 --2- 172.21.15.143:0/2459679656 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f31900cf990 0x7f319015efa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:19:18.878 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:18.876+0000 7f3195647640 1 -- 172.21.15.143:0/2459679656 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f31900cf990 msgr2=0x7f319015efa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-26T07:19:18.878 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:18.876+0000 7f3195647640 1 --2- 172.21.15.143:0/2459679656 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f31900cf990 0x7f319015efa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:19:18.878 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:18.876+0000 7f3195647640 1 -- 172.21.15.143:0/2459679656 >> v1:172.21.15.188:6789/0 conn(0x7f319013e290 legacy=0x7f31900cf280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:19:18.878 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:18.876+0000 7f3195647640 1 -- 172.21.15.143:0/2459679656 --> v2:172.21.15.143:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f319006cdc0 con 0x7f319015f4e0 2023-11-26T07:19:18.880 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:18.876+0000 7f3195647640 1 --2- 172.21.15.143:0/2459679656 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f319015f4e0 0x7f31900cd7b0 secure :-1 s=READY pgs=506 cs=0 l=1 rev1=1 crypto rx=0x7f3188009e50 tx=0x7f318801c9e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=342f92c56ddc366b server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:19:18.880 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:18.876+0000 7f318f7fe640 1 -- 172.21.15.143:0/2459679656 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f31880a58d0 con 0x7f319015f4e0 2023-11-26T07:19:18.880 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:18.876+0000 7f318f7fe640 1 -- 172.21.15.143:0/2459679656 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f31880a5a30 con 0x7f319015f4e0 2023-11-26T07:19:18.880 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:18.876+0000 7f318f7fe640 1 -- 172.21.15.143:0/2459679656 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f31880a5dc0 con 0x7f319015f4e0 2023-11-26T07:19:18.881 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:18.880+0000 7f31970aa640 1 -- 172.21.15.143:0/2459679656 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f319015f4e0 msgr2=0x7f31900cd7b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:19:18.881 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:18.880+0000 7f31970aa640 1 --2- 172.21.15.143:0/2459679656 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f319015f4e0 0x7f31900cd7b0 secure :-1 s=READY pgs=506 cs=0 l=1 rev1=1 crypto rx=0x7f3188009e50 tx=0x7f318801c9e0 comp rx=0 tx=0).stop 2023-11-26T07:19:18.881 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:18.880+0000 7f31970aa640 1 -- 172.21.15.143:0/2459679656 shutdown_connections 2023-11-26T07:19:18.881 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:18.880+0000 7f31970aa640 1 --2- 172.21.15.143:0/2459679656 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f319015f4e0 0x7f31900cd7b0 unknown :-1 s=CLOSED pgs=506 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:19:18.882 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:18.880+0000 7f31970aa640 1 --2- 172.21.15.143:0/2459679656 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f31900cf990 0x7f319015efa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:19:18.882 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:18.880+0000 7f31970aa640 1 -- 172.21.15.143:0/2459679656 >> 172.21.15.143:0/2459679656 conn(0x7f319006a890 msgr2=0x7f319006aca0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:19:18.882 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:18.880+0000 7f31970aa640 1 -- 172.21.15.143:0/2459679656 shutdown_connections 2023-11-26T07:19:18.882 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:18.880+0000 7f31970aa640 1 -- 172.21.15.143:0/2459679656 wait complete. 2023-11-26T07:19:18.882 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:18.880+0000 7f31970aa640 1 Processor -- start 2023-11-26T07:19:18.882 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:18.880+0000 7f31970aa640 1 -- start start 2023-11-26T07:19:18.885 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:18.880+0000 7f31970aa640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f31900cf990 0x7f319015dc00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:19:18.885 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:18.880+0000 7f31970aa640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f319013e290 0x7f319015a3b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:19:18.885 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:18.880+0000 7f31970aa640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f319015f4e0 0x7f3190136690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:19:18.886 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:18.880+0000 7f31970aa640 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3190070370 con 0x7f319015f4e0 2023-11-26T07:19:18.886 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:18.880+0000 7f31970aa640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3190070930 con 0x7f31900cf990 2023-11-26T07:19:18.886 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:18.880+0000 7f31970aa640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f319015e270 con 0x7f319013e290 2023-11-26T07:19:18.886 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:18.880+0000 7f3195647640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f319015f4e0 0x7f3190136690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:19:18.886 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:18.880+0000 7f3195647640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f319015f4e0 0x7f3190136690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:48058/0 (socket says 172.21.15.143:48058) 2023-11-26T07:19:18.886 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:18.880+0000 7f3195647640 1 -- 172.21.15.143:0/1458595036 learned_addr learned my addr 172.21.15.143:0/1458595036 (peer_addr_for_me v2:172.21.15.143:0/0) 2023-11-26T07:19:18.886 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:18.880+0000 7f318ffff640 1 --2- 172.21.15.143:0/1458595036 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f319013e290 0x7f319015a3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:19:18.886 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:18.880+0000 7f3194e46640 1 --2- 172.21.15.143:0/1458595036 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f31900cf990 0x7f319015dc00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:19:18.886 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:18.880+0000 7f3195647640 1 -- 172.21.15.143:0/1458595036 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f319013e290 msgr2=0x7f319015a3b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:19:18.887 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:18.880+0000 7f3195647640 1 --2- 172.21.15.143:0/1458595036 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f319013e290 0x7f319015a3b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:19:18.887 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:18.880+0000 7f3195647640 1 -- 172.21.15.143:0/1458595036 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f31900cf990 msgr2=0x7f319015dc00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:19:18.887 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:18.880+0000 7f3195647640 1 --2- 172.21.15.143:0/1458595036 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f31900cf990 0x7f319015dc00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:19:18.887 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:18.880+0000 7f3195647640 1 -- 172.21.15.143:0/1458595036 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3188009b30 con 0x7f319015f4e0 2023-11-26T07:19:18.887 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:18.884+0000 7f3195647640 1 --2- 172.21.15.143:0/1458595036 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f319015f4e0 0x7f3190136690 secure :-1 s=READY pgs=507 cs=0 l=1 rev1=1 crypto rx=0x7f3188009e50 tx=0x7f3188019780 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:19:18.887 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:18.884+0000 7f318d7fa640 1 -- 172.21.15.143:0/1458595036 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f31880b8670 con 0x7f319015f4e0 2023-11-26T07:19:18.887 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:18.884+0000 7f318d7fa640 1 -- 172.21.15.143:0/1458595036 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f31880b87f0 con 0x7f319015f4e0 2023-11-26T07:19:18.887 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:18.884+0000 7f318d7fa640 1 -- 172.21.15.143:0/1458595036 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f31880b8ba0 con 0x7f319015f4e0 2023-11-26T07:19:18.887 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:18.884+0000 7f31970aa640 1 -- 172.21.15.143:0/1458595036 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f319015e550 con 0x7f319015f4e0 2023-11-26T07:19:18.888 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:18.884+0000 7f31970aa640 1 -- 172.21.15.143:0/1458595036 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3190137920 con 0x7f319015f4e0 2023-11-26T07:19:18.888 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:18.884+0000 7f31970aa640 1 -- 172.21.15.143:0/1458595036 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3190070ac0 con 0x7f319015f4e0 2023-11-26T07:19:18.889 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:18.884+0000 7f318d7fa640 1 -- 172.21.15.143:0/1458595036 <== mon.0 v2:172.21.15.143:3300/0 4 ==== mgrmap(e 7) v1 ==== 86005+0+0 (secure 0 0 0) 0x7f31880b8d00 con 0x7f319015f4e0 2023-11-26T07:19:18.889 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:18.884+0000 7f318d7fa640 1 --2- 172.21.15.143:0/1458595036 >> [v2:172.21.15.143:6832/2150557544,v1:172.21.15.143:6833/2150557544] conn(0x7f3170066650 0x7f3170068b10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:19:18.889 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:18.884+0000 7f318d7fa640 1 -- 172.21.15.143:0/1458595036 <== mon.0 v2:172.21.15.143:3300/0 5 ==== osd_map(33..33 src has 1..33) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f318811eeb0 con 0x7f319015f4e0 2023-11-26T07:19:18.889 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:18.884+0000 7f3194e46640 1 --2- 172.21.15.143:0/1458595036 >> [v2:172.21.15.143:6832/2150557544,v1:172.21.15.143:6833/2150557544] conn(0x7f3170066650 0x7f3170068b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:19:18.889 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:18.888+0000 7f3194e46640 1 --2- 172.21.15.143:0/1458595036 >> [v2:172.21.15.143:6832/2150557544,v1:172.21.15.143:6833/2150557544] conn(0x7f3170066650 0x7f3170068b10 secure :-1 s=READY pgs=260 cs=0 l=1 rev1=1 crypto rx=0x7f3178001810 tx=0x7f3178009040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:19:18.892 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:18.888+0000 7f318d7fa640 1 -- 172.21.15.143:0/1458595036 <== mon.0 v2:172.21.15.143:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172649 (secure 0 0 0) 0x7f31880ec150 con 0x7f319015f4e0 2023-11-26T07:19:19.073 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:19.068+0000 7f31970aa640 1 -- 172.21.15.143:0/1458595036 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs"} v 0) v1 -- 0x7f3190067680 con 0x7f319015f4e0 2023-11-26T07:19:19.073 INFO:tasks.ceph.mon.a.smithi143.stderr:2023-11-26T07:19:19.072+0000 7f6946f73640 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2023-11-26T07:19:19.077 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:19.076+0000 7f318d7fa640 1 -- 172.21.15.143:0/1458595036 <== mon.0 v2:172.21.15.143:3300/0 7 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs"}]=0 cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v18) v1 ==== 161+0+0 (secure 0 0 0) 0x7f31880ef770 con 0x7f319015f4e0 2023-11-26T07:19:19.077 INFO:teuthology.orchestra.run.smithi143.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2023-11-26T07:19:19.079 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:19.076+0000 7f31970aa640 1 -- 172.21.15.143:0/1458595036 >> [v2:172.21.15.143:6832/2150557544,v1:172.21.15.143:6833/2150557544] conn(0x7f3170066650 msgr2=0x7f3170068b10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:19:19.079 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:19.076+0000 7f31970aa640 1 --2- 172.21.15.143:0/1458595036 >> [v2:172.21.15.143:6832/2150557544,v1:172.21.15.143:6833/2150557544] conn(0x7f3170066650 0x7f3170068b10 secure :-1 s=READY pgs=260 cs=0 l=1 rev1=1 crypto rx=0x7f3178001810 tx=0x7f3178009040 comp rx=0 tx=0).stop 2023-11-26T07:19:19.079 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:19.076+0000 7f31970aa640 1 -- 172.21.15.143:0/1458595036 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f319015f4e0 msgr2=0x7f3190136690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:19:19.079 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:19.076+0000 7f31970aa640 1 --2- 172.21.15.143:0/1458595036 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f319015f4e0 0x7f3190136690 secure :-1 s=READY pgs=507 cs=0 l=1 rev1=1 crypto rx=0x7f3188009e50 tx=0x7f3188019780 comp rx=0 tx=0).stop 2023-11-26T07:19:19.079 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:19.076+0000 7f31970aa640 1 -- 172.21.15.143:0/1458595036 shutdown_connections 2023-11-26T07:19:19.079 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:19.076+0000 7f31970aa640 1 --2- 172.21.15.143:0/1458595036 >> [v2:172.21.15.143:6832/2150557544,v1:172.21.15.143:6833/2150557544] conn(0x7f3170066650 0x7f3170068b10 unknown :-1 s=CLOSED pgs=260 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:19:19.080 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:19.076+0000 7f31970aa640 1 --2- 172.21.15.143:0/1458595036 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f319015f4e0 0x7f3190136690 unknown :-1 s=CLOSED pgs=507 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:19:19.080 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:19.076+0000 7f31970aa640 1 --2- 172.21.15.143:0/1458595036 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f319013e290 0x7f319015a3b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:19:19.080 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:19.076+0000 7f31970aa640 1 --2- 172.21.15.143:0/1458595036 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f31900cf990 0x7f319015dc00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:19:19.080 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:19.076+0000 7f31970aa640 1 -- 172.21.15.143:0/1458595036 >> 172.21.15.143:0/1458595036 conn(0x7f319006a890 msgr2=0x7f319013b140 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:19:19.080 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:19.076+0000 7f31970aa640 1 -- 172.21.15.143:0/1458595036 shutdown_connections 2023-11-26T07:19:19.080 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:19.076+0000 7f31970aa640 1 -- 172.21.15.143:0/1458595036 wait complete. 2023-11-26T07:19:19.101 DEBUG:teuthology.orchestra.run.smithi143:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs rm cephfs --yes-i-really-mean-it 2023-11-26T07:19:19.148 INFO:tasks.ceph.mds.c.smithi143.stdout:starting mds.c at 2023-11-26T07:19:19.244 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:19.240+0000 7fd9bd71d640 1 Processor -- start 2023-11-26T07:19:19.244 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:19.240+0000 7fd9bd71d640 1 -- start start 2023-11-26T07:19:19.245 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:19.244+0000 7fd9bd71d640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fd9b80d17f0 0x7fd9b80d1c30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:19:19.245 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:19.244+0000 7fd9bd71d640 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7fd9b80b0490 0x7fd9b80b0920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:19:19.245 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:19.244+0000 7fd9bd71d640 1 -- --> v1:172.21.15.143:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd9b80d0bc0 con 0x7fd9b815d580 2023-11-26T07:19:19.245 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:19.244+0000 7fd9bd71d640 1 -- --> v2:172.21.15.188:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd9b80d0320 con 0x7fd9b80b0490 2023-11-26T07:19:19.245 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:19.244+0000 7fd9bd71d640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd9b80d0600 con 0x7fd9b80d17f0 2023-11-26T07:19:19.246 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:19.244+0000 7fd9b6ffd640 1 --1- >> v1:172.21.15.143:6789/0 conn(0x7fd9b815d580 0x7fd9b80d3920 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.143:6789/0 says I am v1:172.21.15.143:50158/0 (socket says 172.21.15.143:50158) 2023-11-26T07:19:19.246 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:19.244+0000 7fd9b6ffd640 1 -- 172.21.15.143:0/2137162068 learned_addr learned my addr 172.21.15.143:0/2137162068 (peer_addr_for_me v1:172.21.15.143:0/0) 2023-11-26T07:19:19.247 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:19.244+0000 7fd9ae7fc640 1 --2- 172.21.15.143:0/2137162068 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fd9b80d17f0 0x7fd9b80d1c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:19:19.247 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:19.244+0000 7fd9b77fe640 1 --2- 172.21.15.143:0/2137162068 >> v2:172.21.15.188:3300/0 conn(0x7fd9b80b0490 0x7fd9b80b0920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:19:19.247 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:19.244+0000 7fd9b67fc640 1 -- 172.21.15.143:0/2137162068 <== mon.0 v1:172.21.15.143:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2366084115 0 0) 0x7fd9b80d0bc0 con 0x7fd9b815d580 2023-11-26T07:19:19.247 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:19.244+0000 7fd9b67fc640 1 -- 172.21.15.143:0/2137162068 --> v1:172.21.15.143:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fd9a4003530 con 0x7fd9b815d580 2023-11-26T07:19:19.247 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:19.244+0000 7fd9b67fc640 1 -- 172.21.15.143:0/2137162068 <== mon.0 v1:172.21.15.143:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1919035086 0 0) 0x7fd9a4003530 con 0x7fd9b815d580 2023-11-26T07:19:19.247 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:19.244+0000 7fd9ae7fc640 1 -- 172.21.15.143:0/2137162068 >> v2:172.21.15.188:3300/0 conn(0x7fd9b80b0490 msgr2=0x7fd9b80b0920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-26T07:19:19.247 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:19.244+0000 7fd9ae7fc640 1 --2- 172.21.15.143:0/2137162068 >> v2:172.21.15.188:3300/0 conn(0x7fd9b80b0490 0x7fd9b80b0920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:19:19.247 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:19.244+0000 7fd9ae7fc640 1 -- 172.21.15.143:0/2137162068 >> v1:172.21.15.143:6789/0 conn(0x7fd9b815d580 legacy=0x7fd9b80d3920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:19:19.247 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:19.244+0000 7fd9ae7fc640 1 -- 172.21.15.143:0/2137162068 --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd9b80d08e0 con 0x7fd9b80d17f0 2023-11-26T07:19:19.247 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:19.244+0000 7fd9b77fe640 1 --2- 172.21.15.143:0/2137162068 >> v2:172.21.15.188:3300/0 conn(0x7fd9b80b0490 0x7fd9b80b0920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-26T07:19:19.249 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:19.248+0000 7fd9ae7fc640 1 --2- 172.21.15.143:0/2137162068 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fd9b80d17f0 0x7fd9b80d1c30 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7fd9a0009d90 tx=0x7fd9a001c840 comp rx=0 tx=0).ready entity=mon.2 client_cookie=2278e189d25b34e7 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:19:19.249 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:19.248+0000 7fd9b67fc640 1 -- 172.21.15.143:0/2137162068 <== mon.2 v2:172.21.15.188:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd9a8001680 con 0x7fd9b80d17f0 2023-11-26T07:19:19.249 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:19.248+0000 7fd9b67fc640 1 -- 172.21.15.143:0/2137162068 <== mon.2 v2:172.21.15.188:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd9a00a5620 con 0x7fd9b80d17f0 2023-11-26T07:19:19.250 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:19.248+0000 7fd9b67fc640 1 -- 172.21.15.143:0/2137162068 <== mon.2 v2:172.21.15.188:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd9a00a59b0 con 0x7fd9b80d17f0 2023-11-26T07:19:19.250 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:19.248+0000 7fd9bd71d640 1 -- 172.21.15.143:0/2137162068 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fd9b80d17f0 msgr2=0x7fd9b80d1c30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:19:19.250 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:19.248+0000 7fd9bd71d640 1 --2- 172.21.15.143:0/2137162068 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fd9b80d17f0 0x7fd9b80d1c30 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7fd9a0009d90 tx=0x7fd9a001c840 comp rx=0 tx=0).stop 2023-11-26T07:19:19.251 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:19.248+0000 7fd9bd71d640 1 -- 172.21.15.143:0/2137162068 shutdown_connections 2023-11-26T07:19:19.251 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:19.248+0000 7fd9bd71d640 1 --2- 172.21.15.143:0/2137162068 >> v2:172.21.15.188:3300/0 conn(0x7fd9b80b0490 0x7fd9b80b0920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:19:19.251 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:19.248+0000 7fd9bd71d640 1 --2- 172.21.15.143:0/2137162068 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fd9b80d17f0 0x7fd9b80d1c30 unknown :-1 s=CLOSED pgs=79 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:19:19.251 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:19.248+0000 7fd9bd71d640 1 -- 172.21.15.143:0/2137162068 >> 172.21.15.143:0/2137162068 conn(0x7fd9b809c550 msgr2=0x7fd9b80d3210 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:19:19.251 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:19.248+0000 7fd9bd71d640 1 -- 172.21.15.143:0/2137162068 shutdown_connections 2023-11-26T07:19:19.251 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:19.248+0000 7fd9bd71d640 1 -- 172.21.15.143:0/2137162068 wait complete. 2023-11-26T07:19:19.251 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:19.248+0000 7fd9bd71d640 1 Processor -- start 2023-11-26T07:19:19.252 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:19.248+0000 7fd9bd71d640 1 -- start start 2023-11-26T07:19:19.254 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:19.252+0000 7fd9bd71d640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fd9b80b0490 0x7fd9b806b1a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:19:19.254 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:19.252+0000 7fd9bd71d640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fd9b80d17f0 0x7fd9b806b6e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:19:19.254 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:19.252+0000 7fd9bd71d640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fd9b815d580 0x7fd9b8155830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:19:19.254 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:19.252+0000 7fd9bd71d640 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd9b815b590 con 0x7fd9b80d17f0 2023-11-26T07:19:19.254 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:19.252+0000 7fd9bd71d640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd9b80b9000 con 0x7fd9b80b0490 2023-11-26T07:19:19.254 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:19.252+0000 7fd9bd71d640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd9b80ca2e0 con 0x7fd9b815d580 2023-11-26T07:19:19.254 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:19.252+0000 7fd9ae7fc640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fd9b80d17f0 0x7fd9b806b6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:19:19.255 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:19.252+0000 7fd9ae7fc640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fd9b80d17f0 0x7fd9b806b6e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:48086/0 (socket says 172.21.15.143:48086) 2023-11-26T07:19:19.255 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:19.252+0000 7fd9ae7fc640 1 -- 172.21.15.143:0/4284122087 learned_addr learned my addr 172.21.15.143:0/4284122087 (peer_addr_for_me v2:172.21.15.143:0/0) 2023-11-26T07:19:19.255 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:19.252+0000 7fd9b77fe640 1 --2- 172.21.15.143:0/4284122087 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fd9b815d580 0x7fd9b8155830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:19:19.255 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:19.252+0000 7fd9b6ffd640 1 --2- 172.21.15.143:0/4284122087 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fd9b80b0490 0x7fd9b806b1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:19:19.255 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:19.252+0000 7fd9ae7fc640 1 -- 172.21.15.143:0/4284122087 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fd9b815d580 msgr2=0x7fd9b8155830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:19:19.255 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:19.252+0000 7fd9ae7fc640 1 --2- 172.21.15.143:0/4284122087 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fd9b815d580 0x7fd9b8155830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:19:19.255 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:19.252+0000 7fd9ae7fc640 1 -- 172.21.15.143:0/4284122087 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fd9b80b0490 msgr2=0x7fd9b806b1a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:19:19.255 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:19.252+0000 7fd9ae7fc640 1 --2- 172.21.15.143:0/4284122087 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fd9b80b0490 0x7fd9b806b1a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:19:19.255 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:19.252+0000 7fd9ae7fc640 1 -- 172.21.15.143:0/4284122087 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd9b0009420 con 0x7fd9b80d17f0 2023-11-26T07:19:19.256 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:19.252+0000 7fd9b77fe640 1 --2- 172.21.15.143:0/4284122087 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fd9b815d580 0x7fd9b8155830 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-26T07:19:19.256 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:19.252+0000 7fd9b6ffd640 1 --2- 172.21.15.143:0/4284122087 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fd9b80b0490 0x7fd9b806b1a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-26T07:19:19.256 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:19.252+0000 7fd9ae7fc640 1 --2- 172.21.15.143:0/4284122087 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fd9b80d17f0 0x7fd9b806b6e0 secure :-1 s=READY pgs=512 cs=0 l=1 rev1=1 crypto rx=0x7fd9a001aef0 tx=0x7fd9a00195e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:19:19.257 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:19.252+0000 7fd9affff640 1 -- 172.21.15.143:0/4284122087 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd9a00ad850 con 0x7fd9b80d17f0 2023-11-26T07:19:19.257 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:19.252+0000 7fd9affff640 1 -- 172.21.15.143:0/4284122087 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd9a00ad9b0 con 0x7fd9b80d17f0 2023-11-26T07:19:19.257 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:19.252+0000 7fd9bd71d640 1 -- 172.21.15.143:0/4284122087 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd9a0009a70 con 0x7fd9b80d17f0 2023-11-26T07:19:19.257 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:19.252+0000 7fd9affff640 1 -- 172.21.15.143:0/4284122087 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd9a00add60 con 0x7fd9b80d17f0 2023-11-26T07:19:19.257 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:19.252+0000 7fd9bd71d640 1 -- 172.21.15.143:0/4284122087 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd9b815b770 con 0x7fd9b80d17f0 2023-11-26T07:19:19.257 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:19.252+0000 7fd9bd71d640 1 -- 172.21.15.143:0/4284122087 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd9b80b88e0 con 0x7fd9b80d17f0 2023-11-26T07:19:19.258 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:19.252+0000 7fd9affff640 1 -- 172.21.15.143:0/4284122087 <== mon.0 v2:172.21.15.143:3300/0 4 ==== mgrmap(e 7) v1 ==== 86005+0+0 (secure 0 0 0) 0x7fd9a00b4050 con 0x7fd9b80d17f0 2023-11-26T07:19:19.258 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:19.256+0000 7fd9affff640 1 --2- 172.21.15.143:0/4284122087 >> [v2:172.21.15.143:6832/2150557544,v1:172.21.15.143:6833/2150557544] conn(0x7fd998066620 0x7fd998068ae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:19:19.258 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:19.256+0000 7fd9affff640 1 -- 172.21.15.143:0/4284122087 <== mon.0 v2:172.21.15.143:3300/0 5 ==== osd_map(34..34 src has 1..34) v4 ==== 5795+0+0 (secure 0 0 0) 0x7fd9a011f3e0 con 0x7fd9b80d17f0 2023-11-26T07:19:19.258 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:19.256+0000 7fd9b6ffd640 1 --2- 172.21.15.143:0/4284122087 >> [v2:172.21.15.143:6832/2150557544,v1:172.21.15.143:6833/2150557544] conn(0x7fd998066620 0x7fd998068ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:19:19.258 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:19.256+0000 7fd9b6ffd640 1 --2- 172.21.15.143:0/4284122087 >> [v2:172.21.15.143:6832/2150557544,v1:172.21.15.143:6833/2150557544] conn(0x7fd998066620 0x7fd998068ae0 secure :-1 s=READY pgs=261 cs=0 l=1 rev1=1 crypto rx=0x7fd9a8005890 tx=0x7fd9a8009040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:19:19.261 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:19.260+0000 7fd9affff640 1 -- 172.21.15.143:0/4284122087 <== mon.0 v2:172.21.15.143:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172649 (secure 0 0 0) 0x7fd9b80b88e0 con 0x7fd9b80d17f0 2023-11-26T07:19:19.441 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:19.440+0000 7fd9bd71d640 1 -- 172.21.15.143:0/4284122087 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true} v 0) v1 -- 0x7fd9b815f750 con 0x7fd9b80d17f0 2023-11-26T07:19:20.079 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:20.076+0000 7fd9affff640 1 -- 172.21.15.143:0/4284122087 <== mon.0 v2:172.21.15.143:3300/0 7 ==== mon_command_ack([{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]=0 v19) v1 ==== 104+0+0 (secure 0 0 0) 0x7fd9b815f750 con 0x7fd9b80d17f0 2023-11-26T07:19:20.082 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:20.080+0000 7fd9bd71d640 1 -- 172.21.15.143:0/4284122087 >> [v2:172.21.15.143:6832/2150557544,v1:172.21.15.143:6833/2150557544] conn(0x7fd998066620 msgr2=0x7fd998068ae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:19:20.082 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:20.080+0000 7fd9bd71d640 1 --2- 172.21.15.143:0/4284122087 >> [v2:172.21.15.143:6832/2150557544,v1:172.21.15.143:6833/2150557544] conn(0x7fd998066620 0x7fd998068ae0 secure :-1 s=READY pgs=261 cs=0 l=1 rev1=1 crypto rx=0x7fd9a8005890 tx=0x7fd9a8009040 comp rx=0 tx=0).stop 2023-11-26T07:19:20.083 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:20.080+0000 7fd9bd71d640 1 -- 172.21.15.143:0/4284122087 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fd9b80d17f0 msgr2=0x7fd9b806b6e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:19:20.083 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:20.080+0000 7fd9bd71d640 1 --2- 172.21.15.143:0/4284122087 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fd9b80d17f0 0x7fd9b806b6e0 secure :-1 s=READY pgs=512 cs=0 l=1 rev1=1 crypto rx=0x7fd9a001aef0 tx=0x7fd9a00195e0 comp rx=0 tx=0).stop 2023-11-26T07:19:20.083 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:20.080+0000 7fd9bd71d640 1 -- 172.21.15.143:0/4284122087 shutdown_connections 2023-11-26T07:19:20.083 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:20.080+0000 7fd9bd71d640 1 --2- 172.21.15.143:0/4284122087 >> [v2:172.21.15.143:6832/2150557544,v1:172.21.15.143:6833/2150557544] conn(0x7fd998066620 0x7fd998068ae0 unknown :-1 s=CLOSED pgs=261 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:19:20.083 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:20.080+0000 7fd9bd71d640 1 --2- 172.21.15.143:0/4284122087 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fd9b815d580 0x7fd9b8155830 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:19:20.083 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:20.080+0000 7fd9bd71d640 1 --2- 172.21.15.143:0/4284122087 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fd9b80d17f0 0x7fd9b806b6e0 unknown :-1 s=CLOSED pgs=512 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:19:20.083 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:20.080+0000 7fd9bd71d640 1 --2- 172.21.15.143:0/4284122087 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fd9b80b0490 0x7fd9b806b1a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:19:20.083 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:20.080+0000 7fd9bd71d640 1 -- 172.21.15.143:0/4284122087 >> 172.21.15.143:0/4284122087 conn(0x7fd9b809c550 msgr2=0x7fd9b80d0b20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:19:20.083 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:20.080+0000 7fd9bd71d640 1 -- 172.21.15.143:0/4284122087 shutdown_connections 2023-11-26T07:19:20.083 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:20.080+0000 7fd9bd71d640 1 -- 172.21.15.143:0/4284122087 wait complete. 2023-11-26T07:19:20.103 DEBUG:teuthology.orchestra.run.smithi143:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_metadata cephfs_metadata --yes-i-really-really-mean-it 2023-11-26T07:19:20.285 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:20.284+0000 7f7664afe640 1 Processor -- start 2023-11-26T07:19:20.285 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:20.284+0000 7f7664afe640 1 -- start start 2023-11-26T07:19:20.286 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:20.284+0000 7f7664afe640 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7f76600cf710 0x7f76600cd8c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:19:20.287 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:20.284+0000 7f7664afe640 1 -- --> v1:172.21.15.143:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f76600cc730 con 0x7f76600cf280 2023-11-26T07:19:20.287 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:20.284+0000 7f7664afe640 1 -- --> v1:172.21.15.188:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7660070ee0 con 0x7f7660143ac0 2023-11-26T07:19:20.287 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:20.284+0000 7f7664afe640 1 -- --> v2:172.21.15.188:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7660143ea0 con 0x7f76600cf710 2023-11-26T07:19:20.287 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:20.284+0000 7f7657fff640 1 --1- >> v1:172.21.15.143:6789/0 conn(0x7f76600cf280 0x7f76600cd1b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.143:6789/0 says I am v1:172.21.15.143:50164/0 (socket says 172.21.15.143:50164) 2023-11-26T07:19:20.287 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:20.284+0000 7f7657fff640 1 -- 172.21.15.143:0/529820816 learned_addr learned my addr 172.21.15.143:0/529820816 (peer_addr_for_me v1:172.21.15.143:0/0) 2023-11-26T07:19:20.289 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:20.284+0000 7f765ed9d640 1 --2- 172.21.15.143:0/529820816 >> v2:172.21.15.188:3300/0 conn(0x7f76600cf710 0x7f76600cd8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:19:20.289 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:20.284+0000 7f765dd9b640 1 -- 172.21.15.143:0/529820816 <== mon.0 v1:172.21.15.143:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4024374947 0 0) 0x7f76600cc730 con 0x7f76600cf280 2023-11-26T07:19:20.289 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:20.284+0000 7f765dd9b640 1 -- 172.21.15.143:0/529820816 --> v1:172.21.15.143:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f764c003530 con 0x7f76600cf280 2023-11-26T07:19:20.289 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:20.284+0000 7f765dd9b640 1 -- 172.21.15.143:0/529820816 <== mon.0 v1:172.21.15.143:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2727637597 0 0) 0x7f764c003530 con 0x7f76600cf280 2023-11-26T07:19:20.289 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:20.284+0000 7f765dd9b640 1 -- 172.21.15.143:0/529820816 >> v2:172.21.15.188:3300/0 conn(0x7f76600cf710 msgr2=0x7f76600cd8c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-26T07:19:20.289 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:20.284+0000 7f765dd9b640 1 --2- 172.21.15.143:0/529820816 >> v2:172.21.15.188:3300/0 conn(0x7f76600cf710 0x7f76600cd8c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:19:20.289 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:20.284+0000 7f765dd9b640 1 -- 172.21.15.143:0/529820816 >> v1:172.21.15.188:6789/0 conn(0x7f7660143ac0 legacy=0x7f7660070370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:19:20.289 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:20.284+0000 7f765dd9b640 1 -- 172.21.15.143:0/529820816 --> v1:172.21.15.143:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f76600711c0 con 0x7f76600cf280 2023-11-26T07:19:20.290 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:20.284+0000 7f765dd9b640 1 -- 172.21.15.143:0/529820816 <== mon.0 v1:172.21.15.143:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2137882523 0 0) 0x7f7648001680 con 0x7f76600cf280 2023-11-26T07:19:20.290 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:20.284+0000 7f765dd9b640 1 -- 172.21.15.143:0/529820816 >> v1:172.21.15.143:6789/0 conn(0x7f76600cf280 legacy=0x7f76600cd1b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:19:20.290 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:20.284+0000 7f765dd9b640 1 --2- 172.21.15.143:0/529820816 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f764c0039d0 0x7f764c003df0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:19:20.290 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:20.284+0000 7f765dd9b640 1 --2- 172.21.15.143:0/529820816 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f764c0047d0 0x7f764c004c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:19:20.290 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:20.284+0000 7f765dd9b640 1 --2- 172.21.15.143:0/529820816 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f764c005190 0x7f764c00b990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:19:20.290 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:20.284+0000 7f765dd9b640 1 -- 172.21.15.143:0/529820816 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f764c00c060 con 0x7f764c0047d0 2023-11-26T07:19:20.290 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:20.284+0000 7f765dd9b640 1 -- 172.21.15.143:0/529820816 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f764c00c1d0 con 0x7f764c005190 2023-11-26T07:19:20.290 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:20.284+0000 7f765dd9b640 1 -- 172.21.15.143:0/529820816 --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f764c00c390 con 0x7f764c0039d0 2023-11-26T07:19:20.290 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:20.284+0000 7f7657fff640 1 --2- 172.21.15.143:0/529820816 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f764c0047d0 0x7f764c004c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:19:20.290 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:20.284+0000 7f765ed9d640 1 --2- 172.21.15.143:0/529820816 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f764c005190 0x7f764c00b990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:19:20.290 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:20.284+0000 7f765e59c640 1 --2- 172.21.15.143:0/529820816 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f764c0039d0 0x7f764c003df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:19:20.290 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:20.284+0000 7f7657fff640 1 -- 172.21.15.143:0/529820816 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f764c0039d0 msgr2=0x7f764c003df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-26T07:19:20.291 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:20.284+0000 7f7657fff640 1 --2- 172.21.15.143:0/529820816 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f764c0039d0 0x7f764c003df0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:19:20.291 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:20.284+0000 7f7657fff640 1 -- 172.21.15.143:0/529820816 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f764c005190 msgr2=0x7f764c00b990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-26T07:19:20.291 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:20.284+0000 7f7657fff640 1 --2- 172.21.15.143:0/529820816 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f764c005190 0x7f764c00b990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:19:20.291 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:20.284+0000 7f7657fff640 1 -- 172.21.15.143:0/529820816 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f764c003530 con 0x7f764c0047d0 2023-11-26T07:19:20.291 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:20.284+0000 7f765ed9d640 1 -- 172.21.15.143:0/529820816 reap_dead start 2023-11-26T07:19:20.292 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:20.288+0000 7f7657fff640 1 --2- 172.21.15.143:0/529820816 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f764c0047d0 0x7f764c004c50 secure :-1 s=READY pgs=514 cs=0 l=1 rev1=1 crypto rx=0x7f764800c0e0 tx=0x7f764801f290 comp rx=0 tx=0).ready entity=mon.0 client_cookie=61bf5de92e610cdc server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:19:20.292 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:20.288+0000 7f765dd9b640 1 -- 172.21.15.143:0/529820816 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f76480058e0 con 0x7f764c0047d0 2023-11-26T07:19:20.292 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:20.288+0000 7f765dd9b640 1 -- 172.21.15.143:0/529820816 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7648005a40 con 0x7f764c0047d0 2023-11-26T07:19:20.292 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:20.288+0000 7f765dd9b640 1 -- 172.21.15.143:0/529820816 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7648005e70 con 0x7f764c0047d0 2023-11-26T07:19:20.292 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:20.288+0000 7f7664afe640 1 -- 172.21.15.143:0/529820816 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f764c0047d0 msgr2=0x7f764c004c50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:19:20.293 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:20.288+0000 7f7664afe640 1 --2- 172.21.15.143:0/529820816 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f764c0047d0 0x7f764c004c50 secure :-1 s=READY pgs=514 cs=0 l=1 rev1=1 crypto rx=0x7f764800c0e0 tx=0x7f764801f290 comp rx=0 tx=0).stop 2023-11-26T07:19:20.293 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:20.288+0000 7f7664afe640 1 -- 172.21.15.143:0/529820816 shutdown_connections 2023-11-26T07:19:20.293 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:20.288+0000 7f7664afe640 1 --2- 172.21.15.143:0/529820816 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f764c0047d0 0x7f764c004c50 unknown :-1 s=CLOSED pgs=514 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:19:20.293 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:20.288+0000 7f7664afe640 1 -- 172.21.15.143:0/529820816 >> 172.21.15.143:0/529820816 conn(0x7f766006a890 msgr2=0x7f766006aca0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:19:20.293 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:20.288+0000 7f7664afe640 1 -- 172.21.15.143:0/529820816 shutdown_connections 2023-11-26T07:19:20.293 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:20.288+0000 7f7664afe640 1 -- 172.21.15.143:0/529820816 wait complete. 2023-11-26T07:19:20.294 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:20.292+0000 7f7664afe640 1 Processor -- start 2023-11-26T07:19:20.294 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:20.292+0000 7f7664afe640 1 -- start start 2023-11-26T07:19:20.296 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:20.292+0000 7f7664afe640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f764c0047d0 0x7f766013e210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:19:20.296 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:20.292+0000 7f7664afe640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f7660139a50 0x7f7660142420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:19:20.296 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:20.292+0000 7f7664afe640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f7660159bf0 0x7f7660142960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:19:20.296 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:20.292+0000 7f7664afe640 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f766013f800 con 0x7f7660159bf0 2023-11-26T07:19:20.296 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:20.292+0000 7f7664afe640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f766013eb80 con 0x7f764c0047d0 2023-11-26T07:19:20.296 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:20.292+0000 7f7664afe640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f766013ee60 con 0x7f7660139a50 2023-11-26T07:19:20.296 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:20.292+0000 7f765ed9d640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f7660159bf0 0x7f7660142960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:19:20.297 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:20.292+0000 7f765ed9d640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f7660159bf0 0x7f7660142960 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:48106/0 (socket says 172.21.15.143:48106) 2023-11-26T07:19:20.297 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:20.292+0000 7f765ed9d640 1 -- 172.21.15.143:0/3002712045 learned_addr learned my addr 172.21.15.143:0/3002712045 (peer_addr_for_me v2:172.21.15.143:0/0) 2023-11-26T07:19:20.297 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:20.292+0000 7f765e59c640 1 --2- 172.21.15.143:0/3002712045 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f764c0047d0 0x7f766013e210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:19:20.297 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:20.292+0000 7f7657fff640 1 --2- 172.21.15.143:0/3002712045 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f7660139a50 0x7f7660142420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:19:20.297 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:20.292+0000 7f765ed9d640 1 -- 172.21.15.143:0/3002712045 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f7660139a50 msgr2=0x7f7660142420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:19:20.297 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:20.292+0000 7f765ed9d640 1 --2- 172.21.15.143:0/3002712045 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f7660139a50 0x7f7660142420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:19:20.297 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:20.292+0000 7f765ed9d640 1 -- 172.21.15.143:0/3002712045 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f764c0047d0 msgr2=0x7f766013e210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:19:20.297 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:20.292+0000 7f765ed9d640 1 --2- 172.21.15.143:0/3002712045 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f764c0047d0 0x7f766013e210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:19:20.297 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:20.292+0000 7f765ed9d640 1 -- 172.21.15.143:0/3002712045 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f766013f140 con 0x7f7660159bf0 2023-11-26T07:19:20.297 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:20.292+0000 7f765e59c640 1 --2- 172.21.15.143:0/3002712045 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f764c0047d0 0x7f766013e210 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-26T07:19:20.297 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:20.292+0000 7f765ed9d640 1 --2- 172.21.15.143:0/3002712045 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f7660159bf0 0x7f7660142960 secure :-1 s=READY pgs=515 cs=0 l=1 rev1=1 crypto rx=0x7f766006e880 tx=0x7f76580115a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:19:20.297 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:20.292+0000 7f7656ffd640 1 -- 172.21.15.143:0/3002712045 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f765800e070 con 0x7f7660159bf0 2023-11-26T07:19:20.298 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:20.292+0000 7f7656ffd640 1 -- 172.21.15.143:0/3002712045 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7658011db0 con 0x7f7660159bf0 2023-11-26T07:19:20.298 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:20.292+0000 7f7656ffd640 1 -- 172.21.15.143:0/3002712045 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f76580039a0 con 0x7f7660159bf0 2023-11-26T07:19:20.298 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:20.292+0000 7f7664afe640 1 -- 172.21.15.143:0/3002712045 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f766013f420 con 0x7f7660159bf0 2023-11-26T07:19:20.298 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:20.292+0000 7f7664afe640 1 -- 172.21.15.143:0/3002712045 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f766013e920 con 0x7f7660159bf0 2023-11-26T07:19:20.298 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:20.296+0000 7f7664afe640 1 -- 172.21.15.143:0/3002712045 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f766013f940 con 0x7f7660159bf0 2023-11-26T07:19:20.298 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:20.296+0000 7f7656ffd640 1 -- 172.21.15.143:0/3002712045 <== mon.0 v2:172.21.15.143:3300/0 4 ==== mgrmap(e 7) v1 ==== 86005+0+0 (secure 0 0 0) 0x7f7658003b00 con 0x7f7660159bf0 2023-11-26T07:19:20.299 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:20.296+0000 7f7656ffd640 1 --2- 172.21.15.143:0/3002712045 >> [v2:172.21.15.143:6832/2150557544,v1:172.21.15.143:6833/2150557544] conn(0x7f7640066650 0x7f7640068b10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:19:20.299 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:20.296+0000 7f7656ffd640 1 -- 172.21.15.143:0/3002712045 <== mon.0 v2:172.21.15.143:3300/0 5 ==== osd_map(34..34 src has 1..34) v4 ==== 5795+0+0 (secure 0 0 0) 0x7f7658084c20 con 0x7f7660159bf0 2023-11-26T07:19:20.299 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:20.296+0000 7f765e59c640 1 --2- 172.21.15.143:0/3002712045 >> [v2:172.21.15.143:6832/2150557544,v1:172.21.15.143:6833/2150557544] conn(0x7f7640066650 0x7f7640068b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:19:20.300 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:20.296+0000 7f765e59c640 1 --2- 172.21.15.143:0/3002712045 >> [v2:172.21.15.143:6832/2150557544,v1:172.21.15.143:6833/2150557544] conn(0x7f7640066650 0x7f7640068b10 secure :-1 s=READY pgs=263 cs=0 l=1 rev1=1 crypto rx=0x7f7650000c10 tx=0x7f7650009040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:19:20.302 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:20.300+0000 7f7656ffd640 1 -- 172.21.15.143:0/3002712045 <== mon.0 v2:172.21.15.143:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172649 (secure 0 0 0) 0x7f766013f940 con 0x7f7660159bf0 2023-11-26T07:19:20.453 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:20.452+0000 7f7664afe640 1 -- 172.21.15.143:0/3002712045 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7f766013f940 con 0x7f7660159bf0 2023-11-26T07:19:21.082 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:21.080+0000 7f7656ffd640 1 -- 172.21.15.143:0/3002712045 <== mon.0 v2:172.21.15.143:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_metadata' removed v35) v1 ==== 181+0+0 (secure 0 0 0) 0x7f766013f940 con 0x7f7660159bf0 2023-11-26T07:19:21.082 INFO:teuthology.orchestra.run.smithi143.stderr:pool 'cephfs_metadata' removed 2023-11-26T07:19:21.084 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:21.080+0000 7f7664afe640 1 -- 172.21.15.143:0/3002712045 >> [v2:172.21.15.143:6832/2150557544,v1:172.21.15.143:6833/2150557544] conn(0x7f7640066650 msgr2=0x7f7640068b10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:19:21.084 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:21.080+0000 7f7664afe640 1 --2- 172.21.15.143:0/3002712045 >> [v2:172.21.15.143:6832/2150557544,v1:172.21.15.143:6833/2150557544] conn(0x7f7640066650 0x7f7640068b10 secure :-1 s=READY pgs=263 cs=0 l=1 rev1=1 crypto rx=0x7f7650000c10 tx=0x7f7650009040 comp rx=0 tx=0).stop 2023-11-26T07:19:21.085 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:21.080+0000 7f7664afe640 1 -- 172.21.15.143:0/3002712045 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f7660159bf0 msgr2=0x7f7660142960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:19:21.085 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:21.080+0000 7f7664afe640 1 --2- 172.21.15.143:0/3002712045 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f7660159bf0 0x7f7660142960 secure :-1 s=READY pgs=515 cs=0 l=1 rev1=1 crypto rx=0x7f766006e880 tx=0x7f76580115a0 comp rx=0 tx=0).stop 2023-11-26T07:19:21.085 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:21.080+0000 7f7664afe640 1 -- 172.21.15.143:0/3002712045 shutdown_connections 2023-11-26T07:19:21.085 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:21.080+0000 7f7664afe640 1 --2- 172.21.15.143:0/3002712045 >> [v2:172.21.15.143:6832/2150557544,v1:172.21.15.143:6833/2150557544] conn(0x7f7640066650 0x7f7640068b10 unknown :-1 s=CLOSED pgs=263 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:19:21.085 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:21.080+0000 7f7664afe640 1 --2- 172.21.15.143:0/3002712045 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f7660159bf0 0x7f7660142960 unknown :-1 s=CLOSED pgs=515 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:19:21.085 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:21.080+0000 7f7664afe640 1 --2- 172.21.15.143:0/3002712045 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f7660139a50 0x7f7660142420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:19:21.085 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:21.080+0000 7f7664afe640 1 --2- 172.21.15.143:0/3002712045 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f764c0047d0 0x7f766013e210 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:19:21.085 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:21.080+0000 7f7664afe640 1 -- 172.21.15.143:0/3002712045 >> 172.21.15.143:0/3002712045 conn(0x7f766006a890 msgr2=0x7f766015ca30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:19:21.086 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:21.080+0000 7f7664afe640 1 -- 172.21.15.143:0/3002712045 shutdown_connections 2023-11-26T07:19:21.086 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:21.080+0000 7f7664afe640 1 -- 172.21.15.143:0/3002712045 wait complete. 2023-11-26T07:19:21.108 DEBUG:teuthology.orchestra.run.smithi143:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_data cephfs_data --yes-i-really-really-mean-it 2023-11-26T07:19:21.252 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:21.248+0000 7f56ea7a1640 1 Processor -- start 2023-11-26T07:19:21.252 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:21.248+0000 7f56ea7a1640 1 -- start start 2023-11-26T07:19:21.255 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:21.252+0000 7f56ea7a1640 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7f56e4143ac0 0x7f56e4070370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:19:21.256 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:21.252+0000 7f56ea7a1640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f56e40708b0 0x7f56e40cf280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:19:21.256 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:21.252+0000 7f56ea7a1640 1 -- --> v1:172.21.15.143:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f56e40cfbf0 con 0x7f56e40cf7c0 2023-11-26T07:19:21.256 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:21.252+0000 7f56ea7a1640 1 -- --> v2:172.21.15.188:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f56e4143ea0 con 0x7f56e4143ac0 2023-11-26T07:19:21.256 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:21.252+0000 7f56ea7a1640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f56e40cc730 con 0x7f56e40708b0 2023-11-26T07:19:21.256 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:21.252+0000 7f56e8d3e640 1 --1- >> v1:172.21.15.143:6789/0 conn(0x7f56e40cf7c0 0x7f56e40b1ef0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.143:6789/0 says I am v1:172.21.15.143:50176/0 (socket says 172.21.15.143:50176) 2023-11-26T07:19:21.256 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:21.252+0000 7f56e8d3e640 1 -- 172.21.15.143:0/3830640544 learned_addr learned my addr 172.21.15.143:0/3830640544 (peer_addr_for_me v1:172.21.15.143:0/0) 2023-11-26T07:19:21.256 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:21.252+0000 7f56e3fff640 1 --2- 172.21.15.143:0/3830640544 >> v2:172.21.15.188:3300/0 conn(0x7f56e4143ac0 0x7f56e4070370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:19:21.256 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:21.252+0000 7f56e37fe640 1 --2- 172.21.15.143:0/3830640544 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f56e40708b0 0x7f56e40cf280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:19:21.256 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:21.252+0000 7f56e2ffd640 1 -- 172.21.15.143:0/3830640544 <== mon.0 v1:172.21.15.143:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 896816489 0 0) 0x7f56e40cfbf0 con 0x7f56e40cf7c0 2023-11-26T07:19:21.256 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:21.252+0000 7f56e2ffd640 1 -- 172.21.15.143:0/3830640544 --> v1:172.21.15.143:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f56cc005740 con 0x7f56e40cf7c0 2023-11-26T07:19:21.256 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:21.252+0000 7f56e2ffd640 1 -- 172.21.15.143:0/3830640544 <== mon.0 v1:172.21.15.143:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 420455108 0 0) 0x7f56cc005740 con 0x7f56e40cf7c0 2023-11-26T07:19:21.256 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:21.252+0000 7f56e2ffd640 1 -- 172.21.15.143:0/3830640544 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f56e40708b0 msgr2=0x7f56e40cf280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-26T07:19:21.257 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:21.252+0000 7f56e2ffd640 1 --2- 172.21.15.143:0/3830640544 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f56e40708b0 0x7f56e40cf280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:19:21.257 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:21.252+0000 7f56e2ffd640 1 -- 172.21.15.143:0/3830640544 >> v2:172.21.15.188:3300/0 conn(0x7f56e4143ac0 msgr2=0x7f56e4070370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-26T07:19:21.257 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:21.252+0000 7f56e2ffd640 1 --2- 172.21.15.143:0/3830640544 >> v2:172.21.15.188:3300/0 conn(0x7f56e4143ac0 0x7f56e4070370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:19:21.257 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:21.252+0000 7f56e2ffd640 1 -- 172.21.15.143:0/3830640544 --> v1:172.21.15.143:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f56e4070ee0 con 0x7f56e40cf7c0 2023-11-26T07:19:21.257 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:21.252+0000 7f56e2ffd640 1 -- 172.21.15.143:0/3830640544 <== mon.0 v1:172.21.15.143:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2137882523 0 0) 0x7f56dc001a20 con 0x7f56e40cf7c0 2023-11-26T07:19:21.257 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:21.252+0000 7f56e2ffd640 1 -- 172.21.15.143:0/3830640544 >> v1:172.21.15.143:6789/0 conn(0x7f56e40cf7c0 legacy=0x7f56e40b1ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:19:21.257 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:21.252+0000 7f56e2ffd640 1 --2- 172.21.15.143:0/3830640544 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f56cc005b90 0x7f56cc005fb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:19:21.257 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:21.252+0000 7f56e2ffd640 1 --2- 172.21.15.143:0/3830640544 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f56cc0069e0 0x7f56cc006e60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:19:21.257 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:21.252+0000 7f56e2ffd640 1 --2- 172.21.15.143:0/3830640544 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f56e40708b0 0x7f56cc0073a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:19:21.257 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:21.252+0000 7f56e2ffd640 1 -- 172.21.15.143:0/3830640544 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f56e40cc730 con 0x7f56cc0069e0 2023-11-26T07:19:21.257 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:21.252+0000 7f56e2ffd640 1 -- 172.21.15.143:0/3830640544 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f56cc00dba0 con 0x7f56cc005b90 2023-11-26T07:19:21.257 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:21.252+0000 7f56e2ffd640 1 -- 172.21.15.143:0/3830640544 --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f56cc00dce0 con 0x7f56e40708b0 2023-11-26T07:19:21.258 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:21.252+0000 7f56e3fff640 1 --2- 172.21.15.143:0/3830640544 >> v2:172.21.15.188:3300/0 conn(0x7f56e4143ac0 0x7f56e4070370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-26T07:19:21.258 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:21.252+0000 7f56e37fe640 1 --2- 172.21.15.143:0/3830640544 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f56cc0069e0 0x7f56cc006e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:19:21.258 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:21.252+0000 7f56e8d3e640 1 --2- 172.21.15.143:0/3830640544 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f56e40708b0 0x7f56cc0073a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:19:21.258 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:21.252+0000 7f56e3fff640 1 --2- 172.21.15.143:0/3830640544 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f56cc005b90 0x7f56cc005fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:19:21.258 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:21.252+0000 7f56e37fe640 1 -- 172.21.15.143:0/3830640544 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f56e40708b0 msgr2=0x7f56cc0073a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-26T07:19:21.258 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:21.252+0000 7f56e37fe640 1 --2- 172.21.15.143:0/3830640544 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f56e40708b0 0x7f56cc0073a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:19:21.258 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:21.252+0000 7f56e37fe640 1 -- 172.21.15.143:0/3830640544 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f56cc005b90 msgr2=0x7f56cc005fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-26T07:19:21.258 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:21.252+0000 7f56e37fe640 1 --2- 172.21.15.143:0/3830640544 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f56cc005b90 0x7f56cc005fb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:19:21.258 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:21.252+0000 7f56e37fe640 1 -- 172.21.15.143:0/3830640544 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f56cc005740 con 0x7f56cc0069e0 2023-11-26T07:19:21.259 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:21.256+0000 7f56e37fe640 1 --2- 172.21.15.143:0/3830640544 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f56cc0069e0 0x7f56cc006e60 secure :-1 s=READY pgs=517 cs=0 l=1 rev1=1 crypto rx=0x7f56d8009ae0 tx=0x7f56d801ca50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7eb915bcd7a1c9dd server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:19:21.260 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:21.256+0000 7f56e2ffd640 1 -- 172.21.15.143:0/3830640544 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f56d80a5560 con 0x7f56cc0069e0 2023-11-26T07:19:21.260 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:21.256+0000 7f56e2ffd640 1 -- 172.21.15.143:0/3830640544 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f56d80a56c0 con 0x7f56cc0069e0 2023-11-26T07:19:21.260 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:21.256+0000 7f56e2ffd640 1 -- 172.21.15.143:0/3830640544 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f56d80a5a70 con 0x7f56cc0069e0 2023-11-26T07:19:21.260 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:21.256+0000 7f56ea7a1640 1 -- 172.21.15.143:0/3830640544 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f56cc0069e0 msgr2=0x7f56cc006e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:19:21.260 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:21.256+0000 7f56ea7a1640 1 --2- 172.21.15.143:0/3830640544 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f56cc0069e0 0x7f56cc006e60 secure :-1 s=READY pgs=517 cs=0 l=1 rev1=1 crypto rx=0x7f56d8009ae0 tx=0x7f56d801ca50 comp rx=0 tx=0).stop 2023-11-26T07:19:21.260 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:21.256+0000 7f56e8d3e640 1 -- 172.21.15.143:0/3830640544 reap_dead start 2023-11-26T07:19:21.260 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:21.256+0000 7f56ea7a1640 1 -- 172.21.15.143:0/3830640544 shutdown_connections 2023-11-26T07:19:21.260 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:21.256+0000 7f56ea7a1640 1 -- 172.21.15.143:0/3830640544 >> 172.21.15.143:0/3830640544 conn(0x7f56e406a890 msgr2=0x7f56e406aca0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:19:21.260 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:21.256+0000 7f56ea7a1640 1 -- 172.21.15.143:0/3830640544 shutdown_connections 2023-11-26T07:19:21.260 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:21.256+0000 7f56ea7a1640 1 -- 172.21.15.143:0/3830640544 wait complete. 2023-11-26T07:19:21.261 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:21.260+0000 7f56ea7a1640 1 Processor -- start 2023-11-26T07:19:21.261 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:21.260+0000 7f56ea7a1640 1 -- start start 2023-11-26T07:19:21.263 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:21.260+0000 7f56ea7a1640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f56cc0069e0 0x7f56e40b2ce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:19:21.263 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:21.260+0000 7f56ea7a1640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f56e413d440 0x7f56e413d8c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:19:21.263 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:21.260+0000 7f56ea7a1640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f56e40b0bd0 0x7f56e40b1050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:19:21.263 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:21.260+0000 7f56ea7a1640 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f56e41fa800 con 0x7f56cc0069e0 2023-11-26T07:19:21.263 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:21.260+0000 7f56ea7a1640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f56e413de00 con 0x7f56e413d440 2023-11-26T07:19:21.263 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:21.260+0000 7f56ea7a1640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f56e40b1870 con 0x7f56e40b0bd0 2023-11-26T07:19:21.263 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:21.260+0000 7f56e3fff640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f56cc0069e0 0x7f56e40b2ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:19:21.264 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:21.260+0000 7f56e3fff640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f56cc0069e0 0x7f56e40b2ce0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:48116/0 (socket says 172.21.15.143:48116) 2023-11-26T07:19:21.264 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:21.260+0000 7f56e3fff640 1 -- 172.21.15.143:0/2471364580 learned_addr learned my addr 172.21.15.143:0/2471364580 (peer_addr_for_me v2:172.21.15.143:0/0) 2023-11-26T07:19:21.264 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:21.260+0000 7f56e8d3e640 1 --2- 172.21.15.143:0/2471364580 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f56e40b0bd0 0x7f56e40b1050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:19:21.264 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:21.260+0000 7f56e37fe640 1 --2- 172.21.15.143:0/2471364580 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f56e413d440 0x7f56e413d8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:19:21.264 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:21.260+0000 7f56e3fff640 1 -- 172.21.15.143:0/2471364580 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f56e40b0bd0 msgr2=0x7f56e40b1050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:19:21.264 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:21.260+0000 7f56e3fff640 1 --2- 172.21.15.143:0/2471364580 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f56e40b0bd0 0x7f56e40b1050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:19:21.264 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:21.260+0000 7f56e3fff640 1 -- 172.21.15.143:0/2471364580 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f56e413d440 msgr2=0x7f56e413d8c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:19:21.264 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:21.260+0000 7f56e3fff640 1 --2- 172.21.15.143:0/2471364580 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f56e413d440 0x7f56e413d8c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:19:21.264 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:21.260+0000 7f56e3fff640 1 -- 172.21.15.143:0/2471364580 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f56e41faa50 con 0x7f56cc0069e0 2023-11-26T07:19:21.264 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:21.260+0000 7f56e3fff640 1 --2- 172.21.15.143:0/2471364580 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f56cc0069e0 0x7f56e40b2ce0 secure :-1 s=READY pgs=518 cs=0 l=1 rev1=1 crypto rx=0x7f56d4011cd0 tx=0x7f56d400f610 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:19:21.265 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:21.260+0000 7f56e0ff9640 1 -- 172.21.15.143:0/2471364580 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f56d4011d00 con 0x7f56cc0069e0 2023-11-26T07:19:21.265 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:21.260+0000 7f56e0ff9640 1 -- 172.21.15.143:0/2471364580 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f56d4011e60 con 0x7f56cc0069e0 2023-11-26T07:19:21.265 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:21.260+0000 7f56e0ff9640 1 -- 172.21.15.143:0/2471364580 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f56d4004270 con 0x7f56cc0069e0 2023-11-26T07:19:21.265 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:21.260+0000 7f56ea7a1640 1 -- 172.21.15.143:0/2471364580 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f56e41fad30 con 0x7f56cc0069e0 2023-11-26T07:19:21.266 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:21.260+0000 7f56ea7a1640 1 -- 172.21.15.143:0/2471364580 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f56e40b1590 con 0x7f56cc0069e0 2023-11-26T07:19:21.266 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:21.260+0000 7f56ea7a1640 1 -- 172.21.15.143:0/2471364580 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f56e4067680 con 0x7f56cc0069e0 2023-11-26T07:19:21.266 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:21.264+0000 7f56e0ff9640 1 -- 172.21.15.143:0/2471364580 <== mon.0 v2:172.21.15.143:3300/0 4 ==== mgrmap(e 7) v1 ==== 86005+0+0 (secure 0 0 0) 0x7f56d40043d0 con 0x7f56cc0069e0 2023-11-26T07:19:21.266 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:21.264+0000 7f56e0ff9640 1 --2- 172.21.15.143:0/2471364580 >> [v2:172.21.15.143:6832/2150557544,v1:172.21.15.143:6833/2150557544] conn(0x7f56d0066650 0x7f56d0068b10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:19:21.266 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:21.264+0000 7f56e0ff9640 1 -- 172.21.15.143:0/2471364580 <== mon.0 v2:172.21.15.143:3300/0 5 ==== osd_map(35..35 src has 1..35) v4 ==== 5357+0+0 (secure 0 0 0) 0x7f56d4084710 con 0x7f56cc0069e0 2023-11-26T07:19:21.267 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:21.264+0000 7f56e37fe640 1 --2- 172.21.15.143:0/2471364580 >> [v2:172.21.15.143:6832/2150557544,v1:172.21.15.143:6833/2150557544] conn(0x7f56d0066650 0x7f56d0068b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:19:21.267 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:21.264+0000 7f56e37fe640 1 --2- 172.21.15.143:0/2471364580 >> [v2:172.21.15.143:6832/2150557544,v1:172.21.15.143:6833/2150557544] conn(0x7f56d0066650 0x7f56d0068b10 secure :-1 s=READY pgs=264 cs=0 l=1 rev1=1 crypto rx=0x7f56d80af000 tx=0x7f56d80046d0 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:19:21.270 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:21.268+0000 7f56e0ff9640 1 -- 172.21.15.143:0/2471364580 <== mon.0 v2:172.21.15.143:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172649 (secure 0 0 0) 0x7f56d4051a80 con 0x7f56cc0069e0 2023-11-26T07:19:21.423 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:21.420+0000 7f56ea7a1640 1 -- 172.21.15.143:0/2471364580 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7f56e415a210 con 0x7f56cc0069e0 2023-11-26T07:19:21.832 DEBUG:teuthology.orchestra.run.smithi143:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-26T07:19:21.836 DEBUG:teuthology.orchestra.run.smithi188:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-26T07:19:22.084 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:22.080+0000 7f56e0ff9640 1 -- 172.21.15.143:0/2471364580 <== mon.0 v2:172.21.15.143:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_data' removed v36) v1 ==== 169+0+0 (secure 0 0 0) 0x7f56d4022020 con 0x7f56cc0069e0 2023-11-26T07:19:22.084 INFO:teuthology.orchestra.run.smithi143.stderr:pool 'cephfs_data' removed 2023-11-26T07:19:22.087 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:22.084+0000 7f56ea7a1640 1 -- 172.21.15.143:0/2471364580 >> [v2:172.21.15.143:6832/2150557544,v1:172.21.15.143:6833/2150557544] conn(0x7f56d0066650 msgr2=0x7f56d0068b10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:19:22.087 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:22.084+0000 7f56ea7a1640 1 --2- 172.21.15.143:0/2471364580 >> [v2:172.21.15.143:6832/2150557544,v1:172.21.15.143:6833/2150557544] conn(0x7f56d0066650 0x7f56d0068b10 secure :-1 s=READY pgs=264 cs=0 l=1 rev1=1 crypto rx=0x7f56d80af000 tx=0x7f56d80046d0 comp rx=0 tx=0).stop 2023-11-26T07:19:22.087 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:22.084+0000 7f56ea7a1640 1 -- 172.21.15.143:0/2471364580 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f56cc0069e0 msgr2=0x7f56e40b2ce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:19:22.087 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:22.084+0000 7f56ea7a1640 1 --2- 172.21.15.143:0/2471364580 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f56cc0069e0 0x7f56e40b2ce0 secure :-1 s=READY pgs=518 cs=0 l=1 rev1=1 crypto rx=0x7f56d4011cd0 tx=0x7f56d400f610 comp rx=0 tx=0).stop 2023-11-26T07:19:22.087 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:22.084+0000 7f56ea7a1640 1 -- 172.21.15.143:0/2471364580 shutdown_connections 2023-11-26T07:19:22.087 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:22.084+0000 7f56ea7a1640 1 --2- 172.21.15.143:0/2471364580 >> [v2:172.21.15.143:6832/2150557544,v1:172.21.15.143:6833/2150557544] conn(0x7f56d0066650 0x7f56d0068b10 unknown :-1 s=CLOSED pgs=264 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:19:22.087 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:22.084+0000 7f56ea7a1640 1 --2- 172.21.15.143:0/2471364580 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f56e40b0bd0 0x7f56e40b1050 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:19:22.087 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:22.084+0000 7f56ea7a1640 1 --2- 172.21.15.143:0/2471364580 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f56e413d440 0x7f56e413d8c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:19:22.087 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:22.084+0000 7f56ea7a1640 1 --2- 172.21.15.143:0/2471364580 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f56cc0069e0 0x7f56e40b2ce0 unknown :-1 s=CLOSED pgs=518 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:19:22.087 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:22.084+0000 7f56ea7a1640 1 -- 172.21.15.143:0/2471364580 >> 172.21.15.143:0/2471364580 conn(0x7f56e406a890 msgr2=0x7f56e40b3840 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:19:22.088 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:22.084+0000 7f56ea7a1640 1 -- 172.21.15.143:0/2471364580 shutdown_connections 2023-11-26T07:19:22.088 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:22.084+0000 7f56ea7a1640 1 -- 172.21.15.143:0/2471364580 wait complete. 2023-11-26T07:19:22.111 DEBUG:teuthology.orchestra.run.smithi143:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config rm mon mon_allow_pool_delete 2023-11-26T07:19:22.266 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:22.264+0000 7f415129a640 1 Processor -- start 2023-11-26T07:19:22.266 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:22.264+0000 7f415129a640 1 -- start start 2023-11-26T07:19:22.267 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:22.264+0000 7f415129a640 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7f414c13e290 0x7f414c0cf280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:19:22.267 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:22.264+0000 7f415129a640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f414c15f6b0 0x7f414c0cd7b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:19:22.267 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:22.264+0000 7f415129a640 1 -- --> v1:172.21.15.143:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f414c06c520 con 0x7f414c0cf7c0 2023-11-26T07:19:22.267 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:22.264+0000 7f415129a640 1 -- --> v2:172.21.15.188:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f414c0cfc40 con 0x7f414c13e290 2023-11-26T07:19:22.267 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:22.264+0000 7f415129a640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f414c15fb40 con 0x7f414c15f6b0 2023-11-26T07:19:22.269 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:22.264+0000 7f414a7fc640 1 --1- >> v1:172.21.15.143:6789/0 conn(0x7f414c0cf7c0 0x7f414c15efa0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.143:6789/0 says I am v1:172.21.15.143:50184/0 (socket says 172.21.15.143:50184) 2023-11-26T07:19:22.269 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:22.264+0000 7f414a7fc640 1 -- 172.21.15.143:0/1047019785 learned_addr learned my addr 172.21.15.143:0/1047019785 (peer_addr_for_me v1:172.21.15.143:0/0) 2023-11-26T07:19:22.269 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:22.264+0000 7f414affd640 1 --2- 172.21.15.143:0/1047019785 >> v2:172.21.15.188:3300/0 conn(0x7f414c13e290 0x7f414c0cf280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:19:22.269 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:22.264+0000 7f414b7fe640 1 --2- 172.21.15.143:0/1047019785 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f414c15f6b0 0x7f414c0cd7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:19:22.269 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:22.264+0000 7f4149ffb640 1 -- 172.21.15.143:0/1047019785 <== mon.0 v1:172.21.15.143:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1263490786 0 0) 0x7f414c06c520 con 0x7f414c0cf7c0 2023-11-26T07:19:22.269 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:22.264+0000 7f4149ffb640 1 -- 172.21.15.143:0/1047019785 --> v1:172.21.15.143:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f4134003530 con 0x7f414c0cf7c0 2023-11-26T07:19:22.269 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:22.264+0000 7f414b7fe640 1 -- 172.21.15.143:0/1047019785 >> v2:172.21.15.188:3300/0 conn(0x7f414c13e290 msgr2=0x7f414c0cf280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-26T07:19:22.269 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:22.264+0000 7f414b7fe640 1 --2- 172.21.15.143:0/1047019785 >> v2:172.21.15.188:3300/0 conn(0x7f414c13e290 0x7f414c0cf280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:19:22.269 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:22.264+0000 7f414b7fe640 1 -- 172.21.15.143:0/1047019785 >> v1:172.21.15.143:6789/0 conn(0x7f414c0cf7c0 legacy=0x7f414c15efa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:19:22.270 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:22.264+0000 7f414b7fe640 1 -- 172.21.15.143:0/1047019785 --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f414c06c800 con 0x7f414c15f6b0 2023-11-26T07:19:22.271 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:22.268+0000 7f414b7fe640 1 --2- 172.21.15.143:0/1047019785 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f414c15f6b0 0x7f414c0cd7b0 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7f4144009d90 tx=0x7f414401c920 comp rx=0 tx=0).ready entity=mon.2 client_cookie=56363aff2bdf2b63 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:19:22.272 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:22.268+0000 7f4149ffb640 1 -- 172.21.15.143:0/1047019785 <== mon.2 v2:172.21.15.188:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f41440a59e0 con 0x7f414c15f6b0 2023-11-26T07:19:22.272 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:22.268+0000 7f4149ffb640 1 -- 172.21.15.143:0/1047019785 <== mon.2 v2:172.21.15.188:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f41440a5b40 con 0x7f414c15f6b0 2023-11-26T07:19:22.272 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:22.268+0000 7f4149ffb640 1 -- 172.21.15.143:0/1047019785 <== mon.2 v2:172.21.15.188:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f41440ad440 con 0x7f414c15f6b0 2023-11-26T07:19:22.273 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:22.268+0000 7f415129a640 1 -- 172.21.15.143:0/1047019785 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f414c15f6b0 msgr2=0x7f414c0cd7b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:19:22.273 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:22.268+0000 7f415129a640 1 --2- 172.21.15.143:0/1047019785 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f414c15f6b0 0x7f414c0cd7b0 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7f4144009d90 tx=0x7f414401c920 comp rx=0 tx=0).stop 2023-11-26T07:19:22.273 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:22.268+0000 7f415129a640 1 -- 172.21.15.143:0/1047019785 shutdown_connections 2023-11-26T07:19:22.273 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:22.268+0000 7f415129a640 1 --2- 172.21.15.143:0/1047019785 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f414c15f6b0 0x7f414c0cd7b0 unknown :-1 s=CLOSED pgs=80 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:19:22.273 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:22.268+0000 7f415129a640 1 --2- 172.21.15.143:0/1047019785 >> v2:172.21.15.188:3300/0 conn(0x7f414c13e290 0x7f414c0cf280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:19:22.273 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:22.268+0000 7f415129a640 1 -- 172.21.15.143:0/1047019785 >> 172.21.15.143:0/1047019785 conn(0x7f414c06a890 msgr2=0x7f414c06aca0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:19:22.273 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:22.268+0000 7f415129a640 1 -- 172.21.15.143:0/1047019785 shutdown_connections 2023-11-26T07:19:22.273 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:22.268+0000 7f415129a640 1 -- 172.21.15.143:0/1047019785 wait complete. 2023-11-26T07:19:22.274 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:22.272+0000 7f415129a640 1 Processor -- start 2023-11-26T07:19:22.274 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:22.272+0000 7f415129a640 1 -- start start 2023-11-26T07:19:22.276 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:22.272+0000 7f415129a640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f414c0cf7c0 0x7f414c0aeb60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:19:22.276 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:22.272+0000 7f415129a640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f414c13e290 0x7f414c0b14d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:19:22.276 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:22.272+0000 7f415129a640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f414c15f6b0 0x7f414c1396f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:19:22.276 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:22.272+0000 7f415129a640 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f414c0af0a0 con 0x7f414c15f6b0 2023-11-26T07:19:22.276 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:22.272+0000 7f415129a640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f414c0b1a10 con 0x7f414c13e290 2023-11-26T07:19:22.276 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:22.272+0000 7f415129a640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f414c137e20 con 0x7f414c0cf7c0 2023-11-26T07:19:22.277 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:22.272+0000 7f414b7fe640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f414c15f6b0 0x7f414c1396f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:19:22.277 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:22.272+0000 7f414b7fe640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f414c15f6b0 0x7f414c1396f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:48122/0 (socket says 172.21.15.143:48122) 2023-11-26T07:19:22.277 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:22.272+0000 7f414b7fe640 1 -- 172.21.15.143:0/1027117329 learned_addr learned my addr 172.21.15.143:0/1027117329 (peer_addr_for_me v2:172.21.15.143:0/0) 2023-11-26T07:19:22.277 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:22.272+0000 7f414a7fc640 1 --2- 172.21.15.143:0/1027117329 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f414c13e290 0x7f414c0b14d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:19:22.277 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:22.272+0000 7f414affd640 1 --2- 172.21.15.143:0/1027117329 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f414c0cf7c0 0x7f414c0aeb60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:19:22.277 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:22.272+0000 7f414b7fe640 1 -- 172.21.15.143:0/1027117329 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f414c0cf7c0 msgr2=0x7f414c0aeb60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:19:22.277 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:22.272+0000 7f414b7fe640 1 --2- 172.21.15.143:0/1027117329 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f414c0cf7c0 0x7f414c0aeb60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:19:22.277 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:22.272+0000 7f414b7fe640 1 -- 172.21.15.143:0/1027117329 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f414c13e290 msgr2=0x7f414c0b14d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:19:22.277 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:22.272+0000 7f414b7fe640 1 --2- 172.21.15.143:0/1027117329 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f414c13e290 0x7f414c0b14d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:19:22.277 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:22.272+0000 7f414b7fe640 1 -- 172.21.15.143:0/1027117329 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4144009a70 con 0x7f414c15f6b0 2023-11-26T07:19:22.279 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:22.272+0000 7f414b7fe640 1 --2- 172.21.15.143:0/1027117329 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f414c15f6b0 0x7f414c1396f0 secure :-1 s=READY pgs=520 cs=0 l=1 rev1=1 crypto rx=0x7f414401a550 tx=0x7f41440196c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:19:22.279 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:22.272+0000 7f41337fe640 1 -- 172.21.15.143:0/1027117329 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4144007000 con 0x7f414c15f6b0 2023-11-26T07:19:22.279 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:22.272+0000 7f41337fe640 1 -- 172.21.15.143:0/1027117329 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f41440aed30 con 0x7f414c15f6b0 2023-11-26T07:19:22.279 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:22.272+0000 7f415129a640 1 -- 172.21.15.143:0/1027117329 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f414c150dd0 con 0x7f414c15f6b0 2023-11-26T07:19:22.279 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:22.276+0000 7f41337fe640 1 -- 172.21.15.143:0/1027117329 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f41440aeea0 con 0x7f414c15f6b0 2023-11-26T07:19:22.279 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:22.276+0000 7f415129a640 1 -- 172.21.15.143:0/1027117329 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f414c151070 con 0x7f414c15f6b0 2023-11-26T07:19:22.279 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:22.276+0000 7f415129a640 1 -- 172.21.15.143:0/1027117329 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f414c0b11a0 con 0x7f414c15f6b0 2023-11-26T07:19:22.279 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:22.276+0000 7f41337fe640 1 -- 172.21.15.143:0/1027117329 <== mon.0 v2:172.21.15.143:3300/0 4 ==== mgrmap(e 7) v1 ==== 86005+0+0 (secure 0 0 0) 0x7f41440b4050 con 0x7f414c15f6b0 2023-11-26T07:19:22.280 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:22.276+0000 7f41337fe640 1 --2- 172.21.15.143:0/1027117329 >> [v2:172.21.15.143:6832/2150557544,v1:172.21.15.143:6833/2150557544] conn(0x7f4138066620 0x7f4138068ae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:19:22.280 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:22.276+0000 7f41337fe640 1 -- 172.21.15.143:0/1027117329 <== mon.0 v2:172.21.15.143:3300/0 5 ==== osd_map(36..36 src has 1..36) v4 ==== 4943+0+0 (secure 0 0 0) 0x7f414411eea0 con 0x7f414c15f6b0 2023-11-26T07:19:22.280 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:22.276+0000 7f414affd640 1 --2- 172.21.15.143:0/1027117329 >> [v2:172.21.15.143:6832/2150557544,v1:172.21.15.143:6833/2150557544] conn(0x7f4138066620 0x7f4138068ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:19:22.281 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:22.280+0000 7f414affd640 1 --2- 172.21.15.143:0/1027117329 >> [v2:172.21.15.143:6832/2150557544,v1:172.21.15.143:6833/2150557544] conn(0x7f4138066620 0x7f4138068ae0 secure :-1 s=READY pgs=265 cs=0 l=1 rev1=1 crypto rx=0x7f413c009b60 tx=0x7f413c009190 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:19:22.284 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:22.280+0000 7f41337fe640 1 -- 172.21.15.143:0/1027117329 <== mon.0 v2:172.21.15.143:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172649 (secure 0 0 0) 0x7f41440ec3b0 con 0x7f414c15f6b0 2023-11-26T07:19:22.458 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:22.456+0000 7f415129a640 1 -- 172.21.15.143:0/1027117329 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "config rm", "who": "mon", "name": "mon_allow_pool_delete"} v 0) v1 -- 0x7f414c067680 con 0x7f414c15f6b0 2023-11-26T07:19:22.463 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:22.460+0000 7f41337fe640 1 -- 172.21.15.143:0/1027117329 <== mon.0 v2:172.21.15.143:3300/0 7 ==== mon_command_ack([{"prefix": "config rm", "who": "mon", "name": "mon_allow_pool_delete"}]=0 v11) v1 ==== 104+0+0 (secure 0 0 0) 0x7f4144009450 con 0x7f414c15f6b0 2023-11-26T07:19:22.466 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:22.464+0000 7f415129a640 1 -- 172.21.15.143:0/1027117329 >> [v2:172.21.15.143:6832/2150557544,v1:172.21.15.143:6833/2150557544] conn(0x7f4138066620 msgr2=0x7f4138068ae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:19:22.466 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:22.464+0000 7f415129a640 1 --2- 172.21.15.143:0/1027117329 >> [v2:172.21.15.143:6832/2150557544,v1:172.21.15.143:6833/2150557544] conn(0x7f4138066620 0x7f4138068ae0 secure :-1 s=READY pgs=265 cs=0 l=1 rev1=1 crypto rx=0x7f413c009b60 tx=0x7f413c009190 comp rx=0 tx=0).stop 2023-11-26T07:19:22.466 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:22.464+0000 7f415129a640 1 -- 172.21.15.143:0/1027117329 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f414c15f6b0 msgr2=0x7f414c1396f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:19:22.466 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:22.464+0000 7f415129a640 1 --2- 172.21.15.143:0/1027117329 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f414c15f6b0 0x7f414c1396f0 secure :-1 s=READY pgs=520 cs=0 l=1 rev1=1 crypto rx=0x7f414401a550 tx=0x7f41440196c0 comp rx=0 tx=0).stop 2023-11-26T07:19:22.466 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:22.464+0000 7f415129a640 1 -- 172.21.15.143:0/1027117329 shutdown_connections 2023-11-26T07:19:22.466 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:22.464+0000 7f415129a640 1 --2- 172.21.15.143:0/1027117329 >> [v2:172.21.15.143:6832/2150557544,v1:172.21.15.143:6833/2150557544] conn(0x7f4138066620 0x7f4138068ae0 unknown :-1 s=CLOSED pgs=265 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:19:22.466 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:22.464+0000 7f415129a640 1 --2- 172.21.15.143:0/1027117329 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f414c15f6b0 0x7f414c1396f0 unknown :-1 s=CLOSED pgs=520 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:19:22.466 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:22.464+0000 7f415129a640 1 --2- 172.21.15.143:0/1027117329 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f414c13e290 0x7f414c0b14d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:19:22.466 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:22.464+0000 7f415129a640 1 --2- 172.21.15.143:0/1027117329 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f414c0cf7c0 0x7f414c0aeb60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:19:22.467 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:22.464+0000 7f415129a640 1 -- 172.21.15.143:0/1027117329 >> 172.21.15.143:0/1027117329 conn(0x7f414c06a890 msgr2=0x7f414c13b0c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:19:22.467 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:22.464+0000 7f415129a640 1 -- 172.21.15.143:0/1027117329 shutdown_connections 2023-11-26T07:19:22.467 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:22.464+0000 7f415129a640 1 -- 172.21.15.143:0/1027117329 wait complete. 2023-11-26T07:19:22.485 DEBUG:teuthology.orchestra.run.smithi143:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Ended test tasks.cephfs.test_volumes.TestMisc.test_binary_metafile_on_legacy_to_v1_upgrade' 2023-11-26T07:19:22.623 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:22.620+0000 7f6021757640 1 Processor -- start 2023-11-26T07:19:22.624 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:22.620+0000 7f6021757640 1 -- start start 2023-11-26T07:19:22.625 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:22.624+0000 7f6021757640 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7f601c15d580 0x7f601c0d3920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:19:22.625 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:22.624+0000 7f6021757640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f601c0d3e60 0x7f601c0d17f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:19:22.625 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:22.624+0000 7f6021757640 1 -- --> v1:172.21.15.188:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f601c0d21e0 con 0x7f601c0d1d30 2023-11-26T07:19:22.625 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:22.624+0000 7f6021757640 1 -- --> v2:172.21.15.188:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f601c0d0bc0 con 0x7f601c15d580 2023-11-26T07:19:22.625 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:22.624+0000 7f6021757640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f601c0d0320 con 0x7f601c0d3e60 2023-11-26T07:19:22.626 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:22.624+0000 7f601affd640 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7f601c15d580 0x7f601c0d3920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:19:22.626 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:22.624+0000 7f60127fc640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f601c0d3e60 0x7f601c0d17f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:19:22.626 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:22.624+0000 7f601affd640 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7f601c15d580 0x7f601c0d3920 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.188:3300/0 says I am v2:172.21.15.143:37790/0 (socket says 172.21.15.143:37790) 2023-11-26T07:19:22.626 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:22.624+0000 7f601affd640 1 -- 172.21.15.143:0/3506158848 learned_addr learned my addr 172.21.15.143:0/3506158848 (peer_addr_for_me v2:172.21.15.143:0/0) 2023-11-26T07:19:22.626 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:22.624+0000 7f60127fc640 1 -- 172.21.15.143:0/3506158848 >> v2:172.21.15.188:3300/0 conn(0x7f601c15d580 msgr2=0x7f601c0d3920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-26T07:19:22.627 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:22.624+0000 7f60127fc640 1 --2- 172.21.15.143:0/3506158848 >> v2:172.21.15.188:3300/0 conn(0x7f601c15d580 0x7f601c0d3920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:19:22.627 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:22.624+0000 7f60127fc640 1 -- 172.21.15.143:0/3506158848 >> v1:172.21.15.188:6789/0 conn(0x7f601c0d1d30 legacy=0x7f601c0cdce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:19:22.627 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:22.624+0000 7f60127fc640 1 -- 172.21.15.143:0/3506158848 --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f601c0d0600 con 0x7f601c0d3e60 2023-11-26T07:19:22.629 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:22.628+0000 7f60127fc640 1 --2- 172.21.15.143:0/3506158848 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f601c0d3e60 0x7f601c0d17f0 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7f600c0097c0 tx=0x7f600c01c450 comp rx=0 tx=0).ready entity=mon.2 client_cookie=d1f0d792edf1f154 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:19:22.630 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:22.628+0000 7f601a7fc640 1 -- 172.21.15.143:0/3506158848 <== mon.2 v2:172.21.15.188:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f600c0ab650 con 0x7f601c0d3e60 2023-11-26T07:19:22.630 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:22.628+0000 7f601a7fc640 1 -- 172.21.15.143:0/3506158848 <== mon.2 v2:172.21.15.188:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f600c0ab7b0 con 0x7f601c0d3e60 2023-11-26T07:19:22.630 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:22.628+0000 7f601a7fc640 1 -- 172.21.15.143:0/3506158848 <== mon.2 v2:172.21.15.188:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f600c0abb40 con 0x7f601c0d3e60 2023-11-26T07:19:22.631 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:22.628+0000 7f6021757640 1 -- 172.21.15.143:0/3506158848 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f601c0d3e60 msgr2=0x7f601c0d17f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:19:22.631 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:22.628+0000 7f6021757640 1 --2- 172.21.15.143:0/3506158848 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f601c0d3e60 0x7f601c0d17f0 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7f600c0097c0 tx=0x7f600c01c450 comp rx=0 tx=0).stop 2023-11-26T07:19:22.631 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:22.628+0000 7f6021757640 1 -- 172.21.15.143:0/3506158848 shutdown_connections 2023-11-26T07:19:22.631 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:22.628+0000 7f6021757640 1 --2- 172.21.15.143:0/3506158848 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f601c0d3e60 0x7f601c0d17f0 unknown :-1 s=CLOSED pgs=81 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:19:22.631 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:22.628+0000 7f6021757640 1 --2- 172.21.15.143:0/3506158848 >> v2:172.21.15.188:3300/0 conn(0x7f601c15d580 0x7f601c0d3920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:19:22.631 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:22.628+0000 7f6021757640 1 -- 172.21.15.143:0/3506158848 >> 172.21.15.143:0/3506158848 conn(0x7f601c09c550 msgr2=0x7f601c0d3210 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:19:22.631 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:22.628+0000 7f6021757640 1 -- 172.21.15.143:0/3506158848 shutdown_connections 2023-11-26T07:19:22.631 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:22.628+0000 7f6021757640 1 -- 172.21.15.143:0/3506158848 wait complete. 2023-11-26T07:19:22.632 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:22.628+0000 7f6021757640 1 Processor -- start 2023-11-26T07:19:22.632 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:22.628+0000 7f6021757640 1 -- start start 2023-11-26T07:19:22.635 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:22.632+0000 7f6021757640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f601c0d1d30 0x7f601c15c320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:19:22.635 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:22.632+0000 7f6021757640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f601c0d3e60 0x7f601c14bd90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:19:22.635 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:22.632+0000 7f6021757640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f601c15d580 0x7f601c0b3960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:19:22.635 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:22.632+0000 7f6021757640 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f601c0c80d0 con 0x7f601c0d3e60 2023-11-26T07:19:22.635 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:22.632+0000 7f6021757640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f601c15b050 con 0x7f601c0d1d30 2023-11-26T07:19:22.635 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:22.632+0000 7f6021757640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f601c15c860 con 0x7f601c15d580 2023-11-26T07:19:22.635 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:22.632+0000 7f60127fc640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f601c0d3e60 0x7f601c14bd90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:19:22.635 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:22.632+0000 7f60127fc640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f601c0d3e60 0x7f601c14bd90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:48128/0 (socket says 172.21.15.143:48128) 2023-11-26T07:19:22.635 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:22.632+0000 7f60127fc640 1 -- 172.21.15.143:0/1451541542 learned_addr learned my addr 172.21.15.143:0/1451541542 (peer_addr_for_me v2:172.21.15.143:0/0) 2023-11-26T07:19:22.635 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:22.632+0000 7f601affd640 1 --2- 172.21.15.143:0/1451541542 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f601c0d1d30 0x7f601c15c320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:19:22.635 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:22.632+0000 7f601b7fe640 1 --2- 172.21.15.143:0/1451541542 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f601c15d580 0x7f601c0b3960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:19:22.636 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:22.632+0000 7f60127fc640 1 -- 172.21.15.143:0/1451541542 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f601c15d580 msgr2=0x7f601c0b3960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:19:22.636 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:22.632+0000 7f60127fc640 1 --2- 172.21.15.143:0/1451541542 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f601c15d580 0x7f601c0b3960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:19:22.636 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:22.632+0000 7f60127fc640 1 -- 172.21.15.143:0/1451541542 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f601c0d1d30 msgr2=0x7f601c15c320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:19:22.636 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:22.632+0000 7f60127fc640 1 --2- 172.21.15.143:0/1451541542 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f601c0d1d30 0x7f601c15c320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:19:22.636 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:22.632+0000 7f60127fc640 1 -- 172.21.15.143:0/1451541542 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f600c009420 con 0x7f601c0d3e60 2023-11-26T07:19:22.636 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:22.632+0000 7f60127fc640 1 --2- 172.21.15.143:0/1451541542 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f601c0d3e60 0x7f601c14bd90 secure :-1 s=READY pgs=521 cs=0 l=1 rev1=1 crypto rx=0x7f600c0ad040 tx=0x7f600c01ac90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:19:22.636 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:22.632+0000 7f6013fff640 1 -- 172.21.15.143:0/1451541542 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f600c0bf610 con 0x7f601c0d3e60 2023-11-26T07:19:22.636 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:22.632+0000 7f6013fff640 1 -- 172.21.15.143:0/1451541542 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f600c0bf790 con 0x7f601c0d3e60 2023-11-26T07:19:22.636 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:22.632+0000 7f6021757640 1 -- 172.21.15.143:0/1451541542 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f601c15a780 con 0x7f601c0d3e60 2023-11-26T07:19:22.636 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:22.632+0000 7f6013fff640 1 -- 172.21.15.143:0/1451541542 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f600c0bfb40 con 0x7f601c0d3e60 2023-11-26T07:19:22.636 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:22.632+0000 7f6021757640 1 -- 172.21.15.143:0/1451541542 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f601c14c8d0 con 0x7f601c0d3e60 2023-11-26T07:19:22.637 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:22.632+0000 7f6021757640 1 -- 172.21.15.143:0/1451541542 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f601c15b1c0 con 0x7f601c0d3e60 2023-11-26T07:19:22.637 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:22.636+0000 7f6013fff640 1 -- 172.21.15.143:0/1451541542 <== mon.0 v2:172.21.15.143:3300/0 4 ==== mgrmap(e 7) v1 ==== 86005+0+0 (secure 0 0 0) 0x7f600c0bfca0 con 0x7f601c0d3e60 2023-11-26T07:19:22.638 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:22.636+0000 7f6013fff640 1 --2- 172.21.15.143:0/1451541542 >> [v2:172.21.15.143:6832/2150557544,v1:172.21.15.143:6833/2150557544] conn(0x7f5ffc066620 0x7f5ffc068ae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:19:22.638 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:22.636+0000 7f6013fff640 1 -- 172.21.15.143:0/1451541542 <== mon.0 v2:172.21.15.143:3300/0 5 ==== osd_map(36..36 src has 1..36) v4 ==== 4943+0+0 (secure 0 0 0) 0x7f600c11e960 con 0x7f601c0d3e60 2023-11-26T07:19:22.638 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:22.636+0000 7f601affd640 1 --2- 172.21.15.143:0/1451541542 >> [v2:172.21.15.143:6832/2150557544,v1:172.21.15.143:6833/2150557544] conn(0x7f5ffc066620 0x7f5ffc068ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:19:22.638 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:22.636+0000 7f601affd640 1 --2- 172.21.15.143:0/1451541542 >> [v2:172.21.15.143:6832/2150557544,v1:172.21.15.143:6833/2150557544] conn(0x7f5ffc066620 0x7f5ffc068ae0 secure :-1 s=READY pgs=266 cs=0 l=1 rev1=1 crypto rx=0x7f601c0b4860 tx=0x7f6004009240 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:19:22.642 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:22.640+0000 7f6013fff640 1 -- 172.21.15.143:0/1451541542 <== mon.0 v2:172.21.15.143:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172649 (secure 0 0 0) 0x7f600c0ec0c0 con 0x7f601c0d3e60 2023-11-26T07:19:22.843 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:22.840+0000 7f6021757640 1 -- 172.21.15.143:0/1451541542 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_volumes.TestMisc.test_binary_metafile_on_legacy_to_v1_upgrade"]} v 0) v1 -- 0x7f601c067680 con 0x7f601c0d3e60 2023-11-26T07:19:23.086 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:23.084+0000 7f6013fff640 1 -- 172.21.15.143:0/1451541542 <== mon.0 v2:172.21.15.143:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_volumes.TestMisc.test_binary_metafile_on_legacy_to_v1_upgrade"]}]=0 v105) v1 ==== 158+0+0 (secure 0 0 0) 0x7f600c0ef6e0 con 0x7f601c0d3e60 2023-11-26T07:19:23.089 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:23.084+0000 7f6021757640 1 -- 172.21.15.143:0/1451541542 >> [v2:172.21.15.143:6832/2150557544,v1:172.21.15.143:6833/2150557544] conn(0x7f5ffc066620 msgr2=0x7f5ffc068ae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:19:23.089 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:23.084+0000 7f6021757640 1 --2- 172.21.15.143:0/1451541542 >> [v2:172.21.15.143:6832/2150557544,v1:172.21.15.143:6833/2150557544] conn(0x7f5ffc066620 0x7f5ffc068ae0 secure :-1 s=READY pgs=266 cs=0 l=1 rev1=1 crypto rx=0x7f601c0b4860 tx=0x7f6004009240 comp rx=0 tx=0).stop 2023-11-26T07:19:23.089 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:23.084+0000 7f6021757640 1 -- 172.21.15.143:0/1451541542 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f601c0d3e60 msgr2=0x7f601c14bd90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:19:23.089 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:23.084+0000 7f6021757640 1 --2- 172.21.15.143:0/1451541542 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f601c0d3e60 0x7f601c14bd90 secure :-1 s=READY pgs=521 cs=0 l=1 rev1=1 crypto rx=0x7f600c0ad040 tx=0x7f600c01ac90 comp rx=0 tx=0).stop 2023-11-26T07:19:23.089 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:23.084+0000 7f6021757640 1 -- 172.21.15.143:0/1451541542 shutdown_connections 2023-11-26T07:19:23.089 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:23.084+0000 7f6021757640 1 --2- 172.21.15.143:0/1451541542 >> [v2:172.21.15.143:6832/2150557544,v1:172.21.15.143:6833/2150557544] conn(0x7f5ffc066620 0x7f5ffc068ae0 unknown :-1 s=CLOSED pgs=266 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:19:23.089 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:23.084+0000 7f6021757640 1 --2- 172.21.15.143:0/1451541542 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f601c15d580 0x7f601c0b3960 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:19:23.090 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:23.084+0000 7f6021757640 1 --2- 172.21.15.143:0/1451541542 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f601c0d3e60 0x7f601c14bd90 unknown :-1 s=CLOSED pgs=521 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:19:23.090 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:23.084+0000 7f6021757640 1 --2- 172.21.15.143:0/1451541542 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f601c0d1d30 0x7f601c15c320 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:19:23.090 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:23.084+0000 7f6021757640 1 -- 172.21.15.143:0/1451541542 >> 172.21.15.143:0/1451541542 conn(0x7f601c09c550 msgr2=0x7f601c155fd0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:19:23.090 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:23.088+0000 7f6021757640 1 -- 172.21.15.143:0/1451541542 shutdown_connections 2023-11-26T07:19:23.090 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:23.088+0000 7f6021757640 1 -- 172.21.15.143:0/1451541542 wait complete. 2023-11-26T07:19:23.109 INFO:tasks.cephfs_test_runner:Validate binary .meta file on legacy subvol root doesn't break the system ... ok 2023-11-26T07:19:23.109 INFO:tasks.cephfs_test_runner:Starting test: test_connection_expiration (tasks.cephfs.test_volumes.TestMisc) 2023-11-26T07:19:23.109 DEBUG:teuthology.orchestra.run.smithi143:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-11-26T07:19:23.247 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:23.244+0000 7fdca17f5640 1 Processor -- start 2023-11-26T07:19:23.248 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:23.244+0000 7fdca17f5640 1 -- start start 2023-11-26T07:19:23.249 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:23.244+0000 7fdca17f5640 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7fdc9c0b14a0 0x7fdc9c1681a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:19:23.249 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:23.244+0000 7fdca17f5640 1 -- --> v1:172.21.15.143:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fdc9c0d0bc0 con 0x7fdc9c15d580 2023-11-26T07:19:23.249 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:23.244+0000 7fdca17f5640 1 -- --> v1:172.21.15.188:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fdc9c0d0320 con 0x7fdc9c0d17f0 2023-11-26T07:19:23.249 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:23.244+0000 7fdca17f5640 1 -- --> v2:172.21.15.188:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fdc9c0d0600 con 0x7fdc9c0b14a0 2023-11-26T07:19:23.249 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:23.248+0000 7fdc9affd640 1 --1- >> v1:172.21.15.143:6789/0 conn(0x7fdc9c15d580 0x7fdc9c0d3920 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.143:6789/0 says I am v1:172.21.15.143:50190/0 (socket says 172.21.15.143:50190) 2023-11-26T07:19:23.249 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:23.248+0000 7fdc9affd640 1 -- 172.21.15.143:0/357492985 learned_addr learned my addr 172.21.15.143:0/357492985 (peer_addr_for_me v1:172.21.15.143:0/0) 2023-11-26T07:19:23.251 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:23.248+0000 7fdc9b7fe640 1 --2- 172.21.15.143:0/357492985 >> v2:172.21.15.188:3300/0 conn(0x7fdc9c0b14a0 0x7fdc9c1681a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:19:23.251 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:23.248+0000 7fdc99ffb640 1 -- 172.21.15.143:0/357492985 <== mon.0 v1:172.21.15.143:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1859643011 0 0) 0x7fdc9c0d0bc0 con 0x7fdc9c15d580 2023-11-26T07:19:23.251 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:23.248+0000 7fdc99ffb640 1 -- 172.21.15.143:0/357492985 --> v1:172.21.15.143:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fdc84003530 con 0x7fdc9c15d580 2023-11-26T07:19:23.252 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:23.248+0000 7fdc99ffb640 1 -- 172.21.15.143:0/357492985 <== mon.0 v1:172.21.15.143:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 4125973436 0 0) 0x7fdc84003530 con 0x7fdc9c15d580 2023-11-26T07:19:23.252 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:23.248+0000 7fdc99ffb640 1 -- 172.21.15.143:0/357492985 >> v2:172.21.15.188:3300/0 conn(0x7fdc9c0b14a0 msgr2=0x7fdc9c1681a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-26T07:19:23.252 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:23.248+0000 7fdc99ffb640 1 --2- 172.21.15.143:0/357492985 >> v2:172.21.15.188:3300/0 conn(0x7fdc9c0b14a0 0x7fdc9c1681a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:19:23.252 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:23.248+0000 7fdc99ffb640 1 -- 172.21.15.143:0/357492985 >> v1:172.21.15.188:6789/0 conn(0x7fdc9c0d17f0 legacy=0x7fdc9c0d1c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:19:23.252 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:23.248+0000 7fdc99ffb640 1 -- 172.21.15.143:0/357492985 --> v1:172.21.15.143:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdc9c0d08e0 con 0x7fdc9c15d580 2023-11-26T07:19:23.252 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:23.248+0000 7fdc99ffb640 1 -- 172.21.15.143:0/357492985 <== mon.0 v1:172.21.15.143:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2137882523 0 0) 0x7fdc8c001680 con 0x7fdc9c15d580 2023-11-26T07:19:23.252 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:23.248+0000 7fdc99ffb640 1 -- 172.21.15.143:0/357492985 >> v1:172.21.15.143:6789/0 conn(0x7fdc9c15d580 legacy=0x7fdc9c0d3920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:19:23.252 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:23.248+0000 7fdc99ffb640 1 --2- 172.21.15.143:0/357492985 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fdc840039d0 0x7fdc84003df0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:19:23.253 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:23.248+0000 7fdc99ffb640 1 --2- 172.21.15.143:0/357492985 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fdc840047d0 0x7fdc84004c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:19:23.253 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:23.248+0000 7fdc99ffb640 1 --2- 172.21.15.143:0/357492985 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fdc84005190 0x7fdc8400b990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:19:23.253 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:23.248+0000 7fdc99ffb640 1 -- 172.21.15.143:0/357492985 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdc9c0d0600 con 0x7fdc84005190 2023-11-26T07:19:23.253 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:23.248+0000 7fdc99ffb640 1 -- 172.21.15.143:0/357492985 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdc8400c090 con 0x7fdc840039d0 2023-11-26T07:19:23.253 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:23.248+0000 7fdc99ffb640 1 -- 172.21.15.143:0/357492985 --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdc8400c250 con 0x7fdc840047d0 2023-11-26T07:19:23.253 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:23.248+0000 7fdc9b7fe640 1 --2- 172.21.15.143:0/357492985 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fdc84005190 0x7fdc8400b990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:19:23.253 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:23.248+0000 7fdc9affd640 1 --2- 172.21.15.143:0/357492985 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fdc840039d0 0x7fdc84003df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:19:23.253 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:23.248+0000 7fdc9a7fc640 1 --2- 172.21.15.143:0/357492985 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fdc840047d0 0x7fdc84004c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:19:23.253 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:23.248+0000 7fdc9b7fe640 1 -- 172.21.15.143:0/357492985 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fdc840047d0 msgr2=0x7fdc84004c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-26T07:19:23.253 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:23.248+0000 7fdc9b7fe640 1 --2- 172.21.15.143:0/357492985 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fdc840047d0 0x7fdc84004c50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:19:23.254 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:23.248+0000 7fdc9b7fe640 1 -- 172.21.15.143:0/357492985 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fdc840039d0 msgr2=0x7fdc84003df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-26T07:19:23.254 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:23.248+0000 7fdc9b7fe640 1 --2- 172.21.15.143:0/357492985 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fdc840039d0 0x7fdc84003df0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:19:23.254 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:23.248+0000 7fdc9b7fe640 1 -- 172.21.15.143:0/357492985 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdc84003530 con 0x7fdc84005190 2023-11-26T07:19:23.255 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:23.252+0000 7fdc9b7fe640 1 -- 172.21.15.143:0/357492985 reap_dead start 2023-11-26T07:19:23.255 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:23.252+0000 7fdc9b7fe640 1 --2- 172.21.15.143:0/357492985 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fdc84005190 0x7fdc8400b990 secure :-1 s=READY pgs=523 cs=0 l=1 rev1=1 crypto rx=0x7fdc94009f90 tx=0x7fdc9401cc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=59fc22c008abae66 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:19:23.256 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:23.252+0000 7fdc99ffb640 1 -- 172.21.15.143:0/357492985 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdc940a5c70 con 0x7fdc84005190 2023-11-26T07:19:23.256 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:23.252+0000 7fdc99ffb640 1 -- 172.21.15.143:0/357492985 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdc9400f040 con 0x7fdc84005190 2023-11-26T07:19:23.256 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:23.252+0000 7fdc99ffb640 1 -- 172.21.15.143:0/357492985 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdc940ad440 con 0x7fdc84005190 2023-11-26T07:19:23.256 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:23.252+0000 7fdca17f5640 1 -- 172.21.15.143:0/357492985 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fdc84005190 msgr2=0x7fdc8400b990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:19:23.256 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:23.252+0000 7fdca17f5640 1 --2- 172.21.15.143:0/357492985 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fdc84005190 0x7fdc8400b990 secure :-1 s=READY pgs=523 cs=0 l=1 rev1=1 crypto rx=0x7fdc94009f90 tx=0x7fdc9401cc60 comp rx=0 tx=0).stop 2023-11-26T07:19:23.256 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:23.252+0000 7fdca17f5640 1 -- 172.21.15.143:0/357492985 shutdown_connections 2023-11-26T07:19:23.256 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:23.252+0000 7fdca17f5640 1 --2- 172.21.15.143:0/357492985 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fdc84005190 0x7fdc8400b990 unknown :-1 s=CLOSED pgs=523 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:19:23.256 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:23.252+0000 7fdca17f5640 1 -- 172.21.15.143:0/357492985 >> 172.21.15.143:0/357492985 conn(0x7fdc9c09c550 msgr2=0x7fdc9c0d3210 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:19:23.256 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:23.252+0000 7fdca17f5640 1 -- 172.21.15.143:0/357492985 shutdown_connections 2023-11-26T07:19:23.256 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:23.252+0000 7fdca17f5640 1 -- 172.21.15.143:0/357492985 wait complete. 2023-11-26T07:19:23.259 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:23.252+0000 7fdca17f5640 1 Processor -- start 2023-11-26T07:19:23.259 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:23.252+0000 7fdca17f5640 1 -- start start 2023-11-26T07:19:23.259 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:23.256+0000 7fdca17f5640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fdc84005190 0x7fdc9c15a4e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:19:23.259 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:23.256+0000 7fdca17f5640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fdc9c0b82b0 0x7fdc9c0b8730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:19:23.259 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:23.256+0000 7fdca17f5640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fdc9c15dcd0 0x7fdc9c15e110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:19:23.259 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:23.256+0000 7fdca17f5640 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdc9c15e650 con 0x7fdc9c15dcd0 2023-11-26T07:19:23.259 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:23.256+0000 7fdca17f5640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdc9c0b6060 con 0x7fdc9c0b82b0 2023-11-26T07:19:23.259 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:23.256+0000 7fdca17f5640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdc9c0b6300 con 0x7fdc84005190 2023-11-26T07:19:23.259 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:23.256+0000 7fdc9b7fe640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fdc9c15dcd0 0x7fdc9c15e110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:19:23.259 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:23.256+0000 7fdc9b7fe640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fdc9c15dcd0 0x7fdc9c15e110 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:48158/0 (socket says 172.21.15.143:48158) 2023-11-26T07:19:23.259 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:23.256+0000 7fdc9b7fe640 1 -- 172.21.15.143:0/672036297 learned_addr learned my addr 172.21.15.143:0/672036297 (peer_addr_for_me v2:172.21.15.143:0/0) 2023-11-26T07:19:23.260 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:23.256+0000 7fdc9a7fc640 1 --2- 172.21.15.143:0/672036297 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fdc9c0b82b0 0x7fdc9c0b8730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:19:23.260 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:23.256+0000 7fdc9affd640 1 --2- 172.21.15.143:0/672036297 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fdc84005190 0x7fdc9c15a4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:19:23.260 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:23.256+0000 7fdc9b7fe640 1 -- 172.21.15.143:0/672036297 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fdc84005190 msgr2=0x7fdc9c15a4e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:19:23.260 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:23.256+0000 7fdc9b7fe640 1 --2- 172.21.15.143:0/672036297 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fdc84005190 0x7fdc9c15a4e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:19:23.260 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:23.256+0000 7fdc9b7fe640 1 -- 172.21.15.143:0/672036297 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fdc9c0b82b0 msgr2=0x7fdc9c0b8730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:19:23.260 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:23.256+0000 7fdc9b7fe640 1 --2- 172.21.15.143:0/672036297 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fdc9c0b82b0 0x7fdc9c0b8730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:19:23.260 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:23.256+0000 7fdc9b7fe640 1 -- 172.21.15.143:0/672036297 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdc9c0b65e0 con 0x7fdc9c15dcd0 2023-11-26T07:19:23.260 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:23.256+0000 7fdc9b7fe640 1 --2- 172.21.15.143:0/672036297 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fdc9c15dcd0 0x7fdc9c15e110 secure :-1 s=READY pgs=524 cs=0 l=1 rev1=1 crypto rx=0x7fdc9401ca90 tx=0x7fdc94009750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:19:23.260 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:23.256+0000 7fdc837fe640 1 -- 172.21.15.143:0/672036297 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdc940adc00 con 0x7fdc9c15dcd0 2023-11-26T07:19:23.260 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:23.256+0000 7fdc837fe640 1 -- 172.21.15.143:0/672036297 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdc940add60 con 0x7fdc9c15dcd0 2023-11-26T07:19:23.260 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:23.256+0000 7fdca17f5640 1 -- 172.21.15.143:0/672036297 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdc9c0b68c0 con 0x7fdc9c15dcd0 2023-11-26T07:19:23.260 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:23.256+0000 7fdc837fe640 1 -- 172.21.15.143:0/672036297 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdc940b8680 con 0x7fdc9c15dcd0 2023-11-26T07:19:23.261 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:23.256+0000 7fdca17f5640 1 -- 172.21.15.143:0/672036297 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdc9c0b6e80 con 0x7fdc9c15dcd0 2023-11-26T07:19:23.261 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:23.256+0000 7fdca17f5640 1 -- 172.21.15.143:0/672036297 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fdc9c1f2f10 con 0x7fdc9c15dcd0 2023-11-26T07:19:23.261 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:23.256+0000 7fdc837fe640 1 -- 172.21.15.143:0/672036297 <== mon.0 v2:172.21.15.143:3300/0 4 ==== mgrmap(e 7) v1 ==== 86005+0+0 (secure 0 0 0) 0x7fdc940b4040 con 0x7fdc9c15dcd0 2023-11-26T07:19:23.261 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:23.260+0000 7fdc837fe640 1 --2- 172.21.15.143:0/672036297 >> [v2:172.21.15.143:6832/2150557544,v1:172.21.15.143:6833/2150557544] conn(0x7fdc88066650 0x7fdc88068b10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:19:23.262 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:23.260+0000 7fdc837fe640 1 -- 172.21.15.143:0/672036297 <== mon.0 v2:172.21.15.143:3300/0 5 ==== osd_map(36..36 src has 1..36) v4 ==== 4943+0+0 (secure 0 0 0) 0x7fdc9411e180 con 0x7fdc9c15dcd0 2023-11-26T07:19:23.262 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:23.260+0000 7fdc9affd640 1 --2- 172.21.15.143:0/672036297 >> [v2:172.21.15.143:6832/2150557544,v1:172.21.15.143:6833/2150557544] conn(0x7fdc88066650 0x7fdc88068b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:19:23.262 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:23.260+0000 7fdc9affd640 1 --2- 172.21.15.143:0/672036297 >> [v2:172.21.15.143:6832/2150557544,v1:172.21.15.143:6833/2150557544] conn(0x7fdc88066650 0x7fdc88068b10 secure :-1 s=READY pgs=267 cs=0 l=1 rev1=1 crypto rx=0x7fdc8c0017b0 tx=0x7fdc8c0015c0 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:19:23.265 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:23.264+0000 7fdc837fe640 1 -- 172.21.15.143:0/672036297 <== mon.0 v2:172.21.15.143:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172649 (secure 0 0 0) 0x7fdc9c1f2f10 con 0x7fdc9c15dcd0 2023-11-26T07:19:23.415 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:23.412+0000 7fdca17f5640 1 -- 172.21.15.143:0/672036297 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fdc9c1f2f10 con 0x7fdc9c15dcd0 2023-11-26T07:19:23.416 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:23.412+0000 7fdc837fe640 1 -- 172.21.15.143:0/672036297 <== mon.0 v2:172.21.15.143:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v36) v1 ==== 74+0+13024 (secure 0 0 0) 0x7fdc9c1f2f10 con 0x7fdc9c15dcd0 2023-11-26T07:19:23.416 INFO:teuthology.orchestra.run.smithi143.stdout: 2023-11-26T07:19:23.417 INFO:teuthology.orchestra.run.smithi143.stdout:{"epoch":36,"fsid":"f5033060-2469-4a07-95fe-2b76f5404bc1","created":"2023-11-26T07:17:28.613663+0000","modified":"2023-11-26T07:19:22.081498+0000","last_up_change":"2023-11-26T07:17:41.157813+0000","last_in_change":"2023-11-26T07:17:31.181658+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":5,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-11-26T07:17:41.758597+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"ebbee2c6-023e-44b9-9f1d-6f7bfd309730","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6817","nonce":639807369},{"type":"v1","addr":"172.21.15.143:6819","nonce":639807369}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6821","nonce":639807369},{"type":"v1","addr":"172.21.15.143:6823","nonce":639807369}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6830","nonce":639807369},{"type":"v1","addr":"172.21.15.143:6831","nonce":639807369}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6825","nonce":639807369},{"type":"v1","addr":"172.21.15.143:6828","nonce":639807369}]},"public_addr":"172.21.15.143:6819/639807369","cluster_addr":"172.21.15.143:6823/639807369","heartbeat_back_addr":"172.21.15.143:6831/639807369","heartbeat_front_addr":"172.21.15.143:6828/639807369","state":["exists","up"]},{"osd":1,"uuid":"fec4b8be-7122-49e9-9a19-5850a0700001","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6816","nonce":1144942530},{"type":"v1","addr":"172.21.15.143:6818","nonce":1144942530}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6820","nonce":1144942530},{"type":"v1","addr":"172.21.15.143:6822","nonce":1144942530}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6827","nonce":1144942530},{"type":"v1","addr":"172.21.15.143:6829","nonce":1144942530}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6824","nonce":1144942530},{"type":"v1","addr":"172.21.15.143:6826","nonce":1144942530}]},"public_addr":"172.21.15.143:6818/1144942530","cluster_addr":"172.21.15.143:6822/1144942530","heartbeat_back_addr":"172.21.15.143:6829/1144942530","heartbeat_front_addr":"172.21.15.143:6826/1144942530","state":["exists","up"]},{"osd":2,"uuid":"2888af5b-f933-46ca-8f95-6dd494ff43f6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6808","nonce":3909095186},{"type":"v1","addr":"172.21.15.143:6809","nonce":3909095186}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6810","nonce":3909095186},{"type":"v1","addr":"172.21.15.143:6811","nonce":3909095186}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6814","nonce":3909095186},{"type":"v1","addr":"172.21.15.143:6815","nonce":3909095186}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6812","nonce":3909095186},{"type":"v1","addr":"172.21.15.143:6813","nonce":3909095186}]},"public_addr":"172.21.15.143:6809/3909095186","cluster_addr":"172.21.15.143:6811/3909095186","heartbeat_back_addr":"172.21.15.143:6815/3909095186","heartbeat_front_addr":"172.21.15.143:6813/3909095186","state":["exists","up"]},{"osd":3,"uuid":"d1f52435-7bc1-4081-9550-9cd1ba55f549","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6800","nonce":3854210991},{"type":"v1","addr":"172.21.15.143:6801","nonce":3854210991}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6802","nonce":3854210991},{"type":"v1","addr":"172.21.15.143:6803","nonce":3854210991}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6806","nonce":3854210991},{"type":"v1","addr":"172.21.15.143:6807","nonce":3854210991}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6804","nonce":3854210991},{"type":"v1","addr":"172.21.15.143:6805","nonce":3854210991}]},"public_addr":"172.21.15.143:6801/3854210991","cluster_addr":"172.21.15.143:6803/3854210991","heartbeat_back_addr":"172.21.15.143:6807/3854210991","heartbeat_front_addr":"172.21.15.143:6805/3854210991","state":["exists","up"]},{"osd":4,"uuid":"e2ae8b6e-6e25-49b2-bcbf-aacce68f06a3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6800","nonce":3924606902},{"type":"v1","addr":"172.21.15.188:6801","nonce":3924606902}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6802","nonce":3924606902},{"type":"v1","addr":"172.21.15.188:6803","nonce":3924606902}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6806","nonce":3924606902},{"type":"v1","addr":"172.21.15.188:6807","nonce":3924606902}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6804","nonce":3924606902},{"type":"v1","addr":"172.21.15.188:6805","nonce":3924606902}]},"public_addr":"172.21.15.188:6801/3924606902","cluster_addr":"172.21.15.188:6803/3924606902","heartbeat_back_addr":"172.21.15.188:6807/3924606902","heartbeat_front_addr":"172.21.15.188:6805/3924606902","state":["exists","up"]},{"osd":5,"uuid":"3cec8a32-7698-4c36-8e89-e4ba11934f33","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6813","nonce":186841544},{"type":"v1","addr":"172.21.15.188:6815","nonce":186841544}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6817","nonce":186841544},{"type":"v1","addr":"172.21.15.188:6818","nonce":186841544}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6822","nonce":186841544},{"type":"v1","addr":"172.21.15.188:6823","nonce":186841544}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6820","nonce":186841544},{"type":"v1","addr":"172.21.15.188:6821","nonce":186841544}]},"public_addr":"172.21.15.188:6815/186841544","cluster_addr":"172.21.15.188:6818/186841544","heartbeat_back_addr":"172.21.15.188:6823/186841544","heartbeat_front_addr":"172.21.15.188:6821/186841544","state":["exists","up"]},{"osd":6,"uuid":"58a65d5d-e090-473a-81b4-ad38e85029f9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6808","nonce":2153499405},{"type":"v1","addr":"172.21.15.188:6809","nonce":2153499405}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6810","nonce":2153499405},{"type":"v1","addr":"172.21.15.188:6811","nonce":2153499405}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6816","nonce":2153499405},{"type":"v1","addr":"172.21.15.188:6819","nonce":2153499405}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6812","nonce":2153499405},{"type":"v1","addr":"172.21.15.188:6814","nonce":2153499405}]},"public_addr":"172.21.15.188:6809/2153499405","cluster_addr":"172.21.15.188:6811/2153499405","heartbeat_back_addr":"172.21.15.188:6819/2153499405","heartbeat_front_addr":"172.21.15.188:6814/2153499405","state":["exists","up"]},{"osd":7,"uuid":"ea5b7f70-3554-4cf8-ac30-f7c59b80678d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6824","nonce":2722096729},{"type":"v1","addr":"172.21.15.188:6825","nonce":2722096729}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6826","nonce":2722096729},{"type":"v1","addr":"172.21.15.188:6827","nonce":2722096729}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6830","nonce":2722096729},{"type":"v1","addr":"172.21.15.188:6831","nonce":2722096729}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6828","nonce":2722096729},{"type":"v1","addr":"172.21.15.188:6829","nonce":2722096729}]},"public_addr":"172.21.15.188:6825/2722096729","cluster_addr":"172.21.15.188:6827/2722096729","heartbeat_back_addr":"172.21.15.188:6831/2722096729","heartbeat_front_addr":"172.21.15.188:6829/2722096729","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-26T07:17:36.995032+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-26T07:17:36.932266+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-26T07:17:36.829304+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-26T07:17:36.804297+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-26T07:17:36.667892+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-26T07:17:36.964868+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-26T07:17:36.827332+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-26T07:17:36.893179+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.143:6837/2658667363":"2023-11-27T07:19:19.073168+0000","172.21.15.143:6836/2658667363":"2023-11-27T07:19:19.073168+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-11-26T07:19:23.418 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:23.416+0000 7fdca17f5640 1 -- 172.21.15.143:0/672036297 >> [v2:172.21.15.143:6832/2150557544,v1:172.21.15.143:6833/2150557544] conn(0x7fdc88066650 msgr2=0x7fdc88068b10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:19:23.419 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:23.416+0000 7fdca17f5640 1 --2- 172.21.15.143:0/672036297 >> [v2:172.21.15.143:6832/2150557544,v1:172.21.15.143:6833/2150557544] conn(0x7fdc88066650 0x7fdc88068b10 secure :-1 s=READY pgs=267 cs=0 l=1 rev1=1 crypto rx=0x7fdc8c0017b0 tx=0x7fdc8c0015c0 comp rx=0 tx=0).stop 2023-11-26T07:19:23.419 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:23.416+0000 7fdca17f5640 1 -- 172.21.15.143:0/672036297 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fdc9c15dcd0 msgr2=0x7fdc9c15e110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:19:23.419 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:23.416+0000 7fdca17f5640 1 --2- 172.21.15.143:0/672036297 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fdc9c15dcd0 0x7fdc9c15e110 secure :-1 s=READY pgs=524 cs=0 l=1 rev1=1 crypto rx=0x7fdc9401ca90 tx=0x7fdc94009750 comp rx=0 tx=0).stop 2023-11-26T07:19:23.419 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:23.416+0000 7fdca17f5640 1 -- 172.21.15.143:0/672036297 shutdown_connections 2023-11-26T07:19:23.419 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:23.416+0000 7fdca17f5640 1 --2- 172.21.15.143:0/672036297 >> [v2:172.21.15.143:6832/2150557544,v1:172.21.15.143:6833/2150557544] conn(0x7fdc88066650 0x7fdc88068b10 unknown :-1 s=CLOSED pgs=267 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:19:23.419 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:23.416+0000 7fdca17f5640 1 --2- 172.21.15.143:0/672036297 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fdc9c15dcd0 0x7fdc9c15e110 unknown :-1 s=CLOSED pgs=524 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:19:23.419 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:23.416+0000 7fdca17f5640 1 --2- 172.21.15.143:0/672036297 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fdc9c0b82b0 0x7fdc9c0b8730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:19:23.419 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:23.416+0000 7fdca17f5640 1 --2- 172.21.15.143:0/672036297 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fdc84005190 0x7fdc9c15a4e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:19:23.420 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:23.416+0000 7fdca17f5640 1 -- 172.21.15.143:0/672036297 >> 172.21.15.143:0/672036297 conn(0x7fdc9c09c550 msgr2=0x7fdc9c15b420 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:19:23.420 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:23.416+0000 7fdca17f5640 1 -- 172.21.15.143:0/672036297 shutdown_connections 2023-11-26T07:19:23.420 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:23.416+0000 7fdca17f5640 1 -- 172.21.15.143:0/672036297 wait complete. 2023-11-26T07:19:23.438 INFO:tasks.ceph_test_case.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-11-26T07:17:41.758597+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2023-11-26T07:19:23.438 DEBUG:teuthology.orchestra.run.smithi143:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-11-26T07:19:23.579 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:23.576+0000 7f3816606640 1 Processor -- start 2023-11-26T07:19:23.579 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:23.576+0000 7f3816606640 1 -- start start 2023-11-26T07:19:23.581 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:23.576+0000 7f3816606640 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7f3810143ac0 0x7f3810070370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:19:23.581 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:23.576+0000 7f3816606640 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7f38100cf990 0x7f38100cd1b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:19:23.582 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:23.576+0000 7f3816606640 1 -- --> v1:172.21.15.143:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f38100cc730 con 0x7f38100708b0 2023-11-26T07:19:23.582 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:23.576+0000 7f3816606640 1 -- --> v2:172.21.15.143:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3810143ea0 con 0x7f3810143ac0 2023-11-26T07:19:23.582 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:23.576+0000 7f3816606640 1 -- --> v2:172.21.15.188:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3810070ee0 con 0x7f38100cf990 2023-11-26T07:19:23.582 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:23.576+0000 7f380ffff640 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7f3810143ac0 0x7f3810070370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:19:23.582 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:23.576+0000 7f380f7fe640 1 --1- >> v1:172.21.15.143:6789/0 conn(0x7f38100708b0 0x7f38100cf280 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.143:6789/0 says I am v1:172.21.15.143:50196/0 (socket says 172.21.15.143:50196) 2023-11-26T07:19:23.582 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:23.576+0000 7f380f7fe640 1 -- 172.21.15.143:0/2515747225 learned_addr learned my addr 172.21.15.143:0/2515747225 (peer_addr_for_me v1:172.21.15.143:0/0) 2023-11-26T07:19:23.582 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:23.580+0000 7f3814ba3640 1 --2- 172.21.15.143:0/2515747225 >> v2:172.21.15.188:3300/0 conn(0x7f38100cf990 0x7f38100cd1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:19:23.582 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:23.580+0000 7f380ffff640 1 -- 172.21.15.143:0/2515747225 >> v2:172.21.15.188:3300/0 conn(0x7f38100cf990 msgr2=0x7f38100cd1b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-26T07:19:23.582 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:23.580+0000 7f380ffff640 1 --2- 172.21.15.143:0/2515747225 >> v2:172.21.15.188:3300/0 conn(0x7f38100cf990 0x7f38100cd1b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:19:23.583 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:23.580+0000 7f380ffff640 1 -- 172.21.15.143:0/2515747225 >> v1:172.21.15.143:6789/0 conn(0x7f38100708b0 legacy=0x7f38100cf280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:19:23.583 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:23.580+0000 7f380ffff640 1 -- 172.21.15.143:0/2515747225 --> v2:172.21.15.143:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f38100711c0 con 0x7f3810143ac0 2023-11-26T07:19:23.583 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:23.580+0000 7f380effd640 1 -- 172.21.15.143:0/2515747225 <== mon.0 v1:172.21.15.143:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1354988573 0 0) 0x7f38100cc730 con 0x7f38100708b0 2023-11-26T07:19:23.584 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:23.580+0000 7f380ffff640 1 --2- 172.21.15.143:0/2515747225 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f3810143ac0 0x7f3810070370 secure :-1 s=READY pgs=526 cs=0 l=1 rev1=1 crypto rx=0x7f381006e570 tx=0x7f380001c460 comp rx=0 tx=0).ready entity=mon.0 client_cookie=befaf604a0727d89 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:19:23.584 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:23.580+0000 7f380effd640 1 -- 172.21.15.143:0/2515747225 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f38000ab600 con 0x7f3810143ac0 2023-11-26T07:19:23.585 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:23.580+0000 7f380effd640 1 -- 172.21.15.143:0/2515747225 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f38000a7050 con 0x7f3810143ac0 2023-11-26T07:19:23.585 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:23.580+0000 7f380effd640 1 -- 172.21.15.143:0/2515747225 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f38000ab980 con 0x7f3810143ac0 2023-11-26T07:19:23.586 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:23.584+0000 7f3816606640 1 -- 172.21.15.143:0/2515747225 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f3810143ac0 msgr2=0x7f3810070370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:19:23.586 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:23.584+0000 7f3816606640 1 --2- 172.21.15.143:0/2515747225 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f3810143ac0 0x7f3810070370 secure :-1 s=READY pgs=526 cs=0 l=1 rev1=1 crypto rx=0x7f381006e570 tx=0x7f380001c460 comp rx=0 tx=0).stop 2023-11-26T07:19:23.586 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:23.584+0000 7f3816606640 1 -- 172.21.15.143:0/2515747225 shutdown_connections 2023-11-26T07:19:23.586 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:23.584+0000 7f3816606640 1 --2- 172.21.15.143:0/2515747225 >> v2:172.21.15.188:3300/0 conn(0x7f38100cf990 0x7f38100cd1b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:19:23.586 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:23.584+0000 7f3816606640 1 --2- 172.21.15.143:0/2515747225 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f3810143ac0 0x7f3810070370 unknown :-1 s=CLOSED pgs=526 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:19:23.586 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:23.584+0000 7f3816606640 1 -- 172.21.15.143:0/2515747225 >> 172.21.15.143:0/2515747225 conn(0x7f381006a890 msgr2=0x7f381006aca0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:19:23.586 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:23.584+0000 7f3816606640 1 -- 172.21.15.143:0/2515747225 shutdown_connections 2023-11-26T07:19:23.586 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:23.584+0000 7f3816606640 1 -- 172.21.15.143:0/2515747225 wait complete. 2023-11-26T07:19:23.587 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:23.584+0000 7f3816606640 1 Processor -- start 2023-11-26T07:19:23.587 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:23.584+0000 7f3816606640 1 -- start start 2023-11-26T07:19:23.589 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:23.584+0000 7f3816606640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f38100708b0 0x7f38100b16e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:19:23.589 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:23.584+0000 7f3816606640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f38100cf990 0x7f38100ad2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:19:23.589 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:23.584+0000 7f3816606640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f3810143ac0 0x7f3810139920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:19:23.589 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:23.584+0000 7f3816606640 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f381013a1e0 con 0x7f3810143ac0 2023-11-26T07:19:23.589 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:23.584+0000 7f3816606640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f38100ad830 con 0x7f38100cf990 2023-11-26T07:19:23.589 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:23.584+0000 7f3816606640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f38100adad0 con 0x7f38100708b0 2023-11-26T07:19:23.589 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:23.584+0000 7f3814ba3640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f3810143ac0 0x7f3810139920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:19:23.590 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:23.584+0000 7f3814ba3640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f3810143ac0 0x7f3810139920 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:48174/0 (socket says 172.21.15.143:48174) 2023-11-26T07:19:23.590 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:23.584+0000 7f380ffff640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f38100708b0 0x7f38100b16e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:19:23.590 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:23.584+0000 7f3814ba3640 1 -- 172.21.15.143:0/4286753404 learned_addr learned my addr 172.21.15.143:0/4286753404 (peer_addr_for_me v2:172.21.15.143:0/0) 2023-11-26T07:19:23.590 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:23.584+0000 7f380f7fe640 1 --2- 172.21.15.143:0/4286753404 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f38100cf990 0x7f38100ad2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:19:23.590 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:23.588+0000 7f3814ba3640 1 -- 172.21.15.143:0/4286753404 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f38100708b0 msgr2=0x7f38100b16e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:19:23.590 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:23.588+0000 7f3814ba3640 1 --2- 172.21.15.143:0/4286753404 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f38100708b0 0x7f38100b16e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:19:23.590 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:23.588+0000 7f3814ba3640 1 -- 172.21.15.143:0/4286753404 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f38100cf990 msgr2=0x7f38100ad2f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:19:23.590 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:23.588+0000 7f3814ba3640 1 --2- 172.21.15.143:0/4286753404 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f38100cf990 0x7f38100ad2f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:19:23.591 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:23.588+0000 7f3814ba3640 1 -- 172.21.15.143:0/4286753404 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3800009420 con 0x7f3810143ac0 2023-11-26T07:19:23.591 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:23.588+0000 7f3814ba3640 1 --2- 172.21.15.143:0/4286753404 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f3810143ac0 0x7f3810139920 secure :-1 s=READY pgs=527 cs=0 l=1 rev1=1 crypto rx=0x7f3808007e40 tx=0x7f380800e4b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:19:23.592 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:23.588+0000 7f380cff9640 1 -- 172.21.15.143:0/4286753404 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3808014620 con 0x7f3810143ac0 2023-11-26T07:19:23.593 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:23.588+0000 7f380cff9640 1 -- 172.21.15.143:0/4286753404 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f380800ee90 con 0x7f3810143ac0 2023-11-26T07:19:23.593 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:23.588+0000 7f3816606640 1 -- 172.21.15.143:0/4286753404 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f38100addb0 con 0x7f3810143ac0 2023-11-26T07:19:23.593 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:23.588+0000 7f380cff9640 1 -- 172.21.15.143:0/4286753404 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f38080149e0 con 0x7f3810143ac0 2023-11-26T07:19:23.593 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:23.588+0000 7f3816606640 1 -- 172.21.15.143:0/4286753404 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f381013a3f0 con 0x7f3810143ac0 2023-11-26T07:19:23.593 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:23.588+0000 7f3816606640 1 -- 172.21.15.143:0/4286753404 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f38100ae2d0 con 0x7f3810143ac0 2023-11-26T07:19:23.593 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:23.588+0000 7f380cff9640 1 -- 172.21.15.143:0/4286753404 <== mon.0 v2:172.21.15.143:3300/0 4 ==== mgrmap(e 7) v1 ==== 86005+0+0 (secure 0 0 0) 0x7f3808014b40 con 0x7f3810143ac0 2023-11-26T07:19:23.593 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:23.588+0000 7f380cff9640 1 --2- 172.21.15.143:0/4286753404 >> [v2:172.21.15.143:6832/2150557544,v1:172.21.15.143:6833/2150557544] conn(0x7f37fc066620 0x7f37fc068ae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:19:23.593 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:23.588+0000 7f380cff9640 1 -- 172.21.15.143:0/4286753404 <== mon.0 v2:172.21.15.143:3300/0 5 ==== osd_map(36..36 src has 1..36) v4 ==== 4943+0+0 (secure 0 0 0) 0x7f38080846e0 con 0x7f3810143ac0 2023-11-26T07:19:23.593 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:23.588+0000 7f380ffff640 1 --2- 172.21.15.143:0/4286753404 >> [v2:172.21.15.143:6832/2150557544,v1:172.21.15.143:6833/2150557544] conn(0x7f37fc066620 0x7f37fc068ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:19:23.594 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:23.592+0000 7f380ffff640 1 --2- 172.21.15.143:0/4286753404 >> [v2:172.21.15.143:6832/2150557544,v1:172.21.15.143:6833/2150557544] conn(0x7f37fc066620 0x7f37fc068ae0 secure :-1 s=READY pgs=268 cs=0 l=1 rev1=1 crypto rx=0x7f38000ad000 tx=0x7f3800008040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:19:23.596 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:23.592+0000 7f380cff9640 1 -- 172.21.15.143:0/4286753404 <== mon.0 v2:172.21.15.143:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172649 (secure 0 0 0) 0x7f3808051c90 con 0x7f3810143ac0 2023-11-26T07:19:23.750 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:23.748+0000 7f3816606640 1 -- 172.21.15.143:0/4286753404 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f381015d560 con 0x7f3810143ac0 2023-11-26T07:19:23.751 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:23.748+0000 7f380cff9640 1 -- 172.21.15.143:0/4286753404 <== mon.0 v2:172.21.15.143:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v36) v1 ==== 93+0+10 (secure 0 0 0) 0x7f38080552b0 con 0x7f3810143ac0 2023-11-26T07:19:23.751 INFO:teuthology.orchestra.run.smithi143.stdout:pg_num: 1 2023-11-26T07:19:23.753 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:23.752+0000 7f3816606640 1 -- 172.21.15.143:0/4286753404 >> [v2:172.21.15.143:6832/2150557544,v1:172.21.15.143:6833/2150557544] conn(0x7f37fc066620 msgr2=0x7f37fc068ae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:19:23.753 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:23.752+0000 7f3816606640 1 --2- 172.21.15.143:0/4286753404 >> [v2:172.21.15.143:6832/2150557544,v1:172.21.15.143:6833/2150557544] conn(0x7f37fc066620 0x7f37fc068ae0 secure :-1 s=READY pgs=268 cs=0 l=1 rev1=1 crypto rx=0x7f38000ad000 tx=0x7f3800008040 comp rx=0 tx=0).stop 2023-11-26T07:19:23.754 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:23.752+0000 7f3816606640 1 -- 172.21.15.143:0/4286753404 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f3810143ac0 msgr2=0x7f3810139920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:19:23.754 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:23.752+0000 7f3816606640 1 --2- 172.21.15.143:0/4286753404 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f3810143ac0 0x7f3810139920 secure :-1 s=READY pgs=527 cs=0 l=1 rev1=1 crypto rx=0x7f3808007e40 tx=0x7f380800e4b0 comp rx=0 tx=0).stop 2023-11-26T07:19:23.754 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:23.752+0000 7f3816606640 1 -- 172.21.15.143:0/4286753404 shutdown_connections 2023-11-26T07:19:23.754 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:23.752+0000 7f3816606640 1 --2- 172.21.15.143:0/4286753404 >> [v2:172.21.15.143:6832/2150557544,v1:172.21.15.143:6833/2150557544] conn(0x7f37fc066620 0x7f37fc068ae0 unknown :-1 s=CLOSED pgs=268 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:19:23.754 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:23.752+0000 7f3816606640 1 --2- 172.21.15.143:0/4286753404 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f3810143ac0 0x7f3810139920 unknown :-1 s=CLOSED pgs=527 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:19:23.754 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:23.752+0000 7f3816606640 1 --2- 172.21.15.143:0/4286753404 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f38100cf990 0x7f38100ad2f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:19:23.754 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:23.752+0000 7f3816606640 1 --2- 172.21.15.143:0/4286753404 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f38100708b0 0x7f38100b16e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:19:23.754 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:23.752+0000 7f3816606640 1 -- 172.21.15.143:0/4286753404 >> 172.21.15.143:0/4286753404 conn(0x7f381006a890 msgr2=0x7f381015c5e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:19:23.755 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:23.752+0000 7f3816606640 1 -- 172.21.15.143:0/4286753404 shutdown_connections 2023-11-26T07:19:23.755 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:23.752+0000 7f3816606640 1 -- 172.21.15.143:0/4286753404 wait complete. 2023-11-26T07:19:23.772 DEBUG:teuthology.orchestra.run.smithi143:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Starting test tasks.cephfs.test_volumes.TestMisc.test_connection_expiration' 2023-11-26T07:19:23.911 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:23.908+0000 7f4ca0e32640 1 Processor -- start 2023-11-26T07:19:23.911 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:23.908+0000 7f4ca0e32640 1 -- start start 2023-11-26T07:19:23.913 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:23.908+0000 7f4ca0e32640 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7f4c9c0cf280 0x7f4c9c0cf710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:19:23.913 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:23.908+0000 7f4ca0e32640 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7f4c9c0cd1b0 0x7f4c9c0cd5c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:19:23.913 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:23.908+0000 7f4ca0e32640 1 -- --> v1:172.21.15.188:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4c9c0cc730 con 0x7f4c9c143ac0 2023-11-26T07:19:23.913 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:23.908+0000 7f4ca0e32640 1 -- --> v2:172.21.15.143:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4c9c143ea0 con 0x7f4c9c0cf280 2023-11-26T07:19:23.913 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:23.908+0000 7f4ca0e32640 1 -- --> v2:172.21.15.188:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4c9c0cfc50 con 0x7f4c9c0cd1b0 2023-11-26T07:19:23.913 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:23.908+0000 7f4c99d9b640 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7f4c9c0cf280 0x7f4c9c0cf710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:19:23.914 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:23.908+0000 7f4c99d9b640 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7f4c9c0cf280 0x7f4c9c0cf710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:48184/0 (socket says 172.21.15.143:48184) 2023-11-26T07:19:23.914 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:23.908+0000 7f4c99d9b640 1 -- 172.21.15.143:0/1658410015 learned_addr learned my addr 172.21.15.143:0/1658410015 (peer_addr_for_me v2:172.21.15.143:0/0) 2023-11-26T07:19:23.914 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:23.908+0000 7f4c9ad9d640 1 --2- 172.21.15.143:0/1658410015 >> v2:172.21.15.188:3300/0 conn(0x7f4c9c0cd1b0 0x7f4c9c0cd5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:19:23.914 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:23.912+0000 7f4c99d9b640 1 -- 172.21.15.143:0/1658410015 >> v2:172.21.15.188:3300/0 conn(0x7f4c9c0cd1b0 msgr2=0x7f4c9c0cd5c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-26T07:19:23.914 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:23.912+0000 7f4c99d9b640 1 --2- 172.21.15.143:0/1658410015 >> v2:172.21.15.188:3300/0 conn(0x7f4c9c0cd1b0 0x7f4c9c0cd5c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:19:23.914 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:23.912+0000 7f4c99d9b640 1 -- 172.21.15.143:0/1658410015 >> v1:172.21.15.188:6789/0 conn(0x7f4c9c143ac0 legacy=0x7f4c9c070370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:19:23.914 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:23.912+0000 7f4c99d9b640 1 -- 172.21.15.143:0/1658410015 --> v2:172.21.15.143:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4c9c070ee0 con 0x7f4c9c0cf280 2023-11-26T07:19:23.915 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:23.912+0000 7f4c99d9b640 1 --2- 172.21.15.143:0/1658410015 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f4c9c0cf280 0x7f4c9c0cf710 secure :-1 s=READY pgs=528 cs=0 l=1 rev1=1 crypto rx=0x7f4c9c06e570 tx=0x7f4c9001c500 comp rx=0 tx=0).ready entity=mon.0 client_cookie=958c97ee2c655b63 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:19:23.916 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:23.912+0000 7f4c9959a640 1 -- 172.21.15.143:0/1658410015 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4c900aa660 con 0x7f4c9c0cf280 2023-11-26T07:19:23.916 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:23.912+0000 7f4c9959a640 1 -- 172.21.15.143:0/1658410015 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4c900aa7c0 con 0x7f4c9c0cf280 2023-11-26T07:19:23.917 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:23.912+0000 7f4c9959a640 1 -- 172.21.15.143:0/1658410015 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4c900aab50 con 0x7f4c9c0cf280 2023-11-26T07:19:23.917 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:23.912+0000 7f4ca0e32640 1 -- 172.21.15.143:0/1658410015 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f4c9c0cf280 msgr2=0x7f4c9c0cf710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:19:23.917 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:23.916+0000 7f4ca0e32640 1 --2- 172.21.15.143:0/1658410015 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f4c9c0cf280 0x7f4c9c0cf710 secure :-1 s=READY pgs=528 cs=0 l=1 rev1=1 crypto rx=0x7f4c9c06e570 tx=0x7f4c9001c500 comp rx=0 tx=0).stop 2023-11-26T07:19:23.918 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:23.916+0000 7f4ca0e32640 1 -- 172.21.15.143:0/1658410015 shutdown_connections 2023-11-26T07:19:23.918 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:23.916+0000 7f4ca0e32640 1 --2- 172.21.15.143:0/1658410015 >> v2:172.21.15.188:3300/0 conn(0x7f4c9c0cd1b0 0x7f4c9c0cd5c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:19:23.918 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:23.916+0000 7f4ca0e32640 1 --2- 172.21.15.143:0/1658410015 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f4c9c0cf280 0x7f4c9c0cf710 unknown :-1 s=CLOSED pgs=528 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:19:23.918 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:23.916+0000 7f4ca0e32640 1 -- 172.21.15.143:0/1658410015 >> 172.21.15.143:0/1658410015 conn(0x7f4c9c06a890 msgr2=0x7f4c9c06aca0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:19:23.918 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:23.916+0000 7f4ca0e32640 1 -- 172.21.15.143:0/1658410015 shutdown_connections 2023-11-26T07:19:23.918 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:23.916+0000 7f4ca0e32640 1 -- 172.21.15.143:0/1658410015 wait complete. 2023-11-26T07:19:23.919 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:23.916+0000 7f4ca0e32640 1 Processor -- start 2023-11-26T07:19:23.919 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:23.916+0000 7f4ca0e32640 1 -- start start 2023-11-26T07:19:23.922 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:23.916+0000 7f4ca0e32640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f4c9c0cd1b0 0x7f4c9c159db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:19:23.922 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:23.916+0000 7f4ca0e32640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f4c9c0cf280 0x7f4c9c13ae90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:19:23.922 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:23.916+0000 7f4ca0e32640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f4c9c143ac0 0x7f4c9c13b3d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:19:23.922 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:23.916+0000 7f4ca0e32640 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4c9c0afd60 con 0x7f4c9c143ac0 2023-11-26T07:19:23.922 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:23.916+0000 7f4ca0e32640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4c9c15a2f0 con 0x7f4c9c0cd1b0 2023-11-26T07:19:23.922 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:23.916+0000 7f4ca0e32640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4c9c13ba70 con 0x7f4c9c0cf280 2023-11-26T07:19:23.922 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:23.916+0000 7f4c9ad9d640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f4c9c143ac0 0x7f4c9c13b3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:19:23.922 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:23.916+0000 7f4c9ad9d640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f4c9c143ac0 0x7f4c9c13b3d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:48194/0 (socket says 172.21.15.143:48194) 2023-11-26T07:19:23.922 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:23.916+0000 7f4c9ad9d640 1 -- 172.21.15.143:0/1266848731 learned_addr learned my addr 172.21.15.143:0/1266848731 (peer_addr_for_me v2:172.21.15.143:0/0) 2023-11-26T07:19:23.923 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:23.920+0000 7f4c99d9b640 1 --2- 172.21.15.143:0/1266848731 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f4c9c0cf280 0x7f4c9c13ae90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:19:23.923 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:23.920+0000 7f4c9ad9d640 1 -- 172.21.15.143:0/1266848731 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f4c9c0cf280 msgr2=0x7f4c9c13ae90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:19:23.923 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:23.920+0000 7f4c9a59c640 1 --2- 172.21.15.143:0/1266848731 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f4c9c0cd1b0 0x7f4c9c159db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:19:23.923 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:23.920+0000 7f4c9ad9d640 1 --2- 172.21.15.143:0/1266848731 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f4c9c0cf280 0x7f4c9c13ae90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:19:23.923 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:23.920+0000 7f4c9ad9d640 1 -- 172.21.15.143:0/1266848731 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f4c9c0cd1b0 msgr2=0x7f4c9c159db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:19:23.923 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:23.920+0000 7f4c9ad9d640 1 --2- 172.21.15.143:0/1266848731 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f4c9c0cd1b0 0x7f4c9c159db0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:19:23.923 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:23.920+0000 7f4c9ad9d640 1 -- 172.21.15.143:0/1266848731 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4c900094d0 con 0x7f4c9c143ac0 2023-11-26T07:19:23.923 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:23.920+0000 7f4c9a59c640 1 --2- 172.21.15.143:0/1266848731 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f4c9c0cd1b0 0x7f4c9c159db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-26T07:19:23.923 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:23.920+0000 7f4c99d9b640 1 --2- 172.21.15.143:0/1266848731 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f4c9c0cf280 0x7f4c9c13ae90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-26T07:19:23.923 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:23.920+0000 7f4c9ad9d640 1 --2- 172.21.15.143:0/1266848731 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f4c9c143ac0 0x7f4c9c13b3d0 secure :-1 s=READY pgs=529 cs=0 l=1 rev1=1 crypto rx=0x7f4c9400efc0 tx=0x7f4c9400c490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:19:23.923 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:23.920+0000 7f4c8affd640 1 -- 172.21.15.143:0/1266848731 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4c94009ba0 con 0x7f4c9c143ac0 2023-11-26T07:19:23.923 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:23.920+0000 7f4c8affd640 1 -- 172.21.15.143:0/1266848731 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4c9400f040 con 0x7f4c9c143ac0 2023-11-26T07:19:23.923 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:23.920+0000 7f4c8affd640 1 -- 172.21.15.143:0/1266848731 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4c940146c0 con 0x7f4c9c143ac0 2023-11-26T07:19:23.924 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:23.920+0000 7f4ca0e32640 1 -- 172.21.15.143:0/1266848731 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4c9c13bd50 con 0x7f4c9c143ac0 2023-11-26T07:19:23.924 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:23.920+0000 7f4ca0e32640 1 -- 172.21.15.143:0/1266848731 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4c9c0b02e0 con 0x7f4c9c143ac0 2023-11-26T07:19:23.925 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:23.920+0000 7f4ca0e32640 1 -- 172.21.15.143:0/1266848731 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4c9c067680 con 0x7f4c9c143ac0 2023-11-26T07:19:23.925 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:23.920+0000 7f4c8affd640 1 -- 172.21.15.143:0/1266848731 <== mon.0 v2:172.21.15.143:3300/0 4 ==== mgrmap(e 7) v1 ==== 86005+0+0 (secure 0 0 0) 0x7f4c94014820 con 0x7f4c9c143ac0 2023-11-26T07:19:23.925 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:23.920+0000 7f4c8affd640 1 --2- 172.21.15.143:0/1266848731 >> [v2:172.21.15.143:6832/2150557544,v1:172.21.15.143:6833/2150557544] conn(0x7f4c7c066620 0x7f4c7c068ae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:19:23.925 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:23.920+0000 7f4c8affd640 1 -- 172.21.15.143:0/1266848731 <== mon.0 v2:172.21.15.143:3300/0 5 ==== osd_map(36..36 src has 1..36) v4 ==== 4943+0+0 (secure 0 0 0) 0x7f4c94084e50 con 0x7f4c9c143ac0 2023-11-26T07:19:23.925 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:23.920+0000 7f4c9a59c640 1 --2- 172.21.15.143:0/1266848731 >> [v2:172.21.15.143:6832/2150557544,v1:172.21.15.143:6833/2150557544] conn(0x7f4c7c066620 0x7f4c7c068ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:19:23.925 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:23.924+0000 7f4c9a59c640 1 --2- 172.21.15.143:0/1266848731 >> [v2:172.21.15.143:6832/2150557544,v1:172.21.15.143:6833/2150557544] conn(0x7f4c7c066620 0x7f4c7c068ae0 secure :-1 s=READY pgs=269 cs=0 l=1 rev1=1 crypto rx=0x7f4c8c000f70 tx=0x7f4c8c009040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:19:23.928 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:23.924+0000 7f4c8affd640 1 -- 172.21.15.143:0/1266848731 <== mon.0 v2:172.21.15.143:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172649 (secure 0 0 0) 0x7f4c940521a0 con 0x7f4c9c143ac0 2023-11-26T07:19:24.131 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:24.128+0000 7f4ca0e32640 1 -- 172.21.15.143:0/1266848731 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_volumes.TestMisc.test_connection_expiration"]} v 0) v1 -- 0x7f4c9c0aef20 con 0x7f4c9c143ac0 2023-11-26T07:19:25.093 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:25.088+0000 7f4c8affd640 1 -- 172.21.15.143:0/1266848731 <== mon.0 v2:172.21.15.143:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_volumes.TestMisc.test_connection_expiration"]}]=0 v107) v1 ==== 143+0+0 (secure 0 0 0) 0x7f4c940557c0 con 0x7f4c9c143ac0 2023-11-26T07:19:25.095 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:25.092+0000 7f4ca0e32640 1 -- 172.21.15.143:0/1266848731 >> [v2:172.21.15.143:6832/2150557544,v1:172.21.15.143:6833/2150557544] conn(0x7f4c7c066620 msgr2=0x7f4c7c068ae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:19:25.095 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:25.092+0000 7f4ca0e32640 1 --2- 172.21.15.143:0/1266848731 >> [v2:172.21.15.143:6832/2150557544,v1:172.21.15.143:6833/2150557544] conn(0x7f4c7c066620 0x7f4c7c068ae0 secure :-1 s=READY pgs=269 cs=0 l=1 rev1=1 crypto rx=0x7f4c8c000f70 tx=0x7f4c8c009040 comp rx=0 tx=0).stop 2023-11-26T07:19:25.096 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:25.092+0000 7f4ca0e32640 1 -- 172.21.15.143:0/1266848731 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f4c9c143ac0 msgr2=0x7f4c9c13b3d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:19:25.096 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:25.092+0000 7f4ca0e32640 1 --2- 172.21.15.143:0/1266848731 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f4c9c143ac0 0x7f4c9c13b3d0 secure :-1 s=READY pgs=529 cs=0 l=1 rev1=1 crypto rx=0x7f4c9400efc0 tx=0x7f4c9400c490 comp rx=0 tx=0).stop 2023-11-26T07:19:25.096 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:25.092+0000 7f4ca0e32640 1 -- 172.21.15.143:0/1266848731 shutdown_connections 2023-11-26T07:19:25.096 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:25.092+0000 7f4ca0e32640 1 --2- 172.21.15.143:0/1266848731 >> [v2:172.21.15.143:6832/2150557544,v1:172.21.15.143:6833/2150557544] conn(0x7f4c7c066620 0x7f4c7c068ae0 unknown :-1 s=CLOSED pgs=269 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:19:25.096 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:25.092+0000 7f4ca0e32640 1 --2- 172.21.15.143:0/1266848731 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f4c9c143ac0 0x7f4c9c13b3d0 unknown :-1 s=CLOSED pgs=529 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:19:25.096 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:25.092+0000 7f4ca0e32640 1 --2- 172.21.15.143:0/1266848731 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f4c9c0cf280 0x7f4c9c13ae90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:19:25.096 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:25.092+0000 7f4ca0e32640 1 --2- 172.21.15.143:0/1266848731 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f4c9c0cd1b0 0x7f4c9c159db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:19:25.096 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:25.092+0000 7f4ca0e32640 1 -- 172.21.15.143:0/1266848731 >> 172.21.15.143:0/1266848731 conn(0x7f4c9c06a890 msgr2=0x7f4c9c0cb0e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:19:25.096 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:25.092+0000 7f4ca0e32640 1 -- 172.21.15.143:0/1266848731 shutdown_connections 2023-11-26T07:19:25.096 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:25.092+0000 7f4ca0e32640 1 -- 172.21.15.143:0/1266848731 wait complete. 2023-11-26T07:19:25.118 DEBUG:teuthology.orchestra.run.smithi143:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mon mon_allow_pool_delete True 2023-11-26T07:19:25.261 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:25.260+0000 7f8890dd1640 1 Processor -- start 2023-11-26T07:19:25.261 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:25.260+0000 7f8890dd1640 1 -- start start 2023-11-26T07:19:25.263 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:25.260+0000 7f8890dd1640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f888c1596d0 0x7f888c071850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:19:25.263 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:25.260+0000 7f8890dd1640 1 -- --> v1:172.21.15.143:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f888c06c750 con 0x7f888c0ce680 2023-11-26T07:19:25.263 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:25.260+0000 7f8890dd1640 1 -- --> v1:172.21.15.188:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f888c0b3550 con 0x7f888c0b2f80 2023-11-26T07:19:25.263 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:25.260+0000 7f8890dd1640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f888c0cbe00 con 0x7f888c1596d0 2023-11-26T07:19:25.263 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:25.260+0000 7f8889d9b640 1 --1- >> v1:172.21.15.143:6789/0 conn(0x7f888c0ce680 0x7f888c0b2870 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.143:6789/0 says I am v1:172.21.15.143:58314/0 (socket says 172.21.15.143:58314) 2023-11-26T07:19:25.263 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:25.260+0000 7f8889d9b640 1 -- 172.21.15.143:0/1910955046 learned_addr learned my addr 172.21.15.143:0/1910955046 (peer_addr_for_me v1:172.21.15.143:0/0) 2023-11-26T07:19:25.265 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:25.260+0000 7f888a59c640 1 --2- 172.21.15.143:0/1910955046 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f888c1596d0 0x7f888c071850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:19:25.265 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:25.260+0000 7f888959a640 1 -- 172.21.15.143:0/1910955046 <== mon.0 v1:172.21.15.143:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 752009831 0 0) 0x7f888c06c750 con 0x7f888c0ce680 2023-11-26T07:19:25.265 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:25.260+0000 7f888959a640 1 -- 172.21.15.143:0/1910955046 --> v1:172.21.15.143:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f8874003530 con 0x7f888c0ce680 2023-11-26T07:19:25.265 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:25.260+0000 7f888959a640 1 -- 172.21.15.143:0/1910955046 <== mon.0 v1:172.21.15.143:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2487583004 0 0) 0x7f8874003530 con 0x7f888c0ce680 2023-11-26T07:19:25.265 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:25.260+0000 7f888959a640 1 -- 172.21.15.143:0/1910955046 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f888c1596d0 msgr2=0x7f888c071850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-26T07:19:25.265 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:25.260+0000 7f888959a640 1 --2- 172.21.15.143:0/1910955046 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f888c1596d0 0x7f888c071850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:19:25.265 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:25.260+0000 7f888959a640 1 -- 172.21.15.143:0/1910955046 >> v1:172.21.15.188:6789/0 conn(0x7f888c0b2f80 legacy=0x7f888c06cbe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:19:25.266 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:25.260+0000 7f888959a640 1 -- 172.21.15.143:0/1910955046 --> v1:172.21.15.143:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f888c071e80 con 0x7f888c0ce680 2023-11-26T07:19:25.266 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:25.260+0000 7f888959a640 1 -- 172.21.15.143:0/1910955046 <== mon.0 v1:172.21.15.143:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2137882523 0 0) 0x7f8880001680 con 0x7f888c0ce680 2023-11-26T07:19:25.266 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:25.260+0000 7f888959a640 1 -- 172.21.15.143:0/1910955046 >> v1:172.21.15.143:6789/0 conn(0x7f888c0ce680 legacy=0x7f888c0b2870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:19:25.266 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:25.260+0000 7f888959a640 1 --2- 172.21.15.143:0/1910955046 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f88740039b0 0x7f8874003d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:19:25.266 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:25.260+0000 7f888959a640 1 --2- 172.21.15.143:0/1910955046 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f888c1596d0 0x7f88740047d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:19:25.266 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:25.260+0000 7f888959a640 1 --2- 172.21.15.143:0/1910955046 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f8874004d10 0x7f8874005190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:19:25.266 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:25.260+0000 7f888959a640 1 -- 172.21.15.143:0/1910955046 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f888c0cbe00 con 0x7f88740039b0 2023-11-26T07:19:25.266 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:25.260+0000 7f888959a640 1 -- 172.21.15.143:0/1910955046 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f887400b990 con 0x7f8874004d10 2023-11-26T07:19:25.266 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:25.260+0000 7f888959a640 1 -- 172.21.15.143:0/1910955046 --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f887400dd60 con 0x7f888c1596d0 2023-11-26T07:19:25.267 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:25.260+0000 7f888a59c640 1 --2- 172.21.15.143:0/1910955046 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f88740039b0 0x7f8874003d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:19:25.267 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:25.260+0000 7f8889d9b640 1 --2- 172.21.15.143:0/1910955046 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f888c1596d0 0x7f88740047d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:19:25.267 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:25.260+0000 7f888ad9d640 1 --2- 172.21.15.143:0/1910955046 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f8874004d10 0x7f8874005190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:19:25.267 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:25.260+0000 7f888a59c640 1 -- 172.21.15.143:0/1910955046 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f888c1596d0 msgr2=0x7f88740047d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-26T07:19:25.267 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:25.260+0000 7f888a59c640 1 --2- 172.21.15.143:0/1910955046 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f888c1596d0 0x7f88740047d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:19:25.267 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:25.260+0000 7f888a59c640 1 -- 172.21.15.143:0/1910955046 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f8874004d10 msgr2=0x7f8874005190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-26T07:19:25.267 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:25.260+0000 7f888a59c640 1 --2- 172.21.15.143:0/1910955046 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f8874004d10 0x7f8874005190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:19:25.267 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:25.260+0000 7f888a59c640 1 -- 172.21.15.143:0/1910955046 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8874003530 con 0x7f88740039b0 2023-11-26T07:19:25.267 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:25.260+0000 7f8889d9b640 1 --2- 172.21.15.143:0/1910955046 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f888c1596d0 0x7f88740047d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-26T07:19:25.269 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:25.264+0000 7f888a59c640 1 --2- 172.21.15.143:0/1910955046 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f88740039b0 0x7f8874003d90 secure :-1 s=READY pgs=531 cs=0 l=1 rev1=1 crypto rx=0x7f887c00aa60 tx=0x7f887c01cb50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=16bd798507f4886c server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:19:25.269 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:25.264+0000 7f888959a640 1 -- 172.21.15.143:0/1910955046 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f887c0a5b90 con 0x7f88740039b0 2023-11-26T07:19:25.269 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:25.264+0000 7f888959a640 1 -- 172.21.15.143:0/1910955046 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f887c0a5cf0 con 0x7f88740039b0 2023-11-26T07:19:25.269 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:25.264+0000 7f888959a640 1 -- 172.21.15.143:0/1910955046 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f887c0ad680 con 0x7f88740039b0 2023-11-26T07:19:25.269 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:25.264+0000 7f8890dd1640 1 -- 172.21.15.143:0/1910955046 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f88740039b0 msgr2=0x7f8874003d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:19:25.269 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:25.264+0000 7f8890dd1640 1 --2- 172.21.15.143:0/1910955046 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f88740039b0 0x7f8874003d90 secure :-1 s=READY pgs=531 cs=0 l=1 rev1=1 crypto rx=0x7f887c00aa60 tx=0x7f887c01cb50 comp rx=0 tx=0).stop 2023-11-26T07:19:25.269 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:25.264+0000 7f888ad9d640 1 -- 172.21.15.143:0/1910955046 reap_dead start 2023-11-26T07:19:25.269 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:25.264+0000 7f8890dd1640 1 -- 172.21.15.143:0/1910955046 shutdown_connections 2023-11-26T07:19:25.269 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:25.264+0000 7f8890dd1640 1 -- 172.21.15.143:0/1910955046 >> 172.21.15.143:0/1910955046 conn(0x7f888c06a890 msgr2=0x7f888c06aca0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:19:25.270 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:25.264+0000 7f8890dd1640 1 -- 172.21.15.143:0/1910955046 shutdown_connections 2023-11-26T07:19:25.270 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:25.264+0000 7f8890dd1640 1 -- 172.21.15.143:0/1910955046 wait complete. 2023-11-26T07:19:25.270 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:25.268+0000 7f8890dd1640 1 Processor -- start 2023-11-26T07:19:25.270 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:25.268+0000 7f8890dd1640 1 -- start start 2023-11-26T07:19:25.273 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:25.268+0000 7f8890dd1640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f88740039b0 0x7f888c0c3e60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:19:25.273 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:25.268+0000 7f8890dd1640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f888c0c43a0 0x7f888c1f6260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:19:25.273 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:25.268+0000 7f8890dd1640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f888c1f67a0 0x7f888c0cac70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:19:25.273 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:25.268+0000 7f8890dd1640 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f888c157fd0 con 0x7f888c0c43a0 2023-11-26T07:19:25.273 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:25.268+0000 7f8890dd1640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f888c1f6c20 con 0x7f888c1f67a0 2023-11-26T07:19:25.273 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:25.268+0000 7f8890dd1640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f888c1476d0 con 0x7f88740039b0 2023-11-26T07:19:25.273 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:25.268+0000 7f8889d9b640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f888c0c43a0 0x7f888c1f6260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:19:25.273 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:25.268+0000 7f8889d9b640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f888c0c43a0 0x7f888c1f6260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:40496/0 (socket says 172.21.15.143:40496) 2023-11-26T07:19:25.273 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:25.268+0000 7f8889d9b640 1 -- 172.21.15.143:0/2614611659 learned_addr learned my addr 172.21.15.143:0/2614611659 (peer_addr_for_me v2:172.21.15.143:0/0) 2023-11-26T07:19:25.273 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:25.268+0000 7f888ad9d640 1 --2- 172.21.15.143:0/2614611659 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f888c1f67a0 0x7f888c0cac70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:19:25.273 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:25.268+0000 7f888a59c640 1 --2- 172.21.15.143:0/2614611659 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f88740039b0 0x7f888c0c3e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:19:25.274 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:25.268+0000 7f8889d9b640 1 -- 172.21.15.143:0/2614611659 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f88740039b0 msgr2=0x7f888c0c3e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:19:25.274 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:25.268+0000 7f8889d9b640 1 --2- 172.21.15.143:0/2614611659 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f88740039b0 0x7f888c0c3e60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:19:25.274 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:25.268+0000 7f8889d9b640 1 -- 172.21.15.143:0/2614611659 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f888c1f67a0 msgr2=0x7f888c0cac70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:19:25.274 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:25.268+0000 7f8889d9b640 1 --2- 172.21.15.143:0/2614611659 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f888c1f67a0 0x7f888c0cac70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:19:25.274 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:25.268+0000 7f8889d9b640 1 -- 172.21.15.143:0/2614611659 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f888c157350 con 0x7f888c0c43a0 2023-11-26T07:19:25.274 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:25.268+0000 7f888ad9d640 1 --2- 172.21.15.143:0/2614611659 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f888c1f67a0 0x7f888c0cac70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-26T07:19:25.274 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:25.268+0000 7f8889d9b640 1 --2- 172.21.15.143:0/2614611659 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f888c0c43a0 0x7f888c1f6260 secure :-1 s=READY pgs=532 cs=0 l=1 rev1=1 crypto rx=0x7f8880005fd0 tx=0x7f888000ec80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:19:25.274 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:25.268+0000 7f887affd640 1 -- 172.21.15.143:0/2614611659 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f888000c7f0 con 0x7f888c0c43a0 2023-11-26T07:19:25.274 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:25.268+0000 7f887affd640 1 -- 172.21.15.143:0/2614611659 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f888000c950 con 0x7f888c0c43a0 2023-11-26T07:19:25.274 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:25.268+0000 7f887affd640 1 -- 172.21.15.143:0/2614611659 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f888000ccc0 con 0x7f888c0c43a0 2023-11-26T07:19:25.274 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:25.268+0000 7f8890dd1640 1 -- 172.21.15.143:0/2614611659 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f888c157630 con 0x7f888c0c43a0 2023-11-26T07:19:25.274 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:25.268+0000 7f8890dd1640 1 -- 172.21.15.143:0/2614611659 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f888c157bf0 con 0x7f888c0c43a0 2023-11-26T07:19:25.275 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:25.272+0000 7f8890dd1640 1 -- 172.21.15.143:0/2614611659 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f888c157db0 con 0x7f888c0c43a0 2023-11-26T07:19:25.275 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:25.272+0000 7f887affd640 1 -- 172.21.15.143:0/2614611659 <== mon.0 v2:172.21.15.143:3300/0 4 ==== mgrmap(e 7) v1 ==== 86005+0+0 (secure 0 0 0) 0x7f888001d460 con 0x7f888c0c43a0 2023-11-26T07:19:25.275 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:25.272+0000 7f887affd640 1 --2- 172.21.15.143:0/2614611659 >> [v2:172.21.15.143:6832/2150557544,v1:172.21.15.143:6833/2150557544] conn(0x7f886c066620 0x7f886c068ae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:19:25.276 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:25.272+0000 7f887affd640 1 -- 172.21.15.143:0/2614611659 <== mon.0 v2:172.21.15.143:3300/0 5 ==== osd_map(36..36 src has 1..36) v4 ==== 4943+0+0 (secure 0 0 0) 0x7f8880083d60 con 0x7f888c0c43a0 2023-11-26T07:19:25.276 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:25.272+0000 7f888a59c640 1 --2- 172.21.15.143:0/2614611659 >> [v2:172.21.15.143:6832/2150557544,v1:172.21.15.143:6833/2150557544] conn(0x7f886c066620 0x7f886c068ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:19:25.276 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:25.272+0000 7f888a59c640 1 --2- 172.21.15.143:0/2614611659 >> [v2:172.21.15.143:6832/2150557544,v1:172.21.15.143:6833/2150557544] conn(0x7f886c066620 0x7f886c068ae0 secure :-1 s=READY pgs=270 cs=0 l=1 rev1=1 crypto rx=0x7f887c0af000 tx=0x7f887c0046d0 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:19:25.279 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:25.276+0000 7f887affd640 1 -- 172.21.15.143:0/2614611659 <== mon.0 v2:172.21.15.143:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172649 (secure 0 0 0) 0x7f88800511c0 con 0x7f888c0c43a0 2023-11-26T07:19:25.428 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:25.424+0000 7f8890dd1640 1 -- 172.21.15.143:0/2614611659 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command([{prefix=config set, name=mon_allow_pool_delete}] v 0) v1 -- 0x7f888c071800 con 0x7f888c0c43a0 2023-11-26T07:19:25.432 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:25.428+0000 7f887affd640 1 -- 172.21.15.143:0/2614611659 <== mon.0 v2:172.21.15.143:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mon_allow_pool_delete}]=0 v12)=0 v12) v1 ==== 122+0+0 (secure 0 0 0) 0x7f88800547e0 con 0x7f888c0c43a0 2023-11-26T07:19:25.435 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:25.432+0000 7f8890dd1640 1 -- 172.21.15.143:0/2614611659 >> [v2:172.21.15.143:6832/2150557544,v1:172.21.15.143:6833/2150557544] conn(0x7f886c066620 msgr2=0x7f886c068ae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:19:25.435 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:25.432+0000 7f8890dd1640 1 --2- 172.21.15.143:0/2614611659 >> [v2:172.21.15.143:6832/2150557544,v1:172.21.15.143:6833/2150557544] conn(0x7f886c066620 0x7f886c068ae0 secure :-1 s=READY pgs=270 cs=0 l=1 rev1=1 crypto rx=0x7f887c0af000 tx=0x7f887c0046d0 comp rx=0 tx=0).stop 2023-11-26T07:19:25.435 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:25.432+0000 7f8890dd1640 1 -- 172.21.15.143:0/2614611659 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f888c0c43a0 msgr2=0x7f888c1f6260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:19:25.435 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:25.432+0000 7f8890dd1640 1 --2- 172.21.15.143:0/2614611659 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f888c0c43a0 0x7f888c1f6260 secure :-1 s=READY pgs=532 cs=0 l=1 rev1=1 crypto rx=0x7f8880005fd0 tx=0x7f888000ec80 comp rx=0 tx=0).stop 2023-11-26T07:19:25.435 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:25.432+0000 7f8890dd1640 1 -- 172.21.15.143:0/2614611659 shutdown_connections 2023-11-26T07:19:25.435 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:25.432+0000 7f8890dd1640 1 --2- 172.21.15.143:0/2614611659 >> [v2:172.21.15.143:6832/2150557544,v1:172.21.15.143:6833/2150557544] conn(0x7f886c066620 0x7f886c068ae0 unknown :-1 s=CLOSED pgs=270 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:19:25.435 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:25.432+0000 7f8890dd1640 1 --2- 172.21.15.143:0/2614611659 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f888c1f67a0 0x7f888c0cac70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:19:25.435 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:25.432+0000 7f8890dd1640 1 --2- 172.21.15.143:0/2614611659 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f888c0c43a0 0x7f888c1f6260 unknown :-1 s=CLOSED pgs=532 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:19:25.436 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:25.432+0000 7f8890dd1640 1 --2- 172.21.15.143:0/2614611659 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f88740039b0 0x7f888c0c3e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:19:25.436 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:25.432+0000 7f8890dd1640 1 -- 172.21.15.143:0/2614611659 >> 172.21.15.143:0/2614611659 conn(0x7f888c06a890 msgr2=0x7f888c154ee0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:19:25.436 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:25.432+0000 7f8890dd1640 1 -- 172.21.15.143:0/2614611659 shutdown_connections 2023-11-26T07:19:25.436 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:25.432+0000 7f8890dd1640 1 -- 172.21.15.143:0/2614611659 wait complete. 2023-11-26T07:19:25.454 INFO:teuthology.nuke.actions:Clearing teuthology firewall rules... 2023-11-26T07:19:25.454 DEBUG:teuthology.orchestra.run.smithi143:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2023-11-26T07:19:25.559 DEBUG:teuthology.orchestra.run.smithi188:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2023-11-26T07:19:25.652 INFO:teuthology.nuke.actions:Cleared teuthology firewall rules. 2023-11-26T07:19:25.652 DEBUG:teuthology.orchestra.run.smithi143:> set -ex 2023-11-26T07:19:25.653 DEBUG:teuthology.orchestra.run.smithi143:> dd if=/proc/self/mounts of=/dev/stdout 2023-11-26T07:19:25.662 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,relatime,size=16354672k,nr_inodes=4088668,mode=755 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3275384k,mode=755 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=3455 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysusers.service ramfs ro,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3275380k,nr_inodes=818845,mode=700,uid=1000,gid=1271 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3275384k,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2023-11-26T07:19:25.662 DEBUG:teuthology.orchestra.run.smithi188:> set -ex 2023-11-26T07:19:25.662 DEBUG:teuthology.orchestra.run.smithi188:> dd if=/proc/self/mounts of=/dev/stdout 2023-11-26T07:19:25.671 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,relatime,size=16354628k,nr_inodes=4088657,mode=755 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3275376k,mode=755 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=787 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysusers.service ramfs ro,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3275372k,nr_inodes=818843,mode=700,uid=1000,gid=1271 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3275376k,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2023-11-26T07:19:25.672 DEBUG:teuthology.orchestra.run.smithi143:> set -ex 2023-11-26T07:19:25.672 DEBUG:teuthology.orchestra.run.smithi143:> dd if=/proc/self/mounts of=/dev/stdout 2023-11-26T07:19:25.717 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,relatime,size=16354672k,nr_inodes=4088668,mode=755 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3275384k,mode=755 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=3455 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysusers.service ramfs ro,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3275380k,nr_inodes=818845,mode=700,uid=1000,gid=1271 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3275384k,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2023-11-26T07:19:25.718 DEBUG:teuthology.orchestra.run.smithi188:> set -ex 2023-11-26T07:19:25.718 DEBUG:teuthology.orchestra.run.smithi188:> dd if=/proc/self/mounts of=/dev/stdout 2023-11-26T07:19:25.728 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,relatime,size=16354628k,nr_inodes=4088657,mode=755 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3275376k,mode=755 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=787 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysusers.service ramfs ro,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3275372k,nr_inodes=818843,mode=700,uid=1000,gid=1271 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3275376k,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2023-11-26T07:19:25.728 INFO:tasks.cephfs.cephfs_test_case:[, , , ] 2023-11-26T07:19:25.729 DEBUG:teuthology.orchestra.run.smithi143:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-11-26T07:19:25.901 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:25.896+0000 7ff1ac1df640 1 Processor -- start 2023-11-26T07:19:25.901 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:25.900+0000 7ff1ac1df640 1 -- start start 2023-11-26T07:19:25.903 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:25.900+0000 7ff1ac1df640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7ff1a40cf990 0x7ff1a415efa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:19:25.903 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:25.900+0000 7ff1ac1df640 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7ff1a415f4e0 0x7ff1a40cd7b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:19:25.903 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:25.900+0000 7ff1ac1df640 1 -- --> v1:172.21.15.188:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff1a406c520 con 0x7ff1a413e290 2023-11-26T07:19:25.903 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:25.900+0000 7ff1ac1df640 1 -- --> v2:172.21.15.188:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff1a406c800 con 0x7ff1a415f4e0 2023-11-26T07:19:25.903 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:25.900+0000 7ff1ac1df640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff1a406cae0 con 0x7ff1a40cf990 2023-11-26T07:19:25.903 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:25.900+0000 7ff1a977a640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7ff1a40cf990 0x7ff1a415efa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:19:25.903 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:25.900+0000 7ff1aa77c640 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7ff1a415f4e0 0x7ff1a40cd7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:19:25.903 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:25.900+0000 7ff1a977a640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7ff1a40cf990 0x7ff1a415efa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.188:3301/0 says I am v2:172.21.15.143:57034/0 (socket says 172.21.15.143:57034) 2023-11-26T07:19:25.903 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:25.900+0000 7ff1a977a640 1 -- 172.21.15.143:0/385228104 learned_addr learned my addr 172.21.15.143:0/385228104 (peer_addr_for_me v2:172.21.15.143:0/0) 2023-11-26T07:19:25.903 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:25.900+0000 7ff1a977a640 1 -- 172.21.15.143:0/385228104 >> v2:172.21.15.188:3300/0 conn(0x7ff1a415f4e0 msgr2=0x7ff1a40cd7b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-26T07:19:25.903 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:25.900+0000 7ff1a977a640 1 --2- 172.21.15.143:0/385228104 >> v2:172.21.15.188:3300/0 conn(0x7ff1a415f4e0 0x7ff1a40cd7b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:19:25.903 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:25.900+0000 7ff1a977a640 1 -- 172.21.15.143:0/385228104 >> v1:172.21.15.188:6789/0 conn(0x7ff1a413e290 legacy=0x7ff1a40cf280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:19:25.904 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:25.900+0000 7ff1a977a640 1 -- 172.21.15.143:0/385228104 --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff1a406cdc0 con 0x7ff1a40cf990 2023-11-26T07:19:25.906 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:25.904+0000 7ff1a977a640 1 --2- 172.21.15.143:0/385228104 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7ff1a40cf990 0x7ff1a415efa0 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7ff194009870 tx=0x7ff19401c510 comp rx=0 tx=0).ready entity=mon.2 client_cookie=319539e2b1c25921 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:19:25.906 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:25.904+0000 7ff1a8f79640 1 -- 172.21.15.143:0/385228104 <== mon.2 v2:172.21.15.188:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff1940a95f0 con 0x7ff1a40cf990 2023-11-26T07:19:25.906 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:25.904+0000 7ff1a8f79640 1 -- 172.21.15.143:0/385228104 <== mon.2 v2:172.21.15.188:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff1940a9750 con 0x7ff1a40cf990 2023-11-26T07:19:25.906 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:25.904+0000 7ff1a8f79640 1 -- 172.21.15.143:0/385228104 <== mon.2 v2:172.21.15.188:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff1940a9ae0 con 0x7ff1a40cf990 2023-11-26T07:19:25.907 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:25.904+0000 7ff1ac1df640 1 -- 172.21.15.143:0/385228104 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7ff1a40cf990 msgr2=0x7ff1a415efa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:19:25.907 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:25.904+0000 7ff1ac1df640 1 --2- 172.21.15.143:0/385228104 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7ff1a40cf990 0x7ff1a415efa0 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7ff194009870 tx=0x7ff19401c510 comp rx=0 tx=0).stop 2023-11-26T07:19:25.907 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:25.904+0000 7ff1ac1df640 1 -- 172.21.15.143:0/385228104 shutdown_connections 2023-11-26T07:19:25.907 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:25.904+0000 7ff1ac1df640 1 --2- 172.21.15.143:0/385228104 >> v2:172.21.15.188:3300/0 conn(0x7ff1a415f4e0 0x7ff1a40cd7b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:19:25.907 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:25.904+0000 7ff1ac1df640 1 --2- 172.21.15.143:0/385228104 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7ff1a40cf990 0x7ff1a415efa0 unknown :-1 s=CLOSED pgs=82 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:19:25.907 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:25.904+0000 7ff1ac1df640 1 -- 172.21.15.143:0/385228104 >> 172.21.15.143:0/385228104 conn(0x7ff1a406a890 msgr2=0x7ff1a406aca0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:19:25.908 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:25.904+0000 7ff1ac1df640 1 -- 172.21.15.143:0/385228104 shutdown_connections 2023-11-26T07:19:25.908 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:25.904+0000 7ff1ac1df640 1 -- 172.21.15.143:0/385228104 wait complete. 2023-11-26T07:19:25.909 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:25.908+0000 7ff1ac1df640 1 Processor -- start 2023-11-26T07:19:25.909 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:25.908+0000 7ff1ac1df640 1 -- start start 2023-11-26T07:19:25.910 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:25.908+0000 7ff1ac1df640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7ff1a40cf990 0x7ff1a41369f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:19:25.911 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:25.908+0000 7ff1ac1df640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7ff1a413e290 0x7ff1a4153690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:19:25.911 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:25.908+0000 7ff1ac1df640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7ff1a415f4e0 0x7ff1a4151210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:19:25.911 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:25.908+0000 7ff1ac1df640 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff1a4152f20 con 0x7ff1a40cf990 2023-11-26T07:19:25.911 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:25.908+0000 7ff1ac1df640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff1a4153200 con 0x7ff1a413e290 2023-11-26T07:19:25.911 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:25.908+0000 7ff1ac1df640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff1a41549e0 con 0x7ff1a415f4e0 2023-11-26T07:19:25.911 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:25.908+0000 7ff1a9f7b640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7ff1a40cf990 0x7ff1a41369f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:19:25.911 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:25.908+0000 7ff1a9f7b640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7ff1a40cf990 0x7ff1a41369f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:40504/0 (socket says 172.21.15.143:40504) 2023-11-26T07:19:25.911 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:25.908+0000 7ff1a9f7b640 1 -- 172.21.15.143:0/4049931793 learned_addr learned my addr 172.21.15.143:0/4049931793 (peer_addr_for_me v2:172.21.15.143:0/0) 2023-11-26T07:19:25.911 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:25.908+0000 7ff1a977a640 1 --2- 172.21.15.143:0/4049931793 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7ff1a413e290 0x7ff1a4153690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:19:25.911 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:25.908+0000 7ff1aa77c640 1 --2- 172.21.15.143:0/4049931793 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7ff1a415f4e0 0x7ff1a4151210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:19:25.911 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:25.908+0000 7ff1a9f7b640 1 -- 172.21.15.143:0/4049931793 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7ff1a415f4e0 msgr2=0x7ff1a4151210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:19:25.911 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:25.908+0000 7ff1a9f7b640 1 --2- 172.21.15.143:0/4049931793 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7ff1a415f4e0 0x7ff1a4151210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:19:25.912 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:25.908+0000 7ff1a9f7b640 1 -- 172.21.15.143:0/4049931793 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7ff1a413e290 msgr2=0x7ff1a4153690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:19:25.912 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:25.908+0000 7ff1a9f7b640 1 --2- 172.21.15.143:0/4049931793 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7ff1a413e290 0x7ff1a4153690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:19:25.912 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:25.908+0000 7ff1a9f7b640 1 -- 172.21.15.143:0/4049931793 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff1940094d0 con 0x7ff1a40cf990 2023-11-26T07:19:25.912 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:25.908+0000 7ff1aa77c640 1 --2- 172.21.15.143:0/4049931793 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7ff1a415f4e0 0x7ff1a4151210 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-26T07:19:25.912 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:25.908+0000 7ff1a977a640 1 --2- 172.21.15.143:0/4049931793 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7ff1a413e290 0x7ff1a4153690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-26T07:19:25.912 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:25.908+0000 7ff1a9f7b640 1 --2- 172.21.15.143:0/4049931793 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7ff1a40cf990 0x7ff1a41369f0 secure :-1 s=READY pgs=533 cs=0 l=1 rev1=1 crypto rx=0x7ff1a000c530 tx=0x7ff1a000c9f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:19:25.913 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:25.908+0000 7ff19a7fc640 1 -- 172.21.15.143:0/4049931793 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff1a0005220 con 0x7ff1a40cf990 2023-11-26T07:19:25.913 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:25.908+0000 7ff19a7fc640 1 -- 172.21.15.143:0/4049931793 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff1a0005380 con 0x7ff1a40cf990 2023-11-26T07:19:25.913 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:25.908+0000 7ff1ac1df640 1 -- 172.21.15.143:0/4049931793 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff1a4154cc0 con 0x7ff1a40cf990 2023-11-26T07:19:25.913 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:25.908+0000 7ff19a7fc640 1 -- 172.21.15.143:0/4049931793 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff1a0007270 con 0x7ff1a40cf990 2023-11-26T07:19:25.913 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:25.908+0000 7ff1ac1df640 1 -- 172.21.15.143:0/4049931793 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff1a4136f30 con 0x7ff1a40cf990 2023-11-26T07:19:25.913 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:25.908+0000 7ff1ac1df640 1 -- 172.21.15.143:0/4049931793 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff1a4155170 con 0x7ff1a40cf990 2023-11-26T07:19:25.913 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:25.912+0000 7ff19a7fc640 1 -- 172.21.15.143:0/4049931793 <== mon.0 v2:172.21.15.143:3300/0 4 ==== mgrmap(e 7) v1 ==== 86005+0+0 (secure 0 0 0) 0x7ff1a00073d0 con 0x7ff1a40cf990 2023-11-26T07:19:25.914 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:25.912+0000 7ff19a7fc640 1 --2- 172.21.15.143:0/4049931793 >> [v2:172.21.15.143:6832/2150557544,v1:172.21.15.143:6833/2150557544] conn(0x7ff188066620 0x7ff188068ae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:19:25.914 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:25.912+0000 7ff19a7fc640 1 -- 172.21.15.143:0/4049931793 <== mon.0 v2:172.21.15.143:3300/0 5 ==== osd_map(36..36 src has 1..36) v4 ==== 4943+0+0 (secure 0 0 0) 0x7ff1a00837b0 con 0x7ff1a40cf990 2023-11-26T07:19:25.914 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:25.912+0000 7ff1a977a640 1 --2- 172.21.15.143:0/4049931793 >> [v2:172.21.15.143:6832/2150557544,v1:172.21.15.143:6833/2150557544] conn(0x7ff188066620 0x7ff188068ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:19:25.914 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:25.912+0000 7ff1a977a640 1 --2- 172.21.15.143:0/4049931793 >> [v2:172.21.15.143:6832/2150557544,v1:172.21.15.143:6833/2150557544] conn(0x7ff188066620 0x7ff188068ae0 secure :-1 s=READY pgs=271 cs=0 l=1 rev1=1 crypto rx=0x7ff1940ae000 tx=0x7ff1940b3040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:19:25.917 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:25.916+0000 7ff19a7fc640 1 -- 172.21.15.143:0/4049931793 <== mon.0 v2:172.21.15.143:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172649 (secure 0 0 0) 0x7ff1a0050ca0 con 0x7ff1a40cf990 2023-11-26T07:19:26.098 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:26.096+0000 7ff1ac1df640 1 -- 172.21.15.143:0/4049931793 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7ff1a4067680 con 0x7ff1a40cf990 2023-11-26T07:19:26.098 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:26.096+0000 7ff19a7fc640 1 -- 172.21.15.143:0/4049931793 <== mon.0 v2:172.21.15.143:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 19 v19) v1 ==== 94+0+3711 (secure 0 0 0) 0x7ff1a00542c0 con 0x7ff1a40cf990 2023-11-26T07:19:26.099 INFO:teuthology.orchestra.run.smithi143.stdout: 2023-11-26T07:19:26.099 INFO:teuthology.orchestra.run.smithi143.stdout:{"epoch":19,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4847,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.188:6834/1547410397","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6832","nonce":1547410397},{"type":"v1","addr":"172.21.15.188:6834","nonce":1547410397}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":12},{"gid":4850,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.188:6835/3247640329","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6833","nonce":3247640329},{"type":"v1","addr":"172.21.15.188:6835","nonce":3247640329}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":12},{"gid":5217,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.143:6835/2123707650","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6834","nonce":2123707650},{"type":"v1","addr":"172.21.15.143:6835","nonce":2123707650}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":11},{"gid":5664,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.143:6837/4004464005","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6836","nonce":4004464005},{"type":"v1","addr":"172.21.15.143:6837","nonce":4004464005}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":19}],"filesystems":[]} 2023-11-26T07:19:26.099 INFO:teuthology.orchestra.run.smithi143.stderr:dumped fsmap epoch 19 2023-11-26T07:19:26.101 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:26.096+0000 7ff1ac1df640 1 -- 172.21.15.143:0/4049931793 >> [v2:172.21.15.143:6832/2150557544,v1:172.21.15.143:6833/2150557544] conn(0x7ff188066620 msgr2=0x7ff188068ae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:19:26.101 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:26.096+0000 7ff1ac1df640 1 --2- 172.21.15.143:0/4049931793 >> [v2:172.21.15.143:6832/2150557544,v1:172.21.15.143:6833/2150557544] conn(0x7ff188066620 0x7ff188068ae0 secure :-1 s=READY pgs=271 cs=0 l=1 rev1=1 crypto rx=0x7ff1940ae000 tx=0x7ff1940b3040 comp rx=0 tx=0).stop 2023-11-26T07:19:26.101 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:26.096+0000 7ff1ac1df640 1 -- 172.21.15.143:0/4049931793 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7ff1a40cf990 msgr2=0x7ff1a41369f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:19:26.101 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:26.096+0000 7ff1ac1df640 1 --2- 172.21.15.143:0/4049931793 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7ff1a40cf990 0x7ff1a41369f0 secure :-1 s=READY pgs=533 cs=0 l=1 rev1=1 crypto rx=0x7ff1a000c530 tx=0x7ff1a000c9f0 comp rx=0 tx=0).stop 2023-11-26T07:19:26.101 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:26.096+0000 7ff1ac1df640 1 -- 172.21.15.143:0/4049931793 shutdown_connections 2023-11-26T07:19:26.101 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:26.096+0000 7ff1ac1df640 1 --2- 172.21.15.143:0/4049931793 >> [v2:172.21.15.143:6832/2150557544,v1:172.21.15.143:6833/2150557544] conn(0x7ff188066620 0x7ff188068ae0 unknown :-1 s=CLOSED pgs=271 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:19:26.101 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:26.096+0000 7ff1ac1df640 1 --2- 172.21.15.143:0/4049931793 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7ff1a415f4e0 0x7ff1a4151210 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:19:26.101 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:26.096+0000 7ff1ac1df640 1 --2- 172.21.15.143:0/4049931793 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7ff1a413e290 0x7ff1a4153690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:19:26.101 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:26.096+0000 7ff1ac1df640 1 --2- 172.21.15.143:0/4049931793 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7ff1a40cf990 0x7ff1a41369f0 unknown :-1 s=CLOSED pgs=533 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:19:26.102 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:26.096+0000 7ff1ac1df640 1 -- 172.21.15.143:0/4049931793 >> 172.21.15.143:0/4049931793 conn(0x7ff1a406a890 msgr2=0x7ff1a40b25e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:19:26.102 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:26.100+0000 7ff1ac1df640 1 -- 172.21.15.143:0/4049931793 shutdown_connections 2023-11-26T07:19:26.102 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:26.100+0000 7ff1ac1df640 1 -- 172.21.15.143:0/4049931793 wait complete. 2023-11-26T07:19:26.120 INFO:tasks.ceph.mds.a:Restarting daemon 2023-11-26T07:19:26.120 INFO:tasks.ceph.mds.a:Stopping old one... 2023-11-26T07:19:26.120 DEBUG:tasks.ceph.mds.a:waiting for process to exit 2023-11-26T07:19:26.120 INFO:teuthology.orchestra.run:waiting for 300 2023-11-26T07:19:26.120 INFO:tasks.ceph.mds.c:Restarting daemon 2023-11-26T07:19:26.121 INFO:tasks.ceph.mds.c:Stopping old one... 2023-11-26T07:19:26.121 DEBUG:tasks.ceph.mds.c:waiting for process to exit 2023-11-26T07:19:26.121 INFO:teuthology.orchestra.run:waiting for 300 2023-11-26T07:19:26.121 INFO:tasks.ceph.mds.b:Restarting daemon 2023-11-26T07:19:26.121 INFO:tasks.ceph.mds.b:Stopping old one... 2023-11-26T07:19:26.121 DEBUG:tasks.ceph.mds.b:waiting for process to exit 2023-11-26T07:19:26.121 INFO:teuthology.orchestra.run:waiting for 300 2023-11-26T07:19:26.122 INFO:tasks.ceph.mds.d:Restarting daemon 2023-11-26T07:19:26.122 INFO:tasks.ceph.mds.d:Stopping old one... 2023-11-26T07:19:26.122 DEBUG:tasks.ceph.mds.d:waiting for process to exit 2023-11-26T07:19:26.122 INFO:teuthology.orchestra.run:waiting for 300 2023-11-26T07:19:26.133 INFO:tasks.ceph.mds.a:Stopped 2023-11-26T07:19:26.133 DEBUG:teuthology.orchestra.run.smithi143:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i a 2023-11-26T07:19:26.134 INFO:tasks.ceph.mds.c:Stopped 2023-11-26T07:19:26.134 DEBUG:teuthology.orchestra.run.smithi143:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i c 2023-11-26T07:19:26.136 INFO:tasks.ceph.mds.b:Stopped 2023-11-26T07:19:26.137 DEBUG:teuthology.orchestra.run.smithi188:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i b 2023-11-26T07:19:26.137 INFO:tasks.ceph.mds.d:Stopped 2023-11-26T07:19:26.138 DEBUG:teuthology.orchestra.run.smithi188:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i d 2023-11-26T07:19:26.179 INFO:tasks.ceph.mds.a:Started 2023-11-26T07:19:26.179 DEBUG:teuthology.parallel:result is None 2023-11-26T07:19:26.182 INFO:tasks.ceph.mds.c:Started 2023-11-26T07:19:26.183 DEBUG:teuthology.parallel:result is None 2023-11-26T07:19:26.184 INFO:tasks.ceph.mds.b:Started 2023-11-26T07:19:26.185 DEBUG:teuthology.parallel:result is None 2023-11-26T07:19:26.186 INFO:tasks.ceph.mds.d:Started 2023-11-26T07:19:26.187 DEBUG:teuthology.parallel:result is None 2023-11-26T07:19:26.187 DEBUG:teuthology.orchestra.run.smithi143:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist clear 2023-11-26T07:19:26.266 INFO:tasks.ceph.mds.a.smithi143.stdout:starting mds.a at 2023-11-26T07:19:26.269 INFO:tasks.ceph.mds.c.smithi143.stdout:starting mds.c at 2023-11-26T07:19:26.274 INFO:tasks.ceph.mds.b.smithi188.stdout:starting mds.b at 2023-11-26T07:19:26.275 INFO:tasks.ceph.mds.d.smithi188.stdout:starting mds.d at 2023-11-26T07:19:26.334 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:26.332+0000 7fb91f59e640 1 Processor -- start 2023-11-26T07:19:26.334 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:26.332+0000 7fb91f59e640 1 -- start start 2023-11-26T07:19:26.335 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:26.332+0000 7fb91f59e640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fb9200714e0 0x7fb920071940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:19:26.335 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:26.332+0000 7fb91f59e640 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7fb92015bde0 0x7fb92015c250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:19:26.335 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:26.332+0000 7fb91f59e640 1 -- --> v1:172.21.15.188:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb920071e80 con 0x7fb920144720 2023-11-26T07:19:26.335 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:26.332+0000 7fb91f59e640 1 -- --> v2:172.21.15.188:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb92006e3f0 con 0x7fb92015bde0 2023-11-26T07:19:26.335 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:26.332+0000 7fb91f59e640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb9200722f0 con 0x7fb9200714e0 2023-11-26T07:19:26.336 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:26.332+0000 7fb917fff640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fb9200714e0 0x7fb920071940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:19:26.336 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:26.332+0000 7fb91ed9d640 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7fb92015bde0 0x7fb92015c250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:19:26.336 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:26.332+0000 7fb917fff640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fb9200714e0 0x7fb920071940 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.188:3301/0 says I am v2:172.21.15.143:57076/0 (socket says 172.21.15.143:57076) 2023-11-26T07:19:26.337 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:26.332+0000 7fb917fff640 1 -- 172.21.15.143:0/1141138062 learned_addr learned my addr 172.21.15.143:0/1141138062 (peer_addr_for_me v2:172.21.15.143:0/0) 2023-11-26T07:19:26.337 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:26.332+0000 7fb91ed9d640 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7fb92015bde0 0x7fb92015c250 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.188:3300/0 says I am v2:172.21.15.143:33646/0 (socket says 172.21.15.143:33646) 2023-11-26T07:19:26.337 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:26.332+0000 7fb91e59c640 1 --1- 172.21.15.143:0/1141138062 >> v1:172.21.15.188:6789/0 conn(0x7fb920144720 0x7fb92006d300 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.188:6789/0 says I am v1:172.21.15.143:42300/0 (socket says 172.21.15.143:42300) 2023-11-26T07:19:26.337 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:26.332+0000 7fb91ed9d640 1 -- 172.21.15.143:0/1141138062 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fb9200714e0 msgr2=0x7fb920071940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-26T07:19:26.337 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:26.332+0000 7fb91ed9d640 1 --2- 172.21.15.143:0/1141138062 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fb9200714e0 0x7fb920071940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:19:26.337 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:26.332+0000 7fb91ed9d640 1 -- 172.21.15.143:0/1141138062 >> v1:172.21.15.188:6789/0 conn(0x7fb920144720 legacy=0x7fb92006d300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:19:26.337 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:26.332+0000 7fb91ed9d640 1 -- 172.21.15.143:0/1141138062 --> v2:172.21.15.188:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb92006de30 con 0x7fb92015bde0 2023-11-26T07:19:26.337 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:26.332+0000 7fb91dd9b640 1 -- 172.21.15.143:0/1141138062 <== mon.1 v1:172.21.15.188:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 982352687 0 0) 0x7fb920071e80 con 0x7fb920144720 2023-11-26T07:19:26.339 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:26.336+0000 7fb91ed9d640 1 --2- 172.21.15.143:0/1141138062 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fb92015bde0 0x7fb92015c250 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7fb918009d90 tx=0x7fb91801c920 comp rx=0 tx=0).ready entity=mon.1 client_cookie=8a1c8b33877df099 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:19:26.339 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:26.336+0000 7fb91dd9b640 1 -- 172.21.15.143:0/1141138062 <== mon.1 v2:172.21.15.188:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb9180a59e0 con 0x7fb92015bde0 2023-11-26T07:19:26.339 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:26.336+0000 7fb91dd9b640 1 -- 172.21.15.143:0/1141138062 <== mon.1 v2:172.21.15.188:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb9180a5b40 con 0x7fb92015bde0 2023-11-26T07:19:26.339 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:26.336+0000 7fb91dd9b640 1 -- 172.21.15.143:0/1141138062 <== mon.1 v2:172.21.15.188:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb9180ad440 con 0x7fb92015bde0 2023-11-26T07:19:26.340 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:26.336+0000 7fb91f59e640 1 -- 172.21.15.143:0/1141138062 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fb92015bde0 msgr2=0x7fb92015c250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:19:26.340 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:26.336+0000 7fb91f59e640 1 --2- 172.21.15.143:0/1141138062 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fb92015bde0 0x7fb92015c250 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7fb918009d90 tx=0x7fb91801c920 comp rx=0 tx=0).stop 2023-11-26T07:19:26.340 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:26.336+0000 7fb91f59e640 1 -- 172.21.15.143:0/1141138062 shutdown_connections 2023-11-26T07:19:26.340 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:26.336+0000 7fb91f59e640 1 --2- 172.21.15.143:0/1141138062 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fb92015bde0 0x7fb92015c250 unknown :-1 s=CLOSED pgs=238 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:19:26.340 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:26.336+0000 7fb91f59e640 1 --2- 172.21.15.143:0/1141138062 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fb9200714e0 0x7fb920071940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:19:26.340 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:26.336+0000 7fb91f59e640 1 -- 172.21.15.143:0/1141138062 >> 172.21.15.143:0/1141138062 conn(0x7fb92006a890 msgr2=0x7fb92006aca0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:19:26.340 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:26.336+0000 7fb91f59e640 1 -- 172.21.15.143:0/1141138062 shutdown_connections 2023-11-26T07:19:26.340 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:26.336+0000 7fb91f59e640 1 -- 172.21.15.143:0/1141138062 wait complete. 2023-11-26T07:19:26.341 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:26.340+0000 7fb91f59e640 1 Processor -- start 2023-11-26T07:19:26.342 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:26.340+0000 7fb91f59e640 1 -- start start 2023-11-26T07:19:26.343 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:26.340+0000 7fb91f59e640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fb9200714e0 0x7fb920142650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:19:26.344 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:26.340+0000 7fb91f59e640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fb920144720 0x7fb92015a7b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:19:26.344 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:26.340+0000 7fb91f59e640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fb92015bde0 0x7fb9200c4a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:19:26.344 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:26.340+0000 7fb91f59e640 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb9201f5b60 con 0x7fb92015bde0 2023-11-26T07:19:26.344 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:26.340+0000 7fb91f59e640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb9201f5cc0 con 0x7fb9200714e0 2023-11-26T07:19:26.344 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:26.340+0000 7fb91f59e640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb92015acf0 con 0x7fb920144720 2023-11-26T07:19:26.344 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:26.340+0000 7fb91ed9d640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fb92015bde0 0x7fb9200c4a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:19:26.344 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:26.340+0000 7fb91ed9d640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fb92015bde0 0x7fb9200c4a30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:40548/0 (socket says 172.21.15.143:40548) 2023-11-26T07:19:26.344 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:26.340+0000 7fb91ed9d640 1 -- 172.21.15.143:0/3066716808 learned_addr learned my addr 172.21.15.143:0/3066716808 (peer_addr_for_me v2:172.21.15.143:0/0) 2023-11-26T07:19:26.344 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:26.340+0000 7fb917fff640 1 --2- 172.21.15.143:0/3066716808 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fb920144720 0x7fb92015a7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:19:26.344 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:26.340+0000 7fb91e59c640 1 --2- 172.21.15.143:0/3066716808 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fb9200714e0 0x7fb920142650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:19:26.344 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:26.340+0000 7fb91ed9d640 1 -- 172.21.15.143:0/3066716808 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fb920144720 msgr2=0x7fb92015a7b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:19:26.344 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:26.340+0000 7fb91ed9d640 1 --2- 172.21.15.143:0/3066716808 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fb920144720 0x7fb92015a7b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:19:26.344 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:26.340+0000 7fb91ed9d640 1 -- 172.21.15.143:0/3066716808 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fb9200714e0 msgr2=0x7fb920142650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:19:26.344 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:26.340+0000 7fb91ed9d640 1 --2- 172.21.15.143:0/3066716808 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fb9200714e0 0x7fb920142650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:19:26.345 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:26.340+0000 7fb91ed9d640 1 -- 172.21.15.143:0/3066716808 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb918009a70 con 0x7fb92015bde0 2023-11-26T07:19:26.345 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:26.340+0000 7fb91ed9d640 1 --2- 172.21.15.143:0/3066716808 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fb92015bde0 0x7fb9200c4a30 secure :-1 s=READY pgs=543 cs=0 l=1 rev1=1 crypto rx=0x7fb9180b0040 tx=0x7fb9180196c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:19:26.345 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:26.340+0000 7fb916ffd640 1 -- 172.21.15.143:0/3066716808 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb9180add90 con 0x7fb92015bde0 2023-11-26T07:19:26.346 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:26.340+0000 7fb916ffd640 1 -- 172.21.15.143:0/3066716808 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb918009450 con 0x7fb92015bde0 2023-11-26T07:19:26.346 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:26.340+0000 7fb916ffd640 1 -- 172.21.15.143:0/3066716808 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb9180b9690 con 0x7fb92015bde0 2023-11-26T07:19:26.346 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:26.340+0000 7fb91f59e640 1 -- 172.21.15.143:0/3066716808 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb920142cc0 con 0x7fb92015bde0 2023-11-26T07:19:26.346 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:26.340+0000 7fb91f59e640 1 -- 172.21.15.143:0/3066716808 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb920143280 con 0x7fb92015bde0 2023-11-26T07:19:26.346 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:26.344+0000 7fb91f59e640 1 -- 172.21.15.143:0/3066716808 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb9201f5e50 con 0x7fb92015bde0 2023-11-26T07:19:26.346 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:26.344+0000 7fb916ffd640 1 -- 172.21.15.143:0/3066716808 <== mon.0 v2:172.21.15.143:3300/0 4 ==== mgrmap(e 7) v1 ==== 86005+0+0 (secure 0 0 0) 0x7fb9180b97f0 con 0x7fb92015bde0 2023-11-26T07:19:26.347 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:26.344+0000 7fb916ffd640 1 --2- 172.21.15.143:0/3066716808 >> [v2:172.21.15.143:6832/2150557544,v1:172.21.15.143:6833/2150557544] conn(0x7fb900066620 0x7fb900068ae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:19:26.347 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:26.344+0000 7fb916ffd640 1 -- 172.21.15.143:0/3066716808 <== mon.0 v2:172.21.15.143:3300/0 5 ==== osd_map(36..36 src has 1..36) v4 ==== 4943+0+0 (secure 0 0 0) 0x7fb91811f6e0 con 0x7fb92015bde0 2023-11-26T07:19:26.347 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:26.344+0000 7fb91e59c640 1 --2- 172.21.15.143:0/3066716808 >> [v2:172.21.15.143:6832/2150557544,v1:172.21.15.143:6833/2150557544] conn(0x7fb900066620 0x7fb900068ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:19:26.348 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:26.344+0000 7fb91e59c640 1 --2- 172.21.15.143:0/3066716808 >> [v2:172.21.15.143:6832/2150557544,v1:172.21.15.143:6833/2150557544] conn(0x7fb900066620 0x7fb900068ae0 secure :-1 s=READY pgs=272 cs=0 l=1 rev1=1 crypto rx=0x7fb910000f70 tx=0x7fb910007b60 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:19:26.351 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:26.348+0000 7fb916ffd640 1 -- 172.21.15.143:0/3066716808 <== mon.0 v2:172.21.15.143:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172649 (secure 0 0 0) 0x7fb9180eccf0 con 0x7fb92015bde0 2023-11-26T07:19:26.516 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:26.512+0000 7fb91f59e640 1 -- 172.21.15.143:0/3066716808 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "osd blocklist clear"} v 0) v1 -- 0x7fb920067500 con 0x7fb92015bde0 2023-11-26T07:19:27.455 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:27.452+0000 7fb916ffd640 1 -- 172.21.15.143:0/3066716808 <== mon.0 v2:172.21.15.143:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist clear"}]=0 removed all blocklist entries v41) v1 ==== 97+0+0 (secure 0 0 0) 0x7fb9180f0310 con 0x7fb92015bde0 2023-11-26T07:19:27.455 INFO:teuthology.orchestra.run.smithi143.stderr: removed all blocklist entries 2023-11-26T07:19:27.458 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:27.456+0000 7fb91f59e640 1 -- 172.21.15.143:0/3066716808 >> [v2:172.21.15.143:6832/2150557544,v1:172.21.15.143:6833/2150557544] conn(0x7fb900066620 msgr2=0x7fb900068ae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:19:27.459 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:27.456+0000 7fb91f59e640 1 --2- 172.21.15.143:0/3066716808 >> [v2:172.21.15.143:6832/2150557544,v1:172.21.15.143:6833/2150557544] conn(0x7fb900066620 0x7fb900068ae0 secure :-1 s=READY pgs=272 cs=0 l=1 rev1=1 crypto rx=0x7fb910000f70 tx=0x7fb910007b60 comp rx=0 tx=0).stop 2023-11-26T07:19:27.459 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:27.456+0000 7fb91f59e640 1 -- 172.21.15.143:0/3066716808 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fb92015bde0 msgr2=0x7fb9200c4a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:19:27.459 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:27.456+0000 7fb91f59e640 1 --2- 172.21.15.143:0/3066716808 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fb92015bde0 0x7fb9200c4a30 secure :-1 s=READY pgs=543 cs=0 l=1 rev1=1 crypto rx=0x7fb9180b0040 tx=0x7fb9180196c0 comp rx=0 tx=0).stop 2023-11-26T07:19:27.459 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:27.456+0000 7fb91f59e640 1 -- 172.21.15.143:0/3066716808 shutdown_connections 2023-11-26T07:19:27.459 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:27.456+0000 7fb91f59e640 1 --2- 172.21.15.143:0/3066716808 >> [v2:172.21.15.143:6832/2150557544,v1:172.21.15.143:6833/2150557544] conn(0x7fb900066620 0x7fb900068ae0 unknown :-1 s=CLOSED pgs=272 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:19:27.459 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:27.456+0000 7fb91f59e640 1 --2- 172.21.15.143:0/3066716808 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fb92015bde0 0x7fb9200c4a30 unknown :-1 s=CLOSED pgs=543 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:19:27.459 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:27.456+0000 7fb91f59e640 1 --2- 172.21.15.143:0/3066716808 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fb920144720 0x7fb92015a7b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:19:27.459 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:27.456+0000 7fb91f59e640 1 --2- 172.21.15.143:0/3066716808 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fb9200714e0 0x7fb920142650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:19:27.459 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:27.456+0000 7fb91f59e640 1 -- 172.21.15.143:0/3066716808 >> 172.21.15.143:0/3066716808 conn(0x7fb92006a890 msgr2=0x7fb9200ad8d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:19:27.459 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:27.456+0000 7fb91f59e640 1 -- 172.21.15.143:0/3066716808 shutdown_connections 2023-11-26T07:19:27.459 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:27.456+0000 7fb91f59e640 1 -- 172.21.15.143:0/3066716808 wait complete. 2023-11-26T07:19:27.480 DEBUG:teuthology.orchestra.run.smithi143:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth ls --format=json-pretty 2023-11-26T07:19:27.620 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:27.616+0000 7f3d5c31c640 1 Processor -- start 2023-11-26T07:19:27.620 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:27.616+0000 7f3d5c31c640 1 -- start start 2023-11-26T07:19:27.622 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:27.620+0000 7f3d5c31c640 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7f3d541596d0 0x7f3d54071850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:19:27.622 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:27.620+0000 7f3d5c31c640 1 -- --> v1:172.21.15.143:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3d5406c750 con 0x7f3d540b2f80 2023-11-26T07:19:27.622 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:27.620+0000 7f3d5c31c640 1 -- --> v1:172.21.15.188:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3d540b3550 con 0x7f3d540ce680 2023-11-26T07:19:27.623 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:27.620+0000 7f3d5c31c640 1 -- --> v2:172.21.15.188:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3d540cbe00 con 0x7f3d541596d0 2023-11-26T07:19:27.623 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:27.620+0000 7f3d5a8b9640 1 --1- >> v1:172.21.15.143:6789/0 conn(0x7f3d540b2f80 0x7f3d5406cbe0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.143:6789/0 says I am v1:172.21.15.143:58330/0 (socket says 172.21.15.143:58330) 2023-11-26T07:19:27.623 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:27.620+0000 7f3d5a8b9640 1 -- 172.21.15.143:0/161821689 learned_addr learned my addr 172.21.15.143:0/161821689 (peer_addr_for_me v1:172.21.15.143:0/0) 2023-11-26T07:19:27.623 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:27.620+0000 7f3d5a0b8640 1 --2- 172.21.15.143:0/161821689 >> v2:172.21.15.188:3300/0 conn(0x7f3d541596d0 0x7f3d54071850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:19:27.623 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:27.620+0000 7f3d590b6640 1 -- 172.21.15.143:0/161821689 <== mon.0 v1:172.21.15.143:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 997772978 0 0) 0x7f3d5406c750 con 0x7f3d540b2f80 2023-11-26T07:19:27.623 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:27.620+0000 7f3d590b6640 1 -- 172.21.15.143:0/161821689 --> v1:172.21.15.143:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f3d40003530 con 0x7f3d540b2f80 2023-11-26T07:19:27.624 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:27.620+0000 7f3d590b6640 1 -- 172.21.15.143:0/161821689 <== mon.0 v1:172.21.15.143:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3973671726 0 0) 0x7f3d40003530 con 0x7f3d540b2f80 2023-11-26T07:19:27.624 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:27.620+0000 7f3d590b6640 1 -- 172.21.15.143:0/161821689 >> v2:172.21.15.188:3300/0 conn(0x7f3d541596d0 msgr2=0x7f3d54071850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-26T07:19:27.624 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:27.620+0000 7f3d590b6640 1 --2- 172.21.15.143:0/161821689 >> v2:172.21.15.188:3300/0 conn(0x7f3d541596d0 0x7f3d54071850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:19:27.624 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:27.620+0000 7f3d590b6640 1 -- 172.21.15.143:0/161821689 >> v1:172.21.15.188:6789/0 conn(0x7f3d540ce680 legacy=0x7f3d540b2870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:19:27.625 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:27.620+0000 7f3d590b6640 1 -- 172.21.15.143:0/161821689 --> v1:172.21.15.143:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3d54071e80 con 0x7f3d540b2f80 2023-11-26T07:19:27.625 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:27.620+0000 7f3d590b6640 1 -- 172.21.15.143:0/161821689 <== mon.0 v1:172.21.15.143:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2137882523 0 0) 0x7f3d4c001a00 con 0x7f3d540b2f80 2023-11-26T07:19:27.625 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:27.620+0000 7f3d590b6640 1 -- 172.21.15.143:0/161821689 >> v1:172.21.15.143:6789/0 conn(0x7f3d540b2f80 legacy=0x7f3d5406cbe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:19:27.625 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:27.620+0000 7f3d590b6640 1 --2- 172.21.15.143:0/161821689 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f3d40003980 0x7f3d40003d60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:19:27.625 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:27.620+0000 7f3d590b6640 1 --2- 172.21.15.143:0/161821689 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f3d40004380 0x7f3d40004850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:19:27.625 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:27.620+0000 7f3d590b6640 1 --2- 172.21.15.143:0/161821689 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f3d40005050 0x7f3d4000b990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:19:27.625 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:27.620+0000 7f3d590b6640 1 -- 172.21.15.143:0/161821689 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3d540cbe00 con 0x7f3d40005050 2023-11-26T07:19:27.625 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:27.620+0000 7f3d590b6640 1 -- 172.21.15.143:0/161821689 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3d4000e1c0 con 0x7f3d40003980 2023-11-26T07:19:27.625 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:27.620+0000 7f3d590b6640 1 -- 172.21.15.143:0/161821689 --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3d4000c000 con 0x7f3d40004380 2023-11-26T07:19:27.625 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:27.620+0000 7f3d5a8b9640 1 --2- 172.21.15.143:0/161821689 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f3d40005050 0x7f3d4000b990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:19:27.625 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:27.620+0000 7f3d598b7640 1 --2- 172.21.15.143:0/161821689 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f3d40004380 0x7f3d40004850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:19:27.625 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:27.620+0000 7f3d5a0b8640 1 --2- 172.21.15.143:0/161821689 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f3d40003980 0x7f3d40003d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:19:27.625 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:27.620+0000 7f3d5a8b9640 1 -- 172.21.15.143:0/161821689 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f3d40004380 msgr2=0x7f3d40004850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-26T07:19:27.625 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:27.620+0000 7f3d5a8b9640 1 --2- 172.21.15.143:0/161821689 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f3d40004380 0x7f3d40004850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:19:27.626 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:27.620+0000 7f3d5a8b9640 1 -- 172.21.15.143:0/161821689 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f3d40003980 msgr2=0x7f3d40003d60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-26T07:19:27.626 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:27.620+0000 7f3d5a8b9640 1 --2- 172.21.15.143:0/161821689 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f3d40003980 0x7f3d40003d60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:19:27.626 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:27.620+0000 7f3d5a8b9640 1 -- 172.21.15.143:0/161821689 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3d40003530 con 0x7f3d40005050 2023-11-26T07:19:27.626 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:27.624+0000 7f3d5a8b9640 1 -- 172.21.15.143:0/161821689 reap_dead start 2023-11-26T07:19:27.626 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:27.624+0000 7f3d5a8b9640 1 --2- 172.21.15.143:0/161821689 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f3d40005050 0x7f3d4000b990 secure :-1 s=READY pgs=545 cs=0 l=1 rev1=1 crypto rx=0x7f3d4c00a810 tx=0x7f3d4c01fa60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=517c95ea4ebbaf78 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:19:27.627 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:27.624+0000 7f3d590b6640 1 -- 172.21.15.143:0/161821689 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3d4c0af070 con 0x7f3d40005050 2023-11-26T07:19:27.627 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:27.624+0000 7f3d590b6640 1 -- 172.21.15.143:0/161821689 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3d4c0aa490 con 0x7f3d40005050 2023-11-26T07:19:27.627 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:27.624+0000 7f3d590b6640 1 -- 172.21.15.143:0/161821689 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3d4c0aa840 con 0x7f3d40005050 2023-11-26T07:19:27.627 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:27.624+0000 7f3d5c31c640 1 -- 172.21.15.143:0/161821689 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f3d40005050 msgr2=0x7f3d4000b990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:19:27.627 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:27.624+0000 7f3d5c31c640 1 --2- 172.21.15.143:0/161821689 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f3d40005050 0x7f3d4000b990 secure :-1 s=READY pgs=545 cs=0 l=1 rev1=1 crypto rx=0x7f3d4c00a810 tx=0x7f3d4c01fa60 comp rx=0 tx=0).stop 2023-11-26T07:19:27.628 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:27.624+0000 7f3d5c31c640 1 -- 172.21.15.143:0/161821689 shutdown_connections 2023-11-26T07:19:27.628 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:27.624+0000 7f3d5c31c640 1 --2- 172.21.15.143:0/161821689 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f3d40005050 0x7f3d4000b990 unknown :-1 s=CLOSED pgs=545 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:19:27.628 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:27.624+0000 7f3d5c31c640 1 -- 172.21.15.143:0/161821689 >> 172.21.15.143:0/161821689 conn(0x7f3d5406a890 msgr2=0x7f3d5406aca0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:19:27.628 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:27.624+0000 7f3d5c31c640 1 -- 172.21.15.143:0/161821689 shutdown_connections 2023-11-26T07:19:27.628 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:27.624+0000 7f3d5c31c640 1 -- 172.21.15.143:0/161821689 wait complete. 2023-11-26T07:19:27.629 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:27.628+0000 7f3d5c31c640 1 Processor -- start 2023-11-26T07:19:27.629 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:27.628+0000 7f3d5c31c640 1 -- start start 2023-11-26T07:19:27.631 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:27.628+0000 7f3d5c31c640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f3d40005050 0x7f3d541523b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:19:27.631 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:27.628+0000 7f3d5c31c640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f3d54159c30 0x7f3d54154c20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:19:27.631 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:27.628+0000 7f3d5c31c640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f3d5415a0b0 0x7f3d54155160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:19:27.631 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:27.628+0000 7f3d5c31c640 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3d54156880 con 0x7f3d5415a0b0 2023-11-26T07:19:27.632 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:27.628+0000 7f3d5c31c640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3d5415a680 con 0x7f3d54159c30 2023-11-26T07:19:27.632 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:27.628+0000 7f3d5c31c640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3d54156230 con 0x7f3d40005050 2023-11-26T07:19:27.632 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:27.628+0000 7f3d5a8b9640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f3d5415a0b0 0x7f3d54155160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:19:27.632 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:27.628+0000 7f3d5a8b9640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f3d5415a0b0 0x7f3d54155160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:40574/0 (socket says 172.21.15.143:40574) 2023-11-26T07:19:27.632 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:27.628+0000 7f3d5a8b9640 1 -- 172.21.15.143:0/2026749584 learned_addr learned my addr 172.21.15.143:0/2026749584 (peer_addr_for_me v2:172.21.15.143:0/0) 2023-11-26T07:19:27.632 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:27.628+0000 7f3d5a0b8640 1 --2- 172.21.15.143:0/2026749584 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f3d40005050 0x7f3d541523b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:19:27.632 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:27.628+0000 7f3d598b7640 1 --2- 172.21.15.143:0/2026749584 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f3d54159c30 0x7f3d54154c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:19:27.632 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:27.628+0000 7f3d5a8b9640 1 -- 172.21.15.143:0/2026749584 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f3d40005050 msgr2=0x7f3d541523b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:19:27.632 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:27.628+0000 7f3d5a8b9640 1 --2- 172.21.15.143:0/2026749584 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f3d40005050 0x7f3d541523b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:19:27.632 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:27.628+0000 7f3d5a8b9640 1 -- 172.21.15.143:0/2026749584 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f3d54159c30 msgr2=0x7f3d54154c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:19:27.632 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:27.628+0000 7f3d5a8b9640 1 --2- 172.21.15.143:0/2026749584 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f3d54159c30 0x7f3d54154c20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:19:27.632 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:27.628+0000 7f3d5a8b9640 1 -- 172.21.15.143:0/2026749584 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3d54156510 con 0x7f3d5415a0b0 2023-11-26T07:19:27.633 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:27.628+0000 7f3d598b7640 1 --2- 172.21.15.143:0/2026749584 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f3d54159c30 0x7f3d54154c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-26T07:19:27.633 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:27.628+0000 7f3d5a0b8640 1 --2- 172.21.15.143:0/2026749584 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f3d40005050 0x7f3d541523b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-26T07:19:27.633 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:27.628+0000 7f3d5a8b9640 1 --2- 172.21.15.143:0/2026749584 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f3d5415a0b0 0x7f3d54155160 secure :-1 s=READY pgs=546 cs=0 l=1 rev1=1 crypto rx=0x7f3d4c020300 tx=0x7f3d40005800 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:19:27.633 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:27.628+0000 7f3d4affd640 1 -- 172.21.15.143:0/2026749584 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3d4c0af040 con 0x7f3d5415a0b0 2023-11-26T07:19:27.633 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:27.628+0000 7f3d4affd640 1 -- 172.21.15.143:0/2026749584 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3d4c0aa800 con 0x7f3d5415a0b0 2023-11-26T07:19:27.633 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:27.628+0000 7f3d5c31c640 1 -- 172.21.15.143:0/2026749584 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3d5415a960 con 0x7f3d5415a0b0 2023-11-26T07:19:27.633 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:27.628+0000 7f3d4affd640 1 -- 172.21.15.143:0/2026749584 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3d4c0aabb0 con 0x7f3d5415a0b0 2023-11-26T07:19:27.633 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:27.628+0000 7f3d5c31c640 1 -- 172.21.15.143:0/2026749584 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3d54152dd0 con 0x7f3d5415a0b0 2023-11-26T07:19:27.633 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:27.628+0000 7f3d5c31c640 1 -- 172.21.15.143:0/2026749584 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3d54067680 con 0x7f3d5415a0b0 2023-11-26T07:19:27.634 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:27.632+0000 7f3d4affd640 1 -- 172.21.15.143:0/2026749584 <== mon.0 v2:172.21.15.143:3300/0 4 ==== mgrmap(e 7) v1 ==== 86005+0+0 (secure 0 0 0) 0x7f3d4c0aad10 con 0x7f3d5415a0b0 2023-11-26T07:19:27.634 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:27.632+0000 7f3d4affd640 1 --2- 172.21.15.143:0/2026749584 >> [v2:172.21.15.143:6832/2150557544,v1:172.21.15.143:6833/2150557544] conn(0x7f3d38066650 0x7f3d38068b10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:19:27.634 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:27.632+0000 7f3d4affd640 1 -- 172.21.15.143:0/2026749584 <== mon.0 v2:172.21.15.143:3300/0 5 ==== osd_map(41..41 src has 1..41) v4 ==== 4857+0+0 (secure 0 0 0) 0x7f3d4c121070 con 0x7f3d5415a0b0 2023-11-26T07:19:27.634 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:27.632+0000 7f3d5a0b8640 1 --2- 172.21.15.143:0/2026749584 >> [v2:172.21.15.143:6832/2150557544,v1:172.21.15.143:6833/2150557544] conn(0x7f3d38066650 0x7f3d38068b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:19:27.635 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:27.632+0000 7f3d5a0b8640 1 --2- 172.21.15.143:0/2026749584 >> [v2:172.21.15.143:6832/2150557544,v1:172.21.15.143:6833/2150557544] conn(0x7f3d38066650 0x7f3d38068b10 secure :-1 s=READY pgs=277 cs=0 l=1 rev1=1 crypto rx=0x7f3d50009b60 tx=0x7f3d50008040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:19:27.638 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:27.636+0000 7f3d4affd640 1 -- 172.21.15.143:0/2026749584 <== mon.0 v2:172.21.15.143:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172649 (secure 0 0 0) 0x7f3d4c0ee4c0 con 0x7f3d5415a0b0 2023-11-26T07:19:27.842 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:27.840+0000 7f3d5c31c640 1 -- 172.21.15.143:0/2026749584 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "auth ls", "format": "json-pretty"} v 0) v1 -- 0x7f3d540709f0 con 0x7f3d5415a0b0 2023-11-26T07:19:27.843 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:27.840+0000 7f3d4affd640 1 -- 172.21.15.143:0/2026749584 <== mon.0 v2:172.21.15.143:3300/0 7 ==== mon_command_ack([{"prefix": "auth ls", "format": "json-pretty"}]=0 v13) v1 ==== 80+0+5304 (secure 0 0 0) 0x7f3d4c0f1ae0 con 0x7f3d5415a0b0 2023-11-26T07:19:27.843 INFO:teuthology.orchestra.run.smithi143.stdout: 2023-11-26T07:19:27.843 INFO:teuthology.orchestra.run.smithi143.stdout:{ 2023-11-26T07:19:27.843 INFO:teuthology.orchestra.run.smithi143.stdout: "auth_dump": [ 2023-11-26T07:19:27.843 INFO:teuthology.orchestra.run.smithi143.stdout: { 2023-11-26T07:19:27.843 INFO:teuthology.orchestra.run.smithi143.stdout: "entity": "mds.a", 2023-11-26T07:19:27.844 INFO:teuthology.orchestra.run.smithi143.stdout: "key": "AQDl8GJldeBsOBAAhk75XZnQRURKcr0MrW17Lg==", 2023-11-26T07:19:27.844 INFO:teuthology.orchestra.run.smithi143.stdout: "caps": { 2023-11-26T07:19:27.844 INFO:teuthology.orchestra.run.smithi143.stdout: "mds": "allow", 2023-11-26T07:19:27.844 INFO:teuthology.orchestra.run.smithi143.stdout: "mgr": "allow *", 2023-11-26T07:19:27.844 INFO:teuthology.orchestra.run.smithi143.stdout: "mon": "allow *", 2023-11-26T07:19:27.844 INFO:teuthology.orchestra.run.smithi143.stdout: "osd": "allow *" 2023-11-26T07:19:27.844 INFO:teuthology.orchestra.run.smithi143.stdout: } 2023-11-26T07:19:27.844 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2023-11-26T07:19:27.844 INFO:teuthology.orchestra.run.smithi143.stdout: { 2023-11-26T07:19:27.844 INFO:teuthology.orchestra.run.smithi143.stdout: "entity": "mds.b", 2023-11-26T07:19:27.844 INFO:teuthology.orchestra.run.smithi143.stdout: "key": "AQDm8GJl0lvhCBAAXuvHKoB8TAhp0bJ4DcV94g==", 2023-11-26T07:19:27.844 INFO:teuthology.orchestra.run.smithi143.stdout: "caps": { 2023-11-26T07:19:27.844 INFO:teuthology.orchestra.run.smithi143.stdout: "mds": "allow", 2023-11-26T07:19:27.844 INFO:teuthology.orchestra.run.smithi143.stdout: "mgr": "allow *", 2023-11-26T07:19:27.845 INFO:teuthology.orchestra.run.smithi143.stdout: "mon": "allow *", 2023-11-26T07:19:27.845 INFO:teuthology.orchestra.run.smithi143.stdout: "osd": "allow *" 2023-11-26T07:19:27.845 INFO:teuthology.orchestra.run.smithi143.stdout: } 2023-11-26T07:19:27.845 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2023-11-26T07:19:27.845 INFO:teuthology.orchestra.run.smithi143.stdout: { 2023-11-26T07:19:27.845 INFO:teuthology.orchestra.run.smithi143.stdout: "entity": "mds.c", 2023-11-26T07:19:27.845 INFO:teuthology.orchestra.run.smithi143.stdout: "key": "AQDm8GJluQ0PBBAA23EmVdD3bVk52h6izUXDRQ==", 2023-11-26T07:19:27.845 INFO:teuthology.orchestra.run.smithi143.stdout: "caps": { 2023-11-26T07:19:27.845 INFO:teuthology.orchestra.run.smithi143.stdout: "mds": "allow", 2023-11-26T07:19:27.845 INFO:teuthology.orchestra.run.smithi143.stdout: "mgr": "allow *", 2023-11-26T07:19:27.845 INFO:teuthology.orchestra.run.smithi143.stdout: "mon": "allow *", 2023-11-26T07:19:27.845 INFO:teuthology.orchestra.run.smithi143.stdout: "osd": "allow *" 2023-11-26T07:19:27.845 INFO:teuthology.orchestra.run.smithi143.stdout: } 2023-11-26T07:19:27.846 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2023-11-26T07:19:27.846 INFO:teuthology.orchestra.run.smithi143.stdout: { 2023-11-26T07:19:27.846 INFO:teuthology.orchestra.run.smithi143.stdout: "entity": "mds.d", 2023-11-26T07:19:27.846 INFO:teuthology.orchestra.run.smithi143.stdout: "key": "AQDm8GJlkt9nEBAAg+LYtKVNHAN/fwYjEcNJHQ==", 2023-11-26T07:19:27.846 INFO:teuthology.orchestra.run.smithi143.stdout: "caps": { 2023-11-26T07:19:27.846 INFO:teuthology.orchestra.run.smithi143.stdout: "mds": "allow", 2023-11-26T07:19:27.846 INFO:teuthology.orchestra.run.smithi143.stdout: "mgr": "allow *", 2023-11-26T07:19:27.846 INFO:teuthology.orchestra.run.smithi143.stdout: "mon": "allow *", 2023-11-26T07:19:27.846 INFO:teuthology.orchestra.run.smithi143.stdout: "osd": "allow *" 2023-11-26T07:19:27.846 INFO:teuthology.orchestra.run.smithi143.stdout: } 2023-11-26T07:19:27.846 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2023-11-26T07:19:27.846 INFO:teuthology.orchestra.run.smithi143.stdout: { 2023-11-26T07:19:27.846 INFO:teuthology.orchestra.run.smithi143.stdout: "entity": "osd.0", 2023-11-26T07:19:27.847 INFO:teuthology.orchestra.run.smithi143.stdout: "key": "AQDp8GJlNscZLhAA31zK4vTcnxwiRIRgxzewpA==", 2023-11-26T07:19:27.847 INFO:teuthology.orchestra.run.smithi143.stdout: "caps": { 2023-11-26T07:19:27.847 INFO:teuthology.orchestra.run.smithi143.stdout: "mgr": "allow profile osd", 2023-11-26T07:19:27.847 INFO:teuthology.orchestra.run.smithi143.stdout: "mon": "allow profile osd", 2023-11-26T07:19:27.847 INFO:teuthology.orchestra.run.smithi143.stdout: "osd": "allow *" 2023-11-26T07:19:27.847 INFO:teuthology.orchestra.run.smithi143.stdout: } 2023-11-26T07:19:27.847 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2023-11-26T07:19:27.847 INFO:teuthology.orchestra.run.smithi143.stdout: { 2023-11-26T07:19:27.847 INFO:teuthology.orchestra.run.smithi143.stdout: "entity": "osd.1", 2023-11-26T07:19:27.847 INFO:teuthology.orchestra.run.smithi143.stdout: "key": "AQDs8GJlsyKUNBAA+dQ1ElGPrdP1GV7lk4JQjQ==", 2023-11-26T07:19:27.847 INFO:teuthology.orchestra.run.smithi143.stdout: "caps": { 2023-11-26T07:19:27.847 INFO:teuthology.orchestra.run.smithi143.stdout: "mgr": "allow profile osd", 2023-11-26T07:19:27.847 INFO:teuthology.orchestra.run.smithi143.stdout: "mon": "allow profile osd", 2023-11-26T07:19:27.847 INFO:teuthology.orchestra.run.smithi143.stdout: "osd": "allow *" 2023-11-26T07:19:27.848 INFO:teuthology.orchestra.run.smithi143.stdout: } 2023-11-26T07:19:27.848 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2023-11-26T07:19:27.848 INFO:teuthology.orchestra.run.smithi143.stdout: { 2023-11-26T07:19:27.848 INFO:teuthology.orchestra.run.smithi143.stdout: "entity": "osd.2", 2023-11-26T07:19:27.848 INFO:teuthology.orchestra.run.smithi143.stdout: "key": "AQDv8GJlXsy7OhAAJJdldsodgGZVtyV2HuyC4A==", 2023-11-26T07:19:27.848 INFO:teuthology.orchestra.run.smithi143.stdout: "caps": { 2023-11-26T07:19:27.848 INFO:teuthology.orchestra.run.smithi143.stdout: "mgr": "allow profile osd", 2023-11-26T07:19:27.848 INFO:teuthology.orchestra.run.smithi143.stdout: "mon": "allow profile osd", 2023-11-26T07:19:27.848 INFO:teuthology.orchestra.run.smithi143.stdout: "osd": "allow *" 2023-11-26T07:19:27.848 INFO:teuthology.orchestra.run.smithi143.stdout: } 2023-11-26T07:19:27.848 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2023-11-26T07:19:27.848 INFO:teuthology.orchestra.run.smithi143.stdout: { 2023-11-26T07:19:27.848 INFO:teuthology.orchestra.run.smithi143.stdout: "entity": "osd.3", 2023-11-26T07:19:27.848 INFO:teuthology.orchestra.run.smithi143.stdout: "key": "AQDz8GJloLhMBxAAOWD9ec2y4c8nkdJmQnne8Q==", 2023-11-26T07:19:27.848 INFO:teuthology.orchestra.run.smithi143.stdout: "caps": { 2023-11-26T07:19:27.849 INFO:teuthology.orchestra.run.smithi143.stdout: "mgr": "allow profile osd", 2023-11-26T07:19:27.849 INFO:teuthology.orchestra.run.smithi143.stdout: "mon": "allow profile osd", 2023-11-26T07:19:27.849 INFO:teuthology.orchestra.run.smithi143.stdout: "osd": "allow *" 2023-11-26T07:19:27.849 INFO:teuthology.orchestra.run.smithi143.stdout: } 2023-11-26T07:19:27.849 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2023-11-26T07:19:27.849 INFO:teuthology.orchestra.run.smithi143.stdout: { 2023-11-26T07:19:27.849 INFO:teuthology.orchestra.run.smithi143.stdout: "entity": "osd.4", 2023-11-26T07:19:27.849 INFO:teuthology.orchestra.run.smithi143.stdout: "key": "AQD58GJlDlZsDBAAZNauAPPPfAZ+KIqSmgSQhg==", 2023-11-26T07:19:27.849 INFO:teuthology.orchestra.run.smithi143.stdout: "caps": { 2023-11-26T07:19:27.849 INFO:teuthology.orchestra.run.smithi143.stdout: "mgr": "allow profile osd", 2023-11-26T07:19:27.849 INFO:teuthology.orchestra.run.smithi143.stdout: "mon": "allow profile osd", 2023-11-26T07:19:27.849 INFO:teuthology.orchestra.run.smithi143.stdout: "osd": "allow *" 2023-11-26T07:19:27.849 INFO:teuthology.orchestra.run.smithi143.stdout: } 2023-11-26T07:19:27.849 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2023-11-26T07:19:27.850 INFO:teuthology.orchestra.run.smithi143.stdout: { 2023-11-26T07:19:27.850 INFO:teuthology.orchestra.run.smithi143.stdout: "entity": "osd.5", 2023-11-26T07:19:27.850 INFO:teuthology.orchestra.run.smithi143.stdout: "key": "AQD88GJlNE4mExAA5jYkCa0tE3oR6rgK1Dt5cA==", 2023-11-26T07:19:27.850 INFO:teuthology.orchestra.run.smithi143.stdout: "caps": { 2023-11-26T07:19:27.850 INFO:teuthology.orchestra.run.smithi143.stdout: "mgr": "allow profile osd", 2023-11-26T07:19:27.850 INFO:teuthology.orchestra.run.smithi143.stdout: "mon": "allow profile osd", 2023-11-26T07:19:27.850 INFO:teuthology.orchestra.run.smithi143.stdout: "osd": "allow *" 2023-11-26T07:19:27.850 INFO:teuthology.orchestra.run.smithi143.stdout: } 2023-11-26T07:19:27.850 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2023-11-26T07:19:27.850 INFO:teuthology.orchestra.run.smithi143.stdout: { 2023-11-26T07:19:27.850 INFO:teuthology.orchestra.run.smithi143.stdout: "entity": "osd.6", 2023-11-26T07:19:27.850 INFO:teuthology.orchestra.run.smithi143.stdout: "key": "AQD/8GJlh0lYGRAArHalCyZ36UvBOVMZor+N/g==", 2023-11-26T07:19:27.850 INFO:teuthology.orchestra.run.smithi143.stdout: "caps": { 2023-11-26T07:19:27.850 INFO:teuthology.orchestra.run.smithi143.stdout: "mgr": "allow profile osd", 2023-11-26T07:19:27.851 INFO:teuthology.orchestra.run.smithi143.stdout: "mon": "allow profile osd", 2023-11-26T07:19:27.851 INFO:teuthology.orchestra.run.smithi143.stdout: "osd": "allow *" 2023-11-26T07:19:27.851 INFO:teuthology.orchestra.run.smithi143.stdout: } 2023-11-26T07:19:27.851 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2023-11-26T07:19:27.851 INFO:teuthology.orchestra.run.smithi143.stdout: { 2023-11-26T07:19:27.851 INFO:teuthology.orchestra.run.smithi143.stdout: "entity": "osd.7", 2023-11-26T07:19:27.851 INFO:teuthology.orchestra.run.smithi143.stdout: "key": "AQAC8WJlJavtIBAAKMvnR4AnnrDVOXlzN7j05w==", 2023-11-26T07:19:27.851 INFO:teuthology.orchestra.run.smithi143.stdout: "caps": { 2023-11-26T07:19:27.851 INFO:teuthology.orchestra.run.smithi143.stdout: "mgr": "allow profile osd", 2023-11-26T07:19:27.851 INFO:teuthology.orchestra.run.smithi143.stdout: "mon": "allow profile osd", 2023-11-26T07:19:27.851 INFO:teuthology.orchestra.run.smithi143.stdout: "osd": "allow *" 2023-11-26T07:19:27.851 INFO:teuthology.orchestra.run.smithi143.stdout: } 2023-11-26T07:19:27.851 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2023-11-26T07:19:27.851 INFO:teuthology.orchestra.run.smithi143.stdout: { 2023-11-26T07:19:27.851 INFO:teuthology.orchestra.run.smithi143.stdout: "entity": "client.0", 2023-11-26T07:19:27.852 INFO:teuthology.orchestra.run.smithi143.stdout: "key": "AQDm8GJlWMh9FBAAslVzi6fm4/wt7vr/rHCHHQ==", 2023-11-26T07:19:27.852 INFO:teuthology.orchestra.run.smithi143.stdout: "caps": { 2023-11-26T07:19:27.852 INFO:teuthology.orchestra.run.smithi143.stdout: "mds": "allow", 2023-11-26T07:19:27.852 INFO:teuthology.orchestra.run.smithi143.stdout: "mon": "allow r", 2023-11-26T07:19:27.852 INFO:teuthology.orchestra.run.smithi143.stdout: "osd": "allow rw tag cephfs data=cephfs" 2023-11-26T07:19:27.852 INFO:teuthology.orchestra.run.smithi143.stdout: } 2023-11-26T07:19:27.852 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2023-11-26T07:19:27.852 INFO:teuthology.orchestra.run.smithi143.stdout: { 2023-11-26T07:19:27.852 INFO:teuthology.orchestra.run.smithi143.stdout: "entity": "client.1", 2023-11-26T07:19:27.852 INFO:teuthology.orchestra.run.smithi143.stdout: "key": "AQDm8GJl5vkYHhAAgP+gKH2+lpl3aTlgqZNIXg==", 2023-11-26T07:19:27.852 INFO:teuthology.orchestra.run.smithi143.stdout: "caps": { 2023-11-26T07:19:27.852 INFO:teuthology.orchestra.run.smithi143.stdout: "mds": "allow", 2023-11-26T07:19:27.852 INFO:teuthology.orchestra.run.smithi143.stdout: "mgr": "allow r", 2023-11-26T07:19:27.852 INFO:teuthology.orchestra.run.smithi143.stdout: "mon": "allow rw", 2023-11-26T07:19:27.853 INFO:teuthology.orchestra.run.smithi143.stdout: "osd": "allow rwx" 2023-11-26T07:19:27.853 INFO:teuthology.orchestra.run.smithi143.stdout: } 2023-11-26T07:19:27.853 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2023-11-26T07:19:27.853 INFO:teuthology.orchestra.run.smithi143.stdout: { 2023-11-26T07:19:27.853 INFO:teuthology.orchestra.run.smithi143.stdout: "entity": "client.2", 2023-11-26T07:19:27.853 INFO:teuthology.orchestra.run.smithi143.stdout: "key": "AQDm8GJlW4KlGhAANFKsyVJems0ZoE6bo3WmfQ==", 2023-11-26T07:19:27.853 INFO:teuthology.orchestra.run.smithi143.stdout: "caps": { 2023-11-26T07:19:27.853 INFO:teuthology.orchestra.run.smithi143.stdout: "mds": "allow", 2023-11-26T07:19:27.853 INFO:teuthology.orchestra.run.smithi143.stdout: "mgr": "allow r", 2023-11-26T07:19:27.853 INFO:teuthology.orchestra.run.smithi143.stdout: "mon": "allow rw", 2023-11-26T07:19:27.853 INFO:teuthology.orchestra.run.smithi143.stdout: "osd": "allow rwx" 2023-11-26T07:19:27.853 INFO:teuthology.orchestra.run.smithi143.stdout: } 2023-11-26T07:19:27.853 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2023-11-26T07:19:27.853 INFO:teuthology.orchestra.run.smithi143.stdout: { 2023-11-26T07:19:27.854 INFO:teuthology.orchestra.run.smithi143.stdout: "entity": "client.3", 2023-11-26T07:19:27.854 INFO:teuthology.orchestra.run.smithi143.stdout: "key": "AQDm8GJlAJZSJBAAUSUNcwRBY01hOuilBJjlFQ==", 2023-11-26T07:19:27.854 INFO:teuthology.orchestra.run.smithi143.stdout: "caps": { 2023-11-26T07:19:27.854 INFO:teuthology.orchestra.run.smithi143.stdout: "mds": "allow", 2023-11-26T07:19:27.854 INFO:teuthology.orchestra.run.smithi143.stdout: "mgr": "allow r", 2023-11-26T07:19:27.854 INFO:teuthology.orchestra.run.smithi143.stdout: "mon": "allow rw", 2023-11-26T07:19:27.854 INFO:teuthology.orchestra.run.smithi143.stdout: "osd": "allow rwx" 2023-11-26T07:19:27.854 INFO:teuthology.orchestra.run.smithi143.stdout: } 2023-11-26T07:19:27.854 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2023-11-26T07:19:27.854 INFO:teuthology.orchestra.run.smithi143.stdout: { 2023-11-26T07:19:27.854 INFO:teuthology.orchestra.run.smithi143.stdout: "entity": "client.admin", 2023-11-26T07:19:27.854 INFO:teuthology.orchestra.run.smithi143.stdout: "key": "AQDl8GJlbQv7GxAA0Midh8tAO2KO+dH10OV8Xg==", 2023-11-26T07:19:27.854 INFO:teuthology.orchestra.run.smithi143.stdout: "caps": { 2023-11-26T07:19:27.854 INFO:teuthology.orchestra.run.smithi143.stdout: "mds": "allow *", 2023-11-26T07:19:27.855 INFO:teuthology.orchestra.run.smithi143.stdout: "mgr": "allow *", 2023-11-26T07:19:27.855 INFO:teuthology.orchestra.run.smithi143.stdout: "mon": "allow *", 2023-11-26T07:19:27.855 INFO:teuthology.orchestra.run.smithi143.stdout: "osd": "allow *" 2023-11-26T07:19:27.855 INFO:teuthology.orchestra.run.smithi143.stdout: } 2023-11-26T07:19:27.855 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2023-11-26T07:19:27.855 INFO:teuthology.orchestra.run.smithi143.stdout: { 2023-11-26T07:19:27.855 INFO:teuthology.orchestra.run.smithi143.stdout: "entity": "mgr.x", 2023-11-26T07:19:27.855 INFO:teuthology.orchestra.run.smithi143.stdout: "key": "AQDl8GJlXy8cNBAA4f5RDC1MASCqZPcJQvnUDA==", 2023-11-26T07:19:27.855 INFO:teuthology.orchestra.run.smithi143.stdout: "caps": { 2023-11-26T07:19:27.855 INFO:teuthology.orchestra.run.smithi143.stdout: "mds": "allow *", 2023-11-26T07:19:27.855 INFO:teuthology.orchestra.run.smithi143.stdout: "mon": "allow profile mgr", 2023-11-26T07:19:27.855 INFO:teuthology.orchestra.run.smithi143.stdout: "osd": "allow *" 2023-11-26T07:19:27.855 INFO:teuthology.orchestra.run.smithi143.stdout: } 2023-11-26T07:19:27.855 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2023-11-26T07:19:27.855 INFO:teuthology.orchestra.run.smithi143.stdout: { 2023-11-26T07:19:27.856 INFO:teuthology.orchestra.run.smithi143.stdout: "entity": "mgr.y", 2023-11-26T07:19:27.856 INFO:teuthology.orchestra.run.smithi143.stdout: "key": "AQDl8GJlfuFPMBAA3TYEFvdfk4DyGsBkbpv0hg==", 2023-11-26T07:19:27.856 INFO:teuthology.orchestra.run.smithi143.stdout: "caps": { 2023-11-26T07:19:27.856 INFO:teuthology.orchestra.run.smithi143.stdout: "mds": "allow *", 2023-11-26T07:19:27.856 INFO:teuthology.orchestra.run.smithi143.stdout: "mon": "allow profile mgr", 2023-11-26T07:19:27.856 INFO:teuthology.orchestra.run.smithi143.stdout: "osd": "allow *" 2023-11-26T07:19:27.856 INFO:teuthology.orchestra.run.smithi143.stdout: } 2023-11-26T07:19:27.856 INFO:teuthology.orchestra.run.smithi143.stdout: } 2023-11-26T07:19:27.856 INFO:teuthology.orchestra.run.smithi143.stdout: ] 2023-11-26T07:19:27.856 INFO:teuthology.orchestra.run.smithi143.stdout:} 2023-11-26T07:19:27.857 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:27.844+0000 7f3d5c31c640 1 -- 172.21.15.143:0/2026749584 >> [v2:172.21.15.143:6832/2150557544,v1:172.21.15.143:6833/2150557544] conn(0x7f3d38066650 msgr2=0x7f3d38068b10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:19:27.857 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:27.844+0000 7f3d5c31c640 1 --2- 172.21.15.143:0/2026749584 >> [v2:172.21.15.143:6832/2150557544,v1:172.21.15.143:6833/2150557544] conn(0x7f3d38066650 0x7f3d38068b10 secure :-1 s=READY pgs=277 cs=0 l=1 rev1=1 crypto rx=0x7f3d50009b60 tx=0x7f3d50008040 comp rx=0 tx=0).stop 2023-11-26T07:19:27.857 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:27.844+0000 7f3d5c31c640 1 -- 172.21.15.143:0/2026749584 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f3d5415a0b0 msgr2=0x7f3d54155160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:19:27.857 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:27.844+0000 7f3d5c31c640 1 --2- 172.21.15.143:0/2026749584 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f3d5415a0b0 0x7f3d54155160 secure :-1 s=READY pgs=546 cs=0 l=1 rev1=1 crypto rx=0x7f3d4c020300 tx=0x7f3d40005800 comp rx=0 tx=0).stop 2023-11-26T07:19:27.857 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:27.844+0000 7f3d5c31c640 1 -- 172.21.15.143:0/2026749584 shutdown_connections 2023-11-26T07:19:27.857 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:27.844+0000 7f3d5c31c640 1 --2- 172.21.15.143:0/2026749584 >> [v2:172.21.15.143:6832/2150557544,v1:172.21.15.143:6833/2150557544] conn(0x7f3d38066650 0x7f3d38068b10 unknown :-1 s=CLOSED pgs=277 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:19:27.857 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:27.844+0000 7f3d5c31c640 1 --2- 172.21.15.143:0/2026749584 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f3d5415a0b0 0x7f3d54155160 unknown :-1 s=CLOSED pgs=546 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:19:27.857 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:27.844+0000 7f3d5c31c640 1 --2- 172.21.15.143:0/2026749584 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f3d54159c30 0x7f3d54154c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:19:27.858 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:27.844+0000 7f3d5c31c640 1 --2- 172.21.15.143:0/2026749584 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f3d40005050 0x7f3d541523b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:19:27.858 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:27.844+0000 7f3d5c31c640 1 -- 172.21.15.143:0/2026749584 >> 172.21.15.143:0/2026749584 conn(0x7f3d5406a890 msgr2=0x7f3d540af530 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:19:27.858 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:27.844+0000 7f3d5c31c640 1 -- 172.21.15.143:0/2026749584 shutdown_connections 2023-11-26T07:19:27.858 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:27.844+0000 7f3d5c31c640 1 -- 172.21.15.143:0/2026749584 wait complete. 2023-11-26T07:19:27.864 DEBUG:teuthology.orchestra.run.smithi143:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-11-26T07:19:28.003 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:28.000+0000 7f53089df640 1 Processor -- start 2023-11-26T07:19:28.003 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:28.000+0000 7f53089df640 1 -- start start 2023-11-26T07:19:28.005 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:28.004+0000 7f53089df640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f53040cd610 0x7f5304071970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:19:28.005 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:28.004+0000 7f53089df640 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7f5304070ba0 0x7f5304071010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:19:28.005 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:28.004+0000 7f53089df640 1 -- --> v1:172.21.15.143:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5304071eb0 con 0x7f53041614b0 2023-11-26T07:19:28.005 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:28.004+0000 7f53089df640 1 -- --> v2:172.21.15.188:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5304071550 con 0x7f5304070ba0 2023-11-26T07:19:28.005 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:28.004+0000 7f53089df640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f530415d970 con 0x7f53040cd610 2023-11-26T07:19:28.005 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:28.004+0000 7f530259c640 1 --1- >> v1:172.21.15.143:6789/0 conn(0x7f53041614b0 0x7f5304161890 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.143:6789/0 says I am v1:172.21.15.143:58344/0 (socket says 172.21.15.143:58344) 2023-11-26T07:19:28.005 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:28.004+0000 7f530259c640 1 -- 172.21.15.143:0/224460016 learned_addr learned my addr 172.21.15.143:0/224460016 (peer_addr_for_me v1:172.21.15.143:0/0) 2023-11-26T07:19:28.006 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:28.004+0000 7f5302d9d640 1 --2- 172.21.15.143:0/224460016 >> v2:172.21.15.188:3300/0 conn(0x7f5304070ba0 0x7f5304071010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:19:28.006 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:28.004+0000 7f52f9d9b640 1 --2- 172.21.15.143:0/224460016 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f53040cd610 0x7f5304071970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:19:28.006 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:28.004+0000 7f5301d9b640 1 -- 172.21.15.143:0/224460016 <== mon.0 v1:172.21.15.143:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2515649897 0 0) 0x7f5304071eb0 con 0x7f53041614b0 2023-11-26T07:19:28.006 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:28.004+0000 7f5301d9b640 1 -- 172.21.15.143:0/224460016 --> v1:172.21.15.143:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f52f0003530 con 0x7f53041614b0 2023-11-26T07:19:28.006 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:28.004+0000 7f5302d9d640 1 -- 172.21.15.143:0/224460016 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f53040cd610 msgr2=0x7f5304071970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-26T07:19:28.007 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:28.004+0000 7f5302d9d640 1 --2- 172.21.15.143:0/224460016 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f53040cd610 0x7f5304071970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:19:28.007 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:28.004+0000 7f5302d9d640 1 -- 172.21.15.143:0/224460016 >> v1:172.21.15.143:6789/0 conn(0x7f53041614b0 legacy=0x7f5304161890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:19:28.007 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:28.004+0000 7f5301d9b640 1 -- 172.21.15.143:0/224460016 <== mon.0 v1:172.21.15.143:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2553770188 0 0) 0x7f52f0003530 con 0x7f53041614b0 2023-11-26T07:19:28.007 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:28.004+0000 7f5302d9d640 1 -- 172.21.15.143:0/224460016 --> v2:172.21.15.188:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f530415dc30 con 0x7f5304070ba0 2023-11-26T07:19:28.009 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:28.008+0000 7f5302d9d640 1 --2- 172.21.15.143:0/224460016 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f5304070ba0 0x7f5304071010 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7f52fc0097c0 tx=0x7f52fc01c3a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=bed9c9421cf405ae server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:19:28.009 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:28.008+0000 7f5301d9b640 1 -- 172.21.15.143:0/224460016 <== mon.1 v2:172.21.15.188:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f52f4001680 con 0x7f5304070ba0 2023-11-26T07:19:28.009 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:28.008+0000 7f5301d9b640 1 -- 172.21.15.143:0/224460016 <== mon.1 v2:172.21.15.188:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f52fc0a7050 con 0x7f5304070ba0 2023-11-26T07:19:28.009 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:28.008+0000 7f5301d9b640 1 -- 172.21.15.143:0/224460016 <== mon.1 v2:172.21.15.188:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f52fc0ab600 con 0x7f5304070ba0 2023-11-26T07:19:28.010 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:28.008+0000 7f53089df640 1 -- 172.21.15.143:0/224460016 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f5304070ba0 msgr2=0x7f5304071010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:19:28.010 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:28.008+0000 7f53089df640 1 --2- 172.21.15.143:0/224460016 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f5304070ba0 0x7f5304071010 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7f52fc0097c0 tx=0x7f52fc01c3a0 comp rx=0 tx=0).stop 2023-11-26T07:19:28.010 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:28.008+0000 7f53089df640 1 -- 172.21.15.143:0/224460016 shutdown_connections 2023-11-26T07:19:28.010 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:28.008+0000 7f53089df640 1 --2- 172.21.15.143:0/224460016 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f5304070ba0 0x7f5304071010 unknown :-1 s=CLOSED pgs=240 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:19:28.010 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:28.008+0000 7f53089df640 1 --2- 172.21.15.143:0/224460016 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f53040cd610 0x7f5304071970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:19:28.010 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:28.008+0000 7f53089df640 1 -- 172.21.15.143:0/224460016 >> 172.21.15.143:0/224460016 conn(0x7f53040cf710 msgr2=0x7f53040cfb20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:19:28.010 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:28.008+0000 7f53089df640 1 -- 172.21.15.143:0/224460016 shutdown_connections 2023-11-26T07:19:28.010 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:28.008+0000 7f53089df640 1 -- 172.21.15.143:0/224460016 wait complete. 2023-11-26T07:19:28.012 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:28.008+0000 7f53089df640 1 Processor -- start 2023-11-26T07:19:28.012 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:28.008+0000 7f53089df640 1 -- start start 2023-11-26T07:19:28.014 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:28.012+0000 7f53089df640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f5304070ba0 0x7f5304156bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:19:28.014 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:28.012+0000 7f53089df640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f53040cd610 0x7f53040b3950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:19:28.014 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:28.012+0000 7f53089df640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f53041614b0 0x7f5304151a40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:19:28.014 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:28.012+0000 7f53089df640 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5304151f80 con 0x7f5304070ba0 2023-11-26T07:19:28.014 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:28.012+0000 7f53089df640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f53040b3fe0 con 0x7f53040cd610 2023-11-26T07:19:28.014 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:28.012+0000 7f53089df640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5304153960 con 0x7f53041614b0 2023-11-26T07:19:28.014 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:28.012+0000 7f530259c640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f5304070ba0 0x7f5304156bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:19:28.014 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:28.012+0000 7f530259c640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f5304070ba0 0x7f5304156bf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:40576/0 (socket says 172.21.15.143:40576) 2023-11-26T07:19:28.014 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:28.012+0000 7f530259c640 1 -- 172.21.15.143:0/1698072524 learned_addr learned my addr 172.21.15.143:0/1698072524 (peer_addr_for_me v2:172.21.15.143:0/0) 2023-11-26T07:19:28.014 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:28.012+0000 7f52f9d9b640 1 --2- 172.21.15.143:0/1698072524 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f53040cd610 0x7f53040b3950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:19:28.014 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:28.012+0000 7f5302d9d640 1 --2- 172.21.15.143:0/1698072524 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f53041614b0 0x7f5304151a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:19:28.014 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:28.012+0000 7f530259c640 1 -- 172.21.15.143:0/1698072524 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f53041614b0 msgr2=0x7f5304151a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:19:28.015 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:28.012+0000 7f530259c640 1 --2- 172.21.15.143:0/1698072524 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f53041614b0 0x7f5304151a40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:19:28.015 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:28.012+0000 7f530259c640 1 -- 172.21.15.143:0/1698072524 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f53040cd610 msgr2=0x7f53040b3950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:19:28.015 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:28.012+0000 7f530259c640 1 --2- 172.21.15.143:0/1698072524 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f53040cd610 0x7f53040b3950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:19:28.015 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:28.012+0000 7f530259c640 1 -- 172.21.15.143:0/1698072524 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f52fc009420 con 0x7f5304070ba0 2023-11-26T07:19:28.015 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:28.012+0000 7f530259c640 1 --2- 172.21.15.143:0/1698072524 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f5304070ba0 0x7f5304156bf0 secure :-1 s=READY pgs=548 cs=0 l=1 rev1=1 crypto rx=0x7f52f400c750 tx=0x7f52f400cc10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:19:28.016 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:28.012+0000 7f52fb7fe640 1 -- 172.21.15.143:0/1698072524 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f52f4005ae0 con 0x7f5304070ba0 2023-11-26T07:19:28.016 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:28.012+0000 7f52fb7fe640 1 -- 172.21.15.143:0/1698072524 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f52f4005c40 con 0x7f5304070ba0 2023-11-26T07:19:28.016 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:28.012+0000 7f52fb7fe640 1 -- 172.21.15.143:0/1698072524 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f52f4007070 con 0x7f5304070ba0 2023-11-26T07:19:28.016 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:28.012+0000 7f53089df640 1 -- 172.21.15.143:0/1698072524 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f53041580b0 con 0x7f5304070ba0 2023-11-26T07:19:28.016 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:28.012+0000 7f53089df640 1 -- 172.21.15.143:0/1698072524 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5304155930 con 0x7f5304070ba0 2023-11-26T07:19:28.016 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:28.012+0000 7f53089df640 1 -- 172.21.15.143:0/1698072524 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5304067680 con 0x7f5304070ba0 2023-11-26T07:19:28.016 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:28.012+0000 7f52fb7fe640 1 -- 172.21.15.143:0/1698072524 <== mon.0 v2:172.21.15.143:3300/0 4 ==== mgrmap(e 7) v1 ==== 86005+0+0 (secure 0 0 0) 0x7f52f40071d0 con 0x7f5304070ba0 2023-11-26T07:19:28.017 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:28.016+0000 7f52fb7fe640 1 --2- 172.21.15.143:0/1698072524 >> [v2:172.21.15.143:6832/2150557544,v1:172.21.15.143:6833/2150557544] conn(0x7f52e4066620 0x7f52e4068ae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:19:28.017 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:28.016+0000 7f52fb7fe640 1 -- 172.21.15.143:0/1698072524 <== mon.0 v2:172.21.15.143:3300/0 5 ==== osd_map(41..41 src has 1..41) v4 ==== 4857+0+0 (secure 0 0 0) 0x7f52f4083b10 con 0x7f5304070ba0 2023-11-26T07:19:28.017 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:28.016+0000 7f52f9d9b640 1 --2- 172.21.15.143:0/1698072524 >> [v2:172.21.15.143:6832/2150557544,v1:172.21.15.143:6833/2150557544] conn(0x7f52e4066620 0x7f52e4068ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:19:28.017 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:28.016+0000 7f52f9d9b640 1 --2- 172.21.15.143:0/1698072524 >> [v2:172.21.15.143:6832/2150557544,v1:172.21.15.143:6833/2150557544] conn(0x7f52e4066620 0x7f52e4068ae0 secure :-1 s=READY pgs=278 cs=0 l=1 rev1=1 crypto rx=0x7f53041f9240 tx=0x7f52ec00b040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:19:28.021 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:28.020+0000 7f52fb7fe640 1 -- 172.21.15.143:0/1698072524 <== mon.0 v2:172.21.15.143:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172649 (secure 0 0 0) 0x7f52f4051100 con 0x7f5304070ba0 2023-11-26T07:19:28.172 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:28.168+0000 7f53089df640 1 -- 172.21.15.143:0/1698072524 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f53040b4150 con 0x7f5304070ba0 2023-11-26T07:19:28.173 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:28.172+0000 7f52fb7fe640 1 -- 172.21.15.143:0/1698072524 <== mon.0 v2:172.21.15.143:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v41) v1 ==== 74+0+12893 (secure 0 0 0) 0x7f52f4054720 con 0x7f5304070ba0 2023-11-26T07:19:28.173 INFO:teuthology.orchestra.run.smithi143.stdout: 2023-11-26T07:19:28.174 INFO:teuthology.orchestra.run.smithi143.stdout:{"epoch":41,"fsid":"f5033060-2469-4a07-95fe-2b76f5404bc1","created":"2023-11-26T07:17:28.613663+0000","modified":"2023-11-26T07:19:27.452975+0000","last_up_change":"2023-11-26T07:17:41.157813+0000","last_in_change":"2023-11-26T07:17:31.181658+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":5,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-11-26T07:17:41.758597+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"ebbee2c6-023e-44b9-9f1d-6f7bfd309730","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6817","nonce":639807369},{"type":"v1","addr":"172.21.15.143:6819","nonce":639807369}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6821","nonce":639807369},{"type":"v1","addr":"172.21.15.143:6823","nonce":639807369}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6830","nonce":639807369},{"type":"v1","addr":"172.21.15.143:6831","nonce":639807369}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6825","nonce":639807369},{"type":"v1","addr":"172.21.15.143:6828","nonce":639807369}]},"public_addr":"172.21.15.143:6819/639807369","cluster_addr":"172.21.15.143:6823/639807369","heartbeat_back_addr":"172.21.15.143:6831/639807369","heartbeat_front_addr":"172.21.15.143:6828/639807369","state":["exists","up"]},{"osd":1,"uuid":"fec4b8be-7122-49e9-9a19-5850a0700001","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6816","nonce":1144942530},{"type":"v1","addr":"172.21.15.143:6818","nonce":1144942530}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6820","nonce":1144942530},{"type":"v1","addr":"172.21.15.143:6822","nonce":1144942530}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6827","nonce":1144942530},{"type":"v1","addr":"172.21.15.143:6829","nonce":1144942530}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6824","nonce":1144942530},{"type":"v1","addr":"172.21.15.143:6826","nonce":1144942530}]},"public_addr":"172.21.15.143:6818/1144942530","cluster_addr":"172.21.15.143:6822/1144942530","heartbeat_back_addr":"172.21.15.143:6829/1144942530","heartbeat_front_addr":"172.21.15.143:6826/1144942530","state":["exists","up"]},{"osd":2,"uuid":"2888af5b-f933-46ca-8f95-6dd494ff43f6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6808","nonce":3909095186},{"type":"v1","addr":"172.21.15.143:6809","nonce":3909095186}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6810","nonce":3909095186},{"type":"v1","addr":"172.21.15.143:6811","nonce":3909095186}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6814","nonce":3909095186},{"type":"v1","addr":"172.21.15.143:6815","nonce":3909095186}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6812","nonce":3909095186},{"type":"v1","addr":"172.21.15.143:6813","nonce":3909095186}]},"public_addr":"172.21.15.143:6809/3909095186","cluster_addr":"172.21.15.143:6811/3909095186","heartbeat_back_addr":"172.21.15.143:6815/3909095186","heartbeat_front_addr":"172.21.15.143:6813/3909095186","state":["exists","up"]},{"osd":3,"uuid":"d1f52435-7bc1-4081-9550-9cd1ba55f549","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6800","nonce":3854210991},{"type":"v1","addr":"172.21.15.143:6801","nonce":3854210991}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6802","nonce":3854210991},{"type":"v1","addr":"172.21.15.143:6803","nonce":3854210991}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6806","nonce":3854210991},{"type":"v1","addr":"172.21.15.143:6807","nonce":3854210991}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6804","nonce":3854210991},{"type":"v1","addr":"172.21.15.143:6805","nonce":3854210991}]},"public_addr":"172.21.15.143:6801/3854210991","cluster_addr":"172.21.15.143:6803/3854210991","heartbeat_back_addr":"172.21.15.143:6807/3854210991","heartbeat_front_addr":"172.21.15.143:6805/3854210991","state":["exists","up"]},{"osd":4,"uuid":"e2ae8b6e-6e25-49b2-bcbf-aacce68f06a3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6800","nonce":3924606902},{"type":"v1","addr":"172.21.15.188:6801","nonce":3924606902}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6802","nonce":3924606902},{"type":"v1","addr":"172.21.15.188:6803","nonce":3924606902}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6806","nonce":3924606902},{"type":"v1","addr":"172.21.15.188:6807","nonce":3924606902}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6804","nonce":3924606902},{"type":"v1","addr":"172.21.15.188:6805","nonce":3924606902}]},"public_addr":"172.21.15.188:6801/3924606902","cluster_addr":"172.21.15.188:6803/3924606902","heartbeat_back_addr":"172.21.15.188:6807/3924606902","heartbeat_front_addr":"172.21.15.188:6805/3924606902","state":["exists","up"]},{"osd":5,"uuid":"3cec8a32-7698-4c36-8e89-e4ba11934f33","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6813","nonce":186841544},{"type":"v1","addr":"172.21.15.188:6815","nonce":186841544}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6817","nonce":186841544},{"type":"v1","addr":"172.21.15.188:6818","nonce":186841544}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6822","nonce":186841544},{"type":"v1","addr":"172.21.15.188:6823","nonce":186841544}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6820","nonce":186841544},{"type":"v1","addr":"172.21.15.188:6821","nonce":186841544}]},"public_addr":"172.21.15.188:6815/186841544","cluster_addr":"172.21.15.188:6818/186841544","heartbeat_back_addr":"172.21.15.188:6823/186841544","heartbeat_front_addr":"172.21.15.188:6821/186841544","state":["exists","up"]},{"osd":6,"uuid":"58a65d5d-e090-473a-81b4-ad38e85029f9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6808","nonce":2153499405},{"type":"v1","addr":"172.21.15.188:6809","nonce":2153499405}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6810","nonce":2153499405},{"type":"v1","addr":"172.21.15.188:6811","nonce":2153499405}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6816","nonce":2153499405},{"type":"v1","addr":"172.21.15.188:6819","nonce":2153499405}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6812","nonce":2153499405},{"type":"v1","addr":"172.21.15.188:6814","nonce":2153499405}]},"public_addr":"172.21.15.188:6809/2153499405","cluster_addr":"172.21.15.188:6811/2153499405","heartbeat_back_addr":"172.21.15.188:6819/2153499405","heartbeat_front_addr":"172.21.15.188:6814/2153499405","state":["exists","up"]},{"osd":7,"uuid":"ea5b7f70-3554-4cf8-ac30-f7c59b80678d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6824","nonce":2722096729},{"type":"v1","addr":"172.21.15.188:6825","nonce":2722096729}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6826","nonce":2722096729},{"type":"v1","addr":"172.21.15.188:6827","nonce":2722096729}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6830","nonce":2722096729},{"type":"v1","addr":"172.21.15.188:6831","nonce":2722096729}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6828","nonce":2722096729},{"type":"v1","addr":"172.21.15.188:6829","nonce":2722096729}]},"public_addr":"172.21.15.188:6825/2722096729","cluster_addr":"172.21.15.188:6827/2722096729","heartbeat_back_addr":"172.21.15.188:6831/2722096729","heartbeat_front_addr":"172.21.15.188:6829/2722096729","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-26T07:17:36.995032+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-26T07:17:36.932266+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-26T07:17:36.829304+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-26T07:17:36.804297+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-26T07:17:36.667892+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-26T07:17:36.964868+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-26T07:17:36.827332+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-26T07:17:36.893179+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-11-26T07:19:28.175 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:28.172+0000 7f53089df640 1 -- 172.21.15.143:0/1698072524 >> [v2:172.21.15.143:6832/2150557544,v1:172.21.15.143:6833/2150557544] conn(0x7f52e4066620 msgr2=0x7f52e4068ae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:19:28.176 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:28.172+0000 7f53089df640 1 --2- 172.21.15.143:0/1698072524 >> [v2:172.21.15.143:6832/2150557544,v1:172.21.15.143:6833/2150557544] conn(0x7f52e4066620 0x7f52e4068ae0 secure :-1 s=READY pgs=278 cs=0 l=1 rev1=1 crypto rx=0x7f53041f9240 tx=0x7f52ec00b040 comp rx=0 tx=0).stop 2023-11-26T07:19:28.176 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:28.172+0000 7f53089df640 1 -- 172.21.15.143:0/1698072524 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f5304070ba0 msgr2=0x7f5304156bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:19:28.176 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:28.172+0000 7f53089df640 1 --2- 172.21.15.143:0/1698072524 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f5304070ba0 0x7f5304156bf0 secure :-1 s=READY pgs=548 cs=0 l=1 rev1=1 crypto rx=0x7f52f400c750 tx=0x7f52f400cc10 comp rx=0 tx=0).stop 2023-11-26T07:19:28.176 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:28.172+0000 7f53089df640 1 -- 172.21.15.143:0/1698072524 shutdown_connections 2023-11-26T07:19:28.176 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:28.172+0000 7f53089df640 1 --2- 172.21.15.143:0/1698072524 >> [v2:172.21.15.143:6832/2150557544,v1:172.21.15.143:6833/2150557544] conn(0x7f52e4066620 0x7f52e4068ae0 unknown :-1 s=CLOSED pgs=278 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:19:28.176 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:28.172+0000 7f53089df640 1 --2- 172.21.15.143:0/1698072524 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f53041614b0 0x7f5304151a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:19:28.176 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:28.172+0000 7f53089df640 1 --2- 172.21.15.143:0/1698072524 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f53040cd610 0x7f53040b3950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:19:28.176 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:28.172+0000 7f53089df640 1 --2- 172.21.15.143:0/1698072524 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f5304070ba0 0x7f5304156bf0 unknown :-1 s=CLOSED pgs=548 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:19:28.176 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:28.172+0000 7f53089df640 1 -- 172.21.15.143:0/1698072524 >> 172.21.15.143:0/1698072524 conn(0x7f53040cf710 msgr2=0x7f530406b730 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:19:28.176 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:28.172+0000 7f53089df640 1 -- 172.21.15.143:0/1698072524 shutdown_connections 2023-11-26T07:19:28.176 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:28.172+0000 7f53089df640 1 -- 172.21.15.143:0/1698072524 wait complete. 2023-11-26T07:19:28.195 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-11-26T07:17:41.758597+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2023-11-26T07:19:28.195 DEBUG:teuthology.orchestra.run.smithi143:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-11-26T07:19:28.335 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:28.332+0000 7f0939668640 1 Processor -- start 2023-11-26T07:19:28.335 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:28.332+0000 7f0939668640 1 -- start start 2023-11-26T07:19:28.337 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:28.332+0000 7f0939668640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f093415d6e0 0x7f09340d0df0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:19:28.337 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:28.332+0000 7f0939668640 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7f09340d1bc0 0x7f09340b0750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:19:28.337 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:28.332+0000 7f0939668640 1 -- --> v1:172.21.15.188:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f09340cf0e0 con 0x7f09340b2160 2023-11-26T07:19:28.337 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:28.332+0000 7f0939668640 1 -- --> v2:172.21.15.188:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f093415dac0 con 0x7f09340d1bc0 2023-11-26T07:19:28.337 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:28.332+0000 7f0939668640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f09340ceb20 con 0x7f093415d6e0 2023-11-26T07:19:28.337 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:28.336+0000 7f09337fe640 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7f09340d1bc0 0x7f09340b0750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:19:28.337 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:28.336+0000 7f0932ffd640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f093415d6e0 0x7f09340d0df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:19:28.337 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:28.336+0000 7f0932ffd640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f093415d6e0 0x7f09340d0df0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.188:3301/0 says I am v2:172.21.15.143:57134/0 (socket says 172.21.15.143:57134) 2023-11-26T07:19:28.338 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:28.336+0000 7f0932ffd640 1 -- 172.21.15.143:0/754419572 learned_addr learned my addr 172.21.15.143:0/754419572 (peer_addr_for_me v2:172.21.15.143:0/0) 2023-11-26T07:19:28.338 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:28.336+0000 7f092a7fc640 1 --1- 172.21.15.143:0/754419572 >> v1:172.21.15.188:6789/0 conn(0x7f09340b2160 0x7f09340d14b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.188:6789/0 says I am v1:172.21.15.143:42316/0 (socket says 172.21.15.143:42316) 2023-11-26T07:19:28.338 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:28.336+0000 7f0932ffd640 1 -- 172.21.15.143:0/754419572 >> v2:172.21.15.188:3300/0 conn(0x7f09340d1bc0 msgr2=0x7f09340b0750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-26T07:19:28.338 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:28.336+0000 7f0932ffd640 1 --2- 172.21.15.143:0/754419572 >> v2:172.21.15.188:3300/0 conn(0x7f09340d1bc0 0x7f09340b0750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:19:28.338 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:28.336+0000 7f0932ffd640 1 -- 172.21.15.143:0/754419572 >> v1:172.21.15.188:6789/0 conn(0x7f09340b2160 legacy=0x7f09340d14b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:19:28.338 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:28.336+0000 7f0932ffd640 1 -- 172.21.15.143:0/754419572 --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f09340cee00 con 0x7f093415d6e0 2023-11-26T07:19:28.339 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:28.336+0000 7f09337fe640 1 --2- 172.21.15.143:0/754419572 >> v2:172.21.15.188:3300/0 conn(0x7f09340d1bc0 0x7f09340b0750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-26T07:19:28.340 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:28.336+0000 7f0932ffd640 1 --2- 172.21.15.143:0/754419572 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f093415d6e0 0x7f09340d0df0 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7f0924009e50 tx=0x7f092401ca10 comp rx=0 tx=0).ready entity=mon.2 client_cookie=fd1ec8336b5c0f29 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:19:28.341 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:28.340+0000 7f09327fc640 1 -- 172.21.15.143:0/754419572 <== mon.2 v2:172.21.15.188:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f09240a57d0 con 0x7f093415d6e0 2023-11-26T07:19:28.341 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:28.340+0000 7f09327fc640 1 -- 172.21.15.143:0/754419572 <== mon.2 v2:172.21.15.188:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f09240a5930 con 0x7f093415d6e0 2023-11-26T07:19:28.341 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:28.340+0000 7f09327fc640 1 -- 172.21.15.143:0/754419572 <== mon.2 v2:172.21.15.188:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f09240a5cc0 con 0x7f093415d6e0 2023-11-26T07:19:28.342 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:28.340+0000 7f0939668640 1 -- 172.21.15.143:0/754419572 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f093415d6e0 msgr2=0x7f09340d0df0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:19:28.342 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:28.340+0000 7f0939668640 1 --2- 172.21.15.143:0/754419572 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f093415d6e0 0x7f09340d0df0 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7f0924009e50 tx=0x7f092401ca10 comp rx=0 tx=0).stop 2023-11-26T07:19:28.342 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:28.340+0000 7f0939668640 1 -- 172.21.15.143:0/754419572 shutdown_connections 2023-11-26T07:19:28.342 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:28.340+0000 7f0939668640 1 --2- 172.21.15.143:0/754419572 >> v2:172.21.15.188:3300/0 conn(0x7f09340d1bc0 0x7f09340b0750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:19:28.342 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:28.340+0000 7f0939668640 1 --2- 172.21.15.143:0/754419572 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f093415d6e0 0x7f09340d0df0 unknown :-1 s=CLOSED pgs=84 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:19:28.342 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:28.340+0000 7f0939668640 1 -- 172.21.15.143:0/754419572 >> 172.21.15.143:0/754419572 conn(0x7f093409c550 msgr2=0x7f09340d3220 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:19:28.342 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:28.340+0000 7f0939668640 1 -- 172.21.15.143:0/754419572 shutdown_connections 2023-11-26T07:19:28.342 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:28.340+0000 7f0939668640 1 -- 172.21.15.143:0/754419572 wait complete. 2023-11-26T07:19:28.343 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:28.340+0000 7f0939668640 1 Processor -- start 2023-11-26T07:19:28.343 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:28.340+0000 7f0939668640 1 -- start start 2023-11-26T07:19:28.345 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:28.344+0000 7f0939668640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f09340b2160 0x7f09340c9460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:19:28.346 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:28.344+0000 7f0939668640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f09340d1bc0 0x7f09340cc740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:19:28.346 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:28.344+0000 7f0939668640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f093415d6e0 0x7f093415eeb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:19:28.346 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:28.344+0000 7f0939668640 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f09340cb6a0 con 0x7f09340d1bc0 2023-11-26T07:19:28.346 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:28.344+0000 7f0939668640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f09340cb980 con 0x7f093415d6e0 2023-11-26T07:19:28.346 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:28.344+0000 7f0939668640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f09340cbc60 con 0x7f09340b2160 2023-11-26T07:19:28.346 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:28.344+0000 7f092a7fc640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f09340d1bc0 0x7f09340cc740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:19:28.346 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:28.344+0000 7f092a7fc640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f09340d1bc0 0x7f09340cc740 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:40592/0 (socket says 172.21.15.143:40592) 2023-11-26T07:19:28.346 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:28.344+0000 7f092a7fc640 1 -- 172.21.15.143:0/4280520017 learned_addr learned my addr 172.21.15.143:0/4280520017 (peer_addr_for_me v2:172.21.15.143:0/0) 2023-11-26T07:19:28.346 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:28.344+0000 7f0932ffd640 1 --2- 172.21.15.143:0/4280520017 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f09340b2160 0x7f09340c9460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:19:28.346 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:28.344+0000 7f09337fe640 1 --2- 172.21.15.143:0/4280520017 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f093415d6e0 0x7f093415eeb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:19:28.346 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:28.344+0000 7f092a7fc640 1 -- 172.21.15.143:0/4280520017 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f09340b2160 msgr2=0x7f09340c9460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:19:28.346 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:28.344+0000 7f092a7fc640 1 --2- 172.21.15.143:0/4280520017 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f09340b2160 0x7f09340c9460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:19:28.346 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:28.344+0000 7f092a7fc640 1 -- 172.21.15.143:0/4280520017 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f093415d6e0 msgr2=0x7f093415eeb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:19:28.347 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:28.344+0000 7f092a7fc640 1 --2- 172.21.15.143:0/4280520017 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f093415d6e0 0x7f093415eeb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:19:28.347 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:28.344+0000 7f092a7fc640 1 -- 172.21.15.143:0/4280520017 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f092c009420 con 0x7f09340d1bc0 2023-11-26T07:19:28.347 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:28.344+0000 7f092a7fc640 1 --2- 172.21.15.143:0/4280520017 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f09340d1bc0 0x7f09340cc740 secure :-1 s=READY pgs=549 cs=0 l=1 rev1=1 crypto rx=0x7f091c00c490 tx=0x7f091c00c890 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:19:28.347 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:28.344+0000 7f092bfff640 1 -- 172.21.15.143:0/4280520017 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f091c0074e0 con 0x7f09340d1bc0 2023-11-26T07:19:28.347 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:28.344+0000 7f092bfff640 1 -- 172.21.15.143:0/4280520017 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f091c007640 con 0x7f09340d1bc0 2023-11-26T07:19:28.348 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:28.344+0000 7f0939668640 1 -- 172.21.15.143:0/4280520017 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0924009b30 con 0x7f09340d1bc0 2023-11-26T07:19:28.348 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:28.344+0000 7f092bfff640 1 -- 172.21.15.143:0/4280520017 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f091c004a10 con 0x7f09340d1bc0 2023-11-26T07:19:28.348 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:28.344+0000 7f0939668640 1 -- 172.21.15.143:0/4280520017 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f09340cc220 con 0x7f09340d1bc0 2023-11-26T07:19:28.348 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:28.344+0000 7f0939668640 1 -- 172.21.15.143:0/4280520017 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0934067680 con 0x7f09340d1bc0 2023-11-26T07:19:28.348 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:28.344+0000 7f092bfff640 1 -- 172.21.15.143:0/4280520017 <== mon.0 v2:172.21.15.143:3300/0 4 ==== mgrmap(e 7) v1 ==== 86005+0+0 (secure 0 0 0) 0x7f091c004b70 con 0x7f09340d1bc0 2023-11-26T07:19:28.349 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:28.348+0000 7f092bfff640 1 --2- 172.21.15.143:0/4280520017 >> [v2:172.21.15.143:6832/2150557544,v1:172.21.15.143:6833/2150557544] conn(0x7f0914066620 0x7f0914068ae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:19:28.349 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:28.348+0000 7f092bfff640 1 -- 172.21.15.143:0/4280520017 <== mon.0 v2:172.21.15.143:3300/0 5 ==== osd_map(41..41 src has 1..41) v4 ==== 4857+0+0 (secure 0 0 0) 0x7f091c083800 con 0x7f09340d1bc0 2023-11-26T07:19:28.349 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:28.348+0000 7f0932ffd640 1 --2- 172.21.15.143:0/4280520017 >> [v2:172.21.15.143:6832/2150557544,v1:172.21.15.143:6833/2150557544] conn(0x7f0914066620 0x7f0914068ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:19:28.349 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:28.348+0000 7f0932ffd640 1 --2- 172.21.15.143:0/4280520017 >> [v2:172.21.15.143:6832/2150557544,v1:172.21.15.143:6833/2150557544] conn(0x7f0914066620 0x7f0914068ae0 secure :-1 s=READY pgs=279 cs=0 l=1 rev1=1 crypto rx=0x7f09240ae000 tx=0x7f09240a7070 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:19:28.353 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:28.352+0000 7f092bfff640 1 -- 172.21.15.143:0/4280520017 <== mon.0 v2:172.21.15.143:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172649 (secure 0 0 0) 0x7f091c050cc0 con 0x7f09340d1bc0 2023-11-26T07:19:28.503 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:28.500+0000 7f0939668640 1 -- 172.21.15.143:0/4280520017 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f09340cc3e0 con 0x7f09340d1bc0 2023-11-26T07:19:28.504 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:28.500+0000 7f092bfff640 1 -- 172.21.15.143:0/4280520017 <== mon.0 v2:172.21.15.143:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v41) v1 ==== 93+0+10 (secure 0 0 0) 0x7f091c0542e0 con 0x7f09340d1bc0 2023-11-26T07:19:28.504 INFO:teuthology.orchestra.run.smithi143.stdout:pg_num: 1 2023-11-26T07:19:28.506 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:28.504+0000 7f0939668640 1 -- 172.21.15.143:0/4280520017 >> [v2:172.21.15.143:6832/2150557544,v1:172.21.15.143:6833/2150557544] conn(0x7f0914066620 msgr2=0x7f0914068ae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:19:28.506 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:28.504+0000 7f0939668640 1 --2- 172.21.15.143:0/4280520017 >> [v2:172.21.15.143:6832/2150557544,v1:172.21.15.143:6833/2150557544] conn(0x7f0914066620 0x7f0914068ae0 secure :-1 s=READY pgs=279 cs=0 l=1 rev1=1 crypto rx=0x7f09240ae000 tx=0x7f09240a7070 comp rx=0 tx=0).stop 2023-11-26T07:19:28.506 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:28.504+0000 7f0939668640 1 -- 172.21.15.143:0/4280520017 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f09340d1bc0 msgr2=0x7f09340cc740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:19:28.507 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:28.504+0000 7f0939668640 1 --2- 172.21.15.143:0/4280520017 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f09340d1bc0 0x7f09340cc740 secure :-1 s=READY pgs=549 cs=0 l=1 rev1=1 crypto rx=0x7f091c00c490 tx=0x7f091c00c890 comp rx=0 tx=0).stop 2023-11-26T07:19:28.507 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:28.504+0000 7f0939668640 1 -- 172.21.15.143:0/4280520017 shutdown_connections 2023-11-26T07:19:28.507 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:28.504+0000 7f0939668640 1 --2- 172.21.15.143:0/4280520017 >> [v2:172.21.15.143:6832/2150557544,v1:172.21.15.143:6833/2150557544] conn(0x7f0914066620 0x7f0914068ae0 unknown :-1 s=CLOSED pgs=279 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:19:28.507 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:28.504+0000 7f0939668640 1 --2- 172.21.15.143:0/4280520017 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f093415d6e0 0x7f093415eeb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:19:28.507 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:28.504+0000 7f0939668640 1 --2- 172.21.15.143:0/4280520017 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f09340d1bc0 0x7f09340cc740 unknown :-1 s=CLOSED pgs=549 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:19:28.507 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:28.504+0000 7f0939668640 1 --2- 172.21.15.143:0/4280520017 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f09340b2160 0x7f09340c9460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:19:28.507 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:28.504+0000 7f0939668640 1 -- 172.21.15.143:0/4280520017 >> 172.21.15.143:0/4280520017 conn(0x7f093409c550 msgr2=0x7f09340cf490 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:19:28.507 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:28.504+0000 7f0939668640 1 -- 172.21.15.143:0/4280520017 shutdown_connections 2023-11-26T07:19:28.507 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:28.504+0000 7f0939668640 1 -- 172.21.15.143:0/4280520017 wait complete. 2023-11-26T07:19:28.525 DEBUG:teuthology.orchestra.run.smithi143:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --format=json-pretty osd lspools 2023-11-26T07:19:28.662 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:28.660+0000 7f21147df640 1 Processor -- start 2023-11-26T07:19:28.662 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:28.660+0000 7f21147df640 1 -- start start 2023-11-26T07:19:28.663 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:28.660+0000 7f21147df640 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7f210c15d6e0 0x7f210c0d0df0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:19:28.664 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:28.660+0000 7f21147df640 1 -- --> v1:172.21.15.143:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f210c0cf0e0 con 0x7f210c0b2160 2023-11-26T07:19:28.664 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:28.660+0000 7f21147df640 1 -- --> v1:172.21.15.188:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f210c0ceb20 con 0x7f210c0d1bc0 2023-11-26T07:19:28.664 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:28.660+0000 7f21147df640 1 -- --> v2:172.21.15.188:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f210c15dac0 con 0x7f210c15d6e0 2023-11-26T07:19:28.664 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:28.660+0000 7f2111d7a640 1 --1- >> v1:172.21.15.143:6789/0 conn(0x7f210c0b2160 0x7f210c0d14b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.143:6789/0 says I am v1:172.21.15.143:58352/0 (socket says 172.21.15.143:58352) 2023-11-26T07:19:28.664 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:28.660+0000 7f2111d7a640 1 -- 172.21.15.143:0/3911376755 learned_addr learned my addr 172.21.15.143:0/3911376755 (peer_addr_for_me v1:172.21.15.143:0/0) 2023-11-26T07:19:28.666 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:28.660+0000 7f211257b640 1 --2- 172.21.15.143:0/3911376755 >> v2:172.21.15.188:3300/0 conn(0x7f210c15d6e0 0x7f210c0d0df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:19:28.666 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:28.660+0000 7f2111579640 1 -- 172.21.15.143:0/3911376755 <== mon.0 v1:172.21.15.143:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1936712247 0 0) 0x7f210c0cf0e0 con 0x7f210c0b2160 2023-11-26T07:19:28.666 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:28.660+0000 7f2111579640 1 -- 172.21.15.143:0/3911376755 --> v1:172.21.15.143:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f20f8003530 con 0x7f210c0b2160 2023-11-26T07:19:28.666 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:28.664+0000 7f2111579640 1 -- 172.21.15.143:0/3911376755 <== mon.0 v1:172.21.15.143:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 376522462 0 0) 0x7f20f8003530 con 0x7f210c0b2160 2023-11-26T07:19:28.666 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:28.664+0000 7f2111579640 1 -- 172.21.15.143:0/3911376755 >> v2:172.21.15.188:3300/0 conn(0x7f210c15d6e0 msgr2=0x7f210c0d0df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-26T07:19:28.666 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:28.664+0000 7f2111579640 1 --2- 172.21.15.143:0/3911376755 >> v2:172.21.15.188:3300/0 conn(0x7f210c15d6e0 0x7f210c0d0df0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:19:28.666 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:28.664+0000 7f2111579640 1 -- 172.21.15.143:0/3911376755 >> v1:172.21.15.188:6789/0 conn(0x7f210c0d1bc0 legacy=0x7f210c0b0750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:19:28.666 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:28.664+0000 7f2111579640 1 -- 172.21.15.143:0/3911376755 --> v1:172.21.15.143:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f210c0cee00 con 0x7f210c0b2160 2023-11-26T07:19:28.667 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:28.664+0000 7f2111579640 1 -- 172.21.15.143:0/3911376755 <== mon.0 v1:172.21.15.143:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2137882523 0 0) 0x7f20fc001680 con 0x7f210c0b2160 2023-11-26T07:19:28.667 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:28.664+0000 7f211257b640 1 --2- 172.21.15.143:0/3911376755 >> v2:172.21.15.188:3300/0 conn(0x7f210c15d6e0 0x7f210c0d0df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-26T07:19:28.667 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:28.664+0000 7f2111579640 1 -- 172.21.15.143:0/3911376755 >> v1:172.21.15.143:6789/0 conn(0x7f210c0b2160 legacy=0x7f210c0d14b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:19:28.667 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:28.664+0000 7f2111579640 1 --2- 172.21.15.143:0/3911376755 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f20f8003980 0x7f20f8003d60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:19:28.667 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:28.664+0000 7f2111579640 1 --2- 172.21.15.143:0/3911376755 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f20f8004380 0x7f20f8004850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:19:28.667 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:28.664+0000 7f2111579640 1 --2- 172.21.15.143:0/3911376755 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f20f8005050 0x7f20f800b990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:19:28.667 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:28.664+0000 7f2111579640 1 -- 172.21.15.143:0/3911376755 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f210c15dac0 con 0x7f20f8003980 2023-11-26T07:19:28.667 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:28.664+0000 7f2111579640 1 -- 172.21.15.143:0/3911376755 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f20f800bfb0 con 0x7f20f8004380 2023-11-26T07:19:28.667 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:28.664+0000 7f2111579640 1 -- 172.21.15.143:0/3911376755 --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f20f800c170 con 0x7f20f8005050 2023-11-26T07:19:28.667 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:28.664+0000 7f211257b640 1 --2- 172.21.15.143:0/3911376755 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f20f8003980 0x7f20f8003d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:19:28.667 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:28.664+0000 7f2111d7a640 1 --2- 172.21.15.143:0/3911376755 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f20f8004380 0x7f20f8004850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:19:28.667 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:28.664+0000 7f2112d7c640 1 --2- 172.21.15.143:0/3911376755 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f20f8005050 0x7f20f800b990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:19:28.667 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:28.664+0000 7f211257b640 1 -- 172.21.15.143:0/3911376755 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f20f8005050 msgr2=0x7f20f800b990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-26T07:19:28.667 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:28.664+0000 7f211257b640 1 --2- 172.21.15.143:0/3911376755 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f20f8005050 0x7f20f800b990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:19:28.668 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:28.664+0000 7f211257b640 1 -- 172.21.15.143:0/3911376755 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f20f8004380 msgr2=0x7f20f8004850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-26T07:19:28.668 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:28.664+0000 7f211257b640 1 --2- 172.21.15.143:0/3911376755 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f20f8004380 0x7f20f8004850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:19:28.668 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:28.664+0000 7f211257b640 1 -- 172.21.15.143:0/3911376755 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f20f8003530 con 0x7f20f8003980 2023-11-26T07:19:28.668 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:28.664+0000 7f2112d7c640 1 -- 172.21.15.143:0/3911376755 reap_dead start 2023-11-26T07:19:28.668 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:28.664+0000 7f211257b640 1 --2- 172.21.15.143:0/3911376755 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f20f8003980 0x7f20f8003d60 secure :-1 s=READY pgs=551 cs=0 l=1 rev1=1 crypto rx=0x7f2108009cb0 tx=0x7f210801cc10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=71fb59819176e118 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:19:28.668 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:28.664+0000 7f2111579640 1 -- 172.21.15.143:0/3911376755 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f21080a58d0 con 0x7f20f8003980 2023-11-26T07:19:28.669 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:28.664+0000 7f2111579640 1 -- 172.21.15.143:0/3911376755 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f21080a5a30 con 0x7f20f8003980 2023-11-26T07:19:28.669 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:28.668+0000 7f2111579640 1 -- 172.21.15.143:0/3911376755 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f21080a5e20 con 0x7f20f8003980 2023-11-26T07:19:28.669 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:28.668+0000 7f21147df640 1 -- 172.21.15.143:0/3911376755 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f20f8003980 msgr2=0x7f20f8003d60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:19:28.669 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:28.668+0000 7f21147df640 1 --2- 172.21.15.143:0/3911376755 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f20f8003980 0x7f20f8003d60 secure :-1 s=READY pgs=551 cs=0 l=1 rev1=1 crypto rx=0x7f2108009cb0 tx=0x7f210801cc10 comp rx=0 tx=0).stop 2023-11-26T07:19:28.670 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:28.668+0000 7f21147df640 1 -- 172.21.15.143:0/3911376755 shutdown_connections 2023-11-26T07:19:28.670 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:28.668+0000 7f21147df640 1 --2- 172.21.15.143:0/3911376755 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f20f8003980 0x7f20f8003d60 unknown :-1 s=CLOSED pgs=551 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:19:28.670 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:28.668+0000 7f21147df640 1 -- 172.21.15.143:0/3911376755 >> 172.21.15.143:0/3911376755 conn(0x7f210c09c550 msgr2=0x7f210c0d3220 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:19:28.670 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:28.668+0000 7f21147df640 1 -- 172.21.15.143:0/3911376755 shutdown_connections 2023-11-26T07:19:28.670 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:28.668+0000 7f21147df640 1 -- 172.21.15.143:0/3911376755 wait complete. 2023-11-26T07:19:28.671 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:28.668+0000 7f21147df640 1 Processor -- start 2023-11-26T07:19:28.671 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:28.668+0000 7f21147df640 1 -- start start 2023-11-26T07:19:28.673 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:28.668+0000 7f21147df640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f20f8003980 0x7f210c1f5a90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:19:28.673 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:28.668+0000 7f21147df640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f210c0b43c0 0x7f210c0b4840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:19:28.673 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:28.668+0000 7f21147df640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f210c0b0470 0x7f210c0b08f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:19:28.673 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:28.668+0000 7f21147df640 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f210c0b0e30 con 0x7f20f8003980 2023-11-26T07:19:28.673 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:28.668+0000 7f21147df640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f210c0b4f10 con 0x7f210c0b43c0 2023-11-26T07:19:28.673 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:28.668+0000 7f21147df640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f210c0b51f0 con 0x7f210c0b0470 2023-11-26T07:19:28.673 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:28.668+0000 7f211257b640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f20f8003980 0x7f210c1f5a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:19:28.673 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:28.668+0000 7f211257b640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f20f8003980 0x7f210c1f5a90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:40604/0 (socket says 172.21.15.143:40604) 2023-11-26T07:19:28.673 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:28.668+0000 7f211257b640 1 -- 172.21.15.143:0/1881796967 learned_addr learned my addr 172.21.15.143:0/1881796967 (peer_addr_for_me v2:172.21.15.143:0/0) 2023-11-26T07:19:28.673 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:28.668+0000 7f2111d7a640 1 --2- 172.21.15.143:0/1881796967 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f210c0b43c0 0x7f210c0b4840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:19:28.673 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:28.668+0000 7f2112d7c640 1 --2- 172.21.15.143:0/1881796967 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f210c0b0470 0x7f210c0b08f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:19:28.673 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:28.672+0000 7f211257b640 1 -- 172.21.15.143:0/1881796967 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f210c0b0470 msgr2=0x7f210c0b08f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:19:28.674 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:28.672+0000 7f211257b640 1 --2- 172.21.15.143:0/1881796967 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f210c0b0470 0x7f210c0b08f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:19:28.674 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:28.672+0000 7f211257b640 1 -- 172.21.15.143:0/1881796967 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f210c0b43c0 msgr2=0x7f210c0b4840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:19:28.674 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:28.672+0000 7f211257b640 1 --2- 172.21.15.143:0/1881796967 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f210c0b43c0 0x7f210c0b4840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:19:28.674 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:28.672+0000 7f211257b640 1 -- 172.21.15.143:0/1881796967 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f210c0b54d0 con 0x7f20f8003980 2023-11-26T07:19:28.674 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:28.672+0000 7f2111d7a640 1 --2- 172.21.15.143:0/1881796967 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f210c0b43c0 0x7f210c0b4840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-26T07:19:28.675 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:28.672+0000 7f211257b640 1 --2- 172.21.15.143:0/1881796967 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f20f8003980 0x7f210c1f5a90 secure :-1 s=READY pgs=552 cs=0 l=1 rev1=1 crypto rx=0x7f210801ca40 tx=0x7f21080199b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:19:28.675 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:28.672+0000 7f2102ffd640 1 -- 172.21.15.143:0/1881796967 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f21080b8920 con 0x7f20f8003980 2023-11-26T07:19:28.675 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:28.672+0000 7f2102ffd640 1 -- 172.21.15.143:0/1881796967 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f21080b8a80 con 0x7f20f8003980 2023-11-26T07:19:28.675 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:28.672+0000 7f21147df640 1 -- 172.21.15.143:0/1881796967 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f210c0b6a00 con 0x7f20f8003980 2023-11-26T07:19:28.675 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:28.672+0000 7f2102ffd640 1 -- 172.21.15.143:0/1881796967 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f21080b8e10 con 0x7f20f8003980 2023-11-26T07:19:28.675 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:28.672+0000 7f21147df640 1 -- 172.21.15.143:0/1881796967 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f210c1f5fd0 con 0x7f20f8003980 2023-11-26T07:19:28.675 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:28.672+0000 7f21147df640 1 -- 172.21.15.143:0/1881796967 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f210c0b6f20 con 0x7f20f8003980 2023-11-26T07:19:28.675 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:28.672+0000 7f2102ffd640 1 -- 172.21.15.143:0/1881796967 <== mon.0 v2:172.21.15.143:3300/0 4 ==== mgrmap(e 7) v1 ==== 86005+0+0 (secure 0 0 0) 0x7f21080b4070 con 0x7f20f8003980 2023-11-26T07:19:28.676 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:28.672+0000 7f2102ffd640 1 --2- 172.21.15.143:0/1881796967 >> [v2:172.21.15.143:6832/2150557544,v1:172.21.15.143:6833/2150557544] conn(0x7f20f0066620 0x7f20f0068ae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:19:28.676 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:28.672+0000 7f2102ffd640 1 -- 172.21.15.143:0/1881796967 <== mon.0 v2:172.21.15.143:3300/0 5 ==== osd_map(41..41 src has 1..41) v4 ==== 4857+0+0 (secure 0 0 0) 0x7f210811ebb0 con 0x7f20f8003980 2023-11-26T07:19:28.676 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:28.672+0000 7f2111d7a640 1 --2- 172.21.15.143:0/1881796967 >> [v2:172.21.15.143:6832/2150557544,v1:172.21.15.143:6833/2150557544] conn(0x7f20f0066620 0x7f20f0068ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:19:28.677 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:28.676+0000 7f2111d7a640 1 --2- 172.21.15.143:0/1881796967 >> [v2:172.21.15.143:6832/2150557544,v1:172.21.15.143:6833/2150557544] conn(0x7f20f0066620 0x7f20f0068ae0 secure :-1 s=READY pgs=280 cs=0 l=1 rev1=1 crypto rx=0x7f210c151720 tx=0x7f20fc009040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:19:28.680 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:28.676+0000 7f2102ffd640 1 -- 172.21.15.143:0/1881796967 <== mon.0 v2:172.21.15.143:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172649 (secure 0 0 0) 0x7f210c0b6f20 con 0x7f20f8003980 2023-11-26T07:19:28.833 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:28.828+0000 7f21147df640 1 -- 172.21.15.143:0/1881796967 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "osd lspools", "format": "json-pretty"} v 0) v1 -- 0x7f210c0b6f20 con 0x7f20f8003980 2023-11-26T07:19:28.833 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:28.832+0000 7f2102ffd640 1 -- 172.21.15.143:0/1881796967 <== mon.0 v2:172.21.15.143:3300/0 7 ==== mon_command_ack([{"prefix": "osd lspools", "format": "json-pretty"}]=0 v41) v1 ==== 84+0+65 (secure 0 0 0) 0x7f210c0b6f20 con 0x7f20f8003980 2023-11-26T07:19:28.833 INFO:teuthology.orchestra.run.smithi143.stdout: 2023-11-26T07:19:28.834 INFO:teuthology.orchestra.run.smithi143.stdout:[ 2023-11-26T07:19:28.834 INFO:teuthology.orchestra.run.smithi143.stdout: { 2023-11-26T07:19:28.834 INFO:teuthology.orchestra.run.smithi143.stdout: "poolnum": 1, 2023-11-26T07:19:28.834 INFO:teuthology.orchestra.run.smithi143.stdout: "poolname": ".mgr" 2023-11-26T07:19:28.834 INFO:teuthology.orchestra.run.smithi143.stdout: } 2023-11-26T07:19:28.834 INFO:teuthology.orchestra.run.smithi143.stdout:] 2023-11-26T07:19:28.835 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:28.832+0000 7f21147df640 1 -- 172.21.15.143:0/1881796967 >> [v2:172.21.15.143:6832/2150557544,v1:172.21.15.143:6833/2150557544] conn(0x7f20f0066620 msgr2=0x7f20f0068ae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:19:28.835 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:28.832+0000 7f21147df640 1 --2- 172.21.15.143:0/1881796967 >> [v2:172.21.15.143:6832/2150557544,v1:172.21.15.143:6833/2150557544] conn(0x7f20f0066620 0x7f20f0068ae0 secure :-1 s=READY pgs=280 cs=0 l=1 rev1=1 crypto rx=0x7f210c151720 tx=0x7f20fc009040 comp rx=0 tx=0).stop 2023-11-26T07:19:28.836 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:28.832+0000 7f21147df640 1 -- 172.21.15.143:0/1881796967 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f20f8003980 msgr2=0x7f210c1f5a90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:19:28.836 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:28.832+0000 7f21147df640 1 --2- 172.21.15.143:0/1881796967 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f20f8003980 0x7f210c1f5a90 secure :-1 s=READY pgs=552 cs=0 l=1 rev1=1 crypto rx=0x7f210801ca40 tx=0x7f21080199b0 comp rx=0 tx=0).stop 2023-11-26T07:19:28.836 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:28.832+0000 7f21147df640 1 -- 172.21.15.143:0/1881796967 shutdown_connections 2023-11-26T07:19:28.836 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:28.832+0000 7f21147df640 1 --2- 172.21.15.143:0/1881796967 >> [v2:172.21.15.143:6832/2150557544,v1:172.21.15.143:6833/2150557544] conn(0x7f20f0066620 0x7f20f0068ae0 unknown :-1 s=CLOSED pgs=280 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:19:28.836 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:28.832+0000 7f21147df640 1 --2- 172.21.15.143:0/1881796967 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f210c0b0470 0x7f210c0b08f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:19:28.836 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:28.832+0000 7f21147df640 1 --2- 172.21.15.143:0/1881796967 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f210c0b43c0 0x7f210c0b4840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:19:28.836 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:28.832+0000 7f21147df640 1 --2- 172.21.15.143:0/1881796967 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f20f8003980 0x7f210c1f5a90 unknown :-1 s=CLOSED pgs=552 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:19:28.837 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:28.832+0000 7f21147df640 1 -- 172.21.15.143:0/1881796967 >> 172.21.15.143:0/1881796967 conn(0x7f210c09c550 msgr2=0x7f210c15e020 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:19:28.837 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:28.832+0000 7f21147df640 1 -- 172.21.15.143:0/1881796967 shutdown_connections 2023-11-26T07:19:28.837 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:28.832+0000 7f21147df640 1 -- 172.21.15.143:0/1881796967 wait complete. 2023-11-26T07:19:28.854 DEBUG:tasks.cephfs.filesystem:Creating filesystem 'cephfs' 2023-11-26T07:19:28.854 DEBUG:teuthology.orchestra.run.smithi143:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_metadata --pg_num_min 64 2023-11-26T07:19:28.996 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:28.992+0000 7f7bc3a13640 1 Processor -- start 2023-11-26T07:19:28.996 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:28.992+0000 7f7bc3a13640 1 -- start start 2023-11-26T07:19:28.998 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:28.996+0000 7f7bc3a13640 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7f7bbc1596d0 0x7f7bbc071850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:19:28.999 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:28.996+0000 7f7bc3a13640 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7f7bbc0ce680 0x7f7bbc0b2870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:19:28.999 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:28.996+0000 7f7bc3a13640 1 -- --> v1:172.21.15.143:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7bbc0b3200 con 0x7f7bbc0b2db0 2023-11-26T07:19:28.999 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:28.996+0000 7f7bc3a13640 1 -- --> v2:172.21.15.143:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7bbc0cbe00 con 0x7f7bbc1596d0 2023-11-26T07:19:28.999 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:28.996+0000 7f7bc3a13640 1 -- --> v2:172.21.15.188:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7bbc06c750 con 0x7f7bbc0ce680 2023-11-26T07:19:28.999 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:28.996+0000 7f7bc17af640 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7f7bbc1596d0 0x7f7bbc071850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:19:28.999 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:28.996+0000 7f7bc17af640 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7f7bbc1596d0 0x7f7bbc071850 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:40606/0 (socket says 172.21.15.143:40606) 2023-11-26T07:19:28.999 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:28.996+0000 7f7bc17af640 1 -- 172.21.15.143:0/4217183905 learned_addr learned my addr 172.21.15.143:0/4217183905 (peer_addr_for_me v2:172.21.15.143:0/0) 2023-11-26T07:19:28.999 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:28.996+0000 7f7bc1fb0640 1 --1- >> v1:172.21.15.143:6789/0 conn(0x7f7bbc0b2db0 0x7f7bbc15d2c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.143:6789/0 says I am v1:172.21.15.143:58364/0 (socket says 172.21.15.143:58364) 2023-11-26T07:19:28.999 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:28.996+0000 7f7bc0fae640 1 --2- 172.21.15.143:0/4217183905 >> v2:172.21.15.188:3300/0 conn(0x7f7bbc0ce680 0x7f7bbc0b2870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:19:28.999 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:28.996+0000 7f7bc17af640 1 -- 172.21.15.143:0/4217183905 >> v2:172.21.15.188:3300/0 conn(0x7f7bbc0ce680 msgr2=0x7f7bbc0b2870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-26T07:19:28.999 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:28.996+0000 7f7bc17af640 1 --2- 172.21.15.143:0/4217183905 >> v2:172.21.15.188:3300/0 conn(0x7f7bbc0ce680 0x7f7bbc0b2870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:19:28.999 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:28.996+0000 7f7bc17af640 1 -- 172.21.15.143:0/4217183905 >> v1:172.21.15.143:6789/0 conn(0x7f7bbc0b2db0 legacy=0x7f7bbc15d2c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:19:28.999 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:28.996+0000 7f7bc17af640 1 -- 172.21.15.143:0/4217183905 --> v2:172.21.15.143:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7bbc0b3550 con 0x7f7bbc1596d0 2023-11-26T07:19:29.001 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:29.000+0000 7f7bc17af640 1 --2- 172.21.15.143:0/4217183905 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f7bbc1596d0 0x7f7bbc071850 secure :-1 s=READY pgs=554 cs=0 l=1 rev1=1 crypto rx=0x7f7bbc0cbf40 tx=0x7f7bb801d8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=80e145b687ccd9ff server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:19:29.002 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:29.000+0000 7f7bb3fff640 1 -- 172.21.15.143:0/4217183905 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7bb80a67f0 con 0x7f7bbc1596d0 2023-11-26T07:19:29.002 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:29.000+0000 7f7bb3fff640 1 -- 172.21.15.143:0/4217183905 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7bb80a6950 con 0x7f7bbc1596d0 2023-11-26T07:19:29.002 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:29.000+0000 7f7bb3fff640 1 -- 172.21.15.143:0/4217183905 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7bb80a6ce0 con 0x7f7bbc1596d0 2023-11-26T07:19:29.002 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:29.000+0000 7f7bc3a13640 1 -- 172.21.15.143:0/4217183905 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f7bbc1596d0 msgr2=0x7f7bbc071850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:19:29.003 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:29.000+0000 7f7bc3a13640 1 --2- 172.21.15.143:0/4217183905 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f7bbc1596d0 0x7f7bbc071850 secure :-1 s=READY pgs=554 cs=0 l=1 rev1=1 crypto rx=0x7f7bbc0cbf40 tx=0x7f7bb801d8d0 comp rx=0 tx=0).stop 2023-11-26T07:19:29.003 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:29.000+0000 7f7bc3a13640 1 -- 172.21.15.143:0/4217183905 shutdown_connections 2023-11-26T07:19:29.003 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:29.000+0000 7f7bc3a13640 1 --2- 172.21.15.143:0/4217183905 >> v2:172.21.15.188:3300/0 conn(0x7f7bbc0ce680 0x7f7bbc0b2870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:19:29.003 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:29.000+0000 7f7bc3a13640 1 --2- 172.21.15.143:0/4217183905 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f7bbc1596d0 0x7f7bbc071850 unknown :-1 s=CLOSED pgs=554 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:19:29.003 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:29.000+0000 7f7bc3a13640 1 -- 172.21.15.143:0/4217183905 >> 172.21.15.143:0/4217183905 conn(0x7f7bbc06a890 msgr2=0x7f7bbc06aca0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:19:29.003 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:29.000+0000 7f7bc3a13640 1 -- 172.21.15.143:0/4217183905 shutdown_connections 2023-11-26T07:19:29.003 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:29.000+0000 7f7bc3a13640 1 -- 172.21.15.143:0/4217183905 wait complete. 2023-11-26T07:19:29.004 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:29.000+0000 7f7bc3a13640 1 Processor -- start 2023-11-26T07:19:29.004 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:29.000+0000 7f7bc3a13640 1 -- start start 2023-11-26T07:19:29.006 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:29.004+0000 7f7bc3a13640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f7bbc0b2db0 0x7f7bbc1f5bd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:19:29.007 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:29.004+0000 7f7bc3a13640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f7bbc0ce680 0x7f7bbc1fa1e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:19:29.007 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:29.004+0000 7f7bc3a13640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f7bbc1596d0 0x7f7bbc157a40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:19:29.007 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:29.004+0000 7f7bc3a13640 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7bbc1f6110 con 0x7f7bbc0ce680 2023-11-26T07:19:29.007 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:29.004+0000 7f7bc3a13640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7bbc1575b0 con 0x7f7bbc0b2db0 2023-11-26T07:19:29.007 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:29.004+0000 7f7bc3a13640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7bbc1fa720 con 0x7f7bbc1596d0 2023-11-26T07:19:29.007 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:29.004+0000 7f7bc0fae640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f7bbc0ce680 0x7f7bbc1fa1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:19:29.007 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:29.004+0000 7f7bc0fae640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f7bbc0ce680 0x7f7bbc1fa1e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:40614/0 (socket says 172.21.15.143:40614) 2023-11-26T07:19:29.007 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:29.004+0000 7f7bc0fae640 1 -- 172.21.15.143:0/581564583 learned_addr learned my addr 172.21.15.143:0/581564583 (peer_addr_for_me v2:172.21.15.143:0/0) 2023-11-26T07:19:29.007 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:29.004+0000 7f7bc17af640 1 --2- 172.21.15.143:0/581564583 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f7bbc0b2db0 0x7f7bbc1f5bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:19:29.007 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:29.004+0000 7f7bc1fb0640 1 --2- 172.21.15.143:0/581564583 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f7bbc1596d0 0x7f7bbc157a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:19:29.007 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:29.004+0000 7f7bc0fae640 1 -- 172.21.15.143:0/581564583 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f7bbc1596d0 msgr2=0x7f7bbc157a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:19:29.008 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:29.004+0000 7f7bc0fae640 1 --2- 172.21.15.143:0/581564583 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f7bbc1596d0 0x7f7bbc157a40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:19:29.008 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:29.004+0000 7f7bc0fae640 1 -- 172.21.15.143:0/581564583 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f7bbc0b2db0 msgr2=0x7f7bbc1f5bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:19:29.008 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:29.004+0000 7f7bc0fae640 1 --2- 172.21.15.143:0/581564583 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f7bbc0b2db0 0x7f7bbc1f5bd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:19:29.008 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:29.004+0000 7f7bc0fae640 1 -- 172.21.15.143:0/581564583 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7bb800a670 con 0x7f7bbc0ce680 2023-11-26T07:19:29.008 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:29.004+0000 7f7bc1fb0640 1 --2- 172.21.15.143:0/581564583 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f7bbc1596d0 0x7f7bbc157a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-26T07:19:29.008 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:29.004+0000 7f7bc0fae640 1 --2- 172.21.15.143:0/581564583 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f7bbc0ce680 0x7f7bbc1fa1e0 secure :-1 s=READY pgs=555 cs=0 l=1 rev1=1 crypto rx=0x7f7bac00d800 tx=0x7f7bac00dcc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:19:29.008 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:29.004+0000 7f7bb1ffb640 1 -- 172.21.15.143:0/581564583 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7bac00b6c0 con 0x7f7bbc0ce680 2023-11-26T07:19:29.008 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:29.004+0000 7f7bb1ffb640 1 -- 172.21.15.143:0/581564583 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7bac00b820 con 0x7f7bbc0ce680 2023-11-26T07:19:29.008 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:29.004+0000 7f7bb1ffb640 1 -- 172.21.15.143:0/581564583 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7bac00bb90 con 0x7f7bbc0ce680 2023-11-26T07:19:29.008 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:29.004+0000 7f7bc3a13640 1 -- 172.21.15.143:0/581564583 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7bbc1569a0 con 0x7f7bbc0ce680 2023-11-26T07:19:29.008 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:29.004+0000 7f7bc3a13640 1 -- 172.21.15.143:0/581564583 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7bbc156f60 con 0x7f7bbc0ce680 2023-11-26T07:19:29.009 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:29.004+0000 7f7bc3a13640 1 -- 172.21.15.143:0/581564583 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7bbc157740 con 0x7f7bbc0ce680 2023-11-26T07:19:29.009 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:29.004+0000 7f7bb1ffb640 1 -- 172.21.15.143:0/581564583 <== mon.0 v2:172.21.15.143:3300/0 4 ==== mgrmap(e 7) v1 ==== 86005+0+0 (secure 0 0 0) 0x7f7bac00bcf0 con 0x7f7bbc0ce680 2023-11-26T07:19:29.010 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:29.008+0000 7f7bb1ffb640 1 --2- 172.21.15.143:0/581564583 >> [v2:172.21.15.143:6832/2150557544,v1:172.21.15.143:6833/2150557544] conn(0x7f7ba0066620 0x7f7ba0068ae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:19:29.010 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:29.008+0000 7f7bb1ffb640 1 -- 172.21.15.143:0/581564583 <== mon.0 v2:172.21.15.143:3300/0 5 ==== osd_map(41..41 src has 1..41) v4 ==== 4857+0+0 (secure 0 0 0) 0x7f7bac084370 con 0x7f7bbc0ce680 2023-11-26T07:19:29.010 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:29.008+0000 7f7bc17af640 1 --2- 172.21.15.143:0/581564583 >> [v2:172.21.15.143:6832/2150557544,v1:172.21.15.143:6833/2150557544] conn(0x7f7ba0066620 0x7f7ba0068ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:19:29.010 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:29.008+0000 7f7bc17af640 1 --2- 172.21.15.143:0/581564583 >> [v2:172.21.15.143:6832/2150557544,v1:172.21.15.143:6833/2150557544] conn(0x7f7ba0066620 0x7f7ba0068ae0 secure :-1 s=READY pgs=281 cs=0 l=1 rev1=1 crypto rx=0x7f7bb80b0000 tx=0x7f7bb8008040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:19:29.013 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:29.012+0000 7f7bb1ffb640 1 -- 172.21.15.143:0/581564583 <== mon.0 v2:172.21.15.143:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172649 (secure 0 0 0) 0x7f7bac051820 con 0x7f7bbc0ce680 2023-11-26T07:19:29.165 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:29.164+0000 7f7bc3a13640 1 -- 172.21.15.143:0/581564583 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64} v 0) v1 -- 0x7f7bbc067680 con 0x7f7bbc0ce680 2023-11-26T07:19:30.113 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:30.112+0000 7f7bb1ffb640 1 -- 172.21.15.143:0/581564583 <== mon.0 v2:172.21.15.143:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]=0 pool 'cephfs_metadata' created v42) v1 ==== 138+0+0 (secure 0 0 0) 0x7f7bac054e40 con 0x7f7bbc0ce680 2023-11-26T07:19:30.113 INFO:teuthology.orchestra.run.smithi143.stderr:pool 'cephfs_metadata' created 2023-11-26T07:19:30.117 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:30.112+0000 7f7bc3a13640 1 -- 172.21.15.143:0/581564583 >> [v2:172.21.15.143:6832/2150557544,v1:172.21.15.143:6833/2150557544] conn(0x7f7ba0066620 msgr2=0x7f7ba0068ae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:19:30.117 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:30.112+0000 7f7bc3a13640 1 --2- 172.21.15.143:0/581564583 >> [v2:172.21.15.143:6832/2150557544,v1:172.21.15.143:6833/2150557544] conn(0x7f7ba0066620 0x7f7ba0068ae0 secure :-1 s=READY pgs=281 cs=0 l=1 rev1=1 crypto rx=0x7f7bb80b0000 tx=0x7f7bb8008040 comp rx=0 tx=0).stop 2023-11-26T07:19:30.117 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:30.112+0000 7f7bc3a13640 1 -- 172.21.15.143:0/581564583 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f7bbc0ce680 msgr2=0x7f7bbc1fa1e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:19:30.117 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:30.112+0000 7f7bc3a13640 1 --2- 172.21.15.143:0/581564583 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f7bbc0ce680 0x7f7bbc1fa1e0 secure :-1 s=READY pgs=555 cs=0 l=1 rev1=1 crypto rx=0x7f7bac00d800 tx=0x7f7bac00dcc0 comp rx=0 tx=0).stop 2023-11-26T07:19:30.117 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:30.112+0000 7f7bc3a13640 1 -- 172.21.15.143:0/581564583 shutdown_connections 2023-11-26T07:19:30.117 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:30.112+0000 7f7bc3a13640 1 --2- 172.21.15.143:0/581564583 >> [v2:172.21.15.143:6832/2150557544,v1:172.21.15.143:6833/2150557544] conn(0x7f7ba0066620 0x7f7ba0068ae0 unknown :-1 s=CLOSED pgs=281 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:19:30.117 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:30.112+0000 7f7bc3a13640 1 --2- 172.21.15.143:0/581564583 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f7bbc1596d0 0x7f7bbc157a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:19:30.117 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:30.112+0000 7f7bc3a13640 1 --2- 172.21.15.143:0/581564583 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f7bbc0ce680 0x7f7bbc1fa1e0 unknown :-1 s=CLOSED pgs=555 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:19:30.117 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:30.112+0000 7f7bc3a13640 1 --2- 172.21.15.143:0/581564583 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f7bbc0b2db0 0x7f7bbc1f5bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:19:30.117 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:30.112+0000 7f7bc3a13640 1 -- 172.21.15.143:0/581564583 >> 172.21.15.143:0/581564583 conn(0x7f7bbc06a890 msgr2=0x7f7bbc06ac70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:19:30.118 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:30.116+0000 7f7bc3a13640 1 -- 172.21.15.143:0/581564583 shutdown_connections 2023-11-26T07:19:30.118 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:30.116+0000 7f7bc3a13640 1 -- 172.21.15.143:0/581564583 wait complete. 2023-11-26T07:19:30.144 DEBUG:teuthology.orchestra.run.smithi143:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_data 64 --pg_num_min 64 --target_size_ratio 0.9 2023-11-26T07:19:30.290 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:30.288+0000 7fcd429df640 1 Processor -- start 2023-11-26T07:19:30.290 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:30.288+0000 7fcd429df640 1 -- start start 2023-11-26T07:19:30.292 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:30.288+0000 7fcd429df640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fcd3c143c00 0x7fcd3c0b1ef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:19:30.292 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:30.288+0000 7fcd429df640 1 -- --> v1:172.21.15.143:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fcd3c0b25f0 con 0x7fcd3c070390 2023-11-26T07:19:30.292 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:30.288+0000 7fcd429df640 1 -- --> v1:172.21.15.188:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fcd3c0b28d0 con 0x7fcd3c071550 2023-11-26T07:19:30.292 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:30.288+0000 7fcd429df640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcd3c0b2bb0 con 0x7fcd3c143c00 2023-11-26T07:19:30.292 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:30.288+0000 7fcd3b7fe640 1 --1- >> v1:172.21.15.143:6789/0 conn(0x7fcd3c070390 0x7fcd3c0cc7a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.143:6789/0 says I am v1:172.21.15.143:58366/0 (socket says 172.21.15.143:58366) 2023-11-26T07:19:30.292 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:30.288+0000 7fcd3b7fe640 1 -- 172.21.15.143:0/3287558399 learned_addr learned my addr 172.21.15.143:0/3287558399 (peer_addr_for_me v1:172.21.15.143:0/0) 2023-11-26T07:19:30.294 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:30.288+0000 7fcd3bfff640 1 --2- 172.21.15.143:0/3287558399 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fcd3c143c00 0x7fcd3c0b1ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:19:30.294 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:30.288+0000 7fcd3affd640 1 -- 172.21.15.143:0/3287558399 <== mon.0 v1:172.21.15.143:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3607971810 0 0) 0x7fcd3c0b25f0 con 0x7fcd3c070390 2023-11-26T07:19:30.294 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:30.288+0000 7fcd3affd640 1 -- 172.21.15.143:0/3287558399 --> v1:172.21.15.143:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fcd24005740 con 0x7fcd3c070390 2023-11-26T07:19:30.294 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:30.292+0000 7fcd3affd640 1 -- 172.21.15.143:0/3287558399 <== mon.1 v1:172.21.15.188:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 190001675 0 0) 0x7fcd3c0b28d0 con 0x7fcd3c071550 2023-11-26T07:19:30.294 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:30.292+0000 7fcd3affd640 1 -- 172.21.15.143:0/3287558399 --> v1:172.21.15.188:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fcd3c0b25f0 con 0x7fcd3c071550 2023-11-26T07:19:30.294 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:30.292+0000 7fcd3affd640 1 -- 172.21.15.143:0/3287558399 <== mon.0 v1:172.21.15.143:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 911514694 0 0) 0x7fcd24005740 con 0x7fcd3c070390 2023-11-26T07:19:30.294 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:30.292+0000 7fcd3affd640 1 -- 172.21.15.143:0/3287558399 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fcd3c143c00 msgr2=0x7fcd3c0b1ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-26T07:19:30.294 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:30.292+0000 7fcd3affd640 1 --2- 172.21.15.143:0/3287558399 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fcd3c143c00 0x7fcd3c0b1ef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:19:30.294 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:30.292+0000 7fcd3affd640 1 -- 172.21.15.143:0/3287558399 >> v1:172.21.15.188:6789/0 conn(0x7fcd3c071550 legacy=0x7fcd3c0b3690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:19:30.295 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:30.292+0000 7fcd3affd640 1 -- 172.21.15.143:0/3287558399 --> v1:172.21.15.143:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcd3c0b2e90 con 0x7fcd3c070390 2023-11-26T07:19:30.295 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:30.292+0000 7fcd3affd640 1 -- 172.21.15.143:0/3287558399 <== mon.0 v1:172.21.15.143:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2137882523 0 0) 0x7fcd30001680 con 0x7fcd3c070390 2023-11-26T07:19:30.295 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:30.292+0000 7fcd3affd640 1 -- 172.21.15.143:0/3287558399 >> v1:172.21.15.143:6789/0 conn(0x7fcd3c070390 legacy=0x7fcd3c0cc7a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:19:30.295 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:30.292+0000 7fcd3affd640 1 --2- 172.21.15.143:0/3287558399 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fcd3c143c00 0x7fcd24005d50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:19:30.295 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:30.292+0000 7fcd3affd640 1 --2- 172.21.15.143:0/3287558399 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fcd240062b0 0x7fcd240069e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:19:30.295 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:30.292+0000 7fcd3affd640 1 --2- 172.21.15.143:0/3287558399 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fcd24006f20 0x7fcd240073a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:19:30.295 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:30.292+0000 7fcd3affd640 1 -- 172.21.15.143:0/3287558399 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcd3c0b2bb0 con 0x7fcd24006f20 2023-11-26T07:19:30.295 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:30.292+0000 7fcd3affd640 1 -- 172.21.15.143:0/3287558399 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcd2400dd30 con 0x7fcd240062b0 2023-11-26T07:19:30.295 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:30.292+0000 7fcd3affd640 1 -- 172.21.15.143:0/3287558399 --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcd2400dec0 con 0x7fcd3c143c00 2023-11-26T07:19:30.295 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:30.292+0000 7fcd40f7c640 1 --2- 172.21.15.143:0/3287558399 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fcd24006f20 0x7fcd240073a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:19:30.295 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:30.292+0000 7fcd3b7fe640 1 --2- 172.21.15.143:0/3287558399 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fcd240062b0 0x7fcd240069e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:19:30.295 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:30.292+0000 7fcd3bfff640 1 --2- 172.21.15.143:0/3287558399 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fcd3c143c00 0x7fcd24005d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:19:30.296 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:30.292+0000 7fcd40f7c640 1 -- 172.21.15.143:0/3287558399 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fcd3c143c00 msgr2=0x7fcd24005d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-26T07:19:30.296 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:30.292+0000 7fcd40f7c640 1 --2- 172.21.15.143:0/3287558399 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fcd3c143c00 0x7fcd24005d50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:19:30.296 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:30.292+0000 7fcd40f7c640 1 -- 172.21.15.143:0/3287558399 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fcd240062b0 msgr2=0x7fcd240069e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-26T07:19:30.296 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:30.292+0000 7fcd40f7c640 1 --2- 172.21.15.143:0/3287558399 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fcd240062b0 0x7fcd240069e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:19:30.296 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:30.292+0000 7fcd40f7c640 1 -- 172.21.15.143:0/3287558399 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcd24005740 con 0x7fcd24006f20 2023-11-26T07:19:30.296 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:30.292+0000 7fcd3bfff640 1 --2- 172.21.15.143:0/3287558399 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fcd3c143c00 0x7fcd24005d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-26T07:19:30.297 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:30.292+0000 7fcd40f7c640 1 --2- 172.21.15.143:0/3287558399 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fcd24006f20 0x7fcd240073a0 secure :-1 s=READY pgs=557 cs=0 l=1 rev1=1 crypto rx=0x7fcd34001fd0 tx=0x7fcd3401f050 comp rx=0 tx=0).ready entity=mon.0 client_cookie=66bf2fd808c10e13 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:19:30.297 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:30.296+0000 7fcd3affd640 1 -- 172.21.15.143:0/3287558399 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcd340a8c70 con 0x7fcd24006f20 2023-11-26T07:19:30.297 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:30.296+0000 7fcd3affd640 1 -- 172.21.15.143:0/3287558399 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcd340a8df0 con 0x7fcd24006f20 2023-11-26T07:19:30.297 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:30.296+0000 7fcd3affd640 1 -- 172.21.15.143:0/3287558399 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcd34003bc0 con 0x7fcd24006f20 2023-11-26T07:19:30.297 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:30.296+0000 7fcd429df640 1 -- 172.21.15.143:0/3287558399 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fcd24006f20 msgr2=0x7fcd240073a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:19:30.298 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:30.296+0000 7fcd429df640 1 --2- 172.21.15.143:0/3287558399 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fcd24006f20 0x7fcd240073a0 secure :-1 s=READY pgs=557 cs=0 l=1 rev1=1 crypto rx=0x7fcd34001fd0 tx=0x7fcd3401f050 comp rx=0 tx=0).stop 2023-11-26T07:19:30.298 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:30.296+0000 7fcd40f7c640 1 -- 172.21.15.143:0/3287558399 reap_dead start 2023-11-26T07:19:30.298 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:30.296+0000 7fcd429df640 1 -- 172.21.15.143:0/3287558399 shutdown_connections 2023-11-26T07:19:30.298 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:30.296+0000 7fcd429df640 1 -- 172.21.15.143:0/3287558399 >> 172.21.15.143:0/3287558399 conn(0x7fcd3c0b0bf0 msgr2=0x7fcd3c0b1000 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:19:30.298 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:30.296+0000 7fcd429df640 1 -- 172.21.15.143:0/3287558399 shutdown_connections 2023-11-26T07:19:30.298 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:30.296+0000 7fcd429df640 1 -- 172.21.15.143:0/3287558399 wait complete. 2023-11-26T07:19:30.299 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:30.296+0000 7fcd429df640 1 Processor -- start 2023-11-26T07:19:30.299 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:30.296+0000 7fcd429df640 1 -- start start 2023-11-26T07:19:30.301 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:30.296+0000 7fcd429df640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fcd24006f20 0x7fcd3c1f6230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:19:30.301 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:30.296+0000 7fcd429df640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fcd3c0c3e60 0x7fcd3c1599a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:19:30.301 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:30.296+0000 7fcd429df640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fcd3c0ce260 0x7fcd3c0ce710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:19:30.301 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:30.296+0000 7fcd429df640 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcd3c0cec50 con 0x7fcd3c0ce260 2023-11-26T07:19:30.301 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:30.296+0000 7fcd429df640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcd3c1f3e00 con 0x7fcd24006f20 2023-11-26T07:19:30.301 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:30.296+0000 7fcd429df640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcd3c1f6a80 con 0x7fcd3c0c3e60 2023-11-26T07:19:30.301 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:30.296+0000 7fcd40f7c640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fcd3c0ce260 0x7fcd3c0ce710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:19:30.301 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:30.296+0000 7fcd40f7c640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fcd3c0ce260 0x7fcd3c0ce710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:40620/0 (socket says 172.21.15.143:40620) 2023-11-26T07:19:30.302 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:30.296+0000 7fcd40f7c640 1 -- 172.21.15.143:0/4294374720 learned_addr learned my addr 172.21.15.143:0/4294374720 (peer_addr_for_me v2:172.21.15.143:0/0) 2023-11-26T07:19:30.302 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:30.296+0000 7fcd3bfff640 1 --2- 172.21.15.143:0/4294374720 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fcd24006f20 0x7fcd3c1f6230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:19:30.302 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:30.296+0000 7fcd3b7fe640 1 --2- 172.21.15.143:0/4294374720 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fcd3c0c3e60 0x7fcd3c1599a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:19:30.302 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:30.300+0000 7fcd40f7c640 1 -- 172.21.15.143:0/4294374720 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fcd3c0c3e60 msgr2=0x7fcd3c1599a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:19:30.302 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:30.300+0000 7fcd40f7c640 1 --2- 172.21.15.143:0/4294374720 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fcd3c0c3e60 0x7fcd3c1599a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:19:30.302 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:30.300+0000 7fcd40f7c640 1 -- 172.21.15.143:0/4294374720 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fcd24006f20 msgr2=0x7fcd3c1f6230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:19:30.302 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:30.300+0000 7fcd40f7c640 1 --2- 172.21.15.143:0/4294374720 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fcd24006f20 0x7fcd3c1f6230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:19:30.303 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:30.300+0000 7fcd40f7c640 1 -- 172.21.15.143:0/4294374720 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcd3c1f6d60 con 0x7fcd3c0ce260 2023-11-26T07:19:30.303 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:30.300+0000 7fcd3b7fe640 1 --2- 172.21.15.143:0/4294374720 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fcd3c0c3e60 0x7fcd3c1599a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-26T07:19:30.303 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:30.300+0000 7fcd3bfff640 1 --2- 172.21.15.143:0/4294374720 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fcd24006f20 0x7fcd3c1f6230 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-26T07:19:30.303 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:30.300+0000 7fcd40f7c640 1 --2- 172.21.15.143:0/4294374720 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fcd3c0ce260 0x7fcd3c0ce710 secure :-1 s=READY pgs=558 cs=0 l=1 rev1=1 crypto rx=0x7fcd34005040 tx=0x7fcd24007a10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:19:30.304 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:30.300+0000 7fcd38ff9640 1 -- 172.21.15.143:0/4294374720 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcd34003e80 con 0x7fcd3c0ce260 2023-11-26T07:19:30.304 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:30.300+0000 7fcd38ff9640 1 -- 172.21.15.143:0/4294374720 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcd34007e00 con 0x7fcd3c0ce260 2023-11-26T07:19:30.304 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:30.300+0000 7fcd429df640 1 -- 172.21.15.143:0/4294374720 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcd3c1f7040 con 0x7fcd3c0ce260 2023-11-26T07:19:30.304 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:30.300+0000 7fcd38ff9640 1 -- 172.21.15.143:0/4294374720 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcd340a8c70 con 0x7fcd3c0ce260 2023-11-26T07:19:30.304 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:30.300+0000 7fcd429df640 1 -- 172.21.15.143:0/4294374720 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcd3c1f6770 con 0x7fcd3c0ce260 2023-11-26T07:19:30.304 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:30.300+0000 7fcd429df640 1 -- 172.21.15.143:0/4294374720 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fcd3c067720 con 0x7fcd3c0ce260 2023-11-26T07:19:30.304 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:30.300+0000 7fcd38ff9640 1 -- 172.21.15.143:0/4294374720 <== mon.0 v2:172.21.15.143:3300/0 4 ==== mgrmap(e 7) v1 ==== 86005+0+0 (secure 0 0 0) 0x7fcd34006b60 con 0x7fcd3c0ce260 2023-11-26T07:19:30.305 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:30.300+0000 7fcd38ff9640 1 --2- 172.21.15.143:0/4294374720 >> [v2:172.21.15.143:6832/2150557544,v1:172.21.15.143:6833/2150557544] conn(0x7fcd28066650 0x7fcd28068b10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:19:30.305 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:30.300+0000 7fcd38ff9640 1 -- 172.21.15.143:0/4294374720 <== mon.0 v2:172.21.15.143:3300/0 5 ==== osd_map(42..42 src has 1..42) v4 ==== 5227+0+0 (secure 0 0 0) 0x7fcd34120b10 con 0x7fcd3c0ce260 2023-11-26T07:19:30.305 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:30.304+0000 7fcd3bfff640 1 --2- 172.21.15.143:0/4294374720 >> [v2:172.21.15.143:6832/2150557544,v1:172.21.15.143:6833/2150557544] conn(0x7fcd28066650 0x7fcd28068b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:19:30.305 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:30.304+0000 7fcd3bfff640 1 --2- 172.21.15.143:0/4294374720 >> [v2:172.21.15.143:6832/2150557544,v1:172.21.15.143:6833/2150557544] conn(0x7fcd28066650 0x7fcd28068b10 secure :-1 s=READY pgs=282 cs=0 l=1 rev1=1 crypto rx=0x7fcd3c0b2150 tx=0x7fcd2c008040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:19:30.309 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:30.308+0000 7fcd38ff9640 1 -- 172.21.15.143:0/4294374720 <== mon.0 v2:172.21.15.143:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172649 (secure 0 0 0) 0x7fcd340eddc0 con 0x7fcd3c0ce260 2023-11-26T07:19:30.460 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:30.456+0000 7fcd429df640 1 -- 172.21.15.143:0/4294374720 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9} v 0) v1 -- 0x7fcd3c067720 con 0x7fcd3c0ce260 2023-11-26T07:19:31.121 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:31.120+0000 7fcd38ff9640 1 -- 172.21.15.143:0/4294374720 <== mon.0 v2:172.21.15.143:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]=0 pool 'cephfs_data' created v43) v1 ==== 170+0+0 (secure 0 0 0) 0x7fcd340bf020 con 0x7fcd3c0ce260 2023-11-26T07:19:31.121 INFO:teuthology.orchestra.run.smithi143.stderr:pool 'cephfs_data' created 2023-11-26T07:19:31.123 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:31.120+0000 7fcd429df640 1 -- 172.21.15.143:0/4294374720 >> [v2:172.21.15.143:6832/2150557544,v1:172.21.15.143:6833/2150557544] conn(0x7fcd28066650 msgr2=0x7fcd28068b10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:19:31.124 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:31.120+0000 7fcd429df640 1 --2- 172.21.15.143:0/4294374720 >> [v2:172.21.15.143:6832/2150557544,v1:172.21.15.143:6833/2150557544] conn(0x7fcd28066650 0x7fcd28068b10 secure :-1 s=READY pgs=282 cs=0 l=1 rev1=1 crypto rx=0x7fcd3c0b2150 tx=0x7fcd2c008040 comp rx=0 tx=0).stop 2023-11-26T07:19:31.124 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:31.120+0000 7fcd429df640 1 -- 172.21.15.143:0/4294374720 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fcd3c0ce260 msgr2=0x7fcd3c0ce710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:19:31.124 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:31.120+0000 7fcd429df640 1 --2- 172.21.15.143:0/4294374720 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fcd3c0ce260 0x7fcd3c0ce710 secure :-1 s=READY pgs=558 cs=0 l=1 rev1=1 crypto rx=0x7fcd34005040 tx=0x7fcd24007a10 comp rx=0 tx=0).stop 2023-11-26T07:19:31.124 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:31.120+0000 7fcd429df640 1 -- 172.21.15.143:0/4294374720 shutdown_connections 2023-11-26T07:19:31.124 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:31.120+0000 7fcd429df640 1 --2- 172.21.15.143:0/4294374720 >> [v2:172.21.15.143:6832/2150557544,v1:172.21.15.143:6833/2150557544] conn(0x7fcd28066650 0x7fcd28068b10 unknown :-1 s=CLOSED pgs=282 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:19:31.125 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:31.120+0000 7fcd429df640 1 --2- 172.21.15.143:0/4294374720 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fcd3c0ce260 0x7fcd3c0ce710 unknown :-1 s=CLOSED pgs=558 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:19:31.125 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:31.120+0000 7fcd429df640 1 --2- 172.21.15.143:0/4294374720 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fcd3c0c3e60 0x7fcd3c1599a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:19:31.125 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:31.120+0000 7fcd429df640 1 --2- 172.21.15.143:0/4294374720 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fcd24006f20 0x7fcd3c1f6230 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:19:31.125 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:31.120+0000 7fcd429df640 1 -- 172.21.15.143:0/4294374720 >> 172.21.15.143:0/4294374720 conn(0x7fcd3c0b0bf0 msgr2=0x7fcd3c070850 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:19:31.125 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:31.120+0000 7fcd429df640 1 -- 172.21.15.143:0/4294374720 shutdown_connections 2023-11-26T07:19:31.125 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:31.120+0000 7fcd429df640 1 -- 172.21.15.143:0/4294374720 wait complete. 2023-11-26T07:19:31.164 DEBUG:teuthology.orchestra.run.smithi143:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs new cephfs cephfs_metadata cephfs_data 2023-11-26T07:19:31.304 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:31.300+0000 7f32b7032640 1 Processor -- start 2023-11-26T07:19:31.305 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:31.300+0000 7f32b7032640 1 -- start start 2023-11-26T07:19:31.306 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:31.304+0000 7f32b7032640 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7f32b00cf990 0x7f32b015efa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:19:31.306 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:31.304+0000 7f32b7032640 1 -- --> v1:172.21.15.143:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f32b006c520 con 0x7f32b013e290 2023-11-26T07:19:31.306 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:31.304+0000 7f32b7032640 1 -- --> v1:172.21.15.188:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f32b006c800 con 0x7f32b015f4e0 2023-11-26T07:19:31.306 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:31.304+0000 7f32b7032640 1 -- --> v2:172.21.15.143:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f32b006cae0 con 0x7f32b00cf990 2023-11-26T07:19:31.306 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:31.304+0000 7f32a7fff640 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7f32b00cf990 0x7f32b015efa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:19:31.307 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:31.304+0000 7f32b4dce640 1 --1- >> v1:172.21.15.143:6789/0 conn(0x7f32b013e290 0x7f32b00cf280 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.143:6789/0 says I am v1:172.21.15.143:58378/0 (socket says 172.21.15.143:58378) 2023-11-26T07:19:31.307 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:31.304+0000 7f32a7fff640 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7f32b00cf990 0x7f32b015efa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:40634/0 (socket says 172.21.15.143:40634) 2023-11-26T07:19:31.307 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:31.304+0000 7f32b4dce640 1 -- 172.21.15.143:0/4032656808 learned_addr learned my addr 172.21.15.143:0/4032656808 (peer_addr_for_me v1:172.21.15.143:0/0) 2023-11-26T07:19:31.307 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:31.304+0000 7f32a7fff640 1 -- 172.21.15.143:0/4032656808 >> v1:172.21.15.188:6789/0 conn(0x7f32b015f4e0 legacy=0x7f32b00cd7b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:19:31.308 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:31.304+0000 7f32a7fff640 1 -- 172.21.15.143:0/4032656808 >> v1:172.21.15.143:6789/0 conn(0x7f32b013e290 legacy=0x7f32b00cf280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:19:31.308 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:31.304+0000 7f32a7fff640 1 -- 172.21.15.143:0/4032656808 --> v2:172.21.15.143:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f32b006cdc0 con 0x7f32b00cf990 2023-11-26T07:19:31.309 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:31.308+0000 7f32a7fff640 1 --2- 172.21.15.143:0/4032656808 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f32b00cf990 0x7f32b015efa0 secure :-1 s=READY pgs=560 cs=0 l=1 rev1=1 crypto rx=0x7f32a00097c0 tx=0x7f32a001c450 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b833c667bd815c33 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:19:31.310 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:31.308+0000 7f32a77fe640 1 -- 172.21.15.143:0/4032656808 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f32a00ab670 con 0x7f32b00cf990 2023-11-26T07:19:31.310 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:31.308+0000 7f32a77fe640 1 -- 172.21.15.143:0/4032656808 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f32a00ab7d0 con 0x7f32b00cf990 2023-11-26T07:19:31.310 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:31.308+0000 7f32a77fe640 1 -- 172.21.15.143:0/4032656808 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f32a00abb60 con 0x7f32b00cf990 2023-11-26T07:19:31.311 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:31.308+0000 7f32b7032640 1 -- 172.21.15.143:0/4032656808 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f32b00cf990 msgr2=0x7f32b015efa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:19:31.311 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:31.308+0000 7f32b7032640 1 --2- 172.21.15.143:0/4032656808 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f32b00cf990 0x7f32b015efa0 secure :-1 s=READY pgs=560 cs=0 l=1 rev1=1 crypto rx=0x7f32a00097c0 tx=0x7f32a001c450 comp rx=0 tx=0).stop 2023-11-26T07:19:31.311 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:31.308+0000 7f32b7032640 1 -- 172.21.15.143:0/4032656808 shutdown_connections 2023-11-26T07:19:31.311 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:31.308+0000 7f32b7032640 1 --2- 172.21.15.143:0/4032656808 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f32b00cf990 0x7f32b015efa0 unknown :-1 s=CLOSED pgs=560 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:19:31.311 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:31.308+0000 7f32b7032640 1 -- 172.21.15.143:0/4032656808 >> 172.21.15.143:0/4032656808 conn(0x7f32b006a890 msgr2=0x7f32b006aca0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:19:31.311 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:31.308+0000 7f32b7032640 1 -- 172.21.15.143:0/4032656808 shutdown_connections 2023-11-26T07:19:31.311 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:31.308+0000 7f32b7032640 1 -- 172.21.15.143:0/4032656808 wait complete. 2023-11-26T07:19:31.312 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:31.308+0000 7f32b7032640 1 Processor -- start 2023-11-26T07:19:31.312 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:31.308+0000 7f32b7032640 1 -- start start 2023-11-26T07:19:31.314 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:31.312+0000 7f32b7032640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f32b00cf990 0x7f32b01341f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:19:31.314 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:31.312+0000 7f32b7032640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f32b013e290 0x7f32b0135b70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:19:31.314 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:31.312+0000 7f32b7032640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f32b015f4e0 0x7f32b01360b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:19:31.314 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:31.312+0000 7f32b7032640 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f32b0131b00 con 0x7f32b00cf990 2023-11-26T07:19:31.314 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:31.312+0000 7f32b7032640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f32b00b1aa0 con 0x7f32b015f4e0 2023-11-26T07:19:31.314 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:31.312+0000 7f32b7032640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f32b0154020 con 0x7f32b013e290 2023-11-26T07:19:31.314 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:31.312+0000 7f32b4dce640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f32b00cf990 0x7f32b01341f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:19:31.314 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:31.312+0000 7f32b4dce640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f32b00cf990 0x7f32b01341f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:40642/0 (socket says 172.21.15.143:40642) 2023-11-26T07:19:31.314 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:31.312+0000 7f32b4dce640 1 -- 172.21.15.143:0/953522007 learned_addr learned my addr 172.21.15.143:0/953522007 (peer_addr_for_me v2:172.21.15.143:0/0) 2023-11-26T07:19:31.314 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:31.312+0000 7f32b55cf640 1 --2- 172.21.15.143:0/953522007 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f32b015f4e0 0x7f32b01360b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:19:31.314 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:31.312+0000 7f32a7fff640 1 --2- 172.21.15.143:0/953522007 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f32b013e290 0x7f32b0135b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:19:31.315 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:31.312+0000 7f32b4dce640 1 -- 172.21.15.143:0/953522007 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f32b013e290 msgr2=0x7f32b0135b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:19:31.315 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:31.312+0000 7f32b4dce640 1 --2- 172.21.15.143:0/953522007 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f32b013e290 0x7f32b0135b70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:19:31.315 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:31.312+0000 7f32b4dce640 1 -- 172.21.15.143:0/953522007 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f32b015f4e0 msgr2=0x7f32b01360b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:19:31.315 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:31.312+0000 7f32b4dce640 1 --2- 172.21.15.143:0/953522007 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f32b015f4e0 0x7f32b01360b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:19:31.315 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:31.312+0000 7f32b4dce640 1 -- 172.21.15.143:0/953522007 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f32a0009420 con 0x7f32b00cf990 2023-11-26T07:19:31.315 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:31.312+0000 7f32b55cf640 1 --2- 172.21.15.143:0/953522007 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f32b015f4e0 0x7f32b01360b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-26T07:19:31.315 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:31.312+0000 7f32b4dce640 1 --2- 172.21.15.143:0/953522007 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f32b00cf990 0x7f32b01341f0 secure :-1 s=READY pgs=561 cs=0 l=1 rev1=1 crypto rx=0x7f32ac00c500 tx=0x7f32ac00c9c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:19:31.316 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:31.312+0000 7f32a57fa640 1 -- 172.21.15.143:0/953522007 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f32ac005cf0 con 0x7f32b00cf990 2023-11-26T07:19:31.316 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:31.312+0000 7f32a57fa640 1 -- 172.21.15.143:0/953522007 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f32ac005e50 con 0x7f32b00cf990 2023-11-26T07:19:31.316 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:31.312+0000 7f32a57fa640 1 -- 172.21.15.143:0/953522007 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f32ac0072f0 con 0x7f32b00cf990 2023-11-26T07:19:31.316 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:31.312+0000 7f32b7032640 1 -- 172.21.15.143:0/953522007 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f32b0153640 con 0x7f32b00cf990 2023-11-26T07:19:31.316 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:31.312+0000 7f32b7032640 1 -- 172.21.15.143:0/953522007 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f32b0131720 con 0x7f32b00cf990 2023-11-26T07:19:31.316 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:31.312+0000 7f32b7032640 1 -- 172.21.15.143:0/953522007 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f32b0067680 con 0x7f32b00cf990 2023-11-26T07:19:31.316 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:31.316+0000 7f32a57fa640 1 -- 172.21.15.143:0/953522007 <== mon.0 v2:172.21.15.143:3300/0 4 ==== mgrmap(e 7) v1 ==== 86005+0+0 (secure 0 0 0) 0x7f32ac007450 con 0x7f32b00cf990 2023-11-26T07:19:31.317 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:31.316+0000 7f32a57fa640 1 --2- 172.21.15.143:0/953522007 >> [v2:172.21.15.143:6832/2150557544,v1:172.21.15.143:6833/2150557544] conn(0x7f329c066620 0x7f329c068ae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:19:31.317 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:31.316+0000 7f32a57fa640 1 -- 172.21.15.143:0/953522007 <== mon.0 v2:172.21.15.143:3300/0 5 ==== osd_map(43..43 src has 1..43) v4 ==== 5609+0+0 (secure 0 0 0) 0x7f32ac0838c0 con 0x7f32b00cf990 2023-11-26T07:19:31.317 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:31.316+0000 7f32a7fff640 1 --2- 172.21.15.143:0/953522007 >> [v2:172.21.15.143:6832/2150557544,v1:172.21.15.143:6833/2150557544] conn(0x7f329c066620 0x7f329c068ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:19:31.318 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:31.316+0000 7f32a7fff640 1 --2- 172.21.15.143:0/953522007 >> [v2:172.21.15.143:6832/2150557544,v1:172.21.15.143:6833/2150557544] conn(0x7f329c066620 0x7f329c068ae0 secure :-1 s=READY pgs=283 cs=0 l=1 rev1=1 crypto rx=0x7f32a00af000 tx=0x7f32a00a7000 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:19:31.321 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:31.320+0000 7f32a57fa640 1 -- 172.21.15.143:0/953522007 <== mon.0 v2:172.21.15.143:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172649 (secure 0 0 0) 0x7f32ac050df0 con 0x7f32b00cf990 2023-11-26T07:19:31.503 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:31.499+0000 7f32b7032640 1 -- 172.21.15.143:0/953522007 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"} v 0) v1 -- 0x7f32b006df50 con 0x7f32b00cf990 2023-11-26T07:19:32.138 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:32.135+0000 7f32a57fa640 1 -- 172.21.15.143:0/953522007 <== mon.0 v2:172.21.15.143:3300/0 7 ==== mon_command_ack([{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]=0 new fs with metadata pool 6 and data pool 7 v21) v1 ==== 172+0+0 (secure 0 0 0) 0x7f32ac054410 con 0x7f32b00cf990 2023-11-26T07:19:32.145 INFO:teuthology.orchestra.run.smithi143.stderr:new fs with metadata pool 6 and data pool 7 2023-11-26T07:19:32.147 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:32.143+0000 7f32b7032640 1 -- 172.21.15.143:0/953522007 >> [v2:172.21.15.143:6832/2150557544,v1:172.21.15.143:6833/2150557544] conn(0x7f329c066620 msgr2=0x7f329c068ae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:19:32.148 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:32.143+0000 7f32b7032640 1 --2- 172.21.15.143:0/953522007 >> [v2:172.21.15.143:6832/2150557544,v1:172.21.15.143:6833/2150557544] conn(0x7f329c066620 0x7f329c068ae0 secure :-1 s=READY pgs=283 cs=0 l=1 rev1=1 crypto rx=0x7f32a00af000 tx=0x7f32a00a7000 comp rx=0 tx=0).stop 2023-11-26T07:19:32.148 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:32.143+0000 7f32b7032640 1 -- 172.21.15.143:0/953522007 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f32b00cf990 msgr2=0x7f32b01341f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:19:32.148 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:32.143+0000 7f32b7032640 1 --2- 172.21.15.143:0/953522007 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f32b00cf990 0x7f32b01341f0 secure :-1 s=READY pgs=561 cs=0 l=1 rev1=1 crypto rx=0x7f32ac00c500 tx=0x7f32ac00c9c0 comp rx=0 tx=0).stop 2023-11-26T07:19:32.151 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:32.147+0000 7f32b7032640 1 -- 172.21.15.143:0/953522007 shutdown_connections 2023-11-26T07:19:32.152 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:32.147+0000 7f32b7032640 1 --2- 172.21.15.143:0/953522007 >> [v2:172.21.15.143:6832/2150557544,v1:172.21.15.143:6833/2150557544] conn(0x7f329c066620 0x7f329c068ae0 unknown :-1 s=CLOSED pgs=283 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:19:32.152 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:32.147+0000 7f32b7032640 1 --2- 172.21.15.143:0/953522007 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f32b015f4e0 0x7f32b01360b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:19:32.152 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:32.147+0000 7f32b7032640 1 --2- 172.21.15.143:0/953522007 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f32b013e290 0x7f32b0135b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:19:32.152 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:32.147+0000 7f32b7032640 1 --2- 172.21.15.143:0/953522007 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f32b00cf990 0x7f32b01341f0 unknown :-1 s=CLOSED pgs=561 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:19:32.152 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:32.147+0000 7f32b7032640 1 -- 172.21.15.143:0/953522007 >> 172.21.15.143:0/953522007 conn(0x7f32b006a890 msgr2=0x7f32b0071430 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:19:32.152 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:32.147+0000 7f32b7032640 1 -- 172.21.15.143:0/953522007 shutdown_connections 2023-11-26T07:19:32.152 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:32.147+0000 7f32b7032640 1 -- 172.21.15.143:0/953522007 wait complete. 2023-11-26T07:19:32.185 DEBUG:teuthology.orchestra.run.smithi143:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-11-26T07:19:32.324 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:32.319+0000 7fa06ac9a640 1 Processor -- start 2023-11-26T07:19:32.324 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:32.319+0000 7fa06ac9a640 1 -- start start 2023-11-26T07:19:32.325 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:32.323+0000 7fa06ac9a640 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7fa0640cf280 0x7fa0640cf710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:19:32.325 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:32.323+0000 7fa06ac9a640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fa0640cd1b0 0x7fa0640cd5c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:19:32.325 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:32.323+0000 7fa06ac9a640 1 -- --> v1:172.21.15.188:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa0640cc730 con 0x7fa064143ac0 2023-11-26T07:19:32.325 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:32.323+0000 7fa06ac9a640 1 -- --> v2:172.21.15.188:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa064143ea0 con 0x7fa0640cf280 2023-11-26T07:19:32.326 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:32.323+0000 7fa06ac9a640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa0640cfc50 con 0x7fa0640cd1b0 2023-11-26T07:19:32.326 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:32.323+0000 7fa069237640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fa0640cd1b0 0x7fa0640cd5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:19:32.326 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:32.323+0000 7fa05bfff640 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7fa0640cf280 0x7fa0640cf710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:19:32.327 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:32.323+0000 7fa069237640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fa0640cd1b0 0x7fa0640cd5c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.188:3301/0 says I am v2:172.21.15.143:57204/0 (socket says 172.21.15.143:57204) 2023-11-26T07:19:32.327 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:32.323+0000 7fa069237640 1 -- 172.21.15.143:0/795627 learned_addr learned my addr 172.21.15.143:0/795627 (peer_addr_for_me v2:172.21.15.143:0/0) 2023-11-26T07:19:32.327 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:32.323+0000 7fa05bfff640 1 -- 172.21.15.143:0/795627 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fa0640cd1b0 msgr2=0x7fa0640cd5c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-26T07:19:32.327 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:32.323+0000 7fa05bfff640 1 --2- 172.21.15.143:0/795627 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fa0640cd1b0 0x7fa0640cd5c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:19:32.327 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:32.323+0000 7fa05bfff640 1 -- 172.21.15.143:0/795627 >> v1:172.21.15.188:6789/0 conn(0x7fa064143ac0 legacy=0x7fa064070370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:19:32.327 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:32.323+0000 7fa05b7fe640 1 -- 172.21.15.143:0/795627 <== mon.1 v1:172.21.15.188:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2913528357 0 0) 0x7fa0640cc730 con 0x7fa064143ac0 2023-11-26T07:19:32.327 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:32.323+0000 7fa05bfff640 1 -- 172.21.15.143:0/795627 --> v2:172.21.15.188:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa064070ee0 con 0x7fa0640cf280 2023-11-26T07:19:32.327 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:32.323+0000 7fa069237640 1 --2- 172.21.15.143:0/795627 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fa0640cd1b0 0x7fa0640cd5c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-26T07:19:32.329 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:32.327+0000 7fa05bfff640 1 --2- 172.21.15.143:0/795627 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fa0640cf280 0x7fa0640cf710 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x7fa0540097c0 tx=0x7fa05401c480 comp rx=0 tx=0).ready entity=mon.1 client_cookie=507a1a2b3eb12c39 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:19:32.330 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:32.327+0000 7fa05b7fe640 1 -- 172.21.15.143:0/795627 <== mon.1 v2:172.21.15.188:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa0540ab680 con 0x7fa0640cf280 2023-11-26T07:19:32.330 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:32.327+0000 7fa05b7fe640 1 -- 172.21.15.143:0/795627 <== mon.1 v2:172.21.15.188:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa0540ab7e0 con 0x7fa0640cf280 2023-11-26T07:19:32.330 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:32.327+0000 7fa05b7fe640 1 -- 172.21.15.143:0/795627 <== mon.1 v2:172.21.15.188:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa0540abb70 con 0x7fa0640cf280 2023-11-26T07:19:32.331 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:32.327+0000 7fa06ac9a640 1 -- 172.21.15.143:0/795627 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fa0640cf280 msgr2=0x7fa0640cf710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:19:32.331 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:32.327+0000 7fa06ac9a640 1 --2- 172.21.15.143:0/795627 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fa0640cf280 0x7fa0640cf710 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x7fa0540097c0 tx=0x7fa05401c480 comp rx=0 tx=0).stop 2023-11-26T07:19:32.331 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:32.327+0000 7fa06ac9a640 1 -- 172.21.15.143:0/795627 shutdown_connections 2023-11-26T07:19:32.331 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:32.327+0000 7fa06ac9a640 1 --2- 172.21.15.143:0/795627 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fa0640cf280 0x7fa0640cf710 unknown :-1 s=CLOSED pgs=246 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:19:32.331 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:32.327+0000 7fa06ac9a640 1 --2- 172.21.15.143:0/795627 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fa0640cd1b0 0x7fa0640cd5c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:19:32.331 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:32.327+0000 7fa06ac9a640 1 -- 172.21.15.143:0/795627 >> 172.21.15.143:0/795627 conn(0x7fa06406a890 msgr2=0x7fa06406aca0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:19:32.331 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:32.327+0000 7fa06ac9a640 1 -- 172.21.15.143:0/795627 shutdown_connections 2023-11-26T07:19:32.331 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:32.327+0000 7fa06ac9a640 1 -- 172.21.15.143:0/795627 wait complete. 2023-11-26T07:19:32.332 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:32.327+0000 7fa06ac9a640 1 Processor -- start 2023-11-26T07:19:32.332 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:32.327+0000 7fa06ac9a640 1 -- start start 2023-11-26T07:19:32.334 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:32.331+0000 7fa06ac9a640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fa0640cd1b0 0x7fa0640adfa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:19:32.334 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:32.331+0000 7fa06ac9a640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fa0640cf280 0x7fa06413c080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:19:32.334 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:32.331+0000 7fa06ac9a640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fa064143ac0 0x7fa064156350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:19:32.334 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:32.331+0000 7fa06ac9a640 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa0640aeb60 con 0x7fa064143ac0 2023-11-26T07:19:32.334 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:32.331+0000 7fa06ac9a640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa06413c780 con 0x7fa0640cd1b0 2023-11-26T07:19:32.334 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:32.331+0000 7fa06ac9a640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa064158620 con 0x7fa0640cf280 2023-11-26T07:19:32.334 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:32.331+0000 7fa069237640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fa064143ac0 0x7fa064156350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:19:32.334 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:32.331+0000 7fa069237640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fa064143ac0 0x7fa064156350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:40646/0 (socket says 172.21.15.143:40646) 2023-11-26T07:19:32.334 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:32.331+0000 7fa069237640 1 -- 172.21.15.143:0/3187878108 learned_addr learned my addr 172.21.15.143:0/3187878108 (peer_addr_for_me v2:172.21.15.143:0/0) 2023-11-26T07:19:32.334 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:32.331+0000 7fa05bfff640 1 --2- 172.21.15.143:0/3187878108 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fa0640cf280 0x7fa06413c080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:19:32.335 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:32.331+0000 7fa068a36640 1 --2- 172.21.15.143:0/3187878108 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fa0640cd1b0 0x7fa0640adfa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:19:32.335 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:32.331+0000 7fa069237640 1 -- 172.21.15.143:0/3187878108 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fa0640cf280 msgr2=0x7fa06413c080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:19:32.336 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:32.331+0000 7fa069237640 1 --2- 172.21.15.143:0/3187878108 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fa0640cf280 0x7fa06413c080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:19:32.336 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:32.331+0000 7fa069237640 1 -- 172.21.15.143:0/3187878108 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fa0640cd1b0 msgr2=0x7fa0640adfa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:19:32.336 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:32.331+0000 7fa069237640 1 --2- 172.21.15.143:0/3187878108 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fa0640cd1b0 0x7fa0640adfa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:19:32.336 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:32.331+0000 7fa069237640 1 -- 172.21.15.143:0/3187878108 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa05c009b30 con 0x7fa064143ac0 2023-11-26T07:19:32.336 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:32.331+0000 7fa05bfff640 1 --2- 172.21.15.143:0/3187878108 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fa0640cf280 0x7fa06413c080 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-26T07:19:32.336 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:32.331+0000 7fa069237640 1 --2- 172.21.15.143:0/3187878108 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fa064143ac0 0x7fa064156350 secure :-1 s=READY pgs=562 cs=0 l=1 rev1=1 crypto rx=0x7fa05c00efc0 tx=0x7fa05c00c490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:19:32.336 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:32.331+0000 7fa0597fa640 1 -- 172.21.15.143:0/3187878108 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa05c018070 con 0x7fa064143ac0 2023-11-26T07:19:32.336 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:32.331+0000 7fa0597fa640 1 -- 172.21.15.143:0/3187878108 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa05c00f040 con 0x7fa064143ac0 2023-11-26T07:19:32.336 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:32.331+0000 7fa06ac9a640 1 -- 172.21.15.143:0/3187878108 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa054009420 con 0x7fa064143ac0 2023-11-26T07:19:32.336 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:32.331+0000 7fa0597fa640 1 -- 172.21.15.143:0/3187878108 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa05c00ce00 con 0x7fa064143ac0 2023-11-26T07:19:32.336 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:32.331+0000 7fa06ac9a640 1 -- 172.21.15.143:0/3187878108 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa064158be0 con 0x7fa064143ac0 2023-11-26T07:19:32.337 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:32.331+0000 7fa06ac9a640 1 -- 172.21.15.143:0/3187878108 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa064067680 con 0x7fa064143ac0 2023-11-26T07:19:32.337 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:32.335+0000 7fa0597fa640 1 -- 172.21.15.143:0/3187878108 <== mon.0 v2:172.21.15.143:3300/0 4 ==== mgrmap(e 7) v1 ==== 86005+0+0 (secure 0 0 0) 0x7fa05c014480 con 0x7fa064143ac0 2023-11-26T07:19:32.337 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:32.335+0000 7fa0597fa640 1 --2- 172.21.15.143:0/3187878108 >> [v2:172.21.15.143:6832/2150557544,v1:172.21.15.143:6833/2150557544] conn(0x7fa050066620 0x7fa050068ae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:19:32.338 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:32.335+0000 7fa0597fa640 1 -- 172.21.15.143:0/3187878108 <== mon.0 v2:172.21.15.143:3300/0 5 ==== osd_map(44..44 src has 1..44) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fa05c084420 con 0x7fa064143ac0 2023-11-26T07:19:32.338 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:32.335+0000 7fa068a36640 1 --2- 172.21.15.143:0/3187878108 >> [v2:172.21.15.143:6832/2150557544,v1:172.21.15.143:6833/2150557544] conn(0x7fa050066620 0x7fa050068ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:19:32.338 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:32.335+0000 7fa068a36640 1 --2- 172.21.15.143:0/3187878108 >> [v2:172.21.15.143:6832/2150557544,v1:172.21.15.143:6833/2150557544] conn(0x7fa050066620 0x7fa050068ae0 secure :-1 s=READY pgs=284 cs=0 l=1 rev1=1 crypto rx=0x7fa060000f70 tx=0x7fa060009040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:19:32.341 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:32.339+0000 7fa0597fa640 1 -- 172.21.15.143:0/3187878108 <== mon.0 v2:172.21.15.143:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172649 (secure 0 0 0) 0x7fa05c0515b0 con 0x7fa064143ac0 2023-11-26T07:19:32.492 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:32.487+0000 7fa06ac9a640 1 -- 172.21.15.143:0/3187878108 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fa064158da0 con 0x7fa064143ac0 2023-11-26T07:19:32.493 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:32.491+0000 7fa0597fa640 1 -- 172.21.15.143:0/3187878108 <== mon.0 v2:172.21.15.143:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v44) v1 ==== 74+0+16562 (secure 0 0 0) 0x7fa05c054bd0 con 0x7fa064143ac0 2023-11-26T07:19:32.494 INFO:teuthology.orchestra.run.smithi143.stdout: 2023-11-26T07:19:32.494 INFO:teuthology.orchestra.run.smithi143.stdout:{"epoch":44,"fsid":"f5033060-2469-4a07-95fe-2b76f5404bc1","created":"2023-11-26T07:17:28.613663+0000","modified":"2023-11-26T07:19:31.503249+0000","last_up_change":"2023-11-26T07:17:41.157813+0000","last_in_change":"2023-11-26T07:17:31.181658+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":7,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-11-26T07:17:41.758597+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":6,"pool_name":"cephfs_metadata","create_time":"2023-11-26T07:19:29.208075+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"44","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":7,"pool_name":"cephfs_data","create_time":"2023-11-26T07:19:30.503743+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"44","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"ebbee2c6-023e-44b9-9f1d-6f7bfd309730","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":43,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6817","nonce":639807369},{"type":"v1","addr":"172.21.15.143:6819","nonce":639807369}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6821","nonce":639807369},{"type":"v1","addr":"172.21.15.143:6823","nonce":639807369}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6830","nonce":639807369},{"type":"v1","addr":"172.21.15.143:6831","nonce":639807369}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6825","nonce":639807369},{"type":"v1","addr":"172.21.15.143:6828","nonce":639807369}]},"public_addr":"172.21.15.143:6819/639807369","cluster_addr":"172.21.15.143:6823/639807369","heartbeat_back_addr":"172.21.15.143:6831/639807369","heartbeat_front_addr":"172.21.15.143:6828/639807369","state":["exists","up"]},{"osd":1,"uuid":"fec4b8be-7122-49e9-9a19-5850a0700001","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":43,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6816","nonce":1144942530},{"type":"v1","addr":"172.21.15.143:6818","nonce":1144942530}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6820","nonce":1144942530},{"type":"v1","addr":"172.21.15.143:6822","nonce":1144942530}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6827","nonce":1144942530},{"type":"v1","addr":"172.21.15.143:6829","nonce":1144942530}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6824","nonce":1144942530},{"type":"v1","addr":"172.21.15.143:6826","nonce":1144942530}]},"public_addr":"172.21.15.143:6818/1144942530","cluster_addr":"172.21.15.143:6822/1144942530","heartbeat_back_addr":"172.21.15.143:6829/1144942530","heartbeat_front_addr":"172.21.15.143:6826/1144942530","state":["exists","up"]},{"osd":2,"uuid":"2888af5b-f933-46ca-8f95-6dd494ff43f6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":43,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6808","nonce":3909095186},{"type":"v1","addr":"172.21.15.143:6809","nonce":3909095186}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6810","nonce":3909095186},{"type":"v1","addr":"172.21.15.143:6811","nonce":3909095186}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6814","nonce":3909095186},{"type":"v1","addr":"172.21.15.143:6815","nonce":3909095186}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6812","nonce":3909095186},{"type":"v1","addr":"172.21.15.143:6813","nonce":3909095186}]},"public_addr":"172.21.15.143:6809/3909095186","cluster_addr":"172.21.15.143:6811/3909095186","heartbeat_back_addr":"172.21.15.143:6815/3909095186","heartbeat_front_addr":"172.21.15.143:6813/3909095186","state":["exists","up"]},{"osd":3,"uuid":"d1f52435-7bc1-4081-9550-9cd1ba55f549","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":43,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6800","nonce":3854210991},{"type":"v1","addr":"172.21.15.143:6801","nonce":3854210991}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6802","nonce":3854210991},{"type":"v1","addr":"172.21.15.143:6803","nonce":3854210991}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6806","nonce":3854210991},{"type":"v1","addr":"172.21.15.143:6807","nonce":3854210991}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6804","nonce":3854210991},{"type":"v1","addr":"172.21.15.143:6805","nonce":3854210991}]},"public_addr":"172.21.15.143:6801/3854210991","cluster_addr":"172.21.15.143:6803/3854210991","heartbeat_back_addr":"172.21.15.143:6807/3854210991","heartbeat_front_addr":"172.21.15.143:6805/3854210991","state":["exists","up"]},{"osd":4,"uuid":"e2ae8b6e-6e25-49b2-bcbf-aacce68f06a3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":43,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6800","nonce":3924606902},{"type":"v1","addr":"172.21.15.188:6801","nonce":3924606902}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6802","nonce":3924606902},{"type":"v1","addr":"172.21.15.188:6803","nonce":3924606902}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6806","nonce":3924606902},{"type":"v1","addr":"172.21.15.188:6807","nonce":3924606902}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6804","nonce":3924606902},{"type":"v1","addr":"172.21.15.188:6805","nonce":3924606902}]},"public_addr":"172.21.15.188:6801/3924606902","cluster_addr":"172.21.15.188:6803/3924606902","heartbeat_back_addr":"172.21.15.188:6807/3924606902","heartbeat_front_addr":"172.21.15.188:6805/3924606902","state":["exists","up"]},{"osd":5,"uuid":"3cec8a32-7698-4c36-8e89-e4ba11934f33","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":43,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6813","nonce":186841544},{"type":"v1","addr":"172.21.15.188:6815","nonce":186841544}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6817","nonce":186841544},{"type":"v1","addr":"172.21.15.188:6818","nonce":186841544}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6822","nonce":186841544},{"type":"v1","addr":"172.21.15.188:6823","nonce":186841544}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6820","nonce":186841544},{"type":"v1","addr":"172.21.15.188:6821","nonce":186841544}]},"public_addr":"172.21.15.188:6815/186841544","cluster_addr":"172.21.15.188:6818/186841544","heartbeat_back_addr":"172.21.15.188:6823/186841544","heartbeat_front_addr":"172.21.15.188:6821/186841544","state":["exists","up"]},{"osd":6,"uuid":"58a65d5d-e090-473a-81b4-ad38e85029f9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":43,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6808","nonce":2153499405},{"type":"v1","addr":"172.21.15.188:6809","nonce":2153499405}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6810","nonce":2153499405},{"type":"v1","addr":"172.21.15.188:6811","nonce":2153499405}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6816","nonce":2153499405},{"type":"v1","addr":"172.21.15.188:6819","nonce":2153499405}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6812","nonce":2153499405},{"type":"v1","addr":"172.21.15.188:6814","nonce":2153499405}]},"public_addr":"172.21.15.188:6809/2153499405","cluster_addr":"172.21.15.188:6811/2153499405","heartbeat_back_addr":"172.21.15.188:6819/2153499405","heartbeat_front_addr":"172.21.15.188:6814/2153499405","state":["exists","up"]},{"osd":7,"uuid":"ea5b7f70-3554-4cf8-ac30-f7c59b80678d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":43,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6824","nonce":2722096729},{"type":"v1","addr":"172.21.15.188:6825","nonce":2722096729}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6826","nonce":2722096729},{"type":"v1","addr":"172.21.15.188:6827","nonce":2722096729}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6830","nonce":2722096729},{"type":"v1","addr":"172.21.15.188:6831","nonce":2722096729}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6828","nonce":2722096729},{"type":"v1","addr":"172.21.15.188:6829","nonce":2722096729}]},"public_addr":"172.21.15.188:6825/2722096729","cluster_addr":"172.21.15.188:6827/2722096729","heartbeat_back_addr":"172.21.15.188:6831/2722096729","heartbeat_front_addr":"172.21.15.188:6829/2722096729","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-26T07:17:36.995032+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-26T07:17:36.932266+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-26T07:17:36.829304+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-26T07:17:36.804297+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-26T07:17:36.667892+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-26T07:17:36.964868+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-26T07:17:36.827332+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-26T07:17:36.893179+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-11-26T07:19:32.495 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:32.491+0000 7fa06ac9a640 1 -- 172.21.15.143:0/3187878108 >> [v2:172.21.15.143:6832/2150557544,v1:172.21.15.143:6833/2150557544] conn(0x7fa050066620 msgr2=0x7fa050068ae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:19:32.496 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:32.491+0000 7fa06ac9a640 1 --2- 172.21.15.143:0/3187878108 >> [v2:172.21.15.143:6832/2150557544,v1:172.21.15.143:6833/2150557544] conn(0x7fa050066620 0x7fa050068ae0 secure :-1 s=READY pgs=284 cs=0 l=1 rev1=1 crypto rx=0x7fa060000f70 tx=0x7fa060009040 comp rx=0 tx=0).stop 2023-11-26T07:19:32.496 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:32.491+0000 7fa06ac9a640 1 -- 172.21.15.143:0/3187878108 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fa064143ac0 msgr2=0x7fa064156350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:19:32.496 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:32.491+0000 7fa06ac9a640 1 --2- 172.21.15.143:0/3187878108 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fa064143ac0 0x7fa064156350 secure :-1 s=READY pgs=562 cs=0 l=1 rev1=1 crypto rx=0x7fa05c00efc0 tx=0x7fa05c00c490 comp rx=0 tx=0).stop 2023-11-26T07:19:32.496 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:32.491+0000 7fa06ac9a640 1 -- 172.21.15.143:0/3187878108 shutdown_connections 2023-11-26T07:19:32.496 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:32.491+0000 7fa06ac9a640 1 --2- 172.21.15.143:0/3187878108 >> [v2:172.21.15.143:6832/2150557544,v1:172.21.15.143:6833/2150557544] conn(0x7fa050066620 0x7fa050068ae0 unknown :-1 s=CLOSED pgs=284 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:19:32.496 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:32.491+0000 7fa06ac9a640 1 --2- 172.21.15.143:0/3187878108 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fa064143ac0 0x7fa064156350 unknown :-1 s=CLOSED pgs=562 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:19:32.496 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:32.491+0000 7fa06ac9a640 1 --2- 172.21.15.143:0/3187878108 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fa0640cf280 0x7fa06413c080 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:19:32.496 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:32.491+0000 7fa06ac9a640 1 --2- 172.21.15.143:0/3187878108 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fa0640cd1b0 0x7fa0640adfa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:19:32.497 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:32.491+0000 7fa06ac9a640 1 -- 172.21.15.143:0/3187878108 >> 172.21.15.143:0/3187878108 conn(0x7fa06406a890 msgr2=0x7fa06415a8d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:19:32.497 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:32.491+0000 7fa06ac9a640 1 -- 172.21.15.143:0/3187878108 shutdown_connections 2023-11-26T07:19:32.497 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:32.491+0000 7fa06ac9a640 1 -- 172.21.15.143:0/3187878108 wait complete. 2023-11-26T07:19:32.515 DEBUG:teuthology.orchestra.run.smithi143:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-11-26T07:19:32.654 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:32.651+0000 7fa3996ea640 1 Processor -- start 2023-11-26T07:19:32.654 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:32.651+0000 7fa3996ea640 1 -- start start 2023-11-26T07:19:32.655 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:32.651+0000 7fa3996ea640 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7fa3940d3850 0x7fa3940afed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:19:32.655 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:32.651+0000 7fa3996ea640 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7fa39415d530 0x7fa3940b0410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:19:32.655 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:32.651+0000 7fa3996ea640 1 -- --> v1:172.21.15.188:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa3940b0b20 con 0x7fa39415d150 2023-11-26T07:19:32.655 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:32.651+0000 7fa3996ea640 1 -- --> v2:172.21.15.143:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa3940d0bf0 con 0x7fa3940d3850 2023-11-26T07:19:32.655 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:32.651+0000 7fa3996ea640 1 -- --> v2:172.21.15.188:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa3940d1a20 con 0x7fa39415d530 2023-11-26T07:19:32.655 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:32.651+0000 7fa38bfff640 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7fa3940d3850 0x7fa3940afed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:19:32.656 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:32.651+0000 7fa38bfff640 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7fa3940d3850 0x7fa3940afed0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:40654/0 (socket says 172.21.15.143:40654) 2023-11-26T07:19:32.656 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:32.651+0000 7fa38bfff640 1 -- 172.21.15.143:0/768820521 learned_addr learned my addr 172.21.15.143:0/768820521 (peer_addr_for_me v2:172.21.15.143:0/0) 2023-11-26T07:19:32.656 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:32.651+0000 7fa3937fe640 1 --2- 172.21.15.143:0/768820521 >> v2:172.21.15.188:3300/0 conn(0x7fa39415d530 0x7fa3940b0410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:19:32.656 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:32.651+0000 7fa38bfff640 1 -- 172.21.15.143:0/768820521 >> v2:172.21.15.188:3300/0 conn(0x7fa39415d530 msgr2=0x7fa3940b0410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-26T07:19:32.656 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:32.651+0000 7fa38bfff640 1 --2- 172.21.15.143:0/768820521 >> v2:172.21.15.188:3300/0 conn(0x7fa39415d530 0x7fa3940b0410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:19:32.656 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:32.651+0000 7fa38bfff640 1 -- 172.21.15.143:0/768820521 >> v1:172.21.15.188:6789/0 conn(0x7fa39415d150 legacy=0x7fa3940d3140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:19:32.657 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:32.651+0000 7fa38bfff640 1 -- 172.21.15.143:0/768820521 --> v2:172.21.15.143:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa3940b0e00 con 0x7fa3940d3850 2023-11-26T07:19:32.658 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:32.655+0000 7fa38bfff640 1 --2- 172.21.15.143:0/768820521 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fa3940d3850 0x7fa3940afed0 secure :-1 s=READY pgs=563 cs=0 l=1 rev1=1 crypto rx=0x7fa37c0098a0 tx=0x7fa37c01ecd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2bfbe58a9d46e9f3 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:19:32.659 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:32.655+0000 7fa3927fc640 1 -- 172.21.15.143:0/768820521 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa37c0a89d0 con 0x7fa3940d3850 2023-11-26T07:19:32.659 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:32.655+0000 7fa3927fc640 1 -- 172.21.15.143:0/768820521 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa37c0a8b30 con 0x7fa3940d3850 2023-11-26T07:19:32.660 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:32.655+0000 7fa3927fc640 1 -- 172.21.15.143:0/768820521 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa37c0b0440 con 0x7fa3940d3850 2023-11-26T07:19:32.660 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:32.655+0000 7fa3996ea640 1 -- 172.21.15.143:0/768820521 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fa3940d3850 msgr2=0x7fa3940afed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:19:32.660 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:32.655+0000 7fa3996ea640 1 --2- 172.21.15.143:0/768820521 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fa3940d3850 0x7fa3940afed0 secure :-1 s=READY pgs=563 cs=0 l=1 rev1=1 crypto rx=0x7fa37c0098a0 tx=0x7fa37c01ecd0 comp rx=0 tx=0).stop 2023-11-26T07:19:32.660 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:32.655+0000 7fa3996ea640 1 -- 172.21.15.143:0/768820521 shutdown_connections 2023-11-26T07:19:32.660 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:32.655+0000 7fa3996ea640 1 --2- 172.21.15.143:0/768820521 >> v2:172.21.15.188:3300/0 conn(0x7fa39415d530 0x7fa3940b0410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:19:32.660 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:32.655+0000 7fa3996ea640 1 --2- 172.21.15.143:0/768820521 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fa3940d3850 0x7fa3940afed0 unknown :-1 s=CLOSED pgs=563 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:19:32.660 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:32.655+0000 7fa3996ea640 1 -- 172.21.15.143:0/768820521 >> 172.21.15.143:0/768820521 conn(0x7fa39408ccc0 msgr2=0x7fa39409c5e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:19:32.660 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:32.655+0000 7fa3996ea640 1 -- 172.21.15.143:0/768820521 shutdown_connections 2023-11-26T07:19:32.660 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:32.655+0000 7fa3996ea640 1 -- 172.21.15.143:0/768820521 wait complete. 2023-11-26T07:19:32.661 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:32.659+0000 7fa3996ea640 1 Processor -- start 2023-11-26T07:19:32.661 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:32.659+0000 7fa3996ea640 1 -- start start 2023-11-26T07:19:32.664 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:32.659+0000 7fa3996ea640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fa3940d3850 0x7fa39415e4f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:19:32.664 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:32.659+0000 7fa3996ea640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fa39415d150 0x7fa3940b6c10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:19:32.664 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:32.659+0000 7fa3996ea640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fa39415d530 0x7fa39415b5f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:19:32.664 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:32.659+0000 7fa3996ea640 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa39415f640 con 0x7fa39415d150 2023-11-26T07:19:32.664 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:32.659+0000 7fa3996ea640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa3940b7150 con 0x7fa39415d530 2023-11-26T07:19:32.664 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:32.659+0000 7fa3996ea640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa3940b7420 con 0x7fa3940d3850 2023-11-26T07:19:32.664 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:32.659+0000 7fa38bfff640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fa39415d150 0x7fa3940b6c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:19:32.664 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:32.659+0000 7fa38bfff640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fa39415d150 0x7fa3940b6c10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:40662/0 (socket says 172.21.15.143:40662) 2023-11-26T07:19:32.664 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:32.659+0000 7fa38bfff640 1 -- 172.21.15.143:0/2953044383 learned_addr learned my addr 172.21.15.143:0/2953044383 (peer_addr_for_me v2:172.21.15.143:0/0) 2023-11-26T07:19:32.665 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:32.659+0000 7fa3937fe640 1 --2- 172.21.15.143:0/2953044383 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fa39415d530 0x7fa39415b5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:19:32.665 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:32.659+0000 7fa392ffd640 1 --2- 172.21.15.143:0/2953044383 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fa3940d3850 0x7fa39415e4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:19:32.665 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:32.659+0000 7fa38bfff640 1 -- 172.21.15.143:0/2953044383 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fa3940d3850 msgr2=0x7fa39415e4f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:19:32.665 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:32.659+0000 7fa38bfff640 1 --2- 172.21.15.143:0/2953044383 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fa3940d3850 0x7fa39415e4f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:19:32.665 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:32.659+0000 7fa38bfff640 1 -- 172.21.15.143:0/2953044383 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fa39415d530 msgr2=0x7fa39415b5f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:19:32.665 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:32.659+0000 7fa38bfff640 1 --2- 172.21.15.143:0/2953044383 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fa39415d530 0x7fa39415b5f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:19:32.665 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:32.659+0000 7fa38bfff640 1 -- 172.21.15.143:0/2953044383 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa37c0094d0 con 0x7fa39415d150 2023-11-26T07:19:32.665 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:32.659+0000 7fa392ffd640 1 --2- 172.21.15.143:0/2953044383 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fa3940d3850 0x7fa39415e4f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-26T07:19:32.665 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:32.659+0000 7fa38bfff640 1 --2- 172.21.15.143:0/2953044383 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fa39415d150 0x7fa3940b6c10 secure :-1 s=READY pgs=564 cs=0 l=1 rev1=1 crypto rx=0x7fa37c0b3040 tx=0x7fa37c01ba70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:19:32.665 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:32.659+0000 7fa38b7fe640 1 -- 172.21.15.143:0/2953044383 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa37c0be070 con 0x7fa39415d150 2023-11-26T07:19:32.666 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:32.659+0000 7fa38b7fe640 1 -- 172.21.15.143:0/2953044383 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa37c0ba400 con 0x7fa39415d150 2023-11-26T07:19:32.666 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:32.659+0000 7fa38b7fe640 1 -- 172.21.15.143:0/2953044383 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa37c0ba7d0 con 0x7fa39415d150 2023-11-26T07:19:32.666 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:32.659+0000 7fa3996ea640 1 -- 172.21.15.143:0/2953044383 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa39415eff0 con 0x7fa39415d150 2023-11-26T07:19:32.666 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:32.659+0000 7fa3996ea640 1 -- 172.21.15.143:0/2953044383 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa3940b7700 con 0x7fa39415d150 2023-11-26T07:19:32.667 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:32.663+0000 7fa3996ea640 1 -- 172.21.15.143:0/2953044383 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa394067500 con 0x7fa39415d150 2023-11-26T07:19:32.667 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:32.663+0000 7fa38b7fe640 1 -- 172.21.15.143:0/2953044383 <== mon.0 v2:172.21.15.143:3300/0 4 ==== mgrmap(e 7) v1 ==== 86005+0+0 (secure 0 0 0) 0x7fa37c0c3410 con 0x7fa39415d150 2023-11-26T07:19:32.667 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:32.663+0000 7fa38b7fe640 1 --2- 172.21.15.143:0/2953044383 >> [v2:172.21.15.143:6832/2150557544,v1:172.21.15.143:6833/2150557544] conn(0x7fa374066620 0x7fa374068ae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:19:32.667 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:32.663+0000 7fa38b7fe640 1 -- 172.21.15.143:0/2953044383 <== mon.0 v2:172.21.15.143:3300/0 5 ==== osd_map(44..44 src has 1..44) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fa37c121c40 con 0x7fa39415d150 2023-11-26T07:19:32.667 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:32.663+0000 7fa392ffd640 1 --2- 172.21.15.143:0/2953044383 >> [v2:172.21.15.143:6832/2150557544,v1:172.21.15.143:6833/2150557544] conn(0x7fa374066620 0x7fa374068ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:19:32.668 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:32.663+0000 7fa392ffd640 1 --2- 172.21.15.143:0/2953044383 >> [v2:172.21.15.143:6832/2150557544,v1:172.21.15.143:6833/2150557544] conn(0x7fa374066620 0x7fa374068ae0 secure :-1 s=READY pgs=285 cs=0 l=1 rev1=1 crypto rx=0x7fa384000f70 tx=0x7fa384009040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:19:32.670 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:32.667+0000 7fa38b7fe640 1 -- 172.21.15.143:0/2953044383 <== mon.0 v2:172.21.15.143:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172649 (secure 0 0 0) 0x7fa37c0eeeb0 con 0x7fa39415d150 2023-11-26T07:19:32.822 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:32.819+0000 7fa3996ea640 1 -- 172.21.15.143:0/2953044383 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fa39406bae0 con 0x7fa39415d150 2023-11-26T07:19:32.823 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:32.819+0000 7fa38b7fe640 1 -- 172.21.15.143:0/2953044383 <== mon.0 v2:172.21.15.143:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v44) v1 ==== 74+0+16562 (secure 0 0 0) 0x7fa37c0b5090 con 0x7fa39415d150 2023-11-26T07:19:32.824 INFO:teuthology.orchestra.run.smithi143.stdout: 2023-11-26T07:19:32.824 INFO:teuthology.orchestra.run.smithi143.stdout:{"epoch":44,"fsid":"f5033060-2469-4a07-95fe-2b76f5404bc1","created":"2023-11-26T07:17:28.613663+0000","modified":"2023-11-26T07:19:31.503249+0000","last_up_change":"2023-11-26T07:17:41.157813+0000","last_in_change":"2023-11-26T07:17:31.181658+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":7,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-11-26T07:17:41.758597+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":6,"pool_name":"cephfs_metadata","create_time":"2023-11-26T07:19:29.208075+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"44","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":7,"pool_name":"cephfs_data","create_time":"2023-11-26T07:19:30.503743+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"44","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"ebbee2c6-023e-44b9-9f1d-6f7bfd309730","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":43,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6817","nonce":639807369},{"type":"v1","addr":"172.21.15.143:6819","nonce":639807369}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6821","nonce":639807369},{"type":"v1","addr":"172.21.15.143:6823","nonce":639807369}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6830","nonce":639807369},{"type":"v1","addr":"172.21.15.143:6831","nonce":639807369}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6825","nonce":639807369},{"type":"v1","addr":"172.21.15.143:6828","nonce":639807369}]},"public_addr":"172.21.15.143:6819/639807369","cluster_addr":"172.21.15.143:6823/639807369","heartbeat_back_addr":"172.21.15.143:6831/639807369","heartbeat_front_addr":"172.21.15.143:6828/639807369","state":["exists","up"]},{"osd":1,"uuid":"fec4b8be-7122-49e9-9a19-5850a0700001","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":43,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6816","nonce":1144942530},{"type":"v1","addr":"172.21.15.143:6818","nonce":1144942530}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6820","nonce":1144942530},{"type":"v1","addr":"172.21.15.143:6822","nonce":1144942530}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6827","nonce":1144942530},{"type":"v1","addr":"172.21.15.143:6829","nonce":1144942530}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6824","nonce":1144942530},{"type":"v1","addr":"172.21.15.143:6826","nonce":1144942530}]},"public_addr":"172.21.15.143:6818/1144942530","cluster_addr":"172.21.15.143:6822/1144942530","heartbeat_back_addr":"172.21.15.143:6829/1144942530","heartbeat_front_addr":"172.21.15.143:6826/1144942530","state":["exists","up"]},{"osd":2,"uuid":"2888af5b-f933-46ca-8f95-6dd494ff43f6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":43,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6808","nonce":3909095186},{"type":"v1","addr":"172.21.15.143:6809","nonce":3909095186}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6810","nonce":3909095186},{"type":"v1","addr":"172.21.15.143:6811","nonce":3909095186}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6814","nonce":3909095186},{"type":"v1","addr":"172.21.15.143:6815","nonce":3909095186}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6812","nonce":3909095186},{"type":"v1","addr":"172.21.15.143:6813","nonce":3909095186}]},"public_addr":"172.21.15.143:6809/3909095186","cluster_addr":"172.21.15.143:6811/3909095186","heartbeat_back_addr":"172.21.15.143:6815/3909095186","heartbeat_front_addr":"172.21.15.143:6813/3909095186","state":["exists","up"]},{"osd":3,"uuid":"d1f52435-7bc1-4081-9550-9cd1ba55f549","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":43,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6800","nonce":3854210991},{"type":"v1","addr":"172.21.15.143:6801","nonce":3854210991}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6802","nonce":3854210991},{"type":"v1","addr":"172.21.15.143:6803","nonce":3854210991}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6806","nonce":3854210991},{"type":"v1","addr":"172.21.15.143:6807","nonce":3854210991}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6804","nonce":3854210991},{"type":"v1","addr":"172.21.15.143:6805","nonce":3854210991}]},"public_addr":"172.21.15.143:6801/3854210991","cluster_addr":"172.21.15.143:6803/3854210991","heartbeat_back_addr":"172.21.15.143:6807/3854210991","heartbeat_front_addr":"172.21.15.143:6805/3854210991","state":["exists","up"]},{"osd":4,"uuid":"e2ae8b6e-6e25-49b2-bcbf-aacce68f06a3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":43,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6800","nonce":3924606902},{"type":"v1","addr":"172.21.15.188:6801","nonce":3924606902}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6802","nonce":3924606902},{"type":"v1","addr":"172.21.15.188:6803","nonce":3924606902}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6806","nonce":3924606902},{"type":"v1","addr":"172.21.15.188:6807","nonce":3924606902}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6804","nonce":3924606902},{"type":"v1","addr":"172.21.15.188:6805","nonce":3924606902}]},"public_addr":"172.21.15.188:6801/3924606902","cluster_addr":"172.21.15.188:6803/3924606902","heartbeat_back_addr":"172.21.15.188:6807/3924606902","heartbeat_front_addr":"172.21.15.188:6805/3924606902","state":["exists","up"]},{"osd":5,"uuid":"3cec8a32-7698-4c36-8e89-e4ba11934f33","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":43,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6813","nonce":186841544},{"type":"v1","addr":"172.21.15.188:6815","nonce":186841544}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6817","nonce":186841544},{"type":"v1","addr":"172.21.15.188:6818","nonce":186841544}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6822","nonce":186841544},{"type":"v1","addr":"172.21.15.188:6823","nonce":186841544}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6820","nonce":186841544},{"type":"v1","addr":"172.21.15.188:6821","nonce":186841544}]},"public_addr":"172.21.15.188:6815/186841544","cluster_addr":"172.21.15.188:6818/186841544","heartbeat_back_addr":"172.21.15.188:6823/186841544","heartbeat_front_addr":"172.21.15.188:6821/186841544","state":["exists","up"]},{"osd":6,"uuid":"58a65d5d-e090-473a-81b4-ad38e85029f9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":43,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6808","nonce":2153499405},{"type":"v1","addr":"172.21.15.188:6809","nonce":2153499405}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6810","nonce":2153499405},{"type":"v1","addr":"172.21.15.188:6811","nonce":2153499405}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6816","nonce":2153499405},{"type":"v1","addr":"172.21.15.188:6819","nonce":2153499405}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6812","nonce":2153499405},{"type":"v1","addr":"172.21.15.188:6814","nonce":2153499405}]},"public_addr":"172.21.15.188:6809/2153499405","cluster_addr":"172.21.15.188:6811/2153499405","heartbeat_back_addr":"172.21.15.188:6819/2153499405","heartbeat_front_addr":"172.21.15.188:6814/2153499405","state":["exists","up"]},{"osd":7,"uuid":"ea5b7f70-3554-4cf8-ac30-f7c59b80678d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":43,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6824","nonce":2722096729},{"type":"v1","addr":"172.21.15.188:6825","nonce":2722096729}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6826","nonce":2722096729},{"type":"v1","addr":"172.21.15.188:6827","nonce":2722096729}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6830","nonce":2722096729},{"type":"v1","addr":"172.21.15.188:6831","nonce":2722096729}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6828","nonce":2722096729},{"type":"v1","addr":"172.21.15.188:6829","nonce":2722096729}]},"public_addr":"172.21.15.188:6825/2722096729","cluster_addr":"172.21.15.188:6827/2722096729","heartbeat_back_addr":"172.21.15.188:6831/2722096729","heartbeat_front_addr":"172.21.15.188:6829/2722096729","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-26T07:17:36.995032+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-26T07:17:36.932266+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-26T07:17:36.829304+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-26T07:17:36.804297+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-26T07:17:36.667892+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-26T07:17:36.964868+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-26T07:17:36.827332+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-26T07:17:36.893179+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-11-26T07:19:32.826 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:32.823+0000 7fa3996ea640 1 -- 172.21.15.143:0/2953044383 >> [v2:172.21.15.143:6832/2150557544,v1:172.21.15.143:6833/2150557544] conn(0x7fa374066620 msgr2=0x7fa374068ae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:19:32.826 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:32.823+0000 7fa3996ea640 1 --2- 172.21.15.143:0/2953044383 >> [v2:172.21.15.143:6832/2150557544,v1:172.21.15.143:6833/2150557544] conn(0x7fa374066620 0x7fa374068ae0 secure :-1 s=READY pgs=285 cs=0 l=1 rev1=1 crypto rx=0x7fa384000f70 tx=0x7fa384009040 comp rx=0 tx=0).stop 2023-11-26T07:19:32.826 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:32.823+0000 7fa3996ea640 1 -- 172.21.15.143:0/2953044383 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fa39415d150 msgr2=0x7fa3940b6c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:19:32.826 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:32.823+0000 7fa3996ea640 1 --2- 172.21.15.143:0/2953044383 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fa39415d150 0x7fa3940b6c10 secure :-1 s=READY pgs=564 cs=0 l=1 rev1=1 crypto rx=0x7fa37c0b3040 tx=0x7fa37c01ba70 comp rx=0 tx=0).stop 2023-11-26T07:19:32.826 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:32.823+0000 7fa3996ea640 1 -- 172.21.15.143:0/2953044383 shutdown_connections 2023-11-26T07:19:32.826 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:32.823+0000 7fa3996ea640 1 --2- 172.21.15.143:0/2953044383 >> [v2:172.21.15.143:6832/2150557544,v1:172.21.15.143:6833/2150557544] conn(0x7fa374066620 0x7fa374068ae0 unknown :-1 s=CLOSED pgs=285 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:19:32.827 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:32.823+0000 7fa3996ea640 1 --2- 172.21.15.143:0/2953044383 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fa39415d530 0x7fa39415b5f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:19:32.827 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:32.823+0000 7fa3996ea640 1 --2- 172.21.15.143:0/2953044383 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fa39415d150 0x7fa3940b6c10 unknown :-1 s=CLOSED pgs=564 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:19:32.827 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:32.823+0000 7fa3996ea640 1 --2- 172.21.15.143:0/2953044383 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fa3940d3850 0x7fa39415e4f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:19:32.827 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:32.823+0000 7fa3996ea640 1 -- 172.21.15.143:0/2953044383 >> 172.21.15.143:0/2953044383 conn(0x7fa39408ccc0 msgr2=0x7fa3940d3140 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:19:32.827 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:32.823+0000 7fa3996ea640 1 -- 172.21.15.143:0/2953044383 shutdown_connections 2023-11-26T07:19:32.827 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:32.823+0000 7fa3996ea640 1 -- 172.21.15.143:0/2953044383 wait complete. 2023-11-26T07:19:32.845 DEBUG:teuthology.orchestra.run.smithi143:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs standby_count_wanted 0 2023-11-26T07:19:32.986 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:32.983+0000 7f232825a640 1 Processor -- start 2023-11-26T07:19:32.986 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:32.983+0000 7f232825a640 1 -- start start 2023-11-26T07:19:32.988 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:32.983+0000 7f232825a640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f23201596d0 0x7f2320071850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:19:32.988 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:32.983+0000 7f232825a640 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7f23200b2f80 0x7f232006cbe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:19:32.988 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:32.983+0000 7f232825a640 1 -- --> v1:172.21.15.188:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f232006c750 con 0x7f23200ce680 2023-11-26T07:19:32.988 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:32.983+0000 7f232825a640 1 -- --> v2:172.21.15.143:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f23200cbe00 con 0x7f23200b2f80 2023-11-26T07:19:32.988 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:32.983+0000 7f232825a640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f23200b3550 con 0x7f23201596d0 2023-11-26T07:19:32.988 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:32.983+0000 7f23267f7640 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7f23200b2f80 0x7f232006cbe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:19:32.988 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:32.983+0000 7f23267f7640 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7f23200b2f80 0x7f232006cbe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:40668/0 (socket says 172.21.15.143:40668) 2023-11-26T07:19:32.988 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:32.983+0000 7f23267f7640 1 -- 172.21.15.143:0/916949536 learned_addr learned my addr 172.21.15.143:0/916949536 (peer_addr_for_me v2:172.21.15.143:0/0) 2023-11-26T07:19:32.989 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:32.983+0000 7f2325ff6640 1 --2- 172.21.15.143:0/916949536 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f23201596d0 0x7f2320071850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:19:32.989 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:32.983+0000 7f23267f7640 1 -- 172.21.15.143:0/916949536 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f23201596d0 msgr2=0x7f2320071850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-26T07:19:32.989 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:32.983+0000 7f23267f7640 1 --2- 172.21.15.143:0/916949536 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f23201596d0 0x7f2320071850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:19:32.989 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:32.983+0000 7f23267f7640 1 -- 172.21.15.143:0/916949536 >> v1:172.21.15.188:6789/0 conn(0x7f23200ce680 legacy=0x7f23200b2870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-26T07:19:32.989 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:32.983+0000 7f23267f7640 1 -- 172.21.15.143:0/916949536 --> v2:172.21.15.143:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2320071e80 con 0x7f23200b2f80 2023-11-26T07:19:32.990 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:32.987+0000 7f23267f7640 1 --2- 172.21.15.143:0/916949536 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f23200b2f80 0x7f232006cbe0 secure :-1 s=READY pgs=565 cs=0 l=1 rev1=1 crypto rx=0x7f231800af60 tx=0x7f231801db10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=15dfb83b4df20bca server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:19:32.991 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:32.987+0000 7f2324ff4640 1 -- 172.21.15.143:0/916949536 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f231800a2d0 con 0x7f23200b2f80 2023-11-26T07:19:32.991 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:32.987+0000 7f2324ff4640 1 -- 172.21.15.143:0/916949536 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f23180a6840 con 0x7f23200b2f80 2023-11-26T07:19:32.991 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:32.987+0000 7f2324ff4640 1 -- 172.21.15.143:0/916949536 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f23180a6bd0 con 0x7f23200b2f80 2023-11-26T07:19:32.992 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:32.987+0000 7f232825a640 1 -- 172.21.15.143:0/916949536 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f23200b2f80 msgr2=0x7f232006cbe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:19:32.992 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:32.987+0000 7f232825a640 1 --2- 172.21.15.143:0/916949536 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f23200b2f80 0x7f232006cbe0 secure :-1 s=READY pgs=565 cs=0 l=1 rev1=1 crypto rx=0x7f231800af60 tx=0x7f231801db10 comp rx=0 tx=0).stop 2023-11-26T07:19:32.992 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:32.987+0000 7f232825a640 1 -- 172.21.15.143:0/916949536 shutdown_connections 2023-11-26T07:19:32.992 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:32.987+0000 7f232825a640 1 --2- 172.21.15.143:0/916949536 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f23200b2f80 0x7f232006cbe0 unknown :-1 s=CLOSED pgs=565 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:19:32.992 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:32.987+0000 7f232825a640 1 --2- 172.21.15.143:0/916949536 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f23201596d0 0x7f2320071850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:19:32.992 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:32.987+0000 7f232825a640 1 -- 172.21.15.143:0/916949536 >> 172.21.15.143:0/916949536 conn(0x7f232006a890 msgr2=0x7f232006aca0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:19:32.993 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:32.987+0000 7f232825a640 1 -- 172.21.15.143:0/916949536 shutdown_connections 2023-11-26T07:19:32.993 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:32.987+0000 7f232825a640 1 -- 172.21.15.143:0/916949536 wait complete. 2023-11-26T07:19:32.993 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:32.991+0000 7f232825a640 1 Processor -- start 2023-11-26T07:19:32.994 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:32.991+0000 7f232825a640 1 -- start start 2023-11-26T07:19:32.996 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:32.991+0000 7f232825a640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f23200b2f80 0x7f2320153f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:19:32.996 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:32.991+0000 7f232825a640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f23200ce680 0x7f23201f5a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:19:32.996 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:32.991+0000 7f232825a640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f23201596d0 0x7f2320154d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:19:32.996 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:32.991+0000 7f232825a640 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f232015c4c0 con 0x7f23201596d0 2023-11-26T07:19:32.997 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:32.991+0000 7f232825a640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f232015c030 con 0x7f23200ce680 2023-11-26T07:19:32.997 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:32.991+0000 7f232825a640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f232015b420 con 0x7f23200b2f80 2023-11-26T07:19:32.997 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:32.991+0000 7f23267f7640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f23201596d0 0x7f2320154d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:19:32.997 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:32.991+0000 7f23267f7640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f23201596d0 0x7f2320154d90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:40680/0 (socket says 172.21.15.143:40680) 2023-11-26T07:19:32.997 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:32.991+0000 7f23267f7640 1 -- 172.21.15.143:0/15873583 learned_addr learned my addr 172.21.15.143:0/15873583 (peer_addr_for_me v2:172.21.15.143:0/0) 2023-11-26T07:19:32.997 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:32.991+0000 7f2325ff6640 1 --2- 172.21.15.143:0/15873583 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f23200b2f80 0x7f2320153f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:19:32.997 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:32.991+0000 7f23257f5640 1 --2- 172.21.15.143:0/15873583 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f23200ce680 0x7f23201f5a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:19:32.997 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:32.991+0000 7f23267f7640 1 -- 172.21.15.143:0/15873583 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f23200b2f80 msgr2=0x7f2320153f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:19:32.997 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:32.991+0000 7f23267f7640 1 --2- 172.21.15.143:0/15873583 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f23200b2f80 0x7f2320153f20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:19:32.997 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:32.991+0000 7f23267f7640 1 -- 172.21.15.143:0/15873583 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f23200ce680 msgr2=0x7f23201f5a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:19:32.998 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:32.991+0000 7f23267f7640 1 --2- 172.21.15.143:0/15873583 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f23200ce680 0x7f23201f5a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:19:32.998 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:32.991+0000 7f23267f7640 1 -- 172.21.15.143:0/15873583 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f231800ac40 con 0x7f23201596d0 2023-11-26T07:19:32.998 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:32.991+0000 7f23267f7640 1 --2- 172.21.15.143:0/15873583 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f23201596d0 0x7f2320154d90 secure :-1 s=READY pgs=566 cs=0 l=1 rev1=1 crypto rx=0x7f23180b0040 tx=0x7f231801a8b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:19:32.998 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:32.991+0000 7f23167fc640 1 -- 172.21.15.143:0/15873583 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2318008700 con 0x7f23201596d0 2023-11-26T07:19:32.998 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:32.991+0000 7f23167fc640 1 -- 172.21.15.143:0/15873583 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f23180088c0 con 0x7f23201596d0 2023-11-26T07:19:32.998 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:32.991+0000 7f23167fc640 1 -- 172.21.15.143:0/15873583 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2318008c70 con 0x7f23201596d0 2023-11-26T07:19:32.998 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:32.991+0000 7f232825a640 1 -- 172.21.15.143:0/15873583 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f232015b700 con 0x7f23201596d0 2023-11-26T07:19:32.998 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:32.991+0000 7f232825a640 1 -- 172.21.15.143:0/15873583 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f232015bcc0 con 0x7f23201596d0 2023-11-26T07:19:32.999 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:32.995+0000 7f232825a640 1 -- 172.21.15.143:0/15873583 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2320067680 con 0x7f23201596d0 2023-11-26T07:19:33.000 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:32.995+0000 7f23167fc640 1 -- 172.21.15.143:0/15873583 <== mon.0 v2:172.21.15.143:3300/0 4 ==== mgrmap(e 7) v1 ==== 86005+0+0 (secure 0 0 0) 0x7f23180b7480 con 0x7f23201596d0 2023-11-26T07:19:33.000 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:32.995+0000 7f23167fc640 1 --2- 172.21.15.143:0/15873583 >> [v2:172.21.15.143:6832/2150557544,v1:172.21.15.143:6833/2150557544] conn(0x7f2304066620 0x7f2304068ae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:19:33.000 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:32.995+0000 7f23167fc640 1 -- 172.21.15.143:0/15873583 <== mon.0 v2:172.21.15.143:3300/0 5 ==== osd_map(44..44 src has 1..44) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f231811f580 con 0x7f23201596d0 2023-11-26T07:19:33.000 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:32.995+0000 7f2325ff6640 1 --2- 172.21.15.143:0/15873583 >> [v2:172.21.15.143:6832/2150557544,v1:172.21.15.143:6833/2150557544] conn(0x7f2304066620 0x7f2304068ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:19:33.000 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:32.995+0000 7f2325ff6640 1 --2- 172.21.15.143:0/15873583 >> [v2:172.21.15.143:6832/2150557544,v1:172.21.15.143:6833/2150557544] conn(0x7f2304066620 0x7f2304068ae0 secure :-1 s=READY pgs=286 cs=0 l=1 rev1=1 crypto rx=0x7f231c0079c0 tx=0x7f231c008040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:19:33.003 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:32.999+0000 7f23167fc640 1 -- 172.21.15.143:0/15873583 <== mon.0 v2:172.21.15.143:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172649 (secure 0 0 0) 0x7f23180eca00 con 0x7f23201596d0 2023-11-26T07:19:33.191 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:33.187+0000 7f232825a640 1 -- 172.21.15.143:0/15873583 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"} v 0) v1 -- 0x7f23200709f0 con 0x7f23201596d0 2023-11-26T07:19:34.149 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:34.147+0000 7f23167fc640 1 -- 172.21.15.143:0/15873583 <== mon.0 v2:172.21.15.143:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]=0 v24) v1 ==== 118+0+0 (secure 0 0 0) 0x7f23180f0020 con 0x7f23201596d0 2023-11-26T07:19:34.153 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:34.147+0000 7f232825a640 1 -- 172.21.15.143:0/15873583 >> [v2:172.21.15.143:6832/2150557544,v1:172.21.15.143:6833/2150557544] conn(0x7f2304066620 msgr2=0x7f2304068ae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:19:34.153 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:34.147+0000 7f232825a640 1 --2- 172.21.15.143:0/15873583 >> [v2:172.21.15.143:6832/2150557544,v1:172.21.15.143:6833/2150557544] conn(0x7f2304066620 0x7f2304068ae0 secure :-1 s=READY pgs=286 cs=0 l=1 rev1=1 crypto rx=0x7f231c0079c0 tx=0x7f231c008040 comp rx=0 tx=0).stop 2023-11-26T07:19:34.153 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:34.147+0000 7f232825a640 1 -- 172.21.15.143:0/15873583 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f23201596d0 msgr2=0x7f2320154d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:19:34.153 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:34.147+0000 7f232825a640 1 --2- 172.21.15.143:0/15873583 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f23201596d0 0x7f2320154d90 secure :-1 s=READY pgs=566 cs=0 l=1 rev1=1 crypto rx=0x7f23180b0040 tx=0x7f231801a8b0 comp rx=0 tx=0).stop 2023-11-26T07:19:34.153 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:34.147+0000 7f232825a640 1 -- 172.21.15.143:0/15873583 shutdown_connections 2023-11-26T07:19:34.153 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:34.147+0000 7f232825a640 1 --2- 172.21.15.143:0/15873583 >> [v2:172.21.15.143:6832/2150557544,v1:172.21.15.143:6833/2150557544] conn(0x7f2304066620 0x7f2304068ae0 unknown :-1 s=CLOSED pgs=286 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:19:34.153 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:34.147+0000 7f232825a640 1 --2- 172.21.15.143:0/15873583 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f23201596d0 0x7f2320154d90 unknown :-1 s=CLOSED pgs=566 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:19:34.153 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:34.147+0000 7f232825a640 1 --2- 172.21.15.143:0/15873583 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f23200ce680 0x7f23201f5a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:19:34.153 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:34.147+0000 7f232825a640 1 --2- 172.21.15.143:0/15873583 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f23200b2f80 0x7f2320153f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:19:34.153 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:34.147+0000 7f232825a640 1 -- 172.21.15.143:0/15873583 >> 172.21.15.143:0/15873583 conn(0x7f232006a890 msgr2=0x7f2320071620 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:19:34.153 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:34.147+0000 7f232825a640 1 -- 172.21.15.143:0/15873583 shutdown_connections 2023-11-26T07:19:34.154 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:34.147+0000 7f232825a640 1 -- 172.21.15.143:0/15873583 wait complete. 2023-11-26T07:19:34.175 DEBUG:tasks.cephfs.filesystem:fs_config: {} 2023-11-26T07:19:34.175 DEBUG:teuthology.orchestra.run.smithi143:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs allow_standby_replay false 2023-11-26T07:19:34.320 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:34.315+0000 7eff5d016640 1 Processor -- start 2023-11-26T07:19:34.320 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:34.315+0000 7eff5d016640 1 -- start start 2023-11-26T07:19:34.321 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:34.319+0000 7eff5d016640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7eff580cf710 0x7eff580cd8c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:19:34.322 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:34.319+0000 7eff5d016640 1 -- --> v1:172.21.15.143:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7eff580cc730 con 0x7eff580cf280 2023-11-26T07:19:34.322 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:34.319+0000 7eff5d016640 1 -- --> v1:172.21.15.188:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7eff58070ee0 con 0x7eff58143ac0 2023-11-26T07:19:34.322 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:34.319+0000 7eff5d016640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7eff58143ea0 con 0x7eff580cf710 2023-11-26T07:19:34.322 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:34.319+0000 7eff4ffff640 1 --1- >> v1:172.21.15.143:6789/0 conn(0x7eff580cf280 0x7eff580cd1b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.143:6789/0 says I am v1:172.21.15.143:58392/0 (socket says 172.21.15.143:58392) 2023-11-26T07:19:34.322 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:34.319+0000 7eff4ffff640 1 -- 172.21.15.143:0/1746604434 learned_addr learned my addr 172.21.15.143:0/1746604434 (peer_addr_for_me v1:172.21.15.143:0/0) 2023-11-26T07:19:34.324 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:34.319+0000 7eff5759e640 1 --2- 172.21.15.143:0/1746604434 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7eff580cf710 0x7eff580cd8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:19:34.324 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:34.319+0000 7eff5659c640 1 -- 172.21.15.143:0/1746604434 <== mon.0 v1:172.21.15.143:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2151075215 0 0) 0x7eff580cc730 con 0x7eff580cf280 2023-11-26T07:19:34.324 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:34.319+0000 7eff5659c640 1 -- 172.21.15.143:0/1746604434 --> v1:172.21.15.143:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7eff44005740 con 0x7eff580cf280 2023-11-26T07:19:34.324 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:34.319+0000 7eff5659c640 1 -- 172.21.15.143:0/1746604434 <== mon.1 v1:172.21.15.188:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3732706705 0 0) 0x7eff58070ee0 con 0x7eff58143ac0 2023-11-26T07:19:34.324 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:34.319+0000 7eff5659c640 1 -- 172.21.15.143:0/1746604434 --> v1:172.21.15.188:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7eff580cc730 con 0x7eff58143ac0 2023-11-26T07:19:34.324 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:34.319+0000 7eff5659c640 1 -- 172.21.15.143:0/1746604434 <== mon.0 v1:172.21.15.143:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3728154085 0 0) 0x7eff44005740 con 0x7eff580cf280 2023-11-26T07:19:34.324 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:34.319+0000 7eff5659c640 1 -- 172.21.15.143:0/1746604434 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7eff580cf710 msgr2=0x7eff580cd8c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-26T07:19:34.324 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:34.319+0000 7eff5659c640 1 --2- 172.21.15.143:0/1746604434 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7eff580cf710 0x7eff580cd8c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:19:34.325 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:34.319+0000 7eff5659c640 1 -- 172.21.15.143:0/1746604434 >> v1:172.21.15.188:6789/0 conn(0x7eff58143ac0 legacy=0x7eff58070370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:19:34.325 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:34.319+0000 7eff5659c640 1 -- 172.21.15.143:0/1746604434 --> v1:172.21.15.143:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7eff580711c0 con 0x7eff580cf280 2023-11-26T07:19:34.325 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:34.319+0000 7eff5659c640 1 -- 172.21.15.143:0/1746604434 <== mon.0 v1:172.21.15.143:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2137882523 0 0) 0x7eff400016a0 con 0x7eff580cf280 2023-11-26T07:19:34.325 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:34.319+0000 7eff5659c640 1 -- 172.21.15.143:0/1746604434 >> v1:172.21.15.143:6789/0 conn(0x7eff580cf280 legacy=0x7eff580cd1b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:19:34.325 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:34.319+0000 7eff5659c640 1 --2- 172.21.15.143:0/1746604434 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7eff44005d50 0x7eff44006150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:19:34.325 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:34.319+0000 7eff5659c640 1 --2- 172.21.15.143:0/1746604434 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7eff440069e0 0x7eff44006e60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:19:34.325 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:34.319+0000 7eff5659c640 1 --2- 172.21.15.143:0/1746604434 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7eff580cf710 0x7eff440073a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:19:34.325 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:34.319+0000 7eff5659c640 1 -- 172.21.15.143:0/1746604434 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x7eff4400dc50 con 0x7eff44005d50 2023-11-26T07:19:34.325 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:34.319+0000 7eff5659c640 1 -- 172.21.15.143:0/1746604434 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7eff4400dd90 con 0x7eff440069e0 2023-11-26T07:19:34.326 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:34.319+0000 7eff5659c640 1 -- 172.21.15.143:0/1746604434 --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7eff4400df50 con 0x7eff580cf710 2023-11-26T07:19:34.326 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:34.319+0000 7eff56d9d640 1 --2- 172.21.15.143:0/1746604434 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7eff44005d50 0x7eff44006150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:19:34.326 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:34.319+0000 7eff5759e640 1 --2- 172.21.15.143:0/1746604434 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7eff580cf710 0x7eff440073a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:19:34.326 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:34.319+0000 7eff4ffff640 1 --2- 172.21.15.143:0/1746604434 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7eff440069e0 0x7eff44006e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:19:34.326 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:34.319+0000 7eff56d9d640 1 -- 172.21.15.143:0/1746604434 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7eff580cf710 msgr2=0x7eff440073a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-26T07:19:34.326 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:34.319+0000 7eff56d9d640 1 --2- 172.21.15.143:0/1746604434 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7eff580cf710 0x7eff440073a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:19:34.326 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:34.319+0000 7eff56d9d640 1 -- 172.21.15.143:0/1746604434 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7eff440069e0 msgr2=0x7eff44006e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-26T07:19:34.326 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:34.319+0000 7eff56d9d640 1 --2- 172.21.15.143:0/1746604434 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7eff440069e0 0x7eff44006e60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:19:34.326 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:34.319+0000 7eff56d9d640 1 -- 172.21.15.143:0/1746604434 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7eff44005740 con 0x7eff44005d50 2023-11-26T07:19:34.327 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:34.323+0000 7eff56d9d640 1 --2- 172.21.15.143:0/1746604434 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7eff44005d50 0x7eff44006150 secure :-1 s=READY pgs=568 cs=0 l=1 rev1=1 crypto rx=0x7eff48007fd0 tx=0x7eff4801c520 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b90c3cc9c0a7bb65 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:19:34.328 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:34.323+0000 7eff5659c640 1 -- 172.21.15.143:0/1746604434 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7eff480052d0 con 0x7eff44005d50 2023-11-26T07:19:34.328 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:34.323+0000 7eff5659c640 1 -- 172.21.15.143:0/1746604434 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7eff480a7070 con 0x7eff44005d50 2023-11-26T07:19:34.328 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:34.323+0000 7eff5659c640 1 -- 172.21.15.143:0/1746604434 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7eff48003600 con 0x7eff44005d50 2023-11-26T07:19:34.328 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:34.323+0000 7eff5d016640 1 -- 172.21.15.143:0/1746604434 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7eff44005d50 msgr2=0x7eff44006150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:19:34.328 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:34.323+0000 7eff5d016640 1 --2- 172.21.15.143:0/1746604434 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7eff44005d50 0x7eff44006150 secure :-1 s=READY pgs=568 cs=0 l=1 rev1=1 crypto rx=0x7eff48007fd0 tx=0x7eff4801c520 comp rx=0 tx=0).stop 2023-11-26T07:19:34.328 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:34.323+0000 7eff5759e640 1 -- 172.21.15.143:0/1746604434 reap_dead start 2023-11-26T07:19:34.328 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:34.323+0000 7eff5d016640 1 -- 172.21.15.143:0/1746604434 shutdown_connections 2023-11-26T07:19:34.328 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:34.323+0000 7eff5d016640 1 -- 172.21.15.143:0/1746604434 >> 172.21.15.143:0/1746604434 conn(0x7eff5806a890 msgr2=0x7eff5806aca0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:19:34.328 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:34.323+0000 7eff5d016640 1 -- 172.21.15.143:0/1746604434 shutdown_connections 2023-11-26T07:19:34.328 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:34.323+0000 7eff5d016640 1 -- 172.21.15.143:0/1746604434 wait complete. 2023-11-26T07:19:34.329 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:34.327+0000 7eff5d016640 1 Processor -- start 2023-11-26T07:19:34.329 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:34.327+0000 7eff5d016640 1 -- start start 2023-11-26T07:19:34.331 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:34.327+0000 7eff5d016640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7eff44005d50 0x7eff581387f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:19:34.332 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:34.327+0000 7eff5d016640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7eff58159840 0x7eff5813fad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:19:34.332 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:34.327+0000 7eff5d016640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7eff580b0a30 0x7eff580ce160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:19:34.332 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:34.327+0000 7eff5d016640 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x7eff580b19f0 con 0x7eff580b0a30 2023-11-26T07:19:34.332 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:34.327+0000 7eff5d016640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7eff580b1430 con 0x7eff58159840 2023-11-26T07:19:34.332 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:34.327+0000 7eff5d016640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7eff580b1710 con 0x7eff44005d50 2023-11-26T07:19:34.332 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:34.327+0000 7eff5759e640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7eff580b0a30 0x7eff580ce160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:19:34.332 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:34.327+0000 7eff5759e640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7eff580b0a30 0x7eff580ce160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:40694/0 (socket says 172.21.15.143:40694) 2023-11-26T07:19:34.332 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:34.327+0000 7eff5759e640 1 -- 172.21.15.143:0/1740763973 learned_addr learned my addr 172.21.15.143:0/1740763973 (peer_addr_for_me v2:172.21.15.143:0/0) 2023-11-26T07:19:34.332 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:34.327+0000 7eff56d9d640 1 --2- 172.21.15.143:0/1740763973 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7eff44005d50 0x7eff581387f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:19:34.332 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:34.327+0000 7eff4ffff640 1 --2- 172.21.15.143:0/1740763973 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7eff58159840 0x7eff5813fad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:19:34.333 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:34.327+0000 7eff5759e640 1 -- 172.21.15.143:0/1740763973 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7eff44005d50 msgr2=0x7eff581387f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:19:34.333 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:34.327+0000 7eff5759e640 1 --2- 172.21.15.143:0/1740763973 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7eff44005d50 0x7eff581387f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:19:34.333 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:34.327+0000 7eff5759e640 1 -- 172.21.15.143:0/1740763973 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7eff58159840 msgr2=0x7eff5813fad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:19:34.333 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:34.327+0000 7eff5759e640 1 --2- 172.21.15.143:0/1740763973 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7eff58159840 0x7eff5813fad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:19:34.333 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:34.327+0000 7eff5759e640 1 -- 172.21.15.143:0/1740763973 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7eff581583a0 con 0x7eff580b0a30 2023-11-26T07:19:34.333 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:34.327+0000 7eff4ffff640 1 --2- 172.21.15.143:0/1740763973 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7eff58159840 0x7eff5813fad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-26T07:19:34.333 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:34.327+0000 7eff5759e640 1 --2- 172.21.15.143:0/1740763973 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7eff580b0a30 0x7eff580ce160 secure :-1 s=READY pgs=569 cs=0 l=1 rev1=1 crypto rx=0x7eff5806e880 tx=0x7eff5000f580 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:19:34.333 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:34.327+0000 7eff4f7fe640 1 -- 172.21.15.143:0/1740763973 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7eff5000e040 con 0x7eff580b0a30 2023-11-26T07:19:34.333 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:34.327+0000 7eff4f7fe640 1 -- 172.21.15.143:0/1740763973 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7eff5000f040 con 0x7eff580b0a30 2023-11-26T07:19:34.333 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:34.327+0000 7eff4f7fe640 1 -- 172.21.15.143:0/1740763973 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7eff50015670 con 0x7eff580b0a30 2023-11-26T07:19:34.333 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:34.327+0000 7eff5d016640 1 -- 172.21.15.143:0/1740763973 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7eff58158680 con 0x7eff580b0a30 2023-11-26T07:19:34.333 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:34.327+0000 7eff5d016640 1 -- 172.21.15.143:0/1740763973 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7eff58140010 con 0x7eff580b0a30 2023-11-26T07:19:34.334 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:34.331+0000 7eff5d016640 1 -- 172.21.15.143:0/1740763973 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7eff58158870 con 0x7eff580b0a30 2023-11-26T07:19:34.334 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:34.331+0000 7eff4f7fe640 1 -- 172.21.15.143:0/1740763973 <== mon.0 v2:172.21.15.143:3300/0 4 ==== mgrmap(e 7) v1 ==== 86005+0+0 (secure 0 0 0) 0x7eff500157d0 con 0x7eff580b0a30 2023-11-26T07:19:34.335 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:34.331+0000 7eff4f7fe640 1 --2- 172.21.15.143:0/1740763973 >> [v2:172.21.15.143:6832/2150557544,v1:172.21.15.143:6833/2150557544] conn(0x7eff38066650 0x7eff38068b10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:19:34.335 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:34.331+0000 7eff4f7fe640 1 -- 172.21.15.143:0/1740763973 <== mon.0 v2:172.21.15.143:3300/0 5 ==== osd_map(45..45 src has 1..45) v4 ==== 5709+0+0 (secure 0 0 0) 0x7eff50084d40 con 0x7eff580b0a30 2023-11-26T07:19:34.335 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:34.331+0000 7eff56d9d640 1 --2- 172.21.15.143:0/1740763973 >> [v2:172.21.15.143:6832/2150557544,v1:172.21.15.143:6833/2150557544] conn(0x7eff38066650 0x7eff38068b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:19:34.336 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:34.331+0000 7eff56d9d640 1 --2- 172.21.15.143:0/1740763973 >> [v2:172.21.15.143:6832/2150557544,v1:172.21.15.143:6833/2150557544] conn(0x7eff38066650 0x7eff38068b10 secure :-1 s=READY pgs=287 cs=0 l=1 rev1=1 crypto rx=0x7eff48007fd0 tx=0x7eff480a7000 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:19:34.339 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:34.335+0000 7eff4f7fe640 1 -- 172.21.15.143:0/1740763973 <== mon.0 v2:172.21.15.143:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172649 (secure 0 0 0) 0x7eff50051e20 con 0x7eff580b0a30 2023-11-26T07:19:34.523 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:34.519+0000 7eff5d016640 1 -- 172.21.15.143:0/1740763973 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"} v 0) v1 -- 0x7eff58067680 con 0x7eff580b0a30 2023-11-26T07:19:35.444 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:35.439+0000 7eff4f7fe640 1 -- 172.21.15.143:0/1740763973 <== mon.0 v2:172.21.15.143:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"}]=0 v25) v1 ==== 122+0+0 (secure 0 0 0) 0x7eff50055440 con 0x7eff580b0a30 2023-11-26T07:19:35.447 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:35.443+0000 7eff5d016640 1 -- 172.21.15.143:0/1740763973 >> [v2:172.21.15.143:6832/2150557544,v1:172.21.15.143:6833/2150557544] conn(0x7eff38066650 msgr2=0x7eff38068b10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:19:35.447 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:35.443+0000 7eff5d016640 1 --2- 172.21.15.143:0/1740763973 >> [v2:172.21.15.143:6832/2150557544,v1:172.21.15.143:6833/2150557544] conn(0x7eff38066650 0x7eff38068b10 secure :-1 s=READY pgs=287 cs=0 l=1 rev1=1 crypto rx=0x7eff48007fd0 tx=0x7eff480a7000 comp rx=0 tx=0).stop 2023-11-26T07:19:35.447 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:35.443+0000 7eff5d016640 1 -- 172.21.15.143:0/1740763973 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7eff580b0a30 msgr2=0x7eff580ce160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:19:35.447 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:35.443+0000 7eff5d016640 1 --2- 172.21.15.143:0/1740763973 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7eff580b0a30 0x7eff580ce160 secure :-1 s=READY pgs=569 cs=0 l=1 rev1=1 crypto rx=0x7eff5806e880 tx=0x7eff5000f580 comp rx=0 tx=0).stop 2023-11-26T07:19:35.447 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:35.443+0000 7eff5d016640 1 -- 172.21.15.143:0/1740763973 shutdown_connections 2023-11-26T07:19:35.447 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:35.443+0000 7eff5d016640 1 --2- 172.21.15.143:0/1740763973 >> [v2:172.21.15.143:6832/2150557544,v1:172.21.15.143:6833/2150557544] conn(0x7eff38066650 0x7eff38068b10 unknown :-1 s=CLOSED pgs=287 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:19:35.447 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:35.443+0000 7eff5d016640 1 --2- 172.21.15.143:0/1740763973 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7eff580b0a30 0x7eff580ce160 unknown :-1 s=CLOSED pgs=569 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:19:35.447 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:35.443+0000 7eff5d016640 1 --2- 172.21.15.143:0/1740763973 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7eff58159840 0x7eff5813fad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:19:35.447 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:35.443+0000 7eff5d016640 1 --2- 172.21.15.143:0/1740763973 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7eff44005d50 0x7eff581387f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:19:35.448 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:35.443+0000 7eff5d016640 1 -- 172.21.15.143:0/1740763973 >> 172.21.15.143:0/1740763973 conn(0x7eff5806a890 msgr2=0x7eff5813b420 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:19:35.448 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:35.443+0000 7eff5d016640 1 -- 172.21.15.143:0/1740763973 shutdown_connections 2023-11-26T07:19:35.448 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:35.443+0000 7eff5d016640 1 -- 172.21.15.143:0/1740763973 wait complete. 2023-11-26T07:19:35.470 DEBUG:tasks.cephfs.filesystem:Not Creating any subvols for filesystem 'cephfs' 2023-11-26T07:19:35.470 DEBUG:teuthology.orchestra.run.smithi143:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-11-26T07:19:35.611 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:35.607+0000 7f0573ec9640 1 Processor -- start 2023-11-26T07:19:35.611 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:35.607+0000 7f0573ec9640 1 -- start start 2023-11-26T07:19:35.612 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:35.607+0000 7f0573ec9640 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7f056c14b1e0 0x7f056c0cf280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:19:35.612 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:35.607+0000 7f0573ec9640 1 -- --> v1:172.21.15.143:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f056c0cfc20 con 0x7f056c0cf7c0 2023-11-26T07:19:35.612 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:35.607+0000 7f0573ec9640 1 -- --> v1:172.21.15.188:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f056c162080 con 0x7f056c070fd0 2023-11-26T07:19:35.613 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:35.607+0000 7f0573ec9640 1 -- --> v2:172.21.15.188:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f056c161d70 con 0x7f056c14b1e0 2023-11-26T07:19:35.614 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:35.611+0000 7f0571464640 1 --1- >> v1:172.21.15.143:6789/0 conn(0x7f056c0cf7c0 0x7f056c0708c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.143:6789/0 says I am v1:172.21.15.143:41720/0 (socket says 172.21.15.143:41720) 2023-11-26T07:19:35.615 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:35.611+0000 7f0571464640 1 -- 172.21.15.143:0/2091564191 learned_addr learned my addr 172.21.15.143:0/2091564191 (peer_addr_for_me v1:172.21.15.143:0/0) 2023-11-26T07:19:35.615 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:35.611+0000 7f0571c65640 1 --2- 172.21.15.143:0/2091564191 >> v2:172.21.15.188:3300/0 conn(0x7f056c14b1e0 0x7f056c0cf280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:19:35.615 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:35.611+0000 7f0570c63640 1 -- 172.21.15.143:0/2091564191 <== mon.0 v1:172.21.15.143:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2790342520 0 0) 0x7f056c0cfc20 con 0x7f056c0cf7c0 2023-11-26T07:19:35.615 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:35.611+0000 7f0570c63640 1 -- 172.21.15.143:0/2091564191 --> v1:172.21.15.143:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f0558003530 con 0x7f056c0cf7c0 2023-11-26T07:19:35.615 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:35.611+0000 7f0570c63640 1 -- 172.21.15.143:0/2091564191 <== mon.0 v1:172.21.15.143:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 501645849 0 0) 0x7f0558003530 con 0x7f056c0cf7c0 2023-11-26T07:19:35.615 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:35.611+0000 7f0570c63640 1 -- 172.21.15.143:0/2091564191 >> v2:172.21.15.188:3300/0 conn(0x7f056c14b1e0 msgr2=0x7f056c0cf280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-26T07:19:35.615 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:35.611+0000 7f0570c63640 1 --2- 172.21.15.143:0/2091564191 >> v2:172.21.15.188:3300/0 conn(0x7f056c14b1e0 0x7f056c0cf280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:19:35.615 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:35.611+0000 7f0570c63640 1 -- 172.21.15.143:0/2091564191 >> v1:172.21.15.188:6789/0 conn(0x7f056c070fd0 legacy=0x7f056c162510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:19:35.615 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:35.611+0000 7f0570c63640 1 -- 172.21.15.143:0/2091564191 --> v1:172.21.15.143:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f056c14b5c0 con 0x7f056c0cf7c0 2023-11-26T07:19:35.615 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:35.611+0000 7f0570c63640 1 -- 172.21.15.143:0/2091564191 <== mon.0 v1:172.21.15.143:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2137882523 0 0) 0x7f055c001680 con 0x7f056c0cf7c0 2023-11-26T07:19:35.615 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:35.611+0000 7f0570c63640 1 -- 172.21.15.143:0/2091564191 >> v1:172.21.15.143:6789/0 conn(0x7f056c0cf7c0 legacy=0x7f056c0708c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:19:35.615 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:35.611+0000 7f0570c63640 1 --2- 172.21.15.143:0/2091564191 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f0558003980 0x7f0558003da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:19:35.616 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:35.611+0000 7f0570c63640 1 --2- 172.21.15.143:0/2091564191 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f05580047d0 0x7f0558004c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:19:35.616 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:35.611+0000 7f0570c63640 1 --2- 172.21.15.143:0/2091564191 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f0558005190 0x7f055800b990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:19:35.616 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:35.611+0000 7f0570c63640 1 -- 172.21.15.143:0/2091564191 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f056c161d70 con 0x7f0558003980 2023-11-26T07:19:35.616 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:35.611+0000 7f0570c63640 1 -- 172.21.15.143:0/2091564191 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f055800c030 con 0x7f0558005190 2023-11-26T07:19:35.616 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:35.611+0000 7f0570c63640 1 -- 172.21.15.143:0/2091564191 --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f055800c1f0 con 0x7f05580047d0 2023-11-26T07:19:35.616 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:35.611+0000 7f0571c65640 1 --2- 172.21.15.143:0/2091564191 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f0558003980 0x7f0558003da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:19:35.616 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:35.611+0000 7f0572466640 1 --2- 172.21.15.143:0/2091564191 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f0558005190 0x7f055800b990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:19:35.616 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:35.611+0000 7f0571464640 1 --2- 172.21.15.143:0/2091564191 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f05580047d0 0x7f0558004c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:19:35.616 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:35.611+0000 7f0571c65640 1 -- 172.21.15.143:0/2091564191 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f05580047d0 msgr2=0x7f0558004c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-26T07:19:35.616 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:35.611+0000 7f0571c65640 1 --2- 172.21.15.143:0/2091564191 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f05580047d0 0x7f0558004c50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:19:35.616 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:35.611+0000 7f0571c65640 1 -- 172.21.15.143:0/2091564191 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f0558005190 msgr2=0x7f055800b990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-26T07:19:35.616 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:35.611+0000 7f0571c65640 1 --2- 172.21.15.143:0/2091564191 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f0558005190 0x7f055800b990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:19:35.617 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:35.611+0000 7f0571c65640 1 -- 172.21.15.143:0/2091564191 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0558003530 con 0x7f0558003980 2023-11-26T07:19:35.617 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:35.611+0000 7f0572466640 1 -- 172.21.15.143:0/2091564191 reap_dead start 2023-11-26T07:19:35.617 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:35.615+0000 7f0571c65640 1 --2- 172.21.15.143:0/2091564191 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f0558003980 0x7f0558003da0 secure :-1 s=READY pgs=571 cs=0 l=1 rev1=1 crypto rx=0x7f0568009ae0 tx=0x7f056801ca20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=598cea27fe8f8a04 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:19:35.617 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:35.615+0000 7f0570c63640 1 -- 172.21.15.143:0/2091564191 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f05680a6510 con 0x7f0558003980 2023-11-26T07:19:35.617 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:35.615+0000 7f0570c63640 1 -- 172.21.15.143:0/2091564191 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f05680a6670 con 0x7f0558003980 2023-11-26T07:19:35.617 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:35.615+0000 7f0570c63640 1 -- 172.21.15.143:0/2091564191 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f05680a6a20 con 0x7f0558003980 2023-11-26T07:19:35.618 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:35.615+0000 7f0573ec9640 1 -- 172.21.15.143:0/2091564191 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f0558003980 msgr2=0x7f0558003da0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:19:35.618 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:35.615+0000 7f0573ec9640 1 --2- 172.21.15.143:0/2091564191 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f0558003980 0x7f0558003da0 secure :-1 s=READY pgs=571 cs=0 l=1 rev1=1 crypto rx=0x7f0568009ae0 tx=0x7f056801ca20 comp rx=0 tx=0).stop 2023-11-26T07:19:35.618 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:35.615+0000 7f0573ec9640 1 -- 172.21.15.143:0/2091564191 shutdown_connections 2023-11-26T07:19:35.618 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:35.615+0000 7f0573ec9640 1 --2- 172.21.15.143:0/2091564191 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f0558003980 0x7f0558003da0 unknown :-1 s=CLOSED pgs=571 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:19:35.618 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:35.615+0000 7f0573ec9640 1 -- 172.21.15.143:0/2091564191 >> 172.21.15.143:0/2091564191 conn(0x7f056c06a890 msgr2=0x7f056c06aca0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:19:35.619 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:35.615+0000 7f0573ec9640 1 -- 172.21.15.143:0/2091564191 shutdown_connections 2023-11-26T07:19:35.619 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:35.615+0000 7f0573ec9640 1 -- 172.21.15.143:0/2091564191 wait complete. 2023-11-26T07:19:35.620 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:35.615+0000 7f0573ec9640 1 Processor -- start 2023-11-26T07:19:35.620 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:35.615+0000 7f0573ec9640 1 -- start start 2023-11-26T07:19:35.622 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:35.619+0000 7f0573ec9640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f0558003980 0x7f056c15eb60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:19:35.622 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:35.619+0000 7f0573ec9640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f056c1f0580 0x7f056c15f240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:19:35.622 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:35.619+0000 7f0573ec9640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f056c15f780 0x7f056c15cd70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:19:35.622 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:35.619+0000 7f0573ec9640 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f056c15f0a0 con 0x7f0558003980 2023-11-26T07:19:35.622 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:35.619+0000 7f0573ec9640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f056c15e6d0 con 0x7f056c15f780 2023-11-26T07:19:35.623 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:35.619+0000 7f0573ec9640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f056c15fc00 con 0x7f056c1f0580 2023-11-26T07:19:35.623 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:35.619+0000 7f0571c65640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f0558003980 0x7f056c15eb60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:19:35.623 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:35.619+0000 7f0571c65640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f0558003980 0x7f056c15eb60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:52160/0 (socket says 172.21.15.143:52160) 2023-11-26T07:19:35.623 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:35.619+0000 7f0571c65640 1 -- 172.21.15.143:0/2358085617 learned_addr learned my addr 172.21.15.143:0/2358085617 (peer_addr_for_me v2:172.21.15.143:0/0) 2023-11-26T07:19:35.623 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:35.619+0000 7f0571464640 1 --2- 172.21.15.143:0/2358085617 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f056c1f0580 0x7f056c15f240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:19:35.623 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:35.619+0000 7f0572466640 1 --2- 172.21.15.143:0/2358085617 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f056c15f780 0x7f056c15cd70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:19:35.623 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:35.619+0000 7f0571c65640 1 -- 172.21.15.143:0/2358085617 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f056c1f0580 msgr2=0x7f056c15f240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:19:35.623 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:35.619+0000 7f0571c65640 1 --2- 172.21.15.143:0/2358085617 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f056c1f0580 0x7f056c15f240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:19:35.623 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:35.619+0000 7f0571c65640 1 -- 172.21.15.143:0/2358085617 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f056c15f780 msgr2=0x7f056c15cd70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:19:35.623 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:35.619+0000 7f0571c65640 1 --2- 172.21.15.143:0/2358085617 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f056c15f780 0x7f056c15cd70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:19:35.623 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:35.619+0000 7f0571c65640 1 -- 172.21.15.143:0/2358085617 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f056c15dac0 con 0x7f0558003980 2023-11-26T07:19:35.623 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:35.619+0000 7f0571c65640 1 --2- 172.21.15.143:0/2358085617 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f0558003980 0x7f056c15eb60 secure :-1 s=READY pgs=572 cs=0 l=1 rev1=1 crypto rx=0x7f05680b0040 tx=0x7f0558005800 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:19:35.624 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:35.619+0000 7f05627fc640 1 -- 172.21.15.143:0/2358085617 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f056800f0c0 con 0x7f0558003980 2023-11-26T07:19:35.624 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:35.619+0000 7f05627fc640 1 -- 172.21.15.143:0/2358085617 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f05680a67e0 con 0x7f0558003980 2023-11-26T07:19:35.624 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:35.619+0000 7f05627fc640 1 -- 172.21.15.143:0/2358085617 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f05680a6b70 con 0x7f0558003980 2023-11-26T07:19:35.624 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:35.619+0000 7f0573ec9640 1 -- 172.21.15.143:0/2358085617 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f056c15dda0 con 0x7f0558003980 2023-11-26T07:19:35.624 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:35.619+0000 7f0573ec9640 1 -- 172.21.15.143:0/2358085617 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f056c15e360 con 0x7f0558003980 2023-11-26T07:19:35.625 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:35.619+0000 7f0573ec9640 1 -- 172.21.15.143:0/2358085617 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f056c067680 con 0x7f0558003980 2023-11-26T07:19:35.625 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:35.619+0000 7f05627fc640 1 -- 172.21.15.143:0/2358085617 <== mon.0 v2:172.21.15.143:3300/0 4 ==== mgrmap(e 7) v1 ==== 86005+0+0 (secure 0 0 0) 0x7f05680a6cd0 con 0x7f0558003980 2023-11-26T07:19:35.625 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:35.623+0000 7f05627fc640 1 --2- 172.21.15.143:0/2358085617 >> [v2:172.21.15.143:6832/2150557544,v1:172.21.15.143:6833/2150557544] conn(0x7f0550066620 0x7f0550068ae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:19:35.625 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:35.623+0000 7f05627fc640 1 -- 172.21.15.143:0/2358085617 <== mon.0 v2:172.21.15.143:3300/0 5 ==== osd_map(45..45 src has 1..45) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f05680aa070 con 0x7f0558003980 2023-11-26T07:19:35.625 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:35.623+0000 7f0571464640 1 --2- 172.21.15.143:0/2358085617 >> [v2:172.21.15.143:6832/2150557544,v1:172.21.15.143:6833/2150557544] conn(0x7f0550066620 0x7f0550068ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:19:35.625 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:35.623+0000 7f0571464640 1 --2- 172.21.15.143:0/2358085617 >> [v2:172.21.15.143:6832/2150557544,v1:172.21.15.143:6833/2150557544] conn(0x7f0550066620 0x7f0550068ae0 secure :-1 s=READY pgs=288 cs=0 l=1 rev1=1 crypto rx=0x7f055c001ca0 tx=0x7f055c009040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:19:35.628 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:35.627+0000 7f05627fc640 1 -- 172.21.15.143:0/2358085617 <== mon.0 v2:172.21.15.143:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172649 (secure 0 0 0) 0x7f05680ec050 con 0x7f0558003980 2023-11-26T07:19:35.812 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:35.807+0000 7f0573ec9640 1 -- 172.21.15.143:0/2358085617 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f056c0aff00 con 0x7f0558003980 2023-11-26T07:19:35.813 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:35.811+0000 7f05627fc640 1 -- 172.21.15.143:0/2358085617 <== mon.0 v2:172.21.15.143:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 25 v25) v1 ==== 94+0+4922 (secure 0 0 0) 0x7f05680ef670 con 0x7f0558003980 2023-11-26T07:19:35.813 INFO:teuthology.orchestra.run.smithi143.stdout: 2023-11-26T07:19:35.813 INFO:teuthology.orchestra.run.smithi143.stdout:{"epoch":25,"default_fscid":3,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5201,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.188:6835/319464848","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6833","nonce":319464848},{"type":"v1","addr":"172.21.15.188:6835","nonce":319464848}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":20},{"gid":5584,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.188:6834/2113408070","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6832","nonce":2113408070},{"type":"v1","addr":"172.21.15.188:6834","nonce":2113408070}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":20},{"gid":5745,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.143:6835/1616742593","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6834","nonce":1616742593},{"type":"v1","addr":"172.21.15.143:6835","nonce":1616742593}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":20}],"filesystems":[{"mdsmap":{"epoch":25,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-11-26T07:19:31.503587+0000","modified":"2023-11-26T07:19:34.523605+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":5748},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5748":{"gid":5748,"name":"c","rank":0,"incarnation":21,"state":"up:active","state_seq":3,"addr":"172.21.15.143:6837/2872033338","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6836","nonce":2872033338},{"type":"v1","addr":"172.21.15.143:6837","nonce":2872033338}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[7],"metadata_pool":6,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":3}]} 2023-11-26T07:19:35.813 INFO:teuthology.orchestra.run.smithi143.stderr:dumped fsmap epoch 25 2023-11-26T07:19:35.815 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:35.811+0000 7f0573ec9640 1 -- 172.21.15.143:0/2358085617 >> [v2:172.21.15.143:6832/2150557544,v1:172.21.15.143:6833/2150557544] conn(0x7f0550066620 msgr2=0x7f0550068ae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:19:35.815 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:35.811+0000 7f0573ec9640 1 --2- 172.21.15.143:0/2358085617 >> [v2:172.21.15.143:6832/2150557544,v1:172.21.15.143:6833/2150557544] conn(0x7f0550066620 0x7f0550068ae0 secure :-1 s=READY pgs=288 cs=0 l=1 rev1=1 crypto rx=0x7f055c001ca0 tx=0x7f055c009040 comp rx=0 tx=0).stop 2023-11-26T07:19:35.815 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:35.811+0000 7f0573ec9640 1 -- 172.21.15.143:0/2358085617 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f0558003980 msgr2=0x7f056c15eb60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:19:35.815 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:35.811+0000 7f0573ec9640 1 --2- 172.21.15.143:0/2358085617 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f0558003980 0x7f056c15eb60 secure :-1 s=READY pgs=572 cs=0 l=1 rev1=1 crypto rx=0x7f05680b0040 tx=0x7f0558005800 comp rx=0 tx=0).stop 2023-11-26T07:19:35.815 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:35.811+0000 7f0573ec9640 1 -- 172.21.15.143:0/2358085617 shutdown_connections 2023-11-26T07:19:35.815 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:35.811+0000 7f0573ec9640 1 --2- 172.21.15.143:0/2358085617 >> [v2:172.21.15.143:6832/2150557544,v1:172.21.15.143:6833/2150557544] conn(0x7f0550066620 0x7f0550068ae0 unknown :-1 s=CLOSED pgs=288 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:19:35.816 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:35.811+0000 7f0573ec9640 1 --2- 172.21.15.143:0/2358085617 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f056c15f780 0x7f056c15cd70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:19:35.816 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:35.811+0000 7f0573ec9640 1 --2- 172.21.15.143:0/2358085617 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f056c1f0580 0x7f056c15f240 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:19:35.816 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:35.811+0000 7f0573ec9640 1 --2- 172.21.15.143:0/2358085617 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f0558003980 0x7f056c15eb60 unknown :-1 s=CLOSED pgs=572 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:19:35.816 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:35.811+0000 7f0573ec9640 1 -- 172.21.15.143:0/2358085617 >> 172.21.15.143:0/2358085617 conn(0x7f056c06a890 msgr2=0x7f056c0af2e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:19:35.816 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:35.811+0000 7f0573ec9640 1 -- 172.21.15.143:0/2358085617 shutdown_connections 2023-11-26T07:19:35.816 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:35.811+0000 7f0573ec9640 1 -- 172.21.15.143:0/2358085617 wait complete. 2023-11-26T07:19:35.834 DEBUG:teuthology.orchestra.run.smithi143:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-11-26T07:19:35.972 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:35.967+0000 7fef28668640 1 Processor -- start 2023-11-26T07:19:35.973 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:35.971+0000 7fef28668640 1 -- start start 2023-11-26T07:19:35.974 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:35.971+0000 7fef28668640 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7fef2015d6e0 0x7fef200d0df0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:19:35.975 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:35.971+0000 7fef28668640 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7fef200d1bc0 0x7fef200b0750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:19:35.975 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:35.971+0000 7fef28668640 1 -- --> v1:172.21.15.143:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fef200cf0e0 con 0x7fef200b2160 2023-11-26T07:19:35.975 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:35.971+0000 7fef28668640 1 -- --> v2:172.21.15.143:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fef2015dac0 con 0x7fef200d1bc0 2023-11-26T07:19:35.975 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:35.971+0000 7fef28668640 1 -- --> v2:172.21.15.188:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fef200ceb20 con 0x7fef2015d6e0 2023-11-26T07:19:35.975 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:35.971+0000 7fef26c05640 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7fef200d1bc0 0x7fef200b0750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:19:35.975 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:35.971+0000 7fef26c05640 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7fef200d1bc0 0x7fef200b0750 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:52174/0 (socket says 172.21.15.143:52174) 2023-11-26T07:19:35.975 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:35.971+0000 7fef26c05640 1 -- 172.21.15.143:0/1380633872 learned_addr learned my addr 172.21.15.143:0/1380633872 (peer_addr_for_me v2:172.21.15.143:0/0) 2023-11-26T07:19:35.975 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:35.971+0000 7fef26404640 1 --2- 172.21.15.143:0/1380633872 >> v2:172.21.15.188:3300/0 conn(0x7fef2015d6e0 0x7fef200d0df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:19:35.975 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:35.971+0000 7fef26c05640 1 -- 172.21.15.143:0/1380633872 >> v2:172.21.15.188:3300/0 conn(0x7fef2015d6e0 msgr2=0x7fef200d0df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-26T07:19:35.975 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:35.971+0000 7fef26c05640 1 --2- 172.21.15.143:0/1380633872 >> v2:172.21.15.188:3300/0 conn(0x7fef2015d6e0 0x7fef200d0df0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:19:35.975 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:35.971+0000 7fef26c05640 1 -- 172.21.15.143:0/1380633872 >> v1:172.21.15.143:6789/0 conn(0x7fef200b2160 legacy=0x7fef200d14b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:19:35.975 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:35.971+0000 7fef26c05640 1 -- 172.21.15.143:0/1380633872 --> v2:172.21.15.143:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fef200cee00 con 0x7fef200d1bc0 2023-11-26T07:19:35.976 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:35.971+0000 7fef26404640 1 --2- 172.21.15.143:0/1380633872 >> v2:172.21.15.188:3300/0 conn(0x7fef2015d6e0 0x7fef200d0df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-26T07:19:35.977 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:35.975+0000 7fef26c05640 1 --2- 172.21.15.143:0/1380633872 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fef200d1bc0 0x7fef200b0750 secure :-1 s=READY pgs=574 cs=0 l=1 rev1=1 crypto rx=0x7fef200b5db0 tx=0x7fef1801c9d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=10c768bb643c8b90 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:19:35.978 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:35.975+0000 7fef25402640 1 -- 172.21.15.143:0/1380633872 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fef180a5780 con 0x7fef200d1bc0 2023-11-26T07:19:35.978 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:35.975+0000 7fef25402640 1 -- 172.21.15.143:0/1380633872 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fef180a58e0 con 0x7fef200d1bc0 2023-11-26T07:19:35.979 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:35.975+0000 7fef25402640 1 -- 172.21.15.143:0/1380633872 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fef180a5c70 con 0x7fef200d1bc0 2023-11-26T07:19:35.983 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:35.975+0000 7fef28668640 1 -- 172.21.15.143:0/1380633872 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fef200d1bc0 msgr2=0x7fef200b0750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:19:35.983 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:35.975+0000 7fef28668640 1 --2- 172.21.15.143:0/1380633872 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fef200d1bc0 0x7fef200b0750 secure :-1 s=READY pgs=574 cs=0 l=1 rev1=1 crypto rx=0x7fef200b5db0 tx=0x7fef1801c9d0 comp rx=0 tx=0).stop 2023-11-26T07:19:35.983 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:35.975+0000 7fef28668640 1 -- 172.21.15.143:0/1380633872 shutdown_connections 2023-11-26T07:19:35.983 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:35.975+0000 7fef28668640 1 --2- 172.21.15.143:0/1380633872 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fef200d1bc0 0x7fef200b0750 unknown :-1 s=CLOSED pgs=574 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:19:35.984 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:35.975+0000 7fef28668640 1 --2- 172.21.15.143:0/1380633872 >> v2:172.21.15.188:3300/0 conn(0x7fef2015d6e0 0x7fef200d0df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:19:35.984 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:35.975+0000 7fef28668640 1 -- 172.21.15.143:0/1380633872 >> 172.21.15.143:0/1380633872 conn(0x7fef2009c550 msgr2=0x7fef200d3220 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:19:35.984 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:35.975+0000 7fef28668640 1 -- 172.21.15.143:0/1380633872 shutdown_connections 2023-11-26T07:19:35.984 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:35.975+0000 7fef28668640 1 -- 172.21.15.143:0/1380633872 wait complete. 2023-11-26T07:19:35.984 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:35.975+0000 7fef28668640 1 Processor -- start 2023-11-26T07:19:35.984 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:35.979+0000 7fef28668640 1 -- start start 2023-11-26T07:19:35.984 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:35.979+0000 7fef28668640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fef200b2160 0x7fef20160780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:19:35.984 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:35.979+0000 7fef28668640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fef200d1bc0 0x7fef2015dc00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:19:35.984 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:35.979+0000 7fef28668640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fef2015d6e0 0x7fef200b36d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:19:35.984 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:35.979+0000 7fef28668640 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fef2015e2c0 con 0x7fef200d1bc0 2023-11-26T07:19:35.984 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:35.979+0000 7fef28668640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fef2015fca0 con 0x7fef200b2160 2023-11-26T07:19:35.984 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:35.979+0000 7fef28668640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fef2015ff80 con 0x7fef2015d6e0 2023-11-26T07:19:35.985 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:35.979+0000 7fef25c03640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fef200d1bc0 0x7fef2015dc00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:19:35.985 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:35.979+0000 7fef25c03640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fef200d1bc0 0x7fef2015dc00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:52188/0 (socket says 172.21.15.143:52188) 2023-11-26T07:19:35.985 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:35.979+0000 7fef25c03640 1 -- 172.21.15.143:0/287195429 learned_addr learned my addr 172.21.15.143:0/287195429 (peer_addr_for_me v2:172.21.15.143:0/0) 2023-11-26T07:19:35.985 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:35.979+0000 7fef26404640 1 --2- 172.21.15.143:0/287195429 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fef200b2160 0x7fef20160780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:19:35.985 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:35.979+0000 7fef26c05640 1 --2- 172.21.15.143:0/287195429 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fef2015d6e0 0x7fef200b36d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:19:35.985 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:35.979+0000 7fef25c03640 1 -- 172.21.15.143:0/287195429 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fef2015d6e0 msgr2=0x7fef200b36d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:19:35.985 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:35.979+0000 7fef25c03640 1 --2- 172.21.15.143:0/287195429 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fef2015d6e0 0x7fef200b36d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:19:35.985 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:35.979+0000 7fef25c03640 1 -- 172.21.15.143:0/287195429 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fef200b2160 msgr2=0x7fef20160780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:19:35.985 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:35.979+0000 7fef25c03640 1 --2- 172.21.15.143:0/287195429 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fef200b2160 0x7fef20160780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:19:35.985 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:35.979+0000 7fef25c03640 1 -- 172.21.15.143:0/287195429 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fef18009b30 con 0x7fef200d1bc0 2023-11-26T07:19:35.985 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:35.979+0000 7fef26404640 1 --2- 172.21.15.143:0/287195429 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fef200b2160 0x7fef20160780 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-26T07:19:35.985 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:35.979+0000 7fef25c03640 1 --2- 172.21.15.143:0/287195429 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fef200d1bc0 0x7fef2015dc00 secure :-1 s=READY pgs=575 cs=0 l=1 rev1=1 crypto rx=0x7fef1000c500 tx=0x7fef1000c9c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:19:35.986 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:35.979+0000 7fef16ffd640 1 -- 172.21.15.143:0/287195429 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fef10005220 con 0x7fef200d1bc0 2023-11-26T07:19:35.986 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:35.979+0000 7fef16ffd640 1 -- 172.21.15.143:0/287195429 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fef10005380 con 0x7fef200d1bc0 2023-11-26T07:19:35.986 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:35.979+0000 7fef16ffd640 1 -- 172.21.15.143:0/287195429 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fef10007270 con 0x7fef200d1bc0 2023-11-26T07:19:35.986 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:35.979+0000 7fef28668640 1 -- 172.21.15.143:0/287195429 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fef20160260 con 0x7fef200d1bc0 2023-11-26T07:19:35.986 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:35.979+0000 7fef28668640 1 -- 172.21.15.143:0/287195429 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fef200b3c10 con 0x7fef200d1bc0 2023-11-26T07:19:35.987 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:35.979+0000 7fef28668640 1 -- 172.21.15.143:0/287195429 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fef2015fad0 con 0x7fef200d1bc0 2023-11-26T07:19:35.987 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:35.983+0000 7fef16ffd640 1 -- 172.21.15.143:0/287195429 <== mon.0 v2:172.21.15.143:3300/0 4 ==== mgrmap(e 7) v1 ==== 86005+0+0 (secure 0 0 0) 0x7fef100073d0 con 0x7fef200d1bc0 2023-11-26T07:19:35.987 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:35.983+0000 7fef16ffd640 1 --2- 172.21.15.143:0/287195429 >> [v2:172.21.15.143:6832/2150557544,v1:172.21.15.143:6833/2150557544] conn(0x7fef04066620 0x7fef04068ae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:19:35.987 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:35.983+0000 7fef16ffd640 1 -- 172.21.15.143:0/287195429 <== mon.0 v2:172.21.15.143:3300/0 5 ==== osd_map(45..45 src has 1..45) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fef10083b10 con 0x7fef200d1bc0 2023-11-26T07:19:35.987 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:35.983+0000 7fef26404640 1 --2- 172.21.15.143:0/287195429 >> [v2:172.21.15.143:6832/2150557544,v1:172.21.15.143:6833/2150557544] conn(0x7fef04066620 0x7fef04068ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:19:35.987 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:35.983+0000 7fef26404640 1 --2- 172.21.15.143:0/287195429 >> [v2:172.21.15.143:6832/2150557544,v1:172.21.15.143:6833/2150557544] conn(0x7fef04066620 0x7fef04068ae0 secure :-1 s=READY pgs=289 cs=0 l=1 rev1=1 crypto rx=0x7fef200d28d0 tx=0x7fef1c008040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:19:35.990 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:35.987+0000 7fef16ffd640 1 -- 172.21.15.143:0/287195429 <== mon.0 v2:172.21.15.143:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172649 (secure 0 0 0) 0x7fef10050c80 con 0x7fef200d1bc0 2023-11-26T07:19:36.139 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:36.135+0000 7fef28668640 1 -- 172.21.15.143:0/287195429 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fef20067680 con 0x7fef200d1bc0 2023-11-26T07:19:36.140 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:36.135+0000 7fef16ffd640 1 -- 172.21.15.143:0/287195429 <== mon.0 v2:172.21.15.143:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v45) v1 ==== 74+0+16549 (secure 0 0 0) 0x7fef100542a0 con 0x7fef200d1bc0 2023-11-26T07:19:36.141 INFO:teuthology.orchestra.run.smithi143.stdout: 2023-11-26T07:19:36.141 INFO:teuthology.orchestra.run.smithi143.stdout:{"epoch":45,"fsid":"f5033060-2469-4a07-95fe-2b76f5404bc1","created":"2023-11-26T07:17:28.613663+0000","modified":"2023-11-26T07:19:33.132471+0000","last_up_change":"2023-11-26T07:17:41.157813+0000","last_in_change":"2023-11-26T07:17:31.181658+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":7,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-11-26T07:17:41.758597+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":6,"pool_name":"cephfs_metadata","create_time":"2023-11-26T07:19:29.208075+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"44","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":7,"pool_name":"cephfs_data","create_time":"2023-11-26T07:19:30.503743+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"45","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"ebbee2c6-023e-44b9-9f1d-6f7bfd309730","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":43,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6817","nonce":639807369},{"type":"v1","addr":"172.21.15.143:6819","nonce":639807369}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6821","nonce":639807369},{"type":"v1","addr":"172.21.15.143:6823","nonce":639807369}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6830","nonce":639807369},{"type":"v1","addr":"172.21.15.143:6831","nonce":639807369}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6825","nonce":639807369},{"type":"v1","addr":"172.21.15.143:6828","nonce":639807369}]},"public_addr":"172.21.15.143:6819/639807369","cluster_addr":"172.21.15.143:6823/639807369","heartbeat_back_addr":"172.21.15.143:6831/639807369","heartbeat_front_addr":"172.21.15.143:6828/639807369","state":["exists","up"]},{"osd":1,"uuid":"fec4b8be-7122-49e9-9a19-5850a0700001","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":43,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6816","nonce":1144942530},{"type":"v1","addr":"172.21.15.143:6818","nonce":1144942530}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6820","nonce":1144942530},{"type":"v1","addr":"172.21.15.143:6822","nonce":1144942530}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6827","nonce":1144942530},{"type":"v1","addr":"172.21.15.143:6829","nonce":1144942530}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6824","nonce":1144942530},{"type":"v1","addr":"172.21.15.143:6826","nonce":1144942530}]},"public_addr":"172.21.15.143:6818/1144942530","cluster_addr":"172.21.15.143:6822/1144942530","heartbeat_back_addr":"172.21.15.143:6829/1144942530","heartbeat_front_addr":"172.21.15.143:6826/1144942530","state":["exists","up"]},{"osd":2,"uuid":"2888af5b-f933-46ca-8f95-6dd494ff43f6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":43,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6808","nonce":3909095186},{"type":"v1","addr":"172.21.15.143:6809","nonce":3909095186}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6810","nonce":3909095186},{"type":"v1","addr":"172.21.15.143:6811","nonce":3909095186}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6814","nonce":3909095186},{"type":"v1","addr":"172.21.15.143:6815","nonce":3909095186}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6812","nonce":3909095186},{"type":"v1","addr":"172.21.15.143:6813","nonce":3909095186}]},"public_addr":"172.21.15.143:6809/3909095186","cluster_addr":"172.21.15.143:6811/3909095186","heartbeat_back_addr":"172.21.15.143:6815/3909095186","heartbeat_front_addr":"172.21.15.143:6813/3909095186","state":["exists","up"]},{"osd":3,"uuid":"d1f52435-7bc1-4081-9550-9cd1ba55f549","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":43,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6800","nonce":3854210991},{"type":"v1","addr":"172.21.15.143:6801","nonce":3854210991}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6802","nonce":3854210991},{"type":"v1","addr":"172.21.15.143:6803","nonce":3854210991}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6806","nonce":3854210991},{"type":"v1","addr":"172.21.15.143:6807","nonce":3854210991}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6804","nonce":3854210991},{"type":"v1","addr":"172.21.15.143:6805","nonce":3854210991}]},"public_addr":"172.21.15.143:6801/3854210991","cluster_addr":"172.21.15.143:6803/3854210991","heartbeat_back_addr":"172.21.15.143:6807/3854210991","heartbeat_front_addr":"172.21.15.143:6805/3854210991","state":["exists","up"]},{"osd":4,"uuid":"e2ae8b6e-6e25-49b2-bcbf-aacce68f06a3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":43,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6800","nonce":3924606902},{"type":"v1","addr":"172.21.15.188:6801","nonce":3924606902}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6802","nonce":3924606902},{"type":"v1","addr":"172.21.15.188:6803","nonce":3924606902}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6806","nonce":3924606902},{"type":"v1","addr":"172.21.15.188:6807","nonce":3924606902}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6804","nonce":3924606902},{"type":"v1","addr":"172.21.15.188:6805","nonce":3924606902}]},"public_addr":"172.21.15.188:6801/3924606902","cluster_addr":"172.21.15.188:6803/3924606902","heartbeat_back_addr":"172.21.15.188:6807/3924606902","heartbeat_front_addr":"172.21.15.188:6805/3924606902","state":["exists","up"]},{"osd":5,"uuid":"3cec8a32-7698-4c36-8e89-e4ba11934f33","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":43,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6813","nonce":186841544},{"type":"v1","addr":"172.21.15.188:6815","nonce":186841544}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6817","nonce":186841544},{"type":"v1","addr":"172.21.15.188:6818","nonce":186841544}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6822","nonce":186841544},{"type":"v1","addr":"172.21.15.188:6823","nonce":186841544}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6820","nonce":186841544},{"type":"v1","addr":"172.21.15.188:6821","nonce":186841544}]},"public_addr":"172.21.15.188:6815/186841544","cluster_addr":"172.21.15.188:6818/186841544","heartbeat_back_addr":"172.21.15.188:6823/186841544","heartbeat_front_addr":"172.21.15.188:6821/186841544","state":["exists","up"]},{"osd":6,"uuid":"58a65d5d-e090-473a-81b4-ad38e85029f9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":43,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6808","nonce":2153499405},{"type":"v1","addr":"172.21.15.188:6809","nonce":2153499405}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6810","nonce":2153499405},{"type":"v1","addr":"172.21.15.188:6811","nonce":2153499405}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6816","nonce":2153499405},{"type":"v1","addr":"172.21.15.188:6819","nonce":2153499405}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6812","nonce":2153499405},{"type":"v1","addr":"172.21.15.188:6814","nonce":2153499405}]},"public_addr":"172.21.15.188:6809/2153499405","cluster_addr":"172.21.15.188:6811/2153499405","heartbeat_back_addr":"172.21.15.188:6819/2153499405","heartbeat_front_addr":"172.21.15.188:6814/2153499405","state":["exists","up"]},{"osd":7,"uuid":"ea5b7f70-3554-4cf8-ac30-f7c59b80678d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":43,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6824","nonce":2722096729},{"type":"v1","addr":"172.21.15.188:6825","nonce":2722096729}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6826","nonce":2722096729},{"type":"v1","addr":"172.21.15.188:6827","nonce":2722096729}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6830","nonce":2722096729},{"type":"v1","addr":"172.21.15.188:6831","nonce":2722096729}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6828","nonce":2722096729},{"type":"v1","addr":"172.21.15.188:6829","nonce":2722096729}]},"public_addr":"172.21.15.188:6825/2722096729","cluster_addr":"172.21.15.188:6827/2722096729","heartbeat_back_addr":"172.21.15.188:6831/2722096729","heartbeat_front_addr":"172.21.15.188:6829/2722096729","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-26T07:17:36.995032+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-26T07:17:36.932266+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-26T07:17:36.829304+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-26T07:17:36.804297+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-26T07:17:36.667892+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-26T07:17:36.964868+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-26T07:17:36.827332+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-26T07:17:36.893179+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-11-26T07:19:36.143 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:36.139+0000 7fef28668640 1 -- 172.21.15.143:0/287195429 >> [v2:172.21.15.143:6832/2150557544,v1:172.21.15.143:6833/2150557544] conn(0x7fef04066620 msgr2=0x7fef04068ae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:19:36.143 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:36.139+0000 7fef28668640 1 --2- 172.21.15.143:0/287195429 >> [v2:172.21.15.143:6832/2150557544,v1:172.21.15.143:6833/2150557544] conn(0x7fef04066620 0x7fef04068ae0 secure :-1 s=READY pgs=289 cs=0 l=1 rev1=1 crypto rx=0x7fef200d28d0 tx=0x7fef1c008040 comp rx=0 tx=0).stop 2023-11-26T07:19:36.143 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:36.139+0000 7fef28668640 1 -- 172.21.15.143:0/287195429 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fef200d1bc0 msgr2=0x7fef2015dc00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:19:36.143 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:36.139+0000 7fef28668640 1 --2- 172.21.15.143:0/287195429 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fef200d1bc0 0x7fef2015dc00 secure :-1 s=READY pgs=575 cs=0 l=1 rev1=1 crypto rx=0x7fef1000c500 tx=0x7fef1000c9c0 comp rx=0 tx=0).stop 2023-11-26T07:19:36.143 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:36.139+0000 7fef28668640 1 -- 172.21.15.143:0/287195429 shutdown_connections 2023-11-26T07:19:36.143 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:36.139+0000 7fef28668640 1 --2- 172.21.15.143:0/287195429 >> [v2:172.21.15.143:6832/2150557544,v1:172.21.15.143:6833/2150557544] conn(0x7fef04066620 0x7fef04068ae0 unknown :-1 s=CLOSED pgs=289 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:19:36.143 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:36.139+0000 7fef28668640 1 --2- 172.21.15.143:0/287195429 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fef2015d6e0 0x7fef200b36d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:19:36.143 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:36.139+0000 7fef28668640 1 --2- 172.21.15.143:0/287195429 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fef200d1bc0 0x7fef2015dc00 unknown :-1 s=CLOSED pgs=575 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:19:36.143 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:36.139+0000 7fef28668640 1 --2- 172.21.15.143:0/287195429 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fef200b2160 0x7fef20160780 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:19:36.143 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:36.139+0000 7fef28668640 1 -- 172.21.15.143:0/287195429 >> 172.21.15.143:0/287195429 conn(0x7fef2009c550 msgr2=0x7fef2006c9f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:19:36.143 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:36.139+0000 7fef28668640 1 -- 172.21.15.143:0/287195429 shutdown_connections 2023-11-26T07:19:36.144 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:36.139+0000 7fef28668640 1 -- 172.21.15.143:0/287195429 wait complete. 2023-11-26T07:19:36.162 INFO:tasks.cephfs.filesystem.ceph_manager:waiting for clean 2023-11-26T07:19:36.162 DEBUG:teuthology.orchestra.run.smithi143:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-11-26T07:19:36.299 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:36.295+0000 7fdb3689a640 1 Processor -- start 2023-11-26T07:19:36.300 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:36.295+0000 7fdb3689a640 1 -- start start 2023-11-26T07:19:36.301 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:36.299+0000 7fdb3689a640 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7fdb30070ba0 0x7fdb3006bae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:19:36.302 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:36.299+0000 7fdb3689a640 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7fdb30161cf0 0x7fdb30070f80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:19:36.302 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:36.299+0000 7fdb3689a640 1 -- --> v1:172.21.15.143:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fdb300714c0 con 0x7fdb300cdd10 2023-11-26T07:19:36.302 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:36.299+0000 7fdb3689a640 1 -- --> v2:172.21.15.143:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fdb300cd610 con 0x7fdb30161cf0 2023-11-26T07:19:36.302 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:36.299+0000 7fdb3689a640 1 -- --> v2:172.21.15.188:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fdb300cd8f0 con 0x7fdb30070ba0 2023-11-26T07:19:36.302 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:36.299+0000 7fdb2f7fe640 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7fdb30161cf0 0x7fdb30070f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:19:36.302 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:36.299+0000 7fdb2f7fe640 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7fdb30161cf0 0x7fdb30070f80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:52202/0 (socket says 172.21.15.143:52202) 2023-11-26T07:19:36.302 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:36.299+0000 7fdb2f7fe640 1 -- 172.21.15.143:0/318671884 learned_addr learned my addr 172.21.15.143:0/318671884 (peer_addr_for_me v2:172.21.15.143:0/0) 2023-11-26T07:19:36.302 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:36.299+0000 7fdb34e37640 1 --1- 172.21.15.143:0/318671884 >> v1:172.21.15.143:6789/0 conn(0x7fdb300cdd10 0x7fdb3006a490 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.143:6789/0 says I am v1:172.21.15.143:41738/0 (socket says 172.21.15.143:41738) 2023-11-26T07:19:36.302 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:36.299+0000 7fdb2ffff640 1 --2- 172.21.15.143:0/318671884 >> v2:172.21.15.188:3300/0 conn(0x7fdb30070ba0 0x7fdb3006bae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:19:36.302 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:36.299+0000 7fdb2f7fe640 1 -- 172.21.15.143:0/318671884 >> v2:172.21.15.188:3300/0 conn(0x7fdb30070ba0 msgr2=0x7fdb3006bae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-26T07:19:36.302 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:36.299+0000 7fdb2f7fe640 1 --2- 172.21.15.143:0/318671884 >> v2:172.21.15.188:3300/0 conn(0x7fdb30070ba0 0x7fdb3006bae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:19:36.303 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:36.299+0000 7fdb2f7fe640 1 -- 172.21.15.143:0/318671884 >> v1:172.21.15.143:6789/0 conn(0x7fdb300cdd10 legacy=0x7fdb3006a490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:19:36.303 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:36.299+0000 7fdb2f7fe640 1 -- 172.21.15.143:0/318671884 --> v2:172.21.15.143:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdb30161210 con 0x7fdb30161cf0 2023-11-26T07:19:36.303 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:36.299+0000 7fdb2ffff640 1 --2- 172.21.15.143:0/318671884 >> v2:172.21.15.188:3300/0 conn(0x7fdb30070ba0 0x7fdb3006bae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-26T07:19:36.304 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:36.299+0000 7fdb2f7fe640 1 --2- 172.21.15.143:0/318671884 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fdb30161cf0 0x7fdb30070f80 secure :-1 s=READY pgs=577 cs=0 l=1 rev1=1 crypto rx=0x7fdb30073fd0 tx=0x7fdb2401c500 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1d3dc75810de48bb server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:19:36.305 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:36.303+0000 7fdb2effd640 1 -- 172.21.15.143:0/318671884 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdb240ab640 con 0x7fdb30161cf0 2023-11-26T07:19:36.305 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:36.303+0000 7fdb2effd640 1 -- 172.21.15.143:0/318671884 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdb240a7070 con 0x7fdb30161cf0 2023-11-26T07:19:36.305 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:36.303+0000 7fdb2effd640 1 -- 172.21.15.143:0/318671884 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdb240ab9c0 con 0x7fdb30161cf0 2023-11-26T07:19:36.306 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:36.303+0000 7fdb3689a640 1 -- 172.21.15.143:0/318671884 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fdb30161cf0 msgr2=0x7fdb30070f80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:19:36.306 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:36.303+0000 7fdb3689a640 1 --2- 172.21.15.143:0/318671884 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fdb30161cf0 0x7fdb30070f80 secure :-1 s=READY pgs=577 cs=0 l=1 rev1=1 crypto rx=0x7fdb30073fd0 tx=0x7fdb2401c500 comp rx=0 tx=0).stop 2023-11-26T07:19:36.306 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:36.303+0000 7fdb3689a640 1 -- 172.21.15.143:0/318671884 shutdown_connections 2023-11-26T07:19:36.306 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:36.303+0000 7fdb3689a640 1 --2- 172.21.15.143:0/318671884 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fdb30161cf0 0x7fdb30070f80 unknown :-1 s=CLOSED pgs=577 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:19:36.306 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:36.303+0000 7fdb3689a640 1 --2- 172.21.15.143:0/318671884 >> v2:172.21.15.188:3300/0 conn(0x7fdb30070ba0 0x7fdb3006bae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:19:36.306 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:36.303+0000 7fdb3689a640 1 -- 172.21.15.143:0/318671884 >> 172.21.15.143:0/318671884 conn(0x7fdb300cf710 msgr2=0x7fdb300cfb20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:19:36.306 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:36.303+0000 7fdb3689a640 1 -- 172.21.15.143:0/318671884 shutdown_connections 2023-11-26T07:19:36.306 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:36.303+0000 7fdb3689a640 1 -- 172.21.15.143:0/318671884 wait complete. 2023-11-26T07:19:36.307 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:36.303+0000 7fdb3689a640 1 Processor -- start 2023-11-26T07:19:36.307 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:36.303+0000 7fdb3689a640 1 -- start start 2023-11-26T07:19:36.309 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:36.303+0000 7fdb3689a640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fdb30070ba0 0x7fdb30148ef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:19:36.310 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:36.303+0000 7fdb3689a640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fdb300cdd10 0x7fdb30154ef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:19:36.310 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:36.303+0000 7fdb3689a640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fdb30161cf0 0x7fdb30160420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:19:36.310 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:36.303+0000 7fdb3689a640 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdb30155430 con 0x7fdb300cdd10 2023-11-26T07:19:36.310 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:36.303+0000 7fdb3689a640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdb301544a0 con 0x7fdb30161cf0 2023-11-26T07:19:36.310 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:36.303+0000 7fdb3689a640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdb30148cd0 con 0x7fdb30070ba0 2023-11-26T07:19:36.310 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:36.303+0000 7fdb2f7fe640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fdb300cdd10 0x7fdb30154ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:19:36.310 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:36.303+0000 7fdb2f7fe640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fdb300cdd10 0x7fdb30154ef0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:52218/0 (socket says 172.21.15.143:52218) 2023-11-26T07:19:36.310 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:36.307+0000 7fdb2f7fe640 1 -- 172.21.15.143:0/3782683074 learned_addr learned my addr 172.21.15.143:0/3782683074 (peer_addr_for_me v2:172.21.15.143:0/0) 2023-11-26T07:19:36.310 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:36.307+0000 7fdb2ffff640 1 --2- 172.21.15.143:0/3782683074 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fdb30070ba0 0x7fdb30148ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:19:36.310 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:36.307+0000 7fdb34e37640 1 --2- 172.21.15.143:0/3782683074 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fdb30161cf0 0x7fdb30160420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:19:36.310 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:36.307+0000 7fdb2f7fe640 1 -- 172.21.15.143:0/3782683074 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fdb30070ba0 msgr2=0x7fdb30148ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:19:36.311 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:36.307+0000 7fdb2f7fe640 1 --2- 172.21.15.143:0/3782683074 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fdb30070ba0 0x7fdb30148ef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:19:36.311 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:36.307+0000 7fdb2f7fe640 1 -- 172.21.15.143:0/3782683074 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fdb30161cf0 msgr2=0x7fdb30160420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:19:36.311 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:36.307+0000 7fdb2f7fe640 1 --2- 172.21.15.143:0/3782683074 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fdb30161cf0 0x7fdb30160420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:19:36.311 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:36.307+0000 7fdb2f7fe640 1 -- 172.21.15.143:0/3782683074 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdb240094d0 con 0x7fdb300cdd10 2023-11-26T07:19:36.311 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:36.307+0000 7fdb34e37640 1 --2- 172.21.15.143:0/3782683074 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fdb30161cf0 0x7fdb30160420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-26T07:19:36.311 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:36.307+0000 7fdb2f7fe640 1 --2- 172.21.15.143:0/3782683074 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fdb300cdd10 0x7fdb30154ef0 secure :-1 s=READY pgs=578 cs=0 l=1 rev1=1 crypto rx=0x7fdb240ad040 tx=0x7fdb240192a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:19:36.311 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:36.307+0000 7fdb2cff9640 1 -- 172.21.15.143:0/3782683074 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdb240bf610 con 0x7fdb300cdd10 2023-11-26T07:19:36.311 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:36.307+0000 7fdb2cff9640 1 -- 172.21.15.143:0/3782683074 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdb240bf790 con 0x7fdb300cdd10 2023-11-26T07:19:36.311 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:36.307+0000 7fdb3689a640 1 -- 172.21.15.143:0/3782683074 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdb30160960 con 0x7fdb300cdd10 2023-11-26T07:19:36.311 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:36.307+0000 7fdb2cff9640 1 -- 172.21.15.143:0/3782683074 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdb240bfb40 con 0x7fdb300cdd10 2023-11-26T07:19:36.311 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:36.307+0000 7fdb3689a640 1 -- 172.21.15.143:0/3782683074 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdb3015edb0 con 0x7fdb300cdd10 2023-11-26T07:19:36.312 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:36.307+0000 7fdb3689a640 1 -- 172.21.15.143:0/3782683074 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fdb3015efa0 con 0x7fdb300cdd10 2023-11-26T07:19:36.312 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:36.307+0000 7fdb2cff9640 1 -- 172.21.15.143:0/3782683074 <== mon.0 v2:172.21.15.143:3300/0 4 ==== mgrmap(e 7) v1 ==== 86005+0+0 (secure 0 0 0) 0x7fdb240b5400 con 0x7fdb300cdd10 2023-11-26T07:19:36.313 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:36.307+0000 7fdb2cff9640 1 --2- 172.21.15.143:0/3782683074 >> [v2:172.21.15.143:6832/2150557544,v1:172.21.15.143:6833/2150557544] conn(0x7fdb1c066620 0x7fdb1c068ae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:19:36.313 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:36.311+0000 7fdb2cff9640 1 -- 172.21.15.143:0/3782683074 <== mon.0 v2:172.21.15.143:3300/0 5 ==== osd_map(45..45 src has 1..45) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fdb2411e930 con 0x7fdb300cdd10 2023-11-26T07:19:36.313 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:36.311+0000 7fdb2ffff640 1 --2- 172.21.15.143:0/3782683074 >> [v2:172.21.15.143:6832/2150557544,v1:172.21.15.143:6833/2150557544] conn(0x7fdb1c066620 0x7fdb1c068ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:19:36.313 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:36.311+0000 7fdb2ffff640 1 --2- 172.21.15.143:0/3782683074 >> [v2:172.21.15.143:6832/2150557544,v1:172.21.15.143:6833/2150557544] conn(0x7fdb1c066620 0x7fdb1c068ae0 secure :-1 s=READY pgs=290 cs=0 l=1 rev1=1 crypto rx=0x7fdb2000ab60 tx=0x7fdb20008040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:19:36.316 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:36.311+0000 7fdb2cff9640 1 -- 172.21.15.143:0/3782683074 <== mon.0 v2:172.21.15.143:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172649 (secure 0 0 0) 0x7fdb240ebb50 con 0x7fdb300cdd10 2023-11-26T07:19:36.465 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:36.459+0000 7fdb3689a640 1 -- 172.21.15.143:0/3782683074 --> [v2:172.21.15.143:6832/2150557544,v1:172.21.15.143:6833/2150557544] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7fdb300ce260 con 0x7fdb1c066620 2023-11-26T07:19:36.474 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:36.471+0000 7fdb2cff9640 1 -- 172.21.15.143:0/3782683074 <== mgr.4100 v2:172.21.15.143:6832/2150557544 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+284367 (secure 0 0 0) 0x7fdb300ce260 con 0x7fdb1c066620 2023-11-26T07:19:36.477 INFO:teuthology.orchestra.run.smithi143.stdout: 2023-11-26T07:19:36.478 INFO:teuthology.orchestra.run.smithi143.stderr:dumped all 2023-11-26T07:19:36.479 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:36.475+0000 7fdb3689a640 1 -- 172.21.15.143:0/3782683074 >> [v2:172.21.15.143:6832/2150557544,v1:172.21.15.143:6833/2150557544] conn(0x7fdb1c066620 msgr2=0x7fdb1c068ae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:19:36.480 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:36.475+0000 7fdb3689a640 1 --2- 172.21.15.143:0/3782683074 >> [v2:172.21.15.143:6832/2150557544,v1:172.21.15.143:6833/2150557544] conn(0x7fdb1c066620 0x7fdb1c068ae0 secure :-1 s=READY pgs=290 cs=0 l=1 rev1=1 crypto rx=0x7fdb2000ab60 tx=0x7fdb20008040 comp rx=0 tx=0).stop 2023-11-26T07:19:36.480 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:36.475+0000 7fdb3689a640 1 -- 172.21.15.143:0/3782683074 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fdb300cdd10 msgr2=0x7fdb30154ef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:19:36.480 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:36.475+0000 7fdb3689a640 1 --2- 172.21.15.143:0/3782683074 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fdb300cdd10 0x7fdb30154ef0 secure :-1 s=READY pgs=578 cs=0 l=1 rev1=1 crypto rx=0x7fdb240ad040 tx=0x7fdb240192a0 comp rx=0 tx=0).stop 2023-11-26T07:19:36.480 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:36.475+0000 7fdb3689a640 1 -- 172.21.15.143:0/3782683074 shutdown_connections 2023-11-26T07:19:36.480 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:36.475+0000 7fdb3689a640 1 --2- 172.21.15.143:0/3782683074 >> [v2:172.21.15.143:6832/2150557544,v1:172.21.15.143:6833/2150557544] conn(0x7fdb1c066620 0x7fdb1c068ae0 unknown :-1 s=CLOSED pgs=290 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:19:36.480 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:36.475+0000 7fdb3689a640 1 --2- 172.21.15.143:0/3782683074 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fdb30161cf0 0x7fdb30160420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:19:36.480 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:36.475+0000 7fdb3689a640 1 --2- 172.21.15.143:0/3782683074 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fdb300cdd10 0x7fdb30154ef0 unknown :-1 s=CLOSED pgs=578 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:19:36.480 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:36.475+0000 7fdb3689a640 1 --2- 172.21.15.143:0/3782683074 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fdb30070ba0 0x7fdb30148ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:19:36.480 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:36.475+0000 7fdb3689a640 1 -- 172.21.15.143:0/3782683074 >> 172.21.15.143:0/3782683074 conn(0x7fdb300cf710 msgr2=0x7fdb300b2490 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:19:36.480 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:36.475+0000 7fdb3689a640 1 -- 172.21.15.143:0/3782683074 shutdown_connections 2023-11-26T07:19:36.480 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:36.475+0000 7fdb3689a640 1 -- 172.21.15.143:0/3782683074 wait complete. 2023-11-26T07:19:36.497 INFO:teuthology.orchestra.run.smithi143.stdout:{"pg_ready":true,"pg_map":{"version":95,"stamp":"2023-11-26T07:19:35.712433+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":24,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":221240,"kb_used_data":5072,"kb_used_omap":12,"kb_used_meta":216051,"kb_avail":754753480,"statfs":{"total":773094113280,"available":772867563520,"internally_reserved":0,"allocated":5193728,"data_stored":2784394,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12749,"internal_metadata":221236787},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":32,"apply_latency_ms":32,"commit_latency_ns":32000000,"apply_latency_ns":32000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"5.598663"},"pg_stats":[{"pgid":"7.3f","version":"0'0","reported_seq":1,"reported_epoch":43,"state":"creating+peering","last_fresh":"2023-11-26T07:19:31.135164+0000","last_change":"2023-11-26T07:19:31.135164+0000","last_active":"2023-11-26T07:19:31.113498+0000","last_peered":"2023-11-26T07:19:31.113498+0000","last_clean":"2023-11-26T07:19:31.113498+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-26T07:19:31.135164+0000","last_undegraded":"2023-11-26T07:19:31.135164+0000","last_fullsized":"2023-11-26T07:19:31.135164+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:19:31.113498+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:19:31.113498+0000","last_clean_scrub_stamp":"2023-11-26T07:19:31.113498+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.3e","version":"0'0","reported_seq":1,"reported_epoch":43,"state":"creating+peering","last_fresh":"2023-11-26T07:19:31.129025+0000","last_change":"2023-11-26T07:19:31.129025+0000","last_active":"2023-11-26T07:19:31.113498+0000","last_peered":"2023-11-26T07:19:31.113498+0000","last_clean":"2023-11-26T07:19:31.113498+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-26T07:19:31.129025+0000","last_undegraded":"2023-11-26T07:19:31.129025+0000","last_fullsized":"2023-11-26T07:19:31.129025+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:19:31.113498+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:19:31.113498+0000","last_clean_scrub_stamp":"2023-11-26T07:19:31.113498+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.3d","version":"0'0","reported_seq":1,"reported_epoch":43,"state":"creating+peering","last_fresh":"2023-11-26T07:19:31.125533+0000","last_change":"2023-11-26T07:19:31.125533+0000","last_active":"2023-11-26T07:19:31.113498+0000","last_peered":"2023-11-26T07:19:31.113498+0000","last_clean":"2023-11-26T07:19:31.113498+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-26T07:19:31.125533+0000","last_undegraded":"2023-11-26T07:19:31.125533+0000","last_fullsized":"2023-11-26T07:19:31.125533+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:19:31.113498+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:19:31.113498+0000","last_clean_scrub_stamp":"2023-11-26T07:19:31.113498+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.3c","version":"0'0","reported_seq":1,"reported_epoch":43,"state":"creating+peering","last_fresh":"2023-11-26T07:19:31.129825+0000","last_change":"2023-11-26T07:19:31.129825+0000","last_active":"2023-11-26T07:19:31.113498+0000","last_peered":"2023-11-26T07:19:31.113498+0000","last_clean":"2023-11-26T07:19:31.113498+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-26T07:19:31.129825+0000","last_undegraded":"2023-11-26T07:19:31.129825+0000","last_fullsized":"2023-11-26T07:19:31.129825+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:19:31.113498+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:19:31.113498+0000","last_clean_scrub_stamp":"2023-11-26T07:19:31.113498+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.3b","version":"0'0","reported_seq":1,"reported_epoch":43,"state":"creating+peering","last_fresh":"2023-11-26T07:19:31.128964+0000","last_change":"2023-11-26T07:19:31.128964+0000","last_active":"2023-11-26T07:19:31.113498+0000","last_peered":"2023-11-26T07:19:31.113498+0000","last_clean":"2023-11-26T07:19:31.113498+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-26T07:19:31.128964+0000","last_undegraded":"2023-11-26T07:19:31.128964+0000","last_fullsized":"2023-11-26T07:19:31.128964+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:19:31.113498+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:19:31.113498+0000","last_clean_scrub_stamp":"2023-11-26T07:19:31.113498+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.3a","version":"0'0","reported_seq":1,"reported_epoch":43,"state":"creating+peering","last_fresh":"2023-11-26T07:19:31.138400+0000","last_change":"2023-11-26T07:19:31.138400+0000","last_active":"2023-11-26T07:19:31.113498+0000","last_peered":"2023-11-26T07:19:31.113498+0000","last_clean":"2023-11-26T07:19:31.113498+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-26T07:19:31.138400+0000","last_undegraded":"2023-11-26T07:19:31.138400+0000","last_fullsized":"2023-11-26T07:19:31.138400+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:19:31.113498+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:19:31.113498+0000","last_clean_scrub_stamp":"2023-11-26T07:19:31.113498+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.39","version":"0'0","reported_seq":1,"reported_epoch":43,"state":"creating+peering","last_fresh":"2023-11-26T07:19:31.134099+0000","last_change":"2023-11-26T07:19:31.134099+0000","last_active":"2023-11-26T07:19:31.113498+0000","last_peered":"2023-11-26T07:19:31.113498+0000","last_clean":"2023-11-26T07:19:31.113498+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-26T07:19:31.134099+0000","last_undegraded":"2023-11-26T07:19:31.134099+0000","last_fullsized":"2023-11-26T07:19:31.134099+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:19:31.113498+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:19:31.113498+0000","last_clean_scrub_stamp":"2023-11-26T07:19:31.113498+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.38","version":"0'0","reported_seq":1,"reported_epoch":43,"state":"creating+peering","last_fresh":"2023-11-26T07:19:31.130100+0000","last_change":"2023-11-26T07:19:31.130100+0000","last_active":"2023-11-26T07:19:31.113498+0000","last_peered":"2023-11-26T07:19:31.113498+0000","last_clean":"2023-11-26T07:19:31.113498+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-26T07:19:31.130100+0000","last_undegraded":"2023-11-26T07:19:31.130100+0000","last_fullsized":"2023-11-26T07:19:31.130100+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:19:31.113498+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:19:31.113498+0000","last_clean_scrub_stamp":"2023-11-26T07:19:31.113498+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.37","version":"0'0","reported_seq":1,"reported_epoch":43,"state":"creating+peering","last_fresh":"2023-11-26T07:19:31.129317+0000","last_change":"2023-11-26T07:19:31.129317+0000","last_active":"2023-11-26T07:19:31.113498+0000","last_peered":"2023-11-26T07:19:31.113498+0000","last_clean":"2023-11-26T07:19:31.113498+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-26T07:19:31.129317+0000","last_undegraded":"2023-11-26T07:19:31.129317+0000","last_fullsized":"2023-11-26T07:19:31.129317+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:19:31.113498+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:19:31.113498+0000","last_clean_scrub_stamp":"2023-11-26T07:19:31.113498+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.36","version":"0'0","reported_seq":1,"reported_epoch":43,"state":"creating+peering","last_fresh":"2023-11-26T07:19:31.130104+0000","last_change":"2023-11-26T07:19:31.130104+0000","last_active":"2023-11-26T07:19:31.113498+0000","last_peered":"2023-11-26T07:19:31.113498+0000","last_clean":"2023-11-26T07:19:31.113498+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-26T07:19:31.130104+0000","last_undegraded":"2023-11-26T07:19:31.130104+0000","last_fullsized":"2023-11-26T07:19:31.130104+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:19:31.113498+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:19:31.113498+0000","last_clean_scrub_stamp":"2023-11-26T07:19:31.113498+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.35","version":"0'0","reported_seq":1,"reported_epoch":43,"state":"creating+peering","last_fresh":"2023-11-26T07:19:31.130252+0000","last_change":"2023-11-26T07:19:31.130252+0000","last_active":"2023-11-26T07:19:31.113498+0000","last_peered":"2023-11-26T07:19:31.113498+0000","last_clean":"2023-11-26T07:19:31.113498+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-26T07:19:31.130252+0000","last_undegraded":"2023-11-26T07:19:31.130252+0000","last_fullsized":"2023-11-26T07:19:31.130252+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:19:31.113498+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:19:31.113498+0000","last_clean_scrub_stamp":"2023-11-26T07:19:31.113498+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.34","version":"0'0","reported_seq":1,"reported_epoch":43,"state":"creating+peering","last_fresh":"2023-11-26T07:19:31.131089+0000","last_change":"2023-11-26T07:19:31.131089+0000","last_active":"2023-11-26T07:19:31.113498+0000","last_peered":"2023-11-26T07:19:31.113498+0000","last_clean":"2023-11-26T07:19:31.113498+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-26T07:19:31.131089+0000","last_undegraded":"2023-11-26T07:19:31.131089+0000","last_fullsized":"2023-11-26T07:19:31.131089+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:19:31.113498+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:19:31.113498+0000","last_clean_scrub_stamp":"2023-11-26T07:19:31.113498+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.33","version":"0'0","reported_seq":1,"reported_epoch":43,"state":"creating+peering","last_fresh":"2023-11-26T07:19:31.132990+0000","last_change":"2023-11-26T07:19:31.132990+0000","last_active":"2023-11-26T07:19:31.113498+0000","last_peered":"2023-11-26T07:19:31.113498+0000","last_clean":"2023-11-26T07:19:31.113498+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-26T07:19:31.132990+0000","last_undegraded":"2023-11-26T07:19:31.132990+0000","last_fullsized":"2023-11-26T07:19:31.132990+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:19:31.113498+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:19:31.113498+0000","last_clean_scrub_stamp":"2023-11-26T07:19:31.113498+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.32","version":"0'0","reported_seq":1,"reported_epoch":43,"state":"creating+peering","last_fresh":"2023-11-26T07:19:31.129881+0000","last_change":"2023-11-26T07:19:31.129881+0000","last_active":"2023-11-26T07:19:31.113498+0000","last_peered":"2023-11-26T07:19:31.113498+0000","last_clean":"2023-11-26T07:19:31.113498+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-26T07:19:31.129881+0000","last_undegraded":"2023-11-26T07:19:31.129881+0000","last_fullsized":"2023-11-26T07:19:31.129881+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:19:31.113498+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:19:31.113498+0000","last_clean_scrub_stamp":"2023-11-26T07:19:31.113498+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.31","version":"0'0","reported_seq":1,"reported_epoch":43,"state":"creating+peering","last_fresh":"2023-11-26T07:19:31.138475+0000","last_change":"2023-11-26T07:19:31.138475+0000","last_active":"2023-11-26T07:19:31.113498+0000","last_peered":"2023-11-26T07:19:31.113498+0000","last_clean":"2023-11-26T07:19:31.113498+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-26T07:19:31.138475+0000","last_undegraded":"2023-11-26T07:19:31.138475+0000","last_fullsized":"2023-11-26T07:19:31.138475+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:19:31.113498+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:19:31.113498+0000","last_clean_scrub_stamp":"2023-11-26T07:19:31.113498+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.30","version":"0'0","reported_seq":1,"reported_epoch":43,"state":"creating+peering","last_fresh":"2023-11-26T07:19:31.131093+0000","last_change":"2023-11-26T07:19:31.131093+0000","last_active":"2023-11-26T07:19:31.113498+0000","last_peered":"2023-11-26T07:19:31.113498+0000","last_clean":"2023-11-26T07:19:31.113498+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-26T07:19:31.131093+0000","last_undegraded":"2023-11-26T07:19:31.131093+0000","last_fullsized":"2023-11-26T07:19:31.131093+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:19:31.113498+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:19:31.113498+0000","last_clean_scrub_stamp":"2023-11-26T07:19:31.113498+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.2f","version":"0'0","reported_seq":1,"reported_epoch":43,"state":"creating+peering","last_fresh":"2023-11-26T07:19:31.125773+0000","last_change":"2023-11-26T07:19:31.125773+0000","last_active":"2023-11-26T07:19:31.113498+0000","last_peered":"2023-11-26T07:19:31.113498+0000","last_clean":"2023-11-26T07:19:31.113498+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-26T07:19:31.125773+0000","last_undegraded":"2023-11-26T07:19:31.125773+0000","last_fullsized":"2023-11-26T07:19:31.125773+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:19:31.113498+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:19:31.113498+0000","last_clean_scrub_stamp":"2023-11-26T07:19:31.113498+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.2e","version":"0'0","reported_seq":1,"reported_epoch":43,"state":"creating+peering","last_fresh":"2023-11-26T07:19:31.126790+0000","last_change":"2023-11-26T07:19:31.126790+0000","last_active":"2023-11-26T07:19:31.113498+0000","last_peered":"2023-11-26T07:19:31.113498+0000","last_clean":"2023-11-26T07:19:31.113498+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-26T07:19:31.126790+0000","last_undegraded":"2023-11-26T07:19:31.126790+0000","last_fullsized":"2023-11-26T07:19:31.126790+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:19:31.113498+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:19:31.113498+0000","last_clean_scrub_stamp":"2023-11-26T07:19:31.113498+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.2d","version":"0'0","reported_seq":1,"reported_epoch":43,"state":"creating+peering","last_fresh":"2023-11-26T07:19:31.133954+0000","last_change":"2023-11-26T07:19:31.133954+0000","last_active":"2023-11-26T07:19:31.113498+0000","last_peered":"2023-11-26T07:19:31.113498+0000","last_clean":"2023-11-26T07:19:31.113498+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-26T07:19:31.133954+0000","last_undegraded":"2023-11-26T07:19:31.133954+0000","last_fullsized":"2023-11-26T07:19:31.133954+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:19:31.113498+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:19:31.113498+0000","last_clean_scrub_stamp":"2023-11-26T07:19:31.113498+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.2c","version":"0'0","reported_seq":1,"reported_epoch":43,"state":"creating+peering","last_fresh":"2023-11-26T07:19:31.135122+0000","last_change":"2023-11-26T07:19:31.135122+0000","last_active":"2023-11-26T07:19:31.113498+0000","last_peered":"2023-11-26T07:19:31.113498+0000","last_clean":"2023-11-26T07:19:31.113498+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-26T07:19:31.135122+0000","last_undegraded":"2023-11-26T07:19:31.135122+0000","last_fullsized":"2023-11-26T07:19:31.135122+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:19:31.113498+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:19:31.113498+0000","last_clean_scrub_stamp":"2023-11-26T07:19:31.113498+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.2b","version":"0'0","reported_seq":1,"reported_epoch":43,"state":"creating+peering","last_fresh":"2023-11-26T07:19:31.137790+0000","last_change":"2023-11-26T07:19:31.137790+0000","last_active":"2023-11-26T07:19:31.113498+0000","last_peered":"2023-11-26T07:19:31.113498+0000","last_clean":"2023-11-26T07:19:31.113498+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-26T07:19:31.137790+0000","last_undegraded":"2023-11-26T07:19:31.137790+0000","last_fullsized":"2023-11-26T07:19:31.137790+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:19:31.113498+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:19:31.113498+0000","last_clean_scrub_stamp":"2023-11-26T07:19:31.113498+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.2a","version":"0'0","reported_seq":1,"reported_epoch":43,"state":"creating+peering","last_fresh":"2023-11-26T07:19:31.129388+0000","last_change":"2023-11-26T07:19:31.129388+0000","last_active":"2023-11-26T07:19:31.113498+0000","last_peered":"2023-11-26T07:19:31.113498+0000","last_clean":"2023-11-26T07:19:31.113498+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-26T07:19:31.129388+0000","last_undegraded":"2023-11-26T07:19:31.129388+0000","last_fullsized":"2023-11-26T07:19:31.129388+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:19:31.113498+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:19:31.113498+0000","last_clean_scrub_stamp":"2023-11-26T07:19:31.113498+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.29","version":"0'0","reported_seq":1,"reported_epoch":43,"state":"creating+peering","last_fresh":"2023-11-26T07:19:31.131006+0000","last_change":"2023-11-26T07:19:31.131006+0000","last_active":"2023-11-26T07:19:31.113498+0000","last_peered":"2023-11-26T07:19:31.113498+0000","last_clean":"2023-11-26T07:19:31.113498+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-26T07:19:31.131006+0000","last_undegraded":"2023-11-26T07:19:31.131006+0000","last_fullsized":"2023-11-26T07:19:31.131006+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:19:31.113498+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:19:31.113498+0000","last_clean_scrub_stamp":"2023-11-26T07:19:31.113498+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.28","version":"0'0","reported_seq":1,"reported_epoch":43,"state":"creating+peering","last_fresh":"2023-11-26T07:19:31.130446+0000","last_change":"2023-11-26T07:19:31.130446+0000","last_active":"2023-11-26T07:19:31.113498+0000","last_peered":"2023-11-26T07:19:31.113498+0000","last_clean":"2023-11-26T07:19:31.113498+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-26T07:19:31.130446+0000","last_undegraded":"2023-11-26T07:19:31.130446+0000","last_fullsized":"2023-11-26T07:19:31.130446+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:19:31.113498+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:19:31.113498+0000","last_clean_scrub_stamp":"2023-11-26T07:19:31.113498+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.27","version":"0'0","reported_seq":1,"reported_epoch":43,"state":"creating+peering","last_fresh":"2023-11-26T07:19:31.134474+0000","last_change":"2023-11-26T07:19:31.134474+0000","last_active":"2023-11-26T07:19:31.113498+0000","last_peered":"2023-11-26T07:19:31.113498+0000","last_clean":"2023-11-26T07:19:31.113498+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-26T07:19:31.134474+0000","last_undegraded":"2023-11-26T07:19:31.134474+0000","last_fullsized":"2023-11-26T07:19:31.134474+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:19:31.113498+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:19:31.113498+0000","last_clean_scrub_stamp":"2023-11-26T07:19:31.113498+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.26","version":"0'0","reported_seq":1,"reported_epoch":43,"state":"creating+peering","last_fresh":"2023-11-26T07:19:31.132954+0000","last_change":"2023-11-26T07:19:31.132954+0000","last_active":"2023-11-26T07:19:31.113498+0000","last_peered":"2023-11-26T07:19:31.113498+0000","last_clean":"2023-11-26T07:19:31.113498+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-26T07:19:31.132954+0000","last_undegraded":"2023-11-26T07:19:31.132954+0000","last_fullsized":"2023-11-26T07:19:31.132954+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:19:31.113498+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:19:31.113498+0000","last_clean_scrub_stamp":"2023-11-26T07:19:31.113498+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.25","version":"0'0","reported_seq":1,"reported_epoch":43,"state":"creating+peering","last_fresh":"2023-11-26T07:19:31.124787+0000","last_change":"2023-11-26T07:19:31.124787+0000","last_active":"2023-11-26T07:19:31.113498+0000","last_peered":"2023-11-26T07:19:31.113498+0000","last_clean":"2023-11-26T07:19:31.113498+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-26T07:19:31.124787+0000","last_undegraded":"2023-11-26T07:19:31.124787+0000","last_fullsized":"2023-11-26T07:19:31.124787+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:19:31.113498+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:19:31.113498+0000","last_clean_scrub_stamp":"2023-11-26T07:19:31.113498+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.24","version":"0'0","reported_seq":1,"reported_epoch":43,"state":"creating+peering","last_fresh":"2023-11-26T07:19:31.130640+0000","last_change":"2023-11-26T07:19:31.130640+0000","last_active":"2023-11-26T07:19:31.113498+0000","last_peered":"2023-11-26T07:19:31.113498+0000","last_clean":"2023-11-26T07:19:31.113498+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-26T07:19:31.130640+0000","last_undegraded":"2023-11-26T07:19:31.130640+0000","last_fullsized":"2023-11-26T07:19:31.130640+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:19:31.113498+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:19:31.113498+0000","last_clean_scrub_stamp":"2023-11-26T07:19:31.113498+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.23","version":"0'0","reported_seq":1,"reported_epoch":43,"state":"creating+peering","last_fresh":"2023-11-26T07:19:31.131030+0000","last_change":"2023-11-26T07:19:31.131030+0000","last_active":"2023-11-26T07:19:31.113498+0000","last_peered":"2023-11-26T07:19:31.113498+0000","last_clean":"2023-11-26T07:19:31.113498+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-26T07:19:31.131030+0000","last_undegraded":"2023-11-26T07:19:31.131030+0000","last_fullsized":"2023-11-26T07:19:31.131030+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:19:31.113498+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:19:31.113498+0000","last_clean_scrub_stamp":"2023-11-26T07:19:31.113498+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.22","version":"0'0","reported_seq":1,"reported_epoch":43,"state":"creating+peering","last_fresh":"2023-11-26T07:19:31.126109+0000","last_change":"2023-11-26T07:19:31.126109+0000","last_active":"2023-11-26T07:19:31.113498+0000","last_peered":"2023-11-26T07:19:31.113498+0000","last_clean":"2023-11-26T07:19:31.113498+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-26T07:19:31.126109+0000","last_undegraded":"2023-11-26T07:19:31.126109+0000","last_fullsized":"2023-11-26T07:19:31.126109+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:19:31.113498+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:19:31.113498+0000","last_clean_scrub_stamp":"2023-11-26T07:19:31.113498+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.21","version":"0'0","reported_seq":1,"reported_epoch":43,"state":"creating+peering","last_fresh":"2023-11-26T07:19:31.134817+0000","last_change":"2023-11-26T07:19:31.134817+0000","last_active":"2023-11-26T07:19:31.113498+0000","last_peered":"2023-11-26T07:19:31.113498+0000","last_clean":"2023-11-26T07:19:31.113498+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-26T07:19:31.134817+0000","last_undegraded":"2023-11-26T07:19:31.134817+0000","last_fullsized":"2023-11-26T07:19:31.134817+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:19:31.113498+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:19:31.113498+0000","last_clean_scrub_stamp":"2023-11-26T07:19:31.113498+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.20","version":"0'0","reported_seq":1,"reported_epoch":43,"state":"creating+peering","last_fresh":"2023-11-26T07:19:31.127285+0000","last_change":"2023-11-26T07:19:31.127285+0000","last_active":"2023-11-26T07:19:31.113498+0000","last_peered":"2023-11-26T07:19:31.113498+0000","last_clean":"2023-11-26T07:19:31.113498+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-26T07:19:31.127285+0000","last_undegraded":"2023-11-26T07:19:31.127285+0000","last_fullsized":"2023-11-26T07:19:31.127285+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:19:31.113498+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:19:31.113498+0000","last_clean_scrub_stamp":"2023-11-26T07:19:31.113498+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.1e","version":"0'0","reported_seq":1,"reported_epoch":43,"state":"creating+peering","last_fresh":"2023-11-26T07:19:31.133294+0000","last_change":"2023-11-26T07:19:31.133294+0000","last_active":"2023-11-26T07:19:31.113498+0000","last_peered":"2023-11-26T07:19:31.113498+0000","last_clean":"2023-11-26T07:19:31.113498+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-26T07:19:31.133294+0000","last_undegraded":"2023-11-26T07:19:31.133294+0000","last_fullsized":"2023-11-26T07:19:31.133294+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:19:31.113498+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:19:31.113498+0000","last_clean_scrub_stamp":"2023-11-26T07:19:31.113498+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.1f","version":"0'0","reported_seq":9,"reported_epoch":43,"state":"active+clean","last_fresh":"2023-11-26T07:19:31.153093+0000","last_change":"2023-11-26T07:19:31.148359+0000","last_active":"2023-11-26T07:19:31.153093+0000","last_peered":"2023-11-26T07:19:31.153093+0000","last_clean":"2023-11-26T07:19:31.153093+0000","last_became_active":"2023-11-26T07:19:31.148066+0000","last_became_peered":"2023-11-26T07:19:31.148066+0000","last_unstale":"2023-11-26T07:19:31.153093+0000","last_undegraded":"2023-11-26T07:19:31.153093+0000","last_fullsized":"2023-11-26T07:19:31.153093+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:19:30.106639+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:19:30.106639+0000","last_clean_scrub_stamp":"2023-11-26T07:19:30.106639+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T15:38:28.537385+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.1f","version":"0'0","reported_seq":1,"reported_epoch":43,"state":"creating+peering","last_fresh":"2023-11-26T07:19:31.134010+0000","last_change":"2023-11-26T07:19:31.134010+0000","last_active":"2023-11-26T07:19:31.113498+0000","last_peered":"2023-11-26T07:19:31.113498+0000","last_clean":"2023-11-26T07:19:31.113498+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-26T07:19:31.134010+0000","last_undegraded":"2023-11-26T07:19:31.134010+0000","last_fullsized":"2023-11-26T07:19:31.134010+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:19:31.113498+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:19:31.113498+0000","last_clean_scrub_stamp":"2023-11-26T07:19:31.113498+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.1e","version":"0'0","reported_seq":9,"reported_epoch":43,"state":"active+clean","last_fresh":"2023-11-26T07:19:31.149264+0000","last_change":"2023-11-26T07:19:31.144305+0000","last_active":"2023-11-26T07:19:31.149264+0000","last_peered":"2023-11-26T07:19:31.149264+0000","last_clean":"2023-11-26T07:19:31.149264+0000","last_became_active":"2023-11-26T07:19:31.143063+0000","last_became_peered":"2023-11-26T07:19:31.143063+0000","last_unstale":"2023-11-26T07:19:31.149264+0000","last_undegraded":"2023-11-26T07:19:31.149264+0000","last_fullsized":"2023-11-26T07:19:31.149264+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:19:30.106639+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:19:30.106639+0000","last_clean_scrub_stamp":"2023-11-26T07:19:30.106639+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T08:28:59.053370+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.1c","version":"0'0","reported_seq":1,"reported_epoch":43,"state":"creating+peering","last_fresh":"2023-11-26T07:19:31.129959+0000","last_change":"2023-11-26T07:19:31.129959+0000","last_active":"2023-11-26T07:19:31.113498+0000","last_peered":"2023-11-26T07:19:31.113498+0000","last_clean":"2023-11-26T07:19:31.113498+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-26T07:19:31.129959+0000","last_undegraded":"2023-11-26T07:19:31.129959+0000","last_fullsized":"2023-11-26T07:19:31.129959+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:19:31.113498+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:19:31.113498+0000","last_clean_scrub_stamp":"2023-11-26T07:19:31.113498+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.1d","version":"0'0","reported_seq":9,"reported_epoch":43,"state":"active+clean","last_fresh":"2023-11-26T07:19:31.153679+0000","last_change":"2023-11-26T07:19:31.150313+0000","last_active":"2023-11-26T07:19:31.153679+0000","last_peered":"2023-11-26T07:19:31.153679+0000","last_clean":"2023-11-26T07:19:31.153679+0000","last_became_active":"2023-11-26T07:19:31.150042+0000","last_became_peered":"2023-11-26T07:19:31.150042+0000","last_unstale":"2023-11-26T07:19:31.153679+0000","last_undegraded":"2023-11-26T07:19:31.153679+0000","last_fullsized":"2023-11-26T07:19:31.153679+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:19:30.106639+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:19:30.106639+0000","last_clean_scrub_stamp":"2023-11-26T07:19:30.106639+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T16:20:23.315619+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.1d","version":"0'0","reported_seq":1,"reported_epoch":43,"state":"creating+peering","last_fresh":"2023-11-26T07:19:31.134552+0000","last_change":"2023-11-26T07:19:31.134552+0000","last_active":"2023-11-26T07:19:31.113498+0000","last_peered":"2023-11-26T07:19:31.113498+0000","last_clean":"2023-11-26T07:19:31.113498+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-26T07:19:31.134552+0000","last_undegraded":"2023-11-26T07:19:31.134552+0000","last_fullsized":"2023-11-26T07:19:31.134552+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:19:31.113498+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:19:31.113498+0000","last_clean_scrub_stamp":"2023-11-26T07:19:31.113498+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.1c","version":"0'0","reported_seq":9,"reported_epoch":43,"state":"active+clean","last_fresh":"2023-11-26T07:19:31.147334+0000","last_change":"2023-11-26T07:19:31.141667+0000","last_active":"2023-11-26T07:19:31.147334+0000","last_peered":"2023-11-26T07:19:31.147334+0000","last_clean":"2023-11-26T07:19:31.147334+0000","last_became_active":"2023-11-26T07:19:31.141393+0000","last_became_peered":"2023-11-26T07:19:31.141393+0000","last_unstale":"2023-11-26T07:19:31.147334+0000","last_undegraded":"2023-11-26T07:19:31.147334+0000","last_fullsized":"2023-11-26T07:19:31.147334+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:19:30.106639+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:19:30.106639+0000","last_clean_scrub_stamp":"2023-11-26T07:19:30.106639+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T10:27:01.949013+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.1a","version":"0'0","reported_seq":1,"reported_epoch":43,"state":"creating+peering","last_fresh":"2023-11-26T07:19:31.125537+0000","last_change":"2023-11-26T07:19:31.125537+0000","last_active":"2023-11-26T07:19:31.113498+0000","last_peered":"2023-11-26T07:19:31.113498+0000","last_clean":"2023-11-26T07:19:31.113498+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-26T07:19:31.125537+0000","last_undegraded":"2023-11-26T07:19:31.125537+0000","last_fullsized":"2023-11-26T07:19:31.125537+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:19:31.113498+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:19:31.113498+0000","last_clean_scrub_stamp":"2023-11-26T07:19:31.113498+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.1b","version":"0'0","reported_seq":9,"reported_epoch":43,"state":"active+clean","last_fresh":"2023-11-26T07:19:31.153191+0000","last_change":"2023-11-26T07:19:31.149354+0000","last_active":"2023-11-26T07:19:31.153191+0000","last_peered":"2023-11-26T07:19:31.153191+0000","last_clean":"2023-11-26T07:19:31.153191+0000","last_became_active":"2023-11-26T07:19:31.149092+0000","last_became_peered":"2023-11-26T07:19:31.149092+0000","last_unstale":"2023-11-26T07:19:31.153191+0000","last_undegraded":"2023-11-26T07:19:31.153191+0000","last_fullsized":"2023-11-26T07:19:31.153191+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:19:30.106639+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:19:30.106639+0000","last_clean_scrub_stamp":"2023-11-26T07:19:30.106639+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T13:58:09.825490+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.1b","version":"0'0","reported_seq":1,"reported_epoch":43,"state":"creating+peering","last_fresh":"2023-11-26T07:19:31.131194+0000","last_change":"2023-11-26T07:19:31.131194+0000","last_active":"2023-11-26T07:19:31.113498+0000","last_peered":"2023-11-26T07:19:31.113498+0000","last_clean":"2023-11-26T07:19:31.113498+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-26T07:19:31.131194+0000","last_undegraded":"2023-11-26T07:19:31.131194+0000","last_fullsized":"2023-11-26T07:19:31.131194+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:19:31.113498+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:19:31.113498+0000","last_clean_scrub_stamp":"2023-11-26T07:19:31.113498+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.1a","version":"0'0","reported_seq":9,"reported_epoch":43,"state":"active+clean","last_fresh":"2023-11-26T07:19:31.142739+0000","last_change":"2023-11-26T07:19:31.139455+0000","last_active":"2023-11-26T07:19:31.142739+0000","last_peered":"2023-11-26T07:19:31.142739+0000","last_clean":"2023-11-26T07:19:31.142739+0000","last_became_active":"2023-11-26T07:19:31.139009+0000","last_became_peered":"2023-11-26T07:19:31.139009+0000","last_unstale":"2023-11-26T07:19:31.142739+0000","last_undegraded":"2023-11-26T07:19:31.142739+0000","last_fullsized":"2023-11-26T07:19:31.142739+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:19:30.106639+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:19:30.106639+0000","last_clean_scrub_stamp":"2023-11-26T07:19:30.106639+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T13:51:18.697846+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.18","version":"0'0","reported_seq":1,"reported_epoch":43,"state":"creating+peering","last_fresh":"2023-11-26T07:19:31.128550+0000","last_change":"2023-11-26T07:19:31.128550+0000","last_active":"2023-11-26T07:19:31.113498+0000","last_peered":"2023-11-26T07:19:31.113498+0000","last_clean":"2023-11-26T07:19:31.113498+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-26T07:19:31.128550+0000","last_undegraded":"2023-11-26T07:19:31.128550+0000","last_fullsized":"2023-11-26T07:19:31.128550+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:19:31.113498+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:19:31.113498+0000","last_clean_scrub_stamp":"2023-11-26T07:19:31.113498+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.19","version":"0'0","reported_seq":9,"reported_epoch":43,"state":"active+clean","last_fresh":"2023-11-26T07:19:31.150741+0000","last_change":"2023-11-26T07:19:31.146327+0000","last_active":"2023-11-26T07:19:31.150741+0000","last_peered":"2023-11-26T07:19:31.150741+0000","last_clean":"2023-11-26T07:19:31.150741+0000","last_became_active":"2023-11-26T07:19:31.145985+0000","last_became_peered":"2023-11-26T07:19:31.145985+0000","last_unstale":"2023-11-26T07:19:31.150741+0000","last_undegraded":"2023-11-26T07:19:31.150741+0000","last_fullsized":"2023-11-26T07:19:31.150741+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:19:30.106639+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:19:30.106639+0000","last_clean_scrub_stamp":"2023-11-26T07:19:30.106639+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T12:39:06.522107+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.19","version":"0'0","reported_seq":1,"reported_epoch":43,"state":"creating+peering","last_fresh":"2023-11-26T07:19:31.131606+0000","last_change":"2023-11-26T07:19:31.131606+0000","last_active":"2023-11-26T07:19:31.113498+0000","last_peered":"2023-11-26T07:19:31.113498+0000","last_clean":"2023-11-26T07:19:31.113498+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-26T07:19:31.131606+0000","last_undegraded":"2023-11-26T07:19:31.131606+0000","last_fullsized":"2023-11-26T07:19:31.131606+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:19:31.113498+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:19:31.113498+0000","last_clean_scrub_stamp":"2023-11-26T07:19:31.113498+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.18","version":"0'0","reported_seq":9,"reported_epoch":43,"state":"active+clean","last_fresh":"2023-11-26T07:19:31.147314+0000","last_change":"2023-11-26T07:19:31.140258+0000","last_active":"2023-11-26T07:19:31.147314+0000","last_peered":"2023-11-26T07:19:31.147314+0000","last_clean":"2023-11-26T07:19:31.147314+0000","last_became_active":"2023-11-26T07:19:31.140009+0000","last_became_peered":"2023-11-26T07:19:31.140009+0000","last_unstale":"2023-11-26T07:19:31.147314+0000","last_undegraded":"2023-11-26T07:19:31.147314+0000","last_fullsized":"2023-11-26T07:19:31.147314+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:19:30.106639+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:19:30.106639+0000","last_clean_scrub_stamp":"2023-11-26T07:19:30.106639+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T13:51:18.697846+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.16","version":"0'0","reported_seq":1,"reported_epoch":43,"state":"creating+peering","last_fresh":"2023-11-26T07:19:31.126247+0000","last_change":"2023-11-26T07:19:31.126247+0000","last_active":"2023-11-26T07:19:31.113498+0000","last_peered":"2023-11-26T07:19:31.113498+0000","last_clean":"2023-11-26T07:19:31.113498+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-26T07:19:31.126247+0000","last_undegraded":"2023-11-26T07:19:31.126247+0000","last_fullsized":"2023-11-26T07:19:31.126247+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:19:31.113498+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:19:31.113498+0000","last_clean_scrub_stamp":"2023-11-26T07:19:31.113498+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.17","version":"0'0","reported_seq":9,"reported_epoch":43,"state":"active+clean","last_fresh":"2023-11-26T07:19:31.141806+0000","last_change":"2023-11-26T07:19:31.134552+0000","last_active":"2023-11-26T07:19:31.141806+0000","last_peered":"2023-11-26T07:19:31.141806+0000","last_clean":"2023-11-26T07:19:31.141806+0000","last_became_active":"2023-11-26T07:19:31.134241+0000","last_became_peered":"2023-11-26T07:19:31.134241+0000","last_unstale":"2023-11-26T07:19:31.141806+0000","last_undegraded":"2023-11-26T07:19:31.141806+0000","last_fullsized":"2023-11-26T07:19:31.141806+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:19:30.106639+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:19:30.106639+0000","last_clean_scrub_stamp":"2023-11-26T07:19:30.106639+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T07:36:26.843027+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.17","version":"0'0","reported_seq":1,"reported_epoch":43,"state":"creating+peering","last_fresh":"2023-11-26T07:19:31.129608+0000","last_change":"2023-11-26T07:19:31.129608+0000","last_active":"2023-11-26T07:19:31.113498+0000","last_peered":"2023-11-26T07:19:31.113498+0000","last_clean":"2023-11-26T07:19:31.113498+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-26T07:19:31.129608+0000","last_undegraded":"2023-11-26T07:19:31.129608+0000","last_fullsized":"2023-11-26T07:19:31.129608+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:19:31.113498+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:19:31.113498+0000","last_clean_scrub_stamp":"2023-11-26T07:19:31.113498+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.16","version":"0'0","reported_seq":9,"reported_epoch":43,"state":"active+clean","last_fresh":"2023-11-26T07:19:31.151072+0000","last_change":"2023-11-26T07:19:31.144647+0000","last_active":"2023-11-26T07:19:31.151072+0000","last_peered":"2023-11-26T07:19:31.151072+0000","last_clean":"2023-11-26T07:19:31.151072+0000","last_became_active":"2023-11-26T07:19:31.143606+0000","last_became_peered":"2023-11-26T07:19:31.143606+0000","last_unstale":"2023-11-26T07:19:31.151072+0000","last_undegraded":"2023-11-26T07:19:31.151072+0000","last_fullsized":"2023-11-26T07:19:31.151072+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:19:30.106639+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:19:30.106639+0000","last_clean_scrub_stamp":"2023-11-26T07:19:30.106639+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T15:42:44.993287+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.14","version":"0'0","reported_seq":1,"reported_epoch":43,"state":"creating+peering","last_fresh":"2023-11-26T07:19:31.133561+0000","last_change":"2023-11-26T07:19:31.133561+0000","last_active":"2023-11-26T07:19:31.113498+0000","last_peered":"2023-11-26T07:19:31.113498+0000","last_clean":"2023-11-26T07:19:31.113498+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-26T07:19:31.133561+0000","last_undegraded":"2023-11-26T07:19:31.133561+0000","last_fullsized":"2023-11-26T07:19:31.133561+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:19:31.113498+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:19:31.113498+0000","last_clean_scrub_stamp":"2023-11-26T07:19:31.113498+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.15","version":"0'0","reported_seq":9,"reported_epoch":43,"state":"active+clean","last_fresh":"2023-11-26T07:19:31.149462+0000","last_change":"2023-11-26T07:19:31.147755+0000","last_active":"2023-11-26T07:19:31.149462+0000","last_peered":"2023-11-26T07:19:31.149462+0000","last_clean":"2023-11-26T07:19:31.149462+0000","last_became_active":"2023-11-26T07:19:31.147461+0000","last_became_peered":"2023-11-26T07:19:31.147461+0000","last_unstale":"2023-11-26T07:19:31.149462+0000","last_undegraded":"2023-11-26T07:19:31.149462+0000","last_fullsized":"2023-11-26T07:19:31.149462+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:19:30.106639+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:19:30.106639+0000","last_clean_scrub_stamp":"2023-11-26T07:19:30.106639+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T17:07:29.328878+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.15","version":"0'0","reported_seq":1,"reported_epoch":43,"state":"creating+peering","last_fresh":"2023-11-26T07:19:31.132934+0000","last_change":"2023-11-26T07:19:31.132934+0000","last_active":"2023-11-26T07:19:31.113498+0000","last_peered":"2023-11-26T07:19:31.113498+0000","last_clean":"2023-11-26T07:19:31.113498+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-26T07:19:31.132934+0000","last_undegraded":"2023-11-26T07:19:31.132934+0000","last_fullsized":"2023-11-26T07:19:31.132934+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:19:31.113498+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:19:31.113498+0000","last_clean_scrub_stamp":"2023-11-26T07:19:31.113498+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.14","version":"0'0","reported_seq":9,"reported_epoch":43,"state":"active+clean","last_fresh":"2023-11-26T07:19:31.136478+0000","last_change":"2023-11-26T07:19:31.131243+0000","last_active":"2023-11-26T07:19:31.136478+0000","last_peered":"2023-11-26T07:19:31.136478+0000","last_clean":"2023-11-26T07:19:31.136478+0000","last_became_active":"2023-11-26T07:19:31.130865+0000","last_became_peered":"2023-11-26T07:19:31.130865+0000","last_unstale":"2023-11-26T07:19:31.136478+0000","last_undegraded":"2023-11-26T07:19:31.136478+0000","last_fullsized":"2023-11-26T07:19:31.136478+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:19:30.106639+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:19:30.106639+0000","last_clean_scrub_stamp":"2023-11-26T07:19:30.106639+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T08:45:01.391060+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.12","version":"0'0","reported_seq":1,"reported_epoch":43,"state":"creating+peering","last_fresh":"2023-11-26T07:19:31.132304+0000","last_change":"2023-11-26T07:19:31.132304+0000","last_active":"2023-11-26T07:19:31.113498+0000","last_peered":"2023-11-26T07:19:31.113498+0000","last_clean":"2023-11-26T07:19:31.113498+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-26T07:19:31.132304+0000","last_undegraded":"2023-11-26T07:19:31.132304+0000","last_fullsized":"2023-11-26T07:19:31.132304+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:19:31.113498+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:19:31.113498+0000","last_clean_scrub_stamp":"2023-11-26T07:19:31.113498+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.13","version":"0'0","reported_seq":9,"reported_epoch":43,"state":"active+clean","last_fresh":"2023-11-26T07:19:31.153151+0000","last_change":"2023-11-26T07:19:31.148780+0000","last_active":"2023-11-26T07:19:31.153151+0000","last_peered":"2023-11-26T07:19:31.153151+0000","last_clean":"2023-11-26T07:19:31.153151+0000","last_became_active":"2023-11-26T07:19:31.148142+0000","last_became_peered":"2023-11-26T07:19:31.148142+0000","last_unstale":"2023-11-26T07:19:31.153151+0000","last_undegraded":"2023-11-26T07:19:31.153151+0000","last_fullsized":"2023-11-26T07:19:31.153151+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:19:30.106639+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:19:30.106639+0000","last_clean_scrub_stamp":"2023-11-26T07:19:30.106639+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T17:07:29.328878+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.13","version":"0'0","reported_seq":1,"reported_epoch":43,"state":"creating+peering","last_fresh":"2023-11-26T07:19:31.130763+0000","last_change":"2023-11-26T07:19:31.130763+0000","last_active":"2023-11-26T07:19:31.113498+0000","last_peered":"2023-11-26T07:19:31.113498+0000","last_clean":"2023-11-26T07:19:31.113498+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-26T07:19:31.130763+0000","last_undegraded":"2023-11-26T07:19:31.130763+0000","last_fullsized":"2023-11-26T07:19:31.130763+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:19:31.113498+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:19:31.113498+0000","last_clean_scrub_stamp":"2023-11-26T07:19:31.113498+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.12","version":"0'0","reported_seq":9,"reported_epoch":43,"state":"active+clean","last_fresh":"2023-11-26T07:19:31.147945+0000","last_change":"2023-11-26T07:19:31.144842+0000","last_active":"2023-11-26T07:19:31.147945+0000","last_peered":"2023-11-26T07:19:31.147945+0000","last_clean":"2023-11-26T07:19:31.147945+0000","last_became_active":"2023-11-26T07:19:31.144554+0000","last_became_peered":"2023-11-26T07:19:31.144554+0000","last_unstale":"2023-11-26T07:19:31.147945+0000","last_undegraded":"2023-11-26T07:19:31.147945+0000","last_fullsized":"2023-11-26T07:19:31.147945+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:19:30.106639+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:19:30.106639+0000","last_clean_scrub_stamp":"2023-11-26T07:19:30.106639+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T15:38:28.537385+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.10","version":"0'0","reported_seq":1,"reported_epoch":43,"state":"creating+peering","last_fresh":"2023-11-26T07:19:31.134915+0000","last_change":"2023-11-26T07:19:31.134915+0000","last_active":"2023-11-26T07:19:31.113498+0000","last_peered":"2023-11-26T07:19:31.113498+0000","last_clean":"2023-11-26T07:19:31.113498+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-26T07:19:31.134915+0000","last_undegraded":"2023-11-26T07:19:31.134915+0000","last_fullsized":"2023-11-26T07:19:31.134915+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:19:31.113498+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:19:31.113498+0000","last_clean_scrub_stamp":"2023-11-26T07:19:31.113498+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.11","version":"0'0","reported_seq":9,"reported_epoch":43,"state":"active+clean","last_fresh":"2023-11-26T07:19:31.153141+0000","last_change":"2023-11-26T07:19:31.148587+0000","last_active":"2023-11-26T07:19:31.153141+0000","last_peered":"2023-11-26T07:19:31.153141+0000","last_clean":"2023-11-26T07:19:31.153141+0000","last_became_active":"2023-11-26T07:19:31.148170+0000","last_became_peered":"2023-11-26T07:19:31.148170+0000","last_unstale":"2023-11-26T07:19:31.153141+0000","last_undegraded":"2023-11-26T07:19:31.153141+0000","last_fullsized":"2023-11-26T07:19:31.153141+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:19:30.106639+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:19:30.106639+0000","last_clean_scrub_stamp":"2023-11-26T07:19:30.106639+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T16:13:47.070273+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.11","version":"0'0","reported_seq":1,"reported_epoch":43,"state":"creating+peering","last_fresh":"2023-11-26T07:19:31.129918+0000","last_change":"2023-11-26T07:19:31.129918+0000","last_active":"2023-11-26T07:19:31.113498+0000","last_peered":"2023-11-26T07:19:31.113498+0000","last_clean":"2023-11-26T07:19:31.113498+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-26T07:19:31.129918+0000","last_undegraded":"2023-11-26T07:19:31.129918+0000","last_fullsized":"2023-11-26T07:19:31.129918+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:19:31.113498+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:19:31.113498+0000","last_clean_scrub_stamp":"2023-11-26T07:19:31.113498+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.10","version":"0'0","reported_seq":9,"reported_epoch":43,"state":"active+clean","last_fresh":"2023-11-26T07:19:31.147291+0000","last_change":"2023-11-26T07:19:31.139691+0000","last_active":"2023-11-26T07:19:31.147291+0000","last_peered":"2023-11-26T07:19:31.147291+0000","last_clean":"2023-11-26T07:19:31.147291+0000","last_became_active":"2023-11-26T07:19:31.139402+0000","last_became_peered":"2023-11-26T07:19:31.139402+0000","last_unstale":"2023-11-26T07:19:31.147291+0000","last_undegraded":"2023-11-26T07:19:31.147291+0000","last_fullsized":"2023-11-26T07:19:31.147291+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:19:30.106639+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:19:30.106639+0000","last_clean_scrub_stamp":"2023-11-26T07:19:30.106639+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T08:28:59.053370+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.e","version":"0'0","reported_seq":1,"reported_epoch":43,"state":"creating+peering","last_fresh":"2023-11-26T07:19:31.126558+0000","last_change":"2023-11-26T07:19:31.126558+0000","last_active":"2023-11-26T07:19:31.113498+0000","last_peered":"2023-11-26T07:19:31.113498+0000","last_clean":"2023-11-26T07:19:31.113498+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-26T07:19:31.126558+0000","last_undegraded":"2023-11-26T07:19:31.126558+0000","last_fullsized":"2023-11-26T07:19:31.126558+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:19:31.113498+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:19:31.113498+0000","last_clean_scrub_stamp":"2023-11-26T07:19:31.113498+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.f","version":"0'0","reported_seq":9,"reported_epoch":43,"state":"active+clean","last_fresh":"2023-11-26T07:19:31.144746+0000","last_change":"2023-11-26T07:19:31.140922+0000","last_active":"2023-11-26T07:19:31.144746+0000","last_peered":"2023-11-26T07:19:31.144746+0000","last_clean":"2023-11-26T07:19:31.144746+0000","last_became_active":"2023-11-26T07:19:31.139513+0000","last_became_peered":"2023-11-26T07:19:31.139513+0000","last_unstale":"2023-11-26T07:19:31.144746+0000","last_undegraded":"2023-11-26T07:19:31.144746+0000","last_fullsized":"2023-11-26T07:19:31.144746+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:19:30.106639+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:19:30.106639+0000","last_clean_scrub_stamp":"2023-11-26T07:19:30.106639+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T09:39:24.076764+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.f","version":"0'0","reported_seq":1,"reported_epoch":43,"state":"creating+peering","last_fresh":"2023-11-26T07:19:31.128236+0000","last_change":"2023-11-26T07:19:31.128236+0000","last_active":"2023-11-26T07:19:31.113498+0000","last_peered":"2023-11-26T07:19:31.113498+0000","last_clean":"2023-11-26T07:19:31.113498+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-26T07:19:31.128236+0000","last_undegraded":"2023-11-26T07:19:31.128236+0000","last_fullsized":"2023-11-26T07:19:31.128236+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:19:31.113498+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:19:31.113498+0000","last_clean_scrub_stamp":"2023-11-26T07:19:31.113498+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.e","version":"0'0","reported_seq":9,"reported_epoch":43,"state":"active+clean","last_fresh":"2023-11-26T07:19:31.146310+0000","last_change":"2023-11-26T07:19:31.141584+0000","last_active":"2023-11-26T07:19:31.146310+0000","last_peered":"2023-11-26T07:19:31.146310+0000","last_clean":"2023-11-26T07:19:31.146310+0000","last_became_active":"2023-11-26T07:19:31.141272+0000","last_became_peered":"2023-11-26T07:19:31.141272+0000","last_unstale":"2023-11-26T07:19:31.146310+0000","last_undegraded":"2023-11-26T07:19:31.146310+0000","last_fullsized":"2023-11-26T07:19:31.146310+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:19:30.106639+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:19:30.106639+0000","last_clean_scrub_stamp":"2023-11-26T07:19:30.106639+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T12:10:52.088348+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.c","version":"0'0","reported_seq":1,"reported_epoch":43,"state":"creating+peering","last_fresh":"2023-11-26T07:19:31.132510+0000","last_change":"2023-11-26T07:19:31.132510+0000","last_active":"2023-11-26T07:19:31.113498+0000","last_peered":"2023-11-26T07:19:31.113498+0000","last_clean":"2023-11-26T07:19:31.113498+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-26T07:19:31.132510+0000","last_undegraded":"2023-11-26T07:19:31.132510+0000","last_fullsized":"2023-11-26T07:19:31.132510+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:19:31.113498+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:19:31.113498+0000","last_clean_scrub_stamp":"2023-11-26T07:19:31.113498+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.d","version":"0'0","reported_seq":9,"reported_epoch":43,"state":"active+clean","last_fresh":"2023-11-26T07:19:31.149609+0000","last_change":"2023-11-26T07:19:31.145210+0000","last_active":"2023-11-26T07:19:31.149609+0000","last_peered":"2023-11-26T07:19:31.149609+0000","last_clean":"2023-11-26T07:19:31.149609+0000","last_became_active":"2023-11-26T07:19:31.144917+0000","last_became_peered":"2023-11-26T07:19:31.144917+0000","last_unstale":"2023-11-26T07:19:31.149609+0000","last_undegraded":"2023-11-26T07:19:31.149609+0000","last_fullsized":"2023-11-26T07:19:31.149609+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:19:30.106639+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:19:30.106639+0000","last_clean_scrub_stamp":"2023-11-26T07:19:30.106639+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T19:18:59.191940+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.d","version":"0'0","reported_seq":1,"reported_epoch":43,"state":"creating+peering","last_fresh":"2023-11-26T07:19:31.134528+0000","last_change":"2023-11-26T07:19:31.134528+0000","last_active":"2023-11-26T07:19:31.113498+0000","last_peered":"2023-11-26T07:19:31.113498+0000","last_clean":"2023-11-26T07:19:31.113498+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-26T07:19:31.134528+0000","last_undegraded":"2023-11-26T07:19:31.134528+0000","last_fullsized":"2023-11-26T07:19:31.134528+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:19:31.113498+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:19:31.113498+0000","last_clean_scrub_stamp":"2023-11-26T07:19:31.113498+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.c","version":"0'0","reported_seq":9,"reported_epoch":43,"state":"active+clean","last_fresh":"2023-11-26T07:19:31.150641+0000","last_change":"2023-11-26T07:19:31.147668+0000","last_active":"2023-11-26T07:19:31.150641+0000","last_peered":"2023-11-26T07:19:31.150641+0000","last_clean":"2023-11-26T07:19:31.150641+0000","last_became_active":"2023-11-26T07:19:31.146685+0000","last_became_peered":"2023-11-26T07:19:31.146685+0000","last_unstale":"2023-11-26T07:19:31.150641+0000","last_undegraded":"2023-11-26T07:19:31.150641+0000","last_fullsized":"2023-11-26T07:19:31.150641+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:19:30.106639+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:19:30.106639+0000","last_clean_scrub_stamp":"2023-11-26T07:19:30.106639+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T19:14:07.171899+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.a","version":"0'0","reported_seq":1,"reported_epoch":43,"state":"creating+peering","last_fresh":"2023-11-26T07:19:31.137790+0000","last_change":"2023-11-26T07:19:31.137790+0000","last_active":"2023-11-26T07:19:31.113498+0000","last_peered":"2023-11-26T07:19:31.113498+0000","last_clean":"2023-11-26T07:19:31.113498+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-26T07:19:31.137790+0000","last_undegraded":"2023-11-26T07:19:31.137790+0000","last_fullsized":"2023-11-26T07:19:31.137790+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:19:31.113498+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:19:31.113498+0000","last_clean_scrub_stamp":"2023-11-26T07:19:31.113498+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.b","version":"0'0","reported_seq":9,"reported_epoch":43,"state":"active+clean","last_fresh":"2023-11-26T07:19:31.153174+0000","last_change":"2023-11-26T07:19:31.149345+0000","last_active":"2023-11-26T07:19:31.153174+0000","last_peered":"2023-11-26T07:19:31.153174+0000","last_clean":"2023-11-26T07:19:31.153174+0000","last_became_active":"2023-11-26T07:19:31.149072+0000","last_became_peered":"2023-11-26T07:19:31.149072+0000","last_unstale":"2023-11-26T07:19:31.153174+0000","last_undegraded":"2023-11-26T07:19:31.153174+0000","last_fullsized":"2023-11-26T07:19:31.153174+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:19:30.106639+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:19:30.106639+0000","last_clean_scrub_stamp":"2023-11-26T07:19:30.106639+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T10:27:01.949013+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.b","version":"0'0","reported_seq":1,"reported_epoch":43,"state":"creating+peering","last_fresh":"2023-11-26T07:19:31.132864+0000","last_change":"2023-11-26T07:19:31.132864+0000","last_active":"2023-11-26T07:19:31.113498+0000","last_peered":"2023-11-26T07:19:31.113498+0000","last_clean":"2023-11-26T07:19:31.113498+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-26T07:19:31.132864+0000","last_undegraded":"2023-11-26T07:19:31.132864+0000","last_fullsized":"2023-11-26T07:19:31.132864+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:19:31.113498+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:19:31.113498+0000","last_clean_scrub_stamp":"2023-11-26T07:19:31.113498+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.a","version":"0'0","reported_seq":9,"reported_epoch":43,"state":"active+clean","last_fresh":"2023-11-26T07:19:31.150714+0000","last_change":"2023-11-26T07:19:31.146363+0000","last_active":"2023-11-26T07:19:31.150714+0000","last_peered":"2023-11-26T07:19:31.150714+0000","last_clean":"2023-11-26T07:19:31.150714+0000","last_became_active":"2023-11-26T07:19:31.146013+0000","last_became_peered":"2023-11-26T07:19:31.146013+0000","last_unstale":"2023-11-26T07:19:31.150714+0000","last_undegraded":"2023-11-26T07:19:31.150714+0000","last_fullsized":"2023-11-26T07:19:31.150714+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:19:30.106639+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:19:30.106639+0000","last_clean_scrub_stamp":"2023-11-26T07:19:30.106639+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T12:54:51.008938+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.8","version":"0'0","reported_seq":1,"reported_epoch":43,"state":"creating+peering","last_fresh":"2023-11-26T07:19:31.126547+0000","last_change":"2023-11-26T07:19:31.126547+0000","last_active":"2023-11-26T07:19:31.113498+0000","last_peered":"2023-11-26T07:19:31.113498+0000","last_clean":"2023-11-26T07:19:31.113498+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-26T07:19:31.126547+0000","last_undegraded":"2023-11-26T07:19:31.126547+0000","last_fullsized":"2023-11-26T07:19:31.126547+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:19:31.113498+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:19:31.113498+0000","last_clean_scrub_stamp":"2023-11-26T07:19:31.113498+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.9","version":"0'0","reported_seq":9,"reported_epoch":43,"state":"active+clean","last_fresh":"2023-11-26T07:19:31.153191+0000","last_change":"2023-11-26T07:19:31.148399+0000","last_active":"2023-11-26T07:19:31.153191+0000","last_peered":"2023-11-26T07:19:31.153191+0000","last_clean":"2023-11-26T07:19:31.153191+0000","last_became_active":"2023-11-26T07:19:31.148127+0000","last_became_peered":"2023-11-26T07:19:31.148127+0000","last_unstale":"2023-11-26T07:19:31.153191+0000","last_undegraded":"2023-11-26T07:19:31.153191+0000","last_fullsized":"2023-11-26T07:19:31.153191+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:19:30.106639+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:19:30.106639+0000","last_clean_scrub_stamp":"2023-11-26T07:19:30.106639+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T16:49:27.472665+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.9","version":"0'0","reported_seq":1,"reported_epoch":43,"state":"creating+peering","last_fresh":"2023-11-26T07:19:31.129887+0000","last_change":"2023-11-26T07:19:31.129887+0000","last_active":"2023-11-26T07:19:31.113498+0000","last_peered":"2023-11-26T07:19:31.113498+0000","last_clean":"2023-11-26T07:19:31.113498+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-26T07:19:31.129887+0000","last_undegraded":"2023-11-26T07:19:31.129887+0000","last_fullsized":"2023-11-26T07:19:31.129887+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:19:31.113498+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:19:31.113498+0000","last_clean_scrub_stamp":"2023-11-26T07:19:31.113498+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.8","version":"0'0","reported_seq":9,"reported_epoch":43,"state":"active+clean","last_fresh":"2023-11-26T07:19:31.147571+0000","last_change":"2023-11-26T07:19:31.144956+0000","last_active":"2023-11-26T07:19:31.147571+0000","last_peered":"2023-11-26T07:19:31.147571+0000","last_clean":"2023-11-26T07:19:31.147571+0000","last_became_active":"2023-11-26T07:19:31.144653+0000","last_became_peered":"2023-11-26T07:19:31.144653+0000","last_unstale":"2023-11-26T07:19:31.147571+0000","last_undegraded":"2023-11-26T07:19:31.147571+0000","last_fullsized":"2023-11-26T07:19:31.147571+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:19:30.106639+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:19:30.106639+0000","last_clean_scrub_stamp":"2023-11-26T07:19:30.106639+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T13:42:55.807051+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.7","version":"0'0","reported_seq":1,"reported_epoch":43,"state":"creating+peering","last_fresh":"2023-11-26T07:19:31.132550+0000","last_change":"2023-11-26T07:19:31.132550+0000","last_active":"2023-11-26T07:19:31.113498+0000","last_peered":"2023-11-26T07:19:31.113498+0000","last_clean":"2023-11-26T07:19:31.113498+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-26T07:19:31.132550+0000","last_undegraded":"2023-11-26T07:19:31.132550+0000","last_fullsized":"2023-11-26T07:19:31.132550+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:19:31.113498+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:19:31.113498+0000","last_clean_scrub_stamp":"2023-11-26T07:19:31.113498+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.6","version":"0'0","reported_seq":9,"reported_epoch":43,"state":"active+clean","last_fresh":"2023-11-26T07:19:31.148470+0000","last_change":"2023-11-26T07:19:31.143699+0000","last_active":"2023-11-26T07:19:31.148470+0000","last_peered":"2023-11-26T07:19:31.148470+0000","last_clean":"2023-11-26T07:19:31.148470+0000","last_became_active":"2023-11-26T07:19:31.143421+0000","last_became_peered":"2023-11-26T07:19:31.143421+0000","last_unstale":"2023-11-26T07:19:31.148470+0000","last_undegraded":"2023-11-26T07:19:31.148470+0000","last_fullsized":"2023-11-26T07:19:31.148470+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:19:30.106639+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:19:30.106639+0000","last_clean_scrub_stamp":"2023-11-26T07:19:30.106639+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T13:42:55.807051+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.4","version":"0'0","reported_seq":1,"reported_epoch":43,"state":"creating+peering","last_fresh":"2023-11-26T07:19:31.128178+0000","last_change":"2023-11-26T07:19:31.128178+0000","last_active":"2023-11-26T07:19:31.113498+0000","last_peered":"2023-11-26T07:19:31.113498+0000","last_clean":"2023-11-26T07:19:31.113498+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-26T07:19:31.128178+0000","last_undegraded":"2023-11-26T07:19:31.128178+0000","last_fullsized":"2023-11-26T07:19:31.128178+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:19:31.113498+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:19:31.113498+0000","last_clean_scrub_stamp":"2023-11-26T07:19:31.113498+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.5","version":"0'0","reported_seq":9,"reported_epoch":43,"state":"active+clean","last_fresh":"2023-11-26T07:19:31.150074+0000","last_change":"2023-11-26T07:19:31.148100+0000","last_active":"2023-11-26T07:19:31.150074+0000","last_peered":"2023-11-26T07:19:31.150074+0000","last_clean":"2023-11-26T07:19:31.150074+0000","last_became_active":"2023-11-26T07:19:31.147766+0000","last_became_peered":"2023-11-26T07:19:31.147766+0000","last_unstale":"2023-11-26T07:19:31.150074+0000","last_undegraded":"2023-11-26T07:19:31.150074+0000","last_fullsized":"2023-11-26T07:19:31.150074+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:19:30.106639+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:19:30.106639+0000","last_clean_scrub_stamp":"2023-11-26T07:19:30.106639+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T13:58:09.825490+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.5","version":"0'0","reported_seq":1,"reported_epoch":43,"state":"creating+peering","last_fresh":"2023-11-26T07:19:31.132476+0000","last_change":"2023-11-26T07:19:31.132476+0000","last_active":"2023-11-26T07:19:31.113498+0000","last_peered":"2023-11-26T07:19:31.113498+0000","last_clean":"2023-11-26T07:19:31.113498+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-26T07:19:31.132476+0000","last_undegraded":"2023-11-26T07:19:31.132476+0000","last_fullsized":"2023-11-26T07:19:31.132476+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:19:31.113498+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:19:31.113498+0000","last_clean_scrub_stamp":"2023-11-26T07:19:31.113498+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.4","version":"0'0","reported_seq":9,"reported_epoch":43,"state":"active+clean","last_fresh":"2023-11-26T07:19:31.150706+0000","last_change":"2023-11-26T07:19:31.144386+0000","last_active":"2023-11-26T07:19:31.150706+0000","last_peered":"2023-11-26T07:19:31.150706+0000","last_clean":"2023-11-26T07:19:31.150706+0000","last_became_active":"2023-11-26T07:19:31.144106+0000","last_became_peered":"2023-11-26T07:19:31.144106+0000","last_unstale":"2023-11-26T07:19:31.150706+0000","last_undegraded":"2023-11-26T07:19:31.150706+0000","last_fullsized":"2023-11-26T07:19:31.150706+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:19:30.106639+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:19:30.106639+0000","last_clean_scrub_stamp":"2023-11-26T07:19:30.106639+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T08:08:30.523864+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.2","version":"0'0","reported_seq":1,"reported_epoch":43,"state":"creating+peering","last_fresh":"2023-11-26T07:19:31.134609+0000","last_change":"2023-11-26T07:19:31.134609+0000","last_active":"2023-11-26T07:19:31.113498+0000","last_peered":"2023-11-26T07:19:31.113498+0000","last_clean":"2023-11-26T07:19:31.113498+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-26T07:19:31.134609+0000","last_undegraded":"2023-11-26T07:19:31.134609+0000","last_fullsized":"2023-11-26T07:19:31.134609+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:19:31.113498+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:19:31.113498+0000","last_clean_scrub_stamp":"2023-11-26T07:19:31.113498+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.3","version":"0'0","reported_seq":9,"reported_epoch":43,"state":"active+clean","last_fresh":"2023-11-26T07:19:31.149359+0000","last_change":"2023-11-26T07:19:31.146929+0000","last_active":"2023-11-26T07:19:31.149359+0000","last_peered":"2023-11-26T07:19:31.149359+0000","last_clean":"2023-11-26T07:19:31.149359+0000","last_became_active":"2023-11-26T07:19:31.146630+0000","last_became_peered":"2023-11-26T07:19:31.146630+0000","last_unstale":"2023-11-26T07:19:31.149359+0000","last_undegraded":"2023-11-26T07:19:31.149359+0000","last_fullsized":"2023-11-26T07:19:31.149359+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:19:30.106639+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:19:30.106639+0000","last_clean_scrub_stamp":"2023-11-26T07:19:30.106639+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T19:14:07.171899+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.3","version":"0'0","reported_seq":1,"reported_epoch":43,"state":"creating+peering","last_fresh":"2023-11-26T07:19:31.134861+0000","last_change":"2023-11-26T07:19:31.134861+0000","last_active":"2023-11-26T07:19:31.113498+0000","last_peered":"2023-11-26T07:19:31.113498+0000","last_clean":"2023-11-26T07:19:31.113498+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-26T07:19:31.134861+0000","last_undegraded":"2023-11-26T07:19:31.134861+0000","last_fullsized":"2023-11-26T07:19:31.134861+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:19:31.113498+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:19:31.113498+0000","last_clean_scrub_stamp":"2023-11-26T07:19:31.113498+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.2","version":"0'0","reported_seq":9,"reported_epoch":43,"state":"active+clean","last_fresh":"2023-11-26T07:19:31.149756+0000","last_change":"2023-11-26T07:19:31.147730+0000","last_active":"2023-11-26T07:19:31.149756+0000","last_peered":"2023-11-26T07:19:31.149756+0000","last_clean":"2023-11-26T07:19:31.149756+0000","last_became_active":"2023-11-26T07:19:31.147374+0000","last_became_peered":"2023-11-26T07:19:31.147374+0000","last_unstale":"2023-11-26T07:19:31.149756+0000","last_undegraded":"2023-11-26T07:19:31.149756+0000","last_fullsized":"2023-11-26T07:19:31.149756+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:19:30.106639+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:19:30.106639+0000","last_clean_scrub_stamp":"2023-11-26T07:19:30.106639+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T16:49:27.472665+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.0","version":"0'0","reported_seq":1,"reported_epoch":43,"state":"creating+peering","last_fresh":"2023-11-26T07:19:31.130272+0000","last_change":"2023-11-26T07:19:31.130272+0000","last_active":"2023-11-26T07:19:31.113498+0000","last_peered":"2023-11-26T07:19:31.113498+0000","last_clean":"2023-11-26T07:19:31.113498+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-26T07:19:31.130272+0000","last_undegraded":"2023-11-26T07:19:31.130272+0000","last_fullsized":"2023-11-26T07:19:31.130272+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:19:31.113498+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:19:31.113498+0000","last_clean_scrub_stamp":"2023-11-26T07:19:31.113498+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.1","version":"0'0","reported_seq":9,"reported_epoch":43,"state":"active+clean","last_fresh":"2023-11-26T07:19:31.152909+0000","last_change":"2023-11-26T07:19:31.145269+0000","last_active":"2023-11-26T07:19:31.152909+0000","last_peered":"2023-11-26T07:19:31.152909+0000","last_clean":"2023-11-26T07:19:31.152909+0000","last_became_active":"2023-11-26T07:19:31.144994+0000","last_became_peered":"2023-11-26T07:19:31.144994+0000","last_unstale":"2023-11-26T07:19:31.152909+0000","last_undegraded":"2023-11-26T07:19:31.152909+0000","last_fullsized":"2023-11-26T07:19:31.152909+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:19:30.106639+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:19:30.106639+0000","last_clean_scrub_stamp":"2023-11-26T07:19:30.106639+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T15:32:40.529918+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.1","version":"0'0","reported_seq":1,"reported_epoch":43,"state":"creating+peering","last_fresh":"2023-11-26T07:19:31.124694+0000","last_change":"2023-11-26T07:19:31.124694+0000","last_active":"2023-11-26T07:19:31.113498+0000","last_peered":"2023-11-26T07:19:31.113498+0000","last_clean":"2023-11-26T07:19:31.113498+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-26T07:19:31.124694+0000","last_undegraded":"2023-11-26T07:19:31.124694+0000","last_fullsized":"2023-11-26T07:19:31.124694+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:19:31.113498+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:19:31.113498+0000","last_clean_scrub_stamp":"2023-11-26T07:19:31.113498+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.0","version":"0'0","reported_seq":9,"reported_epoch":43,"state":"active+clean","last_fresh":"2023-11-26T07:19:31.151119+0000","last_change":"2023-11-26T07:19:31.146523+0000","last_active":"2023-11-26T07:19:31.151119+0000","last_peered":"2023-11-26T07:19:31.151119+0000","last_clean":"2023-11-26T07:19:31.151119+0000","last_became_active":"2023-11-26T07:19:31.146168+0000","last_became_peered":"2023-11-26T07:19:31.146168+0000","last_unstale":"2023-11-26T07:19:31.151119+0000","last_undegraded":"2023-11-26T07:19:31.151119+0000","last_fullsized":"2023-11-26T07:19:31.151119+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:19:30.106639+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:19:30.106639+0000","last_clean_scrub_stamp":"2023-11-26T07:19:30.106639+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T12:03:38.617112+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.6","version":"0'0","reported_seq":1,"reported_epoch":43,"state":"creating+peering","last_fresh":"2023-11-26T07:19:31.128572+0000","last_change":"2023-11-26T07:19:31.128572+0000","last_active":"2023-11-26T07:19:31.113498+0000","last_peered":"2023-11-26T07:19:31.113498+0000","last_clean":"2023-11-26T07:19:31.113498+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-26T07:19:31.128572+0000","last_undegraded":"2023-11-26T07:19:31.128572+0000","last_fullsized":"2023-11-26T07:19:31.128572+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:19:31.113498+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:19:31.113498+0000","last_clean_scrub_stamp":"2023-11-26T07:19:31.113498+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.7","version":"0'0","reported_seq":9,"reported_epoch":43,"state":"active+clean","last_fresh":"2023-11-26T07:19:31.150685+0000","last_change":"2023-11-26T07:19:31.146427+0000","last_active":"2023-11-26T07:19:31.150685+0000","last_peered":"2023-11-26T07:19:31.150685+0000","last_clean":"2023-11-26T07:19:31.150685+0000","last_became_active":"2023-11-26T07:19:31.146044+0000","last_became_peered":"2023-11-26T07:19:31.146044+0000","last_unstale":"2023-11-26T07:19:31.150685+0000","last_undegraded":"2023-11-26T07:19:31.150685+0000","last_fullsized":"2023-11-26T07:19:31.150685+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:19:30.106639+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:19:30.106639+0000","last_clean_scrub_stamp":"2023-11-26T07:19:30.106639+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T15:54:05.005833+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.0","version":"15'117","reported_seq":182,"reported_epoch":42,"state":"active+clean","last_fresh":"2023-11-26T07:19:30.125712+0000","last_change":"2023-11-26T07:17:43.827802+0000","last_active":"2023-11-26T07:19:30.125712+0000","last_peered":"2023-11-26T07:19:30.125712+0000","last_clean":"2023-11-26T07:19:30.125712+0000","last_became_active":"2023-11-26T07:17:43.827308+0000","last_became_peered":"2023-11-26T07:17:43.827308+0000","last_unstale":"2023-11-26T07:19:30.125712+0000","last_undegraded":"2023-11-26T07:19:30.125712+0000","last_fullsized":"2023-11-26T07:19:30.125712+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:17:42.160945+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:17:42.160945+0000","last_clean_scrub_stamp":"2023-11-26T07:17:42.160945+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T08:17:54.167366+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":7,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":6,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574872,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":28084,"kb_used_data":1060,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343756,"statfs":{"total":96636764160,"available":96608006144,"internally_reserved":0,"allocated":1085440,"data_stored":786125,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Nov 26 07:18:42 2023","interfaces":[{"interface":"back","average":{"1min":0.71499999999999997,"5min":0.71499999999999997,"15min":0.71499999999999997},"min":{"1min":0.54300000000000004,"5min":0.54300000000000004,"15min":0.54300000000000004},"max":{"1min":0.91500000000000004,"5min":0.91500000000000004,"15min":0.91500000000000004},"last":0.57699999999999996},{"interface":"front","average":{"1min":0.70899999999999996,"5min":0.70899999999999996,"15min":0.70899999999999996},"min":{"1min":0.52900000000000003,"5min":0.52900000000000003,"15min":0.52900000000000003},"max":{"1min":1.0980000000000001,"5min":1.0980000000000001,"15min":1.0980000000000001},"last":0.64800000000000002}]},{"osd":1,"last update":"Sun Nov 26 07:18:42 2023","interfaces":[{"interface":"back","average":{"1min":0.68200000000000005,"5min":0.68200000000000005,"15min":0.68200000000000005},"min":{"1min":0.47999999999999998,"5min":0.47999999999999998,"15min":0.47999999999999998},"max":{"1min":1.0669999999999999,"5min":1.0669999999999999,"15min":1.0669999999999999},"last":0.73999999999999999},{"interface":"front","average":{"1min":0.83899999999999997,"5min":0.83899999999999997,"15min":0.83899999999999997},"min":{"1min":0.47599999999999998,"5min":0.47599999999999998,"15min":0.47599999999999998},"max":{"1min":4.5309999999999997,"5min":4.5309999999999997,"15min":4.5309999999999997},"last":0.47699999999999998}]},{"osd":2,"last update":"Sun Nov 26 07:18:42 2023","interfaces":[{"interface":"back","average":{"1min":0.67600000000000005,"5min":0.67600000000000005,"15min":0.67600000000000005},"min":{"1min":0.52000000000000002,"5min":0.52000000000000002,"15min":0.52000000000000002},"max":{"1min":0.873,"5min":0.873,"15min":0.873},"last":0.69499999999999995},{"interface":"front","average":{"1min":0.71299999999999997,"5min":0.71299999999999997,"15min":0.71299999999999997},"min":{"1min":0.54700000000000004,"5min":0.54700000000000004,"15min":0.54700000000000004},"max":{"1min":1.1339999999999999,"5min":1.1339999999999999,"15min":1.1339999999999999},"last":0.61599999999999999}]},{"osd":3,"last update":"Sun Nov 26 07:18:42 2023","interfaces":[{"interface":"back","average":{"1min":0.69699999999999995,"5min":0.69699999999999995,"15min":0.69699999999999995},"min":{"1min":0.51800000000000002,"5min":0.51800000000000002,"15min":0.51800000000000002},"max":{"1min":1.026,"5min":1.026,"15min":1.026},"last":0.627},{"interface":"front","average":{"1min":0.73299999999999998,"5min":0.73299999999999998,"15min":0.73299999999999998},"min":{"1min":0.50600000000000001,"5min":0.50600000000000001,"15min":0.50600000000000001},"max":{"1min":1.4490000000000001,"5min":1.4490000000000001,"15min":1.4490000000000001},"last":0.66200000000000003}]},{"osd":4,"last update":"Sun Nov 26 07:18:42 2023","interfaces":[{"interface":"back","average":{"1min":0.59999999999999998,"5min":0.59999999999999998,"15min":0.59999999999999998},"min":{"1min":0.39400000000000002,"5min":0.39400000000000002,"15min":0.39400000000000002},"max":{"1min":0.94799999999999995,"5min":0.94799999999999995,"15min":0.94799999999999995},"last":0.59599999999999997},{"interface":"front","average":{"1min":0.67300000000000004,"5min":0.67300000000000004,"15min":0.67300000000000004},"min":{"1min":0.436,"5min":0.436,"15min":0.436},"max":{"1min":1.5509999999999999,"5min":1.5509999999999999,"15min":1.5509999999999999},"last":0.51300000000000001}]},{"osd":5,"last update":"Sun Nov 26 07:18:42 2023","interfaces":[{"interface":"back","average":{"1min":0.66600000000000004,"5min":0.66600000000000004,"15min":0.66600000000000004},"min":{"1min":0.443,"5min":0.443,"15min":0.443},"max":{"1min":1.415,"5min":1.415,"15min":1.415},"last":0.77300000000000002},{"interface":"front","average":{"1min":0.71399999999999997,"5min":0.71399999999999997,"15min":0.71399999999999997},"min":{"1min":0.53100000000000003,"5min":0.53100000000000003,"15min":0.53100000000000003},"max":{"1min":1.3859999999999999,"5min":1.3859999999999999,"15min":1.3859999999999999},"last":0.67700000000000005}]},{"osd":6,"last update":"Sun Nov 26 07:18:42 2023","interfaces":[{"interface":"back","average":{"1min":0.77100000000000002,"5min":0.77100000000000002,"15min":0.77100000000000002},"min":{"1min":0.64100000000000001,"5min":0.64100000000000001,"15min":0.64100000000000001},"max":{"1min":1.3069999999999999,"5min":1.3069999999999999,"15min":1.3069999999999999},"last":0.75800000000000001},{"interface":"front","average":{"1min":0.77100000000000002,"5min":0.77100000000000002,"15min":0.77100000000000002},"min":{"1min":0.57399999999999995,"5min":0.57399999999999995,"15min":0.57399999999999995},"max":{"1min":1.6759999999999999,"5min":1.6759999999999999,"15min":1.6759999999999999},"last":0.56200000000000006}]}]},{"osd":6,"up_from":13,"seq":55834574874,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27512,"kb_used_data":492,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344328,"statfs":{"total":96636764160,"available":96608591872,"internally_reserved":0,"allocated":503808,"data_stored":202024,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Nov 26 07:18:45 2023","interfaces":[{"interface":"back","average":{"1min":0.68400000000000005,"5min":0.68400000000000005,"15min":0.68400000000000005},"min":{"1min":0.36699999999999999,"5min":0.36699999999999999,"15min":0.36699999999999999},"max":{"1min":1.7549999999999999,"5min":1.7549999999999999,"15min":1.7549999999999999},"last":0.80500000000000005},{"interface":"front","average":{"1min":0.95799999999999996,"5min":0.95799999999999996,"15min":0.95799999999999996},"min":{"1min":0.38300000000000001,"5min":0.38300000000000001,"15min":0.38300000000000001},"max":{"1min":5.7489999999999997,"5min":5.7489999999999997,"15min":5.7489999999999997},"last":0.59599999999999997}]},{"osd":1,"last update":"Sun Nov 26 07:18:45 2023","interfaces":[{"interface":"back","average":{"1min":0.91400000000000003,"5min":0.91400000000000003,"15min":0.91400000000000003},"min":{"1min":0.42099999999999999,"5min":0.42099999999999999,"15min":0.42099999999999999},"max":{"1min":5.4240000000000004,"5min":5.4240000000000004,"15min":5.4240000000000004},"last":0.46400000000000002},{"interface":"front","average":{"1min":0.89700000000000002,"5min":0.89700000000000002,"15min":0.89700000000000002},"min":{"1min":0.499,"5min":0.499,"15min":0.499},"max":{"1min":5.6020000000000003,"5min":5.6020000000000003,"15min":5.6020000000000003},"last":0.64700000000000002}]},{"osd":2,"last update":"Sun Nov 26 07:18:45 2023","interfaces":[{"interface":"back","average":{"1min":0.69699999999999995,"5min":0.69699999999999995,"15min":0.69699999999999995},"min":{"1min":0.47599999999999998,"5min":0.47599999999999998,"15min":0.47599999999999998},"max":{"1min":1.8520000000000001,"5min":1.8520000000000001,"15min":1.8520000000000001},"last":0.624},{"interface":"front","average":{"1min":0.73399999999999999,"5min":0.73399999999999999,"15min":0.73399999999999999},"min":{"1min":0.46700000000000003,"5min":0.46700000000000003,"15min":0.46700000000000003},"max":{"1min":1.9159999999999999,"5min":1.9159999999999999,"15min":1.9159999999999999},"last":0.67400000000000004}]},{"osd":3,"last update":"Sun Nov 26 07:18:45 2023","interfaces":[{"interface":"back","average":{"1min":0.90200000000000002,"5min":0.90200000000000002,"15min":0.90200000000000002},"min":{"1min":0.502,"5min":0.502,"15min":0.502},"max":{"1min":5.4390000000000001,"5min":5.4390000000000001,"15min":5.4390000000000001},"last":0.70099999999999996},{"interface":"front","average":{"1min":0.94499999999999995,"5min":0.94499999999999995,"15min":0.94499999999999995},"min":{"1min":0.54400000000000004,"5min":0.54400000000000004,"15min":0.54400000000000004},"max":{"1min":5.5490000000000004,"5min":5.5490000000000004,"15min":5.5490000000000004},"last":0.71899999999999997}]},{"osd":4,"last update":"Sun Nov 26 07:18:45 2023","interfaces":[{"interface":"back","average":{"1min":0.64200000000000002,"5min":0.64200000000000002,"15min":0.64200000000000002},"min":{"1min":0.46899999999999997,"5min":0.46899999999999997,"15min":0.46899999999999997},"max":{"1min":0.85099999999999998,"5min":0.85099999999999998,"15min":0.85099999999999998},"last":0.85999999999999999},{"interface":"front","average":{"1min":0.67500000000000004,"5min":0.67500000000000004,"15min":0.67500000000000004},"min":{"1min":0.40699999999999997,"5min":0.40699999999999997,"15min":0.40699999999999997},"max":{"1min":0.93300000000000005,"5min":0.93300000000000005,"15min":0.93300000000000005},"last":0.78200000000000003}]},{"osd":5,"last update":"Sun Nov 26 07:18:45 2023","interfaces":[{"interface":"back","average":{"1min":0.65900000000000003,"5min":0.65900000000000003,"15min":0.65900000000000003},"min":{"1min":0.42499999999999999,"5min":0.42499999999999999,"15min":0.42499999999999999},"max":{"1min":0.79300000000000004,"5min":0.79300000000000004,"15min":0.79300000000000004},"last":0.83299999999999996},{"interface":"front","average":{"1min":0.71299999999999997,"5min":0.71299999999999997,"15min":0.71299999999999997},"min":{"1min":0.436,"5min":0.436,"15min":0.436},"max":{"1min":0.93000000000000005,"5min":0.93000000000000005,"15min":0.93000000000000005},"last":0.90100000000000002}]},{"osd":7,"last update":"Sun Nov 26 07:18:45 2023","interfaces":[{"interface":"back","average":{"1min":0.74199999999999999,"5min":0.74199999999999999,"15min":0.74199999999999999},"min":{"1min":0.46600000000000003,"5min":0.46600000000000003,"15min":0.46600000000000003},"max":{"1min":0.998,"5min":0.998,"15min":0.998},"last":0.746},{"interface":"front","average":{"1min":0.71599999999999997,"5min":0.71599999999999997,"15min":0.71599999999999997},"min":{"1min":0.498,"5min":0.498,"15min":0.498},"max":{"1min":0.86499999999999999,"5min":0.86499999999999999,"15min":0.86499999999999999},"last":0.88100000000000001}]}]},{"osd":5,"up_from":13,"seq":55834574872,"num_pgs":4,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27512,"kb_used_data":492,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344328,"statfs":{"total":96636764160,"available":96608591872,"internally_reserved":0,"allocated":503808,"data_stored":202024,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Nov 26 07:18:45 2023","interfaces":[{"interface":"back","average":{"1min":0.66600000000000004,"5min":0.66600000000000004,"15min":0.66600000000000004},"min":{"1min":0.38100000000000001,"5min":0.38100000000000001,"15min":0.38100000000000001},"max":{"1min":1.0289999999999999,"5min":1.0289999999999999,"15min":1.0289999999999999},"last":0.66500000000000004},{"interface":"front","average":{"1min":0.66000000000000003,"5min":0.66000000000000003,"15min":0.66000000000000003},"min":{"1min":0.438,"5min":0.438,"15min":0.438},"max":{"1min":1.05,"5min":1.05,"15min":1.05},"last":0.58099999999999996}]},{"osd":1,"last update":"Sun Nov 26 07:18:45 2023","interfaces":[{"interface":"back","average":{"1min":0.66800000000000004,"5min":0.66800000000000004,"15min":0.66800000000000004},"min":{"1min":0.41599999999999998,"5min":0.41599999999999998,"15min":0.41599999999999998},"max":{"1min":0.86699999999999999,"5min":0.86699999999999999,"15min":0.86699999999999999},"last":0.872},{"interface":"front","average":{"1min":0.68600000000000005,"5min":0.68600000000000005,"15min":0.68600000000000005},"min":{"1min":0.49199999999999999,"5min":0.49199999999999999,"15min":0.49199999999999999},"max":{"1min":0.98899999999999999,"5min":0.98899999999999999,"15min":0.98899999999999999},"last":0.72899999999999998}]},{"osd":2,"last update":"Sun Nov 26 07:18:45 2023","interfaces":[{"interface":"back","average":{"1min":0.67500000000000004,"5min":0.67500000000000004,"15min":0.67500000000000004},"min":{"1min":0.45900000000000002,"5min":0.45900000000000002,"15min":0.45900000000000002},"max":{"1min":0.89800000000000002,"5min":0.89800000000000002,"15min":0.89800000000000002},"last":0.627},{"interface":"front","average":{"1min":0.67000000000000004,"5min":0.67000000000000004,"15min":0.67000000000000004},"min":{"1min":0.51200000000000001,"5min":0.51200000000000001,"15min":0.51200000000000001},"max":{"1min":0.89700000000000002,"5min":0.89700000000000002,"15min":0.89700000000000002},"last":0.76000000000000001}]},{"osd":3,"last update":"Sun Nov 26 07:18:45 2023","interfaces":[{"interface":"back","average":{"1min":0.71999999999999997,"5min":0.71999999999999997,"15min":0.71999999999999997},"min":{"1min":0.43099999999999999,"5min":0.43099999999999999,"15min":0.43099999999999999},"max":{"1min":1.0029999999999999,"5min":1.0029999999999999,"15min":1.0029999999999999},"last":0.69599999999999995},{"interface":"front","average":{"1min":0.67500000000000004,"5min":0.67500000000000004,"15min":0.67500000000000004},"min":{"1min":0.44600000000000001,"5min":0.44600000000000001,"15min":0.44600000000000001},"max":{"1min":1.0209999999999999,"5min":1.0209999999999999,"15min":1.0209999999999999},"last":0.74299999999999999}]},{"osd":4,"last update":"Sun Nov 26 07:18:45 2023","interfaces":[{"interface":"back","average":{"1min":0.63500000000000001,"5min":0.63500000000000001,"15min":0.63500000000000001},"min":{"1min":0.39200000000000002,"5min":0.39200000000000002,"15min":0.39200000000000002},"max":{"1min":0.95099999999999996,"5min":0.95099999999999996,"15min":0.95099999999999996},"last":0.78900000000000003},{"interface":"front","average":{"1min":0.70899999999999996,"5min":0.70899999999999996,"15min":0.70899999999999996},"min":{"1min":0.41299999999999998,"5min":0.41299999999999998,"15min":0.41299999999999998},"max":{"1min":1.254,"5min":1.254,"15min":1.254},"last":0.80900000000000005}]},{"osd":6,"last update":"Sun Nov 26 07:18:45 2023","interfaces":[{"interface":"back","average":{"1min":0.67200000000000004,"5min":0.67200000000000004,"15min":0.67200000000000004},"min":{"1min":0.45600000000000002,"5min":0.45600000000000002,"15min":0.45600000000000002},"max":{"1min":0.94499999999999995,"5min":0.94499999999999995,"15min":0.94499999999999995},"last":0.83199999999999996},{"interface":"front","average":{"1min":0.72699999999999998,"5min":0.72699999999999998,"15min":0.72699999999999998},"min":{"1min":0.44,"5min":0.44,"15min":0.44},"max":{"1min":1.0649999999999999,"5min":1.0649999999999999,"15min":1.0649999999999999},"last":0.85299999999999998}]},{"osd":7,"last update":"Sun Nov 26 07:18:45 2023","interfaces":[{"interface":"back","average":{"1min":0.80400000000000005,"5min":0.80400000000000005,"15min":0.80400000000000005},"min":{"1min":0.48199999999999998,"5min":0.48199999999999998,"15min":0.48199999999999998},"max":{"1min":1.4079999999999999,"5min":1.4079999999999999,"15min":1.4079999999999999},"last":0.89400000000000002},{"interface":"front","average":{"1min":0.77500000000000002,"5min":0.77500000000000002,"15min":0.77500000000000002},"min":{"1min":0.51600000000000001,"5min":0.51600000000000001,"15min":0.51600000000000001},"max":{"1min":1.0169999999999999,"5min":1.0169999999999999,"15min":1.0169999999999999},"last":0.92400000000000004}]}]},{"osd":4,"up_from":13,"seq":55834574874,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27516,"kb_used_data":492,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344324,"statfs":{"total":96636764160,"available":96608587776,"internally_reserved":0,"allocated":503808,"data_stored":202024,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Nov 26 07:18:46 2023","interfaces":[{"interface":"back","average":{"1min":0.79400000000000004,"5min":0.79400000000000004,"15min":0.79400000000000004},"min":{"1min":0.45600000000000002,"5min":0.45600000000000002,"15min":0.45600000000000002},"max":{"1min":4.1689999999999996,"5min":4.1689999999999996,"15min":4.1689999999999996},"last":0.622},{"interface":"front","average":{"1min":1.002,"5min":1.002,"15min":1.002},"min":{"1min":0.47499999999999998,"5min":0.47499999999999998,"15min":0.47499999999999998},"max":{"1min":7.7889999999999997,"5min":7.7889999999999997,"15min":7.7889999999999997},"last":0.499}]},{"osd":1,"last update":"Sun Nov 26 07:18:46 2023","interfaces":[{"interface":"back","average":{"1min":0.64700000000000002,"5min":0.64700000000000002,"15min":0.64700000000000002},"min":{"1min":0.52300000000000002,"5min":0.52300000000000002,"15min":0.52300000000000002},"max":{"1min":0.78400000000000003,"5min":0.78400000000000003,"15min":0.78400000000000003},"last":0.55500000000000005},{"interface":"front","average":{"1min":0.81999999999999995,"5min":0.81999999999999995,"15min":0.81999999999999995},"min":{"1min":0.44500000000000001,"5min":0.44500000000000001,"15min":0.44500000000000001},"max":{"1min":4.3140000000000001,"5min":4.3140000000000001,"15min":4.3140000000000001},"last":0.65700000000000003}]},{"osd":2,"last update":"Sun Nov 26 07:18:46 2023","interfaces":[{"interface":"back","average":{"1min":0.80700000000000005,"5min":0.80700000000000005,"15min":0.80700000000000005},"min":{"1min":0.57799999999999996,"5min":0.57799999999999996,"15min":0.57799999999999996},"max":{"1min":2.7850000000000001,"5min":2.7850000000000001,"15min":2.7850000000000001},"last":0.69299999999999995},{"interface":"front","average":{"1min":1.0660000000000001,"5min":1.0660000000000001,"15min":1.0660000000000001},"min":{"1min":0.53400000000000003,"5min":0.53400000000000003,"15min":0.53400000000000003},"max":{"1min":7.6470000000000002,"5min":7.6470000000000002,"15min":7.6470000000000002},"last":0.68000000000000005}]},{"osd":3,"last update":"Sun Nov 26 07:18:46 2023","interfaces":[{"interface":"back","average":{"1min":0.72999999999999998,"5min":0.72999999999999998,"15min":0.72999999999999998},"min":{"1min":0.48599999999999999,"5min":0.48599999999999999,"15min":0.48599999999999999},"max":{"1min":1.591,"5min":1.591,"15min":1.591},"last":0.58599999999999997},{"interface":"front","average":{"1min":0.879,"5min":0.879,"15min":0.879},"min":{"1min":0.56899999999999995,"5min":0.56899999999999995,"15min":0.56899999999999995},"max":{"1min":3.452,"5min":3.452,"15min":3.452},"last":0.72899999999999998}]},{"osd":5,"last update":"Sun Nov 26 07:18:46 2023","interfaces":[{"interface":"back","average":{"1min":0.61099999999999999,"5min":0.61099999999999999,"15min":0.61099999999999999},"min":{"1min":0.43099999999999999,"5min":0.43099999999999999,"15min":0.43099999999999999},"max":{"1min":0.96199999999999997,"5min":0.96199999999999997,"15min":0.96199999999999997},"last":0.64500000000000002},{"interface":"front","average":{"1min":0.64100000000000001,"5min":0.64100000000000001,"15min":0.64100000000000001},"min":{"1min":0.41499999999999998,"5min":0.41499999999999998,"15min":0.41499999999999998},"max":{"1min":1.538,"5min":1.538,"15min":1.538},"last":0.47099999999999997}]},{"osd":6,"last update":"Sun Nov 26 07:18:46 2023","interfaces":[{"interface":"back","average":{"1min":0.79400000000000004,"5min":0.79400000000000004,"15min":0.79400000000000004},"min":{"1min":0.39400000000000002,"5min":0.39400000000000002,"15min":0.39400000000000002},"max":{"1min":2.8330000000000002,"5min":2.8330000000000002,"15min":2.8330000000000002},"last":0.79100000000000004},{"interface":"front","average":{"1min":1.0089999999999999,"5min":1.0089999999999999,"15min":1.0089999999999999},"min":{"1min":0.42899999999999999,"5min":0.42899999999999999,"15min":0.42899999999999999},"max":{"1min":7.6050000000000004,"5min":7.6050000000000004,"15min":7.6050000000000004},"last":0.60099999999999998}]},{"osd":7,"last update":"Sun Nov 26 07:18:46 2023","interfaces":[{"interface":"back","average":{"1min":0.86099999999999999,"5min":0.86099999999999999,"15min":0.86099999999999999},"min":{"1min":0.44700000000000001,"5min":0.44700000000000001,"15min":0.44700000000000001},"max":{"1min":4.3689999999999998,"5min":4.3689999999999998,"15min":4.3689999999999998},"last":0.53600000000000003},{"interface":"front","average":{"1min":0.89200000000000002,"5min":0.89200000000000002,"15min":0.89200000000000002},"min":{"1min":0.505,"5min":0.505,"15min":0.505},"max":{"1min":3.403,"5min":3.403,"15min":3.403},"last":0.76300000000000001}]}]},{"osd":3,"up_from":13,"seq":55834574875,"num_pgs":7,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27512,"kb_used_data":492,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344328,"statfs":{"total":96636764160,"available":96608591872,"internally_reserved":0,"allocated":503808,"data_stored":202024,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1592,"internal_metadata":27654600},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Nov 26 07:18:43 2023","interfaces":[{"interface":"back","average":{"1min":0.379,"5min":0.379,"15min":0.379},"min":{"1min":0.26100000000000001,"5min":0.26100000000000001,"15min":0.26100000000000001},"max":{"1min":0.58599999999999997,"5min":0.58599999999999997,"15min":0.58599999999999997},"last":0.34999999999999998},{"interface":"front","average":{"1min":0.433,"5min":0.433,"15min":0.433},"min":{"1min":0.30199999999999999,"5min":0.30199999999999999,"15min":0.30199999999999999},"max":{"1min":0.63,"5min":0.63,"15min":0.63},"last":0.503}]},{"osd":1,"last update":"Sun Nov 26 07:18:43 2023","interfaces":[{"interface":"back","average":{"1min":0.42099999999999999,"5min":0.42099999999999999,"15min":0.42099999999999999},"min":{"1min":0.312,"5min":0.312,"15min":0.312},"max":{"1min":0.752,"5min":0.752,"15min":0.752},"last":0.40300000000000002},{"interface":"front","average":{"1min":0.48399999999999999,"5min":0.48399999999999999,"15min":0.48399999999999999},"min":{"1min":0.38,"5min":0.38,"15min":0.38},"max":{"1min":0.628,"5min":0.628,"15min":0.628},"last":0.54100000000000004}]},{"osd":2,"last update":"Sun Nov 26 07:18:43 2023","interfaces":[{"interface":"back","average":{"1min":0.53300000000000003,"5min":0.53300000000000003,"15min":0.53300000000000003},"min":{"1min":0.40300000000000002,"5min":0.40300000000000002,"15min":0.40300000000000002},"max":{"1min":0.82199999999999995,"5min":0.82199999999999995,"15min":0.82199999999999995},"last":0.57399999999999995},{"interface":"front","average":{"1min":0.505,"5min":0.505,"15min":0.505},"min":{"1min":0.35999999999999999,"5min":0.35999999999999999,"15min":0.35999999999999999},"max":{"1min":0.72199999999999998,"5min":0.72199999999999998,"15min":0.72199999999999998},"last":0.46700000000000003}]},{"osd":4,"last update":"Sun Nov 26 07:18:43 2023","interfaces":[{"interface":"back","average":{"1min":0.70599999999999996,"5min":0.70599999999999996,"15min":0.70599999999999996},"min":{"1min":0.48199999999999998,"5min":0.48199999999999998,"15min":0.48199999999999998},"max":{"1min":0.96199999999999997,"5min":0.96199999999999997,"15min":0.96199999999999997},"last":0.81200000000000006},{"interface":"front","average":{"1min":0.72799999999999998,"5min":0.72799999999999998,"15min":0.72799999999999998},"min":{"1min":0.441,"5min":0.441,"15min":0.441},"max":{"1min":1.171,"5min":1.171,"15min":1.171},"last":0.74299999999999999}]},{"osd":5,"last update":"Sun Nov 26 07:18:43 2023","interfaces":[{"interface":"back","average":{"1min":0.73899999999999999,"5min":0.73899999999999999,"15min":0.73899999999999999},"min":{"1min":0.53700000000000003,"5min":0.53700000000000003,"15min":0.53700000000000003},"max":{"1min":0.94099999999999995,"5min":0.94099999999999995,"15min":0.94099999999999995},"last":0.68000000000000005},{"interface":"front","average":{"1min":0.73799999999999999,"5min":0.73799999999999999,"15min":0.73799999999999999},"min":{"1min":0.504,"5min":0.504,"15min":0.504},"max":{"1min":1.0820000000000001,"5min":1.0820000000000001,"15min":1.0820000000000001},"last":0.71599999999999997}]},{"osd":6,"last update":"Sun Nov 26 07:18:43 2023","interfaces":[{"interface":"back","average":{"1min":0.81399999999999995,"5min":0.81399999999999995,"15min":0.81399999999999995},"min":{"1min":0.55200000000000005,"5min":0.55200000000000005,"15min":0.55200000000000005},"max":{"1min":1.2729999999999999,"5min":1.2729999999999999,"15min":1.2729999999999999},"last":0.83599999999999997},{"interface":"front","average":{"1min":0.79300000000000004,"5min":0.79300000000000004,"15min":0.79300000000000004},"min":{"1min":0.47299999999999998,"5min":0.47299999999999998,"15min":0.47299999999999998},"max":{"1min":1.204,"5min":1.204,"15min":1.204},"last":0.75900000000000001}]},{"osd":7,"last update":"Sun Nov 26 07:18:43 2023","interfaces":[{"interface":"back","average":{"1min":0.80100000000000005,"5min":0.80100000000000005,"15min":0.80100000000000005},"min":{"1min":0.63300000000000001,"5min":0.63300000000000001,"15min":0.63300000000000001},"max":{"1min":1.3600000000000001,"5min":1.3600000000000001,"15min":1.3600000000000001},"last":0.63900000000000001},{"interface":"front","average":{"1min":0.82599999999999996,"5min":0.82599999999999996,"15min":0.82599999999999996},"min":{"1min":0.50600000000000001,"5min":0.50600000000000001,"15min":0.50600000000000001},"max":{"1min":1.617,"5min":1.617,"15min":1.617},"last":0.79300000000000004}]}]},{"osd":2,"up_from":13,"seq":55834574875,"num_pgs":11,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27512,"kb_used_data":492,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344328,"statfs":{"total":96636764160,"available":96608591872,"internally_reserved":0,"allocated":503808,"data_stored":202024,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Nov 26 07:18:43 2023","interfaces":[{"interface":"back","average":{"1min":0.71399999999999997,"5min":0.71399999999999997,"15min":0.71399999999999997},"min":{"1min":0.28699999999999998,"5min":0.28699999999999998,"15min":0.28699999999999998},"max":{"1min":4.7030000000000003,"5min":4.7030000000000003,"15min":4.7030000000000003},"last":0.41099999999999998},{"interface":"front","average":{"1min":0.56200000000000006,"5min":0.56200000000000006,"15min":0.56200000000000006},"min":{"1min":0.32300000000000001,"5min":0.32300000000000001,"15min":0.32300000000000001},"max":{"1min":2.4340000000000002,"5min":2.4340000000000002,"15min":2.4340000000000002},"last":0.438}]},{"osd":1,"last update":"Sun Nov 26 07:18:43 2023","interfaces":[{"interface":"back","average":{"1min":0.86399999999999999,"5min":0.86399999999999999,"15min":0.86399999999999999},"min":{"1min":0.34599999999999997,"5min":0.34599999999999997,"15min":0.34599999999999997},"max":{"1min":4.2690000000000001,"5min":4.2690000000000001,"15min":4.2690000000000001},"last":0.46300000000000002},{"interface":"front","average":{"1min":0.74199999999999999,"5min":0.74199999999999999,"15min":0.74199999999999999},"min":{"1min":0.31900000000000001,"5min":0.31900000000000001,"15min":0.31900000000000001},"max":{"1min":3.9940000000000002,"5min":3.9940000000000002,"15min":3.9940000000000002},"last":0.38800000000000001}]},{"osd":3,"last update":"Sun Nov 26 07:18:43 2023","interfaces":[{"interface":"back","average":{"1min":0.64300000000000002,"5min":0.64300000000000002,"15min":0.64300000000000002},"min":{"1min":0.34399999999999997,"5min":0.34399999999999997,"15min":0.34399999999999997},"max":{"1min":4.0300000000000002,"5min":4.0300000000000002,"15min":4.0300000000000002},"last":0.36899999999999999},{"interface":"front","average":{"1min":0.91400000000000003,"5min":0.91400000000000003,"15min":0.91400000000000003},"min":{"1min":0.41699999999999998,"5min":0.41699999999999998,"15min":0.41699999999999998},"max":{"1min":4.5810000000000004,"5min":4.5810000000000004,"15min":4.5810000000000004},"last":0.50900000000000001}]},{"osd":4,"last update":"Sun Nov 26 07:18:43 2023","interfaces":[{"interface":"back","average":{"1min":0.98499999999999999,"5min":0.98499999999999999,"15min":0.98499999999999999},"min":{"1min":0.55100000000000005,"5min":0.55100000000000005,"15min":0.55100000000000005},"max":{"1min":4.4269999999999996,"5min":4.4269999999999996,"15min":4.4269999999999996},"last":0.80900000000000005},{"interface":"front","average":{"1min":1.0149999999999999,"5min":1.0149999999999999,"15min":1.0149999999999999},"min":{"1min":0.60299999999999998,"5min":0.60299999999999998,"15min":0.60299999999999998},"max":{"1min":4.085,"5min":4.085,"15min":4.085},"last":0.68100000000000005}]},{"osd":5,"last update":"Sun Nov 26 07:18:43 2023","interfaces":[{"interface":"back","average":{"1min":0.874,"5min":0.874,"15min":0.874},"min":{"1min":0.52900000000000003,"5min":0.52900000000000003,"15min":0.52900000000000003},"max":{"1min":3.9590000000000001,"5min":3.9590000000000001,"15min":3.9590000000000001},"last":0.65000000000000002},{"interface":"front","average":{"1min":0.98799999999999999,"5min":0.98799999999999999,"15min":0.98799999999999999},"min":{"1min":0.58799999999999997,"5min":0.58799999999999997,"15min":0.58799999999999997},"max":{"1min":4.3849999999999998,"5min":4.3849999999999998,"15min":4.3849999999999998},"last":0.752}]},{"osd":6,"last update":"Sun Nov 26 07:18:43 2023","interfaces":[{"interface":"back","average":{"1min":0.996,"5min":0.996,"15min":0.996},"min":{"1min":0.56599999999999995,"5min":0.56599999999999995,"15min":0.56599999999999995},"max":{"1min":4.0449999999999999,"5min":4.0449999999999999,"15min":4.0449999999999999},"last":0.77700000000000002},{"interface":"front","average":{"1min":0.91800000000000004,"5min":0.91800000000000004,"15min":0.91800000000000004},"min":{"1min":0.57099999999999995,"5min":0.57099999999999995,"15min":0.57099999999999995},"max":{"1min":3.992,"5min":3.992,"15min":3.992},"last":0.60699999999999998}]},{"osd":7,"last update":"Sun Nov 26 07:18:43 2023","interfaces":[{"interface":"back","average":{"1min":1.0349999999999999,"5min":1.0349999999999999,"15min":1.0349999999999999},"min":{"1min":0.63100000000000001,"5min":0.63100000000000001,"15min":0.63100000000000001},"max":{"1min":4.1219999999999999,"5min":4.1219999999999999,"15min":4.1219999999999999},"last":0.63},{"interface":"front","average":{"1min":0.93799999999999994,"5min":0.93799999999999994,"15min":0.93799999999999994},"min":{"1min":0.57699999999999996,"5min":0.57699999999999996,"15min":0.57699999999999996},"max":{"1min":3.9159999999999999,"5min":3.9159999999999999,"15min":3.9159999999999999},"last":0.73599999999999999}]}]},{"osd":1,"up_from":13,"seq":55834574874,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27512,"kb_used_data":492,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344328,"statfs":{"total":96636764160,"available":96608591872,"internally_reserved":0,"allocated":503808,"data_stored":202024,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Nov 26 07:18:41 2023","interfaces":[{"interface":"back","average":{"1min":0.41699999999999998,"5min":0.41699999999999998,"15min":0.41699999999999998},"min":{"1min":0.26300000000000001,"5min":0.26300000000000001,"15min":0.26300000000000001},"max":{"1min":0.61199999999999999,"5min":0.61199999999999999,"15min":0.61199999999999999},"last":0.36299999999999999},{"interface":"front","average":{"1min":0.48599999999999999,"5min":0.48599999999999999,"15min":0.48599999999999999},"min":{"1min":0.30499999999999999,"5min":0.30499999999999999,"15min":0.30499999999999999},"max":{"1min":0.78700000000000003,"5min":0.78700000000000003,"15min":0.78700000000000003},"last":0.433}]},{"osd":2,"last update":"Sun Nov 26 07:18:41 2023","interfaces":[{"interface":"back","average":{"1min":0.45500000000000002,"5min":0.45500000000000002,"15min":0.45500000000000002},"min":{"1min":0.33400000000000002,"5min":0.33400000000000002,"15min":0.33400000000000002},"max":{"1min":0.67600000000000005,"5min":0.67600000000000005,"15min":0.67600000000000005},"last":0.47099999999999997},{"interface":"front","average":{"1min":0.50800000000000001,"5min":0.50800000000000001,"15min":0.50800000000000001},"min":{"1min":0.32300000000000001,"5min":0.32300000000000001,"15min":0.32300000000000001},"max":{"1min":0.66500000000000004,"5min":0.66500000000000004,"15min":0.66500000000000004},"last":0.64400000000000002}]},{"osd":3,"last update":"Sun Nov 26 07:18:41 2023","interfaces":[{"interface":"back","average":{"1min":0.53000000000000003,"5min":0.53000000000000003,"15min":0.53000000000000003},"min":{"1min":0.38400000000000001,"5min":0.38400000000000001,"15min":0.38400000000000001},"max":{"1min":0.73699999999999999,"5min":0.73699999999999999,"15min":0.73699999999999999},"last":0.495},{"interface":"front","average":{"1min":0.497,"5min":0.497,"15min":0.497},"min":{"1min":0.36199999999999999,"5min":0.36199999999999999,"15min":0.36199999999999999},"max":{"1min":0.80700000000000005,"5min":0.80700000000000005,"15min":0.80700000000000005},"last":0.51500000000000001}]},{"osd":4,"last update":"Sun Nov 26 07:18:41 2023","interfaces":[{"interface":"back","average":{"1min":0.69399999999999995,"5min":0.69399999999999995,"15min":0.69399999999999995},"min":{"1min":0.48599999999999999,"5min":0.48599999999999999,"15min":0.48599999999999999},"max":{"1min":1.0229999999999999,"5min":1.0229999999999999,"15min":1.0229999999999999},"last":0.56599999999999995},{"interface":"front","average":{"1min":0.68000000000000005,"5min":0.68000000000000005,"15min":0.68000000000000005},"min":{"1min":0.49199999999999999,"5min":0.49199999999999999,"15min":0.49199999999999999},"max":{"1min":1.0629999999999999,"5min":1.0629999999999999,"15min":1.0629999999999999},"last":0.71399999999999997}]},{"osd":5,"last update":"Sun Nov 26 07:18:41 2023","interfaces":[{"interface":"back","average":{"1min":0.71899999999999997,"5min":0.71899999999999997,"15min":0.71899999999999997},"min":{"1min":0.49299999999999999,"5min":0.49299999999999999,"15min":0.49299999999999999},"max":{"1min":1.0880000000000001,"5min":1.0880000000000001,"15min":1.0880000000000001},"last":0.625},{"interface":"front","average":{"1min":0.71699999999999997,"5min":0.71699999999999997,"15min":0.71699999999999997},"min":{"1min":0.53500000000000003,"5min":0.53500000000000003,"15min":0.53500000000000003},"max":{"1min":1.014,"5min":1.014,"15min":1.014},"last":0.70099999999999996}]},{"osd":6,"last update":"Sun Nov 26 07:18:41 2023","interfaces":[{"interface":"back","average":{"1min":0.70399999999999996,"5min":0.70399999999999996,"15min":0.70399999999999996},"min":{"1min":0.50600000000000001,"5min":0.50600000000000001,"15min":0.50600000000000001},"max":{"1min":0.97599999999999998,"5min":0.97599999999999998,"15min":0.97599999999999998},"last":0.72899999999999998},{"interface":"front","average":{"1min":0.70099999999999996,"5min":0.70099999999999996,"15min":0.70099999999999996},"min":{"1min":0.55000000000000004,"5min":0.55000000000000004,"15min":0.55000000000000004},"max":{"1min":1.1339999999999999,"5min":1.1339999999999999,"15min":1.1339999999999999},"last":0.58699999999999997}]},{"osd":7,"last update":"Sun Nov 26 07:18:41 2023","interfaces":[{"interface":"back","average":{"1min":0.71499999999999997,"5min":0.71499999999999997,"15min":0.71499999999999997},"min":{"1min":0.55100000000000005,"5min":0.55100000000000005,"15min":0.55100000000000005},"max":{"1min":0.92900000000000005,"5min":0.92900000000000005,"15min":0.92900000000000005},"last":0.74099999999999999},{"interface":"front","average":{"1min":0.753,"5min":0.753,"15min":0.753},"min":{"1min":0.56100000000000005,"5min":0.56100000000000005,"15min":0.56100000000000005},"max":{"1min":1.024,"5min":1.024,"15min":1.024},"last":0.68200000000000005}]}]},{"osd":0,"up_from":13,"seq":55834574874,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":28080,"kb_used_data":1060,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343760,"statfs":{"total":96636764160,"available":96608010240,"internally_reserved":0,"allocated":1085440,"data_stored":786125,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Sun Nov 26 07:18:41 2023","interfaces":[{"interface":"back","average":{"1min":0.43099999999999999,"5min":0.43099999999999999,"15min":0.43099999999999999},"min":{"1min":0.28000000000000003,"5min":0.28000000000000003,"15min":0.28000000000000003},"max":{"1min":0.63100000000000001,"5min":0.63100000000000001,"15min":0.63100000000000001},"last":0.40799999999999997},{"interface":"front","average":{"1min":0.48299999999999998,"5min":0.48299999999999998,"15min":0.48299999999999998},"min":{"1min":0.29899999999999999,"5min":0.29899999999999999,"15min":0.29899999999999999},"max":{"1min":1.133,"5min":1.133,"15min":1.133},"last":0.35399999999999998}]},{"osd":2,"last update":"Sun Nov 26 07:18:41 2023","interfaces":[{"interface":"back","average":{"1min":0.53700000000000003,"5min":0.53700000000000003,"15min":0.53700000000000003},"min":{"1min":0.40899999999999997,"5min":0.40899999999999997,"15min":0.40899999999999997},"max":{"1min":0.67400000000000004,"5min":0.67400000000000004,"15min":0.67400000000000004},"last":0.56299999999999994},{"interface":"front","average":{"1min":0.56399999999999995,"5min":0.56399999999999995,"15min":0.56399999999999995},"min":{"1min":0.36399999999999999,"5min":0.36399999999999999,"15min":0.36399999999999999},"max":{"1min":0.72499999999999998,"5min":0.72499999999999998,"15min":0.72499999999999998},"last":0.48599999999999999}]},{"osd":3,"last update":"Sun Nov 26 07:18:41 2023","interfaces":[{"interface":"back","average":{"1min":0.53100000000000003,"5min":0.53100000000000003,"15min":0.53100000000000003},"min":{"1min":0.35799999999999998,"5min":0.35799999999999998,"15min":0.35799999999999998},"max":{"1min":0.97399999999999998,"5min":0.97399999999999998,"15min":0.97399999999999998},"last":1.3149999999999999},{"interface":"front","average":{"1min":0.56200000000000006,"5min":0.56200000000000006,"15min":0.56200000000000006},"min":{"1min":0.40600000000000003,"5min":0.40600000000000003,"15min":0.40600000000000003},"max":{"1min":1.2509999999999999,"5min":1.2509999999999999,"15min":1.2509999999999999},"last":0.65100000000000002}]},{"osd":4,"last update":"Sun Nov 26 07:18:41 2023","interfaces":[{"interface":"back","average":{"1min":0.75900000000000001,"5min":0.75900000000000001,"15min":0.75900000000000001},"min":{"1min":0.60999999999999999,"5min":0.60999999999999999,"15min":0.60999999999999999},"max":{"1min":1.2010000000000001,"5min":1.2010000000000001,"15min":1.2010000000000001},"last":0.60999999999999999},{"interface":"front","average":{"1min":0.78900000000000003,"5min":0.78900000000000003,"15min":0.78900000000000003},"min":{"1min":0.59999999999999998,"5min":0.59999999999999998,"15min":0.59999999999999998},"max":{"1min":0.99399999999999999,"5min":0.99399999999999999,"15min":0.99399999999999999},"last":1.381}]},{"osd":5,"last update":"Sun Nov 26 07:18:41 2023","interfaces":[{"interface":"back","average":{"1min":0.76000000000000001,"5min":0.76000000000000001,"15min":0.76000000000000001},"min":{"1min":0.53000000000000003,"5min":0.53000000000000003,"15min":0.53000000000000003},"max":{"1min":1.1579999999999999,"5min":1.1579999999999999,"15min":1.1579999999999999},"last":0.72699999999999998},{"interface":"front","average":{"1min":0.79700000000000004,"5min":0.79700000000000004,"15min":0.79700000000000004},"min":{"1min":0.59699999999999998,"5min":0.59699999999999998,"15min":0.59699999999999998},"max":{"1min":1.0409999999999999,"5min":1.0409999999999999,"15min":1.0409999999999999},"last":1.399}]},{"osd":6,"last update":"Sun Nov 26 07:18:41 2023","interfaces":[{"interface":"back","average":{"1min":0.76400000000000001,"5min":0.76400000000000001,"15min":0.76400000000000001},"min":{"1min":0.56599999999999995,"5min":0.56599999999999995,"15min":0.56599999999999995},"max":{"1min":1.3029999999999999,"5min":1.3029999999999999,"15min":1.3029999999999999},"last":0.69599999999999995},{"interface":"front","average":{"1min":0.76900000000000002,"5min":0.76900000000000002,"15min":0.76900000000000002},"min":{"1min":0.58599999999999997,"5min":0.58599999999999997,"15min":0.58599999999999997},"max":{"1min":1.2210000000000001,"5min":1.2210000000000001,"15min":1.2210000000000001},"last":1.3600000000000001}]},{"osd":7,"last update":"Sun Nov 26 07:18:41 2023","interfaces":[{"interface":"back","average":{"1min":0.81100000000000005,"5min":0.81100000000000005,"15min":0.81100000000000005},"min":{"1min":0.59999999999999998,"5min":0.59999999999999998,"15min":0.59999999999999998},"max":{"1min":1.177,"5min":1.177,"15min":1.177},"last":0.84099999999999997},{"interface":"front","average":{"1min":0.79900000000000004,"5min":0.79900000000000004,"15min":0.79900000000000004},"min":{"1min":0.60899999999999999,"5min":0.60899999999999999,"15min":0.60899999999999999},"max":{"1min":1.2390000000000001,"5min":1.2390000000000001,"15min":1.2390000000000001},"last":0.67300000000000004}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-11-26T07:19:36.502 DEBUG:teuthology.orchestra.run.smithi143:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-11-26T07:19:36.682 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:36.679+0000 7ff57a69c640 1 Processor -- start 2023-11-26T07:19:36.682 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:36.679+0000 7ff57a69c640 1 -- start start 2023-11-26T07:19:36.684 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:36.679+0000 7ff57a69c640 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7ff5740d0330 0x7ff5740d3de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:19:36.684 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:36.679+0000 7ff57a69c640 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7ff5740b20e0 0x7ff5740b2dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:19:36.684 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:36.679+0000 7ff57a69c640 1 -- --> v1:172.21.15.188:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff5740ce9d0 con 0x7ff574160d30 2023-11-26T07:19:36.684 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:36.679+0000 7ff57a69c640 1 -- --> v2:172.21.15.143:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff5740cf550 con 0x7ff5740b20e0 2023-11-26T07:19:36.684 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:36.679+0000 7ff57a69c640 1 -- --> v2:172.21.15.188:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff574161110 con 0x7ff5740d0330 2023-11-26T07:19:36.684 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:36.679+0000 7ff578c39640 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7ff5740b20e0 0x7ff5740b2dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:19:36.684 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:36.679+0000 7ff578c39640 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7ff5740b20e0 0x7ff5740b2dd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:52220/0 (socket says 172.21.15.143:52220) 2023-11-26T07:19:36.684 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:36.679+0000 7ff578c39640 1 -- 172.21.15.143:0/3431257089 learned_addr learned my addr 172.21.15.143:0/3431257089 (peer_addr_for_me v2:172.21.15.143:0/0) 2023-11-26T07:19:36.684 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:36.679+0000 7ff5737fe640 1 --2- 172.21.15.143:0/3431257089 >> v2:172.21.15.188:3300/0 conn(0x7ff5740d0330 0x7ff5740d3de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:19:36.685 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:36.679+0000 7ff578c39640 1 -- 172.21.15.143:0/3431257089 >> v2:172.21.15.188:3300/0 conn(0x7ff5740d0330 msgr2=0x7ff5740d3de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-26T07:19:36.685 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:36.679+0000 7ff578c39640 1 --2- 172.21.15.143:0/3431257089 >> v2:172.21.15.188:3300/0 conn(0x7ff5740d0330 0x7ff5740d3de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:19:36.685 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:36.679+0000 7ff578c39640 1 -- 172.21.15.143:0/3431257089 >> v1:172.21.15.188:6789/0 conn(0x7ff574160d30 legacy=0x7ff5740cfc20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:19:36.685 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:36.679+0000 7ff578c39640 1 -- 172.21.15.143:0/3431257089 --> v2:172.21.15.143:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff5740cecb0 con 0x7ff5740b20e0 2023-11-26T07:19:36.687 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:36.683+0000 7ff578c39640 1 --2- 172.21.15.143:0/3431257089 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7ff5740b20e0 0x7ff5740b2dd0 secure :-1 s=READY pgs=579 cs=0 l=1 rev1=1 crypto rx=0x7ff5740b5e00 tx=0x7ff56c01c9a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=90514a7f50a126ef server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:19:36.687 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:36.683+0000 7ff572ffd640 1 -- 172.21.15.143:0/3431257089 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff56c0a5860 con 0x7ff5740b20e0 2023-11-26T07:19:36.687 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:36.683+0000 7ff572ffd640 1 -- 172.21.15.143:0/3431257089 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff56c0a59c0 con 0x7ff5740b20e0 2023-11-26T07:19:36.687 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:36.683+0000 7ff572ffd640 1 -- 172.21.15.143:0/3431257089 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff56c0a5d50 con 0x7ff5740b20e0 2023-11-26T07:19:36.688 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:36.683+0000 7ff57a69c640 1 -- 172.21.15.143:0/3431257089 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7ff5740b20e0 msgr2=0x7ff5740b2dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:19:36.688 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:36.683+0000 7ff57a69c640 1 --2- 172.21.15.143:0/3431257089 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7ff5740b20e0 0x7ff5740b2dd0 secure :-1 s=READY pgs=579 cs=0 l=1 rev1=1 crypto rx=0x7ff5740b5e00 tx=0x7ff56c01c9a0 comp rx=0 tx=0).stop 2023-11-26T07:19:36.688 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:36.683+0000 7ff57a69c640 1 -- 172.21.15.143:0/3431257089 shutdown_connections 2023-11-26T07:19:36.688 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:36.683+0000 7ff57a69c640 1 --2- 172.21.15.143:0/3431257089 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7ff5740b20e0 0x7ff5740b2dd0 unknown :-1 s=CLOSED pgs=579 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:19:36.688 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:36.683+0000 7ff57a69c640 1 --2- 172.21.15.143:0/3431257089 >> v2:172.21.15.188:3300/0 conn(0x7ff5740d0330 0x7ff5740d3de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:19:36.689 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:36.683+0000 7ff57a69c640 1 -- 172.21.15.143:0/3431257089 >> 172.21.15.143:0/3431257089 conn(0x7ff57409c560 msgr2=0x7ff5740d3210 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:19:36.689 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:36.683+0000 7ff57a69c640 1 -- 172.21.15.143:0/3431257089 shutdown_connections 2023-11-26T07:19:36.689 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:36.683+0000 7ff57a69c640 1 -- 172.21.15.143:0/3431257089 wait complete. 2023-11-26T07:19:36.689 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:36.687+0000 7ff57a69c640 1 Processor -- start 2023-11-26T07:19:36.689 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:36.687+0000 7ff57a69c640 1 -- start start 2023-11-26T07:19:36.691 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:36.687+0000 7ff57a69c640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7ff5740b20e0 0x7ff5740b7600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:19:36.691 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:36.687+0000 7ff57a69c640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7ff5740d0330 0x7ff5741542e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:19:36.691 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:36.687+0000 7ff57a69c640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7ff574160d30 0x7ff5740cc4b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:19:36.692 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:36.687+0000 7ff57a69c640 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff5740b63c0 con 0x7ff5740b20e0 2023-11-26T07:19:36.692 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:36.687+0000 7ff57a69c640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff5740b66a0 con 0x7ff574160d30 2023-11-26T07:19:36.692 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:36.687+0000 7ff57a69c640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff5740b6980 con 0x7ff5740d0330 2023-11-26T07:19:36.692 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:36.687+0000 7ff573fff640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7ff5740b20e0 0x7ff5740b7600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:19:36.692 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:36.687+0000 7ff573fff640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7ff5740b20e0 0x7ff5740b7600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:52234/0 (socket says 172.21.15.143:52234) 2023-11-26T07:19:36.692 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:36.687+0000 7ff573fff640 1 -- 172.21.15.143:0/4200860822 learned_addr learned my addr 172.21.15.143:0/4200860822 (peer_addr_for_me v2:172.21.15.143:0/0) 2023-11-26T07:19:36.692 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:36.687+0000 7ff5737fe640 1 --2- 172.21.15.143:0/4200860822 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7ff5740d0330 0x7ff5741542e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:19:36.692 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:36.687+0000 7ff578c39640 1 --2- 172.21.15.143:0/4200860822 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7ff574160d30 0x7ff5740cc4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:19:36.692 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:36.687+0000 7ff573fff640 1 -- 172.21.15.143:0/4200860822 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7ff5740d0330 msgr2=0x7ff5741542e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:19:36.692 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:36.687+0000 7ff573fff640 1 --2- 172.21.15.143:0/4200860822 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7ff5740d0330 0x7ff5741542e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:19:36.692 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:36.687+0000 7ff573fff640 1 -- 172.21.15.143:0/4200860822 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7ff574160d30 msgr2=0x7ff5740cc4b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:19:36.692 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:36.687+0000 7ff573fff640 1 --2- 172.21.15.143:0/4200860822 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7ff574160d30 0x7ff5740cc4b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:19:36.693 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:36.687+0000 7ff573fff640 1 -- 172.21.15.143:0/4200860822 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff56c009b30 con 0x7ff5740b20e0 2023-11-26T07:19:36.693 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:36.687+0000 7ff573fff640 1 --2- 172.21.15.143:0/4200860822 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7ff5740b20e0 0x7ff5740b7600 secure :-1 s=READY pgs=580 cs=0 l=1 rev1=1 crypto rx=0x7ff56400c3c0 tx=0x7ff56400c880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:19:36.693 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:36.687+0000 7ff570ff9640 1 -- 172.21.15.143:0/4200860822 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff5640052d0 con 0x7ff5740b20e0 2023-11-26T07:19:36.694 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:36.687+0000 7ff570ff9640 1 -- 172.21.15.143:0/4200860822 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff564005430 con 0x7ff5740b20e0 2023-11-26T07:19:36.694 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:36.687+0000 7ff57a69c640 1 -- 172.21.15.143:0/4200860822 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff5740b6c60 con 0x7ff5740b20e0 2023-11-26T07:19:36.694 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:36.687+0000 7ff570ff9640 1 -- 172.21.15.143:0/4200860822 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff564003840 con 0x7ff5740b20e0 2023-11-26T07:19:36.694 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:36.687+0000 7ff57a69c640 1 -- 172.21.15.143:0/4200860822 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff57415f780 con 0x7ff5740b20e0 2023-11-26T07:19:36.694 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:36.691+0000 7ff57a69c640 1 -- 172.21.15.143:0/4200860822 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff574067680 con 0x7ff5740b20e0 2023-11-26T07:19:36.694 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:36.691+0000 7ff570ff9640 1 -- 172.21.15.143:0/4200860822 <== mon.0 v2:172.21.15.143:3300/0 4 ==== mgrmap(e 7) v1 ==== 86005+0+0 (secure 0 0 0) 0x7ff5640039a0 con 0x7ff5740b20e0 2023-11-26T07:19:36.695 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:36.691+0000 7ff570ff9640 1 --2- 172.21.15.143:0/4200860822 >> [v2:172.21.15.143:6832/2150557544,v1:172.21.15.143:6833/2150557544] conn(0x7ff560066620 0x7ff560068ae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:19:36.695 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:36.691+0000 7ff570ff9640 1 -- 172.21.15.143:0/4200860822 <== mon.0 v2:172.21.15.143:3300/0 5 ==== osd_map(45..45 src has 1..45) v4 ==== 5709+0+0 (secure 0 0 0) 0x7ff564083600 con 0x7ff5740b20e0 2023-11-26T07:19:36.695 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:36.691+0000 7ff5737fe640 1 --2- 172.21.15.143:0/4200860822 >> [v2:172.21.15.143:6832/2150557544,v1:172.21.15.143:6833/2150557544] conn(0x7ff560066620 0x7ff560068ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:19:36.695 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:36.691+0000 7ff5737fe640 1 --2- 172.21.15.143:0/4200860822 >> [v2:172.21.15.143:6832/2150557544,v1:172.21.15.143:6833/2150557544] conn(0x7ff560066620 0x7ff560068ae0 secure :-1 s=READY pgs=291 cs=0 l=1 rev1=1 crypto rx=0x7ff5740ca1a0 tx=0x7ff568009190 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:19:36.698 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:36.695+0000 7ff570ff9640 1 -- 172.21.15.143:0/4200860822 <== mon.0 v2:172.21.15.143:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172649 (secure 0 0 0) 0x7ff5640508c0 con 0x7ff5740b20e0 2023-11-26T07:19:36.849 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:36.847+0000 7ff57a69c640 1 -- 172.21.15.143:0/4200860822 --> [v2:172.21.15.143:6832/2150557544,v1:172.21.15.143:6833/2150557544] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7ff5740d3e70 con 0x7ff560066620 2023-11-26T07:19:36.856 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:36.851+0000 7ff570ff9640 1 -- 172.21.15.143:0/4200860822 <== mgr.4100 v2:172.21.15.143:6832/2150557544 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+284367 (secure 0 0 0) 0x7ff5740d3e70 con 0x7ff560066620 2023-11-26T07:19:36.859 INFO:teuthology.orchestra.run.smithi143.stdout: 2023-11-26T07:19:36.861 INFO:teuthology.orchestra.run.smithi143.stderr:dumped all 2023-11-26T07:19:36.862 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:36.855+0000 7ff57a69c640 1 -- 172.21.15.143:0/4200860822 >> [v2:172.21.15.143:6832/2150557544,v1:172.21.15.143:6833/2150557544] conn(0x7ff560066620 msgr2=0x7ff560068ae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:19:36.862 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:36.855+0000 7ff57a69c640 1 --2- 172.21.15.143:0/4200860822 >> [v2:172.21.15.143:6832/2150557544,v1:172.21.15.143:6833/2150557544] conn(0x7ff560066620 0x7ff560068ae0 secure :-1 s=READY pgs=291 cs=0 l=1 rev1=1 crypto rx=0x7ff5740ca1a0 tx=0x7ff568009190 comp rx=0 tx=0).stop 2023-11-26T07:19:36.862 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:36.855+0000 7ff57a69c640 1 -- 172.21.15.143:0/4200860822 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7ff5740b20e0 msgr2=0x7ff5740b7600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:19:36.862 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:36.855+0000 7ff57a69c640 1 --2- 172.21.15.143:0/4200860822 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7ff5740b20e0 0x7ff5740b7600 secure :-1 s=READY pgs=580 cs=0 l=1 rev1=1 crypto rx=0x7ff56400c3c0 tx=0x7ff56400c880 comp rx=0 tx=0).stop 2023-11-26T07:19:36.862 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:36.855+0000 7ff57a69c640 1 -- 172.21.15.143:0/4200860822 shutdown_connections 2023-11-26T07:19:36.862 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:36.855+0000 7ff57a69c640 1 --2- 172.21.15.143:0/4200860822 >> [v2:172.21.15.143:6832/2150557544,v1:172.21.15.143:6833/2150557544] conn(0x7ff560066620 0x7ff560068ae0 unknown :-1 s=CLOSED pgs=291 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:19:36.863 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:36.855+0000 7ff57a69c640 1 --2- 172.21.15.143:0/4200860822 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7ff574160d30 0x7ff5740cc4b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:19:36.863 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:36.855+0000 7ff57a69c640 1 --2- 172.21.15.143:0/4200860822 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7ff5740d0330 0x7ff5741542e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:19:36.863 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:36.855+0000 7ff57a69c640 1 --2- 172.21.15.143:0/4200860822 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7ff5740b20e0 0x7ff5740b7600 unknown :-1 s=CLOSED pgs=580 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:19:36.863 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:36.855+0000 7ff57a69c640 1 -- 172.21.15.143:0/4200860822 >> 172.21.15.143:0/4200860822 conn(0x7ff57409c560 msgr2=0x7ff57415fab0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:19:36.863 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:36.859+0000 7ff57a69c640 1 -- 172.21.15.143:0/4200860822 shutdown_connections 2023-11-26T07:19:36.863 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:36.859+0000 7ff57a69c640 1 -- 172.21.15.143:0/4200860822 wait complete. 2023-11-26T07:19:36.879 INFO:teuthology.orchestra.run.smithi143.stdout:{"pg_ready":true,"pg_map":{"version":95,"stamp":"2023-11-26T07:19:35.712433+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":24,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":221240,"kb_used_data":5072,"kb_used_omap":12,"kb_used_meta":216051,"kb_avail":754753480,"statfs":{"total":773094113280,"available":772867563520,"internally_reserved":0,"allocated":5193728,"data_stored":2784394,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12749,"internal_metadata":221236787},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":32,"apply_latency_ms":32,"commit_latency_ns":32000000,"apply_latency_ns":32000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"5.598663"},"pg_stats":[{"pgid":"7.3f","version":"0'0","reported_seq":1,"reported_epoch":43,"state":"creating+peering","last_fresh":"2023-11-26T07:19:31.135164+0000","last_change":"2023-11-26T07:19:31.135164+0000","last_active":"2023-11-26T07:19:31.113498+0000","last_peered":"2023-11-26T07:19:31.113498+0000","last_clean":"2023-11-26T07:19:31.113498+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-26T07:19:31.135164+0000","last_undegraded":"2023-11-26T07:19:31.135164+0000","last_fullsized":"2023-11-26T07:19:31.135164+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:19:31.113498+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:19:31.113498+0000","last_clean_scrub_stamp":"2023-11-26T07:19:31.113498+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.3e","version":"0'0","reported_seq":1,"reported_epoch":43,"state":"creating+peering","last_fresh":"2023-11-26T07:19:31.129025+0000","last_change":"2023-11-26T07:19:31.129025+0000","last_active":"2023-11-26T07:19:31.113498+0000","last_peered":"2023-11-26T07:19:31.113498+0000","last_clean":"2023-11-26T07:19:31.113498+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-26T07:19:31.129025+0000","last_undegraded":"2023-11-26T07:19:31.129025+0000","last_fullsized":"2023-11-26T07:19:31.129025+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:19:31.113498+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:19:31.113498+0000","last_clean_scrub_stamp":"2023-11-26T07:19:31.113498+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.3d","version":"0'0","reported_seq":1,"reported_epoch":43,"state":"creating+peering","last_fresh":"2023-11-26T07:19:31.125533+0000","last_change":"2023-11-26T07:19:31.125533+0000","last_active":"2023-11-26T07:19:31.113498+0000","last_peered":"2023-11-26T07:19:31.113498+0000","last_clean":"2023-11-26T07:19:31.113498+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-26T07:19:31.125533+0000","last_undegraded":"2023-11-26T07:19:31.125533+0000","last_fullsized":"2023-11-26T07:19:31.125533+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:19:31.113498+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:19:31.113498+0000","last_clean_scrub_stamp":"2023-11-26T07:19:31.113498+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.3c","version":"0'0","reported_seq":1,"reported_epoch":43,"state":"creating+peering","last_fresh":"2023-11-26T07:19:31.129825+0000","last_change":"2023-11-26T07:19:31.129825+0000","last_active":"2023-11-26T07:19:31.113498+0000","last_peered":"2023-11-26T07:19:31.113498+0000","last_clean":"2023-11-26T07:19:31.113498+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-26T07:19:31.129825+0000","last_undegraded":"2023-11-26T07:19:31.129825+0000","last_fullsized":"2023-11-26T07:19:31.129825+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:19:31.113498+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:19:31.113498+0000","last_clean_scrub_stamp":"2023-11-26T07:19:31.113498+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.3b","version":"0'0","reported_seq":1,"reported_epoch":43,"state":"creating+peering","last_fresh":"2023-11-26T07:19:31.128964+0000","last_change":"2023-11-26T07:19:31.128964+0000","last_active":"2023-11-26T07:19:31.113498+0000","last_peered":"2023-11-26T07:19:31.113498+0000","last_clean":"2023-11-26T07:19:31.113498+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-26T07:19:31.128964+0000","last_undegraded":"2023-11-26T07:19:31.128964+0000","last_fullsized":"2023-11-26T07:19:31.128964+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:19:31.113498+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:19:31.113498+0000","last_clean_scrub_stamp":"2023-11-26T07:19:31.113498+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.3a","version":"0'0","reported_seq":1,"reported_epoch":43,"state":"creating+peering","last_fresh":"2023-11-26T07:19:31.138400+0000","last_change":"2023-11-26T07:19:31.138400+0000","last_active":"2023-11-26T07:19:31.113498+0000","last_peered":"2023-11-26T07:19:31.113498+0000","last_clean":"2023-11-26T07:19:31.113498+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-26T07:19:31.138400+0000","last_undegraded":"2023-11-26T07:19:31.138400+0000","last_fullsized":"2023-11-26T07:19:31.138400+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:19:31.113498+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:19:31.113498+0000","last_clean_scrub_stamp":"2023-11-26T07:19:31.113498+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.39","version":"0'0","reported_seq":1,"reported_epoch":43,"state":"creating+peering","last_fresh":"2023-11-26T07:19:31.134099+0000","last_change":"2023-11-26T07:19:31.134099+0000","last_active":"2023-11-26T07:19:31.113498+0000","last_peered":"2023-11-26T07:19:31.113498+0000","last_clean":"2023-11-26T07:19:31.113498+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-26T07:19:31.134099+0000","last_undegraded":"2023-11-26T07:19:31.134099+0000","last_fullsized":"2023-11-26T07:19:31.134099+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:19:31.113498+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:19:31.113498+0000","last_clean_scrub_stamp":"2023-11-26T07:19:31.113498+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.38","version":"0'0","reported_seq":1,"reported_epoch":43,"state":"creating+peering","last_fresh":"2023-11-26T07:19:31.130100+0000","last_change":"2023-11-26T07:19:31.130100+0000","last_active":"2023-11-26T07:19:31.113498+0000","last_peered":"2023-11-26T07:19:31.113498+0000","last_clean":"2023-11-26T07:19:31.113498+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-26T07:19:31.130100+0000","last_undegraded":"2023-11-26T07:19:31.130100+0000","last_fullsized":"2023-11-26T07:19:31.130100+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:19:31.113498+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:19:31.113498+0000","last_clean_scrub_stamp":"2023-11-26T07:19:31.113498+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.37","version":"0'0","reported_seq":1,"reported_epoch":43,"state":"creating+peering","last_fresh":"2023-11-26T07:19:31.129317+0000","last_change":"2023-11-26T07:19:31.129317+0000","last_active":"2023-11-26T07:19:31.113498+0000","last_peered":"2023-11-26T07:19:31.113498+0000","last_clean":"2023-11-26T07:19:31.113498+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-26T07:19:31.129317+0000","last_undegraded":"2023-11-26T07:19:31.129317+0000","last_fullsized":"2023-11-26T07:19:31.129317+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:19:31.113498+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:19:31.113498+0000","last_clean_scrub_stamp":"2023-11-26T07:19:31.113498+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.36","version":"0'0","reported_seq":1,"reported_epoch":43,"state":"creating+peering","last_fresh":"2023-11-26T07:19:31.130104+0000","last_change":"2023-11-26T07:19:31.130104+0000","last_active":"2023-11-26T07:19:31.113498+0000","last_peered":"2023-11-26T07:19:31.113498+0000","last_clean":"2023-11-26T07:19:31.113498+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-26T07:19:31.130104+0000","last_undegraded":"2023-11-26T07:19:31.130104+0000","last_fullsized":"2023-11-26T07:19:31.130104+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:19:31.113498+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:19:31.113498+0000","last_clean_scrub_stamp":"2023-11-26T07:19:31.113498+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.35","version":"0'0","reported_seq":1,"reported_epoch":43,"state":"creating+peering","last_fresh":"2023-11-26T07:19:31.130252+0000","last_change":"2023-11-26T07:19:31.130252+0000","last_active":"2023-11-26T07:19:31.113498+0000","last_peered":"2023-11-26T07:19:31.113498+0000","last_clean":"2023-11-26T07:19:31.113498+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-26T07:19:31.130252+0000","last_undegraded":"2023-11-26T07:19:31.130252+0000","last_fullsized":"2023-11-26T07:19:31.130252+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:19:31.113498+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:19:31.113498+0000","last_clean_scrub_stamp":"2023-11-26T07:19:31.113498+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.34","version":"0'0","reported_seq":1,"reported_epoch":43,"state":"creating+peering","last_fresh":"2023-11-26T07:19:31.131089+0000","last_change":"2023-11-26T07:19:31.131089+0000","last_active":"2023-11-26T07:19:31.113498+0000","last_peered":"2023-11-26T07:19:31.113498+0000","last_clean":"2023-11-26T07:19:31.113498+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-26T07:19:31.131089+0000","last_undegraded":"2023-11-26T07:19:31.131089+0000","last_fullsized":"2023-11-26T07:19:31.131089+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:19:31.113498+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:19:31.113498+0000","last_clean_scrub_stamp":"2023-11-26T07:19:31.113498+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.33","version":"0'0","reported_seq":1,"reported_epoch":43,"state":"creating+peering","last_fresh":"2023-11-26T07:19:31.132990+0000","last_change":"2023-11-26T07:19:31.132990+0000","last_active":"2023-11-26T07:19:31.113498+0000","last_peered":"2023-11-26T07:19:31.113498+0000","last_clean":"2023-11-26T07:19:31.113498+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-26T07:19:31.132990+0000","last_undegraded":"2023-11-26T07:19:31.132990+0000","last_fullsized":"2023-11-26T07:19:31.132990+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:19:31.113498+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:19:31.113498+0000","last_clean_scrub_stamp":"2023-11-26T07:19:31.113498+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.32","version":"0'0","reported_seq":1,"reported_epoch":43,"state":"creating+peering","last_fresh":"2023-11-26T07:19:31.129881+0000","last_change":"2023-11-26T07:19:31.129881+0000","last_active":"2023-11-26T07:19:31.113498+0000","last_peered":"2023-11-26T07:19:31.113498+0000","last_clean":"2023-11-26T07:19:31.113498+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-26T07:19:31.129881+0000","last_undegraded":"2023-11-26T07:19:31.129881+0000","last_fullsized":"2023-11-26T07:19:31.129881+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:19:31.113498+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:19:31.113498+0000","last_clean_scrub_stamp":"2023-11-26T07:19:31.113498+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.31","version":"0'0","reported_seq":1,"reported_epoch":43,"state":"creating+peering","last_fresh":"2023-11-26T07:19:31.138475+0000","last_change":"2023-11-26T07:19:31.138475+0000","last_active":"2023-11-26T07:19:31.113498+0000","last_peered":"2023-11-26T07:19:31.113498+0000","last_clean":"2023-11-26T07:19:31.113498+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-26T07:19:31.138475+0000","last_undegraded":"2023-11-26T07:19:31.138475+0000","last_fullsized":"2023-11-26T07:19:31.138475+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:19:31.113498+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:19:31.113498+0000","last_clean_scrub_stamp":"2023-11-26T07:19:31.113498+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.30","version":"0'0","reported_seq":1,"reported_epoch":43,"state":"creating+peering","last_fresh":"2023-11-26T07:19:31.131093+0000","last_change":"2023-11-26T07:19:31.131093+0000","last_active":"2023-11-26T07:19:31.113498+0000","last_peered":"2023-11-26T07:19:31.113498+0000","last_clean":"2023-11-26T07:19:31.113498+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-26T07:19:31.131093+0000","last_undegraded":"2023-11-26T07:19:31.131093+0000","last_fullsized":"2023-11-26T07:19:31.131093+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:19:31.113498+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:19:31.113498+0000","last_clean_scrub_stamp":"2023-11-26T07:19:31.113498+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.2f","version":"0'0","reported_seq":1,"reported_epoch":43,"state":"creating+peering","last_fresh":"2023-11-26T07:19:31.125773+0000","last_change":"2023-11-26T07:19:31.125773+0000","last_active":"2023-11-26T07:19:31.113498+0000","last_peered":"2023-11-26T07:19:31.113498+0000","last_clean":"2023-11-26T07:19:31.113498+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-26T07:19:31.125773+0000","last_undegraded":"2023-11-26T07:19:31.125773+0000","last_fullsized":"2023-11-26T07:19:31.125773+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:19:31.113498+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:19:31.113498+0000","last_clean_scrub_stamp":"2023-11-26T07:19:31.113498+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.2e","version":"0'0","reported_seq":1,"reported_epoch":43,"state":"creating+peering","last_fresh":"2023-11-26T07:19:31.126790+0000","last_change":"2023-11-26T07:19:31.126790+0000","last_active":"2023-11-26T07:19:31.113498+0000","last_peered":"2023-11-26T07:19:31.113498+0000","last_clean":"2023-11-26T07:19:31.113498+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-26T07:19:31.126790+0000","last_undegraded":"2023-11-26T07:19:31.126790+0000","last_fullsized":"2023-11-26T07:19:31.126790+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:19:31.113498+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:19:31.113498+0000","last_clean_scrub_stamp":"2023-11-26T07:19:31.113498+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.2d","version":"0'0","reported_seq":1,"reported_epoch":43,"state":"creating+peering","last_fresh":"2023-11-26T07:19:31.133954+0000","last_change":"2023-11-26T07:19:31.133954+0000","last_active":"2023-11-26T07:19:31.113498+0000","last_peered":"2023-11-26T07:19:31.113498+0000","last_clean":"2023-11-26T07:19:31.113498+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-26T07:19:31.133954+0000","last_undegraded":"2023-11-26T07:19:31.133954+0000","last_fullsized":"2023-11-26T07:19:31.133954+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:19:31.113498+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:19:31.113498+0000","last_clean_scrub_stamp":"2023-11-26T07:19:31.113498+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.2c","version":"0'0","reported_seq":1,"reported_epoch":43,"state":"creating+peering","last_fresh":"2023-11-26T07:19:31.135122+0000","last_change":"2023-11-26T07:19:31.135122+0000","last_active":"2023-11-26T07:19:31.113498+0000","last_peered":"2023-11-26T07:19:31.113498+0000","last_clean":"2023-11-26T07:19:31.113498+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-26T07:19:31.135122+0000","last_undegraded":"2023-11-26T07:19:31.135122+0000","last_fullsized":"2023-11-26T07:19:31.135122+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:19:31.113498+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:19:31.113498+0000","last_clean_scrub_stamp":"2023-11-26T07:19:31.113498+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.2b","version":"0'0","reported_seq":1,"reported_epoch":43,"state":"creating+peering","last_fresh":"2023-11-26T07:19:31.137790+0000","last_change":"2023-11-26T07:19:31.137790+0000","last_active":"2023-11-26T07:19:31.113498+0000","last_peered":"2023-11-26T07:19:31.113498+0000","last_clean":"2023-11-26T07:19:31.113498+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-26T07:19:31.137790+0000","last_undegraded":"2023-11-26T07:19:31.137790+0000","last_fullsized":"2023-11-26T07:19:31.137790+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:19:31.113498+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:19:31.113498+0000","last_clean_scrub_stamp":"2023-11-26T07:19:31.113498+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.2a","version":"0'0","reported_seq":1,"reported_epoch":43,"state":"creating+peering","last_fresh":"2023-11-26T07:19:31.129388+0000","last_change":"2023-11-26T07:19:31.129388+0000","last_active":"2023-11-26T07:19:31.113498+0000","last_peered":"2023-11-26T07:19:31.113498+0000","last_clean":"2023-11-26T07:19:31.113498+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-26T07:19:31.129388+0000","last_undegraded":"2023-11-26T07:19:31.129388+0000","last_fullsized":"2023-11-26T07:19:31.129388+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:19:31.113498+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:19:31.113498+0000","last_clean_scrub_stamp":"2023-11-26T07:19:31.113498+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.29","version":"0'0","reported_seq":1,"reported_epoch":43,"state":"creating+peering","last_fresh":"2023-11-26T07:19:31.131006+0000","last_change":"2023-11-26T07:19:31.131006+0000","last_active":"2023-11-26T07:19:31.113498+0000","last_peered":"2023-11-26T07:19:31.113498+0000","last_clean":"2023-11-26T07:19:31.113498+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-26T07:19:31.131006+0000","last_undegraded":"2023-11-26T07:19:31.131006+0000","last_fullsized":"2023-11-26T07:19:31.131006+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:19:31.113498+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:19:31.113498+0000","last_clean_scrub_stamp":"2023-11-26T07:19:31.113498+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.28","version":"0'0","reported_seq":1,"reported_epoch":43,"state":"creating+peering","last_fresh":"2023-11-26T07:19:31.130446+0000","last_change":"2023-11-26T07:19:31.130446+0000","last_active":"2023-11-26T07:19:31.113498+0000","last_peered":"2023-11-26T07:19:31.113498+0000","last_clean":"2023-11-26T07:19:31.113498+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-26T07:19:31.130446+0000","last_undegraded":"2023-11-26T07:19:31.130446+0000","last_fullsized":"2023-11-26T07:19:31.130446+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:19:31.113498+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:19:31.113498+0000","last_clean_scrub_stamp":"2023-11-26T07:19:31.113498+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.27","version":"0'0","reported_seq":1,"reported_epoch":43,"state":"creating+peering","last_fresh":"2023-11-26T07:19:31.134474+0000","last_change":"2023-11-26T07:19:31.134474+0000","last_active":"2023-11-26T07:19:31.113498+0000","last_peered":"2023-11-26T07:19:31.113498+0000","last_clean":"2023-11-26T07:19:31.113498+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-26T07:19:31.134474+0000","last_undegraded":"2023-11-26T07:19:31.134474+0000","last_fullsized":"2023-11-26T07:19:31.134474+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:19:31.113498+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:19:31.113498+0000","last_clean_scrub_stamp":"2023-11-26T07:19:31.113498+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.26","version":"0'0","reported_seq":1,"reported_epoch":43,"state":"creating+peering","last_fresh":"2023-11-26T07:19:31.132954+0000","last_change":"2023-11-26T07:19:31.132954+0000","last_active":"2023-11-26T07:19:31.113498+0000","last_peered":"2023-11-26T07:19:31.113498+0000","last_clean":"2023-11-26T07:19:31.113498+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-26T07:19:31.132954+0000","last_undegraded":"2023-11-26T07:19:31.132954+0000","last_fullsized":"2023-11-26T07:19:31.132954+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:19:31.113498+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:19:31.113498+0000","last_clean_scrub_stamp":"2023-11-26T07:19:31.113498+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.25","version":"0'0","reported_seq":1,"reported_epoch":43,"state":"creating+peering","last_fresh":"2023-11-26T07:19:31.124787+0000","last_change":"2023-11-26T07:19:31.124787+0000","last_active":"2023-11-26T07:19:31.113498+0000","last_peered":"2023-11-26T07:19:31.113498+0000","last_clean":"2023-11-26T07:19:31.113498+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-26T07:19:31.124787+0000","last_undegraded":"2023-11-26T07:19:31.124787+0000","last_fullsized":"2023-11-26T07:19:31.124787+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:19:31.113498+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:19:31.113498+0000","last_clean_scrub_stamp":"2023-11-26T07:19:31.113498+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.24","version":"0'0","reported_seq":1,"reported_epoch":43,"state":"creating+peering","last_fresh":"2023-11-26T07:19:31.130640+0000","last_change":"2023-11-26T07:19:31.130640+0000","last_active":"2023-11-26T07:19:31.113498+0000","last_peered":"2023-11-26T07:19:31.113498+0000","last_clean":"2023-11-26T07:19:31.113498+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-26T07:19:31.130640+0000","last_undegraded":"2023-11-26T07:19:31.130640+0000","last_fullsized":"2023-11-26T07:19:31.130640+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:19:31.113498+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:19:31.113498+0000","last_clean_scrub_stamp":"2023-11-26T07:19:31.113498+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.23","version":"0'0","reported_seq":1,"reported_epoch":43,"state":"creating+peering","last_fresh":"2023-11-26T07:19:31.131030+0000","last_change":"2023-11-26T07:19:31.131030+0000","last_active":"2023-11-26T07:19:31.113498+0000","last_peered":"2023-11-26T07:19:31.113498+0000","last_clean":"2023-11-26T07:19:31.113498+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-26T07:19:31.131030+0000","last_undegraded":"2023-11-26T07:19:31.131030+0000","last_fullsized":"2023-11-26T07:19:31.131030+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:19:31.113498+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:19:31.113498+0000","last_clean_scrub_stamp":"2023-11-26T07:19:31.113498+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.22","version":"0'0","reported_seq":1,"reported_epoch":43,"state":"creating+peering","last_fresh":"2023-11-26T07:19:31.126109+0000","last_change":"2023-11-26T07:19:31.126109+0000","last_active":"2023-11-26T07:19:31.113498+0000","last_peered":"2023-11-26T07:19:31.113498+0000","last_clean":"2023-11-26T07:19:31.113498+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-26T07:19:31.126109+0000","last_undegraded":"2023-11-26T07:19:31.126109+0000","last_fullsized":"2023-11-26T07:19:31.126109+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:19:31.113498+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:19:31.113498+0000","last_clean_scrub_stamp":"2023-11-26T07:19:31.113498+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.21","version":"0'0","reported_seq":1,"reported_epoch":43,"state":"creating+peering","last_fresh":"2023-11-26T07:19:31.134817+0000","last_change":"2023-11-26T07:19:31.134817+0000","last_active":"2023-11-26T07:19:31.113498+0000","last_peered":"2023-11-26T07:19:31.113498+0000","last_clean":"2023-11-26T07:19:31.113498+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-26T07:19:31.134817+0000","last_undegraded":"2023-11-26T07:19:31.134817+0000","last_fullsized":"2023-11-26T07:19:31.134817+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:19:31.113498+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:19:31.113498+0000","last_clean_scrub_stamp":"2023-11-26T07:19:31.113498+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.20","version":"0'0","reported_seq":1,"reported_epoch":43,"state":"creating+peering","last_fresh":"2023-11-26T07:19:31.127285+0000","last_change":"2023-11-26T07:19:31.127285+0000","last_active":"2023-11-26T07:19:31.113498+0000","last_peered":"2023-11-26T07:19:31.113498+0000","last_clean":"2023-11-26T07:19:31.113498+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-26T07:19:31.127285+0000","last_undegraded":"2023-11-26T07:19:31.127285+0000","last_fullsized":"2023-11-26T07:19:31.127285+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:19:31.113498+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:19:31.113498+0000","last_clean_scrub_stamp":"2023-11-26T07:19:31.113498+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.1e","version":"0'0","reported_seq":1,"reported_epoch":43,"state":"creating+peering","last_fresh":"2023-11-26T07:19:31.133294+0000","last_change":"2023-11-26T07:19:31.133294+0000","last_active":"2023-11-26T07:19:31.113498+0000","last_peered":"2023-11-26T07:19:31.113498+0000","last_clean":"2023-11-26T07:19:31.113498+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-26T07:19:31.133294+0000","last_undegraded":"2023-11-26T07:19:31.133294+0000","last_fullsized":"2023-11-26T07:19:31.133294+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:19:31.113498+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:19:31.113498+0000","last_clean_scrub_stamp":"2023-11-26T07:19:31.113498+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.1f","version":"0'0","reported_seq":9,"reported_epoch":43,"state":"active+clean","last_fresh":"2023-11-26T07:19:31.153093+0000","last_change":"2023-11-26T07:19:31.148359+0000","last_active":"2023-11-26T07:19:31.153093+0000","last_peered":"2023-11-26T07:19:31.153093+0000","last_clean":"2023-11-26T07:19:31.153093+0000","last_became_active":"2023-11-26T07:19:31.148066+0000","last_became_peered":"2023-11-26T07:19:31.148066+0000","last_unstale":"2023-11-26T07:19:31.153093+0000","last_undegraded":"2023-11-26T07:19:31.153093+0000","last_fullsized":"2023-11-26T07:19:31.153093+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:19:30.106639+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:19:30.106639+0000","last_clean_scrub_stamp":"2023-11-26T07:19:30.106639+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T15:38:28.537385+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.1f","version":"0'0","reported_seq":1,"reported_epoch":43,"state":"creating+peering","last_fresh":"2023-11-26T07:19:31.134010+0000","last_change":"2023-11-26T07:19:31.134010+0000","last_active":"2023-11-26T07:19:31.113498+0000","last_peered":"2023-11-26T07:19:31.113498+0000","last_clean":"2023-11-26T07:19:31.113498+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-26T07:19:31.134010+0000","last_undegraded":"2023-11-26T07:19:31.134010+0000","last_fullsized":"2023-11-26T07:19:31.134010+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:19:31.113498+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:19:31.113498+0000","last_clean_scrub_stamp":"2023-11-26T07:19:31.113498+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.1e","version":"0'0","reported_seq":9,"reported_epoch":43,"state":"active+clean","last_fresh":"2023-11-26T07:19:31.149264+0000","last_change":"2023-11-26T07:19:31.144305+0000","last_active":"2023-11-26T07:19:31.149264+0000","last_peered":"2023-11-26T07:19:31.149264+0000","last_clean":"2023-11-26T07:19:31.149264+0000","last_became_active":"2023-11-26T07:19:31.143063+0000","last_became_peered":"2023-11-26T07:19:31.143063+0000","last_unstale":"2023-11-26T07:19:31.149264+0000","last_undegraded":"2023-11-26T07:19:31.149264+0000","last_fullsized":"2023-11-26T07:19:31.149264+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:19:30.106639+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:19:30.106639+0000","last_clean_scrub_stamp":"2023-11-26T07:19:30.106639+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T08:28:59.053370+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.1c","version":"0'0","reported_seq":1,"reported_epoch":43,"state":"creating+peering","last_fresh":"2023-11-26T07:19:31.129959+0000","last_change":"2023-11-26T07:19:31.129959+0000","last_active":"2023-11-26T07:19:31.113498+0000","last_peered":"2023-11-26T07:19:31.113498+0000","last_clean":"2023-11-26T07:19:31.113498+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-26T07:19:31.129959+0000","last_undegraded":"2023-11-26T07:19:31.129959+0000","last_fullsized":"2023-11-26T07:19:31.129959+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:19:31.113498+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:19:31.113498+0000","last_clean_scrub_stamp":"2023-11-26T07:19:31.113498+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.1d","version":"0'0","reported_seq":9,"reported_epoch":43,"state":"active+clean","last_fresh":"2023-11-26T07:19:31.153679+0000","last_change":"2023-11-26T07:19:31.150313+0000","last_active":"2023-11-26T07:19:31.153679+0000","last_peered":"2023-11-26T07:19:31.153679+0000","last_clean":"2023-11-26T07:19:31.153679+0000","last_became_active":"2023-11-26T07:19:31.150042+0000","last_became_peered":"2023-11-26T07:19:31.150042+0000","last_unstale":"2023-11-26T07:19:31.153679+0000","last_undegraded":"2023-11-26T07:19:31.153679+0000","last_fullsized":"2023-11-26T07:19:31.153679+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:19:30.106639+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:19:30.106639+0000","last_clean_scrub_stamp":"2023-11-26T07:19:30.106639+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T16:20:23.315619+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.1d","version":"0'0","reported_seq":1,"reported_epoch":43,"state":"creating+peering","last_fresh":"2023-11-26T07:19:31.134552+0000","last_change":"2023-11-26T07:19:31.134552+0000","last_active":"2023-11-26T07:19:31.113498+0000","last_peered":"2023-11-26T07:19:31.113498+0000","last_clean":"2023-11-26T07:19:31.113498+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-26T07:19:31.134552+0000","last_undegraded":"2023-11-26T07:19:31.134552+0000","last_fullsized":"2023-11-26T07:19:31.134552+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:19:31.113498+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:19:31.113498+0000","last_clean_scrub_stamp":"2023-11-26T07:19:31.113498+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.1c","version":"0'0","reported_seq":9,"reported_epoch":43,"state":"active+clean","last_fresh":"2023-11-26T07:19:31.147334+0000","last_change":"2023-11-26T07:19:31.141667+0000","last_active":"2023-11-26T07:19:31.147334+0000","last_peered":"2023-11-26T07:19:31.147334+0000","last_clean":"2023-11-26T07:19:31.147334+0000","last_became_active":"2023-11-26T07:19:31.141393+0000","last_became_peered":"2023-11-26T07:19:31.141393+0000","last_unstale":"2023-11-26T07:19:31.147334+0000","last_undegraded":"2023-11-26T07:19:31.147334+0000","last_fullsized":"2023-11-26T07:19:31.147334+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:19:30.106639+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:19:30.106639+0000","last_clean_scrub_stamp":"2023-11-26T07:19:30.106639+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T10:27:01.949013+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.1a","version":"0'0","reported_seq":1,"reported_epoch":43,"state":"creating+peering","last_fresh":"2023-11-26T07:19:31.125537+0000","last_change":"2023-11-26T07:19:31.125537+0000","last_active":"2023-11-26T07:19:31.113498+0000","last_peered":"2023-11-26T07:19:31.113498+0000","last_clean":"2023-11-26T07:19:31.113498+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-26T07:19:31.125537+0000","last_undegraded":"2023-11-26T07:19:31.125537+0000","last_fullsized":"2023-11-26T07:19:31.125537+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:19:31.113498+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:19:31.113498+0000","last_clean_scrub_stamp":"2023-11-26T07:19:31.113498+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.1b","version":"0'0","reported_seq":9,"reported_epoch":43,"state":"active+clean","last_fresh":"2023-11-26T07:19:31.153191+0000","last_change":"2023-11-26T07:19:31.149354+0000","last_active":"2023-11-26T07:19:31.153191+0000","last_peered":"2023-11-26T07:19:31.153191+0000","last_clean":"2023-11-26T07:19:31.153191+0000","last_became_active":"2023-11-26T07:19:31.149092+0000","last_became_peered":"2023-11-26T07:19:31.149092+0000","last_unstale":"2023-11-26T07:19:31.153191+0000","last_undegraded":"2023-11-26T07:19:31.153191+0000","last_fullsized":"2023-11-26T07:19:31.153191+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:19:30.106639+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:19:30.106639+0000","last_clean_scrub_stamp":"2023-11-26T07:19:30.106639+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T13:58:09.825490+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.1b","version":"0'0","reported_seq":1,"reported_epoch":43,"state":"creating+peering","last_fresh":"2023-11-26T07:19:31.131194+0000","last_change":"2023-11-26T07:19:31.131194+0000","last_active":"2023-11-26T07:19:31.113498+0000","last_peered":"2023-11-26T07:19:31.113498+0000","last_clean":"2023-11-26T07:19:31.113498+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-26T07:19:31.131194+0000","last_undegraded":"2023-11-26T07:19:31.131194+0000","last_fullsized":"2023-11-26T07:19:31.131194+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:19:31.113498+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:19:31.113498+0000","last_clean_scrub_stamp":"2023-11-26T07:19:31.113498+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.1a","version":"0'0","reported_seq":9,"reported_epoch":43,"state":"active+clean","last_fresh":"2023-11-26T07:19:31.142739+0000","last_change":"2023-11-26T07:19:31.139455+0000","last_active":"2023-11-26T07:19:31.142739+0000","last_peered":"2023-11-26T07:19:31.142739+0000","last_clean":"2023-11-26T07:19:31.142739+0000","last_became_active":"2023-11-26T07:19:31.139009+0000","last_became_peered":"2023-11-26T07:19:31.139009+0000","last_unstale":"2023-11-26T07:19:31.142739+0000","last_undegraded":"2023-11-26T07:19:31.142739+0000","last_fullsized":"2023-11-26T07:19:31.142739+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:19:30.106639+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:19:30.106639+0000","last_clean_scrub_stamp":"2023-11-26T07:19:30.106639+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T13:51:18.697846+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.18","version":"0'0","reported_seq":1,"reported_epoch":43,"state":"creating+peering","last_fresh":"2023-11-26T07:19:31.128550+0000","last_change":"2023-11-26T07:19:31.128550+0000","last_active":"2023-11-26T07:19:31.113498+0000","last_peered":"2023-11-26T07:19:31.113498+0000","last_clean":"2023-11-26T07:19:31.113498+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-26T07:19:31.128550+0000","last_undegraded":"2023-11-26T07:19:31.128550+0000","last_fullsized":"2023-11-26T07:19:31.128550+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:19:31.113498+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:19:31.113498+0000","last_clean_scrub_stamp":"2023-11-26T07:19:31.113498+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.19","version":"0'0","reported_seq":9,"reported_epoch":43,"state":"active+clean","last_fresh":"2023-11-26T07:19:31.150741+0000","last_change":"2023-11-26T07:19:31.146327+0000","last_active":"2023-11-26T07:19:31.150741+0000","last_peered":"2023-11-26T07:19:31.150741+0000","last_clean":"2023-11-26T07:19:31.150741+0000","last_became_active":"2023-11-26T07:19:31.145985+0000","last_became_peered":"2023-11-26T07:19:31.145985+0000","last_unstale":"2023-11-26T07:19:31.150741+0000","last_undegraded":"2023-11-26T07:19:31.150741+0000","last_fullsized":"2023-11-26T07:19:31.150741+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:19:30.106639+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:19:30.106639+0000","last_clean_scrub_stamp":"2023-11-26T07:19:30.106639+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T12:39:06.522107+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.19","version":"0'0","reported_seq":1,"reported_epoch":43,"state":"creating+peering","last_fresh":"2023-11-26T07:19:31.131606+0000","last_change":"2023-11-26T07:19:31.131606+0000","last_active":"2023-11-26T07:19:31.113498+0000","last_peered":"2023-11-26T07:19:31.113498+0000","last_clean":"2023-11-26T07:19:31.113498+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-26T07:19:31.131606+0000","last_undegraded":"2023-11-26T07:19:31.131606+0000","last_fullsized":"2023-11-26T07:19:31.131606+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:19:31.113498+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:19:31.113498+0000","last_clean_scrub_stamp":"2023-11-26T07:19:31.113498+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.18","version":"0'0","reported_seq":9,"reported_epoch":43,"state":"active+clean","last_fresh":"2023-11-26T07:19:31.147314+0000","last_change":"2023-11-26T07:19:31.140258+0000","last_active":"2023-11-26T07:19:31.147314+0000","last_peered":"2023-11-26T07:19:31.147314+0000","last_clean":"2023-11-26T07:19:31.147314+0000","last_became_active":"2023-11-26T07:19:31.140009+0000","last_became_peered":"2023-11-26T07:19:31.140009+0000","last_unstale":"2023-11-26T07:19:31.147314+0000","last_undegraded":"2023-11-26T07:19:31.147314+0000","last_fullsized":"2023-11-26T07:19:31.147314+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:19:30.106639+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:19:30.106639+0000","last_clean_scrub_stamp":"2023-11-26T07:19:30.106639+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T13:51:18.697846+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.16","version":"0'0","reported_seq":1,"reported_epoch":43,"state":"creating+peering","last_fresh":"2023-11-26T07:19:31.126247+0000","last_change":"2023-11-26T07:19:31.126247+0000","last_active":"2023-11-26T07:19:31.113498+0000","last_peered":"2023-11-26T07:19:31.113498+0000","last_clean":"2023-11-26T07:19:31.113498+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-26T07:19:31.126247+0000","last_undegraded":"2023-11-26T07:19:31.126247+0000","last_fullsized":"2023-11-26T07:19:31.126247+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:19:31.113498+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:19:31.113498+0000","last_clean_scrub_stamp":"2023-11-26T07:19:31.113498+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.17","version":"0'0","reported_seq":9,"reported_epoch":43,"state":"active+clean","last_fresh":"2023-11-26T07:19:31.141806+0000","last_change":"2023-11-26T07:19:31.134552+0000","last_active":"2023-11-26T07:19:31.141806+0000","last_peered":"2023-11-26T07:19:31.141806+0000","last_clean":"2023-11-26T07:19:31.141806+0000","last_became_active":"2023-11-26T07:19:31.134241+0000","last_became_peered":"2023-11-26T07:19:31.134241+0000","last_unstale":"2023-11-26T07:19:31.141806+0000","last_undegraded":"2023-11-26T07:19:31.141806+0000","last_fullsized":"2023-11-26T07:19:31.141806+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:19:30.106639+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:19:30.106639+0000","last_clean_scrub_stamp":"2023-11-26T07:19:30.106639+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T07:36:26.843027+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.17","version":"0'0","reported_seq":1,"reported_epoch":43,"state":"creating+peering","last_fresh":"2023-11-26T07:19:31.129608+0000","last_change":"2023-11-26T07:19:31.129608+0000","last_active":"2023-11-26T07:19:31.113498+0000","last_peered":"2023-11-26T07:19:31.113498+0000","last_clean":"2023-11-26T07:19:31.113498+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-26T07:19:31.129608+0000","last_undegraded":"2023-11-26T07:19:31.129608+0000","last_fullsized":"2023-11-26T07:19:31.129608+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:19:31.113498+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:19:31.113498+0000","last_clean_scrub_stamp":"2023-11-26T07:19:31.113498+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.16","version":"0'0","reported_seq":9,"reported_epoch":43,"state":"active+clean","last_fresh":"2023-11-26T07:19:31.151072+0000","last_change":"2023-11-26T07:19:31.144647+0000","last_active":"2023-11-26T07:19:31.151072+0000","last_peered":"2023-11-26T07:19:31.151072+0000","last_clean":"2023-11-26T07:19:31.151072+0000","last_became_active":"2023-11-26T07:19:31.143606+0000","last_became_peered":"2023-11-26T07:19:31.143606+0000","last_unstale":"2023-11-26T07:19:31.151072+0000","last_undegraded":"2023-11-26T07:19:31.151072+0000","last_fullsized":"2023-11-26T07:19:31.151072+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:19:30.106639+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:19:30.106639+0000","last_clean_scrub_stamp":"2023-11-26T07:19:30.106639+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T15:42:44.993287+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.14","version":"0'0","reported_seq":1,"reported_epoch":43,"state":"creating+peering","last_fresh":"2023-11-26T07:19:31.133561+0000","last_change":"2023-11-26T07:19:31.133561+0000","last_active":"2023-11-26T07:19:31.113498+0000","last_peered":"2023-11-26T07:19:31.113498+0000","last_clean":"2023-11-26T07:19:31.113498+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-26T07:19:31.133561+0000","last_undegraded":"2023-11-26T07:19:31.133561+0000","last_fullsized":"2023-11-26T07:19:31.133561+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:19:31.113498+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:19:31.113498+0000","last_clean_scrub_stamp":"2023-11-26T07:19:31.113498+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.15","version":"0'0","reported_seq":9,"reported_epoch":43,"state":"active+clean","last_fresh":"2023-11-26T07:19:31.149462+0000","last_change":"2023-11-26T07:19:31.147755+0000","last_active":"2023-11-26T07:19:31.149462+0000","last_peered":"2023-11-26T07:19:31.149462+0000","last_clean":"2023-11-26T07:19:31.149462+0000","last_became_active":"2023-11-26T07:19:31.147461+0000","last_became_peered":"2023-11-26T07:19:31.147461+0000","last_unstale":"2023-11-26T07:19:31.149462+0000","last_undegraded":"2023-11-26T07:19:31.149462+0000","last_fullsized":"2023-11-26T07:19:31.149462+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:19:30.106639+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:19:30.106639+0000","last_clean_scrub_stamp":"2023-11-26T07:19:30.106639+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T17:07:29.328878+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.15","version":"0'0","reported_seq":1,"reported_epoch":43,"state":"creating+peering","last_fresh":"2023-11-26T07:19:31.132934+0000","last_change":"2023-11-26T07:19:31.132934+0000","last_active":"2023-11-26T07:19:31.113498+0000","last_peered":"2023-11-26T07:19:31.113498+0000","last_clean":"2023-11-26T07:19:31.113498+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-26T07:19:31.132934+0000","last_undegraded":"2023-11-26T07:19:31.132934+0000","last_fullsized":"2023-11-26T07:19:31.132934+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:19:31.113498+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:19:31.113498+0000","last_clean_scrub_stamp":"2023-11-26T07:19:31.113498+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.14","version":"0'0","reported_seq":9,"reported_epoch":43,"state":"active+clean","last_fresh":"2023-11-26T07:19:31.136478+0000","last_change":"2023-11-26T07:19:31.131243+0000","last_active":"2023-11-26T07:19:31.136478+0000","last_peered":"2023-11-26T07:19:31.136478+0000","last_clean":"2023-11-26T07:19:31.136478+0000","last_became_active":"2023-11-26T07:19:31.130865+0000","last_became_peered":"2023-11-26T07:19:31.130865+0000","last_unstale":"2023-11-26T07:19:31.136478+0000","last_undegraded":"2023-11-26T07:19:31.136478+0000","last_fullsized":"2023-11-26T07:19:31.136478+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:19:30.106639+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:19:30.106639+0000","last_clean_scrub_stamp":"2023-11-26T07:19:30.106639+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T08:45:01.391060+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.12","version":"0'0","reported_seq":1,"reported_epoch":43,"state":"creating+peering","last_fresh":"2023-11-26T07:19:31.132304+0000","last_change":"2023-11-26T07:19:31.132304+0000","last_active":"2023-11-26T07:19:31.113498+0000","last_peered":"2023-11-26T07:19:31.113498+0000","last_clean":"2023-11-26T07:19:31.113498+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-26T07:19:31.132304+0000","last_undegraded":"2023-11-26T07:19:31.132304+0000","last_fullsized":"2023-11-26T07:19:31.132304+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:19:31.113498+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:19:31.113498+0000","last_clean_scrub_stamp":"2023-11-26T07:19:31.113498+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.13","version":"0'0","reported_seq":9,"reported_epoch":43,"state":"active+clean","last_fresh":"2023-11-26T07:19:31.153151+0000","last_change":"2023-11-26T07:19:31.148780+0000","last_active":"2023-11-26T07:19:31.153151+0000","last_peered":"2023-11-26T07:19:31.153151+0000","last_clean":"2023-11-26T07:19:31.153151+0000","last_became_active":"2023-11-26T07:19:31.148142+0000","last_became_peered":"2023-11-26T07:19:31.148142+0000","last_unstale":"2023-11-26T07:19:31.153151+0000","last_undegraded":"2023-11-26T07:19:31.153151+0000","last_fullsized":"2023-11-26T07:19:31.153151+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:19:30.106639+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:19:30.106639+0000","last_clean_scrub_stamp":"2023-11-26T07:19:30.106639+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T17:07:29.328878+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.13","version":"0'0","reported_seq":1,"reported_epoch":43,"state":"creating+peering","last_fresh":"2023-11-26T07:19:31.130763+0000","last_change":"2023-11-26T07:19:31.130763+0000","last_active":"2023-11-26T07:19:31.113498+0000","last_peered":"2023-11-26T07:19:31.113498+0000","last_clean":"2023-11-26T07:19:31.113498+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-26T07:19:31.130763+0000","last_undegraded":"2023-11-26T07:19:31.130763+0000","last_fullsized":"2023-11-26T07:19:31.130763+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:19:31.113498+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:19:31.113498+0000","last_clean_scrub_stamp":"2023-11-26T07:19:31.113498+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.12","version":"0'0","reported_seq":9,"reported_epoch":43,"state":"active+clean","last_fresh":"2023-11-26T07:19:31.147945+0000","last_change":"2023-11-26T07:19:31.144842+0000","last_active":"2023-11-26T07:19:31.147945+0000","last_peered":"2023-11-26T07:19:31.147945+0000","last_clean":"2023-11-26T07:19:31.147945+0000","last_became_active":"2023-11-26T07:19:31.144554+0000","last_became_peered":"2023-11-26T07:19:31.144554+0000","last_unstale":"2023-11-26T07:19:31.147945+0000","last_undegraded":"2023-11-26T07:19:31.147945+0000","last_fullsized":"2023-11-26T07:19:31.147945+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:19:30.106639+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:19:30.106639+0000","last_clean_scrub_stamp":"2023-11-26T07:19:30.106639+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T15:38:28.537385+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.10","version":"0'0","reported_seq":1,"reported_epoch":43,"state":"creating+peering","last_fresh":"2023-11-26T07:19:31.134915+0000","last_change":"2023-11-26T07:19:31.134915+0000","last_active":"2023-11-26T07:19:31.113498+0000","last_peered":"2023-11-26T07:19:31.113498+0000","last_clean":"2023-11-26T07:19:31.113498+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-26T07:19:31.134915+0000","last_undegraded":"2023-11-26T07:19:31.134915+0000","last_fullsized":"2023-11-26T07:19:31.134915+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:19:31.113498+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:19:31.113498+0000","last_clean_scrub_stamp":"2023-11-26T07:19:31.113498+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.11","version":"0'0","reported_seq":9,"reported_epoch":43,"state":"active+clean","last_fresh":"2023-11-26T07:19:31.153141+0000","last_change":"2023-11-26T07:19:31.148587+0000","last_active":"2023-11-26T07:19:31.153141+0000","last_peered":"2023-11-26T07:19:31.153141+0000","last_clean":"2023-11-26T07:19:31.153141+0000","last_became_active":"2023-11-26T07:19:31.148170+0000","last_became_peered":"2023-11-26T07:19:31.148170+0000","last_unstale":"2023-11-26T07:19:31.153141+0000","last_undegraded":"2023-11-26T07:19:31.153141+0000","last_fullsized":"2023-11-26T07:19:31.153141+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:19:30.106639+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:19:30.106639+0000","last_clean_scrub_stamp":"2023-11-26T07:19:30.106639+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T16:13:47.070273+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.11","version":"0'0","reported_seq":1,"reported_epoch":43,"state":"creating+peering","last_fresh":"2023-11-26T07:19:31.129918+0000","last_change":"2023-11-26T07:19:31.129918+0000","last_active":"2023-11-26T07:19:31.113498+0000","last_peered":"2023-11-26T07:19:31.113498+0000","last_clean":"2023-11-26T07:19:31.113498+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-26T07:19:31.129918+0000","last_undegraded":"2023-11-26T07:19:31.129918+0000","last_fullsized":"2023-11-26T07:19:31.129918+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:19:31.113498+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:19:31.113498+0000","last_clean_scrub_stamp":"2023-11-26T07:19:31.113498+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.10","version":"0'0","reported_seq":9,"reported_epoch":43,"state":"active+clean","last_fresh":"2023-11-26T07:19:31.147291+0000","last_change":"2023-11-26T07:19:31.139691+0000","last_active":"2023-11-26T07:19:31.147291+0000","last_peered":"2023-11-26T07:19:31.147291+0000","last_clean":"2023-11-26T07:19:31.147291+0000","last_became_active":"2023-11-26T07:19:31.139402+0000","last_became_peered":"2023-11-26T07:19:31.139402+0000","last_unstale":"2023-11-26T07:19:31.147291+0000","last_undegraded":"2023-11-26T07:19:31.147291+0000","last_fullsized":"2023-11-26T07:19:31.147291+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:19:30.106639+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:19:30.106639+0000","last_clean_scrub_stamp":"2023-11-26T07:19:30.106639+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T08:28:59.053370+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.e","version":"0'0","reported_seq":1,"reported_epoch":43,"state":"creating+peering","last_fresh":"2023-11-26T07:19:31.126558+0000","last_change":"2023-11-26T07:19:31.126558+0000","last_active":"2023-11-26T07:19:31.113498+0000","last_peered":"2023-11-26T07:19:31.113498+0000","last_clean":"2023-11-26T07:19:31.113498+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-26T07:19:31.126558+0000","last_undegraded":"2023-11-26T07:19:31.126558+0000","last_fullsized":"2023-11-26T07:19:31.126558+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:19:31.113498+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:19:31.113498+0000","last_clean_scrub_stamp":"2023-11-26T07:19:31.113498+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.f","version":"0'0","reported_seq":9,"reported_epoch":43,"state":"active+clean","last_fresh":"2023-11-26T07:19:31.144746+0000","last_change":"2023-11-26T07:19:31.140922+0000","last_active":"2023-11-26T07:19:31.144746+0000","last_peered":"2023-11-26T07:19:31.144746+0000","last_clean":"2023-11-26T07:19:31.144746+0000","last_became_active":"2023-11-26T07:19:31.139513+0000","last_became_peered":"2023-11-26T07:19:31.139513+0000","last_unstale":"2023-11-26T07:19:31.144746+0000","last_undegraded":"2023-11-26T07:19:31.144746+0000","last_fullsized":"2023-11-26T07:19:31.144746+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:19:30.106639+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:19:30.106639+0000","last_clean_scrub_stamp":"2023-11-26T07:19:30.106639+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T09:39:24.076764+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.f","version":"0'0","reported_seq":1,"reported_epoch":43,"state":"creating+peering","last_fresh":"2023-11-26T07:19:31.128236+0000","last_change":"2023-11-26T07:19:31.128236+0000","last_active":"2023-11-26T07:19:31.113498+0000","last_peered":"2023-11-26T07:19:31.113498+0000","last_clean":"2023-11-26T07:19:31.113498+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-26T07:19:31.128236+0000","last_undegraded":"2023-11-26T07:19:31.128236+0000","last_fullsized":"2023-11-26T07:19:31.128236+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:19:31.113498+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:19:31.113498+0000","last_clean_scrub_stamp":"2023-11-26T07:19:31.113498+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.e","version":"0'0","reported_seq":9,"reported_epoch":43,"state":"active+clean","last_fresh":"2023-11-26T07:19:31.146310+0000","last_change":"2023-11-26T07:19:31.141584+0000","last_active":"2023-11-26T07:19:31.146310+0000","last_peered":"2023-11-26T07:19:31.146310+0000","last_clean":"2023-11-26T07:19:31.146310+0000","last_became_active":"2023-11-26T07:19:31.141272+0000","last_became_peered":"2023-11-26T07:19:31.141272+0000","last_unstale":"2023-11-26T07:19:31.146310+0000","last_undegraded":"2023-11-26T07:19:31.146310+0000","last_fullsized":"2023-11-26T07:19:31.146310+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:19:30.106639+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:19:30.106639+0000","last_clean_scrub_stamp":"2023-11-26T07:19:30.106639+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T12:10:52.088348+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.c","version":"0'0","reported_seq":1,"reported_epoch":43,"state":"creating+peering","last_fresh":"2023-11-26T07:19:31.132510+0000","last_change":"2023-11-26T07:19:31.132510+0000","last_active":"2023-11-26T07:19:31.113498+0000","last_peered":"2023-11-26T07:19:31.113498+0000","last_clean":"2023-11-26T07:19:31.113498+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-26T07:19:31.132510+0000","last_undegraded":"2023-11-26T07:19:31.132510+0000","last_fullsized":"2023-11-26T07:19:31.132510+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:19:31.113498+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:19:31.113498+0000","last_clean_scrub_stamp":"2023-11-26T07:19:31.113498+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.d","version":"0'0","reported_seq":9,"reported_epoch":43,"state":"active+clean","last_fresh":"2023-11-26T07:19:31.149609+0000","last_change":"2023-11-26T07:19:31.145210+0000","last_active":"2023-11-26T07:19:31.149609+0000","last_peered":"2023-11-26T07:19:31.149609+0000","last_clean":"2023-11-26T07:19:31.149609+0000","last_became_active":"2023-11-26T07:19:31.144917+0000","last_became_peered":"2023-11-26T07:19:31.144917+0000","last_unstale":"2023-11-26T07:19:31.149609+0000","last_undegraded":"2023-11-26T07:19:31.149609+0000","last_fullsized":"2023-11-26T07:19:31.149609+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:19:30.106639+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:19:30.106639+0000","last_clean_scrub_stamp":"2023-11-26T07:19:30.106639+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T19:18:59.191940+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.d","version":"0'0","reported_seq":1,"reported_epoch":43,"state":"creating+peering","last_fresh":"2023-11-26T07:19:31.134528+0000","last_change":"2023-11-26T07:19:31.134528+0000","last_active":"2023-11-26T07:19:31.113498+0000","last_peered":"2023-11-26T07:19:31.113498+0000","last_clean":"2023-11-26T07:19:31.113498+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-26T07:19:31.134528+0000","last_undegraded":"2023-11-26T07:19:31.134528+0000","last_fullsized":"2023-11-26T07:19:31.134528+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:19:31.113498+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:19:31.113498+0000","last_clean_scrub_stamp":"2023-11-26T07:19:31.113498+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.c","version":"0'0","reported_seq":9,"reported_epoch":43,"state":"active+clean","last_fresh":"2023-11-26T07:19:31.150641+0000","last_change":"2023-11-26T07:19:31.147668+0000","last_active":"2023-11-26T07:19:31.150641+0000","last_peered":"2023-11-26T07:19:31.150641+0000","last_clean":"2023-11-26T07:19:31.150641+0000","last_became_active":"2023-11-26T07:19:31.146685+0000","last_became_peered":"2023-11-26T07:19:31.146685+0000","last_unstale":"2023-11-26T07:19:31.150641+0000","last_undegraded":"2023-11-26T07:19:31.150641+0000","last_fullsized":"2023-11-26T07:19:31.150641+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:19:30.106639+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:19:30.106639+0000","last_clean_scrub_stamp":"2023-11-26T07:19:30.106639+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T19:14:07.171899+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.a","version":"0'0","reported_seq":1,"reported_epoch":43,"state":"creating+peering","last_fresh":"2023-11-26T07:19:31.137790+0000","last_change":"2023-11-26T07:19:31.137790+0000","last_active":"2023-11-26T07:19:31.113498+0000","last_peered":"2023-11-26T07:19:31.113498+0000","last_clean":"2023-11-26T07:19:31.113498+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-26T07:19:31.137790+0000","last_undegraded":"2023-11-26T07:19:31.137790+0000","last_fullsized":"2023-11-26T07:19:31.137790+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:19:31.113498+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:19:31.113498+0000","last_clean_scrub_stamp":"2023-11-26T07:19:31.113498+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.b","version":"0'0","reported_seq":9,"reported_epoch":43,"state":"active+clean","last_fresh":"2023-11-26T07:19:31.153174+0000","last_change":"2023-11-26T07:19:31.149345+0000","last_active":"2023-11-26T07:19:31.153174+0000","last_peered":"2023-11-26T07:19:31.153174+0000","last_clean":"2023-11-26T07:19:31.153174+0000","last_became_active":"2023-11-26T07:19:31.149072+0000","last_became_peered":"2023-11-26T07:19:31.149072+0000","last_unstale":"2023-11-26T07:19:31.153174+0000","last_undegraded":"2023-11-26T07:19:31.153174+0000","last_fullsized":"2023-11-26T07:19:31.153174+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:19:30.106639+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:19:30.106639+0000","last_clean_scrub_stamp":"2023-11-26T07:19:30.106639+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T10:27:01.949013+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.b","version":"0'0","reported_seq":1,"reported_epoch":43,"state":"creating+peering","last_fresh":"2023-11-26T07:19:31.132864+0000","last_change":"2023-11-26T07:19:31.132864+0000","last_active":"2023-11-26T07:19:31.113498+0000","last_peered":"2023-11-26T07:19:31.113498+0000","last_clean":"2023-11-26T07:19:31.113498+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-26T07:19:31.132864+0000","last_undegraded":"2023-11-26T07:19:31.132864+0000","last_fullsized":"2023-11-26T07:19:31.132864+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:19:31.113498+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:19:31.113498+0000","last_clean_scrub_stamp":"2023-11-26T07:19:31.113498+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.a","version":"0'0","reported_seq":9,"reported_epoch":43,"state":"active+clean","last_fresh":"2023-11-26T07:19:31.150714+0000","last_change":"2023-11-26T07:19:31.146363+0000","last_active":"2023-11-26T07:19:31.150714+0000","last_peered":"2023-11-26T07:19:31.150714+0000","last_clean":"2023-11-26T07:19:31.150714+0000","last_became_active":"2023-11-26T07:19:31.146013+0000","last_became_peered":"2023-11-26T07:19:31.146013+0000","last_unstale":"2023-11-26T07:19:31.150714+0000","last_undegraded":"2023-11-26T07:19:31.150714+0000","last_fullsized":"2023-11-26T07:19:31.150714+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:19:30.106639+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:19:30.106639+0000","last_clean_scrub_stamp":"2023-11-26T07:19:30.106639+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T12:54:51.008938+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.8","version":"0'0","reported_seq":1,"reported_epoch":43,"state":"creating+peering","last_fresh":"2023-11-26T07:19:31.126547+0000","last_change":"2023-11-26T07:19:31.126547+0000","last_active":"2023-11-26T07:19:31.113498+0000","last_peered":"2023-11-26T07:19:31.113498+0000","last_clean":"2023-11-26T07:19:31.113498+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-26T07:19:31.126547+0000","last_undegraded":"2023-11-26T07:19:31.126547+0000","last_fullsized":"2023-11-26T07:19:31.126547+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:19:31.113498+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:19:31.113498+0000","last_clean_scrub_stamp":"2023-11-26T07:19:31.113498+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.9","version":"0'0","reported_seq":9,"reported_epoch":43,"state":"active+clean","last_fresh":"2023-11-26T07:19:31.153191+0000","last_change":"2023-11-26T07:19:31.148399+0000","last_active":"2023-11-26T07:19:31.153191+0000","last_peered":"2023-11-26T07:19:31.153191+0000","last_clean":"2023-11-26T07:19:31.153191+0000","last_became_active":"2023-11-26T07:19:31.148127+0000","last_became_peered":"2023-11-26T07:19:31.148127+0000","last_unstale":"2023-11-26T07:19:31.153191+0000","last_undegraded":"2023-11-26T07:19:31.153191+0000","last_fullsized":"2023-11-26T07:19:31.153191+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:19:30.106639+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:19:30.106639+0000","last_clean_scrub_stamp":"2023-11-26T07:19:30.106639+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T16:49:27.472665+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.9","version":"0'0","reported_seq":1,"reported_epoch":43,"state":"creating+peering","last_fresh":"2023-11-26T07:19:31.129887+0000","last_change":"2023-11-26T07:19:31.129887+0000","last_active":"2023-11-26T07:19:31.113498+0000","last_peered":"2023-11-26T07:19:31.113498+0000","last_clean":"2023-11-26T07:19:31.113498+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-26T07:19:31.129887+0000","last_undegraded":"2023-11-26T07:19:31.129887+0000","last_fullsized":"2023-11-26T07:19:31.129887+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:19:31.113498+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:19:31.113498+0000","last_clean_scrub_stamp":"2023-11-26T07:19:31.113498+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.8","version":"0'0","reported_seq":9,"reported_epoch":43,"state":"active+clean","last_fresh":"2023-11-26T07:19:31.147571+0000","last_change":"2023-11-26T07:19:31.144956+0000","last_active":"2023-11-26T07:19:31.147571+0000","last_peered":"2023-11-26T07:19:31.147571+0000","last_clean":"2023-11-26T07:19:31.147571+0000","last_became_active":"2023-11-26T07:19:31.144653+0000","last_became_peered":"2023-11-26T07:19:31.144653+0000","last_unstale":"2023-11-26T07:19:31.147571+0000","last_undegraded":"2023-11-26T07:19:31.147571+0000","last_fullsized":"2023-11-26T07:19:31.147571+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:19:30.106639+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:19:30.106639+0000","last_clean_scrub_stamp":"2023-11-26T07:19:30.106639+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T13:42:55.807051+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.7","version":"0'0","reported_seq":1,"reported_epoch":43,"state":"creating+peering","last_fresh":"2023-11-26T07:19:31.132550+0000","last_change":"2023-11-26T07:19:31.132550+0000","last_active":"2023-11-26T07:19:31.113498+0000","last_peered":"2023-11-26T07:19:31.113498+0000","last_clean":"2023-11-26T07:19:31.113498+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-26T07:19:31.132550+0000","last_undegraded":"2023-11-26T07:19:31.132550+0000","last_fullsized":"2023-11-26T07:19:31.132550+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:19:31.113498+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:19:31.113498+0000","last_clean_scrub_stamp":"2023-11-26T07:19:31.113498+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.6","version":"0'0","reported_seq":9,"reported_epoch":43,"state":"active+clean","last_fresh":"2023-11-26T07:19:31.148470+0000","last_change":"2023-11-26T07:19:31.143699+0000","last_active":"2023-11-26T07:19:31.148470+0000","last_peered":"2023-11-26T07:19:31.148470+0000","last_clean":"2023-11-26T07:19:31.148470+0000","last_became_active":"2023-11-26T07:19:31.143421+0000","last_became_peered":"2023-11-26T07:19:31.143421+0000","last_unstale":"2023-11-26T07:19:31.148470+0000","last_undegraded":"2023-11-26T07:19:31.148470+0000","last_fullsized":"2023-11-26T07:19:31.148470+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:19:30.106639+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:19:30.106639+0000","last_clean_scrub_stamp":"2023-11-26T07:19:30.106639+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T13:42:55.807051+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.4","version":"0'0","reported_seq":1,"reported_epoch":43,"state":"creating+peering","last_fresh":"2023-11-26T07:19:31.128178+0000","last_change":"2023-11-26T07:19:31.128178+0000","last_active":"2023-11-26T07:19:31.113498+0000","last_peered":"2023-11-26T07:19:31.113498+0000","last_clean":"2023-11-26T07:19:31.113498+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-26T07:19:31.128178+0000","last_undegraded":"2023-11-26T07:19:31.128178+0000","last_fullsized":"2023-11-26T07:19:31.128178+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:19:31.113498+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:19:31.113498+0000","last_clean_scrub_stamp":"2023-11-26T07:19:31.113498+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.5","version":"0'0","reported_seq":9,"reported_epoch":43,"state":"active+clean","last_fresh":"2023-11-26T07:19:31.150074+0000","last_change":"2023-11-26T07:19:31.148100+0000","last_active":"2023-11-26T07:19:31.150074+0000","last_peered":"2023-11-26T07:19:31.150074+0000","last_clean":"2023-11-26T07:19:31.150074+0000","last_became_active":"2023-11-26T07:19:31.147766+0000","last_became_peered":"2023-11-26T07:19:31.147766+0000","last_unstale":"2023-11-26T07:19:31.150074+0000","last_undegraded":"2023-11-26T07:19:31.150074+0000","last_fullsized":"2023-11-26T07:19:31.150074+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:19:30.106639+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:19:30.106639+0000","last_clean_scrub_stamp":"2023-11-26T07:19:30.106639+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T13:58:09.825490+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.5","version":"0'0","reported_seq":1,"reported_epoch":43,"state":"creating+peering","last_fresh":"2023-11-26T07:19:31.132476+0000","last_change":"2023-11-26T07:19:31.132476+0000","last_active":"2023-11-26T07:19:31.113498+0000","last_peered":"2023-11-26T07:19:31.113498+0000","last_clean":"2023-11-26T07:19:31.113498+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-26T07:19:31.132476+0000","last_undegraded":"2023-11-26T07:19:31.132476+0000","last_fullsized":"2023-11-26T07:19:31.132476+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:19:31.113498+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:19:31.113498+0000","last_clean_scrub_stamp":"2023-11-26T07:19:31.113498+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.4","version":"0'0","reported_seq":9,"reported_epoch":43,"state":"active+clean","last_fresh":"2023-11-26T07:19:31.150706+0000","last_change":"2023-11-26T07:19:31.144386+0000","last_active":"2023-11-26T07:19:31.150706+0000","last_peered":"2023-11-26T07:19:31.150706+0000","last_clean":"2023-11-26T07:19:31.150706+0000","last_became_active":"2023-11-26T07:19:31.144106+0000","last_became_peered":"2023-11-26T07:19:31.144106+0000","last_unstale":"2023-11-26T07:19:31.150706+0000","last_undegraded":"2023-11-26T07:19:31.150706+0000","last_fullsized":"2023-11-26T07:19:31.150706+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:19:30.106639+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:19:30.106639+0000","last_clean_scrub_stamp":"2023-11-26T07:19:30.106639+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T08:08:30.523864+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.2","version":"0'0","reported_seq":1,"reported_epoch":43,"state":"creating+peering","last_fresh":"2023-11-26T07:19:31.134609+0000","last_change":"2023-11-26T07:19:31.134609+0000","last_active":"2023-11-26T07:19:31.113498+0000","last_peered":"2023-11-26T07:19:31.113498+0000","last_clean":"2023-11-26T07:19:31.113498+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-26T07:19:31.134609+0000","last_undegraded":"2023-11-26T07:19:31.134609+0000","last_fullsized":"2023-11-26T07:19:31.134609+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:19:31.113498+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:19:31.113498+0000","last_clean_scrub_stamp":"2023-11-26T07:19:31.113498+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.3","version":"0'0","reported_seq":9,"reported_epoch":43,"state":"active+clean","last_fresh":"2023-11-26T07:19:31.149359+0000","last_change":"2023-11-26T07:19:31.146929+0000","last_active":"2023-11-26T07:19:31.149359+0000","last_peered":"2023-11-26T07:19:31.149359+0000","last_clean":"2023-11-26T07:19:31.149359+0000","last_became_active":"2023-11-26T07:19:31.146630+0000","last_became_peered":"2023-11-26T07:19:31.146630+0000","last_unstale":"2023-11-26T07:19:31.149359+0000","last_undegraded":"2023-11-26T07:19:31.149359+0000","last_fullsized":"2023-11-26T07:19:31.149359+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:19:30.106639+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:19:30.106639+0000","last_clean_scrub_stamp":"2023-11-26T07:19:30.106639+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T19:14:07.171899+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.3","version":"0'0","reported_seq":1,"reported_epoch":43,"state":"creating+peering","last_fresh":"2023-11-26T07:19:31.134861+0000","last_change":"2023-11-26T07:19:31.134861+0000","last_active":"2023-11-26T07:19:31.113498+0000","last_peered":"2023-11-26T07:19:31.113498+0000","last_clean":"2023-11-26T07:19:31.113498+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-26T07:19:31.134861+0000","last_undegraded":"2023-11-26T07:19:31.134861+0000","last_fullsized":"2023-11-26T07:19:31.134861+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:19:31.113498+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:19:31.113498+0000","last_clean_scrub_stamp":"2023-11-26T07:19:31.113498+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.2","version":"0'0","reported_seq":9,"reported_epoch":43,"state":"active+clean","last_fresh":"2023-11-26T07:19:31.149756+0000","last_change":"2023-11-26T07:19:31.147730+0000","last_active":"2023-11-26T07:19:31.149756+0000","last_peered":"2023-11-26T07:19:31.149756+0000","last_clean":"2023-11-26T07:19:31.149756+0000","last_became_active":"2023-11-26T07:19:31.147374+0000","last_became_peered":"2023-11-26T07:19:31.147374+0000","last_unstale":"2023-11-26T07:19:31.149756+0000","last_undegraded":"2023-11-26T07:19:31.149756+0000","last_fullsized":"2023-11-26T07:19:31.149756+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:19:30.106639+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:19:30.106639+0000","last_clean_scrub_stamp":"2023-11-26T07:19:30.106639+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T16:49:27.472665+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.0","version":"0'0","reported_seq":1,"reported_epoch":43,"state":"creating+peering","last_fresh":"2023-11-26T07:19:31.130272+0000","last_change":"2023-11-26T07:19:31.130272+0000","last_active":"2023-11-26T07:19:31.113498+0000","last_peered":"2023-11-26T07:19:31.113498+0000","last_clean":"2023-11-26T07:19:31.113498+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-26T07:19:31.130272+0000","last_undegraded":"2023-11-26T07:19:31.130272+0000","last_fullsized":"2023-11-26T07:19:31.130272+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:19:31.113498+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:19:31.113498+0000","last_clean_scrub_stamp":"2023-11-26T07:19:31.113498+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.1","version":"0'0","reported_seq":9,"reported_epoch":43,"state":"active+clean","last_fresh":"2023-11-26T07:19:31.152909+0000","last_change":"2023-11-26T07:19:31.145269+0000","last_active":"2023-11-26T07:19:31.152909+0000","last_peered":"2023-11-26T07:19:31.152909+0000","last_clean":"2023-11-26T07:19:31.152909+0000","last_became_active":"2023-11-26T07:19:31.144994+0000","last_became_peered":"2023-11-26T07:19:31.144994+0000","last_unstale":"2023-11-26T07:19:31.152909+0000","last_undegraded":"2023-11-26T07:19:31.152909+0000","last_fullsized":"2023-11-26T07:19:31.152909+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:19:30.106639+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:19:30.106639+0000","last_clean_scrub_stamp":"2023-11-26T07:19:30.106639+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T15:32:40.529918+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.1","version":"0'0","reported_seq":1,"reported_epoch":43,"state":"creating+peering","last_fresh":"2023-11-26T07:19:31.124694+0000","last_change":"2023-11-26T07:19:31.124694+0000","last_active":"2023-11-26T07:19:31.113498+0000","last_peered":"2023-11-26T07:19:31.113498+0000","last_clean":"2023-11-26T07:19:31.113498+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-26T07:19:31.124694+0000","last_undegraded":"2023-11-26T07:19:31.124694+0000","last_fullsized":"2023-11-26T07:19:31.124694+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:19:31.113498+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:19:31.113498+0000","last_clean_scrub_stamp":"2023-11-26T07:19:31.113498+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.0","version":"0'0","reported_seq":9,"reported_epoch":43,"state":"active+clean","last_fresh":"2023-11-26T07:19:31.151119+0000","last_change":"2023-11-26T07:19:31.146523+0000","last_active":"2023-11-26T07:19:31.151119+0000","last_peered":"2023-11-26T07:19:31.151119+0000","last_clean":"2023-11-26T07:19:31.151119+0000","last_became_active":"2023-11-26T07:19:31.146168+0000","last_became_peered":"2023-11-26T07:19:31.146168+0000","last_unstale":"2023-11-26T07:19:31.151119+0000","last_undegraded":"2023-11-26T07:19:31.151119+0000","last_fullsized":"2023-11-26T07:19:31.151119+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:19:30.106639+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:19:30.106639+0000","last_clean_scrub_stamp":"2023-11-26T07:19:30.106639+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T12:03:38.617112+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.6","version":"0'0","reported_seq":1,"reported_epoch":43,"state":"creating+peering","last_fresh":"2023-11-26T07:19:31.128572+0000","last_change":"2023-11-26T07:19:31.128572+0000","last_active":"2023-11-26T07:19:31.113498+0000","last_peered":"2023-11-26T07:19:31.113498+0000","last_clean":"2023-11-26T07:19:31.113498+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-26T07:19:31.128572+0000","last_undegraded":"2023-11-26T07:19:31.128572+0000","last_fullsized":"2023-11-26T07:19:31.128572+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:19:31.113498+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:19:31.113498+0000","last_clean_scrub_stamp":"2023-11-26T07:19:31.113498+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.7","version":"0'0","reported_seq":9,"reported_epoch":43,"state":"active+clean","last_fresh":"2023-11-26T07:19:31.150685+0000","last_change":"2023-11-26T07:19:31.146427+0000","last_active":"2023-11-26T07:19:31.150685+0000","last_peered":"2023-11-26T07:19:31.150685+0000","last_clean":"2023-11-26T07:19:31.150685+0000","last_became_active":"2023-11-26T07:19:31.146044+0000","last_became_peered":"2023-11-26T07:19:31.146044+0000","last_unstale":"2023-11-26T07:19:31.150685+0000","last_undegraded":"2023-11-26T07:19:31.150685+0000","last_fullsized":"2023-11-26T07:19:31.150685+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:19:30.106639+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:19:30.106639+0000","last_clean_scrub_stamp":"2023-11-26T07:19:30.106639+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T15:54:05.005833+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.0","version":"15'117","reported_seq":182,"reported_epoch":42,"state":"active+clean","last_fresh":"2023-11-26T07:19:30.125712+0000","last_change":"2023-11-26T07:17:43.827802+0000","last_active":"2023-11-26T07:19:30.125712+0000","last_peered":"2023-11-26T07:19:30.125712+0000","last_clean":"2023-11-26T07:19:30.125712+0000","last_became_active":"2023-11-26T07:17:43.827308+0000","last_became_peered":"2023-11-26T07:17:43.827308+0000","last_unstale":"2023-11-26T07:19:30.125712+0000","last_undegraded":"2023-11-26T07:19:30.125712+0000","last_fullsized":"2023-11-26T07:19:30.125712+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:17:42.160945+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:17:42.160945+0000","last_clean_scrub_stamp":"2023-11-26T07:17:42.160945+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T08:17:54.167366+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":7,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":6,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574872,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":28084,"kb_used_data":1060,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343756,"statfs":{"total":96636764160,"available":96608006144,"internally_reserved":0,"allocated":1085440,"data_stored":786125,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Nov 26 07:18:42 2023","interfaces":[{"interface":"back","average":{"1min":0.71499999999999997,"5min":0.71499999999999997,"15min":0.71499999999999997},"min":{"1min":0.54300000000000004,"5min":0.54300000000000004,"15min":0.54300000000000004},"max":{"1min":0.91500000000000004,"5min":0.91500000000000004,"15min":0.91500000000000004},"last":0.57699999999999996},{"interface":"front","average":{"1min":0.70899999999999996,"5min":0.70899999999999996,"15min":0.70899999999999996},"min":{"1min":0.52900000000000003,"5min":0.52900000000000003,"15min":0.52900000000000003},"max":{"1min":1.0980000000000001,"5min":1.0980000000000001,"15min":1.0980000000000001},"last":0.64800000000000002}]},{"osd":1,"last update":"Sun Nov 26 07:18:42 2023","interfaces":[{"interface":"back","average":{"1min":0.68200000000000005,"5min":0.68200000000000005,"15min":0.68200000000000005},"min":{"1min":0.47999999999999998,"5min":0.47999999999999998,"15min":0.47999999999999998},"max":{"1min":1.0669999999999999,"5min":1.0669999999999999,"15min":1.0669999999999999},"last":0.73999999999999999},{"interface":"front","average":{"1min":0.83899999999999997,"5min":0.83899999999999997,"15min":0.83899999999999997},"min":{"1min":0.47599999999999998,"5min":0.47599999999999998,"15min":0.47599999999999998},"max":{"1min":4.5309999999999997,"5min":4.5309999999999997,"15min":4.5309999999999997},"last":0.47699999999999998}]},{"osd":2,"last update":"Sun Nov 26 07:18:42 2023","interfaces":[{"interface":"back","average":{"1min":0.67600000000000005,"5min":0.67600000000000005,"15min":0.67600000000000005},"min":{"1min":0.52000000000000002,"5min":0.52000000000000002,"15min":0.52000000000000002},"max":{"1min":0.873,"5min":0.873,"15min":0.873},"last":0.69499999999999995},{"interface":"front","average":{"1min":0.71299999999999997,"5min":0.71299999999999997,"15min":0.71299999999999997},"min":{"1min":0.54700000000000004,"5min":0.54700000000000004,"15min":0.54700000000000004},"max":{"1min":1.1339999999999999,"5min":1.1339999999999999,"15min":1.1339999999999999},"last":0.61599999999999999}]},{"osd":3,"last update":"Sun Nov 26 07:18:42 2023","interfaces":[{"interface":"back","average":{"1min":0.69699999999999995,"5min":0.69699999999999995,"15min":0.69699999999999995},"min":{"1min":0.51800000000000002,"5min":0.51800000000000002,"15min":0.51800000000000002},"max":{"1min":1.026,"5min":1.026,"15min":1.026},"last":0.627},{"interface":"front","average":{"1min":0.73299999999999998,"5min":0.73299999999999998,"15min":0.73299999999999998},"min":{"1min":0.50600000000000001,"5min":0.50600000000000001,"15min":0.50600000000000001},"max":{"1min":1.4490000000000001,"5min":1.4490000000000001,"15min":1.4490000000000001},"last":0.66200000000000003}]},{"osd":4,"last update":"Sun Nov 26 07:18:42 2023","interfaces":[{"interface":"back","average":{"1min":0.59999999999999998,"5min":0.59999999999999998,"15min":0.59999999999999998},"min":{"1min":0.39400000000000002,"5min":0.39400000000000002,"15min":0.39400000000000002},"max":{"1min":0.94799999999999995,"5min":0.94799999999999995,"15min":0.94799999999999995},"last":0.59599999999999997},{"interface":"front","average":{"1min":0.67300000000000004,"5min":0.67300000000000004,"15min":0.67300000000000004},"min":{"1min":0.436,"5min":0.436,"15min":0.436},"max":{"1min":1.5509999999999999,"5min":1.5509999999999999,"15min":1.5509999999999999},"last":0.51300000000000001}]},{"osd":5,"last update":"Sun Nov 26 07:18:42 2023","interfaces":[{"interface":"back","average":{"1min":0.66600000000000004,"5min":0.66600000000000004,"15min":0.66600000000000004},"min":{"1min":0.443,"5min":0.443,"15min":0.443},"max":{"1min":1.415,"5min":1.415,"15min":1.415},"last":0.77300000000000002},{"interface":"front","average":{"1min":0.71399999999999997,"5min":0.71399999999999997,"15min":0.71399999999999997},"min":{"1min":0.53100000000000003,"5min":0.53100000000000003,"15min":0.53100000000000003},"max":{"1min":1.3859999999999999,"5min":1.3859999999999999,"15min":1.3859999999999999},"last":0.67700000000000005}]},{"osd":6,"last update":"Sun Nov 26 07:18:42 2023","interfaces":[{"interface":"back","average":{"1min":0.77100000000000002,"5min":0.77100000000000002,"15min":0.77100000000000002},"min":{"1min":0.64100000000000001,"5min":0.64100000000000001,"15min":0.64100000000000001},"max":{"1min":1.3069999999999999,"5min":1.3069999999999999,"15min":1.3069999999999999},"last":0.75800000000000001},{"interface":"front","average":{"1min":0.77100000000000002,"5min":0.77100000000000002,"15min":0.77100000000000002},"min":{"1min":0.57399999999999995,"5min":0.57399999999999995,"15min":0.57399999999999995},"max":{"1min":1.6759999999999999,"5min":1.6759999999999999,"15min":1.6759999999999999},"last":0.56200000000000006}]}]},{"osd":6,"up_from":13,"seq":55834574874,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27512,"kb_used_data":492,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344328,"statfs":{"total":96636764160,"available":96608591872,"internally_reserved":0,"allocated":503808,"data_stored":202024,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Nov 26 07:18:45 2023","interfaces":[{"interface":"back","average":{"1min":0.68400000000000005,"5min":0.68400000000000005,"15min":0.68400000000000005},"min":{"1min":0.36699999999999999,"5min":0.36699999999999999,"15min":0.36699999999999999},"max":{"1min":1.7549999999999999,"5min":1.7549999999999999,"15min":1.7549999999999999},"last":0.80500000000000005},{"interface":"front","average":{"1min":0.95799999999999996,"5min":0.95799999999999996,"15min":0.95799999999999996},"min":{"1min":0.38300000000000001,"5min":0.38300000000000001,"15min":0.38300000000000001},"max":{"1min":5.7489999999999997,"5min":5.7489999999999997,"15min":5.7489999999999997},"last":0.59599999999999997}]},{"osd":1,"last update":"Sun Nov 26 07:18:45 2023","interfaces":[{"interface":"back","average":{"1min":0.91400000000000003,"5min":0.91400000000000003,"15min":0.91400000000000003},"min":{"1min":0.42099999999999999,"5min":0.42099999999999999,"15min":0.42099999999999999},"max":{"1min":5.4240000000000004,"5min":5.4240000000000004,"15min":5.4240000000000004},"last":0.46400000000000002},{"interface":"front","average":{"1min":0.89700000000000002,"5min":0.89700000000000002,"15min":0.89700000000000002},"min":{"1min":0.499,"5min":0.499,"15min":0.499},"max":{"1min":5.6020000000000003,"5min":5.6020000000000003,"15min":5.6020000000000003},"last":0.64700000000000002}]},{"osd":2,"last update":"Sun Nov 26 07:18:45 2023","interfaces":[{"interface":"back","average":{"1min":0.69699999999999995,"5min":0.69699999999999995,"15min":0.69699999999999995},"min":{"1min":0.47599999999999998,"5min":0.47599999999999998,"15min":0.47599999999999998},"max":{"1min":1.8520000000000001,"5min":1.8520000000000001,"15min":1.8520000000000001},"last":0.624},{"interface":"front","average":{"1min":0.73399999999999999,"5min":0.73399999999999999,"15min":0.73399999999999999},"min":{"1min":0.46700000000000003,"5min":0.46700000000000003,"15min":0.46700000000000003},"max":{"1min":1.9159999999999999,"5min":1.9159999999999999,"15min":1.9159999999999999},"last":0.67400000000000004}]},{"osd":3,"last update":"Sun Nov 26 07:18:45 2023","interfaces":[{"interface":"back","average":{"1min":0.90200000000000002,"5min":0.90200000000000002,"15min":0.90200000000000002},"min":{"1min":0.502,"5min":0.502,"15min":0.502},"max":{"1min":5.4390000000000001,"5min":5.4390000000000001,"15min":5.4390000000000001},"last":0.70099999999999996},{"interface":"front","average":{"1min":0.94499999999999995,"5min":0.94499999999999995,"15min":0.94499999999999995},"min":{"1min":0.54400000000000004,"5min":0.54400000000000004,"15min":0.54400000000000004},"max":{"1min":5.5490000000000004,"5min":5.5490000000000004,"15min":5.5490000000000004},"last":0.71899999999999997}]},{"osd":4,"last update":"Sun Nov 26 07:18:45 2023","interfaces":[{"interface":"back","average":{"1min":0.64200000000000002,"5min":0.64200000000000002,"15min":0.64200000000000002},"min":{"1min":0.46899999999999997,"5min":0.46899999999999997,"15min":0.46899999999999997},"max":{"1min":0.85099999999999998,"5min":0.85099999999999998,"15min":0.85099999999999998},"last":0.85999999999999999},{"interface":"front","average":{"1min":0.67500000000000004,"5min":0.67500000000000004,"15min":0.67500000000000004},"min":{"1min":0.40699999999999997,"5min":0.40699999999999997,"15min":0.40699999999999997},"max":{"1min":0.93300000000000005,"5min":0.93300000000000005,"15min":0.93300000000000005},"last":0.78200000000000003}]},{"osd":5,"last update":"Sun Nov 26 07:18:45 2023","interfaces":[{"interface":"back","average":{"1min":0.65900000000000003,"5min":0.65900000000000003,"15min":0.65900000000000003},"min":{"1min":0.42499999999999999,"5min":0.42499999999999999,"15min":0.42499999999999999},"max":{"1min":0.79300000000000004,"5min":0.79300000000000004,"15min":0.79300000000000004},"last":0.83299999999999996},{"interface":"front","average":{"1min":0.71299999999999997,"5min":0.71299999999999997,"15min":0.71299999999999997},"min":{"1min":0.436,"5min":0.436,"15min":0.436},"max":{"1min":0.93000000000000005,"5min":0.93000000000000005,"15min":0.93000000000000005},"last":0.90100000000000002}]},{"osd":7,"last update":"Sun Nov 26 07:18:45 2023","interfaces":[{"interface":"back","average":{"1min":0.74199999999999999,"5min":0.74199999999999999,"15min":0.74199999999999999},"min":{"1min":0.46600000000000003,"5min":0.46600000000000003,"15min":0.46600000000000003},"max":{"1min":0.998,"5min":0.998,"15min":0.998},"last":0.746},{"interface":"front","average":{"1min":0.71599999999999997,"5min":0.71599999999999997,"15min":0.71599999999999997},"min":{"1min":0.498,"5min":0.498,"15min":0.498},"max":{"1min":0.86499999999999999,"5min":0.86499999999999999,"15min":0.86499999999999999},"last":0.88100000000000001}]}]},{"osd":5,"up_from":13,"seq":55834574872,"num_pgs":4,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27512,"kb_used_data":492,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344328,"statfs":{"total":96636764160,"available":96608591872,"internally_reserved":0,"allocated":503808,"data_stored":202024,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Nov 26 07:18:45 2023","interfaces":[{"interface":"back","average":{"1min":0.66600000000000004,"5min":0.66600000000000004,"15min":0.66600000000000004},"min":{"1min":0.38100000000000001,"5min":0.38100000000000001,"15min":0.38100000000000001},"max":{"1min":1.0289999999999999,"5min":1.0289999999999999,"15min":1.0289999999999999},"last":0.66500000000000004},{"interface":"front","average":{"1min":0.66000000000000003,"5min":0.66000000000000003,"15min":0.66000000000000003},"min":{"1min":0.438,"5min":0.438,"15min":0.438},"max":{"1min":1.05,"5min":1.05,"15min":1.05},"last":0.58099999999999996}]},{"osd":1,"last update":"Sun Nov 26 07:18:45 2023","interfaces":[{"interface":"back","average":{"1min":0.66800000000000004,"5min":0.66800000000000004,"15min":0.66800000000000004},"min":{"1min":0.41599999999999998,"5min":0.41599999999999998,"15min":0.41599999999999998},"max":{"1min":0.86699999999999999,"5min":0.86699999999999999,"15min":0.86699999999999999},"last":0.872},{"interface":"front","average":{"1min":0.68600000000000005,"5min":0.68600000000000005,"15min":0.68600000000000005},"min":{"1min":0.49199999999999999,"5min":0.49199999999999999,"15min":0.49199999999999999},"max":{"1min":0.98899999999999999,"5min":0.98899999999999999,"15min":0.98899999999999999},"last":0.72899999999999998}]},{"osd":2,"last update":"Sun Nov 26 07:18:45 2023","interfaces":[{"interface":"back","average":{"1min":0.67500000000000004,"5min":0.67500000000000004,"15min":0.67500000000000004},"min":{"1min":0.45900000000000002,"5min":0.45900000000000002,"15min":0.45900000000000002},"max":{"1min":0.89800000000000002,"5min":0.89800000000000002,"15min":0.89800000000000002},"last":0.627},{"interface":"front","average":{"1min":0.67000000000000004,"5min":0.67000000000000004,"15min":0.67000000000000004},"min":{"1min":0.51200000000000001,"5min":0.51200000000000001,"15min":0.51200000000000001},"max":{"1min":0.89700000000000002,"5min":0.89700000000000002,"15min":0.89700000000000002},"last":0.76000000000000001}]},{"osd":3,"last update":"Sun Nov 26 07:18:45 2023","interfaces":[{"interface":"back","average":{"1min":0.71999999999999997,"5min":0.71999999999999997,"15min":0.71999999999999997},"min":{"1min":0.43099999999999999,"5min":0.43099999999999999,"15min":0.43099999999999999},"max":{"1min":1.0029999999999999,"5min":1.0029999999999999,"15min":1.0029999999999999},"last":0.69599999999999995},{"interface":"front","average":{"1min":0.67500000000000004,"5min":0.67500000000000004,"15min":0.67500000000000004},"min":{"1min":0.44600000000000001,"5min":0.44600000000000001,"15min":0.44600000000000001},"max":{"1min":1.0209999999999999,"5min":1.0209999999999999,"15min":1.0209999999999999},"last":0.74299999999999999}]},{"osd":4,"last update":"Sun Nov 26 07:18:45 2023","interfaces":[{"interface":"back","average":{"1min":0.63500000000000001,"5min":0.63500000000000001,"15min":0.63500000000000001},"min":{"1min":0.39200000000000002,"5min":0.39200000000000002,"15min":0.39200000000000002},"max":{"1min":0.95099999999999996,"5min":0.95099999999999996,"15min":0.95099999999999996},"last":0.78900000000000003},{"interface":"front","average":{"1min":0.70899999999999996,"5min":0.70899999999999996,"15min":0.70899999999999996},"min":{"1min":0.41299999999999998,"5min":0.41299999999999998,"15min":0.41299999999999998},"max":{"1min":1.254,"5min":1.254,"15min":1.254},"last":0.80900000000000005}]},{"osd":6,"last update":"Sun Nov 26 07:18:45 2023","interfaces":[{"interface":"back","average":{"1min":0.67200000000000004,"5min":0.67200000000000004,"15min":0.67200000000000004},"min":{"1min":0.45600000000000002,"5min":0.45600000000000002,"15min":0.45600000000000002},"max":{"1min":0.94499999999999995,"5min":0.94499999999999995,"15min":0.94499999999999995},"last":0.83199999999999996},{"interface":"front","average":{"1min":0.72699999999999998,"5min":0.72699999999999998,"15min":0.72699999999999998},"min":{"1min":0.44,"5min":0.44,"15min":0.44},"max":{"1min":1.0649999999999999,"5min":1.0649999999999999,"15min":1.0649999999999999},"last":0.85299999999999998}]},{"osd":7,"last update":"Sun Nov 26 07:18:45 2023","interfaces":[{"interface":"back","average":{"1min":0.80400000000000005,"5min":0.80400000000000005,"15min":0.80400000000000005},"min":{"1min":0.48199999999999998,"5min":0.48199999999999998,"15min":0.48199999999999998},"max":{"1min":1.4079999999999999,"5min":1.4079999999999999,"15min":1.4079999999999999},"last":0.89400000000000002},{"interface":"front","average":{"1min":0.77500000000000002,"5min":0.77500000000000002,"15min":0.77500000000000002},"min":{"1min":0.51600000000000001,"5min":0.51600000000000001,"15min":0.51600000000000001},"max":{"1min":1.0169999999999999,"5min":1.0169999999999999,"15min":1.0169999999999999},"last":0.92400000000000004}]}]},{"osd":4,"up_from":13,"seq":55834574874,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27516,"kb_used_data":492,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344324,"statfs":{"total":96636764160,"available":96608587776,"internally_reserved":0,"allocated":503808,"data_stored":202024,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Nov 26 07:18:46 2023","interfaces":[{"interface":"back","average":{"1min":0.79400000000000004,"5min":0.79400000000000004,"15min":0.79400000000000004},"min":{"1min":0.45600000000000002,"5min":0.45600000000000002,"15min":0.45600000000000002},"max":{"1min":4.1689999999999996,"5min":4.1689999999999996,"15min":4.1689999999999996},"last":0.622},{"interface":"front","average":{"1min":1.002,"5min":1.002,"15min":1.002},"min":{"1min":0.47499999999999998,"5min":0.47499999999999998,"15min":0.47499999999999998},"max":{"1min":7.7889999999999997,"5min":7.7889999999999997,"15min":7.7889999999999997},"last":0.499}]},{"osd":1,"last update":"Sun Nov 26 07:18:46 2023","interfaces":[{"interface":"back","average":{"1min":0.64700000000000002,"5min":0.64700000000000002,"15min":0.64700000000000002},"min":{"1min":0.52300000000000002,"5min":0.52300000000000002,"15min":0.52300000000000002},"max":{"1min":0.78400000000000003,"5min":0.78400000000000003,"15min":0.78400000000000003},"last":0.55500000000000005},{"interface":"front","average":{"1min":0.81999999999999995,"5min":0.81999999999999995,"15min":0.81999999999999995},"min":{"1min":0.44500000000000001,"5min":0.44500000000000001,"15min":0.44500000000000001},"max":{"1min":4.3140000000000001,"5min":4.3140000000000001,"15min":4.3140000000000001},"last":0.65700000000000003}]},{"osd":2,"last update":"Sun Nov 26 07:18:46 2023","interfaces":[{"interface":"back","average":{"1min":0.80700000000000005,"5min":0.80700000000000005,"15min":0.80700000000000005},"min":{"1min":0.57799999999999996,"5min":0.57799999999999996,"15min":0.57799999999999996},"max":{"1min":2.7850000000000001,"5min":2.7850000000000001,"15min":2.7850000000000001},"last":0.69299999999999995},{"interface":"front","average":{"1min":1.0660000000000001,"5min":1.0660000000000001,"15min":1.0660000000000001},"min":{"1min":0.53400000000000003,"5min":0.53400000000000003,"15min":0.53400000000000003},"max":{"1min":7.6470000000000002,"5min":7.6470000000000002,"15min":7.6470000000000002},"last":0.68000000000000005}]},{"osd":3,"last update":"Sun Nov 26 07:18:46 2023","interfaces":[{"interface":"back","average":{"1min":0.72999999999999998,"5min":0.72999999999999998,"15min":0.72999999999999998},"min":{"1min":0.48599999999999999,"5min":0.48599999999999999,"15min":0.48599999999999999},"max":{"1min":1.591,"5min":1.591,"15min":1.591},"last":0.58599999999999997},{"interface":"front","average":{"1min":0.879,"5min":0.879,"15min":0.879},"min":{"1min":0.56899999999999995,"5min":0.56899999999999995,"15min":0.56899999999999995},"max":{"1min":3.452,"5min":3.452,"15min":3.452},"last":0.72899999999999998}]},{"osd":5,"last update":"Sun Nov 26 07:18:46 2023","interfaces":[{"interface":"back","average":{"1min":0.61099999999999999,"5min":0.61099999999999999,"15min":0.61099999999999999},"min":{"1min":0.43099999999999999,"5min":0.43099999999999999,"15min":0.43099999999999999},"max":{"1min":0.96199999999999997,"5min":0.96199999999999997,"15min":0.96199999999999997},"last":0.64500000000000002},{"interface":"front","average":{"1min":0.64100000000000001,"5min":0.64100000000000001,"15min":0.64100000000000001},"min":{"1min":0.41499999999999998,"5min":0.41499999999999998,"15min":0.41499999999999998},"max":{"1min":1.538,"5min":1.538,"15min":1.538},"last":0.47099999999999997}]},{"osd":6,"last update":"Sun Nov 26 07:18:46 2023","interfaces":[{"interface":"back","average":{"1min":0.79400000000000004,"5min":0.79400000000000004,"15min":0.79400000000000004},"min":{"1min":0.39400000000000002,"5min":0.39400000000000002,"15min":0.39400000000000002},"max":{"1min":2.8330000000000002,"5min":2.8330000000000002,"15min":2.8330000000000002},"last":0.79100000000000004},{"interface":"front","average":{"1min":1.0089999999999999,"5min":1.0089999999999999,"15min":1.0089999999999999},"min":{"1min":0.42899999999999999,"5min":0.42899999999999999,"15min":0.42899999999999999},"max":{"1min":7.6050000000000004,"5min":7.6050000000000004,"15min":7.6050000000000004},"last":0.60099999999999998}]},{"osd":7,"last update":"Sun Nov 26 07:18:46 2023","interfaces":[{"interface":"back","average":{"1min":0.86099999999999999,"5min":0.86099999999999999,"15min":0.86099999999999999},"min":{"1min":0.44700000000000001,"5min":0.44700000000000001,"15min":0.44700000000000001},"max":{"1min":4.3689999999999998,"5min":4.3689999999999998,"15min":4.3689999999999998},"last":0.53600000000000003},{"interface":"front","average":{"1min":0.89200000000000002,"5min":0.89200000000000002,"15min":0.89200000000000002},"min":{"1min":0.505,"5min":0.505,"15min":0.505},"max":{"1min":3.403,"5min":3.403,"15min":3.403},"last":0.76300000000000001}]}]},{"osd":3,"up_from":13,"seq":55834574875,"num_pgs":7,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27512,"kb_used_data":492,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344328,"statfs":{"total":96636764160,"available":96608591872,"internally_reserved":0,"allocated":503808,"data_stored":202024,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1592,"internal_metadata":27654600},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Nov 26 07:18:43 2023","interfaces":[{"interface":"back","average":{"1min":0.379,"5min":0.379,"15min":0.379},"min":{"1min":0.26100000000000001,"5min":0.26100000000000001,"15min":0.26100000000000001},"max":{"1min":0.58599999999999997,"5min":0.58599999999999997,"15min":0.58599999999999997},"last":0.34999999999999998},{"interface":"front","average":{"1min":0.433,"5min":0.433,"15min":0.433},"min":{"1min":0.30199999999999999,"5min":0.30199999999999999,"15min":0.30199999999999999},"max":{"1min":0.63,"5min":0.63,"15min":0.63},"last":0.503}]},{"osd":1,"last update":"Sun Nov 26 07:18:43 2023","interfaces":[{"interface":"back","average":{"1min":0.42099999999999999,"5min":0.42099999999999999,"15min":0.42099999999999999},"min":{"1min":0.312,"5min":0.312,"15min":0.312},"max":{"1min":0.752,"5min":0.752,"15min":0.752},"last":0.40300000000000002},{"interface":"front","average":{"1min":0.48399999999999999,"5min":0.48399999999999999,"15min":0.48399999999999999},"min":{"1min":0.38,"5min":0.38,"15min":0.38},"max":{"1min":0.628,"5min":0.628,"15min":0.628},"last":0.54100000000000004}]},{"osd":2,"last update":"Sun Nov 26 07:18:43 2023","interfaces":[{"interface":"back","average":{"1min":0.53300000000000003,"5min":0.53300000000000003,"15min":0.53300000000000003},"min":{"1min":0.40300000000000002,"5min":0.40300000000000002,"15min":0.40300000000000002},"max":{"1min":0.82199999999999995,"5min":0.82199999999999995,"15min":0.82199999999999995},"last":0.57399999999999995},{"interface":"front","average":{"1min":0.505,"5min":0.505,"15min":0.505},"min":{"1min":0.35999999999999999,"5min":0.35999999999999999,"15min":0.35999999999999999},"max":{"1min":0.72199999999999998,"5min":0.72199999999999998,"15min":0.72199999999999998},"last":0.46700000000000003}]},{"osd":4,"last update":"Sun Nov 26 07:18:43 2023","interfaces":[{"interface":"back","average":{"1min":0.70599999999999996,"5min":0.70599999999999996,"15min":0.70599999999999996},"min":{"1min":0.48199999999999998,"5min":0.48199999999999998,"15min":0.48199999999999998},"max":{"1min":0.96199999999999997,"5min":0.96199999999999997,"15min":0.96199999999999997},"last":0.81200000000000006},{"interface":"front","average":{"1min":0.72799999999999998,"5min":0.72799999999999998,"15min":0.72799999999999998},"min":{"1min":0.441,"5min":0.441,"15min":0.441},"max":{"1min":1.171,"5min":1.171,"15min":1.171},"last":0.74299999999999999}]},{"osd":5,"last update":"Sun Nov 26 07:18:43 2023","interfaces":[{"interface":"back","average":{"1min":0.73899999999999999,"5min":0.73899999999999999,"15min":0.73899999999999999},"min":{"1min":0.53700000000000003,"5min":0.53700000000000003,"15min":0.53700000000000003},"max":{"1min":0.94099999999999995,"5min":0.94099999999999995,"15min":0.94099999999999995},"last":0.68000000000000005},{"interface":"front","average":{"1min":0.73799999999999999,"5min":0.73799999999999999,"15min":0.73799999999999999},"min":{"1min":0.504,"5min":0.504,"15min":0.504},"max":{"1min":1.0820000000000001,"5min":1.0820000000000001,"15min":1.0820000000000001},"last":0.71599999999999997}]},{"osd":6,"last update":"Sun Nov 26 07:18:43 2023","interfaces":[{"interface":"back","average":{"1min":0.81399999999999995,"5min":0.81399999999999995,"15min":0.81399999999999995},"min":{"1min":0.55200000000000005,"5min":0.55200000000000005,"15min":0.55200000000000005},"max":{"1min":1.2729999999999999,"5min":1.2729999999999999,"15min":1.2729999999999999},"last":0.83599999999999997},{"interface":"front","average":{"1min":0.79300000000000004,"5min":0.79300000000000004,"15min":0.79300000000000004},"min":{"1min":0.47299999999999998,"5min":0.47299999999999998,"15min":0.47299999999999998},"max":{"1min":1.204,"5min":1.204,"15min":1.204},"last":0.75900000000000001}]},{"osd":7,"last update":"Sun Nov 26 07:18:43 2023","interfaces":[{"interface":"back","average":{"1min":0.80100000000000005,"5min":0.80100000000000005,"15min":0.80100000000000005},"min":{"1min":0.63300000000000001,"5min":0.63300000000000001,"15min":0.63300000000000001},"max":{"1min":1.3600000000000001,"5min":1.3600000000000001,"15min":1.3600000000000001},"last":0.63900000000000001},{"interface":"front","average":{"1min":0.82599999999999996,"5min":0.82599999999999996,"15min":0.82599999999999996},"min":{"1min":0.50600000000000001,"5min":0.50600000000000001,"15min":0.50600000000000001},"max":{"1min":1.617,"5min":1.617,"15min":1.617},"last":0.79300000000000004}]}]},{"osd":2,"up_from":13,"seq":55834574875,"num_pgs":11,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27512,"kb_used_data":492,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344328,"statfs":{"total":96636764160,"available":96608591872,"internally_reserved":0,"allocated":503808,"data_stored":202024,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Nov 26 07:18:43 2023","interfaces":[{"interface":"back","average":{"1min":0.71399999999999997,"5min":0.71399999999999997,"15min":0.71399999999999997},"min":{"1min":0.28699999999999998,"5min":0.28699999999999998,"15min":0.28699999999999998},"max":{"1min":4.7030000000000003,"5min":4.7030000000000003,"15min":4.7030000000000003},"last":0.41099999999999998},{"interface":"front","average":{"1min":0.56200000000000006,"5min":0.56200000000000006,"15min":0.56200000000000006},"min":{"1min":0.32300000000000001,"5min":0.32300000000000001,"15min":0.32300000000000001},"max":{"1min":2.4340000000000002,"5min":2.4340000000000002,"15min":2.4340000000000002},"last":0.438}]},{"osd":1,"last update":"Sun Nov 26 07:18:43 2023","interfaces":[{"interface":"back","average":{"1min":0.86399999999999999,"5min":0.86399999999999999,"15min":0.86399999999999999},"min":{"1min":0.34599999999999997,"5min":0.34599999999999997,"15min":0.34599999999999997},"max":{"1min":4.2690000000000001,"5min":4.2690000000000001,"15min":4.2690000000000001},"last":0.46300000000000002},{"interface":"front","average":{"1min":0.74199999999999999,"5min":0.74199999999999999,"15min":0.74199999999999999},"min":{"1min":0.31900000000000001,"5min":0.31900000000000001,"15min":0.31900000000000001},"max":{"1min":3.9940000000000002,"5min":3.9940000000000002,"15min":3.9940000000000002},"last":0.38800000000000001}]},{"osd":3,"last update":"Sun Nov 26 07:18:43 2023","interfaces":[{"interface":"back","average":{"1min":0.64300000000000002,"5min":0.64300000000000002,"15min":0.64300000000000002},"min":{"1min":0.34399999999999997,"5min":0.34399999999999997,"15min":0.34399999999999997},"max":{"1min":4.0300000000000002,"5min":4.0300000000000002,"15min":4.0300000000000002},"last":0.36899999999999999},{"interface":"front","average":{"1min":0.91400000000000003,"5min":0.91400000000000003,"15min":0.91400000000000003},"min":{"1min":0.41699999999999998,"5min":0.41699999999999998,"15min":0.41699999999999998},"max":{"1min":4.5810000000000004,"5min":4.5810000000000004,"15min":4.5810000000000004},"last":0.50900000000000001}]},{"osd":4,"last update":"Sun Nov 26 07:18:43 2023","interfaces":[{"interface":"back","average":{"1min":0.98499999999999999,"5min":0.98499999999999999,"15min":0.98499999999999999},"min":{"1min":0.55100000000000005,"5min":0.55100000000000005,"15min":0.55100000000000005},"max":{"1min":4.4269999999999996,"5min":4.4269999999999996,"15min":4.4269999999999996},"last":0.80900000000000005},{"interface":"front","average":{"1min":1.0149999999999999,"5min":1.0149999999999999,"15min":1.0149999999999999},"min":{"1min":0.60299999999999998,"5min":0.60299999999999998,"15min":0.60299999999999998},"max":{"1min":4.085,"5min":4.085,"15min":4.085},"last":0.68100000000000005}]},{"osd":5,"last update":"Sun Nov 26 07:18:43 2023","interfaces":[{"interface":"back","average":{"1min":0.874,"5min":0.874,"15min":0.874},"min":{"1min":0.52900000000000003,"5min":0.52900000000000003,"15min":0.52900000000000003},"max":{"1min":3.9590000000000001,"5min":3.9590000000000001,"15min":3.9590000000000001},"last":0.65000000000000002},{"interface":"front","average":{"1min":0.98799999999999999,"5min":0.98799999999999999,"15min":0.98799999999999999},"min":{"1min":0.58799999999999997,"5min":0.58799999999999997,"15min":0.58799999999999997},"max":{"1min":4.3849999999999998,"5min":4.3849999999999998,"15min":4.3849999999999998},"last":0.752}]},{"osd":6,"last update":"Sun Nov 26 07:18:43 2023","interfaces":[{"interface":"back","average":{"1min":0.996,"5min":0.996,"15min":0.996},"min":{"1min":0.56599999999999995,"5min":0.56599999999999995,"15min":0.56599999999999995},"max":{"1min":4.0449999999999999,"5min":4.0449999999999999,"15min":4.0449999999999999},"last":0.77700000000000002},{"interface":"front","average":{"1min":0.91800000000000004,"5min":0.91800000000000004,"15min":0.91800000000000004},"min":{"1min":0.57099999999999995,"5min":0.57099999999999995,"15min":0.57099999999999995},"max":{"1min":3.992,"5min":3.992,"15min":3.992},"last":0.60699999999999998}]},{"osd":7,"last update":"Sun Nov 26 07:18:43 2023","interfaces":[{"interface":"back","average":{"1min":1.0349999999999999,"5min":1.0349999999999999,"15min":1.0349999999999999},"min":{"1min":0.63100000000000001,"5min":0.63100000000000001,"15min":0.63100000000000001},"max":{"1min":4.1219999999999999,"5min":4.1219999999999999,"15min":4.1219999999999999},"last":0.63},{"interface":"front","average":{"1min":0.93799999999999994,"5min":0.93799999999999994,"15min":0.93799999999999994},"min":{"1min":0.57699999999999996,"5min":0.57699999999999996,"15min":0.57699999999999996},"max":{"1min":3.9159999999999999,"5min":3.9159999999999999,"15min":3.9159999999999999},"last":0.73599999999999999}]}]},{"osd":1,"up_from":13,"seq":55834574874,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27512,"kb_used_data":492,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344328,"statfs":{"total":96636764160,"available":96608591872,"internally_reserved":0,"allocated":503808,"data_stored":202024,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Nov 26 07:18:41 2023","interfaces":[{"interface":"back","average":{"1min":0.41699999999999998,"5min":0.41699999999999998,"15min":0.41699999999999998},"min":{"1min":0.26300000000000001,"5min":0.26300000000000001,"15min":0.26300000000000001},"max":{"1min":0.61199999999999999,"5min":0.61199999999999999,"15min":0.61199999999999999},"last":0.36299999999999999},{"interface":"front","average":{"1min":0.48599999999999999,"5min":0.48599999999999999,"15min":0.48599999999999999},"min":{"1min":0.30499999999999999,"5min":0.30499999999999999,"15min":0.30499999999999999},"max":{"1min":0.78700000000000003,"5min":0.78700000000000003,"15min":0.78700000000000003},"last":0.433}]},{"osd":2,"last update":"Sun Nov 26 07:18:41 2023","interfaces":[{"interface":"back","average":{"1min":0.45500000000000002,"5min":0.45500000000000002,"15min":0.45500000000000002},"min":{"1min":0.33400000000000002,"5min":0.33400000000000002,"15min":0.33400000000000002},"max":{"1min":0.67600000000000005,"5min":0.67600000000000005,"15min":0.67600000000000005},"last":0.47099999999999997},{"interface":"front","average":{"1min":0.50800000000000001,"5min":0.50800000000000001,"15min":0.50800000000000001},"min":{"1min":0.32300000000000001,"5min":0.32300000000000001,"15min":0.32300000000000001},"max":{"1min":0.66500000000000004,"5min":0.66500000000000004,"15min":0.66500000000000004},"last":0.64400000000000002}]},{"osd":3,"last update":"Sun Nov 26 07:18:41 2023","interfaces":[{"interface":"back","average":{"1min":0.53000000000000003,"5min":0.53000000000000003,"15min":0.53000000000000003},"min":{"1min":0.38400000000000001,"5min":0.38400000000000001,"15min":0.38400000000000001},"max":{"1min":0.73699999999999999,"5min":0.73699999999999999,"15min":0.73699999999999999},"last":0.495},{"interface":"front","average":{"1min":0.497,"5min":0.497,"15min":0.497},"min":{"1min":0.36199999999999999,"5min":0.36199999999999999,"15min":0.36199999999999999},"max":{"1min":0.80700000000000005,"5min":0.80700000000000005,"15min":0.80700000000000005},"last":0.51500000000000001}]},{"osd":4,"last update":"Sun Nov 26 07:18:41 2023","interfaces":[{"interface":"back","average":{"1min":0.69399999999999995,"5min":0.69399999999999995,"15min":0.69399999999999995},"min":{"1min":0.48599999999999999,"5min":0.48599999999999999,"15min":0.48599999999999999},"max":{"1min":1.0229999999999999,"5min":1.0229999999999999,"15min":1.0229999999999999},"last":0.56599999999999995},{"interface":"front","average":{"1min":0.68000000000000005,"5min":0.68000000000000005,"15min":0.68000000000000005},"min":{"1min":0.49199999999999999,"5min":0.49199999999999999,"15min":0.49199999999999999},"max":{"1min":1.0629999999999999,"5min":1.0629999999999999,"15min":1.0629999999999999},"last":0.71399999999999997}]},{"osd":5,"last update":"Sun Nov 26 07:18:41 2023","interfaces":[{"interface":"back","average":{"1min":0.71899999999999997,"5min":0.71899999999999997,"15min":0.71899999999999997},"min":{"1min":0.49299999999999999,"5min":0.49299999999999999,"15min":0.49299999999999999},"max":{"1min":1.0880000000000001,"5min":1.0880000000000001,"15min":1.0880000000000001},"last":0.625},{"interface":"front","average":{"1min":0.71699999999999997,"5min":0.71699999999999997,"15min":0.71699999999999997},"min":{"1min":0.53500000000000003,"5min":0.53500000000000003,"15min":0.53500000000000003},"max":{"1min":1.014,"5min":1.014,"15min":1.014},"last":0.70099999999999996}]},{"osd":6,"last update":"Sun Nov 26 07:18:41 2023","interfaces":[{"interface":"back","average":{"1min":0.70399999999999996,"5min":0.70399999999999996,"15min":0.70399999999999996},"min":{"1min":0.50600000000000001,"5min":0.50600000000000001,"15min":0.50600000000000001},"max":{"1min":0.97599999999999998,"5min":0.97599999999999998,"15min":0.97599999999999998},"last":0.72899999999999998},{"interface":"front","average":{"1min":0.70099999999999996,"5min":0.70099999999999996,"15min":0.70099999999999996},"min":{"1min":0.55000000000000004,"5min":0.55000000000000004,"15min":0.55000000000000004},"max":{"1min":1.1339999999999999,"5min":1.1339999999999999,"15min":1.1339999999999999},"last":0.58699999999999997}]},{"osd":7,"last update":"Sun Nov 26 07:18:41 2023","interfaces":[{"interface":"back","average":{"1min":0.71499999999999997,"5min":0.71499999999999997,"15min":0.71499999999999997},"min":{"1min":0.55100000000000005,"5min":0.55100000000000005,"15min":0.55100000000000005},"max":{"1min":0.92900000000000005,"5min":0.92900000000000005,"15min":0.92900000000000005},"last":0.74099999999999999},{"interface":"front","average":{"1min":0.753,"5min":0.753,"15min":0.753},"min":{"1min":0.56100000000000005,"5min":0.56100000000000005,"15min":0.56100000000000005},"max":{"1min":1.024,"5min":1.024,"15min":1.024},"last":0.68200000000000005}]}]},{"osd":0,"up_from":13,"seq":55834574874,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":28080,"kb_used_data":1060,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343760,"statfs":{"total":96636764160,"available":96608010240,"internally_reserved":0,"allocated":1085440,"data_stored":786125,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Sun Nov 26 07:18:41 2023","interfaces":[{"interface":"back","average":{"1min":0.43099999999999999,"5min":0.43099999999999999,"15min":0.43099999999999999},"min":{"1min":0.28000000000000003,"5min":0.28000000000000003,"15min":0.28000000000000003},"max":{"1min":0.63100000000000001,"5min":0.63100000000000001,"15min":0.63100000000000001},"last":0.40799999999999997},{"interface":"front","average":{"1min":0.48299999999999998,"5min":0.48299999999999998,"15min":0.48299999999999998},"min":{"1min":0.29899999999999999,"5min":0.29899999999999999,"15min":0.29899999999999999},"max":{"1min":1.133,"5min":1.133,"15min":1.133},"last":0.35399999999999998}]},{"osd":2,"last update":"Sun Nov 26 07:18:41 2023","interfaces":[{"interface":"back","average":{"1min":0.53700000000000003,"5min":0.53700000000000003,"15min":0.53700000000000003},"min":{"1min":0.40899999999999997,"5min":0.40899999999999997,"15min":0.40899999999999997},"max":{"1min":0.67400000000000004,"5min":0.67400000000000004,"15min":0.67400000000000004},"last":0.56299999999999994},{"interface":"front","average":{"1min":0.56399999999999995,"5min":0.56399999999999995,"15min":0.56399999999999995},"min":{"1min":0.36399999999999999,"5min":0.36399999999999999,"15min":0.36399999999999999},"max":{"1min":0.72499999999999998,"5min":0.72499999999999998,"15min":0.72499999999999998},"last":0.48599999999999999}]},{"osd":3,"last update":"Sun Nov 26 07:18:41 2023","interfaces":[{"interface":"back","average":{"1min":0.53100000000000003,"5min":0.53100000000000003,"15min":0.53100000000000003},"min":{"1min":0.35799999999999998,"5min":0.35799999999999998,"15min":0.35799999999999998},"max":{"1min":0.97399999999999998,"5min":0.97399999999999998,"15min":0.97399999999999998},"last":1.3149999999999999},{"interface":"front","average":{"1min":0.56200000000000006,"5min":0.56200000000000006,"15min":0.56200000000000006},"min":{"1min":0.40600000000000003,"5min":0.40600000000000003,"15min":0.40600000000000003},"max":{"1min":1.2509999999999999,"5min":1.2509999999999999,"15min":1.2509999999999999},"last":0.65100000000000002}]},{"osd":4,"last update":"Sun Nov 26 07:18:41 2023","interfaces":[{"interface":"back","average":{"1min":0.75900000000000001,"5min":0.75900000000000001,"15min":0.75900000000000001},"min":{"1min":0.60999999999999999,"5min":0.60999999999999999,"15min":0.60999999999999999},"max":{"1min":1.2010000000000001,"5min":1.2010000000000001,"15min":1.2010000000000001},"last":0.60999999999999999},{"interface":"front","average":{"1min":0.78900000000000003,"5min":0.78900000000000003,"15min":0.78900000000000003},"min":{"1min":0.59999999999999998,"5min":0.59999999999999998,"15min":0.59999999999999998},"max":{"1min":0.99399999999999999,"5min":0.99399999999999999,"15min":0.99399999999999999},"last":1.381}]},{"osd":5,"last update":"Sun Nov 26 07:18:41 2023","interfaces":[{"interface":"back","average":{"1min":0.76000000000000001,"5min":0.76000000000000001,"15min":0.76000000000000001},"min":{"1min":0.53000000000000003,"5min":0.53000000000000003,"15min":0.53000000000000003},"max":{"1min":1.1579999999999999,"5min":1.1579999999999999,"15min":1.1579999999999999},"last":0.72699999999999998},{"interface":"front","average":{"1min":0.79700000000000004,"5min":0.79700000000000004,"15min":0.79700000000000004},"min":{"1min":0.59699999999999998,"5min":0.59699999999999998,"15min":0.59699999999999998},"max":{"1min":1.0409999999999999,"5min":1.0409999999999999,"15min":1.0409999999999999},"last":1.399}]},{"osd":6,"last update":"Sun Nov 26 07:18:41 2023","interfaces":[{"interface":"back","average":{"1min":0.76400000000000001,"5min":0.76400000000000001,"15min":0.76400000000000001},"min":{"1min":0.56599999999999995,"5min":0.56599999999999995,"15min":0.56599999999999995},"max":{"1min":1.3029999999999999,"5min":1.3029999999999999,"15min":1.3029999999999999},"last":0.69599999999999995},{"interface":"front","average":{"1min":0.76900000000000002,"5min":0.76900000000000002,"15min":0.76900000000000002},"min":{"1min":0.58599999999999997,"5min":0.58599999999999997,"15min":0.58599999999999997},"max":{"1min":1.2210000000000001,"5min":1.2210000000000001,"15min":1.2210000000000001},"last":1.3600000000000001}]},{"osd":7,"last update":"Sun Nov 26 07:18:41 2023","interfaces":[{"interface":"back","average":{"1min":0.81100000000000005,"5min":0.81100000000000005,"15min":0.81100000000000005},"min":{"1min":0.59999999999999998,"5min":0.59999999999999998,"15min":0.59999999999999998},"max":{"1min":1.177,"5min":1.177,"15min":1.177},"last":0.84099999999999997},{"interface":"front","average":{"1min":0.79900000000000004,"5min":0.79900000000000004,"15min":0.79900000000000004},"min":{"1min":0.60899999999999999,"5min":0.60899999999999999,"15min":0.60899999999999999},"max":{"1min":1.2390000000000001,"5min":1.2390000000000001,"15min":1.2390000000000001},"last":0.67300000000000004}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-11-26T07:19:36.885 DEBUG:teuthology.orchestra.run.smithi143:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-11-26T07:19:37.056 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:37.051+0000 7fc08292b640 1 Processor -- start 2023-11-26T07:19:37.057 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:37.055+0000 7fc08292b640 1 -- start start 2023-11-26T07:19:37.058 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:37.055+0000 7fc08292b640 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7fc07c143810 0x7fc07c072140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:19:37.059 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:37.055+0000 7fc08292b640 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7fc07c15e0f0 0x7fc07c0ce030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:19:37.059 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:37.055+0000 7fc08292b640 1 -- --> v1:172.21.15.188:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc07c15e860 con 0x7fc07c0b1e10 2023-11-26T07:19:37.059 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:37.055+0000 7fc08292b640 1 -- --> v2:172.21.15.143:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc07c0cdba0 con 0x7fc07c143810 2023-11-26T07:19:37.059 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:37.055+0000 7fc08292b640 1 -- --> v2:172.21.15.188:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc07c143bf0 con 0x7fc07c15e0f0 2023-11-26T07:19:37.059 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:37.055+0000 7fc081929640 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7fc07c143810 0x7fc07c072140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:19:37.059 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:37.055+0000 7fc081929640 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7fc07c143810 0x7fc07c072140 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:52246/0 (socket says 172.21.15.143:52246) 2023-11-26T07:19:37.059 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:37.055+0000 7fc081929640 1 -- 172.21.15.143:0/3610611187 learned_addr learned my addr 172.21.15.143:0/3610611187 (peer_addr_for_me v2:172.21.15.143:0/0) 2023-11-26T07:19:37.059 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:37.055+0000 7fc08212a640 1 --2- 172.21.15.143:0/3610611187 >> v2:172.21.15.188:3300/0 conn(0x7fc07c15e0f0 0x7fc07c0ce030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:19:37.059 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:37.055+0000 7fc081929640 1 -- 172.21.15.143:0/3610611187 >> v2:172.21.15.188:3300/0 conn(0x7fc07c15e0f0 msgr2=0x7fc07c0ce030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-26T07:19:37.059 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:37.055+0000 7fc081929640 1 --2- 172.21.15.143:0/3610611187 >> v2:172.21.15.188:3300/0 conn(0x7fc07c15e0f0 0x7fc07c0ce030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:19:37.060 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:37.055+0000 7fc081929640 1 -- 172.21.15.143:0/3610611187 >> v1:172.21.15.188:6789/0 conn(0x7fc07c0b1e10 legacy=0x7fc07c15d9e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:19:37.060 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:37.055+0000 7fc081929640 1 -- 172.21.15.143:0/3610611187 --> v2:172.21.15.143:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc07c0714f0 con 0x7fc07c143810 2023-11-26T07:19:37.061 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:37.059+0000 7fc081929640 1 --2- 172.21.15.143:0/3610611187 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fc07c143810 0x7fc07c072140 secure :-1 s=READY pgs=581 cs=0 l=1 rev1=1 crypto rx=0x7fc07c06d020 tx=0x7fc07801c4d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f94339fd7421de2 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:19:37.062 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:37.059+0000 7fc080927640 1 -- 172.21.15.143:0/3610611187 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc0780aa430 con 0x7fc07c143810 2023-11-26T07:19:37.062 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:37.059+0000 7fc080927640 1 -- 172.21.15.143:0/3610611187 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc0780aa590 con 0x7fc07c143810 2023-11-26T07:19:37.062 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:37.059+0000 7fc080927640 1 -- 172.21.15.143:0/3610611187 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc0780aa920 con 0x7fc07c143810 2023-11-26T07:19:37.063 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:37.059+0000 7fc08292b640 1 -- 172.21.15.143:0/3610611187 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fc07c143810 msgr2=0x7fc07c072140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:19:37.063 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:37.059+0000 7fc08292b640 1 --2- 172.21.15.143:0/3610611187 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fc07c143810 0x7fc07c072140 secure :-1 s=READY pgs=581 cs=0 l=1 rev1=1 crypto rx=0x7fc07c06d020 tx=0x7fc07801c4d0 comp rx=0 tx=0).stop 2023-11-26T07:19:37.063 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:37.059+0000 7fc08292b640 1 -- 172.21.15.143:0/3610611187 shutdown_connections 2023-11-26T07:19:37.063 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:37.059+0000 7fc08292b640 1 --2- 172.21.15.143:0/3610611187 >> v2:172.21.15.188:3300/0 conn(0x7fc07c15e0f0 0x7fc07c0ce030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:19:37.063 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:37.059+0000 7fc08292b640 1 --2- 172.21.15.143:0/3610611187 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fc07c143810 0x7fc07c072140 unknown :-1 s=CLOSED pgs=581 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:19:37.063 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:37.059+0000 7fc08292b640 1 -- 172.21.15.143:0/3610611187 >> 172.21.15.143:0/3610611187 conn(0x7fc07c06a890 msgr2=0x7fc07c06aca0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:19:37.063 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:37.059+0000 7fc08292b640 1 -- 172.21.15.143:0/3610611187 shutdown_connections 2023-11-26T07:19:37.063 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:37.059+0000 7fc08292b640 1 -- 172.21.15.143:0/3610611187 wait complete. 2023-11-26T07:19:37.064 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:37.059+0000 7fc08292b640 1 Processor -- start 2023-11-26T07:19:37.064 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:37.059+0000 7fc08292b640 1 -- start start 2023-11-26T07:19:37.066 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:37.063+0000 7fc08292b640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fc07c0b1e10 0x7fc07c166250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:19:37.066 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:37.063+0000 7fc08292b640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fc07c143810 0x7fc07c153aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:19:37.066 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:37.063+0000 7fc08292b640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fc07c15e0f0 0x7fc07c0b2600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:19:37.066 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:37.063+0000 7fc08292b640 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc07c13a7e0 con 0x7fc07c143810 2023-11-26T07:19:37.066 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:37.063+0000 7fc08292b640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc07c1668e0 con 0x7fc07c0b1e10 2023-11-26T07:19:37.066 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:37.063+0000 7fc08292b640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc07c166bc0 con 0x7fc07c15e0f0 2023-11-26T07:19:37.066 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:37.063+0000 7fc081128640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fc07c143810 0x7fc07c153aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:19:37.066 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:37.063+0000 7fc081128640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fc07c143810 0x7fc07c153aa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:52256/0 (socket says 172.21.15.143:52256) 2023-11-26T07:19:37.066 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:37.063+0000 7fc081128640 1 -- 172.21.15.143:0/2066883910 learned_addr learned my addr 172.21.15.143:0/2066883910 (peer_addr_for_me v2:172.21.15.143:0/0) 2023-11-26T07:19:37.067 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:37.063+0000 7fc081929640 1 --2- 172.21.15.143:0/2066883910 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fc07c0b1e10 0x7fc07c166250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:19:37.067 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:37.063+0000 7fc08212a640 1 --2- 172.21.15.143:0/2066883910 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fc07c15e0f0 0x7fc07c0b2600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:19:37.068 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:37.063+0000 7fc081128640 1 -- 172.21.15.143:0/2066883910 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fc07c15e0f0 msgr2=0x7fc07c0b2600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:19:37.068 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:37.063+0000 7fc081128640 1 --2- 172.21.15.143:0/2066883910 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fc07c15e0f0 0x7fc07c0b2600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:19:37.068 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:37.063+0000 7fc081128640 1 -- 172.21.15.143:0/2066883910 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fc07c0b1e10 msgr2=0x7fc07c166250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:19:37.068 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:37.063+0000 7fc081128640 1 --2- 172.21.15.143:0/2066883910 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fc07c0b1e10 0x7fc07c166250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:19:37.068 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:37.063+0000 7fc081128640 1 -- 172.21.15.143:0/2066883910 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc0780094d0 con 0x7fc07c143810 2023-11-26T07:19:37.068 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:37.063+0000 7fc081128640 1 --2- 172.21.15.143:0/2066883910 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fc07c143810 0x7fc07c153aa0 secure :-1 s=READY pgs=582 cs=0 l=1 rev1=1 crypto rx=0x7fc06c00c3c0 tx=0x7fc06c00c880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:19:37.068 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:37.063+0000 7fc0727fc640 1 -- 172.21.15.143:0/2066883910 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc06c0052d0 con 0x7fc07c143810 2023-11-26T07:19:37.068 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:37.063+0000 7fc0727fc640 1 -- 172.21.15.143:0/2066883910 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc06c005430 con 0x7fc07c143810 2023-11-26T07:19:37.068 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:37.063+0000 7fc0727fc640 1 -- 172.21.15.143:0/2066883910 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc06c003840 con 0x7fc07c143810 2023-11-26T07:19:37.069 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:37.063+0000 7fc08292b640 1 -- 172.21.15.143:0/2066883910 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc07c166ea0 con 0x7fc07c143810 2023-11-26T07:19:37.069 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:37.063+0000 7fc08292b640 1 -- 172.21.15.143:0/2066883910 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc07c13e750 con 0x7fc07c143810 2023-11-26T07:19:37.069 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:37.063+0000 7fc08292b640 1 -- 172.21.15.143:0/2066883910 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc07c0ce010 con 0x7fc07c143810 2023-11-26T07:19:37.069 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:37.067+0000 7fc0727fc640 1 -- 172.21.15.143:0/2066883910 <== mon.0 v2:172.21.15.143:3300/0 4 ==== mgrmap(e 7) v1 ==== 86005+0+0 (secure 0 0 0) 0x7fc06c0039a0 con 0x7fc07c143810 2023-11-26T07:19:37.069 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:37.067+0000 7fc0727fc640 1 --2- 172.21.15.143:0/2066883910 >> [v2:172.21.15.143:6832/2150557544,v1:172.21.15.143:6833/2150557544] conn(0x7fc060066620 0x7fc060068ae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:19:37.070 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:37.067+0000 7fc0727fc640 1 -- 172.21.15.143:0/2066883910 <== mon.0 v2:172.21.15.143:3300/0 5 ==== osd_map(45..45 src has 1..45) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fc06c0835d0 con 0x7fc07c143810 2023-11-26T07:19:37.070 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:37.067+0000 7fc081929640 1 --2- 172.21.15.143:0/2066883910 >> [v2:172.21.15.143:6832/2150557544,v1:172.21.15.143:6833/2150557544] conn(0x7fc060066620 0x7fc060068ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:19:37.070 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:37.067+0000 7fc081929640 1 --2- 172.21.15.143:0/2066883910 >> [v2:172.21.15.143:6832/2150557544,v1:172.21.15.143:6833/2150557544] conn(0x7fc060066620 0x7fc060068ae0 secure :-1 s=READY pgs=292 cs=0 l=1 rev1=1 crypto rx=0x7fc0780ae000 tx=0x7fc0780b5040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:19:37.074 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:37.071+0000 7fc0727fc640 1 -- 172.21.15.143:0/2066883910 <== mon.0 v2:172.21.15.143:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172649 (secure 0 0 0) 0x7fc06c050920 con 0x7fc07c143810 2023-11-26T07:19:37.108 INFO:tasks.ceph.mds.c.smithi143.stderr:2023-11-26T07:19:37.103+0000 7f978e6e6640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2023-11-26T07:19:37.220 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:37.219+0000 7fc08292b640 1 -- 172.21.15.143:0/2066883910 --> [v2:172.21.15.143:6832/2150557544,v1:172.21.15.143:6833/2150557544] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7fc07c0ce9f0 con 0x7fc060066620 2023-11-26T07:19:37.228 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:37.223+0000 7fc0727fc640 1 -- 172.21.15.143:0/2066883910 <== mgr.4100 v2:172.21.15.143:6832/2150557544 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+284367 (secure 0 0 0) 0x7fc07c0ce9f0 con 0x7fc060066620 2023-11-26T07:19:37.231 INFO:teuthology.orchestra.run.smithi143.stdout: 2023-11-26T07:19:37.232 INFO:teuthology.orchestra.run.smithi143.stderr:dumped all 2023-11-26T07:19:37.233 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:37.227+0000 7fc08292b640 1 -- 172.21.15.143:0/2066883910 >> [v2:172.21.15.143:6832/2150557544,v1:172.21.15.143:6833/2150557544] conn(0x7fc060066620 msgr2=0x7fc060068ae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:19:37.233 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:37.227+0000 7fc08292b640 1 --2- 172.21.15.143:0/2066883910 >> [v2:172.21.15.143:6832/2150557544,v1:172.21.15.143:6833/2150557544] conn(0x7fc060066620 0x7fc060068ae0 secure :-1 s=READY pgs=292 cs=0 l=1 rev1=1 crypto rx=0x7fc0780ae000 tx=0x7fc0780b5040 comp rx=0 tx=0).stop 2023-11-26T07:19:37.233 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:37.227+0000 7fc08292b640 1 -- 172.21.15.143:0/2066883910 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fc07c143810 msgr2=0x7fc07c153aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:19:37.233 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:37.227+0000 7fc08292b640 1 --2- 172.21.15.143:0/2066883910 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fc07c143810 0x7fc07c153aa0 secure :-1 s=READY pgs=582 cs=0 l=1 rev1=1 crypto rx=0x7fc06c00c3c0 tx=0x7fc06c00c880 comp rx=0 tx=0).stop 2023-11-26T07:19:37.233 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:37.227+0000 7fc08292b640 1 -- 172.21.15.143:0/2066883910 shutdown_connections 2023-11-26T07:19:37.234 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:37.227+0000 7fc08292b640 1 --2- 172.21.15.143:0/2066883910 >> [v2:172.21.15.143:6832/2150557544,v1:172.21.15.143:6833/2150557544] conn(0x7fc060066620 0x7fc060068ae0 unknown :-1 s=CLOSED pgs=292 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:19:37.234 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:37.227+0000 7fc08292b640 1 --2- 172.21.15.143:0/2066883910 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fc07c15e0f0 0x7fc07c0b2600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:19:37.234 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:37.227+0000 7fc08292b640 1 --2- 172.21.15.143:0/2066883910 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fc07c143810 0x7fc07c153aa0 unknown :-1 s=CLOSED pgs=582 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:19:37.234 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:37.227+0000 7fc08292b640 1 --2- 172.21.15.143:0/2066883910 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fc07c0b1e10 0x7fc07c166250 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:19:37.234 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:37.227+0000 7fc08292b640 1 -- 172.21.15.143:0/2066883910 >> 172.21.15.143:0/2066883910 conn(0x7fc07c06a890 msgr2=0x7fc07c06ac70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:19:37.234 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:37.227+0000 7fc08292b640 1 -- 172.21.15.143:0/2066883910 shutdown_connections 2023-11-26T07:19:37.234 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:37.227+0000 7fc08292b640 1 -- 172.21.15.143:0/2066883910 wait complete. 2023-11-26T07:19:37.250 INFO:teuthology.orchestra.run.smithi143.stdout:{"pg_ready":true,"pg_map":{"version":95,"stamp":"2023-11-26T07:19:35.712433+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":24,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":221240,"kb_used_data":5072,"kb_used_omap":12,"kb_used_meta":216051,"kb_avail":754753480,"statfs":{"total":773094113280,"available":772867563520,"internally_reserved":0,"allocated":5193728,"data_stored":2784394,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12749,"internal_metadata":221236787},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":32,"apply_latency_ms":32,"commit_latency_ns":32000000,"apply_latency_ns":32000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"5.598663"},"pg_stats":[{"pgid":"7.3f","version":"0'0","reported_seq":1,"reported_epoch":43,"state":"creating+peering","last_fresh":"2023-11-26T07:19:31.135164+0000","last_change":"2023-11-26T07:19:31.135164+0000","last_active":"2023-11-26T07:19:31.113498+0000","last_peered":"2023-11-26T07:19:31.113498+0000","last_clean":"2023-11-26T07:19:31.113498+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-26T07:19:31.135164+0000","last_undegraded":"2023-11-26T07:19:31.135164+0000","last_fullsized":"2023-11-26T07:19:31.135164+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:19:31.113498+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:19:31.113498+0000","last_clean_scrub_stamp":"2023-11-26T07:19:31.113498+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.3e","version":"0'0","reported_seq":1,"reported_epoch":43,"state":"creating+peering","last_fresh":"2023-11-26T07:19:31.129025+0000","last_change":"2023-11-26T07:19:31.129025+0000","last_active":"2023-11-26T07:19:31.113498+0000","last_peered":"2023-11-26T07:19:31.113498+0000","last_clean":"2023-11-26T07:19:31.113498+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-26T07:19:31.129025+0000","last_undegraded":"2023-11-26T07:19:31.129025+0000","last_fullsized":"2023-11-26T07:19:31.129025+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:19:31.113498+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:19:31.113498+0000","last_clean_scrub_stamp":"2023-11-26T07:19:31.113498+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.3d","version":"0'0","reported_seq":1,"reported_epoch":43,"state":"creating+peering","last_fresh":"2023-11-26T07:19:31.125533+0000","last_change":"2023-11-26T07:19:31.125533+0000","last_active":"2023-11-26T07:19:31.113498+0000","last_peered":"2023-11-26T07:19:31.113498+0000","last_clean":"2023-11-26T07:19:31.113498+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-26T07:19:31.125533+0000","last_undegraded":"2023-11-26T07:19:31.125533+0000","last_fullsized":"2023-11-26T07:19:31.125533+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:19:31.113498+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:19:31.113498+0000","last_clean_scrub_stamp":"2023-11-26T07:19:31.113498+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.3c","version":"0'0","reported_seq":1,"reported_epoch":43,"state":"creating+peering","last_fresh":"2023-11-26T07:19:31.129825+0000","last_change":"2023-11-26T07:19:31.129825+0000","last_active":"2023-11-26T07:19:31.113498+0000","last_peered":"2023-11-26T07:19:31.113498+0000","last_clean":"2023-11-26T07:19:31.113498+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-26T07:19:31.129825+0000","last_undegraded":"2023-11-26T07:19:31.129825+0000","last_fullsized":"2023-11-26T07:19:31.129825+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:19:31.113498+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:19:31.113498+0000","last_clean_scrub_stamp":"2023-11-26T07:19:31.113498+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.3b","version":"0'0","reported_seq":1,"reported_epoch":43,"state":"creating+peering","last_fresh":"2023-11-26T07:19:31.128964+0000","last_change":"2023-11-26T07:19:31.128964+0000","last_active":"2023-11-26T07:19:31.113498+0000","last_peered":"2023-11-26T07:19:31.113498+0000","last_clean":"2023-11-26T07:19:31.113498+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-26T07:19:31.128964+0000","last_undegraded":"2023-11-26T07:19:31.128964+0000","last_fullsized":"2023-11-26T07:19:31.128964+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:19:31.113498+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:19:31.113498+0000","last_clean_scrub_stamp":"2023-11-26T07:19:31.113498+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.3a","version":"0'0","reported_seq":1,"reported_epoch":43,"state":"creating+peering","last_fresh":"2023-11-26T07:19:31.138400+0000","last_change":"2023-11-26T07:19:31.138400+0000","last_active":"2023-11-26T07:19:31.113498+0000","last_peered":"2023-11-26T07:19:31.113498+0000","last_clean":"2023-11-26T07:19:31.113498+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-26T07:19:31.138400+0000","last_undegraded":"2023-11-26T07:19:31.138400+0000","last_fullsized":"2023-11-26T07:19:31.138400+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:19:31.113498+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:19:31.113498+0000","last_clean_scrub_stamp":"2023-11-26T07:19:31.113498+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.39","version":"0'0","reported_seq":1,"reported_epoch":43,"state":"creating+peering","last_fresh":"2023-11-26T07:19:31.134099+0000","last_change":"2023-11-26T07:19:31.134099+0000","last_active":"2023-11-26T07:19:31.113498+0000","last_peered":"2023-11-26T07:19:31.113498+0000","last_clean":"2023-11-26T07:19:31.113498+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-26T07:19:31.134099+0000","last_undegraded":"2023-11-26T07:19:31.134099+0000","last_fullsized":"2023-11-26T07:19:31.134099+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:19:31.113498+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:19:31.113498+0000","last_clean_scrub_stamp":"2023-11-26T07:19:31.113498+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.38","version":"0'0","reported_seq":1,"reported_epoch":43,"state":"creating+peering","last_fresh":"2023-11-26T07:19:31.130100+0000","last_change":"2023-11-26T07:19:31.130100+0000","last_active":"2023-11-26T07:19:31.113498+0000","last_peered":"2023-11-26T07:19:31.113498+0000","last_clean":"2023-11-26T07:19:31.113498+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-26T07:19:31.130100+0000","last_undegraded":"2023-11-26T07:19:31.130100+0000","last_fullsized":"2023-11-26T07:19:31.130100+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:19:31.113498+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:19:31.113498+0000","last_clean_scrub_stamp":"2023-11-26T07:19:31.113498+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.37","version":"0'0","reported_seq":1,"reported_epoch":43,"state":"creating+peering","last_fresh":"2023-11-26T07:19:31.129317+0000","last_change":"2023-11-26T07:19:31.129317+0000","last_active":"2023-11-26T07:19:31.113498+0000","last_peered":"2023-11-26T07:19:31.113498+0000","last_clean":"2023-11-26T07:19:31.113498+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-26T07:19:31.129317+0000","last_undegraded":"2023-11-26T07:19:31.129317+0000","last_fullsized":"2023-11-26T07:19:31.129317+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:19:31.113498+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:19:31.113498+0000","last_clean_scrub_stamp":"2023-11-26T07:19:31.113498+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.36","version":"0'0","reported_seq":1,"reported_epoch":43,"state":"creating+peering","last_fresh":"2023-11-26T07:19:31.130104+0000","last_change":"2023-11-26T07:19:31.130104+0000","last_active":"2023-11-26T07:19:31.113498+0000","last_peered":"2023-11-26T07:19:31.113498+0000","last_clean":"2023-11-26T07:19:31.113498+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-26T07:19:31.130104+0000","last_undegraded":"2023-11-26T07:19:31.130104+0000","last_fullsized":"2023-11-26T07:19:31.130104+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:19:31.113498+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:19:31.113498+0000","last_clean_scrub_stamp":"2023-11-26T07:19:31.113498+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.35","version":"0'0","reported_seq":1,"reported_epoch":43,"state":"creating+peering","last_fresh":"2023-11-26T07:19:31.130252+0000","last_change":"2023-11-26T07:19:31.130252+0000","last_active":"2023-11-26T07:19:31.113498+0000","last_peered":"2023-11-26T07:19:31.113498+0000","last_clean":"2023-11-26T07:19:31.113498+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-26T07:19:31.130252+0000","last_undegraded":"2023-11-26T07:19:31.130252+0000","last_fullsized":"2023-11-26T07:19:31.130252+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:19:31.113498+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:19:31.113498+0000","last_clean_scrub_stamp":"2023-11-26T07:19:31.113498+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.34","version":"0'0","reported_seq":1,"reported_epoch":43,"state":"creating+peering","last_fresh":"2023-11-26T07:19:31.131089+0000","last_change":"2023-11-26T07:19:31.131089+0000","last_active":"2023-11-26T07:19:31.113498+0000","last_peered":"2023-11-26T07:19:31.113498+0000","last_clean":"2023-11-26T07:19:31.113498+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-26T07:19:31.131089+0000","last_undegraded":"2023-11-26T07:19:31.131089+0000","last_fullsized":"2023-11-26T07:19:31.131089+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:19:31.113498+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:19:31.113498+0000","last_clean_scrub_stamp":"2023-11-26T07:19:31.113498+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.33","version":"0'0","reported_seq":1,"reported_epoch":43,"state":"creating+peering","last_fresh":"2023-11-26T07:19:31.132990+0000","last_change":"2023-11-26T07:19:31.132990+0000","last_active":"2023-11-26T07:19:31.113498+0000","last_peered":"2023-11-26T07:19:31.113498+0000","last_clean":"2023-11-26T07:19:31.113498+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-26T07:19:31.132990+0000","last_undegraded":"2023-11-26T07:19:31.132990+0000","last_fullsized":"2023-11-26T07:19:31.132990+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:19:31.113498+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:19:31.113498+0000","last_clean_scrub_stamp":"2023-11-26T07:19:31.113498+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.32","version":"0'0","reported_seq":1,"reported_epoch":43,"state":"creating+peering","last_fresh":"2023-11-26T07:19:31.129881+0000","last_change":"2023-11-26T07:19:31.129881+0000","last_active":"2023-11-26T07:19:31.113498+0000","last_peered":"2023-11-26T07:19:31.113498+0000","last_clean":"2023-11-26T07:19:31.113498+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-26T07:19:31.129881+0000","last_undegraded":"2023-11-26T07:19:31.129881+0000","last_fullsized":"2023-11-26T07:19:31.129881+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:19:31.113498+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:19:31.113498+0000","last_clean_scrub_stamp":"2023-11-26T07:19:31.113498+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.31","version":"0'0","reported_seq":1,"reported_epoch":43,"state":"creating+peering","last_fresh":"2023-11-26T07:19:31.138475+0000","last_change":"2023-11-26T07:19:31.138475+0000","last_active":"2023-11-26T07:19:31.113498+0000","last_peered":"2023-11-26T07:19:31.113498+0000","last_clean":"2023-11-26T07:19:31.113498+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-26T07:19:31.138475+0000","last_undegraded":"2023-11-26T07:19:31.138475+0000","last_fullsized":"2023-11-26T07:19:31.138475+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:19:31.113498+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:19:31.113498+0000","last_clean_scrub_stamp":"2023-11-26T07:19:31.113498+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.30","version":"0'0","reported_seq":1,"reported_epoch":43,"state":"creating+peering","last_fresh":"2023-11-26T07:19:31.131093+0000","last_change":"2023-11-26T07:19:31.131093+0000","last_active":"2023-11-26T07:19:31.113498+0000","last_peered":"2023-11-26T07:19:31.113498+0000","last_clean":"2023-11-26T07:19:31.113498+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-26T07:19:31.131093+0000","last_undegraded":"2023-11-26T07:19:31.131093+0000","last_fullsized":"2023-11-26T07:19:31.131093+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:19:31.113498+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:19:31.113498+0000","last_clean_scrub_stamp":"2023-11-26T07:19:31.113498+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.2f","version":"0'0","reported_seq":1,"reported_epoch":43,"state":"creating+peering","last_fresh":"2023-11-26T07:19:31.125773+0000","last_change":"2023-11-26T07:19:31.125773+0000","last_active":"2023-11-26T07:19:31.113498+0000","last_peered":"2023-11-26T07:19:31.113498+0000","last_clean":"2023-11-26T07:19:31.113498+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-26T07:19:31.125773+0000","last_undegraded":"2023-11-26T07:19:31.125773+0000","last_fullsized":"2023-11-26T07:19:31.125773+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:19:31.113498+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:19:31.113498+0000","last_clean_scrub_stamp":"2023-11-26T07:19:31.113498+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.2e","version":"0'0","reported_seq":1,"reported_epoch":43,"state":"creating+peering","last_fresh":"2023-11-26T07:19:31.126790+0000","last_change":"2023-11-26T07:19:31.126790+0000","last_active":"2023-11-26T07:19:31.113498+0000","last_peered":"2023-11-26T07:19:31.113498+0000","last_clean":"2023-11-26T07:19:31.113498+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-26T07:19:31.126790+0000","last_undegraded":"2023-11-26T07:19:31.126790+0000","last_fullsized":"2023-11-26T07:19:31.126790+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:19:31.113498+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:19:31.113498+0000","last_clean_scrub_stamp":"2023-11-26T07:19:31.113498+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.2d","version":"0'0","reported_seq":1,"reported_epoch":43,"state":"creating+peering","last_fresh":"2023-11-26T07:19:31.133954+0000","last_change":"2023-11-26T07:19:31.133954+0000","last_active":"2023-11-26T07:19:31.113498+0000","last_peered":"2023-11-26T07:19:31.113498+0000","last_clean":"2023-11-26T07:19:31.113498+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-26T07:19:31.133954+0000","last_undegraded":"2023-11-26T07:19:31.133954+0000","last_fullsized":"2023-11-26T07:19:31.133954+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:19:31.113498+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:19:31.113498+0000","last_clean_scrub_stamp":"2023-11-26T07:19:31.113498+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.2c","version":"0'0","reported_seq":1,"reported_epoch":43,"state":"creating+peering","last_fresh":"2023-11-26T07:19:31.135122+0000","last_change":"2023-11-26T07:19:31.135122+0000","last_active":"2023-11-26T07:19:31.113498+0000","last_peered":"2023-11-26T07:19:31.113498+0000","last_clean":"2023-11-26T07:19:31.113498+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-26T07:19:31.135122+0000","last_undegraded":"2023-11-26T07:19:31.135122+0000","last_fullsized":"2023-11-26T07:19:31.135122+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:19:31.113498+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:19:31.113498+0000","last_clean_scrub_stamp":"2023-11-26T07:19:31.113498+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.2b","version":"0'0","reported_seq":1,"reported_epoch":43,"state":"creating+peering","last_fresh":"2023-11-26T07:19:31.137790+0000","last_change":"2023-11-26T07:19:31.137790+0000","last_active":"2023-11-26T07:19:31.113498+0000","last_peered":"2023-11-26T07:19:31.113498+0000","last_clean":"2023-11-26T07:19:31.113498+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-26T07:19:31.137790+0000","last_undegraded":"2023-11-26T07:19:31.137790+0000","last_fullsized":"2023-11-26T07:19:31.137790+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:19:31.113498+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:19:31.113498+0000","last_clean_scrub_stamp":"2023-11-26T07:19:31.113498+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.2a","version":"0'0","reported_seq":1,"reported_epoch":43,"state":"creating+peering","last_fresh":"2023-11-26T07:19:31.129388+0000","last_change":"2023-11-26T07:19:31.129388+0000","last_active":"2023-11-26T07:19:31.113498+0000","last_peered":"2023-11-26T07:19:31.113498+0000","last_clean":"2023-11-26T07:19:31.113498+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-26T07:19:31.129388+0000","last_undegraded":"2023-11-26T07:19:31.129388+0000","last_fullsized":"2023-11-26T07:19:31.129388+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:19:31.113498+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:19:31.113498+0000","last_clean_scrub_stamp":"2023-11-26T07:19:31.113498+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.29","version":"0'0","reported_seq":1,"reported_epoch":43,"state":"creating+peering","last_fresh":"2023-11-26T07:19:31.131006+0000","last_change":"2023-11-26T07:19:31.131006+0000","last_active":"2023-11-26T07:19:31.113498+0000","last_peered":"2023-11-26T07:19:31.113498+0000","last_clean":"2023-11-26T07:19:31.113498+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-26T07:19:31.131006+0000","last_undegraded":"2023-11-26T07:19:31.131006+0000","last_fullsized":"2023-11-26T07:19:31.131006+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:19:31.113498+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:19:31.113498+0000","last_clean_scrub_stamp":"2023-11-26T07:19:31.113498+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.28","version":"0'0","reported_seq":1,"reported_epoch":43,"state":"creating+peering","last_fresh":"2023-11-26T07:19:31.130446+0000","last_change":"2023-11-26T07:19:31.130446+0000","last_active":"2023-11-26T07:19:31.113498+0000","last_peered":"2023-11-26T07:19:31.113498+0000","last_clean":"2023-11-26T07:19:31.113498+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-26T07:19:31.130446+0000","last_undegraded":"2023-11-26T07:19:31.130446+0000","last_fullsized":"2023-11-26T07:19:31.130446+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:19:31.113498+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:19:31.113498+0000","last_clean_scrub_stamp":"2023-11-26T07:19:31.113498+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.27","version":"0'0","reported_seq":1,"reported_epoch":43,"state":"creating+peering","last_fresh":"2023-11-26T07:19:31.134474+0000","last_change":"2023-11-26T07:19:31.134474+0000","last_active":"2023-11-26T07:19:31.113498+0000","last_peered":"2023-11-26T07:19:31.113498+0000","last_clean":"2023-11-26T07:19:31.113498+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-26T07:19:31.134474+0000","last_undegraded":"2023-11-26T07:19:31.134474+0000","last_fullsized":"2023-11-26T07:19:31.134474+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:19:31.113498+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:19:31.113498+0000","last_clean_scrub_stamp":"2023-11-26T07:19:31.113498+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.26","version":"0'0","reported_seq":1,"reported_epoch":43,"state":"creating+peering","last_fresh":"2023-11-26T07:19:31.132954+0000","last_change":"2023-11-26T07:19:31.132954+0000","last_active":"2023-11-26T07:19:31.113498+0000","last_peered":"2023-11-26T07:19:31.113498+0000","last_clean":"2023-11-26T07:19:31.113498+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-26T07:19:31.132954+0000","last_undegraded":"2023-11-26T07:19:31.132954+0000","last_fullsized":"2023-11-26T07:19:31.132954+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:19:31.113498+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:19:31.113498+0000","last_clean_scrub_stamp":"2023-11-26T07:19:31.113498+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.25","version":"0'0","reported_seq":1,"reported_epoch":43,"state":"creating+peering","last_fresh":"2023-11-26T07:19:31.124787+0000","last_change":"2023-11-26T07:19:31.124787+0000","last_active":"2023-11-26T07:19:31.113498+0000","last_peered":"2023-11-26T07:19:31.113498+0000","last_clean":"2023-11-26T07:19:31.113498+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-26T07:19:31.124787+0000","last_undegraded":"2023-11-26T07:19:31.124787+0000","last_fullsized":"2023-11-26T07:19:31.124787+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:19:31.113498+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:19:31.113498+0000","last_clean_scrub_stamp":"2023-11-26T07:19:31.113498+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.24","version":"0'0","reported_seq":1,"reported_epoch":43,"state":"creating+peering","last_fresh":"2023-11-26T07:19:31.130640+0000","last_change":"2023-11-26T07:19:31.130640+0000","last_active":"2023-11-26T07:19:31.113498+0000","last_peered":"2023-11-26T07:19:31.113498+0000","last_clean":"2023-11-26T07:19:31.113498+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-26T07:19:31.130640+0000","last_undegraded":"2023-11-26T07:19:31.130640+0000","last_fullsized":"2023-11-26T07:19:31.130640+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:19:31.113498+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:19:31.113498+0000","last_clean_scrub_stamp":"2023-11-26T07:19:31.113498+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.23","version":"0'0","reported_seq":1,"reported_epoch":43,"state":"creating+peering","last_fresh":"2023-11-26T07:19:31.131030+0000","last_change":"2023-11-26T07:19:31.131030+0000","last_active":"2023-11-26T07:19:31.113498+0000","last_peered":"2023-11-26T07:19:31.113498+0000","last_clean":"2023-11-26T07:19:31.113498+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-26T07:19:31.131030+0000","last_undegraded":"2023-11-26T07:19:31.131030+0000","last_fullsized":"2023-11-26T07:19:31.131030+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:19:31.113498+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:19:31.113498+0000","last_clean_scrub_stamp":"2023-11-26T07:19:31.113498+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.22","version":"0'0","reported_seq":1,"reported_epoch":43,"state":"creating+peering","last_fresh":"2023-11-26T07:19:31.126109+0000","last_change":"2023-11-26T07:19:31.126109+0000","last_active":"2023-11-26T07:19:31.113498+0000","last_peered":"2023-11-26T07:19:31.113498+0000","last_clean":"2023-11-26T07:19:31.113498+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-26T07:19:31.126109+0000","last_undegraded":"2023-11-26T07:19:31.126109+0000","last_fullsized":"2023-11-26T07:19:31.126109+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:19:31.113498+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:19:31.113498+0000","last_clean_scrub_stamp":"2023-11-26T07:19:31.113498+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.21","version":"0'0","reported_seq":1,"reported_epoch":43,"state":"creating+peering","last_fresh":"2023-11-26T07:19:31.134817+0000","last_change":"2023-11-26T07:19:31.134817+0000","last_active":"2023-11-26T07:19:31.113498+0000","last_peered":"2023-11-26T07:19:31.113498+0000","last_clean":"2023-11-26T07:19:31.113498+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-26T07:19:31.134817+0000","last_undegraded":"2023-11-26T07:19:31.134817+0000","last_fullsized":"2023-11-26T07:19:31.134817+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:19:31.113498+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:19:31.113498+0000","last_clean_scrub_stamp":"2023-11-26T07:19:31.113498+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.20","version":"0'0","reported_seq":1,"reported_epoch":43,"state":"creating+peering","last_fresh":"2023-11-26T07:19:31.127285+0000","last_change":"2023-11-26T07:19:31.127285+0000","last_active":"2023-11-26T07:19:31.113498+0000","last_peered":"2023-11-26T07:19:31.113498+0000","last_clean":"2023-11-26T07:19:31.113498+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-26T07:19:31.127285+0000","last_undegraded":"2023-11-26T07:19:31.127285+0000","last_fullsized":"2023-11-26T07:19:31.127285+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:19:31.113498+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:19:31.113498+0000","last_clean_scrub_stamp":"2023-11-26T07:19:31.113498+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.1e","version":"0'0","reported_seq":1,"reported_epoch":43,"state":"creating+peering","last_fresh":"2023-11-26T07:19:31.133294+0000","last_change":"2023-11-26T07:19:31.133294+0000","last_active":"2023-11-26T07:19:31.113498+0000","last_peered":"2023-11-26T07:19:31.113498+0000","last_clean":"2023-11-26T07:19:31.113498+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-26T07:19:31.133294+0000","last_undegraded":"2023-11-26T07:19:31.133294+0000","last_fullsized":"2023-11-26T07:19:31.133294+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:19:31.113498+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:19:31.113498+0000","last_clean_scrub_stamp":"2023-11-26T07:19:31.113498+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.1f","version":"0'0","reported_seq":9,"reported_epoch":43,"state":"active+clean","last_fresh":"2023-11-26T07:19:31.153093+0000","last_change":"2023-11-26T07:19:31.148359+0000","last_active":"2023-11-26T07:19:31.153093+0000","last_peered":"2023-11-26T07:19:31.153093+0000","last_clean":"2023-11-26T07:19:31.153093+0000","last_became_active":"2023-11-26T07:19:31.148066+0000","last_became_peered":"2023-11-26T07:19:31.148066+0000","last_unstale":"2023-11-26T07:19:31.153093+0000","last_undegraded":"2023-11-26T07:19:31.153093+0000","last_fullsized":"2023-11-26T07:19:31.153093+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:19:30.106639+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:19:30.106639+0000","last_clean_scrub_stamp":"2023-11-26T07:19:30.106639+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T15:38:28.537385+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.1f","version":"0'0","reported_seq":1,"reported_epoch":43,"state":"creating+peering","last_fresh":"2023-11-26T07:19:31.134010+0000","last_change":"2023-11-26T07:19:31.134010+0000","last_active":"2023-11-26T07:19:31.113498+0000","last_peered":"2023-11-26T07:19:31.113498+0000","last_clean":"2023-11-26T07:19:31.113498+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-26T07:19:31.134010+0000","last_undegraded":"2023-11-26T07:19:31.134010+0000","last_fullsized":"2023-11-26T07:19:31.134010+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:19:31.113498+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:19:31.113498+0000","last_clean_scrub_stamp":"2023-11-26T07:19:31.113498+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.1e","version":"0'0","reported_seq":9,"reported_epoch":43,"state":"active+clean","last_fresh":"2023-11-26T07:19:31.149264+0000","last_change":"2023-11-26T07:19:31.144305+0000","last_active":"2023-11-26T07:19:31.149264+0000","last_peered":"2023-11-26T07:19:31.149264+0000","last_clean":"2023-11-26T07:19:31.149264+0000","last_became_active":"2023-11-26T07:19:31.143063+0000","last_became_peered":"2023-11-26T07:19:31.143063+0000","last_unstale":"2023-11-26T07:19:31.149264+0000","last_undegraded":"2023-11-26T07:19:31.149264+0000","last_fullsized":"2023-11-26T07:19:31.149264+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:19:30.106639+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:19:30.106639+0000","last_clean_scrub_stamp":"2023-11-26T07:19:30.106639+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T08:28:59.053370+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.1c","version":"0'0","reported_seq":1,"reported_epoch":43,"state":"creating+peering","last_fresh":"2023-11-26T07:19:31.129959+0000","last_change":"2023-11-26T07:19:31.129959+0000","last_active":"2023-11-26T07:19:31.113498+0000","last_peered":"2023-11-26T07:19:31.113498+0000","last_clean":"2023-11-26T07:19:31.113498+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-26T07:19:31.129959+0000","last_undegraded":"2023-11-26T07:19:31.129959+0000","last_fullsized":"2023-11-26T07:19:31.129959+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:19:31.113498+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:19:31.113498+0000","last_clean_scrub_stamp":"2023-11-26T07:19:31.113498+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.1d","version":"0'0","reported_seq":9,"reported_epoch":43,"state":"active+clean","last_fresh":"2023-11-26T07:19:31.153679+0000","last_change":"2023-11-26T07:19:31.150313+0000","last_active":"2023-11-26T07:19:31.153679+0000","last_peered":"2023-11-26T07:19:31.153679+0000","last_clean":"2023-11-26T07:19:31.153679+0000","last_became_active":"2023-11-26T07:19:31.150042+0000","last_became_peered":"2023-11-26T07:19:31.150042+0000","last_unstale":"2023-11-26T07:19:31.153679+0000","last_undegraded":"2023-11-26T07:19:31.153679+0000","last_fullsized":"2023-11-26T07:19:31.153679+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:19:30.106639+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:19:30.106639+0000","last_clean_scrub_stamp":"2023-11-26T07:19:30.106639+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T16:20:23.315619+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.1d","version":"0'0","reported_seq":1,"reported_epoch":43,"state":"creating+peering","last_fresh":"2023-11-26T07:19:31.134552+0000","last_change":"2023-11-26T07:19:31.134552+0000","last_active":"2023-11-26T07:19:31.113498+0000","last_peered":"2023-11-26T07:19:31.113498+0000","last_clean":"2023-11-26T07:19:31.113498+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-26T07:19:31.134552+0000","last_undegraded":"2023-11-26T07:19:31.134552+0000","last_fullsized":"2023-11-26T07:19:31.134552+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:19:31.113498+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:19:31.113498+0000","last_clean_scrub_stamp":"2023-11-26T07:19:31.113498+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.1c","version":"0'0","reported_seq":9,"reported_epoch":43,"state":"active+clean","last_fresh":"2023-11-26T07:19:31.147334+0000","last_change":"2023-11-26T07:19:31.141667+0000","last_active":"2023-11-26T07:19:31.147334+0000","last_peered":"2023-11-26T07:19:31.147334+0000","last_clean":"2023-11-26T07:19:31.147334+0000","last_became_active":"2023-11-26T07:19:31.141393+0000","last_became_peered":"2023-11-26T07:19:31.141393+0000","last_unstale":"2023-11-26T07:19:31.147334+0000","last_undegraded":"2023-11-26T07:19:31.147334+0000","last_fullsized":"2023-11-26T07:19:31.147334+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:19:30.106639+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:19:30.106639+0000","last_clean_scrub_stamp":"2023-11-26T07:19:30.106639+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T10:27:01.949013+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.1a","version":"0'0","reported_seq":1,"reported_epoch":43,"state":"creating+peering","last_fresh":"2023-11-26T07:19:31.125537+0000","last_change":"2023-11-26T07:19:31.125537+0000","last_active":"2023-11-26T07:19:31.113498+0000","last_peered":"2023-11-26T07:19:31.113498+0000","last_clean":"2023-11-26T07:19:31.113498+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-26T07:19:31.125537+0000","last_undegraded":"2023-11-26T07:19:31.125537+0000","last_fullsized":"2023-11-26T07:19:31.125537+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:19:31.113498+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:19:31.113498+0000","last_clean_scrub_stamp":"2023-11-26T07:19:31.113498+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.1b","version":"0'0","reported_seq":9,"reported_epoch":43,"state":"active+clean","last_fresh":"2023-11-26T07:19:31.153191+0000","last_change":"2023-11-26T07:19:31.149354+0000","last_active":"2023-11-26T07:19:31.153191+0000","last_peered":"2023-11-26T07:19:31.153191+0000","last_clean":"2023-11-26T07:19:31.153191+0000","last_became_active":"2023-11-26T07:19:31.149092+0000","last_became_peered":"2023-11-26T07:19:31.149092+0000","last_unstale":"2023-11-26T07:19:31.153191+0000","last_undegraded":"2023-11-26T07:19:31.153191+0000","last_fullsized":"2023-11-26T07:19:31.153191+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:19:30.106639+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:19:30.106639+0000","last_clean_scrub_stamp":"2023-11-26T07:19:30.106639+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T13:58:09.825490+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.1b","version":"0'0","reported_seq":1,"reported_epoch":43,"state":"creating+peering","last_fresh":"2023-11-26T07:19:31.131194+0000","last_change":"2023-11-26T07:19:31.131194+0000","last_active":"2023-11-26T07:19:31.113498+0000","last_peered":"2023-11-26T07:19:31.113498+0000","last_clean":"2023-11-26T07:19:31.113498+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-26T07:19:31.131194+0000","last_undegraded":"2023-11-26T07:19:31.131194+0000","last_fullsized":"2023-11-26T07:19:31.131194+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:19:31.113498+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:19:31.113498+0000","last_clean_scrub_stamp":"2023-11-26T07:19:31.113498+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.1a","version":"0'0","reported_seq":9,"reported_epoch":43,"state":"active+clean","last_fresh":"2023-11-26T07:19:31.142739+0000","last_change":"2023-11-26T07:19:31.139455+0000","last_active":"2023-11-26T07:19:31.142739+0000","last_peered":"2023-11-26T07:19:31.142739+0000","last_clean":"2023-11-26T07:19:31.142739+0000","last_became_active":"2023-11-26T07:19:31.139009+0000","last_became_peered":"2023-11-26T07:19:31.139009+0000","last_unstale":"2023-11-26T07:19:31.142739+0000","last_undegraded":"2023-11-26T07:19:31.142739+0000","last_fullsized":"2023-11-26T07:19:31.142739+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:19:30.106639+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:19:30.106639+0000","last_clean_scrub_stamp":"2023-11-26T07:19:30.106639+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T13:51:18.697846+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.18","version":"0'0","reported_seq":1,"reported_epoch":43,"state":"creating+peering","last_fresh":"2023-11-26T07:19:31.128550+0000","last_change":"2023-11-26T07:19:31.128550+0000","last_active":"2023-11-26T07:19:31.113498+0000","last_peered":"2023-11-26T07:19:31.113498+0000","last_clean":"2023-11-26T07:19:31.113498+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-26T07:19:31.128550+0000","last_undegraded":"2023-11-26T07:19:31.128550+0000","last_fullsized":"2023-11-26T07:19:31.128550+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:19:31.113498+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:19:31.113498+0000","last_clean_scrub_stamp":"2023-11-26T07:19:31.113498+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.19","version":"0'0","reported_seq":9,"reported_epoch":43,"state":"active+clean","last_fresh":"2023-11-26T07:19:31.150741+0000","last_change":"2023-11-26T07:19:31.146327+0000","last_active":"2023-11-26T07:19:31.150741+0000","last_peered":"2023-11-26T07:19:31.150741+0000","last_clean":"2023-11-26T07:19:31.150741+0000","last_became_active":"2023-11-26T07:19:31.145985+0000","last_became_peered":"2023-11-26T07:19:31.145985+0000","last_unstale":"2023-11-26T07:19:31.150741+0000","last_undegraded":"2023-11-26T07:19:31.150741+0000","last_fullsized":"2023-11-26T07:19:31.150741+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:19:30.106639+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:19:30.106639+0000","last_clean_scrub_stamp":"2023-11-26T07:19:30.106639+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T12:39:06.522107+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.19","version":"0'0","reported_seq":1,"reported_epoch":43,"state":"creating+peering","last_fresh":"2023-11-26T07:19:31.131606+0000","last_change":"2023-11-26T07:19:31.131606+0000","last_active":"2023-11-26T07:19:31.113498+0000","last_peered":"2023-11-26T07:19:31.113498+0000","last_clean":"2023-11-26T07:19:31.113498+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-26T07:19:31.131606+0000","last_undegraded":"2023-11-26T07:19:31.131606+0000","last_fullsized":"2023-11-26T07:19:31.131606+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:19:31.113498+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:19:31.113498+0000","last_clean_scrub_stamp":"2023-11-26T07:19:31.113498+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.18","version":"0'0","reported_seq":9,"reported_epoch":43,"state":"active+clean","last_fresh":"2023-11-26T07:19:31.147314+0000","last_change":"2023-11-26T07:19:31.140258+0000","last_active":"2023-11-26T07:19:31.147314+0000","last_peered":"2023-11-26T07:19:31.147314+0000","last_clean":"2023-11-26T07:19:31.147314+0000","last_became_active":"2023-11-26T07:19:31.140009+0000","last_became_peered":"2023-11-26T07:19:31.140009+0000","last_unstale":"2023-11-26T07:19:31.147314+0000","last_undegraded":"2023-11-26T07:19:31.147314+0000","last_fullsized":"2023-11-26T07:19:31.147314+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:19:30.106639+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:19:30.106639+0000","last_clean_scrub_stamp":"2023-11-26T07:19:30.106639+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T13:51:18.697846+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.16","version":"0'0","reported_seq":1,"reported_epoch":43,"state":"creating+peering","last_fresh":"2023-11-26T07:19:31.126247+0000","last_change":"2023-11-26T07:19:31.126247+0000","last_active":"2023-11-26T07:19:31.113498+0000","last_peered":"2023-11-26T07:19:31.113498+0000","last_clean":"2023-11-26T07:19:31.113498+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-26T07:19:31.126247+0000","last_undegraded":"2023-11-26T07:19:31.126247+0000","last_fullsized":"2023-11-26T07:19:31.126247+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:19:31.113498+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:19:31.113498+0000","last_clean_scrub_stamp":"2023-11-26T07:19:31.113498+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.17","version":"0'0","reported_seq":9,"reported_epoch":43,"state":"active+clean","last_fresh":"2023-11-26T07:19:31.141806+0000","last_change":"2023-11-26T07:19:31.134552+0000","last_active":"2023-11-26T07:19:31.141806+0000","last_peered":"2023-11-26T07:19:31.141806+0000","last_clean":"2023-11-26T07:19:31.141806+0000","last_became_active":"2023-11-26T07:19:31.134241+0000","last_became_peered":"2023-11-26T07:19:31.134241+0000","last_unstale":"2023-11-26T07:19:31.141806+0000","last_undegraded":"2023-11-26T07:19:31.141806+0000","last_fullsized":"2023-11-26T07:19:31.141806+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:19:30.106639+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:19:30.106639+0000","last_clean_scrub_stamp":"2023-11-26T07:19:30.106639+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T07:36:26.843027+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.17","version":"0'0","reported_seq":1,"reported_epoch":43,"state":"creating+peering","last_fresh":"2023-11-26T07:19:31.129608+0000","last_change":"2023-11-26T07:19:31.129608+0000","last_active":"2023-11-26T07:19:31.113498+0000","last_peered":"2023-11-26T07:19:31.113498+0000","last_clean":"2023-11-26T07:19:31.113498+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-26T07:19:31.129608+0000","last_undegraded":"2023-11-26T07:19:31.129608+0000","last_fullsized":"2023-11-26T07:19:31.129608+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:19:31.113498+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:19:31.113498+0000","last_clean_scrub_stamp":"2023-11-26T07:19:31.113498+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.16","version":"0'0","reported_seq":9,"reported_epoch":43,"state":"active+clean","last_fresh":"2023-11-26T07:19:31.151072+0000","last_change":"2023-11-26T07:19:31.144647+0000","last_active":"2023-11-26T07:19:31.151072+0000","last_peered":"2023-11-26T07:19:31.151072+0000","last_clean":"2023-11-26T07:19:31.151072+0000","last_became_active":"2023-11-26T07:19:31.143606+0000","last_became_peered":"2023-11-26T07:19:31.143606+0000","last_unstale":"2023-11-26T07:19:31.151072+0000","last_undegraded":"2023-11-26T07:19:31.151072+0000","last_fullsized":"2023-11-26T07:19:31.151072+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:19:30.106639+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:19:30.106639+0000","last_clean_scrub_stamp":"2023-11-26T07:19:30.106639+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T15:42:44.993287+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.14","version":"0'0","reported_seq":1,"reported_epoch":43,"state":"creating+peering","last_fresh":"2023-11-26T07:19:31.133561+0000","last_change":"2023-11-26T07:19:31.133561+0000","last_active":"2023-11-26T07:19:31.113498+0000","last_peered":"2023-11-26T07:19:31.113498+0000","last_clean":"2023-11-26T07:19:31.113498+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-26T07:19:31.133561+0000","last_undegraded":"2023-11-26T07:19:31.133561+0000","last_fullsized":"2023-11-26T07:19:31.133561+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:19:31.113498+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:19:31.113498+0000","last_clean_scrub_stamp":"2023-11-26T07:19:31.113498+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.15","version":"0'0","reported_seq":9,"reported_epoch":43,"state":"active+clean","last_fresh":"2023-11-26T07:19:31.149462+0000","last_change":"2023-11-26T07:19:31.147755+0000","last_active":"2023-11-26T07:19:31.149462+0000","last_peered":"2023-11-26T07:19:31.149462+0000","last_clean":"2023-11-26T07:19:31.149462+0000","last_became_active":"2023-11-26T07:19:31.147461+0000","last_became_peered":"2023-11-26T07:19:31.147461+0000","last_unstale":"2023-11-26T07:19:31.149462+0000","last_undegraded":"2023-11-26T07:19:31.149462+0000","last_fullsized":"2023-11-26T07:19:31.149462+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:19:30.106639+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:19:30.106639+0000","last_clean_scrub_stamp":"2023-11-26T07:19:30.106639+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T17:07:29.328878+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.15","version":"0'0","reported_seq":1,"reported_epoch":43,"state":"creating+peering","last_fresh":"2023-11-26T07:19:31.132934+0000","last_change":"2023-11-26T07:19:31.132934+0000","last_active":"2023-11-26T07:19:31.113498+0000","last_peered":"2023-11-26T07:19:31.113498+0000","last_clean":"2023-11-26T07:19:31.113498+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-26T07:19:31.132934+0000","last_undegraded":"2023-11-26T07:19:31.132934+0000","last_fullsized":"2023-11-26T07:19:31.132934+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:19:31.113498+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:19:31.113498+0000","last_clean_scrub_stamp":"2023-11-26T07:19:31.113498+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.14","version":"0'0","reported_seq":9,"reported_epoch":43,"state":"active+clean","last_fresh":"2023-11-26T07:19:31.136478+0000","last_change":"2023-11-26T07:19:31.131243+0000","last_active":"2023-11-26T07:19:31.136478+0000","last_peered":"2023-11-26T07:19:31.136478+0000","last_clean":"2023-11-26T07:19:31.136478+0000","last_became_active":"2023-11-26T07:19:31.130865+0000","last_became_peered":"2023-11-26T07:19:31.130865+0000","last_unstale":"2023-11-26T07:19:31.136478+0000","last_undegraded":"2023-11-26T07:19:31.136478+0000","last_fullsized":"2023-11-26T07:19:31.136478+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:19:30.106639+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:19:30.106639+0000","last_clean_scrub_stamp":"2023-11-26T07:19:30.106639+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T08:45:01.391060+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.12","version":"0'0","reported_seq":1,"reported_epoch":43,"state":"creating+peering","last_fresh":"2023-11-26T07:19:31.132304+0000","last_change":"2023-11-26T07:19:31.132304+0000","last_active":"2023-11-26T07:19:31.113498+0000","last_peered":"2023-11-26T07:19:31.113498+0000","last_clean":"2023-11-26T07:19:31.113498+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-26T07:19:31.132304+0000","last_undegraded":"2023-11-26T07:19:31.132304+0000","last_fullsized":"2023-11-26T07:19:31.132304+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:19:31.113498+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:19:31.113498+0000","last_clean_scrub_stamp":"2023-11-26T07:19:31.113498+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.13","version":"0'0","reported_seq":9,"reported_epoch":43,"state":"active+clean","last_fresh":"2023-11-26T07:19:31.153151+0000","last_change":"2023-11-26T07:19:31.148780+0000","last_active":"2023-11-26T07:19:31.153151+0000","last_peered":"2023-11-26T07:19:31.153151+0000","last_clean":"2023-11-26T07:19:31.153151+0000","last_became_active":"2023-11-26T07:19:31.148142+0000","last_became_peered":"2023-11-26T07:19:31.148142+0000","last_unstale":"2023-11-26T07:19:31.153151+0000","last_undegraded":"2023-11-26T07:19:31.153151+0000","last_fullsized":"2023-11-26T07:19:31.153151+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:19:30.106639+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:19:30.106639+0000","last_clean_scrub_stamp":"2023-11-26T07:19:30.106639+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T17:07:29.328878+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.13","version":"0'0","reported_seq":1,"reported_epoch":43,"state":"creating+peering","last_fresh":"2023-11-26T07:19:31.130763+0000","last_change":"2023-11-26T07:19:31.130763+0000","last_active":"2023-11-26T07:19:31.113498+0000","last_peered":"2023-11-26T07:19:31.113498+0000","last_clean":"2023-11-26T07:19:31.113498+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-26T07:19:31.130763+0000","last_undegraded":"2023-11-26T07:19:31.130763+0000","last_fullsized":"2023-11-26T07:19:31.130763+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:19:31.113498+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:19:31.113498+0000","last_clean_scrub_stamp":"2023-11-26T07:19:31.113498+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.12","version":"0'0","reported_seq":9,"reported_epoch":43,"state":"active+clean","last_fresh":"2023-11-26T07:19:31.147945+0000","last_change":"2023-11-26T07:19:31.144842+0000","last_active":"2023-11-26T07:19:31.147945+0000","last_peered":"2023-11-26T07:19:31.147945+0000","last_clean":"2023-11-26T07:19:31.147945+0000","last_became_active":"2023-11-26T07:19:31.144554+0000","last_became_peered":"2023-11-26T07:19:31.144554+0000","last_unstale":"2023-11-26T07:19:31.147945+0000","last_undegraded":"2023-11-26T07:19:31.147945+0000","last_fullsized":"2023-11-26T07:19:31.147945+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:19:30.106639+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:19:30.106639+0000","last_clean_scrub_stamp":"2023-11-26T07:19:30.106639+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T15:38:28.537385+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.10","version":"0'0","reported_seq":1,"reported_epoch":43,"state":"creating+peering","last_fresh":"2023-11-26T07:19:31.134915+0000","last_change":"2023-11-26T07:19:31.134915+0000","last_active":"2023-11-26T07:19:31.113498+0000","last_peered":"2023-11-26T07:19:31.113498+0000","last_clean":"2023-11-26T07:19:31.113498+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-26T07:19:31.134915+0000","last_undegraded":"2023-11-26T07:19:31.134915+0000","last_fullsized":"2023-11-26T07:19:31.134915+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:19:31.113498+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:19:31.113498+0000","last_clean_scrub_stamp":"2023-11-26T07:19:31.113498+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.11","version":"0'0","reported_seq":9,"reported_epoch":43,"state":"active+clean","last_fresh":"2023-11-26T07:19:31.153141+0000","last_change":"2023-11-26T07:19:31.148587+0000","last_active":"2023-11-26T07:19:31.153141+0000","last_peered":"2023-11-26T07:19:31.153141+0000","last_clean":"2023-11-26T07:19:31.153141+0000","last_became_active":"2023-11-26T07:19:31.148170+0000","last_became_peered":"2023-11-26T07:19:31.148170+0000","last_unstale":"2023-11-26T07:19:31.153141+0000","last_undegraded":"2023-11-26T07:19:31.153141+0000","last_fullsized":"2023-11-26T07:19:31.153141+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:19:30.106639+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:19:30.106639+0000","last_clean_scrub_stamp":"2023-11-26T07:19:30.106639+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T16:13:47.070273+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.11","version":"0'0","reported_seq":1,"reported_epoch":43,"state":"creating+peering","last_fresh":"2023-11-26T07:19:31.129918+0000","last_change":"2023-11-26T07:19:31.129918+0000","last_active":"2023-11-26T07:19:31.113498+0000","last_peered":"2023-11-26T07:19:31.113498+0000","last_clean":"2023-11-26T07:19:31.113498+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-26T07:19:31.129918+0000","last_undegraded":"2023-11-26T07:19:31.129918+0000","last_fullsized":"2023-11-26T07:19:31.129918+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:19:31.113498+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:19:31.113498+0000","last_clean_scrub_stamp":"2023-11-26T07:19:31.113498+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.10","version":"0'0","reported_seq":9,"reported_epoch":43,"state":"active+clean","last_fresh":"2023-11-26T07:19:31.147291+0000","last_change":"2023-11-26T07:19:31.139691+0000","last_active":"2023-11-26T07:19:31.147291+0000","last_peered":"2023-11-26T07:19:31.147291+0000","last_clean":"2023-11-26T07:19:31.147291+0000","last_became_active":"2023-11-26T07:19:31.139402+0000","last_became_peered":"2023-11-26T07:19:31.139402+0000","last_unstale":"2023-11-26T07:19:31.147291+0000","last_undegraded":"2023-11-26T07:19:31.147291+0000","last_fullsized":"2023-11-26T07:19:31.147291+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:19:30.106639+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:19:30.106639+0000","last_clean_scrub_stamp":"2023-11-26T07:19:30.106639+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T08:28:59.053370+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.e","version":"0'0","reported_seq":1,"reported_epoch":43,"state":"creating+peering","last_fresh":"2023-11-26T07:19:31.126558+0000","last_change":"2023-11-26T07:19:31.126558+0000","last_active":"2023-11-26T07:19:31.113498+0000","last_peered":"2023-11-26T07:19:31.113498+0000","last_clean":"2023-11-26T07:19:31.113498+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-26T07:19:31.126558+0000","last_undegraded":"2023-11-26T07:19:31.126558+0000","last_fullsized":"2023-11-26T07:19:31.126558+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:19:31.113498+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:19:31.113498+0000","last_clean_scrub_stamp":"2023-11-26T07:19:31.113498+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.f","version":"0'0","reported_seq":9,"reported_epoch":43,"state":"active+clean","last_fresh":"2023-11-26T07:19:31.144746+0000","last_change":"2023-11-26T07:19:31.140922+0000","last_active":"2023-11-26T07:19:31.144746+0000","last_peered":"2023-11-26T07:19:31.144746+0000","last_clean":"2023-11-26T07:19:31.144746+0000","last_became_active":"2023-11-26T07:19:31.139513+0000","last_became_peered":"2023-11-26T07:19:31.139513+0000","last_unstale":"2023-11-26T07:19:31.144746+0000","last_undegraded":"2023-11-26T07:19:31.144746+0000","last_fullsized":"2023-11-26T07:19:31.144746+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:19:30.106639+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:19:30.106639+0000","last_clean_scrub_stamp":"2023-11-26T07:19:30.106639+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T09:39:24.076764+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.f","version":"0'0","reported_seq":1,"reported_epoch":43,"state":"creating+peering","last_fresh":"2023-11-26T07:19:31.128236+0000","last_change":"2023-11-26T07:19:31.128236+0000","last_active":"2023-11-26T07:19:31.113498+0000","last_peered":"2023-11-26T07:19:31.113498+0000","last_clean":"2023-11-26T07:19:31.113498+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-26T07:19:31.128236+0000","last_undegraded":"2023-11-26T07:19:31.128236+0000","last_fullsized":"2023-11-26T07:19:31.128236+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:19:31.113498+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:19:31.113498+0000","last_clean_scrub_stamp":"2023-11-26T07:19:31.113498+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.e","version":"0'0","reported_seq":9,"reported_epoch":43,"state":"active+clean","last_fresh":"2023-11-26T07:19:31.146310+0000","last_change":"2023-11-26T07:19:31.141584+0000","last_active":"2023-11-26T07:19:31.146310+0000","last_peered":"2023-11-26T07:19:31.146310+0000","last_clean":"2023-11-26T07:19:31.146310+0000","last_became_active":"2023-11-26T07:19:31.141272+0000","last_became_peered":"2023-11-26T07:19:31.141272+0000","last_unstale":"2023-11-26T07:19:31.146310+0000","last_undegraded":"2023-11-26T07:19:31.146310+0000","last_fullsized":"2023-11-26T07:19:31.146310+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:19:30.106639+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:19:30.106639+0000","last_clean_scrub_stamp":"2023-11-26T07:19:30.106639+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T12:10:52.088348+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.c","version":"0'0","reported_seq":1,"reported_epoch":43,"state":"creating+peering","last_fresh":"2023-11-26T07:19:31.132510+0000","last_change":"2023-11-26T07:19:31.132510+0000","last_active":"2023-11-26T07:19:31.113498+0000","last_peered":"2023-11-26T07:19:31.113498+0000","last_clean":"2023-11-26T07:19:31.113498+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-26T07:19:31.132510+0000","last_undegraded":"2023-11-26T07:19:31.132510+0000","last_fullsized":"2023-11-26T07:19:31.132510+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:19:31.113498+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:19:31.113498+0000","last_clean_scrub_stamp":"2023-11-26T07:19:31.113498+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.d","version":"0'0","reported_seq":9,"reported_epoch":43,"state":"active+clean","last_fresh":"2023-11-26T07:19:31.149609+0000","last_change":"2023-11-26T07:19:31.145210+0000","last_active":"2023-11-26T07:19:31.149609+0000","last_peered":"2023-11-26T07:19:31.149609+0000","last_clean":"2023-11-26T07:19:31.149609+0000","last_became_active":"2023-11-26T07:19:31.144917+0000","last_became_peered":"2023-11-26T07:19:31.144917+0000","last_unstale":"2023-11-26T07:19:31.149609+0000","last_undegraded":"2023-11-26T07:19:31.149609+0000","last_fullsized":"2023-11-26T07:19:31.149609+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:19:30.106639+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:19:30.106639+0000","last_clean_scrub_stamp":"2023-11-26T07:19:30.106639+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T19:18:59.191940+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.d","version":"0'0","reported_seq":1,"reported_epoch":43,"state":"creating+peering","last_fresh":"2023-11-26T07:19:31.134528+0000","last_change":"2023-11-26T07:19:31.134528+0000","last_active":"2023-11-26T07:19:31.113498+0000","last_peered":"2023-11-26T07:19:31.113498+0000","last_clean":"2023-11-26T07:19:31.113498+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-26T07:19:31.134528+0000","last_undegraded":"2023-11-26T07:19:31.134528+0000","last_fullsized":"2023-11-26T07:19:31.134528+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:19:31.113498+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:19:31.113498+0000","last_clean_scrub_stamp":"2023-11-26T07:19:31.113498+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.c","version":"0'0","reported_seq":9,"reported_epoch":43,"state":"active+clean","last_fresh":"2023-11-26T07:19:31.150641+0000","last_change":"2023-11-26T07:19:31.147668+0000","last_active":"2023-11-26T07:19:31.150641+0000","last_peered":"2023-11-26T07:19:31.150641+0000","last_clean":"2023-11-26T07:19:31.150641+0000","last_became_active":"2023-11-26T07:19:31.146685+0000","last_became_peered":"2023-11-26T07:19:31.146685+0000","last_unstale":"2023-11-26T07:19:31.150641+0000","last_undegraded":"2023-11-26T07:19:31.150641+0000","last_fullsized":"2023-11-26T07:19:31.150641+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:19:30.106639+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:19:30.106639+0000","last_clean_scrub_stamp":"2023-11-26T07:19:30.106639+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T19:14:07.171899+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.a","version":"0'0","reported_seq":1,"reported_epoch":43,"state":"creating+peering","last_fresh":"2023-11-26T07:19:31.137790+0000","last_change":"2023-11-26T07:19:31.137790+0000","last_active":"2023-11-26T07:19:31.113498+0000","last_peered":"2023-11-26T07:19:31.113498+0000","last_clean":"2023-11-26T07:19:31.113498+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-26T07:19:31.137790+0000","last_undegraded":"2023-11-26T07:19:31.137790+0000","last_fullsized":"2023-11-26T07:19:31.137790+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:19:31.113498+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:19:31.113498+0000","last_clean_scrub_stamp":"2023-11-26T07:19:31.113498+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.b","version":"0'0","reported_seq":9,"reported_epoch":43,"state":"active+clean","last_fresh":"2023-11-26T07:19:31.153174+0000","last_change":"2023-11-26T07:19:31.149345+0000","last_active":"2023-11-26T07:19:31.153174+0000","last_peered":"2023-11-26T07:19:31.153174+0000","last_clean":"2023-11-26T07:19:31.153174+0000","last_became_active":"2023-11-26T07:19:31.149072+0000","last_became_peered":"2023-11-26T07:19:31.149072+0000","last_unstale":"2023-11-26T07:19:31.153174+0000","last_undegraded":"2023-11-26T07:19:31.153174+0000","last_fullsized":"2023-11-26T07:19:31.153174+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:19:30.106639+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:19:30.106639+0000","last_clean_scrub_stamp":"2023-11-26T07:19:30.106639+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T10:27:01.949013+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.b","version":"0'0","reported_seq":1,"reported_epoch":43,"state":"creating+peering","last_fresh":"2023-11-26T07:19:31.132864+0000","last_change":"2023-11-26T07:19:31.132864+0000","last_active":"2023-11-26T07:19:31.113498+0000","last_peered":"2023-11-26T07:19:31.113498+0000","last_clean":"2023-11-26T07:19:31.113498+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-26T07:19:31.132864+0000","last_undegraded":"2023-11-26T07:19:31.132864+0000","last_fullsized":"2023-11-26T07:19:31.132864+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:19:31.113498+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:19:31.113498+0000","last_clean_scrub_stamp":"2023-11-26T07:19:31.113498+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.a","version":"0'0","reported_seq":9,"reported_epoch":43,"state":"active+clean","last_fresh":"2023-11-26T07:19:31.150714+0000","last_change":"2023-11-26T07:19:31.146363+0000","last_active":"2023-11-26T07:19:31.150714+0000","last_peered":"2023-11-26T07:19:31.150714+0000","last_clean":"2023-11-26T07:19:31.150714+0000","last_became_active":"2023-11-26T07:19:31.146013+0000","last_became_peered":"2023-11-26T07:19:31.146013+0000","last_unstale":"2023-11-26T07:19:31.150714+0000","last_undegraded":"2023-11-26T07:19:31.150714+0000","last_fullsized":"2023-11-26T07:19:31.150714+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:19:30.106639+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:19:30.106639+0000","last_clean_scrub_stamp":"2023-11-26T07:19:30.106639+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T12:54:51.008938+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.8","version":"0'0","reported_seq":1,"reported_epoch":43,"state":"creating+peering","last_fresh":"2023-11-26T07:19:31.126547+0000","last_change":"2023-11-26T07:19:31.126547+0000","last_active":"2023-11-26T07:19:31.113498+0000","last_peered":"2023-11-26T07:19:31.113498+0000","last_clean":"2023-11-26T07:19:31.113498+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-26T07:19:31.126547+0000","last_undegraded":"2023-11-26T07:19:31.126547+0000","last_fullsized":"2023-11-26T07:19:31.126547+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:19:31.113498+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:19:31.113498+0000","last_clean_scrub_stamp":"2023-11-26T07:19:31.113498+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.9","version":"0'0","reported_seq":9,"reported_epoch":43,"state":"active+clean","last_fresh":"2023-11-26T07:19:31.153191+0000","last_change":"2023-11-26T07:19:31.148399+0000","last_active":"2023-11-26T07:19:31.153191+0000","last_peered":"2023-11-26T07:19:31.153191+0000","last_clean":"2023-11-26T07:19:31.153191+0000","last_became_active":"2023-11-26T07:19:31.148127+0000","last_became_peered":"2023-11-26T07:19:31.148127+0000","last_unstale":"2023-11-26T07:19:31.153191+0000","last_undegraded":"2023-11-26T07:19:31.153191+0000","last_fullsized":"2023-11-26T07:19:31.153191+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:19:30.106639+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:19:30.106639+0000","last_clean_scrub_stamp":"2023-11-26T07:19:30.106639+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T16:49:27.472665+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.9","version":"0'0","reported_seq":1,"reported_epoch":43,"state":"creating+peering","last_fresh":"2023-11-26T07:19:31.129887+0000","last_change":"2023-11-26T07:19:31.129887+0000","last_active":"2023-11-26T07:19:31.113498+0000","last_peered":"2023-11-26T07:19:31.113498+0000","last_clean":"2023-11-26T07:19:31.113498+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-26T07:19:31.129887+0000","last_undegraded":"2023-11-26T07:19:31.129887+0000","last_fullsized":"2023-11-26T07:19:31.129887+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:19:31.113498+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:19:31.113498+0000","last_clean_scrub_stamp":"2023-11-26T07:19:31.113498+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.8","version":"0'0","reported_seq":9,"reported_epoch":43,"state":"active+clean","last_fresh":"2023-11-26T07:19:31.147571+0000","last_change":"2023-11-26T07:19:31.144956+0000","last_active":"2023-11-26T07:19:31.147571+0000","last_peered":"2023-11-26T07:19:31.147571+0000","last_clean":"2023-11-26T07:19:31.147571+0000","last_became_active":"2023-11-26T07:19:31.144653+0000","last_became_peered":"2023-11-26T07:19:31.144653+0000","last_unstale":"2023-11-26T07:19:31.147571+0000","last_undegraded":"2023-11-26T07:19:31.147571+0000","last_fullsized":"2023-11-26T07:19:31.147571+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:19:30.106639+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:19:30.106639+0000","last_clean_scrub_stamp":"2023-11-26T07:19:30.106639+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T13:42:55.807051+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.7","version":"0'0","reported_seq":1,"reported_epoch":43,"state":"creating+peering","last_fresh":"2023-11-26T07:19:31.132550+0000","last_change":"2023-11-26T07:19:31.132550+0000","last_active":"2023-11-26T07:19:31.113498+0000","last_peered":"2023-11-26T07:19:31.113498+0000","last_clean":"2023-11-26T07:19:31.113498+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-26T07:19:31.132550+0000","last_undegraded":"2023-11-26T07:19:31.132550+0000","last_fullsized":"2023-11-26T07:19:31.132550+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:19:31.113498+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:19:31.113498+0000","last_clean_scrub_stamp":"2023-11-26T07:19:31.113498+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.6","version":"0'0","reported_seq":9,"reported_epoch":43,"state":"active+clean","last_fresh":"2023-11-26T07:19:31.148470+0000","last_change":"2023-11-26T07:19:31.143699+0000","last_active":"2023-11-26T07:19:31.148470+0000","last_peered":"2023-11-26T07:19:31.148470+0000","last_clean":"2023-11-26T07:19:31.148470+0000","last_became_active":"2023-11-26T07:19:31.143421+0000","last_became_peered":"2023-11-26T07:19:31.143421+0000","last_unstale":"2023-11-26T07:19:31.148470+0000","last_undegraded":"2023-11-26T07:19:31.148470+0000","last_fullsized":"2023-11-26T07:19:31.148470+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:19:30.106639+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:19:30.106639+0000","last_clean_scrub_stamp":"2023-11-26T07:19:30.106639+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T13:42:55.807051+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.4","version":"0'0","reported_seq":1,"reported_epoch":43,"state":"creating+peering","last_fresh":"2023-11-26T07:19:31.128178+0000","last_change":"2023-11-26T07:19:31.128178+0000","last_active":"2023-11-26T07:19:31.113498+0000","last_peered":"2023-11-26T07:19:31.113498+0000","last_clean":"2023-11-26T07:19:31.113498+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-26T07:19:31.128178+0000","last_undegraded":"2023-11-26T07:19:31.128178+0000","last_fullsized":"2023-11-26T07:19:31.128178+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:19:31.113498+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:19:31.113498+0000","last_clean_scrub_stamp":"2023-11-26T07:19:31.113498+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.5","version":"0'0","reported_seq":9,"reported_epoch":43,"state":"active+clean","last_fresh":"2023-11-26T07:19:31.150074+0000","last_change":"2023-11-26T07:19:31.148100+0000","last_active":"2023-11-26T07:19:31.150074+0000","last_peered":"2023-11-26T07:19:31.150074+0000","last_clean":"2023-11-26T07:19:31.150074+0000","last_became_active":"2023-11-26T07:19:31.147766+0000","last_became_peered":"2023-11-26T07:19:31.147766+0000","last_unstale":"2023-11-26T07:19:31.150074+0000","last_undegraded":"2023-11-26T07:19:31.150074+0000","last_fullsized":"2023-11-26T07:19:31.150074+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:19:30.106639+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:19:30.106639+0000","last_clean_scrub_stamp":"2023-11-26T07:19:30.106639+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T13:58:09.825490+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.5","version":"0'0","reported_seq":1,"reported_epoch":43,"state":"creating+peering","last_fresh":"2023-11-26T07:19:31.132476+0000","last_change":"2023-11-26T07:19:31.132476+0000","last_active":"2023-11-26T07:19:31.113498+0000","last_peered":"2023-11-26T07:19:31.113498+0000","last_clean":"2023-11-26T07:19:31.113498+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-26T07:19:31.132476+0000","last_undegraded":"2023-11-26T07:19:31.132476+0000","last_fullsized":"2023-11-26T07:19:31.132476+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:19:31.113498+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:19:31.113498+0000","last_clean_scrub_stamp":"2023-11-26T07:19:31.113498+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.4","version":"0'0","reported_seq":9,"reported_epoch":43,"state":"active+clean","last_fresh":"2023-11-26T07:19:31.150706+0000","last_change":"2023-11-26T07:19:31.144386+0000","last_active":"2023-11-26T07:19:31.150706+0000","last_peered":"2023-11-26T07:19:31.150706+0000","last_clean":"2023-11-26T07:19:31.150706+0000","last_became_active":"2023-11-26T07:19:31.144106+0000","last_became_peered":"2023-11-26T07:19:31.144106+0000","last_unstale":"2023-11-26T07:19:31.150706+0000","last_undegraded":"2023-11-26T07:19:31.150706+0000","last_fullsized":"2023-11-26T07:19:31.150706+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:19:30.106639+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:19:30.106639+0000","last_clean_scrub_stamp":"2023-11-26T07:19:30.106639+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T08:08:30.523864+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.2","version":"0'0","reported_seq":1,"reported_epoch":43,"state":"creating+peering","last_fresh":"2023-11-26T07:19:31.134609+0000","last_change":"2023-11-26T07:19:31.134609+0000","last_active":"2023-11-26T07:19:31.113498+0000","last_peered":"2023-11-26T07:19:31.113498+0000","last_clean":"2023-11-26T07:19:31.113498+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-26T07:19:31.134609+0000","last_undegraded":"2023-11-26T07:19:31.134609+0000","last_fullsized":"2023-11-26T07:19:31.134609+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:19:31.113498+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:19:31.113498+0000","last_clean_scrub_stamp":"2023-11-26T07:19:31.113498+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.3","version":"0'0","reported_seq":9,"reported_epoch":43,"state":"active+clean","last_fresh":"2023-11-26T07:19:31.149359+0000","last_change":"2023-11-26T07:19:31.146929+0000","last_active":"2023-11-26T07:19:31.149359+0000","last_peered":"2023-11-26T07:19:31.149359+0000","last_clean":"2023-11-26T07:19:31.149359+0000","last_became_active":"2023-11-26T07:19:31.146630+0000","last_became_peered":"2023-11-26T07:19:31.146630+0000","last_unstale":"2023-11-26T07:19:31.149359+0000","last_undegraded":"2023-11-26T07:19:31.149359+0000","last_fullsized":"2023-11-26T07:19:31.149359+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:19:30.106639+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:19:30.106639+0000","last_clean_scrub_stamp":"2023-11-26T07:19:30.106639+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T19:14:07.171899+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.3","version":"0'0","reported_seq":1,"reported_epoch":43,"state":"creating+peering","last_fresh":"2023-11-26T07:19:31.134861+0000","last_change":"2023-11-26T07:19:31.134861+0000","last_active":"2023-11-26T07:19:31.113498+0000","last_peered":"2023-11-26T07:19:31.113498+0000","last_clean":"2023-11-26T07:19:31.113498+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-26T07:19:31.134861+0000","last_undegraded":"2023-11-26T07:19:31.134861+0000","last_fullsized":"2023-11-26T07:19:31.134861+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:19:31.113498+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:19:31.113498+0000","last_clean_scrub_stamp":"2023-11-26T07:19:31.113498+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.2","version":"0'0","reported_seq":9,"reported_epoch":43,"state":"active+clean","last_fresh":"2023-11-26T07:19:31.149756+0000","last_change":"2023-11-26T07:19:31.147730+0000","last_active":"2023-11-26T07:19:31.149756+0000","last_peered":"2023-11-26T07:19:31.149756+0000","last_clean":"2023-11-26T07:19:31.149756+0000","last_became_active":"2023-11-26T07:19:31.147374+0000","last_became_peered":"2023-11-26T07:19:31.147374+0000","last_unstale":"2023-11-26T07:19:31.149756+0000","last_undegraded":"2023-11-26T07:19:31.149756+0000","last_fullsized":"2023-11-26T07:19:31.149756+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:19:30.106639+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:19:30.106639+0000","last_clean_scrub_stamp":"2023-11-26T07:19:30.106639+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T16:49:27.472665+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.0","version":"0'0","reported_seq":1,"reported_epoch":43,"state":"creating+peering","last_fresh":"2023-11-26T07:19:31.130272+0000","last_change":"2023-11-26T07:19:31.130272+0000","last_active":"2023-11-26T07:19:31.113498+0000","last_peered":"2023-11-26T07:19:31.113498+0000","last_clean":"2023-11-26T07:19:31.113498+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-26T07:19:31.130272+0000","last_undegraded":"2023-11-26T07:19:31.130272+0000","last_fullsized":"2023-11-26T07:19:31.130272+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:19:31.113498+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:19:31.113498+0000","last_clean_scrub_stamp":"2023-11-26T07:19:31.113498+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.1","version":"0'0","reported_seq":9,"reported_epoch":43,"state":"active+clean","last_fresh":"2023-11-26T07:19:31.152909+0000","last_change":"2023-11-26T07:19:31.145269+0000","last_active":"2023-11-26T07:19:31.152909+0000","last_peered":"2023-11-26T07:19:31.152909+0000","last_clean":"2023-11-26T07:19:31.152909+0000","last_became_active":"2023-11-26T07:19:31.144994+0000","last_became_peered":"2023-11-26T07:19:31.144994+0000","last_unstale":"2023-11-26T07:19:31.152909+0000","last_undegraded":"2023-11-26T07:19:31.152909+0000","last_fullsized":"2023-11-26T07:19:31.152909+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:19:30.106639+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:19:30.106639+0000","last_clean_scrub_stamp":"2023-11-26T07:19:30.106639+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T15:32:40.529918+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.1","version":"0'0","reported_seq":1,"reported_epoch":43,"state":"creating+peering","last_fresh":"2023-11-26T07:19:31.124694+0000","last_change":"2023-11-26T07:19:31.124694+0000","last_active":"2023-11-26T07:19:31.113498+0000","last_peered":"2023-11-26T07:19:31.113498+0000","last_clean":"2023-11-26T07:19:31.113498+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-26T07:19:31.124694+0000","last_undegraded":"2023-11-26T07:19:31.124694+0000","last_fullsized":"2023-11-26T07:19:31.124694+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:19:31.113498+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:19:31.113498+0000","last_clean_scrub_stamp":"2023-11-26T07:19:31.113498+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.0","version":"0'0","reported_seq":9,"reported_epoch":43,"state":"active+clean","last_fresh":"2023-11-26T07:19:31.151119+0000","last_change":"2023-11-26T07:19:31.146523+0000","last_active":"2023-11-26T07:19:31.151119+0000","last_peered":"2023-11-26T07:19:31.151119+0000","last_clean":"2023-11-26T07:19:31.151119+0000","last_became_active":"2023-11-26T07:19:31.146168+0000","last_became_peered":"2023-11-26T07:19:31.146168+0000","last_unstale":"2023-11-26T07:19:31.151119+0000","last_undegraded":"2023-11-26T07:19:31.151119+0000","last_fullsized":"2023-11-26T07:19:31.151119+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:19:30.106639+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:19:30.106639+0000","last_clean_scrub_stamp":"2023-11-26T07:19:30.106639+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T12:03:38.617112+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.6","version":"0'0","reported_seq":1,"reported_epoch":43,"state":"creating+peering","last_fresh":"2023-11-26T07:19:31.128572+0000","last_change":"2023-11-26T07:19:31.128572+0000","last_active":"2023-11-26T07:19:31.113498+0000","last_peered":"2023-11-26T07:19:31.113498+0000","last_clean":"2023-11-26T07:19:31.113498+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-26T07:19:31.128572+0000","last_undegraded":"2023-11-26T07:19:31.128572+0000","last_fullsized":"2023-11-26T07:19:31.128572+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:19:31.113498+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:19:31.113498+0000","last_clean_scrub_stamp":"2023-11-26T07:19:31.113498+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.7","version":"0'0","reported_seq":9,"reported_epoch":43,"state":"active+clean","last_fresh":"2023-11-26T07:19:31.150685+0000","last_change":"2023-11-26T07:19:31.146427+0000","last_active":"2023-11-26T07:19:31.150685+0000","last_peered":"2023-11-26T07:19:31.150685+0000","last_clean":"2023-11-26T07:19:31.150685+0000","last_became_active":"2023-11-26T07:19:31.146044+0000","last_became_peered":"2023-11-26T07:19:31.146044+0000","last_unstale":"2023-11-26T07:19:31.150685+0000","last_undegraded":"2023-11-26T07:19:31.150685+0000","last_fullsized":"2023-11-26T07:19:31.150685+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:19:30.106639+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:19:30.106639+0000","last_clean_scrub_stamp":"2023-11-26T07:19:30.106639+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T15:54:05.005833+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.0","version":"15'117","reported_seq":182,"reported_epoch":42,"state":"active+clean","last_fresh":"2023-11-26T07:19:30.125712+0000","last_change":"2023-11-26T07:17:43.827802+0000","last_active":"2023-11-26T07:19:30.125712+0000","last_peered":"2023-11-26T07:19:30.125712+0000","last_clean":"2023-11-26T07:19:30.125712+0000","last_became_active":"2023-11-26T07:17:43.827308+0000","last_became_peered":"2023-11-26T07:17:43.827308+0000","last_unstale":"2023-11-26T07:19:30.125712+0000","last_undegraded":"2023-11-26T07:19:30.125712+0000","last_fullsized":"2023-11-26T07:19:30.125712+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:17:42.160945+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:17:42.160945+0000","last_clean_scrub_stamp":"2023-11-26T07:17:42.160945+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T08:17:54.167366+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":7,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":6,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574872,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":28084,"kb_used_data":1060,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343756,"statfs":{"total":96636764160,"available":96608006144,"internally_reserved":0,"allocated":1085440,"data_stored":786125,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Nov 26 07:18:42 2023","interfaces":[{"interface":"back","average":{"1min":0.71499999999999997,"5min":0.71499999999999997,"15min":0.71499999999999997},"min":{"1min":0.54300000000000004,"5min":0.54300000000000004,"15min":0.54300000000000004},"max":{"1min":0.91500000000000004,"5min":0.91500000000000004,"15min":0.91500000000000004},"last":0.57699999999999996},{"interface":"front","average":{"1min":0.70899999999999996,"5min":0.70899999999999996,"15min":0.70899999999999996},"min":{"1min":0.52900000000000003,"5min":0.52900000000000003,"15min":0.52900000000000003},"max":{"1min":1.0980000000000001,"5min":1.0980000000000001,"15min":1.0980000000000001},"last":0.64800000000000002}]},{"osd":1,"last update":"Sun Nov 26 07:18:42 2023","interfaces":[{"interface":"back","average":{"1min":0.68200000000000005,"5min":0.68200000000000005,"15min":0.68200000000000005},"min":{"1min":0.47999999999999998,"5min":0.47999999999999998,"15min":0.47999999999999998},"max":{"1min":1.0669999999999999,"5min":1.0669999999999999,"15min":1.0669999999999999},"last":0.73999999999999999},{"interface":"front","average":{"1min":0.83899999999999997,"5min":0.83899999999999997,"15min":0.83899999999999997},"min":{"1min":0.47599999999999998,"5min":0.47599999999999998,"15min":0.47599999999999998},"max":{"1min":4.5309999999999997,"5min":4.5309999999999997,"15min":4.5309999999999997},"last":0.47699999999999998}]},{"osd":2,"last update":"Sun Nov 26 07:18:42 2023","interfaces":[{"interface":"back","average":{"1min":0.67600000000000005,"5min":0.67600000000000005,"15min":0.67600000000000005},"min":{"1min":0.52000000000000002,"5min":0.52000000000000002,"15min":0.52000000000000002},"max":{"1min":0.873,"5min":0.873,"15min":0.873},"last":0.69499999999999995},{"interface":"front","average":{"1min":0.71299999999999997,"5min":0.71299999999999997,"15min":0.71299999999999997},"min":{"1min":0.54700000000000004,"5min":0.54700000000000004,"15min":0.54700000000000004},"max":{"1min":1.1339999999999999,"5min":1.1339999999999999,"15min":1.1339999999999999},"last":0.61599999999999999}]},{"osd":3,"last update":"Sun Nov 26 07:18:42 2023","interfaces":[{"interface":"back","average":{"1min":0.69699999999999995,"5min":0.69699999999999995,"15min":0.69699999999999995},"min":{"1min":0.51800000000000002,"5min":0.51800000000000002,"15min":0.51800000000000002},"max":{"1min":1.026,"5min":1.026,"15min":1.026},"last":0.627},{"interface":"front","average":{"1min":0.73299999999999998,"5min":0.73299999999999998,"15min":0.73299999999999998},"min":{"1min":0.50600000000000001,"5min":0.50600000000000001,"15min":0.50600000000000001},"max":{"1min":1.4490000000000001,"5min":1.4490000000000001,"15min":1.4490000000000001},"last":0.66200000000000003}]},{"osd":4,"last update":"Sun Nov 26 07:18:42 2023","interfaces":[{"interface":"back","average":{"1min":0.59999999999999998,"5min":0.59999999999999998,"15min":0.59999999999999998},"min":{"1min":0.39400000000000002,"5min":0.39400000000000002,"15min":0.39400000000000002},"max":{"1min":0.94799999999999995,"5min":0.94799999999999995,"15min":0.94799999999999995},"last":0.59599999999999997},{"interface":"front","average":{"1min":0.67300000000000004,"5min":0.67300000000000004,"15min":0.67300000000000004},"min":{"1min":0.436,"5min":0.436,"15min":0.436},"max":{"1min":1.5509999999999999,"5min":1.5509999999999999,"15min":1.5509999999999999},"last":0.51300000000000001}]},{"osd":5,"last update":"Sun Nov 26 07:18:42 2023","interfaces":[{"interface":"back","average":{"1min":0.66600000000000004,"5min":0.66600000000000004,"15min":0.66600000000000004},"min":{"1min":0.443,"5min":0.443,"15min":0.443},"max":{"1min":1.415,"5min":1.415,"15min":1.415},"last":0.77300000000000002},{"interface":"front","average":{"1min":0.71399999999999997,"5min":0.71399999999999997,"15min":0.71399999999999997},"min":{"1min":0.53100000000000003,"5min":0.53100000000000003,"15min":0.53100000000000003},"max":{"1min":1.3859999999999999,"5min":1.3859999999999999,"15min":1.3859999999999999},"last":0.67700000000000005}]},{"osd":6,"last update":"Sun Nov 26 07:18:42 2023","interfaces":[{"interface":"back","average":{"1min":0.77100000000000002,"5min":0.77100000000000002,"15min":0.77100000000000002},"min":{"1min":0.64100000000000001,"5min":0.64100000000000001,"15min":0.64100000000000001},"max":{"1min":1.3069999999999999,"5min":1.3069999999999999,"15min":1.3069999999999999},"last":0.75800000000000001},{"interface":"front","average":{"1min":0.77100000000000002,"5min":0.77100000000000002,"15min":0.77100000000000002},"min":{"1min":0.57399999999999995,"5min":0.57399999999999995,"15min":0.57399999999999995},"max":{"1min":1.6759999999999999,"5min":1.6759999999999999,"15min":1.6759999999999999},"last":0.56200000000000006}]}]},{"osd":6,"up_from":13,"seq":55834574874,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27512,"kb_used_data":492,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344328,"statfs":{"total":96636764160,"available":96608591872,"internally_reserved":0,"allocated":503808,"data_stored":202024,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Nov 26 07:18:45 2023","interfaces":[{"interface":"back","average":{"1min":0.68400000000000005,"5min":0.68400000000000005,"15min":0.68400000000000005},"min":{"1min":0.36699999999999999,"5min":0.36699999999999999,"15min":0.36699999999999999},"max":{"1min":1.7549999999999999,"5min":1.7549999999999999,"15min":1.7549999999999999},"last":0.80500000000000005},{"interface":"front","average":{"1min":0.95799999999999996,"5min":0.95799999999999996,"15min":0.95799999999999996},"min":{"1min":0.38300000000000001,"5min":0.38300000000000001,"15min":0.38300000000000001},"max":{"1min":5.7489999999999997,"5min":5.7489999999999997,"15min":5.7489999999999997},"last":0.59599999999999997}]},{"osd":1,"last update":"Sun Nov 26 07:18:45 2023","interfaces":[{"interface":"back","average":{"1min":0.91400000000000003,"5min":0.91400000000000003,"15min":0.91400000000000003},"min":{"1min":0.42099999999999999,"5min":0.42099999999999999,"15min":0.42099999999999999},"max":{"1min":5.4240000000000004,"5min":5.4240000000000004,"15min":5.4240000000000004},"last":0.46400000000000002},{"interface":"front","average":{"1min":0.89700000000000002,"5min":0.89700000000000002,"15min":0.89700000000000002},"min":{"1min":0.499,"5min":0.499,"15min":0.499},"max":{"1min":5.6020000000000003,"5min":5.6020000000000003,"15min":5.6020000000000003},"last":0.64700000000000002}]},{"osd":2,"last update":"Sun Nov 26 07:18:45 2023","interfaces":[{"interface":"back","average":{"1min":0.69699999999999995,"5min":0.69699999999999995,"15min":0.69699999999999995},"min":{"1min":0.47599999999999998,"5min":0.47599999999999998,"15min":0.47599999999999998},"max":{"1min":1.8520000000000001,"5min":1.8520000000000001,"15min":1.8520000000000001},"last":0.624},{"interface":"front","average":{"1min":0.73399999999999999,"5min":0.73399999999999999,"15min":0.73399999999999999},"min":{"1min":0.46700000000000003,"5min":0.46700000000000003,"15min":0.46700000000000003},"max":{"1min":1.9159999999999999,"5min":1.9159999999999999,"15min":1.9159999999999999},"last":0.67400000000000004}]},{"osd":3,"last update":"Sun Nov 26 07:18:45 2023","interfaces":[{"interface":"back","average":{"1min":0.90200000000000002,"5min":0.90200000000000002,"15min":0.90200000000000002},"min":{"1min":0.502,"5min":0.502,"15min":0.502},"max":{"1min":5.4390000000000001,"5min":5.4390000000000001,"15min":5.4390000000000001},"last":0.70099999999999996},{"interface":"front","average":{"1min":0.94499999999999995,"5min":0.94499999999999995,"15min":0.94499999999999995},"min":{"1min":0.54400000000000004,"5min":0.54400000000000004,"15min":0.54400000000000004},"max":{"1min":5.5490000000000004,"5min":5.5490000000000004,"15min":5.5490000000000004},"last":0.71899999999999997}]},{"osd":4,"last update":"Sun Nov 26 07:18:45 2023","interfaces":[{"interface":"back","average":{"1min":0.64200000000000002,"5min":0.64200000000000002,"15min":0.64200000000000002},"min":{"1min":0.46899999999999997,"5min":0.46899999999999997,"15min":0.46899999999999997},"max":{"1min":0.85099999999999998,"5min":0.85099999999999998,"15min":0.85099999999999998},"last":0.85999999999999999},{"interface":"front","average":{"1min":0.67500000000000004,"5min":0.67500000000000004,"15min":0.67500000000000004},"min":{"1min":0.40699999999999997,"5min":0.40699999999999997,"15min":0.40699999999999997},"max":{"1min":0.93300000000000005,"5min":0.93300000000000005,"15min":0.93300000000000005},"last":0.78200000000000003}]},{"osd":5,"last update":"Sun Nov 26 07:18:45 2023","interfaces":[{"interface":"back","average":{"1min":0.65900000000000003,"5min":0.65900000000000003,"15min":0.65900000000000003},"min":{"1min":0.42499999999999999,"5min":0.42499999999999999,"15min":0.42499999999999999},"max":{"1min":0.79300000000000004,"5min":0.79300000000000004,"15min":0.79300000000000004},"last":0.83299999999999996},{"interface":"front","average":{"1min":0.71299999999999997,"5min":0.71299999999999997,"15min":0.71299999999999997},"min":{"1min":0.436,"5min":0.436,"15min":0.436},"max":{"1min":0.93000000000000005,"5min":0.93000000000000005,"15min":0.93000000000000005},"last":0.90100000000000002}]},{"osd":7,"last update":"Sun Nov 26 07:18:45 2023","interfaces":[{"interface":"back","average":{"1min":0.74199999999999999,"5min":0.74199999999999999,"15min":0.74199999999999999},"min":{"1min":0.46600000000000003,"5min":0.46600000000000003,"15min":0.46600000000000003},"max":{"1min":0.998,"5min":0.998,"15min":0.998},"last":0.746},{"interface":"front","average":{"1min":0.71599999999999997,"5min":0.71599999999999997,"15min":0.71599999999999997},"min":{"1min":0.498,"5min":0.498,"15min":0.498},"max":{"1min":0.86499999999999999,"5min":0.86499999999999999,"15min":0.86499999999999999},"last":0.88100000000000001}]}]},{"osd":5,"up_from":13,"seq":55834574872,"num_pgs":4,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27512,"kb_used_data":492,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344328,"statfs":{"total":96636764160,"available":96608591872,"internally_reserved":0,"allocated":503808,"data_stored":202024,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Nov 26 07:18:45 2023","interfaces":[{"interface":"back","average":{"1min":0.66600000000000004,"5min":0.66600000000000004,"15min":0.66600000000000004},"min":{"1min":0.38100000000000001,"5min":0.38100000000000001,"15min":0.38100000000000001},"max":{"1min":1.0289999999999999,"5min":1.0289999999999999,"15min":1.0289999999999999},"last":0.66500000000000004},{"interface":"front","average":{"1min":0.66000000000000003,"5min":0.66000000000000003,"15min":0.66000000000000003},"min":{"1min":0.438,"5min":0.438,"15min":0.438},"max":{"1min":1.05,"5min":1.05,"15min":1.05},"last":0.58099999999999996}]},{"osd":1,"last update":"Sun Nov 26 07:18:45 2023","interfaces":[{"interface":"back","average":{"1min":0.66800000000000004,"5min":0.66800000000000004,"15min":0.66800000000000004},"min":{"1min":0.41599999999999998,"5min":0.41599999999999998,"15min":0.41599999999999998},"max":{"1min":0.86699999999999999,"5min":0.86699999999999999,"15min":0.86699999999999999},"last":0.872},{"interface":"front","average":{"1min":0.68600000000000005,"5min":0.68600000000000005,"15min":0.68600000000000005},"min":{"1min":0.49199999999999999,"5min":0.49199999999999999,"15min":0.49199999999999999},"max":{"1min":0.98899999999999999,"5min":0.98899999999999999,"15min":0.98899999999999999},"last":0.72899999999999998}]},{"osd":2,"last update":"Sun Nov 26 07:18:45 2023","interfaces":[{"interface":"back","average":{"1min":0.67500000000000004,"5min":0.67500000000000004,"15min":0.67500000000000004},"min":{"1min":0.45900000000000002,"5min":0.45900000000000002,"15min":0.45900000000000002},"max":{"1min":0.89800000000000002,"5min":0.89800000000000002,"15min":0.89800000000000002},"last":0.627},{"interface":"front","average":{"1min":0.67000000000000004,"5min":0.67000000000000004,"15min":0.67000000000000004},"min":{"1min":0.51200000000000001,"5min":0.51200000000000001,"15min":0.51200000000000001},"max":{"1min":0.89700000000000002,"5min":0.89700000000000002,"15min":0.89700000000000002},"last":0.76000000000000001}]},{"osd":3,"last update":"Sun Nov 26 07:18:45 2023","interfaces":[{"interface":"back","average":{"1min":0.71999999999999997,"5min":0.71999999999999997,"15min":0.71999999999999997},"min":{"1min":0.43099999999999999,"5min":0.43099999999999999,"15min":0.43099999999999999},"max":{"1min":1.0029999999999999,"5min":1.0029999999999999,"15min":1.0029999999999999},"last":0.69599999999999995},{"interface":"front","average":{"1min":0.67500000000000004,"5min":0.67500000000000004,"15min":0.67500000000000004},"min":{"1min":0.44600000000000001,"5min":0.44600000000000001,"15min":0.44600000000000001},"max":{"1min":1.0209999999999999,"5min":1.0209999999999999,"15min":1.0209999999999999},"last":0.74299999999999999}]},{"osd":4,"last update":"Sun Nov 26 07:18:45 2023","interfaces":[{"interface":"back","average":{"1min":0.63500000000000001,"5min":0.63500000000000001,"15min":0.63500000000000001},"min":{"1min":0.39200000000000002,"5min":0.39200000000000002,"15min":0.39200000000000002},"max":{"1min":0.95099999999999996,"5min":0.95099999999999996,"15min":0.95099999999999996},"last":0.78900000000000003},{"interface":"front","average":{"1min":0.70899999999999996,"5min":0.70899999999999996,"15min":0.70899999999999996},"min":{"1min":0.41299999999999998,"5min":0.41299999999999998,"15min":0.41299999999999998},"max":{"1min":1.254,"5min":1.254,"15min":1.254},"last":0.80900000000000005}]},{"osd":6,"last update":"Sun Nov 26 07:18:45 2023","interfaces":[{"interface":"back","average":{"1min":0.67200000000000004,"5min":0.67200000000000004,"15min":0.67200000000000004},"min":{"1min":0.45600000000000002,"5min":0.45600000000000002,"15min":0.45600000000000002},"max":{"1min":0.94499999999999995,"5min":0.94499999999999995,"15min":0.94499999999999995},"last":0.83199999999999996},{"interface":"front","average":{"1min":0.72699999999999998,"5min":0.72699999999999998,"15min":0.72699999999999998},"min":{"1min":0.44,"5min":0.44,"15min":0.44},"max":{"1min":1.0649999999999999,"5min":1.0649999999999999,"15min":1.0649999999999999},"last":0.85299999999999998}]},{"osd":7,"last update":"Sun Nov 26 07:18:45 2023","interfaces":[{"interface":"back","average":{"1min":0.80400000000000005,"5min":0.80400000000000005,"15min":0.80400000000000005},"min":{"1min":0.48199999999999998,"5min":0.48199999999999998,"15min":0.48199999999999998},"max":{"1min":1.4079999999999999,"5min":1.4079999999999999,"15min":1.4079999999999999},"last":0.89400000000000002},{"interface":"front","average":{"1min":0.77500000000000002,"5min":0.77500000000000002,"15min":0.77500000000000002},"min":{"1min":0.51600000000000001,"5min":0.51600000000000001,"15min":0.51600000000000001},"max":{"1min":1.0169999999999999,"5min":1.0169999999999999,"15min":1.0169999999999999},"last":0.92400000000000004}]}]},{"osd":4,"up_from":13,"seq":55834574874,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27516,"kb_used_data":492,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344324,"statfs":{"total":96636764160,"available":96608587776,"internally_reserved":0,"allocated":503808,"data_stored":202024,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Nov 26 07:18:46 2023","interfaces":[{"interface":"back","average":{"1min":0.79400000000000004,"5min":0.79400000000000004,"15min":0.79400000000000004},"min":{"1min":0.45600000000000002,"5min":0.45600000000000002,"15min":0.45600000000000002},"max":{"1min":4.1689999999999996,"5min":4.1689999999999996,"15min":4.1689999999999996},"last":0.622},{"interface":"front","average":{"1min":1.002,"5min":1.002,"15min":1.002},"min":{"1min":0.47499999999999998,"5min":0.47499999999999998,"15min":0.47499999999999998},"max":{"1min":7.7889999999999997,"5min":7.7889999999999997,"15min":7.7889999999999997},"last":0.499}]},{"osd":1,"last update":"Sun Nov 26 07:18:46 2023","interfaces":[{"interface":"back","average":{"1min":0.64700000000000002,"5min":0.64700000000000002,"15min":0.64700000000000002},"min":{"1min":0.52300000000000002,"5min":0.52300000000000002,"15min":0.52300000000000002},"max":{"1min":0.78400000000000003,"5min":0.78400000000000003,"15min":0.78400000000000003},"last":0.55500000000000005},{"interface":"front","average":{"1min":0.81999999999999995,"5min":0.81999999999999995,"15min":0.81999999999999995},"min":{"1min":0.44500000000000001,"5min":0.44500000000000001,"15min":0.44500000000000001},"max":{"1min":4.3140000000000001,"5min":4.3140000000000001,"15min":4.3140000000000001},"last":0.65700000000000003}]},{"osd":2,"last update":"Sun Nov 26 07:18:46 2023","interfaces":[{"interface":"back","average":{"1min":0.80700000000000005,"5min":0.80700000000000005,"15min":0.80700000000000005},"min":{"1min":0.57799999999999996,"5min":0.57799999999999996,"15min":0.57799999999999996},"max":{"1min":2.7850000000000001,"5min":2.7850000000000001,"15min":2.7850000000000001},"last":0.69299999999999995},{"interface":"front","average":{"1min":1.0660000000000001,"5min":1.0660000000000001,"15min":1.0660000000000001},"min":{"1min":0.53400000000000003,"5min":0.53400000000000003,"15min":0.53400000000000003},"max":{"1min":7.6470000000000002,"5min":7.6470000000000002,"15min":7.6470000000000002},"last":0.68000000000000005}]},{"osd":3,"last update":"Sun Nov 26 07:18:46 2023","interfaces":[{"interface":"back","average":{"1min":0.72999999999999998,"5min":0.72999999999999998,"15min":0.72999999999999998},"min":{"1min":0.48599999999999999,"5min":0.48599999999999999,"15min":0.48599999999999999},"max":{"1min":1.591,"5min":1.591,"15min":1.591},"last":0.58599999999999997},{"interface":"front","average":{"1min":0.879,"5min":0.879,"15min":0.879},"min":{"1min":0.56899999999999995,"5min":0.56899999999999995,"15min":0.56899999999999995},"max":{"1min":3.452,"5min":3.452,"15min":3.452},"last":0.72899999999999998}]},{"osd":5,"last update":"Sun Nov 26 07:18:46 2023","interfaces":[{"interface":"back","average":{"1min":0.61099999999999999,"5min":0.61099999999999999,"15min":0.61099999999999999},"min":{"1min":0.43099999999999999,"5min":0.43099999999999999,"15min":0.43099999999999999},"max":{"1min":0.96199999999999997,"5min":0.96199999999999997,"15min":0.96199999999999997},"last":0.64500000000000002},{"interface":"front","average":{"1min":0.64100000000000001,"5min":0.64100000000000001,"15min":0.64100000000000001},"min":{"1min":0.41499999999999998,"5min":0.41499999999999998,"15min":0.41499999999999998},"max":{"1min":1.538,"5min":1.538,"15min":1.538},"last":0.47099999999999997}]},{"osd":6,"last update":"Sun Nov 26 07:18:46 2023","interfaces":[{"interface":"back","average":{"1min":0.79400000000000004,"5min":0.79400000000000004,"15min":0.79400000000000004},"min":{"1min":0.39400000000000002,"5min":0.39400000000000002,"15min":0.39400000000000002},"max":{"1min":2.8330000000000002,"5min":2.8330000000000002,"15min":2.8330000000000002},"last":0.79100000000000004},{"interface":"front","average":{"1min":1.0089999999999999,"5min":1.0089999999999999,"15min":1.0089999999999999},"min":{"1min":0.42899999999999999,"5min":0.42899999999999999,"15min":0.42899999999999999},"max":{"1min":7.6050000000000004,"5min":7.6050000000000004,"15min":7.6050000000000004},"last":0.60099999999999998}]},{"osd":7,"last update":"Sun Nov 26 07:18:46 2023","interfaces":[{"interface":"back","average":{"1min":0.86099999999999999,"5min":0.86099999999999999,"15min":0.86099999999999999},"min":{"1min":0.44700000000000001,"5min":0.44700000000000001,"15min":0.44700000000000001},"max":{"1min":4.3689999999999998,"5min":4.3689999999999998,"15min":4.3689999999999998},"last":0.53600000000000003},{"interface":"front","average":{"1min":0.89200000000000002,"5min":0.89200000000000002,"15min":0.89200000000000002},"min":{"1min":0.505,"5min":0.505,"15min":0.505},"max":{"1min":3.403,"5min":3.403,"15min":3.403},"last":0.76300000000000001}]}]},{"osd":3,"up_from":13,"seq":55834574875,"num_pgs":7,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27512,"kb_used_data":492,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344328,"statfs":{"total":96636764160,"available":96608591872,"internally_reserved":0,"allocated":503808,"data_stored":202024,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1592,"internal_metadata":27654600},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Nov 26 07:18:43 2023","interfaces":[{"interface":"back","average":{"1min":0.379,"5min":0.379,"15min":0.379},"min":{"1min":0.26100000000000001,"5min":0.26100000000000001,"15min":0.26100000000000001},"max":{"1min":0.58599999999999997,"5min":0.58599999999999997,"15min":0.58599999999999997},"last":0.34999999999999998},{"interface":"front","average":{"1min":0.433,"5min":0.433,"15min":0.433},"min":{"1min":0.30199999999999999,"5min":0.30199999999999999,"15min":0.30199999999999999},"max":{"1min":0.63,"5min":0.63,"15min":0.63},"last":0.503}]},{"osd":1,"last update":"Sun Nov 26 07:18:43 2023","interfaces":[{"interface":"back","average":{"1min":0.42099999999999999,"5min":0.42099999999999999,"15min":0.42099999999999999},"min":{"1min":0.312,"5min":0.312,"15min":0.312},"max":{"1min":0.752,"5min":0.752,"15min":0.752},"last":0.40300000000000002},{"interface":"front","average":{"1min":0.48399999999999999,"5min":0.48399999999999999,"15min":0.48399999999999999},"min":{"1min":0.38,"5min":0.38,"15min":0.38},"max":{"1min":0.628,"5min":0.628,"15min":0.628},"last":0.54100000000000004}]},{"osd":2,"last update":"Sun Nov 26 07:18:43 2023","interfaces":[{"interface":"back","average":{"1min":0.53300000000000003,"5min":0.53300000000000003,"15min":0.53300000000000003},"min":{"1min":0.40300000000000002,"5min":0.40300000000000002,"15min":0.40300000000000002},"max":{"1min":0.82199999999999995,"5min":0.82199999999999995,"15min":0.82199999999999995},"last":0.57399999999999995},{"interface":"front","average":{"1min":0.505,"5min":0.505,"15min":0.505},"min":{"1min":0.35999999999999999,"5min":0.35999999999999999,"15min":0.35999999999999999},"max":{"1min":0.72199999999999998,"5min":0.72199999999999998,"15min":0.72199999999999998},"last":0.46700000000000003}]},{"osd":4,"last update":"Sun Nov 26 07:18:43 2023","interfaces":[{"interface":"back","average":{"1min":0.70599999999999996,"5min":0.70599999999999996,"15min":0.70599999999999996},"min":{"1min":0.48199999999999998,"5min":0.48199999999999998,"15min":0.48199999999999998},"max":{"1min":0.96199999999999997,"5min":0.96199999999999997,"15min":0.96199999999999997},"last":0.81200000000000006},{"interface":"front","average":{"1min":0.72799999999999998,"5min":0.72799999999999998,"15min":0.72799999999999998},"min":{"1min":0.441,"5min":0.441,"15min":0.441},"max":{"1min":1.171,"5min":1.171,"15min":1.171},"last":0.74299999999999999}]},{"osd":5,"last update":"Sun Nov 26 07:18:43 2023","interfaces":[{"interface":"back","average":{"1min":0.73899999999999999,"5min":0.73899999999999999,"15min":0.73899999999999999},"min":{"1min":0.53700000000000003,"5min":0.53700000000000003,"15min":0.53700000000000003},"max":{"1min":0.94099999999999995,"5min":0.94099999999999995,"15min":0.94099999999999995},"last":0.68000000000000005},{"interface":"front","average":{"1min":0.73799999999999999,"5min":0.73799999999999999,"15min":0.73799999999999999},"min":{"1min":0.504,"5min":0.504,"15min":0.504},"max":{"1min":1.0820000000000001,"5min":1.0820000000000001,"15min":1.0820000000000001},"last":0.71599999999999997}]},{"osd":6,"last update":"Sun Nov 26 07:18:43 2023","interfaces":[{"interface":"back","average":{"1min":0.81399999999999995,"5min":0.81399999999999995,"15min":0.81399999999999995},"min":{"1min":0.55200000000000005,"5min":0.55200000000000005,"15min":0.55200000000000005},"max":{"1min":1.2729999999999999,"5min":1.2729999999999999,"15min":1.2729999999999999},"last":0.83599999999999997},{"interface":"front","average":{"1min":0.79300000000000004,"5min":0.79300000000000004,"15min":0.79300000000000004},"min":{"1min":0.47299999999999998,"5min":0.47299999999999998,"15min":0.47299999999999998},"max":{"1min":1.204,"5min":1.204,"15min":1.204},"last":0.75900000000000001}]},{"osd":7,"last update":"Sun Nov 26 07:18:43 2023","interfaces":[{"interface":"back","average":{"1min":0.80100000000000005,"5min":0.80100000000000005,"15min":0.80100000000000005},"min":{"1min":0.63300000000000001,"5min":0.63300000000000001,"15min":0.63300000000000001},"max":{"1min":1.3600000000000001,"5min":1.3600000000000001,"15min":1.3600000000000001},"last":0.63900000000000001},{"interface":"front","average":{"1min":0.82599999999999996,"5min":0.82599999999999996,"15min":0.82599999999999996},"min":{"1min":0.50600000000000001,"5min":0.50600000000000001,"15min":0.50600000000000001},"max":{"1min":1.617,"5min":1.617,"15min":1.617},"last":0.79300000000000004}]}]},{"osd":2,"up_from":13,"seq":55834574875,"num_pgs":11,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27512,"kb_used_data":492,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344328,"statfs":{"total":96636764160,"available":96608591872,"internally_reserved":0,"allocated":503808,"data_stored":202024,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Nov 26 07:18:43 2023","interfaces":[{"interface":"back","average":{"1min":0.71399999999999997,"5min":0.71399999999999997,"15min":0.71399999999999997},"min":{"1min":0.28699999999999998,"5min":0.28699999999999998,"15min":0.28699999999999998},"max":{"1min":4.7030000000000003,"5min":4.7030000000000003,"15min":4.7030000000000003},"last":0.41099999999999998},{"interface":"front","average":{"1min":0.56200000000000006,"5min":0.56200000000000006,"15min":0.56200000000000006},"min":{"1min":0.32300000000000001,"5min":0.32300000000000001,"15min":0.32300000000000001},"max":{"1min":2.4340000000000002,"5min":2.4340000000000002,"15min":2.4340000000000002},"last":0.438}]},{"osd":1,"last update":"Sun Nov 26 07:18:43 2023","interfaces":[{"interface":"back","average":{"1min":0.86399999999999999,"5min":0.86399999999999999,"15min":0.86399999999999999},"min":{"1min":0.34599999999999997,"5min":0.34599999999999997,"15min":0.34599999999999997},"max":{"1min":4.2690000000000001,"5min":4.2690000000000001,"15min":4.2690000000000001},"last":0.46300000000000002},{"interface":"front","average":{"1min":0.74199999999999999,"5min":0.74199999999999999,"15min":0.74199999999999999},"min":{"1min":0.31900000000000001,"5min":0.31900000000000001,"15min":0.31900000000000001},"max":{"1min":3.9940000000000002,"5min":3.9940000000000002,"15min":3.9940000000000002},"last":0.38800000000000001}]},{"osd":3,"last update":"Sun Nov 26 07:18:43 2023","interfaces":[{"interface":"back","average":{"1min":0.64300000000000002,"5min":0.64300000000000002,"15min":0.64300000000000002},"min":{"1min":0.34399999999999997,"5min":0.34399999999999997,"15min":0.34399999999999997},"max":{"1min":4.0300000000000002,"5min":4.0300000000000002,"15min":4.0300000000000002},"last":0.36899999999999999},{"interface":"front","average":{"1min":0.91400000000000003,"5min":0.91400000000000003,"15min":0.91400000000000003},"min":{"1min":0.41699999999999998,"5min":0.41699999999999998,"15min":0.41699999999999998},"max":{"1min":4.5810000000000004,"5min":4.5810000000000004,"15min":4.5810000000000004},"last":0.50900000000000001}]},{"osd":4,"last update":"Sun Nov 26 07:18:43 2023","interfaces":[{"interface":"back","average":{"1min":0.98499999999999999,"5min":0.98499999999999999,"15min":0.98499999999999999},"min":{"1min":0.55100000000000005,"5min":0.55100000000000005,"15min":0.55100000000000005},"max":{"1min":4.4269999999999996,"5min":4.4269999999999996,"15min":4.4269999999999996},"last":0.80900000000000005},{"interface":"front","average":{"1min":1.0149999999999999,"5min":1.0149999999999999,"15min":1.0149999999999999},"min":{"1min":0.60299999999999998,"5min":0.60299999999999998,"15min":0.60299999999999998},"max":{"1min":4.085,"5min":4.085,"15min":4.085},"last":0.68100000000000005}]},{"osd":5,"last update":"Sun Nov 26 07:18:43 2023","interfaces":[{"interface":"back","average":{"1min":0.874,"5min":0.874,"15min":0.874},"min":{"1min":0.52900000000000003,"5min":0.52900000000000003,"15min":0.52900000000000003},"max":{"1min":3.9590000000000001,"5min":3.9590000000000001,"15min":3.9590000000000001},"last":0.65000000000000002},{"interface":"front","average":{"1min":0.98799999999999999,"5min":0.98799999999999999,"15min":0.98799999999999999},"min":{"1min":0.58799999999999997,"5min":0.58799999999999997,"15min":0.58799999999999997},"max":{"1min":4.3849999999999998,"5min":4.3849999999999998,"15min":4.3849999999999998},"last":0.752}]},{"osd":6,"last update":"Sun Nov 26 07:18:43 2023","interfaces":[{"interface":"back","average":{"1min":0.996,"5min":0.996,"15min":0.996},"min":{"1min":0.56599999999999995,"5min":0.56599999999999995,"15min":0.56599999999999995},"max":{"1min":4.0449999999999999,"5min":4.0449999999999999,"15min":4.0449999999999999},"last":0.77700000000000002},{"interface":"front","average":{"1min":0.91800000000000004,"5min":0.91800000000000004,"15min":0.91800000000000004},"min":{"1min":0.57099999999999995,"5min":0.57099999999999995,"15min":0.57099999999999995},"max":{"1min":3.992,"5min":3.992,"15min":3.992},"last":0.60699999999999998}]},{"osd":7,"last update":"Sun Nov 26 07:18:43 2023","interfaces":[{"interface":"back","average":{"1min":1.0349999999999999,"5min":1.0349999999999999,"15min":1.0349999999999999},"min":{"1min":0.63100000000000001,"5min":0.63100000000000001,"15min":0.63100000000000001},"max":{"1min":4.1219999999999999,"5min":4.1219999999999999,"15min":4.1219999999999999},"last":0.63},{"interface":"front","average":{"1min":0.93799999999999994,"5min":0.93799999999999994,"15min":0.93799999999999994},"min":{"1min":0.57699999999999996,"5min":0.57699999999999996,"15min":0.57699999999999996},"max":{"1min":3.9159999999999999,"5min":3.9159999999999999,"15min":3.9159999999999999},"last":0.73599999999999999}]}]},{"osd":1,"up_from":13,"seq":55834574874,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27512,"kb_used_data":492,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344328,"statfs":{"total":96636764160,"available":96608591872,"internally_reserved":0,"allocated":503808,"data_stored":202024,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Nov 26 07:18:41 2023","interfaces":[{"interface":"back","average":{"1min":0.41699999999999998,"5min":0.41699999999999998,"15min":0.41699999999999998},"min":{"1min":0.26300000000000001,"5min":0.26300000000000001,"15min":0.26300000000000001},"max":{"1min":0.61199999999999999,"5min":0.61199999999999999,"15min":0.61199999999999999},"last":0.36299999999999999},{"interface":"front","average":{"1min":0.48599999999999999,"5min":0.48599999999999999,"15min":0.48599999999999999},"min":{"1min":0.30499999999999999,"5min":0.30499999999999999,"15min":0.30499999999999999},"max":{"1min":0.78700000000000003,"5min":0.78700000000000003,"15min":0.78700000000000003},"last":0.433}]},{"osd":2,"last update":"Sun Nov 26 07:18:41 2023","interfaces":[{"interface":"back","average":{"1min":0.45500000000000002,"5min":0.45500000000000002,"15min":0.45500000000000002},"min":{"1min":0.33400000000000002,"5min":0.33400000000000002,"15min":0.33400000000000002},"max":{"1min":0.67600000000000005,"5min":0.67600000000000005,"15min":0.67600000000000005},"last":0.47099999999999997},{"interface":"front","average":{"1min":0.50800000000000001,"5min":0.50800000000000001,"15min":0.50800000000000001},"min":{"1min":0.32300000000000001,"5min":0.32300000000000001,"15min":0.32300000000000001},"max":{"1min":0.66500000000000004,"5min":0.66500000000000004,"15min":0.66500000000000004},"last":0.64400000000000002}]},{"osd":3,"last update":"Sun Nov 26 07:18:41 2023","interfaces":[{"interface":"back","average":{"1min":0.53000000000000003,"5min":0.53000000000000003,"15min":0.53000000000000003},"min":{"1min":0.38400000000000001,"5min":0.38400000000000001,"15min":0.38400000000000001},"max":{"1min":0.73699999999999999,"5min":0.73699999999999999,"15min":0.73699999999999999},"last":0.495},{"interface":"front","average":{"1min":0.497,"5min":0.497,"15min":0.497},"min":{"1min":0.36199999999999999,"5min":0.36199999999999999,"15min":0.36199999999999999},"max":{"1min":0.80700000000000005,"5min":0.80700000000000005,"15min":0.80700000000000005},"last":0.51500000000000001}]},{"osd":4,"last update":"Sun Nov 26 07:18:41 2023","interfaces":[{"interface":"back","average":{"1min":0.69399999999999995,"5min":0.69399999999999995,"15min":0.69399999999999995},"min":{"1min":0.48599999999999999,"5min":0.48599999999999999,"15min":0.48599999999999999},"max":{"1min":1.0229999999999999,"5min":1.0229999999999999,"15min":1.0229999999999999},"last":0.56599999999999995},{"interface":"front","average":{"1min":0.68000000000000005,"5min":0.68000000000000005,"15min":0.68000000000000005},"min":{"1min":0.49199999999999999,"5min":0.49199999999999999,"15min":0.49199999999999999},"max":{"1min":1.0629999999999999,"5min":1.0629999999999999,"15min":1.0629999999999999},"last":0.71399999999999997}]},{"osd":5,"last update":"Sun Nov 26 07:18:41 2023","interfaces":[{"interface":"back","average":{"1min":0.71899999999999997,"5min":0.71899999999999997,"15min":0.71899999999999997},"min":{"1min":0.49299999999999999,"5min":0.49299999999999999,"15min":0.49299999999999999},"max":{"1min":1.0880000000000001,"5min":1.0880000000000001,"15min":1.0880000000000001},"last":0.625},{"interface":"front","average":{"1min":0.71699999999999997,"5min":0.71699999999999997,"15min":0.71699999999999997},"min":{"1min":0.53500000000000003,"5min":0.53500000000000003,"15min":0.53500000000000003},"max":{"1min":1.014,"5min":1.014,"15min":1.014},"last":0.70099999999999996}]},{"osd":6,"last update":"Sun Nov 26 07:18:41 2023","interfaces":[{"interface":"back","average":{"1min":0.70399999999999996,"5min":0.70399999999999996,"15min":0.70399999999999996},"min":{"1min":0.50600000000000001,"5min":0.50600000000000001,"15min":0.50600000000000001},"max":{"1min":0.97599999999999998,"5min":0.97599999999999998,"15min":0.97599999999999998},"last":0.72899999999999998},{"interface":"front","average":{"1min":0.70099999999999996,"5min":0.70099999999999996,"15min":0.70099999999999996},"min":{"1min":0.55000000000000004,"5min":0.55000000000000004,"15min":0.55000000000000004},"max":{"1min":1.1339999999999999,"5min":1.1339999999999999,"15min":1.1339999999999999},"last":0.58699999999999997}]},{"osd":7,"last update":"Sun Nov 26 07:18:41 2023","interfaces":[{"interface":"back","average":{"1min":0.71499999999999997,"5min":0.71499999999999997,"15min":0.71499999999999997},"min":{"1min":0.55100000000000005,"5min":0.55100000000000005,"15min":0.55100000000000005},"max":{"1min":0.92900000000000005,"5min":0.92900000000000005,"15min":0.92900000000000005},"last":0.74099999999999999},{"interface":"front","average":{"1min":0.753,"5min":0.753,"15min":0.753},"min":{"1min":0.56100000000000005,"5min":0.56100000000000005,"15min":0.56100000000000005},"max":{"1min":1.024,"5min":1.024,"15min":1.024},"last":0.68200000000000005}]}]},{"osd":0,"up_from":13,"seq":55834574874,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":28080,"kb_used_data":1060,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343760,"statfs":{"total":96636764160,"available":96608010240,"internally_reserved":0,"allocated":1085440,"data_stored":786125,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Sun Nov 26 07:18:41 2023","interfaces":[{"interface":"back","average":{"1min":0.43099999999999999,"5min":0.43099999999999999,"15min":0.43099999999999999},"min":{"1min":0.28000000000000003,"5min":0.28000000000000003,"15min":0.28000000000000003},"max":{"1min":0.63100000000000001,"5min":0.63100000000000001,"15min":0.63100000000000001},"last":0.40799999999999997},{"interface":"front","average":{"1min":0.48299999999999998,"5min":0.48299999999999998,"15min":0.48299999999999998},"min":{"1min":0.29899999999999999,"5min":0.29899999999999999,"15min":0.29899999999999999},"max":{"1min":1.133,"5min":1.133,"15min":1.133},"last":0.35399999999999998}]},{"osd":2,"last update":"Sun Nov 26 07:18:41 2023","interfaces":[{"interface":"back","average":{"1min":0.53700000000000003,"5min":0.53700000000000003,"15min":0.53700000000000003},"min":{"1min":0.40899999999999997,"5min":0.40899999999999997,"15min":0.40899999999999997},"max":{"1min":0.67400000000000004,"5min":0.67400000000000004,"15min":0.67400000000000004},"last":0.56299999999999994},{"interface":"front","average":{"1min":0.56399999999999995,"5min":0.56399999999999995,"15min":0.56399999999999995},"min":{"1min":0.36399999999999999,"5min":0.36399999999999999,"15min":0.36399999999999999},"max":{"1min":0.72499999999999998,"5min":0.72499999999999998,"15min":0.72499999999999998},"last":0.48599999999999999}]},{"osd":3,"last update":"Sun Nov 26 07:18:41 2023","interfaces":[{"interface":"back","average":{"1min":0.53100000000000003,"5min":0.53100000000000003,"15min":0.53100000000000003},"min":{"1min":0.35799999999999998,"5min":0.35799999999999998,"15min":0.35799999999999998},"max":{"1min":0.97399999999999998,"5min":0.97399999999999998,"15min":0.97399999999999998},"last":1.3149999999999999},{"interface":"front","average":{"1min":0.56200000000000006,"5min":0.56200000000000006,"15min":0.56200000000000006},"min":{"1min":0.40600000000000003,"5min":0.40600000000000003,"15min":0.40600000000000003},"max":{"1min":1.2509999999999999,"5min":1.2509999999999999,"15min":1.2509999999999999},"last":0.65100000000000002}]},{"osd":4,"last update":"Sun Nov 26 07:18:41 2023","interfaces":[{"interface":"back","average":{"1min":0.75900000000000001,"5min":0.75900000000000001,"15min":0.75900000000000001},"min":{"1min":0.60999999999999999,"5min":0.60999999999999999,"15min":0.60999999999999999},"max":{"1min":1.2010000000000001,"5min":1.2010000000000001,"15min":1.2010000000000001},"last":0.60999999999999999},{"interface":"front","average":{"1min":0.78900000000000003,"5min":0.78900000000000003,"15min":0.78900000000000003},"min":{"1min":0.59999999999999998,"5min":0.59999999999999998,"15min":0.59999999999999998},"max":{"1min":0.99399999999999999,"5min":0.99399999999999999,"15min":0.99399999999999999},"last":1.381}]},{"osd":5,"last update":"Sun Nov 26 07:18:41 2023","interfaces":[{"interface":"back","average":{"1min":0.76000000000000001,"5min":0.76000000000000001,"15min":0.76000000000000001},"min":{"1min":0.53000000000000003,"5min":0.53000000000000003,"15min":0.53000000000000003},"max":{"1min":1.1579999999999999,"5min":1.1579999999999999,"15min":1.1579999999999999},"last":0.72699999999999998},{"interface":"front","average":{"1min":0.79700000000000004,"5min":0.79700000000000004,"15min":0.79700000000000004},"min":{"1min":0.59699999999999998,"5min":0.59699999999999998,"15min":0.59699999999999998},"max":{"1min":1.0409999999999999,"5min":1.0409999999999999,"15min":1.0409999999999999},"last":1.399}]},{"osd":6,"last update":"Sun Nov 26 07:18:41 2023","interfaces":[{"interface":"back","average":{"1min":0.76400000000000001,"5min":0.76400000000000001,"15min":0.76400000000000001},"min":{"1min":0.56599999999999995,"5min":0.56599999999999995,"15min":0.56599999999999995},"max":{"1min":1.3029999999999999,"5min":1.3029999999999999,"15min":1.3029999999999999},"last":0.69599999999999995},{"interface":"front","average":{"1min":0.76900000000000002,"5min":0.76900000000000002,"15min":0.76900000000000002},"min":{"1min":0.58599999999999997,"5min":0.58599999999999997,"15min":0.58599999999999997},"max":{"1min":1.2210000000000001,"5min":1.2210000000000001,"15min":1.2210000000000001},"last":1.3600000000000001}]},{"osd":7,"last update":"Sun Nov 26 07:18:41 2023","interfaces":[{"interface":"back","average":{"1min":0.81100000000000005,"5min":0.81100000000000005,"15min":0.81100000000000005},"min":{"1min":0.59999999999999998,"5min":0.59999999999999998,"15min":0.59999999999999998},"max":{"1min":1.177,"5min":1.177,"15min":1.177},"last":0.84099999999999997},{"interface":"front","average":{"1min":0.79900000000000004,"5min":0.79900000000000004,"15min":0.79900000000000004},"min":{"1min":0.60899999999999999,"5min":0.60899999999999999,"15min":0.60899999999999999},"max":{"1min":1.2390000000000001,"5min":1.2390000000000001,"15min":1.2390000000000001},"last":0.67300000000000004}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-11-26T07:19:37.255 INFO:tasks.cephfs.filesystem.ceph_manager:PG 7.3f is not active+clean 2023-11-26T07:19:37.255 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '7.3f', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 43, 'state': 'creating+peering', 'last_fresh': '2023-11-26T07:19:31.135164+0000', 'last_change': '2023-11-26T07:19:31.135164+0000', 'last_active': '2023-11-26T07:19:31.113498+0000', 'last_peered': '2023-11-26T07:19:31.113498+0000', 'last_clean': '2023-11-26T07:19:31.113498+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-26T07:19:31.135164+0000', 'last_undegraded': '2023-11-26T07:19:31.135164+0000', 'last_fullsized': '2023-11-26T07:19:31.135164+0000', 'mapping_epoch': 43, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 43, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-26T07:19:31.113498+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-26T07:19:31.113498+0000', 'last_clean_scrub_stamp': '2023-11-26T07:19:31.113498+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [7, 4], 'acting': [7, 4], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 7, 'acting_primary': 7, 'purged_snaps': []} 2023-11-26T07:19:37.255 INFO:tasks.cephfs.filesystem.ceph_manager:PG 7.3e is not active+clean 2023-11-26T07:19:37.255 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '7.3e', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 43, 'state': 'creating+peering', 'last_fresh': '2023-11-26T07:19:31.129025+0000', 'last_change': '2023-11-26T07:19:31.129025+0000', 'last_active': '2023-11-26T07:19:31.113498+0000', 'last_peered': '2023-11-26T07:19:31.113498+0000', 'last_clean': '2023-11-26T07:19:31.113498+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-26T07:19:31.129025+0000', 'last_undegraded': '2023-11-26T07:19:31.129025+0000', 'last_fullsized': '2023-11-26T07:19:31.129025+0000', 'mapping_epoch': 43, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 43, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-26T07:19:31.113498+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-26T07:19:31.113498+0000', 'last_clean_scrub_stamp': '2023-11-26T07:19:31.113498+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [5, 0], 'acting': [5, 0], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 5, 'acting_primary': 5, 'purged_snaps': []} 2023-11-26T07:19:37.255 INFO:tasks.cephfs.filesystem.ceph_manager:PG 7.3d is not active+clean 2023-11-26T07:19:37.256 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '7.3d', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 43, 'state': 'creating+peering', 'last_fresh': '2023-11-26T07:19:31.125533+0000', 'last_change': '2023-11-26T07:19:31.125533+0000', 'last_active': '2023-11-26T07:19:31.113498+0000', 'last_peered': '2023-11-26T07:19:31.113498+0000', 'last_clean': '2023-11-26T07:19:31.113498+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-26T07:19:31.125533+0000', 'last_undegraded': '2023-11-26T07:19:31.125533+0000', 'last_fullsized': '2023-11-26T07:19:31.125533+0000', 'mapping_epoch': 43, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 43, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-26T07:19:31.113498+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-26T07:19:31.113498+0000', 'last_clean_scrub_stamp': '2023-11-26T07:19:31.113498+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [4, 2], 'acting': [4, 2], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 4, 'acting_primary': 4, 'purged_snaps': []} 2023-11-26T07:19:37.256 INFO:tasks.cephfs.filesystem.ceph_manager:PG 7.3c is not active+clean 2023-11-26T07:19:37.256 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '7.3c', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 43, 'state': 'creating+peering', 'last_fresh': '2023-11-26T07:19:31.129825+0000', 'last_change': '2023-11-26T07:19:31.129825+0000', 'last_active': '2023-11-26T07:19:31.113498+0000', 'last_peered': '2023-11-26T07:19:31.113498+0000', 'last_clean': '2023-11-26T07:19:31.113498+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-26T07:19:31.129825+0000', 'last_undegraded': '2023-11-26T07:19:31.129825+0000', 'last_fullsized': '2023-11-26T07:19:31.129825+0000', 'mapping_epoch': 43, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 43, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-26T07:19:31.113498+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-26T07:19:31.113498+0000', 'last_clean_scrub_stamp': '2023-11-26T07:19:31.113498+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 6], 'acting': [0, 6], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2023-11-26T07:19:37.256 INFO:tasks.cephfs.filesystem.ceph_manager:PG 7.3b is not active+clean 2023-11-26T07:19:37.256 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '7.3b', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 43, 'state': 'creating+peering', 'last_fresh': '2023-11-26T07:19:31.128964+0000', 'last_change': '2023-11-26T07:19:31.128964+0000', 'last_active': '2023-11-26T07:19:31.113498+0000', 'last_peered': '2023-11-26T07:19:31.113498+0000', 'last_clean': '2023-11-26T07:19:31.113498+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-26T07:19:31.128964+0000', 'last_undegraded': '2023-11-26T07:19:31.128964+0000', 'last_fullsized': '2023-11-26T07:19:31.128964+0000', 'mapping_epoch': 43, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 43, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-26T07:19:31.113498+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-26T07:19:31.113498+0000', 'last_clean_scrub_stamp': '2023-11-26T07:19:31.113498+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [5, 0], 'acting': [5, 0], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 5, 'acting_primary': 5, 'purged_snaps': []} 2023-11-26T07:19:37.256 INFO:tasks.cephfs.filesystem.ceph_manager:PG 7.3a is not active+clean 2023-11-26T07:19:37.256 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '7.3a', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 43, 'state': 'creating+peering', 'last_fresh': '2023-11-26T07:19:31.138400+0000', 'last_change': '2023-11-26T07:19:31.138400+0000', 'last_active': '2023-11-26T07:19:31.113498+0000', 'last_peered': '2023-11-26T07:19:31.113498+0000', 'last_clean': '2023-11-26T07:19:31.113498+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-26T07:19:31.138400+0000', 'last_undegraded': '2023-11-26T07:19:31.138400+0000', 'last_fullsized': '2023-11-26T07:19:31.138400+0000', 'mapping_epoch': 43, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 43, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-26T07:19:31.113498+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-26T07:19:31.113498+0000', 'last_clean_scrub_stamp': '2023-11-26T07:19:31.113498+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 7], 'acting': [3, 7], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2023-11-26T07:19:37.256 INFO:tasks.cephfs.filesystem.ceph_manager:PG 7.39 is not active+clean 2023-11-26T07:19:37.256 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '7.39', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 43, 'state': 'creating+peering', 'last_fresh': '2023-11-26T07:19:31.134099+0000', 'last_change': '2023-11-26T07:19:31.134099+0000', 'last_active': '2023-11-26T07:19:31.113498+0000', 'last_peered': '2023-11-26T07:19:31.113498+0000', 'last_clean': '2023-11-26T07:19:31.113498+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-26T07:19:31.134099+0000', 'last_undegraded': '2023-11-26T07:19:31.134099+0000', 'last_fullsized': '2023-11-26T07:19:31.134099+0000', 'mapping_epoch': 43, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 43, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-26T07:19:31.113498+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-26T07:19:31.113498+0000', 'last_clean_scrub_stamp': '2023-11-26T07:19:31.113498+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [7, 1], 'acting': [7, 1], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 7, 'acting_primary': 7, 'purged_snaps': []} 2023-11-26T07:19:37.256 INFO:tasks.cephfs.filesystem.ceph_manager:PG 7.38 is not active+clean 2023-11-26T07:19:37.257 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '7.38', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 43, 'state': 'creating+peering', 'last_fresh': '2023-11-26T07:19:31.130100+0000', 'last_change': '2023-11-26T07:19:31.130100+0000', 'last_active': '2023-11-26T07:19:31.113498+0000', 'last_peered': '2023-11-26T07:19:31.113498+0000', 'last_clean': '2023-11-26T07:19:31.113498+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-26T07:19:31.130100+0000', 'last_undegraded': '2023-11-26T07:19:31.130100+0000', 'last_fullsized': '2023-11-26T07:19:31.130100+0000', 'mapping_epoch': 43, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 43, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-26T07:19:31.113498+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-26T07:19:31.113498+0000', 'last_clean_scrub_stamp': '2023-11-26T07:19:31.113498+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 6], 'acting': [0, 6], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2023-11-26T07:19:37.257 INFO:tasks.cephfs.filesystem.ceph_manager:PG 7.37 is not active+clean 2023-11-26T07:19:37.257 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '7.37', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 43, 'state': 'creating+peering', 'last_fresh': '2023-11-26T07:19:31.129317+0000', 'last_change': '2023-11-26T07:19:31.129317+0000', 'last_active': '2023-11-26T07:19:31.113498+0000', 'last_peered': '2023-11-26T07:19:31.113498+0000', 'last_clean': '2023-11-26T07:19:31.113498+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-26T07:19:31.129317+0000', 'last_undegraded': '2023-11-26T07:19:31.129317+0000', 'last_fullsized': '2023-11-26T07:19:31.129317+0000', 'mapping_epoch': 43, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 43, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-26T07:19:31.113498+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-26T07:19:31.113498+0000', 'last_clean_scrub_stamp': '2023-11-26T07:19:31.113498+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0, 1], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2023-11-26T07:19:37.257 INFO:tasks.cephfs.filesystem.ceph_manager:PG 7.36 is not active+clean 2023-11-26T07:19:37.257 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '7.36', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 43, 'state': 'creating+peering', 'last_fresh': '2023-11-26T07:19:31.130104+0000', 'last_change': '2023-11-26T07:19:31.130104+0000', 'last_active': '2023-11-26T07:19:31.113498+0000', 'last_peered': '2023-11-26T07:19:31.113498+0000', 'last_clean': '2023-11-26T07:19:31.113498+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-26T07:19:31.130104+0000', 'last_undegraded': '2023-11-26T07:19:31.130104+0000', 'last_fullsized': '2023-11-26T07:19:31.130104+0000', 'mapping_epoch': 43, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 43, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-26T07:19:31.113498+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-26T07:19:31.113498+0000', 'last_clean_scrub_stamp': '2023-11-26T07:19:31.113498+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 6], 'acting': [0, 6], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2023-11-26T07:19:37.257 INFO:tasks.cephfs.filesystem.ceph_manager:PG 7.35 is not active+clean 2023-11-26T07:19:37.257 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '7.35', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 43, 'state': 'creating+peering', 'last_fresh': '2023-11-26T07:19:31.130252+0000', 'last_change': '2023-11-26T07:19:31.130252+0000', 'last_active': '2023-11-26T07:19:31.113498+0000', 'last_peered': '2023-11-26T07:19:31.113498+0000', 'last_clean': '2023-11-26T07:19:31.113498+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-26T07:19:31.130252+0000', 'last_undegraded': '2023-11-26T07:19:31.130252+0000', 'last_fullsized': '2023-11-26T07:19:31.130252+0000', 'mapping_epoch': 43, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 43, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-26T07:19:31.113498+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-26T07:19:31.113498+0000', 'last_clean_scrub_stamp': '2023-11-26T07:19:31.113498+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0, 1], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2023-11-26T07:19:37.257 INFO:tasks.cephfs.filesystem.ceph_manager:PG 7.34 is not active+clean 2023-11-26T07:19:37.257 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '7.34', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 43, 'state': 'creating+peering', 'last_fresh': '2023-11-26T07:19:31.131089+0000', 'last_change': '2023-11-26T07:19:31.131089+0000', 'last_active': '2023-11-26T07:19:31.113498+0000', 'last_peered': '2023-11-26T07:19:31.113498+0000', 'last_clean': '2023-11-26T07:19:31.113498+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-26T07:19:31.131089+0000', 'last_undegraded': '2023-11-26T07:19:31.131089+0000', 'last_fullsized': '2023-11-26T07:19:31.131089+0000', 'mapping_epoch': 43, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 43, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-26T07:19:31.113498+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-26T07:19:31.113498+0000', 'last_clean_scrub_stamp': '2023-11-26T07:19:31.113498+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [5, 2], 'acting': [5, 2], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 5, 'acting_primary': 5, 'purged_snaps': []} 2023-11-26T07:19:37.257 INFO:tasks.cephfs.filesystem.ceph_manager:PG 7.33 is not active+clean 2023-11-26T07:19:37.258 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '7.33', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 43, 'state': 'creating+peering', 'last_fresh': '2023-11-26T07:19:31.132990+0000', 'last_change': '2023-11-26T07:19:31.132990+0000', 'last_active': '2023-11-26T07:19:31.113498+0000', 'last_peered': '2023-11-26T07:19:31.113498+0000', 'last_clean': '2023-11-26T07:19:31.113498+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-26T07:19:31.132990+0000', 'last_undegraded': '2023-11-26T07:19:31.132990+0000', 'last_fullsized': '2023-11-26T07:19:31.132990+0000', 'mapping_epoch': 43, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 43, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-26T07:19:31.113498+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-26T07:19:31.113498+0000', 'last_clean_scrub_stamp': '2023-11-26T07:19:31.113498+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 3], 'acting': [2, 3], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2023-11-26T07:19:37.258 INFO:tasks.cephfs.filesystem.ceph_manager:PG 7.32 is not active+clean 2023-11-26T07:19:37.258 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '7.32', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 43, 'state': 'creating+peering', 'last_fresh': '2023-11-26T07:19:31.129881+0000', 'last_change': '2023-11-26T07:19:31.129881+0000', 'last_active': '2023-11-26T07:19:31.113498+0000', 'last_peered': '2023-11-26T07:19:31.113498+0000', 'last_clean': '2023-11-26T07:19:31.113498+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-26T07:19:31.129881+0000', 'last_undegraded': '2023-11-26T07:19:31.129881+0000', 'last_fullsized': '2023-11-26T07:19:31.129881+0000', 'mapping_epoch': 43, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 43, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-26T07:19:31.113498+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-26T07:19:31.113498+0000', 'last_clean_scrub_stamp': '2023-11-26T07:19:31.113498+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0, 1], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2023-11-26T07:19:37.258 INFO:tasks.cephfs.filesystem.ceph_manager:PG 7.31 is not active+clean 2023-11-26T07:19:37.258 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '7.31', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 43, 'state': 'creating+peering', 'last_fresh': '2023-11-26T07:19:31.138475+0000', 'last_change': '2023-11-26T07:19:31.138475+0000', 'last_active': '2023-11-26T07:19:31.113498+0000', 'last_peered': '2023-11-26T07:19:31.113498+0000', 'last_clean': '2023-11-26T07:19:31.113498+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-26T07:19:31.138475+0000', 'last_undegraded': '2023-11-26T07:19:31.138475+0000', 'last_fullsized': '2023-11-26T07:19:31.138475+0000', 'mapping_epoch': 43, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 43, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-26T07:19:31.113498+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-26T07:19:31.113498+0000', 'last_clean_scrub_stamp': '2023-11-26T07:19:31.113498+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 4], 'acting': [3, 4], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2023-11-26T07:19:37.258 INFO:tasks.cephfs.filesystem.ceph_manager:PG 7.30 is not active+clean 2023-11-26T07:19:37.258 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '7.30', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 43, 'state': 'creating+peering', 'last_fresh': '2023-11-26T07:19:31.131093+0000', 'last_change': '2023-11-26T07:19:31.131093+0000', 'last_active': '2023-11-26T07:19:31.113498+0000', 'last_peered': '2023-11-26T07:19:31.113498+0000', 'last_clean': '2023-11-26T07:19:31.113498+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-26T07:19:31.131093+0000', 'last_undegraded': '2023-11-26T07:19:31.131093+0000', 'last_fullsized': '2023-11-26T07:19:31.131093+0000', 'mapping_epoch': 43, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 43, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-26T07:19:31.113498+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-26T07:19:31.113498+0000', 'last_clean_scrub_stamp': '2023-11-26T07:19:31.113498+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [6, 0], 'acting': [6, 0], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 6, 'acting_primary': 6, 'purged_snaps': []} 2023-11-26T07:19:37.258 INFO:tasks.cephfs.filesystem.ceph_manager:PG 7.2f is not active+clean 2023-11-26T07:19:37.258 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '7.2f', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 43, 'state': 'creating+peering', 'last_fresh': '2023-11-26T07:19:31.125773+0000', 'last_change': '2023-11-26T07:19:31.125773+0000', 'last_active': '2023-11-26T07:19:31.113498+0000', 'last_peered': '2023-11-26T07:19:31.113498+0000', 'last_clean': '2023-11-26T07:19:31.113498+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-26T07:19:31.125773+0000', 'last_undegraded': '2023-11-26T07:19:31.125773+0000', 'last_fullsized': '2023-11-26T07:19:31.125773+0000', 'mapping_epoch': 43, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 43, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-26T07:19:31.113498+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-26T07:19:31.113498+0000', 'last_clean_scrub_stamp': '2023-11-26T07:19:31.113498+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [4, 5], 'acting': [4, 5], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 4, 'acting_primary': 4, 'purged_snaps': []} 2023-11-26T07:19:37.259 INFO:tasks.cephfs.filesystem.ceph_manager:PG 7.2e is not active+clean 2023-11-26T07:19:37.259 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '7.2e', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 43, 'state': 'creating+peering', 'last_fresh': '2023-11-26T07:19:31.126790+0000', 'last_change': '2023-11-26T07:19:31.126790+0000', 'last_active': '2023-11-26T07:19:31.113498+0000', 'last_peered': '2023-11-26T07:19:31.113498+0000', 'last_clean': '2023-11-26T07:19:31.113498+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-26T07:19:31.126790+0000', 'last_undegraded': '2023-11-26T07:19:31.126790+0000', 'last_fullsized': '2023-11-26T07:19:31.126790+0000', 'mapping_epoch': 43, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 43, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-26T07:19:31.113498+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-26T07:19:31.113498+0000', 'last_clean_scrub_stamp': '2023-11-26T07:19:31.113498+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [4, 0], 'acting': [4, 0], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 4, 'acting_primary': 4, 'purged_snaps': []} 2023-11-26T07:19:37.259 INFO:tasks.cephfs.filesystem.ceph_manager:PG 7.2d is not active+clean 2023-11-26T07:19:37.259 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '7.2d', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 43, 'state': 'creating+peering', 'last_fresh': '2023-11-26T07:19:31.133954+0000', 'last_change': '2023-11-26T07:19:31.133954+0000', 'last_active': '2023-11-26T07:19:31.113498+0000', 'last_peered': '2023-11-26T07:19:31.113498+0000', 'last_clean': '2023-11-26T07:19:31.113498+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-26T07:19:31.133954+0000', 'last_undegraded': '2023-11-26T07:19:31.133954+0000', 'last_fullsized': '2023-11-26T07:19:31.133954+0000', 'mapping_epoch': 43, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 43, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-26T07:19:31.113498+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-26T07:19:31.113498+0000', 'last_clean_scrub_stamp': '2023-11-26T07:19:31.113498+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [6, 0], 'acting': [6, 0], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 6, 'acting_primary': 6, 'purged_snaps': []} 2023-11-26T07:19:37.259 INFO:tasks.cephfs.filesystem.ceph_manager:PG 7.2c is not active+clean 2023-11-26T07:19:37.259 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '7.2c', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 43, 'state': 'creating+peering', 'last_fresh': '2023-11-26T07:19:31.135122+0000', 'last_change': '2023-11-26T07:19:31.135122+0000', 'last_active': '2023-11-26T07:19:31.113498+0000', 'last_peered': '2023-11-26T07:19:31.113498+0000', 'last_clean': '2023-11-26T07:19:31.113498+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-26T07:19:31.135122+0000', 'last_undegraded': '2023-11-26T07:19:31.135122+0000', 'last_fullsized': '2023-11-26T07:19:31.135122+0000', 'mapping_epoch': 43, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 43, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-26T07:19:31.113498+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-26T07:19:31.113498+0000', 'last_clean_scrub_stamp': '2023-11-26T07:19:31.113498+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 0], 'acting': [2, 0], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2023-11-26T07:19:37.259 INFO:tasks.cephfs.filesystem.ceph_manager:PG 7.2b is not active+clean 2023-11-26T07:19:37.259 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '7.2b', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 43, 'state': 'creating+peering', 'last_fresh': '2023-11-26T07:19:31.137790+0000', 'last_change': '2023-11-26T07:19:31.137790+0000', 'last_active': '2023-11-26T07:19:31.113498+0000', 'last_peered': '2023-11-26T07:19:31.113498+0000', 'last_clean': '2023-11-26T07:19:31.113498+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-26T07:19:31.137790+0000', 'last_undegraded': '2023-11-26T07:19:31.137790+0000', 'last_fullsized': '2023-11-26T07:19:31.137790+0000', 'mapping_epoch': 43, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 43, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-26T07:19:31.113498+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-26T07:19:31.113498+0000', 'last_clean_scrub_stamp': '2023-11-26T07:19:31.113498+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 6], 'acting': [3, 6], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2023-11-26T07:19:37.259 INFO:tasks.cephfs.filesystem.ceph_manager:PG 7.2a is not active+clean 2023-11-26T07:19:37.259 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '7.2a', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 43, 'state': 'creating+peering', 'last_fresh': '2023-11-26T07:19:31.129388+0000', 'last_change': '2023-11-26T07:19:31.129388+0000', 'last_active': '2023-11-26T07:19:31.113498+0000', 'last_peered': '2023-11-26T07:19:31.113498+0000', 'last_clean': '2023-11-26T07:19:31.113498+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-26T07:19:31.129388+0000', 'last_undegraded': '2023-11-26T07:19:31.129388+0000', 'last_fullsized': '2023-11-26T07:19:31.129388+0000', 'mapping_epoch': 43, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 43, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-26T07:19:31.113498+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-26T07:19:31.113498+0000', 'last_clean_scrub_stamp': '2023-11-26T07:19:31.113498+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 6], 'acting': [0, 6], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2023-11-26T07:19:37.259 INFO:tasks.cephfs.filesystem.ceph_manager:PG 7.29 is not active+clean 2023-11-26T07:19:37.259 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '7.29', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 43, 'state': 'creating+peering', 'last_fresh': '2023-11-26T07:19:31.131006+0000', 'last_change': '2023-11-26T07:19:31.131006+0000', 'last_active': '2023-11-26T07:19:31.113498+0000', 'last_peered': '2023-11-26T07:19:31.113498+0000', 'last_clean': '2023-11-26T07:19:31.113498+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-26T07:19:31.131006+0000', 'last_undegraded': '2023-11-26T07:19:31.131006+0000', 'last_fullsized': '2023-11-26T07:19:31.131006+0000', 'mapping_epoch': 43, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 43, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-26T07:19:31.113498+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-26T07:19:31.113498+0000', 'last_clean_scrub_stamp': '2023-11-26T07:19:31.113498+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [6, 7], 'acting': [6, 7], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 6, 'acting_primary': 6, 'purged_snaps': []} 2023-11-26T07:19:37.260 INFO:tasks.cephfs.filesystem.ceph_manager:PG 7.28 is not active+clean 2023-11-26T07:19:37.260 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '7.28', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 43, 'state': 'creating+peering', 'last_fresh': '2023-11-26T07:19:31.130446+0000', 'last_change': '2023-11-26T07:19:31.130446+0000', 'last_active': '2023-11-26T07:19:31.113498+0000', 'last_peered': '2023-11-26T07:19:31.113498+0000', 'last_clean': '2023-11-26T07:19:31.113498+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-26T07:19:31.130446+0000', 'last_undegraded': '2023-11-26T07:19:31.130446+0000', 'last_fullsized': '2023-11-26T07:19:31.130446+0000', 'mapping_epoch': 43, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 43, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-26T07:19:31.113498+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-26T07:19:31.113498+0000', 'last_clean_scrub_stamp': '2023-11-26T07:19:31.113498+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [6, 7], 'acting': [6, 7], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 6, 'acting_primary': 6, 'purged_snaps': []} 2023-11-26T07:19:37.260 INFO:tasks.cephfs.filesystem.ceph_manager:PG 7.27 is not active+clean 2023-11-26T07:19:37.260 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '7.27', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 43, 'state': 'creating+peering', 'last_fresh': '2023-11-26T07:19:31.134474+0000', 'last_change': '2023-11-26T07:19:31.134474+0000', 'last_active': '2023-11-26T07:19:31.113498+0000', 'last_peered': '2023-11-26T07:19:31.113498+0000', 'last_clean': '2023-11-26T07:19:31.113498+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-26T07:19:31.134474+0000', 'last_undegraded': '2023-11-26T07:19:31.134474+0000', 'last_fullsized': '2023-11-26T07:19:31.134474+0000', 'mapping_epoch': 43, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 43, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-26T07:19:31.113498+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-26T07:19:31.113498+0000', 'last_clean_scrub_stamp': '2023-11-26T07:19:31.113498+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [6, 1], 'acting': [6, 1], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 6, 'acting_primary': 6, 'purged_snaps': []} 2023-11-26T07:19:37.260 INFO:tasks.cephfs.filesystem.ceph_manager:PG 7.26 is not active+clean 2023-11-26T07:19:37.260 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '7.26', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 43, 'state': 'creating+peering', 'last_fresh': '2023-11-26T07:19:31.132954+0000', 'last_change': '2023-11-26T07:19:31.132954+0000', 'last_active': '2023-11-26T07:19:31.113498+0000', 'last_peered': '2023-11-26T07:19:31.113498+0000', 'last_clean': '2023-11-26T07:19:31.113498+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-26T07:19:31.132954+0000', 'last_undegraded': '2023-11-26T07:19:31.132954+0000', 'last_fullsized': '2023-11-26T07:19:31.132954+0000', 'mapping_epoch': 43, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 43, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-26T07:19:31.113498+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-26T07:19:31.113498+0000', 'last_clean_scrub_stamp': '2023-11-26T07:19:31.113498+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 3], 'acting': [2, 3], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2023-11-26T07:19:37.260 INFO:tasks.cephfs.filesystem.ceph_manager:PG 7.25 is not active+clean 2023-11-26T07:19:37.260 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '7.25', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 43, 'state': 'creating+peering', 'last_fresh': '2023-11-26T07:19:31.124787+0000', 'last_change': '2023-11-26T07:19:31.124787+0000', 'last_active': '2023-11-26T07:19:31.113498+0000', 'last_peered': '2023-11-26T07:19:31.113498+0000', 'last_clean': '2023-11-26T07:19:31.113498+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-26T07:19:31.124787+0000', 'last_undegraded': '2023-11-26T07:19:31.124787+0000', 'last_fullsized': '2023-11-26T07:19:31.124787+0000', 'mapping_epoch': 43, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 43, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-26T07:19:31.113498+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-26T07:19:31.113498+0000', 'last_clean_scrub_stamp': '2023-11-26T07:19:31.113498+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [4, 5], 'acting': [4, 5], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 4, 'acting_primary': 4, 'purged_snaps': []} 2023-11-26T07:19:37.260 INFO:tasks.cephfs.filesystem.ceph_manager:PG 7.24 is not active+clean 2023-11-26T07:19:37.260 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '7.24', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 43, 'state': 'creating+peering', 'last_fresh': '2023-11-26T07:19:31.130640+0000', 'last_change': '2023-11-26T07:19:31.130640+0000', 'last_active': '2023-11-26T07:19:31.113498+0000', 'last_peered': '2023-11-26T07:19:31.113498+0000', 'last_clean': '2023-11-26T07:19:31.113498+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-26T07:19:31.130640+0000', 'last_undegraded': '2023-11-26T07:19:31.130640+0000', 'last_fullsized': '2023-11-26T07:19:31.130640+0000', 'mapping_epoch': 43, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 43, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-26T07:19:31.113498+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-26T07:19:31.113498+0000', 'last_clean_scrub_stamp': '2023-11-26T07:19:31.113498+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [5, 2], 'acting': [5, 2], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 5, 'acting_primary': 5, 'purged_snaps': []} 2023-11-26T07:19:37.261 INFO:tasks.cephfs.filesystem.ceph_manager:PG 7.23 is not active+clean 2023-11-26T07:19:37.261 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '7.23', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 43, 'state': 'creating+peering', 'last_fresh': '2023-11-26T07:19:31.131030+0000', 'last_change': '2023-11-26T07:19:31.131030+0000', 'last_active': '2023-11-26T07:19:31.113498+0000', 'last_peered': '2023-11-26T07:19:31.113498+0000', 'last_clean': '2023-11-26T07:19:31.113498+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-26T07:19:31.131030+0000', 'last_undegraded': '2023-11-26T07:19:31.131030+0000', 'last_fullsized': '2023-11-26T07:19:31.131030+0000', 'mapping_epoch': 43, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 43, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-26T07:19:31.113498+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-26T07:19:31.113498+0000', 'last_clean_scrub_stamp': '2023-11-26T07:19:31.113498+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [6, 1], 'acting': [6, 1], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 6, 'acting_primary': 6, 'purged_snaps': []} 2023-11-26T07:19:37.261 INFO:tasks.cephfs.filesystem.ceph_manager:PG 7.22 is not active+clean 2023-11-26T07:19:37.261 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '7.22', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 43, 'state': 'creating+peering', 'last_fresh': '2023-11-26T07:19:31.126109+0000', 'last_change': '2023-11-26T07:19:31.126109+0000', 'last_active': '2023-11-26T07:19:31.113498+0000', 'last_peered': '2023-11-26T07:19:31.113498+0000', 'last_clean': '2023-11-26T07:19:31.113498+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-26T07:19:31.126109+0000', 'last_undegraded': '2023-11-26T07:19:31.126109+0000', 'last_fullsized': '2023-11-26T07:19:31.126109+0000', 'mapping_epoch': 43, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 43, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-26T07:19:31.113498+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-26T07:19:31.113498+0000', 'last_clean_scrub_stamp': '2023-11-26T07:19:31.113498+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [4, 7], 'acting': [4, 7], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 4, 'acting_primary': 4, 'purged_snaps': []} 2023-11-26T07:19:37.261 INFO:tasks.cephfs.filesystem.ceph_manager:PG 7.21 is not active+clean 2023-11-26T07:19:37.261 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '7.21', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 43, 'state': 'creating+peering', 'last_fresh': '2023-11-26T07:19:31.134817+0000', 'last_change': '2023-11-26T07:19:31.134817+0000', 'last_active': '2023-11-26T07:19:31.113498+0000', 'last_peered': '2023-11-26T07:19:31.113498+0000', 'last_clean': '2023-11-26T07:19:31.113498+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-26T07:19:31.134817+0000', 'last_undegraded': '2023-11-26T07:19:31.134817+0000', 'last_fullsized': '2023-11-26T07:19:31.134817+0000', 'mapping_epoch': 43, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 43, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-26T07:19:31.113498+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-26T07:19:31.113498+0000', 'last_clean_scrub_stamp': '2023-11-26T07:19:31.113498+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 5], 'acting': [3, 5], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2023-11-26T07:19:37.261 INFO:tasks.cephfs.filesystem.ceph_manager:PG 7.20 is not active+clean 2023-11-26T07:19:37.261 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '7.20', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 43, 'state': 'creating+peering', 'last_fresh': '2023-11-26T07:19:31.127285+0000', 'last_change': '2023-11-26T07:19:31.127285+0000', 'last_active': '2023-11-26T07:19:31.113498+0000', 'last_peered': '2023-11-26T07:19:31.113498+0000', 'last_clean': '2023-11-26T07:19:31.113498+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-26T07:19:31.127285+0000', 'last_undegraded': '2023-11-26T07:19:31.127285+0000', 'last_fullsized': '2023-11-26T07:19:31.127285+0000', 'mapping_epoch': 43, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 43, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-26T07:19:31.113498+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-26T07:19:31.113498+0000', 'last_clean_scrub_stamp': '2023-11-26T07:19:31.113498+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 1], 'acting': [2, 1], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2023-11-26T07:19:37.261 INFO:tasks.cephfs.filesystem.ceph_manager:PG 7.1e is not active+clean 2023-11-26T07:19:37.261 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '7.1e', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 43, 'state': 'creating+peering', 'last_fresh': '2023-11-26T07:19:31.133294+0000', 'last_change': '2023-11-26T07:19:31.133294+0000', 'last_active': '2023-11-26T07:19:31.113498+0000', 'last_peered': '2023-11-26T07:19:31.113498+0000', 'last_clean': '2023-11-26T07:19:31.113498+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-26T07:19:31.133294+0000', 'last_undegraded': '2023-11-26T07:19:31.133294+0000', 'last_fullsized': '2023-11-26T07:19:31.133294+0000', 'mapping_epoch': 43, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 43, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-26T07:19:31.113498+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-26T07:19:31.113498+0000', 'last_clean_scrub_stamp': '2023-11-26T07:19:31.113498+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 3], 'acting': [1, 3], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 1, 'purged_snaps': []} 2023-11-26T07:19:37.262 INFO:tasks.cephfs.filesystem.ceph_manager:PG 7.1f is not active+clean 2023-11-26T07:19:37.262 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '7.1f', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 43, 'state': 'creating+peering', 'last_fresh': '2023-11-26T07:19:31.134010+0000', 'last_change': '2023-11-26T07:19:31.134010+0000', 'last_active': '2023-11-26T07:19:31.113498+0000', 'last_peered': '2023-11-26T07:19:31.113498+0000', 'last_clean': '2023-11-26T07:19:31.113498+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-26T07:19:31.134010+0000', 'last_undegraded': '2023-11-26T07:19:31.134010+0000', 'last_fullsized': '2023-11-26T07:19:31.134010+0000', 'mapping_epoch': 43, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 43, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-26T07:19:31.113498+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-26T07:19:31.113498+0000', 'last_clean_scrub_stamp': '2023-11-26T07:19:31.113498+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [6, 4], 'acting': [6, 4], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 6, 'acting_primary': 6, 'purged_snaps': []} 2023-11-26T07:19:37.262 INFO:tasks.cephfs.filesystem.ceph_manager:PG 7.1c is not active+clean 2023-11-26T07:19:37.262 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '7.1c', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 43, 'state': 'creating+peering', 'last_fresh': '2023-11-26T07:19:31.129959+0000', 'last_change': '2023-11-26T07:19:31.129959+0000', 'last_active': '2023-11-26T07:19:31.113498+0000', 'last_peered': '2023-11-26T07:19:31.113498+0000', 'last_clean': '2023-11-26T07:19:31.113498+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-26T07:19:31.129959+0000', 'last_undegraded': '2023-11-26T07:19:31.129959+0000', 'last_fullsized': '2023-11-26T07:19:31.129959+0000', 'mapping_epoch': 43, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 43, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-26T07:19:31.113498+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-26T07:19:31.113498+0000', 'last_clean_scrub_stamp': '2023-11-26T07:19:31.113498+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [5, 3], 'acting': [5, 3], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 5, 'acting_primary': 5, 'purged_snaps': []} 2023-11-26T07:19:37.262 INFO:tasks.cephfs.filesystem.ceph_manager:PG 7.1d is not active+clean 2023-11-26T07:19:37.262 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '7.1d', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 43, 'state': 'creating+peering', 'last_fresh': '2023-11-26T07:19:31.134552+0000', 'last_change': '2023-11-26T07:19:31.134552+0000', 'last_active': '2023-11-26T07:19:31.113498+0000', 'last_peered': '2023-11-26T07:19:31.113498+0000', 'last_clean': '2023-11-26T07:19:31.113498+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-26T07:19:31.134552+0000', 'last_undegraded': '2023-11-26T07:19:31.134552+0000', 'last_fullsized': '2023-11-26T07:19:31.134552+0000', 'mapping_epoch': 43, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 43, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-26T07:19:31.113498+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-26T07:19:31.113498+0000', 'last_clean_scrub_stamp': '2023-11-26T07:19:31.113498+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 6], 'acting': [1, 6], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 1, 'purged_snaps': []} 2023-11-26T07:19:37.262 INFO:tasks.cephfs.filesystem.ceph_manager:PG 7.1a is not active+clean 2023-11-26T07:19:37.262 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '7.1a', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 43, 'state': 'creating+peering', 'last_fresh': '2023-11-26T07:19:31.125537+0000', 'last_change': '2023-11-26T07:19:31.125537+0000', 'last_active': '2023-11-26T07:19:31.113498+0000', 'last_peered': '2023-11-26T07:19:31.113498+0000', 'last_clean': '2023-11-26T07:19:31.113498+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-26T07:19:31.125537+0000', 'last_undegraded': '2023-11-26T07:19:31.125537+0000', 'last_fullsized': '2023-11-26T07:19:31.125537+0000', 'mapping_epoch': 43, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 43, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-26T07:19:31.113498+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-26T07:19:31.113498+0000', 'last_clean_scrub_stamp': '2023-11-26T07:19:31.113498+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [4, 6], 'acting': [4, 6], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 4, 'acting_primary': 4, 'purged_snaps': []} 2023-11-26T07:19:37.262 INFO:tasks.cephfs.filesystem.ceph_manager:PG 7.1b is not active+clean 2023-11-26T07:19:37.262 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '7.1b', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 43, 'state': 'creating+peering', 'last_fresh': '2023-11-26T07:19:31.131194+0000', 'last_change': '2023-11-26T07:19:31.131194+0000', 'last_active': '2023-11-26T07:19:31.113498+0000', 'last_peered': '2023-11-26T07:19:31.113498+0000', 'last_clean': '2023-11-26T07:19:31.113498+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-26T07:19:31.131194+0000', 'last_undegraded': '2023-11-26T07:19:31.131194+0000', 'last_fullsized': '2023-11-26T07:19:31.131194+0000', 'mapping_epoch': 43, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 43, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-26T07:19:31.113498+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-26T07:19:31.113498+0000', 'last_clean_scrub_stamp': '2023-11-26T07:19:31.113498+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 2], 'acting': [0, 2], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2023-11-26T07:19:37.263 INFO:tasks.cephfs.filesystem.ceph_manager:PG 7.18 is not active+clean 2023-11-26T07:19:37.263 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '7.18', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 43, 'state': 'creating+peering', 'last_fresh': '2023-11-26T07:19:31.128550+0000', 'last_change': '2023-11-26T07:19:31.128550+0000', 'last_active': '2023-11-26T07:19:31.113498+0000', 'last_peered': '2023-11-26T07:19:31.113498+0000', 'last_clean': '2023-11-26T07:19:31.113498+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-26T07:19:31.128550+0000', 'last_undegraded': '2023-11-26T07:19:31.128550+0000', 'last_fullsized': '2023-11-26T07:19:31.128550+0000', 'mapping_epoch': 43, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 43, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-26T07:19:31.113498+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-26T07:19:31.113498+0000', 'last_clean_scrub_stamp': '2023-11-26T07:19:31.113498+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 6], 'acting': [0, 6], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2023-11-26T07:19:37.263 INFO:tasks.cephfs.filesystem.ceph_manager:PG 7.19 is not active+clean 2023-11-26T07:19:37.263 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '7.19', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 43, 'state': 'creating+peering', 'last_fresh': '2023-11-26T07:19:31.131606+0000', 'last_change': '2023-11-26T07:19:31.131606+0000', 'last_active': '2023-11-26T07:19:31.113498+0000', 'last_peered': '2023-11-26T07:19:31.113498+0000', 'last_clean': '2023-11-26T07:19:31.113498+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-26T07:19:31.131606+0000', 'last_undegraded': '2023-11-26T07:19:31.131606+0000', 'last_fullsized': '2023-11-26T07:19:31.131606+0000', 'mapping_epoch': 43, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 43, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-26T07:19:31.113498+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-26T07:19:31.113498+0000', 'last_clean_scrub_stamp': '2023-11-26T07:19:31.113498+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [6, 7], 'acting': [6, 7], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 6, 'acting_primary': 6, 'purged_snaps': []} 2023-11-26T07:19:37.263 INFO:tasks.cephfs.filesystem.ceph_manager:PG 7.16 is not active+clean 2023-11-26T07:19:37.263 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '7.16', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 43, 'state': 'creating+peering', 'last_fresh': '2023-11-26T07:19:31.126247+0000', 'last_change': '2023-11-26T07:19:31.126247+0000', 'last_active': '2023-11-26T07:19:31.113498+0000', 'last_peered': '2023-11-26T07:19:31.113498+0000', 'last_clean': '2023-11-26T07:19:31.113498+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-26T07:19:31.126247+0000', 'last_undegraded': '2023-11-26T07:19:31.126247+0000', 'last_fullsized': '2023-11-26T07:19:31.126247+0000', 'mapping_epoch': 43, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 43, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-26T07:19:31.113498+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-26T07:19:31.113498+0000', 'last_clean_scrub_stamp': '2023-11-26T07:19:31.113498+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [4, 0], 'acting': [4, 0], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 4, 'acting_primary': 4, 'purged_snaps': []} 2023-11-26T07:19:37.263 INFO:tasks.cephfs.filesystem.ceph_manager:PG 7.17 is not active+clean 2023-11-26T07:19:37.263 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '7.17', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 43, 'state': 'creating+peering', 'last_fresh': '2023-11-26T07:19:31.129608+0000', 'last_change': '2023-11-26T07:19:31.129608+0000', 'last_active': '2023-11-26T07:19:31.113498+0000', 'last_peered': '2023-11-26T07:19:31.113498+0000', 'last_clean': '2023-11-26T07:19:31.113498+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-26T07:19:31.129608+0000', 'last_undegraded': '2023-11-26T07:19:31.129608+0000', 'last_fullsized': '2023-11-26T07:19:31.129608+0000', 'mapping_epoch': 43, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 43, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-26T07:19:31.113498+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-26T07:19:31.113498+0000', 'last_clean_scrub_stamp': '2023-11-26T07:19:31.113498+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [7, 5], 'acting': [7, 5], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 7, 'acting_primary': 7, 'purged_snaps': []} 2023-11-26T07:19:37.263 INFO:tasks.cephfs.filesystem.ceph_manager:PG 7.14 is not active+clean 2023-11-26T07:19:37.263 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '7.14', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 43, 'state': 'creating+peering', 'last_fresh': '2023-11-26T07:19:31.133561+0000', 'last_change': '2023-11-26T07:19:31.133561+0000', 'last_active': '2023-11-26T07:19:31.113498+0000', 'last_peered': '2023-11-26T07:19:31.113498+0000', 'last_clean': '2023-11-26T07:19:31.113498+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-26T07:19:31.133561+0000', 'last_undegraded': '2023-11-26T07:19:31.133561+0000', 'last_fullsized': '2023-11-26T07:19:31.133561+0000', 'mapping_epoch': 43, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 43, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-26T07:19:31.113498+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-26T07:19:31.113498+0000', 'last_clean_scrub_stamp': '2023-11-26T07:19:31.113498+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 2], 'acting': [3, 2], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2023-11-26T07:19:37.264 INFO:tasks.cephfs.filesystem.ceph_manager:PG 7.15 is not active+clean 2023-11-26T07:19:37.264 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '7.15', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 43, 'state': 'creating+peering', 'last_fresh': '2023-11-26T07:19:31.132934+0000', 'last_change': '2023-11-26T07:19:31.132934+0000', 'last_active': '2023-11-26T07:19:31.113498+0000', 'last_peered': '2023-11-26T07:19:31.113498+0000', 'last_clean': '2023-11-26T07:19:31.113498+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-26T07:19:31.132934+0000', 'last_undegraded': '2023-11-26T07:19:31.132934+0000', 'last_fullsized': '2023-11-26T07:19:31.132934+0000', 'mapping_epoch': 43, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 43, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-26T07:19:31.113498+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-26T07:19:31.113498+0000', 'last_clean_scrub_stamp': '2023-11-26T07:19:31.113498+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [7, 1], 'acting': [7, 1], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 7, 'acting_primary': 7, 'purged_snaps': []} 2023-11-26T07:19:37.264 INFO:tasks.cephfs.filesystem.ceph_manager:PG 7.12 is not active+clean 2023-11-26T07:19:37.264 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '7.12', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 43, 'state': 'creating+peering', 'last_fresh': '2023-11-26T07:19:31.132304+0000', 'last_change': '2023-11-26T07:19:31.132304+0000', 'last_active': '2023-11-26T07:19:31.113498+0000', 'last_peered': '2023-11-26T07:19:31.113498+0000', 'last_clean': '2023-11-26T07:19:31.113498+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-26T07:19:31.132304+0000', 'last_undegraded': '2023-11-26T07:19:31.132304+0000', 'last_fullsized': '2023-11-26T07:19:31.132304+0000', 'mapping_epoch': 43, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 43, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-26T07:19:31.113498+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-26T07:19:31.113498+0000', 'last_clean_scrub_stamp': '2023-11-26T07:19:31.113498+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [6, 0], 'acting': [6, 0], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 6, 'acting_primary': 6, 'purged_snaps': []} 2023-11-26T07:19:37.264 INFO:tasks.cephfs.filesystem.ceph_manager:PG 7.13 is not active+clean 2023-11-26T07:19:37.264 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '7.13', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 43, 'state': 'creating+peering', 'last_fresh': '2023-11-26T07:19:31.130763+0000', 'last_change': '2023-11-26T07:19:31.130763+0000', 'last_active': '2023-11-26T07:19:31.113498+0000', 'last_peered': '2023-11-26T07:19:31.113498+0000', 'last_clean': '2023-11-26T07:19:31.113498+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-26T07:19:31.130763+0000', 'last_undegraded': '2023-11-26T07:19:31.130763+0000', 'last_fullsized': '2023-11-26T07:19:31.130763+0000', 'mapping_epoch': 43, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 43, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-26T07:19:31.113498+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-26T07:19:31.113498+0000', 'last_clean_scrub_stamp': '2023-11-26T07:19:31.113498+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 7], 'acting': [0, 7], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2023-11-26T07:19:37.264 INFO:tasks.cephfs.filesystem.ceph_manager:PG 7.10 is not active+clean 2023-11-26T07:19:37.264 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '7.10', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 43, 'state': 'creating+peering', 'last_fresh': '2023-11-26T07:19:31.134915+0000', 'last_change': '2023-11-26T07:19:31.134915+0000', 'last_active': '2023-11-26T07:19:31.113498+0000', 'last_peered': '2023-11-26T07:19:31.113498+0000', 'last_clean': '2023-11-26T07:19:31.113498+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-26T07:19:31.134915+0000', 'last_undegraded': '2023-11-26T07:19:31.134915+0000', 'last_fullsized': '2023-11-26T07:19:31.134915+0000', 'mapping_epoch': 43, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 43, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-26T07:19:31.113498+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-26T07:19:31.113498+0000', 'last_clean_scrub_stamp': '2023-11-26T07:19:31.113498+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 2], 'acting': [3, 2], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2023-11-26T07:19:37.264 INFO:tasks.cephfs.filesystem.ceph_manager:PG 7.11 is not active+clean 2023-11-26T07:19:37.264 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '7.11', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 43, 'state': 'creating+peering', 'last_fresh': '2023-11-26T07:19:31.129918+0000', 'last_change': '2023-11-26T07:19:31.129918+0000', 'last_active': '2023-11-26T07:19:31.113498+0000', 'last_peered': '2023-11-26T07:19:31.113498+0000', 'last_clean': '2023-11-26T07:19:31.113498+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-26T07:19:31.129918+0000', 'last_undegraded': '2023-11-26T07:19:31.129918+0000', 'last_fullsized': '2023-11-26T07:19:31.129918+0000', 'mapping_epoch': 43, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 43, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-26T07:19:31.113498+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-26T07:19:31.113498+0000', 'last_clean_scrub_stamp': '2023-11-26T07:19:31.113498+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [7, 6], 'acting': [7, 6], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 7, 'acting_primary': 7, 'purged_snaps': []} 2023-11-26T07:19:37.265 INFO:tasks.cephfs.filesystem.ceph_manager:PG 7.e is not active+clean 2023-11-26T07:19:37.265 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '7.e', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 43, 'state': 'creating+peering', 'last_fresh': '2023-11-26T07:19:31.126558+0000', 'last_change': '2023-11-26T07:19:31.126558+0000', 'last_active': '2023-11-26T07:19:31.113498+0000', 'last_peered': '2023-11-26T07:19:31.113498+0000', 'last_clean': '2023-11-26T07:19:31.113498+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-26T07:19:31.126558+0000', 'last_undegraded': '2023-11-26T07:19:31.126558+0000', 'last_fullsized': '2023-11-26T07:19:31.126558+0000', 'mapping_epoch': 43, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 43, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-26T07:19:31.113498+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-26T07:19:31.113498+0000', 'last_clean_scrub_stamp': '2023-11-26T07:19:31.113498+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 5], 'acting': [2, 5], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2023-11-26T07:19:37.265 INFO:tasks.cephfs.filesystem.ceph_manager:PG 7.f is not active+clean 2023-11-26T07:19:37.265 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '7.f', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 43, 'state': 'creating+peering', 'last_fresh': '2023-11-26T07:19:31.128236+0000', 'last_change': '2023-11-26T07:19:31.128236+0000', 'last_active': '2023-11-26T07:19:31.113498+0000', 'last_peered': '2023-11-26T07:19:31.113498+0000', 'last_clean': '2023-11-26T07:19:31.113498+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-26T07:19:31.128236+0000', 'last_undegraded': '2023-11-26T07:19:31.128236+0000', 'last_fullsized': '2023-11-26T07:19:31.128236+0000', 'mapping_epoch': 43, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 43, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-26T07:19:31.113498+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-26T07:19:31.113498+0000', 'last_clean_scrub_stamp': '2023-11-26T07:19:31.113498+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 3], 'acting': [0, 3], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2023-11-26T07:19:37.265 INFO:tasks.cephfs.filesystem.ceph_manager:PG 7.c is not active+clean 2023-11-26T07:19:37.265 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '7.c', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 43, 'state': 'creating+peering', 'last_fresh': '2023-11-26T07:19:31.132510+0000', 'last_change': '2023-11-26T07:19:31.132510+0000', 'last_active': '2023-11-26T07:19:31.113498+0000', 'last_peered': '2023-11-26T07:19:31.113498+0000', 'last_clean': '2023-11-26T07:19:31.113498+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-26T07:19:31.132510+0000', 'last_undegraded': '2023-11-26T07:19:31.132510+0000', 'last_fullsized': '2023-11-26T07:19:31.132510+0000', 'mapping_epoch': 43, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 43, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-26T07:19:31.113498+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-26T07:19:31.113498+0000', 'last_clean_scrub_stamp': '2023-11-26T07:19:31.113498+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [6, 3], 'acting': [6, 3], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 6, 'acting_primary': 6, 'purged_snaps': []} 2023-11-26T07:19:37.265 INFO:tasks.cephfs.filesystem.ceph_manager:PG 7.d is not active+clean 2023-11-26T07:19:37.265 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '7.d', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 43, 'state': 'creating+peering', 'last_fresh': '2023-11-26T07:19:31.134528+0000', 'last_change': '2023-11-26T07:19:31.134528+0000', 'last_active': '2023-11-26T07:19:31.113498+0000', 'last_peered': '2023-11-26T07:19:31.113498+0000', 'last_clean': '2023-11-26T07:19:31.113498+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-26T07:19:31.134528+0000', 'last_undegraded': '2023-11-26T07:19:31.134528+0000', 'last_fullsized': '2023-11-26T07:19:31.134528+0000', 'mapping_epoch': 43, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 43, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-26T07:19:31.113498+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-26T07:19:31.113498+0000', 'last_clean_scrub_stamp': '2023-11-26T07:19:31.113498+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [6, 1], 'acting': [6, 1], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 6, 'acting_primary': 6, 'purged_snaps': []} 2023-11-26T07:19:37.265 INFO:tasks.cephfs.filesystem.ceph_manager:PG 7.a is not active+clean 2023-11-26T07:19:37.265 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '7.a', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 43, 'state': 'creating+peering', 'last_fresh': '2023-11-26T07:19:31.137790+0000', 'last_change': '2023-11-26T07:19:31.137790+0000', 'last_active': '2023-11-26T07:19:31.113498+0000', 'last_peered': '2023-11-26T07:19:31.113498+0000', 'last_clean': '2023-11-26T07:19:31.113498+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-26T07:19:31.137790+0000', 'last_undegraded': '2023-11-26T07:19:31.137790+0000', 'last_fullsized': '2023-11-26T07:19:31.137790+0000', 'mapping_epoch': 43, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 43, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-26T07:19:31.113498+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-26T07:19:31.113498+0000', 'last_clean_scrub_stamp': '2023-11-26T07:19:31.113498+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 2], 'acting': [3, 2], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2023-11-26T07:19:37.266 INFO:tasks.cephfs.filesystem.ceph_manager:PG 7.b is not active+clean 2023-11-26T07:19:37.266 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '7.b', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 43, 'state': 'creating+peering', 'last_fresh': '2023-11-26T07:19:31.132864+0000', 'last_change': '2023-11-26T07:19:31.132864+0000', 'last_active': '2023-11-26T07:19:31.113498+0000', 'last_peered': '2023-11-26T07:19:31.113498+0000', 'last_clean': '2023-11-26T07:19:31.113498+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-26T07:19:31.132864+0000', 'last_undegraded': '2023-11-26T07:19:31.132864+0000', 'last_fullsized': '2023-11-26T07:19:31.132864+0000', 'mapping_epoch': 43, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 43, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-26T07:19:31.113498+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-26T07:19:31.113498+0000', 'last_clean_scrub_stamp': '2023-11-26T07:19:31.113498+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 5], 'acting': [1, 5], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 1, 'purged_snaps': []} 2023-11-26T07:19:37.266 INFO:tasks.cephfs.filesystem.ceph_manager:PG 7.8 is not active+clean 2023-11-26T07:19:37.266 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '7.8', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 43, 'state': 'creating+peering', 'last_fresh': '2023-11-26T07:19:31.126547+0000', 'last_change': '2023-11-26T07:19:31.126547+0000', 'last_active': '2023-11-26T07:19:31.113498+0000', 'last_peered': '2023-11-26T07:19:31.113498+0000', 'last_clean': '2023-11-26T07:19:31.113498+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-26T07:19:31.126547+0000', 'last_undegraded': '2023-11-26T07:19:31.126547+0000', 'last_fullsized': '2023-11-26T07:19:31.126547+0000', 'mapping_epoch': 43, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 43, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-26T07:19:31.113498+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-26T07:19:31.113498+0000', 'last_clean_scrub_stamp': '2023-11-26T07:19:31.113498+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 7], 'acting': [2, 7], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2023-11-26T07:19:37.266 INFO:tasks.cephfs.filesystem.ceph_manager:PG 7.9 is not active+clean 2023-11-26T07:19:37.266 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '7.9', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 43, 'state': 'creating+peering', 'last_fresh': '2023-11-26T07:19:31.129887+0000', 'last_change': '2023-11-26T07:19:31.129887+0000', 'last_active': '2023-11-26T07:19:31.113498+0000', 'last_peered': '2023-11-26T07:19:31.113498+0000', 'last_clean': '2023-11-26T07:19:31.113498+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-26T07:19:31.129887+0000', 'last_undegraded': '2023-11-26T07:19:31.129887+0000', 'last_fullsized': '2023-11-26T07:19:31.129887+0000', 'mapping_epoch': 43, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 43, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-26T07:19:31.113498+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-26T07:19:31.113498+0000', 'last_clean_scrub_stamp': '2023-11-26T07:19:31.113498+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0, 1], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2023-11-26T07:19:37.266 INFO:tasks.cephfs.filesystem.ceph_manager:PG 7.7 is not active+clean 2023-11-26T07:19:37.266 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '7.7', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 43, 'state': 'creating+peering', 'last_fresh': '2023-11-26T07:19:31.132550+0000', 'last_change': '2023-11-26T07:19:31.132550+0000', 'last_active': '2023-11-26T07:19:31.113498+0000', 'last_peered': '2023-11-26T07:19:31.113498+0000', 'last_clean': '2023-11-26T07:19:31.113498+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-26T07:19:31.132550+0000', 'last_undegraded': '2023-11-26T07:19:31.132550+0000', 'last_fullsized': '2023-11-26T07:19:31.132550+0000', 'mapping_epoch': 43, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 43, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-26T07:19:31.113498+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-26T07:19:31.113498+0000', 'last_clean_scrub_stamp': '2023-11-26T07:19:31.113498+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [6, 0], 'acting': [6, 0], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 6, 'acting_primary': 6, 'purged_snaps': []} 2023-11-26T07:19:37.266 INFO:tasks.cephfs.filesystem.ceph_manager:PG 7.4 is not active+clean 2023-11-26T07:19:37.267 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '7.4', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 43, 'state': 'creating+peering', 'last_fresh': '2023-11-26T07:19:31.128178+0000', 'last_change': '2023-11-26T07:19:31.128178+0000', 'last_active': '2023-11-26T07:19:31.113498+0000', 'last_peered': '2023-11-26T07:19:31.113498+0000', 'last_clean': '2023-11-26T07:19:31.113498+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-26T07:19:31.128178+0000', 'last_undegraded': '2023-11-26T07:19:31.128178+0000', 'last_fullsized': '2023-11-26T07:19:31.128178+0000', 'mapping_epoch': 43, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 43, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-26T07:19:31.113498+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-26T07:19:31.113498+0000', 'last_clean_scrub_stamp': '2023-11-26T07:19:31.113498+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 4], 'acting': [0, 4], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2023-11-26T07:19:37.267 INFO:tasks.cephfs.filesystem.ceph_manager:PG 7.5 is not active+clean 2023-11-26T07:19:37.267 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '7.5', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 43, 'state': 'creating+peering', 'last_fresh': '2023-11-26T07:19:31.132476+0000', 'last_change': '2023-11-26T07:19:31.132476+0000', 'last_active': '2023-11-26T07:19:31.113498+0000', 'last_peered': '2023-11-26T07:19:31.113498+0000', 'last_clean': '2023-11-26T07:19:31.113498+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-26T07:19:31.132476+0000', 'last_undegraded': '2023-11-26T07:19:31.132476+0000', 'last_fullsized': '2023-11-26T07:19:31.132476+0000', 'mapping_epoch': 43, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 43, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-26T07:19:31.113498+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-26T07:19:31.113498+0000', 'last_clean_scrub_stamp': '2023-11-26T07:19:31.113498+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [7, 2], 'acting': [7, 2], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 7, 'acting_primary': 7, 'purged_snaps': []} 2023-11-26T07:19:37.267 INFO:tasks.cephfs.filesystem.ceph_manager:PG 7.2 is not active+clean 2023-11-26T07:19:37.267 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '7.2', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 43, 'state': 'creating+peering', 'last_fresh': '2023-11-26T07:19:31.134609+0000', 'last_change': '2023-11-26T07:19:31.134609+0000', 'last_active': '2023-11-26T07:19:31.113498+0000', 'last_peered': '2023-11-26T07:19:31.113498+0000', 'last_clean': '2023-11-26T07:19:31.113498+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-26T07:19:31.134609+0000', 'last_undegraded': '2023-11-26T07:19:31.134609+0000', 'last_fullsized': '2023-11-26T07:19:31.134609+0000', 'mapping_epoch': 43, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 43, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-26T07:19:31.113498+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-26T07:19:31.113498+0000', 'last_clean_scrub_stamp': '2023-11-26T07:19:31.113498+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 1], 'acting': [3, 1], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2023-11-26T07:19:37.267 INFO:tasks.cephfs.filesystem.ceph_manager:PG 7.3 is not active+clean 2023-11-26T07:19:37.267 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '7.3', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 43, 'state': 'creating+peering', 'last_fresh': '2023-11-26T07:19:31.134861+0000', 'last_change': '2023-11-26T07:19:31.134861+0000', 'last_active': '2023-11-26T07:19:31.113498+0000', 'last_peered': '2023-11-26T07:19:31.113498+0000', 'last_clean': '2023-11-26T07:19:31.113498+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-26T07:19:31.134861+0000', 'last_undegraded': '2023-11-26T07:19:31.134861+0000', 'last_fullsized': '2023-11-26T07:19:31.134861+0000', 'mapping_epoch': 43, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 43, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-26T07:19:31.113498+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-26T07:19:31.113498+0000', 'last_clean_scrub_stamp': '2023-11-26T07:19:31.113498+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 2], 'acting': [3, 2], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2023-11-26T07:19:37.267 INFO:tasks.cephfs.filesystem.ceph_manager:PG 7.0 is not active+clean 2023-11-26T07:19:37.267 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '7.0', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 43, 'state': 'creating+peering', 'last_fresh': '2023-11-26T07:19:31.130272+0000', 'last_change': '2023-11-26T07:19:31.130272+0000', 'last_active': '2023-11-26T07:19:31.113498+0000', 'last_peered': '2023-11-26T07:19:31.113498+0000', 'last_clean': '2023-11-26T07:19:31.113498+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-26T07:19:31.130272+0000', 'last_undegraded': '2023-11-26T07:19:31.130272+0000', 'last_fullsized': '2023-11-26T07:19:31.130272+0000', 'mapping_epoch': 43, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 43, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-26T07:19:31.113498+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-26T07:19:31.113498+0000', 'last_clean_scrub_stamp': '2023-11-26T07:19:31.113498+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [5, 1], 'acting': [5, 1], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 5, 'acting_primary': 5, 'purged_snaps': []} 2023-11-26T07:19:37.267 INFO:tasks.cephfs.filesystem.ceph_manager:PG 7.1 is not active+clean 2023-11-26T07:19:37.267 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '7.1', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 43, 'state': 'creating+peering', 'last_fresh': '2023-11-26T07:19:31.124694+0000', 'last_change': '2023-11-26T07:19:31.124694+0000', 'last_active': '2023-11-26T07:19:31.113498+0000', 'last_peered': '2023-11-26T07:19:31.113498+0000', 'last_clean': '2023-11-26T07:19:31.113498+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-26T07:19:31.124694+0000', 'last_undegraded': '2023-11-26T07:19:31.124694+0000', 'last_fullsized': '2023-11-26T07:19:31.124694+0000', 'mapping_epoch': 43, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 43, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-26T07:19:31.113498+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-26T07:19:31.113498+0000', 'last_clean_scrub_stamp': '2023-11-26T07:19:31.113498+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [4, 3], 'acting': [4, 3], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 4, 'acting_primary': 4, 'purged_snaps': []} 2023-11-26T07:19:37.268 INFO:tasks.cephfs.filesystem.ceph_manager:PG 7.6 is not active+clean 2023-11-26T07:19:37.268 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '7.6', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 43, 'state': 'creating+peering', 'last_fresh': '2023-11-26T07:19:31.128572+0000', 'last_change': '2023-11-26T07:19:31.128572+0000', 'last_active': '2023-11-26T07:19:31.113498+0000', 'last_peered': '2023-11-26T07:19:31.113498+0000', 'last_clean': '2023-11-26T07:19:31.113498+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-26T07:19:31.128572+0000', 'last_undegraded': '2023-11-26T07:19:31.128572+0000', 'last_fullsized': '2023-11-26T07:19:31.128572+0000', 'mapping_epoch': 43, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 43, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-26T07:19:31.113498+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-26T07:19:31.113498+0000', 'last_clean_scrub_stamp': '2023-11-26T07:19:31.113498+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 7], 'acting': [0, 7], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2023-11-26T07:19:37.268 DEBUG:teuthology.orchestra.run.smithi143:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2023-11-26T07:19:37.429 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:37.427+0000 7fb8d1679640 1 Processor -- start 2023-11-26T07:19:37.429 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:37.427+0000 7fb8d1679640 1 -- start start 2023-11-26T07:19:37.430 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:37.427+0000 7fb8d1679640 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7fb8cc15d6e0 0x7fb8cc0d0df0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:19:37.430 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:37.427+0000 7fb8d1679640 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7fb8cc0d1bc0 0x7fb8cc0b0750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:19:37.430 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:37.427+0000 7fb8d1679640 1 -- --> v1:172.21.15.188:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb8cc0cf0e0 con 0x7fb8cc0b2160 2023-11-26T07:19:37.430 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:37.427+0000 7fb8d1679640 1 -- --> v2:172.21.15.143:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb8cc15dac0 con 0x7fb8cc0d1bc0 2023-11-26T07:19:37.431 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:37.427+0000 7fb8d1679640 1 -- --> v2:172.21.15.188:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb8cc0ceb20 con 0x7fb8cc15d6e0 2023-11-26T07:19:37.431 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:37.427+0000 7fb8cb7fe640 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7fb8cc0d1bc0 0x7fb8cc0b0750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:19:37.431 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:37.427+0000 7fb8cb7fe640 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7fb8cc0d1bc0 0x7fb8cc0b0750 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:52262/0 (socket says 172.21.15.143:52262) 2023-11-26T07:19:37.431 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:37.427+0000 7fb8cb7fe640 1 -- 172.21.15.143:0/389425223 learned_addr learned my addr 172.21.15.143:0/389425223 (peer_addr_for_me v2:172.21.15.143:0/0) 2023-11-26T07:19:37.431 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:37.427+0000 7fb8caffd640 1 --2- 172.21.15.143:0/389425223 >> v2:172.21.15.188:3300/0 conn(0x7fb8cc15d6e0 0x7fb8cc0d0df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:19:37.432 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:37.427+0000 7fb8cb7fe640 1 -- 172.21.15.143:0/389425223 >> v2:172.21.15.188:3300/0 conn(0x7fb8cc15d6e0 msgr2=0x7fb8cc0d0df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-26T07:19:37.432 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:37.427+0000 7fb8cb7fe640 1 --2- 172.21.15.143:0/389425223 >> v2:172.21.15.188:3300/0 conn(0x7fb8cc15d6e0 0x7fb8cc0d0df0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:19:37.432 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:37.427+0000 7fb8cb7fe640 1 -- 172.21.15.143:0/389425223 >> v1:172.21.15.188:6789/0 conn(0x7fb8cc0b2160 legacy=0x7fb8cc0d14b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-26T07:19:37.432 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:37.427+0000 7fb8cb7fe640 1 -- 172.21.15.143:0/389425223 --> v2:172.21.15.143:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb8cc0cee00 con 0x7fb8cc0d1bc0 2023-11-26T07:19:37.433 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:37.431+0000 7fb8cb7fe640 1 --2- 172.21.15.143:0/389425223 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fb8cc0d1bc0 0x7fb8cc0b0750 secure :-1 s=READY pgs=583 cs=0 l=1 rev1=1 crypto rx=0x7fb8cc0b5db0 tx=0x7fb8c401c990 comp rx=0 tx=0).ready entity=mon.0 client_cookie=35813da619e87e2a server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:19:37.434 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:37.431+0000 7fb8c9ffb640 1 -- 172.21.15.143:0/389425223 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb8c40a5740 con 0x7fb8cc0d1bc0 2023-11-26T07:19:37.434 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:37.431+0000 7fb8c9ffb640 1 -- 172.21.15.143:0/389425223 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb8c40a58a0 con 0x7fb8cc0d1bc0 2023-11-26T07:19:37.434 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:37.431+0000 7fb8c9ffb640 1 -- 172.21.15.143:0/389425223 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb8c40a5c30 con 0x7fb8cc0d1bc0 2023-11-26T07:19:37.435 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:37.431+0000 7fb8d1679640 1 -- 172.21.15.143:0/389425223 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fb8cc0d1bc0 msgr2=0x7fb8cc0b0750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:19:37.435 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:37.431+0000 7fb8d1679640 1 --2- 172.21.15.143:0/389425223 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fb8cc0d1bc0 0x7fb8cc0b0750 secure :-1 s=READY pgs=583 cs=0 l=1 rev1=1 crypto rx=0x7fb8cc0b5db0 tx=0x7fb8c401c990 comp rx=0 tx=0).stop 2023-11-26T07:19:37.435 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:37.431+0000 7fb8d1679640 1 -- 172.21.15.143:0/389425223 shutdown_connections 2023-11-26T07:19:37.435 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:37.431+0000 7fb8d1679640 1 --2- 172.21.15.143:0/389425223 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fb8cc0d1bc0 0x7fb8cc0b0750 unknown :-1 s=CLOSED pgs=583 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:19:37.435 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:37.431+0000 7fb8d1679640 1 --2- 172.21.15.143:0/389425223 >> v2:172.21.15.188:3300/0 conn(0x7fb8cc15d6e0 0x7fb8cc0d0df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:19:37.435 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:37.431+0000 7fb8d1679640 1 -- 172.21.15.143:0/389425223 >> 172.21.15.143:0/389425223 conn(0x7fb8cc09c550 msgr2=0x7fb8cc0d3220 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:19:37.436 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:37.431+0000 7fb8d1679640 1 -- 172.21.15.143:0/389425223 shutdown_connections 2023-11-26T07:19:37.436 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:37.431+0000 7fb8d1679640 1 -- 172.21.15.143:0/389425223 wait complete. 2023-11-26T07:19:37.436 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:37.431+0000 7fb8d1679640 1 Processor -- start 2023-11-26T07:19:37.436 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:37.435+0000 7fb8d1679640 1 -- start start 2023-11-26T07:19:37.438 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:37.435+0000 7fb8d1679640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fb8cc0b2160 0x7fb8cc1538c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:19:37.438 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:37.435+0000 7fb8d1679640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fb8cc0d1bc0 0x7fb8cc0b5e40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:19:37.438 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:37.435+0000 7fb8d1679640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fb8cc15d6e0 0x7fb8cc06b130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:19:37.438 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:37.435+0000 7fb8d1679640 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb8cc155250 con 0x7fb8cc0d1bc0 2023-11-26T07:19:37.439 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:37.435+0000 7fb8d1679640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb8cc153e00 con 0x7fb8cc15d6e0 2023-11-26T07:19:37.439 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:37.435+0000 7fb8d1679640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb8cc154980 con 0x7fb8cc0b2160 2023-11-26T07:19:37.439 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:37.435+0000 7fb8ca7fc640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fb8cc0d1bc0 0x7fb8cc0b5e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:19:37.439 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:37.435+0000 7fb8ca7fc640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fb8cc0d1bc0 0x7fb8cc0b5e40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:52270/0 (socket says 172.21.15.143:52270) 2023-11-26T07:19:37.439 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:37.435+0000 7fb8ca7fc640 1 -- 172.21.15.143:0/1110921540 learned_addr learned my addr 172.21.15.143:0/1110921540 (peer_addr_for_me v2:172.21.15.143:0/0) 2023-11-26T07:19:37.439 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:37.435+0000 7fb8caffd640 1 --2- 172.21.15.143:0/1110921540 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fb8cc0b2160 0x7fb8cc1538c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:19:37.439 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:37.435+0000 7fb8cb7fe640 1 --2- 172.21.15.143:0/1110921540 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fb8cc15d6e0 0x7fb8cc06b130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:19:37.439 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:37.435+0000 7fb8ca7fc640 1 -- 172.21.15.143:0/1110921540 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fb8cc0b2160 msgr2=0x7fb8cc1538c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:19:37.439 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:37.435+0000 7fb8ca7fc640 1 --2- 172.21.15.143:0/1110921540 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fb8cc0b2160 0x7fb8cc1538c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:19:37.439 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:37.435+0000 7fb8ca7fc640 1 -- 172.21.15.143:0/1110921540 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fb8cc15d6e0 msgr2=0x7fb8cc06b130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:19:37.439 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:37.435+0000 7fb8ca7fc640 1 --2- 172.21.15.143:0/1110921540 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fb8cc15d6e0 0x7fb8cc06b130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:19:37.439 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:37.435+0000 7fb8ca7fc640 1 -- 172.21.15.143:0/1110921540 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb8c4009b30 con 0x7fb8cc0d1bc0 2023-11-26T07:19:37.440 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:37.435+0000 7fb8cb7fe640 1 --2- 172.21.15.143:0/1110921540 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fb8cc15d6e0 0x7fb8cc06b130 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-26T07:19:37.440 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:37.435+0000 7fb8ca7fc640 1 --2- 172.21.15.143:0/1110921540 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fb8cc0d1bc0 0x7fb8cc0b5e40 secure :-1 s=READY pgs=584 cs=0 l=1 rev1=1 crypto rx=0x7fb8c000c450 tx=0x7fb8c000c910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:19:37.441 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:37.435+0000 7fb8b37fe640 1 -- 172.21.15.143:0/1110921540 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb8c0012830 con 0x7fb8cc0d1bc0 2023-11-26T07:19:37.441 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:37.435+0000 7fb8b37fe640 1 -- 172.21.15.143:0/1110921540 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb8c00129b0 con 0x7fb8cc0d1bc0 2023-11-26T07:19:37.441 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:37.435+0000 7fb8b37fe640 1 -- 172.21.15.143:0/1110921540 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb8c0012d60 con 0x7fb8cc0d1bc0 2023-11-26T07:19:37.441 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:37.435+0000 7fb8d1679640 1 -- 172.21.15.143:0/1110921540 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb8cc0b6380 con 0x7fb8cc0d1bc0 2023-11-26T07:19:37.441 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:37.435+0000 7fb8d1679640 1 -- 172.21.15.143:0/1110921540 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb8cc154b90 con 0x7fb8cc0d1bc0 2023-11-26T07:19:37.441 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:37.435+0000 7fb8d1679640 1 -- 172.21.15.143:0/1110921540 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb8cc067680 con 0x7fb8cc0d1bc0 2023-11-26T07:19:37.441 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:37.439+0000 7fb8b37fe640 1 -- 172.21.15.143:0/1110921540 <== mon.0 v2:172.21.15.143:3300/0 4 ==== mgrmap(e 7) v1 ==== 86005+0+0 (secure 0 0 0) 0x7fb8c001c460 con 0x7fb8cc0d1bc0 2023-11-26T07:19:37.442 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:37.439+0000 7fb8b37fe640 1 --2- 172.21.15.143:0/1110921540 >> [v2:172.21.15.143:6832/2150557544,v1:172.21.15.143:6833/2150557544] conn(0x7fb8b8066620 0x7fb8b8068ae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:19:37.442 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:37.439+0000 7fb8b37fe640 1 -- 172.21.15.143:0/1110921540 <== mon.0 v2:172.21.15.143:3300/0 5 ==== osd_map(45..45 src has 1..45) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fb8c0084000 con 0x7fb8cc0d1bc0 2023-11-26T07:19:37.442 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:37.439+0000 7fb8caffd640 1 --2- 172.21.15.143:0/1110921540 >> [v2:172.21.15.143:6832/2150557544,v1:172.21.15.143:6833/2150557544] conn(0x7fb8b8066620 0x7fb8b8068ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:19:37.442 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:37.439+0000 7fb8caffd640 1 --2- 172.21.15.143:0/1110921540 >> [v2:172.21.15.143:6832/2150557544,v1:172.21.15.143:6833/2150557544] conn(0x7fb8b8066620 0x7fb8b8068ae0 secure :-1 s=READY pgs=293 cs=0 l=1 rev1=1 crypto rx=0x7fb8bc0079c0 tx=0x7fb8bc008040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:19:37.445 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:37.443+0000 7fb8b37fe640 1 -- 172.21.15.143:0/1110921540 <== mon.0 v2:172.21.15.143:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172649 (secure 0 0 0) 0x7fb8c0051160 con 0x7fb8cc0d1bc0 2023-11-26T07:19:37.651 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:37.647+0000 7fb8d1679640 1 -- 172.21.15.143:0/1110921540 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "status", "format": "json"} v 0) v1 -- 0x7fb8cc159fd0 con 0x7fb8cc0d1bc0 2023-11-26T07:19:37.652 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:37.647+0000 7fb8b37fe640 1 -- 172.21.15.143:0/1110921540 <== mon.0 v2:172.21.15.143:3300/0 7 ==== mon_command_ack([{"prefix": "status", "format": "json"}]=0 v0) v1 ==== 72+0+1452 (secure 0 0 0) 0x7fb8c0054780 con 0x7fb8cc0d1bc0 2023-11-26T07:19:37.652 INFO:teuthology.orchestra.run.smithi143.stdout: 2023-11-26T07:19:37.652 INFO:teuthology.orchestra.run.smithi143.stdout:{"fsid":"f5033060-2469-4a07-95fe-2b76f5404bc1","health":{"status":"HEALTH_OK","checks":{},"mutes":[]},"election_epoch":4,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":129,"monmap":{"epoch":1,"min_mon_release_name":"reef","num_mons":3},"osdmap":{"epoch":45,"num_osds":8,"num_up_osds":8,"osd_up_since":1700983061,"num_in_osds":8,"osd_in_since":1700983051,"num_remapped_pgs":0},"pgmap":{"pgs_by_state":[{"state_name":"creating+peering","count":64},{"state_name":"active+clean","count":33}],"num_pgs":97,"num_pools":3,"num_objects":2,"data_bytes":590368,"bytes_used":226549760,"bytes_avail":772867563520,"bytes_total":773094113280,"inactive_pgs_ratio":0.65979379415512085},"fsmap":{"epoch":25,"id":3,"up":1,"in":1,"max":1,"by_rank":[{"filesystem_id":3,"rank":0,"name":"c","status":"up:active","gid":5748}],"up:standby":3},"mgrmap":{"available":true,"num_standbys":1,"modules":["iostat","nfs","restful"],"services":{}},"servicemap":{"epoch":4,"modified":"2023-11-26T07:19:19.710006+0000","services":{"mds":{"daemons":{"summary":"","a":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}},"b":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}},"d":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}}}}}},"progress_events":{}} 2023-11-26T07:19:37.654 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:37.651+0000 7fb8d1679640 1 -- 172.21.15.143:0/1110921540 >> [v2:172.21.15.143:6832/2150557544,v1:172.21.15.143:6833/2150557544] conn(0x7fb8b8066620 msgr2=0x7fb8b8068ae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:19:37.654 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:37.651+0000 7fb8d1679640 1 --2- 172.21.15.143:0/1110921540 >> [v2:172.21.15.143:6832/2150557544,v1:172.21.15.143:6833/2150557544] conn(0x7fb8b8066620 0x7fb8b8068ae0 secure :-1 s=READY pgs=293 cs=0 l=1 rev1=1 crypto rx=0x7fb8bc0079c0 tx=0x7fb8bc008040 comp rx=0 tx=0).stop 2023-11-26T07:19:37.654 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:37.651+0000 7fb8d1679640 1 -- 172.21.15.143:0/1110921540 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fb8cc0d1bc0 msgr2=0x7fb8cc0b5e40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:19:37.654 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:37.651+0000 7fb8d1679640 1 --2- 172.21.15.143:0/1110921540 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fb8cc0d1bc0 0x7fb8cc0b5e40 secure :-1 s=READY pgs=584 cs=0 l=1 rev1=1 crypto rx=0x7fb8c000c450 tx=0x7fb8c000c910 comp rx=0 tx=0).stop 2023-11-26T07:19:37.654 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:37.651+0000 7fb8d1679640 1 -- 172.21.15.143:0/1110921540 shutdown_connections 2023-11-26T07:19:37.654 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:37.651+0000 7fb8d1679640 1 --2- 172.21.15.143:0/1110921540 >> [v2:172.21.15.143:6832/2150557544,v1:172.21.15.143:6833/2150557544] conn(0x7fb8b8066620 0x7fb8b8068ae0 unknown :-1 s=CLOSED pgs=293 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:19:37.654 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:37.651+0000 7fb8d1679640 1 --2- 172.21.15.143:0/1110921540 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fb8cc15d6e0 0x7fb8cc06b130 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:19:37.654 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:37.651+0000 7fb8d1679640 1 --2- 172.21.15.143:0/1110921540 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fb8cc0d1bc0 0x7fb8cc0b5e40 unknown :-1 s=CLOSED pgs=584 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:19:37.654 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:37.651+0000 7fb8d1679640 1 --2- 172.21.15.143:0/1110921540 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fb8cc0b2160 0x7fb8cc1538c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:19:37.655 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:37.651+0000 7fb8d1679640 1 -- 172.21.15.143:0/1110921540 >> 172.21.15.143:0/1110921540 conn(0x7fb8cc09c550 msgr2=0x7fb8cc0b25c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:19:37.655 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:37.651+0000 7fb8d1679640 1 -- 172.21.15.143:0/1110921540 shutdown_connections 2023-11-26T07:19:37.655 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:37.651+0000 7fb8d1679640 1 -- 172.21.15.143:0/1110921540 wait complete. 2023-11-26T07:19:37.673 INFO:tasks.cephfs.filesystem.ceph_manager:no progress seen, keeping timeout for now 2023-11-26T07:19:37.673 DEBUG:teuthology.orchestra.run.smithi143:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-11-26T07:19:37.811 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:37.807+0000 7f78ae357640 1 Processor -- start 2023-11-26T07:19:37.811 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:37.807+0000 7f78ae357640 1 -- start start 2023-11-26T07:19:37.812 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:37.807+0000 7f78ae357640 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7f78a8143ac0 0x7f78a8070370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:19:37.812 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:37.807+0000 7f78ae357640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f78a80708b0 0x7f78a80cf280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:19:37.812 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:37.807+0000 7f78ae357640 1 -- --> v1:172.21.15.188:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f78a80cfbf0 con 0x7f78a80cf7c0 2023-11-26T07:19:37.812 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:37.807+0000 7f78ae357640 1 -- --> v2:172.21.15.188:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f78a8143ea0 con 0x7f78a8143ac0 2023-11-26T07:19:37.812 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:37.807+0000 7f78ae357640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f78a80cc730 con 0x7f78a80708b0 2023-11-26T07:19:37.813 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:37.811+0000 7f78a77fe640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f78a80708b0 0x7f78a80cf280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:19:37.813 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:37.811+0000 7f78a7fff640 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7f78a8143ac0 0x7f78a8070370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:19:37.813 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:37.811+0000 7f78a77fe640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f78a80708b0 0x7f78a80cf280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.188:3301/0 says I am v2:172.21.15.143:54860/0 (socket says 172.21.15.143:54860) 2023-11-26T07:19:37.813 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:37.811+0000 7f78a77fe640 1 -- 172.21.15.143:0/3283507632 learned_addr learned my addr 172.21.15.143:0/3283507632 (peer_addr_for_me v2:172.21.15.143:0/0) 2023-11-26T07:19:37.814 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:37.811+0000 7f78a77fe640 1 -- 172.21.15.143:0/3283507632 >> v2:172.21.15.188:3300/0 conn(0x7f78a8143ac0 msgr2=0x7f78a8070370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-26T07:19:37.814 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:37.811+0000 7f78a77fe640 1 --2- 172.21.15.143:0/3283507632 >> v2:172.21.15.188:3300/0 conn(0x7f78a8143ac0 0x7f78a8070370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:19:37.814 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:37.811+0000 7f78a77fe640 1 -- 172.21.15.143:0/3283507632 >> v1:172.21.15.188:6789/0 conn(0x7f78a80cf7c0 legacy=0x7f78a80b1ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:19:37.814 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:37.811+0000 7f78a77fe640 1 -- 172.21.15.143:0/3283507632 --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f78a8070ee0 con 0x7f78a80708b0 2023-11-26T07:19:37.816 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:37.811+0000 7f78a77fe640 1 --2- 172.21.15.143:0/3283507632 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f78a80708b0 0x7f78a80cf280 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7f789c00bb10 tx=0x7f789c01ec10 comp rx=0 tx=0).ready entity=mon.2 client_cookie=f2798fc2cbb82239 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:19:37.816 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:37.811+0000 7f78a6ffd640 1 -- 172.21.15.143:0/3283507632 <== mon.2 v2:172.21.15.188:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f789c0a7bf0 con 0x7f78a80708b0 2023-11-26T07:19:37.816 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:37.811+0000 7f78a6ffd640 1 -- 172.21.15.143:0/3283507632 <== mon.2 v2:172.21.15.188:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f789c0a7d50 con 0x7f78a80708b0 2023-11-26T07:19:37.816 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:37.811+0000 7f78a6ffd640 1 -- 172.21.15.143:0/3283507632 <== mon.2 v2:172.21.15.188:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f789c0af660 con 0x7f78a80708b0 2023-11-26T07:19:37.817 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:37.815+0000 7f78ae357640 1 -- 172.21.15.143:0/3283507632 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f78a80708b0 msgr2=0x7f78a80cf280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:19:37.817 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:37.815+0000 7f78ae357640 1 --2- 172.21.15.143:0/3283507632 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f78a80708b0 0x7f78a80cf280 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7f789c00bb10 tx=0x7f789c01ec10 comp rx=0 tx=0).stop 2023-11-26T07:19:37.817 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:37.815+0000 7f78ae357640 1 -- 172.21.15.143:0/3283507632 shutdown_connections 2023-11-26T07:19:37.817 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:37.815+0000 7f78ae357640 1 --2- 172.21.15.143:0/3283507632 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f78a80708b0 0x7f78a80cf280 unknown :-1 s=CLOSED pgs=85 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:19:37.818 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:37.815+0000 7f78ae357640 1 --2- 172.21.15.143:0/3283507632 >> v2:172.21.15.188:3300/0 conn(0x7f78a8143ac0 0x7f78a8070370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:19:37.818 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:37.815+0000 7f78ae357640 1 -- 172.21.15.143:0/3283507632 >> 172.21.15.143:0/3283507632 conn(0x7f78a806a890 msgr2=0x7f78a806aca0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:19:37.818 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:37.815+0000 7f78ae357640 1 -- 172.21.15.143:0/3283507632 shutdown_connections 2023-11-26T07:19:37.818 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:37.815+0000 7f78ae357640 1 -- 172.21.15.143:0/3283507632 wait complete. 2023-11-26T07:19:37.819 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:37.815+0000 7f78ae357640 1 Processor -- start 2023-11-26T07:19:37.819 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:37.815+0000 7f78ae357640 1 -- start start 2023-11-26T07:19:37.820 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:37.815+0000 7f78ae357640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f78a80708b0 0x7f78a8156c10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:19:37.820 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:37.815+0000 7f78ae357640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f78a80cf7c0 0x7f78a8139bd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:19:37.820 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:37.815+0000 7f78ae357640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f78a8143ac0 0x7f78a8159290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:19:37.820 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:37.815+0000 7f78ae357640 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f78a81400a0 con 0x7f78a80708b0 2023-11-26T07:19:37.821 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:37.815+0000 7f78ae357640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f78a8158e00 con 0x7f78a8143ac0 2023-11-26T07:19:37.821 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:37.815+0000 7f78ae357640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f78a813fdc0 con 0x7f78a80cf7c0 2023-11-26T07:19:37.821 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:37.815+0000 7f78a7fff640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f78a80708b0 0x7f78a8156c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:19:37.821 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:37.815+0000 7f78a7fff640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f78a80708b0 0x7f78a8156c10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:52276/0 (socket says 172.21.15.143:52276) 2023-11-26T07:19:37.821 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:37.815+0000 7f78a7fff640 1 -- 172.21.15.143:0/1446499161 learned_addr learned my addr 172.21.15.143:0/1446499161 (peer_addr_for_me v2:172.21.15.143:0/0) 2023-11-26T07:19:37.822 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:37.819+0000 7f78a77fe640 1 --2- 172.21.15.143:0/1446499161 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f78a80cf7c0 0x7f78a8139bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:19:37.822 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:37.819+0000 7f78ac8f4640 1 --2- 172.21.15.143:0/1446499161 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f78a8143ac0 0x7f78a8159290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:19:37.822 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:37.819+0000 7f78a7fff640 1 -- 172.21.15.143:0/1446499161 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f78a80cf7c0 msgr2=0x7f78a8139bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:19:37.822 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:37.819+0000 7f78a7fff640 1 --2- 172.21.15.143:0/1446499161 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f78a80cf7c0 0x7f78a8139bd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:19:37.822 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:37.819+0000 7f78a7fff640 1 -- 172.21.15.143:0/1446499161 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f78a8143ac0 msgr2=0x7f78a8159290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:19:37.822 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:37.819+0000 7f78a7fff640 1 --2- 172.21.15.143:0/1446499161 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f78a8143ac0 0x7f78a8159290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:19:37.823 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:37.819+0000 7f78a7fff640 1 -- 172.21.15.143:0/1446499161 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f789c00b770 con 0x7f78a80708b0 2023-11-26T07:19:37.823 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:37.819+0000 7f78ac8f4640 1 --2- 172.21.15.143:0/1446499161 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f78a8143ac0 0x7f78a8159290 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-26T07:19:37.823 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:37.819+0000 7f78a7fff640 1 --2- 172.21.15.143:0/1446499161 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f78a80708b0 0x7f78a8156c10 secure :-1 s=READY pgs=585 cs=0 l=1 rev1=1 crypto rx=0x7f789800e870 tx=0x7f789800ec70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:19:37.823 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:37.819+0000 7f78a4ff9640 1 -- 172.21.15.143:0/1446499161 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7898009510 con 0x7f78a80708b0 2023-11-26T07:19:37.823 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:37.819+0000 7f78a4ff9640 1 -- 172.21.15.143:0/1446499161 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f789800eec0 con 0x7f78a80708b0 2023-11-26T07:19:37.823 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:37.819+0000 7f78a4ff9640 1 -- 172.21.15.143:0/1446499161 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f789800cb40 con 0x7f78a80708b0 2023-11-26T07:19:37.823 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:37.819+0000 7f78ae357640 1 -- 172.21.15.143:0/1446499161 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f78a81587b0 con 0x7f78a80708b0 2023-11-26T07:19:37.823 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:37.819+0000 7f78ae357640 1 -- 172.21.15.143:0/1446499161 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f78a8157150 con 0x7f78a80708b0 2023-11-26T07:19:37.823 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:37.819+0000 7f78ae357640 1 -- 172.21.15.143:0/1446499161 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f78a8067680 con 0x7f78a80708b0 2023-11-26T07:19:37.824 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:37.819+0000 7f78a4ff9640 1 -- 172.21.15.143:0/1446499161 <== mon.0 v2:172.21.15.143:3300/0 4 ==== mgrmap(e 7) v1 ==== 86005+0+0 (secure 0 0 0) 0x7f789800cca0 con 0x7f78a80708b0 2023-11-26T07:19:37.824 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:37.819+0000 7f78a4ff9640 1 --2- 172.21.15.143:0/1446499161 >> [v2:172.21.15.143:6832/2150557544,v1:172.21.15.143:6833/2150557544] conn(0x7f7894066620 0x7f7894068ae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:19:37.824 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:37.819+0000 7f78a4ff9640 1 -- 172.21.15.143:0/1446499161 <== mon.0 v2:172.21.15.143:3300/0 5 ==== osd_map(45..45 src has 1..45) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f7898084330 con 0x7f78a80708b0 2023-11-26T07:19:37.824 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:37.819+0000 7f78a77fe640 1 --2- 172.21.15.143:0/1446499161 >> [v2:172.21.15.143:6832/2150557544,v1:172.21.15.143:6833/2150557544] conn(0x7f7894066620 0x7f7894068ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:19:37.825 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:37.823+0000 7f78a77fe640 1 --2- 172.21.15.143:0/1446499161 >> [v2:172.21.15.143:6832/2150557544,v1:172.21.15.143:6833/2150557544] conn(0x7f7894066620 0x7f7894068ae0 secure :-1 s=READY pgs=294 cs=0 l=1 rev1=1 crypto rx=0x7f789c0b1000 tx=0x7f789c0a9070 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:19:37.828 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:37.823+0000 7f78a4ff9640 1 -- 172.21.15.143:0/1446499161 <== mon.0 v2:172.21.15.143:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172649 (secure 0 0 0) 0x7f7898051520 con 0x7f78a80708b0 2023-11-26T07:19:37.976 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:37.971+0000 7f78ae357640 1 -- 172.21.15.143:0/1446499161 --> [v2:172.21.15.143:6832/2150557544,v1:172.21.15.143:6833/2150557544] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f78a8071bd0 con 0x7f7894066620 2023-11-26T07:19:37.983 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:37.979+0000 7f78a4ff9640 1 -- 172.21.15.143:0/1446499161 <== mgr.4100 v2:172.21.15.143:6832/2150557544 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+289690 (secure 0 0 0) 0x7f78a8071bd0 con 0x7f7894066620 2023-11-26T07:19:37.986 INFO:teuthology.orchestra.run.smithi143.stdout: 2023-11-26T07:19:37.987 INFO:teuthology.orchestra.run.smithi143.stderr:dumped all 2023-11-26T07:19:37.988 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:37.983+0000 7f78ae357640 1 -- 172.21.15.143:0/1446499161 >> [v2:172.21.15.143:6832/2150557544,v1:172.21.15.143:6833/2150557544] conn(0x7f7894066620 msgr2=0x7f7894068ae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:19:37.989 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:37.983+0000 7f78ae357640 1 --2- 172.21.15.143:0/1446499161 >> [v2:172.21.15.143:6832/2150557544,v1:172.21.15.143:6833/2150557544] conn(0x7f7894066620 0x7f7894068ae0 secure :-1 s=READY pgs=294 cs=0 l=1 rev1=1 crypto rx=0x7f789c0b1000 tx=0x7f789c0a9070 comp rx=0 tx=0).stop 2023-11-26T07:19:37.989 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:37.983+0000 7f78ae357640 1 -- 172.21.15.143:0/1446499161 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f78a80708b0 msgr2=0x7f78a8156c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:19:37.989 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:37.983+0000 7f78ae357640 1 --2- 172.21.15.143:0/1446499161 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f78a80708b0 0x7f78a8156c10 secure :-1 s=READY pgs=585 cs=0 l=1 rev1=1 crypto rx=0x7f789800e870 tx=0x7f789800ec70 comp rx=0 tx=0).stop 2023-11-26T07:19:37.989 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:37.983+0000 7f78ae357640 1 -- 172.21.15.143:0/1446499161 shutdown_connections 2023-11-26T07:19:37.989 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:37.983+0000 7f78ae357640 1 --2- 172.21.15.143:0/1446499161 >> [v2:172.21.15.143:6832/2150557544,v1:172.21.15.143:6833/2150557544] conn(0x7f7894066620 0x7f7894068ae0 unknown :-1 s=CLOSED pgs=294 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:19:37.989 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:37.983+0000 7f78ae357640 1 --2- 172.21.15.143:0/1446499161 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f78a8143ac0 0x7f78a8159290 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:19:37.989 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:37.983+0000 7f78ae357640 1 --2- 172.21.15.143:0/1446499161 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f78a80cf7c0 0x7f78a8139bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:19:37.989 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:37.983+0000 7f78ae357640 1 --2- 172.21.15.143:0/1446499161 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f78a80708b0 0x7f78a8156c10 unknown :-1 s=CLOSED pgs=585 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:19:37.989 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:37.983+0000 7f78ae357640 1 -- 172.21.15.143:0/1446499161 >> 172.21.15.143:0/1446499161 conn(0x7f78a806a890 msgr2=0x7f78a806ca30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:19:37.989 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:37.983+0000 7f78ae357640 1 -- 172.21.15.143:0/1446499161 shutdown_connections 2023-11-26T07:19:37.989 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:37.983+0000 7f78ae357640 1 -- 172.21.15.143:0/1446499161 wait complete. 2023-11-26T07:19:38.007 INFO:teuthology.orchestra.run.smithi143.stdout:{"pg_ready":true,"pg_map":{"version":96,"stamp":"2023-11-26T07:19:37.712850+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592778,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":204,"num_write_kb":2430,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":156,"ondisk_log_size":156,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":185,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":221520,"kb_used_data":5352,"kb_used_omap":12,"kb_used_meta":216051,"kb_avail":754753200,"statfs":{"total":773094113280,"available":772867276800,"internally_reserved":0,"allocated":5480448,"data_stored":2906348,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12749,"internal_metadata":221236787},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":47,"apply_latency_ms":47,"commit_latency_ns":47000000,"apply_latency_ns":47000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"6.590757"},"pg_stats":[{"pgid":"7.3f","version":"0'0","reported_seq":11,"reported_epoch":45,"state":"active+clean","last_fresh":"2023-11-26T07:19:33.479237+0000","last_change":"2023-11-26T07:19:32.144487+0000","last_active":"2023-11-26T07:19:33.479237+0000","last_peered":"2023-11-26T07:19:33.479237+0000","last_clean":"2023-11-26T07:19:33.479237+0000","last_became_active":"2023-11-26T07:19:32.144186+0000","last_became_peered":"2023-11-26T07:19:32.144186+0000","last_unstale":"2023-11-26T07:19:33.479237+0000","last_undegraded":"2023-11-26T07:19:33.479237+0000","last_fullsized":"2023-11-26T07:19:33.479237+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:19:31.113498+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:19:31.113498+0000","last_clean_scrub_stamp":"2023-11-26T07:19:31.113498+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T11:39:14.851287+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.3e","version":"0'0","reported_seq":11,"reported_epoch":45,"state":"active+clean","last_fresh":"2023-11-26T07:19:35.452781+0000","last_change":"2023-11-26T07:19:32.149166+0000","last_active":"2023-11-26T07:19:35.452781+0000","last_peered":"2023-11-26T07:19:35.452781+0000","last_clean":"2023-11-26T07:19:35.452781+0000","last_became_active":"2023-11-26T07:19:32.148894+0000","last_became_peered":"2023-11-26T07:19:32.148894+0000","last_unstale":"2023-11-26T07:19:35.452781+0000","last_undegraded":"2023-11-26T07:19:35.452781+0000","last_fullsized":"2023-11-26T07:19:35.452781+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:19:31.113498+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:19:31.113498+0000","last_clean_scrub_stamp":"2023-11-26T07:19:31.113498+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T07:36:12.082239+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.3d","version":"0'0","reported_seq":11,"reported_epoch":45,"state":"active+clean","last_fresh":"2023-11-26T07:19:33.141402+0000","last_change":"2023-11-26T07:19:32.144951+0000","last_active":"2023-11-26T07:19:33.141402+0000","last_peered":"2023-11-26T07:19:33.141402+0000","last_clean":"2023-11-26T07:19:33.141402+0000","last_became_active":"2023-11-26T07:19:32.144447+0000","last_became_peered":"2023-11-26T07:19:32.144447+0000","last_unstale":"2023-11-26T07:19:33.141402+0000","last_undegraded":"2023-11-26T07:19:33.141402+0000","last_fullsized":"2023-11-26T07:19:33.141402+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:19:31.113498+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:19:31.113498+0000","last_clean_scrub_stamp":"2023-11-26T07:19:31.113498+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T17:07:30.335737+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.3c","version":"0'0","reported_seq":11,"reported_epoch":45,"state":"active+clean","last_fresh":"2023-11-26T07:19:35.781277+0000","last_change":"2023-11-26T07:19:32.151382+0000","last_active":"2023-11-26T07:19:35.781277+0000","last_peered":"2023-11-26T07:19:35.781277+0000","last_clean":"2023-11-26T07:19:35.781277+0000","last_became_active":"2023-11-26T07:19:32.151123+0000","last_became_peered":"2023-11-26T07:19:32.151123+0000","last_unstale":"2023-11-26T07:19:35.781277+0000","last_undegraded":"2023-11-26T07:19:35.781277+0000","last_fullsized":"2023-11-26T07:19:35.781277+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:19:31.113498+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:19:31.113498+0000","last_clean_scrub_stamp":"2023-11-26T07:19:31.113498+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T10:13:02.758558+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.3b","version":"0'0","reported_seq":11,"reported_epoch":45,"state":"active+clean","last_fresh":"2023-11-26T07:19:35.452479+0000","last_change":"2023-11-26T07:19:32.148869+0000","last_active":"2023-11-26T07:19:35.452479+0000","last_peered":"2023-11-26T07:19:35.452479+0000","last_clean":"2023-11-26T07:19:35.452479+0000","last_became_active":"2023-11-26T07:19:32.148584+0000","last_became_peered":"2023-11-26T07:19:32.148584+0000","last_unstale":"2023-11-26T07:19:35.452479+0000","last_undegraded":"2023-11-26T07:19:35.452479+0000","last_fullsized":"2023-11-26T07:19:35.452479+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:19:31.113498+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:19:31.113498+0000","last_clean_scrub_stamp":"2023-11-26T07:19:31.113498+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T11:13:03.640215+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.3a","version":"0'0","reported_seq":11,"reported_epoch":45,"state":"active+clean","last_fresh":"2023-11-26T07:19:35.045949+0000","last_change":"2023-11-26T07:19:32.147494+0000","last_active":"2023-11-26T07:19:35.045949+0000","last_peered":"2023-11-26T07:19:35.045949+0000","last_clean":"2023-11-26T07:19:35.045949+0000","last_became_active":"2023-11-26T07:19:32.147228+0000","last_became_peered":"2023-11-26T07:19:32.147228+0000","last_unstale":"2023-11-26T07:19:35.045949+0000","last_undegraded":"2023-11-26T07:19:35.045949+0000","last_fullsized":"2023-11-26T07:19:35.045949+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:19:31.113498+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:19:31.113498+0000","last_clean_scrub_stamp":"2023-11-26T07:19:31.113498+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T16:40:21.382290+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.39","version":"0'0","reported_seq":11,"reported_epoch":45,"state":"active+clean","last_fresh":"2023-11-26T07:19:33.478901+0000","last_change":"2023-11-26T07:19:32.161946+0000","last_active":"2023-11-26T07:19:33.478901+0000","last_peered":"2023-11-26T07:19:33.478901+0000","last_clean":"2023-11-26T07:19:33.478901+0000","last_became_active":"2023-11-26T07:19:32.161569+0000","last_became_peered":"2023-11-26T07:19:32.161569+0000","last_unstale":"2023-11-26T07:19:33.478901+0000","last_undegraded":"2023-11-26T07:19:33.478901+0000","last_fullsized":"2023-11-26T07:19:33.478901+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:19:31.113498+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:19:31.113498+0000","last_clean_scrub_stamp":"2023-11-26T07:19:31.113498+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T09:57:31.429974+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.38","version":"0'0","reported_seq":11,"reported_epoch":45,"state":"active+clean","last_fresh":"2023-11-26T07:19:35.780688+0000","last_change":"2023-11-26T07:19:32.150270+0000","last_active":"2023-11-26T07:19:35.780688+0000","last_peered":"2023-11-26T07:19:35.780688+0000","last_clean":"2023-11-26T07:19:35.780688+0000","last_became_active":"2023-11-26T07:19:32.149871+0000","last_became_peered":"2023-11-26T07:19:32.149871+0000","last_unstale":"2023-11-26T07:19:35.780688+0000","last_undegraded":"2023-11-26T07:19:35.780688+0000","last_fullsized":"2023-11-26T07:19:35.780688+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:19:31.113498+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:19:31.113498+0000","last_clean_scrub_stamp":"2023-11-26T07:19:31.113498+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T11:13:03.640215+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.37","version":"0'0","reported_seq":11,"reported_epoch":45,"state":"active+clean","last_fresh":"2023-11-26T07:19:35.781815+0000","last_change":"2023-11-26T07:19:32.162754+0000","last_active":"2023-11-26T07:19:35.781815+0000","last_peered":"2023-11-26T07:19:35.781815+0000","last_clean":"2023-11-26T07:19:35.781815+0000","last_became_active":"2023-11-26T07:19:32.162469+0000","last_became_peered":"2023-11-26T07:19:32.162469+0000","last_unstale":"2023-11-26T07:19:35.781815+0000","last_undegraded":"2023-11-26T07:19:35.781815+0000","last_fullsized":"2023-11-26T07:19:35.781815+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:19:31.113498+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:19:31.113498+0000","last_clean_scrub_stamp":"2023-11-26T07:19:31.113498+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T09:57:31.429974+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.36","version":"0'0","reported_seq":11,"reported_epoch":45,"state":"active+clean","last_fresh":"2023-11-26T07:19:35.781818+0000","last_change":"2023-11-26T07:19:32.152984+0000","last_active":"2023-11-26T07:19:35.781818+0000","last_peered":"2023-11-26T07:19:35.781818+0000","last_clean":"2023-11-26T07:19:35.781818+0000","last_became_active":"2023-11-26T07:19:32.152182+0000","last_became_peered":"2023-11-26T07:19:32.152182+0000","last_unstale":"2023-11-26T07:19:35.781818+0000","last_undegraded":"2023-11-26T07:19:35.781818+0000","last_fullsized":"2023-11-26T07:19:35.781818+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:19:31.113498+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:19:31.113498+0000","last_clean_scrub_stamp":"2023-11-26T07:19:31.113498+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T14:38:21.642327+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.35","version":"0'0","reported_seq":11,"reported_epoch":45,"state":"active+clean","last_fresh":"2023-11-26T07:19:35.781548+0000","last_change":"2023-11-26T07:19:32.169986+0000","last_active":"2023-11-26T07:19:35.781548+0000","last_peered":"2023-11-26T07:19:35.781548+0000","last_clean":"2023-11-26T07:19:35.781548+0000","last_became_active":"2023-11-26T07:19:32.166254+0000","last_became_peered":"2023-11-26T07:19:32.166254+0000","last_unstale":"2023-11-26T07:19:35.781548+0000","last_undegraded":"2023-11-26T07:19:35.781548+0000","last_fullsized":"2023-11-26T07:19:35.781548+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:19:31.113498+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:19:31.113498+0000","last_clean_scrub_stamp":"2023-11-26T07:19:31.113498+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T13:08:06.236414+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.34","version":"0'0","reported_seq":11,"reported_epoch":45,"state":"active+clean","last_fresh":"2023-11-26T07:19:35.452584+0000","last_change":"2023-11-26T07:19:32.152810+0000","last_active":"2023-11-26T07:19:35.452584+0000","last_peered":"2023-11-26T07:19:35.452584+0000","last_clean":"2023-11-26T07:19:35.452584+0000","last_became_active":"2023-11-26T07:19:32.152478+0000","last_became_peered":"2023-11-26T07:19:32.152478+0000","last_unstale":"2023-11-26T07:19:35.452584+0000","last_undegraded":"2023-11-26T07:19:35.452584+0000","last_fullsized":"2023-11-26T07:19:35.452584+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:19:31.113498+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:19:31.113498+0000","last_clean_scrub_stamp":"2023-11-26T07:19:31.113498+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T14:52:03.571786+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.33","version":"0'0","reported_seq":11,"reported_epoch":45,"state":"active+clean","last_fresh":"2023-11-26T07:19:33.141758+0000","last_change":"2023-11-26T07:19:32.144139+0000","last_active":"2023-11-26T07:19:33.141758+0000","last_peered":"2023-11-26T07:19:33.141758+0000","last_clean":"2023-11-26T07:19:33.141758+0000","last_became_active":"2023-11-26T07:19:32.143833+0000","last_became_peered":"2023-11-26T07:19:32.143833+0000","last_unstale":"2023-11-26T07:19:33.141758+0000","last_undegraded":"2023-11-26T07:19:33.141758+0000","last_fullsized":"2023-11-26T07:19:33.141758+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:19:31.113498+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:19:31.113498+0000","last_clean_scrub_stamp":"2023-11-26T07:19:31.113498+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T12:03:39.623971+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.32","version":"0'0","reported_seq":11,"reported_epoch":45,"state":"active+clean","last_fresh":"2023-11-26T07:19:35.781536+0000","last_change":"2023-11-26T07:19:32.162812+0000","last_active":"2023-11-26T07:19:35.781536+0000","last_peered":"2023-11-26T07:19:35.781536+0000","last_clean":"2023-11-26T07:19:35.781536+0000","last_became_active":"2023-11-26T07:19:32.162479+0000","last_became_peered":"2023-11-26T07:19:32.162479+0000","last_unstale":"2023-11-26T07:19:35.781536+0000","last_undegraded":"2023-11-26T07:19:35.781536+0000","last_fullsized":"2023-11-26T07:19:35.781536+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:19:31.113498+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:19:31.113498+0000","last_clean_scrub_stamp":"2023-11-26T07:19:31.113498+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T17:06:09.448286+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.31","version":"0'0","reported_seq":11,"reported_epoch":45,"state":"active+clean","last_fresh":"2023-11-26T07:19:35.045857+0000","last_change":"2023-11-26T07:19:32.151586+0000","last_active":"2023-11-26T07:19:35.045857+0000","last_peered":"2023-11-26T07:19:35.045857+0000","last_clean":"2023-11-26T07:19:35.045857+0000","last_became_active":"2023-11-26T07:19:32.151044+0000","last_became_peered":"2023-11-26T07:19:32.151044+0000","last_unstale":"2023-11-26T07:19:35.045857+0000","last_undegraded":"2023-11-26T07:19:35.045857+0000","last_fullsized":"2023-11-26T07:19:35.045857+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:19:31.113498+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:19:31.113498+0000","last_clean_scrub_stamp":"2023-11-26T07:19:31.113498+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T08:05:46.483096+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.30","version":"0'0","reported_seq":11,"reported_epoch":45,"state":"active+clean","last_fresh":"2023-11-26T07:19:35.779486+0000","last_change":"2023-11-26T07:19:32.141346+0000","last_active":"2023-11-26T07:19:35.779486+0000","last_peered":"2023-11-26T07:19:35.779486+0000","last_clean":"2023-11-26T07:19:35.779486+0000","last_became_active":"2023-11-26T07:19:32.140936+0000","last_became_peered":"2023-11-26T07:19:32.140936+0000","last_unstale":"2023-11-26T07:19:35.779486+0000","last_undegraded":"2023-11-26T07:19:35.779486+0000","last_fullsized":"2023-11-26T07:19:35.779486+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:19:31.113498+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:19:31.113498+0000","last_clean_scrub_stamp":"2023-11-26T07:19:31.113498+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T16:20:24.322478+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.2f","version":"0'0","reported_seq":11,"reported_epoch":45,"state":"active+clean","last_fresh":"2023-11-26T07:19:33.141851+0000","last_change":"2023-11-26T07:19:32.146031+0000","last_active":"2023-11-26T07:19:33.141851+0000","last_peered":"2023-11-26T07:19:33.141851+0000","last_clean":"2023-11-26T07:19:33.141851+0000","last_became_active":"2023-11-26T07:19:32.145722+0000","last_became_peered":"2023-11-26T07:19:32.145722+0000","last_unstale":"2023-11-26T07:19:33.141851+0000","last_undegraded":"2023-11-26T07:19:33.141851+0000","last_fullsized":"2023-11-26T07:19:33.141851+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:19:31.113498+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:19:31.113498+0000","last_clean_scrub_stamp":"2023-11-26T07:19:31.113498+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T14:52:03.571786+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.2e","version":"0'0","reported_seq":11,"reported_epoch":45,"state":"active+clean","last_fresh":"2023-11-26T07:19:33.141559+0000","last_change":"2023-11-26T07:19:32.152229+0000","last_active":"2023-11-26T07:19:33.141559+0000","last_peered":"2023-11-26T07:19:33.141559+0000","last_clean":"2023-11-26T07:19:33.141559+0000","last_became_active":"2023-11-26T07:19:32.151091+0000","last_became_peered":"2023-11-26T07:19:32.151091+0000","last_unstale":"2023-11-26T07:19:33.141559+0000","last_undegraded":"2023-11-26T07:19:33.141559+0000","last_fullsized":"2023-11-26T07:19:33.141559+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:19:31.113498+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:19:31.113498+0000","last_clean_scrub_stamp":"2023-11-26T07:19:31.113498+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T16:13:48.077132+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.2d","version":"0'0","reported_seq":11,"reported_epoch":45,"state":"active+clean","last_fresh":"2023-11-26T07:19:35.779927+0000","last_change":"2023-11-26T07:19:32.142222+0000","last_active":"2023-11-26T07:19:35.779927+0000","last_peered":"2023-11-26T07:19:35.779927+0000","last_clean":"2023-11-26T07:19:35.779927+0000","last_became_active":"2023-11-26T07:19:32.141931+0000","last_became_peered":"2023-11-26T07:19:32.141931+0000","last_unstale":"2023-11-26T07:19:35.779927+0000","last_undegraded":"2023-11-26T07:19:35.779927+0000","last_fullsized":"2023-11-26T07:19:35.779927+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:19:31.113498+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:19:31.113498+0000","last_clean_scrub_stamp":"2023-11-26T07:19:31.113498+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T19:14:08.178758+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.2c","version":"0'0","reported_seq":11,"reported_epoch":45,"state":"active+clean","last_fresh":"2023-11-26T07:19:33.141948+0000","last_change":"2023-11-26T07:19:32.144361+0000","last_active":"2023-11-26T07:19:33.141948+0000","last_peered":"2023-11-26T07:19:33.141948+0000","last_clean":"2023-11-26T07:19:33.141948+0000","last_became_active":"2023-11-26T07:19:32.144067+0000","last_became_peered":"2023-11-26T07:19:32.144067+0000","last_unstale":"2023-11-26T07:19:33.141948+0000","last_undegraded":"2023-11-26T07:19:33.141948+0000","last_fullsized":"2023-11-26T07:19:33.141948+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:19:31.113498+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:19:31.113498+0000","last_clean_scrub_stamp":"2023-11-26T07:19:31.113498+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T08:29:00.060229+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.2b","version":"0'0","reported_seq":11,"reported_epoch":45,"state":"active+clean","last_fresh":"2023-11-26T07:19:35.046029+0000","last_change":"2023-11-26T07:19:32.143656+0000","last_active":"2023-11-26T07:19:35.046029+0000","last_peered":"2023-11-26T07:19:35.046029+0000","last_clean":"2023-11-26T07:19:35.046029+0000","last_became_active":"2023-11-26T07:19:32.143366+0000","last_became_peered":"2023-11-26T07:19:32.143366+0000","last_unstale":"2023-11-26T07:19:35.046029+0000","last_undegraded":"2023-11-26T07:19:35.046029+0000","last_fullsized":"2023-11-26T07:19:35.046029+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:19:31.113498+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:19:31.113498+0000","last_clean_scrub_stamp":"2023-11-26T07:19:31.113498+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T14:26:16.211661+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.2a","version":"0'0","reported_seq":11,"reported_epoch":45,"state":"active+clean","last_fresh":"2023-11-26T07:19:35.781581+0000","last_change":"2023-11-26T07:19:32.157944+0000","last_active":"2023-11-26T07:19:35.781581+0000","last_peered":"2023-11-26T07:19:35.781581+0000","last_clean":"2023-11-26T07:19:35.781581+0000","last_became_active":"2023-11-26T07:19:32.157635+0000","last_became_peered":"2023-11-26T07:19:32.157635+0000","last_unstale":"2023-11-26T07:19:35.781581+0000","last_undegraded":"2023-11-26T07:19:35.781581+0000","last_fullsized":"2023-11-26T07:19:35.781581+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:19:31.113498+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:19:31.113498+0000","last_clean_scrub_stamp":"2023-11-26T07:19:31.113498+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T13:57:09.650424+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.29","version":"0'0","reported_seq":11,"reported_epoch":45,"state":"active+clean","last_fresh":"2023-11-26T07:19:35.779675+0000","last_change":"2023-11-26T07:19:32.141578+0000","last_active":"2023-11-26T07:19:35.779675+0000","last_peered":"2023-11-26T07:19:35.779675+0000","last_clean":"2023-11-26T07:19:35.779675+0000","last_became_active":"2023-11-26T07:19:32.141062+0000","last_became_peered":"2023-11-26T07:19:32.141062+0000","last_unstale":"2023-11-26T07:19:35.779675+0000","last_undegraded":"2023-11-26T07:19:35.779675+0000","last_fullsized":"2023-11-26T07:19:35.779675+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:19:31.113498+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:19:31.113498+0000","last_clean_scrub_stamp":"2023-11-26T07:19:31.113498+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T19:19:00.198799+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.28","version":"0'0","reported_seq":11,"reported_epoch":45,"state":"active+clean","last_fresh":"2023-11-26T07:19:35.779529+0000","last_change":"2023-11-26T07:19:32.142515+0000","last_active":"2023-11-26T07:19:35.779529+0000","last_peered":"2023-11-26T07:19:35.779529+0000","last_clean":"2023-11-26T07:19:35.779529+0000","last_became_active":"2023-11-26T07:19:32.141329+0000","last_became_peered":"2023-11-26T07:19:32.141329+0000","last_unstale":"2023-11-26T07:19:35.779529+0000","last_undegraded":"2023-11-26T07:19:35.779529+0000","last_fullsized":"2023-11-26T07:19:35.779529+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:19:31.113498+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:19:31.113498+0000","last_clean_scrub_stamp":"2023-11-26T07:19:31.113498+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T12:39:07.528966+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.27","version":"0'0","reported_seq":11,"reported_epoch":45,"state":"active+clean","last_fresh":"2023-11-26T07:19:35.780024+0000","last_change":"2023-11-26T07:19:32.165013+0000","last_active":"2023-11-26T07:19:35.780024+0000","last_peered":"2023-11-26T07:19:35.780024+0000","last_clean":"2023-11-26T07:19:35.780024+0000","last_became_active":"2023-11-26T07:19:32.164626+0000","last_became_peered":"2023-11-26T07:19:32.164626+0000","last_unstale":"2023-11-26T07:19:35.780024+0000","last_undegraded":"2023-11-26T07:19:35.780024+0000","last_fullsized":"2023-11-26T07:19:35.780024+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:19:31.113498+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:19:31.113498+0000","last_clean_scrub_stamp":"2023-11-26T07:19:31.113498+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T15:38:29.544244+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.26","version":"0'0","reported_seq":11,"reported_epoch":45,"state":"active+clean","last_fresh":"2023-11-26T07:19:33.141894+0000","last_change":"2023-11-26T07:19:32.148455+0000","last_active":"2023-11-26T07:19:33.141894+0000","last_peered":"2023-11-26T07:19:33.141894+0000","last_clean":"2023-11-26T07:19:33.141894+0000","last_became_active":"2023-11-26T07:19:32.148153+0000","last_became_peered":"2023-11-26T07:19:32.148153+0000","last_unstale":"2023-11-26T07:19:33.141894+0000","last_undegraded":"2023-11-26T07:19:33.141894+0000","last_fullsized":"2023-11-26T07:19:33.141894+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:19:31.113498+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:19:31.113498+0000","last_clean_scrub_stamp":"2023-11-26T07:19:31.113498+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T15:42:46.000146+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.25","version":"0'0","reported_seq":11,"reported_epoch":45,"state":"active+clean","last_fresh":"2023-11-26T07:19:33.141651+0000","last_change":"2023-11-26T07:19:32.144732+0000","last_active":"2023-11-26T07:19:33.141651+0000","last_peered":"2023-11-26T07:19:33.141651+0000","last_clean":"2023-11-26T07:19:33.141651+0000","last_became_active":"2023-11-26T07:19:32.144080+0000","last_became_peered":"2023-11-26T07:19:32.144080+0000","last_unstale":"2023-11-26T07:19:33.141651+0000","last_undegraded":"2023-11-26T07:19:33.141651+0000","last_fullsized":"2023-11-26T07:19:33.141651+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:19:31.113498+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:19:31.113498+0000","last_clean_scrub_stamp":"2023-11-26T07:19:31.113498+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T11:13:03.640215+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.24","version":"0'0","reported_seq":11,"reported_epoch":45,"state":"active+clean","last_fresh":"2023-11-26T07:19:35.452719+0000","last_change":"2023-11-26T07:19:32.152207+0000","last_active":"2023-11-26T07:19:35.452719+0000","last_peered":"2023-11-26T07:19:35.452719+0000","last_clean":"2023-11-26T07:19:35.452719+0000","last_became_active":"2023-11-26T07:19:32.151884+0000","last_became_peered":"2023-11-26T07:19:32.151884+0000","last_unstale":"2023-11-26T07:19:35.452719+0000","last_undegraded":"2023-11-26T07:19:35.452719+0000","last_fullsized":"2023-11-26T07:19:35.452719+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:19:31.113498+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:19:31.113498+0000","last_clean_scrub_stamp":"2023-11-26T07:19:31.113498+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T10:13:02.758558+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.23","version":"0'0","reported_seq":11,"reported_epoch":45,"state":"active+clean","last_fresh":"2023-11-26T07:19:35.779763+0000","last_change":"2023-11-26T07:19:32.161293+0000","last_active":"2023-11-26T07:19:35.779763+0000","last_peered":"2023-11-26T07:19:35.779763+0000","last_clean":"2023-11-26T07:19:35.779763+0000","last_became_active":"2023-11-26T07:19:32.160980+0000","last_became_peered":"2023-11-26T07:19:32.160980+0000","last_unstale":"2023-11-26T07:19:35.779763+0000","last_undegraded":"2023-11-26T07:19:35.779763+0000","last_fullsized":"2023-11-26T07:19:35.779763+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:19:31.113498+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:19:31.113498+0000","last_clean_scrub_stamp":"2023-11-26T07:19:31.113498+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T15:54:06.012692+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.22","version":"0'0","reported_seq":11,"reported_epoch":45,"state":"active+clean","last_fresh":"2023-11-26T07:19:33.141366+0000","last_change":"2023-11-26T07:19:32.148594+0000","last_active":"2023-11-26T07:19:33.141366+0000","last_peered":"2023-11-26T07:19:33.141366+0000","last_clean":"2023-11-26T07:19:33.141366+0000","last_became_active":"2023-11-26T07:19:32.146474+0000","last_became_peered":"2023-11-26T07:19:32.146474+0000","last_unstale":"2023-11-26T07:19:33.141366+0000","last_undegraded":"2023-11-26T07:19:33.141366+0000","last_fullsized":"2023-11-26T07:19:33.141366+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:19:31.113498+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:19:31.113498+0000","last_clean_scrub_stamp":"2023-11-26T07:19:31.113498+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T13:58:10.832349+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.21","version":"0'0","reported_seq":11,"reported_epoch":45,"state":"active+clean","last_fresh":"2023-11-26T07:19:35.045939+0000","last_change":"2023-11-26T07:19:32.146135+0000","last_active":"2023-11-26T07:19:35.045939+0000","last_peered":"2023-11-26T07:19:35.045939+0000","last_clean":"2023-11-26T07:19:35.045939+0000","last_became_active":"2023-11-26T07:19:32.145871+0000","last_became_peered":"2023-11-26T07:19:32.145871+0000","last_unstale":"2023-11-26T07:19:35.045939+0000","last_undegraded":"2023-11-26T07:19:35.045939+0000","last_fullsized":"2023-11-26T07:19:35.045939+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:19:31.113498+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:19:31.113498+0000","last_clean_scrub_stamp":"2023-11-26T07:19:31.113498+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T11:38:43.936569+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.20","version":"0'0","reported_seq":11,"reported_epoch":45,"state":"active+clean","last_fresh":"2023-11-26T07:19:33.141620+0000","last_change":"2023-11-26T07:19:32.162325+0000","last_active":"2023-11-26T07:19:33.141620+0000","last_peered":"2023-11-26T07:19:33.141620+0000","last_clean":"2023-11-26T07:19:33.141620+0000","last_became_active":"2023-11-26T07:19:32.161954+0000","last_became_peered":"2023-11-26T07:19:32.161954+0000","last_unstale":"2023-11-26T07:19:33.141620+0000","last_undegraded":"2023-11-26T07:19:33.141620+0000","last_fullsized":"2023-11-26T07:19:33.141620+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:19:31.113498+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:19:31.113498+0000","last_clean_scrub_stamp":"2023-11-26T07:19:31.113498+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T08:37:11.440586+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.1e","version":"0'0","reported_seq":11,"reported_epoch":45,"state":"active+clean","last_fresh":"2023-11-26T07:19:35.781173+0000","last_change":"2023-11-26T07:19:32.163638+0000","last_active":"2023-11-26T07:19:35.781173+0000","last_peered":"2023-11-26T07:19:35.781173+0000","last_clean":"2023-11-26T07:19:35.781173+0000","last_became_active":"2023-11-26T07:19:32.163380+0000","last_became_peered":"2023-11-26T07:19:32.163380+0000","last_unstale":"2023-11-26T07:19:35.781173+0000","last_undegraded":"2023-11-26T07:19:35.781173+0000","last_fullsized":"2023-11-26T07:19:35.781173+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:19:31.113498+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:19:31.113498+0000","last_clean_scrub_stamp":"2023-11-26T07:19:31.113498+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T10:27:02.955872+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.1f","version":"44'4","reported_seq":16,"reported_epoch":44,"state":"active+clean","last_fresh":"2023-11-26T07:19:32.172842+0000","last_change":"2023-11-26T07:19:31.148359+0000","last_active":"2023-11-26T07:19:32.172842+0000","last_peered":"2023-11-26T07:19:32.172842+0000","last_clean":"2023-11-26T07:19:32.172842+0000","last_became_active":"2023-11-26T07:19:31.148066+0000","last_became_peered":"2023-11-26T07:19:31.148066+0000","last_unstale":"2023-11-26T07:19:32.172842+0000","last_undegraded":"2023-11-26T07:19:32.172842+0000","last_fullsized":"2023-11-26T07:19:32.172842+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:19:30.106639+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:19:30.106639+0000","last_clean_scrub_stamp":"2023-11-26T07:19:30.106639+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T12:08:31.504338+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.1f","version":"0'0","reported_seq":11,"reported_epoch":45,"state":"active+clean","last_fresh":"2023-11-26T07:19:35.780092+0000","last_change":"2023-11-26T07:19:32.142782+0000","last_active":"2023-11-26T07:19:35.780092+0000","last_peered":"2023-11-26T07:19:35.780092+0000","last_clean":"2023-11-26T07:19:35.780092+0000","last_became_active":"2023-11-26T07:19:32.142493+0000","last_became_peered":"2023-11-26T07:19:32.142493+0000","last_unstale":"2023-11-26T07:19:35.780092+0000","last_undegraded":"2023-11-26T07:19:35.780092+0000","last_fullsized":"2023-11-26T07:19:35.780092+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:19:31.113498+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:19:31.113498+0000","last_clean_scrub_stamp":"2023-11-26T07:19:31.113498+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T10:27:02.955872+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.1e","version":"44'1","reported_seq":13,"reported_epoch":44,"state":"active+clean","last_fresh":"2023-11-26T07:19:32.159452+0000","last_change":"2023-11-26T07:19:31.144305+0000","last_active":"2023-11-26T07:19:32.159452+0000","last_peered":"2023-11-26T07:19:32.159452+0000","last_clean":"2023-11-26T07:19:32.159452+0000","last_became_active":"2023-11-26T07:19:31.143063+0000","last_became_peered":"2023-11-26T07:19:31.143063+0000","last_unstale":"2023-11-26T07:19:32.159452+0000","last_undegraded":"2023-11-26T07:19:32.159452+0000","last_fullsized":"2023-11-26T07:19:32.159452+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:19:30.106639+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:19:30.106639+0000","last_clean_scrub_stamp":"2023-11-26T07:19:30.106639+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T19:14:07.171899+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.1c","version":"0'0","reported_seq":11,"reported_epoch":45,"state":"active+clean","last_fresh":"2023-11-26T07:19:35.452754+0000","last_change":"2023-11-26T07:19:32.145834+0000","last_active":"2023-11-26T07:19:35.452754+0000","last_peered":"2023-11-26T07:19:35.452754+0000","last_clean":"2023-11-26T07:19:35.452754+0000","last_became_active":"2023-11-26T07:19:32.145557+0000","last_became_peered":"2023-11-26T07:19:32.145557+0000","last_unstale":"2023-11-26T07:19:35.452754+0000","last_undegraded":"2023-11-26T07:19:35.452754+0000","last_fullsized":"2023-11-26T07:19:35.452754+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:19:31.113498+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:19:31.113498+0000","last_clean_scrub_stamp":"2023-11-26T07:19:31.113498+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T12:30:43.967303+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.1d","version":"44'2","reported_seq":14,"reported_epoch":44,"state":"active+clean","last_fresh":"2023-11-26T07:19:32.173683+0000","last_change":"2023-11-26T07:19:31.150313+0000","last_active":"2023-11-26T07:19:32.173683+0000","last_peered":"2023-11-26T07:19:32.173683+0000","last_clean":"2023-11-26T07:19:32.173683+0000","last_became_active":"2023-11-26T07:19:31.150042+0000","last_became_peered":"2023-11-26T07:19:31.150042+0000","last_unstale":"2023-11-26T07:19:32.173683+0000","last_undegraded":"2023-11-26T07:19:32.173683+0000","last_fullsized":"2023-11-26T07:19:32.173683+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:19:30.106639+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:19:30.106639+0000","last_clean_scrub_stamp":"2023-11-26T07:19:30.106639+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T15:54:05.005833+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.1d","version":"0'0","reported_seq":11,"reported_epoch":45,"state":"active+clean","last_fresh":"2023-11-26T07:19:35.781089+0000","last_change":"2023-11-26T07:19:32.162941+0000","last_active":"2023-11-26T07:19:35.781089+0000","last_peered":"2023-11-26T07:19:35.781089+0000","last_clean":"2023-11-26T07:19:35.781089+0000","last_became_active":"2023-11-26T07:19:32.162043+0000","last_became_peered":"2023-11-26T07:19:32.162043+0000","last_unstale":"2023-11-26T07:19:35.781089+0000","last_undegraded":"2023-11-26T07:19:35.781089+0000","last_fullsized":"2023-11-26T07:19:35.781089+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:19:31.113498+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:19:31.113498+0000","last_clean_scrub_stamp":"2023-11-26T07:19:31.113498+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T15:38:29.544244+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.1c","version":"44'2","reported_seq":14,"reported_epoch":44,"state":"active+clean","last_fresh":"2023-11-26T07:19:32.167534+0000","last_change":"2023-11-26T07:19:31.141667+0000","last_active":"2023-11-26T07:19:32.167534+0000","last_peered":"2023-11-26T07:19:32.167534+0000","last_clean":"2023-11-26T07:19:32.167534+0000","last_became_active":"2023-11-26T07:19:31.141393+0000","last_became_peered":"2023-11-26T07:19:31.141393+0000","last_unstale":"2023-11-26T07:19:32.167534+0000","last_undegraded":"2023-11-26T07:19:32.167534+0000","last_fullsized":"2023-11-26T07:19:32.167534+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:19:30.106639+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:19:30.106639+0000","last_clean_scrub_stamp":"2023-11-26T07:19:30.106639+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T07:36:11.075380+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.1a","version":"0'0","reported_seq":11,"reported_epoch":45,"state":"active+clean","last_fresh":"2023-11-26T07:19:33.141556+0000","last_change":"2023-11-26T07:19:32.149961+0000","last_active":"2023-11-26T07:19:33.141556+0000","last_peered":"2023-11-26T07:19:33.141556+0000","last_clean":"2023-11-26T07:19:33.141556+0000","last_became_active":"2023-11-26T07:19:32.149627+0000","last_became_peered":"2023-11-26T07:19:32.149627+0000","last_unstale":"2023-11-26T07:19:33.141556+0000","last_undegraded":"2023-11-26T07:19:33.141556+0000","last_fullsized":"2023-11-26T07:19:33.141556+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:19:31.113498+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:19:31.113498+0000","last_clean_scrub_stamp":"2023-11-26T07:19:31.113498+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T08:11:43.416968+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.1b","version":"44'1","reported_seq":13,"reported_epoch":44,"state":"active+clean","last_fresh":"2023-11-26T07:19:32.173306+0000","last_change":"2023-11-26T07:19:31.149354+0000","last_active":"2023-11-26T07:19:32.173306+0000","last_peered":"2023-11-26T07:19:32.173306+0000","last_clean":"2023-11-26T07:19:32.173306+0000","last_became_active":"2023-11-26T07:19:31.149092+0000","last_became_peered":"2023-11-26T07:19:31.149092+0000","last_unstale":"2023-11-26T07:19:32.173306+0000","last_undegraded":"2023-11-26T07:19:32.173306+0000","last_fullsized":"2023-11-26T07:19:32.173306+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:19:30.106639+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:19:30.106639+0000","last_clean_scrub_stamp":"2023-11-26T07:19:30.106639+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T17:06:08.441427+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.1b","version":"0'0","reported_seq":11,"reported_epoch":45,"state":"active+clean","last_fresh":"2023-11-26T07:19:35.781370+0000","last_change":"2023-11-26T07:19:32.147918+0000","last_active":"2023-11-26T07:19:35.781370+0000","last_peered":"2023-11-26T07:19:35.781370+0000","last_clean":"2023-11-26T07:19:35.781370+0000","last_became_active":"2023-11-26T07:19:32.144906+0000","last_became_peered":"2023-11-26T07:19:32.144906+0000","last_unstale":"2023-11-26T07:19:35.781370+0000","last_undegraded":"2023-11-26T07:19:35.781370+0000","last_fullsized":"2023-11-26T07:19:35.781370+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:19:31.113498+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:19:31.113498+0000","last_clean_scrub_stamp":"2023-11-26T07:19:31.113498+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T07:36:12.082239+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.1a","version":"44'1","reported_seq":13,"reported_epoch":44,"state":"active+clean","last_fresh":"2023-11-26T07:19:32.158326+0000","last_change":"2023-11-26T07:19:31.139455+0000","last_active":"2023-11-26T07:19:32.158326+0000","last_peered":"2023-11-26T07:19:32.158326+0000","last_clean":"2023-11-26T07:19:32.158326+0000","last_became_active":"2023-11-26T07:19:31.139009+0000","last_became_peered":"2023-11-26T07:19:31.139009+0000","last_unstale":"2023-11-26T07:19:32.158326+0000","last_undegraded":"2023-11-26T07:19:32.158326+0000","last_fullsized":"2023-11-26T07:19:32.158326+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:19:30.106639+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:19:30.106639+0000","last_clean_scrub_stamp":"2023-11-26T07:19:30.106639+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T12:54:51.008938+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.18","version":"0'0","reported_seq":11,"reported_epoch":45,"state":"active+clean","last_fresh":"2023-11-26T07:19:35.780736+0000","last_change":"2023-11-26T07:19:32.150102+0000","last_active":"2023-11-26T07:19:35.780736+0000","last_peered":"2023-11-26T07:19:35.780736+0000","last_clean":"2023-11-26T07:19:35.780736+0000","last_became_active":"2023-11-26T07:19:32.149530+0000","last_became_peered":"2023-11-26T07:19:32.149530+0000","last_unstale":"2023-11-26T07:19:35.780736+0000","last_undegraded":"2023-11-26T07:19:35.780736+0000","last_fullsized":"2023-11-26T07:19:35.780736+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:19:31.113498+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:19:31.113498+0000","last_clean_scrub_stamp":"2023-11-26T07:19:31.113498+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T12:30:43.967303+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.19","version":"0'0","reported_seq":12,"reported_epoch":44,"state":"active+clean","last_fresh":"2023-11-26T07:19:32.131503+0000","last_change":"2023-11-26T07:19:31.146327+0000","last_active":"2023-11-26T07:19:32.131503+0000","last_peered":"2023-11-26T07:19:32.131503+0000","last_clean":"2023-11-26T07:19:32.131503+0000","last_became_active":"2023-11-26T07:19:31.145985+0000","last_became_peered":"2023-11-26T07:19:31.145985+0000","last_unstale":"2023-11-26T07:19:32.131503+0000","last_undegraded":"2023-11-26T07:19:32.131503+0000","last_fullsized":"2023-11-26T07:19:32.131503+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:19:30.106639+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:19:30.106639+0000","last_clean_scrub_stamp":"2023-11-26T07:19:30.106639+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T17:48:35.887457+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.19","version":"0'0","reported_seq":11,"reported_epoch":45,"state":"active+clean","last_fresh":"2023-11-26T07:19:35.779506+0000","last_change":"2023-11-26T07:19:32.141885+0000","last_active":"2023-11-26T07:19:35.779506+0000","last_peered":"2023-11-26T07:19:35.779506+0000","last_clean":"2023-11-26T07:19:35.779506+0000","last_became_active":"2023-11-26T07:19:32.141579+0000","last_became_peered":"2023-11-26T07:19:32.141579+0000","last_unstale":"2023-11-26T07:19:35.779506+0000","last_undegraded":"2023-11-26T07:19:35.779506+0000","last_fullsized":"2023-11-26T07:19:35.779506+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:19:31.113498+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:19:31.113498+0000","last_clean_scrub_stamp":"2023-11-26T07:19:31.113498+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T12:54:52.015797+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.18","version":"0'0","reported_seq":12,"reported_epoch":44,"state":"active+clean","last_fresh":"2023-11-26T07:19:32.134241+0000","last_change":"2023-11-26T07:19:31.140258+0000","last_active":"2023-11-26T07:19:32.134241+0000","last_peered":"2023-11-26T07:19:32.134241+0000","last_clean":"2023-11-26T07:19:32.134241+0000","last_became_active":"2023-11-26T07:19:31.140009+0000","last_became_peered":"2023-11-26T07:19:31.140009+0000","last_unstale":"2023-11-26T07:19:32.134241+0000","last_undegraded":"2023-11-26T07:19:32.134241+0000","last_fullsized":"2023-11-26T07:19:32.134241+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:19:30.106639+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:19:30.106639+0000","last_clean_scrub_stamp":"2023-11-26T07:19:30.106639+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T15:54:05.005833+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.16","version":"0'0","reported_seq":11,"reported_epoch":45,"state":"active+clean","last_fresh":"2023-11-26T07:19:33.141587+0000","last_change":"2023-11-26T07:19:32.150893+0000","last_active":"2023-11-26T07:19:33.141587+0000","last_peered":"2023-11-26T07:19:33.141587+0000","last_clean":"2023-11-26T07:19:33.141587+0000","last_became_active":"2023-11-26T07:19:32.150499+0000","last_became_peered":"2023-11-26T07:19:32.150499+0000","last_unstale":"2023-11-26T07:19:33.141587+0000","last_undegraded":"2023-11-26T07:19:33.141587+0000","last_fullsized":"2023-11-26T07:19:33.141587+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:19:31.113498+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:19:31.113498+0000","last_clean_scrub_stamp":"2023-11-26T07:19:31.113498+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T12:08:32.511197+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.17","version":"0'0","reported_seq":12,"reported_epoch":44,"state":"active+clean","last_fresh":"2023-11-26T07:19:32.135153+0000","last_change":"2023-11-26T07:19:31.134552+0000","last_active":"2023-11-26T07:19:32.135153+0000","last_peered":"2023-11-26T07:19:32.135153+0000","last_clean":"2023-11-26T07:19:32.135153+0000","last_became_active":"2023-11-26T07:19:31.134241+0000","last_became_peered":"2023-11-26T07:19:31.134241+0000","last_unstale":"2023-11-26T07:19:32.135153+0000","last_undegraded":"2023-11-26T07:19:32.135153+0000","last_fullsized":"2023-11-26T07:19:32.135153+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:19:30.106639+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:19:30.106639+0000","last_clean_scrub_stamp":"2023-11-26T07:19:30.106639+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T15:38:28.537385+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.17","version":"0'0","reported_seq":11,"reported_epoch":45,"state":"active+clean","last_fresh":"2023-11-26T07:19:33.479303+0000","last_change":"2023-11-26T07:19:32.145750+0000","last_active":"2023-11-26T07:19:33.479303+0000","last_peered":"2023-11-26T07:19:33.479303+0000","last_clean":"2023-11-26T07:19:33.479303+0000","last_became_active":"2023-11-26T07:19:32.145490+0000","last_became_peered":"2023-11-26T07:19:32.145490+0000","last_unstale":"2023-11-26T07:19:33.479303+0000","last_undegraded":"2023-11-26T07:19:33.479303+0000","last_fullsized":"2023-11-26T07:19:33.479303+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:19:31.113498+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:19:31.113498+0000","last_clean_scrub_stamp":"2023-11-26T07:19:31.113498+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T08:05:46.483096+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.16","version":"0'0","reported_seq":12,"reported_epoch":44,"state":"active+clean","last_fresh":"2023-11-26T07:19:32.136290+0000","last_change":"2023-11-26T07:19:31.144647+0000","last_active":"2023-11-26T07:19:32.136290+0000","last_peered":"2023-11-26T07:19:32.136290+0000","last_clean":"2023-11-26T07:19:32.136290+0000","last_became_active":"2023-11-26T07:19:31.143606+0000","last_became_peered":"2023-11-26T07:19:31.143606+0000","last_unstale":"2023-11-26T07:19:32.136290+0000","last_undegraded":"2023-11-26T07:19:32.136290+0000","last_fullsized":"2023-11-26T07:19:32.136290+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:19:30.106639+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:19:30.106639+0000","last_clean_scrub_stamp":"2023-11-26T07:19:30.106639+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T17:07:29.328878+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.14","version":"0'0","reported_seq":11,"reported_epoch":45,"state":"active+clean","last_fresh":"2023-11-26T07:19:35.046247+0000","last_change":"2023-11-26T07:19:32.157436+0000","last_active":"2023-11-26T07:19:35.046247+0000","last_peered":"2023-11-26T07:19:35.046247+0000","last_clean":"2023-11-26T07:19:35.046247+0000","last_became_active":"2023-11-26T07:19:32.157158+0000","last_became_peered":"2023-11-26T07:19:32.157158+0000","last_unstale":"2023-11-26T07:19:35.046247+0000","last_undegraded":"2023-11-26T07:19:35.046247+0000","last_fullsized":"2023-11-26T07:19:35.046247+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:19:31.113498+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:19:31.113498+0000","last_clean_scrub_stamp":"2023-11-26T07:19:31.113498+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T13:56:46.827916+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.15","version":"44'3","reported_seq":15,"reported_epoch":44,"state":"active+clean","last_fresh":"2023-11-26T07:19:32.170458+0000","last_change":"2023-11-26T07:19:31.147755+0000","last_active":"2023-11-26T07:19:32.170458+0000","last_peered":"2023-11-26T07:19:32.170458+0000","last_clean":"2023-11-26T07:19:32.170458+0000","last_became_active":"2023-11-26T07:19:31.147461+0000","last_became_peered":"2023-11-26T07:19:31.147461+0000","last_unstale":"2023-11-26T07:19:32.170458+0000","last_undegraded":"2023-11-26T07:19:32.170458+0000","last_fullsized":"2023-11-26T07:19:32.170458+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:19:30.106639+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:19:30.106639+0000","last_clean_scrub_stamp":"2023-11-26T07:19:30.106639+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T13:40:11.907176+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.15","version":"0'0","reported_seq":11,"reported_epoch":45,"state":"active+clean","last_fresh":"2023-11-26T07:19:33.479056+0000","last_change":"2023-11-26T07:19:32.164123+0000","last_active":"2023-11-26T07:19:33.479056+0000","last_peered":"2023-11-26T07:19:33.479056+0000","last_clean":"2023-11-26T07:19:33.479056+0000","last_became_active":"2023-11-26T07:19:32.163813+0000","last_became_peered":"2023-11-26T07:19:32.163813+0000","last_unstale":"2023-11-26T07:19:33.479056+0000","last_undegraded":"2023-11-26T07:19:33.479056+0000","last_fullsized":"2023-11-26T07:19:33.479056+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:19:31.113498+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:19:31.113498+0000","last_clean_scrub_stamp":"2023-11-26T07:19:31.113498+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T14:46:10.067648+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.14","version":"44'6","reported_seq":18,"reported_epoch":44,"state":"active+clean","last_fresh":"2023-11-26T07:19:32.176465+0000","last_change":"2023-11-26T07:19:31.131243+0000","last_active":"2023-11-26T07:19:32.176465+0000","last_peered":"2023-11-26T07:19:32.176465+0000","last_clean":"2023-11-26T07:19:32.176465+0000","last_became_active":"2023-11-26T07:19:31.130865+0000","last_became_peered":"2023-11-26T07:19:31.130865+0000","last_unstale":"2023-11-26T07:19:32.176465+0000","last_undegraded":"2023-11-26T07:19:32.176465+0000","last_fullsized":"2023-11-26T07:19:32.176465+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:19:30.106639+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:19:30.106639+0000","last_clean_scrub_stamp":"2023-11-26T07:19:30.106639+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T10:23:43.187890+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.12","version":"0'0","reported_seq":11,"reported_epoch":45,"state":"active+clean","last_fresh":"2023-11-26T07:19:35.779828+0000","last_change":"2023-11-26T07:19:32.140004+0000","last_active":"2023-11-26T07:19:35.779828+0000","last_peered":"2023-11-26T07:19:35.779828+0000","last_clean":"2023-11-26T07:19:35.779828+0000","last_became_active":"2023-11-26T07:19:32.139750+0000","last_became_peered":"2023-11-26T07:19:32.139750+0000","last_unstale":"2023-11-26T07:19:35.779828+0000","last_undegraded":"2023-11-26T07:19:35.779828+0000","last_fullsized":"2023-11-26T07:19:35.779828+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:19:31.113498+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:19:31.113498+0000","last_clean_scrub_stamp":"2023-11-26T07:19:31.113498+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T13:18:35.574099+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.13","version":"44'2","reported_seq":14,"reported_epoch":44,"state":"active+clean","last_fresh":"2023-11-26T07:19:32.175839+0000","last_change":"2023-11-26T07:19:31.148780+0000","last_active":"2023-11-26T07:19:32.175839+0000","last_peered":"2023-11-26T07:19:32.175839+0000","last_clean":"2023-11-26T07:19:32.175839+0000","last_became_active":"2023-11-26T07:19:31.148142+0000","last_became_peered":"2023-11-26T07:19:31.148142+0000","last_unstale":"2023-11-26T07:19:32.175839+0000","last_undegraded":"2023-11-26T07:19:32.175839+0000","last_fullsized":"2023-11-26T07:19:32.175839+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:19:30.106639+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:19:30.106639+0000","last_clean_scrub_stamp":"2023-11-26T07:19:30.106639+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T13:08:05.229555+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.13","version":"0'0","reported_seq":11,"reported_epoch":45,"state":"active+clean","last_fresh":"2023-11-26T07:19:35.781446+0000","last_change":"2023-11-26T07:19:32.146408+0000","last_active":"2023-11-26T07:19:35.781446+0000","last_peered":"2023-11-26T07:19:35.781446+0000","last_clean":"2023-11-26T07:19:35.781446+0000","last_became_active":"2023-11-26T07:19:32.146146+0000","last_became_peered":"2023-11-26T07:19:32.146146+0000","last_unstale":"2023-11-26T07:19:35.781446+0000","last_undegraded":"2023-11-26T07:19:35.781446+0000","last_fullsized":"2023-11-26T07:19:35.781446+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:19:31.113498+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:19:31.113498+0000","last_clean_scrub_stamp":"2023-11-26T07:19:31.113498+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T13:40:12.914035+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.12","version":"0'0","reported_seq":12,"reported_epoch":44,"state":"active+clean","last_fresh":"2023-11-26T07:19:32.131375+0000","last_change":"2023-11-26T07:19:31.144842+0000","last_active":"2023-11-26T07:19:32.131375+0000","last_peered":"2023-11-26T07:19:32.131375+0000","last_clean":"2023-11-26T07:19:32.131375+0000","last_became_active":"2023-11-26T07:19:31.144554+0000","last_became_peered":"2023-11-26T07:19:31.144554+0000","last_unstale":"2023-11-26T07:19:32.131375+0000","last_undegraded":"2023-11-26T07:19:32.131375+0000","last_fullsized":"2023-11-26T07:19:32.131375+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:19:30.106639+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:19:30.106639+0000","last_clean_scrub_stamp":"2023-11-26T07:19:30.106639+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T08:11:42.410109+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.10","version":"0'0","reported_seq":11,"reported_epoch":45,"state":"active+clean","last_fresh":"2023-11-26T07:19:35.045846+0000","last_change":"2023-11-26T07:19:32.152524+0000","last_active":"2023-11-26T07:19:35.045846+0000","last_peered":"2023-11-26T07:19:35.045846+0000","last_clean":"2023-11-26T07:19:35.045846+0000","last_became_active":"2023-11-26T07:19:32.152218+0000","last_became_peered":"2023-11-26T07:19:32.152218+0000","last_unstale":"2023-11-26T07:19:35.045846+0000","last_undegraded":"2023-11-26T07:19:35.045846+0000","last_fullsized":"2023-11-26T07:19:35.045846+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:19:31.113498+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:19:31.113498+0000","last_clean_scrub_stamp":"2023-11-26T07:19:31.113498+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T11:46:02.755553+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.11","version":"0'0","reported_seq":12,"reported_epoch":44,"state":"active+clean","last_fresh":"2023-11-26T07:19:32.132190+0000","last_change":"2023-11-26T07:19:31.148587+0000","last_active":"2023-11-26T07:19:32.132190+0000","last_peered":"2023-11-26T07:19:32.132190+0000","last_clean":"2023-11-26T07:19:32.132190+0000","last_became_active":"2023-11-26T07:19:31.148170+0000","last_became_peered":"2023-11-26T07:19:31.148170+0000","last_unstale":"2023-11-26T07:19:32.132190+0000","last_undegraded":"2023-11-26T07:19:32.132190+0000","last_fullsized":"2023-11-26T07:19:32.132190+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:19:30.106639+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:19:30.106639+0000","last_clean_scrub_stamp":"2023-11-26T07:19:30.106639+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T14:52:02.564927+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.11","version":"0'0","reported_seq":11,"reported_epoch":45,"state":"active+clean","last_fresh":"2023-11-26T07:19:33.479035+0000","last_change":"2023-11-26T07:19:32.140495+0000","last_active":"2023-11-26T07:19:33.479035+0000","last_peered":"2023-11-26T07:19:33.479035+0000","last_clean":"2023-11-26T07:19:33.479035+0000","last_became_active":"2023-11-26T07:19:32.140249+0000","last_became_peered":"2023-11-26T07:19:32.140249+0000","last_unstale":"2023-11-26T07:19:33.479035+0000","last_undegraded":"2023-11-26T07:19:33.479035+0000","last_fullsized":"2023-11-26T07:19:33.479035+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:19:31.113498+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:19:31.113498+0000","last_clean_scrub_stamp":"2023-11-26T07:19:31.113498+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T14:38:21.642327+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.10","version":"44'8","reported_seq":20,"reported_epoch":44,"state":"active+clean","last_fresh":"2023-11-26T07:19:32.173764+0000","last_change":"2023-11-26T07:19:31.139691+0000","last_active":"2023-11-26T07:19:32.173764+0000","last_peered":"2023-11-26T07:19:32.173764+0000","last_clean":"2023-11-26T07:19:32.173764+0000","last_became_active":"2023-11-26T07:19:31.139402+0000","last_became_peered":"2023-11-26T07:19:31.139402+0000","last_unstale":"2023-11-26T07:19:32.173764+0000","last_undegraded":"2023-11-26T07:19:32.173764+0000","last_fullsized":"2023-11-26T07:19:32.173764+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:19:30.106639+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:19:30.106639+0000","last_clean_scrub_stamp":"2023-11-26T07:19:30.106639+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T17:48:35.887457+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.e","version":"0'0","reported_seq":11,"reported_epoch":45,"state":"active+clean","last_fresh":"2023-11-26T07:19:33.142000+0000","last_change":"2023-11-26T07:19:32.147956+0000","last_active":"2023-11-26T07:19:33.142000+0000","last_peered":"2023-11-26T07:19:33.142000+0000","last_clean":"2023-11-26T07:19:33.142000+0000","last_became_active":"2023-11-26T07:19:32.147524+0000","last_became_peered":"2023-11-26T07:19:32.147524+0000","last_unstale":"2023-11-26T07:19:33.142000+0000","last_undegraded":"2023-11-26T07:19:33.142000+0000","last_fullsized":"2023-11-26T07:19:33.142000+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:19:31.113498+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:19:31.113498+0000","last_clean_scrub_stamp":"2023-11-26T07:19:31.113498+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T16:49:28.479524+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.f","version":"44'1","reported_seq":13,"reported_epoch":44,"state":"active+clean","last_fresh":"2023-11-26T07:19:32.173455+0000","last_change":"2023-11-26T07:19:31.140922+0000","last_active":"2023-11-26T07:19:32.173455+0000","last_peered":"2023-11-26T07:19:32.173455+0000","last_clean":"2023-11-26T07:19:32.173455+0000","last_became_active":"2023-11-26T07:19:31.139513+0000","last_became_peered":"2023-11-26T07:19:31.139513+0000","last_unstale":"2023-11-26T07:19:32.173455+0000","last_undegraded":"2023-11-26T07:19:32.173455+0000","last_fullsized":"2023-11-26T07:19:32.173455+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:19:30.106639+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:19:30.106639+0000","last_clean_scrub_stamp":"2023-11-26T07:19:30.106639+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T13:48:18.741812+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.f","version":"0'0","reported_seq":11,"reported_epoch":45,"state":"active+clean","last_fresh":"2023-11-26T07:19:35.781895+0000","last_change":"2023-11-26T07:19:32.145588+0000","last_active":"2023-11-26T07:19:35.781895+0000","last_peered":"2023-11-26T07:19:35.781895+0000","last_clean":"2023-11-26T07:19:35.781895+0000","last_became_active":"2023-11-26T07:19:32.145118+0000","last_became_peered":"2023-11-26T07:19:32.145118+0000","last_unstale":"2023-11-26T07:19:35.781895+0000","last_undegraded":"2023-11-26T07:19:35.781895+0000","last_fullsized":"2023-11-26T07:19:35.781895+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:19:31.113498+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:19:31.113498+0000","last_clean_scrub_stamp":"2023-11-26T07:19:31.113498+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T18:10:41.853254+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.e","version":"44'2","reported_seq":14,"reported_epoch":44,"state":"active+clean","last_fresh":"2023-11-26T07:19:32.165352+0000","last_change":"2023-11-26T07:19:31.141584+0000","last_active":"2023-11-26T07:19:32.165352+0000","last_peered":"2023-11-26T07:19:32.165352+0000","last_clean":"2023-11-26T07:19:32.165352+0000","last_became_active":"2023-11-26T07:19:31.141272+0000","last_became_peered":"2023-11-26T07:19:31.141272+0000","last_unstale":"2023-11-26T07:19:32.165352+0000","last_undegraded":"2023-11-26T07:19:32.165352+0000","last_fullsized":"2023-11-26T07:19:32.165352+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:19:30.106639+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:19:30.106639+0000","last_clean_scrub_stamp":"2023-11-26T07:19:30.106639+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T17:05:24.936030+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.c","version":"0'0","reported_seq":11,"reported_epoch":45,"state":"active+clean","last_fresh":"2023-11-26T07:19:35.779829+0000","last_change":"2023-11-26T07:19:32.144748+0000","last_active":"2023-11-26T07:19:35.779829+0000","last_peered":"2023-11-26T07:19:35.779829+0000","last_clean":"2023-11-26T07:19:35.779829+0000","last_became_active":"2023-11-26T07:19:32.144491+0000","last_became_peered":"2023-11-26T07:19:32.144491+0000","last_unstale":"2023-11-26T07:19:35.779829+0000","last_undegraded":"2023-11-26T07:19:35.779829+0000","last_fullsized":"2023-11-26T07:19:35.779829+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:19:31.113498+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:19:31.113498+0000","last_clean_scrub_stamp":"2023-11-26T07:19:31.113498+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T17:48:36.894316+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.d","version":"44'1","reported_seq":13,"reported_epoch":44,"state":"active+clean","last_fresh":"2023-11-26T07:19:32.172698+0000","last_change":"2023-11-26T07:19:31.145210+0000","last_active":"2023-11-26T07:19:32.172698+0000","last_peered":"2023-11-26T07:19:32.172698+0000","last_clean":"2023-11-26T07:19:32.172698+0000","last_became_active":"2023-11-26T07:19:31.144917+0000","last_became_peered":"2023-11-26T07:19:31.144917+0000","last_unstale":"2023-11-26T07:19:32.172698+0000","last_undegraded":"2023-11-26T07:19:32.172698+0000","last_fullsized":"2023-11-26T07:19:32.172698+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:19:30.106639+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:19:30.106639+0000","last_clean_scrub_stamp":"2023-11-26T07:19:30.106639+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T17:07:29.328878+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.d","version":"0'0","reported_seq":11,"reported_epoch":45,"state":"active+clean","last_fresh":"2023-11-26T07:19:35.779828+0000","last_change":"2023-11-26T07:19:32.165304+0000","last_active":"2023-11-26T07:19:35.779828+0000","last_peered":"2023-11-26T07:19:35.779828+0000","last_clean":"2023-11-26T07:19:35.779828+0000","last_became_active":"2023-11-26T07:19:32.165070+0000","last_became_peered":"2023-11-26T07:19:32.165070+0000","last_unstale":"2023-11-26T07:19:35.779828+0000","last_undegraded":"2023-11-26T07:19:35.779828+0000","last_fullsized":"2023-11-26T07:19:35.779828+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:19:31.113498+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:19:31.113498+0000","last_clean_scrub_stamp":"2023-11-26T07:19:31.113498+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T17:05:25.942889+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.c","version":"0'0","reported_seq":12,"reported_epoch":44,"state":"active+clean","last_fresh":"2023-11-26T07:19:32.134711+0000","last_change":"2023-11-26T07:19:31.147668+0000","last_active":"2023-11-26T07:19:32.134711+0000","last_peered":"2023-11-26T07:19:32.134711+0000","last_clean":"2023-11-26T07:19:32.134711+0000","last_became_active":"2023-11-26T07:19:31.146685+0000","last_became_peered":"2023-11-26T07:19:31.146685+0000","last_unstale":"2023-11-26T07:19:32.134711+0000","last_undegraded":"2023-11-26T07:19:32.134711+0000","last_fullsized":"2023-11-26T07:19:32.134711+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:19:30.106639+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:19:30.106639+0000","last_clean_scrub_stamp":"2023-11-26T07:19:30.106639+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T18:10:40.846395+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.a","version":"0'0","reported_seq":11,"reported_epoch":45,"state":"active+clean","last_fresh":"2023-11-26T07:19:35.045943+0000","last_change":"2023-11-26T07:19:32.156472+0000","last_active":"2023-11-26T07:19:35.045943+0000","last_peered":"2023-11-26T07:19:35.045943+0000","last_clean":"2023-11-26T07:19:35.045943+0000","last_became_active":"2023-11-26T07:19:32.156160+0000","last_became_peered":"2023-11-26T07:19:32.156160+0000","last_unstale":"2023-11-26T07:19:35.045943+0000","last_undegraded":"2023-11-26T07:19:35.045943+0000","last_fullsized":"2023-11-26T07:19:35.045943+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:19:31.113498+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:19:31.113498+0000","last_clean_scrub_stamp":"2023-11-26T07:19:31.113498+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T10:15:08.536351+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.b","version":"44'2","reported_seq":14,"reported_epoch":44,"state":"active+clean","last_fresh":"2023-11-26T07:19:32.173221+0000","last_change":"2023-11-26T07:19:31.149345+0000","last_active":"2023-11-26T07:19:32.173221+0000","last_peered":"2023-11-26T07:19:32.173221+0000","last_clean":"2023-11-26T07:19:32.173221+0000","last_became_active":"2023-11-26T07:19:31.149072+0000","last_became_peered":"2023-11-26T07:19:31.149072+0000","last_unstale":"2023-11-26T07:19:32.173221+0000","last_undegraded":"2023-11-26T07:19:32.173221+0000","last_fullsized":"2023-11-26T07:19:32.173221+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:19:30.106639+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:19:30.106639+0000","last_clean_scrub_stamp":"2023-11-26T07:19:30.106639+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T13:57:08.643565+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.b","version":"0'0","reported_seq":11,"reported_epoch":45,"state":"active+clean","last_fresh":"2023-11-26T07:19:35.781055+0000","last_change":"2023-11-26T07:19:32.160785+0000","last_active":"2023-11-26T07:19:35.781055+0000","last_peered":"2023-11-26T07:19:35.781055+0000","last_clean":"2023-11-26T07:19:35.781055+0000","last_became_active":"2023-11-26T07:19:32.160504+0000","last_became_peered":"2023-11-26T07:19:32.160504+0000","last_unstale":"2023-11-26T07:19:35.781055+0000","last_undegraded":"2023-11-26T07:19:35.781055+0000","last_fullsized":"2023-11-26T07:19:35.781055+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:19:31.113498+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:19:31.113498+0000","last_clean_scrub_stamp":"2023-11-26T07:19:31.113498+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T19:14:08.178758+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.a","version":"0'0","reported_seq":12,"reported_epoch":44,"state":"active+clean","last_fresh":"2023-11-26T07:19:32.138088+0000","last_change":"2023-11-26T07:19:31.146363+0000","last_active":"2023-11-26T07:19:32.138088+0000","last_peered":"2023-11-26T07:19:32.138088+0000","last_clean":"2023-11-26T07:19:32.138088+0000","last_became_active":"2023-11-26T07:19:31.146013+0000","last_became_peered":"2023-11-26T07:19:31.146013+0000","last_unstale":"2023-11-26T07:19:32.138088+0000","last_undegraded":"2023-11-26T07:19:32.138088+0000","last_fullsized":"2023-11-26T07:19:32.138088+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:19:30.106639+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:19:30.106639+0000","last_clean_scrub_stamp":"2023-11-26T07:19:30.106639+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T13:58:09.825490+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.8","version":"0'0","reported_seq":11,"reported_epoch":45,"state":"active+clean","last_fresh":"2023-11-26T07:19:33.141617+0000","last_change":"2023-11-26T07:19:32.142623+0000","last_active":"2023-11-26T07:19:33.141617+0000","last_peered":"2023-11-26T07:19:33.141617+0000","last_clean":"2023-11-26T07:19:33.141617+0000","last_became_active":"2023-11-26T07:19:32.142279+0000","last_became_peered":"2023-11-26T07:19:32.142279+0000","last_unstale":"2023-11-26T07:19:33.141617+0000","last_undegraded":"2023-11-26T07:19:33.141617+0000","last_fullsized":"2023-11-26T07:19:33.141617+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:19:31.113498+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:19:31.113498+0000","last_clean_scrub_stamp":"2023-11-26T07:19:31.113498+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T17:24:01.360879+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.9","version":"0'0","reported_seq":12,"reported_epoch":44,"state":"active+clean","last_fresh":"2023-11-26T07:19:32.131814+0000","last_change":"2023-11-26T07:19:31.148399+0000","last_active":"2023-11-26T07:19:32.131814+0000","last_peered":"2023-11-26T07:19:32.131814+0000","last_clean":"2023-11-26T07:19:32.131814+0000","last_became_active":"2023-11-26T07:19:31.148127+0000","last_became_peered":"2023-11-26T07:19:31.148127+0000","last_unstale":"2023-11-26T07:19:32.131814+0000","last_undegraded":"2023-11-26T07:19:32.131814+0000","last_fullsized":"2023-11-26T07:19:32.131814+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:19:30.106639+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:19:30.106639+0000","last_clean_scrub_stamp":"2023-11-26T07:19:30.106639+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T15:32:40.529918+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.9","version":"0'0","reported_seq":11,"reported_epoch":45,"state":"active+clean","last_fresh":"2023-11-26T07:19:35.780767+0000","last_change":"2023-11-26T07:19:32.172061+0000","last_active":"2023-11-26T07:19:35.780767+0000","last_peered":"2023-11-26T07:19:35.780767+0000","last_clean":"2023-11-26T07:19:35.780767+0000","last_became_active":"2023-11-26T07:19:32.171767+0000","last_became_peered":"2023-11-26T07:19:32.171767+0000","last_unstale":"2023-11-26T07:19:35.780767+0000","last_undegraded":"2023-11-26T07:19:35.780767+0000","last_fullsized":"2023-11-26T07:19:35.780767+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:19:31.113498+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:19:31.113498+0000","last_clean_scrub_stamp":"2023-11-26T07:19:31.113498+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T14:52:03.571786+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.8","version":"0'0","reported_seq":12,"reported_epoch":44,"state":"active+clean","last_fresh":"2023-11-26T07:19:32.131173+0000","last_change":"2023-11-26T07:19:31.144956+0000","last_active":"2023-11-26T07:19:32.131173+0000","last_peered":"2023-11-26T07:19:32.131173+0000","last_clean":"2023-11-26T07:19:32.131173+0000","last_became_active":"2023-11-26T07:19:31.144653+0000","last_became_peered":"2023-11-26T07:19:31.144653+0000","last_unstale":"2023-11-26T07:19:32.131173+0000","last_undegraded":"2023-11-26T07:19:32.131173+0000","last_fullsized":"2023-11-26T07:19:32.131173+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:19:30.106639+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:19:30.106639+0000","last_clean_scrub_stamp":"2023-11-26T07:19:30.106639+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T16:13:47.070273+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.7","version":"0'0","reported_seq":11,"reported_epoch":45,"state":"active+clean","last_fresh":"2023-11-26T07:19:35.780130+0000","last_change":"2023-11-26T07:19:32.152689+0000","last_active":"2023-11-26T07:19:35.780130+0000","last_peered":"2023-11-26T07:19:35.780130+0000","last_clean":"2023-11-26T07:19:35.780130+0000","last_became_active":"2023-11-26T07:19:32.151484+0000","last_became_peered":"2023-11-26T07:19:32.151484+0000","last_unstale":"2023-11-26T07:19:35.780130+0000","last_undegraded":"2023-11-26T07:19:35.780130+0000","last_fullsized":"2023-11-26T07:19:35.780130+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:19:31.113498+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:19:31.113498+0000","last_clean_scrub_stamp":"2023-11-26T07:19:31.113498+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T13:42:56.813910+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.6","version":"45'1","reported_seq":13,"reported_epoch":45,"state":"active+clean","last_fresh":"2023-11-26T07:19:35.046338+0000","last_change":"2023-11-26T07:19:31.143699+0000","last_active":"2023-11-26T07:19:35.046338+0000","last_peered":"2023-11-26T07:19:35.046338+0000","last_clean":"2023-11-26T07:19:35.046338+0000","last_became_active":"2023-11-26T07:19:31.143421+0000","last_became_peered":"2023-11-26T07:19:31.143421+0000","last_unstale":"2023-11-26T07:19:35.046338+0000","last_undegraded":"2023-11-26T07:19:35.046338+0000","last_fullsized":"2023-11-26T07:19:35.046338+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:19:30.106639+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:19:30.106639+0000","last_clean_scrub_stamp":"2023-11-26T07:19:30.106639+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T14:46:09.060789+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.4","version":"0'0","reported_seq":11,"reported_epoch":45,"state":"active+clean","last_fresh":"2023-11-26T07:19:35.781425+0000","last_change":"2023-11-26T07:19:32.154214+0000","last_active":"2023-11-26T07:19:35.781425+0000","last_peered":"2023-11-26T07:19:35.781425+0000","last_clean":"2023-11-26T07:19:35.781425+0000","last_became_active":"2023-11-26T07:19:32.153615+0000","last_became_peered":"2023-11-26T07:19:32.153615+0000","last_unstale":"2023-11-26T07:19:35.781425+0000","last_undegraded":"2023-11-26T07:19:35.781425+0000","last_fullsized":"2023-11-26T07:19:35.781425+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:19:31.113498+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:19:31.113498+0000","last_clean_scrub_stamp":"2023-11-26T07:19:31.113498+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T18:36:17.645226+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.5","version":"0'0","reported_seq":12,"reported_epoch":44,"state":"active+clean","last_fresh":"2023-11-26T07:19:32.132832+0000","last_change":"2023-11-26T07:19:31.148100+0000","last_active":"2023-11-26T07:19:32.132832+0000","last_peered":"2023-11-26T07:19:32.132832+0000","last_clean":"2023-11-26T07:19:32.132832+0000","last_became_active":"2023-11-26T07:19:31.147766+0000","last_became_peered":"2023-11-26T07:19:31.147766+0000","last_unstale":"2023-11-26T07:19:32.132832+0000","last_undegraded":"2023-11-26T07:19:32.132832+0000","last_fullsized":"2023-11-26T07:19:32.132832+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:19:30.106639+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:19:30.106639+0000","last_clean_scrub_stamp":"2023-11-26T07:19:30.106639+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T13:08:05.229555+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.5","version":"0'0","reported_seq":11,"reported_epoch":45,"state":"active+clean","last_fresh":"2023-11-26T07:19:33.479141+0000","last_change":"2023-11-26T07:19:32.145003+0000","last_active":"2023-11-26T07:19:33.479141+0000","last_peered":"2023-11-26T07:19:33.479141+0000","last_clean":"2023-11-26T07:19:33.479141+0000","last_became_active":"2023-11-26T07:19:32.143594+0000","last_became_peered":"2023-11-26T07:19:32.143594+0000","last_unstale":"2023-11-26T07:19:33.479141+0000","last_undegraded":"2023-11-26T07:19:33.479141+0000","last_fullsized":"2023-11-26T07:19:33.479141+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:19:31.113498+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:19:31.113498+0000","last_clean_scrub_stamp":"2023-11-26T07:19:31.113498+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T18:10:41.853254+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.4","version":"0'0","reported_seq":12,"reported_epoch":44,"state":"active+clean","last_fresh":"2023-11-26T07:19:32.146730+0000","last_change":"2023-11-26T07:19:31.144386+0000","last_active":"2023-11-26T07:19:32.146730+0000","last_peered":"2023-11-26T07:19:32.146730+0000","last_clean":"2023-11-26T07:19:32.146730+0000","last_became_active":"2023-11-26T07:19:31.144106+0000","last_became_peered":"2023-11-26T07:19:31.144106+0000","last_unstale":"2023-11-26T07:19:32.146730+0000","last_undegraded":"2023-11-26T07:19:32.146730+0000","last_fullsized":"2023-11-26T07:19:32.146730+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:19:30.106639+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:19:30.106639+0000","last_clean_scrub_stamp":"2023-11-26T07:19:30.106639+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T13:18:34.567240+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.2","version":"0'0","reported_seq":11,"reported_epoch":45,"state":"active+clean","last_fresh":"2023-11-26T07:19:35.046072+0000","last_change":"2023-11-26T07:19:32.163779+0000","last_active":"2023-11-26T07:19:35.046072+0000","last_peered":"2023-11-26T07:19:35.046072+0000","last_clean":"2023-11-26T07:19:35.046072+0000","last_became_active":"2023-11-26T07:19:32.163503+0000","last_became_peered":"2023-11-26T07:19:32.163503+0000","last_unstale":"2023-11-26T07:19:35.046072+0000","last_undegraded":"2023-11-26T07:19:35.046072+0000","last_fullsized":"2023-11-26T07:19:35.046072+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:19:31.113498+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:19:31.113498+0000","last_clean_scrub_stamp":"2023-11-26T07:19:31.113498+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T10:09:45.601591+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.3","version":"0'0","reported_seq":12,"reported_epoch":44,"state":"active+clean","last_fresh":"2023-11-26T07:19:32.132520+0000","last_change":"2023-11-26T07:19:31.146929+0000","last_active":"2023-11-26T07:19:32.132520+0000","last_peered":"2023-11-26T07:19:32.132520+0000","last_clean":"2023-11-26T07:19:32.132520+0000","last_became_active":"2023-11-26T07:19:31.146630+0000","last_became_peered":"2023-11-26T07:19:31.146630+0000","last_unstale":"2023-11-26T07:19:32.132520+0000","last_undegraded":"2023-11-26T07:19:32.132520+0000","last_fullsized":"2023-11-26T07:19:32.132520+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:19:30.106639+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:19:30.106639+0000","last_clean_scrub_stamp":"2023-11-26T07:19:30.106639+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T18:36:16.638367+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.3","version":"0'0","reported_seq":11,"reported_epoch":45,"state":"active+clean","last_fresh":"2023-11-26T07:19:35.046021+0000","last_change":"2023-11-26T07:19:32.156475+0000","last_active":"2023-11-26T07:19:35.046021+0000","last_peered":"2023-11-26T07:19:35.046021+0000","last_clean":"2023-11-26T07:19:35.046021+0000","last_became_active":"2023-11-26T07:19:32.156262+0000","last_became_peered":"2023-11-26T07:19:32.156262+0000","last_unstale":"2023-11-26T07:19:35.046021+0000","last_undegraded":"2023-11-26T07:19:35.046021+0000","last_fullsized":"2023-11-26T07:19:35.046021+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:19:31.113498+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:19:31.113498+0000","last_clean_scrub_stamp":"2023-11-26T07:19:31.113498+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T17:37:48.397190+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.2","version":"0'0","reported_seq":12,"reported_epoch":44,"state":"active+clean","last_fresh":"2023-11-26T07:19:32.133786+0000","last_change":"2023-11-26T07:19:31.147730+0000","last_active":"2023-11-26T07:19:32.133786+0000","last_peered":"2023-11-26T07:19:32.133786+0000","last_clean":"2023-11-26T07:19:32.133786+0000","last_became_active":"2023-11-26T07:19:31.147374+0000","last_became_peered":"2023-11-26T07:19:31.147374+0000","last_unstale":"2023-11-26T07:19:32.133786+0000","last_undegraded":"2023-11-26T07:19:32.133786+0000","last_fullsized":"2023-11-26T07:19:32.133786+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:19:30.106639+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:19:30.106639+0000","last_clean_scrub_stamp":"2023-11-26T07:19:30.106639+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T15:54:05.005833+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.0","version":"0'0","reported_seq":11,"reported_epoch":45,"state":"active+clean","last_fresh":"2023-11-26T07:19:35.452409+0000","last_change":"2023-11-26T07:19:32.164869+0000","last_active":"2023-11-26T07:19:35.452409+0000","last_peered":"2023-11-26T07:19:35.452409+0000","last_clean":"2023-11-26T07:19:35.452409+0000","last_became_active":"2023-11-26T07:19:32.164562+0000","last_became_peered":"2023-11-26T07:19:32.164562+0000","last_unstale":"2023-11-26T07:19:35.452409+0000","last_undegraded":"2023-11-26T07:19:35.452409+0000","last_fullsized":"2023-11-26T07:19:35.452409+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:19:31.113498+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:19:31.113498+0000","last_clean_scrub_stamp":"2023-11-26T07:19:31.113498+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T12:08:32.511197+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.1","version":"0'0","reported_seq":12,"reported_epoch":44,"state":"active+clean","last_fresh":"2023-11-26T07:19:32.143839+0000","last_change":"2023-11-26T07:19:31.145269+0000","last_active":"2023-11-26T07:19:32.143839+0000","last_peered":"2023-11-26T07:19:32.143839+0000","last_clean":"2023-11-26T07:19:32.143839+0000","last_became_active":"2023-11-26T07:19:31.144994+0000","last_became_peered":"2023-11-26T07:19:31.144994+0000","last_unstale":"2023-11-26T07:19:32.143839+0000","last_undegraded":"2023-11-26T07:19:32.143839+0000","last_fullsized":"2023-11-26T07:19:32.143839+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:19:30.106639+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:19:30.106639+0000","last_clean_scrub_stamp":"2023-11-26T07:19:30.106639+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T17:48:35.887457+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.1","version":"0'0","reported_seq":11,"reported_epoch":45,"state":"active+clean","last_fresh":"2023-11-26T07:19:33.141359+0000","last_change":"2023-11-26T07:19:32.144600+0000","last_active":"2023-11-26T07:19:33.141359+0000","last_peered":"2023-11-26T07:19:33.141359+0000","last_clean":"2023-11-26T07:19:33.141359+0000","last_became_active":"2023-11-26T07:19:32.144402+0000","last_became_peered":"2023-11-26T07:19:32.144402+0000","last_unstale":"2023-11-26T07:19:33.141359+0000","last_undegraded":"2023-11-26T07:19:33.141359+0000","last_fullsized":"2023-11-26T07:19:33.141359+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:19:31.113498+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:19:31.113498+0000","last_clean_scrub_stamp":"2023-11-26T07:19:31.113498+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T13:42:56.813910+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.0","version":"0'0","reported_seq":12,"reported_epoch":44,"state":"active+clean","last_fresh":"2023-11-26T07:19:32.134488+0000","last_change":"2023-11-26T07:19:31.146523+0000","last_active":"2023-11-26T07:19:32.134488+0000","last_peered":"2023-11-26T07:19:32.134488+0000","last_clean":"2023-11-26T07:19:32.134488+0000","last_became_active":"2023-11-26T07:19:31.146168+0000","last_became_peered":"2023-11-26T07:19:31.146168+0000","last_unstale":"2023-11-26T07:19:32.134488+0000","last_undegraded":"2023-11-26T07:19:32.134488+0000","last_fullsized":"2023-11-26T07:19:32.134488+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:19:30.106639+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:19:30.106639+0000","last_clean_scrub_stamp":"2023-11-26T07:19:30.106639+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T13:18:34.567240+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.6","version":"0'0","reported_seq":11,"reported_epoch":45,"state":"active+clean","last_fresh":"2023-11-26T07:19:35.781869+0000","last_change":"2023-11-26T07:19:32.153162+0000","last_active":"2023-11-26T07:19:35.781869+0000","last_peered":"2023-11-26T07:19:35.781869+0000","last_clean":"2023-11-26T07:19:35.781869+0000","last_became_active":"2023-11-26T07:19:32.152536+0000","last_became_peered":"2023-11-26T07:19:32.152536+0000","last_unstale":"2023-11-26T07:19:35.781869+0000","last_undegraded":"2023-11-26T07:19:35.781869+0000","last_fullsized":"2023-11-26T07:19:35.781869+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:19:31.113498+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:19:31.113498+0000","last_clean_scrub_stamp":"2023-11-26T07:19:31.113498+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T14:46:10.067648+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.7","version":"44'2","reported_seq":14,"reported_epoch":44,"state":"active+clean","last_fresh":"2023-11-26T07:19:32.171344+0000","last_change":"2023-11-26T07:19:31.146427+0000","last_active":"2023-11-26T07:19:32.171344+0000","last_peered":"2023-11-26T07:19:32.171344+0000","last_clean":"2023-11-26T07:19:32.171344+0000","last_became_active":"2023-11-26T07:19:31.146044+0000","last_became_peered":"2023-11-26T07:19:31.146044+0000","last_unstale":"2023-11-26T07:19:32.171344+0000","last_undegraded":"2023-11-26T07:19:32.171344+0000","last_fullsized":"2023-11-26T07:19:32.171344+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:19:30.106639+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:19:30.106639+0000","last_clean_scrub_stamp":"2023-11-26T07:19:30.106639+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T16:13:47.070273+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.0","version":"15'117","reported_seq":184,"reported_epoch":44,"state":"active+clean","last_fresh":"2023-11-26T07:19:32.131171+0000","last_change":"2023-11-26T07:17:43.827802+0000","last_active":"2023-11-26T07:19:32.131171+0000","last_peered":"2023-11-26T07:19:32.131171+0000","last_clean":"2023-11-26T07:19:32.131171+0000","last_became_active":"2023-11-26T07:17:43.827308+0000","last_became_peered":"2023-11-26T07:17:43.827308+0000","last_unstale":"2023-11-26T07:19:32.131171+0000","last_undegraded":"2023-11-26T07:19:32.131171+0000","last_fullsized":"2023-11-26T07:19:32.131171+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:17:42.160945+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:17:42.160945+0000","last_clean_scrub_stamp":"2023-11-26T07:17:42.160945+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T08:17:54.167366+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":7,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":6,"num_pg":32,"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":39,"ondisk_log_size":39,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574873,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":28120,"kb_used_data":1100,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343720,"statfs":{"total":96636764160,"available":96607969280,"internally_reserved":0,"allocated":1126400,"data_stored":805557,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Nov 26 07:18:42 2023","interfaces":[{"interface":"back","average":{"1min":0.71499999999999997,"5min":0.71499999999999997,"15min":0.71499999999999997},"min":{"1min":0.54300000000000004,"5min":0.54300000000000004,"15min":0.54300000000000004},"max":{"1min":0.91500000000000004,"5min":0.91500000000000004,"15min":0.91500000000000004},"last":0.63500000000000001},{"interface":"front","average":{"1min":0.70899999999999996,"5min":0.70899999999999996,"15min":0.70899999999999996},"min":{"1min":0.52900000000000003,"5min":0.52900000000000003,"15min":0.52900000000000003},"max":{"1min":1.0980000000000001,"5min":1.0980000000000001,"15min":1.0980000000000001},"last":0.86699999999999999}]},{"osd":1,"last update":"Sun Nov 26 07:18:42 2023","interfaces":[{"interface":"back","average":{"1min":0.68200000000000005,"5min":0.68200000000000005,"15min":0.68200000000000005},"min":{"1min":0.47999999999999998,"5min":0.47999999999999998,"15min":0.47999999999999998},"max":{"1min":1.0669999999999999,"5min":1.0669999999999999,"15min":1.0669999999999999},"last":0.68700000000000006},{"interface":"front","average":{"1min":0.83899999999999997,"5min":0.83899999999999997,"15min":0.83899999999999997},"min":{"1min":0.47599999999999998,"5min":0.47599999999999998,"15min":0.47599999999999998},"max":{"1min":4.5309999999999997,"5min":4.5309999999999997,"15min":4.5309999999999997},"last":0.48899999999999999}]},{"osd":2,"last update":"Sun Nov 26 07:18:42 2023","interfaces":[{"interface":"back","average":{"1min":0.67600000000000005,"5min":0.67600000000000005,"15min":0.67600000000000005},"min":{"1min":0.52000000000000002,"5min":0.52000000000000002,"15min":0.52000000000000002},"max":{"1min":0.873,"5min":0.873,"15min":0.873},"last":0.56200000000000006},{"interface":"front","average":{"1min":0.71299999999999997,"5min":0.71299999999999997,"15min":0.71299999999999997},"min":{"1min":0.54700000000000004,"5min":0.54700000000000004,"15min":0.54700000000000004},"max":{"1min":1.1339999999999999,"5min":1.1339999999999999,"15min":1.1339999999999999},"last":0.73999999999999999}]},{"osd":3,"last update":"Sun Nov 26 07:18:42 2023","interfaces":[{"interface":"back","average":{"1min":0.69699999999999995,"5min":0.69699999999999995,"15min":0.69699999999999995},"min":{"1min":0.51800000000000002,"5min":0.51800000000000002,"15min":0.51800000000000002},"max":{"1min":1.026,"5min":1.026,"15min":1.026},"last":0.626},{"interface":"front","average":{"1min":0.73299999999999998,"5min":0.73299999999999998,"15min":0.73299999999999998},"min":{"1min":0.50600000000000001,"5min":0.50600000000000001,"15min":0.50600000000000001},"max":{"1min":1.4490000000000001,"5min":1.4490000000000001,"15min":1.4490000000000001},"last":0.71899999999999997}]},{"osd":4,"last update":"Sun Nov 26 07:18:42 2023","interfaces":[{"interface":"back","average":{"1min":0.59999999999999998,"5min":0.59999999999999998,"15min":0.59999999999999998},"min":{"1min":0.39400000000000002,"5min":0.39400000000000002,"15min":0.39400000000000002},"max":{"1min":0.94799999999999995,"5min":0.94799999999999995,"15min":0.94799999999999995},"last":0.77000000000000002},{"interface":"front","average":{"1min":0.67300000000000004,"5min":0.67300000000000004,"15min":0.67300000000000004},"min":{"1min":0.436,"5min":0.436,"15min":0.436},"max":{"1min":1.5509999999999999,"5min":1.5509999999999999,"15min":1.5509999999999999},"last":0.67100000000000004}]},{"osd":5,"last update":"Sun Nov 26 07:18:42 2023","interfaces":[{"interface":"back","average":{"1min":0.66600000000000004,"5min":0.66600000000000004,"15min":0.66600000000000004},"min":{"1min":0.443,"5min":0.443,"15min":0.443},"max":{"1min":1.415,"5min":1.415,"15min":1.415},"last":0.84999999999999998},{"interface":"front","average":{"1min":0.71399999999999997,"5min":0.71399999999999997,"15min":0.71399999999999997},"min":{"1min":0.53100000000000003,"5min":0.53100000000000003,"15min":0.53100000000000003},"max":{"1min":1.3859999999999999,"5min":1.3859999999999999,"15min":1.3859999999999999},"last":0.98599999999999999}]},{"osd":6,"last update":"Sun Nov 26 07:18:42 2023","interfaces":[{"interface":"back","average":{"1min":0.77100000000000002,"5min":0.77100000000000002,"15min":0.77100000000000002},"min":{"1min":0.64100000000000001,"5min":0.64100000000000001,"15min":0.64100000000000001},"max":{"1min":1.3069999999999999,"5min":1.3069999999999999,"15min":1.3069999999999999},"last":1.0349999999999999},{"interface":"front","average":{"1min":0.77100000000000002,"5min":0.77100000000000002,"15min":0.77100000000000002},"min":{"1min":0.57399999999999995,"5min":0.57399999999999995,"15min":0.57399999999999995},"max":{"1min":1.6759999999999999,"5min":1.6759999999999999,"15min":1.6759999999999999},"last":0.79600000000000004}]}]},{"osd":6,"up_from":13,"seq":55834574875,"num_pgs":29,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27544,"kb_used_data":524,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344296,"statfs":{"total":96636764160,"available":96608559104,"internally_reserved":0,"allocated":536576,"data_stored":215771,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Nov 26 07:18:45 2023","interfaces":[{"interface":"back","average":{"1min":0.68400000000000005,"5min":0.68400000000000005,"15min":0.68400000000000005},"min":{"1min":0.36699999999999999,"5min":0.36699999999999999,"15min":0.36699999999999999},"max":{"1min":1.7549999999999999,"5min":1.7549999999999999,"15min":1.7549999999999999},"last":0.55400000000000005},{"interface":"front","average":{"1min":0.95799999999999996,"5min":0.95799999999999996,"15min":0.95799999999999996},"min":{"1min":0.38300000000000001,"5min":0.38300000000000001,"15min":0.38300000000000001},"max":{"1min":5.7489999999999997,"5min":5.7489999999999997,"15min":5.7489999999999997},"last":0.57899999999999996}]},{"osd":1,"last update":"Sun Nov 26 07:18:45 2023","interfaces":[{"interface":"back","average":{"1min":0.91400000000000003,"5min":0.91400000000000003,"15min":0.91400000000000003},"min":{"1min":0.42099999999999999,"5min":0.42099999999999999,"15min":0.42099999999999999},"max":{"1min":5.4240000000000004,"5min":5.4240000000000004,"15min":5.4240000000000004},"last":0.745},{"interface":"front","average":{"1min":0.89700000000000002,"5min":0.89700000000000002,"15min":0.89700000000000002},"min":{"1min":0.499,"5min":0.499,"15min":0.499},"max":{"1min":5.6020000000000003,"5min":5.6020000000000003,"15min":5.6020000000000003},"last":0.64100000000000001}]},{"osd":2,"last update":"Sun Nov 26 07:18:45 2023","interfaces":[{"interface":"back","average":{"1min":0.69699999999999995,"5min":0.69699999999999995,"15min":0.69699999999999995},"min":{"1min":0.47599999999999998,"5min":0.47599999999999998,"15min":0.47599999999999998},"max":{"1min":1.8520000000000001,"5min":1.8520000000000001,"15min":1.8520000000000001},"last":0.61399999999999999},{"interface":"front","average":{"1min":0.73399999999999999,"5min":0.73399999999999999,"15min":0.73399999999999999},"min":{"1min":0.46700000000000003,"5min":0.46700000000000003,"15min":0.46700000000000003},"max":{"1min":1.9159999999999999,"5min":1.9159999999999999,"15min":1.9159999999999999},"last":0.79400000000000004}]},{"osd":3,"last update":"Sun Nov 26 07:18:45 2023","interfaces":[{"interface":"back","average":{"1min":0.90200000000000002,"5min":0.90200000000000002,"15min":0.90200000000000002},"min":{"1min":0.502,"5min":0.502,"15min":0.502},"max":{"1min":5.4390000000000001,"5min":5.4390000000000001,"15min":5.4390000000000001},"last":0.69299999999999995},{"interface":"front","average":{"1min":0.94499999999999995,"5min":0.94499999999999995,"15min":0.94499999999999995},"min":{"1min":0.54400000000000004,"5min":0.54400000000000004,"15min":0.54400000000000004},"max":{"1min":5.5490000000000004,"5min":5.5490000000000004,"15min":5.5490000000000004},"last":0.77100000000000002}]},{"osd":4,"last update":"Sun Nov 26 07:18:45 2023","interfaces":[{"interface":"back","average":{"1min":0.64200000000000002,"5min":0.64200000000000002,"15min":0.64200000000000002},"min":{"1min":0.46899999999999997,"5min":0.46899999999999997,"15min":0.46899999999999997},"max":{"1min":0.85099999999999998,"5min":0.85099999999999998,"15min":0.85099999999999998},"last":0.82599999999999996},{"interface":"front","average":{"1min":0.67500000000000004,"5min":0.67500000000000004,"15min":0.67500000000000004},"min":{"1min":0.40699999999999997,"5min":0.40699999999999997,"15min":0.40699999999999997},"max":{"1min":0.93300000000000005,"5min":0.93300000000000005,"15min":0.93300000000000005},"last":0.85899999999999999}]},{"osd":5,"last update":"Sun Nov 26 07:18:45 2023","interfaces":[{"interface":"back","average":{"1min":0.65900000000000003,"5min":0.65900000000000003,"15min":0.65900000000000003},"min":{"1min":0.42499999999999999,"5min":0.42499999999999999,"15min":0.42499999999999999},"max":{"1min":0.79300000000000004,"5min":0.79300000000000004,"15min":0.79300000000000004},"last":0.71299999999999997},{"interface":"front","average":{"1min":0.71299999999999997,"5min":0.71299999999999997,"15min":0.71299999999999997},"min":{"1min":0.436,"5min":0.436,"15min":0.436},"max":{"1min":0.93000000000000005,"5min":0.93000000000000005,"15min":0.93000000000000005},"last":0.65800000000000003}]},{"osd":7,"last update":"Sun Nov 26 07:18:45 2023","interfaces":[{"interface":"back","average":{"1min":0.74199999999999999,"5min":0.74199999999999999,"15min":0.74199999999999999},"min":{"1min":0.46600000000000003,"5min":0.46600000000000003,"15min":0.46600000000000003},"max":{"1min":0.998,"5min":0.998,"15min":0.998},"last":0.81100000000000005},{"interface":"front","average":{"1min":0.71599999999999997,"5min":0.71599999999999997,"15min":0.71599999999999997},"min":{"1min":0.498,"5min":0.498,"15min":0.498},"max":{"1min":0.86499999999999999,"5min":0.86499999999999999,"15min":0.86499999999999999},"last":0.83799999999999997}]}]},{"osd":5,"up_from":13,"seq":55834574873,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27540,"kb_used_data":520,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344300,"statfs":{"total":96636764160,"available":96608563200,"internally_reserved":0,"allocated":532480,"data_stored":215169,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Nov 26 07:18:45 2023","interfaces":[{"interface":"back","average":{"1min":0.66600000000000004,"5min":0.66600000000000004,"15min":0.66600000000000004},"min":{"1min":0.38100000000000001,"5min":0.38100000000000001,"15min":0.38100000000000001},"max":{"1min":1.0289999999999999,"5min":1.0289999999999999,"15min":1.0289999999999999},"last":0.42099999999999999},{"interface":"front","average":{"1min":0.66000000000000003,"5min":0.66000000000000003,"15min":0.66000000000000003},"min":{"1min":0.438,"5min":0.438,"15min":0.438},"max":{"1min":1.05,"5min":1.05,"15min":1.05},"last":0.46700000000000003}]},{"osd":1,"last update":"Sun Nov 26 07:18:45 2023","interfaces":[{"interface":"back","average":{"1min":0.66800000000000004,"5min":0.66800000000000004,"15min":0.66800000000000004},"min":{"1min":0.41599999999999998,"5min":0.41599999999999998,"15min":0.41599999999999998},"max":{"1min":0.86699999999999999,"5min":0.86699999999999999,"15min":0.86699999999999999},"last":0.44700000000000001},{"interface":"front","average":{"1min":0.68600000000000005,"5min":0.68600000000000005,"15min":0.68600000000000005},"min":{"1min":0.49199999999999999,"5min":0.49199999999999999,"15min":0.49199999999999999},"max":{"1min":0.98899999999999999,"5min":0.98899999999999999,"15min":0.98899999999999999},"last":0.504}]},{"osd":2,"last update":"Sun Nov 26 07:18:45 2023","interfaces":[{"interface":"back","average":{"1min":0.67500000000000004,"5min":0.67500000000000004,"15min":0.67500000000000004},"min":{"1min":0.45900000000000002,"5min":0.45900000000000002,"15min":0.45900000000000002},"max":{"1min":0.89800000000000002,"5min":0.89800000000000002,"15min":0.89800000000000002},"last":0.61499999999999999},{"interface":"front","average":{"1min":0.67000000000000004,"5min":0.67000000000000004,"15min":0.67000000000000004},"min":{"1min":0.51200000000000001,"5min":0.51200000000000001,"15min":0.51200000000000001},"max":{"1min":0.89700000000000002,"5min":0.89700000000000002,"15min":0.89700000000000002},"last":0.53800000000000003}]},{"osd":3,"last update":"Sun Nov 26 07:18:45 2023","interfaces":[{"interface":"back","average":{"1min":0.71999999999999997,"5min":0.71999999999999997,"15min":0.71999999999999997},"min":{"1min":0.43099999999999999,"5min":0.43099999999999999,"15min":0.43099999999999999},"max":{"1min":1.0029999999999999,"5min":1.0029999999999999,"15min":1.0029999999999999},"last":0.70299999999999996},{"interface":"front","average":{"1min":0.67500000000000004,"5min":0.67500000000000004,"15min":0.67500000000000004},"min":{"1min":0.44600000000000001,"5min":0.44600000000000001,"15min":0.44600000000000001},"max":{"1min":1.0209999999999999,"5min":1.0209999999999999,"15min":1.0209999999999999},"last":0.56299999999999994}]},{"osd":4,"last update":"Sun Nov 26 07:18:45 2023","interfaces":[{"interface":"back","average":{"1min":0.63500000000000001,"5min":0.63500000000000001,"15min":0.63500000000000001},"min":{"1min":0.39200000000000002,"5min":0.39200000000000002,"15min":0.39200000000000002},"max":{"1min":0.95099999999999996,"5min":0.95099999999999996,"15min":0.95099999999999996},"last":0.63200000000000001},{"interface":"front","average":{"1min":0.70899999999999996,"5min":0.70899999999999996,"15min":0.70899999999999996},"min":{"1min":0.41299999999999998,"5min":0.41299999999999998,"15min":0.41299999999999998},"max":{"1min":1.254,"5min":1.254,"15min":1.254},"last":0.66400000000000003}]},{"osd":6,"last update":"Sun Nov 26 07:18:45 2023","interfaces":[{"interface":"back","average":{"1min":0.67200000000000004,"5min":0.67200000000000004,"15min":0.67200000000000004},"min":{"1min":0.45600000000000002,"5min":0.45600000000000002,"15min":0.45600000000000002},"max":{"1min":0.94499999999999995,"5min":0.94499999999999995,"15min":0.94499999999999995},"last":0.97199999999999998},{"interface":"front","average":{"1min":0.72699999999999998,"5min":0.72699999999999998,"15min":0.72699999999999998},"min":{"1min":0.44,"5min":0.44,"15min":0.44},"max":{"1min":1.0649999999999999,"5min":1.0649999999999999,"15min":1.0649999999999999},"last":0.79800000000000004}]},{"osd":7,"last update":"Sun Nov 26 07:18:45 2023","interfaces":[{"interface":"back","average":{"1min":0.80400000000000005,"5min":0.80400000000000005,"15min":0.80400000000000005},"min":{"1min":0.48199999999999998,"5min":0.48199999999999998,"15min":0.48199999999999998},"max":{"1min":1.4079999999999999,"5min":1.4079999999999999,"15min":1.4079999999999999},"last":0.878},{"interface":"front","average":{"1min":0.77500000000000002,"5min":0.77500000000000002,"15min":0.77500000000000002},"min":{"1min":0.51600000000000001,"5min":0.51600000000000001,"15min":0.51600000000000001},"max":{"1min":1.0169999999999999,"5min":1.0169999999999999,"15min":1.0169999999999999},"last":0.73099999999999998}]}]},{"osd":4,"up_from":13,"seq":55834574875,"num_pgs":19,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27548,"kb_used_data":528,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344292,"statfs":{"total":96636764160,"available":96608555008,"internally_reserved":0,"allocated":540672,"data_stored":216711,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Nov 26 07:18:46 2023","interfaces":[{"interface":"back","average":{"1min":0.79400000000000004,"5min":0.79400000000000004,"15min":0.79400000000000004},"min":{"1min":0.45600000000000002,"5min":0.45600000000000002,"15min":0.45600000000000002},"max":{"1min":4.1689999999999996,"5min":4.1689999999999996,"15min":4.1689999999999996},"last":0.54700000000000004},{"interface":"front","average":{"1min":1.002,"5min":1.002,"15min":1.002},"min":{"1min":0.47499999999999998,"5min":0.47499999999999998,"15min":0.47499999999999998},"max":{"1min":7.7889999999999997,"5min":7.7889999999999997,"15min":7.7889999999999997},"last":0.46200000000000002}]},{"osd":1,"last update":"Sun Nov 26 07:18:46 2023","interfaces":[{"interface":"back","average":{"1min":0.64700000000000002,"5min":0.64700000000000002,"15min":0.64700000000000002},"min":{"1min":0.52300000000000002,"5min":0.52300000000000002,"15min":0.52300000000000002},"max":{"1min":0.78400000000000003,"5min":0.78400000000000003,"15min":0.78400000000000003},"last":0.52100000000000002},{"interface":"front","average":{"1min":0.81999999999999995,"5min":0.81999999999999995,"15min":0.81999999999999995},"min":{"1min":0.44500000000000001,"5min":0.44500000000000001,"15min":0.44500000000000001},"max":{"1min":4.3140000000000001,"5min":4.3140000000000001,"15min":4.3140000000000001},"last":0.435}]},{"osd":2,"last update":"Sun Nov 26 07:18:46 2023","interfaces":[{"interface":"back","average":{"1min":0.80700000000000005,"5min":0.80700000000000005,"15min":0.80700000000000005},"min":{"1min":0.57799999999999996,"5min":0.57799999999999996,"15min":0.57799999999999996},"max":{"1min":2.7850000000000001,"5min":2.7850000000000001,"15min":2.7850000000000001},"last":0.73299999999999998},{"interface":"front","average":{"1min":1.0660000000000001,"5min":1.0660000000000001,"15min":1.0660000000000001},"min":{"1min":0.53400000000000003,"5min":0.53400000000000003,"15min":0.53400000000000003},"max":{"1min":7.6470000000000002,"5min":7.6470000000000002,"15min":7.6470000000000002},"last":0.67400000000000004}]},{"osd":3,"last update":"Sun Nov 26 07:18:46 2023","interfaces":[{"interface":"back","average":{"1min":0.72999999999999998,"5min":0.72999999999999998,"15min":0.72999999999999998},"min":{"1min":0.48599999999999999,"5min":0.48599999999999999,"15min":0.48599999999999999},"max":{"1min":1.591,"5min":1.591,"15min":1.591},"last":0.70499999999999996},{"interface":"front","average":{"1min":0.879,"5min":0.879,"15min":0.879},"min":{"1min":0.56899999999999995,"5min":0.56899999999999995,"15min":0.56899999999999995},"max":{"1min":3.452,"5min":3.452,"15min":3.452},"last":0.64400000000000002}]},{"osd":5,"last update":"Sun Nov 26 07:18:46 2023","interfaces":[{"interface":"back","average":{"1min":0.61099999999999999,"5min":0.61099999999999999,"15min":0.61099999999999999},"min":{"1min":0.43099999999999999,"5min":0.43099999999999999,"15min":0.43099999999999999},"max":{"1min":0.96199999999999997,"5min":0.96199999999999997,"15min":0.96199999999999997},"last":0.56899999999999995},{"interface":"front","average":{"1min":0.64100000000000001,"5min":0.64100000000000001,"15min":0.64100000000000001},"min":{"1min":0.41499999999999998,"5min":0.41499999999999998,"15min":0.41499999999999998},"max":{"1min":1.538,"5min":1.538,"15min":1.538},"last":0.749}]},{"osd":6,"last update":"Sun Nov 26 07:18:46 2023","interfaces":[{"interface":"back","average":{"1min":0.79400000000000004,"5min":0.79400000000000004,"15min":0.79400000000000004},"min":{"1min":0.39400000000000002,"5min":0.39400000000000002,"15min":0.39400000000000002},"max":{"1min":2.8330000000000002,"5min":2.8330000000000002,"15min":2.8330000000000002},"last":0.77300000000000002},{"interface":"front","average":{"1min":1.0089999999999999,"5min":1.0089999999999999,"15min":1.0089999999999999},"min":{"1min":0.42899999999999999,"5min":0.42899999999999999,"15min":0.42899999999999999},"max":{"1min":7.6050000000000004,"5min":7.6050000000000004,"15min":7.6050000000000004},"last":0.623}]},{"osd":7,"last update":"Sun Nov 26 07:18:46 2023","interfaces":[{"interface":"back","average":{"1min":0.86099999999999999,"5min":0.86099999999999999,"15min":0.86099999999999999},"min":{"1min":0.44700000000000001,"5min":0.44700000000000001,"15min":0.44700000000000001},"max":{"1min":4.3689999999999998,"5min":4.3689999999999998,"15min":4.3689999999999998},"last":0.59799999999999998},{"interface":"front","average":{"1min":0.89200000000000002,"5min":0.89200000000000002,"15min":0.89200000000000002},"min":{"1min":0.505,"5min":0.505,"15min":0.505},"max":{"1min":3.403,"5min":3.403,"15min":3.403},"last":0.79900000000000004}]}]},{"osd":3,"up_from":13,"seq":55834574876,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27548,"kb_used_data":528,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344292,"statfs":{"total":96636764160,"available":96608555008,"internally_reserved":0,"allocated":540672,"data_stored":215737,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1592,"internal_metadata":27654600},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Nov 26 07:18:43 2023","interfaces":[{"interface":"back","average":{"1min":0.379,"5min":0.379,"15min":0.379},"min":{"1min":0.26100000000000001,"5min":0.26100000000000001,"15min":0.26100000000000001},"max":{"1min":0.58599999999999997,"5min":0.58599999999999997,"15min":0.58599999999999997},"last":0.44800000000000001},{"interface":"front","average":{"1min":0.433,"5min":0.433,"15min":0.433},"min":{"1min":0.30199999999999999,"5min":0.30199999999999999,"15min":0.30199999999999999},"max":{"1min":0.63,"5min":0.63,"15min":0.63},"last":0.51700000000000002}]},{"osd":1,"last update":"Sun Nov 26 07:18:43 2023","interfaces":[{"interface":"back","average":{"1min":0.42099999999999999,"5min":0.42099999999999999,"15min":0.42099999999999999},"min":{"1min":0.312,"5min":0.312,"15min":0.312},"max":{"1min":0.752,"5min":0.752,"15min":0.752},"last":0.54500000000000004},{"interface":"front","average":{"1min":0.48399999999999999,"5min":0.48399999999999999,"15min":0.48399999999999999},"min":{"1min":0.38,"5min":0.38,"15min":0.38},"max":{"1min":0.628,"5min":0.628,"15min":0.628},"last":0.56599999999999995}]},{"osd":2,"last update":"Sun Nov 26 07:18:43 2023","interfaces":[{"interface":"back","average":{"1min":0.53300000000000003,"5min":0.53300000000000003,"15min":0.53300000000000003},"min":{"1min":0.40300000000000002,"5min":0.40300000000000002,"15min":0.40300000000000002},"max":{"1min":0.82199999999999995,"5min":0.82199999999999995,"15min":0.82199999999999995},"last":0.70499999999999996},{"interface":"front","average":{"1min":0.505,"5min":0.505,"15min":0.505},"min":{"1min":0.35999999999999999,"5min":0.35999999999999999,"15min":0.35999999999999999},"max":{"1min":0.72199999999999998,"5min":0.72199999999999998,"15min":0.72199999999999998},"last":0.67800000000000005}]},{"osd":4,"last update":"Sun Nov 26 07:18:43 2023","interfaces":[{"interface":"back","average":{"1min":0.70599999999999996,"5min":0.70599999999999996,"15min":0.70599999999999996},"min":{"1min":0.48199999999999998,"5min":0.48199999999999998,"15min":0.48199999999999998},"max":{"1min":0.96199999999999997,"5min":0.96199999999999997,"15min":0.96199999999999997},"last":1.004},{"interface":"front","average":{"1min":0.72799999999999998,"5min":0.72799999999999998,"15min":0.72799999999999998},"min":{"1min":0.441,"5min":0.441,"15min":0.441},"max":{"1min":1.171,"5min":1.171,"15min":1.171},"last":0.84899999999999998}]},{"osd":5,"last update":"Sun Nov 26 07:18:43 2023","interfaces":[{"interface":"back","average":{"1min":0.73899999999999999,"5min":0.73899999999999999,"15min":0.73899999999999999},"min":{"1min":0.53700000000000003,"5min":0.53700000000000003,"15min":0.53700000000000003},"max":{"1min":0.94099999999999995,"5min":0.94099999999999995,"15min":0.94099999999999995},"last":0.88600000000000001},{"interface":"front","average":{"1min":0.73799999999999999,"5min":0.73799999999999999,"15min":0.73799999999999999},"min":{"1min":0.504,"5min":0.504,"15min":0.504},"max":{"1min":1.0820000000000001,"5min":1.0820000000000001,"15min":1.0820000000000001},"last":0.86799999999999999}]},{"osd":6,"last update":"Sun Nov 26 07:18:43 2023","interfaces":[{"interface":"back","average":{"1min":0.81399999999999995,"5min":0.81399999999999995,"15min":0.81399999999999995},"min":{"1min":0.55200000000000005,"5min":0.55200000000000005,"15min":0.55200000000000005},"max":{"1min":1.2729999999999999,"5min":1.2729999999999999,"15min":1.2729999999999999},"last":0.80700000000000005},{"interface":"front","average":{"1min":0.79300000000000004,"5min":0.79300000000000004,"15min":0.79300000000000004},"min":{"1min":0.47299999999999998,"5min":0.47299999999999998,"15min":0.47299999999999998},"max":{"1min":1.204,"5min":1.204,"15min":1.204},"last":0.93600000000000005}]},{"osd":7,"last update":"Sun Nov 26 07:18:43 2023","interfaces":[{"interface":"back","average":{"1min":0.80100000000000005,"5min":0.80100000000000005,"15min":0.80100000000000005},"min":{"1min":0.63300000000000001,"5min":0.63300000000000001,"15min":0.63300000000000001},"max":{"1min":1.3600000000000001,"5min":1.3600000000000001,"15min":1.3600000000000001},"last":1.125},{"interface":"front","average":{"1min":0.82599999999999996,"5min":0.82599999999999996,"15min":0.82599999999999996},"min":{"1min":0.50600000000000001,"5min":0.50600000000000001,"15min":0.50600000000000001},"max":{"1min":1.617,"5min":1.617,"15min":1.617},"last":1.0940000000000001}]}]},{"osd":2,"up_from":13,"seq":55834574876,"num_pgs":11,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27560,"kb_used_data":532,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344280,"statfs":{"total":96636764160,"available":96608542720,"internally_reserved":0,"allocated":544768,"data_stored":216733,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Nov 26 07:18:43 2023","interfaces":[{"interface":"back","average":{"1min":0.71399999999999997,"5min":0.71399999999999997,"15min":0.71399999999999997},"min":{"1min":0.28699999999999998,"5min":0.28699999999999998,"15min":0.28699999999999998},"max":{"1min":4.7030000000000003,"5min":4.7030000000000003,"15min":4.7030000000000003},"last":0.41099999999999998},{"interface":"front","average":{"1min":0.56200000000000006,"5min":0.56200000000000006,"15min":0.56200000000000006},"min":{"1min":0.32300000000000001,"5min":0.32300000000000001,"15min":0.32300000000000001},"max":{"1min":2.4340000000000002,"5min":2.4340000000000002,"15min":2.4340000000000002},"last":0.438}]},{"osd":1,"last update":"Sun Nov 26 07:18:43 2023","interfaces":[{"interface":"back","average":{"1min":0.86399999999999999,"5min":0.86399999999999999,"15min":0.86399999999999999},"min":{"1min":0.34599999999999997,"5min":0.34599999999999997,"15min":0.34599999999999997},"max":{"1min":4.2690000000000001,"5min":4.2690000000000001,"15min":4.2690000000000001},"last":0.46300000000000002},{"interface":"front","average":{"1min":0.74199999999999999,"5min":0.74199999999999999,"15min":0.74199999999999999},"min":{"1min":0.31900000000000001,"5min":0.31900000000000001,"15min":0.31900000000000001},"max":{"1min":3.9940000000000002,"5min":3.9940000000000002,"15min":3.9940000000000002},"last":0.38800000000000001}]},{"osd":3,"last update":"Sun Nov 26 07:18:43 2023","interfaces":[{"interface":"back","average":{"1min":0.64300000000000002,"5min":0.64300000000000002,"15min":0.64300000000000002},"min":{"1min":0.34399999999999997,"5min":0.34399999999999997,"15min":0.34399999999999997},"max":{"1min":4.0300000000000002,"5min":4.0300000000000002,"15min":4.0300000000000002},"last":0.36899999999999999},{"interface":"front","average":{"1min":0.91400000000000003,"5min":0.91400000000000003,"15min":0.91400000000000003},"min":{"1min":0.41699999999999998,"5min":0.41699999999999998,"15min":0.41699999999999998},"max":{"1min":4.5810000000000004,"5min":4.5810000000000004,"15min":4.5810000000000004},"last":0.50900000000000001}]},{"osd":4,"last update":"Sun Nov 26 07:18:43 2023","interfaces":[{"interface":"back","average":{"1min":0.98499999999999999,"5min":0.98499999999999999,"15min":0.98499999999999999},"min":{"1min":0.55100000000000005,"5min":0.55100000000000005,"15min":0.55100000000000005},"max":{"1min":4.4269999999999996,"5min":4.4269999999999996,"15min":4.4269999999999996},"last":0.80900000000000005},{"interface":"front","average":{"1min":1.0149999999999999,"5min":1.0149999999999999,"15min":1.0149999999999999},"min":{"1min":0.60299999999999998,"5min":0.60299999999999998,"15min":0.60299999999999998},"max":{"1min":4.085,"5min":4.085,"15min":4.085},"last":0.68100000000000005}]},{"osd":5,"last update":"Sun Nov 26 07:18:43 2023","interfaces":[{"interface":"back","average":{"1min":0.874,"5min":0.874,"15min":0.874},"min":{"1min":0.52900000000000003,"5min":0.52900000000000003,"15min":0.52900000000000003},"max":{"1min":3.9590000000000001,"5min":3.9590000000000001,"15min":3.9590000000000001},"last":0.65000000000000002},{"interface":"front","average":{"1min":0.98799999999999999,"5min":0.98799999999999999,"15min":0.98799999999999999},"min":{"1min":0.58799999999999997,"5min":0.58799999999999997,"15min":0.58799999999999997},"max":{"1min":4.3849999999999998,"5min":4.3849999999999998,"15min":4.3849999999999998},"last":0.752}]},{"osd":6,"last update":"Sun Nov 26 07:18:43 2023","interfaces":[{"interface":"back","average":{"1min":0.996,"5min":0.996,"15min":0.996},"min":{"1min":0.56599999999999995,"5min":0.56599999999999995,"15min":0.56599999999999995},"max":{"1min":4.0449999999999999,"5min":4.0449999999999999,"15min":4.0449999999999999},"last":0.77700000000000002},{"interface":"front","average":{"1min":0.91800000000000004,"5min":0.91800000000000004,"15min":0.91800000000000004},"min":{"1min":0.57099999999999995,"5min":0.57099999999999995,"15min":0.57099999999999995},"max":{"1min":3.992,"5min":3.992,"15min":3.992},"last":0.60699999999999998}]},{"osd":7,"last update":"Sun Nov 26 07:18:43 2023","interfaces":[{"interface":"back","average":{"1min":1.0349999999999999,"5min":1.0349999999999999,"15min":1.0349999999999999},"min":{"1min":0.63100000000000001,"5min":0.63100000000000001,"15min":0.63100000000000001},"max":{"1min":4.1219999999999999,"5min":4.1219999999999999,"15min":4.1219999999999999},"last":0.63},{"interface":"front","average":{"1min":0.93799999999999994,"5min":0.93799999999999994,"15min":0.93799999999999994},"min":{"1min":0.57699999999999996,"5min":0.57699999999999996,"15min":0.57699999999999996},"max":{"1min":3.9159999999999999,"5min":3.9159999999999999,"15min":3.9159999999999999},"last":0.73599999999999999}]}]},{"osd":1,"up_from":13,"seq":55834574875,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27540,"kb_used_data":520,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344300,"statfs":{"total":96636764160,"available":96608563200,"internally_reserved":0,"allocated":532480,"data_stored":215169,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":12,"apply_latency_ms":12,"commit_latency_ns":12000000,"apply_latency_ns":12000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Nov 26 07:18:41 2023","interfaces":[{"interface":"back","average":{"1min":0.41699999999999998,"5min":0.41699999999999998,"15min":0.41699999999999998},"min":{"1min":0.26300000000000001,"5min":0.26300000000000001,"15min":0.26300000000000001},"max":{"1min":0.61199999999999999,"5min":0.61199999999999999,"15min":0.61199999999999999},"last":0.40500000000000003},{"interface":"front","average":{"1min":0.48599999999999999,"5min":0.48599999999999999,"15min":0.48599999999999999},"min":{"1min":0.30499999999999999,"5min":0.30499999999999999,"15min":0.30499999999999999},"max":{"1min":0.78700000000000003,"5min":0.78700000000000003,"15min":0.78700000000000003},"last":0.26700000000000002}]},{"osd":2,"last update":"Sun Nov 26 07:18:41 2023","interfaces":[{"interface":"back","average":{"1min":0.45500000000000002,"5min":0.45500000000000002,"15min":0.45500000000000002},"min":{"1min":0.33400000000000002,"5min":0.33400000000000002,"15min":0.33400000000000002},"max":{"1min":0.67600000000000005,"5min":0.67600000000000005,"15min":0.67600000000000005},"last":0.42299999999999999},{"interface":"front","average":{"1min":0.50800000000000001,"5min":0.50800000000000001,"15min":0.50800000000000001},"min":{"1min":0.32300000000000001,"5min":0.32300000000000001,"15min":0.32300000000000001},"max":{"1min":0.66500000000000004,"5min":0.66500000000000004,"15min":0.66500000000000004},"last":0.36299999999999999}]},{"osd":3,"last update":"Sun Nov 26 07:18:41 2023","interfaces":[{"interface":"back","average":{"1min":0.53000000000000003,"5min":0.53000000000000003,"15min":0.53000000000000003},"min":{"1min":0.38400000000000001,"5min":0.38400000000000001,"15min":0.38400000000000001},"max":{"1min":0.73699999999999999,"5min":0.73699999999999999,"15min":0.73699999999999999},"last":0.44500000000000001},{"interface":"front","average":{"1min":0.497,"5min":0.497,"15min":0.497},"min":{"1min":0.36199999999999999,"5min":0.36199999999999999,"15min":0.36199999999999999},"max":{"1min":0.80700000000000005,"5min":0.80700000000000005,"15min":0.80700000000000005},"last":0.31}]},{"osd":4,"last update":"Sun Nov 26 07:18:41 2023","interfaces":[{"interface":"back","average":{"1min":0.69399999999999995,"5min":0.69399999999999995,"15min":0.69399999999999995},"min":{"1min":0.48599999999999999,"5min":0.48599999999999999,"15min":0.48599999999999999},"max":{"1min":1.0229999999999999,"5min":1.0229999999999999,"15min":1.0229999999999999},"last":0.60499999999999998},{"interface":"front","average":{"1min":0.68000000000000005,"5min":0.68000000000000005,"15min":0.68000000000000005},"min":{"1min":0.49199999999999999,"5min":0.49199999999999999,"15min":0.49199999999999999},"max":{"1min":1.0629999999999999,"5min":1.0629999999999999,"15min":1.0629999999999999},"last":0.61599999999999999}]},{"osd":5,"last update":"Sun Nov 26 07:18:41 2023","interfaces":[{"interface":"back","average":{"1min":0.71899999999999997,"5min":0.71899999999999997,"15min":0.71899999999999997},"min":{"1min":0.49299999999999999,"5min":0.49299999999999999,"15min":0.49299999999999999},"max":{"1min":1.0880000000000001,"5min":1.0880000000000001,"15min":1.0880000000000001},"last":0.58799999999999997},{"interface":"front","average":{"1min":0.71699999999999997,"5min":0.71699999999999997,"15min":0.71699999999999997},"min":{"1min":0.53500000000000003,"5min":0.53500000000000003,"15min":0.53500000000000003},"max":{"1min":1.014,"5min":1.014,"15min":1.014},"last":0.63}]},{"osd":6,"last update":"Sun Nov 26 07:18:41 2023","interfaces":[{"interface":"back","average":{"1min":0.70399999999999996,"5min":0.70399999999999996,"15min":0.70399999999999996},"min":{"1min":0.50600000000000001,"5min":0.50600000000000001,"15min":0.50600000000000001},"max":{"1min":0.97599999999999998,"5min":0.97599999999999998,"15min":0.97599999999999998},"last":0.64500000000000002},{"interface":"front","average":{"1min":0.70099999999999996,"5min":0.70099999999999996,"15min":0.70099999999999996},"min":{"1min":0.55000000000000004,"5min":0.55000000000000004,"15min":0.55000000000000004},"max":{"1min":1.1339999999999999,"5min":1.1339999999999999,"15min":1.1339999999999999},"last":0.71499999999999997}]},{"osd":7,"last update":"Sun Nov 26 07:18:41 2023","interfaces":[{"interface":"back","average":{"1min":0.71499999999999997,"5min":0.71499999999999997,"15min":0.71499999999999997},"min":{"1min":0.55100000000000005,"5min":0.55100000000000005,"15min":0.55100000000000005},"max":{"1min":0.92900000000000005,"5min":0.92900000000000005,"15min":0.92900000000000005},"last":0.69999999999999996},{"interface":"front","average":{"1min":0.753,"5min":0.753,"15min":0.753},"min":{"1min":0.56100000000000005,"5min":0.56100000000000005,"15min":0.56100000000000005},"max":{"1min":1.024,"5min":1.024,"15min":1.024},"last":0.55000000000000004}]}]},{"osd":0,"up_from":13,"seq":55834574875,"num_pgs":31,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":28120,"kb_used_data":1100,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343720,"statfs":{"total":96636764160,"available":96607969280,"internally_reserved":0,"allocated":1126400,"data_stored":805501,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":7,"apply_latency_ms":7,"commit_latency_ns":7000000,"apply_latency_ns":7000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Sun Nov 26 07:18:41 2023","interfaces":[{"interface":"back","average":{"1min":0.43099999999999999,"5min":0.43099999999999999,"15min":0.43099999999999999},"min":{"1min":0.28000000000000003,"5min":0.28000000000000003,"15min":0.28000000000000003},"max":{"1min":0.63100000000000001,"5min":0.63100000000000001,"15min":0.63100000000000001},"last":0.35299999999999998},{"interface":"front","average":{"1min":0.48299999999999998,"5min":0.48299999999999998,"15min":0.48299999999999998},"min":{"1min":0.29899999999999999,"5min":0.29899999999999999,"15min":0.29899999999999999},"max":{"1min":1.133,"5min":1.133,"15min":1.133},"last":0.48299999999999998}]},{"osd":2,"last update":"Sun Nov 26 07:18:41 2023","interfaces":[{"interface":"back","average":{"1min":0.53700000000000003,"5min":0.53700000000000003,"15min":0.53700000000000003},"min":{"1min":0.40899999999999997,"5min":0.40899999999999997,"15min":0.40899999999999997},"max":{"1min":0.67400000000000004,"5min":0.67400000000000004,"15min":0.67400000000000004},"last":0.40300000000000002},{"interface":"front","average":{"1min":0.56399999999999995,"5min":0.56399999999999995,"15min":0.56399999999999995},"min":{"1min":0.36399999999999999,"5min":0.36399999999999999,"15min":0.36399999999999999},"max":{"1min":0.72499999999999998,"5min":0.72499999999999998,"15min":0.72499999999999998},"last":0.45900000000000002}]},{"osd":3,"last update":"Sun Nov 26 07:18:41 2023","interfaces":[{"interface":"back","average":{"1min":0.53100000000000003,"5min":0.53100000000000003,"15min":0.53100000000000003},"min":{"1min":0.35799999999999998,"5min":0.35799999999999998,"15min":0.35799999999999998},"max":{"1min":0.97399999999999998,"5min":0.97399999999999998,"15min":0.97399999999999998},"last":0.437},{"interface":"front","average":{"1min":0.56200000000000006,"5min":0.56200000000000006,"15min":0.56200000000000006},"min":{"1min":0.40600000000000003,"5min":0.40600000000000003,"15min":0.40600000000000003},"max":{"1min":1.2509999999999999,"5min":1.2509999999999999,"15min":1.2509999999999999},"last":0.41699999999999998}]},{"osd":4,"last update":"Sun Nov 26 07:18:41 2023","interfaces":[{"interface":"back","average":{"1min":0.75900000000000001,"5min":0.75900000000000001,"15min":0.75900000000000001},"min":{"1min":0.60999999999999999,"5min":0.60999999999999999,"15min":0.60999999999999999},"max":{"1min":1.2010000000000001,"5min":1.2010000000000001,"15min":1.2010000000000001},"last":0.67700000000000005},{"interface":"front","average":{"1min":0.78900000000000003,"5min":0.78900000000000003,"15min":0.78900000000000003},"min":{"1min":0.59999999999999998,"5min":0.59999999999999998,"15min":0.59999999999999998},"max":{"1min":0.99399999999999999,"5min":0.99399999999999999,"15min":0.99399999999999999},"last":0.63}]},{"osd":5,"last update":"Sun Nov 26 07:18:41 2023","interfaces":[{"interface":"back","average":{"1min":0.76000000000000001,"5min":0.76000000000000001,"15min":0.76000000000000001},"min":{"1min":0.53000000000000003,"5min":0.53000000000000003,"15min":0.53000000000000003},"max":{"1min":1.1579999999999999,"5min":1.1579999999999999,"15min":1.1579999999999999},"last":0.70699999999999996},{"interface":"front","average":{"1min":0.79700000000000004,"5min":0.79700000000000004,"15min":0.79700000000000004},"min":{"1min":0.59699999999999998,"5min":0.59699999999999998,"15min":0.59699999999999998},"max":{"1min":1.0409999999999999,"5min":1.0409999999999999,"15min":1.0409999999999999},"last":0.67100000000000004}]},{"osd":6,"last update":"Sun Nov 26 07:18:41 2023","interfaces":[{"interface":"back","average":{"1min":0.76400000000000001,"5min":0.76400000000000001,"15min":0.76400000000000001},"min":{"1min":0.56599999999999995,"5min":0.56599999999999995,"15min":0.56599999999999995},"max":{"1min":1.3029999999999999,"5min":1.3029999999999999,"15min":1.3029999999999999},"last":0.66200000000000003},{"interface":"front","average":{"1min":0.76900000000000002,"5min":0.76900000000000002,"15min":0.76900000000000002},"min":{"1min":0.58599999999999997,"5min":0.58599999999999997,"15min":0.58599999999999997},"max":{"1min":1.2210000000000001,"5min":1.2210000000000001,"15min":1.2210000000000001},"last":0.59699999999999998}]},{"osd":7,"last update":"Sun Nov 26 07:18:41 2023","interfaces":[{"interface":"back","average":{"1min":0.81100000000000005,"5min":0.81100000000000005,"15min":0.81100000000000005},"min":{"1min":0.59999999999999998,"5min":0.59999999999999998,"15min":0.59999999999999998},"max":{"1min":1.177,"5min":1.177,"15min":1.177},"last":0.69199999999999995},{"interface":"front","average":{"1min":0.79900000000000004,"5min":0.79900000000000004,"15min":0.79900000000000004},"min":{"1min":0.60899999999999999,"5min":0.60899999999999999,"15min":0.60899999999999999},"max":{"1min":1.2390000000000001,"5min":1.2390000000000001,"15min":1.2390000000000001},"last":0.55600000000000005}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":34,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":70,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1634,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":638,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":70,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":672,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-11-26T07:19:41.012 DEBUG:teuthology.orchestra.run.smithi143:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-11-26T07:19:41.161 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:41.159+0000 7f2b421e4640 1 Processor -- start 2023-11-26T07:19:41.161 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:41.159+0000 7f2b421e4640 1 -- start start 2023-11-26T07:19:41.163 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:41.159+0000 7f2b421e4640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f2b3c0d01b0 0x7f2b3c0d3be0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:19:41.163 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:41.159+0000 7f2b421e4640 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7f2b3c15d610 0x7f2b3c0cdc40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:19:41.163 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:41.159+0000 7f2b421e4640 1 -- --> v1:172.21.15.143:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2b3c0ce180 con 0x7f2b3c15d230 2023-11-26T07:19:41.163 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:41.159+0000 7f2b421e4640 1 -- --> v2:172.21.15.143:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2b3c0ce3b0 con 0x7f2b3c15d610 2023-11-26T07:19:41.164 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:41.159+0000 7f2b421e4640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2b3c0ce690 con 0x7f2b3c0d01b0 2023-11-26T07:19:41.164 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:41.159+0000 7f2b419e3640 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7f2b3c15d610 0x7f2b3c0cdc40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:19:41.164 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:41.159+0000 7f2b411e2640 1 --1- >> v1:172.21.15.143:6789/0 conn(0x7f2b3c15d230 0x7f2b3c0cfaa0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.143:6789/0 says I am v1:172.21.15.143:41754/0 (socket says 172.21.15.143:41754) 2023-11-26T07:19:41.164 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:41.159+0000 7f2b411e2640 1 -- 172.21.15.143:0/3543118279 learned_addr learned my addr 172.21.15.143:0/3543118279 (peer_addr_for_me v1:172.21.15.143:0/0) 2023-11-26T07:19:41.164 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:41.159+0000 7f2b409e1640 1 --2- 172.21.15.143:0/3543118279 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f2b3c0d01b0 0x7f2b3c0d3be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:19:41.164 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:41.159+0000 7f2b419e3640 1 -- 172.21.15.143:0/3543118279 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f2b3c0d01b0 msgr2=0x7f2b3c0d3be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-26T07:19:41.164 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:41.159+0000 7f2b419e3640 1 --2- 172.21.15.143:0/3543118279 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f2b3c0d01b0 0x7f2b3c0d3be0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:19:41.164 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:41.159+0000 7f2b419e3640 1 -- 172.21.15.143:0/3543118279 >> v1:172.21.15.143:6789/0 conn(0x7f2b3c15d230 legacy=0x7f2b3c0cfaa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:19:41.164 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:41.159+0000 7f2b33fff640 1 -- 172.21.15.143:0/3543118279 <== mon.0 v1:172.21.15.143:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1795345146 0 0) 0x7f2b3c0ce180 con 0x7f2b3c15d230 2023-11-26T07:19:41.165 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:41.159+0000 7f2b419e3640 1 -- 172.21.15.143:0/3543118279 --> v2:172.21.15.143:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2b3c0ce970 con 0x7f2b3c15d610 2023-11-26T07:19:41.166 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:41.163+0000 7f2b419e3640 1 --2- 172.21.15.143:0/3543118279 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f2b3c15d610 0x7f2b3c0cdc40 secure :-1 s=READY pgs=587 cs=0 l=1 rev1=1 crypto rx=0x7f2b34009d90 tx=0x7f2b3401c8c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=63879117247063fc server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:19:41.166 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:41.163+0000 7f2b33fff640 1 -- 172.21.15.143:0/3543118279 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2b340a5980 con 0x7f2b3c15d610 2023-11-26T07:19:41.167 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:41.163+0000 7f2b33fff640 1 -- 172.21.15.143:0/3543118279 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2b340a5ae0 con 0x7f2b3c15d610 2023-11-26T07:19:41.167 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:41.163+0000 7f2b33fff640 1 -- 172.21.15.143:0/3543118279 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2b340a5e70 con 0x7f2b3c15d610 2023-11-26T07:19:41.167 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:41.163+0000 7f2b421e4640 1 -- 172.21.15.143:0/3543118279 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f2b3c15d610 msgr2=0x7f2b3c0cdc40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:19:41.168 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:41.163+0000 7f2b421e4640 1 --2- 172.21.15.143:0/3543118279 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f2b3c15d610 0x7f2b3c0cdc40 secure :-1 s=READY pgs=587 cs=0 l=1 rev1=1 crypto rx=0x7f2b34009d90 tx=0x7f2b3401c8c0 comp rx=0 tx=0).stop 2023-11-26T07:19:41.168 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:41.163+0000 7f2b421e4640 1 -- 172.21.15.143:0/3543118279 shutdown_connections 2023-11-26T07:19:41.168 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:41.163+0000 7f2b421e4640 1 --2- 172.21.15.143:0/3543118279 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f2b3c15d610 0x7f2b3c0cdc40 unknown :-1 s=CLOSED pgs=587 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:19:41.168 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:41.163+0000 7f2b421e4640 1 --2- 172.21.15.143:0/3543118279 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f2b3c0d01b0 0x7f2b3c0d3be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:19:41.168 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:41.163+0000 7f2b421e4640 1 -- 172.21.15.143:0/3543118279 >> 172.21.15.143:0/3543118279 conn(0x7f2b3c0d30c0 msgr2=0x7f2b3c0d34d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:19:41.168 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:41.163+0000 7f2b421e4640 1 -- 172.21.15.143:0/3543118279 shutdown_connections 2023-11-26T07:19:41.168 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:41.163+0000 7f2b421e4640 1 -- 172.21.15.143:0/3543118279 wait complete. 2023-11-26T07:19:41.169 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:41.167+0000 7f2b421e4640 1 Processor -- start 2023-11-26T07:19:41.169 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:41.167+0000 7f2b421e4640 1 -- start start 2023-11-26T07:19:41.170 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:41.167+0000 7f2b421e4640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f2b3c0d01b0 0x7f2b3c154a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:19:41.170 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:41.167+0000 7f2b421e4640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f2b3c15d230 0x7f2b3c15bf10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:19:41.170 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:41.167+0000 7f2b421e4640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f2b3c15d610 0x7f2b3c0cc360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:19:41.170 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:41.167+0000 7f2b421e4640 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2b3c15c8f0 con 0x7f2b3c15d230 2023-11-26T07:19:41.171 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:41.167+0000 7f2b421e4640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2b3c15a510 con 0x7f2b3c0d01b0 2023-11-26T07:19:41.171 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:41.167+0000 7f2b421e4640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2b3c15a7f0 con 0x7f2b3c15d610 2023-11-26T07:19:41.171 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:41.167+0000 7f2b409e1640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f2b3c15d230 0x7f2b3c15bf10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:19:41.171 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:41.167+0000 7f2b409e1640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f2b3c15d230 0x7f2b3c15bf10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:52298/0 (socket says 172.21.15.143:52298) 2023-11-26T07:19:41.171 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:41.167+0000 7f2b409e1640 1 -- 172.21.15.143:0/955258283 learned_addr learned my addr 172.21.15.143:0/955258283 (peer_addr_for_me v2:172.21.15.143:0/0) 2023-11-26T07:19:41.171 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:41.167+0000 7f2b411e2640 1 --2- 172.21.15.143:0/955258283 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f2b3c0d01b0 0x7f2b3c154a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:19:41.171 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:41.167+0000 7f2b419e3640 1 --2- 172.21.15.143:0/955258283 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f2b3c15d610 0x7f2b3c0cc360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:19:41.172 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:41.167+0000 7f2b409e1640 1 -- 172.21.15.143:0/955258283 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f2b3c15d610 msgr2=0x7f2b3c0cc360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:19:41.172 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:41.167+0000 7f2b409e1640 1 --2- 172.21.15.143:0/955258283 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f2b3c15d610 0x7f2b3c0cc360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:19:41.173 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:41.167+0000 7f2b409e1640 1 -- 172.21.15.143:0/955258283 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f2b3c0d01b0 msgr2=0x7f2b3c154a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:19:41.173 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:41.167+0000 7f2b409e1640 1 --2- 172.21.15.143:0/955258283 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f2b3c0d01b0 0x7f2b3c154a20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:19:41.173 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:41.167+0000 7f2b409e1640 1 -- 172.21.15.143:0/955258283 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2b34009a70 con 0x7f2b3c15d230 2023-11-26T07:19:41.173 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:41.167+0000 7f2b409e1640 1 --2- 172.21.15.143:0/955258283 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f2b3c15d230 0x7f2b3c15bf10 secure :-1 s=READY pgs=588 cs=0 l=1 rev1=1 crypto rx=0x7f2b2c00d800 tx=0x7f2b2c00dcc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:19:41.173 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:41.167+0000 7f2b31ffb640 1 -- 172.21.15.143:0/955258283 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2b2c00b6c0 con 0x7f2b3c15d230 2023-11-26T07:19:41.173 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:41.167+0000 7f2b31ffb640 1 -- 172.21.15.143:0/955258283 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2b2c00b820 con 0x7f2b3c15d230 2023-11-26T07:19:41.173 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:41.167+0000 7f2b31ffb640 1 -- 172.21.15.143:0/955258283 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2b2c00bb90 con 0x7f2b3c15d230 2023-11-26T07:19:41.173 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:41.167+0000 7f2b421e4640 1 -- 172.21.15.143:0/955258283 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2b3c0b4a20 con 0x7f2b3c15d230 2023-11-26T07:19:41.174 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:41.167+0000 7f2b421e4640 1 -- 172.21.15.143:0/955258283 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2b3c157720 con 0x7f2b3c15d230 2023-11-26T07:19:41.174 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:41.167+0000 7f2b421e4640 1 -- 172.21.15.143:0/955258283 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2b3c067c10 con 0x7f2b3c15d230 2023-11-26T07:19:41.174 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:41.171+0000 7f2b31ffb640 1 -- 172.21.15.143:0/955258283 <== mon.0 v2:172.21.15.143:3300/0 4 ==== mgrmap(e 7) v1 ==== 86005+0+0 (secure 0 0 0) 0x7f2b2c00bcf0 con 0x7f2b3c15d230 2023-11-26T07:19:41.174 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:41.171+0000 7f2b31ffb640 1 --2- 172.21.15.143:0/955258283 >> [v2:172.21.15.143:6832/2150557544,v1:172.21.15.143:6833/2150557544] conn(0x7f2b20066620 0x7f2b20068ae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:19:41.175 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:41.171+0000 7f2b31ffb640 1 -- 172.21.15.143:0/955258283 <== mon.0 v2:172.21.15.143:3300/0 5 ==== osd_map(45..45 src has 1..45) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f2b2c084530 con 0x7f2b3c15d230 2023-11-26T07:19:41.175 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:41.171+0000 7f2b411e2640 1 --2- 172.21.15.143:0/955258283 >> [v2:172.21.15.143:6832/2150557544,v1:172.21.15.143:6833/2150557544] conn(0x7f2b20066620 0x7f2b20068ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:19:41.175 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:41.171+0000 7f2b411e2640 1 --2- 172.21.15.143:0/955258283 >> [v2:172.21.15.143:6832/2150557544,v1:172.21.15.143:6833/2150557544] conn(0x7f2b20066620 0x7f2b20068ae0 secure :-1 s=READY pgs=295 cs=0 l=1 rev1=1 crypto rx=0x7f2b38007fd0 tx=0x7f2b38007d60 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:19:41.179 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:41.175+0000 7f2b31ffb640 1 -- 172.21.15.143:0/955258283 <== mon.0 v2:172.21.15.143:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172649 (secure 0 0 0) 0x7f2b2c051850 con 0x7f2b3c15d230 2023-11-26T07:19:41.326 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:41.323+0000 7f2b421e4640 1 -- 172.21.15.143:0/955258283 --> [v2:172.21.15.143:6832/2150557544,v1:172.21.15.143:6833/2150557544] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f2b3c15aca0 con 0x7f2b20066620 2023-11-26T07:19:41.334 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:41.331+0000 7f2b31ffb640 1 -- 172.21.15.143:0/955258283 <== mgr.4100 v2:172.21.15.143:6832/2150557544 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+289690 (secure 0 0 0) 0x7f2b38004c80 con 0x7f2b20066620 2023-11-26T07:19:41.337 INFO:teuthology.orchestra.run.smithi143.stdout: 2023-11-26T07:19:41.338 INFO:teuthology.orchestra.run.smithi143.stderr:dumped all 2023-11-26T07:19:41.340 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:41.335+0000 7f2b421e4640 1 -- 172.21.15.143:0/955258283 >> [v2:172.21.15.143:6832/2150557544,v1:172.21.15.143:6833/2150557544] conn(0x7f2b20066620 msgr2=0x7f2b20068ae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:19:41.340 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:41.335+0000 7f2b421e4640 1 --2- 172.21.15.143:0/955258283 >> [v2:172.21.15.143:6832/2150557544,v1:172.21.15.143:6833/2150557544] conn(0x7f2b20066620 0x7f2b20068ae0 secure :-1 s=READY pgs=295 cs=0 l=1 rev1=1 crypto rx=0x7f2b38007fd0 tx=0x7f2b38007d60 comp rx=0 tx=0).stop 2023-11-26T07:19:41.340 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:41.335+0000 7f2b421e4640 1 -- 172.21.15.143:0/955258283 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f2b3c15d230 msgr2=0x7f2b3c15bf10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:19:41.340 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:41.335+0000 7f2b421e4640 1 --2- 172.21.15.143:0/955258283 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f2b3c15d230 0x7f2b3c15bf10 secure :-1 s=READY pgs=588 cs=0 l=1 rev1=1 crypto rx=0x7f2b2c00d800 tx=0x7f2b2c00dcc0 comp rx=0 tx=0).stop 2023-11-26T07:19:41.340 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:41.335+0000 7f2b421e4640 1 -- 172.21.15.143:0/955258283 shutdown_connections 2023-11-26T07:19:41.340 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:41.335+0000 7f2b421e4640 1 --2- 172.21.15.143:0/955258283 >> [v2:172.21.15.143:6832/2150557544,v1:172.21.15.143:6833/2150557544] conn(0x7f2b20066620 0x7f2b20068ae0 unknown :-1 s=CLOSED pgs=295 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:19:41.340 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:41.335+0000 7f2b421e4640 1 --2- 172.21.15.143:0/955258283 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f2b3c15d610 0x7f2b3c0cc360 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:19:41.340 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:41.335+0000 7f2b421e4640 1 --2- 172.21.15.143:0/955258283 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f2b3c15d230 0x7f2b3c15bf10 unknown :-1 s=CLOSED pgs=588 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:19:41.340 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:41.335+0000 7f2b421e4640 1 --2- 172.21.15.143:0/955258283 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f2b3c0d01b0 0x7f2b3c154a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:19:41.340 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:41.335+0000 7f2b421e4640 1 -- 172.21.15.143:0/955258283 >> 172.21.15.143:0/955258283 conn(0x7f2b3c0d30c0 msgr2=0x7f2b3c0b5300 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:19:41.340 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:41.335+0000 7f2b421e4640 1 -- 172.21.15.143:0/955258283 shutdown_connections 2023-11-26T07:19:41.341 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:41.335+0000 7f2b421e4640 1 -- 172.21.15.143:0/955258283 wait complete. 2023-11-26T07:19:41.357 INFO:teuthology.orchestra.run.smithi143.stdout:{"pg_ready":true,"pg_map":{"version":97,"stamp":"2023-11-26T07:19:39.713381+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592778,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":204,"num_write_kb":2430,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":156,"ondisk_log_size":156,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":185,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":221520,"kb_used_data":5352,"kb_used_omap":12,"kb_used_meta":216051,"kb_avail":754753200,"statfs":{"total":773094113280,"available":772867276800,"internally_reserved":0,"allocated":5480448,"data_stored":2906348,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12749,"internal_metadata":221236787},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":47,"apply_latency_ms":47,"commit_latency_ns":47000000,"apply_latency_ns":47000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"8.001659"},"pg_stats":[{"pgid":"7.3f","version":"0'0","reported_seq":11,"reported_epoch":45,"state":"active+clean","last_fresh":"2023-11-26T07:19:33.479237+0000","last_change":"2023-11-26T07:19:32.144487+0000","last_active":"2023-11-26T07:19:33.479237+0000","last_peered":"2023-11-26T07:19:33.479237+0000","last_clean":"2023-11-26T07:19:33.479237+0000","last_became_active":"2023-11-26T07:19:32.144186+0000","last_became_peered":"2023-11-26T07:19:32.144186+0000","last_unstale":"2023-11-26T07:19:33.479237+0000","last_undegraded":"2023-11-26T07:19:33.479237+0000","last_fullsized":"2023-11-26T07:19:33.479237+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:19:31.113498+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:19:31.113498+0000","last_clean_scrub_stamp":"2023-11-26T07:19:31.113498+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T11:39:14.851287+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.3e","version":"0'0","reported_seq":11,"reported_epoch":45,"state":"active+clean","last_fresh":"2023-11-26T07:19:35.452781+0000","last_change":"2023-11-26T07:19:32.149166+0000","last_active":"2023-11-26T07:19:35.452781+0000","last_peered":"2023-11-26T07:19:35.452781+0000","last_clean":"2023-11-26T07:19:35.452781+0000","last_became_active":"2023-11-26T07:19:32.148894+0000","last_became_peered":"2023-11-26T07:19:32.148894+0000","last_unstale":"2023-11-26T07:19:35.452781+0000","last_undegraded":"2023-11-26T07:19:35.452781+0000","last_fullsized":"2023-11-26T07:19:35.452781+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:19:31.113498+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:19:31.113498+0000","last_clean_scrub_stamp":"2023-11-26T07:19:31.113498+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T07:36:12.082239+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.3d","version":"0'0","reported_seq":11,"reported_epoch":45,"state":"active+clean","last_fresh":"2023-11-26T07:19:33.141402+0000","last_change":"2023-11-26T07:19:32.144951+0000","last_active":"2023-11-26T07:19:33.141402+0000","last_peered":"2023-11-26T07:19:33.141402+0000","last_clean":"2023-11-26T07:19:33.141402+0000","last_became_active":"2023-11-26T07:19:32.144447+0000","last_became_peered":"2023-11-26T07:19:32.144447+0000","last_unstale":"2023-11-26T07:19:33.141402+0000","last_undegraded":"2023-11-26T07:19:33.141402+0000","last_fullsized":"2023-11-26T07:19:33.141402+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:19:31.113498+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:19:31.113498+0000","last_clean_scrub_stamp":"2023-11-26T07:19:31.113498+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T17:07:30.335737+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.3c","version":"0'0","reported_seq":11,"reported_epoch":45,"state":"active+clean","last_fresh":"2023-11-26T07:19:35.781277+0000","last_change":"2023-11-26T07:19:32.151382+0000","last_active":"2023-11-26T07:19:35.781277+0000","last_peered":"2023-11-26T07:19:35.781277+0000","last_clean":"2023-11-26T07:19:35.781277+0000","last_became_active":"2023-11-26T07:19:32.151123+0000","last_became_peered":"2023-11-26T07:19:32.151123+0000","last_unstale":"2023-11-26T07:19:35.781277+0000","last_undegraded":"2023-11-26T07:19:35.781277+0000","last_fullsized":"2023-11-26T07:19:35.781277+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:19:31.113498+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:19:31.113498+0000","last_clean_scrub_stamp":"2023-11-26T07:19:31.113498+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T10:13:02.758558+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.3b","version":"0'0","reported_seq":11,"reported_epoch":45,"state":"active+clean","last_fresh":"2023-11-26T07:19:35.452479+0000","last_change":"2023-11-26T07:19:32.148869+0000","last_active":"2023-11-26T07:19:35.452479+0000","last_peered":"2023-11-26T07:19:35.452479+0000","last_clean":"2023-11-26T07:19:35.452479+0000","last_became_active":"2023-11-26T07:19:32.148584+0000","last_became_peered":"2023-11-26T07:19:32.148584+0000","last_unstale":"2023-11-26T07:19:35.452479+0000","last_undegraded":"2023-11-26T07:19:35.452479+0000","last_fullsized":"2023-11-26T07:19:35.452479+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:19:31.113498+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:19:31.113498+0000","last_clean_scrub_stamp":"2023-11-26T07:19:31.113498+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T11:13:03.640215+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.3a","version":"0'0","reported_seq":11,"reported_epoch":45,"state":"active+clean","last_fresh":"2023-11-26T07:19:35.045949+0000","last_change":"2023-11-26T07:19:32.147494+0000","last_active":"2023-11-26T07:19:35.045949+0000","last_peered":"2023-11-26T07:19:35.045949+0000","last_clean":"2023-11-26T07:19:35.045949+0000","last_became_active":"2023-11-26T07:19:32.147228+0000","last_became_peered":"2023-11-26T07:19:32.147228+0000","last_unstale":"2023-11-26T07:19:35.045949+0000","last_undegraded":"2023-11-26T07:19:35.045949+0000","last_fullsized":"2023-11-26T07:19:35.045949+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:19:31.113498+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:19:31.113498+0000","last_clean_scrub_stamp":"2023-11-26T07:19:31.113498+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T16:40:21.382290+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.39","version":"0'0","reported_seq":11,"reported_epoch":45,"state":"active+clean","last_fresh":"2023-11-26T07:19:33.478901+0000","last_change":"2023-11-26T07:19:32.161946+0000","last_active":"2023-11-26T07:19:33.478901+0000","last_peered":"2023-11-26T07:19:33.478901+0000","last_clean":"2023-11-26T07:19:33.478901+0000","last_became_active":"2023-11-26T07:19:32.161569+0000","last_became_peered":"2023-11-26T07:19:32.161569+0000","last_unstale":"2023-11-26T07:19:33.478901+0000","last_undegraded":"2023-11-26T07:19:33.478901+0000","last_fullsized":"2023-11-26T07:19:33.478901+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:19:31.113498+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:19:31.113498+0000","last_clean_scrub_stamp":"2023-11-26T07:19:31.113498+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T09:57:31.429974+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.38","version":"0'0","reported_seq":11,"reported_epoch":45,"state":"active+clean","last_fresh":"2023-11-26T07:19:35.780688+0000","last_change":"2023-11-26T07:19:32.150270+0000","last_active":"2023-11-26T07:19:35.780688+0000","last_peered":"2023-11-26T07:19:35.780688+0000","last_clean":"2023-11-26T07:19:35.780688+0000","last_became_active":"2023-11-26T07:19:32.149871+0000","last_became_peered":"2023-11-26T07:19:32.149871+0000","last_unstale":"2023-11-26T07:19:35.780688+0000","last_undegraded":"2023-11-26T07:19:35.780688+0000","last_fullsized":"2023-11-26T07:19:35.780688+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:19:31.113498+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:19:31.113498+0000","last_clean_scrub_stamp":"2023-11-26T07:19:31.113498+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T11:13:03.640215+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.37","version":"0'0","reported_seq":11,"reported_epoch":45,"state":"active+clean","last_fresh":"2023-11-26T07:19:35.781815+0000","last_change":"2023-11-26T07:19:32.162754+0000","last_active":"2023-11-26T07:19:35.781815+0000","last_peered":"2023-11-26T07:19:35.781815+0000","last_clean":"2023-11-26T07:19:35.781815+0000","last_became_active":"2023-11-26T07:19:32.162469+0000","last_became_peered":"2023-11-26T07:19:32.162469+0000","last_unstale":"2023-11-26T07:19:35.781815+0000","last_undegraded":"2023-11-26T07:19:35.781815+0000","last_fullsized":"2023-11-26T07:19:35.781815+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:19:31.113498+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:19:31.113498+0000","last_clean_scrub_stamp":"2023-11-26T07:19:31.113498+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T09:57:31.429974+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.36","version":"0'0","reported_seq":11,"reported_epoch":45,"state":"active+clean","last_fresh":"2023-11-26T07:19:35.781818+0000","last_change":"2023-11-26T07:19:32.152984+0000","last_active":"2023-11-26T07:19:35.781818+0000","last_peered":"2023-11-26T07:19:35.781818+0000","last_clean":"2023-11-26T07:19:35.781818+0000","last_became_active":"2023-11-26T07:19:32.152182+0000","last_became_peered":"2023-11-26T07:19:32.152182+0000","last_unstale":"2023-11-26T07:19:35.781818+0000","last_undegraded":"2023-11-26T07:19:35.781818+0000","last_fullsized":"2023-11-26T07:19:35.781818+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:19:31.113498+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:19:31.113498+0000","last_clean_scrub_stamp":"2023-11-26T07:19:31.113498+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T14:38:21.642327+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.35","version":"0'0","reported_seq":11,"reported_epoch":45,"state":"active+clean","last_fresh":"2023-11-26T07:19:35.781548+0000","last_change":"2023-11-26T07:19:32.169986+0000","last_active":"2023-11-26T07:19:35.781548+0000","last_peered":"2023-11-26T07:19:35.781548+0000","last_clean":"2023-11-26T07:19:35.781548+0000","last_became_active":"2023-11-26T07:19:32.166254+0000","last_became_peered":"2023-11-26T07:19:32.166254+0000","last_unstale":"2023-11-26T07:19:35.781548+0000","last_undegraded":"2023-11-26T07:19:35.781548+0000","last_fullsized":"2023-11-26T07:19:35.781548+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:19:31.113498+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:19:31.113498+0000","last_clean_scrub_stamp":"2023-11-26T07:19:31.113498+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T13:08:06.236414+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.34","version":"0'0","reported_seq":11,"reported_epoch":45,"state":"active+clean","last_fresh":"2023-11-26T07:19:35.452584+0000","last_change":"2023-11-26T07:19:32.152810+0000","last_active":"2023-11-26T07:19:35.452584+0000","last_peered":"2023-11-26T07:19:35.452584+0000","last_clean":"2023-11-26T07:19:35.452584+0000","last_became_active":"2023-11-26T07:19:32.152478+0000","last_became_peered":"2023-11-26T07:19:32.152478+0000","last_unstale":"2023-11-26T07:19:35.452584+0000","last_undegraded":"2023-11-26T07:19:35.452584+0000","last_fullsized":"2023-11-26T07:19:35.452584+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:19:31.113498+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:19:31.113498+0000","last_clean_scrub_stamp":"2023-11-26T07:19:31.113498+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T14:52:03.571786+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.33","version":"0'0","reported_seq":11,"reported_epoch":45,"state":"active+clean","last_fresh":"2023-11-26T07:19:33.141758+0000","last_change":"2023-11-26T07:19:32.144139+0000","last_active":"2023-11-26T07:19:33.141758+0000","last_peered":"2023-11-26T07:19:33.141758+0000","last_clean":"2023-11-26T07:19:33.141758+0000","last_became_active":"2023-11-26T07:19:32.143833+0000","last_became_peered":"2023-11-26T07:19:32.143833+0000","last_unstale":"2023-11-26T07:19:33.141758+0000","last_undegraded":"2023-11-26T07:19:33.141758+0000","last_fullsized":"2023-11-26T07:19:33.141758+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:19:31.113498+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:19:31.113498+0000","last_clean_scrub_stamp":"2023-11-26T07:19:31.113498+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T12:03:39.623971+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.32","version":"0'0","reported_seq":11,"reported_epoch":45,"state":"active+clean","last_fresh":"2023-11-26T07:19:35.781536+0000","last_change":"2023-11-26T07:19:32.162812+0000","last_active":"2023-11-26T07:19:35.781536+0000","last_peered":"2023-11-26T07:19:35.781536+0000","last_clean":"2023-11-26T07:19:35.781536+0000","last_became_active":"2023-11-26T07:19:32.162479+0000","last_became_peered":"2023-11-26T07:19:32.162479+0000","last_unstale":"2023-11-26T07:19:35.781536+0000","last_undegraded":"2023-11-26T07:19:35.781536+0000","last_fullsized":"2023-11-26T07:19:35.781536+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:19:31.113498+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:19:31.113498+0000","last_clean_scrub_stamp":"2023-11-26T07:19:31.113498+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T17:06:09.448286+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.31","version":"0'0","reported_seq":11,"reported_epoch":45,"state":"active+clean","last_fresh":"2023-11-26T07:19:35.045857+0000","last_change":"2023-11-26T07:19:32.151586+0000","last_active":"2023-11-26T07:19:35.045857+0000","last_peered":"2023-11-26T07:19:35.045857+0000","last_clean":"2023-11-26T07:19:35.045857+0000","last_became_active":"2023-11-26T07:19:32.151044+0000","last_became_peered":"2023-11-26T07:19:32.151044+0000","last_unstale":"2023-11-26T07:19:35.045857+0000","last_undegraded":"2023-11-26T07:19:35.045857+0000","last_fullsized":"2023-11-26T07:19:35.045857+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:19:31.113498+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:19:31.113498+0000","last_clean_scrub_stamp":"2023-11-26T07:19:31.113498+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T08:05:46.483096+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.30","version":"0'0","reported_seq":11,"reported_epoch":45,"state":"active+clean","last_fresh":"2023-11-26T07:19:35.779486+0000","last_change":"2023-11-26T07:19:32.141346+0000","last_active":"2023-11-26T07:19:35.779486+0000","last_peered":"2023-11-26T07:19:35.779486+0000","last_clean":"2023-11-26T07:19:35.779486+0000","last_became_active":"2023-11-26T07:19:32.140936+0000","last_became_peered":"2023-11-26T07:19:32.140936+0000","last_unstale":"2023-11-26T07:19:35.779486+0000","last_undegraded":"2023-11-26T07:19:35.779486+0000","last_fullsized":"2023-11-26T07:19:35.779486+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:19:31.113498+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:19:31.113498+0000","last_clean_scrub_stamp":"2023-11-26T07:19:31.113498+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T16:20:24.322478+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.2f","version":"0'0","reported_seq":11,"reported_epoch":45,"state":"active+clean","last_fresh":"2023-11-26T07:19:33.141851+0000","last_change":"2023-11-26T07:19:32.146031+0000","last_active":"2023-11-26T07:19:33.141851+0000","last_peered":"2023-11-26T07:19:33.141851+0000","last_clean":"2023-11-26T07:19:33.141851+0000","last_became_active":"2023-11-26T07:19:32.145722+0000","last_became_peered":"2023-11-26T07:19:32.145722+0000","last_unstale":"2023-11-26T07:19:33.141851+0000","last_undegraded":"2023-11-26T07:19:33.141851+0000","last_fullsized":"2023-11-26T07:19:33.141851+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:19:31.113498+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:19:31.113498+0000","last_clean_scrub_stamp":"2023-11-26T07:19:31.113498+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T14:52:03.571786+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.2e","version":"0'0","reported_seq":11,"reported_epoch":45,"state":"active+clean","last_fresh":"2023-11-26T07:19:33.141559+0000","last_change":"2023-11-26T07:19:32.152229+0000","last_active":"2023-11-26T07:19:33.141559+0000","last_peered":"2023-11-26T07:19:33.141559+0000","last_clean":"2023-11-26T07:19:33.141559+0000","last_became_active":"2023-11-26T07:19:32.151091+0000","last_became_peered":"2023-11-26T07:19:32.151091+0000","last_unstale":"2023-11-26T07:19:33.141559+0000","last_undegraded":"2023-11-26T07:19:33.141559+0000","last_fullsized":"2023-11-26T07:19:33.141559+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:19:31.113498+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:19:31.113498+0000","last_clean_scrub_stamp":"2023-11-26T07:19:31.113498+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T16:13:48.077132+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.2d","version":"0'0","reported_seq":11,"reported_epoch":45,"state":"active+clean","last_fresh":"2023-11-26T07:19:35.779927+0000","last_change":"2023-11-26T07:19:32.142222+0000","last_active":"2023-11-26T07:19:35.779927+0000","last_peered":"2023-11-26T07:19:35.779927+0000","last_clean":"2023-11-26T07:19:35.779927+0000","last_became_active":"2023-11-26T07:19:32.141931+0000","last_became_peered":"2023-11-26T07:19:32.141931+0000","last_unstale":"2023-11-26T07:19:35.779927+0000","last_undegraded":"2023-11-26T07:19:35.779927+0000","last_fullsized":"2023-11-26T07:19:35.779927+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:19:31.113498+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:19:31.113498+0000","last_clean_scrub_stamp":"2023-11-26T07:19:31.113498+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T19:14:08.178758+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.2c","version":"0'0","reported_seq":11,"reported_epoch":45,"state":"active+clean","last_fresh":"2023-11-26T07:19:33.141948+0000","last_change":"2023-11-26T07:19:32.144361+0000","last_active":"2023-11-26T07:19:33.141948+0000","last_peered":"2023-11-26T07:19:33.141948+0000","last_clean":"2023-11-26T07:19:33.141948+0000","last_became_active":"2023-11-26T07:19:32.144067+0000","last_became_peered":"2023-11-26T07:19:32.144067+0000","last_unstale":"2023-11-26T07:19:33.141948+0000","last_undegraded":"2023-11-26T07:19:33.141948+0000","last_fullsized":"2023-11-26T07:19:33.141948+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:19:31.113498+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:19:31.113498+0000","last_clean_scrub_stamp":"2023-11-26T07:19:31.113498+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T08:29:00.060229+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.2b","version":"0'0","reported_seq":11,"reported_epoch":45,"state":"active+clean","last_fresh":"2023-11-26T07:19:35.046029+0000","last_change":"2023-11-26T07:19:32.143656+0000","last_active":"2023-11-26T07:19:35.046029+0000","last_peered":"2023-11-26T07:19:35.046029+0000","last_clean":"2023-11-26T07:19:35.046029+0000","last_became_active":"2023-11-26T07:19:32.143366+0000","last_became_peered":"2023-11-26T07:19:32.143366+0000","last_unstale":"2023-11-26T07:19:35.046029+0000","last_undegraded":"2023-11-26T07:19:35.046029+0000","last_fullsized":"2023-11-26T07:19:35.046029+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:19:31.113498+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:19:31.113498+0000","last_clean_scrub_stamp":"2023-11-26T07:19:31.113498+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T14:26:16.211661+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.2a","version":"0'0","reported_seq":11,"reported_epoch":45,"state":"active+clean","last_fresh":"2023-11-26T07:19:35.781581+0000","last_change":"2023-11-26T07:19:32.157944+0000","last_active":"2023-11-26T07:19:35.781581+0000","last_peered":"2023-11-26T07:19:35.781581+0000","last_clean":"2023-11-26T07:19:35.781581+0000","last_became_active":"2023-11-26T07:19:32.157635+0000","last_became_peered":"2023-11-26T07:19:32.157635+0000","last_unstale":"2023-11-26T07:19:35.781581+0000","last_undegraded":"2023-11-26T07:19:35.781581+0000","last_fullsized":"2023-11-26T07:19:35.781581+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:19:31.113498+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:19:31.113498+0000","last_clean_scrub_stamp":"2023-11-26T07:19:31.113498+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T13:57:09.650424+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.29","version":"0'0","reported_seq":11,"reported_epoch":45,"state":"active+clean","last_fresh":"2023-11-26T07:19:35.779675+0000","last_change":"2023-11-26T07:19:32.141578+0000","last_active":"2023-11-26T07:19:35.779675+0000","last_peered":"2023-11-26T07:19:35.779675+0000","last_clean":"2023-11-26T07:19:35.779675+0000","last_became_active":"2023-11-26T07:19:32.141062+0000","last_became_peered":"2023-11-26T07:19:32.141062+0000","last_unstale":"2023-11-26T07:19:35.779675+0000","last_undegraded":"2023-11-26T07:19:35.779675+0000","last_fullsized":"2023-11-26T07:19:35.779675+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:19:31.113498+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:19:31.113498+0000","last_clean_scrub_stamp":"2023-11-26T07:19:31.113498+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T19:19:00.198799+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.28","version":"0'0","reported_seq":11,"reported_epoch":45,"state":"active+clean","last_fresh":"2023-11-26T07:19:35.779529+0000","last_change":"2023-11-26T07:19:32.142515+0000","last_active":"2023-11-26T07:19:35.779529+0000","last_peered":"2023-11-26T07:19:35.779529+0000","last_clean":"2023-11-26T07:19:35.779529+0000","last_became_active":"2023-11-26T07:19:32.141329+0000","last_became_peered":"2023-11-26T07:19:32.141329+0000","last_unstale":"2023-11-26T07:19:35.779529+0000","last_undegraded":"2023-11-26T07:19:35.779529+0000","last_fullsized":"2023-11-26T07:19:35.779529+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:19:31.113498+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:19:31.113498+0000","last_clean_scrub_stamp":"2023-11-26T07:19:31.113498+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T12:39:07.528966+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.27","version":"0'0","reported_seq":11,"reported_epoch":45,"state":"active+clean","last_fresh":"2023-11-26T07:19:35.780024+0000","last_change":"2023-11-26T07:19:32.165013+0000","last_active":"2023-11-26T07:19:35.780024+0000","last_peered":"2023-11-26T07:19:35.780024+0000","last_clean":"2023-11-26T07:19:35.780024+0000","last_became_active":"2023-11-26T07:19:32.164626+0000","last_became_peered":"2023-11-26T07:19:32.164626+0000","last_unstale":"2023-11-26T07:19:35.780024+0000","last_undegraded":"2023-11-26T07:19:35.780024+0000","last_fullsized":"2023-11-26T07:19:35.780024+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:19:31.113498+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:19:31.113498+0000","last_clean_scrub_stamp":"2023-11-26T07:19:31.113498+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T15:38:29.544244+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.26","version":"0'0","reported_seq":11,"reported_epoch":45,"state":"active+clean","last_fresh":"2023-11-26T07:19:33.141894+0000","last_change":"2023-11-26T07:19:32.148455+0000","last_active":"2023-11-26T07:19:33.141894+0000","last_peered":"2023-11-26T07:19:33.141894+0000","last_clean":"2023-11-26T07:19:33.141894+0000","last_became_active":"2023-11-26T07:19:32.148153+0000","last_became_peered":"2023-11-26T07:19:32.148153+0000","last_unstale":"2023-11-26T07:19:33.141894+0000","last_undegraded":"2023-11-26T07:19:33.141894+0000","last_fullsized":"2023-11-26T07:19:33.141894+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:19:31.113498+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:19:31.113498+0000","last_clean_scrub_stamp":"2023-11-26T07:19:31.113498+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T15:42:46.000146+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.25","version":"0'0","reported_seq":11,"reported_epoch":45,"state":"active+clean","last_fresh":"2023-11-26T07:19:33.141651+0000","last_change":"2023-11-26T07:19:32.144732+0000","last_active":"2023-11-26T07:19:33.141651+0000","last_peered":"2023-11-26T07:19:33.141651+0000","last_clean":"2023-11-26T07:19:33.141651+0000","last_became_active":"2023-11-26T07:19:32.144080+0000","last_became_peered":"2023-11-26T07:19:32.144080+0000","last_unstale":"2023-11-26T07:19:33.141651+0000","last_undegraded":"2023-11-26T07:19:33.141651+0000","last_fullsized":"2023-11-26T07:19:33.141651+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:19:31.113498+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:19:31.113498+0000","last_clean_scrub_stamp":"2023-11-26T07:19:31.113498+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T11:13:03.640215+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.24","version":"0'0","reported_seq":11,"reported_epoch":45,"state":"active+clean","last_fresh":"2023-11-26T07:19:35.452719+0000","last_change":"2023-11-26T07:19:32.152207+0000","last_active":"2023-11-26T07:19:35.452719+0000","last_peered":"2023-11-26T07:19:35.452719+0000","last_clean":"2023-11-26T07:19:35.452719+0000","last_became_active":"2023-11-26T07:19:32.151884+0000","last_became_peered":"2023-11-26T07:19:32.151884+0000","last_unstale":"2023-11-26T07:19:35.452719+0000","last_undegraded":"2023-11-26T07:19:35.452719+0000","last_fullsized":"2023-11-26T07:19:35.452719+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:19:31.113498+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:19:31.113498+0000","last_clean_scrub_stamp":"2023-11-26T07:19:31.113498+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T10:13:02.758558+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.23","version":"0'0","reported_seq":11,"reported_epoch":45,"state":"active+clean","last_fresh":"2023-11-26T07:19:35.779763+0000","last_change":"2023-11-26T07:19:32.161293+0000","last_active":"2023-11-26T07:19:35.779763+0000","last_peered":"2023-11-26T07:19:35.779763+0000","last_clean":"2023-11-26T07:19:35.779763+0000","last_became_active":"2023-11-26T07:19:32.160980+0000","last_became_peered":"2023-11-26T07:19:32.160980+0000","last_unstale":"2023-11-26T07:19:35.779763+0000","last_undegraded":"2023-11-26T07:19:35.779763+0000","last_fullsized":"2023-11-26T07:19:35.779763+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:19:31.113498+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:19:31.113498+0000","last_clean_scrub_stamp":"2023-11-26T07:19:31.113498+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T15:54:06.012692+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.22","version":"0'0","reported_seq":11,"reported_epoch":45,"state":"active+clean","last_fresh":"2023-11-26T07:19:33.141366+0000","last_change":"2023-11-26T07:19:32.148594+0000","last_active":"2023-11-26T07:19:33.141366+0000","last_peered":"2023-11-26T07:19:33.141366+0000","last_clean":"2023-11-26T07:19:33.141366+0000","last_became_active":"2023-11-26T07:19:32.146474+0000","last_became_peered":"2023-11-26T07:19:32.146474+0000","last_unstale":"2023-11-26T07:19:33.141366+0000","last_undegraded":"2023-11-26T07:19:33.141366+0000","last_fullsized":"2023-11-26T07:19:33.141366+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:19:31.113498+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:19:31.113498+0000","last_clean_scrub_stamp":"2023-11-26T07:19:31.113498+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T13:58:10.832349+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.21","version":"0'0","reported_seq":11,"reported_epoch":45,"state":"active+clean","last_fresh":"2023-11-26T07:19:35.045939+0000","last_change":"2023-11-26T07:19:32.146135+0000","last_active":"2023-11-26T07:19:35.045939+0000","last_peered":"2023-11-26T07:19:35.045939+0000","last_clean":"2023-11-26T07:19:35.045939+0000","last_became_active":"2023-11-26T07:19:32.145871+0000","last_became_peered":"2023-11-26T07:19:32.145871+0000","last_unstale":"2023-11-26T07:19:35.045939+0000","last_undegraded":"2023-11-26T07:19:35.045939+0000","last_fullsized":"2023-11-26T07:19:35.045939+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:19:31.113498+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:19:31.113498+0000","last_clean_scrub_stamp":"2023-11-26T07:19:31.113498+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T11:38:43.936569+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.20","version":"0'0","reported_seq":11,"reported_epoch":45,"state":"active+clean","last_fresh":"2023-11-26T07:19:33.141620+0000","last_change":"2023-11-26T07:19:32.162325+0000","last_active":"2023-11-26T07:19:33.141620+0000","last_peered":"2023-11-26T07:19:33.141620+0000","last_clean":"2023-11-26T07:19:33.141620+0000","last_became_active":"2023-11-26T07:19:32.161954+0000","last_became_peered":"2023-11-26T07:19:32.161954+0000","last_unstale":"2023-11-26T07:19:33.141620+0000","last_undegraded":"2023-11-26T07:19:33.141620+0000","last_fullsized":"2023-11-26T07:19:33.141620+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:19:31.113498+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:19:31.113498+0000","last_clean_scrub_stamp":"2023-11-26T07:19:31.113498+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T08:37:11.440586+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.1e","version":"0'0","reported_seq":11,"reported_epoch":45,"state":"active+clean","last_fresh":"2023-11-26T07:19:35.781173+0000","last_change":"2023-11-26T07:19:32.163638+0000","last_active":"2023-11-26T07:19:35.781173+0000","last_peered":"2023-11-26T07:19:35.781173+0000","last_clean":"2023-11-26T07:19:35.781173+0000","last_became_active":"2023-11-26T07:19:32.163380+0000","last_became_peered":"2023-11-26T07:19:32.163380+0000","last_unstale":"2023-11-26T07:19:35.781173+0000","last_undegraded":"2023-11-26T07:19:35.781173+0000","last_fullsized":"2023-11-26T07:19:35.781173+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:19:31.113498+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:19:31.113498+0000","last_clean_scrub_stamp":"2023-11-26T07:19:31.113498+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T10:27:02.955872+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.1f","version":"44'4","reported_seq":16,"reported_epoch":44,"state":"active+clean","last_fresh":"2023-11-26T07:19:32.172842+0000","last_change":"2023-11-26T07:19:31.148359+0000","last_active":"2023-11-26T07:19:32.172842+0000","last_peered":"2023-11-26T07:19:32.172842+0000","last_clean":"2023-11-26T07:19:32.172842+0000","last_became_active":"2023-11-26T07:19:31.148066+0000","last_became_peered":"2023-11-26T07:19:31.148066+0000","last_unstale":"2023-11-26T07:19:32.172842+0000","last_undegraded":"2023-11-26T07:19:32.172842+0000","last_fullsized":"2023-11-26T07:19:32.172842+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:19:30.106639+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:19:30.106639+0000","last_clean_scrub_stamp":"2023-11-26T07:19:30.106639+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T12:08:31.504338+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.1f","version":"0'0","reported_seq":11,"reported_epoch":45,"state":"active+clean","last_fresh":"2023-11-26T07:19:35.780092+0000","last_change":"2023-11-26T07:19:32.142782+0000","last_active":"2023-11-26T07:19:35.780092+0000","last_peered":"2023-11-26T07:19:35.780092+0000","last_clean":"2023-11-26T07:19:35.780092+0000","last_became_active":"2023-11-26T07:19:32.142493+0000","last_became_peered":"2023-11-26T07:19:32.142493+0000","last_unstale":"2023-11-26T07:19:35.780092+0000","last_undegraded":"2023-11-26T07:19:35.780092+0000","last_fullsized":"2023-11-26T07:19:35.780092+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:19:31.113498+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:19:31.113498+0000","last_clean_scrub_stamp":"2023-11-26T07:19:31.113498+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T10:27:02.955872+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.1e","version":"44'1","reported_seq":13,"reported_epoch":44,"state":"active+clean","last_fresh":"2023-11-26T07:19:32.159452+0000","last_change":"2023-11-26T07:19:31.144305+0000","last_active":"2023-11-26T07:19:32.159452+0000","last_peered":"2023-11-26T07:19:32.159452+0000","last_clean":"2023-11-26T07:19:32.159452+0000","last_became_active":"2023-11-26T07:19:31.143063+0000","last_became_peered":"2023-11-26T07:19:31.143063+0000","last_unstale":"2023-11-26T07:19:32.159452+0000","last_undegraded":"2023-11-26T07:19:32.159452+0000","last_fullsized":"2023-11-26T07:19:32.159452+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:19:30.106639+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:19:30.106639+0000","last_clean_scrub_stamp":"2023-11-26T07:19:30.106639+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T19:14:07.171899+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.1c","version":"0'0","reported_seq":11,"reported_epoch":45,"state":"active+clean","last_fresh":"2023-11-26T07:19:35.452754+0000","last_change":"2023-11-26T07:19:32.145834+0000","last_active":"2023-11-26T07:19:35.452754+0000","last_peered":"2023-11-26T07:19:35.452754+0000","last_clean":"2023-11-26T07:19:35.452754+0000","last_became_active":"2023-11-26T07:19:32.145557+0000","last_became_peered":"2023-11-26T07:19:32.145557+0000","last_unstale":"2023-11-26T07:19:35.452754+0000","last_undegraded":"2023-11-26T07:19:35.452754+0000","last_fullsized":"2023-11-26T07:19:35.452754+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:19:31.113498+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:19:31.113498+0000","last_clean_scrub_stamp":"2023-11-26T07:19:31.113498+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T12:30:43.967303+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.1d","version":"44'2","reported_seq":14,"reported_epoch":44,"state":"active+clean","last_fresh":"2023-11-26T07:19:32.173683+0000","last_change":"2023-11-26T07:19:31.150313+0000","last_active":"2023-11-26T07:19:32.173683+0000","last_peered":"2023-11-26T07:19:32.173683+0000","last_clean":"2023-11-26T07:19:32.173683+0000","last_became_active":"2023-11-26T07:19:31.150042+0000","last_became_peered":"2023-11-26T07:19:31.150042+0000","last_unstale":"2023-11-26T07:19:32.173683+0000","last_undegraded":"2023-11-26T07:19:32.173683+0000","last_fullsized":"2023-11-26T07:19:32.173683+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:19:30.106639+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:19:30.106639+0000","last_clean_scrub_stamp":"2023-11-26T07:19:30.106639+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T15:54:05.005833+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.1d","version":"0'0","reported_seq":11,"reported_epoch":45,"state":"active+clean","last_fresh":"2023-11-26T07:19:35.781089+0000","last_change":"2023-11-26T07:19:32.162941+0000","last_active":"2023-11-26T07:19:35.781089+0000","last_peered":"2023-11-26T07:19:35.781089+0000","last_clean":"2023-11-26T07:19:35.781089+0000","last_became_active":"2023-11-26T07:19:32.162043+0000","last_became_peered":"2023-11-26T07:19:32.162043+0000","last_unstale":"2023-11-26T07:19:35.781089+0000","last_undegraded":"2023-11-26T07:19:35.781089+0000","last_fullsized":"2023-11-26T07:19:35.781089+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:19:31.113498+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:19:31.113498+0000","last_clean_scrub_stamp":"2023-11-26T07:19:31.113498+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T15:38:29.544244+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.1c","version":"44'2","reported_seq":14,"reported_epoch":44,"state":"active+clean","last_fresh":"2023-11-26T07:19:32.167534+0000","last_change":"2023-11-26T07:19:31.141667+0000","last_active":"2023-11-26T07:19:32.167534+0000","last_peered":"2023-11-26T07:19:32.167534+0000","last_clean":"2023-11-26T07:19:32.167534+0000","last_became_active":"2023-11-26T07:19:31.141393+0000","last_became_peered":"2023-11-26T07:19:31.141393+0000","last_unstale":"2023-11-26T07:19:32.167534+0000","last_undegraded":"2023-11-26T07:19:32.167534+0000","last_fullsized":"2023-11-26T07:19:32.167534+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:19:30.106639+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:19:30.106639+0000","last_clean_scrub_stamp":"2023-11-26T07:19:30.106639+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T07:36:11.075380+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.1a","version":"0'0","reported_seq":11,"reported_epoch":45,"state":"active+clean","last_fresh":"2023-11-26T07:19:33.141556+0000","last_change":"2023-11-26T07:19:32.149961+0000","last_active":"2023-11-26T07:19:33.141556+0000","last_peered":"2023-11-26T07:19:33.141556+0000","last_clean":"2023-11-26T07:19:33.141556+0000","last_became_active":"2023-11-26T07:19:32.149627+0000","last_became_peered":"2023-11-26T07:19:32.149627+0000","last_unstale":"2023-11-26T07:19:33.141556+0000","last_undegraded":"2023-11-26T07:19:33.141556+0000","last_fullsized":"2023-11-26T07:19:33.141556+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:19:31.113498+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:19:31.113498+0000","last_clean_scrub_stamp":"2023-11-26T07:19:31.113498+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T08:11:43.416968+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.1b","version":"44'1","reported_seq":13,"reported_epoch":44,"state":"active+clean","last_fresh":"2023-11-26T07:19:32.173306+0000","last_change":"2023-11-26T07:19:31.149354+0000","last_active":"2023-11-26T07:19:32.173306+0000","last_peered":"2023-11-26T07:19:32.173306+0000","last_clean":"2023-11-26T07:19:32.173306+0000","last_became_active":"2023-11-26T07:19:31.149092+0000","last_became_peered":"2023-11-26T07:19:31.149092+0000","last_unstale":"2023-11-26T07:19:32.173306+0000","last_undegraded":"2023-11-26T07:19:32.173306+0000","last_fullsized":"2023-11-26T07:19:32.173306+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:19:30.106639+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:19:30.106639+0000","last_clean_scrub_stamp":"2023-11-26T07:19:30.106639+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T17:06:08.441427+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.1b","version":"0'0","reported_seq":11,"reported_epoch":45,"state":"active+clean","last_fresh":"2023-11-26T07:19:35.781370+0000","last_change":"2023-11-26T07:19:32.147918+0000","last_active":"2023-11-26T07:19:35.781370+0000","last_peered":"2023-11-26T07:19:35.781370+0000","last_clean":"2023-11-26T07:19:35.781370+0000","last_became_active":"2023-11-26T07:19:32.144906+0000","last_became_peered":"2023-11-26T07:19:32.144906+0000","last_unstale":"2023-11-26T07:19:35.781370+0000","last_undegraded":"2023-11-26T07:19:35.781370+0000","last_fullsized":"2023-11-26T07:19:35.781370+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:19:31.113498+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:19:31.113498+0000","last_clean_scrub_stamp":"2023-11-26T07:19:31.113498+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T07:36:12.082239+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.1a","version":"44'1","reported_seq":13,"reported_epoch":44,"state":"active+clean","last_fresh":"2023-11-26T07:19:32.158326+0000","last_change":"2023-11-26T07:19:31.139455+0000","last_active":"2023-11-26T07:19:32.158326+0000","last_peered":"2023-11-26T07:19:32.158326+0000","last_clean":"2023-11-26T07:19:32.158326+0000","last_became_active":"2023-11-26T07:19:31.139009+0000","last_became_peered":"2023-11-26T07:19:31.139009+0000","last_unstale":"2023-11-26T07:19:32.158326+0000","last_undegraded":"2023-11-26T07:19:32.158326+0000","last_fullsized":"2023-11-26T07:19:32.158326+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:19:30.106639+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:19:30.106639+0000","last_clean_scrub_stamp":"2023-11-26T07:19:30.106639+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T12:54:51.008938+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.18","version":"0'0","reported_seq":11,"reported_epoch":45,"state":"active+clean","last_fresh":"2023-11-26T07:19:35.780736+0000","last_change":"2023-11-26T07:19:32.150102+0000","last_active":"2023-11-26T07:19:35.780736+0000","last_peered":"2023-11-26T07:19:35.780736+0000","last_clean":"2023-11-26T07:19:35.780736+0000","last_became_active":"2023-11-26T07:19:32.149530+0000","last_became_peered":"2023-11-26T07:19:32.149530+0000","last_unstale":"2023-11-26T07:19:35.780736+0000","last_undegraded":"2023-11-26T07:19:35.780736+0000","last_fullsized":"2023-11-26T07:19:35.780736+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:19:31.113498+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:19:31.113498+0000","last_clean_scrub_stamp":"2023-11-26T07:19:31.113498+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T12:30:43.967303+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.19","version":"0'0","reported_seq":12,"reported_epoch":44,"state":"active+clean","last_fresh":"2023-11-26T07:19:32.131503+0000","last_change":"2023-11-26T07:19:31.146327+0000","last_active":"2023-11-26T07:19:32.131503+0000","last_peered":"2023-11-26T07:19:32.131503+0000","last_clean":"2023-11-26T07:19:32.131503+0000","last_became_active":"2023-11-26T07:19:31.145985+0000","last_became_peered":"2023-11-26T07:19:31.145985+0000","last_unstale":"2023-11-26T07:19:32.131503+0000","last_undegraded":"2023-11-26T07:19:32.131503+0000","last_fullsized":"2023-11-26T07:19:32.131503+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:19:30.106639+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:19:30.106639+0000","last_clean_scrub_stamp":"2023-11-26T07:19:30.106639+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T17:48:35.887457+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.19","version":"0'0","reported_seq":11,"reported_epoch":45,"state":"active+clean","last_fresh":"2023-11-26T07:19:35.779506+0000","last_change":"2023-11-26T07:19:32.141885+0000","last_active":"2023-11-26T07:19:35.779506+0000","last_peered":"2023-11-26T07:19:35.779506+0000","last_clean":"2023-11-26T07:19:35.779506+0000","last_became_active":"2023-11-26T07:19:32.141579+0000","last_became_peered":"2023-11-26T07:19:32.141579+0000","last_unstale":"2023-11-26T07:19:35.779506+0000","last_undegraded":"2023-11-26T07:19:35.779506+0000","last_fullsized":"2023-11-26T07:19:35.779506+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:19:31.113498+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:19:31.113498+0000","last_clean_scrub_stamp":"2023-11-26T07:19:31.113498+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T12:54:52.015797+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.18","version":"0'0","reported_seq":12,"reported_epoch":44,"state":"active+clean","last_fresh":"2023-11-26T07:19:32.134241+0000","last_change":"2023-11-26T07:19:31.140258+0000","last_active":"2023-11-26T07:19:32.134241+0000","last_peered":"2023-11-26T07:19:32.134241+0000","last_clean":"2023-11-26T07:19:32.134241+0000","last_became_active":"2023-11-26T07:19:31.140009+0000","last_became_peered":"2023-11-26T07:19:31.140009+0000","last_unstale":"2023-11-26T07:19:32.134241+0000","last_undegraded":"2023-11-26T07:19:32.134241+0000","last_fullsized":"2023-11-26T07:19:32.134241+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:19:30.106639+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:19:30.106639+0000","last_clean_scrub_stamp":"2023-11-26T07:19:30.106639+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T15:54:05.005833+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.16","version":"0'0","reported_seq":11,"reported_epoch":45,"state":"active+clean","last_fresh":"2023-11-26T07:19:33.141587+0000","last_change":"2023-11-26T07:19:32.150893+0000","last_active":"2023-11-26T07:19:33.141587+0000","last_peered":"2023-11-26T07:19:33.141587+0000","last_clean":"2023-11-26T07:19:33.141587+0000","last_became_active":"2023-11-26T07:19:32.150499+0000","last_became_peered":"2023-11-26T07:19:32.150499+0000","last_unstale":"2023-11-26T07:19:33.141587+0000","last_undegraded":"2023-11-26T07:19:33.141587+0000","last_fullsized":"2023-11-26T07:19:33.141587+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:19:31.113498+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:19:31.113498+0000","last_clean_scrub_stamp":"2023-11-26T07:19:31.113498+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T12:08:32.511197+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.17","version":"0'0","reported_seq":12,"reported_epoch":44,"state":"active+clean","last_fresh":"2023-11-26T07:19:32.135153+0000","last_change":"2023-11-26T07:19:31.134552+0000","last_active":"2023-11-26T07:19:32.135153+0000","last_peered":"2023-11-26T07:19:32.135153+0000","last_clean":"2023-11-26T07:19:32.135153+0000","last_became_active":"2023-11-26T07:19:31.134241+0000","last_became_peered":"2023-11-26T07:19:31.134241+0000","last_unstale":"2023-11-26T07:19:32.135153+0000","last_undegraded":"2023-11-26T07:19:32.135153+0000","last_fullsized":"2023-11-26T07:19:32.135153+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:19:30.106639+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:19:30.106639+0000","last_clean_scrub_stamp":"2023-11-26T07:19:30.106639+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T15:38:28.537385+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.17","version":"0'0","reported_seq":11,"reported_epoch":45,"state":"active+clean","last_fresh":"2023-11-26T07:19:33.479303+0000","last_change":"2023-11-26T07:19:32.145750+0000","last_active":"2023-11-26T07:19:33.479303+0000","last_peered":"2023-11-26T07:19:33.479303+0000","last_clean":"2023-11-26T07:19:33.479303+0000","last_became_active":"2023-11-26T07:19:32.145490+0000","last_became_peered":"2023-11-26T07:19:32.145490+0000","last_unstale":"2023-11-26T07:19:33.479303+0000","last_undegraded":"2023-11-26T07:19:33.479303+0000","last_fullsized":"2023-11-26T07:19:33.479303+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:19:31.113498+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:19:31.113498+0000","last_clean_scrub_stamp":"2023-11-26T07:19:31.113498+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T08:05:46.483096+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.16","version":"0'0","reported_seq":12,"reported_epoch":44,"state":"active+clean","last_fresh":"2023-11-26T07:19:32.136290+0000","last_change":"2023-11-26T07:19:31.144647+0000","last_active":"2023-11-26T07:19:32.136290+0000","last_peered":"2023-11-26T07:19:32.136290+0000","last_clean":"2023-11-26T07:19:32.136290+0000","last_became_active":"2023-11-26T07:19:31.143606+0000","last_became_peered":"2023-11-26T07:19:31.143606+0000","last_unstale":"2023-11-26T07:19:32.136290+0000","last_undegraded":"2023-11-26T07:19:32.136290+0000","last_fullsized":"2023-11-26T07:19:32.136290+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:19:30.106639+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:19:30.106639+0000","last_clean_scrub_stamp":"2023-11-26T07:19:30.106639+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T17:07:29.328878+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.14","version":"0'0","reported_seq":11,"reported_epoch":45,"state":"active+clean","last_fresh":"2023-11-26T07:19:35.046247+0000","last_change":"2023-11-26T07:19:32.157436+0000","last_active":"2023-11-26T07:19:35.046247+0000","last_peered":"2023-11-26T07:19:35.046247+0000","last_clean":"2023-11-26T07:19:35.046247+0000","last_became_active":"2023-11-26T07:19:32.157158+0000","last_became_peered":"2023-11-26T07:19:32.157158+0000","last_unstale":"2023-11-26T07:19:35.046247+0000","last_undegraded":"2023-11-26T07:19:35.046247+0000","last_fullsized":"2023-11-26T07:19:35.046247+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:19:31.113498+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:19:31.113498+0000","last_clean_scrub_stamp":"2023-11-26T07:19:31.113498+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T13:56:46.827916+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.15","version":"44'3","reported_seq":15,"reported_epoch":44,"state":"active+clean","last_fresh":"2023-11-26T07:19:32.170458+0000","last_change":"2023-11-26T07:19:31.147755+0000","last_active":"2023-11-26T07:19:32.170458+0000","last_peered":"2023-11-26T07:19:32.170458+0000","last_clean":"2023-11-26T07:19:32.170458+0000","last_became_active":"2023-11-26T07:19:31.147461+0000","last_became_peered":"2023-11-26T07:19:31.147461+0000","last_unstale":"2023-11-26T07:19:32.170458+0000","last_undegraded":"2023-11-26T07:19:32.170458+0000","last_fullsized":"2023-11-26T07:19:32.170458+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:19:30.106639+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:19:30.106639+0000","last_clean_scrub_stamp":"2023-11-26T07:19:30.106639+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T13:40:11.907176+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.15","version":"0'0","reported_seq":11,"reported_epoch":45,"state":"active+clean","last_fresh":"2023-11-26T07:19:33.479056+0000","last_change":"2023-11-26T07:19:32.164123+0000","last_active":"2023-11-26T07:19:33.479056+0000","last_peered":"2023-11-26T07:19:33.479056+0000","last_clean":"2023-11-26T07:19:33.479056+0000","last_became_active":"2023-11-26T07:19:32.163813+0000","last_became_peered":"2023-11-26T07:19:32.163813+0000","last_unstale":"2023-11-26T07:19:33.479056+0000","last_undegraded":"2023-11-26T07:19:33.479056+0000","last_fullsized":"2023-11-26T07:19:33.479056+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:19:31.113498+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:19:31.113498+0000","last_clean_scrub_stamp":"2023-11-26T07:19:31.113498+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T14:46:10.067648+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.14","version":"44'6","reported_seq":18,"reported_epoch":44,"state":"active+clean","last_fresh":"2023-11-26T07:19:32.176465+0000","last_change":"2023-11-26T07:19:31.131243+0000","last_active":"2023-11-26T07:19:32.176465+0000","last_peered":"2023-11-26T07:19:32.176465+0000","last_clean":"2023-11-26T07:19:32.176465+0000","last_became_active":"2023-11-26T07:19:31.130865+0000","last_became_peered":"2023-11-26T07:19:31.130865+0000","last_unstale":"2023-11-26T07:19:32.176465+0000","last_undegraded":"2023-11-26T07:19:32.176465+0000","last_fullsized":"2023-11-26T07:19:32.176465+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:19:30.106639+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:19:30.106639+0000","last_clean_scrub_stamp":"2023-11-26T07:19:30.106639+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T10:23:43.187890+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.12","version":"0'0","reported_seq":11,"reported_epoch":45,"state":"active+clean","last_fresh":"2023-11-26T07:19:35.779828+0000","last_change":"2023-11-26T07:19:32.140004+0000","last_active":"2023-11-26T07:19:35.779828+0000","last_peered":"2023-11-26T07:19:35.779828+0000","last_clean":"2023-11-26T07:19:35.779828+0000","last_became_active":"2023-11-26T07:19:32.139750+0000","last_became_peered":"2023-11-26T07:19:32.139750+0000","last_unstale":"2023-11-26T07:19:35.779828+0000","last_undegraded":"2023-11-26T07:19:35.779828+0000","last_fullsized":"2023-11-26T07:19:35.779828+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:19:31.113498+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:19:31.113498+0000","last_clean_scrub_stamp":"2023-11-26T07:19:31.113498+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T13:18:35.574099+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.13","version":"44'2","reported_seq":14,"reported_epoch":44,"state":"active+clean","last_fresh":"2023-11-26T07:19:32.175839+0000","last_change":"2023-11-26T07:19:31.148780+0000","last_active":"2023-11-26T07:19:32.175839+0000","last_peered":"2023-11-26T07:19:32.175839+0000","last_clean":"2023-11-26T07:19:32.175839+0000","last_became_active":"2023-11-26T07:19:31.148142+0000","last_became_peered":"2023-11-26T07:19:31.148142+0000","last_unstale":"2023-11-26T07:19:32.175839+0000","last_undegraded":"2023-11-26T07:19:32.175839+0000","last_fullsized":"2023-11-26T07:19:32.175839+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:19:30.106639+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:19:30.106639+0000","last_clean_scrub_stamp":"2023-11-26T07:19:30.106639+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T13:08:05.229555+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.13","version":"0'0","reported_seq":11,"reported_epoch":45,"state":"active+clean","last_fresh":"2023-11-26T07:19:35.781446+0000","last_change":"2023-11-26T07:19:32.146408+0000","last_active":"2023-11-26T07:19:35.781446+0000","last_peered":"2023-11-26T07:19:35.781446+0000","last_clean":"2023-11-26T07:19:35.781446+0000","last_became_active":"2023-11-26T07:19:32.146146+0000","last_became_peered":"2023-11-26T07:19:32.146146+0000","last_unstale":"2023-11-26T07:19:35.781446+0000","last_undegraded":"2023-11-26T07:19:35.781446+0000","last_fullsized":"2023-11-26T07:19:35.781446+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:19:31.113498+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:19:31.113498+0000","last_clean_scrub_stamp":"2023-11-26T07:19:31.113498+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T13:40:12.914035+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.12","version":"0'0","reported_seq":12,"reported_epoch":44,"state":"active+clean","last_fresh":"2023-11-26T07:19:32.131375+0000","last_change":"2023-11-26T07:19:31.144842+0000","last_active":"2023-11-26T07:19:32.131375+0000","last_peered":"2023-11-26T07:19:32.131375+0000","last_clean":"2023-11-26T07:19:32.131375+0000","last_became_active":"2023-11-26T07:19:31.144554+0000","last_became_peered":"2023-11-26T07:19:31.144554+0000","last_unstale":"2023-11-26T07:19:32.131375+0000","last_undegraded":"2023-11-26T07:19:32.131375+0000","last_fullsized":"2023-11-26T07:19:32.131375+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:19:30.106639+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:19:30.106639+0000","last_clean_scrub_stamp":"2023-11-26T07:19:30.106639+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T08:11:42.410109+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.10","version":"0'0","reported_seq":11,"reported_epoch":45,"state":"active+clean","last_fresh":"2023-11-26T07:19:35.045846+0000","last_change":"2023-11-26T07:19:32.152524+0000","last_active":"2023-11-26T07:19:35.045846+0000","last_peered":"2023-11-26T07:19:35.045846+0000","last_clean":"2023-11-26T07:19:35.045846+0000","last_became_active":"2023-11-26T07:19:32.152218+0000","last_became_peered":"2023-11-26T07:19:32.152218+0000","last_unstale":"2023-11-26T07:19:35.045846+0000","last_undegraded":"2023-11-26T07:19:35.045846+0000","last_fullsized":"2023-11-26T07:19:35.045846+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:19:31.113498+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:19:31.113498+0000","last_clean_scrub_stamp":"2023-11-26T07:19:31.113498+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T11:46:02.755553+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.11","version":"0'0","reported_seq":12,"reported_epoch":44,"state":"active+clean","last_fresh":"2023-11-26T07:19:32.132190+0000","last_change":"2023-11-26T07:19:31.148587+0000","last_active":"2023-11-26T07:19:32.132190+0000","last_peered":"2023-11-26T07:19:32.132190+0000","last_clean":"2023-11-26T07:19:32.132190+0000","last_became_active":"2023-11-26T07:19:31.148170+0000","last_became_peered":"2023-11-26T07:19:31.148170+0000","last_unstale":"2023-11-26T07:19:32.132190+0000","last_undegraded":"2023-11-26T07:19:32.132190+0000","last_fullsized":"2023-11-26T07:19:32.132190+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:19:30.106639+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:19:30.106639+0000","last_clean_scrub_stamp":"2023-11-26T07:19:30.106639+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T14:52:02.564927+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.11","version":"0'0","reported_seq":11,"reported_epoch":45,"state":"active+clean","last_fresh":"2023-11-26T07:19:33.479035+0000","last_change":"2023-11-26T07:19:32.140495+0000","last_active":"2023-11-26T07:19:33.479035+0000","last_peered":"2023-11-26T07:19:33.479035+0000","last_clean":"2023-11-26T07:19:33.479035+0000","last_became_active":"2023-11-26T07:19:32.140249+0000","last_became_peered":"2023-11-26T07:19:32.140249+0000","last_unstale":"2023-11-26T07:19:33.479035+0000","last_undegraded":"2023-11-26T07:19:33.479035+0000","last_fullsized":"2023-11-26T07:19:33.479035+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:19:31.113498+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:19:31.113498+0000","last_clean_scrub_stamp":"2023-11-26T07:19:31.113498+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T14:38:21.642327+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.10","version":"44'8","reported_seq":20,"reported_epoch":44,"state":"active+clean","last_fresh":"2023-11-26T07:19:32.173764+0000","last_change":"2023-11-26T07:19:31.139691+0000","last_active":"2023-11-26T07:19:32.173764+0000","last_peered":"2023-11-26T07:19:32.173764+0000","last_clean":"2023-11-26T07:19:32.173764+0000","last_became_active":"2023-11-26T07:19:31.139402+0000","last_became_peered":"2023-11-26T07:19:31.139402+0000","last_unstale":"2023-11-26T07:19:32.173764+0000","last_undegraded":"2023-11-26T07:19:32.173764+0000","last_fullsized":"2023-11-26T07:19:32.173764+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:19:30.106639+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:19:30.106639+0000","last_clean_scrub_stamp":"2023-11-26T07:19:30.106639+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T17:48:35.887457+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.e","version":"0'0","reported_seq":11,"reported_epoch":45,"state":"active+clean","last_fresh":"2023-11-26T07:19:33.142000+0000","last_change":"2023-11-26T07:19:32.147956+0000","last_active":"2023-11-26T07:19:33.142000+0000","last_peered":"2023-11-26T07:19:33.142000+0000","last_clean":"2023-11-26T07:19:33.142000+0000","last_became_active":"2023-11-26T07:19:32.147524+0000","last_became_peered":"2023-11-26T07:19:32.147524+0000","last_unstale":"2023-11-26T07:19:33.142000+0000","last_undegraded":"2023-11-26T07:19:33.142000+0000","last_fullsized":"2023-11-26T07:19:33.142000+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:19:31.113498+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:19:31.113498+0000","last_clean_scrub_stamp":"2023-11-26T07:19:31.113498+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T16:49:28.479524+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.f","version":"44'1","reported_seq":13,"reported_epoch":44,"state":"active+clean","last_fresh":"2023-11-26T07:19:32.173455+0000","last_change":"2023-11-26T07:19:31.140922+0000","last_active":"2023-11-26T07:19:32.173455+0000","last_peered":"2023-11-26T07:19:32.173455+0000","last_clean":"2023-11-26T07:19:32.173455+0000","last_became_active":"2023-11-26T07:19:31.139513+0000","last_became_peered":"2023-11-26T07:19:31.139513+0000","last_unstale":"2023-11-26T07:19:32.173455+0000","last_undegraded":"2023-11-26T07:19:32.173455+0000","last_fullsized":"2023-11-26T07:19:32.173455+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:19:30.106639+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:19:30.106639+0000","last_clean_scrub_stamp":"2023-11-26T07:19:30.106639+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T13:48:18.741812+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.f","version":"0'0","reported_seq":11,"reported_epoch":45,"state":"active+clean","last_fresh":"2023-11-26T07:19:35.781895+0000","last_change":"2023-11-26T07:19:32.145588+0000","last_active":"2023-11-26T07:19:35.781895+0000","last_peered":"2023-11-26T07:19:35.781895+0000","last_clean":"2023-11-26T07:19:35.781895+0000","last_became_active":"2023-11-26T07:19:32.145118+0000","last_became_peered":"2023-11-26T07:19:32.145118+0000","last_unstale":"2023-11-26T07:19:35.781895+0000","last_undegraded":"2023-11-26T07:19:35.781895+0000","last_fullsized":"2023-11-26T07:19:35.781895+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:19:31.113498+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:19:31.113498+0000","last_clean_scrub_stamp":"2023-11-26T07:19:31.113498+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T18:10:41.853254+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.e","version":"44'2","reported_seq":14,"reported_epoch":44,"state":"active+clean","last_fresh":"2023-11-26T07:19:32.165352+0000","last_change":"2023-11-26T07:19:31.141584+0000","last_active":"2023-11-26T07:19:32.165352+0000","last_peered":"2023-11-26T07:19:32.165352+0000","last_clean":"2023-11-26T07:19:32.165352+0000","last_became_active":"2023-11-26T07:19:31.141272+0000","last_became_peered":"2023-11-26T07:19:31.141272+0000","last_unstale":"2023-11-26T07:19:32.165352+0000","last_undegraded":"2023-11-26T07:19:32.165352+0000","last_fullsized":"2023-11-26T07:19:32.165352+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:19:30.106639+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:19:30.106639+0000","last_clean_scrub_stamp":"2023-11-26T07:19:30.106639+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T17:05:24.936030+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.c","version":"0'0","reported_seq":11,"reported_epoch":45,"state":"active+clean","last_fresh":"2023-11-26T07:19:35.779829+0000","last_change":"2023-11-26T07:19:32.144748+0000","last_active":"2023-11-26T07:19:35.779829+0000","last_peered":"2023-11-26T07:19:35.779829+0000","last_clean":"2023-11-26T07:19:35.779829+0000","last_became_active":"2023-11-26T07:19:32.144491+0000","last_became_peered":"2023-11-26T07:19:32.144491+0000","last_unstale":"2023-11-26T07:19:35.779829+0000","last_undegraded":"2023-11-26T07:19:35.779829+0000","last_fullsized":"2023-11-26T07:19:35.779829+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:19:31.113498+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:19:31.113498+0000","last_clean_scrub_stamp":"2023-11-26T07:19:31.113498+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T17:48:36.894316+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.d","version":"44'1","reported_seq":13,"reported_epoch":44,"state":"active+clean","last_fresh":"2023-11-26T07:19:32.172698+0000","last_change":"2023-11-26T07:19:31.145210+0000","last_active":"2023-11-26T07:19:32.172698+0000","last_peered":"2023-11-26T07:19:32.172698+0000","last_clean":"2023-11-26T07:19:32.172698+0000","last_became_active":"2023-11-26T07:19:31.144917+0000","last_became_peered":"2023-11-26T07:19:31.144917+0000","last_unstale":"2023-11-26T07:19:32.172698+0000","last_undegraded":"2023-11-26T07:19:32.172698+0000","last_fullsized":"2023-11-26T07:19:32.172698+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:19:30.106639+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:19:30.106639+0000","last_clean_scrub_stamp":"2023-11-26T07:19:30.106639+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T17:07:29.328878+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.d","version":"0'0","reported_seq":11,"reported_epoch":45,"state":"active+clean","last_fresh":"2023-11-26T07:19:35.779828+0000","last_change":"2023-11-26T07:19:32.165304+0000","last_active":"2023-11-26T07:19:35.779828+0000","last_peered":"2023-11-26T07:19:35.779828+0000","last_clean":"2023-11-26T07:19:35.779828+0000","last_became_active":"2023-11-26T07:19:32.165070+0000","last_became_peered":"2023-11-26T07:19:32.165070+0000","last_unstale":"2023-11-26T07:19:35.779828+0000","last_undegraded":"2023-11-26T07:19:35.779828+0000","last_fullsized":"2023-11-26T07:19:35.779828+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:19:31.113498+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:19:31.113498+0000","last_clean_scrub_stamp":"2023-11-26T07:19:31.113498+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T17:05:25.942889+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.c","version":"0'0","reported_seq":12,"reported_epoch":44,"state":"active+clean","last_fresh":"2023-11-26T07:19:32.134711+0000","last_change":"2023-11-26T07:19:31.147668+0000","last_active":"2023-11-26T07:19:32.134711+0000","last_peered":"2023-11-26T07:19:32.134711+0000","last_clean":"2023-11-26T07:19:32.134711+0000","last_became_active":"2023-11-26T07:19:31.146685+0000","last_became_peered":"2023-11-26T07:19:31.146685+0000","last_unstale":"2023-11-26T07:19:32.134711+0000","last_undegraded":"2023-11-26T07:19:32.134711+0000","last_fullsized":"2023-11-26T07:19:32.134711+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:19:30.106639+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:19:30.106639+0000","last_clean_scrub_stamp":"2023-11-26T07:19:30.106639+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T18:10:40.846395+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.a","version":"0'0","reported_seq":11,"reported_epoch":45,"state":"active+clean","last_fresh":"2023-11-26T07:19:35.045943+0000","last_change":"2023-11-26T07:19:32.156472+0000","last_active":"2023-11-26T07:19:35.045943+0000","last_peered":"2023-11-26T07:19:35.045943+0000","last_clean":"2023-11-26T07:19:35.045943+0000","last_became_active":"2023-11-26T07:19:32.156160+0000","last_became_peered":"2023-11-26T07:19:32.156160+0000","last_unstale":"2023-11-26T07:19:35.045943+0000","last_undegraded":"2023-11-26T07:19:35.045943+0000","last_fullsized":"2023-11-26T07:19:35.045943+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:19:31.113498+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:19:31.113498+0000","last_clean_scrub_stamp":"2023-11-26T07:19:31.113498+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T10:15:08.536351+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.b","version":"44'2","reported_seq":14,"reported_epoch":44,"state":"active+clean","last_fresh":"2023-11-26T07:19:32.173221+0000","last_change":"2023-11-26T07:19:31.149345+0000","last_active":"2023-11-26T07:19:32.173221+0000","last_peered":"2023-11-26T07:19:32.173221+0000","last_clean":"2023-11-26T07:19:32.173221+0000","last_became_active":"2023-11-26T07:19:31.149072+0000","last_became_peered":"2023-11-26T07:19:31.149072+0000","last_unstale":"2023-11-26T07:19:32.173221+0000","last_undegraded":"2023-11-26T07:19:32.173221+0000","last_fullsized":"2023-11-26T07:19:32.173221+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:19:30.106639+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:19:30.106639+0000","last_clean_scrub_stamp":"2023-11-26T07:19:30.106639+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T13:57:08.643565+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.b","version":"0'0","reported_seq":11,"reported_epoch":45,"state":"active+clean","last_fresh":"2023-11-26T07:19:35.781055+0000","last_change":"2023-11-26T07:19:32.160785+0000","last_active":"2023-11-26T07:19:35.781055+0000","last_peered":"2023-11-26T07:19:35.781055+0000","last_clean":"2023-11-26T07:19:35.781055+0000","last_became_active":"2023-11-26T07:19:32.160504+0000","last_became_peered":"2023-11-26T07:19:32.160504+0000","last_unstale":"2023-11-26T07:19:35.781055+0000","last_undegraded":"2023-11-26T07:19:35.781055+0000","last_fullsized":"2023-11-26T07:19:35.781055+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:19:31.113498+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:19:31.113498+0000","last_clean_scrub_stamp":"2023-11-26T07:19:31.113498+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T19:14:08.178758+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.a","version":"0'0","reported_seq":12,"reported_epoch":44,"state":"active+clean","last_fresh":"2023-11-26T07:19:32.138088+0000","last_change":"2023-11-26T07:19:31.146363+0000","last_active":"2023-11-26T07:19:32.138088+0000","last_peered":"2023-11-26T07:19:32.138088+0000","last_clean":"2023-11-26T07:19:32.138088+0000","last_became_active":"2023-11-26T07:19:31.146013+0000","last_became_peered":"2023-11-26T07:19:31.146013+0000","last_unstale":"2023-11-26T07:19:32.138088+0000","last_undegraded":"2023-11-26T07:19:32.138088+0000","last_fullsized":"2023-11-26T07:19:32.138088+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:19:30.106639+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:19:30.106639+0000","last_clean_scrub_stamp":"2023-11-26T07:19:30.106639+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T13:58:09.825490+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.8","version":"0'0","reported_seq":11,"reported_epoch":45,"state":"active+clean","last_fresh":"2023-11-26T07:19:33.141617+0000","last_change":"2023-11-26T07:19:32.142623+0000","last_active":"2023-11-26T07:19:33.141617+0000","last_peered":"2023-11-26T07:19:33.141617+0000","last_clean":"2023-11-26T07:19:33.141617+0000","last_became_active":"2023-11-26T07:19:32.142279+0000","last_became_peered":"2023-11-26T07:19:32.142279+0000","last_unstale":"2023-11-26T07:19:33.141617+0000","last_undegraded":"2023-11-26T07:19:33.141617+0000","last_fullsized":"2023-11-26T07:19:33.141617+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:19:31.113498+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:19:31.113498+0000","last_clean_scrub_stamp":"2023-11-26T07:19:31.113498+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T17:24:01.360879+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.9","version":"0'0","reported_seq":12,"reported_epoch":44,"state":"active+clean","last_fresh":"2023-11-26T07:19:32.131814+0000","last_change":"2023-11-26T07:19:31.148399+0000","last_active":"2023-11-26T07:19:32.131814+0000","last_peered":"2023-11-26T07:19:32.131814+0000","last_clean":"2023-11-26T07:19:32.131814+0000","last_became_active":"2023-11-26T07:19:31.148127+0000","last_became_peered":"2023-11-26T07:19:31.148127+0000","last_unstale":"2023-11-26T07:19:32.131814+0000","last_undegraded":"2023-11-26T07:19:32.131814+0000","last_fullsized":"2023-11-26T07:19:32.131814+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:19:30.106639+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:19:30.106639+0000","last_clean_scrub_stamp":"2023-11-26T07:19:30.106639+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T15:32:40.529918+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.9","version":"0'0","reported_seq":11,"reported_epoch":45,"state":"active+clean","last_fresh":"2023-11-26T07:19:35.780767+0000","last_change":"2023-11-26T07:19:32.172061+0000","last_active":"2023-11-26T07:19:35.780767+0000","last_peered":"2023-11-26T07:19:35.780767+0000","last_clean":"2023-11-26T07:19:35.780767+0000","last_became_active":"2023-11-26T07:19:32.171767+0000","last_became_peered":"2023-11-26T07:19:32.171767+0000","last_unstale":"2023-11-26T07:19:35.780767+0000","last_undegraded":"2023-11-26T07:19:35.780767+0000","last_fullsized":"2023-11-26T07:19:35.780767+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:19:31.113498+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:19:31.113498+0000","last_clean_scrub_stamp":"2023-11-26T07:19:31.113498+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T14:52:03.571786+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.8","version":"0'0","reported_seq":12,"reported_epoch":44,"state":"active+clean","last_fresh":"2023-11-26T07:19:32.131173+0000","last_change":"2023-11-26T07:19:31.144956+0000","last_active":"2023-11-26T07:19:32.131173+0000","last_peered":"2023-11-26T07:19:32.131173+0000","last_clean":"2023-11-26T07:19:32.131173+0000","last_became_active":"2023-11-26T07:19:31.144653+0000","last_became_peered":"2023-11-26T07:19:31.144653+0000","last_unstale":"2023-11-26T07:19:32.131173+0000","last_undegraded":"2023-11-26T07:19:32.131173+0000","last_fullsized":"2023-11-26T07:19:32.131173+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:19:30.106639+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:19:30.106639+0000","last_clean_scrub_stamp":"2023-11-26T07:19:30.106639+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T16:13:47.070273+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.7","version":"0'0","reported_seq":11,"reported_epoch":45,"state":"active+clean","last_fresh":"2023-11-26T07:19:35.780130+0000","last_change":"2023-11-26T07:19:32.152689+0000","last_active":"2023-11-26T07:19:35.780130+0000","last_peered":"2023-11-26T07:19:35.780130+0000","last_clean":"2023-11-26T07:19:35.780130+0000","last_became_active":"2023-11-26T07:19:32.151484+0000","last_became_peered":"2023-11-26T07:19:32.151484+0000","last_unstale":"2023-11-26T07:19:35.780130+0000","last_undegraded":"2023-11-26T07:19:35.780130+0000","last_fullsized":"2023-11-26T07:19:35.780130+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:19:31.113498+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:19:31.113498+0000","last_clean_scrub_stamp":"2023-11-26T07:19:31.113498+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T13:42:56.813910+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.6","version":"45'1","reported_seq":13,"reported_epoch":45,"state":"active+clean","last_fresh":"2023-11-26T07:19:35.046338+0000","last_change":"2023-11-26T07:19:31.143699+0000","last_active":"2023-11-26T07:19:35.046338+0000","last_peered":"2023-11-26T07:19:35.046338+0000","last_clean":"2023-11-26T07:19:35.046338+0000","last_became_active":"2023-11-26T07:19:31.143421+0000","last_became_peered":"2023-11-26T07:19:31.143421+0000","last_unstale":"2023-11-26T07:19:35.046338+0000","last_undegraded":"2023-11-26T07:19:35.046338+0000","last_fullsized":"2023-11-26T07:19:35.046338+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:19:30.106639+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:19:30.106639+0000","last_clean_scrub_stamp":"2023-11-26T07:19:30.106639+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T14:46:09.060789+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.4","version":"0'0","reported_seq":11,"reported_epoch":45,"state":"active+clean","last_fresh":"2023-11-26T07:19:35.781425+0000","last_change":"2023-11-26T07:19:32.154214+0000","last_active":"2023-11-26T07:19:35.781425+0000","last_peered":"2023-11-26T07:19:35.781425+0000","last_clean":"2023-11-26T07:19:35.781425+0000","last_became_active":"2023-11-26T07:19:32.153615+0000","last_became_peered":"2023-11-26T07:19:32.153615+0000","last_unstale":"2023-11-26T07:19:35.781425+0000","last_undegraded":"2023-11-26T07:19:35.781425+0000","last_fullsized":"2023-11-26T07:19:35.781425+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:19:31.113498+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:19:31.113498+0000","last_clean_scrub_stamp":"2023-11-26T07:19:31.113498+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T18:36:17.645226+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.5","version":"0'0","reported_seq":12,"reported_epoch":44,"state":"active+clean","last_fresh":"2023-11-26T07:19:32.132832+0000","last_change":"2023-11-26T07:19:31.148100+0000","last_active":"2023-11-26T07:19:32.132832+0000","last_peered":"2023-11-26T07:19:32.132832+0000","last_clean":"2023-11-26T07:19:32.132832+0000","last_became_active":"2023-11-26T07:19:31.147766+0000","last_became_peered":"2023-11-26T07:19:31.147766+0000","last_unstale":"2023-11-26T07:19:32.132832+0000","last_undegraded":"2023-11-26T07:19:32.132832+0000","last_fullsized":"2023-11-26T07:19:32.132832+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:19:30.106639+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:19:30.106639+0000","last_clean_scrub_stamp":"2023-11-26T07:19:30.106639+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T13:08:05.229555+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.5","version":"0'0","reported_seq":11,"reported_epoch":45,"state":"active+clean","last_fresh":"2023-11-26T07:19:33.479141+0000","last_change":"2023-11-26T07:19:32.145003+0000","last_active":"2023-11-26T07:19:33.479141+0000","last_peered":"2023-11-26T07:19:33.479141+0000","last_clean":"2023-11-26T07:19:33.479141+0000","last_became_active":"2023-11-26T07:19:32.143594+0000","last_became_peered":"2023-11-26T07:19:32.143594+0000","last_unstale":"2023-11-26T07:19:33.479141+0000","last_undegraded":"2023-11-26T07:19:33.479141+0000","last_fullsized":"2023-11-26T07:19:33.479141+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:19:31.113498+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:19:31.113498+0000","last_clean_scrub_stamp":"2023-11-26T07:19:31.113498+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T18:10:41.853254+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.4","version":"0'0","reported_seq":12,"reported_epoch":44,"state":"active+clean","last_fresh":"2023-11-26T07:19:32.146730+0000","last_change":"2023-11-26T07:19:31.144386+0000","last_active":"2023-11-26T07:19:32.146730+0000","last_peered":"2023-11-26T07:19:32.146730+0000","last_clean":"2023-11-26T07:19:32.146730+0000","last_became_active":"2023-11-26T07:19:31.144106+0000","last_became_peered":"2023-11-26T07:19:31.144106+0000","last_unstale":"2023-11-26T07:19:32.146730+0000","last_undegraded":"2023-11-26T07:19:32.146730+0000","last_fullsized":"2023-11-26T07:19:32.146730+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:19:30.106639+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:19:30.106639+0000","last_clean_scrub_stamp":"2023-11-26T07:19:30.106639+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T13:18:34.567240+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.2","version":"0'0","reported_seq":11,"reported_epoch":45,"state":"active+clean","last_fresh":"2023-11-26T07:19:35.046072+0000","last_change":"2023-11-26T07:19:32.163779+0000","last_active":"2023-11-26T07:19:35.046072+0000","last_peered":"2023-11-26T07:19:35.046072+0000","last_clean":"2023-11-26T07:19:35.046072+0000","last_became_active":"2023-11-26T07:19:32.163503+0000","last_became_peered":"2023-11-26T07:19:32.163503+0000","last_unstale":"2023-11-26T07:19:35.046072+0000","last_undegraded":"2023-11-26T07:19:35.046072+0000","last_fullsized":"2023-11-26T07:19:35.046072+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:19:31.113498+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:19:31.113498+0000","last_clean_scrub_stamp":"2023-11-26T07:19:31.113498+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T10:09:45.601591+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.3","version":"0'0","reported_seq":12,"reported_epoch":44,"state":"active+clean","last_fresh":"2023-11-26T07:19:32.132520+0000","last_change":"2023-11-26T07:19:31.146929+0000","last_active":"2023-11-26T07:19:32.132520+0000","last_peered":"2023-11-26T07:19:32.132520+0000","last_clean":"2023-11-26T07:19:32.132520+0000","last_became_active":"2023-11-26T07:19:31.146630+0000","last_became_peered":"2023-11-26T07:19:31.146630+0000","last_unstale":"2023-11-26T07:19:32.132520+0000","last_undegraded":"2023-11-26T07:19:32.132520+0000","last_fullsized":"2023-11-26T07:19:32.132520+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:19:30.106639+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:19:30.106639+0000","last_clean_scrub_stamp":"2023-11-26T07:19:30.106639+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T18:36:16.638367+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.3","version":"0'0","reported_seq":11,"reported_epoch":45,"state":"active+clean","last_fresh":"2023-11-26T07:19:35.046021+0000","last_change":"2023-11-26T07:19:32.156475+0000","last_active":"2023-11-26T07:19:35.046021+0000","last_peered":"2023-11-26T07:19:35.046021+0000","last_clean":"2023-11-26T07:19:35.046021+0000","last_became_active":"2023-11-26T07:19:32.156262+0000","last_became_peered":"2023-11-26T07:19:32.156262+0000","last_unstale":"2023-11-26T07:19:35.046021+0000","last_undegraded":"2023-11-26T07:19:35.046021+0000","last_fullsized":"2023-11-26T07:19:35.046021+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:19:31.113498+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:19:31.113498+0000","last_clean_scrub_stamp":"2023-11-26T07:19:31.113498+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T17:37:48.397190+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.2","version":"0'0","reported_seq":12,"reported_epoch":44,"state":"active+clean","last_fresh":"2023-11-26T07:19:32.133786+0000","last_change":"2023-11-26T07:19:31.147730+0000","last_active":"2023-11-26T07:19:32.133786+0000","last_peered":"2023-11-26T07:19:32.133786+0000","last_clean":"2023-11-26T07:19:32.133786+0000","last_became_active":"2023-11-26T07:19:31.147374+0000","last_became_peered":"2023-11-26T07:19:31.147374+0000","last_unstale":"2023-11-26T07:19:32.133786+0000","last_undegraded":"2023-11-26T07:19:32.133786+0000","last_fullsized":"2023-11-26T07:19:32.133786+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:19:30.106639+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:19:30.106639+0000","last_clean_scrub_stamp":"2023-11-26T07:19:30.106639+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T15:54:05.005833+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.0","version":"0'0","reported_seq":11,"reported_epoch":45,"state":"active+clean","last_fresh":"2023-11-26T07:19:35.452409+0000","last_change":"2023-11-26T07:19:32.164869+0000","last_active":"2023-11-26T07:19:35.452409+0000","last_peered":"2023-11-26T07:19:35.452409+0000","last_clean":"2023-11-26T07:19:35.452409+0000","last_became_active":"2023-11-26T07:19:32.164562+0000","last_became_peered":"2023-11-26T07:19:32.164562+0000","last_unstale":"2023-11-26T07:19:35.452409+0000","last_undegraded":"2023-11-26T07:19:35.452409+0000","last_fullsized":"2023-11-26T07:19:35.452409+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:19:31.113498+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:19:31.113498+0000","last_clean_scrub_stamp":"2023-11-26T07:19:31.113498+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T12:08:32.511197+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.1","version":"0'0","reported_seq":12,"reported_epoch":44,"state":"active+clean","last_fresh":"2023-11-26T07:19:32.143839+0000","last_change":"2023-11-26T07:19:31.145269+0000","last_active":"2023-11-26T07:19:32.143839+0000","last_peered":"2023-11-26T07:19:32.143839+0000","last_clean":"2023-11-26T07:19:32.143839+0000","last_became_active":"2023-11-26T07:19:31.144994+0000","last_became_peered":"2023-11-26T07:19:31.144994+0000","last_unstale":"2023-11-26T07:19:32.143839+0000","last_undegraded":"2023-11-26T07:19:32.143839+0000","last_fullsized":"2023-11-26T07:19:32.143839+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:19:30.106639+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:19:30.106639+0000","last_clean_scrub_stamp":"2023-11-26T07:19:30.106639+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T17:48:35.887457+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.1","version":"0'0","reported_seq":11,"reported_epoch":45,"state":"active+clean","last_fresh":"2023-11-26T07:19:33.141359+0000","last_change":"2023-11-26T07:19:32.144600+0000","last_active":"2023-11-26T07:19:33.141359+0000","last_peered":"2023-11-26T07:19:33.141359+0000","last_clean":"2023-11-26T07:19:33.141359+0000","last_became_active":"2023-11-26T07:19:32.144402+0000","last_became_peered":"2023-11-26T07:19:32.144402+0000","last_unstale":"2023-11-26T07:19:33.141359+0000","last_undegraded":"2023-11-26T07:19:33.141359+0000","last_fullsized":"2023-11-26T07:19:33.141359+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:19:31.113498+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:19:31.113498+0000","last_clean_scrub_stamp":"2023-11-26T07:19:31.113498+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T13:42:56.813910+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.0","version":"0'0","reported_seq":12,"reported_epoch":44,"state":"active+clean","last_fresh":"2023-11-26T07:19:32.134488+0000","last_change":"2023-11-26T07:19:31.146523+0000","last_active":"2023-11-26T07:19:32.134488+0000","last_peered":"2023-11-26T07:19:32.134488+0000","last_clean":"2023-11-26T07:19:32.134488+0000","last_became_active":"2023-11-26T07:19:31.146168+0000","last_became_peered":"2023-11-26T07:19:31.146168+0000","last_unstale":"2023-11-26T07:19:32.134488+0000","last_undegraded":"2023-11-26T07:19:32.134488+0000","last_fullsized":"2023-11-26T07:19:32.134488+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:19:30.106639+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:19:30.106639+0000","last_clean_scrub_stamp":"2023-11-26T07:19:30.106639+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T13:18:34.567240+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.6","version":"0'0","reported_seq":11,"reported_epoch":45,"state":"active+clean","last_fresh":"2023-11-26T07:19:35.781869+0000","last_change":"2023-11-26T07:19:32.153162+0000","last_active":"2023-11-26T07:19:35.781869+0000","last_peered":"2023-11-26T07:19:35.781869+0000","last_clean":"2023-11-26T07:19:35.781869+0000","last_became_active":"2023-11-26T07:19:32.152536+0000","last_became_peered":"2023-11-26T07:19:32.152536+0000","last_unstale":"2023-11-26T07:19:35.781869+0000","last_undegraded":"2023-11-26T07:19:35.781869+0000","last_fullsized":"2023-11-26T07:19:35.781869+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:19:31.113498+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:19:31.113498+0000","last_clean_scrub_stamp":"2023-11-26T07:19:31.113498+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T14:46:10.067648+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.7","version":"44'2","reported_seq":14,"reported_epoch":44,"state":"active+clean","last_fresh":"2023-11-26T07:19:32.171344+0000","last_change":"2023-11-26T07:19:31.146427+0000","last_active":"2023-11-26T07:19:32.171344+0000","last_peered":"2023-11-26T07:19:32.171344+0000","last_clean":"2023-11-26T07:19:32.171344+0000","last_became_active":"2023-11-26T07:19:31.146044+0000","last_became_peered":"2023-11-26T07:19:31.146044+0000","last_unstale":"2023-11-26T07:19:32.171344+0000","last_undegraded":"2023-11-26T07:19:32.171344+0000","last_fullsized":"2023-11-26T07:19:32.171344+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:19:30.106639+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:19:30.106639+0000","last_clean_scrub_stamp":"2023-11-26T07:19:30.106639+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T16:13:47.070273+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.0","version":"15'117","reported_seq":184,"reported_epoch":44,"state":"active+clean","last_fresh":"2023-11-26T07:19:32.131171+0000","last_change":"2023-11-26T07:17:43.827802+0000","last_active":"2023-11-26T07:19:32.131171+0000","last_peered":"2023-11-26T07:19:32.131171+0000","last_clean":"2023-11-26T07:19:32.131171+0000","last_became_active":"2023-11-26T07:17:43.827308+0000","last_became_peered":"2023-11-26T07:17:43.827308+0000","last_unstale":"2023-11-26T07:19:32.131171+0000","last_undegraded":"2023-11-26T07:19:32.131171+0000","last_fullsized":"2023-11-26T07:19:32.131171+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:17:42.160945+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:17:42.160945+0000","last_clean_scrub_stamp":"2023-11-26T07:17:42.160945+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T08:17:54.167366+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":7,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":6,"num_pg":32,"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":39,"ondisk_log_size":39,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574873,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":28120,"kb_used_data":1100,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343720,"statfs":{"total":96636764160,"available":96607969280,"internally_reserved":0,"allocated":1126400,"data_stored":805557,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Nov 26 07:18:42 2023","interfaces":[{"interface":"back","average":{"1min":0.71499999999999997,"5min":0.71499999999999997,"15min":0.71499999999999997},"min":{"1min":0.54300000000000004,"5min":0.54300000000000004,"15min":0.54300000000000004},"max":{"1min":0.91500000000000004,"5min":0.91500000000000004,"15min":0.91500000000000004},"last":0.63500000000000001},{"interface":"front","average":{"1min":0.70899999999999996,"5min":0.70899999999999996,"15min":0.70899999999999996},"min":{"1min":0.52900000000000003,"5min":0.52900000000000003,"15min":0.52900000000000003},"max":{"1min":1.0980000000000001,"5min":1.0980000000000001,"15min":1.0980000000000001},"last":0.86699999999999999}]},{"osd":1,"last update":"Sun Nov 26 07:18:42 2023","interfaces":[{"interface":"back","average":{"1min":0.68200000000000005,"5min":0.68200000000000005,"15min":0.68200000000000005},"min":{"1min":0.47999999999999998,"5min":0.47999999999999998,"15min":0.47999999999999998},"max":{"1min":1.0669999999999999,"5min":1.0669999999999999,"15min":1.0669999999999999},"last":0.68700000000000006},{"interface":"front","average":{"1min":0.83899999999999997,"5min":0.83899999999999997,"15min":0.83899999999999997},"min":{"1min":0.47599999999999998,"5min":0.47599999999999998,"15min":0.47599999999999998},"max":{"1min":4.5309999999999997,"5min":4.5309999999999997,"15min":4.5309999999999997},"last":0.48899999999999999}]},{"osd":2,"last update":"Sun Nov 26 07:18:42 2023","interfaces":[{"interface":"back","average":{"1min":0.67600000000000005,"5min":0.67600000000000005,"15min":0.67600000000000005},"min":{"1min":0.52000000000000002,"5min":0.52000000000000002,"15min":0.52000000000000002},"max":{"1min":0.873,"5min":0.873,"15min":0.873},"last":0.56200000000000006},{"interface":"front","average":{"1min":0.71299999999999997,"5min":0.71299999999999997,"15min":0.71299999999999997},"min":{"1min":0.54700000000000004,"5min":0.54700000000000004,"15min":0.54700000000000004},"max":{"1min":1.1339999999999999,"5min":1.1339999999999999,"15min":1.1339999999999999},"last":0.73999999999999999}]},{"osd":3,"last update":"Sun Nov 26 07:18:42 2023","interfaces":[{"interface":"back","average":{"1min":0.69699999999999995,"5min":0.69699999999999995,"15min":0.69699999999999995},"min":{"1min":0.51800000000000002,"5min":0.51800000000000002,"15min":0.51800000000000002},"max":{"1min":1.026,"5min":1.026,"15min":1.026},"last":0.626},{"interface":"front","average":{"1min":0.73299999999999998,"5min":0.73299999999999998,"15min":0.73299999999999998},"min":{"1min":0.50600000000000001,"5min":0.50600000000000001,"15min":0.50600000000000001},"max":{"1min":1.4490000000000001,"5min":1.4490000000000001,"15min":1.4490000000000001},"last":0.71899999999999997}]},{"osd":4,"last update":"Sun Nov 26 07:18:42 2023","interfaces":[{"interface":"back","average":{"1min":0.59999999999999998,"5min":0.59999999999999998,"15min":0.59999999999999998},"min":{"1min":0.39400000000000002,"5min":0.39400000000000002,"15min":0.39400000000000002},"max":{"1min":0.94799999999999995,"5min":0.94799999999999995,"15min":0.94799999999999995},"last":0.77000000000000002},{"interface":"front","average":{"1min":0.67300000000000004,"5min":0.67300000000000004,"15min":0.67300000000000004},"min":{"1min":0.436,"5min":0.436,"15min":0.436},"max":{"1min":1.5509999999999999,"5min":1.5509999999999999,"15min":1.5509999999999999},"last":0.67100000000000004}]},{"osd":5,"last update":"Sun Nov 26 07:18:42 2023","interfaces":[{"interface":"back","average":{"1min":0.66600000000000004,"5min":0.66600000000000004,"15min":0.66600000000000004},"min":{"1min":0.443,"5min":0.443,"15min":0.443},"max":{"1min":1.415,"5min":1.415,"15min":1.415},"last":0.84999999999999998},{"interface":"front","average":{"1min":0.71399999999999997,"5min":0.71399999999999997,"15min":0.71399999999999997},"min":{"1min":0.53100000000000003,"5min":0.53100000000000003,"15min":0.53100000000000003},"max":{"1min":1.3859999999999999,"5min":1.3859999999999999,"15min":1.3859999999999999},"last":0.98599999999999999}]},{"osd":6,"last update":"Sun Nov 26 07:18:42 2023","interfaces":[{"interface":"back","average":{"1min":0.77100000000000002,"5min":0.77100000000000002,"15min":0.77100000000000002},"min":{"1min":0.64100000000000001,"5min":0.64100000000000001,"15min":0.64100000000000001},"max":{"1min":1.3069999999999999,"5min":1.3069999999999999,"15min":1.3069999999999999},"last":1.0349999999999999},{"interface":"front","average":{"1min":0.77100000000000002,"5min":0.77100000000000002,"15min":0.77100000000000002},"min":{"1min":0.57399999999999995,"5min":0.57399999999999995,"15min":0.57399999999999995},"max":{"1min":1.6759999999999999,"5min":1.6759999999999999,"15min":1.6759999999999999},"last":0.79600000000000004}]}]},{"osd":6,"up_from":13,"seq":55834574875,"num_pgs":29,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27544,"kb_used_data":524,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344296,"statfs":{"total":96636764160,"available":96608559104,"internally_reserved":0,"allocated":536576,"data_stored":215771,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Nov 26 07:18:45 2023","interfaces":[{"interface":"back","average":{"1min":0.68400000000000005,"5min":0.68400000000000005,"15min":0.68400000000000005},"min":{"1min":0.36699999999999999,"5min":0.36699999999999999,"15min":0.36699999999999999},"max":{"1min":1.7549999999999999,"5min":1.7549999999999999,"15min":1.7549999999999999},"last":0.55400000000000005},{"interface":"front","average":{"1min":0.95799999999999996,"5min":0.95799999999999996,"15min":0.95799999999999996},"min":{"1min":0.38300000000000001,"5min":0.38300000000000001,"15min":0.38300000000000001},"max":{"1min":5.7489999999999997,"5min":5.7489999999999997,"15min":5.7489999999999997},"last":0.57899999999999996}]},{"osd":1,"last update":"Sun Nov 26 07:18:45 2023","interfaces":[{"interface":"back","average":{"1min":0.91400000000000003,"5min":0.91400000000000003,"15min":0.91400000000000003},"min":{"1min":0.42099999999999999,"5min":0.42099999999999999,"15min":0.42099999999999999},"max":{"1min":5.4240000000000004,"5min":5.4240000000000004,"15min":5.4240000000000004},"last":0.745},{"interface":"front","average":{"1min":0.89700000000000002,"5min":0.89700000000000002,"15min":0.89700000000000002},"min":{"1min":0.499,"5min":0.499,"15min":0.499},"max":{"1min":5.6020000000000003,"5min":5.6020000000000003,"15min":5.6020000000000003},"last":0.64100000000000001}]},{"osd":2,"last update":"Sun Nov 26 07:18:45 2023","interfaces":[{"interface":"back","average":{"1min":0.69699999999999995,"5min":0.69699999999999995,"15min":0.69699999999999995},"min":{"1min":0.47599999999999998,"5min":0.47599999999999998,"15min":0.47599999999999998},"max":{"1min":1.8520000000000001,"5min":1.8520000000000001,"15min":1.8520000000000001},"last":0.61399999999999999},{"interface":"front","average":{"1min":0.73399999999999999,"5min":0.73399999999999999,"15min":0.73399999999999999},"min":{"1min":0.46700000000000003,"5min":0.46700000000000003,"15min":0.46700000000000003},"max":{"1min":1.9159999999999999,"5min":1.9159999999999999,"15min":1.9159999999999999},"last":0.79400000000000004}]},{"osd":3,"last update":"Sun Nov 26 07:18:45 2023","interfaces":[{"interface":"back","average":{"1min":0.90200000000000002,"5min":0.90200000000000002,"15min":0.90200000000000002},"min":{"1min":0.502,"5min":0.502,"15min":0.502},"max":{"1min":5.4390000000000001,"5min":5.4390000000000001,"15min":5.4390000000000001},"last":0.69299999999999995},{"interface":"front","average":{"1min":0.94499999999999995,"5min":0.94499999999999995,"15min":0.94499999999999995},"min":{"1min":0.54400000000000004,"5min":0.54400000000000004,"15min":0.54400000000000004},"max":{"1min":5.5490000000000004,"5min":5.5490000000000004,"15min":5.5490000000000004},"last":0.77100000000000002}]},{"osd":4,"last update":"Sun Nov 26 07:18:45 2023","interfaces":[{"interface":"back","average":{"1min":0.64200000000000002,"5min":0.64200000000000002,"15min":0.64200000000000002},"min":{"1min":0.46899999999999997,"5min":0.46899999999999997,"15min":0.46899999999999997},"max":{"1min":0.85099999999999998,"5min":0.85099999999999998,"15min":0.85099999999999998},"last":0.82599999999999996},{"interface":"front","average":{"1min":0.67500000000000004,"5min":0.67500000000000004,"15min":0.67500000000000004},"min":{"1min":0.40699999999999997,"5min":0.40699999999999997,"15min":0.40699999999999997},"max":{"1min":0.93300000000000005,"5min":0.93300000000000005,"15min":0.93300000000000005},"last":0.85899999999999999}]},{"osd":5,"last update":"Sun Nov 26 07:18:45 2023","interfaces":[{"interface":"back","average":{"1min":0.65900000000000003,"5min":0.65900000000000003,"15min":0.65900000000000003},"min":{"1min":0.42499999999999999,"5min":0.42499999999999999,"15min":0.42499999999999999},"max":{"1min":0.79300000000000004,"5min":0.79300000000000004,"15min":0.79300000000000004},"last":0.71299999999999997},{"interface":"front","average":{"1min":0.71299999999999997,"5min":0.71299999999999997,"15min":0.71299999999999997},"min":{"1min":0.436,"5min":0.436,"15min":0.436},"max":{"1min":0.93000000000000005,"5min":0.93000000000000005,"15min":0.93000000000000005},"last":0.65800000000000003}]},{"osd":7,"last update":"Sun Nov 26 07:18:45 2023","interfaces":[{"interface":"back","average":{"1min":0.74199999999999999,"5min":0.74199999999999999,"15min":0.74199999999999999},"min":{"1min":0.46600000000000003,"5min":0.46600000000000003,"15min":0.46600000000000003},"max":{"1min":0.998,"5min":0.998,"15min":0.998},"last":0.81100000000000005},{"interface":"front","average":{"1min":0.71599999999999997,"5min":0.71599999999999997,"15min":0.71599999999999997},"min":{"1min":0.498,"5min":0.498,"15min":0.498},"max":{"1min":0.86499999999999999,"5min":0.86499999999999999,"15min":0.86499999999999999},"last":0.83799999999999997}]}]},{"osd":5,"up_from":13,"seq":55834574873,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27540,"kb_used_data":520,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344300,"statfs":{"total":96636764160,"available":96608563200,"internally_reserved":0,"allocated":532480,"data_stored":215169,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Nov 26 07:18:45 2023","interfaces":[{"interface":"back","average":{"1min":0.66600000000000004,"5min":0.66600000000000004,"15min":0.66600000000000004},"min":{"1min":0.38100000000000001,"5min":0.38100000000000001,"15min":0.38100000000000001},"max":{"1min":1.0289999999999999,"5min":1.0289999999999999,"15min":1.0289999999999999},"last":0.42099999999999999},{"interface":"front","average":{"1min":0.66000000000000003,"5min":0.66000000000000003,"15min":0.66000000000000003},"min":{"1min":0.438,"5min":0.438,"15min":0.438},"max":{"1min":1.05,"5min":1.05,"15min":1.05},"last":0.46700000000000003}]},{"osd":1,"last update":"Sun Nov 26 07:18:45 2023","interfaces":[{"interface":"back","average":{"1min":0.66800000000000004,"5min":0.66800000000000004,"15min":0.66800000000000004},"min":{"1min":0.41599999999999998,"5min":0.41599999999999998,"15min":0.41599999999999998},"max":{"1min":0.86699999999999999,"5min":0.86699999999999999,"15min":0.86699999999999999},"last":0.44700000000000001},{"interface":"front","average":{"1min":0.68600000000000005,"5min":0.68600000000000005,"15min":0.68600000000000005},"min":{"1min":0.49199999999999999,"5min":0.49199999999999999,"15min":0.49199999999999999},"max":{"1min":0.98899999999999999,"5min":0.98899999999999999,"15min":0.98899999999999999},"last":0.504}]},{"osd":2,"last update":"Sun Nov 26 07:18:45 2023","interfaces":[{"interface":"back","average":{"1min":0.67500000000000004,"5min":0.67500000000000004,"15min":0.67500000000000004},"min":{"1min":0.45900000000000002,"5min":0.45900000000000002,"15min":0.45900000000000002},"max":{"1min":0.89800000000000002,"5min":0.89800000000000002,"15min":0.89800000000000002},"last":0.61499999999999999},{"interface":"front","average":{"1min":0.67000000000000004,"5min":0.67000000000000004,"15min":0.67000000000000004},"min":{"1min":0.51200000000000001,"5min":0.51200000000000001,"15min":0.51200000000000001},"max":{"1min":0.89700000000000002,"5min":0.89700000000000002,"15min":0.89700000000000002},"last":0.53800000000000003}]},{"osd":3,"last update":"Sun Nov 26 07:18:45 2023","interfaces":[{"interface":"back","average":{"1min":0.71999999999999997,"5min":0.71999999999999997,"15min":0.71999999999999997},"min":{"1min":0.43099999999999999,"5min":0.43099999999999999,"15min":0.43099999999999999},"max":{"1min":1.0029999999999999,"5min":1.0029999999999999,"15min":1.0029999999999999},"last":0.70299999999999996},{"interface":"front","average":{"1min":0.67500000000000004,"5min":0.67500000000000004,"15min":0.67500000000000004},"min":{"1min":0.44600000000000001,"5min":0.44600000000000001,"15min":0.44600000000000001},"max":{"1min":1.0209999999999999,"5min":1.0209999999999999,"15min":1.0209999999999999},"last":0.56299999999999994}]},{"osd":4,"last update":"Sun Nov 26 07:18:45 2023","interfaces":[{"interface":"back","average":{"1min":0.63500000000000001,"5min":0.63500000000000001,"15min":0.63500000000000001},"min":{"1min":0.39200000000000002,"5min":0.39200000000000002,"15min":0.39200000000000002},"max":{"1min":0.95099999999999996,"5min":0.95099999999999996,"15min":0.95099999999999996},"last":0.63200000000000001},{"interface":"front","average":{"1min":0.70899999999999996,"5min":0.70899999999999996,"15min":0.70899999999999996},"min":{"1min":0.41299999999999998,"5min":0.41299999999999998,"15min":0.41299999999999998},"max":{"1min":1.254,"5min":1.254,"15min":1.254},"last":0.66400000000000003}]},{"osd":6,"last update":"Sun Nov 26 07:18:45 2023","interfaces":[{"interface":"back","average":{"1min":0.67200000000000004,"5min":0.67200000000000004,"15min":0.67200000000000004},"min":{"1min":0.45600000000000002,"5min":0.45600000000000002,"15min":0.45600000000000002},"max":{"1min":0.94499999999999995,"5min":0.94499999999999995,"15min":0.94499999999999995},"last":0.97199999999999998},{"interface":"front","average":{"1min":0.72699999999999998,"5min":0.72699999999999998,"15min":0.72699999999999998},"min":{"1min":0.44,"5min":0.44,"15min":0.44},"max":{"1min":1.0649999999999999,"5min":1.0649999999999999,"15min":1.0649999999999999},"last":0.79800000000000004}]},{"osd":7,"last update":"Sun Nov 26 07:18:45 2023","interfaces":[{"interface":"back","average":{"1min":0.80400000000000005,"5min":0.80400000000000005,"15min":0.80400000000000005},"min":{"1min":0.48199999999999998,"5min":0.48199999999999998,"15min":0.48199999999999998},"max":{"1min":1.4079999999999999,"5min":1.4079999999999999,"15min":1.4079999999999999},"last":0.878},{"interface":"front","average":{"1min":0.77500000000000002,"5min":0.77500000000000002,"15min":0.77500000000000002},"min":{"1min":0.51600000000000001,"5min":0.51600000000000001,"15min":0.51600000000000001},"max":{"1min":1.0169999999999999,"5min":1.0169999999999999,"15min":1.0169999999999999},"last":0.73099999999999998}]}]},{"osd":4,"up_from":13,"seq":55834574875,"num_pgs":19,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27548,"kb_used_data":528,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344292,"statfs":{"total":96636764160,"available":96608555008,"internally_reserved":0,"allocated":540672,"data_stored":216711,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Nov 26 07:18:46 2023","interfaces":[{"interface":"back","average":{"1min":0.79400000000000004,"5min":0.79400000000000004,"15min":0.79400000000000004},"min":{"1min":0.45600000000000002,"5min":0.45600000000000002,"15min":0.45600000000000002},"max":{"1min":4.1689999999999996,"5min":4.1689999999999996,"15min":4.1689999999999996},"last":0.54700000000000004},{"interface":"front","average":{"1min":1.002,"5min":1.002,"15min":1.002},"min":{"1min":0.47499999999999998,"5min":0.47499999999999998,"15min":0.47499999999999998},"max":{"1min":7.7889999999999997,"5min":7.7889999999999997,"15min":7.7889999999999997},"last":0.46200000000000002}]},{"osd":1,"last update":"Sun Nov 26 07:18:46 2023","interfaces":[{"interface":"back","average":{"1min":0.64700000000000002,"5min":0.64700000000000002,"15min":0.64700000000000002},"min":{"1min":0.52300000000000002,"5min":0.52300000000000002,"15min":0.52300000000000002},"max":{"1min":0.78400000000000003,"5min":0.78400000000000003,"15min":0.78400000000000003},"last":0.52100000000000002},{"interface":"front","average":{"1min":0.81999999999999995,"5min":0.81999999999999995,"15min":0.81999999999999995},"min":{"1min":0.44500000000000001,"5min":0.44500000000000001,"15min":0.44500000000000001},"max":{"1min":4.3140000000000001,"5min":4.3140000000000001,"15min":4.3140000000000001},"last":0.435}]},{"osd":2,"last update":"Sun Nov 26 07:18:46 2023","interfaces":[{"interface":"back","average":{"1min":0.80700000000000005,"5min":0.80700000000000005,"15min":0.80700000000000005},"min":{"1min":0.57799999999999996,"5min":0.57799999999999996,"15min":0.57799999999999996},"max":{"1min":2.7850000000000001,"5min":2.7850000000000001,"15min":2.7850000000000001},"last":0.73299999999999998},{"interface":"front","average":{"1min":1.0660000000000001,"5min":1.0660000000000001,"15min":1.0660000000000001},"min":{"1min":0.53400000000000003,"5min":0.53400000000000003,"15min":0.53400000000000003},"max":{"1min":7.6470000000000002,"5min":7.6470000000000002,"15min":7.6470000000000002},"last":0.67400000000000004}]},{"osd":3,"last update":"Sun Nov 26 07:18:46 2023","interfaces":[{"interface":"back","average":{"1min":0.72999999999999998,"5min":0.72999999999999998,"15min":0.72999999999999998},"min":{"1min":0.48599999999999999,"5min":0.48599999999999999,"15min":0.48599999999999999},"max":{"1min":1.591,"5min":1.591,"15min":1.591},"last":0.70499999999999996},{"interface":"front","average":{"1min":0.879,"5min":0.879,"15min":0.879},"min":{"1min":0.56899999999999995,"5min":0.56899999999999995,"15min":0.56899999999999995},"max":{"1min":3.452,"5min":3.452,"15min":3.452},"last":0.64400000000000002}]},{"osd":5,"last update":"Sun Nov 26 07:18:46 2023","interfaces":[{"interface":"back","average":{"1min":0.61099999999999999,"5min":0.61099999999999999,"15min":0.61099999999999999},"min":{"1min":0.43099999999999999,"5min":0.43099999999999999,"15min":0.43099999999999999},"max":{"1min":0.96199999999999997,"5min":0.96199999999999997,"15min":0.96199999999999997},"last":0.56899999999999995},{"interface":"front","average":{"1min":0.64100000000000001,"5min":0.64100000000000001,"15min":0.64100000000000001},"min":{"1min":0.41499999999999998,"5min":0.41499999999999998,"15min":0.41499999999999998},"max":{"1min":1.538,"5min":1.538,"15min":1.538},"last":0.749}]},{"osd":6,"last update":"Sun Nov 26 07:18:46 2023","interfaces":[{"interface":"back","average":{"1min":0.79400000000000004,"5min":0.79400000000000004,"15min":0.79400000000000004},"min":{"1min":0.39400000000000002,"5min":0.39400000000000002,"15min":0.39400000000000002},"max":{"1min":2.8330000000000002,"5min":2.8330000000000002,"15min":2.8330000000000002},"last":0.77300000000000002},{"interface":"front","average":{"1min":1.0089999999999999,"5min":1.0089999999999999,"15min":1.0089999999999999},"min":{"1min":0.42899999999999999,"5min":0.42899999999999999,"15min":0.42899999999999999},"max":{"1min":7.6050000000000004,"5min":7.6050000000000004,"15min":7.6050000000000004},"last":0.623}]},{"osd":7,"last update":"Sun Nov 26 07:18:46 2023","interfaces":[{"interface":"back","average":{"1min":0.86099999999999999,"5min":0.86099999999999999,"15min":0.86099999999999999},"min":{"1min":0.44700000000000001,"5min":0.44700000000000001,"15min":0.44700000000000001},"max":{"1min":4.3689999999999998,"5min":4.3689999999999998,"15min":4.3689999999999998},"last":0.59799999999999998},{"interface":"front","average":{"1min":0.89200000000000002,"5min":0.89200000000000002,"15min":0.89200000000000002},"min":{"1min":0.505,"5min":0.505,"15min":0.505},"max":{"1min":3.403,"5min":3.403,"15min":3.403},"last":0.79900000000000004}]}]},{"osd":3,"up_from":13,"seq":55834574876,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27548,"kb_used_data":528,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344292,"statfs":{"total":96636764160,"available":96608555008,"internally_reserved":0,"allocated":540672,"data_stored":215737,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1592,"internal_metadata":27654600},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Nov 26 07:18:43 2023","interfaces":[{"interface":"back","average":{"1min":0.379,"5min":0.379,"15min":0.379},"min":{"1min":0.26100000000000001,"5min":0.26100000000000001,"15min":0.26100000000000001},"max":{"1min":0.58599999999999997,"5min":0.58599999999999997,"15min":0.58599999999999997},"last":0.44800000000000001},{"interface":"front","average":{"1min":0.433,"5min":0.433,"15min":0.433},"min":{"1min":0.30199999999999999,"5min":0.30199999999999999,"15min":0.30199999999999999},"max":{"1min":0.63,"5min":0.63,"15min":0.63},"last":0.51700000000000002}]},{"osd":1,"last update":"Sun Nov 26 07:18:43 2023","interfaces":[{"interface":"back","average":{"1min":0.42099999999999999,"5min":0.42099999999999999,"15min":0.42099999999999999},"min":{"1min":0.312,"5min":0.312,"15min":0.312},"max":{"1min":0.752,"5min":0.752,"15min":0.752},"last":0.54500000000000004},{"interface":"front","average":{"1min":0.48399999999999999,"5min":0.48399999999999999,"15min":0.48399999999999999},"min":{"1min":0.38,"5min":0.38,"15min":0.38},"max":{"1min":0.628,"5min":0.628,"15min":0.628},"last":0.56599999999999995}]},{"osd":2,"last update":"Sun Nov 26 07:18:43 2023","interfaces":[{"interface":"back","average":{"1min":0.53300000000000003,"5min":0.53300000000000003,"15min":0.53300000000000003},"min":{"1min":0.40300000000000002,"5min":0.40300000000000002,"15min":0.40300000000000002},"max":{"1min":0.82199999999999995,"5min":0.82199999999999995,"15min":0.82199999999999995},"last":0.70499999999999996},{"interface":"front","average":{"1min":0.505,"5min":0.505,"15min":0.505},"min":{"1min":0.35999999999999999,"5min":0.35999999999999999,"15min":0.35999999999999999},"max":{"1min":0.72199999999999998,"5min":0.72199999999999998,"15min":0.72199999999999998},"last":0.67800000000000005}]},{"osd":4,"last update":"Sun Nov 26 07:18:43 2023","interfaces":[{"interface":"back","average":{"1min":0.70599999999999996,"5min":0.70599999999999996,"15min":0.70599999999999996},"min":{"1min":0.48199999999999998,"5min":0.48199999999999998,"15min":0.48199999999999998},"max":{"1min":0.96199999999999997,"5min":0.96199999999999997,"15min":0.96199999999999997},"last":1.004},{"interface":"front","average":{"1min":0.72799999999999998,"5min":0.72799999999999998,"15min":0.72799999999999998},"min":{"1min":0.441,"5min":0.441,"15min":0.441},"max":{"1min":1.171,"5min":1.171,"15min":1.171},"last":0.84899999999999998}]},{"osd":5,"last update":"Sun Nov 26 07:18:43 2023","interfaces":[{"interface":"back","average":{"1min":0.73899999999999999,"5min":0.73899999999999999,"15min":0.73899999999999999},"min":{"1min":0.53700000000000003,"5min":0.53700000000000003,"15min":0.53700000000000003},"max":{"1min":0.94099999999999995,"5min":0.94099999999999995,"15min":0.94099999999999995},"last":0.88600000000000001},{"interface":"front","average":{"1min":0.73799999999999999,"5min":0.73799999999999999,"15min":0.73799999999999999},"min":{"1min":0.504,"5min":0.504,"15min":0.504},"max":{"1min":1.0820000000000001,"5min":1.0820000000000001,"15min":1.0820000000000001},"last":0.86799999999999999}]},{"osd":6,"last update":"Sun Nov 26 07:18:43 2023","interfaces":[{"interface":"back","average":{"1min":0.81399999999999995,"5min":0.81399999999999995,"15min":0.81399999999999995},"min":{"1min":0.55200000000000005,"5min":0.55200000000000005,"15min":0.55200000000000005},"max":{"1min":1.2729999999999999,"5min":1.2729999999999999,"15min":1.2729999999999999},"last":0.80700000000000005},{"interface":"front","average":{"1min":0.79300000000000004,"5min":0.79300000000000004,"15min":0.79300000000000004},"min":{"1min":0.47299999999999998,"5min":0.47299999999999998,"15min":0.47299999999999998},"max":{"1min":1.204,"5min":1.204,"15min":1.204},"last":0.93600000000000005}]},{"osd":7,"last update":"Sun Nov 26 07:18:43 2023","interfaces":[{"interface":"back","average":{"1min":0.80100000000000005,"5min":0.80100000000000005,"15min":0.80100000000000005},"min":{"1min":0.63300000000000001,"5min":0.63300000000000001,"15min":0.63300000000000001},"max":{"1min":1.3600000000000001,"5min":1.3600000000000001,"15min":1.3600000000000001},"last":1.125},{"interface":"front","average":{"1min":0.82599999999999996,"5min":0.82599999999999996,"15min":0.82599999999999996},"min":{"1min":0.50600000000000001,"5min":0.50600000000000001,"15min":0.50600000000000001},"max":{"1min":1.617,"5min":1.617,"15min":1.617},"last":1.0940000000000001}]}]},{"osd":2,"up_from":13,"seq":55834574876,"num_pgs":11,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27560,"kb_used_data":532,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344280,"statfs":{"total":96636764160,"available":96608542720,"internally_reserved":0,"allocated":544768,"data_stored":216733,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Nov 26 07:18:43 2023","interfaces":[{"interface":"back","average":{"1min":0.71399999999999997,"5min":0.71399999999999997,"15min":0.71399999999999997},"min":{"1min":0.28699999999999998,"5min":0.28699999999999998,"15min":0.28699999999999998},"max":{"1min":4.7030000000000003,"5min":4.7030000000000003,"15min":4.7030000000000003},"last":0.41099999999999998},{"interface":"front","average":{"1min":0.56200000000000006,"5min":0.56200000000000006,"15min":0.56200000000000006},"min":{"1min":0.32300000000000001,"5min":0.32300000000000001,"15min":0.32300000000000001},"max":{"1min":2.4340000000000002,"5min":2.4340000000000002,"15min":2.4340000000000002},"last":0.438}]},{"osd":1,"last update":"Sun Nov 26 07:18:43 2023","interfaces":[{"interface":"back","average":{"1min":0.86399999999999999,"5min":0.86399999999999999,"15min":0.86399999999999999},"min":{"1min":0.34599999999999997,"5min":0.34599999999999997,"15min":0.34599999999999997},"max":{"1min":4.2690000000000001,"5min":4.2690000000000001,"15min":4.2690000000000001},"last":0.46300000000000002},{"interface":"front","average":{"1min":0.74199999999999999,"5min":0.74199999999999999,"15min":0.74199999999999999},"min":{"1min":0.31900000000000001,"5min":0.31900000000000001,"15min":0.31900000000000001},"max":{"1min":3.9940000000000002,"5min":3.9940000000000002,"15min":3.9940000000000002},"last":0.38800000000000001}]},{"osd":3,"last update":"Sun Nov 26 07:18:43 2023","interfaces":[{"interface":"back","average":{"1min":0.64300000000000002,"5min":0.64300000000000002,"15min":0.64300000000000002},"min":{"1min":0.34399999999999997,"5min":0.34399999999999997,"15min":0.34399999999999997},"max":{"1min":4.0300000000000002,"5min":4.0300000000000002,"15min":4.0300000000000002},"last":0.36899999999999999},{"interface":"front","average":{"1min":0.91400000000000003,"5min":0.91400000000000003,"15min":0.91400000000000003},"min":{"1min":0.41699999999999998,"5min":0.41699999999999998,"15min":0.41699999999999998},"max":{"1min":4.5810000000000004,"5min":4.5810000000000004,"15min":4.5810000000000004},"last":0.50900000000000001}]},{"osd":4,"last update":"Sun Nov 26 07:18:43 2023","interfaces":[{"interface":"back","average":{"1min":0.98499999999999999,"5min":0.98499999999999999,"15min":0.98499999999999999},"min":{"1min":0.55100000000000005,"5min":0.55100000000000005,"15min":0.55100000000000005},"max":{"1min":4.4269999999999996,"5min":4.4269999999999996,"15min":4.4269999999999996},"last":0.80900000000000005},{"interface":"front","average":{"1min":1.0149999999999999,"5min":1.0149999999999999,"15min":1.0149999999999999},"min":{"1min":0.60299999999999998,"5min":0.60299999999999998,"15min":0.60299999999999998},"max":{"1min":4.085,"5min":4.085,"15min":4.085},"last":0.68100000000000005}]},{"osd":5,"last update":"Sun Nov 26 07:18:43 2023","interfaces":[{"interface":"back","average":{"1min":0.874,"5min":0.874,"15min":0.874},"min":{"1min":0.52900000000000003,"5min":0.52900000000000003,"15min":0.52900000000000003},"max":{"1min":3.9590000000000001,"5min":3.9590000000000001,"15min":3.9590000000000001},"last":0.65000000000000002},{"interface":"front","average":{"1min":0.98799999999999999,"5min":0.98799999999999999,"15min":0.98799999999999999},"min":{"1min":0.58799999999999997,"5min":0.58799999999999997,"15min":0.58799999999999997},"max":{"1min":4.3849999999999998,"5min":4.3849999999999998,"15min":4.3849999999999998},"last":0.752}]},{"osd":6,"last update":"Sun Nov 26 07:18:43 2023","interfaces":[{"interface":"back","average":{"1min":0.996,"5min":0.996,"15min":0.996},"min":{"1min":0.56599999999999995,"5min":0.56599999999999995,"15min":0.56599999999999995},"max":{"1min":4.0449999999999999,"5min":4.0449999999999999,"15min":4.0449999999999999},"last":0.77700000000000002},{"interface":"front","average":{"1min":0.91800000000000004,"5min":0.91800000000000004,"15min":0.91800000000000004},"min":{"1min":0.57099999999999995,"5min":0.57099999999999995,"15min":0.57099999999999995},"max":{"1min":3.992,"5min":3.992,"15min":3.992},"last":0.60699999999999998}]},{"osd":7,"last update":"Sun Nov 26 07:18:43 2023","interfaces":[{"interface":"back","average":{"1min":1.0349999999999999,"5min":1.0349999999999999,"15min":1.0349999999999999},"min":{"1min":0.63100000000000001,"5min":0.63100000000000001,"15min":0.63100000000000001},"max":{"1min":4.1219999999999999,"5min":4.1219999999999999,"15min":4.1219999999999999},"last":0.63},{"interface":"front","average":{"1min":0.93799999999999994,"5min":0.93799999999999994,"15min":0.93799999999999994},"min":{"1min":0.57699999999999996,"5min":0.57699999999999996,"15min":0.57699999999999996},"max":{"1min":3.9159999999999999,"5min":3.9159999999999999,"15min":3.9159999999999999},"last":0.73599999999999999}]}]},{"osd":1,"up_from":13,"seq":55834574875,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27540,"kb_used_data":520,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344300,"statfs":{"total":96636764160,"available":96608563200,"internally_reserved":0,"allocated":532480,"data_stored":215169,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":12,"apply_latency_ms":12,"commit_latency_ns":12000000,"apply_latency_ns":12000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Nov 26 07:18:41 2023","interfaces":[{"interface":"back","average":{"1min":0.41699999999999998,"5min":0.41699999999999998,"15min":0.41699999999999998},"min":{"1min":0.26300000000000001,"5min":0.26300000000000001,"15min":0.26300000000000001},"max":{"1min":0.61199999999999999,"5min":0.61199999999999999,"15min":0.61199999999999999},"last":0.40500000000000003},{"interface":"front","average":{"1min":0.48599999999999999,"5min":0.48599999999999999,"15min":0.48599999999999999},"min":{"1min":0.30499999999999999,"5min":0.30499999999999999,"15min":0.30499999999999999},"max":{"1min":0.78700000000000003,"5min":0.78700000000000003,"15min":0.78700000000000003},"last":0.26700000000000002}]},{"osd":2,"last update":"Sun Nov 26 07:18:41 2023","interfaces":[{"interface":"back","average":{"1min":0.45500000000000002,"5min":0.45500000000000002,"15min":0.45500000000000002},"min":{"1min":0.33400000000000002,"5min":0.33400000000000002,"15min":0.33400000000000002},"max":{"1min":0.67600000000000005,"5min":0.67600000000000005,"15min":0.67600000000000005},"last":0.42299999999999999},{"interface":"front","average":{"1min":0.50800000000000001,"5min":0.50800000000000001,"15min":0.50800000000000001},"min":{"1min":0.32300000000000001,"5min":0.32300000000000001,"15min":0.32300000000000001},"max":{"1min":0.66500000000000004,"5min":0.66500000000000004,"15min":0.66500000000000004},"last":0.36299999999999999}]},{"osd":3,"last update":"Sun Nov 26 07:18:41 2023","interfaces":[{"interface":"back","average":{"1min":0.53000000000000003,"5min":0.53000000000000003,"15min":0.53000000000000003},"min":{"1min":0.38400000000000001,"5min":0.38400000000000001,"15min":0.38400000000000001},"max":{"1min":0.73699999999999999,"5min":0.73699999999999999,"15min":0.73699999999999999},"last":0.44500000000000001},{"interface":"front","average":{"1min":0.497,"5min":0.497,"15min":0.497},"min":{"1min":0.36199999999999999,"5min":0.36199999999999999,"15min":0.36199999999999999},"max":{"1min":0.80700000000000005,"5min":0.80700000000000005,"15min":0.80700000000000005},"last":0.31}]},{"osd":4,"last update":"Sun Nov 26 07:18:41 2023","interfaces":[{"interface":"back","average":{"1min":0.69399999999999995,"5min":0.69399999999999995,"15min":0.69399999999999995},"min":{"1min":0.48599999999999999,"5min":0.48599999999999999,"15min":0.48599999999999999},"max":{"1min":1.0229999999999999,"5min":1.0229999999999999,"15min":1.0229999999999999},"last":0.60499999999999998},{"interface":"front","average":{"1min":0.68000000000000005,"5min":0.68000000000000005,"15min":0.68000000000000005},"min":{"1min":0.49199999999999999,"5min":0.49199999999999999,"15min":0.49199999999999999},"max":{"1min":1.0629999999999999,"5min":1.0629999999999999,"15min":1.0629999999999999},"last":0.61599999999999999}]},{"osd":5,"last update":"Sun Nov 26 07:18:41 2023","interfaces":[{"interface":"back","average":{"1min":0.71899999999999997,"5min":0.71899999999999997,"15min":0.71899999999999997},"min":{"1min":0.49299999999999999,"5min":0.49299999999999999,"15min":0.49299999999999999},"max":{"1min":1.0880000000000001,"5min":1.0880000000000001,"15min":1.0880000000000001},"last":0.58799999999999997},{"interface":"front","average":{"1min":0.71699999999999997,"5min":0.71699999999999997,"15min":0.71699999999999997},"min":{"1min":0.53500000000000003,"5min":0.53500000000000003,"15min":0.53500000000000003},"max":{"1min":1.014,"5min":1.014,"15min":1.014},"last":0.63}]},{"osd":6,"last update":"Sun Nov 26 07:18:41 2023","interfaces":[{"interface":"back","average":{"1min":0.70399999999999996,"5min":0.70399999999999996,"15min":0.70399999999999996},"min":{"1min":0.50600000000000001,"5min":0.50600000000000001,"15min":0.50600000000000001},"max":{"1min":0.97599999999999998,"5min":0.97599999999999998,"15min":0.97599999999999998},"last":0.64500000000000002},{"interface":"front","average":{"1min":0.70099999999999996,"5min":0.70099999999999996,"15min":0.70099999999999996},"min":{"1min":0.55000000000000004,"5min":0.55000000000000004,"15min":0.55000000000000004},"max":{"1min":1.1339999999999999,"5min":1.1339999999999999,"15min":1.1339999999999999},"last":0.71499999999999997}]},{"osd":7,"last update":"Sun Nov 26 07:18:41 2023","interfaces":[{"interface":"back","average":{"1min":0.71499999999999997,"5min":0.71499999999999997,"15min":0.71499999999999997},"min":{"1min":0.55100000000000005,"5min":0.55100000000000005,"15min":0.55100000000000005},"max":{"1min":0.92900000000000005,"5min":0.92900000000000005,"15min":0.92900000000000005},"last":0.69999999999999996},{"interface":"front","average":{"1min":0.753,"5min":0.753,"15min":0.753},"min":{"1min":0.56100000000000005,"5min":0.56100000000000005,"15min":0.56100000000000005},"max":{"1min":1.024,"5min":1.024,"15min":1.024},"last":0.55000000000000004}]}]},{"osd":0,"up_from":13,"seq":55834574875,"num_pgs":31,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":28120,"kb_used_data":1100,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343720,"statfs":{"total":96636764160,"available":96607969280,"internally_reserved":0,"allocated":1126400,"data_stored":805501,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":7,"apply_latency_ms":7,"commit_latency_ns":7000000,"apply_latency_ns":7000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Sun Nov 26 07:18:41 2023","interfaces":[{"interface":"back","average":{"1min":0.43099999999999999,"5min":0.43099999999999999,"15min":0.43099999999999999},"min":{"1min":0.28000000000000003,"5min":0.28000000000000003,"15min":0.28000000000000003},"max":{"1min":0.63100000000000001,"5min":0.63100000000000001,"15min":0.63100000000000001},"last":0.35299999999999998},{"interface":"front","average":{"1min":0.48299999999999998,"5min":0.48299999999999998,"15min":0.48299999999999998},"min":{"1min":0.29899999999999999,"5min":0.29899999999999999,"15min":0.29899999999999999},"max":{"1min":1.133,"5min":1.133,"15min":1.133},"last":0.48299999999999998}]},{"osd":2,"last update":"Sun Nov 26 07:18:41 2023","interfaces":[{"interface":"back","average":{"1min":0.53700000000000003,"5min":0.53700000000000003,"15min":0.53700000000000003},"min":{"1min":0.40899999999999997,"5min":0.40899999999999997,"15min":0.40899999999999997},"max":{"1min":0.67400000000000004,"5min":0.67400000000000004,"15min":0.67400000000000004},"last":0.40300000000000002},{"interface":"front","average":{"1min":0.56399999999999995,"5min":0.56399999999999995,"15min":0.56399999999999995},"min":{"1min":0.36399999999999999,"5min":0.36399999999999999,"15min":0.36399999999999999},"max":{"1min":0.72499999999999998,"5min":0.72499999999999998,"15min":0.72499999999999998},"last":0.45900000000000002}]},{"osd":3,"last update":"Sun Nov 26 07:18:41 2023","interfaces":[{"interface":"back","average":{"1min":0.53100000000000003,"5min":0.53100000000000003,"15min":0.53100000000000003},"min":{"1min":0.35799999999999998,"5min":0.35799999999999998,"15min":0.35799999999999998},"max":{"1min":0.97399999999999998,"5min":0.97399999999999998,"15min":0.97399999999999998},"last":0.437},{"interface":"front","average":{"1min":0.56200000000000006,"5min":0.56200000000000006,"15min":0.56200000000000006},"min":{"1min":0.40600000000000003,"5min":0.40600000000000003,"15min":0.40600000000000003},"max":{"1min":1.2509999999999999,"5min":1.2509999999999999,"15min":1.2509999999999999},"last":0.41699999999999998}]},{"osd":4,"last update":"Sun Nov 26 07:18:41 2023","interfaces":[{"interface":"back","average":{"1min":0.75900000000000001,"5min":0.75900000000000001,"15min":0.75900000000000001},"min":{"1min":0.60999999999999999,"5min":0.60999999999999999,"15min":0.60999999999999999},"max":{"1min":1.2010000000000001,"5min":1.2010000000000001,"15min":1.2010000000000001},"last":0.67700000000000005},{"interface":"front","average":{"1min":0.78900000000000003,"5min":0.78900000000000003,"15min":0.78900000000000003},"min":{"1min":0.59999999999999998,"5min":0.59999999999999998,"15min":0.59999999999999998},"max":{"1min":0.99399999999999999,"5min":0.99399999999999999,"15min":0.99399999999999999},"last":0.63}]},{"osd":5,"last update":"Sun Nov 26 07:18:41 2023","interfaces":[{"interface":"back","average":{"1min":0.76000000000000001,"5min":0.76000000000000001,"15min":0.76000000000000001},"min":{"1min":0.53000000000000003,"5min":0.53000000000000003,"15min":0.53000000000000003},"max":{"1min":1.1579999999999999,"5min":1.1579999999999999,"15min":1.1579999999999999},"last":0.70699999999999996},{"interface":"front","average":{"1min":0.79700000000000004,"5min":0.79700000000000004,"15min":0.79700000000000004},"min":{"1min":0.59699999999999998,"5min":0.59699999999999998,"15min":0.59699999999999998},"max":{"1min":1.0409999999999999,"5min":1.0409999999999999,"15min":1.0409999999999999},"last":0.67100000000000004}]},{"osd":6,"last update":"Sun Nov 26 07:18:41 2023","interfaces":[{"interface":"back","average":{"1min":0.76400000000000001,"5min":0.76400000000000001,"15min":0.76400000000000001},"min":{"1min":0.56599999999999995,"5min":0.56599999999999995,"15min":0.56599999999999995},"max":{"1min":1.3029999999999999,"5min":1.3029999999999999,"15min":1.3029999999999999},"last":0.66200000000000003},{"interface":"front","average":{"1min":0.76900000000000002,"5min":0.76900000000000002,"15min":0.76900000000000002},"min":{"1min":0.58599999999999997,"5min":0.58599999999999997,"15min":0.58599999999999997},"max":{"1min":1.2210000000000001,"5min":1.2210000000000001,"15min":1.2210000000000001},"last":0.59699999999999998}]},{"osd":7,"last update":"Sun Nov 26 07:18:41 2023","interfaces":[{"interface":"back","average":{"1min":0.81100000000000005,"5min":0.81100000000000005,"15min":0.81100000000000005},"min":{"1min":0.59999999999999998,"5min":0.59999999999999998,"15min":0.59999999999999998},"max":{"1min":1.177,"5min":1.177,"15min":1.177},"last":0.69199999999999995},{"interface":"front","average":{"1min":0.79900000000000004,"5min":0.79900000000000004,"15min":0.79900000000000004},"min":{"1min":0.60899999999999999,"5min":0.60899999999999999,"15min":0.60899999999999999},"max":{"1min":1.2390000000000001,"5min":1.2390000000000001,"15min":1.2390000000000001},"last":0.55600000000000005}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":34,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":70,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1634,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":638,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":70,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":672,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-11-26T07:19:41.362 INFO:tasks.cephfs.filesystem.ceph_manager:clean! 2023-11-26T07:19:41.362 DEBUG:teuthology.orchestra.run.smithi143:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth caps client.0 mon 'allow r' osd 'allow rw tag cephfs data=cephfs' mds allow 2023-11-26T07:19:41.544 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:41.539+0000 7f7a3cafe640 1 Processor -- start 2023-11-26T07:19:41.544 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:41.539+0000 7f7a3cafe640 1 -- start start 2023-11-26T07:19:41.545 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:41.543+0000 7f7a3cafe640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f7a38143c00 0x7f7a380b1ef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:19:41.545 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:41.543+0000 7f7a3cafe640 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7f7a38071550 0x7f7a380719e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:19:41.545 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:41.543+0000 7f7a3cafe640 1 -- --> v1:172.21.15.143:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7a380b25f0 con 0x7f7a38070390 2023-11-26T07:19:41.546 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:41.543+0000 7f7a3cafe640 1 -- --> v2:172.21.15.188:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7a38071f20 con 0x7f7a38071550 2023-11-26T07:19:41.546 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:41.543+0000 7f7a3cafe640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7a380b28d0 con 0x7f7a38143c00 2023-11-26T07:19:41.546 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:41.543+0000 7f7a35d9b640 1 --1- >> v1:172.21.15.143:6789/0 conn(0x7f7a38070390 0x7f7a380cc7a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.143:6789/0 says I am v1:172.21.15.143:41768/0 (socket says 172.21.15.143:41768) 2023-11-26T07:19:41.546 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:41.543+0000 7f7a35d9b640 1 -- 172.21.15.143:0/2031998110 learned_addr learned my addr 172.21.15.143:0/2031998110 (peer_addr_for_me v1:172.21.15.143:0/0) 2023-11-26T07:19:41.546 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:41.543+0000 7f7a36d9d640 1 --2- 172.21.15.143:0/2031998110 >> v2:172.21.15.188:3300/0 conn(0x7f7a38071550 0x7f7a380719e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:19:41.546 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:41.543+0000 7f7a3659c640 1 --2- 172.21.15.143:0/2031998110 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f7a38143c00 0x7f7a380b1ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:19:41.547 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:41.543+0000 7f7a3559a640 1 -- 172.21.15.143:0/2031998110 <== mon.0 v1:172.21.15.143:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3117325010 0 0) 0x7f7a380b25f0 con 0x7f7a38070390 2023-11-26T07:19:41.547 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:41.543+0000 7f7a3559a640 1 -- 172.21.15.143:0/2031998110 --> v1:172.21.15.143:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f7a20003530 con 0x7f7a38070390 2023-11-26T07:19:41.547 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:41.543+0000 7f7a3659c640 1 -- 172.21.15.143:0/2031998110 >> v2:172.21.15.188:3300/0 conn(0x7f7a38071550 msgr2=0x7f7a380719e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-26T07:19:41.547 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:41.543+0000 7f7a3659c640 1 --2- 172.21.15.143:0/2031998110 >> v2:172.21.15.188:3300/0 conn(0x7f7a38071550 0x7f7a380719e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:19:41.547 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:41.543+0000 7f7a3659c640 1 -- 172.21.15.143:0/2031998110 >> v1:172.21.15.143:6789/0 conn(0x7f7a38070390 legacy=0x7f7a380cc7a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:19:41.547 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:41.543+0000 7f7a3659c640 1 -- 172.21.15.143:0/2031998110 --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7a380b2bb0 con 0x7f7a38143c00 2023-11-26T07:19:41.549 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:41.547+0000 7f7a3659c640 1 --2- 172.21.15.143:0/2031998110 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f7a38143c00 0x7f7a380b1ef0 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7f7a280097c0 tx=0x7f7a2801c3f0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=778e565c65580f1e server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:19:41.549 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:41.547+0000 7f7a3559a640 1 -- 172.21.15.143:0/2031998110 <== mon.2 v2:172.21.15.188:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7a280ab600 con 0x7f7a38143c00 2023-11-26T07:19:41.549 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:41.547+0000 7f7a3559a640 1 -- 172.21.15.143:0/2031998110 <== mon.2 v2:172.21.15.188:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7a280a7050 con 0x7f7a38143c00 2023-11-26T07:19:41.549 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:41.547+0000 7f7a3559a640 1 -- 172.21.15.143:0/2031998110 <== mon.2 v2:172.21.15.188:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7a280ab980 con 0x7f7a38143c00 2023-11-26T07:19:41.550 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:41.547+0000 7f7a3cafe640 1 -- 172.21.15.143:0/2031998110 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f7a38143c00 msgr2=0x7f7a380b1ef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:19:41.550 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:41.547+0000 7f7a3cafe640 1 --2- 172.21.15.143:0/2031998110 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f7a38143c00 0x7f7a380b1ef0 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7f7a280097c0 tx=0x7f7a2801c3f0 comp rx=0 tx=0).stop 2023-11-26T07:19:41.550 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:41.547+0000 7f7a3cafe640 1 -- 172.21.15.143:0/2031998110 shutdown_connections 2023-11-26T07:19:41.550 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:41.547+0000 7f7a3cafe640 1 --2- 172.21.15.143:0/2031998110 >> v2:172.21.15.188:3300/0 conn(0x7f7a38071550 0x7f7a380719e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:19:41.551 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:41.547+0000 7f7a3cafe640 1 --2- 172.21.15.143:0/2031998110 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f7a38143c00 0x7f7a380b1ef0 unknown :-1 s=CLOSED pgs=86 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:19:41.551 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:41.547+0000 7f7a3cafe640 1 -- 172.21.15.143:0/2031998110 >> 172.21.15.143:0/2031998110 conn(0x7f7a380b0bf0 msgr2=0x7f7a380b1000 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:19:41.551 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:41.547+0000 7f7a3cafe640 1 -- 172.21.15.143:0/2031998110 shutdown_connections 2023-11-26T07:19:41.551 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:41.547+0000 7f7a3cafe640 1 -- 172.21.15.143:0/2031998110 wait complete. 2023-11-26T07:19:41.552 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:41.547+0000 7f7a3cafe640 1 Processor -- start 2023-11-26T07:19:41.552 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:41.547+0000 7f7a3cafe640 1 -- start start 2023-11-26T07:19:41.554 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:41.551+0000 7f7a3cafe640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f7a38070390 0x7f7a38140b40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:19:41.554 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:41.551+0000 7f7a3cafe640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f7a38071550 0x7f7a38165310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:19:41.554 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:41.551+0000 7f7a3cafe640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f7a38143c00 0x7f7a38166be0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:19:41.554 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:41.551+0000 7f7a3cafe640 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7a38166750 con 0x7f7a38143c00 2023-11-26T07:19:41.554 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:41.551+0000 7f7a3cafe640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7a38141080 con 0x7f7a38070390 2023-11-26T07:19:41.554 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:41.551+0000 7f7a3cafe640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7a38165b40 con 0x7f7a38071550 2023-11-26T07:19:41.554 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:41.551+0000 7f7a36d9d640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f7a38143c00 0x7f7a38166be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:19:41.555 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:41.551+0000 7f7a36d9d640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f7a38143c00 0x7f7a38166be0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:52306/0 (socket says 172.21.15.143:52306) 2023-11-26T07:19:41.555 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:41.551+0000 7f7a3659c640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f7a38070390 0x7f7a38140b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:19:41.555 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:41.551+0000 7f7a36d9d640 1 -- 172.21.15.143:0/728925976 learned_addr learned my addr 172.21.15.143:0/728925976 (peer_addr_for_me v2:172.21.15.143:0/0) 2023-11-26T07:19:41.555 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:41.551+0000 7f7a35d9b640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f7a38071550 0x7f7a38165310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:19:41.555 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:41.551+0000 7f7a3659c640 1 -- 172.21.15.143:0/728925976 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f7a38071550 msgr2=0x7f7a38165310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:19:41.555 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:41.551+0000 7f7a3659c640 1 --2- 172.21.15.143:0/728925976 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f7a38071550 0x7f7a38165310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:19:41.555 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:41.551+0000 7f7a3659c640 1 -- 172.21.15.143:0/728925976 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f7a38143c00 msgr2=0x7f7a38166be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:19:41.555 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:41.551+0000 7f7a3659c640 1 --2- 172.21.15.143:0/728925976 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f7a38143c00 0x7f7a38166be0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:19:41.555 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:41.551+0000 7f7a3659c640 1 -- 172.21.15.143:0/728925976 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7a28009420 con 0x7f7a38070390 2023-11-26T07:19:41.556 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:41.551+0000 7f7a36d9d640 1 --2- 172.21.15.143:0/728925976 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f7a38143c00 0x7f7a38166be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-26T07:19:41.557 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:41.551+0000 7f7a3659c640 1 --2- 172.21.15.143:0/728925976 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f7a38070390 0x7f7a38140b40 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 crypto rx=0x7f7a280ab000 tx=0x7f7a28019190 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:19:41.557 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:41.551+0000 7f7a26ffd640 1 -- 172.21.15.143:0/728925976 <== mon.1 v2:172.21.15.188:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7a280ba070 con 0x7f7a38070390 2023-11-26T07:19:41.557 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:41.551+0000 7f7a26ffd640 1 -- 172.21.15.143:0/728925976 <== mon.1 v2:172.21.15.188:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7a280ab910 con 0x7f7a38070390 2023-11-26T07:19:41.557 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:41.551+0000 7f7a3cafe640 1 -- 172.21.15.143:0/728925976 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7a38165e20 con 0x7f7a38070390 2023-11-26T07:19:41.557 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:41.551+0000 7f7a26ffd640 1 -- 172.21.15.143:0/728925976 <== mon.1 v2:172.21.15.188:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7a280abcc0 con 0x7f7a38070390 2023-11-26T07:19:41.557 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:41.551+0000 7f7a3cafe640 1 -- 172.21.15.143:0/728925976 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7a381663e0 con 0x7f7a38070390 2023-11-26T07:19:41.557 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:41.551+0000 7f7a3cafe640 1 -- 172.21.15.143:0/728925976 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7a38067720 con 0x7f7a38070390 2023-11-26T07:19:41.557 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:41.551+0000 7f7a26ffd640 1 -- 172.21.15.143:0/728925976 <== mon.1 v2:172.21.15.188:3300/0 4 ==== mgrmap(e 7) v1 ==== 86005+0+0 (secure 0 0 0) 0x7f7a280b2040 con 0x7f7a38070390 2023-11-26T07:19:41.558 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:41.555+0000 7f7a26ffd640 1 --2- 172.21.15.143:0/728925976 >> [v2:172.21.15.143:6832/2150557544,v1:172.21.15.143:6833/2150557544] conn(0x7f7a18066620 0x7f7a18068ae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:19:41.558 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:41.555+0000 7f7a26ffd640 1 -- 172.21.15.143:0/728925976 <== mon.1 v2:172.21.15.188:3300/0 5 ==== osd_map(45..45 src has 1..45) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f7a2811d2b0 con 0x7f7a38070390 2023-11-26T07:19:41.558 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:41.555+0000 7f7a35d9b640 1 --2- 172.21.15.143:0/728925976 >> [v2:172.21.15.143:6832/2150557544,v1:172.21.15.143:6833/2150557544] conn(0x7f7a18066620 0x7f7a18068ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:19:41.559 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:41.555+0000 7f7a35d9b640 1 --2- 172.21.15.143:0/728925976 >> [v2:172.21.15.143:6832/2150557544,v1:172.21.15.143:6833/2150557544] conn(0x7f7a18066620 0x7f7a18068ae0 secure :-1 s=READY pgs=296 cs=0 l=1 rev1=1 crypto rx=0x7f7a380ae600 tx=0x7f7a2c007d40 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:19:41.561 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:41.559+0000 7f7a26ffd640 1 -- 172.21.15.143:0/728925976 <== mon.1 v2:172.21.15.188:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172649 (secure 0 0 0) 0x7f7a280ea320 con 0x7f7a38070390 2023-11-26T07:19:41.763 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:41.759+0000 7f7a3cafe640 1 -- 172.21.15.143:0/728925976 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_command({"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]} v 0) v1 -- 0x7f7a380b17a0 con 0x7f7a38070390 2023-11-26T07:19:41.769 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:41.767+0000 7f7a26ffd640 1 -- 172.21.15.143:0/728925976 <== mon.1 v2:172.21.15.188:3300/0 7 ==== mon_command_ack([{"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]}]=0 updated caps for client.0 v14) v1 ==== 190+0+147 (secure 0 0 0) 0x7f7a280b00e0 con 0x7f7a38070390 2023-11-26T07:19:41.769 INFO:teuthology.orchestra.run.smithi143.stdout:[client.0] 2023-11-26T07:19:41.769 INFO:teuthology.orchestra.run.smithi143.stdout: key = AQDm8GJlWMh9FBAAslVzi6fm4/wt7vr/rHCHHQ== 2023-11-26T07:19:41.769 INFO:teuthology.orchestra.run.smithi143.stdout: caps mds = "allow" 2023-11-26T07:19:41.769 INFO:teuthology.orchestra.run.smithi143.stdout: caps mon = "allow r" 2023-11-26T07:19:41.770 INFO:teuthology.orchestra.run.smithi143.stdout: caps osd = "allow rw tag cephfs data=cephfs" 2023-11-26T07:19:41.770 INFO:teuthology.orchestra.run.smithi143.stderr:updated caps for client.0 2023-11-26T07:19:41.771 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:41.767+0000 7f7a3cafe640 1 -- 172.21.15.143:0/728925976 >> [v2:172.21.15.143:6832/2150557544,v1:172.21.15.143:6833/2150557544] conn(0x7f7a18066620 msgr2=0x7f7a18068ae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:19:41.771 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:41.767+0000 7f7a3cafe640 1 --2- 172.21.15.143:0/728925976 >> [v2:172.21.15.143:6832/2150557544,v1:172.21.15.143:6833/2150557544] conn(0x7f7a18066620 0x7f7a18068ae0 secure :-1 s=READY pgs=296 cs=0 l=1 rev1=1 crypto rx=0x7f7a380ae600 tx=0x7f7a2c007d40 comp rx=0 tx=0).stop 2023-11-26T07:19:41.771 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:41.767+0000 7f7a3cafe640 1 -- 172.21.15.143:0/728925976 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f7a38070390 msgr2=0x7f7a38140b40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:19:41.771 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:41.767+0000 7f7a3cafe640 1 --2- 172.21.15.143:0/728925976 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f7a38070390 0x7f7a38140b40 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 crypto rx=0x7f7a280ab000 tx=0x7f7a28019190 comp rx=0 tx=0).stop 2023-11-26T07:19:41.771 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:41.767+0000 7f7a3cafe640 1 -- 172.21.15.143:0/728925976 shutdown_connections 2023-11-26T07:19:41.771 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:41.767+0000 7f7a3cafe640 1 --2- 172.21.15.143:0/728925976 >> [v2:172.21.15.143:6832/2150557544,v1:172.21.15.143:6833/2150557544] conn(0x7f7a18066620 0x7f7a18068ae0 unknown :-1 s=CLOSED pgs=296 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:19:41.771 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:41.767+0000 7f7a3cafe640 1 --2- 172.21.15.143:0/728925976 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f7a38143c00 0x7f7a38166be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:19:41.771 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:41.767+0000 7f7a3cafe640 1 --2- 172.21.15.143:0/728925976 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f7a38071550 0x7f7a38165310 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:19:41.771 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:41.767+0000 7f7a3cafe640 1 --2- 172.21.15.143:0/728925976 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f7a38070390 0x7f7a38140b40 unknown :-1 s=CLOSED pgs=255 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:19:41.772 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:41.767+0000 7f7a3cafe640 1 -- 172.21.15.143:0/728925976 >> 172.21.15.143:0/728925976 conn(0x7f7a380b0bf0 msgr2=0x7f7a38070c60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:19:41.772 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:41.767+0000 7f7a3cafe640 1 -- 172.21.15.143:0/728925976 shutdown_connections 2023-11-26T07:19:41.772 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:41.767+0000 7f7a3cafe640 1 -- 172.21.15.143:0/728925976 wait complete. 2023-11-26T07:19:41.790 DEBUG:teuthology.orchestra.run.smithi143:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-11-26T07:19:41.928 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:41.923+0000 7f5541501640 1 Processor -- start 2023-11-26T07:19:41.928 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:41.923+0000 7f5541501640 1 -- start start 2023-11-26T07:19:41.929 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:41.927+0000 7f5541501640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f553c143ac0 0x7f553c070370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:19:41.929 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:41.927+0000 7f5541501640 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7f553c0cf990 0x7f553c0cd1b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:19:41.930 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:41.927+0000 7f5541501640 1 -- --> v1:172.21.15.143:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f553c0cc730 con 0x7f553c0708b0 2023-11-26T07:19:41.930 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:41.927+0000 7f5541501640 1 -- --> v2:172.21.15.143:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f553c143ea0 con 0x7f553c0cf990 2023-11-26T07:19:41.930 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:41.927+0000 7f5541501640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f553c070ee0 con 0x7f553c143ac0 2023-11-26T07:19:41.930 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:41.927+0000 7f553b7fe640 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7f553c0cf990 0x7f553c0cd1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:19:41.930 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:41.927+0000 7f553b7fe640 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7f553c0cf990 0x7f553c0cd1b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:52322/0 (socket says 172.21.15.143:52322) 2023-11-26T07:19:41.930 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:41.927+0000 7f553b7fe640 1 -- 172.21.15.143:0/1051703680 learned_addr learned my addr 172.21.15.143:0/1051703680 (peer_addr_for_me v2:172.21.15.143:0/0) 2023-11-26T07:19:41.930 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:41.927+0000 7f55327fc640 1 --1- >> v1:172.21.15.143:6789/0 conn(0x7f553c0708b0 0x7f553c0cf280 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.143:6789/0 says I am v1:172.21.15.143:41770/0 (socket says 172.21.15.143:41770) 2023-11-26T07:19:41.930 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:41.927+0000 7f553affd640 1 --2- 172.21.15.143:0/1051703680 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f553c143ac0 0x7f553c070370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:19:41.930 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:41.927+0000 7f553b7fe640 1 -- 172.21.15.143:0/1051703680 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f553c143ac0 msgr2=0x7f553c070370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-26T07:19:41.930 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:41.927+0000 7f553b7fe640 1 --2- 172.21.15.143:0/1051703680 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f553c143ac0 0x7f553c070370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:19:41.930 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:41.927+0000 7f553b7fe640 1 -- 172.21.15.143:0/1051703680 >> v1:172.21.15.143:6789/0 conn(0x7f553c0708b0 legacy=0x7f553c0cf280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:19:41.931 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:41.927+0000 7f553b7fe640 1 -- 172.21.15.143:0/1051703680 --> v2:172.21.15.143:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f553c0711c0 con 0x7f553c0cf990 2023-11-26T07:19:41.932 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:41.927+0000 7f553b7fe640 1 --2- 172.21.15.143:0/1051703680 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f553c0cf990 0x7f553c0cd1b0 secure :-1 s=READY pgs=591 cs=0 l=1 rev1=1 crypto rx=0x7f5534009e50 tx=0x7f553401ca10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=afaab3dd2164d643 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:19:41.933 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:41.931+0000 7f553a7fc640 1 -- 172.21.15.143:0/1051703680 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f55340a57f0 con 0x7f553c0cf990 2023-11-26T07:19:41.933 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:41.931+0000 7f553a7fc640 1 -- 172.21.15.143:0/1051703680 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f55340a5950 con 0x7f553c0cf990 2023-11-26T07:19:41.933 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:41.931+0000 7f553a7fc640 1 -- 172.21.15.143:0/1051703680 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f55340a5ce0 con 0x7f553c0cf990 2023-11-26T07:19:41.934 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:41.931+0000 7f5541501640 1 -- 172.21.15.143:0/1051703680 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f553c0cf990 msgr2=0x7f553c0cd1b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:19:41.934 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:41.931+0000 7f5541501640 1 --2- 172.21.15.143:0/1051703680 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f553c0cf990 0x7f553c0cd1b0 secure :-1 s=READY pgs=591 cs=0 l=1 rev1=1 crypto rx=0x7f5534009e50 tx=0x7f553401ca10 comp rx=0 tx=0).stop 2023-11-26T07:19:41.934 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:41.931+0000 7f5541501640 1 -- 172.21.15.143:0/1051703680 shutdown_connections 2023-11-26T07:19:41.934 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:41.931+0000 7f5541501640 1 --2- 172.21.15.143:0/1051703680 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f553c0cf990 0x7f553c0cd1b0 unknown :-1 s=CLOSED pgs=591 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:19:41.934 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:41.931+0000 7f5541501640 1 --2- 172.21.15.143:0/1051703680 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f553c143ac0 0x7f553c070370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:19:41.934 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:41.931+0000 7f5541501640 1 -- 172.21.15.143:0/1051703680 >> 172.21.15.143:0/1051703680 conn(0x7f553c06a890 msgr2=0x7f553c06aca0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:19:41.934 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:41.931+0000 7f5541501640 1 -- 172.21.15.143:0/1051703680 shutdown_connections 2023-11-26T07:19:41.934 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:41.931+0000 7f5541501640 1 -- 172.21.15.143:0/1051703680 wait complete. 2023-11-26T07:19:41.935 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:41.931+0000 7f5541501640 1 Processor -- start 2023-11-26T07:19:41.935 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:41.931+0000 7f5541501640 1 -- start start 2023-11-26T07:19:41.937 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:41.931+0000 7f5541501640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f553c0708b0 0x7f553c0b1120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:19:41.937 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:41.931+0000 7f5541501640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f553c0cf990 0x7f553c14b8b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:19:41.937 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:41.931+0000 7f5541501640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f553c143ac0 0x7f553c0a8ec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:19:41.937 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:41.931+0000 7f5541501640 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f553c14bdf0 con 0x7f553c143ac0 2023-11-26T07:19:41.937 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:41.931+0000 7f5541501640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f553c0a96b0 con 0x7f553c0708b0 2023-11-26T07:19:41.937 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:41.931+0000 7f5541501640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f553c0a9400 con 0x7f553c0cf990 2023-11-26T07:19:41.937 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:41.931+0000 7f553b7fe640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f553c143ac0 0x7f553c0a8ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:19:41.937 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:41.931+0000 7f553b7fe640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f553c143ac0 0x7f553c0a8ec0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:52324/0 (socket says 172.21.15.143:52324) 2023-11-26T07:19:41.938 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:41.931+0000 7f553b7fe640 1 -- 172.21.15.143:0/945840672 learned_addr learned my addr 172.21.15.143:0/945840672 (peer_addr_for_me v2:172.21.15.143:0/0) 2023-11-26T07:19:41.938 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:41.935+0000 7f55327fc640 1 --2- 172.21.15.143:0/945840672 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f553c0cf990 0x7f553c14b8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:19:41.938 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:41.935+0000 7f553affd640 1 --2- 172.21.15.143:0/945840672 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f553c0708b0 0x7f553c0b1120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:19:41.938 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:41.935+0000 7f553b7fe640 1 -- 172.21.15.143:0/945840672 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f553c0cf990 msgr2=0x7f553c14b8b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:19:41.938 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:41.935+0000 7f553b7fe640 1 --2- 172.21.15.143:0/945840672 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f553c0cf990 0x7f553c14b8b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:19:41.938 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:41.935+0000 7f553b7fe640 1 -- 172.21.15.143:0/945840672 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f553c0708b0 msgr2=0x7f553c0b1120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:19:41.938 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:41.935+0000 7f553b7fe640 1 --2- 172.21.15.143:0/945840672 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f553c0708b0 0x7f553c0b1120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:19:41.938 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:41.935+0000 7f553b7fe640 1 -- 172.21.15.143:0/945840672 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5534009b30 con 0x7f553c143ac0 2023-11-26T07:19:41.938 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:41.935+0000 7f55327fc640 1 --2- 172.21.15.143:0/945840672 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f553c0cf990 0x7f553c14b8b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-26T07:19:41.938 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:41.935+0000 7f553b7fe640 1 --2- 172.21.15.143:0/945840672 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f553c143ac0 0x7f553c0a8ec0 secure :-1 s=READY pgs=592 cs=0 l=1 rev1=1 crypto rx=0x7f55340af040 tx=0x7f55340197b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:19:41.939 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:41.935+0000 7f5533fff640 1 -- 172.21.15.143:0/945840672 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f55340b8670 con 0x7f553c143ac0 2023-11-26T07:19:41.939 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:41.935+0000 7f5533fff640 1 -- 172.21.15.143:0/945840672 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f55340b87d0 con 0x7f553c143ac0 2023-11-26T07:19:41.939 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:41.935+0000 7f5541501640 1 -- 172.21.15.143:0/945840672 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f553c0b0360 con 0x7f553c143ac0 2023-11-26T07:19:41.939 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:41.935+0000 7f5533fff640 1 -- 172.21.15.143:0/945840672 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f55340b8b80 con 0x7f553c143ac0 2023-11-26T07:19:41.940 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:41.935+0000 7f5541501640 1 -- 172.21.15.143:0/945840672 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f553c0b0920 con 0x7f553c143ac0 2023-11-26T07:19:41.940 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:41.935+0000 7f5541501640 1 -- 172.21.15.143:0/945840672 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f553c067680 con 0x7f553c143ac0 2023-11-26T07:19:41.940 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:41.935+0000 7f5533fff640 1 -- 172.21.15.143:0/945840672 <== mon.0 v2:172.21.15.143:3300/0 4 ==== mgrmap(e 7) v1 ==== 86005+0+0 (secure 0 0 0) 0x7f55340b8ce0 con 0x7f553c143ac0 2023-11-26T07:19:41.941 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:41.939+0000 7f5533fff640 1 --2- 172.21.15.143:0/945840672 >> [v2:172.21.15.143:6832/2150557544,v1:172.21.15.143:6833/2150557544] conn(0x7f551c066620 0x7f551c068ae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:19:41.941 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:41.939+0000 7f5533fff640 1 -- 172.21.15.143:0/945840672 <== mon.0 v2:172.21.15.143:3300/0 5 ==== osd_map(45..45 src has 1..45) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f553411f710 con 0x7f553c143ac0 2023-11-26T07:19:41.941 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:41.939+0000 7f553affd640 1 --2- 172.21.15.143:0/945840672 >> [v2:172.21.15.143:6832/2150557544,v1:172.21.15.143:6833/2150557544] conn(0x7f551c066620 0x7f551c068ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:19:41.941 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:41.939+0000 7f553affd640 1 --2- 172.21.15.143:0/945840672 >> [v2:172.21.15.143:6832/2150557544,v1:172.21.15.143:6833/2150557544] conn(0x7f551c066620 0x7f551c068ae0 secure :-1 s=READY pgs=297 cs=0 l=1 rev1=1 crypto rx=0x7f552c00b930 tx=0x7f552c008040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:19:41.944 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:41.943+0000 7f5533fff640 1 -- 172.21.15.143:0/945840672 <== mon.0 v2:172.21.15.143:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172649 (secure 0 0 0) 0x7f55340ec880 con 0x7f553c143ac0 2023-11-26T07:19:42.126 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:42.123+0000 7f5541501640 1 -- 172.21.15.143:0/945840672 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f553c13a150 con 0x7f553c143ac0 2023-11-26T07:19:42.127 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:42.123+0000 7f5533fff640 1 -- 172.21.15.143:0/945840672 <== mon.0 v2:172.21.15.143:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 25 v25) v1 ==== 94+0+4922 (secure 0 0 0) 0x7f55340efea0 con 0x7f553c143ac0 2023-11-26T07:19:42.127 INFO:teuthology.orchestra.run.smithi143.stdout: 2023-11-26T07:19:42.127 INFO:teuthology.orchestra.run.smithi143.stdout:{"epoch":25,"default_fscid":3,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5201,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.188:6835/319464848","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6833","nonce":319464848},{"type":"v1","addr":"172.21.15.188:6835","nonce":319464848}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":20},{"gid":5584,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.188:6834/2113408070","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6832","nonce":2113408070},{"type":"v1","addr":"172.21.15.188:6834","nonce":2113408070}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":20},{"gid":5745,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.143:6835/1616742593","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6834","nonce":1616742593},{"type":"v1","addr":"172.21.15.143:6835","nonce":1616742593}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":20}],"filesystems":[{"mdsmap":{"epoch":25,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-11-26T07:19:31.503587+0000","modified":"2023-11-26T07:19:34.523605+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":5748},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5748":{"gid":5748,"name":"c","rank":0,"incarnation":21,"state":"up:active","state_seq":3,"addr":"172.21.15.143:6837/2872033338","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6836","nonce":2872033338},{"type":"v1","addr":"172.21.15.143:6837","nonce":2872033338}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[7],"metadata_pool":6,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":3}]} 2023-11-26T07:19:42.128 INFO:teuthology.orchestra.run.smithi143.stderr:dumped fsmap epoch 25 2023-11-26T07:19:42.129 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:42.127+0000 7f5541501640 1 -- 172.21.15.143:0/945840672 >> [v2:172.21.15.143:6832/2150557544,v1:172.21.15.143:6833/2150557544] conn(0x7f551c066620 msgr2=0x7f551c068ae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:19:42.130 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:42.127+0000 7f5541501640 1 --2- 172.21.15.143:0/945840672 >> [v2:172.21.15.143:6832/2150557544,v1:172.21.15.143:6833/2150557544] conn(0x7f551c066620 0x7f551c068ae0 secure :-1 s=READY pgs=297 cs=0 l=1 rev1=1 crypto rx=0x7f552c00b930 tx=0x7f552c008040 comp rx=0 tx=0).stop 2023-11-26T07:19:42.130 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:42.127+0000 7f5541501640 1 -- 172.21.15.143:0/945840672 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f553c143ac0 msgr2=0x7f553c0a8ec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:19:42.130 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:42.127+0000 7f5541501640 1 --2- 172.21.15.143:0/945840672 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f553c143ac0 0x7f553c0a8ec0 secure :-1 s=READY pgs=592 cs=0 l=1 rev1=1 crypto rx=0x7f55340af040 tx=0x7f55340197b0 comp rx=0 tx=0).stop 2023-11-26T07:19:42.130 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:42.127+0000 7f5541501640 1 -- 172.21.15.143:0/945840672 shutdown_connections 2023-11-26T07:19:42.130 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:42.127+0000 7f5541501640 1 --2- 172.21.15.143:0/945840672 >> [v2:172.21.15.143:6832/2150557544,v1:172.21.15.143:6833/2150557544] conn(0x7f551c066620 0x7f551c068ae0 unknown :-1 s=CLOSED pgs=297 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:19:42.130 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:42.127+0000 7f5541501640 1 --2- 172.21.15.143:0/945840672 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f553c143ac0 0x7f553c0a8ec0 unknown :-1 s=CLOSED pgs=592 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:19:42.130 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:42.127+0000 7f5541501640 1 --2- 172.21.15.143:0/945840672 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f553c0cf990 0x7f553c14b8b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:19:42.130 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:42.127+0000 7f5541501640 1 --2- 172.21.15.143:0/945840672 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f553c0708b0 0x7f553c0b1120 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:19:42.130 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:42.127+0000 7f5541501640 1 -- 172.21.15.143:0/945840672 >> 172.21.15.143:0/945840672 conn(0x7f553c06a890 msgr2=0x7f553c15c7a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:19:42.131 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:42.127+0000 7f5541501640 1 -- 172.21.15.143:0/945840672 shutdown_connections 2023-11-26T07:19:42.131 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:42.127+0000 7f5541501640 1 -- 172.21.15.143:0/945840672 wait complete. 2023-11-26T07:19:42.551 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 25, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2023-11-26T07:19:31.503587+0000', 'modified': '2023-11-26T07:19:34.523605+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 5748}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_5748': {'gid': 5748, 'name': 'c', 'rank': 0, 'incarnation': 21, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.143:6837/2872033338', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.143:6836', 'nonce': 2872033338}, {'type': 'v1', 'addr': '172.21.15.143:6837', 'nonce': 2872033338}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138322906710015, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}}}, 'data_pools': [7], 'metadata_pool': 6, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0} 2023-11-26T07:19:42.551 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2023-11-26T07:19:42.551 DEBUG:teuthology.orchestra.run.smithi143:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.c status 2023-11-26T07:19:42.690 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:42.687+0000 7f75f7b2a640 1 Processor -- start 2023-11-26T07:19:42.690 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:42.687+0000 7f75f7b2a640 1 -- start start 2023-11-26T07:19:42.692 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:42.687+0000 7f75f7b2a640 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7f75f00cf990 0x7f75f00708c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:19:42.693 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:42.687+0000 7f75f7b2a640 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7f75f0070e00 0x7f75f0162510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:19:42.693 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:42.687+0000 7f75f7b2a640 1 -- --> v1:172.21.15.188:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f75f0162080 con 0x7f75f014b1e0 2023-11-26T07:19:42.693 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:42.687+0000 7f75f7b2a640 1 -- --> v2:172.21.15.143:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f75f0161d70 con 0x7f75f0070e00 2023-11-26T07:19:42.693 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:42.687+0000 7f75f7b2a640 1 -- --> v2:172.21.15.188:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f75f014b5c0 con 0x7f75f00cf990 2023-11-26T07:19:42.693 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:42.687+0000 7f75f60c7640 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7f75f0070e00 0x7f75f0162510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:19:42.693 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:42.687+0000 7f75f60c7640 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7f75f0070e00 0x7f75f0162510 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:52340/0 (socket says 172.21.15.143:52340) 2023-11-26T07:19:42.693 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:42.687+0000 7f75f60c7640 1 -- 172.21.15.143:0/410838077 learned_addr learned my addr 172.21.15.143:0/410838077 (peer_addr_for_me v2:172.21.15.143:0/0) 2023-11-26T07:19:42.693 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:42.687+0000 7f75f50c5640 1 --2- 172.21.15.143:0/410838077 >> v2:172.21.15.188:3300/0 conn(0x7f75f00cf990 0x7f75f00708c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:19:42.693 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:42.687+0000 7f75f60c7640 1 -- 172.21.15.143:0/410838077 >> v2:172.21.15.188:3300/0 conn(0x7f75f00cf990 msgr2=0x7f75f00708c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-26T07:19:42.693 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:42.687+0000 7f75f60c7640 1 --2- 172.21.15.143:0/410838077 >> v2:172.21.15.188:3300/0 conn(0x7f75f00cf990 0x7f75f00708c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:19:42.693 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:42.687+0000 7f75f60c7640 1 -- 172.21.15.143:0/410838077 >> v1:172.21.15.188:6789/0 conn(0x7f75f014b1e0 legacy=0x7f75f00cf280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-26T07:19:42.693 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:42.691+0000 7f75f60c7640 1 -- 172.21.15.143:0/410838077 --> v2:172.21.15.143:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f75f0071580 con 0x7f75f0070e00 2023-11-26T07:19:42.695 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:42.691+0000 7f75f60c7640 1 --2- 172.21.15.143:0/410838077 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f75f0070e00 0x7f75f0162510 secure :-1 s=READY pgs=593 cs=0 l=1 rev1=1 crypto rx=0x7f75f006dfa0 tx=0x7f75e801c960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=14ff0b67948f999b server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:19:42.695 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:42.691+0000 7f75f48c4640 1 -- 172.21.15.143:0/410838077 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f75e80a5820 con 0x7f75f0070e00 2023-11-26T07:19:42.696 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:42.691+0000 7f75f48c4640 1 -- 172.21.15.143:0/410838077 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f75e80a5980 con 0x7f75f0070e00 2023-11-26T07:19:42.696 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:42.691+0000 7f75f48c4640 1 -- 172.21.15.143:0/410838077 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f75e80a5d10 con 0x7f75f0070e00 2023-11-26T07:19:42.696 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:42.691+0000 7f75f7b2a640 1 -- 172.21.15.143:0/410838077 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f75f0070e00 msgr2=0x7f75f0162510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:19:42.697 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:42.691+0000 7f75f7b2a640 1 --2- 172.21.15.143:0/410838077 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f75f0070e00 0x7f75f0162510 secure :-1 s=READY pgs=593 cs=0 l=1 rev1=1 crypto rx=0x7f75f006dfa0 tx=0x7f75e801c960 comp rx=0 tx=0).stop 2023-11-26T07:19:42.697 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:42.691+0000 7f75f7b2a640 1 -- 172.21.15.143:0/410838077 shutdown_connections 2023-11-26T07:19:42.697 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:42.691+0000 7f75f7b2a640 1 --2- 172.21.15.143:0/410838077 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f75f0070e00 0x7f75f0162510 unknown :-1 s=CLOSED pgs=593 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:19:42.697 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:42.691+0000 7f75f7b2a640 1 --2- 172.21.15.143:0/410838077 >> v2:172.21.15.188:3300/0 conn(0x7f75f00cf990 0x7f75f00708c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:19:42.697 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:42.691+0000 7f75f7b2a640 1 -- 172.21.15.143:0/410838077 >> 172.21.15.143:0/410838077 conn(0x7f75f006a890 msgr2=0x7f75f006aca0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:19:42.697 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:42.691+0000 7f75f7b2a640 1 -- 172.21.15.143:0/410838077 shutdown_connections 2023-11-26T07:19:42.697 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:42.691+0000 7f75f7b2a640 1 -- 172.21.15.143:0/410838077 wait complete. 2023-11-26T07:19:42.698 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:42.695+0000 7f75f7b2a640 1 Processor -- start 2023-11-26T07:19:42.698 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:42.695+0000 7f75f7b2a640 1 -- start start 2023-11-26T07:19:42.700 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:42.695+0000 7f75f7b2a640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f75f0070e00 0x7f75f0142cf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:19:42.700 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:42.695+0000 7f75f7b2a640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f75f00cf990 0x7f75f015c2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:19:42.700 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:42.695+0000 7f75f7b2a640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f75f014b1e0 0x7f75f015c830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:19:42.700 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:42.695+0000 7f75f7b2a640 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f75f015aa80 con 0x7f75f0070e00 2023-11-26T07:19:42.700 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:42.695+0000 7f75f7b2a640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f75f0159e00 con 0x7f75f014b1e0 2023-11-26T07:19:42.700 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:42.695+0000 7f75f7b2a640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f75f015a0e0 con 0x7f75f00cf990 2023-11-26T07:19:42.700 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:42.695+0000 7f75f58c6640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f75f0070e00 0x7f75f0142cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:19:42.700 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:42.695+0000 7f75f58c6640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f75f0070e00 0x7f75f0142cf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:52350/0 (socket says 172.21.15.143:52350) 2023-11-26T07:19:42.700 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:42.695+0000 7f75f58c6640 1 -- 172.21.15.143:0/2068237987 learned_addr learned my addr 172.21.15.143:0/2068237987 (peer_addr_for_me v2:172.21.15.143:0/0) 2023-11-26T07:19:42.700 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:42.695+0000 7f75f50c5640 1 --2- 172.21.15.143:0/2068237987 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f75f00cf990 0x7f75f015c2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:19:42.701 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:42.695+0000 7f75f60c7640 1 --2- 172.21.15.143:0/2068237987 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f75f014b1e0 0x7f75f015c830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:19:42.701 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:42.695+0000 7f75f58c6640 1 -- 172.21.15.143:0/2068237987 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f75f00cf990 msgr2=0x7f75f015c2f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:19:42.701 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:42.695+0000 7f75f58c6640 1 --2- 172.21.15.143:0/2068237987 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f75f00cf990 0x7f75f015c2f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:19:42.701 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:42.695+0000 7f75f58c6640 1 -- 172.21.15.143:0/2068237987 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f75f014b1e0 msgr2=0x7f75f015c830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:19:42.701 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:42.695+0000 7f75f58c6640 1 --2- 172.21.15.143:0/2068237987 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f75f014b1e0 0x7f75f015c830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:19:42.701 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:42.695+0000 7f75f58c6640 1 -- 172.21.15.143:0/2068237987 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f75e8009b30 con 0x7f75f0070e00 2023-11-26T07:19:42.701 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:42.695+0000 7f75f58c6640 1 --2- 172.21.15.143:0/2068237987 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f75f0070e00 0x7f75f0142cf0 secure :-1 s=READY pgs=594 cs=0 l=1 rev1=1 crypto rx=0x7f75ec00b410 tx=0x7f75ec00b8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:19:42.702 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:42.695+0000 7f75e67fc640 1 -- 172.21.15.143:0/2068237987 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f75ec004020 con 0x7f75f0070e00 2023-11-26T07:19:42.702 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:42.695+0000 7f75e67fc640 1 -- 172.21.15.143:0/2068237987 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f75ec004180 con 0x7f75f0070e00 2023-11-26T07:19:42.702 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:42.695+0000 7f75e67fc640 1 -- 172.21.15.143:0/2068237987 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f75ec011630 con 0x7f75f0070e00 2023-11-26T07:19:42.702 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:42.695+0000 7f75f7b2a640 1 -- 172.21.15.143:0/2068237987 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f75f015a3c0 con 0x7f75f0070e00 2023-11-26T07:19:42.702 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:42.699+0000 7f75f7b2a640 1 -- 172.21.15.143:0/2068237987 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f75f015b730 con 0x7f75f0070e00 2023-11-26T07:19:42.702 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:42.699+0000 7f75e67fc640 1 -- 172.21.15.143:0/2068237987 <== mon.0 v2:172.21.15.143:3300/0 4 ==== mgrmap(e 7) v1 ==== 86005+0+0 (secure 0 0 0) 0x7f75ec011790 con 0x7f75f0070e00 2023-11-26T07:19:42.703 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:42.699+0000 7f75e67fc640 1 --2- 172.21.15.143:0/2068237987 >> [v2:172.21.15.143:6832/2150557544,v1:172.21.15.143:6833/2150557544] conn(0x7f75d4066620 0x7f75d4068ae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:19:42.703 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:42.699+0000 7f75e67fc640 1 -- 172.21.15.143:0/2068237987 <== mon.0 v2:172.21.15.143:3300/0 5 ==== osd_map(45..45 src has 1..45) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f75ec083da0 con 0x7f75f0070e00 2023-11-26T07:19:42.704 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:42.699+0000 7f75f50c5640 1 --2- 172.21.15.143:0/2068237987 >> [v2:172.21.15.143:6832/2150557544,v1:172.21.15.143:6833/2150557544] conn(0x7f75d4066620 0x7f75d4068ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:19:42.704 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:42.699+0000 7f75faeaf000 1 Processor -- start 2023-11-26T07:19:42.704 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:42.699+0000 7f75faeaf000 1 -- start start 2023-11-26T07:19:42.704 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:42.703+0000 7f75f50c5640 1 --2- 172.21.15.143:0/2068237987 >> [v2:172.21.15.143:6832/2150557544,v1:172.21.15.143:6833/2150557544] conn(0x7f75d4066620 0x7f75d4068ae0 secure :-1 s=READY pgs=298 cs=0 l=1 rev1=1 crypto rx=0x7f75f0162250 tx=0x7f75e0008040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:19:42.706 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:42.703+0000 7f75faeaf000 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55b8e9fdbc90 0x55b8e9fdc070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:19:42.706 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:42.703+0000 7f75faeaf000 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55b8e9fdc5b0 0x55b8e9f91de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:19:42.706 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:42.703+0000 7f75faeaf000 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x55b8e9f92320 0x55b8e9f92700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:19:42.706 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:42.703+0000 7f75faeaf000 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b8e9dfc130 con 0x55b8e9fdbc90 2023-11-26T07:19:42.706 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:42.703+0000 7f75faeaf000 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b8e9de99a0 con 0x55b8e9fdc5b0 2023-11-26T07:19:42.706 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:42.703+0000 7f75faeaf000 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x55b8e9de6ed0 con 0x55b8e9f92320 2023-11-26T07:19:42.706 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:42.703+0000 7f75f58c6640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55b8e9fdbc90 0x55b8e9fdc070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:19:42.706 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:42.703+0000 7f75f58c6640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55b8e9fdbc90 0x55b8e9fdc070 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:52360/0 (socket says 172.21.15.143:52360) 2023-11-26T07:19:42.706 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:42.703+0000 7f75f58c6640 1 -- 172.21.15.143:0/2007257785 learned_addr learned my addr 172.21.15.143:0/2007257785 (peer_addr_for_me v2:172.21.15.143:0/0) 2023-11-26T07:19:42.707 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:42.703+0000 7f75f58c6640 1 -- 172.21.15.143:0/2007257785 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x55b8e9f92320 msgr2=0x55b8e9f92700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-26T07:19:42.707 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:42.703+0000 7f75f58c6640 1 --2- 172.21.15.143:0/2007257785 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x55b8e9f92320 0x55b8e9f92700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:19:42.707 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:42.703+0000 7f75f58c6640 1 -- 172.21.15.143:0/2007257785 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55b8e9fdc5b0 msgr2=0x55b8e9f91de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-26T07:19:42.707 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:42.703+0000 7f75f58c6640 1 --2- 172.21.15.143:0/2007257785 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55b8e9fdc5b0 0x55b8e9f91de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:19:42.707 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:42.703+0000 7f75f58c6640 1 -- 172.21.15.143:0/2007257785 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55b8e9ada890 con 0x55b8e9fdbc90 2023-11-26T07:19:42.707 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:42.703+0000 7f75f58c6640 1 --2- 172.21.15.143:0/2007257785 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55b8e9fdbc90 0x55b8e9fdc070 secure :-1 s=READY pgs=595 cs=0 l=1 rev1=1 crypto rx=0x7f75ec051200 tx=0x7f75ec054e40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=50ce74c7817131a7 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:19:42.708 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:42.703+0000 7f75d37fe640 1 -- 172.21.15.143:0/2007257785 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f75ec051410 con 0x55b8e9fdbc90 2023-11-26T07:19:42.708 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:42.703+0000 7f75d37fe640 1 -- 172.21.15.143:0/2007257785 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f75ec051570 con 0x55b8e9fdbc90 2023-11-26T07:19:42.708 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:42.703+0000 7f75d37fe640 1 -- 172.21.15.143:0/2007257785 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f75ec0823d0 con 0x55b8e9fdbc90 2023-11-26T07:19:42.708 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:42.703+0000 7f75faeaf000 1 -- 172.21.15.143:0/2007257785 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55b8e9fdbc90 msgr2=0x55b8e9fdc070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:19:42.708 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:42.703+0000 7f75faeaf000 1 --2- 172.21.15.143:0/2007257785 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55b8e9fdbc90 0x55b8e9fdc070 secure :-1 s=READY pgs=595 cs=0 l=1 rev1=1 crypto rx=0x7f75ec051200 tx=0x7f75ec054e40 comp rx=0 tx=0).stop 2023-11-26T07:19:42.708 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:42.703+0000 7f75faeaf000 1 -- 172.21.15.143:0/2007257785 shutdown_connections 2023-11-26T07:19:42.709 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:42.703+0000 7f75faeaf000 1 --2- 172.21.15.143:0/2007257785 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x55b8e9f92320 0x55b8e9f92700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:19:42.709 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:42.703+0000 7f75faeaf000 1 --2- 172.21.15.143:0/2007257785 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55b8e9fdc5b0 0x55b8e9f91de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:19:42.709 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:42.703+0000 7f75faeaf000 1 --2- 172.21.15.143:0/2007257785 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55b8e9fdbc90 0x55b8e9fdc070 unknown :-1 s=CLOSED pgs=595 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:19:42.709 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:42.703+0000 7f75faeaf000 1 -- 172.21.15.143:0/2007257785 >> 172.21.15.143:0/2007257785 conn(0x55b8e9f8e2d0 msgr2=0x55b8e9f906c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:19:42.709 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:42.703+0000 7f75faeaf000 1 -- 172.21.15.143:0/2007257785 shutdown_connections 2023-11-26T07:19:42.709 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:42.703+0000 7f75faeaf000 1 -- 172.21.15.143:0/2007257785 wait complete. 2023-11-26T07:19:42.709 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:42.703+0000 7f75faeaf000 1 -- start start 2023-11-26T07:19:42.709 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:42.703+0000 7f75faeaf000 1 Processor -- start 2023-11-26T07:19:42.711 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:42.707+0000 7f75faeaf000 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55b8e9f92320 0x55b8e9f96d40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:19:42.712 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:42.707+0000 7f75faeaf000 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x55b8e9fdbc90 0x55b8e9f97280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:19:42.712 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:42.707+0000 7f75faeaf000 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55b8e9fdc5b0 0x55b8e9faa910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:19:42.712 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:42.707+0000 7f75faeaf000 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b8e9de27c0 con 0x55b8e9f92320 2023-11-26T07:19:42.712 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:42.707+0000 7f75faeaf000 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b8e9d9abf0 con 0x55b8e9fdc5b0 2023-11-26T07:19:42.712 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:42.707+0000 7f75faeaf000 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x55b8e9da0a90 con 0x55b8e9fdbc90 2023-11-26T07:19:42.712 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:42.707+0000 7f75faeaf000 10 client.0 _ll_register_callbacks cb 0x55b8e9f7d3c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-11-26T07:19:42.712 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:42.707+0000 7f75f58c6640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55b8e9f92320 0x55b8e9f96d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:19:42.712 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:42.707+0000 7f75f58c6640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55b8e9f92320 0x55b8e9f96d40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:52364/0 (socket says 172.21.15.143:52364) 2023-11-26T07:19:42.712 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:42.707+0000 7f75f58c6640 1 -- 172.21.15.143:0/3633363013 learned_addr learned my addr 172.21.15.143:0/3633363013 (peer_addr_for_me v2:172.21.15.143:0/0) 2023-11-26T07:19:42.712 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:42.707+0000 7f75f60c7640 1 --2- 172.21.15.143:0/3633363013 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55b8e9fdc5b0 0x55b8e9faa910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:19:42.712 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:42.707+0000 7f75f50c5640 1 --2- 172.21.15.143:0/3633363013 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x55b8e9fdbc90 0x55b8e9f97280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:19:42.712 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:42.707+0000 7f75f58c6640 1 -- 172.21.15.143:0/3633363013 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x55b8e9fdbc90 msgr2=0x55b8e9f97280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-26T07:19:42.713 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:42.707+0000 7f75f58c6640 1 --2- 172.21.15.143:0/3633363013 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x55b8e9fdbc90 0x55b8e9f97280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:19:42.713 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:42.707+0000 7f75f58c6640 1 -- 172.21.15.143:0/3633363013 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55b8e9fdc5b0 msgr2=0x55b8e9faa910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-26T07:19:42.713 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:42.707+0000 7f75f58c6640 1 --2- 172.21.15.143:0/3633363013 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55b8e9fdc5b0 0x55b8e9faa910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:19:42.713 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:42.707+0000 7f75f58c6640 1 -- 172.21.15.143:0/3633363013 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f75f015b730 con 0x55b8e9f92320 2023-11-26T07:19:42.713 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:42.707+0000 7f75f50c5640 1 --2- 172.21.15.143:0/3633363013 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x55b8e9fdbc90 0x55b8e9f97280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-26T07:19:42.713 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:42.707+0000 7f75f58c6640 1 --2- 172.21.15.143:0/3633363013 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55b8e9f92320 0x55b8e9f96d40 secure :-1 s=READY pgs=596 cs=0 l=1 rev1=1 crypto rx=0x7f75ec053050 tx=0x7f75ec004690 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c92951bd4ac8b676 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:19:42.713 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:42.707+0000 7f75d0ff9640 10 client.0 ms_handle_connect on v2:172.21.15.143:3300/0 2023-11-26T07:19:42.713 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:42.707+0000 7f75d0ff9640 1 -- 172.21.15.143:0/3633363013 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f75ec082390 con 0x55b8e9f92320 2023-11-26T07:19:42.713 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:42.707+0000 7f75d0ff9640 1 -- 172.21.15.143:0/3633363013 <== mon.0 v2:172.21.15.143:3300/0 2 ==== osd_map(45..45 src has 1..45) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f75ec05cac0 con 0x55b8e9f92320 2023-11-26T07:19:42.713 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:42.707+0000 7f75faeaf000 1 -- 172.21.15.143:0/3633363013 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55b8e9da4470 con 0x55b8e9f92320 2023-11-26T07:19:42.713 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:42.707+0000 7f75d0ff9640 1 -- 172.21.15.143:0/3633363013 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) v2 ==== 24+0+0 (secure 0 0 0) 0x7f75ec05cca0 con 0x55b8e9f92320 2023-11-26T07:19:42.713 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:42.707+0000 7f75faeaf000 10 client.5904 fetch_fsmap learned FSMap version 25 2023-11-26T07:19:42.713 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:42.707+0000 7f75faeaf000 1 -- 172.21.15.143:0/3633363013 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=2+}) v3 -- 0x55b8e9da3190 con 0x55b8e9f92320 2023-11-26T07:19:42.714 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:42.707+0000 7f75d0ff9640 1 -- 172.21.15.143:0/3633363013 <== mon.0 v2:172.21.15.143:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x55b8e9da4470 con 0x55b8e9f92320 2023-11-26T07:19:42.714 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:42.707+0000 7f75d0ff9640 1 -- 172.21.15.143:0/3633363013 <== mon.0 v2:172.21.15.143:3300/0 5 ==== fsmap(e 25) v1 ==== 3294+0+0 (secure 0 0 0) 0x7f75ec06c210 con 0x55b8e9f92320 2023-11-26T07:19:42.714 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:42.707+0000 7f75faeaf000 10 client.5904 fetch_fsmap finished waiting for FSMap version 25 2023-11-26T07:19:42.714 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:42.707+0000 7f75faeaf000 10 client.5904 resolve_mds: resolved name 'c' to daemon mds.c 2023-11-26T07:19:42.714 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:42.707+0000 7f75faeaf000 20 client.5904 populate_metadata read hostname 'smithi143' 2023-11-26T07:19:42.714 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:42.707+0000 7f75faeaf000 1 --2- 172.21.15.143:0/3633363013 >> [v2:172.21.15.143:6836/2872033338,v1:172.21.15.143:6837/2872033338] conn(0x55b8e9fab320 0x55b8e9fedf50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:19:42.714 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:42.707+0000 7f75faeaf000 4 client.5904 mds_command: new command op to 5748 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2023-11-26T07:19:42.714 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:42.707+0000 7f75faeaf000 1 -- 172.21.15.143:0/3633363013 --> [v2:172.21.15.143:6836/2872033338,v1:172.21.15.143:6837/2872033338] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55b8e9da7b50 con 0x55b8e9fab320 2023-11-26T07:19:42.714 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:42.707+0000 7f75f50c5640 1 --2- 172.21.15.143:0/3633363013 >> [v2:172.21.15.143:6836/2872033338,v1:172.21.15.143:6837/2872033338] conn(0x55b8e9fab320 0x55b8e9fedf50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:19:42.714 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:42.707+0000 7f75d0ff9640 10 client.5904 ms_handle_connect on v2:172.21.15.143:6836/2872033338 2023-11-26T07:19:42.714 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:42.707+0000 7f75f50c5640 1 --2- 172.21.15.143:0/3633363013 >> [v2:172.21.15.143:6836/2872033338,v1:172.21.15.143:6837/2872033338] conn(0x55b8e9fab320 0x55b8e9fedf50 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=54109f0bce327a4c server_cookie=c3cbfcb3fb4a815c in_seq=0 out_seq=0 2023-11-26T07:19:42.715 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:42.707+0000 7f75d0ff9640 1 -- 172.21.15.143:0/3633363013 <== mds.0 v2:172.21.15.143:6836/2872033338 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x55b8e9da7b50 con 0x55b8e9fab320 2023-11-26T07:19:42.715 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:42.707+0000 7f75d0ff9640 10 client.5904 handle_command_reply: tid=0 2023-11-26T07:19:42.716 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:42.707+0000 7f75faeaf000 1 client.5904 shutdown 2023-11-26T07:19:42.716 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:42.707+0000 7f75faeaf000 1 -- 172.21.15.143:0/3633363013 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55b8e9f92320 msgr2=0x55b8e9f96d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:19:42.716 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:42.707+0000 7f75faeaf000 1 --2- 172.21.15.143:0/3633363013 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55b8e9f92320 0x55b8e9f96d40 secure :-1 s=READY pgs=596 cs=0 l=1 rev1=1 crypto rx=0x7f75ec053050 tx=0x7f75ec004690 comp rx=0 tx=0).stop 2023-11-26T07:19:42.716 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:42.707+0000 7f75faeaf000 1 -- 172.21.15.143:0/3633363013 shutdown_connections 2023-11-26T07:19:42.716 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:42.707+0000 7f75faeaf000 1 --2- 172.21.15.143:0/3633363013 >> [v2:172.21.15.143:6836/2872033338,v1:172.21.15.143:6837/2872033338] conn(0x55b8e9fab320 0x55b8e9fedf50 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:19:42.716 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:42.707+0000 7f75faeaf000 1 --2- 172.21.15.143:0/3633363013 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55b8e9fdc5b0 0x55b8e9faa910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:19:42.716 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:42.707+0000 7f75faeaf000 1 --2- 172.21.15.143:0/3633363013 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x55b8e9fdbc90 0x55b8e9f97280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:19:42.716 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:42.707+0000 7f75faeaf000 1 --2- 172.21.15.143:0/3633363013 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55b8e9f92320 0x55b8e9f96d40 unknown :-1 s=CLOSED pgs=596 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:19:42.716 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:42.707+0000 7f75d0ff9640 0 client.5904 ms_handle_reset on v2:172.21.15.143:6836/2872033338 2023-11-26T07:19:42.716 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:42.711+0000 7f75faeaf000 1 -- 172.21.15.143:0/3633363013 >> 172.21.15.143:0/3633363013 conn(0x55b8e9f8e2d0 msgr2=0x55b8e9f906c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:19:42.716 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:42.711+0000 7f75faeaf000 1 -- 172.21.15.143:0/3633363013 shutdown_connections 2023-11-26T07:19:42.716 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:42.711+0000 7f75faeaf000 1 -- 172.21.15.143:0/3633363013 wait complete. 2023-11-26T07:19:42.716 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:42.711+0000 7f75faeaf000 20 client.5904 trim_cache size 0 max 16384 2023-11-26T07:19:42.728 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:42.723+0000 7f75faeaf000 1 Processor -- start 2023-11-26T07:19:42.728 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:42.723+0000 7f75faeaf000 1 -- start start 2023-11-26T07:19:42.730 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:42.727+0000 7f75faeaf000 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55b8e9f8e2d0 0x55b8e9fd9b30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:19:42.730 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:42.727+0000 7f75faeaf000 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55b8e9f92320 0x55b8e9fda070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:19:42.730 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:42.727+0000 7f75faeaf000 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x55b8e9fdbc90 0x55b8e9f92700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:19:42.730 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:42.727+0000 7f75faeaf000 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b8e9a9b940 con 0x55b8e9f92320 2023-11-26T07:19:42.730 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:42.727+0000 7f75faeaf000 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b8e9dbbe10 con 0x55b8e9f8e2d0 2023-11-26T07:19:42.730 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:42.727+0000 7f75faeaf000 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x55b8e9de3070 con 0x55b8e9fdbc90 2023-11-26T07:19:42.730 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:42.727+0000 7f75f50c5640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55b8e9f92320 0x55b8e9fda070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:19:42.731 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:42.727+0000 7f75f50c5640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55b8e9f92320 0x55b8e9fda070 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:52366/0 (socket says 172.21.15.143:52366) 2023-11-26T07:19:42.731 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:42.727+0000 7f75f50c5640 1 -- 172.21.15.143:0/1766446467 learned_addr learned my addr 172.21.15.143:0/1766446467 (peer_addr_for_me v2:172.21.15.143:0/0) 2023-11-26T07:19:42.731 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:42.727+0000 7f75f60c7640 1 --2- 172.21.15.143:0/1766446467 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x55b8e9fdbc90 0x55b8e9f92700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:19:42.731 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:42.727+0000 7f75f58c6640 1 --2- 172.21.15.143:0/1766446467 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55b8e9f8e2d0 0x55b8e9fd9b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:19:42.731 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:42.727+0000 7f75f50c5640 1 -- 172.21.15.143:0/1766446467 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x55b8e9fdbc90 msgr2=0x55b8e9f92700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-26T07:19:42.731 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:42.727+0000 7f75f50c5640 1 --2- 172.21.15.143:0/1766446467 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x55b8e9fdbc90 0x55b8e9f92700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:19:42.731 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:42.727+0000 7f75f50c5640 1 -- 172.21.15.143:0/1766446467 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55b8e9f8e2d0 msgr2=0x55b8e9fd9b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-26T07:19:42.731 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:42.727+0000 7f75f50c5640 1 --2- 172.21.15.143:0/1766446467 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55b8e9f8e2d0 0x55b8e9fd9b30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:19:42.731 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:42.727+0000 7f75f50c5640 1 -- 172.21.15.143:0/1766446467 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55b8e9d89490 con 0x55b8e9f92320 2023-11-26T07:19:42.731 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:42.727+0000 7f75f50c5640 1 --2- 172.21.15.143:0/1766446467 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55b8e9f92320 0x55b8e9fda070 secure :-1 s=READY pgs=597 cs=0 l=1 rev1=1 crypto rx=0x7f75e0010ce0 tx=0x7f75e0013000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e92a522b0e4b82bc server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:19:42.731 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:42.727+0000 7f75d3fff640 1 -- 172.21.15.143:0/1766446467 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f75e0010990 con 0x55b8e9f92320 2023-11-26T07:19:42.731 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:42.727+0000 7f75d3fff640 1 -- 172.21.15.143:0/1766446467 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f75e0004030 con 0x55b8e9f92320 2023-11-26T07:19:42.732 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:42.727+0000 7f75d3fff640 1 -- 172.21.15.143:0/1766446467 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f75e00041a0 con 0x55b8e9f92320 2023-11-26T07:19:42.733 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:42.727+0000 7f75faeaf000 1 -- 172.21.15.143:0/1766446467 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55b8e9f92320 msgr2=0x55b8e9fda070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:19:42.733 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:42.727+0000 7f75faeaf000 1 --2- 172.21.15.143:0/1766446467 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55b8e9f92320 0x55b8e9fda070 secure :-1 s=READY pgs=597 cs=0 l=1 rev1=1 crypto rx=0x7f75e0010ce0 tx=0x7f75e0013000 comp rx=0 tx=0).stop 2023-11-26T07:19:42.733 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:42.727+0000 7f75faeaf000 1 -- 172.21.15.143:0/1766446467 shutdown_connections 2023-11-26T07:19:42.734 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:42.727+0000 7f75faeaf000 1 --2- 172.21.15.143:0/1766446467 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x55b8e9fdbc90 0x55b8e9f92700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:19:42.734 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:42.727+0000 7f75faeaf000 1 --2- 172.21.15.143:0/1766446467 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55b8e9f92320 0x55b8e9fda070 unknown :-1 s=CLOSED pgs=597 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:19:42.734 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:42.727+0000 7f75faeaf000 1 --2- 172.21.15.143:0/1766446467 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55b8e9f8e2d0 0x55b8e9fd9b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:19:42.734 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:42.727+0000 7f75faeaf000 1 -- 172.21.15.143:0/1766446467 >> 172.21.15.143:0/1766446467 conn(0x55b8e9f96f10 msgr2=0x55b8e9f906c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:19:42.734 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:42.727+0000 7f75faeaf000 1 -- 172.21.15.143:0/1766446467 shutdown_connections 2023-11-26T07:19:42.734 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:42.727+0000 7f75faeaf000 1 -- 172.21.15.143:0/1766446467 wait complete. 2023-11-26T07:19:42.734 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:42.727+0000 7f75faeaf000 1 -- start start 2023-11-26T07:19:42.734 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:42.727+0000 7f75faeaf000 1 Processor -- start 2023-11-26T07:19:42.734 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:42.731+0000 7f75faeaf000 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x55b8e9f8e2d0 0x55b8e9feefa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:19:42.734 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:42.731+0000 7f75faeaf000 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55b8e9f92320 0x55b8e9fef4e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:19:42.734 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:42.731+0000 7f75faeaf000 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55b8e9fdbc90 0x55b8e9ff99e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:19:42.734 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:42.731+0000 7f75faeaf000 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b8e9f99890 con 0x55b8e9f92320 2023-11-26T07:19:42.735 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:42.731+0000 7f75faeaf000 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b8e9ddffd0 con 0x55b8e9fdbc90 2023-11-26T07:19:42.735 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:42.731+0000 7f75faeaf000 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x55b8e9adb1b0 con 0x55b8e9f8e2d0 2023-11-26T07:19:42.735 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:42.731+0000 7f75faeaf000 10 client.0 _ll_register_callbacks cb 0x55b8e9e30c00 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-11-26T07:19:42.735 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:42.731+0000 7f75f50c5640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55b8e9f92320 0x55b8e9fef4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:19:42.735 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:42.731+0000 7f75f50c5640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55b8e9f92320 0x55b8e9fef4e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:52378/0 (socket says 172.21.15.143:52378) 2023-11-26T07:19:42.735 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:42.731+0000 7f75f50c5640 1 -- 172.21.15.143:0/653224151 learned_addr learned my addr 172.21.15.143:0/653224151 (peer_addr_for_me v2:172.21.15.143:0/0) 2023-11-26T07:19:42.735 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:42.731+0000 7f75f58c6640 1 --2- 172.21.15.143:0/653224151 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x55b8e9f8e2d0 0x55b8e9feefa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:19:42.735 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:42.731+0000 7f75f60c7640 1 --2- 172.21.15.143:0/653224151 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55b8e9fdbc90 0x55b8e9ff99e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:19:42.735 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:42.731+0000 7f75f50c5640 1 -- 172.21.15.143:0/653224151 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x55b8e9f8e2d0 msgr2=0x55b8e9feefa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-26T07:19:42.735 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:42.731+0000 7f75f50c5640 1 --2- 172.21.15.143:0/653224151 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x55b8e9f8e2d0 0x55b8e9feefa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:19:42.735 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:42.731+0000 7f75f50c5640 1 -- 172.21.15.143:0/653224151 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55b8e9fdbc90 msgr2=0x55b8e9ff99e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-26T07:19:42.735 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:42.731+0000 7f75f50c5640 1 --2- 172.21.15.143:0/653224151 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55b8e9fdbc90 0x55b8e9ff99e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:19:42.735 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:42.731+0000 7f75f50c5640 1 -- 172.21.15.143:0/653224151 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f75e001e040 con 0x55b8e9f92320 2023-11-26T07:19:42.735 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:42.731+0000 7f75f60c7640 1 --2- 172.21.15.143:0/653224151 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55b8e9fdbc90 0x55b8e9ff99e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-26T07:19:42.737 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:42.731+0000 7f75f50c5640 1 --2- 172.21.15.143:0/653224151 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55b8e9f92320 0x55b8e9fef4e0 secure :-1 s=READY pgs=598 cs=0 l=1 rev1=1 crypto rx=0x7f75e0010ce0 tx=0x7f75e0004740 comp rx=0 tx=0).ready entity=mon.0 client_cookie=213e41627774b1b9 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:19:42.738 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:42.731+0000 7f75d0ff9640 10 client.0 ms_handle_connect on v2:172.21.15.143:3300/0 2023-11-26T07:19:42.738 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:42.731+0000 7f75d0ff9640 1 -- 172.21.15.143:0/653224151 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f75e00044b0 con 0x55b8e9f92320 2023-11-26T07:19:42.738 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:42.731+0000 7f75d0ff9640 1 -- 172.21.15.143:0/653224151 <== mon.0 v2:172.21.15.143:3300/0 2 ==== osd_map(45..45 src has 1..45) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f75e00059a0 con 0x55b8e9f92320 2023-11-26T07:19:42.738 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:42.731+0000 7f75faeaf000 1 -- 172.21.15.143:0/653224151 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55b8e9de1110 con 0x55b8e9f92320 2023-11-26T07:19:42.738 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:42.731+0000 7f75d0ff9640 1 -- 172.21.15.143:0/653224151 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) v2 ==== 24+0+0 (secure 0 0 0) 0x7f75e0005b60 con 0x55b8e9f92320 2023-11-26T07:19:42.738 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:42.731+0000 7f75faeaf000 10 client.5910 fetch_fsmap learned FSMap version 25 2023-11-26T07:19:42.738 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:42.731+0000 7f75faeaf000 1 -- 172.21.15.143:0/653224151 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=2+}) v3 -- 0x55b8e9adacc0 con 0x55b8e9f92320 2023-11-26T07:19:42.738 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:42.731+0000 7f75d0ff9640 1 -- 172.21.15.143:0/653224151 <== mon.0 v2:172.21.15.143:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x55b8e9de1110 con 0x55b8e9f92320 2023-11-26T07:19:42.738 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:42.731+0000 7f75d0ff9640 1 -- 172.21.15.143:0/653224151 <== mon.0 v2:172.21.15.143:3300/0 5 ==== fsmap(e 25) v1 ==== 3294+0+0 (secure 0 0 0) 0x7f75e0021040 con 0x55b8e9f92320 2023-11-26T07:19:42.738 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:42.731+0000 7f75faeaf000 10 client.5910 fetch_fsmap finished waiting for FSMap version 25 2023-11-26T07:19:42.738 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:42.731+0000 7f75faeaf000 10 client.5910 resolve_mds: resolved name 'c' to daemon mds.c 2023-11-26T07:19:42.739 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:42.731+0000 7f75faeaf000 20 client.5910 populate_metadata read hostname 'smithi143' 2023-11-26T07:19:42.739 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:42.731+0000 7f75faeaf000 1 --2- 172.21.15.143:0/653224151 >> [v2:172.21.15.143:6836/2872033338,v1:172.21.15.143:6837/2872033338] conn(0x55b8e9fdc5b0 0x55b8ea044e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:19:42.739 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:42.731+0000 7f75faeaf000 4 client.5910 mds_command: new command op to 5748 tid=0 multi_id=0 [{"prefix": "status"}] 2023-11-26T07:19:42.739 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:42.731+0000 7f75faeaf000 1 -- 172.21.15.143:0/653224151 --> [v2:172.21.15.143:6836/2872033338,v1:172.21.15.143:6837/2872033338] -- command(tid 0: {"prefix": "status"}) v1 -- 0x55b8e9e109a0 con 0x55b8e9fdc5b0 2023-11-26T07:19:42.739 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:42.731+0000 7f75f58c6640 1 --2- 172.21.15.143:0/653224151 >> [v2:172.21.15.143:6836/2872033338,v1:172.21.15.143:6837/2872033338] conn(0x55b8e9fdc5b0 0x55b8ea044e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:19:42.739 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:42.731+0000 7f75d0ff9640 10 client.5910 ms_handle_connect on v2:172.21.15.143:6836/2872033338 2023-11-26T07:19:42.739 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:42.731+0000 7f75f58c6640 1 --2- 172.21.15.143:0/653224151 >> [v2:172.21.15.143:6836/2872033338,v1:172.21.15.143:6837/2872033338] conn(0x55b8e9fdc5b0 0x55b8ea044e50 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=267ee3360ef2fa65 server_cookie=aaf7fffb53b02c1d in_seq=0 out_seq=0 2023-11-26T07:19:42.739 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:42.731+0000 7f75d0ff9640 1 -- 172.21.15.143:0/653224151 <== mds.0 v2:172.21.15.143:6836/2872033338 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+331 (crc 0 0 0) 0x55b8e9e109a0 con 0x55b8e9fdc5b0 2023-11-26T07:19:42.739 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:42.731+0000 7f75d0ff9640 10 client.5910 handle_command_reply: tid=0 2023-11-26T07:19:42.739 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:42.731+0000 7f75faeaf000 1 client.5910 shutdown 2023-11-26T07:19:42.739 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:42.731+0000 7f75faeaf000 1 -- 172.21.15.143:0/653224151 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55b8e9f92320 msgr2=0x55b8e9fef4e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:19:42.739 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:42.731+0000 7f75faeaf000 1 --2- 172.21.15.143:0/653224151 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55b8e9f92320 0x55b8e9fef4e0 secure :-1 s=READY pgs=598 cs=0 l=1 rev1=1 crypto rx=0x7f75e0010ce0 tx=0x7f75e0004740 comp rx=0 tx=0).stop 2023-11-26T07:19:42.739 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:42.731+0000 7f75faeaf000 1 -- 172.21.15.143:0/653224151 shutdown_connections 2023-11-26T07:19:42.740 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:42.731+0000 7f75faeaf000 1 --2- 172.21.15.143:0/653224151 >> [v2:172.21.15.143:6836/2872033338,v1:172.21.15.143:6837/2872033338] conn(0x55b8e9fdc5b0 0x55b8ea044e50 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:19:42.740 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:42.731+0000 7f75faeaf000 1 --2- 172.21.15.143:0/653224151 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55b8e9fdbc90 0x55b8e9ff99e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:19:42.740 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:42.731+0000 7f75faeaf000 1 --2- 172.21.15.143:0/653224151 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55b8e9f92320 0x55b8e9fef4e0 unknown :-1 s=CLOSED pgs=598 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:19:42.740 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:42.731+0000 7f75faeaf000 1 --2- 172.21.15.143:0/653224151 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x55b8e9f8e2d0 0x55b8e9feefa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:19:42.740 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:42.731+0000 7f75d0ff9640 0 client.5910 ms_handle_reset on v2:172.21.15.143:6836/2872033338 2023-11-26T07:19:42.740 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:42.731+0000 7f75faeaf000 1 -- 172.21.15.143:0/653224151 >> 172.21.15.143:0/653224151 conn(0x55b8e9f96f10 msgr2=0x55b8e9f90f50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:19:42.740 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:42.731+0000 7f75faeaf000 1 -- 172.21.15.143:0/653224151 shutdown_connections 2023-11-26T07:19:42.740 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:42.731+0000 7f75faeaf000 1 -- 172.21.15.143:0/653224151 wait complete. 2023-11-26T07:19:42.740 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:42.731+0000 7f75faeaf000 20 client.5910 trim_cache size 0 max 16384 2023-11-26T07:19:42.740 INFO:teuthology.orchestra.run.smithi143.stdout:{ 2023-11-26T07:19:42.741 INFO:teuthology.orchestra.run.smithi143.stdout: "cluster_fsid": "f5033060-2469-4a07-95fe-2b76f5404bc1", 2023-11-26T07:19:42.741 INFO:teuthology.orchestra.run.smithi143.stdout: "whoami": 0, 2023-11-26T07:19:42.741 INFO:teuthology.orchestra.run.smithi143.stdout: "id": 5748, 2023-11-26T07:19:42.741 INFO:teuthology.orchestra.run.smithi143.stdout: "want_state": "up:active", 2023-11-26T07:19:42.741 INFO:teuthology.orchestra.run.smithi143.stdout: "state": "up:active", 2023-11-26T07:19:42.741 INFO:teuthology.orchestra.run.smithi143.stdout: "fs_name": "cephfs", 2023-11-26T07:19:42.741 INFO:teuthology.orchestra.run.smithi143.stdout: "rank_uptime": 10.597803692999999, 2023-11-26T07:19:42.741 INFO:teuthology.orchestra.run.smithi143.stdout: "mdsmap_epoch": 25, 2023-11-26T07:19:42.741 INFO:teuthology.orchestra.run.smithi143.stdout: "osdmap_epoch": 45, 2023-11-26T07:19:42.741 INFO:teuthology.orchestra.run.smithi143.stdout: "osdmap_epoch_barrier": 45, 2023-11-26T07:19:42.741 INFO:teuthology.orchestra.run.smithi143.stdout: "uptime": 16.461301090999999 2023-11-26T07:19:42.741 INFO:teuthology.orchestra.run.smithi143.stdout:} 2023-11-26T07:19:42.742 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:42.735+0000 7f75f7b2a640 1 -- 172.21.15.143:0/2068237987 >> [v2:172.21.15.143:6832/2150557544,v1:172.21.15.143:6833/2150557544] conn(0x7f75d4066620 msgr2=0x7f75d4068ae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:19:42.742 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:42.735+0000 7f75f7b2a640 1 --2- 172.21.15.143:0/2068237987 >> [v2:172.21.15.143:6832/2150557544,v1:172.21.15.143:6833/2150557544] conn(0x7f75d4066620 0x7f75d4068ae0 secure :-1 s=READY pgs=298 cs=0 l=1 rev1=1 crypto rx=0x7f75f0162250 tx=0x7f75e0008040 comp rx=0 tx=0).stop 2023-11-26T07:19:42.742 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:42.735+0000 7f75f7b2a640 1 -- 172.21.15.143:0/2068237987 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f75f0070e00 msgr2=0x7f75f0142cf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:19:42.742 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:42.735+0000 7f75f7b2a640 1 --2- 172.21.15.143:0/2068237987 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f75f0070e00 0x7f75f0142cf0 secure :-1 s=READY pgs=594 cs=0 l=1 rev1=1 crypto rx=0x7f75ec00b410 tx=0x7f75ec00b8d0 comp rx=0 tx=0).stop 2023-11-26T07:19:42.743 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:42.735+0000 7f75f7b2a640 1 -- 172.21.15.143:0/2068237987 shutdown_connections 2023-11-26T07:19:42.743 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:42.735+0000 7f75f7b2a640 1 --2- 172.21.15.143:0/2068237987 >> [v2:172.21.15.143:6832/2150557544,v1:172.21.15.143:6833/2150557544] conn(0x7f75d4066620 0x7f75d4068ae0 unknown :-1 s=CLOSED pgs=298 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:19:42.743 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:42.735+0000 7f75f7b2a640 1 --2- 172.21.15.143:0/2068237987 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f75f014b1e0 0x7f75f015c830 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:19:42.743 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:42.735+0000 7f75f7b2a640 1 --2- 172.21.15.143:0/2068237987 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f75f00cf990 0x7f75f015c2f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:19:42.743 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:42.735+0000 7f75f7b2a640 1 --2- 172.21.15.143:0/2068237987 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f75f0070e00 0x7f75f0142cf0 unknown :-1 s=CLOSED pgs=594 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:19:42.743 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:42.735+0000 7f75f7b2a640 1 -- 172.21.15.143:0/2068237987 >> 172.21.15.143:0/2068237987 conn(0x7f75f006a890 msgr2=0x7f75f00b14a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:19:42.743 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:42.735+0000 7f75f7b2a640 1 -- 172.21.15.143:0/2068237987 shutdown_connections 2023-11-26T07:19:42.743 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:42.735+0000 7f75f7b2a640 1 -- 172.21.15.143:0/2068237987 wait complete. 2023-11-26T07:19:42.755 DEBUG:teuthology.orchestra.run.smithi143:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2023-11-26T07:19:42.893 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:42.891+0000 7fee3b59e640 1 Processor -- start 2023-11-26T07:19:42.893 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:42.891+0000 7fee3b59e640 1 -- start start 2023-11-26T07:19:42.895 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:42.891+0000 7fee3b59e640 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7fee3c15e0f0 0x7fee3c0ce030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:19:42.895 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:42.891+0000 7fee3b59e640 1 -- --> v1:172.21.15.143:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fee3c15e860 con 0x7fee3c0b1e10 2023-11-26T07:19:42.895 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:42.891+0000 7fee3b59e640 1 -- --> v1:172.21.15.188:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fee3c0cdba0 con 0x7fee3c143810 2023-11-26T07:19:42.895 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:42.891+0000 7fee3b59e640 1 -- --> v2:172.21.15.143:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fee3c143bf0 con 0x7fee3c15e0f0 2023-11-26T07:19:42.895 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:42.891+0000 7fee3ad9d640 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7fee3c15e0f0 0x7fee3c0ce030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:19:42.895 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:42.891+0000 7fee3ad9d640 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7fee3c15e0f0 0x7fee3c0ce030 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:52380/0 (socket says 172.21.15.143:52380) 2023-11-26T07:19:42.895 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:42.891+0000 7fee3ad9d640 1 -- 172.21.15.143:0/3986957787 learned_addr learned my addr 172.21.15.143:0/3986957787 (peer_addr_for_me v2:172.21.15.143:0/0) 2023-11-26T07:19:42.896 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:42.891+0000 7fee3ad9d640 1 -- 172.21.15.143:0/3986957787 >> v1:172.21.15.188:6789/0 conn(0x7fee3c143810 legacy=0x7fee3c072140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:19:42.896 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:42.891+0000 7fee3ad9d640 1 -- 172.21.15.143:0/3986957787 >> v1:172.21.15.143:6789/0 conn(0x7fee3c0b1e10 legacy=0x7fee3c15d9e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:19:42.896 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:42.891+0000 7fee3ad9d640 1 -- 172.21.15.143:0/3986957787 --> v2:172.21.15.143:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fee3c0714f0 con 0x7fee3c15e0f0 2023-11-26T07:19:42.896 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:42.891+0000 7fee3959a640 1 -- 172.21.15.143:0/3986957787 <== mon.0 v1:172.21.15.143:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1586226490 0 0) 0x7fee3c15e860 con 0x7fee3c0b1e10 2023-11-26T07:19:42.898 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:42.895+0000 7fee3ad9d640 1 --2- 172.21.15.143:0/3986957787 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fee3c15e0f0 0x7fee3c0ce030 secure :-1 s=READY pgs=600 cs=0 l=1 rev1=1 crypto rx=0x7fee3400d150 tx=0x7fee3401ebe0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=787580f0f2f45c78 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:19:42.898 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:42.895+0000 7fee3959a640 1 -- 172.21.15.143:0/3986957787 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fee3400b2d0 con 0x7fee3c15e0f0 2023-11-26T07:19:42.898 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:42.895+0000 7fee3959a640 1 -- 172.21.15.143:0/3986957787 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fee3400b430 con 0x7fee3c15e0f0 2023-11-26T07:19:42.898 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:42.895+0000 7fee3959a640 1 -- 172.21.15.143:0/3986957787 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fee340a7bf0 con 0x7fee3c15e0f0 2023-11-26T07:19:42.899 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:42.895+0000 7fee3b59e640 1 -- 172.21.15.143:0/3986957787 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fee3c15e0f0 msgr2=0x7fee3c0ce030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:19:42.899 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:42.895+0000 7fee3b59e640 1 --2- 172.21.15.143:0/3986957787 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fee3c15e0f0 0x7fee3c0ce030 secure :-1 s=READY pgs=600 cs=0 l=1 rev1=1 crypto rx=0x7fee3400d150 tx=0x7fee3401ebe0 comp rx=0 tx=0).stop 2023-11-26T07:19:42.899 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:42.895+0000 7fee3b59e640 1 -- 172.21.15.143:0/3986957787 shutdown_connections 2023-11-26T07:19:42.899 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:42.895+0000 7fee3b59e640 1 --2- 172.21.15.143:0/3986957787 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fee3c15e0f0 0x7fee3c0ce030 unknown :-1 s=CLOSED pgs=600 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:19:42.900 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:42.895+0000 7fee3b59e640 1 -- 172.21.15.143:0/3986957787 >> 172.21.15.143:0/3986957787 conn(0x7fee3c06a890 msgr2=0x7fee3c06aca0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:19:42.900 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:42.895+0000 7fee3b59e640 1 -- 172.21.15.143:0/3986957787 shutdown_connections 2023-11-26T07:19:42.900 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:42.895+0000 7fee3b59e640 1 -- 172.21.15.143:0/3986957787 wait complete. 2023-11-26T07:19:42.901 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:42.899+0000 7fee3b59e640 1 Processor -- start 2023-11-26T07:19:42.901 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:42.899+0000 7fee3b59e640 1 -- start start 2023-11-26T07:19:42.903 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:42.899+0000 7fee3b59e640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fee3c0b1e10 0x7fee3c140000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:19:42.903 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:42.899+0000 7fee3b59e640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fee3c143810 0x7fee3c1531a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:19:42.903 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:42.899+0000 7fee3b59e640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fee3c15e0f0 0x7fee3c158ec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:19:42.903 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:42.899+0000 7fee3b59e640 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fee3c13bfe0 con 0x7fee3c0b1e10 2023-11-26T07:19:42.903 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:42.899+0000 7fee3b59e640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fee3c13b3d0 con 0x7fee3c143810 2023-11-26T07:19:42.903 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:42.899+0000 7fee3b59e640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fee3c13b6b0 con 0x7fee3c15e0f0 2023-11-26T07:19:42.903 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:42.899+0000 7fee3a59c640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fee3c0b1e10 0x7fee3c140000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:19:42.903 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:42.899+0000 7fee3a59c640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fee3c0b1e10 0x7fee3c140000 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:52386/0 (socket says 172.21.15.143:52386) 2023-11-26T07:19:42.903 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:42.899+0000 7fee3a59c640 1 -- 172.21.15.143:0/1959521100 learned_addr learned my addr 172.21.15.143:0/1959521100 (peer_addr_for_me v2:172.21.15.143:0/0) 2023-11-26T07:19:42.904 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:42.899+0000 7fee3ad9d640 1 --2- 172.21.15.143:0/1959521100 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fee3c15e0f0 0x7fee3c158ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:19:42.904 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:42.899+0000 7fee39d9b640 1 --2- 172.21.15.143:0/1959521100 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fee3c143810 0x7fee3c1531a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:19:42.904 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:42.899+0000 7fee3a59c640 1 -- 172.21.15.143:0/1959521100 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fee3c15e0f0 msgr2=0x7fee3c158ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:19:42.904 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:42.899+0000 7fee3a59c640 1 --2- 172.21.15.143:0/1959521100 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fee3c15e0f0 0x7fee3c158ec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:19:42.904 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:42.899+0000 7fee3a59c640 1 -- 172.21.15.143:0/1959521100 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fee3c143810 msgr2=0x7fee3c1531a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:19:42.904 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:42.899+0000 7fee3a59c640 1 --2- 172.21.15.143:0/1959521100 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fee3c143810 0x7fee3c1531a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:19:42.904 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:42.899+0000 7fee3a59c640 1 -- 172.21.15.143:0/1959521100 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fee3400bd40 con 0x7fee3c0b1e10 2023-11-26T07:19:42.904 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:42.899+0000 7fee3a59c640 1 --2- 172.21.15.143:0/1959521100 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fee3c0b1e10 0x7fee3c140000 secure :-1 s=READY pgs=601 cs=0 l=1 rev1=1 crypto rx=0x7fee2c00c500 tx=0x7fee2c00c9c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:19:42.905 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:42.899+0000 7fee2affd640 1 -- 172.21.15.143:0/1959521100 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fee2c005220 con 0x7fee3c0b1e10 2023-11-26T07:19:42.905 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:42.899+0000 7fee2affd640 1 -- 172.21.15.143:0/1959521100 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fee2c005380 con 0x7fee3c0b1e10 2023-11-26T07:19:42.905 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:42.899+0000 7fee2affd640 1 -- 172.21.15.143:0/1959521100 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fee2c007270 con 0x7fee3c0b1e10 2023-11-26T07:19:42.905 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:42.899+0000 7fee3b59e640 1 -- 172.21.15.143:0/1959521100 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fee3c13b990 con 0x7fee3c0b1e10 2023-11-26T07:19:42.905 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:42.899+0000 7fee3b59e640 1 -- 172.21.15.143:0/1959521100 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fee3c140540 con 0x7fee3c0b1e10 2023-11-26T07:19:42.905 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:42.899+0000 7fee3b59e640 1 -- 172.21.15.143:0/1959521100 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fee3c067500 con 0x7fee3c0b1e10 2023-11-26T07:19:42.906 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:42.903+0000 7fee2affd640 1 -- 172.21.15.143:0/1959521100 <== mon.0 v2:172.21.15.143:3300/0 4 ==== mgrmap(e 7) v1 ==== 86005+0+0 (secure 0 0 0) 0x7fee2c0073d0 con 0x7fee3c0b1e10 2023-11-26T07:19:42.906 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:42.903+0000 7fee2affd640 1 --2- 172.21.15.143:0/1959521100 >> [v2:172.21.15.143:6832/2150557544,v1:172.21.15.143:6833/2150557544] conn(0x7fee1c066620 0x7fee1c068ae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:19:42.906 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:42.903+0000 7fee2affd640 1 -- 172.21.15.143:0/1959521100 <== mon.0 v2:172.21.15.143:3300/0 5 ==== osd_map(45..45 src has 1..45) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fee2c0839b0 con 0x7fee3c0b1e10 2023-11-26T07:19:42.907 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:42.903+0000 7fee39d9b640 1 --2- 172.21.15.143:0/1959521100 >> [v2:172.21.15.143:6832/2150557544,v1:172.21.15.143:6833/2150557544] conn(0x7fee1c066620 0x7fee1c068ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:19:42.907 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:42.903+0000 7fee39d9b640 1 --2- 172.21.15.143:0/1959521100 >> [v2:172.21.15.143:6832/2150557544,v1:172.21.15.143:6833/2150557544] conn(0x7fee1c066620 0x7fee1c068ae0 secure :-1 s=READY pgs=299 cs=0 l=1 rev1=1 crypto rx=0x7fee3c13b000 tx=0x7fee30009040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:19:42.910 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:42.907+0000 7fee2affd640 1 -- 172.21.15.143:0/1959521100 <== mon.0 v2:172.21.15.143:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172649 (secure 0 0 0) 0x7fee2c050bd0 con 0x7fee3c0b1e10 2023-11-26T07:19:43.091 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:43.087+0000 7fee3b59e640 1 -- 172.21.15.143:0/1959521100 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "fs ls"} v 0) v1 -- 0x7fee3c0b4010 con 0x7fee3c0b1e10 2023-11-26T07:19:43.092 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:43.087+0000 7fee2affd640 1 -- 172.21.15.143:0/1959521100 <== mon.0 v2:172.21.15.143:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v25) v1 ==== 53+0+73 (secure 0 0 0) 0x7fee2c0541f0 con 0x7fee3c0b1e10 2023-11-26T07:19:43.092 INFO:teuthology.orchestra.run.smithi143.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2023-11-26T07:19:43.094 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:43.091+0000 7fee3b59e640 1 -- 172.21.15.143:0/1959521100 >> [v2:172.21.15.143:6832/2150557544,v1:172.21.15.143:6833/2150557544] conn(0x7fee1c066620 msgr2=0x7fee1c068ae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:19:43.094 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:43.091+0000 7fee3b59e640 1 --2- 172.21.15.143:0/1959521100 >> [v2:172.21.15.143:6832/2150557544,v1:172.21.15.143:6833/2150557544] conn(0x7fee1c066620 0x7fee1c068ae0 secure :-1 s=READY pgs=299 cs=0 l=1 rev1=1 crypto rx=0x7fee3c13b000 tx=0x7fee30009040 comp rx=0 tx=0).stop 2023-11-26T07:19:43.094 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:43.091+0000 7fee3b59e640 1 -- 172.21.15.143:0/1959521100 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fee3c0b1e10 msgr2=0x7fee3c140000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:19:43.094 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:43.091+0000 7fee3b59e640 1 --2- 172.21.15.143:0/1959521100 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fee3c0b1e10 0x7fee3c140000 secure :-1 s=READY pgs=601 cs=0 l=1 rev1=1 crypto rx=0x7fee2c00c500 tx=0x7fee2c00c9c0 comp rx=0 tx=0).stop 2023-11-26T07:19:43.094 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:43.091+0000 7fee3b59e640 1 -- 172.21.15.143:0/1959521100 shutdown_connections 2023-11-26T07:19:43.094 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:43.091+0000 7fee3b59e640 1 --2- 172.21.15.143:0/1959521100 >> [v2:172.21.15.143:6832/2150557544,v1:172.21.15.143:6833/2150557544] conn(0x7fee1c066620 0x7fee1c068ae0 unknown :-1 s=CLOSED pgs=299 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:19:43.094 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:43.091+0000 7fee3b59e640 1 --2- 172.21.15.143:0/1959521100 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fee3c15e0f0 0x7fee3c158ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:19:43.095 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:43.091+0000 7fee3b59e640 1 --2- 172.21.15.143:0/1959521100 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fee3c143810 0x7fee3c1531a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:19:43.095 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:43.091+0000 7fee3b59e640 1 --2- 172.21.15.143:0/1959521100 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fee3c0b1e10 0x7fee3c140000 unknown :-1 s=CLOSED pgs=601 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:19:43.095 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:43.091+0000 7fee3b59e640 1 -- 172.21.15.143:0/1959521100 >> 172.21.15.143:0/1959521100 conn(0x7fee3c06a890 msgr2=0x7fee3c0b3450 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:19:43.095 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:43.091+0000 7fee3b59e640 1 -- 172.21.15.143:0/1959521100 shutdown_connections 2023-11-26T07:19:43.095 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:43.091+0000 7fee3b59e640 1 -- 172.21.15.143:0/1959521100 wait complete. 2023-11-26T07:19:43.113 INFO:tasks.cephfs.mount:Mounting default Ceph FS; just confirmed its presence on cluster 2023-11-26T07:19:43.113 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2023-11-26T07:19:43.113 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi143.front.sepia.ceph.com 2023-11-26T07:19:43.113 INFO:tasks.cephfs.mount:self.client.name = client.0 2023-11-26T07:19:43.113 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2023-11-26T07:19:43.113 INFO:tasks.cephfs.mount:self.cephfs_name = None 2023-11-26T07:19:43.113 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2023-11-26T07:19:43.113 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2023-11-26T07:19:43.113 INFO:tasks.cephfs.mount:Setting the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2023-11-26T07:19:43.113 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-26T07:19:43.114 DEBUG:teuthology.orchestra.run.smithi143:> ip addr 2023-11-26T07:19:43.123 INFO:teuthology.orchestra.run.smithi143.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2023-11-26T07:19:43.123 INFO:teuthology.orchestra.run.smithi143.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2023-11-26T07:19:43.124 INFO:teuthology.orchestra.run.smithi143.stdout: inet 127.0.0.1/8 scope host lo 2023-11-26T07:19:43.124 INFO:teuthology.orchestra.run.smithi143.stdout: valid_lft forever preferred_lft forever 2023-11-26T07:19:43.124 INFO:teuthology.orchestra.run.smithi143.stdout: inet6 ::1/128 scope host 2023-11-26T07:19:43.124 INFO:teuthology.orchestra.run.smithi143.stdout: valid_lft forever preferred_lft forever 2023-11-26T07:19:43.124 INFO:teuthology.orchestra.run.smithi143.stdout:2: eno1: mtu 1500 qdisc noop state DOWN group default qlen 1000 2023-11-26T07:19:43.124 INFO:teuthology.orchestra.run.smithi143.stdout: link/ether 0c:c4:7a:d9:3b:94 brd ff:ff:ff:ff:ff:ff 2023-11-26T07:19:43.124 INFO:teuthology.orchestra.run.smithi143.stdout: altname enp7s0f0 2023-11-26T07:19:43.124 INFO:teuthology.orchestra.run.smithi143.stdout:3: eno2: mtu 1500 qdisc noop state DOWN group default qlen 1000 2023-11-26T07:19:43.124 INFO:teuthology.orchestra.run.smithi143.stdout: link/ether 0c:c4:7a:d9:3b:95 brd ff:ff:ff:ff:ff:ff 2023-11-26T07:19:43.124 INFO:teuthology.orchestra.run.smithi143.stdout: altname enp7s0f1 2023-11-26T07:19:43.124 INFO:teuthology.orchestra.run.smithi143.stdout:4: enp3s0f0: mtu 1500 qdisc noop state DOWN group default qlen 1000 2023-11-26T07:19:43.125 INFO:teuthology.orchestra.run.smithi143.stdout: link/ether 0c:c4:7a:88:77:82 brd ff:ff:ff:ff:ff:ff 2023-11-26T07:19:43.125 INFO:teuthology.orchestra.run.smithi143.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2023-11-26T07:19:43.125 INFO:teuthology.orchestra.run.smithi143.stdout: link/ether 0c:c4:7a:88:77:83 brd ff:ff:ff:ff:ff:ff 2023-11-26T07:19:43.125 INFO:teuthology.orchestra.run.smithi143.stdout: inet 172.21.15.143/20 metric 100 brd 172.21.15.255 scope global dynamic enp3s0f1 2023-11-26T07:19:43.125 INFO:teuthology.orchestra.run.smithi143.stdout: valid_lft 42291sec preferred_lft 42291sec 2023-11-26T07:19:43.125 INFO:teuthology.orchestra.run.smithi143.stdout: inet6 fe80::ec4:7aff:fe88:7783/64 scope link 2023-11-26T07:19:43.125 INFO:teuthology.orchestra.run.smithi143.stdout: valid_lft forever preferred_lft forever 2023-11-26T07:19:43.125 INFO:teuthology.orchestra.run.smithi143.stdout:6: docker0: mtu 1500 qdisc noqueue state DOWN group default 2023-11-26T07:19:43.125 INFO:teuthology.orchestra.run.smithi143.stdout: link/ether 02:42:c4:b4:2f:eb brd ff:ff:ff:ff:ff:ff 2023-11-26T07:19:43.125 INFO:teuthology.orchestra.run.smithi143.stdout: inet 172.17.0.1/16 brd 172.17.255.255 scope global docker0 2023-11-26T07:19:43.125 INFO:teuthology.orchestra.run.smithi143.stdout: valid_lft forever preferred_lft forever 2023-11-26T07:19:43.125 INFO:teuthology.orchestra.run.smithi143.stdout:7: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2023-11-26T07:19:43.125 INFO:teuthology.orchestra.run.smithi143.stdout: link/ether 5e:1f:84:6d:79:ae brd ff:ff:ff:ff:ff:ff 2023-11-26T07:19:43.126 INFO:teuthology.orchestra.run.smithi143.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2023-11-26T07:19:43.126 INFO:teuthology.orchestra.run.smithi143.stdout: valid_lft forever preferred_lft forever 2023-11-26T07:19:43.126 INFO:teuthology.orchestra.run.smithi143.stdout: inet6 fe80::5c1f:84ff:fe6d:79ae/64 scope link 2023-11-26T07:19:43.126 INFO:teuthology.orchestra.run.smithi143.stdout: valid_lft forever preferred_lft forever 2023-11-26T07:19:43.126 INFO:teuthology.orchestra.run.smithi143.stdout:8: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2023-11-26T07:19:43.126 INFO:teuthology.orchestra.run.smithi143.stdout: link/ether 0e:5c:f6:1f:82:2f brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.0 2023-11-26T07:19:43.126 INFO:teuthology.orchestra.run.smithi143.stdout: inet6 fe80::c5c:f6ff:fe1f:822f/64 scope link 2023-11-26T07:19:43.126 INFO:teuthology.orchestra.run.smithi143.stdout: valid_lft forever preferred_lft forever 2023-11-26T07:19:43.126 INFO:teuthology.orchestra.run.smithi143.stdout:9: brx.1@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2023-11-26T07:19:43.126 INFO:teuthology.orchestra.run.smithi143.stdout: link/ether c6:a6:3d:69:d0:db brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.2 2023-11-26T07:19:43.126 INFO:teuthology.orchestra.run.smithi143.stdout: inet6 fe80::c4a6:3dff:fe69:d0db/64 scope link 2023-11-26T07:19:43.126 INFO:teuthology.orchestra.run.smithi143.stdout: valid_lft forever preferred_lft forever 2023-11-26T07:19:43.127 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-26T07:19:43.127 DEBUG:teuthology.orchestra.run.smithi143:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2023-11-26T07:19:43.182 INFO:teuthology.orchestra.run.smithi143.stdout:1 2023-11-26T07:19:43.184 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-26T07:19:43.185 DEBUG:teuthology.orchestra.run.smithi143:> route 2023-11-26T07:19:43.239 INFO:teuthology.orchestra.run.smithi143.stdout:Kernel IP routing table 2023-11-26T07:19:43.239 INFO:teuthology.orchestra.run.smithi143.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2023-11-26T07:19:43.239 INFO:teuthology.orchestra.run.smithi143.stdout:default _gateway 0.0.0.0 UG 0 0 0 enp3s0f1 2023-11-26T07:19:43.239 INFO:teuthology.orchestra.run.smithi143.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2023-11-26T07:19:43.239 INFO:teuthology.orchestra.run.smithi143.stdout:172.17.0.0 0.0.0.0 255.255.0.0 U 0 0 0 docker0 2023-11-26T07:19:43.239 INFO:teuthology.orchestra.run.smithi143.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 enp3s0f1 2023-11-26T07:19:43.239 INFO:teuthology.orchestra.run.smithi143.stdout:ns1.front.sepia 0.0.0.0 255.255.255.255 UH 100 0 0 enp3s0f1 2023-11-26T07:19:43.239 INFO:teuthology.orchestra.run.smithi143.stdout:172.21.0.2 0.0.0.0 255.255.255.255 UH 100 0 0 enp3s0f1 2023-11-26T07:19:43.240 INFO:teuthology.orchestra.run.smithi143.stdout:_gateway 0.0.0.0 255.255.255.255 UH 100 0 0 enp3s0f1 2023-11-26T07:19:43.240 INFO:teuthology.orchestra.run.smithi143.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2023-11-26T07:19:43.240 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-26T07:19:43.240 DEBUG:teuthology.orchestra.run.smithi143:> (cd / && exec bash -c ' 2023-11-26T07:19:43.240 DEBUG:teuthology.orchestra.run.smithi143:> set -e 2023-11-26T07:19:43.240 DEBUG:teuthology.orchestra.run.smithi143:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2023-11-26T07:19:43.240 DEBUG:teuthology.orchestra.run.smithi143:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2023-11-26T07:19:43.240 DEBUG:teuthology.orchestra.run.smithi143:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2023-11-26T07:19:43.240 DEBUG:teuthology.orchestra.run.smithi143:> ') 2023-11-26T07:19:43.334 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-26T07:19:43.334 DEBUG:teuthology.orchestra.run.smithi143:> ip netns list 2023-11-26T07:19:43.344 INFO:teuthology.orchestra.run.smithi143.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2023-11-26T07:19:43.344 INFO:teuthology.orchestra.run.smithi143.stdout:ceph-ns--home-ubuntu-cephtest-mnt.2 (id: 1) 2023-11-26T07:19:43.344 INFO:tasks.cephfs.mount:Resuming the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2023-11-26T07:19:43.344 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-26T07:19:43.344 DEBUG:teuthology.orchestra.run.smithi143:> sudo ip link set brx.0 up 2023-11-26T07:19:43.404 INFO:tasks.cephfs.fuse_mount:Client client.0 config is {} 2023-11-26T07:19:43.404 INFO:teuthology.orchestra.run:Running command with timeout 60 2023-11-26T07:19:43.404 DEBUG:teuthology.orchestra.run.smithi143:> mkdir -p -v /home/ubuntu/cephtest/mnt.0 2023-11-26T07:19:43.452 INFO:teuthology.orchestra.run.smithi143.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.0' 2023-11-26T07:19:43.453 INFO:teuthology.orchestra.run:Running command with timeout 60 2023-11-26T07:19:43.453 DEBUG:teuthology.orchestra.run.smithi143:> chmod 0000 /home/ubuntu/cephtest/mnt.0 2023-11-26T07:19:43.508 DEBUG:teuthology.orchestra.run.smithi143:> sudo modprobe fuse 2023-11-26T07:19:43.569 DEBUG:teuthology.orchestra.run.smithi143:> cat /proc/self/mounts | awk '{print $2}' 2023-11-26T07:19:43.626 INFO:teuthology.orchestra.run.smithi143.stdout:/sys 2023-11-26T07:19:43.626 INFO:teuthology.orchestra.run.smithi143.stdout:/proc 2023-11-26T07:19:43.626 INFO:teuthology.orchestra.run.smithi143.stdout:/dev 2023-11-26T07:19:43.626 INFO:teuthology.orchestra.run.smithi143.stdout:/dev/pts 2023-11-26T07:19:43.627 INFO:teuthology.orchestra.run.smithi143.stdout:/run 2023-11-26T07:19:43.627 INFO:teuthology.orchestra.run.smithi143.stdout:/ 2023-11-26T07:19:43.627 INFO:teuthology.orchestra.run.smithi143.stdout:/sys/kernel/security 2023-11-26T07:19:43.627 INFO:teuthology.orchestra.run.smithi143.stdout:/dev/shm 2023-11-26T07:19:43.627 INFO:teuthology.orchestra.run.smithi143.stdout:/run/lock 2023-11-26T07:19:43.627 INFO:teuthology.orchestra.run.smithi143.stdout:/sys/fs/cgroup 2023-11-26T07:19:43.627 INFO:teuthology.orchestra.run.smithi143.stdout:/sys/fs/pstore 2023-11-26T07:19:43.627 INFO:teuthology.orchestra.run.smithi143.stdout:/sys/fs/bpf 2023-11-26T07:19:43.627 INFO:teuthology.orchestra.run.smithi143.stdout:/proc/sys/fs/binfmt_misc 2023-11-26T07:19:43.627 INFO:teuthology.orchestra.run.smithi143.stdout:/dev/hugepages 2023-11-26T07:19:43.628 INFO:teuthology.orchestra.run.smithi143.stdout:/dev/mqueue 2023-11-26T07:19:43.628 INFO:teuthology.orchestra.run.smithi143.stdout:/sys/kernel/debug 2023-11-26T07:19:43.628 INFO:teuthology.orchestra.run.smithi143.stdout:/sys/kernel/tracing 2023-11-26T07:19:43.628 INFO:teuthology.orchestra.run.smithi143.stdout:/proc/fs/nfsd 2023-11-26T07:19:43.628 INFO:teuthology.orchestra.run.smithi143.stdout:/sys/fs/fuse/connections 2023-11-26T07:19:43.628 INFO:teuthology.orchestra.run.smithi143.stdout:/sys/kernel/config 2023-11-26T07:19:43.628 INFO:teuthology.orchestra.run.smithi143.stdout:/run/credentials/systemd-sysusers.service 2023-11-26T07:19:43.628 INFO:teuthology.orchestra.run.smithi143.stdout:/run/rpc_pipefs 2023-11-26T07:19:43.628 INFO:teuthology.orchestra.run.smithi143.stdout:/run/user/1000 2023-11-26T07:19:43.628 INFO:teuthology.orchestra.run.smithi143.stdout:/var/lib/ceph 2023-11-26T07:19:43.629 INFO:teuthology.orchestra.run.smithi143.stdout:/var/lib/ceph/osd/ceph-0 2023-11-26T07:19:43.629 INFO:teuthology.orchestra.run.smithi143.stdout:/var/lib/ceph/osd/ceph-1 2023-11-26T07:19:43.629 INFO:teuthology.orchestra.run.smithi143.stdout:/var/lib/ceph/osd/ceph-2 2023-11-26T07:19:43.629 INFO:teuthology.orchestra.run.smithi143.stdout:/var/lib/ceph/osd/ceph-3 2023-11-26T07:19:43.629 INFO:teuthology.orchestra.run.smithi143.stdout:/run/netns 2023-11-26T07:19:43.629 INFO:teuthology.orchestra.run.smithi143.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2023-11-26T07:19:43.629 INFO:teuthology.orchestra.run.smithi143.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2023-11-26T07:19:43.629 INFO:teuthology.orchestra.run.smithi143.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 2023-11-26T07:19:43.629 INFO:teuthology.orchestra.run.smithi143.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 2023-11-26T07:19:43.630 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-26T07:19:43.630 DEBUG:teuthology.orchestra.run.smithi143:> ls /sys/fs/fuse/connections 2023-11-26T07:19:43.680 INFO:tasks.cephfs.fuse_mount:Pre-mount connections: [] 2023-11-26T07:19:43.681 DEBUG:teuthology.orchestra.run.smithi143:> (cd /home/ubuntu/cephtest && exec sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-fuse -f --admin-socket '/var/run/ceph/$cluster-$name.$pid.asok' /home/ubuntu/cephtest/mnt.0 --id 0) 2023-11-26T07:19:43.727 DEBUG:teuthology.orchestra.run.smithi143:> sudo modprobe fuse 2023-11-26T07:19:43.748 DEBUG:teuthology.orchestra.run.smithi143:> cat /proc/self/mounts | awk '{print $2}' 2023-11-26T07:19:43.804 INFO:teuthology.orchestra.run.smithi143.stdout:/sys 2023-11-26T07:19:43.804 INFO:teuthology.orchestra.run.smithi143.stdout:/proc 2023-11-26T07:19:43.804 INFO:teuthology.orchestra.run.smithi143.stdout:/dev 2023-11-26T07:19:43.804 INFO:teuthology.orchestra.run.smithi143.stdout:/dev/pts 2023-11-26T07:19:43.804 INFO:teuthology.orchestra.run.smithi143.stdout:/run 2023-11-26T07:19:43.804 INFO:teuthology.orchestra.run.smithi143.stdout:/ 2023-11-26T07:19:43.804 INFO:teuthology.orchestra.run.smithi143.stdout:/sys/kernel/security 2023-11-26T07:19:43.804 INFO:teuthology.orchestra.run.smithi143.stdout:/dev/shm 2023-11-26T07:19:43.804 INFO:teuthology.orchestra.run.smithi143.stdout:/run/lock 2023-11-26T07:19:43.804 INFO:teuthology.orchestra.run.smithi143.stdout:/sys/fs/cgroup 2023-11-26T07:19:43.804 INFO:teuthology.orchestra.run.smithi143.stdout:/sys/fs/pstore 2023-11-26T07:19:43.805 INFO:teuthology.orchestra.run.smithi143.stdout:/sys/fs/bpf 2023-11-26T07:19:43.805 INFO:teuthology.orchestra.run.smithi143.stdout:/proc/sys/fs/binfmt_misc 2023-11-26T07:19:43.805 INFO:teuthology.orchestra.run.smithi143.stdout:/dev/hugepages 2023-11-26T07:19:43.805 INFO:teuthology.orchestra.run.smithi143.stdout:/dev/mqueue 2023-11-26T07:19:43.805 INFO:teuthology.orchestra.run.smithi143.stdout:/sys/kernel/debug 2023-11-26T07:19:43.805 INFO:teuthology.orchestra.run.smithi143.stdout:/sys/kernel/tracing 2023-11-26T07:19:43.805 INFO:teuthology.orchestra.run.smithi143.stdout:/proc/fs/nfsd 2023-11-26T07:19:43.805 INFO:teuthology.orchestra.run.smithi143.stdout:/sys/fs/fuse/connections 2023-11-26T07:19:43.805 INFO:teuthology.orchestra.run.smithi143.stdout:/sys/kernel/config 2023-11-26T07:19:43.805 INFO:teuthology.orchestra.run.smithi143.stdout:/run/credentials/systemd-sysusers.service 2023-11-26T07:19:43.805 INFO:teuthology.orchestra.run.smithi143.stdout:/run/rpc_pipefs 2023-11-26T07:19:43.805 INFO:teuthology.orchestra.run.smithi143.stdout:/run/user/1000 2023-11-26T07:19:43.806 INFO:teuthology.orchestra.run.smithi143.stdout:/var/lib/ceph 2023-11-26T07:19:43.806 INFO:teuthology.orchestra.run.smithi143.stdout:/var/lib/ceph/osd/ceph-0 2023-11-26T07:19:43.806 INFO:teuthology.orchestra.run.smithi143.stdout:/var/lib/ceph/osd/ceph-1 2023-11-26T07:19:43.806 INFO:teuthology.orchestra.run.smithi143.stdout:/var/lib/ceph/osd/ceph-2 2023-11-26T07:19:43.806 INFO:teuthology.orchestra.run.smithi143.stdout:/var/lib/ceph/osd/ceph-3 2023-11-26T07:19:43.806 INFO:teuthology.orchestra.run.smithi143.stdout:/run/netns 2023-11-26T07:19:43.806 INFO:teuthology.orchestra.run.smithi143.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2023-11-26T07:19:43.806 INFO:teuthology.orchestra.run.smithi143.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2023-11-26T07:19:43.806 INFO:teuthology.orchestra.run.smithi143.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 2023-11-26T07:19:43.806 INFO:teuthology.orchestra.run.smithi143.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 2023-11-26T07:19:43.807 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-26T07:19:43.807 DEBUG:teuthology.orchestra.run.smithi143:> ls /sys/fs/fuse/connections 2023-11-26T07:19:43.825 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi143.stderr:2023-11-26T07:19:43.823+0000 7fe691845580 -1 init, newargv = 0x55a498918830 newargc=15 2023-11-26T07:19:43.825 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi143.stderr:ceph-fuse[50877]: starting ceph client 2023-11-26T07:19:43.832 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi143.stderr:ceph-fuse[50877]: starting fuse 2023-11-26T07:19:43.834 INFO:teuthology.orchestra.run.smithi143.stdout:43 2023-11-26T07:19:43.834 INFO:tasks.cephfs.fuse_mount:Post-mount connections: [43] 2023-11-26T07:19:43.835 DEBUG:teuthology.orchestra.run.smithi143:> sudo stdin-killer -- python3 -c ' 2023-11-26T07:19:43.835 DEBUG:teuthology.orchestra.run.smithi143:> import glob 2023-11-26T07:19:43.835 DEBUG:teuthology.orchestra.run.smithi143:> import re 2023-11-26T07:19:43.835 DEBUG:teuthology.orchestra.run.smithi143:> import os 2023-11-26T07:19:43.835 DEBUG:teuthology.orchestra.run.smithi143:> import subprocess 2023-11-26T07:19:43.835 DEBUG:teuthology.orchestra.run.smithi143:> 2023-11-26T07:19:43.835 DEBUG:teuthology.orchestra.run.smithi143:> def _find_admin_socket(client_name): 2023-11-26T07:19:43.835 DEBUG:teuthology.orchestra.run.smithi143:> asok_path = "/var/run/ceph/ceph-client.0.*.asok" 2023-11-26T07:19:43.835 DEBUG:teuthology.orchestra.run.smithi143:> files = glob.glob(asok_path) 2023-11-26T07:19:43.835 DEBUG:teuthology.orchestra.run.smithi143:> mountpoint = "/home/ubuntu/cephtest/mnt.0" 2023-11-26T07:19:43.835 DEBUG:teuthology.orchestra.run.smithi143:> 2023-11-26T07:19:43.835 DEBUG:teuthology.orchestra.run.smithi143:> # Given a non-glob path, it better be there 2023-11-26T07:19:43.836 DEBUG:teuthology.orchestra.run.smithi143:> if "*" not in asok_path: 2023-11-26T07:19:43.836 DEBUG:teuthology.orchestra.run.smithi143:> assert(len(files) == 1) 2023-11-26T07:19:43.836 DEBUG:teuthology.orchestra.run.smithi143:> return files[0] 2023-11-26T07:19:43.836 DEBUG:teuthology.orchestra.run.smithi143:> 2023-11-26T07:19:43.836 DEBUG:teuthology.orchestra.run.smithi143:> for f in files: 2023-11-26T07:19:43.836 DEBUG:teuthology.orchestra.run.smithi143:> pid = re.match(".*\.(\d+)\.asok$", f).group(1) 2023-11-26T07:19:43.836 DEBUG:teuthology.orchestra.run.smithi143:> if os.path.exists("/proc/{0}".format(pid)): 2023-11-26T07:19:43.836 DEBUG:teuthology.orchestra.run.smithi143:> with open("/proc/{0}/cmdline".format(pid), '"'"'r'"'"') as proc_f: 2023-11-26T07:19:43.836 DEBUG:teuthology.orchestra.run.smithi143:> contents = proc_f.read() 2023-11-26T07:19:43.836 DEBUG:teuthology.orchestra.run.smithi143:> if mountpoint in contents: 2023-11-26T07:19:43.836 DEBUG:teuthology.orchestra.run.smithi143:> return f 2023-11-26T07:19:43.836 DEBUG:teuthology.orchestra.run.smithi143:> raise RuntimeError("Client socket {0} not found".format(client_name)) 2023-11-26T07:19:43.836 DEBUG:teuthology.orchestra.run.smithi143:> 2023-11-26T07:19:43.837 DEBUG:teuthology.orchestra.run.smithi143:> print(_find_admin_socket("client.0")) 2023-11-26T07:19:43.837 DEBUG:teuthology.orchestra.run.smithi143:> ' 2023-11-26T07:19:43.957 INFO:teuthology.orchestra.run.smithi143.stdout:/var/run/ceph/ceph-client.0.50877.asok 2023-11-26T07:19:43.960 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:43 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2023-11-26T07:19:43.968 INFO:tasks.cephfs.fuse_mount:Found client admin socket at /var/run/ceph/ceph-client.0.50877.asok 2023-11-26T07:19:43.968 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-26T07:19:43.968 DEBUG:teuthology.orchestra.run.smithi143:> sudo ceph --admin-daemon /var/run/ceph/ceph-client.0.50877.asok status 2023-11-26T07:19:44.139 INFO:teuthology.orchestra.run.smithi143.stdout:{ 2023-11-26T07:19:44.140 INFO:teuthology.orchestra.run.smithi143.stdout: "metadata": { 2023-11-26T07:19:44.140 INFO:teuthology.orchestra.run.smithi143.stdout: "ceph_sha1": "44c2237fe77713214148c1fd622c872f346f1b99", 2023-11-26T07:19:44.140 INFO:teuthology.orchestra.run.smithi143.stdout: "ceph_version": "ceph version 18.0.0-7484-g44c2237f (44c2237fe77713214148c1fd622c872f346f1b99) reef (dev)", 2023-11-26T07:19:44.140 INFO:teuthology.orchestra.run.smithi143.stdout: "entity_id": "0", 2023-11-26T07:19:44.140 INFO:teuthology.orchestra.run.smithi143.stdout: "hostname": "smithi143", 2023-11-26T07:19:44.140 INFO:teuthology.orchestra.run.smithi143.stdout: "mount_point": "/home/ubuntu/cephtest/mnt.0", 2023-11-26T07:19:44.140 INFO:teuthology.orchestra.run.smithi143.stdout: "pid": "50877", 2023-11-26T07:19:44.140 INFO:teuthology.orchestra.run.smithi143.stdout: "root": "/" 2023-11-26T07:19:44.140 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2023-11-26T07:19:44.140 INFO:teuthology.orchestra.run.smithi143.stdout: "dentry_count": 0, 2023-11-26T07:19:44.140 INFO:teuthology.orchestra.run.smithi143.stdout: "dentry_pinned_count": 0, 2023-11-26T07:19:44.141 INFO:teuthology.orchestra.run.smithi143.stdout: "id": 5925, 2023-11-26T07:19:44.141 INFO:teuthology.orchestra.run.smithi143.stdout: "inst": { 2023-11-26T07:19:44.141 INFO:teuthology.orchestra.run.smithi143.stdout: "name": { 2023-11-26T07:19:44.141 INFO:teuthology.orchestra.run.smithi143.stdout: "type": "client", 2023-11-26T07:19:44.141 INFO:teuthology.orchestra.run.smithi143.stdout: "num": 5925 2023-11-26T07:19:44.141 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2023-11-26T07:19:44.141 INFO:teuthology.orchestra.run.smithi143.stdout: "addr": { 2023-11-26T07:19:44.141 INFO:teuthology.orchestra.run.smithi143.stdout: "type": "v1", 2023-11-26T07:19:44.141 INFO:teuthology.orchestra.run.smithi143.stdout: "addr": "192.168.0.1:0", 2023-11-26T07:19:44.141 INFO:teuthology.orchestra.run.smithi143.stdout: "nonce": 3063012995 2023-11-26T07:19:44.141 INFO:teuthology.orchestra.run.smithi143.stdout: } 2023-11-26T07:19:44.141 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2023-11-26T07:19:44.141 INFO:teuthology.orchestra.run.smithi143.stdout: "addr": { 2023-11-26T07:19:44.142 INFO:teuthology.orchestra.run.smithi143.stdout: "type": "v1", 2023-11-26T07:19:44.142 INFO:teuthology.orchestra.run.smithi143.stdout: "addr": "192.168.0.1:0", 2023-11-26T07:19:44.142 INFO:teuthology.orchestra.run.smithi143.stdout: "nonce": 3063012995 2023-11-26T07:19:44.142 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2023-11-26T07:19:44.142 INFO:teuthology.orchestra.run.smithi143.stdout: "inst_str": "client.5925 192.168.0.1:0/3063012995", 2023-11-26T07:19:44.142 INFO:teuthology.orchestra.run.smithi143.stdout: "addr_str": "192.168.0.1:0/3063012995", 2023-11-26T07:19:44.142 INFO:teuthology.orchestra.run.smithi143.stdout: "inode_count": 1, 2023-11-26T07:19:44.142 INFO:teuthology.orchestra.run.smithi143.stdout: "mds_epoch": 25, 2023-11-26T07:19:44.142 INFO:teuthology.orchestra.run.smithi143.stdout: "osd_epoch": 45, 2023-11-26T07:19:44.142 INFO:teuthology.orchestra.run.smithi143.stdout: "osd_epoch_barrier": 0, 2023-11-26T07:19:44.142 INFO:teuthology.orchestra.run.smithi143.stdout: "blocklisted": false, 2023-11-26T07:19:44.142 INFO:teuthology.orchestra.run.smithi143.stdout: "fs_name": "cephfs" 2023-11-26T07:19:44.142 INFO:teuthology.orchestra.run.smithi143.stdout:} 2023-11-26T07:19:44.154 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-26T07:19:44.155 DEBUG:teuthology.orchestra.run.smithi143:> stat --file-system '--printf=%T 2023-11-26T07:19:44.155 DEBUG:teuthology.orchestra.run.smithi143:> ' -- /home/ubuntu/cephtest/mnt.0 2023-11-26T07:19:44.164 INFO:teuthology.orchestra.run.smithi143.stdout:fuseblk 2023-11-26T07:19:44.164 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.0 2023-11-26T07:19:44.164 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-26T07:19:44.165 DEBUG:teuthology.orchestra.run.smithi143:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.0 2023-11-26T07:19:44.232 DEBUG:teuthology.orchestra.run.smithi143:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2023-11-26T07:19:44.413 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:44.411+0000 7fa191ddf640 1 Processor -- start 2023-11-26T07:19:44.414 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:44.411+0000 7fa191ddf640 1 -- start start 2023-11-26T07:19:44.414 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:44.411+0000 7fa191ddf640 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7fa18c15d6e0 0x7fa18c0d0df0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:19:44.415 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:44.411+0000 7fa191ddf640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fa18c0d1bc0 0x7fa18c0b0750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:19:44.415 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:44.411+0000 7fa191ddf640 1 -- --> v1:172.21.15.143:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa18c0cf0e0 con 0x7fa18c0b2160 2023-11-26T07:19:44.415 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:44.411+0000 7fa191ddf640 1 -- --> v2:172.21.15.188:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa18c15dac0 con 0x7fa18c15d6e0 2023-11-26T07:19:44.415 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:44.411+0000 7fa191ddf640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa18c0ceb20 con 0x7fa18c0d1bc0 2023-11-26T07:19:44.415 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:44.411+0000 7fa183fff640 1 --1- >> v1:172.21.15.143:6789/0 conn(0x7fa18c0b2160 0x7fa18c0d14b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.143:6789/0 says I am v1:172.21.15.143:50382/0 (socket says 172.21.15.143:50382) 2023-11-26T07:19:44.415 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:44.411+0000 7fa183fff640 1 -- 172.21.15.143:0/2383486805 learned_addr learned my addr 172.21.15.143:0/2383486805 (peer_addr_for_me v1:172.21.15.143:0/0) 2023-11-26T07:19:44.416 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:44.411+0000 7fa18bfff640 1 --2- 172.21.15.143:0/2383486805 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fa18c0d1bc0 0x7fa18c0b0750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:19:44.416 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:44.411+0000 7fa18b7fe640 1 --2- 172.21.15.143:0/2383486805 >> v2:172.21.15.188:3300/0 conn(0x7fa18c15d6e0 0x7fa18c0d0df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:19:44.416 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:44.411+0000 7fa18affd640 1 -- 172.21.15.143:0/2383486805 <== mon.0 v1:172.21.15.143:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4050311341 0 0) 0x7fa18c0cf0e0 con 0x7fa18c0b2160 2023-11-26T07:19:44.416 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:44.411+0000 7fa18affd640 1 -- 172.21.15.143:0/2383486805 --> v1:172.21.15.143:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fa178003530 con 0x7fa18c0b2160 2023-11-26T07:19:44.416 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:44.411+0000 7fa18bfff640 1 -- 172.21.15.143:0/2383486805 >> v2:172.21.15.188:3300/0 conn(0x7fa18c15d6e0 msgr2=0x7fa18c0d0df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-26T07:19:44.416 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:44.411+0000 7fa18bfff640 1 --2- 172.21.15.143:0/2383486805 >> v2:172.21.15.188:3300/0 conn(0x7fa18c15d6e0 0x7fa18c0d0df0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:19:44.416 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:44.411+0000 7fa18bfff640 1 -- 172.21.15.143:0/2383486805 >> v1:172.21.15.143:6789/0 conn(0x7fa18c0b2160 legacy=0x7fa18c0d14b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:19:44.417 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:44.411+0000 7fa18bfff640 1 -- 172.21.15.143:0/2383486805 --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa18c0cee00 con 0x7fa18c0d1bc0 2023-11-26T07:19:44.418 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:44.415+0000 7fa18bfff640 1 --2- 172.21.15.143:0/2383486805 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fa18c0d1bc0 0x7fa18c0b0750 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7fa184009d90 tx=0x7fa18401c950 comp rx=0 tx=0).ready entity=mon.2 client_cookie=43e7730290c2d279 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:19:44.419 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:44.415+0000 7fa18affd640 1 -- 172.21.15.143:0/2383486805 <== mon.2 v2:172.21.15.188:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa1840a5900 con 0x7fa18c0d1bc0 2023-11-26T07:19:44.419 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:44.415+0000 7fa18affd640 1 -- 172.21.15.143:0/2383486805 <== mon.2 v2:172.21.15.188:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa1840a5a60 con 0x7fa18c0d1bc0 2023-11-26T07:19:44.419 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:44.415+0000 7fa18affd640 1 -- 172.21.15.143:0/2383486805 <== mon.2 v2:172.21.15.188:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa1840a5dd0 con 0x7fa18c0d1bc0 2023-11-26T07:19:44.420 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:44.415+0000 7fa191ddf640 1 -- 172.21.15.143:0/2383486805 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fa18c0d1bc0 msgr2=0x7fa18c0b0750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:19:44.420 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:44.415+0000 7fa191ddf640 1 --2- 172.21.15.143:0/2383486805 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fa18c0d1bc0 0x7fa18c0b0750 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7fa184009d90 tx=0x7fa18401c950 comp rx=0 tx=0).stop 2023-11-26T07:19:44.420 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:44.415+0000 7fa191ddf640 1 -- 172.21.15.143:0/2383486805 shutdown_connections 2023-11-26T07:19:44.420 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:44.415+0000 7fa191ddf640 1 --2- 172.21.15.143:0/2383486805 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fa18c0d1bc0 0x7fa18c0b0750 unknown :-1 s=CLOSED pgs=87 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:19:44.420 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:44.415+0000 7fa191ddf640 1 --2- 172.21.15.143:0/2383486805 >> v2:172.21.15.188:3300/0 conn(0x7fa18c15d6e0 0x7fa18c0d0df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:19:44.420 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:44.415+0000 7fa191ddf640 1 -- 172.21.15.143:0/2383486805 >> 172.21.15.143:0/2383486805 conn(0x7fa18c09c550 msgr2=0x7fa18c0d3220 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:19:44.420 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:44.415+0000 7fa191ddf640 1 -- 172.21.15.143:0/2383486805 shutdown_connections 2023-11-26T07:19:44.420 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:44.415+0000 7fa191ddf640 1 -- 172.21.15.143:0/2383486805 wait complete. 2023-11-26T07:19:44.421 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:44.419+0000 7fa191ddf640 1 Processor -- start 2023-11-26T07:19:44.421 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:44.419+0000 7fa191ddf640 1 -- start start 2023-11-26T07:19:44.423 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:44.419+0000 7fa191ddf640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fa18c0b2160 0x7fa18c15de10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:19:44.423 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:44.419+0000 7fa191ddf640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fa18c0d1bc0 0x7fa18c15ffe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:19:44.423 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:44.419+0000 7fa191ddf640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fa18c15d6e0 0x7fa18c160520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:19:44.423 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:44.419+0000 7fa191ddf640 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa18c160a60 con 0x7fa18c0d1bc0 2023-11-26T07:19:44.423 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:44.419+0000 7fa191ddf640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa18c06ce70 con 0x7fa18c0b2160 2023-11-26T07:19:44.423 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:44.419+0000 7fa191ddf640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa18c0b3e30 con 0x7fa18c15d6e0 2023-11-26T07:19:44.423 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:44.419+0000 7fa183fff640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fa18c0d1bc0 0x7fa18c15ffe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:19:44.424 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:44.419+0000 7fa183fff640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fa18c0d1bc0 0x7fa18c15ffe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:58102/0 (socket says 172.21.15.143:58102) 2023-11-26T07:19:44.424 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:44.419+0000 7fa183fff640 1 -- 172.21.15.143:0/2613947263 learned_addr learned my addr 172.21.15.143:0/2613947263 (peer_addr_for_me v2:172.21.15.143:0/0) 2023-11-26T07:19:44.424 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:44.419+0000 7fa18bfff640 1 --2- 172.21.15.143:0/2613947263 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fa18c15d6e0 0x7fa18c160520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:19:44.424 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:44.419+0000 7fa18b7fe640 1 --2- 172.21.15.143:0/2613947263 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fa18c0b2160 0x7fa18c15de10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:19:44.424 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:44.419+0000 7fa183fff640 1 -- 172.21.15.143:0/2613947263 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fa18c15d6e0 msgr2=0x7fa18c160520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:19:44.424 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:44.419+0000 7fa183fff640 1 --2- 172.21.15.143:0/2613947263 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fa18c15d6e0 0x7fa18c160520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:19:44.424 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:44.419+0000 7fa183fff640 1 -- 172.21.15.143:0/2613947263 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fa18c0b2160 msgr2=0x7fa18c15de10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:19:44.424 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:44.419+0000 7fa183fff640 1 --2- 172.21.15.143:0/2613947263 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fa18c0b2160 0x7fa18c15de10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:19:44.424 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:44.419+0000 7fa183fff640 1 -- 172.21.15.143:0/2613947263 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa184009a70 con 0x7fa18c0d1bc0 2023-11-26T07:19:44.424 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:44.419+0000 7fa18b7fe640 1 --2- 172.21.15.143:0/2613947263 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fa18c0b2160 0x7fa18c15de10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-26T07:19:44.424 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:44.419+0000 7fa183fff640 1 --2- 172.21.15.143:0/2613947263 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fa18c0d1bc0 0x7fa18c15ffe0 secure :-1 s=READY pgs=606 cs=0 l=1 rev1=1 crypto rx=0x7fa17400c450 tx=0x7fa17400c910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:19:44.425 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:44.419+0000 7fa188ff9640 1 -- 172.21.15.143:0/2613947263 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa174005cf0 con 0x7fa18c0d1bc0 2023-11-26T07:19:44.425 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:44.419+0000 7fa188ff9640 1 -- 172.21.15.143:0/2613947263 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa174005e50 con 0x7fa18c0d1bc0 2023-11-26T07:19:44.425 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:44.419+0000 7fa191ddf640 1 -- 172.21.15.143:0/2613947263 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa18c15e6b0 con 0x7fa18c0d1bc0 2023-11-26T07:19:44.425 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:44.419+0000 7fa188ff9640 1 -- 172.21.15.143:0/2613947263 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa174007290 con 0x7fa18c0d1bc0 2023-11-26T07:19:44.426 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:44.419+0000 7fa191ddf640 1 -- 172.21.15.143:0/2613947263 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa18c06c670 con 0x7fa18c0d1bc0 2023-11-26T07:19:44.426 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:44.423+0000 7fa191ddf640 1 -- 172.21.15.143:0/2613947263 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa18c067680 con 0x7fa18c0d1bc0 2023-11-26T07:19:44.426 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:44.423+0000 7fa188ff9640 1 -- 172.21.15.143:0/2613947263 <== mon.0 v2:172.21.15.143:3300/0 4 ==== mgrmap(e 7) v1 ==== 86005+0+0 (secure 0 0 0) 0x7fa1740073f0 con 0x7fa18c0d1bc0 2023-11-26T07:19:44.426 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:44.423+0000 7fa188ff9640 1 --2- 172.21.15.143:0/2613947263 >> [v2:172.21.15.143:6832/2150557544,v1:172.21.15.143:6833/2150557544] conn(0x7fa16c066620 0x7fa16c068ae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:19:44.426 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:44.423+0000 7fa188ff9640 1 -- 172.21.15.143:0/2613947263 <== mon.0 v2:172.21.15.143:3300/0 5 ==== osd_map(45..45 src has 1..45) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fa174083760 con 0x7fa18c0d1bc0 2023-11-26T07:19:44.427 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:44.423+0000 7fa18b7fe640 1 --2- 172.21.15.143:0/2613947263 >> [v2:172.21.15.143:6832/2150557544,v1:172.21.15.143:6833/2150557544] conn(0x7fa16c066620 0x7fa16c068ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:19:44.427 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:44.423+0000 7fa18b7fe640 1 --2- 172.21.15.143:0/2613947263 >> [v2:172.21.15.143:6832/2150557544,v1:172.21.15.143:6833/2150557544] conn(0x7fa16c066620 0x7fa16c068ae0 secure :-1 s=READY pgs=300 cs=0 l=1 rev1=1 crypto rx=0x7fa18c160cc0 tx=0x7fa17c008040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:19:44.430 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:44.427+0000 7fa188ff9640 1 -- 172.21.15.143:0/2613947263 <== mon.0 v2:172.21.15.143:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172649 (secure 0 0 0) 0x7fa174050970 con 0x7fa18c0d1bc0 2023-11-26T07:19:44.614 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:44.611+0000 7fa191ddf640 1 -- 172.21.15.143:0/2613947263 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "fs ls"} v 0) v1 -- 0x7fa18c15e3f0 con 0x7fa18c0d1bc0 2023-11-26T07:19:44.615 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:44.611+0000 7fa188ff9640 1 -- 172.21.15.143:0/2613947263 <== mon.0 v2:172.21.15.143:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v25) v1 ==== 53+0+73 (secure 0 0 0) 0x7fa174017020 con 0x7fa18c0d1bc0 2023-11-26T07:19:44.615 INFO:teuthology.orchestra.run.smithi143.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2023-11-26T07:19:44.617 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:44.611+0000 7fa191ddf640 1 -- 172.21.15.143:0/2613947263 >> [v2:172.21.15.143:6832/2150557544,v1:172.21.15.143:6833/2150557544] conn(0x7fa16c066620 msgr2=0x7fa16c068ae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:19:44.617 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:44.615+0000 7fa191ddf640 1 --2- 172.21.15.143:0/2613947263 >> [v2:172.21.15.143:6832/2150557544,v1:172.21.15.143:6833/2150557544] conn(0x7fa16c066620 0x7fa16c068ae0 secure :-1 s=READY pgs=300 cs=0 l=1 rev1=1 crypto rx=0x7fa18c160cc0 tx=0x7fa17c008040 comp rx=0 tx=0).stop 2023-11-26T07:19:44.617 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:44.615+0000 7fa191ddf640 1 -- 172.21.15.143:0/2613947263 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fa18c0d1bc0 msgr2=0x7fa18c15ffe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:19:44.617 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:44.615+0000 7fa191ddf640 1 --2- 172.21.15.143:0/2613947263 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fa18c0d1bc0 0x7fa18c15ffe0 secure :-1 s=READY pgs=606 cs=0 l=1 rev1=1 crypto rx=0x7fa17400c450 tx=0x7fa17400c910 comp rx=0 tx=0).stop 2023-11-26T07:19:44.617 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:44.615+0000 7fa191ddf640 1 -- 172.21.15.143:0/2613947263 shutdown_connections 2023-11-26T07:19:44.617 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:44.615+0000 7fa191ddf640 1 --2- 172.21.15.143:0/2613947263 >> [v2:172.21.15.143:6832/2150557544,v1:172.21.15.143:6833/2150557544] conn(0x7fa16c066620 0x7fa16c068ae0 unknown :-1 s=CLOSED pgs=300 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:19:44.617 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:44.615+0000 7fa191ddf640 1 --2- 172.21.15.143:0/2613947263 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fa18c15d6e0 0x7fa18c160520 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:19:44.618 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:44.615+0000 7fa191ddf640 1 --2- 172.21.15.143:0/2613947263 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fa18c0d1bc0 0x7fa18c15ffe0 unknown :-1 s=CLOSED pgs=606 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:19:44.618 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:44.615+0000 7fa191ddf640 1 --2- 172.21.15.143:0/2613947263 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fa18c0b2160 0x7fa18c15de10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:19:44.618 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:44.615+0000 7fa191ddf640 1 -- 172.21.15.143:0/2613947263 >> 172.21.15.143:0/2613947263 conn(0x7fa18c09c550 msgr2=0x7fa18c0b30b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:19:44.618 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:44.615+0000 7fa191ddf640 1 -- 172.21.15.143:0/2613947263 shutdown_connections 2023-11-26T07:19:44.618 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:44.615+0000 7fa191ddf640 1 -- 172.21.15.143:0/2613947263 wait complete. 2023-11-26T07:19:44.636 INFO:tasks.cephfs.mount:Mounting default Ceph FS; just confirmed its presence on cluster 2023-11-26T07:19:44.636 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2023-11-26T07:19:44.636 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi188.front.sepia.ceph.com 2023-11-26T07:19:44.636 INFO:tasks.cephfs.mount:self.client.name = client.1 2023-11-26T07:19:44.636 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.1 2023-11-26T07:19:44.636 INFO:tasks.cephfs.mount:self.cephfs_name = None 2023-11-26T07:19:44.636 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2023-11-26T07:19:44.636 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2023-11-26T07:19:44.636 INFO:tasks.cephfs.mount:Setting the 'ceph-ns--home-ubuntu-cephtest-mnt.1' netns for '/home/ubuntu/cephtest/mnt.1' 2023-11-26T07:19:44.637 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-26T07:19:44.637 DEBUG:teuthology.orchestra.run.smithi188:> ip addr 2023-11-26T07:19:44.647 INFO:teuthology.orchestra.run.smithi188.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2023-11-26T07:19:44.647 INFO:teuthology.orchestra.run.smithi188.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2023-11-26T07:19:44.648 INFO:teuthology.orchestra.run.smithi188.stdout: inet 127.0.0.1/8 scope host lo 2023-11-26T07:19:44.648 INFO:teuthology.orchestra.run.smithi188.stdout: valid_lft forever preferred_lft forever 2023-11-26T07:19:44.648 INFO:teuthology.orchestra.run.smithi188.stdout: inet6 ::1/128 scope host 2023-11-26T07:19:44.648 INFO:teuthology.orchestra.run.smithi188.stdout: valid_lft forever preferred_lft forever 2023-11-26T07:19:44.648 INFO:teuthology.orchestra.run.smithi188.stdout:2: eno1: mtu 1500 qdisc noop state DOWN group default qlen 1000 2023-11-26T07:19:44.648 INFO:teuthology.orchestra.run.smithi188.stdout: link/ether 0c:c4:7a:d9:37:48 brd ff:ff:ff:ff:ff:ff 2023-11-26T07:19:44.648 INFO:teuthology.orchestra.run.smithi188.stdout: altname enp7s0f0 2023-11-26T07:19:44.649 INFO:teuthology.orchestra.run.smithi188.stdout:3: eno2: mtu 1500 qdisc noop state DOWN group default qlen 1000 2023-11-26T07:19:44.649 INFO:teuthology.orchestra.run.smithi188.stdout: link/ether 0c:c4:7a:d9:37:49 brd ff:ff:ff:ff:ff:ff 2023-11-26T07:19:44.649 INFO:teuthology.orchestra.run.smithi188.stdout: altname enp7s0f1 2023-11-26T07:19:44.649 INFO:teuthology.orchestra.run.smithi188.stdout:4: enp3s0f0: mtu 1500 qdisc noop state DOWN group default qlen 1000 2023-11-26T07:19:44.649 INFO:teuthology.orchestra.run.smithi188.stdout: link/ether 0c:c4:7a:88:80:38 brd ff:ff:ff:ff:ff:ff 2023-11-26T07:19:44.649 INFO:teuthology.orchestra.run.smithi188.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2023-11-26T07:19:44.649 INFO:teuthology.orchestra.run.smithi188.stdout: link/ether 0c:c4:7a:88:80:39 brd ff:ff:ff:ff:ff:ff 2023-11-26T07:19:44.650 INFO:teuthology.orchestra.run.smithi188.stdout: inet 172.21.15.188/20 metric 100 brd 172.21.15.255 scope global dynamic enp3s0f1 2023-11-26T07:19:44.650 INFO:teuthology.orchestra.run.smithi188.stdout: valid_lft 42283sec preferred_lft 42283sec 2023-11-26T07:19:44.650 INFO:teuthology.orchestra.run.smithi188.stdout: inet6 fe80::ec4:7aff:fe88:8039/64 scope link 2023-11-26T07:19:44.650 INFO:teuthology.orchestra.run.smithi188.stdout: valid_lft forever preferred_lft forever 2023-11-26T07:19:44.650 INFO:teuthology.orchestra.run.smithi188.stdout:6: docker0: mtu 1500 qdisc noqueue state DOWN group default 2023-11-26T07:19:44.650 INFO:teuthology.orchestra.run.smithi188.stdout: link/ether 02:42:d8:0d:a0:fa brd ff:ff:ff:ff:ff:ff 2023-11-26T07:19:44.650 INFO:teuthology.orchestra.run.smithi188.stdout: inet 172.17.0.1/16 brd 172.17.255.255 scope global docker0 2023-11-26T07:19:44.651 INFO:teuthology.orchestra.run.smithi188.stdout: valid_lft forever preferred_lft forever 2023-11-26T07:19:44.651 INFO:teuthology.orchestra.run.smithi188.stdout:7: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2023-11-26T07:19:44.651 INFO:teuthology.orchestra.run.smithi188.stdout: link/ether 5e:1f:84:6d:79:ae brd ff:ff:ff:ff:ff:ff 2023-11-26T07:19:44.651 INFO:teuthology.orchestra.run.smithi188.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2023-11-26T07:19:44.651 INFO:teuthology.orchestra.run.smithi188.stdout: valid_lft forever preferred_lft forever 2023-11-26T07:19:44.651 INFO:teuthology.orchestra.run.smithi188.stdout: inet6 fe80::5c1f:84ff:fe6d:79ae/64 scope link 2023-11-26T07:19:44.652 INFO:teuthology.orchestra.run.smithi188.stdout: valid_lft forever preferred_lft forever 2023-11-26T07:19:44.652 INFO:teuthology.orchestra.run.smithi188.stdout:8: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2023-11-26T07:19:44.652 INFO:teuthology.orchestra.run.smithi188.stdout: link/ether 0e:5c:f6:1f:82:2f brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.1 2023-11-26T07:19:44.652 INFO:teuthology.orchestra.run.smithi188.stdout: inet6 fe80::c5c:f6ff:fe1f:822f/64 scope link 2023-11-26T07:19:44.652 INFO:teuthology.orchestra.run.smithi188.stdout: valid_lft forever preferred_lft forever 2023-11-26T07:19:44.652 INFO:teuthology.orchestra.run.smithi188.stdout:9: brx.1@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2023-11-26T07:19:44.652 INFO:teuthology.orchestra.run.smithi188.stdout: link/ether c6:a6:3d:69:d0:db brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.3 2023-11-26T07:19:44.653 INFO:teuthology.orchestra.run.smithi188.stdout: inet6 fe80::c4a6:3dff:fe69:d0db/64 scope link 2023-11-26T07:19:44.653 INFO:teuthology.orchestra.run.smithi188.stdout: valid_lft forever preferred_lft forever 2023-11-26T07:19:44.654 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-26T07:19:44.654 DEBUG:teuthology.orchestra.run.smithi188:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2023-11-26T07:19:44.722 INFO:teuthology.orchestra.run.smithi188.stdout:1 2023-11-26T07:19:44.724 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-26T07:19:44.724 DEBUG:teuthology.orchestra.run.smithi188:> route 2023-11-26T07:19:44.777 INFO:teuthology.orchestra.run.smithi188.stdout:Kernel IP routing table 2023-11-26T07:19:44.778 INFO:teuthology.orchestra.run.smithi188.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2023-11-26T07:19:44.778 INFO:teuthology.orchestra.run.smithi188.stdout:default _gateway 0.0.0.0 UG 0 0 0 enp3s0f1 2023-11-26T07:19:44.778 INFO:teuthology.orchestra.run.smithi188.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2023-11-26T07:19:44.778 INFO:teuthology.orchestra.run.smithi188.stdout:172.17.0.0 0.0.0.0 255.255.0.0 U 0 0 0 docker0 2023-11-26T07:19:44.778 INFO:teuthology.orchestra.run.smithi188.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 enp3s0f1 2023-11-26T07:19:44.778 INFO:teuthology.orchestra.run.smithi188.stdout:ns1.front.sepia 0.0.0.0 255.255.255.255 UH 100 0 0 enp3s0f1 2023-11-26T07:19:44.778 INFO:teuthology.orchestra.run.smithi188.stdout:172.21.0.2 0.0.0.0 255.255.255.255 UH 100 0 0 enp3s0f1 2023-11-26T07:19:44.778 INFO:teuthology.orchestra.run.smithi188.stdout:_gateway 0.0.0.0 255.255.255.255 UH 100 0 0 enp3s0f1 2023-11-26T07:19:44.778 INFO:teuthology.orchestra.run.smithi188.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2023-11-26T07:19:44.779 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-26T07:19:44.779 DEBUG:teuthology.orchestra.run.smithi188:> (cd / && exec bash -c ' 2023-11-26T07:19:44.779 DEBUG:teuthology.orchestra.run.smithi188:> set -e 2023-11-26T07:19:44.780 DEBUG:teuthology.orchestra.run.smithi188:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2023-11-26T07:19:44.780 DEBUG:teuthology.orchestra.run.smithi188:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2023-11-26T07:19:44.780 DEBUG:teuthology.orchestra.run.smithi188:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2023-11-26T07:19:44.780 DEBUG:teuthology.orchestra.run.smithi188:> ') 2023-11-26T07:19:44.877 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-26T07:19:44.878 DEBUG:teuthology.orchestra.run.smithi188:> ip netns list 2023-11-26T07:19:44.887 INFO:teuthology.orchestra.run.smithi188.stdout:ceph-ns--home-ubuntu-cephtest-mnt.1 (id: 0) 2023-11-26T07:19:44.887 INFO:teuthology.orchestra.run.smithi188.stdout:ceph-ns--home-ubuntu-cephtest-mnt.3 (id: 1) 2023-11-26T07:19:44.888 INFO:tasks.cephfs.mount:Resuming the 'ceph-ns--home-ubuntu-cephtest-mnt.1' netns for '/home/ubuntu/cephtest/mnt.1' 2023-11-26T07:19:44.888 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-26T07:19:44.888 DEBUG:teuthology.orchestra.run.smithi188:> sudo ip link set brx.0 up 2023-11-26T07:19:44.950 INFO:tasks.cephfs.fuse_mount:Client client.1 config is {} 2023-11-26T07:19:44.950 INFO:teuthology.orchestra.run:Running command with timeout 60 2023-11-26T07:19:44.950 DEBUG:teuthology.orchestra.run.smithi188:> mkdir -p -v /home/ubuntu/cephtest/mnt.1 2023-11-26T07:19:45.001 INFO:teuthology.orchestra.run.smithi188.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.1' 2023-11-26T07:19:45.001 INFO:teuthology.orchestra.run:Running command with timeout 60 2023-11-26T07:19:45.002 DEBUG:teuthology.orchestra.run.smithi188:> chmod 0000 /home/ubuntu/cephtest/mnt.1 2023-11-26T07:19:45.053 DEBUG:teuthology.orchestra.run.smithi188:> sudo modprobe fuse 2023-11-26T07:19:45.119 DEBUG:teuthology.orchestra.run.smithi188:> cat /proc/self/mounts | awk '{print $2}' 2023-11-26T07:19:45.171 INFO:teuthology.orchestra.run.smithi188.stdout:/sys 2023-11-26T07:19:45.172 INFO:teuthology.orchestra.run.smithi188.stdout:/proc 2023-11-26T07:19:45.172 INFO:teuthology.orchestra.run.smithi188.stdout:/dev 2023-11-26T07:19:45.172 INFO:teuthology.orchestra.run.smithi188.stdout:/dev/pts 2023-11-26T07:19:45.172 INFO:teuthology.orchestra.run.smithi188.stdout:/run 2023-11-26T07:19:45.173 INFO:teuthology.orchestra.run.smithi188.stdout:/ 2023-11-26T07:19:45.173 INFO:teuthology.orchestra.run.smithi188.stdout:/sys/kernel/security 2023-11-26T07:19:45.173 INFO:teuthology.orchestra.run.smithi188.stdout:/dev/shm 2023-11-26T07:19:45.173 INFO:teuthology.orchestra.run.smithi188.stdout:/run/lock 2023-11-26T07:19:45.173 INFO:teuthology.orchestra.run.smithi188.stdout:/sys/fs/cgroup 2023-11-26T07:19:45.174 INFO:teuthology.orchestra.run.smithi188.stdout:/sys/fs/pstore 2023-11-26T07:19:45.174 INFO:teuthology.orchestra.run.smithi188.stdout:/sys/fs/bpf 2023-11-26T07:19:45.174 INFO:teuthology.orchestra.run.smithi188.stdout:/proc/sys/fs/binfmt_misc 2023-11-26T07:19:45.174 INFO:teuthology.orchestra.run.smithi188.stdout:/dev/hugepages 2023-11-26T07:19:45.174 INFO:teuthology.orchestra.run.smithi188.stdout:/dev/mqueue 2023-11-26T07:19:45.174 INFO:teuthology.orchestra.run.smithi188.stdout:/sys/kernel/debug 2023-11-26T07:19:45.175 INFO:teuthology.orchestra.run.smithi188.stdout:/sys/kernel/tracing 2023-11-26T07:19:45.175 INFO:teuthology.orchestra.run.smithi188.stdout:/proc/fs/nfsd 2023-11-26T07:19:45.175 INFO:teuthology.orchestra.run.smithi188.stdout:/sys/fs/fuse/connections 2023-11-26T07:19:45.175 INFO:teuthology.orchestra.run.smithi188.stdout:/sys/kernel/config 2023-11-26T07:19:45.175 INFO:teuthology.orchestra.run.smithi188.stdout:/run/credentials/systemd-sysusers.service 2023-11-26T07:19:45.176 INFO:teuthology.orchestra.run.smithi188.stdout:/run/rpc_pipefs 2023-11-26T07:19:45.176 INFO:teuthology.orchestra.run.smithi188.stdout:/run/user/1000 2023-11-26T07:19:45.176 INFO:teuthology.orchestra.run.smithi188.stdout:/var/lib/ceph 2023-11-26T07:19:45.176 INFO:teuthology.orchestra.run.smithi188.stdout:/var/lib/ceph/osd/ceph-4 2023-11-26T07:19:45.176 INFO:teuthology.orchestra.run.smithi188.stdout:/var/lib/ceph/osd/ceph-5 2023-11-26T07:19:45.176 INFO:teuthology.orchestra.run.smithi188.stdout:/var/lib/ceph/osd/ceph-6 2023-11-26T07:19:45.177 INFO:teuthology.orchestra.run.smithi188.stdout:/var/lib/ceph/osd/ceph-7 2023-11-26T07:19:45.177 INFO:teuthology.orchestra.run.smithi188.stdout:/run/netns 2023-11-26T07:19:45.177 INFO:teuthology.orchestra.run.smithi188.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 2023-11-26T07:19:45.177 INFO:teuthology.orchestra.run.smithi188.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 2023-11-26T07:19:45.177 INFO:teuthology.orchestra.run.smithi188.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 2023-11-26T07:19:45.177 INFO:teuthology.orchestra.run.smithi188.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 2023-11-26T07:19:45.178 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-26T07:19:45.179 DEBUG:teuthology.orchestra.run.smithi188:> ls /sys/fs/fuse/connections 2023-11-26T07:19:45.230 INFO:tasks.cephfs.fuse_mount:Pre-mount connections: [] 2023-11-26T07:19:45.230 DEBUG:teuthology.orchestra.run.smithi188:> (cd /home/ubuntu/cephtest && exec sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-fuse -f --admin-socket '/var/run/ceph/$cluster-$name.$pid.asok' /home/ubuntu/cephtest/mnt.1 --id 1) 2023-11-26T07:19:45.276 DEBUG:teuthology.orchestra.run.smithi188:> sudo modprobe fuse 2023-11-26T07:19:45.299 DEBUG:teuthology.orchestra.run.smithi188:> cat /proc/self/mounts | awk '{print $2}' 2023-11-26T07:19:45.355 INFO:teuthology.orchestra.run.smithi188.stdout:/sys 2023-11-26T07:19:45.355 INFO:teuthology.orchestra.run.smithi188.stdout:/proc 2023-11-26T07:19:45.355 INFO:teuthology.orchestra.run.smithi188.stdout:/dev 2023-11-26T07:19:45.355 INFO:teuthology.orchestra.run.smithi188.stdout:/dev/pts 2023-11-26T07:19:45.355 INFO:teuthology.orchestra.run.smithi188.stdout:/run 2023-11-26T07:19:45.355 INFO:teuthology.orchestra.run.smithi188.stdout:/ 2023-11-26T07:19:45.355 INFO:teuthology.orchestra.run.smithi188.stdout:/sys/kernel/security 2023-11-26T07:19:45.356 INFO:teuthology.orchestra.run.smithi188.stdout:/dev/shm 2023-11-26T07:19:45.356 INFO:teuthology.orchestra.run.smithi188.stdout:/run/lock 2023-11-26T07:19:45.356 INFO:teuthology.orchestra.run.smithi188.stdout:/sys/fs/cgroup 2023-11-26T07:19:45.356 INFO:teuthology.orchestra.run.smithi188.stdout:/sys/fs/pstore 2023-11-26T07:19:45.356 INFO:teuthology.orchestra.run.smithi188.stdout:/sys/fs/bpf 2023-11-26T07:19:45.356 INFO:teuthology.orchestra.run.smithi188.stdout:/proc/sys/fs/binfmt_misc 2023-11-26T07:19:45.356 INFO:teuthology.orchestra.run.smithi188.stdout:/dev/hugepages 2023-11-26T07:19:45.356 INFO:teuthology.orchestra.run.smithi188.stdout:/dev/mqueue 2023-11-26T07:19:45.356 INFO:teuthology.orchestra.run.smithi188.stdout:/sys/kernel/debug 2023-11-26T07:19:45.356 INFO:teuthology.orchestra.run.smithi188.stdout:/sys/kernel/tracing 2023-11-26T07:19:45.357 INFO:teuthology.orchestra.run.smithi188.stdout:/proc/fs/nfsd 2023-11-26T07:19:45.357 INFO:teuthology.orchestra.run.smithi188.stdout:/sys/fs/fuse/connections 2023-11-26T07:19:45.357 INFO:teuthology.orchestra.run.smithi188.stdout:/sys/kernel/config 2023-11-26T07:19:45.357 INFO:teuthology.orchestra.run.smithi188.stdout:/run/credentials/systemd-sysusers.service 2023-11-26T07:19:45.357 INFO:teuthology.orchestra.run.smithi188.stdout:/run/rpc_pipefs 2023-11-26T07:19:45.357 INFO:teuthology.orchestra.run.smithi188.stdout:/run/user/1000 2023-11-26T07:19:45.357 INFO:teuthology.orchestra.run.smithi188.stdout:/var/lib/ceph 2023-11-26T07:19:45.357 INFO:teuthology.orchestra.run.smithi188.stdout:/var/lib/ceph/osd/ceph-4 2023-11-26T07:19:45.358 INFO:teuthology.orchestra.run.smithi188.stdout:/var/lib/ceph/osd/ceph-5 2023-11-26T07:19:45.358 INFO:teuthology.orchestra.run.smithi188.stdout:/var/lib/ceph/osd/ceph-6 2023-11-26T07:19:45.358 INFO:teuthology.orchestra.run.smithi188.stdout:/var/lib/ceph/osd/ceph-7 2023-11-26T07:19:45.358 INFO:teuthology.orchestra.run.smithi188.stdout:/run/netns 2023-11-26T07:19:45.358 INFO:teuthology.orchestra.run.smithi188.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 2023-11-26T07:19:45.358 INFO:teuthology.orchestra.run.smithi188.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 2023-11-26T07:19:45.358 INFO:teuthology.orchestra.run.smithi188.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 2023-11-26T07:19:45.358 INFO:teuthology.orchestra.run.smithi188.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 2023-11-26T07:19:45.359 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-26T07:19:45.359 DEBUG:teuthology.orchestra.run.smithi188:> ls /sys/fs/fuse/connections 2023-11-26T07:19:45.378 INFO:tasks.cephfs.fuse_mount.ceph-fuse.1.smithi188.stderr:2023-11-26T07:19:45.372+0000 7f5186660580 -1 init, newargv = 0x5588f0c70840 newargc=15 2023-11-26T07:19:45.378 INFO:tasks.cephfs.fuse_mount.ceph-fuse.1.smithi188.stderr:ceph-fuse[44538]: starting ceph client 2023-11-26T07:19:45.386 INFO:tasks.cephfs.fuse_mount.ceph-fuse.1.smithi188.stderr:ceph-fuse[44538]: starting fuse 2023-11-26T07:19:45.388 INFO:teuthology.orchestra.run.smithi188.stdout:43 2023-11-26T07:19:45.388 INFO:tasks.cephfs.fuse_mount:Post-mount connections: [43] 2023-11-26T07:19:45.389 DEBUG:teuthology.orchestra.run.smithi188:> sudo stdin-killer -- python3 -c ' 2023-11-26T07:19:45.389 DEBUG:teuthology.orchestra.run.smithi188:> import glob 2023-11-26T07:19:45.389 DEBUG:teuthology.orchestra.run.smithi188:> import re 2023-11-26T07:19:45.389 DEBUG:teuthology.orchestra.run.smithi188:> import os 2023-11-26T07:19:45.389 DEBUG:teuthology.orchestra.run.smithi188:> import subprocess 2023-11-26T07:19:45.389 DEBUG:teuthology.orchestra.run.smithi188:> 2023-11-26T07:19:45.389 DEBUG:teuthology.orchestra.run.smithi188:> def _find_admin_socket(client_name): 2023-11-26T07:19:45.389 DEBUG:teuthology.orchestra.run.smithi188:> asok_path = "/var/run/ceph/ceph-client.1.*.asok" 2023-11-26T07:19:45.389 DEBUG:teuthology.orchestra.run.smithi188:> files = glob.glob(asok_path) 2023-11-26T07:19:45.389 DEBUG:teuthology.orchestra.run.smithi188:> mountpoint = "/home/ubuntu/cephtest/mnt.1" 2023-11-26T07:19:45.389 DEBUG:teuthology.orchestra.run.smithi188:> 2023-11-26T07:19:45.389 DEBUG:teuthology.orchestra.run.smithi188:> # Given a non-glob path, it better be there 2023-11-26T07:19:45.389 DEBUG:teuthology.orchestra.run.smithi188:> if "*" not in asok_path: 2023-11-26T07:19:45.390 DEBUG:teuthology.orchestra.run.smithi188:> assert(len(files) == 1) 2023-11-26T07:19:45.390 DEBUG:teuthology.orchestra.run.smithi188:> return files[0] 2023-11-26T07:19:45.390 DEBUG:teuthology.orchestra.run.smithi188:> 2023-11-26T07:19:45.390 DEBUG:teuthology.orchestra.run.smithi188:> for f in files: 2023-11-26T07:19:45.390 DEBUG:teuthology.orchestra.run.smithi188:> pid = re.match(".*\.(\d+)\.asok$", f).group(1) 2023-11-26T07:19:45.390 DEBUG:teuthology.orchestra.run.smithi188:> if os.path.exists("/proc/{0}".format(pid)): 2023-11-26T07:19:45.390 DEBUG:teuthology.orchestra.run.smithi188:> with open("/proc/{0}/cmdline".format(pid), '"'"'r'"'"') as proc_f: 2023-11-26T07:19:45.390 DEBUG:teuthology.orchestra.run.smithi188:> contents = proc_f.read() 2023-11-26T07:19:45.390 DEBUG:teuthology.orchestra.run.smithi188:> if mountpoint in contents: 2023-11-26T07:19:45.391 DEBUG:teuthology.orchestra.run.smithi188:> return f 2023-11-26T07:19:45.391 DEBUG:teuthology.orchestra.run.smithi188:> raise RuntimeError("Client socket {0} not found".format(client_name)) 2023-11-26T07:19:45.391 DEBUG:teuthology.orchestra.run.smithi188:> 2023-11-26T07:19:45.391 DEBUG:teuthology.orchestra.run.smithi188:> print(_find_admin_socket("client.1")) 2023-11-26T07:19:45.391 DEBUG:teuthology.orchestra.run.smithi188:> ' 2023-11-26T07:19:45.511 INFO:teuthology.orchestra.run.smithi188.stdout:/var/run/ceph/ceph-client.1.44538.asok 2023-11-26T07:19:45.515 INFO:teuthology.orchestra.run.smithi188.stderr:2023-11-26T07:19:45 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2023-11-26T07:19:45.522 INFO:tasks.cephfs.fuse_mount:Found client admin socket at /var/run/ceph/ceph-client.1.44538.asok 2023-11-26T07:19:45.523 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-26T07:19:45.523 DEBUG:teuthology.orchestra.run.smithi188:> sudo ceph --admin-daemon /var/run/ceph/ceph-client.1.44538.asok status 2023-11-26T07:19:45.692 INFO:teuthology.orchestra.run.smithi188.stdout:{ 2023-11-26T07:19:45.693 INFO:teuthology.orchestra.run.smithi188.stdout: "metadata": { 2023-11-26T07:19:45.693 INFO:teuthology.orchestra.run.smithi188.stdout: "ceph_sha1": "44c2237fe77713214148c1fd622c872f346f1b99", 2023-11-26T07:19:45.693 INFO:teuthology.orchestra.run.smithi188.stdout: "ceph_version": "ceph version 18.0.0-7484-g44c2237f (44c2237fe77713214148c1fd622c872f346f1b99) reef (dev)", 2023-11-26T07:19:45.693 INFO:teuthology.orchestra.run.smithi188.stdout: "entity_id": "1", 2023-11-26T07:19:45.693 INFO:teuthology.orchestra.run.smithi188.stdout: "hostname": "smithi188", 2023-11-26T07:19:45.693 INFO:teuthology.orchestra.run.smithi188.stdout: "mount_point": "/home/ubuntu/cephtest/mnt.1", 2023-11-26T07:19:45.693 INFO:teuthology.orchestra.run.smithi188.stdout: "pid": "44538", 2023-11-26T07:19:45.693 INFO:teuthology.orchestra.run.smithi188.stdout: "root": "/" 2023-11-26T07:19:45.693 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2023-11-26T07:19:45.693 INFO:teuthology.orchestra.run.smithi188.stdout: "dentry_count": 0, 2023-11-26T07:19:45.694 INFO:teuthology.orchestra.run.smithi188.stdout: "dentry_pinned_count": 0, 2023-11-26T07:19:45.694 INFO:teuthology.orchestra.run.smithi188.stdout: "id": 5327, 2023-11-26T07:19:45.694 INFO:teuthology.orchestra.run.smithi188.stdout: "inst": { 2023-11-26T07:19:45.694 INFO:teuthology.orchestra.run.smithi188.stdout: "name": { 2023-11-26T07:19:45.694 INFO:teuthology.orchestra.run.smithi188.stdout: "type": "client", 2023-11-26T07:19:45.694 INFO:teuthology.orchestra.run.smithi188.stdout: "num": 5327 2023-11-26T07:19:45.694 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2023-11-26T07:19:45.694 INFO:teuthology.orchestra.run.smithi188.stdout: "addr": { 2023-11-26T07:19:45.694 INFO:teuthology.orchestra.run.smithi188.stdout: "type": "v1", 2023-11-26T07:19:45.695 INFO:teuthology.orchestra.run.smithi188.stdout: "addr": "192.168.0.1:0", 2023-11-26T07:19:45.695 INFO:teuthology.orchestra.run.smithi188.stdout: "nonce": 2748460334 2023-11-26T07:19:45.695 INFO:teuthology.orchestra.run.smithi188.stdout: } 2023-11-26T07:19:45.695 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2023-11-26T07:19:45.695 INFO:teuthology.orchestra.run.smithi188.stdout: "addr": { 2023-11-26T07:19:45.695 INFO:teuthology.orchestra.run.smithi188.stdout: "type": "v1", 2023-11-26T07:19:45.695 INFO:teuthology.orchestra.run.smithi188.stdout: "addr": "192.168.0.1:0", 2023-11-26T07:19:45.695 INFO:teuthology.orchestra.run.smithi188.stdout: "nonce": 2748460334 2023-11-26T07:19:45.695 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2023-11-26T07:19:45.695 INFO:teuthology.orchestra.run.smithi188.stdout: "inst_str": "client.5327 192.168.0.1:0/2748460334", 2023-11-26T07:19:45.695 INFO:teuthology.orchestra.run.smithi188.stdout: "addr_str": "192.168.0.1:0/2748460334", 2023-11-26T07:19:45.695 INFO:teuthology.orchestra.run.smithi188.stdout: "inode_count": 1, 2023-11-26T07:19:45.695 INFO:teuthology.orchestra.run.smithi188.stdout: "mds_epoch": 25, 2023-11-26T07:19:45.696 INFO:teuthology.orchestra.run.smithi188.stdout: "osd_epoch": 45, 2023-11-26T07:19:45.696 INFO:teuthology.orchestra.run.smithi188.stdout: "osd_epoch_barrier": 0, 2023-11-26T07:19:45.696 INFO:teuthology.orchestra.run.smithi188.stdout: "blocklisted": false, 2023-11-26T07:19:45.696 INFO:teuthology.orchestra.run.smithi188.stdout: "fs_name": "cephfs" 2023-11-26T07:19:45.696 INFO:teuthology.orchestra.run.smithi188.stdout:} 2023-11-26T07:19:45.709 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-26T07:19:45.709 DEBUG:teuthology.orchestra.run.smithi188:> stat --file-system '--printf=%T 2023-11-26T07:19:45.709 DEBUG:teuthology.orchestra.run.smithi188:> ' -- /home/ubuntu/cephtest/mnt.1 2023-11-26T07:19:45.718 INFO:teuthology.orchestra.run.smithi188.stdout:fuseblk 2023-11-26T07:19:45.718 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.1 2023-11-26T07:19:45.719 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-26T07:19:45.719 DEBUG:teuthology.orchestra.run.smithi188:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.1 2023-11-26T07:19:45.780 DEBUG:teuthology.orchestra.run.smithi143:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs flag set enable_multiple true --yes-i-really-mean-it 2023-11-26T07:19:45.932 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:45.927+0000 7fbcc77c4640 1 Processor -- start 2023-11-26T07:19:45.932 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:45.927+0000 7fbcc77c4640 1 -- start start 2023-11-26T07:19:45.933 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:45.931+0000 7fbcc77c4640 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7fbcc0143ac0 0x7fbcc0070370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:19:45.934 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:45.931+0000 7fbcc77c4640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fbcc00cf990 0x7fbcc00cd1b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:19:45.934 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:45.931+0000 7fbcc77c4640 1 -- --> v1:172.21.15.143:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbcc00cc730 con 0x7fbcc00708b0 2023-11-26T07:19:45.934 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:45.931+0000 7fbcc77c4640 1 -- --> v2:172.21.15.143:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fbcc0143ea0 con 0x7fbcc0143ac0 2023-11-26T07:19:45.934 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:45.931+0000 7fbcc77c4640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbcc0070ee0 con 0x7fbcc00cf990 2023-11-26T07:19:45.935 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:45.931+0000 7fbcc4d5f640 1 --1- >> v1:172.21.15.143:6789/0 conn(0x7fbcc00708b0 0x7fbcc00cf280 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.143:6789/0 says I am v1:172.21.15.143:50390/0 (socket says 172.21.15.143:50390) 2023-11-26T07:19:45.935 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:45.931+0000 7fbcc4d5f640 1 -- 172.21.15.143:0/4187588263 learned_addr learned my addr 172.21.15.143:0/4187588263 (peer_addr_for_me v1:172.21.15.143:0/0) 2023-11-26T07:19:45.935 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:45.931+0000 7fbcc5560640 1 --2- 172.21.15.143:0/4187588263 >> v2:172.21.15.143:3300/0 conn(0x7fbcc0143ac0 0x7fbcc0070370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:19:45.935 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:45.931+0000 7fbcc5d61640 1 --2- 172.21.15.143:0/4187588263 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fbcc00cf990 0x7fbcc00cd1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:19:45.935 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:45.931+0000 7fbcb7fff640 1 -- 172.21.15.143:0/4187588263 <== mon.0 v1:172.21.15.143:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2024535039 0 0) 0x7fbcc00cc730 con 0x7fbcc00708b0 2023-11-26T07:19:45.935 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:45.931+0000 7fbcb7fff640 1 -- 172.21.15.143:0/4187588263 --> v1:172.21.15.143:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fbcac003530 con 0x7fbcc00708b0 2023-11-26T07:19:45.935 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:45.931+0000 7fbcc5560640 1 -- 172.21.15.143:0/4187588263 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fbcc00cf990 msgr2=0x7fbcc00cd1b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-26T07:19:45.935 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:45.931+0000 7fbcc5560640 1 --2- 172.21.15.143:0/4187588263 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fbcc00cf990 0x7fbcc00cd1b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:19:45.936 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:45.931+0000 7fbcc5560640 1 -- 172.21.15.143:0/4187588263 >> v1:172.21.15.143:6789/0 conn(0x7fbcc00708b0 legacy=0x7fbcc00cf280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:19:45.936 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:45.931+0000 7fbcc5560640 1 -- 172.21.15.143:0/4187588263 --> v2:172.21.15.143:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbcc00711c0 con 0x7fbcc0143ac0 2023-11-26T07:19:45.937 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:45.935+0000 7fbcc5560640 1 --2- 172.21.15.143:0/4187588263 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fbcc0143ac0 0x7fbcc0070370 secure :-1 s=READY pgs=609 cs=0 l=1 rev1=1 crypto rx=0x7fbcbc0097c0 tx=0x7fbcbc01c480 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3ef5f11c0d7a0b9f server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:19:45.937 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:45.935+0000 7fbcb7fff640 1 -- 172.21.15.143:0/4187588263 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbcbc0ad640 con 0x7fbcc0143ac0 2023-11-26T07:19:45.937 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:45.935+0000 7fbcb7fff640 1 -- 172.21.15.143:0/4187588263 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbcbc0ad7a0 con 0x7fbcc0143ac0 2023-11-26T07:19:45.937 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:45.935+0000 7fbcb7fff640 1 -- 172.21.15.143:0/4187588263 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbcbc0adb10 con 0x7fbcc0143ac0 2023-11-26T07:19:45.938 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:45.935+0000 7fbcc77c4640 1 -- 172.21.15.143:0/4187588263 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fbcc0143ac0 msgr2=0x7fbcc0070370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:19:45.938 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:45.935+0000 7fbcc77c4640 1 --2- 172.21.15.143:0/4187588263 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fbcc0143ac0 0x7fbcc0070370 secure :-1 s=READY pgs=609 cs=0 l=1 rev1=1 crypto rx=0x7fbcbc0097c0 tx=0x7fbcbc01c480 comp rx=0 tx=0).stop 2023-11-26T07:19:45.939 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:45.935+0000 7fbcc77c4640 1 -- 172.21.15.143:0/4187588263 shutdown_connections 2023-11-26T07:19:45.939 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:45.935+0000 7fbcc77c4640 1 --2- 172.21.15.143:0/4187588263 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fbcc00cf990 0x7fbcc00cd1b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:19:45.939 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:45.935+0000 7fbcc77c4640 1 --2- 172.21.15.143:0/4187588263 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fbcc0143ac0 0x7fbcc0070370 unknown :-1 s=CLOSED pgs=609 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:19:45.939 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:45.935+0000 7fbcc77c4640 1 -- 172.21.15.143:0/4187588263 >> 172.21.15.143:0/4187588263 conn(0x7fbcc006a890 msgr2=0x7fbcc006aca0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:19:45.939 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:45.935+0000 7fbcc77c4640 1 -- 172.21.15.143:0/4187588263 shutdown_connections 2023-11-26T07:19:45.939 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:45.935+0000 7fbcc77c4640 1 -- 172.21.15.143:0/4187588263 wait complete. 2023-11-26T07:19:45.940 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:45.935+0000 7fbcc77c4640 1 Processor -- start 2023-11-26T07:19:45.940 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:45.935+0000 7fbcc77c4640 1 -- start start 2023-11-26T07:19:45.941 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:45.939+0000 7fbcc77c4640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fbcc00708b0 0x7fbcc013f320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:19:45.942 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:45.939+0000 7fbcc77c4640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fbcc00cf990 0x7fbcc0158f60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:19:45.942 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:45.939+0000 7fbcc77c4640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fbcc0143ac0 0x7fbcc0140300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:19:45.942 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:45.939+0000 7fbcc77c4640 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbcc01594a0 con 0x7fbcc00708b0 2023-11-26T07:19:45.942 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:45.939+0000 7fbcc77c4640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbcc015a590 con 0x7fbcc0143ac0 2023-11-26T07:19:45.942 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:45.939+0000 7fbcc77c4640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbcc013f900 con 0x7fbcc00cf990 2023-11-26T07:19:45.942 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:45.939+0000 7fbcc5560640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fbcc00708b0 0x7fbcc013f320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:19:45.942 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:45.939+0000 7fbcc5560640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fbcc00708b0 0x7fbcc013f320 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:58120/0 (socket says 172.21.15.143:58120) 2023-11-26T07:19:45.942 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:45.939+0000 7fbcc5560640 1 -- 172.21.15.143:0/1373126734 learned_addr learned my addr 172.21.15.143:0/1373126734 (peer_addr_for_me v2:172.21.15.143:0/0) 2023-11-26T07:19:45.942 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:45.939+0000 7fbcc4d5f640 1 --2- 172.21.15.143:0/1373126734 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fbcc00cf990 0x7fbcc0158f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:19:45.942 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:45.939+0000 7fbcc5d61640 1 --2- 172.21.15.143:0/1373126734 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fbcc0143ac0 0x7fbcc0140300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:19:45.942 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:45.939+0000 7fbcc5560640 1 -- 172.21.15.143:0/1373126734 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fbcc00cf990 msgr2=0x7fbcc0158f60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:19:45.942 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:45.939+0000 7fbcc5560640 1 --2- 172.21.15.143:0/1373126734 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fbcc00cf990 0x7fbcc0158f60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:19:45.943 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:45.939+0000 7fbcc5560640 1 -- 172.21.15.143:0/1373126734 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fbcc0143ac0 msgr2=0x7fbcc0140300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:19:45.943 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:45.939+0000 7fbcc5560640 1 --2- 172.21.15.143:0/1373126734 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fbcc0143ac0 0x7fbcc0140300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:19:45.943 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:45.939+0000 7fbcc5560640 1 -- 172.21.15.143:0/1373126734 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbcbc009420 con 0x7fbcc00708b0 2023-11-26T07:19:45.944 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:45.939+0000 7fbcc5560640 1 --2- 172.21.15.143:0/1373126734 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fbcc00708b0 0x7fbcc013f320 secure :-1 s=READY pgs=610 cs=0 l=1 rev1=1 crypto rx=0x7fbcbc0097c0 tx=0x7fbcbc019220 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:19:45.944 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:45.939+0000 7fbcb5ffb640 1 -- 172.21.15.143:0/1373126734 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbcbc0b6630 con 0x7fbcc00708b0 2023-11-26T07:19:45.944 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:45.939+0000 7fbcb5ffb640 1 -- 172.21.15.143:0/1373126734 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbcbc0a6050 con 0x7fbcc00708b0 2023-11-26T07:19:45.944 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:45.939+0000 7fbcb5ffb640 1 -- 172.21.15.143:0/1373126734 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbcbc0b6990 con 0x7fbcc00708b0 2023-11-26T07:19:45.944 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:45.939+0000 7fbcc77c4640 1 -- 172.21.15.143:0/1373126734 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbcc015c920 con 0x7fbcc00708b0 2023-11-26T07:19:45.944 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:45.939+0000 7fbcc77c4640 1 -- 172.21.15.143:0/1373126734 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbcc015cee0 con 0x7fbcc00708b0 2023-11-26T07:19:45.944 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:45.939+0000 7fbcc77c4640 1 -- 172.21.15.143:0/1373126734 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbcc015d0d0 con 0x7fbcc00708b0 2023-11-26T07:19:45.944 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:45.939+0000 7fbcb5ffb640 1 -- 172.21.15.143:0/1373126734 <== mon.0 v2:172.21.15.143:3300/0 4 ==== mgrmap(e 7) v1 ==== 86005+0+0 (secure 0 0 0) 0x7fbcbc0b4040 con 0x7fbcc00708b0 2023-11-26T07:19:45.946 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:45.943+0000 7fbcb5ffb640 1 --2- 172.21.15.143:0/1373126734 >> [v2:172.21.15.143:6832/2150557544,v1:172.21.15.143:6833/2150557544] conn(0x7fbca4066620 0x7fbca4068ae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:19:45.946 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:45.943+0000 7fbcb5ffb640 1 -- 172.21.15.143:0/1373126734 <== mon.0 v2:172.21.15.143:3300/0 5 ==== osd_map(45..45 src has 1..45) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fbcbc11dff0 con 0x7fbcc00708b0 2023-11-26T07:19:45.946 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:45.943+0000 7fbcc4d5f640 1 --2- 172.21.15.143:0/1373126734 >> [v2:172.21.15.143:6832/2150557544,v1:172.21.15.143:6833/2150557544] conn(0x7fbca4066620 0x7fbca4068ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:19:45.946 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:45.943+0000 7fbcc4d5f640 1 --2- 172.21.15.143:0/1373126734 >> [v2:172.21.15.143:6832/2150557544,v1:172.21.15.143:6833/2150557544] conn(0x7fbca4066620 0x7fbca4068ae0 secure :-1 s=READY pgs=301 cs=0 l=1 rev1=1 crypto rx=0x7fbcc015c890 tx=0x7fbcb0000bd0 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:19:45.950 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:45.947+0000 7fbcb5ffb640 1 -- 172.21.15.143:0/1373126734 <== mon.0 v2:172.21.15.143:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172649 (secure 0 0 0) 0x7fbcc015d0d0 con 0x7fbcc00708b0 2023-11-26T07:19:46.132 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:46.127+0000 7fbcc77c4640 1 -- 172.21.15.143:0/1373126734 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "fs flag set", "flag_name": "enable_multiple", "val": "true", "yes_i_really_mean_it": true} v 0) v1 -- 0x7fbcc015d0d0 con 0x7fbcc00708b0 2023-11-26T07:19:46.490 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:46.487+0000 7fbcb5ffb640 1 -- 172.21.15.143:0/1373126734 <== mon.0 v2:172.21.15.143:3300/0 7 ==== mon_command_ack([{"prefix": "fs flag set", "flag_name": "enable_multiple", "val": "true", "yes_i_really_mean_it": true}]=0 v26) v1 ==== 136+0+0 (secure 0 0 0) 0x7fbcc015d0d0 con 0x7fbcc00708b0 2023-11-26T07:19:46.492 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:46.487+0000 7fbcc77c4640 1 -- 172.21.15.143:0/1373126734 >> [v2:172.21.15.143:6832/2150557544,v1:172.21.15.143:6833/2150557544] conn(0x7fbca4066620 msgr2=0x7fbca4068ae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:19:46.492 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:46.487+0000 7fbcc77c4640 1 --2- 172.21.15.143:0/1373126734 >> [v2:172.21.15.143:6832/2150557544,v1:172.21.15.143:6833/2150557544] conn(0x7fbca4066620 0x7fbca4068ae0 secure :-1 s=READY pgs=301 cs=0 l=1 rev1=1 crypto rx=0x7fbcc015c890 tx=0x7fbcb0000bd0 comp rx=0 tx=0).stop 2023-11-26T07:19:46.493 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:46.487+0000 7fbcc77c4640 1 -- 172.21.15.143:0/1373126734 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fbcc00708b0 msgr2=0x7fbcc013f320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:19:46.493 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:46.487+0000 7fbcc77c4640 1 --2- 172.21.15.143:0/1373126734 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fbcc00708b0 0x7fbcc013f320 secure :-1 s=READY pgs=610 cs=0 l=1 rev1=1 crypto rx=0x7fbcbc0097c0 tx=0x7fbcbc019220 comp rx=0 tx=0).stop 2023-11-26T07:19:46.493 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:46.487+0000 7fbcc77c4640 1 -- 172.21.15.143:0/1373126734 shutdown_connections 2023-11-26T07:19:46.493 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:46.487+0000 7fbcc77c4640 1 --2- 172.21.15.143:0/1373126734 >> [v2:172.21.15.143:6832/2150557544,v1:172.21.15.143:6833/2150557544] conn(0x7fbca4066620 0x7fbca4068ae0 unknown :-1 s=CLOSED pgs=301 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:19:46.493 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:46.487+0000 7fbcc77c4640 1 --2- 172.21.15.143:0/1373126734 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fbcc0143ac0 0x7fbcc0140300 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:19:46.493 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:46.487+0000 7fbcc77c4640 1 --2- 172.21.15.143:0/1373126734 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fbcc00cf990 0x7fbcc0158f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:19:46.493 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:46.487+0000 7fbcc77c4640 1 --2- 172.21.15.143:0/1373126734 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fbcc00708b0 0x7fbcc013f320 unknown :-1 s=CLOSED pgs=610 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:19:46.493 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:46.487+0000 7fbcc77c4640 1 -- 172.21.15.143:0/1373126734 >> 172.21.15.143:0/1373126734 conn(0x7fbcc006a890 msgr2=0x7fbcc00add90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:19:46.493 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:46.487+0000 7fbcc77c4640 1 -- 172.21.15.143:0/1373126734 shutdown_connections 2023-11-26T07:19:46.493 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:46.487+0000 7fbcc77c4640 1 -- 172.21.15.143:0/1373126734 wait complete. 2023-11-26T07:19:46.511 DEBUG:teuthology.orchestra.run.smithi143:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs volume ls 2023-11-26T07:19:46.650 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:46.647+0000 7fa5ac0fe640 1 Processor -- start 2023-11-26T07:19:46.650 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:46.647+0000 7fa5ac0fe640 1 -- start start 2023-11-26T07:19:46.652 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:46.647+0000 7fa5ac0fe640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fa5a40cf280 0x7fa5a40cf6e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:19:46.652 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:46.647+0000 7fa5ac0fe640 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7fa5a40cd1b0 0x7fa5a40cd5e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:19:46.652 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:46.647+0000 7fa5ac0fe640 1 -- --> v1:172.21.15.143:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa5a40cfc20 con 0x7fa5a4143ac0 2023-11-26T07:19:46.652 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:46.647+0000 7fa5ac0fe640 1 -- --> v2:172.21.15.143:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa5a4143ea0 con 0x7fa5a40cd1b0 2023-11-26T07:19:46.652 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:46.647+0000 7fa5ac0fe640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa5a40cc730 con 0x7fa5a40cf280 2023-11-26T07:19:46.652 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:46.647+0000 7fa5a9e9a640 1 --1- >> v1:172.21.15.143:6789/0 conn(0x7fa5a4143ac0 0x7fa5a4070370 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.143:6789/0 says I am v1:172.21.15.143:50400/0 (socket says 172.21.15.143:50400) 2023-11-26T07:19:46.652 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:46.647+0000 7fa5a9e9a640 1 -- 172.21.15.143:0/3817427114 learned_addr learned my addr 172.21.15.143:0/3817427114 (peer_addr_for_me v1:172.21.15.143:0/0) 2023-11-26T07:19:46.653 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:46.647+0000 7fa5aa69b640 1 --2- 172.21.15.143:0/3817427114 >> v2:172.21.15.143:3300/0 conn(0x7fa5a40cd1b0 0x7fa5a40cd5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:19:46.653 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:46.647+0000 7fa5a9699640 1 --2- 172.21.15.143:0/3817427114 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fa5a40cf280 0x7fa5a40cf6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:19:46.653 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:46.647+0000 7fa5a8e98640 1 -- 172.21.15.143:0/3817427114 <== mon.0 v1:172.21.15.143:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3242976214 0 0) 0x7fa5a40cfc20 con 0x7fa5a4143ac0 2023-11-26T07:19:46.653 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:46.647+0000 7fa5aa69b640 1 -- 172.21.15.143:0/3817427114 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fa5a40cf280 msgr2=0x7fa5a40cf6e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-26T07:19:46.653 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:46.647+0000 7fa5aa69b640 1 --2- 172.21.15.143:0/3817427114 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fa5a40cf280 0x7fa5a40cf6e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:19:46.653 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:46.647+0000 7fa5aa69b640 1 -- 172.21.15.143:0/3817427114 >> v1:172.21.15.143:6789/0 conn(0x7fa5a4143ac0 legacy=0x7fa5a4070370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:19:46.654 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:46.647+0000 7fa5aa69b640 1 -- 172.21.15.143:0/3817427114 --> v2:172.21.15.143:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa5a4070ee0 con 0x7fa5a40cd1b0 2023-11-26T07:19:46.655 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:46.651+0000 7fa5aa69b640 1 --2- 172.21.15.143:0/3817427114 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fa5a40cd1b0 0x7fa5a40cd5e0 secure :-1 s=READY pgs=612 cs=0 l=1 rev1=1 crypto rx=0x7fa59c009d90 tx=0x7fa59c01c950 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f8f0eecb3fdafad0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:19:46.655 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:46.651+0000 7fa5a8e98640 1 -- 172.21.15.143:0/3817427114 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa59c0a5a10 con 0x7fa5a40cd1b0 2023-11-26T07:19:46.656 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:46.651+0000 7fa5a8e98640 1 -- 172.21.15.143:0/3817427114 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa59c0a5b70 con 0x7fa5a40cd1b0 2023-11-26T07:19:46.656 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:46.651+0000 7fa5a8e98640 1 -- 172.21.15.143:0/3817427114 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa59c0ad440 con 0x7fa5a40cd1b0 2023-11-26T07:19:46.656 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:46.651+0000 7fa5ac0fe640 1 -- 172.21.15.143:0/3817427114 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fa5a40cd1b0 msgr2=0x7fa5a40cd5e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:19:46.657 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:46.651+0000 7fa5ac0fe640 1 --2- 172.21.15.143:0/3817427114 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fa5a40cd1b0 0x7fa5a40cd5e0 secure :-1 s=READY pgs=612 cs=0 l=1 rev1=1 crypto rx=0x7fa59c009d90 tx=0x7fa59c01c950 comp rx=0 tx=0).stop 2023-11-26T07:19:46.657 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:46.651+0000 7fa5ac0fe640 1 -- 172.21.15.143:0/3817427114 shutdown_connections 2023-11-26T07:19:46.657 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:46.651+0000 7fa5ac0fe640 1 --2- 172.21.15.143:0/3817427114 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fa5a40cd1b0 0x7fa5a40cd5e0 unknown :-1 s=CLOSED pgs=612 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:19:46.657 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:46.651+0000 7fa5ac0fe640 1 --2- 172.21.15.143:0/3817427114 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fa5a40cf280 0x7fa5a40cf6e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:19:46.657 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:46.651+0000 7fa5ac0fe640 1 -- 172.21.15.143:0/3817427114 >> 172.21.15.143:0/3817427114 conn(0x7fa5a406a890 msgr2=0x7fa5a406aca0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:19:46.657 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:46.651+0000 7fa5ac0fe640 1 -- 172.21.15.143:0/3817427114 shutdown_connections 2023-11-26T07:19:46.657 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:46.651+0000 7fa5ac0fe640 1 -- 172.21.15.143:0/3817427114 wait complete. 2023-11-26T07:19:46.658 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:46.655+0000 7fa5ac0fe640 1 Processor -- start 2023-11-26T07:19:46.658 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:46.655+0000 7fa5ac0fe640 1 -- start start 2023-11-26T07:19:46.662 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:46.655+0000 7fa5ac0fe640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fa5a40cd1b0 0x7fa5a40ad2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:19:46.662 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:46.655+0000 7fa5ac0fe640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fa5a40cf280 0x7fa5a40b0f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:19:46.662 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:46.655+0000 7fa5ac0fe640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fa5a4143ac0 0x7fa5a414d840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:19:46.662 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:46.655+0000 7fa5ac0fe640 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa5a40b1470 con 0x7fa5a40cf280 2023-11-26T07:19:46.663 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:46.655+0000 7fa5ac0fe640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa5a40ae220 con 0x7fa5a4143ac0 2023-11-26T07:19:46.663 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:46.655+0000 7fa5ac0fe640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa5a415ad70 con 0x7fa5a40cd1b0 2023-11-26T07:19:46.663 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:46.655+0000 7fa5a9699640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fa5a40cf280 0x7fa5a40b0f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:19:46.663 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:46.655+0000 7fa5a9699640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fa5a40cf280 0x7fa5a40b0f30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:58148/0 (socket says 172.21.15.143:58148) 2023-11-26T07:19:46.663 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:46.655+0000 7fa5a9699640 1 -- 172.21.15.143:0/875324108 learned_addr learned my addr 172.21.15.143:0/875324108 (peer_addr_for_me v2:172.21.15.143:0/0) 2023-11-26T07:19:46.663 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:46.655+0000 7fa5a9e9a640 1 --2- 172.21.15.143:0/875324108 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fa5a40cd1b0 0x7fa5a40ad2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:19:46.663 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:46.655+0000 7fa5aa69b640 1 --2- 172.21.15.143:0/875324108 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fa5a4143ac0 0x7fa5a414d840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:19:46.664 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:46.655+0000 7fa5a9699640 1 -- 172.21.15.143:0/875324108 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fa5a40cd1b0 msgr2=0x7fa5a40ad2f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:19:46.664 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:46.655+0000 7fa5a9699640 1 --2- 172.21.15.143:0/875324108 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fa5a40cd1b0 0x7fa5a40ad2f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:19:46.664 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:46.655+0000 7fa5a9699640 1 -- 172.21.15.143:0/875324108 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fa5a4143ac0 msgr2=0x7fa5a414d840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:19:46.664 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:46.655+0000 7fa5a9699640 1 --2- 172.21.15.143:0/875324108 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fa5a4143ac0 0x7fa5a414d840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:19:46.664 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:46.655+0000 7fa5a9699640 1 -- 172.21.15.143:0/875324108 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa59c009a70 con 0x7fa5a40cf280 2023-11-26T07:19:46.664 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:46.655+0000 7fa5aa69b640 1 --2- 172.21.15.143:0/875324108 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fa5a4143ac0 0x7fa5a414d840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-26T07:19:46.664 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:46.655+0000 7fa5a9699640 1 --2- 172.21.15.143:0/875324108 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fa5a40cf280 0x7fa5a40b0f30 secure :-1 s=READY pgs=613 cs=0 l=1 rev1=1 crypto rx=0x7fa59400e930 tx=0x7fa59400edf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:19:46.664 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:46.655+0000 7fa59a7fc640 1 -- 172.21.15.143:0/875324108 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa594009690 con 0x7fa5a40cf280 2023-11-26T07:19:46.664 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:46.655+0000 7fa59a7fc640 1 -- 172.21.15.143:0/875324108 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa59400ca00 con 0x7fa5a40cf280 2023-11-26T07:19:46.664 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:46.655+0000 7fa5ac0fe640 1 -- 172.21.15.143:0/875324108 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa5a414dd80 con 0x7fa5a40cf280 2023-11-26T07:19:46.665 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:46.655+0000 7fa59a7fc640 1 -- 172.21.15.143:0/875324108 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa59400cd70 con 0x7fa5a40cf280 2023-11-26T07:19:46.665 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:46.655+0000 7fa5ac0fe640 1 -- 172.21.15.143:0/875324108 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa5a40adc60 con 0x7fa5a40cf280 2023-11-26T07:19:46.665 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:46.659+0000 7fa5ac0fe640 1 -- 172.21.15.143:0/875324108 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa5a415af00 con 0x7fa5a40cf280 2023-11-26T07:19:46.665 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:46.659+0000 7fa59a7fc640 1 -- 172.21.15.143:0/875324108 <== mon.0 v2:172.21.15.143:3300/0 4 ==== mgrmap(e 7) v1 ==== 86005+0+0 (secure 0 0 0) 0x7fa594013470 con 0x7fa5a40cf280 2023-11-26T07:19:46.666 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:46.659+0000 7fa59a7fc640 1 --2- 172.21.15.143:0/875324108 >> [v2:172.21.15.143:6832/2150557544,v1:172.21.15.143:6833/2150557544] conn(0x7fa588066620 0x7fa588068ae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:19:46.666 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:46.659+0000 7fa59a7fc640 1 -- 172.21.15.143:0/875324108 <== mon.0 v2:172.21.15.143:3300/0 5 ==== osd_map(45..45 src has 1..45) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fa594084870 con 0x7fa5a40cf280 2023-11-26T07:19:46.666 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:46.659+0000 7fa5a9e9a640 1 --2- 172.21.15.143:0/875324108 >> [v2:172.21.15.143:6832/2150557544,v1:172.21.15.143:6833/2150557544] conn(0x7fa588066620 0x7fa588068ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:19:46.666 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:46.659+0000 7fa5a9e9a640 1 --2- 172.21.15.143:0/875324108 >> [v2:172.21.15.143:6832/2150557544,v1:172.21.15.143:6833/2150557544] conn(0x7fa588066620 0x7fa588068ae0 secure :-1 s=READY pgs=302 cs=0 l=1 rev1=1 crypto rx=0x7fa5a0000f70 tx=0x7fa5a0007b60 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:19:46.667 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:46.663+0000 7fa59a7fc640 1 -- 172.21.15.143:0/875324108 <== mon.0 v2:172.21.15.143:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172649 (secure 0 0 0) 0x7fa594051a90 con 0x7fa5a40cf280 2023-11-26T07:19:46.839 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:46.835+0000 7fa5ac0fe640 1 -- 172.21.15.143:0/875324108 --> [v2:172.21.15.143:6832/2150557544,v1:172.21.15.143:6833/2150557544] -- mgr_command(tid 0: {"prefix": "fs volume ls", "target": ["mon-mgr", ""]}) v1 -- 0x7fa5a4071c30 con 0x7fa588066620 2023-11-26T07:19:46.840 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:46.835+0000 7fa59a7fc640 1 -- 172.21.15.143:0/875324108 <== mgr.4100 v2:172.21.15.143:6832/2150557544 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+41 (secure 0 0 0) 0x7fa5a00071f0 con 0x7fa588066620 2023-11-26T07:19:46.841 INFO:teuthology.orchestra.run.smithi143.stdout:[ 2023-11-26T07:19:46.841 INFO:teuthology.orchestra.run.smithi143.stdout: { 2023-11-26T07:19:46.841 INFO:teuthology.orchestra.run.smithi143.stdout: "name": "cephfs" 2023-11-26T07:19:46.841 INFO:teuthology.orchestra.run.smithi143.stdout: } 2023-11-26T07:19:46.841 INFO:teuthology.orchestra.run.smithi143.stdout:] 2023-11-26T07:19:46.843 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:46.839+0000 7fa5ac0fe640 1 -- 172.21.15.143:0/875324108 >> [v2:172.21.15.143:6832/2150557544,v1:172.21.15.143:6833/2150557544] conn(0x7fa588066620 msgr2=0x7fa588068ae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:19:46.843 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:46.839+0000 7fa5ac0fe640 1 --2- 172.21.15.143:0/875324108 >> [v2:172.21.15.143:6832/2150557544,v1:172.21.15.143:6833/2150557544] conn(0x7fa588066620 0x7fa588068ae0 secure :-1 s=READY pgs=302 cs=0 l=1 rev1=1 crypto rx=0x7fa5a0000f70 tx=0x7fa5a0007b60 comp rx=0 tx=0).stop 2023-11-26T07:19:46.843 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:46.839+0000 7fa5ac0fe640 1 -- 172.21.15.143:0/875324108 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fa5a40cf280 msgr2=0x7fa5a40b0f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:19:46.843 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:46.839+0000 7fa5ac0fe640 1 --2- 172.21.15.143:0/875324108 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fa5a40cf280 0x7fa5a40b0f30 secure :-1 s=READY pgs=613 cs=0 l=1 rev1=1 crypto rx=0x7fa59400e930 tx=0x7fa59400edf0 comp rx=0 tx=0).stop 2023-11-26T07:19:46.843 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:46.839+0000 7fa5ac0fe640 1 -- 172.21.15.143:0/875324108 shutdown_connections 2023-11-26T07:19:46.843 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:46.839+0000 7fa5ac0fe640 1 --2- 172.21.15.143:0/875324108 >> [v2:172.21.15.143:6832/2150557544,v1:172.21.15.143:6833/2150557544] conn(0x7fa588066620 0x7fa588068ae0 unknown :-1 s=CLOSED pgs=302 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:19:46.843 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:46.839+0000 7fa5ac0fe640 1 --2- 172.21.15.143:0/875324108 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fa5a4143ac0 0x7fa5a414d840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:19:46.843 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:46.839+0000 7fa5ac0fe640 1 --2- 172.21.15.143:0/875324108 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fa5a40cf280 0x7fa5a40b0f30 unknown :-1 s=CLOSED pgs=613 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:19:46.843 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:46.839+0000 7fa5ac0fe640 1 --2- 172.21.15.143:0/875324108 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fa5a40cd1b0 0x7fa5a40ad2f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:19:46.844 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:46.839+0000 7fa5ac0fe640 1 -- 172.21.15.143:0/875324108 >> 172.21.15.143:0/875324108 conn(0x7fa5a406a890 msgr2=0x7fa5a415c7a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:19:46.844 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:46.839+0000 7fa5ac0fe640 1 -- 172.21.15.143:0/875324108 shutdown_connections 2023-11-26T07:19:46.844 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:46.839+0000 7fa5ac0fe640 1 -- 172.21.15.143:0/875324108 wait complete. 2023-11-26T07:19:46.862 DEBUG:teuthology.orchestra.run.smithi143:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mon mon_allow_pool_delete True 2023-11-26T07:19:47.002 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:46.999+0000 7f7afac91640 1 Processor -- start 2023-11-26T07:19:47.002 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:46.999+0000 7f7afac91640 1 -- start start 2023-11-26T07:19:47.003 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:46.999+0000 7f7afac91640 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7f7af40b2160 0x7f7af40d0df0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:19:47.003 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:46.999+0000 7f7afac91640 1 -- --> v1:172.21.15.143:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7af40cf0e0 con 0x7f7af415d6e0 2023-11-26T07:19:47.004 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:46.999+0000 7f7afac91640 1 -- --> v1:172.21.15.188:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7af40ceb20 con 0x7f7af40d1bc0 2023-11-26T07:19:47.004 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:46.999+0000 7f7afac91640 1 -- --> v2:172.21.15.143:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7af415dac0 con 0x7f7af40b2160 2023-11-26T07:19:47.004 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:46.999+0000 7f7af922e640 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7f7af40b2160 0x7f7af40d0df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:19:47.004 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:46.999+0000 7f7af8a2d640 1 --1- >> v1:172.21.15.143:6789/0 conn(0x7f7af415d6e0 0x7f7af40d14b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.143:6789/0 says I am v1:172.21.15.143:50412/0 (socket says 172.21.15.143:50412) 2023-11-26T07:19:47.004 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:46.999+0000 7f7af8a2d640 1 -- 172.21.15.143:0/135496880 learned_addr learned my addr 172.21.15.143:0/135496880 (peer_addr_for_me v1:172.21.15.143:0/0) 2023-11-26T07:19:47.004 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:46.999+0000 7f7af922e640 1 -- 172.21.15.143:0/135496880 >> v1:172.21.15.188:6789/0 conn(0x7f7af40d1bc0 legacy=0x7f7af40b0750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:19:47.005 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:46.999+0000 7f7af922e640 1 -- 172.21.15.143:0/135496880 >> v1:172.21.15.143:6789/0 conn(0x7f7af415d6e0 legacy=0x7f7af40d14b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:19:47.005 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:46.999+0000 7f7af922e640 1 -- 172.21.15.143:0/135496880 --> v2:172.21.15.143:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7af40cee00 con 0x7f7af40b2160 2023-11-26T07:19:47.005 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:46.999+0000 7f7aeb7fe640 1 -- 172.21.15.143:0/135496880 <== mon.0 v1:172.21.15.143:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 610048009 0 0) 0x7f7af40cf0e0 con 0x7f7af415d6e0 2023-11-26T07:19:47.007 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:47.003+0000 7f7af922e640 1 --2- 172.21.15.143:0/135496880 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f7af40b2160 0x7f7af40d0df0 secure :-1 s=READY pgs=615 cs=0 l=1 rev1=1 crypto rx=0x7f7aec009d90 tx=0x7f7aec01c950 comp rx=0 tx=0).ready entity=mon.0 client_cookie=222446a06ea0f091 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:19:47.007 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:47.003+0000 7f7aeb7fe640 1 -- 172.21.15.143:0/135496880 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7aec0a5900 con 0x7f7af40b2160 2023-11-26T07:19:47.007 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:47.003+0000 7f7aeb7fe640 1 -- 172.21.15.143:0/135496880 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7aec0a5a60 con 0x7f7af40b2160 2023-11-26T07:19:47.007 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:47.003+0000 7f7aeb7fe640 1 -- 172.21.15.143:0/135496880 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7aec0a5df0 con 0x7f7af40b2160 2023-11-26T07:19:47.008 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:47.003+0000 7f7afac91640 1 -- 172.21.15.143:0/135496880 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f7af40b2160 msgr2=0x7f7af40d0df0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:19:47.008 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:47.003+0000 7f7afac91640 1 --2- 172.21.15.143:0/135496880 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f7af40b2160 0x7f7af40d0df0 secure :-1 s=READY pgs=615 cs=0 l=1 rev1=1 crypto rx=0x7f7aec009d90 tx=0x7f7aec01c950 comp rx=0 tx=0).stop 2023-11-26T07:19:47.008 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:47.003+0000 7f7afac91640 1 -- 172.21.15.143:0/135496880 shutdown_connections 2023-11-26T07:19:47.008 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:47.003+0000 7f7afac91640 1 --2- 172.21.15.143:0/135496880 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f7af40b2160 0x7f7af40d0df0 unknown :-1 s=CLOSED pgs=615 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:19:47.008 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:47.003+0000 7f7afac91640 1 -- 172.21.15.143:0/135496880 >> 172.21.15.143:0/135496880 conn(0x7f7af409c550 msgr2=0x7f7af40d3220 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:19:47.008 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:47.003+0000 7f7afac91640 1 -- 172.21.15.143:0/135496880 shutdown_connections 2023-11-26T07:19:47.008 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:47.003+0000 7f7afac91640 1 -- 172.21.15.143:0/135496880 wait complete. 2023-11-26T07:19:47.009 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:47.007+0000 7f7afac91640 1 Processor -- start 2023-11-26T07:19:47.010 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:47.007+0000 7f7afac91640 1 -- start start 2023-11-26T07:19:47.012 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:47.007+0000 7f7afac91640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f7af40b2160 0x7f7af40cc080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:19:47.012 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:47.007+0000 7f7afac91640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f7af40d1bc0 0x7f7af4151700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:19:47.012 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:47.007+0000 7f7afac91640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f7af415d6e0 0x7f7af4152050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:19:47.012 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:47.007+0000 7f7afac91640 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7af40cc5c0 con 0x7f7af40b2160 2023-11-26T07:19:47.013 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:47.007+0000 7f7afac91640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7af40cad00 con 0x7f7af40d1bc0 2023-11-26T07:19:47.013 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:47.007+0000 7f7afac91640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7af40cafe0 con 0x7f7af415d6e0 2023-11-26T07:19:47.013 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:47.007+0000 7f7af8a2d640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f7af40b2160 0x7f7af40cc080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:19:47.013 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:47.007+0000 7f7af8a2d640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f7af40b2160 0x7f7af40cc080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:58174/0 (socket says 172.21.15.143:58174) 2023-11-26T07:19:47.013 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:47.007+0000 7f7af8a2d640 1 -- 172.21.15.143:0/338467294 learned_addr learned my addr 172.21.15.143:0/338467294 (peer_addr_for_me v2:172.21.15.143:0/0) 2023-11-26T07:19:47.013 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:47.007+0000 7f7af922e640 1 --2- 172.21.15.143:0/338467294 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f7af415d6e0 0x7f7af4152050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:19:47.013 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:47.007+0000 7f7aebfff640 1 --2- 172.21.15.143:0/338467294 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f7af40d1bc0 0x7f7af4151700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:19:47.013 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:47.007+0000 7f7af8a2d640 1 -- 172.21.15.143:0/338467294 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f7af415d6e0 msgr2=0x7f7af4152050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:19:47.013 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:47.007+0000 7f7af8a2d640 1 --2- 172.21.15.143:0/338467294 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f7af415d6e0 0x7f7af4152050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:19:47.013 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:47.007+0000 7f7af8a2d640 1 -- 172.21.15.143:0/338467294 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f7af40d1bc0 msgr2=0x7f7af4151700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:19:47.013 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:47.007+0000 7f7af8a2d640 1 --2- 172.21.15.143:0/338467294 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f7af40d1bc0 0x7f7af4151700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:19:47.013 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:47.007+0000 7f7af8a2d640 1 -- 172.21.15.143:0/338467294 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7aec009a70 con 0x7f7af40b2160 2023-11-26T07:19:47.013 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:47.007+0000 7f7af8a2d640 1 --2- 172.21.15.143:0/338467294 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f7af40b2160 0x7f7af40cc080 secure :-1 s=READY pgs=616 cs=0 l=1 rev1=1 crypto rx=0x7f7af000c410 tx=0x7f7af000c8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:19:47.014 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:47.007+0000 7f7ae97fa640 1 -- 172.21.15.143:0/338467294 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7af00074e0 con 0x7f7af40b2160 2023-11-26T07:19:47.014 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:47.007+0000 7f7ae97fa640 1 -- 172.21.15.143:0/338467294 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7af0007640 con 0x7f7af40b2160 2023-11-26T07:19:47.014 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:47.007+0000 7f7ae97fa640 1 -- 172.21.15.143:0/338467294 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7af0004a30 con 0x7f7af40b2160 2023-11-26T07:19:47.014 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:47.007+0000 7f7afac91640 1 -- 172.21.15.143:0/338467294 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7af40cb2c0 con 0x7f7af40b2160 2023-11-26T07:19:47.014 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:47.007+0000 7f7afac91640 1 -- 172.21.15.143:0/338467294 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7af40cb880 con 0x7f7af40b2160 2023-11-26T07:19:47.014 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:47.011+0000 7f7afac91640 1 -- 172.21.15.143:0/338467294 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7af40d2690 con 0x7f7af40b2160 2023-11-26T07:19:47.015 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:47.011+0000 7f7ae97fa640 1 -- 172.21.15.143:0/338467294 <== mon.0 v2:172.21.15.143:3300/0 4 ==== mgrmap(e 7) v1 ==== 86005+0+0 (secure 0 0 0) 0x7f7af0004b90 con 0x7f7af40b2160 2023-11-26T07:19:47.015 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:47.011+0000 7f7ae97fa640 1 --2- 172.21.15.143:0/338467294 >> [v2:172.21.15.143:6832/2150557544,v1:172.21.15.143:6833/2150557544] conn(0x7f7ae0066620 0x7f7ae0068ae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:19:47.015 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:47.011+0000 7f7ae97fa640 1 -- 172.21.15.143:0/338467294 <== mon.0 v2:172.21.15.143:3300/0 5 ==== osd_map(45..45 src has 1..45) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f7af0083ec0 con 0x7f7af40b2160 2023-11-26T07:19:47.015 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:47.011+0000 7f7aebfff640 1 --2- 172.21.15.143:0/338467294 >> [v2:172.21.15.143:6832/2150557544,v1:172.21.15.143:6833/2150557544] conn(0x7f7ae0066620 0x7f7ae0068ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:19:47.016 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:47.011+0000 7f7aebfff640 1 --2- 172.21.15.143:0/338467294 >> [v2:172.21.15.143:6832/2150557544,v1:172.21.15.143:6833/2150557544] conn(0x7f7ae0066620 0x7f7ae0068ae0 secure :-1 s=READY pgs=303 cs=0 l=1 rev1=1 crypto rx=0x7f7af40d0ea0 tx=0x7f7ae4009040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:19:47.019 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:47.015+0000 7f7ae97fa640 1 -- 172.21.15.143:0/338467294 <== mon.0 v2:172.21.15.143:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172649 (secure 0 0 0) 0x7f7af0051160 con 0x7f7af40b2160 2023-11-26T07:19:47.170 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:47.167+0000 7f7afac91640 1 -- 172.21.15.143:0/338467294 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command([{prefix=config set, name=mon_allow_pool_delete}] v 0) v1 -- 0x7f7af40d0ef0 con 0x7f7af40b2160 2023-11-26T07:19:47.170 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:47.167+0000 7f7ae97fa640 1 -- 172.21.15.143:0/338467294 <== mon.0 v2:172.21.15.143:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mon_allow_pool_delete}]=0 v12)=0 v12) v1 ==== 122+0+0 (secure 0 0 0) 0x7f7af0054780 con 0x7f7af40b2160 2023-11-26T07:19:47.173 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:47.171+0000 7f7afac91640 1 -- 172.21.15.143:0/338467294 >> [v2:172.21.15.143:6832/2150557544,v1:172.21.15.143:6833/2150557544] conn(0x7f7ae0066620 msgr2=0x7f7ae0068ae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:19:47.173 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:47.171+0000 7f7afac91640 1 --2- 172.21.15.143:0/338467294 >> [v2:172.21.15.143:6832/2150557544,v1:172.21.15.143:6833/2150557544] conn(0x7f7ae0066620 0x7f7ae0068ae0 secure :-1 s=READY pgs=303 cs=0 l=1 rev1=1 crypto rx=0x7f7af40d0ea0 tx=0x7f7ae4009040 comp rx=0 tx=0).stop 2023-11-26T07:19:47.173 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:47.171+0000 7f7afac91640 1 -- 172.21.15.143:0/338467294 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f7af40b2160 msgr2=0x7f7af40cc080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:19:47.173 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:47.171+0000 7f7afac91640 1 --2- 172.21.15.143:0/338467294 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f7af40b2160 0x7f7af40cc080 secure :-1 s=READY pgs=616 cs=0 l=1 rev1=1 crypto rx=0x7f7af000c410 tx=0x7f7af000c8d0 comp rx=0 tx=0).stop 2023-11-26T07:19:47.173 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:47.171+0000 7f7afac91640 1 -- 172.21.15.143:0/338467294 shutdown_connections 2023-11-26T07:19:47.173 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:47.171+0000 7f7afac91640 1 --2- 172.21.15.143:0/338467294 >> [v2:172.21.15.143:6832/2150557544,v1:172.21.15.143:6833/2150557544] conn(0x7f7ae0066620 0x7f7ae0068ae0 unknown :-1 s=CLOSED pgs=303 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:19:47.173 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:47.171+0000 7f7afac91640 1 --2- 172.21.15.143:0/338467294 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f7af415d6e0 0x7f7af4152050 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:19:47.174 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:47.171+0000 7f7afac91640 1 --2- 172.21.15.143:0/338467294 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f7af40d1bc0 0x7f7af4151700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:19:47.174 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:47.171+0000 7f7afac91640 1 --2- 172.21.15.143:0/338467294 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f7af40b2160 0x7f7af40cc080 unknown :-1 s=CLOSED pgs=616 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:19:47.174 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:47.171+0000 7f7afac91640 1 -- 172.21.15.143:0/338467294 >> 172.21.15.143:0/338467294 conn(0x7f7af409c550 msgr2=0x7f7af40cfd20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:19:47.174 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:47.171+0000 7f7afac91640 1 -- 172.21.15.143:0/338467294 shutdown_connections 2023-11-26T07:19:47.174 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:47.171+0000 7f7afac91640 1 -- 172.21.15.143:0/338467294 wait complete. 2023-11-26T07:19:47.192 DEBUG:teuthology.orchestra.run.smithi143:> set -ex 2023-11-26T07:19:47.192 DEBUG:teuthology.orchestra.run.smithi143:> dd if=/proc/self/mounts of=/dev/stdout 2023-11-26T07:19:47.201 DEBUG:teuthology.orchestra.run.smithi143:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-11-26T07:19:47.380 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:47.375+0000 7fd0426fe640 1 Processor -- start 2023-11-26T07:19:47.381 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:47.375+0000 7fd0426fe640 1 -- start start 2023-11-26T07:19:47.382 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:47.379+0000 7fd0426fe640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fd03c0b2f80 0x7fd03c071850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:19:47.382 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:47.379+0000 7fd0426fe640 1 -- --> v1:172.21.15.143:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd03c06c750 con 0x7fd03c1596d0 2023-11-26T07:19:47.382 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:47.379+0000 7fd0426fe640 1 -- --> v1:172.21.15.188:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd03c0b3550 con 0x7fd03c0ce680 2023-11-26T07:19:47.382 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:47.379+0000 7fd0426fe640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd03c0cbe00 con 0x7fd03c0b2f80 2023-11-26T07:19:47.382 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:47.379+0000 7fd03bfff640 1 --1- >> v1:172.21.15.143:6789/0 conn(0x7fd03c1596d0 0x7fd03c0b2870 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.143:6789/0 says I am v1:172.21.15.143:50426/0 (socket says 172.21.15.143:50426) 2023-11-26T07:19:47.382 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:47.379+0000 7fd03bfff640 1 -- 172.21.15.143:0/1742096196 learned_addr learned my addr 172.21.15.143:0/1742096196 (peer_addr_for_me v1:172.21.15.143:0/0) 2023-11-26T07:19:47.382 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:47.379+0000 7fd03b7fe640 1 --2- 172.21.15.143:0/1742096196 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fd03c0b2f80 0x7fd03c071850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:19:47.383 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:47.379+0000 7fd03affd640 1 -- 172.21.15.143:0/1742096196 <== mon.0 v1:172.21.15.143:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3432537770 0 0) 0x7fd03c06c750 con 0x7fd03c1596d0 2023-11-26T07:19:47.383 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:47.379+0000 7fd03affd640 1 -- 172.21.15.143:0/1742096196 --> v1:172.21.15.143:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fd024003450 con 0x7fd03c1596d0 2023-11-26T07:19:47.383 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:47.379+0000 7fd03b7fe640 1 -- 172.21.15.143:0/1742096196 >> v1:172.21.15.188:6789/0 conn(0x7fd03c0ce680 legacy=0x7fd03c06cbe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:19:47.383 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:47.379+0000 7fd03affd640 1 -- 172.21.15.143:0/1742096196 <== mon.1 v1:172.21.15.188:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 241684377 0 0) 0x7fd03c0b3550 con 0x7fd03c0ce680 2023-11-26T07:19:47.383 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:47.379+0000 7fd03b7fe640 1 -- 172.21.15.143:0/1742096196 >> v1:172.21.15.143:6789/0 conn(0x7fd03c1596d0 legacy=0x7fd03c0b2870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:19:47.383 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:47.379+0000 7fd03b7fe640 1 -- 172.21.15.143:0/1742096196 --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd03c071e80 con 0x7fd03c0b2f80 2023-11-26T07:19:47.385 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:47.383+0000 7fd03b7fe640 1 --2- 172.21.15.143:0/1742096196 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fd03c0b2f80 0x7fd03c071850 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7fd0300097f0 tx=0x7fd03001c4c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=e265a7cef644e78d server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:19:47.386 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:47.383+0000 7fd03affd640 1 -- 172.21.15.143:0/1742096196 <== mon.2 v2:172.21.15.188:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd0300ab600 con 0x7fd03c0b2f80 2023-11-26T07:19:47.386 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:47.383+0000 7fd03affd640 1 -- 172.21.15.143:0/1742096196 <== mon.2 v2:172.21.15.188:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd0300a7050 con 0x7fd03c0b2f80 2023-11-26T07:19:47.386 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:47.383+0000 7fd03affd640 1 -- 172.21.15.143:0/1742096196 <== mon.2 v2:172.21.15.188:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd0300ab980 con 0x7fd03c0b2f80 2023-11-26T07:19:47.386 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:47.383+0000 7fd0426fe640 1 -- 172.21.15.143:0/1742096196 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fd03c0b2f80 msgr2=0x7fd03c071850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:19:47.387 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:47.383+0000 7fd0426fe640 1 --2- 172.21.15.143:0/1742096196 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fd03c0b2f80 0x7fd03c071850 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7fd0300097f0 tx=0x7fd03001c4c0 comp rx=0 tx=0).stop 2023-11-26T07:19:47.387 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:47.383+0000 7fd0426fe640 1 -- 172.21.15.143:0/1742096196 shutdown_connections 2023-11-26T07:19:47.387 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:47.383+0000 7fd0426fe640 1 --2- 172.21.15.143:0/1742096196 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fd03c0b2f80 0x7fd03c071850 unknown :-1 s=CLOSED pgs=90 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:19:47.387 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:47.383+0000 7fd0426fe640 1 -- 172.21.15.143:0/1742096196 >> 172.21.15.143:0/1742096196 conn(0x7fd03c06a890 msgr2=0x7fd03c06aca0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:19:47.387 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:47.383+0000 7fd0426fe640 1 -- 172.21.15.143:0/1742096196 shutdown_connections 2023-11-26T07:19:47.387 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:47.383+0000 7fd0426fe640 1 -- 172.21.15.143:0/1742096196 wait complete. 2023-11-26T07:19:47.388 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:47.383+0000 7fd0426fe640 1 Processor -- start 2023-11-26T07:19:47.388 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:47.383+0000 7fd0426fe640 1 -- start start 2023-11-26T07:19:47.389 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:47.387+0000 7fd0426fe640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fd03c0b2f80 0x7fd03c13c1d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:19:47.389 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:47.387+0000 7fd0426fe640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fd03c0ce680 0x7fd03c156360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:19:47.389 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:47.387+0000 7fd0426fe640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fd03c1596d0 0x7fd03c15c6d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:19:47.389 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:47.387+0000 7fd0426fe640 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd03c1568a0 con 0x7fd03c1596d0 2023-11-26T07:19:47.390 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:47.387+0000 7fd0426fe640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd03c15cc10 con 0x7fd03c0b2f80 2023-11-26T07:19:47.390 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:47.387+0000 7fd0426fe640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd03c151360 con 0x7fd03c0ce680 2023-11-26T07:19:47.391 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:47.387+0000 7fd040c9b640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fd03c1596d0 0x7fd03c15c6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:19:47.391 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:47.387+0000 7fd040c9b640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fd03c1596d0 0x7fd03c15c6d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:58178/0 (socket says 172.21.15.143:58178) 2023-11-26T07:19:47.391 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:47.387+0000 7fd040c9b640 1 -- 172.21.15.143:0/4069153475 learned_addr learned my addr 172.21.15.143:0/4069153475 (peer_addr_for_me v2:172.21.15.143:0/0) 2023-11-26T07:19:47.391 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:47.387+0000 7fd03b7fe640 1 --2- 172.21.15.143:0/4069153475 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fd03c0ce680 0x7fd03c156360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:19:47.391 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:47.387+0000 7fd03bfff640 1 --2- 172.21.15.143:0/4069153475 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fd03c0b2f80 0x7fd03c13c1d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:19:47.392 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:47.387+0000 7fd040c9b640 1 -- 172.21.15.143:0/4069153475 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fd03c0ce680 msgr2=0x7fd03c156360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:19:47.392 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:47.387+0000 7fd040c9b640 1 --2- 172.21.15.143:0/4069153475 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fd03c0ce680 0x7fd03c156360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:19:47.392 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:47.387+0000 7fd040c9b640 1 -- 172.21.15.143:0/4069153475 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fd03c0b2f80 msgr2=0x7fd03c13c1d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:19:47.392 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:47.387+0000 7fd040c9b640 1 --2- 172.21.15.143:0/4069153475 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fd03c0b2f80 0x7fd03c13c1d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:19:47.392 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:47.387+0000 7fd040c9b640 1 -- 172.21.15.143:0/4069153475 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd030009420 con 0x7fd03c1596d0 2023-11-26T07:19:47.392 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:47.387+0000 7fd040c9b640 1 --2- 172.21.15.143:0/4069153475 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fd03c1596d0 0x7fd03c15c6d0 secure :-1 s=READY pgs=618 cs=0 l=1 rev1=1 crypto rx=0x7fd034001570 tx=0x7fd03400cd20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:19:47.392 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:47.387+0000 7fd038ff9640 1 -- 172.21.15.143:0/4069153475 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd034003db0 con 0x7fd03c1596d0 2023-11-26T07:19:47.392 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:47.387+0000 7fd038ff9640 1 -- 172.21.15.143:0/4069153475 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd034005070 con 0x7fd03c1596d0 2023-11-26T07:19:47.392 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:47.387+0000 7fd0426fe640 1 -- 172.21.15.143:0/4069153475 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd03c13ca40 con 0x7fd03c1596d0 2023-11-26T07:19:47.392 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:47.387+0000 7fd038ff9640 1 -- 172.21.15.143:0/4069153475 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd034005420 con 0x7fd03c1596d0 2023-11-26T07:19:47.392 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:47.387+0000 7fd0426fe640 1 -- 172.21.15.143:0/4069153475 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd03c151850 con 0x7fd03c1596d0 2023-11-26T07:19:47.393 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:47.387+0000 7fd0426fe640 1 -- 172.21.15.143:0/4069153475 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd03c13c810 con 0x7fd03c1596d0 2023-11-26T07:19:47.393 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:47.391+0000 7fd038ff9640 1 -- 172.21.15.143:0/4069153475 <== mon.0 v2:172.21.15.143:3300/0 4 ==== mgrmap(e 7) v1 ==== 86005+0+0 (secure 0 0 0) 0x7fd034005580 con 0x7fd03c1596d0 2023-11-26T07:19:47.394 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:47.391+0000 7fd038ff9640 1 --2- 172.21.15.143:0/4069153475 >> [v2:172.21.15.143:6832/2150557544,v1:172.21.15.143:6833/2150557544] conn(0x7fd028066620 0x7fd028068ae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:19:47.394 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:47.391+0000 7fd038ff9640 1 -- 172.21.15.143:0/4069153475 <== mon.0 v2:172.21.15.143:3300/0 5 ==== osd_map(45..45 src has 1..45) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fd034083920 con 0x7fd03c1596d0 2023-11-26T07:19:47.394 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:47.391+0000 7fd03bfff640 1 --2- 172.21.15.143:0/4069153475 >> [v2:172.21.15.143:6832/2150557544,v1:172.21.15.143:6833/2150557544] conn(0x7fd028066620 0x7fd028068ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:19:47.394 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:47.391+0000 7fd03bfff640 1 --2- 172.21.15.143:0/4069153475 >> [v2:172.21.15.143:6832/2150557544,v1:172.21.15.143:6833/2150557544] conn(0x7fd028066620 0x7fd028068ae0 secure :-1 s=READY pgs=304 cs=0 l=1 rev1=1 crypto rx=0x7fd02c007fd0 tx=0x7fd02c009040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:19:47.398 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:47.395+0000 7fd038ff9640 1 -- 172.21.15.143:0/4069153475 <== mon.0 v2:172.21.15.143:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172649 (secure 0 0 0) 0x7fd03c13c810 con 0x7fd03c1596d0 2023-11-26T07:19:47.549 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:47.547+0000 7fd0426fe640 1 -- 172.21.15.143:0/4069153475 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fd03c13c810 con 0x7fd03c1596d0 2023-11-26T07:19:47.550 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:47.547+0000 7fd038ff9640 1 -- 172.21.15.143:0/4069153475 <== mon.0 v2:172.21.15.143:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v45) v1 ==== 74+0+16549 (secure 0 0 0) 0x7fd03c13c810 con 0x7fd03c1596d0 2023-11-26T07:19:47.550 INFO:teuthology.orchestra.run.smithi143.stdout: 2023-11-26T07:19:47.550 INFO:teuthology.orchestra.run.smithi143.stdout:{"epoch":45,"fsid":"f5033060-2469-4a07-95fe-2b76f5404bc1","created":"2023-11-26T07:17:28.613663+0000","modified":"2023-11-26T07:19:33.132471+0000","last_up_change":"2023-11-26T07:17:41.157813+0000","last_in_change":"2023-11-26T07:17:31.181658+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":7,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-11-26T07:17:41.758597+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":6,"pool_name":"cephfs_metadata","create_time":"2023-11-26T07:19:29.208075+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"44","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":7,"pool_name":"cephfs_data","create_time":"2023-11-26T07:19:30.503743+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"45","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"ebbee2c6-023e-44b9-9f1d-6f7bfd309730","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":43,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6817","nonce":639807369},{"type":"v1","addr":"172.21.15.143:6819","nonce":639807369}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6821","nonce":639807369},{"type":"v1","addr":"172.21.15.143:6823","nonce":639807369}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6830","nonce":639807369},{"type":"v1","addr":"172.21.15.143:6831","nonce":639807369}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6825","nonce":639807369},{"type":"v1","addr":"172.21.15.143:6828","nonce":639807369}]},"public_addr":"172.21.15.143:6819/639807369","cluster_addr":"172.21.15.143:6823/639807369","heartbeat_back_addr":"172.21.15.143:6831/639807369","heartbeat_front_addr":"172.21.15.143:6828/639807369","state":["exists","up"]},{"osd":1,"uuid":"fec4b8be-7122-49e9-9a19-5850a0700001","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":43,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6816","nonce":1144942530},{"type":"v1","addr":"172.21.15.143:6818","nonce":1144942530}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6820","nonce":1144942530},{"type":"v1","addr":"172.21.15.143:6822","nonce":1144942530}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6827","nonce":1144942530},{"type":"v1","addr":"172.21.15.143:6829","nonce":1144942530}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6824","nonce":1144942530},{"type":"v1","addr":"172.21.15.143:6826","nonce":1144942530}]},"public_addr":"172.21.15.143:6818/1144942530","cluster_addr":"172.21.15.143:6822/1144942530","heartbeat_back_addr":"172.21.15.143:6829/1144942530","heartbeat_front_addr":"172.21.15.143:6826/1144942530","state":["exists","up"]},{"osd":2,"uuid":"2888af5b-f933-46ca-8f95-6dd494ff43f6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":43,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6808","nonce":3909095186},{"type":"v1","addr":"172.21.15.143:6809","nonce":3909095186}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6810","nonce":3909095186},{"type":"v1","addr":"172.21.15.143:6811","nonce":3909095186}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6814","nonce":3909095186},{"type":"v1","addr":"172.21.15.143:6815","nonce":3909095186}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6812","nonce":3909095186},{"type":"v1","addr":"172.21.15.143:6813","nonce":3909095186}]},"public_addr":"172.21.15.143:6809/3909095186","cluster_addr":"172.21.15.143:6811/3909095186","heartbeat_back_addr":"172.21.15.143:6815/3909095186","heartbeat_front_addr":"172.21.15.143:6813/3909095186","state":["exists","up"]},{"osd":3,"uuid":"d1f52435-7bc1-4081-9550-9cd1ba55f549","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":43,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6800","nonce":3854210991},{"type":"v1","addr":"172.21.15.143:6801","nonce":3854210991}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6802","nonce":3854210991},{"type":"v1","addr":"172.21.15.143:6803","nonce":3854210991}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6806","nonce":3854210991},{"type":"v1","addr":"172.21.15.143:6807","nonce":3854210991}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6804","nonce":3854210991},{"type":"v1","addr":"172.21.15.143:6805","nonce":3854210991}]},"public_addr":"172.21.15.143:6801/3854210991","cluster_addr":"172.21.15.143:6803/3854210991","heartbeat_back_addr":"172.21.15.143:6807/3854210991","heartbeat_front_addr":"172.21.15.143:6805/3854210991","state":["exists","up"]},{"osd":4,"uuid":"e2ae8b6e-6e25-49b2-bcbf-aacce68f06a3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":43,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6800","nonce":3924606902},{"type":"v1","addr":"172.21.15.188:6801","nonce":3924606902}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6802","nonce":3924606902},{"type":"v1","addr":"172.21.15.188:6803","nonce":3924606902}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6806","nonce":3924606902},{"type":"v1","addr":"172.21.15.188:6807","nonce":3924606902}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6804","nonce":3924606902},{"type":"v1","addr":"172.21.15.188:6805","nonce":3924606902}]},"public_addr":"172.21.15.188:6801/3924606902","cluster_addr":"172.21.15.188:6803/3924606902","heartbeat_back_addr":"172.21.15.188:6807/3924606902","heartbeat_front_addr":"172.21.15.188:6805/3924606902","state":["exists","up"]},{"osd":5,"uuid":"3cec8a32-7698-4c36-8e89-e4ba11934f33","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":43,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6813","nonce":186841544},{"type":"v1","addr":"172.21.15.188:6815","nonce":186841544}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6817","nonce":186841544},{"type":"v1","addr":"172.21.15.188:6818","nonce":186841544}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6822","nonce":186841544},{"type":"v1","addr":"172.21.15.188:6823","nonce":186841544}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6820","nonce":186841544},{"type":"v1","addr":"172.21.15.188:6821","nonce":186841544}]},"public_addr":"172.21.15.188:6815/186841544","cluster_addr":"172.21.15.188:6818/186841544","heartbeat_back_addr":"172.21.15.188:6823/186841544","heartbeat_front_addr":"172.21.15.188:6821/186841544","state":["exists","up"]},{"osd":6,"uuid":"58a65d5d-e090-473a-81b4-ad38e85029f9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":43,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6808","nonce":2153499405},{"type":"v1","addr":"172.21.15.188:6809","nonce":2153499405}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6810","nonce":2153499405},{"type":"v1","addr":"172.21.15.188:6811","nonce":2153499405}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6816","nonce":2153499405},{"type":"v1","addr":"172.21.15.188:6819","nonce":2153499405}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6812","nonce":2153499405},{"type":"v1","addr":"172.21.15.188:6814","nonce":2153499405}]},"public_addr":"172.21.15.188:6809/2153499405","cluster_addr":"172.21.15.188:6811/2153499405","heartbeat_back_addr":"172.21.15.188:6819/2153499405","heartbeat_front_addr":"172.21.15.188:6814/2153499405","state":["exists","up"]},{"osd":7,"uuid":"ea5b7f70-3554-4cf8-ac30-f7c59b80678d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":43,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6824","nonce":2722096729},{"type":"v1","addr":"172.21.15.188:6825","nonce":2722096729}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6826","nonce":2722096729},{"type":"v1","addr":"172.21.15.188:6827","nonce":2722096729}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6830","nonce":2722096729},{"type":"v1","addr":"172.21.15.188:6831","nonce":2722096729}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6828","nonce":2722096729},{"type":"v1","addr":"172.21.15.188:6829","nonce":2722096729}]},"public_addr":"172.21.15.188:6825/2722096729","cluster_addr":"172.21.15.188:6827/2722096729","heartbeat_back_addr":"172.21.15.188:6831/2722096729","heartbeat_front_addr":"172.21.15.188:6829/2722096729","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-26T07:17:36.995032+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-26T07:17:36.932266+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-26T07:17:36.829304+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-26T07:17:36.804297+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-26T07:17:36.667892+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-26T07:17:36.964868+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-26T07:17:36.827332+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-26T07:17:36.893179+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-11-26T07:19:47.552 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:47.547+0000 7fd0426fe640 1 -- 172.21.15.143:0/4069153475 >> [v2:172.21.15.143:6832/2150557544,v1:172.21.15.143:6833/2150557544] conn(0x7fd028066620 msgr2=0x7fd028068ae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:19:47.552 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:47.547+0000 7fd0426fe640 1 --2- 172.21.15.143:0/4069153475 >> [v2:172.21.15.143:6832/2150557544,v1:172.21.15.143:6833/2150557544] conn(0x7fd028066620 0x7fd028068ae0 secure :-1 s=READY pgs=304 cs=0 l=1 rev1=1 crypto rx=0x7fd02c007fd0 tx=0x7fd02c009040 comp rx=0 tx=0).stop 2023-11-26T07:19:47.552 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:47.547+0000 7fd0426fe640 1 -- 172.21.15.143:0/4069153475 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fd03c1596d0 msgr2=0x7fd03c15c6d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:19:47.552 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:47.547+0000 7fd0426fe640 1 --2- 172.21.15.143:0/4069153475 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fd03c1596d0 0x7fd03c15c6d0 secure :-1 s=READY pgs=618 cs=0 l=1 rev1=1 crypto rx=0x7fd034001570 tx=0x7fd03400cd20 comp rx=0 tx=0).stop 2023-11-26T07:19:47.552 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:47.547+0000 7fd0426fe640 1 -- 172.21.15.143:0/4069153475 shutdown_connections 2023-11-26T07:19:47.552 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:47.547+0000 7fd0426fe640 1 --2- 172.21.15.143:0/4069153475 >> [v2:172.21.15.143:6832/2150557544,v1:172.21.15.143:6833/2150557544] conn(0x7fd028066620 0x7fd028068ae0 unknown :-1 s=CLOSED pgs=304 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:19:47.552 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:47.547+0000 7fd0426fe640 1 --2- 172.21.15.143:0/4069153475 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fd03c1596d0 0x7fd03c15c6d0 unknown :-1 s=CLOSED pgs=618 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:19:47.552 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:47.547+0000 7fd0426fe640 1 --2- 172.21.15.143:0/4069153475 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fd03c0ce680 0x7fd03c156360 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:19:47.553 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:47.547+0000 7fd0426fe640 1 --2- 172.21.15.143:0/4069153475 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fd03c0b2f80 0x7fd03c13c1d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:19:47.553 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:47.547+0000 7fd0426fe640 1 -- 172.21.15.143:0/4069153475 >> 172.21.15.143:0/4069153475 conn(0x7fd03c06a890 msgr2=0x7fd03c15d7e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:19:47.553 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:47.547+0000 7fd0426fe640 1 -- 172.21.15.143:0/4069153475 shutdown_connections 2023-11-26T07:19:47.553 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:47.547+0000 7fd0426fe640 1 -- 172.21.15.143:0/4069153475 wait complete. 2023-11-26T07:19:47.571 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-11-26T07:17:41.758597+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 6, 'pool_name': 'cephfs_metadata', 'create_time': '2023-11-26T07:19:29.208075+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '44', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 1.75, 'score_stable': 1.75, 'optimal_score': 1, 'raw_score_acting': 1.75, 'raw_score_stable': 1.75, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 7, 'pool_name': 'cephfs_data', 'create_time': '2023-11-26T07:19:30.503743+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '45', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.75, 'score_stable': 1.75, 'optimal_score': 1, 'raw_score_acting': 1.75, 'raw_score_stable': 1.75, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2023-11-26T07:19:47.572 DEBUG:teuthology.orchestra.run.smithi143:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-11-26T07:19:47.712 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:47.707+0000 7f4e60668640 1 Processor -- start 2023-11-26T07:19:47.712 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:47.707+0000 7f4e60668640 1 -- start start 2023-11-26T07:19:47.713 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:47.711+0000 7f4e60668640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f4e580ce7c0 0x7f4e5815f6b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:19:47.713 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:47.711+0000 7f4e60668640 1 -- --> v1:172.21.15.143:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4e5806c520 con 0x7f4e5813e290 2023-11-26T07:19:47.713 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:47.711+0000 7f4e60668640 1 -- --> v1:172.21.15.188:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4e5806c800 con 0x7f4e580cf990 2023-11-26T07:19:47.714 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:47.711+0000 7f4e60668640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4e5806cae0 con 0x7f4e580ce7c0 2023-11-26T07:19:47.714 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:47.711+0000 7f4e5e404640 1 --1- >> v1:172.21.15.143:6789/0 conn(0x7f4e5813e290 0x7f4e580cf280 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.143:6789/0 says I am v1:172.21.15.143:50432/0 (socket says 172.21.15.143:50432) 2023-11-26T07:19:47.714 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:47.711+0000 7f4e5e404640 1 -- 172.21.15.143:0/3872460118 learned_addr learned my addr 172.21.15.143:0/3872460118 (peer_addr_for_me v1:172.21.15.143:0/0) 2023-11-26T07:19:47.715 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:47.711+0000 7f4e5ec05640 1 --2- 172.21.15.143:0/3872460118 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f4e580ce7c0 0x7f4e5815f6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:19:47.716 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:47.711+0000 7f4e5d402640 1 -- 172.21.15.143:0/3872460118 <== mon.0 v1:172.21.15.143:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2690014978 0 0) 0x7f4e5806c520 con 0x7f4e5813e290 2023-11-26T07:19:47.716 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:47.711+0000 7f4e5d402640 1 -- 172.21.15.143:0/3872460118 --> v1:172.21.15.143:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f4e44007940 con 0x7f4e5813e290 2023-11-26T07:19:47.716 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:47.711+0000 7f4e5d402640 1 -- 172.21.15.143:0/3872460118 <== mon.0 v1:172.21.15.143:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2757179736 0 0) 0x7f4e44007940 con 0x7f4e5813e290 2023-11-26T07:19:47.716 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:47.711+0000 7f4e5d402640 1 -- 172.21.15.143:0/3872460118 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f4e580ce7c0 msgr2=0x7f4e5815f6b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-26T07:19:47.716 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:47.711+0000 7f4e5d402640 1 --2- 172.21.15.143:0/3872460118 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f4e580ce7c0 0x7f4e5815f6b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:19:47.716 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:47.711+0000 7f4e5d402640 1 -- 172.21.15.143:0/3872460118 >> v1:172.21.15.188:6789/0 conn(0x7f4e580cf990 legacy=0x7f4e5815efa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:19:47.716 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:47.711+0000 7f4e5d402640 1 -- 172.21.15.143:0/3872460118 --> v1:172.21.15.143:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4e5806cdc0 con 0x7f4e5813e290 2023-11-26T07:19:47.716 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:47.711+0000 7f4e5d402640 1 -- 172.21.15.143:0/3872460118 <== mon.0 v1:172.21.15.143:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2137882523 0 0) 0x7f4e54001680 con 0x7f4e5813e290 2023-11-26T07:19:47.716 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:47.711+0000 7f4e5d402640 1 -- 172.21.15.143:0/3872460118 >> v1:172.21.15.143:6789/0 conn(0x7f4e5813e290 legacy=0x7f4e580cf280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:19:47.716 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:47.711+0000 7f4e5d402640 1 --2- 172.21.15.143:0/3872460118 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f4e44007dc0 0x7f4e440081a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:19:47.717 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:47.711+0000 7f4e5d402640 1 --2- 172.21.15.143:0/3872460118 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f4e580ce7c0 0x7f4e44008be0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:19:47.717 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:47.711+0000 7f4e5d402640 1 --2- 172.21.15.143:0/3872460118 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f4e44009120 0x7f4e440095a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:19:47.717 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:47.711+0000 7f4e5d402640 1 -- 172.21.15.143:0/3872460118 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4e5806cae0 con 0x7f4e44007dc0 2023-11-26T07:19:47.717 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:47.711+0000 7f4e5d402640 1 -- 172.21.15.143:0/3872460118 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4e4400fda0 con 0x7f4e44009120 2023-11-26T07:19:47.717 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:47.711+0000 7f4e5d402640 1 -- 172.21.15.143:0/3872460118 --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4e4400ff60 con 0x7f4e580ce7c0 2023-11-26T07:19:47.717 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:47.711+0000 7f4e5e404640 1 --2- 172.21.15.143:0/3872460118 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f4e44007dc0 0x7f4e440081a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:19:47.717 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:47.711+0000 7f4e5dc03640 1 --2- 172.21.15.143:0/3872460118 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f4e580ce7c0 0x7f4e44008be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:19:47.717 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:47.711+0000 7f4e5ec05640 1 --2- 172.21.15.143:0/3872460118 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f4e44009120 0x7f4e440095a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:19:47.717 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:47.711+0000 7f4e5e404640 1 -- 172.21.15.143:0/3872460118 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f4e580ce7c0 msgr2=0x7f4e44008be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-26T07:19:47.717 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:47.711+0000 7f4e5e404640 1 --2- 172.21.15.143:0/3872460118 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f4e580ce7c0 0x7f4e44008be0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:19:47.717 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:47.711+0000 7f4e5e404640 1 -- 172.21.15.143:0/3872460118 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f4e44009120 msgr2=0x7f4e440095a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-26T07:19:47.717 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:47.711+0000 7f4e5e404640 1 --2- 172.21.15.143:0/3872460118 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f4e44009120 0x7f4e440095a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:19:47.718 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:47.711+0000 7f4e5e404640 1 -- 172.21.15.143:0/3872460118 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4e44007940 con 0x7f4e44007dc0 2023-11-26T07:19:47.719 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:47.715+0000 7f4e5e404640 1 --2- 172.21.15.143:0/3872460118 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f4e44007dc0 0x7f4e440081a0 secure :-1 s=READY pgs=620 cs=0 l=1 rev1=1 crypto rx=0x7f4e5400c210 tx=0x7f4e5401cb70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=65552d37e0c428ba server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:19:47.719 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:47.715+0000 7f4e5d402640 1 -- 172.21.15.143:0/3872460118 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4e540058e0 con 0x7f4e44007dc0 2023-11-26T07:19:47.719 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:47.715+0000 7f4e5d402640 1 -- 172.21.15.143:0/3872460118 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4e54005a60 con 0x7f4e44007dc0 2023-11-26T07:19:47.719 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:47.715+0000 7f4e5d402640 1 -- 172.21.15.143:0/3872460118 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4e54005e10 con 0x7f4e44007dc0 2023-11-26T07:19:47.720 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:47.715+0000 7f4e60668640 1 -- 172.21.15.143:0/3872460118 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f4e44007dc0 msgr2=0x7f4e440081a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:19:47.720 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:47.715+0000 7f4e60668640 1 --2- 172.21.15.143:0/3872460118 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f4e44007dc0 0x7f4e440081a0 secure :-1 s=READY pgs=620 cs=0 l=1 rev1=1 crypto rx=0x7f4e5400c210 tx=0x7f4e5401cb70 comp rx=0 tx=0).stop 2023-11-26T07:19:47.720 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:47.715+0000 7f4e5ec05640 1 -- 172.21.15.143:0/3872460118 reap_dead start 2023-11-26T07:19:47.720 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:47.715+0000 7f4e60668640 1 -- 172.21.15.143:0/3872460118 shutdown_connections 2023-11-26T07:19:47.720 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:47.715+0000 7f4e60668640 1 -- 172.21.15.143:0/3872460118 >> 172.21.15.143:0/3872460118 conn(0x7f4e5806a890 msgr2=0x7f4e5806aca0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:19:47.720 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:47.715+0000 7f4e60668640 1 -- 172.21.15.143:0/3872460118 shutdown_connections 2023-11-26T07:19:47.720 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:47.715+0000 7f4e60668640 1 -- 172.21.15.143:0/3872460118 wait complete. 2023-11-26T07:19:47.721 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:47.719+0000 7f4e60668640 1 Processor -- start 2023-11-26T07:19:47.722 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:47.719+0000 7f4e60668640 1 -- start start 2023-11-26T07:19:47.723 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:47.719+0000 7f4e60668640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f4e44007dc0 0x7f4e5815a7c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:19:47.723 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:47.719+0000 7f4e60668640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f4e5813b140 0x7f4e580b2da0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:19:47.723 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:47.719+0000 7f4e60668640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f4e5813b670 0x7f4e58155d20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:19:47.723 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:47.719+0000 7f4e60668640 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4e58162be0 con 0x7f4e5813b670 2023-11-26T07:19:47.723 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:47.719+0000 7f4e60668640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4e581633f0 con 0x7f4e5813b140 2023-11-26T07:19:47.723 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:47.719+0000 7f4e60668640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4e58162e30 con 0x7f4e44007dc0 2023-11-26T07:19:47.723 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:47.719+0000 7f4e5ec05640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f4e5813b670 0x7f4e58155d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:19:47.723 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:47.719+0000 7f4e5ec05640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f4e5813b670 0x7f4e58155d20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:58196/0 (socket says 172.21.15.143:58196) 2023-11-26T07:19:47.724 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:47.719+0000 7f4e5ec05640 1 -- 172.21.15.143:0/1894060018 learned_addr learned my addr 172.21.15.143:0/1894060018 (peer_addr_for_me v2:172.21.15.143:0/0) 2023-11-26T07:19:47.724 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:47.719+0000 7f4e5dc03640 1 --2- 172.21.15.143:0/1894060018 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f4e5813b140 0x7f4e580b2da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:19:47.724 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:47.719+0000 7f4e5e404640 1 --2- 172.21.15.143:0/1894060018 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f4e44007dc0 0x7f4e5815a7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:19:47.725 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:47.719+0000 7f4e5ec05640 1 -- 172.21.15.143:0/1894060018 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f4e44007dc0 msgr2=0x7f4e5815a7c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:19:47.725 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:47.719+0000 7f4e5ec05640 1 --2- 172.21.15.143:0/1894060018 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f4e44007dc0 0x7f4e5815a7c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:19:47.725 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:47.719+0000 7f4e5ec05640 1 -- 172.21.15.143:0/1894060018 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f4e5813b140 msgr2=0x7f4e580b2da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:19:47.725 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:47.719+0000 7f4e5ec05640 1 --2- 172.21.15.143:0/1894060018 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f4e5813b140 0x7f4e580b2da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:19:47.725 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:47.719+0000 7f4e5ec05640 1 -- 172.21.15.143:0/1894060018 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4e58163110 con 0x7f4e5813b670 2023-11-26T07:19:47.725 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:47.719+0000 7f4e5ec05640 1 --2- 172.21.15.143:0/1894060018 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f4e5813b670 0x7f4e58155d20 secure :-1 s=READY pgs=621 cs=0 l=1 rev1=1 crypto rx=0x7f4e580b1c00 tx=0x7f4e5000f580 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:19:47.726 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:47.719+0000 7f4e4effd640 1 -- 172.21.15.143:0/1894060018 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4e5000e040 con 0x7f4e5813b670 2023-11-26T07:19:47.726 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:47.719+0000 7f4e4effd640 1 -- 172.21.15.143:0/1894060018 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4e5000f040 con 0x7f4e5813b670 2023-11-26T07:19:47.726 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:47.719+0000 7f4e4effd640 1 -- 172.21.15.143:0/1894060018 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4e50015430 con 0x7f4e5813b670 2023-11-26T07:19:47.726 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:47.719+0000 7f4e60668640 1 -- 172.21.15.143:0/1894060018 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4e58159fc0 con 0x7f4e5813b670 2023-11-26T07:19:47.726 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:47.719+0000 7f4e60668640 1 -- 172.21.15.143:0/1894060018 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4e5815ad00 con 0x7f4e5813b670 2023-11-26T07:19:47.726 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:47.723+0000 7f4e60668640 1 -- 172.21.15.143:0/1894060018 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4e5815a4e0 con 0x7f4e5813b670 2023-11-26T07:19:47.727 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:47.723+0000 7f4e4effd640 1 -- 172.21.15.143:0/1894060018 <== mon.0 v2:172.21.15.143:3300/0 4 ==== mgrmap(e 7) v1 ==== 86005+0+0 (secure 0 0 0) 0x7f4e50015590 con 0x7f4e5813b670 2023-11-26T07:19:47.727 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:47.723+0000 7f4e4effd640 1 --2- 172.21.15.143:0/1894060018 >> [v2:172.21.15.143:6832/2150557544,v1:172.21.15.143:6833/2150557544] conn(0x7f4e3c066620 0x7f4e3c068ae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:19:47.727 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:47.723+0000 7f4e4effd640 1 -- 172.21.15.143:0/1894060018 <== mon.0 v2:172.21.15.143:3300/0 5 ==== osd_map(45..45 src has 1..45) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f4e500846f0 con 0x7f4e5813b670 2023-11-26T07:19:47.727 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:47.723+0000 7f4e5e404640 1 --2- 172.21.15.143:0/1894060018 >> [v2:172.21.15.143:6832/2150557544,v1:172.21.15.143:6833/2150557544] conn(0x7f4e3c066620 0x7f4e3c068ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:19:47.728 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:47.723+0000 7f4e5e404640 1 --2- 172.21.15.143:0/1894060018 >> [v2:172.21.15.143:6832/2150557544,v1:172.21.15.143:6833/2150557544] conn(0x7f4e3c066620 0x7f4e3c068ae0 secure :-1 s=READY pgs=305 cs=0 l=1 rev1=1 crypto rx=0x7f4e540ab000 tx=0x7f4e540a7070 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:19:47.730 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:47.727+0000 7f4e4effd640 1 -- 172.21.15.143:0/1894060018 <== mon.0 v2:172.21.15.143:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172649 (secure 0 0 0) 0x7f4e5815a4e0 con 0x7f4e5813b670 2023-11-26T07:19:47.882 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:47.879+0000 7f4e60668640 1 -- 172.21.15.143:0/1894060018 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f4e5815a4e0 con 0x7f4e5813b670 2023-11-26T07:19:47.882 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:47.879+0000 7f4e4effd640 1 -- 172.21.15.143:0/1894060018 <== mon.0 v2:172.21.15.143:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v45) v1 ==== 93+0+10 (secure 0 0 0) 0x7f4e5815a4e0 con 0x7f4e5813b670 2023-11-26T07:19:47.882 INFO:teuthology.orchestra.run.smithi143.stdout:pg_num: 1 2023-11-26T07:19:47.884 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:47.879+0000 7f4e60668640 1 -- 172.21.15.143:0/1894060018 >> [v2:172.21.15.143:6832/2150557544,v1:172.21.15.143:6833/2150557544] conn(0x7f4e3c066620 msgr2=0x7f4e3c068ae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:19:47.884 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:47.879+0000 7f4e60668640 1 --2- 172.21.15.143:0/1894060018 >> [v2:172.21.15.143:6832/2150557544,v1:172.21.15.143:6833/2150557544] conn(0x7f4e3c066620 0x7f4e3c068ae0 secure :-1 s=READY pgs=305 cs=0 l=1 rev1=1 crypto rx=0x7f4e540ab000 tx=0x7f4e540a7070 comp rx=0 tx=0).stop 2023-11-26T07:19:47.885 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:47.879+0000 7f4e60668640 1 -- 172.21.15.143:0/1894060018 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f4e5813b670 msgr2=0x7f4e58155d20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:19:47.885 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:47.879+0000 7f4e60668640 1 --2- 172.21.15.143:0/1894060018 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f4e5813b670 0x7f4e58155d20 secure :-1 s=READY pgs=621 cs=0 l=1 rev1=1 crypto rx=0x7f4e580b1c00 tx=0x7f4e5000f580 comp rx=0 tx=0).stop 2023-11-26T07:19:47.885 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:47.879+0000 7f4e60668640 1 -- 172.21.15.143:0/1894060018 shutdown_connections 2023-11-26T07:19:47.885 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:47.879+0000 7f4e60668640 1 --2- 172.21.15.143:0/1894060018 >> [v2:172.21.15.143:6832/2150557544,v1:172.21.15.143:6833/2150557544] conn(0x7f4e3c066620 0x7f4e3c068ae0 unknown :-1 s=CLOSED pgs=305 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:19:47.885 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:47.879+0000 7f4e60668640 1 --2- 172.21.15.143:0/1894060018 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f4e5813b670 0x7f4e58155d20 unknown :-1 s=CLOSED pgs=621 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:19:47.885 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:47.879+0000 7f4e60668640 1 --2- 172.21.15.143:0/1894060018 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f4e5813b140 0x7f4e580b2da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:19:47.885 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:47.879+0000 7f4e60668640 1 --2- 172.21.15.143:0/1894060018 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f4e44007dc0 0x7f4e5815a7c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:19:47.885 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:47.879+0000 7f4e60668640 1 -- 172.21.15.143:0/1894060018 >> 172.21.15.143:0/1894060018 conn(0x7f4e5806a890 msgr2=0x7f4e580b0b50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:19:47.885 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:47.879+0000 7f4e60668640 1 -- 172.21.15.143:0/1894060018 shutdown_connections 2023-11-26T07:19:47.885 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:47.879+0000 7f4e60668640 1 -- 172.21.15.143:0/1894060018 wait complete. 2023-11-26T07:19:47.903 DEBUG:teuthology.orchestra.run.smithi143:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-11-26T07:19:48.043 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:48.039+0000 7fed2a032640 1 Processor -- start 2023-11-26T07:19:48.043 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:48.039+0000 7fed2a032640 1 -- start start 2023-11-26T07:19:48.045 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:48.039+0000 7fed2a032640 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7fed240ce680 0x7fed24071850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:19:48.045 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:48.039+0000 7fed2a032640 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7fed240b2870 0x7fed240b2cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:19:48.045 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:48.039+0000 7fed2a032640 1 -- --> v1:172.21.15.143:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fed240b3200 con 0x7fed241596d0 2023-11-26T07:19:48.045 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:48.039+0000 7fed2a032640 1 -- --> v2:172.21.15.143:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fed240cbe00 con 0x7fed240b2870 2023-11-26T07:19:48.046 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:48.039+0000 7fed2a032640 1 -- --> v2:172.21.15.188:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fed240cf990 con 0x7fed240ce680 2023-11-26T07:19:48.046 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:48.043+0000 7fed22ffd640 1 --1- >> v1:172.21.15.143:6789/0 conn(0x7fed241596d0 0x7fed240cf280 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.143:6789/0 says I am v1:172.21.15.143:50440/0 (socket says 172.21.15.143:50440) 2023-11-26T07:19:48.046 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:48.043+0000 7fed22ffd640 1 -- 172.21.15.143:0/2907746289 learned_addr learned my addr 172.21.15.143:0/2907746289 (peer_addr_for_me v1:172.21.15.143:0/0) 2023-11-26T07:19:48.046 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:48.043+0000 7fed23fff640 1 --2- 172.21.15.143:0/2907746289 >> v2:172.21.15.143:3300/0 conn(0x7fed240b2870 0x7fed240b2cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:19:48.046 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:48.043+0000 7fed237fe640 1 --2- 172.21.15.143:0/2907746289 >> v2:172.21.15.188:3300/0 conn(0x7fed240ce680 0x7fed24071850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:19:48.046 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:48.043+0000 7fed23fff640 1 -- 172.21.15.143:0/2907746289 >> v2:172.21.15.188:3300/0 conn(0x7fed240ce680 msgr2=0x7fed24071850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-26T07:19:48.046 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:48.043+0000 7fed23fff640 1 --2- 172.21.15.143:0/2907746289 >> v2:172.21.15.188:3300/0 conn(0x7fed240ce680 0x7fed24071850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:19:48.046 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:48.043+0000 7fed23fff640 1 -- 172.21.15.143:0/2907746289 >> v1:172.21.15.143:6789/0 conn(0x7fed241596d0 legacy=0x7fed240cf280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:19:48.046 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:48.043+0000 7fed23fff640 1 -- 172.21.15.143:0/2907746289 --> v2:172.21.15.143:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fed2406c750 con 0x7fed240b2870 2023-11-26T07:19:48.048 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:48.043+0000 7fed23fff640 1 --2- 172.21.15.143:0/2907746289 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fed240b2870 0x7fed240b2cc0 secure :-1 s=READY pgs=623 cs=0 l=1 rev1=1 crypto rx=0x7fed2406e4d0 tx=0x7fed1c01c910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=879d2f8222c1983a server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:19:48.048 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:48.043+0000 7fed227fc640 1 -- 172.21.15.143:0/2907746289 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fed1c0a57d0 con 0x7fed240b2870 2023-11-26T07:19:48.048 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:48.043+0000 7fed227fc640 1 -- 172.21.15.143:0/2907746289 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fed1c0a5930 con 0x7fed240b2870 2023-11-26T07:19:48.048 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:48.043+0000 7fed227fc640 1 -- 172.21.15.143:0/2907746289 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fed1c0a5cc0 con 0x7fed240b2870 2023-11-26T07:19:48.049 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:48.047+0000 7fed2a032640 1 -- 172.21.15.143:0/2907746289 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fed240b2870 msgr2=0x7fed240b2cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:19:48.049 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:48.047+0000 7fed2a032640 1 --2- 172.21.15.143:0/2907746289 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fed240b2870 0x7fed240b2cc0 secure :-1 s=READY pgs=623 cs=0 l=1 rev1=1 crypto rx=0x7fed2406e4d0 tx=0x7fed1c01c910 comp rx=0 tx=0).stop 2023-11-26T07:19:48.049 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:48.047+0000 7fed2a032640 1 -- 172.21.15.143:0/2907746289 shutdown_connections 2023-11-26T07:19:48.049 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:48.047+0000 7fed2a032640 1 --2- 172.21.15.143:0/2907746289 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fed240b2870 0x7fed240b2cc0 unknown :-1 s=CLOSED pgs=623 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:19:48.049 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:48.047+0000 7fed2a032640 1 --2- 172.21.15.143:0/2907746289 >> v2:172.21.15.188:3300/0 conn(0x7fed240ce680 0x7fed24071850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:19:48.049 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:48.047+0000 7fed2a032640 1 -- 172.21.15.143:0/2907746289 >> 172.21.15.143:0/2907746289 conn(0x7fed2406a890 msgr2=0x7fed2406aca0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:19:48.050 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:48.047+0000 7fed2a032640 1 -- 172.21.15.143:0/2907746289 shutdown_connections 2023-11-26T07:19:48.050 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:48.047+0000 7fed2a032640 1 -- 172.21.15.143:0/2907746289 wait complete. 2023-11-26T07:19:48.050 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:48.047+0000 7fed2a032640 1 Processor -- start 2023-11-26T07:19:48.051 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:48.047+0000 7fed2a032640 1 -- start start 2023-11-26T07:19:48.053 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:48.047+0000 7fed2a032640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fed240b2870 0x7fed240b0000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:19:48.053 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:48.047+0000 7fed2a032640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fed240ce680 0x7fed24146010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:19:48.053 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:48.047+0000 7fed2a032640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fed241596d0 0x7fed2414dd70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:19:48.053 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:48.047+0000 7fed2a032640 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fed240afb70 con 0x7fed240b2870 2023-11-26T07:19:48.053 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:48.047+0000 7fed2a032640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fed241f9e30 con 0x7fed241596d0 2023-11-26T07:19:48.053 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:48.047+0000 7fed2a032640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fed240af820 con 0x7fed240ce680 2023-11-26T07:19:48.053 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:48.047+0000 7fed237fe640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fed240b2870 0x7fed240b0000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:19:48.053 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:48.047+0000 7fed237fe640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fed240b2870 0x7fed240b0000 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:58222/0 (socket says 172.21.15.143:58222) 2023-11-26T07:19:48.053 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:48.047+0000 7fed237fe640 1 -- 172.21.15.143:0/536009039 learned_addr learned my addr 172.21.15.143:0/536009039 (peer_addr_for_me v2:172.21.15.143:0/0) 2023-11-26T07:19:48.054 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:48.047+0000 7fed23fff640 1 --2- 172.21.15.143:0/536009039 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fed241596d0 0x7fed2414dd70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:19:48.054 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:48.047+0000 7fed22ffd640 1 --2- 172.21.15.143:0/536009039 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fed240ce680 0x7fed24146010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:19:48.054 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:48.051+0000 7fed237fe640 1 -- 172.21.15.143:0/536009039 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fed240ce680 msgr2=0x7fed24146010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:19:48.054 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:48.051+0000 7fed237fe640 1 --2- 172.21.15.143:0/536009039 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fed240ce680 0x7fed24146010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:19:48.054 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:48.051+0000 7fed237fe640 1 -- 172.21.15.143:0/536009039 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fed241596d0 msgr2=0x7fed2414dd70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:19:48.054 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:48.051+0000 7fed237fe640 1 --2- 172.21.15.143:0/536009039 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fed241596d0 0x7fed2414dd70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:19:48.054 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:48.051+0000 7fed237fe640 1 -- 172.21.15.143:0/536009039 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fed1c009a70 con 0x7fed240b2870 2023-11-26T07:19:48.054 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:48.051+0000 7fed23fff640 1 --2- 172.21.15.143:0/536009039 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fed241596d0 0x7fed2414dd70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-26T07:19:48.054 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:48.051+0000 7fed22ffd640 1 --2- 172.21.15.143:0/536009039 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fed240ce680 0x7fed24146010 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-26T07:19:48.054 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:48.051+0000 7fed237fe640 1 --2- 172.21.15.143:0/536009039 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fed240b2870 0x7fed240b0000 secure :-1 s=READY pgs=624 cs=0 l=1 rev1=1 crypto rx=0x7fed1400e930 tx=0x7fed1400edf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:19:48.055 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:48.051+0000 7fed0bfff640 1 -- 172.21.15.143:0/536009039 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fed14009690 con 0x7fed240b2870 2023-11-26T07:19:48.055 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:48.051+0000 7fed0bfff640 1 -- 172.21.15.143:0/536009039 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fed1400ca00 con 0x7fed240b2870 2023-11-26T07:19:48.055 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:48.051+0000 7fed0bfff640 1 -- 172.21.15.143:0/536009039 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fed1400cd70 con 0x7fed240b2870 2023-11-26T07:19:48.055 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:48.051+0000 7fed2a032640 1 -- 172.21.15.143:0/536009039 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fed241f6260 con 0x7fed240b2870 2023-11-26T07:19:48.055 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:48.051+0000 7fed2a032640 1 -- 172.21.15.143:0/536009039 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fed241f6820 con 0x7fed240b2870 2023-11-26T07:19:48.055 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:48.051+0000 7fed2a032640 1 -- 172.21.15.143:0/536009039 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fed24067680 con 0x7fed240b2870 2023-11-26T07:19:48.056 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:48.051+0000 7fed0bfff640 1 -- 172.21.15.143:0/536009039 <== mon.0 v2:172.21.15.143:3300/0 4 ==== mgrmap(e 7) v1 ==== 86005+0+0 (secure 0 0 0) 0x7fed14013470 con 0x7fed240b2870 2023-11-26T07:19:48.056 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:48.051+0000 7fed0bfff640 1 --2- 172.21.15.143:0/536009039 >> [v2:172.21.15.143:6832/2150557544,v1:172.21.15.143:6833/2150557544] conn(0x7fed10066650 0x7fed10068b10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:19:48.056 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:48.051+0000 7fed0bfff640 1 -- 172.21.15.143:0/536009039 <== mon.0 v2:172.21.15.143:3300/0 5 ==== osd_map(45..45 src has 1..45) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fed140844e0 con 0x7fed240b2870 2023-11-26T07:19:48.056 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:48.051+0000 7fed22ffd640 1 --2- 172.21.15.143:0/536009039 >> [v2:172.21.15.143:6832/2150557544,v1:172.21.15.143:6833/2150557544] conn(0x7fed10066650 0x7fed10068b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:19:48.057 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:48.055+0000 7fed22ffd640 1 --2- 172.21.15.143:0/536009039 >> [v2:172.21.15.143:6832/2150557544,v1:172.21.15.143:6833/2150557544] conn(0x7fed10066650 0x7fed10068b10 secure :-1 s=READY pgs=306 cs=0 l=1 rev1=1 crypto rx=0x7fed240adbc0 tx=0x7fed18009040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:19:48.059 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:48.055+0000 7fed0bfff640 1 -- 172.21.15.143:0/536009039 <== mon.0 v2:172.21.15.143:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172649 (secure 0 0 0) 0x7fed14051700 con 0x7fed240b2870 2023-11-26T07:19:48.215 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:48.211+0000 7fed2a032640 1 -- 172.21.15.143:0/536009039 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7fed240709f0 con 0x7fed240b2870 2023-11-26T07:19:48.215 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:48.211+0000 7fed0bfff640 1 -- 172.21.15.143:0/536009039 <== mon.0 v2:172.21.15.143:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v45) v1 ==== 104+0+11 (secure 0 0 0) 0x7fed14054d20 con 0x7fed240b2870 2023-11-26T07:19:48.216 INFO:teuthology.orchestra.run.smithi143.stdout:pg_num: 32 2023-11-26T07:19:48.218 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:48.215+0000 7fed2a032640 1 -- 172.21.15.143:0/536009039 >> [v2:172.21.15.143:6832/2150557544,v1:172.21.15.143:6833/2150557544] conn(0x7fed10066650 msgr2=0x7fed10068b10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:19:48.218 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:48.215+0000 7fed2a032640 1 --2- 172.21.15.143:0/536009039 >> [v2:172.21.15.143:6832/2150557544,v1:172.21.15.143:6833/2150557544] conn(0x7fed10066650 0x7fed10068b10 secure :-1 s=READY pgs=306 cs=0 l=1 rev1=1 crypto rx=0x7fed240adbc0 tx=0x7fed18009040 comp rx=0 tx=0).stop 2023-11-26T07:19:48.218 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:48.215+0000 7fed2a032640 1 -- 172.21.15.143:0/536009039 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fed240b2870 msgr2=0x7fed240b0000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:19:48.218 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:48.215+0000 7fed2a032640 1 --2- 172.21.15.143:0/536009039 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fed240b2870 0x7fed240b0000 secure :-1 s=READY pgs=624 cs=0 l=1 rev1=1 crypto rx=0x7fed1400e930 tx=0x7fed1400edf0 comp rx=0 tx=0).stop 2023-11-26T07:19:48.218 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:48.215+0000 7fed2a032640 1 -- 172.21.15.143:0/536009039 shutdown_connections 2023-11-26T07:19:48.218 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:48.215+0000 7fed2a032640 1 --2- 172.21.15.143:0/536009039 >> [v2:172.21.15.143:6832/2150557544,v1:172.21.15.143:6833/2150557544] conn(0x7fed10066650 0x7fed10068b10 unknown :-1 s=CLOSED pgs=306 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:19:48.218 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:48.215+0000 7fed2a032640 1 --2- 172.21.15.143:0/536009039 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fed241596d0 0x7fed2414dd70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:19:48.219 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:48.215+0000 7fed2a032640 1 --2- 172.21.15.143:0/536009039 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fed240ce680 0x7fed24146010 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:19:48.219 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:48.215+0000 7fed2a032640 1 --2- 172.21.15.143:0/536009039 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fed240b2870 0x7fed240b0000 unknown :-1 s=CLOSED pgs=624 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:19:48.219 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:48.215+0000 7fed2a032640 1 -- 172.21.15.143:0/536009039 >> 172.21.15.143:0/536009039 conn(0x7fed2406a890 msgr2=0x7fed240cb0e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:19:48.219 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:48.215+0000 7fed2a032640 1 -- 172.21.15.143:0/536009039 shutdown_connections 2023-11-26T07:19:48.219 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:48.215+0000 7fed2a032640 1 -- 172.21.15.143:0/536009039 wait complete. 2023-11-26T07:19:48.237 DEBUG:teuthology.orchestra.run.smithi143:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-11-26T07:19:48.378 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:48.375+0000 7fa8837c1640 1 Processor -- start 2023-11-26T07:19:48.378 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:48.375+0000 7fa8837c1640 1 -- start start 2023-11-26T07:19:48.379 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:48.375+0000 7fa8837c1640 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7fa87c14b1e0 0x7fa87c0cf280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:19:48.379 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:48.375+0000 7fa8837c1640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fa87c070fd0 0x7fa87c162510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:19:48.380 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:48.375+0000 7fa8837c1640 1 -- --> v1:172.21.15.143:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa87c0cfc20 con 0x7fa87c0cf7c0 2023-11-26T07:19:48.380 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:48.375+0000 7fa8837c1640 1 -- --> v2:172.21.15.143:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa87c162080 con 0x7fa87c14b1e0 2023-11-26T07:19:48.380 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:48.375+0000 7fa8837c1640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa87c161d70 con 0x7fa87c070fd0 2023-11-26T07:19:48.380 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:48.375+0000 7fa880d5c640 1 --1- >> v1:172.21.15.143:6789/0 conn(0x7fa87c0cf7c0 0x7fa87c0708c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.143:6789/0 says I am v1:172.21.15.143:50452/0 (socket says 172.21.15.143:50452) 2023-11-26T07:19:48.380 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:48.375+0000 7fa88155d640 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7fa87c14b1e0 0x7fa87c0cf280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:19:48.380 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:48.375+0000 7fa880d5c640 1 -- 172.21.15.143:0/965666012 learned_addr learned my addr 172.21.15.143:0/965666012 (peer_addr_for_me v1:172.21.15.143:0/0) 2023-11-26T07:19:48.380 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:48.375+0000 7fa881d5e640 1 --2- 172.21.15.143:0/965666012 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fa87c070fd0 0x7fa87c162510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:19:48.381 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:48.375+0000 7fa88155d640 1 -- 172.21.15.143:0/965666012 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fa87c070fd0 msgr2=0x7fa87c162510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-26T07:19:48.381 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:48.375+0000 7fa88155d640 1 --2- 172.21.15.143:0/965666012 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fa87c070fd0 0x7fa87c162510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:19:48.381 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:48.375+0000 7fa88155d640 1 -- 172.21.15.143:0/965666012 >> v1:172.21.15.143:6789/0 conn(0x7fa87c0cf7c0 legacy=0x7fa87c0708c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:19:48.381 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:48.375+0000 7fa88155d640 1 -- 172.21.15.143:0/965666012 --> v2:172.21.15.143:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa87c14b5c0 con 0x7fa87c14b1e0 2023-11-26T07:19:48.381 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:48.375+0000 7fa873fff640 1 -- 172.21.15.143:0/965666012 <== mon.0 v1:172.21.15.143:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 654350189 0 0) 0x7fa87c0cfc20 con 0x7fa87c0cf7c0 2023-11-26T07:19:48.383 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:48.379+0000 7fa88155d640 1 --2- 172.21.15.143:0/965666012 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fa87c14b1e0 0x7fa87c0cf280 secure :-1 s=READY pgs=626 cs=0 l=1 rev1=1 crypto rx=0x7fa8780097c0 tx=0x7fa87801c450 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8015a2d69897246e server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:19:48.383 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:48.379+0000 7fa873fff640 1 -- 172.21.15.143:0/965666012 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa8780ab650 con 0x7fa87c14b1e0 2023-11-26T07:19:48.383 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:48.379+0000 7fa873fff640 1 -- 172.21.15.143:0/965666012 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa8780ab7b0 con 0x7fa87c14b1e0 2023-11-26T07:19:48.383 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:48.379+0000 7fa873fff640 1 -- 172.21.15.143:0/965666012 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa8780abb40 con 0x7fa87c14b1e0 2023-11-26T07:19:48.384 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:48.379+0000 7fa8837c1640 1 -- 172.21.15.143:0/965666012 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fa87c14b1e0 msgr2=0x7fa87c0cf280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:19:48.384 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:48.379+0000 7fa8837c1640 1 --2- 172.21.15.143:0/965666012 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fa87c14b1e0 0x7fa87c0cf280 secure :-1 s=READY pgs=626 cs=0 l=1 rev1=1 crypto rx=0x7fa8780097c0 tx=0x7fa87801c450 comp rx=0 tx=0).stop 2023-11-26T07:19:48.384 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:48.379+0000 7fa8837c1640 1 -- 172.21.15.143:0/965666012 shutdown_connections 2023-11-26T07:19:48.384 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:48.379+0000 7fa8837c1640 1 --2- 172.21.15.143:0/965666012 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fa87c070fd0 0x7fa87c162510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:19:48.384 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:48.379+0000 7fa8837c1640 1 --2- 172.21.15.143:0/965666012 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fa87c14b1e0 0x7fa87c0cf280 unknown :-1 s=CLOSED pgs=626 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:19:48.384 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:48.379+0000 7fa8837c1640 1 -- 172.21.15.143:0/965666012 >> 172.21.15.143:0/965666012 conn(0x7fa87c06a890 msgr2=0x7fa87c06aca0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:19:48.384 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:48.379+0000 7fa8837c1640 1 -- 172.21.15.143:0/965666012 shutdown_connections 2023-11-26T07:19:48.384 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:48.379+0000 7fa8837c1640 1 -- 172.21.15.143:0/965666012 wait complete. 2023-11-26T07:19:48.385 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:48.383+0000 7fa8837c1640 1 Processor -- start 2023-11-26T07:19:48.385 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:48.383+0000 7fa8837c1640 1 -- start start 2023-11-26T07:19:48.387 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:48.383+0000 7fa8837c1640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fa87c070fd0 0x7fa87c15faf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:19:48.387 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:48.383+0000 7fa8837c1640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fa87c0cf7c0 0x7fa87c0a96e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:19:48.387 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:48.383+0000 7fa8837c1640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fa87c14b1e0 0x7fa87c1430b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:19:48.388 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:48.383+0000 7fa8837c1640 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa87c0a9c20 con 0x7fa87c070fd0 2023-11-26T07:19:48.388 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:48.383+0000 7fa8837c1640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa87c1435f0 con 0x7fa87c14b1e0 2023-11-26T07:19:48.388 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:48.383+0000 7fa8837c1640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa87c1601f0 con 0x7fa87c0cf7c0 2023-11-26T07:19:48.388 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:48.383+0000 7fa88155d640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fa87c070fd0 0x7fa87c15faf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:19:48.388 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:48.383+0000 7fa88155d640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fa87c070fd0 0x7fa87c15faf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:58234/0 (socket says 172.21.15.143:58234) 2023-11-26T07:19:48.388 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:48.383+0000 7fa88155d640 1 -- 172.21.15.143:0/3476437115 learned_addr learned my addr 172.21.15.143:0/3476437115 (peer_addr_for_me v2:172.21.15.143:0/0) 2023-11-26T07:19:48.388 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:48.383+0000 7fa880d5c640 1 --2- 172.21.15.143:0/3476437115 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fa87c0cf7c0 0x7fa87c0a96e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:19:48.388 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:48.383+0000 7fa881d5e640 1 --2- 172.21.15.143:0/3476437115 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fa87c14b1e0 0x7fa87c1430b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:19:48.388 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:48.383+0000 7fa88155d640 1 -- 172.21.15.143:0/3476437115 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fa87c0cf7c0 msgr2=0x7fa87c0a96e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:19:48.388 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:48.383+0000 7fa88155d640 1 --2- 172.21.15.143:0/3476437115 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fa87c0cf7c0 0x7fa87c0a96e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:19:48.388 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:48.383+0000 7fa88155d640 1 -- 172.21.15.143:0/3476437115 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fa87c14b1e0 msgr2=0x7fa87c1430b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:19:48.388 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:48.383+0000 7fa88155d640 1 --2- 172.21.15.143:0/3476437115 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fa87c14b1e0 0x7fa87c1430b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:19:48.389 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:48.383+0000 7fa88155d640 1 -- 172.21.15.143:0/3476437115 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa878009420 con 0x7fa87c070fd0 2023-11-26T07:19:48.389 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:48.383+0000 7fa881d5e640 1 --2- 172.21.15.143:0/3476437115 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fa87c14b1e0 0x7fa87c1430b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-26T07:19:48.389 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:48.383+0000 7fa880d5c640 1 --2- 172.21.15.143:0/3476437115 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fa87c0cf7c0 0x7fa87c0a96e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-26T07:19:48.389 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:48.383+0000 7fa88155d640 1 --2- 172.21.15.143:0/3476437115 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fa87c070fd0 0x7fa87c15faf0 secure :-1 s=READY pgs=627 cs=0 l=1 rev1=1 crypto rx=0x7fa8780ad040 tx=0x7fa87801ac90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:19:48.389 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:48.383+0000 7fa871ffb640 1 -- 172.21.15.143:0/3476437115 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa8780b5810 con 0x7fa87c070fd0 2023-11-26T07:19:48.390 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:48.383+0000 7fa871ffb640 1 -- 172.21.15.143:0/3476437115 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa8780b5990 con 0x7fa87c070fd0 2023-11-26T07:19:48.390 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:48.383+0000 7fa871ffb640 1 -- 172.21.15.143:0/3476437115 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa8780b5d40 con 0x7fa87c070fd0 2023-11-26T07:19:48.390 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:48.383+0000 7fa8837c1640 1 -- 172.21.15.143:0/3476437115 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa87c1604d0 con 0x7fa87c070fd0 2023-11-26T07:19:48.390 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:48.383+0000 7fa8837c1640 1 -- 172.21.15.143:0/3476437115 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa87c160a90 con 0x7fa87c070fd0 2023-11-26T07:19:48.390 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:48.387+0000 7fa8837c1640 1 -- 172.21.15.143:0/3476437115 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa87c162820 con 0x7fa87c070fd0 2023-11-26T07:19:48.390 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:48.387+0000 7fa871ffb640 1 -- 172.21.15.143:0/3476437115 <== mon.0 v2:172.21.15.143:3300/0 4 ==== mgrmap(e 7) v1 ==== 86005+0+0 (secure 0 0 0) 0x7fa8780b4390 con 0x7fa87c070fd0 2023-11-26T07:19:48.391 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:48.387+0000 7fa871ffb640 1 --2- 172.21.15.143:0/3476437115 >> [v2:172.21.15.143:6832/2150557544,v1:172.21.15.143:6833/2150557544] conn(0x7fa860066650 0x7fa860068b10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:19:48.391 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:48.387+0000 7fa871ffb640 1 -- 172.21.15.143:0/3476437115 <== mon.0 v2:172.21.15.143:3300/0 5 ==== osd_map(45..45 src has 1..45) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fa87811ec60 con 0x7fa87c070fd0 2023-11-26T07:19:48.391 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:48.387+0000 7fa880d5c640 1 --2- 172.21.15.143:0/3476437115 >> [v2:172.21.15.143:6832/2150557544,v1:172.21.15.143:6833/2150557544] conn(0x7fa860066650 0x7fa860068b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:19:48.392 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:48.387+0000 7fa880d5c640 1 --2- 172.21.15.143:0/3476437115 >> [v2:172.21.15.143:6832/2150557544,v1:172.21.15.143:6833/2150557544] conn(0x7fa860066650 0x7fa860068b10 secure :-1 s=READY pgs=307 cs=0 l=1 rev1=1 crypto rx=0x7fa87c0ae3d0 tx=0x7fa86c007c40 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:19:48.395 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:48.391+0000 7fa871ffb640 1 -- 172.21.15.143:0/3476437115 <== mon.0 v2:172.21.15.143:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172649 (secure 0 0 0) 0x7fa8780ebdc0 con 0x7fa87c070fd0 2023-11-26T07:19:48.548 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:48.543+0000 7fa8837c1640 1 -- 172.21.15.143:0/3476437115 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7fa87c067680 con 0x7fa87c070fd0 2023-11-26T07:19:48.548 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:48.543+0000 7fa871ffb640 1 -- 172.21.15.143:0/3476437115 <== mon.0 v2:172.21.15.143:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v45) v1 ==== 100+0+11 (secure 0 0 0) 0x7fa8780ab610 con 0x7fa87c070fd0 2023-11-26T07:19:48.548 INFO:teuthology.orchestra.run.smithi143.stdout:pg_num: 64 2023-11-26T07:19:48.550 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:48.547+0000 7fa8837c1640 1 -- 172.21.15.143:0/3476437115 >> [v2:172.21.15.143:6832/2150557544,v1:172.21.15.143:6833/2150557544] conn(0x7fa860066650 msgr2=0x7fa860068b10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:19:48.550 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:48.547+0000 7fa8837c1640 1 --2- 172.21.15.143:0/3476437115 >> [v2:172.21.15.143:6832/2150557544,v1:172.21.15.143:6833/2150557544] conn(0x7fa860066650 0x7fa860068b10 secure :-1 s=READY pgs=307 cs=0 l=1 rev1=1 crypto rx=0x7fa87c0ae3d0 tx=0x7fa86c007c40 comp rx=0 tx=0).stop 2023-11-26T07:19:48.550 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:48.547+0000 7fa8837c1640 1 -- 172.21.15.143:0/3476437115 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fa87c070fd0 msgr2=0x7fa87c15faf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:19:48.551 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:48.547+0000 7fa8837c1640 1 --2- 172.21.15.143:0/3476437115 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fa87c070fd0 0x7fa87c15faf0 secure :-1 s=READY pgs=627 cs=0 l=1 rev1=1 crypto rx=0x7fa8780ad040 tx=0x7fa87801ac90 comp rx=0 tx=0).stop 2023-11-26T07:19:48.551 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:48.547+0000 7fa8837c1640 1 -- 172.21.15.143:0/3476437115 shutdown_connections 2023-11-26T07:19:48.551 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:48.547+0000 7fa8837c1640 1 --2- 172.21.15.143:0/3476437115 >> [v2:172.21.15.143:6832/2150557544,v1:172.21.15.143:6833/2150557544] conn(0x7fa860066650 0x7fa860068b10 unknown :-1 s=CLOSED pgs=307 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:19:48.551 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:48.547+0000 7fa8837c1640 1 --2- 172.21.15.143:0/3476437115 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fa87c14b1e0 0x7fa87c1430b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:19:48.551 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:48.547+0000 7fa8837c1640 1 --2- 172.21.15.143:0/3476437115 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fa87c0cf7c0 0x7fa87c0a96e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:19:48.551 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:48.547+0000 7fa8837c1640 1 --2- 172.21.15.143:0/3476437115 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fa87c070fd0 0x7fa87c15faf0 unknown :-1 s=CLOSED pgs=627 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:19:48.551 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:48.547+0000 7fa8837c1640 1 -- 172.21.15.143:0/3476437115 >> 172.21.15.143:0/3476437115 conn(0x7fa87c06a890 msgr2=0x7fa87c0b0d20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:19:48.551 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:48.547+0000 7fa8837c1640 1 -- 172.21.15.143:0/3476437115 shutdown_connections 2023-11-26T07:19:48.551 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:48.547+0000 7fa8837c1640 1 -- 172.21.15.143:0/3476437115 wait complete. 2023-11-26T07:19:48.569 DEBUG:teuthology.orchestra.run.smithi143:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2023-11-26T07:19:48.708 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:48.703+0000 7f9ef4d72640 1 Processor -- start 2023-11-26T07:19:48.708 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:48.703+0000 7f9ef4d72640 1 -- start start 2023-11-26T07:19:48.709 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:48.707+0000 7f9ef4d72640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f9ef0143ac0 0x7f9ef0070370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:19:48.709 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:48.707+0000 7f9ef4d72640 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7f9ef00cf990 0x7f9ef00cd1b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:19:48.709 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:48.707+0000 7f9ef4d72640 1 -- --> v1:172.21.15.188:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9ef00cc730 con 0x7f9ef00708b0 2023-11-26T07:19:48.709 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:48.707+0000 7f9ef4d72640 1 -- --> v2:172.21.15.188:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9ef0143ea0 con 0x7f9ef00cf990 2023-11-26T07:19:48.710 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:48.707+0000 7f9ef4d72640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9ef0070ee0 con 0x7f9ef0143ac0 2023-11-26T07:19:48.710 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:48.707+0000 7f9eee59c640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f9ef0143ac0 0x7f9ef0070370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:19:48.711 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:48.707+0000 7f9eeed9d640 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7f9ef00cf990 0x7f9ef00cd1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:19:48.711 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:48.707+0000 7f9eee59c640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f9ef0143ac0 0x7f9ef0070370 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.188:3301/0 says I am v2:172.21.15.143:43244/0 (socket says 172.21.15.143:43244) 2023-11-26T07:19:48.711 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:48.707+0000 7f9eeed9d640 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7f9ef00cf990 0x7f9ef00cd1b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.188:3300/0 says I am v2:172.21.15.143:49624/0 (socket says 172.21.15.143:49624) 2023-11-26T07:19:48.711 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:48.707+0000 7f9eee59c640 1 -- 172.21.15.143:0/268969447 learned_addr learned my addr 172.21.15.143:0/268969447 (peer_addr_for_me v2:172.21.15.143:0/0) 2023-11-26T07:19:48.711 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:48.707+0000 7f9eee59c640 1 -- 172.21.15.143:0/268969447 >> v2:172.21.15.188:3300/0 conn(0x7f9ef00cf990 msgr2=0x7f9ef00cd1b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-26T07:19:48.711 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:48.707+0000 7f9eee59c640 1 --2- 172.21.15.143:0/268969447 >> v2:172.21.15.188:3300/0 conn(0x7f9ef00cf990 0x7f9ef00cd1b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:19:48.711 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:48.707+0000 7f9eee59c640 1 -- 172.21.15.143:0/268969447 >> v1:172.21.15.188:6789/0 conn(0x7f9ef00708b0 legacy=0x7f9ef00cf280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:19:48.711 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:48.707+0000 7f9eee59c640 1 -- 172.21.15.143:0/268969447 --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9ef00711c0 con 0x7f9ef0143ac0 2023-11-26T07:19:48.711 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:48.707+0000 7f9eeed9d640 1 --2- 172.21.15.143:0/268969447 >> v2:172.21.15.188:3300/0 conn(0x7f9ef00cf990 0x7f9ef00cd1b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-26T07:19:48.713 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:48.711+0000 7f9eee59c640 1 --2- 172.21.15.143:0/268969447 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f9ef0143ac0 0x7f9ef0070370 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7f9ee0009870 tx=0x7f9ee001c540 comp rx=0 tx=0).ready entity=mon.2 client_cookie=fecc925b560af872 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:19:48.713 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:48.711+0000 7f9eed59a640 1 -- 172.21.15.143:0/268969447 <== mon.2 v2:172.21.15.188:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9ee00ab650 con 0x7f9ef0143ac0 2023-11-26T07:19:48.714 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:48.711+0000 7f9eed59a640 1 -- 172.21.15.143:0/268969447 <== mon.2 v2:172.21.15.188:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9ee00ab7b0 con 0x7f9ef0143ac0 2023-11-26T07:19:48.714 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:48.711+0000 7f9eed59a640 1 -- 172.21.15.143:0/268969447 <== mon.2 v2:172.21.15.188:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9ee00abb40 con 0x7f9ef0143ac0 2023-11-26T07:19:48.714 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:48.711+0000 7f9ef4d72640 1 -- 172.21.15.143:0/268969447 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f9ef0143ac0 msgr2=0x7f9ef0070370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:19:48.715 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:48.711+0000 7f9ef4d72640 1 --2- 172.21.15.143:0/268969447 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f9ef0143ac0 0x7f9ef0070370 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7f9ee0009870 tx=0x7f9ee001c540 comp rx=0 tx=0).stop 2023-11-26T07:19:48.715 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:48.711+0000 7f9ef4d72640 1 -- 172.21.15.143:0/268969447 shutdown_connections 2023-11-26T07:19:48.715 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:48.711+0000 7f9ef4d72640 1 --2- 172.21.15.143:0/268969447 >> v2:172.21.15.188:3300/0 conn(0x7f9ef00cf990 0x7f9ef00cd1b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:19:48.715 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:48.711+0000 7f9ef4d72640 1 --2- 172.21.15.143:0/268969447 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f9ef0143ac0 0x7f9ef0070370 unknown :-1 s=CLOSED pgs=91 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:19:48.715 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:48.711+0000 7f9ef4d72640 1 -- 172.21.15.143:0/268969447 >> 172.21.15.143:0/268969447 conn(0x7f9ef006a890 msgr2=0x7f9ef006aca0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:19:48.715 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:48.711+0000 7f9ef4d72640 1 -- 172.21.15.143:0/268969447 shutdown_connections 2023-11-26T07:19:48.715 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:48.711+0000 7f9ef4d72640 1 -- 172.21.15.143:0/268969447 wait complete. 2023-11-26T07:19:48.716 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:48.711+0000 7f9ef4d72640 1 Processor -- start 2023-11-26T07:19:48.716 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:48.711+0000 7f9ef4d72640 1 -- start start 2023-11-26T07:19:48.717 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:48.715+0000 7f9ef4d72640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f9ef00708b0 0x7f9ef013ca20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:19:48.718 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:48.715+0000 7f9ef4d72640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f9ef00cf990 0x7f9ef0158170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:19:48.718 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:48.715+0000 7f9ef4d72640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f9ef0143ac0 0x7f9ef013bb80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:19:48.718 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:48.715+0000 7f9ef4d72640 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9ef01590a0 con 0x7f9ef00cf990 2023-11-26T07:19:48.718 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:48.715+0000 7f9ef4d72640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9ef013c0c0 con 0x7f9ef00708b0 2023-11-26T07:19:48.718 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:48.715+0000 7f9ef4d72640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9ef013d100 con 0x7f9ef0143ac0 2023-11-26T07:19:48.718 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:48.715+0000 7f9eedd9b640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f9ef00cf990 0x7f9ef0158170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:19:48.718 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:48.715+0000 7f9eedd9b640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f9ef00cf990 0x7f9ef0158170 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:58244/0 (socket says 172.21.15.143:58244) 2023-11-26T07:19:48.718 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:48.715+0000 7f9eedd9b640 1 -- 172.21.15.143:0/1799990142 learned_addr learned my addr 172.21.15.143:0/1799990142 (peer_addr_for_me v2:172.21.15.143:0/0) 2023-11-26T07:19:48.719 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:48.715+0000 7f9eeed9d640 1 --2- 172.21.15.143:0/1799990142 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f9ef0143ac0 0x7f9ef013bb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:19:48.719 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:48.715+0000 7f9eee59c640 1 --2- 172.21.15.143:0/1799990142 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f9ef00708b0 0x7f9ef013ca20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:19:48.719 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:48.715+0000 7f9eedd9b640 1 -- 172.21.15.143:0/1799990142 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f9ef0143ac0 msgr2=0x7f9ef013bb80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:19:48.720 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:48.715+0000 7f9eedd9b640 1 --2- 172.21.15.143:0/1799990142 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f9ef0143ac0 0x7f9ef013bb80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:19:48.720 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:48.715+0000 7f9eedd9b640 1 -- 172.21.15.143:0/1799990142 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f9ef00708b0 msgr2=0x7f9ef013ca20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:19:48.720 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:48.715+0000 7f9eedd9b640 1 --2- 172.21.15.143:0/1799990142 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f9ef00708b0 0x7f9ef013ca20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:19:48.720 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:48.715+0000 7f9eedd9b640 1 -- 172.21.15.143:0/1799990142 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9ee8009a70 con 0x7f9ef00cf990 2023-11-26T07:19:48.720 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:48.715+0000 7f9eeed9d640 1 --2- 172.21.15.143:0/1799990142 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f9ef0143ac0 0x7f9ef013bb80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-26T07:19:48.720 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:48.715+0000 7f9eedd9b640 1 --2- 172.21.15.143:0/1799990142 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f9ef00cf990 0x7f9ef0158170 secure :-1 s=READY pgs=628 cs=0 l=1 rev1=1 crypto rx=0x7f9ee400c450 tx=0x7f9ee400c910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:19:48.720 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:48.715+0000 7f9edeffd640 1 -- 172.21.15.143:0/1799990142 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9ee40052d0 con 0x7f9ef00cf990 2023-11-26T07:19:48.720 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:48.715+0000 7f9edeffd640 1 -- 172.21.15.143:0/1799990142 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9ee4005450 con 0x7f9ef00cf990 2023-11-26T07:19:48.720 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:48.715+0000 7f9edeffd640 1 -- 172.21.15.143:0/1799990142 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9ee4012870 con 0x7f9ef00cf990 2023-11-26T07:19:48.720 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:48.715+0000 7f9ef4d72640 1 -- 172.21.15.143:0/1799990142 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9ee00094d0 con 0x7f9ef00cf990 2023-11-26T07:19:48.720 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:48.715+0000 7f9ef4d72640 1 -- 172.21.15.143:0/1799990142 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9ef013b0a0 con 0x7f9ef00cf990 2023-11-26T07:19:48.720 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:48.715+0000 7f9ef4d72640 1 -- 172.21.15.143:0/1799990142 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9ef0067680 con 0x7f9ef00cf990 2023-11-26T07:19:48.721 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:48.719+0000 7f9edeffd640 1 -- 172.21.15.143:0/1799990142 <== mon.0 v2:172.21.15.143:3300/0 4 ==== mgrmap(e 7) v1 ==== 86005+0+0 (secure 0 0 0) 0x7f9ee40129d0 con 0x7f9ef00cf990 2023-11-26T07:19:48.721 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:48.719+0000 7f9edeffd640 1 --2- 172.21.15.143:0/1799990142 >> [v2:172.21.15.143:6832/2150557544,v1:172.21.15.143:6833/2150557544] conn(0x7f9ed0066620 0x7f9ed0068ae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:19:48.722 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:48.719+0000 7f9edeffd640 1 -- 172.21.15.143:0/1799990142 <== mon.0 v2:172.21.15.143:3300/0 5 ==== osd_map(45..45 src has 1..45) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f9ee4083cb0 con 0x7f9ef00cf990 2023-11-26T07:19:48.722 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:48.719+0000 7f9eee59c640 1 --2- 172.21.15.143:0/1799990142 >> [v2:172.21.15.143:6832/2150557544,v1:172.21.15.143:6833/2150557544] conn(0x7f9ed0066620 0x7f9ed0068ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:19:48.722 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:48.719+0000 7f9eee59c640 1 --2- 172.21.15.143:0/1799990142 >> [v2:172.21.15.143:6832/2150557544,v1:172.21.15.143:6833/2150557544] conn(0x7f9ed0066620 0x7f9ed0068ae0 secure :-1 s=READY pgs=308 cs=0 l=1 rev1=1 crypto rx=0x7f9ee00af000 tx=0x7f9ee00a7000 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:19:48.726 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:48.723+0000 7f9edeffd640 1 -- 172.21.15.143:0/1799990142 <== mon.0 v2:172.21.15.143:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172649 (secure 0 0 0) 0x7f9ee4050ea0 con 0x7f9ef00cf990 2023-11-26T07:19:48.890 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:48.887+0000 7f9ef4d72640 1 -- 172.21.15.143:0/1799990142 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) v1 -- 0x7f9ef00b23a0 con 0x7f9ef00cf990 2023-11-26T07:19:48.890 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:48.887+0000 7f9edeffd640 1 -- 172.21.15.143:0/1799990142 <== mon.0 v2:172.21.15.143:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v45) v1 ==== 80+0+0 (secure 0 0 0) 0x7f9ee40544c0 con 0x7f9ef00cf990 2023-11-26T07:19:48.890 INFO:teuthology.orchestra.run.smithi143.stderr:listed 0 entries 2023-11-26T07:19:48.892 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:48.887+0000 7f9ef4d72640 1 -- 172.21.15.143:0/1799990142 >> [v2:172.21.15.143:6832/2150557544,v1:172.21.15.143:6833/2150557544] conn(0x7f9ed0066620 msgr2=0x7f9ed0068ae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:19:48.893 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:48.887+0000 7f9ef4d72640 1 --2- 172.21.15.143:0/1799990142 >> [v2:172.21.15.143:6832/2150557544,v1:172.21.15.143:6833/2150557544] conn(0x7f9ed0066620 0x7f9ed0068ae0 secure :-1 s=READY pgs=308 cs=0 l=1 rev1=1 crypto rx=0x7f9ee00af000 tx=0x7f9ee00a7000 comp rx=0 tx=0).stop 2023-11-26T07:19:48.893 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:48.887+0000 7f9ef4d72640 1 -- 172.21.15.143:0/1799990142 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f9ef00cf990 msgr2=0x7f9ef0158170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:19:48.893 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:48.887+0000 7f9ef4d72640 1 --2- 172.21.15.143:0/1799990142 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f9ef00cf990 0x7f9ef0158170 secure :-1 s=READY pgs=628 cs=0 l=1 rev1=1 crypto rx=0x7f9ee400c450 tx=0x7f9ee400c910 comp rx=0 tx=0).stop 2023-11-26T07:19:48.893 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:48.887+0000 7f9ef4d72640 1 -- 172.21.15.143:0/1799990142 shutdown_connections 2023-11-26T07:19:48.893 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:48.887+0000 7f9ef4d72640 1 --2- 172.21.15.143:0/1799990142 >> [v2:172.21.15.143:6832/2150557544,v1:172.21.15.143:6833/2150557544] conn(0x7f9ed0066620 0x7f9ed0068ae0 unknown :-1 s=CLOSED pgs=308 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:19:48.893 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:48.887+0000 7f9ef4d72640 1 --2- 172.21.15.143:0/1799990142 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f9ef0143ac0 0x7f9ef013bb80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:19:48.893 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:48.887+0000 7f9ef4d72640 1 --2- 172.21.15.143:0/1799990142 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f9ef00cf990 0x7f9ef0158170 unknown :-1 s=CLOSED pgs=628 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:19:48.893 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:48.887+0000 7f9ef4d72640 1 --2- 172.21.15.143:0/1799990142 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f9ef00708b0 0x7f9ef013ca20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:19:48.893 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:48.887+0000 7f9ef4d72640 1 -- 172.21.15.143:0/1799990142 >> 172.21.15.143:0/1799990142 conn(0x7f9ef006a890 msgr2=0x7f9ef00720c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:19:48.893 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:48.887+0000 7f9ef4d72640 1 -- 172.21.15.143:0/1799990142 shutdown_connections 2023-11-26T07:19:48.893 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:48.887+0000 7f9ef4d72640 1 -- 172.21.15.143:0/1799990142 wait complete. 2023-11-26T07:19:48.911 DEBUG:teuthology.orchestra.run.smithi143:> set -ex 2023-11-26T07:19:48.911 DEBUG:teuthology.orchestra.run.smithi143:> dd if=/proc/self/mounts of=/dev/stdout 2023-11-26T07:19:48.920 DEBUG:teuthology.orchestra.run.smithi143:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-11-26T07:19:49.100 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:49.095+0000 7fd25be54640 1 Processor -- start 2023-11-26T07:19:49.100 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:49.095+0000 7fd25be54640 1 -- start start 2023-11-26T07:19:49.102 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:49.099+0000 7fd25be54640 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7fd254143ac0 0x7fd254070370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:19:49.102 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:49.099+0000 7fd25be54640 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7fd2540708b0 0x7fd2540cf280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:19:49.102 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:49.099+0000 7fd25be54640 1 -- --> v1:172.21.15.143:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd2540cc730 con 0x7fd2540cf7c0 2023-11-26T07:19:49.102 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:49.099+0000 7fd25be54640 1 -- --> v2:172.21.15.143:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd254143ea0 con 0x7fd254143ac0 2023-11-26T07:19:49.102 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:49.099+0000 7fd25be54640 1 -- --> v2:172.21.15.188:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd2540cfbd0 con 0x7fd2540708b0 2023-11-26T07:19:49.102 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:49.099+0000 7fd25a3f1640 1 --1- >> v1:172.21.15.143:6789/0 conn(0x7fd2540cf7c0 0x7fd2540b1ef0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.143:6789/0 says I am v1:172.21.15.143:50460/0 (socket says 172.21.15.143:50460) 2023-11-26T07:19:49.102 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:49.099+0000 7fd25a3f1640 1 -- 172.21.15.143:0/697708639 learned_addr learned my addr 172.21.15.143:0/697708639 (peer_addr_for_me v1:172.21.15.143:0/0) 2023-11-26T07:19:49.103 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:49.099+0000 7fd259bf0640 1 --2- 172.21.15.143:0/697708639 >> v2:172.21.15.143:3300/0 conn(0x7fd254143ac0 0x7fd254070370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:19:49.103 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:49.099+0000 7fd2593ef640 1 --2- 172.21.15.143:0/697708639 >> v2:172.21.15.188:3300/0 conn(0x7fd2540708b0 0x7fd2540cf280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:19:49.103 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:49.099+0000 7fd258bee640 1 -- 172.21.15.143:0/697708639 <== mon.0 v1:172.21.15.143:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2208510288 0 0) 0x7fd2540cc730 con 0x7fd2540cf7c0 2023-11-26T07:19:49.103 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:49.099+0000 7fd258bee640 1 -- 172.21.15.143:0/697708639 --> v1:172.21.15.143:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fd240003530 con 0x7fd2540cf7c0 2023-11-26T07:19:49.103 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:49.099+0000 7fd259bf0640 1 -- 172.21.15.143:0/697708639 >> v2:172.21.15.188:3300/0 conn(0x7fd2540708b0 msgr2=0x7fd2540cf280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-26T07:19:49.104 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:49.099+0000 7fd259bf0640 1 --2- 172.21.15.143:0/697708639 >> v2:172.21.15.188:3300/0 conn(0x7fd2540708b0 0x7fd2540cf280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:19:49.104 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:49.099+0000 7fd259bf0640 1 -- 172.21.15.143:0/697708639 >> v1:172.21.15.143:6789/0 conn(0x7fd2540cf7c0 legacy=0x7fd2540b1ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:19:49.104 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:49.099+0000 7fd259bf0640 1 -- 172.21.15.143:0/697708639 --> v2:172.21.15.143:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd254070ee0 con 0x7fd254143ac0 2023-11-26T07:19:49.105 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:49.103+0000 7fd259bf0640 1 --2- 172.21.15.143:0/697708639 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fd254143ac0 0x7fd254070370 secure :-1 s=READY pgs=630 cs=0 l=1 rev1=1 crypto rx=0x7fd2540aeb10 tx=0x7fd25001c460 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6dcd236ab66d7528 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:19:49.106 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:49.103+0000 7fd258bee640 1 -- 172.21.15.143:0/697708639 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd2500ab600 con 0x7fd254143ac0 2023-11-26T07:19:49.106 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:49.103+0000 7fd258bee640 1 -- 172.21.15.143:0/697708639 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd2500a7050 con 0x7fd254143ac0 2023-11-26T07:19:49.106 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:49.103+0000 7fd258bee640 1 -- 172.21.15.143:0/697708639 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd2500ab980 con 0x7fd254143ac0 2023-11-26T07:19:49.106 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:49.103+0000 7fd25be54640 1 -- 172.21.15.143:0/697708639 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fd254143ac0 msgr2=0x7fd254070370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:19:49.107 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:49.103+0000 7fd25be54640 1 --2- 172.21.15.143:0/697708639 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fd254143ac0 0x7fd254070370 secure :-1 s=READY pgs=630 cs=0 l=1 rev1=1 crypto rx=0x7fd2540aeb10 tx=0x7fd25001c460 comp rx=0 tx=0).stop 2023-11-26T07:19:49.107 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:49.103+0000 7fd25be54640 1 -- 172.21.15.143:0/697708639 shutdown_connections 2023-11-26T07:19:49.107 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:49.103+0000 7fd25be54640 1 --2- 172.21.15.143:0/697708639 >> v2:172.21.15.188:3300/0 conn(0x7fd2540708b0 0x7fd2540cf280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:19:49.107 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:49.103+0000 7fd25be54640 1 --2- 172.21.15.143:0/697708639 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fd254143ac0 0x7fd254070370 unknown :-1 s=CLOSED pgs=630 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:19:49.107 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:49.103+0000 7fd25be54640 1 -- 172.21.15.143:0/697708639 >> 172.21.15.143:0/697708639 conn(0x7fd25406a890 msgr2=0x7fd25406aca0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:19:49.107 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:49.103+0000 7fd25be54640 1 -- 172.21.15.143:0/697708639 shutdown_connections 2023-11-26T07:19:49.107 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:49.103+0000 7fd25be54640 1 -- 172.21.15.143:0/697708639 wait complete. 2023-11-26T07:19:49.108 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:49.103+0000 7fd25be54640 1 Processor -- start 2023-11-26T07:19:49.108 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:49.107+0000 7fd25be54640 1 -- start start 2023-11-26T07:19:49.111 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:49.107+0000 7fd25be54640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fd2540708b0 0x7fd254157620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:19:49.111 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:49.107+0000 7fd25be54640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fd2540cf7c0 0x7fd25413b3f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:19:49.111 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:49.107+0000 7fd25be54640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fd254143ac0 0x7fd254141390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:19:49.111 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:49.107+0000 7fd25be54640 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd2541f6c20 con 0x7fd2540708b0 2023-11-26T07:19:49.111 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:49.107+0000 7fd25be54640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd254141be0 con 0x7fd254143ac0 2023-11-26T07:19:49.111 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:49.107+0000 7fd25be54640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd2541f62f0 con 0x7fd2540cf7c0 2023-11-26T07:19:49.111 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:49.107+0000 7fd259bf0640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fd2540708b0 0x7fd254157620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:19:49.111 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:49.107+0000 7fd259bf0640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fd2540708b0 0x7fd254157620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:58260/0 (socket says 172.21.15.143:58260) 2023-11-26T07:19:49.112 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:49.107+0000 7fd259bf0640 1 -- 172.21.15.143:0/1413462103 learned_addr learned my addr 172.21.15.143:0/1413462103 (peer_addr_for_me v2:172.21.15.143:0/0) 2023-11-26T07:19:49.112 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:49.107+0000 7fd2593ef640 1 --2- 172.21.15.143:0/1413462103 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fd2540cf7c0 0x7fd25413b3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:19:49.112 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:49.107+0000 7fd25a3f1640 1 --2- 172.21.15.143:0/1413462103 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fd254143ac0 0x7fd254141390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:19:49.112 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:49.107+0000 7fd259bf0640 1 -- 172.21.15.143:0/1413462103 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fd2540cf7c0 msgr2=0x7fd25413b3f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:19:49.112 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:49.107+0000 7fd259bf0640 1 --2- 172.21.15.143:0/1413462103 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fd2540cf7c0 0x7fd25413b3f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:19:49.112 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:49.107+0000 7fd259bf0640 1 -- 172.21.15.143:0/1413462103 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fd254143ac0 msgr2=0x7fd254141390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:19:49.112 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:49.107+0000 7fd259bf0640 1 --2- 172.21.15.143:0/1413462103 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fd254143ac0 0x7fd254141390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:19:49.112 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:49.107+0000 7fd259bf0640 1 -- 172.21.15.143:0/1413462103 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd250009420 con 0x7fd2540708b0 2023-11-26T07:19:49.112 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:49.107+0000 7fd2593ef640 1 --2- 172.21.15.143:0/1413462103 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fd2540cf7c0 0x7fd25413b3f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-26T07:19:49.112 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:49.107+0000 7fd25a3f1640 1 --2- 172.21.15.143:0/1413462103 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fd254143ac0 0x7fd254141390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-26T07:19:49.112 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:49.107+0000 7fd259bf0640 1 --2- 172.21.15.143:0/1413462103 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fd2540708b0 0x7fd254157620 secure :-1 s=READY pgs=631 cs=0 l=1 rev1=1 crypto rx=0x7fd2500ae040 tx=0x7fd250019200 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:19:49.112 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:49.107+0000 7fd24a7fc640 1 -- 172.21.15.143:0/1413462103 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd2500b4410 con 0x7fd2540708b0 2023-11-26T07:19:49.113 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:49.107+0000 7fd24a7fc640 1 -- 172.21.15.143:0/1413462103 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd2500b4570 con 0x7fd2540708b0 2023-11-26T07:19:49.113 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:49.107+0000 7fd24a7fc640 1 -- 172.21.15.143:0/1413462103 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd2500b4920 con 0x7fd2540708b0 2023-11-26T07:19:49.113 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:49.107+0000 7fd25be54640 1 -- 172.21.15.143:0/1413462103 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd2541f65d0 con 0x7fd2540708b0 2023-11-26T07:19:49.113 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:49.107+0000 7fd25be54640 1 -- 172.21.15.143:0/1413462103 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd25413b930 con 0x7fd2540708b0 2023-11-26T07:19:49.113 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:49.107+0000 7fd25be54640 1 -- 172.21.15.143:0/1413462103 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd254067680 con 0x7fd2540708b0 2023-11-26T07:19:49.114 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:49.111+0000 7fd24a7fc640 1 -- 172.21.15.143:0/1413462103 <== mon.0 v2:172.21.15.143:3300/0 4 ==== mgrmap(e 7) v1 ==== 86005+0+0 (secure 0 0 0) 0x7fd2500b2040 con 0x7fd2540708b0 2023-11-26T07:19:49.114 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:49.111+0000 7fd24a7fc640 1 --2- 172.21.15.143:0/1413462103 >> [v2:172.21.15.143:6832/2150557544,v1:172.21.15.143:6833/2150557544] conn(0x7fd238066620 0x7fd238068ae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:19:49.114 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:49.111+0000 7fd24a7fc640 1 -- 172.21.15.143:0/1413462103 <== mon.0 v2:172.21.15.143:3300/0 5 ==== osd_map(45..45 src has 1..45) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fd25011e0b0 con 0x7fd2540708b0 2023-11-26T07:19:49.114 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:49.111+0000 7fd2593ef640 1 --2- 172.21.15.143:0/1413462103 >> [v2:172.21.15.143:6832/2150557544,v1:172.21.15.143:6833/2150557544] conn(0x7fd238066620 0x7fd238068ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:19:49.114 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:49.111+0000 7fd2593ef640 1 --2- 172.21.15.143:0/1413462103 >> [v2:172.21.15.143:6832/2150557544,v1:172.21.15.143:6833/2150557544] conn(0x7fd238066620 0x7fd238068ae0 secure :-1 s=READY pgs=309 cs=0 l=1 rev1=1 crypto rx=0x7fd25413f820 tx=0x7fd244008040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:19:49.118 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:49.115+0000 7fd24a7fc640 1 -- 172.21.15.143:0/1413462103 <== mon.0 v2:172.21.15.143:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172649 (secure 0 0 0) 0x7fd2500eb180 con 0x7fd2540708b0 2023-11-26T07:19:49.268 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:49.263+0000 7fd25be54640 1 -- 172.21.15.143:0/1413462103 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fd25413e4d0 con 0x7fd2540708b0 2023-11-26T07:19:49.270 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:49.267+0000 7fd24a7fc640 1 -- 172.21.15.143:0/1413462103 <== mon.0 v2:172.21.15.143:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v45) v1 ==== 74+0+16549 (secure 0 0 0) 0x7fd2500ee7a0 con 0x7fd2540708b0 2023-11-26T07:19:49.270 INFO:teuthology.orchestra.run.smithi143.stdout: 2023-11-26T07:19:49.270 INFO:teuthology.orchestra.run.smithi143.stdout:{"epoch":45,"fsid":"f5033060-2469-4a07-95fe-2b76f5404bc1","created":"2023-11-26T07:17:28.613663+0000","modified":"2023-11-26T07:19:33.132471+0000","last_up_change":"2023-11-26T07:17:41.157813+0000","last_in_change":"2023-11-26T07:17:31.181658+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":7,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-11-26T07:17:41.758597+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":6,"pool_name":"cephfs_metadata","create_time":"2023-11-26T07:19:29.208075+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"44","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":7,"pool_name":"cephfs_data","create_time":"2023-11-26T07:19:30.503743+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"45","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"ebbee2c6-023e-44b9-9f1d-6f7bfd309730","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":43,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6817","nonce":639807369},{"type":"v1","addr":"172.21.15.143:6819","nonce":639807369}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6821","nonce":639807369},{"type":"v1","addr":"172.21.15.143:6823","nonce":639807369}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6830","nonce":639807369},{"type":"v1","addr":"172.21.15.143:6831","nonce":639807369}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6825","nonce":639807369},{"type":"v1","addr":"172.21.15.143:6828","nonce":639807369}]},"public_addr":"172.21.15.143:6819/639807369","cluster_addr":"172.21.15.143:6823/639807369","heartbeat_back_addr":"172.21.15.143:6831/639807369","heartbeat_front_addr":"172.21.15.143:6828/639807369","state":["exists","up"]},{"osd":1,"uuid":"fec4b8be-7122-49e9-9a19-5850a0700001","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":43,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6816","nonce":1144942530},{"type":"v1","addr":"172.21.15.143:6818","nonce":1144942530}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6820","nonce":1144942530},{"type":"v1","addr":"172.21.15.143:6822","nonce":1144942530}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6827","nonce":1144942530},{"type":"v1","addr":"172.21.15.143:6829","nonce":1144942530}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6824","nonce":1144942530},{"type":"v1","addr":"172.21.15.143:6826","nonce":1144942530}]},"public_addr":"172.21.15.143:6818/1144942530","cluster_addr":"172.21.15.143:6822/1144942530","heartbeat_back_addr":"172.21.15.143:6829/1144942530","heartbeat_front_addr":"172.21.15.143:6826/1144942530","state":["exists","up"]},{"osd":2,"uuid":"2888af5b-f933-46ca-8f95-6dd494ff43f6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":43,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6808","nonce":3909095186},{"type":"v1","addr":"172.21.15.143:6809","nonce":3909095186}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6810","nonce":3909095186},{"type":"v1","addr":"172.21.15.143:6811","nonce":3909095186}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6814","nonce":3909095186},{"type":"v1","addr":"172.21.15.143:6815","nonce":3909095186}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6812","nonce":3909095186},{"type":"v1","addr":"172.21.15.143:6813","nonce":3909095186}]},"public_addr":"172.21.15.143:6809/3909095186","cluster_addr":"172.21.15.143:6811/3909095186","heartbeat_back_addr":"172.21.15.143:6815/3909095186","heartbeat_front_addr":"172.21.15.143:6813/3909095186","state":["exists","up"]},{"osd":3,"uuid":"d1f52435-7bc1-4081-9550-9cd1ba55f549","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":43,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6800","nonce":3854210991},{"type":"v1","addr":"172.21.15.143:6801","nonce":3854210991}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6802","nonce":3854210991},{"type":"v1","addr":"172.21.15.143:6803","nonce":3854210991}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6806","nonce":3854210991},{"type":"v1","addr":"172.21.15.143:6807","nonce":3854210991}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6804","nonce":3854210991},{"type":"v1","addr":"172.21.15.143:6805","nonce":3854210991}]},"public_addr":"172.21.15.143:6801/3854210991","cluster_addr":"172.21.15.143:6803/3854210991","heartbeat_back_addr":"172.21.15.143:6807/3854210991","heartbeat_front_addr":"172.21.15.143:6805/3854210991","state":["exists","up"]},{"osd":4,"uuid":"e2ae8b6e-6e25-49b2-bcbf-aacce68f06a3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":43,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6800","nonce":3924606902},{"type":"v1","addr":"172.21.15.188:6801","nonce":3924606902}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6802","nonce":3924606902},{"type":"v1","addr":"172.21.15.188:6803","nonce":3924606902}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6806","nonce":3924606902},{"type":"v1","addr":"172.21.15.188:6807","nonce":3924606902}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6804","nonce":3924606902},{"type":"v1","addr":"172.21.15.188:6805","nonce":3924606902}]},"public_addr":"172.21.15.188:6801/3924606902","cluster_addr":"172.21.15.188:6803/3924606902","heartbeat_back_addr":"172.21.15.188:6807/3924606902","heartbeat_front_addr":"172.21.15.188:6805/3924606902","state":["exists","up"]},{"osd":5,"uuid":"3cec8a32-7698-4c36-8e89-e4ba11934f33","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":43,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6813","nonce":186841544},{"type":"v1","addr":"172.21.15.188:6815","nonce":186841544}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6817","nonce":186841544},{"type":"v1","addr":"172.21.15.188:6818","nonce":186841544}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6822","nonce":186841544},{"type":"v1","addr":"172.21.15.188:6823","nonce":186841544}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6820","nonce":186841544},{"type":"v1","addr":"172.21.15.188:6821","nonce":186841544}]},"public_addr":"172.21.15.188:6815/186841544","cluster_addr":"172.21.15.188:6818/186841544","heartbeat_back_addr":"172.21.15.188:6823/186841544","heartbeat_front_addr":"172.21.15.188:6821/186841544","state":["exists","up"]},{"osd":6,"uuid":"58a65d5d-e090-473a-81b4-ad38e85029f9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":43,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6808","nonce":2153499405},{"type":"v1","addr":"172.21.15.188:6809","nonce":2153499405}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6810","nonce":2153499405},{"type":"v1","addr":"172.21.15.188:6811","nonce":2153499405}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6816","nonce":2153499405},{"type":"v1","addr":"172.21.15.188:6819","nonce":2153499405}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6812","nonce":2153499405},{"type":"v1","addr":"172.21.15.188:6814","nonce":2153499405}]},"public_addr":"172.21.15.188:6809/2153499405","cluster_addr":"172.21.15.188:6811/2153499405","heartbeat_back_addr":"172.21.15.188:6819/2153499405","heartbeat_front_addr":"172.21.15.188:6814/2153499405","state":["exists","up"]},{"osd":7,"uuid":"ea5b7f70-3554-4cf8-ac30-f7c59b80678d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":43,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6824","nonce":2722096729},{"type":"v1","addr":"172.21.15.188:6825","nonce":2722096729}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6826","nonce":2722096729},{"type":"v1","addr":"172.21.15.188:6827","nonce":2722096729}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6830","nonce":2722096729},{"type":"v1","addr":"172.21.15.188:6831","nonce":2722096729}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6828","nonce":2722096729},{"type":"v1","addr":"172.21.15.188:6829","nonce":2722096729}]},"public_addr":"172.21.15.188:6825/2722096729","cluster_addr":"172.21.15.188:6827/2722096729","heartbeat_back_addr":"172.21.15.188:6831/2722096729","heartbeat_front_addr":"172.21.15.188:6829/2722096729","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-26T07:17:36.995032+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-26T07:17:36.932266+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-26T07:17:36.829304+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-26T07:17:36.804297+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-26T07:17:36.667892+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-26T07:17:36.964868+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-26T07:17:36.827332+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-26T07:17:36.893179+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-11-26T07:19:49.272 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:49.267+0000 7fd25be54640 1 -- 172.21.15.143:0/1413462103 >> [v2:172.21.15.143:6832/2150557544,v1:172.21.15.143:6833/2150557544] conn(0x7fd238066620 msgr2=0x7fd238068ae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:19:49.272 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:49.267+0000 7fd25be54640 1 --2- 172.21.15.143:0/1413462103 >> [v2:172.21.15.143:6832/2150557544,v1:172.21.15.143:6833/2150557544] conn(0x7fd238066620 0x7fd238068ae0 secure :-1 s=READY pgs=309 cs=0 l=1 rev1=1 crypto rx=0x7fd25413f820 tx=0x7fd244008040 comp rx=0 tx=0).stop 2023-11-26T07:19:49.272 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:49.267+0000 7fd25be54640 1 -- 172.21.15.143:0/1413462103 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fd2540708b0 msgr2=0x7fd254157620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:19:49.272 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:49.267+0000 7fd25be54640 1 --2- 172.21.15.143:0/1413462103 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fd2540708b0 0x7fd254157620 secure :-1 s=READY pgs=631 cs=0 l=1 rev1=1 crypto rx=0x7fd2500ae040 tx=0x7fd250019200 comp rx=0 tx=0).stop 2023-11-26T07:19:49.272 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:49.267+0000 7fd25be54640 1 -- 172.21.15.143:0/1413462103 shutdown_connections 2023-11-26T07:19:49.272 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:49.267+0000 7fd25be54640 1 --2- 172.21.15.143:0/1413462103 >> [v2:172.21.15.143:6832/2150557544,v1:172.21.15.143:6833/2150557544] conn(0x7fd238066620 0x7fd238068ae0 unknown :-1 s=CLOSED pgs=309 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:19:49.272 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:49.267+0000 7fd25be54640 1 --2- 172.21.15.143:0/1413462103 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fd254143ac0 0x7fd254141390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:19:49.272 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:49.267+0000 7fd25be54640 1 --2- 172.21.15.143:0/1413462103 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fd2540cf7c0 0x7fd25413b3f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:19:49.273 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:49.267+0000 7fd25be54640 1 --2- 172.21.15.143:0/1413462103 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fd2540708b0 0x7fd254157620 unknown :-1 s=CLOSED pgs=631 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:19:49.273 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:49.267+0000 7fd25be54640 1 -- 172.21.15.143:0/1413462103 >> 172.21.15.143:0/1413462103 conn(0x7fd25406a890 msgr2=0x7fd2540ad220 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:19:49.273 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:49.267+0000 7fd25be54640 1 -- 172.21.15.143:0/1413462103 shutdown_connections 2023-11-26T07:19:49.273 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:49.267+0000 7fd25be54640 1 -- 172.21.15.143:0/1413462103 wait complete. 2023-11-26T07:19:49.291 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-11-26T07:17:41.758597+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 6, 'pool_name': 'cephfs_metadata', 'create_time': '2023-11-26T07:19:29.208075+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '44', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 1.75, 'score_stable': 1.75, 'optimal_score': 1, 'raw_score_acting': 1.75, 'raw_score_stable': 1.75, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 7, 'pool_name': 'cephfs_data', 'create_time': '2023-11-26T07:19:30.503743+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '45', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.75, 'score_stable': 1.75, 'optimal_score': 1, 'raw_score_acting': 1.75, 'raw_score_stable': 1.75, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2023-11-26T07:19:49.292 DEBUG:teuthology.orchestra.run.smithi143:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-11-26T07:19:49.431 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:49.427+0000 7fe0e7868640 1 Processor -- start 2023-11-26T07:19:49.431 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:49.427+0000 7fe0e7868640 1 -- start start 2023-11-26T07:19:49.432 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:49.427+0000 7fe0e7868640 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7fe0e0143ac0 0x7fe0e0070370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:19:49.432 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:49.427+0000 7fe0e7868640 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7fe0e00708b0 0x7fe0e00cf280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:19:49.432 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:49.427+0000 7fe0e7868640 1 -- --> v1:172.21.15.143:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe0e00cc730 con 0x7fe0e00cf7c0 2023-11-26T07:19:49.432 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:49.427+0000 7fe0e7868640 1 -- --> v2:172.21.15.143:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe0e0143ea0 con 0x7fe0e0143ac0 2023-11-26T07:19:49.432 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:49.427+0000 7fe0e7868640 1 -- --> v2:172.21.15.188:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe0e00cfbd0 con 0x7fe0e00708b0 2023-11-26T07:19:49.433 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:49.431+0000 7fe0e5e05640 1 --1- >> v1:172.21.15.143:6789/0 conn(0x7fe0e00cf7c0 0x7fe0e00b1ef0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.143:6789/0 says I am v1:172.21.15.143:50476/0 (socket says 172.21.15.143:50476) 2023-11-26T07:19:49.434 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:49.431+0000 7fe0e5e05640 1 -- 172.21.15.143:0/3557547575 learned_addr learned my addr 172.21.15.143:0/3557547575 (peer_addr_for_me v1:172.21.15.143:0/0) 2023-11-26T07:19:49.434 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:49.431+0000 7fe0e4e03640 1 --2- 172.21.15.143:0/3557547575 >> v2:172.21.15.188:3300/0 conn(0x7fe0e00708b0 0x7fe0e00cf280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:19:49.434 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:49.431+0000 7fe0e5604640 1 --2- 172.21.15.143:0/3557547575 >> v2:172.21.15.143:3300/0 conn(0x7fe0e0143ac0 0x7fe0e0070370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:19:49.434 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:49.431+0000 7fe0d7fff640 1 -- 172.21.15.143:0/3557547575 <== mon.0 v1:172.21.15.143:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 309053171 0 0) 0x7fe0e00cc730 con 0x7fe0e00cf7c0 2023-11-26T07:19:49.434 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:49.431+0000 7fe0d7fff640 1 -- 172.21.15.143:0/3557547575 --> v1:172.21.15.143:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fe0cc003530 con 0x7fe0e00cf7c0 2023-11-26T07:19:49.434 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:49.431+0000 7fe0e4e03640 1 -- 172.21.15.143:0/3557547575 >> v2:172.21.15.143:3300/0 conn(0x7fe0e0143ac0 msgr2=0x7fe0e0070370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-26T07:19:49.434 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:49.431+0000 7fe0e4e03640 1 --2- 172.21.15.143:0/3557547575 >> v2:172.21.15.143:3300/0 conn(0x7fe0e0143ac0 0x7fe0e0070370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:19:49.434 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:49.431+0000 7fe0e4e03640 1 -- 172.21.15.143:0/3557547575 >> v1:172.21.15.143:6789/0 conn(0x7fe0e00cf7c0 legacy=0x7fe0e00b1ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:19:49.434 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:49.431+0000 7fe0e4e03640 1 -- 172.21.15.143:0/3557547575 --> v2:172.21.15.188:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe0e0070ee0 con 0x7fe0e00708b0 2023-11-26T07:19:49.434 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:49.431+0000 7fe0e5604640 1 --2- 172.21.15.143:0/3557547575 >> v2:172.21.15.143:3300/0 conn(0x7fe0e0143ac0 0x7fe0e0070370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-26T07:19:49.436 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:49.431+0000 7fe0e4e03640 1 --2- 172.21.15.143:0/3557547575 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fe0e00708b0 0x7fe0e00cf280 secure :-1 s=READY pgs=264 cs=0 l=1 rev1=1 crypto rx=0x7fe0e00aeb10 tx=0x7fe0d001c460 comp rx=0 tx=0).ready entity=mon.1 client_cookie=eadf472325f53c2c server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:19:49.437 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:49.435+0000 7fe0d7fff640 1 -- 172.21.15.143:0/3557547575 <== mon.1 v2:172.21.15.188:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe0d00ab660 con 0x7fe0e00708b0 2023-11-26T07:19:49.437 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:49.435+0000 7fe0d7fff640 1 -- 172.21.15.143:0/3557547575 <== mon.1 v2:172.21.15.188:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe0d00ab7c0 con 0x7fe0e00708b0 2023-11-26T07:19:49.437 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:49.435+0000 7fe0d7fff640 1 -- 172.21.15.143:0/3557547575 <== mon.1 v2:172.21.15.188:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe0d00abb50 con 0x7fe0e00708b0 2023-11-26T07:19:49.438 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:49.435+0000 7fe0e7868640 1 -- 172.21.15.143:0/3557547575 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fe0e00708b0 msgr2=0x7fe0e00cf280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:19:49.438 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:49.435+0000 7fe0e7868640 1 --2- 172.21.15.143:0/3557547575 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fe0e00708b0 0x7fe0e00cf280 secure :-1 s=READY pgs=264 cs=0 l=1 rev1=1 crypto rx=0x7fe0e00aeb10 tx=0x7fe0d001c460 comp rx=0 tx=0).stop 2023-11-26T07:19:49.438 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:49.435+0000 7fe0e7868640 1 -- 172.21.15.143:0/3557547575 shutdown_connections 2023-11-26T07:19:49.438 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:49.435+0000 7fe0e7868640 1 --2- 172.21.15.143:0/3557547575 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fe0e00708b0 0x7fe0e00cf280 unknown :-1 s=CLOSED pgs=264 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:19:49.438 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:49.435+0000 7fe0e7868640 1 --2- 172.21.15.143:0/3557547575 >> v2:172.21.15.143:3300/0 conn(0x7fe0e0143ac0 0x7fe0e0070370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:19:49.438 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:49.435+0000 7fe0e7868640 1 -- 172.21.15.143:0/3557547575 >> 172.21.15.143:0/3557547575 conn(0x7fe0e006a890 msgr2=0x7fe0e006aca0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:19:49.438 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:49.435+0000 7fe0e7868640 1 -- 172.21.15.143:0/3557547575 shutdown_connections 2023-11-26T07:19:49.438 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:49.435+0000 7fe0e7868640 1 -- 172.21.15.143:0/3557547575 wait complete. 2023-11-26T07:19:49.439 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:49.435+0000 7fe0e7868640 1 Processor -- start 2023-11-26T07:19:49.439 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:49.435+0000 7fe0e7868640 1 -- start start 2023-11-26T07:19:49.441 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:49.435+0000 7fe0e7868640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fe0e00708b0 0x7fe0e0158c30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:19:49.441 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:49.435+0000 7fe0e7868640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fe0e00cf7c0 0x7fe0e00b03f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:19:49.441 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:49.435+0000 7fe0e7868640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fe0e0143ac0 0x7fe0e01f3cd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:19:49.441 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:49.435+0000 7fe0e7868640 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe0e00af3e0 con 0x7fe0e0143ac0 2023-11-26T07:19:49.441 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:49.435+0000 7fe0e7868640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe0e00af630 con 0x7fe0e00708b0 2023-11-26T07:19:49.441 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:49.435+0000 7fe0e7868640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe0e00af910 con 0x7fe0e00cf7c0 2023-11-26T07:19:49.442 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:49.439+0000 7fe0e5e05640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fe0e0143ac0 0x7fe0e01f3cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:19:49.442 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:49.439+0000 7fe0e5e05640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fe0e0143ac0 0x7fe0e01f3cd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:58286/0 (socket says 172.21.15.143:58286) 2023-11-26T07:19:49.442 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:49.439+0000 7fe0e5e05640 1 -- 172.21.15.143:0/3228963558 learned_addr learned my addr 172.21.15.143:0/3228963558 (peer_addr_for_me v2:172.21.15.143:0/0) 2023-11-26T07:19:49.442 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:49.439+0000 7fe0e5604640 1 --2- 172.21.15.143:0/3228963558 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fe0e00708b0 0x7fe0e0158c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:19:49.442 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:49.439+0000 7fe0e4e03640 1 --2- 172.21.15.143:0/3228963558 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fe0e00cf7c0 0x7fe0e00b03f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:19:49.442 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:49.439+0000 7fe0e5e05640 1 -- 172.21.15.143:0/3228963558 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fe0e00cf7c0 msgr2=0x7fe0e00b03f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:19:49.442 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:49.439+0000 7fe0e5e05640 1 --2- 172.21.15.143:0/3228963558 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fe0e00cf7c0 0x7fe0e00b03f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:19:49.442 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:49.439+0000 7fe0e5e05640 1 -- 172.21.15.143:0/3228963558 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fe0e00708b0 msgr2=0x7fe0e0158c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:19:49.442 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:49.439+0000 7fe0e5e05640 1 --2- 172.21.15.143:0/3228963558 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fe0e00708b0 0x7fe0e0158c30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:19:49.442 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:49.439+0000 7fe0e5e05640 1 -- 172.21.15.143:0/3228963558 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe0dc009420 con 0x7fe0e0143ac0 2023-11-26T07:19:49.443 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:49.439+0000 7fe0e5e05640 1 --2- 172.21.15.143:0/3228963558 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fe0e0143ac0 0x7fe0e01f3cd0 secure :-1 s=READY pgs=633 cs=0 l=1 rev1=1 crypto rx=0x7fe0d800cb40 tx=0x7fe0d800a060 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:19:49.443 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:49.439+0000 7fe0d5ffb640 1 -- 172.21.15.143:0/3228963558 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe0d800aca0 con 0x7fe0e0143ac0 2023-11-26T07:19:49.443 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:49.439+0000 7fe0d5ffb640 1 -- 172.21.15.143:0/3228963558 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe0d800ae00 con 0x7fe0e0143ac0 2023-11-26T07:19:49.444 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:49.439+0000 7fe0d5ffb640 1 -- 172.21.15.143:0/3228963558 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe0d8012620 con 0x7fe0e0143ac0 2023-11-26T07:19:49.444 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:49.439+0000 7fe0e7868640 1 -- 172.21.15.143:0/3228963558 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe0d0009420 con 0x7fe0e0143ac0 2023-11-26T07:19:49.444 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:49.439+0000 7fe0e7868640 1 -- 172.21.15.143:0/3228963558 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe0e00afed0 con 0x7fe0e0143ac0 2023-11-26T07:19:49.444 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:49.439+0000 7fe0e7868640 1 -- 172.21.15.143:0/3228963558 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe0e00b00c0 con 0x7fe0e0143ac0 2023-11-26T07:19:49.444 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:49.439+0000 7fe0d5ffb640 1 -- 172.21.15.143:0/3228963558 <== mon.0 v2:172.21.15.143:3300/0 4 ==== mgrmap(e 7) v1 ==== 86005+0+0 (secure 0 0 0) 0x7fe0d8007070 con 0x7fe0e0143ac0 2023-11-26T07:19:49.445 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:49.443+0000 7fe0d5ffb640 1 --2- 172.21.15.143:0/3228963558 >> [v2:172.21.15.143:6832/2150557544,v1:172.21.15.143:6833/2150557544] conn(0x7fe0c4066650 0x7fe0c4068b10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:19:49.445 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:49.443+0000 7fe0d5ffb640 1 -- 172.21.15.143:0/3228963558 <== mon.0 v2:172.21.15.143:3300/0 5 ==== osd_map(45..45 src has 1..45) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fe0d8084120 con 0x7fe0e0143ac0 2023-11-26T07:19:49.445 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:49.443+0000 7fe0e5604640 1 --2- 172.21.15.143:0/3228963558 >> [v2:172.21.15.143:6832/2150557544,v1:172.21.15.143:6833/2150557544] conn(0x7fe0c4066650 0x7fe0c4068b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:19:49.445 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:49.443+0000 7fe0e5604640 1 --2- 172.21.15.143:0/3228963558 >> [v2:172.21.15.143:6832/2150557544,v1:172.21.15.143:6833/2150557544] conn(0x7fe0c4066650 0x7fe0c4068b10 secure :-1 s=READY pgs=310 cs=0 l=1 rev1=1 crypto rx=0x7fe0dc009cf0 tx=0x7fe0dc009190 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:19:49.448 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:49.447+0000 7fe0d5ffb640 1 -- 172.21.15.143:0/3228963558 <== mon.0 v2:172.21.15.143:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172649 (secure 0 0 0) 0x7fe0d80513e0 con 0x7fe0e0143ac0 2023-11-26T07:19:49.600 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:49.595+0000 7fe0e7868640 1 -- 172.21.15.143:0/3228963558 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7fe0e0067680 con 0x7fe0e0143ac0 2023-11-26T07:19:49.600 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:49.595+0000 7fe0d5ffb640 1 -- 172.21.15.143:0/3228963558 <== mon.0 v2:172.21.15.143:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v45) v1 ==== 93+0+10 (secure 0 0 0) 0x7fe0d8054a00 con 0x7fe0e0143ac0 2023-11-26T07:19:49.600 INFO:teuthology.orchestra.run.smithi143.stdout:pg_num: 1 2023-11-26T07:19:49.602 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:49.599+0000 7fe0e7868640 1 -- 172.21.15.143:0/3228963558 >> [v2:172.21.15.143:6832/2150557544,v1:172.21.15.143:6833/2150557544] conn(0x7fe0c4066650 msgr2=0x7fe0c4068b10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:19:49.603 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:49.599+0000 7fe0e7868640 1 --2- 172.21.15.143:0/3228963558 >> [v2:172.21.15.143:6832/2150557544,v1:172.21.15.143:6833/2150557544] conn(0x7fe0c4066650 0x7fe0c4068b10 secure :-1 s=READY pgs=310 cs=0 l=1 rev1=1 crypto rx=0x7fe0dc009cf0 tx=0x7fe0dc009190 comp rx=0 tx=0).stop 2023-11-26T07:19:49.603 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:49.599+0000 7fe0e7868640 1 -- 172.21.15.143:0/3228963558 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fe0e0143ac0 msgr2=0x7fe0e01f3cd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:19:49.603 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:49.599+0000 7fe0e7868640 1 --2- 172.21.15.143:0/3228963558 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fe0e0143ac0 0x7fe0e01f3cd0 secure :-1 s=READY pgs=633 cs=0 l=1 rev1=1 crypto rx=0x7fe0d800cb40 tx=0x7fe0d800a060 comp rx=0 tx=0).stop 2023-11-26T07:19:49.603 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:49.599+0000 7fe0e7868640 1 -- 172.21.15.143:0/3228963558 shutdown_connections 2023-11-26T07:19:49.603 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:49.599+0000 7fe0e7868640 1 --2- 172.21.15.143:0/3228963558 >> [v2:172.21.15.143:6832/2150557544,v1:172.21.15.143:6833/2150557544] conn(0x7fe0c4066650 0x7fe0c4068b10 unknown :-1 s=CLOSED pgs=310 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:19:49.603 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:49.599+0000 7fe0e7868640 1 --2- 172.21.15.143:0/3228963558 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fe0e0143ac0 0x7fe0e01f3cd0 unknown :-1 s=CLOSED pgs=633 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:19:49.603 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:49.599+0000 7fe0e7868640 1 --2- 172.21.15.143:0/3228963558 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fe0e00cf7c0 0x7fe0e00b03f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:19:49.603 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:49.599+0000 7fe0e7868640 1 --2- 172.21.15.143:0/3228963558 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fe0e00708b0 0x7fe0e0158c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:19:49.603 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:49.599+0000 7fe0e7868640 1 -- 172.21.15.143:0/3228963558 >> 172.21.15.143:0/3228963558 conn(0x7fe0e006a890 msgr2=0x7fe0e013c7f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:19:49.603 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:49.599+0000 7fe0e7868640 1 -- 172.21.15.143:0/3228963558 shutdown_connections 2023-11-26T07:19:49.603 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:49.599+0000 7fe0e7868640 1 -- 172.21.15.143:0/3228963558 wait complete. 2023-11-26T07:19:49.621 DEBUG:teuthology.orchestra.run.smithi143:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-11-26T07:19:49.763 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:49.759+0000 7ff991406640 1 Processor -- start 2023-11-26T07:19:49.763 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:49.759+0000 7ff991406640 1 -- start start 2023-11-26T07:19:49.764 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:49.759+0000 7ff991406640 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7ff98c0cf990 0x7ff98c15efa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:19:49.764 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:49.759+0000 7ff991406640 1 -- --> v1:172.21.15.143:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff98c06c520 con 0x7ff98c13e290 2023-11-26T07:19:49.764 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:49.759+0000 7ff991406640 1 -- --> v1:172.21.15.188:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff98c06c800 con 0x7ff98c15f4e0 2023-11-26T07:19:49.764 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:49.759+0000 7ff991406640 1 -- --> v2:172.21.15.143:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff98c06cae0 con 0x7ff98c0cf990 2023-11-26T07:19:49.765 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:49.759+0000 7ff98affd640 1 --1- >> v1:172.21.15.143:6789/0 conn(0x7ff98c13e290 0x7ff98c0cf280 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.143:6789/0 says I am v1:172.21.15.143:50480/0 (socket says 172.21.15.143:50480) 2023-11-26T07:19:49.765 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:49.759+0000 7ff98affd640 1 -- 172.21.15.143:0/2564212369 learned_addr learned my addr 172.21.15.143:0/2564212369 (peer_addr_for_me v1:172.21.15.143:0/0) 2023-11-26T07:19:49.765 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:49.759+0000 7ff9827fc640 1 --2- 172.21.15.143:0/2564212369 >> v2:172.21.15.143:3300/0 conn(0x7ff98c0cf990 0x7ff98c15efa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:19:49.765 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:49.763+0000 7ff98a7fc640 1 -- 172.21.15.143:0/2564212369 <== mon.0 v1:172.21.15.143:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 759378386 0 0) 0x7ff98c06c520 con 0x7ff98c13e290 2023-11-26T07:19:49.765 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:49.763+0000 7ff98a7fc640 1 -- 172.21.15.143:0/2564212369 --> v1:172.21.15.143:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7ff978003530 con 0x7ff98c13e290 2023-11-26T07:19:49.766 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:49.763+0000 7ff9827fc640 1 -- 172.21.15.143:0/2564212369 >> v1:172.21.15.188:6789/0 conn(0x7ff98c15f4e0 legacy=0x7ff98c0cd7b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-26T07:19:49.766 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:49.763+0000 7ff9827fc640 1 -- 172.21.15.143:0/2564212369 >> v1:172.21.15.143:6789/0 conn(0x7ff98c13e290 legacy=0x7ff98c0cf280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:19:49.766 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:49.763+0000 7ff9827fc640 1 -- 172.21.15.143:0/2564212369 --> v2:172.21.15.143:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff98c06cdc0 con 0x7ff98c0cf990 2023-11-26T07:19:49.768 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:49.763+0000 7ff9827fc640 1 --2- 172.21.15.143:0/2564212369 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7ff98c0cf990 0x7ff98c15efa0 secure :-1 s=READY pgs=635 cs=0 l=1 rev1=1 crypto rx=0x7ff9740097c0 tx=0x7ff97401c410 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e85adc1928e82ab3 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:19:49.768 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:49.763+0000 7ff98a7fc640 1 -- 172.21.15.143:0/2564212369 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff9740a7070 con 0x7ff98c0cf990 2023-11-26T07:19:49.768 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:49.763+0000 7ff98a7fc640 1 -- 172.21.15.143:0/2564212369 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff9740ab430 con 0x7ff98c0cf990 2023-11-26T07:19:49.768 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:49.763+0000 7ff98a7fc640 1 -- 172.21.15.143:0/2564212369 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff98c15fb60 con 0x7ff98c0cf990 2023-11-26T07:19:49.768 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:49.763+0000 7ff991406640 1 -- 172.21.15.143:0/2564212369 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7ff98c0cf990 msgr2=0x7ff98c15efa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:19:49.769 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:49.763+0000 7ff991406640 1 --2- 172.21.15.143:0/2564212369 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7ff98c0cf990 0x7ff98c15efa0 secure :-1 s=READY pgs=635 cs=0 l=1 rev1=1 crypto rx=0x7ff9740097c0 tx=0x7ff97401c410 comp rx=0 tx=0).stop 2023-11-26T07:19:49.769 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:49.763+0000 7ff991406640 1 -- 172.21.15.143:0/2564212369 shutdown_connections 2023-11-26T07:19:49.769 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:49.763+0000 7ff991406640 1 --2- 172.21.15.143:0/2564212369 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7ff98c0cf990 0x7ff98c15efa0 unknown :-1 s=CLOSED pgs=635 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:19:49.769 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:49.763+0000 7ff991406640 1 -- 172.21.15.143:0/2564212369 >> 172.21.15.143:0/2564212369 conn(0x7ff98c06a890 msgr2=0x7ff98c06aca0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:19:49.769 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:49.767+0000 7ff991406640 1 -- 172.21.15.143:0/2564212369 shutdown_connections 2023-11-26T07:19:49.769 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:49.767+0000 7ff991406640 1 -- 172.21.15.143:0/2564212369 wait complete. 2023-11-26T07:19:49.770 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:49.767+0000 7ff991406640 1 Processor -- start 2023-11-26T07:19:49.770 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:49.767+0000 7ff991406640 1 -- start start 2023-11-26T07:19:49.773 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:49.767+0000 7ff991406640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7ff98c0cf990 0x7ff98c0b1220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:19:49.773 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:49.767+0000 7ff991406640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7ff98c13e290 0x7ff98c071c70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:19:49.773 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:49.767+0000 7ff991406640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7ff98c15f4e0 0x7ff98c15a150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:19:49.773 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:49.767+0000 7ff991406640 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff98c137f20 con 0x7ff98c0cf990 2023-11-26T07:19:49.773 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:49.767+0000 7ff991406640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff98c137310 con 0x7ff98c13e290 2023-11-26T07:19:49.773 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:49.767+0000 7ff991406640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff98c1375f0 con 0x7ff98c15f4e0 2023-11-26T07:19:49.773 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:49.767+0000 7ff98affd640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7ff98c0cf990 0x7ff98c0b1220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:19:49.773 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:49.767+0000 7ff98affd640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7ff98c0cf990 0x7ff98c0b1220 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:58310/0 (socket says 172.21.15.143:58310) 2023-11-26T07:19:49.774 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:49.767+0000 7ff98affd640 1 -- 172.21.15.143:0/1977090705 learned_addr learned my addr 172.21.15.143:0/1977090705 (peer_addr_for_me v2:172.21.15.143:0/0) 2023-11-26T07:19:49.774 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:49.767+0000 7ff9827fc640 1 --2- 172.21.15.143:0/1977090705 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7ff98c13e290 0x7ff98c071c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:19:49.774 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:49.767+0000 7ff98b7fe640 1 --2- 172.21.15.143:0/1977090705 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7ff98c15f4e0 0x7ff98c15a150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:19:49.774 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:49.767+0000 7ff98affd640 1 -- 172.21.15.143:0/1977090705 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7ff98c15f4e0 msgr2=0x7ff98c15a150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:19:49.774 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:49.767+0000 7ff98affd640 1 --2- 172.21.15.143:0/1977090705 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7ff98c15f4e0 0x7ff98c15a150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:19:49.774 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:49.767+0000 7ff98affd640 1 -- 172.21.15.143:0/1977090705 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7ff98c13e290 msgr2=0x7ff98c071c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:19:49.774 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:49.767+0000 7ff98affd640 1 --2- 172.21.15.143:0/1977090705 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7ff98c13e290 0x7ff98c071c70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:19:49.774 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:49.767+0000 7ff98affd640 1 -- 172.21.15.143:0/1977090705 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff974009420 con 0x7ff98c0cf990 2023-11-26T07:19:49.774 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:49.767+0000 7ff98affd640 1 --2- 172.21.15.143:0/1977090705 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7ff98c0cf990 0x7ff98c0b1220 secure :-1 s=READY pgs=636 cs=0 l=1 rev1=1 crypto rx=0x7ff97c00cbc0 tx=0x7ff97c00a430 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:19:49.774 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:49.771+0000 7ff983fff640 1 -- 172.21.15.143:0/1977090705 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff97c00ad40 con 0x7ff98c0cf990 2023-11-26T07:19:49.774 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:49.771+0000 7ff983fff640 1 -- 172.21.15.143:0/1977090705 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff97c012400 con 0x7ff98c0cf990 2023-11-26T07:19:49.774 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:49.771+0000 7ff983fff640 1 -- 172.21.15.143:0/1977090705 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff97c00aea0 con 0x7ff98c0cf990 2023-11-26T07:19:49.775 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:49.771+0000 7ff991406640 1 -- 172.21.15.143:0/1977090705 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff98c1378d0 con 0x7ff98c0cf990 2023-11-26T07:19:49.775 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:49.771+0000 7ff991406640 1 -- 172.21.15.143:0/1977090705 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff98c0721b0 con 0x7ff98c0cf990 2023-11-26T07:19:49.775 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:49.771+0000 7ff991406640 1 -- 172.21.15.143:0/1977090705 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff98c067680 con 0x7ff98c0cf990 2023-11-26T07:19:49.775 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:49.771+0000 7ff983fff640 1 -- 172.21.15.143:0/1977090705 <== mon.0 v2:172.21.15.143:3300/0 4 ==== mgrmap(e 7) v1 ==== 86005+0+0 (secure 0 0 0) 0x7ff97c007070 con 0x7ff98c0cf990 2023-11-26T07:19:49.775 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:49.771+0000 7ff983fff640 1 --2- 172.21.15.143:0/1977090705 >> [v2:172.21.15.143:6832/2150557544,v1:172.21.15.143:6833/2150557544] conn(0x7ff96c066620 0x7ff96c068ae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:19:49.776 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:49.771+0000 7ff983fff640 1 -- 172.21.15.143:0/1977090705 <== mon.0 v2:172.21.15.143:3300/0 5 ==== osd_map(45..45 src has 1..45) v4 ==== 5709+0+0 (secure 0 0 0) 0x7ff97c0841e0 con 0x7ff98c0cf990 2023-11-26T07:19:49.776 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:49.771+0000 7ff9827fc640 1 --2- 172.21.15.143:0/1977090705 >> [v2:172.21.15.143:6832/2150557544,v1:172.21.15.143:6833/2150557544] conn(0x7ff96c066620 0x7ff96c068ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:19:49.776 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:49.775+0000 7ff9827fc640 1 --2- 172.21.15.143:0/1977090705 >> [v2:172.21.15.143:6832/2150557544,v1:172.21.15.143:6833/2150557544] conn(0x7ff96c066620 0x7ff96c068ae0 secure :-1 s=READY pgs=311 cs=0 l=1 rev1=1 crypto rx=0x7ff9740af000 tx=0x7ff9740a7000 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:19:49.779 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:49.775+0000 7ff983fff640 1 -- 172.21.15.143:0/1977090705 <== mon.0 v2:172.21.15.143:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172649 (secure 0 0 0) 0x7ff97c0513f0 con 0x7ff98c0cf990 2023-11-26T07:19:49.929 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:49.927+0000 7ff991406640 1 -- 172.21.15.143:0/1977090705 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7ff98c06df50 con 0x7ff98c0cf990 2023-11-26T07:19:49.930 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:49.927+0000 7ff983fff640 1 -- 172.21.15.143:0/1977090705 <== mon.0 v2:172.21.15.143:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v45) v1 ==== 104+0+11 (secure 0 0 0) 0x7ff97c054a10 con 0x7ff98c0cf990 2023-11-26T07:19:49.930 INFO:teuthology.orchestra.run.smithi143.stdout:pg_num: 32 2023-11-26T07:19:49.932 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:49.927+0000 7ff991406640 1 -- 172.21.15.143:0/1977090705 >> [v2:172.21.15.143:6832/2150557544,v1:172.21.15.143:6833/2150557544] conn(0x7ff96c066620 msgr2=0x7ff96c068ae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:19:49.932 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:49.927+0000 7ff991406640 1 --2- 172.21.15.143:0/1977090705 >> [v2:172.21.15.143:6832/2150557544,v1:172.21.15.143:6833/2150557544] conn(0x7ff96c066620 0x7ff96c068ae0 secure :-1 s=READY pgs=311 cs=0 l=1 rev1=1 crypto rx=0x7ff9740af000 tx=0x7ff9740a7000 comp rx=0 tx=0).stop 2023-11-26T07:19:49.932 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:49.927+0000 7ff991406640 1 -- 172.21.15.143:0/1977090705 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7ff98c0cf990 msgr2=0x7ff98c0b1220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:19:49.932 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:49.927+0000 7ff991406640 1 --2- 172.21.15.143:0/1977090705 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7ff98c0cf990 0x7ff98c0b1220 secure :-1 s=READY pgs=636 cs=0 l=1 rev1=1 crypto rx=0x7ff97c00cbc0 tx=0x7ff97c00a430 comp rx=0 tx=0).stop 2023-11-26T07:19:49.932 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:49.927+0000 7ff991406640 1 -- 172.21.15.143:0/1977090705 shutdown_connections 2023-11-26T07:19:49.932 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:49.927+0000 7ff991406640 1 --2- 172.21.15.143:0/1977090705 >> [v2:172.21.15.143:6832/2150557544,v1:172.21.15.143:6833/2150557544] conn(0x7ff96c066620 0x7ff96c068ae0 unknown :-1 s=CLOSED pgs=311 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:19:49.932 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:49.927+0000 7ff991406640 1 --2- 172.21.15.143:0/1977090705 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7ff98c15f4e0 0x7ff98c15a150 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:19:49.932 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:49.927+0000 7ff991406640 1 --2- 172.21.15.143:0/1977090705 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7ff98c13e290 0x7ff98c071c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:19:49.933 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:49.927+0000 7ff991406640 1 --2- 172.21.15.143:0/1977090705 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7ff98c0cf990 0x7ff98c0b1220 unknown :-1 s=CLOSED pgs=636 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:19:49.933 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:49.927+0000 7ff991406640 1 -- 172.21.15.143:0/1977090705 >> 172.21.15.143:0/1977090705 conn(0x7ff98c06a890 msgr2=0x7ff98c06ac70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:19:49.933 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:49.927+0000 7ff991406640 1 -- 172.21.15.143:0/1977090705 shutdown_connections 2023-11-26T07:19:49.933 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:49.927+0000 7ff991406640 1 -- 172.21.15.143:0/1977090705 wait complete. 2023-11-26T07:19:49.951 DEBUG:teuthology.orchestra.run.smithi143:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-11-26T07:19:50.093 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:50.091+0000 7f7fba135640 1 Processor -- start 2023-11-26T07:19:50.093 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:50.091+0000 7f7fba135640 1 -- start start 2023-11-26T07:19:50.094 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:50.091+0000 7f7fba135640 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7f7fb40d1bc0 0x7f7fb40d0df0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:19:50.094 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:50.091+0000 7f7fba135640 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7f7fb40b2160 0x7f7fb40b0750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:19:50.095 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:50.091+0000 7f7fba135640 1 -- --> v1:172.21.15.143:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7fb40cf0e0 con 0x7f7fb415d6e0 2023-11-26T07:19:50.095 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:50.091+0000 7f7fba135640 1 -- --> v2:172.21.15.143:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7fb415dac0 con 0x7f7fb40d1bc0 2023-11-26T07:19:50.095 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:50.091+0000 7f7fba135640 1 -- --> v2:172.21.15.188:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7fb40ceb20 con 0x7f7fb40b2160 2023-11-26T07:19:50.095 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:50.091+0000 7f7fb37fe640 1 --1- >> v1:172.21.15.143:6789/0 conn(0x7f7fb415d6e0 0x7f7fb40d14b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.143:6789/0 says I am v1:172.21.15.143:50496/0 (socket says 172.21.15.143:50496) 2023-11-26T07:19:50.095 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:50.091+0000 7f7fb37fe640 1 -- 172.21.15.143:0/1156739158 learned_addr learned my addr 172.21.15.143:0/1156739158 (peer_addr_for_me v1:172.21.15.143:0/0) 2023-11-26T07:19:50.095 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:50.091+0000 7f7fb2ffd640 1 --2- 172.21.15.143:0/1156739158 >> v2:172.21.15.143:3300/0 conn(0x7f7fb40d1bc0 0x7f7fb40d0df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:19:50.096 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:50.091+0000 7f7fb3fff640 1 --2- 172.21.15.143:0/1156739158 >> v2:172.21.15.188:3300/0 conn(0x7f7fb40b2160 0x7f7fb40b0750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:19:50.096 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:50.091+0000 7f7fb2ffd640 1 -- 172.21.15.143:0/1156739158 >> v2:172.21.15.188:3300/0 conn(0x7f7fb40b2160 msgr2=0x7f7fb40b0750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-26T07:19:50.096 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:50.091+0000 7f7fb2ffd640 1 --2- 172.21.15.143:0/1156739158 >> v2:172.21.15.188:3300/0 conn(0x7f7fb40b2160 0x7f7fb40b0750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:19:50.096 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:50.091+0000 7f7fb2ffd640 1 -- 172.21.15.143:0/1156739158 >> v1:172.21.15.143:6789/0 conn(0x7f7fb415d6e0 legacy=0x7f7fb40d14b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:19:50.096 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:50.091+0000 7f7fb2ffd640 1 -- 172.21.15.143:0/1156739158 --> v2:172.21.15.143:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7fb40cee00 con 0x7f7fb40d1bc0 2023-11-26T07:19:50.098 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:50.095+0000 7f7fb2ffd640 1 --2- 172.21.15.143:0/1156739158 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f7fb40d1bc0 0x7f7fb40d0df0 secure :-1 s=READY pgs=638 cs=0 l=1 rev1=1 crypto rx=0x7f7fb408d820 tx=0x7f7fa801c460 comp rx=0 tx=0).ready entity=mon.0 client_cookie=431c111bc2df4fe2 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:19:50.098 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:50.095+0000 7f7fb27fc640 1 -- 172.21.15.143:0/1156739158 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7fa8004230 con 0x7f7fb40d1bc0 2023-11-26T07:19:50.098 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:50.095+0000 7f7fb27fc640 1 -- 172.21.15.143:0/1156739158 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7fa80a7050 con 0x7f7fb40d1bc0 2023-11-26T07:19:50.098 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:50.095+0000 7f7fb27fc640 1 -- 172.21.15.143:0/1156739158 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7fa8003620 con 0x7f7fb40d1bc0 2023-11-26T07:19:50.099 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:50.095+0000 7f7fba135640 1 -- 172.21.15.143:0/1156739158 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f7fb40d1bc0 msgr2=0x7f7fb40d0df0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:19:50.099 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:50.095+0000 7f7fba135640 1 --2- 172.21.15.143:0/1156739158 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f7fb40d1bc0 0x7f7fb40d0df0 secure :-1 s=READY pgs=638 cs=0 l=1 rev1=1 crypto rx=0x7f7fb408d820 tx=0x7f7fa801c460 comp rx=0 tx=0).stop 2023-11-26T07:19:50.099 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:50.095+0000 7f7fba135640 1 -- 172.21.15.143:0/1156739158 shutdown_connections 2023-11-26T07:19:50.099 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:50.095+0000 7f7fba135640 1 --2- 172.21.15.143:0/1156739158 >> v2:172.21.15.188:3300/0 conn(0x7f7fb40b2160 0x7f7fb40b0750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:19:50.099 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:50.095+0000 7f7fba135640 1 --2- 172.21.15.143:0/1156739158 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f7fb40d1bc0 0x7f7fb40d0df0 unknown :-1 s=CLOSED pgs=638 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:19:50.099 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:50.095+0000 7f7fba135640 1 -- 172.21.15.143:0/1156739158 >> 172.21.15.143:0/1156739158 conn(0x7f7fb409c550 msgr2=0x7f7fb40d3220 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:19:50.099 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:50.095+0000 7f7fba135640 1 -- 172.21.15.143:0/1156739158 shutdown_connections 2023-11-26T07:19:50.100 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:50.095+0000 7f7fba135640 1 -- 172.21.15.143:0/1156739158 wait complete. 2023-11-26T07:19:50.100 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:50.099+0000 7f7fba135640 1 Processor -- start 2023-11-26T07:19:50.101 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:50.099+0000 7f7fba135640 1 -- start start 2023-11-26T07:19:50.102 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:50.099+0000 7f7fba135640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f7fb40b2160 0x7f7fb40ca410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:19:50.102 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:50.099+0000 7f7fba135640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f7fb40d1bc0 0x7f7fb4154f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:19:50.102 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:50.099+0000 7f7fba135640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f7fb415d6e0 0x7f7fb406b560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:19:50.102 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:50.099+0000 7f7fba135640 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7fb40cd230 con 0x7f7fb415d6e0 2023-11-26T07:19:50.102 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:50.099+0000 7f7fba135640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7fb40cbef0 con 0x7f7fb40b2160 2023-11-26T07:19:50.102 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:50.099+0000 7f7fba135640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7fb40cc190 con 0x7f7fb40d1bc0 2023-11-26T07:19:50.103 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:50.099+0000 7f7fb3fff640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f7fb415d6e0 0x7f7fb406b560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:19:50.103 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:50.099+0000 7f7fb3fff640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f7fb415d6e0 0x7f7fb406b560 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:58338/0 (socket says 172.21.15.143:58338) 2023-11-26T07:19:50.103 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:50.099+0000 7f7fb3fff640 1 -- 172.21.15.143:0/3527391176 learned_addr learned my addr 172.21.15.143:0/3527391176 (peer_addr_for_me v2:172.21.15.143:0/0) 2023-11-26T07:19:50.103 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:50.099+0000 7f7fb37fe640 1 --2- 172.21.15.143:0/3527391176 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f7fb40b2160 0x7f7fb40ca410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:19:50.103 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:50.099+0000 7f7fb2ffd640 1 --2- 172.21.15.143:0/3527391176 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f7fb40d1bc0 0x7f7fb4154f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:19:50.104 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:50.099+0000 7f7fb3fff640 1 -- 172.21.15.143:0/3527391176 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f7fb40d1bc0 msgr2=0x7f7fb4154f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:19:50.104 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:50.099+0000 7f7fb3fff640 1 --2- 172.21.15.143:0/3527391176 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f7fb40d1bc0 0x7f7fb4154f20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:19:50.104 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:50.099+0000 7f7fb3fff640 1 -- 172.21.15.143:0/3527391176 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f7fb40b2160 msgr2=0x7f7fb40ca410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:19:50.104 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:50.099+0000 7f7fb3fff640 1 --2- 172.21.15.143:0/3527391176 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f7fb40b2160 0x7f7fb40ca410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:19:50.105 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:50.099+0000 7f7fb3fff640 1 -- 172.21.15.143:0/3527391176 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7fa8009420 con 0x7f7fb415d6e0 2023-11-26T07:19:50.105 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:50.099+0000 7f7fb3fff640 1 --2- 172.21.15.143:0/3527391176 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f7fb415d6e0 0x7f7fb406b560 secure :-1 s=READY pgs=639 cs=0 l=1 rev1=1 crypto rx=0x7f7fac007e30 tx=0x7f7fac00e4b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:19:50.105 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:50.099+0000 7f7f9bfff640 1 -- 172.21.15.143:0/3527391176 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7fac014620 con 0x7f7fb415d6e0 2023-11-26T07:19:50.105 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:50.099+0000 7f7f9bfff640 1 -- 172.21.15.143:0/3527391176 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7fac00ee90 con 0x7f7fb415d6e0 2023-11-26T07:19:50.105 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:50.099+0000 7f7f9bfff640 1 -- 172.21.15.143:0/3527391176 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7fac0149c0 con 0x7f7fb415d6e0 2023-11-26T07:19:50.105 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:50.099+0000 7f7fba135640 1 -- 172.21.15.143:0/3527391176 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7fb40cc470 con 0x7f7fb415d6e0 2023-11-26T07:19:50.105 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:50.099+0000 7f7fba135640 1 -- 172.21.15.143:0/3527391176 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7fb40cca30 con 0x7f7fb415d6e0 2023-11-26T07:19:50.105 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:50.099+0000 7f7fba135640 1 -- 172.21.15.143:0/3527391176 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7fb4067680 con 0x7f7fb415d6e0 2023-11-26T07:19:50.105 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:50.103+0000 7f7f9bfff640 1 -- 172.21.15.143:0/3527391176 <== mon.0 v2:172.21.15.143:3300/0 4 ==== mgrmap(e 7) v1 ==== 86005+0+0 (secure 0 0 0) 0x7f7fac014b20 con 0x7f7fb415d6e0 2023-11-26T07:19:50.106 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:50.103+0000 7f7f9bfff640 1 --2- 172.21.15.143:0/3527391176 >> [v2:172.21.15.143:6832/2150557544,v1:172.21.15.143:6833/2150557544] conn(0x7f7fa0066620 0x7f7fa0068ae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:19:50.106 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:50.103+0000 7f7f9bfff640 1 -- 172.21.15.143:0/3527391176 <== mon.0 v2:172.21.15.143:3300/0 5 ==== osd_map(45..45 src has 1..45) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f7fac0849d0 con 0x7f7fb415d6e0 2023-11-26T07:19:50.106 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:50.103+0000 7f7fb37fe640 1 --2- 172.21.15.143:0/3527391176 >> [v2:172.21.15.143:6832/2150557544,v1:172.21.15.143:6833/2150557544] conn(0x7f7fa0066620 0x7f7fa0068ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:19:50.107 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:50.103+0000 7f7fb37fe640 1 --2- 172.21.15.143:0/3527391176 >> [v2:172.21.15.143:6832/2150557544,v1:172.21.15.143:6833/2150557544] conn(0x7f7fa0066620 0x7f7fa0068ae0 secure :-1 s=READY pgs=312 cs=0 l=1 rev1=1 crypto rx=0x7f7fa4005890 tx=0x7f7fa4009040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:19:50.110 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:50.107+0000 7f7f9bfff640 1 -- 172.21.15.143:0/3527391176 <== mon.0 v2:172.21.15.143:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172649 (secure 0 0 0) 0x7f7fac051c80 con 0x7f7fb415d6e0 2023-11-26T07:19:50.260 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:50.255+0000 7f7fba135640 1 -- 172.21.15.143:0/3527391176 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f7fb40d2f40 con 0x7f7fb415d6e0 2023-11-26T07:19:50.260 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:50.255+0000 7f7f9bfff640 1 -- 172.21.15.143:0/3527391176 <== mon.0 v2:172.21.15.143:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v45) v1 ==== 100+0+11 (secure 0 0 0) 0x7f7fac0552a0 con 0x7f7fb415d6e0 2023-11-26T07:19:50.260 INFO:teuthology.orchestra.run.smithi143.stdout:pg_num: 64 2023-11-26T07:19:50.262 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:50.259+0000 7f7fba135640 1 -- 172.21.15.143:0/3527391176 >> [v2:172.21.15.143:6832/2150557544,v1:172.21.15.143:6833/2150557544] conn(0x7f7fa0066620 msgr2=0x7f7fa0068ae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:19:50.263 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:50.259+0000 7f7fba135640 1 --2- 172.21.15.143:0/3527391176 >> [v2:172.21.15.143:6832/2150557544,v1:172.21.15.143:6833/2150557544] conn(0x7f7fa0066620 0x7f7fa0068ae0 secure :-1 s=READY pgs=312 cs=0 l=1 rev1=1 crypto rx=0x7f7fa4005890 tx=0x7f7fa4009040 comp rx=0 tx=0).stop 2023-11-26T07:19:50.263 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:50.259+0000 7f7fba135640 1 -- 172.21.15.143:0/3527391176 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f7fb415d6e0 msgr2=0x7f7fb406b560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:19:50.263 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:50.259+0000 7f7fba135640 1 --2- 172.21.15.143:0/3527391176 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f7fb415d6e0 0x7f7fb406b560 secure :-1 s=READY pgs=639 cs=0 l=1 rev1=1 crypto rx=0x7f7fac007e30 tx=0x7f7fac00e4b0 comp rx=0 tx=0).stop 2023-11-26T07:19:50.263 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:50.259+0000 7f7fba135640 1 -- 172.21.15.143:0/3527391176 shutdown_connections 2023-11-26T07:19:50.263 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:50.259+0000 7f7fba135640 1 --2- 172.21.15.143:0/3527391176 >> [v2:172.21.15.143:6832/2150557544,v1:172.21.15.143:6833/2150557544] conn(0x7f7fa0066620 0x7f7fa0068ae0 unknown :-1 s=CLOSED pgs=312 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:19:50.263 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:50.259+0000 7f7fba135640 1 --2- 172.21.15.143:0/3527391176 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f7fb415d6e0 0x7f7fb406b560 unknown :-1 s=CLOSED pgs=639 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:19:50.263 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:50.259+0000 7f7fba135640 1 --2- 172.21.15.143:0/3527391176 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f7fb40d1bc0 0x7f7fb4154f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:19:50.263 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:50.259+0000 7f7fba135640 1 --2- 172.21.15.143:0/3527391176 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f7fb40b2160 0x7f7fb40ca410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:19:50.263 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:50.259+0000 7f7fba135640 1 -- 172.21.15.143:0/3527391176 >> 172.21.15.143:0/3527391176 conn(0x7f7fb409c550 msgr2=0x7f7fb40d05a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:19:50.263 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:50.259+0000 7f7fba135640 1 -- 172.21.15.143:0/3527391176 shutdown_connections 2023-11-26T07:19:50.263 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:50.259+0000 7f7fba135640 1 -- 172.21.15.143:0/3527391176 wait complete. 2023-11-26T07:19:50.282 DEBUG:teuthology.orchestra.run.smithi143:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2023-11-26T07:19:50.423 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:50.419+0000 7fecf2fa1640 1 Processor -- start 2023-11-26T07:19:50.423 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:50.419+0000 7fecf2fa1640 1 -- start start 2023-11-26T07:19:50.424 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:50.419+0000 7fecf2fa1640 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7fecec15d6e0 0x7fecec0d0df0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:19:50.424 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:50.419+0000 7fecf2fa1640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fecec0b2160 0x7fecec0d14b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:19:50.424 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:50.419+0000 7fecf2fa1640 1 -- --> v1:172.21.15.188:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fecec0cf0e0 con 0x7fecec0d19f0 2023-11-26T07:19:50.424 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:50.419+0000 7fecf2fa1640 1 -- --> v2:172.21.15.143:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fecec15dac0 con 0x7fecec15d6e0 2023-11-26T07:19:50.424 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:50.419+0000 7fecf2fa1640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fecec0d1e80 con 0x7fecec0b2160 2023-11-26T07:19:50.425 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:50.419+0000 7fecf0d3d640 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7fecec15d6e0 0x7fecec0d0df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:19:50.425 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:50.419+0000 7fecf0d3d640 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7fecec15d6e0 0x7fecec0d0df0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:58352/0 (socket says 172.21.15.143:58352) 2023-11-26T07:19:50.425 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:50.419+0000 7fecf0d3d640 1 -- 172.21.15.143:0/3864349259 learned_addr learned my addr 172.21.15.143:0/3864349259 (peer_addr_for_me v2:172.21.15.143:0/0) 2023-11-26T07:19:50.425 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:50.419+0000 7fecebfff640 1 --2- 172.21.15.143:0/3864349259 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fecec0b2160 0x7fecec0d14b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:19:50.425 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:50.423+0000 7fecf0d3d640 1 -- 172.21.15.143:0/3864349259 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fecec0b2160 msgr2=0x7fecec0d14b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-26T07:19:50.425 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:50.423+0000 7fecf0d3d640 1 --2- 172.21.15.143:0/3864349259 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fecec0b2160 0x7fecec0d14b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:19:50.426 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:50.423+0000 7fecf0d3d640 1 -- 172.21.15.143:0/3864349259 >> v1:172.21.15.188:6789/0 conn(0x7fecec0d19f0 legacy=0x7fecec0cfbf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:19:50.426 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:50.423+0000 7fecf0d3d640 1 -- 172.21.15.143:0/3864349259 --> v2:172.21.15.143:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fecec0ceb20 con 0x7fecec15d6e0 2023-11-26T07:19:50.427 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:50.423+0000 7fecf0d3d640 1 --2- 172.21.15.143:0/3864349259 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fecec15d6e0 0x7fecec0d0df0 secure :-1 s=READY pgs=640 cs=0 l=1 rev1=1 crypto rx=0x7fece0009870 tx=0x7fece001c540 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8ee9ee5e849ba0f4 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:19:50.428 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:50.423+0000 7feceb7fe640 1 -- 172.21.15.143:0/3864349259 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fece00a7070 con 0x7fecec15d6e0 2023-11-26T07:19:50.428 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:50.423+0000 7feceb7fe640 1 -- 172.21.15.143:0/3864349259 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fece00ab430 con 0x7fecec15d6e0 2023-11-26T07:19:50.428 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:50.423+0000 7feceb7fe640 1 -- 172.21.15.143:0/3864349259 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fece00ab7c0 con 0x7fecec15d6e0 2023-11-26T07:19:50.428 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:50.423+0000 7fecf2fa1640 1 -- 172.21.15.143:0/3864349259 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fecec15d6e0 msgr2=0x7fecec0d0df0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:19:50.428 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:50.423+0000 7fecf2fa1640 1 --2- 172.21.15.143:0/3864349259 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fecec15d6e0 0x7fecec0d0df0 secure :-1 s=READY pgs=640 cs=0 l=1 rev1=1 crypto rx=0x7fece0009870 tx=0x7fece001c540 comp rx=0 tx=0).stop 2023-11-26T07:19:50.429 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:50.423+0000 7fecf2fa1640 1 -- 172.21.15.143:0/3864349259 shutdown_connections 2023-11-26T07:19:50.429 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:50.423+0000 7fecf2fa1640 1 --2- 172.21.15.143:0/3864349259 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fecec0b2160 0x7fecec0d14b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:19:50.429 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:50.423+0000 7fecf2fa1640 1 --2- 172.21.15.143:0/3864349259 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fecec15d6e0 0x7fecec0d0df0 unknown :-1 s=CLOSED pgs=640 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:19:50.429 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:50.423+0000 7fecf2fa1640 1 -- 172.21.15.143:0/3864349259 >> 172.21.15.143:0/3864349259 conn(0x7fecec09c550 msgr2=0x7fecec0d3220 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:19:50.429 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:50.427+0000 7fecf2fa1640 1 -- 172.21.15.143:0/3864349259 shutdown_connections 2023-11-26T07:19:50.429 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:50.427+0000 7fecf2fa1640 1 -- 172.21.15.143:0/3864349259 wait complete. 2023-11-26T07:19:50.430 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:50.427+0000 7fecf2fa1640 1 Processor -- start 2023-11-26T07:19:50.430 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:50.427+0000 7fecf2fa1640 1 -- start start 2023-11-26T07:19:50.432 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:50.427+0000 7fecf2fa1640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fecec0b2160 0x7fecec0b86c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:19:50.432 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:50.427+0000 7fecf2fa1640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fecec0d19f0 0x7fecec151be0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:19:50.432 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:50.427+0000 7fecf2fa1640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fecec15d6e0 0x7fecec15fbd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:19:50.432 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:50.427+0000 7fecf2fa1640 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fecec0b8c00 con 0x7fecec0b2160 2023-11-26T07:19:50.432 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:50.427+0000 7fecf2fa1640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fecec15e7b0 con 0x7fecec15d6e0 2023-11-26T07:19:50.432 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:50.427+0000 7fecf2fa1640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fecec15ea90 con 0x7fecec0d19f0 2023-11-26T07:19:50.432 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:50.427+0000 7fecf0d3d640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fecec0b2160 0x7fecec0b86c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:19:50.432 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:50.427+0000 7fecf0d3d640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fecec0b2160 0x7fecec0b86c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:58360/0 (socket says 172.21.15.143:58360) 2023-11-26T07:19:50.433 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:50.427+0000 7fecf0d3d640 1 -- 172.21.15.143:0/3075265128 learned_addr learned my addr 172.21.15.143:0/3075265128 (peer_addr_for_me v2:172.21.15.143:0/0) 2023-11-26T07:19:50.433 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:50.427+0000 7fecf153e640 1 --2- 172.21.15.143:0/3075265128 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fecec15d6e0 0x7fecec15fbd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:19:50.433 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:50.427+0000 7fecebfff640 1 --2- 172.21.15.143:0/3075265128 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fecec0d19f0 0x7fecec151be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:19:50.433 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:50.427+0000 7fecf0d3d640 1 -- 172.21.15.143:0/3075265128 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fecec0d19f0 msgr2=0x7fecec151be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:19:50.433 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:50.427+0000 7fecf0d3d640 1 --2- 172.21.15.143:0/3075265128 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fecec0d19f0 0x7fecec151be0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:19:50.433 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:50.427+0000 7fecf0d3d640 1 -- 172.21.15.143:0/3075265128 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fecec15d6e0 msgr2=0x7fecec15fbd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:19:50.433 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:50.427+0000 7fecf0d3d640 1 --2- 172.21.15.143:0/3075265128 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fecec15d6e0 0x7fecec15fbd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:19:50.433 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:50.427+0000 7fecf0d3d640 1 -- 172.21.15.143:0/3075265128 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fece00094d0 con 0x7fecec0b2160 2023-11-26T07:19:50.433 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:50.427+0000 7fecebfff640 1 --2- 172.21.15.143:0/3075265128 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fecec0d19f0 0x7fecec151be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-26T07:19:50.434 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:50.431+0000 7fecf0d3d640 1 --2- 172.21.15.143:0/3075265128 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fecec0b2160 0x7fecec0b86c0 secure :-1 s=READY pgs=641 cs=0 l=1 rev1=1 crypto rx=0x7fece00ad040 tx=0x7fece00192e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:19:50.434 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:50.431+0000 7fece97fa640 1 -- 172.21.15.143:0/3075265128 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fece00bc070 con 0x7fecec0b2160 2023-11-26T07:19:50.434 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:50.431+0000 7fece97fa640 1 -- 172.21.15.143:0/3075265128 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fece00b4420 con 0x7fecec0b2160 2023-11-26T07:19:50.434 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:50.431+0000 7fece97fa640 1 -- 172.21.15.143:0/3075265128 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fece00b47d0 con 0x7fecec0b2160 2023-11-26T07:19:50.434 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:50.431+0000 7fecf2fa1640 1 -- 172.21.15.143:0/3075265128 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fecec15ed70 con 0x7fecec0b2160 2023-11-26T07:19:50.434 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:50.431+0000 7fecf2fa1640 1 -- 172.21.15.143:0/3075265128 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fecec152120 con 0x7fecec0b2160 2023-11-26T07:19:50.435 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:50.431+0000 7fecf2fa1640 1 -- 172.21.15.143:0/3075265128 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fecec067fa0 con 0x7fecec0b2160 2023-11-26T07:19:50.435 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:50.431+0000 7fece97fa640 1 -- 172.21.15.143:0/3075265128 <== mon.0 v2:172.21.15.143:3300/0 4 ==== mgrmap(e 7) v1 ==== 86005+0+0 (secure 0 0 0) 0x7fece00b4930 con 0x7fecec0b2160 2023-11-26T07:19:50.436 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:50.431+0000 7fece97fa640 1 --2- 172.21.15.143:0/3075265128 >> [v2:172.21.15.143:6832/2150557544,v1:172.21.15.143:6833/2150557544] conn(0x7fecd0066620 0x7fecd0068ae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:19:50.436 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:50.435+0000 7fece97fa640 1 -- 172.21.15.143:0/3075265128 <== mon.0 v2:172.21.15.143:3300/0 5 ==== osd_map(45..45 src has 1..45) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fece011db00 con 0x7fecec0b2160 2023-11-26T07:19:50.436 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:50.435+0000 7fecebfff640 1 --2- 172.21.15.143:0/3075265128 >> [v2:172.21.15.143:6832/2150557544,v1:172.21.15.143:6833/2150557544] conn(0x7fecd0066620 0x7fecd0068ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:19:50.437 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:50.435+0000 7fecebfff640 1 --2- 172.21.15.143:0/3075265128 >> [v2:172.21.15.143:6832/2150557544,v1:172.21.15.143:6833/2150557544] conn(0x7fecd0066620 0x7fecd0068ae0 secure :-1 s=READY pgs=313 cs=0 l=1 rev1=1 crypto rx=0x7fecec0b20d0 tx=0x7fecd8008040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:19:50.440 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:50.435+0000 7fece97fa640 1 -- 172.21.15.143:0/3075265128 <== mon.0 v2:172.21.15.143:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172649 (secure 0 0 0) 0x7fece00eac80 con 0x7fecec0b2160 2023-11-26T07:19:50.611 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:50.607+0000 7fecf2fa1640 1 -- 172.21.15.143:0/3075265128 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) v1 -- 0x7fecec15ebd0 con 0x7fecec0b2160 2023-11-26T07:19:50.612 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:50.607+0000 7fece97fa640 1 -- 172.21.15.143:0/3075265128 <== mon.0 v2:172.21.15.143:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v45) v1 ==== 80+0+0 (secure 0 0 0) 0x7fecec15ebd0 con 0x7fecec0b2160 2023-11-26T07:19:50.612 INFO:teuthology.orchestra.run.smithi143.stderr:listed 0 entries 2023-11-26T07:19:50.614 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:50.611+0000 7fecf2fa1640 1 -- 172.21.15.143:0/3075265128 >> [v2:172.21.15.143:6832/2150557544,v1:172.21.15.143:6833/2150557544] conn(0x7fecd0066620 msgr2=0x7fecd0068ae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:19:50.614 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:50.611+0000 7fecf2fa1640 1 --2- 172.21.15.143:0/3075265128 >> [v2:172.21.15.143:6832/2150557544,v1:172.21.15.143:6833/2150557544] conn(0x7fecd0066620 0x7fecd0068ae0 secure :-1 s=READY pgs=313 cs=0 l=1 rev1=1 crypto rx=0x7fecec0b20d0 tx=0x7fecd8008040 comp rx=0 tx=0).stop 2023-11-26T07:19:50.614 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:50.611+0000 7fecf2fa1640 1 -- 172.21.15.143:0/3075265128 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fecec0b2160 msgr2=0x7fecec0b86c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:19:50.614 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:50.611+0000 7fecf2fa1640 1 --2- 172.21.15.143:0/3075265128 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fecec0b2160 0x7fecec0b86c0 secure :-1 s=READY pgs=641 cs=0 l=1 rev1=1 crypto rx=0x7fece00ad040 tx=0x7fece00192e0 comp rx=0 tx=0).stop 2023-11-26T07:19:50.615 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:50.611+0000 7fecf2fa1640 1 -- 172.21.15.143:0/3075265128 shutdown_connections 2023-11-26T07:19:50.615 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:50.611+0000 7fecf2fa1640 1 --2- 172.21.15.143:0/3075265128 >> [v2:172.21.15.143:6832/2150557544,v1:172.21.15.143:6833/2150557544] conn(0x7fecd0066620 0x7fecd0068ae0 unknown :-1 s=CLOSED pgs=313 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:19:50.615 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:50.611+0000 7fecf2fa1640 1 --2- 172.21.15.143:0/3075265128 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fecec15d6e0 0x7fecec15fbd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:19:50.615 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:50.611+0000 7fecf2fa1640 1 --2- 172.21.15.143:0/3075265128 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fecec0d19f0 0x7fecec151be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:19:50.615 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:50.611+0000 7fecf2fa1640 1 --2- 172.21.15.143:0/3075265128 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fecec0b2160 0x7fecec0b86c0 unknown :-1 s=CLOSED pgs=641 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:19:50.615 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:50.611+0000 7fecf2fa1640 1 -- 172.21.15.143:0/3075265128 >> 172.21.15.143:0/3075265128 conn(0x7fecec09c550 msgr2=0x7fecec0b2870 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:19:50.615 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:50.611+0000 7fecf2fa1640 1 -- 172.21.15.143:0/3075265128 shutdown_connections 2023-11-26T07:19:50.615 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:50.611+0000 7fecf2fa1640 1 -- 172.21.15.143:0/3075265128 wait complete. 2023-11-26T07:19:50.633 INFO:tasks.cephfs.fuse_mount:Running fusermount -u on ubuntu@smithi143.front.sepia.ceph.com... 2023-11-26T07:19:50.633 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-26T07:19:50.634 DEBUG:teuthology.orchestra.run.smithi143:> sudo fusermount -u /home/ubuntu/cephtest/mnt.0 2023-11-26T07:19:50.652 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi143.stderr:ceph-fuse[50877]: fuse finished with error 0 and tester_r 0 2023-11-26T07:19:50.655 INFO:teuthology.orchestra.run:waiting for 300 2023-11-26T07:19:51.923 DEBUG:teuthology.orchestra.run.smithi143:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-26T07:19:51.927 DEBUG:teuthology.orchestra.run.smithi188:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-26T07:19:56.757 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi143.front.sepia.ceph.com 2023-11-26T07:19:56.757 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-26T07:19:56.757 DEBUG:teuthology.orchestra.run.smithi143:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2023-11-26T07:19:56.768 DEBUG:teuthology.orchestra.run.smithi188:> set -ex 2023-11-26T07:19:56.768 DEBUG:teuthology.orchestra.run.smithi188:> dd if=/proc/self/mounts of=/dev/stdout 2023-11-26T07:19:56.778 DEBUG:teuthology.orchestra.run.smithi143:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-11-26T07:19:56.955 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:56.951+0000 7f88d472a640 1 Processor -- start 2023-11-26T07:19:56.955 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:56.951+0000 7f88d472a640 1 -- start start 2023-11-26T07:19:56.956 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:56.951+0000 7f88d472a640 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7f88cc0cf280 0x7f88cc0cf710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:19:56.956 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:56.951+0000 7f88d472a640 1 -- --> v1:172.21.15.143:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f88cc0cc730 con 0x7f88cc143ac0 2023-11-26T07:19:56.956 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:56.951+0000 7f88d472a640 1 -- --> v1:172.21.15.188:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f88cc070ee0 con 0x7f88cc0cd1b0 2023-11-26T07:19:56.956 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:56.951+0000 7f88d472a640 1 -- --> v2:172.21.15.188:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f88cc143ea0 con 0x7f88cc0cf280 2023-11-26T07:19:56.957 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:56.955+0000 7f88d24c6640 1 --1- >> v1:172.21.15.143:6789/0 conn(0x7f88cc143ac0 0x7f88cc070370 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.143:6789/0 says I am v1:172.21.15.143:47554/0 (socket says 172.21.15.143:47554) 2023-11-26T07:19:56.957 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:56.955+0000 7f88d24c6640 1 -- 172.21.15.143:0/2079629585 learned_addr learned my addr 172.21.15.143:0/2079629585 (peer_addr_for_me v1:172.21.15.143:0/0) 2023-11-26T07:19:56.959 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:56.955+0000 7f88d1cc5640 1 --2- 172.21.15.143:0/2079629585 >> v2:172.21.15.188:3300/0 conn(0x7f88cc0cf280 0x7f88cc0cf710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:19:56.959 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:56.955+0000 7f88d14c4640 1 -- 172.21.15.143:0/2079629585 <== mon.0 v1:172.21.15.143:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3937570172 0 0) 0x7f88cc0cc730 con 0x7f88cc143ac0 2023-11-26T07:19:56.959 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:56.955+0000 7f88d14c4640 1 -- 172.21.15.143:0/2079629585 --> v1:172.21.15.143:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f88b8005740 con 0x7f88cc143ac0 2023-11-26T07:19:56.959 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:56.955+0000 7f88d14c4640 1 -- 172.21.15.143:0/2079629585 <== mon.0 v1:172.21.15.143:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1577618269 0 0) 0x7f88b8005740 con 0x7f88cc143ac0 2023-11-26T07:19:56.959 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:56.955+0000 7f88d14c4640 1 -- 172.21.15.143:0/2079629585 >> v2:172.21.15.188:3300/0 conn(0x7f88cc0cf280 msgr2=0x7f88cc0cf710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-26T07:19:56.959 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:56.955+0000 7f88d14c4640 1 --2- 172.21.15.143:0/2079629585 >> v2:172.21.15.188:3300/0 conn(0x7f88cc0cf280 0x7f88cc0cf710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:19:56.959 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:56.955+0000 7f88d14c4640 1 -- 172.21.15.143:0/2079629585 >> v1:172.21.15.188:6789/0 conn(0x7f88cc0cd1b0 legacy=0x7f88cc0cd5c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:19:56.959 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:56.955+0000 7f88d14c4640 1 -- 172.21.15.143:0/2079629585 --> v1:172.21.15.143:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f88cc0711c0 con 0x7f88cc143ac0 2023-11-26T07:19:56.959 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:56.955+0000 7f88d14c4640 1 -- 172.21.15.143:0/2079629585 <== mon.0 v1:172.21.15.143:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2137882523 0 0) 0x7f88c8001680 con 0x7f88cc143ac0 2023-11-26T07:19:56.959 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:56.955+0000 7f88d14c4640 1 -- 172.21.15.143:0/2079629585 >> v1:172.21.15.143:6789/0 conn(0x7f88cc143ac0 legacy=0x7f88cc070370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:19:56.959 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:56.955+0000 7f88d14c4640 1 --2- 172.21.15.143:0/2079629585 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f88b8005b90 0x7f88b8005f70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:19:56.959 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:56.955+0000 7f88d14c4640 1 --2- 172.21.15.143:0/2079629585 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f88b80069e0 0x7f88b8006e60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:19:56.959 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:56.955+0000 7f88d14c4640 1 --2- 172.21.15.143:0/2079629585 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f88b80073a0 0x7f88b800dba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:19:56.960 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:56.955+0000 7f88d14c4640 1 -- 172.21.15.143:0/2079629585 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f88b800e1c0 con 0x7f88b80073a0 2023-11-26T07:19:56.960 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:56.955+0000 7f88d14c4640 1 -- 172.21.15.143:0/2079629585 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f88b800e330 con 0x7f88b80069e0 2023-11-26T07:19:56.960 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:56.955+0000 7f88d14c4640 1 -- 172.21.15.143:0/2079629585 --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f88b800e4f0 con 0x7f88b8005b90 2023-11-26T07:19:56.960 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:56.955+0000 7f88d2cc7640 1 --2- 172.21.15.143:0/2079629585 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f88b80073a0 0x7f88b800dba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:19:56.960 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:56.955+0000 7f88d1cc5640 1 --2- 172.21.15.143:0/2079629585 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f88b80069e0 0x7f88b8006e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:19:56.960 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:56.955+0000 7f88d24c6640 1 --2- 172.21.15.143:0/2079629585 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f88b8005b90 0x7f88b8005f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:19:56.960 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:56.955+0000 7f88d2cc7640 1 -- 172.21.15.143:0/2079629585 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f88b8005b90 msgr2=0x7f88b8005f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-26T07:19:56.960 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:56.955+0000 7f88d2cc7640 1 --2- 172.21.15.143:0/2079629585 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f88b8005b90 0x7f88b8005f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:19:56.960 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:56.955+0000 7f88d2cc7640 1 -- 172.21.15.143:0/2079629585 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f88b80069e0 msgr2=0x7f88b8006e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-26T07:19:56.960 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:56.955+0000 7f88d2cc7640 1 --2- 172.21.15.143:0/2079629585 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f88b80069e0 0x7f88b8006e60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:19:56.960 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:56.955+0000 7f88d2cc7640 1 -- 172.21.15.143:0/2079629585 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f88b8005740 con 0x7f88b80073a0 2023-11-26T07:19:56.961 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:56.959+0000 7f88d2cc7640 1 -- 172.21.15.143:0/2079629585 reap_dead start 2023-11-26T07:19:56.961 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:56.959+0000 7f88d2cc7640 1 --2- 172.21.15.143:0/2079629585 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f88b80073a0 0x7f88b800dba0 secure :-1 s=READY pgs=643 cs=0 l=1 rev1=1 crypto rx=0x7f88c4001f60 tx=0x7f88c401c860 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7735c6d1fa4ab13a server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:19:56.961 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:56.959+0000 7f88d14c4640 1 -- 172.21.15.143:0/2079629585 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f88c4003b80 con 0x7f88b80073a0 2023-11-26T07:19:56.961 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:56.959+0000 7f88d14c4640 1 -- 172.21.15.143:0/2079629585 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f88c4003ce0 con 0x7f88b80073a0 2023-11-26T07:19:56.961 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:56.959+0000 7f88d14c4640 1 -- 172.21.15.143:0/2079629585 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f88c40052f0 con 0x7f88b80073a0 2023-11-26T07:19:56.962 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:56.959+0000 7f88d472a640 1 -- 172.21.15.143:0/2079629585 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f88b80073a0 msgr2=0x7f88b800dba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:19:56.962 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:56.959+0000 7f88d472a640 1 --2- 172.21.15.143:0/2079629585 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f88b80073a0 0x7f88b800dba0 secure :-1 s=READY pgs=643 cs=0 l=1 rev1=1 crypto rx=0x7f88c4001f60 tx=0x7f88c401c860 comp rx=0 tx=0).stop 2023-11-26T07:19:56.962 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:56.959+0000 7f88d472a640 1 -- 172.21.15.143:0/2079629585 shutdown_connections 2023-11-26T07:19:56.962 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:56.959+0000 7f88d472a640 1 --2- 172.21.15.143:0/2079629585 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f88b80073a0 0x7f88b800dba0 unknown :-1 s=CLOSED pgs=643 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:19:56.962 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:56.959+0000 7f88d472a640 1 -- 172.21.15.143:0/2079629585 >> 172.21.15.143:0/2079629585 conn(0x7f88cc06a890 msgr2=0x7f88cc06aca0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:19:56.962 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:56.959+0000 7f88d472a640 1 -- 172.21.15.143:0/2079629585 shutdown_connections 2023-11-26T07:19:56.963 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:56.959+0000 7f88d472a640 1 -- 172.21.15.143:0/2079629585 wait complete. 2023-11-26T07:19:56.964 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:56.959+0000 7f88d472a640 1 Processor -- start 2023-11-26T07:19:56.964 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:56.959+0000 7f88d472a640 1 -- start start 2023-11-26T07:19:56.965 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:56.963+0000 7f88d472a640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f88b80073a0 0x7f88cc1380a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:19:56.966 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:56.963+0000 7f88d472a640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f88cc1ef580 0x7f88cc1596a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:19:56.966 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:56.963+0000 7f88d472a640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f88cc139aa0 0x7f88cc0cdc80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:19:56.966 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:56.963+0000 7f88d472a640 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f88cc137370 con 0x7f88cc139aa0 2023-11-26T07:19:56.966 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:56.963+0000 7f88d472a640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f88cc13a4d0 con 0x7f88b80073a0 2023-11-26T07:19:56.966 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:56.963+0000 7f88d472a640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f88cc1375c0 con 0x7f88cc1ef580 2023-11-26T07:19:56.966 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:56.963+0000 7f88d2cc7640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f88cc139aa0 0x7f88cc0cdc80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:19:56.966 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:56.963+0000 7f88d2cc7640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f88cc139aa0 0x7f88cc0cdc80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:35088/0 (socket says 172.21.15.143:35088) 2023-11-26T07:19:56.966 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:56.963+0000 7f88d2cc7640 1 -- 172.21.15.143:0/2237192060 learned_addr learned my addr 172.21.15.143:0/2237192060 (peer_addr_for_me v2:172.21.15.143:0/0) 2023-11-26T07:19:56.966 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:56.963+0000 7f88d24c6640 1 --2- 172.21.15.143:0/2237192060 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f88b80073a0 0x7f88cc1380a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:19:56.966 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:56.963+0000 7f88d1cc5640 1 --2- 172.21.15.143:0/2237192060 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f88cc1ef580 0x7f88cc1596a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:19:56.966 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:56.963+0000 7f88d2cc7640 1 -- 172.21.15.143:0/2237192060 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f88cc1ef580 msgr2=0x7f88cc1596a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:19:56.966 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:56.963+0000 7f88d2cc7640 1 --2- 172.21.15.143:0/2237192060 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f88cc1ef580 0x7f88cc1596a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:19:56.967 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:56.963+0000 7f88d2cc7640 1 -- 172.21.15.143:0/2237192060 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f88b80073a0 msgr2=0x7f88cc1380a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:19:56.967 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:56.963+0000 7f88d2cc7640 1 --2- 172.21.15.143:0/2237192060 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f88b80073a0 0x7f88cc1380a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:19:56.967 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:56.963+0000 7f88d2cc7640 1 -- 172.21.15.143:0/2237192060 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f88cc1378a0 con 0x7f88cc139aa0 2023-11-26T07:19:56.968 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:56.963+0000 7f88d2cc7640 1 --2- 172.21.15.143:0/2237192060 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f88cc139aa0 0x7f88cc0cdc80 secure :-1 s=READY pgs=644 cs=0 l=1 rev1=1 crypto rx=0x7f88c400a030 tx=0x7f88b8007a10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:19:56.968 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:56.963+0000 7f88c2ffd640 1 -- 172.21.15.143:0/2237192060 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f88c4003b80 con 0x7f88cc139aa0 2023-11-26T07:19:56.968 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:56.963+0000 7f88c2ffd640 1 -- 172.21.15.143:0/2237192060 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f88c4006b60 con 0x7f88cc139aa0 2023-11-26T07:19:56.968 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:56.963+0000 7f88d472a640 1 -- 172.21.15.143:0/2237192060 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f88cc137b80 con 0x7f88cc139aa0 2023-11-26T07:19:56.968 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:56.963+0000 7f88c2ffd640 1 -- 172.21.15.143:0/2237192060 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f88c40a5550 con 0x7f88cc139aa0 2023-11-26T07:19:56.968 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:56.963+0000 7f88d472a640 1 -- 172.21.15.143:0/2237192060 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f88cc159be0 con 0x7f88cc139aa0 2023-11-26T07:19:56.968 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:56.963+0000 7f88d472a640 1 -- 172.21.15.143:0/2237192060 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f88cc067680 con 0x7f88cc139aa0 2023-11-26T07:19:56.968 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:56.963+0000 7f88c2ffd640 1 -- 172.21.15.143:0/2237192060 <== mon.0 v2:172.21.15.143:3300/0 4 ==== mgrmap(e 7) v1 ==== 86005+0+0 (secure 0 0 0) 0x7f88c40a56b0 con 0x7f88cc139aa0 2023-11-26T07:19:56.969 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:56.967+0000 7f88c2ffd640 1 --2- 172.21.15.143:0/2237192060 >> [v2:172.21.15.143:6832/2150557544,v1:172.21.15.143:6833/2150557544] conn(0x7f88b0066650 0x7f88b0068b10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:19:56.969 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:56.967+0000 7f88c2ffd640 1 -- 172.21.15.143:0/2237192060 <== mon.0 v2:172.21.15.143:3300/0 5 ==== osd_map(45..45 src has 1..45) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f88c411d160 con 0x7f88cc139aa0 2023-11-26T07:19:56.969 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:56.967+0000 7f88d24c6640 1 --2- 172.21.15.143:0/2237192060 >> [v2:172.21.15.143:6832/2150557544,v1:172.21.15.143:6833/2150557544] conn(0x7f88b0066650 0x7f88b0068b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:19:56.970 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:56.967+0000 7f88d24c6640 1 --2- 172.21.15.143:0/2237192060 >> [v2:172.21.15.143:6832/2150557544,v1:172.21.15.143:6833/2150557544] conn(0x7f88b0066650 0x7f88b0068b10 secure :-1 s=READY pgs=314 cs=0 l=1 rev1=1 crypto rx=0x7f88c8000c10 tx=0x7f88c8001b60 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:19:56.973 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:56.971+0000 7f88c2ffd640 1 -- 172.21.15.143:0/2237192060 <== mon.0 v2:172.21.15.143:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172649 (secure 0 0 0) 0x7f88c40ea410 con 0x7f88cc139aa0 2023-11-26T07:19:57.123 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:57.119+0000 7f88d472a640 1 -- 172.21.15.143:0/2237192060 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f88cc15d3c0 con 0x7f88cc139aa0 2023-11-26T07:19:57.125 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:57.123+0000 7f88c2ffd640 1 -- 172.21.15.143:0/2237192060 <== mon.0 v2:172.21.15.143:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v45) v1 ==== 74+0+16549 (secure 0 0 0) 0x7f88c40eda30 con 0x7f88cc139aa0 2023-11-26T07:19:57.125 INFO:teuthology.orchestra.run.smithi143.stdout: 2023-11-26T07:19:57.125 INFO:teuthology.orchestra.run.smithi143.stdout:{"epoch":45,"fsid":"f5033060-2469-4a07-95fe-2b76f5404bc1","created":"2023-11-26T07:17:28.613663+0000","modified":"2023-11-26T07:19:33.132471+0000","last_up_change":"2023-11-26T07:17:41.157813+0000","last_in_change":"2023-11-26T07:17:31.181658+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":7,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-11-26T07:17:41.758597+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":6,"pool_name":"cephfs_metadata","create_time":"2023-11-26T07:19:29.208075+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"44","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":7,"pool_name":"cephfs_data","create_time":"2023-11-26T07:19:30.503743+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"45","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"ebbee2c6-023e-44b9-9f1d-6f7bfd309730","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":43,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6817","nonce":639807369},{"type":"v1","addr":"172.21.15.143:6819","nonce":639807369}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6821","nonce":639807369},{"type":"v1","addr":"172.21.15.143:6823","nonce":639807369}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6830","nonce":639807369},{"type":"v1","addr":"172.21.15.143:6831","nonce":639807369}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6825","nonce":639807369},{"type":"v1","addr":"172.21.15.143:6828","nonce":639807369}]},"public_addr":"172.21.15.143:6819/639807369","cluster_addr":"172.21.15.143:6823/639807369","heartbeat_back_addr":"172.21.15.143:6831/639807369","heartbeat_front_addr":"172.21.15.143:6828/639807369","state":["exists","up"]},{"osd":1,"uuid":"fec4b8be-7122-49e9-9a19-5850a0700001","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":43,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6816","nonce":1144942530},{"type":"v1","addr":"172.21.15.143:6818","nonce":1144942530}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6820","nonce":1144942530},{"type":"v1","addr":"172.21.15.143:6822","nonce":1144942530}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6827","nonce":1144942530},{"type":"v1","addr":"172.21.15.143:6829","nonce":1144942530}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6824","nonce":1144942530},{"type":"v1","addr":"172.21.15.143:6826","nonce":1144942530}]},"public_addr":"172.21.15.143:6818/1144942530","cluster_addr":"172.21.15.143:6822/1144942530","heartbeat_back_addr":"172.21.15.143:6829/1144942530","heartbeat_front_addr":"172.21.15.143:6826/1144942530","state":["exists","up"]},{"osd":2,"uuid":"2888af5b-f933-46ca-8f95-6dd494ff43f6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":43,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6808","nonce":3909095186},{"type":"v1","addr":"172.21.15.143:6809","nonce":3909095186}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6810","nonce":3909095186},{"type":"v1","addr":"172.21.15.143:6811","nonce":3909095186}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6814","nonce":3909095186},{"type":"v1","addr":"172.21.15.143:6815","nonce":3909095186}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6812","nonce":3909095186},{"type":"v1","addr":"172.21.15.143:6813","nonce":3909095186}]},"public_addr":"172.21.15.143:6809/3909095186","cluster_addr":"172.21.15.143:6811/3909095186","heartbeat_back_addr":"172.21.15.143:6815/3909095186","heartbeat_front_addr":"172.21.15.143:6813/3909095186","state":["exists","up"]},{"osd":3,"uuid":"d1f52435-7bc1-4081-9550-9cd1ba55f549","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":43,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6800","nonce":3854210991},{"type":"v1","addr":"172.21.15.143:6801","nonce":3854210991}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6802","nonce":3854210991},{"type":"v1","addr":"172.21.15.143:6803","nonce":3854210991}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6806","nonce":3854210991},{"type":"v1","addr":"172.21.15.143:6807","nonce":3854210991}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6804","nonce":3854210991},{"type":"v1","addr":"172.21.15.143:6805","nonce":3854210991}]},"public_addr":"172.21.15.143:6801/3854210991","cluster_addr":"172.21.15.143:6803/3854210991","heartbeat_back_addr":"172.21.15.143:6807/3854210991","heartbeat_front_addr":"172.21.15.143:6805/3854210991","state":["exists","up"]},{"osd":4,"uuid":"e2ae8b6e-6e25-49b2-bcbf-aacce68f06a3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":43,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6800","nonce":3924606902},{"type":"v1","addr":"172.21.15.188:6801","nonce":3924606902}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6802","nonce":3924606902},{"type":"v1","addr":"172.21.15.188:6803","nonce":3924606902}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6806","nonce":3924606902},{"type":"v1","addr":"172.21.15.188:6807","nonce":3924606902}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6804","nonce":3924606902},{"type":"v1","addr":"172.21.15.188:6805","nonce":3924606902}]},"public_addr":"172.21.15.188:6801/3924606902","cluster_addr":"172.21.15.188:6803/3924606902","heartbeat_back_addr":"172.21.15.188:6807/3924606902","heartbeat_front_addr":"172.21.15.188:6805/3924606902","state":["exists","up"]},{"osd":5,"uuid":"3cec8a32-7698-4c36-8e89-e4ba11934f33","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":43,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6813","nonce":186841544},{"type":"v1","addr":"172.21.15.188:6815","nonce":186841544}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6817","nonce":186841544},{"type":"v1","addr":"172.21.15.188:6818","nonce":186841544}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6822","nonce":186841544},{"type":"v1","addr":"172.21.15.188:6823","nonce":186841544}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6820","nonce":186841544},{"type":"v1","addr":"172.21.15.188:6821","nonce":186841544}]},"public_addr":"172.21.15.188:6815/186841544","cluster_addr":"172.21.15.188:6818/186841544","heartbeat_back_addr":"172.21.15.188:6823/186841544","heartbeat_front_addr":"172.21.15.188:6821/186841544","state":["exists","up"]},{"osd":6,"uuid":"58a65d5d-e090-473a-81b4-ad38e85029f9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":43,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6808","nonce":2153499405},{"type":"v1","addr":"172.21.15.188:6809","nonce":2153499405}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6810","nonce":2153499405},{"type":"v1","addr":"172.21.15.188:6811","nonce":2153499405}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6816","nonce":2153499405},{"type":"v1","addr":"172.21.15.188:6819","nonce":2153499405}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6812","nonce":2153499405},{"type":"v1","addr":"172.21.15.188:6814","nonce":2153499405}]},"public_addr":"172.21.15.188:6809/2153499405","cluster_addr":"172.21.15.188:6811/2153499405","heartbeat_back_addr":"172.21.15.188:6819/2153499405","heartbeat_front_addr":"172.21.15.188:6814/2153499405","state":["exists","up"]},{"osd":7,"uuid":"ea5b7f70-3554-4cf8-ac30-f7c59b80678d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":43,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6824","nonce":2722096729},{"type":"v1","addr":"172.21.15.188:6825","nonce":2722096729}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6826","nonce":2722096729},{"type":"v1","addr":"172.21.15.188:6827","nonce":2722096729}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6830","nonce":2722096729},{"type":"v1","addr":"172.21.15.188:6831","nonce":2722096729}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6828","nonce":2722096729},{"type":"v1","addr":"172.21.15.188:6829","nonce":2722096729}]},"public_addr":"172.21.15.188:6825/2722096729","cluster_addr":"172.21.15.188:6827/2722096729","heartbeat_back_addr":"172.21.15.188:6831/2722096729","heartbeat_front_addr":"172.21.15.188:6829/2722096729","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-26T07:17:36.995032+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-26T07:17:36.932266+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-26T07:17:36.829304+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-26T07:17:36.804297+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-26T07:17:36.667892+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-26T07:17:36.964868+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-26T07:17:36.827332+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-26T07:17:36.893179+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-11-26T07:19:57.127 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:57.123+0000 7f88d472a640 1 -- 172.21.15.143:0/2237192060 >> [v2:172.21.15.143:6832/2150557544,v1:172.21.15.143:6833/2150557544] conn(0x7f88b0066650 msgr2=0x7f88b0068b10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:19:57.127 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:57.123+0000 7f88d472a640 1 --2- 172.21.15.143:0/2237192060 >> [v2:172.21.15.143:6832/2150557544,v1:172.21.15.143:6833/2150557544] conn(0x7f88b0066650 0x7f88b0068b10 secure :-1 s=READY pgs=314 cs=0 l=1 rev1=1 crypto rx=0x7f88c8000c10 tx=0x7f88c8001b60 comp rx=0 tx=0).stop 2023-11-26T07:19:57.127 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:57.123+0000 7f88d472a640 1 -- 172.21.15.143:0/2237192060 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f88cc139aa0 msgr2=0x7f88cc0cdc80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:19:57.127 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:57.123+0000 7f88d472a640 1 --2- 172.21.15.143:0/2237192060 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f88cc139aa0 0x7f88cc0cdc80 secure :-1 s=READY pgs=644 cs=0 l=1 rev1=1 crypto rx=0x7f88c400a030 tx=0x7f88b8007a10 comp rx=0 tx=0).stop 2023-11-26T07:19:57.127 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:57.123+0000 7f88d472a640 1 -- 172.21.15.143:0/2237192060 shutdown_connections 2023-11-26T07:19:57.127 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:57.123+0000 7f88d472a640 1 --2- 172.21.15.143:0/2237192060 >> [v2:172.21.15.143:6832/2150557544,v1:172.21.15.143:6833/2150557544] conn(0x7f88b0066650 0x7f88b0068b10 unknown :-1 s=CLOSED pgs=314 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:19:57.127 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:57.123+0000 7f88d472a640 1 --2- 172.21.15.143:0/2237192060 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f88cc139aa0 0x7f88cc0cdc80 unknown :-1 s=CLOSED pgs=644 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:19:57.127 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:57.123+0000 7f88d472a640 1 --2- 172.21.15.143:0/2237192060 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f88cc1ef580 0x7f88cc1596a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:19:57.128 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:57.123+0000 7f88d472a640 1 --2- 172.21.15.143:0/2237192060 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f88b80073a0 0x7f88cc1380a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:19:57.128 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:57.123+0000 7f88d472a640 1 -- 172.21.15.143:0/2237192060 >> 172.21.15.143:0/2237192060 conn(0x7f88cc06a890 msgr2=0x7f88cc142420 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:19:57.128 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:57.123+0000 7f88d472a640 1 -- 172.21.15.143:0/2237192060 shutdown_connections 2023-11-26T07:19:57.128 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:57.123+0000 7f88d472a640 1 -- 172.21.15.143:0/2237192060 wait complete. 2023-11-26T07:19:57.146 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-11-26T07:17:41.758597+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 6, 'pool_name': 'cephfs_metadata', 'create_time': '2023-11-26T07:19:29.208075+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '44', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 1.75, 'score_stable': 1.75, 'optimal_score': 1, 'raw_score_acting': 1.75, 'raw_score_stable': 1.75, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 7, 'pool_name': 'cephfs_data', 'create_time': '2023-11-26T07:19:30.503743+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '45', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.75, 'score_stable': 1.75, 'optimal_score': 1, 'raw_score_acting': 1.75, 'raw_score_stable': 1.75, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2023-11-26T07:19:57.146 DEBUG:teuthology.orchestra.run.smithi143:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-11-26T07:19:57.286 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:57.283+0000 7f9136f2a640 1 Processor -- start 2023-11-26T07:19:57.286 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:57.283+0000 7f9136f2a640 1 -- start start 2023-11-26T07:19:57.287 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:57.283+0000 7f9136f2a640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f913013e290 0x7f91300cf280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:19:57.288 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:57.283+0000 7f9136f2a640 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7f91300cf7c0 0x7f913015efa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:19:57.288 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:57.283+0000 7f9136f2a640 1 -- --> v1:172.21.15.188:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f913006c520 con 0x7f913015f4e0 2023-11-26T07:19:57.288 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:57.283+0000 7f9136f2a640 1 -- --> v2:172.21.15.143:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f91300cfc40 con 0x7f91300cf7c0 2023-11-26T07:19:57.288 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:57.283+0000 7f9136f2a640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f913006c800 con 0x7f913013e290 2023-11-26T07:19:57.288 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:57.283+0000 7f912ffff640 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7f91300cf7c0 0x7f913015efa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:19:57.288 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:57.283+0000 7f912ffff640 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7f91300cf7c0 0x7f913015efa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:35104/0 (socket says 172.21.15.143:35104) 2023-11-26T07:19:57.288 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:57.283+0000 7f912ffff640 1 -- 172.21.15.143:0/1562005769 learned_addr learned my addr 172.21.15.143:0/1562005769 (peer_addr_for_me v2:172.21.15.143:0/0) 2023-11-26T07:19:57.288 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:57.283+0000 7f9134cc6640 1 --2- 172.21.15.143:0/1562005769 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f913013e290 0x7f91300cf280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:19:57.289 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:57.283+0000 7f912ffff640 1 -- 172.21.15.143:0/1562005769 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f913013e290 msgr2=0x7f91300cf280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-26T07:19:57.289 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:57.283+0000 7f912ffff640 1 --2- 172.21.15.143:0/1562005769 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f913013e290 0x7f91300cf280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:19:57.289 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:57.283+0000 7f912ffff640 1 -- 172.21.15.143:0/1562005769 >> v1:172.21.15.188:6789/0 conn(0x7f913015f4e0 legacy=0x7f91301408e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:19:57.289 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:57.283+0000 7f912ffff640 1 -- 172.21.15.143:0/1562005769 --> v2:172.21.15.143:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f913006cae0 con 0x7f91300cf7c0 2023-11-26T07:19:57.290 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:57.287+0000 7f912ffff640 1 --2- 172.21.15.143:0/1562005769 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f91300cf7c0 0x7f913015efa0 secure :-1 s=READY pgs=645 cs=0 l=1 rev1=1 crypto rx=0x7f911c009870 tx=0x7f911c01c510 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d071cfd47e43373a server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:19:57.291 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:57.287+0000 7f912f7fe640 1 -- 172.21.15.143:0/1562005769 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f911c0aa670 con 0x7f91300cf7c0 2023-11-26T07:19:57.291 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:57.287+0000 7f912f7fe640 1 -- 172.21.15.143:0/1562005769 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f911c0aa7d0 con 0x7f91300cf7c0 2023-11-26T07:19:57.291 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:57.287+0000 7f912f7fe640 1 -- 172.21.15.143:0/1562005769 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f911c0aab60 con 0x7f91300cf7c0 2023-11-26T07:19:57.292 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:57.287+0000 7f9136f2a640 1 -- 172.21.15.143:0/1562005769 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f91300cf7c0 msgr2=0x7f913015efa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:19:57.292 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:57.287+0000 7f9136f2a640 1 --2- 172.21.15.143:0/1562005769 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f91300cf7c0 0x7f913015efa0 secure :-1 s=READY pgs=645 cs=0 l=1 rev1=1 crypto rx=0x7f911c009870 tx=0x7f911c01c510 comp rx=0 tx=0).stop 2023-11-26T07:19:57.292 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:57.287+0000 7f9136f2a640 1 -- 172.21.15.143:0/1562005769 shutdown_connections 2023-11-26T07:19:57.292 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:57.287+0000 7f9136f2a640 1 --2- 172.21.15.143:0/1562005769 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f91300cf7c0 0x7f913015efa0 unknown :-1 s=CLOSED pgs=645 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:19:57.292 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:57.287+0000 7f9136f2a640 1 --2- 172.21.15.143:0/1562005769 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f913013e290 0x7f91300cf280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:19:57.292 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:57.287+0000 7f9136f2a640 1 -- 172.21.15.143:0/1562005769 >> 172.21.15.143:0/1562005769 conn(0x7f913006a890 msgr2=0x7f913006aca0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:19:57.292 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:57.287+0000 7f9136f2a640 1 -- 172.21.15.143:0/1562005769 shutdown_connections 2023-11-26T07:19:57.292 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:57.287+0000 7f9136f2a640 1 -- 172.21.15.143:0/1562005769 wait complete. 2023-11-26T07:19:57.293 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:57.291+0000 7f9136f2a640 1 Processor -- start 2023-11-26T07:19:57.293 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:57.291+0000 7f9136f2a640 1 -- start start 2023-11-26T07:19:57.295 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:57.291+0000 7f9136f2a640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f91300cf7c0 0x7f91301548a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:19:57.295 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:57.291+0000 7f9136f2a640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f913013e290 0x7f91301418f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:19:57.296 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:57.291+0000 7f9136f2a640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f913015f4e0 0x7f913013ce40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:19:57.296 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:57.291+0000 7f9136f2a640 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9130142ee0 con 0x7f913013e290 2023-11-26T07:19:57.296 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:57.291+0000 7f9136f2a640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9130142260 con 0x7f913015f4e0 2023-11-26T07:19:57.296 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:57.291+0000 7f9136f2a640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9130142540 con 0x7f91300cf7c0 2023-11-26T07:19:57.296 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:57.291+0000 7f912ffff640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f913013e290 0x7f91301418f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:19:57.296 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:57.291+0000 7f912ffff640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f913013e290 0x7f91301418f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:35108/0 (socket says 172.21.15.143:35108) 2023-11-26T07:19:57.296 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:57.291+0000 7f912ffff640 1 -- 172.21.15.143:0/2194010625 learned_addr learned my addr 172.21.15.143:0/2194010625 (peer_addr_for_me v2:172.21.15.143:0/0) 2023-11-26T07:19:57.296 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:57.291+0000 7f9134cc6640 1 --2- 172.21.15.143:0/2194010625 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f91300cf7c0 0x7f91301548a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:19:57.296 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:57.291+0000 7f91354c7640 1 --2- 172.21.15.143:0/2194010625 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f913015f4e0 0x7f913013ce40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:19:57.296 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:57.291+0000 7f912ffff640 1 -- 172.21.15.143:0/2194010625 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f91300cf7c0 msgr2=0x7f91301548a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:19:57.296 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:57.291+0000 7f912ffff640 1 --2- 172.21.15.143:0/2194010625 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f91300cf7c0 0x7f91301548a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:19:57.296 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:57.291+0000 7f912ffff640 1 -- 172.21.15.143:0/2194010625 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f913015f4e0 msgr2=0x7f913013ce40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:19:57.296 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:57.291+0000 7f912ffff640 1 --2- 172.21.15.143:0/2194010625 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f913015f4e0 0x7f913013ce40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:19:57.296 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:57.291+0000 7f912ffff640 1 -- 172.21.15.143:0/2194010625 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f911c0094d0 con 0x7f913013e290 2023-11-26T07:19:57.297 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:57.291+0000 7f91354c7640 1 --2- 172.21.15.143:0/2194010625 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f913015f4e0 0x7f913013ce40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-26T07:19:57.297 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:57.291+0000 7f912ffff640 1 --2- 172.21.15.143:0/2194010625 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f913013e290 0x7f91301418f0 secure :-1 s=READY pgs=646 cs=0 l=1 rev1=1 crypto rx=0x7f911c01a140 tx=0x7f911c01ad50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:19:57.297 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:57.291+0000 7f912d7fa640 1 -- 172.21.15.143:0/2194010625 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f911c0b3800 con 0x7f913013e290 2023-11-26T07:19:57.297 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:57.291+0000 7f912d7fa640 1 -- 172.21.15.143:0/2194010625 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f911c0b3960 con 0x7f913013e290 2023-11-26T07:19:57.297 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:57.291+0000 7f912d7fa640 1 -- 172.21.15.143:0/2194010625 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f911c0b3d10 con 0x7f913013e290 2023-11-26T07:19:57.298 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:57.291+0000 7f9136f2a640 1 -- 172.21.15.143:0/2194010625 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9130142820 con 0x7f913013e290 2023-11-26T07:19:57.298 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:57.291+0000 7f9136f2a640 1 -- 172.21.15.143:0/2194010625 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9130154de0 con 0x7f913013e290 2023-11-26T07:19:57.298 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:57.295+0000 7f9136f2a640 1 -- 172.21.15.143:0/2194010625 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9130143020 con 0x7f913013e290 2023-11-26T07:19:57.298 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:57.295+0000 7f912d7fa640 1 -- 172.21.15.143:0/2194010625 <== mon.0 v2:172.21.15.143:3300/0 4 ==== mgrmap(e 7) v1 ==== 86005+0+0 (secure 0 0 0) 0x7f911c0b2380 con 0x7f913013e290 2023-11-26T07:19:57.299 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:57.295+0000 7f912d7fa640 1 --2- 172.21.15.143:0/2194010625 >> [v2:172.21.15.143:6832/2150557544,v1:172.21.15.143:6833/2150557544] conn(0x7f9114066620 0x7f9114068ae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:19:57.299 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:57.295+0000 7f912d7fa640 1 -- 172.21.15.143:0/2194010625 <== mon.0 v2:172.21.15.143:3300/0 5 ==== osd_map(45..45 src has 1..45) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f911c11eb10 con 0x7f913013e290 2023-11-26T07:19:57.299 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:57.295+0000 7f9134cc6640 1 --2- 172.21.15.143:0/2194010625 >> [v2:172.21.15.143:6832/2150557544,v1:172.21.15.143:6833/2150557544] conn(0x7f9114066620 0x7f9114068ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:19:57.300 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:57.295+0000 7f9134cc6640 1 --2- 172.21.15.143:0/2194010625 >> [v2:172.21.15.143:6832/2150557544,v1:172.21.15.143:6833/2150557544] conn(0x7f9114066620 0x7f9114068ae0 secure :-1 s=READY pgs=315 cs=0 l=1 rev1=1 crypto rx=0x7f912400b9d0 tx=0x7f9124008040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:19:57.303 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:57.299+0000 7f912d7fa640 1 -- 172.21.15.143:0/2194010625 <== mon.0 v2:172.21.15.143:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172649 (secure 0 0 0) 0x7f9130143020 con 0x7f913013e290 2023-11-26T07:19:57.452 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:57.451+0000 7f9136f2a640 1 -- 172.21.15.143:0/2194010625 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f9130143020 con 0x7f913013e290 2023-11-26T07:19:57.453 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:57.451+0000 7f912d7fa640 1 -- 172.21.15.143:0/2194010625 <== mon.0 v2:172.21.15.143:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v45) v1 ==== 93+0+10 (secure 0 0 0) 0x7f9130143020 con 0x7f913013e290 2023-11-26T07:19:57.453 INFO:teuthology.orchestra.run.smithi143.stdout:pg_num: 1 2023-11-26T07:19:57.455 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:57.451+0000 7f9136f2a640 1 -- 172.21.15.143:0/2194010625 >> [v2:172.21.15.143:6832/2150557544,v1:172.21.15.143:6833/2150557544] conn(0x7f9114066620 msgr2=0x7f9114068ae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:19:57.455 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:57.451+0000 7f9136f2a640 1 --2- 172.21.15.143:0/2194010625 >> [v2:172.21.15.143:6832/2150557544,v1:172.21.15.143:6833/2150557544] conn(0x7f9114066620 0x7f9114068ae0 secure :-1 s=READY pgs=315 cs=0 l=1 rev1=1 crypto rx=0x7f912400b9d0 tx=0x7f9124008040 comp rx=0 tx=0).stop 2023-11-26T07:19:57.455 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:57.451+0000 7f9136f2a640 1 -- 172.21.15.143:0/2194010625 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f913013e290 msgr2=0x7f91301418f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:19:57.456 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:57.451+0000 7f9136f2a640 1 --2- 172.21.15.143:0/2194010625 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f913013e290 0x7f91301418f0 secure :-1 s=READY pgs=646 cs=0 l=1 rev1=1 crypto rx=0x7f911c01a140 tx=0x7f911c01ad50 comp rx=0 tx=0).stop 2023-11-26T07:19:57.456 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:57.451+0000 7f9136f2a640 1 -- 172.21.15.143:0/2194010625 shutdown_connections 2023-11-26T07:19:57.456 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:57.451+0000 7f9136f2a640 1 --2- 172.21.15.143:0/2194010625 >> [v2:172.21.15.143:6832/2150557544,v1:172.21.15.143:6833/2150557544] conn(0x7f9114066620 0x7f9114068ae0 unknown :-1 s=CLOSED pgs=315 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:19:57.456 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:57.451+0000 7f9136f2a640 1 --2- 172.21.15.143:0/2194010625 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f913015f4e0 0x7f913013ce40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:19:57.456 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:57.451+0000 7f9136f2a640 1 --2- 172.21.15.143:0/2194010625 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f913013e290 0x7f91301418f0 unknown :-1 s=CLOSED pgs=646 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:19:57.456 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:57.451+0000 7f9136f2a640 1 --2- 172.21.15.143:0/2194010625 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f91300cf7c0 0x7f91301548a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:19:57.456 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:57.451+0000 7f9136f2a640 1 -- 172.21.15.143:0/2194010625 >> 172.21.15.143:0/2194010625 conn(0x7f913006a890 msgr2=0x7f9130071a80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:19:57.456 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:57.451+0000 7f9136f2a640 1 -- 172.21.15.143:0/2194010625 shutdown_connections 2023-11-26T07:19:57.456 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:57.451+0000 7f9136f2a640 1 -- 172.21.15.143:0/2194010625 wait complete. 2023-11-26T07:19:57.474 DEBUG:teuthology.orchestra.run.smithi143:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-11-26T07:19:57.614 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:57.611+0000 7f500ae32640 1 Processor -- start 2023-11-26T07:19:57.615 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:57.611+0000 7f500ae32640 1 -- start start 2023-11-26T07:19:57.616 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:57.611+0000 7f500ae32640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f5004070ba0 0x7f500406bae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:19:57.616 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:57.611+0000 7f500ae32640 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7f5004161cf0 0x7f5004070f80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:19:57.616 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:57.611+0000 7f500ae32640 1 -- --> v1:172.21.15.143:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f50040714c0 con 0x7f50040cdd10 2023-11-26T07:19:57.616 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:57.611+0000 7f500ae32640 1 -- --> v2:172.21.15.188:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f50040cd610 con 0x7f5004161cf0 2023-11-26T07:19:57.616 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:57.611+0000 7f500ae32640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f50040cd8f0 con 0x7f5004070ba0 2023-11-26T07:19:57.616 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:57.611+0000 7f50093cf640 1 --1- >> v1:172.21.15.143:6789/0 conn(0x7f50040cdd10 0x7f500406a490 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.143:6789/0 says I am v1:172.21.15.143:47556/0 (socket says 172.21.15.143:47556) 2023-11-26T07:19:57.616 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:57.611+0000 7f50093cf640 1 -- 172.21.15.143:0/2031292076 learned_addr learned my addr 172.21.15.143:0/2031292076 (peer_addr_for_me v1:172.21.15.143:0/0) 2023-11-26T07:19:57.618 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:57.611+0000 7f4ffbfff640 1 --2- 172.21.15.143:0/2031292076 >> v2:172.21.15.188:3300/0 conn(0x7f5004161cf0 0x7f5004070f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:19:57.618 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:57.611+0000 7f5008bce640 1 --2- 172.21.15.143:0/2031292076 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f5004070ba0 0x7f500406bae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:19:57.618 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:57.615+0000 7f4ffb7fe640 1 -- 172.21.15.143:0/2031292076 <== mon.0 v1:172.21.15.143:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2488410698 0 0) 0x7f50040714c0 con 0x7f50040cdd10 2023-11-26T07:19:57.618 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:57.615+0000 7f4ffb7fe640 1 -- 172.21.15.143:0/2031292076 --> v1:172.21.15.143:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f4fec003530 con 0x7f50040cdd10 2023-11-26T07:19:57.618 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:57.615+0000 7f4ffb7fe640 1 -- 172.21.15.143:0/2031292076 <== mon.0 v1:172.21.15.143:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1889853349 0 0) 0x7f4fec003530 con 0x7f50040cdd10 2023-11-26T07:19:57.619 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:57.615+0000 7f4ffb7fe640 1 -- 172.21.15.143:0/2031292076 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f5004070ba0 msgr2=0x7f500406bae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-26T07:19:57.619 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:57.615+0000 7f4ffb7fe640 1 --2- 172.21.15.143:0/2031292076 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f5004070ba0 0x7f500406bae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:19:57.619 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:57.615+0000 7f4ffb7fe640 1 -- 172.21.15.143:0/2031292076 >> v2:172.21.15.188:3300/0 conn(0x7f5004161cf0 msgr2=0x7f5004070f80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-26T07:19:57.619 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:57.615+0000 7f4ffb7fe640 1 --2- 172.21.15.143:0/2031292076 >> v2:172.21.15.188:3300/0 conn(0x7f5004161cf0 0x7f5004070f80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:19:57.619 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:57.615+0000 7f4ffb7fe640 1 -- 172.21.15.143:0/2031292076 --> v1:172.21.15.143:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5004161210 con 0x7f50040cdd10 2023-11-26T07:19:57.619 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:57.615+0000 7f4ffb7fe640 1 -- 172.21.15.143:0/2031292076 <== mon.0 v1:172.21.15.143:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2137882523 0 0) 0x7f4ffc001a00 con 0x7f50040cdd10 2023-11-26T07:19:57.619 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:57.615+0000 7f4ffbfff640 1 --2- 172.21.15.143:0/2031292076 >> v2:172.21.15.188:3300/0 conn(0x7f5004161cf0 0x7f5004070f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-26T07:19:57.619 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:57.615+0000 7f4ffb7fe640 1 -- 172.21.15.143:0/2031292076 >> v1:172.21.15.143:6789/0 conn(0x7f50040cdd10 legacy=0x7f500406a490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:19:57.619 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:57.615+0000 7f4ffb7fe640 1 --2- 172.21.15.143:0/2031292076 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f4fec003980 0x7f4fec003da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:19:57.619 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:57.615+0000 7f4ffb7fe640 1 --2- 172.21.15.143:0/2031292076 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f4fec0047d0 0x7f4fec004c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:19:57.619 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:57.615+0000 7f4ffb7fe640 1 --2- 172.21.15.143:0/2031292076 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f5004070ba0 0x7f4fec005190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:19:57.619 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:57.615+0000 7f4ffb7fe640 1 -- 172.21.15.143:0/2031292076 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f50040cd610 con 0x7f4fec0047d0 2023-11-26T07:19:57.619 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:57.615+0000 7f4ffb7fe640 1 -- 172.21.15.143:0/2031292076 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f50040cd8f0 con 0x7f4fec003980 2023-11-26T07:19:57.619 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:57.615+0000 7f4ffb7fe640 1 -- 172.21.15.143:0/2031292076 --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4fec00b990 con 0x7f5004070ba0 2023-11-26T07:19:57.620 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:57.615+0000 7f4ffbfff640 1 --2- 172.21.15.143:0/2031292076 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f4fec0047d0 0x7f4fec004c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:19:57.620 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:57.615+0000 7f5008bce640 1 --2- 172.21.15.143:0/2031292076 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f4fec003980 0x7f4fec003da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:19:57.620 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:57.615+0000 7f50093cf640 1 --2- 172.21.15.143:0/2031292076 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f5004070ba0 0x7f4fec005190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:19:57.620 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:57.615+0000 7f4ffbfff640 1 -- 172.21.15.143:0/2031292076 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f5004070ba0 msgr2=0x7f4fec005190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-26T07:19:57.620 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:57.615+0000 7f4ffbfff640 1 --2- 172.21.15.143:0/2031292076 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f5004070ba0 0x7f4fec005190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:19:57.620 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:57.615+0000 7f4ffbfff640 1 -- 172.21.15.143:0/2031292076 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f4fec003980 msgr2=0x7f4fec003da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-26T07:19:57.620 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:57.615+0000 7f4ffbfff640 1 --2- 172.21.15.143:0/2031292076 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f4fec003980 0x7f4fec003da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:19:57.620 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:57.615+0000 7f4ffbfff640 1 -- 172.21.15.143:0/2031292076 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4fec003530 con 0x7f4fec0047d0 2023-11-26T07:19:57.621 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:57.619+0000 7f4ffbfff640 1 --2- 172.21.15.143:0/2031292076 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f4fec0047d0 0x7f4fec004c50 secure :-1 s=READY pgs=648 cs=0 l=1 rev1=1 crypto rx=0x7f4ff4009cb0 tx=0x7f4ff401cc10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9934c28148cae6a1 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:19:57.621 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:57.619+0000 7f4ffb7fe640 1 -- 172.21.15.143:0/2031292076 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4ff40a5740 con 0x7f4fec0047d0 2023-11-26T07:19:57.621 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:57.619+0000 7f4ffb7fe640 1 -- 172.21.15.143:0/2031292076 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4ff40a58a0 con 0x7f4fec0047d0 2023-11-26T07:19:57.621 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:57.619+0000 7f4ffb7fe640 1 -- 172.21.15.143:0/2031292076 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4ff40a5c50 con 0x7f4fec0047d0 2023-11-26T07:19:57.622 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:57.619+0000 7f500ae32640 1 -- 172.21.15.143:0/2031292076 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f4fec0047d0 msgr2=0x7f4fec004c50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:19:57.622 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:57.619+0000 7f500ae32640 1 --2- 172.21.15.143:0/2031292076 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f4fec0047d0 0x7f4fec004c50 secure :-1 s=READY pgs=648 cs=0 l=1 rev1=1 crypto rx=0x7f4ff4009cb0 tx=0x7f4ff401cc10 comp rx=0 tx=0).stop 2023-11-26T07:19:57.622 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:57.619+0000 7f50093cf640 1 -- 172.21.15.143:0/2031292076 reap_dead start 2023-11-26T07:19:57.622 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:57.619+0000 7f500ae32640 1 -- 172.21.15.143:0/2031292076 shutdown_connections 2023-11-26T07:19:57.622 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:57.619+0000 7f500ae32640 1 -- 172.21.15.143:0/2031292076 >> 172.21.15.143:0/2031292076 conn(0x7f50040cf710 msgr2=0x7f50040cfb20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:19:57.622 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:57.619+0000 7f500ae32640 1 -- 172.21.15.143:0/2031292076 shutdown_connections 2023-11-26T07:19:57.622 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:57.619+0000 7f500ae32640 1 -- 172.21.15.143:0/2031292076 wait complete. 2023-11-26T07:19:57.623 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:57.619+0000 7f500ae32640 1 Processor -- start 2023-11-26T07:19:57.623 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:57.619+0000 7f500ae32640 1 -- start start 2023-11-26T07:19:57.625 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:57.619+0000 7f500ae32640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f4fec0047d0 0x7f50041450b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:19:57.625 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:57.619+0000 7f500ae32640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f5004144b00 0x7f5004160500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:19:57.625 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:57.619+0000 7f500ae32640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f5004159730 0x7f50041470e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:19:57.625 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:57.623+0000 7f500ae32640 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5004159ca0 con 0x7f4fec0047d0 2023-11-26T07:19:57.625 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:57.623+0000 7f500ae32640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5004160a40 con 0x7f5004159730 2023-11-26T07:19:57.625 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:57.623+0000 7f500ae32640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f500415b0c0 con 0x7f5004144b00 2023-11-26T07:19:57.625 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:57.623+0000 7f5008bce640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f4fec0047d0 0x7f50041450b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:19:57.625 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:57.623+0000 7f5008bce640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f4fec0047d0 0x7f50041450b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:35120/0 (socket says 172.21.15.143:35120) 2023-11-26T07:19:57.625 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:57.623+0000 7f5008bce640 1 -- 172.21.15.143:0/816210675 learned_addr learned my addr 172.21.15.143:0/816210675 (peer_addr_for_me v2:172.21.15.143:0/0) 2023-11-26T07:19:57.626 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:57.623+0000 7f4ffbfff640 1 --2- 172.21.15.143:0/816210675 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f5004144b00 0x7f5004160500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:19:57.626 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:57.623+0000 7f50093cf640 1 --2- 172.21.15.143:0/816210675 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f5004159730 0x7f50041470e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:19:57.626 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:57.623+0000 7f5008bce640 1 -- 172.21.15.143:0/816210675 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f5004144b00 msgr2=0x7f5004160500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:19:57.626 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:57.623+0000 7f5008bce640 1 --2- 172.21.15.143:0/816210675 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f5004144b00 0x7f5004160500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:19:57.626 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:57.623+0000 7f5008bce640 1 -- 172.21.15.143:0/816210675 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f5004159730 msgr2=0x7f50041470e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:19:57.626 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:57.623+0000 7f5008bce640 1 --2- 172.21.15.143:0/816210675 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f5004159730 0x7f50041470e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:19:57.626 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:57.623+0000 7f5008bce640 1 -- 172.21.15.143:0/816210675 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5004155430 con 0x7f4fec0047d0 2023-11-26T07:19:57.626 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:57.623+0000 7f4ffbfff640 1 --2- 172.21.15.143:0/816210675 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f5004144b00 0x7f5004160500 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-26T07:19:57.626 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:57.623+0000 7f5008bce640 1 --2- 172.21.15.143:0/816210675 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f4fec0047d0 0x7f50041450b0 secure :-1 s=READY pgs=649 cs=0 l=1 rev1=1 crypto rx=0x7f5000011960 tx=0x7f5000011e20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:19:57.627 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:57.623+0000 7f4ff97fa640 1 -- 172.21.15.143:0/816210675 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f500000faf0 con 0x7f4fec0047d0 2023-11-26T07:19:57.627 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:57.623+0000 7f4ff97fa640 1 -- 172.21.15.143:0/816210675 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f500000fc50 con 0x7f4fec0047d0 2023-11-26T07:19:57.627 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:57.623+0000 7f500ae32640 1 -- 172.21.15.143:0/816210675 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f500415b340 con 0x7f4fec0047d0 2023-11-26T07:19:57.627 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:57.623+0000 7f4ff97fa640 1 -- 172.21.15.143:0/816210675 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5000014440 con 0x7f4fec0047d0 2023-11-26T07:19:57.627 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:57.623+0000 7f500ae32640 1 -- 172.21.15.143:0/816210675 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f500415b900 con 0x7f4fec0047d0 2023-11-26T07:19:57.627 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:57.623+0000 7f500ae32640 1 -- 172.21.15.143:0/816210675 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5004067680 con 0x7f4fec0047d0 2023-11-26T07:19:57.628 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:57.623+0000 7f4ff97fa640 1 -- 172.21.15.143:0/816210675 <== mon.0 v2:172.21.15.143:3300/0 4 ==== mgrmap(e 7) v1 ==== 86005+0+0 (secure 0 0 0) 0x7f500001d400 con 0x7f4fec0047d0 2023-11-26T07:19:57.628 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:57.627+0000 7f4ff97fa640 1 --2- 172.21.15.143:0/816210675 >> [v2:172.21.15.143:6832/2150557544,v1:172.21.15.143:6833/2150557544] conn(0x7f4ff0066620 0x7f4ff0068ae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:19:57.628 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:57.627+0000 7f4ff97fa640 1 -- 172.21.15.143:0/816210675 <== mon.0 v2:172.21.15.143:3300/0 5 ==== osd_map(45..45 src has 1..45) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f5000084980 con 0x7f4fec0047d0 2023-11-26T07:19:57.629 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:57.627+0000 7f4ffbfff640 1 --2- 172.21.15.143:0/816210675 >> [v2:172.21.15.143:6832/2150557544,v1:172.21.15.143:6833/2150557544] conn(0x7f4ff0066620 0x7f4ff0068ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:19:57.629 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:57.627+0000 7f4ffbfff640 1 --2- 172.21.15.143:0/816210675 >> [v2:172.21.15.143:6832/2150557544,v1:172.21.15.143:6833/2150557544] conn(0x7f4ff0066620 0x7f4ff0068ae0 secure :-1 s=READY pgs=316 cs=0 l=1 rev1=1 crypto rx=0x7f4ff40af000 tx=0x7f4ff40046d0 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:19:57.632 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:57.627+0000 7f4ff97fa640 1 -- 172.21.15.143:0/816210675 <== mon.0 v2:172.21.15.143:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172649 (secure 0 0 0) 0x7f5000051b70 con 0x7f4fec0047d0 2023-11-26T07:19:57.783 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:57.779+0000 7f500ae32640 1 -- 172.21.15.143:0/816210675 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f50040ce260 con 0x7f4fec0047d0 2023-11-26T07:19:57.783 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:57.779+0000 7f4ff97fa640 1 -- 172.21.15.143:0/816210675 <== mon.0 v2:172.21.15.143:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v45) v1 ==== 104+0+11 (secure 0 0 0) 0x7f5000055190 con 0x7f4fec0047d0 2023-11-26T07:19:57.783 INFO:teuthology.orchestra.run.smithi143.stdout:pg_num: 32 2023-11-26T07:19:57.785 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:57.783+0000 7f500ae32640 1 -- 172.21.15.143:0/816210675 >> [v2:172.21.15.143:6832/2150557544,v1:172.21.15.143:6833/2150557544] conn(0x7f4ff0066620 msgr2=0x7f4ff0068ae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:19:57.785 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:57.783+0000 7f500ae32640 1 --2- 172.21.15.143:0/816210675 >> [v2:172.21.15.143:6832/2150557544,v1:172.21.15.143:6833/2150557544] conn(0x7f4ff0066620 0x7f4ff0068ae0 secure :-1 s=READY pgs=316 cs=0 l=1 rev1=1 crypto rx=0x7f4ff40af000 tx=0x7f4ff40046d0 comp rx=0 tx=0).stop 2023-11-26T07:19:57.785 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:57.783+0000 7f500ae32640 1 -- 172.21.15.143:0/816210675 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f4fec0047d0 msgr2=0x7f50041450b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:19:57.786 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:57.783+0000 7f500ae32640 1 --2- 172.21.15.143:0/816210675 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f4fec0047d0 0x7f50041450b0 secure :-1 s=READY pgs=649 cs=0 l=1 rev1=1 crypto rx=0x7f5000011960 tx=0x7f5000011e20 comp rx=0 tx=0).stop 2023-11-26T07:19:57.786 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:57.783+0000 7f500ae32640 1 -- 172.21.15.143:0/816210675 shutdown_connections 2023-11-26T07:19:57.786 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:57.783+0000 7f500ae32640 1 --2- 172.21.15.143:0/816210675 >> [v2:172.21.15.143:6832/2150557544,v1:172.21.15.143:6833/2150557544] conn(0x7f4ff0066620 0x7f4ff0068ae0 unknown :-1 s=CLOSED pgs=316 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:19:57.786 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:57.783+0000 7f500ae32640 1 --2- 172.21.15.143:0/816210675 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f5004159730 0x7f50041470e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:19:57.786 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:57.783+0000 7f500ae32640 1 --2- 172.21.15.143:0/816210675 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f5004144b00 0x7f5004160500 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:19:57.786 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:57.783+0000 7f500ae32640 1 --2- 172.21.15.143:0/816210675 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f4fec0047d0 0x7f50041450b0 unknown :-1 s=CLOSED pgs=649 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:19:57.786 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:57.783+0000 7f500ae32640 1 -- 172.21.15.143:0/816210675 >> 172.21.15.143:0/816210675 conn(0x7f50040cf710 msgr2=0x7f50040b0b00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:19:57.786 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:57.783+0000 7f500ae32640 1 -- 172.21.15.143:0/816210675 shutdown_connections 2023-11-26T07:19:57.786 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:57.783+0000 7f500ae32640 1 -- 172.21.15.143:0/816210675 wait complete. 2023-11-26T07:19:57.804 DEBUG:teuthology.orchestra.run.smithi143:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-11-26T07:19:57.944 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:57.939+0000 7f5c5bdd1640 1 Processor -- start 2023-11-26T07:19:57.944 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:57.943+0000 7f5c5bdd1640 1 -- start start 2023-11-26T07:19:57.946 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:57.943+0000 7f5c5bdd1640 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7f5c5413e290 0x7f5c540cf280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:19:57.946 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:57.943+0000 7f5c5bdd1640 1 -- --> v1:172.21.15.143:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5c5415fb20 con 0x7f5c5415f6b0 2023-11-26T07:19:57.946 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:57.943+0000 7f5c5bdd1640 1 -- --> v1:172.21.15.188:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5c5406c520 con 0x7f5c540cf7c0 2023-11-26T07:19:57.946 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:57.943+0000 7f5c5bdd1640 1 -- --> v2:172.21.15.143:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5c540cfc40 con 0x7f5c5413e290 2023-11-26T07:19:57.946 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:57.943+0000 7f5c59b6d640 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7f5c5413e290 0x7f5c540cf280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:19:57.946 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:57.943+0000 7f5c59b6d640 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7f5c5413e290 0x7f5c540cf280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:35124/0 (socket says 172.21.15.143:35124) 2023-11-26T07:19:57.946 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:57.943+0000 7f5c59b6d640 1 -- 172.21.15.143:0/3271196325 learned_addr learned my addr 172.21.15.143:0/3271196325 (peer_addr_for_me v2:172.21.15.143:0/0) 2023-11-26T07:19:57.946 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:57.943+0000 7f5c5a36e640 1 --1- >> v1:172.21.15.143:6789/0 conn(0x7f5c5415f6b0 0x7f5c540cd7b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.143:6789/0 says I am v1:172.21.15.143:47570/0 (socket says 172.21.15.143:47570) 2023-11-26T07:19:57.947 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:57.943+0000 7f5c59b6d640 1 -- 172.21.15.143:0/3271196325 >> v1:172.21.15.188:6789/0 conn(0x7f5c540cf7c0 legacy=0x7f5c5415efa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-26T07:19:57.947 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:57.943+0000 7f5c58b6b640 1 -- 172.21.15.143:0/3271196325 <== mon.0 v1:172.21.15.143:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3513689258 0 0) 0x7f5c4c0014a0 con 0x7f5c5415f6b0 2023-11-26T07:19:57.947 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:57.943+0000 7f5c59b6d640 1 -- 172.21.15.143:0/3271196325 >> v1:172.21.15.143:6789/0 conn(0x7f5c5415f6b0 legacy=0x7f5c540cd7b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:19:57.947 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:57.943+0000 7f5c59b6d640 1 -- 172.21.15.143:0/3271196325 --> v2:172.21.15.143:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5c5406c800 con 0x7f5c5413e290 2023-11-26T07:19:57.949 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:57.947+0000 7f5c59b6d640 1 --2- 172.21.15.143:0/3271196325 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f5c5413e290 0x7f5c540cf280 secure :-1 s=READY pgs=651 cs=0 l=1 rev1=1 crypto rx=0x7f5c50009870 tx=0x7f5c5001c4d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=715b3abd736fafcc server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:19:57.949 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:57.947+0000 7f5c58b6b640 1 -- 172.21.15.143:0/3271196325 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5c500aa430 con 0x7f5c5413e290 2023-11-26T07:19:57.949 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:57.947+0000 7f5c58b6b640 1 -- 172.21.15.143:0/3271196325 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5c500aa590 con 0x7f5c5413e290 2023-11-26T07:19:57.950 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:57.947+0000 7f5c58b6b640 1 -- 172.21.15.143:0/3271196325 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5c500aa920 con 0x7f5c5413e290 2023-11-26T07:19:57.950 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:57.947+0000 7f5c5bdd1640 1 -- 172.21.15.143:0/3271196325 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f5c5413e290 msgr2=0x7f5c540cf280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:19:57.950 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:57.947+0000 7f5c5bdd1640 1 --2- 172.21.15.143:0/3271196325 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f5c5413e290 0x7f5c540cf280 secure :-1 s=READY pgs=651 cs=0 l=1 rev1=1 crypto rx=0x7f5c50009870 tx=0x7f5c5001c4d0 comp rx=0 tx=0).stop 2023-11-26T07:19:57.951 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:57.947+0000 7f5c5bdd1640 1 -- 172.21.15.143:0/3271196325 shutdown_connections 2023-11-26T07:19:57.951 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:57.947+0000 7f5c5bdd1640 1 --2- 172.21.15.143:0/3271196325 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f5c5413e290 0x7f5c540cf280 unknown :-1 s=CLOSED pgs=651 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:19:57.951 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:57.947+0000 7f5c5bdd1640 1 -- 172.21.15.143:0/3271196325 >> 172.21.15.143:0/3271196325 conn(0x7f5c5406a890 msgr2=0x7f5c5406aca0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:19:57.951 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:57.947+0000 7f5c5bdd1640 1 -- 172.21.15.143:0/3271196325 shutdown_connections 2023-11-26T07:19:57.951 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:57.947+0000 7f5c5bdd1640 1 -- 172.21.15.143:0/3271196325 wait complete. 2023-11-26T07:19:57.952 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:57.947+0000 7f5c5bdd1640 1 Processor -- start 2023-11-26T07:19:57.952 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:57.947+0000 7f5c5bdd1640 1 -- start start 2023-11-26T07:19:57.953 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:57.951+0000 7f5c5bdd1640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f5c540cf7c0 0x7f5c54154410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:19:57.953 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:57.951+0000 7f5c5bdd1640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f5c5413e290 0x7f5c54070790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:19:57.953 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:57.951+0000 7f5c5bdd1640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f5c5415f6b0 0x7f5c5415e220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:19:57.954 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:57.951+0000 7f5c5bdd1640 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5c5415e760 con 0x7f5c540cf7c0 2023-11-26T07:19:57.954 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:57.951+0000 7f5c5bdd1640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5c5413c590 con 0x7f5c5415f6b0 2023-11-26T07:19:57.954 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:57.951+0000 7f5c5bdd1640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5c5413c870 con 0x7f5c5413e290 2023-11-26T07:19:57.954 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:57.951+0000 7f5c59b6d640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f5c540cf7c0 0x7f5c54154410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:19:57.954 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:57.951+0000 7f5c59b6d640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f5c540cf7c0 0x7f5c54154410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:35140/0 (socket says 172.21.15.143:35140) 2023-11-26T07:19:57.954 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:57.951+0000 7f5c59b6d640 1 -- 172.21.15.143:0/3968587710 learned_addr learned my addr 172.21.15.143:0/3968587710 (peer_addr_for_me v2:172.21.15.143:0/0) 2023-11-26T07:19:57.954 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:57.951+0000 7f5c59b6d640 1 -- 172.21.15.143:0/3968587710 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f5c5413e290 msgr2=0x7f5c54070790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:19:57.954 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:57.951+0000 7f5c59b6d640 1 --2- 172.21.15.143:0/3968587710 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f5c5413e290 0x7f5c54070790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:19:57.954 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:57.951+0000 7f5c59b6d640 1 -- 172.21.15.143:0/3968587710 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f5c5415f6b0 msgr2=0x7f5c5415e220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:19:57.954 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:57.951+0000 7f5c59b6d640 1 --2- 172.21.15.143:0/3968587710 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f5c5415f6b0 0x7f5c5415e220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:19:57.954 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:57.951+0000 7f5c59b6d640 1 -- 172.21.15.143:0/3968587710 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5c500094d0 con 0x7f5c540cf7c0 2023-11-26T07:19:57.955 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:57.951+0000 7f5c59b6d640 1 --2- 172.21.15.143:0/3968587710 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f5c540cf7c0 0x7f5c54154410 secure :-1 s=READY pgs=652 cs=0 l=1 rev1=1 crypto rx=0x7f5c500ac040 tx=0x7f5c5001ad10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:19:57.955 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:57.951+0000 7f5c4a7fc640 1 -- 172.21.15.143:0/3968587710 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5c500b38c0 con 0x7f5c540cf7c0 2023-11-26T07:19:57.955 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:57.951+0000 7f5c4a7fc640 1 -- 172.21.15.143:0/3968587710 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5c500b3a40 con 0x7f5c540cf7c0 2023-11-26T07:19:57.955 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:57.951+0000 7f5c5bdd1640 1 -- 172.21.15.143:0/3968587710 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5c5413cb50 con 0x7f5c540cf7c0 2023-11-26T07:19:57.955 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:57.951+0000 7f5c4a7fc640 1 -- 172.21.15.143:0/3968587710 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5c500b3df0 con 0x7f5c540cf7c0 2023-11-26T07:19:57.956 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:57.951+0000 7f5c5bdd1640 1 -- 172.21.15.143:0/3968587710 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5c5413ce30 con 0x7f5c540cf7c0 2023-11-26T07:19:57.956 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:57.951+0000 7f5c5bdd1640 1 -- 172.21.15.143:0/3968587710 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5c5415ca40 con 0x7f5c540cf7c0 2023-11-26T07:19:57.956 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:57.951+0000 7f5c4a7fc640 1 -- 172.21.15.143:0/3968587710 <== mon.0 v2:172.21.15.143:3300/0 4 ==== mgrmap(e 7) v1 ==== 86005+0+0 (secure 0 0 0) 0x7f5c500c6390 con 0x7f5c540cf7c0 2023-11-26T07:19:57.957 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:57.955+0000 7f5c4a7fc640 1 --2- 172.21.15.143:0/3968587710 >> [v2:172.21.15.143:6832/2150557544,v1:172.21.15.143:6833/2150557544] conn(0x7f5c38066620 0x7f5c38068ae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:19:57.957 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:57.955+0000 7f5c4a7fc640 1 -- 172.21.15.143:0/3968587710 <== mon.0 v2:172.21.15.143:3300/0 5 ==== osd_map(45..45 src has 1..45) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f5c5011ee20 con 0x7f5c540cf7c0 2023-11-26T07:19:57.957 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:57.955+0000 7f5c5936c640 1 --2- 172.21.15.143:0/3968587710 >> [v2:172.21.15.143:6832/2150557544,v1:172.21.15.143:6833/2150557544] conn(0x7f5c38066620 0x7f5c38068ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:19:57.958 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:57.955+0000 7f5c5936c640 1 --2- 172.21.15.143:0/3968587710 >> [v2:172.21.15.143:6832/2150557544,v1:172.21.15.143:6833/2150557544] conn(0x7f5c38066620 0x7f5c38068ae0 secure :-1 s=READY pgs=317 cs=0 l=1 rev1=1 crypto rx=0x7f5c44003cf0 tx=0x7f5c440079c0 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:19:57.961 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:57.959+0000 7f5c4a7fc640 1 -- 172.21.15.143:0/3968587710 <== mon.0 v2:172.21.15.143:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172649 (secure 0 0 0) 0x7f5c5415ca40 con 0x7f5c540cf7c0 2023-11-26T07:19:58.108 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:58.107+0000 7f5c5bdd1640 1 -- 172.21.15.143:0/3968587710 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f5c5415ca40 con 0x7f5c540cf7c0 2023-11-26T07:19:58.109 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:58.107+0000 7f5c4a7fc640 1 -- 172.21.15.143:0/3968587710 <== mon.0 v2:172.21.15.143:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v45) v1 ==== 100+0+11 (secure 0 0 0) 0x7f5c5415ca40 con 0x7f5c540cf7c0 2023-11-26T07:19:58.109 INFO:teuthology.orchestra.run.smithi143.stdout:pg_num: 64 2023-11-26T07:19:58.111 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:58.107+0000 7f5c5bdd1640 1 -- 172.21.15.143:0/3968587710 >> [v2:172.21.15.143:6832/2150557544,v1:172.21.15.143:6833/2150557544] conn(0x7f5c38066620 msgr2=0x7f5c38068ae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:19:58.111 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:58.107+0000 7f5c5bdd1640 1 --2- 172.21.15.143:0/3968587710 >> [v2:172.21.15.143:6832/2150557544,v1:172.21.15.143:6833/2150557544] conn(0x7f5c38066620 0x7f5c38068ae0 secure :-1 s=READY pgs=317 cs=0 l=1 rev1=1 crypto rx=0x7f5c44003cf0 tx=0x7f5c440079c0 comp rx=0 tx=0).stop 2023-11-26T07:19:58.111 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:58.107+0000 7f5c5bdd1640 1 -- 172.21.15.143:0/3968587710 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f5c540cf7c0 msgr2=0x7f5c54154410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:19:58.111 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:58.107+0000 7f5c5bdd1640 1 --2- 172.21.15.143:0/3968587710 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f5c540cf7c0 0x7f5c54154410 secure :-1 s=READY pgs=652 cs=0 l=1 rev1=1 crypto rx=0x7f5c500ac040 tx=0x7f5c5001ad10 comp rx=0 tx=0).stop 2023-11-26T07:19:58.111 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:58.107+0000 7f5c5bdd1640 1 -- 172.21.15.143:0/3968587710 shutdown_connections 2023-11-26T07:19:58.111 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:58.107+0000 7f5c5bdd1640 1 --2- 172.21.15.143:0/3968587710 >> [v2:172.21.15.143:6832/2150557544,v1:172.21.15.143:6833/2150557544] conn(0x7f5c38066620 0x7f5c38068ae0 unknown :-1 s=CLOSED pgs=317 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:19:58.112 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:58.107+0000 7f5c5bdd1640 1 --2- 172.21.15.143:0/3968587710 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f5c5415f6b0 0x7f5c5415e220 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:19:58.112 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:58.107+0000 7f5c5bdd1640 1 --2- 172.21.15.143:0/3968587710 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f5c5413e290 0x7f5c54070790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:19:58.112 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:58.107+0000 7f5c5bdd1640 1 --2- 172.21.15.143:0/3968587710 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f5c540cf7c0 0x7f5c54154410 unknown :-1 s=CLOSED pgs=652 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:19:58.112 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:58.107+0000 7f5c5bdd1640 1 -- 172.21.15.143:0/3968587710 >> 172.21.15.143:0/3968587710 conn(0x7f5c5406a890 msgr2=0x7f5c540cd7b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:19:58.112 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:58.107+0000 7f5c5bdd1640 1 -- 172.21.15.143:0/3968587710 shutdown_connections 2023-11-26T07:19:58.112 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:58.107+0000 7f5c5bdd1640 1 -- 172.21.15.143:0/3968587710 wait complete. 2023-11-26T07:19:58.130 DEBUG:teuthology.orchestra.run.smithi143:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2023-11-26T07:19:58.270 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:58.267+0000 7f7056f01640 1 Processor -- start 2023-11-26T07:19:58.271 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:58.267+0000 7f7056f01640 1 -- start start 2023-11-26T07:19:58.272 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:58.267+0000 7f7056f01640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f7050143ac0 0x7f7050070370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:19:58.272 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:58.267+0000 7f7056f01640 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7f70500cf990 0x7f70500cd1b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:19:58.272 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:58.267+0000 7f7056f01640 1 -- --> v1:172.21.15.143:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f70500cc730 con 0x7f70500708b0 2023-11-26T07:19:58.272 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:58.267+0000 7f7056f01640 1 -- --> v2:172.21.15.188:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7050143ea0 con 0x7f70500cf990 2023-11-26T07:19:58.272 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:58.267+0000 7f7056f01640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7050070ee0 con 0x7f7050143ac0 2023-11-26T07:19:58.272 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:58.267+0000 7f7047fff640 1 --1- >> v1:172.21.15.143:6789/0 conn(0x7f70500708b0 0x7f70500cf280 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.143:6789/0 says I am v1:172.21.15.143:47574/0 (socket says 172.21.15.143:47574) 2023-11-26T07:19:58.272 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:58.267+0000 7f7047fff640 1 -- 172.21.15.143:0/1833270641 learned_addr learned my addr 172.21.15.143:0/1833270641 (peer_addr_for_me v1:172.21.15.143:0/0) 2023-11-26T07:19:58.274 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:58.267+0000 7f7054c9d640 1 --2- 172.21.15.143:0/1833270641 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f7050143ac0 0x7f7050070370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:19:58.274 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:58.267+0000 7f705549e640 1 --2- 172.21.15.143:0/1833270641 >> v2:172.21.15.188:3300/0 conn(0x7f70500cf990 0x7f70500cd1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:19:58.274 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:58.267+0000 7f70477fe640 1 -- 172.21.15.143:0/1833270641 <== mon.0 v1:172.21.15.143:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 407162822 0 0) 0x7f70500cc730 con 0x7f70500708b0 2023-11-26T07:19:58.274 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:58.271+0000 7f70477fe640 1 -- 172.21.15.143:0/1833270641 --> v1:172.21.15.143:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f7038005740 con 0x7f70500708b0 2023-11-26T07:19:58.274 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:58.271+0000 7f70477fe640 1 -- 172.21.15.143:0/1833270641 <== mon.0 v1:172.21.15.143:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2007569085 0 0) 0x7f7038005740 con 0x7f70500708b0 2023-11-26T07:19:58.274 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:58.271+0000 7f70477fe640 1 -- 172.21.15.143:0/1833270641 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f7050143ac0 msgr2=0x7f7050070370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-26T07:19:58.274 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:58.271+0000 7f70477fe640 1 --2- 172.21.15.143:0/1833270641 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f7050143ac0 0x7f7050070370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:19:58.275 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:58.271+0000 7f70477fe640 1 -- 172.21.15.143:0/1833270641 >> v2:172.21.15.188:3300/0 conn(0x7f70500cf990 msgr2=0x7f70500cd1b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-26T07:19:58.275 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:58.271+0000 7f70477fe640 1 --2- 172.21.15.143:0/1833270641 >> v2:172.21.15.188:3300/0 conn(0x7f70500cf990 0x7f70500cd1b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:19:58.275 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:58.271+0000 7f70477fe640 1 -- 172.21.15.143:0/1833270641 --> v1:172.21.15.143:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f70500711c0 con 0x7f70500708b0 2023-11-26T07:19:58.275 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:58.271+0000 7f70477fe640 1 -- 172.21.15.143:0/1833270641 <== mon.0 v1:172.21.15.143:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2137882523 0 0) 0x7f70400016a0 con 0x7f70500708b0 2023-11-26T07:19:58.275 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:58.271+0000 7f70477fe640 1 -- 172.21.15.143:0/1833270641 >> v1:172.21.15.143:6789/0 conn(0x7f70500708b0 legacy=0x7f70500cf280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:19:58.275 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:58.271+0000 7f70477fe640 1 --2- 172.21.15.143:0/1833270641 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f7038005b90 0x7f7038005fb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:19:58.275 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:58.271+0000 7f70477fe640 1 --2- 172.21.15.143:0/1833270641 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f70380069e0 0x7f7038006e60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:19:58.275 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:58.271+0000 7f70477fe640 1 --2- 172.21.15.143:0/1833270641 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f7050143ac0 0x7f70380073a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:19:58.275 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:58.271+0000 7f70477fe640 1 -- 172.21.15.143:0/1833270641 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7050070ee0 con 0x7f7038005b90 2023-11-26T07:19:58.275 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:58.271+0000 7f70477fe640 1 -- 172.21.15.143:0/1833270641 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f703800dba0 con 0x7f70380069e0 2023-11-26T07:19:58.275 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:58.271+0000 7f70477fe640 1 -- 172.21.15.143:0/1833270641 --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f703800dce0 con 0x7f7050143ac0 2023-11-26T07:19:58.275 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:58.271+0000 7f7054c9d640 1 --2- 172.21.15.143:0/1833270641 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f7038005b90 0x7f7038005fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:19:58.275 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:58.271+0000 7f7047fff640 1 --2- 172.21.15.143:0/1833270641 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f70380069e0 0x7f7038006e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:19:58.275 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:58.271+0000 7f705549e640 1 --2- 172.21.15.143:0/1833270641 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f7050143ac0 0x7f70380073a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:19:58.275 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:58.271+0000 7f7054c9d640 1 -- 172.21.15.143:0/1833270641 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f7050143ac0 msgr2=0x7f70380073a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-26T07:19:58.276 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:58.271+0000 7f7054c9d640 1 --2- 172.21.15.143:0/1833270641 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f7050143ac0 0x7f70380073a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:19:58.276 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:58.271+0000 7f7054c9d640 1 -- 172.21.15.143:0/1833270641 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f70380069e0 msgr2=0x7f7038006e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-26T07:19:58.276 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:58.271+0000 7f7054c9d640 1 --2- 172.21.15.143:0/1833270641 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f70380069e0 0x7f7038006e60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:19:58.276 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:58.271+0000 7f7054c9d640 1 -- 172.21.15.143:0/1833270641 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7038005740 con 0x7f7038005b90 2023-11-26T07:19:58.276 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:58.271+0000 7f7054c9d640 1 --2- 172.21.15.143:0/1833270641 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f7038005b90 0x7f7038005fb0 secure :-1 s=READY pgs=654 cs=0 l=1 rev1=1 crypto rx=0x7f704c009ae0 tx=0x7f704c01ca50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8cb8e5784ae11893 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:19:58.276 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:58.275+0000 7f70477fe640 1 -- 172.21.15.143:0/1833270641 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f704c0a6460 con 0x7f7038005b90 2023-11-26T07:19:58.277 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:58.275+0000 7f70477fe640 1 -- 172.21.15.143:0/1833270641 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f704c0a65c0 con 0x7f7038005b90 2023-11-26T07:19:58.277 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:58.275+0000 7f70477fe640 1 -- 172.21.15.143:0/1833270641 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f704c0a6970 con 0x7f7038005b90 2023-11-26T07:19:58.277 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:58.275+0000 7f7056f01640 1 -- 172.21.15.143:0/1833270641 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f7038005b90 msgr2=0x7f7038005fb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:19:58.277 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:58.275+0000 7f7056f01640 1 --2- 172.21.15.143:0/1833270641 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f7038005b90 0x7f7038005fb0 secure :-1 s=READY pgs=654 cs=0 l=1 rev1=1 crypto rx=0x7f704c009ae0 tx=0x7f704c01ca50 comp rx=0 tx=0).stop 2023-11-26T07:19:58.277 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:58.275+0000 7f705549e640 1 -- 172.21.15.143:0/1833270641 reap_dead start 2023-11-26T07:19:58.278 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:58.275+0000 7f7056f01640 1 -- 172.21.15.143:0/1833270641 shutdown_connections 2023-11-26T07:19:58.278 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:58.275+0000 7f7056f01640 1 -- 172.21.15.143:0/1833270641 >> 172.21.15.143:0/1833270641 conn(0x7f705006a890 msgr2=0x7f705006aca0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:19:58.278 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:58.275+0000 7f7056f01640 1 -- 172.21.15.143:0/1833270641 shutdown_connections 2023-11-26T07:19:58.278 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:58.275+0000 7f7056f01640 1 -- 172.21.15.143:0/1833270641 wait complete. 2023-11-26T07:19:58.279 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:58.275+0000 7f7056f01640 1 Processor -- start 2023-11-26T07:19:58.279 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:58.275+0000 7f7056f01640 1 -- start start 2023-11-26T07:19:58.282 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:58.275+0000 7f7056f01640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f7038005b90 0x7f70500aec40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:19:58.282 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:58.275+0000 7f7056f01640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f70500af3e0 0x7f705013d290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:19:58.282 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:58.275+0000 7f7056f01640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f70501594d0 0x7f7050159950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:19:58.282 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:58.275+0000 7f7056f01640 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7050159e90 con 0x7f7038005b90 2023-11-26T07:19:58.282 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:58.275+0000 7f7056f01640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f705015b2e0 con 0x7f70500af3e0 2023-11-26T07:19:58.282 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:58.275+0000 7f7056f01640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f705015b580 con 0x7f70501594d0 2023-11-26T07:19:58.282 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:58.279+0000 7f7054c9d640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f7038005b90 0x7f70500aec40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:19:58.282 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:58.279+0000 7f7054c9d640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f7038005b90 0x7f70500aec40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:35156/0 (socket says 172.21.15.143:35156) 2023-11-26T07:19:58.282 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:58.279+0000 7f7054c9d640 1 -- 172.21.15.143:0/2355111235 learned_addr learned my addr 172.21.15.143:0/2355111235 (peer_addr_for_me v2:172.21.15.143:0/0) 2023-11-26T07:19:58.282 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:58.279+0000 7f7047fff640 1 --2- 172.21.15.143:0/2355111235 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f70500af3e0 0x7f705013d290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:19:58.282 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:58.279+0000 7f705549e640 1 --2- 172.21.15.143:0/2355111235 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f70501594d0 0x7f7050159950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:19:58.282 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:58.279+0000 7f7054c9d640 1 -- 172.21.15.143:0/2355111235 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f70501594d0 msgr2=0x7f7050159950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:19:58.283 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:58.279+0000 7f7054c9d640 1 --2- 172.21.15.143:0/2355111235 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f70501594d0 0x7f7050159950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:19:58.283 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:58.279+0000 7f7054c9d640 1 -- 172.21.15.143:0/2355111235 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f70500af3e0 msgr2=0x7f705013d290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:19:58.283 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:58.279+0000 7f7054c9d640 1 --2- 172.21.15.143:0/2355111235 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f70500af3e0 0x7f705013d290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:19:58.283 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:58.279+0000 7f7054c9d640 1 -- 172.21.15.143:0/2355111235 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f705015b860 con 0x7f7038005b90 2023-11-26T07:19:58.283 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:58.279+0000 7f7054c9d640 1 --2- 172.21.15.143:0/2355111235 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f7038005b90 0x7f70500aec40 secure :-1 s=READY pgs=655 cs=0 l=1 rev1=1 crypto rx=0x7f704c009ae0 tx=0x7f7038007a10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:19:58.283 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:58.279+0000 7f70457fa640 1 -- 172.21.15.143:0/2355111235 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f704c0a6d30 con 0x7f7038005b90 2023-11-26T07:19:58.283 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:58.279+0000 7f70457fa640 1 -- 172.21.15.143:0/2355111235 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f704c0a6e90 con 0x7f7038005b90 2023-11-26T07:19:58.283 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:58.279+0000 7f70457fa640 1 -- 172.21.15.143:0/2355111235 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f704c004240 con 0x7f7038005b90 2023-11-26T07:19:58.283 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:58.279+0000 7f7056f01640 1 -- 172.21.15.143:0/2355111235 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f705015bb40 con 0x7f7038005b90 2023-11-26T07:19:58.283 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:58.279+0000 7f7056f01640 1 -- 172.21.15.143:0/2355111235 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f70500afb50 con 0x7f7038005b90 2023-11-26T07:19:58.284 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:58.279+0000 7f7056f01640 1 -- 172.21.15.143:0/2355111235 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7050067680 con 0x7f7038005b90 2023-11-26T07:19:58.284 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:58.279+0000 7f70457fa640 1 -- 172.21.15.143:0/2355111235 <== mon.0 v2:172.21.15.143:3300/0 4 ==== mgrmap(e 7) v1 ==== 86005+0+0 (secure 0 0 0) 0x7f704c0043a0 con 0x7f7038005b90 2023-11-26T07:19:58.284 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:58.279+0000 7f70457fa640 1 --2- 172.21.15.143:0/2355111235 >> [v2:172.21.15.143:6832/2150557544,v1:172.21.15.143:6833/2150557544] conn(0x7f703c066650 0x7f703c068b10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:19:58.284 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:58.283+0000 7f70457fa640 1 -- 172.21.15.143:0/2355111235 <== mon.0 v2:172.21.15.143:3300/0 5 ==== osd_map(45..45 src has 1..45) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f704c11dda0 con 0x7f7038005b90 2023-11-26T07:19:58.284 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:58.283+0000 7f7047fff640 1 --2- 172.21.15.143:0/2355111235 >> [v2:172.21.15.143:6832/2150557544,v1:172.21.15.143:6833/2150557544] conn(0x7f703c066650 0x7f703c068b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:19:58.285 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:58.283+0000 7f7047fff640 1 --2- 172.21.15.143:0/2355111235 >> [v2:172.21.15.143:6832/2150557544,v1:172.21.15.143:6833/2150557544] conn(0x7f703c066650 0x7f703c068b10 secure :-1 s=READY pgs=318 cs=0 l=1 rev1=1 crypto rx=0x7f705015d6e0 tx=0x7f7040007900 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:19:58.288 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:58.287+0000 7f70457fa640 1 -- 172.21.15.143:0/2355111235 <== mon.0 v2:172.21.15.143:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172649 (secure 0 0 0) 0x7f704c0eaf20 con 0x7f7038005b90 2023-11-26T07:19:58.448 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:58.443+0000 7f7056f01640 1 -- 172.21.15.143:0/2355111235 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) v1 -- 0x7f7050142cb0 con 0x7f7038005b90 2023-11-26T07:19:58.448 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:58.447+0000 7f70457fa640 1 -- 172.21.15.143:0/2355111235 <== mon.0 v2:172.21.15.143:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v45) v1 ==== 80+0+0 (secure 0 0 0) 0x7f704c0efe60 con 0x7f7038005b90 2023-11-26T07:19:58.449 INFO:teuthology.orchestra.run.smithi143.stderr:listed 0 entries 2023-11-26T07:19:58.451 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:58.447+0000 7f7056f01640 1 -- 172.21.15.143:0/2355111235 >> [v2:172.21.15.143:6832/2150557544,v1:172.21.15.143:6833/2150557544] conn(0x7f703c066650 msgr2=0x7f703c068b10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:19:58.451 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:58.447+0000 7f7056f01640 1 --2- 172.21.15.143:0/2355111235 >> [v2:172.21.15.143:6832/2150557544,v1:172.21.15.143:6833/2150557544] conn(0x7f703c066650 0x7f703c068b10 secure :-1 s=READY pgs=318 cs=0 l=1 rev1=1 crypto rx=0x7f705015d6e0 tx=0x7f7040007900 comp rx=0 tx=0).stop 2023-11-26T07:19:58.451 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:58.447+0000 7f7056f01640 1 -- 172.21.15.143:0/2355111235 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f7038005b90 msgr2=0x7f70500aec40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:19:58.451 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:58.447+0000 7f7056f01640 1 --2- 172.21.15.143:0/2355111235 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f7038005b90 0x7f70500aec40 secure :-1 s=READY pgs=655 cs=0 l=1 rev1=1 crypto rx=0x7f704c009ae0 tx=0x7f7038007a10 comp rx=0 tx=0).stop 2023-11-26T07:19:58.451 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:58.447+0000 7f7056f01640 1 -- 172.21.15.143:0/2355111235 shutdown_connections 2023-11-26T07:19:58.451 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:58.447+0000 7f7056f01640 1 --2- 172.21.15.143:0/2355111235 >> [v2:172.21.15.143:6832/2150557544,v1:172.21.15.143:6833/2150557544] conn(0x7f703c066650 0x7f703c068b10 unknown :-1 s=CLOSED pgs=318 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:19:58.451 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:58.447+0000 7f7056f01640 1 --2- 172.21.15.143:0/2355111235 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f70501594d0 0x7f7050159950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:19:58.451 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:58.447+0000 7f7056f01640 1 --2- 172.21.15.143:0/2355111235 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f70500af3e0 0x7f705013d290 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:19:58.451 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:58.447+0000 7f7056f01640 1 --2- 172.21.15.143:0/2355111235 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f7038005b90 0x7f70500aec40 unknown :-1 s=CLOSED pgs=655 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:19:58.451 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:58.447+0000 7f7056f01640 1 -- 172.21.15.143:0/2355111235 >> 172.21.15.143:0/2355111235 conn(0x7f705006a890 msgr2=0x7f7050072130 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:19:58.452 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:58.447+0000 7f7056f01640 1 -- 172.21.15.143:0/2355111235 shutdown_connections 2023-11-26T07:19:58.452 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:58.447+0000 7f7056f01640 1 -- 172.21.15.143:0/2355111235 wait complete. 2023-11-26T07:19:58.470 DEBUG:teuthology.orchestra.run.smithi188:> set -ex 2023-11-26T07:19:58.470 DEBUG:teuthology.orchestra.run.smithi188:> dd if=/proc/self/mounts of=/dev/stdout 2023-11-26T07:19:58.480 DEBUG:teuthology.orchestra.run.smithi143:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-11-26T07:19:58.619 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:58.615+0000 7f38d9f01640 1 Processor -- start 2023-11-26T07:19:58.619 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:58.615+0000 7f38d9f01640 1 -- start start 2023-11-26T07:19:58.620 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:58.615+0000 7f38d9f01640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f38d40cf280 0x7f38d40cf660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:19:58.620 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:58.615+0000 7f38d9f01640 1 -- --> v1:172.21.15.143:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f38d415b2f0 con 0x7f38d4070d10 2023-11-26T07:19:58.620 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:58.615+0000 7f38d9f01640 1 -- --> v1:172.21.15.188:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f38d406c5c0 con 0x7f38d406cd50 2023-11-26T07:19:58.620 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:58.615+0000 7f38d9f01640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f38d40cfba0 con 0x7f38d40cf280 2023-11-26T07:19:58.620 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:58.615+0000 7f38cbfff640 1 --1- >> v1:172.21.15.143:6789/0 conn(0x7f38d4070d10 0x7f38d415d900 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.143:6789/0 says I am v1:172.21.15.143:47578/0 (socket says 172.21.15.143:47578) 2023-11-26T07:19:58.620 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:58.615+0000 7f38cbfff640 1 -- 172.21.15.143:0/2425885241 learned_addr learned my addr 172.21.15.143:0/2425885241 (peer_addr_for_me v1:172.21.15.143:0/0) 2023-11-26T07:19:58.622 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:58.619+0000 7f38d37fe640 1 --2- 172.21.15.143:0/2425885241 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f38d40cf280 0x7f38d40cf660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:19:58.622 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:58.619+0000 7f38d2ffd640 1 -- 172.21.15.143:0/2425885241 <== mon.0 v1:172.21.15.143:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1585924597 0 0) 0x7f38bc001120 con 0x7f38d4070d10 2023-11-26T07:19:58.623 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:58.619+0000 7f38d2ffd640 1 -- 172.21.15.143:0/2425885241 --> v1:172.21.15.143:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f38c0003530 con 0x7f38d4070d10 2023-11-26T07:19:58.623 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:58.619+0000 7f38d2ffd640 1 -- 172.21.15.143:0/2425885241 <== mon.1 v1:172.21.15.188:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1384132988 0 0) 0x7f38d406c5c0 con 0x7f38d406cd50 2023-11-26T07:19:58.623 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:58.619+0000 7f38d2ffd640 1 -- 172.21.15.143:0/2425885241 --> v1:172.21.15.188:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f38bc001120 con 0x7f38d406cd50 2023-11-26T07:19:58.623 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:58.619+0000 7f38d2ffd640 1 -- 172.21.15.143:0/2425885241 <== mon.0 v1:172.21.15.143:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2565174188 0 0) 0x7f38c0003530 con 0x7f38d4070d10 2023-11-26T07:19:58.623 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:58.619+0000 7f38d2ffd640 1 -- 172.21.15.143:0/2425885241 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f38d40cf280 msgr2=0x7f38d40cf660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-26T07:19:58.623 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:58.619+0000 7f38d2ffd640 1 --2- 172.21.15.143:0/2425885241 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f38d40cf280 0x7f38d40cf660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:19:58.623 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:58.619+0000 7f38d2ffd640 1 -- 172.21.15.143:0/2425885241 >> v1:172.21.15.188:6789/0 conn(0x7f38d406cd50 legacy=0x7f38d40b2180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:19:58.623 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:58.619+0000 7f38d2ffd640 1 -- 172.21.15.143:0/2425885241 --> v1:172.21.15.143:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f38d415e1a0 con 0x7f38d4070d10 2023-11-26T07:19:58.623 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:58.619+0000 7f38d2ffd640 1 -- 172.21.15.143:0/2425885241 <== mon.0 v1:172.21.15.143:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2137882523 0 0) 0x7f38bc001810 con 0x7f38d4070d10 2023-11-26T07:19:58.623 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:58.619+0000 7f38d37fe640 1 --2- 172.21.15.143:0/2425885241 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f38d40cf280 0x7f38d40cf660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-26T07:19:58.623 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:58.619+0000 7f38d2ffd640 1 -- 172.21.15.143:0/2425885241 >> v1:172.21.15.143:6789/0 conn(0x7f38d4070d10 legacy=0x7f38d415d900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:19:58.623 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:58.619+0000 7f38d2ffd640 1 --2- 172.21.15.143:0/2425885241 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f38d40cf280 0x7f38c0003b40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:19:58.623 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:58.619+0000 7f38d2ffd640 1 --2- 172.21.15.143:0/2425885241 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f38c00040d0 0x7f38c00047d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:19:58.624 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:58.619+0000 7f38d2ffd640 1 --2- 172.21.15.143:0/2425885241 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f38c0004d10 0x7f38c0005190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:19:58.624 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:58.619+0000 7f38d2ffd640 1 -- 172.21.15.143:0/2425885241 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f38d40cfba0 con 0x7f38c00040d0 2023-11-26T07:19:58.624 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:58.619+0000 7f38d2ffd640 1 -- 172.21.15.143:0/2425885241 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f38c000baa0 con 0x7f38c0004d10 2023-11-26T07:19:58.624 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:58.619+0000 7f38d2ffd640 1 -- 172.21.15.143:0/2425885241 --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f38c000bc60 con 0x7f38d40cf280 2023-11-26T07:19:58.624 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:58.619+0000 7f38cbfff640 1 --2- 172.21.15.143:0/2425885241 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f38c00040d0 0x7f38c00047d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:19:58.624 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:58.619+0000 7f38d37fe640 1 --2- 172.21.15.143:0/2425885241 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f38d40cf280 0x7f38c0003b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:19:58.624 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:58.619+0000 7f38d3fff640 1 --2- 172.21.15.143:0/2425885241 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f38c0004d10 0x7f38c0005190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:19:58.624 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:58.619+0000 7f38cbfff640 1 -- 172.21.15.143:0/2425885241 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f38d40cf280 msgr2=0x7f38c0003b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-26T07:19:58.624 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:58.619+0000 7f38cbfff640 1 --2- 172.21.15.143:0/2425885241 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f38d40cf280 0x7f38c0003b40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:19:58.624 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:58.619+0000 7f38cbfff640 1 -- 172.21.15.143:0/2425885241 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f38c0004d10 msgr2=0x7f38c0005190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-26T07:19:58.624 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:58.619+0000 7f38cbfff640 1 --2- 172.21.15.143:0/2425885241 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f38c0004d10 0x7f38c0005190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:19:58.624 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:58.619+0000 7f38cbfff640 1 -- 172.21.15.143:0/2425885241 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f38c4009420 con 0x7f38c00040d0 2023-11-26T07:19:58.625 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:58.623+0000 7f38cbfff640 1 --2- 172.21.15.143:0/2425885241 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f38c00040d0 0x7f38c00047d0 secure :-1 s=READY pgs=657 cs=0 l=1 rev1=1 crypto rx=0x7f38bc007f60 tx=0x7f38bc01f210 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4448a15ae791404c server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:19:58.625 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:58.623+0000 7f38d2ffd640 1 -- 172.21.15.143:0/2425885241 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f38bc0a8e30 con 0x7f38c00040d0 2023-11-26T07:19:58.625 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:58.623+0000 7f38d2ffd640 1 -- 172.21.15.143:0/2425885241 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f38bc0058e0 con 0x7f38c00040d0 2023-11-26T07:19:58.625 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:58.623+0000 7f38d2ffd640 1 -- 172.21.15.143:0/2425885241 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f38bc005cd0 con 0x7f38c00040d0 2023-11-26T07:19:58.626 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:58.623+0000 7f38d9f01640 1 -- 172.21.15.143:0/2425885241 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f38c00040d0 msgr2=0x7f38c00047d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:19:58.626 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:58.623+0000 7f38d9f01640 1 --2- 172.21.15.143:0/2425885241 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f38c00040d0 0x7f38c00047d0 secure :-1 s=READY pgs=657 cs=0 l=1 rev1=1 crypto rx=0x7f38bc007f60 tx=0x7f38bc01f210 comp rx=0 tx=0).stop 2023-11-26T07:19:58.626 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:58.623+0000 7f38d3fff640 1 -- 172.21.15.143:0/2425885241 reap_dead start 2023-11-26T07:19:58.626 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:58.623+0000 7f38d9f01640 1 -- 172.21.15.143:0/2425885241 shutdown_connections 2023-11-26T07:19:58.626 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:58.623+0000 7f38d9f01640 1 -- 172.21.15.143:0/2425885241 >> 172.21.15.143:0/2425885241 conn(0x7f38d406a890 msgr2=0x7f38d406aca0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:19:58.626 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:58.623+0000 7f38d9f01640 1 -- 172.21.15.143:0/2425885241 shutdown_connections 2023-11-26T07:19:58.626 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:58.623+0000 7f38d9f01640 1 -- 172.21.15.143:0/2425885241 wait complete. 2023-11-26T07:19:58.627 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:58.623+0000 7f38d9f01640 1 Processor -- start 2023-11-26T07:19:58.627 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:58.623+0000 7f38d9f01640 1 -- start start 2023-11-26T07:19:58.630 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:58.627+0000 7f38d9f01640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f38c00040d0 0x7f38d4153680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:19:58.630 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:58.627+0000 7f38d9f01640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f38d41e35f0 0x7f38d4152e70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:19:58.630 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:58.627+0000 7f38d9f01640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f38d41e4c50 0x7f38d40b0ae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:19:58.630 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:58.627+0000 7f38d9f01640 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f38d41e3bb0 con 0x7f38d41e35f0 2023-11-26T07:19:58.630 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:58.627+0000 7f38d9f01640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f38d41e3e90 con 0x7f38d41e4c50 2023-11-26T07:19:58.630 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:58.627+0000 7f38d9f01640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f38d41e4170 con 0x7f38c00040d0 2023-11-26T07:19:58.630 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:58.627+0000 7f38cbfff640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f38d41e35f0 0x7f38d4152e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:19:58.630 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:58.627+0000 7f38cbfff640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f38d41e35f0 0x7f38d4152e70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:35178/0 (socket says 172.21.15.143:35178) 2023-11-26T07:19:58.630 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:58.627+0000 7f38cbfff640 1 -- 172.21.15.143:0/4009706013 learned_addr learned my addr 172.21.15.143:0/4009706013 (peer_addr_for_me v2:172.21.15.143:0/0) 2023-11-26T07:19:58.630 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:58.627+0000 7f38d3fff640 1 --2- 172.21.15.143:0/4009706013 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f38d41e4c50 0x7f38d40b0ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:19:58.630 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:58.627+0000 7f38d37fe640 1 --2- 172.21.15.143:0/4009706013 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f38c00040d0 0x7f38d4153680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:19:58.631 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:58.627+0000 7f38cbfff640 1 -- 172.21.15.143:0/4009706013 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f38c00040d0 msgr2=0x7f38d4153680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:19:58.631 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:58.627+0000 7f38cbfff640 1 --2- 172.21.15.143:0/4009706013 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f38c00040d0 0x7f38d4153680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:19:58.631 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:58.627+0000 7f38cbfff640 1 -- 172.21.15.143:0/4009706013 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f38d41e4c50 msgr2=0x7f38d40b0ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:19:58.631 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:58.627+0000 7f38cbfff640 1 --2- 172.21.15.143:0/4009706013 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f38d41e4c50 0x7f38d40b0ae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:19:58.631 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:58.627+0000 7f38cbfff640 1 -- 172.21.15.143:0/4009706013 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f38d41e4450 con 0x7f38d41e35f0 2023-11-26T07:19:58.631 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:58.627+0000 7f38cbfff640 1 --2- 172.21.15.143:0/4009706013 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f38d41e35f0 0x7f38d4152e70 secure :-1 s=READY pgs=658 cs=0 l=1 rev1=1 crypto rx=0x7f38bc007f60 tx=0x7f38c0005800 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:19:58.631 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:58.627+0000 7f38d0ff9640 1 -- 172.21.15.143:0/4009706013 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f38bc0b7070 con 0x7f38d41e35f0 2023-11-26T07:19:58.631 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:58.627+0000 7f38d0ff9640 1 -- 172.21.15.143:0/4009706013 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f38bc0016d0 con 0x7f38d41e35f0 2023-11-26T07:19:58.631 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:58.627+0000 7f38d0ff9640 1 -- 172.21.15.143:0/4009706013 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f38bc004a50 con 0x7f38d41e35f0 2023-11-26T07:19:58.631 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:58.627+0000 7f38d9f01640 1 -- 172.21.15.143:0/4009706013 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f38d41e4730 con 0x7f38d41e35f0 2023-11-26T07:19:58.631 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:58.627+0000 7f38d9f01640 1 -- 172.21.15.143:0/4009706013 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f38d4153e00 con 0x7f38d41e35f0 2023-11-26T07:19:58.632 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:58.627+0000 7f38d9f01640 1 -- 172.21.15.143:0/4009706013 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f38d41e48c0 con 0x7f38d41e35f0 2023-11-26T07:19:58.632 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:58.627+0000 7f38d0ff9640 1 -- 172.21.15.143:0/4009706013 <== mon.0 v2:172.21.15.143:3300/0 4 ==== mgrmap(e 7) v1 ==== 86005+0+0 (secure 0 0 0) 0x7f38bc004bb0 con 0x7f38d41e35f0 2023-11-26T07:19:58.632 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:58.631+0000 7f38d0ff9640 1 --2- 172.21.15.143:0/4009706013 >> [v2:172.21.15.143:6832/2150557544,v1:172.21.15.143:6833/2150557544] conn(0x7f38b4066620 0x7f38b4068ae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:19:58.633 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:58.631+0000 7f38d0ff9640 1 -- 172.21.15.143:0/4009706013 <== mon.0 v2:172.21.15.143:3300/0 5 ==== osd_map(45..45 src has 1..45) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f38bc121220 con 0x7f38d41e35f0 2023-11-26T07:19:58.633 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:58.631+0000 7f38d37fe640 1 --2- 172.21.15.143:0/4009706013 >> [v2:172.21.15.143:6832/2150557544,v1:172.21.15.143:6833/2150557544] conn(0x7f38b4066620 0x7f38b4068ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:19:58.633 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:58.631+0000 7f38d37fe640 1 --2- 172.21.15.143:0/4009706013 >> [v2:172.21.15.143:6832/2150557544,v1:172.21.15.143:6833/2150557544] conn(0x7f38b4066620 0x7f38b4068ae0 secure :-1 s=READY pgs=319 cs=0 l=1 rev1=1 crypto rx=0x7f38d4154240 tx=0x7f38c4008040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:19:58.636 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:58.635+0000 7f38d0ff9640 1 -- 172.21.15.143:0/4009706013 <== mon.0 v2:172.21.15.143:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172649 (secure 0 0 0) 0x7f38bc0ee4b0 con 0x7f38d41e35f0 2023-11-26T07:19:58.788 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:58.783+0000 7f38d9f01640 1 -- 172.21.15.143:0/4009706013 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f38d415b2f0 con 0x7f38d41e35f0 2023-11-26T07:19:58.789 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:58.787+0000 7f38d0ff9640 1 -- 172.21.15.143:0/4009706013 <== mon.0 v2:172.21.15.143:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v45) v1 ==== 74+0+16549 (secure 0 0 0) 0x7f38d415b2f0 con 0x7f38d41e35f0 2023-11-26T07:19:58.789 INFO:teuthology.orchestra.run.smithi143.stdout: 2023-11-26T07:19:58.789 INFO:teuthology.orchestra.run.smithi143.stdout:{"epoch":45,"fsid":"f5033060-2469-4a07-95fe-2b76f5404bc1","created":"2023-11-26T07:17:28.613663+0000","modified":"2023-11-26T07:19:33.132471+0000","last_up_change":"2023-11-26T07:17:41.157813+0000","last_in_change":"2023-11-26T07:17:31.181658+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":7,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-11-26T07:17:41.758597+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":6,"pool_name":"cephfs_metadata","create_time":"2023-11-26T07:19:29.208075+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"44","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":7,"pool_name":"cephfs_data","create_time":"2023-11-26T07:19:30.503743+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"45","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"ebbee2c6-023e-44b9-9f1d-6f7bfd309730","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":43,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6817","nonce":639807369},{"type":"v1","addr":"172.21.15.143:6819","nonce":639807369}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6821","nonce":639807369},{"type":"v1","addr":"172.21.15.143:6823","nonce":639807369}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6830","nonce":639807369},{"type":"v1","addr":"172.21.15.143:6831","nonce":639807369}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6825","nonce":639807369},{"type":"v1","addr":"172.21.15.143:6828","nonce":639807369}]},"public_addr":"172.21.15.143:6819/639807369","cluster_addr":"172.21.15.143:6823/639807369","heartbeat_back_addr":"172.21.15.143:6831/639807369","heartbeat_front_addr":"172.21.15.143:6828/639807369","state":["exists","up"]},{"osd":1,"uuid":"fec4b8be-7122-49e9-9a19-5850a0700001","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":43,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6816","nonce":1144942530},{"type":"v1","addr":"172.21.15.143:6818","nonce":1144942530}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6820","nonce":1144942530},{"type":"v1","addr":"172.21.15.143:6822","nonce":1144942530}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6827","nonce":1144942530},{"type":"v1","addr":"172.21.15.143:6829","nonce":1144942530}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6824","nonce":1144942530},{"type":"v1","addr":"172.21.15.143:6826","nonce":1144942530}]},"public_addr":"172.21.15.143:6818/1144942530","cluster_addr":"172.21.15.143:6822/1144942530","heartbeat_back_addr":"172.21.15.143:6829/1144942530","heartbeat_front_addr":"172.21.15.143:6826/1144942530","state":["exists","up"]},{"osd":2,"uuid":"2888af5b-f933-46ca-8f95-6dd494ff43f6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":43,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6808","nonce":3909095186},{"type":"v1","addr":"172.21.15.143:6809","nonce":3909095186}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6810","nonce":3909095186},{"type":"v1","addr":"172.21.15.143:6811","nonce":3909095186}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6814","nonce":3909095186},{"type":"v1","addr":"172.21.15.143:6815","nonce":3909095186}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6812","nonce":3909095186},{"type":"v1","addr":"172.21.15.143:6813","nonce":3909095186}]},"public_addr":"172.21.15.143:6809/3909095186","cluster_addr":"172.21.15.143:6811/3909095186","heartbeat_back_addr":"172.21.15.143:6815/3909095186","heartbeat_front_addr":"172.21.15.143:6813/3909095186","state":["exists","up"]},{"osd":3,"uuid":"d1f52435-7bc1-4081-9550-9cd1ba55f549","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":43,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6800","nonce":3854210991},{"type":"v1","addr":"172.21.15.143:6801","nonce":3854210991}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6802","nonce":3854210991},{"type":"v1","addr":"172.21.15.143:6803","nonce":3854210991}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6806","nonce":3854210991},{"type":"v1","addr":"172.21.15.143:6807","nonce":3854210991}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6804","nonce":3854210991},{"type":"v1","addr":"172.21.15.143:6805","nonce":3854210991}]},"public_addr":"172.21.15.143:6801/3854210991","cluster_addr":"172.21.15.143:6803/3854210991","heartbeat_back_addr":"172.21.15.143:6807/3854210991","heartbeat_front_addr":"172.21.15.143:6805/3854210991","state":["exists","up"]},{"osd":4,"uuid":"e2ae8b6e-6e25-49b2-bcbf-aacce68f06a3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":43,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6800","nonce":3924606902},{"type":"v1","addr":"172.21.15.188:6801","nonce":3924606902}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6802","nonce":3924606902},{"type":"v1","addr":"172.21.15.188:6803","nonce":3924606902}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6806","nonce":3924606902},{"type":"v1","addr":"172.21.15.188:6807","nonce":3924606902}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6804","nonce":3924606902},{"type":"v1","addr":"172.21.15.188:6805","nonce":3924606902}]},"public_addr":"172.21.15.188:6801/3924606902","cluster_addr":"172.21.15.188:6803/3924606902","heartbeat_back_addr":"172.21.15.188:6807/3924606902","heartbeat_front_addr":"172.21.15.188:6805/3924606902","state":["exists","up"]},{"osd":5,"uuid":"3cec8a32-7698-4c36-8e89-e4ba11934f33","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":43,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6813","nonce":186841544},{"type":"v1","addr":"172.21.15.188:6815","nonce":186841544}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6817","nonce":186841544},{"type":"v1","addr":"172.21.15.188:6818","nonce":186841544}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6822","nonce":186841544},{"type":"v1","addr":"172.21.15.188:6823","nonce":186841544}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6820","nonce":186841544},{"type":"v1","addr":"172.21.15.188:6821","nonce":186841544}]},"public_addr":"172.21.15.188:6815/186841544","cluster_addr":"172.21.15.188:6818/186841544","heartbeat_back_addr":"172.21.15.188:6823/186841544","heartbeat_front_addr":"172.21.15.188:6821/186841544","state":["exists","up"]},{"osd":6,"uuid":"58a65d5d-e090-473a-81b4-ad38e85029f9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":43,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6808","nonce":2153499405},{"type":"v1","addr":"172.21.15.188:6809","nonce":2153499405}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6810","nonce":2153499405},{"type":"v1","addr":"172.21.15.188:6811","nonce":2153499405}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6816","nonce":2153499405},{"type":"v1","addr":"172.21.15.188:6819","nonce":2153499405}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6812","nonce":2153499405},{"type":"v1","addr":"172.21.15.188:6814","nonce":2153499405}]},"public_addr":"172.21.15.188:6809/2153499405","cluster_addr":"172.21.15.188:6811/2153499405","heartbeat_back_addr":"172.21.15.188:6819/2153499405","heartbeat_front_addr":"172.21.15.188:6814/2153499405","state":["exists","up"]},{"osd":7,"uuid":"ea5b7f70-3554-4cf8-ac30-f7c59b80678d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":43,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6824","nonce":2722096729},{"type":"v1","addr":"172.21.15.188:6825","nonce":2722096729}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6826","nonce":2722096729},{"type":"v1","addr":"172.21.15.188:6827","nonce":2722096729}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6830","nonce":2722096729},{"type":"v1","addr":"172.21.15.188:6831","nonce":2722096729}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6828","nonce":2722096729},{"type":"v1","addr":"172.21.15.188:6829","nonce":2722096729}]},"public_addr":"172.21.15.188:6825/2722096729","cluster_addr":"172.21.15.188:6827/2722096729","heartbeat_back_addr":"172.21.15.188:6831/2722096729","heartbeat_front_addr":"172.21.15.188:6829/2722096729","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-26T07:17:36.995032+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-26T07:17:36.932266+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-26T07:17:36.829304+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-26T07:17:36.804297+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-26T07:17:36.667892+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-26T07:17:36.964868+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-26T07:17:36.827332+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-26T07:17:36.893179+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-11-26T07:19:58.791 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:58.787+0000 7f38d9f01640 1 -- 172.21.15.143:0/4009706013 >> [v2:172.21.15.143:6832/2150557544,v1:172.21.15.143:6833/2150557544] conn(0x7f38b4066620 msgr2=0x7f38b4068ae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:19:58.791 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:58.787+0000 7f38d9f01640 1 --2- 172.21.15.143:0/4009706013 >> [v2:172.21.15.143:6832/2150557544,v1:172.21.15.143:6833/2150557544] conn(0x7f38b4066620 0x7f38b4068ae0 secure :-1 s=READY pgs=319 cs=0 l=1 rev1=1 crypto rx=0x7f38d4154240 tx=0x7f38c4008040 comp rx=0 tx=0).stop 2023-11-26T07:19:58.791 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:58.787+0000 7f38d9f01640 1 -- 172.21.15.143:0/4009706013 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f38d41e35f0 msgr2=0x7f38d4152e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:19:58.791 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:58.787+0000 7f38d9f01640 1 --2- 172.21.15.143:0/4009706013 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f38d41e35f0 0x7f38d4152e70 secure :-1 s=READY pgs=658 cs=0 l=1 rev1=1 crypto rx=0x7f38bc007f60 tx=0x7f38c0005800 comp rx=0 tx=0).stop 2023-11-26T07:19:58.791 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:58.787+0000 7f38d9f01640 1 -- 172.21.15.143:0/4009706013 shutdown_connections 2023-11-26T07:19:58.792 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:58.787+0000 7f38d9f01640 1 --2- 172.21.15.143:0/4009706013 >> [v2:172.21.15.143:6832/2150557544,v1:172.21.15.143:6833/2150557544] conn(0x7f38b4066620 0x7f38b4068ae0 unknown :-1 s=CLOSED pgs=319 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:19:58.792 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:58.787+0000 7f38d9f01640 1 --2- 172.21.15.143:0/4009706013 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f38d41e4c50 0x7f38d40b0ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:19:58.792 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:58.787+0000 7f38d9f01640 1 --2- 172.21.15.143:0/4009706013 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f38d41e35f0 0x7f38d4152e70 unknown :-1 s=CLOSED pgs=658 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:19:58.792 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:58.787+0000 7f38d9f01640 1 --2- 172.21.15.143:0/4009706013 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f38c00040d0 0x7f38d4153680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:19:58.792 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:58.787+0000 7f38d9f01640 1 -- 172.21.15.143:0/4009706013 >> 172.21.15.143:0/4009706013 conn(0x7f38d406a890 msgr2=0x7f38d40ad780 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:19:58.792 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:58.787+0000 7f38d9f01640 1 -- 172.21.15.143:0/4009706013 shutdown_connections 2023-11-26T07:19:58.792 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:58.787+0000 7f38d9f01640 1 -- 172.21.15.143:0/4009706013 wait complete. 2023-11-26T07:19:58.810 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-11-26T07:17:41.758597+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 6, 'pool_name': 'cephfs_metadata', 'create_time': '2023-11-26T07:19:29.208075+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '44', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 1.75, 'score_stable': 1.75, 'optimal_score': 1, 'raw_score_acting': 1.75, 'raw_score_stable': 1.75, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 7, 'pool_name': 'cephfs_data', 'create_time': '2023-11-26T07:19:30.503743+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '45', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.75, 'score_stable': 1.75, 'optimal_score': 1, 'raw_score_acting': 1.75, 'raw_score_stable': 1.75, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2023-11-26T07:19:58.811 DEBUG:teuthology.orchestra.run.smithi143:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-11-26T07:19:58.951 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:58.947+0000 7f2c551f5640 1 Processor -- start 2023-11-26T07:19:58.951 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:58.947+0000 7f2c551f5640 1 -- start start 2023-11-26T07:19:58.952 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:58.947+0000 7f2c551f5640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f2c500cf710 0x7f2c500cd8c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:19:58.952 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:58.947+0000 7f2c551f5640 1 -- --> v1:172.21.15.143:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2c500cc730 con 0x7f2c50143ac0 2023-11-26T07:19:58.952 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:58.947+0000 7f2c551f5640 1 -- --> v1:172.21.15.188:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2c50070ee0 con 0x7f2c500cf280 2023-11-26T07:19:58.953 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:58.947+0000 7f2c551f5640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2c50143ea0 con 0x7f2c500cf710 2023-11-26T07:19:58.953 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:58.951+0000 7f2c4ed9d640 1 --1- >> v1:172.21.15.143:6789/0 conn(0x7f2c50143ac0 0x7f2c50070370 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.143:6789/0 says I am v1:172.21.15.143:47592/0 (socket says 172.21.15.143:47592) 2023-11-26T07:19:58.953 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:58.951+0000 7f2c4ed9d640 1 -- 172.21.15.143:0/633851964 learned_addr learned my addr 172.21.15.143:0/633851964 (peer_addr_for_me v1:172.21.15.143:0/0) 2023-11-26T07:19:58.953 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:58.951+0000 7f2c4f59e640 1 --2- 172.21.15.143:0/633851964 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f2c500cf710 0x7f2c500cd8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:19:58.953 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:58.951+0000 7f2c4e59c640 1 -- 172.21.15.143:0/633851964 <== mon.0 v1:172.21.15.143:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 599598364 0 0) 0x7f2c500cc730 con 0x7f2c50143ac0 2023-11-26T07:19:58.953 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:58.951+0000 7f2c4e59c640 1 -- 172.21.15.143:0/633851964 --> v1:172.21.15.143:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f2c3c005740 con 0x7f2c50143ac0 2023-11-26T07:19:58.954 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:58.951+0000 7f2c4e59c640 1 -- 172.21.15.143:0/633851964 <== mon.1 v1:172.21.15.188:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2741005269 0 0) 0x7f2c50070ee0 con 0x7f2c500cf280 2023-11-26T07:19:58.954 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:58.951+0000 7f2c4f59e640 1 -- 172.21.15.143:0/633851964 >> v1:172.21.15.188:6789/0 conn(0x7f2c500cf280 legacy=0x7f2c500cd1b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:19:58.954 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:58.951+0000 7f2c4f59e640 1 -- 172.21.15.143:0/633851964 >> v1:172.21.15.143:6789/0 conn(0x7f2c50143ac0 legacy=0x7f2c50070370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:19:58.954 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:58.951+0000 7f2c4f59e640 1 -- 172.21.15.143:0/633851964 --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2c500711c0 con 0x7f2c500cf710 2023-11-26T07:19:58.956 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:58.951+0000 7f2c4f59e640 1 --2- 172.21.15.143:0/633851964 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f2c500cf710 0x7f2c500cd8c0 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7f2c48009dc0 tx=0x7f2c4801c970 comp rx=0 tx=0).ready entity=mon.2 client_cookie=496884cd342c9c27 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:19:58.956 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:58.951+0000 7f2c4e59c640 1 -- 172.21.15.143:0/633851964 <== mon.2 v2:172.21.15.188:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2c480a5a30 con 0x7f2c500cf710 2023-11-26T07:19:58.956 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:58.951+0000 7f2c4e59c640 1 -- 172.21.15.143:0/633851964 <== mon.2 v2:172.21.15.188:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2c480a5b90 con 0x7f2c500cf710 2023-11-26T07:19:58.956 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:58.951+0000 7f2c4e59c640 1 -- 172.21.15.143:0/633851964 <== mon.2 v2:172.21.15.188:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2c480ad440 con 0x7f2c500cf710 2023-11-26T07:19:58.957 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:58.955+0000 7f2c551f5640 1 -- 172.21.15.143:0/633851964 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f2c500cf710 msgr2=0x7f2c500cd8c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:19:58.957 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:58.955+0000 7f2c551f5640 1 --2- 172.21.15.143:0/633851964 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f2c500cf710 0x7f2c500cd8c0 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7f2c48009dc0 tx=0x7f2c4801c970 comp rx=0 tx=0).stop 2023-11-26T07:19:58.957 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:58.955+0000 7f2c551f5640 1 -- 172.21.15.143:0/633851964 shutdown_connections 2023-11-26T07:19:58.957 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:58.955+0000 7f2c551f5640 1 --2- 172.21.15.143:0/633851964 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f2c500cf710 0x7f2c500cd8c0 unknown :-1 s=CLOSED pgs=92 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:19:58.957 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:58.955+0000 7f2c551f5640 1 -- 172.21.15.143:0/633851964 >> 172.21.15.143:0/633851964 conn(0x7f2c5006a890 msgr2=0x7f2c5006aca0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:19:58.957 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:58.955+0000 7f2c551f5640 1 -- 172.21.15.143:0/633851964 shutdown_connections 2023-11-26T07:19:58.958 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:58.955+0000 7f2c551f5640 1 -- 172.21.15.143:0/633851964 wait complete. 2023-11-26T07:19:58.959 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:58.955+0000 7f2c551f5640 1 Processor -- start 2023-11-26T07:19:58.959 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:58.955+0000 7f2c551f5640 1 -- start start 2023-11-26T07:19:58.960 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:58.955+0000 7f2c551f5640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f2c500cf280 0x7f2c50141400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:19:58.960 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:58.955+0000 7f2c551f5640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f2c500cf710 0x7f2c50137ee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:19:58.960 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:58.955+0000 7f2c551f5640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f2c50143ac0 0x7f2c5013f930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:19:58.960 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:58.955+0000 7f2c551f5640 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2c50140f70 con 0x7f2c500cf280 2023-11-26T07:19:58.960 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:58.955+0000 7f2c551f5640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2c50140360 con 0x7f2c500cf710 2023-11-26T07:19:58.961 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:58.955+0000 7f2c551f5640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2c50140640 con 0x7f2c50143ac0 2023-11-26T07:19:58.961 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:58.955+0000 7f2c4ed9d640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f2c500cf280 0x7f2c50141400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:19:58.961 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:58.959+0000 7f2c4ed9d640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f2c500cf280 0x7f2c50141400 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:35192/0 (socket says 172.21.15.143:35192) 2023-11-26T07:19:58.961 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:58.959+0000 7f2c4ed9d640 1 -- 172.21.15.143:0/1463996959 learned_addr learned my addr 172.21.15.143:0/1463996959 (peer_addr_for_me v2:172.21.15.143:0/0) 2023-11-26T07:19:58.962 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:58.959+0000 7f2c4f59e640 1 --2- 172.21.15.143:0/1463996959 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f2c50143ac0 0x7f2c5013f930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:19:58.962 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:58.959+0000 7f2c4659c640 1 --2- 172.21.15.143:0/1463996959 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f2c500cf710 0x7f2c50137ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:19:58.962 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:58.959+0000 7f2c4ed9d640 1 -- 172.21.15.143:0/1463996959 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f2c50143ac0 msgr2=0x7f2c5013f930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:19:58.962 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:58.959+0000 7f2c4ed9d640 1 --2- 172.21.15.143:0/1463996959 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f2c50143ac0 0x7f2c5013f930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:19:58.962 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:58.959+0000 7f2c4ed9d640 1 -- 172.21.15.143:0/1463996959 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f2c500cf710 msgr2=0x7f2c50137ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:19:58.962 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:58.959+0000 7f2c4ed9d640 1 --2- 172.21.15.143:0/1463996959 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f2c500cf710 0x7f2c50137ee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:19:58.962 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:58.959+0000 7f2c4ed9d640 1 -- 172.21.15.143:0/1463996959 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2c48009a70 con 0x7f2c500cf280 2023-11-26T07:19:58.962 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:58.959+0000 7f2c4ed9d640 1 --2- 172.21.15.143:0/1463996959 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f2c500cf280 0x7f2c50141400 secure :-1 s=READY pgs=660 cs=0 l=1 rev1=1 crypto rx=0x7f2c380010f0 tx=0x7f2c3800c8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:19:58.962 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:58.959+0000 7f2c47fff640 1 -- 172.21.15.143:0/1463996959 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2c38005cf0 con 0x7f2c500cf280 2023-11-26T07:19:58.963 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:58.959+0000 7f2c47fff640 1 -- 172.21.15.143:0/1463996959 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2c38005e70 con 0x7f2c500cf280 2023-11-26T07:19:58.963 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:58.959+0000 7f2c551f5640 1 -- 172.21.15.143:0/1463996959 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2c50140920 con 0x7f2c500cf280 2023-11-26T07:19:58.963 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:58.959+0000 7f2c47fff640 1 -- 172.21.15.143:0/1463996959 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2c380072d0 con 0x7f2c500cf280 2023-11-26T07:19:58.963 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:58.959+0000 7f2c551f5640 1 -- 172.21.15.143:0/1463996959 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2c50138420 con 0x7f2c500cf280 2023-11-26T07:19:58.963 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:58.959+0000 7f2c551f5640 1 -- 172.21.15.143:0/1463996959 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2c50067680 con 0x7f2c500cf280 2023-11-26T07:19:58.963 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:58.959+0000 7f2c47fff640 1 -- 172.21.15.143:0/1463996959 <== mon.0 v2:172.21.15.143:3300/0 4 ==== mgrmap(e 7) v1 ==== 86005+0+0 (secure 0 0 0) 0x7f2c38007430 con 0x7f2c500cf280 2023-11-26T07:19:58.964 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:58.959+0000 7f2c47fff640 1 --2- 172.21.15.143:0/1463996959 >> [v2:172.21.15.143:6832/2150557544,v1:172.21.15.143:6833/2150557544] conn(0x7f2c30066650 0x7f2c30068b10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:19:58.964 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:58.963+0000 7f2c47fff640 1 -- 172.21.15.143:0/1463996959 <== mon.0 v2:172.21.15.143:3300/0 5 ==== osd_map(45..45 src has 1..45) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f2c38083870 con 0x7f2c500cf280 2023-11-26T07:19:58.965 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:58.963+0000 7f2c4659c640 1 --2- 172.21.15.143:0/1463996959 >> [v2:172.21.15.143:6832/2150557544,v1:172.21.15.143:6833/2150557544] conn(0x7f2c30066650 0x7f2c30068b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:19:58.965 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:58.963+0000 7f2c4659c640 1 --2- 172.21.15.143:0/1463996959 >> [v2:172.21.15.143:6832/2150557544,v1:172.21.15.143:6833/2150557544] conn(0x7f2c30066650 0x7f2c30068b10 secure :-1 s=READY pgs=320 cs=0 l=1 rev1=1 crypto rx=0x7f2c5013d1b0 tx=0x7f2c40009040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:19:58.968 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:58.963+0000 7f2c47fff640 1 -- 172.21.15.143:0/1463996959 <== mon.0 v2:172.21.15.143:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172649 (secure 0 0 0) 0x7f2c380509d0 con 0x7f2c500cf280 2023-11-26T07:19:59.119 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:59.115+0000 7f2c551f5640 1 -- 172.21.15.143:0/1463996959 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f2c5006d1f0 con 0x7f2c500cf280 2023-11-26T07:19:59.204 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:59.115+0000 7f2c47fff640 1 -- 172.21.15.143:0/1463996959 <== mon.0 v2:172.21.15.143:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v45) v1 ==== 93+0+10 (secure 0 0 0) 0x7f2c38053ff0 con 0x7f2c500cf280 2023-11-26T07:19:59.205 INFO:teuthology.orchestra.run.smithi143.stdout:pg_num: 1 2023-11-26T07:19:59.208 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:59.119+0000 7f2c551f5640 1 -- 172.21.15.143:0/1463996959 >> [v2:172.21.15.143:6832/2150557544,v1:172.21.15.143:6833/2150557544] conn(0x7f2c30066650 msgr2=0x7f2c30068b10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:19:59.208 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:59.119+0000 7f2c551f5640 1 --2- 172.21.15.143:0/1463996959 >> [v2:172.21.15.143:6832/2150557544,v1:172.21.15.143:6833/2150557544] conn(0x7f2c30066650 0x7f2c30068b10 secure :-1 s=READY pgs=320 cs=0 l=1 rev1=1 crypto rx=0x7f2c5013d1b0 tx=0x7f2c40009040 comp rx=0 tx=0).stop 2023-11-26T07:19:59.208 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:59.119+0000 7f2c551f5640 1 -- 172.21.15.143:0/1463996959 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f2c500cf280 msgr2=0x7f2c50141400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:19:59.208 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:59.119+0000 7f2c551f5640 1 --2- 172.21.15.143:0/1463996959 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f2c500cf280 0x7f2c50141400 secure :-1 s=READY pgs=660 cs=0 l=1 rev1=1 crypto rx=0x7f2c380010f0 tx=0x7f2c3800c8f0 comp rx=0 tx=0).stop 2023-11-26T07:19:59.208 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:59.119+0000 7f2c551f5640 1 -- 172.21.15.143:0/1463996959 shutdown_connections 2023-11-26T07:19:59.209 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:59.119+0000 7f2c551f5640 1 --2- 172.21.15.143:0/1463996959 >> [v2:172.21.15.143:6832/2150557544,v1:172.21.15.143:6833/2150557544] conn(0x7f2c30066650 0x7f2c30068b10 unknown :-1 s=CLOSED pgs=320 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:19:59.209 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:59.119+0000 7f2c551f5640 1 --2- 172.21.15.143:0/1463996959 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f2c50143ac0 0x7f2c5013f930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:19:59.209 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:59.119+0000 7f2c551f5640 1 --2- 172.21.15.143:0/1463996959 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f2c500cf710 0x7f2c50137ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:19:59.209 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:59.119+0000 7f2c551f5640 1 --2- 172.21.15.143:0/1463996959 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f2c500cf280 0x7f2c50141400 unknown :-1 s=CLOSED pgs=660 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:19:59.209 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:59.119+0000 7f2c551f5640 1 -- 172.21.15.143:0/1463996959 >> 172.21.15.143:0/1463996959 conn(0x7f2c5006a890 msgr2=0x7f2c5006ac70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:19:59.210 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:59.119+0000 7f2c551f5640 1 -- 172.21.15.143:0/1463996959 shutdown_connections 2023-11-26T07:19:59.210 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:59.119+0000 7f2c551f5640 1 -- 172.21.15.143:0/1463996959 wait complete. 2023-11-26T07:19:59.210 DEBUG:teuthology.orchestra.run.smithi143:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-11-26T07:19:59.351 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:59.347+0000 7f29162fe640 1 Processor -- start 2023-11-26T07:19:59.351 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:59.347+0000 7f29162fe640 1 -- start start 2023-11-26T07:19:59.353 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:59.347+0000 7f29162fe640 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7f29100b2f80 0x7f2910071850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:19:59.353 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:59.347+0000 7f29162fe640 1 -- --> v1:172.21.15.143:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f291006c750 con 0x7f29101596d0 2023-11-26T07:19:59.353 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:59.347+0000 7f29162fe640 1 -- --> v1:172.21.15.188:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f29100b3550 con 0x7f29100ce680 2023-11-26T07:19:59.353 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:59.347+0000 7f29162fe640 1 -- --> v2:172.21.15.188:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f29100cbe00 con 0x7f29100b2f80 2023-11-26T07:19:59.353 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:59.347+0000 7f290ffff640 1 --1- >> v1:172.21.15.143:6789/0 conn(0x7f29101596d0 0x7f29100b2870 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.143:6789/0 says I am v1:172.21.15.143:47600/0 (socket says 172.21.15.143:47600) 2023-11-26T07:19:59.353 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:59.347+0000 7f290ffff640 1 -- 172.21.15.143:0/1373486213 learned_addr learned my addr 172.21.15.143:0/1373486213 (peer_addr_for_me v1:172.21.15.143:0/0) 2023-11-26T07:19:59.353 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:59.351+0000 7f290f7fe640 1 --2- 172.21.15.143:0/1373486213 >> v2:172.21.15.188:3300/0 conn(0x7f29100b2f80 0x7f2910071850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:19:59.353 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:59.351+0000 7f290effd640 1 -- 172.21.15.143:0/1373486213 <== mon.0 v1:172.21.15.143:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3580765366 0 0) 0x7f291006c750 con 0x7f29101596d0 2023-11-26T07:19:59.353 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:59.351+0000 7f290effd640 1 -- 172.21.15.143:0/1373486213 --> v1:172.21.15.143:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f28f8003530 con 0x7f29101596d0 2023-11-26T07:19:59.355 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:59.351+0000 7f290effd640 1 -- 172.21.15.143:0/1373486213 <== mon.0 v1:172.21.15.143:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 300807041 0 0) 0x7f28f8003530 con 0x7f29101596d0 2023-11-26T07:19:59.355 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:59.351+0000 7f290effd640 1 -- 172.21.15.143:0/1373486213 >> v2:172.21.15.188:3300/0 conn(0x7f29100b2f80 msgr2=0x7f2910071850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-26T07:19:59.355 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:59.351+0000 7f290effd640 1 --2- 172.21.15.143:0/1373486213 >> v2:172.21.15.188:3300/0 conn(0x7f29100b2f80 0x7f2910071850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:19:59.356 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:59.351+0000 7f290effd640 1 -- 172.21.15.143:0/1373486213 >> v1:172.21.15.188:6789/0 conn(0x7f29100ce680 legacy=0x7f291006cbe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:19:59.356 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:59.351+0000 7f290effd640 1 -- 172.21.15.143:0/1373486213 --> v1:172.21.15.143:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2910071e80 con 0x7f29101596d0 2023-11-26T07:19:59.356 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:59.351+0000 7f290effd640 1 -- 172.21.15.143:0/1373486213 <== mon.0 v1:172.21.15.143:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2137882523 0 0) 0x7f2900001680 con 0x7f29101596d0 2023-11-26T07:19:59.356 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:59.351+0000 7f290effd640 1 -- 172.21.15.143:0/1373486213 >> v1:172.21.15.143:6789/0 conn(0x7f29101596d0 legacy=0x7f29100b2870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:19:59.356 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:59.351+0000 7f290effd640 1 --2- 172.21.15.143:0/1373486213 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f28f80039d0 0x7f28f8003e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:19:59.356 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:59.351+0000 7f290effd640 1 --2- 172.21.15.143:0/1373486213 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f28f80047d0 0x7f28f8004c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:19:59.356 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:59.351+0000 7f290effd640 1 --2- 172.21.15.143:0/1373486213 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f28f8005200 0x7f28f800b990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:19:59.356 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:59.351+0000 7f290effd640 1 -- 172.21.15.143:0/1373486213 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f29100cbe00 con 0x7f28f8005200 2023-11-26T07:19:59.357 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:59.351+0000 7f290effd640 1 -- 172.21.15.143:0/1373486213 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f28f800c160 con 0x7f28f80047d0 2023-11-26T07:19:59.357 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:59.351+0000 7f290effd640 1 -- 172.21.15.143:0/1373486213 --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f28f800c320 con 0x7f28f80039d0 2023-11-26T07:19:59.357 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:59.351+0000 7f290f7fe640 1 --2- 172.21.15.143:0/1373486213 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f28f80047d0 0x7f28f8004c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:19:59.357 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:59.351+0000 7f290ffff640 1 --2- 172.21.15.143:0/1373486213 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f28f80039d0 0x7f28f8003e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:19:59.357 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:59.351+0000 7f291489b640 1 --2- 172.21.15.143:0/1373486213 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f28f8005200 0x7f28f800b990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:19:59.357 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:59.351+0000 7f291489b640 1 -- 172.21.15.143:0/1373486213 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f28f80039d0 msgr2=0x7f28f8003e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-26T07:19:59.357 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:59.351+0000 7f291489b640 1 --2- 172.21.15.143:0/1373486213 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f28f80039d0 0x7f28f8003e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:19:59.357 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:59.351+0000 7f291489b640 1 -- 172.21.15.143:0/1373486213 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f28f80047d0 msgr2=0x7f28f8004c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-26T07:19:59.358 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:59.351+0000 7f291489b640 1 --2- 172.21.15.143:0/1373486213 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f28f80047d0 0x7f28f8004c50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:19:59.358 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:59.351+0000 7f291489b640 1 -- 172.21.15.143:0/1373486213 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f28f8003530 con 0x7f28f8005200 2023-11-26T07:19:59.358 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:59.351+0000 7f290f7fe640 1 --2- 172.21.15.143:0/1373486213 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f28f80047d0 0x7f28f8004c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-26T07:19:59.359 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:59.355+0000 7f291489b640 1 -- 172.21.15.143:0/1373486213 reap_dead start 2023-11-26T07:19:59.360 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:59.355+0000 7f291489b640 1 --2- 172.21.15.143:0/1373486213 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f28f8005200 0x7f28f800b990 secure :-1 s=READY pgs=662 cs=0 l=1 rev1=1 crypto rx=0x7f2908001e70 tx=0x7f290801c4e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=de30a70a9f8e5e95 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:19:59.360 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:59.355+0000 7f290effd640 1 -- 172.21.15.143:0/1373486213 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2908003b80 con 0x7f28f8005200 2023-11-26T07:19:59.360 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:59.355+0000 7f290effd640 1 -- 172.21.15.143:0/1373486213 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2908003d20 con 0x7f28f8005200 2023-11-26T07:19:59.360 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:59.355+0000 7f290effd640 1 -- 172.21.15.143:0/1373486213 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2908005310 con 0x7f28f8005200 2023-11-26T07:19:59.360 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:59.355+0000 7f29162fe640 1 -- 172.21.15.143:0/1373486213 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f28f8005200 msgr2=0x7f28f800b990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:19:59.360 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:59.355+0000 7f29162fe640 1 --2- 172.21.15.143:0/1373486213 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f28f8005200 0x7f28f800b990 secure :-1 s=READY pgs=662 cs=0 l=1 rev1=1 crypto rx=0x7f2908001e70 tx=0x7f290801c4e0 comp rx=0 tx=0).stop 2023-11-26T07:19:59.360 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:59.355+0000 7f29162fe640 1 -- 172.21.15.143:0/1373486213 shutdown_connections 2023-11-26T07:19:59.360 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:59.355+0000 7f29162fe640 1 --2- 172.21.15.143:0/1373486213 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f28f8005200 0x7f28f800b990 unknown :-1 s=CLOSED pgs=662 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:19:59.361 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:59.355+0000 7f29162fe640 1 -- 172.21.15.143:0/1373486213 >> 172.21.15.143:0/1373486213 conn(0x7f291006a890 msgr2=0x7f291006aca0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:19:59.361 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:59.355+0000 7f29162fe640 1 -- 172.21.15.143:0/1373486213 shutdown_connections 2023-11-26T07:19:59.361 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:59.355+0000 7f29162fe640 1 -- 172.21.15.143:0/1373486213 wait complete. 2023-11-26T07:19:59.361 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:59.355+0000 7f29162fe640 1 Processor -- start 2023-11-26T07:19:59.361 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:59.355+0000 7f29162fe640 1 -- start start 2023-11-26T07:19:59.367 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:59.359+0000 7f29162fe640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f28f8005200 0x7f2910158330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:19:59.367 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:59.359+0000 7f29162fe640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f29101564a0 0x7f291014ced0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:19:59.367 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:59.359+0000 7f29162fe640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f2910156920 0x7f291014d410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:19:59.367 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:59.359+0000 7f29162fe640 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2910156eb0 con 0x7f2910156920 2023-11-26T07:19:59.368 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:59.359+0000 7f29162fe640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2910157190 con 0x7f29101564a0 2023-11-26T07:19:59.368 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:59.359+0000 7f29162fe640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2910157470 con 0x7f28f8005200 2023-11-26T07:19:59.368 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:59.359+0000 7f291489b640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f2910156920 0x7f291014d410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:19:59.368 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:59.359+0000 7f291489b640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f2910156920 0x7f291014d410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:35208/0 (socket says 172.21.15.143:35208) 2023-11-26T07:19:59.368 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:59.359+0000 7f291489b640 1 -- 172.21.15.143:0/4135680982 learned_addr learned my addr 172.21.15.143:0/4135680982 (peer_addr_for_me v2:172.21.15.143:0/0) 2023-11-26T07:19:59.368 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:59.359+0000 7f290f7fe640 1 --2- 172.21.15.143:0/4135680982 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f29101564a0 0x7f291014ced0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:19:59.368 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:59.359+0000 7f290ffff640 1 --2- 172.21.15.143:0/4135680982 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f28f8005200 0x7f2910158330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:19:59.368 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:59.359+0000 7f291489b640 1 -- 172.21.15.143:0/4135680982 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f28f8005200 msgr2=0x7f2910158330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:19:59.368 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:59.359+0000 7f291489b640 1 --2- 172.21.15.143:0/4135680982 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f28f8005200 0x7f2910158330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:19:59.368 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:59.359+0000 7f291489b640 1 -- 172.21.15.143:0/4135680982 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f29101564a0 msgr2=0x7f291014ced0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:19:59.369 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:59.359+0000 7f291489b640 1 --2- 172.21.15.143:0/4135680982 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f29101564a0 0x7f291014ced0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:19:59.369 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:59.359+0000 7f291489b640 1 -- 172.21.15.143:0/4135680982 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2910157750 con 0x7f2910156920 2023-11-26T07:19:59.369 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:59.359+0000 7f291489b640 1 --2- 172.21.15.143:0/4135680982 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f2910156920 0x7f291014d410 secure :-1 s=READY pgs=663 cs=0 l=1 rev1=1 crypto rx=0x7f290801c310 tx=0x7f28f8005800 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:19:59.369 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:59.359+0000 7f290cff9640 1 -- 172.21.15.143:0/4135680982 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2908006d60 con 0x7f2910156920 2023-11-26T07:19:59.369 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:59.359+0000 7f290cff9640 1 -- 172.21.15.143:0/4135680982 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f29080050d0 con 0x7f2910156920 2023-11-26T07:19:59.369 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:59.359+0000 7f290cff9640 1 -- 172.21.15.143:0/4135680982 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2908005480 con 0x7f2910156920 2023-11-26T07:19:59.369 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:59.359+0000 7f29162fe640 1 -- 172.21.15.143:0/4135680982 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2910157a30 con 0x7f2910156920 2023-11-26T07:19:59.369 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:59.359+0000 7f29162fe640 1 -- 172.21.15.143:0/4135680982 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f291015bfb0 con 0x7f2910156920 2023-11-26T07:19:59.369 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:59.359+0000 7f29162fe640 1 -- 172.21.15.143:0/4135680982 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2910157c20 con 0x7f2910156920 2023-11-26T07:19:59.369 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:59.359+0000 7f290cff9640 1 -- 172.21.15.143:0/4135680982 <== mon.0 v2:172.21.15.143:3300/0 4 ==== mgrmap(e 7) v1 ==== 86005+0+0 (secure 0 0 0) 0x7f29080055e0 con 0x7f2910156920 2023-11-26T07:19:59.369 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:59.363+0000 7f290cff9640 1 --2- 172.21.15.143:0/4135680982 >> [v2:172.21.15.143:6832/2150557544,v1:172.21.15.143:6833/2150557544] conn(0x7f28fc066640 0x7f28fc068b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:19:59.370 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:59.363+0000 7f290cff9640 1 -- 172.21.15.143:0/4135680982 <== mon.0 v2:172.21.15.143:3300/0 5 ==== osd_map(45..45 src has 1..45) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f290811cd80 con 0x7f2910156920 2023-11-26T07:19:59.370 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:59.363+0000 7f290ffff640 1 --2- 172.21.15.143:0/4135680982 >> [v2:172.21.15.143:6832/2150557544,v1:172.21.15.143:6833/2150557544] conn(0x7f28fc066640 0x7f28fc068b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:19:59.370 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:59.363+0000 7f290ffff640 1 --2- 172.21.15.143:0/4135680982 >> [v2:172.21.15.143:6832/2150557544,v1:172.21.15.143:6833/2150557544] conn(0x7f28fc066640 0x7f28fc068b00 secure :-1 s=READY pgs=321 cs=0 l=1 rev1=1 crypto rx=0x7f2900001060 tx=0x7f2900001a10 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:19:59.370 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:59.367+0000 7f290cff9640 1 -- 172.21.15.143:0/4135680982 <== mon.0 v2:172.21.15.143:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172649 (secure 0 0 0) 0x7f29080e9e90 con 0x7f2910156920 2023-11-26T07:19:59.519 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:59.515+0000 7f29162fe640 1 -- 172.21.15.143:0/4135680982 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f2910067680 con 0x7f2910156920 2023-11-26T07:19:59.520 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:59.515+0000 7f290cff9640 1 -- 172.21.15.143:0/4135680982 <== mon.0 v2:172.21.15.143:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v45) v1 ==== 104+0+11 (secure 0 0 0) 0x7f29080b90a0 con 0x7f2910156920 2023-11-26T07:19:59.520 INFO:teuthology.orchestra.run.smithi143.stdout:pg_num: 32 2023-11-26T07:19:59.522 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:59.519+0000 7f29162fe640 1 -- 172.21.15.143:0/4135680982 >> [v2:172.21.15.143:6832/2150557544,v1:172.21.15.143:6833/2150557544] conn(0x7f28fc066640 msgr2=0x7f28fc068b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:19:59.522 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:59.519+0000 7f29162fe640 1 --2- 172.21.15.143:0/4135680982 >> [v2:172.21.15.143:6832/2150557544,v1:172.21.15.143:6833/2150557544] conn(0x7f28fc066640 0x7f28fc068b00 secure :-1 s=READY pgs=321 cs=0 l=1 rev1=1 crypto rx=0x7f2900001060 tx=0x7f2900001a10 comp rx=0 tx=0).stop 2023-11-26T07:19:59.523 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:59.519+0000 7f29162fe640 1 -- 172.21.15.143:0/4135680982 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f2910156920 msgr2=0x7f291014d410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:19:59.523 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:59.519+0000 7f29162fe640 1 --2- 172.21.15.143:0/4135680982 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f2910156920 0x7f291014d410 secure :-1 s=READY pgs=663 cs=0 l=1 rev1=1 crypto rx=0x7f290801c310 tx=0x7f28f8005800 comp rx=0 tx=0).stop 2023-11-26T07:19:59.523 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:59.519+0000 7f29162fe640 1 -- 172.21.15.143:0/4135680982 shutdown_connections 2023-11-26T07:19:59.523 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:59.519+0000 7f29162fe640 1 --2- 172.21.15.143:0/4135680982 >> [v2:172.21.15.143:6832/2150557544,v1:172.21.15.143:6833/2150557544] conn(0x7f28fc066640 0x7f28fc068b00 unknown :-1 s=CLOSED pgs=321 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:19:59.523 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:59.519+0000 7f29162fe640 1 --2- 172.21.15.143:0/4135680982 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f2910156920 0x7f291014d410 unknown :-1 s=CLOSED pgs=663 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:19:59.523 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:59.519+0000 7f29162fe640 1 --2- 172.21.15.143:0/4135680982 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f29101564a0 0x7f291014ced0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:19:59.523 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:59.519+0000 7f29162fe640 1 --2- 172.21.15.143:0/4135680982 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f28f8005200 0x7f2910158330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:19:59.523 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:59.519+0000 7f29162fe640 1 -- 172.21.15.143:0/4135680982 >> 172.21.15.143:0/4135680982 conn(0x7f291006a890 msgr2=0x7f2910154990 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:19:59.523 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:59.519+0000 7f29162fe640 1 -- 172.21.15.143:0/4135680982 shutdown_connections 2023-11-26T07:19:59.523 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:59.519+0000 7f29162fe640 1 -- 172.21.15.143:0/4135680982 wait complete. 2023-11-26T07:19:59.542 DEBUG:teuthology.orchestra.run.smithi143:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-11-26T07:19:59.682 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:59.679+0000 7f9afb557640 1 Processor -- start 2023-11-26T07:19:59.682 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:59.679+0000 7f9afb557640 1 -- start start 2023-11-26T07:19:59.684 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:59.679+0000 7f9afb557640 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7f9af415d580 0x7f9af40d3920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:19:59.684 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:59.679+0000 7f9afb557640 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7f9af40d3e60 0x7f9af40d17f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:19:59.684 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:59.679+0000 7f9afb557640 1 -- --> v1:172.21.15.143:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9af40d21e0 con 0x7f9af40d1d30 2023-11-26T07:19:59.684 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:59.679+0000 7f9afb557640 1 -- --> v2:172.21.15.143:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9af40d0bc0 con 0x7f9af415d580 2023-11-26T07:19:59.684 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:59.679+0000 7f9afb557640 1 -- --> v2:172.21.15.188:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9af40d0320 con 0x7f9af40d3e60 2023-11-26T07:19:59.685 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:59.679+0000 7f9af92f3640 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7f9af415d580 0x7f9af40d3920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:19:59.685 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:59.679+0000 7f9af9af4640 1 --1- >> v1:172.21.15.143:6789/0 conn(0x7f9af40d1d30 0x7f9af40cdce0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.143:6789/0 says I am v1:172.21.15.143:47610/0 (socket says 172.21.15.143:47610) 2023-11-26T07:19:59.685 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:59.679+0000 7f9af9af4640 1 -- 172.21.15.143:0/2166525460 learned_addr learned my addr 172.21.15.143:0/2166525460 (peer_addr_for_me v1:172.21.15.143:0/0) 2023-11-26T07:19:59.685 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:59.679+0000 7f9af8af2640 1 --2- 172.21.15.143:0/2166525460 >> v2:172.21.15.188:3300/0 conn(0x7f9af40d3e60 0x7f9af40d17f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:19:59.685 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:59.679+0000 7f9af92f3640 1 -- 172.21.15.143:0/2166525460 >> v2:172.21.15.188:3300/0 conn(0x7f9af40d3e60 msgr2=0x7f9af40d17f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-26T07:19:59.685 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:59.679+0000 7f9af92f3640 1 --2- 172.21.15.143:0/2166525460 >> v2:172.21.15.188:3300/0 conn(0x7f9af40d3e60 0x7f9af40d17f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:19:59.685 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:59.679+0000 7f9af92f3640 1 -- 172.21.15.143:0/2166525460 >> v1:172.21.15.143:6789/0 conn(0x7f9af40d1d30 legacy=0x7f9af40cdce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:19:59.685 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:59.679+0000 7f9af92f3640 1 -- 172.21.15.143:0/2166525460 --> v2:172.21.15.143:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9af40d0600 con 0x7f9af415d580 2023-11-26T07:19:59.685 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:59.679+0000 7f9aebfff640 1 -- 172.21.15.143:0/2166525460 <== mon.0 v1:172.21.15.143:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3963257038 0 0) 0x7f9af40d21e0 con 0x7f9af40d1d30 2023-11-26T07:19:59.685 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:59.679+0000 7f9af8af2640 1 --2- 172.21.15.143:0/2166525460 >> v2:172.21.15.188:3300/0 conn(0x7f9af40d3e60 0x7f9af40d17f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-26T07:19:59.687 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:59.683+0000 7f9af92f3640 1 --2- 172.21.15.143:0/2166525460 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f9af415d580 0x7f9af40d3920 secure :-1 s=READY pgs=665 cs=0 l=1 rev1=1 crypto rx=0x7f9af40a1190 tx=0x7f9af001c430 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8d6a9f007031943f server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:19:59.687 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:59.683+0000 7f9aebfff640 1 -- 172.21.15.143:0/2166525460 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9af00a7070 con 0x7f9af415d580 2023-11-26T07:19:59.687 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:59.683+0000 7f9aebfff640 1 -- 172.21.15.143:0/2166525460 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9af00ab430 con 0x7f9af415d580 2023-11-26T07:19:59.687 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:59.683+0000 7f9aebfff640 1 -- 172.21.15.143:0/2166525460 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9af00ab7c0 con 0x7f9af415d580 2023-11-26T07:19:59.688 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:59.683+0000 7f9afb557640 1 -- 172.21.15.143:0/2166525460 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f9af415d580 msgr2=0x7f9af40d3920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:19:59.688 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:59.683+0000 7f9afb557640 1 --2- 172.21.15.143:0/2166525460 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f9af415d580 0x7f9af40d3920 secure :-1 s=READY pgs=665 cs=0 l=1 rev1=1 crypto rx=0x7f9af40a1190 tx=0x7f9af001c430 comp rx=0 tx=0).stop 2023-11-26T07:19:59.688 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:59.683+0000 7f9afb557640 1 -- 172.21.15.143:0/2166525460 shutdown_connections 2023-11-26T07:19:59.688 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:59.683+0000 7f9afb557640 1 --2- 172.21.15.143:0/2166525460 >> v2:172.21.15.188:3300/0 conn(0x7f9af40d3e60 0x7f9af40d17f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:19:59.689 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:59.683+0000 7f9afb557640 1 --2- 172.21.15.143:0/2166525460 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f9af415d580 0x7f9af40d3920 unknown :-1 s=CLOSED pgs=665 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:19:59.689 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:59.683+0000 7f9afb557640 1 -- 172.21.15.143:0/2166525460 >> 172.21.15.143:0/2166525460 conn(0x7f9af409c550 msgr2=0x7f9af40d3210 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:19:59.689 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:59.683+0000 7f9afb557640 1 -- 172.21.15.143:0/2166525460 shutdown_connections 2023-11-26T07:19:59.689 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:59.683+0000 7f9afb557640 1 -- 172.21.15.143:0/2166525460 wait complete. 2023-11-26T07:19:59.690 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:59.687+0000 7f9afb557640 1 Processor -- start 2023-11-26T07:19:59.690 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:59.687+0000 7f9afb557640 1 -- start start 2023-11-26T07:19:59.692 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:59.687+0000 7f9afb557640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f9af40d1d30 0x7f9af4155860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:19:59.692 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:59.687+0000 7f9afb557640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f9af40d3e60 0x7f9af40b8870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:19:59.692 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:59.687+0000 7f9afb557640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f9af415d580 0x7f9af4156630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:19:59.692 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:59.687+0000 7f9afb557640 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9af4155da0 con 0x7f9af415d580 2023-11-26T07:19:59.692 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:59.687+0000 7f9afb557640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9af40b4560 con 0x7f9af40d3e60 2023-11-26T07:19:59.692 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:59.687+0000 7f9afb557640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9af415e7d0 con 0x7f9af40d1d30 2023-11-26T07:19:59.692 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:59.687+0000 7f9af9af4640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f9af415d580 0x7f9af4156630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:19:59.692 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:59.687+0000 7f9af9af4640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f9af415d580 0x7f9af4156630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:35220/0 (socket says 172.21.15.143:35220) 2023-11-26T07:19:59.692 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:59.687+0000 7f9af9af4640 1 -- 172.21.15.143:0/2237236997 learned_addr learned my addr 172.21.15.143:0/2237236997 (peer_addr_for_me v2:172.21.15.143:0/0) 2023-11-26T07:19:59.692 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:59.687+0000 7f9af92f3640 1 --2- 172.21.15.143:0/2237236997 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f9af40d1d30 0x7f9af4155860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:19:59.693 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:59.687+0000 7f9af8af2640 1 --2- 172.21.15.143:0/2237236997 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f9af40d3e60 0x7f9af40b8870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:19:59.693 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:59.687+0000 7f9af9af4640 1 -- 172.21.15.143:0/2237236997 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f9af40d1d30 msgr2=0x7f9af4155860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:19:59.693 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:59.687+0000 7f9af9af4640 1 --2- 172.21.15.143:0/2237236997 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f9af40d1d30 0x7f9af4155860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:19:59.693 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:59.687+0000 7f9af9af4640 1 -- 172.21.15.143:0/2237236997 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f9af40d3e60 msgr2=0x7f9af40b8870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:19:59.693 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:59.687+0000 7f9af9af4640 1 --2- 172.21.15.143:0/2237236997 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f9af40d3e60 0x7f9af40b8870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:19:59.693 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:59.687+0000 7f9af9af4640 1 -- 172.21.15.143:0/2237236997 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9af0009420 con 0x7f9af415d580 2023-11-26T07:19:59.693 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:59.687+0000 7f9af8af2640 1 --2- 172.21.15.143:0/2237236997 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f9af40d3e60 0x7f9af40b8870 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-26T07:19:59.693 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:59.687+0000 7f9af9af4640 1 --2- 172.21.15.143:0/2237236997 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f9af415d580 0x7f9af4156630 secure :-1 s=READY pgs=666 cs=0 l=1 rev1=1 crypto rx=0x7f9aec001570 tx=0x7f9aec00cca0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:19:59.693 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:59.687+0000 7f9ae9ffb640 1 -- 172.21.15.143:0/2237236997 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9aec00a640 con 0x7f9af415d580 2023-11-26T07:19:59.693 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:59.687+0000 7f9ae9ffb640 1 -- 172.21.15.143:0/2237236997 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9aec00a7c0 con 0x7f9af415d580 2023-11-26T07:19:59.693 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:59.687+0000 7f9afb557640 1 -- 172.21.15.143:0/2237236997 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9af40b3950 con 0x7f9af415d580 2023-11-26T07:19:59.693 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:59.687+0000 7f9ae9ffb640 1 -- 172.21.15.143:0/2237236997 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9aec00ab70 con 0x7f9af415d580 2023-11-26T07:19:59.694 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:59.687+0000 7f9afb557640 1 -- 172.21.15.143:0/2237236997 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9af40b3f10 con 0x7f9af415d580 2023-11-26T07:19:59.694 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:59.691+0000 7f9afb557640 1 -- 172.21.15.143:0/2237236997 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9af4067680 con 0x7f9af415d580 2023-11-26T07:19:59.694 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:59.691+0000 7f9ae9ffb640 1 -- 172.21.15.143:0/2237236997 <== mon.0 v2:172.21.15.143:3300/0 4 ==== mgrmap(e 7) v1 ==== 86005+0+0 (secure 0 0 0) 0x7f9aec00acd0 con 0x7f9af415d580 2023-11-26T07:19:59.695 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:59.691+0000 7f9ae9ffb640 1 --2- 172.21.15.143:0/2237236997 >> [v2:172.21.15.143:6832/2150557544,v1:172.21.15.143:6833/2150557544] conn(0x7f9ad8066620 0x7f9ad8068ae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:19:59.695 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:59.691+0000 7f9ae9ffb640 1 -- 172.21.15.143:0/2237236997 <== mon.0 v2:172.21.15.143:3300/0 5 ==== osd_map(45..45 src has 1..45) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f9aec083a60 con 0x7f9af415d580 2023-11-26T07:19:59.695 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:59.691+0000 7f9af92f3640 1 --2- 172.21.15.143:0/2237236997 >> [v2:172.21.15.143:6832/2150557544,v1:172.21.15.143:6833/2150557544] conn(0x7f9ad8066620 0x7f9ad8068ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:19:59.696 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:59.691+0000 7f9af92f3640 1 --2- 172.21.15.143:0/2237236997 >> [v2:172.21.15.143:6832/2150557544,v1:172.21.15.143:6833/2150557544] conn(0x7f9ad8066620 0x7f9ad8068ae0 secure :-1 s=READY pgs=322 cs=0 l=1 rev1=1 crypto rx=0x7f9af00af000 tx=0x7f9af00a7000 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:19:59.699 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:59.695+0000 7f9ae9ffb640 1 -- 172.21.15.143:0/2237236997 <== mon.0 v2:172.21.15.143:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172649 (secure 0 0 0) 0x7f9aec050c60 con 0x7f9af415d580 2023-11-26T07:19:59.849 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:59.847+0000 7f9afb557640 1 -- 172.21.15.143:0/2237236997 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f9af40b5ad0 con 0x7f9af415d580 2023-11-26T07:19:59.849 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:59.847+0000 7f9ae9ffb640 1 -- 172.21.15.143:0/2237236997 <== mon.0 v2:172.21.15.143:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v45) v1 ==== 100+0+11 (secure 0 0 0) 0x7f9aec018020 con 0x7f9af415d580 2023-11-26T07:19:59.849 INFO:teuthology.orchestra.run.smithi143.stdout:pg_num: 64 2023-11-26T07:19:59.851 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:59.847+0000 7f9afb557640 1 -- 172.21.15.143:0/2237236997 >> [v2:172.21.15.143:6832/2150557544,v1:172.21.15.143:6833/2150557544] conn(0x7f9ad8066620 msgr2=0x7f9ad8068ae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:19:59.851 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:59.847+0000 7f9afb557640 1 --2- 172.21.15.143:0/2237236997 >> [v2:172.21.15.143:6832/2150557544,v1:172.21.15.143:6833/2150557544] conn(0x7f9ad8066620 0x7f9ad8068ae0 secure :-1 s=READY pgs=322 cs=0 l=1 rev1=1 crypto rx=0x7f9af00af000 tx=0x7f9af00a7000 comp rx=0 tx=0).stop 2023-11-26T07:19:59.852 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:59.847+0000 7f9afb557640 1 -- 172.21.15.143:0/2237236997 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f9af415d580 msgr2=0x7f9af4156630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:19:59.852 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:59.847+0000 7f9afb557640 1 --2- 172.21.15.143:0/2237236997 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f9af415d580 0x7f9af4156630 secure :-1 s=READY pgs=666 cs=0 l=1 rev1=1 crypto rx=0x7f9aec001570 tx=0x7f9aec00cca0 comp rx=0 tx=0).stop 2023-11-26T07:19:59.852 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:59.847+0000 7f9afb557640 1 -- 172.21.15.143:0/2237236997 shutdown_connections 2023-11-26T07:19:59.852 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:59.847+0000 7f9afb557640 1 --2- 172.21.15.143:0/2237236997 >> [v2:172.21.15.143:6832/2150557544,v1:172.21.15.143:6833/2150557544] conn(0x7f9ad8066620 0x7f9ad8068ae0 unknown :-1 s=CLOSED pgs=322 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:19:59.852 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:59.847+0000 7f9afb557640 1 --2- 172.21.15.143:0/2237236997 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f9af415d580 0x7f9af4156630 unknown :-1 s=CLOSED pgs=666 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:19:59.852 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:59.847+0000 7f9afb557640 1 --2- 172.21.15.143:0/2237236997 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f9af40d3e60 0x7f9af40b8870 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:19:59.852 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:59.847+0000 7f9afb557640 1 --2- 172.21.15.143:0/2237236997 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f9af40d1d30 0x7f9af4155860 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:19:59.852 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:59.847+0000 7f9afb557640 1 -- 172.21.15.143:0/2237236997 >> 172.21.15.143:0/2237236997 conn(0x7f9af409c550 msgr2=0x7f9af40b53c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:19:59.852 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:59.847+0000 7f9afb557640 1 -- 172.21.15.143:0/2237236997 shutdown_connections 2023-11-26T07:19:59.852 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:19:59.847+0000 7f9afb557640 1 -- 172.21.15.143:0/2237236997 wait complete. 2023-11-26T07:19:59.870 DEBUG:teuthology.orchestra.run.smithi143:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2023-11-26T07:20:00.009 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:20:00.007+0000 7fccb1f72640 1 Processor -- start 2023-11-26T07:20:00.010 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:20:00.007+0000 7fccb1f72640 1 -- start start 2023-11-26T07:20:00.011 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:20:00.007+0000 7fccb1f72640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fccac143ac0 0x7fccac070370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:20:00.011 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:20:00.007+0000 7fccb1f72640 1 -- --> v1:172.21.15.143:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fccac0cc730 con 0x7fccac0708b0 2023-11-26T07:20:00.011 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:20:00.007+0000 7fccb1f72640 1 -- --> v1:172.21.15.188:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fccac070ee0 con 0x7fccac0cf990 2023-11-26T07:20:00.011 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:20:00.007+0000 7fccb1f72640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fccac143ea0 con 0x7fccac143ac0 2023-11-26T07:20:00.011 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:20:00.007+0000 7fccaaffd640 1 --1- >> v1:172.21.15.143:6789/0 conn(0x7fccac0708b0 0x7fccac0cf280 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.143:6789/0 says I am v1:172.21.15.143:47622/0 (socket says 172.21.15.143:47622) 2023-11-26T07:20:00.011 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:20:00.007+0000 7fccaaffd640 1 -- 172.21.15.143:0/3340084779 learned_addr learned my addr 172.21.15.143:0/3340084779 (peer_addr_for_me v1:172.21.15.143:0/0) 2023-11-26T07:20:00.011 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:20:00.007+0000 7fccab7fe640 1 --2- 172.21.15.143:0/3340084779 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fccac143ac0 0x7fccac070370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:20:00.012 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:20:00.007+0000 7fccaa7fc640 1 -- 172.21.15.143:0/3340084779 <== mon.0 v1:172.21.15.143:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2089192812 0 0) 0x7fccac0cc730 con 0x7fccac0708b0 2023-11-26T07:20:00.012 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:20:00.007+0000 7fccaa7fc640 1 -- 172.21.15.143:0/3340084779 --> v1:172.21.15.143:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fcc94003530 con 0x7fccac0708b0 2023-11-26T07:20:00.012 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:20:00.007+0000 7fccab7fe640 1 -- 172.21.15.143:0/3340084779 >> v1:172.21.15.188:6789/0 conn(0x7fccac0cf990 legacy=0x7fccac0cd1b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:20:00.012 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:20:00.007+0000 7fccab7fe640 1 -- 172.21.15.143:0/3340084779 >> v1:172.21.15.143:6789/0 conn(0x7fccac0708b0 legacy=0x7fccac0cf280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:20:00.012 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:20:00.007+0000 7fccab7fe640 1 -- 172.21.15.143:0/3340084779 --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fccac0711c0 con 0x7fccac143ac0 2023-11-26T07:20:00.015 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:20:00.011+0000 7fccab7fe640 1 --2- 172.21.15.143:0/3340084779 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fccac143ac0 0x7fccac070370 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7fcc9c0097f0 tx=0x7fcc9c01c4c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=89383a1ef0010242 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:20:00.015 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:20:00.011+0000 7fccaa7fc640 1 -- 172.21.15.143:0/3340084779 <== mon.2 v2:172.21.15.188:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcc9c0ac630 con 0x7fccac143ac0 2023-11-26T07:20:00.015 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:20:00.011+0000 7fccaa7fc640 1 -- 172.21.15.143:0/3340084779 <== mon.2 v2:172.21.15.188:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcc9c0ac790 con 0x7fccac143ac0 2023-11-26T07:20:00.015 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:20:00.011+0000 7fccaa7fc640 1 -- 172.21.15.143:0/3340084779 <== mon.2 v2:172.21.15.188:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcc9c0acb20 con 0x7fccac143ac0 2023-11-26T07:20:00.016 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:20:00.011+0000 7fccb1f72640 1 -- 172.21.15.143:0/3340084779 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fccac143ac0 msgr2=0x7fccac070370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:20:00.016 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:20:00.011+0000 7fccb1f72640 1 --2- 172.21.15.143:0/3340084779 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fccac143ac0 0x7fccac070370 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7fcc9c0097f0 tx=0x7fcc9c01c4c0 comp rx=0 tx=0).stop 2023-11-26T07:20:00.016 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:20:00.011+0000 7fccb1f72640 1 -- 172.21.15.143:0/3340084779 shutdown_connections 2023-11-26T07:20:00.016 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:20:00.011+0000 7fccb1f72640 1 --2- 172.21.15.143:0/3340084779 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fccac143ac0 0x7fccac070370 unknown :-1 s=CLOSED pgs=93 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:20:00.016 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:20:00.011+0000 7fccb1f72640 1 -- 172.21.15.143:0/3340084779 >> 172.21.15.143:0/3340084779 conn(0x7fccac06a890 msgr2=0x7fccac06aca0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:20:00.016 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:20:00.011+0000 7fccb1f72640 1 -- 172.21.15.143:0/3340084779 shutdown_connections 2023-11-26T07:20:00.016 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:20:00.011+0000 7fccb1f72640 1 -- 172.21.15.143:0/3340084779 wait complete. 2023-11-26T07:20:00.017 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:20:00.015+0000 7fccb1f72640 1 Processor -- start 2023-11-26T07:20:00.017 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:20:00.015+0000 7fccb1f72640 1 -- start start 2023-11-26T07:20:00.019 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:20:00.015+0000 7fccb1f72640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fccac0708b0 0x7fccac157ee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:20:00.020 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:20:00.015+0000 7fccb1f72640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fccac0cf990 0x7fccac141890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:20:00.020 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:20:00.015+0000 7fccb1f72640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fccac143ac0 0x7fccac13a370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:20:00.020 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:20:00.015+0000 7fccb1f72640 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fccac0a9460 con 0x7fccac0708b0 2023-11-26T07:20:00.020 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:20:00.015+0000 7fccb1f72640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fccac0af2d0 con 0x7fccac143ac0 2023-11-26T07:20:00.020 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:20:00.015+0000 7fccb1f72640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fccac158420 con 0x7fccac0cf990 2023-11-26T07:20:00.020 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:20:00.015+0000 7fccab7fe640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fccac0708b0 0x7fccac157ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:20:00.020 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:20:00.015+0000 7fccab7fe640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fccac0708b0 0x7fccac157ee0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:35224/0 (socket says 172.21.15.143:35224) 2023-11-26T07:20:00.020 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:20:00.015+0000 7fccab7fe640 1 -- 172.21.15.143:0/560367646 learned_addr learned my addr 172.21.15.143:0/560367646 (peer_addr_for_me v2:172.21.15.143:0/0) 2023-11-26T07:20:00.020 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:20:00.015+0000 7fccaaffd640 1 --2- 172.21.15.143:0/560367646 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fccac0cf990 0x7fccac141890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:20:00.020 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:20:00.015+0000 7fccabfff640 1 --2- 172.21.15.143:0/560367646 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fccac143ac0 0x7fccac13a370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:20:00.020 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:20:00.015+0000 7fccab7fe640 1 -- 172.21.15.143:0/560367646 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fccac0cf990 msgr2=0x7fccac141890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:20:00.020 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:20:00.015+0000 7fccab7fe640 1 --2- 172.21.15.143:0/560367646 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fccac0cf990 0x7fccac141890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:20:00.021 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:20:00.015+0000 7fccab7fe640 1 -- 172.21.15.143:0/560367646 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fccac143ac0 msgr2=0x7fccac13a370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:20:00.021 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:20:00.015+0000 7fccab7fe640 1 --2- 172.21.15.143:0/560367646 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fccac143ac0 0x7fccac13a370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:20:00.021 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:20:00.015+0000 7fccab7fe640 1 -- 172.21.15.143:0/560367646 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcc9c009420 con 0x7fccac0708b0 2023-11-26T07:20:00.021 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:20:00.015+0000 7fccabfff640 1 --2- 172.21.15.143:0/560367646 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fccac143ac0 0x7fccac13a370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-26T07:20:00.021 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:20:00.015+0000 7fccab7fe640 1 --2- 172.21.15.143:0/560367646 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fccac0708b0 0x7fccac157ee0 secure :-1 s=READY pgs=668 cs=0 l=1 rev1=1 crypto rx=0x7fcc9c0190d0 tx=0x7fcc9c01ad00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:20:00.022 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:20:00.015+0000 7fcc93fff640 1 -- 172.21.15.143:0/560367646 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcc9c0b8650 con 0x7fccac0708b0 2023-11-26T07:20:00.022 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:20:00.015+0000 7fcc93fff640 1 -- 172.21.15.143:0/560367646 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcc9c0b87b0 con 0x7fccac0708b0 2023-11-26T07:20:00.022 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:20:00.015+0000 7fccb1f72640 1 -- 172.21.15.143:0/560367646 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fccac0af520 con 0x7fccac0708b0 2023-11-26T07:20:00.022 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:20:00.015+0000 7fcc93fff640 1 -- 172.21.15.143:0/560367646 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcc9c0b8b60 con 0x7fccac0708b0 2023-11-26T07:20:00.022 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:20:00.015+0000 7fccb1f72640 1 -- 172.21.15.143:0/560367646 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fccac0afae0 con 0x7fccac0708b0 2023-11-26T07:20:00.022 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:20:00.019+0000 7fccb1f72640 1 -- 172.21.15.143:0/560367646 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fccac067680 con 0x7fccac0708b0 2023-11-26T07:20:00.022 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:20:00.019+0000 7fcc93fff640 1 -- 172.21.15.143:0/560367646 <== mon.0 v2:172.21.15.143:3300/0 4 ==== mgrmap(e 7) v1 ==== 86005+0+0 (secure 0 0 0) 0x7fcc9c0b8cc0 con 0x7fccac0708b0 2023-11-26T07:20:00.023 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:20:00.019+0000 7fcc93fff640 1 --2- 172.21.15.143:0/560367646 >> [v2:172.21.15.143:6832/2150557544,v1:172.21.15.143:6833/2150557544] conn(0x7fcc98066620 0x7fcc98068ae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:20:00.023 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:20:00.019+0000 7fcc93fff640 1 -- 172.21.15.143:0/560367646 <== mon.0 v2:172.21.15.143:3300/0 5 ==== osd_map(45..45 src has 1..45) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fcc9c11eb20 con 0x7fccac0708b0 2023-11-26T07:20:00.023 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:20:00.019+0000 7fccaaffd640 1 --2- 172.21.15.143:0/560367646 >> [v2:172.21.15.143:6832/2150557544,v1:172.21.15.143:6833/2150557544] conn(0x7fcc98066620 0x7fcc98068ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:20:00.024 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:20:00.019+0000 7fccaaffd640 1 --2- 172.21.15.143:0/560367646 >> [v2:172.21.15.143:6832/2150557544,v1:172.21.15.143:6833/2150557544] conn(0x7fcc98066620 0x7fcc98068ae0 secure :-1 s=READY pgs=323 cs=0 l=1 rev1=1 crypto rx=0x7fccac0ae590 tx=0x7fcca0009040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:20:00.027 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:20:00.023+0000 7fcc93fff640 1 -- 172.21.15.143:0/560367646 <== mon.0 v2:172.21.15.143:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172649 (secure 0 0 0) 0x7fcc9c0ebb70 con 0x7fccac0708b0 2023-11-26T07:20:00.188 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:20:00.183+0000 7fccb1f72640 1 -- 172.21.15.143:0/560367646 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) v1 -- 0x7fccac0725d0 con 0x7fccac0708b0 2023-11-26T07:20:00.188 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:20:00.183+0000 7fcc93fff640 1 -- 172.21.15.143:0/560367646 <== mon.0 v2:172.21.15.143:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v45) v1 ==== 80+0+0 (secure 0 0 0) 0x7fcc9c0a8080 con 0x7fccac0708b0 2023-11-26T07:20:00.188 INFO:teuthology.orchestra.run.smithi143.stderr:listed 0 entries 2023-11-26T07:20:00.190 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:20:00.187+0000 7fccb1f72640 1 -- 172.21.15.143:0/560367646 >> [v2:172.21.15.143:6832/2150557544,v1:172.21.15.143:6833/2150557544] conn(0x7fcc98066620 msgr2=0x7fcc98068ae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:20:00.190 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:20:00.187+0000 7fccb1f72640 1 --2- 172.21.15.143:0/560367646 >> [v2:172.21.15.143:6832/2150557544,v1:172.21.15.143:6833/2150557544] conn(0x7fcc98066620 0x7fcc98068ae0 secure :-1 s=READY pgs=323 cs=0 l=1 rev1=1 crypto rx=0x7fccac0ae590 tx=0x7fcca0009040 comp rx=0 tx=0).stop 2023-11-26T07:20:00.190 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:20:00.187+0000 7fccb1f72640 1 -- 172.21.15.143:0/560367646 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fccac0708b0 msgr2=0x7fccac157ee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:20:00.191 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:20:00.187+0000 7fccb1f72640 1 --2- 172.21.15.143:0/560367646 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fccac0708b0 0x7fccac157ee0 secure :-1 s=READY pgs=668 cs=0 l=1 rev1=1 crypto rx=0x7fcc9c0190d0 tx=0x7fcc9c01ad00 comp rx=0 tx=0).stop 2023-11-26T07:20:00.191 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:20:00.187+0000 7fccb1f72640 1 -- 172.21.15.143:0/560367646 shutdown_connections 2023-11-26T07:20:00.191 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:20:00.187+0000 7fccb1f72640 1 --2- 172.21.15.143:0/560367646 >> [v2:172.21.15.143:6832/2150557544,v1:172.21.15.143:6833/2150557544] conn(0x7fcc98066620 0x7fcc98068ae0 unknown :-1 s=CLOSED pgs=323 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:20:00.191 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:20:00.187+0000 7fccb1f72640 1 --2- 172.21.15.143:0/560367646 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fccac143ac0 0x7fccac13a370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:20:00.191 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:20:00.187+0000 7fccb1f72640 1 --2- 172.21.15.143:0/560367646 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fccac0cf990 0x7fccac141890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:20:00.191 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:20:00.187+0000 7fccb1f72640 1 --2- 172.21.15.143:0/560367646 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fccac0708b0 0x7fccac157ee0 unknown :-1 s=CLOSED pgs=668 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:20:00.191 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:20:00.187+0000 7fccb1f72640 1 -- 172.21.15.143:0/560367646 >> 172.21.15.143:0/560367646 conn(0x7fccac06a890 msgr2=0x7fccac06ac70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:20:00.191 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:20:00.187+0000 7fccb1f72640 1 -- 172.21.15.143:0/560367646 shutdown_connections 2023-11-26T07:20:00.191 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:20:00.187+0000 7fccb1f72640 1 -- 172.21.15.143:0/560367646 wait complete. 2023-11-26T07:20:00.209 INFO:tasks.cephfs.fuse_mount:Running fusermount -u on ubuntu@smithi188.front.sepia.ceph.com... 2023-11-26T07:20:00.209 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-26T07:20:00.209 DEBUG:teuthology.orchestra.run.smithi188:> sudo fusermount -u /home/ubuntu/cephtest/mnt.1 2023-11-26T07:20:00.228 INFO:tasks.cephfs.fuse_mount.ceph-fuse.1.smithi188.stderr:ceph-fuse[44538]: fuse finished with error 0 and tester_r 0 2023-11-26T07:20:00.231 INFO:teuthology.orchestra.run:waiting for 300 2023-11-26T07:20:06.333 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi188.front.sepia.ceph.com 2023-11-26T07:20:06.334 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-26T07:20:06.334 DEBUG:teuthology.orchestra.run.smithi188:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.1) 2023-11-26T07:20:06.344 DEBUG:teuthology.orchestra.run.smithi143:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-11-26T07:20:06.486 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:20:06.483+0000 7fd497679640 1 Processor -- start 2023-11-26T07:20:06.486 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:20:06.483+0000 7fd497679640 1 -- start start 2023-11-26T07:20:06.487 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:20:06.483+0000 7fd497679640 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7fd4900b2160 0x7fd4900d0df0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:20:06.487 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:20:06.483+0000 7fd497679640 1 -- --> v1:172.21.15.143:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd4900cf0e0 con 0x7fd49015d6e0 2023-11-26T07:20:06.487 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:20:06.483+0000 7fd497679640 1 -- --> v1:172.21.15.188:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd4900ceb20 con 0x7fd4900d1bc0 2023-11-26T07:20:06.487 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:20:06.483+0000 7fd497679640 1 -- --> v2:172.21.15.143:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd49015dac0 con 0x7fd4900b2160 2023-11-26T07:20:06.487 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:20:06.483+0000 7fd495415640 1 --1- >> v1:172.21.15.143:6789/0 conn(0x7fd49015d6e0 0x7fd4900d14b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.143:6789/0 says I am v1:172.21.15.143:56586/0 (socket says 172.21.15.143:56586) 2023-11-26T07:20:06.487 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:20:06.483+0000 7fd495415640 1 -- 172.21.15.143:0/3322760711 learned_addr learned my addr 172.21.15.143:0/3322760711 (peer_addr_for_me v1:172.21.15.143:0/0) 2023-11-26T07:20:06.488 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:20:06.483+0000 7fd495c16640 1 --2- 172.21.15.143:0/3322760711 >> v2:172.21.15.143:3300/0 conn(0x7fd4900b2160 0x7fd4900d0df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:20:06.488 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:20:06.483+0000 7fd487fff640 1 -- 172.21.15.143:0/3322760711 <== mon.0 v1:172.21.15.143:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2323725012 0 0) 0x7fd4900cf0e0 con 0x7fd49015d6e0 2023-11-26T07:20:06.488 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:20:06.483+0000 7fd487fff640 1 -- 172.21.15.143:0/3322760711 --> v1:172.21.15.143:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fd47c003530 con 0x7fd49015d6e0 2023-11-26T07:20:06.488 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:20:06.483+0000 7fd495c16640 1 -- 172.21.15.143:0/3322760711 >> v1:172.21.15.188:6789/0 conn(0x7fd4900d1bc0 legacy=0x7fd4900b0750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-26T07:20:06.488 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:20:06.483+0000 7fd495c16640 1 -- 172.21.15.143:0/3322760711 >> v1:172.21.15.143:6789/0 conn(0x7fd49015d6e0 legacy=0x7fd4900d14b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:20:06.489 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:20:06.483+0000 7fd495c16640 1 -- 172.21.15.143:0/3322760711 --> v2:172.21.15.143:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd4900cee00 con 0x7fd4900b2160 2023-11-26T07:20:06.490 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:20:06.487+0000 7fd495c16640 1 --2- 172.21.15.143:0/3322760711 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fd4900b2160 0x7fd4900d0df0 secure :-1 s=READY pgs=670 cs=0 l=1 rev1=1 crypto rx=0x7fd488009d90 tx=0x7fd48801c910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=432b1600f6b82ac9 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:20:06.491 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:20:06.487+0000 7fd487fff640 1 -- 172.21.15.143:0/3322760711 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd4880a58c0 con 0x7fd4900b2160 2023-11-26T07:20:06.491 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:20:06.487+0000 7fd487fff640 1 -- 172.21.15.143:0/3322760711 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd4880a5a20 con 0x7fd4900b2160 2023-11-26T07:20:06.491 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:20:06.487+0000 7fd487fff640 1 -- 172.21.15.143:0/3322760711 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd4880a5db0 con 0x7fd4900b2160 2023-11-26T07:20:06.492 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:20:06.487+0000 7fd497679640 1 -- 172.21.15.143:0/3322760711 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fd4900b2160 msgr2=0x7fd4900d0df0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:20:06.492 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:20:06.487+0000 7fd497679640 1 --2- 172.21.15.143:0/3322760711 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fd4900b2160 0x7fd4900d0df0 secure :-1 s=READY pgs=670 cs=0 l=1 rev1=1 crypto rx=0x7fd488009d90 tx=0x7fd48801c910 comp rx=0 tx=0).stop 2023-11-26T07:20:06.492 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:20:06.487+0000 7fd497679640 1 -- 172.21.15.143:0/3322760711 shutdown_connections 2023-11-26T07:20:06.492 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:20:06.487+0000 7fd497679640 1 --2- 172.21.15.143:0/3322760711 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fd4900b2160 0x7fd4900d0df0 unknown :-1 s=CLOSED pgs=670 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:20:06.492 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:20:06.487+0000 7fd497679640 1 -- 172.21.15.143:0/3322760711 >> 172.21.15.143:0/3322760711 conn(0x7fd49009c550 msgr2=0x7fd4900d3220 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:20:06.492 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:20:06.487+0000 7fd497679640 1 -- 172.21.15.143:0/3322760711 shutdown_connections 2023-11-26T07:20:06.492 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:20:06.487+0000 7fd497679640 1 -- 172.21.15.143:0/3322760711 wait complete. 2023-11-26T07:20:06.493 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:20:06.491+0000 7fd497679640 1 Processor -- start 2023-11-26T07:20:06.493 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:20:06.491+0000 7fd497679640 1 -- start start 2023-11-26T07:20:06.496 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:20:06.491+0000 7fd497679640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fd4900b2160 0x7fd490157f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:20:06.496 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:20:06.491+0000 7fd497679640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fd4900d1bc0 0x7fd490158470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:20:06.496 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:20:06.491+0000 7fd497679640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fd49015d6e0 0x7fd490154c80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:20:06.496 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:20:06.491+0000 7fd497679640 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd4900c7d00 con 0x7fd4900d1bc0 2023-11-26T07:20:06.496 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:20:06.491+0000 7fd497679640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd4900c7080 con 0x7fd49015d6e0 2023-11-26T07:20:06.496 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:20:06.491+0000 7fd497679640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd4900c7360 con 0x7fd4900b2160 2023-11-26T07:20:06.496 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:20:06.491+0000 7fd494c14640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fd4900d1bc0 0x7fd490158470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:20:06.496 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:20:06.491+0000 7fd494c14640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fd4900d1bc0 0x7fd490158470 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:59620/0 (socket says 172.21.15.143:59620) 2023-11-26T07:20:06.497 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:20:06.491+0000 7fd494c14640 1 -- 172.21.15.143:0/3317623490 learned_addr learned my addr 172.21.15.143:0/3317623490 (peer_addr_for_me v2:172.21.15.143:0/0) 2023-11-26T07:20:06.497 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:20:06.491+0000 7fd495415640 1 --2- 172.21.15.143:0/3317623490 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fd4900b2160 0x7fd490157f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:20:06.497 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:20:06.491+0000 7fd495c16640 1 --2- 172.21.15.143:0/3317623490 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fd49015d6e0 0x7fd490154c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:20:06.497 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:20:06.491+0000 7fd494c14640 1 -- 172.21.15.143:0/3317623490 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fd4900b2160 msgr2=0x7fd490157f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:20:06.497 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:20:06.491+0000 7fd494c14640 1 --2- 172.21.15.143:0/3317623490 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fd4900b2160 0x7fd490157f30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:20:06.497 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:20:06.491+0000 7fd494c14640 1 -- 172.21.15.143:0/3317623490 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fd49015d6e0 msgr2=0x7fd490154c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:20:06.497 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:20:06.491+0000 7fd494c14640 1 --2- 172.21.15.143:0/3317623490 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fd49015d6e0 0x7fd490154c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:20:06.497 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:20:06.491+0000 7fd494c14640 1 -- 172.21.15.143:0/3317623490 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd488009a70 con 0x7fd4900d1bc0 2023-11-26T07:20:06.497 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:20:06.491+0000 7fd495415640 1 --2- 172.21.15.143:0/3317623490 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fd4900b2160 0x7fd490157f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-26T07:20:06.497 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:20:06.491+0000 7fd495c16640 1 --2- 172.21.15.143:0/3317623490 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fd49015d6e0 0x7fd490154c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-26T07:20:06.497 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:20:06.491+0000 7fd494c14640 1 --2- 172.21.15.143:0/3317623490 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fd4900d1bc0 0x7fd490158470 secure :-1 s=READY pgs=671 cs=0 l=1 rev1=1 crypto rx=0x7fd48000c450 tx=0x7fd48000c910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:20:06.497 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:20:06.491+0000 7fd485ffb640 1 -- 172.21.15.143:0/3317623490 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd480012830 con 0x7fd4900d1bc0 2023-11-26T07:20:06.498 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:20:06.491+0000 7fd485ffb640 1 -- 172.21.15.143:0/3317623490 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd4800129b0 con 0x7fd4900d1bc0 2023-11-26T07:20:06.498 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:20:06.491+0000 7fd485ffb640 1 -- 172.21.15.143:0/3317623490 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd480012d60 con 0x7fd4900d1bc0 2023-11-26T07:20:06.498 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:20:06.491+0000 7fd497679640 1 -- 172.21.15.143:0/3317623490 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd4900c7640 con 0x7fd4900d1bc0 2023-11-26T07:20:06.498 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:20:06.491+0000 7fd497679640 1 -- 172.21.15.143:0/3317623490 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd490155b00 con 0x7fd4900d1bc0 2023-11-26T07:20:06.499 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:20:06.495+0000 7fd497679640 1 -- 172.21.15.143:0/3317623490 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd4900c7e40 con 0x7fd4900d1bc0 2023-11-26T07:20:06.499 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:20:06.495+0000 7fd485ffb640 1 -- 172.21.15.143:0/3317623490 <== mon.0 v2:172.21.15.143:3300/0 4 ==== mgrmap(e 7) v1 ==== 86005+0+0 (secure 0 0 0) 0x7fd48001c460 con 0x7fd4900d1bc0 2023-11-26T07:20:06.499 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:20:06.495+0000 7fd485ffb640 1 --2- 172.21.15.143:0/3317623490 >> [v2:172.21.15.143:6832/2150557544,v1:172.21.15.143:6833/2150557544] conn(0x7fd474066620 0x7fd474068ae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:20:06.499 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:20:06.495+0000 7fd485ffb640 1 -- 172.21.15.143:0/3317623490 <== mon.0 v2:172.21.15.143:3300/0 5 ==== osd_map(45..45 src has 1..45) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fd4800840a0 con 0x7fd4900d1bc0 2023-11-26T07:20:06.499 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:20:06.495+0000 7fd495415640 1 --2- 172.21.15.143:0/3317623490 >> [v2:172.21.15.143:6832/2150557544,v1:172.21.15.143:6833/2150557544] conn(0x7fd474066620 0x7fd474068ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:20:06.499 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:20:06.495+0000 7fd495415640 1 --2- 172.21.15.143:0/3317623490 >> [v2:172.21.15.143:6832/2150557544,v1:172.21.15.143:6833/2150557544] conn(0x7fd474066620 0x7fd474068ae0 secure :-1 s=READY pgs=324 cs=0 l=1 rev1=1 crypto rx=0x7fd48c005890 tx=0x7fd48c009040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:20:06.503 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:20:06.499+0000 7fd485ffb640 1 -- 172.21.15.143:0/3317623490 <== mon.0 v2:172.21.15.143:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172649 (secure 0 0 0) 0x7fd4900c7e40 con 0x7fd4900d1bc0 2023-11-26T07:20:06.690 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:20:06.687+0000 7fd497679640 1 -- 172.21.15.143:0/3317623490 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fd4900c7e40 con 0x7fd4900d1bc0 2023-11-26T07:20:06.690 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:20:06.687+0000 7fd485ffb640 1 -- 172.21.15.143:0/3317623490 <== mon.0 v2:172.21.15.143:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 26 v26) v1 ==== 94+0+4922 (secure 0 0 0) 0x7fd4900c7e40 con 0x7fd4900d1bc0 2023-11-26T07:20:06.691 INFO:teuthology.orchestra.run.smithi143.stdout: 2023-11-26T07:20:06.691 INFO:teuthology.orchestra.run.smithi143.stdout:{"epoch":26,"default_fscid":3,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5201,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.188:6835/319464848","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6833","nonce":319464848},{"type":"v1","addr":"172.21.15.188:6835","nonce":319464848}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":20},{"gid":5584,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.188:6834/2113408070","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6832","nonce":2113408070},{"type":"v1","addr":"172.21.15.188:6834","nonce":2113408070}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":20},{"gid":5745,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.143:6835/1616742593","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6834","nonce":1616742593},{"type":"v1","addr":"172.21.15.143:6835","nonce":1616742593}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":20}],"filesystems":[{"mdsmap":{"epoch":25,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-11-26T07:19:31.503587+0000","modified":"2023-11-26T07:19:34.523605+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":5748},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5748":{"gid":5748,"name":"c","rank":0,"incarnation":21,"state":"up:active","state_seq":3,"addr":"172.21.15.143:6837/2872033338","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6836","nonce":2872033338},{"type":"v1","addr":"172.21.15.143:6837","nonce":2872033338}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[7],"metadata_pool":6,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":3}]} 2023-11-26T07:20:06.691 INFO:teuthology.orchestra.run.smithi143.stderr:dumped fsmap epoch 26 2023-11-26T07:20:06.692 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:20:06.687+0000 7fd497679640 1 -- 172.21.15.143:0/3317623490 >> [v2:172.21.15.143:6832/2150557544,v1:172.21.15.143:6833/2150557544] conn(0x7fd474066620 msgr2=0x7fd474068ae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:20:06.692 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:20:06.687+0000 7fd497679640 1 --2- 172.21.15.143:0/3317623490 >> [v2:172.21.15.143:6832/2150557544,v1:172.21.15.143:6833/2150557544] conn(0x7fd474066620 0x7fd474068ae0 secure :-1 s=READY pgs=324 cs=0 l=1 rev1=1 crypto rx=0x7fd48c005890 tx=0x7fd48c009040 comp rx=0 tx=0).stop 2023-11-26T07:20:06.692 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:20:06.687+0000 7fd497679640 1 -- 172.21.15.143:0/3317623490 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fd4900d1bc0 msgr2=0x7fd490158470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:20:06.693 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:20:06.687+0000 7fd497679640 1 --2- 172.21.15.143:0/3317623490 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fd4900d1bc0 0x7fd490158470 secure :-1 s=READY pgs=671 cs=0 l=1 rev1=1 crypto rx=0x7fd48000c450 tx=0x7fd48000c910 comp rx=0 tx=0).stop 2023-11-26T07:20:06.693 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:20:06.691+0000 7fd497679640 1 -- 172.21.15.143:0/3317623490 shutdown_connections 2023-11-26T07:20:06.693 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:20:06.691+0000 7fd497679640 1 --2- 172.21.15.143:0/3317623490 >> [v2:172.21.15.143:6832/2150557544,v1:172.21.15.143:6833/2150557544] conn(0x7fd474066620 0x7fd474068ae0 unknown :-1 s=CLOSED pgs=324 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:20:06.693 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:20:06.691+0000 7fd497679640 1 --2- 172.21.15.143:0/3317623490 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fd49015d6e0 0x7fd490154c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:20:06.693 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:20:06.691+0000 7fd497679640 1 --2- 172.21.15.143:0/3317623490 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fd4900d1bc0 0x7fd490158470 unknown :-1 s=CLOSED pgs=671 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:20:06.693 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:20:06.691+0000 7fd497679640 1 --2- 172.21.15.143:0/3317623490 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fd4900b2160 0x7fd490157f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:20:06.693 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:20:06.691+0000 7fd497679640 1 -- 172.21.15.143:0/3317623490 >> 172.21.15.143:0/3317623490 conn(0x7fd49009c550 msgr2=0x7fd4900d3220 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:20:06.693 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:20:06.691+0000 7fd497679640 1 -- 172.21.15.143:0/3317623490 shutdown_connections 2023-11-26T07:20:06.693 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:20:06.691+0000 7fd497679640 1 -- 172.21.15.143:0/3317623490 wait complete. 2023-11-26T07:20:06.712 DEBUG:teuthology.orchestra.run.smithi143:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.c.asok --format=json session ls 2023-11-26T07:20:06.870 INFO:teuthology.orchestra.run.smithi143.stdout:[] 2023-11-26T07:20:06.870 DEBUG:tasks.cephfs.filesystem:_json_asok output [] 2023-11-26T07:20:06.870 DEBUG:teuthology.orchestra.run.smithi143:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume create cephfs subvol56170 2023-11-26T07:20:07.011 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:20:07.007+0000 7f16b1248640 1 Processor -- start 2023-11-26T07:20:07.012 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:20:07.007+0000 7f16b1248640 1 -- start start 2023-11-26T07:20:07.013 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:20:07.011+0000 7f16b1248640 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7f16ac0b0aa0 0x7f16ac0b2040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:20:07.013 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:20:07.011+0000 7f16b1248640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f16ac164e90 0x7f16ac167280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:20:07.013 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:20:07.011+0000 7f16b1248640 1 -- --> v1:172.21.15.188:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f16ac06b5a0 con 0x7f16ac09a670 2023-11-26T07:20:07.013 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:20:07.011+0000 7f16b1248640 1 -- --> v2:172.21.15.188:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f16ac0ce160 con 0x7f16ac0b0aa0 2023-11-26T07:20:07.013 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:20:07.011+0000 7f16b1248640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f16ac0ce3d0 con 0x7f16ac164e90 2023-11-26T07:20:07.014 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:20:07.011+0000 7f16b0a47640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f16ac164e90 0x7f16ac167280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:20:07.014 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:20:07.011+0000 7f16abfff640 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7f16ac0b0aa0 0x7f16ac0b2040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:20:07.014 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:20:07.011+0000 7f16ab7fe640 1 --1- >> v1:172.21.15.188:6789/0 conn(0x7f16ac09a670 0x7f16ac0b0e80 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.188:6789/0 says I am v1:172.21.15.143:47174/0 (socket says 172.21.15.143:47174) 2023-11-26T07:20:07.014 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:20:07.011+0000 7f16abfff640 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7f16ac0b0aa0 0x7f16ac0b2040 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.188:3300/0 says I am v2:172.21.15.143:34570/0 (socket says 172.21.15.143:34570) 2023-11-26T07:20:07.014 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:20:07.011+0000 7f16b0a47640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f16ac164e90 0x7f16ac167280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.188:3301/0 says I am v2:172.21.15.143:51588/0 (socket says 172.21.15.143:51588) 2023-11-26T07:20:07.015 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:20:07.011+0000 7f16ab7fe640 1 -- 172.21.15.143:0/2243942920 learned_addr learned my addr 172.21.15.143:0/2243942920 (peer_addr_for_me v1:172.21.15.143:0/0) 2023-11-26T07:20:07.015 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:20:07.011+0000 7f16abfff640 1 -- 172.21.15.143:0/2243942920 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f16ac164e90 msgr2=0x7f16ac167280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-26T07:20:07.015 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:20:07.011+0000 7f16abfff640 1 --2- 172.21.15.143:0/2243942920 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f16ac164e90 0x7f16ac167280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:20:07.015 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:20:07.011+0000 7f16abfff640 1 -- 172.21.15.143:0/2243942920 >> v1:172.21.15.188:6789/0 conn(0x7f16ac09a670 legacy=0x7f16ac0b0e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:20:07.015 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:20:07.011+0000 7f16abfff640 1 -- 172.21.15.143:0/2243942920 --> v2:172.21.15.188:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f16ac0ce6b0 con 0x7f16ac0b0aa0 2023-11-26T07:20:07.015 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:20:07.011+0000 7f16b0a47640 1 --2- 172.21.15.143:0/2243942920 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f16ac164e90 0x7f16ac167280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-26T07:20:07.017 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:20:07.015+0000 7f16abfff640 1 --2- 172.21.15.143:0/2243942920 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f16ac0b0aa0 0x7f16ac0b2040 secure :-1 s=READY pgs=276 cs=0 l=1 rev1=1 crypto rx=0x7f169c0097c0 tx=0x7f169c01c480 comp rx=0 tx=0).ready entity=mon.1 client_cookie=90aedb35ccba5f90 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:20:07.018 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:20:07.015+0000 7f16aaffd640 1 -- 172.21.15.143:0/2243942920 <== mon.1 v2:172.21.15.188:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f169c0ab610 con 0x7f16ac0b0aa0 2023-11-26T07:20:07.018 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:20:07.015+0000 7f16aaffd640 1 -- 172.21.15.143:0/2243942920 <== mon.1 v2:172.21.15.188:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f169c0ab770 con 0x7f16ac0b0aa0 2023-11-26T07:20:07.018 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:20:07.015+0000 7f16aaffd640 1 -- 172.21.15.143:0/2243942920 <== mon.1 v2:172.21.15.188:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f169c0abb00 con 0x7f16ac0b0aa0 2023-11-26T07:20:07.019 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:20:07.015+0000 7f16b1248640 1 -- 172.21.15.143:0/2243942920 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f16ac0b0aa0 msgr2=0x7f16ac0b2040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:20:07.019 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:20:07.015+0000 7f16b1248640 1 --2- 172.21.15.143:0/2243942920 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f16ac0b0aa0 0x7f16ac0b2040 secure :-1 s=READY pgs=276 cs=0 l=1 rev1=1 crypto rx=0x7f169c0097c0 tx=0x7f169c01c480 comp rx=0 tx=0).stop 2023-11-26T07:20:07.019 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:20:07.015+0000 7f16b1248640 1 -- 172.21.15.143:0/2243942920 shutdown_connections 2023-11-26T07:20:07.019 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:20:07.015+0000 7f16b1248640 1 --2- 172.21.15.143:0/2243942920 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f16ac164e90 0x7f16ac167280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:20:07.019 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:20:07.015+0000 7f16b1248640 1 --2- 172.21.15.143:0/2243942920 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f16ac0b0aa0 0x7f16ac0b2040 unknown :-1 s=CLOSED pgs=276 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:20:07.019 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:20:07.015+0000 7f16b1248640 1 -- 172.21.15.143:0/2243942920 >> 172.21.15.143:0/2243942920 conn(0x7f16ac099a70 msgr2=0x7f16ac099e80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:20:07.019 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:20:07.015+0000 7f16b1248640 1 -- 172.21.15.143:0/2243942920 shutdown_connections 2023-11-26T07:20:07.019 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:20:07.015+0000 7f16b1248640 1 -- 172.21.15.143:0/2243942920 wait complete. 2023-11-26T07:20:07.020 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:20:07.015+0000 7f16b1248640 1 Processor -- start 2023-11-26T07:20:07.020 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:20:07.019+0000 7f16b1248640 1 -- start start 2023-11-26T07:20:07.022 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:20:07.019+0000 7f16b1248640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f16ac09a670 0x7f16ac14c400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:20:07.022 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:20:07.019+0000 7f16b1248640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f16ac0b0aa0 0x7f16ac14a7b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:20:07.022 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:20:07.019+0000 7f16b1248640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f16ac164e90 0x7f16ac06b990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:20:07.023 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:20:07.019+0000 7f16b1248640 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f16ac155ae0 con 0x7f16ac09a670 2023-11-26T07:20:07.023 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:20:07.019+0000 7f16b1248640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f16ac0b0630 con 0x7f16ac0b0aa0 2023-11-26T07:20:07.023 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:20:07.019+0000 7f16b1248640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f16ac14cc50 con 0x7f16ac164e90 2023-11-26T07:20:07.023 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:20:07.019+0000 7f16abfff640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f16ac09a670 0x7f16ac14c400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:20:07.023 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:20:07.019+0000 7f16abfff640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f16ac09a670 0x7f16ac14c400 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:59634/0 (socket says 172.21.15.143:59634) 2023-11-26T07:20:07.023 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:20:07.019+0000 7f16abfff640 1 -- 172.21.15.143:0/3789506374 learned_addr learned my addr 172.21.15.143:0/3789506374 (peer_addr_for_me v2:172.21.15.143:0/0) 2023-11-26T07:20:07.023 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:20:07.019+0000 7f16b0a47640 1 --2- 172.21.15.143:0/3789506374 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f16ac164e90 0x7f16ac06b990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:20:07.023 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:20:07.019+0000 7f16ab7fe640 1 --2- 172.21.15.143:0/3789506374 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f16ac0b0aa0 0x7f16ac14a7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:20:07.023 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:20:07.019+0000 7f16abfff640 1 -- 172.21.15.143:0/3789506374 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f16ac164e90 msgr2=0x7f16ac06b990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:20:07.023 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:20:07.019+0000 7f16abfff640 1 --2- 172.21.15.143:0/3789506374 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f16ac164e90 0x7f16ac06b990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:20:07.023 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:20:07.019+0000 7f16abfff640 1 -- 172.21.15.143:0/3789506374 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f16ac0b0aa0 msgr2=0x7f16ac14a7b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:20:07.024 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:20:07.019+0000 7f16abfff640 1 --2- 172.21.15.143:0/3789506374 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f16ac0b0aa0 0x7f16ac14a7b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:20:07.024 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:20:07.019+0000 7f16abfff640 1 -- 172.21.15.143:0/3789506374 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f16a4009a70 con 0x7f16ac09a670 2023-11-26T07:20:07.024 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:20:07.019+0000 7f16abfff640 1 --2- 172.21.15.143:0/3789506374 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f16ac09a670 0x7f16ac14c400 secure :-1 s=READY pgs=672 cs=0 l=1 rev1=1 crypto rx=0x7f169c0ad040 tx=0x7f169c01acc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:20:07.025 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:20:07.019+0000 7f16a8ff9640 1 -- 172.21.15.143:0/3789506374 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f169c0bf610 con 0x7f16ac09a670 2023-11-26T07:20:07.025 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:20:07.019+0000 7f16a8ff9640 1 -- 172.21.15.143:0/3789506374 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f169c0bf770 con 0x7f16ac09a670 2023-11-26T07:20:07.025 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:20:07.019+0000 7f16b1248640 1 -- 172.21.15.143:0/3789506374 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f169c009420 con 0x7f16ac09a670 2023-11-26T07:20:07.025 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:20:07.019+0000 7f16a8ff9640 1 -- 172.21.15.143:0/3789506374 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f169c0bfae0 con 0x7f16ac09a670 2023-11-26T07:20:07.025 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:20:07.019+0000 7f16b1248640 1 -- 172.21.15.143:0/3789506374 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f16ac154fe0 con 0x7f16ac09a670 2023-11-26T07:20:07.025 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:20:07.019+0000 7f16b1248640 1 -- 172.21.15.143:0/3789506374 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f16ac067500 con 0x7f16ac09a670 2023-11-26T07:20:07.025 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:20:07.023+0000 7f16a8ff9640 1 -- 172.21.15.143:0/3789506374 <== mon.0 v2:172.21.15.143:3300/0 4 ==== mgrmap(e 7) v1 ==== 86005+0+0 (secure 0 0 0) 0x7f169c0bfc40 con 0x7f16ac09a670 2023-11-26T07:20:07.026 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:20:07.023+0000 7f16a8ff9640 1 --2- 172.21.15.143:0/3789506374 >> [v2:172.21.15.143:6832/2150557544,v1:172.21.15.143:6833/2150557544] conn(0x7f1698066620 0x7f1698068ae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:20:07.026 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:20:07.023+0000 7f16a8ff9640 1 -- 172.21.15.143:0/3789506374 <== mon.0 v2:172.21.15.143:3300/0 5 ==== osd_map(45..45 src has 1..45) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f169c11ee50 con 0x7f16ac09a670 2023-11-26T07:20:07.026 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:20:07.023+0000 7f16ab7fe640 1 --2- 172.21.15.143:0/3789506374 >> [v2:172.21.15.143:6832/2150557544,v1:172.21.15.143:6833/2150557544] conn(0x7f1698066620 0x7f1698068ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:20:07.026 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:20:07.023+0000 7f16ab7fe640 1 --2- 172.21.15.143:0/3789506374 >> [v2:172.21.15.143:6832/2150557544,v1:172.21.15.143:6833/2150557544] conn(0x7f1698066620 0x7f1698068ae0 secure :-1 s=READY pgs=325 cs=0 l=1 rev1=1 crypto rx=0x7f16ac166390 tx=0x7f16a0009040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:20:07.029 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:20:07.027+0000 7f16a8ff9640 1 -- 172.21.15.143:0/3789506374 <== mon.0 v2:172.21.15.143:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172649 (secure 0 0 0) 0x7f169c0ebf20 con 0x7f16ac09a670 2023-11-26T07:20:07.203 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:20:07.199+0000 7f16b1248640 1 -- 172.21.15.143:0/3789506374 --> [v2:172.21.15.143:6832/2150557544,v1:172.21.15.143:6833/2150557544] -- mgr_command(tid 0: {"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "subvol56170", "target": ["mon-mgr", ""]}) v1 -- 0x7f16ac0b2580 con 0x7f1698066620 2023-11-26T07:20:07.209 INFO:tasks.ceph.mgr.y.smithi143.stderr:2023-11-26T07:20:07.207+0000 7f5048ac6640 -1 client.0 error registering admin socket command: (17) File exists 2023-11-26T07:20:07.209 INFO:tasks.ceph.mgr.y.smithi143.stderr:2023-11-26T07:20:07.207+0000 7f5048ac6640 -1 client.0 error registering admin socket command: (17) File exists 2023-11-26T07:20:07.209 INFO:tasks.ceph.mgr.y.smithi143.stderr:2023-11-26T07:20:07.207+0000 7f5048ac6640 -1 client.0 error registering admin socket command: (17) File exists 2023-11-26T07:20:07.209 INFO:tasks.ceph.mgr.y.smithi143.stderr:2023-11-26T07:20:07.207+0000 7f5048ac6640 -1 client.0 error registering admin socket command: (17) File exists 2023-11-26T07:20:07.209 INFO:tasks.ceph.mgr.y.smithi143.stderr:2023-11-26T07:20:07.207+0000 7f5048ac6640 -1 client.0 error registering admin socket command: (17) File exists 2023-11-26T07:20:07.241 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:20:07.239+0000 7f16a8ff9640 1 -- 172.21.15.143:0/3789506374 <== mgr.4100 v2:172.21.15.143:6832/2150557544 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+0 (secure 0 0 0) 0x7f16a0001350 con 0x7f1698066620 2023-11-26T07:20:07.243 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:20:07.239+0000 7f16b1248640 1 -- 172.21.15.143:0/3789506374 >> [v2:172.21.15.143:6832/2150557544,v1:172.21.15.143:6833/2150557544] conn(0x7f1698066620 msgr2=0x7f1698068ae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:20:07.243 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:20:07.239+0000 7f16b1248640 1 --2- 172.21.15.143:0/3789506374 >> [v2:172.21.15.143:6832/2150557544,v1:172.21.15.143:6833/2150557544] conn(0x7f1698066620 0x7f1698068ae0 secure :-1 s=READY pgs=325 cs=0 l=1 rev1=1 crypto rx=0x7f16ac166390 tx=0x7f16a0009040 comp rx=0 tx=0).stop 2023-11-26T07:20:07.243 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:20:07.239+0000 7f16b1248640 1 -- 172.21.15.143:0/3789506374 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f16ac09a670 msgr2=0x7f16ac14c400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:20:07.243 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:20:07.239+0000 7f16b1248640 1 --2- 172.21.15.143:0/3789506374 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f16ac09a670 0x7f16ac14c400 secure :-1 s=READY pgs=672 cs=0 l=1 rev1=1 crypto rx=0x7f169c0ad040 tx=0x7f169c01acc0 comp rx=0 tx=0).stop 2023-11-26T07:20:07.243 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:20:07.239+0000 7f16b1248640 1 -- 172.21.15.143:0/3789506374 shutdown_connections 2023-11-26T07:20:07.243 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:20:07.239+0000 7f16b1248640 1 --2- 172.21.15.143:0/3789506374 >> [v2:172.21.15.143:6832/2150557544,v1:172.21.15.143:6833/2150557544] conn(0x7f1698066620 0x7f1698068ae0 unknown :-1 s=CLOSED pgs=325 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:20:07.243 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:20:07.239+0000 7f16b1248640 1 --2- 172.21.15.143:0/3789506374 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f16ac164e90 0x7f16ac06b990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:20:07.243 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:20:07.239+0000 7f16b1248640 1 --2- 172.21.15.143:0/3789506374 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f16ac0b0aa0 0x7f16ac14a7b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:20:07.243 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:20:07.239+0000 7f16b1248640 1 --2- 172.21.15.143:0/3789506374 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f16ac09a670 0x7f16ac14c400 unknown :-1 s=CLOSED pgs=672 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:20:07.244 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:20:07.239+0000 7f16b1248640 1 -- 172.21.15.143:0/3789506374 >> 172.21.15.143:0/3789506374 conn(0x7f16ac099a70 msgr2=0x7f16ac06cd10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:20:07.244 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:20:07.239+0000 7f16b1248640 1 -- 172.21.15.143:0/3789506374 shutdown_connections 2023-11-26T07:20:07.244 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:20:07.239+0000 7f16b1248640 1 -- 172.21.15.143:0/3789506374 wait complete. 2023-11-26T07:20:07.262 DEBUG:teuthology.orchestra.run.smithi143:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-11-26T07:20:07.399 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:20:07.395+0000 7f663a1df640 1 Processor -- start 2023-11-26T07:20:07.399 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:20:07.395+0000 7f663a1df640 1 -- start start 2023-11-26T07:20:07.400 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:20:07.395+0000 7f663a1df640 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7f663413e290 0x7f66340cf280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:20:07.400 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:20:07.395+0000 7f663a1df640 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7f663415f6b0 0x7f66340cd7b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:20:07.400 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:20:07.395+0000 7f663a1df640 1 -- --> v1:172.21.15.143:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f663415fb20 con 0x7f66340cf7c0 2023-11-26T07:20:07.400 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:20:07.395+0000 7f663a1df640 1 -- --> v2:172.21.15.143:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f66340cfc40 con 0x7f663415f6b0 2023-11-26T07:20:07.401 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:20:07.395+0000 7f663a1df640 1 -- --> v2:172.21.15.188:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f663406c520 con 0x7f663413e290 2023-11-26T07:20:07.402 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:20:07.399+0000 7f6633fff640 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7f663415f6b0 0x7f66340cd7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:20:07.402 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:20:07.399+0000 7f6632ffd640 1 --1- >> v1:172.21.15.143:6789/0 conn(0x7f66340cf7c0 0x7f663415efa0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.143:6789/0 says I am v1:172.21.15.143:56592/0 (socket says 172.21.15.143:56592) 2023-11-26T07:20:07.402 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:20:07.399+0000 7f6633fff640 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7f663415f6b0 0x7f66340cd7b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:59658/0 (socket says 172.21.15.143:59658) 2023-11-26T07:20:07.402 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:20:07.399+0000 7f6632ffd640 1 -- 172.21.15.143:0/1627932982 learned_addr learned my addr 172.21.15.143:0/1627932982 (peer_addr_for_me v1:172.21.15.143:0/0) 2023-11-26T07:20:07.402 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:20:07.399+0000 7f66337fe640 1 --2- 172.21.15.143:0/1627932982 >> v2:172.21.15.188:3300/0 conn(0x7f663413e290 0x7f66340cf280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:20:07.403 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:20:07.399+0000 7f6633fff640 1 -- 172.21.15.143:0/1627932982 >> v2:172.21.15.188:3300/0 conn(0x7f663413e290 msgr2=0x7f66340cf280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-26T07:20:07.403 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:20:07.399+0000 7f6633fff640 1 --2- 172.21.15.143:0/1627932982 >> v2:172.21.15.188:3300/0 conn(0x7f663413e290 0x7f66340cf280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:20:07.403 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:20:07.399+0000 7f6633fff640 1 -- 172.21.15.143:0/1627932982 >> v1:172.21.15.143:6789/0 conn(0x7f66340cf7c0 legacy=0x7f663415efa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:20:07.403 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:20:07.399+0000 7f6633fff640 1 -- 172.21.15.143:0/1627932982 --> v2:172.21.15.143:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f663406c800 con 0x7f663415f6b0 2023-11-26T07:20:07.404 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:20:07.399+0000 7f6633fff640 1 --2- 172.21.15.143:0/1627932982 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f663415f6b0 0x7f66340cd7b0 secure :-1 s=READY pgs=676 cs=0 l=1 rev1=1 crypto rx=0x7f66340725a0 tx=0x7f662c01c8e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=eb21901ba4757dc5 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:20:07.404 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:20:07.399+0000 7f66327fc640 1 -- 172.21.15.143:0/1627932982 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f662c0a59a0 con 0x7f663415f6b0 2023-11-26T07:20:07.404 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:20:07.399+0000 7f66327fc640 1 -- 172.21.15.143:0/1627932982 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f662c0a5b00 con 0x7f663415f6b0 2023-11-26T07:20:07.404 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:20:07.399+0000 7f66327fc640 1 -- 172.21.15.143:0/1627932982 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f662c0a5e90 con 0x7f663415f6b0 2023-11-26T07:20:07.405 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:20:07.403+0000 7f663a1df640 1 -- 172.21.15.143:0/1627932982 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f663415f6b0 msgr2=0x7f66340cd7b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:20:07.405 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:20:07.403+0000 7f663a1df640 1 --2- 172.21.15.143:0/1627932982 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f663415f6b0 0x7f66340cd7b0 secure :-1 s=READY pgs=676 cs=0 l=1 rev1=1 crypto rx=0x7f66340725a0 tx=0x7f662c01c8e0 comp rx=0 tx=0).stop 2023-11-26T07:20:07.406 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:20:07.403+0000 7f663a1df640 1 -- 172.21.15.143:0/1627932982 shutdown_connections 2023-11-26T07:20:07.406 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:20:07.403+0000 7f663a1df640 1 --2- 172.21.15.143:0/1627932982 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f663415f6b0 0x7f66340cd7b0 unknown :-1 s=CLOSED pgs=676 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:20:07.406 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:20:07.403+0000 7f663a1df640 1 --2- 172.21.15.143:0/1627932982 >> v2:172.21.15.188:3300/0 conn(0x7f663413e290 0x7f66340cf280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:20:07.406 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:20:07.403+0000 7f663a1df640 1 -- 172.21.15.143:0/1627932982 >> 172.21.15.143:0/1627932982 conn(0x7f663406a890 msgr2=0x7f663406aca0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:20:07.406 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:20:07.403+0000 7f663a1df640 1 -- 172.21.15.143:0/1627932982 shutdown_connections 2023-11-26T07:20:07.406 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:20:07.403+0000 7f663a1df640 1 -- 172.21.15.143:0/1627932982 wait complete. 2023-11-26T07:20:07.407 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:20:07.403+0000 7f663a1df640 1 Processor -- start 2023-11-26T07:20:07.407 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:20:07.403+0000 7f663a1df640 1 -- start start 2023-11-26T07:20:07.409 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:20:07.403+0000 7f663a1df640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f66340cf7c0 0x7f663415e880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:20:07.409 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:20:07.403+0000 7f663a1df640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f663413e290 0x7f6634132e90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:20:07.409 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:20:07.403+0000 7f663a1df640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f663415f6b0 0x7f6634131ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:20:07.409 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:20:07.403+0000 7f663a1df640 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6634144cb0 con 0x7f663415f6b0 2023-11-26T07:20:07.409 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:20:07.403+0000 7f663a1df640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f66341355f0 con 0x7f66340cf7c0 2023-11-26T07:20:07.410 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:20:07.403+0000 7f663a1df640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6634135310 con 0x7f663413e290 2023-11-26T07:20:07.410 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:20:07.403+0000 7f6633fff640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f663415f6b0 0x7f6634131ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:20:07.410 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:20:07.403+0000 7f6633fff640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f663415f6b0 0x7f6634131ed0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:59666/0 (socket says 172.21.15.143:59666) 2023-11-26T07:20:07.410 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:20:07.403+0000 7f6633fff640 1 -- 172.21.15.143:0/234537988 learned_addr learned my addr 172.21.15.143:0/234537988 (peer_addr_for_me v2:172.21.15.143:0/0) 2023-11-26T07:20:07.410 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:20:07.403+0000 7f66337fe640 1 --2- 172.21.15.143:0/234537988 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f66340cf7c0 0x7f663415e880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:20:07.410 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:20:07.403+0000 7f6632ffd640 1 --2- 172.21.15.143:0/234537988 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f663413e290 0x7f6634132e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:20:07.411 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:20:07.403+0000 7f6633fff640 1 -- 172.21.15.143:0/234537988 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f663413e290 msgr2=0x7f6634132e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:20:07.411 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:20:07.403+0000 7f6633fff640 1 --2- 172.21.15.143:0/234537988 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f663413e290 0x7f6634132e90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:20:07.411 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:20:07.403+0000 7f6633fff640 1 -- 172.21.15.143:0/234537988 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f66340cf7c0 msgr2=0x7f663415e880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:20:07.411 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:20:07.403+0000 7f6633fff640 1 --2- 172.21.15.143:0/234537988 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f66340cf7c0 0x7f663415e880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:20:07.411 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:20:07.407+0000 7f6633fff640 1 -- 172.21.15.143:0/234537988 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f662c009a70 con 0x7f663415f6b0 2023-11-26T07:20:07.411 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:20:07.407+0000 7f6633fff640 1 --2- 172.21.15.143:0/234537988 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f663415f6b0 0x7f6634131ed0 secure :-1 s=READY pgs=677 cs=0 l=1 rev1=1 crypto rx=0x7f662c0194f0 tx=0x7f662c019680 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:20:07.411 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:20:07.407+0000 7f661bfff640 1 -- 172.21.15.143:0/234537988 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f662c0ade00 con 0x7f663415f6b0 2023-11-26T07:20:07.412 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:20:07.407+0000 7f661bfff640 1 -- 172.21.15.143:0/234537988 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f662c009450 con 0x7f663415f6b0 2023-11-26T07:20:07.412 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:20:07.407+0000 7f663a1df640 1 -- 172.21.15.143:0/234537988 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6634152600 con 0x7f663415f6b0 2023-11-26T07:20:07.412 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:20:07.407+0000 7f661bfff640 1 -- 172.21.15.143:0/234537988 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f662c0b9690 con 0x7f663415f6b0 2023-11-26T07:20:07.412 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:20:07.407+0000 7f663a1df640 1 -- 172.21.15.143:0/234537988 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6634152bc0 con 0x7f663415f6b0 2023-11-26T07:20:07.413 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:20:07.407+0000 7f663a1df640 1 -- 172.21.15.143:0/234537988 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6634152db0 con 0x7f663415f6b0 2023-11-26T07:20:07.413 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:20:07.407+0000 7f661bfff640 1 -- 172.21.15.143:0/234537988 <== mon.0 v2:172.21.15.143:3300/0 4 ==== mgrmap(e 7) v1 ==== 86005+0+0 (secure 0 0 0) 0x7f662c0b97f0 con 0x7f663415f6b0 2023-11-26T07:20:07.413 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:20:07.407+0000 7f661bfff640 1 --2- 172.21.15.143:0/234537988 >> [v2:172.21.15.143:6832/2150557544,v1:172.21.15.143:6833/2150557544] conn(0x7f6620066620 0x7f6620068ae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:20:07.413 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:20:07.407+0000 7f661bfff640 1 -- 172.21.15.143:0/234537988 <== mon.0 v2:172.21.15.143:3300/0 5 ==== osd_map(45..45 src has 1..45) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f662c11fa50 con 0x7f663415f6b0 2023-11-26T07:20:07.414 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:20:07.407+0000 7f66337fe640 1 --2- 172.21.15.143:0/234537988 >> [v2:172.21.15.143:6832/2150557544,v1:172.21.15.143:6833/2150557544] conn(0x7f6620066620 0x7f6620068ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:20:07.414 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:20:07.411+0000 7f66337fe640 1 --2- 172.21.15.143:0/234537988 >> [v2:172.21.15.143:6832/2150557544,v1:172.21.15.143:6833/2150557544] conn(0x7f6620066620 0x7f6620068ae0 secure :-1 s=READY pgs=326 cs=0 l=1 rev1=1 crypto rx=0x7f662400b9d0 tx=0x7f6624008040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:20:07.415 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:20:07.411+0000 7f661bfff640 1 -- 172.21.15.143:0/234537988 <== mon.0 v2:172.21.15.143:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172649 (secure 0 0 0) 0x7f662c0ecbc0 con 0x7f663415f6b0 2023-11-26T07:20:07.599 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:20:07.595+0000 7f663a1df640 1 -- 172.21.15.143:0/234537988 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f6634152db0 con 0x7f663415f6b0 2023-11-26T07:20:07.599 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:20:07.595+0000 7f661bfff640 1 -- 172.21.15.143:0/234537988 <== mon.0 v2:172.21.15.143:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 26 v26) v1 ==== 94+0+4922 (secure 0 0 0) 0x7f662c0b2020 con 0x7f663415f6b0 2023-11-26T07:20:07.600 INFO:teuthology.orchestra.run.smithi143.stdout: 2023-11-26T07:20:07.600 INFO:teuthology.orchestra.run.smithi143.stdout:{"epoch":26,"default_fscid":3,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5201,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.188:6835/319464848","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6833","nonce":319464848},{"type":"v1","addr":"172.21.15.188:6835","nonce":319464848}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":20},{"gid":5584,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.188:6834/2113408070","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6832","nonce":2113408070},{"type":"v1","addr":"172.21.15.188:6834","nonce":2113408070}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":20},{"gid":5745,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.143:6835/1616742593","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6834","nonce":1616742593},{"type":"v1","addr":"172.21.15.143:6835","nonce":1616742593}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":20}],"filesystems":[{"mdsmap":{"epoch":25,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-11-26T07:19:31.503587+0000","modified":"2023-11-26T07:19:34.523605+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":5748},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5748":{"gid":5748,"name":"c","rank":0,"incarnation":21,"state":"up:active","state_seq":3,"addr":"172.21.15.143:6837/2872033338","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6836","nonce":2872033338},{"type":"v1","addr":"172.21.15.143:6837","nonce":2872033338}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[7],"metadata_pool":6,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":3}]} 2023-11-26T07:20:07.600 INFO:teuthology.orchestra.run.smithi143.stderr:dumped fsmap epoch 26 2023-11-26T07:20:07.602 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:20:07.599+0000 7f663a1df640 1 -- 172.21.15.143:0/234537988 >> [v2:172.21.15.143:6832/2150557544,v1:172.21.15.143:6833/2150557544] conn(0x7f6620066620 msgr2=0x7f6620068ae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:20:07.602 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:20:07.599+0000 7f663a1df640 1 --2- 172.21.15.143:0/234537988 >> [v2:172.21.15.143:6832/2150557544,v1:172.21.15.143:6833/2150557544] conn(0x7f6620066620 0x7f6620068ae0 secure :-1 s=READY pgs=326 cs=0 l=1 rev1=1 crypto rx=0x7f662400b9d0 tx=0x7f6624008040 comp rx=0 tx=0).stop 2023-11-26T07:20:07.602 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:20:07.599+0000 7f663a1df640 1 -- 172.21.15.143:0/234537988 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f663415f6b0 msgr2=0x7f6634131ed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:20:07.602 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:20:07.599+0000 7f663a1df640 1 --2- 172.21.15.143:0/234537988 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f663415f6b0 0x7f6634131ed0 secure :-1 s=READY pgs=677 cs=0 l=1 rev1=1 crypto rx=0x7f662c0194f0 tx=0x7f662c019680 comp rx=0 tx=0).stop 2023-11-26T07:20:07.602 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:20:07.599+0000 7f663a1df640 1 -- 172.21.15.143:0/234537988 shutdown_connections 2023-11-26T07:20:07.602 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:20:07.599+0000 7f663a1df640 1 --2- 172.21.15.143:0/234537988 >> [v2:172.21.15.143:6832/2150557544,v1:172.21.15.143:6833/2150557544] conn(0x7f6620066620 0x7f6620068ae0 unknown :-1 s=CLOSED pgs=326 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:20:07.602 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:20:07.599+0000 7f663a1df640 1 --2- 172.21.15.143:0/234537988 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f663415f6b0 0x7f6634131ed0 unknown :-1 s=CLOSED pgs=677 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:20:07.602 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:20:07.599+0000 7f663a1df640 1 --2- 172.21.15.143:0/234537988 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f663413e290 0x7f6634132e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:20:07.602 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:20:07.599+0000 7f663a1df640 1 --2- 172.21.15.143:0/234537988 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f66340cf7c0 0x7f663415e880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:20:07.603 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:20:07.599+0000 7f663a1df640 1 -- 172.21.15.143:0/234537988 >> 172.21.15.143:0/234537988 conn(0x7f663406a890 msgr2=0x7f6634139f30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:20:07.603 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:20:07.599+0000 7f663a1df640 1 -- 172.21.15.143:0/234537988 shutdown_connections 2023-11-26T07:20:07.603 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:20:07.599+0000 7f663a1df640 1 -- 172.21.15.143:0/234537988 wait complete. 2023-11-26T07:20:07.621 DEBUG:teuthology.orchestra.run.smithi143:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.c.asok --format=json session ls 2023-11-26T07:20:07.780 INFO:teuthology.orchestra.run.smithi143.stdout:[{"id":6126,"entity":{"name":{"type":"client","num":6126},"addr":{"type":"any","addr":"172.21.15.143:0","nonce":3879234400}},"state":"open","num_leases":1,"num_caps":7,"request_load_avg":14,"uptime":0.551974093,"requests_in_flight":2,"num_completed_requests":0,"num_completed_flushes":1,"reconnecting":false,"recall_caps":{"value":0,"halflife":60},"release_caps":{"value":0,"halflife":60},"recall_caps_throttle":{"value":0,"halflife":1.5},"recall_caps_throttle2o":{"value":0,"halflife":0.5},"session_cache_liveness":{"value":21.972652847475651,"halflife":300},"cap_acquisition":{"value":0,"halflife":30},"last_trim_completed_requests_tid":14,"last_trim_completed_flushes_tid":2,"delegated_inos":[{"start":"0x10000000005","length":500}],"inst":"client.6126 172.21.15.143:0/3879234400","completed_requests":[],"prealloc_inos":[{"start":"0x10000000005","length":996}],"client_metadata":{"client_features":{"feature_bits":"0x00000000003fffff"},"metric_spec":{"metric_flags":{"feature_bits":"0x000000000000ffff"}},"ceph_sha1":"44c2237fe77713214148c1fd622c872f346f1b99","ceph_version":"ceph version 18.0.0-7484-g44c2237f (44c2237fe77713214148c1fd622c872f346f1b99) reef (dev)","entity_id":"y","hostname":"smithi143","pid":"38097","root":"/"}}] 2023-11-26T07:20:07.780 DEBUG:tasks.cephfs.filesystem:_json_asok output [ { "cap_acquisition": { "halflife": 30, "value": 0 }, "client_metadata": { "ceph_sha1": "44c2237fe77713214148c1fd622c872f346f1b99", "ceph_version": "ceph version 18.0.0-7484-g44c2237f (44c2237fe77713214148c1fd622c872f346f1b99) reef (dev)", "client_features": { "feature_bits": "0x00000000003fffff" }, "entity_id": "y", "hostname": "smithi143", "metric_spec": { "metric_flags": { "feature_bits": "0x000000000000ffff" } }, "pid": "38097", "root": "/" }, "completed_requests": [], "delegated_inos": [ { "length": 500, "start": "0x10000000005" } ], "entity": { "addr": { "addr": "172.21.15.143:0", "nonce": 3879234400, "type": "any" }, "name": { "num": 6126, "type": "client" } }, "id": 6126, "inst": "client.6126 172.21.15.143:0/3879234400", "last_trim_completed_flushes_tid": 2, "last_trim_completed_requests_tid": 14, "num_caps": 7, "num_completed_flushes": 1, "num_completed_requests": 0, "num_leases": 1, "prealloc_inos": [ { "length": 996, "start": "0x10000000005" } ], "recall_caps": { "halflife": 60, "value": 0 }, "recall_caps_throttle": { "halflife": 1.5, "value": 0 }, "recall_caps_throttle2o": { "halflife": 0.5, "value": 0 }, "reconnecting": false, "release_caps": { "halflife": 60, "value": 0 }, "request_load_avg": 14, "requests_in_flight": 2, "session_cache_liveness": { "halflife": 300, "value": 21.97265284747565 }, "state": "open", "uptime": 0.551974093 } ] 2023-11-26T07:20:07.781 DEBUG:teuthology.orchestra.run.smithi143:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-11-26T07:20:07.918 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:20:07.915+0000 7ff04abf8640 1 Processor -- start 2023-11-26T07:20:07.918 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:20:07.915+0000 7ff04abf8640 1 -- start start 2023-11-26T07:20:07.920 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:20:07.915+0000 7ff04abf8640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7ff04415e6f0 0x7ff0441610f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:20:07.920 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:20:07.915+0000 7ff04abf8640 1 -- --> v1:172.21.15.143:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff0440cde60 con 0x7ff04406bae0 2023-11-26T07:20:07.920 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:20:07.915+0000 7ff04abf8640 1 -- --> v1:172.21.15.188:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff04415dc10 con 0x7ff044070ba0 2023-11-26T07:20:07.920 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:20:07.915+0000 7ff04abf8640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff04415def0 con 0x7ff04415e6f0 2023-11-26T07:20:07.920 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:20:07.915+0000 7ff03bfff640 1 --1- >> v1:172.21.15.143:6789/0 conn(0x7ff04406bae0 0x7ff0440cd610 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.143:6789/0 says I am v1:172.21.15.143:56596/0 (socket says 172.21.15.143:56596) 2023-11-26T07:20:07.920 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:20:07.915+0000 7ff03bfff640 1 -- 172.21.15.143:0/145726963 learned_addr learned my addr 172.21.15.143:0/145726963 (peer_addr_for_me v1:172.21.15.143:0/0) 2023-11-26T07:20:07.920 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:20:07.915+0000 7ff049195640 1 --2- 172.21.15.143:0/145726963 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7ff04415e6f0 0x7ff0441610f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:20:07.922 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:20:07.915+0000 7ff03b7fe640 1 -- 172.21.15.143:0/145726963 <== mon.0 v1:172.21.15.143:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1843805469 0 0) 0x7ff0440cde60 con 0x7ff04406bae0 2023-11-26T07:20:07.922 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:20:07.915+0000 7ff03b7fe640 1 -- 172.21.15.143:0/145726963 --> v1:172.21.15.143:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7ff02c003530 con 0x7ff04406bae0 2023-11-26T07:20:07.922 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:20:07.919+0000 7ff03b7fe640 1 -- 172.21.15.143:0/145726963 <== mon.1 v1:172.21.15.188:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1918167136 0 0) 0x7ff04415dc10 con 0x7ff044070ba0 2023-11-26T07:20:07.922 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:20:07.919+0000 7ff03b7fe640 1 -- 172.21.15.143:0/145726963 --> v1:172.21.15.188:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7ff0440cde60 con 0x7ff044070ba0 2023-11-26T07:20:07.922 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:20:07.919+0000 7ff03b7fe640 1 -- 172.21.15.143:0/145726963 <== mon.0 v1:172.21.15.143:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3894648400 0 0) 0x7ff02c003530 con 0x7ff04406bae0 2023-11-26T07:20:07.922 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:20:07.919+0000 7ff03b7fe640 1 -- 172.21.15.143:0/145726963 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7ff04415e6f0 msgr2=0x7ff0441610f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-26T07:20:07.923 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:20:07.919+0000 7ff03b7fe640 1 --2- 172.21.15.143:0/145726963 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7ff04415e6f0 0x7ff0441610f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:20:07.923 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:20:07.919+0000 7ff03b7fe640 1 -- 172.21.15.143:0/145726963 >> v1:172.21.15.188:6789/0 conn(0x7ff044070ba0 legacy=0x7ff044070f80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:20:07.923 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:20:07.919+0000 7ff03b7fe640 1 -- 172.21.15.143:0/145726963 --> v1:172.21.15.143:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff04415e1d0 con 0x7ff04406bae0 2023-11-26T07:20:07.923 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:20:07.919+0000 7ff03b7fe640 1 -- 172.21.15.143:0/145726963 <== mon.0 v1:172.21.15.143:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2137882523 0 0) 0x7ff034001680 con 0x7ff04406bae0 2023-11-26T07:20:07.923 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:20:07.919+0000 7ff049195640 1 --2- 172.21.15.143:0/145726963 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7ff04415e6f0 0x7ff0441610f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-26T07:20:07.923 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:20:07.919+0000 7ff03b7fe640 1 -- 172.21.15.143:0/145726963 >> v1:172.21.15.143:6789/0 conn(0x7ff04406bae0 legacy=0x7ff0440cd610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:20:07.923 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:20:07.919+0000 7ff03b7fe640 1 --2- 172.21.15.143:0/145726963 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7ff02c003b40 0x7ff02c003f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:20:07.923 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:20:07.919+0000 7ff03b7fe640 1 --2- 172.21.15.143:0/145726963 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7ff04415e6f0 0x7ff02c0047d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:20:07.923 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:20:07.919+0000 7ff03b7fe640 1 --2- 172.21.15.143:0/145726963 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7ff02c004d10 0x7ff02c005190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:20:07.923 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:20:07.919+0000 7ff03b7fe640 1 -- 172.21.15.143:0/145726963 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff04415def0 con 0x7ff02c004d10 2023-11-26T07:20:07.923 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:20:07.919+0000 7ff03b7fe640 1 -- 172.21.15.143:0/145726963 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff02c00bad0 con 0x7ff02c003b40 2023-11-26T07:20:07.923 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:20:07.919+0000 7ff03b7fe640 1 -- 172.21.15.143:0/145726963 --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff02c00bc90 con 0x7ff04415e6f0 2023-11-26T07:20:07.924 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:20:07.919+0000 7ff049195640 1 --2- 172.21.15.143:0/145726963 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7ff02c004d10 0x7ff02c005190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:20:07.924 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:20:07.919+0000 7ff048994640 1 --2- 172.21.15.143:0/145726963 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7ff02c003b40 0x7ff02c003f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:20:07.924 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:20:07.919+0000 7ff03bfff640 1 --2- 172.21.15.143:0/145726963 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7ff04415e6f0 0x7ff02c0047d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:20:07.924 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:20:07.919+0000 7ff049195640 1 -- 172.21.15.143:0/145726963 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7ff04415e6f0 msgr2=0x7ff02c0047d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-26T07:20:07.924 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:20:07.919+0000 7ff049195640 1 --2- 172.21.15.143:0/145726963 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7ff04415e6f0 0x7ff02c0047d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:20:07.924 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:20:07.919+0000 7ff049195640 1 -- 172.21.15.143:0/145726963 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7ff02c003b40 msgr2=0x7ff02c003f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-26T07:20:07.924 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:20:07.919+0000 7ff049195640 1 --2- 172.21.15.143:0/145726963 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7ff02c003b40 0x7ff02c003f40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:20:07.924 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:20:07.919+0000 7ff049195640 1 -- 172.21.15.143:0/145726963 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff03c009a70 con 0x7ff02c004d10 2023-11-26T07:20:07.924 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:20:07.919+0000 7ff03bfff640 1 --2- 172.21.15.143:0/145726963 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7ff04415e6f0 0x7ff02c0047d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-26T07:20:07.925 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:20:07.919+0000 7ff049195640 1 --2- 172.21.15.143:0/145726963 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7ff02c004d10 0x7ff02c005190 secure :-1 s=READY pgs=679 cs=0 l=1 rev1=1 crypto rx=0x7ff03c00e350 tx=0x7ff03c01cde0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=805719362b163acc server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:20:07.925 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:20:07.919+0000 7ff03b7fe640 1 -- 172.21.15.143:0/145726963 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff03c0a5df0 con 0x7ff02c004d10 2023-11-26T07:20:07.925 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:20:07.923+0000 7ff03b7fe640 1 -- 172.21.15.143:0/145726963 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff03c00f040 con 0x7ff02c004d10 2023-11-26T07:20:07.925 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:20:07.923+0000 7ff03b7fe640 1 -- 172.21.15.143:0/145726963 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff03c0ad640 con 0x7ff02c004d10 2023-11-26T07:20:07.926 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:20:07.923+0000 7ff04abf8640 1 -- 172.21.15.143:0/145726963 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7ff02c004d10 msgr2=0x7ff02c005190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:20:07.926 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:20:07.923+0000 7ff04abf8640 1 --2- 172.21.15.143:0/145726963 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7ff02c004d10 0x7ff02c005190 secure :-1 s=READY pgs=679 cs=0 l=1 rev1=1 crypto rx=0x7ff03c00e350 tx=0x7ff03c01cde0 comp rx=0 tx=0).stop 2023-11-26T07:20:07.926 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:20:07.923+0000 7ff049195640 1 -- 172.21.15.143:0/145726963 reap_dead start 2023-11-26T07:20:07.926 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:20:07.923+0000 7ff04abf8640 1 -- 172.21.15.143:0/145726963 shutdown_connections 2023-11-26T07:20:07.926 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:20:07.923+0000 7ff04abf8640 1 -- 172.21.15.143:0/145726963 >> 172.21.15.143:0/145726963 conn(0x7ff0440cf710 msgr2=0x7ff0440cfb20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:20:07.926 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:20:07.923+0000 7ff04abf8640 1 -- 172.21.15.143:0/145726963 shutdown_connections 2023-11-26T07:20:07.926 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:20:07.923+0000 7ff04abf8640 1 -- 172.21.15.143:0/145726963 wait complete. 2023-11-26T07:20:07.927 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:20:07.923+0000 7ff04abf8640 1 Processor -- start 2023-11-26T07:20:07.927 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:20:07.923+0000 7ff04abf8640 1 -- start start 2023-11-26T07:20:07.928 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:20:07.923+0000 7ff04abf8640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7ff02c004d10 0x7ff04414c1f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:20:07.929 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:20:07.923+0000 7ff04abf8640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7ff0441f6650 0x7ff0440c49b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:20:07.929 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:20:07.923+0000 7ff04abf8640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7ff04415c6a0 0x7ff04415cb50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:20:07.929 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:20:07.923+0000 7ff04abf8640 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff04415d090 con 0x7ff04415c6a0 2023-11-26T07:20:07.929 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:20:07.923+0000 7ff04abf8640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff0441f4170 con 0x7ff0441f6650 2023-11-26T07:20:07.929 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:20:07.923+0000 7ff04abf8640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff04414c730 con 0x7ff02c004d10 2023-11-26T07:20:07.929 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:20:07.923+0000 7ff049195640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7ff04415c6a0 0x7ff04415cb50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:20:07.929 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:20:07.923+0000 7ff048994640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7ff02c004d10 0x7ff04414c1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:20:07.929 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:20:07.923+0000 7ff049195640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7ff04415c6a0 0x7ff04415cb50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:59682/0 (socket says 172.21.15.143:59682) 2023-11-26T07:20:07.929 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:20:07.923+0000 7ff049195640 1 -- 172.21.15.143:0/2784184114 learned_addr learned my addr 172.21.15.143:0/2784184114 (peer_addr_for_me v2:172.21.15.143:0/0) 2023-11-26T07:20:07.929 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:20:07.923+0000 7ff03bfff640 1 --2- 172.21.15.143:0/2784184114 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7ff0441f6650 0x7ff0440c49b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:20:07.929 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:20:07.927+0000 7ff049195640 1 -- 172.21.15.143:0/2784184114 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7ff02c004d10 msgr2=0x7ff04414c1f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:20:07.930 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:20:07.927+0000 7ff049195640 1 --2- 172.21.15.143:0/2784184114 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7ff02c004d10 0x7ff04414c1f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:20:07.930 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:20:07.927+0000 7ff049195640 1 -- 172.21.15.143:0/2784184114 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7ff0441f6650 msgr2=0x7ff0440c49b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:20:07.930 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:20:07.927+0000 7ff049195640 1 --2- 172.21.15.143:0/2784184114 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7ff0441f6650 0x7ff0440c49b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:20:07.930 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:20:07.927+0000 7ff049195640 1 -- 172.21.15.143:0/2784184114 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff0441f6ae0 con 0x7ff04415c6a0 2023-11-26T07:20:07.930 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:20:07.927+0000 7ff049195640 1 --2- 172.21.15.143:0/2784184114 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7ff04415c6a0 0x7ff04415cb50 secure :-1 s=READY pgs=680 cs=0 l=1 rev1=1 crypto rx=0x7ff03c00e350 tx=0x7ff03c009750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:20:07.931 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:20:07.927+0000 7ff0397fa640 1 -- 172.21.15.143:0/2784184114 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff03c0bc070 con 0x7ff04415c6a0 2023-11-26T07:20:07.931 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:20:07.927+0000 7ff0397fa640 1 -- 172.21.15.143:0/2784184114 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff03c0aed70 con 0x7ff04415c6a0 2023-11-26T07:20:07.931 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:20:07.927+0000 7ff04abf8640 1 -- 172.21.15.143:0/2784184114 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff0441f6dc0 con 0x7ff04415c6a0 2023-11-26T07:20:07.931 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:20:07.927+0000 7ff0397fa640 1 -- 172.21.15.143:0/2784184114 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff03c0039a0 con 0x7ff04415c6a0 2023-11-26T07:20:07.931 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:20:07.927+0000 7ff04abf8640 1 -- 172.21.15.143:0/2784184114 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff0441f7380 con 0x7ff04415c6a0 2023-11-26T07:20:07.931 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:20:07.927+0000 7ff04abf8640 1 -- 172.21.15.143:0/2784184114 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff0441f4340 con 0x7ff04415c6a0 2023-11-26T07:20:07.932 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:20:07.927+0000 7ff0397fa640 1 -- 172.21.15.143:0/2784184114 <== mon.0 v2:172.21.15.143:3300/0 4 ==== mgrmap(e 7) v1 ==== 86005+0+0 (secure 0 0 0) 0x7ff03c0b4040 con 0x7ff04415c6a0 2023-11-26T07:20:07.932 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:20:07.927+0000 7ff0397fa640 1 --2- 172.21.15.143:0/2784184114 >> [v2:172.21.15.143:6832/2150557544,v1:172.21.15.143:6833/2150557544] conn(0x7ff030066620 0x7ff030068ae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:20:07.932 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:20:07.927+0000 7ff0397fa640 1 -- 172.21.15.143:0/2784184114 <== mon.0 v2:172.21.15.143:3300/0 5 ==== osd_map(45..45 src has 1..45) v4 ==== 5709+0+0 (secure 0 0 0) 0x7ff040001d30 con 0x7ff04415c6a0 2023-11-26T07:20:07.932 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:20:07.927+0000 7ff048994640 1 --2- 172.21.15.143:0/2784184114 >> [v2:172.21.15.143:6832/2150557544,v1:172.21.15.143:6833/2150557544] conn(0x7ff030066620 0x7ff030068ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:20:07.933 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:20:07.931+0000 7ff048994640 1 --2- 172.21.15.143:0/2784184114 >> [v2:172.21.15.143:6832/2150557544,v1:172.21.15.143:6833/2150557544] conn(0x7ff030066620 0x7ff030068ae0 secure :-1 s=READY pgs=327 cs=0 l=1 rev1=1 crypto rx=0x7ff04415da30 tx=0x7ff040001480 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:20:07.936 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:20:07.931+0000 7ff0397fa640 1 -- 172.21.15.143:0/2784184114 <== mon.0 v2:172.21.15.143:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172649 (secure 0 0 0) 0x7ff03c0eb520 con 0x7ff04415c6a0 2023-11-26T07:20:08.116 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:20:08.111+0000 7ff04abf8640 1 -- 172.21.15.143:0/2784184114 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7ff044067680 con 0x7ff04415c6a0 2023-11-26T07:20:08.117 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:20:08.115+0000 7ff0397fa640 1 -- 172.21.15.143:0/2784184114 <== mon.0 v2:172.21.15.143:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 26 v26) v1 ==== 94+0+4922 (secure 0 0 0) 0x7ff03c009de0 con 0x7ff04415c6a0 2023-11-26T07:20:08.117 INFO:teuthology.orchestra.run.smithi143.stdout: 2023-11-26T07:20:08.117 INFO:teuthology.orchestra.run.smithi143.stdout:{"epoch":26,"default_fscid":3,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5201,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.188:6835/319464848","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6833","nonce":319464848},{"type":"v1","addr":"172.21.15.188:6835","nonce":319464848}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":20},{"gid":5584,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.188:6834/2113408070","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6832","nonce":2113408070},{"type":"v1","addr":"172.21.15.188:6834","nonce":2113408070}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":20},{"gid":5745,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.143:6835/1616742593","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6834","nonce":1616742593},{"type":"v1","addr":"172.21.15.143:6835","nonce":1616742593}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":20}],"filesystems":[{"mdsmap":{"epoch":25,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-11-26T07:19:31.503587+0000","modified":"2023-11-26T07:19:34.523605+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":5748},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5748":{"gid":5748,"name":"c","rank":0,"incarnation":21,"state":"up:active","state_seq":3,"addr":"172.21.15.143:6837/2872033338","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6836","nonce":2872033338},{"type":"v1","addr":"172.21.15.143:6837","nonce":2872033338}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[7],"metadata_pool":6,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":3}]} 2023-11-26T07:20:08.117 INFO:teuthology.orchestra.run.smithi143.stderr:dumped fsmap epoch 26 2023-11-26T07:20:08.119 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:20:08.115+0000 7ff04abf8640 1 -- 172.21.15.143:0/2784184114 >> [v2:172.21.15.143:6832/2150557544,v1:172.21.15.143:6833/2150557544] conn(0x7ff030066620 msgr2=0x7ff030068ae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:20:08.119 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:20:08.115+0000 7ff04abf8640 1 --2- 172.21.15.143:0/2784184114 >> [v2:172.21.15.143:6832/2150557544,v1:172.21.15.143:6833/2150557544] conn(0x7ff030066620 0x7ff030068ae0 secure :-1 s=READY pgs=327 cs=0 l=1 rev1=1 crypto rx=0x7ff04415da30 tx=0x7ff040001480 comp rx=0 tx=0).stop 2023-11-26T07:20:08.119 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:20:08.115+0000 7ff04abf8640 1 -- 172.21.15.143:0/2784184114 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7ff04415c6a0 msgr2=0x7ff04415cb50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:20:08.119 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:20:08.115+0000 7ff04abf8640 1 --2- 172.21.15.143:0/2784184114 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7ff04415c6a0 0x7ff04415cb50 secure :-1 s=READY pgs=680 cs=0 l=1 rev1=1 crypto rx=0x7ff03c00e350 tx=0x7ff03c009750 comp rx=0 tx=0).stop 2023-11-26T07:20:08.119 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:20:08.115+0000 7ff04abf8640 1 -- 172.21.15.143:0/2784184114 shutdown_connections 2023-11-26T07:20:08.119 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:20:08.115+0000 7ff04abf8640 1 --2- 172.21.15.143:0/2784184114 >> [v2:172.21.15.143:6832/2150557544,v1:172.21.15.143:6833/2150557544] conn(0x7ff030066620 0x7ff030068ae0 unknown :-1 s=CLOSED pgs=327 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:20:08.119 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:20:08.115+0000 7ff04abf8640 1 --2- 172.21.15.143:0/2784184114 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7ff04415c6a0 0x7ff04415cb50 unknown :-1 s=CLOSED pgs=680 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:20:08.120 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:20:08.115+0000 7ff04abf8640 1 --2- 172.21.15.143:0/2784184114 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7ff0441f6650 0x7ff0440c49b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:20:08.120 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:20:08.115+0000 7ff04abf8640 1 --2- 172.21.15.143:0/2784184114 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7ff02c004d10 0x7ff04414c1f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:20:08.120 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:20:08.115+0000 7ff04abf8640 1 -- 172.21.15.143:0/2784184114 >> 172.21.15.143:0/2784184114 conn(0x7ff0440cf710 msgr2=0x7ff0440af890 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:20:08.120 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:20:08.115+0000 7ff04abf8640 1 -- 172.21.15.143:0/2784184114 shutdown_connections 2023-11-26T07:20:08.120 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:20:08.115+0000 7ff04abf8640 1 -- 172.21.15.143:0/2784184114 wait complete. 2023-11-26T07:20:08.138 DEBUG:teuthology.orchestra.run.smithi143:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.c.asok --format=json session ls 2023-11-26T07:20:08.297 INFO:teuthology.orchestra.run.smithi143.stdout:[{"id":6126,"entity":{"name":{"type":"client","num":6126},"addr":{"type":"any","addr":"172.21.15.143:0","nonce":3879234400}},"state":"open","num_leases":1,"num_caps":7,"request_load_avg":14,"uptime":1.067950145,"requests_in_flight":2,"num_completed_requests":0,"num_completed_flushes":1,"reconnecting":false,"recall_caps":{"value":0,"halflife":60},"release_caps":{"value":0,"halflife":60},"recall_caps_throttle":{"value":0,"halflife":1.5},"recall_caps_throttle2o":{"value":0,"halflife":0.5},"session_cache_liveness":{"value":21.94647358558727,"halflife":300},"cap_acquisition":{"value":0,"halflife":30},"last_trim_completed_requests_tid":14,"last_trim_completed_flushes_tid":2,"delegated_inos":[{"start":"0x10000000005","length":500}],"inst":"client.6126 172.21.15.143:0/3879234400","completed_requests":[],"prealloc_inos":[{"start":"0x10000000005","length":996}],"client_metadata":{"client_features":{"feature_bits":"0x00000000003fffff"},"metric_spec":{"metric_flags":{"feature_bits":"0x000000000000ffff"}},"ceph_sha1":"44c2237fe77713214148c1fd622c872f346f1b99","ceph_version":"ceph version 18.0.0-7484-g44c2237f (44c2237fe77713214148c1fd622c872f346f1b99) reef (dev)","entity_id":"y","hostname":"smithi143","pid":"38097","root":"/"}}] 2023-11-26T07:20:08.297 DEBUG:tasks.cephfs.filesystem:_json_asok output [ { "cap_acquisition": { "halflife": 30, "value": 0 }, "client_metadata": { "ceph_sha1": "44c2237fe77713214148c1fd622c872f346f1b99", "ceph_version": "ceph version 18.0.0-7484-g44c2237f (44c2237fe77713214148c1fd622c872f346f1b99) reef (dev)", "client_features": { "feature_bits": "0x00000000003fffff" }, "entity_id": "y", "hostname": "smithi143", "metric_spec": { "metric_flags": { "feature_bits": "0x000000000000ffff" } }, "pid": "38097", "root": "/" }, "completed_requests": [], "delegated_inos": [ { "length": 500, "start": "0x10000000005" } ], "entity": { "addr": { "addr": "172.21.15.143:0", "nonce": 3879234400, "type": "any" }, "name": { "num": 6126, "type": "client" } }, "id": 6126, "inst": "client.6126 172.21.15.143:0/3879234400", "last_trim_completed_flushes_tid": 2, "last_trim_completed_requests_tid": 14, "num_caps": 7, "num_completed_flushes": 1, "num_completed_requests": 0, "num_leases": 1, "prealloc_inos": [ { "length": 996, "start": "0x10000000005" } ], "recall_caps": { "halflife": 60, "value": 0 }, "recall_caps_throttle": { "halflife": 1.5, "value": 0 }, "recall_caps_throttle2o": { "halflife": 0.5, "value": 0 }, "reconnecting": false, "release_caps": { "halflife": 60, "value": 0 }, "request_load_avg": 14, "requests_in_flight": 2, "session_cache_liveness": { "halflife": 300, "value": 21.94647358558727 }, "state": "open", "uptime": 1.067950145 } ] 2023-11-26T07:20:08.297 DEBUG:tasks.ceph_test_case:wait_until_true: waiting (timeout=90 retry_count=0)... 2023-11-26T07:20:13.298 DEBUG:teuthology.orchestra.run.smithi143:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-11-26T07:20:13.448 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:20:13.443+0000 7f23f2f72640 1 Processor -- start 2023-11-26T07:20:13.448 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:20:13.443+0000 7f23f2f72640 1 -- start start 2023-11-26T07:20:13.450 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:20:13.447+0000 7f23f2f72640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f23ec160aa0 0x7f23ec0b2b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:20:13.450 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:20:13.447+0000 7f23f2f72640 1 -- --> v1:172.21.15.143:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f23ec0d4140 con 0x7f23ec0d14e0 2023-11-26T07:20:13.450 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:20:13.447+0000 7f23f2f72640 1 -- --> v1:172.21.15.188:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f23ec0d3930 con 0x7f23ec0b2240 2023-11-26T07:20:13.450 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:20:13.447+0000 7f23f2f72640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f23ec0ce440 con 0x7f23ec160aa0 2023-11-26T07:20:13.450 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:20:13.447+0000 7f23f150f640 1 --1- >> v1:172.21.15.143:6789/0 conn(0x7f23ec0d14e0 0x7f23ec0ce860 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.143:6789/0 says I am v1:172.21.15.143:56598/0 (socket says 172.21.15.143:56598) 2023-11-26T07:20:13.450 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:20:13.447+0000 7f23f150f640 1 -- 172.21.15.143:0/1192040565 learned_addr learned my addr 172.21.15.143:0/1192040565 (peer_addr_for_me v1:172.21.15.143:0/0) 2023-11-26T07:20:13.450 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:20:13.447+0000 7f23f0d0e640 1 --2- 172.21.15.143:0/1192040565 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f23ec160aa0 0x7f23ec0b2b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:20:13.451 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:20:13.447+0000 7f23f0d0e640 1 -- 172.21.15.143:0/1192040565 >> v1:172.21.15.188:6789/0 conn(0x7f23ec0b2240 legacy=0x7f23ec0d0590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:20:13.451 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:20:13.447+0000 7f23f0d0e640 1 -- 172.21.15.143:0/1192040565 >> v1:172.21.15.143:6789/0 conn(0x7f23ec0d14e0 legacy=0x7f23ec0ce860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:20:13.451 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:20:13.447+0000 7f23f0d0e640 1 -- 172.21.15.143:0/1192040565 --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f23ec0cddc0 con 0x7f23ec160aa0 2023-11-26T07:20:13.454 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:20:13.451+0000 7f23f0d0e640 1 --2- 172.21.15.143:0/1192040565 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f23ec160aa0 0x7f23ec0b2b80 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7f23e00097f0 tx=0x7f23e001c400 comp rx=0 tx=0).ready entity=mon.2 client_cookie=df65cfddacd025d0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:20:13.454 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:20:13.451+0000 7f23eb7fe640 1 -- 172.21.15.143:0/1192040565 <== mon.2 v2:172.21.15.188:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f23e00ab600 con 0x7f23ec160aa0 2023-11-26T07:20:13.454 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:20:13.451+0000 7f23eb7fe640 1 -- 172.21.15.143:0/1192040565 <== mon.2 v2:172.21.15.188:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f23e00ab760 con 0x7f23ec160aa0 2023-11-26T07:20:13.454 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:20:13.451+0000 7f23eb7fe640 1 -- 172.21.15.143:0/1192040565 <== mon.2 v2:172.21.15.188:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f23e00abaf0 con 0x7f23ec160aa0 2023-11-26T07:20:13.455 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:20:13.451+0000 7f23f2f72640 1 -- 172.21.15.143:0/1192040565 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f23ec160aa0 msgr2=0x7f23ec0b2b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:20:13.455 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:20:13.451+0000 7f23f2f72640 1 --2- 172.21.15.143:0/1192040565 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f23ec160aa0 0x7f23ec0b2b80 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7f23e00097f0 tx=0x7f23e001c400 comp rx=0 tx=0).stop 2023-11-26T07:20:13.455 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:20:13.451+0000 7f23f2f72640 1 -- 172.21.15.143:0/1192040565 shutdown_connections 2023-11-26T07:20:13.455 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:20:13.451+0000 7f23f2f72640 1 --2- 172.21.15.143:0/1192040565 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f23ec160aa0 0x7f23ec0b2b80 unknown :-1 s=CLOSED pgs=94 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:20:13.455 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:20:13.451+0000 7f23f2f72640 1 -- 172.21.15.143:0/1192040565 >> 172.21.15.143:0/1192040565 conn(0x7f23ec09c550 msgr2=0x7f23ec0d3220 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:20:13.456 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:20:13.451+0000 7f23f2f72640 1 -- 172.21.15.143:0/1192040565 shutdown_connections 2023-11-26T07:20:13.456 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:20:13.451+0000 7f23f2f72640 1 -- 172.21.15.143:0/1192040565 wait complete. 2023-11-26T07:20:13.456 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:20:13.455+0000 7f23f2f72640 1 Processor -- start 2023-11-26T07:20:13.457 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:20:13.455+0000 7f23f2f72640 1 -- start start 2023-11-26T07:20:13.458 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:20:13.455+0000 7f23f2f72640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f23ec0b2240 0x7f23ec160e80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:20:13.458 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:20:13.455+0000 7f23f2f72640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f23ec0d14e0 0x7f23ec15fd30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:20:13.459 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:20:13.455+0000 7f23f2f72640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f23ec160aa0 0x7f23ec0c7da0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:20:13.459 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:20:13.455+0000 7f23f2f72640 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f23ec0a0e50 con 0x7f23ec0d14e0 2023-11-26T07:20:13.459 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:20:13.455+0000 7f23f2f72640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f23ec0a0b70 con 0x7f23ec160aa0 2023-11-26T07:20:13.459 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:20:13.455+0000 7f23f2f72640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f23ec0cbc30 con 0x7f23ec0b2240 2023-11-26T07:20:13.459 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:20:13.455+0000 7f23ebfff640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f23ec0d14e0 0x7f23ec15fd30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:20:13.459 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:20:13.455+0000 7f23ebfff640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f23ec0d14e0 0x7f23ec15fd30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:59698/0 (socket says 172.21.15.143:59698) 2023-11-26T07:20:13.459 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:20:13.455+0000 7f23ebfff640 1 -- 172.21.15.143:0/1878438102 learned_addr learned my addr 172.21.15.143:0/1878438102 (peer_addr_for_me v2:172.21.15.143:0/0) 2023-11-26T07:20:13.459 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:20:13.455+0000 7f23f150f640 1 --2- 172.21.15.143:0/1878438102 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f23ec160aa0 0x7f23ec0c7da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:20:13.459 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:20:13.455+0000 7f23f0d0e640 1 --2- 172.21.15.143:0/1878438102 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f23ec0b2240 0x7f23ec160e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:20:13.459 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:20:13.455+0000 7f23ebfff640 1 -- 172.21.15.143:0/1878438102 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f23ec0b2240 msgr2=0x7f23ec160e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:20:13.459 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:20:13.455+0000 7f23ebfff640 1 --2- 172.21.15.143:0/1878438102 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f23ec0b2240 0x7f23ec160e80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:20:13.459 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:20:13.455+0000 7f23ebfff640 1 -- 172.21.15.143:0/1878438102 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f23ec160aa0 msgr2=0x7f23ec0c7da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:20:13.460 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:20:13.455+0000 7f23ebfff640 1 --2- 172.21.15.143:0/1878438102 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f23ec160aa0 0x7f23ec0c7da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:20:13.460 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:20:13.455+0000 7f23ebfff640 1 -- 172.21.15.143:0/1878438102 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f23e0009420 con 0x7f23ec0d14e0 2023-11-26T07:20:13.460 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:20:13.455+0000 7f23f150f640 1 --2- 172.21.15.143:0/1878438102 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f23ec160aa0 0x7f23ec0c7da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-26T07:20:13.461 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:20:13.455+0000 7f23ebfff640 1 --2- 172.21.15.143:0/1878438102 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f23ec0d14e0 0x7f23ec15fd30 secure :-1 s=READY pgs=682 cs=0 l=1 rev1=1 crypto rx=0x7f23d800c3c0 tx=0x7f23d800c880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:20:13.461 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:20:13.455+0000 7f23e97fa640 1 -- 172.21.15.143:0/1878438102 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f23d80052d0 con 0x7f23ec0d14e0 2023-11-26T07:20:13.461 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:20:13.455+0000 7f23e97fa640 1 -- 172.21.15.143:0/1878438102 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f23d8005430 con 0x7f23ec0d14e0 2023-11-26T07:20:13.461 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:20:13.455+0000 7f23f2f72640 1 -- 172.21.15.143:0/1878438102 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f23ec0cbf10 con 0x7f23ec0d14e0 2023-11-26T07:20:13.461 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:20:13.455+0000 7f23e97fa640 1 -- 172.21.15.143:0/1878438102 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f23d8003840 con 0x7f23ec0d14e0 2023-11-26T07:20:13.461 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:20:13.455+0000 7f23f2f72640 1 -- 172.21.15.143:0/1878438102 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f23ec160270 con 0x7f23ec0d14e0 2023-11-26T07:20:13.461 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:20:13.455+0000 7f23f2f72640 1 -- 172.21.15.143:0/1878438102 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f23ec0cc430 con 0x7f23ec0d14e0 2023-11-26T07:20:13.462 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:20:13.459+0000 7f23e97fa640 1 -- 172.21.15.143:0/1878438102 <== mon.0 v2:172.21.15.143:3300/0 4 ==== mgrmap(e 8) v1 ==== 86056+0+0 (secure 0 0 0) 0x7f23d80039a0 con 0x7f23ec0d14e0 2023-11-26T07:20:13.462 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:20:13.459+0000 7f23e97fa640 1 --2- 172.21.15.143:0/1878438102 >> [v2:172.21.15.143:6832/2150557544,v1:172.21.15.143:6833/2150557544] conn(0x7f23d00666b0 0x7f23d0068b70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:20:13.462 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:20:13.459+0000 7f23e97fa640 1 -- 172.21.15.143:0/1878438102 <== mon.0 v2:172.21.15.143:3300/0 5 ==== osd_map(45..45 src has 1..45) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f23d8083710 con 0x7f23ec0d14e0 2023-11-26T07:20:13.462 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:20:13.459+0000 7f23f0d0e640 1 --2- 172.21.15.143:0/1878438102 >> [v2:172.21.15.143:6832/2150557544,v1:172.21.15.143:6833/2150557544] conn(0x7f23d00666b0 0x7f23d0068b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:20:13.462 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:20:13.459+0000 7f23f0d0e640 1 --2- 172.21.15.143:0/1878438102 >> [v2:172.21.15.143:6832/2150557544,v1:172.21.15.143:6833/2150557544] conn(0x7f23d00666b0 0x7f23d0068b70 secure :-1 s=READY pgs=328 cs=0 l=1 rev1=1 crypto rx=0x7f23e00af000 tx=0x7f23e00a7000 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:20:13.466 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:20:13.463+0000 7f23e97fa640 1 -- 172.21.15.143:0/1878438102 <== mon.0 v2:172.21.15.143:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172649 (secure 0 0 0) 0x7f23ec0cc430 con 0x7f23ec0d14e0 2023-11-26T07:20:13.647 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:20:13.643+0000 7f23f2f72640 1 -- 172.21.15.143:0/1878438102 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f23ec0cc430 con 0x7f23ec0d14e0 2023-11-26T07:20:13.648 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:20:13.643+0000 7f23e97fa640 1 -- 172.21.15.143:0/1878438102 <== mon.0 v2:172.21.15.143:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 26 v26) v1 ==== 94+0+4922 (secure 0 0 0) 0x7f23ec0cc430 con 0x7f23ec0d14e0 2023-11-26T07:20:13.648 INFO:teuthology.orchestra.run.smithi143.stdout: 2023-11-26T07:20:13.648 INFO:teuthology.orchestra.run.smithi143.stdout:{"epoch":26,"default_fscid":3,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5201,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.188:6835/319464848","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6833","nonce":319464848},{"type":"v1","addr":"172.21.15.188:6835","nonce":319464848}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":20},{"gid":5584,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.188:6834/2113408070","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6832","nonce":2113408070},{"type":"v1","addr":"172.21.15.188:6834","nonce":2113408070}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":20},{"gid":5745,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.143:6835/1616742593","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6834","nonce":1616742593},{"type":"v1","addr":"172.21.15.143:6835","nonce":1616742593}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":20}],"filesystems":[{"mdsmap":{"epoch":25,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-11-26T07:19:31.503587+0000","modified":"2023-11-26T07:19:34.523605+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":5748},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5748":{"gid":5748,"name":"c","rank":0,"incarnation":21,"state":"up:active","state_seq":3,"addr":"172.21.15.143:6837/2872033338","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6836","nonce":2872033338},{"type":"v1","addr":"172.21.15.143:6837","nonce":2872033338}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[7],"metadata_pool":6,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":3}]} 2023-11-26T07:20:13.649 INFO:teuthology.orchestra.run.smithi143.stderr:dumped fsmap epoch 26 2023-11-26T07:20:13.650 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:20:13.647+0000 7f23f2f72640 1 -- 172.21.15.143:0/1878438102 >> [v2:172.21.15.143:6832/2150557544,v1:172.21.15.143:6833/2150557544] conn(0x7f23d00666b0 msgr2=0x7f23d0068b70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:20:13.650 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:20:13.647+0000 7f23f2f72640 1 --2- 172.21.15.143:0/1878438102 >> [v2:172.21.15.143:6832/2150557544,v1:172.21.15.143:6833/2150557544] conn(0x7f23d00666b0 0x7f23d0068b70 secure :-1 s=READY pgs=328 cs=0 l=1 rev1=1 crypto rx=0x7f23e00af000 tx=0x7f23e00a7000 comp rx=0 tx=0).stop 2023-11-26T07:20:13.650 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:20:13.647+0000 7f23f2f72640 1 -- 172.21.15.143:0/1878438102 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f23ec0d14e0 msgr2=0x7f23ec15fd30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:20:13.651 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:20:13.647+0000 7f23f2f72640 1 --2- 172.21.15.143:0/1878438102 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f23ec0d14e0 0x7f23ec15fd30 secure :-1 s=READY pgs=682 cs=0 l=1 rev1=1 crypto rx=0x7f23d800c3c0 tx=0x7f23d800c880 comp rx=0 tx=0).stop 2023-11-26T07:20:13.651 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:20:13.647+0000 7f23f2f72640 1 -- 172.21.15.143:0/1878438102 shutdown_connections 2023-11-26T07:20:13.651 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:20:13.647+0000 7f23f2f72640 1 --2- 172.21.15.143:0/1878438102 >> [v2:172.21.15.143:6832/2150557544,v1:172.21.15.143:6833/2150557544] conn(0x7f23d00666b0 0x7f23d0068b70 unknown :-1 s=CLOSED pgs=328 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:20:13.651 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:20:13.647+0000 7f23f2f72640 1 --2- 172.21.15.143:0/1878438102 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f23ec160aa0 0x7f23ec0c7da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:20:13.651 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:20:13.647+0000 7f23f2f72640 1 --2- 172.21.15.143:0/1878438102 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f23ec0d14e0 0x7f23ec15fd30 unknown :-1 s=CLOSED pgs=682 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:20:13.651 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:20:13.647+0000 7f23f2f72640 1 --2- 172.21.15.143:0/1878438102 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f23ec0b2240 0x7f23ec160e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:20:13.651 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:20:13.647+0000 7f23f2f72640 1 -- 172.21.15.143:0/1878438102 >> 172.21.15.143:0/1878438102 conn(0x7f23ec09c550 msgr2=0x7f23ec0b3240 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:20:13.651 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:20:13.647+0000 7f23f2f72640 1 -- 172.21.15.143:0/1878438102 shutdown_connections 2023-11-26T07:20:13.651 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:20:13.647+0000 7f23f2f72640 1 -- 172.21.15.143:0/1878438102 wait complete. 2023-11-26T07:20:13.669 DEBUG:teuthology.orchestra.run.smithi143:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.c.asok --format=json session ls 2023-11-26T07:20:13.827 INFO:teuthology.orchestra.run.smithi143.stdout:[{"id":6126,"entity":{"name":{"type":"client","num":6126},"addr":{"type":"any","addr":"172.21.15.143:0","nonce":3879234400}},"state":"open","num_leases":1,"num_caps":7,"request_load_avg":13,"uptime":6.5997100590000004,"requests_in_flight":0,"num_completed_requests":2,"num_completed_flushes":1,"reconnecting":false,"recall_caps":{"value":0,"halflife":60},"release_caps":{"value":0,"halflife":60},"recall_caps_throttle":{"value":0,"halflife":1.5},"recall_caps_throttle2o":{"value":0,"halflife":0.5},"session_cache_liveness":{"value":21.667758897880717,"halflife":300},"cap_acquisition":{"value":0,"halflife":30},"last_trim_completed_requests_tid":14,"last_trim_completed_flushes_tid":2,"delegated_inos":[{"start":"0x10000000005","length":500}],"inst":"client.6126 172.21.15.143:0/3879234400","completed_requests":[{"tid":14,"created_ino":"0x0"},{"tid":15,"created_ino":"0x100000001f9"}],"prealloc_inos":[{"start":"0x10000000005","length":500},{"start":"0x100000001fa","length":1000}],"client_metadata":{"client_features":{"feature_bits":"0x00000000003fffff"},"metric_spec":{"metric_flags":{"feature_bits":"0x000000000000ffff"}},"ceph_sha1":"44c2237fe77713214148c1fd622c872f346f1b99","ceph_version":"ceph version 18.0.0-7484-g44c2237f (44c2237fe77713214148c1fd622c872f346f1b99) reef (dev)","entity_id":"y","hostname":"smithi143","pid":"38097","root":"/"}}] 2023-11-26T07:20:13.828 DEBUG:tasks.cephfs.filesystem:_json_asok output [ { "cap_acquisition": { "halflife": 30, "value": 0 }, "client_metadata": { "ceph_sha1": "44c2237fe77713214148c1fd622c872f346f1b99", "ceph_version": "ceph version 18.0.0-7484-g44c2237f (44c2237fe77713214148c1fd622c872f346f1b99) reef (dev)", "client_features": { "feature_bits": "0x00000000003fffff" }, "entity_id": "y", "hostname": "smithi143", "metric_spec": { "metric_flags": { "feature_bits": "0x000000000000ffff" } }, "pid": "38097", "root": "/" }, "completed_requests": [ { "created_ino": "0x0", "tid": 14 }, { "created_ino": "0x100000001f9", "tid": 15 } ], "delegated_inos": [ { "length": 500, "start": "0x10000000005" } ], "entity": { "addr": { "addr": "172.21.15.143:0", "nonce": 3879234400, "type": "any" }, "name": { "num": 6126, "type": "client" } }, "id": 6126, "inst": "client.6126 172.21.15.143:0/3879234400", "last_trim_completed_flushes_tid": 2, "last_trim_completed_requests_tid": 14, "num_caps": 7, "num_completed_flushes": 1, "num_completed_requests": 2, "num_leases": 1, "prealloc_inos": [ { "length": 500, "start": "0x10000000005" }, { "length": 1000, "start": "0x100000001fa" } ], "recall_caps": { "halflife": 60, "value": 0 }, "recall_caps_throttle": { "halflife": 1.5, "value": 0 }, "recall_caps_throttle2o": { "halflife": 0.5, "value": 0 }, "reconnecting": false, "release_caps": { "halflife": 60, "value": 0 }, "request_load_avg": 13, "requests_in_flight": 0, "session_cache_liveness": { "halflife": 300, "value": 21.667758897880717 }, "state": "open", "uptime": 6.599710059 } ] 2023-11-26T07:20:13.828 DEBUG:tasks.ceph_test_case:wait_until_true: waiting (timeout=90 retry_count=0)... 2023-11-26T07:20:18.829 DEBUG:teuthology.orchestra.run.smithi143:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-11-26T07:20:18.978 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:20:18.974+0000 7f733c3a1640 1 Processor -- start 2023-11-26T07:20:18.978 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:20:18.974+0000 7f733c3a1640 1 -- start start 2023-11-26T07:20:18.980 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:20:18.974+0000 7f733c3a1640 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7f73340cf280 0x7f73340cf710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:20:18.980 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:20:18.974+0000 7f733c3a1640 1 -- --> v1:172.21.15.143:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f73340cc730 con 0x7f7334143ac0 2023-11-26T07:20:18.980 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:20:18.974+0000 7f733c3a1640 1 -- --> v1:172.21.15.188:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7334070ee0 con 0x7f73340cd1b0 2023-11-26T07:20:18.980 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:20:18.974+0000 7f733c3a1640 1 -- --> v2:172.21.15.143:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7334143ea0 con 0x7f73340cf280 2023-11-26T07:20:18.980 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:20:18.974+0000 7f733993c640 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7f73340cf280 0x7f73340cf710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:20:18.980 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:20:18.974+0000 7f733a13d640 1 --1- >> v1:172.21.15.143:6789/0 conn(0x7f7334143ac0 0x7f7334070370 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.143:6789/0 says I am v1:172.21.15.143:50702/0 (socket says 172.21.15.143:50702) 2023-11-26T07:20:18.980 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:20:18.974+0000 7f733a13d640 1 -- 172.21.15.143:0/3253806907 learned_addr learned my addr 172.21.15.143:0/3253806907 (peer_addr_for_me v1:172.21.15.143:0/0) 2023-11-26T07:20:18.980 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:20:18.974+0000 7f733993c640 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7f73340cf280 0x7f73340cf710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:44680/0 (socket says 172.21.15.143:44680) 2023-11-26T07:20:18.981 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:20:18.974+0000 7f733993c640 1 -- 172.21.15.143:0/3253806907 >> v1:172.21.15.188:6789/0 conn(0x7f73340cd1b0 legacy=0x7f73340cd5c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-26T07:20:18.981 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:20:18.974+0000 7f733993c640 1 -- 172.21.15.143:0/3253806907 >> v1:172.21.15.143:6789/0 conn(0x7f7334143ac0 legacy=0x7f7334070370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:20:18.981 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:20:18.974+0000 7f733993c640 1 -- 172.21.15.143:0/3253806907 --> v2:172.21.15.143:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f73340711c0 con 0x7f73340cf280 2023-11-26T07:20:18.981 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:20:18.974+0000 7f733913b640 1 -- 172.21.15.143:0/3253806907 <== mon.0 v1:172.21.15.143:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 795463817 0 0) 0x7f73340cc730 con 0x7f7334143ac0 2023-11-26T07:20:18.983 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:20:18.978+0000 7f733993c640 1 --2- 172.21.15.143:0/3253806907 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f73340cf280 0x7f73340cf710 secure :-1 s=READY pgs=684 cs=0 l=1 rev1=1 crypto rx=0x7f73240097c0 tx=0x7f732401c440 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4f0aa7994cd9568c server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:20:18.983 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:20:18.978+0000 7f733913b640 1 -- 172.21.15.143:0/3253806907 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7324004230 con 0x7f73340cf280 2023-11-26T07:20:18.983 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:20:18.978+0000 7f733913b640 1 -- 172.21.15.143:0/3253806907 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f73240a7050 con 0x7f73340cf280 2023-11-26T07:20:18.983 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:20:18.978+0000 7f733913b640 1 -- 172.21.15.143:0/3253806907 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7324003620 con 0x7f73340cf280 2023-11-26T07:20:18.984 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:20:18.978+0000 7f733c3a1640 1 -- 172.21.15.143:0/3253806907 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f73340cf280 msgr2=0x7f73340cf710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:20:18.984 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:20:18.978+0000 7f733c3a1640 1 --2- 172.21.15.143:0/3253806907 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f73340cf280 0x7f73340cf710 secure :-1 s=READY pgs=684 cs=0 l=1 rev1=1 crypto rx=0x7f73240097c0 tx=0x7f732401c440 comp rx=0 tx=0).stop 2023-11-26T07:20:18.984 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:20:18.978+0000 7f733c3a1640 1 -- 172.21.15.143:0/3253806907 shutdown_connections 2023-11-26T07:20:18.984 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:20:18.978+0000 7f733c3a1640 1 --2- 172.21.15.143:0/3253806907 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f73340cf280 0x7f73340cf710 unknown :-1 s=CLOSED pgs=684 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:20:18.985 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:20:18.978+0000 7f733c3a1640 1 -- 172.21.15.143:0/3253806907 >> 172.21.15.143:0/3253806907 conn(0x7f733406a890 msgr2=0x7f733406aca0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:20:18.985 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:20:18.978+0000 7f733c3a1640 1 -- 172.21.15.143:0/3253806907 shutdown_connections 2023-11-26T07:20:18.985 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:20:18.978+0000 7f733c3a1640 1 -- 172.21.15.143:0/3253806907 wait complete. 2023-11-26T07:20:18.986 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:20:18.982+0000 7f733c3a1640 1 Processor -- start 2023-11-26T07:20:18.986 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:20:18.982+0000 7f733c3a1640 1 -- start start 2023-11-26T07:20:18.988 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:20:18.982+0000 7f733c3a1640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f73340cd1b0 0x7f7334140750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:20:18.988 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:20:18.982+0000 7f733c3a1640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f73340cf280 0x7f73340c3e60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:20:18.988 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:20:18.982+0000 7f733c3a1640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f7334143ac0 0x7f73341f5ec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:20:18.988 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:20:18.982+0000 7f733c3a1640 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f73341f6400 con 0x7f73340cd1b0 2023-11-26T07:20:18.988 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:20:18.982+0000 7f733c3a1640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f733413e9c0 con 0x7f7334143ac0 2023-11-26T07:20:18.989 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:20:18.982+0000 7f733c3a1640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f733413eca0 con 0x7f73340cf280 2023-11-26T07:20:18.989 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:20:18.982+0000 7f733a13d640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f73340cd1b0 0x7f7334140750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:20:18.989 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:20:18.982+0000 7f733a13d640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f73340cd1b0 0x7f7334140750 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:44688/0 (socket says 172.21.15.143:44688) 2023-11-26T07:20:18.989 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:20:18.982+0000 7f733a13d640 1 -- 172.21.15.143:0/4273663587 learned_addr learned my addr 172.21.15.143:0/4273663587 (peer_addr_for_me v2:172.21.15.143:0/0) 2023-11-26T07:20:18.989 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:20:18.982+0000 7f733a93e640 1 --2- 172.21.15.143:0/4273663587 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f7334143ac0 0x7f73341f5ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:20:18.989 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:20:18.982+0000 7f733993c640 1 --2- 172.21.15.143:0/4273663587 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f73340cf280 0x7f73340c3e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:20:18.989 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:20:18.982+0000 7f733a13d640 1 -- 172.21.15.143:0/4273663587 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f73340cf280 msgr2=0x7f73340c3e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:20:18.989 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:20:18.982+0000 7f733a13d640 1 --2- 172.21.15.143:0/4273663587 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f73340cf280 0x7f73340c3e60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:20:18.989 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:20:18.982+0000 7f733a13d640 1 -- 172.21.15.143:0/4273663587 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f7334143ac0 msgr2=0x7f73341f5ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:20:18.989 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:20:18.982+0000 7f733a13d640 1 --2- 172.21.15.143:0/4273663587 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f7334143ac0 0x7f73341f5ec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:20:18.989 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:20:18.982+0000 7f733a13d640 1 -- 172.21.15.143:0/4273663587 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7324009420 con 0x7f73340cd1b0 2023-11-26T07:20:18.989 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:20:18.982+0000 7f733a13d640 1 --2- 172.21.15.143:0/4273663587 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f73340cd1b0 0x7f7334140750 secure :-1 s=READY pgs=685 cs=0 l=1 rev1=1 crypto rx=0x7f733000c480 tx=0x7f733000c940 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:20:18.990 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:20:18.982+0000 7f732affd640 1 -- 172.21.15.143:0/4273663587 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f73300074e0 con 0x7f73340cd1b0 2023-11-26T07:20:18.990 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:20:18.982+0000 7f732affd640 1 -- 172.21.15.143:0/4273663587 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7330007660 con 0x7f73340cd1b0 2023-11-26T07:20:18.990 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:20:18.982+0000 7f732affd640 1 -- 172.21.15.143:0/4273663587 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7330004a30 con 0x7f73340cd1b0 2023-11-26T07:20:18.990 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:20:18.982+0000 7f733c3a1640 1 -- 172.21.15.143:0/4273663587 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f733413ef80 con 0x7f73340cd1b0 2023-11-26T07:20:18.990 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:20:18.982+0000 7f733c3a1640 1 -- 172.21.15.143:0/4273663587 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f73340c4570 con 0x7f73340cd1b0 2023-11-26T07:20:18.990 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:20:18.986+0000 7f733c3a1640 1 -- 172.21.15.143:0/4273663587 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f733413f4a0 con 0x7f73340cd1b0 2023-11-26T07:20:18.990 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:20:18.986+0000 7f732affd640 1 -- 172.21.15.143:0/4273663587 <== mon.0 v2:172.21.15.143:3300/0 4 ==== mgrmap(e 8) v1 ==== 86056+0+0 (secure 0 0 0) 0x7f7330004b90 con 0x7f73340cd1b0 2023-11-26T07:20:18.991 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:20:18.986+0000 7f732affd640 1 --2- 172.21.15.143:0/4273663587 >> [v2:172.21.15.143:6832/2150557544,v1:172.21.15.143:6833/2150557544] conn(0x7f73180666b0 0x7f7318068b70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:20:18.991 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:20:18.986+0000 7f732affd640 1 -- 172.21.15.143:0/4273663587 <== mon.0 v2:172.21.15.143:3300/0 5 ==== osd_map(45..45 src has 1..45) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f73300833f0 con 0x7f73340cd1b0 2023-11-26T07:20:18.991 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:20:18.986+0000 7f733993c640 1 --2- 172.21.15.143:0/4273663587 >> [v2:172.21.15.143:6832/2150557544,v1:172.21.15.143:6833/2150557544] conn(0x7f73180666b0 0x7f7318068b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:20:18.992 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:20:18.986+0000 7f733993c640 1 --2- 172.21.15.143:0/4273663587 >> [v2:172.21.15.143:6832/2150557544,v1:172.21.15.143:6833/2150557544] conn(0x7f73180666b0 0x7f7318068b70 secure :-1 s=READY pgs=329 cs=0 l=1 rev1=1 crypto rx=0x7f73240ad000 tx=0x7f7324008040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:20:18.995 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:20:18.990+0000 7f732affd640 1 -- 172.21.15.143:0/4273663587 <== mon.0 v2:172.21.15.143:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172649 (secure 0 0 0) 0x7f733413f4a0 con 0x7f73340cd1b0 2023-11-26T07:20:19.176 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:20:19.170+0000 7f733c3a1640 1 -- 172.21.15.143:0/4273663587 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f733413f4a0 con 0x7f73340cd1b0 2023-11-26T07:20:19.177 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:20:19.174+0000 7f732affd640 1 -- 172.21.15.143:0/4273663587 <== mon.0 v2:172.21.15.143:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 26 v26) v1 ==== 94+0+4922 (secure 0 0 0) 0x7f733413f4a0 con 0x7f73340cd1b0 2023-11-26T07:20:19.177 INFO:teuthology.orchestra.run.smithi143.stdout: 2023-11-26T07:20:19.177 INFO:teuthology.orchestra.run.smithi143.stdout:{"epoch":26,"default_fscid":3,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5201,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.188:6835/319464848","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6833","nonce":319464848},{"type":"v1","addr":"172.21.15.188:6835","nonce":319464848}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":20},{"gid":5584,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.188:6834/2113408070","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6832","nonce":2113408070},{"type":"v1","addr":"172.21.15.188:6834","nonce":2113408070}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":20},{"gid":5745,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.143:6835/1616742593","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6834","nonce":1616742593},{"type":"v1","addr":"172.21.15.143:6835","nonce":1616742593}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":20}],"filesystems":[{"mdsmap":{"epoch":25,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-11-26T07:19:31.503587+0000","modified":"2023-11-26T07:19:34.523605+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":5748},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5748":{"gid":5748,"name":"c","rank":0,"incarnation":21,"state":"up:active","state_seq":3,"addr":"172.21.15.143:6837/2872033338","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6836","nonce":2872033338},{"type":"v1","addr":"172.21.15.143:6837","nonce":2872033338}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[7],"metadata_pool":6,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":3}]} 2023-11-26T07:20:19.178 INFO:teuthology.orchestra.run.smithi143.stderr:dumped fsmap epoch 26 2023-11-26T07:20:19.179 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:20:19.174+0000 7f733c3a1640 1 -- 172.21.15.143:0/4273663587 >> [v2:172.21.15.143:6832/2150557544,v1:172.21.15.143:6833/2150557544] conn(0x7f73180666b0 msgr2=0x7f7318068b70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:20:19.179 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:20:19.174+0000 7f733c3a1640 1 --2- 172.21.15.143:0/4273663587 >> [v2:172.21.15.143:6832/2150557544,v1:172.21.15.143:6833/2150557544] conn(0x7f73180666b0 0x7f7318068b70 secure :-1 s=READY pgs=329 cs=0 l=1 rev1=1 crypto rx=0x7f73240ad000 tx=0x7f7324008040 comp rx=0 tx=0).stop 2023-11-26T07:20:19.179 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:20:19.174+0000 7f733c3a1640 1 -- 172.21.15.143:0/4273663587 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f73340cd1b0 msgr2=0x7f7334140750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:20:19.180 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:20:19.174+0000 7f733c3a1640 1 --2- 172.21.15.143:0/4273663587 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f73340cd1b0 0x7f7334140750 secure :-1 s=READY pgs=685 cs=0 l=1 rev1=1 crypto rx=0x7f733000c480 tx=0x7f733000c940 comp rx=0 tx=0).stop 2023-11-26T07:20:19.180 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:20:19.174+0000 7f733c3a1640 1 -- 172.21.15.143:0/4273663587 shutdown_connections 2023-11-26T07:20:19.180 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:20:19.174+0000 7f733c3a1640 1 --2- 172.21.15.143:0/4273663587 >> [v2:172.21.15.143:6832/2150557544,v1:172.21.15.143:6833/2150557544] conn(0x7f73180666b0 0x7f7318068b70 unknown :-1 s=CLOSED pgs=329 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:20:19.180 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:20:19.174+0000 7f733c3a1640 1 --2- 172.21.15.143:0/4273663587 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f7334143ac0 0x7f73341f5ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:20:19.180 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:20:19.174+0000 7f733c3a1640 1 --2- 172.21.15.143:0/4273663587 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f73340cf280 0x7f73340c3e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:20:19.180 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:20:19.174+0000 7f733c3a1640 1 --2- 172.21.15.143:0/4273663587 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f73340cd1b0 0x7f7334140750 unknown :-1 s=CLOSED pgs=685 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:20:19.180 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:20:19.174+0000 7f733c3a1640 1 -- 172.21.15.143:0/4273663587 >> 172.21.15.143:0/4273663587 conn(0x7f733406a890 msgr2=0x7f73340acae0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:20:19.181 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:20:19.174+0000 7f733c3a1640 1 -- 172.21.15.143:0/4273663587 shutdown_connections 2023-11-26T07:20:19.181 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:20:19.174+0000 7f733c3a1640 1 -- 172.21.15.143:0/4273663587 wait complete. 2023-11-26T07:20:19.198 DEBUG:teuthology.orchestra.run.smithi143:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.c.asok --format=json session ls 2023-11-26T07:20:19.355 INFO:teuthology.orchestra.run.smithi143.stdout:[{"id":6126,"entity":{"name":{"type":"client","num":6126},"addr":{"type":"any","addr":"172.21.15.143:0","nonce":3879234400}},"state":"open","num_leases":1,"num_caps":7,"request_load_avg":13,"uptime":12.127501754000001,"requests_in_flight":0,"num_completed_requests":2,"num_completed_flushes":1,"reconnecting":false,"recall_caps":{"value":0,"halflife":60},"release_caps":{"value":0,"halflife":60},"recall_caps_throttle":{"value":0,"halflife":1.5},"recall_caps_throttle2o":{"value":0,"halflife":0.5},"session_cache_liveness":{"value":21.392779956022213,"halflife":300},"cap_acquisition":{"value":0,"halflife":30},"last_trim_completed_requests_tid":14,"last_trim_completed_flushes_tid":2,"delegated_inos":[{"start":"0x10000000005","length":500}],"inst":"client.6126 172.21.15.143:0/3879234400","completed_requests":[{"tid":14,"created_ino":"0x0"},{"tid":15,"created_ino":"0x100000001f9"}],"prealloc_inos":[{"start":"0x10000000005","length":500},{"start":"0x100000001fa","length":1000}],"client_metadata":{"client_features":{"feature_bits":"0x00000000003fffff"},"metric_spec":{"metric_flags":{"feature_bits":"0x000000000000ffff"}},"ceph_sha1":"44c2237fe77713214148c1fd622c872f346f1b99","ceph_version":"ceph version 18.0.0-7484-g44c2237f (44c2237fe77713214148c1fd622c872f346f1b99) reef (dev)","entity_id":"y","hostname":"smithi143","pid":"38097","root":"/"}}] 2023-11-26T07:20:19.355 DEBUG:tasks.cephfs.filesystem:_json_asok output [ { "cap_acquisition": { "halflife": 30, "value": 0 }, "client_metadata": { "ceph_sha1": "44c2237fe77713214148c1fd622c872f346f1b99", "ceph_version": "ceph version 18.0.0-7484-g44c2237f (44c2237fe77713214148c1fd622c872f346f1b99) reef (dev)", "client_features": { "feature_bits": "0x00000000003fffff" }, "entity_id": "y", "hostname": "smithi143", "metric_spec": { "metric_flags": { "feature_bits": "0x000000000000ffff" } }, "pid": "38097", "root": "/" }, "completed_requests": [ { "created_ino": "0x0", "tid": 14 }, { "created_ino": "0x100000001f9", "tid": 15 } ], "delegated_inos": [ { "length": 500, "start": "0x10000000005" } ], "entity": { "addr": { "addr": "172.21.15.143:0", "nonce": 3879234400, "type": "any" }, "name": { "num": 6126, "type": "client" } }, "id": 6126, "inst": "client.6126 172.21.15.143:0/3879234400", "last_trim_completed_flushes_tid": 2, "last_trim_completed_requests_tid": 14, "num_caps": 7, "num_completed_flushes": 1, "num_completed_requests": 2, "num_leases": 1, "prealloc_inos": [ { "length": 500, "start": "0x10000000005" }, { "length": 1000, "start": "0x100000001fa" } ], "recall_caps": { "halflife": 60, "value": 0 }, "recall_caps_throttle": { "halflife": 1.5, "value": 0 }, "recall_caps_throttle2o": { "halflife": 0.5, "value": 0 }, "reconnecting": false, "release_caps": { "halflife": 60, "value": 0 }, "request_load_avg": 13, "requests_in_flight": 0, "session_cache_liveness": { "halflife": 300, "value": 21.392779956022213 }, "state": "open", "uptime": 12.127501754 } ] 2023-11-26T07:20:19.356 DEBUG:tasks.ceph_test_case:wait_until_true: waiting (timeout=90 retry_count=0)... 2023-11-26T07:20:21.977 DEBUG:teuthology.orchestra.run.smithi143:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-26T07:20:21.981 DEBUG:teuthology.orchestra.run.smithi188:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-26T07:20:24.357 DEBUG:teuthology.orchestra.run.smithi143:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-11-26T07:20:24.504 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:20:24.498+0000 7f047f017640 1 Processor -- start 2023-11-26T07:20:24.505 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:20:24.502+0000 7f047f017640 1 -- start start 2023-11-26T07:20:24.506 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:20:24.502+0000 7f047f017640 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7f04780cf280 0x7f04780cf710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:20:24.506 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:20:24.502+0000 7f047f017640 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7f04780cd1b0 0x7f04780cd5c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:20:24.506 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:20:24.502+0000 7f047f017640 1 -- --> v1:172.21.15.188:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f04780cc730 con 0x7f0478143ac0 2023-11-26T07:20:24.507 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:20:24.502+0000 7f047f017640 1 -- --> v2:172.21.15.143:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0478143ea0 con 0x7f04780cd1b0 2023-11-26T07:20:24.507 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:20:24.502+0000 7f047f017640 1 -- --> v2:172.21.15.188:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f04780cfc50 con 0x7f04780cf280 2023-11-26T07:20:24.507 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:20:24.502+0000 7f047d5b4640 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7f04780cd1b0 0x7f04780cd5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:20:24.507 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:20:24.502+0000 7f047d5b4640 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7f04780cd1b0 0x7f04780cd5c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:34136/0 (socket says 172.21.15.143:34136) 2023-11-26T07:20:24.507 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:20:24.502+0000 7f047d5b4640 1 -- 172.21.15.143:0/3597218588 learned_addr learned my addr 172.21.15.143:0/3597218588 (peer_addr_for_me v2:172.21.15.143:0/0) 2023-11-26T07:20:24.507 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:20:24.502+0000 7f046ffff640 1 --2- 172.21.15.143:0/3597218588 >> v2:172.21.15.188:3300/0 conn(0x7f04780cf280 0x7f04780cf710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:20:24.507 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:20:24.502+0000 7f047d5b4640 1 -- 172.21.15.143:0/3597218588 >> v2:172.21.15.188:3300/0 conn(0x7f04780cf280 msgr2=0x7f04780cf710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-26T07:20:24.507 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:20:24.502+0000 7f047d5b4640 1 --2- 172.21.15.143:0/3597218588 >> v2:172.21.15.188:3300/0 conn(0x7f04780cf280 0x7f04780cf710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:20:24.507 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:20:24.502+0000 7f047d5b4640 1 -- 172.21.15.143:0/3597218588 >> v1:172.21.15.188:6789/0 conn(0x7f0478143ac0 legacy=0x7f0478070370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:20:24.507 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:20:24.502+0000 7f047d5b4640 1 -- 172.21.15.143:0/3597218588 --> v2:172.21.15.143:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0478070ee0 con 0x7f04780cd1b0 2023-11-26T07:20:24.509 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:20:24.506+0000 7f047d5b4640 1 --2- 172.21.15.143:0/3597218588 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f04780cd1b0 0x7f04780cd5c0 secure :-1 s=READY pgs=686 cs=0 l=1 rev1=1 crypto rx=0x7f04780aeb10 tx=0x7f047001c9d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5503083ea019bb5d server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:20:24.510 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:20:24.506+0000 7f046f7fe640 1 -- 172.21.15.143:0/3597218588 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f04700a5890 con 0x7f04780cd1b0 2023-11-26T07:20:24.510 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:20:24.506+0000 7f046f7fe640 1 -- 172.21.15.143:0/3597218588 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f04700a59f0 con 0x7f04780cd1b0 2023-11-26T07:20:24.510 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:20:24.506+0000 7f046f7fe640 1 -- 172.21.15.143:0/3597218588 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f04700a5d80 con 0x7f04780cd1b0 2023-11-26T07:20:24.510 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:20:24.506+0000 7f047f017640 1 -- 172.21.15.143:0/3597218588 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f04780cd1b0 msgr2=0x7f04780cd5c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:20:24.511 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:20:24.506+0000 7f047f017640 1 --2- 172.21.15.143:0/3597218588 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f04780cd1b0 0x7f04780cd5c0 secure :-1 s=READY pgs=686 cs=0 l=1 rev1=1 crypto rx=0x7f04780aeb10 tx=0x7f047001c9d0 comp rx=0 tx=0).stop 2023-11-26T07:20:24.511 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:20:24.506+0000 7f047f017640 1 -- 172.21.15.143:0/3597218588 shutdown_connections 2023-11-26T07:20:24.511 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:20:24.506+0000 7f047f017640 1 --2- 172.21.15.143:0/3597218588 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f04780cd1b0 0x7f04780cd5c0 unknown :-1 s=CLOSED pgs=686 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:20:24.511 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:20:24.506+0000 7f047f017640 1 --2- 172.21.15.143:0/3597218588 >> v2:172.21.15.188:3300/0 conn(0x7f04780cf280 0x7f04780cf710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:20:24.511 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:20:24.506+0000 7f047f017640 1 -- 172.21.15.143:0/3597218588 >> 172.21.15.143:0/3597218588 conn(0x7f047806a890 msgr2=0x7f047806aca0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:20:24.511 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:20:24.506+0000 7f047f017640 1 -- 172.21.15.143:0/3597218588 shutdown_connections 2023-11-26T07:20:24.511 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:20:24.506+0000 7f047f017640 1 -- 172.21.15.143:0/3597218588 wait complete. 2023-11-26T07:20:24.512 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:20:24.506+0000 7f047f017640 1 Processor -- start 2023-11-26T07:20:24.512 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:20:24.506+0000 7f047f017640 1 -- start start 2023-11-26T07:20:24.514 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:20:24.510+0000 7f047f017640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f04780cd1b0 0x7f04780aca70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:20:24.514 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:20:24.510+0000 7f047f017640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f04780cf280 0x7f04780b1cb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:20:24.515 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:20:24.510+0000 7f047f017640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f0478143ac0 0x7f04780b21f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:20:24.515 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:20:24.510+0000 7f047f017640 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f04780b08b0 con 0x7f04780cf280 2023-11-26T07:20:24.515 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:20:24.510+0000 7f047f017640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f04780acfb0 con 0x7f04780cd1b0 2023-11-26T07:20:24.515 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:20:24.510+0000 7f047f017640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f04781f6800 con 0x7f0478143ac0 2023-11-26T07:20:24.515 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:20:24.510+0000 7f046ffff640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f04780cf280 0x7f04780b1cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:20:24.515 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:20:24.510+0000 7f046ffff640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f04780cf280 0x7f04780b1cb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:34138/0 (socket says 172.21.15.143:34138) 2023-11-26T07:20:24.515 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:20:24.510+0000 7f046ffff640 1 -- 172.21.15.143:0/219080559 learned_addr learned my addr 172.21.15.143:0/219080559 (peer_addr_for_me v2:172.21.15.143:0/0) 2023-11-26T07:20:24.515 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:20:24.510+0000 7f047cdb3640 1 --2- 172.21.15.143:0/219080559 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f04780cd1b0 0x7f04780aca70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:20:24.515 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:20:24.510+0000 7f047d5b4640 1 --2- 172.21.15.143:0/219080559 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f0478143ac0 0x7f04780b21f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:20:24.515 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:20:24.510+0000 7f046ffff640 1 -- 172.21.15.143:0/219080559 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f0478143ac0 msgr2=0x7f04780b21f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:20:24.515 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:20:24.510+0000 7f046ffff640 1 --2- 172.21.15.143:0/219080559 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f0478143ac0 0x7f04780b21f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:20:24.515 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:20:24.510+0000 7f046ffff640 1 -- 172.21.15.143:0/219080559 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f04780cd1b0 msgr2=0x7f04780aca70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:20:24.515 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:20:24.510+0000 7f046ffff640 1 --2- 172.21.15.143:0/219080559 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f04780cd1b0 0x7f04780aca70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:20:24.516 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:20:24.510+0000 7f046ffff640 1 -- 172.21.15.143:0/219080559 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0470009b30 con 0x7f04780cf280 2023-11-26T07:20:24.516 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:20:24.510+0000 7f046ffff640 1 --2- 172.21.15.143:0/219080559 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f04780cf280 0x7f04780b1cb0 secure :-1 s=READY pgs=687 cs=0 l=1 rev1=1 crypto rx=0x7f046800d5f0 tx=0x7f046800dab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:20:24.516 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:20:24.510+0000 7f046d7fa640 1 -- 172.21.15.143:0/219080559 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f04680125f0 con 0x7f04780cf280 2023-11-26T07:20:24.516 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:20:24.510+0000 7f046d7fa640 1 -- 172.21.15.143:0/219080559 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0468012750 con 0x7f04780cf280 2023-11-26T07:20:24.516 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:20:24.510+0000 7f046d7fa640 1 -- 172.21.15.143:0/219080559 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0468012b00 con 0x7f04780cf280 2023-11-26T07:20:24.516 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:20:24.510+0000 7f047f017640 1 -- 172.21.15.143:0/219080559 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0478159b20 con 0x7f04780cf280 2023-11-26T07:20:24.516 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:20:24.510+0000 7f047f017640 1 -- 172.21.15.143:0/219080559 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f04780b04d0 con 0x7f04780cf280 2023-11-26T07:20:24.517 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:20:24.510+0000 7f047f017640 1 -- 172.21.15.143:0/219080559 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f04780b09f0 con 0x7f04780cf280 2023-11-26T07:20:24.517 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:20:24.510+0000 7f046d7fa640 1 -- 172.21.15.143:0/219080559 <== mon.0 v2:172.21.15.143:3300/0 4 ==== mgrmap(e 8) v1 ==== 86056+0+0 (secure 0 0 0) 0x7f0468012c60 con 0x7f04780cf280 2023-11-26T07:20:24.517 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:20:24.514+0000 7f046d7fa640 1 --2- 172.21.15.143:0/219080559 >> [v2:172.21.15.143:6832/2150557544,v1:172.21.15.143:6833/2150557544] conn(0x7f04640666b0 0x7f0464068b70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:20:24.517 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:20:24.514+0000 7f046d7fa640 1 -- 172.21.15.143:0/219080559 <== mon.0 v2:172.21.15.143:3300/0 5 ==== osd_map(45..45 src has 1..45) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f0468084690 con 0x7f04780cf280 2023-11-26T07:20:24.517 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:20:24.514+0000 7f047cdb3640 1 --2- 172.21.15.143:0/219080559 >> [v2:172.21.15.143:6832/2150557544,v1:172.21.15.143:6833/2150557544] conn(0x7f04640666b0 0x7f0464068b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:20:24.518 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:20:24.514+0000 7f047cdb3640 1 --2- 172.21.15.143:0/219080559 >> [v2:172.21.15.143:6832/2150557544,v1:172.21.15.143:6833/2150557544] conn(0x7f04640666b0 0x7f0464068b70 secure :-1 s=READY pgs=330 cs=0 l=1 rev1=1 crypto rx=0x7f047813c930 tx=0x7f0474009040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:20:24.521 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:20:24.518+0000 7f046d7fa640 1 -- 172.21.15.143:0/219080559 <== mon.0 v2:172.21.15.143:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172649 (secure 0 0 0) 0x7f04780b09f0 con 0x7f04780cf280 2023-11-26T07:20:24.712 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:20:24.706+0000 7f047f017640 1 -- 172.21.15.143:0/219080559 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f04780b09f0 con 0x7f04780cf280 2023-11-26T07:20:24.712 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:20:24.706+0000 7f046d7fa640 1 -- 172.21.15.143:0/219080559 <== mon.0 v2:172.21.15.143:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 26 v26) v1 ==== 94+0+4922 (secure 0 0 0) 0x7f04780b09f0 con 0x7f04780cf280 2023-11-26T07:20:24.713 INFO:teuthology.orchestra.run.smithi143.stdout: 2023-11-26T07:20:24.713 INFO:teuthology.orchestra.run.smithi143.stdout:{"epoch":26,"default_fscid":3,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5201,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.188:6835/319464848","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6833","nonce":319464848},{"type":"v1","addr":"172.21.15.188:6835","nonce":319464848}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":20},{"gid":5584,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.188:6834/2113408070","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6832","nonce":2113408070},{"type":"v1","addr":"172.21.15.188:6834","nonce":2113408070}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":20},{"gid":5745,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.143:6835/1616742593","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6834","nonce":1616742593},{"type":"v1","addr":"172.21.15.143:6835","nonce":1616742593}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":20}],"filesystems":[{"mdsmap":{"epoch":25,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-11-26T07:19:31.503587+0000","modified":"2023-11-26T07:19:34.523605+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":5748},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5748":{"gid":5748,"name":"c","rank":0,"incarnation":21,"state":"up:active","state_seq":3,"addr":"172.21.15.143:6837/2872033338","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6836","nonce":2872033338},{"type":"v1","addr":"172.21.15.143:6837","nonce":2872033338}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[7],"metadata_pool":6,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":3}]} 2023-11-26T07:20:24.713 INFO:teuthology.orchestra.run.smithi143.stderr:dumped fsmap epoch 26 2023-11-26T07:20:24.715 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:20:24.710+0000 7f047f017640 1 -- 172.21.15.143:0/219080559 >> [v2:172.21.15.143:6832/2150557544,v1:172.21.15.143:6833/2150557544] conn(0x7f04640666b0 msgr2=0x7f0464068b70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:20:24.715 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:20:24.710+0000 7f047f017640 1 --2- 172.21.15.143:0/219080559 >> [v2:172.21.15.143:6832/2150557544,v1:172.21.15.143:6833/2150557544] conn(0x7f04640666b0 0x7f0464068b70 secure :-1 s=READY pgs=330 cs=0 l=1 rev1=1 crypto rx=0x7f047813c930 tx=0x7f0474009040 comp rx=0 tx=0).stop 2023-11-26T07:20:24.715 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:20:24.710+0000 7f047f017640 1 -- 172.21.15.143:0/219080559 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f04780cf280 msgr2=0x7f04780b1cb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:20:24.715 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:20:24.710+0000 7f047f017640 1 --2- 172.21.15.143:0/219080559 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f04780cf280 0x7f04780b1cb0 secure :-1 s=READY pgs=687 cs=0 l=1 rev1=1 crypto rx=0x7f046800d5f0 tx=0x7f046800dab0 comp rx=0 tx=0).stop 2023-11-26T07:20:24.715 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:20:24.710+0000 7f047f017640 1 -- 172.21.15.143:0/219080559 shutdown_connections 2023-11-26T07:20:24.715 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:20:24.710+0000 7f047f017640 1 --2- 172.21.15.143:0/219080559 >> [v2:172.21.15.143:6832/2150557544,v1:172.21.15.143:6833/2150557544] conn(0x7f04640666b0 0x7f0464068b70 unknown :-1 s=CLOSED pgs=330 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:20:24.715 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:20:24.710+0000 7f047f017640 1 --2- 172.21.15.143:0/219080559 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f0478143ac0 0x7f04780b21f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:20:24.716 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:20:24.710+0000 7f047f017640 1 --2- 172.21.15.143:0/219080559 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f04780cf280 0x7f04780b1cb0 unknown :-1 s=CLOSED pgs=687 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:20:24.716 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:20:24.710+0000 7f047f017640 1 --2- 172.21.15.143:0/219080559 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f04780cd1b0 0x7f04780aca70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:20:24.716 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:20:24.710+0000 7f047f017640 1 -- 172.21.15.143:0/219080559 >> 172.21.15.143:0/219080559 conn(0x7f047806a890 msgr2=0x7f047813fec0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:20:24.716 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:20:24.710+0000 7f047f017640 1 -- 172.21.15.143:0/219080559 shutdown_connections 2023-11-26T07:20:24.716 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:20:24.710+0000 7f047f017640 1 -- 172.21.15.143:0/219080559 wait complete. 2023-11-26T07:20:24.734 DEBUG:teuthology.orchestra.run.smithi143:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.c.asok --format=json session ls 2023-11-26T07:20:24.894 INFO:teuthology.orchestra.run.smithi143.stdout:[{"id":6126,"entity":{"name":{"type":"client","num":6126},"addr":{"type":"any","addr":"172.21.15.143:0","nonce":3879234400}},"state":"open","num_leases":1,"num_caps":7,"request_load_avg":12,"uptime":17.667318700999999,"requests_in_flight":0,"num_completed_requests":2,"num_completed_flushes":1,"reconnecting":false,"recall_caps":{"value":0,"halflife":60},"release_caps":{"value":0,"halflife":60},"recall_caps_throttle":{"value":0,"halflife":1.5},"recall_caps_throttle2o":{"value":0,"halflife":0.5},"session_cache_liveness":{"value":21.120703857647335,"halflife":300},"cap_acquisition":{"value":0,"halflife":30},"last_trim_completed_requests_tid":14,"last_trim_completed_flushes_tid":2,"delegated_inos":[{"start":"0x10000000005","length":500}],"inst":"client.6126 172.21.15.143:0/3879234400","completed_requests":[{"tid":14,"created_ino":"0x0"},{"tid":15,"created_ino":"0x100000001f9"}],"prealloc_inos":[{"start":"0x10000000005","length":500},{"start":"0x100000001fa","length":1000}],"client_metadata":{"client_features":{"feature_bits":"0x00000000003fffff"},"metric_spec":{"metric_flags":{"feature_bits":"0x000000000000ffff"}},"ceph_sha1":"44c2237fe77713214148c1fd622c872f346f1b99","ceph_version":"ceph version 18.0.0-7484-g44c2237f (44c2237fe77713214148c1fd622c872f346f1b99) reef (dev)","entity_id":"y","hostname":"smithi143","pid":"38097","root":"/"}}] 2023-11-26T07:20:24.894 DEBUG:tasks.cephfs.filesystem:_json_asok output [ { "cap_acquisition": { "halflife": 30, "value": 0 }, "client_metadata": { "ceph_sha1": "44c2237fe77713214148c1fd622c872f346f1b99", "ceph_version": "ceph version 18.0.0-7484-g44c2237f (44c2237fe77713214148c1fd622c872f346f1b99) reef (dev)", "client_features": { "feature_bits": "0x00000000003fffff" }, "entity_id": "y", "hostname": "smithi143", "metric_spec": { "metric_flags": { "feature_bits": "0x000000000000ffff" } }, "pid": "38097", "root": "/" }, "completed_requests": [ { "created_ino": "0x0", "tid": 14 }, { "created_ino": "0x100000001f9", "tid": 15 } ], "delegated_inos": [ { "length": 500, "start": "0x10000000005" } ], "entity": { "addr": { "addr": "172.21.15.143:0", "nonce": 3879234400, "type": "any" }, "name": { "num": 6126, "type": "client" } }, "id": 6126, "inst": "client.6126 172.21.15.143:0/3879234400", "last_trim_completed_flushes_tid": 2, "last_trim_completed_requests_tid": 14, "num_caps": 7, "num_completed_flushes": 1, "num_completed_requests": 2, "num_leases": 1, "prealloc_inos": [ { "length": 500, "start": "0x10000000005" }, { "length": 1000, "start": "0x100000001fa" } ], "recall_caps": { "halflife": 60, "value": 0 }, "recall_caps_throttle": { "halflife": 1.5, "value": 0 }, "recall_caps_throttle2o": { "halflife": 0.5, "value": 0 }, "reconnecting": false, "release_caps": { "halflife": 60, "value": 0 }, "request_load_avg": 12, "requests_in_flight": 0, "session_cache_liveness": { "halflife": 300, "value": 21.120703857647335 }, "state": "open", "uptime": 17.667318701 } ] 2023-11-26T07:20:24.894 DEBUG:tasks.ceph_test_case:wait_until_true: waiting (timeout=90 retry_count=0)... 2023-11-26T07:20:29.895 DEBUG:teuthology.orchestra.run.smithi143:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-11-26T07:20:30.043 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:20:30.038+0000 7f6fac032640 1 Processor -- start 2023-11-26T07:20:30.043 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:20:30.038+0000 7f6fac032640 1 -- start start 2023-11-26T07:20:30.045 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:20:30.042+0000 7f6fac032640 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7f6fa40cf280 0x7f6fa40cf710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:20:30.045 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:20:30.042+0000 7f6fac032640 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7f6fa40cd1b0 0x7f6fa40cd5c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:20:30.045 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:20:30.042+0000 7f6fac032640 1 -- --> v1:172.21.15.143:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6fa40cc730 con 0x7f6fa4143ac0 2023-11-26T07:20:30.045 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:20:30.042+0000 7f6fac032640 1 -- --> v2:172.21.15.143:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6fa4143ea0 con 0x7f6fa40cf280 2023-11-26T07:20:30.045 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:20:30.042+0000 7f6fac032640 1 -- --> v2:172.21.15.188:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6fa40cfc50 con 0x7f6fa40cd1b0 2023-11-26T07:20:30.045 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:20:30.042+0000 7f6fa95cd640 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7f6fa40cf280 0x7f6fa40cf710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:20:30.045 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:20:30.042+0000 7f6fa9dce640 1 --1- >> v1:172.21.15.143:6789/0 conn(0x7f6fa4143ac0 0x7f6fa4070370 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.143:6789/0 says I am v1:172.21.15.143:53944/0 (socket says 172.21.15.143:53944) 2023-11-26T07:20:30.045 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:20:30.042+0000 7f6fa9dce640 1 -- 172.21.15.143:0/2149553826 learned_addr learned my addr 172.21.15.143:0/2149553826 (peer_addr_for_me v1:172.21.15.143:0/0) 2023-11-26T07:20:30.046 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:20:30.042+0000 7f6fa95cd640 1 -- 172.21.15.143:0/2149553826 >> v2:172.21.15.188:3300/0 conn(0x7f6fa40cd1b0 msgr2=0x7f6fa40cd5c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-26T07:20:30.046 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:20:30.042+0000 7f6fa95cd640 1 --2- 172.21.15.143:0/2149553826 >> v2:172.21.15.188:3300/0 conn(0x7f6fa40cd1b0 0x7f6fa40cd5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:20:30.046 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:20:30.042+0000 7f6fa95cd640 1 -- 172.21.15.143:0/2149553826 >> v1:172.21.15.143:6789/0 conn(0x7f6fa4143ac0 legacy=0x7f6fa4070370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:20:30.046 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:20:30.042+0000 7f6fa95cd640 1 -- 172.21.15.143:0/2149553826 --> v2:172.21.15.143:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6fa4070ee0 con 0x7f6fa40cf280 2023-11-26T07:20:30.046 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:20:30.042+0000 7f6fa8dcc640 1 -- 172.21.15.143:0/2149553826 <== mon.0 v1:172.21.15.143:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 243868888 0 0) 0x7f6fa40cc730 con 0x7f6fa4143ac0 2023-11-26T07:20:30.048 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:20:30.042+0000 7f6fa95cd640 1 --2- 172.21.15.143:0/2149553826 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f6fa40cf280 0x7f6fa40cf710 secure :-1 s=READY pgs=689 cs=0 l=1 rev1=1 crypto rx=0x7f6fa406e570 tx=0x7f6f9401c460 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3f0d948facf3659a server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:20:30.048 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:20:30.042+0000 7f6fa8dcc640 1 -- 172.21.15.143:0/2149553826 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6f940ab660 con 0x7f6fa40cf280 2023-11-26T07:20:30.048 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:20:30.042+0000 7f6fa8dcc640 1 -- 172.21.15.143:0/2149553826 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6f940ab7c0 con 0x7f6fa40cf280 2023-11-26T07:20:30.048 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:20:30.042+0000 7f6fa8dcc640 1 -- 172.21.15.143:0/2149553826 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6f940abb50 con 0x7f6fa40cf280 2023-11-26T07:20:30.049 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:20:30.046+0000 7f6fac032640 1 -- 172.21.15.143:0/2149553826 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f6fa40cf280 msgr2=0x7f6fa40cf710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:20:30.049 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:20:30.046+0000 7f6fac032640 1 --2- 172.21.15.143:0/2149553826 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f6fa40cf280 0x7f6fa40cf710 secure :-1 s=READY pgs=689 cs=0 l=1 rev1=1 crypto rx=0x7f6fa406e570 tx=0x7f6f9401c460 comp rx=0 tx=0).stop 2023-11-26T07:20:30.049 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:20:30.046+0000 7f6fac032640 1 -- 172.21.15.143:0/2149553826 shutdown_connections 2023-11-26T07:20:30.049 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:20:30.046+0000 7f6fac032640 1 --2- 172.21.15.143:0/2149553826 >> v2:172.21.15.188:3300/0 conn(0x7f6fa40cd1b0 0x7f6fa40cd5c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:20:30.049 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:20:30.046+0000 7f6fac032640 1 --2- 172.21.15.143:0/2149553826 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f6fa40cf280 0x7f6fa40cf710 unknown :-1 s=CLOSED pgs=689 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:20:30.050 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:20:30.046+0000 7f6fac032640 1 -- 172.21.15.143:0/2149553826 >> 172.21.15.143:0/2149553826 conn(0x7f6fa406a890 msgr2=0x7f6fa406aca0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:20:30.050 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:20:30.046+0000 7f6fac032640 1 -- 172.21.15.143:0/2149553826 shutdown_connections 2023-11-26T07:20:30.050 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:20:30.046+0000 7f6fac032640 1 -- 172.21.15.143:0/2149553826 wait complete. 2023-11-26T07:20:30.051 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:20:30.046+0000 7f6fac032640 1 Processor -- start 2023-11-26T07:20:30.139 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:20:30.046+0000 7f6fac032640 1 -- start start 2023-11-26T07:20:30.142 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:20:30.046+0000 7f6fac032640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f6fa40cd1b0 0x7f6fa4167210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:20:30.142 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:20:30.046+0000 7f6fac032640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f6fa40cf280 0x7f6fa4156380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:20:30.142 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:20:30.046+0000 7f6fac032640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f6fa4143ac0 0x7f6fa413ff00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:20:30.142 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:20:30.046+0000 7f6fac032640 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6fa4166d80 con 0x7f6fa40cd1b0 2023-11-26T07:20:30.142 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:20:30.046+0000 7f6fac032640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6fa4157ee0 con 0x7f6fa40cf280 2023-11-26T07:20:30.142 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:20:30.046+0000 7f6fac032640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6fa4166170 con 0x7f6fa4143ac0 2023-11-26T07:20:30.143 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:20:30.046+0000 7f6fa9dce640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f6fa40cd1b0 0x7f6fa4167210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:20:30.143 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:20:30.046+0000 7f6fa9dce640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f6fa40cd1b0 0x7f6fa4167210 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:34158/0 (socket says 172.21.15.143:34158) 2023-11-26T07:20:30.143 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:20:30.046+0000 7f6fa9dce640 1 -- 172.21.15.143:0/1485676182 learned_addr learned my addr 172.21.15.143:0/1485676182 (peer_addr_for_me v2:172.21.15.143:0/0) 2023-11-26T07:20:30.143 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:20:30.046+0000 7f6fa9dce640 1 -- 172.21.15.143:0/1485676182 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f6fa4143ac0 msgr2=0x7f6fa413ff00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:20:30.143 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:20:30.046+0000 7f6fa9dce640 1 --2- 172.21.15.143:0/1485676182 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f6fa4143ac0 0x7f6fa413ff00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:20:30.143 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:20:30.046+0000 7f6fa9dce640 1 -- 172.21.15.143:0/1485676182 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f6fa40cf280 msgr2=0x7f6fa4156380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:20:30.143 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:20:30.046+0000 7f6fa95cd640 1 --2- 172.21.15.143:0/1485676182 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f6fa40cf280 0x7f6fa4156380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:20:30.143 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:20:30.050+0000 7f6fa9dce640 1 --2- 172.21.15.143:0/1485676182 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f6fa40cf280 0x7f6fa4156380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:20:30.144 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:20:30.050+0000 7f6fa9dce640 1 -- 172.21.15.143:0/1485676182 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6f94009420 con 0x7f6fa40cd1b0 2023-11-26T07:20:30.144 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:20:30.050+0000 7f6fa95cd640 1 --2- 172.21.15.143:0/1485676182 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f6fa40cf280 0x7f6fa4156380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-26T07:20:30.144 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:20:30.050+0000 7f6fa9dce640 1 --2- 172.21.15.143:0/1485676182 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f6fa40cd1b0 0x7f6fa4167210 secure :-1 s=READY pgs=690 cs=0 l=1 rev1=1 crypto rx=0x7f6fa00011f0 tx=0x7f6fa000c9b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:20:30.144 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:20:30.050+0000 7f6f9a7fc640 1 -- 172.21.15.143:0/1485676182 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6fa00074e0 con 0x7f6fa40cd1b0 2023-11-26T07:20:30.144 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:20:30.050+0000 7f6f9a7fc640 1 -- 172.21.15.143:0/1485676182 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6fa0007640 con 0x7f6fa40cd1b0 2023-11-26T07:20:30.144 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:20:30.050+0000 7f6fac032640 1 -- 172.21.15.143:0/1485676182 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6fa4166450 con 0x7f6fa40cd1b0 2023-11-26T07:20:30.144 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:20:30.050+0000 7f6f9a7fc640 1 -- 172.21.15.143:0/1485676182 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6fa0004a10 con 0x7f6fa40cd1b0 2023-11-26T07:20:30.144 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:20:30.050+0000 7f6fac032640 1 -- 172.21.15.143:0/1485676182 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6fa4166a10 con 0x7f6fa40cd1b0 2023-11-26T07:20:30.144 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:20:30.050+0000 7f6fac032640 1 -- 172.21.15.143:0/1485676182 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6fa4067680 con 0x7f6fa40cd1b0 2023-11-26T07:20:30.145 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:20:30.050+0000 7f6f9a7fc640 1 -- 172.21.15.143:0/1485676182 <== mon.0 v2:172.21.15.143:3300/0 4 ==== mgrmap(e 8) v1 ==== 86056+0+0 (secure 0 0 0) 0x7f6fa0004b70 con 0x7f6fa40cd1b0 2023-11-26T07:20:30.145 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:20:30.050+0000 7f6f9a7fc640 1 --2- 172.21.15.143:0/1485676182 >> [v2:172.21.15.143:6832/2150557544,v1:172.21.15.143:6833/2150557544] conn(0x7f6f880666b0 0x7f6f88068b70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:20:30.145 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:20:30.050+0000 7f6f9a7fc640 1 -- 172.21.15.143:0/1485676182 <== mon.0 v2:172.21.15.143:3300/0 5 ==== osd_map(45..45 src has 1..45) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f6fa0083890 con 0x7f6fa40cd1b0 2023-11-26T07:20:30.145 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:20:30.050+0000 7f6fa95cd640 1 --2- 172.21.15.143:0/1485676182 >> [v2:172.21.15.143:6832/2150557544,v1:172.21.15.143:6833/2150557544] conn(0x7f6f880666b0 0x7f6f88068b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:20:30.145 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:20:30.054+0000 7f6fa95cd640 1 --2- 172.21.15.143:0/1485676182 >> [v2:172.21.15.143:6832/2150557544,v1:172.21.15.143:6833/2150557544] conn(0x7f6f880666b0 0x7f6f88068b70 secure :-1 s=READY pgs=331 cs=0 l=1 rev1=1 crypto rx=0x7f6f940af000 tx=0x7f6f94008040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:20:30.145 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:20:30.054+0000 7f6f9a7fc640 1 -- 172.21.15.143:0/1485676182 <== mon.0 v2:172.21.15.143:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172649 (secure 0 0 0) 0x7f6fa00509b0 con 0x7f6fa40cd1b0 2023-11-26T07:20:30.240 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:20:30.234+0000 7f6fac032640 1 -- 172.21.15.143:0/1485676182 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f6fa41432d0 con 0x7f6fa40cd1b0 2023-11-26T07:20:30.242 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:20:30.238+0000 7f6f9a7fc640 1 -- 172.21.15.143:0/1485676182 <== mon.0 v2:172.21.15.143:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 26 v26) v1 ==== 94+0+4922 (secure 0 0 0) 0x7f6fa0018020 con 0x7f6fa40cd1b0 2023-11-26T07:20:30.242 INFO:teuthology.orchestra.run.smithi143.stdout: 2023-11-26T07:20:30.242 INFO:teuthology.orchestra.run.smithi143.stdout:{"epoch":26,"default_fscid":3,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5201,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.188:6835/319464848","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6833","nonce":319464848},{"type":"v1","addr":"172.21.15.188:6835","nonce":319464848}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":20},{"gid":5584,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.188:6834/2113408070","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6832","nonce":2113408070},{"type":"v1","addr":"172.21.15.188:6834","nonce":2113408070}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":20},{"gid":5745,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.143:6835/1616742593","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6834","nonce":1616742593},{"type":"v1","addr":"172.21.15.143:6835","nonce":1616742593}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":20}],"filesystems":[{"mdsmap":{"epoch":25,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-11-26T07:19:31.503587+0000","modified":"2023-11-26T07:19:34.523605+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":5748},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5748":{"gid":5748,"name":"c","rank":0,"incarnation":21,"state":"up:active","state_seq":3,"addr":"172.21.15.143:6837/2872033338","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6836","nonce":2872033338},{"type":"v1","addr":"172.21.15.143:6837","nonce":2872033338}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[7],"metadata_pool":6,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":3}]} 2023-11-26T07:20:30.243 INFO:teuthology.orchestra.run.smithi143.stderr:dumped fsmap epoch 26 2023-11-26T07:20:30.243 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:20:30.238+0000 7f6fac032640 1 -- 172.21.15.143:0/1485676182 >> [v2:172.21.15.143:6832/2150557544,v1:172.21.15.143:6833/2150557544] conn(0x7f6f880666b0 msgr2=0x7f6f88068b70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:20:30.244 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:20:30.238+0000 7f6fac032640 1 --2- 172.21.15.143:0/1485676182 >> [v2:172.21.15.143:6832/2150557544,v1:172.21.15.143:6833/2150557544] conn(0x7f6f880666b0 0x7f6f88068b70 secure :-1 s=READY pgs=331 cs=0 l=1 rev1=1 crypto rx=0x7f6f940af000 tx=0x7f6f94008040 comp rx=0 tx=0).stop 2023-11-26T07:20:30.244 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:20:30.238+0000 7f6fac032640 1 -- 172.21.15.143:0/1485676182 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f6fa40cd1b0 msgr2=0x7f6fa4167210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:20:30.244 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:20:30.238+0000 7f6fac032640 1 --2- 172.21.15.143:0/1485676182 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f6fa40cd1b0 0x7f6fa4167210 secure :-1 s=READY pgs=690 cs=0 l=1 rev1=1 crypto rx=0x7f6fa00011f0 tx=0x7f6fa000c9b0 comp rx=0 tx=0).stop 2023-11-26T07:20:30.244 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:20:30.238+0000 7f6fac032640 1 -- 172.21.15.143:0/1485676182 shutdown_connections 2023-11-26T07:20:30.244 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:20:30.238+0000 7f6fac032640 1 --2- 172.21.15.143:0/1485676182 >> [v2:172.21.15.143:6832/2150557544,v1:172.21.15.143:6833/2150557544] conn(0x7f6f880666b0 0x7f6f88068b70 unknown :-1 s=CLOSED pgs=331 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:20:30.244 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:20:30.238+0000 7f6fac032640 1 --2- 172.21.15.143:0/1485676182 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f6fa4143ac0 0x7f6fa413ff00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:20:30.244 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:20:30.238+0000 7f6fac032640 1 --2- 172.21.15.143:0/1485676182 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f6fa40cf280 0x7f6fa4156380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:20:30.244 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:20:30.238+0000 7f6fac032640 1 --2- 172.21.15.143:0/1485676182 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f6fa40cd1b0 0x7f6fa4167210 unknown :-1 s=CLOSED pgs=690 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:20:30.244 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:20:30.238+0000 7f6fac032640 1 -- 172.21.15.143:0/1485676182 >> 172.21.15.143:0/1485676182 conn(0x7f6fa406a890 msgr2=0x7f6fa4070370 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:20:30.244 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:20:30.238+0000 7f6fac032640 1 -- 172.21.15.143:0/1485676182 shutdown_connections 2023-11-26T07:20:30.245 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:20:30.238+0000 7f6fac032640 1 -- 172.21.15.143:0/1485676182 wait complete. 2023-11-26T07:20:30.262 DEBUG:teuthology.orchestra.run.smithi143:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.c.asok --format=json session ls 2023-11-26T07:20:30.420 INFO:teuthology.orchestra.run.smithi143.stdout:[{"id":6126,"entity":{"name":{"type":"client","num":6126},"addr":{"type":"any","addr":"172.21.15.143:0","nonce":3879234400}},"state":"open","num_leases":1,"num_caps":7,"request_load_avg":11,"uptime":23.195157300000002,"requests_in_flight":0,"num_completed_requests":2,"num_completed_flushes":1,"reconnecting":false,"recall_caps":{"value":0,"halflife":60},"release_caps":{"value":0,"halflife":60},"recall_caps_throttle":{"value":0,"halflife":1.5},"recall_caps_throttle2o":{"value":0,"halflife":0.5},"session_cache_liveness":{"value":20.852665165187439,"halflife":300},"cap_acquisition":{"value":0,"halflife":30},"last_trim_completed_requests_tid":14,"last_trim_completed_flushes_tid":2,"delegated_inos":[{"start":"0x10000000005","length":500}],"inst":"client.6126 172.21.15.143:0/3879234400","completed_requests":[{"tid":14,"created_ino":"0x0"},{"tid":15,"created_ino":"0x100000001f9"}],"prealloc_inos":[{"start":"0x10000000005","length":500},{"start":"0x100000001fa","length":1000}],"client_metadata":{"client_features":{"feature_bits":"0x00000000003fffff"},"metric_spec":{"metric_flags":{"feature_bits":"0x000000000000ffff"}},"ceph_sha1":"44c2237fe77713214148c1fd622c872f346f1b99","ceph_version":"ceph version 18.0.0-7484-g44c2237f (44c2237fe77713214148c1fd622c872f346f1b99) reef (dev)","entity_id":"y","hostname":"smithi143","pid":"38097","root":"/"}}] 2023-11-26T07:20:30.421 DEBUG:tasks.cephfs.filesystem:_json_asok output [ { "cap_acquisition": { "halflife": 30, "value": 0 }, "client_metadata": { "ceph_sha1": "44c2237fe77713214148c1fd622c872f346f1b99", "ceph_version": "ceph version 18.0.0-7484-g44c2237f (44c2237fe77713214148c1fd622c872f346f1b99) reef (dev)", "client_features": { "feature_bits": "0x00000000003fffff" }, "entity_id": "y", "hostname": "smithi143", "metric_spec": { "metric_flags": { "feature_bits": "0x000000000000ffff" } }, "pid": "38097", "root": "/" }, "completed_requests": [ { "created_ino": "0x0", "tid": 14 }, { "created_ino": "0x100000001f9", "tid": 15 } ], "delegated_inos": [ { "length": 500, "start": "0x10000000005" } ], "entity": { "addr": { "addr": "172.21.15.143:0", "nonce": 3879234400, "type": "any" }, "name": { "num": 6126, "type": "client" } }, "id": 6126, "inst": "client.6126 172.21.15.143:0/3879234400", "last_trim_completed_flushes_tid": 2, "last_trim_completed_requests_tid": 14, "num_caps": 7, "num_completed_flushes": 1, "num_completed_requests": 2, "num_leases": 1, "prealloc_inos": [ { "length": 500, "start": "0x10000000005" }, { "length": 1000, "start": "0x100000001fa" } ], "recall_caps": { "halflife": 60, "value": 0 }, "recall_caps_throttle": { "halflife": 1.5, "value": 0 }, "recall_caps_throttle2o": { "halflife": 0.5, "value": 0 }, "reconnecting": false, "release_caps": { "halflife": 60, "value": 0 }, "request_load_avg": 11, "requests_in_flight": 0, "session_cache_liveness": { "halflife": 300, "value": 20.85266516518744 }, "state": "open", "uptime": 23.1951573 } ] 2023-11-26T07:20:30.421 DEBUG:tasks.ceph_test_case:wait_until_true: waiting (timeout=90 retry_count=0)... 2023-11-26T07:20:35.422 DEBUG:teuthology.orchestra.run.smithi143:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-11-26T07:20:35.577 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:20:35.570+0000 7f95c44aa640 1 Processor -- start 2023-11-26T07:20:35.578 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:20:35.570+0000 7f95c44aa640 1 -- start start 2023-11-26T07:20:35.579 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:20:35.574+0000 7f95c44aa640 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7f95bc13e290 0x7f95bc0cf280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:20:35.579 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:20:35.574+0000 7f95c44aa640 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7f95bc0cf7c0 0x7f95bc15efa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:20:35.579 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:20:35.574+0000 7f95c44aa640 1 -- --> v1:172.21.15.143:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f95bc06c520 con 0x7f95bc15f4e0 2023-11-26T07:20:35.579 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:20:35.574+0000 7f95c44aa640 1 -- --> v2:172.21.15.143:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f95bc0cfc40 con 0x7f95bc13e290 2023-11-26T07:20:35.580 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:20:35.574+0000 7f95c44aa640 1 -- --> v2:172.21.15.188:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f95bc06c800 con 0x7f95bc0cf7c0 2023-11-26T07:20:35.580 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:20:35.574+0000 7f95c2246640 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7f95bc13e290 0x7f95bc0cf280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:20:35.580 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:20:35.574+0000 7f95c2246640 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7f95bc13e290 0x7f95bc0cf280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:46014/0 (socket says 172.21.15.143:46014) 2023-11-26T07:20:35.580 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:20:35.574+0000 7f95c2246640 1 -- 172.21.15.143:0/109492243 learned_addr learned my addr 172.21.15.143:0/109492243 (peer_addr_for_me v2:172.21.15.143:0/0) 2023-11-26T07:20:35.580 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:20:35.574+0000 7f95c1a45640 1 --2- 172.21.15.143:0/109492243 >> v2:172.21.15.188:3300/0 conn(0x7f95bc0cf7c0 0x7f95bc15efa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:20:35.580 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:20:35.574+0000 7f95c2246640 1 -- 172.21.15.143:0/109492243 >> v2:172.21.15.188:3300/0 conn(0x7f95bc0cf7c0 msgr2=0x7f95bc15efa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-26T07:20:35.581 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:20:35.574+0000 7f95c2246640 1 --2- 172.21.15.143:0/109492243 >> v2:172.21.15.188:3300/0 conn(0x7f95bc0cf7c0 0x7f95bc15efa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:20:35.581 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:20:35.574+0000 7f95c2246640 1 -- 172.21.15.143:0/109492243 >> v1:172.21.15.143:6789/0 conn(0x7f95bc15f4e0 legacy=0x7f95bc1408e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:20:35.581 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:20:35.574+0000 7f95c2246640 1 -- 172.21.15.143:0/109492243 --> v2:172.21.15.143:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f95bc06cae0 con 0x7f95bc13e290 2023-11-26T07:20:35.589 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:20:35.578+0000 7f95c2246640 1 --2- 172.21.15.143:0/109492243 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f95bc13e290 0x7f95bc0cf280 secure :-1 s=READY pgs=692 cs=0 l=1 rev1=1 crypto rx=0x7f95bc0cfe40 tx=0x7f95b801c4d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c9f0d7049da8d46e server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:20:35.590 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:20:35.578+0000 7f95c1244640 1 -- 172.21.15.143:0/109492243 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f95b80aa430 con 0x7f95bc13e290 2023-11-26T07:20:35.590 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:20:35.578+0000 7f95c1244640 1 -- 172.21.15.143:0/109492243 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f95b80aa590 con 0x7f95bc13e290 2023-11-26T07:20:35.590 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:20:35.578+0000 7f95c1244640 1 -- 172.21.15.143:0/109492243 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f95b80aa920 con 0x7f95bc13e290 2023-11-26T07:20:35.591 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:20:35.578+0000 7f95c44aa640 1 -- 172.21.15.143:0/109492243 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f95bc13e290 msgr2=0x7f95bc0cf280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:20:35.591 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:20:35.578+0000 7f95c44aa640 1 --2- 172.21.15.143:0/109492243 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f95bc13e290 0x7f95bc0cf280 secure :-1 s=READY pgs=692 cs=0 l=1 rev1=1 crypto rx=0x7f95bc0cfe40 tx=0x7f95b801c4d0 comp rx=0 tx=0).stop 2023-11-26T07:20:35.591 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:20:35.578+0000 7f95c44aa640 1 -- 172.21.15.143:0/109492243 shutdown_connections 2023-11-26T07:20:35.591 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:20:35.578+0000 7f95c44aa640 1 --2- 172.21.15.143:0/109492243 >> v2:172.21.15.188:3300/0 conn(0x7f95bc0cf7c0 0x7f95bc15efa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:20:35.591 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:20:35.578+0000 7f95c44aa640 1 --2- 172.21.15.143:0/109492243 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f95bc13e290 0x7f95bc0cf280 unknown :-1 s=CLOSED pgs=692 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:20:35.591 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:20:35.578+0000 7f95c44aa640 1 -- 172.21.15.143:0/109492243 >> 172.21.15.143:0/109492243 conn(0x7f95bc06a890 msgr2=0x7f95bc06aca0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:20:35.592 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:20:35.578+0000 7f95c44aa640 1 -- 172.21.15.143:0/109492243 shutdown_connections 2023-11-26T07:20:35.592 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:20:35.578+0000 7f95c44aa640 1 -- 172.21.15.143:0/109492243 wait complete. 2023-11-26T07:20:35.594 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:20:35.578+0000 7f95c44aa640 1 Processor -- start 2023-11-26T07:20:35.594 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:20:35.578+0000 7f95c44aa640 1 -- start start 2023-11-26T07:20:35.595 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:20:35.582+0000 7f95c44aa640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f95bc0cf7c0 0x7f95bc1358a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:20:35.595 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:20:35.582+0000 7f95c44aa640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f95bc13e290 0x7f95bc15df90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:20:35.595 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:20:35.582+0000 7f95c44aa640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f95bc15f4e0 0x7f95bc0b0680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:20:35.595 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:20:35.582+0000 7f95c44aa640 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f95bc0b0e60 con 0x7f95bc0cf7c0 2023-11-26T07:20:35.595 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:20:35.582+0000 7f95c44aa640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f95bc0b0bc0 con 0x7f95bc13e290 2023-11-26T07:20:35.595 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:20:35.582+0000 7f95c44aa640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f95bc15b6b0 con 0x7f95bc15f4e0 2023-11-26T07:20:35.595 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:20:35.582+0000 7f95c2246640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f95bc0cf7c0 0x7f95bc1358a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:20:35.595 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:20:35.582+0000 7f95c2246640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f95bc0cf7c0 0x7f95bc1358a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:46028/0 (socket says 172.21.15.143:46028) 2023-11-26T07:20:35.595 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:20:35.582+0000 7f95c2246640 1 -- 172.21.15.143:0/2086059958 learned_addr learned my addr 172.21.15.143:0/2086059958 (peer_addr_for_me v2:172.21.15.143:0/0) 2023-11-26T07:20:35.595 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:20:35.582+0000 7f95c1a45640 1 --2- 172.21.15.143:0/2086059958 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f95bc13e290 0x7f95bc15df90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:20:35.596 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:20:35.582+0000 7f95c2a47640 1 --2- 172.21.15.143:0/2086059958 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f95bc15f4e0 0x7f95bc0b0680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:20:35.596 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:20:35.582+0000 7f95c2246640 1 -- 172.21.15.143:0/2086059958 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f95bc15f4e0 msgr2=0x7f95bc0b0680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:20:35.596 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:20:35.582+0000 7f95c2246640 1 --2- 172.21.15.143:0/2086059958 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f95bc15f4e0 0x7f95bc0b0680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:20:35.596 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:20:35.582+0000 7f95c2246640 1 -- 172.21.15.143:0/2086059958 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f95bc13e290 msgr2=0x7f95bc15df90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:20:35.596 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:20:35.582+0000 7f95c2246640 1 --2- 172.21.15.143:0/2086059958 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f95bc13e290 0x7f95bc15df90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:20:35.596 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:20:35.582+0000 7f95c2246640 1 -- 172.21.15.143:0/2086059958 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f95b80094d0 con 0x7f95bc0cf7c0 2023-11-26T07:20:35.596 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:20:35.582+0000 7f95c2a47640 1 --2- 172.21.15.143:0/2086059958 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f95bc15f4e0 0x7f95bc0b0680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-26T07:20:35.596 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:20:35.582+0000 7f95c2246640 1 --2- 172.21.15.143:0/2086059958 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f95bc0cf7c0 0x7f95bc1358a0 secure :-1 s=READY pgs=693 cs=0 l=1 rev1=1 crypto rx=0x7f95b80ac040 tx=0x7f95b801ad10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:20:35.596 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:20:35.582+0000 7f95b2ffd640 1 -- 172.21.15.143:0/2086059958 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f95b80b3800 con 0x7f95bc0cf7c0 2023-11-26T07:20:35.597 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:20:35.582+0000 7f95b2ffd640 1 -- 172.21.15.143:0/2086059958 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f95b80b3980 con 0x7f95bc0cf7c0 2023-11-26T07:20:35.597 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:20:35.582+0000 7f95b2ffd640 1 -- 172.21.15.143:0/2086059958 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f95b80b3d30 con 0x7f95bc0cf7c0 2023-11-26T07:20:35.597 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:20:35.582+0000 7f95c44aa640 1 -- 172.21.15.143:0/2086059958 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f95bc15b990 con 0x7f95bc0cf7c0 2023-11-26T07:20:35.597 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:20:35.582+0000 7f95c44aa640 1 -- 172.21.15.143:0/2086059958 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f95bc1537c0 con 0x7f95bc0cf7c0 2023-11-26T07:20:35.597 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:20:35.582+0000 7f95c44aa640 1 -- 172.21.15.143:0/2086059958 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f95bc067680 con 0x7f95bc0cf7c0 2023-11-26T07:20:35.597 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:20:35.586+0000 7f95b2ffd640 1 -- 172.21.15.143:0/2086059958 <== mon.0 v2:172.21.15.143:3300/0 4 ==== mgrmap(e 8) v1 ==== 86056+0+0 (secure 0 0 0) 0x7f95b80b2380 con 0x7f95bc0cf7c0 2023-11-26T07:20:35.597 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:20:35.586+0000 7f95b2ffd640 1 --2- 172.21.15.143:0/2086059958 >> [v2:172.21.15.143:6832/2150557544,v1:172.21.15.143:6833/2150557544] conn(0x7f95a00666b0 0x7f95a0068b70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:20:35.597 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:20:35.586+0000 7f95b2ffd640 1 -- 172.21.15.143:0/2086059958 <== mon.0 v2:172.21.15.143:3300/0 5 ==== osd_map(45..45 src has 1..45) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f95b811e970 con 0x7f95bc0cf7c0 2023-11-26T07:20:35.597 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:20:35.586+0000 7f95c1a45640 1 --2- 172.21.15.143:0/2086059958 >> [v2:172.21.15.143:6832/2150557544,v1:172.21.15.143:6833/2150557544] conn(0x7f95a00666b0 0x7f95a0068b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:20:35.597 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:20:35.586+0000 7f95c1a45640 1 --2- 172.21.15.143:0/2086059958 >> [v2:172.21.15.143:6832/2150557544,v1:172.21.15.143:6833/2150557544] conn(0x7f95a00666b0 0x7f95a0068b70 secure :-1 s=READY pgs=332 cs=0 l=1 rev1=1 crypto rx=0x7f95bc15f1a0 tx=0x7f95ac008040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:20:35.598 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:20:35.590+0000 7f95b2ffd640 1 -- 172.21.15.143:0/2086059958 <== mon.0 v2:172.21.15.143:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172649 (secure 0 0 0) 0x7f95b80eba20 con 0x7f95bc0cf7c0 2023-11-26T07:20:35.778 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:20:35.774+0000 7f95c44aa640 1 -- 172.21.15.143:0/2086059958 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f95bc06df50 con 0x7f95bc0cf7c0 2023-11-26T07:20:35.779 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:20:35.774+0000 7f95b2ffd640 1 -- 172.21.15.143:0/2086059958 <== mon.0 v2:172.21.15.143:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 26 v26) v1 ==== 94+0+4922 (secure 0 0 0) 0x7f95b80c3090 con 0x7f95bc0cf7c0 2023-11-26T07:20:35.779 INFO:teuthology.orchestra.run.smithi143.stdout: 2023-11-26T07:20:35.779 INFO:teuthology.orchestra.run.smithi143.stdout:{"epoch":26,"default_fscid":3,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5201,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.188:6835/319464848","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6833","nonce":319464848},{"type":"v1","addr":"172.21.15.188:6835","nonce":319464848}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":20},{"gid":5584,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.188:6834/2113408070","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6832","nonce":2113408070},{"type":"v1","addr":"172.21.15.188:6834","nonce":2113408070}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":20},{"gid":5745,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.143:6835/1616742593","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6834","nonce":1616742593},{"type":"v1","addr":"172.21.15.143:6835","nonce":1616742593}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":20}],"filesystems":[{"mdsmap":{"epoch":25,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-11-26T07:19:31.503587+0000","modified":"2023-11-26T07:19:34.523605+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":5748},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5748":{"gid":5748,"name":"c","rank":0,"incarnation":21,"state":"up:active","state_seq":3,"addr":"172.21.15.143:6837/2872033338","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6836","nonce":2872033338},{"type":"v1","addr":"172.21.15.143:6837","nonce":2872033338}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[7],"metadata_pool":6,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":3}]} 2023-11-26T07:20:35.780 INFO:teuthology.orchestra.run.smithi143.stderr:dumped fsmap epoch 26 2023-11-26T07:20:35.780 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:20:35.774+0000 7f95c44aa640 1 -- 172.21.15.143:0/2086059958 >> [v2:172.21.15.143:6832/2150557544,v1:172.21.15.143:6833/2150557544] conn(0x7f95a00666b0 msgr2=0x7f95a0068b70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:20:35.781 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:20:35.774+0000 7f95c44aa640 1 --2- 172.21.15.143:0/2086059958 >> [v2:172.21.15.143:6832/2150557544,v1:172.21.15.143:6833/2150557544] conn(0x7f95a00666b0 0x7f95a0068b70 secure :-1 s=READY pgs=332 cs=0 l=1 rev1=1 crypto rx=0x7f95bc15f1a0 tx=0x7f95ac008040 comp rx=0 tx=0).stop 2023-11-26T07:20:35.781 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:20:35.774+0000 7f95c44aa640 1 -- 172.21.15.143:0/2086059958 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f95bc0cf7c0 msgr2=0x7f95bc1358a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:20:35.781 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:20:35.774+0000 7f95c44aa640 1 --2- 172.21.15.143:0/2086059958 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f95bc0cf7c0 0x7f95bc1358a0 secure :-1 s=READY pgs=693 cs=0 l=1 rev1=1 crypto rx=0x7f95b80ac040 tx=0x7f95b801ad10 comp rx=0 tx=0).stop 2023-11-26T07:20:35.781 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:20:35.774+0000 7f95c44aa640 1 -- 172.21.15.143:0/2086059958 shutdown_connections 2023-11-26T07:20:35.781 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:20:35.774+0000 7f95c44aa640 1 --2- 172.21.15.143:0/2086059958 >> [v2:172.21.15.143:6832/2150557544,v1:172.21.15.143:6833/2150557544] conn(0x7f95a00666b0 0x7f95a0068b70 unknown :-1 s=CLOSED pgs=332 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:20:35.781 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:20:35.774+0000 7f95c44aa640 1 --2- 172.21.15.143:0/2086059958 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f95bc15f4e0 0x7f95bc0b0680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:20:35.782 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:20:35.774+0000 7f95c44aa640 1 --2- 172.21.15.143:0/2086059958 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f95bc13e290 0x7f95bc15df90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:20:35.782 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:20:35.774+0000 7f95c44aa640 1 --2- 172.21.15.143:0/2086059958 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f95bc0cf7c0 0x7f95bc1358a0 unknown :-1 s=CLOSED pgs=693 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:20:35.782 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:20:35.774+0000 7f95c44aa640 1 -- 172.21.15.143:0/2086059958 >> 172.21.15.143:0/2086059958 conn(0x7f95bc06a890 msgr2=0x7f95bc13b270 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:20:35.782 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:20:35.774+0000 7f95c44aa640 1 -- 172.21.15.143:0/2086059958 shutdown_connections 2023-11-26T07:20:35.783 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:20:35.774+0000 7f95c44aa640 1 -- 172.21.15.143:0/2086059958 wait complete. 2023-11-26T07:20:35.800 DEBUG:teuthology.orchestra.run.smithi143:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.c.asok --format=json session ls 2023-11-26T07:20:35.959 INFO:teuthology.orchestra.run.smithi143.stdout:[{"id":6126,"entity":{"name":{"type":"client","num":6126},"addr":{"type":"any","addr":"172.21.15.143:0","nonce":3879234400}},"state":"open","num_leases":1,"num_caps":7,"request_load_avg":10,"uptime":28.731013999000002,"requests_in_flight":0,"num_completed_requests":2,"num_completed_flushes":1,"reconnecting":false,"recall_caps":{"value":0,"halflife":60},"release_caps":{"value":0,"halflife":60},"recall_caps_throttle":{"value":0,"halflife":1.5},"recall_caps_throttle2o":{"value":0,"halflife":0.5},"session_cache_liveness":{"value":20.587646693957893,"halflife":300},"cap_acquisition":{"value":0,"halflife":30},"last_trim_completed_requests_tid":14,"last_trim_completed_flushes_tid":2,"delegated_inos":[{"start":"0x10000000005","length":500}],"inst":"client.6126 172.21.15.143:0/3879234400","completed_requests":[{"tid":14,"created_ino":"0x0"},{"tid":15,"created_ino":"0x100000001f9"}],"prealloc_inos":[{"start":"0x10000000005","length":500},{"start":"0x100000001fa","length":1000}],"client_metadata":{"client_features":{"feature_bits":"0x00000000003fffff"},"metric_spec":{"metric_flags":{"feature_bits":"0x000000000000ffff"}},"ceph_sha1":"44c2237fe77713214148c1fd622c872f346f1b99","ceph_version":"ceph version 18.0.0-7484-g44c2237f (44c2237fe77713214148c1fd622c872f346f1b99) reef (dev)","entity_id":"y","hostname":"smithi143","pid":"38097","root":"/"}}] 2023-11-26T07:20:35.960 DEBUG:tasks.cephfs.filesystem:_json_asok output [ { "cap_acquisition": { "halflife": 30, "value": 0 }, "client_metadata": { "ceph_sha1": "44c2237fe77713214148c1fd622c872f346f1b99", "ceph_version": "ceph version 18.0.0-7484-g44c2237f (44c2237fe77713214148c1fd622c872f346f1b99) reef (dev)", "client_features": { "feature_bits": "0x00000000003fffff" }, "entity_id": "y", "hostname": "smithi143", "metric_spec": { "metric_flags": { "feature_bits": "0x000000000000ffff" } }, "pid": "38097", "root": "/" }, "completed_requests": [ { "created_ino": "0x0", "tid": 14 }, { "created_ino": "0x100000001f9", "tid": 15 } ], "delegated_inos": [ { "length": 500, "start": "0x10000000005" } ], "entity": { "addr": { "addr": "172.21.15.143:0", "nonce": 3879234400, "type": "any" }, "name": { "num": 6126, "type": "client" } }, "id": 6126, "inst": "client.6126 172.21.15.143:0/3879234400", "last_trim_completed_flushes_tid": 2, "last_trim_completed_requests_tid": 14, "num_caps": 7, "num_completed_flushes": 1, "num_completed_requests": 2, "num_leases": 1, "prealloc_inos": [ { "length": 500, "start": "0x10000000005" }, { "length": 1000, "start": "0x100000001fa" } ], "recall_caps": { "halflife": 60, "value": 0 }, "recall_caps_throttle": { "halflife": 1.5, "value": 0 }, "recall_caps_throttle2o": { "halflife": 0.5, "value": 0 }, "reconnecting": false, "release_caps": { "halflife": 60, "value": 0 }, "request_load_avg": 10, "requests_in_flight": 0, "session_cache_liveness": { "halflife": 300, "value": 20.587646693957893 }, "state": "open", "uptime": 28.731013999 } ] 2023-11-26T07:20:35.960 DEBUG:tasks.ceph_test_case:wait_until_true: waiting (timeout=90 retry_count=0)... 2023-11-26T07:20:40.961 DEBUG:teuthology.orchestra.run.smithi143:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-11-26T07:20:41.109 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:20:41.106+0000 7f6f270e3640 1 Processor -- start 2023-11-26T07:20:41.110 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:20:41.106+0000 7f6f270e3640 1 -- start start 2023-11-26T07:20:41.111 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:20:41.106+0000 7f6f270e3640 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7f6f20143ac0 0x7f6f20070370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:20:41.111 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:20:41.106+0000 7f6f270e3640 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7f6f200708b0 0x7f6f200cf280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:20:41.112 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:20:41.106+0000 7f6f270e3640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f6f200cf7c0 0x7f6f200b1ef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:20:41.112 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:20:41.106+0000 7f6f270e3640 1 -- --> v2:172.21.15.143:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6f20143ea0 con 0x7f6f20143ac0 2023-11-26T07:20:41.112 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:20:41.106+0000 7f6f270e3640 1 -- --> v2:172.21.15.188:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6f200cc730 con 0x7f6f200708b0 2023-11-26T07:20:41.112 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:20:41.106+0000 7f6f270e3640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6f200cfbd0 con 0x7f6f200cf7c0 2023-11-26T07:20:41.112 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:20:41.106+0000 7f6f24e7f640 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7f6f20143ac0 0x7f6f20070370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:20:41.112 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:20:41.106+0000 7f6f24e7f640 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7f6f20143ac0 0x7f6f20070370 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:46038/0 (socket says 172.21.15.143:46038) 2023-11-26T07:20:41.112 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:20:41.106+0000 7f6f24e7f640 1 -- 172.21.15.143:0/2157514906 learned_addr learned my addr 172.21.15.143:0/2157514906 (peer_addr_for_me v2:172.21.15.143:0/0) 2023-11-26T07:20:41.112 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:20:41.106+0000 7f6f24e7f640 1 -- 172.21.15.143:0/2157514906 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f6f200cf7c0 msgr2=0x7f6f200b1ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-26T07:20:41.112 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:20:41.106+0000 7f6f24e7f640 1 --2- 172.21.15.143:0/2157514906 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f6f200cf7c0 0x7f6f200b1ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:20:41.112 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:20:41.106+0000 7f6f24e7f640 1 -- 172.21.15.143:0/2157514906 >> v2:172.21.15.188:3300/0 conn(0x7f6f200708b0 msgr2=0x7f6f200cf280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-26T07:20:41.112 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:20:41.106+0000 7f6f1ffff640 1 --2- 172.21.15.143:0/2157514906 >> v2:172.21.15.188:3300/0 conn(0x7f6f200708b0 0x7f6f200cf280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:20:41.112 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:20:41.106+0000 7f6f24e7f640 1 --2- 172.21.15.143:0/2157514906 >> v2:172.21.15.188:3300/0 conn(0x7f6f200708b0 0x7f6f200cf280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:20:41.113 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:20:41.106+0000 7f6f24e7f640 1 -- 172.21.15.143:0/2157514906 --> v2:172.21.15.143:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6f20070ee0 con 0x7f6f20143ac0 2023-11-26T07:20:41.113 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:20:41.106+0000 7f6f1ffff640 1 --2- 172.21.15.143:0/2157514906 >> v2:172.21.15.188:3300/0 conn(0x7f6f200708b0 0x7f6f200cf280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-26T07:20:41.114 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:20:41.110+0000 7f6f24e7f640 1 --2- 172.21.15.143:0/2157514906 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f6f20143ac0 0x7f6f20070370 secure :-1 s=READY pgs=694 cs=0 l=1 rev1=1 crypto rx=0x7f6f200aeb10 tx=0x7f6f1401c240 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d889252cc5b1487 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:20:41.115 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:20:41.110+0000 7f6f1f7fe640 1 -- 172.21.15.143:0/2157514906 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6f140ab600 con 0x7f6f20143ac0 2023-11-26T07:20:41.115 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:20:41.110+0000 7f6f1f7fe640 1 -- 172.21.15.143:0/2157514906 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6f140a7050 con 0x7f6f20143ac0 2023-11-26T07:20:41.115 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:20:41.110+0000 7f6f1f7fe640 1 -- 172.21.15.143:0/2157514906 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6f140ab980 con 0x7f6f20143ac0 2023-11-26T07:20:41.115 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:20:41.110+0000 7f6f270e3640 1 -- 172.21.15.143:0/2157514906 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f6f20143ac0 msgr2=0x7f6f20070370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:20:41.116 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:20:41.110+0000 7f6f270e3640 1 --2- 172.21.15.143:0/2157514906 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f6f20143ac0 0x7f6f20070370 secure :-1 s=READY pgs=694 cs=0 l=1 rev1=1 crypto rx=0x7f6f200aeb10 tx=0x7f6f1401c240 comp rx=0 tx=0).stop 2023-11-26T07:20:41.116 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:20:41.110+0000 7f6f270e3640 1 -- 172.21.15.143:0/2157514906 shutdown_connections 2023-11-26T07:20:41.116 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:20:41.110+0000 7f6f270e3640 1 --2- 172.21.15.143:0/2157514906 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f6f200cf7c0 0x7f6f200b1ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:20:41.116 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:20:41.110+0000 7f6f270e3640 1 --2- 172.21.15.143:0/2157514906 >> v2:172.21.15.188:3300/0 conn(0x7f6f200708b0 0x7f6f200cf280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:20:41.116 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:20:41.110+0000 7f6f270e3640 1 --2- 172.21.15.143:0/2157514906 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f6f20143ac0 0x7f6f20070370 unknown :-1 s=CLOSED pgs=694 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:20:41.116 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:20:41.110+0000 7f6f270e3640 1 -- 172.21.15.143:0/2157514906 >> 172.21.15.143:0/2157514906 conn(0x7f6f2006a890 msgr2=0x7f6f2006aca0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:20:41.116 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:20:41.110+0000 7f6f270e3640 1 -- 172.21.15.143:0/2157514906 shutdown_connections 2023-11-26T07:20:41.116 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:20:41.110+0000 7f6f270e3640 1 -- 172.21.15.143:0/2157514906 wait complete. 2023-11-26T07:20:41.117 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:20:41.114+0000 7f6f270e3640 1 Processor -- start 2023-11-26T07:20:41.117 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:20:41.114+0000 7f6f270e3640 1 -- start start 2023-11-26T07:20:41.119 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:20:41.114+0000 7f6f270e3640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f6f200708b0 0x7f6f20140d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:20:41.119 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:20:41.114+0000 7f6f270e3640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f6f200cf7c0 0x7f6f20133370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:20:41.119 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:20:41.114+0000 7f6f270e3640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f6f20143ac0 0x7f6f200af280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:20:41.119 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:20:41.114+0000 7f6f270e3640 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6f20142010 con 0x7f6f20143ac0 2023-11-26T07:20:41.120 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:20:41.114+0000 7f6f270e3640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6f20141390 con 0x7f6f200708b0 2023-11-26T07:20:41.120 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:20:41.114+0000 7f6f270e3640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6f20141670 con 0x7f6f200cf7c0 2023-11-26T07:20:41.120 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:20:41.114+0000 7f6f25680640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f6f20143ac0 0x7f6f200af280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:20:41.120 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:20:41.114+0000 7f6f24e7f640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f6f200708b0 0x7f6f20140d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:20:41.120 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:20:41.114+0000 7f6f24e7f640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f6f200708b0 0x7f6f20140d90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.188:3300/0 says I am v2:172.21.15.143:40300/0 (socket says 172.21.15.143:40300) 2023-11-26T07:20:41.120 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:20:41.114+0000 7f6f25680640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f6f20143ac0 0x7f6f200af280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:46052/0 (socket says 172.21.15.143:46052) 2023-11-26T07:20:41.120 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:20:41.114+0000 7f6f24e7f640 1 -- 172.21.15.143:0/2528675286 learned_addr learned my addr 172.21.15.143:0/2528675286 (peer_addr_for_me v2:172.21.15.143:0/0) 2023-11-26T07:20:41.120 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:20:41.114+0000 7f6f1ffff640 1 --2- 172.21.15.143:0/2528675286 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f6f200cf7c0 0x7f6f20133370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:20:41.120 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:20:41.114+0000 7f6f25680640 1 -- 172.21.15.143:0/2528675286 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f6f200cf7c0 msgr2=0x7f6f20133370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:20:41.120 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:20:41.114+0000 7f6f25680640 1 --2- 172.21.15.143:0/2528675286 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f6f200cf7c0 0x7f6f20133370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:20:41.120 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:20:41.114+0000 7f6f25680640 1 -- 172.21.15.143:0/2528675286 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f6f200708b0 msgr2=0x7f6f20140d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:20:41.120 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:20:41.114+0000 7f6f25680640 1 --2- 172.21.15.143:0/2528675286 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f6f200708b0 0x7f6f20140d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:20:41.121 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:20:41.114+0000 7f6f25680640 1 -- 172.21.15.143:0/2528675286 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6f140094d0 con 0x7f6f20143ac0 2023-11-26T07:20:41.121 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:20:41.114+0000 7f6f24e7f640 1 --2- 172.21.15.143:0/2528675286 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f6f200708b0 0x7f6f20140d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-26T07:20:41.121 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:20:41.114+0000 7f6f25680640 1 --2- 172.21.15.143:0/2528675286 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f6f20143ac0 0x7f6f200af280 secure :-1 s=READY pgs=695 cs=0 l=1 rev1=1 crypto rx=0x7f6f1800be20 tx=0x7f6f1800bf10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:20:41.121 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:20:41.114+0000 7f6f1d7fa640 1 -- 172.21.15.143:0/2528675286 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6f1800ce70 con 0x7f6f20143ac0 2023-11-26T07:20:41.121 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:20:41.114+0000 7f6f1d7fa640 1 -- 172.21.15.143:0/2528675286 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6f18014440 con 0x7f6f20143ac0 2023-11-26T07:20:41.122 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:20:41.114+0000 7f6f270e3640 1 -- 172.21.15.143:0/2528675286 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6f20141950 con 0x7f6f20143ac0 2023-11-26T07:20:41.122 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:20:41.114+0000 7f6f1d7fa640 1 -- 172.21.15.143:0/2528675286 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6f18014810 con 0x7f6f20143ac0 2023-11-26T07:20:41.122 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:20:41.114+0000 7f6f270e3640 1 -- 172.21.15.143:0/2528675286 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6f2013f880 con 0x7f6f20143ac0 2023-11-26T07:20:41.122 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:20:41.118+0000 7f6f270e3640 1 -- 172.21.15.143:0/2528675286 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6f20142150 con 0x7f6f20143ac0 2023-11-26T07:20:41.122 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:20:41.118+0000 7f6f1d7fa640 1 -- 172.21.15.143:0/2528675286 <== mon.0 v2:172.21.15.143:3300/0 4 ==== mgrmap(e 8) v1 ==== 86056+0+0 (secure 0 0 0) 0x7f6f1801d3b0 con 0x7f6f20143ac0 2023-11-26T07:20:41.123 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:20:41.118+0000 7f6f1d7fa640 1 --2- 172.21.15.143:0/2528675286 >> [v2:172.21.15.143:6832/2150557544,v1:172.21.15.143:6833/2150557544] conn(0x7f6f040666b0 0x7f6f04068b70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:20:41.123 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:20:41.118+0000 7f6f1d7fa640 1 -- 172.21.15.143:0/2528675286 <== mon.0 v2:172.21.15.143:3300/0 5 ==== osd_map(47..47 src has 1..47) v4 ==== 5767+0+0 (secure 0 0 0) 0x7f6f18084540 con 0x7f6f20143ac0 2023-11-26T07:20:41.123 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:20:41.118+0000 7f6f24e7f640 1 --2- 172.21.15.143:0/2528675286 >> [v2:172.21.15.143:6832/2150557544,v1:172.21.15.143:6833/2150557544] conn(0x7f6f040666b0 0x7f6f04068b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:20:41.123 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:20:41.118+0000 7f6f24e7f640 1 --2- 172.21.15.143:0/2528675286 >> [v2:172.21.15.143:6832/2150557544,v1:172.21.15.143:6833/2150557544] conn(0x7f6f040666b0 0x7f6f04068b70 secure :-1 s=READY pgs=333 cs=0 l=1 rev1=1 crypto rx=0x7f6f1401c280 tx=0x7f6f140a6000 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:20:41.127 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:20:41.122+0000 7f6f1d7fa640 1 -- 172.21.15.143:0/2528675286 <== mon.0 v2:172.21.15.143:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172649 (secure 0 0 0) 0x7f6f20142150 con 0x7f6f20143ac0 2023-11-26T07:20:41.306 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:20:41.302+0000 7f6f270e3640 1 -- 172.21.15.143:0/2528675286 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f6f20142150 con 0x7f6f20143ac0 2023-11-26T07:20:41.307 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:20:41.302+0000 7f6f1d7fa640 1 -- 172.21.15.143:0/2528675286 <== mon.0 v2:172.21.15.143:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 26 v26) v1 ==== 94+0+4922 (secure 0 0 0) 0x7f6f20142150 con 0x7f6f20143ac0 2023-11-26T07:20:41.307 INFO:teuthology.orchestra.run.smithi143.stdout: 2023-11-26T07:20:41.307 INFO:teuthology.orchestra.run.smithi143.stdout:{"epoch":26,"default_fscid":3,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5201,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.188:6835/319464848","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6833","nonce":319464848},{"type":"v1","addr":"172.21.15.188:6835","nonce":319464848}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":20},{"gid":5584,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.188:6834/2113408070","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6832","nonce":2113408070},{"type":"v1","addr":"172.21.15.188:6834","nonce":2113408070}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":20},{"gid":5745,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.143:6835/1616742593","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6834","nonce":1616742593},{"type":"v1","addr":"172.21.15.143:6835","nonce":1616742593}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":20}],"filesystems":[{"mdsmap":{"epoch":25,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-11-26T07:19:31.503587+0000","modified":"2023-11-26T07:19:34.523605+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":5748},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5748":{"gid":5748,"name":"c","rank":0,"incarnation":21,"state":"up:active","state_seq":3,"addr":"172.21.15.143:6837/2872033338","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6836","nonce":2872033338},{"type":"v1","addr":"172.21.15.143:6837","nonce":2872033338}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[7],"metadata_pool":6,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":3}]} 2023-11-26T07:20:41.308 INFO:teuthology.orchestra.run.smithi143.stderr:dumped fsmap epoch 26 2023-11-26T07:20:41.309 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:20:41.306+0000 7f6f270e3640 1 -- 172.21.15.143:0/2528675286 >> [v2:172.21.15.143:6832/2150557544,v1:172.21.15.143:6833/2150557544] conn(0x7f6f040666b0 msgr2=0x7f6f04068b70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:20:41.309 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:20:41.306+0000 7f6f270e3640 1 --2- 172.21.15.143:0/2528675286 >> [v2:172.21.15.143:6832/2150557544,v1:172.21.15.143:6833/2150557544] conn(0x7f6f040666b0 0x7f6f04068b70 secure :-1 s=READY pgs=333 cs=0 l=1 rev1=1 crypto rx=0x7f6f1401c280 tx=0x7f6f140a6000 comp rx=0 tx=0).stop 2023-11-26T07:20:41.309 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:20:41.306+0000 7f6f270e3640 1 -- 172.21.15.143:0/2528675286 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f6f20143ac0 msgr2=0x7f6f200af280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:20:41.310 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:20:41.306+0000 7f6f270e3640 1 --2- 172.21.15.143:0/2528675286 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f6f20143ac0 0x7f6f200af280 secure :-1 s=READY pgs=695 cs=0 l=1 rev1=1 crypto rx=0x7f6f1800be20 tx=0x7f6f1800bf10 comp rx=0 tx=0).stop 2023-11-26T07:20:41.310 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:20:41.306+0000 7f6f270e3640 1 -- 172.21.15.143:0/2528675286 shutdown_connections 2023-11-26T07:20:41.310 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:20:41.306+0000 7f6f270e3640 1 --2- 172.21.15.143:0/2528675286 >> [v2:172.21.15.143:6832/2150557544,v1:172.21.15.143:6833/2150557544] conn(0x7f6f040666b0 0x7f6f04068b70 unknown :-1 s=CLOSED pgs=333 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:20:41.310 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:20:41.306+0000 7f6f270e3640 1 --2- 172.21.15.143:0/2528675286 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f6f20143ac0 0x7f6f200af280 unknown :-1 s=CLOSED pgs=695 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:20:41.310 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:20:41.306+0000 7f6f270e3640 1 --2- 172.21.15.143:0/2528675286 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f6f200cf7c0 0x7f6f20133370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:20:41.310 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:20:41.306+0000 7f6f270e3640 1 --2- 172.21.15.143:0/2528675286 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f6f200708b0 0x7f6f20140d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:20:41.310 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:20:41.306+0000 7f6f270e3640 1 -- 172.21.15.143:0/2528675286 >> 172.21.15.143:0/2528675286 conn(0x7f6f2006a890 msgr2=0x7f6f2015b210 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:20:41.310 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:20:41.306+0000 7f6f270e3640 1 -- 172.21.15.143:0/2528675286 shutdown_connections 2023-11-26T07:20:41.310 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:20:41.306+0000 7f6f270e3640 1 -- 172.21.15.143:0/2528675286 wait complete. 2023-11-26T07:20:41.328 DEBUG:teuthology.orchestra.run.smithi143:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.c.asok --format=json session ls 2023-11-26T07:20:41.486 INFO:teuthology.orchestra.run.smithi143.stdout:[{"id":6126,"entity":{"name":{"type":"client","num":6126},"addr":{"type":"any","addr":"172.21.15.143:0","nonce":3879234400}},"state":"open","num_leases":0,"num_caps":7,"request_load_avg":10,"uptime":34.25888561,"requests_in_flight":0,"num_completed_requests":2,"num_completed_flushes":1,"reconnecting":false,"recall_caps":{"value":0,"halflife":60},"release_caps":{"value":0,"halflife":60},"recall_caps_throttle":{"value":0,"halflife":1.5},"recall_caps_throttle2o":{"value":0,"halflife":0.5},"session_cache_liveness":{"value":20.326371374592792,"halflife":300},"cap_acquisition":{"value":0,"halflife":30},"last_trim_completed_requests_tid":14,"last_trim_completed_flushes_tid":2,"delegated_inos":[{"start":"0x10000000005","length":500}],"inst":"client.6126 172.21.15.143:0/3879234400","completed_requests":[{"tid":14,"created_ino":"0x0"},{"tid":15,"created_ino":"0x100000001f9"}],"prealloc_inos":[{"start":"0x10000000005","length":500},{"start":"0x100000001fa","length":1000}],"client_metadata":{"client_features":{"feature_bits":"0x00000000003fffff"},"metric_spec":{"metric_flags":{"feature_bits":"0x000000000000ffff"}},"ceph_sha1":"44c2237fe77713214148c1fd622c872f346f1b99","ceph_version":"ceph version 18.0.0-7484-g44c2237f (44c2237fe77713214148c1fd622c872f346f1b99) reef (dev)","entity_id":"y","hostname":"smithi143","pid":"38097","root":"/"}}] 2023-11-26T07:20:41.486 DEBUG:tasks.cephfs.filesystem:_json_asok output [ { "cap_acquisition": { "halflife": 30, "value": 0 }, "client_metadata": { "ceph_sha1": "44c2237fe77713214148c1fd622c872f346f1b99", "ceph_version": "ceph version 18.0.0-7484-g44c2237f (44c2237fe77713214148c1fd622c872f346f1b99) reef (dev)", "client_features": { "feature_bits": "0x00000000003fffff" }, "entity_id": "y", "hostname": "smithi143", "metric_spec": { "metric_flags": { "feature_bits": "0x000000000000ffff" } }, "pid": "38097", "root": "/" }, "completed_requests": [ { "created_ino": "0x0", "tid": 14 }, { "created_ino": "0x100000001f9", "tid": 15 } ], "delegated_inos": [ { "length": 500, "start": "0x10000000005" } ], "entity": { "addr": { "addr": "172.21.15.143:0", "nonce": 3879234400, "type": "any" }, "name": { "num": 6126, "type": "client" } }, "id": 6126, "inst": "client.6126 172.21.15.143:0/3879234400", "last_trim_completed_flushes_tid": 2, "last_trim_completed_requests_tid": 14, "num_caps": 7, "num_completed_flushes": 1, "num_completed_requests": 2, "num_leases": 0, "prealloc_inos": [ { "length": 500, "start": "0x10000000005" }, { "length": 1000, "start": "0x100000001fa" } ], "recall_caps": { "halflife": 60, "value": 0 }, "recall_caps_throttle": { "halflife": 1.5, "value": 0 }, "recall_caps_throttle2o": { "halflife": 0.5, "value": 0 }, "reconnecting": false, "release_caps": { "halflife": 60, "value": 0 }, "request_load_avg": 10, "requests_in_flight": 0, "session_cache_liveness": { "halflife": 300, "value": 20.326371374592792 }, "state": "open", "uptime": 34.25888561 } ] 2023-11-26T07:20:41.486 DEBUG:tasks.ceph_test_case:wait_until_true: waiting (timeout=90 retry_count=0)... 2023-11-26T07:20:46.487 DEBUG:teuthology.orchestra.run.smithi143:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-11-26T07:20:46.630 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:20:46.626+0000 7fca0c1f5640 1 Processor -- start 2023-11-26T07:20:46.630 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:20:46.626+0000 7fca0c1f5640 1 -- start start 2023-11-26T07:20:46.632 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:20:46.626+0000 7fca0c1f5640 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7fca04143ac0 0x7fca04070370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:20:46.632 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:20:46.626+0000 7fca0c1f5640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fca040cf990 0x7fca040cd1b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:20:46.632 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:20:46.626+0000 7fca0c1f5640 1 -- --> v1:172.21.15.188:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fca040cc730 con 0x7fca040708b0 2023-11-26T07:20:46.632 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:20:46.626+0000 7fca0c1f5640 1 -- --> v2:172.21.15.143:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fca04143ea0 con 0x7fca04143ac0 2023-11-26T07:20:46.632 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:20:46.626+0000 7fca0c1f5640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fca04070ee0 con 0x7fca040cf990 2023-11-26T07:20:46.632 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:20:46.626+0000 7fca09f91640 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7fca04143ac0 0x7fca04070370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:20:46.633 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:20:46.626+0000 7fca09f91640 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7fca04143ac0 0x7fca04070370 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:55058/0 (socket says 172.21.15.143:55058) 2023-11-26T07:20:46.633 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:20:46.626+0000 7fca09f91640 1 -- 172.21.15.143:0/2460349618 learned_addr learned my addr 172.21.15.143:0/2460349618 (peer_addr_for_me v2:172.21.15.143:0/0) 2023-11-26T07:20:46.633 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:20:46.626+0000 7fca0a792640 1 --2- 172.21.15.143:0/2460349618 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fca040cf990 0x7fca040cd1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:20:46.633 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:20:46.626+0000 7fca09f91640 1 -- 172.21.15.143:0/2460349618 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fca040cf990 msgr2=0x7fca040cd1b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-26T07:20:46.633 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:20:46.626+0000 7fca09f91640 1 --2- 172.21.15.143:0/2460349618 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fca040cf990 0x7fca040cd1b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:20:46.633 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:20:46.626+0000 7fca09f91640 1 -- 172.21.15.143:0/2460349618 >> v1:172.21.15.188:6789/0 conn(0x7fca040708b0 legacy=0x7fca040cf280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-26T07:20:46.633 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:20:46.626+0000 7fca09f91640 1 -- 172.21.15.143:0/2460349618 --> v2:172.21.15.143:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fca040711c0 con 0x7fca04143ac0 2023-11-26T07:20:46.635 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:20:46.630+0000 7fca09f91640 1 --2- 172.21.15.143:0/2460349618 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fca04143ac0 0x7fca04070370 secure :-1 s=READY pgs=696 cs=0 l=1 rev1=1 crypto rx=0x7fca0406dca0 tx=0x7fca0001c4c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8bc5f36232c5db3e server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:20:46.636 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:20:46.630+0000 7fca08f8f640 1 -- 172.21.15.143:0/2460349618 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fca000ab680 con 0x7fca04143ac0 2023-11-26T07:20:46.636 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:20:46.630+0000 7fca08f8f640 1 -- 172.21.15.143:0/2460349618 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fca000ab7e0 con 0x7fca04143ac0 2023-11-26T07:20:46.636 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:20:46.630+0000 7fca08f8f640 1 -- 172.21.15.143:0/2460349618 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fca000abb70 con 0x7fca04143ac0 2023-11-26T07:20:46.636 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:20:46.630+0000 7fca0c1f5640 1 -- 172.21.15.143:0/2460349618 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fca04143ac0 msgr2=0x7fca04070370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:20:46.636 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:20:46.630+0000 7fca0c1f5640 1 --2- 172.21.15.143:0/2460349618 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fca04143ac0 0x7fca04070370 secure :-1 s=READY pgs=696 cs=0 l=1 rev1=1 crypto rx=0x7fca0406dca0 tx=0x7fca0001c4c0 comp rx=0 tx=0).stop 2023-11-26T07:20:46.636 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:20:46.630+0000 7fca0c1f5640 1 -- 172.21.15.143:0/2460349618 shutdown_connections 2023-11-26T07:20:46.636 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:20:46.630+0000 7fca0c1f5640 1 --2- 172.21.15.143:0/2460349618 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fca040cf990 0x7fca040cd1b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:20:46.637 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:20:46.630+0000 7fca0c1f5640 1 --2- 172.21.15.143:0/2460349618 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fca04143ac0 0x7fca04070370 unknown :-1 s=CLOSED pgs=696 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:20:46.637 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:20:46.630+0000 7fca0c1f5640 1 -- 172.21.15.143:0/2460349618 >> 172.21.15.143:0/2460349618 conn(0x7fca0406a890 msgr2=0x7fca0406aca0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:20:46.637 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:20:46.630+0000 7fca0c1f5640 1 -- 172.21.15.143:0/2460349618 shutdown_connections 2023-11-26T07:20:46.637 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:20:46.630+0000 7fca0c1f5640 1 -- 172.21.15.143:0/2460349618 wait complete. 2023-11-26T07:20:46.638 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:20:46.634+0000 7fca0c1f5640 1 Processor -- start 2023-11-26T07:20:46.638 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:20:46.634+0000 7fca0c1f5640 1 -- start start 2023-11-26T07:20:46.639 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:20:46.634+0000 7fca0c1f5640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fca040708b0 0x7fca04158ef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:20:46.639 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:20:46.634+0000 7fca0c1f5640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fca040cf990 0x7fca041f5c60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:20:46.640 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:20:46.634+0000 7fca0c1f5640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fca04143ac0 0x7fca0413f3b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:20:46.640 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:20:46.634+0000 7fca0c1f5640 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fca0413fe70 con 0x7fca040cf990 2023-11-26T07:20:46.640 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:20:46.634+0000 7fca0c1f5640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fca04159430 con 0x7fca040708b0 2023-11-26T07:20:46.640 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:20:46.634+0000 7fca0c1f5640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fca0415a6c0 con 0x7fca04143ac0 2023-11-26T07:20:46.640 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:20:46.634+0000 7fca09790640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fca040cf990 0x7fca041f5c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:20:46.640 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:20:46.634+0000 7fca09790640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fca040cf990 0x7fca041f5c60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:55064/0 (socket says 172.21.15.143:55064) 2023-11-26T07:20:46.640 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:20:46.634+0000 7fca09790640 1 -- 172.21.15.143:0/2272976772 learned_addr learned my addr 172.21.15.143:0/2272976772 (peer_addr_for_me v2:172.21.15.143:0/0) 2023-11-26T07:20:46.640 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:20:46.634+0000 7fca09f91640 1 --2- 172.21.15.143:0/2272976772 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fca040708b0 0x7fca04158ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:20:46.640 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:20:46.634+0000 7fca0a792640 1 --2- 172.21.15.143:0/2272976772 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fca04143ac0 0x7fca0413f3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:20:46.640 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:20:46.634+0000 7fca09790640 1 -- 172.21.15.143:0/2272976772 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fca04143ac0 msgr2=0x7fca0413f3b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:20:46.640 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:20:46.634+0000 7fca09790640 1 --2- 172.21.15.143:0/2272976772 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fca04143ac0 0x7fca0413f3b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:20:46.640 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:20:46.634+0000 7fca09790640 1 -- 172.21.15.143:0/2272976772 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fca040708b0 msgr2=0x7fca04158ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:20:46.641 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:20:46.634+0000 7fca09790640 1 --2- 172.21.15.143:0/2272976772 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fca040708b0 0x7fca04158ef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:20:46.641 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:20:46.634+0000 7fca09790640 1 -- 172.21.15.143:0/2272976772 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fca000094d0 con 0x7fca040cf990 2023-11-26T07:20:46.641 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:20:46.634+0000 7fca09790640 1 --2- 172.21.15.143:0/2272976772 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fca040cf990 0x7fca041f5c60 secure :-1 s=READY pgs=697 cs=0 l=1 rev1=1 crypto rx=0x7fc9f400c450 tx=0x7fc9f400c910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:20:46.641 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:20:46.634+0000 7fc9fa7fc640 1 -- 172.21.15.143:0/2272976772 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc9f4012830 con 0x7fca040cf990 2023-11-26T07:20:46.642 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:20:46.634+0000 7fc9fa7fc640 1 -- 172.21.15.143:0/2272976772 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc9f40129b0 con 0x7fca040cf990 2023-11-26T07:20:46.642 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:20:46.634+0000 7fc9fa7fc640 1 -- 172.21.15.143:0/2272976772 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc9f4012d60 con 0x7fca040cf990 2023-11-26T07:20:46.642 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:20:46.634+0000 7fca0c1f5640 1 -- 172.21.15.143:0/2272976772 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fca0415a9a0 con 0x7fca040cf990 2023-11-26T07:20:46.642 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:20:46.634+0000 7fca0c1f5640 1 -- 172.21.15.143:0/2272976772 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fca0415af60 con 0x7fca040cf990 2023-11-26T07:20:46.642 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:20:46.638+0000 7fca0c1f5640 1 -- 172.21.15.143:0/2272976772 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fca0415b150 con 0x7fca040cf990 2023-11-26T07:20:46.642 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:20:46.638+0000 7fc9fa7fc640 1 -- 172.21.15.143:0/2272976772 <== mon.0 v2:172.21.15.143:3300/0 4 ==== mgrmap(e 8) v1 ==== 86056+0+0 (secure 0 0 0) 0x7fc9f401c460 con 0x7fca040cf990 2023-11-26T07:20:46.643 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:20:46.638+0000 7fc9fa7fc640 1 --2- 172.21.15.143:0/2272976772 >> [v2:172.21.15.143:6832/2150557544,v1:172.21.15.143:6833/2150557544] conn(0x7fc9e80666b0 0x7fc9e8068b70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:20:46.643 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:20:46.638+0000 7fc9fa7fc640 1 -- 172.21.15.143:0/2272976772 <== mon.0 v2:172.21.15.143:3300/0 5 ==== osd_map(47..47 src has 1..47) v4 ==== 5767+0+0 (secure 0 0 0) 0x7fc9f4083fc0 con 0x7fca040cf990 2023-11-26T07:20:46.643 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:20:46.638+0000 7fca09f91640 1 --2- 172.21.15.143:0/2272976772 >> [v2:172.21.15.143:6832/2150557544,v1:172.21.15.143:6833/2150557544] conn(0x7fc9e80666b0 0x7fc9e8068b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:20:46.643 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:20:46.638+0000 7fca09f91640 1 --2- 172.21.15.143:0/2272976772 >> [v2:172.21.15.143:6832/2150557544,v1:172.21.15.143:6833/2150557544] conn(0x7fc9e80666b0 0x7fc9e8068b70 secure :-1 s=READY pgs=334 cs=0 l=1 rev1=1 crypto rx=0x7fca000af000 tx=0x7fca000b6040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:20:46.647 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:20:46.642+0000 7fc9fa7fc640 1 -- 172.21.15.143:0/2272976772 <== mon.0 v2:172.21.15.143:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172649 (secure 0 0 0) 0x7fc9f4051140 con 0x7fca040cf990 2023-11-26T07:20:46.829 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:20:46.826+0000 7fca0c1f5640 1 -- 172.21.15.143:0/2272976772 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fca04067680 con 0x7fca040cf990 2023-11-26T07:20:46.829 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:20:46.826+0000 7fc9fa7fc640 1 -- 172.21.15.143:0/2272976772 <== mon.0 v2:172.21.15.143:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 26 v26) v1 ==== 94+0+4922 (secure 0 0 0) 0x7fc9f4054760 con 0x7fca040cf990 2023-11-26T07:20:46.830 INFO:teuthology.orchestra.run.smithi143.stdout: 2023-11-26T07:20:46.830 INFO:teuthology.orchestra.run.smithi143.stdout:{"epoch":26,"default_fscid":3,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5201,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.188:6835/319464848","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6833","nonce":319464848},{"type":"v1","addr":"172.21.15.188:6835","nonce":319464848}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":20},{"gid":5584,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.188:6834/2113408070","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6832","nonce":2113408070},{"type":"v1","addr":"172.21.15.188:6834","nonce":2113408070}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":20},{"gid":5745,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.143:6835/1616742593","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6834","nonce":1616742593},{"type":"v1","addr":"172.21.15.143:6835","nonce":1616742593}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":20}],"filesystems":[{"mdsmap":{"epoch":25,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-11-26T07:19:31.503587+0000","modified":"2023-11-26T07:19:34.523605+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":5748},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5748":{"gid":5748,"name":"c","rank":0,"incarnation":21,"state":"up:active","state_seq":3,"addr":"172.21.15.143:6837/2872033338","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6836","nonce":2872033338},{"type":"v1","addr":"172.21.15.143:6837","nonce":2872033338}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[7],"metadata_pool":6,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":3}]} 2023-11-26T07:20:46.830 INFO:teuthology.orchestra.run.smithi143.stderr:dumped fsmap epoch 26 2023-11-26T07:20:46.831 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:20:46.826+0000 7fca0c1f5640 1 -- 172.21.15.143:0/2272976772 >> [v2:172.21.15.143:6832/2150557544,v1:172.21.15.143:6833/2150557544] conn(0x7fc9e80666b0 msgr2=0x7fc9e8068b70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:20:46.832 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:20:46.826+0000 7fca0c1f5640 1 --2- 172.21.15.143:0/2272976772 >> [v2:172.21.15.143:6832/2150557544,v1:172.21.15.143:6833/2150557544] conn(0x7fc9e80666b0 0x7fc9e8068b70 secure :-1 s=READY pgs=334 cs=0 l=1 rev1=1 crypto rx=0x7fca000af000 tx=0x7fca000b6040 comp rx=0 tx=0).stop 2023-11-26T07:20:46.832 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:20:46.826+0000 7fca0c1f5640 1 -- 172.21.15.143:0/2272976772 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fca040cf990 msgr2=0x7fca041f5c60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:20:46.832 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:20:46.826+0000 7fca0c1f5640 1 --2- 172.21.15.143:0/2272976772 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fca040cf990 0x7fca041f5c60 secure :-1 s=READY pgs=697 cs=0 l=1 rev1=1 crypto rx=0x7fc9f400c450 tx=0x7fc9f400c910 comp rx=0 tx=0).stop 2023-11-26T07:20:46.832 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:20:46.826+0000 7fca0c1f5640 1 -- 172.21.15.143:0/2272976772 shutdown_connections 2023-11-26T07:20:46.832 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:20:46.826+0000 7fca0c1f5640 1 --2- 172.21.15.143:0/2272976772 >> [v2:172.21.15.143:6832/2150557544,v1:172.21.15.143:6833/2150557544] conn(0x7fc9e80666b0 0x7fc9e8068b70 unknown :-1 s=CLOSED pgs=334 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:20:46.832 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:20:46.826+0000 7fca0c1f5640 1 --2- 172.21.15.143:0/2272976772 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fca04143ac0 0x7fca0413f3b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:20:46.832 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:20:46.826+0000 7fca0c1f5640 1 --2- 172.21.15.143:0/2272976772 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fca040cf990 0x7fca041f5c60 unknown :-1 s=CLOSED pgs=697 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:20:46.832 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:20:46.826+0000 7fca0c1f5640 1 --2- 172.21.15.143:0/2272976772 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fca040708b0 0x7fca04158ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:20:46.832 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:20:46.826+0000 7fca0c1f5640 1 -- 172.21.15.143:0/2272976772 >> 172.21.15.143:0/2272976772 conn(0x7fca0406a890 msgr2=0x7fca040ace10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:20:46.832 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:20:46.826+0000 7fca0c1f5640 1 -- 172.21.15.143:0/2272976772 shutdown_connections 2023-11-26T07:20:46.832 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:20:46.826+0000 7fca0c1f5640 1 -- 172.21.15.143:0/2272976772 wait complete. 2023-11-26T07:20:46.850 DEBUG:teuthology.orchestra.run.smithi143:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.c.asok --format=json session ls 2023-11-26T07:20:47.007 INFO:teuthology.orchestra.run.smithi143.stdout:[{"id":6126,"entity":{"name":{"type":"client","num":6126},"addr":{"type":"any","addr":"172.21.15.143:0","nonce":3879234400}},"state":"open","num_leases":0,"num_caps":7,"request_load_avg":9,"uptime":39.778770498,"requests_in_flight":0,"num_completed_requests":2,"num_completed_flushes":1,"reconnecting":false,"recall_caps":{"value":0,"halflife":60},"release_caps":{"value":0,"halflife":60},"recall_caps_throttle":{"value":0,"halflife":1.5},"recall_caps_throttle2o":{"value":0,"halflife":0.5},"session_cache_liveness":{"value":20.068782199324009,"halflife":300},"cap_acquisition":{"value":0,"halflife":30},"last_trim_completed_requests_tid":14,"last_trim_completed_flushes_tid":2,"delegated_inos":[{"start":"0x10000000005","length":500}],"inst":"client.6126 172.21.15.143:0/3879234400","completed_requests":[{"tid":14,"created_ino":"0x0"},{"tid":15,"created_ino":"0x100000001f9"}],"prealloc_inos":[{"start":"0x10000000005","length":500},{"start":"0x100000001fa","length":1000}],"client_metadata":{"client_features":{"feature_bits":"0x00000000003fffff"},"metric_spec":{"metric_flags":{"feature_bits":"0x000000000000ffff"}},"ceph_sha1":"44c2237fe77713214148c1fd622c872f346f1b99","ceph_version":"ceph version 18.0.0-7484-g44c2237f (44c2237fe77713214148c1fd622c872f346f1b99) reef (dev)","entity_id":"y","hostname":"smithi143","pid":"38097","root":"/"}}] 2023-11-26T07:20:47.007 DEBUG:tasks.cephfs.filesystem:_json_asok output [ { "cap_acquisition": { "halflife": 30, "value": 0 }, "client_metadata": { "ceph_sha1": "44c2237fe77713214148c1fd622c872f346f1b99", "ceph_version": "ceph version 18.0.0-7484-g44c2237f (44c2237fe77713214148c1fd622c872f346f1b99) reef (dev)", "client_features": { "feature_bits": "0x00000000003fffff" }, "entity_id": "y", "hostname": "smithi143", "metric_spec": { "metric_flags": { "feature_bits": "0x000000000000ffff" } }, "pid": "38097", "root": "/" }, "completed_requests": [ { "created_ino": "0x0", "tid": 14 }, { "created_ino": "0x100000001f9", "tid": 15 } ], "delegated_inos": [ { "length": 500, "start": "0x10000000005" } ], "entity": { "addr": { "addr": "172.21.15.143:0", "nonce": 3879234400, "type": "any" }, "name": { "num": 6126, "type": "client" } }, "id": 6126, "inst": "client.6126 172.21.15.143:0/3879234400", "last_trim_completed_flushes_tid": 2, "last_trim_completed_requests_tid": 14, "num_caps": 7, "num_completed_flushes": 1, "num_completed_requests": 2, "num_leases": 0, "prealloc_inos": [ { "length": 500, "start": "0x10000000005" }, { "length": 1000, "start": "0x100000001fa" } ], "recall_caps": { "halflife": 60, "value": 0 }, "recall_caps_throttle": { "halflife": 1.5, "value": 0 }, "recall_caps_throttle2o": { "halflife": 0.5, "value": 0 }, "reconnecting": false, "release_caps": { "halflife": 60, "value": 0 }, "request_load_avg": 9, "requests_in_flight": 0, "session_cache_liveness": { "halflife": 300, "value": 20.06878219932401 }, "state": "open", "uptime": 39.778770498 } ] 2023-11-26T07:20:47.007 DEBUG:tasks.ceph_test_case:wait_until_true: waiting (timeout=90 retry_count=0)... 2023-11-26T07:20:52.008 DEBUG:teuthology.orchestra.run.smithi143:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-11-26T07:20:52.047 DEBUG:teuthology.orchestra.run.smithi143:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-26T07:20:52.059 DEBUG:teuthology.orchestra.run.smithi188:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-26T07:20:52.151 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:20:52.146+0000 7f668dfaa640 1 Processor -- start 2023-11-26T07:20:52.152 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:20:52.146+0000 7f668dfaa640 1 -- start start 2023-11-26T07:20:52.154 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:20:52.150+0000 7f668dfaa640 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7f66880cf280 0x7f66880cf710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:20:52.154 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:20:52.150+0000 7f668dfaa640 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7f66880cd1b0 0x7f66880cd5c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:20:52.154 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:20:52.150+0000 7f668dfaa640 1 -- --> v1:172.21.15.188:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f66880cc730 con 0x7f6688143ac0 2023-11-26T07:20:52.154 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:20:52.150+0000 7f668dfaa640 1 -- --> v2:172.21.15.143:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6688143ea0 con 0x7f66880cf280 2023-11-26T07:20:52.154 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:20:52.150+0000 7f668dfaa640 1 -- --> v2:172.21.15.188:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f66880cfc50 con 0x7f66880cd1b0 2023-11-26T07:20:52.154 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:20:52.150+0000 7f6686ffd640 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7f66880cf280 0x7f66880cf710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:20:52.154 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:20:52.150+0000 7f6686ffd640 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7f66880cf280 0x7f66880cf710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:55066/0 (socket says 172.21.15.143:55066) 2023-11-26T07:20:52.155 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:20:52.150+0000 7f6686ffd640 1 -- 172.21.15.143:0/1032791404 learned_addr learned my addr 172.21.15.143:0/1032791404 (peer_addr_for_me v2:172.21.15.143:0/0) 2023-11-26T07:20:52.155 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:20:52.150+0000 7f66877fe640 1 --1- >> v1:172.21.15.188:6789/0 conn(0x7f6688143ac0 0x7f6688070370 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.188:6789/0 says I am v1:172.21.15.143:36272/0 (socket says 172.21.15.143:36272) 2023-11-26T07:20:52.155 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:20:52.150+0000 7f6687fff640 1 --2- 172.21.15.143:0/1032791404 >> v2:172.21.15.188:3300/0 conn(0x7f66880cd1b0 0x7f66880cd5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:20:52.155 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:20:52.150+0000 7f6686ffd640 1 -- 172.21.15.143:0/1032791404 >> v2:172.21.15.188:3300/0 conn(0x7f66880cd1b0 msgr2=0x7f66880cd5c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-26T07:20:52.156 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:20:52.150+0000 7f6686ffd640 1 --2- 172.21.15.143:0/1032791404 >> v2:172.21.15.188:3300/0 conn(0x7f66880cd1b0 0x7f66880cd5c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:20:52.156 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:20:52.150+0000 7f6686ffd640 1 -- 172.21.15.143:0/1032791404 >> v1:172.21.15.188:6789/0 conn(0x7f6688143ac0 legacy=0x7f6688070370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:20:52.156 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:20:52.150+0000 7f6686ffd640 1 -- 172.21.15.143:0/1032791404 --> v2:172.21.15.143:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6688070ee0 con 0x7f66880cf280 2023-11-26T07:20:52.157 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:20:52.154+0000 7f6686ffd640 1 --2- 172.21.15.143:0/1032791404 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f66880cf280 0x7f66880cf710 secure :-1 s=READY pgs=698 cs=0 l=1 rev1=1 crypto rx=0x7f668806e570 tx=0x7f667c01c500 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c657b9de397bb01c server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:20:52.157 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:20:52.154+0000 7f66867fc640 1 -- 172.21.15.143:0/1032791404 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f667c0aa660 con 0x7f66880cf280 2023-11-26T07:20:52.158 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:20:52.154+0000 7f66867fc640 1 -- 172.21.15.143:0/1032791404 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f667c0aa7c0 con 0x7f66880cf280 2023-11-26T07:20:52.158 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:20:52.154+0000 7f66867fc640 1 -- 172.21.15.143:0/1032791404 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f667c0aab50 con 0x7f66880cf280 2023-11-26T07:20:52.158 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:20:52.154+0000 7f668dfaa640 1 -- 172.21.15.143:0/1032791404 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f66880cf280 msgr2=0x7f66880cf710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:20:52.159 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:20:52.154+0000 7f668dfaa640 1 --2- 172.21.15.143:0/1032791404 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f66880cf280 0x7f66880cf710 secure :-1 s=READY pgs=698 cs=0 l=1 rev1=1 crypto rx=0x7f668806e570 tx=0x7f667c01c500 comp rx=0 tx=0).stop 2023-11-26T07:20:52.159 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:20:52.154+0000 7f668dfaa640 1 -- 172.21.15.143:0/1032791404 shutdown_connections 2023-11-26T07:20:52.159 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:20:52.154+0000 7f668dfaa640 1 --2- 172.21.15.143:0/1032791404 >> v2:172.21.15.188:3300/0 conn(0x7f66880cd1b0 0x7f66880cd5c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:20:52.159 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:20:52.154+0000 7f668dfaa640 1 --2- 172.21.15.143:0/1032791404 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f66880cf280 0x7f66880cf710 unknown :-1 s=CLOSED pgs=698 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:20:52.159 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:20:52.154+0000 7f668dfaa640 1 -- 172.21.15.143:0/1032791404 >> 172.21.15.143:0/1032791404 conn(0x7f668806a890 msgr2=0x7f668806aca0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:20:52.159 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:20:52.154+0000 7f668dfaa640 1 -- 172.21.15.143:0/1032791404 shutdown_connections 2023-11-26T07:20:52.160 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:20:52.154+0000 7f668dfaa640 1 -- 172.21.15.143:0/1032791404 wait complete. 2023-11-26T07:20:52.160 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:20:52.154+0000 7f668dfaa640 1 Processor -- start 2023-11-26T07:20:52.160 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:20:52.154+0000 7f668dfaa640 1 -- start start 2023-11-26T07:20:52.162 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:20:52.158+0000 7f668dfaa640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f66880cd1b0 0x7f6688157670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:20:52.163 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:20:52.158+0000 7f668dfaa640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f66880cf280 0x7f66881385d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:20:52.163 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:20:52.158+0000 7f668dfaa640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f6688143ac0 0x7f66881582e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:20:52.163 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:20:52.158+0000 7f668dfaa640 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f66881394e0 con 0x7f6688143ac0 2023-11-26T07:20:52.163 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:20:52.158+0000 7f668dfaa640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f66881560e0 con 0x7f66880cf280 2023-11-26T07:20:52.163 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:20:52.158+0000 7f668dfaa640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6688138c40 con 0x7f66880cd1b0 2023-11-26T07:20:52.163 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:20:52.158+0000 7f6687fff640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f6688143ac0 0x7f66881582e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:20:52.164 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:20:52.158+0000 7f6687fff640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f6688143ac0 0x7f66881582e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:55076/0 (socket says 172.21.15.143:55076) 2023-11-26T07:20:52.164 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:20:52.158+0000 7f6687fff640 1 -- 172.21.15.143:0/864888119 learned_addr learned my addr 172.21.15.143:0/864888119 (peer_addr_for_me v2:172.21.15.143:0/0) 2023-11-26T07:20:52.164 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:20:52.158+0000 7f6686ffd640 1 --2- 172.21.15.143:0/864888119 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f66880cf280 0x7f66881385d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:20:52.164 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:20:52.158+0000 7f66877fe640 1 --2- 172.21.15.143:0/864888119 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f66880cd1b0 0x7f6688157670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:20:52.164 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:20:52.158+0000 7f6687fff640 1 -- 172.21.15.143:0/864888119 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f66880cd1b0 msgr2=0x7f6688157670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:20:52.164 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:20:52.158+0000 7f6687fff640 1 --2- 172.21.15.143:0/864888119 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f66880cd1b0 0x7f6688157670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:20:52.164 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:20:52.158+0000 7f6687fff640 1 -- 172.21.15.143:0/864888119 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f66880cf280 msgr2=0x7f66881385d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:20:52.164 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:20:52.158+0000 7f6687fff640 1 --2- 172.21.15.143:0/864888119 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f66880cf280 0x7f66881385d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:20:52.164 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:20:52.158+0000 7f6687fff640 1 -- 172.21.15.143:0/864888119 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f667c0094d0 con 0x7f6688143ac0 2023-11-26T07:20:52.165 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:20:52.158+0000 7f6687fff640 1 --2- 172.21.15.143:0/864888119 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f6688143ac0 0x7f66881582e0 secure :-1 s=READY pgs=699 cs=0 l=1 rev1=1 crypto rx=0x7f668000efc0 tx=0x7f668000c490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:20:52.165 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:20:52.158+0000 7f666ffff640 1 -- 172.21.15.143:0/864888119 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6680009ba0 con 0x7f6688143ac0 2023-11-26T07:20:52.165 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:20:52.158+0000 7f666ffff640 1 -- 172.21.15.143:0/864888119 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f668000f040 con 0x7f6688143ac0 2023-11-26T07:20:52.165 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:20:52.158+0000 7f668dfaa640 1 -- 172.21.15.143:0/864888119 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6688138f20 con 0x7f6688143ac0 2023-11-26T07:20:52.165 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:20:52.158+0000 7f666ffff640 1 -- 172.21.15.143:0/864888119 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f66800146c0 con 0x7f6688143ac0 2023-11-26T07:20:52.165 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:20:52.158+0000 7f668dfaa640 1 -- 172.21.15.143:0/864888119 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f668813b340 con 0x7f6688143ac0 2023-11-26T07:20:52.166 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:20:52.158+0000 7f668dfaa640 1 -- 172.21.15.143:0/864888119 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f668813b1a0 con 0x7f6688143ac0 2023-11-26T07:20:52.166 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:20:52.162+0000 7f666ffff640 1 -- 172.21.15.143:0/864888119 <== mon.0 v2:172.21.15.143:3300/0 4 ==== mgrmap(e 8) v1 ==== 86056+0+0 (secure 0 0 0) 0x7f6680014820 con 0x7f6688143ac0 2023-11-26T07:20:52.167 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:20:52.162+0000 7f666ffff640 1 --2- 172.21.15.143:0/864888119 >> [v2:172.21.15.143:6832/2150557544,v1:172.21.15.143:6833/2150557544] conn(0x7f66740666b0 0x7f6674068b70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:20:52.167 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:20:52.162+0000 7f666ffff640 1 -- 172.21.15.143:0/864888119 <== mon.0 v2:172.21.15.143:3300/0 5 ==== osd_map(47..47 src has 1..47) v4 ==== 5767+0+0 (secure 0 0 0) 0x7f6680084d80 con 0x7f6688143ac0 2023-11-26T07:20:52.167 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:20:52.162+0000 7f66877fe640 1 --2- 172.21.15.143:0/864888119 >> [v2:172.21.15.143:6832/2150557544,v1:172.21.15.143:6833/2150557544] conn(0x7f66740666b0 0x7f6674068b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:20:52.167 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:20:52.162+0000 7f66877fe640 1 --2- 172.21.15.143:0/864888119 >> [v2:172.21.15.143:6832/2150557544,v1:172.21.15.143:6833/2150557544] conn(0x7f66740666b0 0x7f6674068b70 secure :-1 s=READY pgs=335 cs=0 l=1 rev1=1 crypto rx=0x7f6678000f70 tx=0x7f6678007cb0 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:20:52.170 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:20:52.166+0000 7f666ffff640 1 -- 172.21.15.143:0/864888119 <== mon.0 v2:172.21.15.143:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172649 (secure 0 0 0) 0x7f668813b1a0 con 0x7f6688143ac0 2023-11-26T07:20:52.351 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:20:52.346+0000 7f668dfaa640 1 -- 172.21.15.143:0/864888119 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f668813b1a0 con 0x7f6688143ac0 2023-11-26T07:20:52.352 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:20:52.346+0000 7f666ffff640 1 -- 172.21.15.143:0/864888119 <== mon.0 v2:172.21.15.143:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 26 v26) v1 ==== 94+0+4922 (secure 0 0 0) 0x7f668813b1a0 con 0x7f6688143ac0 2023-11-26T07:20:52.352 INFO:teuthology.orchestra.run.smithi143.stdout: 2023-11-26T07:20:52.352 INFO:teuthology.orchestra.run.smithi143.stdout:{"epoch":26,"default_fscid":3,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5201,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.188:6835/319464848","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6833","nonce":319464848},{"type":"v1","addr":"172.21.15.188:6835","nonce":319464848}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":20},{"gid":5584,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.188:6834/2113408070","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6832","nonce":2113408070},{"type":"v1","addr":"172.21.15.188:6834","nonce":2113408070}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":20},{"gid":5745,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.143:6835/1616742593","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6834","nonce":1616742593},{"type":"v1","addr":"172.21.15.143:6835","nonce":1616742593}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":20}],"filesystems":[{"mdsmap":{"epoch":25,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-11-26T07:19:31.503587+0000","modified":"2023-11-26T07:19:34.523605+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":5748},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5748":{"gid":5748,"name":"c","rank":0,"incarnation":21,"state":"up:active","state_seq":3,"addr":"172.21.15.143:6837/2872033338","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6836","nonce":2872033338},{"type":"v1","addr":"172.21.15.143:6837","nonce":2872033338}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[7],"metadata_pool":6,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":3}]} 2023-11-26T07:20:52.353 INFO:teuthology.orchestra.run.smithi143.stderr:dumped fsmap epoch 26 2023-11-26T07:20:52.354 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:20:52.350+0000 7f668dfaa640 1 -- 172.21.15.143:0/864888119 >> [v2:172.21.15.143:6832/2150557544,v1:172.21.15.143:6833/2150557544] conn(0x7f66740666b0 msgr2=0x7f6674068b70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:20:52.354 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:20:52.350+0000 7f668dfaa640 1 --2- 172.21.15.143:0/864888119 >> [v2:172.21.15.143:6832/2150557544,v1:172.21.15.143:6833/2150557544] conn(0x7f66740666b0 0x7f6674068b70 secure :-1 s=READY pgs=335 cs=0 l=1 rev1=1 crypto rx=0x7f6678000f70 tx=0x7f6678007cb0 comp rx=0 tx=0).stop 2023-11-26T07:20:52.354 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:20:52.350+0000 7f668dfaa640 1 -- 172.21.15.143:0/864888119 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f6688143ac0 msgr2=0x7f66881582e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:20:52.354 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:20:52.350+0000 7f668dfaa640 1 --2- 172.21.15.143:0/864888119 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f6688143ac0 0x7f66881582e0 secure :-1 s=READY pgs=699 cs=0 l=1 rev1=1 crypto rx=0x7f668000efc0 tx=0x7f668000c490 comp rx=0 tx=0).stop 2023-11-26T07:20:52.354 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:20:52.350+0000 7f668dfaa640 1 -- 172.21.15.143:0/864888119 shutdown_connections 2023-11-26T07:20:52.354 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:20:52.350+0000 7f668dfaa640 1 --2- 172.21.15.143:0/864888119 >> [v2:172.21.15.143:6832/2150557544,v1:172.21.15.143:6833/2150557544] conn(0x7f66740666b0 0x7f6674068b70 unknown :-1 s=CLOSED pgs=335 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:20:52.354 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:20:52.350+0000 7f668dfaa640 1 --2- 172.21.15.143:0/864888119 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f6688143ac0 0x7f66881582e0 unknown :-1 s=CLOSED pgs=699 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:20:52.354 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:20:52.350+0000 7f668dfaa640 1 --2- 172.21.15.143:0/864888119 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f66880cf280 0x7f66881385d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:20:52.354 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:20:52.350+0000 7f668dfaa640 1 --2- 172.21.15.143:0/864888119 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f66880cd1b0 0x7f6688157670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:20:52.354 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:20:52.350+0000 7f668dfaa640 1 -- 172.21.15.143:0/864888119 >> 172.21.15.143:0/864888119 conn(0x7f668806a890 msgr2=0x7f66880b25a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:20:52.354 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:20:52.350+0000 7f668dfaa640 1 -- 172.21.15.143:0/864888119 shutdown_connections 2023-11-26T07:20:52.355 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:20:52.350+0000 7f668dfaa640 1 -- 172.21.15.143:0/864888119 wait complete. 2023-11-26T07:20:52.373 DEBUG:teuthology.orchestra.run.smithi143:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.c.asok --format=json session ls 2023-11-26T07:20:52.578 INFO:teuthology.orchestra.run.smithi143.stdout:[{"id":6126,"entity":{"name":{"type":"client","num":6126},"addr":{"type":"any","addr":"172.21.15.143:0","nonce":3879234400}},"state":"open","num_leases":0,"num_caps":7,"request_load_avg":8,"uptime":45.350664537,"requests_in_flight":0,"num_completed_requests":2,"num_completed_flushes":1,"reconnecting":false,"recall_caps":{"value":0,"halflife":60},"release_caps":{"value":0,"halflife":60},"recall_caps_throttle":{"value":0,"halflife":1.5},"recall_caps_throttle2o":{"value":0,"halflife":0.5},"session_cache_liveness":{"value":19.812076469863346,"halflife":300},"cap_acquisition":{"value":0,"halflife":30},"last_trim_completed_requests_tid":14,"last_trim_completed_flushes_tid":2,"delegated_inos":[{"start":"0x10000000005","length":500}],"inst":"client.6126 172.21.15.143:0/3879234400","completed_requests":[{"tid":14,"created_ino":"0x0"},{"tid":15,"created_ino":"0x100000001f9"}],"prealloc_inos":[{"start":"0x10000000005","length":500},{"start":"0x100000001fa","length":1000}],"client_metadata":{"client_features":{"feature_bits":"0x00000000003fffff"},"metric_spec":{"metric_flags":{"feature_bits":"0x000000000000ffff"}},"ceph_sha1":"44c2237fe77713214148c1fd622c872f346f1b99","ceph_version":"ceph version 18.0.0-7484-g44c2237f (44c2237fe77713214148c1fd622c872f346f1b99) reef (dev)","entity_id":"y","hostname":"smithi143","pid":"38097","root":"/"}}] 2023-11-26T07:20:52.578 DEBUG:tasks.cephfs.filesystem:_json_asok output [ { "cap_acquisition": { "halflife": 30, "value": 0 }, "client_metadata": { "ceph_sha1": "44c2237fe77713214148c1fd622c872f346f1b99", "ceph_version": "ceph version 18.0.0-7484-g44c2237f (44c2237fe77713214148c1fd622c872f346f1b99) reef (dev)", "client_features": { "feature_bits": "0x00000000003fffff" }, "entity_id": "y", "hostname": "smithi143", "metric_spec": { "metric_flags": { "feature_bits": "0x000000000000ffff" } }, "pid": "38097", "root": "/" }, "completed_requests": [ { "created_ino": "0x0", "tid": 14 }, { "created_ino": "0x100000001f9", "tid": 15 } ], "delegated_inos": [ { "length": 500, "start": "0x10000000005" } ], "entity": { "addr": { "addr": "172.21.15.143:0", "nonce": 3879234400, "type": "any" }, "name": { "num": 6126, "type": "client" } }, "id": 6126, "inst": "client.6126 172.21.15.143:0/3879234400", "last_trim_completed_flushes_tid": 2, "last_trim_completed_requests_tid": 14, "num_caps": 7, "num_completed_flushes": 1, "num_completed_requests": 2, "num_leases": 0, "prealloc_inos": [ { "length": 500, "start": "0x10000000005" }, { "length": 1000, "start": "0x100000001fa" } ], "recall_caps": { "halflife": 60, "value": 0 }, "recall_caps_throttle": { "halflife": 1.5, "value": 0 }, "recall_caps_throttle2o": { "halflife": 0.5, "value": 0 }, "reconnecting": false, "release_caps": { "halflife": 60, "value": 0 }, "request_load_avg": 8, "requests_in_flight": 0, "session_cache_liveness": { "halflife": 300, "value": 19.812076469863346 }, "state": "open", "uptime": 45.350664537 } ] 2023-11-26T07:20:52.578 DEBUG:tasks.ceph_test_case:wait_until_true: waiting (timeout=90 retry_count=0)... 2023-11-26T07:20:57.580 DEBUG:teuthology.orchestra.run.smithi143:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-11-26T07:20:57.729 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:20:57.725+0000 7febbdfd0640 1 Processor -- start 2023-11-26T07:20:57.730 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:20:57.725+0000 7febbdfd0640 1 -- start start 2023-11-26T07:20:57.730 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:20:57.725+0000 7febbdfd0640 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7febb806c0e0 0x7febb80ada60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:20:57.730 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:20:57.725+0000 7febbdfd0640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7febb80cfaf0 0x7febb80cffa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:20:57.730 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:20:57.725+0000 7febbdfd0640 1 -- --> v1:172.21.15.188:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7febb80cc050 con 0x7febb80b23f0 2023-11-26T07:20:57.731 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:20:57.725+0000 7febbdfd0640 1 -- --> v2:172.21.15.188:3300/0 -- mon_getmap magic: 0 v1 -- 0x7febb80cb440 con 0x7febb806c0e0 2023-11-26T07:20:57.731 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:20:57.725+0000 7febbdfd0640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7febb80cb720 con 0x7febb80cfaf0 2023-11-26T07:20:57.731 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:20:57.725+0000 7febb7fff640 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7febb806c0e0 0x7febb80ada60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:20:57.732 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:20:57.725+0000 7febbd7cf640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7febb80cfaf0 0x7febb80cffa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:20:57.732 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:20:57.725+0000 7febbcfce640 1 --1- >> v1:172.21.15.188:6789/0 conn(0x7febb80b23f0 0x7febb806b9d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.188:6789/0 says I am v1:172.21.15.143:46330/0 (socket says 172.21.15.143:46330) 2023-11-26T07:20:57.732 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:20:57.725+0000 7febb7fff640 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7febb806c0e0 0x7febb80ada60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.188:3300/0 says I am v2:172.21.15.143:52350/0 (socket says 172.21.15.143:52350) 2023-11-26T07:20:57.732 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:20:57.725+0000 7febbcfce640 1 -- 172.21.15.143:0/2825254142 learned_addr learned my addr 172.21.15.143:0/2825254142 (peer_addr_for_me v1:172.21.15.143:0/0) 2023-11-26T07:20:57.732 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:20:57.725+0000 7febbd7cf640 1 -- 172.21.15.143:0/2825254142 >> v2:172.21.15.188:3300/0 conn(0x7febb806c0e0 msgr2=0x7febb80ada60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-26T07:20:57.732 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:20:57.725+0000 7febbd7cf640 1 --2- 172.21.15.143:0/2825254142 >> v2:172.21.15.188:3300/0 conn(0x7febb806c0e0 0x7febb80ada60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:20:57.732 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:20:57.725+0000 7febbd7cf640 1 -- 172.21.15.143:0/2825254142 >> v1:172.21.15.188:6789/0 conn(0x7febb80b23f0 legacy=0x7febb806b9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:20:57.732 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:20:57.725+0000 7febbd7cf640 1 -- 172.21.15.143:0/2825254142 --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7febb80cba00 con 0x7febb80cfaf0 2023-11-26T07:20:57.732 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:20:57.725+0000 7febb7fff640 1 --2- 172.21.15.143:0/2825254142 >> v2:172.21.15.188:3300/0 conn(0x7febb806c0e0 0x7febb80ada60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-26T07:20:57.734 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:20:57.729+0000 7febbd7cf640 1 --2- 172.21.15.143:0/2825254142 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7febb80cfaf0 0x7febb80cffa0 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7febb0009d90 tx=0x7febb001d3b0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=57cf83239218736e server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:20:57.735 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:20:57.729+0000 7febb77fe640 1 -- 172.21.15.143:0/2825254142 <== mon.2 v2:172.21.15.188:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7febb00a5920 con 0x7febb80cfaf0 2023-11-26T07:20:57.735 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:20:57.729+0000 7febb77fe640 1 -- 172.21.15.143:0/2825254142 <== mon.2 v2:172.21.15.188:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7febb00a5a80 con 0x7febb80cfaf0 2023-11-26T07:20:57.735 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:20:57.729+0000 7febb77fe640 1 -- 172.21.15.143:0/2825254142 <== mon.2 v2:172.21.15.188:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7febb00a5e10 con 0x7febb80cfaf0 2023-11-26T07:20:57.735 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:20:57.729+0000 7febbdfd0640 1 -- 172.21.15.143:0/2825254142 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7febb80cfaf0 msgr2=0x7febb80cffa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:20:57.735 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:20:57.729+0000 7febbdfd0640 1 --2- 172.21.15.143:0/2825254142 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7febb80cfaf0 0x7febb80cffa0 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7febb0009d90 tx=0x7febb001d3b0 comp rx=0 tx=0).stop 2023-11-26T07:20:57.736 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:20:57.729+0000 7febbdfd0640 1 -- 172.21.15.143:0/2825254142 shutdown_connections 2023-11-26T07:20:57.736 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:20:57.729+0000 7febbdfd0640 1 --2- 172.21.15.143:0/2825254142 >> v2:172.21.15.188:3300/0 conn(0x7febb806c0e0 0x7febb80ada60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:20:57.736 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:20:57.729+0000 7febbdfd0640 1 --2- 172.21.15.143:0/2825254142 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7febb80cfaf0 0x7febb80cffa0 unknown :-1 s=CLOSED pgs=95 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:20:57.736 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:20:57.729+0000 7febbdfd0640 1 -- 172.21.15.143:0/2825254142 >> 172.21.15.143:0/2825254142 conn(0x7febb80d0df0 msgr2=0x7febb80d1200 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:20:57.736 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:20:57.729+0000 7febbdfd0640 1 -- 172.21.15.143:0/2825254142 shutdown_connections 2023-11-26T07:20:57.737 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:20:57.729+0000 7febbdfd0640 1 -- 172.21.15.143:0/2825254142 wait complete. 2023-11-26T07:20:57.737 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:20:57.733+0000 7febbdfd0640 1 Processor -- start 2023-11-26T07:20:57.737 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:20:57.733+0000 7febbdfd0640 1 -- start start 2023-11-26T07:20:57.739 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:20:57.733+0000 7febbdfd0640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7febb806c0e0 0x7febb815d0e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:20:57.739 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:20:57.733+0000 7febbdfd0640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7febb80b23f0 0x7febb80c6920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:20:57.739 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:20:57.733+0000 7febbdfd0640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7febb80cfaf0 0x7febb80c6e60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:20:57.739 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:20:57.733+0000 7febbdfd0640 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x7febb80c7870 con 0x7febb806c0e0 2023-11-26T07:20:57.739 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:20:57.733+0000 7febbdfd0640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7febb80c73a0 con 0x7febb80b23f0 2023-11-26T07:20:57.739 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:20:57.733+0000 7febbdfd0640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7febb815d620 con 0x7febb80cfaf0 2023-11-26T07:20:57.740 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:20:57.733+0000 7febbcfce640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7febb806c0e0 0x7febb815d0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:20:57.740 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:20:57.733+0000 7febbcfce640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7febb806c0e0 0x7febb815d0e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:49528/0 (socket says 172.21.15.143:49528) 2023-11-26T07:20:57.740 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:20:57.733+0000 7febbcfce640 1 -- 172.21.15.143:0/2633546975 learned_addr learned my addr 172.21.15.143:0/2633546975 (peer_addr_for_me v2:172.21.15.143:0/0) 2023-11-26T07:20:57.740 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:20:57.733+0000 7febb7fff640 1 --2- 172.21.15.143:0/2633546975 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7febb80b23f0 0x7febb80c6920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:20:57.740 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:20:57.733+0000 7febbd7cf640 1 --2- 172.21.15.143:0/2633546975 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7febb80cfaf0 0x7febb80c6e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:20:57.740 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:20:57.733+0000 7febbcfce640 1 -- 172.21.15.143:0/2633546975 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7febb80cfaf0 msgr2=0x7febb80c6e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:20:57.740 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:20:57.733+0000 7febbcfce640 1 --2- 172.21.15.143:0/2633546975 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7febb80cfaf0 0x7febb80c6e60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:20:57.740 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:20:57.733+0000 7febbcfce640 1 -- 172.21.15.143:0/2633546975 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7febb80b23f0 msgr2=0x7febb80c6920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:20:57.740 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:20:57.733+0000 7febbcfce640 1 --2- 172.21.15.143:0/2633546975 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7febb80b23f0 0x7febb80c6920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:20:57.740 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:20:57.733+0000 7febbcfce640 1 -- 172.21.15.143:0/2633546975 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7feba4009420 con 0x7febb806c0e0 2023-11-26T07:20:57.740 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:20:57.733+0000 7febbd7cf640 1 --2- 172.21.15.143:0/2633546975 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7febb80cfaf0 0x7febb80c6e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-26T07:20:57.741 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:20:57.733+0000 7febb7fff640 1 --2- 172.21.15.143:0/2633546975 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7febb80b23f0 0x7febb80c6920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-26T07:20:57.741 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:20:57.733+0000 7febbcfce640 1 --2- 172.21.15.143:0/2633546975 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7febb806c0e0 0x7febb815d0e0 secure :-1 s=READY pgs=700 cs=0 l=1 rev1=1 crypto rx=0x7febac00c3a0 tx=0x7febac00c860 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:20:57.741 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:20:57.733+0000 7febb57fa640 1 -- 172.21.15.143:0/2633546975 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7febac005cf0 con 0x7febb806c0e0 2023-11-26T07:20:57.742 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:20:57.733+0000 7febb57fa640 1 -- 172.21.15.143:0/2633546975 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7febac005e50 con 0x7febb806c0e0 2023-11-26T07:20:57.742 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:20:57.733+0000 7febbdfd0640 1 -- 172.21.15.143:0/2633546975 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7febb0009a70 con 0x7febb806c0e0 2023-11-26T07:20:57.742 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:20:57.733+0000 7febb57fa640 1 -- 172.21.15.143:0/2633546975 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7febac0072f0 con 0x7febb806c0e0 2023-11-26T07:20:57.742 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:20:57.733+0000 7febbdfd0640 1 -- 172.21.15.143:0/2633546975 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7febb80c54d0 con 0x7febb806c0e0 2023-11-26T07:20:57.742 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:20:57.737+0000 7febbdfd0640 1 -- 172.21.15.143:0/2633546975 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7febb806d4f0 con 0x7febb806c0e0 2023-11-26T07:20:57.742 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:20:57.737+0000 7febb57fa640 1 -- 172.21.15.143:0/2633546975 <== mon.0 v2:172.21.15.143:3300/0 4 ==== mgrmap(e 8) v1 ==== 86056+0+0 (secure 0 0 0) 0x7febac007450 con 0x7febb806c0e0 2023-11-26T07:20:57.743 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:20:57.737+0000 7febb57fa640 1 --2- 172.21.15.143:0/2633546975 >> [v2:172.21.15.143:6832/2150557544,v1:172.21.15.143:6833/2150557544] conn(0x7feb9c0666b0 0x7feb9c068b70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:20:57.743 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:20:57.737+0000 7febb57fa640 1 -- 172.21.15.143:0/2633546975 <== mon.0 v2:172.21.15.143:3300/0 5 ==== osd_map(47..47 src has 1..47) v4 ==== 5767+0+0 (secure 0 0 0) 0x7febac0838f0 con 0x7febb806c0e0 2023-11-26T07:20:57.743 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:20:57.737+0000 7febb7fff640 1 --2- 172.21.15.143:0/2633546975 >> [v2:172.21.15.143:6832/2150557544,v1:172.21.15.143:6833/2150557544] conn(0x7feb9c0666b0 0x7feb9c068b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:20:57.743 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:20:57.737+0000 7febb7fff640 1 --2- 172.21.15.143:0/2633546975 >> [v2:172.21.15.143:6832/2150557544,v1:172.21.15.143:6833/2150557544] conn(0x7feb9c0666b0 0x7feb9c068b70 secure :-1 s=READY pgs=336 cs=0 l=1 rev1=1 crypto rx=0x7febb815a850 tx=0x7feba4008040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:20:57.746 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:20:57.741+0000 7febb57fa640 1 -- 172.21.15.143:0/2633546975 <== mon.0 v2:172.21.15.143:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172649 (secure 0 0 0) 0x7febac050930 con 0x7febb806c0e0 2023-11-26T07:20:57.927 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:20:57.921+0000 7febbdfd0640 1 -- 172.21.15.143:0/2633546975 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7febb8067500 con 0x7febb806c0e0 2023-11-26T07:20:57.927 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:20:57.921+0000 7febb57fa640 1 -- 172.21.15.143:0/2633546975 <== mon.0 v2:172.21.15.143:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 26 v26) v1 ==== 94+0+4922 (secure 0 0 0) 0x7febac017090 con 0x7febb806c0e0 2023-11-26T07:20:57.928 INFO:teuthology.orchestra.run.smithi143.stdout: 2023-11-26T07:20:57.928 INFO:teuthology.orchestra.run.smithi143.stdout:{"epoch":26,"default_fscid":3,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5201,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.188:6835/319464848","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6833","nonce":319464848},{"type":"v1","addr":"172.21.15.188:6835","nonce":319464848}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":20},{"gid":5584,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.188:6834/2113408070","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6832","nonce":2113408070},{"type":"v1","addr":"172.21.15.188:6834","nonce":2113408070}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":20},{"gid":5745,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.143:6835/1616742593","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6834","nonce":1616742593},{"type":"v1","addr":"172.21.15.143:6835","nonce":1616742593}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":20}],"filesystems":[{"mdsmap":{"epoch":25,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-11-26T07:19:31.503587+0000","modified":"2023-11-26T07:19:34.523605+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":5748},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5748":{"gid":5748,"name":"c","rank":0,"incarnation":21,"state":"up:active","state_seq":3,"addr":"172.21.15.143:6837/2872033338","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6836","nonce":2872033338},{"type":"v1","addr":"172.21.15.143:6837","nonce":2872033338}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[7],"metadata_pool":6,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":3}]} 2023-11-26T07:20:57.928 INFO:teuthology.orchestra.run.smithi143.stderr:dumped fsmap epoch 26 2023-11-26T07:20:57.929 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:20:57.925+0000 7febbdfd0640 1 -- 172.21.15.143:0/2633546975 >> [v2:172.21.15.143:6832/2150557544,v1:172.21.15.143:6833/2150557544] conn(0x7feb9c0666b0 msgr2=0x7feb9c068b70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:20:57.929 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:20:57.925+0000 7febbdfd0640 1 --2- 172.21.15.143:0/2633546975 >> [v2:172.21.15.143:6832/2150557544,v1:172.21.15.143:6833/2150557544] conn(0x7feb9c0666b0 0x7feb9c068b70 secure :-1 s=READY pgs=336 cs=0 l=1 rev1=1 crypto rx=0x7febb815a850 tx=0x7feba4008040 comp rx=0 tx=0).stop 2023-11-26T07:20:57.930 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:20:57.925+0000 7febbdfd0640 1 -- 172.21.15.143:0/2633546975 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7febb806c0e0 msgr2=0x7febb815d0e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:20:57.930 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:20:57.925+0000 7febbdfd0640 1 --2- 172.21.15.143:0/2633546975 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7febb806c0e0 0x7febb815d0e0 secure :-1 s=READY pgs=700 cs=0 l=1 rev1=1 crypto rx=0x7febac00c3a0 tx=0x7febac00c860 comp rx=0 tx=0).stop 2023-11-26T07:20:57.930 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:20:57.925+0000 7febbdfd0640 1 -- 172.21.15.143:0/2633546975 shutdown_connections 2023-11-26T07:20:57.930 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:20:57.925+0000 7febbdfd0640 1 --2- 172.21.15.143:0/2633546975 >> [v2:172.21.15.143:6832/2150557544,v1:172.21.15.143:6833/2150557544] conn(0x7feb9c0666b0 0x7feb9c068b70 unknown :-1 s=CLOSED pgs=336 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:20:57.930 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:20:57.925+0000 7febbdfd0640 1 --2- 172.21.15.143:0/2633546975 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7febb80cfaf0 0x7febb80c6e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:20:57.930 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:20:57.925+0000 7febbdfd0640 1 --2- 172.21.15.143:0/2633546975 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7febb80b23f0 0x7febb80c6920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:20:57.930 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:20:57.925+0000 7febbdfd0640 1 --2- 172.21.15.143:0/2633546975 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7febb806c0e0 0x7febb815d0e0 unknown :-1 s=CLOSED pgs=700 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:20:57.930 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:20:57.925+0000 7febbdfd0640 1 -- 172.21.15.143:0/2633546975 >> 172.21.15.143:0/2633546975 conn(0x7febb80d0df0 msgr2=0x7febb80cf030 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:20:57.930 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:20:57.925+0000 7febbdfd0640 1 -- 172.21.15.143:0/2633546975 shutdown_connections 2023-11-26T07:20:57.930 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:20:57.925+0000 7febbdfd0640 1 -- 172.21.15.143:0/2633546975 wait complete. 2023-11-26T07:20:57.949 DEBUG:teuthology.orchestra.run.smithi143:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.c.asok --format=json session ls 2023-11-26T07:20:58.106 INFO:teuthology.orchestra.run.smithi143.stdout:[{"id":6126,"entity":{"name":{"type":"client","num":6126},"addr":{"type":"any","addr":"172.21.15.143:0","nonce":3879234400}},"state":"open","num_leases":0,"num_caps":7,"request_load_avg":8,"uptime":50.878568487000003,"requests_in_flight":0,"num_completed_requests":2,"num_completed_flushes":1,"reconnecting":false,"recall_caps":{"value":0,"halflife":60},"release_caps":{"value":0,"halflife":60},"recall_caps_throttle":{"value":0,"halflife":1.5},"recall_caps_throttle2o":{"value":0,"halflife":0.5},"session_cache_liveness":{"value":19.560642356300455,"halflife":300},"cap_acquisition":{"value":0,"halflife":30},"last_trim_completed_requests_tid":14,"last_trim_completed_flushes_tid":2,"delegated_inos":[{"start":"0x10000000005","length":500}],"inst":"client.6126 172.21.15.143:0/3879234400","completed_requests":[{"tid":14,"created_ino":"0x0"},{"tid":15,"created_ino":"0x100000001f9"}],"prealloc_inos":[{"start":"0x10000000005","length":500},{"start":"0x100000001fa","length":1000}],"client_metadata":{"client_features":{"feature_bits":"0x00000000003fffff"},"metric_spec":{"metric_flags":{"feature_bits":"0x000000000000ffff"}},"ceph_sha1":"44c2237fe77713214148c1fd622c872f346f1b99","ceph_version":"ceph version 18.0.0-7484-g44c2237f (44c2237fe77713214148c1fd622c872f346f1b99) reef (dev)","entity_id":"y","hostname":"smithi143","pid":"38097","root":"/"}}] 2023-11-26T07:20:58.106 DEBUG:tasks.cephfs.filesystem:_json_asok output [ { "cap_acquisition": { "halflife": 30, "value": 0 }, "client_metadata": { "ceph_sha1": "44c2237fe77713214148c1fd622c872f346f1b99", "ceph_version": "ceph version 18.0.0-7484-g44c2237f (44c2237fe77713214148c1fd622c872f346f1b99) reef (dev)", "client_features": { "feature_bits": "0x00000000003fffff" }, "entity_id": "y", "hostname": "smithi143", "metric_spec": { "metric_flags": { "feature_bits": "0x000000000000ffff" } }, "pid": "38097", "root": "/" }, "completed_requests": [ { "created_ino": "0x0", "tid": 14 }, { "created_ino": "0x100000001f9", "tid": 15 } ], "delegated_inos": [ { "length": 500, "start": "0x10000000005" } ], "entity": { "addr": { "addr": "172.21.15.143:0", "nonce": 3879234400, "type": "any" }, "name": { "num": 6126, "type": "client" } }, "id": 6126, "inst": "client.6126 172.21.15.143:0/3879234400", "last_trim_completed_flushes_tid": 2, "last_trim_completed_requests_tid": 14, "num_caps": 7, "num_completed_flushes": 1, "num_completed_requests": 2, "num_leases": 0, "prealloc_inos": [ { "length": 500, "start": "0x10000000005" }, { "length": 1000, "start": "0x100000001fa" } ], "recall_caps": { "halflife": 60, "value": 0 }, "recall_caps_throttle": { "halflife": 1.5, "value": 0 }, "recall_caps_throttle2o": { "halflife": 0.5, "value": 0 }, "reconnecting": false, "release_caps": { "halflife": 60, "value": 0 }, "request_load_avg": 8, "requests_in_flight": 0, "session_cache_liveness": { "halflife": 300, "value": 19.560642356300455 }, "state": "open", "uptime": 50.878568487 } ] 2023-11-26T07:20:58.106 DEBUG:tasks.ceph_test_case:wait_until_true: waiting (timeout=90 retry_count=0)... 2023-11-26T07:21:03.108 DEBUG:teuthology.orchestra.run.smithi143:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-11-26T07:21:03.257 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:21:03.253+0000 7fbbb6572640 1 Processor -- start 2023-11-26T07:21:03.257 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:21:03.253+0000 7fbbb6572640 1 -- start start 2023-11-26T07:21:03.259 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:21:03.253+0000 7fbbb6572640 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7fbbb01575c0 0x7fbbb006bac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:21:03.259 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:21:03.253+0000 7fbbb6572640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fbbb006aba0 0x7fbbb00cddd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:21:03.259 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:21:03.253+0000 7fbbb6572640 1 -- --> v1:172.21.15.143:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbbb00cd610 con 0x7fbbb0159e10 2023-11-26T07:21:03.259 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:21:03.253+0000 7fbbb6572640 1 -- --> v2:172.21.15.143:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fbbb015a6d0 con 0x7fbbb01575c0 2023-11-26T07:21:03.259 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:21:03.253+0000 7fbbb6572640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbbb015a9b0 con 0x7fbbb006aba0 2023-11-26T07:21:03.260 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:21:03.253+0000 7fbbaffff640 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7fbbb01575c0 0x7fbbb006bac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:21:03.260 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:21:03.253+0000 7fbbaf7fe640 1 --1- >> v1:172.21.15.143:6789/0 conn(0x7fbbb0159e10 0x7fbbb006a490 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.143:6789/0 says I am v1:172.21.15.143:41472/0 (socket says 172.21.15.143:41472) 2023-11-26T07:21:03.260 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:21:03.253+0000 7fbbaf7fe640 1 -- 172.21.15.143:0/227674957 learned_addr learned my addr 172.21.15.143:0/227674957 (peer_addr_for_me v1:172.21.15.143:0/0) 2023-11-26T07:21:03.260 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:21:03.253+0000 7fbbb4b0f640 1 --2- 172.21.15.143:0/227674957 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fbbb006aba0 0x7fbbb00cddd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:21:03.260 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:21:03.253+0000 7fbbaffff640 1 -- 172.21.15.143:0/227674957 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fbbb006aba0 msgr2=0x7fbbb00cddd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-26T07:21:03.260 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:21:03.253+0000 7fbbaffff640 1 --2- 172.21.15.143:0/227674957 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fbbb006aba0 0x7fbbb00cddd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:21:03.260 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:21:03.253+0000 7fbbaffff640 1 -- 172.21.15.143:0/227674957 >> v1:172.21.15.143:6789/0 conn(0x7fbbb0159e10 legacy=0x7fbbb006a490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:21:03.260 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:21:03.253+0000 7fbbaffff640 1 -- 172.21.15.143:0/227674957 --> v2:172.21.15.143:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbbb015ac90 con 0x7fbbb01575c0 2023-11-26T07:21:03.262 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:21:03.257+0000 7fbbaffff640 1 --2- 172.21.15.143:0/227674957 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fbbb01575c0 0x7fbbb006bac0 secure :-1 s=READY pgs=702 cs=0 l=1 rev1=1 crypto rx=0x7fbba00097c0 tx=0x7fbba001c3c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=42ef90b5704dbe93 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:21:03.262 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:21:03.257+0000 7fbbaeffd640 1 -- 172.21.15.143:0/227674957 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbba00ad660 con 0x7fbbb01575c0 2023-11-26T07:21:03.263 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:21:03.257+0000 7fbbaeffd640 1 -- 172.21.15.143:0/227674957 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbba00ad7c0 con 0x7fbbb01575c0 2023-11-26T07:21:03.263 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:21:03.257+0000 7fbbaeffd640 1 -- 172.21.15.143:0/227674957 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbba00adb50 con 0x7fbbb01575c0 2023-11-26T07:21:03.263 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:21:03.257+0000 7fbbb6572640 1 -- 172.21.15.143:0/227674957 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fbbb01575c0 msgr2=0x7fbbb006bac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:21:03.263 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:21:03.257+0000 7fbbb6572640 1 --2- 172.21.15.143:0/227674957 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fbbb01575c0 0x7fbbb006bac0 secure :-1 s=READY pgs=702 cs=0 l=1 rev1=1 crypto rx=0x7fbba00097c0 tx=0x7fbba001c3c0 comp rx=0 tx=0).stop 2023-11-26T07:21:03.264 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:21:03.257+0000 7fbbb6572640 1 -- 172.21.15.143:0/227674957 shutdown_connections 2023-11-26T07:21:03.264 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:21:03.257+0000 7fbbb6572640 1 --2- 172.21.15.143:0/227674957 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fbbb006aba0 0x7fbbb00cddd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:21:03.264 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:21:03.257+0000 7fbbb6572640 1 --2- 172.21.15.143:0/227674957 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fbbb01575c0 0x7fbbb006bac0 unknown :-1 s=CLOSED pgs=702 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:21:03.264 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:21:03.257+0000 7fbbb6572640 1 -- 172.21.15.143:0/227674957 >> 172.21.15.143:0/227674957 conn(0x7fbbb00cf710 msgr2=0x7fbbb00cfb20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:21:03.264 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:21:03.257+0000 7fbbb6572640 1 -- 172.21.15.143:0/227674957 shutdown_connections 2023-11-26T07:21:03.264 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:21:03.257+0000 7fbbb6572640 1 -- 172.21.15.143:0/227674957 wait complete. 2023-11-26T07:21:03.265 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:21:03.261+0000 7fbbb6572640 1 Processor -- start 2023-11-26T07:21:03.265 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:21:03.261+0000 7fbbb6572640 1 -- start start 2023-11-26T07:21:03.266 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:21:03.261+0000 7fbbb6572640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fbbb006aba0 0x7fbbb01f1830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:21:03.267 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:21:03.261+0000 7fbbb6572640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fbbb01575c0 0x7fbbb01f5820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:21:03.267 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:21:03.261+0000 7fbbb6572640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fbbb0159e10 0x7fbbb014e990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:21:03.267 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:21:03.261+0000 7fbbb6572640 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbbb01f5d60 con 0x7fbbb0159e10 2023-11-26T07:21:03.267 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:21:03.261+0000 7fbbb6572640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbbb01f1e50 con 0x7fbbb006aba0 2023-11-26T07:21:03.267 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:21:03.261+0000 7fbbb6572640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbbb01f2130 con 0x7fbbb01575c0 2023-11-26T07:21:03.267 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:21:03.261+0000 7fbbb4b0f640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fbbb0159e10 0x7fbbb014e990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:21:03.267 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:21:03.261+0000 7fbbb4b0f640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fbbb0159e10 0x7fbbb014e990 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:49550/0 (socket says 172.21.15.143:49550) 2023-11-26T07:21:03.267 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:21:03.261+0000 7fbbb4b0f640 1 -- 172.21.15.143:0/3090824760 learned_addr learned my addr 172.21.15.143:0/3090824760 (peer_addr_for_me v2:172.21.15.143:0/0) 2023-11-26T07:21:03.267 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:21:03.261+0000 7fbbaf7fe640 1 --2- 172.21.15.143:0/3090824760 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fbbb01575c0 0x7fbbb01f5820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:21:03.267 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:21:03.261+0000 7fbbaffff640 1 --2- 172.21.15.143:0/3090824760 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fbbb006aba0 0x7fbbb01f1830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:21:03.268 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:21:03.261+0000 7fbbb4b0f640 1 -- 172.21.15.143:0/3090824760 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fbbb01575c0 msgr2=0x7fbbb01f5820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:21:03.268 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:21:03.261+0000 7fbbb4b0f640 1 --2- 172.21.15.143:0/3090824760 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fbbb01575c0 0x7fbbb01f5820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:21:03.269 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:21:03.261+0000 7fbbb4b0f640 1 -- 172.21.15.143:0/3090824760 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fbbb006aba0 msgr2=0x7fbbb01f1830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:21:03.269 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:21:03.261+0000 7fbbb4b0f640 1 --2- 172.21.15.143:0/3090824760 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fbbb006aba0 0x7fbbb01f1830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:21:03.269 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:21:03.261+0000 7fbbb4b0f640 1 -- 172.21.15.143:0/3090824760 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbba0009420 con 0x7fbbb0159e10 2023-11-26T07:21:03.269 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:21:03.261+0000 7fbbaf7fe640 1 --2- 172.21.15.143:0/3090824760 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fbbb01575c0 0x7fbbb01f5820 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-26T07:21:03.269 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:21:03.261+0000 7fbbaffff640 1 --2- 172.21.15.143:0/3090824760 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fbbb006aba0 0x7fbbb01f1830 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-26T07:21:03.269 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:21:03.261+0000 7fbbb4b0f640 1 --2- 172.21.15.143:0/3090824760 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fbbb0159e10 0x7fbbb014e990 secure :-1 s=READY pgs=703 cs=0 l=1 rev1=1 crypto rx=0x7fbba8007e30 tx=0x7fbba800e4b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:21:03.269 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:21:03.261+0000 7fbbacff9640 1 -- 172.21.15.143:0/3090824760 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbba8014620 con 0x7fbbb0159e10 2023-11-26T07:21:03.269 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:21:03.261+0000 7fbbacff9640 1 -- 172.21.15.143:0/3090824760 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbba800ee90 con 0x7fbbb0159e10 2023-11-26T07:21:03.269 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:21:03.261+0000 7fbbacff9640 1 -- 172.21.15.143:0/3090824760 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbba80149c0 con 0x7fbbb0159e10 2023-11-26T07:21:03.269 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:21:03.261+0000 7fbbb6572640 1 -- 172.21.15.143:0/3090824760 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbbb01f2410 con 0x7fbbb0159e10 2023-11-26T07:21:03.269 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:21:03.261+0000 7fbbb6572640 1 -- 172.21.15.143:0/3090824760 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbbb0153350 con 0x7fbbb0159e10 2023-11-26T07:21:03.269 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:21:03.265+0000 7fbbb6572640 1 -- 172.21.15.143:0/3090824760 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbbb0067680 con 0x7fbbb0159e10 2023-11-26T07:21:03.270 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:21:03.265+0000 7fbbacff9640 1 -- 172.21.15.143:0/3090824760 <== mon.0 v2:172.21.15.143:3300/0 4 ==== mgrmap(e 8) v1 ==== 86056+0+0 (secure 0 0 0) 0x7fbba8014b20 con 0x7fbbb0159e10 2023-11-26T07:21:03.270 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:21:03.265+0000 7fbbacff9640 1 --2- 172.21.15.143:0/3090824760 >> [v2:172.21.15.143:6832/2150557544,v1:172.21.15.143:6833/2150557544] conn(0x7fbb9c0666b0 0x7fbb9c068b70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:21:03.270 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:21:03.265+0000 7fbbacff9640 1 -- 172.21.15.143:0/3090824760 <== mon.0 v2:172.21.15.143:3300/0 5 ==== osd_map(47..47 src has 1..47) v4 ==== 5767+0+0 (secure 0 0 0) 0x7fbba8084bf0 con 0x7fbbb0159e10 2023-11-26T07:21:03.271 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:21:03.265+0000 7fbbaffff640 1 --2- 172.21.15.143:0/3090824760 >> [v2:172.21.15.143:6832/2150557544,v1:172.21.15.143:6833/2150557544] conn(0x7fbb9c0666b0 0x7fbb9c068b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:21:03.271 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:21:03.265+0000 7fbbaffff640 1 --2- 172.21.15.143:0/3090824760 >> [v2:172.21.15.143:6832/2150557544,v1:172.21.15.143:6833/2150557544] conn(0x7fbb9c0666b0 0x7fbb9c068b70 secure :-1 s=READY pgs=337 cs=0 l=1 rev1=1 crypto rx=0x7fbba00af000 tx=0x7fbba0008040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:21:03.275 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:21:03.269+0000 7fbbacff9640 1 -- 172.21.15.143:0/3090824760 <== mon.0 v2:172.21.15.143:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172649 (secure 0 0 0) 0x7fbba8051d90 con 0x7fbbb0159e10 2023-11-26T07:21:03.454 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:21:03.449+0000 7fbbb6572640 1 -- 172.21.15.143:0/3090824760 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fbbb0159060 con 0x7fbbb0159e10 2023-11-26T07:21:03.455 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:21:03.449+0000 7fbbacff9640 1 -- 172.21.15.143:0/3090824760 <== mon.0 v2:172.21.15.143:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 26 v26) v1 ==== 94+0+4922 (secure 0 0 0) 0x7fbba80553b0 con 0x7fbbb0159e10 2023-11-26T07:21:03.455 INFO:teuthology.orchestra.run.smithi143.stdout: 2023-11-26T07:21:03.456 INFO:teuthology.orchestra.run.smithi143.stdout:{"epoch":26,"default_fscid":3,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5201,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.188:6835/319464848","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6833","nonce":319464848},{"type":"v1","addr":"172.21.15.188:6835","nonce":319464848}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":20},{"gid":5584,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.188:6834/2113408070","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6832","nonce":2113408070},{"type":"v1","addr":"172.21.15.188:6834","nonce":2113408070}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":20},{"gid":5745,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.143:6835/1616742593","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6834","nonce":1616742593},{"type":"v1","addr":"172.21.15.143:6835","nonce":1616742593}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":20}],"filesystems":[{"mdsmap":{"epoch":25,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-11-26T07:19:31.503587+0000","modified":"2023-11-26T07:19:34.523605+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":5748},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5748":{"gid":5748,"name":"c","rank":0,"incarnation":21,"state":"up:active","state_seq":3,"addr":"172.21.15.143:6837/2872033338","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6836","nonce":2872033338},{"type":"v1","addr":"172.21.15.143:6837","nonce":2872033338}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[7],"metadata_pool":6,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":3}]} 2023-11-26T07:21:03.456 INFO:teuthology.orchestra.run.smithi143.stderr:dumped fsmap epoch 26 2023-11-26T07:21:03.457 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:21:03.453+0000 7fbbb6572640 1 -- 172.21.15.143:0/3090824760 >> [v2:172.21.15.143:6832/2150557544,v1:172.21.15.143:6833/2150557544] conn(0x7fbb9c0666b0 msgr2=0x7fbb9c068b70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:21:03.457 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:21:03.453+0000 7fbbb6572640 1 --2- 172.21.15.143:0/3090824760 >> [v2:172.21.15.143:6832/2150557544,v1:172.21.15.143:6833/2150557544] conn(0x7fbb9c0666b0 0x7fbb9c068b70 secure :-1 s=READY pgs=337 cs=0 l=1 rev1=1 crypto rx=0x7fbba00af000 tx=0x7fbba0008040 comp rx=0 tx=0).stop 2023-11-26T07:21:03.458 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:21:03.453+0000 7fbbb6572640 1 -- 172.21.15.143:0/3090824760 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fbbb0159e10 msgr2=0x7fbbb014e990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:21:03.458 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:21:03.453+0000 7fbbb6572640 1 --2- 172.21.15.143:0/3090824760 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fbbb0159e10 0x7fbbb014e990 secure :-1 s=READY pgs=703 cs=0 l=1 rev1=1 crypto rx=0x7fbba8007e30 tx=0x7fbba800e4b0 comp rx=0 tx=0).stop 2023-11-26T07:21:03.458 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:21:03.453+0000 7fbbb6572640 1 -- 172.21.15.143:0/3090824760 shutdown_connections 2023-11-26T07:21:03.458 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:21:03.453+0000 7fbbb6572640 1 --2- 172.21.15.143:0/3090824760 >> [v2:172.21.15.143:6832/2150557544,v1:172.21.15.143:6833/2150557544] conn(0x7fbb9c0666b0 0x7fbb9c068b70 unknown :-1 s=CLOSED pgs=337 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:21:03.458 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:21:03.453+0000 7fbbb6572640 1 --2- 172.21.15.143:0/3090824760 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fbbb0159e10 0x7fbbb014e990 unknown :-1 s=CLOSED pgs=703 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:21:03.458 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:21:03.453+0000 7fbbb6572640 1 --2- 172.21.15.143:0/3090824760 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fbbb01575c0 0x7fbbb01f5820 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:21:03.458 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:21:03.453+0000 7fbbb6572640 1 --2- 172.21.15.143:0/3090824760 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fbbb006aba0 0x7fbbb01f1830 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:21:03.458 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:21:03.453+0000 7fbbb6572640 1 -- 172.21.15.143:0/3090824760 >> 172.21.15.143:0/3090824760 conn(0x7fbbb00cf710 msgr2=0x7fbbb0157a30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:21:03.458 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:21:03.453+0000 7fbbb6572640 1 -- 172.21.15.143:0/3090824760 shutdown_connections 2023-11-26T07:21:03.458 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:21:03.453+0000 7fbbb6572640 1 -- 172.21.15.143:0/3090824760 wait complete. 2023-11-26T07:21:03.477 DEBUG:teuthology.orchestra.run.smithi143:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.c.asok --format=json session ls 2023-11-26T07:21:03.634 INFO:teuthology.orchestra.run.smithi143.stdout:[{"id":6126,"entity":{"name":{"type":"client","num":6126},"addr":{"type":"any","addr":"172.21.15.143:0","nonce":3879234400}},"state":"open","num_leases":0,"num_caps":7,"request_load_avg":7,"uptime":56.406480096999999,"requests_in_flight":0,"num_completed_requests":2,"num_completed_flushes":1,"reconnecting":false,"recall_caps":{"value":0,"halflife":60},"release_caps":{"value":0,"halflife":60},"recall_caps_throttle":{"value":0,"halflife":1.5},"recall_caps_throttle2o":{"value":0,"halflife":0.5},"session_cache_liveness":{"value":19.31239883923239,"halflife":300},"cap_acquisition":{"value":0,"halflife":30},"last_trim_completed_requests_tid":14,"last_trim_completed_flushes_tid":2,"delegated_inos":[{"start":"0x10000000005","length":500}],"inst":"client.6126 172.21.15.143:0/3879234400","completed_requests":[{"tid":14,"created_ino":"0x0"},{"tid":15,"created_ino":"0x100000001f9"}],"prealloc_inos":[{"start":"0x10000000005","length":500},{"start":"0x100000001fa","length":1000}],"client_metadata":{"client_features":{"feature_bits":"0x00000000003fffff"},"metric_spec":{"metric_flags":{"feature_bits":"0x000000000000ffff"}},"ceph_sha1":"44c2237fe77713214148c1fd622c872f346f1b99","ceph_version":"ceph version 18.0.0-7484-g44c2237f (44c2237fe77713214148c1fd622c872f346f1b99) reef (dev)","entity_id":"y","hostname":"smithi143","pid":"38097","root":"/"}}] 2023-11-26T07:21:03.635 DEBUG:tasks.cephfs.filesystem:_json_asok output [ { "cap_acquisition": { "halflife": 30, "value": 0 }, "client_metadata": { "ceph_sha1": "44c2237fe77713214148c1fd622c872f346f1b99", "ceph_version": "ceph version 18.0.0-7484-g44c2237f (44c2237fe77713214148c1fd622c872f346f1b99) reef (dev)", "client_features": { "feature_bits": "0x00000000003fffff" }, "entity_id": "y", "hostname": "smithi143", "metric_spec": { "metric_flags": { "feature_bits": "0x000000000000ffff" } }, "pid": "38097", "root": "/" }, "completed_requests": [ { "created_ino": "0x0", "tid": 14 }, { "created_ino": "0x100000001f9", "tid": 15 } ], "delegated_inos": [ { "length": 500, "start": "0x10000000005" } ], "entity": { "addr": { "addr": "172.21.15.143:0", "nonce": 3879234400, "type": "any" }, "name": { "num": 6126, "type": "client" } }, "id": 6126, "inst": "client.6126 172.21.15.143:0/3879234400", "last_trim_completed_flushes_tid": 2, "last_trim_completed_requests_tid": 14, "num_caps": 7, "num_completed_flushes": 1, "num_completed_requests": 2, "num_leases": 0, "prealloc_inos": [ { "length": 500, "start": "0x10000000005" }, { "length": 1000, "start": "0x100000001fa" } ], "recall_caps": { "halflife": 60, "value": 0 }, "recall_caps_throttle": { "halflife": 1.5, "value": 0 }, "recall_caps_throttle2o": { "halflife": 0.5, "value": 0 }, "reconnecting": false, "release_caps": { "halflife": 60, "value": 0 }, "request_load_avg": 7, "requests_in_flight": 0, "session_cache_liveness": { "halflife": 300, "value": 19.31239883923239 }, "state": "open", "uptime": 56.406480097 } ] 2023-11-26T07:21:03.635 DEBUG:tasks.ceph_test_case:wait_until_true: waiting (timeout=90 retry_count=0)... 2023-11-26T07:21:08.636 DEBUG:teuthology.orchestra.run.smithi143:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-11-26T07:21:08.784 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:21:08.777+0000 7f5f31de7640 1 Processor -- start 2023-11-26T07:21:08.784 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:21:08.777+0000 7f5f31de7640 1 -- start start 2023-11-26T07:21:08.785 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:21:08.781+0000 7f5f31de7640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f5f2c0b2f80 0x7f5f2c071850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:21:08.785 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:21:08.781+0000 7f5f31de7640 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7f5f2c0ce680 0x7f5f2c06cbe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:21:08.785 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:21:08.781+0000 7f5f31de7640 1 -- --> v1:172.21.15.143:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5f2c06c750 con 0x7f5f2c1596d0 2023-11-26T07:21:08.785 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:21:08.781+0000 7f5f31de7640 1 -- --> v2:172.21.15.143:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5f2c0cbe00 con 0x7f5f2c0ce680 2023-11-26T07:21:08.786 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:21:08.781+0000 7f5f31de7640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5f2c0b3550 con 0x7f5f2c0b2f80 2023-11-26T07:21:08.786 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:21:08.781+0000 7f5f2b7fe640 1 --1- >> v1:172.21.15.143:6789/0 conn(0x7f5f2c1596d0 0x7f5f2c0b2870 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.143:6789/0 says I am v1:172.21.15.143:50908/0 (socket says 172.21.15.143:50908) 2023-11-26T07:21:08.786 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:21:08.781+0000 7f5f2b7fe640 1 -- 172.21.15.143:0/3916559718 learned_addr learned my addr 172.21.15.143:0/3916559718 (peer_addr_for_me v1:172.21.15.143:0/0) 2023-11-26T07:21:08.786 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:21:08.781+0000 7f5f2bfff640 1 --2- 172.21.15.143:0/3916559718 >> v2:172.21.15.143:3300/0 conn(0x7f5f2c0ce680 0x7f5f2c06cbe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:21:08.787 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:21:08.781+0000 7f5f2affd640 1 --2- 172.21.15.143:0/3916559718 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f5f2c0b2f80 0x7f5f2c071850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:21:08.787 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:21:08.781+0000 7f5f2bfff640 1 -- 172.21.15.143:0/3916559718 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f5f2c0b2f80 msgr2=0x7f5f2c071850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-26T07:21:08.787 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:21:08.781+0000 7f5f2bfff640 1 --2- 172.21.15.143:0/3916559718 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f5f2c0b2f80 0x7f5f2c071850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:21:08.787 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:21:08.781+0000 7f5f2bfff640 1 -- 172.21.15.143:0/3916559718 >> v1:172.21.15.143:6789/0 conn(0x7f5f2c1596d0 legacy=0x7f5f2c0b2870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:21:08.787 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:21:08.781+0000 7f5f2bfff640 1 -- 172.21.15.143:0/3916559718 --> v2:172.21.15.143:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5f2c071e80 con 0x7f5f2c0ce680 2023-11-26T07:21:08.787 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:21:08.781+0000 7f5f2a7fc640 1 -- 172.21.15.143:0/3916559718 <== mon.0 v1:172.21.15.143:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1499138849 0 0) 0x7f5f2c06c750 con 0x7f5f2c1596d0 2023-11-26T07:21:08.789 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:21:08.785+0000 7f5f2bfff640 1 --2- 172.21.15.143:0/3916559718 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f5f2c0ce680 0x7f5f2c06cbe0 secure :-1 s=READY pgs=705 cs=0 l=1 rev1=1 crypto rx=0x7f5f24009d90 tx=0x7f5f2401c950 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2c678545cf3d46 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:21:08.789 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:21:08.785+0000 7f5f2a7fc640 1 -- 172.21.15.143:0/3916559718 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5f240a5880 con 0x7f5f2c0ce680 2023-11-26T07:21:08.789 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:21:08.785+0000 7f5f2a7fc640 1 -- 172.21.15.143:0/3916559718 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5f240a59e0 con 0x7f5f2c0ce680 2023-11-26T07:21:08.789 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:21:08.785+0000 7f5f2a7fc640 1 -- 172.21.15.143:0/3916559718 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5f240a5d70 con 0x7f5f2c0ce680 2023-11-26T07:21:08.790 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:21:08.785+0000 7f5f31de7640 1 -- 172.21.15.143:0/3916559718 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f5f2c0ce680 msgr2=0x7f5f2c06cbe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:21:08.790 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:21:08.785+0000 7f5f31de7640 1 --2- 172.21.15.143:0/3916559718 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f5f2c0ce680 0x7f5f2c06cbe0 secure :-1 s=READY pgs=705 cs=0 l=1 rev1=1 crypto rx=0x7f5f24009d90 tx=0x7f5f2401c950 comp rx=0 tx=0).stop 2023-11-26T07:21:08.790 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:21:08.785+0000 7f5f31de7640 1 -- 172.21.15.143:0/3916559718 shutdown_connections 2023-11-26T07:21:08.790 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:21:08.785+0000 7f5f31de7640 1 --2- 172.21.15.143:0/3916559718 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f5f2c0ce680 0x7f5f2c06cbe0 unknown :-1 s=CLOSED pgs=705 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:21:08.791 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:21:08.785+0000 7f5f31de7640 1 --2- 172.21.15.143:0/3916559718 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f5f2c0b2f80 0x7f5f2c071850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:21:08.791 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:21:08.785+0000 7f5f31de7640 1 -- 172.21.15.143:0/3916559718 >> 172.21.15.143:0/3916559718 conn(0x7f5f2c06a890 msgr2=0x7f5f2c06aca0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:21:08.791 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:21:08.785+0000 7f5f31de7640 1 -- 172.21.15.143:0/3916559718 shutdown_connections 2023-11-26T07:21:08.791 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:21:08.785+0000 7f5f31de7640 1 -- 172.21.15.143:0/3916559718 wait complete. 2023-11-26T07:21:08.791 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:21:08.785+0000 7f5f31de7640 1 Processor -- start 2023-11-26T07:21:08.792 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:21:08.785+0000 7f5f31de7640 1 -- start start 2023-11-26T07:21:08.794 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:21:08.789+0000 7f5f31de7640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f5f2c0b2f80 0x7f5f2c1516a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:21:08.794 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:21:08.789+0000 7f5f31de7640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f5f2c0ce680 0x7f5f2c14c950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:21:08.794 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:21:08.789+0000 7f5f31de7640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f5f2c1596d0 0x7f5f2c153d20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:21:08.794 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:21:08.789+0000 7f5f31de7640 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5f2c153820 con 0x7f5f2c0ce680 2023-11-26T07:21:08.794 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:21:08.789+0000 7f5f31de7640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5f2c151be0 con 0x7f5f2c0b2f80 2023-11-26T07:21:08.794 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:21:08.789+0000 7f5f31de7640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5f2c158e60 con 0x7f5f2c1596d0 2023-11-26T07:21:08.794 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:21:08.789+0000 7f5f2affd640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f5f2c0ce680 0x7f5f2c14c950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:21:08.795 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:21:08.789+0000 7f5f2affd640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f5f2c0ce680 0x7f5f2c14c950 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:39360/0 (socket says 172.21.15.143:39360) 2023-11-26T07:21:08.795 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:21:08.789+0000 7f5f2affd640 1 -- 172.21.15.143:0/2896230694 learned_addr learned my addr 172.21.15.143:0/2896230694 (peer_addr_for_me v2:172.21.15.143:0/0) 2023-11-26T07:21:08.795 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:21:08.789+0000 7f5f2b7fe640 1 --2- 172.21.15.143:0/2896230694 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f5f2c0b2f80 0x7f5f2c1516a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:21:08.795 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:21:08.789+0000 7f5f2bfff640 1 --2- 172.21.15.143:0/2896230694 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f5f2c1596d0 0x7f5f2c153d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:21:08.795 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:21:08.789+0000 7f5f2affd640 1 -- 172.21.15.143:0/2896230694 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f5f2c1596d0 msgr2=0x7f5f2c153d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:21:08.795 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:21:08.789+0000 7f5f2affd640 1 --2- 172.21.15.143:0/2896230694 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f5f2c1596d0 0x7f5f2c153d20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:21:08.795 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:21:08.789+0000 7f5f2affd640 1 -- 172.21.15.143:0/2896230694 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f5f2c0b2f80 msgr2=0x7f5f2c1516a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:21:08.795 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:21:08.789+0000 7f5f2affd640 1 --2- 172.21.15.143:0/2896230694 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f5f2c0b2f80 0x7f5f2c1516a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:21:08.795 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:21:08.789+0000 7f5f2affd640 1 -- 172.21.15.143:0/2896230694 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5f24009a70 con 0x7f5f2c0ce680 2023-11-26T07:21:08.795 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:21:08.789+0000 7f5f2b7fe640 1 --2- 172.21.15.143:0/2896230694 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f5f2c0b2f80 0x7f5f2c1516a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-26T07:21:08.795 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:21:08.789+0000 7f5f2affd640 1 --2- 172.21.15.143:0/2896230694 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f5f2c0ce680 0x7f5f2c14c950 secure :-1 s=READY pgs=706 cs=0 l=1 rev1=1 crypto rx=0x7f5f2000d800 tx=0x7f5f2000dcc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:21:08.795 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:21:08.789+0000 7f5f13fff640 1 -- 172.21.15.143:0/2896230694 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5f2000b6c0 con 0x7f5f2c0ce680 2023-11-26T07:21:08.795 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:21:08.789+0000 7f5f13fff640 1 -- 172.21.15.143:0/2896230694 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5f2000b820 con 0x7f5f2c0ce680 2023-11-26T07:21:08.796 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:21:08.789+0000 7f5f13fff640 1 -- 172.21.15.143:0/2896230694 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5f2000bb90 con 0x7f5f2c0ce680 2023-11-26T07:21:08.796 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:21:08.789+0000 7f5f31de7640 1 -- 172.21.15.143:0/2896230694 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5f2c1f9ac0 con 0x7f5f2c0ce680 2023-11-26T07:21:08.796 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:21:08.789+0000 7f5f31de7640 1 -- 172.21.15.143:0/2896230694 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5f2c1fa300 con 0x7f5f2c0ce680 2023-11-26T07:21:08.796 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:21:08.789+0000 7f5f31de7640 1 -- 172.21.15.143:0/2896230694 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5f2c067680 con 0x7f5f2c0ce680 2023-11-26T07:21:08.796 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:21:08.789+0000 7f5f13fff640 1 -- 172.21.15.143:0/2896230694 <== mon.0 v2:172.21.15.143:3300/0 4 ==== mgrmap(e 8) v1 ==== 86056+0+0 (secure 0 0 0) 0x7f5f2000bcf0 con 0x7f5f2c0ce680 2023-11-26T07:21:08.797 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:21:08.793+0000 7f5f13fff640 1 --2- 172.21.15.143:0/2896230694 >> [v2:172.21.15.143:6832/2150557544,v1:172.21.15.143:6833/2150557544] conn(0x7f5f180666b0 0x7f5f18068b70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:21:08.797 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:21:08.793+0000 7f5f13fff640 1 -- 172.21.15.143:0/2896230694 <== mon.0 v2:172.21.15.143:3300/0 5 ==== osd_map(47..47 src has 1..47) v4 ==== 5767+0+0 (secure 0 0 0) 0x7f5f20084890 con 0x7f5f2c0ce680 2023-11-26T07:21:08.797 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:21:08.793+0000 7f5f2b7fe640 1 --2- 172.21.15.143:0/2896230694 >> [v2:172.21.15.143:6832/2150557544,v1:172.21.15.143:6833/2150557544] conn(0x7f5f180666b0 0x7f5f18068b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:21:08.798 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:21:08.793+0000 7f5f2b7fe640 1 --2- 172.21.15.143:0/2896230694 >> [v2:172.21.15.143:6832/2150557544,v1:172.21.15.143:6833/2150557544] conn(0x7f5f180666b0 0x7f5f18068b70 secure :-1 s=READY pgs=338 cs=0 l=1 rev1=1 crypto rx=0x7f5f1c000f70 tx=0x7f5f1c007cb0 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:21:08.801 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:21:08.797+0000 7f5f13fff640 1 -- 172.21.15.143:0/2896230694 <== mon.0 v2:172.21.15.143:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172649 (secure 0 0 0) 0x7f5f20051960 con 0x7f5f2c0ce680 2023-11-26T07:21:08.982 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:21:08.977+0000 7f5f31de7640 1 -- 172.21.15.143:0/2896230694 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f5f2c070a20 con 0x7f5f2c0ce680 2023-11-26T07:21:08.982 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:21:08.977+0000 7f5f13fff640 1 -- 172.21.15.143:0/2896230694 <== mon.0 v2:172.21.15.143:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 26 v26) v1 ==== 94+0+4922 (secure 0 0 0) 0x7f5f20011340 con 0x7f5f2c0ce680 2023-11-26T07:21:08.982 INFO:teuthology.orchestra.run.smithi143.stdout: 2023-11-26T07:21:08.983 INFO:teuthology.orchestra.run.smithi143.stdout:{"epoch":26,"default_fscid":3,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5201,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.188:6835/319464848","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6833","nonce":319464848},{"type":"v1","addr":"172.21.15.188:6835","nonce":319464848}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":20},{"gid":5584,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.188:6834/2113408070","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6832","nonce":2113408070},{"type":"v1","addr":"172.21.15.188:6834","nonce":2113408070}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":20},{"gid":5745,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.143:6835/1616742593","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6834","nonce":1616742593},{"type":"v1","addr":"172.21.15.143:6835","nonce":1616742593}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":20}],"filesystems":[{"mdsmap":{"epoch":25,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-11-26T07:19:31.503587+0000","modified":"2023-11-26T07:19:34.523605+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":5748},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5748":{"gid":5748,"name":"c","rank":0,"incarnation":21,"state":"up:active","state_seq":3,"addr":"172.21.15.143:6837/2872033338","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6836","nonce":2872033338},{"type":"v1","addr":"172.21.15.143:6837","nonce":2872033338}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[7],"metadata_pool":6,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":3}]} 2023-11-26T07:21:08.983 INFO:teuthology.orchestra.run.smithi143.stderr:dumped fsmap epoch 26 2023-11-26T07:21:08.984 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:21:08.977+0000 7f5f31de7640 1 -- 172.21.15.143:0/2896230694 >> [v2:172.21.15.143:6832/2150557544,v1:172.21.15.143:6833/2150557544] conn(0x7f5f180666b0 msgr2=0x7f5f18068b70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:21:08.984 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:21:08.977+0000 7f5f31de7640 1 --2- 172.21.15.143:0/2896230694 >> [v2:172.21.15.143:6832/2150557544,v1:172.21.15.143:6833/2150557544] conn(0x7f5f180666b0 0x7f5f18068b70 secure :-1 s=READY pgs=338 cs=0 l=1 rev1=1 crypto rx=0x7f5f1c000f70 tx=0x7f5f1c007cb0 comp rx=0 tx=0).stop 2023-11-26T07:21:08.984 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:21:08.977+0000 7f5f31de7640 1 -- 172.21.15.143:0/2896230694 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f5f2c0ce680 msgr2=0x7f5f2c14c950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:21:08.985 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:21:08.977+0000 7f5f31de7640 1 --2- 172.21.15.143:0/2896230694 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f5f2c0ce680 0x7f5f2c14c950 secure :-1 s=READY pgs=706 cs=0 l=1 rev1=1 crypto rx=0x7f5f2000d800 tx=0x7f5f2000dcc0 comp rx=0 tx=0).stop 2023-11-26T07:21:08.985 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:21:08.977+0000 7f5f31de7640 1 -- 172.21.15.143:0/2896230694 shutdown_connections 2023-11-26T07:21:08.985 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:21:08.977+0000 7f5f31de7640 1 --2- 172.21.15.143:0/2896230694 >> [v2:172.21.15.143:6832/2150557544,v1:172.21.15.143:6833/2150557544] conn(0x7f5f180666b0 0x7f5f18068b70 unknown :-1 s=CLOSED pgs=338 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:21:08.985 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:21:08.981+0000 7f5f31de7640 1 --2- 172.21.15.143:0/2896230694 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f5f2c1596d0 0x7f5f2c153d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:21:08.985 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:21:08.981+0000 7f5f31de7640 1 --2- 172.21.15.143:0/2896230694 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f5f2c0ce680 0x7f5f2c14c950 unknown :-1 s=CLOSED pgs=706 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:21:08.985 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:21:08.981+0000 7f5f31de7640 1 --2- 172.21.15.143:0/2896230694 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f5f2c0b2f80 0x7f5f2c1516a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:21:08.985 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:21:08.981+0000 7f5f31de7640 1 -- 172.21.15.143:0/2896230694 >> 172.21.15.143:0/2896230694 conn(0x7f5f2c06a890 msgr2=0x7f5f2c0aeb40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:21:08.985 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:21:08.981+0000 7f5f31de7640 1 -- 172.21.15.143:0/2896230694 shutdown_connections 2023-11-26T07:21:08.985 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:21:08.981+0000 7f5f31de7640 1 -- 172.21.15.143:0/2896230694 wait complete. 2023-11-26T07:21:09.004 DEBUG:teuthology.orchestra.run.smithi143:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.c.asok --format=json session ls 2023-11-26T07:21:09.161 INFO:teuthology.orchestra.run.smithi143.stdout:[{"id":6126,"entity":{"name":{"type":"client","num":6126},"addr":{"type":"any","addr":"172.21.15.143:0","nonce":3879234400}},"state":"open","num_leases":0,"num_caps":7,"request_load_avg":7,"uptime":61.934397691000001,"requests_in_flight":0,"num_completed_requests":2,"num_completed_flushes":1,"reconnecting":false,"recall_caps":{"value":0,"halflife":60},"release_caps":{"value":0,"halflife":60},"recall_caps_throttle":{"value":0,"halflife":1.5},"recall_caps_throttle2o":{"value":0,"halflife":0.5},"session_cache_liveness":{"value":19.067305509463239,"halflife":300},"cap_acquisition":{"value":0,"halflife":30},"last_trim_completed_requests_tid":14,"last_trim_completed_flushes_tid":2,"delegated_inos":[{"start":"0x10000000005","length":500}],"inst":"client.6126 172.21.15.143:0/3879234400","completed_requests":[{"tid":14,"created_ino":"0x0"},{"tid":15,"created_ino":"0x100000001f9"}],"prealloc_inos":[{"start":"0x10000000005","length":500},{"start":"0x100000001fa","length":1000}],"client_metadata":{"client_features":{"feature_bits":"0x00000000003fffff"},"metric_spec":{"metric_flags":{"feature_bits":"0x000000000000ffff"}},"ceph_sha1":"44c2237fe77713214148c1fd622c872f346f1b99","ceph_version":"ceph version 18.0.0-7484-g44c2237f (44c2237fe77713214148c1fd622c872f346f1b99) reef (dev)","entity_id":"y","hostname":"smithi143","pid":"38097","root":"/"}}] 2023-11-26T07:21:09.162 DEBUG:tasks.cephfs.filesystem:_json_asok output [ { "cap_acquisition": { "halflife": 30, "value": 0 }, "client_metadata": { "ceph_sha1": "44c2237fe77713214148c1fd622c872f346f1b99", "ceph_version": "ceph version 18.0.0-7484-g44c2237f (44c2237fe77713214148c1fd622c872f346f1b99) reef (dev)", "client_features": { "feature_bits": "0x00000000003fffff" }, "entity_id": "y", "hostname": "smithi143", "metric_spec": { "metric_flags": { "feature_bits": "0x000000000000ffff" } }, "pid": "38097", "root": "/" }, "completed_requests": [ { "created_ino": "0x0", "tid": 14 }, { "created_ino": "0x100000001f9", "tid": 15 } ], "delegated_inos": [ { "length": 500, "start": "0x10000000005" } ], "entity": { "addr": { "addr": "172.21.15.143:0", "nonce": 3879234400, "type": "any" }, "name": { "num": 6126, "type": "client" } }, "id": 6126, "inst": "client.6126 172.21.15.143:0/3879234400", "last_trim_completed_flushes_tid": 2, "last_trim_completed_requests_tid": 14, "num_caps": 7, "num_completed_flushes": 1, "num_completed_requests": 2, "num_leases": 0, "prealloc_inos": [ { "length": 500, "start": "0x10000000005" }, { "length": 1000, "start": "0x100000001fa" } ], "recall_caps": { "halflife": 60, "value": 0 }, "recall_caps_throttle": { "halflife": 1.5, "value": 0 }, "recall_caps_throttle2o": { "halflife": 0.5, "value": 0 }, "reconnecting": false, "release_caps": { "halflife": 60, "value": 0 }, "request_load_avg": 7, "requests_in_flight": 0, "session_cache_liveness": { "halflife": 300, "value": 19.06730550946324 }, "state": "open", "uptime": 61.934397691 } ] 2023-11-26T07:21:09.162 DEBUG:tasks.ceph_test_case:wait_until_true: waiting (timeout=90 retry_count=0)... 2023-11-26T07:21:14.163 DEBUG:teuthology.orchestra.run.smithi143:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-11-26T07:21:14.310 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:21:14.305+0000 7faa8326b640 1 Processor -- start 2023-11-26T07:21:14.310 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:21:14.305+0000 7faa8326b640 1 -- start start 2023-11-26T07:21:14.312 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:21:14.305+0000 7faa8326b640 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7faa7c163520 0x7faa7c16c3a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:21:14.312 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:21:14.305+0000 7faa8326b640 1 -- --> v1:172.21.15.143:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7faa7c162080 con 0x7faa7c0cf990 2023-11-26T07:21:14.312 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:21:14.305+0000 7faa8326b640 1 -- --> v1:172.21.15.188:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7faa7c161d70 con 0x7faa7c14b1e0 2023-11-26T07:21:14.312 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:21:14.305+0000 7faa8326b640 1 -- --> v2:172.21.15.143:3300/0 -- mon_getmap magic: 0 v1 -- 0x7faa7c14b5c0 con 0x7faa7c163520 2023-11-26T07:21:14.312 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:21:14.305+0000 7faa81808640 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7faa7c163520 0x7faa7c16c3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:21:14.313 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:21:14.305+0000 7faa80806640 1 --1- >> v1:172.21.15.143:6789/0 conn(0x7faa7c0cf990 0x7faa7c0708c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.143:6789/0 says I am v1:172.21.15.143:50918/0 (socket says 172.21.15.143:50918) 2023-11-26T07:21:14.313 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:21:14.305+0000 7faa80806640 1 -- 172.21.15.143:0/2030795128 learned_addr learned my addr 172.21.15.143:0/2030795128 (peer_addr_for_me v1:172.21.15.143:0/0) 2023-11-26T07:21:14.313 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:21:14.309+0000 7faa81808640 1 -- 172.21.15.143:0/2030795128 >> v1:172.21.15.188:6789/0 conn(0x7faa7c14b1e0 legacy=0x7faa7c0cf280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-26T07:21:14.313 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:21:14.309+0000 7faa81808640 1 -- 172.21.15.143:0/2030795128 >> v1:172.21.15.143:6789/0 conn(0x7faa7c0cf990 legacy=0x7faa7c0708c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:21:14.313 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:21:14.309+0000 7faa73fff640 1 -- 172.21.15.143:0/2030795128 <== mon.0 v1:172.21.15.143:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 970875813 0 0) 0x7faa7c162080 con 0x7faa7c0cf990 2023-11-26T07:21:14.314 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:21:14.309+0000 7faa81808640 1 -- 172.21.15.143:0/2030795128 --> v2:172.21.15.143:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7faa7c071580 con 0x7faa7c163520 2023-11-26T07:21:14.315 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:21:14.309+0000 7faa81808640 1 --2- 172.21.15.143:0/2030795128 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7faa7c163520 0x7faa7c16c3a0 secure :-1 s=READY pgs=708 cs=0 l=1 rev1=1 crypto rx=0x7faa74009d90 tx=0x7faa7401c8e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3e55def875ec6092 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:21:14.316 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:21:14.309+0000 7faa73fff640 1 -- 172.21.15.143:0/2030795128 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faa740a59a0 con 0x7faa7c163520 2023-11-26T07:21:14.316 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:21:14.309+0000 7faa73fff640 1 -- 172.21.15.143:0/2030795128 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7faa740a5b00 con 0x7faa7c163520 2023-11-26T07:21:14.316 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:21:14.309+0000 7faa73fff640 1 -- 172.21.15.143:0/2030795128 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faa740a5e90 con 0x7faa7c163520 2023-11-26T07:21:14.317 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:21:14.313+0000 7faa8326b640 1 -- 172.21.15.143:0/2030795128 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7faa7c163520 msgr2=0x7faa7c16c3a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:21:14.317 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:21:14.313+0000 7faa8326b640 1 --2- 172.21.15.143:0/2030795128 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7faa7c163520 0x7faa7c16c3a0 secure :-1 s=READY pgs=708 cs=0 l=1 rev1=1 crypto rx=0x7faa74009d90 tx=0x7faa7401c8e0 comp rx=0 tx=0).stop 2023-11-26T07:21:14.317 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:21:14.313+0000 7faa8326b640 1 -- 172.21.15.143:0/2030795128 shutdown_connections 2023-11-26T07:21:14.317 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:21:14.313+0000 7faa8326b640 1 --2- 172.21.15.143:0/2030795128 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7faa7c163520 0x7faa7c16c3a0 unknown :-1 s=CLOSED pgs=708 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:21:14.317 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:21:14.313+0000 7faa8326b640 1 -- 172.21.15.143:0/2030795128 >> 172.21.15.143:0/2030795128 conn(0x7faa7c06a890 msgr2=0x7faa7c06aca0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:21:14.317 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:21:14.313+0000 7faa8326b640 1 -- 172.21.15.143:0/2030795128 shutdown_connections 2023-11-26T07:21:14.317 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:21:14.313+0000 7faa8326b640 1 -- 172.21.15.143:0/2030795128 wait complete. 2023-11-26T07:21:14.318 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:21:14.313+0000 7faa8326b640 1 Processor -- start 2023-11-26T07:21:14.318 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:21:14.313+0000 7faa8326b640 1 -- start start 2023-11-26T07:21:14.319 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:21:14.313+0000 7faa8326b640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7faa7c0cf990 0x7faa7c160750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:21:14.319 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:21:14.313+0000 7faa8326b640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7faa7c14b1e0 0x7faa7c0a9940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:21:14.319 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:21:14.313+0000 7faa8326b640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7faa7c163520 0x7faa7c14a4d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:21:14.319 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:21:14.313+0000 7faa8326b640 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faa7c160c90 con 0x7faa7c0cf990 2023-11-26T07:21:14.319 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:21:14.313+0000 7faa8326b640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faa7c149460 con 0x7faa7c163520 2023-11-26T07:21:14.320 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:21:14.313+0000 7faa8326b640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7faa7c0a9e80 con 0x7faa7c14b1e0 2023-11-26T07:21:14.320 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:21:14.313+0000 7faa81007640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7faa7c0cf990 0x7faa7c160750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:21:14.320 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:21:14.313+0000 7faa81007640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7faa7c0cf990 0x7faa7c160750 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:39380/0 (socket says 172.21.15.143:39380) 2023-11-26T07:21:14.320 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:21:14.313+0000 7faa81007640 1 -- 172.21.15.143:0/3635443286 learned_addr learned my addr 172.21.15.143:0/3635443286 (peer_addr_for_me v2:172.21.15.143:0/0) 2023-11-26T07:21:14.321 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:21:14.313+0000 7faa81808640 1 --2- 172.21.15.143:0/3635443286 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7faa7c163520 0x7faa7c14a4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:21:14.321 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:21:14.313+0000 7faa80806640 1 --2- 172.21.15.143:0/3635443286 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7faa7c14b1e0 0x7faa7c0a9940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:21:14.321 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:21:14.313+0000 7faa81007640 1 -- 172.21.15.143:0/3635443286 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7faa7c14b1e0 msgr2=0x7faa7c0a9940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:21:14.321 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:21:14.313+0000 7faa81007640 1 --2- 172.21.15.143:0/3635443286 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7faa7c14b1e0 0x7faa7c0a9940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:21:14.321 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:21:14.313+0000 7faa81007640 1 -- 172.21.15.143:0/3635443286 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7faa7c163520 msgr2=0x7faa7c14a4d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:21:14.321 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:21:14.313+0000 7faa81007640 1 --2- 172.21.15.143:0/3635443286 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7faa7c163520 0x7faa7c14a4d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:21:14.321 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:21:14.313+0000 7faa81007640 1 -- 172.21.15.143:0/3635443286 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7faa74009a70 con 0x7faa7c0cf990 2023-11-26T07:21:14.321 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:21:14.313+0000 7faa80806640 1 --2- 172.21.15.143:0/3635443286 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7faa7c14b1e0 0x7faa7c0a9940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-26T07:21:14.321 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:21:14.313+0000 7faa81007640 1 --2- 172.21.15.143:0/3635443286 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7faa7c0cf990 0x7faa7c160750 secure :-1 s=READY pgs=709 cs=0 l=1 rev1=1 crypto rx=0x7faa7800c450 tx=0x7faa7800c910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:21:14.322 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:21:14.317+0000 7faa71ffb640 1 -- 172.21.15.143:0/3635443286 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faa78012830 con 0x7faa7c0cf990 2023-11-26T07:21:14.322 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:21:14.317+0000 7faa71ffb640 1 -- 172.21.15.143:0/3635443286 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7faa780129b0 con 0x7faa7c0cf990 2023-11-26T07:21:14.322 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:21:14.317+0000 7faa71ffb640 1 -- 172.21.15.143:0/3635443286 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faa78012d60 con 0x7faa7c0cf990 2023-11-26T07:21:14.322 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:21:14.317+0000 7faa8326b640 1 -- 172.21.15.143:0/3635443286 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7faa7c161580 con 0x7faa7c0cf990 2023-11-26T07:21:14.322 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:21:14.317+0000 7faa8326b640 1 -- 172.21.15.143:0/3635443286 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7faa7c1612a0 con 0x7faa7c0cf990 2023-11-26T07:21:14.322 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:21:14.317+0000 7faa8326b640 1 -- 172.21.15.143:0/3635443286 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7faa7c067680 con 0x7faa7c0cf990 2023-11-26T07:21:14.322 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:21:14.317+0000 7faa71ffb640 1 -- 172.21.15.143:0/3635443286 <== mon.0 v2:172.21.15.143:3300/0 4 ==== mgrmap(e 8) v1 ==== 86056+0+0 (secure 0 0 0) 0x7faa7801c460 con 0x7faa7c0cf990 2023-11-26T07:21:14.323 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:21:14.317+0000 7faa71ffb640 1 --2- 172.21.15.143:0/3635443286 >> [v2:172.21.15.143:6832/2150557544,v1:172.21.15.143:6833/2150557544] conn(0x7faa600666b0 0x7faa60068b70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:21:14.323 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:21:14.317+0000 7faa71ffb640 1 -- 172.21.15.143:0/3635443286 <== mon.0 v2:172.21.15.143:3300/0 5 ==== osd_map(47..47 src has 1..47) v4 ==== 5767+0+0 (secure 0 0 0) 0x7faa78084130 con 0x7faa7c0cf990 2023-11-26T07:21:14.324 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:21:14.317+0000 7faa80806640 1 --2- 172.21.15.143:0/3635443286 >> [v2:172.21.15.143:6832/2150557544,v1:172.21.15.143:6833/2150557544] conn(0x7faa600666b0 0x7faa60068b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:21:14.324 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:21:14.321+0000 7faa80806640 1 --2- 172.21.15.143:0/3635443286 >> [v2:172.21.15.143:6832/2150557544,v1:172.21.15.143:6833/2150557544] conn(0x7faa600666b0 0x7faa60068b70 secure :-1 s=READY pgs=339 cs=0 l=1 rev1=1 crypto rx=0x7faa7c149080 tx=0x7faa6c009040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:21:14.327 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:21:14.321+0000 7faa71ffb640 1 -- 172.21.15.143:0/3635443286 <== mon.0 v2:172.21.15.143:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172649 (secure 0 0 0) 0x7faa780512a0 con 0x7faa7c0cf990 2023-11-26T07:21:14.507 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:21:14.501+0000 7faa8326b640 1 -- 172.21.15.143:0/3635443286 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7faa7c145bd0 con 0x7faa7c0cf990 2023-11-26T07:21:14.508 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:21:14.501+0000 7faa71ffb640 1 -- 172.21.15.143:0/3635443286 <== mon.0 v2:172.21.15.143:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 26 v26) v1 ==== 94+0+4922 (secure 0 0 0) 0x7faa780548c0 con 0x7faa7c0cf990 2023-11-26T07:21:14.508 INFO:teuthology.orchestra.run.smithi143.stdout: 2023-11-26T07:21:14.508 INFO:teuthology.orchestra.run.smithi143.stdout:{"epoch":26,"default_fscid":3,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5201,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.188:6835/319464848","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6833","nonce":319464848},{"type":"v1","addr":"172.21.15.188:6835","nonce":319464848}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":20},{"gid":5584,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.188:6834/2113408070","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6832","nonce":2113408070},{"type":"v1","addr":"172.21.15.188:6834","nonce":2113408070}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":20},{"gid":5745,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.143:6835/1616742593","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6834","nonce":1616742593},{"type":"v1","addr":"172.21.15.143:6835","nonce":1616742593}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":20}],"filesystems":[{"mdsmap":{"epoch":25,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-11-26T07:19:31.503587+0000","modified":"2023-11-26T07:19:34.523605+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":5748},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5748":{"gid":5748,"name":"c","rank":0,"incarnation":21,"state":"up:active","state_seq":3,"addr":"172.21.15.143:6837/2872033338","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6836","nonce":2872033338},{"type":"v1","addr":"172.21.15.143:6837","nonce":2872033338}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[7],"metadata_pool":6,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":3}]} 2023-11-26T07:21:14.508 INFO:teuthology.orchestra.run.smithi143.stderr:dumped fsmap epoch 26 2023-11-26T07:21:14.511 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:21:14.505+0000 7faa8326b640 1 -- 172.21.15.143:0/3635443286 >> [v2:172.21.15.143:6832/2150557544,v1:172.21.15.143:6833/2150557544] conn(0x7faa600666b0 msgr2=0x7faa60068b70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:21:14.511 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:21:14.505+0000 7faa8326b640 1 --2- 172.21.15.143:0/3635443286 >> [v2:172.21.15.143:6832/2150557544,v1:172.21.15.143:6833/2150557544] conn(0x7faa600666b0 0x7faa60068b70 secure :-1 s=READY pgs=339 cs=0 l=1 rev1=1 crypto rx=0x7faa7c149080 tx=0x7faa6c009040 comp rx=0 tx=0).stop 2023-11-26T07:21:14.511 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:21:14.505+0000 7faa8326b640 1 -- 172.21.15.143:0/3635443286 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7faa7c0cf990 msgr2=0x7faa7c160750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:21:14.512 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:21:14.505+0000 7faa8326b640 1 --2- 172.21.15.143:0/3635443286 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7faa7c0cf990 0x7faa7c160750 secure :-1 s=READY pgs=709 cs=0 l=1 rev1=1 crypto rx=0x7faa7800c450 tx=0x7faa7800c910 comp rx=0 tx=0).stop 2023-11-26T07:21:14.512 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:21:14.505+0000 7faa8326b640 1 -- 172.21.15.143:0/3635443286 shutdown_connections 2023-11-26T07:21:14.512 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:21:14.505+0000 7faa8326b640 1 --2- 172.21.15.143:0/3635443286 >> [v2:172.21.15.143:6832/2150557544,v1:172.21.15.143:6833/2150557544] conn(0x7faa600666b0 0x7faa60068b70 unknown :-1 s=CLOSED pgs=339 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:21:14.512 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:21:14.505+0000 7faa8326b640 1 --2- 172.21.15.143:0/3635443286 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7faa7c163520 0x7faa7c14a4d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:21:14.512 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:21:14.505+0000 7faa8326b640 1 --2- 172.21.15.143:0/3635443286 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7faa7c14b1e0 0x7faa7c0a9940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:21:14.512 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:21:14.505+0000 7faa8326b640 1 --2- 172.21.15.143:0/3635443286 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7faa7c0cf990 0x7faa7c160750 unknown :-1 s=CLOSED pgs=709 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:21:14.513 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:21:14.505+0000 7faa8326b640 1 -- 172.21.15.143:0/3635443286 >> 172.21.15.143:0/3635443286 conn(0x7faa7c06a890 msgr2=0x7faa7c06ac70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:21:14.513 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:21:14.505+0000 7faa8326b640 1 -- 172.21.15.143:0/3635443286 shutdown_connections 2023-11-26T07:21:14.513 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:21:14.505+0000 7faa8326b640 1 -- 172.21.15.143:0/3635443286 wait complete. 2023-11-26T07:21:14.529 DEBUG:teuthology.orchestra.run.smithi143:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.c.asok --format=json session ls 2023-11-26T07:21:14.688 INFO:teuthology.orchestra.run.smithi143.stdout:[{"id":6126,"entity":{"name":{"type":"client","num":6126},"addr":{"type":"any","addr":"172.21.15.143:0","nonce":3879234400}},"state":"open","num_leases":0,"num_caps":7,"request_load_avg":6,"uptime":67.462261693000002,"requests_in_flight":0,"num_completed_requests":2,"num_completed_flushes":1,"reconnecting":false,"recall_caps":{"value":0,"halflife":60},"release_caps":{"value":0,"halflife":60},"recall_caps_throttle":{"value":0,"halflife":1.5},"recall_caps_throttle2o":{"value":0,"halflife":0.5},"session_cache_liveness":{"value":18.825324986054948,"halflife":300},"cap_acquisition":{"value":0,"halflife":30},"last_trim_completed_requests_tid":14,"last_trim_completed_flushes_tid":2,"delegated_inos":[{"start":"0x10000000005","length":500}],"inst":"client.6126 172.21.15.143:0/3879234400","completed_requests":[{"tid":14,"created_ino":"0x0"},{"tid":15,"created_ino":"0x100000001f9"}],"prealloc_inos":[{"start":"0x10000000005","length":500},{"start":"0x100000001fa","length":1000}],"client_metadata":{"client_features":{"feature_bits":"0x00000000003fffff"},"metric_spec":{"metric_flags":{"feature_bits":"0x000000000000ffff"}},"ceph_sha1":"44c2237fe77713214148c1fd622c872f346f1b99","ceph_version":"ceph version 18.0.0-7484-g44c2237f (44c2237fe77713214148c1fd622c872f346f1b99) reef (dev)","entity_id":"y","hostname":"smithi143","pid":"38097","root":"/"}}] 2023-11-26T07:21:14.689 DEBUG:tasks.cephfs.filesystem:_json_asok output [ { "cap_acquisition": { "halflife": 30, "value": 0 }, "client_metadata": { "ceph_sha1": "44c2237fe77713214148c1fd622c872f346f1b99", "ceph_version": "ceph version 18.0.0-7484-g44c2237f (44c2237fe77713214148c1fd622c872f346f1b99) reef (dev)", "client_features": { "feature_bits": "0x00000000003fffff" }, "entity_id": "y", "hostname": "smithi143", "metric_spec": { "metric_flags": { "feature_bits": "0x000000000000ffff" } }, "pid": "38097", "root": "/" }, "completed_requests": [ { "created_ino": "0x0", "tid": 14 }, { "created_ino": "0x100000001f9", "tid": 15 } ], "delegated_inos": [ { "length": 500, "start": "0x10000000005" } ], "entity": { "addr": { "addr": "172.21.15.143:0", "nonce": 3879234400, "type": "any" }, "name": { "num": 6126, "type": "client" } }, "id": 6126, "inst": "client.6126 172.21.15.143:0/3879234400", "last_trim_completed_flushes_tid": 2, "last_trim_completed_requests_tid": 14, "num_caps": 7, "num_completed_flushes": 1, "num_completed_requests": 2, "num_leases": 0, "prealloc_inos": [ { "length": 500, "start": "0x10000000005" }, { "length": 1000, "start": "0x100000001fa" } ], "recall_caps": { "halflife": 60, "value": 0 }, "recall_caps_throttle": { "halflife": 1.5, "value": 0 }, "recall_caps_throttle2o": { "halflife": 0.5, "value": 0 }, "reconnecting": false, "release_caps": { "halflife": 60, "value": 0 }, "request_load_avg": 6, "requests_in_flight": 0, "session_cache_liveness": { "halflife": 300, "value": 18.825324986054948 }, "state": "open", "uptime": 67.462261693 } ] 2023-11-26T07:21:14.689 DEBUG:tasks.ceph_test_case:wait_until_true: waiting (timeout=90 retry_count=0)... 2023-11-26T07:21:19.690 DEBUG:teuthology.orchestra.run.smithi143:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-11-26T07:21:19.840 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:21:19.833+0000 7ff5b6b2a640 1 Processor -- start 2023-11-26T07:21:19.841 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:21:19.837+0000 7ff5b6b2a640 1 -- start start 2023-11-26T07:21:19.842 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:21:19.837+0000 7ff5b6b2a640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7ff5b00cf280 0x7ff5b00cf6e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:21:19.842 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:21:19.837+0000 7ff5b6b2a640 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7ff5b00cd1b0 0x7ff5b00cd5e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:21:19.842 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:21:19.837+0000 7ff5b6b2a640 1 -- --> v1:172.21.15.143:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff5b00cfc20 con 0x7ff5b0143ac0 2023-11-26T07:21:19.842 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:21:19.837+0000 7ff5b6b2a640 1 -- --> v2:172.21.15.188:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff5b0143ea0 con 0x7ff5b00cd1b0 2023-11-26T07:21:19.842 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:21:19.837+0000 7ff5b6b2a640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff5b00cc730 con 0x7ff5b00cf280 2023-11-26T07:21:19.842 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:21:19.837+0000 7ff5b48c6640 1 --1- >> v1:172.21.15.143:6789/0 conn(0x7ff5b0143ac0 0x7ff5b0070370 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.143:6789/0 says I am v1:172.21.15.143:44402/0 (socket says 172.21.15.143:44402) 2023-11-26T07:21:19.842 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:21:19.837+0000 7ff5b48c6640 1 -- 172.21.15.143:0/2972139594 learned_addr learned my addr 172.21.15.143:0/2972139594 (peer_addr_for_me v1:172.21.15.143:0/0) 2023-11-26T07:21:19.843 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:21:19.837+0000 7ff5b50c7640 1 --2- 172.21.15.143:0/2972139594 >> v2:172.21.15.188:3300/0 conn(0x7ff5b00cd1b0 0x7ff5b00cd5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:21:19.843 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:21:19.837+0000 7ff5affff640 1 --2- 172.21.15.143:0/2972139594 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7ff5b00cf280 0x7ff5b00cf6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:21:19.843 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:21:19.837+0000 7ff5af7fe640 1 -- 172.21.15.143:0/2972139594 <== mon.0 v1:172.21.15.143:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3213063083 0 0) 0x7ff5b00cfc20 con 0x7ff5b0143ac0 2023-11-26T07:21:19.843 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:21:19.837+0000 7ff5af7fe640 1 -- 172.21.15.143:0/2972139594 --> v1:172.21.15.143:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7ff5a0003530 con 0x7ff5b0143ac0 2023-11-26T07:21:19.843 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:21:19.837+0000 7ff5affff640 1 -- 172.21.15.143:0/2972139594 >> v2:172.21.15.188:3300/0 conn(0x7ff5b00cd1b0 msgr2=0x7ff5b00cd5e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-26T07:21:19.844 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:21:19.837+0000 7ff5affff640 1 --2- 172.21.15.143:0/2972139594 >> v2:172.21.15.188:3300/0 conn(0x7ff5b00cd1b0 0x7ff5b00cd5e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:21:19.844 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:21:19.837+0000 7ff5affff640 1 -- 172.21.15.143:0/2972139594 >> v1:172.21.15.143:6789/0 conn(0x7ff5b0143ac0 legacy=0x7ff5b0070370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:21:19.844 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:21:19.837+0000 7ff5affff640 1 -- 172.21.15.143:0/2972139594 --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff5b0070ee0 con 0x7ff5b00cf280 2023-11-26T07:21:19.845 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:21:19.841+0000 7ff5affff640 1 --2- 172.21.15.143:0/2972139594 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7ff5b00cf280 0x7ff5b00cf6e0 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7ff59c0097c0 tx=0x7ff59c01c480 comp rx=0 tx=0).ready entity=mon.2 client_cookie=2e6db61c0fe7ab04 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:21:19.846 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:21:19.841+0000 7ff5af7fe640 1 -- 172.21.15.143:0/2972139594 <== mon.2 v2:172.21.15.188:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff59c0ab680 con 0x7ff5b00cf280 2023-11-26T07:21:19.847 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:21:19.841+0000 7ff5af7fe640 1 -- 172.21.15.143:0/2972139594 <== mon.2 v2:172.21.15.188:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff59c0ab7e0 con 0x7ff5b00cf280 2023-11-26T07:21:19.847 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:21:19.841+0000 7ff5af7fe640 1 -- 172.21.15.143:0/2972139594 <== mon.2 v2:172.21.15.188:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff59c0abb70 con 0x7ff5b00cf280 2023-11-26T07:21:19.847 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:21:19.841+0000 7ff5b6b2a640 1 -- 172.21.15.143:0/2972139594 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7ff5b00cf280 msgr2=0x7ff5b00cf6e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:21:19.847 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:21:19.841+0000 7ff5b6b2a640 1 --2- 172.21.15.143:0/2972139594 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7ff5b00cf280 0x7ff5b00cf6e0 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7ff59c0097c0 tx=0x7ff59c01c480 comp rx=0 tx=0).stop 2023-11-26T07:21:19.847 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:21:19.841+0000 7ff5b6b2a640 1 -- 172.21.15.143:0/2972139594 shutdown_connections 2023-11-26T07:21:19.847 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:21:19.841+0000 7ff5b6b2a640 1 --2- 172.21.15.143:0/2972139594 >> v2:172.21.15.188:3300/0 conn(0x7ff5b00cd1b0 0x7ff5b00cd5e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:21:19.847 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:21:19.841+0000 7ff5b6b2a640 1 --2- 172.21.15.143:0/2972139594 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7ff5b00cf280 0x7ff5b00cf6e0 unknown :-1 s=CLOSED pgs=96 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:21:19.847 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:21:19.841+0000 7ff5b6b2a640 1 -- 172.21.15.143:0/2972139594 >> 172.21.15.143:0/2972139594 conn(0x7ff5b006a890 msgr2=0x7ff5b006aca0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:21:19.847 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:21:19.841+0000 7ff5b6b2a640 1 -- 172.21.15.143:0/2972139594 shutdown_connections 2023-11-26T07:21:19.847 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:21:19.841+0000 7ff5b6b2a640 1 -- 172.21.15.143:0/2972139594 wait complete. 2023-11-26T07:21:19.848 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:21:19.845+0000 7ff5b6b2a640 1 Processor -- start 2023-11-26T07:21:19.848 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:21:19.845+0000 7ff5b6b2a640 1 -- start start 2023-11-26T07:21:19.850 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:21:19.845+0000 7ff5b6b2a640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7ff5b00cd1b0 0x7ff5b013baf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:21:19.850 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:21:19.845+0000 7ff5b6b2a640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7ff5b00cf280 0x7ff5b013d590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:21:19.850 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:21:19.845+0000 7ff5b6b2a640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7ff5b0143ac0 0x7ff5b013dad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:21:19.851 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:21:19.845+0000 7ff5b6b2a640 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff5b013c030 con 0x7ff5b00cd1b0 2023-11-26T07:21:19.851 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:21:19.845+0000 7ff5b6b2a640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff5b0149670 con 0x7ff5b00cf280 2023-11-26T07:21:19.851 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:21:19.845+0000 7ff5b6b2a640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff5b0149950 con 0x7ff5b0143ac0 2023-11-26T07:21:19.851 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:21:19.845+0000 7ff5b48c6640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7ff5b00cd1b0 0x7ff5b013baf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:21:19.851 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:21:19.845+0000 7ff5b48c6640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7ff5b00cd1b0 0x7ff5b013baf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:54348/0 (socket says 172.21.15.143:54348) 2023-11-26T07:21:19.851 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:21:19.845+0000 7ff5b48c6640 1 -- 172.21.15.143:0/2146094897 learned_addr learned my addr 172.21.15.143:0/2146094897 (peer_addr_for_me v2:172.21.15.143:0/0) 2023-11-26T07:21:19.851 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:21:19.845+0000 7ff5b50c7640 1 --2- 172.21.15.143:0/2146094897 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7ff5b0143ac0 0x7ff5b013dad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:21:19.851 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:21:19.845+0000 7ff5affff640 1 --2- 172.21.15.143:0/2146094897 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7ff5b00cf280 0x7ff5b013d590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:21:19.851 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:21:19.845+0000 7ff5b48c6640 1 -- 172.21.15.143:0/2146094897 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7ff5b0143ac0 msgr2=0x7ff5b013dad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:21:19.851 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:21:19.845+0000 7ff5b48c6640 1 --2- 172.21.15.143:0/2146094897 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7ff5b0143ac0 0x7ff5b013dad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:21:19.851 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:21:19.845+0000 7ff5b48c6640 1 -- 172.21.15.143:0/2146094897 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7ff5b00cf280 msgr2=0x7ff5b013d590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:21:19.851 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:21:19.845+0000 7ff5b48c6640 1 --2- 172.21.15.143:0/2146094897 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7ff5b00cf280 0x7ff5b013d590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:21:19.852 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:21:19.845+0000 7ff5b48c6640 1 -- 172.21.15.143:0/2146094897 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff59c009420 con 0x7ff5b00cd1b0 2023-11-26T07:21:19.852 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:21:19.845+0000 7ff5b48c6640 1 --2- 172.21.15.143:0/2146094897 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7ff5b00cd1b0 0x7ff5b013baf0 secure :-1 s=READY pgs=711 cs=0 l=1 rev1=1 crypto rx=0x7ff5a400c420 tx=0x7ff5a400c8e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:21:19.853 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:21:19.845+0000 7ff5ad7fa640 1 -- 172.21.15.143:0/2146094897 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff5a4005cf0 con 0x7ff5b00cd1b0 2023-11-26T07:21:19.853 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:21:19.845+0000 7ff5ad7fa640 1 -- 172.21.15.143:0/2146094897 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff5a4005e50 con 0x7ff5b00cd1b0 2023-11-26T07:21:19.853 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:21:19.845+0000 7ff5ad7fa640 1 -- 172.21.15.143:0/2146094897 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff5a40072d0 con 0x7ff5b00cd1b0 2023-11-26T07:21:19.853 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:21:19.845+0000 7ff5b6b2a640 1 -- 172.21.15.143:0/2146094897 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff5b0149c30 con 0x7ff5b00cd1b0 2023-11-26T07:21:19.853 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:21:19.845+0000 7ff5b6b2a640 1 -- 172.21.15.143:0/2146094897 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff5b014a1f0 con 0x7ff5b00cd1b0 2023-11-26T07:21:19.853 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:21:19.845+0000 7ff5b6b2a640 1 -- 172.21.15.143:0/2146094897 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff5b0067680 con 0x7ff5b00cd1b0 2023-11-26T07:21:19.853 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:21:19.849+0000 7ff5ad7fa640 1 -- 172.21.15.143:0/2146094897 <== mon.0 v2:172.21.15.143:3300/0 4 ==== mgrmap(e 8) v1 ==== 86056+0+0 (secure 0 0 0) 0x7ff5a4007430 con 0x7ff5b00cd1b0 2023-11-26T07:21:19.854 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:21:19.849+0000 7ff5ad7fa640 1 --2- 172.21.15.143:0/2146094897 >> [v2:172.21.15.143:6832/2150557544,v1:172.21.15.143:6833/2150557544] conn(0x7ff5940666b0 0x7ff594068b70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:21:19.854 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:21:19.849+0000 7ff5ad7fa640 1 -- 172.21.15.143:0/2146094897 <== mon.0 v2:172.21.15.143:3300/0 5 ==== osd_map(47..47 src has 1..47) v4 ==== 5767+0+0 (secure 0 0 0) 0x7ff5a4083930 con 0x7ff5b00cd1b0 2023-11-26T07:21:19.854 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:21:19.849+0000 7ff5affff640 1 --2- 172.21.15.143:0/2146094897 >> [v2:172.21.15.143:6832/2150557544,v1:172.21.15.143:6833/2150557544] conn(0x7ff5940666b0 0x7ff594068b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:21:19.854 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:21:19.849+0000 7ff5affff640 1 --2- 172.21.15.143:0/2146094897 >> [v2:172.21.15.143:6832/2150557544,v1:172.21.15.143:6833/2150557544] conn(0x7ff5940666b0 0x7ff594068b70 secure :-1 s=READY pgs=340 cs=0 l=1 rev1=1 crypto rx=0x7ff59c0af000 tx=0x7ff59c0a7000 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:21:19.858 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:21:19.853+0000 7ff5ad7fa640 1 -- 172.21.15.143:0/2146094897 <== mon.0 v2:172.21.15.143:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172649 (secure 0 0 0) 0x7ff5a4050990 con 0x7ff5b00cd1b0 2023-11-26T07:21:20.042 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:21:20.037+0000 7ff5b6b2a640 1 -- 172.21.15.143:0/2146094897 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7ff5b00b1130 con 0x7ff5b00cd1b0 2023-11-26T07:21:20.043 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:21:20.037+0000 7ff5ad7fa640 1 -- 172.21.15.143:0/2146094897 <== mon.0 v2:172.21.15.143:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 26 v26) v1 ==== 94+0+4922 (secure 0 0 0) 0x7ff5a4017070 con 0x7ff5b00cd1b0 2023-11-26T07:21:20.043 INFO:teuthology.orchestra.run.smithi143.stdout: 2023-11-26T07:21:20.043 INFO:teuthology.orchestra.run.smithi143.stdout:{"epoch":26,"default_fscid":3,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5201,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.188:6835/319464848","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6833","nonce":319464848},{"type":"v1","addr":"172.21.15.188:6835","nonce":319464848}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":20},{"gid":5584,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.188:6834/2113408070","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6832","nonce":2113408070},{"type":"v1","addr":"172.21.15.188:6834","nonce":2113408070}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":20},{"gid":5745,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.143:6835/1616742593","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6834","nonce":1616742593},{"type":"v1","addr":"172.21.15.143:6835","nonce":1616742593}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":20}],"filesystems":[{"mdsmap":{"epoch":25,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-11-26T07:19:31.503587+0000","modified":"2023-11-26T07:19:34.523605+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":5748},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5748":{"gid":5748,"name":"c","rank":0,"incarnation":21,"state":"up:active","state_seq":3,"addr":"172.21.15.143:6837/2872033338","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6836","nonce":2872033338},{"type":"v1","addr":"172.21.15.143:6837","nonce":2872033338}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[7],"metadata_pool":6,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":3}]} 2023-11-26T07:21:20.043 INFO:teuthology.orchestra.run.smithi143.stderr:dumped fsmap epoch 26 2023-11-26T07:21:20.045 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:21:20.041+0000 7ff5b6b2a640 1 -- 172.21.15.143:0/2146094897 >> [v2:172.21.15.143:6832/2150557544,v1:172.21.15.143:6833/2150557544] conn(0x7ff5940666b0 msgr2=0x7ff594068b70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:21:20.045 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:21:20.041+0000 7ff5b6b2a640 1 --2- 172.21.15.143:0/2146094897 >> [v2:172.21.15.143:6832/2150557544,v1:172.21.15.143:6833/2150557544] conn(0x7ff5940666b0 0x7ff594068b70 secure :-1 s=READY pgs=340 cs=0 l=1 rev1=1 crypto rx=0x7ff59c0af000 tx=0x7ff59c0a7000 comp rx=0 tx=0).stop 2023-11-26T07:21:20.045 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:21:20.041+0000 7ff5b6b2a640 1 -- 172.21.15.143:0/2146094897 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7ff5b00cd1b0 msgr2=0x7ff5b013baf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:21:20.045 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:21:20.041+0000 7ff5b6b2a640 1 --2- 172.21.15.143:0/2146094897 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7ff5b00cd1b0 0x7ff5b013baf0 secure :-1 s=READY pgs=711 cs=0 l=1 rev1=1 crypto rx=0x7ff5a400c420 tx=0x7ff5a400c8e0 comp rx=0 tx=0).stop 2023-11-26T07:21:20.045 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:21:20.041+0000 7ff5b6b2a640 1 -- 172.21.15.143:0/2146094897 shutdown_connections 2023-11-26T07:21:20.045 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:21:20.041+0000 7ff5b6b2a640 1 --2- 172.21.15.143:0/2146094897 >> [v2:172.21.15.143:6832/2150557544,v1:172.21.15.143:6833/2150557544] conn(0x7ff5940666b0 0x7ff594068b70 unknown :-1 s=CLOSED pgs=340 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:21:20.045 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:21:20.041+0000 7ff5b6b2a640 1 --2- 172.21.15.143:0/2146094897 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7ff5b0143ac0 0x7ff5b013dad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:21:20.045 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:21:20.041+0000 7ff5b6b2a640 1 --2- 172.21.15.143:0/2146094897 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7ff5b00cf280 0x7ff5b013d590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:21:20.046 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:21:20.041+0000 7ff5b6b2a640 1 --2- 172.21.15.143:0/2146094897 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7ff5b00cd1b0 0x7ff5b013baf0 unknown :-1 s=CLOSED pgs=711 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:21:20.046 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:21:20.041+0000 7ff5b6b2a640 1 -- 172.21.15.143:0/2146094897 >> 172.21.15.143:0/2146094897 conn(0x7ff5b006a890 msgr2=0x7ff5b006c520 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:21:20.046 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:21:20.041+0000 7ff5b6b2a640 1 -- 172.21.15.143:0/2146094897 shutdown_connections 2023-11-26T07:21:20.046 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:21:20.041+0000 7ff5b6b2a640 1 -- 172.21.15.143:0/2146094897 wait complete. 2023-11-26T07:21:20.064 DEBUG:teuthology.orchestra.run.smithi143:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.c.asok --format=json session ls 2023-11-26T07:21:20.223 INFO:teuthology.orchestra.run.smithi143.stdout:[{"id":6126,"entity":{"name":{"type":"client","num":6126},"addr":{"type":"any","addr":"172.21.15.143:0","nonce":3879234400}},"state":"open","num_leases":0,"num_caps":7,"request_load_avg":6,"uptime":72.994011561999997,"requests_in_flight":0,"num_completed_requests":2,"num_completed_flushes":1,"reconnecting":false,"recall_caps":{"value":0,"halflife":60},"release_caps":{"value":0,"halflife":60},"recall_caps_throttle":{"value":0,"halflife":1.5},"recall_caps_throttle2o":{"value":0,"halflife":0.5},"session_cache_liveness":{"value":18.586248530922465,"halflife":300},"cap_acquisition":{"value":0,"halflife":30},"last_trim_completed_requests_tid":14,"last_trim_completed_flushes_tid":2,"delegated_inos":[{"start":"0x10000000005","length":500}],"inst":"client.6126 172.21.15.143:0/3879234400","completed_requests":[{"tid":14,"created_ino":"0x0"},{"tid":15,"created_ino":"0x100000001f9"}],"prealloc_inos":[{"start":"0x10000000005","length":500},{"start":"0x100000001fa","length":1000}],"client_metadata":{"client_features":{"feature_bits":"0x00000000003fffff"},"metric_spec":{"metric_flags":{"feature_bits":"0x000000000000ffff"}},"ceph_sha1":"44c2237fe77713214148c1fd622c872f346f1b99","ceph_version":"ceph version 18.0.0-7484-g44c2237f (44c2237fe77713214148c1fd622c872f346f1b99) reef (dev)","entity_id":"y","hostname":"smithi143","pid":"38097","root":"/"}}] 2023-11-26T07:21:20.223 DEBUG:tasks.cephfs.filesystem:_json_asok output [ { "cap_acquisition": { "halflife": 30, "value": 0 }, "client_metadata": { "ceph_sha1": "44c2237fe77713214148c1fd622c872f346f1b99", "ceph_version": "ceph version 18.0.0-7484-g44c2237f (44c2237fe77713214148c1fd622c872f346f1b99) reef (dev)", "client_features": { "feature_bits": "0x00000000003fffff" }, "entity_id": "y", "hostname": "smithi143", "metric_spec": { "metric_flags": { "feature_bits": "0x000000000000ffff" } }, "pid": "38097", "root": "/" }, "completed_requests": [ { "created_ino": "0x0", "tid": 14 }, { "created_ino": "0x100000001f9", "tid": 15 } ], "delegated_inos": [ { "length": 500, "start": "0x10000000005" } ], "entity": { "addr": { "addr": "172.21.15.143:0", "nonce": 3879234400, "type": "any" }, "name": { "num": 6126, "type": "client" } }, "id": 6126, "inst": "client.6126 172.21.15.143:0/3879234400", "last_trim_completed_flushes_tid": 2, "last_trim_completed_requests_tid": 14, "num_caps": 7, "num_completed_flushes": 1, "num_completed_requests": 2, "num_leases": 0, "prealloc_inos": [ { "length": 500, "start": "0x10000000005" }, { "length": 1000, "start": "0x100000001fa" } ], "recall_caps": { "halflife": 60, "value": 0 }, "recall_caps_throttle": { "halflife": 1.5, "value": 0 }, "recall_caps_throttle2o": { "halflife": 0.5, "value": 0 }, "reconnecting": false, "release_caps": { "halflife": 60, "value": 0 }, "request_load_avg": 6, "requests_in_flight": 0, "session_cache_liveness": { "halflife": 300, "value": 18.586248530922465 }, "state": "open", "uptime": 72.994011562 } ] 2023-11-26T07:21:20.223 DEBUG:tasks.ceph_test_case:wait_until_true: waiting (timeout=90 retry_count=0)... 2023-11-26T07:21:22.116 DEBUG:teuthology.orchestra.run.smithi143:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-26T07:21:22.121 DEBUG:teuthology.orchestra.run.smithi188:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-26T07:21:25.224 DEBUG:teuthology.orchestra.run.smithi143:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-11-26T07:21:27.642 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:21:27.637+0000 7ff6da135640 1 Processor -- start 2023-11-26T07:21:27.642 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:21:27.637+0000 7ff6da135640 1 -- start start 2023-11-26T07:21:27.643 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:21:27.637+0000 7ff6da135640 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7ff6d4143ac0 0x7ff6d4070370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:21:27.643 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:21:27.637+0000 7ff6da135640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7ff6d40708b0 0x7ff6d40cf280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:21:27.644 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:21:27.637+0000 7ff6da135640 1 -- --> v1:172.21.15.188:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff6d40cfbf0 con 0x7ff6d40cf7c0 2023-11-26T07:21:27.644 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:21:27.637+0000 7ff6da135640 1 -- --> v2:172.21.15.188:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff6d4143ea0 con 0x7ff6d4143ac0 2023-11-26T07:21:27.644 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:21:27.637+0000 7ff6da135640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff6d40cc730 con 0x7ff6d40708b0 2023-11-26T07:21:27.645 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:21:27.637+0000 7ff6d2ffd640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7ff6d40708b0 0x7ff6d40cf280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:21:27.645 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:21:27.637+0000 7ff6d37fe640 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7ff6d4143ac0 0x7ff6d4070370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:21:27.645 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:21:27.637+0000 7ff6d3fff640 1 --1- >> v1:172.21.15.188:6789/0 conn(0x7ff6d40cf7c0 0x7ff6d40b1ef0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.188:6789/0 says I am v1:172.21.15.143:39122/0 (socket says 172.21.15.143:39122) 2023-11-26T07:21:27.645 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:21:27.637+0000 7ff6d3fff640 1 -- 172.21.15.143:0/2034444611 learned_addr learned my addr 172.21.15.143:0/2034444611 (peer_addr_for_me v1:172.21.15.143:0/0) 2023-11-26T07:21:27.645 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:21:27.641+0000 7ff6d27fc640 1 -- 172.21.15.143:0/2034444611 <== mon.1 v1:172.21.15.188:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 964677764 0 0) 0x7ff6d40cfbf0 con 0x7ff6d40cf7c0 2023-11-26T07:21:27.645 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:21:27.641+0000 7ff6d27fc640 1 -- 172.21.15.143:0/2034444611 --> v1:172.21.15.188:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7ff6bc003530 con 0x7ff6d40cf7c0 2023-11-26T07:21:27.645 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:21:27.641+0000 7ff6d2ffd640 1 -- 172.21.15.143:0/2034444611 >> v2:172.21.15.188:3300/0 conn(0x7ff6d4143ac0 msgr2=0x7ff6d4070370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-26T07:21:27.645 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:21:27.641+0000 7ff6d2ffd640 1 --2- 172.21.15.143:0/2034444611 >> v2:172.21.15.188:3300/0 conn(0x7ff6d4143ac0 0x7ff6d4070370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:21:27.645 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:21:27.641+0000 7ff6d2ffd640 1 -- 172.21.15.143:0/2034444611 >> v1:172.21.15.188:6789/0 conn(0x7ff6d40cf7c0 legacy=0x7ff6d40b1ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:21:27.645 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:21:27.641+0000 7ff6d2ffd640 1 -- 172.21.15.143:0/2034444611 --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff6d4070ee0 con 0x7ff6d40708b0 2023-11-26T07:21:27.645 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:21:27.641+0000 7ff6d37fe640 1 --2- 172.21.15.143:0/2034444611 >> v2:172.21.15.188:3300/0 conn(0x7ff6d4143ac0 0x7ff6d4070370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-26T07:21:27.650 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:21:27.641+0000 7ff6d2ffd640 1 --2- 172.21.15.143:0/2034444611 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7ff6d40708b0 0x7ff6d40cf280 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7ff6c80097c0 tx=0x7ff6c801c480 comp rx=0 tx=0).ready entity=mon.2 client_cookie=6bf23e12a9cb1fd1 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:21:27.650 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:21:27.645+0000 7ff6d27fc640 1 -- 172.21.15.143:0/2034444611 <== mon.2 v2:172.21.15.188:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff6c80ad640 con 0x7ff6d40708b0 2023-11-26T07:21:27.650 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:21:27.645+0000 7ff6d27fc640 1 -- 172.21.15.143:0/2034444611 <== mon.2 v2:172.21.15.188:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff6c80ad7a0 con 0x7ff6d40708b0 2023-11-26T07:21:27.650 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:21:27.645+0000 7ff6d27fc640 1 -- 172.21.15.143:0/2034444611 <== mon.2 v2:172.21.15.188:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff6c80adb30 con 0x7ff6d40708b0 2023-11-26T07:21:27.650 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:21:27.645+0000 7ff6da135640 1 -- 172.21.15.143:0/2034444611 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7ff6d40708b0 msgr2=0x7ff6d40cf280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:21:27.650 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:21:27.645+0000 7ff6da135640 1 --2- 172.21.15.143:0/2034444611 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7ff6d40708b0 0x7ff6d40cf280 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7ff6c80097c0 tx=0x7ff6c801c480 comp rx=0 tx=0).stop 2023-11-26T07:21:27.650 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:21:27.645+0000 7ff6da135640 1 -- 172.21.15.143:0/2034444611 shutdown_connections 2023-11-26T07:21:27.650 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:21:27.645+0000 7ff6da135640 1 --2- 172.21.15.143:0/2034444611 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7ff6d40708b0 0x7ff6d40cf280 unknown :-1 s=CLOSED pgs=97 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:21:27.650 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:21:27.645+0000 7ff6da135640 1 --2- 172.21.15.143:0/2034444611 >> v2:172.21.15.188:3300/0 conn(0x7ff6d4143ac0 0x7ff6d4070370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:21:27.651 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:21:27.645+0000 7ff6da135640 1 -- 172.21.15.143:0/2034444611 >> 172.21.15.143:0/2034444611 conn(0x7ff6d406a890 msgr2=0x7ff6d406aca0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:21:27.651 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:21:27.645+0000 7ff6da135640 1 -- 172.21.15.143:0/2034444611 shutdown_connections 2023-11-26T07:21:27.651 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:21:27.645+0000 7ff6da135640 1 -- 172.21.15.143:0/2034444611 wait complete. 2023-11-26T07:21:27.651 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:21:27.645+0000 7ff6da135640 1 Processor -- start 2023-11-26T07:21:27.651 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:21:27.645+0000 7ff6da135640 1 -- start start 2023-11-26T07:21:27.652 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:21:27.645+0000 7ff6da135640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7ff6d40708b0 0x7ff6d413afa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:21:27.653 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:21:27.645+0000 7ff6da135640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7ff6d40cf7c0 0x7ff6d413b760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:21:27.653 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:21:27.645+0000 7ff6da135640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7ff6d4143ac0 0x7ff6d413bca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:21:27.653 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:21:27.645+0000 7ff6da135640 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff6d40b0c50 con 0x7ff6d40708b0 2023-11-26T07:21:27.653 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:21:27.645+0000 7ff6da135640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff6d40ad1d0 con 0x7ff6d4143ac0 2023-11-26T07:21:27.653 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:21:27.645+0000 7ff6da135640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff6d40ad4b0 con 0x7ff6d40cf7c0 2023-11-26T07:21:27.653 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:21:27.645+0000 7ff6d37fe640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7ff6d40708b0 0x7ff6d413afa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:21:27.653 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:21:27.645+0000 7ff6d37fe640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7ff6d40708b0 0x7ff6d413afa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:34880/0 (socket says 172.21.15.143:34880) 2023-11-26T07:21:27.653 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:21:27.645+0000 7ff6d37fe640 1 -- 172.21.15.143:0/2317213673 learned_addr learned my addr 172.21.15.143:0/2317213673 (peer_addr_for_me v2:172.21.15.143:0/0) 2023-11-26T07:21:27.653 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:21:27.649+0000 7ff6d3fff640 1 --2- 172.21.15.143:0/2317213673 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7ff6d4143ac0 0x7ff6d413bca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:21:27.653 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:21:27.649+0000 7ff6d2ffd640 1 --2- 172.21.15.143:0/2317213673 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7ff6d40cf7c0 0x7ff6d413b760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:21:27.653 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:21:27.649+0000 7ff6d37fe640 1 -- 172.21.15.143:0/2317213673 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7ff6d40cf7c0 msgr2=0x7ff6d413b760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:21:27.653 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:21:27.649+0000 7ff6d37fe640 1 --2- 172.21.15.143:0/2317213673 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7ff6d40cf7c0 0x7ff6d413b760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:21:27.653 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:21:27.649+0000 7ff6d37fe640 1 -- 172.21.15.143:0/2317213673 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7ff6d4143ac0 msgr2=0x7ff6d413bca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:21:27.653 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:21:27.649+0000 7ff6d37fe640 1 --2- 172.21.15.143:0/2317213673 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7ff6d4143ac0 0x7ff6d413bca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:21:27.654 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:21:27.649+0000 7ff6d37fe640 1 -- 172.21.15.143:0/2317213673 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff6c4009420 con 0x7ff6d40708b0 2023-11-26T07:21:27.654 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:21:27.649+0000 7ff6d37fe640 1 --2- 172.21.15.143:0/2317213673 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7ff6d40708b0 0x7ff6d413afa0 secure :-1 s=READY pgs=712 cs=0 l=1 rev1=1 crypto rx=0x7ff6c400ebe0 tx=0x7ff6c400c520 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:21:27.654 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:21:27.649+0000 7ff6bbfff640 1 -- 172.21.15.143:0/2317213673 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff6c400eee0 con 0x7ff6d40708b0 2023-11-26T07:21:27.655 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:21:27.649+0000 7ff6bbfff640 1 -- 172.21.15.143:0/2317213673 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff6c400cc30 con 0x7ff6d40708b0 2023-11-26T07:21:27.655 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:21:27.649+0000 7ff6bbfff640 1 -- 172.21.15.143:0/2317213673 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff6c4013470 con 0x7ff6d40708b0 2023-11-26T07:21:27.655 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:21:27.649+0000 7ff6da135640 1 -- 172.21.15.143:0/2317213673 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff6c8009420 con 0x7ff6d40708b0 2023-11-26T07:21:27.655 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:21:27.649+0000 7ff6da135640 1 -- 172.21.15.143:0/2317213673 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff6d40b11d0 con 0x7ff6d40708b0 2023-11-26T07:21:27.655 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:21:27.649+0000 7ff6da135640 1 -- 172.21.15.143:0/2317213673 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff6d40ad670 con 0x7ff6d40708b0 2023-11-26T07:21:27.655 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:21:27.649+0000 7ff6bbfff640 1 -- 172.21.15.143:0/2317213673 <== mon.0 v2:172.21.15.143:3300/0 4 ==== mgrmap(e 8) v1 ==== 86056+0+0 (secure 0 0 0) 0x7ff6c40135d0 con 0x7ff6d40708b0 2023-11-26T07:21:27.656 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:21:27.649+0000 7ff6bbfff640 1 --2- 172.21.15.143:0/2317213673 >> [v2:172.21.15.143:6832/2150557544,v1:172.21.15.143:6833/2150557544] conn(0x7ff6c00666b0 0x7ff6c0068b70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:21:27.656 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:21:27.653+0000 7ff6bbfff640 1 -- 172.21.15.143:0/2317213673 <== mon.0 v2:172.21.15.143:3300/0 5 ==== osd_map(47..47 src has 1..47) v4 ==== 5767+0+0 (secure 0 0 0) 0x7ff6c40845b0 con 0x7ff6d40708b0 2023-11-26T07:21:27.656 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:21:27.653+0000 7ff6d2ffd640 1 --2- 172.21.15.143:0/2317213673 >> [v2:172.21.15.143:6832/2150557544,v1:172.21.15.143:6833/2150557544] conn(0x7ff6c00666b0 0x7ff6c0068b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:21:27.657 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:21:27.653+0000 7ff6d2ffd640 1 --2- 172.21.15.143:0/2317213673 >> [v2:172.21.15.143:6832/2150557544,v1:172.21.15.143:6833/2150557544] conn(0x7ff6c00666b0 0x7ff6c0068b70 secure :-1 s=READY pgs=341 cs=0 l=1 rev1=1 crypto rx=0x7ff6c80af000 tx=0x7ff6c8008040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:21:27.661 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:21:27.657+0000 7ff6bbfff640 1 -- 172.21.15.143:0/2317213673 <== mon.0 v2:172.21.15.143:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172649 (secure 0 0 0) 0x7ff6c4051730 con 0x7ff6d40708b0 2023-11-26T07:21:27.839 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:21:27.833+0000 7ff6da135640 1 -- 172.21.15.143:0/2317213673 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7ff6d4067680 con 0x7ff6d40708b0 2023-11-26T07:21:27.840 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:21:27.833+0000 7ff6bbfff640 1 -- 172.21.15.143:0/2317213673 <== mon.0 v2:172.21.15.143:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 26 v26) v1 ==== 94+0+4922 (secure 0 0 0) 0x7ff6c4054d50 con 0x7ff6d40708b0 2023-11-26T07:21:27.840 INFO:teuthology.orchestra.run.smithi143.stdout: 2023-11-26T07:21:27.840 INFO:teuthology.orchestra.run.smithi143.stdout:{"epoch":26,"default_fscid":3,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5201,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.188:6835/319464848","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6833","nonce":319464848},{"type":"v1","addr":"172.21.15.188:6835","nonce":319464848}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":20},{"gid":5584,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.188:6834/2113408070","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6832","nonce":2113408070},{"type":"v1","addr":"172.21.15.188:6834","nonce":2113408070}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":20},{"gid":5745,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.143:6835/1616742593","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6834","nonce":1616742593},{"type":"v1","addr":"172.21.15.143:6835","nonce":1616742593}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":20}],"filesystems":[{"mdsmap":{"epoch":25,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-11-26T07:19:31.503587+0000","modified":"2023-11-26T07:19:34.523605+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":5748},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5748":{"gid":5748,"name":"c","rank":0,"incarnation":21,"state":"up:active","state_seq":3,"addr":"172.21.15.143:6837/2872033338","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6836","nonce":2872033338},{"type":"v1","addr":"172.21.15.143:6837","nonce":2872033338}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[7],"metadata_pool":6,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":3}]} 2023-11-26T07:21:27.841 INFO:teuthology.orchestra.run.smithi143.stderr:dumped fsmap epoch 26 2023-11-26T07:21:27.842 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:21:27.837+0000 7ff6da135640 1 -- 172.21.15.143:0/2317213673 >> [v2:172.21.15.143:6832/2150557544,v1:172.21.15.143:6833/2150557544] conn(0x7ff6c00666b0 msgr2=0x7ff6c0068b70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:21:27.842 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:21:27.837+0000 7ff6da135640 1 --2- 172.21.15.143:0/2317213673 >> [v2:172.21.15.143:6832/2150557544,v1:172.21.15.143:6833/2150557544] conn(0x7ff6c00666b0 0x7ff6c0068b70 secure :-1 s=READY pgs=341 cs=0 l=1 rev1=1 crypto rx=0x7ff6c80af000 tx=0x7ff6c8008040 comp rx=0 tx=0).stop 2023-11-26T07:21:27.843 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:21:27.837+0000 7ff6da135640 1 -- 172.21.15.143:0/2317213673 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7ff6d40708b0 msgr2=0x7ff6d413afa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:21:27.843 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:21:27.837+0000 7ff6da135640 1 --2- 172.21.15.143:0/2317213673 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7ff6d40708b0 0x7ff6d413afa0 secure :-1 s=READY pgs=712 cs=0 l=1 rev1=1 crypto rx=0x7ff6c400ebe0 tx=0x7ff6c400c520 comp rx=0 tx=0).stop 2023-11-26T07:21:27.843 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:21:27.837+0000 7ff6da135640 1 -- 172.21.15.143:0/2317213673 shutdown_connections 2023-11-26T07:21:27.843 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:21:27.837+0000 7ff6da135640 1 --2- 172.21.15.143:0/2317213673 >> [v2:172.21.15.143:6832/2150557544,v1:172.21.15.143:6833/2150557544] conn(0x7ff6c00666b0 0x7ff6c0068b70 unknown :-1 s=CLOSED pgs=341 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:21:27.843 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:21:27.837+0000 7ff6da135640 1 --2- 172.21.15.143:0/2317213673 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7ff6d4143ac0 0x7ff6d413bca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:21:27.843 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:21:27.837+0000 7ff6da135640 1 --2- 172.21.15.143:0/2317213673 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7ff6d40cf7c0 0x7ff6d413b760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:21:27.843 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:21:27.837+0000 7ff6da135640 1 --2- 172.21.15.143:0/2317213673 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7ff6d40708b0 0x7ff6d413afa0 unknown :-1 s=CLOSED pgs=712 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:21:27.843 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:21:27.837+0000 7ff6da135640 1 -- 172.21.15.143:0/2317213673 >> 172.21.15.143:0/2317213673 conn(0x7ff6d406a890 msgr2=0x7ff6d4072130 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:21:27.843 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:21:27.837+0000 7ff6da135640 1 -- 172.21.15.143:0/2317213673 shutdown_connections 2023-11-26T07:21:27.843 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:21:27.837+0000 7ff6da135640 1 -- 172.21.15.143:0/2317213673 wait complete. 2023-11-26T07:21:27.862 DEBUG:teuthology.orchestra.run.smithi143:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.c.asok --format=json session ls 2023-11-26T07:21:28.021 INFO:teuthology.orchestra.run.smithi143.stdout:[{"id":6126,"entity":{"name":{"type":"client","num":6126},"addr":{"type":"any","addr":"172.21.15.143:0","nonce":3879234400}},"state":"open","num_leases":0,"num_caps":7,"request_load_avg":5,"uptime":80.793713272999995,"requests_in_flight":0,"num_completed_requests":2,"num_completed_flushes":1,"reconnecting":false,"recall_caps":{"value":0,"halflife":60},"release_caps":{"value":0,"halflife":60},"recall_caps_throttle":{"value":0,"halflife":1.5},"recall_caps_throttle2o":{"value":0,"halflife":0.5},"session_cache_liveness":{"value":18.25430319037563,"halflife":300},"cap_acquisition":{"value":0,"halflife":30},"last_trim_completed_requests_tid":14,"last_trim_completed_flushes_tid":2,"delegated_inos":[{"start":"0x10000000005","length":500}],"inst":"client.6126 172.21.15.143:0/3879234400","completed_requests":[{"tid":14,"created_ino":"0x0"},{"tid":15,"created_ino":"0x100000001f9"}],"prealloc_inos":[{"start":"0x10000000005","length":500},{"start":"0x100000001fa","length":1000}],"client_metadata":{"client_features":{"feature_bits":"0x00000000003fffff"},"metric_spec":{"metric_flags":{"feature_bits":"0x000000000000ffff"}},"ceph_sha1":"44c2237fe77713214148c1fd622c872f346f1b99","ceph_version":"ceph version 18.0.0-7484-g44c2237f (44c2237fe77713214148c1fd622c872f346f1b99) reef (dev)","entity_id":"y","hostname":"smithi143","pid":"38097","root":"/"}}] 2023-11-26T07:21:28.021 DEBUG:tasks.cephfs.filesystem:_json_asok output [ { "cap_acquisition": { "halflife": 30, "value": 0 }, "client_metadata": { "ceph_sha1": "44c2237fe77713214148c1fd622c872f346f1b99", "ceph_version": "ceph version 18.0.0-7484-g44c2237f (44c2237fe77713214148c1fd622c872f346f1b99) reef (dev)", "client_features": { "feature_bits": "0x00000000003fffff" }, "entity_id": "y", "hostname": "smithi143", "metric_spec": { "metric_flags": { "feature_bits": "0x000000000000ffff" } }, "pid": "38097", "root": "/" }, "completed_requests": [ { "created_ino": "0x0", "tid": 14 }, { "created_ino": "0x100000001f9", "tid": 15 } ], "delegated_inos": [ { "length": 500, "start": "0x10000000005" } ], "entity": { "addr": { "addr": "172.21.15.143:0", "nonce": 3879234400, "type": "any" }, "name": { "num": 6126, "type": "client" } }, "id": 6126, "inst": "client.6126 172.21.15.143:0/3879234400", "last_trim_completed_flushes_tid": 2, "last_trim_completed_requests_tid": 14, "num_caps": 7, "num_completed_flushes": 1, "num_completed_requests": 2, "num_leases": 0, "prealloc_inos": [ { "length": 500, "start": "0x10000000005" }, { "length": 1000, "start": "0x100000001fa" } ], "recall_caps": { "halflife": 60, "value": 0 }, "recall_caps_throttle": { "halflife": 1.5, "value": 0 }, "recall_caps_throttle2o": { "halflife": 0.5, "value": 0 }, "reconnecting": false, "release_caps": { "halflife": 60, "value": 0 }, "request_load_avg": 5, "requests_in_flight": 0, "session_cache_liveness": { "halflife": 300, "value": 18.25430319037563 }, "state": "open", "uptime": 80.793713273 } ] 2023-11-26T07:21:28.021 DEBUG:tasks.ceph_test_case:wait_until_true: waiting (timeout=90 retry_count=0)... 2023-11-26T07:21:33.022 DEBUG:teuthology.orchestra.run.smithi143:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-11-26T07:21:33.170 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:21:33.164+0000 7fd459301640 1 Processor -- start 2023-11-26T07:21:33.171 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:21:33.164+0000 7fd459301640 1 -- start start 2023-11-26T07:21:33.172 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:21:33.164+0000 7fd459301640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fd4540cf280 0x7fd4540cf6e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:21:33.172 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:21:33.164+0000 7fd459301640 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7fd4540cd1b0 0x7fd4540cd5e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:21:33.173 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:21:33.164+0000 7fd459301640 1 -- --> v1:172.21.15.143:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd4540cfc20 con 0x7fd454143ac0 2023-11-26T07:21:33.173 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:21:33.164+0000 7fd459301640 1 -- --> v2:172.21.15.188:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd454143ea0 con 0x7fd4540cd1b0 2023-11-26T07:21:33.173 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:21:33.164+0000 7fd459301640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd4540cc730 con 0x7fd4540cf280 2023-11-26T07:21:33.173 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:21:33.164+0000 7fd452ffd640 1 --1- >> v1:172.21.15.143:6789/0 conn(0x7fd454143ac0 0x7fd454070370 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.143:6789/0 says I am v1:172.21.15.143:46618/0 (socket says 172.21.15.143:46618) 2023-11-26T07:21:33.173 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:21:33.164+0000 7fd452ffd640 1 -- 172.21.15.143:0/2628561188 learned_addr learned my addr 172.21.15.143:0/2628561188 (peer_addr_for_me v1:172.21.15.143:0/0) 2023-11-26T07:21:33.173 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:21:33.164+0000 7fd44a7fc640 1 --2- 172.21.15.143:0/2628561188 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fd4540cf280 0x7fd4540cf6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:21:33.173 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:21:33.164+0000 7fd4537fe640 1 --2- 172.21.15.143:0/2628561188 >> v2:172.21.15.188:3300/0 conn(0x7fd4540cd1b0 0x7fd4540cd5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:21:33.173 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:21:33.168+0000 7fd4527fc640 1 -- 172.21.15.143:0/2628561188 <== mon.0 v1:172.21.15.143:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 435630770 0 0) 0x7fd4540cfc20 con 0x7fd454143ac0 2023-11-26T07:21:33.173 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:21:33.168+0000 7fd4527fc640 1 -- 172.21.15.143:0/2628561188 --> v1:172.21.15.143:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fd440003530 con 0x7fd454143ac0 2023-11-26T07:21:33.174 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:21:33.168+0000 7fd4537fe640 1 -- 172.21.15.143:0/2628561188 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fd4540cf280 msgr2=0x7fd4540cf6e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-26T07:21:33.174 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:21:33.168+0000 7fd4527fc640 1 -- 172.21.15.143:0/2628561188 <== mon.0 v1:172.21.15.143:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3239617047 0 0) 0x7fd440003530 con 0x7fd454143ac0 2023-11-26T07:21:33.174 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:21:33.168+0000 7fd4537fe640 1 --2- 172.21.15.143:0/2628561188 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fd4540cf280 0x7fd4540cf6e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:21:33.174 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:21:33.168+0000 7fd4537fe640 1 -- 172.21.15.143:0/2628561188 >> v1:172.21.15.143:6789/0 conn(0x7fd454143ac0 legacy=0x7fd454070370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:21:33.174 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:21:33.168+0000 7fd4537fe640 1 -- 172.21.15.143:0/2628561188 --> v2:172.21.15.188:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd454070ee0 con 0x7fd4540cd1b0 2023-11-26T07:21:33.175 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:21:33.168+0000 7fd44a7fc640 1 --2- 172.21.15.143:0/2628561188 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fd4540cf280 0x7fd4540cf6e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-26T07:21:33.176 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:21:33.168+0000 7fd4537fe640 1 --2- 172.21.15.143:0/2628561188 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fd4540cd1b0 0x7fd4540cd5e0 secure :-1 s=READY pgs=286 cs=0 l=1 rev1=1 crypto rx=0x7fd44c009d90 tx=0x7fd44c01c870 comp rx=0 tx=0).ready entity=mon.1 client_cookie=8a0680d4b9d0540c server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:21:33.177 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:21:33.168+0000 7fd4527fc640 1 -- 172.21.15.143:0/2628561188 <== mon.1 v2:172.21.15.188:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd43c001680 con 0x7fd4540cd1b0 2023-11-26T07:21:33.177 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:21:33.168+0000 7fd4527fc640 1 -- 172.21.15.143:0/2628561188 <== mon.1 v2:172.21.15.188:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd44c0a5670 con 0x7fd4540cd1b0 2023-11-26T07:21:33.177 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:21:33.168+0000 7fd4527fc640 1 -- 172.21.15.143:0/2628561188 <== mon.1 v2:172.21.15.188:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd44c0a59e0 con 0x7fd4540cd1b0 2023-11-26T07:21:33.177 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:21:33.172+0000 7fd459301640 1 -- 172.21.15.143:0/2628561188 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fd4540cd1b0 msgr2=0x7fd4540cd5e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:21:33.177 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:21:33.172+0000 7fd459301640 1 --2- 172.21.15.143:0/2628561188 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fd4540cd1b0 0x7fd4540cd5e0 secure :-1 s=READY pgs=286 cs=0 l=1 rev1=1 crypto rx=0x7fd44c009d90 tx=0x7fd44c01c870 comp rx=0 tx=0).stop 2023-11-26T07:21:33.178 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:21:33.172+0000 7fd459301640 1 -- 172.21.15.143:0/2628561188 shutdown_connections 2023-11-26T07:21:33.178 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:21:33.172+0000 7fd459301640 1 --2- 172.21.15.143:0/2628561188 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fd4540cd1b0 0x7fd4540cd5e0 unknown :-1 s=CLOSED pgs=286 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:21:33.178 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:21:33.172+0000 7fd459301640 1 --2- 172.21.15.143:0/2628561188 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fd4540cf280 0x7fd4540cf6e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:21:33.178 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:21:33.172+0000 7fd459301640 1 -- 172.21.15.143:0/2628561188 >> 172.21.15.143:0/2628561188 conn(0x7fd45406a890 msgr2=0x7fd45406aca0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:21:33.178 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:21:33.172+0000 7fd459301640 1 -- 172.21.15.143:0/2628561188 shutdown_connections 2023-11-26T07:21:33.178 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:21:33.172+0000 7fd459301640 1 -- 172.21.15.143:0/2628561188 wait complete. 2023-11-26T07:21:33.179 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:21:33.172+0000 7fd459301640 1 Processor -- start 2023-11-26T07:21:33.179 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:21:33.172+0000 7fd459301640 1 -- start start 2023-11-26T07:21:33.181 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:21:33.172+0000 7fd459301640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fd4540cd1b0 0x7fd4541596b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:21:33.181 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:21:33.172+0000 7fd459301640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fd4540cf280 0x7fd45413c7e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:21:33.182 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:21:33.172+0000 7fd459301640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fd454143ac0 0x7fd45413dff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:21:33.182 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:21:33.172+0000 7fd459301640 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd45413cd20 con 0x7fd4540cf280 2023-11-26T07:21:33.182 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:21:33.172+0000 7fd459301640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd4541f9ea0 con 0x7fd4540cd1b0 2023-11-26T07:21:33.182 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:21:33.172+0000 7fd459301640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd45413cf50 con 0x7fd454143ac0 2023-11-26T07:21:33.182 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:21:33.172+0000 7fd44a7fc640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fd4540cf280 0x7fd45413c7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:21:33.182 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:21:33.172+0000 7fd44a7fc640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fd4540cf280 0x7fd45413c7e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:34890/0 (socket says 172.21.15.143:34890) 2023-11-26T07:21:33.182 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:21:33.172+0000 7fd44a7fc640 1 -- 172.21.15.143:0/217504325 learned_addr learned my addr 172.21.15.143:0/217504325 (peer_addr_for_me v2:172.21.15.143:0/0) 2023-11-26T07:21:33.182 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:21:33.176+0000 7fd452ffd640 1 --2- 172.21.15.143:0/217504325 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fd4540cd1b0 0x7fd4541596b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:21:33.182 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:21:33.176+0000 7fd4537fe640 1 --2- 172.21.15.143:0/217504325 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fd454143ac0 0x7fd45413dff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:21:33.183 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:21:33.176+0000 7fd44a7fc640 1 -- 172.21.15.143:0/217504325 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fd454143ac0 msgr2=0x7fd45413dff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:21:33.183 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:21:33.176+0000 7fd44a7fc640 1 --2- 172.21.15.143:0/217504325 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fd454143ac0 0x7fd45413dff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:21:33.183 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:21:33.176+0000 7fd44a7fc640 1 -- 172.21.15.143:0/217504325 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fd4540cd1b0 msgr2=0x7fd4541596b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:21:33.183 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:21:33.176+0000 7fd44a7fc640 1 --2- 172.21.15.143:0/217504325 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fd4540cd1b0 0x7fd4541596b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:21:33.183 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:21:33.176+0000 7fd44a7fc640 1 -- 172.21.15.143:0/217504325 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd444009420 con 0x7fd4540cf280 2023-11-26T07:21:33.183 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:21:33.176+0000 7fd4537fe640 1 --2- 172.21.15.143:0/217504325 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fd454143ac0 0x7fd45413dff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-26T07:21:33.183 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:21:33.176+0000 7fd44a7fc640 1 --2- 172.21.15.143:0/217504325 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fd4540cf280 0x7fd45413c7e0 secure :-1 s=READY pgs=714 cs=0 l=1 rev1=1 crypto rx=0x7fd44400ebe0 tx=0x7fd44400c520 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:21:33.183 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:21:33.176+0000 7fd44bfff640 1 -- 172.21.15.143:0/217504325 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd44400eee0 con 0x7fd4540cf280 2023-11-26T07:21:33.183 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:21:33.176+0000 7fd44bfff640 1 -- 172.21.15.143:0/217504325 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd44400cc30 con 0x7fd4540cf280 2023-11-26T07:21:33.184 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:21:33.176+0000 7fd459301640 1 -- 172.21.15.143:0/217504325 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd44c009a70 con 0x7fd4540cf280 2023-11-26T07:21:33.184 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:21:33.176+0000 7fd44bfff640 1 -- 172.21.15.143:0/217504325 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd444013470 con 0x7fd4540cf280 2023-11-26T07:21:33.184 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:21:33.176+0000 7fd459301640 1 -- 172.21.15.143:0/217504325 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd45413d510 con 0x7fd4540cf280 2023-11-26T07:21:33.185 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:21:33.176+0000 7fd459301640 1 -- 172.21.15.143:0/217504325 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd454067680 con 0x7fd4540cf280 2023-11-26T07:21:33.185 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:21:33.176+0000 7fd44bfff640 1 -- 172.21.15.143:0/217504325 <== mon.0 v2:172.21.15.143:3300/0 4 ==== mgrmap(e 8) v1 ==== 86056+0+0 (secure 0 0 0) 0x7fd4440135d0 con 0x7fd4540cf280 2023-11-26T07:21:33.185 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:21:33.176+0000 7fd44bfff640 1 --2- 172.21.15.143:0/217504325 >> [v2:172.21.15.143:6832/2150557544,v1:172.21.15.143:6833/2150557544] conn(0x7fd4340666b0 0x7fd434068b70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:21:33.185 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:21:33.176+0000 7fd44bfff640 1 -- 172.21.15.143:0/217504325 <== mon.0 v2:172.21.15.143:3300/0 5 ==== osd_map(47..47 src has 1..47) v4 ==== 5767+0+0 (secure 0 0 0) 0x7fd4440846e0 con 0x7fd4540cf280 2023-11-26T07:21:33.185 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:21:33.180+0000 7fd452ffd640 1 --2- 172.21.15.143:0/217504325 >> [v2:172.21.15.143:6832/2150557544,v1:172.21.15.143:6833/2150557544] conn(0x7fd4340666b0 0x7fd434068b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:21:33.185 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:21:33.180+0000 7fd452ffd640 1 --2- 172.21.15.143:0/217504325 >> [v2:172.21.15.143:6832/2150557544,v1:172.21.15.143:6833/2150557544] conn(0x7fd4340666b0 0x7fd434068b70 secure :-1 s=READY pgs=342 cs=0 l=1 rev1=1 crypto rx=0x7fd43c0011f0 tx=0x7fd43c009040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:21:33.188 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:21:33.180+0000 7fd44bfff640 1 -- 172.21.15.143:0/217504325 <== mon.0 v2:172.21.15.143:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172649 (secure 0 0 0) 0x7fd444051690 con 0x7fd4540cf280 2023-11-26T07:21:33.371 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:21:33.364+0000 7fd459301640 1 -- 172.21.15.143:0/217504325 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fd45413f870 con 0x7fd4540cf280 2023-11-26T07:21:33.371 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:21:33.364+0000 7fd44bfff640 1 -- 172.21.15.143:0/217504325 <== mon.0 v2:172.21.15.143:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 26 v26) v1 ==== 94+0+4922 (secure 0 0 0) 0x7fd444054cb0 con 0x7fd4540cf280 2023-11-26T07:21:33.372 INFO:teuthology.orchestra.run.smithi143.stdout: 2023-11-26T07:21:33.372 INFO:teuthology.orchestra.run.smithi143.stdout:{"epoch":26,"default_fscid":3,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5201,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.188:6835/319464848","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6833","nonce":319464848},{"type":"v1","addr":"172.21.15.188:6835","nonce":319464848}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":20},{"gid":5584,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.188:6834/2113408070","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6832","nonce":2113408070},{"type":"v1","addr":"172.21.15.188:6834","nonce":2113408070}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":20},{"gid":5745,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.143:6835/1616742593","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6834","nonce":1616742593},{"type":"v1","addr":"172.21.15.143:6835","nonce":1616742593}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":20}],"filesystems":[{"mdsmap":{"epoch":25,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-11-26T07:19:31.503587+0000","modified":"2023-11-26T07:19:34.523605+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":5748},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5748":{"gid":5748,"name":"c","rank":0,"incarnation":21,"state":"up:active","state_seq":3,"addr":"172.21.15.143:6837/2872033338","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6836","nonce":2872033338},{"type":"v1","addr":"172.21.15.143:6837","nonce":2872033338}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[7],"metadata_pool":6,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":3}]} 2023-11-26T07:21:33.372 INFO:teuthology.orchestra.run.smithi143.stderr:dumped fsmap epoch 26 2023-11-26T07:21:33.374 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:21:33.368+0000 7fd459301640 1 -- 172.21.15.143:0/217504325 >> [v2:172.21.15.143:6832/2150557544,v1:172.21.15.143:6833/2150557544] conn(0x7fd4340666b0 msgr2=0x7fd434068b70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:21:33.374 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:21:33.368+0000 7fd459301640 1 --2- 172.21.15.143:0/217504325 >> [v2:172.21.15.143:6832/2150557544,v1:172.21.15.143:6833/2150557544] conn(0x7fd4340666b0 0x7fd434068b70 secure :-1 s=READY pgs=342 cs=0 l=1 rev1=1 crypto rx=0x7fd43c0011f0 tx=0x7fd43c009040 comp rx=0 tx=0).stop 2023-11-26T07:21:33.374 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:21:33.368+0000 7fd459301640 1 -- 172.21.15.143:0/217504325 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fd4540cf280 msgr2=0x7fd45413c7e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:21:33.374 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:21:33.368+0000 7fd459301640 1 --2- 172.21.15.143:0/217504325 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fd4540cf280 0x7fd45413c7e0 secure :-1 s=READY pgs=714 cs=0 l=1 rev1=1 crypto rx=0x7fd44400ebe0 tx=0x7fd44400c520 comp rx=0 tx=0).stop 2023-11-26T07:21:33.374 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:21:33.368+0000 7fd459301640 1 -- 172.21.15.143:0/217504325 shutdown_connections 2023-11-26T07:21:33.374 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:21:33.368+0000 7fd459301640 1 --2- 172.21.15.143:0/217504325 >> [v2:172.21.15.143:6832/2150557544,v1:172.21.15.143:6833/2150557544] conn(0x7fd4340666b0 0x7fd434068b70 unknown :-1 s=CLOSED pgs=342 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:21:33.374 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:21:33.368+0000 7fd459301640 1 --2- 172.21.15.143:0/217504325 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fd454143ac0 0x7fd45413dff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:21:33.374 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:21:33.368+0000 7fd459301640 1 --2- 172.21.15.143:0/217504325 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fd4540cf280 0x7fd45413c7e0 unknown :-1 s=CLOSED pgs=714 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:21:33.374 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:21:33.368+0000 7fd459301640 1 --2- 172.21.15.143:0/217504325 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fd4540cd1b0 0x7fd4541596b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:21:33.374 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:21:33.368+0000 7fd459301640 1 -- 172.21.15.143:0/217504325 >> 172.21.15.143:0/217504325 conn(0x7fd45406a890 msgr2=0x7fd454070370 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:21:33.375 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:21:33.368+0000 7fd459301640 1 -- 172.21.15.143:0/217504325 shutdown_connections 2023-11-26T07:21:33.375 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:21:33.368+0000 7fd459301640 1 -- 172.21.15.143:0/217504325 wait complete. 2023-11-26T07:21:33.393 DEBUG:teuthology.orchestra.run.smithi143:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.c.asok --format=json session ls 2023-11-26T07:21:33.551 INFO:teuthology.orchestra.run.smithi143.stdout:[{"id":6126,"entity":{"name":{"type":"client","num":6126},"addr":{"type":"any","addr":"172.21.15.143:0","nonce":3879234400}},"state":"open","num_leases":0,"num_caps":7,"request_load_avg":5,"uptime":86.321532783999999,"requests_in_flight":0,"num_completed_requests":2,"num_completed_flushes":1,"reconnecting":false,"recall_caps":{"value":0,"halflife":60},"release_caps":{"value":0,"halflife":60},"recall_caps_throttle":{"value":0,"halflife":1.5},"recall_caps_throttle2o":{"value":0,"halflife":0.5},"session_cache_liveness":{"value":18.022642219028185,"halflife":300},"cap_acquisition":{"value":0,"halflife":30},"last_trim_completed_requests_tid":14,"last_trim_completed_flushes_tid":2,"delegated_inos":[{"start":"0x10000000005","length":500}],"inst":"client.6126 172.21.15.143:0/3879234400","completed_requests":[{"tid":14,"created_ino":"0x0"},{"tid":15,"created_ino":"0x100000001f9"}],"prealloc_inos":[{"start":"0x10000000005","length":500},{"start":"0x100000001fa","length":1000}],"client_metadata":{"client_features":{"feature_bits":"0x00000000003fffff"},"metric_spec":{"metric_flags":{"feature_bits":"0x000000000000ffff"}},"ceph_sha1":"44c2237fe77713214148c1fd622c872f346f1b99","ceph_version":"ceph version 18.0.0-7484-g44c2237f (44c2237fe77713214148c1fd622c872f346f1b99) reef (dev)","entity_id":"y","hostname":"smithi143","pid":"38097","root":"/"}}] 2023-11-26T07:21:33.551 DEBUG:tasks.cephfs.filesystem:_json_asok output [ { "cap_acquisition": { "halflife": 30, "value": 0 }, "client_metadata": { "ceph_sha1": "44c2237fe77713214148c1fd622c872f346f1b99", "ceph_version": "ceph version 18.0.0-7484-g44c2237f (44c2237fe77713214148c1fd622c872f346f1b99) reef (dev)", "client_features": { "feature_bits": "0x00000000003fffff" }, "entity_id": "y", "hostname": "smithi143", "metric_spec": { "metric_flags": { "feature_bits": "0x000000000000ffff" } }, "pid": "38097", "root": "/" }, "completed_requests": [ { "created_ino": "0x0", "tid": 14 }, { "created_ino": "0x100000001f9", "tid": 15 } ], "delegated_inos": [ { "length": 500, "start": "0x10000000005" } ], "entity": { "addr": { "addr": "172.21.15.143:0", "nonce": 3879234400, "type": "any" }, "name": { "num": 6126, "type": "client" } }, "id": 6126, "inst": "client.6126 172.21.15.143:0/3879234400", "last_trim_completed_flushes_tid": 2, "last_trim_completed_requests_tid": 14, "num_caps": 7, "num_completed_flushes": 1, "num_completed_requests": 2, "num_leases": 0, "prealloc_inos": [ { "length": 500, "start": "0x10000000005" }, { "length": 1000, "start": "0x100000001fa" } ], "recall_caps": { "halflife": 60, "value": 0 }, "recall_caps_throttle": { "halflife": 1.5, "value": 0 }, "recall_caps_throttle2o": { "halflife": 0.5, "value": 0 }, "reconnecting": false, "release_caps": { "halflife": 60, "value": 0 }, "request_load_avg": 5, "requests_in_flight": 0, "session_cache_liveness": { "halflife": 300, "value": 18.022642219028185 }, "state": "open", "uptime": 86.321532784 } ] 2023-11-26T07:21:33.552 DEBUG:tasks.ceph_test_case:wait_until_true: waiting (timeout=90 retry_count=0)... 2023-11-26T07:21:38.553 DEBUG:teuthology.orchestra.run.smithi143:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-11-26T07:21:38.702 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:21:38.700+0000 7ff0b9555640 1 Processor -- start 2023-11-26T07:21:38.702 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:21:38.700+0000 7ff0b9555640 1 -- start start 2023-11-26T07:21:38.703 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:21:38.700+0000 7ff0b9555640 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7ff0b40d1bc0 0x7ff0b40d0df0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:21:38.703 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:21:38.700+0000 7ff0b9555640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7ff0b40b2160 0x7ff0b40b0750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:21:38.704 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:21:38.700+0000 7ff0b9555640 1 -- --> v1:172.21.15.188:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff0b40cf0e0 con 0x7ff0b415d6e0 2023-11-26T07:21:38.704 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:21:38.700+0000 7ff0b9555640 1 -- --> v2:172.21.15.143:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff0b415dac0 con 0x7ff0b40d1bc0 2023-11-26T07:21:38.704 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:21:38.700+0000 7ff0b9555640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff0b40ceb20 con 0x7ff0b40b2160 2023-11-26T07:21:38.704 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:21:38.700+0000 7ff0aa7fc640 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7ff0b40d1bc0 0x7ff0b40d0df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:21:38.704 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:21:38.700+0000 7ff0aa7fc640 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7ff0b40d1bc0 0x7ff0b40d0df0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:35670/0 (socket says 172.21.15.143:35670) 2023-11-26T07:21:38.704 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:21:38.700+0000 7ff0aa7fc640 1 -- 172.21.15.143:0/2198925222 learned_addr learned my addr 172.21.15.143:0/2198925222 (peer_addr_for_me v2:172.21.15.143:0/0) 2023-11-26T07:21:38.704 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:21:38.700+0000 7ff0b37fe640 1 --2- 172.21.15.143:0/2198925222 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7ff0b40b2160 0x7ff0b40b0750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:21:38.704 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:21:38.700+0000 7ff0aa7fc640 1 -- 172.21.15.143:0/2198925222 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7ff0b40b2160 msgr2=0x7ff0b40b0750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-26T07:21:38.704 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:21:38.700+0000 7ff0aa7fc640 1 --2- 172.21.15.143:0/2198925222 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7ff0b40b2160 0x7ff0b40b0750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:21:38.704 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:21:38.700+0000 7ff0aa7fc640 1 -- 172.21.15.143:0/2198925222 >> v1:172.21.15.188:6789/0 conn(0x7ff0b415d6e0 legacy=0x7ff0b40d14b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-26T07:21:38.704 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:21:38.700+0000 7ff0aa7fc640 1 -- 172.21.15.143:0/2198925222 --> v2:172.21.15.143:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff0b40cee00 con 0x7ff0b40d1bc0 2023-11-26T07:21:38.706 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:21:38.704+0000 7ff0aa7fc640 1 --2- 172.21.15.143:0/2198925222 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7ff0b40d1bc0 0x7ff0b40d0df0 secure :-1 s=READY pgs=715 cs=0 l=1 rev1=1 crypto rx=0x7ff09c009e50 tx=0x7ff09c01c9d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e579f2583c46d2f5 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:21:38.707 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:21:38.704+0000 7ff0b27fc640 1 -- 172.21.15.143:0/2198925222 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff09c0a5740 con 0x7ff0b40d1bc0 2023-11-26T07:21:38.707 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:21:38.704+0000 7ff0b27fc640 1 -- 172.21.15.143:0/2198925222 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff09c0a58a0 con 0x7ff0b40d1bc0 2023-11-26T07:21:38.707 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:21:38.704+0000 7ff0b27fc640 1 -- 172.21.15.143:0/2198925222 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff09c0a5c30 con 0x7ff0b40d1bc0 2023-11-26T07:21:38.708 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:21:38.704+0000 7ff0b9555640 1 -- 172.21.15.143:0/2198925222 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7ff0b40d1bc0 msgr2=0x7ff0b40d0df0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:21:38.708 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:21:38.704+0000 7ff0b9555640 1 --2- 172.21.15.143:0/2198925222 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7ff0b40d1bc0 0x7ff0b40d0df0 secure :-1 s=READY pgs=715 cs=0 l=1 rev1=1 crypto rx=0x7ff09c009e50 tx=0x7ff09c01c9d0 comp rx=0 tx=0).stop 2023-11-26T07:21:38.708 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:21:38.704+0000 7ff0b9555640 1 -- 172.21.15.143:0/2198925222 shutdown_connections 2023-11-26T07:21:38.708 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:21:38.704+0000 7ff0b9555640 1 --2- 172.21.15.143:0/2198925222 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7ff0b40d1bc0 0x7ff0b40d0df0 unknown :-1 s=CLOSED pgs=715 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:21:38.708 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:21:38.704+0000 7ff0b9555640 1 --2- 172.21.15.143:0/2198925222 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7ff0b40b2160 0x7ff0b40b0750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:21:38.708 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:21:38.704+0000 7ff0b9555640 1 -- 172.21.15.143:0/2198925222 >> 172.21.15.143:0/2198925222 conn(0x7ff0b409c550 msgr2=0x7ff0b40d3220 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:21:38.708 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:21:38.704+0000 7ff0b9555640 1 -- 172.21.15.143:0/2198925222 shutdown_connections 2023-11-26T07:21:38.708 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:21:38.704+0000 7ff0b9555640 1 -- 172.21.15.143:0/2198925222 wait complete. 2023-11-26T07:21:38.709 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:21:38.708+0000 7ff0b9555640 1 Processor -- start 2023-11-26T07:21:38.709 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:21:38.708+0000 7ff0b9555640 1 -- start start 2023-11-26T07:21:38.711 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:21:38.708+0000 7ff0b9555640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7ff0b40b2160 0x7ff0b415dc00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:21:38.711 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:21:38.708+0000 7ff0b9555640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7ff0b40d1bc0 0x7ff0b40c6360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:21:38.711 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:21:38.708+0000 7ff0b9555640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7ff0b415d6e0 0x7ff0b40b3f10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:21:38.711 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:21:38.708+0000 7ff0b9555640 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff0b415e290 con 0x7ff0b40b2160 2023-11-26T07:21:38.711 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:21:38.708+0000 7ff0b9555640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff0b40b5ee0 con 0x7ff0b415d6e0 2023-11-26T07:21:38.711 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:21:38.708+0000 7ff0b9555640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff0b40b61c0 con 0x7ff0b40d1bc0 2023-11-26T07:21:38.711 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:21:38.708+0000 7ff0b2ffd640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7ff0b40b2160 0x7ff0b415dc00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:21:38.712 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:21:38.708+0000 7ff0b2ffd640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7ff0b40b2160 0x7ff0b415dc00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:35684/0 (socket says 172.21.15.143:35684) 2023-11-26T07:21:38.712 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:21:38.708+0000 7ff0b2ffd640 1 -- 172.21.15.143:0/683894170 learned_addr learned my addr 172.21.15.143:0/683894170 (peer_addr_for_me v2:172.21.15.143:0/0) 2023-11-26T07:21:38.712 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:21:38.708+0000 7ff0b37fe640 1 --2- 172.21.15.143:0/683894170 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7ff0b415d6e0 0x7ff0b40b3f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:21:38.712 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:21:38.708+0000 7ff0aa7fc640 1 --2- 172.21.15.143:0/683894170 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7ff0b40d1bc0 0x7ff0b40c6360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:21:38.712 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:21:38.708+0000 7ff0b2ffd640 1 -- 172.21.15.143:0/683894170 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7ff0b40d1bc0 msgr2=0x7ff0b40c6360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:21:38.712 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:21:38.708+0000 7ff0b2ffd640 1 --2- 172.21.15.143:0/683894170 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7ff0b40d1bc0 0x7ff0b40c6360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:21:38.712 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:21:38.708+0000 7ff0b2ffd640 1 -- 172.21.15.143:0/683894170 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7ff0b415d6e0 msgr2=0x7ff0b40b3f10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:21:38.712 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:21:38.708+0000 7ff0b2ffd640 1 --2- 172.21.15.143:0/683894170 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7ff0b415d6e0 0x7ff0b40b3f10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:21:38.712 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:21:38.708+0000 7ff0b2ffd640 1 -- 172.21.15.143:0/683894170 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff09c009b30 con 0x7ff0b40b2160 2023-11-26T07:21:38.712 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:21:38.708+0000 7ff0b2ffd640 1 --2- 172.21.15.143:0/683894170 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7ff0b40b2160 0x7ff0b415dc00 secure :-1 s=READY pgs=716 cs=0 l=1 rev1=1 crypto rx=0x7ff0a400b410 tx=0x7ff0a400b8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:21:38.713 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:21:38.708+0000 7ff0abfff640 1 -- 172.21.15.143:0/683894170 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff0a4004020 con 0x7ff0b40b2160 2023-11-26T07:21:38.713 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:21:38.708+0000 7ff0abfff640 1 -- 172.21.15.143:0/683894170 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff0a4004180 con 0x7ff0b40b2160 2023-11-26T07:21:38.713 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:21:38.708+0000 7ff0b9555640 1 -- 172.21.15.143:0/683894170 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff0b40b64a0 con 0x7ff0b40b2160 2023-11-26T07:21:38.713 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:21:38.708+0000 7ff0abfff640 1 -- 172.21.15.143:0/683894170 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff0a4011630 con 0x7ff0b40b2160 2023-11-26T07:21:38.713 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:21:38.708+0000 7ff0b9555640 1 -- 172.21.15.143:0/683894170 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff0b4160ad0 con 0x7ff0b40b2160 2023-11-26T07:21:38.713 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:21:38.712+0000 7ff0b9555640 1 -- 172.21.15.143:0/683894170 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff0b40b69c0 con 0x7ff0b40b2160 2023-11-26T07:21:38.714 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:21:38.712+0000 7ff0abfff640 1 -- 172.21.15.143:0/683894170 <== mon.0 v2:172.21.15.143:3300/0 4 ==== mgrmap(e 9) v1 ==== 86005+0+0 (secure 0 0 0) 0x7ff0a4011790 con 0x7ff0b40b2160 2023-11-26T07:21:38.714 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:21:38.712+0000 7ff0abfff640 1 --2- 172.21.15.143:0/683894170 >> [v2:172.21.15.143:6832/2150557544,v1:172.21.15.143:6833/2150557544] conn(0x7ff094066620 0x7ff094068ae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:21:38.714 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:21:38.712+0000 7ff0abfff640 1 -- 172.21.15.143:0/683894170 <== mon.0 v2:172.21.15.143:3300/0 5 ==== osd_map(47..47 src has 1..47) v4 ==== 5767+0+0 (secure 0 0 0) 0x7ff0a4083df0 con 0x7ff0b40b2160 2023-11-26T07:21:38.715 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:21:38.712+0000 7ff0aa7fc640 1 --2- 172.21.15.143:0/683894170 >> [v2:172.21.15.143:6832/2150557544,v1:172.21.15.143:6833/2150557544] conn(0x7ff094066620 0x7ff094068ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:21:38.715 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:21:38.712+0000 7ff0aa7fc640 1 --2- 172.21.15.143:0/683894170 >> [v2:172.21.15.143:6832/2150557544,v1:172.21.15.143:6833/2150557544] conn(0x7ff094066620 0x7ff094068ae0 secure :-1 s=READY pgs=343 cs=0 l=1 rev1=1 crypto rx=0x7ff09c0ae000 tx=0x7ff09c0a7070 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:21:38.718 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:21:38.716+0000 7ff0abfff640 1 -- 172.21.15.143:0/683894170 <== mon.0 v2:172.21.15.143:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172649 (secure 0 0 0) 0x7ff0b40b69c0 con 0x7ff0b40b2160 2023-11-26T07:21:38.897 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:21:38.896+0000 7ff0b9555640 1 -- 172.21.15.143:0/683894170 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7ff0b40b69c0 con 0x7ff0b40b2160 2023-11-26T07:21:38.897 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:21:38.896+0000 7ff0abfff640 1 -- 172.21.15.143:0/683894170 <== mon.0 v2:172.21.15.143:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 26 v26) v1 ==== 94+0+4922 (secure 0 0 0) 0x7ff0b40b69c0 con 0x7ff0b40b2160 2023-11-26T07:21:38.898 INFO:teuthology.orchestra.run.smithi143.stdout: 2023-11-26T07:21:38.898 INFO:teuthology.orchestra.run.smithi143.stdout:{"epoch":26,"default_fscid":3,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5201,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.188:6835/319464848","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6833","nonce":319464848},{"type":"v1","addr":"172.21.15.188:6835","nonce":319464848}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":20},{"gid":5584,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.188:6834/2113408070","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6832","nonce":2113408070},{"type":"v1","addr":"172.21.15.188:6834","nonce":2113408070}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":20},{"gid":5745,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.143:6835/1616742593","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6834","nonce":1616742593},{"type":"v1","addr":"172.21.15.143:6835","nonce":1616742593}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":20}],"filesystems":[{"mdsmap":{"epoch":25,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-11-26T07:19:31.503587+0000","modified":"2023-11-26T07:19:34.523605+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":5748},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5748":{"gid":5748,"name":"c","rank":0,"incarnation":21,"state":"up:active","state_seq":3,"addr":"172.21.15.143:6837/2872033338","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6836","nonce":2872033338},{"type":"v1","addr":"172.21.15.143:6837","nonce":2872033338}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[7],"metadata_pool":6,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":3}]} 2023-11-26T07:21:38.898 INFO:teuthology.orchestra.run.smithi143.stderr:dumped fsmap epoch 26 2023-11-26T07:21:38.900 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:21:38.896+0000 7ff0b9555640 1 -- 172.21.15.143:0/683894170 >> [v2:172.21.15.143:6832/2150557544,v1:172.21.15.143:6833/2150557544] conn(0x7ff094066620 msgr2=0x7ff094068ae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:21:38.900 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:21:38.896+0000 7ff0b9555640 1 --2- 172.21.15.143:0/683894170 >> [v2:172.21.15.143:6832/2150557544,v1:172.21.15.143:6833/2150557544] conn(0x7ff094066620 0x7ff094068ae0 secure :-1 s=READY pgs=343 cs=0 l=1 rev1=1 crypto rx=0x7ff09c0ae000 tx=0x7ff09c0a7070 comp rx=0 tx=0).stop 2023-11-26T07:21:38.900 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:21:38.896+0000 7ff0b9555640 1 -- 172.21.15.143:0/683894170 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7ff0b40b2160 msgr2=0x7ff0b415dc00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:21:38.900 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:21:38.896+0000 7ff0b9555640 1 --2- 172.21.15.143:0/683894170 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7ff0b40b2160 0x7ff0b415dc00 secure :-1 s=READY pgs=716 cs=0 l=1 rev1=1 crypto rx=0x7ff0a400b410 tx=0x7ff0a400b8d0 comp rx=0 tx=0).stop 2023-11-26T07:21:38.900 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:21:38.896+0000 7ff0b9555640 1 -- 172.21.15.143:0/683894170 shutdown_connections 2023-11-26T07:21:38.900 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:21:38.896+0000 7ff0b9555640 1 --2- 172.21.15.143:0/683894170 >> [v2:172.21.15.143:6832/2150557544,v1:172.21.15.143:6833/2150557544] conn(0x7ff094066620 0x7ff094068ae0 unknown :-1 s=CLOSED pgs=343 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:21:38.900 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:21:38.896+0000 7ff0b9555640 1 --2- 172.21.15.143:0/683894170 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7ff0b415d6e0 0x7ff0b40b3f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:21:38.900 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:21:38.896+0000 7ff0b9555640 1 --2- 172.21.15.143:0/683894170 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7ff0b40d1bc0 0x7ff0b40c6360 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:21:38.900 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:21:38.896+0000 7ff0b9555640 1 --2- 172.21.15.143:0/683894170 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7ff0b40b2160 0x7ff0b415dc00 unknown :-1 s=CLOSED pgs=716 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:21:38.901 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:21:38.896+0000 7ff0b9555640 1 -- 172.21.15.143:0/683894170 >> 172.21.15.143:0/683894170 conn(0x7ff0b409c550 msgr2=0x7ff0b40d3220 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:21:38.901 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:21:38.896+0000 7ff0b9555640 1 -- 172.21.15.143:0/683894170 shutdown_connections 2023-11-26T07:21:38.901 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:21:38.896+0000 7ff0b9555640 1 -- 172.21.15.143:0/683894170 wait complete. 2023-11-26T07:21:38.919 DEBUG:teuthology.orchestra.run.smithi143:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.c.asok --format=json session ls 2023-11-26T07:21:39.077 INFO:teuthology.orchestra.run.smithi143.stdout:[] 2023-11-26T07:21:39.078 DEBUG:tasks.cephfs.filesystem:_json_asok output [] 2023-11-26T07:21:39.078 DEBUG:tasks.ceph_test_case:wait_until_true: success in 80s and 0 retries 2023-11-26T07:21:39.078 INFO:teuthology.nuke.actions:Clearing teuthology firewall rules... 2023-11-26T07:21:39.078 DEBUG:teuthology.orchestra.run.smithi143:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2023-11-26T07:21:39.131 DEBUG:teuthology.orchestra.run.smithi188:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2023-11-26T07:21:39.223 INFO:teuthology.nuke.actions:Cleared teuthology firewall rules. 2023-11-26T07:21:39.223 DEBUG:teuthology.orchestra.run.smithi143:> set -ex 2023-11-26T07:21:39.223 DEBUG:teuthology.orchestra.run.smithi143:> dd if=/proc/self/mounts of=/dev/stdout 2023-11-26T07:21:39.232 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,relatime,size=16354672k,nr_inodes=4088668,mode=755 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3275384k,mode=755 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=3455 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysusers.service ramfs ro,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3275380k,nr_inodes=818845,mode=700,uid=1000,gid=1271 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3275384k,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2023-11-26T07:21:39.232 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi143.front.sepia.ceph.com 2023-11-26T07:21:39.232 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-26T07:21:39.233 DEBUG:teuthology.orchestra.run.smithi143:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2023-11-26T07:21:39.283 INFO:teuthology.orchestra.run.smithi143.stderr:rmdir: failed to remove '/home/ubuntu/cephtest/mnt.0': No such file or directory 2023-11-26T07:21:39.284 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-11-26T07:21:39.285 DEBUG:teuthology.orchestra.run.smithi188:> set -ex 2023-11-26T07:21:39.285 DEBUG:teuthology.orchestra.run.smithi188:> dd if=/proc/self/mounts of=/dev/stdout 2023-11-26T07:21:39.294 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,relatime,size=16354628k,nr_inodes=4088657,mode=755 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3275376k,mode=755 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=787 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysusers.service ramfs ro,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3275372k,nr_inodes=818843,mode=700,uid=1000,gid=1271 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3275376k,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2023-11-26T07:21:39.295 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi188.front.sepia.ceph.com 2023-11-26T07:21:39.295 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-26T07:21:39.295 DEBUG:teuthology.orchestra.run.smithi188:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.1) 2023-11-26T07:21:39.345 INFO:teuthology.orchestra.run.smithi188.stderr:rmdir: failed to remove '/home/ubuntu/cephtest/mnt.1': No such file or directory 2023-11-26T07:21:39.345 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-11-26T07:21:39.346 DEBUG:teuthology.orchestra.run.smithi143:> set -ex 2023-11-26T07:21:39.346 DEBUG:teuthology.orchestra.run.smithi143:> dd if=/proc/self/mounts of=/dev/stdout 2023-11-26T07:21:39.355 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,relatime,size=16354672k,nr_inodes=4088668,mode=755 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3275384k,mode=755 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=3455 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysusers.service ramfs ro,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3275380k,nr_inodes=818845,mode=700,uid=1000,gid=1271 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3275384k,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2023-11-26T07:21:39.355 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi143.front.sepia.ceph.com 2023-11-26T07:21:39.355 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-26T07:21:39.355 DEBUG:teuthology.orchestra.run.smithi143:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.2) 2023-11-26T07:21:39.404 INFO:teuthology.orchestra.run.smithi143.stderr:rmdir: failed to remove '/home/ubuntu/cephtest/mnt.2': No such file or directory 2023-11-26T07:21:39.404 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-11-26T07:21:39.405 DEBUG:teuthology.orchestra.run.smithi188:> set -ex 2023-11-26T07:21:39.405 DEBUG:teuthology.orchestra.run.smithi188:> dd if=/proc/self/mounts of=/dev/stdout 2023-11-26T07:21:39.414 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,relatime,size=16354628k,nr_inodes=4088657,mode=755 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3275376k,mode=755 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=787 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysusers.service ramfs ro,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3275372k,nr_inodes=818843,mode=700,uid=1000,gid=1271 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3275376k,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2023-11-26T07:21:39.414 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi188.front.sepia.ceph.com 2023-11-26T07:21:39.415 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-26T07:21:39.415 DEBUG:teuthology.orchestra.run.smithi188:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.3) 2023-11-26T07:21:39.465 INFO:teuthology.orchestra.run.smithi188.stderr:rmdir: failed to remove '/home/ubuntu/cephtest/mnt.3': No such file or directory 2023-11-26T07:21:39.465 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-11-26T07:21:39.466 DEBUG:teuthology.orchestra.run.smithi143:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-11-26T07:21:39.602 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:21:39.600+0000 7fe046557640 1 Processor -- start 2023-11-26T07:21:39.603 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:21:39.600+0000 7fe046557640 1 -- start start 2023-11-26T07:21:39.604 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:21:39.600+0000 7fe046557640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fe0401596d0 0x7fe040071850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:21:39.604 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:21:39.600+0000 7fe046557640 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7fe0400ce680 0x7fe0400b2870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:21:39.604 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:21:39.600+0000 7fe046557640 1 -- --> v1:172.21.15.188:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe04006c750 con 0x7fe0400b2db0 2023-11-26T07:21:39.604 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:21:39.600+0000 7fe046557640 1 -- --> v2:172.21.15.188:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe0400b3240 con 0x7fe0400ce680 2023-11-26T07:21:39.604 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:21:39.600+0000 7fe046557640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe0400cbe00 con 0x7fe0401596d0 2023-11-26T07:21:39.605 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:21:39.600+0000 7fe03f7fe640 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7fe0400ce680 0x7fe0400b2870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:21:39.605 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:21:39.600+0000 7fe03ffff640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fe0401596d0 0x7fe040071850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:21:39.605 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:21:39.604+0000 7fe03f7fe640 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7fe0400ce680 0x7fe0400b2870 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.188:3300/0 says I am v2:172.21.15.143:59080/0 (socket says 172.21.15.143:59080) 2023-11-26T07:21:39.605 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:21:39.604+0000 7fe03f7fe640 1 -- 172.21.15.143:0/3392555354 learned_addr learned my addr 172.21.15.143:0/3392555354 (peer_addr_for_me v2:172.21.15.143:0/0) 2023-11-26T07:21:39.605 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:21:39.604+0000 7fe03ffff640 1 -- 172.21.15.143:0/3392555354 >> v2:172.21.15.188:3300/0 conn(0x7fe0400ce680 msgr2=0x7fe0400b2870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-26T07:21:39.605 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:21:39.604+0000 7fe03ffff640 1 --2- 172.21.15.143:0/3392555354 >> v2:172.21.15.188:3300/0 conn(0x7fe0400ce680 0x7fe0400b2870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:21:39.606 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:21:39.604+0000 7fe03ffff640 1 -- 172.21.15.143:0/3392555354 >> v1:172.21.15.188:6789/0 conn(0x7fe0400b2db0 legacy=0x7fe04015d2c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:21:39.606 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:21:39.604+0000 7fe03ffff640 1 -- 172.21.15.143:0/3392555354 --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe0400b3550 con 0x7fe0401596d0 2023-11-26T07:21:39.606 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:21:39.604+0000 7fe03f7fe640 1 --2- 172.21.15.143:0/3392555354 >> v2:172.21.15.188:3300/0 conn(0x7fe0400ce680 0x7fe0400b2870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-26T07:21:39.607 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:21:39.604+0000 7fe03ffff640 1 --2- 172.21.15.143:0/3392555354 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fe0401596d0 0x7fe040071850 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7fe0300097c0 tx=0x7fe03001c480 comp rx=0 tx=0).ready entity=mon.2 client_cookie=d46c65b1df62e5d6 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:21:39.608 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:21:39.604+0000 7fe03effd640 1 -- 172.21.15.143:0/3392555354 <== mon.2 v2:172.21.15.188:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe0300ab600 con 0x7fe0401596d0 2023-11-26T07:21:39.608 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:21:39.604+0000 7fe03effd640 1 -- 172.21.15.143:0/3392555354 <== mon.2 v2:172.21.15.188:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe0300a7050 con 0x7fe0401596d0 2023-11-26T07:21:39.608 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:21:39.604+0000 7fe03effd640 1 -- 172.21.15.143:0/3392555354 <== mon.2 v2:172.21.15.188:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe0300ab980 con 0x7fe0401596d0 2023-11-26T07:21:39.609 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:21:39.608+0000 7fe046557640 1 -- 172.21.15.143:0/3392555354 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fe0401596d0 msgr2=0x7fe040071850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:21:39.609 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:21:39.608+0000 7fe046557640 1 --2- 172.21.15.143:0/3392555354 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fe0401596d0 0x7fe040071850 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7fe0300097c0 tx=0x7fe03001c480 comp rx=0 tx=0).stop 2023-11-26T07:21:39.609 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:21:39.608+0000 7fe046557640 1 -- 172.21.15.143:0/3392555354 shutdown_connections 2023-11-26T07:21:39.609 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:21:39.608+0000 7fe046557640 1 --2- 172.21.15.143:0/3392555354 >> v2:172.21.15.188:3300/0 conn(0x7fe0400ce680 0x7fe0400b2870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:21:39.609 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:21:39.608+0000 7fe046557640 1 --2- 172.21.15.143:0/3392555354 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fe0401596d0 0x7fe040071850 unknown :-1 s=CLOSED pgs=98 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:21:39.609 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:21:39.608+0000 7fe046557640 1 -- 172.21.15.143:0/3392555354 >> 172.21.15.143:0/3392555354 conn(0x7fe04006a890 msgr2=0x7fe04006aca0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:21:39.609 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:21:39.608+0000 7fe046557640 1 -- 172.21.15.143:0/3392555354 shutdown_connections 2023-11-26T07:21:39.610 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:21:39.608+0000 7fe046557640 1 -- 172.21.15.143:0/3392555354 wait complete. 2023-11-26T07:21:39.610 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:21:39.608+0000 7fe046557640 1 Processor -- start 2023-11-26T07:21:39.610 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:21:39.608+0000 7fe046557640 1 -- start start 2023-11-26T07:21:39.612 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:21:39.608+0000 7fe046557640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fe0400b2db0 0x7fe040137ce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:21:39.612 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:21:39.608+0000 7fe046557640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fe0400ce680 0x7fe040168130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:21:39.612 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:21:39.608+0000 7fe046557640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fe0401596d0 0x7fe0400af5b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:21:39.612 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:21:39.608+0000 7fe046557640 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe040167500 con 0x7fe0401596d0 2023-11-26T07:21:39.612 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:21:39.608+0000 7fe046557640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe0401668f0 con 0x7fe0400b2db0 2023-11-26T07:21:39.612 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:21:39.608+0000 7fe046557640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe040166bd0 con 0x7fe0400ce680 2023-11-26T07:21:39.612 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:21:39.608+0000 7fe03f7fe640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fe0400ce680 0x7fe040168130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:21:39.613 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:21:39.608+0000 7fe03ffff640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fe0400b2db0 0x7fe040137ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:21:39.613 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:21:39.608+0000 7fe044af4640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fe0401596d0 0x7fe0400af5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:21:39.613 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:21:39.608+0000 7fe03f7fe640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fe0400ce680 0x7fe040168130 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.188:3301/0 says I am v2:172.21.15.143:57610/0 (socket says 172.21.15.143:57610) 2023-11-26T07:21:39.613 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:21:39.608+0000 7fe03f7fe640 1 -- 172.21.15.143:0/2845924678 learned_addr learned my addr 172.21.15.143:0/2845924678 (peer_addr_for_me v2:172.21.15.143:0/0) 2023-11-26T07:21:39.613 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:21:39.608+0000 7fe03ffff640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fe0400b2db0 0x7fe040137ce0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.188:3300/0 says I am v2:172.21.15.143:59090/0 (socket says 172.21.15.143:59090) 2023-11-26T07:21:39.614 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:21:39.612+0000 7fe03f7fe640 1 -- 172.21.15.143:0/2845924678 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fe0400b2db0 msgr2=0x7fe040137ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:21:39.614 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:21:39.612+0000 7fe03f7fe640 1 --2- 172.21.15.143:0/2845924678 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fe0400b2db0 0x7fe040137ce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:21:39.614 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:21:39.612+0000 7fe03f7fe640 1 -- 172.21.15.143:0/2845924678 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fe0401596d0 msgr2=0x7fe0400af5b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:21:39.614 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:21:39.612+0000 7fe03f7fe640 1 --2- 172.21.15.143:0/2845924678 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fe0401596d0 0x7fe0400af5b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:21:39.614 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:21:39.612+0000 7fe03f7fe640 1 -- 172.21.15.143:0/2845924678 --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe0340094d0 con 0x7fe0400ce680 2023-11-26T07:21:39.614 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:21:39.612+0000 7fe03ffff640 1 --2- 172.21.15.143:0/2845924678 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fe0400b2db0 0x7fe040137ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-26T07:21:39.614 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:21:39.612+0000 7fe044af4640 1 --2- 172.21.15.143:0/2845924678 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fe0401596d0 0x7fe0400af5b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-26T07:21:39.614 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:21:39.612+0000 7fe03f7fe640 1 --2- 172.21.15.143:0/2845924678 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fe0400ce680 0x7fe040168130 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7fe03400e9d0 tx=0x7fe03400ee90 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:21:39.616 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:21:39.612+0000 7fe03cff9640 1 -- 172.21.15.143:0/2845924678 <== mon.2 v2:172.21.15.188:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe03400ca60 con 0x7fe0400ce680 2023-11-26T07:21:39.616 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:21:39.612+0000 7fe03cff9640 1 -- 172.21.15.143:0/2845924678 <== mon.2 v2:172.21.15.188:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe03400cbc0 con 0x7fe0400ce680 2023-11-26T07:21:39.616 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:21:39.612+0000 7fe046557640 1 -- 172.21.15.143:0/2845924678 --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe030009420 con 0x7fe0400ce680 2023-11-26T07:21:39.617 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:21:39.612+0000 7fe03cff9640 1 -- 172.21.15.143:0/2845924678 <== mon.2 v2:172.21.15.188:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe034013470 con 0x7fe0400ce680 2023-11-26T07:21:39.617 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:21:39.612+0000 7fe046557640 1 -- 172.21.15.143:0/2845924678 --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe040167190 con 0x7fe0400ce680 2023-11-26T07:21:39.617 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:21:39.616+0000 7fe046557640 1 -- 172.21.15.143:0/2845924678 --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe040067680 con 0x7fe0400ce680 2023-11-26T07:21:39.618 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:21:39.616+0000 7fe03cff9640 1 -- 172.21.15.143:0/2845924678 <== mon.2 v2:172.21.15.188:3301/0 4 ==== mgrmap(e 9) v1 ==== 86005+0+0 (secure 0 0 0) 0x7fe0340135d0 con 0x7fe0400ce680 2023-11-26T07:21:39.619 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:21:39.616+0000 7fe03cff9640 1 --2- 172.21.15.143:0/2845924678 >> [v2:172.21.15.143:6832/2150557544,v1:172.21.15.143:6833/2150557544] conn(0x7fe02c066650 0x7fe02c068b10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:21:39.619 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:21:39.616+0000 7fe03cff9640 1 -- 172.21.15.143:0/2845924678 <== mon.2 v2:172.21.15.188:3301/0 5 ==== osd_map(47..47 src has 1..47) v4 ==== 5767+0+0 (secure 0 0 0) 0x7fe034084430 con 0x7fe0400ce680 2023-11-26T07:21:39.619 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:21:39.616+0000 7fe03ffff640 1 --2- 172.21.15.143:0/2845924678 >> [v2:172.21.15.143:6832/2150557544,v1:172.21.15.143:6833/2150557544] conn(0x7fe02c066650 0x7fe02c068b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:21:39.620 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:21:39.616+0000 7fe03ffff640 1 --2- 172.21.15.143:0/2845924678 >> [v2:172.21.15.143:6832/2150557544,v1:172.21.15.143:6833/2150557544] conn(0x7fe02c066650 0x7fe02c068b10 secure :-1 s=READY pgs=344 cs=0 l=1 rev1=1 crypto rx=0x7fe0300ad000 tx=0x7fe0300a6000 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:21:39.624 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:21:39.620+0000 7fe03cff9640 1 -- 172.21.15.143:0/2845924678 <== mon.2 v2:172.21.15.188:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172649 (secure 0 0 0) 0x7fe0340516b0 con 0x7fe0400ce680 2023-11-26T07:21:39.805 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:21:39.804+0000 7fe046557640 1 -- 172.21.15.143:0/2845924678 --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fe0400cb0e0 con 0x7fe0400ce680 2023-11-26T07:21:39.806 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:21:39.804+0000 7fe03cff9640 1 -- 172.21.15.143:0/2845924678 <== mon.2 v2:172.21.15.188:3301/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 26 v26) v1 ==== 94+0+4922 (secure 0 0 0) 0x7fe034054cd0 con 0x7fe0400ce680 2023-11-26T07:21:39.806 INFO:teuthology.orchestra.run.smithi143.stdout: 2023-11-26T07:21:39.806 INFO:teuthology.orchestra.run.smithi143.stdout:{"epoch":26,"default_fscid":3,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5201,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.188:6835/319464848","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6833","nonce":319464848},{"type":"v1","addr":"172.21.15.188:6835","nonce":319464848}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":20},{"gid":5584,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.188:6834/2113408070","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6832","nonce":2113408070},{"type":"v1","addr":"172.21.15.188:6834","nonce":2113408070}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":20},{"gid":5745,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.143:6835/1616742593","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6834","nonce":1616742593},{"type":"v1","addr":"172.21.15.143:6835","nonce":1616742593}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":20}],"filesystems":[{"mdsmap":{"epoch":25,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-11-26T07:19:31.503587+0000","modified":"2023-11-26T07:19:34.523605+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":5748},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5748":{"gid":5748,"name":"c","rank":0,"incarnation":21,"state":"up:active","state_seq":3,"addr":"172.21.15.143:6837/2872033338","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6836","nonce":2872033338},{"type":"v1","addr":"172.21.15.143:6837","nonce":2872033338}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[7],"metadata_pool":6,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":3}]} 2023-11-26T07:21:39.806 INFO:teuthology.orchestra.run.smithi143.stderr:dumped fsmap epoch 26 2023-11-26T07:21:39.808 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:21:39.804+0000 7fe046557640 1 -- 172.21.15.143:0/2845924678 >> [v2:172.21.15.143:6832/2150557544,v1:172.21.15.143:6833/2150557544] conn(0x7fe02c066650 msgr2=0x7fe02c068b10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:21:39.808 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:21:39.804+0000 7fe046557640 1 --2- 172.21.15.143:0/2845924678 >> [v2:172.21.15.143:6832/2150557544,v1:172.21.15.143:6833/2150557544] conn(0x7fe02c066650 0x7fe02c068b10 secure :-1 s=READY pgs=344 cs=0 l=1 rev1=1 crypto rx=0x7fe0300ad000 tx=0x7fe0300a6000 comp rx=0 tx=0).stop 2023-11-26T07:21:39.808 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:21:39.804+0000 7fe046557640 1 -- 172.21.15.143:0/2845924678 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fe0400ce680 msgr2=0x7fe040168130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:21:39.808 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:21:39.804+0000 7fe046557640 1 --2- 172.21.15.143:0/2845924678 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fe0400ce680 0x7fe040168130 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7fe03400e9d0 tx=0x7fe03400ee90 comp rx=0 tx=0).stop 2023-11-26T07:21:39.808 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:21:39.804+0000 7fe046557640 1 -- 172.21.15.143:0/2845924678 shutdown_connections 2023-11-26T07:21:39.808 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:21:39.804+0000 7fe046557640 1 --2- 172.21.15.143:0/2845924678 >> [v2:172.21.15.143:6832/2150557544,v1:172.21.15.143:6833/2150557544] conn(0x7fe02c066650 0x7fe02c068b10 unknown :-1 s=CLOSED pgs=344 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:21:39.808 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:21:39.804+0000 7fe046557640 1 --2- 172.21.15.143:0/2845924678 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fe0401596d0 0x7fe0400af5b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:21:39.809 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:21:39.804+0000 7fe046557640 1 --2- 172.21.15.143:0/2845924678 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fe0400ce680 0x7fe040168130 unknown :-1 s=CLOSED pgs=99 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:21:39.809 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:21:39.804+0000 7fe046557640 1 --2- 172.21.15.143:0/2845924678 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fe0400b2db0 0x7fe040137ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:21:39.809 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:21:39.804+0000 7fe046557640 1 -- 172.21.15.143:0/2845924678 >> 172.21.15.143:0/2845924678 conn(0x7fe04006a890 msgr2=0x7fe040071090 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:21:39.809 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:21:39.804+0000 7fe046557640 1 -- 172.21.15.143:0/2845924678 shutdown_connections 2023-11-26T07:21:39.809 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:21:39.804+0000 7fe046557640 1 -- 172.21.15.143:0/2845924678 wait complete. 2023-11-26T07:21:39.827 DEBUG:teuthology.orchestra.run.smithi143:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-11-26T07:21:39.964 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:21:39.960+0000 7f32d26e3640 1 Processor -- start 2023-11-26T07:21:39.964 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:21:39.960+0000 7f32d26e3640 1 -- start start 2023-11-26T07:21:39.965 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:21:39.964+0000 7f32d26e3640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f32cc143ac0 0x7f32cc070370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:21:39.965 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:21:39.964+0000 7f32d26e3640 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7f32cc0708b0 0x7f32cc0cf280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:21:39.965 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:21:39.964+0000 7f32d26e3640 1 -- --> v1:172.21.15.143:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f32cc0cfbf0 con 0x7f32cc0cf7c0 2023-11-26T07:21:39.965 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:21:39.964+0000 7f32d26e3640 1 -- --> v2:172.21.15.143:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f32cc143ea0 con 0x7f32cc0708b0 2023-11-26T07:21:39.966 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:21:39.964+0000 7f32d26e3640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f32cc0cc730 con 0x7f32cc143ac0 2023-11-26T07:21:39.966 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:21:39.964+0000 7f32cb7fe640 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7f32cc0708b0 0x7f32cc0cf280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:21:39.966 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:21:39.964+0000 7f32d0c80640 1 --1- >> v1:172.21.15.143:6789/0 conn(0x7f32cc0cf7c0 0x7f32cc0b1ef0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.143:6789/0 says I am v1:172.21.15.143:36132/0 (socket says 172.21.15.143:36132) 2023-11-26T07:21:39.966 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:21:39.964+0000 7f32d0c80640 1 -- 172.21.15.143:0/2988313890 learned_addr learned my addr 172.21.15.143:0/2988313890 (peer_addr_for_me v1:172.21.15.143:0/0) 2023-11-26T07:21:39.966 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:21:39.964+0000 7f32cb7fe640 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7f32cc0708b0 0x7f32cc0cf280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:35698/0 (socket says 172.21.15.143:35698) 2023-11-26T07:21:39.967 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:21:39.964+0000 7f32cb7fe640 1 -- 172.21.15.143:0/2988313890 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f32cc143ac0 msgr2=0x7f32cc070370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-26T07:21:39.967 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:21:39.964+0000 7f32cbfff640 1 --2- 172.21.15.143:0/2988313890 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f32cc143ac0 0x7f32cc070370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:21:39.967 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:21:39.964+0000 7f32cb7fe640 1 --2- 172.21.15.143:0/2988313890 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f32cc143ac0 0x7f32cc070370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:21:39.967 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:21:39.964+0000 7f32cb7fe640 1 -- 172.21.15.143:0/2988313890 >> v1:172.21.15.143:6789/0 conn(0x7f32cc0cf7c0 legacy=0x7f32cc0b1ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:21:39.967 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:21:39.964+0000 7f32cb7fe640 1 -- 172.21.15.143:0/2988313890 --> v2:172.21.15.143:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f32cc070ee0 con 0x7f32cc0708b0 2023-11-26T07:21:39.967 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:21:39.964+0000 7f32cbfff640 1 --2- 172.21.15.143:0/2988313890 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f32cc143ac0 0x7f32cc070370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-26T07:21:39.967 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:21:39.964+0000 7f32caffd640 1 -- 172.21.15.143:0/2988313890 <== mon.0 v1:172.21.15.143:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 527765824 0 0) 0x7f32cc0cfbf0 con 0x7f32cc0cf7c0 2023-11-26T07:21:39.969 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:21:39.968+0000 7f32cb7fe640 1 --2- 172.21.15.143:0/2988313890 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f32cc0708b0 0x7f32cc0cf280 secure :-1 s=READY pgs=718 cs=0 l=1 rev1=1 crypto rx=0x7f32c000ba60 tx=0x7f32c001eb50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2418939f02a5b149 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:21:39.969 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:21:39.968+0000 7f32caffd640 1 -- 172.21.15.143:0/2988313890 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f32c00a7b30 con 0x7f32cc0708b0 2023-11-26T07:21:39.969 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:21:39.968+0000 7f32caffd640 1 -- 172.21.15.143:0/2988313890 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f32c00a7c90 con 0x7f32cc0708b0 2023-11-26T07:21:39.969 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:21:39.968+0000 7f32caffd640 1 -- 172.21.15.143:0/2988313890 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f32c00af440 con 0x7f32cc0708b0 2023-11-26T07:21:39.970 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:21:39.968+0000 7f32d26e3640 1 -- 172.21.15.143:0/2988313890 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f32cc0708b0 msgr2=0x7f32cc0cf280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:21:39.970 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:21:39.968+0000 7f32d26e3640 1 --2- 172.21.15.143:0/2988313890 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f32cc0708b0 0x7f32cc0cf280 secure :-1 s=READY pgs=718 cs=0 l=1 rev1=1 crypto rx=0x7f32c000ba60 tx=0x7f32c001eb50 comp rx=0 tx=0).stop 2023-11-26T07:21:39.970 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:21:39.968+0000 7f32d26e3640 1 -- 172.21.15.143:0/2988313890 shutdown_connections 2023-11-26T07:21:39.970 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:21:39.968+0000 7f32d26e3640 1 --2- 172.21.15.143:0/2988313890 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f32cc0708b0 0x7f32cc0cf280 unknown :-1 s=CLOSED pgs=718 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:21:39.970 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:21:39.968+0000 7f32d26e3640 1 --2- 172.21.15.143:0/2988313890 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f32cc143ac0 0x7f32cc070370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:21:39.970 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:21:39.968+0000 7f32d26e3640 1 -- 172.21.15.143:0/2988313890 >> 172.21.15.143:0/2988313890 conn(0x7f32cc06a890 msgr2=0x7f32cc06aca0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:21:39.970 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:21:39.968+0000 7f32d26e3640 1 -- 172.21.15.143:0/2988313890 shutdown_connections 2023-11-26T07:21:39.970 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:21:39.968+0000 7f32d26e3640 1 -- 172.21.15.143:0/2988313890 wait complete. 2023-11-26T07:21:39.971 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:21:39.968+0000 7f32d26e3640 1 Processor -- start 2023-11-26T07:21:39.971 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:21:39.968+0000 7f32d26e3640 1 -- start start 2023-11-26T07:21:39.973 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:21:39.972+0000 7f32d26e3640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f32cc0708b0 0x7f32cc1598f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:21:39.973 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:21:39.972+0000 7f32d26e3640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f32cc0cf7c0 0x7f32cc141300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:21:39.973 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:21:39.972+0000 7f32d26e3640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f32cc143ac0 0x7f32cc13cb40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:21:39.973 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:21:39.972+0000 7f32d26e3640 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f32cc1f71d0 con 0x7f32cc0cf7c0 2023-11-26T07:21:39.974 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:21:39.972+0000 7f32d26e3640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f32cc0aeb30 con 0x7f32cc0708b0 2023-11-26T07:21:39.974 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:21:39.972+0000 7f32d26e3640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f32cc15b9a0 con 0x7f32cc143ac0 2023-11-26T07:21:39.974 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:21:39.972+0000 7f32cb7fe640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f32cc0cf7c0 0x7f32cc141300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:21:39.974 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:21:39.972+0000 7f32cb7fe640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f32cc0cf7c0 0x7f32cc141300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:35710/0 (socket says 172.21.15.143:35710) 2023-11-26T07:21:39.974 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:21:39.972+0000 7f32cb7fe640 1 -- 172.21.15.143:0/3497178210 learned_addr learned my addr 172.21.15.143:0/3497178210 (peer_addr_for_me v2:172.21.15.143:0/0) 2023-11-26T07:21:39.974 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:21:39.972+0000 7f32cbfff640 1 --2- 172.21.15.143:0/3497178210 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f32cc0708b0 0x7f32cc1598f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:21:39.974 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:21:39.972+0000 7f32d0c80640 1 --2- 172.21.15.143:0/3497178210 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f32cc143ac0 0x7f32cc13cb40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:21:39.974 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:21:39.972+0000 7f32cb7fe640 1 -- 172.21.15.143:0/3497178210 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f32cc143ac0 msgr2=0x7f32cc13cb40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:21:39.974 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:21:39.972+0000 7f32cb7fe640 1 --2- 172.21.15.143:0/3497178210 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f32cc143ac0 0x7f32cc13cb40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:21:39.974 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:21:39.972+0000 7f32cb7fe640 1 -- 172.21.15.143:0/3497178210 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f32cc0708b0 msgr2=0x7f32cc1598f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:21:39.974 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:21:39.972+0000 7f32cb7fe640 1 --2- 172.21.15.143:0/3497178210 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f32cc0708b0 0x7f32cc1598f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:21:39.974 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:21:39.972+0000 7f32cb7fe640 1 -- 172.21.15.143:0/3497178210 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f32c000b6c0 con 0x7f32cc0cf7c0 2023-11-26T07:21:39.975 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:21:39.972+0000 7f32cb7fe640 1 --2- 172.21.15.143:0/3497178210 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f32cc0cf7c0 0x7f32cc141300 secure :-1 s=READY pgs=719 cs=0 l=1 rev1=1 crypto rx=0x7f32c00b2040 tx=0x7f32c001b8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:21:39.975 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:21:39.972+0000 7f32c8ff9640 1 -- 172.21.15.143:0/3497178210 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f32c00b8460 con 0x7f32cc0cf7c0 2023-11-26T07:21:39.976 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:21:39.972+0000 7f32c8ff9640 1 -- 172.21.15.143:0/3497178210 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f32c00b85e0 con 0x7f32cc0cf7c0 2023-11-26T07:21:39.976 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:21:39.972+0000 7f32d26e3640 1 -- 172.21.15.143:0/3497178210 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f32cc1f6270 con 0x7f32cc0cf7c0 2023-11-26T07:21:39.976 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:21:39.972+0000 7f32c8ff9640 1 -- 172.21.15.143:0/3497178210 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f32c00b8990 con 0x7f32cc0cf7c0 2023-11-26T07:21:39.976 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:21:39.972+0000 7f32d26e3640 1 -- 172.21.15.143:0/3497178210 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f32cc1f6830 con 0x7f32cc0cf7c0 2023-11-26T07:21:39.976 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:21:39.972+0000 7f32d26e3640 1 -- 172.21.15.143:0/3497178210 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f32cc067680 con 0x7f32cc0cf7c0 2023-11-26T07:21:39.976 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:21:39.972+0000 7f32c8ff9640 1 -- 172.21.15.143:0/3497178210 <== mon.0 v2:172.21.15.143:3300/0 4 ==== mgrmap(e 9) v1 ==== 86005+0+0 (secure 0 0 0) 0x7f32c00b6070 con 0x7f32cc0cf7c0 2023-11-26T07:21:39.977 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:21:39.976+0000 7f32c8ff9640 1 --2- 172.21.15.143:0/3497178210 >> [v2:172.21.15.143:6832/2150557544,v1:172.21.15.143:6833/2150557544] conn(0x7f32b8066620 0x7f32b8068ae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:21:39.977 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:21:39.976+0000 7f32c8ff9640 1 -- 172.21.15.143:0/3497178210 <== mon.0 v2:172.21.15.143:3300/0 5 ==== osd_map(47..47 src has 1..47) v4 ==== 5767+0+0 (secure 0 0 0) 0x7f32c011eff0 con 0x7f32cc0cf7c0 2023-11-26T07:21:39.977 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:21:39.976+0000 7f32cbfff640 1 --2- 172.21.15.143:0/3497178210 >> [v2:172.21.15.143:6832/2150557544,v1:172.21.15.143:6833/2150557544] conn(0x7f32b8066620 0x7f32b8068ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:21:39.978 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:21:39.976+0000 7f32cbfff640 1 --2- 172.21.15.143:0/3497178210 >> [v2:172.21.15.143:6832/2150557544,v1:172.21.15.143:6833/2150557544] conn(0x7f32b8066620 0x7f32b8068ae0 secure :-1 s=READY pgs=345 cs=0 l=1 rev1=1 crypto rx=0x7f32bc006fd0 tx=0x7f32bc008040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:21:39.981 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:21:39.980+0000 7f32c8ff9640 1 -- 172.21.15.143:0/3497178210 <== mon.0 v2:172.21.15.143:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172649 (secure 0 0 0) 0x7f32c00ec170 con 0x7f32cc0cf7c0 2023-11-26T07:21:40.136 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:21:40.132+0000 7f32d26e3640 1 -- 172.21.15.143:0/3497178210 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f32cc072710 con 0x7f32cc0cf7c0 2023-11-26T07:21:40.137 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:21:40.136+0000 7f32c8ff9640 1 -- 172.21.15.143:0/3497178210 <== mon.0 v2:172.21.15.143:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v47) v1 ==== 74+0+16645 (secure 0 0 0) 0x7f32c00ef790 con 0x7f32cc0cf7c0 2023-11-26T07:21:40.137 INFO:teuthology.orchestra.run.smithi143.stdout: 2023-11-26T07:21:40.137 INFO:teuthology.orchestra.run.smithi143.stdout:{"epoch":47,"fsid":"f5033060-2469-4a07-95fe-2b76f5404bc1","created":"2023-11-26T07:17:28.613663+0000","modified":"2023-11-26T07:20:36.636824+0000","last_up_change":"2023-11-26T07:17:41.157813+0000","last_in_change":"2023-11-26T07:17:31.181658+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":7,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-11-26T07:17:41.758597+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":6,"pool_name":"cephfs_metadata","create_time":"2023-11-26T07:19:29.208075+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"44","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":7,"pool_name":"cephfs_data","create_time":"2023-11-26T07:19:30.503743+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"45","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"ebbee2c6-023e-44b9-9f1d-6f7bfd309730","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":43,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6817","nonce":639807369},{"type":"v1","addr":"172.21.15.143:6819","nonce":639807369}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6821","nonce":639807369},{"type":"v1","addr":"172.21.15.143:6823","nonce":639807369}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6830","nonce":639807369},{"type":"v1","addr":"172.21.15.143:6831","nonce":639807369}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6825","nonce":639807369},{"type":"v1","addr":"172.21.15.143:6828","nonce":639807369}]},"public_addr":"172.21.15.143:6819/639807369","cluster_addr":"172.21.15.143:6823/639807369","heartbeat_back_addr":"172.21.15.143:6831/639807369","heartbeat_front_addr":"172.21.15.143:6828/639807369","state":["exists","up"]},{"osd":1,"uuid":"fec4b8be-7122-49e9-9a19-5850a0700001","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":43,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6816","nonce":1144942530},{"type":"v1","addr":"172.21.15.143:6818","nonce":1144942530}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6820","nonce":1144942530},{"type":"v1","addr":"172.21.15.143:6822","nonce":1144942530}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6827","nonce":1144942530},{"type":"v1","addr":"172.21.15.143:6829","nonce":1144942530}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6824","nonce":1144942530},{"type":"v1","addr":"172.21.15.143:6826","nonce":1144942530}]},"public_addr":"172.21.15.143:6818/1144942530","cluster_addr":"172.21.15.143:6822/1144942530","heartbeat_back_addr":"172.21.15.143:6829/1144942530","heartbeat_front_addr":"172.21.15.143:6826/1144942530","state":["exists","up"]},{"osd":2,"uuid":"2888af5b-f933-46ca-8f95-6dd494ff43f6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":46,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6808","nonce":3909095186},{"type":"v1","addr":"172.21.15.143:6809","nonce":3909095186}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6810","nonce":3909095186},{"type":"v1","addr":"172.21.15.143:6811","nonce":3909095186}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6814","nonce":3909095186},{"type":"v1","addr":"172.21.15.143:6815","nonce":3909095186}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6812","nonce":3909095186},{"type":"v1","addr":"172.21.15.143:6813","nonce":3909095186}]},"public_addr":"172.21.15.143:6809/3909095186","cluster_addr":"172.21.15.143:6811/3909095186","heartbeat_back_addr":"172.21.15.143:6815/3909095186","heartbeat_front_addr":"172.21.15.143:6813/3909095186","state":["exists","up"]},{"osd":3,"uuid":"d1f52435-7bc1-4081-9550-9cd1ba55f549","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":43,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6800","nonce":3854210991},{"type":"v1","addr":"172.21.15.143:6801","nonce":3854210991}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6802","nonce":3854210991},{"type":"v1","addr":"172.21.15.143:6803","nonce":3854210991}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6806","nonce":3854210991},{"type":"v1","addr":"172.21.15.143:6807","nonce":3854210991}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6804","nonce":3854210991},{"type":"v1","addr":"172.21.15.143:6805","nonce":3854210991}]},"public_addr":"172.21.15.143:6801/3854210991","cluster_addr":"172.21.15.143:6803/3854210991","heartbeat_back_addr":"172.21.15.143:6807/3854210991","heartbeat_front_addr":"172.21.15.143:6805/3854210991","state":["exists","up"]},{"osd":4,"uuid":"e2ae8b6e-6e25-49b2-bcbf-aacce68f06a3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":43,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6800","nonce":3924606902},{"type":"v1","addr":"172.21.15.188:6801","nonce":3924606902}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6802","nonce":3924606902},{"type":"v1","addr":"172.21.15.188:6803","nonce":3924606902}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6806","nonce":3924606902},{"type":"v1","addr":"172.21.15.188:6807","nonce":3924606902}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6804","nonce":3924606902},{"type":"v1","addr":"172.21.15.188:6805","nonce":3924606902}]},"public_addr":"172.21.15.188:6801/3924606902","cluster_addr":"172.21.15.188:6803/3924606902","heartbeat_back_addr":"172.21.15.188:6807/3924606902","heartbeat_front_addr":"172.21.15.188:6805/3924606902","state":["exists","up"]},{"osd":5,"uuid":"3cec8a32-7698-4c36-8e89-e4ba11934f33","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":43,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6813","nonce":186841544},{"type":"v1","addr":"172.21.15.188:6815","nonce":186841544}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6817","nonce":186841544},{"type":"v1","addr":"172.21.15.188:6818","nonce":186841544}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6822","nonce":186841544},{"type":"v1","addr":"172.21.15.188:6823","nonce":186841544}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6820","nonce":186841544},{"type":"v1","addr":"172.21.15.188:6821","nonce":186841544}]},"public_addr":"172.21.15.188:6815/186841544","cluster_addr":"172.21.15.188:6818/186841544","heartbeat_back_addr":"172.21.15.188:6823/186841544","heartbeat_front_addr":"172.21.15.188:6821/186841544","state":["exists","up"]},{"osd":6,"uuid":"58a65d5d-e090-473a-81b4-ad38e85029f9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":46,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6808","nonce":2153499405},{"type":"v1","addr":"172.21.15.188:6809","nonce":2153499405}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6810","nonce":2153499405},{"type":"v1","addr":"172.21.15.188:6811","nonce":2153499405}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6816","nonce":2153499405},{"type":"v1","addr":"172.21.15.188:6819","nonce":2153499405}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6812","nonce":2153499405},{"type":"v1","addr":"172.21.15.188:6814","nonce":2153499405}]},"public_addr":"172.21.15.188:6809/2153499405","cluster_addr":"172.21.15.188:6811/2153499405","heartbeat_back_addr":"172.21.15.188:6819/2153499405","heartbeat_front_addr":"172.21.15.188:6814/2153499405","state":["exists","up"]},{"osd":7,"uuid":"ea5b7f70-3554-4cf8-ac30-f7c59b80678d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":43,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6824","nonce":2722096729},{"type":"v1","addr":"172.21.15.188:6825","nonce":2722096729}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6826","nonce":2722096729},{"type":"v1","addr":"172.21.15.188:6827","nonce":2722096729}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6830","nonce":2722096729},{"type":"v1","addr":"172.21.15.188:6831","nonce":2722096729}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6828","nonce":2722096729},{"type":"v1","addr":"172.21.15.188:6829","nonce":2722096729}]},"public_addr":"172.21.15.188:6825/2722096729","cluster_addr":"172.21.15.188:6827/2722096729","heartbeat_back_addr":"172.21.15.188:6831/2722096729","heartbeat_front_addr":"172.21.15.188:6829/2722096729","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-26T07:17:36.995032+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-26T07:17:36.932266+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-26T07:17:36.829304+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-26T07:17:36.804297+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-26T07:17:36.667892+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-26T07:17:36.964868+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-26T07:17:36.827332+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-26T07:17:36.893179+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"7.2c","mappings":[{"from":0,"to":4}]},{"pgid":"7.2d","mappings":[{"from":0,"to":5}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-11-26T07:21:40.139 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:21:40.136+0000 7f32d26e3640 1 -- 172.21.15.143:0/3497178210 >> [v2:172.21.15.143:6832/2150557544,v1:172.21.15.143:6833/2150557544] conn(0x7f32b8066620 msgr2=0x7f32b8068ae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:21:40.139 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:21:40.136+0000 7f32d26e3640 1 --2- 172.21.15.143:0/3497178210 >> [v2:172.21.15.143:6832/2150557544,v1:172.21.15.143:6833/2150557544] conn(0x7f32b8066620 0x7f32b8068ae0 secure :-1 s=READY pgs=345 cs=0 l=1 rev1=1 crypto rx=0x7f32bc006fd0 tx=0x7f32bc008040 comp rx=0 tx=0).stop 2023-11-26T07:21:40.139 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:21:40.136+0000 7f32d26e3640 1 -- 172.21.15.143:0/3497178210 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f32cc0cf7c0 msgr2=0x7f32cc141300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:21:40.139 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:21:40.136+0000 7f32d26e3640 1 --2- 172.21.15.143:0/3497178210 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f32cc0cf7c0 0x7f32cc141300 secure :-1 s=READY pgs=719 cs=0 l=1 rev1=1 crypto rx=0x7f32c00b2040 tx=0x7f32c001b8f0 comp rx=0 tx=0).stop 2023-11-26T07:21:40.139 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:21:40.136+0000 7f32d26e3640 1 -- 172.21.15.143:0/3497178210 shutdown_connections 2023-11-26T07:21:40.139 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:21:40.136+0000 7f32d26e3640 1 --2- 172.21.15.143:0/3497178210 >> [v2:172.21.15.143:6832/2150557544,v1:172.21.15.143:6833/2150557544] conn(0x7f32b8066620 0x7f32b8068ae0 unknown :-1 s=CLOSED pgs=345 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:21:40.139 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:21:40.136+0000 7f32d26e3640 1 --2- 172.21.15.143:0/3497178210 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f32cc143ac0 0x7f32cc13cb40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:21:40.139 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:21:40.136+0000 7f32d26e3640 1 --2- 172.21.15.143:0/3497178210 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f32cc0cf7c0 0x7f32cc141300 unknown :-1 s=CLOSED pgs=719 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:21:40.139 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:21:40.136+0000 7f32d26e3640 1 --2- 172.21.15.143:0/3497178210 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f32cc0708b0 0x7f32cc1598f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:21:40.140 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:21:40.136+0000 7f32d26e3640 1 -- 172.21.15.143:0/3497178210 >> 172.21.15.143:0/3497178210 conn(0x7f32cc06a890 msgr2=0x7f32cc06ac70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:21:40.140 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:21:40.136+0000 7f32d26e3640 1 -- 172.21.15.143:0/3497178210 shutdown_connections 2023-11-26T07:21:40.140 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:21:40.136+0000 7f32d26e3640 1 -- 172.21.15.143:0/3497178210 wait complete. 2023-11-26T07:21:40.160 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-11-26T07:17:41.758597+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 6, 'pool_name': 'cephfs_metadata', 'create_time': '2023-11-26T07:19:29.208075+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '44', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 1.75, 'score_stable': 1.75, 'optimal_score': 1, 'raw_score_acting': 1.75, 'raw_score_stable': 1.75, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 7, 'pool_name': 'cephfs_data', 'create_time': '2023-11-26T07:19:30.503743+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '45', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.75, 'score_stable': 1.75, 'optimal_score': 1, 'raw_score_acting': 1.75, 'raw_score_stable': 1.75, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2023-11-26T07:21:40.160 DEBUG:teuthology.orchestra.run.smithi143:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-11-26T07:21:40.302 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:21:40.300+0000 7f5124a13640 1 Processor -- start 2023-11-26T07:21:40.302 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:21:40.300+0000 7f5124a13640 1 -- start start 2023-11-26T07:21:40.303 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:21:40.300+0000 7f5124a13640 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7f51200ce7c0 0x7f512015f6b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:21:40.303 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:21:40.300+0000 7f5124a13640 1 -- --> v1:172.21.15.143:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f512006c520 con 0x7f512013e290 2023-11-26T07:21:40.303 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:21:40.300+0000 7f5124a13640 1 -- --> v1:172.21.15.188:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f512006c800 con 0x7f51200cf990 2023-11-26T07:21:40.303 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:21:40.300+0000 7f5124a13640 1 -- --> v2:172.21.15.143:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f512006cae0 con 0x7f51200ce7c0 2023-11-26T07:21:40.303 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:21:40.300+0000 7f511e59c640 1 --1- >> v1:172.21.15.143:6789/0 conn(0x7f512013e290 0x7f51200cf280 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.143:6789/0 says I am v1:172.21.15.143:36142/0 (socket says 172.21.15.143:36142) 2023-11-26T07:21:40.304 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:21:40.300+0000 7f511e59c640 1 -- 172.21.15.143:0/1197224142 learned_addr learned my addr 172.21.15.143:0/1197224142 (peer_addr_for_me v1:172.21.15.143:0/0) 2023-11-26T07:21:40.304 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:21:40.300+0000 7f511ed9d640 1 --2- 172.21.15.143:0/1197224142 >> v2:172.21.15.143:3300/0 conn(0x7f51200ce7c0 0x7f512015f6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:21:40.304 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:21:40.300+0000 7f511dd9b640 1 -- 172.21.15.143:0/1197224142 <== mon.0 v1:172.21.15.143:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3355020663 0 0) 0x7f512006c520 con 0x7f512013e290 2023-11-26T07:21:40.304 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:21:40.300+0000 7f511dd9b640 1 -- 172.21.15.143:0/1197224142 --> v1:172.21.15.143:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f510c003530 con 0x7f512013e290 2023-11-26T07:21:40.304 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:21:40.300+0000 7f511ed9d640 1 -- 172.21.15.143:0/1197224142 >> v1:172.21.15.188:6789/0 conn(0x7f51200cf990 legacy=0x7f512015efa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:21:40.305 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:21:40.300+0000 7f511ed9d640 1 -- 172.21.15.143:0/1197224142 >> v1:172.21.15.143:6789/0 conn(0x7f512013e290 legacy=0x7f51200cf280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:21:40.305 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:21:40.300+0000 7f511ed9d640 1 -- 172.21.15.143:0/1197224142 --> v2:172.21.15.143:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f512006cdc0 con 0x7f51200ce7c0 2023-11-26T07:21:40.307 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:21:40.304+0000 7f511ed9d640 1 --2- 172.21.15.143:0/1197224142 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f51200ce7c0 0x7f512015f6b0 secure :-1 s=READY pgs=721 cs=0 l=1 rev1=1 crypto rx=0x7f5118009d90 tx=0x7f511801c890 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f3eb7df103888700 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:21:40.307 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:21:40.304+0000 7f511dd9b640 1 -- 172.21.15.143:0/1197224142 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f51180a5970 con 0x7f51200ce7c0 2023-11-26T07:21:40.307 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:21:40.304+0000 7f511dd9b640 1 -- 172.21.15.143:0/1197224142 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f51180a5ad0 con 0x7f51200ce7c0 2023-11-26T07:21:40.307 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:21:40.304+0000 7f511dd9b640 1 -- 172.21.15.143:0/1197224142 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f51180a5e60 con 0x7f51200ce7c0 2023-11-26T07:21:40.308 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:21:40.304+0000 7f5124a13640 1 -- 172.21.15.143:0/1197224142 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f51200ce7c0 msgr2=0x7f512015f6b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:21:40.308 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:21:40.304+0000 7f5124a13640 1 --2- 172.21.15.143:0/1197224142 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f51200ce7c0 0x7f512015f6b0 secure :-1 s=READY pgs=721 cs=0 l=1 rev1=1 crypto rx=0x7f5118009d90 tx=0x7f511801c890 comp rx=0 tx=0).stop 2023-11-26T07:21:40.308 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:21:40.304+0000 7f5124a13640 1 -- 172.21.15.143:0/1197224142 shutdown_connections 2023-11-26T07:21:40.308 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:21:40.304+0000 7f5124a13640 1 --2- 172.21.15.143:0/1197224142 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f51200ce7c0 0x7f512015f6b0 unknown :-1 s=CLOSED pgs=721 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:21:40.308 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:21:40.304+0000 7f5124a13640 1 -- 172.21.15.143:0/1197224142 >> 172.21.15.143:0/1197224142 conn(0x7f512006a890 msgr2=0x7f512006aca0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:21:40.308 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:21:40.304+0000 7f5124a13640 1 -- 172.21.15.143:0/1197224142 shutdown_connections 2023-11-26T07:21:40.308 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:21:40.304+0000 7f5124a13640 1 -- 172.21.15.143:0/1197224142 wait complete. 2023-11-26T07:21:40.309 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:21:40.308+0000 7f5124a13640 1 Processor -- start 2023-11-26T07:21:40.309 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:21:40.308+0000 7f5124a13640 1 -- start start 2023-11-26T07:21:40.311 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:21:40.308+0000 7f5124a13640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f51200ce7c0 0x7f5120071ce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:21:40.311 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:21:40.308+0000 7f5124a13640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f51200cf990 0x7f51200b0f90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:21:40.311 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:21:40.308+0000 7f5124a13640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f512013e290 0x7f512015d900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:21:40.311 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:21:40.308+0000 7f5124a13640 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f512015d470 con 0x7f512013e290 2023-11-26T07:21:40.311 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:21:40.308+0000 7f5124a13640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f512015c860 con 0x7f51200cf990 2023-11-26T07:21:40.311 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:21:40.308+0000 7f5124a13640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f512015cb40 con 0x7f51200ce7c0 2023-11-26T07:21:40.312 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:21:40.308+0000 7f511ed9d640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f512013e290 0x7f512015d900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:21:40.312 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:21:40.308+0000 7f511ed9d640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f512013e290 0x7f512015d900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:35734/0 (socket says 172.21.15.143:35734) 2023-11-26T07:21:40.312 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:21:40.308+0000 7f511ed9d640 1 -- 172.21.15.143:0/475672739 learned_addr learned my addr 172.21.15.143:0/475672739 (peer_addr_for_me v2:172.21.15.143:0/0) 2023-11-26T07:21:40.312 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:21:40.308+0000 7f5117fff640 1 --2- 172.21.15.143:0/475672739 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f51200cf990 0x7f51200b0f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:21:40.312 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:21:40.308+0000 7f511e59c640 1 --2- 172.21.15.143:0/475672739 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f51200ce7c0 0x7f5120071ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:21:40.312 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:21:40.308+0000 7f511ed9d640 1 -- 172.21.15.143:0/475672739 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f51200ce7c0 msgr2=0x7f5120071ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:21:40.312 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:21:40.308+0000 7f511ed9d640 1 --2- 172.21.15.143:0/475672739 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f51200ce7c0 0x7f5120071ce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:21:40.312 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:21:40.308+0000 7f511ed9d640 1 -- 172.21.15.143:0/475672739 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f51200cf990 msgr2=0x7f51200b0f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:21:40.312 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:21:40.308+0000 7f511ed9d640 1 --2- 172.21.15.143:0/475672739 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f51200cf990 0x7f51200b0f90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:21:40.312 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:21:40.308+0000 7f511ed9d640 1 -- 172.21.15.143:0/475672739 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5118009a70 con 0x7f512013e290 2023-11-26T07:21:40.312 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:21:40.308+0000 7f511e59c640 1 --2- 172.21.15.143:0/475672739 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f51200ce7c0 0x7f5120071ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-26T07:21:40.312 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:21:40.308+0000 7f511ed9d640 1 --2- 172.21.15.143:0/475672739 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f512013e290 0x7f512015d900 secure :-1 s=READY pgs=722 cs=0 l=1 rev1=1 crypto rx=0x7f51180b0040 tx=0x7f5118019630 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:21:40.313 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:21:40.308+0000 7f5116ffd640 1 -- 172.21.15.143:0/475672739 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f51180a5970 con 0x7f512013e290 2023-11-26T07:21:40.313 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:21:40.308+0000 7f5116ffd640 1 -- 172.21.15.143:0/475672739 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5118009450 con 0x7f512013e290 2023-11-26T07:21:40.313 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:21:40.308+0000 7f5124a13640 1 -- 172.21.15.143:0/475672739 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f512015ce20 con 0x7f512013e290 2023-11-26T07:21:40.313 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:21:40.308+0000 7f5116ffd640 1 -- 172.21.15.143:0/475672739 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f51180b9690 con 0x7f512013e290 2023-11-26T07:21:40.313 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:21:40.308+0000 7f5124a13640 1 -- 172.21.15.143:0/475672739 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5120072220 con 0x7f512013e290 2023-11-26T07:21:40.314 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:21:40.312+0000 7f5124a13640 1 -- 172.21.15.143:0/475672739 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f512015d5e0 con 0x7f512013e290 2023-11-26T07:21:40.314 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:21:40.312+0000 7f5116ffd640 1 -- 172.21.15.143:0/475672739 <== mon.0 v2:172.21.15.143:3300/0 4 ==== mgrmap(e 9) v1 ==== 86005+0+0 (secure 0 0 0) 0x7f51180b97f0 con 0x7f512013e290 2023-11-26T07:21:40.314 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:21:40.312+0000 7f5116ffd640 1 --2- 172.21.15.143:0/475672739 >> [v2:172.21.15.143:6832/2150557544,v1:172.21.15.143:6833/2150557544] conn(0x7f5100066620 0x7f5100068ae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:21:40.314 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:21:40.312+0000 7f5116ffd640 1 -- 172.21.15.143:0/475672739 <== mon.0 v2:172.21.15.143:3300/0 5 ==== osd_map(47..47 src has 1..47) v4 ==== 5767+0+0 (secure 0 0 0) 0x7f511811fb90 con 0x7f512013e290 2023-11-26T07:21:40.315 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:21:40.312+0000 7f511e59c640 1 --2- 172.21.15.143:0/475672739 >> [v2:172.21.15.143:6832/2150557544,v1:172.21.15.143:6833/2150557544] conn(0x7f5100066620 0x7f5100068ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:21:40.315 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:21:40.312+0000 7f511e59c640 1 --2- 172.21.15.143:0/475672739 >> [v2:172.21.15.143:6832/2150557544,v1:172.21.15.143:6833/2150557544] conn(0x7f5100066620 0x7f5100068ae0 secure :-1 s=READY pgs=346 cs=0 l=1 rev1=1 crypto rx=0x7f51100011f0 tx=0x7f5110009040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:21:40.318 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:21:40.316+0000 7f5116ffd640 1 -- 172.21.15.143:0/475672739 <== mon.0 v2:172.21.15.143:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172649 (secure 0 0 0) 0x7f51180ecd40 con 0x7f512013e290 2023-11-26T07:21:40.469 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:21:40.468+0000 7f5124a13640 1 -- 172.21.15.143:0/475672739 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f5120067680 con 0x7f512013e290 2023-11-26T07:21:40.469 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:21:40.468+0000 7f5116ffd640 1 -- 172.21.15.143:0/475672739 <== mon.0 v2:172.21.15.143:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v47) v1 ==== 93+0+10 (secure 0 0 0) 0x7f51180bd020 con 0x7f512013e290 2023-11-26T07:21:40.469 INFO:teuthology.orchestra.run.smithi143.stdout:pg_num: 1 2023-11-26T07:21:40.471 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:21:40.468+0000 7f5124a13640 1 -- 172.21.15.143:0/475672739 >> [v2:172.21.15.143:6832/2150557544,v1:172.21.15.143:6833/2150557544] conn(0x7f5100066620 msgr2=0x7f5100068ae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:21:40.471 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:21:40.468+0000 7f5124a13640 1 --2- 172.21.15.143:0/475672739 >> [v2:172.21.15.143:6832/2150557544,v1:172.21.15.143:6833/2150557544] conn(0x7f5100066620 0x7f5100068ae0 secure :-1 s=READY pgs=346 cs=0 l=1 rev1=1 crypto rx=0x7f51100011f0 tx=0x7f5110009040 comp rx=0 tx=0).stop 2023-11-26T07:21:40.471 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:21:40.468+0000 7f5124a13640 1 -- 172.21.15.143:0/475672739 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f512013e290 msgr2=0x7f512015d900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:21:40.471 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:21:40.468+0000 7f5124a13640 1 --2- 172.21.15.143:0/475672739 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f512013e290 0x7f512015d900 secure :-1 s=READY pgs=722 cs=0 l=1 rev1=1 crypto rx=0x7f51180b0040 tx=0x7f5118019630 comp rx=0 tx=0).stop 2023-11-26T07:21:40.472 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:21:40.468+0000 7f5124a13640 1 -- 172.21.15.143:0/475672739 shutdown_connections 2023-11-26T07:21:40.472 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:21:40.468+0000 7f5124a13640 1 --2- 172.21.15.143:0/475672739 >> [v2:172.21.15.143:6832/2150557544,v1:172.21.15.143:6833/2150557544] conn(0x7f5100066620 0x7f5100068ae0 unknown :-1 s=CLOSED pgs=346 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:21:40.472 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:21:40.468+0000 7f5124a13640 1 --2- 172.21.15.143:0/475672739 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f512013e290 0x7f512015d900 unknown :-1 s=CLOSED pgs=722 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:21:40.472 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:21:40.468+0000 7f5124a13640 1 --2- 172.21.15.143:0/475672739 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f51200cf990 0x7f51200b0f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:21:40.472 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:21:40.468+0000 7f5124a13640 1 --2- 172.21.15.143:0/475672739 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f51200ce7c0 0x7f5120071ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:21:40.472 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:21:40.468+0000 7f5124a13640 1 -- 172.21.15.143:0/475672739 >> 172.21.15.143:0/475672739 conn(0x7f512006a890 msgr2=0x7f512013a3e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:21:40.472 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:21:40.468+0000 7f5124a13640 1 -- 172.21.15.143:0/475672739 shutdown_connections 2023-11-26T07:21:40.472 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:21:40.468+0000 7f5124a13640 1 -- 172.21.15.143:0/475672739 wait complete. 2023-11-26T07:21:40.490 DEBUG:teuthology.orchestra.run.smithi143:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-11-26T07:21:40.631 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:21:40.628+0000 7f96de1b4640 1 Processor -- start 2023-11-26T07:21:40.632 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:21:40.628+0000 7f96de1b4640 1 -- start start 2023-11-26T07:21:40.634 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:21:40.632+0000 7f96de1b4640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f96d80d0200 0x7f96d80ada60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:21:40.634 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:21:40.632+0000 7f96de1b4640 1 -- --> v1:172.21.15.143:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f96d80cc050 con 0x7f96d80b23f0 2023-11-26T07:21:40.634 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:21:40.632+0000 7f96de1b4640 1 -- --> v1:172.21.15.188:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f96d80cb440 con 0x7f96d806c0e0 2023-11-26T07:21:40.634 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:21:40.632+0000 7f96de1b4640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f96d80cb720 con 0x7f96d80d0200 2023-11-26T07:21:40.634 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:21:40.632+0000 7f96dd1b2640 1 --1- >> v1:172.21.15.143:6789/0 conn(0x7f96d80b23f0 0x7f96d806b9d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.143:6789/0 says I am v1:172.21.15.143:36156/0 (socket says 172.21.15.143:36156) 2023-11-26T07:21:40.634 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:21:40.632+0000 7f96dd1b2640 1 -- 172.21.15.143:0/682370467 learned_addr learned my addr 172.21.15.143:0/682370467 (peer_addr_for_me v1:172.21.15.143:0/0) 2023-11-26T07:21:40.634 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:21:40.632+0000 7f96dd9b3640 1 --2- 172.21.15.143:0/682370467 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f96d80d0200 0x7f96d80ada60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:21:40.634 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:21:40.632+0000 7f96cffff640 1 -- 172.21.15.143:0/682370467 <== mon.0 v1:172.21.15.143:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3522129399 0 0) 0x7f96d80cc050 con 0x7f96d80b23f0 2023-11-26T07:21:40.634 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:21:40.632+0000 7f96cffff640 1 -- 172.21.15.143:0/682370467 --> v1:172.21.15.143:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f96c4003530 con 0x7f96d80b23f0 2023-11-26T07:21:40.635 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:21:40.632+0000 7f96cffff640 1 -- 172.21.15.143:0/682370467 <== mon.1 v1:172.21.15.188:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1347135733 0 0) 0x7f96d80cb440 con 0x7f96d806c0e0 2023-11-26T07:21:40.635 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:21:40.632+0000 7f96cffff640 1 -- 172.21.15.143:0/682370467 --> v1:172.21.15.188:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f96d80cc050 con 0x7f96d806c0e0 2023-11-26T07:21:40.635 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:21:40.632+0000 7f96dd9b3640 1 -- 172.21.15.143:0/682370467 >> v1:172.21.15.188:6789/0 conn(0x7f96d806c0e0 legacy=0x7f96d80cfaf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:21:40.635 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:21:40.632+0000 7f96dd9b3640 1 -- 172.21.15.143:0/682370467 >> v1:172.21.15.143:6789/0 conn(0x7f96d80b23f0 legacy=0x7f96d806b9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:21:40.635 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:21:40.632+0000 7f96dd9b3640 1 -- 172.21.15.143:0/682370467 --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f96d80cba00 con 0x7f96d80d0200 2023-11-26T07:21:40.637 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:21:40.636+0000 7f96dd9b3640 1 --2- 172.21.15.143:0/682370467 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f96d80d0200 0x7f96d80ada60 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7f96d0009dc0 tx=0x7f96d001c970 comp rx=0 tx=0).ready entity=mon.2 client_cookie=c6382c1f8ad43b6b server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:21:40.637 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:21:40.636+0000 7f96cffff640 1 -- 172.21.15.143:0/682370467 <== mon.2 v2:172.21.15.188:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f96d00a5940 con 0x7f96d80d0200 2023-11-26T07:21:40.637 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:21:40.636+0000 7f96cffff640 1 -- 172.21.15.143:0/682370467 <== mon.2 v2:172.21.15.188:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f96d00a5aa0 con 0x7f96d80d0200 2023-11-26T07:21:40.637 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:21:40.636+0000 7f96cffff640 1 -- 172.21.15.143:0/682370467 <== mon.2 v2:172.21.15.188:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f96d00a5e30 con 0x7f96d80d0200 2023-11-26T07:21:40.638 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:21:40.636+0000 7f96de1b4640 1 -- 172.21.15.143:0/682370467 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f96d80d0200 msgr2=0x7f96d80ada60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:21:40.638 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:21:40.636+0000 7f96de1b4640 1 --2- 172.21.15.143:0/682370467 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f96d80d0200 0x7f96d80ada60 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7f96d0009dc0 tx=0x7f96d001c970 comp rx=0 tx=0).stop 2023-11-26T07:21:40.638 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:21:40.636+0000 7f96de1b4640 1 -- 172.21.15.143:0/682370467 shutdown_connections 2023-11-26T07:21:40.638 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:21:40.636+0000 7f96de1b4640 1 --2- 172.21.15.143:0/682370467 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f96d80d0200 0x7f96d80ada60 unknown :-1 s=CLOSED pgs=100 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:21:40.639 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:21:40.636+0000 7f96de1b4640 1 -- 172.21.15.143:0/682370467 >> 172.21.15.143:0/682370467 conn(0x7f96d80d0df0 msgr2=0x7f96d80d1200 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:21:40.639 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:21:40.636+0000 7f96de1b4640 1 -- 172.21.15.143:0/682370467 shutdown_connections 2023-11-26T07:21:40.639 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:21:40.636+0000 7f96de1b4640 1 -- 172.21.15.143:0/682370467 wait complete. 2023-11-26T07:21:40.640 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:21:40.636+0000 7f96de1b4640 1 Processor -- start 2023-11-26T07:21:40.640 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:21:40.636+0000 7f96de1b4640 1 -- start start 2023-11-26T07:21:40.642 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:21:40.640+0000 7f96de1b4640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f96d806c0e0 0x7f96d8151d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:21:40.642 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:21:40.640+0000 7f96de1b4640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f96d80b23f0 0x7f96d815f790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:21:40.642 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:21:40.640+0000 7f96de1b4640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f96d80d0200 0x7f96d80c5330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:21:40.643 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:21:40.640+0000 7f96de1b4640 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f96d8151bd0 con 0x7f96d806c0e0 2023-11-26T07:21:40.643 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:21:40.640+0000 7f96de1b4640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f96d815a420 con 0x7f96d80b23f0 2023-11-26T07:21:40.643 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:21:40.640+0000 7f96de1b4640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f96d8154590 con 0x7f96d80d0200 2023-11-26T07:21:40.643 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:21:40.640+0000 7f96dd1b2640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f96d806c0e0 0x7f96d8151d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:21:40.643 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:21:40.640+0000 7f96dd1b2640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f96d806c0e0 0x7f96d8151d40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:35748/0 (socket says 172.21.15.143:35748) 2023-11-26T07:21:40.643 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:21:40.640+0000 7f96dd1b2640 1 -- 172.21.15.143:0/611330697 learned_addr learned my addr 172.21.15.143:0/611330697 (peer_addr_for_me v2:172.21.15.143:0/0) 2023-11-26T07:21:40.643 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:21:40.640+0000 7f96dd9b3640 1 --2- 172.21.15.143:0/611330697 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f96d80d0200 0x7f96d80c5330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:21:40.643 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:21:40.640+0000 7f96dc9b1640 1 --2- 172.21.15.143:0/611330697 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f96d80b23f0 0x7f96d815f790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:21:40.643 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:21:40.640+0000 7f96dd1b2640 1 -- 172.21.15.143:0/611330697 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f96d80d0200 msgr2=0x7f96d80c5330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:21:40.643 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:21:40.640+0000 7f96dd1b2640 1 --2- 172.21.15.143:0/611330697 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f96d80d0200 0x7f96d80c5330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:21:40.643 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:21:40.640+0000 7f96dd1b2640 1 -- 172.21.15.143:0/611330697 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f96d80b23f0 msgr2=0x7f96d815f790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:21:40.644 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:21:40.640+0000 7f96dd1b2640 1 --2- 172.21.15.143:0/611330697 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f96d80b23f0 0x7f96d815f790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:21:40.644 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:21:40.640+0000 7f96dd1b2640 1 -- 172.21.15.143:0/611330697 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f96d0009a70 con 0x7f96d806c0e0 2023-11-26T07:21:40.644 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:21:40.640+0000 7f96dd1b2640 1 --2- 172.21.15.143:0/611330697 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f96d806c0e0 0x7f96d8151d40 secure :-1 s=READY pgs=724 cs=0 l=1 rev1=1 crypto rx=0x7f96d400c420 tx=0x7f96d400c8e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:21:40.644 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:21:40.640+0000 7f96cdffb640 1 -- 172.21.15.143:0/611330697 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f96d4005cf0 con 0x7f96d806c0e0 2023-11-26T07:21:40.644 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:21:40.640+0000 7f96cdffb640 1 -- 172.21.15.143:0/611330697 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f96d4005e50 con 0x7f96d806c0e0 2023-11-26T07:21:40.644 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:21:40.640+0000 7f96cdffb640 1 -- 172.21.15.143:0/611330697 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f96d4007310 con 0x7f96d806c0e0 2023-11-26T07:21:40.644 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:21:40.640+0000 7f96de1b4640 1 -- 172.21.15.143:0/611330697 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f96d80c7310 con 0x7f96d806c0e0 2023-11-26T07:21:40.644 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:21:40.640+0000 7f96de1b4640 1 -- 172.21.15.143:0/611330697 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f96d815a6b0 con 0x7f96d806c0e0 2023-11-26T07:21:40.645 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:21:40.640+0000 7f96de1b4640 1 -- 172.21.15.143:0/611330697 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f96d8067500 con 0x7f96d806c0e0 2023-11-26T07:21:40.645 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:21:40.640+0000 7f96cdffb640 1 -- 172.21.15.143:0/611330697 <== mon.0 v2:172.21.15.143:3300/0 4 ==== mgrmap(e 9) v1 ==== 86005+0+0 (secure 0 0 0) 0x7f96d4007470 con 0x7f96d806c0e0 2023-11-26T07:21:40.645 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:21:40.644+0000 7f96cdffb640 1 --2- 172.21.15.143:0/611330697 >> [v2:172.21.15.143:6832/2150557544,v1:172.21.15.143:6833/2150557544] conn(0x7f96bc066650 0x7f96bc068b10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:21:40.645 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:21:40.644+0000 7f96cdffb640 1 -- 172.21.15.143:0/611330697 <== mon.0 v2:172.21.15.143:3300/0 5 ==== osd_map(47..47 src has 1..47) v4 ==== 5767+0+0 (secure 0 0 0) 0x7f96d4083860 con 0x7f96d806c0e0 2023-11-26T07:21:40.645 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:21:40.644+0000 7f96dc9b1640 1 --2- 172.21.15.143:0/611330697 >> [v2:172.21.15.143:6832/2150557544,v1:172.21.15.143:6833/2150557544] conn(0x7f96bc066650 0x7f96bc068b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:21:40.646 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:21:40.644+0000 7f96dc9b1640 1 --2- 172.21.15.143:0/611330697 >> [v2:172.21.15.143:6832/2150557544,v1:172.21.15.143:6833/2150557544] conn(0x7f96bc066650 0x7f96bc068b10 secure :-1 s=READY pgs=347 cs=0 l=1 rev1=1 crypto rx=0x7f96d81547b0 tx=0x7f96c8009040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:21:40.649 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:21:40.648+0000 7f96cdffb640 1 -- 172.21.15.143:0/611330697 <== mon.0 v2:172.21.15.143:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172649 (secure 0 0 0) 0x7f96d4050980 con 0x7f96d806c0e0 2023-11-26T07:21:40.799 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:21:40.796+0000 7f96de1b4640 1 -- 172.21.15.143:0/611330697 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f96d80af7b0 con 0x7f96d806c0e0 2023-11-26T07:21:40.799 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:21:40.796+0000 7f96cdffb640 1 -- 172.21.15.143:0/611330697 <== mon.0 v2:172.21.15.143:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v47) v1 ==== 104+0+11 (secure 0 0 0) 0x7f96d4017020 con 0x7f96d806c0e0 2023-11-26T07:21:40.800 INFO:teuthology.orchestra.run.smithi143.stdout:pg_num: 32 2023-11-26T07:21:40.802 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:21:40.800+0000 7f96de1b4640 1 -- 172.21.15.143:0/611330697 >> [v2:172.21.15.143:6832/2150557544,v1:172.21.15.143:6833/2150557544] conn(0x7f96bc066650 msgr2=0x7f96bc068b10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:21:40.802 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:21:40.800+0000 7f96de1b4640 1 --2- 172.21.15.143:0/611330697 >> [v2:172.21.15.143:6832/2150557544,v1:172.21.15.143:6833/2150557544] conn(0x7f96bc066650 0x7f96bc068b10 secure :-1 s=READY pgs=347 cs=0 l=1 rev1=1 crypto rx=0x7f96d81547b0 tx=0x7f96c8009040 comp rx=0 tx=0).stop 2023-11-26T07:21:40.802 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:21:40.800+0000 7f96de1b4640 1 -- 172.21.15.143:0/611330697 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f96d806c0e0 msgr2=0x7f96d8151d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:21:40.802 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:21:40.800+0000 7f96de1b4640 1 --2- 172.21.15.143:0/611330697 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f96d806c0e0 0x7f96d8151d40 secure :-1 s=READY pgs=724 cs=0 l=1 rev1=1 crypto rx=0x7f96d400c420 tx=0x7f96d400c8e0 comp rx=0 tx=0).stop 2023-11-26T07:21:40.802 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:21:40.800+0000 7f96de1b4640 1 -- 172.21.15.143:0/611330697 shutdown_connections 2023-11-26T07:21:40.802 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:21:40.800+0000 7f96de1b4640 1 --2- 172.21.15.143:0/611330697 >> [v2:172.21.15.143:6832/2150557544,v1:172.21.15.143:6833/2150557544] conn(0x7f96bc066650 0x7f96bc068b10 unknown :-1 s=CLOSED pgs=347 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:21:40.803 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:21:40.800+0000 7f96de1b4640 1 --2- 172.21.15.143:0/611330697 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f96d80d0200 0x7f96d80c5330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:21:40.803 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:21:40.800+0000 7f96de1b4640 1 --2- 172.21.15.143:0/611330697 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f96d80b23f0 0x7f96d815f790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:21:40.803 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:21:40.800+0000 7f96de1b4640 1 --2- 172.21.15.143:0/611330697 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f96d806c0e0 0x7f96d8151d40 unknown :-1 s=CLOSED pgs=724 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:21:40.803 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:21:40.800+0000 7f96de1b4640 1 -- 172.21.15.143:0/611330697 >> 172.21.15.143:0/611330697 conn(0x7f96d80d0df0 msgr2=0x7f96d806e400 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:21:40.803 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:21:40.800+0000 7f96de1b4640 1 -- 172.21.15.143:0/611330697 shutdown_connections 2023-11-26T07:21:40.803 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:21:40.800+0000 7f96de1b4640 1 -- 172.21.15.143:0/611330697 wait complete. 2023-11-26T07:21:40.820 DEBUG:teuthology.orchestra.run.smithi143:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-11-26T07:21:40.961 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:21:40.960+0000 7f46d1ba1640 1 Processor -- start 2023-11-26T07:21:40.961 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:21:40.960+0000 7f46d1ba1640 1 -- start start 2023-11-26T07:21:40.962 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:21:40.960+0000 7f46d1ba1640 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7f46cc143ac0 0x7f46cc070370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:21:40.962 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:21:40.960+0000 7f46d1ba1640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f46cc0cf990 0x7f46cc0cd1b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:21:40.962 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:21:40.960+0000 7f46d1ba1640 1 -- --> v1:172.21.15.188:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f46cc0cc730 con 0x7f46cc0708b0 2023-11-26T07:21:40.962 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:21:40.960+0000 7f46d1ba1640 1 -- --> v2:172.21.15.188:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f46cc143ea0 con 0x7f46cc143ac0 2023-11-26T07:21:40.962 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:21:40.960+0000 7f46d1ba1640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f46cc070ee0 con 0x7f46cc0cf990 2023-11-26T07:21:40.963 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:21:40.960+0000 7f46cbfff640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f46cc0cf990 0x7f46cc0cd1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:21:40.963 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:21:40.960+0000 7f46cb7fe640 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7f46cc143ac0 0x7f46cc070370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:21:40.963 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:21:40.960+0000 7f46cbfff640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f46cc0cf990 0x7f46cc0cd1b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.188:3301/0 says I am v2:172.21.15.143:57648/0 (socket says 172.21.15.143:57648) 2023-11-26T07:21:40.963 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:21:40.960+0000 7f46cbfff640 1 -- 172.21.15.143:0/2177940375 learned_addr learned my addr 172.21.15.143:0/2177940375 (peer_addr_for_me v2:172.21.15.143:0/0) 2023-11-26T07:21:40.963 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:21:40.960+0000 7f46cbfff640 1 -- 172.21.15.143:0/2177940375 >> v2:172.21.15.188:3300/0 conn(0x7f46cc143ac0 msgr2=0x7f46cc070370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-26T07:21:40.964 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:21:40.960+0000 7f46cbfff640 1 --2- 172.21.15.143:0/2177940375 >> v2:172.21.15.188:3300/0 conn(0x7f46cc143ac0 0x7f46cc070370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:21:40.964 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:21:40.960+0000 7f46cbfff640 1 -- 172.21.15.143:0/2177940375 >> v1:172.21.15.188:6789/0 conn(0x7f46cc0708b0 legacy=0x7f46cc0cf280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:21:40.964 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:21:40.960+0000 7f46cbfff640 1 -- 172.21.15.143:0/2177940375 --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f46cc0711c0 con 0x7f46cc0cf990 2023-11-26T07:21:40.966 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:21:40.964+0000 7f46cbfff640 1 --2- 172.21.15.143:0/2177940375 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f46cc0cf990 0x7f46cc0cd1b0 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7f46c4009e50 tx=0x7f46c401ca10 comp rx=0 tx=0).ready entity=mon.2 client_cookie=755bf774fc382772 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:21:40.966 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:21:40.964+0000 7f46caffd640 1 -- 172.21.15.143:0/2177940375 <== mon.2 v2:172.21.15.188:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f46c40a57f0 con 0x7f46cc0cf990 2023-11-26T07:21:40.966 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:21:40.964+0000 7f46caffd640 1 -- 172.21.15.143:0/2177940375 <== mon.2 v2:172.21.15.188:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f46c40a5950 con 0x7f46cc0cf990 2023-11-26T07:21:40.966 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:21:40.964+0000 7f46caffd640 1 -- 172.21.15.143:0/2177940375 <== mon.2 v2:172.21.15.188:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f46c40a5ce0 con 0x7f46cc0cf990 2023-11-26T07:21:40.967 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:21:40.964+0000 7f46d1ba1640 1 -- 172.21.15.143:0/2177940375 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f46cc0cf990 msgr2=0x7f46cc0cd1b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:21:40.967 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:21:40.964+0000 7f46d1ba1640 1 --2- 172.21.15.143:0/2177940375 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f46cc0cf990 0x7f46cc0cd1b0 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7f46c4009e50 tx=0x7f46c401ca10 comp rx=0 tx=0).stop 2023-11-26T07:21:40.967 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:21:40.964+0000 7f46d1ba1640 1 -- 172.21.15.143:0/2177940375 shutdown_connections 2023-11-26T07:21:40.967 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:21:40.964+0000 7f46d1ba1640 1 --2- 172.21.15.143:0/2177940375 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f46cc0cf990 0x7f46cc0cd1b0 unknown :-1 s=CLOSED pgs=101 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:21:40.967 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:21:40.964+0000 7f46d1ba1640 1 --2- 172.21.15.143:0/2177940375 >> v2:172.21.15.188:3300/0 conn(0x7f46cc143ac0 0x7f46cc070370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:21:40.967 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:21:40.964+0000 7f46d1ba1640 1 -- 172.21.15.143:0/2177940375 >> 172.21.15.143:0/2177940375 conn(0x7f46cc06a890 msgr2=0x7f46cc06aca0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:21:40.968 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:21:40.964+0000 7f46d1ba1640 1 -- 172.21.15.143:0/2177940375 shutdown_connections 2023-11-26T07:21:40.968 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:21:40.964+0000 7f46d1ba1640 1 -- 172.21.15.143:0/2177940375 wait complete. 2023-11-26T07:21:40.968 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:21:40.968+0000 7f46d1ba1640 1 Processor -- start 2023-11-26T07:21:40.968 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:21:40.968+0000 7f46d1ba1640 1 -- start start 2023-11-26T07:21:40.971 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:21:40.968+0000 7f46d1ba1640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f46cc0708b0 0x7f46cc156d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:21:40.971 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:21:40.968+0000 7f46d1ba1640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f46cc0cf990 0x7f46cc139cb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:21:40.971 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:21:40.968+0000 7f46d1ba1640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f46cc143ac0 0x7f46cc15ba40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:21:40.971 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:21:40.968+0000 7f46d1ba1640 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f46cc0ad4b0 con 0x7f46cc0cf990 2023-11-26T07:21:40.971 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:21:40.968+0000 7f46d1ba1640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f46cc13d4b0 con 0x7f46cc143ac0 2023-11-26T07:21:40.971 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:21:40.968+0000 7f46d1ba1640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f46cc1381c0 con 0x7f46cc0708b0 2023-11-26T07:21:40.972 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:21:40.968+0000 7f46c3fff640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f46cc0cf990 0x7f46cc139cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:21:40.972 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:21:40.968+0000 7f46c3fff640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f46cc0cf990 0x7f46cc139cb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:35764/0 (socket says 172.21.15.143:35764) 2023-11-26T07:21:40.972 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:21:40.968+0000 7f46c3fff640 1 -- 172.21.15.143:0/943532721 learned_addr learned my addr 172.21.15.143:0/943532721 (peer_addr_for_me v2:172.21.15.143:0/0) 2023-11-26T07:21:40.972 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:21:40.968+0000 7f46cb7fe640 1 --2- 172.21.15.143:0/943532721 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f46cc0708b0 0x7f46cc156d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:21:40.972 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:21:40.968+0000 7f46cbfff640 1 --2- 172.21.15.143:0/943532721 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f46cc143ac0 0x7f46cc15ba40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:21:40.972 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:21:40.968+0000 7f46c3fff640 1 -- 172.21.15.143:0/943532721 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f46cc0708b0 msgr2=0x7f46cc156d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:21:40.972 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:21:40.968+0000 7f46c3fff640 1 --2- 172.21.15.143:0/943532721 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f46cc0708b0 0x7f46cc156d10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:21:40.972 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:21:40.968+0000 7f46c3fff640 1 -- 172.21.15.143:0/943532721 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f46cc143ac0 msgr2=0x7f46cc15ba40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:21:40.972 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:21:40.968+0000 7f46c3fff640 1 --2- 172.21.15.143:0/943532721 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f46cc143ac0 0x7f46cc15ba40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:21:40.972 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:21:40.968+0000 7f46c3fff640 1 -- 172.21.15.143:0/943532721 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f46c4009b30 con 0x7f46cc0cf990 2023-11-26T07:21:40.973 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:21:40.968+0000 7f46c3fff640 1 --2- 172.21.15.143:0/943532721 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f46cc0cf990 0x7f46cc139cb0 secure :-1 s=READY pgs=725 cs=0 l=1 rev1=1 crypto rx=0x7f46b400c450 tx=0x7f46b400c910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:21:40.973 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:21:40.968+0000 7f46c8ff9640 1 -- 172.21.15.143:0/943532721 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f46b40052d0 con 0x7f46cc0cf990 2023-11-26T07:21:40.973 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:21:40.968+0000 7f46c8ff9640 1 -- 172.21.15.143:0/943532721 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f46b4005450 con 0x7f46cc0cf990 2023-11-26T07:21:40.973 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:21:40.968+0000 7f46d1ba1640 1 -- 172.21.15.143:0/943532721 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f46cc1384a0 con 0x7f46cc0cf990 2023-11-26T07:21:40.973 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:21:40.968+0000 7f46c8ff9640 1 -- 172.21.15.143:0/943532721 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f46b4012870 con 0x7f46cc0cf990 2023-11-26T07:21:40.973 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:21:40.968+0000 7f46d1ba1640 1 -- 172.21.15.143:0/943532721 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f46cc0ad940 con 0x7f46cc0cf990 2023-11-26T07:21:40.974 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:21:40.968+0000 7f46d1ba1640 1 -- 172.21.15.143:0/943532721 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f46cc067680 con 0x7f46cc0cf990 2023-11-26T07:21:40.974 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:21:40.972+0000 7f46c8ff9640 1 -- 172.21.15.143:0/943532721 <== mon.0 v2:172.21.15.143:3300/0 4 ==== mgrmap(e 9) v1 ==== 86005+0+0 (secure 0 0 0) 0x7f46b40129d0 con 0x7f46cc0cf990 2023-11-26T07:21:40.974 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:21:40.972+0000 7f46c8ff9640 1 --2- 172.21.15.143:0/943532721 >> [v2:172.21.15.143:6832/2150557544,v1:172.21.15.143:6833/2150557544] conn(0x7f46ac066620 0x7f46ac068ae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:21:40.974 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:21:40.972+0000 7f46c8ff9640 1 -- 172.21.15.143:0/943532721 <== mon.0 v2:172.21.15.143:3300/0 5 ==== osd_map(47..47 src has 1..47) v4 ==== 5767+0+0 (secure 0 0 0) 0x7f46b4083bd0 con 0x7f46cc0cf990 2023-11-26T07:21:40.974 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:21:40.972+0000 7f46cb7fe640 1 --2- 172.21.15.143:0/943532721 >> [v2:172.21.15.143:6832/2150557544,v1:172.21.15.143:6833/2150557544] conn(0x7f46ac066620 0x7f46ac068ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:21:40.975 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:21:40.972+0000 7f46cb7fe640 1 --2- 172.21.15.143:0/943532721 >> [v2:172.21.15.143:6832/2150557544,v1:172.21.15.143:6833/2150557544] conn(0x7f46ac066620 0x7f46ac068ae0 secure :-1 s=READY pgs=348 cs=0 l=1 rev1=1 crypto rx=0x7f46bc00b240 tx=0x7f46bc00b3a0 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:21:40.977 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:21:40.976+0000 7f46c8ff9640 1 -- 172.21.15.143:0/943532721 <== mon.0 v2:172.21.15.143:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172649 (secure 0 0 0) 0x7f46b4050e30 con 0x7f46cc0cf990 2023-11-26T07:21:41.126 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:21:41.124+0000 7f46d1ba1640 1 -- 172.21.15.143:0/943532721 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f46cc0ade20 con 0x7f46cc0cf990 2023-11-26T07:21:41.126 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:21:41.124+0000 7f46c8ff9640 1 -- 172.21.15.143:0/943532721 <== mon.0 v2:172.21.15.143:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v47) v1 ==== 100+0+11 (secure 0 0 0) 0x7f46b4054450 con 0x7f46cc0cf990 2023-11-26T07:21:41.126 INFO:teuthology.orchestra.run.smithi143.stdout:pg_num: 64 2023-11-26T07:21:41.128 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:21:41.124+0000 7f46d1ba1640 1 -- 172.21.15.143:0/943532721 >> [v2:172.21.15.143:6832/2150557544,v1:172.21.15.143:6833/2150557544] conn(0x7f46ac066620 msgr2=0x7f46ac068ae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:21:41.129 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:21:41.124+0000 7f46d1ba1640 1 --2- 172.21.15.143:0/943532721 >> [v2:172.21.15.143:6832/2150557544,v1:172.21.15.143:6833/2150557544] conn(0x7f46ac066620 0x7f46ac068ae0 secure :-1 s=READY pgs=348 cs=0 l=1 rev1=1 crypto rx=0x7f46bc00b240 tx=0x7f46bc00b3a0 comp rx=0 tx=0).stop 2023-11-26T07:21:41.129 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:21:41.124+0000 7f46d1ba1640 1 -- 172.21.15.143:0/943532721 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f46cc0cf990 msgr2=0x7f46cc139cb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:21:41.129 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:21:41.124+0000 7f46d1ba1640 1 --2- 172.21.15.143:0/943532721 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f46cc0cf990 0x7f46cc139cb0 secure :-1 s=READY pgs=725 cs=0 l=1 rev1=1 crypto rx=0x7f46b400c450 tx=0x7f46b400c910 comp rx=0 tx=0).stop 2023-11-26T07:21:41.129 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:21:41.124+0000 7f46d1ba1640 1 -- 172.21.15.143:0/943532721 shutdown_connections 2023-11-26T07:21:41.129 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:21:41.124+0000 7f46d1ba1640 1 --2- 172.21.15.143:0/943532721 >> [v2:172.21.15.143:6832/2150557544,v1:172.21.15.143:6833/2150557544] conn(0x7f46ac066620 0x7f46ac068ae0 unknown :-1 s=CLOSED pgs=348 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:21:41.129 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:21:41.124+0000 7f46d1ba1640 1 --2- 172.21.15.143:0/943532721 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f46cc143ac0 0x7f46cc15ba40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:21:41.129 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:21:41.124+0000 7f46d1ba1640 1 --2- 172.21.15.143:0/943532721 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f46cc0cf990 0x7f46cc139cb0 unknown :-1 s=CLOSED pgs=725 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:21:41.129 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:21:41.124+0000 7f46d1ba1640 1 --2- 172.21.15.143:0/943532721 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f46cc0708b0 0x7f46cc156d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:21:41.129 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:21:41.124+0000 7f46d1ba1640 1 -- 172.21.15.143:0/943532721 >> 172.21.15.143:0/943532721 conn(0x7f46cc06a890 msgr2=0x7f46cc15a620 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:21:41.129 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:21:41.128+0000 7f46d1ba1640 1 -- 172.21.15.143:0/943532721 shutdown_connections 2023-11-26T07:21:41.129 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:21:41.128+0000 7f46d1ba1640 1 -- 172.21.15.143:0/943532721 wait complete. 2023-11-26T07:21:41.148 DEBUG:teuthology.orchestra.run.smithi143:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-11-26T07:21:41.285 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:21:41.284+0000 7f32f6495640 1 Processor -- start 2023-11-26T07:21:41.285 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:21:41.284+0000 7f32f6495640 1 -- start start 2023-11-26T07:21:41.287 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:21:41.284+0000 7f32f6495640 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7f32f015d2a0 0x7f32f00d00e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:21:41.287 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:21:41.284+0000 7f32f6495640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f32f015d680 0x7f32f00cec30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:21:41.287 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:21:41.284+0000 7f32f6495640 1 -- --> v1:172.21.15.143:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f32f00cdc40 con 0x7f32f00d0e80 2023-11-26T07:21:41.287 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:21:41.284+0000 7f32f6495640 1 -- --> v2:172.21.15.143:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f32f00ce7a0 con 0x7f32f015d2a0 2023-11-26T07:21:41.287 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:21:41.284+0000 7f32f6495640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f32f00d1440 con 0x7f32f015d680 2023-11-26T07:21:41.287 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:21:41.284+0000 7f32f5493640 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7f32f015d2a0 0x7f32f00d00e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:21:41.287 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:21:41.284+0000 7f32f5c94640 1 --1- >> v1:172.21.15.143:6789/0 conn(0x7f32f00d0e80 0x7f32f00d1680 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.143:6789/0 says I am v1:172.21.15.143:36158/0 (socket says 172.21.15.143:36158) 2023-11-26T07:21:41.287 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:21:41.284+0000 7f32f5c94640 1 -- 172.21.15.143:0/2999664825 learned_addr learned my addr 172.21.15.143:0/2999664825 (peer_addr_for_me v1:172.21.15.143:0/0) 2023-11-26T07:21:41.287 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:21:41.284+0000 7f32f4c92640 1 --2- 172.21.15.143:0/2999664825 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f32f015d680 0x7f32f00cec30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:21:41.287 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:21:41.284+0000 7f32f5493640 1 -- 172.21.15.143:0/2999664825 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f32f015d680 msgr2=0x7f32f00cec30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-26T07:21:41.288 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:21:41.284+0000 7f32f5493640 1 --2- 172.21.15.143:0/2999664825 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f32f015d680 0x7f32f00cec30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:21:41.288 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:21:41.284+0000 7f32e7fff640 1 -- 172.21.15.143:0/2999664825 <== mon.0 v1:172.21.15.143:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 380852570 0 0) 0x7f32f00cdc40 con 0x7f32f00d0e80 2023-11-26T07:21:41.288 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:21:41.284+0000 7f32f5493640 1 -- 172.21.15.143:0/2999664825 >> v1:172.21.15.143:6789/0 conn(0x7f32f00d0e80 legacy=0x7f32f00d1680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:21:41.288 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:21:41.284+0000 7f32f5493640 1 -- 172.21.15.143:0/2999664825 --> v2:172.21.15.143:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f32f00cde70 con 0x7f32f015d2a0 2023-11-26T07:21:41.289 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:21:41.288+0000 7f32f5493640 1 --2- 172.21.15.143:0/2999664825 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f32f015d2a0 0x7f32f00d00e0 secure :-1 s=READY pgs=727 cs=0 l=1 rev1=1 crypto rx=0x7f32ec0097c0 tx=0x7f32ec01c3c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f93e10794e62c16b server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:21:41.290 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:21:41.288+0000 7f32e7fff640 1 -- 172.21.15.143:0/2999664825 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f32ec0ac620 con 0x7f32f015d2a0 2023-11-26T07:21:41.290 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:21:41.288+0000 7f32e7fff640 1 -- 172.21.15.143:0/2999664825 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f32ec0ac780 con 0x7f32f015d2a0 2023-11-26T07:21:41.290 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:21:41.288+0000 7f32e7fff640 1 -- 172.21.15.143:0/2999664825 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f32ec0acb10 con 0x7f32f015d2a0 2023-11-26T07:21:41.291 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:21:41.288+0000 7f32f6495640 1 -- 172.21.15.143:0/2999664825 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f32f015d2a0 msgr2=0x7f32f00d00e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:21:41.291 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:21:41.288+0000 7f32f6495640 1 --2- 172.21.15.143:0/2999664825 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f32f015d2a0 0x7f32f00d00e0 secure :-1 s=READY pgs=727 cs=0 l=1 rev1=1 crypto rx=0x7f32ec0097c0 tx=0x7f32ec01c3c0 comp rx=0 tx=0).stop 2023-11-26T07:21:41.291 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:21:41.288+0000 7f32f6495640 1 -- 172.21.15.143:0/2999664825 shutdown_connections 2023-11-26T07:21:41.291 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:21:41.288+0000 7f32f6495640 1 --2- 172.21.15.143:0/2999664825 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f32f015d680 0x7f32f00cec30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:21:41.291 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:21:41.288+0000 7f32f6495640 1 --2- 172.21.15.143:0/2999664825 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f32f015d2a0 0x7f32f00d00e0 unknown :-1 s=CLOSED pgs=727 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:21:41.291 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:21:41.288+0000 7f32f6495640 1 -- 172.21.15.143:0/2999664825 >> 172.21.15.143:0/2999664825 conn(0x7f32f00d30a0 msgr2=0x7f32f00d3480 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:21:41.291 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:21:41.288+0000 7f32f6495640 1 -- 172.21.15.143:0/2999664825 shutdown_connections 2023-11-26T07:21:41.291 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:21:41.288+0000 7f32f6495640 1 -- 172.21.15.143:0/2999664825 wait complete. 2023-11-26T07:21:41.292 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:21:41.292+0000 7f32f6495640 1 Processor -- start 2023-11-26T07:21:41.292 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:21:41.292+0000 7f32f6495640 1 -- start start 2023-11-26T07:21:41.294 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:21:41.292+0000 7f32f6495640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f32f00d0e80 0x7f32f015edd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:21:41.294 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:21:41.292+0000 7f32f6495640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f32f015d2a0 0x7f32f015f310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:21:41.294 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:21:41.292+0000 7f32f6495640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f32f015d680 0x7f32f006afd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:21:41.294 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:21:41.292+0000 7f32f6495640 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f32f00c6a90 con 0x7f32f015d2a0 2023-11-26T07:21:41.294 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:21:41.292+0000 7f32f6495640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f32f0156b40 con 0x7f32f015d680 2023-11-26T07:21:41.295 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:21:41.292+0000 7f32f6495640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f32f00c6ce0 con 0x7f32f00d0e80 2023-11-26T07:21:41.295 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:21:41.292+0000 7f32f4c92640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f32f015d2a0 0x7f32f015f310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:21:41.295 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:21:41.292+0000 7f32f4c92640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f32f015d2a0 0x7f32f015f310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:35782/0 (socket says 172.21.15.143:35782) 2023-11-26T07:21:41.295 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:21:41.292+0000 7f32f4c92640 1 -- 172.21.15.143:0/533300459 learned_addr learned my addr 172.21.15.143:0/533300459 (peer_addr_for_me v2:172.21.15.143:0/0) 2023-11-26T07:21:41.295 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:21:41.292+0000 7f32f5493640 1 --2- 172.21.15.143:0/533300459 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f32f00d0e80 0x7f32f015edd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:21:41.295 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:21:41.292+0000 7f32f5c94640 1 --2- 172.21.15.143:0/533300459 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f32f015d680 0x7f32f006afd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:21:41.295 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:21:41.292+0000 7f32f4c92640 1 -- 172.21.15.143:0/533300459 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f32f00d0e80 msgr2=0x7f32f015edd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:21:41.295 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:21:41.292+0000 7f32f4c92640 1 --2- 172.21.15.143:0/533300459 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f32f00d0e80 0x7f32f015edd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:21:41.295 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:21:41.292+0000 7f32f4c92640 1 -- 172.21.15.143:0/533300459 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f32f015d680 msgr2=0x7f32f006afd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:21:41.295 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:21:41.292+0000 7f32f4c92640 1 --2- 172.21.15.143:0/533300459 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f32f015d680 0x7f32f006afd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:21:41.295 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:21:41.292+0000 7f32f4c92640 1 -- 172.21.15.143:0/533300459 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f32ec009420 con 0x7f32f015d2a0 2023-11-26T07:21:41.295 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:21:41.292+0000 7f32f5c94640 1 --2- 172.21.15.143:0/533300459 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f32f015d680 0x7f32f006afd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-26T07:21:41.296 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:21:41.292+0000 7f32f4c92640 1 --2- 172.21.15.143:0/533300459 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f32f015d2a0 0x7f32f015f310 secure :-1 s=READY pgs=728 cs=0 l=1 rev1=1 crypto rx=0x7f32e000d7d0 tx=0x7f32e000dc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:21:41.296 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:21:41.292+0000 7f32e5ffb640 1 -- 172.21.15.143:0/533300459 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f32e000b6c0 con 0x7f32f015d2a0 2023-11-26T07:21:41.296 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:21:41.292+0000 7f32e5ffb640 1 -- 172.21.15.143:0/533300459 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f32e000b820 con 0x7f32f015d2a0 2023-11-26T07:21:41.296 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:21:41.292+0000 7f32f6495640 1 -- 172.21.15.143:0/533300459 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f32f00c6fc0 con 0x7f32f015d2a0 2023-11-26T07:21:41.297 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:21:41.292+0000 7f32e5ffb640 1 -- 172.21.15.143:0/533300459 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f32e000bbb0 con 0x7f32f015d2a0 2023-11-26T07:21:41.297 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:21:41.292+0000 7f32f6495640 1 -- 172.21.15.143:0/533300459 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f32f00c7580 con 0x7f32f015d2a0 2023-11-26T07:21:41.297 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:21:41.292+0000 7f32f6495640 1 -- 172.21.15.143:0/533300459 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f32f00c7740 con 0x7f32f015d2a0 2023-11-26T07:21:41.297 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:21:41.296+0000 7f32e5ffb640 1 -- 172.21.15.143:0/533300459 <== mon.0 v2:172.21.15.143:3300/0 4 ==== mgrmap(e 9) v1 ==== 86005+0+0 (secure 0 0 0) 0x7f32e000bd10 con 0x7f32f015d2a0 2023-11-26T07:21:41.298 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:21:41.296+0000 7f32e5ffb640 1 --2- 172.21.15.143:0/533300459 >> [v2:172.21.15.143:6832/2150557544,v1:172.21.15.143:6833/2150557544] conn(0x7f32d4066620 0x7f32d4068ae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:21:41.298 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:21:41.296+0000 7f32e5ffb640 1 -- 172.21.15.143:0/533300459 <== mon.0 v2:172.21.15.143:3300/0 5 ==== osd_map(47..47 src has 1..47) v4 ==== 5767+0+0 (secure 0 0 0) 0x7f32e00846c0 con 0x7f32f015d2a0 2023-11-26T07:21:41.298 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:21:41.296+0000 7f32f5493640 1 --2- 172.21.15.143:0/533300459 >> [v2:172.21.15.143:6832/2150557544,v1:172.21.15.143:6833/2150557544] conn(0x7f32d4066620 0x7f32d4068ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:21:41.299 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:21:41.296+0000 7f32f5493640 1 --2- 172.21.15.143:0/533300459 >> [v2:172.21.15.143:6832/2150557544,v1:172.21.15.143:6833/2150557544] conn(0x7f32d4066620 0x7f32d4068ae0 secure :-1 s=READY pgs=349 cs=0 l=1 rev1=1 crypto rx=0x7f32ec0ae000 tx=0x7f32ec0a7000 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:21:41.302 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:21:41.300+0000 7f32e5ffb640 1 -- 172.21.15.143:0/533300459 <== mon.0 v2:172.21.15.143:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172649 (secure 0 0 0) 0x7f32e00517e0 con 0x7f32f015d2a0 2023-11-26T07:21:41.482 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:21:41.480+0000 7f32f6495640 1 -- 172.21.15.143:0/533300459 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f32f00d3520 con 0x7f32f015d2a0 2023-11-26T07:21:41.482 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:21:41.480+0000 7f32e5ffb640 1 -- 172.21.15.143:0/533300459 <== mon.0 v2:172.21.15.143:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 26 v26) v1 ==== 94+0+4922 (secure 0 0 0) 0x7f32e0054e00 con 0x7f32f015d2a0 2023-11-26T07:21:41.483 INFO:teuthology.orchestra.run.smithi143.stdout: 2023-11-26T07:21:41.483 INFO:teuthology.orchestra.run.smithi143.stdout:{"epoch":26,"default_fscid":3,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5201,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.188:6835/319464848","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6833","nonce":319464848},{"type":"v1","addr":"172.21.15.188:6835","nonce":319464848}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":20},{"gid":5584,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.188:6834/2113408070","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6832","nonce":2113408070},{"type":"v1","addr":"172.21.15.188:6834","nonce":2113408070}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":20},{"gid":5745,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.143:6835/1616742593","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6834","nonce":1616742593},{"type":"v1","addr":"172.21.15.143:6835","nonce":1616742593}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":20}],"filesystems":[{"mdsmap":{"epoch":25,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-11-26T07:19:31.503587+0000","modified":"2023-11-26T07:19:34.523605+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":5748},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5748":{"gid":5748,"name":"c","rank":0,"incarnation":21,"state":"up:active","state_seq":3,"addr":"172.21.15.143:6837/2872033338","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6836","nonce":2872033338},{"type":"v1","addr":"172.21.15.143:6837","nonce":2872033338}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[7],"metadata_pool":6,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":3}]} 2023-11-26T07:21:41.483 INFO:teuthology.orchestra.run.smithi143.stderr:dumped fsmap epoch 26 2023-11-26T07:21:41.484 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:21:41.480+0000 7f32f6495640 1 -- 172.21.15.143:0/533300459 >> [v2:172.21.15.143:6832/2150557544,v1:172.21.15.143:6833/2150557544] conn(0x7f32d4066620 msgr2=0x7f32d4068ae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:21:41.485 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:21:41.480+0000 7f32f6495640 1 --2- 172.21.15.143:0/533300459 >> [v2:172.21.15.143:6832/2150557544,v1:172.21.15.143:6833/2150557544] conn(0x7f32d4066620 0x7f32d4068ae0 secure :-1 s=READY pgs=349 cs=0 l=1 rev1=1 crypto rx=0x7f32ec0ae000 tx=0x7f32ec0a7000 comp rx=0 tx=0).stop 2023-11-26T07:21:41.485 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:21:41.480+0000 7f32f6495640 1 -- 172.21.15.143:0/533300459 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f32f015d2a0 msgr2=0x7f32f015f310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:21:41.485 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:21:41.480+0000 7f32f6495640 1 --2- 172.21.15.143:0/533300459 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f32f015d2a0 0x7f32f015f310 secure :-1 s=READY pgs=728 cs=0 l=1 rev1=1 crypto rx=0x7f32e000d7d0 tx=0x7f32e000dc90 comp rx=0 tx=0).stop 2023-11-26T07:21:41.485 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:21:41.480+0000 7f32f6495640 1 -- 172.21.15.143:0/533300459 shutdown_connections 2023-11-26T07:21:41.485 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:21:41.480+0000 7f32f6495640 1 --2- 172.21.15.143:0/533300459 >> [v2:172.21.15.143:6832/2150557544,v1:172.21.15.143:6833/2150557544] conn(0x7f32d4066620 0x7f32d4068ae0 unknown :-1 s=CLOSED pgs=349 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:21:41.485 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:21:41.480+0000 7f32f6495640 1 --2- 172.21.15.143:0/533300459 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f32f015d680 0x7f32f006afd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:21:41.485 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:21:41.480+0000 7f32f6495640 1 --2- 172.21.15.143:0/533300459 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f32f015d2a0 0x7f32f015f310 unknown :-1 s=CLOSED pgs=728 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:21:41.485 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:21:41.480+0000 7f32f6495640 1 --2- 172.21.15.143:0/533300459 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f32f00d0e80 0x7f32f015edd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:21:41.485 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:21:41.480+0000 7f32f6495640 1 -- 172.21.15.143:0/533300459 >> 172.21.15.143:0/533300459 conn(0x7f32f00d30a0 msgr2=0x7f32f00cfb20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:21:41.485 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:21:41.484+0000 7f32f6495640 1 -- 172.21.15.143:0/533300459 shutdown_connections 2023-11-26T07:21:41.485 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:21:41.484+0000 7f32f6495640 1 -- 172.21.15.143:0/533300459 wait complete. 2023-11-26T07:21:41.504 DEBUG:teuthology.orchestra.run.smithi143:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-11-26T07:21:41.643 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:21:41.640+0000 7fe908632640 1 Processor -- start 2023-11-26T07:21:41.643 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:21:41.640+0000 7fe908632640 1 -- start start 2023-11-26T07:21:41.645 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:21:41.644+0000 7fe908632640 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7fe90013e290 0x7fe9000cf280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:21:41.646 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:21:41.644+0000 7fe908632640 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7fe90015f6b0 0x7fe9000cd7b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:21:41.646 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:21:41.644+0000 7fe908632640 1 -- --> v1:172.21.15.188:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe90015fb20 con 0x7fe9000cf7c0 2023-11-26T07:21:41.646 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:21:41.644+0000 7fe908632640 1 -- --> v2:172.21.15.143:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe9000cfc40 con 0x7fe90013e290 2023-11-26T07:21:41.646 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:21:41.644+0000 7fe908632640 1 -- --> v2:172.21.15.188:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe90006c520 con 0x7fe90015f6b0 2023-11-26T07:21:41.646 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:21:41.644+0000 7fe9063ce640 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7fe90013e290 0x7fe9000cf280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:21:41.646 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:21:41.644+0000 7fe9063ce640 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7fe90013e290 0x7fe9000cf280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:35784/0 (socket says 172.21.15.143:35784) 2023-11-26T07:21:41.646 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:21:41.644+0000 7fe9063ce640 1 -- 172.21.15.143:0/3965661546 learned_addr learned my addr 172.21.15.143:0/3965661546 (peer_addr_for_me v2:172.21.15.143:0/0) 2023-11-26T07:21:41.646 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:21:41.644+0000 7fe906bcf640 1 --2- 172.21.15.143:0/3965661546 >> v2:172.21.15.188:3300/0 conn(0x7fe90015f6b0 0x7fe9000cd7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:21:41.646 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:21:41.644+0000 7fe9063ce640 1 -- 172.21.15.143:0/3965661546 >> v2:172.21.15.188:3300/0 conn(0x7fe90015f6b0 msgr2=0x7fe9000cd7b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-26T07:21:41.646 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:21:41.644+0000 7fe9063ce640 1 --2- 172.21.15.143:0/3965661546 >> v2:172.21.15.188:3300/0 conn(0x7fe90015f6b0 0x7fe9000cd7b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:21:41.646 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:21:41.644+0000 7fe9063ce640 1 -- 172.21.15.143:0/3965661546 >> v1:172.21.15.188:6789/0 conn(0x7fe9000cf7c0 legacy=0x7fe90015efa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:21:41.646 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:21:41.644+0000 7fe9063ce640 1 -- 172.21.15.143:0/3965661546 --> v2:172.21.15.143:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe90006c800 con 0x7fe90013e290 2023-11-26T07:21:41.647 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:21:41.644+0000 7fe9053cc640 1 -- 172.21.15.143:0/3965661546 <== mon.1 v1:172.21.15.188:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2604329746 0 0) 0x7fe8f0001120 con 0x7fe9000cf7c0 2023-11-26T07:21:41.648 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:21:41.644+0000 7fe9063ce640 1 --2- 172.21.15.143:0/3965661546 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fe90013e290 0x7fe9000cf280 secure :-1 s=READY pgs=729 cs=0 l=1 rev1=1 crypto rx=0x7fe90006e780 tx=0x7fe8fc01c4d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=72295341258e1122 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:21:41.648 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:21:41.648+0000 7fe9053cc640 1 -- 172.21.15.143:0/3965661546 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe8fc0aa430 con 0x7fe90013e290 2023-11-26T07:21:41.649 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:21:41.648+0000 7fe9053cc640 1 -- 172.21.15.143:0/3965661546 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe8fc0aa590 con 0x7fe90013e290 2023-11-26T07:21:41.649 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:21:41.648+0000 7fe9053cc640 1 -- 172.21.15.143:0/3965661546 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe8fc0aa920 con 0x7fe90013e290 2023-11-26T07:21:41.649 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:21:41.648+0000 7fe908632640 1 -- 172.21.15.143:0/3965661546 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fe90013e290 msgr2=0x7fe9000cf280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:21:41.649 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:21:41.648+0000 7fe908632640 1 --2- 172.21.15.143:0/3965661546 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fe90013e290 0x7fe9000cf280 secure :-1 s=READY pgs=729 cs=0 l=1 rev1=1 crypto rx=0x7fe90006e780 tx=0x7fe8fc01c4d0 comp rx=0 tx=0).stop 2023-11-26T07:21:41.649 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:21:41.648+0000 7fe908632640 1 -- 172.21.15.143:0/3965661546 shutdown_connections 2023-11-26T07:21:41.650 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:21:41.648+0000 7fe908632640 1 --2- 172.21.15.143:0/3965661546 >> v2:172.21.15.188:3300/0 conn(0x7fe90015f6b0 0x7fe9000cd7b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:21:41.650 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:21:41.648+0000 7fe908632640 1 --2- 172.21.15.143:0/3965661546 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fe90013e290 0x7fe9000cf280 unknown :-1 s=CLOSED pgs=729 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:21:41.650 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:21:41.648+0000 7fe908632640 1 -- 172.21.15.143:0/3965661546 >> 172.21.15.143:0/3965661546 conn(0x7fe90006a890 msgr2=0x7fe90006aca0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:21:41.650 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:21:41.648+0000 7fe908632640 1 -- 172.21.15.143:0/3965661546 shutdown_connections 2023-11-26T07:21:41.650 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:21:41.648+0000 7fe908632640 1 -- 172.21.15.143:0/3965661546 wait complete. 2023-11-26T07:21:41.651 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:21:41.648+0000 7fe908632640 1 Processor -- start 2023-11-26T07:21:41.651 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:21:41.648+0000 7fe908632640 1 -- start start 2023-11-26T07:21:41.652 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:21:41.648+0000 7fe908632640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fe9000cf7c0 0x7fe9001542a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:21:41.652 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:21:41.648+0000 7fe908632640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fe90013e290 0x7fe90013c970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:21:41.652 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:21:41.648+0000 7fe908632640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fe90015f6b0 0x7fe90013ceb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:21:41.652 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:21:41.648+0000 7fe908632640 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe900154e20 con 0x7fe9000cf7c0 2023-11-26T07:21:41.653 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:21:41.648+0000 7fe908632640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe900133580 con 0x7fe90013e290 2023-11-26T07:21:41.653 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:21:41.648+0000 7fe908632640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe9001332a0 con 0x7fe90015f6b0 2023-11-26T07:21:41.653 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:21:41.648+0000 7fe9063ce640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fe9000cf7c0 0x7fe9001542a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:21:41.653 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:21:41.648+0000 7fe9063ce640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fe9000cf7c0 0x7fe9001542a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:35792/0 (socket says 172.21.15.143:35792) 2023-11-26T07:21:41.653 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:21:41.648+0000 7fe9063ce640 1 -- 172.21.15.143:0/1964878004 learned_addr learned my addr 172.21.15.143:0/1964878004 (peer_addr_for_me v2:172.21.15.143:0/0) 2023-11-26T07:21:41.653 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:21:41.648+0000 7fe905bcd640 1 --2- 172.21.15.143:0/1964878004 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fe90013e290 0x7fe90013c970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:21:41.653 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:21:41.648+0000 7fe906bcf640 1 --2- 172.21.15.143:0/1964878004 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fe90015f6b0 0x7fe90013ceb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:21:41.653 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:21:41.652+0000 7fe9063ce640 1 -- 172.21.15.143:0/1964878004 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fe90015f6b0 msgr2=0x7fe90013ceb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:21:41.653 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:21:41.652+0000 7fe9063ce640 1 --2- 172.21.15.143:0/1964878004 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fe90015f6b0 0x7fe90013ceb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:21:41.653 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:21:41.652+0000 7fe9063ce640 1 -- 172.21.15.143:0/1964878004 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fe90013e290 msgr2=0x7fe90013c970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:21:41.653 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:21:41.652+0000 7fe9063ce640 1 --2- 172.21.15.143:0/1964878004 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fe90013e290 0x7fe90013c970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:21:41.653 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:21:41.652+0000 7fe9063ce640 1 -- 172.21.15.143:0/1964878004 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe8fc0094d0 con 0x7fe9000cf7c0 2023-11-26T07:21:41.654 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:21:41.652+0000 7fe906bcf640 1 --2- 172.21.15.143:0/1964878004 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fe90015f6b0 0x7fe90013ceb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-26T07:21:41.654 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:21:41.652+0000 7fe905bcd640 1 --2- 172.21.15.143:0/1964878004 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fe90013e290 0x7fe90013c970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-26T07:21:41.654 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:21:41.652+0000 7fe9063ce640 1 --2- 172.21.15.143:0/1964878004 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fe9000cf7c0 0x7fe9001542a0 secure :-1 s=READY pgs=730 cs=0 l=1 rev1=1 crypto rx=0x7fe8fc0ac040 tx=0x7fe8fc01ad10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:21:41.654 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:21:41.652+0000 7fe8f6ffd640 1 -- 172.21.15.143:0/1964878004 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe8fc0aa430 con 0x7fe9000cf7c0 2023-11-26T07:21:41.655 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:21:41.652+0000 7fe8f6ffd640 1 -- 172.21.15.143:0/1964878004 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe8fc0b3820 con 0x7fe9000cf7c0 2023-11-26T07:21:41.655 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:21:41.652+0000 7fe908632640 1 -- 172.21.15.143:0/1964878004 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe900135ab0 con 0x7fe9000cf7c0 2023-11-26T07:21:41.655 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:21:41.652+0000 7fe8f6ffd640 1 -- 172.21.15.143:0/1964878004 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe8fc0b3bd0 con 0x7fe9000cf7c0 2023-11-26T07:21:41.655 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:21:41.652+0000 7fe908632640 1 -- 172.21.15.143:0/1964878004 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe900136070 con 0x7fe9000cf7c0 2023-11-26T07:21:41.655 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:21:41.652+0000 7fe908632640 1 -- 172.21.15.143:0/1964878004 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe900135910 con 0x7fe9000cf7c0 2023-11-26T07:21:41.655 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:21:41.652+0000 7fe8f6ffd640 1 -- 172.21.15.143:0/1964878004 <== mon.0 v2:172.21.15.143:3300/0 4 ==== mgrmap(e 9) v1 ==== 86005+0+0 (secure 0 0 0) 0x7fe8fc0b3d30 con 0x7fe9000cf7c0 2023-11-26T07:21:41.655 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:21:41.652+0000 7fe8f6ffd640 1 --2- 172.21.15.143:0/1964878004 >> [v2:172.21.15.143:6832/2150557544,v1:172.21.15.143:6833/2150557544] conn(0x7fe8e4066650 0x7fe8e4068b10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:21:41.656 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:21:41.652+0000 7fe8f6ffd640 1 -- 172.21.15.143:0/1964878004 <== mon.0 v2:172.21.15.143:3300/0 5 ==== osd_map(47..47 src has 1..47) v4 ==== 5767+0+0 (secure 0 0 0) 0x7fe8fc11e750 con 0x7fe9000cf7c0 2023-11-26T07:21:41.656 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:21:41.652+0000 7fe905bcd640 1 --2- 172.21.15.143:0/1964878004 >> [v2:172.21.15.143:6832/2150557544,v1:172.21.15.143:6833/2150557544] conn(0x7fe8e4066650 0x7fe8e4068b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:21:41.656 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:21:41.656+0000 7fe905bcd640 1 --2- 172.21.15.143:0/1964878004 >> [v2:172.21.15.143:6832/2150557544,v1:172.21.15.143:6833/2150557544] conn(0x7fe8e4066650 0x7fe8e4068b10 secure :-1 s=READY pgs=350 cs=0 l=1 rev1=1 crypto rx=0x7fe9001f7180 tx=0x7fe8f0009040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:21:41.660 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:21:41.656+0000 7fe8f6ffd640 1 -- 172.21.15.143:0/1964878004 <== mon.0 v2:172.21.15.143:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172649 (secure 0 0 0) 0x7fe900135910 con 0x7fe9000cf7c0 2023-11-26T07:21:41.813 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:21:41.812+0000 7fe908632640 1 -- 172.21.15.143:0/1964878004 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fe900135910 con 0x7fe9000cf7c0 2023-11-26T07:21:41.814 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:21:41.812+0000 7fe8f6ffd640 1 -- 172.21.15.143:0/1964878004 <== mon.0 v2:172.21.15.143:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v47) v1 ==== 74+0+16645 (secure 0 0 0) 0x7fe900135910 con 0x7fe9000cf7c0 2023-11-26T07:21:41.814 INFO:teuthology.orchestra.run.smithi143.stdout: 2023-11-26T07:21:41.814 INFO:teuthology.orchestra.run.smithi143.stdout:{"epoch":47,"fsid":"f5033060-2469-4a07-95fe-2b76f5404bc1","created":"2023-11-26T07:17:28.613663+0000","modified":"2023-11-26T07:20:36.636824+0000","last_up_change":"2023-11-26T07:17:41.157813+0000","last_in_change":"2023-11-26T07:17:31.181658+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":7,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-11-26T07:17:41.758597+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":6,"pool_name":"cephfs_metadata","create_time":"2023-11-26T07:19:29.208075+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"44","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":7,"pool_name":"cephfs_data","create_time":"2023-11-26T07:19:30.503743+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"45","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"ebbee2c6-023e-44b9-9f1d-6f7bfd309730","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":43,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6817","nonce":639807369},{"type":"v1","addr":"172.21.15.143:6819","nonce":639807369}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6821","nonce":639807369},{"type":"v1","addr":"172.21.15.143:6823","nonce":639807369}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6830","nonce":639807369},{"type":"v1","addr":"172.21.15.143:6831","nonce":639807369}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6825","nonce":639807369},{"type":"v1","addr":"172.21.15.143:6828","nonce":639807369}]},"public_addr":"172.21.15.143:6819/639807369","cluster_addr":"172.21.15.143:6823/639807369","heartbeat_back_addr":"172.21.15.143:6831/639807369","heartbeat_front_addr":"172.21.15.143:6828/639807369","state":["exists","up"]},{"osd":1,"uuid":"fec4b8be-7122-49e9-9a19-5850a0700001","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":43,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6816","nonce":1144942530},{"type":"v1","addr":"172.21.15.143:6818","nonce":1144942530}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6820","nonce":1144942530},{"type":"v1","addr":"172.21.15.143:6822","nonce":1144942530}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6827","nonce":1144942530},{"type":"v1","addr":"172.21.15.143:6829","nonce":1144942530}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6824","nonce":1144942530},{"type":"v1","addr":"172.21.15.143:6826","nonce":1144942530}]},"public_addr":"172.21.15.143:6818/1144942530","cluster_addr":"172.21.15.143:6822/1144942530","heartbeat_back_addr":"172.21.15.143:6829/1144942530","heartbeat_front_addr":"172.21.15.143:6826/1144942530","state":["exists","up"]},{"osd":2,"uuid":"2888af5b-f933-46ca-8f95-6dd494ff43f6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":46,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6808","nonce":3909095186},{"type":"v1","addr":"172.21.15.143:6809","nonce":3909095186}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6810","nonce":3909095186},{"type":"v1","addr":"172.21.15.143:6811","nonce":3909095186}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6814","nonce":3909095186},{"type":"v1","addr":"172.21.15.143:6815","nonce":3909095186}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6812","nonce":3909095186},{"type":"v1","addr":"172.21.15.143:6813","nonce":3909095186}]},"public_addr":"172.21.15.143:6809/3909095186","cluster_addr":"172.21.15.143:6811/3909095186","heartbeat_back_addr":"172.21.15.143:6815/3909095186","heartbeat_front_addr":"172.21.15.143:6813/3909095186","state":["exists","up"]},{"osd":3,"uuid":"d1f52435-7bc1-4081-9550-9cd1ba55f549","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":43,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6800","nonce":3854210991},{"type":"v1","addr":"172.21.15.143:6801","nonce":3854210991}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6802","nonce":3854210991},{"type":"v1","addr":"172.21.15.143:6803","nonce":3854210991}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6806","nonce":3854210991},{"type":"v1","addr":"172.21.15.143:6807","nonce":3854210991}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6804","nonce":3854210991},{"type":"v1","addr":"172.21.15.143:6805","nonce":3854210991}]},"public_addr":"172.21.15.143:6801/3854210991","cluster_addr":"172.21.15.143:6803/3854210991","heartbeat_back_addr":"172.21.15.143:6807/3854210991","heartbeat_front_addr":"172.21.15.143:6805/3854210991","state":["exists","up"]},{"osd":4,"uuid":"e2ae8b6e-6e25-49b2-bcbf-aacce68f06a3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":43,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6800","nonce":3924606902},{"type":"v1","addr":"172.21.15.188:6801","nonce":3924606902}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6802","nonce":3924606902},{"type":"v1","addr":"172.21.15.188:6803","nonce":3924606902}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6806","nonce":3924606902},{"type":"v1","addr":"172.21.15.188:6807","nonce":3924606902}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6804","nonce":3924606902},{"type":"v1","addr":"172.21.15.188:6805","nonce":3924606902}]},"public_addr":"172.21.15.188:6801/3924606902","cluster_addr":"172.21.15.188:6803/3924606902","heartbeat_back_addr":"172.21.15.188:6807/3924606902","heartbeat_front_addr":"172.21.15.188:6805/3924606902","state":["exists","up"]},{"osd":5,"uuid":"3cec8a32-7698-4c36-8e89-e4ba11934f33","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":43,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6813","nonce":186841544},{"type":"v1","addr":"172.21.15.188:6815","nonce":186841544}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6817","nonce":186841544},{"type":"v1","addr":"172.21.15.188:6818","nonce":186841544}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6822","nonce":186841544},{"type":"v1","addr":"172.21.15.188:6823","nonce":186841544}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6820","nonce":186841544},{"type":"v1","addr":"172.21.15.188:6821","nonce":186841544}]},"public_addr":"172.21.15.188:6815/186841544","cluster_addr":"172.21.15.188:6818/186841544","heartbeat_back_addr":"172.21.15.188:6823/186841544","heartbeat_front_addr":"172.21.15.188:6821/186841544","state":["exists","up"]},{"osd":6,"uuid":"58a65d5d-e090-473a-81b4-ad38e85029f9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":46,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6808","nonce":2153499405},{"type":"v1","addr":"172.21.15.188:6809","nonce":2153499405}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6810","nonce":2153499405},{"type":"v1","addr":"172.21.15.188:6811","nonce":2153499405}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6816","nonce":2153499405},{"type":"v1","addr":"172.21.15.188:6819","nonce":2153499405}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6812","nonce":2153499405},{"type":"v1","addr":"172.21.15.188:6814","nonce":2153499405}]},"public_addr":"172.21.15.188:6809/2153499405","cluster_addr":"172.21.15.188:6811/2153499405","heartbeat_back_addr":"172.21.15.188:6819/2153499405","heartbeat_front_addr":"172.21.15.188:6814/2153499405","state":["exists","up"]},{"osd":7,"uuid":"ea5b7f70-3554-4cf8-ac30-f7c59b80678d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":43,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6824","nonce":2722096729},{"type":"v1","addr":"172.21.15.188:6825","nonce":2722096729}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6826","nonce":2722096729},{"type":"v1","addr":"172.21.15.188:6827","nonce":2722096729}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6830","nonce":2722096729},{"type":"v1","addr":"172.21.15.188:6831","nonce":2722096729}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6828","nonce":2722096729},{"type":"v1","addr":"172.21.15.188:6829","nonce":2722096729}]},"public_addr":"172.21.15.188:6825/2722096729","cluster_addr":"172.21.15.188:6827/2722096729","heartbeat_back_addr":"172.21.15.188:6831/2722096729","heartbeat_front_addr":"172.21.15.188:6829/2722096729","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-26T07:17:36.995032+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-26T07:17:36.932266+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-26T07:17:36.829304+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-26T07:17:36.804297+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-26T07:17:36.667892+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-26T07:17:36.964868+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-26T07:17:36.827332+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-26T07:17:36.893179+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"7.2c","mappings":[{"from":0,"to":4}]},{"pgid":"7.2d","mappings":[{"from":0,"to":5}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-11-26T07:21:41.816 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:21:41.812+0000 7fe908632640 1 -- 172.21.15.143:0/1964878004 >> [v2:172.21.15.143:6832/2150557544,v1:172.21.15.143:6833/2150557544] conn(0x7fe8e4066650 msgr2=0x7fe8e4068b10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:21:41.816 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:21:41.812+0000 7fe908632640 1 --2- 172.21.15.143:0/1964878004 >> [v2:172.21.15.143:6832/2150557544,v1:172.21.15.143:6833/2150557544] conn(0x7fe8e4066650 0x7fe8e4068b10 secure :-1 s=READY pgs=350 cs=0 l=1 rev1=1 crypto rx=0x7fe9001f7180 tx=0x7fe8f0009040 comp rx=0 tx=0).stop 2023-11-26T07:21:41.816 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:21:41.812+0000 7fe908632640 1 -- 172.21.15.143:0/1964878004 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fe9000cf7c0 msgr2=0x7fe9001542a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:21:41.816 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:21:41.812+0000 7fe908632640 1 --2- 172.21.15.143:0/1964878004 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fe9000cf7c0 0x7fe9001542a0 secure :-1 s=READY pgs=730 cs=0 l=1 rev1=1 crypto rx=0x7fe8fc0ac040 tx=0x7fe8fc01ad10 comp rx=0 tx=0).stop 2023-11-26T07:21:41.817 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:21:41.812+0000 7fe908632640 1 -- 172.21.15.143:0/1964878004 shutdown_connections 2023-11-26T07:21:41.817 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:21:41.812+0000 7fe908632640 1 --2- 172.21.15.143:0/1964878004 >> [v2:172.21.15.143:6832/2150557544,v1:172.21.15.143:6833/2150557544] conn(0x7fe8e4066650 0x7fe8e4068b10 unknown :-1 s=CLOSED pgs=350 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:21:41.817 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:21:41.812+0000 7fe908632640 1 --2- 172.21.15.143:0/1964878004 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fe90015f6b0 0x7fe90013ceb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:21:41.817 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:21:41.812+0000 7fe908632640 1 --2- 172.21.15.143:0/1964878004 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fe90013e290 0x7fe90013c970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:21:41.817 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:21:41.812+0000 7fe908632640 1 --2- 172.21.15.143:0/1964878004 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fe9000cf7c0 0x7fe9001542a0 unknown :-1 s=CLOSED pgs=730 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:21:41.817 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:21:41.812+0000 7fe908632640 1 -- 172.21.15.143:0/1964878004 >> 172.21.15.143:0/1964878004 conn(0x7fe90006a890 msgr2=0x7fe90015b510 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:21:41.817 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:21:41.812+0000 7fe908632640 1 -- 172.21.15.143:0/1964878004 shutdown_connections 2023-11-26T07:21:41.817 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:21:41.812+0000 7fe908632640 1 -- 172.21.15.143:0/1964878004 wait complete. 2023-11-26T07:21:41.835 INFO:tasks.cephfs.filesystem:Destroying file system cephfs and related pools 2023-11-26T07:21:41.835 DEBUG:teuthology.orchestra.run.smithi143:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-11-26T07:21:41.972 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:21:41.968+0000 7f7c5b29c640 1 Processor -- start 2023-11-26T07:21:41.972 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:21:41.972+0000 7f7c5b29c640 1 -- start start 2023-11-26T07:21:41.974 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:21:41.972+0000 7f7c5b29c640 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7f7c5415d6e0 0x7f7c540d0df0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:21:41.974 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:21:41.972+0000 7f7c5b29c640 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7f7c540b2160 0x7f7c540d14b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:21:41.974 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:21:41.972+0000 7f7c5b29c640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f7c540d19f0 0x7f7c540cfbf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:21:41.974 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:21:41.972+0000 7f7c5b29c640 1 -- --> v2:172.21.15.143:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7c5415dac0 con 0x7f7c540b2160 2023-11-26T07:21:41.974 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:21:41.972+0000 7f7c5b29c640 1 -- --> v2:172.21.15.188:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7c540d1e80 con 0x7f7c5415d6e0 2023-11-26T07:21:41.974 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:21:41.972+0000 7f7c5b29c640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7c540cf0e0 con 0x7f7c540d19f0 2023-11-26T07:21:41.974 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:21:41.972+0000 7f7c58837640 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7f7c540b2160 0x7f7c540d14b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:21:41.974 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:21:41.972+0000 7f7c58837640 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7f7c540b2160 0x7f7c540d14b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:35806/0 (socket says 172.21.15.143:35806) 2023-11-26T07:21:41.974 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:21:41.972+0000 7f7c58837640 1 -- 172.21.15.143:0/3869756235 learned_addr learned my addr 172.21.15.143:0/3869756235 (peer_addr_for_me v2:172.21.15.143:0/0) 2023-11-26T07:21:41.975 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:21:41.972+0000 7f7c59038640 1 --2- 172.21.15.143:0/3869756235 >> v2:172.21.15.188:3300/0 conn(0x7f7c5415d6e0 0x7f7c540d0df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:21:41.975 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:21:41.972+0000 7f7c59839640 1 --2- 172.21.15.143:0/3869756235 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f7c540d19f0 0x7f7c540cfbf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:21:41.975 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:21:41.972+0000 7f7c58837640 1 -- 172.21.15.143:0/3869756235 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f7c540d19f0 msgr2=0x7f7c540cfbf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-26T07:21:41.975 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:21:41.972+0000 7f7c58837640 1 --2- 172.21.15.143:0/3869756235 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f7c540d19f0 0x7f7c540cfbf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:21:41.975 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:21:41.972+0000 7f7c58837640 1 -- 172.21.15.143:0/3869756235 >> v2:172.21.15.188:3300/0 conn(0x7f7c5415d6e0 msgr2=0x7f7c540d0df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-26T07:21:41.976 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:21:41.972+0000 7f7c58837640 1 --2- 172.21.15.143:0/3869756235 >> v2:172.21.15.188:3300/0 conn(0x7f7c5415d6e0 0x7f7c540d0df0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:21:41.976 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:21:41.972+0000 7f7c58837640 1 -- 172.21.15.143:0/3869756235 --> v2:172.21.15.143:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7c540ceb20 con 0x7f7c540b2160 2023-11-26T07:21:41.976 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:21:41.972+0000 7f7c59038640 1 --2- 172.21.15.143:0/3869756235 >> v2:172.21.15.188:3300/0 conn(0x7f7c5415d6e0 0x7f7c540d0df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-26T07:21:41.976 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:21:41.972+0000 7f7c59839640 1 --2- 172.21.15.143:0/3869756235 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f7c540d19f0 0x7f7c540cfbf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-26T07:21:41.977 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:21:41.976+0000 7f7c58837640 1 --2- 172.21.15.143:0/3869756235 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f7c540b2160 0x7f7c540d14b0 secure :-1 s=READY pgs=731 cs=0 l=1 rev1=1 crypto rx=0x7f7c540b5db0 tx=0x7f7c4401c240 comp rx=0 tx=0).ready entity=mon.0 client_cookie=18fdce1819d43639 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:21:41.978 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:21:41.976+0000 7f7c4bfff640 1 -- 172.21.15.143:0/3869756235 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7c440ab600 con 0x7f7c540b2160 2023-11-26T07:21:41.978 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:21:41.976+0000 7f7c4bfff640 1 -- 172.21.15.143:0/3869756235 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7c440a7050 con 0x7f7c540b2160 2023-11-26T07:21:41.978 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:21:41.976+0000 7f7c4bfff640 1 -- 172.21.15.143:0/3869756235 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7c440ab980 con 0x7f7c540b2160 2023-11-26T07:21:41.979 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:21:41.976+0000 7f7c5b29c640 1 -- 172.21.15.143:0/3869756235 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f7c540b2160 msgr2=0x7f7c540d14b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:21:41.979 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:21:41.976+0000 7f7c5b29c640 1 --2- 172.21.15.143:0/3869756235 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f7c540b2160 0x7f7c540d14b0 secure :-1 s=READY pgs=731 cs=0 l=1 rev1=1 crypto rx=0x7f7c540b5db0 tx=0x7f7c4401c240 comp rx=0 tx=0).stop 2023-11-26T07:21:41.979 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:21:41.976+0000 7f7c5b29c640 1 -- 172.21.15.143:0/3869756235 shutdown_connections 2023-11-26T07:21:41.979 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:21:41.976+0000 7f7c5b29c640 1 --2- 172.21.15.143:0/3869756235 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f7c540d19f0 0x7f7c540cfbf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:21:41.979 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:21:41.976+0000 7f7c5b29c640 1 --2- 172.21.15.143:0/3869756235 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f7c540b2160 0x7f7c540d14b0 unknown :-1 s=CLOSED pgs=731 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:21:41.979 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:21:41.976+0000 7f7c5b29c640 1 --2- 172.21.15.143:0/3869756235 >> v2:172.21.15.188:3300/0 conn(0x7f7c5415d6e0 0x7f7c540d0df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:21:41.979 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:21:41.976+0000 7f7c5b29c640 1 -- 172.21.15.143:0/3869756235 >> 172.21.15.143:0/3869756235 conn(0x7f7c5409c550 msgr2=0x7f7c540d3220 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:21:41.979 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:21:41.976+0000 7f7c5b29c640 1 -- 172.21.15.143:0/3869756235 shutdown_connections 2023-11-26T07:21:41.979 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:21:41.976+0000 7f7c5b29c640 1 -- 172.21.15.143:0/3869756235 wait complete. 2023-11-26T07:21:41.980 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:21:41.976+0000 7f7c5b29c640 1 Processor -- start 2023-11-26T07:21:41.980 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:21:41.976+0000 7f7c5b29c640 1 -- start start 2023-11-26T07:21:41.982 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:21:41.980+0000 7f7c5b29c640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f7c540b2160 0x7f7c540cceb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:21:41.982 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:21:41.980+0000 7f7c5b29c640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f7c540d19f0 0x7f7c541560c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:21:41.982 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:21:41.980+0000 7f7c5b29c640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f7c5415d6e0 0x7f7c54154950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:21:41.982 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:21:41.980+0000 7f7c5b29c640 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7c54156600 con 0x7f7c540b2160 2023-11-26T07:21:41.982 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:21:41.980+0000 7f7c5b29c640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7c54156880 con 0x7f7c540d19f0 2023-11-26T07:21:41.982 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:21:41.980+0000 7f7c5b29c640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7c5406c490 con 0x7f7c5415d6e0 2023-11-26T07:21:41.982 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:21:41.980+0000 7f7c59038640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f7c540b2160 0x7f7c540cceb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:21:41.982 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:21:41.980+0000 7f7c59038640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f7c540b2160 0x7f7c540cceb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:35822/0 (socket says 172.21.15.143:35822) 2023-11-26T07:21:41.982 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:21:41.980+0000 7f7c59038640 1 -- 172.21.15.143:0/3069471144 learned_addr learned my addr 172.21.15.143:0/3069471144 (peer_addr_for_me v2:172.21.15.143:0/0) 2023-11-26T07:21:41.982 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:21:41.980+0000 7f7c59839640 1 --2- 172.21.15.143:0/3069471144 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f7c5415d6e0 0x7f7c54154950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:21:41.982 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:21:41.980+0000 7f7c58837640 1 --2- 172.21.15.143:0/3069471144 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f7c540d19f0 0x7f7c541560c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:21:41.983 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:21:41.980+0000 7f7c59038640 1 -- 172.21.15.143:0/3069471144 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f7c5415d6e0 msgr2=0x7f7c54154950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:21:41.983 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:21:41.980+0000 7f7c59038640 1 --2- 172.21.15.143:0/3069471144 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f7c5415d6e0 0x7f7c54154950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:21:41.983 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:21:41.980+0000 7f7c59038640 1 -- 172.21.15.143:0/3069471144 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f7c540d19f0 msgr2=0x7f7c541560c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:21:41.983 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:21:41.980+0000 7f7c59038640 1 --2- 172.21.15.143:0/3069471144 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f7c540d19f0 0x7f7c541560c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:21:41.983 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:21:41.980+0000 7f7c59038640 1 -- 172.21.15.143:0/3069471144 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7c440094d0 con 0x7f7c540b2160 2023-11-26T07:21:41.983 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:21:41.980+0000 7f7c59038640 1 --2- 172.21.15.143:0/3069471144 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f7c540b2160 0x7f7c540cceb0 secure :-1 s=READY pgs=732 cs=0 l=1 rev1=1 crypto rx=0x7f7c5000c7f0 tx=0x7f7c5000ccb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:21:41.984 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:21:41.980+0000 7f7c49ffb640 1 -- 172.21.15.143:0/3069471144 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7c50007820 con 0x7f7c540b2160 2023-11-26T07:21:41.984 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:21:41.980+0000 7f7c49ffb640 1 -- 172.21.15.143:0/3069471144 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7c50007980 con 0x7f7c540b2160 2023-11-26T07:21:41.984 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:21:41.980+0000 7f7c49ffb640 1 -- 172.21.15.143:0/3069471144 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7c50007d30 con 0x7f7c540b2160 2023-11-26T07:21:41.984 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:21:41.980+0000 7f7c5b29c640 1 -- 172.21.15.143:0/3069471144 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7c5406c770 con 0x7f7c540b2160 2023-11-26T07:21:41.984 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:21:41.980+0000 7f7c5b29c640 1 -- 172.21.15.143:0/3069471144 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7c54154e90 con 0x7f7c540b2160 2023-11-26T07:21:41.984 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:21:41.980+0000 7f7c5b29c640 1 -- 172.21.15.143:0/3069471144 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7c540b5e40 con 0x7f7c540b2160 2023-11-26T07:21:41.984 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:21:41.980+0000 7f7c49ffb640 1 -- 172.21.15.143:0/3069471144 <== mon.0 v2:172.21.15.143:3300/0 4 ==== mgrmap(e 9) v1 ==== 86005+0+0 (secure 0 0 0) 0x7f7c500113f0 con 0x7f7c540b2160 2023-11-26T07:21:41.985 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:21:41.984+0000 7f7c49ffb640 1 --2- 172.21.15.143:0/3069471144 >> [v2:172.21.15.143:6832/2150557544,v1:172.21.15.143:6833/2150557544] conn(0x7f7c38066650 0x7f7c38068b10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:21:41.985 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:21:41.984+0000 7f7c49ffb640 1 -- 172.21.15.143:0/3069471144 <== mon.0 v2:172.21.15.143:3300/0 5 ==== osd_map(47..47 src has 1..47) v4 ==== 5767+0+0 (secure 0 0 0) 0x7f7c500845d0 con 0x7f7c540b2160 2023-11-26T07:21:41.986 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:21:41.984+0000 7f7c58837640 1 --2- 172.21.15.143:0/3069471144 >> [v2:172.21.15.143:6832/2150557544,v1:172.21.15.143:6833/2150557544] conn(0x7f7c38066650 0x7f7c38068b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:21:41.986 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:21:41.984+0000 7f7c58837640 1 --2- 172.21.15.143:0/3069471144 >> [v2:172.21.15.143:6832/2150557544,v1:172.21.15.143:6833/2150557544] conn(0x7f7c38066650 0x7f7c38068b10 secure :-1 s=READY pgs=351 cs=0 l=1 rev1=1 crypto rx=0x7f7c440ad000 tx=0x7f7c44008040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:21:41.989 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:21:41.988+0000 7f7c49ffb640 1 -- 172.21.15.143:0/3069471144 <== mon.0 v2:172.21.15.143:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172649 (secure 0 0 0) 0x7f7c540b5e40 con 0x7f7c540b2160 2023-11-26T07:21:42.168 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:21:42.164+0000 7f7c5b29c640 1 -- 172.21.15.143:0/3069471144 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f7c540b5e40 con 0x7f7c540b2160 2023-11-26T07:21:42.169 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:21:42.168+0000 7f7c49ffb640 1 -- 172.21.15.143:0/3069471144 <== mon.0 v2:172.21.15.143:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 26 v26) v1 ==== 94+0+4922 (secure 0 0 0) 0x7f7c540b5e40 con 0x7f7c540b2160 2023-11-26T07:21:42.169 INFO:teuthology.orchestra.run.smithi143.stdout: 2023-11-26T07:21:42.169 INFO:teuthology.orchestra.run.smithi143.stdout:{"epoch":26,"default_fscid":3,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5201,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.188:6835/319464848","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6833","nonce":319464848},{"type":"v1","addr":"172.21.15.188:6835","nonce":319464848}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":20},{"gid":5584,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.188:6834/2113408070","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6832","nonce":2113408070},{"type":"v1","addr":"172.21.15.188:6834","nonce":2113408070}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":20},{"gid":5745,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.143:6835/1616742593","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6834","nonce":1616742593},{"type":"v1","addr":"172.21.15.143:6835","nonce":1616742593}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":20}],"filesystems":[{"mdsmap":{"epoch":25,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-11-26T07:19:31.503587+0000","modified":"2023-11-26T07:19:34.523605+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":5748},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5748":{"gid":5748,"name":"c","rank":0,"incarnation":21,"state":"up:active","state_seq":3,"addr":"172.21.15.143:6837/2872033338","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6836","nonce":2872033338},{"type":"v1","addr":"172.21.15.143:6837","nonce":2872033338}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[7],"metadata_pool":6,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":3}]} 2023-11-26T07:21:42.169 INFO:teuthology.orchestra.run.smithi143.stderr:dumped fsmap epoch 26 2023-11-26T07:21:42.171 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:21:42.168+0000 7f7c5b29c640 1 -- 172.21.15.143:0/3069471144 >> [v2:172.21.15.143:6832/2150557544,v1:172.21.15.143:6833/2150557544] conn(0x7f7c38066650 msgr2=0x7f7c38068b10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:21:42.171 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:21:42.168+0000 7f7c5b29c640 1 --2- 172.21.15.143:0/3069471144 >> [v2:172.21.15.143:6832/2150557544,v1:172.21.15.143:6833/2150557544] conn(0x7f7c38066650 0x7f7c38068b10 secure :-1 s=READY pgs=351 cs=0 l=1 rev1=1 crypto rx=0x7f7c440ad000 tx=0x7f7c44008040 comp rx=0 tx=0).stop 2023-11-26T07:21:42.171 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:21:42.168+0000 7f7c5b29c640 1 -- 172.21.15.143:0/3069471144 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f7c540b2160 msgr2=0x7f7c540cceb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:21:42.171 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:21:42.168+0000 7f7c5b29c640 1 --2- 172.21.15.143:0/3069471144 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f7c540b2160 0x7f7c540cceb0 secure :-1 s=READY pgs=732 cs=0 l=1 rev1=1 crypto rx=0x7f7c5000c7f0 tx=0x7f7c5000ccb0 comp rx=0 tx=0).stop 2023-11-26T07:21:42.171 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:21:42.168+0000 7f7c5b29c640 1 -- 172.21.15.143:0/3069471144 shutdown_connections 2023-11-26T07:21:42.171 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:21:42.168+0000 7f7c5b29c640 1 --2- 172.21.15.143:0/3069471144 >> [v2:172.21.15.143:6832/2150557544,v1:172.21.15.143:6833/2150557544] conn(0x7f7c38066650 0x7f7c38068b10 unknown :-1 s=CLOSED pgs=351 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:21:42.171 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:21:42.168+0000 7f7c5b29c640 1 --2- 172.21.15.143:0/3069471144 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f7c5415d6e0 0x7f7c54154950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:21:42.171 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:21:42.168+0000 7f7c5b29c640 1 --2- 172.21.15.143:0/3069471144 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f7c540d19f0 0x7f7c541560c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:21:42.171 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:21:42.168+0000 7f7c5b29c640 1 --2- 172.21.15.143:0/3069471144 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f7c540b2160 0x7f7c540cceb0 unknown :-1 s=CLOSED pgs=732 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:21:42.171 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:21:42.168+0000 7f7c5b29c640 1 -- 172.21.15.143:0/3069471144 >> 172.21.15.143:0/3069471144 conn(0x7f7c5409c550 msgr2=0x7f7c540d3220 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:21:42.172 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:21:42.168+0000 7f7c5b29c640 1 -- 172.21.15.143:0/3069471144 shutdown_connections 2023-11-26T07:21:42.172 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:21:42.168+0000 7f7c5b29c640 1 -- 172.21.15.143:0/3069471144 wait complete. 2023-11-26T07:21:42.189 DEBUG:teuthology.orchestra.run.smithi143:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-11-26T07:21:42.328 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:21:42.324+0000 7f48f1b57640 1 Processor -- start 2023-11-26T07:21:42.328 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:21:42.324+0000 7f48f1b57640 1 -- start start 2023-11-26T07:21:42.330 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:21:42.328+0000 7f48f1b57640 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7f48ec143ac0 0x7f48ec070370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:21:42.330 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:21:42.328+0000 7f48f1b57640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f48ec0708b0 0x7f48ec0cf280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:21:42.330 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:21:42.328+0000 7f48f1b57640 1 -- --> v1:172.21.15.143:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f48ec0cfbf0 con 0x7f48ec0cf7c0 2023-11-26T07:21:42.330 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:21:42.328+0000 7f48f1b57640 1 -- --> v2:172.21.15.143:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f48ec143ea0 con 0x7f48ec143ac0 2023-11-26T07:21:42.330 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:21:42.328+0000 7f48f1b57640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f48ec0cc730 con 0x7f48ec0708b0 2023-11-26T07:21:42.330 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:21:42.328+0000 7f48eb7fe640 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7f48ec143ac0 0x7f48ec070370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:21:42.330 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:21:42.328+0000 7f48eb7fe640 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7f48ec143ac0 0x7f48ec070370 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:35834/0 (socket says 172.21.15.143:35834) 2023-11-26T07:21:42.330 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:21:42.328+0000 7f48eb7fe640 1 -- 172.21.15.143:0/229888237 learned_addr learned my addr 172.21.15.143:0/229888237 (peer_addr_for_me v2:172.21.15.143:0/0) 2023-11-26T07:21:42.330 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:21:42.328+0000 7f48eaffd640 1 --2- 172.21.15.143:0/229888237 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f48ec0708b0 0x7f48ec0cf280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:21:42.331 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:21:42.328+0000 7f48eb7fe640 1 -- 172.21.15.143:0/229888237 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f48ec0708b0 msgr2=0x7f48ec0cf280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-26T07:21:42.331 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:21:42.328+0000 7f48eb7fe640 1 --2- 172.21.15.143:0/229888237 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f48ec0708b0 0x7f48ec0cf280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:21:42.331 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:21:42.328+0000 7f48eb7fe640 1 -- 172.21.15.143:0/229888237 >> v1:172.21.15.143:6789/0 conn(0x7f48ec0cf7c0 legacy=0x7f48ec0b1ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:21:42.331 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:21:42.328+0000 7f48eb7fe640 1 -- 172.21.15.143:0/229888237 --> v2:172.21.15.143:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f48ec070ee0 con 0x7f48ec143ac0 2023-11-26T07:21:42.331 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:21:42.328+0000 7f48eaffd640 1 --2- 172.21.15.143:0/229888237 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f48ec0708b0 0x7f48ec0cf280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-26T07:21:42.331 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:21:42.328+0000 7f48ea7fc640 1 -- 172.21.15.143:0/229888237 <== mon.0 v1:172.21.15.143:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3631154966 0 0) 0x7f48ec0cfbf0 con 0x7f48ec0cf7c0 2023-11-26T07:21:42.333 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:21:42.332+0000 7f48eb7fe640 1 --2- 172.21.15.143:0/229888237 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f48ec143ac0 0x7f48ec070370 secure :-1 s=READY pgs=734 cs=0 l=1 rev1=1 crypto rx=0x7f48dc009870 tx=0x7f48dc01c540 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f5f27205a871d627 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:21:42.334 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:21:42.332+0000 7f48ea7fc640 1 -- 172.21.15.143:0/229888237 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f48dc0ab650 con 0x7f48ec143ac0 2023-11-26T07:21:42.334 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:21:42.332+0000 7f48ea7fc640 1 -- 172.21.15.143:0/229888237 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f48dc0ab7b0 con 0x7f48ec143ac0 2023-11-26T07:21:42.334 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:21:42.332+0000 7f48ea7fc640 1 -- 172.21.15.143:0/229888237 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f48dc0abb40 con 0x7f48ec143ac0 2023-11-26T07:21:42.335 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:21:42.332+0000 7f48f1b57640 1 -- 172.21.15.143:0/229888237 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f48ec143ac0 msgr2=0x7f48ec070370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:21:42.335 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:21:42.332+0000 7f48f1b57640 1 --2- 172.21.15.143:0/229888237 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f48ec143ac0 0x7f48ec070370 secure :-1 s=READY pgs=734 cs=0 l=1 rev1=1 crypto rx=0x7f48dc009870 tx=0x7f48dc01c540 comp rx=0 tx=0).stop 2023-11-26T07:21:42.335 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:21:42.332+0000 7f48f1b57640 1 -- 172.21.15.143:0/229888237 shutdown_connections 2023-11-26T07:21:42.335 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:21:42.332+0000 7f48f1b57640 1 --2- 172.21.15.143:0/229888237 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f48ec0708b0 0x7f48ec0cf280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:21:42.335 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:21:42.332+0000 7f48f1b57640 1 --2- 172.21.15.143:0/229888237 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f48ec143ac0 0x7f48ec070370 unknown :-1 s=CLOSED pgs=734 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:21:42.335 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:21:42.332+0000 7f48f1b57640 1 -- 172.21.15.143:0/229888237 >> 172.21.15.143:0/229888237 conn(0x7f48ec06a890 msgr2=0x7f48ec06aca0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:21:42.335 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:21:42.332+0000 7f48f1b57640 1 -- 172.21.15.143:0/229888237 shutdown_connections 2023-11-26T07:21:42.335 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:21:42.332+0000 7f48f1b57640 1 -- 172.21.15.143:0/229888237 wait complete. 2023-11-26T07:21:42.336 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:21:42.332+0000 7f48f1b57640 1 Processor -- start 2023-11-26T07:21:42.336 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:21:42.332+0000 7f48f1b57640 1 -- start start 2023-11-26T07:21:42.338 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:21:42.336+0000 7f48f1b57640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f48ec0708b0 0x7f48ec1418c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:21:42.338 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:21:42.336+0000 7f48f1b57640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f48ec0cf7c0 0x7f48ec13b6a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:21:42.338 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:21:42.336+0000 7f48f1b57640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f48ec143ac0 0x7f48ec0aed40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:21:42.338 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:21:42.336+0000 7f48f1b57640 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f48ec157170 con 0x7f48ec143ac0 2023-11-26T07:21:42.338 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:21:42.336+0000 7f48f1b57640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f48ec15a590 con 0x7f48ec0708b0 2023-11-26T07:21:42.338 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:21:42.336+0000 7f48f1b57640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f48ec13bbe0 con 0x7f48ec0cf7c0 2023-11-26T07:21:42.338 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:21:42.336+0000 7f48ebfff640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f48ec143ac0 0x7f48ec0aed40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:21:42.338 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:21:42.336+0000 7f48ebfff640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f48ec143ac0 0x7f48ec0aed40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:35838/0 (socket says 172.21.15.143:35838) 2023-11-26T07:21:42.338 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:21:42.336+0000 7f48ebfff640 1 -- 172.21.15.143:0/1211962399 learned_addr learned my addr 172.21.15.143:0/1211962399 (peer_addr_for_me v2:172.21.15.143:0/0) 2023-11-26T07:21:42.339 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:21:42.336+0000 7f48eaffd640 1 --2- 172.21.15.143:0/1211962399 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f48ec0cf7c0 0x7f48ec13b6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:21:42.339 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:21:42.336+0000 7f48eb7fe640 1 --2- 172.21.15.143:0/1211962399 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f48ec0708b0 0x7f48ec1418c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:21:42.339 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:21:42.336+0000 7f48ebfff640 1 -- 172.21.15.143:0/1211962399 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f48ec0cf7c0 msgr2=0x7f48ec13b6a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:21:42.339 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:21:42.336+0000 7f48ebfff640 1 --2- 172.21.15.143:0/1211962399 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f48ec0cf7c0 0x7f48ec13b6a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:21:42.339 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:21:42.336+0000 7f48ebfff640 1 -- 172.21.15.143:0/1211962399 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f48ec0708b0 msgr2=0x7f48ec1418c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:21:42.339 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:21:42.336+0000 7f48ebfff640 1 --2- 172.21.15.143:0/1211962399 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f48ec0708b0 0x7f48ec1418c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:21:42.339 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:21:42.336+0000 7f48ebfff640 1 -- 172.21.15.143:0/1211962399 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f48dc0094d0 con 0x7f48ec143ac0 2023-11-26T07:21:42.339 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:21:42.336+0000 7f48eb7fe640 1 --2- 172.21.15.143:0/1211962399 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f48ec0708b0 0x7f48ec1418c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-26T07:21:42.340 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:21:42.336+0000 7f48ebfff640 1 --2- 172.21.15.143:0/1211962399 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f48ec143ac0 0x7f48ec0aed40 secure :-1 s=READY pgs=735 cs=0 l=1 rev1=1 crypto rx=0x7f48e400c780 tx=0x7f48e400cc40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:21:42.340 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:21:42.336+0000 7f48d3fff640 1 -- 172.21.15.143:0/1211962399 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f48e4003ba0 con 0x7f48ec143ac0 2023-11-26T07:21:42.340 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:21:42.336+0000 7f48d3fff640 1 -- 172.21.15.143:0/1211962399 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f48e4003d00 con 0x7f48ec143ac0 2023-11-26T07:21:42.340 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:21:42.336+0000 7f48f1b57640 1 -- 172.21.15.143:0/1211962399 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f48ec156bb0 con 0x7f48ec143ac0 2023-11-26T07:21:42.340 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:21:42.336+0000 7f48d3fff640 1 -- 172.21.15.143:0/1211962399 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f48e400aa90 con 0x7f48ec143ac0 2023-11-26T07:21:42.340 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:21:42.336+0000 7f48f1b57640 1 -- 172.21.15.143:0/1211962399 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f48ec13fbe0 con 0x7f48ec143ac0 2023-11-26T07:21:42.340 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:21:42.336+0000 7f48f1b57640 1 -- 172.21.15.143:0/1211962399 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f48ec13fa40 con 0x7f48ec143ac0 2023-11-26T07:21:42.341 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:21:42.340+0000 7f48d3fff640 1 -- 172.21.15.143:0/1211962399 <== mon.0 v2:172.21.15.143:3300/0 4 ==== mgrmap(e 9) v1 ==== 86005+0+0 (secure 0 0 0) 0x7f48e400abf0 con 0x7f48ec143ac0 2023-11-26T07:21:42.341 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:21:42.340+0000 7f48d3fff640 1 --2- 172.21.15.143:0/1211962399 >> [v2:172.21.15.143:6832/2150557544,v1:172.21.15.143:6833/2150557544] conn(0x7f48d8066620 0x7f48d8068ae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:21:42.341 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:21:42.340+0000 7f48d3fff640 1 -- 172.21.15.143:0/1211962399 <== mon.0 v2:172.21.15.143:3300/0 5 ==== osd_map(47..47 src has 1..47) v4 ==== 5767+0+0 (secure 0 0 0) 0x7f48e4083e70 con 0x7f48ec143ac0 2023-11-26T07:21:42.342 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:21:42.340+0000 7f48eb7fe640 1 --2- 172.21.15.143:0/1211962399 >> [v2:172.21.15.143:6832/2150557544,v1:172.21.15.143:6833/2150557544] conn(0x7f48d8066620 0x7f48d8068ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:21:42.342 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:21:42.340+0000 7f48eb7fe640 1 --2- 172.21.15.143:0/1211962399 >> [v2:172.21.15.143:6832/2150557544,v1:172.21.15.143:6833/2150557544] conn(0x7f48d8066620 0x7f48d8068ae0 secure :-1 s=READY pgs=352 cs=0 l=1 rev1=1 crypto rx=0x7f48dc0af000 tx=0x7f48dc0a7000 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:21:42.345 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:21:42.344+0000 7f48d3fff640 1 -- 172.21.15.143:0/1211962399 <== mon.0 v2:172.21.15.143:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172649 (secure 0 0 0) 0x7f48ec13fa40 con 0x7f48ec143ac0 2023-11-26T07:21:42.536 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:21:42.532+0000 7f48f1b57640 1 -- 172.21.15.143:0/1211962399 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f48ec13fa40 con 0x7f48ec143ac0 2023-11-26T07:21:42.536 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:21:42.532+0000 7f48d3fff640 1 -- 172.21.15.143:0/1211962399 <== mon.0 v2:172.21.15.143:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 26 v26) v1 ==== 94+0+4922 (secure 0 0 0) 0x7f48ec13fa40 con 0x7f48ec143ac0 2023-11-26T07:21:42.537 INFO:teuthology.orchestra.run.smithi143.stdout: 2023-11-26T07:21:42.537 INFO:teuthology.orchestra.run.smithi143.stdout:{"epoch":26,"default_fscid":3,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5201,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.188:6835/319464848","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6833","nonce":319464848},{"type":"v1","addr":"172.21.15.188:6835","nonce":319464848}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":20},{"gid":5584,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.188:6834/2113408070","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6832","nonce":2113408070},{"type":"v1","addr":"172.21.15.188:6834","nonce":2113408070}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":20},{"gid":5745,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.143:6835/1616742593","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6834","nonce":1616742593},{"type":"v1","addr":"172.21.15.143:6835","nonce":1616742593}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":20}],"filesystems":[{"mdsmap":{"epoch":25,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-11-26T07:19:31.503587+0000","modified":"2023-11-26T07:19:34.523605+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":5748},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5748":{"gid":5748,"name":"c","rank":0,"incarnation":21,"state":"up:active","state_seq":3,"addr":"172.21.15.143:6837/2872033338","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6836","nonce":2872033338},{"type":"v1","addr":"172.21.15.143:6837","nonce":2872033338}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[7],"metadata_pool":6,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":3}]} 2023-11-26T07:21:42.537 INFO:teuthology.orchestra.run.smithi143.stderr:dumped fsmap epoch 26 2023-11-26T07:21:42.539 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:21:42.536+0000 7f48f1b57640 1 -- 172.21.15.143:0/1211962399 >> [v2:172.21.15.143:6832/2150557544,v1:172.21.15.143:6833/2150557544] conn(0x7f48d8066620 msgr2=0x7f48d8068ae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:21:42.539 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:21:42.536+0000 7f48f1b57640 1 --2- 172.21.15.143:0/1211962399 >> [v2:172.21.15.143:6832/2150557544,v1:172.21.15.143:6833/2150557544] conn(0x7f48d8066620 0x7f48d8068ae0 secure :-1 s=READY pgs=352 cs=0 l=1 rev1=1 crypto rx=0x7f48dc0af000 tx=0x7f48dc0a7000 comp rx=0 tx=0).stop 2023-11-26T07:21:42.539 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:21:42.536+0000 7f48f1b57640 1 -- 172.21.15.143:0/1211962399 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f48ec143ac0 msgr2=0x7f48ec0aed40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:21:42.539 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:21:42.536+0000 7f48f1b57640 1 --2- 172.21.15.143:0/1211962399 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f48ec143ac0 0x7f48ec0aed40 secure :-1 s=READY pgs=735 cs=0 l=1 rev1=1 crypto rx=0x7f48e400c780 tx=0x7f48e400cc40 comp rx=0 tx=0).stop 2023-11-26T07:21:42.539 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:21:42.536+0000 7f48f1b57640 1 -- 172.21.15.143:0/1211962399 shutdown_connections 2023-11-26T07:21:42.539 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:21:42.536+0000 7f48f1b57640 1 --2- 172.21.15.143:0/1211962399 >> [v2:172.21.15.143:6832/2150557544,v1:172.21.15.143:6833/2150557544] conn(0x7f48d8066620 0x7f48d8068ae0 unknown :-1 s=CLOSED pgs=352 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:21:42.539 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:21:42.536+0000 7f48f1b57640 1 --2- 172.21.15.143:0/1211962399 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f48ec143ac0 0x7f48ec0aed40 unknown :-1 s=CLOSED pgs=735 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:21:42.539 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:21:42.536+0000 7f48f1b57640 1 --2- 172.21.15.143:0/1211962399 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f48ec0cf7c0 0x7f48ec13b6a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:21:42.539 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:21:42.536+0000 7f48f1b57640 1 --2- 172.21.15.143:0/1211962399 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f48ec0708b0 0x7f48ec1418c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:21:42.540 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:21:42.536+0000 7f48f1b57640 1 -- 172.21.15.143:0/1211962399 >> 172.21.15.143:0/1211962399 conn(0x7f48ec06a890 msgr2=0x7f48ec06ac70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:21:42.540 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:21:42.536+0000 7f48f1b57640 1 -- 172.21.15.143:0/1211962399 shutdown_connections 2023-11-26T07:21:42.540 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:21:42.536+0000 7f48f1b57640 1 -- 172.21.15.143:0/1211962399 wait complete. 2023-11-26T07:21:42.558 DEBUG:teuthology.orchestra.run.smithi143:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-11-26T07:21:42.697 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:21:42.696+0000 7f6fa3a79640 1 Processor -- start 2023-11-26T07:21:42.697 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:21:42.696+0000 7f6fa3a79640 1 -- start start 2023-11-26T07:21:42.699 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:21:42.696+0000 7f6fa3a79640 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7f6f9c0cf280 0x7f6f9c0cf660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:21:42.699 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:21:42.696+0000 7f6fa3a79640 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7f6f9c070d10 0x7f6f9c15d900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:21:42.699 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:21:42.696+0000 7f6fa3a79640 1 -- --> v1:172.21.15.188:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6f9c15b2f0 con 0x7f6f9c06cd50 2023-11-26T07:21:42.699 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:21:42.696+0000 7f6fa3a79640 1 -- --> v2:172.21.15.143:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6f9c06c5c0 con 0x7f6f9c0cf280 2023-11-26T07:21:42.699 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:21:42.696+0000 7f6fa3a79640 1 -- --> v2:172.21.15.188:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6f9c0cfba0 con 0x7f6f9c070d10 2023-11-26T07:21:42.699 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:21:42.696+0000 7f6fa1815640 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7f6f9c0cf280 0x7f6f9c0cf660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:21:42.700 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:21:42.696+0000 7f6fa1815640 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7f6f9c0cf280 0x7f6f9c0cf660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:35844/0 (socket says 172.21.15.143:35844) 2023-11-26T07:21:42.700 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:21:42.696+0000 7f6fa1815640 1 -- 172.21.15.143:0/4249804766 learned_addr learned my addr 172.21.15.143:0/4249804766 (peer_addr_for_me v2:172.21.15.143:0/0) 2023-11-26T07:21:42.700 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:21:42.696+0000 7f6fa1815640 1 -- 172.21.15.143:0/4249804766 >> v2:172.21.15.188:3300/0 conn(0x7f6f9c070d10 msgr2=0x7f6f9c15d900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-26T07:21:42.700 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:21:42.696+0000 7f6fa1815640 1 --2- 172.21.15.143:0/4249804766 >> v2:172.21.15.188:3300/0 conn(0x7f6f9c070d10 0x7f6f9c15d900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:21:42.700 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:21:42.696+0000 7f6fa1815640 1 -- 172.21.15.143:0/4249804766 >> v1:172.21.15.188:6789/0 conn(0x7f6f9c06cd50 legacy=0x7f6f9c0b2180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-26T07:21:42.700 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:21:42.696+0000 7f6fa1815640 1 -- 172.21.15.143:0/4249804766 --> v2:172.21.15.143:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6f9c15e1a0 con 0x7f6f9c0cf280 2023-11-26T07:21:42.702 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:21:42.700+0000 7f6fa1815640 1 --2- 172.21.15.143:0/4249804766 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f6f9c0cf280 0x7f6f9c0cf660 secure :-1 s=READY pgs=736 cs=0 l=1 rev1=1 crypto rx=0x7f6f9c06e480 tx=0x7f6f9801ecb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=947a13fbe3861eb0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:21:42.702 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:21:42.700+0000 7f6fa0813640 1 -- 172.21.15.143:0/4249804766 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6f980a89b0 con 0x7f6f9c0cf280 2023-11-26T07:21:42.702 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:21:42.700+0000 7f6fa0813640 1 -- 172.21.15.143:0/4249804766 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6f980a8b10 con 0x7f6f9c0cf280 2023-11-26T07:21:42.702 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:21:42.700+0000 7f6fa0813640 1 -- 172.21.15.143:0/4249804766 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6f980a8ea0 con 0x7f6f9c0cf280 2023-11-26T07:21:42.703 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:21:42.700+0000 7f6fa3a79640 1 -- 172.21.15.143:0/4249804766 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f6f9c0cf280 msgr2=0x7f6f9c0cf660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:21:42.703 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:21:42.700+0000 7f6fa3a79640 1 --2- 172.21.15.143:0/4249804766 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f6f9c0cf280 0x7f6f9c0cf660 secure :-1 s=READY pgs=736 cs=0 l=1 rev1=1 crypto rx=0x7f6f9c06e480 tx=0x7f6f9801ecb0 comp rx=0 tx=0).stop 2023-11-26T07:21:42.703 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:21:42.700+0000 7f6fa3a79640 1 -- 172.21.15.143:0/4249804766 shutdown_connections 2023-11-26T07:21:42.703 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:21:42.700+0000 7f6fa3a79640 1 --2- 172.21.15.143:0/4249804766 >> v2:172.21.15.188:3300/0 conn(0x7f6f9c070d10 0x7f6f9c15d900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:21:42.704 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:21:42.700+0000 7f6fa3a79640 1 --2- 172.21.15.143:0/4249804766 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f6f9c0cf280 0x7f6f9c0cf660 unknown :-1 s=CLOSED pgs=736 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:21:42.704 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:21:42.700+0000 7f6fa3a79640 1 -- 172.21.15.143:0/4249804766 >> 172.21.15.143:0/4249804766 conn(0x7f6f9c06a890 msgr2=0x7f6f9c06aca0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:21:42.704 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:21:42.700+0000 7f6fa3a79640 1 -- 172.21.15.143:0/4249804766 shutdown_connections 2023-11-26T07:21:42.704 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:21:42.700+0000 7f6fa3a79640 1 -- 172.21.15.143:0/4249804766 wait complete. 2023-11-26T07:21:42.705 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:21:42.704+0000 7f6fa3a79640 1 Processor -- start 2023-11-26T07:21:42.705 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:21:42.704+0000 7f6fa3a79640 1 -- start start 2023-11-26T07:21:42.707 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:21:42.704+0000 7f6fa3a79640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f6f9c06cd50 0x7f6f9c1f5d70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:21:42.707 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:21:42.704+0000 7f6fa3a79640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f6f9c070d10 0x7f6f9c0c3e60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:21:42.707 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:21:42.704+0000 7f6fa3a79640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f6f9c0cf280 0x7f6f9c0c4c30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:21:42.707 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:21:42.704+0000 7f6fa3a79640 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6f9c0c5860 con 0x7f6f9c06cd50 2023-11-26T07:21:42.708 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:21:42.704+0000 7f6fa3a79640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6f9c14dd30 con 0x7f6f9c0cf280 2023-11-26T07:21:42.708 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:21:42.704+0000 7f6fa3a79640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6f9c14e010 con 0x7f6f9c070d10 2023-11-26T07:21:42.708 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:21:42.704+0000 7f6fa1815640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f6f9c06cd50 0x7f6f9c1f5d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:21:42.708 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:21:42.704+0000 7f6fa1815640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f6f9c06cd50 0x7f6f9c1f5d70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:35860/0 (socket says 172.21.15.143:35860) 2023-11-26T07:21:42.708 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:21:42.704+0000 7f6fa1815640 1 -- 172.21.15.143:0/3032577465 learned_addr learned my addr 172.21.15.143:0/3032577465 (peer_addr_for_me v2:172.21.15.143:0/0) 2023-11-26T07:21:42.708 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:21:42.704+0000 7f6fa1815640 1 -- 172.21.15.143:0/3032577465 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f6f9c070d10 msgr2=0x7f6f9c0c3e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:21:42.708 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:21:42.704+0000 7f6fa1014640 1 --2- 172.21.15.143:0/3032577465 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f6f9c070d10 0x7f6f9c0c3e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:21:42.708 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:21:42.704+0000 7f6fa2016640 1 --2- 172.21.15.143:0/3032577465 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f6f9c0cf280 0x7f6f9c0c4c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:21:42.708 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:21:42.704+0000 7f6fa1815640 1 --2- 172.21.15.143:0/3032577465 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f6f9c070d10 0x7f6f9c0c3e60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:21:42.708 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:21:42.704+0000 7f6fa1815640 1 -- 172.21.15.143:0/3032577465 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f6f9c0cf280 msgr2=0x7f6f9c0c4c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:21:42.709 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:21:42.704+0000 7f6fa1815640 1 --2- 172.21.15.143:0/3032577465 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f6f9c0cf280 0x7f6f9c0c4c30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:21:42.709 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:21:42.704+0000 7f6fa1815640 1 -- 172.21.15.143:0/3032577465 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6f980094d0 con 0x7f6f9c06cd50 2023-11-26T07:21:42.709 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:21:42.704+0000 7f6fa2016640 1 --2- 172.21.15.143:0/3032577465 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f6f9c0cf280 0x7f6f9c0c4c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-26T07:21:42.709 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:21:42.704+0000 7f6fa1014640 1 --2- 172.21.15.143:0/3032577465 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f6f9c070d10 0x7f6f9c0c3e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-26T07:21:42.709 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:21:42.704+0000 7f6fa1815640 1 --2- 172.21.15.143:0/3032577465 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f6f9c06cd50 0x7f6f9c1f5d70 secure :-1 s=READY pgs=737 cs=0 l=1 rev1=1 crypto rx=0x7f6f980b3040 tx=0x7f6f9801ba50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:21:42.709 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:21:42.704+0000 7f6f927fc640 1 -- 172.21.15.143:0/3032577465 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6f980b0e00 con 0x7f6f9c06cd50 2023-11-26T07:21:42.709 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:21:42.704+0000 7f6f927fc640 1 -- 172.21.15.143:0/3032577465 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6f980bb470 con 0x7f6f9c06cd50 2023-11-26T07:21:42.709 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:21:42.704+0000 7f6f927fc640 1 -- 172.21.15.143:0/3032577465 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6f980bb820 con 0x7f6f9c06cd50 2023-11-26T07:21:42.709 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:21:42.704+0000 7f6fa3a79640 1 -- 172.21.15.143:0/3032577465 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6f9c14e2f0 con 0x7f6f9c06cd50 2023-11-26T07:21:42.709 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:21:42.704+0000 7f6fa3a79640 1 -- 172.21.15.143:0/3032577465 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6f9c14e8b0 con 0x7f6f9c06cd50 2023-11-26T07:21:42.710 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:21:42.704+0000 7f6fa3a79640 1 -- 172.21.15.143:0/3032577465 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6f9c14eaa0 con 0x7f6f9c06cd50 2023-11-26T07:21:42.710 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:21:42.708+0000 7f6f927fc640 1 -- 172.21.15.143:0/3032577465 <== mon.0 v2:172.21.15.143:3300/0 4 ==== mgrmap(e 9) v1 ==== 86005+0+0 (secure 0 0 0) 0x7f6f980b7070 con 0x7f6f9c06cd50 2023-11-26T07:21:42.710 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:21:42.708+0000 7f6f927fc640 1 --2- 172.21.15.143:0/3032577465 >> [v2:172.21.15.143:6832/2150557544,v1:172.21.15.143:6833/2150557544] conn(0x7f6f80066620 0x7f6f80068ae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:21:42.710 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:21:42.708+0000 7f6f927fc640 1 -- 172.21.15.143:0/3032577465 <== mon.0 v2:172.21.15.143:3300/0 5 ==== osd_map(47..47 src has 1..47) v4 ==== 5767+0+0 (secure 0 0 0) 0x7f6f98122420 con 0x7f6f9c06cd50 2023-11-26T07:21:42.710 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:21:42.708+0000 7f6fa1014640 1 --2- 172.21.15.143:0/3032577465 >> [v2:172.21.15.143:6832/2150557544,v1:172.21.15.143:6833/2150557544] conn(0x7f6f80066620 0x7f6f80068ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:21:42.711 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:21:42.708+0000 7f6fa1014640 1 --2- 172.21.15.143:0/3032577465 >> [v2:172.21.15.143:6832/2150557544,v1:172.21.15.143:6833/2150557544] conn(0x7f6f80066620 0x7f6f80068ae0 secure :-1 s=READY pgs=353 cs=0 l=1 rev1=1 crypto rx=0x7f6f9c0cedf0 tx=0x7f6f8c008040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:21:42.714 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:21:42.712+0000 7f6f927fc640 1 -- 172.21.15.143:0/3032577465 <== mon.0 v2:172.21.15.143:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172649 (secure 0 0 0) 0x7f6f980ef670 con 0x7f6f9c06cd50 2023-11-26T07:21:42.863 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:21:42.860+0000 7f6fa3a79640 1 -- 172.21.15.143:0/3032577465 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f6f9c067680 con 0x7f6f9c06cd50 2023-11-26T07:21:42.865 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:21:42.864+0000 7f6f927fc640 1 -- 172.21.15.143:0/3032577465 <== mon.0 v2:172.21.15.143:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v47) v1 ==== 74+0+16645 (secure 0 0 0) 0x7f6f980f2c90 con 0x7f6f9c06cd50 2023-11-26T07:21:42.865 INFO:teuthology.orchestra.run.smithi143.stdout: 2023-11-26T07:21:42.865 INFO:teuthology.orchestra.run.smithi143.stdout:{"epoch":47,"fsid":"f5033060-2469-4a07-95fe-2b76f5404bc1","created":"2023-11-26T07:17:28.613663+0000","modified":"2023-11-26T07:20:36.636824+0000","last_up_change":"2023-11-26T07:17:41.157813+0000","last_in_change":"2023-11-26T07:17:31.181658+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":7,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-11-26T07:17:41.758597+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":6,"pool_name":"cephfs_metadata","create_time":"2023-11-26T07:19:29.208075+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"44","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":7,"pool_name":"cephfs_data","create_time":"2023-11-26T07:19:30.503743+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"45","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"ebbee2c6-023e-44b9-9f1d-6f7bfd309730","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":43,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6817","nonce":639807369},{"type":"v1","addr":"172.21.15.143:6819","nonce":639807369}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6821","nonce":639807369},{"type":"v1","addr":"172.21.15.143:6823","nonce":639807369}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6830","nonce":639807369},{"type":"v1","addr":"172.21.15.143:6831","nonce":639807369}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6825","nonce":639807369},{"type":"v1","addr":"172.21.15.143:6828","nonce":639807369}]},"public_addr":"172.21.15.143:6819/639807369","cluster_addr":"172.21.15.143:6823/639807369","heartbeat_back_addr":"172.21.15.143:6831/639807369","heartbeat_front_addr":"172.21.15.143:6828/639807369","state":["exists","up"]},{"osd":1,"uuid":"fec4b8be-7122-49e9-9a19-5850a0700001","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":43,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6816","nonce":1144942530},{"type":"v1","addr":"172.21.15.143:6818","nonce":1144942530}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6820","nonce":1144942530},{"type":"v1","addr":"172.21.15.143:6822","nonce":1144942530}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6827","nonce":1144942530},{"type":"v1","addr":"172.21.15.143:6829","nonce":1144942530}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6824","nonce":1144942530},{"type":"v1","addr":"172.21.15.143:6826","nonce":1144942530}]},"public_addr":"172.21.15.143:6818/1144942530","cluster_addr":"172.21.15.143:6822/1144942530","heartbeat_back_addr":"172.21.15.143:6829/1144942530","heartbeat_front_addr":"172.21.15.143:6826/1144942530","state":["exists","up"]},{"osd":2,"uuid":"2888af5b-f933-46ca-8f95-6dd494ff43f6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":46,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6808","nonce":3909095186},{"type":"v1","addr":"172.21.15.143:6809","nonce":3909095186}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6810","nonce":3909095186},{"type":"v1","addr":"172.21.15.143:6811","nonce":3909095186}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6814","nonce":3909095186},{"type":"v1","addr":"172.21.15.143:6815","nonce":3909095186}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6812","nonce":3909095186},{"type":"v1","addr":"172.21.15.143:6813","nonce":3909095186}]},"public_addr":"172.21.15.143:6809/3909095186","cluster_addr":"172.21.15.143:6811/3909095186","heartbeat_back_addr":"172.21.15.143:6815/3909095186","heartbeat_front_addr":"172.21.15.143:6813/3909095186","state":["exists","up"]},{"osd":3,"uuid":"d1f52435-7bc1-4081-9550-9cd1ba55f549","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":43,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6800","nonce":3854210991},{"type":"v1","addr":"172.21.15.143:6801","nonce":3854210991}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6802","nonce":3854210991},{"type":"v1","addr":"172.21.15.143:6803","nonce":3854210991}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6806","nonce":3854210991},{"type":"v1","addr":"172.21.15.143:6807","nonce":3854210991}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6804","nonce":3854210991},{"type":"v1","addr":"172.21.15.143:6805","nonce":3854210991}]},"public_addr":"172.21.15.143:6801/3854210991","cluster_addr":"172.21.15.143:6803/3854210991","heartbeat_back_addr":"172.21.15.143:6807/3854210991","heartbeat_front_addr":"172.21.15.143:6805/3854210991","state":["exists","up"]},{"osd":4,"uuid":"e2ae8b6e-6e25-49b2-bcbf-aacce68f06a3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":43,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6800","nonce":3924606902},{"type":"v1","addr":"172.21.15.188:6801","nonce":3924606902}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6802","nonce":3924606902},{"type":"v1","addr":"172.21.15.188:6803","nonce":3924606902}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6806","nonce":3924606902},{"type":"v1","addr":"172.21.15.188:6807","nonce":3924606902}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6804","nonce":3924606902},{"type":"v1","addr":"172.21.15.188:6805","nonce":3924606902}]},"public_addr":"172.21.15.188:6801/3924606902","cluster_addr":"172.21.15.188:6803/3924606902","heartbeat_back_addr":"172.21.15.188:6807/3924606902","heartbeat_front_addr":"172.21.15.188:6805/3924606902","state":["exists","up"]},{"osd":5,"uuid":"3cec8a32-7698-4c36-8e89-e4ba11934f33","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":43,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6813","nonce":186841544},{"type":"v1","addr":"172.21.15.188:6815","nonce":186841544}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6817","nonce":186841544},{"type":"v1","addr":"172.21.15.188:6818","nonce":186841544}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6822","nonce":186841544},{"type":"v1","addr":"172.21.15.188:6823","nonce":186841544}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6820","nonce":186841544},{"type":"v1","addr":"172.21.15.188:6821","nonce":186841544}]},"public_addr":"172.21.15.188:6815/186841544","cluster_addr":"172.21.15.188:6818/186841544","heartbeat_back_addr":"172.21.15.188:6823/186841544","heartbeat_front_addr":"172.21.15.188:6821/186841544","state":["exists","up"]},{"osd":6,"uuid":"58a65d5d-e090-473a-81b4-ad38e85029f9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":46,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6808","nonce":2153499405},{"type":"v1","addr":"172.21.15.188:6809","nonce":2153499405}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6810","nonce":2153499405},{"type":"v1","addr":"172.21.15.188:6811","nonce":2153499405}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6816","nonce":2153499405},{"type":"v1","addr":"172.21.15.188:6819","nonce":2153499405}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6812","nonce":2153499405},{"type":"v1","addr":"172.21.15.188:6814","nonce":2153499405}]},"public_addr":"172.21.15.188:6809/2153499405","cluster_addr":"172.21.15.188:6811/2153499405","heartbeat_back_addr":"172.21.15.188:6819/2153499405","heartbeat_front_addr":"172.21.15.188:6814/2153499405","state":["exists","up"]},{"osd":7,"uuid":"ea5b7f70-3554-4cf8-ac30-f7c59b80678d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":43,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6824","nonce":2722096729},{"type":"v1","addr":"172.21.15.188:6825","nonce":2722096729}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6826","nonce":2722096729},{"type":"v1","addr":"172.21.15.188:6827","nonce":2722096729}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6830","nonce":2722096729},{"type":"v1","addr":"172.21.15.188:6831","nonce":2722096729}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6828","nonce":2722096729},{"type":"v1","addr":"172.21.15.188:6829","nonce":2722096729}]},"public_addr":"172.21.15.188:6825/2722096729","cluster_addr":"172.21.15.188:6827/2722096729","heartbeat_back_addr":"172.21.15.188:6831/2722096729","heartbeat_front_addr":"172.21.15.188:6829/2722096729","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-26T07:17:36.995032+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-26T07:17:36.932266+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-26T07:17:36.829304+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-26T07:17:36.804297+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-26T07:17:36.667892+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-26T07:17:36.964868+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-26T07:17:36.827332+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-26T07:17:36.893179+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"7.2c","mappings":[{"from":0,"to":4}]},{"pgid":"7.2d","mappings":[{"from":0,"to":5}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-11-26T07:21:42.867 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:21:42.864+0000 7f6fa3a79640 1 -- 172.21.15.143:0/3032577465 >> [v2:172.21.15.143:6832/2150557544,v1:172.21.15.143:6833/2150557544] conn(0x7f6f80066620 msgr2=0x7f6f80068ae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:21:42.867 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:21:42.864+0000 7f6fa3a79640 1 --2- 172.21.15.143:0/3032577465 >> [v2:172.21.15.143:6832/2150557544,v1:172.21.15.143:6833/2150557544] conn(0x7f6f80066620 0x7f6f80068ae0 secure :-1 s=READY pgs=353 cs=0 l=1 rev1=1 crypto rx=0x7f6f9c0cedf0 tx=0x7f6f8c008040 comp rx=0 tx=0).stop 2023-11-26T07:21:42.867 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:21:42.864+0000 7f6fa3a79640 1 -- 172.21.15.143:0/3032577465 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f6f9c06cd50 msgr2=0x7f6f9c1f5d70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:21:42.867 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:21:42.864+0000 7f6fa3a79640 1 --2- 172.21.15.143:0/3032577465 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f6f9c06cd50 0x7f6f9c1f5d70 secure :-1 s=READY pgs=737 cs=0 l=1 rev1=1 crypto rx=0x7f6f980b3040 tx=0x7f6f9801ba50 comp rx=0 tx=0).stop 2023-11-26T07:21:42.868 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:21:42.864+0000 7f6fa3a79640 1 -- 172.21.15.143:0/3032577465 shutdown_connections 2023-11-26T07:21:42.868 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:21:42.864+0000 7f6fa3a79640 1 --2- 172.21.15.143:0/3032577465 >> [v2:172.21.15.143:6832/2150557544,v1:172.21.15.143:6833/2150557544] conn(0x7f6f80066620 0x7f6f80068ae0 unknown :-1 s=CLOSED pgs=353 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:21:42.868 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:21:42.864+0000 7f6fa3a79640 1 --2- 172.21.15.143:0/3032577465 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f6f9c0cf280 0x7f6f9c0c4c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:21:42.868 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:21:42.864+0000 7f6fa3a79640 1 --2- 172.21.15.143:0/3032577465 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f6f9c070d10 0x7f6f9c0c3e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:21:42.868 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:21:42.864+0000 7f6fa3a79640 1 --2- 172.21.15.143:0/3032577465 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f6f9c06cd50 0x7f6f9c1f5d70 unknown :-1 s=CLOSED pgs=737 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:21:42.868 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:21:42.864+0000 7f6fa3a79640 1 -- 172.21.15.143:0/3032577465 >> 172.21.15.143:0/3032577465 conn(0x7f6f9c06a890 msgr2=0x7f6f9c15c7a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:21:42.868 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:21:42.864+0000 7f6fa3a79640 1 -- 172.21.15.143:0/3032577465 shutdown_connections 2023-11-26T07:21:42.868 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:21:42.864+0000 7f6fa3a79640 1 -- 172.21.15.143:0/3032577465 wait complete. 2023-11-26T07:21:42.886 DEBUG:teuthology.orchestra.run.smithi143:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs fail cephfs 2023-11-26T07:21:43.024 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:21:43.020+0000 7f3885eaa640 1 Processor -- start 2023-11-26T07:21:43.024 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:21:43.020+0000 7f3885eaa640 1 -- start start 2023-11-26T07:21:43.026 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:21:43.024+0000 7f3885eaa640 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7f38801596d0 0x7f3880071850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:21:43.026 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:21:43.024+0000 7f3885eaa640 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7f38800ce680 0x7f38800b2870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:21:43.026 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:21:43.024+0000 7f3885eaa640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f38800b2db0 0x7f388015d2c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:21:43.026 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:21:43.024+0000 7f3885eaa640 1 -- --> v2:172.21.15.143:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f38800b3240 con 0x7f38801596d0 2023-11-26T07:21:43.026 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:21:43.024+0000 7f3885eaa640 1 -- --> v2:172.21.15.188:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f38800cbe00 con 0x7f38800ce680 2023-11-26T07:21:43.026 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:21:43.024+0000 7f3885eaa640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f388006c750 con 0x7f38800b2db0 2023-11-26T07:21:43.026 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:21:43.024+0000 7f387f7fe640 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7f38801596d0 0x7f3880071850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:21:43.026 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:21:43.024+0000 7f387f7fe640 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7f38801596d0 0x7f3880071850 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:35864/0 (socket says 172.21.15.143:35864) 2023-11-26T07:21:43.027 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:21:43.024+0000 7f387f7fe640 1 -- 172.21.15.143:0/767627842 learned_addr learned my addr 172.21.15.143:0/767627842 (peer_addr_for_me v2:172.21.15.143:0/0) 2023-11-26T07:21:43.027 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:21:43.024+0000 7f387f7fe640 1 -- 172.21.15.143:0/767627842 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f38800b2db0 msgr2=0x7f388015d2c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-26T07:21:43.027 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:21:43.024+0000 7f387ffff640 1 --2- 172.21.15.143:0/767627842 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f38800b2db0 0x7f388015d2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:21:43.027 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:21:43.024+0000 7f387effd640 1 --2- 172.21.15.143:0/767627842 >> v2:172.21.15.188:3300/0 conn(0x7f38800ce680 0x7f38800b2870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:21:43.028 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:21:43.024+0000 7f387f7fe640 1 --2- 172.21.15.143:0/767627842 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f38800b2db0 0x7f388015d2c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:21:43.028 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:21:43.024+0000 7f387f7fe640 1 -- 172.21.15.143:0/767627842 >> v2:172.21.15.188:3300/0 conn(0x7f38800ce680 msgr2=0x7f38800b2870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-26T07:21:43.028 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:21:43.024+0000 7f387f7fe640 1 --2- 172.21.15.143:0/767627842 >> v2:172.21.15.188:3300/0 conn(0x7f38800ce680 0x7f38800b2870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:21:43.028 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:21:43.024+0000 7f387f7fe640 1 -- 172.21.15.143:0/767627842 --> v2:172.21.15.143:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f38800b3550 con 0x7f38801596d0 2023-11-26T07:21:43.028 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:21:43.024+0000 7f387ffff640 1 --2- 172.21.15.143:0/767627842 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f38800b2db0 0x7f388015d2c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-26T07:21:43.028 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:21:43.024+0000 7f387effd640 1 --2- 172.21.15.143:0/767627842 >> v2:172.21.15.188:3300/0 conn(0x7f38800ce680 0x7f38800b2870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-26T07:21:43.029 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:21:43.028+0000 7f387f7fe640 1 --2- 172.21.15.143:0/767627842 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f38801596d0 0x7f3880071850 secure :-1 s=READY pgs=738 cs=0 l=1 rev1=1 crypto rx=0x7f38800ce440 tx=0x7f387001c500 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b997f83669f7410 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:21:43.029 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:21:43.028+0000 7f387e7fc640 1 -- 172.21.15.143:0/767627842 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f38700ab600 con 0x7f38801596d0 2023-11-26T07:21:43.029 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:21:43.028+0000 7f387e7fc640 1 -- 172.21.15.143:0/767627842 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f38700a7050 con 0x7f38801596d0 2023-11-26T07:21:43.030 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:21:43.028+0000 7f387e7fc640 1 -- 172.21.15.143:0/767627842 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f38700ab980 con 0x7f38801596d0 2023-11-26T07:21:43.030 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:21:43.028+0000 7f3885eaa640 1 -- 172.21.15.143:0/767627842 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f38801596d0 msgr2=0x7f3880071850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:21:43.030 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:21:43.028+0000 7f3885eaa640 1 --2- 172.21.15.143:0/767627842 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f38801596d0 0x7f3880071850 secure :-1 s=READY pgs=738 cs=0 l=1 rev1=1 crypto rx=0x7f38800ce440 tx=0x7f387001c500 comp rx=0 tx=0).stop 2023-11-26T07:21:43.031 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:21:43.028+0000 7f3885eaa640 1 -- 172.21.15.143:0/767627842 shutdown_connections 2023-11-26T07:21:43.031 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:21:43.028+0000 7f3885eaa640 1 --2- 172.21.15.143:0/767627842 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f38800b2db0 0x7f388015d2c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:21:43.031 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:21:43.028+0000 7f3885eaa640 1 --2- 172.21.15.143:0/767627842 >> v2:172.21.15.188:3300/0 conn(0x7f38800ce680 0x7f38800b2870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:21:43.031 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:21:43.028+0000 7f3885eaa640 1 --2- 172.21.15.143:0/767627842 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f38801596d0 0x7f3880071850 unknown :-1 s=CLOSED pgs=738 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:21:43.031 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:21:43.028+0000 7f3885eaa640 1 -- 172.21.15.143:0/767627842 >> 172.21.15.143:0/767627842 conn(0x7f388006a890 msgr2=0x7f388006aca0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:21:43.031 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:21:43.028+0000 7f3885eaa640 1 -- 172.21.15.143:0/767627842 shutdown_connections 2023-11-26T07:21:43.031 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:21:43.028+0000 7f3885eaa640 1 -- 172.21.15.143:0/767627842 wait complete. 2023-11-26T07:21:43.032 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:21:43.028+0000 7f3885eaa640 1 Processor -- start 2023-11-26T07:21:43.032 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:21:43.028+0000 7f3885eaa640 1 -- start start 2023-11-26T07:21:43.034 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:21:43.032+0000 7f3885eaa640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f38800b2db0 0x7f388013e690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:21:43.034 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:21:43.032+0000 7f3885eaa640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f38800ce680 0x7f3880151070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:21:43.034 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:21:43.032+0000 7f3885eaa640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f38801596d0 0x7f3880070c10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:21:43.034 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:21:43.032+0000 7f3885eaa640 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f38801f5a60 con 0x7f38800b2db0 2023-11-26T07:21:43.034 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:21:43.032+0000 7f3885eaa640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3880151aa0 con 0x7f38800ce680 2023-11-26T07:21:43.034 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:21:43.032+0000 7f3885eaa640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3880151d80 con 0x7f38801596d0 2023-11-26T07:21:43.034 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:21:43.032+0000 7f387f7fe640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f38800b2db0 0x7f388013e690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:21:43.034 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:21:43.032+0000 7f387f7fe640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f38800b2db0 0x7f388013e690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:35872/0 (socket says 172.21.15.143:35872) 2023-11-26T07:21:43.034 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:21:43.032+0000 7f387f7fe640 1 -- 172.21.15.143:0/267754149 learned_addr learned my addr 172.21.15.143:0/267754149 (peer_addr_for_me v2:172.21.15.143:0/0) 2023-11-26T07:21:43.035 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:21:43.032+0000 7f387f7fe640 1 -- 172.21.15.143:0/267754149 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f38801596d0 msgr2=0x7f3880070c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:21:43.035 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:21:43.032+0000 7f387f7fe640 1 --2- 172.21.15.143:0/267754149 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f38801596d0 0x7f3880070c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:21:43.035 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:21:43.032+0000 7f387f7fe640 1 -- 172.21.15.143:0/267754149 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f38800ce680 msgr2=0x7f3880151070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:21:43.035 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:21:43.032+0000 7f387effd640 1 --2- 172.21.15.143:0/267754149 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f38800ce680 0x7f3880151070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:21:43.035 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:21:43.032+0000 7f387f7fe640 1 --2- 172.21.15.143:0/267754149 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f38800ce680 0x7f3880151070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:21:43.035 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:21:43.032+0000 7f387f7fe640 1 -- 172.21.15.143:0/267754149 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f38700094d0 con 0x7f38800b2db0 2023-11-26T07:21:43.035 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:21:43.032+0000 7f387effd640 1 --2- 172.21.15.143:0/267754149 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f38800ce680 0x7f3880151070 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-26T07:21:43.035 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:21:43.032+0000 7f387f7fe640 1 --2- 172.21.15.143:0/267754149 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f38800b2db0 0x7f388013e690 secure :-1 s=READY pgs=739 cs=0 l=1 rev1=1 crypto rx=0x7f387001abb0 tx=0x7f38700192a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:21:43.036 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:21:43.032+0000 7f3867fff640 1 -- 172.21.15.143:0/267754149 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f38700b4640 con 0x7f38800b2db0 2023-11-26T07:21:43.036 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:21:43.032+0000 7f3867fff640 1 -- 172.21.15.143:0/267754149 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f38700abe90 con 0x7f38800b2db0 2023-11-26T07:21:43.036 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:21:43.032+0000 7f3867fff640 1 -- 172.21.15.143:0/267754149 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f38700b4a00 con 0x7f38800b2db0 2023-11-26T07:21:43.036 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:21:43.032+0000 7f3885eaa640 1 -- 172.21.15.143:0/267754149 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3880152060 con 0x7f38800b2db0 2023-11-26T07:21:43.036 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:21:43.032+0000 7f3885eaa640 1 -- 172.21.15.143:0/267754149 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f388013ebd0 con 0x7f38800b2db0 2023-11-26T07:21:43.036 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:21:43.032+0000 7f3885eaa640 1 -- 172.21.15.143:0/267754149 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f388013edc0 con 0x7f38800b2db0 2023-11-26T07:21:43.037 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:21:43.036+0000 7f3867fff640 1 -- 172.21.15.143:0/267754149 <== mon.0 v2:172.21.15.143:3300/0 4 ==== mgrmap(e 9) v1 ==== 86005+0+0 (secure 0 0 0) 0x7f38700b2070 con 0x7f38800b2db0 2023-11-26T07:21:43.037 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:21:43.036+0000 7f3867fff640 1 --2- 172.21.15.143:0/267754149 >> [v2:172.21.15.143:6832/2150557544,v1:172.21.15.143:6833/2150557544] conn(0x7f386c066620 0x7f386c068ae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:21:43.037 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:21:43.036+0000 7f3867fff640 1 -- 172.21.15.143:0/267754149 <== mon.0 v2:172.21.15.143:3300/0 5 ==== osd_map(47..47 src has 1..47) v4 ==== 5767+0+0 (secure 0 0 0) 0x7f387011e2c0 con 0x7f38800b2db0 2023-11-26T07:21:43.037 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:21:43.036+0000 7f387effd640 1 --2- 172.21.15.143:0/267754149 >> [v2:172.21.15.143:6832/2150557544,v1:172.21.15.143:6833/2150557544] conn(0x7f386c066620 0x7f386c068ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:21:43.038 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:21:43.036+0000 7f387effd640 1 --2- 172.21.15.143:0/267754149 >> [v2:172.21.15.143:6832/2150557544,v1:172.21.15.143:6833/2150557544] conn(0x7f386c066620 0x7f386c068ae0 secure :-1 s=READY pgs=354 cs=0 l=1 rev1=1 crypto rx=0x7f3874006fd0 tx=0x7f3874008040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:21:43.041 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:21:43.040+0000 7f3867fff640 1 -- 172.21.15.143:0/267754149 <== mon.0 v2:172.21.15.143:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172649 (secure 0 0 0) 0x7f38700eb360 con 0x7f38800b2db0 2023-11-26T07:21:43.220 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:21:43.216+0000 7f3885eaa640 1 -- 172.21.15.143:0/267754149 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs"} v 0) v1 -- 0x7f3880067680 con 0x7f38800b2db0 2023-11-26T07:21:43.221 INFO:tasks.ceph.mon.a.smithi143.stderr:2023-11-26T07:21:43.220+0000 7f6946f73640 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2023-11-26T07:21:43.223 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:21:43.220+0000 7f3867fff640 1 -- 172.21.15.143:0/267754149 <== mon.0 v2:172.21.15.143:3300/0 7 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs"}]=0 cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v27) v1 ==== 161+0+0 (secure 0 0 0) 0x7f38700ee980 con 0x7f38800b2db0 2023-11-26T07:21:43.223 INFO:teuthology.orchestra.run.smithi143.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2023-11-26T07:21:43.227 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:21:43.224+0000 7f3885eaa640 1 -- 172.21.15.143:0/267754149 >> [v2:172.21.15.143:6832/2150557544,v1:172.21.15.143:6833/2150557544] conn(0x7f386c066620 msgr2=0x7f386c068ae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:21:43.228 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:21:43.224+0000 7f3885eaa640 1 --2- 172.21.15.143:0/267754149 >> [v2:172.21.15.143:6832/2150557544,v1:172.21.15.143:6833/2150557544] conn(0x7f386c066620 0x7f386c068ae0 secure :-1 s=READY pgs=354 cs=0 l=1 rev1=1 crypto rx=0x7f3874006fd0 tx=0x7f3874008040 comp rx=0 tx=0).stop 2023-11-26T07:21:43.228 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:21:43.224+0000 7f3885eaa640 1 -- 172.21.15.143:0/267754149 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f38800b2db0 msgr2=0x7f388013e690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:21:43.228 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:21:43.224+0000 7f3885eaa640 1 --2- 172.21.15.143:0/267754149 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f38800b2db0 0x7f388013e690 secure :-1 s=READY pgs=739 cs=0 l=1 rev1=1 crypto rx=0x7f387001abb0 tx=0x7f38700192a0 comp rx=0 tx=0).stop 2023-11-26T07:21:43.228 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:21:43.224+0000 7f3885eaa640 1 -- 172.21.15.143:0/267754149 shutdown_connections 2023-11-26T07:21:43.228 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:21:43.224+0000 7f3885eaa640 1 --2- 172.21.15.143:0/267754149 >> [v2:172.21.15.143:6832/2150557544,v1:172.21.15.143:6833/2150557544] conn(0x7f386c066620 0x7f386c068ae0 unknown :-1 s=CLOSED pgs=354 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:21:43.228 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:21:43.224+0000 7f3885eaa640 1 --2- 172.21.15.143:0/267754149 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f38801596d0 0x7f3880070c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:21:43.228 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:21:43.224+0000 7f3885eaa640 1 --2- 172.21.15.143:0/267754149 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f38800ce680 0x7f3880151070 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:21:43.228 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:21:43.224+0000 7f3885eaa640 1 --2- 172.21.15.143:0/267754149 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f38800b2db0 0x7f388013e690 unknown :-1 s=CLOSED pgs=739 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:21:43.228 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:21:43.224+0000 7f3885eaa640 1 -- 172.21.15.143:0/267754149 >> 172.21.15.143:0/267754149 conn(0x7f388006a890 msgr2=0x7f38800b0090 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:21:43.228 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:21:43.224+0000 7f3885eaa640 1 -- 172.21.15.143:0/267754149 shutdown_connections 2023-11-26T07:21:43.228 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:21:43.224+0000 7f3885eaa640 1 -- 172.21.15.143:0/267754149 wait complete. 2023-11-26T07:21:43.242 INFO:tasks.ceph.mds.c.smithi143.stderr: -9790> 2023-11-26T07:19:37.103+0000 7f978e6e6640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2023-11-26T07:21:43.256 DEBUG:teuthology.orchestra.run.smithi143:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs rm cephfs --yes-i-really-mean-it 2023-11-26T07:21:43.301 INFO:tasks.ceph.mds.c.smithi143.stdout:starting mds.c at 2023-11-26T07:21:43.398 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:21:43.396+0000 7f21070e3640 1 Processor -- start 2023-11-26T07:21:43.398 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:21:43.396+0000 7f21070e3640 1 -- start start 2023-11-26T07:21:43.400 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:21:43.396+0000 7f21070e3640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f21000cf280 0x7f21000cf6e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:21:43.400 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:21:43.396+0000 7f21070e3640 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7f21000cd1b0 0x7f21000cd5e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:21:43.400 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:21:43.396+0000 7f21070e3640 1 -- --> v1:172.21.15.143:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f21000cfc20 con 0x7f2100143ac0 2023-11-26T07:21:43.400 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:21:43.396+0000 7f21070e3640 1 -- --> v2:172.21.15.143:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2100143ea0 con 0x7f21000cd1b0 2023-11-26T07:21:43.400 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:21:43.396+0000 7f21070e3640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f21000cc730 con 0x7f21000cf280 2023-11-26T07:21:43.401 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:21:43.396+0000 7f2105680640 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7f21000cd1b0 0x7f21000cd5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:21:43.401 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:21:43.396+0000 7f2104e7f640 1 --1- >> v1:172.21.15.143:6789/0 conn(0x7f2100143ac0 0x7f2100070370 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.143:6789/0 says I am v1:172.21.15.143:36200/0 (socket says 172.21.15.143:36200) 2023-11-26T07:21:43.401 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:21:43.396+0000 7f2104e7f640 1 -- 172.21.15.143:0/3268230263 learned_addr learned my addr 172.21.15.143:0/3268230263 (peer_addr_for_me v1:172.21.15.143:0/0) 2023-11-26T07:21:43.401 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:21:43.396+0000 7f20f7fff640 1 --2- 172.21.15.143:0/3268230263 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f21000cf280 0x7f21000cf6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:21:43.401 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:21:43.396+0000 7f2105680640 1 -- 172.21.15.143:0/3268230263 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f21000cf280 msgr2=0x7f21000cf6e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-26T07:21:43.401 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:21:43.396+0000 7f2105680640 1 --2- 172.21.15.143:0/3268230263 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f21000cf280 0x7f21000cf6e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:21:43.401 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:21:43.396+0000 7f2105680640 1 -- 172.21.15.143:0/3268230263 >> v1:172.21.15.143:6789/0 conn(0x7f2100143ac0 legacy=0x7f2100070370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:21:43.401 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:21:43.396+0000 7f2105680640 1 -- 172.21.15.143:0/3268230263 --> v2:172.21.15.143:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2100070ee0 con 0x7f21000cd1b0 2023-11-26T07:21:43.401 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:21:43.396+0000 7f20f77fe640 1 -- 172.21.15.143:0/3268230263 <== mon.0 v1:172.21.15.143:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3836615821 0 0) 0x7f21000cfc20 con 0x7f2100143ac0 2023-11-26T07:21:43.403 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:21:43.400+0000 7f2105680640 1 --2- 172.21.15.143:0/3268230263 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f21000cd1b0 0x7f21000cd5e0 secure :-1 s=READY pgs=744 cs=0 l=1 rev1=1 crypto rx=0x7f20f8009d90 tx=0x7f20f801c950 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2f8b047fe17d8111 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:21:43.403 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:21:43.400+0000 7f20f77fe640 1 -- 172.21.15.143:0/3268230263 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f20f80a5a10 con 0x7f21000cd1b0 2023-11-26T07:21:43.403 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:21:43.400+0000 7f20f77fe640 1 -- 172.21.15.143:0/3268230263 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f20f80a5b70 con 0x7f21000cd1b0 2023-11-26T07:21:43.403 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:21:43.400+0000 7f20f77fe640 1 -- 172.21.15.143:0/3268230263 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f20f80ad440 con 0x7f21000cd1b0 2023-11-26T07:21:43.404 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:21:43.400+0000 7f21070e3640 1 -- 172.21.15.143:0/3268230263 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f21000cd1b0 msgr2=0x7f21000cd5e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:21:43.404 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:21:43.400+0000 7f21070e3640 1 --2- 172.21.15.143:0/3268230263 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f21000cd1b0 0x7f21000cd5e0 secure :-1 s=READY pgs=744 cs=0 l=1 rev1=1 crypto rx=0x7f20f8009d90 tx=0x7f20f801c950 comp rx=0 tx=0).stop 2023-11-26T07:21:43.404 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:21:43.400+0000 7f21070e3640 1 -- 172.21.15.143:0/3268230263 shutdown_connections 2023-11-26T07:21:43.404 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:21:43.400+0000 7f21070e3640 1 --2- 172.21.15.143:0/3268230263 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f21000cd1b0 0x7f21000cd5e0 unknown :-1 s=CLOSED pgs=744 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:21:43.405 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:21:43.400+0000 7f21070e3640 1 --2- 172.21.15.143:0/3268230263 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f21000cf280 0x7f21000cf6e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:21:43.405 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:21:43.400+0000 7f21070e3640 1 -- 172.21.15.143:0/3268230263 >> 172.21.15.143:0/3268230263 conn(0x7f210006a890 msgr2=0x7f210006aca0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:21:43.405 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:21:43.400+0000 7f21070e3640 1 -- 172.21.15.143:0/3268230263 shutdown_connections 2023-11-26T07:21:43.405 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:21:43.400+0000 7f21070e3640 1 -- 172.21.15.143:0/3268230263 wait complete. 2023-11-26T07:21:43.405 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:21:43.404+0000 7f21070e3640 1 Processor -- start 2023-11-26T07:21:43.406 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:21:43.404+0000 7f21070e3640 1 -- start start 2023-11-26T07:21:43.408 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:21:43.404+0000 7f21070e3640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f21000cd1b0 0x7f2100157640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:21:43.408 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:21:43.404+0000 7f21070e3640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f21000cf280 0x7f2100159460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:21:43.408 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:21:43.404+0000 7f21070e3640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f2100143ac0 0x7f21001599a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:21:43.408 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:21:43.404+0000 7f21070e3640 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2100158010 con 0x7f2100143ac0 2023-11-26T07:21:43.408 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:21:43.404+0000 7f21070e3640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f21001582b0 con 0x7f21000cf280 2023-11-26T07:21:43.408 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:21:43.404+0000 7f21070e3640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2100158590 con 0x7f21000cd1b0 2023-11-26T07:21:43.408 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:21:43.404+0000 7f2105680640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f2100143ac0 0x7f21001599a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:21:43.409 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:21:43.404+0000 7f2105680640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f2100143ac0 0x7f21001599a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:35920/0 (socket says 172.21.15.143:35920) 2023-11-26T07:21:43.409 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:21:43.404+0000 7f2105680640 1 -- 172.21.15.143:0/1320395817 learned_addr learned my addr 172.21.15.143:0/1320395817 (peer_addr_for_me v2:172.21.15.143:0/0) 2023-11-26T07:21:43.409 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:21:43.404+0000 7f2104e7f640 1 --2- 172.21.15.143:0/1320395817 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f21000cd1b0 0x7f2100157640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:21:43.409 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:21:43.404+0000 7f20f7fff640 1 --2- 172.21.15.143:0/1320395817 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f21000cf280 0x7f2100159460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:21:43.409 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:21:43.404+0000 7f2105680640 1 -- 172.21.15.143:0/1320395817 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f21000cd1b0 msgr2=0x7f2100157640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:21:43.409 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:21:43.404+0000 7f2105680640 1 --2- 172.21.15.143:0/1320395817 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f21000cd1b0 0x7f2100157640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:21:43.409 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:21:43.404+0000 7f2105680640 1 -- 172.21.15.143:0/1320395817 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f21000cf280 msgr2=0x7f2100159460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:21:43.409 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:21:43.404+0000 7f2105680640 1 --2- 172.21.15.143:0/1320395817 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f21000cf280 0x7f2100159460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:21:43.410 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:21:43.404+0000 7f2105680640 1 -- 172.21.15.143:0/1320395817 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f20f8009a70 con 0x7f2100143ac0 2023-11-26T07:21:43.410 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:21:43.404+0000 7f2105680640 1 --2- 172.21.15.143:0/1320395817 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f2100143ac0 0x7f21001599a0 secure :-1 s=READY pgs=745 cs=0 l=1 rev1=1 crypto rx=0x7f20f801b000 tx=0x7f20f80196f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:21:43.410 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:21:43.404+0000 7f20f57fa640 1 -- 172.21.15.143:0/1320395817 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f20f80addc0 con 0x7f2100143ac0 2023-11-26T07:21:43.410 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:21:43.404+0000 7f20f57fa640 1 -- 172.21.15.143:0/1320395817 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f20f8009450 con 0x7f2100143ac0 2023-11-26T07:21:43.410 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:21:43.404+0000 7f20f57fa640 1 -- 172.21.15.143:0/1320395817 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f20f80b9650 con 0x7f2100143ac0 2023-11-26T07:21:43.410 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:21:43.404+0000 7f21070e3640 1 -- 172.21.15.143:0/1320395817 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2100158870 con 0x7f2100143ac0 2023-11-26T07:21:43.410 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:21:43.404+0000 7f21070e3640 1 -- 172.21.15.143:0/1320395817 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2100159070 con 0x7f2100143ac0 2023-11-26T07:21:43.411 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:21:43.408+0000 7f21070e3640 1 -- 172.21.15.143:0/1320395817 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2100067680 con 0x7f2100143ac0 2023-11-26T07:21:43.411 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:21:43.408+0000 7f20f57fa640 1 -- 172.21.15.143:0/1320395817 <== mon.0 v2:172.21.15.143:3300/0 4 ==== mgrmap(e 9) v1 ==== 86005+0+0 (secure 0 0 0) 0x7f20f80b97b0 con 0x7f2100143ac0 2023-11-26T07:21:43.412 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:21:43.408+0000 7f20f57fa640 1 --2- 172.21.15.143:0/1320395817 >> [v2:172.21.15.143:6832/2150557544,v1:172.21.15.143:6833/2150557544] conn(0x7f20ec066620 0x7f20ec068ae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:21:43.412 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:21:43.408+0000 7f20f57fa640 1 -- 172.21.15.143:0/1320395817 <== mon.0 v2:172.21.15.143:3300/0 5 ==== osd_map(48..48 src has 1..48) v4 ==== 5853+0+0 (secure 0 0 0) 0x7f20f811f930 con 0x7f2100143ac0 2023-11-26T07:21:43.412 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:21:43.408+0000 7f2104e7f640 1 --2- 172.21.15.143:0/1320395817 >> [v2:172.21.15.143:6832/2150557544,v1:172.21.15.143:6833/2150557544] conn(0x7f20ec066620 0x7f20ec068ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:21:43.412 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:21:43.408+0000 7f2104e7f640 1 --2- 172.21.15.143:0/1320395817 >> [v2:172.21.15.143:6832/2150557544,v1:172.21.15.143:6833/2150557544] conn(0x7f20ec066620 0x7f20ec068ae0 secure :-1 s=READY pgs=355 cs=0 l=1 rev1=1 crypto rx=0x7f20fc000f70 tx=0x7f20fc007b80 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:21:43.415 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:21:43.412+0000 7f20f57fa640 1 -- 172.21.15.143:0/1320395817 <== mon.0 v2:172.21.15.143:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172649 (secure 0 0 0) 0x7f20f80ecaa0 con 0x7f2100143ac0 2023-11-26T07:21:43.601 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:21:43.600+0000 7f21070e3640 1 -- 172.21.15.143:0/1320395817 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true} v 0) v1 -- 0x7f2100158d20 con 0x7f2100143ac0 2023-11-26T07:21:44.226 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:21:44.224+0000 7f20f57fa640 1 -- 172.21.15.143:0/1320395817 <== mon.0 v2:172.21.15.143:3300/0 7 ==== mon_command_ack([{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]=0 v28) v1 ==== 104+0+0 (secure 0 0 0) 0x7f20f80b20a0 con 0x7f2100143ac0 2023-11-26T07:21:44.228 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:21:44.224+0000 7f21070e3640 1 -- 172.21.15.143:0/1320395817 >> [v2:172.21.15.143:6832/2150557544,v1:172.21.15.143:6833/2150557544] conn(0x7f20ec066620 msgr2=0x7f20ec068ae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:21:44.228 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:21:44.224+0000 7f21070e3640 1 --2- 172.21.15.143:0/1320395817 >> [v2:172.21.15.143:6832/2150557544,v1:172.21.15.143:6833/2150557544] conn(0x7f20ec066620 0x7f20ec068ae0 secure :-1 s=READY pgs=355 cs=0 l=1 rev1=1 crypto rx=0x7f20fc000f70 tx=0x7f20fc007b80 comp rx=0 tx=0).stop 2023-11-26T07:21:44.228 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:21:44.224+0000 7f21070e3640 1 -- 172.21.15.143:0/1320395817 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f2100143ac0 msgr2=0x7f21001599a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:21:44.228 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:21:44.224+0000 7f21070e3640 1 --2- 172.21.15.143:0/1320395817 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f2100143ac0 0x7f21001599a0 secure :-1 s=READY pgs=745 cs=0 l=1 rev1=1 crypto rx=0x7f20f801b000 tx=0x7f20f80196f0 comp rx=0 tx=0).stop 2023-11-26T07:21:44.229 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:21:44.228+0000 7f21070e3640 1 -- 172.21.15.143:0/1320395817 shutdown_connections 2023-11-26T07:21:44.229 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:21:44.228+0000 7f21070e3640 1 --2- 172.21.15.143:0/1320395817 >> [v2:172.21.15.143:6832/2150557544,v1:172.21.15.143:6833/2150557544] conn(0x7f20ec066620 0x7f20ec068ae0 unknown :-1 s=CLOSED pgs=355 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:21:44.229 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:21:44.228+0000 7f21070e3640 1 --2- 172.21.15.143:0/1320395817 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f2100143ac0 0x7f21001599a0 unknown :-1 s=CLOSED pgs=745 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:21:44.229 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:21:44.228+0000 7f21070e3640 1 --2- 172.21.15.143:0/1320395817 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f21000cf280 0x7f2100159460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:21:44.229 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:21:44.228+0000 7f21070e3640 1 --2- 172.21.15.143:0/1320395817 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f21000cd1b0 0x7f2100157640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:21:44.229 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:21:44.228+0000 7f21070e3640 1 -- 172.21.15.143:0/1320395817 >> 172.21.15.143:0/1320395817 conn(0x7f210006a890 msgr2=0x7f21000b2fd0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:21:44.230 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:21:44.228+0000 7f21070e3640 1 -- 172.21.15.143:0/1320395817 shutdown_connections 2023-11-26T07:21:44.230 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:21:44.228+0000 7f21070e3640 1 -- 172.21.15.143:0/1320395817 wait complete. 2023-11-26T07:21:44.252 DEBUG:teuthology.orchestra.run.smithi143:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_metadata cephfs_metadata --yes-i-really-really-mean-it 2023-11-26T07:21:44.441 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:21:44.440+0000 7f48b5617640 1 Processor -- start 2023-11-26T07:21:44.441 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:21:44.440+0000 7f48b5617640 1 -- start start 2023-11-26T07:21:44.442 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:21:44.440+0000 7f48b5617640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f48b00b1e10 0x7f48b015d9e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:21:44.443 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:21:44.440+0000 7f48b5617640 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7f48b015df20 0x7f48b00ce030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:21:44.443 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:21:44.440+0000 7f48b5617640 1 -- --> v1:172.21.15.188:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f48b015e860 con 0x7f48b0143810 2023-11-26T07:21:44.443 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:21:44.440+0000 7f48b5617640 1 -- --> v2:172.21.15.143:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f48b015e3b0 con 0x7f48b015df20 2023-11-26T07:21:44.443 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:21:44.440+0000 7f48b5617640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f48b00cdba0 con 0x7f48b00b1e10 2023-11-26T07:21:44.443 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:21:44.440+0000 7f48b4e16640 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7f48b015df20 0x7f48b00ce030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:21:44.443 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:21:44.440+0000 7f48b4e16640 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7f48b015df20 0x7f48b00ce030 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:49226/0 (socket says 172.21.15.143:49226) 2023-11-26T07:21:44.443 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:21:44.440+0000 7f48b4e16640 1 -- 172.21.15.143:0/2713774393 learned_addr learned my addr 172.21.15.143:0/2713774393 (peer_addr_for_me v2:172.21.15.143:0/0) 2023-11-26T07:21:44.443 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:21:44.440+0000 7f48af7fe640 1 --2- 172.21.15.143:0/2713774393 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f48b00b1e10 0x7f48b015d9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:21:44.443 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:21:44.440+0000 7f48b4e16640 1 -- 172.21.15.143:0/2713774393 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f48b00b1e10 msgr2=0x7f48b015d9e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-26T07:21:44.443 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:21:44.440+0000 7f48b4e16640 1 --2- 172.21.15.143:0/2713774393 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f48b00b1e10 0x7f48b015d9e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:21:44.444 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:21:44.440+0000 7f48b4e16640 1 -- 172.21.15.143:0/2713774393 >> v1:172.21.15.188:6789/0 conn(0x7f48b0143810 legacy=0x7f48b0072140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-26T07:21:44.444 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:21:44.440+0000 7f48b4e16640 1 -- 172.21.15.143:0/2713774393 --> v2:172.21.15.143:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f48b0143bf0 con 0x7f48b015df20 2023-11-26T07:21:44.445 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:21:44.444+0000 7f48b4e16640 1 --2- 172.21.15.143:0/2713774393 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f48b015df20 0x7f48b00ce030 secure :-1 s=READY pgs=746 cs=0 l=1 rev1=1 crypto rx=0x7f48a800d150 tx=0x7f48a801ebe0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3639f8f6d7291752 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:21:44.446 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:21:44.444+0000 7f48aeffd640 1 -- 172.21.15.143:0/2713774393 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f48a800b2d0 con 0x7f48b015df20 2023-11-26T07:21:44.447 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:21:44.444+0000 7f48aeffd640 1 -- 172.21.15.143:0/2713774393 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f48a800b430 con 0x7f48b015df20 2023-11-26T07:21:44.447 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:21:44.444+0000 7f48aeffd640 1 -- 172.21.15.143:0/2713774393 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f48a80a7bf0 con 0x7f48b015df20 2023-11-26T07:21:44.447 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:21:44.444+0000 7f48b5617640 1 -- 172.21.15.143:0/2713774393 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f48b015df20 msgr2=0x7f48b00ce030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:21:44.447 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:21:44.444+0000 7f48b5617640 1 --2- 172.21.15.143:0/2713774393 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f48b015df20 0x7f48b00ce030 secure :-1 s=READY pgs=746 cs=0 l=1 rev1=1 crypto rx=0x7f48a800d150 tx=0x7f48a801ebe0 comp rx=0 tx=0).stop 2023-11-26T07:21:44.447 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:21:44.444+0000 7f48b5617640 1 -- 172.21.15.143:0/2713774393 shutdown_connections 2023-11-26T07:21:44.447 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:21:44.444+0000 7f48b5617640 1 --2- 172.21.15.143:0/2713774393 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f48b015df20 0x7f48b00ce030 unknown :-1 s=CLOSED pgs=746 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:21:44.447 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:21:44.444+0000 7f48b5617640 1 --2- 172.21.15.143:0/2713774393 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f48b00b1e10 0x7f48b015d9e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:21:44.447 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:21:44.444+0000 7f48b5617640 1 -- 172.21.15.143:0/2713774393 >> 172.21.15.143:0/2713774393 conn(0x7f48b006a890 msgr2=0x7f48b006aca0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:21:44.447 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:21:44.444+0000 7f48b5617640 1 -- 172.21.15.143:0/2713774393 shutdown_connections 2023-11-26T07:21:44.447 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:21:44.444+0000 7f48b5617640 1 -- 172.21.15.143:0/2713774393 wait complete. 2023-11-26T07:21:44.448 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:21:44.444+0000 7f48b5617640 1 Processor -- start 2023-11-26T07:21:44.448 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:21:44.448+0000 7f48b5617640 1 -- start start 2023-11-26T07:21:44.450 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:21:44.448+0000 7f48b5617640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f48b00b1e10 0x7f48b013b500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:21:44.451 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:21:44.448+0000 7f48b5617640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f48b0143810 0x7f48b0141990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:21:44.451 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:21:44.448+0000 7f48b5617640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f48b015df20 0x7f48b0139510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:21:44.451 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:21:44.448+0000 7f48b5617640 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f48b00b0820 con 0x7f48b0143810 2023-11-26T07:21:44.451 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:21:44.448+0000 7f48b5617640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f48b013aab0 con 0x7f48b00b1e10 2023-11-26T07:21:44.451 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:21:44.448+0000 7f48b5617640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f48b013f280 con 0x7f48b015df20 2023-11-26T07:21:44.451 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:21:44.448+0000 7f48af7fe640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f48b0143810 0x7f48b0141990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:21:44.451 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:21:44.448+0000 7f48af7fe640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f48b0143810 0x7f48b0141990 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:49232/0 (socket says 172.21.15.143:49232) 2023-11-26T07:21:44.451 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:21:44.448+0000 7f48af7fe640 1 -- 172.21.15.143:0/640290441 learned_addr learned my addr 172.21.15.143:0/640290441 (peer_addr_for_me v2:172.21.15.143:0/0) 2023-11-26T07:21:44.451 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:21:44.448+0000 7f48affff640 1 --2- 172.21.15.143:0/640290441 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f48b00b1e10 0x7f48b013b500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:21:44.451 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:21:44.448+0000 7f48b4e16640 1 --2- 172.21.15.143:0/640290441 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f48b015df20 0x7f48b0139510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:21:44.451 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:21:44.448+0000 7f48af7fe640 1 -- 172.21.15.143:0/640290441 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f48b015df20 msgr2=0x7f48b0139510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:21:44.452 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:21:44.448+0000 7f48af7fe640 1 --2- 172.21.15.143:0/640290441 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f48b015df20 0x7f48b0139510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:21:44.452 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:21:44.448+0000 7f48af7fe640 1 -- 172.21.15.143:0/640290441 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f48b00b1e10 msgr2=0x7f48b013b500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:21:44.452 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:21:44.448+0000 7f48af7fe640 1 --2- 172.21.15.143:0/640290441 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f48b00b1e10 0x7f48b013b500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:21:44.452 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:21:44.448+0000 7f48af7fe640 1 -- 172.21.15.143:0/640290441 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f48a800bd40 con 0x7f48b0143810 2023-11-26T07:21:44.452 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:21:44.448+0000 7f48affff640 1 --2- 172.21.15.143:0/640290441 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f48b00b1e10 0x7f48b013b500 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-26T07:21:44.452 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:21:44.448+0000 7f48b4e16640 1 --2- 172.21.15.143:0/640290441 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f48b015df20 0x7f48b0139510 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-26T07:21:44.452 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:21:44.448+0000 7f48af7fe640 1 --2- 172.21.15.143:0/640290441 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f48b0143810 0x7f48b0141990 secure :-1 s=READY pgs=747 cs=0 l=1 rev1=1 crypto rx=0x7f48a400d800 tx=0x7f48a400dcc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:21:44.452 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:21:44.448+0000 7f48acff9640 1 -- 172.21.15.143:0/640290441 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f48a400b6c0 con 0x7f48b0143810 2023-11-26T07:21:44.452 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:21:44.448+0000 7f48acff9640 1 -- 172.21.15.143:0/640290441 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f48a400b820 con 0x7f48b0143810 2023-11-26T07:21:44.452 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:21:44.448+0000 7f48acff9640 1 -- 172.21.15.143:0/640290441 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f48a400bb90 con 0x7f48b0143810 2023-11-26T07:21:44.452 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:21:44.448+0000 7f48b5617640 1 -- 172.21.15.143:0/640290441 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f48b013f560 con 0x7f48b0143810 2023-11-26T07:21:44.453 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:21:44.448+0000 7f48b5617640 1 -- 172.21.15.143:0/640290441 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f48b013afe0 con 0x7f48b0143810 2023-11-26T07:21:44.453 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:21:44.448+0000 7f48b5617640 1 -- 172.21.15.143:0/640290441 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f48b013b1a0 con 0x7f48b0143810 2023-11-26T07:21:44.453 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:21:44.452+0000 7f48acff9640 1 -- 172.21.15.143:0/640290441 <== mon.0 v2:172.21.15.143:3300/0 4 ==== mgrmap(e 9) v1 ==== 86005+0+0 (secure 0 0 0) 0x7f48a400bcf0 con 0x7f48b0143810 2023-11-26T07:21:44.454 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:21:44.452+0000 7f48acff9640 1 --2- 172.21.15.143:0/640290441 >> [v2:172.21.15.143:6832/2150557544,v1:172.21.15.143:6833/2150557544] conn(0x7f489c066620 0x7f489c068ae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:21:44.454 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:21:44.452+0000 7f48acff9640 1 -- 172.21.15.143:0/640290441 <== mon.0 v2:172.21.15.143:3300/0 5 ==== osd_map(48..48 src has 1..48) v4 ==== 5853+0+0 (secure 0 0 0) 0x7f48a40846c0 con 0x7f48b0143810 2023-11-26T07:21:44.454 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:21:44.452+0000 7f48affff640 1 --2- 172.21.15.143:0/640290441 >> [v2:172.21.15.143:6832/2150557544,v1:172.21.15.143:6833/2150557544] conn(0x7f489c066620 0x7f489c068ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:21:44.454 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:21:44.452+0000 7f48affff640 1 --2- 172.21.15.143:0/640290441 >> [v2:172.21.15.143:6832/2150557544,v1:172.21.15.143:6833/2150557544] conn(0x7f489c066620 0x7f489c068ae0 secure :-1 s=READY pgs=357 cs=0 l=1 rev1=1 crypto rx=0x7f48b00cca70 tx=0x7f48a0009040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:21:44.458 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:21:44.456+0000 7f48acff9640 1 -- 172.21.15.143:0/640290441 <== mon.0 v2:172.21.15.143:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172649 (secure 0 0 0) 0x7f48a40518c0 con 0x7f48b0143810 2023-11-26T07:21:44.608 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:21:44.604+0000 7f48b5617640 1 -- 172.21.15.143:0/640290441 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7f48b0067500 con 0x7f48b0143810 2023-11-26T07:21:45.228 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:21:45.224+0000 7f48acff9640 1 -- 172.21.15.143:0/640290441 <== mon.0 v2:172.21.15.143:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_metadata' removed v49) v1 ==== 181+0+0 (secure 0 0 0) 0x7f48a4054ee0 con 0x7f48b0143810 2023-11-26T07:21:45.228 INFO:teuthology.orchestra.run.smithi143.stderr:pool 'cephfs_metadata' removed 2023-11-26T07:21:45.230 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:21:45.228+0000 7f48b5617640 1 -- 172.21.15.143:0/640290441 >> [v2:172.21.15.143:6832/2150557544,v1:172.21.15.143:6833/2150557544] conn(0x7f489c066620 msgr2=0x7f489c068ae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:21:45.230 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:21:45.228+0000 7f48b5617640 1 --2- 172.21.15.143:0/640290441 >> [v2:172.21.15.143:6832/2150557544,v1:172.21.15.143:6833/2150557544] conn(0x7f489c066620 0x7f489c068ae0 secure :-1 s=READY pgs=357 cs=0 l=1 rev1=1 crypto rx=0x7f48b00cca70 tx=0x7f48a0009040 comp rx=0 tx=0).stop 2023-11-26T07:21:45.230 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:21:45.228+0000 7f48b5617640 1 -- 172.21.15.143:0/640290441 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f48b0143810 msgr2=0x7f48b0141990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:21:45.231 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:21:45.228+0000 7f48b5617640 1 --2- 172.21.15.143:0/640290441 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f48b0143810 0x7f48b0141990 secure :-1 s=READY pgs=747 cs=0 l=1 rev1=1 crypto rx=0x7f48a400d800 tx=0x7f48a400dcc0 comp rx=0 tx=0).stop 2023-11-26T07:21:45.231 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:21:45.228+0000 7f48b5617640 1 -- 172.21.15.143:0/640290441 shutdown_connections 2023-11-26T07:21:45.231 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:21:45.228+0000 7f48b5617640 1 --2- 172.21.15.143:0/640290441 >> [v2:172.21.15.143:6832/2150557544,v1:172.21.15.143:6833/2150557544] conn(0x7f489c066620 0x7f489c068ae0 unknown :-1 s=CLOSED pgs=357 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:21:45.231 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:21:45.228+0000 7f48b5617640 1 --2- 172.21.15.143:0/640290441 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f48b015df20 0x7f48b0139510 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:21:45.231 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:21:45.228+0000 7f48b5617640 1 --2- 172.21.15.143:0/640290441 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f48b0143810 0x7f48b0141990 unknown :-1 s=CLOSED pgs=747 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:21:45.231 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:21:45.228+0000 7f48b5617640 1 --2- 172.21.15.143:0/640290441 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f48b00b1e10 0x7f48b013b500 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:21:45.231 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:21:45.228+0000 7f48b5617640 1 -- 172.21.15.143:0/640290441 >> 172.21.15.143:0/640290441 conn(0x7f48b006a890 msgr2=0x7f48b00afea0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:21:45.231 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:21:45.228+0000 7f48b5617640 1 -- 172.21.15.143:0/640290441 shutdown_connections 2023-11-26T07:21:45.232 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:21:45.228+0000 7f48b5617640 1 -- 172.21.15.143:0/640290441 wait complete. 2023-11-26T07:21:45.257 DEBUG:teuthology.orchestra.run.smithi143:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_data cephfs_data --yes-i-really-really-mean-it 2023-11-26T07:21:45.400 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:21:45.396+0000 7f6eea1df640 1 Processor -- start 2023-11-26T07:21:45.400 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:21:45.396+0000 7f6eea1df640 1 -- start start 2023-11-26T07:21:45.401 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:21:45.400+0000 7f6eea1df640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f6ee40cf710 0x7f6ee40cd8c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:21:45.401 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:21:45.400+0000 7f6eea1df640 1 -- --> v1:172.21.15.143:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6ee40cc730 con 0x7f6ee40cf280 2023-11-26T07:21:45.402 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:21:45.400+0000 7f6eea1df640 1 -- --> v1:172.21.15.188:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6ee4070ee0 con 0x7f6ee4143ac0 2023-11-26T07:21:45.402 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:21:45.400+0000 7f6eea1df640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6ee4143ea0 con 0x7f6ee40cf710 2023-11-26T07:21:45.402 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:21:45.400+0000 7f6edbfff640 1 --1- >> v1:172.21.15.143:6789/0 conn(0x7f6ee40cf280 0x7f6ee40cd1b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.143:6789/0 says I am v1:172.21.15.143:52714/0 (socket says 172.21.15.143:52714) 2023-11-26T07:21:45.402 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:21:45.400+0000 7f6edbfff640 1 -- 172.21.15.143:0/4050157191 learned_addr learned my addr 172.21.15.143:0/4050157191 (peer_addr_for_me v1:172.21.15.143:0/0) 2023-11-26T07:21:45.402 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:21:45.400+0000 7f6ee3fff640 1 --2- 172.21.15.143:0/4050157191 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f6ee40cf710 0x7f6ee40cd8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:21:45.403 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:21:45.400+0000 7f6ee2ffd640 1 -- 172.21.15.143:0/4050157191 <== mon.0 v1:172.21.15.143:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2220934604 0 0) 0x7f6ee40cc730 con 0x7f6ee40cf280 2023-11-26T07:21:45.403 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:21:45.400+0000 7f6ee2ffd640 1 -- 172.21.15.143:0/4050157191 --> v1:172.21.15.143:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f6ed0003530 con 0x7f6ee40cf280 2023-11-26T07:21:45.403 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:21:45.400+0000 7f6ee2ffd640 1 -- 172.21.15.143:0/4050157191 <== mon.1 v1:172.21.15.188:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1362310156 0 0) 0x7f6ee4070ee0 con 0x7f6ee4143ac0 2023-11-26T07:21:45.403 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:21:45.400+0000 7f6ee2ffd640 1 -- 172.21.15.143:0/4050157191 --> v1:172.21.15.188:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f6ee40cc730 con 0x7f6ee4143ac0 2023-11-26T07:21:45.403 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:21:45.400+0000 7f6ee3fff640 1 -- 172.21.15.143:0/4050157191 >> v1:172.21.15.188:6789/0 conn(0x7f6ee4143ac0 legacy=0x7f6ee4070370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:21:45.403 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:21:45.400+0000 7f6ee3fff640 1 -- 172.21.15.143:0/4050157191 >> v1:172.21.15.143:6789/0 conn(0x7f6ee40cf280 legacy=0x7f6ee40cd1b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:21:45.403 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:21:45.400+0000 7f6ee3fff640 1 -- 172.21.15.143:0/4050157191 --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6ee40711c0 con 0x7f6ee40cf710 2023-11-26T07:21:45.405 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:21:45.404+0000 7f6ee3fff640 1 --2- 172.21.15.143:0/4050157191 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f6ee40cf710 0x7f6ee40cd8c0 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7f6edc009dc0 tx=0x7f6edc01c970 comp rx=0 tx=0).ready entity=mon.2 client_cookie=696de133b40227c6 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:21:45.405 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:21:45.404+0000 7f6ee2ffd640 1 -- 172.21.15.143:0/4050157191 <== mon.2 v2:172.21.15.188:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6edc0a5a30 con 0x7f6ee40cf710 2023-11-26T07:21:45.405 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:21:45.404+0000 7f6ee2ffd640 1 -- 172.21.15.143:0/4050157191 <== mon.2 v2:172.21.15.188:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6edc0a5b90 con 0x7f6ee40cf710 2023-11-26T07:21:45.405 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:21:45.404+0000 7f6ee2ffd640 1 -- 172.21.15.143:0/4050157191 <== mon.2 v2:172.21.15.188:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6edc0ad440 con 0x7f6ee40cf710 2023-11-26T07:21:45.406 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:21:45.404+0000 7f6eea1df640 1 -- 172.21.15.143:0/4050157191 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f6ee40cf710 msgr2=0x7f6ee40cd8c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:21:45.406 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:21:45.404+0000 7f6eea1df640 1 --2- 172.21.15.143:0/4050157191 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f6ee40cf710 0x7f6ee40cd8c0 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7f6edc009dc0 tx=0x7f6edc01c970 comp rx=0 tx=0).stop 2023-11-26T07:21:45.406 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:21:45.404+0000 7f6eea1df640 1 -- 172.21.15.143:0/4050157191 shutdown_connections 2023-11-26T07:21:45.406 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:21:45.404+0000 7f6eea1df640 1 --2- 172.21.15.143:0/4050157191 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f6ee40cf710 0x7f6ee40cd8c0 unknown :-1 s=CLOSED pgs=102 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:21:45.407 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:21:45.404+0000 7f6eea1df640 1 -- 172.21.15.143:0/4050157191 >> 172.21.15.143:0/4050157191 conn(0x7f6ee406a890 msgr2=0x7f6ee406aca0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:21:45.407 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:21:45.404+0000 7f6eea1df640 1 -- 172.21.15.143:0/4050157191 shutdown_connections 2023-11-26T07:21:45.407 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:21:45.404+0000 7f6eea1df640 1 -- 172.21.15.143:0/4050157191 wait complete. 2023-11-26T07:21:45.410 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:21:45.404+0000 7f6eea1df640 1 Processor -- start 2023-11-26T07:21:45.411 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:21:45.404+0000 7f6eea1df640 1 -- start start 2023-11-26T07:21:45.411 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:21:45.408+0000 7f6eea1df640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f6ee40cf280 0x7f6ee413ac50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:21:45.411 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:21:45.408+0000 7f6eea1df640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f6ee40cf710 0x7f6ee40b0240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:21:45.411 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:21:45.408+0000 7f6eea1df640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f6ee4143ac0 0x7f6ee413c4a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:21:45.411 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:21:45.408+0000 7f6eea1df640 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6ee413b190 con 0x7f6ee4143ac0 2023-11-26T07:21:45.411 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:21:45.408+0000 7f6eea1df640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6ee40b0a90 con 0x7f6ee40cf280 2023-11-26T07:21:45.411 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:21:45.408+0000 7f6eea1df640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6ee40b0d70 con 0x7f6ee40cf710 2023-11-26T07:21:45.411 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:21:45.408+0000 7f6ee3fff640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f6ee4143ac0 0x7f6ee413c4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:21:45.411 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:21:45.408+0000 7f6ee3fff640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f6ee4143ac0 0x7f6ee413c4a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:49248/0 (socket says 172.21.15.143:49248) 2023-11-26T07:21:45.411 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:21:45.408+0000 7f6ee3fff640 1 -- 172.21.15.143:0/384017036 learned_addr learned my addr 172.21.15.143:0/384017036 (peer_addr_for_me v2:172.21.15.143:0/0) 2023-11-26T07:21:45.412 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:21:45.408+0000 7f6ee37fe640 1 --2- 172.21.15.143:0/384017036 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f6ee40cf280 0x7f6ee413ac50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:21:45.412 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:21:45.408+0000 7f6edbfff640 1 --2- 172.21.15.143:0/384017036 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f6ee40cf710 0x7f6ee40b0240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:21:45.412 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:21:45.408+0000 7f6ee3fff640 1 -- 172.21.15.143:0/384017036 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f6ee40cf710 msgr2=0x7f6ee40b0240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:21:45.412 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:21:45.408+0000 7f6ee3fff640 1 --2- 172.21.15.143:0/384017036 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f6ee40cf710 0x7f6ee40b0240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:21:45.412 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:21:45.408+0000 7f6ee3fff640 1 -- 172.21.15.143:0/384017036 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f6ee40cf280 msgr2=0x7f6ee413ac50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:21:45.412 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:21:45.408+0000 7f6ee3fff640 1 --2- 172.21.15.143:0/384017036 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f6ee40cf280 0x7f6ee413ac50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:21:45.412 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:21:45.408+0000 7f6ee3fff640 1 -- 172.21.15.143:0/384017036 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6edc009a70 con 0x7f6ee4143ac0 2023-11-26T07:21:45.412 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:21:45.408+0000 7f6ee3fff640 1 --2- 172.21.15.143:0/384017036 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f6ee4143ac0 0x7f6ee413c4a0 secure :-1 s=READY pgs=749 cs=0 l=1 rev1=1 crypto rx=0x7f6edc0b0040 tx=0x7f6ee40ce610 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:21:45.412 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:21:45.408+0000 7f6ee0ff9640 1 -- 172.21.15.143:0/384017036 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6edc0adcc0 con 0x7f6ee4143ac0 2023-11-26T07:21:45.412 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:21:45.408+0000 7f6ee0ff9640 1 -- 172.21.15.143:0/384017036 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6edc0ade20 con 0x7f6ee4143ac0 2023-11-26T07:21:45.412 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:21:45.408+0000 7f6ee0ff9640 1 -- 172.21.15.143:0/384017036 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6edc0b8660 con 0x7f6ee4143ac0 2023-11-26T07:21:45.412 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:21:45.408+0000 7f6eea1df640 1 -- 172.21.15.143:0/384017036 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6ee40b1050 con 0x7f6ee4143ac0 2023-11-26T07:21:45.413 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:21:45.408+0000 7f6eea1df640 1 -- 172.21.15.143:0/384017036 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6ee40b1610 con 0x7f6ee4143ac0 2023-11-26T07:21:45.413 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:21:45.408+0000 7f6eea1df640 1 -- 172.21.15.143:0/384017036 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6ee40b1800 con 0x7f6ee4143ac0 2023-11-26T07:21:45.413 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:21:45.408+0000 7f6ee0ff9640 1 -- 172.21.15.143:0/384017036 <== mon.0 v2:172.21.15.143:3300/0 4 ==== mgrmap(e 9) v1 ==== 86005+0+0 (secure 0 0 0) 0x7f6edc0b4040 con 0x7f6ee4143ac0 2023-11-26T07:21:45.413 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:21:45.412+0000 7f6ee0ff9640 1 --2- 172.21.15.143:0/384017036 >> [v2:172.21.15.143:6832/2150557544,v1:172.21.15.143:6833/2150557544] conn(0x7f6ec4066620 0x7f6ec4068ae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:21:45.414 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:21:45.412+0000 7f6ee0ff9640 1 -- 172.21.15.143:0/384017036 <== mon.0 v2:172.21.15.143:3300/0 5 ==== osd_map(49..49 src has 1..49) v4 ==== 5415+0+0 (secure 0 0 0) 0x7f6edc11efd0 con 0x7f6ee4143ac0 2023-11-26T07:21:45.414 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:21:45.412+0000 7f6ee37fe640 1 --2- 172.21.15.143:0/384017036 >> [v2:172.21.15.143:6832/2150557544,v1:172.21.15.143:6833/2150557544] conn(0x7f6ec4066620 0x7f6ec4068ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:21:45.414 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:21:45.412+0000 7f6ee37fe640 1 --2- 172.21.15.143:0/384017036 >> [v2:172.21.15.143:6832/2150557544,v1:172.21.15.143:6833/2150557544] conn(0x7f6ec4066620 0x7f6ec4068ae0 secure :-1 s=READY pgs=358 cs=0 l=1 rev1=1 crypto rx=0x7f6ed40011f0 tx=0x7f6ed4009040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:21:45.417 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:21:45.416+0000 7f6ee0ff9640 1 -- 172.21.15.143:0/384017036 <== mon.0 v2:172.21.15.143:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172649 (secure 0 0 0) 0x7f6edc0ec340 con 0x7f6ee4143ac0 2023-11-26T07:21:45.566 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:21:45.564+0000 7f6eea1df640 1 -- 172.21.15.143:0/384017036 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7f6ee4067680 con 0x7f6ee4143ac0 2023-11-26T07:21:46.230 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:21:46.228+0000 7f6ee0ff9640 1 -- 172.21.15.143:0/384017036 <== mon.0 v2:172.21.15.143:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_data' removed v50) v1 ==== 169+0+0 (secure 0 0 0) 0x7f6edc009450 con 0x7f6ee4143ac0 2023-11-26T07:21:46.230 INFO:teuthology.orchestra.run.smithi143.stderr:pool 'cephfs_data' removed 2023-11-26T07:21:46.232 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:21:46.228+0000 7f6eea1df640 1 -- 172.21.15.143:0/384017036 >> [v2:172.21.15.143:6832/2150557544,v1:172.21.15.143:6833/2150557544] conn(0x7f6ec4066620 msgr2=0x7f6ec4068ae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:21:46.232 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:21:46.228+0000 7f6eea1df640 1 --2- 172.21.15.143:0/384017036 >> [v2:172.21.15.143:6832/2150557544,v1:172.21.15.143:6833/2150557544] conn(0x7f6ec4066620 0x7f6ec4068ae0 secure :-1 s=READY pgs=358 cs=0 l=1 rev1=1 crypto rx=0x7f6ed40011f0 tx=0x7f6ed4009040 comp rx=0 tx=0).stop 2023-11-26T07:21:46.233 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:21:46.228+0000 7f6eea1df640 1 -- 172.21.15.143:0/384017036 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f6ee4143ac0 msgr2=0x7f6ee413c4a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:21:46.233 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:21:46.228+0000 7f6eea1df640 1 --2- 172.21.15.143:0/384017036 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f6ee4143ac0 0x7f6ee413c4a0 secure :-1 s=READY pgs=749 cs=0 l=1 rev1=1 crypto rx=0x7f6edc0b0040 tx=0x7f6ee40ce610 comp rx=0 tx=0).stop 2023-11-26T07:21:46.233 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:21:46.228+0000 7f6eea1df640 1 -- 172.21.15.143:0/384017036 shutdown_connections 2023-11-26T07:21:46.233 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:21:46.228+0000 7f6eea1df640 1 --2- 172.21.15.143:0/384017036 >> [v2:172.21.15.143:6832/2150557544,v1:172.21.15.143:6833/2150557544] conn(0x7f6ec4066620 0x7f6ec4068ae0 unknown :-1 s=CLOSED pgs=358 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:21:46.233 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:21:46.228+0000 7f6eea1df640 1 --2- 172.21.15.143:0/384017036 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f6ee4143ac0 0x7f6ee413c4a0 unknown :-1 s=CLOSED pgs=749 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:21:46.233 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:21:46.228+0000 7f6eea1df640 1 --2- 172.21.15.143:0/384017036 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f6ee40cf710 0x7f6ee40b0240 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:21:46.233 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:21:46.232+0000 7f6eea1df640 1 --2- 172.21.15.143:0/384017036 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f6ee40cf280 0x7f6ee413ac50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:21:46.233 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:21:46.232+0000 7f6eea1df640 1 -- 172.21.15.143:0/384017036 >> 172.21.15.143:0/384017036 conn(0x7f6ee406a890 msgr2=0x7f6ee413d770 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:21:46.233 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:21:46.232+0000 7f6eea1df640 1 -- 172.21.15.143:0/384017036 shutdown_connections 2023-11-26T07:21:46.233 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:21:46.232+0000 7f6eea1df640 1 -- 172.21.15.143:0/384017036 wait complete. 2023-11-26T07:21:46.258 DEBUG:teuthology.orchestra.run.smithi143:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config rm mon mon_allow_pool_delete 2023-11-26T07:21:46.403 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:21:46.400+0000 7ff6b4868640 1 Processor -- start 2023-11-26T07:21:46.403 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:21:46.400+0000 7ff6b4868640 1 -- start start 2023-11-26T07:21:46.405 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:21:46.400+0000 7ff6b4868640 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7ff6b0143ac0 0x7ff6b0070370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:21:46.405 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:21:46.400+0000 7ff6b4868640 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7ff6b00cf990 0x7ff6b00cd1b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:21:46.405 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:21:46.400+0000 7ff6b4868640 1 -- --> v1:172.21.15.143:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff6b00cc730 con 0x7ff6b00708b0 2023-11-26T07:21:46.405 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:21:46.400+0000 7ff6b4868640 1 -- --> v2:172.21.15.143:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff6b0143ea0 con 0x7ff6b00cf990 2023-11-26T07:21:46.405 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:21:46.400+0000 7ff6b4868640 1 -- --> v2:172.21.15.188:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff6b0070ee0 con 0x7ff6b0143ac0 2023-11-26T07:21:46.406 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:21:46.404+0000 7ff6aed9d640 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7ff6b00cf990 0x7ff6b00cd1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:21:46.406 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:21:46.404+0000 7ff6aed9d640 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7ff6b00cf990 0x7ff6b00cd1b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:49252/0 (socket says 172.21.15.143:49252) 2023-11-26T07:21:46.406 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:21:46.404+0000 7ff6aed9d640 1 -- 172.21.15.143:0/3966752634 learned_addr learned my addr 172.21.15.143:0/3966752634 (peer_addr_for_me v2:172.21.15.143:0/0) 2023-11-26T07:21:46.406 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:21:46.404+0000 7ff6ae59c640 1 --2- 172.21.15.143:0/3966752634 >> v2:172.21.15.188:3300/0 conn(0x7ff6b0143ac0 0x7ff6b0070370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:21:46.406 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:21:46.404+0000 7ff6aed9d640 1 -- 172.21.15.143:0/3966752634 >> v2:172.21.15.188:3300/0 conn(0x7ff6b0143ac0 msgr2=0x7ff6b0070370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-26T07:21:46.406 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:21:46.404+0000 7ff6aed9d640 1 --2- 172.21.15.143:0/3966752634 >> v2:172.21.15.188:3300/0 conn(0x7ff6b0143ac0 0x7ff6b0070370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:21:46.406 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:21:46.404+0000 7ff6aed9d640 1 -- 172.21.15.143:0/3966752634 >> v1:172.21.15.143:6789/0 conn(0x7ff6b00708b0 legacy=0x7ff6b00cf280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:21:46.406 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:21:46.404+0000 7ff6aed9d640 1 -- 172.21.15.143:0/3966752634 --> v2:172.21.15.143:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff6b00711c0 con 0x7ff6b00cf990 2023-11-26T07:21:46.406 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:21:46.404+0000 7ff6ae59c640 1 --2- 172.21.15.143:0/3966752634 >> v2:172.21.15.188:3300/0 conn(0x7ff6b0143ac0 0x7ff6b0070370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-26T07:21:46.407 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:21:46.404+0000 7ff6aed9d640 1 --2- 172.21.15.143:0/3966752634 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7ff6b00cf990 0x7ff6b00cd1b0 secure :-1 s=READY pgs=751 cs=0 l=1 rev1=1 crypto rx=0x7ff6b006dca0 tx=0x7ff6a801c9d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d1750f48f2ac2d71 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:21:46.408 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:21:46.404+0000 7ff6add9b640 1 -- 172.21.15.143:0/3966752634 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff6a80a57a0 con 0x7ff6b00cf990 2023-11-26T07:21:46.408 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:21:46.404+0000 7ff6add9b640 1 -- 172.21.15.143:0/3966752634 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff6a80a5900 con 0x7ff6b00cf990 2023-11-26T07:21:46.408 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:21:46.404+0000 7ff6add9b640 1 -- 172.21.15.143:0/3966752634 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff6a80a5c90 con 0x7ff6b00cf990 2023-11-26T07:21:46.409 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:21:46.408+0000 7ff6b4868640 1 -- 172.21.15.143:0/3966752634 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7ff6b00cf990 msgr2=0x7ff6b00cd1b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:21:46.409 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:21:46.408+0000 7ff6b4868640 1 --2- 172.21.15.143:0/3966752634 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7ff6b00cf990 0x7ff6b00cd1b0 secure :-1 s=READY pgs=751 cs=0 l=1 rev1=1 crypto rx=0x7ff6b006dca0 tx=0x7ff6a801c9d0 comp rx=0 tx=0).stop 2023-11-26T07:21:46.410 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:21:46.408+0000 7ff6b4868640 1 -- 172.21.15.143:0/3966752634 shutdown_connections 2023-11-26T07:21:46.410 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:21:46.408+0000 7ff6b4868640 1 --2- 172.21.15.143:0/3966752634 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7ff6b00cf990 0x7ff6b00cd1b0 unknown :-1 s=CLOSED pgs=751 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:21:46.410 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:21:46.408+0000 7ff6b4868640 1 --2- 172.21.15.143:0/3966752634 >> v2:172.21.15.188:3300/0 conn(0x7ff6b0143ac0 0x7ff6b0070370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:21:46.410 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:21:46.408+0000 7ff6b4868640 1 -- 172.21.15.143:0/3966752634 >> 172.21.15.143:0/3966752634 conn(0x7ff6b006a890 msgr2=0x7ff6b006aca0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:21:46.410 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:21:46.408+0000 7ff6b4868640 1 -- 172.21.15.143:0/3966752634 shutdown_connections 2023-11-26T07:21:46.410 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:21:46.408+0000 7ff6b4868640 1 -- 172.21.15.143:0/3966752634 wait complete. 2023-11-26T07:21:46.411 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:21:46.408+0000 7ff6b4868640 1 Processor -- start 2023-11-26T07:21:46.411 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:21:46.408+0000 7ff6b4868640 1 -- start start 2023-11-26T07:21:46.413 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:21:46.408+0000 7ff6b4868640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7ff6b00708b0 0x7ff6b013b0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:21:46.413 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:21:46.408+0000 7ff6b4868640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7ff6b00cf990 0x7ff6b00b13d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:21:46.413 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:21:46.408+0000 7ff6b4868640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7ff6b0143ac0 0x7ff6b013b880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:21:46.414 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:21:46.408+0000 7ff6b4868640 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff6b013bdc0 con 0x7ff6b00708b0 2023-11-26T07:21:46.414 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:21:46.408+0000 7ff6b4868640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff6b013abf0 con 0x7ff6b00cf990 2023-11-26T07:21:46.414 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:21:46.408+0000 7ff6b4868640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff6b013bfd0 con 0x7ff6b0143ac0 2023-11-26T07:21:46.414 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:21:46.408+0000 7ff6ae59c640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7ff6b00708b0 0x7ff6b013b0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:21:46.414 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:21:46.408+0000 7ff6ae59c640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7ff6b00708b0 0x7ff6b013b0d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:49254/0 (socket says 172.21.15.143:49254) 2023-11-26T07:21:46.414 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:21:46.408+0000 7ff6ae59c640 1 -- 172.21.15.143:0/1343696251 learned_addr learned my addr 172.21.15.143:0/1343696251 (peer_addr_for_me v2:172.21.15.143:0/0) 2023-11-26T07:21:46.414 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:21:46.408+0000 7ff6a5d9b640 1 --2- 172.21.15.143:0/1343696251 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7ff6b00cf990 0x7ff6b00b13d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:21:46.414 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:21:46.408+0000 7ff6aed9d640 1 --2- 172.21.15.143:0/1343696251 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7ff6b0143ac0 0x7ff6b013b880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:21:46.414 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:21:46.408+0000 7ff6ae59c640 1 -- 172.21.15.143:0/1343696251 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7ff6b0143ac0 msgr2=0x7ff6b013b880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:21:46.414 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:21:46.408+0000 7ff6ae59c640 1 --2- 172.21.15.143:0/1343696251 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7ff6b0143ac0 0x7ff6b013b880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:21:46.414 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:21:46.412+0000 7ff6ae59c640 1 -- 172.21.15.143:0/1343696251 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7ff6b00cf990 msgr2=0x7ff6b00b13d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:21:46.415 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:21:46.412+0000 7ff6ae59c640 1 --2- 172.21.15.143:0/1343696251 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7ff6b00cf990 0x7ff6b00b13d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:21:46.415 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:21:46.412+0000 7ff6ae59c640 1 -- 172.21.15.143:0/1343696251 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff6a8009b30 con 0x7ff6b00708b0 2023-11-26T07:21:46.415 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:21:46.412+0000 7ff6aed9d640 1 --2- 172.21.15.143:0/1343696251 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7ff6b0143ac0 0x7ff6b013b880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-26T07:21:46.415 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:21:46.412+0000 7ff6ae59c640 1 --2- 172.21.15.143:0/1343696251 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7ff6b00708b0 0x7ff6b013b0d0 secure :-1 s=READY pgs=752 cs=0 l=1 rev1=1 crypto rx=0x7ff6a000c7f0 tx=0x7ff6a000ccb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:21:46.415 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:21:46.412+0000 7ff6a77fe640 1 -- 172.21.15.143:0/1343696251 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff6a0007820 con 0x7ff6b00708b0 2023-11-26T07:21:46.415 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:21:46.412+0000 7ff6a77fe640 1 -- 172.21.15.143:0/1343696251 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff6a0007980 con 0x7ff6b00708b0 2023-11-26T07:21:46.415 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:21:46.412+0000 7ff6a77fe640 1 -- 172.21.15.143:0/1343696251 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff6a0007d30 con 0x7ff6b00708b0 2023-11-26T07:21:46.415 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:21:46.412+0000 7ff6b4868640 1 -- 172.21.15.143:0/1343696251 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff6b01642a0 con 0x7ff6b00708b0 2023-11-26T07:21:46.415 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:21:46.412+0000 7ff6b4868640 1 -- 172.21.15.143:0/1343696251 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff6b0164860 con 0x7ff6b00708b0 2023-11-26T07:21:46.416 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:21:46.412+0000 7ff6b4868640 1 -- 172.21.15.143:0/1343696251 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff6b013adf0 con 0x7ff6b00708b0 2023-11-26T07:21:46.416 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:21:46.412+0000 7ff6a77fe640 1 -- 172.21.15.143:0/1343696251 <== mon.0 v2:172.21.15.143:3300/0 4 ==== mgrmap(e 9) v1 ==== 86005+0+0 (secure 0 0 0) 0x7ff6a00113f0 con 0x7ff6b00708b0 2023-11-26T07:21:46.417 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:21:46.412+0000 7ff6a77fe640 1 --2- 172.21.15.143:0/1343696251 >> [v2:172.21.15.143:6832/2150557544,v1:172.21.15.143:6833/2150557544] conn(0x7ff690066620 0x7ff690068ae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:21:46.417 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:21:46.412+0000 7ff6a77fe640 1 -- 172.21.15.143:0/1343696251 <== mon.0 v2:172.21.15.143:3300/0 5 ==== osd_map(50..50 src has 1..50) v4 ==== 4943+0+0 (secure 0 0 0) 0x7ff6a00843b0 con 0x7ff6b00708b0 2023-11-26T07:21:46.417 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:21:46.412+0000 7ff6a5d9b640 1 --2- 172.21.15.143:0/1343696251 >> [v2:172.21.15.143:6832/2150557544,v1:172.21.15.143:6833/2150557544] conn(0x7ff690066620 0x7ff690068ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:21:46.417 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:21:46.416+0000 7ff6a5d9b640 1 --2- 172.21.15.143:0/1343696251 >> [v2:172.21.15.143:6832/2150557544,v1:172.21.15.143:6833/2150557544] conn(0x7ff690066620 0x7ff690068ae0 secure :-1 s=READY pgs=359 cs=0 l=1 rev1=1 crypto rx=0x7ff6b015a190 tx=0x7ff698009040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:21:46.419 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:21:46.416+0000 7ff6a77fe640 1 -- 172.21.15.143:0/1343696251 <== mon.0 v2:172.21.15.143:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172649 (secure 0 0 0) 0x7ff6b013adf0 con 0x7ff6b00708b0 2023-11-26T07:21:46.569 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:21:46.568+0000 7ff6b4868640 1 -- 172.21.15.143:0/1343696251 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "config rm", "who": "mon", "name": "mon_allow_pool_delete"} v 0) v1 -- 0x7ff6b0067680 con 0x7ff6b00708b0 2023-11-26T07:21:46.573 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:21:46.572+0000 7ff6a77fe640 1 -- 172.21.15.143:0/1343696251 <== mon.0 v2:172.21.15.143:3300/0 7 ==== mon_command_ack([{"prefix": "config rm", "who": "mon", "name": "mon_allow_pool_delete"}]=0 v13) v1 ==== 104+0+0 (secure 0 0 0) 0x7ff6a0051700 con 0x7ff6b00708b0 2023-11-26T07:21:46.576 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:21:46.572+0000 7ff6b4868640 1 -- 172.21.15.143:0/1343696251 >> [v2:172.21.15.143:6832/2150557544,v1:172.21.15.143:6833/2150557544] conn(0x7ff690066620 msgr2=0x7ff690068ae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:21:46.576 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:21:46.572+0000 7ff6b4868640 1 --2- 172.21.15.143:0/1343696251 >> [v2:172.21.15.143:6832/2150557544,v1:172.21.15.143:6833/2150557544] conn(0x7ff690066620 0x7ff690068ae0 secure :-1 s=READY pgs=359 cs=0 l=1 rev1=1 crypto rx=0x7ff6b015a190 tx=0x7ff698009040 comp rx=0 tx=0).stop 2023-11-26T07:21:46.576 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:21:46.572+0000 7ff6b4868640 1 -- 172.21.15.143:0/1343696251 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7ff6b00708b0 msgr2=0x7ff6b013b0d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:21:46.576 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:21:46.572+0000 7ff6b4868640 1 --2- 172.21.15.143:0/1343696251 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7ff6b00708b0 0x7ff6b013b0d0 secure :-1 s=READY pgs=752 cs=0 l=1 rev1=1 crypto rx=0x7ff6a000c7f0 tx=0x7ff6a000ccb0 comp rx=0 tx=0).stop 2023-11-26T07:21:46.576 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:21:46.572+0000 7ff6b4868640 1 -- 172.21.15.143:0/1343696251 shutdown_connections 2023-11-26T07:21:46.576 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:21:46.572+0000 7ff6b4868640 1 --2- 172.21.15.143:0/1343696251 >> [v2:172.21.15.143:6832/2150557544,v1:172.21.15.143:6833/2150557544] conn(0x7ff690066620 0x7ff690068ae0 unknown :-1 s=CLOSED pgs=359 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:21:46.576 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:21:46.572+0000 7ff6b4868640 1 --2- 172.21.15.143:0/1343696251 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7ff6b0143ac0 0x7ff6b013b880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:21:46.576 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:21:46.572+0000 7ff6b4868640 1 --2- 172.21.15.143:0/1343696251 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7ff6b00cf990 0x7ff6b00b13d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:21:46.576 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:21:46.572+0000 7ff6b4868640 1 --2- 172.21.15.143:0/1343696251 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7ff6b00708b0 0x7ff6b013b0d0 unknown :-1 s=CLOSED pgs=752 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:21:46.576 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:21:46.572+0000 7ff6b4868640 1 -- 172.21.15.143:0/1343696251 >> 172.21.15.143:0/1343696251 conn(0x7ff6b006a890 msgr2=0x7ff6b0071f20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:21:46.577 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:21:46.572+0000 7ff6b4868640 1 -- 172.21.15.143:0/1343696251 shutdown_connections 2023-11-26T07:21:46.577 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:21:46.572+0000 7ff6b4868640 1 -- 172.21.15.143:0/1343696251 wait complete. 2023-11-26T07:21:46.594 DEBUG:teuthology.orchestra.run.smithi143:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Ended test tasks.cephfs.test_volumes.TestMisc.test_connection_expiration' 2023-11-26T07:21:46.733 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:21:46.732+0000 7f94cd58d640 1 Processor -- start 2023-11-26T07:21:46.733 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:21:46.732+0000 7f94cd58d640 1 -- start start 2023-11-26T07:21:46.735 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:21:46.732+0000 7f94cd58d640 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7f94c80d0150 0x7f94c80c6b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:21:46.735 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:21:46.732+0000 7f94cd58d640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f94c80b45d0 0x7f94c80c70c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:21:46.735 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:21:46.732+0000 7f94cd58d640 1 -- --> v1:172.21.15.188:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f94c80cefa0 con 0x7f94c80b3610 2023-11-26T07:21:46.735 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:21:46.732+0000 7f94cd58d640 1 -- --> v2:172.21.15.143:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f94c80c9fa0 con 0x7f94c80d0150 2023-11-26T07:21:46.735 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:21:46.732+0000 7f94cd58d640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f94c80b3ef0 con 0x7f94c80b45d0 2023-11-26T07:21:46.735 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:21:46.732+0000 7f94be7fc640 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7f94c80d0150 0x7f94c80c6b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:21:46.735 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:21:46.732+0000 7f94be7fc640 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7f94c80d0150 0x7f94c80c6b80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:49262/0 (socket says 172.21.15.143:49262) 2023-11-26T07:21:46.735 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:21:46.732+0000 7f94be7fc640 1 -- 172.21.15.143:0/1953686860 learned_addr learned my addr 172.21.15.143:0/1953686860 (peer_addr_for_me v2:172.21.15.143:0/0) 2023-11-26T07:21:46.736 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:21:46.732+0000 7f94c77fe640 1 --2- 172.21.15.143:0/1953686860 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f94c80b45d0 0x7f94c80c70c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:21:46.736 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:21:46.732+0000 7f94be7fc640 1 -- 172.21.15.143:0/1953686860 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f94c80b45d0 msgr2=0x7f94c80c70c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-26T07:21:46.736 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:21:46.732+0000 7f94be7fc640 1 --2- 172.21.15.143:0/1953686860 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f94c80b45d0 0x7f94c80c70c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:21:46.736 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:21:46.732+0000 7f94be7fc640 1 -- 172.21.15.143:0/1953686860 >> v1:172.21.15.188:6789/0 conn(0x7f94c80b3610 legacy=0x7f94c80cfa40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:21:46.736 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:21:46.732+0000 7f94be7fc640 1 -- 172.21.15.143:0/1953686860 --> v2:172.21.15.143:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f94c80d0f90 con 0x7f94c80d0150 2023-11-26T07:21:46.738 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:21:46.736+0000 7f94be7fc640 1 --2- 172.21.15.143:0/1953686860 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f94c80d0150 0x7f94c80c6b80 secure :-1 s=READY pgs=753 cs=0 l=1 rev1=1 crypto rx=0x7f94b0009870 tx=0x7f94b001ed30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=64c4285d015cf17d server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:21:46.738 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:21:46.736+0000 7f94c67fc640 1 -- 172.21.15.143:0/1953686860 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f94b00a8a30 con 0x7f94c80d0150 2023-11-26T07:21:46.738 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:21:46.736+0000 7f94c67fc640 1 -- 172.21.15.143:0/1953686860 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f94b00a8b90 con 0x7f94c80d0150 2023-11-26T07:21:46.738 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:21:46.736+0000 7f94c67fc640 1 -- 172.21.15.143:0/1953686860 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f94b00b0440 con 0x7f94c80d0150 2023-11-26T07:21:46.739 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:21:46.736+0000 7f94cd58d640 1 -- 172.21.15.143:0/1953686860 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f94c80d0150 msgr2=0x7f94c80c6b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:21:46.739 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:21:46.736+0000 7f94cd58d640 1 --2- 172.21.15.143:0/1953686860 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f94c80d0150 0x7f94c80c6b80 secure :-1 s=READY pgs=753 cs=0 l=1 rev1=1 crypto rx=0x7f94b0009870 tx=0x7f94b001ed30 comp rx=0 tx=0).stop 2023-11-26T07:21:46.739 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:21:46.736+0000 7f94cd58d640 1 -- 172.21.15.143:0/1953686860 shutdown_connections 2023-11-26T07:21:46.739 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:21:46.736+0000 7f94cd58d640 1 --2- 172.21.15.143:0/1953686860 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f94c80d0150 0x7f94c80c6b80 unknown :-1 s=CLOSED pgs=753 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:21:46.739 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:21:46.736+0000 7f94cd58d640 1 --2- 172.21.15.143:0/1953686860 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f94c80b45d0 0x7f94c80c70c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:21:46.740 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:21:46.736+0000 7f94cd58d640 1 -- 172.21.15.143:0/1953686860 >> 172.21.15.143:0/1953686860 conn(0x7f94c80d3030 msgr2=0x7f94c80d3440 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:21:46.740 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:21:46.736+0000 7f94cd58d640 1 -- 172.21.15.143:0/1953686860 shutdown_connections 2023-11-26T07:21:46.740 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:21:46.736+0000 7f94cd58d640 1 -- 172.21.15.143:0/1953686860 wait complete. 2023-11-26T07:21:46.740 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:21:46.740+0000 7f94cd58d640 1 Processor -- start 2023-11-26T07:21:46.741 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:21:46.740+0000 7f94cd58d640 1 -- start start 2023-11-26T07:21:46.743 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:21:46.740+0000 7f94cd58d640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f94c80b3610 0x7f94c81f2830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:21:46.743 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:21:46.740+0000 7f94cd58d640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f94c80b45d0 0x7f94c815b090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:21:46.743 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:21:46.740+0000 7f94cd58d640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f94c80d0150 0x7f94c815cca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:21:46.743 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:21:46.740+0000 7f94cd58d640 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f94c80b7df0 con 0x7f94c80b45d0 2023-11-26T07:21:46.743 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:21:46.740+0000 7f94cd58d640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f94c80b8090 con 0x7f94c80d0150 2023-11-26T07:21:46.743 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:21:46.740+0000 7f94cd58d640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f94c80b8370 con 0x7f94c80b3610 2023-11-26T07:21:46.743 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:21:46.740+0000 7f94be7fc640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f94c80b45d0 0x7f94c815b090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:21:46.743 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:21:46.740+0000 7f94be7fc640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f94c80b45d0 0x7f94c815b090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:49274/0 (socket says 172.21.15.143:49274) 2023-11-26T07:21:46.743 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:21:46.740+0000 7f94be7fc640 1 -- 172.21.15.143:0/839345357 learned_addr learned my addr 172.21.15.143:0/839345357 (peer_addr_for_me v2:172.21.15.143:0/0) 2023-11-26T07:21:46.743 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:21:46.740+0000 7f94c6ffd640 1 --2- 172.21.15.143:0/839345357 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f94c80b3610 0x7f94c81f2830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:21:46.744 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:21:46.740+0000 7f94c77fe640 1 --2- 172.21.15.143:0/839345357 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f94c80d0150 0x7f94c815cca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:21:46.744 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:21:46.740+0000 7f94be7fc640 1 -- 172.21.15.143:0/839345357 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f94c80b3610 msgr2=0x7f94c81f2830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:21:46.744 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:21:46.740+0000 7f94be7fc640 1 --2- 172.21.15.143:0/839345357 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f94c80b3610 0x7f94c81f2830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:21:46.744 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:21:46.740+0000 7f94be7fc640 1 -- 172.21.15.143:0/839345357 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f94c80d0150 msgr2=0x7f94c815cca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:21:46.744 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:21:46.740+0000 7f94be7fc640 1 --2- 172.21.15.143:0/839345357 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f94c80d0150 0x7f94c815cca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:21:46.744 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:21:46.740+0000 7f94be7fc640 1 -- 172.21.15.143:0/839345357 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f94b00094d0 con 0x7f94c80b45d0 2023-11-26T07:21:46.744 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:21:46.740+0000 7f94be7fc640 1 --2- 172.21.15.143:0/839345357 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f94c80b45d0 0x7f94c815b090 secure :-1 s=READY pgs=754 cs=0 l=1 rev1=1 crypto rx=0x7f94b00b3040 tx=0x7f94b001bad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:21:46.745 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:21:46.740+0000 7f94bffff640 1 -- 172.21.15.143:0/839345357 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f94b00be070 con 0x7f94c80b45d0 2023-11-26T07:21:46.745 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:21:46.740+0000 7f94bffff640 1 -- 172.21.15.143:0/839345357 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f94b00ba400 con 0x7f94c80b45d0 2023-11-26T07:21:46.745 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:21:46.740+0000 7f94bffff640 1 -- 172.21.15.143:0/839345357 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f94b00ba7d0 con 0x7f94c80b45d0 2023-11-26T07:21:46.745 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:21:46.740+0000 7f94cd58d640 1 -- 172.21.15.143:0/839345357 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f94c80b8650 con 0x7f94c80b45d0 2023-11-26T07:21:46.745 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:21:46.740+0000 7f94cd58d640 1 -- 172.21.15.143:0/839345357 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f94c81583e0 con 0x7f94c80b45d0 2023-11-26T07:21:46.745 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:21:46.740+0000 7f94cd58d640 1 -- 172.21.15.143:0/839345357 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f94c8067500 con 0x7f94c80b45d0 2023-11-26T07:21:46.746 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:21:46.744+0000 7f94bffff640 1 -- 172.21.15.143:0/839345357 <== mon.0 v2:172.21.15.143:3300/0 4 ==== mgrmap(e 9) v1 ==== 86005+0+0 (secure 0 0 0) 0x7f94b00c3410 con 0x7f94c80b45d0 2023-11-26T07:21:46.747 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:21:46.744+0000 7f94bffff640 1 --2- 172.21.15.143:0/839345357 >> [v2:172.21.15.143:6832/2150557544,v1:172.21.15.143:6833/2150557544] conn(0x7f94a8066620 0x7f94a8068ae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:21:46.747 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:21:46.744+0000 7f94bffff640 1 -- 172.21.15.143:0/839345357 <== mon.0 v2:172.21.15.143:3300/0 5 ==== osd_map(50..50 src has 1..50) v4 ==== 4943+0+0 (secure 0 0 0) 0x7f94b01219b0 con 0x7f94c80b45d0 2023-11-26T07:21:46.747 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:21:46.744+0000 7f94c6ffd640 1 --2- 172.21.15.143:0/839345357 >> [v2:172.21.15.143:6832/2150557544,v1:172.21.15.143:6833/2150557544] conn(0x7f94a8066620 0x7f94a8068ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:21:46.747 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:21:46.744+0000 7f94c6ffd640 1 --2- 172.21.15.143:0/839345357 >> [v2:172.21.15.143:6832/2150557544,v1:172.21.15.143:6833/2150557544] conn(0x7f94a8066620 0x7f94a8068ae0 secure :-1 s=READY pgs=360 cs=0 l=1 rev1=1 crypto rx=0x7f94b8000f70 tx=0x7f94b8009040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:21:46.750 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:21:46.748+0000 7f94bffff640 1 -- 172.21.15.143:0/839345357 <== mon.0 v2:172.21.15.143:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172649 (secure 0 0 0) 0x7f94b00eef40 con 0x7f94c80b45d0 2023-11-26T07:21:46.952 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:21:46.948+0000 7f94cd58d640 1 -- 172.21.15.143:0/839345357 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_volumes.TestMisc.test_connection_expiration"]} v 0) v1 -- 0x7f94c80b8b00 con 0x7f94c80b45d0 2023-11-26T07:21:47.232 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:21:47.228+0000 7f94bffff640 1 -- 172.21.15.143:0/839345357 <== mon.0 v2:172.21.15.143:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_volumes.TestMisc.test_connection_expiration"]}]=0 v203) v1 ==== 140+0+0 (secure 0 0 0) 0x7f94b00f2560 con 0x7f94c80b45d0 2023-11-26T07:21:47.235 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:21:47.232+0000 7f94cd58d640 1 -- 172.21.15.143:0/839345357 >> [v2:172.21.15.143:6832/2150557544,v1:172.21.15.143:6833/2150557544] conn(0x7f94a8066620 msgr2=0x7f94a8068ae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:21:47.235 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:21:47.232+0000 7f94cd58d640 1 --2- 172.21.15.143:0/839345357 >> [v2:172.21.15.143:6832/2150557544,v1:172.21.15.143:6833/2150557544] conn(0x7f94a8066620 0x7f94a8068ae0 secure :-1 s=READY pgs=360 cs=0 l=1 rev1=1 crypto rx=0x7f94b8000f70 tx=0x7f94b8009040 comp rx=0 tx=0).stop 2023-11-26T07:21:47.235 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:21:47.232+0000 7f94cd58d640 1 -- 172.21.15.143:0/839345357 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f94c80b45d0 msgr2=0x7f94c815b090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:21:47.235 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:21:47.232+0000 7f94cd58d640 1 --2- 172.21.15.143:0/839345357 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f94c80b45d0 0x7f94c815b090 secure :-1 s=READY pgs=754 cs=0 l=1 rev1=1 crypto rx=0x7f94b00b3040 tx=0x7f94b001bad0 comp rx=0 tx=0).stop 2023-11-26T07:21:47.235 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:21:47.232+0000 7f94cd58d640 1 -- 172.21.15.143:0/839345357 shutdown_connections 2023-11-26T07:21:47.235 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:21:47.232+0000 7f94cd58d640 1 --2- 172.21.15.143:0/839345357 >> [v2:172.21.15.143:6832/2150557544,v1:172.21.15.143:6833/2150557544] conn(0x7f94a8066620 0x7f94a8068ae0 unknown :-1 s=CLOSED pgs=360 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:21:47.235 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:21:47.232+0000 7f94cd58d640 1 --2- 172.21.15.143:0/839345357 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f94c80d0150 0x7f94c815cca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:21:47.235 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:21:47.232+0000 7f94cd58d640 1 --2- 172.21.15.143:0/839345357 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f94c80b45d0 0x7f94c815b090 unknown :-1 s=CLOSED pgs=754 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:21:47.235 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:21:47.232+0000 7f94cd58d640 1 --2- 172.21.15.143:0/839345357 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f94c80b3610 0x7f94c81f2830 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:21:47.235 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:21:47.232+0000 7f94cd58d640 1 -- 172.21.15.143:0/839345357 >> 172.21.15.143:0/839345357 conn(0x7f94c80d3030 msgr2=0x7f94c806aed0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:21:47.236 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:21:47.232+0000 7f94cd58d640 1 -- 172.21.15.143:0/839345357 shutdown_connections 2023-11-26T07:21:47.236 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:21:47.232+0000 7f94cd58d640 1 -- 172.21.15.143:0/839345357 wait complete. 2023-11-26T07:21:47.254 INFO:tasks.cephfs_test_runner:test_connection_expiration (tasks.cephfs.test_volumes.TestMisc) ... ok 2023-11-26T07:21:47.254 INFO:tasks.cephfs_test_runner:Starting test: test_malicious_metafile_on_legacy_to_v1_upgrade (tasks.cephfs.test_volumes.TestMisc) Validate handcrafted .meta file on legacy subvol root doesn't break the system 2023-11-26T07:21:47.254 INFO:tasks.cephfs_test_runner:test_malicious_metafile_on_legacy_to_v1_upgrade (tasks.cephfs.test_volumes.TestMisc) 2023-11-26T07:21:47.255 DEBUG:teuthology.orchestra.run.smithi143:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-11-26T07:21:47.392 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:21:47.388+0000 7f99c4b57640 1 Processor -- start 2023-11-26T07:21:47.392 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:21:47.388+0000 7f99c4b57640 1 -- start start 2023-11-26T07:21:47.394 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:21:47.392+0000 7f99c4b57640 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7f99c0143ac0 0x7f99c0070370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:21:47.394 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:21:47.392+0000 7f99c4b57640 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7f99c00cf990 0x7f99c00cd1b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:21:47.394 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:21:47.392+0000 7f99c4b57640 1 -- --> v1:172.21.15.188:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f99c00cc730 con 0x7f99c00708b0 2023-11-26T07:21:47.394 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:21:47.392+0000 7f99c4b57640 1 -- --> v2:172.21.15.143:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f99c0143ea0 con 0x7f99c0143ac0 2023-11-26T07:21:47.394 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:21:47.392+0000 7f99c4b57640 1 -- --> v2:172.21.15.188:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f99c0070ee0 con 0x7f99c00cf990 2023-11-26T07:21:47.394 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:21:47.392+0000 7f99be59c640 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7f99c0143ac0 0x7f99c0070370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:21:47.395 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:21:47.392+0000 7f99be59c640 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7f99c0143ac0 0x7f99c0070370 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:49282/0 (socket says 172.21.15.143:49282) 2023-11-26T07:21:47.395 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:21:47.392+0000 7f99be59c640 1 -- 172.21.15.143:0/2370985263 learned_addr learned my addr 172.21.15.143:0/2370985263 (peer_addr_for_me v2:172.21.15.143:0/0) 2023-11-26T07:21:47.395 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:21:47.392+0000 7f99bed9d640 1 --2- 172.21.15.143:0/2370985263 >> v2:172.21.15.188:3300/0 conn(0x7f99c00cf990 0x7f99c00cd1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:21:47.395 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:21:47.392+0000 7f99be59c640 1 -- 172.21.15.143:0/2370985263 >> v2:172.21.15.188:3300/0 conn(0x7f99c00cf990 msgr2=0x7f99c00cd1b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-26T07:21:47.395 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:21:47.392+0000 7f99be59c640 1 --2- 172.21.15.143:0/2370985263 >> v2:172.21.15.188:3300/0 conn(0x7f99c00cf990 0x7f99c00cd1b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:21:47.395 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:21:47.392+0000 7f99be59c640 1 -- 172.21.15.143:0/2370985263 >> v1:172.21.15.188:6789/0 conn(0x7f99c00708b0 legacy=0x7f99c00cf280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-26T07:21:47.395 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:21:47.392+0000 7f99be59c640 1 -- 172.21.15.143:0/2370985263 --> v2:172.21.15.143:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f99c00711c0 con 0x7f99c0143ac0 2023-11-26T07:21:47.397 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:21:47.396+0000 7f99be59c640 1 --2- 172.21.15.143:0/2370985263 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f99c0143ac0 0x7f99c0070370 secure :-1 s=READY pgs=755 cs=0 l=1 rev1=1 crypto rx=0x7f99c006e570 tx=0x7f99b001eb90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9880b4bcf1b35d9b server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:21:47.397 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:21:47.396+0000 7f99bdd9b640 1 -- 172.21.15.143:0/2370985263 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f99b00a7c60 con 0x7f99c0143ac0 2023-11-26T07:21:47.397 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:21:47.396+0000 7f99bdd9b640 1 -- 172.21.15.143:0/2370985263 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f99b00a7dc0 con 0x7f99c0143ac0 2023-11-26T07:21:47.397 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:21:47.396+0000 7f99bdd9b640 1 -- 172.21.15.143:0/2370985263 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f99b00af660 con 0x7f99c0143ac0 2023-11-26T07:21:47.398 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:21:47.396+0000 7f99c4b57640 1 -- 172.21.15.143:0/2370985263 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f99c0143ac0 msgr2=0x7f99c0070370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:21:47.398 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:21:47.396+0000 7f99c4b57640 1 --2- 172.21.15.143:0/2370985263 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f99c0143ac0 0x7f99c0070370 secure :-1 s=READY pgs=755 cs=0 l=1 rev1=1 crypto rx=0x7f99c006e570 tx=0x7f99b001eb90 comp rx=0 tx=0).stop 2023-11-26T07:21:47.398 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:21:47.396+0000 7f99c4b57640 1 -- 172.21.15.143:0/2370985263 shutdown_connections 2023-11-26T07:21:47.398 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:21:47.396+0000 7f99c4b57640 1 --2- 172.21.15.143:0/2370985263 >> v2:172.21.15.188:3300/0 conn(0x7f99c00cf990 0x7f99c00cd1b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:21:47.398 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:21:47.396+0000 7f99c4b57640 1 --2- 172.21.15.143:0/2370985263 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f99c0143ac0 0x7f99c0070370 unknown :-1 s=CLOSED pgs=755 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:21:47.399 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:21:47.396+0000 7f99c4b57640 1 -- 172.21.15.143:0/2370985263 >> 172.21.15.143:0/2370985263 conn(0x7f99c006a890 msgr2=0x7f99c006aca0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:21:47.399 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:21:47.396+0000 7f99c4b57640 1 -- 172.21.15.143:0/2370985263 shutdown_connections 2023-11-26T07:21:47.399 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:21:47.396+0000 7f99c4b57640 1 -- 172.21.15.143:0/2370985263 wait complete. 2023-11-26T07:21:47.400 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:21:47.396+0000 7f99c4b57640 1 Processor -- start 2023-11-26T07:21:47.400 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:21:47.396+0000 7f99c4b57640 1 -- start start 2023-11-26T07:21:47.402 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:21:47.400+0000 7f99c4b57640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f99c00708b0 0x7f99c0137180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:21:47.402 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:21:47.400+0000 7f99c4b57640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f99c00cf990 0x7f99c013ee50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:21:47.403 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:21:47.400+0000 7f99c4b57640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f99c0143ac0 0x7f99c0140a50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:21:47.403 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:21:47.400+0000 7f99c4b57640 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f99c0157a90 con 0x7f99c00cf990 2023-11-26T07:21:47.403 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:21:47.400+0000 7f99c4b57640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f99c013cbe0 con 0x7f99c0143ac0 2023-11-26T07:21:47.403 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:21:47.400+0000 7f99c4b57640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f99c013cec0 con 0x7f99c00708b0 2023-11-26T07:21:47.403 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:21:47.400+0000 7f99b5d9b640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f99c00cf990 0x7f99c013ee50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:21:47.403 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:21:47.400+0000 7f99b5d9b640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f99c00cf990 0x7f99c013ee50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:49292/0 (socket says 172.21.15.143:49292) 2023-11-26T07:21:47.403 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:21:47.400+0000 7f99b5d9b640 1 -- 172.21.15.143:0/504037534 learned_addr learned my addr 172.21.15.143:0/504037534 (peer_addr_for_me v2:172.21.15.143:0/0) 2023-11-26T07:21:47.403 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:21:47.400+0000 7f99be59c640 1 --2- 172.21.15.143:0/504037534 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f99c00708b0 0x7f99c0137180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:21:47.403 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:21:47.400+0000 7f99bed9d640 1 --2- 172.21.15.143:0/504037534 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f99c0143ac0 0x7f99c0140a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:21:47.403 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:21:47.400+0000 7f99b5d9b640 1 -- 172.21.15.143:0/504037534 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f99c00708b0 msgr2=0x7f99c0137180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:21:47.403 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:21:47.400+0000 7f99b5d9b640 1 --2- 172.21.15.143:0/504037534 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f99c00708b0 0x7f99c0137180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:21:47.403 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:21:47.400+0000 7f99b5d9b640 1 -- 172.21.15.143:0/504037534 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f99c0143ac0 msgr2=0x7f99c0140a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:21:47.404 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:21:47.400+0000 7f99b5d9b640 1 --2- 172.21.15.143:0/504037534 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f99c0143ac0 0x7f99c0140a50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:21:47.404 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:21:47.400+0000 7f99b5d9b640 1 -- 172.21.15.143:0/504037534 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f99b000b770 con 0x7f99c00cf990 2023-11-26T07:21:47.404 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:21:47.400+0000 7f99b5d9b640 1 --2- 172.21.15.143:0/504037534 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f99c00cf990 0x7f99c013ee50 secure :-1 s=READY pgs=756 cs=0 l=1 rev1=1 crypto rx=0x7f99a800c7c0 tx=0x7f99a800cc80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:21:47.404 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:21:47.400+0000 7f99b77fe640 1 -- 172.21.15.143:0/504037534 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f99a800aa50 con 0x7f99c00cf990 2023-11-26T07:21:47.404 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:21:47.400+0000 7f99b77fe640 1 -- 172.21.15.143:0/504037534 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f99a800abb0 con 0x7f99c00cf990 2023-11-26T07:21:47.404 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:21:47.400+0000 7f99c4b57640 1 -- 172.21.15.143:0/504037534 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f99c013d1a0 con 0x7f99c00cf990 2023-11-26T07:21:47.404 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:21:47.400+0000 7f99b77fe640 1 -- 172.21.15.143:0/504037534 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f99a8012420 con 0x7f99c00cf990 2023-11-26T07:21:47.404 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:21:47.400+0000 7f99c4b57640 1 -- 172.21.15.143:0/504037534 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f99c013f560 con 0x7f99c00cf990 2023-11-26T07:21:47.405 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:21:47.400+0000 7f99c4b57640 1 -- 172.21.15.143:0/504037534 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f99c013d3e0 con 0x7f99c00cf990 2023-11-26T07:21:47.405 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:21:47.400+0000 7f99b77fe640 1 -- 172.21.15.143:0/504037534 <== mon.0 v2:172.21.15.143:3300/0 4 ==== mgrmap(e 9) v1 ==== 86005+0+0 (secure 0 0 0) 0x7f99a8012580 con 0x7f99c00cf990 2023-11-26T07:21:47.405 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:21:47.404+0000 7f99b77fe640 1 --2- 172.21.15.143:0/504037534 >> [v2:172.21.15.143:6832/2150557544,v1:172.21.15.143:6833/2150557544] conn(0x7f99a0066650 0x7f99a0068b10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:21:47.405 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:21:47.404+0000 7f99b77fe640 1 -- 172.21.15.143:0/504037534 <== mon.0 v2:172.21.15.143:3300/0 5 ==== osd_map(50..50 src has 1..50) v4 ==== 4943+0+0 (secure 0 0 0) 0x7f99a80840e0 con 0x7f99c00cf990 2023-11-26T07:21:47.405 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:21:47.404+0000 7f99be59c640 1 --2- 172.21.15.143:0/504037534 >> [v2:172.21.15.143:6832/2150557544,v1:172.21.15.143:6833/2150557544] conn(0x7f99a0066650 0x7f99a0068b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:21:47.406 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:21:47.404+0000 7f99be59c640 1 --2- 172.21.15.143:0/504037534 >> [v2:172.21.15.143:6832/2150557544,v1:172.21.15.143:6833/2150557544] conn(0x7f99a0066650 0x7f99a0068b10 secure :-1 s=READY pgs=361 cs=0 l=1 rev1=1 crypto rx=0x7f99b00b1000 tx=0x7f99b00a9070 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:21:47.409 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:21:47.408+0000 7f99b77fe640 1 -- 172.21.15.143:0/504037534 <== mon.0 v2:172.21.15.143:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172649 (secure 0 0 0) 0x7f99c013d3e0 con 0x7f99c00cf990 2023-11-26T07:21:47.571 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:21:47.568+0000 7f99c4b57640 1 -- 172.21.15.143:0/504037534 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f99c0157bd0 con 0x7f99c00cf990 2023-11-26T07:21:47.572 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:21:47.568+0000 7f99b77fe640 1 -- 172.21.15.143:0/504037534 <== mon.0 v2:172.21.15.143:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v50) v1 ==== 74+0+13024 (secure 0 0 0) 0x7f99c0157bd0 con 0x7f99c00cf990 2023-11-26T07:21:47.572 INFO:teuthology.orchestra.run.smithi143.stdout: 2023-11-26T07:21:47.572 INFO:teuthology.orchestra.run.smithi143.stdout:{"epoch":50,"fsid":"f5033060-2469-4a07-95fe-2b76f5404bc1","created":"2023-11-26T07:17:28.613663+0000","modified":"2023-11-26T07:21:46.228057+0000","last_up_change":"2023-11-26T07:17:41.157813+0000","last_in_change":"2023-11-26T07:17:31.181658+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":7,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-11-26T07:17:41.758597+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"ebbee2c6-023e-44b9-9f1d-6f7bfd309730","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":43,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6817","nonce":639807369},{"type":"v1","addr":"172.21.15.143:6819","nonce":639807369}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6821","nonce":639807369},{"type":"v1","addr":"172.21.15.143:6823","nonce":639807369}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6830","nonce":639807369},{"type":"v1","addr":"172.21.15.143:6831","nonce":639807369}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6825","nonce":639807369},{"type":"v1","addr":"172.21.15.143:6828","nonce":639807369}]},"public_addr":"172.21.15.143:6819/639807369","cluster_addr":"172.21.15.143:6823/639807369","heartbeat_back_addr":"172.21.15.143:6831/639807369","heartbeat_front_addr":"172.21.15.143:6828/639807369","state":["exists","up"]},{"osd":1,"uuid":"fec4b8be-7122-49e9-9a19-5850a0700001","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":43,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6816","nonce":1144942530},{"type":"v1","addr":"172.21.15.143:6818","nonce":1144942530}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6820","nonce":1144942530},{"type":"v1","addr":"172.21.15.143:6822","nonce":1144942530}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6827","nonce":1144942530},{"type":"v1","addr":"172.21.15.143:6829","nonce":1144942530}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6824","nonce":1144942530},{"type":"v1","addr":"172.21.15.143:6826","nonce":1144942530}]},"public_addr":"172.21.15.143:6818/1144942530","cluster_addr":"172.21.15.143:6822/1144942530","heartbeat_back_addr":"172.21.15.143:6829/1144942530","heartbeat_front_addr":"172.21.15.143:6826/1144942530","state":["exists","up"]},{"osd":2,"uuid":"2888af5b-f933-46ca-8f95-6dd494ff43f6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":46,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6808","nonce":3909095186},{"type":"v1","addr":"172.21.15.143:6809","nonce":3909095186}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6810","nonce":3909095186},{"type":"v1","addr":"172.21.15.143:6811","nonce":3909095186}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6814","nonce":3909095186},{"type":"v1","addr":"172.21.15.143:6815","nonce":3909095186}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6812","nonce":3909095186},{"type":"v1","addr":"172.21.15.143:6813","nonce":3909095186}]},"public_addr":"172.21.15.143:6809/3909095186","cluster_addr":"172.21.15.143:6811/3909095186","heartbeat_back_addr":"172.21.15.143:6815/3909095186","heartbeat_front_addr":"172.21.15.143:6813/3909095186","state":["exists","up"]},{"osd":3,"uuid":"d1f52435-7bc1-4081-9550-9cd1ba55f549","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":43,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6800","nonce":3854210991},{"type":"v1","addr":"172.21.15.143:6801","nonce":3854210991}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6802","nonce":3854210991},{"type":"v1","addr":"172.21.15.143:6803","nonce":3854210991}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6806","nonce":3854210991},{"type":"v1","addr":"172.21.15.143:6807","nonce":3854210991}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6804","nonce":3854210991},{"type":"v1","addr":"172.21.15.143:6805","nonce":3854210991}]},"public_addr":"172.21.15.143:6801/3854210991","cluster_addr":"172.21.15.143:6803/3854210991","heartbeat_back_addr":"172.21.15.143:6807/3854210991","heartbeat_front_addr":"172.21.15.143:6805/3854210991","state":["exists","up"]},{"osd":4,"uuid":"e2ae8b6e-6e25-49b2-bcbf-aacce68f06a3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":43,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6800","nonce":3924606902},{"type":"v1","addr":"172.21.15.188:6801","nonce":3924606902}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6802","nonce":3924606902},{"type":"v1","addr":"172.21.15.188:6803","nonce":3924606902}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6806","nonce":3924606902},{"type":"v1","addr":"172.21.15.188:6807","nonce":3924606902}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6804","nonce":3924606902},{"type":"v1","addr":"172.21.15.188:6805","nonce":3924606902}]},"public_addr":"172.21.15.188:6801/3924606902","cluster_addr":"172.21.15.188:6803/3924606902","heartbeat_back_addr":"172.21.15.188:6807/3924606902","heartbeat_front_addr":"172.21.15.188:6805/3924606902","state":["exists","up"]},{"osd":5,"uuid":"3cec8a32-7698-4c36-8e89-e4ba11934f33","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":43,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6813","nonce":186841544},{"type":"v1","addr":"172.21.15.188:6815","nonce":186841544}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6817","nonce":186841544},{"type":"v1","addr":"172.21.15.188:6818","nonce":186841544}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6822","nonce":186841544},{"type":"v1","addr":"172.21.15.188:6823","nonce":186841544}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6820","nonce":186841544},{"type":"v1","addr":"172.21.15.188:6821","nonce":186841544}]},"public_addr":"172.21.15.188:6815/186841544","cluster_addr":"172.21.15.188:6818/186841544","heartbeat_back_addr":"172.21.15.188:6823/186841544","heartbeat_front_addr":"172.21.15.188:6821/186841544","state":["exists","up"]},{"osd":6,"uuid":"58a65d5d-e090-473a-81b4-ad38e85029f9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":46,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6808","nonce":2153499405},{"type":"v1","addr":"172.21.15.188:6809","nonce":2153499405}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6810","nonce":2153499405},{"type":"v1","addr":"172.21.15.188:6811","nonce":2153499405}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6816","nonce":2153499405},{"type":"v1","addr":"172.21.15.188:6819","nonce":2153499405}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6812","nonce":2153499405},{"type":"v1","addr":"172.21.15.188:6814","nonce":2153499405}]},"public_addr":"172.21.15.188:6809/2153499405","cluster_addr":"172.21.15.188:6811/2153499405","heartbeat_back_addr":"172.21.15.188:6819/2153499405","heartbeat_front_addr":"172.21.15.188:6814/2153499405","state":["exists","up"]},{"osd":7,"uuid":"ea5b7f70-3554-4cf8-ac30-f7c59b80678d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":43,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6824","nonce":2722096729},{"type":"v1","addr":"172.21.15.188:6825","nonce":2722096729}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6826","nonce":2722096729},{"type":"v1","addr":"172.21.15.188:6827","nonce":2722096729}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6830","nonce":2722096729},{"type":"v1","addr":"172.21.15.188:6831","nonce":2722096729}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6828","nonce":2722096729},{"type":"v1","addr":"172.21.15.188:6829","nonce":2722096729}]},"public_addr":"172.21.15.188:6825/2722096729","cluster_addr":"172.21.15.188:6827/2722096729","heartbeat_back_addr":"172.21.15.188:6831/2722096729","heartbeat_front_addr":"172.21.15.188:6829/2722096729","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-26T07:17:36.995032+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-26T07:17:36.932266+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-26T07:17:36.829304+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-26T07:17:36.804297+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-26T07:17:36.667892+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-26T07:17:36.964868+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-26T07:17:36.827332+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-26T07:17:36.893179+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.143:6837/2872033338":"2023-11-27T07:21:43.220829+0000","172.21.15.143:6836/2872033338":"2023-11-27T07:21:43.220829+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-11-26T07:21:47.574 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:21:47.572+0000 7f99c4b57640 1 -- 172.21.15.143:0/504037534 >> [v2:172.21.15.143:6832/2150557544,v1:172.21.15.143:6833/2150557544] conn(0x7f99a0066650 msgr2=0x7f99a0068b10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:21:47.574 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:21:47.572+0000 7f99c4b57640 1 --2- 172.21.15.143:0/504037534 >> [v2:172.21.15.143:6832/2150557544,v1:172.21.15.143:6833/2150557544] conn(0x7f99a0066650 0x7f99a0068b10 secure :-1 s=READY pgs=361 cs=0 l=1 rev1=1 crypto rx=0x7f99b00b1000 tx=0x7f99b00a9070 comp rx=0 tx=0).stop 2023-11-26T07:21:47.575 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:21:47.572+0000 7f99c4b57640 1 -- 172.21.15.143:0/504037534 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f99c00cf990 msgr2=0x7f99c013ee50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:21:47.575 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:21:47.572+0000 7f99c4b57640 1 --2- 172.21.15.143:0/504037534 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f99c00cf990 0x7f99c013ee50 secure :-1 s=READY pgs=756 cs=0 l=1 rev1=1 crypto rx=0x7f99a800c7c0 tx=0x7f99a800cc80 comp rx=0 tx=0).stop 2023-11-26T07:21:47.575 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:21:47.572+0000 7f99c4b57640 1 -- 172.21.15.143:0/504037534 shutdown_connections 2023-11-26T07:21:47.575 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:21:47.572+0000 7f99c4b57640 1 --2- 172.21.15.143:0/504037534 >> [v2:172.21.15.143:6832/2150557544,v1:172.21.15.143:6833/2150557544] conn(0x7f99a0066650 0x7f99a0068b10 unknown :-1 s=CLOSED pgs=361 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:21:47.575 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:21:47.572+0000 7f99c4b57640 1 --2- 172.21.15.143:0/504037534 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f99c0143ac0 0x7f99c0140a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:21:47.575 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:21:47.572+0000 7f99c4b57640 1 --2- 172.21.15.143:0/504037534 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f99c00cf990 0x7f99c013ee50 unknown :-1 s=CLOSED pgs=756 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:21:47.575 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:21:47.572+0000 7f99c4b57640 1 --2- 172.21.15.143:0/504037534 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f99c00708b0 0x7f99c0137180 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:21:47.575 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:21:47.572+0000 7f99c4b57640 1 -- 172.21.15.143:0/504037534 >> 172.21.15.143:0/504037534 conn(0x7f99c006a890 msgr2=0x7f99c006c520 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:21:47.575 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:21:47.572+0000 7f99c4b57640 1 -- 172.21.15.143:0/504037534 shutdown_connections 2023-11-26T07:21:47.575 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:21:47.572+0000 7f99c4b57640 1 -- 172.21.15.143:0/504037534 wait complete. 2023-11-26T07:21:47.593 INFO:tasks.ceph_test_case.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-11-26T07:17:41.758597+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2023-11-26T07:21:47.593 DEBUG:teuthology.orchestra.run.smithi143:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-11-26T07:21:47.733 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:21:47.732+0000 7fb30df35640 1 Processor -- start 2023-11-26T07:21:47.733 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:21:47.732+0000 7fb30df35640 1 -- start start 2023-11-26T07:21:47.735 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:21:47.732+0000 7fb30df35640 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7fb30813e290 0x7fb3080cf280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:21:47.735 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:21:47.732+0000 7fb30df35640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fb30815f6b0 0x7fb3080cd7b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:21:47.735 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:21:47.732+0000 7fb30df35640 1 -- --> v1:172.21.15.143:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb30806c520 con 0x7fb3080cf7c0 2023-11-26T07:21:47.735 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:21:47.732+0000 7fb30df35640 1 -- --> v2:172.21.15.143:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb3080cfc40 con 0x7fb30813e290 2023-11-26T07:21:47.735 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:21:47.732+0000 7fb30df35640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb30815fb40 con 0x7fb30815f6b0 2023-11-26T07:21:47.735 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:21:47.732+0000 7fb3077fe640 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7fb30813e290 0x7fb3080cf280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:21:47.735 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:21:47.732+0000 7fb306ffd640 1 --1- >> v1:172.21.15.143:6789/0 conn(0x7fb3080cf7c0 0x7fb30815efa0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.143:6789/0 says I am v1:172.21.15.143:52718/0 (socket says 172.21.15.143:52718) 2023-11-26T07:21:47.735 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:21:47.732+0000 7fb306ffd640 1 -- 172.21.15.143:0/4233950814 learned_addr learned my addr 172.21.15.143:0/4233950814 (peer_addr_for_me v1:172.21.15.143:0/0) 2023-11-26T07:21:47.735 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:21:47.732+0000 7fb307fff640 1 --2- 172.21.15.143:0/4233950814 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fb30815f6b0 0x7fb3080cd7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:21:47.736 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:21:47.732+0000 7fb3077fe640 1 -- 172.21.15.143:0/4233950814 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fb30815f6b0 msgr2=0x7fb3080cd7b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-26T07:21:47.736 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:21:47.732+0000 7fb3077fe640 1 --2- 172.21.15.143:0/4233950814 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fb30815f6b0 0x7fb3080cd7b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:21:47.736 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:21:47.732+0000 7fb3077fe640 1 -- 172.21.15.143:0/4233950814 >> v1:172.21.15.143:6789/0 conn(0x7fb3080cf7c0 legacy=0x7fb30815efa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:21:47.736 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:21:47.732+0000 7fb3077fe640 1 -- 172.21.15.143:0/4233950814 --> v2:172.21.15.143:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb30806c800 con 0x7fb30813e290 2023-11-26T07:21:47.736 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:21:47.732+0000 7fb307fff640 1 --2- 172.21.15.143:0/4233950814 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fb30815f6b0 0x7fb3080cd7b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-26T07:21:47.738 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:21:47.736+0000 7fb3077fe640 1 --2- 172.21.15.143:0/4233950814 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fb30813e290 0x7fb3080cf280 secure :-1 s=READY pgs=758 cs=0 l=1 rev1=1 crypto rx=0x7fb2f80097c0 tx=0x7fb2f801c450 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2533bd783ae9d625 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:21:47.738 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:21:47.736+0000 7fb3067fc640 1 -- 172.21.15.143:0/4233950814 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb2f80ab650 con 0x7fb30813e290 2023-11-26T07:21:47.738 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:21:47.736+0000 7fb3067fc640 1 -- 172.21.15.143:0/4233950814 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb2f80ab7b0 con 0x7fb30813e290 2023-11-26T07:21:47.738 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:21:47.736+0000 7fb3067fc640 1 -- 172.21.15.143:0/4233950814 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb2f80abb40 con 0x7fb30813e290 2023-11-26T07:21:47.739 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:21:47.736+0000 7fb30df35640 1 -- 172.21.15.143:0/4233950814 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fb30813e290 msgr2=0x7fb3080cf280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:21:47.739 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:21:47.736+0000 7fb30df35640 1 --2- 172.21.15.143:0/4233950814 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fb30813e290 0x7fb3080cf280 secure :-1 s=READY pgs=758 cs=0 l=1 rev1=1 crypto rx=0x7fb2f80097c0 tx=0x7fb2f801c450 comp rx=0 tx=0).stop 2023-11-26T07:21:47.739 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:21:47.736+0000 7fb30df35640 1 -- 172.21.15.143:0/4233950814 shutdown_connections 2023-11-26T07:21:47.739 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:21:47.736+0000 7fb30df35640 1 --2- 172.21.15.143:0/4233950814 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fb30815f6b0 0x7fb3080cd7b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:21:47.739 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:21:47.736+0000 7fb30df35640 1 --2- 172.21.15.143:0/4233950814 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fb30813e290 0x7fb3080cf280 unknown :-1 s=CLOSED pgs=758 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:21:47.740 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:21:47.736+0000 7fb30df35640 1 -- 172.21.15.143:0/4233950814 >> 172.21.15.143:0/4233950814 conn(0x7fb30806a890 msgr2=0x7fb30806aca0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:21:47.740 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:21:47.736+0000 7fb30df35640 1 -- 172.21.15.143:0/4233950814 shutdown_connections 2023-11-26T07:21:47.740 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:21:47.736+0000 7fb30df35640 1 -- 172.21.15.143:0/4233950814 wait complete. 2023-11-26T07:21:47.740 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:21:47.740+0000 7fb30df35640 1 Processor -- start 2023-11-26T07:21:47.741 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:21:47.740+0000 7fb30df35640 1 -- start start 2023-11-26T07:21:47.743 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:21:47.740+0000 7fb30df35640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fb3080cf7c0 0x7fb30813ae80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:21:47.743 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:21:47.740+0000 7fb30df35640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fb30813e290 0x7fb308141270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:21:47.743 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:21:47.740+0000 7fb30df35640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fb30815f6b0 0x7fb30813f210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:21:47.744 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:21:47.740+0000 7fb30df35640 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb308158840 con 0x7fb30815f6b0 2023-11-26T07:21:47.744 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:21:47.740+0000 7fb30df35640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb3081417b0 con 0x7fb30813e290 2023-11-26T07:21:47.744 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:21:47.740+0000 7fb30df35640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb308159280 con 0x7fb3080cf7c0 2023-11-26T07:21:47.744 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:21:47.740+0000 7fb307fff640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fb30815f6b0 0x7fb30813f210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:21:47.744 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:21:47.740+0000 7fb307fff640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fb30815f6b0 0x7fb30813f210 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:49310/0 (socket says 172.21.15.143:49310) 2023-11-26T07:21:47.744 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:21:47.740+0000 7fb307fff640 1 -- 172.21.15.143:0/3562105553 learned_addr learned my addr 172.21.15.143:0/3562105553 (peer_addr_for_me v2:172.21.15.143:0/0) 2023-11-26T07:21:47.744 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:21:47.740+0000 7fb3077fe640 1 --2- 172.21.15.143:0/3562105553 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fb3080cf7c0 0x7fb30813ae80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:21:47.744 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:21:47.740+0000 7fb306ffd640 1 --2- 172.21.15.143:0/3562105553 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fb30813e290 0x7fb308141270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:21:47.744 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:21:47.740+0000 7fb307fff640 1 -- 172.21.15.143:0/3562105553 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fb3080cf7c0 msgr2=0x7fb30813ae80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:21:47.744 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:21:47.740+0000 7fb307fff640 1 --2- 172.21.15.143:0/3562105553 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fb3080cf7c0 0x7fb30813ae80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:21:47.744 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:21:47.740+0000 7fb307fff640 1 -- 172.21.15.143:0/3562105553 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fb30813e290 msgr2=0x7fb308141270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:21:47.745 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:21:47.740+0000 7fb307fff640 1 --2- 172.21.15.143:0/3562105553 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fb30813e290 0x7fb308141270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:21:47.745 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:21:47.740+0000 7fb307fff640 1 -- 172.21.15.143:0/3562105553 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb2f8009420 con 0x7fb30815f6b0 2023-11-26T07:21:47.745 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:21:47.740+0000 7fb306ffd640 1 --2- 172.21.15.143:0/3562105553 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fb30813e290 0x7fb308141270 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-26T07:21:47.745 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:21:47.740+0000 7fb307fff640 1 --2- 172.21.15.143:0/3562105553 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fb30815f6b0 0x7fb30813f210 secure :-1 s=READY pgs=759 cs=0 l=1 rev1=1 crypto rx=0x7fb30000cd50 tx=0x7fb30000a650 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:21:47.745 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:21:47.740+0000 7fb2effff640 1 -- 172.21.15.143:0/3562105553 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb30000f680 con 0x7fb30815f6b0 2023-11-26T07:21:47.745 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:21:47.740+0000 7fb2effff640 1 -- 172.21.15.143:0/3562105553 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb30000f800 con 0x7fb30815f6b0 2023-11-26T07:21:47.745 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:21:47.740+0000 7fb2effff640 1 -- 172.21.15.143:0/3562105553 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb30000fbb0 con 0x7fb30815f6b0 2023-11-26T07:21:47.745 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:21:47.740+0000 7fb30df35640 1 -- 172.21.15.143:0/3562105553 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb308159560 con 0x7fb30815f6b0 2023-11-26T07:21:47.745 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:21:47.740+0000 7fb30df35640 1 -- 172.21.15.143:0/3562105553 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb308159b20 con 0x7fb30815f6b0 2023-11-26T07:21:47.746 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:21:47.740+0000 7fb30df35640 1 -- 172.21.15.143:0/3562105553 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb308159d40 con 0x7fb30815f6b0 2023-11-26T07:21:47.746 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:21:47.744+0000 7fb2effff640 1 -- 172.21.15.143:0/3562105553 <== mon.0 v2:172.21.15.143:3300/0 4 ==== mgrmap(e 9) v1 ==== 86005+0+0 (secure 0 0 0) 0x7fb30000b040 con 0x7fb30815f6b0 2023-11-26T07:21:47.747 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:21:47.744+0000 7fb2effff640 1 --2- 172.21.15.143:0/3562105553 >> [v2:172.21.15.143:6832/2150557544,v1:172.21.15.143:6833/2150557544] conn(0x7fb2f4066620 0x7fb2f4068ae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:21:47.747 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:21:47.744+0000 7fb2effff640 1 -- 172.21.15.143:0/3562105553 <== mon.0 v2:172.21.15.143:3300/0 5 ==== osd_map(50..50 src has 1..50) v4 ==== 4943+0+0 (secure 0 0 0) 0x7fb3000849a0 con 0x7fb30815f6b0 2023-11-26T07:21:47.747 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:21:47.744+0000 7fb3077fe640 1 --2- 172.21.15.143:0/3562105553 >> [v2:172.21.15.143:6832/2150557544,v1:172.21.15.143:6833/2150557544] conn(0x7fb2f4066620 0x7fb2f4068ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:21:47.747 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:21:47.744+0000 7fb3077fe640 1 --2- 172.21.15.143:0/3562105553 >> [v2:172.21.15.143:6832/2150557544,v1:172.21.15.143:6833/2150557544] conn(0x7fb2f4066620 0x7fb2f4068ae0 secure :-1 s=READY pgs=362 cs=0 l=1 rev1=1 crypto rx=0x7fb2f80af000 tx=0x7fb2f80a7000 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:21:47.750 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:21:47.748+0000 7fb2effff640 1 -- 172.21.15.143:0/3562105553 <== mon.0 v2:172.21.15.143:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172649 (secure 0 0 0) 0x7fb300051e20 con 0x7fb30815f6b0 2023-11-26T07:21:47.902 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:21:47.900+0000 7fb30df35640 1 -- 172.21.15.143:0/3562105553 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7fb308067680 con 0x7fb30815f6b0 2023-11-26T07:21:47.903 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:21:47.900+0000 7fb2effff640 1 -- 172.21.15.143:0/3562105553 <== mon.0 v2:172.21.15.143:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v50) v1 ==== 93+0+10 (secure 0 0 0) 0x7fb30001d020 con 0x7fb30815f6b0 2023-11-26T07:21:47.903 INFO:teuthology.orchestra.run.smithi143.stdout:pg_num: 1 2023-11-26T07:21:47.905 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:21:47.904+0000 7fb30df35640 1 -- 172.21.15.143:0/3562105553 >> [v2:172.21.15.143:6832/2150557544,v1:172.21.15.143:6833/2150557544] conn(0x7fb2f4066620 msgr2=0x7fb2f4068ae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:21:47.905 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:21:47.904+0000 7fb30df35640 1 --2- 172.21.15.143:0/3562105553 >> [v2:172.21.15.143:6832/2150557544,v1:172.21.15.143:6833/2150557544] conn(0x7fb2f4066620 0x7fb2f4068ae0 secure :-1 s=READY pgs=362 cs=0 l=1 rev1=1 crypto rx=0x7fb2f80af000 tx=0x7fb2f80a7000 comp rx=0 tx=0).stop 2023-11-26T07:21:47.905 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:21:47.904+0000 7fb30df35640 1 -- 172.21.15.143:0/3562105553 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fb30815f6b0 msgr2=0x7fb30813f210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:21:47.905 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:21:47.904+0000 7fb30df35640 1 --2- 172.21.15.143:0/3562105553 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fb30815f6b0 0x7fb30813f210 secure :-1 s=READY pgs=759 cs=0 l=1 rev1=1 crypto rx=0x7fb30000cd50 tx=0x7fb30000a650 comp rx=0 tx=0).stop 2023-11-26T07:21:47.905 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:21:47.904+0000 7fb30df35640 1 -- 172.21.15.143:0/3562105553 shutdown_connections 2023-11-26T07:21:47.905 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:21:47.904+0000 7fb30df35640 1 --2- 172.21.15.143:0/3562105553 >> [v2:172.21.15.143:6832/2150557544,v1:172.21.15.143:6833/2150557544] conn(0x7fb2f4066620 0x7fb2f4068ae0 unknown :-1 s=CLOSED pgs=362 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:21:47.905 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:21:47.904+0000 7fb30df35640 1 --2- 172.21.15.143:0/3562105553 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fb30815f6b0 0x7fb30813f210 unknown :-1 s=CLOSED pgs=759 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:21:47.905 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:21:47.904+0000 7fb30df35640 1 --2- 172.21.15.143:0/3562105553 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fb30813e290 0x7fb308141270 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:21:47.905 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:21:47.904+0000 7fb30df35640 1 --2- 172.21.15.143:0/3562105553 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fb3080cf7c0 0x7fb30813ae80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:21:47.906 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:21:47.904+0000 7fb30df35640 1 -- 172.21.15.143:0/3562105553 >> 172.21.15.143:0/3562105553 conn(0x7fb30806a890 msgr2=0x7fb30815b510 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:21:47.906 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:21:47.904+0000 7fb30df35640 1 -- 172.21.15.143:0/3562105553 shutdown_connections 2023-11-26T07:21:47.906 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:21:47.904+0000 7fb30df35640 1 -- 172.21.15.143:0/3562105553 wait complete. 2023-11-26T07:21:47.924 DEBUG:teuthology.orchestra.run.smithi143:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Starting test tasks.cephfs.test_volumes.TestMisc.test_malicious_metafile_on_legacy_to_v1_upgrade' 2023-11-26T07:21:48.061 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:21:48.060+0000 7efdff59e640 1 Processor -- start 2023-11-26T07:21:48.061 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:21:48.060+0000 7efdff59e640 1 -- start start 2023-11-26T07:21:48.062 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:21:48.060+0000 7efdff59e640 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7efe0015d680 0x7efe000d00e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:21:48.063 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:21:48.060+0000 7efdff59e640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7efe000d1d90 0x7efe000cec30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:21:48.063 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:21:48.060+0000 7efdff59e640 1 -- --> v1:172.21.15.143:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7efe000cdc40 con 0x7efe0015d2a0 2023-11-26T07:21:48.063 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:21:48.060+0000 7efdff59e640 1 -- --> v2:172.21.15.188:3300/0 -- mon_getmap magic: 0 v1 -- 0x7efe000ce7a0 con 0x7efe0015d680 2023-11-26T07:21:48.063 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:21:48.060+0000 7efdff59e640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7efe000d1440 con 0x7efe000d1d90 2023-11-26T07:21:48.063 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:21:48.060+0000 7efdfe59c640 1 --1- >> v1:172.21.15.143:6789/0 conn(0x7efe0015d2a0 0x7efe000d1680 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.143:6789/0 says I am v1:172.21.15.143:52722/0 (socket says 172.21.15.143:52722) 2023-11-26T07:21:48.063 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:21:48.060+0000 7efdfe59c640 1 -- 172.21.15.143:0/1895882589 learned_addr learned my addr 172.21.15.143:0/1895882589 (peer_addr_for_me v1:172.21.15.143:0/0) 2023-11-26T07:21:48.063 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:21:48.060+0000 7efdf5d9b640 1 --2- 172.21.15.143:0/1895882589 >> v2:172.21.15.188:3300/0 conn(0x7efe0015d680 0x7efe000d00e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:21:48.063 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:21:48.060+0000 7efdfed9d640 1 --2- 172.21.15.143:0/1895882589 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7efe000d1d90 0x7efe000cec30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:21:48.064 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:21:48.060+0000 7efdfdd9b640 1 -- 172.21.15.143:0/1895882589 <== mon.0 v1:172.21.15.143:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1820228498 0 0) 0x7efe000cdc40 con 0x7efe0015d2a0 2023-11-26T07:21:48.064 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:21:48.060+0000 7efdfdd9b640 1 -- 172.21.15.143:0/1895882589 --> v1:172.21.15.143:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7efdec003530 con 0x7efe0015d2a0 2023-11-26T07:21:48.064 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:21:48.060+0000 7efdf5d9b640 1 -- 172.21.15.143:0/1895882589 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7efe000d1d90 msgr2=0x7efe000cec30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-26T07:21:48.064 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:21:48.060+0000 7efdf5d9b640 1 --2- 172.21.15.143:0/1895882589 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7efe000d1d90 0x7efe000cec30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:21:48.064 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:21:48.060+0000 7efdf5d9b640 1 -- 172.21.15.143:0/1895882589 >> v1:172.21.15.143:6789/0 conn(0x7efe0015d2a0 legacy=0x7efe000d1680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:21:48.064 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:21:48.060+0000 7efdf5d9b640 1 -- 172.21.15.143:0/1895882589 --> v2:172.21.15.188:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efe000cde70 con 0x7efe0015d680 2023-11-26T07:21:48.064 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:21:48.060+0000 7efdfed9d640 1 --2- 172.21.15.143:0/1895882589 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7efe000d1d90 0x7efe000cec30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-26T07:21:48.066 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:21:48.064+0000 7efdf5d9b640 1 --2- 172.21.15.143:0/1895882589 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7efe0015d680 0x7efe000d00e0 secure :-1 s=READY pgs=297 cs=0 l=1 rev1=1 crypto rx=0x7efde80097c0 tx=0x7efde801c450 comp rx=0 tx=0).ready entity=mon.1 client_cookie=1118a13a0b9f9bf4 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:21:48.067 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:21:48.064+0000 7efdfdd9b640 1 -- 172.21.15.143:0/1895882589 <== mon.1 v2:172.21.15.188:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efde80ad660 con 0x7efe0015d680 2023-11-26T07:21:48.067 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:21:48.064+0000 7efdfdd9b640 1 -- 172.21.15.143:0/1895882589 <== mon.1 v2:172.21.15.188:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7efde80ad7c0 con 0x7efe0015d680 2023-11-26T07:21:48.067 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:21:48.064+0000 7efdfdd9b640 1 -- 172.21.15.143:0/1895882589 <== mon.1 v2:172.21.15.188:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efde80adb30 con 0x7efe0015d680 2023-11-26T07:21:48.067 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:21:48.064+0000 7efdff59e640 1 -- 172.21.15.143:0/1895882589 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7efe0015d680 msgr2=0x7efe000d00e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:21:48.067 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:21:48.064+0000 7efdff59e640 1 --2- 172.21.15.143:0/1895882589 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7efe0015d680 0x7efe000d00e0 secure :-1 s=READY pgs=297 cs=0 l=1 rev1=1 crypto rx=0x7efde80097c0 tx=0x7efde801c450 comp rx=0 tx=0).stop 2023-11-26T07:21:48.067 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:21:48.064+0000 7efdff59e640 1 -- 172.21.15.143:0/1895882589 shutdown_connections 2023-11-26T07:21:48.068 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:21:48.064+0000 7efdff59e640 1 --2- 172.21.15.143:0/1895882589 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7efe000d1d90 0x7efe000cec30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:21:48.068 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:21:48.064+0000 7efdff59e640 1 --2- 172.21.15.143:0/1895882589 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7efe0015d680 0x7efe000d00e0 unknown :-1 s=CLOSED pgs=297 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:21:48.068 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:21:48.064+0000 7efdff59e640 1 -- 172.21.15.143:0/1895882589 >> 172.21.15.143:0/1895882589 conn(0x7efe000d30a0 msgr2=0x7efe000d3480 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:21:48.068 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:21:48.064+0000 7efdff59e640 1 -- 172.21.15.143:0/1895882589 shutdown_connections 2023-11-26T07:21:48.068 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:21:48.064+0000 7efdff59e640 1 -- 172.21.15.143:0/1895882589 wait complete. 2023-11-26T07:21:48.069 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:21:48.068+0000 7efdff59e640 1 Processor -- start 2023-11-26T07:21:48.069 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:21:48.068+0000 7efdff59e640 1 -- start start 2023-11-26T07:21:48.071 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:21:48.068+0000 7efdff59e640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7efe000d1d90 0x7efe000b4070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:21:48.071 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:21:48.068+0000 7efdff59e640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7efe0015d2a0 0x7efe000cabc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:21:48.071 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:21:48.068+0000 7efdff59e640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7efe0015d680 0x7efe00154a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:21:48.071 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:21:48.068+0000 7efdff59e640 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efe000b45b0 con 0x7efe0015d680 2023-11-26T07:21:48.071 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:21:48.068+0000 7efdff59e640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efe000cb100 con 0x7efe000d1d90 2023-11-26T07:21:48.071 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:21:48.068+0000 7efdff59e640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7efe000ca170 con 0x7efe0015d2a0 2023-11-26T07:21:48.072 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:21:48.068+0000 7efdfed9d640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7efe0015d680 0x7efe00154a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:21:48.072 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:21:48.068+0000 7efdfed9d640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7efe0015d680 0x7efe00154a20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:49324/0 (socket says 172.21.15.143:49324) 2023-11-26T07:21:48.072 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:21:48.068+0000 7efdfed9d640 1 -- 172.21.15.143:0/2515173499 learned_addr learned my addr 172.21.15.143:0/2515173499 (peer_addr_for_me v2:172.21.15.143:0/0) 2023-11-26T07:21:48.072 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:21:48.068+0000 7efdfe59c640 1 --2- 172.21.15.143:0/2515173499 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7efe000d1d90 0x7efe000b4070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:21:48.072 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:21:48.068+0000 7efdf5d9b640 1 --2- 172.21.15.143:0/2515173499 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7efe0015d2a0 0x7efe000cabc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:21:48.072 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:21:48.068+0000 7efdfed9d640 1 -- 172.21.15.143:0/2515173499 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7efe0015d2a0 msgr2=0x7efe000cabc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:21:48.072 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:21:48.068+0000 7efdfed9d640 1 --2- 172.21.15.143:0/2515173499 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7efe0015d2a0 0x7efe000cabc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:21:48.072 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:21:48.068+0000 7efdfed9d640 1 -- 172.21.15.143:0/2515173499 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7efe000d1d90 msgr2=0x7efe000b4070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:21:48.072 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:21:48.068+0000 7efdfed9d640 1 --2- 172.21.15.143:0/2515173499 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7efe000d1d90 0x7efe000b4070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:21:48.072 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:21:48.068+0000 7efdfed9d640 1 -- 172.21.15.143:0/2515173499 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efdf8009a70 con 0x7efe0015d680 2023-11-26T07:21:48.072 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:21:48.068+0000 7efdf5d9b640 1 --2- 172.21.15.143:0/2515173499 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7efe0015d2a0 0x7efe000cabc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-26T07:21:48.073 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:21:48.068+0000 7efdfed9d640 1 --2- 172.21.15.143:0/2515173499 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7efe0015d680 0x7efe00154a20 secure :-1 s=READY pgs=761 cs=0 l=1 rev1=1 crypto rx=0x7efdf800ef90 tx=0x7efdf8009f90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:21:48.073 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:21:48.068+0000 7efdf77fe640 1 -- 172.21.15.143:0/2515173499 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efdf8009d70 con 0x7efe0015d680 2023-11-26T07:21:48.073 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:21:48.068+0000 7efdf77fe640 1 -- 172.21.15.143:0/2515173499 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7efdf800f040 con 0x7efe0015d680 2023-11-26T07:21:48.074 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:21:48.068+0000 7efdff59e640 1 -- 172.21.15.143:0/2515173499 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7efde8009420 con 0x7efe0015d680 2023-11-26T07:21:48.074 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:21:48.068+0000 7efdf77fe640 1 -- 172.21.15.143:0/2515173499 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efdf80146c0 con 0x7efe0015d680 2023-11-26T07:21:48.074 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:21:48.068+0000 7efdff59e640 1 -- 172.21.15.143:0/2515173499 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7efe00154f60 con 0x7efe0015d680 2023-11-26T07:21:48.074 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:21:48.072+0000 7efdff59e640 1 -- 172.21.15.143:0/2515173499 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7efe00067500 con 0x7efe0015d680 2023-11-26T07:21:48.074 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:21:48.072+0000 7efdf77fe640 1 -- 172.21.15.143:0/2515173499 <== mon.0 v2:172.21.15.143:3300/0 4 ==== mgrmap(e 9) v1 ==== 86005+0+0 (secure 0 0 0) 0x7efdf8014820 con 0x7efe0015d680 2023-11-26T07:21:48.074 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:21:48.072+0000 7efdf77fe640 1 --2- 172.21.15.143:0/2515173499 >> [v2:172.21.15.143:6832/2150557544,v1:172.21.15.143:6833/2150557544] conn(0x7efde0066620 0x7efde0068ae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:21:48.075 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:21:48.072+0000 7efdf77fe640 1 -- 172.21.15.143:0/2515173499 <== mon.0 v2:172.21.15.143:3300/0 5 ==== osd_map(50..50 src has 1..50) v4 ==== 4943+0+0 (secure 0 0 0) 0x7efdf8084f50 con 0x7efe0015d680 2023-11-26T07:21:48.075 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:21:48.072+0000 7efdfe59c640 1 --2- 172.21.15.143:0/2515173499 >> [v2:172.21.15.143:6832/2150557544,v1:172.21.15.143:6833/2150557544] conn(0x7efde0066620 0x7efde0068ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:21:48.075 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:21:48.072+0000 7efdfe59c640 1 --2- 172.21.15.143:0/2515173499 >> [v2:172.21.15.143:6832/2150557544,v1:172.21.15.143:6833/2150557544] conn(0x7efde0066620 0x7efde0068ae0 secure :-1 s=READY pgs=363 cs=0 l=1 rev1=1 crypto rx=0x7efdf0007fd0 tx=0x7efdf0009040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:21:48.078 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:21:48.076+0000 7efdf77fe640 1 -- 172.21.15.143:0/2515173499 <== mon.0 v2:172.21.15.143:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172649 (secure 0 0 0) 0x7efdf8052340 con 0x7efe0015d680 2023-11-26T07:21:48.282 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:21:48.280+0000 7efdff59e640 1 -- 172.21.15.143:0/2515173499 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_volumes.TestMisc.test_malicious_metafile_on_legacy_to_v1_upgrade"]} v 0) v1 -- 0x7efe0008c7f0 con 0x7efe0015d680 2023-11-26T07:21:49.241 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:21:49.240+0000 7efdf77fe640 1 -- 172.21.15.143:0/2515173499 <== mon.0 v2:172.21.15.143:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_volumes.TestMisc.test_malicious_metafile_on_legacy_to_v1_upgrade"]}]=0 v205) v1 ==== 164+0+0 (secure 0 0 0) 0x7efdf80180e0 con 0x7efe0015d680 2023-11-26T07:21:49.244 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:21:49.240+0000 7efdff59e640 1 -- 172.21.15.143:0/2515173499 >> [v2:172.21.15.143:6832/2150557544,v1:172.21.15.143:6833/2150557544] conn(0x7efde0066620 msgr2=0x7efde0068ae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:21:49.244 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:21:49.240+0000 7efdff59e640 1 --2- 172.21.15.143:0/2515173499 >> [v2:172.21.15.143:6832/2150557544,v1:172.21.15.143:6833/2150557544] conn(0x7efde0066620 0x7efde0068ae0 secure :-1 s=READY pgs=363 cs=0 l=1 rev1=1 crypto rx=0x7efdf0007fd0 tx=0x7efdf0009040 comp rx=0 tx=0).stop 2023-11-26T07:21:49.244 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:21:49.240+0000 7efdff59e640 1 -- 172.21.15.143:0/2515173499 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7efe0015d680 msgr2=0x7efe00154a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:21:49.244 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:21:49.240+0000 7efdff59e640 1 --2- 172.21.15.143:0/2515173499 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7efe0015d680 0x7efe00154a20 secure :-1 s=READY pgs=761 cs=0 l=1 rev1=1 crypto rx=0x7efdf800ef90 tx=0x7efdf8009f90 comp rx=0 tx=0).stop 2023-11-26T07:21:49.244 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:21:49.240+0000 7efdff59e640 1 -- 172.21.15.143:0/2515173499 shutdown_connections 2023-11-26T07:21:49.244 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:21:49.240+0000 7efdff59e640 1 --2- 172.21.15.143:0/2515173499 >> [v2:172.21.15.143:6832/2150557544,v1:172.21.15.143:6833/2150557544] conn(0x7efde0066620 0x7efde0068ae0 unknown :-1 s=CLOSED pgs=363 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:21:49.244 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:21:49.240+0000 7efdff59e640 1 --2- 172.21.15.143:0/2515173499 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7efe0015d680 0x7efe00154a20 unknown :-1 s=CLOSED pgs=761 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:21:49.245 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:21:49.240+0000 7efdff59e640 1 --2- 172.21.15.143:0/2515173499 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7efe0015d2a0 0x7efe000cabc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:21:49.245 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:21:49.240+0000 7efdff59e640 1 --2- 172.21.15.143:0/2515173499 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7efe000d1d90 0x7efe000b4070 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:21:49.245 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:21:49.240+0000 7efdff59e640 1 -- 172.21.15.143:0/2515173499 >> 172.21.15.143:0/2515173499 conn(0x7efe000d30a0 msgr2=0x7efe000d1680 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:21:49.245 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:21:49.240+0000 7efdff59e640 1 -- 172.21.15.143:0/2515173499 shutdown_connections 2023-11-26T07:21:49.245 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:21:49.240+0000 7efdff59e640 1 -- 172.21.15.143:0/2515173499 wait complete. 2023-11-26T07:21:49.269 DEBUG:teuthology.orchestra.run.smithi143:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mon mon_allow_pool_delete True 2023-11-26T07:21:49.416 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:21:49.412+0000 7f18ae7aa640 1 Processor -- start 2023-11-26T07:21:49.416 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:21:49.412+0000 7f18ae7aa640 1 -- start start 2023-11-26T07:21:49.417 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:21:49.416+0000 7f18ae7aa640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f18a80cf280 0x7f18a80cf6e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:21:49.417 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:21:49.416+0000 7f18ae7aa640 1 -- --> v1:172.21.15.143:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f18a80cfc20 con 0x7f18a80cd1b0 2023-11-26T07:21:49.417 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:21:49.416+0000 7f18ae7aa640 1 -- --> v1:172.21.15.188:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f18a80cc730 con 0x7f18a8143ac0 2023-11-26T07:21:49.417 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:21:49.416+0000 7f18ae7aa640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f18a8143ea0 con 0x7f18a80cf280 2023-11-26T07:21:49.418 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:21:49.416+0000 7f18acd47640 1 --1- >> v1:172.21.15.143:6789/0 conn(0x7f18a80cd1b0 0x7f18a80cd5e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.143:6789/0 says I am v1:172.21.15.143:52730/0 (socket says 172.21.15.143:52730) 2023-11-26T07:21:49.418 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:21:49.416+0000 7f18acd47640 1 -- 172.21.15.143:0/558206392 learned_addr learned my addr 172.21.15.143:0/558206392 (peer_addr_for_me v1:172.21.15.143:0/0) 2023-11-26T07:21:49.420 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:21:49.416+0000 7f18a77fe640 1 --2- 172.21.15.143:0/558206392 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f18a80cf280 0x7f18a80cf6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:21:49.420 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:21:49.416+0000 7f18a6ffd640 1 -- 172.21.15.143:0/558206392 <== mon.0 v1:172.21.15.143:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2669262654 0 0) 0x7f18a80cfc20 con 0x7f18a80cd1b0 2023-11-26T07:21:49.420 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:21:49.416+0000 7f18a6ffd640 1 -- 172.21.15.143:0/558206392 --> v1:172.21.15.143:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f1890003530 con 0x7f18a80cd1b0 2023-11-26T07:21:49.420 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:21:49.416+0000 7f18a6ffd640 1 -- 172.21.15.143:0/558206392 <== mon.1 v1:172.21.15.188:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1475070914 0 0) 0x7f18a80cc730 con 0x7f18a8143ac0 2023-11-26T07:21:49.420 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:21:49.416+0000 7f18a6ffd640 1 -- 172.21.15.143:0/558206392 --> v1:172.21.15.188:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f18a80cfc20 con 0x7f18a8143ac0 2023-11-26T07:21:49.420 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:21:49.416+0000 7f18a6ffd640 1 -- 172.21.15.143:0/558206392 <== mon.0 v1:172.21.15.143:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2761660034 0 0) 0x7f1890003530 con 0x7f18a80cd1b0 2023-11-26T07:21:49.420 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:21:49.416+0000 7f18a6ffd640 1 -- 172.21.15.143:0/558206392 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f18a80cf280 msgr2=0x7f18a80cf6e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-26T07:21:49.420 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:21:49.416+0000 7f18a6ffd640 1 --2- 172.21.15.143:0/558206392 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f18a80cf280 0x7f18a80cf6e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:21:49.420 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:21:49.416+0000 7f18a6ffd640 1 -- 172.21.15.143:0/558206392 >> v1:172.21.15.188:6789/0 conn(0x7f18a8143ac0 legacy=0x7f18a8070370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:21:49.420 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:21:49.416+0000 7f18a6ffd640 1 -- 172.21.15.143:0/558206392 --> v1:172.21.15.143:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f18a8070ee0 con 0x7f18a80cd1b0 2023-11-26T07:21:49.420 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:21:49.416+0000 7f18a6ffd640 1 -- 172.21.15.143:0/558206392 <== mon.0 v1:172.21.15.143:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2137882523 0 0) 0x7f18a0001a00 con 0x7f18a80cd1b0 2023-11-26T07:21:49.420 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:21:49.416+0000 7f18a6ffd640 1 -- 172.21.15.143:0/558206392 >> v1:172.21.15.143:6789/0 conn(0x7f18a80cd1b0 legacy=0x7f18a80cd5e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:21:49.421 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:21:49.416+0000 7f18a6ffd640 1 --2- 172.21.15.143:0/558206392 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f1890003b40 0x7f1890003f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:21:49.421 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:21:49.416+0000 7f18a6ffd640 1 --2- 172.21.15.143:0/558206392 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f18a80cf280 0x7f18900047d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:21:49.421 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:21:49.416+0000 7f18a6ffd640 1 --2- 172.21.15.143:0/558206392 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f1890004d10 0x7f1890005190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:21:49.421 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:21:49.416+0000 7f18a6ffd640 1 -- 172.21.15.143:0/558206392 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f189000ba40 con 0x7f1890004d10 2023-11-26T07:21:49.421 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:21:49.416+0000 7f18a6ffd640 1 -- 172.21.15.143:0/558206392 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f189000bbb0 con 0x7f1890003b40 2023-11-26T07:21:49.421 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:21:49.416+0000 7f18a6ffd640 1 -- 172.21.15.143:0/558206392 --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f189000df80 con 0x7f18a80cf280 2023-11-26T07:21:49.421 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:21:49.416+0000 7f18acd47640 1 --2- 172.21.15.143:0/558206392 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f1890004d10 0x7f1890005190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:21:49.421 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:21:49.416+0000 7f18a7fff640 1 --2- 172.21.15.143:0/558206392 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f1890003b40 0x7f1890003f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:21:49.421 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:21:49.416+0000 7f18a77fe640 1 --2- 172.21.15.143:0/558206392 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f18a80cf280 0x7f18900047d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:21:49.421 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:21:49.416+0000 7f18acd47640 1 -- 172.21.15.143:0/558206392 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f18a80cf280 msgr2=0x7f18900047d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-26T07:21:49.421 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:21:49.416+0000 7f18acd47640 1 --2- 172.21.15.143:0/558206392 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f18a80cf280 0x7f18900047d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:21:49.421 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:21:49.416+0000 7f18acd47640 1 -- 172.21.15.143:0/558206392 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f1890003b40 msgr2=0x7f1890003f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-26T07:21:49.422 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:21:49.416+0000 7f18acd47640 1 --2- 172.21.15.143:0/558206392 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f1890003b40 0x7f1890003f40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:21:49.422 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:21:49.416+0000 7f18acd47640 1 -- 172.21.15.143:0/558206392 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1890003530 con 0x7f1890004d10 2023-11-26T07:21:49.422 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:21:49.420+0000 7f18acd47640 1 --2- 172.21.15.143:0/558206392 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f1890004d10 0x7f1890005190 secure :-1 s=READY pgs=763 cs=0 l=1 rev1=1 crypto rx=0x7f18a000a6f0 tx=0x7f18a001f940 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bc246bc185adf89c server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:21:49.422 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:21:49.420+0000 7f18a6ffd640 1 -- 172.21.15.143:0/558206392 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f18a00ac6b0 con 0x7f1890004d10 2023-11-26T07:21:49.422 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:21:49.420+0000 7f18a6ffd640 1 -- 172.21.15.143:0/558206392 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f18a00ac810 con 0x7f1890004d10 2023-11-26T07:21:49.422 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:21:49.420+0000 7f18a6ffd640 1 -- 172.21.15.143:0/558206392 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f18a00acc00 con 0x7f1890004d10 2023-11-26T07:21:49.423 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:21:49.420+0000 7f18ae7aa640 1 -- 172.21.15.143:0/558206392 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f1890004d10 msgr2=0x7f1890005190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:21:49.423 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:21:49.420+0000 7f18ae7aa640 1 --2- 172.21.15.143:0/558206392 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f1890004d10 0x7f1890005190 secure :-1 s=READY pgs=763 cs=0 l=1 rev1=1 crypto rx=0x7f18a000a6f0 tx=0x7f18a001f940 comp rx=0 tx=0).stop 2023-11-26T07:21:49.423 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:21:49.420+0000 7f18acd47640 1 -- 172.21.15.143:0/558206392 reap_dead start 2023-11-26T07:21:49.423 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:21:49.420+0000 7f18ae7aa640 1 -- 172.21.15.143:0/558206392 shutdown_connections 2023-11-26T07:21:49.423 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:21:49.420+0000 7f18ae7aa640 1 -- 172.21.15.143:0/558206392 >> 172.21.15.143:0/558206392 conn(0x7f18a806a890 msgr2=0x7f18a806aca0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:21:49.424 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:21:49.420+0000 7f18ae7aa640 1 -- 172.21.15.143:0/558206392 shutdown_connections 2023-11-26T07:21:49.424 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:21:49.420+0000 7f18ae7aa640 1 -- 172.21.15.143:0/558206392 wait complete. 2023-11-26T07:21:49.424 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:21:49.424+0000 7f18ae7aa640 1 Processor -- start 2023-11-26T07:21:49.425 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:21:49.424+0000 7f18ae7aa640 1 -- start start 2023-11-26T07:21:49.426 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:21:49.424+0000 7f18ae7aa640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f18a81e43f0 0x7f18a81e2650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:21:49.427 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:21:49.424+0000 7f18ae7aa640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f18a813fba0 0x7f18a8159890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:21:49.427 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:21:49.424+0000 7f18ae7aa640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f18a81e47d0 0x7f18a8159dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:21:49.427 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:21:49.424+0000 7f18ae7aa640 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f18a81e2f50 con 0x7f18a81e47d0 2023-11-26T07:21:49.427 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:21:49.424+0000 7f18ae7aa640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f18a81e3230 con 0x7f18a813fba0 2023-11-26T07:21:49.427 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:21:49.424+0000 7f18ae7aa640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f18a81e3510 con 0x7f18a81e43f0 2023-11-26T07:21:49.427 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:21:49.424+0000 7f18acd47640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f18a81e47d0 0x7f18a8159dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:21:49.427 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:21:49.424+0000 7f18acd47640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f18a81e47d0 0x7f18a8159dd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:49336/0 (socket says 172.21.15.143:49336) 2023-11-26T07:21:49.427 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:21:49.424+0000 7f18acd47640 1 -- 172.21.15.143:0/1379293821 learned_addr learned my addr 172.21.15.143:0/1379293821 (peer_addr_for_me v2:172.21.15.143:0/0) 2023-11-26T07:21:49.427 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:21:49.424+0000 7f18a7fff640 1 --2- 172.21.15.143:0/1379293821 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f18a81e43f0 0x7f18a81e2650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:21:49.427 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:21:49.424+0000 7f18a77fe640 1 --2- 172.21.15.143:0/1379293821 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f18a813fba0 0x7f18a8159890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:21:49.427 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:21:49.424+0000 7f18acd47640 1 -- 172.21.15.143:0/1379293821 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f18a81e43f0 msgr2=0x7f18a81e2650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:21:49.428 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:21:49.424+0000 7f18acd47640 1 --2- 172.21.15.143:0/1379293821 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f18a81e43f0 0x7f18a81e2650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:21:49.428 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:21:49.424+0000 7f18acd47640 1 -- 172.21.15.143:0/1379293821 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f18a813fba0 msgr2=0x7f18a8159890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:21:49.428 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:21:49.424+0000 7f18acd47640 1 --2- 172.21.15.143:0/1379293821 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f18a813fba0 0x7f18a8159890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:21:49.428 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:21:49.424+0000 7f18acd47640 1 -- 172.21.15.143:0/1379293821 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f18a81e37f0 con 0x7f18a81e47d0 2023-11-26T07:21:49.428 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:21:49.424+0000 7f18a77fe640 1 --2- 172.21.15.143:0/1379293821 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f18a813fba0 0x7f18a8159890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-26T07:21:49.428 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:21:49.424+0000 7f18acd47640 1 --2- 172.21.15.143:0/1379293821 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f18a81e47d0 0x7f18a8159dd0 secure :-1 s=READY pgs=764 cs=0 l=1 rev1=1 crypto rx=0x7f18a00ae040 tx=0x7f1890005800 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:21:49.429 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:21:49.424+0000 7f18a4ff9640 1 -- 172.21.15.143:0/1379293821 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f18a00ac6b0 con 0x7f18a81e47d0 2023-11-26T07:21:49.429 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:21:49.424+0000 7f18a4ff9640 1 -- 172.21.15.143:0/1379293821 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f18a00ace80 con 0x7f18a81e47d0 2023-11-26T07:21:49.429 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:21:49.424+0000 7f18a4ff9640 1 -- 172.21.15.143:0/1379293821 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f18a0001a00 con 0x7f18a81e47d0 2023-11-26T07:21:49.429 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:21:49.424+0000 7f18ae7aa640 1 -- 172.21.15.143:0/1379293821 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f18a81e2b90 con 0x7f18a81e47d0 2023-11-26T07:21:49.429 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:21:49.424+0000 7f18ae7aa640 1 -- 172.21.15.143:0/1379293821 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f18a81f65d0 con 0x7f18a81e47d0 2023-11-26T07:21:49.429 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:21:49.424+0000 7f18ae7aa640 1 -- 172.21.15.143:0/1379293821 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f18a8067680 con 0x7f18a81e47d0 2023-11-26T07:21:49.429 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:21:49.428+0000 7f18a4ff9640 1 -- 172.21.15.143:0/1379293821 <== mon.0 v2:172.21.15.143:3300/0 4 ==== mgrmap(e 9) v1 ==== 86005+0+0 (secure 0 0 0) 0x7f18a00c7450 con 0x7f18a81e47d0 2023-11-26T07:21:49.430 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:21:49.428+0000 7f18a4ff9640 1 --2- 172.21.15.143:0/1379293821 >> [v2:172.21.15.143:6832/2150557544,v1:172.21.15.143:6833/2150557544] conn(0x7f1894066620 0x7f1894068ae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:21:49.430 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:21:49.428+0000 7f18a4ff9640 1 -- 172.21.15.143:0/1379293821 <== mon.0 v2:172.21.15.143:3300/0 5 ==== osd_map(50..50 src has 1..50) v4 ==== 4943+0+0 (secure 0 0 0) 0x7f18a0121f70 con 0x7f18a81e47d0 2023-11-26T07:21:49.430 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:21:49.428+0000 7f18a7fff640 1 --2- 172.21.15.143:0/1379293821 >> [v2:172.21.15.143:6832/2150557544,v1:172.21.15.143:6833/2150557544] conn(0x7f1894066620 0x7f1894068ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:21:49.431 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:21:49.428+0000 7f18a7fff640 1 --2- 172.21.15.143:0/1379293821 >> [v2:172.21.15.143:6832/2150557544,v1:172.21.15.143:6833/2150557544] conn(0x7f1894066620 0x7f1894068ae0 secure :-1 s=READY pgs=364 cs=0 l=1 rev1=1 crypto rx=0x7f1898007fd0 tx=0x7f1898009040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:21:49.434 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:21:49.432+0000 7f18a4ff9640 1 -- 172.21.15.143:0/1379293821 <== mon.0 v2:172.21.15.143:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172649 (secure 0 0 0) 0x7f18a00ef520 con 0x7f18a81e47d0 2023-11-26T07:21:49.585 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:21:49.584+0000 7f18ae7aa640 1 -- 172.21.15.143:0/1379293821 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command([{prefix=config set, name=mon_allow_pool_delete}] v 0) v1 -- 0x7f18a80b28f0 con 0x7f18a81e47d0 2023-11-26T07:21:49.590 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:21:49.588+0000 7f18a4ff9640 1 -- 172.21.15.143:0/1379293821 <== mon.0 v2:172.21.15.143:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mon_allow_pool_delete}]=0 v14)=0 v14) v1 ==== 122+0+0 (secure 0 0 0) 0x7f18a00011d0 con 0x7f18a81e47d0 2023-11-26T07:21:49.592 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:21:49.588+0000 7f18ae7aa640 1 -- 172.21.15.143:0/1379293821 >> [v2:172.21.15.143:6832/2150557544,v1:172.21.15.143:6833/2150557544] conn(0x7f1894066620 msgr2=0x7f1894068ae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:21:49.592 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:21:49.588+0000 7f18ae7aa640 1 --2- 172.21.15.143:0/1379293821 >> [v2:172.21.15.143:6832/2150557544,v1:172.21.15.143:6833/2150557544] conn(0x7f1894066620 0x7f1894068ae0 secure :-1 s=READY pgs=364 cs=0 l=1 rev1=1 crypto rx=0x7f1898007fd0 tx=0x7f1898009040 comp rx=0 tx=0).stop 2023-11-26T07:21:49.593 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:21:49.588+0000 7f18ae7aa640 1 -- 172.21.15.143:0/1379293821 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f18a81e47d0 msgr2=0x7f18a8159dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:21:49.593 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:21:49.588+0000 7f18ae7aa640 1 --2- 172.21.15.143:0/1379293821 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f18a81e47d0 0x7f18a8159dd0 secure :-1 s=READY pgs=764 cs=0 l=1 rev1=1 crypto rx=0x7f18a00ae040 tx=0x7f1890005800 comp rx=0 tx=0).stop 2023-11-26T07:21:49.593 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:21:49.588+0000 7f18ae7aa640 1 -- 172.21.15.143:0/1379293821 shutdown_connections 2023-11-26T07:21:49.593 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:21:49.588+0000 7f18ae7aa640 1 --2- 172.21.15.143:0/1379293821 >> [v2:172.21.15.143:6832/2150557544,v1:172.21.15.143:6833/2150557544] conn(0x7f1894066620 0x7f1894068ae0 unknown :-1 s=CLOSED pgs=364 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:21:49.593 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:21:49.588+0000 7f18ae7aa640 1 --2- 172.21.15.143:0/1379293821 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f18a81e47d0 0x7f18a8159dd0 unknown :-1 s=CLOSED pgs=764 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:21:49.593 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:21:49.588+0000 7f18ae7aa640 1 --2- 172.21.15.143:0/1379293821 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f18a813fba0 0x7f18a8159890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:21:49.593 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:21:49.588+0000 7f18ae7aa640 1 --2- 172.21.15.143:0/1379293821 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f18a81e43f0 0x7f18a81e2650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:21:49.593 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:21:49.588+0000 7f18ae7aa640 1 -- 172.21.15.143:0/1379293821 >> 172.21.15.143:0/1379293821 conn(0x7f18a806a890 msgr2=0x7f18a806aca0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:21:49.593 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:21:49.592+0000 7f18ae7aa640 1 -- 172.21.15.143:0/1379293821 shutdown_connections 2023-11-26T07:21:49.593 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:21:49.592+0000 7f18ae7aa640 1 -- 172.21.15.143:0/1379293821 wait complete. 2023-11-26T07:21:49.611 INFO:teuthology.nuke.actions:Clearing teuthology firewall rules... 2023-11-26T07:21:49.611 DEBUG:teuthology.orchestra.run.smithi143:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2023-11-26T07:21:49.698 DEBUG:teuthology.orchestra.run.smithi188:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2023-11-26T07:21:49.775 INFO:teuthology.nuke.actions:Cleared teuthology firewall rules. 2023-11-26T07:21:49.775 DEBUG:teuthology.orchestra.run.smithi143:> set -ex 2023-11-26T07:21:49.775 DEBUG:teuthology.orchestra.run.smithi143:> dd if=/proc/self/mounts of=/dev/stdout 2023-11-26T07:21:49.784 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,relatime,size=16354672k,nr_inodes=4088668,mode=755 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3275384k,mode=755 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=3455 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysusers.service ramfs ro,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3275380k,nr_inodes=818845,mode=700,uid=1000,gid=1271 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3275384k,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2023-11-26T07:21:49.784 DEBUG:teuthology.orchestra.run.smithi188:> set -ex 2023-11-26T07:21:49.784 DEBUG:teuthology.orchestra.run.smithi188:> dd if=/proc/self/mounts of=/dev/stdout 2023-11-26T07:21:49.794 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,relatime,size=16354628k,nr_inodes=4088657,mode=755 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3275376k,mode=755 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=787 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysusers.service ramfs ro,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3275372k,nr_inodes=818843,mode=700,uid=1000,gid=1271 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3275376k,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2023-11-26T07:21:49.794 DEBUG:teuthology.orchestra.run.smithi143:> set -ex 2023-11-26T07:21:49.794 DEBUG:teuthology.orchestra.run.smithi143:> dd if=/proc/self/mounts of=/dev/stdout 2023-11-26T07:21:49.836 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,relatime,size=16354672k,nr_inodes=4088668,mode=755 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3275384k,mode=755 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=3455 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysusers.service ramfs ro,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3275380k,nr_inodes=818845,mode=700,uid=1000,gid=1271 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3275384k,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2023-11-26T07:21:49.836 DEBUG:teuthology.orchestra.run.smithi188:> set -ex 2023-11-26T07:21:49.836 DEBUG:teuthology.orchestra.run.smithi188:> dd if=/proc/self/mounts of=/dev/stdout 2023-11-26T07:21:49.845 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,relatime,size=16354628k,nr_inodes=4088657,mode=755 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3275376k,mode=755 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=787 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysusers.service ramfs ro,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3275372k,nr_inodes=818843,mode=700,uid=1000,gid=1271 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3275376k,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2023-11-26T07:21:49.845 INFO:tasks.cephfs.cephfs_test_case:[, , , ] 2023-11-26T07:21:49.845 DEBUG:teuthology.orchestra.run.smithi143:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-11-26T07:21:50.016 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:21:50.012+0000 7f2d07535640 1 Processor -- start 2023-11-26T07:21:50.016 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:21:50.016+0000 7f2d07535640 1 -- start start 2023-11-26T07:21:50.018 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:21:50.016+0000 7f2d07535640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f2d0015d6e0 0x7f2d000d0df0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:21:50.019 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:21:50.016+0000 7f2d07535640 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7f2d000b2160 0x7f2d000d14b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:21:50.019 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:21:50.016+0000 7f2d07535640 1 -- --> v1:172.21.15.143:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2d000cf0e0 con 0x7f2d000d19f0 2023-11-26T07:21:50.019 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:21:50.016+0000 7f2d07535640 1 -- --> v2:172.21.15.143:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2d0015dac0 con 0x7f2d000b2160 2023-11-26T07:21:50.019 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:21:50.016+0000 7f2d07535640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2d000d1e80 con 0x7f2d0015d6e0 2023-11-26T07:21:50.019 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:21:50.016+0000 7f2d04ad0640 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7f2d000b2160 0x7f2d000d14b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:21:50.019 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:21:50.016+0000 7f2d05ad2640 1 --1- >> v1:172.21.15.143:6789/0 conn(0x7f2d000d19f0 0x7f2d000cfbf0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.143:6789/0 says I am v1:172.21.15.143:52736/0 (socket says 172.21.15.143:52736) 2023-11-26T07:21:50.019 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:21:50.016+0000 7f2d05ad2640 1 -- 172.21.15.143:0/436517970 learned_addr learned my addr 172.21.15.143:0/436517970 (peer_addr_for_me v1:172.21.15.143:0/0) 2023-11-26T07:21:50.020 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:21:50.016+0000 7f2d052d1640 1 --2- 172.21.15.143:0/436517970 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f2d0015d6e0 0x7f2d000d0df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:21:50.020 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:21:50.016+0000 7f2d04ad0640 1 -- 172.21.15.143:0/436517970 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f2d0015d6e0 msgr2=0x7f2d000d0df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-26T07:21:50.020 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:21:50.016+0000 7f2d04ad0640 1 --2- 172.21.15.143:0/436517970 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f2d0015d6e0 0x7f2d000d0df0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:21:50.020 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:21:50.016+0000 7f2d04ad0640 1 -- 172.21.15.143:0/436517970 >> v1:172.21.15.143:6789/0 conn(0x7f2d000d19f0 legacy=0x7f2d000cfbf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:21:50.020 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:21:50.016+0000 7f2d04ad0640 1 -- 172.21.15.143:0/436517970 --> v2:172.21.15.143:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2d000ceb20 con 0x7f2d000b2160 2023-11-26T07:21:50.020 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:21:50.016+0000 7f2cf7fff640 1 -- 172.21.15.143:0/436517970 <== mon.0 v1:172.21.15.143:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 238569648 0 0) 0x7f2d000cf0e0 con 0x7f2d000d19f0 2023-11-26T07:21:50.020 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:21:50.016+0000 7f2d052d1640 1 --2- 172.21.15.143:0/436517970 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f2d0015d6e0 0x7f2d000d0df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-26T07:21:50.022 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:21:50.020+0000 7f2d04ad0640 1 --2- 172.21.15.143:0/436517970 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f2d000b2160 0x7f2d000d14b0 secure :-1 s=READY pgs=766 cs=0 l=1 rev1=1 crypto rx=0x7f2cf00097c0 tx=0x7f2cf001c480 comp rx=0 tx=0).ready entity=mon.0 client_cookie=12b49322cf7dc5f5 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:21:50.022 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:21:50.020+0000 7f2cf7fff640 1 -- 172.21.15.143:0/436517970 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2cf00ab600 con 0x7f2d000b2160 2023-11-26T07:21:50.022 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:21:50.020+0000 7f2cf7fff640 1 -- 172.21.15.143:0/436517970 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2cf00a7050 con 0x7f2d000b2160 2023-11-26T07:21:50.022 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:21:50.020+0000 7f2cf7fff640 1 -- 172.21.15.143:0/436517970 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2cf00ab980 con 0x7f2d000b2160 2023-11-26T07:21:50.023 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:21:50.020+0000 7f2d07535640 1 -- 172.21.15.143:0/436517970 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f2d000b2160 msgr2=0x7f2d000d14b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:21:50.023 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:21:50.020+0000 7f2d07535640 1 --2- 172.21.15.143:0/436517970 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f2d000b2160 0x7f2d000d14b0 secure :-1 s=READY pgs=766 cs=0 l=1 rev1=1 crypto rx=0x7f2cf00097c0 tx=0x7f2cf001c480 comp rx=0 tx=0).stop 2023-11-26T07:21:50.023 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:21:50.020+0000 7f2d07535640 1 -- 172.21.15.143:0/436517970 shutdown_connections 2023-11-26T07:21:50.023 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:21:50.020+0000 7f2d07535640 1 --2- 172.21.15.143:0/436517970 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f2d000b2160 0x7f2d000d14b0 unknown :-1 s=CLOSED pgs=766 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:21:50.023 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:21:50.020+0000 7f2d07535640 1 --2- 172.21.15.143:0/436517970 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f2d0015d6e0 0x7f2d000d0df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:21:50.024 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:21:50.020+0000 7f2d07535640 1 -- 172.21.15.143:0/436517970 >> 172.21.15.143:0/436517970 conn(0x7f2d0009c550 msgr2=0x7f2d000d3220 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:21:50.024 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:21:50.020+0000 7f2d07535640 1 -- 172.21.15.143:0/436517970 shutdown_connections 2023-11-26T07:21:50.024 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:21:50.020+0000 7f2d07535640 1 -- 172.21.15.143:0/436517970 wait complete. 2023-11-26T07:21:50.025 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:21:50.024+0000 7f2d07535640 1 Processor -- start 2023-11-26T07:21:50.026 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:21:50.024+0000 7f2d07535640 1 -- start start 2023-11-26T07:21:50.028 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:21:50.024+0000 7f2d07535640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f2d000b2160 0x7f2d00151690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:21:50.028 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:21:50.024+0000 7f2d07535640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f2d000d19f0 0x7f2d000c80c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:21:50.028 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:21:50.024+0000 7f2d07535640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f2d0015d6e0 0x7f2d000c8600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:21:50.028 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:21:50.024+0000 7f2d07535640 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2d000c78f0 con 0x7f2d000b2160 2023-11-26T07:21:50.029 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:21:50.024+0000 7f2d07535640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2d000c94b0 con 0x7f2d0015d6e0 2023-11-26T07:21:50.029 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:21:50.024+0000 7f2d07535640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2d000c8c10 con 0x7f2d000d19f0 2023-11-26T07:21:50.029 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:21:50.024+0000 7f2d052d1640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f2d000b2160 0x7f2d00151690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:21:50.029 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:21:50.024+0000 7f2d052d1640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f2d000b2160 0x7f2d00151690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:49354/0 (socket says 172.21.15.143:49354) 2023-11-26T07:21:50.029 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:21:50.024+0000 7f2d052d1640 1 -- 172.21.15.143:0/1835156217 learned_addr learned my addr 172.21.15.143:0/1835156217 (peer_addr_for_me v2:172.21.15.143:0/0) 2023-11-26T07:21:50.029 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:21:50.024+0000 7f2d052d1640 1 -- 172.21.15.143:0/1835156217 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f2d000d19f0 msgr2=0x7f2d000c80c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:21:50.029 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:21:50.024+0000 7f2d052d1640 1 --2- 172.21.15.143:0/1835156217 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f2d000d19f0 0x7f2d000c80c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:21:50.029 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:21:50.024+0000 7f2d052d1640 1 -- 172.21.15.143:0/1835156217 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f2d0015d6e0 msgr2=0x7f2d000c8600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:21:50.029 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:21:50.024+0000 7f2d052d1640 1 --2- 172.21.15.143:0/1835156217 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f2d0015d6e0 0x7f2d000c8600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:21:50.029 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:21:50.024+0000 7f2d052d1640 1 -- 172.21.15.143:0/1835156217 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2cf0009420 con 0x7f2d000b2160 2023-11-26T07:21:50.029 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:21:50.028+0000 7f2d052d1640 1 --2- 172.21.15.143:0/1835156217 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f2d000b2160 0x7f2d00151690 secure :-1 s=READY pgs=767 cs=0 l=1 rev1=1 crypto rx=0x7f2cfc00c7f0 tx=0x7f2cfc00ccb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:21:50.030 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:21:50.028+0000 7f2cf5ffb640 1 -- 172.21.15.143:0/1835156217 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2cfc007820 con 0x7f2d000b2160 2023-11-26T07:21:50.030 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:21:50.028+0000 7f2cf5ffb640 1 -- 172.21.15.143:0/1835156217 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2cfc007980 con 0x7f2d000b2160 2023-11-26T07:21:50.030 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:21:50.028+0000 7f2d07535640 1 -- 172.21.15.143:0/1835156217 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2d000c8ef0 con 0x7f2d000b2160 2023-11-26T07:21:50.031 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:21:50.028+0000 7f2cf5ffb640 1 -- 172.21.15.143:0/1835156217 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2cfc007d50 con 0x7f2d000b2160 2023-11-26T07:21:50.031 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:21:50.028+0000 7f2d07535640 1 -- 172.21.15.143:0/1835156217 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2d000b5f40 con 0x7f2d000b2160 2023-11-26T07:21:50.031 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:21:50.028+0000 7f2d07535640 1 -- 172.21.15.143:0/1835156217 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2d000b6130 con 0x7f2d000b2160 2023-11-26T07:21:50.031 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:21:50.028+0000 7f2cf5ffb640 1 -- 172.21.15.143:0/1835156217 <== mon.0 v2:172.21.15.143:3300/0 4 ==== mgrmap(e 9) v1 ==== 86005+0+0 (secure 0 0 0) 0x7f2cfc0113f0 con 0x7f2d000b2160 2023-11-26T07:21:50.032 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:21:50.028+0000 7f2cf5ffb640 1 --2- 172.21.15.143:0/1835156217 >> [v2:172.21.15.143:6832/2150557544,v1:172.21.15.143:6833/2150557544] conn(0x7f2ce4066620 0x7f2ce4068ae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:21:50.032 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:21:50.032+0000 7f2cf5ffb640 1 -- 172.21.15.143:0/1835156217 <== mon.0 v2:172.21.15.143:3300/0 5 ==== osd_map(50..50 src has 1..50) v4 ==== 4943+0+0 (secure 0 0 0) 0x7f2cfc084300 con 0x7f2d000b2160 2023-11-26T07:21:50.032 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:21:50.032+0000 7f2d04ad0640 1 --2- 172.21.15.143:0/1835156217 >> [v2:172.21.15.143:6832/2150557544,v1:172.21.15.143:6833/2150557544] conn(0x7f2ce4066620 0x7f2ce4068ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:21:50.033 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:21:50.032+0000 7f2d04ad0640 1 --2- 172.21.15.143:0/1835156217 >> [v2:172.21.15.143:6832/2150557544,v1:172.21.15.143:6833/2150557544] conn(0x7f2ce4066620 0x7f2ce4068ae0 secure :-1 s=READY pgs=365 cs=0 l=1 rev1=1 crypto rx=0x7f2cf00ad000 tx=0x7f2cf0004740 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:21:50.036 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:21:50.032+0000 7f2cf5ffb640 1 -- 172.21.15.143:0/1835156217 <== mon.0 v2:172.21.15.143:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172649 (secure 0 0 0) 0x7f2cfc0516e0 con 0x7f2d000b2160 2023-11-26T07:21:50.218 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:21:50.216+0000 7f2d07535640 1 -- 172.21.15.143:0/1835156217 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f2d000d2d00 con 0x7f2d000b2160 2023-11-26T07:21:50.218 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:21:50.216+0000 7f2cf5ffb640 1 -- 172.21.15.143:0/1835156217 <== mon.0 v2:172.21.15.143:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 28 v28) v1 ==== 94+0+3708 (secure 0 0 0) 0x7f2cfc01fe90 con 0x7f2d000b2160 2023-11-26T07:21:50.219 INFO:teuthology.orchestra.run.smithi143.stdout: 2023-11-26T07:21:50.219 INFO:teuthology.orchestra.run.smithi143.stdout:{"epoch":28,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5201,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.188:6835/319464848","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6833","nonce":319464848},{"type":"v1","addr":"172.21.15.188:6835","nonce":319464848}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":20},{"gid":5584,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.188:6834/2113408070","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6832","nonce":2113408070},{"type":"v1","addr":"172.21.15.188:6834","nonce":2113408070}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":20},{"gid":5745,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.143:6835/1616742593","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6834","nonce":1616742593},{"type":"v1","addr":"172.21.15.143:6835","nonce":1616742593}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":20},{"gid":6327,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.143:6837/1891955455","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6836","nonce":1891955455},{"type":"v1","addr":"172.21.15.143:6837","nonce":1891955455}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":28}],"filesystems":[]} 2023-11-26T07:21:50.219 INFO:teuthology.orchestra.run.smithi143.stderr:dumped fsmap epoch 28 2023-11-26T07:21:50.221 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:21:50.220+0000 7f2d07535640 1 -- 172.21.15.143:0/1835156217 >> [v2:172.21.15.143:6832/2150557544,v1:172.21.15.143:6833/2150557544] conn(0x7f2ce4066620 msgr2=0x7f2ce4068ae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:21:50.221 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:21:50.220+0000 7f2d07535640 1 --2- 172.21.15.143:0/1835156217 >> [v2:172.21.15.143:6832/2150557544,v1:172.21.15.143:6833/2150557544] conn(0x7f2ce4066620 0x7f2ce4068ae0 secure :-1 s=READY pgs=365 cs=0 l=1 rev1=1 crypto rx=0x7f2cf00ad000 tx=0x7f2cf0004740 comp rx=0 tx=0).stop 2023-11-26T07:21:50.221 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:21:50.220+0000 7f2d07535640 1 -- 172.21.15.143:0/1835156217 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f2d000b2160 msgr2=0x7f2d00151690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:21:50.221 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:21:50.220+0000 7f2d07535640 1 --2- 172.21.15.143:0/1835156217 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f2d000b2160 0x7f2d00151690 secure :-1 s=READY pgs=767 cs=0 l=1 rev1=1 crypto rx=0x7f2cfc00c7f0 tx=0x7f2cfc00ccb0 comp rx=0 tx=0).stop 2023-11-26T07:21:50.221 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:21:50.220+0000 7f2d07535640 1 -- 172.21.15.143:0/1835156217 shutdown_connections 2023-11-26T07:21:50.221 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:21:50.220+0000 7f2d07535640 1 --2- 172.21.15.143:0/1835156217 >> [v2:172.21.15.143:6832/2150557544,v1:172.21.15.143:6833/2150557544] conn(0x7f2ce4066620 0x7f2ce4068ae0 unknown :-1 s=CLOSED pgs=365 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:21:50.221 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:21:50.220+0000 7f2d07535640 1 --2- 172.21.15.143:0/1835156217 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f2d0015d6e0 0x7f2d000c8600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:21:50.221 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:21:50.220+0000 7f2d07535640 1 --2- 172.21.15.143:0/1835156217 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f2d000d19f0 0x7f2d000c80c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:21:50.222 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:21:50.220+0000 7f2d07535640 1 --2- 172.21.15.143:0/1835156217 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f2d000b2160 0x7f2d00151690 unknown :-1 s=CLOSED pgs=767 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:21:50.222 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:21:50.220+0000 7f2d07535640 1 -- 172.21.15.143:0/1835156217 >> 172.21.15.143:0/1835156217 conn(0x7f2d0009c550 msgr2=0x7f2d0015c060 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:21:50.222 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:21:50.220+0000 7f2d07535640 1 -- 172.21.15.143:0/1835156217 shutdown_connections 2023-11-26T07:21:50.222 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:21:50.220+0000 7f2d07535640 1 -- 172.21.15.143:0/1835156217 wait complete. 2023-11-26T07:21:50.240 INFO:tasks.ceph.mds.a:Restarting daemon 2023-11-26T07:21:50.240 INFO:tasks.ceph.mds.a:Stopping old one... 2023-11-26T07:21:50.240 DEBUG:tasks.ceph.mds.a:waiting for process to exit 2023-11-26T07:21:50.241 INFO:teuthology.orchestra.run:waiting for 300 2023-11-26T07:21:50.241 INFO:tasks.ceph.mds.c:Restarting daemon 2023-11-26T07:21:50.241 INFO:tasks.ceph.mds.c:Stopping old one... 2023-11-26T07:21:50.241 DEBUG:tasks.ceph.mds.c:waiting for process to exit 2023-11-26T07:21:50.241 INFO:teuthology.orchestra.run:waiting for 300 2023-11-26T07:21:50.241 INFO:tasks.ceph.mds.b:Restarting daemon 2023-11-26T07:21:50.242 INFO:tasks.ceph.mds.b:Stopping old one... 2023-11-26T07:21:50.242 DEBUG:tasks.ceph.mds.b:waiting for process to exit 2023-11-26T07:21:50.242 INFO:teuthology.orchestra.run:waiting for 300 2023-11-26T07:21:50.242 INFO:tasks.ceph.mds.d:Restarting daemon 2023-11-26T07:21:50.242 INFO:tasks.ceph.mds.d:Stopping old one... 2023-11-26T07:21:50.242 DEBUG:tasks.ceph.mds.d:waiting for process to exit 2023-11-26T07:21:50.242 INFO:teuthology.orchestra.run:waiting for 300 2023-11-26T07:21:50.254 INFO:tasks.ceph.mds.a:Stopped 2023-11-26T07:21:50.254 DEBUG:teuthology.orchestra.run.smithi143:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i a 2023-11-26T07:21:50.256 INFO:tasks.ceph.mds.c:Stopped 2023-11-26T07:21:50.256 DEBUG:teuthology.orchestra.run.smithi143:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i c 2023-11-26T07:21:50.263 INFO:tasks.ceph.mds.d:Stopped 2023-11-26T07:21:50.263 DEBUG:teuthology.orchestra.run.smithi188:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i d 2023-11-26T07:21:50.265 INFO:tasks.ceph.mds.b:Stopped 2023-11-26T07:21:50.265 DEBUG:teuthology.orchestra.run.smithi188:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i b 2023-11-26T07:21:50.298 INFO:tasks.ceph.mds.a:Started 2023-11-26T07:21:50.299 DEBUG:teuthology.parallel:result is None 2023-11-26T07:21:50.301 INFO:tasks.ceph.mds.c:Started 2023-11-26T07:21:50.301 DEBUG:teuthology.parallel:result is None 2023-11-26T07:21:50.308 INFO:tasks.ceph.mds.d:Started 2023-11-26T07:21:50.308 DEBUG:teuthology.parallel:result is None 2023-11-26T07:21:50.309 INFO:tasks.ceph.mds.b:Started 2023-11-26T07:21:50.310 DEBUG:teuthology.parallel:result is None 2023-11-26T07:21:50.310 DEBUG:teuthology.orchestra.run.smithi143:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist clear 2023-11-26T07:21:50.387 INFO:tasks.ceph.mds.a.smithi143.stdout:starting mds.a at 2023-11-26T07:21:50.391 INFO:tasks.ceph.mds.c.smithi143.stdout:starting mds.c at 2023-11-26T07:21:50.400 INFO:tasks.ceph.mds.d.smithi188.stdout:starting mds.d at 2023-11-26T07:21:50.400 INFO:tasks.ceph.mds.b.smithi188.stdout:starting mds.b at 2023-11-26T07:21:50.458 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:21:50.456+0000 7fd0fda32640 1 Processor -- start 2023-11-26T07:21:50.459 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:21:50.456+0000 7fd0fda32640 1 -- start start 2023-11-26T07:21:50.461 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:21:50.456+0000 7fd0fda32640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fd0f813e290 0x7fd0f80cf280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:21:50.461 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:21:50.456+0000 7fd0fda32640 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7fd0f815f6b0 0x7fd0f80cd7b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:21:50.461 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:21:50.456+0000 7fd0fda32640 1 -- --> v1:172.21.15.143:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd0f806c520 con 0x7fd0f80cf7c0 2023-11-26T07:21:50.461 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:21:50.456+0000 7fd0fda32640 1 -- --> v2:172.21.15.143:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd0f80cfc40 con 0x7fd0f815f6b0 2023-11-26T07:21:50.461 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:21:50.456+0000 7fd0fda32640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd0f815fb40 con 0x7fd0f813e290 2023-11-26T07:21:50.461 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:21:50.456+0000 7fd0effff640 1 --1- >> v1:172.21.15.143:6789/0 conn(0x7fd0f80cf7c0 0x7fd0f815efa0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.143:6789/0 says I am v1:172.21.15.143:52756/0 (socket says 172.21.15.143:52756) 2023-11-26T07:21:50.461 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:21:50.456+0000 7fd0effff640 1 -- 172.21.15.143:0/3941331476 learned_addr learned my addr 172.21.15.143:0/3941331476 (peer_addr_for_me v1:172.21.15.143:0/0) 2023-11-26T07:21:50.461 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:21:50.456+0000 7fd0f77fe640 1 --2- 172.21.15.143:0/3941331476 >> v2:172.21.15.143:3300/0 conn(0x7fd0f815f6b0 0x7fd0f80cd7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:21:50.461 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:21:50.460+0000 7fd0f6ffd640 1 --2- 172.21.15.143:0/3941331476 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fd0f813e290 0x7fd0f80cf280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:21:50.462 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:21:50.460+0000 7fd0f67fc640 1 -- 172.21.15.143:0/3941331476 <== mon.0 v1:172.21.15.143:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3889791399 0 0) 0x7fd0f806c520 con 0x7fd0f80cf7c0 2023-11-26T07:21:50.462 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:21:50.460+0000 7fd0f67fc640 1 -- 172.21.15.143:0/3941331476 --> v1:172.21.15.143:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fd0e4003530 con 0x7fd0f80cf7c0 2023-11-26T07:21:50.462 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:21:50.460+0000 7fd0f77fe640 1 -- 172.21.15.143:0/3941331476 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fd0f813e290 msgr2=0x7fd0f80cf280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-26T07:21:50.462 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:21:50.460+0000 7fd0f77fe640 1 --2- 172.21.15.143:0/3941331476 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fd0f813e290 0x7fd0f80cf280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:21:50.462 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:21:50.460+0000 7fd0f77fe640 1 -- 172.21.15.143:0/3941331476 >> v1:172.21.15.143:6789/0 conn(0x7fd0f80cf7c0 legacy=0x7fd0f815efa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:21:50.462 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:21:50.460+0000 7fd0f77fe640 1 -- 172.21.15.143:0/3941331476 --> v2:172.21.15.143:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd0f806c800 con 0x7fd0f815f6b0 2023-11-26T07:21:50.462 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:21:50.460+0000 7fd0f6ffd640 1 --2- 172.21.15.143:0/3941331476 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fd0f813e290 0x7fd0f80cf280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-26T07:21:50.464 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:21:50.460+0000 7fd0f77fe640 1 --2- 172.21.15.143:0/3941331476 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fd0f815f6b0 0x7fd0f80cd7b0 secure :-1 s=READY pgs=772 cs=0 l=1 rev1=1 crypto rx=0x7fd0f0009d90 tx=0x7fd0f001c920 comp rx=0 tx=0).ready entity=mon.0 client_cookie=41a098a0268a5c0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:21:50.464 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:21:50.460+0000 7fd0f67fc640 1 -- 172.21.15.143:0/3941331476 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd0f00a59e0 con 0x7fd0f815f6b0 2023-11-26T07:21:50.464 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:21:50.460+0000 7fd0f67fc640 1 -- 172.21.15.143:0/3941331476 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd0f00a5b40 con 0x7fd0f815f6b0 2023-11-26T07:21:50.464 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:21:50.460+0000 7fd0f67fc640 1 -- 172.21.15.143:0/3941331476 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd0f00ad440 con 0x7fd0f815f6b0 2023-11-26T07:21:50.465 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:21:50.464+0000 7fd0fda32640 1 -- 172.21.15.143:0/3941331476 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fd0f815f6b0 msgr2=0x7fd0f80cd7b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:21:50.465 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:21:50.464+0000 7fd0fda32640 1 --2- 172.21.15.143:0/3941331476 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fd0f815f6b0 0x7fd0f80cd7b0 secure :-1 s=READY pgs=772 cs=0 l=1 rev1=1 crypto rx=0x7fd0f0009d90 tx=0x7fd0f001c920 comp rx=0 tx=0).stop 2023-11-26T07:21:50.465 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:21:50.464+0000 7fd0fda32640 1 -- 172.21.15.143:0/3941331476 shutdown_connections 2023-11-26T07:21:50.465 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:21:50.464+0000 7fd0fda32640 1 --2- 172.21.15.143:0/3941331476 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fd0f815f6b0 0x7fd0f80cd7b0 unknown :-1 s=CLOSED pgs=772 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:21:50.465 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:21:50.464+0000 7fd0fda32640 1 --2- 172.21.15.143:0/3941331476 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fd0f813e290 0x7fd0f80cf280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:21:50.466 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:21:50.464+0000 7fd0fda32640 1 -- 172.21.15.143:0/3941331476 >> 172.21.15.143:0/3941331476 conn(0x7fd0f806a890 msgr2=0x7fd0f806aca0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:21:50.466 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:21:50.464+0000 7fd0fda32640 1 -- 172.21.15.143:0/3941331476 shutdown_connections 2023-11-26T07:21:50.466 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:21:50.464+0000 7fd0fda32640 1 -- 172.21.15.143:0/3941331476 wait complete. 2023-11-26T07:21:50.466 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:21:50.464+0000 7fd0fda32640 1 Processor -- start 2023-11-26T07:21:50.466 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:21:50.464+0000 7fd0fda32640 1 -- start start 2023-11-26T07:21:50.468 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:21:50.464+0000 7fd0fda32640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fd0f80cf7c0 0x7fd0f8137440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:21:50.468 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:21:50.464+0000 7fd0fda32640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fd0f813e290 0x7fd0f815b510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:21:50.468 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:21:50.464+0000 7fd0fda32640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fd0f815f6b0 0x7fd0f815d7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:21:50.469 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:21:50.464+0000 7fd0fda32640 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd0f814ff00 con 0x7fd0f815f6b0 2023-11-26T07:21:50.469 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:21:50.464+0000 7fd0fda32640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd0f815c3d0 con 0x7fd0f80cf7c0 2023-11-26T07:21:50.469 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:21:50.464+0000 7fd0fda32640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd0f814f910 con 0x7fd0f813e290 2023-11-26T07:21:50.469 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:21:50.464+0000 7fd0f77fe640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fd0f815f6b0 0x7fd0f815d7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:21:50.469 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:21:50.464+0000 7fd0f77fe640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fd0f815f6b0 0x7fd0f815d7a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:49428/0 (socket says 172.21.15.143:49428) 2023-11-26T07:21:50.469 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:21:50.464+0000 7fd0f77fe640 1 -- 172.21.15.143:0/3145583966 learned_addr learned my addr 172.21.15.143:0/3145583966 (peer_addr_for_me v2:172.21.15.143:0/0) 2023-11-26T07:21:50.469 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:21:50.464+0000 7fd0f6ffd640 1 --2- 172.21.15.143:0/3145583966 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fd0f80cf7c0 0x7fd0f8137440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:21:50.469 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:21:50.464+0000 7fd0effff640 1 --2- 172.21.15.143:0/3145583966 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fd0f813e290 0x7fd0f815b510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:21:50.469 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:21:50.464+0000 7fd0f77fe640 1 -- 172.21.15.143:0/3145583966 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fd0f813e290 msgr2=0x7fd0f815b510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:21:50.469 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:21:50.464+0000 7fd0f77fe640 1 --2- 172.21.15.143:0/3145583966 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fd0f813e290 0x7fd0f815b510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:21:50.469 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:21:50.464+0000 7fd0f77fe640 1 -- 172.21.15.143:0/3145583966 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fd0f80cf7c0 msgr2=0x7fd0f8137440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:21:50.469 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:21:50.464+0000 7fd0f77fe640 1 --2- 172.21.15.143:0/3145583966 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fd0f80cf7c0 0x7fd0f8137440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:21:50.470 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:21:50.464+0000 7fd0f77fe640 1 -- 172.21.15.143:0/3145583966 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd0f0009a70 con 0x7fd0f815f6b0 2023-11-26T07:21:50.470 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:21:50.468+0000 7fd0f77fe640 1 --2- 172.21.15.143:0/3145583966 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fd0f815f6b0 0x7fd0f815d7a0 secure :-1 s=READY pgs=773 cs=0 l=1 rev1=1 crypto rx=0x7fd0f0009d90 tx=0x7fd0f00196c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:21:50.470 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:21:50.468+0000 7fd0f4ff9640 1 -- 172.21.15.143:0/3145583966 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd0f00add90 con 0x7fd0f815f6b0 2023-11-26T07:21:50.471 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:21:50.468+0000 7fd0f4ff9640 1 -- 172.21.15.143:0/3145583966 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd0f0009450 con 0x7fd0f815f6b0 2023-11-26T07:21:50.471 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:21:50.468+0000 7fd0fda32640 1 -- 172.21.15.143:0/3145583966 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd0f815be10 con 0x7fd0f815f6b0 2023-11-26T07:21:50.471 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:21:50.468+0000 7fd0f4ff9640 1 -- 172.21.15.143:0/3145583966 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd0f00b9690 con 0x7fd0f815f6b0 2023-11-26T07:21:50.471 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:21:50.468+0000 7fd0fda32640 1 -- 172.21.15.143:0/3145583966 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd0f814fb20 con 0x7fd0f815f6b0 2023-11-26T07:21:50.471 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:21:50.468+0000 7fd0fda32640 1 -- 172.21.15.143:0/3145583966 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd0f8150040 con 0x7fd0f815f6b0 2023-11-26T07:21:50.471 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:21:50.468+0000 7fd0f4ff9640 1 -- 172.21.15.143:0/3145583966 <== mon.0 v2:172.21.15.143:3300/0 4 ==== mgrmap(e 9) v1 ==== 86005+0+0 (secure 0 0 0) 0x7fd0f00039a0 con 0x7fd0f815f6b0 2023-11-26T07:21:50.472 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:21:50.468+0000 7fd0f4ff9640 1 --2- 172.21.15.143:0/3145583966 >> [v2:172.21.15.143:6832/2150557544,v1:172.21.15.143:6833/2150557544] conn(0x7fd0d8066650 0x7fd0d8068b10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:21:50.472 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:21:50.468+0000 7fd0f4ff9640 1 -- 172.21.15.143:0/3145583966 <== mon.0 v2:172.21.15.143:3300/0 5 ==== osd_map(50..50 src has 1..50) v4 ==== 4943+0+0 (secure 0 0 0) 0x7fd0f011fa50 con 0x7fd0f815f6b0 2023-11-26T07:21:50.472 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:21:50.468+0000 7fd0f6ffd640 1 --2- 172.21.15.143:0/3145583966 >> [v2:172.21.15.143:6832/2150557544,v1:172.21.15.143:6833/2150557544] conn(0x7fd0d8066650 0x7fd0d8068b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:21:50.473 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:21:50.472+0000 7fd0f6ffd640 1 --2- 172.21.15.143:0/3145583966 >> [v2:172.21.15.143:6832/2150557544,v1:172.21.15.143:6833/2150557544] conn(0x7fd0d8066650 0x7fd0d8068b10 secure :-1 s=READY pgs=366 cs=0 l=1 rev1=1 crypto rx=0x7fd0e8009950 tx=0x7fd0e8008040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:21:50.475 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:21:50.472+0000 7fd0f4ff9640 1 -- 172.21.15.143:0/3145583966 <== mon.0 v2:172.21.15.143:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172649 (secure 0 0 0) 0x7fd0f00ecea0 con 0x7fd0f815f6b0 2023-11-26T07:21:50.644 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:21:50.640+0000 7fd0fda32640 1 -- 172.21.15.143:0/3145583966 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "osd blocklist clear"} v 0) v1 -- 0x7fd0f8150040 con 0x7fd0f815f6b0 2023-11-26T07:21:51.618 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:21:51.616+0000 7fd0f4ff9640 1 -- 172.21.15.143:0/3145583966 <== mon.0 v2:172.21.15.143:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist clear"}]=0 removed all blocklist entries v55) v1 ==== 97+0+0 (secure 0 0 0) 0x7fd0f8150040 con 0x7fd0f815f6b0 2023-11-26T07:21:51.618 INFO:teuthology.orchestra.run.smithi143.stderr: removed all blocklist entries 2023-11-26T07:21:51.622 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:21:51.620+0000 7fd0fda32640 1 -- 172.21.15.143:0/3145583966 >> [v2:172.21.15.143:6832/2150557544,v1:172.21.15.143:6833/2150557544] conn(0x7fd0d8066650 msgr2=0x7fd0d8068b10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:21:51.622 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:21:51.620+0000 7fd0fda32640 1 --2- 172.21.15.143:0/3145583966 >> [v2:172.21.15.143:6832/2150557544,v1:172.21.15.143:6833/2150557544] conn(0x7fd0d8066650 0x7fd0d8068b10 secure :-1 s=READY pgs=366 cs=0 l=1 rev1=1 crypto rx=0x7fd0e8009950 tx=0x7fd0e8008040 comp rx=0 tx=0).stop 2023-11-26T07:21:51.622 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:21:51.620+0000 7fd0fda32640 1 -- 172.21.15.143:0/3145583966 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fd0f815f6b0 msgr2=0x7fd0f815d7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:21:51.622 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:21:51.620+0000 7fd0fda32640 1 --2- 172.21.15.143:0/3145583966 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fd0f815f6b0 0x7fd0f815d7a0 secure :-1 s=READY pgs=773 cs=0 l=1 rev1=1 crypto rx=0x7fd0f0009d90 tx=0x7fd0f00196c0 comp rx=0 tx=0).stop 2023-11-26T07:21:51.622 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:21:51.620+0000 7fd0fda32640 1 -- 172.21.15.143:0/3145583966 shutdown_connections 2023-11-26T07:21:51.622 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:21:51.620+0000 7fd0fda32640 1 --2- 172.21.15.143:0/3145583966 >> [v2:172.21.15.143:6832/2150557544,v1:172.21.15.143:6833/2150557544] conn(0x7fd0d8066650 0x7fd0d8068b10 unknown :-1 s=CLOSED pgs=366 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:21:51.622 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:21:51.620+0000 7fd0fda32640 1 --2- 172.21.15.143:0/3145583966 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fd0f815f6b0 0x7fd0f815d7a0 unknown :-1 s=CLOSED pgs=773 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:21:51.623 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:21:51.620+0000 7fd0fda32640 1 --2- 172.21.15.143:0/3145583966 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fd0f813e290 0x7fd0f815b510 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:21:51.623 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:21:51.620+0000 7fd0fda32640 1 --2- 172.21.15.143:0/3145583966 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fd0f80cf7c0 0x7fd0f8137440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:21:51.623 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:21:51.620+0000 7fd0fda32640 1 -- 172.21.15.143:0/3145583966 >> 172.21.15.143:0/3145583966 conn(0x7fd0f806a890 msgr2=0x7fd0f806ac70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:21:51.623 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:21:51.620+0000 7fd0fda32640 1 -- 172.21.15.143:0/3145583966 shutdown_connections 2023-11-26T07:21:51.623 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:21:51.620+0000 7fd0fda32640 1 -- 172.21.15.143:0/3145583966 wait complete. 2023-11-26T07:21:51.645 DEBUG:teuthology.orchestra.run.smithi143:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth ls --format=json-pretty 2023-11-26T07:21:51.788 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:21:51.784+0000 7fae8289c640 1 Processor -- start 2023-11-26T07:21:51.788 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:21:51.784+0000 7fae8289c640 1 -- start start 2023-11-26T07:21:51.790 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:21:51.788+0000 7fae8289c640 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7fae7c0cf280 0x7fae7c0cf710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:21:51.790 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:21:51.788+0000 7fae8289c640 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7fae7c0cd1b0 0x7fae7c0cd5c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:21:51.790 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:21:51.788+0000 7fae8289c640 1 -- --> v1:172.21.15.188:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fae7c0cc730 con 0x7fae7c143ac0 2023-11-26T07:21:51.790 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:21:51.788+0000 7fae8289c640 1 -- --> v2:172.21.15.143:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fae7c143ea0 con 0x7fae7c0cf280 2023-11-26T07:21:51.790 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:21:51.788+0000 7fae8289c640 1 -- --> v2:172.21.15.188:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fae7c0cfc50 con 0x7fae7c0cd1b0 2023-11-26T07:21:51.790 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:21:51.788+0000 7fae7b7fe640 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7fae7c0cf280 0x7fae7c0cf710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:21:51.790 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:21:51.788+0000 7fae7b7fe640 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7fae7c0cf280 0x7fae7c0cf710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:49432/0 (socket says 172.21.15.143:49432) 2023-11-26T07:21:51.791 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:21:51.788+0000 7fae7b7fe640 1 -- 172.21.15.143:0/1747808665 learned_addr learned my addr 172.21.15.143:0/1747808665 (peer_addr_for_me v2:172.21.15.143:0/0) 2023-11-26T07:21:51.791 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:21:51.788+0000 7fae7b7fe640 1 -- 172.21.15.143:0/1747808665 >> v2:172.21.15.188:3300/0 conn(0x7fae7c0cd1b0 msgr2=0x7fae7c0cd5c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-26T07:21:51.791 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:21:51.788+0000 7fae80e39640 1 --2- 172.21.15.143:0/1747808665 >> v2:172.21.15.188:3300/0 conn(0x7fae7c0cd1b0 0x7fae7c0cd5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:21:51.791 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:21:51.788+0000 7fae7b7fe640 1 --2- 172.21.15.143:0/1747808665 >> v2:172.21.15.188:3300/0 conn(0x7fae7c0cd1b0 0x7fae7c0cd5c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:21:51.791 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:21:51.788+0000 7fae7b7fe640 1 -- 172.21.15.143:0/1747808665 >> v1:172.21.15.188:6789/0 conn(0x7fae7c143ac0 legacy=0x7fae7c070370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-26T07:21:51.791 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:21:51.788+0000 7fae7b7fe640 1 -- 172.21.15.143:0/1747808665 --> v2:172.21.15.143:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fae7c070ee0 con 0x7fae7c0cf280 2023-11-26T07:21:51.791 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:21:51.788+0000 7fae80e39640 1 --2- 172.21.15.143:0/1747808665 >> v2:172.21.15.188:3300/0 conn(0x7fae7c0cd1b0 0x7fae7c0cd5c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-26T07:21:51.793 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:21:51.792+0000 7fae7b7fe640 1 --2- 172.21.15.143:0/1747808665 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fae7c0cf280 0x7fae7c0cf710 secure :-1 s=READY pgs=774 cs=0 l=1 rev1=1 crypto rx=0x7fae7c06e570 tx=0x7fae7001c4c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c5faca8890607b server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:21:51.793 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:21:51.792+0000 7fae7affd640 1 -- 172.21.15.143:0/1747808665 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fae700aa620 con 0x7fae7c0cf280 2023-11-26T07:21:51.793 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:21:51.792+0000 7fae7affd640 1 -- 172.21.15.143:0/1747808665 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fae700aa780 con 0x7fae7c0cf280 2023-11-26T07:21:51.793 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:21:51.792+0000 7fae7affd640 1 -- 172.21.15.143:0/1747808665 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fae700aab10 con 0x7fae7c0cf280 2023-11-26T07:21:51.794 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:21:51.792+0000 7fae8289c640 1 -- 172.21.15.143:0/1747808665 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fae7c0cf280 msgr2=0x7fae7c0cf710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:21:51.794 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:21:51.792+0000 7fae8289c640 1 --2- 172.21.15.143:0/1747808665 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fae7c0cf280 0x7fae7c0cf710 secure :-1 s=READY pgs=774 cs=0 l=1 rev1=1 crypto rx=0x7fae7c06e570 tx=0x7fae7001c4c0 comp rx=0 tx=0).stop 2023-11-26T07:21:51.794 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:21:51.792+0000 7fae8289c640 1 -- 172.21.15.143:0/1747808665 shutdown_connections 2023-11-26T07:21:51.794 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:21:51.792+0000 7fae8289c640 1 --2- 172.21.15.143:0/1747808665 >> v2:172.21.15.188:3300/0 conn(0x7fae7c0cd1b0 0x7fae7c0cd5c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:21:51.794 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:21:51.792+0000 7fae8289c640 1 --2- 172.21.15.143:0/1747808665 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fae7c0cf280 0x7fae7c0cf710 unknown :-1 s=CLOSED pgs=774 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:21:51.794 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:21:51.792+0000 7fae8289c640 1 -- 172.21.15.143:0/1747808665 >> 172.21.15.143:0/1747808665 conn(0x7fae7c06a890 msgr2=0x7fae7c06aca0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:21:51.795 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:21:51.792+0000 7fae8289c640 1 -- 172.21.15.143:0/1747808665 shutdown_connections 2023-11-26T07:21:51.795 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:21:51.792+0000 7fae8289c640 1 -- 172.21.15.143:0/1747808665 wait complete. 2023-11-26T07:21:51.795 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:21:51.792+0000 7fae8289c640 1 Processor -- start 2023-11-26T07:21:51.796 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:21:51.792+0000 7fae8289c640 1 -- start start 2023-11-26T07:21:51.797 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:21:51.796+0000 7fae8289c640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fae7c0cd1b0 0x7fae7c14a380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:21:51.798 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:21:51.796+0000 7fae8289c640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fae7c0cf280 0x7fae7c15ccd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:21:51.798 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:21:51.796+0000 7fae8289c640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fae7c143ac0 0x7fae7c15afe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:21:51.798 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:21:51.796+0000 7fae8289c640 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fae7c15d210 con 0x7fae7c0cf280 2023-11-26T07:21:51.798 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:21:51.796+0000 7fae8289c640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fae7c139400 con 0x7fae7c143ac0 2023-11-26T07:21:51.798 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:21:51.796+0000 7fae8289c640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fae7c1396e0 con 0x7fae7c0cd1b0 2023-11-26T07:21:51.798 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:21:51.796+0000 7fae7b7fe640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fae7c0cf280 0x7fae7c15ccd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:21:51.798 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:21:51.796+0000 7fae7b7fe640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fae7c0cf280 0x7fae7c15ccd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:49446/0 (socket says 172.21.15.143:49446) 2023-11-26T07:21:51.798 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:21:51.796+0000 7fae7b7fe640 1 -- 172.21.15.143:0/4072794391 learned_addr learned my addr 172.21.15.143:0/4072794391 (peer_addr_for_me v2:172.21.15.143:0/0) 2023-11-26T07:21:51.798 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:21:51.796+0000 7fae80e39640 1 --2- 172.21.15.143:0/4072794391 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fae7c143ac0 0x7fae7c15afe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:21:51.798 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:21:51.796+0000 7fae7bfff640 1 --2- 172.21.15.143:0/4072794391 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fae7c0cd1b0 0x7fae7c14a380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:21:51.798 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:21:51.796+0000 7fae7b7fe640 1 -- 172.21.15.143:0/4072794391 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fae7c0cd1b0 msgr2=0x7fae7c14a380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:21:51.799 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:21:51.796+0000 7fae7b7fe640 1 --2- 172.21.15.143:0/4072794391 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fae7c0cd1b0 0x7fae7c14a380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:21:51.799 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:21:51.796+0000 7fae7b7fe640 1 -- 172.21.15.143:0/4072794391 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fae7c143ac0 msgr2=0x7fae7c15afe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:21:51.799 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:21:51.796+0000 7fae7b7fe640 1 --2- 172.21.15.143:0/4072794391 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fae7c143ac0 0x7fae7c15afe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:21:51.799 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:21:51.796+0000 7fae7b7fe640 1 -- 172.21.15.143:0/4072794391 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fae700094d0 con 0x7fae7c0cf280 2023-11-26T07:21:51.799 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:21:51.796+0000 7fae7bfff640 1 --2- 172.21.15.143:0/4072794391 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fae7c0cd1b0 0x7fae7c14a380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-26T07:21:51.799 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:21:51.796+0000 7fae7b7fe640 1 --2- 172.21.15.143:0/4072794391 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fae7c0cf280 0x7fae7c15ccd0 secure :-1 s=READY pgs=775 cs=0 l=1 rev1=1 crypto rx=0x7fae7001a0f0 tx=0x7fae7001ad00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:21:51.800 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:21:51.796+0000 7fae78ff9640 1 -- 172.21.15.143:0/4072794391 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fae700bd5f0 con 0x7fae7c0cf280 2023-11-26T07:21:51.800 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:21:51.796+0000 7fae78ff9640 1 -- 172.21.15.143:0/4072794391 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fae700bd750 con 0x7fae7c0cf280 2023-11-26T07:21:51.800 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:21:51.796+0000 7fae8289c640 1 -- 172.21.15.143:0/4072794391 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fae7c1399c0 con 0x7fae7c0cf280 2023-11-26T07:21:51.800 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:21:51.796+0000 7fae78ff9640 1 -- 172.21.15.143:0/4072794391 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fae700bdb00 con 0x7fae7c0cf280 2023-11-26T07:21:51.800 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:21:51.796+0000 7fae8289c640 1 -- 172.21.15.143:0/4072794391 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fae7c139f80 con 0x7fae7c0cf280 2023-11-26T07:21:51.800 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:21:51.796+0000 7fae8289c640 1 -- 172.21.15.143:0/4072794391 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fae7c067680 con 0x7fae7c0cf280 2023-11-26T07:21:51.800 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:21:51.796+0000 7fae78ff9640 1 -- 172.21.15.143:0/4072794391 <== mon.0 v2:172.21.15.143:3300/0 4 ==== mgrmap(e 9) v1 ==== 86005+0+0 (secure 0 0 0) 0x7fae700b33f0 con 0x7fae7c0cf280 2023-11-26T07:21:51.801 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:21:51.800+0000 7fae78ff9640 1 --2- 172.21.15.143:0/4072794391 >> [v2:172.21.15.143:6832/2150557544,v1:172.21.15.143:6833/2150557544] conn(0x7fae68066620 0x7fae68068ae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:21:51.801 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:21:51.800+0000 7fae78ff9640 1 -- 172.21.15.143:0/4072794391 <== mon.0 v2:172.21.15.143:3300/0 5 ==== osd_map(55..55 src has 1..55) v4 ==== 4857+0+0 (secure 0 0 0) 0x7fae7011eeb0 con 0x7fae7c0cf280 2023-11-26T07:21:51.802 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:21:51.800+0000 7fae7bfff640 1 --2- 172.21.15.143:0/4072794391 >> [v2:172.21.15.143:6832/2150557544,v1:172.21.15.143:6833/2150557544] conn(0x7fae68066620 0x7fae68068ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:21:51.802 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:21:51.800+0000 7fae7bfff640 1 --2- 172.21.15.143:0/4072794391 >> [v2:172.21.15.143:6832/2150557544,v1:172.21.15.143:6833/2150557544] conn(0x7fae68066620 0x7fae68068ae0 secure :-1 s=READY pgs=371 cs=0 l=1 rev1=1 crypto rx=0x7fae6c003e40 tx=0x7fae6c009040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:21:51.806 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:21:51.804+0000 7fae78ff9640 1 -- 172.21.15.143:0/4072794391 <== mon.0 v2:172.21.15.143:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172649 (secure 0 0 0) 0x7fae700ec400 con 0x7fae7c0cf280 2023-11-26T07:21:52.006 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:21:52.004+0000 7fae8289c640 1 -- 172.21.15.143:0/4072794391 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "auth ls", "format": "json-pretty"} v 0) v1 -- 0x7fae7c15a370 con 0x7fae7c0cf280 2023-11-26T07:21:52.007 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:21:52.004+0000 7fae78ff9640 1 -- 172.21.15.143:0/4072794391 <== mon.0 v2:172.21.15.143:3300/0 7 ==== mon_command_ack([{"prefix": "auth ls", "format": "json-pretty"}]=0 v14) v1 ==== 80+0+5304 (secure 0 0 0) 0x7fae700efa20 con 0x7fae7c0cf280 2023-11-26T07:21:52.007 INFO:teuthology.orchestra.run.smithi143.stdout: 2023-11-26T07:21:52.007 INFO:teuthology.orchestra.run.smithi143.stdout:{ 2023-11-26T07:21:52.008 INFO:teuthology.orchestra.run.smithi143.stdout: "auth_dump": [ 2023-11-26T07:21:52.008 INFO:teuthology.orchestra.run.smithi143.stdout: { 2023-11-26T07:21:52.008 INFO:teuthology.orchestra.run.smithi143.stdout: "entity": "mds.a", 2023-11-26T07:21:52.008 INFO:teuthology.orchestra.run.smithi143.stdout: "key": "AQDl8GJldeBsOBAAhk75XZnQRURKcr0MrW17Lg==", 2023-11-26T07:21:52.008 INFO:teuthology.orchestra.run.smithi143.stdout: "caps": { 2023-11-26T07:21:52.008 INFO:teuthology.orchestra.run.smithi143.stdout: "mds": "allow", 2023-11-26T07:21:52.008 INFO:teuthology.orchestra.run.smithi143.stdout: "mgr": "allow *", 2023-11-26T07:21:52.008 INFO:teuthology.orchestra.run.smithi143.stdout: "mon": "allow *", 2023-11-26T07:21:52.008 INFO:teuthology.orchestra.run.smithi143.stdout: "osd": "allow *" 2023-11-26T07:21:52.008 INFO:teuthology.orchestra.run.smithi143.stdout: } 2023-11-26T07:21:52.008 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2023-11-26T07:21:52.008 INFO:teuthology.orchestra.run.smithi143.stdout: { 2023-11-26T07:21:52.009 INFO:teuthology.orchestra.run.smithi143.stdout: "entity": "mds.b", 2023-11-26T07:21:52.009 INFO:teuthology.orchestra.run.smithi143.stdout: "key": "AQDm8GJl0lvhCBAAXuvHKoB8TAhp0bJ4DcV94g==", 2023-11-26T07:21:52.009 INFO:teuthology.orchestra.run.smithi143.stdout: "caps": { 2023-11-26T07:21:52.009 INFO:teuthology.orchestra.run.smithi143.stdout: "mds": "allow", 2023-11-26T07:21:52.009 INFO:teuthology.orchestra.run.smithi143.stdout: "mgr": "allow *", 2023-11-26T07:21:52.009 INFO:teuthology.orchestra.run.smithi143.stdout: "mon": "allow *", 2023-11-26T07:21:52.009 INFO:teuthology.orchestra.run.smithi143.stdout: "osd": "allow *" 2023-11-26T07:21:52.009 INFO:teuthology.orchestra.run.smithi143.stdout: } 2023-11-26T07:21:52.009 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2023-11-26T07:21:52.009 INFO:teuthology.orchestra.run.smithi143.stdout: { 2023-11-26T07:21:52.009 INFO:teuthology.orchestra.run.smithi143.stdout: "entity": "mds.c", 2023-11-26T07:21:52.009 INFO:teuthology.orchestra.run.smithi143.stdout: "key": "AQDm8GJluQ0PBBAA23EmVdD3bVk52h6izUXDRQ==", 2023-11-26T07:21:52.009 INFO:teuthology.orchestra.run.smithi143.stdout: "caps": { 2023-11-26T07:21:52.010 INFO:teuthology.orchestra.run.smithi143.stdout: "mds": "allow", 2023-11-26T07:21:52.010 INFO:teuthology.orchestra.run.smithi143.stdout: "mgr": "allow *", 2023-11-26T07:21:52.010 INFO:teuthology.orchestra.run.smithi143.stdout: "mon": "allow *", 2023-11-26T07:21:52.010 INFO:teuthology.orchestra.run.smithi143.stdout: "osd": "allow *" 2023-11-26T07:21:52.010 INFO:teuthology.orchestra.run.smithi143.stdout: } 2023-11-26T07:21:52.010 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2023-11-26T07:21:52.010 INFO:teuthology.orchestra.run.smithi143.stdout: { 2023-11-26T07:21:52.010 INFO:teuthology.orchestra.run.smithi143.stdout: "entity": "mds.d", 2023-11-26T07:21:52.010 INFO:teuthology.orchestra.run.smithi143.stdout: "key": "AQDm8GJlkt9nEBAAg+LYtKVNHAN/fwYjEcNJHQ==", 2023-11-26T07:21:52.010 INFO:teuthology.orchestra.run.smithi143.stdout: "caps": { 2023-11-26T07:21:52.010 INFO:teuthology.orchestra.run.smithi143.stdout: "mds": "allow", 2023-11-26T07:21:52.010 INFO:teuthology.orchestra.run.smithi143.stdout: "mgr": "allow *", 2023-11-26T07:21:52.011 INFO:teuthology.orchestra.run.smithi143.stdout: "mon": "allow *", 2023-11-26T07:21:52.011 INFO:teuthology.orchestra.run.smithi143.stdout: "osd": "allow *" 2023-11-26T07:21:52.011 INFO:teuthology.orchestra.run.smithi143.stdout: } 2023-11-26T07:21:52.011 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2023-11-26T07:21:52.011 INFO:teuthology.orchestra.run.smithi143.stdout: { 2023-11-26T07:21:52.011 INFO:teuthology.orchestra.run.smithi143.stdout: "entity": "osd.0", 2023-11-26T07:21:52.011 INFO:teuthology.orchestra.run.smithi143.stdout: "key": "AQDp8GJlNscZLhAA31zK4vTcnxwiRIRgxzewpA==", 2023-11-26T07:21:52.011 INFO:teuthology.orchestra.run.smithi143.stdout: "caps": { 2023-11-26T07:21:52.011 INFO:teuthology.orchestra.run.smithi143.stdout: "mgr": "allow profile osd", 2023-11-26T07:21:52.011 INFO:teuthology.orchestra.run.smithi143.stdout: "mon": "allow profile osd", 2023-11-26T07:21:52.011 INFO:teuthology.orchestra.run.smithi143.stdout: "osd": "allow *" 2023-11-26T07:21:52.011 INFO:teuthology.orchestra.run.smithi143.stdout: } 2023-11-26T07:21:52.011 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2023-11-26T07:21:52.012 INFO:teuthology.orchestra.run.smithi143.stdout: { 2023-11-26T07:21:52.012 INFO:teuthology.orchestra.run.smithi143.stdout: "entity": "osd.1", 2023-11-26T07:21:52.012 INFO:teuthology.orchestra.run.smithi143.stdout: "key": "AQDs8GJlsyKUNBAA+dQ1ElGPrdP1GV7lk4JQjQ==", 2023-11-26T07:21:52.012 INFO:teuthology.orchestra.run.smithi143.stdout: "caps": { 2023-11-26T07:21:52.012 INFO:teuthology.orchestra.run.smithi143.stdout: "mgr": "allow profile osd", 2023-11-26T07:21:52.012 INFO:teuthology.orchestra.run.smithi143.stdout: "mon": "allow profile osd", 2023-11-26T07:21:52.012 INFO:teuthology.orchestra.run.smithi143.stdout: "osd": "allow *" 2023-11-26T07:21:52.012 INFO:teuthology.orchestra.run.smithi143.stdout: } 2023-11-26T07:21:52.012 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2023-11-26T07:21:52.012 INFO:teuthology.orchestra.run.smithi143.stdout: { 2023-11-26T07:21:52.012 INFO:teuthology.orchestra.run.smithi143.stdout: "entity": "osd.2", 2023-11-26T07:21:52.012 INFO:teuthology.orchestra.run.smithi143.stdout: "key": "AQDv8GJlXsy7OhAAJJdldsodgGZVtyV2HuyC4A==", 2023-11-26T07:21:52.013 INFO:teuthology.orchestra.run.smithi143.stdout: "caps": { 2023-11-26T07:21:52.013 INFO:teuthology.orchestra.run.smithi143.stdout: "mgr": "allow profile osd", 2023-11-26T07:21:52.013 INFO:teuthology.orchestra.run.smithi143.stdout: "mon": "allow profile osd", 2023-11-26T07:21:52.013 INFO:teuthology.orchestra.run.smithi143.stdout: "osd": "allow *" 2023-11-26T07:21:52.013 INFO:teuthology.orchestra.run.smithi143.stdout: } 2023-11-26T07:21:52.013 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2023-11-26T07:21:52.013 INFO:teuthology.orchestra.run.smithi143.stdout: { 2023-11-26T07:21:52.013 INFO:teuthology.orchestra.run.smithi143.stdout: "entity": "osd.3", 2023-11-26T07:21:52.013 INFO:teuthology.orchestra.run.smithi143.stdout: "key": "AQDz8GJloLhMBxAAOWD9ec2y4c8nkdJmQnne8Q==", 2023-11-26T07:21:52.013 INFO:teuthology.orchestra.run.smithi143.stdout: "caps": { 2023-11-26T07:21:52.013 INFO:teuthology.orchestra.run.smithi143.stdout: "mgr": "allow profile osd", 2023-11-26T07:21:52.013 INFO:teuthology.orchestra.run.smithi143.stdout: "mon": "allow profile osd", 2023-11-26T07:21:52.014 INFO:teuthology.orchestra.run.smithi143.stdout: "osd": "allow *" 2023-11-26T07:21:52.014 INFO:teuthology.orchestra.run.smithi143.stdout: } 2023-11-26T07:21:52.014 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2023-11-26T07:21:52.014 INFO:teuthology.orchestra.run.smithi143.stdout: { 2023-11-26T07:21:52.014 INFO:teuthology.orchestra.run.smithi143.stdout: "entity": "osd.4", 2023-11-26T07:21:52.014 INFO:teuthology.orchestra.run.smithi143.stdout: "key": "AQD58GJlDlZsDBAAZNauAPPPfAZ+KIqSmgSQhg==", 2023-11-26T07:21:52.014 INFO:teuthology.orchestra.run.smithi143.stdout: "caps": { 2023-11-26T07:21:52.014 INFO:teuthology.orchestra.run.smithi143.stdout: "mgr": "allow profile osd", 2023-11-26T07:21:52.014 INFO:teuthology.orchestra.run.smithi143.stdout: "mon": "allow profile osd", 2023-11-26T07:21:52.014 INFO:teuthology.orchestra.run.smithi143.stdout: "osd": "allow *" 2023-11-26T07:21:52.014 INFO:teuthology.orchestra.run.smithi143.stdout: } 2023-11-26T07:21:52.015 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2023-11-26T07:21:52.015 INFO:teuthology.orchestra.run.smithi143.stdout: { 2023-11-26T07:21:52.015 INFO:teuthology.orchestra.run.smithi143.stdout: "entity": "osd.5", 2023-11-26T07:21:52.015 INFO:teuthology.orchestra.run.smithi143.stdout: "key": "AQD88GJlNE4mExAA5jYkCa0tE3oR6rgK1Dt5cA==", 2023-11-26T07:21:52.015 INFO:teuthology.orchestra.run.smithi143.stdout: "caps": { 2023-11-26T07:21:52.015 INFO:teuthology.orchestra.run.smithi143.stdout: "mgr": "allow profile osd", 2023-11-26T07:21:52.015 INFO:teuthology.orchestra.run.smithi143.stdout: "mon": "allow profile osd", 2023-11-26T07:21:52.015 INFO:teuthology.orchestra.run.smithi143.stdout: "osd": "allow *" 2023-11-26T07:21:52.015 INFO:teuthology.orchestra.run.smithi143.stdout: } 2023-11-26T07:21:52.015 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2023-11-26T07:21:52.015 INFO:teuthology.orchestra.run.smithi143.stdout: { 2023-11-26T07:21:52.015 INFO:teuthology.orchestra.run.smithi143.stdout: "entity": "osd.6", 2023-11-26T07:21:52.016 INFO:teuthology.orchestra.run.smithi143.stdout: "key": "AQD/8GJlh0lYGRAArHalCyZ36UvBOVMZor+N/g==", 2023-11-26T07:21:52.016 INFO:teuthology.orchestra.run.smithi143.stdout: "caps": { 2023-11-26T07:21:52.016 INFO:teuthology.orchestra.run.smithi143.stdout: "mgr": "allow profile osd", 2023-11-26T07:21:52.016 INFO:teuthology.orchestra.run.smithi143.stdout: "mon": "allow profile osd", 2023-11-26T07:21:52.016 INFO:teuthology.orchestra.run.smithi143.stdout: "osd": "allow *" 2023-11-26T07:21:52.016 INFO:teuthology.orchestra.run.smithi143.stdout: } 2023-11-26T07:21:52.016 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2023-11-26T07:21:52.016 INFO:teuthology.orchestra.run.smithi143.stdout: { 2023-11-26T07:21:52.016 INFO:teuthology.orchestra.run.smithi143.stdout: "entity": "osd.7", 2023-11-26T07:21:52.016 INFO:teuthology.orchestra.run.smithi143.stdout: "key": "AQAC8WJlJavtIBAAKMvnR4AnnrDVOXlzN7j05w==", 2023-11-26T07:21:52.016 INFO:teuthology.orchestra.run.smithi143.stdout: "caps": { 2023-11-26T07:21:52.016 INFO:teuthology.orchestra.run.smithi143.stdout: "mgr": "allow profile osd", 2023-11-26T07:21:52.016 INFO:teuthology.orchestra.run.smithi143.stdout: "mon": "allow profile osd", 2023-11-26T07:21:52.017 INFO:teuthology.orchestra.run.smithi143.stdout: "osd": "allow *" 2023-11-26T07:21:52.017 INFO:teuthology.orchestra.run.smithi143.stdout: } 2023-11-26T07:21:52.017 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2023-11-26T07:21:52.017 INFO:teuthology.orchestra.run.smithi143.stdout: { 2023-11-26T07:21:52.017 INFO:teuthology.orchestra.run.smithi143.stdout: "entity": "client.0", 2023-11-26T07:21:52.017 INFO:teuthology.orchestra.run.smithi143.stdout: "key": "AQDm8GJlWMh9FBAAslVzi6fm4/wt7vr/rHCHHQ==", 2023-11-26T07:21:52.017 INFO:teuthology.orchestra.run.smithi143.stdout: "caps": { 2023-11-26T07:21:52.017 INFO:teuthology.orchestra.run.smithi143.stdout: "mds": "allow", 2023-11-26T07:21:52.017 INFO:teuthology.orchestra.run.smithi143.stdout: "mon": "allow r", 2023-11-26T07:21:52.017 INFO:teuthology.orchestra.run.smithi143.stdout: "osd": "allow rw tag cephfs data=cephfs" 2023-11-26T07:21:52.017 INFO:teuthology.orchestra.run.smithi143.stdout: } 2023-11-26T07:21:52.017 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2023-11-26T07:21:52.018 INFO:teuthology.orchestra.run.smithi143.stdout: { 2023-11-26T07:21:52.018 INFO:teuthology.orchestra.run.smithi143.stdout: "entity": "client.1", 2023-11-26T07:21:52.018 INFO:teuthology.orchestra.run.smithi143.stdout: "key": "AQDm8GJl5vkYHhAAgP+gKH2+lpl3aTlgqZNIXg==", 2023-11-26T07:21:52.018 INFO:teuthology.orchestra.run.smithi143.stdout: "caps": { 2023-11-26T07:21:52.018 INFO:teuthology.orchestra.run.smithi143.stdout: "mds": "allow", 2023-11-26T07:21:52.018 INFO:teuthology.orchestra.run.smithi143.stdout: "mgr": "allow r", 2023-11-26T07:21:52.018 INFO:teuthology.orchestra.run.smithi143.stdout: "mon": "allow rw", 2023-11-26T07:21:52.018 INFO:teuthology.orchestra.run.smithi143.stdout: "osd": "allow rwx" 2023-11-26T07:21:52.018 INFO:teuthology.orchestra.run.smithi143.stdout: } 2023-11-26T07:21:52.018 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2023-11-26T07:21:52.018 INFO:teuthology.orchestra.run.smithi143.stdout: { 2023-11-26T07:21:52.018 INFO:teuthology.orchestra.run.smithi143.stdout: "entity": "client.2", 2023-11-26T07:21:52.019 INFO:teuthology.orchestra.run.smithi143.stdout: "key": "AQDm8GJlW4KlGhAANFKsyVJems0ZoE6bo3WmfQ==", 2023-11-26T07:21:52.019 INFO:teuthology.orchestra.run.smithi143.stdout: "caps": { 2023-11-26T07:21:52.019 INFO:teuthology.orchestra.run.smithi143.stdout: "mds": "allow", 2023-11-26T07:21:52.019 INFO:teuthology.orchestra.run.smithi143.stdout: "mgr": "allow r", 2023-11-26T07:21:52.019 INFO:teuthology.orchestra.run.smithi143.stdout: "mon": "allow rw", 2023-11-26T07:21:52.019 INFO:teuthology.orchestra.run.smithi143.stdout: "osd": "allow rwx" 2023-11-26T07:21:52.019 INFO:teuthology.orchestra.run.smithi143.stdout: } 2023-11-26T07:21:52.019 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2023-11-26T07:21:52.019 INFO:teuthology.orchestra.run.smithi143.stdout: { 2023-11-26T07:21:52.019 INFO:teuthology.orchestra.run.smithi143.stdout: "entity": "client.3", 2023-11-26T07:21:52.019 INFO:teuthology.orchestra.run.smithi143.stdout: "key": "AQDm8GJlAJZSJBAAUSUNcwRBY01hOuilBJjlFQ==", 2023-11-26T07:21:52.019 INFO:teuthology.orchestra.run.smithi143.stdout: "caps": { 2023-11-26T07:21:52.020 INFO:teuthology.orchestra.run.smithi143.stdout: "mds": "allow", 2023-11-26T07:21:52.020 INFO:teuthology.orchestra.run.smithi143.stdout: "mgr": "allow r", 2023-11-26T07:21:52.020 INFO:teuthology.orchestra.run.smithi143.stdout: "mon": "allow rw", 2023-11-26T07:21:52.020 INFO:teuthology.orchestra.run.smithi143.stdout: "osd": "allow rwx" 2023-11-26T07:21:52.020 INFO:teuthology.orchestra.run.smithi143.stdout: } 2023-11-26T07:21:52.020 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2023-11-26T07:21:52.020 INFO:teuthology.orchestra.run.smithi143.stdout: { 2023-11-26T07:21:52.020 INFO:teuthology.orchestra.run.smithi143.stdout: "entity": "client.admin", 2023-11-26T07:21:52.020 INFO:teuthology.orchestra.run.smithi143.stdout: "key": "AQDl8GJlbQv7GxAA0Midh8tAO2KO+dH10OV8Xg==", 2023-11-26T07:21:52.020 INFO:teuthology.orchestra.run.smithi143.stdout: "caps": { 2023-11-26T07:21:52.020 INFO:teuthology.orchestra.run.smithi143.stdout: "mds": "allow *", 2023-11-26T07:21:52.020 INFO:teuthology.orchestra.run.smithi143.stdout: "mgr": "allow *", 2023-11-26T07:21:52.020 INFO:teuthology.orchestra.run.smithi143.stdout: "mon": "allow *", 2023-11-26T07:21:52.021 INFO:teuthology.orchestra.run.smithi143.stdout: "osd": "allow *" 2023-11-26T07:21:52.021 INFO:teuthology.orchestra.run.smithi143.stdout: } 2023-11-26T07:21:52.021 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2023-11-26T07:21:52.021 INFO:teuthology.orchestra.run.smithi143.stdout: { 2023-11-26T07:21:52.021 INFO:teuthology.orchestra.run.smithi143.stdout: "entity": "mgr.x", 2023-11-26T07:21:52.021 INFO:teuthology.orchestra.run.smithi143.stdout: "key": "AQDl8GJlXy8cNBAA4f5RDC1MASCqZPcJQvnUDA==", 2023-11-26T07:21:52.021 INFO:teuthology.orchestra.run.smithi143.stdout: "caps": { 2023-11-26T07:21:52.021 INFO:teuthology.orchestra.run.smithi143.stdout: "mds": "allow *", 2023-11-26T07:21:52.021 INFO:teuthology.orchestra.run.smithi143.stdout: "mon": "allow profile mgr", 2023-11-26T07:21:52.021 INFO:teuthology.orchestra.run.smithi143.stdout: "osd": "allow *" 2023-11-26T07:21:52.021 INFO:teuthology.orchestra.run.smithi143.stdout: } 2023-11-26T07:21:52.021 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2023-11-26T07:21:52.022 INFO:teuthology.orchestra.run.smithi143.stdout: { 2023-11-26T07:21:52.022 INFO:teuthology.orchestra.run.smithi143.stdout: "entity": "mgr.y", 2023-11-26T07:21:52.022 INFO:teuthology.orchestra.run.smithi143.stdout: "key": "AQDl8GJlfuFPMBAA3TYEFvdfk4DyGsBkbpv0hg==", 2023-11-26T07:21:52.022 INFO:teuthology.orchestra.run.smithi143.stdout: "caps": { 2023-11-26T07:21:52.022 INFO:teuthology.orchestra.run.smithi143.stdout: "mds": "allow *", 2023-11-26T07:21:52.022 INFO:teuthology.orchestra.run.smithi143.stdout: "mon": "allow profile mgr", 2023-11-26T07:21:52.022 INFO:teuthology.orchestra.run.smithi143.stdout: "osd": "allow *" 2023-11-26T07:21:52.022 INFO:teuthology.orchestra.run.smithi143.stdout: } 2023-11-26T07:21:52.022 INFO:teuthology.orchestra.run.smithi143.stdout: } 2023-11-26T07:21:52.022 INFO:teuthology.orchestra.run.smithi143.stdout: ] 2023-11-26T07:21:52.022 INFO:teuthology.orchestra.run.smithi143.stdout:} 2023-11-26T07:21:52.023 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:21:52.008+0000 7fae8289c640 1 -- 172.21.15.143:0/4072794391 >> [v2:172.21.15.143:6832/2150557544,v1:172.21.15.143:6833/2150557544] conn(0x7fae68066620 msgr2=0x7fae68068ae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:21:52.023 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:21:52.008+0000 7fae8289c640 1 --2- 172.21.15.143:0/4072794391 >> [v2:172.21.15.143:6832/2150557544,v1:172.21.15.143:6833/2150557544] conn(0x7fae68066620 0x7fae68068ae0 secure :-1 s=READY pgs=371 cs=0 l=1 rev1=1 crypto rx=0x7fae6c003e40 tx=0x7fae6c009040 comp rx=0 tx=0).stop 2023-11-26T07:21:52.024 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:21:52.008+0000 7fae8289c640 1 -- 172.21.15.143:0/4072794391 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fae7c0cf280 msgr2=0x7fae7c15ccd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:21:52.024 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:21:52.008+0000 7fae8289c640 1 --2- 172.21.15.143:0/4072794391 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fae7c0cf280 0x7fae7c15ccd0 secure :-1 s=READY pgs=775 cs=0 l=1 rev1=1 crypto rx=0x7fae7001a0f0 tx=0x7fae7001ad00 comp rx=0 tx=0).stop 2023-11-26T07:21:52.024 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:21:52.008+0000 7fae8289c640 1 -- 172.21.15.143:0/4072794391 shutdown_connections 2023-11-26T07:21:52.024 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:21:52.008+0000 7fae8289c640 1 --2- 172.21.15.143:0/4072794391 >> [v2:172.21.15.143:6832/2150557544,v1:172.21.15.143:6833/2150557544] conn(0x7fae68066620 0x7fae68068ae0 unknown :-1 s=CLOSED pgs=371 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:21:52.024 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:21:52.008+0000 7fae8289c640 1 --2- 172.21.15.143:0/4072794391 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fae7c143ac0 0x7fae7c15afe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:21:52.024 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:21:52.008+0000 7fae8289c640 1 --2- 172.21.15.143:0/4072794391 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fae7c0cf280 0x7fae7c15ccd0 unknown :-1 s=CLOSED pgs=775 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:21:52.024 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:21:52.008+0000 7fae8289c640 1 --2- 172.21.15.143:0/4072794391 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fae7c0cd1b0 0x7fae7c14a380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:21:52.024 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:21:52.008+0000 7fae8289c640 1 -- 172.21.15.143:0/4072794391 >> 172.21.15.143:0/4072794391 conn(0x7fae7c06a890 msgr2=0x7fae7c0ace30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:21:52.024 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:21:52.008+0000 7fae8289c640 1 -- 172.21.15.143:0/4072794391 shutdown_connections 2023-11-26T07:21:52.024 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:21:52.008+0000 7fae8289c640 1 -- 172.21.15.143:0/4072794391 wait complete. 2023-11-26T07:21:52.029 DEBUG:teuthology.orchestra.run.smithi143:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-11-26T07:21:52.166 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:21:52.164+0000 7fdc3d565640 1 Processor -- start 2023-11-26T07:21:52.166 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:21:52.164+0000 7fdc3d565640 1 -- start start 2023-11-26T07:21:52.168 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:21:52.164+0000 7fdc3d565640 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7fdc3815d580 0x7fdc380d3920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:21:52.168 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:21:52.164+0000 7fdc3d565640 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7fdc380d3e60 0x7fdc380d17f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:21:52.168 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:21:52.164+0000 7fdc3d565640 1 -- --> v1:172.21.15.188:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fdc380d21e0 con 0x7fdc380d1d30 2023-11-26T07:21:52.168 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:21:52.164+0000 7fdc3d565640 1 -- --> v2:172.21.15.143:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fdc380d0bc0 con 0x7fdc3815d580 2023-11-26T07:21:52.168 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:21:52.164+0000 7fdc3d565640 1 -- --> v2:172.21.15.188:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fdc380d0320 con 0x7fdc380d3e60 2023-11-26T07:21:52.168 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:21:52.164+0000 7fdc36ffd640 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7fdc3815d580 0x7fdc380d3920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:21:52.169 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:21:52.164+0000 7fdc36ffd640 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7fdc3815d580 0x7fdc380d3920 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:49458/0 (socket says 172.21.15.143:49458) 2023-11-26T07:21:52.169 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:21:52.164+0000 7fdc36ffd640 1 -- 172.21.15.143:0/4189026665 learned_addr learned my addr 172.21.15.143:0/4189026665 (peer_addr_for_me v2:172.21.15.143:0/0) 2023-11-26T07:21:52.169 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:21:52.164+0000 7fdc2e7fc640 1 --2- 172.21.15.143:0/4189026665 >> v2:172.21.15.188:3300/0 conn(0x7fdc380d3e60 0x7fdc380d17f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:21:52.169 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:21:52.164+0000 7fdc36ffd640 1 -- 172.21.15.143:0/4189026665 >> v2:172.21.15.188:3300/0 conn(0x7fdc380d3e60 msgr2=0x7fdc380d17f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-26T07:21:52.169 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:21:52.164+0000 7fdc36ffd640 1 --2- 172.21.15.143:0/4189026665 >> v2:172.21.15.188:3300/0 conn(0x7fdc380d3e60 0x7fdc380d17f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:21:52.169 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:21:52.164+0000 7fdc36ffd640 1 -- 172.21.15.143:0/4189026665 >> v1:172.21.15.188:6789/0 conn(0x7fdc380d1d30 legacy=0x7fdc380cdce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-26T07:21:52.169 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:21:52.164+0000 7fdc36ffd640 1 -- 172.21.15.143:0/4189026665 --> v2:172.21.15.143:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdc380d0600 con 0x7fdc3815d580 2023-11-26T07:21:52.169 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:21:52.164+0000 7fdc2e7fc640 1 --2- 172.21.15.143:0/4189026665 >> v2:172.21.15.188:3300/0 conn(0x7fdc380d3e60 0x7fdc380d17f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-26T07:21:52.171 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:21:52.168+0000 7fdc36ffd640 1 --2- 172.21.15.143:0/4189026665 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fdc3815d580 0x7fdc380d3920 secure :-1 s=READY pgs=776 cs=0 l=1 rev1=1 crypto rx=0x7fdc380a1190 tx=0x7fdc2801c490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a77870fa8aa390e2 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:21:52.171 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:21:52.168+0000 7fdc367fc640 1 -- 172.21.15.143:0/4189026665 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdc280aa5f0 con 0x7fdc3815d580 2023-11-26T07:21:52.172 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:21:52.168+0000 7fdc367fc640 1 -- 172.21.15.143:0/4189026665 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdc280aa750 con 0x7fdc3815d580 2023-11-26T07:21:52.172 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:21:52.168+0000 7fdc367fc640 1 -- 172.21.15.143:0/4189026665 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdc280aaae0 con 0x7fdc3815d580 2023-11-26T07:21:52.173 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:21:52.168+0000 7fdc3d565640 1 -- 172.21.15.143:0/4189026665 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fdc3815d580 msgr2=0x7fdc380d3920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:21:52.173 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:21:52.168+0000 7fdc3d565640 1 --2- 172.21.15.143:0/4189026665 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fdc3815d580 0x7fdc380d3920 secure :-1 s=READY pgs=776 cs=0 l=1 rev1=1 crypto rx=0x7fdc380a1190 tx=0x7fdc2801c490 comp rx=0 tx=0).stop 2023-11-26T07:21:52.173 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:21:52.168+0000 7fdc3d565640 1 -- 172.21.15.143:0/4189026665 shutdown_connections 2023-11-26T07:21:52.173 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:21:52.168+0000 7fdc3d565640 1 --2- 172.21.15.143:0/4189026665 >> v2:172.21.15.188:3300/0 conn(0x7fdc380d3e60 0x7fdc380d17f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:21:52.173 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:21:52.168+0000 7fdc3d565640 1 --2- 172.21.15.143:0/4189026665 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fdc3815d580 0x7fdc380d3920 unknown :-1 s=CLOSED pgs=776 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:21:52.173 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:21:52.168+0000 7fdc3d565640 1 -- 172.21.15.143:0/4189026665 >> 172.21.15.143:0/4189026665 conn(0x7fdc3809c550 msgr2=0x7fdc380d3210 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:21:52.174 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:21:52.168+0000 7fdc3d565640 1 -- 172.21.15.143:0/4189026665 shutdown_connections 2023-11-26T07:21:52.174 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:21:52.168+0000 7fdc3d565640 1 -- 172.21.15.143:0/4189026665 wait complete. 2023-11-26T07:21:52.174 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:21:52.172+0000 7fdc3d565640 1 Processor -- start 2023-11-26T07:21:52.174 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:21:52.172+0000 7fdc3d565640 1 -- start start 2023-11-26T07:21:52.175 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:21:52.172+0000 7fdc3d565640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fdc380d1d30 0x7fdc38153730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:21:52.176 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:21:52.172+0000 7fdc3d565640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fdc380d3e60 0x7fdc381582a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:21:52.176 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:21:52.172+0000 7fdc3d565640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fdc3815d580 0x7fdc381587e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:21:52.176 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:21:52.172+0000 7fdc3d565640 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdc381557f0 con 0x7fdc380d1d30 2023-11-26T07:21:52.176 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:21:52.172+0000 7fdc3d565640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdc3815cde0 con 0x7fdc380d3e60 2023-11-26T07:21:52.176 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:21:52.172+0000 7fdc3d565640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdc380b6190 con 0x7fdc3815d580 2023-11-26T07:21:52.176 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:21:52.172+0000 7fdc36ffd640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fdc380d1d30 0x7fdc38153730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:21:52.176 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:21:52.172+0000 7fdc36ffd640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fdc380d1d30 0x7fdc38153730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:49460/0 (socket says 172.21.15.143:49460) 2023-11-26T07:21:52.176 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:21:52.172+0000 7fdc36ffd640 1 -- 172.21.15.143:0/1816917075 learned_addr learned my addr 172.21.15.143:0/1816917075 (peer_addr_for_me v2:172.21.15.143:0/0) 2023-11-26T07:21:52.176 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:21:52.172+0000 7fdc2e7fc640 1 --2- 172.21.15.143:0/1816917075 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fdc380d3e60 0x7fdc381582a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:21:52.176 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:21:52.172+0000 7fdc377fe640 1 --2- 172.21.15.143:0/1816917075 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fdc3815d580 0x7fdc381587e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:21:52.176 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:21:52.172+0000 7fdc36ffd640 1 -- 172.21.15.143:0/1816917075 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fdc3815d580 msgr2=0x7fdc381587e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:21:52.177 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:21:52.172+0000 7fdc36ffd640 1 --2- 172.21.15.143:0/1816917075 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fdc3815d580 0x7fdc381587e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:21:52.177 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:21:52.172+0000 7fdc36ffd640 1 -- 172.21.15.143:0/1816917075 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fdc380d3e60 msgr2=0x7fdc381582a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:21:52.177 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:21:52.172+0000 7fdc36ffd640 1 --2- 172.21.15.143:0/1816917075 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fdc380d3e60 0x7fdc381582a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:21:52.177 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:21:52.172+0000 7fdc36ffd640 1 -- 172.21.15.143:0/1816917075 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdc280094d0 con 0x7fdc380d1d30 2023-11-26T07:21:52.177 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:21:52.172+0000 7fdc36ffd640 1 --2- 172.21.15.143:0/1816917075 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fdc380d1d30 0x7fdc38153730 secure :-1 s=READY pgs=777 cs=0 l=1 rev1=1 crypto rx=0x7fdc2801a0c0 tx=0x7fdc2801acd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:21:52.177 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:21:52.172+0000 7fdc2ffff640 1 -- 172.21.15.143:0/1816917075 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdc280b3800 con 0x7fdc380d1d30 2023-11-26T07:21:52.178 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:21:52.172+0000 7fdc2ffff640 1 -- 172.21.15.143:0/1816917075 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdc280b3980 con 0x7fdc380d1d30 2023-11-26T07:21:52.178 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:21:52.172+0000 7fdc2ffff640 1 -- 172.21.15.143:0/1816917075 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdc280b3d30 con 0x7fdc380d1d30 2023-11-26T07:21:52.178 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:21:52.172+0000 7fdc3d565640 1 -- 172.21.15.143:0/1816917075 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdc380b6430 con 0x7fdc380d1d30 2023-11-26T07:21:52.178 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:21:52.172+0000 7fdc3d565640 1 -- 172.21.15.143:0/1816917075 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdc380b69f0 con 0x7fdc380d1d30 2023-11-26T07:21:52.178 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:21:52.176+0000 7fdc3d565640 1 -- 172.21.15.143:0/1816917075 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fdc380b6be0 con 0x7fdc380d1d30 2023-11-26T07:21:52.178 DEBUG:teuthology.orchestra.run.smithi143:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-26T07:21:52.179 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:21:52.176+0000 7fdc2ffff640 1 -- 172.21.15.143:0/1816917075 <== mon.0 v2:172.21.15.143:3300/0 4 ==== mgrmap(e 9) v1 ==== 86005+0+0 (secure 0 0 0) 0x7fdc280b2380 con 0x7fdc380d1d30 2023-11-26T07:21:52.179 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:21:52.176+0000 7fdc2ffff640 1 --2- 172.21.15.143:0/1816917075 >> [v2:172.21.15.143:6832/2150557544,v1:172.21.15.143:6833/2150557544] conn(0x7fdc18066620 0x7fdc18068ae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:21:52.179 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:21:52.176+0000 7fdc2ffff640 1 -- 172.21.15.143:0/1816917075 <== mon.0 v2:172.21.15.143:3300/0 5 ==== osd_map(55..55 src has 1..55) v4 ==== 4857+0+0 (secure 0 0 0) 0x7fdc280a7070 con 0x7fdc380d1d30 2023-11-26T07:21:52.180 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:21:52.176+0000 7fdc2e7fc640 1 --2- 172.21.15.143:0/1816917075 >> [v2:172.21.15.143:6832/2150557544,v1:172.21.15.143:6833/2150557544] conn(0x7fdc18066620 0x7fdc18068ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:21:52.180 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:21:52.176+0000 7fdc2e7fc640 1 --2- 172.21.15.143:0/1816917075 >> [v2:172.21.15.143:6832/2150557544,v1:172.21.15.143:6833/2150557544] conn(0x7fdc18066620 0x7fdc18068ae0 secure :-1 s=READY pgs=372 cs=0 l=1 rev1=1 crypto rx=0x7fdc380b76e0 tx=0x7fdc20008040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:21:52.182 DEBUG:teuthology.orchestra.run.smithi188:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-26T07:21:52.183 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:21:52.180+0000 7fdc2ffff640 1 -- 172.21.15.143:0/1816917075 <== mon.0 v2:172.21.15.143:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172649 (secure 0 0 0) 0x7fdc280eb7e0 con 0x7fdc380d1d30 2023-11-26T07:21:52.336 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:21:52.336+0000 7fdc3d565640 1 -- 172.21.15.143:0/1816917075 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fdc38067680 con 0x7fdc380d1d30 2023-11-26T07:21:52.337 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:21:52.336+0000 7fdc2ffff640 1 -- 172.21.15.143:0/1816917075 <== mon.0 v2:172.21.15.143:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v55) v1 ==== 74+0+12893 (secure 0 0 0) 0x7fdc280eee00 con 0x7fdc380d1d30 2023-11-26T07:21:52.337 INFO:teuthology.orchestra.run.smithi143.stdout: 2023-11-26T07:21:52.338 INFO:teuthology.orchestra.run.smithi143.stdout:{"epoch":55,"fsid":"f5033060-2469-4a07-95fe-2b76f5404bc1","created":"2023-11-26T07:17:28.613663+0000","modified":"2023-11-26T07:21:51.615487+0000","last_up_change":"2023-11-26T07:17:41.157813+0000","last_in_change":"2023-11-26T07:17:31.181658+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":7,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-11-26T07:17:41.758597+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"ebbee2c6-023e-44b9-9f1d-6f7bfd309730","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":43,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6817","nonce":639807369},{"type":"v1","addr":"172.21.15.143:6819","nonce":639807369}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6821","nonce":639807369},{"type":"v1","addr":"172.21.15.143:6823","nonce":639807369}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6830","nonce":639807369},{"type":"v1","addr":"172.21.15.143:6831","nonce":639807369}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6825","nonce":639807369},{"type":"v1","addr":"172.21.15.143:6828","nonce":639807369}]},"public_addr":"172.21.15.143:6819/639807369","cluster_addr":"172.21.15.143:6823/639807369","heartbeat_back_addr":"172.21.15.143:6831/639807369","heartbeat_front_addr":"172.21.15.143:6828/639807369","state":["exists","up"]},{"osd":1,"uuid":"fec4b8be-7122-49e9-9a19-5850a0700001","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":43,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6816","nonce":1144942530},{"type":"v1","addr":"172.21.15.143:6818","nonce":1144942530}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6820","nonce":1144942530},{"type":"v1","addr":"172.21.15.143:6822","nonce":1144942530}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6827","nonce":1144942530},{"type":"v1","addr":"172.21.15.143:6829","nonce":1144942530}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6824","nonce":1144942530},{"type":"v1","addr":"172.21.15.143:6826","nonce":1144942530}]},"public_addr":"172.21.15.143:6818/1144942530","cluster_addr":"172.21.15.143:6822/1144942530","heartbeat_back_addr":"172.21.15.143:6829/1144942530","heartbeat_front_addr":"172.21.15.143:6826/1144942530","state":["exists","up"]},{"osd":2,"uuid":"2888af5b-f933-46ca-8f95-6dd494ff43f6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":46,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6808","nonce":3909095186},{"type":"v1","addr":"172.21.15.143:6809","nonce":3909095186}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6810","nonce":3909095186},{"type":"v1","addr":"172.21.15.143:6811","nonce":3909095186}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6814","nonce":3909095186},{"type":"v1","addr":"172.21.15.143:6815","nonce":3909095186}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6812","nonce":3909095186},{"type":"v1","addr":"172.21.15.143:6813","nonce":3909095186}]},"public_addr":"172.21.15.143:6809/3909095186","cluster_addr":"172.21.15.143:6811/3909095186","heartbeat_back_addr":"172.21.15.143:6815/3909095186","heartbeat_front_addr":"172.21.15.143:6813/3909095186","state":["exists","up"]},{"osd":3,"uuid":"d1f52435-7bc1-4081-9550-9cd1ba55f549","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":43,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6800","nonce":3854210991},{"type":"v1","addr":"172.21.15.143:6801","nonce":3854210991}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6802","nonce":3854210991},{"type":"v1","addr":"172.21.15.143:6803","nonce":3854210991}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6806","nonce":3854210991},{"type":"v1","addr":"172.21.15.143:6807","nonce":3854210991}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6804","nonce":3854210991},{"type":"v1","addr":"172.21.15.143:6805","nonce":3854210991}]},"public_addr":"172.21.15.143:6801/3854210991","cluster_addr":"172.21.15.143:6803/3854210991","heartbeat_back_addr":"172.21.15.143:6807/3854210991","heartbeat_front_addr":"172.21.15.143:6805/3854210991","state":["exists","up"]},{"osd":4,"uuid":"e2ae8b6e-6e25-49b2-bcbf-aacce68f06a3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":43,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6800","nonce":3924606902},{"type":"v1","addr":"172.21.15.188:6801","nonce":3924606902}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6802","nonce":3924606902},{"type":"v1","addr":"172.21.15.188:6803","nonce":3924606902}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6806","nonce":3924606902},{"type":"v1","addr":"172.21.15.188:6807","nonce":3924606902}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6804","nonce":3924606902},{"type":"v1","addr":"172.21.15.188:6805","nonce":3924606902}]},"public_addr":"172.21.15.188:6801/3924606902","cluster_addr":"172.21.15.188:6803/3924606902","heartbeat_back_addr":"172.21.15.188:6807/3924606902","heartbeat_front_addr":"172.21.15.188:6805/3924606902","state":["exists","up"]},{"osd":5,"uuid":"3cec8a32-7698-4c36-8e89-e4ba11934f33","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":43,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6813","nonce":186841544},{"type":"v1","addr":"172.21.15.188:6815","nonce":186841544}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6817","nonce":186841544},{"type":"v1","addr":"172.21.15.188:6818","nonce":186841544}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6822","nonce":186841544},{"type":"v1","addr":"172.21.15.188:6823","nonce":186841544}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6820","nonce":186841544},{"type":"v1","addr":"172.21.15.188:6821","nonce":186841544}]},"public_addr":"172.21.15.188:6815/186841544","cluster_addr":"172.21.15.188:6818/186841544","heartbeat_back_addr":"172.21.15.188:6823/186841544","heartbeat_front_addr":"172.21.15.188:6821/186841544","state":["exists","up"]},{"osd":6,"uuid":"58a65d5d-e090-473a-81b4-ad38e85029f9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":46,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6808","nonce":2153499405},{"type":"v1","addr":"172.21.15.188:6809","nonce":2153499405}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6810","nonce":2153499405},{"type":"v1","addr":"172.21.15.188:6811","nonce":2153499405}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6816","nonce":2153499405},{"type":"v1","addr":"172.21.15.188:6819","nonce":2153499405}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6812","nonce":2153499405},{"type":"v1","addr":"172.21.15.188:6814","nonce":2153499405}]},"public_addr":"172.21.15.188:6809/2153499405","cluster_addr":"172.21.15.188:6811/2153499405","heartbeat_back_addr":"172.21.15.188:6819/2153499405","heartbeat_front_addr":"172.21.15.188:6814/2153499405","state":["exists","up"]},{"osd":7,"uuid":"ea5b7f70-3554-4cf8-ac30-f7c59b80678d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":43,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6824","nonce":2722096729},{"type":"v1","addr":"172.21.15.188:6825","nonce":2722096729}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6826","nonce":2722096729},{"type":"v1","addr":"172.21.15.188:6827","nonce":2722096729}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6830","nonce":2722096729},{"type":"v1","addr":"172.21.15.188:6831","nonce":2722096729}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6828","nonce":2722096729},{"type":"v1","addr":"172.21.15.188:6829","nonce":2722096729}]},"public_addr":"172.21.15.188:6825/2722096729","cluster_addr":"172.21.15.188:6827/2722096729","heartbeat_back_addr":"172.21.15.188:6831/2722096729","heartbeat_front_addr":"172.21.15.188:6829/2722096729","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-26T07:17:36.995032+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-26T07:17:36.932266+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-26T07:17:36.829304+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-26T07:17:36.804297+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-26T07:17:36.667892+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-26T07:17:36.964868+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-26T07:17:36.827332+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-26T07:17:36.893179+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-11-26T07:21:52.339 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:21:52.336+0000 7fdc3d565640 1 -- 172.21.15.143:0/1816917075 >> [v2:172.21.15.143:6832/2150557544,v1:172.21.15.143:6833/2150557544] conn(0x7fdc18066620 msgr2=0x7fdc18068ae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:21:52.339 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:21:52.336+0000 7fdc3d565640 1 --2- 172.21.15.143:0/1816917075 >> [v2:172.21.15.143:6832/2150557544,v1:172.21.15.143:6833/2150557544] conn(0x7fdc18066620 0x7fdc18068ae0 secure :-1 s=READY pgs=372 cs=0 l=1 rev1=1 crypto rx=0x7fdc380b76e0 tx=0x7fdc20008040 comp rx=0 tx=0).stop 2023-11-26T07:21:52.339 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:21:52.336+0000 7fdc3d565640 1 -- 172.21.15.143:0/1816917075 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fdc380d1d30 msgr2=0x7fdc38153730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:21:52.340 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:21:52.336+0000 7fdc3d565640 1 --2- 172.21.15.143:0/1816917075 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fdc380d1d30 0x7fdc38153730 secure :-1 s=READY pgs=777 cs=0 l=1 rev1=1 crypto rx=0x7fdc2801a0c0 tx=0x7fdc2801acd0 comp rx=0 tx=0).stop 2023-11-26T07:21:52.340 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:21:52.336+0000 7fdc3d565640 1 -- 172.21.15.143:0/1816917075 shutdown_connections 2023-11-26T07:21:52.340 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:21:52.336+0000 7fdc3d565640 1 --2- 172.21.15.143:0/1816917075 >> [v2:172.21.15.143:6832/2150557544,v1:172.21.15.143:6833/2150557544] conn(0x7fdc18066620 0x7fdc18068ae0 unknown :-1 s=CLOSED pgs=372 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:21:52.340 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:21:52.336+0000 7fdc3d565640 1 --2- 172.21.15.143:0/1816917075 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fdc3815d580 0x7fdc381587e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:21:52.340 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:21:52.336+0000 7fdc3d565640 1 --2- 172.21.15.143:0/1816917075 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fdc380d3e60 0x7fdc381582a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:21:52.340 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:21:52.336+0000 7fdc3d565640 1 --2- 172.21.15.143:0/1816917075 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fdc380d1d30 0x7fdc38153730 unknown :-1 s=CLOSED pgs=777 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:21:52.340 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:21:52.336+0000 7fdc3d565640 1 -- 172.21.15.143:0/1816917075 >> 172.21.15.143:0/1816917075 conn(0x7fdc3809c550 msgr2=0x7fdc38156550 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:21:52.340 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:21:52.336+0000 7fdc3d565640 1 -- 172.21.15.143:0/1816917075 shutdown_connections 2023-11-26T07:21:52.340 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:21:52.336+0000 7fdc3d565640 1 -- 172.21.15.143:0/1816917075 wait complete. 2023-11-26T07:21:52.359 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-11-26T07:17:41.758597+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2023-11-26T07:21:52.359 DEBUG:teuthology.orchestra.run.smithi143:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-11-26T07:21:52.499 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:21:52.496+0000 7f945af72640 1 Processor -- start 2023-11-26T07:21:52.499 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:21:52.496+0000 7f945af72640 1 -- start start 2023-11-26T07:21:52.500 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:21:52.496+0000 7f945af72640 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7f9454143ac0 0x7f9454070370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:21:52.500 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:21:52.496+0000 7f945af72640 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7f94540cf990 0x7f94540cd1b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:21:52.501 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:21:52.496+0000 7f945af72640 1 -- --> v1:172.21.15.188:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f94540cc730 con 0x7f94540708b0 2023-11-26T07:21:52.501 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:21:52.496+0000 7f945af72640 1 -- --> v2:172.21.15.143:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9454143ea0 con 0x7f9454143ac0 2023-11-26T07:21:52.501 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:21:52.496+0000 7f945af72640 1 -- --> v2:172.21.15.188:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9454070ee0 con 0x7f94540cf990 2023-11-26T07:21:52.501 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:21:52.500+0000 7f9458d0e640 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7f9454143ac0 0x7f9454070370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:21:52.501 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:21:52.500+0000 7f9458d0e640 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7f9454143ac0 0x7f9454070370 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:49470/0 (socket says 172.21.15.143:49470) 2023-11-26T07:21:52.501 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:21:52.500+0000 7f9458d0e640 1 -- 172.21.15.143:0/987349486 learned_addr learned my addr 172.21.15.143:0/987349486 (peer_addr_for_me v2:172.21.15.143:0/0) 2023-11-26T07:21:52.501 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:21:52.500+0000 7f945950f640 1 --2- 172.21.15.143:0/987349486 >> v2:172.21.15.188:3300/0 conn(0x7f94540cf990 0x7f94540cd1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:21:52.501 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:21:52.500+0000 7f9458d0e640 1 -- 172.21.15.143:0/987349486 >> v2:172.21.15.188:3300/0 conn(0x7f94540cf990 msgr2=0x7f94540cd1b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-26T07:21:52.502 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:21:52.500+0000 7f9458d0e640 1 --2- 172.21.15.143:0/987349486 >> v2:172.21.15.188:3300/0 conn(0x7f94540cf990 0x7f94540cd1b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:21:52.502 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:21:52.500+0000 7f9458d0e640 1 -- 172.21.15.143:0/987349486 >> v1:172.21.15.188:6789/0 conn(0x7f94540708b0 legacy=0x7f94540cf280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-26T07:21:52.502 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:21:52.500+0000 7f9458d0e640 1 -- 172.21.15.143:0/987349486 --> v2:172.21.15.143:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f94540711c0 con 0x7f9454143ac0 2023-11-26T07:21:52.504 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:21:52.500+0000 7f9458d0e640 1 --2- 172.21.15.143:0/987349486 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f9454143ac0 0x7f9454070370 secure :-1 s=READY pgs=778 cs=0 l=1 rev1=1 crypto rx=0x7f945406e570 tx=0x7f944801c4c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1cd06981fa3c64c9 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:21:52.504 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:21:52.500+0000 7f94537fe640 1 -- 172.21.15.143:0/987349486 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f94480aa620 con 0x7f9454143ac0 2023-11-26T07:21:52.504 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:21:52.500+0000 7f94537fe640 1 -- 172.21.15.143:0/987349486 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f94480aa780 con 0x7f9454143ac0 2023-11-26T07:21:52.504 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:21:52.500+0000 7f94537fe640 1 -- 172.21.15.143:0/987349486 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f94480aab10 con 0x7f9454143ac0 2023-11-26T07:21:52.505 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:21:52.504+0000 7f945af72640 1 -- 172.21.15.143:0/987349486 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f9454143ac0 msgr2=0x7f9454070370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:21:52.505 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:21:52.504+0000 7f945af72640 1 --2- 172.21.15.143:0/987349486 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f9454143ac0 0x7f9454070370 secure :-1 s=READY pgs=778 cs=0 l=1 rev1=1 crypto rx=0x7f945406e570 tx=0x7f944801c4c0 comp rx=0 tx=0).stop 2023-11-26T07:21:52.505 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:21:52.504+0000 7f945af72640 1 -- 172.21.15.143:0/987349486 shutdown_connections 2023-11-26T07:21:52.505 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:21:52.504+0000 7f945af72640 1 --2- 172.21.15.143:0/987349486 >> v2:172.21.15.188:3300/0 conn(0x7f94540cf990 0x7f94540cd1b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:21:52.505 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:21:52.504+0000 7f945af72640 1 --2- 172.21.15.143:0/987349486 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f9454143ac0 0x7f9454070370 unknown :-1 s=CLOSED pgs=778 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:21:52.505 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:21:52.504+0000 7f945af72640 1 -- 172.21.15.143:0/987349486 >> 172.21.15.143:0/987349486 conn(0x7f945406a890 msgr2=0x7f945406aca0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:21:52.505 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:21:52.504+0000 7f945af72640 1 -- 172.21.15.143:0/987349486 shutdown_connections 2023-11-26T07:21:52.505 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:21:52.504+0000 7f945af72640 1 -- 172.21.15.143:0/987349486 wait complete. 2023-11-26T07:21:52.506 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:21:52.504+0000 7f945af72640 1 Processor -- start 2023-11-26T07:21:52.506 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:21:52.504+0000 7f945af72640 1 -- start start 2023-11-26T07:21:52.508 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:21:52.504+0000 7f945af72640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f94540708b0 0x7f9454157db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:21:52.508 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:21:52.504+0000 7f945af72640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f94540cf990 0x7f94540aeb60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:21:52.509 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:21:52.504+0000 7f945af72640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f9454143ac0 0x7f945413eac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:21:52.509 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:21:52.504+0000 7f945af72640 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f94540aff20 con 0x7f9454143ac0 2023-11-26T07:21:52.509 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:21:52.504+0000 7f945af72640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f94540af620 con 0x7f94540708b0 2023-11-26T07:21:52.509 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:21:52.504+0000 7f945af72640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f94540af900 con 0x7f94540cf990 2023-11-26T07:21:52.509 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:21:52.504+0000 7f945950f640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f9454143ac0 0x7f945413eac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:21:52.509 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:21:52.504+0000 7f945950f640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f9454143ac0 0x7f945413eac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:49472/0 (socket says 172.21.15.143:49472) 2023-11-26T07:21:52.509 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:21:52.504+0000 7f945950f640 1 -- 172.21.15.143:0/4149104261 learned_addr learned my addr 172.21.15.143:0/4149104261 (peer_addr_for_me v2:172.21.15.143:0/0) 2023-11-26T07:21:52.509 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:21:52.504+0000 7f9458d0e640 1 --2- 172.21.15.143:0/4149104261 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f94540708b0 0x7f9454157db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:21:52.509 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:21:52.504+0000 7f9453fff640 1 --2- 172.21.15.143:0/4149104261 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f94540cf990 0x7f94540aeb60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:21:52.509 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:21:52.508+0000 7f945950f640 1 -- 172.21.15.143:0/4149104261 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f94540cf990 msgr2=0x7f94540aeb60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:21:52.509 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:21:52.508+0000 7f945950f640 1 --2- 172.21.15.143:0/4149104261 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f94540cf990 0x7f94540aeb60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:21:52.509 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:21:52.508+0000 7f945950f640 1 -- 172.21.15.143:0/4149104261 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f94540708b0 msgr2=0x7f9454157db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:21:52.509 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:21:52.508+0000 7f945950f640 1 --2- 172.21.15.143:0/4149104261 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f94540708b0 0x7f9454157db0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:21:52.510 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:21:52.508+0000 7f945950f640 1 -- 172.21.15.143:0/4149104261 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f94480094d0 con 0x7f9454143ac0 2023-11-26T07:21:52.510 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:21:52.508+0000 7f945950f640 1 --2- 172.21.15.143:0/4149104261 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f9454143ac0 0x7f945413eac0 secure :-1 s=READY pgs=779 cs=0 l=1 rev1=1 crypto rx=0x7f944c007e40 tx=0x7f944c00e4b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:21:52.511 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:21:52.508+0000 7f94517fa640 1 -- 172.21.15.143:0/4149104261 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f944c014620 con 0x7f9454143ac0 2023-11-26T07:21:52.511 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:21:52.508+0000 7f94517fa640 1 -- 172.21.15.143:0/4149104261 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f944c00ee90 con 0x7f9454143ac0 2023-11-26T07:21:52.511 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:21:52.508+0000 7f945af72640 1 -- 172.21.15.143:0/4149104261 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f94540b01c0 con 0x7f9454143ac0 2023-11-26T07:21:52.511 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:21:52.508+0000 7f94517fa640 1 -- 172.21.15.143:0/4149104261 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f944c0149e0 con 0x7f9454143ac0 2023-11-26T07:21:52.511 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:21:52.508+0000 7f945af72640 1 -- 172.21.15.143:0/4149104261 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f94540afbe0 con 0x7f9454143ac0 2023-11-26T07:21:52.511 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:21:52.508+0000 7f945af72640 1 -- 172.21.15.143:0/4149104261 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f94540ae000 con 0x7f9454143ac0 2023-11-26T07:21:52.511 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:21:52.508+0000 7f94517fa640 1 -- 172.21.15.143:0/4149104261 <== mon.0 v2:172.21.15.143:3300/0 4 ==== mgrmap(e 9) v1 ==== 86005+0+0 (secure 0 0 0) 0x7f944c014b40 con 0x7f9454143ac0 2023-11-26T07:21:52.512 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:21:52.508+0000 7f94517fa640 1 --2- 172.21.15.143:0/4149104261 >> [v2:172.21.15.143:6832/2150557544,v1:172.21.15.143:6833/2150557544] conn(0x7f9438066620 0x7f9438068ae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:21:52.512 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:21:52.508+0000 7f94517fa640 1 -- 172.21.15.143:0/4149104261 <== mon.0 v2:172.21.15.143:3300/0 5 ==== osd_map(55..55 src has 1..55) v4 ==== 4857+0+0 (secure 0 0 0) 0x7f944c0846f0 con 0x7f9454143ac0 2023-11-26T07:21:52.512 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:21:52.508+0000 7f9458d0e640 1 --2- 172.21.15.143:0/4149104261 >> [v2:172.21.15.143:6832/2150557544,v1:172.21.15.143:6833/2150557544] conn(0x7f9438066620 0x7f9438068ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:21:52.512 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:21:52.512+0000 7f9458d0e640 1 --2- 172.21.15.143:0/4149104261 >> [v2:172.21.15.143:6832/2150557544,v1:172.21.15.143:6833/2150557544] conn(0x7f9438066620 0x7f9438068ae0 secure :-1 s=READY pgs=373 cs=0 l=1 rev1=1 crypto rx=0x7f94480ae000 tx=0x7f94480b5040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:21:52.515 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:21:52.512+0000 7f94517fa640 1 -- 172.21.15.143:0/4149104261 <== mon.0 v2:172.21.15.143:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172649 (secure 0 0 0) 0x7f944c051ce0 con 0x7f9454143ac0 2023-11-26T07:21:52.666 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:21:52.664+0000 7f945af72640 1 -- 172.21.15.143:0/4149104261 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f94540ae220 con 0x7f9454143ac0 2023-11-26T07:21:52.667 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:21:52.664+0000 7f94517fa640 1 -- 172.21.15.143:0/4149104261 <== mon.0 v2:172.21.15.143:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v55) v1 ==== 93+0+10 (secure 0 0 0) 0x7f944c055300 con 0x7f9454143ac0 2023-11-26T07:21:52.667 INFO:teuthology.orchestra.run.smithi143.stdout:pg_num: 1 2023-11-26T07:21:52.669 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:21:52.668+0000 7f945af72640 1 -- 172.21.15.143:0/4149104261 >> [v2:172.21.15.143:6832/2150557544,v1:172.21.15.143:6833/2150557544] conn(0x7f9438066620 msgr2=0x7f9438068ae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:21:52.669 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:21:52.668+0000 7f945af72640 1 --2- 172.21.15.143:0/4149104261 >> [v2:172.21.15.143:6832/2150557544,v1:172.21.15.143:6833/2150557544] conn(0x7f9438066620 0x7f9438068ae0 secure :-1 s=READY pgs=373 cs=0 l=1 rev1=1 crypto rx=0x7f94480ae000 tx=0x7f94480b5040 comp rx=0 tx=0).stop 2023-11-26T07:21:52.669 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:21:52.668+0000 7f945af72640 1 -- 172.21.15.143:0/4149104261 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f9454143ac0 msgr2=0x7f945413eac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:21:52.669 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:21:52.668+0000 7f945af72640 1 --2- 172.21.15.143:0/4149104261 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f9454143ac0 0x7f945413eac0 secure :-1 s=READY pgs=779 cs=0 l=1 rev1=1 crypto rx=0x7f944c007e40 tx=0x7f944c00e4b0 comp rx=0 tx=0).stop 2023-11-26T07:21:52.669 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:21:52.668+0000 7f945af72640 1 -- 172.21.15.143:0/4149104261 shutdown_connections 2023-11-26T07:21:52.669 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:21:52.668+0000 7f945af72640 1 --2- 172.21.15.143:0/4149104261 >> [v2:172.21.15.143:6832/2150557544,v1:172.21.15.143:6833/2150557544] conn(0x7f9438066620 0x7f9438068ae0 unknown :-1 s=CLOSED pgs=373 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:21:52.669 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:21:52.668+0000 7f945af72640 1 --2- 172.21.15.143:0/4149104261 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f9454143ac0 0x7f945413eac0 unknown :-1 s=CLOSED pgs=779 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:21:52.669 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:21:52.668+0000 7f945af72640 1 --2- 172.21.15.143:0/4149104261 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f94540cf990 0x7f94540aeb60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:21:52.669 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:21:52.668+0000 7f945af72640 1 --2- 172.21.15.143:0/4149104261 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f94540708b0 0x7f9454157db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:21:52.669 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:21:52.668+0000 7f945af72640 1 -- 172.21.15.143:0/4149104261 >> 172.21.15.143:0/4149104261 conn(0x7f945406a890 msgr2=0x7f94540b2810 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:21:52.670 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:21:52.668+0000 7f945af72640 1 -- 172.21.15.143:0/4149104261 shutdown_connections 2023-11-26T07:21:52.670 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:21:52.668+0000 7f945af72640 1 -- 172.21.15.143:0/4149104261 wait complete. 2023-11-26T07:21:52.688 DEBUG:teuthology.orchestra.run.smithi143:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --format=json-pretty osd lspools 2023-11-26T07:21:52.827 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:21:52.824+0000 7f0d336d6640 1 Processor -- start 2023-11-26T07:21:52.827 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:21:52.824+0000 7f0d336d6640 1 -- start start 2023-11-26T07:21:52.828 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:21:52.824+0000 7f0d336d6640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f0d2c1614b0 0x7f0d2c0cd610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:21:52.828 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:21:52.824+0000 7f0d336d6640 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7f0d2c071970 0x7f0d2c161890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:21:52.828 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:21:52.824+0000 7f0d336d6640 1 -- --> v1:172.21.15.188:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0d2c15d970 con 0x7f0d2c070ba0 2023-11-26T07:21:52.828 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:21:52.824+0000 7f0d336d6640 1 -- --> v2:172.21.15.188:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0d2c161dd0 con 0x7f0d2c071970 2023-11-26T07:21:52.828 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:21:52.824+0000 7f0d336d6640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0d2c15dc30 con 0x7f0d2c1614b0 2023-11-26T07:21:52.829 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:21:52.824+0000 7f0d31472640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f0d2c1614b0 0x7f0d2c0cd610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:21:52.829 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:21:52.824+0000 7f0d31472640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f0d2c1614b0 0x7f0d2c0cd610 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.188:3301/0 says I am v2:172.21.15.143:51452/0 (socket says 172.21.15.143:51452) 2023-11-26T07:21:52.830 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:21:52.824+0000 7f0d31472640 1 -- 172.21.15.143:0/1046849776 learned_addr learned my addr 172.21.15.143:0/1046849776 (peer_addr_for_me v2:172.21.15.143:0/0) 2023-11-26T07:21:52.830 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:21:52.828+0000 7f0d30c71640 1 --2- 172.21.15.143:0/1046849776 >> v2:172.21.15.188:3300/0 conn(0x7f0d2c071970 0x7f0d2c161890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:21:52.830 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:21:52.828+0000 7f0d23fff640 1 -- 172.21.15.143:0/1046849776 <== mon.1 v1:172.21.15.188:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3868712235 0 0) 0x7f0d2c15d970 con 0x7f0d2c070ba0 2023-11-26T07:21:52.830 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:21:52.828+0000 7f0d30c71640 1 -- 172.21.15.143:0/1046849776 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f0d2c1614b0 msgr2=0x7f0d2c0cd610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-26T07:21:52.830 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:21:52.828+0000 7f0d30c71640 1 --2- 172.21.15.143:0/1046849776 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f0d2c1614b0 0x7f0d2c0cd610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:21:52.830 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:21:52.828+0000 7f0d30c71640 1 -- 172.21.15.143:0/1046849776 >> v1:172.21.15.188:6789/0 conn(0x7f0d2c070ba0 legacy=0x7f0d2c06dd10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:21:52.830 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:21:52.828+0000 7f0d30c71640 1 -- 172.21.15.143:0/1046849776 --> v2:172.21.15.188:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0d2c15df10 con 0x7f0d2c071970 2023-11-26T07:21:52.830 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:21:52.828+0000 7f0d31472640 1 --2- 172.21.15.143:0/1046849776 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f0d2c1614b0 0x7f0d2c0cd610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-26T07:21:52.831 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:21:52.828+0000 7f0d30c71640 1 --2- 172.21.15.143:0/1046849776 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f0d2c071970 0x7f0d2c161890 secure :-1 s=READY pgs=311 cs=0 l=1 rev1=1 crypto rx=0x7f0d1c0097c0 tx=0x7f0d1c01c450 comp rx=0 tx=0).ready entity=mon.1 client_cookie=4ca9448947013fd5 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:21:52.832 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:21:52.828+0000 7f0d23fff640 1 -- 172.21.15.143:0/1046849776 <== mon.1 v2:172.21.15.188:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0d1c0ab620 con 0x7f0d2c071970 2023-11-26T07:21:52.832 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:21:52.828+0000 7f0d23fff640 1 -- 172.21.15.143:0/1046849776 <== mon.1 v2:172.21.15.188:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0d1c0ab780 con 0x7f0d2c071970 2023-11-26T07:21:52.832 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:21:52.828+0000 7f0d23fff640 1 -- 172.21.15.143:0/1046849776 <== mon.1 v2:172.21.15.188:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0d1c0abb10 con 0x7f0d2c071970 2023-11-26T07:21:52.833 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:21:52.832+0000 7f0d336d6640 1 -- 172.21.15.143:0/1046849776 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f0d2c071970 msgr2=0x7f0d2c161890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:21:52.833 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:21:52.832+0000 7f0d336d6640 1 --2- 172.21.15.143:0/1046849776 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f0d2c071970 0x7f0d2c161890 secure :-1 s=READY pgs=311 cs=0 l=1 rev1=1 crypto rx=0x7f0d1c0097c0 tx=0x7f0d1c01c450 comp rx=0 tx=0).stop 2023-11-26T07:21:52.833 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:21:52.832+0000 7f0d336d6640 1 -- 172.21.15.143:0/1046849776 shutdown_connections 2023-11-26T07:21:52.833 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:21:52.832+0000 7f0d336d6640 1 --2- 172.21.15.143:0/1046849776 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f0d2c071970 0x7f0d2c161890 unknown :-1 s=CLOSED pgs=311 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:21:52.834 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:21:52.832+0000 7f0d336d6640 1 --2- 172.21.15.143:0/1046849776 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f0d2c1614b0 0x7f0d2c0cd610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:21:52.834 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:21:52.832+0000 7f0d336d6640 1 -- 172.21.15.143:0/1046849776 >> 172.21.15.143:0/1046849776 conn(0x7f0d2c0cf710 msgr2=0x7f0d2c0cfb20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:21:52.834 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:21:52.832+0000 7f0d336d6640 1 -- 172.21.15.143:0/1046849776 shutdown_connections 2023-11-26T07:21:52.834 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:21:52.832+0000 7f0d336d6640 1 -- 172.21.15.143:0/1046849776 wait complete. 2023-11-26T07:21:52.834 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:21:52.832+0000 7f0d336d6640 1 Processor -- start 2023-11-26T07:21:52.834 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:21:52.832+0000 7f0d336d6640 1 -- start start 2023-11-26T07:21:52.836 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:21:52.832+0000 7f0d336d6640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f0d2c070ba0 0x7f0d2c0b2d60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:21:52.836 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:21:52.832+0000 7f0d336d6640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f0d2c071970 0x7f0d2c0cdb00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:21:52.836 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:21:52.832+0000 7f0d336d6640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f0d2c1614b0 0x7f0d2c151170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:21:52.836 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:21:52.832+0000 7f0d336d6640 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0d2c0b46f0 con 0x7f0d2c070ba0 2023-11-26T07:21:52.836 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:21:52.832+0000 7f0d336d6640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0d2c0adec0 con 0x7f0d2c071970 2023-11-26T07:21:52.837 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:21:52.832+0000 7f0d336d6640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0d2c0b3ae0 con 0x7f0d2c1614b0 2023-11-26T07:21:52.837 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:21:52.832+0000 7f0d31472640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f0d2c070ba0 0x7f0d2c0b2d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:21:52.837 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:21:52.832+0000 7f0d31472640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f0d2c070ba0 0x7f0d2c0b2d60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:49484/0 (socket says 172.21.15.143:49484) 2023-11-26T07:21:52.837 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:21:52.832+0000 7f0d31472640 1 -- 172.21.15.143:0/3663185887 learned_addr learned my addr 172.21.15.143:0/3663185887 (peer_addr_for_me v2:172.21.15.143:0/0) 2023-11-26T07:21:52.838 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:21:52.832+0000 7f0d30c71640 1 --2- 172.21.15.143:0/3663185887 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f0d2c071970 0x7f0d2c0cdb00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:21:52.838 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:21:52.832+0000 7f0d31c73640 1 --2- 172.21.15.143:0/3663185887 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f0d2c1614b0 0x7f0d2c151170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:21:52.838 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:21:52.832+0000 7f0d31472640 1 -- 172.21.15.143:0/3663185887 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f0d2c1614b0 msgr2=0x7f0d2c151170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:21:52.839 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:21:52.832+0000 7f0d31472640 1 --2- 172.21.15.143:0/3663185887 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f0d2c1614b0 0x7f0d2c151170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:21:52.839 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:21:52.832+0000 7f0d31472640 1 -- 172.21.15.143:0/3663185887 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f0d2c071970 msgr2=0x7f0d2c0cdb00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:21:52.839 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:21:52.832+0000 7f0d31472640 1 --2- 172.21.15.143:0/3663185887 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f0d2c071970 0x7f0d2c0cdb00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:21:52.839 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:21:52.832+0000 7f0d31472640 1 -- 172.21.15.143:0/3663185887 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0d280094d0 con 0x7f0d2c070ba0 2023-11-26T07:21:52.839 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:21:52.832+0000 7f0d30c71640 1 --2- 172.21.15.143:0/3663185887 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f0d2c071970 0x7f0d2c0cdb00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-26T07:21:52.839 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:21:52.836+0000 7f0d31472640 1 --2- 172.21.15.143:0/3663185887 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f0d2c070ba0 0x7f0d2c0b2d60 secure :-1 s=READY pgs=780 cs=0 l=1 rev1=1 crypto rx=0x7f0d2800e9d0 tx=0x7f0d2800ee90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:21:52.840 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:21:52.836+0000 7f0d21ffb640 1 -- 172.21.15.143:0/3663185887 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0d2800ca60 con 0x7f0d2c070ba0 2023-11-26T07:21:52.840 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:21:52.836+0000 7f0d21ffb640 1 -- 172.21.15.143:0/3663185887 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0d2800cbc0 con 0x7f0d2c070ba0 2023-11-26T07:21:52.840 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:21:52.836+0000 7f0d21ffb640 1 -- 172.21.15.143:0/3663185887 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0d28013470 con 0x7f0d2c070ba0 2023-11-26T07:21:52.840 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:21:52.836+0000 7f0d336d6640 1 -- 172.21.15.143:0/3663185887 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0d1c009420 con 0x7f0d2c070ba0 2023-11-26T07:21:52.840 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:21:52.836+0000 7f0d336d6640 1 -- 172.21.15.143:0/3663185887 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0d2c0b40a0 con 0x7f0d2c070ba0 2023-11-26T07:21:52.840 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:21:52.836+0000 7f0d336d6640 1 -- 172.21.15.143:0/3663185887 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0d2c067680 con 0x7f0d2c070ba0 2023-11-26T07:21:52.840 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:21:52.840+0000 7f0d21ffb640 1 -- 172.21.15.143:0/3663185887 <== mon.0 v2:172.21.15.143:3300/0 4 ==== mgrmap(e 9) v1 ==== 86005+0+0 (secure 0 0 0) 0x7f0d280135d0 con 0x7f0d2c070ba0 2023-11-26T07:21:52.842 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:21:52.840+0000 7f0d21ffb640 1 --2- 172.21.15.143:0/3663185887 >> [v2:172.21.15.143:6832/2150557544,v1:172.21.15.143:6833/2150557544] conn(0x7f0d10066620 0x7f0d10068ae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:21:52.842 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:21:52.840+0000 7f0d21ffb640 1 -- 172.21.15.143:0/3663185887 <== mon.0 v2:172.21.15.143:3300/0 5 ==== osd_map(55..55 src has 1..55) v4 ==== 4857+0+0 (secure 0 0 0) 0x7f0d280848d0 con 0x7f0d2c070ba0 2023-11-26T07:21:52.842 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:21:52.840+0000 7f0d30c71640 1 --2- 172.21.15.143:0/3663185887 >> [v2:172.21.15.143:6832/2150557544,v1:172.21.15.143:6833/2150557544] conn(0x7f0d10066620 0x7f0d10068ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:21:52.843 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:21:52.840+0000 7f0d30c71640 1 --2- 172.21.15.143:0/3663185887 >> [v2:172.21.15.143:6832/2150557544,v1:172.21.15.143:6833/2150557544] conn(0x7f0d10066620 0x7f0d10068ae0 secure :-1 s=READY pgs=374 cs=0 l=1 rev1=1 crypto rx=0x7f0d1c0af000 tx=0x7f0d1c0a7000 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:21:52.844 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:21:52.844+0000 7f0d21ffb640 1 -- 172.21.15.143:0/3663185887 <== mon.0 v2:172.21.15.143:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172649 (secure 0 0 0) 0x7f0d28051f00 con 0x7f0d2c070ba0 2023-11-26T07:21:52.998 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:21:52.992+0000 7f0d336d6640 1 -- 172.21.15.143:0/3663185887 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "osd lspools", "format": "json-pretty"} v 0) v1 -- 0x7f0d2c0d0510 con 0x7f0d2c070ba0 2023-11-26T07:21:53.000 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:21:52.992+0000 7f0d21ffb640 1 -- 172.21.15.143:0/3663185887 <== mon.0 v2:172.21.15.143:3300/0 7 ==== mon_command_ack([{"prefix": "osd lspools", "format": "json-pretty"}]=0 v55) v1 ==== 84+0+65 (secure 0 0 0) 0x7f0d28021090 con 0x7f0d2c070ba0 2023-11-26T07:21:53.000 INFO:teuthology.orchestra.run.smithi143.stdout: 2023-11-26T07:21:53.000 INFO:teuthology.orchestra.run.smithi143.stdout:[ 2023-11-26T07:21:53.000 INFO:teuthology.orchestra.run.smithi143.stdout: { 2023-11-26T07:21:53.001 INFO:teuthology.orchestra.run.smithi143.stdout: "poolnum": 1, 2023-11-26T07:21:53.001 INFO:teuthology.orchestra.run.smithi143.stdout: "poolname": ".mgr" 2023-11-26T07:21:53.001 INFO:teuthology.orchestra.run.smithi143.stdout: } 2023-11-26T07:21:53.001 INFO:teuthology.orchestra.run.smithi143.stdout:] 2023-11-26T07:21:53.002 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:21:52.996+0000 7f0d336d6640 1 -- 172.21.15.143:0/3663185887 >> [v2:172.21.15.143:6832/2150557544,v1:172.21.15.143:6833/2150557544] conn(0x7f0d10066620 msgr2=0x7f0d10068ae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:21:53.002 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:21:52.996+0000 7f0d336d6640 1 --2- 172.21.15.143:0/3663185887 >> [v2:172.21.15.143:6832/2150557544,v1:172.21.15.143:6833/2150557544] conn(0x7f0d10066620 0x7f0d10068ae0 secure :-1 s=READY pgs=374 cs=0 l=1 rev1=1 crypto rx=0x7f0d1c0af000 tx=0x7f0d1c0a7000 comp rx=0 tx=0).stop 2023-11-26T07:21:53.002 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:21:52.996+0000 7f0d336d6640 1 -- 172.21.15.143:0/3663185887 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f0d2c070ba0 msgr2=0x7f0d2c0b2d60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:21:53.003 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:21:52.996+0000 7f0d336d6640 1 --2- 172.21.15.143:0/3663185887 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f0d2c070ba0 0x7f0d2c0b2d60 secure :-1 s=READY pgs=780 cs=0 l=1 rev1=1 crypto rx=0x7f0d2800e9d0 tx=0x7f0d2800ee90 comp rx=0 tx=0).stop 2023-11-26T07:21:53.003 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:21:52.996+0000 7f0d336d6640 1 -- 172.21.15.143:0/3663185887 shutdown_connections 2023-11-26T07:21:53.004 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:21:52.996+0000 7f0d336d6640 1 --2- 172.21.15.143:0/3663185887 >> [v2:172.21.15.143:6832/2150557544,v1:172.21.15.143:6833/2150557544] conn(0x7f0d10066620 0x7f0d10068ae0 unknown :-1 s=CLOSED pgs=374 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:21:53.004 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:21:52.996+0000 7f0d336d6640 1 --2- 172.21.15.143:0/3663185887 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f0d2c1614b0 0x7f0d2c151170 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:21:53.004 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:21:52.996+0000 7f0d336d6640 1 --2- 172.21.15.143:0/3663185887 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f0d2c071970 0x7f0d2c0cdb00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:21:53.004 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:21:52.996+0000 7f0d336d6640 1 --2- 172.21.15.143:0/3663185887 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f0d2c070ba0 0x7f0d2c0b2d60 unknown :-1 s=CLOSED pgs=780 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:21:53.004 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:21:52.996+0000 7f0d336d6640 1 -- 172.21.15.143:0/3663185887 >> 172.21.15.143:0/3663185887 conn(0x7f0d2c0cf710 msgr2=0x7f0d2c15f100 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:21:53.004 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:21:52.996+0000 7f0d336d6640 1 -- 172.21.15.143:0/3663185887 shutdown_connections 2023-11-26T07:21:53.005 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:21:52.996+0000 7f0d336d6640 1 -- 172.21.15.143:0/3663185887 wait complete. 2023-11-26T07:21:53.019 DEBUG:tasks.cephfs.filesystem:Creating filesystem 'cephfs' 2023-11-26T07:21:53.020 DEBUG:teuthology.orchestra.run.smithi143:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_metadata --pg_num_min 64 2023-11-26T07:21:53.170 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:21:53.168+0000 7f73361df640 1 Processor -- start 2023-11-26T07:21:53.170 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:21:53.168+0000 7f73361df640 1 -- start start 2023-11-26T07:21:53.176 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:21:53.168+0000 7f73361df640 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7f73300cf280 0x7f73300cf710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:21:53.176 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:21:53.168+0000 7f73361df640 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7f73300cd1b0 0x7f73300cd5c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:21:53.176 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:21:53.168+0000 7f73361df640 1 -- --> v1:172.21.15.143:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f73300cc730 con 0x7f7330143ac0 2023-11-26T07:21:53.176 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:21:53.168+0000 7f73361df640 1 -- --> v2:172.21.15.143:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7330143ea0 con 0x7f73300cd1b0 2023-11-26T07:21:53.177 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:21:53.168+0000 7f73361df640 1 -- --> v2:172.21.15.188:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f73300cfc50 con 0x7f73300cf280 2023-11-26T07:21:53.177 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:21:53.168+0000 7f732ffff640 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7f73300cd1b0 0x7f73300cd5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:21:53.177 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:21:53.168+0000 7f732f7fe640 1 --1- >> v1:172.21.15.143:6789/0 conn(0x7f7330143ac0 0x7f7330070370 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.143:6789/0 says I am v1:172.21.15.143:52762/0 (socket says 172.21.15.143:52762) 2023-11-26T07:21:53.178 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:21:53.168+0000 7f732ffff640 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7f73300cd1b0 0x7f73300cd5c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:49494/0 (socket says 172.21.15.143:49494) 2023-11-26T07:21:53.178 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:21:53.168+0000 7f732f7fe640 1 -- 172.21.15.143:0/1548910640 learned_addr learned my addr 172.21.15.143:0/1548910640 (peer_addr_for_me v1:172.21.15.143:0/0) 2023-11-26T07:21:53.178 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:21:53.168+0000 7f732effd640 1 --2- 172.21.15.143:0/1548910640 >> v2:172.21.15.188:3300/0 conn(0x7f73300cf280 0x7f73300cf710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:21:53.178 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:21:53.168+0000 7f732ffff640 1 -- 172.21.15.143:0/1548910640 >> v2:172.21.15.188:3300/0 conn(0x7f73300cf280 msgr2=0x7f73300cf710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-26T07:21:53.178 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:21:53.168+0000 7f732ffff640 1 --2- 172.21.15.143:0/1548910640 >> v2:172.21.15.188:3300/0 conn(0x7f73300cf280 0x7f73300cf710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:21:53.179 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:21:53.168+0000 7f732ffff640 1 -- 172.21.15.143:0/1548910640 >> v1:172.21.15.143:6789/0 conn(0x7f7330143ac0 legacy=0x7f7330070370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:21:53.179 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:21:53.168+0000 7f732ffff640 1 -- 172.21.15.143:0/1548910640 --> v2:172.21.15.143:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7330070ee0 con 0x7f73300cd1b0 2023-11-26T07:21:53.179 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:21:53.168+0000 7f732effd640 1 --2- 172.21.15.143:0/1548910640 >> v2:172.21.15.188:3300/0 conn(0x7f73300cf280 0x7f73300cf710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-26T07:21:53.179 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:21:53.172+0000 7f732ffff640 1 --2- 172.21.15.143:0/1548910640 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f73300cd1b0 0x7f73300cd5c0 secure :-1 s=READY pgs=782 cs=0 l=1 rev1=1 crypto rx=0x7f73300aeb10 tx=0x7f732801c910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d5c62b7c338cb342 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:21:53.180 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:21:53.172+0000 7f732e7fc640 1 -- 172.21.15.143:0/1548910640 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f73280a59d0 con 0x7f73300cd1b0 2023-11-26T07:21:53.180 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:21:53.172+0000 7f732e7fc640 1 -- 172.21.15.143:0/1548910640 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f73280a5b30 con 0x7f73300cd1b0 2023-11-26T07:21:53.180 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:21:53.172+0000 7f732e7fc640 1 -- 172.21.15.143:0/1548910640 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f73280ad440 con 0x7f73300cd1b0 2023-11-26T07:21:53.180 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:21:53.172+0000 7f73361df640 1 -- 172.21.15.143:0/1548910640 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f73300cd1b0 msgr2=0x7f73300cd5c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:21:53.180 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:21:53.172+0000 7f73361df640 1 --2- 172.21.15.143:0/1548910640 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f73300cd1b0 0x7f73300cd5c0 secure :-1 s=READY pgs=782 cs=0 l=1 rev1=1 crypto rx=0x7f73300aeb10 tx=0x7f732801c910 comp rx=0 tx=0).stop 2023-11-26T07:21:53.181 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:21:53.172+0000 7f73361df640 1 -- 172.21.15.143:0/1548910640 shutdown_connections 2023-11-26T07:21:53.181 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:21:53.172+0000 7f73361df640 1 --2- 172.21.15.143:0/1548910640 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f73300cd1b0 0x7f73300cd5c0 unknown :-1 s=CLOSED pgs=782 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:21:53.181 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:21:53.172+0000 7f73361df640 1 --2- 172.21.15.143:0/1548910640 >> v2:172.21.15.188:3300/0 conn(0x7f73300cf280 0x7f73300cf710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:21:53.181 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:21:53.172+0000 7f73361df640 1 -- 172.21.15.143:0/1548910640 >> 172.21.15.143:0/1548910640 conn(0x7f733006a890 msgr2=0x7f733006aca0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:21:53.181 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:21:53.172+0000 7f73361df640 1 -- 172.21.15.143:0/1548910640 shutdown_connections 2023-11-26T07:21:53.182 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:21:53.172+0000 7f73361df640 1 -- 172.21.15.143:0/1548910640 wait complete. 2023-11-26T07:21:53.188 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:21:53.172+0000 7f73361df640 1 Processor -- start 2023-11-26T07:21:53.188 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:21:53.176+0000 7f73361df640 1 -- start start 2023-11-26T07:21:53.188 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:21:53.176+0000 7f73361df640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f73300cd1b0 0x7f7330140270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:21:53.189 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:21:53.176+0000 7f73361df640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f73300cf280 0x7f7330140f80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:21:53.189 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:21:53.176+0000 7f73361df640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f7330143ac0 0x7f73301414c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:21:53.189 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:21:53.176+0000 7f73361df640 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f733013dd10 con 0x7f73300cd1b0 2023-11-26T07:21:53.189 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:21:53.176+0000 7f73361df640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f73301367b0 con 0x7f7330143ac0 2023-11-26T07:21:53.189 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:21:53.176+0000 7f73361df640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7330136a90 con 0x7f73300cf280 2023-11-26T07:21:53.190 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:21:53.176+0000 7f732f7fe640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f73300cd1b0 0x7f7330140270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:21:53.190 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:21:53.176+0000 7f732f7fe640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f73300cd1b0 0x7f7330140270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:49506/0 (socket says 172.21.15.143:49506) 2023-11-26T07:21:53.190 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:21:53.176+0000 7f732f7fe640 1 -- 172.21.15.143:0/2932702314 learned_addr learned my addr 172.21.15.143:0/2932702314 (peer_addr_for_me v2:172.21.15.143:0/0) 2023-11-26T07:21:53.190 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:21:53.176+0000 7f732effd640 1 --2- 172.21.15.143:0/2932702314 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f73300cf280 0x7f7330140f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:21:53.191 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:21:53.176+0000 7f732ffff640 1 --2- 172.21.15.143:0/2932702314 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f7330143ac0 0x7f73301414c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:21:53.191 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:21:53.176+0000 7f732f7fe640 1 -- 172.21.15.143:0/2932702314 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f73300cf280 msgr2=0x7f7330140f80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:21:53.191 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:21:53.176+0000 7f732f7fe640 1 --2- 172.21.15.143:0/2932702314 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f73300cf280 0x7f7330140f80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:21:53.191 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:21:53.176+0000 7f732f7fe640 1 -- 172.21.15.143:0/2932702314 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f7330143ac0 msgr2=0x7f73301414c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:21:53.192 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:21:53.176+0000 7f732f7fe640 1 --2- 172.21.15.143:0/2932702314 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f7330143ac0 0x7f73301414c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:21:53.192 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:21:53.176+0000 7f732f7fe640 1 -- 172.21.15.143:0/2932702314 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7328009a70 con 0x7f73300cd1b0 2023-11-26T07:21:53.192 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:21:53.176+0000 7f732ffff640 1 --2- 172.21.15.143:0/2932702314 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f7330143ac0 0x7f73301414c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-26T07:21:53.192 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:21:53.176+0000 7f732effd640 1 --2- 172.21.15.143:0/2932702314 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f73300cf280 0x7f7330140f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-26T07:21:53.192 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:21:53.176+0000 7f732f7fe640 1 --2- 172.21.15.143:0/2932702314 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f73300cd1b0 0x7f7330140270 secure :-1 s=READY pgs=783 cs=0 l=1 rev1=1 crypto rx=0x7f732000c500 tx=0x7f732000c9c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:21:53.193 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:21:53.176+0000 7f7317fff640 1 -- 172.21.15.143:0/2932702314 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7320005cf0 con 0x7f73300cd1b0 2023-11-26T07:21:53.193 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:21:53.176+0000 7f7317fff640 1 -- 172.21.15.143:0/2932702314 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7320005e50 con 0x7f73300cd1b0 2023-11-26T07:21:53.193 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:21:53.176+0000 7f73361df640 1 -- 172.21.15.143:0/2932702314 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f733013df60 con 0x7f73300cd1b0 2023-11-26T07:21:53.193 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:21:53.176+0000 7f7317fff640 1 -- 172.21.15.143:0/2932702314 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f73200072f0 con 0x7f73300cd1b0 2023-11-26T07:21:53.193 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:21:53.176+0000 7f73361df640 1 -- 172.21.15.143:0/2932702314 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f73301407b0 con 0x7f73300cd1b0 2023-11-26T07:21:53.193 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:21:53.176+0000 7f73361df640 1 -- 172.21.15.143:0/2932702314 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f733015d560 con 0x7f73300cd1b0 2023-11-26T07:21:53.193 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:21:53.180+0000 7f7317fff640 1 -- 172.21.15.143:0/2932702314 <== mon.0 v2:172.21.15.143:3300/0 4 ==== mgrmap(e 9) v1 ==== 86005+0+0 (secure 0 0 0) 0x7f7320007450 con 0x7f73300cd1b0 2023-11-26T07:21:53.194 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:21:53.180+0000 7f7317fff640 1 --2- 172.21.15.143:0/2932702314 >> [v2:172.21.15.143:6832/2150557544,v1:172.21.15.143:6833/2150557544] conn(0x7f731c066650 0x7f731c068b10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:21:53.194 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:21:53.180+0000 7f7317fff640 1 -- 172.21.15.143:0/2932702314 <== mon.0 v2:172.21.15.143:3300/0 5 ==== osd_map(55..55 src has 1..55) v4 ==== 4857+0+0 (secure 0 0 0) 0x7f7320083300 con 0x7f73300cd1b0 2023-11-26T07:21:53.194 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:21:53.180+0000 7f732effd640 1 --2- 172.21.15.143:0/2932702314 >> [v2:172.21.15.143:6832/2150557544,v1:172.21.15.143:6833/2150557544] conn(0x7f731c066650 0x7f731c068b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:21:53.194 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:21:53.180+0000 7f732effd640 1 --2- 172.21.15.143:0/2932702314 >> [v2:172.21.15.143:6832/2150557544,v1:172.21.15.143:6833/2150557544] conn(0x7f731c066650 0x7f731c068b10 secure :-1 s=READY pgs=375 cs=0 l=1 rev1=1 crypto rx=0x7f7330140d60 tx=0x7f7324008040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:21:53.194 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:21:53.184+0000 7f7317fff640 1 -- 172.21.15.143:0/2932702314 <== mon.0 v2:172.21.15.143:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172649 (secure 0 0 0) 0x7f73200508f0 con 0x7f73300cd1b0 2023-11-26T07:21:53.341 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:21:53.336+0000 7f73361df640 1 -- 172.21.15.143:0/2932702314 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64} v 0) v1 -- 0x7f7330067680 con 0x7f73300cd1b0 2023-11-26T07:21:54.263 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:21:54.260+0000 7f7317fff640 1 -- 172.21.15.143:0/2932702314 <== mon.0 v2:172.21.15.143:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]=0 pool 'cephfs_metadata' created v56) v1 ==== 138+0+0 (secure 0 0 0) 0x7f7320053f10 con 0x7f73300cd1b0 2023-11-26T07:21:54.264 INFO:teuthology.orchestra.run.smithi143.stderr:pool 'cephfs_metadata' created 2023-11-26T07:21:54.267 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:21:54.264+0000 7f73361df640 1 -- 172.21.15.143:0/2932702314 >> [v2:172.21.15.143:6832/2150557544,v1:172.21.15.143:6833/2150557544] conn(0x7f731c066650 msgr2=0x7f731c068b10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:21:54.267 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:21:54.264+0000 7f73361df640 1 --2- 172.21.15.143:0/2932702314 >> [v2:172.21.15.143:6832/2150557544,v1:172.21.15.143:6833/2150557544] conn(0x7f731c066650 0x7f731c068b10 secure :-1 s=READY pgs=375 cs=0 l=1 rev1=1 crypto rx=0x7f7330140d60 tx=0x7f7324008040 comp rx=0 tx=0).stop 2023-11-26T07:21:54.267 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:21:54.264+0000 7f73361df640 1 -- 172.21.15.143:0/2932702314 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f73300cd1b0 msgr2=0x7f7330140270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:21:54.267 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:21:54.264+0000 7f73361df640 1 --2- 172.21.15.143:0/2932702314 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f73300cd1b0 0x7f7330140270 secure :-1 s=READY pgs=783 cs=0 l=1 rev1=1 crypto rx=0x7f732000c500 tx=0x7f732000c9c0 comp rx=0 tx=0).stop 2023-11-26T07:21:54.268 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:21:54.264+0000 7f73361df640 1 -- 172.21.15.143:0/2932702314 shutdown_connections 2023-11-26T07:21:54.268 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:21:54.264+0000 7f73361df640 1 --2- 172.21.15.143:0/2932702314 >> [v2:172.21.15.143:6832/2150557544,v1:172.21.15.143:6833/2150557544] conn(0x7f731c066650 0x7f731c068b10 unknown :-1 s=CLOSED pgs=375 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:21:54.268 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:21:54.264+0000 7f73361df640 1 --2- 172.21.15.143:0/2932702314 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f7330143ac0 0x7f73301414c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:21:54.268 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:21:54.264+0000 7f73361df640 1 --2- 172.21.15.143:0/2932702314 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f73300cf280 0x7f7330140f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:21:54.268 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:21:54.264+0000 7f73361df640 1 --2- 172.21.15.143:0/2932702314 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f73300cd1b0 0x7f7330140270 unknown :-1 s=CLOSED pgs=783 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:21:54.268 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:21:54.264+0000 7f73361df640 1 -- 172.21.15.143:0/2932702314 >> 172.21.15.143:0/2932702314 conn(0x7f733006a890 msgr2=0x7f73300cb0e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:21:54.268 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:21:54.264+0000 7f73361df640 1 -- 172.21.15.143:0/2932702314 shutdown_connections 2023-11-26T07:21:54.268 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:21:54.264+0000 7f73361df640 1 -- 172.21.15.143:0/2932702314 wait complete. 2023-11-26T07:21:54.296 DEBUG:teuthology.orchestra.run.smithi143:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_data 64 --pg_num_min 64 --target_size_ratio 0.9 2023-11-26T07:21:54.443 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:21:54.440+0000 7fb402594640 1 Processor -- start 2023-11-26T07:21:54.444 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:21:54.440+0000 7fb402594640 1 -- start start 2023-11-26T07:21:54.445 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:21:54.444+0000 7fb402594640 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7fb3fc143c00 0x7fb3fc0b1ef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:21:54.446 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:21:54.444+0000 7fb402594640 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7fb3fc070390 0x7fb3fc0cc7a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:21:54.446 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:21:54.444+0000 7fb402594640 1 -- --> v1:172.21.15.188:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb3fc0b25f0 con 0x7fb3fc0ccce0 2023-11-26T07:21:54.446 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:21:54.444+0000 7fb402594640 1 -- --> v2:172.21.15.143:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb3fc0b28d0 con 0x7fb3fc070390 2023-11-26T07:21:54.446 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:21:54.444+0000 7fb402594640 1 -- --> v2:172.21.15.188:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb3fc0b2bb0 con 0x7fb3fc143c00 2023-11-26T07:21:54.446 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:21:54.444+0000 7fb3fb7fe640 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7fb3fc070390 0x7fb3fc0cc7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:21:54.446 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:21:54.444+0000 7fb3fb7fe640 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7fb3fc070390 0x7fb3fc0cc7a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:57140/0 (socket says 172.21.15.143:57140) 2023-11-26T07:21:54.446 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:21:54.444+0000 7fb3fb7fe640 1 -- 172.21.15.143:0/384919122 learned_addr learned my addr 172.21.15.143:0/384919122 (peer_addr_for_me v2:172.21.15.143:0/0) 2023-11-26T07:21:54.446 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:21:54.444+0000 7fb3fbfff640 1 --2- 172.21.15.143:0/384919122 >> v2:172.21.15.188:3300/0 conn(0x7fb3fc143c00 0x7fb3fc0b1ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:21:54.446 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:21:54.444+0000 7fb3fb7fe640 1 -- 172.21.15.143:0/384919122 >> v2:172.21.15.188:3300/0 conn(0x7fb3fc143c00 msgr2=0x7fb3fc0b1ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-26T07:21:54.446 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:21:54.444+0000 7fb3fb7fe640 1 --2- 172.21.15.143:0/384919122 >> v2:172.21.15.188:3300/0 conn(0x7fb3fc143c00 0x7fb3fc0b1ef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:21:54.446 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:21:54.444+0000 7fb3fb7fe640 1 -- 172.21.15.143:0/384919122 >> v1:172.21.15.188:6789/0 conn(0x7fb3fc0ccce0 legacy=0x7fb3fc071550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:21:54.447 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:21:54.444+0000 7fb3fb7fe640 1 -- 172.21.15.143:0/384919122 --> v2:172.21.15.143:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb3fc0b2e90 con 0x7fb3fc070390 2023-11-26T07:21:54.448 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:21:54.448+0000 7fb3fb7fe640 1 --2- 172.21.15.143:0/384919122 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fb3fc070390 0x7fb3fc0cc7a0 secure :-1 s=READY pgs=784 cs=0 l=1 rev1=1 crypto rx=0x7fb3fc0cfef0 tx=0x7fb3f001c470 comp rx=0 tx=0).ready entity=mon.0 client_cookie=48cde09e65755a98 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:21:54.449 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:21:54.448+0000 7fb3faffd640 1 -- 172.21.15.143:0/384919122 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb3f00aa5f0 con 0x7fb3fc070390 2023-11-26T07:21:54.449 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:21:54.448+0000 7fb3faffd640 1 -- 172.21.15.143:0/384919122 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb3f00aa750 con 0x7fb3fc070390 2023-11-26T07:21:54.449 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:21:54.448+0000 7fb3faffd640 1 -- 172.21.15.143:0/384919122 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb3f00aaae0 con 0x7fb3fc070390 2023-11-26T07:21:54.450 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:21:54.448+0000 7fb402594640 1 -- 172.21.15.143:0/384919122 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fb3fc070390 msgr2=0x7fb3fc0cc7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:21:54.450 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:21:54.448+0000 7fb402594640 1 --2- 172.21.15.143:0/384919122 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fb3fc070390 0x7fb3fc0cc7a0 secure :-1 s=READY pgs=784 cs=0 l=1 rev1=1 crypto rx=0x7fb3fc0cfef0 tx=0x7fb3f001c470 comp rx=0 tx=0).stop 2023-11-26T07:21:54.450 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:21:54.448+0000 7fb402594640 1 -- 172.21.15.143:0/384919122 shutdown_connections 2023-11-26T07:21:54.450 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:21:54.448+0000 7fb402594640 1 --2- 172.21.15.143:0/384919122 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fb3fc070390 0x7fb3fc0cc7a0 unknown :-1 s=CLOSED pgs=784 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:21:54.450 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:21:54.448+0000 7fb402594640 1 --2- 172.21.15.143:0/384919122 >> v2:172.21.15.188:3300/0 conn(0x7fb3fc143c00 0x7fb3fc0b1ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:21:54.450 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:21:54.448+0000 7fb402594640 1 -- 172.21.15.143:0/384919122 >> 172.21.15.143:0/384919122 conn(0x7fb3fc0b0bf0 msgr2=0x7fb3fc0b1000 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:21:54.450 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:21:54.448+0000 7fb402594640 1 -- 172.21.15.143:0/384919122 shutdown_connections 2023-11-26T07:21:54.450 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:21:54.448+0000 7fb402594640 1 -- 172.21.15.143:0/384919122 wait complete. 2023-11-26T07:21:54.451 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:21:54.448+0000 7fb402594640 1 Processor -- start 2023-11-26T07:21:54.451 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:21:54.448+0000 7fb402594640 1 -- start start 2023-11-26T07:21:54.453 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:21:54.452+0000 7fb402594640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fb3fc070390 0x7fb3fc138450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:21:54.453 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:21:54.452+0000 7fb402594640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fb3fc0ccce0 0x7fb3fc157bb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:21:54.453 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:21:54.452+0000 7fb402594640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fb3fc143c00 0x7fb3fc1580f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:21:54.453 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:21:54.452+0000 7fb402594640 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb3fc155480 con 0x7fb3fc070390 2023-11-26T07:21:54.453 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:21:54.452+0000 7fb402594640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb3fc1548b0 con 0x7fb3fc0ccce0 2023-11-26T07:21:54.453 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:21:54.452+0000 7fb402594640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb3fc154b50 con 0x7fb3fc143c00 2023-11-26T07:21:54.453 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:21:54.452+0000 7fb3fbfff640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fb3fc070390 0x7fb3fc138450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:21:54.454 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:21:54.452+0000 7fb3fbfff640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fb3fc070390 0x7fb3fc138450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:57144/0 (socket says 172.21.15.143:57144) 2023-11-26T07:21:54.454 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:21:54.452+0000 7fb3fbfff640 1 -- 172.21.15.143:0/901540896 learned_addr learned my addr 172.21.15.143:0/901540896 (peer_addr_for_me v2:172.21.15.143:0/0) 2023-11-26T07:21:54.454 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:21:54.452+0000 7fb3fb7fe640 1 --2- 172.21.15.143:0/901540896 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fb3fc0ccce0 0x7fb3fc157bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:21:54.454 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:21:54.452+0000 7fb400b31640 1 --2- 172.21.15.143:0/901540896 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fb3fc143c00 0x7fb3fc1580f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:21:54.454 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:21:54.452+0000 7fb3fbfff640 1 -- 172.21.15.143:0/901540896 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fb3fc143c00 msgr2=0x7fb3fc1580f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:21:54.454 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:21:54.452+0000 7fb3fbfff640 1 --2- 172.21.15.143:0/901540896 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fb3fc143c00 0x7fb3fc1580f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:21:54.454 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:21:54.452+0000 7fb3fbfff640 1 -- 172.21.15.143:0/901540896 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fb3fc0ccce0 msgr2=0x7fb3fc157bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:21:54.454 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:21:54.452+0000 7fb3fbfff640 1 --2- 172.21.15.143:0/901540896 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fb3fc0ccce0 0x7fb3fc157bb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:21:54.454 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:21:54.452+0000 7fb3fbfff640 1 -- 172.21.15.143:0/901540896 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb3f00094d0 con 0x7fb3fc070390 2023-11-26T07:21:54.455 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:21:54.452+0000 7fb3fbfff640 1 --2- 172.21.15.143:0/901540896 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fb3fc070390 0x7fb3fc138450 secure :-1 s=READY pgs=785 cs=0 l=1 rev1=1 crypto rx=0x7fb3ec00d5f0 tx=0x7fb3ec00dab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:21:54.455 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:21:54.452+0000 7fb3f8ff9640 1 -- 172.21.15.143:0/901540896 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb3ec0125f0 con 0x7fb3fc070390 2023-11-26T07:21:54.455 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:21:54.452+0000 7fb3f8ff9640 1 -- 172.21.15.143:0/901540896 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb3ec012750 con 0x7fb3fc070390 2023-11-26T07:21:54.455 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:21:54.452+0000 7fb3f8ff9640 1 -- 172.21.15.143:0/901540896 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb3ec012b00 con 0x7fb3fc070390 2023-11-26T07:21:54.455 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:21:54.452+0000 7fb402594640 1 -- 172.21.15.143:0/901540896 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb3fc154e30 con 0x7fb3fc070390 2023-11-26T07:21:54.456 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:21:54.452+0000 7fb402594640 1 -- 172.21.15.143:0/901540896 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb3fc13e7b0 con 0x7fb3fc070390 2023-11-26T07:21:54.456 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:21:54.452+0000 7fb402594640 1 -- 172.21.15.143:0/901540896 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb3fc067720 con 0x7fb3fc070390 2023-11-26T07:21:54.456 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:21:54.452+0000 7fb3f8ff9640 1 -- 172.21.15.143:0/901540896 <== mon.0 v2:172.21.15.143:3300/0 4 ==== mgrmap(e 9) v1 ==== 86005+0+0 (secure 0 0 0) 0x7fb3ec012c60 con 0x7fb3fc070390 2023-11-26T07:21:54.457 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:21:54.456+0000 7fb3f8ff9640 1 --2- 172.21.15.143:0/901540896 >> [v2:172.21.15.143:6832/2150557544,v1:172.21.15.143:6833/2150557544] conn(0x7fb3e8066620 0x7fb3e8068ae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:21:54.457 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:21:54.456+0000 7fb3f8ff9640 1 -- 172.21.15.143:0/901540896 <== mon.0 v2:172.21.15.143:3300/0 5 ==== osd_map(56..56 src has 1..56) v4 ==== 5227+0+0 (secure 0 0 0) 0x7fb3ec084440 con 0x7fb3fc070390 2023-11-26T07:21:54.457 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:21:54.456+0000 7fb3fb7fe640 1 --2- 172.21.15.143:0/901540896 >> [v2:172.21.15.143:6832/2150557544,v1:172.21.15.143:6833/2150557544] conn(0x7fb3e8066620 0x7fb3e8068ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:21:54.458 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:21:54.456+0000 7fb3fb7fe640 1 --2- 172.21.15.143:0/901540896 >> [v2:172.21.15.143:6832/2150557544,v1:172.21.15.143:6833/2150557544] conn(0x7fb3e8066620 0x7fb3e8068ae0 secure :-1 s=READY pgs=376 cs=0 l=1 rev1=1 crypto rx=0x7fb3f00ae000 tx=0x7fb3f00b5040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:21:54.461 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:21:54.460+0000 7fb3f8ff9640 1 -- 172.21.15.143:0/901540896 <== mon.0 v2:172.21.15.143:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172649 (secure 0 0 0) 0x7fb3ec051840 con 0x7fb3fc070390 2023-11-26T07:21:54.613 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:21:54.612+0000 7fb402594640 1 -- 172.21.15.143:0/901540896 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9} v 0) v1 -- 0x7fb3fc0b17a0 con 0x7fb3fc070390 2023-11-26T07:21:55.272 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:21:55.268+0000 7fb3f8ff9640 1 -- 172.21.15.143:0/901540896 <== mon.0 v2:172.21.15.143:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]=0 pool 'cephfs_data' created v57) v1 ==== 170+0+0 (secure 0 0 0) 0x7fb3ec054e60 con 0x7fb3fc070390 2023-11-26T07:21:55.272 INFO:teuthology.orchestra.run.smithi143.stderr:pool 'cephfs_data' created 2023-11-26T07:21:55.275 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:21:55.272+0000 7fb402594640 1 -- 172.21.15.143:0/901540896 >> [v2:172.21.15.143:6832/2150557544,v1:172.21.15.143:6833/2150557544] conn(0x7fb3e8066620 msgr2=0x7fb3e8068ae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:21:55.275 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:21:55.272+0000 7fb402594640 1 --2- 172.21.15.143:0/901540896 >> [v2:172.21.15.143:6832/2150557544,v1:172.21.15.143:6833/2150557544] conn(0x7fb3e8066620 0x7fb3e8068ae0 secure :-1 s=READY pgs=376 cs=0 l=1 rev1=1 crypto rx=0x7fb3f00ae000 tx=0x7fb3f00b5040 comp rx=0 tx=0).stop 2023-11-26T07:21:55.275 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:21:55.272+0000 7fb402594640 1 -- 172.21.15.143:0/901540896 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fb3fc070390 msgr2=0x7fb3fc138450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:21:55.275 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:21:55.272+0000 7fb402594640 1 --2- 172.21.15.143:0/901540896 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fb3fc070390 0x7fb3fc138450 secure :-1 s=READY pgs=785 cs=0 l=1 rev1=1 crypto rx=0x7fb3ec00d5f0 tx=0x7fb3ec00dab0 comp rx=0 tx=0).stop 2023-11-26T07:21:55.276 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:21:55.272+0000 7fb402594640 1 -- 172.21.15.143:0/901540896 shutdown_connections 2023-11-26T07:21:55.276 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:21:55.272+0000 7fb402594640 1 --2- 172.21.15.143:0/901540896 >> [v2:172.21.15.143:6832/2150557544,v1:172.21.15.143:6833/2150557544] conn(0x7fb3e8066620 0x7fb3e8068ae0 unknown :-1 s=CLOSED pgs=376 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:21:55.276 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:21:55.272+0000 7fb402594640 1 --2- 172.21.15.143:0/901540896 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fb3fc143c00 0x7fb3fc1580f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:21:55.276 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:21:55.272+0000 7fb402594640 1 --2- 172.21.15.143:0/901540896 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fb3fc0ccce0 0x7fb3fc157bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:21:55.276 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:21:55.272+0000 7fb402594640 1 --2- 172.21.15.143:0/901540896 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fb3fc070390 0x7fb3fc138450 unknown :-1 s=CLOSED pgs=785 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:21:55.276 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:21:55.272+0000 7fb402594640 1 -- 172.21.15.143:0/901540896 >> 172.21.15.143:0/901540896 conn(0x7fb3fc0b0bf0 msgr2=0x7fb3fc0b0fd0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:21:55.277 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:21:55.272+0000 7fb402594640 1 -- 172.21.15.143:0/901540896 shutdown_connections 2023-11-26T07:21:55.278 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:21:55.276+0000 7fb402594640 1 -- 172.21.15.143:0/901540896 wait complete. 2023-11-26T07:21:55.316 DEBUG:teuthology.orchestra.run.smithi143:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs new cephfs cephfs_metadata cephfs_data 2023-11-26T07:21:55.457 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:21:55.456+0000 7fac0949a640 1 Processor -- start 2023-11-26T07:21:55.457 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:21:55.456+0000 7fac0949a640 1 -- start start 2023-11-26T07:21:55.458 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:21:55.456+0000 7fac0949a640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fac040cf280 0x7fac040cf660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:21:55.459 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:21:55.456+0000 7fac0949a640 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7fac0406cd50 0x7fac040b2180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:21:55.459 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:21:55.456+0000 7fac0949a640 1 -- --> v1:172.21.15.188:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fac0415b2f0 con 0x7fac04070d10 2023-11-26T07:21:55.459 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:21:55.456+0000 7fac0949a640 1 -- --> v2:172.21.15.188:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fac0406c5c0 con 0x7fac0406cd50 2023-11-26T07:21:55.459 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:21:55.456+0000 7fac0949a640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fac040cfba0 con 0x7fac040cf280 2023-11-26T07:21:55.460 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:21:55.456+0000 7fac02ffd640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fac040cf280 0x7fac040cf660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:21:55.460 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:21:55.456+0000 7fac037fe640 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7fac0406cd50 0x7fac040b2180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:21:55.461 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:21:55.456+0000 7fac02ffd640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fac040cf280 0x7fac040cf660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.188:3301/0 says I am v2:172.21.15.143:55098/0 (socket says 172.21.15.143:55098) 2023-11-26T07:21:55.461 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:21:55.456+0000 7fac02ffd640 1 -- 172.21.15.143:0/2429492774 learned_addr learned my addr 172.21.15.143:0/2429492774 (peer_addr_for_me v2:172.21.15.143:0/0) 2023-11-26T07:21:55.461 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:21:55.456+0000 7fac02ffd640 1 -- 172.21.15.143:0/2429492774 >> v2:172.21.15.188:3300/0 conn(0x7fac0406cd50 msgr2=0x7fac040b2180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-26T07:21:55.461 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:21:55.456+0000 7fac02ffd640 1 --2- 172.21.15.143:0/2429492774 >> v2:172.21.15.188:3300/0 conn(0x7fac0406cd50 0x7fac040b2180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:21:55.461 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:21:55.456+0000 7fac02ffd640 1 -- 172.21.15.143:0/2429492774 >> v1:172.21.15.188:6789/0 conn(0x7fac04070d10 legacy=0x7fac0415d900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:21:55.461 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:21:55.456+0000 7fac02ffd640 1 -- 172.21.15.143:0/2429492774 --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fac0415e1a0 con 0x7fac040cf280 2023-11-26T07:21:55.462 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:21:55.456+0000 7fac027fc640 1 -- 172.21.15.143:0/2429492774 <== mon.1 v1:172.21.15.188:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1949306767 0 0) 0x7fabec001120 con 0x7fac04070d10 2023-11-26T07:21:55.462 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:21:55.460+0000 7fac02ffd640 1 --2- 172.21.15.143:0/2429492774 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fac040cf280 0x7fac040cf660 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7fabf4009870 tx=0x7fabf401ed30 comp rx=0 tx=0).ready entity=mon.2 client_cookie=2c34aaa6ba8d3845 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:21:55.463 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:21:55.460+0000 7fac027fc640 1 -- 172.21.15.143:0/2429492774 <== mon.2 v2:172.21.15.188:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fabf40a8a30 con 0x7fac040cf280 2023-11-26T07:21:55.463 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:21:55.460+0000 7fac027fc640 1 -- 172.21.15.143:0/2429492774 <== mon.2 v2:172.21.15.188:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fabf40a8b90 con 0x7fac040cf280 2023-11-26T07:21:55.463 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:21:55.460+0000 7fac027fc640 1 -- 172.21.15.143:0/2429492774 <== mon.2 v2:172.21.15.188:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fabf40b0440 con 0x7fac040cf280 2023-11-26T07:21:55.464 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:21:55.460+0000 7fac0949a640 1 -- 172.21.15.143:0/2429492774 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fac040cf280 msgr2=0x7fac040cf660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:21:55.464 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:21:55.460+0000 7fac0949a640 1 --2- 172.21.15.143:0/2429492774 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fac040cf280 0x7fac040cf660 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7fabf4009870 tx=0x7fabf401ed30 comp rx=0 tx=0).stop 2023-11-26T07:21:55.464 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:21:55.460+0000 7fac0949a640 1 -- 172.21.15.143:0/2429492774 shutdown_connections 2023-11-26T07:21:55.465 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:21:55.460+0000 7fac0949a640 1 --2- 172.21.15.143:0/2429492774 >> v2:172.21.15.188:3300/0 conn(0x7fac0406cd50 0x7fac040b2180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:21:55.465 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:21:55.460+0000 7fac0949a640 1 --2- 172.21.15.143:0/2429492774 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fac040cf280 0x7fac040cf660 unknown :-1 s=CLOSED pgs=106 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:21:55.465 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:21:55.460+0000 7fac0949a640 1 -- 172.21.15.143:0/2429492774 >> 172.21.15.143:0/2429492774 conn(0x7fac0406a890 msgr2=0x7fac0406aca0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:21:55.465 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:21:55.460+0000 7fac0949a640 1 -- 172.21.15.143:0/2429492774 shutdown_connections 2023-11-26T07:21:55.465 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:21:55.460+0000 7fac0949a640 1 -- 172.21.15.143:0/2429492774 wait complete. 2023-11-26T07:21:55.468 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:21:55.464+0000 7fac0949a640 1 Processor -- start 2023-11-26T07:21:55.468 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:21:55.464+0000 7fac0949a640 1 -- start start 2023-11-26T07:21:55.468 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:21:55.464+0000 7fac0949a640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fac0406cd50 0x7fac040cc4a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:21:55.468 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:21:55.464+0000 7fac0949a640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fac04070d10 0x7fac0414ba40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:21:55.468 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:21:55.464+0000 7fac0949a640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fac040cf280 0x7fac040c4cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:21:55.469 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:21:55.464+0000 7fac0949a640 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fac040cc010 con 0x7fac0406cd50 2023-11-26T07:21:55.469 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:21:55.464+0000 7fac0949a640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fac040c5630 con 0x7fac040cf280 2023-11-26T07:21:55.469 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:21:55.464+0000 7fac0949a640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fac040cb9c0 con 0x7fac04070d10 2023-11-26T07:21:55.469 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:21:55.464+0000 7fac02ffd640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fac0406cd50 0x7fac040cc4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:21:55.469 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:21:55.464+0000 7fac02ffd640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fac0406cd50 0x7fac040cc4a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:57148/0 (socket says 172.21.15.143:57148) 2023-11-26T07:21:55.469 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:21:55.464+0000 7fac02ffd640 1 -- 172.21.15.143:0/3838491569 learned_addr learned my addr 172.21.15.143:0/3838491569 (peer_addr_for_me v2:172.21.15.143:0/0) 2023-11-26T07:21:55.470 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:21:55.464+0000 7fac037fe640 1 --2- 172.21.15.143:0/3838491569 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fac040cf280 0x7fac040c4cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:21:55.470 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:21:55.464+0000 7fabfbfff640 1 --2- 172.21.15.143:0/3838491569 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fac04070d10 0x7fac0414ba40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:21:55.470 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:21:55.464+0000 7fac02ffd640 1 -- 172.21.15.143:0/3838491569 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fac04070d10 msgr2=0x7fac0414ba40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:21:55.470 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:21:55.464+0000 7fac02ffd640 1 --2- 172.21.15.143:0/3838491569 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fac04070d10 0x7fac0414ba40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:21:55.470 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:21:55.464+0000 7fac02ffd640 1 -- 172.21.15.143:0/3838491569 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fac040cf280 msgr2=0x7fac040c4cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:21:55.470 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:21:55.464+0000 7fac02ffd640 1 --2- 172.21.15.143:0/3838491569 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fac040cf280 0x7fac040c4cc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:21:55.471 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:21:55.464+0000 7fac02ffd640 1 -- 172.21.15.143:0/3838491569 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fabf40094d0 con 0x7fac0406cd50 2023-11-26T07:21:55.471 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:21:55.464+0000 7fac02ffd640 1 --2- 172.21.15.143:0/3838491569 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fac0406cd50 0x7fac040cc4a0 secure :-1 s=READY pgs=786 cs=0 l=1 rev1=1 crypto rx=0x7fabf40b3040 tx=0x7fabf401bad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:21:55.471 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:21:55.464+0000 7fabfb7fe640 1 -- 172.21.15.143:0/3838491569 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fabf40b9620 con 0x7fac0406cd50 2023-11-26T07:21:55.471 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:21:55.464+0000 7fabfb7fe640 1 -- 172.21.15.143:0/3838491569 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fabf40b97c0 con 0x7fac0406cd50 2023-11-26T07:21:55.471 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:21:55.464+0000 7fabfb7fe640 1 -- 172.21.15.143:0/3838491569 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fabf40b9b70 con 0x7fac0406cd50 2023-11-26T07:21:55.471 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:21:55.464+0000 7fac0949a640 1 -- 172.21.15.143:0/3838491569 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fac040cbca0 con 0x7fac0406cd50 2023-11-26T07:21:55.472 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:21:55.464+0000 7fac0949a640 1 -- 172.21.15.143:0/3838491569 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fac040c5200 con 0x7fac0406cd50 2023-11-26T07:21:55.473 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:21:55.468+0000 7fac0949a640 1 -- 172.21.15.143:0/3838491569 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fac04067680 con 0x7fac0406cd50 2023-11-26T07:21:55.473 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:21:55.468+0000 7fabfb7fe640 1 -- 172.21.15.143:0/3838491569 <== mon.0 v2:172.21.15.143:3300/0 4 ==== mgrmap(e 9) v1 ==== 86005+0+0 (secure 0 0 0) 0x7fabf40b7070 con 0x7fac0406cd50 2023-11-26T07:21:55.473 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:21:55.468+0000 7fabfb7fe640 1 --2- 172.21.15.143:0/3838491569 >> [v2:172.21.15.143:6832/2150557544,v1:172.21.15.143:6833/2150557544] conn(0x7fabe4066620 0x7fabe4068ae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:21:55.473 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:21:55.468+0000 7fabfb7fe640 1 -- 172.21.15.143:0/3838491569 <== mon.0 v2:172.21.15.143:3300/0 5 ==== osd_map(57..57 src has 1..57) v4 ==== 5609+0+0 (secure 0 0 0) 0x7fabf4122b70 con 0x7fac0406cd50 2023-11-26T07:21:55.473 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:21:55.468+0000 7fabfbfff640 1 --2- 172.21.15.143:0/3838491569 >> [v2:172.21.15.143:6832/2150557544,v1:172.21.15.143:6833/2150557544] conn(0x7fabe4066620 0x7fabe4068ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:21:55.473 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:21:55.468+0000 7fabfbfff640 1 --2- 172.21.15.143:0/3838491569 >> [v2:172.21.15.143:6832/2150557544,v1:172.21.15.143:6833/2150557544] conn(0x7fabe4066620 0x7fabe4068ae0 secure :-1 s=READY pgs=377 cs=0 l=1 rev1=1 crypto rx=0x7fac040c4b20 tx=0x7fabec009040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:21:55.474 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:21:55.472+0000 7fabfb7fe640 1 -- 172.21.15.143:0/3838491569 <== mon.0 v2:172.21.15.143:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172649 (secure 0 0 0) 0x7fabf40efdb0 con 0x7fac0406cd50 2023-11-26T07:21:55.655 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:21:55.652+0000 7fac0949a640 1 -- 172.21.15.143:0/3838491569 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"} v 0) v1 -- 0x7fac040707d0 con 0x7fac0406cd50 2023-11-26T07:21:56.287 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:21:56.284+0000 7fabfb7fe640 1 -- 172.21.15.143:0/3838491569 <== mon.0 v2:172.21.15.143:3300/0 7 ==== mon_command_ack([{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]=0 new fs with metadata pool 8 and data pool 9 v30) v1 ==== 172+0+0 (secure 0 0 0) 0x7fabf40b50c0 con 0x7fac0406cd50 2023-11-26T07:21:56.287 INFO:teuthology.orchestra.run.smithi143.stderr:new fs with metadata pool 8 and data pool 9 2023-11-26T07:21:56.294 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:21:56.292+0000 7fac0949a640 1 -- 172.21.15.143:0/3838491569 >> [v2:172.21.15.143:6832/2150557544,v1:172.21.15.143:6833/2150557544] conn(0x7fabe4066620 msgr2=0x7fabe4068ae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:21:56.294 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:21:56.292+0000 7fac0949a640 1 --2- 172.21.15.143:0/3838491569 >> [v2:172.21.15.143:6832/2150557544,v1:172.21.15.143:6833/2150557544] conn(0x7fabe4066620 0x7fabe4068ae0 secure :-1 s=READY pgs=377 cs=0 l=1 rev1=1 crypto rx=0x7fac040c4b20 tx=0x7fabec009040 comp rx=0 tx=0).stop 2023-11-26T07:21:56.295 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:21:56.292+0000 7fac0949a640 1 -- 172.21.15.143:0/3838491569 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fac0406cd50 msgr2=0x7fac040cc4a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:21:56.295 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:21:56.292+0000 7fac0949a640 1 --2- 172.21.15.143:0/3838491569 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fac0406cd50 0x7fac040cc4a0 secure :-1 s=READY pgs=786 cs=0 l=1 rev1=1 crypto rx=0x7fabf40b3040 tx=0x7fabf401bad0 comp rx=0 tx=0).stop 2023-11-26T07:21:56.295 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:21:56.292+0000 7fac0949a640 1 -- 172.21.15.143:0/3838491569 shutdown_connections 2023-11-26T07:21:56.295 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:21:56.292+0000 7fac0949a640 1 --2- 172.21.15.143:0/3838491569 >> [v2:172.21.15.143:6832/2150557544,v1:172.21.15.143:6833/2150557544] conn(0x7fabe4066620 0x7fabe4068ae0 unknown :-1 s=CLOSED pgs=377 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:21:56.295 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:21:56.292+0000 7fac0949a640 1 --2- 172.21.15.143:0/3838491569 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fac040cf280 0x7fac040c4cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:21:56.296 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:21:56.292+0000 7fac0949a640 1 --2- 172.21.15.143:0/3838491569 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fac04070d10 0x7fac0414ba40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:21:56.296 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:21:56.292+0000 7fac0949a640 1 --2- 172.21.15.143:0/3838491569 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fac0406cd50 0x7fac040cc4a0 secure :-1 s=CLOSED pgs=786 cs=0 l=1 rev1=1 crypto rx=0x7fabf40b3040 tx=0x7fabf401bad0 comp rx=0 tx=0).stop 2023-11-26T07:21:56.296 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:21:56.292+0000 7fac0949a640 1 -- 172.21.15.143:0/3838491569 >> 172.21.15.143:0/3838491569 conn(0x7fac0406a890 msgr2=0x7fac040af540 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:21:56.297 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:21:56.292+0000 7fac0949a640 1 -- 172.21.15.143:0/3838491569 shutdown_connections 2023-11-26T07:21:56.298 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:21:56.292+0000 7fac0949a640 1 -- 172.21.15.143:0/3838491569 wait complete. 2023-11-26T07:21:56.329 DEBUG:teuthology.orchestra.run.smithi143:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-11-26T07:21:56.466 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:21:56.464+0000 7f1ff0432640 1 Processor -- start 2023-11-26T07:21:56.466 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:21:56.464+0000 7f1ff0432640 1 -- start start 2023-11-26T07:21:56.468 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:21:56.464+0000 7f1ff0432640 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7f1fe815d6e0 0x7f1fe80d0df0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:21:56.468 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:21:56.464+0000 7f1ff0432640 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7f1fe80b2160 0x7f1fe80d14b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:21:56.468 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:21:56.464+0000 7f1ff0432640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f1fe80d19f0 0x7f1fe80cfbf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:21:56.468 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:21:56.464+0000 7f1ff0432640 1 -- --> v2:172.21.15.143:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1fe815dac0 con 0x7f1fe815d6e0 2023-11-26T07:21:56.468 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:21:56.464+0000 7f1ff0432640 1 -- --> v2:172.21.15.188:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1fe80d1e80 con 0x7f1fe80b2160 2023-11-26T07:21:56.468 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:21:56.464+0000 7f1ff0432640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1fe80cf0e0 con 0x7f1fe80d19f0 2023-11-26T07:21:56.468 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:21:56.464+0000 7f1fee1ce640 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7f1fe815d6e0 0x7f1fe80d0df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:21:56.468 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:21:56.464+0000 7f1fee1ce640 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7f1fe815d6e0 0x7f1fe80d0df0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:57158/0 (socket says 172.21.15.143:57158) 2023-11-26T07:21:56.468 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:21:56.464+0000 7f1fee1ce640 1 -- 172.21.15.143:0/2219956911 learned_addr learned my addr 172.21.15.143:0/2219956911 (peer_addr_for_me v2:172.21.15.143:0/0) 2023-11-26T07:21:56.468 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:21:56.464+0000 7f1fee9cf640 1 --2- 172.21.15.143:0/2219956911 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f1fe80d19f0 0x7f1fe80cfbf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:21:56.469 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:21:56.464+0000 7f1fed9cd640 1 --2- 172.21.15.143:0/2219956911 >> v2:172.21.15.188:3300/0 conn(0x7f1fe80b2160 0x7f1fe80d14b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:21:56.469 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:21:56.464+0000 7f1fee1ce640 1 -- 172.21.15.143:0/2219956911 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f1fe80d19f0 msgr2=0x7f1fe80cfbf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-26T07:21:56.469 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:21:56.464+0000 7f1fee1ce640 1 --2- 172.21.15.143:0/2219956911 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f1fe80d19f0 0x7f1fe80cfbf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:21:56.469 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:21:56.464+0000 7f1fee1ce640 1 -- 172.21.15.143:0/2219956911 >> v2:172.21.15.188:3300/0 conn(0x7f1fe80b2160 msgr2=0x7f1fe80d14b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-26T07:21:56.469 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:21:56.464+0000 7f1fee1ce640 1 --2- 172.21.15.143:0/2219956911 >> v2:172.21.15.188:3300/0 conn(0x7f1fe80b2160 0x7f1fe80d14b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:21:56.469 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:21:56.464+0000 7f1fee1ce640 1 -- 172.21.15.143:0/2219956911 --> v2:172.21.15.143:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1fe80ceb20 con 0x7f1fe815d6e0 2023-11-26T07:21:56.470 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:21:56.468+0000 7f1fee1ce640 1 --2- 172.21.15.143:0/2219956911 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f1fe815d6e0 0x7f1fe80d0df0 secure :-1 s=READY pgs=787 cs=0 l=1 rev1=1 crypto rx=0x7f1fe80ca2a0 tx=0x7f1fe401c240 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f8d452561385a03b server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:21:56.471 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:21:56.468+0000 7f1fed1cc640 1 -- 172.21.15.143:0/2219956911 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1fe40ab600 con 0x7f1fe815d6e0 2023-11-26T07:21:56.471 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:21:56.468+0000 7f1fed1cc640 1 -- 172.21.15.143:0/2219956911 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1fe40a7050 con 0x7f1fe815d6e0 2023-11-26T07:21:56.471 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:21:56.468+0000 7f1fed1cc640 1 -- 172.21.15.143:0/2219956911 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1fe40ab980 con 0x7f1fe815d6e0 2023-11-26T07:21:56.472 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:21:56.468+0000 7f1ff0432640 1 -- 172.21.15.143:0/2219956911 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f1fe815d6e0 msgr2=0x7f1fe80d0df0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:21:56.472 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:21:56.468+0000 7f1ff0432640 1 --2- 172.21.15.143:0/2219956911 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f1fe815d6e0 0x7f1fe80d0df0 secure :-1 s=READY pgs=787 cs=0 l=1 rev1=1 crypto rx=0x7f1fe80ca2a0 tx=0x7f1fe401c240 comp rx=0 tx=0).stop 2023-11-26T07:21:56.472 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:21:56.468+0000 7f1ff0432640 1 -- 172.21.15.143:0/2219956911 shutdown_connections 2023-11-26T07:21:56.472 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:21:56.468+0000 7f1ff0432640 1 --2- 172.21.15.143:0/2219956911 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f1fe80d19f0 0x7f1fe80cfbf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:21:56.472 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:21:56.468+0000 7f1ff0432640 1 --2- 172.21.15.143:0/2219956911 >> v2:172.21.15.188:3300/0 conn(0x7f1fe80b2160 0x7f1fe80d14b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:21:56.472 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:21:56.468+0000 7f1ff0432640 1 --2- 172.21.15.143:0/2219956911 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f1fe815d6e0 0x7f1fe80d0df0 unknown :-1 s=CLOSED pgs=787 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:21:56.472 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:21:56.468+0000 7f1ff0432640 1 -- 172.21.15.143:0/2219956911 >> 172.21.15.143:0/2219956911 conn(0x7f1fe809c550 msgr2=0x7f1fe80d3220 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:21:56.472 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:21:56.468+0000 7f1ff0432640 1 -- 172.21.15.143:0/2219956911 shutdown_connections 2023-11-26T07:21:56.472 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:21:56.468+0000 7f1ff0432640 1 -- 172.21.15.143:0/2219956911 wait complete. 2023-11-26T07:21:56.473 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:21:56.472+0000 7f1ff0432640 1 Processor -- start 2023-11-26T07:21:56.473 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:21:56.472+0000 7f1ff0432640 1 -- start start 2023-11-26T07:21:56.475 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:21:56.472+0000 7f1ff0432640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f1fe80b2160 0x7f1fe80ccd30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:21:56.475 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:21:56.472+0000 7f1ff0432640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f1fe80d19f0 0x7f1fe80b0f10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:21:56.475 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:21:56.472+0000 7f1ff0432640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f1fe815d6e0 0x7f1fe815e460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:21:56.476 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:21:56.472+0000 7f1ff0432640 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1fe815dca0 con 0x7f1fe815d6e0 2023-11-26T07:21:56.476 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:21:56.472+0000 7f1ff0432640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1fe80cc2e0 con 0x7f1fe80b2160 2023-11-26T07:21:56.476 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:21:56.472+0000 7f1ff0432640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1fe80cc5c0 con 0x7f1fe80d19f0 2023-11-26T07:21:56.476 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:21:56.472+0000 7f1fee9cf640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f1fe815d6e0 0x7f1fe815e460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:21:56.476 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:21:56.472+0000 7f1fee9cf640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f1fe815d6e0 0x7f1fe815e460 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:57168/0 (socket says 172.21.15.143:57168) 2023-11-26T07:21:56.476 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:21:56.472+0000 7f1fee9cf640 1 -- 172.21.15.143:0/3567326552 learned_addr learned my addr 172.21.15.143:0/3567326552 (peer_addr_for_me v2:172.21.15.143:0/0) 2023-11-26T07:21:56.476 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:21:56.472+0000 7f1fee1ce640 1 --2- 172.21.15.143:0/3567326552 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f1fe80b2160 0x7f1fe80ccd30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:21:56.476 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:21:56.472+0000 7f1fed9cd640 1 --2- 172.21.15.143:0/3567326552 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f1fe80d19f0 0x7f1fe80b0f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:21:56.476 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:21:56.472+0000 7f1fee9cf640 1 -- 172.21.15.143:0/3567326552 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f1fe80d19f0 msgr2=0x7f1fe80b0f10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:21:56.476 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:21:56.472+0000 7f1fee9cf640 1 --2- 172.21.15.143:0/3567326552 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f1fe80d19f0 0x7f1fe80b0f10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:21:56.476 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:21:56.472+0000 7f1fee9cf640 1 -- 172.21.15.143:0/3567326552 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f1fe80b2160 msgr2=0x7f1fe80ccd30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:21:56.476 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:21:56.472+0000 7f1fee9cf640 1 --2- 172.21.15.143:0/3567326552 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f1fe80b2160 0x7f1fe80ccd30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:21:56.477 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:21:56.472+0000 7f1fee9cf640 1 -- 172.21.15.143:0/3567326552 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1fe40094d0 con 0x7f1fe815d6e0 2023-11-26T07:21:56.477 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:21:56.472+0000 7f1fee9cf640 1 --2- 172.21.15.143:0/3567326552 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f1fe815d6e0 0x7f1fe815e460 secure :-1 s=READY pgs=788 cs=0 l=1 rev1=1 crypto rx=0x7f1fe0007e40 tx=0x7f1fe000e4b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:21:56.477 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:21:56.472+0000 7f1fdeffd640 1 -- 172.21.15.143:0/3567326552 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1fe0014620 con 0x7f1fe815d6e0 2023-11-26T07:21:56.477 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:21:56.472+0000 7f1fdeffd640 1 -- 172.21.15.143:0/3567326552 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1fe000ee90 con 0x7f1fe815d6e0 2023-11-26T07:21:56.478 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:21:56.472+0000 7f1fdeffd640 1 -- 172.21.15.143:0/3567326552 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1fe00149e0 con 0x7f1fe815d6e0 2023-11-26T07:21:56.478 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:21:56.472+0000 7f1ff0432640 1 -- 172.21.15.143:0/3567326552 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1fe815df40 con 0x7f1fe815d6e0 2023-11-26T07:21:56.478 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:21:56.472+0000 7f1ff0432640 1 -- 172.21.15.143:0/3567326552 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1fe81514b0 con 0x7f1fe815d6e0 2023-11-26T07:21:56.478 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:21:56.476+0000 7f1ff0432640 1 -- 172.21.15.143:0/3567326552 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1fe8067680 con 0x7f1fe815d6e0 2023-11-26T07:21:56.478 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:21:56.476+0000 7f1fdeffd640 1 -- 172.21.15.143:0/3567326552 <== mon.0 v2:172.21.15.143:3300/0 4 ==== mgrmap(e 9) v1 ==== 86005+0+0 (secure 0 0 0) 0x7f1fe0014b40 con 0x7f1fe815d6e0 2023-11-26T07:21:56.479 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:21:56.476+0000 7f1fdeffd640 1 --2- 172.21.15.143:0/3567326552 >> [v2:172.21.15.143:6832/2150557544,v1:172.21.15.143:6833/2150557544] conn(0x7f1fcc066620 0x7f1fcc068ae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:21:56.479 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:21:56.476+0000 7f1fdeffd640 1 -- 172.21.15.143:0/3567326552 <== mon.0 v2:172.21.15.143:3300/0 5 ==== osd_map(58..58 src has 1..58) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f1fe0084ab0 con 0x7f1fe815d6e0 2023-11-26T07:21:56.479 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:21:56.476+0000 7f1fee1ce640 1 --2- 172.21.15.143:0/3567326552 >> [v2:172.21.15.143:6832/2150557544,v1:172.21.15.143:6833/2150557544] conn(0x7f1fcc066620 0x7f1fcc068ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:21:56.479 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:21:56.476+0000 7f1fee1ce640 1 --2- 172.21.15.143:0/3567326552 >> [v2:172.21.15.143:6832/2150557544,v1:172.21.15.143:6833/2150557544] conn(0x7f1fcc066620 0x7f1fcc068ae0 secure :-1 s=READY pgs=378 cs=0 l=1 rev1=1 crypto rx=0x7f1fe40ad000 tx=0x7f1fe4008040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:21:56.482 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:21:56.480+0000 7f1fdeffd640 1 -- 172.21.15.143:0/3567326552 <== mon.0 v2:172.21.15.143:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172649 (secure 0 0 0) 0x7f1fe0051cc0 con 0x7f1fe815d6e0 2023-11-26T07:21:56.633 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:21:56.632+0000 7f1ff0432640 1 -- 172.21.15.143:0/3567326552 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f1fe806bc90 con 0x7f1fe815d6e0 2023-11-26T07:21:56.634 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:21:56.632+0000 7f1fdeffd640 1 -- 172.21.15.143:0/3567326552 <== mon.0 v2:172.21.15.143:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v58) v1 ==== 74+0+16614 (secure 0 0 0) 0x7f1fe00552e0 con 0x7f1fe815d6e0 2023-11-26T07:21:56.635 INFO:teuthology.orchestra.run.smithi143.stdout: 2023-11-26T07:21:56.635 INFO:teuthology.orchestra.run.smithi143.stdout:{"epoch":58,"fsid":"f5033060-2469-4a07-95fe-2b76f5404bc1","created":"2023-11-26T07:17:28.613663+0000","modified":"2023-11-26T07:21:55.655844+0000","last_up_change":"2023-11-26T07:17:41.157813+0000","last_in_change":"2023-11-26T07:17:31.181658+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":9,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-11-26T07:17:41.758597+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":8,"pool_name":"cephfs_metadata","create_time":"2023-11-26T07:21:53.382407+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"58","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":9,"pool_name":"cephfs_data","create_time":"2023-11-26T07:21:54.656298+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"58","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.6299999952316284,"score_stable":1.6299999952316284,"optimal_score":1,"raw_score_acting":1.6299999952316284,"raw_score_stable":1.6299999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"ebbee2c6-023e-44b9-9f1d-6f7bfd309730","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":57,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6817","nonce":639807369},{"type":"v1","addr":"172.21.15.143:6819","nonce":639807369}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6821","nonce":639807369},{"type":"v1","addr":"172.21.15.143:6823","nonce":639807369}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6830","nonce":639807369},{"type":"v1","addr":"172.21.15.143:6831","nonce":639807369}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6825","nonce":639807369},{"type":"v1","addr":"172.21.15.143:6828","nonce":639807369}]},"public_addr":"172.21.15.143:6819/639807369","cluster_addr":"172.21.15.143:6823/639807369","heartbeat_back_addr":"172.21.15.143:6831/639807369","heartbeat_front_addr":"172.21.15.143:6828/639807369","state":["exists","up"]},{"osd":1,"uuid":"fec4b8be-7122-49e9-9a19-5850a0700001","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":57,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6816","nonce":1144942530},{"type":"v1","addr":"172.21.15.143:6818","nonce":1144942530}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6820","nonce":1144942530},{"type":"v1","addr":"172.21.15.143:6822","nonce":1144942530}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6827","nonce":1144942530},{"type":"v1","addr":"172.21.15.143:6829","nonce":1144942530}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6824","nonce":1144942530},{"type":"v1","addr":"172.21.15.143:6826","nonce":1144942530}]},"public_addr":"172.21.15.143:6818/1144942530","cluster_addr":"172.21.15.143:6822/1144942530","heartbeat_back_addr":"172.21.15.143:6829/1144942530","heartbeat_front_addr":"172.21.15.143:6826/1144942530","state":["exists","up"]},{"osd":2,"uuid":"2888af5b-f933-46ca-8f95-6dd494ff43f6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":57,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6808","nonce":3909095186},{"type":"v1","addr":"172.21.15.143:6809","nonce":3909095186}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6810","nonce":3909095186},{"type":"v1","addr":"172.21.15.143:6811","nonce":3909095186}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6814","nonce":3909095186},{"type":"v1","addr":"172.21.15.143:6815","nonce":3909095186}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6812","nonce":3909095186},{"type":"v1","addr":"172.21.15.143:6813","nonce":3909095186}]},"public_addr":"172.21.15.143:6809/3909095186","cluster_addr":"172.21.15.143:6811/3909095186","heartbeat_back_addr":"172.21.15.143:6815/3909095186","heartbeat_front_addr":"172.21.15.143:6813/3909095186","state":["exists","up"]},{"osd":3,"uuid":"d1f52435-7bc1-4081-9550-9cd1ba55f549","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":57,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6800","nonce":3854210991},{"type":"v1","addr":"172.21.15.143:6801","nonce":3854210991}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6802","nonce":3854210991},{"type":"v1","addr":"172.21.15.143:6803","nonce":3854210991}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6806","nonce":3854210991},{"type":"v1","addr":"172.21.15.143:6807","nonce":3854210991}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6804","nonce":3854210991},{"type":"v1","addr":"172.21.15.143:6805","nonce":3854210991}]},"public_addr":"172.21.15.143:6801/3854210991","cluster_addr":"172.21.15.143:6803/3854210991","heartbeat_back_addr":"172.21.15.143:6807/3854210991","heartbeat_front_addr":"172.21.15.143:6805/3854210991","state":["exists","up"]},{"osd":4,"uuid":"e2ae8b6e-6e25-49b2-bcbf-aacce68f06a3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":57,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6800","nonce":3924606902},{"type":"v1","addr":"172.21.15.188:6801","nonce":3924606902}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6802","nonce":3924606902},{"type":"v1","addr":"172.21.15.188:6803","nonce":3924606902}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6806","nonce":3924606902},{"type":"v1","addr":"172.21.15.188:6807","nonce":3924606902}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6804","nonce":3924606902},{"type":"v1","addr":"172.21.15.188:6805","nonce":3924606902}]},"public_addr":"172.21.15.188:6801/3924606902","cluster_addr":"172.21.15.188:6803/3924606902","heartbeat_back_addr":"172.21.15.188:6807/3924606902","heartbeat_front_addr":"172.21.15.188:6805/3924606902","state":["exists","up"]},{"osd":5,"uuid":"3cec8a32-7698-4c36-8e89-e4ba11934f33","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":57,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6813","nonce":186841544},{"type":"v1","addr":"172.21.15.188:6815","nonce":186841544}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6817","nonce":186841544},{"type":"v1","addr":"172.21.15.188:6818","nonce":186841544}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6822","nonce":186841544},{"type":"v1","addr":"172.21.15.188:6823","nonce":186841544}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6820","nonce":186841544},{"type":"v1","addr":"172.21.15.188:6821","nonce":186841544}]},"public_addr":"172.21.15.188:6815/186841544","cluster_addr":"172.21.15.188:6818/186841544","heartbeat_back_addr":"172.21.15.188:6823/186841544","heartbeat_front_addr":"172.21.15.188:6821/186841544","state":["exists","up"]},{"osd":6,"uuid":"58a65d5d-e090-473a-81b4-ad38e85029f9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":57,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6808","nonce":2153499405},{"type":"v1","addr":"172.21.15.188:6809","nonce":2153499405}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6810","nonce":2153499405},{"type":"v1","addr":"172.21.15.188:6811","nonce":2153499405}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6816","nonce":2153499405},{"type":"v1","addr":"172.21.15.188:6819","nonce":2153499405}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6812","nonce":2153499405},{"type":"v1","addr":"172.21.15.188:6814","nonce":2153499405}]},"public_addr":"172.21.15.188:6809/2153499405","cluster_addr":"172.21.15.188:6811/2153499405","heartbeat_back_addr":"172.21.15.188:6819/2153499405","heartbeat_front_addr":"172.21.15.188:6814/2153499405","state":["exists","up"]},{"osd":7,"uuid":"ea5b7f70-3554-4cf8-ac30-f7c59b80678d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":57,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6824","nonce":2722096729},{"type":"v1","addr":"172.21.15.188:6825","nonce":2722096729}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6826","nonce":2722096729},{"type":"v1","addr":"172.21.15.188:6827","nonce":2722096729}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6830","nonce":2722096729},{"type":"v1","addr":"172.21.15.188:6831","nonce":2722096729}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6828","nonce":2722096729},{"type":"v1","addr":"172.21.15.188:6829","nonce":2722096729}]},"public_addr":"172.21.15.188:6825/2722096729","cluster_addr":"172.21.15.188:6827/2722096729","heartbeat_back_addr":"172.21.15.188:6831/2722096729","heartbeat_front_addr":"172.21.15.188:6829/2722096729","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-26T07:17:36.995032+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-26T07:17:36.932266+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-26T07:17:36.829304+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-26T07:17:36.804297+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-26T07:17:36.667892+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-26T07:17:36.964868+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-26T07:17:36.827332+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-26T07:17:36.893179+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-11-26T07:21:56.637 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:21:56.636+0000 7f1ff0432640 1 -- 172.21.15.143:0/3567326552 >> [v2:172.21.15.143:6832/2150557544,v1:172.21.15.143:6833/2150557544] conn(0x7f1fcc066620 msgr2=0x7f1fcc068ae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:21:56.637 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:21:56.636+0000 7f1ff0432640 1 --2- 172.21.15.143:0/3567326552 >> [v2:172.21.15.143:6832/2150557544,v1:172.21.15.143:6833/2150557544] conn(0x7f1fcc066620 0x7f1fcc068ae0 secure :-1 s=READY pgs=378 cs=0 l=1 rev1=1 crypto rx=0x7f1fe40ad000 tx=0x7f1fe4008040 comp rx=0 tx=0).stop 2023-11-26T07:21:56.637 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:21:56.636+0000 7f1ff0432640 1 -- 172.21.15.143:0/3567326552 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f1fe815d6e0 msgr2=0x7f1fe815e460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:21:56.637 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:21:56.636+0000 7f1ff0432640 1 --2- 172.21.15.143:0/3567326552 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f1fe815d6e0 0x7f1fe815e460 secure :-1 s=READY pgs=788 cs=0 l=1 rev1=1 crypto rx=0x7f1fe0007e40 tx=0x7f1fe000e4b0 comp rx=0 tx=0).stop 2023-11-26T07:21:56.637 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:21:56.636+0000 7f1ff0432640 1 -- 172.21.15.143:0/3567326552 shutdown_connections 2023-11-26T07:21:56.637 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:21:56.636+0000 7f1ff0432640 1 --2- 172.21.15.143:0/3567326552 >> [v2:172.21.15.143:6832/2150557544,v1:172.21.15.143:6833/2150557544] conn(0x7f1fcc066620 0x7f1fcc068ae0 unknown :-1 s=CLOSED pgs=378 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:21:56.638 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:21:56.636+0000 7f1ff0432640 1 --2- 172.21.15.143:0/3567326552 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f1fe815d6e0 0x7f1fe815e460 unknown :-1 s=CLOSED pgs=788 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:21:56.638 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:21:56.636+0000 7f1ff0432640 1 --2- 172.21.15.143:0/3567326552 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f1fe80d19f0 0x7f1fe80b0f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:21:56.638 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:21:56.636+0000 7f1ff0432640 1 --2- 172.21.15.143:0/3567326552 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f1fe80b2160 0x7f1fe80ccd30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:21:56.638 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:21:56.636+0000 7f1ff0432640 1 -- 172.21.15.143:0/3567326552 >> 172.21.15.143:0/3567326552 conn(0x7f1fe809c550 msgr2=0x7f1fe80b2c70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:21:56.638 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:21:56.636+0000 7f1ff0432640 1 -- 172.21.15.143:0/3567326552 shutdown_connections 2023-11-26T07:21:56.638 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:21:56.636+0000 7f1ff0432640 1 -- 172.21.15.143:0/3567326552 wait complete. 2023-11-26T07:21:56.656 DEBUG:teuthology.orchestra.run.smithi143:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-11-26T07:21:56.795 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:21:56.792+0000 7fd1f218d640 1 Processor -- start 2023-11-26T07:21:56.795 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:21:56.792+0000 7fd1f218d640 1 -- start start 2023-11-26T07:21:56.797 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:21:56.792+0000 7fd1f218d640 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7fd1ec143ac0 0x7fd1ec070370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:21:56.797 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:21:56.792+0000 7fd1f218d640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fd1ec0cf990 0x7fd1ec0cd1b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:21:56.797 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:21:56.792+0000 7fd1f218d640 1 -- --> v1:172.21.15.188:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd1ec0cc730 con 0x7fd1ec0708b0 2023-11-26T07:21:56.797 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:21:56.792+0000 7fd1f218d640 1 -- --> v2:172.21.15.143:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd1ec143ea0 con 0x7fd1ec143ac0 2023-11-26T07:21:56.797 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:21:56.792+0000 7fd1f218d640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd1ec070ee0 con 0x7fd1ec0cf990 2023-11-26T07:21:56.797 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:21:56.796+0000 7fd1eb7fe640 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7fd1ec143ac0 0x7fd1ec070370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:21:56.797 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:21:56.796+0000 7fd1eb7fe640 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7fd1ec143ac0 0x7fd1ec070370 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:57174/0 (socket says 172.21.15.143:57174) 2023-11-26T07:21:56.797 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:21:56.796+0000 7fd1eb7fe640 1 -- 172.21.15.143:0/431749978 learned_addr learned my addr 172.21.15.143:0/431749978 (peer_addr_for_me v2:172.21.15.143:0/0) 2023-11-26T07:21:56.797 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:21:56.796+0000 7fd1ebfff640 1 --2- 172.21.15.143:0/431749978 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fd1ec0cf990 0x7fd1ec0cd1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:21:56.798 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:21:56.796+0000 7fd1eb7fe640 1 -- 172.21.15.143:0/431749978 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fd1ec0cf990 msgr2=0x7fd1ec0cd1b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-26T07:21:56.798 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:21:56.796+0000 7fd1eb7fe640 1 --2- 172.21.15.143:0/431749978 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fd1ec0cf990 0x7fd1ec0cd1b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:21:56.798 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:21:56.796+0000 7fd1eb7fe640 1 -- 172.21.15.143:0/431749978 >> v1:172.21.15.188:6789/0 conn(0x7fd1ec0708b0 legacy=0x7fd1ec0cf280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-26T07:21:56.798 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:21:56.796+0000 7fd1eb7fe640 1 -- 172.21.15.143:0/431749978 --> v2:172.21.15.143:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd1ec0711c0 con 0x7fd1ec143ac0 2023-11-26T07:21:56.800 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:21:56.796+0000 7fd1eb7fe640 1 --2- 172.21.15.143:0/431749978 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fd1ec143ac0 0x7fd1ec070370 secure :-1 s=READY pgs=789 cs=0 l=1 rev1=1 crypto rx=0x7fd1dc009870 tx=0x7fd1dc01c540 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f5b4f1551dd00fbd server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:21:56.800 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:21:56.796+0000 7fd1ea7fc640 1 -- 172.21.15.143:0/431749978 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd1dc0ab650 con 0x7fd1ec143ac0 2023-11-26T07:21:56.800 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:21:56.796+0000 7fd1ea7fc640 1 -- 172.21.15.143:0/431749978 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd1dc0ab7b0 con 0x7fd1ec143ac0 2023-11-26T07:21:56.800 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:21:56.796+0000 7fd1ea7fc640 1 -- 172.21.15.143:0/431749978 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd1dc0abb40 con 0x7fd1ec143ac0 2023-11-26T07:21:56.801 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:21:56.800+0000 7fd1f218d640 1 -- 172.21.15.143:0/431749978 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fd1ec143ac0 msgr2=0x7fd1ec070370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:21:56.801 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:21:56.800+0000 7fd1f218d640 1 --2- 172.21.15.143:0/431749978 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fd1ec143ac0 0x7fd1ec070370 secure :-1 s=READY pgs=789 cs=0 l=1 rev1=1 crypto rx=0x7fd1dc009870 tx=0x7fd1dc01c540 comp rx=0 tx=0).stop 2023-11-26T07:21:56.801 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:21:56.800+0000 7fd1f218d640 1 -- 172.21.15.143:0/431749978 shutdown_connections 2023-11-26T07:21:56.801 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:21:56.800+0000 7fd1f218d640 1 --2- 172.21.15.143:0/431749978 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fd1ec0cf990 0x7fd1ec0cd1b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:21:56.801 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:21:56.800+0000 7fd1f218d640 1 --2- 172.21.15.143:0/431749978 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fd1ec143ac0 0x7fd1ec070370 unknown :-1 s=CLOSED pgs=789 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:21:56.801 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:21:56.800+0000 7fd1f218d640 1 -- 172.21.15.143:0/431749978 >> 172.21.15.143:0/431749978 conn(0x7fd1ec06a890 msgr2=0x7fd1ec06aca0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:21:56.801 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:21:56.800+0000 7fd1f218d640 1 -- 172.21.15.143:0/431749978 shutdown_connections 2023-11-26T07:21:56.802 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:21:56.800+0000 7fd1f218d640 1 -- 172.21.15.143:0/431749978 wait complete. 2023-11-26T07:21:56.802 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:21:56.800+0000 7fd1f218d640 1 Processor -- start 2023-11-26T07:21:56.802 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:21:56.800+0000 7fd1f218d640 1 -- start start 2023-11-26T07:21:56.804 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:21:56.800+0000 7fd1f218d640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fd1ec0708b0 0x7fd1ec138bc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:21:56.804 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:21:56.800+0000 7fd1f218d640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fd1ec0cf990 0x7fd1ec0b2490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:21:56.805 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:21:56.800+0000 7fd1f218d640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fd1ec143ac0 0x7fd1ec0b29d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:21:56.805 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:21:56.800+0000 7fd1f218d640 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd1ec157cd0 con 0x7fd1ec143ac0 2023-11-26T07:21:56.805 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:21:56.800+0000 7fd1f218d640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd1ec157fb0 con 0x7fd1ec0708b0 2023-11-26T07:21:56.805 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:21:56.800+0000 7fd1f218d640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd1ec158290 con 0x7fd1ec0cf990 2023-11-26T07:21:56.805 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:21:56.800+0000 7fd1ebfff640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fd1ec143ac0 0x7fd1ec0b29d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:21:56.805 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:21:56.800+0000 7fd1eaffd640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fd1ec0cf990 0x7fd1ec0b2490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:21:56.805 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:21:56.800+0000 7fd1ebfff640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fd1ec143ac0 0x7fd1ec0b29d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:57176/0 (socket says 172.21.15.143:57176) 2023-11-26T07:21:56.805 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:21:56.800+0000 7fd1ebfff640 1 -- 172.21.15.143:0/3652346942 learned_addr learned my addr 172.21.15.143:0/3652346942 (peer_addr_for_me v2:172.21.15.143:0/0) 2023-11-26T07:21:56.806 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:21:56.800+0000 7fd1eb7fe640 1 --2- 172.21.15.143:0/3652346942 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fd1ec0708b0 0x7fd1ec138bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:21:56.806 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:21:56.804+0000 7fd1ebfff640 1 -- 172.21.15.143:0/3652346942 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fd1ec0cf990 msgr2=0x7fd1ec0b2490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:21:56.806 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:21:56.804+0000 7fd1ebfff640 1 --2- 172.21.15.143:0/3652346942 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fd1ec0cf990 0x7fd1ec0b2490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:21:56.806 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:21:56.804+0000 7fd1ebfff640 1 -- 172.21.15.143:0/3652346942 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fd1ec0708b0 msgr2=0x7fd1ec138bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:21:56.806 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:21:56.804+0000 7fd1ebfff640 1 --2- 172.21.15.143:0/3652346942 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fd1ec0708b0 0x7fd1ec138bc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:21:56.806 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:21:56.804+0000 7fd1ebfff640 1 -- 172.21.15.143:0/3652346942 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd1dc0094d0 con 0x7fd1ec143ac0 2023-11-26T07:21:56.806 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:21:56.804+0000 7fd1ebfff640 1 --2- 172.21.15.143:0/3652346942 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fd1ec143ac0 0x7fd1ec0b29d0 secure :-1 s=READY pgs=790 cs=0 l=1 rev1=1 crypto rx=0x7fd1e4007e40 tx=0x7fd1e400e4b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:21:56.807 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:21:56.804+0000 7fd1d3fff640 1 -- 172.21.15.143:0/3652346942 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd1e4014620 con 0x7fd1ec143ac0 2023-11-26T07:21:56.807 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:21:56.804+0000 7fd1d3fff640 1 -- 172.21.15.143:0/3652346942 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd1e400ee90 con 0x7fd1ec143ac0 2023-11-26T07:21:56.808 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:21:56.804+0000 7fd1d3fff640 1 -- 172.21.15.143:0/3652346942 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd1e40149e0 con 0x7fd1ec143ac0 2023-11-26T07:21:56.808 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:21:56.804+0000 7fd1f218d640 1 -- 172.21.15.143:0/3652346942 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd1ec158570 con 0x7fd1ec143ac0 2023-11-26T07:21:56.808 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:21:56.804+0000 7fd1f218d640 1 -- 172.21.15.143:0/3652346942 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd1ec157860 con 0x7fd1ec143ac0 2023-11-26T07:21:56.808 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:21:56.804+0000 7fd1f218d640 1 -- 172.21.15.143:0/3652346942 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd1ec067680 con 0x7fd1ec143ac0 2023-11-26T07:21:56.808 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:21:56.804+0000 7fd1d3fff640 1 -- 172.21.15.143:0/3652346942 <== mon.0 v2:172.21.15.143:3300/0 4 ==== mgrmap(e 9) v1 ==== 86005+0+0 (secure 0 0 0) 0x7fd1e4014b40 con 0x7fd1ec143ac0 2023-11-26T07:21:56.809 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:21:56.804+0000 7fd1d3fff640 1 --2- 172.21.15.143:0/3652346942 >> [v2:172.21.15.143:6832/2150557544,v1:172.21.15.143:6833/2150557544] conn(0x7fd1d8066620 0x7fd1d8068ae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:21:56.809 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:21:56.804+0000 7fd1d3fff640 1 -- 172.21.15.143:0/3652346942 <== mon.0 v2:172.21.15.143:3300/0 5 ==== osd_map(58..58 src has 1..58) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fd1e4084a10 con 0x7fd1ec143ac0 2023-11-26T07:21:56.809 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:21:56.804+0000 7fd1eb7fe640 1 --2- 172.21.15.143:0/3652346942 >> [v2:172.21.15.143:6832/2150557544,v1:172.21.15.143:6833/2150557544] conn(0x7fd1d8066620 0x7fd1d8068ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:21:56.809 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:21:56.808+0000 7fd1eb7fe640 1 --2- 172.21.15.143:0/3652346942 >> [v2:172.21.15.143:6832/2150557544,v1:172.21.15.143:6833/2150557544] conn(0x7fd1d8066620 0x7fd1d8068ae0 secure :-1 s=READY pgs=379 cs=0 l=1 rev1=1 crypto rx=0x7fd1dc0af000 tx=0x7fd1dc0b6040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:21:56.811 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:21:56.808+0000 7fd1d3fff640 1 -- 172.21.15.143:0/3652346942 <== mon.0 v2:172.21.15.143:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172649 (secure 0 0 0) 0x7fd1e4051cc0 con 0x7fd1ec143ac0 2023-11-26T07:21:56.962 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:21:56.960+0000 7fd1f218d640 1 -- 172.21.15.143:0/3652346942 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fd1ec158a20 con 0x7fd1ec143ac0 2023-11-26T07:21:56.963 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:21:56.960+0000 7fd1d3fff640 1 -- 172.21.15.143:0/3652346942 <== mon.0 v2:172.21.15.143:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v58) v1 ==== 74+0+16614 (secure 0 0 0) 0x7fd1e40552e0 con 0x7fd1ec143ac0 2023-11-26T07:21:56.963 INFO:teuthology.orchestra.run.smithi143.stdout: 2023-11-26T07:21:56.964 INFO:teuthology.orchestra.run.smithi143.stdout:{"epoch":58,"fsid":"f5033060-2469-4a07-95fe-2b76f5404bc1","created":"2023-11-26T07:17:28.613663+0000","modified":"2023-11-26T07:21:55.655844+0000","last_up_change":"2023-11-26T07:17:41.157813+0000","last_in_change":"2023-11-26T07:17:31.181658+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":9,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-11-26T07:17:41.758597+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":8,"pool_name":"cephfs_metadata","create_time":"2023-11-26T07:21:53.382407+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"58","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":9,"pool_name":"cephfs_data","create_time":"2023-11-26T07:21:54.656298+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"58","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.6299999952316284,"score_stable":1.6299999952316284,"optimal_score":1,"raw_score_acting":1.6299999952316284,"raw_score_stable":1.6299999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"ebbee2c6-023e-44b9-9f1d-6f7bfd309730","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":57,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6817","nonce":639807369},{"type":"v1","addr":"172.21.15.143:6819","nonce":639807369}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6821","nonce":639807369},{"type":"v1","addr":"172.21.15.143:6823","nonce":639807369}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6830","nonce":639807369},{"type":"v1","addr":"172.21.15.143:6831","nonce":639807369}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6825","nonce":639807369},{"type":"v1","addr":"172.21.15.143:6828","nonce":639807369}]},"public_addr":"172.21.15.143:6819/639807369","cluster_addr":"172.21.15.143:6823/639807369","heartbeat_back_addr":"172.21.15.143:6831/639807369","heartbeat_front_addr":"172.21.15.143:6828/639807369","state":["exists","up"]},{"osd":1,"uuid":"fec4b8be-7122-49e9-9a19-5850a0700001","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":57,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6816","nonce":1144942530},{"type":"v1","addr":"172.21.15.143:6818","nonce":1144942530}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6820","nonce":1144942530},{"type":"v1","addr":"172.21.15.143:6822","nonce":1144942530}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6827","nonce":1144942530},{"type":"v1","addr":"172.21.15.143:6829","nonce":1144942530}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6824","nonce":1144942530},{"type":"v1","addr":"172.21.15.143:6826","nonce":1144942530}]},"public_addr":"172.21.15.143:6818/1144942530","cluster_addr":"172.21.15.143:6822/1144942530","heartbeat_back_addr":"172.21.15.143:6829/1144942530","heartbeat_front_addr":"172.21.15.143:6826/1144942530","state":["exists","up"]},{"osd":2,"uuid":"2888af5b-f933-46ca-8f95-6dd494ff43f6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":57,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6808","nonce":3909095186},{"type":"v1","addr":"172.21.15.143:6809","nonce":3909095186}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6810","nonce":3909095186},{"type":"v1","addr":"172.21.15.143:6811","nonce":3909095186}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6814","nonce":3909095186},{"type":"v1","addr":"172.21.15.143:6815","nonce":3909095186}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6812","nonce":3909095186},{"type":"v1","addr":"172.21.15.143:6813","nonce":3909095186}]},"public_addr":"172.21.15.143:6809/3909095186","cluster_addr":"172.21.15.143:6811/3909095186","heartbeat_back_addr":"172.21.15.143:6815/3909095186","heartbeat_front_addr":"172.21.15.143:6813/3909095186","state":["exists","up"]},{"osd":3,"uuid":"d1f52435-7bc1-4081-9550-9cd1ba55f549","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":57,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6800","nonce":3854210991},{"type":"v1","addr":"172.21.15.143:6801","nonce":3854210991}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6802","nonce":3854210991},{"type":"v1","addr":"172.21.15.143:6803","nonce":3854210991}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6806","nonce":3854210991},{"type":"v1","addr":"172.21.15.143:6807","nonce":3854210991}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6804","nonce":3854210991},{"type":"v1","addr":"172.21.15.143:6805","nonce":3854210991}]},"public_addr":"172.21.15.143:6801/3854210991","cluster_addr":"172.21.15.143:6803/3854210991","heartbeat_back_addr":"172.21.15.143:6807/3854210991","heartbeat_front_addr":"172.21.15.143:6805/3854210991","state":["exists","up"]},{"osd":4,"uuid":"e2ae8b6e-6e25-49b2-bcbf-aacce68f06a3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":57,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6800","nonce":3924606902},{"type":"v1","addr":"172.21.15.188:6801","nonce":3924606902}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6802","nonce":3924606902},{"type":"v1","addr":"172.21.15.188:6803","nonce":3924606902}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6806","nonce":3924606902},{"type":"v1","addr":"172.21.15.188:6807","nonce":3924606902}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6804","nonce":3924606902},{"type":"v1","addr":"172.21.15.188:6805","nonce":3924606902}]},"public_addr":"172.21.15.188:6801/3924606902","cluster_addr":"172.21.15.188:6803/3924606902","heartbeat_back_addr":"172.21.15.188:6807/3924606902","heartbeat_front_addr":"172.21.15.188:6805/3924606902","state":["exists","up"]},{"osd":5,"uuid":"3cec8a32-7698-4c36-8e89-e4ba11934f33","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":57,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6813","nonce":186841544},{"type":"v1","addr":"172.21.15.188:6815","nonce":186841544}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6817","nonce":186841544},{"type":"v1","addr":"172.21.15.188:6818","nonce":186841544}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6822","nonce":186841544},{"type":"v1","addr":"172.21.15.188:6823","nonce":186841544}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6820","nonce":186841544},{"type":"v1","addr":"172.21.15.188:6821","nonce":186841544}]},"public_addr":"172.21.15.188:6815/186841544","cluster_addr":"172.21.15.188:6818/186841544","heartbeat_back_addr":"172.21.15.188:6823/186841544","heartbeat_front_addr":"172.21.15.188:6821/186841544","state":["exists","up"]},{"osd":6,"uuid":"58a65d5d-e090-473a-81b4-ad38e85029f9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":57,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6808","nonce":2153499405},{"type":"v1","addr":"172.21.15.188:6809","nonce":2153499405}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6810","nonce":2153499405},{"type":"v1","addr":"172.21.15.188:6811","nonce":2153499405}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6816","nonce":2153499405},{"type":"v1","addr":"172.21.15.188:6819","nonce":2153499405}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6812","nonce":2153499405},{"type":"v1","addr":"172.21.15.188:6814","nonce":2153499405}]},"public_addr":"172.21.15.188:6809/2153499405","cluster_addr":"172.21.15.188:6811/2153499405","heartbeat_back_addr":"172.21.15.188:6819/2153499405","heartbeat_front_addr":"172.21.15.188:6814/2153499405","state":["exists","up"]},{"osd":7,"uuid":"ea5b7f70-3554-4cf8-ac30-f7c59b80678d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":57,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6824","nonce":2722096729},{"type":"v1","addr":"172.21.15.188:6825","nonce":2722096729}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6826","nonce":2722096729},{"type":"v1","addr":"172.21.15.188:6827","nonce":2722096729}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6830","nonce":2722096729},{"type":"v1","addr":"172.21.15.188:6831","nonce":2722096729}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6828","nonce":2722096729},{"type":"v1","addr":"172.21.15.188:6829","nonce":2722096729}]},"public_addr":"172.21.15.188:6825/2722096729","cluster_addr":"172.21.15.188:6827/2722096729","heartbeat_back_addr":"172.21.15.188:6831/2722096729","heartbeat_front_addr":"172.21.15.188:6829/2722096729","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-26T07:17:36.995032+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-26T07:17:36.932266+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-26T07:17:36.829304+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-26T07:17:36.804297+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-26T07:17:36.667892+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-26T07:17:36.964868+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-26T07:17:36.827332+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-26T07:17:36.893179+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-11-26T07:21:56.965 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:21:56.964+0000 7fd1f218d640 1 -- 172.21.15.143:0/3652346942 >> [v2:172.21.15.143:6832/2150557544,v1:172.21.15.143:6833/2150557544] conn(0x7fd1d8066620 msgr2=0x7fd1d8068ae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:21:56.966 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:21:56.964+0000 7fd1f218d640 1 --2- 172.21.15.143:0/3652346942 >> [v2:172.21.15.143:6832/2150557544,v1:172.21.15.143:6833/2150557544] conn(0x7fd1d8066620 0x7fd1d8068ae0 secure :-1 s=READY pgs=379 cs=0 l=1 rev1=1 crypto rx=0x7fd1dc0af000 tx=0x7fd1dc0b6040 comp rx=0 tx=0).stop 2023-11-26T07:21:56.966 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:21:56.964+0000 7fd1f218d640 1 -- 172.21.15.143:0/3652346942 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fd1ec143ac0 msgr2=0x7fd1ec0b29d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:21:56.966 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:21:56.964+0000 7fd1f218d640 1 --2- 172.21.15.143:0/3652346942 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fd1ec143ac0 0x7fd1ec0b29d0 secure :-1 s=READY pgs=790 cs=0 l=1 rev1=1 crypto rx=0x7fd1e4007e40 tx=0x7fd1e400e4b0 comp rx=0 tx=0).stop 2023-11-26T07:21:56.966 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:21:56.964+0000 7fd1f218d640 1 -- 172.21.15.143:0/3652346942 shutdown_connections 2023-11-26T07:21:56.966 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:21:56.964+0000 7fd1f218d640 1 --2- 172.21.15.143:0/3652346942 >> [v2:172.21.15.143:6832/2150557544,v1:172.21.15.143:6833/2150557544] conn(0x7fd1d8066620 0x7fd1d8068ae0 unknown :-1 s=CLOSED pgs=379 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:21:56.966 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:21:56.964+0000 7fd1f218d640 1 --2- 172.21.15.143:0/3652346942 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fd1ec143ac0 0x7fd1ec0b29d0 unknown :-1 s=CLOSED pgs=790 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:21:56.966 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:21:56.964+0000 7fd1f218d640 1 --2- 172.21.15.143:0/3652346942 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fd1ec0cf990 0x7fd1ec0b2490 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:21:56.967 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:21:56.964+0000 7fd1f218d640 1 --2- 172.21.15.143:0/3652346942 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fd1ec0708b0 0x7fd1ec138bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:21:56.967 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:21:56.964+0000 7fd1f218d640 1 -- 172.21.15.143:0/3652346942 >> 172.21.15.143:0/3652346942 conn(0x7fd1ec06a890 msgr2=0x7fd1ec06cdb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:21:56.967 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:21:56.964+0000 7fd1f218d640 1 -- 172.21.15.143:0/3652346942 shutdown_connections 2023-11-26T07:21:56.967 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:21:56.964+0000 7fd1f218d640 1 -- 172.21.15.143:0/3652346942 wait complete. 2023-11-26T07:21:56.985 DEBUG:teuthology.orchestra.run.smithi143:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs standby_count_wanted 0 2023-11-26T07:21:57.125 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:21:57.124+0000 7f50c4a46640 1 Processor -- start 2023-11-26T07:21:57.125 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:21:57.124+0000 7f50c4a46640 1 -- start start 2023-11-26T07:21:57.127 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:21:57.124+0000 7f50c4a46640 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7f50c0070d10 0x7f50c015d900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:21:57.127 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:21:57.124+0000 7f50c4a46640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f50c006cd50 0x7f50c00b2180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:21:57.127 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:21:57.124+0000 7f50c4a46640 1 -- --> v1:172.21.15.143:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f50c015b2f0 con 0x7f50c00cf280 2023-11-26T07:21:57.127 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:21:57.124+0000 7f50c4a46640 1 -- --> v2:172.21.15.188:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f50c006c5c0 con 0x7f50c0070d10 2023-11-26T07:21:57.127 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:21:57.124+0000 7f50c4a46640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f50c015e1a0 con 0x7f50c006cd50 2023-11-26T07:21:57.127 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:21:57.124+0000 7f50be59c640 1 --1- >> v1:172.21.15.143:6789/0 conn(0x7f50c00cf280 0x7f50c00cf660 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.143:6789/0 says I am v1:172.21.15.143:57224/0 (socket says 172.21.15.143:57224) 2023-11-26T07:21:57.127 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:21:57.124+0000 7f50be59c640 1 -- 172.21.15.143:0/1787529903 learned_addr learned my addr 172.21.15.143:0/1787529903 (peer_addr_for_me v1:172.21.15.143:0/0) 2023-11-26T07:21:57.127 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:21:57.124+0000 7f50bed9d640 1 --2- 172.21.15.143:0/1787529903 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f50c006cd50 0x7f50c00b2180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:21:57.128 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:21:57.124+0000 7f50b7fff640 1 --2- 172.21.15.143:0/1787529903 >> v2:172.21.15.188:3300/0 conn(0x7f50c0070d10 0x7f50c015d900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:21:57.128 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:21:57.124+0000 7f50bdd9b640 1 -- 172.21.15.143:0/1787529903 <== mon.0 v1:172.21.15.143:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2703772493 0 0) 0x7f50b0001120 con 0x7f50c00cf280 2023-11-26T07:21:57.128 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:21:57.124+0000 7f50bdd9b640 1 -- 172.21.15.143:0/1787529903 --> v1:172.21.15.143:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f50ac003530 con 0x7f50c00cf280 2023-11-26T07:21:57.129 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:21:57.124+0000 7f50b7fff640 1 -- 172.21.15.143:0/1787529903 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f50c006cd50 msgr2=0x7f50c00b2180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-26T07:21:57.129 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:21:57.124+0000 7f50b7fff640 1 --2- 172.21.15.143:0/1787529903 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f50c006cd50 0x7f50c00b2180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:21:57.129 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:21:57.124+0000 7f50b7fff640 1 -- 172.21.15.143:0/1787529903 >> v1:172.21.15.143:6789/0 conn(0x7f50c00cf280 legacy=0x7f50c00cf660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:21:57.129 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:21:57.124+0000 7f50b7fff640 1 -- 172.21.15.143:0/1787529903 --> v2:172.21.15.188:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f50c015e480 con 0x7f50c0070d10 2023-11-26T07:21:57.129 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:21:57.124+0000 7f50bed9d640 1 --2- 172.21.15.143:0/1787529903 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f50c006cd50 0x7f50c00b2180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-26T07:21:57.130 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:21:57.128+0000 7f50b7fff640 1 --2- 172.21.15.143:0/1787529903 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f50c0070d10 0x7f50c015d900 secure :-1 s=READY pgs=315 cs=0 l=1 rev1=1 crypto rx=0x7f50a80097c0 tx=0x7f50a801ec70 comp rx=0 tx=0).ready entity=mon.1 client_cookie=b7df86d3089dbbca server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:21:57.131 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:21:57.128+0000 7f50bdd9b640 1 -- 172.21.15.143:0/1787529903 <== mon.1 v2:172.21.15.188:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f50a80a8970 con 0x7f50c0070d10 2023-11-26T07:21:57.131 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:21:57.128+0000 7f50bdd9b640 1 -- 172.21.15.143:0/1787529903 <== mon.1 v2:172.21.15.188:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f50a80a8ad0 con 0x7f50c0070d10 2023-11-26T07:21:57.131 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:21:57.128+0000 7f50bdd9b640 1 -- 172.21.15.143:0/1787529903 <== mon.1 v2:172.21.15.188:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f50a80a8e60 con 0x7f50c0070d10 2023-11-26T07:21:57.132 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:21:57.128+0000 7f50c4a46640 1 -- 172.21.15.143:0/1787529903 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f50c0070d10 msgr2=0x7f50c015d900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:21:57.132 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:21:57.128+0000 7f50c4a46640 1 --2- 172.21.15.143:0/1787529903 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f50c0070d10 0x7f50c015d900 secure :-1 s=READY pgs=315 cs=0 l=1 rev1=1 crypto rx=0x7f50a80097c0 tx=0x7f50a801ec70 comp rx=0 tx=0).stop 2023-11-26T07:21:57.132 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:21:57.128+0000 7f50c4a46640 1 -- 172.21.15.143:0/1787529903 shutdown_connections 2023-11-26T07:21:57.132 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:21:57.128+0000 7f50c4a46640 1 --2- 172.21.15.143:0/1787529903 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f50c006cd50 0x7f50c00b2180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:21:57.132 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:21:57.128+0000 7f50c4a46640 1 --2- 172.21.15.143:0/1787529903 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f50c0070d10 0x7f50c015d900 unknown :-1 s=CLOSED pgs=315 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:21:57.132 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:21:57.128+0000 7f50c4a46640 1 -- 172.21.15.143:0/1787529903 >> 172.21.15.143:0/1787529903 conn(0x7f50c006a890 msgr2=0x7f50c006aca0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:21:57.132 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:21:57.128+0000 7f50c4a46640 1 -- 172.21.15.143:0/1787529903 shutdown_connections 2023-11-26T07:21:57.132 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:21:57.128+0000 7f50c4a46640 1 -- 172.21.15.143:0/1787529903 wait complete. 2023-11-26T07:21:57.133 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:21:57.132+0000 7f50c4a46640 1 Processor -- start 2023-11-26T07:21:57.133 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:21:57.132+0000 7f50c4a46640 1 -- start start 2023-11-26T07:21:57.135 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:21:57.132+0000 7f50c4a46640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f50c006cd50 0x7f50c0156ce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:21:57.135 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:21:57.132+0000 7f50c4a46640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f50c0070d10 0x7f50c00ce560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:21:57.136 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:21:57.132+0000 7f50c4a46640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f50c00cf280 0x7f50c01e35f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:21:57.136 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:21:57.132+0000 7f50c4a46640 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f50c00ceaa0 con 0x7f50c0070d10 2023-11-26T07:21:57.136 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:21:57.132+0000 7f50c4a46640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f50c01e3b30 con 0x7f50c00cf280 2023-11-26T07:21:57.136 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:21:57.132+0000 7f50c4a46640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f50c01e3d40 con 0x7f50c006cd50 2023-11-26T07:21:57.136 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:21:57.132+0000 7f50b7fff640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f50c0070d10 0x7f50c00ce560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:21:57.136 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:21:57.132+0000 7f50b7fff640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f50c0070d10 0x7f50c00ce560 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:57186/0 (socket says 172.21.15.143:57186) 2023-11-26T07:21:57.136 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:21:57.132+0000 7f50b7fff640 1 -- 172.21.15.143:0/2051902718 learned_addr learned my addr 172.21.15.143:0/2051902718 (peer_addr_for_me v2:172.21.15.143:0/0) 2023-11-26T07:21:57.136 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:21:57.132+0000 7f50be59c640 1 --2- 172.21.15.143:0/2051902718 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f50c006cd50 0x7f50c0156ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:21:57.136 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:21:57.132+0000 7f50b7fff640 1 -- 172.21.15.143:0/2051902718 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f50c006cd50 msgr2=0x7f50c0156ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:21:57.136 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:21:57.132+0000 7f50bed9d640 1 --2- 172.21.15.143:0/2051902718 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f50c00cf280 0x7f50c01e35f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:21:57.136 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:21:57.132+0000 7f50b7fff640 1 --2- 172.21.15.143:0/2051902718 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f50c006cd50 0x7f50c0156ce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:21:57.136 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:21:57.132+0000 7f50b7fff640 1 -- 172.21.15.143:0/2051902718 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f50c00cf280 msgr2=0x7f50c01e35f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:21:57.137 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:21:57.132+0000 7f50b7fff640 1 --2- 172.21.15.143:0/2051902718 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f50c00cf280 0x7f50c01e35f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:21:57.137 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:21:57.132+0000 7f50b7fff640 1 -- 172.21.15.143:0/2051902718 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f50b8009a70 con 0x7f50c0070d10 2023-11-26T07:21:57.137 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:21:57.132+0000 7f50be59c640 1 --2- 172.21.15.143:0/2051902718 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f50c006cd50 0x7f50c0156ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-26T07:21:57.137 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:21:57.132+0000 7f50bed9d640 1 --2- 172.21.15.143:0/2051902718 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f50c00cf280 0x7f50c01e35f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-26T07:21:57.137 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:21:57.132+0000 7f50b7fff640 1 --2- 172.21.15.143:0/2051902718 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f50c0070d10 0x7f50c00ce560 secure :-1 s=READY pgs=792 cs=0 l=1 rev1=1 crypto rx=0x7f50a80b3040 tx=0x7f50a801ba10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:21:57.137 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:21:57.132+0000 7f50b6ffd640 1 -- 172.21.15.143:0/2051902718 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f50a80b0d60 con 0x7f50c0070d10 2023-11-26T07:21:57.137 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:21:57.132+0000 7f50b6ffd640 1 -- 172.21.15.143:0/2051902718 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f50a80b9430 con 0x7f50c0070d10 2023-11-26T07:21:57.137 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:21:57.132+0000 7f50b6ffd640 1 -- 172.21.15.143:0/2051902718 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f50a80b97e0 con 0x7f50c0070d10 2023-11-26T07:21:57.137 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:21:57.132+0000 7f50c4a46640 1 -- 172.21.15.143:0/2051902718 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f50a8009420 con 0x7f50c0070d10 2023-11-26T07:21:57.137 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:21:57.132+0000 7f50c4a46640 1 -- 172.21.15.143:0/2051902718 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f50c01e4300 con 0x7f50c0070d10 2023-11-26T07:21:57.138 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:21:57.136+0000 7f50c4a46640 1 -- 172.21.15.143:0/2051902718 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f50c0067680 con 0x7f50c0070d10 2023-11-26T07:21:57.138 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:21:57.136+0000 7f50b6ffd640 1 -- 172.21.15.143:0/2051902718 <== mon.0 v2:172.21.15.143:3300/0 4 ==== mgrmap(e 9) v1 ==== 86005+0+0 (secure 0 0 0) 0x7f50a80b7040 con 0x7f50c0070d10 2023-11-26T07:21:57.139 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:21:57.136+0000 7f50b6ffd640 1 --2- 172.21.15.143:0/2051902718 >> [v2:172.21.15.143:6832/2150557544,v1:172.21.15.143:6833/2150557544] conn(0x7f50a0066650 0x7f50a0068b10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:21:57.139 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:21:57.136+0000 7f50b6ffd640 1 -- 172.21.15.143:0/2051902718 <== mon.0 v2:172.21.15.143:3300/0 5 ==== osd_map(58..58 src has 1..58) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f50a8121c10 con 0x7f50c0070d10 2023-11-26T07:21:57.139 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:21:57.136+0000 7f50be59c640 1 --2- 172.21.15.143:0/2051902718 >> [v2:172.21.15.143:6832/2150557544,v1:172.21.15.143:6833/2150557544] conn(0x7f50a0066650 0x7f50a0068b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:21:57.139 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:21:57.136+0000 7f50be59c640 1 --2- 172.21.15.143:0/2051902718 >> [v2:172.21.15.143:6832/2150557544,v1:172.21.15.143:6833/2150557544] conn(0x7f50a0066650 0x7f50a0068b10 secure :-1 s=READY pgs=380 cs=0 l=1 rev1=1 crypto rx=0x7f50b0007fd0 tx=0x7f50b0009040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:21:57.143 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:21:57.140+0000 7f50b6ffd640 1 -- 172.21.15.143:0/2051902718 <== mon.0 v2:172.21.15.143:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172649 (secure 0 0 0) 0x7f50a80eee10 con 0x7f50c0070d10 2023-11-26T07:21:57.328 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:21:57.324+0000 7f50c4a46640 1 -- 172.21.15.143:0/2051902718 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"} v 0) v1 -- 0x7f50c00705f0 con 0x7f50c0070d10 2023-11-26T07:21:58.302 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:21:58.300+0000 7f50b6ffd640 1 -- 172.21.15.143:0/2051902718 <== mon.0 v2:172.21.15.143:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]=0 v33) v1 ==== 118+0+0 (secure 0 0 0) 0x7f50a80b5020 con 0x7f50c0070d10 2023-11-26T07:21:58.305 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:21:58.304+0000 7f50c4a46640 1 -- 172.21.15.143:0/2051902718 >> [v2:172.21.15.143:6832/2150557544,v1:172.21.15.143:6833/2150557544] conn(0x7f50a0066650 msgr2=0x7f50a0068b10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:21:58.305 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:21:58.304+0000 7f50c4a46640 1 --2- 172.21.15.143:0/2051902718 >> [v2:172.21.15.143:6832/2150557544,v1:172.21.15.143:6833/2150557544] conn(0x7f50a0066650 0x7f50a0068b10 secure :-1 s=READY pgs=380 cs=0 l=1 rev1=1 crypto rx=0x7f50b0007fd0 tx=0x7f50b0009040 comp rx=0 tx=0).stop 2023-11-26T07:21:58.305 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:21:58.304+0000 7f50c4a46640 1 -- 172.21.15.143:0/2051902718 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f50c0070d10 msgr2=0x7f50c00ce560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:21:58.306 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:21:58.304+0000 7f50c4a46640 1 --2- 172.21.15.143:0/2051902718 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f50c0070d10 0x7f50c00ce560 secure :-1 s=READY pgs=792 cs=0 l=1 rev1=1 crypto rx=0x7f50a80b3040 tx=0x7f50a801ba10 comp rx=0 tx=0).stop 2023-11-26T07:21:58.306 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:21:58.304+0000 7f50c4a46640 1 -- 172.21.15.143:0/2051902718 shutdown_connections 2023-11-26T07:21:58.306 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:21:58.304+0000 7f50c4a46640 1 --2- 172.21.15.143:0/2051902718 >> [v2:172.21.15.143:6832/2150557544,v1:172.21.15.143:6833/2150557544] conn(0x7f50a0066650 0x7f50a0068b10 unknown :-1 s=CLOSED pgs=380 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:21:58.306 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:21:58.304+0000 7f50c4a46640 1 --2- 172.21.15.143:0/2051902718 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f50c00cf280 0x7f50c01e35f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:21:58.306 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:21:58.304+0000 7f50c4a46640 1 --2- 172.21.15.143:0/2051902718 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f50c0070d10 0x7f50c00ce560 unknown :-1 s=CLOSED pgs=792 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:21:58.306 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:21:58.304+0000 7f50c4a46640 1 --2- 172.21.15.143:0/2051902718 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f50c006cd50 0x7f50c0156ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:21:58.306 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:21:58.304+0000 7f50c4a46640 1 -- 172.21.15.143:0/2051902718 >> 172.21.15.143:0/2051902718 conn(0x7f50c006a890 msgr2=0x7f50c00af4b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:21:58.306 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:21:58.304+0000 7f50c4a46640 1 -- 172.21.15.143:0/2051902718 shutdown_connections 2023-11-26T07:21:58.306 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:21:58.304+0000 7f50c4a46640 1 -- 172.21.15.143:0/2051902718 wait complete. 2023-11-26T07:21:58.328 DEBUG:tasks.cephfs.filesystem:fs_config: {} 2023-11-26T07:21:58.328 DEBUG:teuthology.orchestra.run.smithi143:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs allow_standby_replay false 2023-11-26T07:21:58.474 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:21:58.472+0000 7fe91ef62640 1 Processor -- start 2023-11-26T07:21:58.474 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:21:58.472+0000 7fe91ef62640 1 -- start start 2023-11-26T07:21:58.476 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:21:58.472+0000 7fe91ef62640 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7fe91815d2a0 0x7fe9180d00e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:21:58.476 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:21:58.472+0000 7fe91ef62640 1 -- --> v1:172.21.15.143:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe9180cdc40 con 0x7fe9180d1d90 2023-11-26T07:21:58.476 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:21:58.472+0000 7fe91ef62640 1 -- --> v1:172.21.15.188:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe9180ce7a0 con 0x7fe91815d680 2023-11-26T07:21:58.476 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:21:58.472+0000 7fe91ef62640 1 -- --> v2:172.21.15.188:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe9180d1440 con 0x7fe91815d2a0 2023-11-26T07:21:58.476 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:21:58.472+0000 7fe91e761640 1 --1- >> v1:172.21.15.143:6789/0 conn(0x7fe9180d1d90 0x7fe9180b2480 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.143:6789/0 says I am v1:172.21.15.143:57238/0 (socket says 172.21.15.143:57238) 2023-11-26T07:21:58.476 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:21:58.472+0000 7fe91e761640 1 -- 172.21.15.143:0/3824579110 learned_addr learned my addr 172.21.15.143:0/3824579110 (peer_addr_for_me v1:172.21.15.143:0/0) 2023-11-26T07:21:58.476 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:21:58.472+0000 7fe91df60640 1 --2- 172.21.15.143:0/3824579110 >> v2:172.21.15.188:3300/0 conn(0x7fe91815d2a0 0x7fe9180d00e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:21:58.477 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:21:58.476+0000 7fe91cf5e640 1 -- 172.21.15.143:0/3824579110 <== mon.0 v1:172.21.15.143:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4061580676 0 0) 0x7fe9180cdc40 con 0x7fe9180d1d90 2023-11-26T07:21:58.477 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:21:58.476+0000 7fe91cf5e640 1 -- 172.21.15.143:0/3824579110 --> v1:172.21.15.143:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fe904003530 con 0x7fe9180d1d90 2023-11-26T07:21:58.477 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:21:58.476+0000 7fe91df60640 1 -- 172.21.15.143:0/3824579110 >> v1:172.21.15.188:6789/0 conn(0x7fe91815d680 legacy=0x7fe9180d1680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:21:58.477 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:21:58.476+0000 7fe91df60640 1 -- 172.21.15.143:0/3824579110 >> v1:172.21.15.143:6789/0 conn(0x7fe9180d1d90 legacy=0x7fe9180b2480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:21:58.477 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:21:58.476+0000 7fe91df60640 1 -- 172.21.15.143:0/3824579110 --> v2:172.21.15.188:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe9180cde70 con 0x7fe91815d2a0 2023-11-26T07:21:58.479 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:21:58.476+0000 7fe91df60640 1 --2- 172.21.15.143:0/3824579110 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fe91815d2a0 0x7fe9180d00e0 secure :-1 s=READY pgs=317 cs=0 l=1 rev1=1 crypto rx=0x7fe9140097c0 tx=0x7fe91401c450 comp rx=0 tx=0).ready entity=mon.1 client_cookie=bf5641a2f2883392 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:21:58.481 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:21:58.480+0000 7fe91cf5e640 1 -- 172.21.15.143:0/3824579110 <== mon.1 v2:172.21.15.188:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe9140ad660 con 0x7fe91815d2a0 2023-11-26T07:21:58.481 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:21:58.480+0000 7fe91cf5e640 1 -- 172.21.15.143:0/3824579110 <== mon.1 v2:172.21.15.188:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe9140ad7c0 con 0x7fe91815d2a0 2023-11-26T07:21:58.481 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:21:58.480+0000 7fe91cf5e640 1 -- 172.21.15.143:0/3824579110 <== mon.1 v2:172.21.15.188:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe9140adb50 con 0x7fe91815d2a0 2023-11-26T07:21:58.482 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:21:58.480+0000 7fe91ef62640 1 -- 172.21.15.143:0/3824579110 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fe91815d2a0 msgr2=0x7fe9180d00e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:21:58.482 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:21:58.480+0000 7fe91ef62640 1 --2- 172.21.15.143:0/3824579110 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fe91815d2a0 0x7fe9180d00e0 secure :-1 s=READY pgs=317 cs=0 l=1 rev1=1 crypto rx=0x7fe9140097c0 tx=0x7fe91401c450 comp rx=0 tx=0).stop 2023-11-26T07:21:58.482 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:21:58.480+0000 7fe91ef62640 1 -- 172.21.15.143:0/3824579110 shutdown_connections 2023-11-26T07:21:58.482 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:21:58.480+0000 7fe91ef62640 1 --2- 172.21.15.143:0/3824579110 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fe91815d2a0 0x7fe9180d00e0 unknown :-1 s=CLOSED pgs=317 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:21:58.483 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:21:58.480+0000 7fe91ef62640 1 -- 172.21.15.143:0/3824579110 >> 172.21.15.143:0/3824579110 conn(0x7fe9180d30a0 msgr2=0x7fe9180d3480 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:21:58.483 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:21:58.480+0000 7fe91ef62640 1 -- 172.21.15.143:0/3824579110 shutdown_connections 2023-11-26T07:21:58.483 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:21:58.480+0000 7fe91ef62640 1 -- 172.21.15.143:0/3824579110 wait complete. 2023-11-26T07:21:58.483 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:21:58.480+0000 7fe91ef62640 1 Processor -- start 2023-11-26T07:21:58.484 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:21:58.480+0000 7fe91ef62640 1 -- start start 2023-11-26T07:21:58.485 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:21:58.484+0000 7fe91ef62640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fe9180d1d90 0x7fe918157ce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:21:58.485 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:21:58.484+0000 7fe91ef62640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fe91815d2a0 0x7fe918151510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:21:58.485 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:21:58.484+0000 7fe91ef62640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fe91815d680 0x7fe9180ca170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:21:58.486 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:21:58.484+0000 7fe91ef62640 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe918151a50 con 0x7fe9180d1d90 2023-11-26T07:21:58.486 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:21:58.484+0000 7fe91ef62640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe9180ca9c0 con 0x7fe91815d680 2023-11-26T07:21:58.486 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:21:58.484+0000 7fe91ef62640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe9180caca0 con 0x7fe91815d2a0 2023-11-26T07:21:58.486 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:21:58.484+0000 7fe91df60640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fe9180d1d90 0x7fe918157ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:21:58.486 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:21:58.484+0000 7fe91df60640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fe9180d1d90 0x7fe918157ce0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:57194/0 (socket says 172.21.15.143:57194) 2023-11-26T07:21:58.486 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:21:58.484+0000 7fe91df60640 1 -- 172.21.15.143:0/1346688828 learned_addr learned my addr 172.21.15.143:0/1346688828 (peer_addr_for_me v2:172.21.15.143:0/0) 2023-11-26T07:21:58.486 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:21:58.484+0000 7fe91e761640 1 --2- 172.21.15.143:0/1346688828 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fe91815d680 0x7fe9180ca170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:21:58.486 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:21:58.484+0000 7fe91d75f640 1 --2- 172.21.15.143:0/1346688828 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fe91815d2a0 0x7fe918151510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:21:58.487 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:21:58.484+0000 7fe91df60640 1 -- 172.21.15.143:0/1346688828 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fe91815d2a0 msgr2=0x7fe918151510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:21:58.487 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:21:58.484+0000 7fe91df60640 1 --2- 172.21.15.143:0/1346688828 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fe91815d2a0 0x7fe918151510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:21:58.487 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:21:58.484+0000 7fe91df60640 1 -- 172.21.15.143:0/1346688828 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fe91815d680 msgr2=0x7fe9180ca170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:21:58.487 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:21:58.484+0000 7fe91df60640 1 --2- 172.21.15.143:0/1346688828 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fe91815d680 0x7fe9180ca170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:21:58.487 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:21:58.484+0000 7fe91df60640 1 -- 172.21.15.143:0/1346688828 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe914009420 con 0x7fe9180d1d90 2023-11-26T07:21:58.487 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:21:58.484+0000 7fe91e761640 1 --2- 172.21.15.143:0/1346688828 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fe91815d680 0x7fe9180ca170 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-26T07:21:58.487 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:21:58.484+0000 7fe91df60640 1 --2- 172.21.15.143:0/1346688828 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fe9180d1d90 0x7fe918157ce0 secure :-1 s=READY pgs=794 cs=0 l=1 rev1=1 crypto rx=0x7fe9140b0040 tx=0x7fe9140191f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:21:58.488 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:21:58.484+0000 7fe90e7fc640 1 -- 172.21.15.143:0/1346688828 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe9140b6610 con 0x7fe9180d1d90 2023-11-26T07:21:58.488 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:21:58.484+0000 7fe90e7fc640 1 -- 172.21.15.143:0/1346688828 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe9140b6790 con 0x7fe9180d1d90 2023-11-26T07:21:58.488 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:21:58.484+0000 7fe90e7fc640 1 -- 172.21.15.143:0/1346688828 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe9140b6b40 con 0x7fe9180d1d90 2023-11-26T07:21:58.489 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:21:58.484+0000 7fe91ef62640 1 -- 172.21.15.143:0/1346688828 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe9180caf80 con 0x7fe9180d1d90 2023-11-26T07:21:58.489 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:21:58.484+0000 7fe91ef62640 1 -- 172.21.15.143:0/1346688828 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe9180cb540 con 0x7fe9180d1d90 2023-11-26T07:21:58.489 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:21:58.484+0000 7fe91ef62640 1 -- 172.21.15.143:0/1346688828 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe918067500 con 0x7fe9180d1d90 2023-11-26T07:21:58.489 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:21:58.484+0000 7fe90e7fc640 1 -- 172.21.15.143:0/1346688828 <== mon.0 v2:172.21.15.143:3300/0 4 ==== mgrmap(e 9) v1 ==== 86005+0+0 (secure 0 0 0) 0x7fe9140b4040 con 0x7fe9180d1d90 2023-11-26T07:21:58.489 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:21:58.488+0000 7fe90e7fc640 1 --2- 172.21.15.143:0/1346688828 >> [v2:172.21.15.143:6832/2150557544,v1:172.21.15.143:6833/2150557544] conn(0x7fe8fc066620 0x7fe8fc068ae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:21:58.489 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:21:58.488+0000 7fe90e7fc640 1 -- 172.21.15.143:0/1346688828 <== mon.0 v2:172.21.15.143:3300/0 5 ==== osd_map(59..59 src has 1..59) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fe91411e530 con 0x7fe9180d1d90 2023-11-26T07:21:58.490 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:21:58.488+0000 7fe91d75f640 1 --2- 172.21.15.143:0/1346688828 >> [v2:172.21.15.143:6832/2150557544,v1:172.21.15.143:6833/2150557544] conn(0x7fe8fc066620 0x7fe8fc068ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:21:58.490 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:21:58.488+0000 7fe91d75f640 1 --2- 172.21.15.143:0/1346688828 >> [v2:172.21.15.143:6832/2150557544,v1:172.21.15.143:6833/2150557544] conn(0x7fe8fc066620 0x7fe8fc068ae0 secure :-1 s=READY pgs=381 cs=0 l=1 rev1=1 crypto rx=0x7fe918156c40 tx=0x7fe908009040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:21:58.493 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:21:58.492+0000 7fe90e7fc640 1 -- 172.21.15.143:0/1346688828 <== mon.0 v2:172.21.15.143:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172649 (secure 0 0 0) 0x7fe9140eb590 con 0x7fe9180d1d90 2023-11-26T07:21:58.674 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:21:58.672+0000 7fe91ef62640 1 -- 172.21.15.143:0/1346688828 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"} v 0) v1 -- 0x7fe9180cd560 con 0x7fe9180d1d90 2023-11-26T07:21:59.479 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:21:59.476+0000 7fe90e7fc640 1 -- 172.21.15.143:0/1346688828 <== mon.0 v2:172.21.15.143:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"}]=0 v34) v1 ==== 122+0+0 (secure 0 0 0) 0x7fe9140eebb0 con 0x7fe9180d1d90 2023-11-26T07:21:59.483 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:21:59.480+0000 7fe91ef62640 1 -- 172.21.15.143:0/1346688828 >> [v2:172.21.15.143:6832/2150557544,v1:172.21.15.143:6833/2150557544] conn(0x7fe8fc066620 msgr2=0x7fe8fc068ae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:21:59.483 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:21:59.480+0000 7fe91ef62640 1 --2- 172.21.15.143:0/1346688828 >> [v2:172.21.15.143:6832/2150557544,v1:172.21.15.143:6833/2150557544] conn(0x7fe8fc066620 0x7fe8fc068ae0 secure :-1 s=READY pgs=381 cs=0 l=1 rev1=1 crypto rx=0x7fe918156c40 tx=0x7fe908009040 comp rx=0 tx=0).stop 2023-11-26T07:21:59.483 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:21:59.480+0000 7fe91ef62640 1 -- 172.21.15.143:0/1346688828 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fe9180d1d90 msgr2=0x7fe918157ce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:21:59.483 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:21:59.480+0000 7fe91ef62640 1 --2- 172.21.15.143:0/1346688828 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fe9180d1d90 0x7fe918157ce0 secure :-1 s=READY pgs=794 cs=0 l=1 rev1=1 crypto rx=0x7fe9140b0040 tx=0x7fe9140191f0 comp rx=0 tx=0).stop 2023-11-26T07:21:59.483 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:21:59.480+0000 7fe91ef62640 1 -- 172.21.15.143:0/1346688828 shutdown_connections 2023-11-26T07:21:59.483 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:21:59.480+0000 7fe91ef62640 1 --2- 172.21.15.143:0/1346688828 >> [v2:172.21.15.143:6832/2150557544,v1:172.21.15.143:6833/2150557544] conn(0x7fe8fc066620 0x7fe8fc068ae0 unknown :-1 s=CLOSED pgs=381 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:21:59.483 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:21:59.480+0000 7fe91ef62640 1 --2- 172.21.15.143:0/1346688828 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fe91815d680 0x7fe9180ca170 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:21:59.483 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:21:59.480+0000 7fe91ef62640 1 --2- 172.21.15.143:0/1346688828 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fe91815d2a0 0x7fe918151510 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:21:59.483 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:21:59.480+0000 7fe91ef62640 1 --2- 172.21.15.143:0/1346688828 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fe9180d1d90 0x7fe918157ce0 unknown :-1 s=CLOSED pgs=794 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:21:59.483 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:21:59.480+0000 7fe91ef62640 1 -- 172.21.15.143:0/1346688828 >> 172.21.15.143:0/1346688828 conn(0x7fe9180d30a0 msgr2=0x7fe91806bdf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:21:59.483 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:21:59.480+0000 7fe91ef62640 1 -- 172.21.15.143:0/1346688828 shutdown_connections 2023-11-26T07:21:59.483 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:21:59.480+0000 7fe91ef62640 1 -- 172.21.15.143:0/1346688828 wait complete. 2023-11-26T07:21:59.507 DEBUG:tasks.cephfs.filesystem:Not Creating any subvols for filesystem 'cephfs' 2023-11-26T07:21:59.507 DEBUG:teuthology.orchestra.run.smithi143:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-11-26T07:21:59.647 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:21:59.644+0000 7f29a2e17640 1 Processor -- start 2023-11-26T07:21:59.647 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:21:59.644+0000 7f29a2e17640 1 -- start start 2023-11-26T07:21:59.649 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:21:59.648+0000 7f29a2e17640 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7f299c06f000 0x7f299c06d030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:21:59.649 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:21:59.648+0000 7f29a2e17640 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7f299c06f3e0 0x7f299c06b5a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:21:59.649 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:21:59.648+0000 7f29a2e17640 1 -- --> v1:172.21.15.143:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f299c0cf260 con 0x7f299c099a70 2023-11-26T07:21:59.649 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:21:59.648+0000 7f29a2e17640 1 -- --> v2:172.21.15.143:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f299c06d570 con 0x7f299c06f000 2023-11-26T07:21:59.650 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:21:59.648+0000 7f29a2e17640 1 -- --> v2:172.21.15.188:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f299c09a010 con 0x7f299c06f3e0 2023-11-26T07:21:59.650 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:21:59.648+0000 7f29a0bb3640 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7f299c06f000 0x7f299c06d030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:21:59.650 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:21:59.648+0000 7f29a0bb3640 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7f299c06f000 0x7f299c06d030 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:57198/0 (socket says 172.21.15.143:57198) 2023-11-26T07:21:59.650 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:21:59.648+0000 7f29a0bb3640 1 -- 172.21.15.143:0/2516463475 learned_addr learned my addr 172.21.15.143:0/2516463475 (peer_addr_for_me v2:172.21.15.143:0/0) 2023-11-26T07:21:59.650 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:21:59.648+0000 7f29a13b4640 1 --1- 172.21.15.143:0/2516463475 >> v1:172.21.15.143:6789/0 conn(0x7f299c099a70 0x7f299c06bae0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.143:6789/0 says I am v1:172.21.15.143:57246/0 (socket says 172.21.15.143:57246) 2023-11-26T07:21:59.650 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:21:59.648+0000 7f2993fff640 1 --2- 172.21.15.143:0/2516463475 >> v2:172.21.15.188:3300/0 conn(0x7f299c06f3e0 0x7f299c06b5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:21:59.650 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:21:59.648+0000 7f29a0bb3640 1 -- 172.21.15.143:0/2516463475 >> v2:172.21.15.188:3300/0 conn(0x7f299c06f3e0 msgr2=0x7f299c06b5a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-26T07:21:59.650 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:21:59.648+0000 7f29a0bb3640 1 --2- 172.21.15.143:0/2516463475 >> v2:172.21.15.188:3300/0 conn(0x7f299c06f3e0 0x7f299c06b5a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:21:59.650 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:21:59.648+0000 7f29a0bb3640 1 -- 172.21.15.143:0/2516463475 >> v1:172.21.15.143:6789/0 conn(0x7f299c099a70 legacy=0x7f299c06bae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:21:59.650 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:21:59.648+0000 7f29a0bb3640 1 -- 172.21.15.143:0/2516463475 --> v2:172.21.15.143:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f299c09a2f0 con 0x7f299c06f000 2023-11-26T07:21:59.650 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:21:59.648+0000 7f29937fe640 1 -- 172.21.15.143:0/2516463475 <== mon.0 v1:172.21.15.143:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1001336546 0 0) 0x7f299c0cf260 con 0x7f299c099a70 2023-11-26T07:21:59.652 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:21:59.652+0000 7f29a0bb3640 1 --2- 172.21.15.143:0/2516463475 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f299c06f000 0x7f299c06d030 secure :-1 s=READY pgs=796 cs=0 l=1 rev1=1 crypto rx=0x7f299c06e4d0 tx=0x7f299801f1c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fed451574f032152 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:21:59.652 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:21:59.652+0000 7f29937fe640 1 -- 172.21.15.143:0/2516463475 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f29980af400 con 0x7f299c06f000 2023-11-26T07:21:59.652 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:21:59.652+0000 7f29937fe640 1 -- 172.21.15.143:0/2516463475 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f29980ab050 con 0x7f299c06f000 2023-11-26T07:21:59.652 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:21:59.652+0000 7f29937fe640 1 -- 172.21.15.143:0/2516463475 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f29980af780 con 0x7f299c06f000 2023-11-26T07:21:59.653 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:21:59.652+0000 7f29a2e17640 1 -- 172.21.15.143:0/2516463475 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f299c06f000 msgr2=0x7f299c06d030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:21:59.653 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:21:59.652+0000 7f29a2e17640 1 --2- 172.21.15.143:0/2516463475 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f299c06f000 0x7f299c06d030 secure :-1 s=READY pgs=796 cs=0 l=1 rev1=1 crypto rx=0x7f299c06e4d0 tx=0x7f299801f1c0 comp rx=0 tx=0).stop 2023-11-26T07:21:59.653 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:21:59.652+0000 7f29a2e17640 1 -- 172.21.15.143:0/2516463475 shutdown_connections 2023-11-26T07:21:59.654 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:21:59.652+0000 7f29a2e17640 1 --2- 172.21.15.143:0/2516463475 >> v2:172.21.15.188:3300/0 conn(0x7f299c06f3e0 0x7f299c06b5a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:21:59.654 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:21:59.652+0000 7f29a2e17640 1 --2- 172.21.15.143:0/2516463475 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f299c06f000 0x7f299c06d030 unknown :-1 s=CLOSED pgs=796 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:21:59.654 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:21:59.652+0000 7f29a2e17640 1 -- 172.21.15.143:0/2516463475 >> 172.21.15.143:0/2516463475 conn(0x7f299c0b2d10 msgr2=0x7f299c0b3120 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:21:59.654 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:21:59.652+0000 7f29a2e17640 1 -- 172.21.15.143:0/2516463475 shutdown_connections 2023-11-26T07:21:59.654 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:21:59.652+0000 7f29a2e17640 1 -- 172.21.15.143:0/2516463475 wait complete. 2023-11-26T07:21:59.655 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:21:59.652+0000 7f29a2e17640 1 Processor -- start 2023-11-26T07:21:59.655 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:21:59.652+0000 7f29a2e17640 1 -- start start 2023-11-26T07:21:59.658 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:21:59.652+0000 7f29a2e17640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f299c06f000 0x7f299c1f3010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:21:59.658 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:21:59.656+0000 7f29a2e17640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f299c06f3e0 0x7f299c0cd530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:21:59.658 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:21:59.656+0000 7f29a2e17640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f299c099a70 0x7f299c1e6480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:21:59.658 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:21:59.656+0000 7f29a2e17640 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f299c0c93e0 con 0x7f299c06f000 2023-11-26T07:21:59.658 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:21:59.656+0000 7f29a2e17640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f299c0cda70 con 0x7f299c06f3e0 2023-11-26T07:21:59.658 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:21:59.656+0000 7f29a2e17640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f299c1f3660 con 0x7f299c099a70 2023-11-26T07:21:59.658 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:21:59.656+0000 7f29a0bb3640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f299c06f000 0x7f299c1f3010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:21:59.658 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:21:59.656+0000 7f29a0bb3640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f299c06f000 0x7f299c1f3010 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:57210/0 (socket says 172.21.15.143:57210) 2023-11-26T07:21:59.658 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:21:59.656+0000 7f29a0bb3640 1 -- 172.21.15.143:0/1090383543 learned_addr learned my addr 172.21.15.143:0/1090383543 (peer_addr_for_me v2:172.21.15.143:0/0) 2023-11-26T07:21:59.658 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:21:59.656+0000 7f2993fff640 1 --2- 172.21.15.143:0/1090383543 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f299c06f3e0 0x7f299c0cd530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:21:59.659 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:21:59.656+0000 7f29a0bb3640 1 -- 172.21.15.143:0/1090383543 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f299c099a70 msgr2=0x7f299c1e6480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:21:59.659 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:21:59.656+0000 7f29a0bb3640 1 --2- 172.21.15.143:0/1090383543 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f299c099a70 0x7f299c1e6480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:21:59.659 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:21:59.656+0000 7f29a0bb3640 1 -- 172.21.15.143:0/1090383543 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f299c06f3e0 msgr2=0x7f299c0cd530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:21:59.659 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:21:59.656+0000 7f29a0bb3640 1 --2- 172.21.15.143:0/1090383543 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f299c06f3e0 0x7f299c0cd530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:21:59.659 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:21:59.656+0000 7f29a0bb3640 1 -- 172.21.15.143:0/1090383543 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2998009b30 con 0x7f299c06f000 2023-11-26T07:21:59.659 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:21:59.656+0000 7f2993fff640 1 --2- 172.21.15.143:0/1090383543 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f299c06f3e0 0x7f299c0cd530 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-26T07:21:59.659 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:21:59.656+0000 7f29a0bb3640 1 --2- 172.21.15.143:0/1090383543 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f299c06f000 0x7f299c1f3010 secure :-1 s=READY pgs=797 cs=0 l=1 rev1=1 crypto rx=0x7f299801bdd0 tx=0x7f299801bf60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:21:59.659 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:21:59.656+0000 7f29917fa640 1 -- 172.21.15.143:0/1090383543 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f29980afe80 con 0x7f299c06f000 2023-11-26T07:21:59.659 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:21:59.656+0000 7f29917fa640 1 -- 172.21.15.143:0/1090383543 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2998009510 con 0x7f299c06f000 2023-11-26T07:21:59.659 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:21:59.656+0000 7f29a2e17640 1 -- 172.21.15.143:0/1090383543 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f299c1f3940 con 0x7f299c06f000 2023-11-26T07:21:59.659 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:21:59.656+0000 7f29917fa640 1 -- 172.21.15.143:0/1090383543 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f29980ba660 con 0x7f299c06f000 2023-11-26T07:21:59.659 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:21:59.656+0000 7f29a2e17640 1 -- 172.21.15.143:0/1090383543 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f299c0ce0b0 con 0x7f299c06f000 2023-11-26T07:21:59.660 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:21:59.656+0000 7f29a2e17640 1 -- 172.21.15.143:0/1090383543 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f299c0c9550 con 0x7f299c06f000 2023-11-26T07:21:59.660 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:21:59.656+0000 7f29917fa640 1 -- 172.21.15.143:0/1090383543 <== mon.0 v2:172.21.15.143:3300/0 4 ==== mgrmap(e 9) v1 ==== 86005+0+0 (secure 0 0 0) 0x7f29980b6040 con 0x7f299c06f000 2023-11-26T07:21:59.660 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:21:59.660+0000 7f29917fa640 1 --2- 172.21.15.143:0/1090383543 >> [v2:172.21.15.143:6832/2150557544,v1:172.21.15.143:6833/2150557544] conn(0x7f2988066620 0x7f2988068ae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:21:59.660 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:21:59.660+0000 7f29917fa640 1 -- 172.21.15.143:0/1090383543 <== mon.0 v2:172.21.15.143:3300/0 5 ==== osd_map(59..59 src has 1..59) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f2998122500 con 0x7f299c06f000 2023-11-26T07:21:59.661 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:21:59.660+0000 7f2993fff640 1 --2- 172.21.15.143:0/1090383543 >> [v2:172.21.15.143:6832/2150557544,v1:172.21.15.143:6833/2150557544] conn(0x7f2988066620 0x7f2988068ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:21:59.661 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:21:59.660+0000 7f2993fff640 1 --2- 172.21.15.143:0/1090383543 >> [v2:172.21.15.143:6832/2150557544,v1:172.21.15.143:6833/2150557544] conn(0x7f2988066620 0x7f2988068ae0 secure :-1 s=READY pgs=382 cs=0 l=1 rev1=1 crypto rx=0x7f299c0b28d0 tx=0x7f298c008040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:21:59.665 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:21:59.664+0000 7f29917fa640 1 -- 172.21.15.143:0/1090383543 <== mon.0 v2:172.21.15.143:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172649 (secure 0 0 0) 0x7f29980ef680 con 0x7f299c06f000 2023-11-26T07:21:59.845 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:21:59.844+0000 7f29a2e17640 1 -- 172.21.15.143:0/1090383543 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f299c067680 con 0x7f299c06f000 2023-11-26T07:21:59.846 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:21:59.844+0000 7f29917fa640 1 -- 172.21.15.143:0/1090383543 <== mon.0 v2:172.21.15.143:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 34 v34) v1 ==== 94+0+4925 (secure 0 0 0) 0x7f29980f2ca0 con 0x7f299c06f000 2023-11-26T07:21:59.846 INFO:teuthology.orchestra.run.smithi143.stdout: 2023-11-26T07:21:59.846 INFO:teuthology.orchestra.run.smithi143.stdout:{"epoch":34,"default_fscid":4,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5612,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.143:6837/1372238588","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6836","nonce":1372238588},{"type":"v1","addr":"172.21.15.143:6837","nonce":1372238588}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":29},{"gid":6103,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.143:6835/1078738477","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6834","nonce":1078738477},{"type":"v1","addr":"172.21.15.143:6835","nonce":1078738477}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":29},{"gid":6118,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.188:6834/1634083777","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6832","nonce":1634083777},{"type":"v1","addr":"172.21.15.188:6834","nonce":1634083777}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":29}],"filesystems":[{"mdsmap":{"epoch":34,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-11-26T07:21:55.656168+0000","modified":"2023-11-26T07:21:58.674916+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":6121},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6121":{"gid":6121,"name":"b","rank":0,"incarnation":30,"state":"up:active","state_seq":3,"addr":"172.21.15.188:6835/1958716738","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6833","nonce":1958716738},{"type":"v1","addr":"172.21.15.188:6835","nonce":1958716738}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[9],"metadata_pool":8,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":4}]} 2023-11-26T07:21:59.846 INFO:teuthology.orchestra.run.smithi143.stderr:dumped fsmap epoch 34 2023-11-26T07:21:59.848 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:21:59.844+0000 7f29a2e17640 1 -- 172.21.15.143:0/1090383543 >> [v2:172.21.15.143:6832/2150557544,v1:172.21.15.143:6833/2150557544] conn(0x7f2988066620 msgr2=0x7f2988068ae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:21:59.848 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:21:59.844+0000 7f29a2e17640 1 --2- 172.21.15.143:0/1090383543 >> [v2:172.21.15.143:6832/2150557544,v1:172.21.15.143:6833/2150557544] conn(0x7f2988066620 0x7f2988068ae0 secure :-1 s=READY pgs=382 cs=0 l=1 rev1=1 crypto rx=0x7f299c0b28d0 tx=0x7f298c008040 comp rx=0 tx=0).stop 2023-11-26T07:21:59.848 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:21:59.844+0000 7f29a2e17640 1 -- 172.21.15.143:0/1090383543 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f299c06f000 msgr2=0x7f299c1f3010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:21:59.848 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:21:59.844+0000 7f29a2e17640 1 --2- 172.21.15.143:0/1090383543 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f299c06f000 0x7f299c1f3010 secure :-1 s=READY pgs=797 cs=0 l=1 rev1=1 crypto rx=0x7f299801bdd0 tx=0x7f299801bf60 comp rx=0 tx=0).stop 2023-11-26T07:21:59.848 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:21:59.844+0000 7f29a2e17640 1 -- 172.21.15.143:0/1090383543 shutdown_connections 2023-11-26T07:21:59.848 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:21:59.844+0000 7f29a2e17640 1 --2- 172.21.15.143:0/1090383543 >> [v2:172.21.15.143:6832/2150557544,v1:172.21.15.143:6833/2150557544] conn(0x7f2988066620 0x7f2988068ae0 unknown :-1 s=CLOSED pgs=382 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:21:59.848 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:21:59.844+0000 7f29a2e17640 1 --2- 172.21.15.143:0/1090383543 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f299c099a70 0x7f299c1e6480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:21:59.849 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:21:59.844+0000 7f29a2e17640 1 --2- 172.21.15.143:0/1090383543 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f299c06f3e0 0x7f299c0cd530 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:21:59.849 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:21:59.844+0000 7f29a2e17640 1 --2- 172.21.15.143:0/1090383543 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f299c06f000 0x7f299c1f3010 unknown :-1 s=CLOSED pgs=797 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:21:59.849 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:21:59.844+0000 7f29a2e17640 1 -- 172.21.15.143:0/1090383543 >> 172.21.15.143:0/1090383543 conn(0x7f299c0b2d10 msgr2=0x7f299c06cc50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:21:59.849 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:21:59.844+0000 7f29a2e17640 1 -- 172.21.15.143:0/1090383543 shutdown_connections 2023-11-26T07:21:59.849 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:21:59.844+0000 7f29a2e17640 1 -- 172.21.15.143:0/1090383543 wait complete. 2023-11-26T07:21:59.868 DEBUG:teuthology.orchestra.run.smithi143:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-11-26T07:22:00.006 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:00.004+0000 7f9aad196640 1 Processor -- start 2023-11-26T07:22:00.006 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:00.004+0000 7f9aad196640 1 -- start start 2023-11-26T07:22:00.008 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:00.004+0000 7f9aad196640 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7f9aa8143810 0x7f9aa8072140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:22:00.008 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:00.004+0000 7f9aad196640 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7f9aa815e0f0 0x7f9aa80ce030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:22:00.008 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:00.004+0000 7f9aad196640 1 -- --> v1:172.21.15.188:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9aa815e860 con 0x7f9aa80b1e10 2023-11-26T07:22:00.008 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:00.004+0000 7f9aad196640 1 -- --> v2:172.21.15.143:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9aa80cdba0 con 0x7f9aa8143810 2023-11-26T07:22:00.008 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:00.004+0000 7f9aad196640 1 -- --> v2:172.21.15.188:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9aa8143bf0 con 0x7f9aa815e0f0 2023-11-26T07:22:00.008 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:00.004+0000 7f9aa7fff640 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7f9aa8143810 0x7f9aa8072140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:22:00.008 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:00.004+0000 7f9aa7fff640 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7f9aa8143810 0x7f9aa8072140 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:57212/0 (socket says 172.21.15.143:57212) 2023-11-26T07:22:00.009 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:00.004+0000 7f9aa7fff640 1 -- 172.21.15.143:0/3892058838 learned_addr learned my addr 172.21.15.143:0/3892058838 (peer_addr_for_me v2:172.21.15.143:0/0) 2023-11-26T07:22:00.009 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:00.004+0000 7f9aac995640 1 --2- 172.21.15.143:0/3892058838 >> v2:172.21.15.188:3300/0 conn(0x7f9aa815e0f0 0x7f9aa80ce030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:22:00.009 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:00.008+0000 7f9aa7fff640 1 -- 172.21.15.143:0/3892058838 >> v2:172.21.15.188:3300/0 conn(0x7f9aa815e0f0 msgr2=0x7f9aa80ce030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-26T07:22:00.009 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:00.008+0000 7f9aa7fff640 1 --2- 172.21.15.143:0/3892058838 >> v2:172.21.15.188:3300/0 conn(0x7f9aa815e0f0 0x7f9aa80ce030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:22:00.009 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:00.008+0000 7f9aa7fff640 1 -- 172.21.15.143:0/3892058838 >> v1:172.21.15.188:6789/0 conn(0x7f9aa80b1e10 legacy=0x7f9aa815d9e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:22:00.009 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:00.008+0000 7f9aa7fff640 1 -- 172.21.15.143:0/3892058838 --> v2:172.21.15.143:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9aa80714f0 con 0x7f9aa8143810 2023-11-26T07:22:00.009 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:00.008+0000 7f9aac995640 1 --2- 172.21.15.143:0/3892058838 >> v2:172.21.15.188:3300/0 conn(0x7f9aa815e0f0 0x7f9aa80ce030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-26T07:22:00.011 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:00.008+0000 7f9aa7fff640 1 --2- 172.21.15.143:0/3892058838 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f9aa8143810 0x7f9aa8072140 secure :-1 s=READY pgs=798 cs=0 l=1 rev1=1 crypto rx=0x7f9aa806d020 tx=0x7f9a9801c4d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6e8f8731e7ad641b server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:22:00.011 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:00.008+0000 7f9aa6ffd640 1 -- 172.21.15.143:0/3892058838 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9a980aa430 con 0x7f9aa8143810 2023-11-26T07:22:00.012 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:00.008+0000 7f9aa6ffd640 1 -- 172.21.15.143:0/3892058838 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9a980aa590 con 0x7f9aa8143810 2023-11-26T07:22:00.012 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:00.008+0000 7f9aa6ffd640 1 -- 172.21.15.143:0/3892058838 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9a980aa920 con 0x7f9aa8143810 2023-11-26T07:22:00.012 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:00.012+0000 7f9aad196640 1 -- 172.21.15.143:0/3892058838 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f9aa8143810 msgr2=0x7f9aa8072140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:22:00.012 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:00.012+0000 7f9aad196640 1 --2- 172.21.15.143:0/3892058838 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f9aa8143810 0x7f9aa8072140 secure :-1 s=READY pgs=798 cs=0 l=1 rev1=1 crypto rx=0x7f9aa806d020 tx=0x7f9a9801c4d0 comp rx=0 tx=0).stop 2023-11-26T07:22:00.013 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:00.012+0000 7f9aad196640 1 -- 172.21.15.143:0/3892058838 shutdown_connections 2023-11-26T07:22:00.013 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:00.012+0000 7f9aad196640 1 --2- 172.21.15.143:0/3892058838 >> v2:172.21.15.188:3300/0 conn(0x7f9aa815e0f0 0x7f9aa80ce030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:22:00.013 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:00.012+0000 7f9aad196640 1 --2- 172.21.15.143:0/3892058838 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f9aa8143810 0x7f9aa8072140 unknown :-1 s=CLOSED pgs=798 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:22:00.013 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:00.012+0000 7f9aad196640 1 -- 172.21.15.143:0/3892058838 >> 172.21.15.143:0/3892058838 conn(0x7f9aa806a890 msgr2=0x7f9aa806aca0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:22:00.013 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:00.012+0000 7f9aad196640 1 -- 172.21.15.143:0/3892058838 shutdown_connections 2023-11-26T07:22:00.013 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:00.012+0000 7f9aad196640 1 -- 172.21.15.143:0/3892058838 wait complete. 2023-11-26T07:22:00.014 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:00.012+0000 7f9aad196640 1 Processor -- start 2023-11-26T07:22:00.014 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:00.012+0000 7f9aad196640 1 -- start start 2023-11-26T07:22:00.016 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:00.012+0000 7f9aad196640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f9aa80b1e10 0x7f9aa81556b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:22:00.016 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:00.012+0000 7f9aad196640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f9aa8143810 0x7f9aa813c470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:22:00.016 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:00.012+0000 7f9aad196640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f9aa815e0f0 0x7f9aa81592b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:22:00.016 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:00.012+0000 7f9aad196640 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9aa8158210 con 0x7f9aa815e0f0 2023-11-26T07:22:00.016 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:00.012+0000 7f9aad196640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9aa81584f0 con 0x7f9aa8143810 2023-11-26T07:22:00.016 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:00.012+0000 7f9aad196640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9aa81587d0 con 0x7f9aa80b1e10 2023-11-26T07:22:00.016 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:00.012+0000 7f9aac995640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f9aa815e0f0 0x7f9aa81592b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:22:00.016 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:00.012+0000 7f9aac995640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f9aa815e0f0 0x7f9aa81592b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:57216/0 (socket says 172.21.15.143:57216) 2023-11-26T07:22:00.016 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:00.012+0000 7f9aac995640 1 -- 172.21.15.143:0/4294493511 learned_addr learned my addr 172.21.15.143:0/4294493511 (peer_addr_for_me v2:172.21.15.143:0/0) 2023-11-26T07:22:00.017 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:00.012+0000 7f9aa7fff640 1 --2- 172.21.15.143:0/4294493511 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f9aa80b1e10 0x7f9aa81556b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:22:00.017 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:00.012+0000 7f9aa77fe640 1 --2- 172.21.15.143:0/4294493511 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f9aa8143810 0x7f9aa813c470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:22:00.017 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:00.012+0000 7f9aac995640 1 -- 172.21.15.143:0/4294493511 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f9aa80b1e10 msgr2=0x7f9aa81556b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:22:00.017 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:00.012+0000 7f9aac995640 1 --2- 172.21.15.143:0/4294493511 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f9aa80b1e10 0x7f9aa81556b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:22:00.017 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:00.012+0000 7f9aac995640 1 -- 172.21.15.143:0/4294493511 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f9aa8143810 msgr2=0x7f9aa813c470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:22:00.017 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:00.012+0000 7f9aac995640 1 --2- 172.21.15.143:0/4294493511 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f9aa8143810 0x7f9aa813c470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:22:00.017 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:00.012+0000 7f9aac995640 1 -- 172.21.15.143:0/4294493511 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9a980094d0 con 0x7f9aa815e0f0 2023-11-26T07:22:00.018 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:00.016+0000 7f9aac995640 1 --2- 172.21.15.143:0/4294493511 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f9aa815e0f0 0x7f9aa81592b0 secure :-1 s=READY pgs=799 cs=0 l=1 rev1=1 crypto rx=0x7f9aa0007df0 tx=0x7f9aa000d4a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:22:00.018 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:00.016+0000 7f9aa4ff9640 1 -- 172.21.15.143:0/4294493511 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9aa00039a0 con 0x7f9aa815e0f0 2023-11-26T07:22:00.018 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:00.016+0000 7f9aa4ff9640 1 -- 172.21.15.143:0/4294493511 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9aa000f040 con 0x7f9aa815e0f0 2023-11-26T07:22:00.018 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:00.016+0000 7f9aa4ff9640 1 -- 172.21.15.143:0/4294493511 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9aa0003d40 con 0x7f9aa815e0f0 2023-11-26T07:22:00.018 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:00.016+0000 7f9aad196640 1 -- 172.21.15.143:0/4294493511 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9aa8158ab0 con 0x7f9aa815e0f0 2023-11-26T07:22:00.018 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:00.016+0000 7f9aad196640 1 -- 172.21.15.143:0/4294493511 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9aa8155bf0 con 0x7f9aa815e0f0 2023-11-26T07:22:00.018 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:00.016+0000 7f9aad196640 1 -- 172.21.15.143:0/4294493511 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9aa8067500 con 0x7f9aa815e0f0 2023-11-26T07:22:00.019 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:00.016+0000 7f9aa4ff9640 1 -- 172.21.15.143:0/4294493511 <== mon.0 v2:172.21.15.143:3300/0 4 ==== mgrmap(e 9) v1 ==== 86005+0+0 (secure 0 0 0) 0x7f9aa0004030 con 0x7f9aa815e0f0 2023-11-26T07:22:00.019 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:00.016+0000 7f9aa4ff9640 1 --2- 172.21.15.143:0/4294493511 >> [v2:172.21.15.143:6832/2150557544,v1:172.21.15.143:6833/2150557544] conn(0x7f9a94066620 0x7f9a94068ae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:22:00.020 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:00.016+0000 7f9aa4ff9640 1 -- 172.21.15.143:0/4294493511 <== mon.0 v2:172.21.15.143:3300/0 5 ==== osd_map(59..59 src has 1..59) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f9aa0084450 con 0x7f9aa815e0f0 2023-11-26T07:22:00.020 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:00.016+0000 7f9aa7fff640 1 --2- 172.21.15.143:0/4294493511 >> [v2:172.21.15.143:6832/2150557544,v1:172.21.15.143:6833/2150557544] conn(0x7f9a94066620 0x7f9a94068ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:22:00.020 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:00.020+0000 7f9aa7fff640 1 --2- 172.21.15.143:0/4294493511 >> [v2:172.21.15.143:6832/2150557544,v1:172.21.15.143:6833/2150557544] conn(0x7f9a94066620 0x7f9a94068ae0 secure :-1 s=READY pgs=383 cs=0 l=1 rev1=1 crypto rx=0x7f9a980ae000 tx=0x7f9a980b5040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:22:00.023 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:00.020+0000 7f9aa4ff9640 1 -- 172.21.15.143:0/4294493511 <== mon.0 v2:172.21.15.143:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172649 (secure 0 0 0) 0x7f9aa0051680 con 0x7f9aa815e0f0 2023-11-26T07:22:00.174 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:00.172+0000 7f9aad196640 1 -- 172.21.15.143:0/4294493511 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f9aa80b4010 con 0x7f9aa815e0f0 2023-11-26T07:22:00.175 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:00.172+0000 7f9aa4ff9640 1 -- 172.21.15.143:0/4294493511 <== mon.0 v2:172.21.15.143:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v59) v1 ==== 74+0+16601 (secure 0 0 0) 0x7f9aa0054ca0 con 0x7f9aa815e0f0 2023-11-26T07:22:00.176 INFO:teuthology.orchestra.run.smithi143.stdout: 2023-11-26T07:22:00.176 INFO:teuthology.orchestra.run.smithi143.stdout:{"epoch":59,"fsid":"f5033060-2469-4a07-95fe-2b76f5404bc1","created":"2023-11-26T07:17:28.613663+0000","modified":"2023-11-26T07:21:57.286555+0000","last_up_change":"2023-11-26T07:17:41.157813+0000","last_in_change":"2023-11-26T07:17:31.181658+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":9,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-11-26T07:17:41.758597+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":8,"pool_name":"cephfs_metadata","create_time":"2023-11-26T07:21:53.382407+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"58","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":9,"pool_name":"cephfs_data","create_time":"2023-11-26T07:21:54.656298+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"59","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.6299999952316284,"score_stable":1.6299999952316284,"optimal_score":1,"raw_score_acting":1.6299999952316284,"raw_score_stable":1.6299999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"ebbee2c6-023e-44b9-9f1d-6f7bfd309730","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":57,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6817","nonce":639807369},{"type":"v1","addr":"172.21.15.143:6819","nonce":639807369}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6821","nonce":639807369},{"type":"v1","addr":"172.21.15.143:6823","nonce":639807369}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6830","nonce":639807369},{"type":"v1","addr":"172.21.15.143:6831","nonce":639807369}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6825","nonce":639807369},{"type":"v1","addr":"172.21.15.143:6828","nonce":639807369}]},"public_addr":"172.21.15.143:6819/639807369","cluster_addr":"172.21.15.143:6823/639807369","heartbeat_back_addr":"172.21.15.143:6831/639807369","heartbeat_front_addr":"172.21.15.143:6828/639807369","state":["exists","up"]},{"osd":1,"uuid":"fec4b8be-7122-49e9-9a19-5850a0700001","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":57,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6816","nonce":1144942530},{"type":"v1","addr":"172.21.15.143:6818","nonce":1144942530}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6820","nonce":1144942530},{"type":"v1","addr":"172.21.15.143:6822","nonce":1144942530}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6827","nonce":1144942530},{"type":"v1","addr":"172.21.15.143:6829","nonce":1144942530}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6824","nonce":1144942530},{"type":"v1","addr":"172.21.15.143:6826","nonce":1144942530}]},"public_addr":"172.21.15.143:6818/1144942530","cluster_addr":"172.21.15.143:6822/1144942530","heartbeat_back_addr":"172.21.15.143:6829/1144942530","heartbeat_front_addr":"172.21.15.143:6826/1144942530","state":["exists","up"]},{"osd":2,"uuid":"2888af5b-f933-46ca-8f95-6dd494ff43f6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":57,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6808","nonce":3909095186},{"type":"v1","addr":"172.21.15.143:6809","nonce":3909095186}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6810","nonce":3909095186},{"type":"v1","addr":"172.21.15.143:6811","nonce":3909095186}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6814","nonce":3909095186},{"type":"v1","addr":"172.21.15.143:6815","nonce":3909095186}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6812","nonce":3909095186},{"type":"v1","addr":"172.21.15.143:6813","nonce":3909095186}]},"public_addr":"172.21.15.143:6809/3909095186","cluster_addr":"172.21.15.143:6811/3909095186","heartbeat_back_addr":"172.21.15.143:6815/3909095186","heartbeat_front_addr":"172.21.15.143:6813/3909095186","state":["exists","up"]},{"osd":3,"uuid":"d1f52435-7bc1-4081-9550-9cd1ba55f549","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":57,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6800","nonce":3854210991},{"type":"v1","addr":"172.21.15.143:6801","nonce":3854210991}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6802","nonce":3854210991},{"type":"v1","addr":"172.21.15.143:6803","nonce":3854210991}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6806","nonce":3854210991},{"type":"v1","addr":"172.21.15.143:6807","nonce":3854210991}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6804","nonce":3854210991},{"type":"v1","addr":"172.21.15.143:6805","nonce":3854210991}]},"public_addr":"172.21.15.143:6801/3854210991","cluster_addr":"172.21.15.143:6803/3854210991","heartbeat_back_addr":"172.21.15.143:6807/3854210991","heartbeat_front_addr":"172.21.15.143:6805/3854210991","state":["exists","up"]},{"osd":4,"uuid":"e2ae8b6e-6e25-49b2-bcbf-aacce68f06a3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":57,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6800","nonce":3924606902},{"type":"v1","addr":"172.21.15.188:6801","nonce":3924606902}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6802","nonce":3924606902},{"type":"v1","addr":"172.21.15.188:6803","nonce":3924606902}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6806","nonce":3924606902},{"type":"v1","addr":"172.21.15.188:6807","nonce":3924606902}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6804","nonce":3924606902},{"type":"v1","addr":"172.21.15.188:6805","nonce":3924606902}]},"public_addr":"172.21.15.188:6801/3924606902","cluster_addr":"172.21.15.188:6803/3924606902","heartbeat_back_addr":"172.21.15.188:6807/3924606902","heartbeat_front_addr":"172.21.15.188:6805/3924606902","state":["exists","up"]},{"osd":5,"uuid":"3cec8a32-7698-4c36-8e89-e4ba11934f33","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":57,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6813","nonce":186841544},{"type":"v1","addr":"172.21.15.188:6815","nonce":186841544}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6817","nonce":186841544},{"type":"v1","addr":"172.21.15.188:6818","nonce":186841544}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6822","nonce":186841544},{"type":"v1","addr":"172.21.15.188:6823","nonce":186841544}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6820","nonce":186841544},{"type":"v1","addr":"172.21.15.188:6821","nonce":186841544}]},"public_addr":"172.21.15.188:6815/186841544","cluster_addr":"172.21.15.188:6818/186841544","heartbeat_back_addr":"172.21.15.188:6823/186841544","heartbeat_front_addr":"172.21.15.188:6821/186841544","state":["exists","up"]},{"osd":6,"uuid":"58a65d5d-e090-473a-81b4-ad38e85029f9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":57,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6808","nonce":2153499405},{"type":"v1","addr":"172.21.15.188:6809","nonce":2153499405}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6810","nonce":2153499405},{"type":"v1","addr":"172.21.15.188:6811","nonce":2153499405}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6816","nonce":2153499405},{"type":"v1","addr":"172.21.15.188:6819","nonce":2153499405}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6812","nonce":2153499405},{"type":"v1","addr":"172.21.15.188:6814","nonce":2153499405}]},"public_addr":"172.21.15.188:6809/2153499405","cluster_addr":"172.21.15.188:6811/2153499405","heartbeat_back_addr":"172.21.15.188:6819/2153499405","heartbeat_front_addr":"172.21.15.188:6814/2153499405","state":["exists","up"]},{"osd":7,"uuid":"ea5b7f70-3554-4cf8-ac30-f7c59b80678d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":57,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6824","nonce":2722096729},{"type":"v1","addr":"172.21.15.188:6825","nonce":2722096729}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6826","nonce":2722096729},{"type":"v1","addr":"172.21.15.188:6827","nonce":2722096729}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6830","nonce":2722096729},{"type":"v1","addr":"172.21.15.188:6831","nonce":2722096729}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6828","nonce":2722096729},{"type":"v1","addr":"172.21.15.188:6829","nonce":2722096729}]},"public_addr":"172.21.15.188:6825/2722096729","cluster_addr":"172.21.15.188:6827/2722096729","heartbeat_back_addr":"172.21.15.188:6831/2722096729","heartbeat_front_addr":"172.21.15.188:6829/2722096729","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-26T07:17:36.995032+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-26T07:17:36.932266+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-26T07:17:36.829304+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-26T07:17:36.804297+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-26T07:17:36.667892+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-26T07:17:36.964868+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-26T07:17:36.827332+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-26T07:17:36.893179+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-11-26T07:22:00.178 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:00.176+0000 7f9aad196640 1 -- 172.21.15.143:0/4294493511 >> [v2:172.21.15.143:6832/2150557544,v1:172.21.15.143:6833/2150557544] conn(0x7f9a94066620 msgr2=0x7f9a94068ae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:22:00.178 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:00.176+0000 7f9aad196640 1 --2- 172.21.15.143:0/4294493511 >> [v2:172.21.15.143:6832/2150557544,v1:172.21.15.143:6833/2150557544] conn(0x7f9a94066620 0x7f9a94068ae0 secure :-1 s=READY pgs=383 cs=0 l=1 rev1=1 crypto rx=0x7f9a980ae000 tx=0x7f9a980b5040 comp rx=0 tx=0).stop 2023-11-26T07:22:00.178 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:00.176+0000 7f9aad196640 1 -- 172.21.15.143:0/4294493511 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f9aa815e0f0 msgr2=0x7f9aa81592b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:22:00.178 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:00.176+0000 7f9aad196640 1 --2- 172.21.15.143:0/4294493511 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f9aa815e0f0 0x7f9aa81592b0 secure :-1 s=READY pgs=799 cs=0 l=1 rev1=1 crypto rx=0x7f9aa0007df0 tx=0x7f9aa000d4a0 comp rx=0 tx=0).stop 2023-11-26T07:22:00.178 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:00.176+0000 7f9aad196640 1 -- 172.21.15.143:0/4294493511 shutdown_connections 2023-11-26T07:22:00.178 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:00.176+0000 7f9aad196640 1 --2- 172.21.15.143:0/4294493511 >> [v2:172.21.15.143:6832/2150557544,v1:172.21.15.143:6833/2150557544] conn(0x7f9a94066620 0x7f9a94068ae0 unknown :-1 s=CLOSED pgs=383 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:22:00.178 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:00.176+0000 7f9aad196640 1 --2- 172.21.15.143:0/4294493511 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f9aa815e0f0 0x7f9aa81592b0 unknown :-1 s=CLOSED pgs=799 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:22:00.178 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:00.176+0000 7f9aad196640 1 --2- 172.21.15.143:0/4294493511 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f9aa8143810 0x7f9aa813c470 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:22:00.178 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:00.176+0000 7f9aad196640 1 --2- 172.21.15.143:0/4294493511 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f9aa80b1e10 0x7f9aa81556b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:22:00.178 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:00.176+0000 7f9aad196640 1 -- 172.21.15.143:0/4294493511 >> 172.21.15.143:0/4294493511 conn(0x7f9aa806a890 msgr2=0x7f9aa80b3230 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:22:00.178 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:00.176+0000 7f9aad196640 1 -- 172.21.15.143:0/4294493511 shutdown_connections 2023-11-26T07:22:00.179 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:00.176+0000 7f9aad196640 1 -- 172.21.15.143:0/4294493511 wait complete. 2023-11-26T07:22:00.197 INFO:tasks.cephfs.filesystem.ceph_manager:waiting for clean 2023-11-26T07:22:00.197 DEBUG:teuthology.orchestra.run.smithi143:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-11-26T07:22:00.334 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:00.332+0000 7f5fec10f640 1 Processor -- start 2023-11-26T07:22:00.334 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:00.332+0000 7f5fec10f640 1 -- start start 2023-11-26T07:22:00.335 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:00.332+0000 7f5fec10f640 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7f5fe414b1e0 0x7f5fe40cf280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:22:00.335 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:00.332+0000 7f5fec10f640 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7f5fe4070fd0 0x7f5fe4162510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:22:00.335 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:00.332+0000 7f5fec10f640 1 -- --> v1:172.21.15.143:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5fe40cfc20 con 0x7f5fe40cf7c0 2023-11-26T07:22:00.336 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:00.332+0000 7f5fec10f640 1 -- --> v2:172.21.15.143:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5fe4162080 con 0x7f5fe414b1e0 2023-11-26T07:22:00.336 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:00.332+0000 7f5fec10f640 1 -- --> v2:172.21.15.188:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5fe4161d70 con 0x7f5fe4070fd0 2023-11-26T07:22:00.336 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:00.332+0000 7f5fe9eab640 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7f5fe414b1e0 0x7f5fe40cf280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:22:00.336 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:00.332+0000 7f5fe9eab640 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7f5fe414b1e0 0x7f5fe40cf280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:57222/0 (socket says 172.21.15.143:57222) 2023-11-26T07:22:00.336 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:00.332+0000 7f5fe9eab640 1 -- 172.21.15.143:0/1722039578 learned_addr learned my addr 172.21.15.143:0/1722039578 (peer_addr_for_me v2:172.21.15.143:0/0) 2023-11-26T07:22:00.336 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:00.332+0000 7f5fe96aa640 1 --1- 172.21.15.143:0/1722039578 >> v1:172.21.15.143:6789/0 conn(0x7f5fe40cf7c0 0x7f5fe40708c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.143:6789/0 says I am v1:172.21.15.143:57252/0 (socket says 172.21.15.143:57252) 2023-11-26T07:22:00.336 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:00.332+0000 7f5fea6ac640 1 --2- 172.21.15.143:0/1722039578 >> v2:172.21.15.188:3300/0 conn(0x7f5fe4070fd0 0x7f5fe4162510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:22:00.336 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:00.332+0000 7f5fe9eab640 1 -- 172.21.15.143:0/1722039578 >> v2:172.21.15.188:3300/0 conn(0x7f5fe4070fd0 msgr2=0x7f5fe4162510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-26T07:22:00.337 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:00.332+0000 7f5fe9eab640 1 --2- 172.21.15.143:0/1722039578 >> v2:172.21.15.188:3300/0 conn(0x7f5fe4070fd0 0x7f5fe4162510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:22:00.337 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:00.332+0000 7f5fe9eab640 1 -- 172.21.15.143:0/1722039578 >> v1:172.21.15.143:6789/0 conn(0x7f5fe40cf7c0 legacy=0x7f5fe40708c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:22:00.337 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:00.332+0000 7f5fe9eab640 1 -- 172.21.15.143:0/1722039578 --> v2:172.21.15.143:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5fe414b5c0 con 0x7f5fe414b1e0 2023-11-26T07:22:00.339 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:00.336+0000 7f5fe9eab640 1 --2- 172.21.15.143:0/1722039578 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f5fe414b1e0 0x7f5fe40cf280 secure :-1 s=READY pgs=801 cs=0 l=1 rev1=1 crypto rx=0x7f5fe4072110 tx=0x7f5fe001c4d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e7e133a5c48a5e53 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:22:00.339 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:00.336+0000 7f5fe8ea9640 1 -- 172.21.15.143:0/1722039578 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5fe00aa430 con 0x7f5fe414b1e0 2023-11-26T07:22:00.339 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:00.336+0000 7f5fe8ea9640 1 -- 172.21.15.143:0/1722039578 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5fe00aa590 con 0x7f5fe414b1e0 2023-11-26T07:22:00.339 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:00.336+0000 7f5fe8ea9640 1 -- 172.21.15.143:0/1722039578 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5fe00aa920 con 0x7f5fe414b1e0 2023-11-26T07:22:00.340 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:00.336+0000 7f5fec10f640 1 -- 172.21.15.143:0/1722039578 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f5fe414b1e0 msgr2=0x7f5fe40cf280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:22:00.340 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:00.336+0000 7f5fec10f640 1 --2- 172.21.15.143:0/1722039578 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f5fe414b1e0 0x7f5fe40cf280 secure :-1 s=READY pgs=801 cs=0 l=1 rev1=1 crypto rx=0x7f5fe4072110 tx=0x7f5fe001c4d0 comp rx=0 tx=0).stop 2023-11-26T07:22:00.340 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:00.336+0000 7f5fec10f640 1 -- 172.21.15.143:0/1722039578 shutdown_connections 2023-11-26T07:22:00.340 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:00.336+0000 7f5fec10f640 1 --2- 172.21.15.143:0/1722039578 >> v2:172.21.15.188:3300/0 conn(0x7f5fe4070fd0 0x7f5fe4162510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:22:00.340 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:00.336+0000 7f5fec10f640 1 --2- 172.21.15.143:0/1722039578 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f5fe414b1e0 0x7f5fe40cf280 unknown :-1 s=CLOSED pgs=801 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:22:00.340 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:00.336+0000 7f5fec10f640 1 -- 172.21.15.143:0/1722039578 >> 172.21.15.143:0/1722039578 conn(0x7f5fe406a890 msgr2=0x7f5fe406aca0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:22:00.340 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:00.336+0000 7f5fec10f640 1 -- 172.21.15.143:0/1722039578 shutdown_connections 2023-11-26T07:22:00.340 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:00.336+0000 7f5fec10f640 1 -- 172.21.15.143:0/1722039578 wait complete. 2023-11-26T07:22:00.341 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:00.340+0000 7f5fec10f640 1 Processor -- start 2023-11-26T07:22:00.341 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:00.340+0000 7f5fec10f640 1 -- start start 2023-11-26T07:22:00.343 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:00.340+0000 7f5fec10f640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f5fe4070fd0 0x7f5fe4160810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:22:00.343 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:00.340+0000 7f5fec10f640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f5fe40cf7c0 0x7f5fe4140f50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:22:00.343 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:00.340+0000 7f5fec10f640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f5fe414b1e0 0x7f5fe4141490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:22:00.344 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:00.340+0000 7f5fec10f640 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5fe415a4c0 con 0x7f5fe4070fd0 2023-11-26T07:22:00.344 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:00.340+0000 7f5fec10f640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5fe415a7a0 con 0x7f5fe414b1e0 2023-11-26T07:22:00.344 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:00.340+0000 7f5fec10f640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5fe415aa80 con 0x7f5fe40cf7c0 2023-11-26T07:22:00.344 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:00.340+0000 7f5fe9eab640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f5fe4070fd0 0x7f5fe4160810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:22:00.344 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:00.340+0000 7f5fe9eab640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f5fe4070fd0 0x7f5fe4160810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:57236/0 (socket says 172.21.15.143:57236) 2023-11-26T07:22:00.344 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:00.340+0000 7f5fe9eab640 1 -- 172.21.15.143:0/1312618963 learned_addr learned my addr 172.21.15.143:0/1312618963 (peer_addr_for_me v2:172.21.15.143:0/0) 2023-11-26T07:22:00.344 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:00.340+0000 7f5fe96aa640 1 --2- 172.21.15.143:0/1312618963 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f5fe40cf7c0 0x7f5fe4140f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:22:00.344 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:00.340+0000 7f5fea6ac640 1 --2- 172.21.15.143:0/1312618963 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f5fe414b1e0 0x7f5fe4141490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:22:00.344 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:00.340+0000 7f5fe9eab640 1 -- 172.21.15.143:0/1312618963 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f5fe40cf7c0 msgr2=0x7f5fe4140f50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:22:00.344 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:00.340+0000 7f5fe9eab640 1 --2- 172.21.15.143:0/1312618963 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f5fe40cf7c0 0x7f5fe4140f50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:22:00.345 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:00.340+0000 7f5fe9eab640 1 -- 172.21.15.143:0/1312618963 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f5fe414b1e0 msgr2=0x7f5fe4141490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:22:00.345 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:00.340+0000 7f5fe9eab640 1 --2- 172.21.15.143:0/1312618963 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f5fe414b1e0 0x7f5fe4141490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:22:00.345 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:00.340+0000 7f5fe9eab640 1 -- 172.21.15.143:0/1312618963 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5fe00094d0 con 0x7f5fe4070fd0 2023-11-26T07:22:00.345 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:00.340+0000 7f5fea6ac640 1 --2- 172.21.15.143:0/1312618963 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f5fe414b1e0 0x7f5fe4141490 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-26T07:22:00.345 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:00.340+0000 7f5fe9eab640 1 --2- 172.21.15.143:0/1312618963 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f5fe4070fd0 0x7f5fe4160810 secure :-1 s=READY pgs=802 cs=0 l=1 rev1=1 crypto rx=0x7f5fe00ac040 tx=0x7f5fe001ad10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:22:00.346 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:00.340+0000 7f5fda7fc640 1 -- 172.21.15.143:0/1312618963 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5fe00aa430 con 0x7f5fe4070fd0 2023-11-26T07:22:00.346 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:00.340+0000 7f5fda7fc640 1 -- 172.21.15.143:0/1312618963 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5fe00b3820 con 0x7f5fe4070fd0 2023-11-26T07:22:00.346 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:00.340+0000 7f5fec10f640 1 -- 172.21.15.143:0/1312618963 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5fe415ad60 con 0x7f5fe4070fd0 2023-11-26T07:22:00.346 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:00.340+0000 7f5fda7fc640 1 -- 172.21.15.143:0/1312618963 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5fe00b3bd0 con 0x7f5fe4070fd0 2023-11-26T07:22:00.346 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:00.340+0000 7f5fec10f640 1 -- 172.21.15.143:0/1312618963 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5fe4160d50 con 0x7f5fe4070fd0 2023-11-26T07:22:00.346 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:00.344+0000 7f5fec10f640 1 -- 172.21.15.143:0/1312618963 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5fe4160f40 con 0x7f5fe4070fd0 2023-11-26T07:22:00.347 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:00.344+0000 7f5fda7fc640 1 -- 172.21.15.143:0/1312618963 <== mon.0 v2:172.21.15.143:3300/0 4 ==== mgrmap(e 9) v1 ==== 86005+0+0 (secure 0 0 0) 0x7f5fe00b3d30 con 0x7f5fe4070fd0 2023-11-26T07:22:00.347 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:00.344+0000 7f5fda7fc640 1 --2- 172.21.15.143:0/1312618963 >> [v2:172.21.15.143:6832/2150557544,v1:172.21.15.143:6833/2150557544] conn(0x7f5fc8066650 0x7f5fc8068b10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:22:00.347 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:00.344+0000 7f5fda7fc640 1 -- 172.21.15.143:0/1312618963 <== mon.0 v2:172.21.15.143:3300/0 5 ==== osd_map(59..59 src has 1..59) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f5fe011eb10 con 0x7f5fe4070fd0 2023-11-26T07:22:00.347 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:00.344+0000 7f5fe96aa640 1 --2- 172.21.15.143:0/1312618963 >> [v2:172.21.15.143:6832/2150557544,v1:172.21.15.143:6833/2150557544] conn(0x7f5fc8066650 0x7f5fc8068b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:22:00.348 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:00.344+0000 7f5fe96aa640 1 --2- 172.21.15.143:0/1312618963 >> [v2:172.21.15.143:6832/2150557544,v1:172.21.15.143:6833/2150557544] conn(0x7f5fc8066650 0x7f5fc8068b10 secure :-1 s=READY pgs=384 cs=0 l=1 rev1=1 crypto rx=0x7f5fe415c6d0 tx=0x7f5fd4007e50 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:22:00.350 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:00.348+0000 7f5fda7fc640 1 -- 172.21.15.143:0/1312618963 <== mon.0 v2:172.21.15.143:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172649 (secure 0 0 0) 0x7f5fe4160f40 con 0x7f5fe4070fd0 2023-11-26T07:22:00.499 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:00.496+0000 7f5fec10f640 1 -- 172.21.15.143:0/1312618963 --> [v2:172.21.15.143:6832/2150557544,v1:172.21.15.143:6833/2150557544] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f5fe40b3b40 con 0x7f5fc8066650 2023-11-26T07:22:00.508 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:00.508+0000 7f5fda7fc640 1 -- 172.21.15.143:0/1312618963 <== mgr.4100 v2:172.21.15.143:6832/2150557544 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+289118 (secure 0 0 0) 0x7f5fe40b3b40 con 0x7f5fc8066650 2023-11-26T07:22:00.511 INFO:teuthology.orchestra.run.smithi143.stdout: 2023-11-26T07:22:00.513 INFO:teuthology.orchestra.run.smithi143.stderr:dumped all 2023-11-26T07:22:00.514 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:00.508+0000 7f5fec10f640 1 -- 172.21.15.143:0/1312618963 >> [v2:172.21.15.143:6832/2150557544,v1:172.21.15.143:6833/2150557544] conn(0x7f5fc8066650 msgr2=0x7f5fc8068b10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:22:00.514 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:00.508+0000 7f5fec10f640 1 --2- 172.21.15.143:0/1312618963 >> [v2:172.21.15.143:6832/2150557544,v1:172.21.15.143:6833/2150557544] conn(0x7f5fc8066650 0x7f5fc8068b10 secure :-1 s=READY pgs=384 cs=0 l=1 rev1=1 crypto rx=0x7f5fe415c6d0 tx=0x7f5fd4007e50 comp rx=0 tx=0).stop 2023-11-26T07:22:00.514 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:00.508+0000 7f5fec10f640 1 -- 172.21.15.143:0/1312618963 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f5fe4070fd0 msgr2=0x7f5fe4160810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:22:00.515 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:00.508+0000 7f5fec10f640 1 --2- 172.21.15.143:0/1312618963 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f5fe4070fd0 0x7f5fe4160810 secure :-1 s=READY pgs=802 cs=0 l=1 rev1=1 crypto rx=0x7f5fe00ac040 tx=0x7f5fe001ad10 comp rx=0 tx=0).stop 2023-11-26T07:22:00.515 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:00.508+0000 7f5fec10f640 1 -- 172.21.15.143:0/1312618963 shutdown_connections 2023-11-26T07:22:00.515 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:00.508+0000 7f5fec10f640 1 --2- 172.21.15.143:0/1312618963 >> [v2:172.21.15.143:6832/2150557544,v1:172.21.15.143:6833/2150557544] conn(0x7f5fc8066650 0x7f5fc8068b10 unknown :-1 s=CLOSED pgs=384 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:22:00.515 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:00.508+0000 7f5fec10f640 1 --2- 172.21.15.143:0/1312618963 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f5fe414b1e0 0x7f5fe4141490 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:22:00.515 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:00.508+0000 7f5fec10f640 1 --2- 172.21.15.143:0/1312618963 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f5fe40cf7c0 0x7f5fe4140f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:22:00.515 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:00.508+0000 7f5fec10f640 1 --2- 172.21.15.143:0/1312618963 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f5fe4070fd0 0x7f5fe4160810 unknown :-1 s=CLOSED pgs=802 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:22:00.515 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:00.508+0000 7f5fec10f640 1 -- 172.21.15.143:0/1312618963 >> 172.21.15.143:0/1312618963 conn(0x7f5fe406a890 msgr2=0x7f5fe40b1110 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:22:00.515 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:00.512+0000 7f5fec10f640 1 -- 172.21.15.143:0/1312618963 shutdown_connections 2023-11-26T07:22:00.515 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:00.512+0000 7f5fec10f640 1 -- 172.21.15.143:0/1312618963 wait complete. 2023-11-26T07:22:00.531 INFO:teuthology.orchestra.run.smithi143.stdout:{"pg_ready":true,"pg_map":{"version":181,"stamp":"2023-11-26T07:21:59.748730+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592778,"num_objects":23,"num_object_clones":0,"num_object_copies":46,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":23,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":203,"num_write_kb":2430,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":155,"ondisk_log_size":155,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":68,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":231100,"kb_used_data":6596,"kb_used_omap":12,"kb_used_meta":224371,"kb_avail":754743620,"statfs":{"total":773094113280,"available":772857466880,"internally_reserved":0,"allocated":6754304,"data_stored":3478041,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12749,"internal_metadata":229756467},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":41,"apply_latency_ms":41,"commit_latency_ns":41000000,"apply_latency_ns":41000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2410,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"5.484128"},"pg_stats":[{"pgid":"9.3f","version":"0'0","reported_seq":9,"reported_epoch":58,"state":"active+clean","last_fresh":"2023-11-26T07:21:56.300698+0000","last_change":"2023-11-26T07:21:56.291621+0000","last_active":"2023-11-26T07:21:56.300698+0000","last_peered":"2023-11-26T07:21:56.300698+0000","last_clean":"2023-11-26T07:21:56.300698+0000","last_became_active":"2023-11-26T07:21:56.291367+0000","last_became_peered":"2023-11-26T07:21:56.291367+0000","last_unstale":"2023-11-26T07:21:56.300698+0000","last_undegraded":"2023-11-26T07:21:56.300698+0000","last_fullsized":"2023-11-26T07:21:56.300698+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:21:55.264078+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:21:55.264078+0000","last_clean_scrub_stamp":"2023-11-26T07:21:55.264078+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T10:12:09.752171+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.3e","version":"0'0","reported_seq":9,"reported_epoch":58,"state":"active+clean","last_fresh":"2023-11-26T07:21:56.304718+0000","last_change":"2023-11-26T07:21:56.299056+0000","last_active":"2023-11-26T07:21:56.304718+0000","last_peered":"2023-11-26T07:21:56.304718+0000","last_clean":"2023-11-26T07:21:56.304718+0000","last_became_active":"2023-11-26T07:21:56.297285+0000","last_became_peered":"2023-11-26T07:21:56.297285+0000","last_unstale":"2023-11-26T07:21:56.304718+0000","last_undegraded":"2023-11-26T07:21:56.304718+0000","last_fullsized":"2023-11-26T07:21:56.304718+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:21:55.264078+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:21:55.264078+0000","last_clean_scrub_stamp":"2023-11-26T07:21:55.264078+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T09:40:09.904648+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.3d","version":"0'0","reported_seq":9,"reported_epoch":58,"state":"active+clean","last_fresh":"2023-11-26T07:21:56.305388+0000","last_change":"2023-11-26T07:21:56.296573+0000","last_active":"2023-11-26T07:21:56.305388+0000","last_peered":"2023-11-26T07:21:56.305388+0000","last_clean":"2023-11-26T07:21:56.305388+0000","last_became_active":"2023-11-26T07:21:56.296280+0000","last_became_peered":"2023-11-26T07:21:56.296280+0000","last_unstale":"2023-11-26T07:21:56.305388+0000","last_undegraded":"2023-11-26T07:21:56.305388+0000","last_fullsized":"2023-11-26T07:21:56.305388+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:21:55.264078+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:21:55.264078+0000","last_clean_scrub_stamp":"2023-11-26T07:21:55.264078+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T09:35:29.250849+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.3c","version":"0'0","reported_seq":9,"reported_epoch":58,"state":"active+clean","last_fresh":"2023-11-26T07:21:56.312843+0000","last_change":"2023-11-26T07:21:56.306604+0000","last_active":"2023-11-26T07:21:56.312843+0000","last_peered":"2023-11-26T07:21:56.312843+0000","last_clean":"2023-11-26T07:21:56.312843+0000","last_became_active":"2023-11-26T07:21:56.306030+0000","last_became_peered":"2023-11-26T07:21:56.306030+0000","last_unstale":"2023-11-26T07:21:56.312843+0000","last_undegraded":"2023-11-26T07:21:56.312843+0000","last_fullsized":"2023-11-26T07:21:56.312843+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:21:55.264078+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:21:55.264078+0000","last_clean_scrub_stamp":"2023-11-26T07:21:55.264078+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T07:38:48.885316+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.3b","version":"0'0","reported_seq":9,"reported_epoch":58,"state":"active+clean","last_fresh":"2023-11-26T07:21:56.302339+0000","last_change":"2023-11-26T07:21:56.298880+0000","last_active":"2023-11-26T07:21:56.302339+0000","last_peered":"2023-11-26T07:21:56.302339+0000","last_clean":"2023-11-26T07:21:56.302339+0000","last_became_active":"2023-11-26T07:21:56.298620+0000","last_became_peered":"2023-11-26T07:21:56.298620+0000","last_unstale":"2023-11-26T07:21:56.302339+0000","last_undegraded":"2023-11-26T07:21:56.302339+0000","last_fullsized":"2023-11-26T07:21:56.302339+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:21:55.264078+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:21:55.264078+0000","last_clean_scrub_stamp":"2023-11-26T07:21:55.264078+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T10:17:32.686931+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.3a","version":"0'0","reported_seq":9,"reported_epoch":58,"state":"active+clean","last_fresh":"2023-11-26T07:21:56.304452+0000","last_change":"2023-11-26T07:21:56.300801+0000","last_active":"2023-11-26T07:21:56.304452+0000","last_peered":"2023-11-26T07:21:56.304452+0000","last_clean":"2023-11-26T07:21:56.304452+0000","last_became_active":"2023-11-26T07:21:56.300531+0000","last_became_peered":"2023-11-26T07:21:56.300531+0000","last_unstale":"2023-11-26T07:21:56.304452+0000","last_undegraded":"2023-11-26T07:21:56.304452+0000","last_fullsized":"2023-11-26T07:21:56.304452+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:21:55.264078+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:21:55.264078+0000","last_clean_scrub_stamp":"2023-11-26T07:21:55.264078+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T12:17:05.630587+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.39","version":"0'0","reported_seq":9,"reported_epoch":58,"state":"active+clean","last_fresh":"2023-11-26T07:21:56.313592+0000","last_change":"2023-11-26T07:21:56.311679+0000","last_active":"2023-11-26T07:21:56.313592+0000","last_peered":"2023-11-26T07:21:56.313592+0000","last_clean":"2023-11-26T07:21:56.313592+0000","last_became_active":"2023-11-26T07:21:56.311496+0000","last_became_peered":"2023-11-26T07:21:56.311496+0000","last_unstale":"2023-11-26T07:21:56.313592+0000","last_undegraded":"2023-11-26T07:21:56.313592+0000","last_fullsized":"2023-11-26T07:21:56.313592+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:21:55.264078+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:21:55.264078+0000","last_clean_scrub_stamp":"2023-11-26T07:21:55.264078+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T17:06:42.726725+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.38","version":"0'0","reported_seq":9,"reported_epoch":58,"state":"active+clean","last_fresh":"2023-11-26T07:21:56.302704+0000","last_change":"2023-11-26T07:21:56.297195+0000","last_active":"2023-11-26T07:21:56.302704+0000","last_peered":"2023-11-26T07:21:56.302704+0000","last_clean":"2023-11-26T07:21:56.302704+0000","last_became_active":"2023-11-26T07:21:56.296928+0000","last_became_peered":"2023-11-26T07:21:56.296928+0000","last_unstale":"2023-11-26T07:21:56.302704+0000","last_undegraded":"2023-11-26T07:21:56.302704+0000","last_fullsized":"2023-11-26T07:21:56.302704+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:21:55.264078+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:21:55.264078+0000","last_clean_scrub_stamp":"2023-11-26T07:21:55.264078+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T15:11:39.329445+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.37","version":"0'0","reported_seq":9,"reported_epoch":58,"state":"active+clean","last_fresh":"2023-11-26T07:21:56.307129+0000","last_change":"2023-11-26T07:21:56.297210+0000","last_active":"2023-11-26T07:21:56.307129+0000","last_peered":"2023-11-26T07:21:56.307129+0000","last_clean":"2023-11-26T07:21:56.307129+0000","last_became_active":"2023-11-26T07:21:56.296897+0000","last_became_peered":"2023-11-26T07:21:56.296897+0000","last_unstale":"2023-11-26T07:21:56.307129+0000","last_undegraded":"2023-11-26T07:21:56.307129+0000","last_fullsized":"2023-11-26T07:21:56.307129+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:21:55.264078+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:21:55.264078+0000","last_clean_scrub_stamp":"2023-11-26T07:21:55.264078+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T17:56:48.875217+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.36","version":"0'0","reported_seq":9,"reported_epoch":58,"state":"active+clean","last_fresh":"2023-11-26T07:21:56.312516+0000","last_change":"2023-11-26T07:21:56.304514+0000","last_active":"2023-11-26T07:21:56.312516+0000","last_peered":"2023-11-26T07:21:56.312516+0000","last_clean":"2023-11-26T07:21:56.312516+0000","last_became_active":"2023-11-26T07:21:56.303710+0000","last_became_peered":"2023-11-26T07:21:56.303710+0000","last_unstale":"2023-11-26T07:21:56.312516+0000","last_undegraded":"2023-11-26T07:21:56.312516+0000","last_fullsized":"2023-11-26T07:21:56.312516+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:21:55.264078+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:21:55.264078+0000","last_clean_scrub_stamp":"2023-11-26T07:21:55.264078+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T17:00:33.710521+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.35","version":"0'0","reported_seq":9,"reported_epoch":58,"state":"active+clean","last_fresh":"2023-11-26T07:21:56.305120+0000","last_change":"2023-11-26T07:21:56.301384+0000","last_active":"2023-11-26T07:21:56.305120+0000","last_peered":"2023-11-26T07:21:56.305120+0000","last_clean":"2023-11-26T07:21:56.305120+0000","last_became_active":"2023-11-26T07:21:56.301085+0000","last_became_peered":"2023-11-26T07:21:56.301085+0000","last_unstale":"2023-11-26T07:21:56.305120+0000","last_undegraded":"2023-11-26T07:21:56.305120+0000","last_fullsized":"2023-11-26T07:21:56.305120+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:21:55.264078+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:21:55.264078+0000","last_clean_scrub_stamp":"2023-11-26T07:21:55.264078+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T12:10:56.964250+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.34","version":"0'0","reported_seq":9,"reported_epoch":58,"state":"active+clean","last_fresh":"2023-11-26T07:21:56.312862+0000","last_change":"2023-11-26T07:21:56.306724+0000","last_active":"2023-11-26T07:21:56.312862+0000","last_peered":"2023-11-26T07:21:56.312862+0000","last_clean":"2023-11-26T07:21:56.312862+0000","last_became_active":"2023-11-26T07:21:56.306369+0000","last_became_peered":"2023-11-26T07:21:56.306369+0000","last_unstale":"2023-11-26T07:21:56.312862+0000","last_undegraded":"2023-11-26T07:21:56.312862+0000","last_fullsized":"2023-11-26T07:21:56.312862+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:21:55.264078+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:21:55.264078+0000","last_clean_scrub_stamp":"2023-11-26T07:21:55.264078+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T15:20:41.424795+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.33","version":"0'0","reported_seq":9,"reported_epoch":58,"state":"active+clean","last_fresh":"2023-11-26T07:21:56.310948+0000","last_change":"2023-11-26T07:21:56.301597+0000","last_active":"2023-11-26T07:21:56.310948+0000","last_peered":"2023-11-26T07:21:56.310948+0000","last_clean":"2023-11-26T07:21:56.310948+0000","last_became_active":"2023-11-26T07:21:56.301265+0000","last_became_peered":"2023-11-26T07:21:56.301265+0000","last_unstale":"2023-11-26T07:21:56.310948+0000","last_undegraded":"2023-11-26T07:21:56.310948+0000","last_fullsized":"2023-11-26T07:21:56.310948+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:21:55.264078+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:21:55.264078+0000","last_clean_scrub_stamp":"2023-11-26T07:21:55.264078+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T07:44:25.322195+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.32","version":"0'0","reported_seq":9,"reported_epoch":58,"state":"active+clean","last_fresh":"2023-11-26T07:21:56.307644+0000","last_change":"2023-11-26T07:21:56.298573+0000","last_active":"2023-11-26T07:21:56.307644+0000","last_peered":"2023-11-26T07:21:56.307644+0000","last_clean":"2023-11-26T07:21:56.307644+0000","last_became_active":"2023-11-26T07:21:56.298282+0000","last_became_peered":"2023-11-26T07:21:56.298282+0000","last_unstale":"2023-11-26T07:21:56.307644+0000","last_undegraded":"2023-11-26T07:21:56.307644+0000","last_fullsized":"2023-11-26T07:21:56.307644+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:21:55.264078+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:21:55.264078+0000","last_clean_scrub_stamp":"2023-11-26T07:21:55.264078+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T10:27:44.765406+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.31","version":"0'0","reported_seq":9,"reported_epoch":58,"state":"active+clean","last_fresh":"2023-11-26T07:21:56.307734+0000","last_change":"2023-11-26T07:21:56.299019+0000","last_active":"2023-11-26T07:21:56.307734+0000","last_peered":"2023-11-26T07:21:56.307734+0000","last_clean":"2023-11-26T07:21:56.307734+0000","last_became_active":"2023-11-26T07:21:56.298693+0000","last_became_peered":"2023-11-26T07:21:56.298693+0000","last_unstale":"2023-11-26T07:21:56.307734+0000","last_undegraded":"2023-11-26T07:21:56.307734+0000","last_fullsized":"2023-11-26T07:21:56.307734+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:21:55.264078+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:21:55.264078+0000","last_clean_scrub_stamp":"2023-11-26T07:21:55.264078+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T12:10:56.964250+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.30","version":"0'0","reported_seq":9,"reported_epoch":58,"state":"active+clean","last_fresh":"2023-11-26T07:21:56.301573+0000","last_change":"2023-11-26T07:21:56.292573+0000","last_active":"2023-11-26T07:21:56.301573+0000","last_peered":"2023-11-26T07:21:56.301573+0000","last_clean":"2023-11-26T07:21:56.301573+0000","last_became_active":"2023-11-26T07:21:56.291812+0000","last_became_peered":"2023-11-26T07:21:56.291812+0000","last_unstale":"2023-11-26T07:21:56.301573+0000","last_undegraded":"2023-11-26T07:21:56.301573+0000","last_fullsized":"2023-11-26T07:21:56.301573+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:21:55.264078+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:21:55.264078+0000","last_clean_scrub_stamp":"2023-11-26T07:21:55.264078+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T08:06:02.760482+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.2f","version":"0'0","reported_seq":9,"reported_epoch":58,"state":"active+clean","last_fresh":"2023-11-26T07:21:56.302731+0000","last_change":"2023-11-26T07:21:56.296668+0000","last_active":"2023-11-26T07:21:56.302731+0000","last_peered":"2023-11-26T07:21:56.302731+0000","last_clean":"2023-11-26T07:21:56.302731+0000","last_became_active":"2023-11-26T07:21:56.296332+0000","last_became_peered":"2023-11-26T07:21:56.296332+0000","last_unstale":"2023-11-26T07:21:56.302731+0000","last_undegraded":"2023-11-26T07:21:56.302731+0000","last_fullsized":"2023-11-26T07:21:56.302731+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:21:55.264078+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:21:55.264078+0000","last_clean_scrub_stamp":"2023-11-26T07:21:55.264078+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T19:16:16.815018+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.2e","version":"0'0","reported_seq":9,"reported_epoch":58,"state":"active+clean","last_fresh":"2023-11-26T07:21:56.309667+0000","last_change":"2023-11-26T07:21:56.301067+0000","last_active":"2023-11-26T07:21:56.309667+0000","last_peered":"2023-11-26T07:21:56.309667+0000","last_clean":"2023-11-26T07:21:56.309667+0000","last_became_active":"2023-11-26T07:21:56.300756+0000","last_became_peered":"2023-11-26T07:21:56.300756+0000","last_unstale":"2023-11-26T07:21:56.309667+0000","last_undegraded":"2023-11-26T07:21:56.309667+0000","last_fullsized":"2023-11-26T07:21:56.309667+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:21:55.264078+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:21:55.264078+0000","last_clean_scrub_stamp":"2023-11-26T07:21:55.264078+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T12:18:44.648658+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.2d","version":"0'0","reported_seq":9,"reported_epoch":58,"state":"active+clean","last_fresh":"2023-11-26T07:21:56.297345+0000","last_change":"2023-11-26T07:21:56.290886+0000","last_active":"2023-11-26T07:21:56.297345+0000","last_peered":"2023-11-26T07:21:56.297345+0000","last_clean":"2023-11-26T07:21:56.297345+0000","last_became_active":"2023-11-26T07:21:56.290612+0000","last_became_peered":"2023-11-26T07:21:56.290612+0000","last_unstale":"2023-11-26T07:21:56.297345+0000","last_undegraded":"2023-11-26T07:21:56.297345+0000","last_fullsized":"2023-11-26T07:21:56.297345+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:21:55.264078+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:21:55.264078+0000","last_clean_scrub_stamp":"2023-11-26T07:21:55.264078+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T14:28:40.362241+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.2c","version":"0'0","reported_seq":9,"reported_epoch":58,"state":"active+clean","last_fresh":"2023-11-26T07:21:56.303054+0000","last_change":"2023-11-26T07:21:56.298126+0000","last_active":"2023-11-26T07:21:56.303054+0000","last_peered":"2023-11-26T07:21:56.303054+0000","last_clean":"2023-11-26T07:21:56.303054+0000","last_became_active":"2023-11-26T07:21:56.297867+0000","last_became_peered":"2023-11-26T07:21:56.297867+0000","last_unstale":"2023-11-26T07:21:56.303054+0000","last_undegraded":"2023-11-26T07:21:56.303054+0000","last_fullsized":"2023-11-26T07:21:56.303054+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:21:55.264078+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:21:55.264078+0000","last_clean_scrub_stamp":"2023-11-26T07:21:55.264078+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T14:55:04.672096+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.2b","version":"0'0","reported_seq":9,"reported_epoch":58,"state":"active+clean","last_fresh":"2023-11-26T07:21:56.314071+0000","last_change":"2023-11-26T07:21:56.309268+0000","last_active":"2023-11-26T07:21:56.314071+0000","last_peered":"2023-11-26T07:21:56.314071+0000","last_clean":"2023-11-26T07:21:56.314071+0000","last_became_active":"2023-11-26T07:21:56.308973+0000","last_became_peered":"2023-11-26T07:21:56.308973+0000","last_unstale":"2023-11-26T07:21:56.314071+0000","last_undegraded":"2023-11-26T07:21:56.314071+0000","last_fullsized":"2023-11-26T07:21:56.314071+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:21:55.264078+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:21:55.264078+0000","last_clean_scrub_stamp":"2023-11-26T07:21:55.264078+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T13:30:25.417889+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.2a","version":"0'0","reported_seq":9,"reported_epoch":58,"state":"active+clean","last_fresh":"2023-11-26T07:21:56.299203+0000","last_change":"2023-11-26T07:21:56.293241+0000","last_active":"2023-11-26T07:21:56.299203+0000","last_peered":"2023-11-26T07:21:56.299203+0000","last_clean":"2023-11-26T07:21:56.299203+0000","last_became_active":"2023-11-26T07:21:56.292578+0000","last_became_peered":"2023-11-26T07:21:56.292578+0000","last_unstale":"2023-11-26T07:21:56.299203+0000","last_undegraded":"2023-11-26T07:21:56.299203+0000","last_fullsized":"2023-11-26T07:21:56.299203+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:21:55.264078+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:21:55.264078+0000","last_clean_scrub_stamp":"2023-11-26T07:21:55.264078+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T19:05:58.460154+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.29","version":"0'0","reported_seq":9,"reported_epoch":58,"state":"active+clean","last_fresh":"2023-11-26T07:21:56.309723+0000","last_change":"2023-11-26T07:21:56.303654+0000","last_active":"2023-11-26T07:21:56.309723+0000","last_peered":"2023-11-26T07:21:56.309723+0000","last_clean":"2023-11-26T07:21:56.309723+0000","last_became_active":"2023-11-26T07:21:56.300036+0000","last_became_peered":"2023-11-26T07:21:56.300036+0000","last_unstale":"2023-11-26T07:21:56.309723+0000","last_undegraded":"2023-11-26T07:21:56.309723+0000","last_fullsized":"2023-11-26T07:21:56.309723+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:21:55.264078+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:21:55.264078+0000","last_clean_scrub_stamp":"2023-11-26T07:21:55.264078+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T13:30:24.283597+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.28","version":"0'0","reported_seq":9,"reported_epoch":58,"state":"active+clean","last_fresh":"2023-11-26T07:21:56.309060+0000","last_change":"2023-11-26T07:21:56.292011+0000","last_active":"2023-11-26T07:21:56.309060+0000","last_peered":"2023-11-26T07:21:56.309060+0000","last_clean":"2023-11-26T07:21:56.309060+0000","last_became_active":"2023-11-26T07:21:56.291728+0000","last_became_peered":"2023-11-26T07:21:56.291728+0000","last_unstale":"2023-11-26T07:21:56.309060+0000","last_undegraded":"2023-11-26T07:21:56.309060+0000","last_fullsized":"2023-11-26T07:21:56.309060+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:21:55.264078+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:21:55.264078+0000","last_clean_scrub_stamp":"2023-11-26T07:21:55.264078+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T12:41:20.887450+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.27","version":"0'0","reported_seq":9,"reported_epoch":58,"state":"active+clean","last_fresh":"2023-11-26T07:21:56.300715+0000","last_change":"2023-11-26T07:21:56.296395+0000","last_active":"2023-11-26T07:21:56.300715+0000","last_peered":"2023-11-26T07:21:56.300715+0000","last_clean":"2023-11-26T07:21:56.300715+0000","last_became_active":"2023-11-26T07:21:56.296077+0000","last_became_peered":"2023-11-26T07:21:56.296077+0000","last_unstale":"2023-11-26T07:21:56.300715+0000","last_undegraded":"2023-11-26T07:21:56.300715+0000","last_fullsized":"2023-11-26T07:21:56.300715+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:21:55.264078+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:21:55.264078+0000","last_clean_scrub_stamp":"2023-11-26T07:21:55.264078+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T13:59:10.978496+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.26","version":"0'0","reported_seq":9,"reported_epoch":58,"state":"active+clean","last_fresh":"2023-11-26T07:21:56.310283+0000","last_change":"2023-11-26T07:21:56.297590+0000","last_active":"2023-11-26T07:21:56.310283+0000","last_peered":"2023-11-26T07:21:56.310283+0000","last_clean":"2023-11-26T07:21:56.310283+0000","last_became_active":"2023-11-26T07:21:56.296279+0000","last_became_peered":"2023-11-26T07:21:56.296279+0000","last_unstale":"2023-11-26T07:21:56.310283+0000","last_undegraded":"2023-11-26T07:21:56.310283+0000","last_fullsized":"2023-11-26T07:21:56.310283+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:21:55.264078+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:21:55.264078+0000","last_clean_scrub_stamp":"2023-11-26T07:21:55.264078+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T19:19:55.894699+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.25","version":"0'0","reported_seq":9,"reported_epoch":58,"state":"active+clean","last_fresh":"2023-11-26T07:21:56.304371+0000","last_change":"2023-11-26T07:21:56.301048+0000","last_active":"2023-11-26T07:21:56.304371+0000","last_peered":"2023-11-26T07:21:56.304371+0000","last_clean":"2023-11-26T07:21:56.304371+0000","last_became_active":"2023-11-26T07:21:56.299240+0000","last_became_peered":"2023-11-26T07:21:56.299240+0000","last_unstale":"2023-11-26T07:21:56.304371+0000","last_undegraded":"2023-11-26T07:21:56.304371+0000","last_fullsized":"2023-11-26T07:21:56.304371+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:21:55.264078+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:21:55.264078+0000","last_clean_scrub_stamp":"2023-11-26T07:21:55.264078+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T10:22:37.629253+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.24","version":"0'0","reported_seq":9,"reported_epoch":58,"state":"active+clean","last_fresh":"2023-11-26T07:21:56.302367+0000","last_change":"2023-11-26T07:21:56.298575+0000","last_active":"2023-11-26T07:21:56.302367+0000","last_peered":"2023-11-26T07:21:56.302367+0000","last_clean":"2023-11-26T07:21:56.302367+0000","last_became_active":"2023-11-26T07:21:56.298312+0000","last_became_peered":"2023-11-26T07:21:56.298312+0000","last_unstale":"2023-11-26T07:21:56.302367+0000","last_undegraded":"2023-11-26T07:21:56.302367+0000","last_fullsized":"2023-11-26T07:21:56.302367+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:21:55.264078+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:21:55.264078+0000","last_clean_scrub_stamp":"2023-11-26T07:21:55.264078+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T17:40:12.547770+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.23","version":"0'0","reported_seq":9,"reported_epoch":58,"state":"active+clean","last_fresh":"2023-11-26T07:21:56.315097+0000","last_change":"2023-11-26T07:21:56.312013+0000","last_active":"2023-11-26T07:21:56.315097+0000","last_peered":"2023-11-26T07:21:56.315097+0000","last_clean":"2023-11-26T07:21:56.315097+0000","last_became_active":"2023-11-26T07:21:56.311699+0000","last_became_peered":"2023-11-26T07:21:56.311699+0000","last_unstale":"2023-11-26T07:21:56.315097+0000","last_undegraded":"2023-11-26T07:21:56.315097+0000","last_fullsized":"2023-11-26T07:21:56.315097+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:21:55.264078+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:21:55.264078+0000","last_clean_scrub_stamp":"2023-11-26T07:21:55.264078+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T12:18:44.648658+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.22","version":"0'0","reported_seq":9,"reported_epoch":58,"state":"active+clean","last_fresh":"2023-11-26T07:21:56.309681+0000","last_change":"2023-11-26T07:21:56.298035+0000","last_active":"2023-11-26T07:21:56.309681+0000","last_peered":"2023-11-26T07:21:56.309681+0000","last_clean":"2023-11-26T07:21:56.309681+0000","last_became_active":"2023-11-26T07:21:56.297613+0000","last_became_peered":"2023-11-26T07:21:56.297613+0000","last_unstale":"2023-11-26T07:21:56.309681+0000","last_undegraded":"2023-11-26T07:21:56.309681+0000","last_fullsized":"2023-11-26T07:21:56.309681+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:21:55.264078+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:21:55.264078+0000","last_clean_scrub_stamp":"2023-11-26T07:21:55.264078+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T10:32:38.427322+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.21","version":"0'0","reported_seq":9,"reported_epoch":58,"state":"active+clean","last_fresh":"2023-11-26T07:21:56.292934+0000","last_change":"2023-11-26T07:21:56.289763+0000","last_active":"2023-11-26T07:21:56.292934+0000","last_peered":"2023-11-26T07:21:56.292934+0000","last_clean":"2023-11-26T07:21:56.292934+0000","last_became_active":"2023-11-26T07:21:56.289468+0000","last_became_peered":"2023-11-26T07:21:56.289468+0000","last_unstale":"2023-11-26T07:21:56.292934+0000","last_undegraded":"2023-11-26T07:21:56.292934+0000","last_fullsized":"2023-11-26T07:21:56.292934+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:21:55.264078+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:21:55.264078+0000","last_clean_scrub_stamp":"2023-11-26T07:21:55.264078+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T16:42:45.532870+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.20","version":"0'0","reported_seq":9,"reported_epoch":58,"state":"active+clean","last_fresh":"2023-11-26T07:21:56.301553+0000","last_change":"2023-11-26T07:21:56.291707+0000","last_active":"2023-11-26T07:21:56.301553+0000","last_peered":"2023-11-26T07:21:56.301553+0000","last_clean":"2023-11-26T07:21:56.301553+0000","last_became_active":"2023-11-26T07:21:56.291419+0000","last_became_peered":"2023-11-26T07:21:56.291419+0000","last_unstale":"2023-11-26T07:21:56.301553+0000","last_undegraded":"2023-11-26T07:21:56.301553+0000","last_fullsized":"2023-11-26T07:21:56.301553+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:21:55.264078+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:21:55.264078+0000","last_clean_scrub_stamp":"2023-11-26T07:21:55.264078+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T09:35:29.250849+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.1e","version":"0'0","reported_seq":9,"reported_epoch":58,"state":"active+clean","last_fresh":"2023-11-26T07:21:56.310315+0000","last_change":"2023-11-26T07:21:56.302129+0000","last_active":"2023-11-26T07:21:56.310315+0000","last_peered":"2023-11-26T07:21:56.310315+0000","last_clean":"2023-11-26T07:21:56.310315+0000","last_became_active":"2023-11-26T07:21:56.301845+0000","last_became_peered":"2023-11-26T07:21:56.301845+0000","last_unstale":"2023-11-26T07:21:56.310315+0000","last_undegraded":"2023-11-26T07:21:56.310315+0000","last_fullsized":"2023-11-26T07:21:56.310315+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:21:55.264078+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:21:55.264078+0000","last_clean_scrub_stamp":"2023-11-26T07:21:55.264078+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T11:54:47.454823+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.1f","version":"58'4","reported_seq":15,"reported_epoch":58,"state":"active+clean","last_fresh":"2023-11-26T07:21:56.319313+0000","last_change":"2023-11-26T07:21:55.298427+0000","last_active":"2023-11-26T07:21:56.319313+0000","last_peered":"2023-11-26T07:21:56.319313+0000","last_clean":"2023-11-26T07:21:56.319313+0000","last_became_active":"2023-11-26T07:21:55.298125+0000","last_became_peered":"2023-11-26T07:21:55.298125+0000","last_unstale":"2023-11-26T07:21:56.319313+0000","last_undegraded":"2023-11-26T07:21:56.319313+0000","last_fullsized":"2023-11-26T07:21:56.319313+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:21:54.255568+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:21:54.255568+0000","last_clean_scrub_stamp":"2023-11-26T07:21:54.255568+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T13:30:23.275087+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.1f","version":"0'0","reported_seq":9,"reported_epoch":58,"state":"active+clean","last_fresh":"2023-11-26T07:21:56.310682+0000","last_change":"2023-11-26T07:21:56.304018+0000","last_active":"2023-11-26T07:21:56.310682+0000","last_peered":"2023-11-26T07:21:56.310682+0000","last_clean":"2023-11-26T07:21:56.310682+0000","last_became_active":"2023-11-26T07:21:56.303695+0000","last_became_peered":"2023-11-26T07:21:56.303695+0000","last_unstale":"2023-11-26T07:21:56.310682+0000","last_undegraded":"2023-11-26T07:21:56.310682+0000","last_fullsized":"2023-11-26T07:21:56.310682+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:21:55.264078+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:21:55.264078+0000","last_clean_scrub_stamp":"2023-11-26T07:21:55.264078+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T17:06:42.726725+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.1e","version":"58'1","reported_seq":12,"reported_epoch":58,"state":"active+clean","last_fresh":"2023-11-26T07:21:56.314736+0000","last_change":"2023-11-26T07:21:55.299423+0000","last_active":"2023-11-26T07:21:56.314736+0000","last_peered":"2023-11-26T07:21:56.314736+0000","last_clean":"2023-11-26T07:21:56.314736+0000","last_became_active":"2023-11-26T07:21:55.299124+0000","last_became_peered":"2023-11-26T07:21:55.299124+0000","last_unstale":"2023-11-26T07:21:56.314736+0000","last_undegraded":"2023-11-26T07:21:56.314736+0000","last_fullsized":"2023-11-26T07:21:56.314736+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:21:54.255568+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:21:54.255568+0000","last_clean_scrub_stamp":"2023-11-26T07:21:54.255568+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T16:56:28.898546+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.1c","version":"0'0","reported_seq":9,"reported_epoch":58,"state":"active+clean","last_fresh":"2023-11-26T07:21:56.301667+0000","last_change":"2023-11-26T07:21:56.294816+0000","last_active":"2023-11-26T07:21:56.301667+0000","last_peered":"2023-11-26T07:21:56.301667+0000","last_clean":"2023-11-26T07:21:56.301667+0000","last_became_active":"2023-11-26T07:21:56.294549+0000","last_became_peered":"2023-11-26T07:21:56.294549+0000","last_unstale":"2023-11-26T07:21:56.301667+0000","last_undegraded":"2023-11-26T07:21:56.301667+0000","last_fullsized":"2023-11-26T07:21:56.301667+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:21:55.264078+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:21:55.264078+0000","last_clean_scrub_stamp":"2023-11-26T07:21:55.264078+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T13:30:25.417889+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.1d","version":"58'2","reported_seq":13,"reported_epoch":58,"state":"active+clean","last_fresh":"2023-11-26T07:21:56.316221+0000","last_change":"2023-11-26T07:21:55.296693+0000","last_active":"2023-11-26T07:21:56.316221+0000","last_peered":"2023-11-26T07:21:56.316221+0000","last_clean":"2023-11-26T07:21:56.316221+0000","last_became_active":"2023-11-26T07:21:55.296389+0000","last_became_peered":"2023-11-26T07:21:55.296389+0000","last_unstale":"2023-11-26T07:21:56.316221+0000","last_undegraded":"2023-11-26T07:21:56.316221+0000","last_fullsized":"2023-11-26T07:21:56.316221+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:21:54.255568+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:21:54.255568+0000","last_clean_scrub_stamp":"2023-11-26T07:21:54.255568+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T11:09:01.721510+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.1d","version":"0'0","reported_seq":9,"reported_epoch":58,"state":"active+clean","last_fresh":"2023-11-26T07:21:56.304263+0000","last_change":"2023-11-26T07:21:56.299896+0000","last_active":"2023-11-26T07:21:56.304263+0000","last_peered":"2023-11-26T07:21:56.304263+0000","last_clean":"2023-11-26T07:21:56.304263+0000","last_became_active":"2023-11-26T07:21:56.299617+0000","last_became_peered":"2023-11-26T07:21:56.299617+0000","last_unstale":"2023-11-26T07:21:56.304263+0000","last_undegraded":"2023-11-26T07:21:56.304263+0000","last_fullsized":"2023-11-26T07:21:56.304263+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:21:55.264078+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:21:55.264078+0000","last_clean_scrub_stamp":"2023-11-26T07:21:55.264078+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T08:10:58.660224+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.1c","version":"58'2","reported_seq":13,"reported_epoch":58,"state":"active+clean","last_fresh":"2023-11-26T07:21:56.318020+0000","last_change":"2023-11-26T07:21:55.295656+0000","last_active":"2023-11-26T07:21:56.318020+0000","last_peered":"2023-11-26T07:21:56.318020+0000","last_clean":"2023-11-26T07:21:56.318020+0000","last_became_active":"2023-11-26T07:21:55.294764+0000","last_became_peered":"2023-11-26T07:21:55.294764+0000","last_unstale":"2023-11-26T07:21:56.318020+0000","last_undegraded":"2023-11-26T07:21:56.318020+0000","last_fullsized":"2023-11-26T07:21:56.318020+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:21:54.255568+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:21:54.255568+0000","last_clean_scrub_stamp":"2023-11-26T07:21:54.255568+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T09:35:28.242339+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.1a","version":"0'0","reported_seq":9,"reported_epoch":58,"state":"active+clean","last_fresh":"2023-11-26T07:21:56.299184+0000","last_change":"2023-11-26T07:21:56.293138+0000","last_active":"2023-11-26T07:21:56.299184+0000","last_peered":"2023-11-26T07:21:56.299184+0000","last_clean":"2023-11-26T07:21:56.299184+0000","last_became_active":"2023-11-26T07:21:56.292236+0000","last_became_peered":"2023-11-26T07:21:56.292236+0000","last_unstale":"2023-11-26T07:21:56.299184+0000","last_undegraded":"2023-11-26T07:21:56.299184+0000","last_fullsized":"2023-11-26T07:21:56.299184+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:21:55.264078+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:21:55.264078+0000","last_clean_scrub_stamp":"2023-11-26T07:21:55.264078+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T10:56:18.452471+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.1b","version":"58'1","reported_seq":12,"reported_epoch":58,"state":"active+clean","last_fresh":"2023-11-26T07:21:56.315453+0000","last_change":"2023-11-26T07:21:55.286486+0000","last_active":"2023-11-26T07:21:56.315453+0000","last_peered":"2023-11-26T07:21:56.315453+0000","last_clean":"2023-11-26T07:21:56.315453+0000","last_became_active":"2023-11-26T07:21:55.286095+0000","last_became_peered":"2023-11-26T07:21:55.286095+0000","last_unstale":"2023-11-26T07:21:56.315453+0000","last_undegraded":"2023-11-26T07:21:56.315453+0000","last_fullsized":"2023-11-26T07:21:56.315453+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:21:54.255568+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:21:54.255568+0000","last_clean_scrub_stamp":"2023-11-26T07:21:54.255568+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T12:13:26.816359+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.1b","version":"0'0","reported_seq":9,"reported_epoch":58,"state":"active+clean","last_fresh":"2023-11-26T07:21:56.304741+0000","last_change":"2023-11-26T07:21:56.302073+0000","last_active":"2023-11-26T07:21:56.304741+0000","last_peered":"2023-11-26T07:21:56.304741+0000","last_clean":"2023-11-26T07:21:56.304741+0000","last_became_active":"2023-11-26T07:21:56.301822+0000","last_became_peered":"2023-11-26T07:21:56.301822+0000","last_unstale":"2023-11-26T07:21:56.304741+0000","last_undegraded":"2023-11-26T07:21:56.304741+0000","last_fullsized":"2023-11-26T07:21:56.304741+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:21:55.264078+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:21:55.264078+0000","last_clean_scrub_stamp":"2023-11-26T07:21:55.264078+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T10:03:32.111285+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.1a","version":"58'1","reported_seq":12,"reported_epoch":58,"state":"active+clean","last_fresh":"2023-11-26T07:21:56.313461+0000","last_change":"2023-11-26T07:21:55.292401+0000","last_active":"2023-11-26T07:21:56.313461+0000","last_peered":"2023-11-26T07:21:56.313461+0000","last_clean":"2023-11-26T07:21:56.313461+0000","last_became_active":"2023-11-26T07:21:55.292081+0000","last_became_peered":"2023-11-26T07:21:55.292081+0000","last_unstale":"2023-11-26T07:21:56.313461+0000","last_undegraded":"2023-11-26T07:21:56.313461+0000","last_fullsized":"2023-11-26T07:21:56.313461+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:21:54.255568+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:21:54.255568+0000","last_clean_scrub_stamp":"2023-11-26T07:21:54.255568+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T10:56:17.443961+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.18","version":"0'0","reported_seq":9,"reported_epoch":58,"state":"active+clean","last_fresh":"2023-11-26T07:21:56.310785+0000","last_change":"2023-11-26T07:21:56.298980+0000","last_active":"2023-11-26T07:21:56.310785+0000","last_peered":"2023-11-26T07:21:56.310785+0000","last_clean":"2023-11-26T07:21:56.310785+0000","last_became_active":"2023-11-26T07:21:56.298626+0000","last_became_peered":"2023-11-26T07:21:56.298626+0000","last_unstale":"2023-11-26T07:21:56.310785+0000","last_undegraded":"2023-11-26T07:21:56.310785+0000","last_fullsized":"2023-11-26T07:21:56.310785+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:21:55.264078+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:21:55.264078+0000","last_clean_scrub_stamp":"2023-11-26T07:21:55.264078+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T18:59:37.088334+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"8.19","version":"0'0","reported_seq":11,"reported_epoch":58,"state":"active+clean","last_fresh":"2023-11-26T07:21:56.284919+0000","last_change":"2023-11-26T07:21:55.288232+0000","last_active":"2023-11-26T07:21:56.284919+0000","last_peered":"2023-11-26T07:21:56.284919+0000","last_clean":"2023-11-26T07:21:56.284919+0000","last_became_active":"2023-11-26T07:21:55.287513+0000","last_became_peered":"2023-11-26T07:21:55.287513+0000","last_unstale":"2023-11-26T07:21:56.284919+0000","last_undegraded":"2023-11-26T07:21:56.284919+0000","last_fullsized":"2023-11-26T07:21:56.284919+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:21:54.255568+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:21:54.255568+0000","last_clean_scrub_stamp":"2023-11-26T07:21:54.255568+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T13:38:08.175522+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.19","version":"0'0","reported_seq":9,"reported_epoch":58,"state":"active+clean","last_fresh":"2023-11-26T07:21:56.302907+0000","last_change":"2023-11-26T07:21:56.296395+0000","last_active":"2023-11-26T07:21:56.302907+0000","last_peered":"2023-11-26T07:21:56.302907+0000","last_clean":"2023-11-26T07:21:56.302907+0000","last_became_active":"2023-11-26T07:21:56.296121+0000","last_became_peered":"2023-11-26T07:21:56.296121+0000","last_unstale":"2023-11-26T07:21:56.302907+0000","last_undegraded":"2023-11-26T07:21:56.302907+0000","last_fullsized":"2023-11-26T07:21:56.302907+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:21:55.264078+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:21:55.264078+0000","last_clean_scrub_stamp":"2023-11-26T07:21:55.264078+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T13:30:24.283597+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.18","version":"0'0","reported_seq":11,"reported_epoch":58,"state":"active+clean","last_fresh":"2023-11-26T07:21:56.283557+0000","last_change":"2023-11-26T07:21:55.299260+0000","last_active":"2023-11-26T07:21:56.283557+0000","last_peered":"2023-11-26T07:21:56.283557+0000","last_clean":"2023-11-26T07:21:56.283557+0000","last_became_active":"2023-11-26T07:21:55.298821+0000","last_became_peered":"2023-11-26T07:21:55.298821+0000","last_unstale":"2023-11-26T07:21:56.283557+0000","last_undegraded":"2023-11-26T07:21:56.283557+0000","last_fullsized":"2023-11-26T07:21:56.283557+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:21:54.255568+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:21:54.255568+0000","last_clean_scrub_stamp":"2023-11-26T07:21:54.255568+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T15:18:49.985683+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.16","version":"0'0","reported_seq":9,"reported_epoch":58,"state":"active+clean","last_fresh":"2023-11-26T07:21:56.313490+0000","last_change":"2023-11-26T07:21:56.305168+0000","last_active":"2023-11-26T07:21:56.313490+0000","last_peered":"2023-11-26T07:21:56.313490+0000","last_clean":"2023-11-26T07:21:56.313490+0000","last_became_active":"2023-11-26T07:21:56.304849+0000","last_became_peered":"2023-11-26T07:21:56.304849+0000","last_unstale":"2023-11-26T07:21:56.313490+0000","last_undegraded":"2023-11-26T07:21:56.313490+0000","last_fullsized":"2023-11-26T07:21:56.313490+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:21:55.264078+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:21:55.264078+0000","last_clean_scrub_stamp":"2023-11-26T07:21:55.264078+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T09:56:06.708591+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"8.17","version":"0'0","reported_seq":11,"reported_epoch":58,"state":"active+clean","last_fresh":"2023-11-26T07:21:56.284545+0000","last_change":"2023-11-26T07:21:55.302413+0000","last_active":"2023-11-26T07:21:56.284545+0000","last_peered":"2023-11-26T07:21:56.284545+0000","last_clean":"2023-11-26T07:21:56.284545+0000","last_became_active":"2023-11-26T07:21:55.302038+0000","last_became_peered":"2023-11-26T07:21:55.302038+0000","last_unstale":"2023-11-26T07:21:56.284545+0000","last_undegraded":"2023-11-26T07:21:56.284545+0000","last_fullsized":"2023-11-26T07:21:56.284545+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:21:54.255568+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:21:54.255568+0000","last_clean_scrub_stamp":"2023-11-26T07:21:54.255568+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T11:54:46.446314+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.17","version":"0'0","reported_seq":9,"reported_epoch":58,"state":"active+clean","last_fresh":"2023-11-26T07:21:56.310836+0000","last_change":"2023-11-26T07:21:56.306728+0000","last_active":"2023-11-26T07:21:56.310836+0000","last_peered":"2023-11-26T07:21:56.310836+0000","last_clean":"2023-11-26T07:21:56.310836+0000","last_became_active":"2023-11-26T07:21:56.306343+0000","last_became_peered":"2023-11-26T07:21:56.306343+0000","last_unstale":"2023-11-26T07:21:56.310836+0000","last_undegraded":"2023-11-26T07:21:56.310836+0000","last_fullsized":"2023-11-26T07:21:56.310836+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:21:55.264078+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:21:55.264078+0000","last_clean_scrub_stamp":"2023-11-26T07:21:55.264078+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T08:10:58.660224+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.16","version":"0'0","reported_seq":11,"reported_epoch":58,"state":"active+clean","last_fresh":"2023-11-26T07:21:56.285592+0000","last_change":"2023-11-26T07:21:55.297031+0000","last_active":"2023-11-26T07:21:56.285592+0000","last_peered":"2023-11-26T07:21:56.285592+0000","last_clean":"2023-11-26T07:21:56.285592+0000","last_became_active":"2023-11-26T07:21:55.296694+0000","last_became_peered":"2023-11-26T07:21:55.296694+0000","last_unstale":"2023-11-26T07:21:56.285592+0000","last_undegraded":"2023-11-26T07:21:56.285592+0000","last_fullsized":"2023-11-26T07:21:56.285592+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:21:54.255568+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:21:54.255568+0000","last_clean_scrub_stamp":"2023-11-26T07:21:54.255568+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T07:40:23.543416+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.14","version":"0'0","reported_seq":9,"reported_epoch":58,"state":"active+clean","last_fresh":"2023-11-26T07:21:56.302348+0000","last_change":"2023-11-26T07:21:56.299253+0000","last_active":"2023-11-26T07:21:56.302348+0000","last_peered":"2023-11-26T07:21:56.302348+0000","last_clean":"2023-11-26T07:21:56.302348+0000","last_became_active":"2023-11-26T07:21:56.298929+0000","last_became_peered":"2023-11-26T07:21:56.298929+0000","last_unstale":"2023-11-26T07:21:56.302348+0000","last_undegraded":"2023-11-26T07:21:56.302348+0000","last_fullsized":"2023-11-26T07:21:56.302348+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:21:55.264078+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:21:55.264078+0000","last_clean_scrub_stamp":"2023-11-26T07:21:55.264078+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T07:45:52.457913+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.15","version":"58'3","reported_seq":14,"reported_epoch":58,"state":"active+clean","last_fresh":"2023-11-26T07:21:56.315399+0000","last_change":"2023-11-26T07:21:55.302350+0000","last_active":"2023-11-26T07:21:56.315399+0000","last_peered":"2023-11-26T07:21:56.315399+0000","last_clean":"2023-11-26T07:21:56.315399+0000","last_became_active":"2023-11-26T07:21:55.302092+0000","last_became_peered":"2023-11-26T07:21:55.302092+0000","last_unstale":"2023-11-26T07:21:56.315399+0000","last_undegraded":"2023-11-26T07:21:56.315399+0000","last_fullsized":"2023-11-26T07:21:56.315399+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:21:54.255568+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:21:54.255568+0000","last_clean_scrub_stamp":"2023-11-26T07:21:54.255568+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T10:00:06.562625+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.15","version":"0'0","reported_seq":9,"reported_epoch":58,"state":"active+clean","last_fresh":"2023-11-26T07:21:56.312157+0000","last_change":"2023-11-26T07:21:56.301162+0000","last_active":"2023-11-26T07:21:56.312157+0000","last_peered":"2023-11-26T07:21:56.312157+0000","last_clean":"2023-11-26T07:21:56.312157+0000","last_became_active":"2023-11-26T07:21:56.299984+0000","last_became_peered":"2023-11-26T07:21:56.299984+0000","last_unstale":"2023-11-26T07:21:56.312157+0000","last_undegraded":"2023-11-26T07:21:56.312157+0000","last_fullsized":"2023-11-26T07:21:56.312157+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:21:55.264078+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:21:55.264078+0000","last_clean_scrub_stamp":"2023-11-26T07:21:55.264078+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T08:06:02.760482+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.14","version":"58'6","reported_seq":17,"reported_epoch":58,"state":"active+clean","last_fresh":"2023-11-26T07:21:56.318891+0000","last_change":"2023-11-26T07:21:55.288706+0000","last_active":"2023-11-26T07:21:56.318891+0000","last_peered":"2023-11-26T07:21:56.318891+0000","last_clean":"2023-11-26T07:21:56.318891+0000","last_became_active":"2023-11-26T07:21:55.287110+0000","last_became_peered":"2023-11-26T07:21:55.287110+0000","last_unstale":"2023-11-26T07:21:56.318891+0000","last_undegraded":"2023-11-26T07:21:56.318891+0000","last_fullsized":"2023-11-26T07:21:56.318891+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:21:54.255568+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:21:54.255568+0000","last_clean_scrub_stamp":"2023-11-26T07:21:54.255568+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T17:00:32.702011+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.12","version":"0'0","reported_seq":9,"reported_epoch":58,"state":"active+clean","last_fresh":"2023-11-26T07:21:56.314585+0000","last_change":"2023-11-26T07:21:56.302363+0000","last_active":"2023-11-26T07:21:56.314585+0000","last_peered":"2023-11-26T07:21:56.314585+0000","last_clean":"2023-11-26T07:21:56.314585+0000","last_became_active":"2023-11-26T07:21:56.301926+0000","last_became_peered":"2023-11-26T07:21:56.301926+0000","last_unstale":"2023-11-26T07:21:56.314585+0000","last_undegraded":"2023-11-26T07:21:56.314585+0000","last_fullsized":"2023-11-26T07:21:56.314585+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:21:55.264078+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:21:55.264078+0000","last_clean_scrub_stamp":"2023-11-26T07:21:55.264078+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T14:21:21.645228+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.13","version":"58'2","reported_seq":13,"reported_epoch":58,"state":"active+clean","last_fresh":"2023-11-26T07:21:56.315041+0000","last_change":"2023-11-26T07:21:55.297366+0000","last_active":"2023-11-26T07:21:56.315041+0000","last_peered":"2023-11-26T07:21:56.315041+0000","last_clean":"2023-11-26T07:21:56.315041+0000","last_became_active":"2023-11-26T07:21:55.297074+0000","last_became_peered":"2023-11-26T07:21:55.297074+0000","last_unstale":"2023-11-26T07:21:56.315041+0000","last_undegraded":"2023-11-26T07:21:56.315041+0000","last_fullsized":"2023-11-26T07:21:56.315041+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:21:54.255568+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:21:54.255568+0000","last_clean_scrub_stamp":"2023-11-26T07:21:54.255568+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T17:19:44.854745+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.13","version":"0'0","reported_seq":9,"reported_epoch":58,"state":"active+clean","last_fresh":"2023-11-26T07:21:56.309729+0000","last_change":"2023-11-26T07:21:56.303041+0000","last_active":"2023-11-26T07:21:56.309729+0000","last_peered":"2023-11-26T07:21:56.309729+0000","last_clean":"2023-11-26T07:21:56.309729+0000","last_became_active":"2023-11-26T07:21:56.302706+0000","last_became_peered":"2023-11-26T07:21:56.302706+0000","last_unstale":"2023-11-26T07:21:56.309729+0000","last_undegraded":"2023-11-26T07:21:56.309729+0000","last_fullsized":"2023-11-26T07:21:56.309729+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:21:55.264078+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:21:55.264078+0000","last_clean_scrub_stamp":"2023-11-26T07:21:55.264078+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T08:28:48.252754+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.12","version":"0'0","reported_seq":11,"reported_epoch":58,"state":"active+clean","last_fresh":"2023-11-26T07:21:56.277947+0000","last_change":"2023-11-26T07:21:55.295331+0000","last_active":"2023-11-26T07:21:56.277947+0000","last_peered":"2023-11-26T07:21:56.277947+0000","last_clean":"2023-11-26T07:21:56.277947+0000","last_became_active":"2023-11-26T07:21:55.295043+0000","last_became_peered":"2023-11-26T07:21:55.295043+0000","last_unstale":"2023-11-26T07:21:56.277947+0000","last_undegraded":"2023-11-26T07:21:56.277947+0000","last_fullsized":"2023-11-26T07:21:56.277947+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:21:54.255568+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:21:54.255568+0000","last_clean_scrub_stamp":"2023-11-26T07:21:54.255568+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T07:45:51.449403+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.10","version":"0'0","reported_seq":9,"reported_epoch":58,"state":"active+clean","last_fresh":"2023-11-26T07:21:56.313476+0000","last_change":"2023-11-26T07:21:56.300018+0000","last_active":"2023-11-26T07:21:56.313476+0000","last_peered":"2023-11-26T07:21:56.313476+0000","last_clean":"2023-11-26T07:21:56.313476+0000","last_became_active":"2023-11-26T07:21:56.299715+0000","last_became_peered":"2023-11-26T07:21:56.299715+0000","last_unstale":"2023-11-26T07:21:56.313476+0000","last_undegraded":"2023-11-26T07:21:56.313476+0000","last_fullsized":"2023-11-26T07:21:56.313476+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:21:55.264078+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:21:55.264078+0000","last_clean_scrub_stamp":"2023-11-26T07:21:55.264078+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T15:06:34.387124+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"8.11","version":"0'0","reported_seq":11,"reported_epoch":58,"state":"active+clean","last_fresh":"2023-11-26T07:21:56.285066+0000","last_change":"2023-11-26T07:21:55.298972+0000","last_active":"2023-11-26T07:21:56.285066+0000","last_peered":"2023-11-26T07:21:56.285066+0000","last_clean":"2023-11-26T07:21:56.285066+0000","last_became_active":"2023-11-26T07:21:55.297927+0000","last_became_peered":"2023-11-26T07:21:55.297927+0000","last_unstale":"2023-11-26T07:21:56.285066+0000","last_undegraded":"2023-11-26T07:21:56.285066+0000","last_fullsized":"2023-11-26T07:21:56.285066+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:21:54.255568+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:21:54.255568+0000","last_clean_scrub_stamp":"2023-11-26T07:21:54.255568+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T10:32:37.418812+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.11","version":"0'0","reported_seq":9,"reported_epoch":58,"state":"active+clean","last_fresh":"2023-11-26T07:21:56.309741+0000","last_change":"2023-11-26T07:21:56.304278+0000","last_active":"2023-11-26T07:21:56.309741+0000","last_peered":"2023-11-26T07:21:56.309741+0000","last_clean":"2023-11-26T07:21:56.309741+0000","last_became_active":"2023-11-26T07:21:56.303968+0000","last_became_peered":"2023-11-26T07:21:56.303968+0000","last_unstale":"2023-11-26T07:21:56.309741+0000","last_undegraded":"2023-11-26T07:21:56.309741+0000","last_fullsized":"2023-11-26T07:21:56.309741+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:21:55.264078+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:21:55.264078+0000","last_clean_scrub_stamp":"2023-11-26T07:21:55.264078+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T15:11:39.329445+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.10","version":"58'8","reported_seq":19,"reported_epoch":58,"state":"active+clean","last_fresh":"2023-11-26T07:21:56.318625+0000","last_change":"2023-11-26T07:21:55.303796+0000","last_active":"2023-11-26T07:21:56.318625+0000","last_peered":"2023-11-26T07:21:56.318625+0000","last_clean":"2023-11-26T07:21:56.318625+0000","last_became_active":"2023-11-26T07:21:55.303540+0000","last_became_peered":"2023-11-26T07:21:55.303540+0000","last_unstale":"2023-11-26T07:21:56.318625+0000","last_undegraded":"2023-11-26T07:21:56.318625+0000","last_fullsized":"2023-11-26T07:21:56.318625+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:21:54.255568+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:21:54.255568+0000","last_clean_scrub_stamp":"2023-11-26T07:21:54.255568+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T10:22:36.620743+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.e","version":"0'0","reported_seq":9,"reported_epoch":58,"state":"active+clean","last_fresh":"2023-11-26T07:21:56.310750+0000","last_change":"2023-11-26T07:21:56.305051+0000","last_active":"2023-11-26T07:21:56.310750+0000","last_peered":"2023-11-26T07:21:56.310750+0000","last_clean":"2023-11-26T07:21:56.310750+0000","last_became_active":"2023-11-26T07:21:56.304788+0000","last_became_peered":"2023-11-26T07:21:56.304788+0000","last_unstale":"2023-11-26T07:21:56.310750+0000","last_undegraded":"2023-11-26T07:21:56.310750+0000","last_fullsized":"2023-11-26T07:21:56.310750+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:21:55.264078+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:21:55.264078+0000","last_clean_scrub_stamp":"2023-11-26T07:21:55.264078+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T11:09:02.730019+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.f","version":"58'1","reported_seq":12,"reported_epoch":58,"state":"active+clean","last_fresh":"2023-11-26T07:21:56.314900+0000","last_change":"2023-11-26T07:21:55.299931+0000","last_active":"2023-11-26T07:21:56.314900+0000","last_peered":"2023-11-26T07:21:56.314900+0000","last_clean":"2023-11-26T07:21:56.314900+0000","last_became_active":"2023-11-26T07:21:55.299643+0000","last_became_peered":"2023-11-26T07:21:55.299643+0000","last_unstale":"2023-11-26T07:21:56.314900+0000","last_undegraded":"2023-11-26T07:21:56.314900+0000","last_fullsized":"2023-11-26T07:21:56.314900+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:21:54.255568+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:21:54.255568+0000","last_clean_scrub_stamp":"2023-11-26T07:21:54.255568+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T19:14:16.437130+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.f","version":"0'0","reported_seq":9,"reported_epoch":58,"state":"active+clean","last_fresh":"2023-11-26T07:21:56.311823+0000","last_change":"2023-11-26T07:21:56.301020+0000","last_active":"2023-11-26T07:21:56.311823+0000","last_peered":"2023-11-26T07:21:56.311823+0000","last_clean":"2023-11-26T07:21:56.311823+0000","last_became_active":"2023-11-26T07:21:56.300756+0000","last_became_peered":"2023-11-26T07:21:56.300756+0000","last_unstale":"2023-11-26T07:21:56.311823+0000","last_undegraded":"2023-11-26T07:21:56.311823+0000","last_fullsized":"2023-11-26T07:21:56.311823+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:21:55.264078+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:21:55.264078+0000","last_clean_scrub_stamp":"2023-11-26T07:21:55.264078+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T09:18:26.215842+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.e","version":"58'2","reported_seq":13,"reported_epoch":58,"state":"active+clean","last_fresh":"2023-11-26T07:21:56.314229+0000","last_change":"2023-11-26T07:21:55.295052+0000","last_active":"2023-11-26T07:21:56.314229+0000","last_peered":"2023-11-26T07:21:56.314229+0000","last_clean":"2023-11-26T07:21:56.314229+0000","last_became_active":"2023-11-26T07:21:55.294786+0000","last_became_peered":"2023-11-26T07:21:55.294786+0000","last_unstale":"2023-11-26T07:21:56.314229+0000","last_undegraded":"2023-11-26T07:21:56.314229+0000","last_fullsized":"2023-11-26T07:21:56.314229+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:21:54.255568+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:21:54.255568+0000","last_clean_scrub_stamp":"2023-11-26T07:21:54.255568+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T15:11:38.320935+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.c","version":"0'0","reported_seq":9,"reported_epoch":58,"state":"active+clean","last_fresh":"2023-11-26T07:21:56.302708+0000","last_change":"2023-11-26T07:21:56.295721+0000","last_active":"2023-11-26T07:21:56.302708+0000","last_peered":"2023-11-26T07:21:56.302708+0000","last_clean":"2023-11-26T07:21:56.302708+0000","last_became_active":"2023-11-26T07:21:56.294569+0000","last_became_peered":"2023-11-26T07:21:56.294569+0000","last_unstale":"2023-11-26T07:21:56.302708+0000","last_undegraded":"2023-11-26T07:21:56.302708+0000","last_fullsized":"2023-11-26T07:21:56.302708+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:21:55.264078+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:21:55.264078+0000","last_clean_scrub_stamp":"2023-11-26T07:21:55.264078+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T18:00:54.173424+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.d","version":"58'1","reported_seq":12,"reported_epoch":58,"state":"active+clean","last_fresh":"2023-11-26T07:21:56.318099+0000","last_change":"2023-11-26T07:21:55.292563+0000","last_active":"2023-11-26T07:21:56.318099+0000","last_peered":"2023-11-26T07:21:56.318099+0000","last_clean":"2023-11-26T07:21:56.318099+0000","last_became_active":"2023-11-26T07:21:55.292240+0000","last_became_peered":"2023-11-26T07:21:55.292240+0000","last_unstale":"2023-11-26T07:21:56.318099+0000","last_undegraded":"2023-11-26T07:21:56.318099+0000","last_fullsized":"2023-11-26T07:21:56.318099+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:21:54.255568+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:21:54.255568+0000","last_clean_scrub_stamp":"2023-11-26T07:21:54.255568+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T08:06:01.751972+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.d","version":"0'0","reported_seq":9,"reported_epoch":58,"state":"active+clean","last_fresh":"2023-11-26T07:21:56.312174+0000","last_change":"2023-11-26T07:21:56.307531+0000","last_active":"2023-11-26T07:21:56.312174+0000","last_peered":"2023-11-26T07:21:56.312174+0000","last_clean":"2023-11-26T07:21:56.312174+0000","last_became_active":"2023-11-26T07:21:56.307217+0000","last_became_peered":"2023-11-26T07:21:56.307217+0000","last_unstale":"2023-11-26T07:21:56.312174+0000","last_undegraded":"2023-11-26T07:21:56.312174+0000","last_fullsized":"2023-11-26T07:21:56.312174+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:21:55.264078+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:21:55.264078+0000","last_clean_scrub_stamp":"2023-11-26T07:21:55.264078+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T19:11:16.774662+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.c","version":"0'0","reported_seq":11,"reported_epoch":58,"state":"active+clean","last_fresh":"2023-11-26T07:21:56.282330+0000","last_change":"2023-11-26T07:21:55.297889+0000","last_active":"2023-11-26T07:21:56.282330+0000","last_peered":"2023-11-26T07:21:56.282330+0000","last_clean":"2023-11-26T07:21:56.282330+0000","last_became_active":"2023-11-26T07:21:55.297620+0000","last_became_peered":"2023-11-26T07:21:55.297620+0000","last_unstale":"2023-11-26T07:21:56.282330+0000","last_undegraded":"2023-11-26T07:21:56.282330+0000","last_fullsized":"2023-11-26T07:21:56.282330+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:21:54.255568+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:21:54.255568+0000","last_clean_scrub_stamp":"2023-11-26T07:21:54.255568+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T19:11:15.766152+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.a","version":"0'0","reported_seq":9,"reported_epoch":58,"state":"active+clean","last_fresh":"2023-11-26T07:21:56.314565+0000","last_change":"2023-11-26T07:21:56.302195+0000","last_active":"2023-11-26T07:21:56.314565+0000","last_peered":"2023-11-26T07:21:56.314565+0000","last_clean":"2023-11-26T07:21:56.314565+0000","last_became_active":"2023-11-26T07:21:56.301576+0000","last_became_peered":"2023-11-26T07:21:56.301576+0000","last_unstale":"2023-11-26T07:21:56.314565+0000","last_undegraded":"2023-11-26T07:21:56.314565+0000","last_fullsized":"2023-11-26T07:21:56.314565+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:21:55.264078+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:21:55.264078+0000","last_clean_scrub_stamp":"2023-11-26T07:21:55.264078+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T10:27:44.765406+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.b","version":"58'2","reported_seq":13,"reported_epoch":58,"state":"active+clean","last_fresh":"2023-11-26T07:21:56.316778+0000","last_change":"2023-11-26T07:21:55.299504+0000","last_active":"2023-11-26T07:21:56.316778+0000","last_peered":"2023-11-26T07:21:56.316778+0000","last_clean":"2023-11-26T07:21:56.316778+0000","last_became_active":"2023-11-26T07:21:55.299171+0000","last_became_peered":"2023-11-26T07:21:55.299171+0000","last_unstale":"2023-11-26T07:21:56.316778+0000","last_undegraded":"2023-11-26T07:21:56.316778+0000","last_fullsized":"2023-11-26T07:21:56.316778+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:21:54.255568+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:21:54.255568+0000","last_clean_scrub_stamp":"2023-11-26T07:21:54.255568+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T16:39:35.277308+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.b","version":"0'0","reported_seq":9,"reported_epoch":58,"state":"active+clean","last_fresh":"2023-11-26T07:21:56.304317+0000","last_change":"2023-11-26T07:21:56.300647+0000","last_active":"2023-11-26T07:21:56.304317+0000","last_peered":"2023-11-26T07:21:56.304317+0000","last_clean":"2023-11-26T07:21:56.304317+0000","last_became_active":"2023-11-26T07:21:56.296266+0000","last_became_peered":"2023-11-26T07:21:56.296266+0000","last_unstale":"2023-11-26T07:21:56.304317+0000","last_undegraded":"2023-11-26T07:21:56.304317+0000","last_fullsized":"2023-11-26T07:21:56.304317+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:21:55.264078+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:21:55.264078+0000","last_clean_scrub_stamp":"2023-11-26T07:21:55.264078+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T12:18:44.648658+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.a","version":"0'0","reported_seq":11,"reported_epoch":58,"state":"active+clean","last_fresh":"2023-11-26T07:21:56.284049+0000","last_change":"2023-11-26T07:21:55.285429+0000","last_active":"2023-11-26T07:21:56.284049+0000","last_peered":"2023-11-26T07:21:56.284049+0000","last_clean":"2023-11-26T07:21:56.284049+0000","last_became_active":"2023-11-26T07:21:55.285087+0000","last_became_peered":"2023-11-26T07:21:55.285087+0000","last_unstale":"2023-11-26T07:21:56.284049+0000","last_undegraded":"2023-11-26T07:21:56.284049+0000","last_fullsized":"2023-11-26T07:21:56.284049+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:21:54.255568+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:21:54.255568+0000","last_clean_scrub_stamp":"2023-11-26T07:21:54.255568+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T07:44:24.313686+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.9","version":"0'0","reported_seq":9,"reported_epoch":58,"state":"active+clean","last_fresh":"2023-11-26T07:21:56.307885+0000","last_change":"2023-11-26T07:21:56.301561+0000","last_active":"2023-11-26T07:21:56.307885+0000","last_peered":"2023-11-26T07:21:56.307885+0000","last_clean":"2023-11-26T07:21:56.307885+0000","last_became_active":"2023-11-26T07:21:56.301217+0000","last_became_peered":"2023-11-26T07:21:56.301217+0000","last_unstale":"2023-11-26T07:21:56.307885+0000","last_undegraded":"2023-11-26T07:21:56.307885+0000","last_fullsized":"2023-11-26T07:21:56.307885+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:21:55.264078+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:21:55.264078+0000","last_clean_scrub_stamp":"2023-11-26T07:21:55.264078+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T08:28:48.252754+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.8","version":"0'0","reported_seq":11,"reported_epoch":58,"state":"active+clean","last_fresh":"2023-11-26T07:21:56.283515+0000","last_change":"2023-11-26T07:21:55.298840+0000","last_active":"2023-11-26T07:21:56.283515+0000","last_peered":"2023-11-26T07:21:56.283515+0000","last_clean":"2023-11-26T07:21:56.283515+0000","last_became_active":"2023-11-26T07:21:55.297942+0000","last_became_peered":"2023-11-26T07:21:55.297942+0000","last_unstale":"2023-11-26T07:21:56.283515+0000","last_undegraded":"2023-11-26T07:21:56.283515+0000","last_fullsized":"2023-11-26T07:21:56.283515+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:21:54.255568+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:21:54.255568+0000","last_clean_scrub_stamp":"2023-11-26T07:21:54.255568+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T13:02:38.473374+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.6","version":"0'0","reported_seq":9,"reported_epoch":58,"state":"active+clean","last_fresh":"2023-11-26T07:21:56.312471+0000","last_change":"2023-11-26T07:21:56.305431+0000","last_active":"2023-11-26T07:21:56.312471+0000","last_peered":"2023-11-26T07:21:56.312471+0000","last_clean":"2023-11-26T07:21:56.312471+0000","last_became_active":"2023-11-26T07:21:56.305100+0000","last_became_peered":"2023-11-26T07:21:56.305100+0000","last_unstale":"2023-11-26T07:21:56.312471+0000","last_undegraded":"2023-11-26T07:21:56.312471+0000","last_fullsized":"2023-11-26T07:21:56.312471+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:21:55.264078+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:21:55.264078+0000","last_clean_scrub_stamp":"2023-11-26T07:21:55.264078+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T08:28:48.583712+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.7","version":"58'2","reported_seq":13,"reported_epoch":58,"state":"active+clean","last_fresh":"2023-11-26T07:21:56.315445+0000","last_change":"2023-11-26T07:21:55.298487+0000","last_active":"2023-11-26T07:21:56.315445+0000","last_peered":"2023-11-26T07:21:56.315445+0000","last_clean":"2023-11-26T07:21:56.315445+0000","last_became_active":"2023-11-26T07:21:55.298222+0000","last_became_peered":"2023-11-26T07:21:55.298222+0000","last_unstale":"2023-11-26T07:21:56.315445+0000","last_undegraded":"2023-11-26T07:21:56.315445+0000","last_fullsized":"2023-11-26T07:21:56.315445+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:21:54.255568+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:21:54.255568+0000","last_clean_scrub_stamp":"2023-11-26T07:21:54.255568+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T15:18:49.985683+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.7","version":"0'0","reported_seq":9,"reported_epoch":58,"state":"active+clean","last_fresh":"2023-11-26T07:21:56.314603+0000","last_change":"2023-11-26T07:21:56.306517+0000","last_active":"2023-11-26T07:21:56.314603+0000","last_peered":"2023-11-26T07:21:56.314603+0000","last_clean":"2023-11-26T07:21:56.314603+0000","last_became_active":"2023-11-26T07:21:56.306225+0000","last_became_peered":"2023-11-26T07:21:56.306225+0000","last_unstale":"2023-11-26T07:21:56.314603+0000","last_undegraded":"2023-11-26T07:21:56.314603+0000","last_fullsized":"2023-11-26T07:21:56.314603+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:21:55.264078+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:21:55.264078+0000","last_clean_scrub_stamp":"2023-11-26T07:21:55.264078+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T15:10:29.063723+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.6","version":"0'0","reported_seq":11,"reported_epoch":58,"state":"active+clean","last_fresh":"2023-11-26T07:21:56.283407+0000","last_change":"2023-11-26T07:21:55.293050+0000","last_active":"2023-11-26T07:21:56.283407+0000","last_peered":"2023-11-26T07:21:56.283407+0000","last_clean":"2023-11-26T07:21:56.283407+0000","last_became_active":"2023-11-26T07:21:55.292351+0000","last_became_peered":"2023-11-26T07:21:55.292351+0000","last_unstale":"2023-11-26T07:21:56.283407+0000","last_undegraded":"2023-11-26T07:21:56.283407+0000","last_fullsized":"2023-11-26T07:21:56.283407+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:21:54.255568+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:21:54.255568+0000","last_clean_scrub_stamp":"2023-11-26T07:21:54.255568+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T10:03:31.102775+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.4","version":"0'0","reported_seq":9,"reported_epoch":58,"state":"active+clean","last_fresh":"2023-11-26T07:21:56.301571+0000","last_change":"2023-11-26T07:21:56.295014+0000","last_active":"2023-11-26T07:21:56.301571+0000","last_peered":"2023-11-26T07:21:56.301571+0000","last_clean":"2023-11-26T07:21:56.301571+0000","last_became_active":"2023-11-26T07:21:56.294562+0000","last_became_peered":"2023-11-26T07:21:56.294562+0000","last_unstale":"2023-11-26T07:21:56.301571+0000","last_undegraded":"2023-11-26T07:21:56.301571+0000","last_fullsized":"2023-11-26T07:21:56.301571+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:21:55.264078+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:21:55.264078+0000","last_clean_scrub_stamp":"2023-11-26T07:21:55.264078+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T12:41:20.887450+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.5","version":"0'0","reported_seq":11,"reported_epoch":58,"state":"active+clean","last_fresh":"2023-11-26T07:21:56.278026+0000","last_change":"2023-11-26T07:21:55.292346+0000","last_active":"2023-11-26T07:21:56.278026+0000","last_peered":"2023-11-26T07:21:56.278026+0000","last_clean":"2023-11-26T07:21:56.278026+0000","last_became_active":"2023-11-26T07:21:55.292035+0000","last_became_peered":"2023-11-26T07:21:55.292035+0000","last_unstale":"2023-11-26T07:21:56.278026+0000","last_undegraded":"2023-11-26T07:21:56.278026+0000","last_fullsized":"2023-11-26T07:21:56.278026+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:21:54.255568+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:21:54.255568+0000","last_clean_scrub_stamp":"2023-11-26T07:21:54.255568+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T12:10:55.955741+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.5","version":"0'0","reported_seq":9,"reported_epoch":58,"state":"active+clean","last_fresh":"2023-11-26T07:21:56.313014+0000","last_change":"2023-11-26T07:21:56.307591+0000","last_active":"2023-11-26T07:21:56.313014+0000","last_peered":"2023-11-26T07:21:56.313014+0000","last_clean":"2023-11-26T07:21:56.313014+0000","last_became_active":"2023-11-26T07:21:56.307309+0000","last_became_peered":"2023-11-26T07:21:56.307309+0000","last_unstale":"2023-11-26T07:21:56.313014+0000","last_undegraded":"2023-11-26T07:21:56.313014+0000","last_fullsized":"2023-11-26T07:21:56.313014+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:21:55.264078+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:21:55.264078+0000","last_clean_scrub_stamp":"2023-11-26T07:21:55.264078+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T10:00:07.571135+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.4","version":"0'0","reported_seq":11,"reported_epoch":58,"state":"active+clean","last_fresh":"2023-11-26T07:21:56.286622+0000","last_change":"2023-11-26T07:21:55.292520+0000","last_active":"2023-11-26T07:21:56.286622+0000","last_peered":"2023-11-26T07:21:56.286622+0000","last_clean":"2023-11-26T07:21:56.286622+0000","last_became_active":"2023-11-26T07:21:55.292203+0000","last_became_peered":"2023-11-26T07:21:55.292203+0000","last_unstale":"2023-11-26T07:21:56.286622+0000","last_undegraded":"2023-11-26T07:21:56.286622+0000","last_fullsized":"2023-11-26T07:21:56.286622+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:21:54.255568+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:21:54.255568+0000","last_clean_scrub_stamp":"2023-11-26T07:21:54.255568+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T13:30:24.409380+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.2","version":"0'0","reported_seq":9,"reported_epoch":58,"state":"active+clean","last_fresh":"2023-11-26T07:21:56.299219+0000","last_change":"2023-11-26T07:21:56.293314+0000","last_active":"2023-11-26T07:21:56.299219+0000","last_peered":"2023-11-26T07:21:56.299219+0000","last_clean":"2023-11-26T07:21:56.299219+0000","last_became_active":"2023-11-26T07:21:56.292857+0000","last_became_peered":"2023-11-26T07:21:56.292857+0000","last_unstale":"2023-11-26T07:21:56.299219+0000","last_undegraded":"2023-11-26T07:21:56.299219+0000","last_fullsized":"2023-11-26T07:21:56.299219+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:21:55.264078+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:21:55.264078+0000","last_clean_scrub_stamp":"2023-11-26T07:21:55.264078+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T09:31:12.063590+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.3","version":"0'0","reported_seq":11,"reported_epoch":58,"state":"active+clean","last_fresh":"2023-11-26T07:21:56.284050+0000","last_change":"2023-11-26T07:21:55.293825+0000","last_active":"2023-11-26T07:21:56.284050+0000","last_peered":"2023-11-26T07:21:56.284050+0000","last_clean":"2023-11-26T07:21:56.284050+0000","last_became_active":"2023-11-26T07:21:55.293536+0000","last_became_peered":"2023-11-26T07:21:55.293536+0000","last_unstale":"2023-11-26T07:21:56.284050+0000","last_undegraded":"2023-11-26T07:21:56.284050+0000","last_fullsized":"2023-11-26T07:21:56.284050+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:21:54.255568+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:21:54.255568+0000","last_clean_scrub_stamp":"2023-11-26T07:21:54.255568+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T18:59:36.079825+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.3","version":"0'0","reported_seq":9,"reported_epoch":58,"state":"active+clean","last_fresh":"2023-11-26T07:21:56.305354+0000","last_change":"2023-11-26T07:21:56.296316+0000","last_active":"2023-11-26T07:21:56.305354+0000","last_peered":"2023-11-26T07:21:56.305354+0000","last_clean":"2023-11-26T07:21:56.305354+0000","last_became_active":"2023-11-26T07:21:56.295949+0000","last_became_peered":"2023-11-26T07:21:56.295949+0000","last_unstale":"2023-11-26T07:21:56.305354+0000","last_undegraded":"2023-11-26T07:21:56.305354+0000","last_fullsized":"2023-11-26T07:21:56.305354+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:21:55.264078+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:21:55.264078+0000","last_clean_scrub_stamp":"2023-11-26T07:21:55.264078+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T12:17:05.630587+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.2","version":"0'0","reported_seq":11,"reported_epoch":58,"state":"active+clean","last_fresh":"2023-11-26T07:21:56.286018+0000","last_change":"2023-11-26T07:21:55.295847+0000","last_active":"2023-11-26T07:21:56.286018+0000","last_peered":"2023-11-26T07:21:56.286018+0000","last_clean":"2023-11-26T07:21:56.286018+0000","last_became_active":"2023-11-26T07:21:55.295562+0000","last_became_peered":"2023-11-26T07:21:55.295562+0000","last_unstale":"2023-11-26T07:21:56.286018+0000","last_undegraded":"2023-11-26T07:21:56.286018+0000","last_fullsized":"2023-11-26T07:21:56.286018+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:21:54.255568+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:21:54.255568+0000","last_clean_scrub_stamp":"2023-11-26T07:21:54.255568+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T15:10:28.055213+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.0","version":"0'0","reported_seq":9,"reported_epoch":58,"state":"active+clean","last_fresh":"2023-11-26T07:21:56.312723+0000","last_change":"2023-11-26T07:21:56.305723+0000","last_active":"2023-11-26T07:21:56.312723+0000","last_peered":"2023-11-26T07:21:56.312723+0000","last_clean":"2023-11-26T07:21:56.312723+0000","last_became_active":"2023-11-26T07:21:56.305464+0000","last_became_peered":"2023-11-26T07:21:56.305464+0000","last_unstale":"2023-11-26T07:21:56.312723+0000","last_undegraded":"2023-11-26T07:21:56.312723+0000","last_fullsized":"2023-11-26T07:21:56.312723+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:21:55.264078+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:21:55.264078+0000","last_clean_scrub_stamp":"2023-11-26T07:21:55.264078+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T08:21:37.149660+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.1","version":"0'0","reported_seq":11,"reported_epoch":58,"state":"active+clean","last_fresh":"2023-11-26T07:21:56.285447+0000","last_change":"2023-11-26T07:21:55.293187+0000","last_active":"2023-11-26T07:21:56.285447+0000","last_peered":"2023-11-26T07:21:56.285447+0000","last_clean":"2023-11-26T07:21:56.285447+0000","last_became_active":"2023-11-26T07:21:55.292900+0000","last_became_peered":"2023-11-26T07:21:55.292900+0000","last_unstale":"2023-11-26T07:21:56.285447+0000","last_undegraded":"2023-11-26T07:21:56.285447+0000","last_fullsized":"2023-11-26T07:21:56.285447+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:21:54.255568+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:21:54.255568+0000","last_clean_scrub_stamp":"2023-11-26T07:21:54.255568+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T15:06:33.378614+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.1","version":"0'0","reported_seq":9,"reported_epoch":58,"state":"active+clean","last_fresh":"2023-11-26T07:21:56.302928+0000","last_change":"2023-11-26T07:21:56.296770+0000","last_active":"2023-11-26T07:21:56.302928+0000","last_peered":"2023-11-26T07:21:56.302928+0000","last_clean":"2023-11-26T07:21:56.302928+0000","last_became_active":"2023-11-26T07:21:56.296481+0000","last_became_peered":"2023-11-26T07:21:56.296481+0000","last_unstale":"2023-11-26T07:21:56.302928+0000","last_undegraded":"2023-11-26T07:21:56.302928+0000","last_fullsized":"2023-11-26T07:21:56.302928+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:21:55.264078+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:21:55.264078+0000","last_clean_scrub_stamp":"2023-11-26T07:21:55.264078+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T11:09:02.730019+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.0","version":"0'0","reported_seq":11,"reported_epoch":58,"state":"active+clean","last_fresh":"2023-11-26T07:21:56.284312+0000","last_change":"2023-11-26T07:21:55.298176+0000","last_active":"2023-11-26T07:21:56.284312+0000","last_peered":"2023-11-26T07:21:56.284312+0000","last_clean":"2023-11-26T07:21:56.284312+0000","last_became_active":"2023-11-26T07:21:55.297908+0000","last_became_peered":"2023-11-26T07:21:55.297908+0000","last_unstale":"2023-11-26T07:21:56.284312+0000","last_undegraded":"2023-11-26T07:21:56.284312+0000","last_fullsized":"2023-11-26T07:21:56.284312+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:21:54.255568+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:21:54.255568+0000","last_clean_scrub_stamp":"2023-11-26T07:21:54.255568+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T13:59:39.325631+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.8","version":"0'0","reported_seq":9,"reported_epoch":58,"state":"active+clean","last_fresh":"2023-11-26T07:21:56.302819+0000","last_change":"2023-11-26T07:21:56.297889+0000","last_active":"2023-11-26T07:21:56.302819+0000","last_peered":"2023-11-26T07:21:56.302819+0000","last_clean":"2023-11-26T07:21:56.302819+0000","last_became_active":"2023-11-26T07:21:56.297618+0000","last_became_peered":"2023-11-26T07:21:56.297618+0000","last_unstale":"2023-11-26T07:21:56.302819+0000","last_undegraded":"2023-11-26T07:21:56.302819+0000","last_fullsized":"2023-11-26T07:21:56.302819+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:21:55.264078+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:21:55.264078+0000","last_clean_scrub_stamp":"2023-11-26T07:21:55.264078+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T10:22:37.629253+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.9","version":"0'0","reported_seq":11,"reported_epoch":58,"state":"active+clean","last_fresh":"2023-11-26T07:21:56.277661+0000","last_change":"2023-11-26T07:21:55.291675+0000","last_active":"2023-11-26T07:21:56.277661+0000","last_peered":"2023-11-26T07:21:56.277661+0000","last_clean":"2023-11-26T07:21:56.277661+0000","last_became_active":"2023-11-26T07:21:55.291359+0000","last_became_peered":"2023-11-26T07:21:55.291359+0000","last_unstale":"2023-11-26T07:21:56.277661+0000","last_undegraded":"2023-11-26T07:21:56.277661+0000","last_fullsized":"2023-11-26T07:21:56.277661+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:21:54.255568+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:21:54.255568+0000","last_clean_scrub_stamp":"2023-11-26T07:21:54.255568+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T08:21:36.141151+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"1.0","version":"15'117","reported_seq":195,"reported_epoch":57,"state":"active+clean","last_fresh":"2023-11-26T07:21:55.294411+0000","last_change":"2023-11-26T07:17:43.827802+0000","last_active":"2023-11-26T07:21:55.294411+0000","last_peered":"2023-11-26T07:21:55.294411+0000","last_clean":"2023-11-26T07:21:55.294411+0000","last_became_active":"2023-11-26T07:17:43.827308+0000","last_became_peered":"2023-11-26T07:17:43.827308+0000","last_unstale":"2023-11-26T07:21:55.294411+0000","last_undegraded":"2023-11-26T07:21:55.294411+0000","last_fullsized":"2023-11-26T07:21:55.294411+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:17:42.160945+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:17:42.160945+0000","last_clean_scrub_stamp":"2023-11-26T07:17:42.160945+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T08:17:54.167366+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":9,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":8,"num_pg":32,"stat_sum":{"num_bytes":2410,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":38,"ondisk_log_size":38,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574901,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":28292,"kb_used_data":1268,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343548,"statfs":{"total":96636764160,"available":96607793152,"internally_reserved":0,"allocated":1298432,"data_stored":879992,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Nov 26 07:21:51 2023","interfaces":[{"interface":"back","average":{"1min":0.76600000000000001,"5min":0.71899999999999997,"15min":0.71599999999999997},"min":{"1min":0.42999999999999999,"5min":0.42999999999999999,"15min":0.42999999999999999},"max":{"1min":0.95999999999999996,"5min":0.98199999999999998,"15min":0.98199999999999998},"last":0.437},{"interface":"front","average":{"1min":0.76400000000000001,"5min":0.72799999999999998,"15min":0.71499999999999997},"min":{"1min":0.53300000000000003,"5min":0.52900000000000003,"15min":0.52900000000000003},"max":{"1min":0.98599999999999999,"5min":1.0980000000000001,"15min":1.0980000000000001},"last":0.52900000000000003}]},{"osd":1,"last update":"Sun Nov 26 07:21:51 2023","interfaces":[{"interface":"back","average":{"1min":0.70599999999999996,"5min":0.68700000000000006,"15min":0.68300000000000005},"min":{"1min":0.441,"5min":0.441,"15min":0.441},"max":{"1min":1.012,"5min":1.0669999999999999,"15min":1.0669999999999999},"last":0.84599999999999997},{"interface":"front","average":{"1min":0.73599999999999999,"5min":0.74399999999999999,"15min":0.80700000000000005},"min":{"1min":0.58099999999999996,"5min":0.47599999999999998,"15min":0.47599999999999998},"max":{"1min":0.94199999999999995,"5min":4.5309999999999997,"15min":4.5309999999999997},"last":0.58899999999999997}]},{"osd":2,"last update":"Sun Nov 26 07:21:51 2023","interfaces":[{"interface":"back","average":{"1min":0.73399999999999999,"5min":0.69299999999999995,"15min":0.68100000000000005},"min":{"1min":0.39700000000000002,"5min":0.39700000000000002,"15min":0.39700000000000002},"max":{"1min":1.0800000000000001,"5min":1.0800000000000001,"15min":1.0800000000000001},"last":0.71099999999999997},{"interface":"front","average":{"1min":0.76800000000000002,"5min":0.71299999999999997,"15min":0.71299999999999997},"min":{"1min":0.497,"5min":0.497,"15min":0.497},"max":{"1min":0.96299999999999997,"5min":1.1339999999999999,"15min":1.1339999999999999},"last":0.80700000000000005}]},{"osd":3,"last update":"Sun Nov 26 07:21:51 2023","interfaces":[{"interface":"back","average":{"1min":0.73799999999999999,"5min":0.71899999999999997,"15min":0.70399999999999996},"min":{"1min":0.53000000000000003,"5min":0.47799999999999998,"15min":0.47799999999999998},"max":{"1min":1.1240000000000001,"5min":1.1240000000000001,"15min":1.1240000000000001},"last":0.67300000000000004},{"interface":"front","average":{"1min":0.81599999999999995,"5min":0.748,"15min":0.73799999999999999},"min":{"1min":0.54600000000000004,"5min":0.433,"15min":0.433},"max":{"1min":1.036,"5min":1.4490000000000001,"15min":1.4490000000000001},"last":0.747}]},{"osd":4,"last update":"Sun Nov 26 07:21:51 2023","interfaces":[{"interface":"back","average":{"1min":0.59999999999999998,"5min":0.61199999999999999,"15min":0.60399999999999998},"min":{"1min":0.47099999999999997,"5min":0.39400000000000002,"15min":0.39400000000000002},"max":{"1min":0.96199999999999997,"5min":0.96299999999999997,"15min":0.96299999999999997},"last":0.88},{"interface":"front","average":{"1min":0.66000000000000003,"5min":0.66100000000000003,"15min":0.66900000000000004},"min":{"1min":0.44600000000000001,"5min":0.436,"15min":0.436},"max":{"1min":1.004,"5min":1.5509999999999999,"15min":1.5509999999999999},"last":0.625}]},{"osd":5,"last update":"Sun Nov 26 07:21:51 2023","interfaces":[{"interface":"back","average":{"1min":0.69299999999999995,"5min":0.68200000000000005,"15min":0.67100000000000004},"min":{"1min":0.46300000000000002,"5min":0.443,"15min":0.443},"max":{"1min":0.93300000000000005,"5min":1.415,"15min":1.415},"last":0.68500000000000005},{"interface":"front","average":{"1min":0.69299999999999995,"5min":0.69399999999999995,"15min":0.70699999999999996},"min":{"1min":0.48699999999999999,"5min":0.48599999999999999,"15min":0.48599999999999999},"max":{"1min":1.0840000000000001,"5min":1.3859999999999999,"15min":1.3859999999999999},"last":0.93200000000000005}]},{"osd":6,"last update":"Sun Nov 26 07:21:51 2023","interfaces":[{"interface":"back","average":{"1min":0.79200000000000004,"5min":0.77800000000000002,"15min":0.77300000000000002},"min":{"1min":0.56399999999999995,"5min":0.51700000000000002,"15min":0.51700000000000002},"max":{"1min":1.123,"5min":1.3069999999999999,"15min":1.3069999999999999},"last":0.97599999999999998},{"interface":"front","average":{"1min":0.75600000000000001,"5min":0.76200000000000001,"15min":0.76800000000000002},"min":{"1min":0.58399999999999996,"5min":0.46899999999999997,"15min":0.46899999999999997},"max":{"1min":0.90800000000000003,"5min":1.6759999999999999,"15min":1.6759999999999999},"last":0.77600000000000002}]}]},{"osd":6,"up_from":13,"seq":55834574903,"num_pgs":17,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27704,"kb_used_data":680,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344136,"statfs":{"total":96636764160,"available":96608395264,"internally_reserved":0,"allocated":696320,"data_stored":289464,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Nov 26 07:21:53 2023","interfaces":[{"interface":"back","average":{"1min":0.69799999999999995,"5min":0.69799999999999995,"15min":0.68799999999999994},"min":{"1min":0.48999999999999999,"5min":0.36699999999999999,"15min":0.36699999999999999},"max":{"1min":0.91800000000000004,"5min":1.7549999999999999,"15min":1.7549999999999999},"last":0.63100000000000001},{"interface":"front","average":{"1min":0.72899999999999998,"5min":0.81799999999999995,"15min":0.91100000000000003},"min":{"1min":0.46600000000000003,"5min":0.38300000000000001,"15min":0.38300000000000001},"max":{"1min":0.874,"5min":5.7489999999999997,"15min":5.7489999999999997},"last":0.60399999999999998}]},{"osd":1,"last update":"Sun Nov 26 07:21:53 2023","interfaces":[{"interface":"back","average":{"1min":0.74199999999999999,"5min":0.80300000000000005,"15min":0.877},"min":{"1min":0.48699999999999999,"5min":0.41199999999999998,"15min":0.41199999999999998},"max":{"1min":0.97799999999999998,"5min":5.4240000000000004,"15min":5.4240000000000004},"last":0.59099999999999997},{"interface":"front","average":{"1min":0.72999999999999998,"5min":0.79500000000000004,"15min":0.86299999999999999},"min":{"1min":0.52200000000000002,"5min":0.499,"15min":0.499},"max":{"1min":0.96499999999999997,"5min":5.6020000000000003,"15min":5.6020000000000003},"last":0.71399999999999997}]},{"osd":2,"last update":"Sun Nov 26 07:21:53 2023","interfaces":[{"interface":"back","average":{"1min":0.70199999999999996,"5min":0.68999999999999995,"15min":0.69399999999999995},"min":{"1min":0.499,"5min":0.45100000000000001,"15min":0.45100000000000001},"max":{"1min":0.93600000000000005,"5min":1.8520000000000001,"15min":1.8520000000000001},"last":0.57999999999999996},{"interface":"front","average":{"1min":0.752,"5min":0.73399999999999999,"15min":0.73399999999999999},"min":{"1min":0.54300000000000004,"5min":0.437,"15min":0.437},"max":{"1min":0.95199999999999996,"5min":1.9159999999999999,"15min":1.9159999999999999},"last":0.69199999999999995}]},{"osd":3,"last update":"Sun Nov 26 07:21:53 2023","interfaces":[{"interface":"back","average":{"1min":0.72699999999999998,"5min":0.80000000000000004,"15min":0.86799999999999999},"min":{"1min":0.52400000000000002,"5min":0.46899999999999997,"15min":0.46899999999999997},"max":{"1min":0.91500000000000004,"5min":5.4390000000000001,"15min":5.4390000000000001},"last":0.64500000000000002},{"interface":"front","average":{"1min":0.74299999999999999,"5min":0.81499999999999995,"15min":0.90100000000000002},"min":{"1min":0.52300000000000002,"5min":0.48399999999999999,"15min":0.48399999999999999},"max":{"1min":0.89800000000000002,"5min":5.5490000000000004,"15min":5.5490000000000004},"last":0.66200000000000003}]},{"osd":4,"last update":"Sun Nov 26 07:21:53 2023","interfaces":[{"interface":"back","average":{"1min":0.56399999999999995,"5min":0.626,"15min":0.63600000000000001},"min":{"1min":0.40600000000000003,"5min":0.371,"15min":0.371},"max":{"1min":0.82999999999999996,"5min":1.0129999999999999,"15min":1.0129999999999999},"last":0.45100000000000001},{"interface":"front","average":{"1min":0.58899999999999997,"5min":0.65300000000000002,"15min":0.66700000000000004},"min":{"1min":0.38600000000000001,"5min":0.38600000000000001,"15min":0.38600000000000001},"max":{"1min":0.86399999999999999,"5min":1.0349999999999999,"15min":1.0349999999999999},"last":0.505}]},{"osd":5,"last update":"Sun Nov 26 07:21:53 2023","interfaces":[{"interface":"back","average":{"1min":0.62,"5min":0.63800000000000001,"15min":0.65200000000000002},"min":{"1min":0.40699999999999997,"5min":0.38200000000000001,"15min":0.38200000000000001},"max":{"1min":0.84499999999999997,"5min":0.873,"15min":0.873},"last":0.53200000000000003},{"interface":"front","average":{"1min":0.65700000000000003,"5min":0.69899999999999995,"15min":0.70799999999999996},"min":{"1min":0.44,"5min":0.436,"15min":0.436},"max":{"1min":0.92900000000000005,"5min":1.0960000000000001,"15min":1.0960000000000001},"last":0.55500000000000005}]},{"osd":7,"last update":"Sun Nov 26 07:21:53 2023","interfaces":[{"interface":"back","average":{"1min":0.67900000000000005,"5min":0.71999999999999997,"15min":0.73399999999999999},"min":{"1min":0.42599999999999999,"5min":0.42599999999999999,"15min":0.42599999999999999},"max":{"1min":0.93999999999999995,"5min":0.998,"15min":0.998},"last":0.67800000000000005},{"interface":"front","average":{"1min":0.64800000000000002,"5min":0.69799999999999995,"15min":0.70999999999999996},"min":{"1min":0.46400000000000002,"5min":0.46400000000000002,"15min":0.46400000000000002},"max":{"1min":0.85299999999999998,"5min":0.98799999999999999,"15min":0.98799999999999999},"last":0.61799999999999999}]}]},{"osd":5,"up_from":13,"seq":55834574901,"num_pgs":6,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27688,"kb_used_data":668,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344152,"statfs":{"total":96636764160,"available":96608411648,"internally_reserved":0,"allocated":684032,"data_stored":282689,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Nov 26 07:21:48 2023","interfaces":[{"interface":"back","average":{"1min":0.72199999999999998,"5min":0.68799999999999994,"15min":0.67300000000000004},"min":{"1min":0.56200000000000006,"5min":0.38100000000000001,"15min":0.38100000000000001},"max":{"1min":0.872,"5min":1.0289999999999999,"15min":1.0289999999999999},"last":0.84799999999999998},{"interface":"front","average":{"1min":0.71399999999999997,"5min":0.69599999999999995,"15min":0.67200000000000004},"min":{"1min":0.50600000000000001,"5min":0.438,"15min":0.438},"max":{"1min":1.028,"5min":1.05,"15min":1.05},"last":0.92200000000000004}]},{"osd":1,"last update":"Sun Nov 26 07:21:48 2023","interfaces":[{"interface":"back","average":{"1min":0.71199999999999997,"5min":0.69499999999999995,"15min":0.67700000000000005},"min":{"1min":0.45700000000000002,"5min":0.41599999999999998,"15min":0.41599999999999998},"max":{"1min":0.95399999999999996,"5min":1.0389999999999999,"15min":1.0389999999999999},"last":0.79900000000000004},{"interface":"front","average":{"1min":0.72199999999999998,"5min":0.70899999999999996,"15min":0.69299999999999995},"min":{"1min":0.53500000000000003,"5min":0.443,"15min":0.443},"max":{"1min":0.99299999999999999,"5min":1.006,"15min":1.006},"last":0.90700000000000003}]},{"osd":2,"last update":"Sun Nov 26 07:21:48 2023","interfaces":[{"interface":"back","average":{"1min":0.72199999999999998,"5min":0.69799999999999995,"15min":0.68200000000000005},"min":{"1min":0.51300000000000001,"5min":0.45900000000000002,"15min":0.45900000000000002},"max":{"1min":0.92900000000000005,"5min":0.95699999999999996,"15min":0.95699999999999996},"last":0.86199999999999999},{"interface":"front","average":{"1min":0.78700000000000003,"5min":0.72999999999999998,"15min":0.68999999999999995},"min":{"1min":0.55100000000000005,"5min":0.51200000000000001,"15min":0.51200000000000001},"max":{"1min":1.1379999999999999,"5min":1.1379999999999999,"15min":1.1379999999999999},"last":0.88600000000000001}]},{"osd":3,"last update":"Sun Nov 26 07:21:48 2023","interfaces":[{"interface":"back","average":{"1min":0.75700000000000001,"5min":0.73299999999999998,"15min":0.72399999999999998},"min":{"1min":0.44700000000000001,"5min":0.43099999999999999,"15min":0.43099999999999999},"max":{"1min":1.1739999999999999,"5min":1.1739999999999999,"15min":1.1739999999999999},"last":0.83399999999999996},{"interface":"front","average":{"1min":0.77800000000000002,"5min":0.72299999999999998,"15min":0.69099999999999995},"min":{"1min":0.55600000000000005,"5min":0.44600000000000001,"15min":0.44600000000000001},"max":{"1min":1.0029999999999999,"5min":1.0209999999999999,"15min":1.0209999999999999},"last":0.93100000000000005}]},{"osd":4,"last update":"Sun Nov 26 07:21:48 2023","interfaces":[{"interface":"back","average":{"1min":0.56799999999999995,"5min":0.59999999999999998,"15min":0.623},"min":{"1min":0.41199999999999998,"5min":0.35299999999999998,"15min":0.35299999999999998},"max":{"1min":0.92700000000000005,"5min":0.95099999999999996,"15min":0.95099999999999996},"last":0.81799999999999995},{"interface":"front","average":{"1min":0.66200000000000003,"5min":0.67000000000000004,"15min":0.69599999999999995},"min":{"1min":0.45000000000000001,"5min":0.41299999999999998,"15min":0.41299999999999998},"max":{"1min":0.94699999999999995,"5min":1.254,"15min":1.254},"last":0.71699999999999997}]},{"osd":6,"last update":"Sun Nov 26 07:21:48 2023","interfaces":[{"interface":"back","average":{"1min":0.61399999999999999,"5min":0.66200000000000003,"15min":0.66800000000000004},"min":{"1min":0.44400000000000001,"5min":0.439,"15min":0.439},"max":{"1min":1.0880000000000001,"5min":1.0880000000000001,"15min":1.0880000000000001},"last":0.67800000000000005},{"interface":"front","average":{"1min":0.66600000000000004,"5min":0.70299999999999996,"15min":0.71899999999999997},"min":{"1min":0.47699999999999998,"5min":0.436,"15min":0.436},"max":{"1min":1.052,"5min":1.0649999999999999,"15min":1.0649999999999999},"last":0.76700000000000002}]},{"osd":7,"last update":"Sun Nov 26 07:21:48 2023","interfaces":[{"interface":"back","average":{"1min":0.73099999999999998,"5min":0.75900000000000001,"15min":0.78900000000000003},"min":{"1min":0.51800000000000002,"5min":0.48199999999999998,"15min":0.48199999999999998},"max":{"1min":1.2010000000000001,"5min":1.4079999999999999,"15min":1.4079999999999999},"last":0.74399999999999999},{"interface":"front","average":{"1min":0.72899999999999998,"5min":0.751,"15min":0.76700000000000002},"min":{"1min":0.52400000000000002,"5min":0.45000000000000001,"15min":0.45000000000000001},"max":{"1min":1.115,"5min":1.115,"15min":1.115},"last":0.98299999999999998}]}]},{"osd":4,"up_from":13,"seq":55834574903,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27716,"kb_used_data":692,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344124,"statfs":{"total":96636764160,"available":96608382976,"internally_reserved":0,"allocated":708608,"data_stored":291076,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Nov 26 07:21:53 2023","interfaces":[{"interface":"back","average":{"1min":0.67300000000000004,"5min":0.70899999999999996,"15min":0.76500000000000001},"min":{"1min":0.46100000000000002,"5min":0.32100000000000001,"15min":0.32100000000000001},"max":{"1min":0.94499999999999995,"5min":4.1689999999999996,"15min":4.1689999999999996},"last":0.67800000000000005},{"interface":"front","average":{"1min":1.609,"5min":0.98299999999999998,"15min":0.995},"min":{"1min":0.52800000000000002,"5min":0.46200000000000002,"15min":0.46200000000000002},"max":{"1min":18.021000000000001,"5min":18.021000000000001,"15min":18.021000000000001},"last":0.83599999999999997}]},{"osd":1,"last update":"Sun Nov 26 07:21:53 2023","interfaces":[{"interface":"back","average":{"1min":0.90500000000000003,"5min":0.71199999999999997,"15min":0.66800000000000004},"min":{"1min":0.47499999999999998,"5min":0.43099999999999999,"15min":0.43099999999999999},"max":{"1min":4.3159999999999998,"5min":4.3159999999999998,"15min":4.3159999999999998},"last":0.749},{"interface":"front","average":{"1min":1.2150000000000001,"5min":0.82799999999999996,"15min":0.82199999999999995},"min":{"1min":0.51900000000000002,"5min":0.39800000000000002,"15min":0.39800000000000002},"max":{"1min":11.206,"5min":11.206,"15min":11.206},"last":0.78100000000000003}]},{"osd":2,"last update":"Sun Nov 26 07:21:53 2023","interfaces":[{"interface":"back","average":{"1min":1.492,"5min":0.89800000000000002,"15min":0.83699999999999997},"min":{"1min":0.47199999999999998,"5min":0.433,"15min":0.433},"max":{"1min":15.441000000000001,"5min":15.441000000000001,"15min":15.441000000000001},"last":0.621},{"interface":"front","average":{"1min":1.2350000000000001,"5min":0.97399999999999998,"15min":1.0349999999999999},"min":{"1min":0.59399999999999997,"5min":0.53400000000000003,"15min":0.53400000000000003},"max":{"1min":9.4800000000000004,"5min":9.4800000000000004,"15min":9.4800000000000004},"last":0.88400000000000001}]},{"osd":3,"last update":"Sun Nov 26 07:21:53 2023","interfaces":[{"interface":"back","average":{"1min":1.3700000000000001,"5min":0.83599999999999997,"15min":0.76500000000000001},"min":{"1min":0.57199999999999995,"5min":0.441,"15min":0.441},"max":{"1min":12.715,"5min":12.715,"15min":12.715},"last":0.90200000000000002},{"interface":"front","average":{"1min":1.145,"5min":0.87,"15min":0.876},"min":{"1min":0.50700000000000001,"5min":0.50700000000000001,"15min":0.50700000000000001},"max":{"1min":8.3239999999999998,"5min":8.3239999999999998,"15min":8.3239999999999998},"last":0.73299999999999998}]},{"osd":5,"last update":"Sun Nov 26 07:21:53 2023","interfaces":[{"interface":"back","average":{"1min":0.63800000000000001,"5min":0.61299999999999999,"15min":0.61099999999999999},"min":{"1min":0.47099999999999997,"5min":0.38200000000000001,"15min":0.38200000000000001},"max":{"1min":0.93999999999999995,"5min":0.96199999999999997,"15min":0.96199999999999997},"last":0.93999999999999995},{"interface":"front","average":{"1min":1.165,"5min":0.72799999999999998,"15min":0.67000000000000004},"min":{"1min":0.442,"5min":0.35599999999999998,"15min":0.35599999999999998},"max":{"1min":11.148999999999999,"5min":11.148999999999999,"15min":11.148999999999999},"last":0.86299999999999999}]},{"osd":6,"last update":"Sun Nov 26 07:21:53 2023","interfaces":[{"interface":"back","average":{"1min":0.624,"5min":0.67500000000000004,"15min":0.754},"min":{"1min":0.41699999999999998,"5min":0.372,"15min":0.372},"max":{"1min":0.81799999999999995,"5min":2.8330000000000002,"15min":2.8330000000000002},"last":0.81799999999999995},{"interface":"front","average":{"1min":0.79200000000000004,"5min":0.83299999999999996,"15min":0.94999999999999996},"min":{"1min":0.54000000000000004,"5min":0.41299999999999998,"15min":0.41299999999999998},"max":{"1min":2.472,"5min":7.6050000000000004,"15min":7.6050000000000004},"last":0.97699999999999998}]},{"osd":7,"last update":"Sun Nov 26 07:21:53 2023","interfaces":[{"interface":"back","average":{"1min":1.339,"5min":0.88100000000000001,"15min":0.86699999999999999},"min":{"1min":0.57999999999999996,"5min":0.44700000000000001,"15min":0.44700000000000001},"max":{"1min":12.653,"5min":12.653,"15min":12.653},"last":0.95999999999999996},{"interface":"front","average":{"1min":0.90100000000000002,"5min":0.82199999999999995,"15min":0.86799999999999999},"min":{"1min":0.47499999999999998,"5min":0.33300000000000002,"15min":0.33300000000000002},"max":{"1min":3.8319999999999999,"5min":3.8319999999999999,"15min":3.8319999999999999},"last":0.91900000000000004}]}]},{"osd":3,"up_from":13,"seq":55834574904,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31872,"kb_used_data":688,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339968,"statfs":{"total":96636764160,"available":96604127232,"internally_reserved":0,"allocated":704512,"data_stored":289520,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1592,"internal_metadata":31914440},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Nov 26 07:21:48 2023","interfaces":[{"interface":"back","average":{"1min":0.46000000000000002,"5min":0.41299999999999998,"15min":0.39000000000000001},"min":{"1min":0.25800000000000001,"5min":0.25800000000000001,"15min":0.25800000000000001},"max":{"1min":0.72799999999999998,"5min":0.72799999999999998,"15min":0.72799999999999998},"last":0.318},{"interface":"front","average":{"1min":0.48599999999999999,"5min":0.45900000000000002,"15min":0.441},"min":{"1min":0.316,"5min":0.26900000000000002,"15min":0.26900000000000002},"max":{"1min":0.71299999999999997,"5min":0.77300000000000002,"15min":0.77300000000000002},"last":0.54500000000000004}]},{"osd":1,"last update":"Sun Nov 26 07:21:48 2023","interfaces":[{"interface":"back","average":{"1min":0.443,"5min":0.438,"15min":0.42599999999999999},"min":{"1min":0.34100000000000003,"5min":0.28799999999999998,"15min":0.28799999999999998},"max":{"1min":0.59299999999999997,"5min":0.752,"15min":0.752},"last":0.38300000000000001},{"interface":"front","average":{"1min":0.503,"5min":0.498,"15min":0.48799999999999999},"min":{"1min":0.374,"5min":0.34599999999999997,"15min":0.34599999999999997},"max":{"1min":0.76200000000000001,"5min":0.872,"15min":0.872},"last":0.35599999999999998}]},{"osd":2,"last update":"Sun Nov 26 07:21:48 2023","interfaces":[{"interface":"back","average":{"1min":0.58399999999999996,"5min":0.56299999999999994,"15min":0.54300000000000004},"min":{"1min":0.376,"5min":0.376,"15min":0.376},"max":{"1min":0.97199999999999998,"5min":0.97199999999999998,"15min":0.97199999999999998},"last":0.44700000000000001},{"interface":"front","average":{"1min":0.56799999999999995,"5min":0.53700000000000003,"15min":0.51500000000000001},"min":{"1min":0.40899999999999997,"5min":0.33000000000000002,"15min":0.33000000000000002},"max":{"1min":0.81999999999999995,"5min":0.89700000000000002,"15min":0.89700000000000002},"last":0.40899999999999997}]},{"osd":4,"last update":"Sun Nov 26 07:21:48 2023","interfaces":[{"interface":"back","average":{"1min":0.71099999999999997,"5min":0.71799999999999997,"15min":0.70999999999999996},"min":{"1min":0.45800000000000002,"5min":0.45800000000000002,"15min":0.45800000000000002},"max":{"1min":0.93000000000000005,"5min":1.091,"15min":1.091},"last":0.52400000000000002},{"interface":"front","average":{"1min":0.73999999999999999,"5min":0.73899999999999999,"15min":0.73099999999999998},"min":{"1min":0.39900000000000002,"5min":0.39900000000000002,"15min":0.39900000000000002},"max":{"1min":1.0289999999999999,"5min":1.242,"15min":1.242},"last":0.58299999999999996}]},{"osd":5,"last update":"Sun Nov 26 07:21:48 2023","interfaces":[{"interface":"back","average":{"1min":0.70999999999999996,"5min":0.73299999999999998,"15min":0.73699999999999999},"min":{"1min":0.47099999999999997,"5min":0.47099999999999997,"15min":0.47099999999999997},"max":{"1min":0.94699999999999995,"5min":1.105,"15min":1.105},"last":0.501},{"interface":"front","average":{"1min":0.73499999999999999,"5min":0.75,"15min":0.74199999999999999},"min":{"1min":0.48799999999999999,"5min":0.48799999999999999,"15min":0.48799999999999999},"max":{"1min":0.97399999999999998,"5min":1.1319999999999999,"15min":1.1319999999999999},"last":0.48799999999999999}]},{"osd":6,"last update":"Sun Nov 26 07:21:48 2023","interfaces":[{"interface":"back","average":{"1min":0.76100000000000001,"5min":0.79400000000000004,"15min":0.80700000000000005},"min":{"1min":0.503,"5min":0.503,"15min":0.503},"max":{"1min":0.89100000000000001,"5min":1.2729999999999999,"15min":1.2729999999999999},"last":0.67500000000000004},{"interface":"front","average":{"1min":0.77500000000000002,"5min":0.77900000000000003,"15min":0.78800000000000003},"min":{"1min":0.51100000000000001,"5min":0.47299999999999998,"15min":0.47299999999999998},"max":{"1min":1.002,"5min":1.204,"15min":1.204},"last":0.66500000000000004}]},{"osd":7,"last update":"Sun Nov 26 07:21:48 2023","interfaces":[{"interface":"back","average":{"1min":0.78900000000000003,"5min":0.77700000000000002,"15min":0.79300000000000004},"min":{"1min":0.54500000000000004,"5min":0.5,"15min":0.5},"max":{"1min":0.999,"5min":1.3600000000000001,"15min":1.3600000000000001},"last":0.65200000000000002},{"interface":"front","average":{"1min":0.77600000000000002,"5min":0.81699999999999995,"15min":0.82299999999999995},"min":{"1min":0.47299999999999998,"5min":0.47299999999999998,"15min":0.47299999999999998},"max":{"1min":1.0349999999999999,"5min":1.617,"15min":1.617},"last":0.63700000000000001}]}]},{"osd":2,"up_from":13,"seq":55834574904,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27688,"kb_used_data":668,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344152,"statfs":{"total":96636764160,"available":96608411648,"internally_reserved":0,"allocated":684032,"data_stored":282689,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Nov 26 07:21:47 2023","interfaces":[{"interface":"back","average":{"1min":0.42299999999999999,"5min":0.55200000000000005,"15min":0.66000000000000003},"min":{"1min":0.29799999999999999,"5min":0.28699999999999998,"15min":0.28699999999999998},"max":{"1min":0.72899999999999998,"5min":4.7030000000000003,"15min":4.7030000000000003},"last":0.29599999999999999},{"interface":"front","average":{"1min":0.48299999999999998,"5min":0.51300000000000001,"15min":0.54500000000000004},"min":{"1min":0.27300000000000002,"5min":0.27300000000000002,"15min":0.27300000000000002},"max":{"1min":0.71899999999999997,"5min":2.4340000000000002,"15min":2.4340000000000002},"last":0.33600000000000002}]},{"osd":1,"last update":"Sun Nov 26 07:21:47 2023","interfaces":[{"interface":"back","average":{"1min":0.52400000000000002,"5min":0.64900000000000002,"15min":0.79200000000000004},"min":{"1min":0.36099999999999999,"5min":0.34599999999999997,"15min":0.34599999999999997},"max":{"1min":0.86099999999999999,"5min":4.2690000000000001,"15min":4.2690000000000001},"last":0.46100000000000002},{"interface":"front","average":{"1min":0.52700000000000002,"5min":0.60799999999999998,"15min":0.69699999999999995},"min":{"1min":0.376,"5min":0.31900000000000001,"15min":0.31900000000000001},"max":{"1min":0.70899999999999996,"5min":3.9940000000000002,"15min":3.9940000000000002},"last":0.39800000000000002}]},{"osd":3,"last update":"Sun Nov 26 07:21:47 2023","interfaces":[{"interface":"back","average":{"1min":0.497,"5min":0.55200000000000005,"15min":0.61199999999999999},"min":{"1min":0.32200000000000001,"5min":0.32000000000000001,"15min":0.32000000000000001},"max":{"1min":0.82699999999999996,"5min":4.0300000000000002,"15min":4.0300000000000002},"last":0.318},{"interface":"front","average":{"1min":0.56399999999999995,"5min":0.70299999999999996,"15min":0.84299999999999997},"min":{"1min":0.36099999999999999,"5min":0.36099999999999999,"15min":0.36099999999999999},"max":{"1min":0.86699999999999999,"5min":4.5810000000000004,"15min":4.5810000000000004},"last":0.48699999999999999}]},{"osd":4,"last update":"Sun Nov 26 07:21:47 2023","interfaces":[{"interface":"back","average":{"1min":0.78500000000000003,"5min":0.85599999999999998,"15min":0.94199999999999995},"min":{"1min":0.58899999999999997,"5min":0.51400000000000001,"15min":0.51400000000000001},"max":{"1min":0.98699999999999999,"5min":4.4269999999999996,"15min":4.4269999999999996},"last":0.91300000000000003},{"interface":"front","average":{"1min":0.83799999999999997,"5min":0.88100000000000001,"15min":0.96999999999999997},"min":{"1min":0.55500000000000005,"5min":0.55500000000000005,"15min":0.55500000000000005},"max":{"1min":1.0780000000000001,"5min":4.085,"15min":4.085},"last":0.77600000000000002}]},{"osd":5,"last update":"Sun Nov 26 07:21:47 2023","interfaces":[{"interface":"back","average":{"1min":0.81200000000000006,"5min":0.81000000000000005,"15min":0.85199999999999998},"min":{"1min":0.60699999999999998,"5min":0.49199999999999999,"15min":0.49199999999999999},"max":{"1min":1,"5min":3.9590000000000001,"15min":3.9590000000000001},"last":0.79700000000000004},{"interface":"front","average":{"1min":0.81200000000000006,"5min":0.874,"15min":0.94999999999999996},"min":{"1min":0.499,"5min":0.499,"15min":0.499},"max":{"1min":1.024,"5min":4.3849999999999998,"15min":4.3849999999999998},"last":0.86099999999999999}]},{"osd":6,"last update":"Sun Nov 26 07:21:47 2023","interfaces":[{"interface":"back","average":{"1min":0.79100000000000004,"5min":0.86199999999999999,"15min":0.95099999999999996},"min":{"1min":0.51800000000000002,"5min":0.51800000000000002,"15min":0.51800000000000002},"max":{"1min":0.94999999999999996,"5min":4.0449999999999999,"15min":4.0449999999999999},"last":0.88300000000000001},{"interface":"front","average":{"1min":0.77000000000000002,"5min":0.83499999999999996,"15min":0.89000000000000001},"min":{"1min":0.48099999999999998,"5min":0.48099999999999998,"15min":0.48099999999999998},"max":{"1min":1,"5min":3.992,"15min":3.992},"last":0.76000000000000001}]},{"osd":7,"last update":"Sun Nov 26 07:21:47 2023","interfaces":[{"interface":"back","average":{"1min":0.81599999999999995,"5min":0.88600000000000001,"15min":0.98499999999999999},"min":{"1min":0.63900000000000001,"5min":0.52200000000000002,"15min":0.52200000000000002},"max":{"1min":1.0149999999999999,"5min":4.1219999999999999,"15min":4.1219999999999999},"last":0.84899999999999998},{"interface":"front","average":{"1min":0.80100000000000005,"5min":0.85699999999999998,"15min":0.91100000000000003},"min":{"1min":0.50700000000000001,"5min":0.50700000000000001,"15min":0.50700000000000001},"max":{"1min":0.96599999999999997,"5min":3.9159999999999999,"15min":3.9159999999999999},"last":0.72399999999999998}]}]},{"osd":1,"up_from":13,"seq":55834574903,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27708,"kb_used_data":684,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344132,"statfs":{"total":96636764160,"available":96608391168,"internally_reserved":0,"allocated":700416,"data_stored":289554,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Nov 26 07:21:45 2023","interfaces":[{"interface":"back","average":{"1min":0.39400000000000002,"5min":0.40999999999999998,"15min":0.41399999999999998},"min":{"1min":0.29899999999999999,"5min":0.23599999999999999,"15min":0.23599999999999999},"max":{"1min":0.57499999999999996,"5min":0.61199999999999999,"15min":0.61199999999999999},"last":0.34000000000000002},{"interface":"front","average":{"1min":0.435,"5min":0.46200000000000002,"15min":0.47799999999999998},"min":{"1min":0.28899999999999998,"5min":0.26700000000000002,"15min":0.26700000000000002},"max":{"1min":0.61699999999999999,"5min":0.78700000000000003,"15min":0.78700000000000003},"last":0.44400000000000001}]},{"osd":2,"last update":"Sun Nov 26 07:21:45 2023","interfaces":[{"interface":"back","average":{"1min":0.42999999999999999,"5min":0.443,"15min":0.45100000000000001},"min":{"1min":0.35499999999999998,"5min":0.27800000000000002,"15min":0.27800000000000002},"max":{"1min":0.53600000000000003,"5min":0.67600000000000005,"15min":0.67600000000000005},"last":0.47299999999999998},{"interface":"front","average":{"1min":0.47399999999999998,"5min":0.498,"15min":0.504},"min":{"1min":0.35999999999999999,"5min":0.27700000000000002,"15min":0.27700000000000002},"max":{"1min":0.64200000000000002,"5min":0.80600000000000005,"15min":0.80600000000000005},"last":0.50900000000000001}]},{"osd":3,"last update":"Sun Nov 26 07:21:45 2023","interfaces":[{"interface":"back","average":{"1min":0.51200000000000001,"5min":0.52800000000000002,"15min":0.52900000000000003},"min":{"1min":0.42099999999999999,"5min":0.38400000000000001,"15min":0.38400000000000001},"max":{"1min":0.60799999999999998,"5min":0.748,"15min":0.748},"last":0.628},{"interface":"front","average":{"1min":0.496,"5min":0.499,"15min":0.497},"min":{"1min":0.35499999999999998,"5min":0.31,"15min":0.31},"max":{"1min":0.68600000000000005,"5min":0.80700000000000005,"15min":0.80700000000000005},"last":0.48999999999999999}]},{"osd":4,"last update":"Sun Nov 26 07:21:45 2023","interfaces":[{"interface":"back","average":{"1min":0.749,"5min":0.71999999999999997,"15min":0.70199999999999996},"min":{"1min":0.51500000000000001,"5min":0.41299999999999998,"15min":0.41299999999999998},"max":{"1min":1.0589999999999999,"5min":1.1000000000000001,"15min":1.1000000000000001},"last":0.53100000000000003},{"interface":"front","average":{"1min":0.71999999999999997,"5min":0.71399999999999997,"15min":0.69099999999999995},"min":{"1min":0.44500000000000001,"5min":0.44500000000000001,"15min":0.44500000000000001},"max":{"1min":0.93300000000000005,"5min":1.131,"15min":1.131},"last":0.54400000000000004}]},{"osd":5,"last update":"Sun Nov 26 07:21:45 2023","interfaces":[{"interface":"back","average":{"1min":0.73399999999999999,"5min":0.73099999999999998,"15min":0.72299999999999998},"min":{"1min":0.52000000000000002,"5min":0.49299999999999999,"15min":0.49299999999999999},"max":{"1min":0.89900000000000002,"5min":1.1919999999999999,"15min":1.1919999999999999},"last":0.57999999999999996},{"interface":"front","average":{"1min":0.70399999999999996,"5min":0.72699999999999998,"15min":0.71999999999999997},"min":{"1min":0.54700000000000004,"5min":0.48799999999999999,"15min":0.48799999999999999},"max":{"1min":0.93400000000000005,"5min":1.0609999999999999,"15min":1.0609999999999999},"last":0.58799999999999997}]},{"osd":6,"last update":"Sun Nov 26 07:21:45 2023","interfaces":[{"interface":"back","average":{"1min":0.76000000000000001,"5min":0.73199999999999998,"15min":0.71299999999999997},"min":{"1min":0.55000000000000004,"5min":0.50600000000000001,"15min":0.50600000000000001},"max":{"1min":0.97799999999999998,"5min":1.0369999999999999,"15min":1.0369999999999999},"last":0.60699999999999998},{"interface":"front","average":{"1min":0.74399999999999999,"5min":0.73399999999999999,"15min":0.71199999999999997},"min":{"1min":0.59999999999999998,"5min":0.504,"15min":0.504},"max":{"1min":0.94999999999999996,"5min":1.1339999999999999,"15min":1.1339999999999999},"last":0.60099999999999998}]},{"osd":7,"last update":"Sun Nov 26 07:21:45 2023","interfaces":[{"interface":"back","average":{"1min":0.75,"5min":0.74099999999999999,"15min":0.72299999999999998},"min":{"1min":0.50600000000000001,"5min":0.50600000000000001,"15min":0.50600000000000001},"max":{"1min":0.92400000000000004,"5min":1.002,"15min":1.002},"last":0.66200000000000003},{"interface":"front","average":{"1min":0.73699999999999999,"5min":0.747,"15min":0.751},"min":{"1min":0.56999999999999995,"5min":0.48499999999999999,"15min":0.48499999999999999},"max":{"1min":1.018,"5min":1.024,"15min":1.024},"last":0.56899999999999995}]}]},{"osd":0,"up_from":13,"seq":55834574903,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32432,"kb_used_data":1248,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339408,"statfs":{"total":96636764160,"available":96603553792,"internally_reserved":0,"allocated":1277952,"data_stored":873057,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Sun Nov 26 07:21:43 2023","interfaces":[{"interface":"back","average":{"1min":0.438,"5min":0.42599999999999999,"15min":0.42899999999999999},"min":{"1min":0.27000000000000002,"5min":0.27000000000000002,"15min":0.27000000000000002},"max":{"1min":0.624,"5min":0.64300000000000002,"15min":0.64300000000000002},"last":0.39800000000000002},{"interface":"front","average":{"1min":0.47299999999999998,"5min":0.46800000000000003,"15min":0.47799999999999998},"min":{"1min":0.32200000000000001,"5min":0.29899999999999999,"15min":0.29899999999999999},"max":{"1min":0.77200000000000002,"5min":1.133,"15min":1.133},"last":0.54900000000000004}]},{"osd":2,"last update":"Sun Nov 26 07:21:43 2023","interfaces":[{"interface":"back","average":{"1min":0.496,"5min":0.51800000000000002,"15min":0.53000000000000003},"min":{"1min":0.34699999999999998,"5min":0.34699999999999998,"15min":0.34699999999999998},"max":{"1min":0.84199999999999997,"5min":0.84199999999999997,"15min":0.84199999999999997},"last":0.53400000000000003},{"interface":"front","average":{"1min":0.56299999999999994,"5min":0.55700000000000005,"15min":0.56100000000000005},"min":{"1min":0.41399999999999998,"5min":0.36399999999999999,"15min":0.36399999999999999},"max":{"1min":0.77300000000000002,"5min":0.84199999999999997,"15min":0.84199999999999997},"last":0.58899999999999997}]},{"osd":3,"last update":"Sun Nov 26 07:21:43 2023","interfaces":[{"interface":"back","average":{"1min":0.52200000000000002,"5min":0.52200000000000002,"15min":0.52800000000000002},"min":{"1min":0.39000000000000001,"5min":0.34999999999999998,"15min":0.34999999999999998},"max":{"1min":0.75700000000000001,"5min":1.3149999999999999,"15min":1.3149999999999999},"last":0.47599999999999998},{"interface":"front","average":{"1min":0.55900000000000005,"5min":0.54400000000000004,"15min":0.55600000000000005},"min":{"1min":0.40000000000000002,"5min":0.371,"15min":0.371},"max":{"1min":0.72799999999999998,"5min":1.2509999999999999,"15min":1.2509999999999999},"last":0.61599999999999999}]},{"osd":4,"last update":"Sun Nov 26 07:21:43 2023","interfaces":[{"interface":"back","average":{"1min":0.747,"5min":0.749,"15min":0.755},"min":{"1min":0.44700000000000001,"5min":0.44700000000000001,"15min":0.44700000000000001},"max":{"1min":1.0900000000000001,"5min":1.2010000000000001,"15min":1.2010000000000001},"last":0.85499999999999998},{"interface":"front","average":{"1min":0.78000000000000003,"5min":0.78800000000000003,"15min":0.78800000000000003},"min":{"1min":0.51100000000000001,"5min":0.51100000000000001,"15min":0.51100000000000001},"max":{"1min":0.99199999999999999,"5min":1.381,"15min":1.381},"last":0.84399999999999997}]},{"osd":5,"last update":"Sun Nov 26 07:21:43 2023","interfaces":[{"interface":"back","average":{"1min":0.75900000000000001,"5min":0.76100000000000001,"15min":0.76000000000000001},"min":{"1min":0.54800000000000004,"5min":0.42199999999999999,"15min":0.42199999999999999},"max":{"1min":0.97999999999999998,"5min":1.1579999999999999,"15min":1.1579999999999999},"last":0.79000000000000004},{"interface":"front","average":{"1min":0.81499999999999995,"5min":0.80600000000000005,"15min":0.80000000000000004},"min":{"1min":0.53600000000000003,"5min":0.53600000000000003,"15min":0.53600000000000003},"max":{"1min":1.0680000000000001,"5min":1.399,"15min":1.399},"last":0.86699999999999999}]},{"osd":6,"last update":"Sun Nov 26 07:21:43 2023","interfaces":[{"interface":"back","average":{"1min":0.72299999999999998,"5min":0.72199999999999998,"15min":0.75},"min":{"1min":0.52400000000000002,"5min":0.50800000000000001,"15min":0.50800000000000001},"max":{"1min":0.89300000000000002,"5min":1.3029999999999999,"15min":1.3029999999999999},"last":0.68700000000000006},{"interface":"front","average":{"1min":0.747,"5min":0.75700000000000001,"15min":0.76500000000000001},"min":{"1min":0.47199999999999998,"5min":0.47199999999999998,"15min":0.47199999999999998},"max":{"1min":1.0489999999999999,"5min":1.3600000000000001,"15min":1.3600000000000001},"last":0.80900000000000005}]},{"osd":7,"last update":"Sun Nov 26 07:21:43 2023","interfaces":[{"interface":"back","average":{"1min":0.77200000000000002,"5min":0.78800000000000003,"15min":0.80300000000000005},"min":{"1min":0.56899999999999995,"5min":0.56899999999999995,"15min":0.56899999999999995},"max":{"1min":0.98899999999999999,"5min":1.177,"15min":1.177},"last":0.82999999999999996},{"interface":"front","average":{"1min":0.78600000000000003,"5min":0.76700000000000002,"15min":0.78800000000000003},"min":{"1min":0.51200000000000001,"5min":0.51200000000000001,"15min":0.51200000000000001},"max":{"1min":1.0629999999999999,"5min":1.2390000000000001,"15min":1.2390000000000001},"last":0.88400000000000001}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":2194,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":56,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":56,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":652,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":160,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-11-26T07:22:00.536 DEBUG:teuthology.orchestra.run.smithi143:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-11-26T07:22:00.713 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:00.712+0000 7f5aa7380640 1 Processor -- start 2023-11-26T07:22:00.713 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:00.712+0000 7f5aa7380640 1 -- start start 2023-11-26T07:22:00.715 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:00.712+0000 7f5aa7380640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f5aa015d6e0 0x7f5aa00d0df0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:22:00.715 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:00.712+0000 7f5aa7380640 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7f5aa00d1bc0 0x7f5aa00b0750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:22:00.715 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:00.712+0000 7f5aa7380640 1 -- --> v1:172.21.15.143:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5aa00cf0e0 con 0x7f5aa00b2160 2023-11-26T07:22:00.715 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:00.712+0000 7f5aa7380640 1 -- --> v2:172.21.15.143:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5aa015dac0 con 0x7f5aa00d1bc0 2023-11-26T07:22:00.715 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:00.712+0000 7f5aa7380640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5aa00ceb20 con 0x7f5aa015d6e0 2023-11-26T07:22:00.715 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:00.712+0000 7f5aa591d640 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7f5aa00d1bc0 0x7f5aa00b0750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:22:00.715 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:00.712+0000 7f5aa591d640 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7f5aa00d1bc0 0x7f5aa00b0750 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:57238/0 (socket says 172.21.15.143:57238) 2023-11-26T07:22:00.715 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:00.712+0000 7f5aa491b640 1 --1- >> v1:172.21.15.143:6789/0 conn(0x7f5aa00b2160 0x7f5aa00d14b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.143:6789/0 says I am v1:172.21.15.143:57258/0 (socket says 172.21.15.143:57258) 2023-11-26T07:22:00.715 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:00.712+0000 7f5aa591d640 1 -- 172.21.15.143:0/1422601581 learned_addr learned my addr 172.21.15.143:0/1422601581 (peer_addr_for_me v2:172.21.15.143:0/0) 2023-11-26T07:22:00.715 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:00.712+0000 7f5aa511c640 1 --2- 172.21.15.143:0/1422601581 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f5aa015d6e0 0x7f5aa00d0df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:22:00.716 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:00.712+0000 7f5aa591d640 1 -- 172.21.15.143:0/1422601581 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f5aa015d6e0 msgr2=0x7f5aa00d0df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-26T07:22:00.716 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:00.712+0000 7f5aa591d640 1 --2- 172.21.15.143:0/1422601581 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f5aa015d6e0 0x7f5aa00d0df0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:22:00.716 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:00.712+0000 7f5aa591d640 1 -- 172.21.15.143:0/1422601581 >> v1:172.21.15.143:6789/0 conn(0x7f5aa00b2160 legacy=0x7f5aa00d14b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:22:00.716 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:00.712+0000 7f5aa591d640 1 -- 172.21.15.143:0/1422601581 --> v2:172.21.15.143:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5aa00cee00 con 0x7f5aa00d1bc0 2023-11-26T07:22:00.717 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:00.716+0000 7f5aa591d640 1 --2- 172.21.15.143:0/1422601581 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f5aa00d1bc0 0x7f5aa00b0750 secure :-1 s=READY pgs=804 cs=0 l=1 rev1=1 crypto rx=0x7f5a98009e50 tx=0x7f5a9801ca10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=38d1efdb5a6d6588 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:22:00.718 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:00.716+0000 7f5a97fff640 1 -- 172.21.15.143:0/1422601581 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5a980a57d0 con 0x7f5aa00d1bc0 2023-11-26T07:22:00.718 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:00.716+0000 7f5a97fff640 1 -- 172.21.15.143:0/1422601581 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5a980a5930 con 0x7f5aa00d1bc0 2023-11-26T07:22:00.718 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:00.716+0000 7f5a97fff640 1 -- 172.21.15.143:0/1422601581 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5a980a5cc0 con 0x7f5aa00d1bc0 2023-11-26T07:22:00.719 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:00.716+0000 7f5aa7380640 1 -- 172.21.15.143:0/1422601581 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f5aa00d1bc0 msgr2=0x7f5aa00b0750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:22:00.719 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:00.716+0000 7f5aa7380640 1 --2- 172.21.15.143:0/1422601581 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f5aa00d1bc0 0x7f5aa00b0750 secure :-1 s=READY pgs=804 cs=0 l=1 rev1=1 crypto rx=0x7f5a98009e50 tx=0x7f5a9801ca10 comp rx=0 tx=0).stop 2023-11-26T07:22:00.719 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:00.716+0000 7f5aa7380640 1 -- 172.21.15.143:0/1422601581 shutdown_connections 2023-11-26T07:22:00.719 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:00.716+0000 7f5aa7380640 1 --2- 172.21.15.143:0/1422601581 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f5aa00d1bc0 0x7f5aa00b0750 unknown :-1 s=CLOSED pgs=804 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:22:00.719 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:00.716+0000 7f5aa7380640 1 --2- 172.21.15.143:0/1422601581 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f5aa015d6e0 0x7f5aa00d0df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:22:00.719 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:00.716+0000 7f5aa7380640 1 -- 172.21.15.143:0/1422601581 >> 172.21.15.143:0/1422601581 conn(0x7f5aa009c550 msgr2=0x7f5aa00d3220 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:22:00.719 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:00.716+0000 7f5aa7380640 1 -- 172.21.15.143:0/1422601581 shutdown_connections 2023-11-26T07:22:00.719 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:00.716+0000 7f5aa7380640 1 -- 172.21.15.143:0/1422601581 wait complete. 2023-11-26T07:22:00.720 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:00.720+0000 7f5aa7380640 1 Processor -- start 2023-11-26T07:22:00.720 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:00.720+0000 7f5aa7380640 1 -- start start 2023-11-26T07:22:00.722 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:00.720+0000 7f5aa7380640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f5aa00b2160 0x7f5aa0155760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:22:00.722 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:00.720+0000 7f5aa7380640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f5aa00d1bc0 0x7f5aa00cb9d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:22:00.722 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:00.720+0000 7f5aa7380640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f5aa015d6e0 0x7f5aa00cbf10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:22:00.722 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:00.720+0000 7f5aa7380640 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5aa015eb60 con 0x7f5aa00d1bc0 2023-11-26T07:22:00.722 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:00.720+0000 7f5aa7380640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5aa0155ca0 con 0x7f5aa015d6e0 2023-11-26T07:22:00.722 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:00.720+0000 7f5aa7380640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5aa015dc00 con 0x7f5aa00b2160 2023-11-26T07:22:00.723 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:00.720+0000 7f5aa491b640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f5aa00d1bc0 0x7f5aa00cb9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:22:00.723 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:00.720+0000 7f5aa511c640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f5aa00b2160 0x7f5aa0155760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:22:00.723 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:00.720+0000 7f5aa491b640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f5aa00d1bc0 0x7f5aa00cb9d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:57252/0 (socket says 172.21.15.143:57252) 2023-11-26T07:22:00.723 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:00.720+0000 7f5aa491b640 1 -- 172.21.15.143:0/3125572288 learned_addr learned my addr 172.21.15.143:0/3125572288 (peer_addr_for_me v2:172.21.15.143:0/0) 2023-11-26T07:22:00.723 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:00.720+0000 7f5aa591d640 1 --2- 172.21.15.143:0/3125572288 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f5aa015d6e0 0x7f5aa00cbf10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:22:00.723 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:00.720+0000 7f5aa491b640 1 -- 172.21.15.143:0/3125572288 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f5aa00b2160 msgr2=0x7f5aa0155760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:22:00.723 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:00.720+0000 7f5aa491b640 1 --2- 172.21.15.143:0/3125572288 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f5aa00b2160 0x7f5aa0155760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:22:00.723 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:00.720+0000 7f5aa491b640 1 -- 172.21.15.143:0/3125572288 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f5aa015d6e0 msgr2=0x7f5aa00cbf10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:22:00.723 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:00.720+0000 7f5aa491b640 1 --2- 172.21.15.143:0/3125572288 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f5aa015d6e0 0x7f5aa00cbf10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:22:00.723 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:00.720+0000 7f5aa491b640 1 -- 172.21.15.143:0/3125572288 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5a98009b30 con 0x7f5aa00d1bc0 2023-11-26T07:22:00.723 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:00.720+0000 7f5aa491b640 1 --2- 172.21.15.143:0/3125572288 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f5aa00d1bc0 0x7f5aa00cb9d0 secure :-1 s=READY pgs=805 cs=0 l=1 rev1=1 crypto rx=0x7f5a9000c490 tx=0x7f5a9000c950 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:22:00.724 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:00.720+0000 7f5a95ffb640 1 -- 172.21.15.143:0/3125572288 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5a90007480 con 0x7f5aa00d1bc0 2023-11-26T07:22:00.724 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:00.720+0000 7f5a95ffb640 1 -- 172.21.15.143:0/3125572288 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5a900075e0 con 0x7f5aa00d1bc0 2023-11-26T07:22:00.724 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:00.720+0000 7f5a95ffb640 1 -- 172.21.15.143:0/3125572288 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5a90004a10 con 0x7f5aa00d1bc0 2023-11-26T07:22:00.724 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:00.720+0000 7f5aa7380640 1 -- 172.21.15.143:0/3125572288 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5aa015dee0 con 0x7f5aa00d1bc0 2023-11-26T07:22:00.725 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:00.720+0000 7f5aa7380640 1 -- 172.21.15.143:0/3125572288 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5aa015e4a0 con 0x7f5aa00d1bc0 2023-11-26T07:22:00.725 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:00.724+0000 7f5aa7380640 1 -- 172.21.15.143:0/3125572288 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5aa015eca0 con 0x7f5aa00d1bc0 2023-11-26T07:22:00.725 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:00.724+0000 7f5a95ffb640 1 -- 172.21.15.143:0/3125572288 <== mon.0 v2:172.21.15.143:3300/0 4 ==== mgrmap(e 9) v1 ==== 86005+0+0 (secure 0 0 0) 0x7f5a90004b70 con 0x7f5aa00d1bc0 2023-11-26T07:22:00.726 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:00.724+0000 7f5a95ffb640 1 --2- 172.21.15.143:0/3125572288 >> [v2:172.21.15.143:6832/2150557544,v1:172.21.15.143:6833/2150557544] conn(0x7f5a84066620 0x7f5a84068ae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:22:00.726 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:00.724+0000 7f5a95ffb640 1 -- 172.21.15.143:0/3125572288 <== mon.0 v2:172.21.15.143:3300/0 5 ==== osd_map(59..59 src has 1..59) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f5a90083220 con 0x7f5aa00d1bc0 2023-11-26T07:22:00.726 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:00.724+0000 7f5aa511c640 1 --2- 172.21.15.143:0/3125572288 >> [v2:172.21.15.143:6832/2150557544,v1:172.21.15.143:6833/2150557544] conn(0x7f5a84066620 0x7f5a84068ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:22:00.726 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:00.724+0000 7f5aa511c640 1 --2- 172.21.15.143:0/3125572288 >> [v2:172.21.15.143:6832/2150557544,v1:172.21.15.143:6833/2150557544] conn(0x7f5a84066620 0x7f5a84068ae0 secure :-1 s=READY pgs=385 cs=0 l=1 rev1=1 crypto rx=0x7f5a9c0079c0 tx=0x7f5a9c008040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:22:00.730 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:00.728+0000 7f5a95ffb640 1 -- 172.21.15.143:0/3125572288 <== mon.0 v2:172.21.15.143:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172649 (secure 0 0 0) 0x7f5aa015eca0 con 0x7f5aa00d1bc0 2023-11-26T07:22:00.876 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:00.872+0000 7f5aa7380640 1 -- 172.21.15.143:0/3125572288 --> [v2:172.21.15.143:6832/2150557544,v1:172.21.15.143:6833/2150557544] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f5aa015eca0 con 0x7f5a84066620 2023-11-26T07:22:00.883 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:00.880+0000 7f5a95ffb640 1 -- 172.21.15.143:0/3125572288 <== mgr.4100 v2:172.21.15.143:6832/2150557544 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+289118 (secure 0 0 0) 0x7f5aa015eca0 con 0x7f5a84066620 2023-11-26T07:22:00.886 INFO:teuthology.orchestra.run.smithi143.stdout: 2023-11-26T07:22:00.887 INFO:teuthology.orchestra.run.smithi143.stderr:dumped all 2023-11-26T07:22:00.889 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:00.884+0000 7f5aa7380640 1 -- 172.21.15.143:0/3125572288 >> [v2:172.21.15.143:6832/2150557544,v1:172.21.15.143:6833/2150557544] conn(0x7f5a84066620 msgr2=0x7f5a84068ae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:22:00.889 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:00.884+0000 7f5aa7380640 1 --2- 172.21.15.143:0/3125572288 >> [v2:172.21.15.143:6832/2150557544,v1:172.21.15.143:6833/2150557544] conn(0x7f5a84066620 0x7f5a84068ae0 secure :-1 s=READY pgs=385 cs=0 l=1 rev1=1 crypto rx=0x7f5a9c0079c0 tx=0x7f5a9c008040 comp rx=0 tx=0).stop 2023-11-26T07:22:00.889 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:00.884+0000 7f5aa7380640 1 -- 172.21.15.143:0/3125572288 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f5aa00d1bc0 msgr2=0x7f5aa00cb9d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:22:00.889 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:00.884+0000 7f5aa7380640 1 --2- 172.21.15.143:0/3125572288 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f5aa00d1bc0 0x7f5aa00cb9d0 secure :-1 s=READY pgs=805 cs=0 l=1 rev1=1 crypto rx=0x7f5a9000c490 tx=0x7f5a9000c950 comp rx=0 tx=0).stop 2023-11-26T07:22:00.889 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:00.884+0000 7f5aa7380640 1 -- 172.21.15.143:0/3125572288 shutdown_connections 2023-11-26T07:22:00.889 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:00.884+0000 7f5aa7380640 1 --2- 172.21.15.143:0/3125572288 >> [v2:172.21.15.143:6832/2150557544,v1:172.21.15.143:6833/2150557544] conn(0x7f5a84066620 0x7f5a84068ae0 unknown :-1 s=CLOSED pgs=385 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:22:00.889 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:00.884+0000 7f5aa7380640 1 --2- 172.21.15.143:0/3125572288 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f5aa015d6e0 0x7f5aa00cbf10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:22:00.889 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:00.884+0000 7f5aa7380640 1 --2- 172.21.15.143:0/3125572288 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f5aa00d1bc0 0x7f5aa00cb9d0 unknown :-1 s=CLOSED pgs=805 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:22:00.889 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:00.884+0000 7f5aa7380640 1 --2- 172.21.15.143:0/3125572288 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f5aa00b2160 0x7f5aa0155760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:22:00.889 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:00.884+0000 7f5aa7380640 1 -- 172.21.15.143:0/3125572288 >> 172.21.15.143:0/3125572288 conn(0x7f5aa009c550 msgr2=0x7f5aa00b3420 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:22:00.890 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:00.884+0000 7f5aa7380640 1 -- 172.21.15.143:0/3125572288 shutdown_connections 2023-11-26T07:22:00.890 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:00.884+0000 7f5aa7380640 1 -- 172.21.15.143:0/3125572288 wait complete. 2023-11-26T07:22:00.906 INFO:teuthology.orchestra.run.smithi143.stdout:{"pg_ready":true,"pg_map":{"version":181,"stamp":"2023-11-26T07:21:59.748730+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592778,"num_objects":23,"num_object_clones":0,"num_object_copies":46,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":23,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":203,"num_write_kb":2430,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":155,"ondisk_log_size":155,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":68,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":231100,"kb_used_data":6596,"kb_used_omap":12,"kb_used_meta":224371,"kb_avail":754743620,"statfs":{"total":773094113280,"available":772857466880,"internally_reserved":0,"allocated":6754304,"data_stored":3478041,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12749,"internal_metadata":229756467},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":41,"apply_latency_ms":41,"commit_latency_ns":41000000,"apply_latency_ns":41000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2410,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"5.484128"},"pg_stats":[{"pgid":"9.3f","version":"0'0","reported_seq":9,"reported_epoch":58,"state":"active+clean","last_fresh":"2023-11-26T07:21:56.300698+0000","last_change":"2023-11-26T07:21:56.291621+0000","last_active":"2023-11-26T07:21:56.300698+0000","last_peered":"2023-11-26T07:21:56.300698+0000","last_clean":"2023-11-26T07:21:56.300698+0000","last_became_active":"2023-11-26T07:21:56.291367+0000","last_became_peered":"2023-11-26T07:21:56.291367+0000","last_unstale":"2023-11-26T07:21:56.300698+0000","last_undegraded":"2023-11-26T07:21:56.300698+0000","last_fullsized":"2023-11-26T07:21:56.300698+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:21:55.264078+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:21:55.264078+0000","last_clean_scrub_stamp":"2023-11-26T07:21:55.264078+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T10:12:09.752171+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.3e","version":"0'0","reported_seq":9,"reported_epoch":58,"state":"active+clean","last_fresh":"2023-11-26T07:21:56.304718+0000","last_change":"2023-11-26T07:21:56.299056+0000","last_active":"2023-11-26T07:21:56.304718+0000","last_peered":"2023-11-26T07:21:56.304718+0000","last_clean":"2023-11-26T07:21:56.304718+0000","last_became_active":"2023-11-26T07:21:56.297285+0000","last_became_peered":"2023-11-26T07:21:56.297285+0000","last_unstale":"2023-11-26T07:21:56.304718+0000","last_undegraded":"2023-11-26T07:21:56.304718+0000","last_fullsized":"2023-11-26T07:21:56.304718+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:21:55.264078+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:21:55.264078+0000","last_clean_scrub_stamp":"2023-11-26T07:21:55.264078+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T09:40:09.904648+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.3d","version":"0'0","reported_seq":9,"reported_epoch":58,"state":"active+clean","last_fresh":"2023-11-26T07:21:56.305388+0000","last_change":"2023-11-26T07:21:56.296573+0000","last_active":"2023-11-26T07:21:56.305388+0000","last_peered":"2023-11-26T07:21:56.305388+0000","last_clean":"2023-11-26T07:21:56.305388+0000","last_became_active":"2023-11-26T07:21:56.296280+0000","last_became_peered":"2023-11-26T07:21:56.296280+0000","last_unstale":"2023-11-26T07:21:56.305388+0000","last_undegraded":"2023-11-26T07:21:56.305388+0000","last_fullsized":"2023-11-26T07:21:56.305388+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:21:55.264078+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:21:55.264078+0000","last_clean_scrub_stamp":"2023-11-26T07:21:55.264078+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T09:35:29.250849+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.3c","version":"0'0","reported_seq":9,"reported_epoch":58,"state":"active+clean","last_fresh":"2023-11-26T07:21:56.312843+0000","last_change":"2023-11-26T07:21:56.306604+0000","last_active":"2023-11-26T07:21:56.312843+0000","last_peered":"2023-11-26T07:21:56.312843+0000","last_clean":"2023-11-26T07:21:56.312843+0000","last_became_active":"2023-11-26T07:21:56.306030+0000","last_became_peered":"2023-11-26T07:21:56.306030+0000","last_unstale":"2023-11-26T07:21:56.312843+0000","last_undegraded":"2023-11-26T07:21:56.312843+0000","last_fullsized":"2023-11-26T07:21:56.312843+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:21:55.264078+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:21:55.264078+0000","last_clean_scrub_stamp":"2023-11-26T07:21:55.264078+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T07:38:48.885316+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.3b","version":"0'0","reported_seq":9,"reported_epoch":58,"state":"active+clean","last_fresh":"2023-11-26T07:21:56.302339+0000","last_change":"2023-11-26T07:21:56.298880+0000","last_active":"2023-11-26T07:21:56.302339+0000","last_peered":"2023-11-26T07:21:56.302339+0000","last_clean":"2023-11-26T07:21:56.302339+0000","last_became_active":"2023-11-26T07:21:56.298620+0000","last_became_peered":"2023-11-26T07:21:56.298620+0000","last_unstale":"2023-11-26T07:21:56.302339+0000","last_undegraded":"2023-11-26T07:21:56.302339+0000","last_fullsized":"2023-11-26T07:21:56.302339+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:21:55.264078+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:21:55.264078+0000","last_clean_scrub_stamp":"2023-11-26T07:21:55.264078+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T10:17:32.686931+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.3a","version":"0'0","reported_seq":9,"reported_epoch":58,"state":"active+clean","last_fresh":"2023-11-26T07:21:56.304452+0000","last_change":"2023-11-26T07:21:56.300801+0000","last_active":"2023-11-26T07:21:56.304452+0000","last_peered":"2023-11-26T07:21:56.304452+0000","last_clean":"2023-11-26T07:21:56.304452+0000","last_became_active":"2023-11-26T07:21:56.300531+0000","last_became_peered":"2023-11-26T07:21:56.300531+0000","last_unstale":"2023-11-26T07:21:56.304452+0000","last_undegraded":"2023-11-26T07:21:56.304452+0000","last_fullsized":"2023-11-26T07:21:56.304452+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:21:55.264078+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:21:55.264078+0000","last_clean_scrub_stamp":"2023-11-26T07:21:55.264078+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T12:17:05.630587+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.39","version":"0'0","reported_seq":9,"reported_epoch":58,"state":"active+clean","last_fresh":"2023-11-26T07:21:56.313592+0000","last_change":"2023-11-26T07:21:56.311679+0000","last_active":"2023-11-26T07:21:56.313592+0000","last_peered":"2023-11-26T07:21:56.313592+0000","last_clean":"2023-11-26T07:21:56.313592+0000","last_became_active":"2023-11-26T07:21:56.311496+0000","last_became_peered":"2023-11-26T07:21:56.311496+0000","last_unstale":"2023-11-26T07:21:56.313592+0000","last_undegraded":"2023-11-26T07:21:56.313592+0000","last_fullsized":"2023-11-26T07:21:56.313592+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:21:55.264078+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:21:55.264078+0000","last_clean_scrub_stamp":"2023-11-26T07:21:55.264078+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T17:06:42.726725+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.38","version":"0'0","reported_seq":9,"reported_epoch":58,"state":"active+clean","last_fresh":"2023-11-26T07:21:56.302704+0000","last_change":"2023-11-26T07:21:56.297195+0000","last_active":"2023-11-26T07:21:56.302704+0000","last_peered":"2023-11-26T07:21:56.302704+0000","last_clean":"2023-11-26T07:21:56.302704+0000","last_became_active":"2023-11-26T07:21:56.296928+0000","last_became_peered":"2023-11-26T07:21:56.296928+0000","last_unstale":"2023-11-26T07:21:56.302704+0000","last_undegraded":"2023-11-26T07:21:56.302704+0000","last_fullsized":"2023-11-26T07:21:56.302704+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:21:55.264078+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:21:55.264078+0000","last_clean_scrub_stamp":"2023-11-26T07:21:55.264078+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T15:11:39.329445+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.37","version":"0'0","reported_seq":9,"reported_epoch":58,"state":"active+clean","last_fresh":"2023-11-26T07:21:56.307129+0000","last_change":"2023-11-26T07:21:56.297210+0000","last_active":"2023-11-26T07:21:56.307129+0000","last_peered":"2023-11-26T07:21:56.307129+0000","last_clean":"2023-11-26T07:21:56.307129+0000","last_became_active":"2023-11-26T07:21:56.296897+0000","last_became_peered":"2023-11-26T07:21:56.296897+0000","last_unstale":"2023-11-26T07:21:56.307129+0000","last_undegraded":"2023-11-26T07:21:56.307129+0000","last_fullsized":"2023-11-26T07:21:56.307129+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:21:55.264078+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:21:55.264078+0000","last_clean_scrub_stamp":"2023-11-26T07:21:55.264078+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T17:56:48.875217+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.36","version":"0'0","reported_seq":9,"reported_epoch":58,"state":"active+clean","last_fresh":"2023-11-26T07:21:56.312516+0000","last_change":"2023-11-26T07:21:56.304514+0000","last_active":"2023-11-26T07:21:56.312516+0000","last_peered":"2023-11-26T07:21:56.312516+0000","last_clean":"2023-11-26T07:21:56.312516+0000","last_became_active":"2023-11-26T07:21:56.303710+0000","last_became_peered":"2023-11-26T07:21:56.303710+0000","last_unstale":"2023-11-26T07:21:56.312516+0000","last_undegraded":"2023-11-26T07:21:56.312516+0000","last_fullsized":"2023-11-26T07:21:56.312516+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:21:55.264078+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:21:55.264078+0000","last_clean_scrub_stamp":"2023-11-26T07:21:55.264078+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T17:00:33.710521+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.35","version":"0'0","reported_seq":9,"reported_epoch":58,"state":"active+clean","last_fresh":"2023-11-26T07:21:56.305120+0000","last_change":"2023-11-26T07:21:56.301384+0000","last_active":"2023-11-26T07:21:56.305120+0000","last_peered":"2023-11-26T07:21:56.305120+0000","last_clean":"2023-11-26T07:21:56.305120+0000","last_became_active":"2023-11-26T07:21:56.301085+0000","last_became_peered":"2023-11-26T07:21:56.301085+0000","last_unstale":"2023-11-26T07:21:56.305120+0000","last_undegraded":"2023-11-26T07:21:56.305120+0000","last_fullsized":"2023-11-26T07:21:56.305120+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:21:55.264078+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:21:55.264078+0000","last_clean_scrub_stamp":"2023-11-26T07:21:55.264078+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T12:10:56.964250+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.34","version":"0'0","reported_seq":9,"reported_epoch":58,"state":"active+clean","last_fresh":"2023-11-26T07:21:56.312862+0000","last_change":"2023-11-26T07:21:56.306724+0000","last_active":"2023-11-26T07:21:56.312862+0000","last_peered":"2023-11-26T07:21:56.312862+0000","last_clean":"2023-11-26T07:21:56.312862+0000","last_became_active":"2023-11-26T07:21:56.306369+0000","last_became_peered":"2023-11-26T07:21:56.306369+0000","last_unstale":"2023-11-26T07:21:56.312862+0000","last_undegraded":"2023-11-26T07:21:56.312862+0000","last_fullsized":"2023-11-26T07:21:56.312862+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:21:55.264078+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:21:55.264078+0000","last_clean_scrub_stamp":"2023-11-26T07:21:55.264078+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T15:20:41.424795+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.33","version":"0'0","reported_seq":9,"reported_epoch":58,"state":"active+clean","last_fresh":"2023-11-26T07:21:56.310948+0000","last_change":"2023-11-26T07:21:56.301597+0000","last_active":"2023-11-26T07:21:56.310948+0000","last_peered":"2023-11-26T07:21:56.310948+0000","last_clean":"2023-11-26T07:21:56.310948+0000","last_became_active":"2023-11-26T07:21:56.301265+0000","last_became_peered":"2023-11-26T07:21:56.301265+0000","last_unstale":"2023-11-26T07:21:56.310948+0000","last_undegraded":"2023-11-26T07:21:56.310948+0000","last_fullsized":"2023-11-26T07:21:56.310948+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:21:55.264078+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:21:55.264078+0000","last_clean_scrub_stamp":"2023-11-26T07:21:55.264078+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T07:44:25.322195+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.32","version":"0'0","reported_seq":9,"reported_epoch":58,"state":"active+clean","last_fresh":"2023-11-26T07:21:56.307644+0000","last_change":"2023-11-26T07:21:56.298573+0000","last_active":"2023-11-26T07:21:56.307644+0000","last_peered":"2023-11-26T07:21:56.307644+0000","last_clean":"2023-11-26T07:21:56.307644+0000","last_became_active":"2023-11-26T07:21:56.298282+0000","last_became_peered":"2023-11-26T07:21:56.298282+0000","last_unstale":"2023-11-26T07:21:56.307644+0000","last_undegraded":"2023-11-26T07:21:56.307644+0000","last_fullsized":"2023-11-26T07:21:56.307644+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:21:55.264078+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:21:55.264078+0000","last_clean_scrub_stamp":"2023-11-26T07:21:55.264078+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T10:27:44.765406+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.31","version":"0'0","reported_seq":9,"reported_epoch":58,"state":"active+clean","last_fresh":"2023-11-26T07:21:56.307734+0000","last_change":"2023-11-26T07:21:56.299019+0000","last_active":"2023-11-26T07:21:56.307734+0000","last_peered":"2023-11-26T07:21:56.307734+0000","last_clean":"2023-11-26T07:21:56.307734+0000","last_became_active":"2023-11-26T07:21:56.298693+0000","last_became_peered":"2023-11-26T07:21:56.298693+0000","last_unstale":"2023-11-26T07:21:56.307734+0000","last_undegraded":"2023-11-26T07:21:56.307734+0000","last_fullsized":"2023-11-26T07:21:56.307734+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:21:55.264078+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:21:55.264078+0000","last_clean_scrub_stamp":"2023-11-26T07:21:55.264078+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T12:10:56.964250+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.30","version":"0'0","reported_seq":9,"reported_epoch":58,"state":"active+clean","last_fresh":"2023-11-26T07:21:56.301573+0000","last_change":"2023-11-26T07:21:56.292573+0000","last_active":"2023-11-26T07:21:56.301573+0000","last_peered":"2023-11-26T07:21:56.301573+0000","last_clean":"2023-11-26T07:21:56.301573+0000","last_became_active":"2023-11-26T07:21:56.291812+0000","last_became_peered":"2023-11-26T07:21:56.291812+0000","last_unstale":"2023-11-26T07:21:56.301573+0000","last_undegraded":"2023-11-26T07:21:56.301573+0000","last_fullsized":"2023-11-26T07:21:56.301573+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:21:55.264078+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:21:55.264078+0000","last_clean_scrub_stamp":"2023-11-26T07:21:55.264078+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T08:06:02.760482+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.2f","version":"0'0","reported_seq":9,"reported_epoch":58,"state":"active+clean","last_fresh":"2023-11-26T07:21:56.302731+0000","last_change":"2023-11-26T07:21:56.296668+0000","last_active":"2023-11-26T07:21:56.302731+0000","last_peered":"2023-11-26T07:21:56.302731+0000","last_clean":"2023-11-26T07:21:56.302731+0000","last_became_active":"2023-11-26T07:21:56.296332+0000","last_became_peered":"2023-11-26T07:21:56.296332+0000","last_unstale":"2023-11-26T07:21:56.302731+0000","last_undegraded":"2023-11-26T07:21:56.302731+0000","last_fullsized":"2023-11-26T07:21:56.302731+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:21:55.264078+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:21:55.264078+0000","last_clean_scrub_stamp":"2023-11-26T07:21:55.264078+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T19:16:16.815018+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.2e","version":"0'0","reported_seq":9,"reported_epoch":58,"state":"active+clean","last_fresh":"2023-11-26T07:21:56.309667+0000","last_change":"2023-11-26T07:21:56.301067+0000","last_active":"2023-11-26T07:21:56.309667+0000","last_peered":"2023-11-26T07:21:56.309667+0000","last_clean":"2023-11-26T07:21:56.309667+0000","last_became_active":"2023-11-26T07:21:56.300756+0000","last_became_peered":"2023-11-26T07:21:56.300756+0000","last_unstale":"2023-11-26T07:21:56.309667+0000","last_undegraded":"2023-11-26T07:21:56.309667+0000","last_fullsized":"2023-11-26T07:21:56.309667+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:21:55.264078+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:21:55.264078+0000","last_clean_scrub_stamp":"2023-11-26T07:21:55.264078+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T12:18:44.648658+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.2d","version":"0'0","reported_seq":9,"reported_epoch":58,"state":"active+clean","last_fresh":"2023-11-26T07:21:56.297345+0000","last_change":"2023-11-26T07:21:56.290886+0000","last_active":"2023-11-26T07:21:56.297345+0000","last_peered":"2023-11-26T07:21:56.297345+0000","last_clean":"2023-11-26T07:21:56.297345+0000","last_became_active":"2023-11-26T07:21:56.290612+0000","last_became_peered":"2023-11-26T07:21:56.290612+0000","last_unstale":"2023-11-26T07:21:56.297345+0000","last_undegraded":"2023-11-26T07:21:56.297345+0000","last_fullsized":"2023-11-26T07:21:56.297345+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:21:55.264078+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:21:55.264078+0000","last_clean_scrub_stamp":"2023-11-26T07:21:55.264078+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T14:28:40.362241+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.2c","version":"0'0","reported_seq":9,"reported_epoch":58,"state":"active+clean","last_fresh":"2023-11-26T07:21:56.303054+0000","last_change":"2023-11-26T07:21:56.298126+0000","last_active":"2023-11-26T07:21:56.303054+0000","last_peered":"2023-11-26T07:21:56.303054+0000","last_clean":"2023-11-26T07:21:56.303054+0000","last_became_active":"2023-11-26T07:21:56.297867+0000","last_became_peered":"2023-11-26T07:21:56.297867+0000","last_unstale":"2023-11-26T07:21:56.303054+0000","last_undegraded":"2023-11-26T07:21:56.303054+0000","last_fullsized":"2023-11-26T07:21:56.303054+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:21:55.264078+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:21:55.264078+0000","last_clean_scrub_stamp":"2023-11-26T07:21:55.264078+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T14:55:04.672096+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.2b","version":"0'0","reported_seq":9,"reported_epoch":58,"state":"active+clean","last_fresh":"2023-11-26T07:21:56.314071+0000","last_change":"2023-11-26T07:21:56.309268+0000","last_active":"2023-11-26T07:21:56.314071+0000","last_peered":"2023-11-26T07:21:56.314071+0000","last_clean":"2023-11-26T07:21:56.314071+0000","last_became_active":"2023-11-26T07:21:56.308973+0000","last_became_peered":"2023-11-26T07:21:56.308973+0000","last_unstale":"2023-11-26T07:21:56.314071+0000","last_undegraded":"2023-11-26T07:21:56.314071+0000","last_fullsized":"2023-11-26T07:21:56.314071+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:21:55.264078+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:21:55.264078+0000","last_clean_scrub_stamp":"2023-11-26T07:21:55.264078+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T13:30:25.417889+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.2a","version":"0'0","reported_seq":9,"reported_epoch":58,"state":"active+clean","last_fresh":"2023-11-26T07:21:56.299203+0000","last_change":"2023-11-26T07:21:56.293241+0000","last_active":"2023-11-26T07:21:56.299203+0000","last_peered":"2023-11-26T07:21:56.299203+0000","last_clean":"2023-11-26T07:21:56.299203+0000","last_became_active":"2023-11-26T07:21:56.292578+0000","last_became_peered":"2023-11-26T07:21:56.292578+0000","last_unstale":"2023-11-26T07:21:56.299203+0000","last_undegraded":"2023-11-26T07:21:56.299203+0000","last_fullsized":"2023-11-26T07:21:56.299203+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:21:55.264078+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:21:55.264078+0000","last_clean_scrub_stamp":"2023-11-26T07:21:55.264078+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T19:05:58.460154+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.29","version":"0'0","reported_seq":9,"reported_epoch":58,"state":"active+clean","last_fresh":"2023-11-26T07:21:56.309723+0000","last_change":"2023-11-26T07:21:56.303654+0000","last_active":"2023-11-26T07:21:56.309723+0000","last_peered":"2023-11-26T07:21:56.309723+0000","last_clean":"2023-11-26T07:21:56.309723+0000","last_became_active":"2023-11-26T07:21:56.300036+0000","last_became_peered":"2023-11-26T07:21:56.300036+0000","last_unstale":"2023-11-26T07:21:56.309723+0000","last_undegraded":"2023-11-26T07:21:56.309723+0000","last_fullsized":"2023-11-26T07:21:56.309723+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:21:55.264078+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:21:55.264078+0000","last_clean_scrub_stamp":"2023-11-26T07:21:55.264078+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T13:30:24.283597+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.28","version":"0'0","reported_seq":9,"reported_epoch":58,"state":"active+clean","last_fresh":"2023-11-26T07:21:56.309060+0000","last_change":"2023-11-26T07:21:56.292011+0000","last_active":"2023-11-26T07:21:56.309060+0000","last_peered":"2023-11-26T07:21:56.309060+0000","last_clean":"2023-11-26T07:21:56.309060+0000","last_became_active":"2023-11-26T07:21:56.291728+0000","last_became_peered":"2023-11-26T07:21:56.291728+0000","last_unstale":"2023-11-26T07:21:56.309060+0000","last_undegraded":"2023-11-26T07:21:56.309060+0000","last_fullsized":"2023-11-26T07:21:56.309060+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:21:55.264078+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:21:55.264078+0000","last_clean_scrub_stamp":"2023-11-26T07:21:55.264078+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T12:41:20.887450+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.27","version":"0'0","reported_seq":9,"reported_epoch":58,"state":"active+clean","last_fresh":"2023-11-26T07:21:56.300715+0000","last_change":"2023-11-26T07:21:56.296395+0000","last_active":"2023-11-26T07:21:56.300715+0000","last_peered":"2023-11-26T07:21:56.300715+0000","last_clean":"2023-11-26T07:21:56.300715+0000","last_became_active":"2023-11-26T07:21:56.296077+0000","last_became_peered":"2023-11-26T07:21:56.296077+0000","last_unstale":"2023-11-26T07:21:56.300715+0000","last_undegraded":"2023-11-26T07:21:56.300715+0000","last_fullsized":"2023-11-26T07:21:56.300715+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:21:55.264078+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:21:55.264078+0000","last_clean_scrub_stamp":"2023-11-26T07:21:55.264078+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T13:59:10.978496+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.26","version":"0'0","reported_seq":9,"reported_epoch":58,"state":"active+clean","last_fresh":"2023-11-26T07:21:56.310283+0000","last_change":"2023-11-26T07:21:56.297590+0000","last_active":"2023-11-26T07:21:56.310283+0000","last_peered":"2023-11-26T07:21:56.310283+0000","last_clean":"2023-11-26T07:21:56.310283+0000","last_became_active":"2023-11-26T07:21:56.296279+0000","last_became_peered":"2023-11-26T07:21:56.296279+0000","last_unstale":"2023-11-26T07:21:56.310283+0000","last_undegraded":"2023-11-26T07:21:56.310283+0000","last_fullsized":"2023-11-26T07:21:56.310283+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:21:55.264078+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:21:55.264078+0000","last_clean_scrub_stamp":"2023-11-26T07:21:55.264078+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T19:19:55.894699+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.25","version":"0'0","reported_seq":9,"reported_epoch":58,"state":"active+clean","last_fresh":"2023-11-26T07:21:56.304371+0000","last_change":"2023-11-26T07:21:56.301048+0000","last_active":"2023-11-26T07:21:56.304371+0000","last_peered":"2023-11-26T07:21:56.304371+0000","last_clean":"2023-11-26T07:21:56.304371+0000","last_became_active":"2023-11-26T07:21:56.299240+0000","last_became_peered":"2023-11-26T07:21:56.299240+0000","last_unstale":"2023-11-26T07:21:56.304371+0000","last_undegraded":"2023-11-26T07:21:56.304371+0000","last_fullsized":"2023-11-26T07:21:56.304371+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:21:55.264078+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:21:55.264078+0000","last_clean_scrub_stamp":"2023-11-26T07:21:55.264078+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T10:22:37.629253+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.24","version":"0'0","reported_seq":9,"reported_epoch":58,"state":"active+clean","last_fresh":"2023-11-26T07:21:56.302367+0000","last_change":"2023-11-26T07:21:56.298575+0000","last_active":"2023-11-26T07:21:56.302367+0000","last_peered":"2023-11-26T07:21:56.302367+0000","last_clean":"2023-11-26T07:21:56.302367+0000","last_became_active":"2023-11-26T07:21:56.298312+0000","last_became_peered":"2023-11-26T07:21:56.298312+0000","last_unstale":"2023-11-26T07:21:56.302367+0000","last_undegraded":"2023-11-26T07:21:56.302367+0000","last_fullsized":"2023-11-26T07:21:56.302367+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:21:55.264078+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:21:55.264078+0000","last_clean_scrub_stamp":"2023-11-26T07:21:55.264078+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T17:40:12.547770+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.23","version":"0'0","reported_seq":9,"reported_epoch":58,"state":"active+clean","last_fresh":"2023-11-26T07:21:56.315097+0000","last_change":"2023-11-26T07:21:56.312013+0000","last_active":"2023-11-26T07:21:56.315097+0000","last_peered":"2023-11-26T07:21:56.315097+0000","last_clean":"2023-11-26T07:21:56.315097+0000","last_became_active":"2023-11-26T07:21:56.311699+0000","last_became_peered":"2023-11-26T07:21:56.311699+0000","last_unstale":"2023-11-26T07:21:56.315097+0000","last_undegraded":"2023-11-26T07:21:56.315097+0000","last_fullsized":"2023-11-26T07:21:56.315097+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:21:55.264078+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:21:55.264078+0000","last_clean_scrub_stamp":"2023-11-26T07:21:55.264078+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T12:18:44.648658+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.22","version":"0'0","reported_seq":9,"reported_epoch":58,"state":"active+clean","last_fresh":"2023-11-26T07:21:56.309681+0000","last_change":"2023-11-26T07:21:56.298035+0000","last_active":"2023-11-26T07:21:56.309681+0000","last_peered":"2023-11-26T07:21:56.309681+0000","last_clean":"2023-11-26T07:21:56.309681+0000","last_became_active":"2023-11-26T07:21:56.297613+0000","last_became_peered":"2023-11-26T07:21:56.297613+0000","last_unstale":"2023-11-26T07:21:56.309681+0000","last_undegraded":"2023-11-26T07:21:56.309681+0000","last_fullsized":"2023-11-26T07:21:56.309681+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:21:55.264078+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:21:55.264078+0000","last_clean_scrub_stamp":"2023-11-26T07:21:55.264078+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T10:32:38.427322+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.21","version":"0'0","reported_seq":9,"reported_epoch":58,"state":"active+clean","last_fresh":"2023-11-26T07:21:56.292934+0000","last_change":"2023-11-26T07:21:56.289763+0000","last_active":"2023-11-26T07:21:56.292934+0000","last_peered":"2023-11-26T07:21:56.292934+0000","last_clean":"2023-11-26T07:21:56.292934+0000","last_became_active":"2023-11-26T07:21:56.289468+0000","last_became_peered":"2023-11-26T07:21:56.289468+0000","last_unstale":"2023-11-26T07:21:56.292934+0000","last_undegraded":"2023-11-26T07:21:56.292934+0000","last_fullsized":"2023-11-26T07:21:56.292934+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:21:55.264078+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:21:55.264078+0000","last_clean_scrub_stamp":"2023-11-26T07:21:55.264078+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T16:42:45.532870+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.20","version":"0'0","reported_seq":9,"reported_epoch":58,"state":"active+clean","last_fresh":"2023-11-26T07:21:56.301553+0000","last_change":"2023-11-26T07:21:56.291707+0000","last_active":"2023-11-26T07:21:56.301553+0000","last_peered":"2023-11-26T07:21:56.301553+0000","last_clean":"2023-11-26T07:21:56.301553+0000","last_became_active":"2023-11-26T07:21:56.291419+0000","last_became_peered":"2023-11-26T07:21:56.291419+0000","last_unstale":"2023-11-26T07:21:56.301553+0000","last_undegraded":"2023-11-26T07:21:56.301553+0000","last_fullsized":"2023-11-26T07:21:56.301553+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:21:55.264078+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:21:55.264078+0000","last_clean_scrub_stamp":"2023-11-26T07:21:55.264078+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T09:35:29.250849+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.1e","version":"0'0","reported_seq":9,"reported_epoch":58,"state":"active+clean","last_fresh":"2023-11-26T07:21:56.310315+0000","last_change":"2023-11-26T07:21:56.302129+0000","last_active":"2023-11-26T07:21:56.310315+0000","last_peered":"2023-11-26T07:21:56.310315+0000","last_clean":"2023-11-26T07:21:56.310315+0000","last_became_active":"2023-11-26T07:21:56.301845+0000","last_became_peered":"2023-11-26T07:21:56.301845+0000","last_unstale":"2023-11-26T07:21:56.310315+0000","last_undegraded":"2023-11-26T07:21:56.310315+0000","last_fullsized":"2023-11-26T07:21:56.310315+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:21:55.264078+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:21:55.264078+0000","last_clean_scrub_stamp":"2023-11-26T07:21:55.264078+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T11:54:47.454823+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.1f","version":"58'4","reported_seq":15,"reported_epoch":58,"state":"active+clean","last_fresh":"2023-11-26T07:21:56.319313+0000","last_change":"2023-11-26T07:21:55.298427+0000","last_active":"2023-11-26T07:21:56.319313+0000","last_peered":"2023-11-26T07:21:56.319313+0000","last_clean":"2023-11-26T07:21:56.319313+0000","last_became_active":"2023-11-26T07:21:55.298125+0000","last_became_peered":"2023-11-26T07:21:55.298125+0000","last_unstale":"2023-11-26T07:21:56.319313+0000","last_undegraded":"2023-11-26T07:21:56.319313+0000","last_fullsized":"2023-11-26T07:21:56.319313+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:21:54.255568+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:21:54.255568+0000","last_clean_scrub_stamp":"2023-11-26T07:21:54.255568+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T13:30:23.275087+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.1f","version":"0'0","reported_seq":9,"reported_epoch":58,"state":"active+clean","last_fresh":"2023-11-26T07:21:56.310682+0000","last_change":"2023-11-26T07:21:56.304018+0000","last_active":"2023-11-26T07:21:56.310682+0000","last_peered":"2023-11-26T07:21:56.310682+0000","last_clean":"2023-11-26T07:21:56.310682+0000","last_became_active":"2023-11-26T07:21:56.303695+0000","last_became_peered":"2023-11-26T07:21:56.303695+0000","last_unstale":"2023-11-26T07:21:56.310682+0000","last_undegraded":"2023-11-26T07:21:56.310682+0000","last_fullsized":"2023-11-26T07:21:56.310682+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:21:55.264078+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:21:55.264078+0000","last_clean_scrub_stamp":"2023-11-26T07:21:55.264078+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T17:06:42.726725+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.1e","version":"58'1","reported_seq":12,"reported_epoch":58,"state":"active+clean","last_fresh":"2023-11-26T07:21:56.314736+0000","last_change":"2023-11-26T07:21:55.299423+0000","last_active":"2023-11-26T07:21:56.314736+0000","last_peered":"2023-11-26T07:21:56.314736+0000","last_clean":"2023-11-26T07:21:56.314736+0000","last_became_active":"2023-11-26T07:21:55.299124+0000","last_became_peered":"2023-11-26T07:21:55.299124+0000","last_unstale":"2023-11-26T07:21:56.314736+0000","last_undegraded":"2023-11-26T07:21:56.314736+0000","last_fullsized":"2023-11-26T07:21:56.314736+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:21:54.255568+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:21:54.255568+0000","last_clean_scrub_stamp":"2023-11-26T07:21:54.255568+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T16:56:28.898546+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.1c","version":"0'0","reported_seq":9,"reported_epoch":58,"state":"active+clean","last_fresh":"2023-11-26T07:21:56.301667+0000","last_change":"2023-11-26T07:21:56.294816+0000","last_active":"2023-11-26T07:21:56.301667+0000","last_peered":"2023-11-26T07:21:56.301667+0000","last_clean":"2023-11-26T07:21:56.301667+0000","last_became_active":"2023-11-26T07:21:56.294549+0000","last_became_peered":"2023-11-26T07:21:56.294549+0000","last_unstale":"2023-11-26T07:21:56.301667+0000","last_undegraded":"2023-11-26T07:21:56.301667+0000","last_fullsized":"2023-11-26T07:21:56.301667+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:21:55.264078+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:21:55.264078+0000","last_clean_scrub_stamp":"2023-11-26T07:21:55.264078+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T13:30:25.417889+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.1d","version":"58'2","reported_seq":13,"reported_epoch":58,"state":"active+clean","last_fresh":"2023-11-26T07:21:56.316221+0000","last_change":"2023-11-26T07:21:55.296693+0000","last_active":"2023-11-26T07:21:56.316221+0000","last_peered":"2023-11-26T07:21:56.316221+0000","last_clean":"2023-11-26T07:21:56.316221+0000","last_became_active":"2023-11-26T07:21:55.296389+0000","last_became_peered":"2023-11-26T07:21:55.296389+0000","last_unstale":"2023-11-26T07:21:56.316221+0000","last_undegraded":"2023-11-26T07:21:56.316221+0000","last_fullsized":"2023-11-26T07:21:56.316221+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:21:54.255568+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:21:54.255568+0000","last_clean_scrub_stamp":"2023-11-26T07:21:54.255568+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T11:09:01.721510+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.1d","version":"0'0","reported_seq":9,"reported_epoch":58,"state":"active+clean","last_fresh":"2023-11-26T07:21:56.304263+0000","last_change":"2023-11-26T07:21:56.299896+0000","last_active":"2023-11-26T07:21:56.304263+0000","last_peered":"2023-11-26T07:21:56.304263+0000","last_clean":"2023-11-26T07:21:56.304263+0000","last_became_active":"2023-11-26T07:21:56.299617+0000","last_became_peered":"2023-11-26T07:21:56.299617+0000","last_unstale":"2023-11-26T07:21:56.304263+0000","last_undegraded":"2023-11-26T07:21:56.304263+0000","last_fullsized":"2023-11-26T07:21:56.304263+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:21:55.264078+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:21:55.264078+0000","last_clean_scrub_stamp":"2023-11-26T07:21:55.264078+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T08:10:58.660224+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.1c","version":"58'2","reported_seq":13,"reported_epoch":58,"state":"active+clean","last_fresh":"2023-11-26T07:21:56.318020+0000","last_change":"2023-11-26T07:21:55.295656+0000","last_active":"2023-11-26T07:21:56.318020+0000","last_peered":"2023-11-26T07:21:56.318020+0000","last_clean":"2023-11-26T07:21:56.318020+0000","last_became_active":"2023-11-26T07:21:55.294764+0000","last_became_peered":"2023-11-26T07:21:55.294764+0000","last_unstale":"2023-11-26T07:21:56.318020+0000","last_undegraded":"2023-11-26T07:21:56.318020+0000","last_fullsized":"2023-11-26T07:21:56.318020+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:21:54.255568+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:21:54.255568+0000","last_clean_scrub_stamp":"2023-11-26T07:21:54.255568+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T09:35:28.242339+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.1a","version":"0'0","reported_seq":9,"reported_epoch":58,"state":"active+clean","last_fresh":"2023-11-26T07:21:56.299184+0000","last_change":"2023-11-26T07:21:56.293138+0000","last_active":"2023-11-26T07:21:56.299184+0000","last_peered":"2023-11-26T07:21:56.299184+0000","last_clean":"2023-11-26T07:21:56.299184+0000","last_became_active":"2023-11-26T07:21:56.292236+0000","last_became_peered":"2023-11-26T07:21:56.292236+0000","last_unstale":"2023-11-26T07:21:56.299184+0000","last_undegraded":"2023-11-26T07:21:56.299184+0000","last_fullsized":"2023-11-26T07:21:56.299184+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:21:55.264078+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:21:55.264078+0000","last_clean_scrub_stamp":"2023-11-26T07:21:55.264078+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T10:56:18.452471+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.1b","version":"58'1","reported_seq":12,"reported_epoch":58,"state":"active+clean","last_fresh":"2023-11-26T07:21:56.315453+0000","last_change":"2023-11-26T07:21:55.286486+0000","last_active":"2023-11-26T07:21:56.315453+0000","last_peered":"2023-11-26T07:21:56.315453+0000","last_clean":"2023-11-26T07:21:56.315453+0000","last_became_active":"2023-11-26T07:21:55.286095+0000","last_became_peered":"2023-11-26T07:21:55.286095+0000","last_unstale":"2023-11-26T07:21:56.315453+0000","last_undegraded":"2023-11-26T07:21:56.315453+0000","last_fullsized":"2023-11-26T07:21:56.315453+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:21:54.255568+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:21:54.255568+0000","last_clean_scrub_stamp":"2023-11-26T07:21:54.255568+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T12:13:26.816359+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.1b","version":"0'0","reported_seq":9,"reported_epoch":58,"state":"active+clean","last_fresh":"2023-11-26T07:21:56.304741+0000","last_change":"2023-11-26T07:21:56.302073+0000","last_active":"2023-11-26T07:21:56.304741+0000","last_peered":"2023-11-26T07:21:56.304741+0000","last_clean":"2023-11-26T07:21:56.304741+0000","last_became_active":"2023-11-26T07:21:56.301822+0000","last_became_peered":"2023-11-26T07:21:56.301822+0000","last_unstale":"2023-11-26T07:21:56.304741+0000","last_undegraded":"2023-11-26T07:21:56.304741+0000","last_fullsized":"2023-11-26T07:21:56.304741+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:21:55.264078+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:21:55.264078+0000","last_clean_scrub_stamp":"2023-11-26T07:21:55.264078+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T10:03:32.111285+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.1a","version":"58'1","reported_seq":12,"reported_epoch":58,"state":"active+clean","last_fresh":"2023-11-26T07:21:56.313461+0000","last_change":"2023-11-26T07:21:55.292401+0000","last_active":"2023-11-26T07:21:56.313461+0000","last_peered":"2023-11-26T07:21:56.313461+0000","last_clean":"2023-11-26T07:21:56.313461+0000","last_became_active":"2023-11-26T07:21:55.292081+0000","last_became_peered":"2023-11-26T07:21:55.292081+0000","last_unstale":"2023-11-26T07:21:56.313461+0000","last_undegraded":"2023-11-26T07:21:56.313461+0000","last_fullsized":"2023-11-26T07:21:56.313461+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:21:54.255568+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:21:54.255568+0000","last_clean_scrub_stamp":"2023-11-26T07:21:54.255568+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T10:56:17.443961+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.18","version":"0'0","reported_seq":9,"reported_epoch":58,"state":"active+clean","last_fresh":"2023-11-26T07:21:56.310785+0000","last_change":"2023-11-26T07:21:56.298980+0000","last_active":"2023-11-26T07:21:56.310785+0000","last_peered":"2023-11-26T07:21:56.310785+0000","last_clean":"2023-11-26T07:21:56.310785+0000","last_became_active":"2023-11-26T07:21:56.298626+0000","last_became_peered":"2023-11-26T07:21:56.298626+0000","last_unstale":"2023-11-26T07:21:56.310785+0000","last_undegraded":"2023-11-26T07:21:56.310785+0000","last_fullsized":"2023-11-26T07:21:56.310785+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:21:55.264078+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:21:55.264078+0000","last_clean_scrub_stamp":"2023-11-26T07:21:55.264078+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T18:59:37.088334+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"8.19","version":"0'0","reported_seq":11,"reported_epoch":58,"state":"active+clean","last_fresh":"2023-11-26T07:21:56.284919+0000","last_change":"2023-11-26T07:21:55.288232+0000","last_active":"2023-11-26T07:21:56.284919+0000","last_peered":"2023-11-26T07:21:56.284919+0000","last_clean":"2023-11-26T07:21:56.284919+0000","last_became_active":"2023-11-26T07:21:55.287513+0000","last_became_peered":"2023-11-26T07:21:55.287513+0000","last_unstale":"2023-11-26T07:21:56.284919+0000","last_undegraded":"2023-11-26T07:21:56.284919+0000","last_fullsized":"2023-11-26T07:21:56.284919+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:21:54.255568+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:21:54.255568+0000","last_clean_scrub_stamp":"2023-11-26T07:21:54.255568+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T13:38:08.175522+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.19","version":"0'0","reported_seq":9,"reported_epoch":58,"state":"active+clean","last_fresh":"2023-11-26T07:21:56.302907+0000","last_change":"2023-11-26T07:21:56.296395+0000","last_active":"2023-11-26T07:21:56.302907+0000","last_peered":"2023-11-26T07:21:56.302907+0000","last_clean":"2023-11-26T07:21:56.302907+0000","last_became_active":"2023-11-26T07:21:56.296121+0000","last_became_peered":"2023-11-26T07:21:56.296121+0000","last_unstale":"2023-11-26T07:21:56.302907+0000","last_undegraded":"2023-11-26T07:21:56.302907+0000","last_fullsized":"2023-11-26T07:21:56.302907+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:21:55.264078+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:21:55.264078+0000","last_clean_scrub_stamp":"2023-11-26T07:21:55.264078+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T13:30:24.283597+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.18","version":"0'0","reported_seq":11,"reported_epoch":58,"state":"active+clean","last_fresh":"2023-11-26T07:21:56.283557+0000","last_change":"2023-11-26T07:21:55.299260+0000","last_active":"2023-11-26T07:21:56.283557+0000","last_peered":"2023-11-26T07:21:56.283557+0000","last_clean":"2023-11-26T07:21:56.283557+0000","last_became_active":"2023-11-26T07:21:55.298821+0000","last_became_peered":"2023-11-26T07:21:55.298821+0000","last_unstale":"2023-11-26T07:21:56.283557+0000","last_undegraded":"2023-11-26T07:21:56.283557+0000","last_fullsized":"2023-11-26T07:21:56.283557+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:21:54.255568+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:21:54.255568+0000","last_clean_scrub_stamp":"2023-11-26T07:21:54.255568+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T15:18:49.985683+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.16","version":"0'0","reported_seq":9,"reported_epoch":58,"state":"active+clean","last_fresh":"2023-11-26T07:21:56.313490+0000","last_change":"2023-11-26T07:21:56.305168+0000","last_active":"2023-11-26T07:21:56.313490+0000","last_peered":"2023-11-26T07:21:56.313490+0000","last_clean":"2023-11-26T07:21:56.313490+0000","last_became_active":"2023-11-26T07:21:56.304849+0000","last_became_peered":"2023-11-26T07:21:56.304849+0000","last_unstale":"2023-11-26T07:21:56.313490+0000","last_undegraded":"2023-11-26T07:21:56.313490+0000","last_fullsized":"2023-11-26T07:21:56.313490+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:21:55.264078+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:21:55.264078+0000","last_clean_scrub_stamp":"2023-11-26T07:21:55.264078+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T09:56:06.708591+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"8.17","version":"0'0","reported_seq":11,"reported_epoch":58,"state":"active+clean","last_fresh":"2023-11-26T07:21:56.284545+0000","last_change":"2023-11-26T07:21:55.302413+0000","last_active":"2023-11-26T07:21:56.284545+0000","last_peered":"2023-11-26T07:21:56.284545+0000","last_clean":"2023-11-26T07:21:56.284545+0000","last_became_active":"2023-11-26T07:21:55.302038+0000","last_became_peered":"2023-11-26T07:21:55.302038+0000","last_unstale":"2023-11-26T07:21:56.284545+0000","last_undegraded":"2023-11-26T07:21:56.284545+0000","last_fullsized":"2023-11-26T07:21:56.284545+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:21:54.255568+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:21:54.255568+0000","last_clean_scrub_stamp":"2023-11-26T07:21:54.255568+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T11:54:46.446314+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.17","version":"0'0","reported_seq":9,"reported_epoch":58,"state":"active+clean","last_fresh":"2023-11-26T07:21:56.310836+0000","last_change":"2023-11-26T07:21:56.306728+0000","last_active":"2023-11-26T07:21:56.310836+0000","last_peered":"2023-11-26T07:21:56.310836+0000","last_clean":"2023-11-26T07:21:56.310836+0000","last_became_active":"2023-11-26T07:21:56.306343+0000","last_became_peered":"2023-11-26T07:21:56.306343+0000","last_unstale":"2023-11-26T07:21:56.310836+0000","last_undegraded":"2023-11-26T07:21:56.310836+0000","last_fullsized":"2023-11-26T07:21:56.310836+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:21:55.264078+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:21:55.264078+0000","last_clean_scrub_stamp":"2023-11-26T07:21:55.264078+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T08:10:58.660224+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.16","version":"0'0","reported_seq":11,"reported_epoch":58,"state":"active+clean","last_fresh":"2023-11-26T07:21:56.285592+0000","last_change":"2023-11-26T07:21:55.297031+0000","last_active":"2023-11-26T07:21:56.285592+0000","last_peered":"2023-11-26T07:21:56.285592+0000","last_clean":"2023-11-26T07:21:56.285592+0000","last_became_active":"2023-11-26T07:21:55.296694+0000","last_became_peered":"2023-11-26T07:21:55.296694+0000","last_unstale":"2023-11-26T07:21:56.285592+0000","last_undegraded":"2023-11-26T07:21:56.285592+0000","last_fullsized":"2023-11-26T07:21:56.285592+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:21:54.255568+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:21:54.255568+0000","last_clean_scrub_stamp":"2023-11-26T07:21:54.255568+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T07:40:23.543416+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.14","version":"0'0","reported_seq":9,"reported_epoch":58,"state":"active+clean","last_fresh":"2023-11-26T07:21:56.302348+0000","last_change":"2023-11-26T07:21:56.299253+0000","last_active":"2023-11-26T07:21:56.302348+0000","last_peered":"2023-11-26T07:21:56.302348+0000","last_clean":"2023-11-26T07:21:56.302348+0000","last_became_active":"2023-11-26T07:21:56.298929+0000","last_became_peered":"2023-11-26T07:21:56.298929+0000","last_unstale":"2023-11-26T07:21:56.302348+0000","last_undegraded":"2023-11-26T07:21:56.302348+0000","last_fullsized":"2023-11-26T07:21:56.302348+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:21:55.264078+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:21:55.264078+0000","last_clean_scrub_stamp":"2023-11-26T07:21:55.264078+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T07:45:52.457913+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.15","version":"58'3","reported_seq":14,"reported_epoch":58,"state":"active+clean","last_fresh":"2023-11-26T07:21:56.315399+0000","last_change":"2023-11-26T07:21:55.302350+0000","last_active":"2023-11-26T07:21:56.315399+0000","last_peered":"2023-11-26T07:21:56.315399+0000","last_clean":"2023-11-26T07:21:56.315399+0000","last_became_active":"2023-11-26T07:21:55.302092+0000","last_became_peered":"2023-11-26T07:21:55.302092+0000","last_unstale":"2023-11-26T07:21:56.315399+0000","last_undegraded":"2023-11-26T07:21:56.315399+0000","last_fullsized":"2023-11-26T07:21:56.315399+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:21:54.255568+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:21:54.255568+0000","last_clean_scrub_stamp":"2023-11-26T07:21:54.255568+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T10:00:06.562625+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.15","version":"0'0","reported_seq":9,"reported_epoch":58,"state":"active+clean","last_fresh":"2023-11-26T07:21:56.312157+0000","last_change":"2023-11-26T07:21:56.301162+0000","last_active":"2023-11-26T07:21:56.312157+0000","last_peered":"2023-11-26T07:21:56.312157+0000","last_clean":"2023-11-26T07:21:56.312157+0000","last_became_active":"2023-11-26T07:21:56.299984+0000","last_became_peered":"2023-11-26T07:21:56.299984+0000","last_unstale":"2023-11-26T07:21:56.312157+0000","last_undegraded":"2023-11-26T07:21:56.312157+0000","last_fullsized":"2023-11-26T07:21:56.312157+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:21:55.264078+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:21:55.264078+0000","last_clean_scrub_stamp":"2023-11-26T07:21:55.264078+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T08:06:02.760482+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.14","version":"58'6","reported_seq":17,"reported_epoch":58,"state":"active+clean","last_fresh":"2023-11-26T07:21:56.318891+0000","last_change":"2023-11-26T07:21:55.288706+0000","last_active":"2023-11-26T07:21:56.318891+0000","last_peered":"2023-11-26T07:21:56.318891+0000","last_clean":"2023-11-26T07:21:56.318891+0000","last_became_active":"2023-11-26T07:21:55.287110+0000","last_became_peered":"2023-11-26T07:21:55.287110+0000","last_unstale":"2023-11-26T07:21:56.318891+0000","last_undegraded":"2023-11-26T07:21:56.318891+0000","last_fullsized":"2023-11-26T07:21:56.318891+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:21:54.255568+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:21:54.255568+0000","last_clean_scrub_stamp":"2023-11-26T07:21:54.255568+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T17:00:32.702011+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.12","version":"0'0","reported_seq":9,"reported_epoch":58,"state":"active+clean","last_fresh":"2023-11-26T07:21:56.314585+0000","last_change":"2023-11-26T07:21:56.302363+0000","last_active":"2023-11-26T07:21:56.314585+0000","last_peered":"2023-11-26T07:21:56.314585+0000","last_clean":"2023-11-26T07:21:56.314585+0000","last_became_active":"2023-11-26T07:21:56.301926+0000","last_became_peered":"2023-11-26T07:21:56.301926+0000","last_unstale":"2023-11-26T07:21:56.314585+0000","last_undegraded":"2023-11-26T07:21:56.314585+0000","last_fullsized":"2023-11-26T07:21:56.314585+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:21:55.264078+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:21:55.264078+0000","last_clean_scrub_stamp":"2023-11-26T07:21:55.264078+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T14:21:21.645228+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.13","version":"58'2","reported_seq":13,"reported_epoch":58,"state":"active+clean","last_fresh":"2023-11-26T07:21:56.315041+0000","last_change":"2023-11-26T07:21:55.297366+0000","last_active":"2023-11-26T07:21:56.315041+0000","last_peered":"2023-11-26T07:21:56.315041+0000","last_clean":"2023-11-26T07:21:56.315041+0000","last_became_active":"2023-11-26T07:21:55.297074+0000","last_became_peered":"2023-11-26T07:21:55.297074+0000","last_unstale":"2023-11-26T07:21:56.315041+0000","last_undegraded":"2023-11-26T07:21:56.315041+0000","last_fullsized":"2023-11-26T07:21:56.315041+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:21:54.255568+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:21:54.255568+0000","last_clean_scrub_stamp":"2023-11-26T07:21:54.255568+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T17:19:44.854745+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.13","version":"0'0","reported_seq":9,"reported_epoch":58,"state":"active+clean","last_fresh":"2023-11-26T07:21:56.309729+0000","last_change":"2023-11-26T07:21:56.303041+0000","last_active":"2023-11-26T07:21:56.309729+0000","last_peered":"2023-11-26T07:21:56.309729+0000","last_clean":"2023-11-26T07:21:56.309729+0000","last_became_active":"2023-11-26T07:21:56.302706+0000","last_became_peered":"2023-11-26T07:21:56.302706+0000","last_unstale":"2023-11-26T07:21:56.309729+0000","last_undegraded":"2023-11-26T07:21:56.309729+0000","last_fullsized":"2023-11-26T07:21:56.309729+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:21:55.264078+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:21:55.264078+0000","last_clean_scrub_stamp":"2023-11-26T07:21:55.264078+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T08:28:48.252754+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.12","version":"0'0","reported_seq":11,"reported_epoch":58,"state":"active+clean","last_fresh":"2023-11-26T07:21:56.277947+0000","last_change":"2023-11-26T07:21:55.295331+0000","last_active":"2023-11-26T07:21:56.277947+0000","last_peered":"2023-11-26T07:21:56.277947+0000","last_clean":"2023-11-26T07:21:56.277947+0000","last_became_active":"2023-11-26T07:21:55.295043+0000","last_became_peered":"2023-11-26T07:21:55.295043+0000","last_unstale":"2023-11-26T07:21:56.277947+0000","last_undegraded":"2023-11-26T07:21:56.277947+0000","last_fullsized":"2023-11-26T07:21:56.277947+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:21:54.255568+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:21:54.255568+0000","last_clean_scrub_stamp":"2023-11-26T07:21:54.255568+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T07:45:51.449403+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.10","version":"0'0","reported_seq":9,"reported_epoch":58,"state":"active+clean","last_fresh":"2023-11-26T07:21:56.313476+0000","last_change":"2023-11-26T07:21:56.300018+0000","last_active":"2023-11-26T07:21:56.313476+0000","last_peered":"2023-11-26T07:21:56.313476+0000","last_clean":"2023-11-26T07:21:56.313476+0000","last_became_active":"2023-11-26T07:21:56.299715+0000","last_became_peered":"2023-11-26T07:21:56.299715+0000","last_unstale":"2023-11-26T07:21:56.313476+0000","last_undegraded":"2023-11-26T07:21:56.313476+0000","last_fullsized":"2023-11-26T07:21:56.313476+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:21:55.264078+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:21:55.264078+0000","last_clean_scrub_stamp":"2023-11-26T07:21:55.264078+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T15:06:34.387124+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"8.11","version":"0'0","reported_seq":11,"reported_epoch":58,"state":"active+clean","last_fresh":"2023-11-26T07:21:56.285066+0000","last_change":"2023-11-26T07:21:55.298972+0000","last_active":"2023-11-26T07:21:56.285066+0000","last_peered":"2023-11-26T07:21:56.285066+0000","last_clean":"2023-11-26T07:21:56.285066+0000","last_became_active":"2023-11-26T07:21:55.297927+0000","last_became_peered":"2023-11-26T07:21:55.297927+0000","last_unstale":"2023-11-26T07:21:56.285066+0000","last_undegraded":"2023-11-26T07:21:56.285066+0000","last_fullsized":"2023-11-26T07:21:56.285066+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:21:54.255568+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:21:54.255568+0000","last_clean_scrub_stamp":"2023-11-26T07:21:54.255568+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T10:32:37.418812+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.11","version":"0'0","reported_seq":9,"reported_epoch":58,"state":"active+clean","last_fresh":"2023-11-26T07:21:56.309741+0000","last_change":"2023-11-26T07:21:56.304278+0000","last_active":"2023-11-26T07:21:56.309741+0000","last_peered":"2023-11-26T07:21:56.309741+0000","last_clean":"2023-11-26T07:21:56.309741+0000","last_became_active":"2023-11-26T07:21:56.303968+0000","last_became_peered":"2023-11-26T07:21:56.303968+0000","last_unstale":"2023-11-26T07:21:56.309741+0000","last_undegraded":"2023-11-26T07:21:56.309741+0000","last_fullsized":"2023-11-26T07:21:56.309741+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:21:55.264078+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:21:55.264078+0000","last_clean_scrub_stamp":"2023-11-26T07:21:55.264078+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T15:11:39.329445+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.10","version":"58'8","reported_seq":19,"reported_epoch":58,"state":"active+clean","last_fresh":"2023-11-26T07:21:56.318625+0000","last_change":"2023-11-26T07:21:55.303796+0000","last_active":"2023-11-26T07:21:56.318625+0000","last_peered":"2023-11-26T07:21:56.318625+0000","last_clean":"2023-11-26T07:21:56.318625+0000","last_became_active":"2023-11-26T07:21:55.303540+0000","last_became_peered":"2023-11-26T07:21:55.303540+0000","last_unstale":"2023-11-26T07:21:56.318625+0000","last_undegraded":"2023-11-26T07:21:56.318625+0000","last_fullsized":"2023-11-26T07:21:56.318625+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:21:54.255568+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:21:54.255568+0000","last_clean_scrub_stamp":"2023-11-26T07:21:54.255568+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T10:22:36.620743+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.e","version":"0'0","reported_seq":9,"reported_epoch":58,"state":"active+clean","last_fresh":"2023-11-26T07:21:56.310750+0000","last_change":"2023-11-26T07:21:56.305051+0000","last_active":"2023-11-26T07:21:56.310750+0000","last_peered":"2023-11-26T07:21:56.310750+0000","last_clean":"2023-11-26T07:21:56.310750+0000","last_became_active":"2023-11-26T07:21:56.304788+0000","last_became_peered":"2023-11-26T07:21:56.304788+0000","last_unstale":"2023-11-26T07:21:56.310750+0000","last_undegraded":"2023-11-26T07:21:56.310750+0000","last_fullsized":"2023-11-26T07:21:56.310750+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:21:55.264078+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:21:55.264078+0000","last_clean_scrub_stamp":"2023-11-26T07:21:55.264078+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T11:09:02.730019+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.f","version":"58'1","reported_seq":12,"reported_epoch":58,"state":"active+clean","last_fresh":"2023-11-26T07:21:56.314900+0000","last_change":"2023-11-26T07:21:55.299931+0000","last_active":"2023-11-26T07:21:56.314900+0000","last_peered":"2023-11-26T07:21:56.314900+0000","last_clean":"2023-11-26T07:21:56.314900+0000","last_became_active":"2023-11-26T07:21:55.299643+0000","last_became_peered":"2023-11-26T07:21:55.299643+0000","last_unstale":"2023-11-26T07:21:56.314900+0000","last_undegraded":"2023-11-26T07:21:56.314900+0000","last_fullsized":"2023-11-26T07:21:56.314900+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:21:54.255568+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:21:54.255568+0000","last_clean_scrub_stamp":"2023-11-26T07:21:54.255568+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T19:14:16.437130+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.f","version":"0'0","reported_seq":9,"reported_epoch":58,"state":"active+clean","last_fresh":"2023-11-26T07:21:56.311823+0000","last_change":"2023-11-26T07:21:56.301020+0000","last_active":"2023-11-26T07:21:56.311823+0000","last_peered":"2023-11-26T07:21:56.311823+0000","last_clean":"2023-11-26T07:21:56.311823+0000","last_became_active":"2023-11-26T07:21:56.300756+0000","last_became_peered":"2023-11-26T07:21:56.300756+0000","last_unstale":"2023-11-26T07:21:56.311823+0000","last_undegraded":"2023-11-26T07:21:56.311823+0000","last_fullsized":"2023-11-26T07:21:56.311823+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:21:55.264078+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:21:55.264078+0000","last_clean_scrub_stamp":"2023-11-26T07:21:55.264078+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T09:18:26.215842+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.e","version":"58'2","reported_seq":13,"reported_epoch":58,"state":"active+clean","last_fresh":"2023-11-26T07:21:56.314229+0000","last_change":"2023-11-26T07:21:55.295052+0000","last_active":"2023-11-26T07:21:56.314229+0000","last_peered":"2023-11-26T07:21:56.314229+0000","last_clean":"2023-11-26T07:21:56.314229+0000","last_became_active":"2023-11-26T07:21:55.294786+0000","last_became_peered":"2023-11-26T07:21:55.294786+0000","last_unstale":"2023-11-26T07:21:56.314229+0000","last_undegraded":"2023-11-26T07:21:56.314229+0000","last_fullsized":"2023-11-26T07:21:56.314229+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:21:54.255568+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:21:54.255568+0000","last_clean_scrub_stamp":"2023-11-26T07:21:54.255568+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T15:11:38.320935+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.c","version":"0'0","reported_seq":9,"reported_epoch":58,"state":"active+clean","last_fresh":"2023-11-26T07:21:56.302708+0000","last_change":"2023-11-26T07:21:56.295721+0000","last_active":"2023-11-26T07:21:56.302708+0000","last_peered":"2023-11-26T07:21:56.302708+0000","last_clean":"2023-11-26T07:21:56.302708+0000","last_became_active":"2023-11-26T07:21:56.294569+0000","last_became_peered":"2023-11-26T07:21:56.294569+0000","last_unstale":"2023-11-26T07:21:56.302708+0000","last_undegraded":"2023-11-26T07:21:56.302708+0000","last_fullsized":"2023-11-26T07:21:56.302708+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:21:55.264078+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:21:55.264078+0000","last_clean_scrub_stamp":"2023-11-26T07:21:55.264078+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T18:00:54.173424+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.d","version":"58'1","reported_seq":12,"reported_epoch":58,"state":"active+clean","last_fresh":"2023-11-26T07:21:56.318099+0000","last_change":"2023-11-26T07:21:55.292563+0000","last_active":"2023-11-26T07:21:56.318099+0000","last_peered":"2023-11-26T07:21:56.318099+0000","last_clean":"2023-11-26T07:21:56.318099+0000","last_became_active":"2023-11-26T07:21:55.292240+0000","last_became_peered":"2023-11-26T07:21:55.292240+0000","last_unstale":"2023-11-26T07:21:56.318099+0000","last_undegraded":"2023-11-26T07:21:56.318099+0000","last_fullsized":"2023-11-26T07:21:56.318099+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:21:54.255568+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:21:54.255568+0000","last_clean_scrub_stamp":"2023-11-26T07:21:54.255568+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T08:06:01.751972+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.d","version":"0'0","reported_seq":9,"reported_epoch":58,"state":"active+clean","last_fresh":"2023-11-26T07:21:56.312174+0000","last_change":"2023-11-26T07:21:56.307531+0000","last_active":"2023-11-26T07:21:56.312174+0000","last_peered":"2023-11-26T07:21:56.312174+0000","last_clean":"2023-11-26T07:21:56.312174+0000","last_became_active":"2023-11-26T07:21:56.307217+0000","last_became_peered":"2023-11-26T07:21:56.307217+0000","last_unstale":"2023-11-26T07:21:56.312174+0000","last_undegraded":"2023-11-26T07:21:56.312174+0000","last_fullsized":"2023-11-26T07:21:56.312174+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:21:55.264078+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:21:55.264078+0000","last_clean_scrub_stamp":"2023-11-26T07:21:55.264078+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T19:11:16.774662+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.c","version":"0'0","reported_seq":11,"reported_epoch":58,"state":"active+clean","last_fresh":"2023-11-26T07:21:56.282330+0000","last_change":"2023-11-26T07:21:55.297889+0000","last_active":"2023-11-26T07:21:56.282330+0000","last_peered":"2023-11-26T07:21:56.282330+0000","last_clean":"2023-11-26T07:21:56.282330+0000","last_became_active":"2023-11-26T07:21:55.297620+0000","last_became_peered":"2023-11-26T07:21:55.297620+0000","last_unstale":"2023-11-26T07:21:56.282330+0000","last_undegraded":"2023-11-26T07:21:56.282330+0000","last_fullsized":"2023-11-26T07:21:56.282330+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:21:54.255568+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:21:54.255568+0000","last_clean_scrub_stamp":"2023-11-26T07:21:54.255568+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T19:11:15.766152+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.a","version":"0'0","reported_seq":9,"reported_epoch":58,"state":"active+clean","last_fresh":"2023-11-26T07:21:56.314565+0000","last_change":"2023-11-26T07:21:56.302195+0000","last_active":"2023-11-26T07:21:56.314565+0000","last_peered":"2023-11-26T07:21:56.314565+0000","last_clean":"2023-11-26T07:21:56.314565+0000","last_became_active":"2023-11-26T07:21:56.301576+0000","last_became_peered":"2023-11-26T07:21:56.301576+0000","last_unstale":"2023-11-26T07:21:56.314565+0000","last_undegraded":"2023-11-26T07:21:56.314565+0000","last_fullsized":"2023-11-26T07:21:56.314565+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:21:55.264078+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:21:55.264078+0000","last_clean_scrub_stamp":"2023-11-26T07:21:55.264078+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T10:27:44.765406+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.b","version":"58'2","reported_seq":13,"reported_epoch":58,"state":"active+clean","last_fresh":"2023-11-26T07:21:56.316778+0000","last_change":"2023-11-26T07:21:55.299504+0000","last_active":"2023-11-26T07:21:56.316778+0000","last_peered":"2023-11-26T07:21:56.316778+0000","last_clean":"2023-11-26T07:21:56.316778+0000","last_became_active":"2023-11-26T07:21:55.299171+0000","last_became_peered":"2023-11-26T07:21:55.299171+0000","last_unstale":"2023-11-26T07:21:56.316778+0000","last_undegraded":"2023-11-26T07:21:56.316778+0000","last_fullsized":"2023-11-26T07:21:56.316778+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:21:54.255568+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:21:54.255568+0000","last_clean_scrub_stamp":"2023-11-26T07:21:54.255568+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T16:39:35.277308+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.b","version":"0'0","reported_seq":9,"reported_epoch":58,"state":"active+clean","last_fresh":"2023-11-26T07:21:56.304317+0000","last_change":"2023-11-26T07:21:56.300647+0000","last_active":"2023-11-26T07:21:56.304317+0000","last_peered":"2023-11-26T07:21:56.304317+0000","last_clean":"2023-11-26T07:21:56.304317+0000","last_became_active":"2023-11-26T07:21:56.296266+0000","last_became_peered":"2023-11-26T07:21:56.296266+0000","last_unstale":"2023-11-26T07:21:56.304317+0000","last_undegraded":"2023-11-26T07:21:56.304317+0000","last_fullsized":"2023-11-26T07:21:56.304317+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:21:55.264078+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:21:55.264078+0000","last_clean_scrub_stamp":"2023-11-26T07:21:55.264078+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T12:18:44.648658+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.a","version":"0'0","reported_seq":11,"reported_epoch":58,"state":"active+clean","last_fresh":"2023-11-26T07:21:56.284049+0000","last_change":"2023-11-26T07:21:55.285429+0000","last_active":"2023-11-26T07:21:56.284049+0000","last_peered":"2023-11-26T07:21:56.284049+0000","last_clean":"2023-11-26T07:21:56.284049+0000","last_became_active":"2023-11-26T07:21:55.285087+0000","last_became_peered":"2023-11-26T07:21:55.285087+0000","last_unstale":"2023-11-26T07:21:56.284049+0000","last_undegraded":"2023-11-26T07:21:56.284049+0000","last_fullsized":"2023-11-26T07:21:56.284049+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:21:54.255568+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:21:54.255568+0000","last_clean_scrub_stamp":"2023-11-26T07:21:54.255568+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T07:44:24.313686+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.9","version":"0'0","reported_seq":9,"reported_epoch":58,"state":"active+clean","last_fresh":"2023-11-26T07:21:56.307885+0000","last_change":"2023-11-26T07:21:56.301561+0000","last_active":"2023-11-26T07:21:56.307885+0000","last_peered":"2023-11-26T07:21:56.307885+0000","last_clean":"2023-11-26T07:21:56.307885+0000","last_became_active":"2023-11-26T07:21:56.301217+0000","last_became_peered":"2023-11-26T07:21:56.301217+0000","last_unstale":"2023-11-26T07:21:56.307885+0000","last_undegraded":"2023-11-26T07:21:56.307885+0000","last_fullsized":"2023-11-26T07:21:56.307885+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:21:55.264078+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:21:55.264078+0000","last_clean_scrub_stamp":"2023-11-26T07:21:55.264078+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T08:28:48.252754+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.8","version":"0'0","reported_seq":11,"reported_epoch":58,"state":"active+clean","last_fresh":"2023-11-26T07:21:56.283515+0000","last_change":"2023-11-26T07:21:55.298840+0000","last_active":"2023-11-26T07:21:56.283515+0000","last_peered":"2023-11-26T07:21:56.283515+0000","last_clean":"2023-11-26T07:21:56.283515+0000","last_became_active":"2023-11-26T07:21:55.297942+0000","last_became_peered":"2023-11-26T07:21:55.297942+0000","last_unstale":"2023-11-26T07:21:56.283515+0000","last_undegraded":"2023-11-26T07:21:56.283515+0000","last_fullsized":"2023-11-26T07:21:56.283515+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:21:54.255568+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:21:54.255568+0000","last_clean_scrub_stamp":"2023-11-26T07:21:54.255568+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T13:02:38.473374+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.6","version":"0'0","reported_seq":9,"reported_epoch":58,"state":"active+clean","last_fresh":"2023-11-26T07:21:56.312471+0000","last_change":"2023-11-26T07:21:56.305431+0000","last_active":"2023-11-26T07:21:56.312471+0000","last_peered":"2023-11-26T07:21:56.312471+0000","last_clean":"2023-11-26T07:21:56.312471+0000","last_became_active":"2023-11-26T07:21:56.305100+0000","last_became_peered":"2023-11-26T07:21:56.305100+0000","last_unstale":"2023-11-26T07:21:56.312471+0000","last_undegraded":"2023-11-26T07:21:56.312471+0000","last_fullsized":"2023-11-26T07:21:56.312471+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:21:55.264078+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:21:55.264078+0000","last_clean_scrub_stamp":"2023-11-26T07:21:55.264078+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T08:28:48.583712+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.7","version":"58'2","reported_seq":13,"reported_epoch":58,"state":"active+clean","last_fresh":"2023-11-26T07:21:56.315445+0000","last_change":"2023-11-26T07:21:55.298487+0000","last_active":"2023-11-26T07:21:56.315445+0000","last_peered":"2023-11-26T07:21:56.315445+0000","last_clean":"2023-11-26T07:21:56.315445+0000","last_became_active":"2023-11-26T07:21:55.298222+0000","last_became_peered":"2023-11-26T07:21:55.298222+0000","last_unstale":"2023-11-26T07:21:56.315445+0000","last_undegraded":"2023-11-26T07:21:56.315445+0000","last_fullsized":"2023-11-26T07:21:56.315445+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:21:54.255568+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:21:54.255568+0000","last_clean_scrub_stamp":"2023-11-26T07:21:54.255568+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T15:18:49.985683+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.7","version":"0'0","reported_seq":9,"reported_epoch":58,"state":"active+clean","last_fresh":"2023-11-26T07:21:56.314603+0000","last_change":"2023-11-26T07:21:56.306517+0000","last_active":"2023-11-26T07:21:56.314603+0000","last_peered":"2023-11-26T07:21:56.314603+0000","last_clean":"2023-11-26T07:21:56.314603+0000","last_became_active":"2023-11-26T07:21:56.306225+0000","last_became_peered":"2023-11-26T07:21:56.306225+0000","last_unstale":"2023-11-26T07:21:56.314603+0000","last_undegraded":"2023-11-26T07:21:56.314603+0000","last_fullsized":"2023-11-26T07:21:56.314603+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:21:55.264078+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:21:55.264078+0000","last_clean_scrub_stamp":"2023-11-26T07:21:55.264078+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T15:10:29.063723+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.6","version":"0'0","reported_seq":11,"reported_epoch":58,"state":"active+clean","last_fresh":"2023-11-26T07:21:56.283407+0000","last_change":"2023-11-26T07:21:55.293050+0000","last_active":"2023-11-26T07:21:56.283407+0000","last_peered":"2023-11-26T07:21:56.283407+0000","last_clean":"2023-11-26T07:21:56.283407+0000","last_became_active":"2023-11-26T07:21:55.292351+0000","last_became_peered":"2023-11-26T07:21:55.292351+0000","last_unstale":"2023-11-26T07:21:56.283407+0000","last_undegraded":"2023-11-26T07:21:56.283407+0000","last_fullsized":"2023-11-26T07:21:56.283407+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:21:54.255568+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:21:54.255568+0000","last_clean_scrub_stamp":"2023-11-26T07:21:54.255568+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T10:03:31.102775+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.4","version":"0'0","reported_seq":9,"reported_epoch":58,"state":"active+clean","last_fresh":"2023-11-26T07:21:56.301571+0000","last_change":"2023-11-26T07:21:56.295014+0000","last_active":"2023-11-26T07:21:56.301571+0000","last_peered":"2023-11-26T07:21:56.301571+0000","last_clean":"2023-11-26T07:21:56.301571+0000","last_became_active":"2023-11-26T07:21:56.294562+0000","last_became_peered":"2023-11-26T07:21:56.294562+0000","last_unstale":"2023-11-26T07:21:56.301571+0000","last_undegraded":"2023-11-26T07:21:56.301571+0000","last_fullsized":"2023-11-26T07:21:56.301571+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:21:55.264078+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:21:55.264078+0000","last_clean_scrub_stamp":"2023-11-26T07:21:55.264078+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T12:41:20.887450+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.5","version":"0'0","reported_seq":11,"reported_epoch":58,"state":"active+clean","last_fresh":"2023-11-26T07:21:56.278026+0000","last_change":"2023-11-26T07:21:55.292346+0000","last_active":"2023-11-26T07:21:56.278026+0000","last_peered":"2023-11-26T07:21:56.278026+0000","last_clean":"2023-11-26T07:21:56.278026+0000","last_became_active":"2023-11-26T07:21:55.292035+0000","last_became_peered":"2023-11-26T07:21:55.292035+0000","last_unstale":"2023-11-26T07:21:56.278026+0000","last_undegraded":"2023-11-26T07:21:56.278026+0000","last_fullsized":"2023-11-26T07:21:56.278026+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:21:54.255568+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:21:54.255568+0000","last_clean_scrub_stamp":"2023-11-26T07:21:54.255568+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T12:10:55.955741+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.5","version":"0'0","reported_seq":9,"reported_epoch":58,"state":"active+clean","last_fresh":"2023-11-26T07:21:56.313014+0000","last_change":"2023-11-26T07:21:56.307591+0000","last_active":"2023-11-26T07:21:56.313014+0000","last_peered":"2023-11-26T07:21:56.313014+0000","last_clean":"2023-11-26T07:21:56.313014+0000","last_became_active":"2023-11-26T07:21:56.307309+0000","last_became_peered":"2023-11-26T07:21:56.307309+0000","last_unstale":"2023-11-26T07:21:56.313014+0000","last_undegraded":"2023-11-26T07:21:56.313014+0000","last_fullsized":"2023-11-26T07:21:56.313014+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:21:55.264078+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:21:55.264078+0000","last_clean_scrub_stamp":"2023-11-26T07:21:55.264078+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T10:00:07.571135+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.4","version":"0'0","reported_seq":11,"reported_epoch":58,"state":"active+clean","last_fresh":"2023-11-26T07:21:56.286622+0000","last_change":"2023-11-26T07:21:55.292520+0000","last_active":"2023-11-26T07:21:56.286622+0000","last_peered":"2023-11-26T07:21:56.286622+0000","last_clean":"2023-11-26T07:21:56.286622+0000","last_became_active":"2023-11-26T07:21:55.292203+0000","last_became_peered":"2023-11-26T07:21:55.292203+0000","last_unstale":"2023-11-26T07:21:56.286622+0000","last_undegraded":"2023-11-26T07:21:56.286622+0000","last_fullsized":"2023-11-26T07:21:56.286622+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:21:54.255568+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:21:54.255568+0000","last_clean_scrub_stamp":"2023-11-26T07:21:54.255568+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T13:30:24.409380+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.2","version":"0'0","reported_seq":9,"reported_epoch":58,"state":"active+clean","last_fresh":"2023-11-26T07:21:56.299219+0000","last_change":"2023-11-26T07:21:56.293314+0000","last_active":"2023-11-26T07:21:56.299219+0000","last_peered":"2023-11-26T07:21:56.299219+0000","last_clean":"2023-11-26T07:21:56.299219+0000","last_became_active":"2023-11-26T07:21:56.292857+0000","last_became_peered":"2023-11-26T07:21:56.292857+0000","last_unstale":"2023-11-26T07:21:56.299219+0000","last_undegraded":"2023-11-26T07:21:56.299219+0000","last_fullsized":"2023-11-26T07:21:56.299219+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:21:55.264078+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:21:55.264078+0000","last_clean_scrub_stamp":"2023-11-26T07:21:55.264078+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T09:31:12.063590+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.3","version":"0'0","reported_seq":11,"reported_epoch":58,"state":"active+clean","last_fresh":"2023-11-26T07:21:56.284050+0000","last_change":"2023-11-26T07:21:55.293825+0000","last_active":"2023-11-26T07:21:56.284050+0000","last_peered":"2023-11-26T07:21:56.284050+0000","last_clean":"2023-11-26T07:21:56.284050+0000","last_became_active":"2023-11-26T07:21:55.293536+0000","last_became_peered":"2023-11-26T07:21:55.293536+0000","last_unstale":"2023-11-26T07:21:56.284050+0000","last_undegraded":"2023-11-26T07:21:56.284050+0000","last_fullsized":"2023-11-26T07:21:56.284050+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:21:54.255568+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:21:54.255568+0000","last_clean_scrub_stamp":"2023-11-26T07:21:54.255568+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T18:59:36.079825+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.3","version":"0'0","reported_seq":9,"reported_epoch":58,"state":"active+clean","last_fresh":"2023-11-26T07:21:56.305354+0000","last_change":"2023-11-26T07:21:56.296316+0000","last_active":"2023-11-26T07:21:56.305354+0000","last_peered":"2023-11-26T07:21:56.305354+0000","last_clean":"2023-11-26T07:21:56.305354+0000","last_became_active":"2023-11-26T07:21:56.295949+0000","last_became_peered":"2023-11-26T07:21:56.295949+0000","last_unstale":"2023-11-26T07:21:56.305354+0000","last_undegraded":"2023-11-26T07:21:56.305354+0000","last_fullsized":"2023-11-26T07:21:56.305354+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:21:55.264078+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:21:55.264078+0000","last_clean_scrub_stamp":"2023-11-26T07:21:55.264078+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T12:17:05.630587+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.2","version":"0'0","reported_seq":11,"reported_epoch":58,"state":"active+clean","last_fresh":"2023-11-26T07:21:56.286018+0000","last_change":"2023-11-26T07:21:55.295847+0000","last_active":"2023-11-26T07:21:56.286018+0000","last_peered":"2023-11-26T07:21:56.286018+0000","last_clean":"2023-11-26T07:21:56.286018+0000","last_became_active":"2023-11-26T07:21:55.295562+0000","last_became_peered":"2023-11-26T07:21:55.295562+0000","last_unstale":"2023-11-26T07:21:56.286018+0000","last_undegraded":"2023-11-26T07:21:56.286018+0000","last_fullsized":"2023-11-26T07:21:56.286018+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:21:54.255568+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:21:54.255568+0000","last_clean_scrub_stamp":"2023-11-26T07:21:54.255568+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T15:10:28.055213+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.0","version":"0'0","reported_seq":9,"reported_epoch":58,"state":"active+clean","last_fresh":"2023-11-26T07:21:56.312723+0000","last_change":"2023-11-26T07:21:56.305723+0000","last_active":"2023-11-26T07:21:56.312723+0000","last_peered":"2023-11-26T07:21:56.312723+0000","last_clean":"2023-11-26T07:21:56.312723+0000","last_became_active":"2023-11-26T07:21:56.305464+0000","last_became_peered":"2023-11-26T07:21:56.305464+0000","last_unstale":"2023-11-26T07:21:56.312723+0000","last_undegraded":"2023-11-26T07:21:56.312723+0000","last_fullsized":"2023-11-26T07:21:56.312723+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:21:55.264078+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:21:55.264078+0000","last_clean_scrub_stamp":"2023-11-26T07:21:55.264078+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T08:21:37.149660+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.1","version":"0'0","reported_seq":11,"reported_epoch":58,"state":"active+clean","last_fresh":"2023-11-26T07:21:56.285447+0000","last_change":"2023-11-26T07:21:55.293187+0000","last_active":"2023-11-26T07:21:56.285447+0000","last_peered":"2023-11-26T07:21:56.285447+0000","last_clean":"2023-11-26T07:21:56.285447+0000","last_became_active":"2023-11-26T07:21:55.292900+0000","last_became_peered":"2023-11-26T07:21:55.292900+0000","last_unstale":"2023-11-26T07:21:56.285447+0000","last_undegraded":"2023-11-26T07:21:56.285447+0000","last_fullsized":"2023-11-26T07:21:56.285447+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:21:54.255568+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:21:54.255568+0000","last_clean_scrub_stamp":"2023-11-26T07:21:54.255568+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T15:06:33.378614+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.1","version":"0'0","reported_seq":9,"reported_epoch":58,"state":"active+clean","last_fresh":"2023-11-26T07:21:56.302928+0000","last_change":"2023-11-26T07:21:56.296770+0000","last_active":"2023-11-26T07:21:56.302928+0000","last_peered":"2023-11-26T07:21:56.302928+0000","last_clean":"2023-11-26T07:21:56.302928+0000","last_became_active":"2023-11-26T07:21:56.296481+0000","last_became_peered":"2023-11-26T07:21:56.296481+0000","last_unstale":"2023-11-26T07:21:56.302928+0000","last_undegraded":"2023-11-26T07:21:56.302928+0000","last_fullsized":"2023-11-26T07:21:56.302928+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:21:55.264078+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:21:55.264078+0000","last_clean_scrub_stamp":"2023-11-26T07:21:55.264078+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T11:09:02.730019+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.0","version":"0'0","reported_seq":11,"reported_epoch":58,"state":"active+clean","last_fresh":"2023-11-26T07:21:56.284312+0000","last_change":"2023-11-26T07:21:55.298176+0000","last_active":"2023-11-26T07:21:56.284312+0000","last_peered":"2023-11-26T07:21:56.284312+0000","last_clean":"2023-11-26T07:21:56.284312+0000","last_became_active":"2023-11-26T07:21:55.297908+0000","last_became_peered":"2023-11-26T07:21:55.297908+0000","last_unstale":"2023-11-26T07:21:56.284312+0000","last_undegraded":"2023-11-26T07:21:56.284312+0000","last_fullsized":"2023-11-26T07:21:56.284312+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:21:54.255568+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:21:54.255568+0000","last_clean_scrub_stamp":"2023-11-26T07:21:54.255568+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T13:59:39.325631+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.8","version":"0'0","reported_seq":9,"reported_epoch":58,"state":"active+clean","last_fresh":"2023-11-26T07:21:56.302819+0000","last_change":"2023-11-26T07:21:56.297889+0000","last_active":"2023-11-26T07:21:56.302819+0000","last_peered":"2023-11-26T07:21:56.302819+0000","last_clean":"2023-11-26T07:21:56.302819+0000","last_became_active":"2023-11-26T07:21:56.297618+0000","last_became_peered":"2023-11-26T07:21:56.297618+0000","last_unstale":"2023-11-26T07:21:56.302819+0000","last_undegraded":"2023-11-26T07:21:56.302819+0000","last_fullsized":"2023-11-26T07:21:56.302819+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:21:55.264078+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:21:55.264078+0000","last_clean_scrub_stamp":"2023-11-26T07:21:55.264078+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T10:22:37.629253+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.9","version":"0'0","reported_seq":11,"reported_epoch":58,"state":"active+clean","last_fresh":"2023-11-26T07:21:56.277661+0000","last_change":"2023-11-26T07:21:55.291675+0000","last_active":"2023-11-26T07:21:56.277661+0000","last_peered":"2023-11-26T07:21:56.277661+0000","last_clean":"2023-11-26T07:21:56.277661+0000","last_became_active":"2023-11-26T07:21:55.291359+0000","last_became_peered":"2023-11-26T07:21:55.291359+0000","last_unstale":"2023-11-26T07:21:56.277661+0000","last_undegraded":"2023-11-26T07:21:56.277661+0000","last_fullsized":"2023-11-26T07:21:56.277661+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:21:54.255568+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:21:54.255568+0000","last_clean_scrub_stamp":"2023-11-26T07:21:54.255568+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T08:21:36.141151+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"1.0","version":"15'117","reported_seq":195,"reported_epoch":57,"state":"active+clean","last_fresh":"2023-11-26T07:21:55.294411+0000","last_change":"2023-11-26T07:17:43.827802+0000","last_active":"2023-11-26T07:21:55.294411+0000","last_peered":"2023-11-26T07:21:55.294411+0000","last_clean":"2023-11-26T07:21:55.294411+0000","last_became_active":"2023-11-26T07:17:43.827308+0000","last_became_peered":"2023-11-26T07:17:43.827308+0000","last_unstale":"2023-11-26T07:21:55.294411+0000","last_undegraded":"2023-11-26T07:21:55.294411+0000","last_fullsized":"2023-11-26T07:21:55.294411+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:17:42.160945+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:17:42.160945+0000","last_clean_scrub_stamp":"2023-11-26T07:17:42.160945+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T08:17:54.167366+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":9,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":8,"num_pg":32,"stat_sum":{"num_bytes":2410,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":38,"ondisk_log_size":38,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574901,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":28292,"kb_used_data":1268,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343548,"statfs":{"total":96636764160,"available":96607793152,"internally_reserved":0,"allocated":1298432,"data_stored":879992,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Nov 26 07:21:51 2023","interfaces":[{"interface":"back","average":{"1min":0.76600000000000001,"5min":0.71899999999999997,"15min":0.71599999999999997},"min":{"1min":0.42999999999999999,"5min":0.42999999999999999,"15min":0.42999999999999999},"max":{"1min":0.95999999999999996,"5min":0.98199999999999998,"15min":0.98199999999999998},"last":0.437},{"interface":"front","average":{"1min":0.76400000000000001,"5min":0.72799999999999998,"15min":0.71499999999999997},"min":{"1min":0.53300000000000003,"5min":0.52900000000000003,"15min":0.52900000000000003},"max":{"1min":0.98599999999999999,"5min":1.0980000000000001,"15min":1.0980000000000001},"last":0.52900000000000003}]},{"osd":1,"last update":"Sun Nov 26 07:21:51 2023","interfaces":[{"interface":"back","average":{"1min":0.70599999999999996,"5min":0.68700000000000006,"15min":0.68300000000000005},"min":{"1min":0.441,"5min":0.441,"15min":0.441},"max":{"1min":1.012,"5min":1.0669999999999999,"15min":1.0669999999999999},"last":0.84599999999999997},{"interface":"front","average":{"1min":0.73599999999999999,"5min":0.74399999999999999,"15min":0.80700000000000005},"min":{"1min":0.58099999999999996,"5min":0.47599999999999998,"15min":0.47599999999999998},"max":{"1min":0.94199999999999995,"5min":4.5309999999999997,"15min":4.5309999999999997},"last":0.58899999999999997}]},{"osd":2,"last update":"Sun Nov 26 07:21:51 2023","interfaces":[{"interface":"back","average":{"1min":0.73399999999999999,"5min":0.69299999999999995,"15min":0.68100000000000005},"min":{"1min":0.39700000000000002,"5min":0.39700000000000002,"15min":0.39700000000000002},"max":{"1min":1.0800000000000001,"5min":1.0800000000000001,"15min":1.0800000000000001},"last":0.71099999999999997},{"interface":"front","average":{"1min":0.76800000000000002,"5min":0.71299999999999997,"15min":0.71299999999999997},"min":{"1min":0.497,"5min":0.497,"15min":0.497},"max":{"1min":0.96299999999999997,"5min":1.1339999999999999,"15min":1.1339999999999999},"last":0.80700000000000005}]},{"osd":3,"last update":"Sun Nov 26 07:21:51 2023","interfaces":[{"interface":"back","average":{"1min":0.73799999999999999,"5min":0.71899999999999997,"15min":0.70399999999999996},"min":{"1min":0.53000000000000003,"5min":0.47799999999999998,"15min":0.47799999999999998},"max":{"1min":1.1240000000000001,"5min":1.1240000000000001,"15min":1.1240000000000001},"last":0.67300000000000004},{"interface":"front","average":{"1min":0.81599999999999995,"5min":0.748,"15min":0.73799999999999999},"min":{"1min":0.54600000000000004,"5min":0.433,"15min":0.433},"max":{"1min":1.036,"5min":1.4490000000000001,"15min":1.4490000000000001},"last":0.747}]},{"osd":4,"last update":"Sun Nov 26 07:21:51 2023","interfaces":[{"interface":"back","average":{"1min":0.59999999999999998,"5min":0.61199999999999999,"15min":0.60399999999999998},"min":{"1min":0.47099999999999997,"5min":0.39400000000000002,"15min":0.39400000000000002},"max":{"1min":0.96199999999999997,"5min":0.96299999999999997,"15min":0.96299999999999997},"last":0.88},{"interface":"front","average":{"1min":0.66000000000000003,"5min":0.66100000000000003,"15min":0.66900000000000004},"min":{"1min":0.44600000000000001,"5min":0.436,"15min":0.436},"max":{"1min":1.004,"5min":1.5509999999999999,"15min":1.5509999999999999},"last":0.625}]},{"osd":5,"last update":"Sun Nov 26 07:21:51 2023","interfaces":[{"interface":"back","average":{"1min":0.69299999999999995,"5min":0.68200000000000005,"15min":0.67100000000000004},"min":{"1min":0.46300000000000002,"5min":0.443,"15min":0.443},"max":{"1min":0.93300000000000005,"5min":1.415,"15min":1.415},"last":0.68500000000000005},{"interface":"front","average":{"1min":0.69299999999999995,"5min":0.69399999999999995,"15min":0.70699999999999996},"min":{"1min":0.48699999999999999,"5min":0.48599999999999999,"15min":0.48599999999999999},"max":{"1min":1.0840000000000001,"5min":1.3859999999999999,"15min":1.3859999999999999},"last":0.93200000000000005}]},{"osd":6,"last update":"Sun Nov 26 07:21:51 2023","interfaces":[{"interface":"back","average":{"1min":0.79200000000000004,"5min":0.77800000000000002,"15min":0.77300000000000002},"min":{"1min":0.56399999999999995,"5min":0.51700000000000002,"15min":0.51700000000000002},"max":{"1min":1.123,"5min":1.3069999999999999,"15min":1.3069999999999999},"last":0.97599999999999998},{"interface":"front","average":{"1min":0.75600000000000001,"5min":0.76200000000000001,"15min":0.76800000000000002},"min":{"1min":0.58399999999999996,"5min":0.46899999999999997,"15min":0.46899999999999997},"max":{"1min":0.90800000000000003,"5min":1.6759999999999999,"15min":1.6759999999999999},"last":0.77600000000000002}]}]},{"osd":6,"up_from":13,"seq":55834574903,"num_pgs":17,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27704,"kb_used_data":680,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344136,"statfs":{"total":96636764160,"available":96608395264,"internally_reserved":0,"allocated":696320,"data_stored":289464,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Nov 26 07:21:53 2023","interfaces":[{"interface":"back","average":{"1min":0.69799999999999995,"5min":0.69799999999999995,"15min":0.68799999999999994},"min":{"1min":0.48999999999999999,"5min":0.36699999999999999,"15min":0.36699999999999999},"max":{"1min":0.91800000000000004,"5min":1.7549999999999999,"15min":1.7549999999999999},"last":0.63100000000000001},{"interface":"front","average":{"1min":0.72899999999999998,"5min":0.81799999999999995,"15min":0.91100000000000003},"min":{"1min":0.46600000000000003,"5min":0.38300000000000001,"15min":0.38300000000000001},"max":{"1min":0.874,"5min":5.7489999999999997,"15min":5.7489999999999997},"last":0.60399999999999998}]},{"osd":1,"last update":"Sun Nov 26 07:21:53 2023","interfaces":[{"interface":"back","average":{"1min":0.74199999999999999,"5min":0.80300000000000005,"15min":0.877},"min":{"1min":0.48699999999999999,"5min":0.41199999999999998,"15min":0.41199999999999998},"max":{"1min":0.97799999999999998,"5min":5.4240000000000004,"15min":5.4240000000000004},"last":0.59099999999999997},{"interface":"front","average":{"1min":0.72999999999999998,"5min":0.79500000000000004,"15min":0.86299999999999999},"min":{"1min":0.52200000000000002,"5min":0.499,"15min":0.499},"max":{"1min":0.96499999999999997,"5min":5.6020000000000003,"15min":5.6020000000000003},"last":0.71399999999999997}]},{"osd":2,"last update":"Sun Nov 26 07:21:53 2023","interfaces":[{"interface":"back","average":{"1min":0.70199999999999996,"5min":0.68999999999999995,"15min":0.69399999999999995},"min":{"1min":0.499,"5min":0.45100000000000001,"15min":0.45100000000000001},"max":{"1min":0.93600000000000005,"5min":1.8520000000000001,"15min":1.8520000000000001},"last":0.57999999999999996},{"interface":"front","average":{"1min":0.752,"5min":0.73399999999999999,"15min":0.73399999999999999},"min":{"1min":0.54300000000000004,"5min":0.437,"15min":0.437},"max":{"1min":0.95199999999999996,"5min":1.9159999999999999,"15min":1.9159999999999999},"last":0.69199999999999995}]},{"osd":3,"last update":"Sun Nov 26 07:21:53 2023","interfaces":[{"interface":"back","average":{"1min":0.72699999999999998,"5min":0.80000000000000004,"15min":0.86799999999999999},"min":{"1min":0.52400000000000002,"5min":0.46899999999999997,"15min":0.46899999999999997},"max":{"1min":0.91500000000000004,"5min":5.4390000000000001,"15min":5.4390000000000001},"last":0.64500000000000002},{"interface":"front","average":{"1min":0.74299999999999999,"5min":0.81499999999999995,"15min":0.90100000000000002},"min":{"1min":0.52300000000000002,"5min":0.48399999999999999,"15min":0.48399999999999999},"max":{"1min":0.89800000000000002,"5min":5.5490000000000004,"15min":5.5490000000000004},"last":0.66200000000000003}]},{"osd":4,"last update":"Sun Nov 26 07:21:53 2023","interfaces":[{"interface":"back","average":{"1min":0.56399999999999995,"5min":0.626,"15min":0.63600000000000001},"min":{"1min":0.40600000000000003,"5min":0.371,"15min":0.371},"max":{"1min":0.82999999999999996,"5min":1.0129999999999999,"15min":1.0129999999999999},"last":0.45100000000000001},{"interface":"front","average":{"1min":0.58899999999999997,"5min":0.65300000000000002,"15min":0.66700000000000004},"min":{"1min":0.38600000000000001,"5min":0.38600000000000001,"15min":0.38600000000000001},"max":{"1min":0.86399999999999999,"5min":1.0349999999999999,"15min":1.0349999999999999},"last":0.505}]},{"osd":5,"last update":"Sun Nov 26 07:21:53 2023","interfaces":[{"interface":"back","average":{"1min":0.62,"5min":0.63800000000000001,"15min":0.65200000000000002},"min":{"1min":0.40699999999999997,"5min":0.38200000000000001,"15min":0.38200000000000001},"max":{"1min":0.84499999999999997,"5min":0.873,"15min":0.873},"last":0.53200000000000003},{"interface":"front","average":{"1min":0.65700000000000003,"5min":0.69899999999999995,"15min":0.70799999999999996},"min":{"1min":0.44,"5min":0.436,"15min":0.436},"max":{"1min":0.92900000000000005,"5min":1.0960000000000001,"15min":1.0960000000000001},"last":0.55500000000000005}]},{"osd":7,"last update":"Sun Nov 26 07:21:53 2023","interfaces":[{"interface":"back","average":{"1min":0.67900000000000005,"5min":0.71999999999999997,"15min":0.73399999999999999},"min":{"1min":0.42599999999999999,"5min":0.42599999999999999,"15min":0.42599999999999999},"max":{"1min":0.93999999999999995,"5min":0.998,"15min":0.998},"last":0.67800000000000005},{"interface":"front","average":{"1min":0.64800000000000002,"5min":0.69799999999999995,"15min":0.70999999999999996},"min":{"1min":0.46400000000000002,"5min":0.46400000000000002,"15min":0.46400000000000002},"max":{"1min":0.85299999999999998,"5min":0.98799999999999999,"15min":0.98799999999999999},"last":0.61799999999999999}]}]},{"osd":5,"up_from":13,"seq":55834574901,"num_pgs":6,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27688,"kb_used_data":668,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344152,"statfs":{"total":96636764160,"available":96608411648,"internally_reserved":0,"allocated":684032,"data_stored":282689,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Nov 26 07:21:48 2023","interfaces":[{"interface":"back","average":{"1min":0.72199999999999998,"5min":0.68799999999999994,"15min":0.67300000000000004},"min":{"1min":0.56200000000000006,"5min":0.38100000000000001,"15min":0.38100000000000001},"max":{"1min":0.872,"5min":1.0289999999999999,"15min":1.0289999999999999},"last":0.84799999999999998},{"interface":"front","average":{"1min":0.71399999999999997,"5min":0.69599999999999995,"15min":0.67200000000000004},"min":{"1min":0.50600000000000001,"5min":0.438,"15min":0.438},"max":{"1min":1.028,"5min":1.05,"15min":1.05},"last":0.92200000000000004}]},{"osd":1,"last update":"Sun Nov 26 07:21:48 2023","interfaces":[{"interface":"back","average":{"1min":0.71199999999999997,"5min":0.69499999999999995,"15min":0.67700000000000005},"min":{"1min":0.45700000000000002,"5min":0.41599999999999998,"15min":0.41599999999999998},"max":{"1min":0.95399999999999996,"5min":1.0389999999999999,"15min":1.0389999999999999},"last":0.79900000000000004},{"interface":"front","average":{"1min":0.72199999999999998,"5min":0.70899999999999996,"15min":0.69299999999999995},"min":{"1min":0.53500000000000003,"5min":0.443,"15min":0.443},"max":{"1min":0.99299999999999999,"5min":1.006,"15min":1.006},"last":0.90700000000000003}]},{"osd":2,"last update":"Sun Nov 26 07:21:48 2023","interfaces":[{"interface":"back","average":{"1min":0.72199999999999998,"5min":0.69799999999999995,"15min":0.68200000000000005},"min":{"1min":0.51300000000000001,"5min":0.45900000000000002,"15min":0.45900000000000002},"max":{"1min":0.92900000000000005,"5min":0.95699999999999996,"15min":0.95699999999999996},"last":0.86199999999999999},{"interface":"front","average":{"1min":0.78700000000000003,"5min":0.72999999999999998,"15min":0.68999999999999995},"min":{"1min":0.55100000000000005,"5min":0.51200000000000001,"15min":0.51200000000000001},"max":{"1min":1.1379999999999999,"5min":1.1379999999999999,"15min":1.1379999999999999},"last":0.88600000000000001}]},{"osd":3,"last update":"Sun Nov 26 07:21:48 2023","interfaces":[{"interface":"back","average":{"1min":0.75700000000000001,"5min":0.73299999999999998,"15min":0.72399999999999998},"min":{"1min":0.44700000000000001,"5min":0.43099999999999999,"15min":0.43099999999999999},"max":{"1min":1.1739999999999999,"5min":1.1739999999999999,"15min":1.1739999999999999},"last":0.83399999999999996},{"interface":"front","average":{"1min":0.77800000000000002,"5min":0.72299999999999998,"15min":0.69099999999999995},"min":{"1min":0.55600000000000005,"5min":0.44600000000000001,"15min":0.44600000000000001},"max":{"1min":1.0029999999999999,"5min":1.0209999999999999,"15min":1.0209999999999999},"last":0.93100000000000005}]},{"osd":4,"last update":"Sun Nov 26 07:21:48 2023","interfaces":[{"interface":"back","average":{"1min":0.56799999999999995,"5min":0.59999999999999998,"15min":0.623},"min":{"1min":0.41199999999999998,"5min":0.35299999999999998,"15min":0.35299999999999998},"max":{"1min":0.92700000000000005,"5min":0.95099999999999996,"15min":0.95099999999999996},"last":0.81799999999999995},{"interface":"front","average":{"1min":0.66200000000000003,"5min":0.67000000000000004,"15min":0.69599999999999995},"min":{"1min":0.45000000000000001,"5min":0.41299999999999998,"15min":0.41299999999999998},"max":{"1min":0.94699999999999995,"5min":1.254,"15min":1.254},"last":0.71699999999999997}]},{"osd":6,"last update":"Sun Nov 26 07:21:48 2023","interfaces":[{"interface":"back","average":{"1min":0.61399999999999999,"5min":0.66200000000000003,"15min":0.66800000000000004},"min":{"1min":0.44400000000000001,"5min":0.439,"15min":0.439},"max":{"1min":1.0880000000000001,"5min":1.0880000000000001,"15min":1.0880000000000001},"last":0.67800000000000005},{"interface":"front","average":{"1min":0.66600000000000004,"5min":0.70299999999999996,"15min":0.71899999999999997},"min":{"1min":0.47699999999999998,"5min":0.436,"15min":0.436},"max":{"1min":1.052,"5min":1.0649999999999999,"15min":1.0649999999999999},"last":0.76700000000000002}]},{"osd":7,"last update":"Sun Nov 26 07:21:48 2023","interfaces":[{"interface":"back","average":{"1min":0.73099999999999998,"5min":0.75900000000000001,"15min":0.78900000000000003},"min":{"1min":0.51800000000000002,"5min":0.48199999999999998,"15min":0.48199999999999998},"max":{"1min":1.2010000000000001,"5min":1.4079999999999999,"15min":1.4079999999999999},"last":0.74399999999999999},{"interface":"front","average":{"1min":0.72899999999999998,"5min":0.751,"15min":0.76700000000000002},"min":{"1min":0.52400000000000002,"5min":0.45000000000000001,"15min":0.45000000000000001},"max":{"1min":1.115,"5min":1.115,"15min":1.115},"last":0.98299999999999998}]}]},{"osd":4,"up_from":13,"seq":55834574903,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27716,"kb_used_data":692,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344124,"statfs":{"total":96636764160,"available":96608382976,"internally_reserved":0,"allocated":708608,"data_stored":291076,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Nov 26 07:21:53 2023","interfaces":[{"interface":"back","average":{"1min":0.67300000000000004,"5min":0.70899999999999996,"15min":0.76500000000000001},"min":{"1min":0.46100000000000002,"5min":0.32100000000000001,"15min":0.32100000000000001},"max":{"1min":0.94499999999999995,"5min":4.1689999999999996,"15min":4.1689999999999996},"last":0.67800000000000005},{"interface":"front","average":{"1min":1.609,"5min":0.98299999999999998,"15min":0.995},"min":{"1min":0.52800000000000002,"5min":0.46200000000000002,"15min":0.46200000000000002},"max":{"1min":18.021000000000001,"5min":18.021000000000001,"15min":18.021000000000001},"last":0.83599999999999997}]},{"osd":1,"last update":"Sun Nov 26 07:21:53 2023","interfaces":[{"interface":"back","average":{"1min":0.90500000000000003,"5min":0.71199999999999997,"15min":0.66800000000000004},"min":{"1min":0.47499999999999998,"5min":0.43099999999999999,"15min":0.43099999999999999},"max":{"1min":4.3159999999999998,"5min":4.3159999999999998,"15min":4.3159999999999998},"last":0.749},{"interface":"front","average":{"1min":1.2150000000000001,"5min":0.82799999999999996,"15min":0.82199999999999995},"min":{"1min":0.51900000000000002,"5min":0.39800000000000002,"15min":0.39800000000000002},"max":{"1min":11.206,"5min":11.206,"15min":11.206},"last":0.78100000000000003}]},{"osd":2,"last update":"Sun Nov 26 07:21:53 2023","interfaces":[{"interface":"back","average":{"1min":1.492,"5min":0.89800000000000002,"15min":0.83699999999999997},"min":{"1min":0.47199999999999998,"5min":0.433,"15min":0.433},"max":{"1min":15.441000000000001,"5min":15.441000000000001,"15min":15.441000000000001},"last":0.621},{"interface":"front","average":{"1min":1.2350000000000001,"5min":0.97399999999999998,"15min":1.0349999999999999},"min":{"1min":0.59399999999999997,"5min":0.53400000000000003,"15min":0.53400000000000003},"max":{"1min":9.4800000000000004,"5min":9.4800000000000004,"15min":9.4800000000000004},"last":0.88400000000000001}]},{"osd":3,"last update":"Sun Nov 26 07:21:53 2023","interfaces":[{"interface":"back","average":{"1min":1.3700000000000001,"5min":0.83599999999999997,"15min":0.76500000000000001},"min":{"1min":0.57199999999999995,"5min":0.441,"15min":0.441},"max":{"1min":12.715,"5min":12.715,"15min":12.715},"last":0.90200000000000002},{"interface":"front","average":{"1min":1.145,"5min":0.87,"15min":0.876},"min":{"1min":0.50700000000000001,"5min":0.50700000000000001,"15min":0.50700000000000001},"max":{"1min":8.3239999999999998,"5min":8.3239999999999998,"15min":8.3239999999999998},"last":0.73299999999999998}]},{"osd":5,"last update":"Sun Nov 26 07:21:53 2023","interfaces":[{"interface":"back","average":{"1min":0.63800000000000001,"5min":0.61299999999999999,"15min":0.61099999999999999},"min":{"1min":0.47099999999999997,"5min":0.38200000000000001,"15min":0.38200000000000001},"max":{"1min":0.93999999999999995,"5min":0.96199999999999997,"15min":0.96199999999999997},"last":0.93999999999999995},{"interface":"front","average":{"1min":1.165,"5min":0.72799999999999998,"15min":0.67000000000000004},"min":{"1min":0.442,"5min":0.35599999999999998,"15min":0.35599999999999998},"max":{"1min":11.148999999999999,"5min":11.148999999999999,"15min":11.148999999999999},"last":0.86299999999999999}]},{"osd":6,"last update":"Sun Nov 26 07:21:53 2023","interfaces":[{"interface":"back","average":{"1min":0.624,"5min":0.67500000000000004,"15min":0.754},"min":{"1min":0.41699999999999998,"5min":0.372,"15min":0.372},"max":{"1min":0.81799999999999995,"5min":2.8330000000000002,"15min":2.8330000000000002},"last":0.81799999999999995},{"interface":"front","average":{"1min":0.79200000000000004,"5min":0.83299999999999996,"15min":0.94999999999999996},"min":{"1min":0.54000000000000004,"5min":0.41299999999999998,"15min":0.41299999999999998},"max":{"1min":2.472,"5min":7.6050000000000004,"15min":7.6050000000000004},"last":0.97699999999999998}]},{"osd":7,"last update":"Sun Nov 26 07:21:53 2023","interfaces":[{"interface":"back","average":{"1min":1.339,"5min":0.88100000000000001,"15min":0.86699999999999999},"min":{"1min":0.57999999999999996,"5min":0.44700000000000001,"15min":0.44700000000000001},"max":{"1min":12.653,"5min":12.653,"15min":12.653},"last":0.95999999999999996},{"interface":"front","average":{"1min":0.90100000000000002,"5min":0.82199999999999995,"15min":0.86799999999999999},"min":{"1min":0.47499999999999998,"5min":0.33300000000000002,"15min":0.33300000000000002},"max":{"1min":3.8319999999999999,"5min":3.8319999999999999,"15min":3.8319999999999999},"last":0.91900000000000004}]}]},{"osd":3,"up_from":13,"seq":55834574904,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31872,"kb_used_data":688,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339968,"statfs":{"total":96636764160,"available":96604127232,"internally_reserved":0,"allocated":704512,"data_stored":289520,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1592,"internal_metadata":31914440},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Nov 26 07:21:48 2023","interfaces":[{"interface":"back","average":{"1min":0.46000000000000002,"5min":0.41299999999999998,"15min":0.39000000000000001},"min":{"1min":0.25800000000000001,"5min":0.25800000000000001,"15min":0.25800000000000001},"max":{"1min":0.72799999999999998,"5min":0.72799999999999998,"15min":0.72799999999999998},"last":0.318},{"interface":"front","average":{"1min":0.48599999999999999,"5min":0.45900000000000002,"15min":0.441},"min":{"1min":0.316,"5min":0.26900000000000002,"15min":0.26900000000000002},"max":{"1min":0.71299999999999997,"5min":0.77300000000000002,"15min":0.77300000000000002},"last":0.54500000000000004}]},{"osd":1,"last update":"Sun Nov 26 07:21:48 2023","interfaces":[{"interface":"back","average":{"1min":0.443,"5min":0.438,"15min":0.42599999999999999},"min":{"1min":0.34100000000000003,"5min":0.28799999999999998,"15min":0.28799999999999998},"max":{"1min":0.59299999999999997,"5min":0.752,"15min":0.752},"last":0.38300000000000001},{"interface":"front","average":{"1min":0.503,"5min":0.498,"15min":0.48799999999999999},"min":{"1min":0.374,"5min":0.34599999999999997,"15min":0.34599999999999997},"max":{"1min":0.76200000000000001,"5min":0.872,"15min":0.872},"last":0.35599999999999998}]},{"osd":2,"last update":"Sun Nov 26 07:21:48 2023","interfaces":[{"interface":"back","average":{"1min":0.58399999999999996,"5min":0.56299999999999994,"15min":0.54300000000000004},"min":{"1min":0.376,"5min":0.376,"15min":0.376},"max":{"1min":0.97199999999999998,"5min":0.97199999999999998,"15min":0.97199999999999998},"last":0.44700000000000001},{"interface":"front","average":{"1min":0.56799999999999995,"5min":0.53700000000000003,"15min":0.51500000000000001},"min":{"1min":0.40899999999999997,"5min":0.33000000000000002,"15min":0.33000000000000002},"max":{"1min":0.81999999999999995,"5min":0.89700000000000002,"15min":0.89700000000000002},"last":0.40899999999999997}]},{"osd":4,"last update":"Sun Nov 26 07:21:48 2023","interfaces":[{"interface":"back","average":{"1min":0.71099999999999997,"5min":0.71799999999999997,"15min":0.70999999999999996},"min":{"1min":0.45800000000000002,"5min":0.45800000000000002,"15min":0.45800000000000002},"max":{"1min":0.93000000000000005,"5min":1.091,"15min":1.091},"last":0.52400000000000002},{"interface":"front","average":{"1min":0.73999999999999999,"5min":0.73899999999999999,"15min":0.73099999999999998},"min":{"1min":0.39900000000000002,"5min":0.39900000000000002,"15min":0.39900000000000002},"max":{"1min":1.0289999999999999,"5min":1.242,"15min":1.242},"last":0.58299999999999996}]},{"osd":5,"last update":"Sun Nov 26 07:21:48 2023","interfaces":[{"interface":"back","average":{"1min":0.70999999999999996,"5min":0.73299999999999998,"15min":0.73699999999999999},"min":{"1min":0.47099999999999997,"5min":0.47099999999999997,"15min":0.47099999999999997},"max":{"1min":0.94699999999999995,"5min":1.105,"15min":1.105},"last":0.501},{"interface":"front","average":{"1min":0.73499999999999999,"5min":0.75,"15min":0.74199999999999999},"min":{"1min":0.48799999999999999,"5min":0.48799999999999999,"15min":0.48799999999999999},"max":{"1min":0.97399999999999998,"5min":1.1319999999999999,"15min":1.1319999999999999},"last":0.48799999999999999}]},{"osd":6,"last update":"Sun Nov 26 07:21:48 2023","interfaces":[{"interface":"back","average":{"1min":0.76100000000000001,"5min":0.79400000000000004,"15min":0.80700000000000005},"min":{"1min":0.503,"5min":0.503,"15min":0.503},"max":{"1min":0.89100000000000001,"5min":1.2729999999999999,"15min":1.2729999999999999},"last":0.67500000000000004},{"interface":"front","average":{"1min":0.77500000000000002,"5min":0.77900000000000003,"15min":0.78800000000000003},"min":{"1min":0.51100000000000001,"5min":0.47299999999999998,"15min":0.47299999999999998},"max":{"1min":1.002,"5min":1.204,"15min":1.204},"last":0.66500000000000004}]},{"osd":7,"last update":"Sun Nov 26 07:21:48 2023","interfaces":[{"interface":"back","average":{"1min":0.78900000000000003,"5min":0.77700000000000002,"15min":0.79300000000000004},"min":{"1min":0.54500000000000004,"5min":0.5,"15min":0.5},"max":{"1min":0.999,"5min":1.3600000000000001,"15min":1.3600000000000001},"last":0.65200000000000002},{"interface":"front","average":{"1min":0.77600000000000002,"5min":0.81699999999999995,"15min":0.82299999999999995},"min":{"1min":0.47299999999999998,"5min":0.47299999999999998,"15min":0.47299999999999998},"max":{"1min":1.0349999999999999,"5min":1.617,"15min":1.617},"last":0.63700000000000001}]}]},{"osd":2,"up_from":13,"seq":55834574904,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27688,"kb_used_data":668,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344152,"statfs":{"total":96636764160,"available":96608411648,"internally_reserved":0,"allocated":684032,"data_stored":282689,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Nov 26 07:21:47 2023","interfaces":[{"interface":"back","average":{"1min":0.42299999999999999,"5min":0.55200000000000005,"15min":0.66000000000000003},"min":{"1min":0.29799999999999999,"5min":0.28699999999999998,"15min":0.28699999999999998},"max":{"1min":0.72899999999999998,"5min":4.7030000000000003,"15min":4.7030000000000003},"last":0.29599999999999999},{"interface":"front","average":{"1min":0.48299999999999998,"5min":0.51300000000000001,"15min":0.54500000000000004},"min":{"1min":0.27300000000000002,"5min":0.27300000000000002,"15min":0.27300000000000002},"max":{"1min":0.71899999999999997,"5min":2.4340000000000002,"15min":2.4340000000000002},"last":0.33600000000000002}]},{"osd":1,"last update":"Sun Nov 26 07:21:47 2023","interfaces":[{"interface":"back","average":{"1min":0.52400000000000002,"5min":0.64900000000000002,"15min":0.79200000000000004},"min":{"1min":0.36099999999999999,"5min":0.34599999999999997,"15min":0.34599999999999997},"max":{"1min":0.86099999999999999,"5min":4.2690000000000001,"15min":4.2690000000000001},"last":0.46100000000000002},{"interface":"front","average":{"1min":0.52700000000000002,"5min":0.60799999999999998,"15min":0.69699999999999995},"min":{"1min":0.376,"5min":0.31900000000000001,"15min":0.31900000000000001},"max":{"1min":0.70899999999999996,"5min":3.9940000000000002,"15min":3.9940000000000002},"last":0.39800000000000002}]},{"osd":3,"last update":"Sun Nov 26 07:21:47 2023","interfaces":[{"interface":"back","average":{"1min":0.497,"5min":0.55200000000000005,"15min":0.61199999999999999},"min":{"1min":0.32200000000000001,"5min":0.32000000000000001,"15min":0.32000000000000001},"max":{"1min":0.82699999999999996,"5min":4.0300000000000002,"15min":4.0300000000000002},"last":0.318},{"interface":"front","average":{"1min":0.56399999999999995,"5min":0.70299999999999996,"15min":0.84299999999999997},"min":{"1min":0.36099999999999999,"5min":0.36099999999999999,"15min":0.36099999999999999},"max":{"1min":0.86699999999999999,"5min":4.5810000000000004,"15min":4.5810000000000004},"last":0.48699999999999999}]},{"osd":4,"last update":"Sun Nov 26 07:21:47 2023","interfaces":[{"interface":"back","average":{"1min":0.78500000000000003,"5min":0.85599999999999998,"15min":0.94199999999999995},"min":{"1min":0.58899999999999997,"5min":0.51400000000000001,"15min":0.51400000000000001},"max":{"1min":0.98699999999999999,"5min":4.4269999999999996,"15min":4.4269999999999996},"last":0.91300000000000003},{"interface":"front","average":{"1min":0.83799999999999997,"5min":0.88100000000000001,"15min":0.96999999999999997},"min":{"1min":0.55500000000000005,"5min":0.55500000000000005,"15min":0.55500000000000005},"max":{"1min":1.0780000000000001,"5min":4.085,"15min":4.085},"last":0.77600000000000002}]},{"osd":5,"last update":"Sun Nov 26 07:21:47 2023","interfaces":[{"interface":"back","average":{"1min":0.81200000000000006,"5min":0.81000000000000005,"15min":0.85199999999999998},"min":{"1min":0.60699999999999998,"5min":0.49199999999999999,"15min":0.49199999999999999},"max":{"1min":1,"5min":3.9590000000000001,"15min":3.9590000000000001},"last":0.79700000000000004},{"interface":"front","average":{"1min":0.81200000000000006,"5min":0.874,"15min":0.94999999999999996},"min":{"1min":0.499,"5min":0.499,"15min":0.499},"max":{"1min":1.024,"5min":4.3849999999999998,"15min":4.3849999999999998},"last":0.86099999999999999}]},{"osd":6,"last update":"Sun Nov 26 07:21:47 2023","interfaces":[{"interface":"back","average":{"1min":0.79100000000000004,"5min":0.86199999999999999,"15min":0.95099999999999996},"min":{"1min":0.51800000000000002,"5min":0.51800000000000002,"15min":0.51800000000000002},"max":{"1min":0.94999999999999996,"5min":4.0449999999999999,"15min":4.0449999999999999},"last":0.88300000000000001},{"interface":"front","average":{"1min":0.77000000000000002,"5min":0.83499999999999996,"15min":0.89000000000000001},"min":{"1min":0.48099999999999998,"5min":0.48099999999999998,"15min":0.48099999999999998},"max":{"1min":1,"5min":3.992,"15min":3.992},"last":0.76000000000000001}]},{"osd":7,"last update":"Sun Nov 26 07:21:47 2023","interfaces":[{"interface":"back","average":{"1min":0.81599999999999995,"5min":0.88600000000000001,"15min":0.98499999999999999},"min":{"1min":0.63900000000000001,"5min":0.52200000000000002,"15min":0.52200000000000002},"max":{"1min":1.0149999999999999,"5min":4.1219999999999999,"15min":4.1219999999999999},"last":0.84899999999999998},{"interface":"front","average":{"1min":0.80100000000000005,"5min":0.85699999999999998,"15min":0.91100000000000003},"min":{"1min":0.50700000000000001,"5min":0.50700000000000001,"15min":0.50700000000000001},"max":{"1min":0.96599999999999997,"5min":3.9159999999999999,"15min":3.9159999999999999},"last":0.72399999999999998}]}]},{"osd":1,"up_from":13,"seq":55834574903,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27708,"kb_used_data":684,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344132,"statfs":{"total":96636764160,"available":96608391168,"internally_reserved":0,"allocated":700416,"data_stored":289554,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Nov 26 07:21:45 2023","interfaces":[{"interface":"back","average":{"1min":0.39400000000000002,"5min":0.40999999999999998,"15min":0.41399999999999998},"min":{"1min":0.29899999999999999,"5min":0.23599999999999999,"15min":0.23599999999999999},"max":{"1min":0.57499999999999996,"5min":0.61199999999999999,"15min":0.61199999999999999},"last":0.34000000000000002},{"interface":"front","average":{"1min":0.435,"5min":0.46200000000000002,"15min":0.47799999999999998},"min":{"1min":0.28899999999999998,"5min":0.26700000000000002,"15min":0.26700000000000002},"max":{"1min":0.61699999999999999,"5min":0.78700000000000003,"15min":0.78700000000000003},"last":0.44400000000000001}]},{"osd":2,"last update":"Sun Nov 26 07:21:45 2023","interfaces":[{"interface":"back","average":{"1min":0.42999999999999999,"5min":0.443,"15min":0.45100000000000001},"min":{"1min":0.35499999999999998,"5min":0.27800000000000002,"15min":0.27800000000000002},"max":{"1min":0.53600000000000003,"5min":0.67600000000000005,"15min":0.67600000000000005},"last":0.47299999999999998},{"interface":"front","average":{"1min":0.47399999999999998,"5min":0.498,"15min":0.504},"min":{"1min":0.35999999999999999,"5min":0.27700000000000002,"15min":0.27700000000000002},"max":{"1min":0.64200000000000002,"5min":0.80600000000000005,"15min":0.80600000000000005},"last":0.50900000000000001}]},{"osd":3,"last update":"Sun Nov 26 07:21:45 2023","interfaces":[{"interface":"back","average":{"1min":0.51200000000000001,"5min":0.52800000000000002,"15min":0.52900000000000003},"min":{"1min":0.42099999999999999,"5min":0.38400000000000001,"15min":0.38400000000000001},"max":{"1min":0.60799999999999998,"5min":0.748,"15min":0.748},"last":0.628},{"interface":"front","average":{"1min":0.496,"5min":0.499,"15min":0.497},"min":{"1min":0.35499999999999998,"5min":0.31,"15min":0.31},"max":{"1min":0.68600000000000005,"5min":0.80700000000000005,"15min":0.80700000000000005},"last":0.48999999999999999}]},{"osd":4,"last update":"Sun Nov 26 07:21:45 2023","interfaces":[{"interface":"back","average":{"1min":0.749,"5min":0.71999999999999997,"15min":0.70199999999999996},"min":{"1min":0.51500000000000001,"5min":0.41299999999999998,"15min":0.41299999999999998},"max":{"1min":1.0589999999999999,"5min":1.1000000000000001,"15min":1.1000000000000001},"last":0.53100000000000003},{"interface":"front","average":{"1min":0.71999999999999997,"5min":0.71399999999999997,"15min":0.69099999999999995},"min":{"1min":0.44500000000000001,"5min":0.44500000000000001,"15min":0.44500000000000001},"max":{"1min":0.93300000000000005,"5min":1.131,"15min":1.131},"last":0.54400000000000004}]},{"osd":5,"last update":"Sun Nov 26 07:21:45 2023","interfaces":[{"interface":"back","average":{"1min":0.73399999999999999,"5min":0.73099999999999998,"15min":0.72299999999999998},"min":{"1min":0.52000000000000002,"5min":0.49299999999999999,"15min":0.49299999999999999},"max":{"1min":0.89900000000000002,"5min":1.1919999999999999,"15min":1.1919999999999999},"last":0.57999999999999996},{"interface":"front","average":{"1min":0.70399999999999996,"5min":0.72699999999999998,"15min":0.71999999999999997},"min":{"1min":0.54700000000000004,"5min":0.48799999999999999,"15min":0.48799999999999999},"max":{"1min":0.93400000000000005,"5min":1.0609999999999999,"15min":1.0609999999999999},"last":0.58799999999999997}]},{"osd":6,"last update":"Sun Nov 26 07:21:45 2023","interfaces":[{"interface":"back","average":{"1min":0.76000000000000001,"5min":0.73199999999999998,"15min":0.71299999999999997},"min":{"1min":0.55000000000000004,"5min":0.50600000000000001,"15min":0.50600000000000001},"max":{"1min":0.97799999999999998,"5min":1.0369999999999999,"15min":1.0369999999999999},"last":0.60699999999999998},{"interface":"front","average":{"1min":0.74399999999999999,"5min":0.73399999999999999,"15min":0.71199999999999997},"min":{"1min":0.59999999999999998,"5min":0.504,"15min":0.504},"max":{"1min":0.94999999999999996,"5min":1.1339999999999999,"15min":1.1339999999999999},"last":0.60099999999999998}]},{"osd":7,"last update":"Sun Nov 26 07:21:45 2023","interfaces":[{"interface":"back","average":{"1min":0.75,"5min":0.74099999999999999,"15min":0.72299999999999998},"min":{"1min":0.50600000000000001,"5min":0.50600000000000001,"15min":0.50600000000000001},"max":{"1min":0.92400000000000004,"5min":1.002,"15min":1.002},"last":0.66200000000000003},{"interface":"front","average":{"1min":0.73699999999999999,"5min":0.747,"15min":0.751},"min":{"1min":0.56999999999999995,"5min":0.48499999999999999,"15min":0.48499999999999999},"max":{"1min":1.018,"5min":1.024,"15min":1.024},"last":0.56899999999999995}]}]},{"osd":0,"up_from":13,"seq":55834574903,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32432,"kb_used_data":1248,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339408,"statfs":{"total":96636764160,"available":96603553792,"internally_reserved":0,"allocated":1277952,"data_stored":873057,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Sun Nov 26 07:21:43 2023","interfaces":[{"interface":"back","average":{"1min":0.438,"5min":0.42599999999999999,"15min":0.42899999999999999},"min":{"1min":0.27000000000000002,"5min":0.27000000000000002,"15min":0.27000000000000002},"max":{"1min":0.624,"5min":0.64300000000000002,"15min":0.64300000000000002},"last":0.39800000000000002},{"interface":"front","average":{"1min":0.47299999999999998,"5min":0.46800000000000003,"15min":0.47799999999999998},"min":{"1min":0.32200000000000001,"5min":0.29899999999999999,"15min":0.29899999999999999},"max":{"1min":0.77200000000000002,"5min":1.133,"15min":1.133},"last":0.54900000000000004}]},{"osd":2,"last update":"Sun Nov 26 07:21:43 2023","interfaces":[{"interface":"back","average":{"1min":0.496,"5min":0.51800000000000002,"15min":0.53000000000000003},"min":{"1min":0.34699999999999998,"5min":0.34699999999999998,"15min":0.34699999999999998},"max":{"1min":0.84199999999999997,"5min":0.84199999999999997,"15min":0.84199999999999997},"last":0.53400000000000003},{"interface":"front","average":{"1min":0.56299999999999994,"5min":0.55700000000000005,"15min":0.56100000000000005},"min":{"1min":0.41399999999999998,"5min":0.36399999999999999,"15min":0.36399999999999999},"max":{"1min":0.77300000000000002,"5min":0.84199999999999997,"15min":0.84199999999999997},"last":0.58899999999999997}]},{"osd":3,"last update":"Sun Nov 26 07:21:43 2023","interfaces":[{"interface":"back","average":{"1min":0.52200000000000002,"5min":0.52200000000000002,"15min":0.52800000000000002},"min":{"1min":0.39000000000000001,"5min":0.34999999999999998,"15min":0.34999999999999998},"max":{"1min":0.75700000000000001,"5min":1.3149999999999999,"15min":1.3149999999999999},"last":0.47599999999999998},{"interface":"front","average":{"1min":0.55900000000000005,"5min":0.54400000000000004,"15min":0.55600000000000005},"min":{"1min":0.40000000000000002,"5min":0.371,"15min":0.371},"max":{"1min":0.72799999999999998,"5min":1.2509999999999999,"15min":1.2509999999999999},"last":0.61599999999999999}]},{"osd":4,"last update":"Sun Nov 26 07:21:43 2023","interfaces":[{"interface":"back","average":{"1min":0.747,"5min":0.749,"15min":0.755},"min":{"1min":0.44700000000000001,"5min":0.44700000000000001,"15min":0.44700000000000001},"max":{"1min":1.0900000000000001,"5min":1.2010000000000001,"15min":1.2010000000000001},"last":0.85499999999999998},{"interface":"front","average":{"1min":0.78000000000000003,"5min":0.78800000000000003,"15min":0.78800000000000003},"min":{"1min":0.51100000000000001,"5min":0.51100000000000001,"15min":0.51100000000000001},"max":{"1min":0.99199999999999999,"5min":1.381,"15min":1.381},"last":0.84399999999999997}]},{"osd":5,"last update":"Sun Nov 26 07:21:43 2023","interfaces":[{"interface":"back","average":{"1min":0.75900000000000001,"5min":0.76100000000000001,"15min":0.76000000000000001},"min":{"1min":0.54800000000000004,"5min":0.42199999999999999,"15min":0.42199999999999999},"max":{"1min":0.97999999999999998,"5min":1.1579999999999999,"15min":1.1579999999999999},"last":0.79000000000000004},{"interface":"front","average":{"1min":0.81499999999999995,"5min":0.80600000000000005,"15min":0.80000000000000004},"min":{"1min":0.53600000000000003,"5min":0.53600000000000003,"15min":0.53600000000000003},"max":{"1min":1.0680000000000001,"5min":1.399,"15min":1.399},"last":0.86699999999999999}]},{"osd":6,"last update":"Sun Nov 26 07:21:43 2023","interfaces":[{"interface":"back","average":{"1min":0.72299999999999998,"5min":0.72199999999999998,"15min":0.75},"min":{"1min":0.52400000000000002,"5min":0.50800000000000001,"15min":0.50800000000000001},"max":{"1min":0.89300000000000002,"5min":1.3029999999999999,"15min":1.3029999999999999},"last":0.68700000000000006},{"interface":"front","average":{"1min":0.747,"5min":0.75700000000000001,"15min":0.76500000000000001},"min":{"1min":0.47199999999999998,"5min":0.47199999999999998,"15min":0.47199999999999998},"max":{"1min":1.0489999999999999,"5min":1.3600000000000001,"15min":1.3600000000000001},"last":0.80900000000000005}]},{"osd":7,"last update":"Sun Nov 26 07:21:43 2023","interfaces":[{"interface":"back","average":{"1min":0.77200000000000002,"5min":0.78800000000000003,"15min":0.80300000000000005},"min":{"1min":0.56899999999999995,"5min":0.56899999999999995,"15min":0.56899999999999995},"max":{"1min":0.98899999999999999,"5min":1.177,"15min":1.177},"last":0.82999999999999996},{"interface":"front","average":{"1min":0.78600000000000003,"5min":0.76700000000000002,"15min":0.78800000000000003},"min":{"1min":0.51200000000000001,"5min":0.51200000000000001,"15min":0.51200000000000001},"max":{"1min":1.0629999999999999,"5min":1.2390000000000001,"15min":1.2390000000000001},"last":0.88400000000000001}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":2194,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":56,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":56,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":652,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":160,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-11-26T07:22:00.911 INFO:tasks.cephfs.filesystem.ceph_manager:clean! 2023-11-26T07:22:00.911 DEBUG:teuthology.orchestra.run.smithi143:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth caps client.0 mon 'allow r' osd 'allow rw tag cephfs data=cephfs' mds allow 2023-11-26T07:22:01.090 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:01.088+0000 7fc470357640 1 Processor -- start 2023-11-26T07:22:01.090 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:01.088+0000 7fc470357640 1 -- start start 2023-11-26T07:22:01.091 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:01.088+0000 7fc470357640 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7fc4680cbfe0 0x7fc46806c540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:22:01.092 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:01.088+0000 7fc470357640 1 -- --> v1:172.21.15.143:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc46806d7e0 con 0x7fc4681449f0 2023-11-26T07:22:01.092 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:01.088+0000 7fc470357640 1 -- --> v1:172.21.15.188:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc46806cbd0 con 0x7fc46806dc70 2023-11-26T07:22:01.092 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:01.088+0000 7fc470357640 1 -- --> v2:172.21.15.188:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc46806ceb0 con 0x7fc4680cbfe0 2023-11-26T07:22:01.092 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:01.088+0000 7fc46e0f3640 1 --1- >> v1:172.21.15.143:6789/0 conn(0x7fc4681449f0 0x7fc4680ce6b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.143:6789/0 says I am v1:172.21.15.143:57274/0 (socket says 172.21.15.143:57274) 2023-11-26T07:22:01.092 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:01.088+0000 7fc46e0f3640 1 -- 172.21.15.143:0/3688624290 learned_addr learned my addr 172.21.15.143:0/3688624290 (peer_addr_for_me v1:172.21.15.143:0/0) 2023-11-26T07:22:01.093 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:01.088+0000 7fc46e8f4640 1 --2- 172.21.15.143:0/3688624290 >> v2:172.21.15.188:3300/0 conn(0x7fc4680cbfe0 0x7fc46806c540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:22:01.093 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:01.092+0000 7fc46d0f1640 1 -- 172.21.15.143:0/3688624290 <== mon.0 v1:172.21.15.143:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1128560077 0 0) 0x7fc46806d7e0 con 0x7fc4681449f0 2023-11-26T07:22:01.093 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:01.092+0000 7fc46d0f1640 1 -- 172.21.15.143:0/3688624290 --> v1:172.21.15.143:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fc454003530 con 0x7fc4681449f0 2023-11-26T07:22:01.093 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:01.092+0000 7fc46e8f4640 1 -- 172.21.15.143:0/3688624290 >> v1:172.21.15.188:6789/0 conn(0x7fc46806dc70 legacy=0x7fc4680cb8d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:22:01.093 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:01.092+0000 7fc46e8f4640 1 -- 172.21.15.143:0/3688624290 >> v1:172.21.15.143:6789/0 conn(0x7fc4681449f0 legacy=0x7fc4680ce6b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:22:01.093 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:01.092+0000 7fc46e8f4640 1 -- 172.21.15.143:0/3688624290 --> v2:172.21.15.188:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc46806d190 con 0x7fc4680cbfe0 2023-11-26T07:22:01.095 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:01.092+0000 7fc46e8f4640 1 --2- 172.21.15.143:0/3688624290 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fc4680cbfe0 0x7fc46806c540 secure :-1 s=READY pgs=320 cs=0 l=1 rev1=1 crypto rx=0x7fc460009d90 tx=0x7fc46001c920 comp rx=0 tx=0).ready entity=mon.1 client_cookie=4db6d933880179b3 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:22:01.095 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:01.092+0000 7fc46d0f1640 1 -- 172.21.15.143:0/3688624290 <== mon.1 v2:172.21.15.188:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc4600a59a0 con 0x7fc4680cbfe0 2023-11-26T07:22:01.096 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:01.092+0000 7fc46d0f1640 1 -- 172.21.15.143:0/3688624290 <== mon.1 v2:172.21.15.188:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc4600a5b00 con 0x7fc4680cbfe0 2023-11-26T07:22:01.096 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:01.092+0000 7fc46d0f1640 1 -- 172.21.15.143:0/3688624290 <== mon.1 v2:172.21.15.188:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc4600a5e90 con 0x7fc4680cbfe0 2023-11-26T07:22:01.096 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:01.092+0000 7fc470357640 1 -- 172.21.15.143:0/3688624290 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fc4680cbfe0 msgr2=0x7fc46806c540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:22:01.096 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:01.092+0000 7fc470357640 1 --2- 172.21.15.143:0/3688624290 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fc4680cbfe0 0x7fc46806c540 secure :-1 s=READY pgs=320 cs=0 l=1 rev1=1 crypto rx=0x7fc460009d90 tx=0x7fc46001c920 comp rx=0 tx=0).stop 2023-11-26T07:22:01.097 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:01.096+0000 7fc470357640 1 -- 172.21.15.143:0/3688624290 shutdown_connections 2023-11-26T07:22:01.097 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:01.096+0000 7fc470357640 1 --2- 172.21.15.143:0/3688624290 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fc4680cbfe0 0x7fc46806c540 unknown :-1 s=CLOSED pgs=320 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:22:01.097 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:01.096+0000 7fc470357640 1 -- 172.21.15.143:0/3688624290 >> 172.21.15.143:0/3688624290 conn(0x7fc4680b0bf0 msgr2=0x7fc4680b1000 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:22:01.097 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:01.096+0000 7fc470357640 1 -- 172.21.15.143:0/3688624290 shutdown_connections 2023-11-26T07:22:01.097 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:01.096+0000 7fc470357640 1 -- 172.21.15.143:0/3688624290 wait complete. 2023-11-26T07:22:01.098 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:01.096+0000 7fc470357640 1 Processor -- start 2023-11-26T07:22:01.098 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:01.096+0000 7fc470357640 1 -- start start 2023-11-26T07:22:01.099 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:01.096+0000 7fc470357640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fc46806dc70 0x7fc468159960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:22:01.099 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:01.096+0000 7fc470357640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fc4680cbfe0 0x7fc468135c30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:22:01.099 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:01.096+0000 7fc470357640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fc4681449f0 0x7fc468136170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:22:01.099 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:01.096+0000 7fc470357640 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc468159ea0 con 0x7fc4681449f0 2023-11-26T07:22:01.099 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:01.096+0000 7fc470357640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc4680b20e0 con 0x7fc46806dc70 2023-11-26T07:22:01.099 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:01.096+0000 7fc470357640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc46813cf80 con 0x7fc4680cbfe0 2023-11-26T07:22:01.100 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:01.096+0000 7fc46e8f4640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fc4681449f0 0x7fc468136170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:22:01.100 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:01.096+0000 7fc46e8f4640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fc4681449f0 0x7fc468136170 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:57264/0 (socket says 172.21.15.143:57264) 2023-11-26T07:22:01.100 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:01.096+0000 7fc46e8f4640 1 -- 172.21.15.143:0/3796312687 learned_addr learned my addr 172.21.15.143:0/3796312687 (peer_addr_for_me v2:172.21.15.143:0/0) 2023-11-26T07:22:01.101 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:01.096+0000 7fc46e0f3640 1 --2- 172.21.15.143:0/3796312687 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fc46806dc70 0x7fc468159960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:22:01.101 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:01.096+0000 7fc46d8f2640 1 --2- 172.21.15.143:0/3796312687 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fc4680cbfe0 0x7fc468135c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:22:01.101 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:01.096+0000 7fc46e8f4640 1 -- 172.21.15.143:0/3796312687 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fc4680cbfe0 msgr2=0x7fc468135c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:22:01.101 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:01.096+0000 7fc46e8f4640 1 --2- 172.21.15.143:0/3796312687 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fc4680cbfe0 0x7fc468135c30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:22:01.101 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:01.096+0000 7fc46e8f4640 1 -- 172.21.15.143:0/3796312687 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fc46806dc70 msgr2=0x7fc468159960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:22:01.101 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:01.096+0000 7fc46e8f4640 1 --2- 172.21.15.143:0/3796312687 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fc46806dc70 0x7fc468159960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:22:01.101 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:01.096+0000 7fc46e8f4640 1 -- 172.21.15.143:0/3796312687 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc460009a70 con 0x7fc4681449f0 2023-11-26T07:22:01.101 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:01.096+0000 7fc46d8f2640 1 --2- 172.21.15.143:0/3796312687 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fc4680cbfe0 0x7fc468135c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-26T07:22:01.101 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:01.096+0000 7fc46e0f3640 1 --2- 172.21.15.143:0/3796312687 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fc46806dc70 0x7fc468159960 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-26T07:22:01.102 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:01.096+0000 7fc46e8f4640 1 --2- 172.21.15.143:0/3796312687 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fc4681449f0 0x7fc468136170 secure :-1 s=READY pgs=807 cs=0 l=1 rev1=1 crypto rx=0x7fc46001a550 tx=0x7fc4600196c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:22:01.102 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:01.100+0000 7fc45effd640 1 -- 172.21.15.143:0/3796312687 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc4600add90 con 0x7fc4681449f0 2023-11-26T07:22:01.102 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:01.100+0000 7fc45effd640 1 -- 172.21.15.143:0/3796312687 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc460009450 con 0x7fc4681449f0 2023-11-26T07:22:01.102 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:01.100+0000 7fc45effd640 1 -- 172.21.15.143:0/3796312687 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc4600b9670 con 0x7fc4681449f0 2023-11-26T07:22:01.102 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:01.100+0000 7fc470357640 1 -- 172.21.15.143:0/3796312687 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc46813d260 con 0x7fc4681449f0 2023-11-26T07:22:01.102 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:01.100+0000 7fc470357640 1 -- 172.21.15.143:0/3796312687 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc46815a780 con 0x7fc4681449f0 2023-11-26T07:22:01.102 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:01.100+0000 7fc470357640 1 -- 172.21.15.143:0/3796312687 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc468067720 con 0x7fc4681449f0 2023-11-26T07:22:01.103 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:01.100+0000 7fc45effd640 1 -- 172.21.15.143:0/3796312687 <== mon.0 v2:172.21.15.143:3300/0 4 ==== mgrmap(e 9) v1 ==== 86005+0+0 (secure 0 0 0) 0x7fc4600b97d0 con 0x7fc4681449f0 2023-11-26T07:22:01.103 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:01.100+0000 7fc45effd640 1 --2- 172.21.15.143:0/3796312687 >> [v2:172.21.15.143:6832/2150557544,v1:172.21.15.143:6833/2150557544] conn(0x7fc44c066620 0x7fc44c068ae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:22:01.103 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:01.100+0000 7fc45effd640 1 -- 172.21.15.143:0/3796312687 <== mon.0 v2:172.21.15.143:3300/0 5 ==== osd_map(59..59 src has 1..59) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fc46011f910 con 0x7fc4681449f0 2023-11-26T07:22:01.103 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:01.100+0000 7fc46e0f3640 1 --2- 172.21.15.143:0/3796312687 >> [v2:172.21.15.143:6832/2150557544,v1:172.21.15.143:6833/2150557544] conn(0x7fc44c066620 0x7fc44c068ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:22:01.104 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:01.100+0000 7fc46e0f3640 1 --2- 172.21.15.143:0/3796312687 >> [v2:172.21.15.143:6832/2150557544,v1:172.21.15.143:6833/2150557544] conn(0x7fc44c066620 0x7fc44c068ae0 secure :-1 s=READY pgs=386 cs=0 l=1 rev1=1 crypto rx=0x7fc4640011f0 tx=0x7fc464009040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:22:01.107 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:01.104+0000 7fc45effd640 1 -- 172.21.15.143:0/3796312687 <== mon.0 v2:172.21.15.143:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172649 (secure 0 0 0) 0x7fc4600ecb10 con 0x7fc4681449f0 2023-11-26T07:22:01.259 INFO:tasks.ceph.mds.b.smithi188.stderr:2023-11-26T07:22:01.254+0000 7f72a92dd640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2023-11-26T07:22:01.312 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:01.308+0000 7fc470357640 1 -- 172.21.15.143:0/3796312687 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]} v 0) v1 -- 0x7fc46806e210 con 0x7fc4681449f0 2023-11-26T07:22:01.315 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:01.312+0000 7fc45effd640 1 -- 172.21.15.143:0/3796312687 <== mon.0 v2:172.21.15.143:3300/0 7 ==== mon_command_ack([{"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]}]=0 updated caps for client.0 v15) v1 ==== 190+0+147 (secure 0 0 0) 0x7fc4600b20e0 con 0x7fc4681449f0 2023-11-26T07:22:01.315 INFO:teuthology.orchestra.run.smithi143.stdout:[client.0] 2023-11-26T07:22:01.315 INFO:teuthology.orchestra.run.smithi143.stdout: key = AQDm8GJlWMh9FBAAslVzi6fm4/wt7vr/rHCHHQ== 2023-11-26T07:22:01.315 INFO:teuthology.orchestra.run.smithi143.stdout: caps mds = "allow" 2023-11-26T07:22:01.316 INFO:teuthology.orchestra.run.smithi143.stdout: caps mon = "allow r" 2023-11-26T07:22:01.316 INFO:teuthology.orchestra.run.smithi143.stdout: caps osd = "allow rw tag cephfs data=cephfs" 2023-11-26T07:22:01.316 INFO:teuthology.orchestra.run.smithi143.stderr:updated caps for client.0 2023-11-26T07:22:01.317 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:01.316+0000 7fc470357640 1 -- 172.21.15.143:0/3796312687 >> [v2:172.21.15.143:6832/2150557544,v1:172.21.15.143:6833/2150557544] conn(0x7fc44c066620 msgr2=0x7fc44c068ae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:22:01.318 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:01.316+0000 7fc470357640 1 --2- 172.21.15.143:0/3796312687 >> [v2:172.21.15.143:6832/2150557544,v1:172.21.15.143:6833/2150557544] conn(0x7fc44c066620 0x7fc44c068ae0 secure :-1 s=READY pgs=386 cs=0 l=1 rev1=1 crypto rx=0x7fc4640011f0 tx=0x7fc464009040 comp rx=0 tx=0).stop 2023-11-26T07:22:01.318 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:01.316+0000 7fc470357640 1 -- 172.21.15.143:0/3796312687 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fc4681449f0 msgr2=0x7fc468136170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:22:01.318 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:01.316+0000 7fc470357640 1 --2- 172.21.15.143:0/3796312687 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fc4681449f0 0x7fc468136170 secure :-1 s=READY pgs=807 cs=0 l=1 rev1=1 crypto rx=0x7fc46001a550 tx=0x7fc4600196c0 comp rx=0 tx=0).stop 2023-11-26T07:22:01.318 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:01.316+0000 7fc470357640 1 -- 172.21.15.143:0/3796312687 shutdown_connections 2023-11-26T07:22:01.318 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:01.316+0000 7fc470357640 1 --2- 172.21.15.143:0/3796312687 >> [v2:172.21.15.143:6832/2150557544,v1:172.21.15.143:6833/2150557544] conn(0x7fc44c066620 0x7fc44c068ae0 unknown :-1 s=CLOSED pgs=386 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:22:01.318 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:01.316+0000 7fc470357640 1 --2- 172.21.15.143:0/3796312687 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fc4681449f0 0x7fc468136170 unknown :-1 s=CLOSED pgs=807 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:22:01.318 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:01.316+0000 7fc470357640 1 --2- 172.21.15.143:0/3796312687 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fc4680cbfe0 0x7fc468135c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:22:01.319 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:01.316+0000 7fc470357640 1 --2- 172.21.15.143:0/3796312687 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fc46806dc70 0x7fc468159960 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:22:01.319 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:01.316+0000 7fc470357640 1 -- 172.21.15.143:0/3796312687 >> 172.21.15.143:0/3796312687 conn(0x7fc4680b0bf0 msgr2=0x7fc4680cee60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:22:01.319 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:01.316+0000 7fc470357640 1 -- 172.21.15.143:0/3796312687 shutdown_connections 2023-11-26T07:22:01.319 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:01.316+0000 7fc470357640 1 -- 172.21.15.143:0/3796312687 wait complete. 2023-11-26T07:22:01.336 DEBUG:teuthology.orchestra.run.smithi143:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-11-26T07:22:01.473 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:01.472+0000 7f5e63e32640 1 Processor -- start 2023-11-26T07:22:01.473 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:01.472+0000 7f5e63e32640 1 -- start start 2023-11-26T07:22:01.475 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:01.472+0000 7f5e63e32640 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7f5e5c15d6e0 0x7f5e5c0d0df0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:22:01.475 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:01.472+0000 7f5e63e32640 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7f5e5c0b2160 0x7f5e5c0d14b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:22:01.475 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:01.472+0000 7f5e63e32640 1 -- --> v1:172.21.15.143:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5e5c0cf0e0 con 0x7f5e5c0d19f0 2023-11-26T07:22:01.475 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:01.472+0000 7f5e63e32640 1 -- --> v2:172.21.15.143:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5e5c15dac0 con 0x7f5e5c15d6e0 2023-11-26T07:22:01.476 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:01.472+0000 7f5e63e32640 1 -- --> v2:172.21.15.188:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5e5c0d1e80 con 0x7f5e5c0b2160 2023-11-26T07:22:01.476 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:01.472+0000 7f5e61bce640 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7f5e5c15d6e0 0x7f5e5c0d0df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:22:01.476 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:01.472+0000 7f5e61bce640 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7f5e5c15d6e0 0x7f5e5c0d0df0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:57270/0 (socket says 172.21.15.143:57270) 2023-11-26T07:22:01.476 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:01.472+0000 7f5e61bce640 1 -- 172.21.15.143:0/3808312409 learned_addr learned my addr 172.21.15.143:0/3808312409 (peer_addr_for_me v2:172.21.15.143:0/0) 2023-11-26T07:22:01.476 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:01.472+0000 7f5e623cf640 1 --1- >> v1:172.21.15.143:6789/0 conn(0x7f5e5c0d19f0 0x7f5e5c0cfbf0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.143:6789/0 says I am v1:172.21.15.143:57290/0 (socket says 172.21.15.143:57290) 2023-11-26T07:22:01.476 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:01.472+0000 7f5e613cd640 1 --2- 172.21.15.143:0/3808312409 >> v2:172.21.15.188:3300/0 conn(0x7f5e5c0b2160 0x7f5e5c0d14b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:22:01.476 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:01.472+0000 7f5e61bce640 1 -- 172.21.15.143:0/3808312409 >> v2:172.21.15.188:3300/0 conn(0x7f5e5c0b2160 msgr2=0x7f5e5c0d14b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-26T07:22:01.476 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:01.472+0000 7f5e61bce640 1 --2- 172.21.15.143:0/3808312409 >> v2:172.21.15.188:3300/0 conn(0x7f5e5c0b2160 0x7f5e5c0d14b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:22:01.476 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:01.472+0000 7f5e61bce640 1 -- 172.21.15.143:0/3808312409 >> v1:172.21.15.143:6789/0 conn(0x7f5e5c0d19f0 legacy=0x7f5e5c0cfbf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:22:01.476 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:01.472+0000 7f5e60bcc640 1 -- 172.21.15.143:0/3808312409 <== mon.0 v1:172.21.15.143:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3331155021 0 0) 0x7f5e5c0cf0e0 con 0x7f5e5c0d19f0 2023-11-26T07:22:01.476 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:01.472+0000 7f5e61bce640 1 -- 172.21.15.143:0/3808312409 --> v2:172.21.15.143:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5e5c0ceb20 con 0x7f5e5c15d6e0 2023-11-26T07:22:01.478 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:01.476+0000 7f5e61bce640 1 --2- 172.21.15.143:0/3808312409 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f5e5c15d6e0 0x7f5e5c0d0df0 secure :-1 s=READY pgs=809 cs=0 l=1 rev1=1 crypto rx=0x7f5e5c0ca2a0 tx=0x7f5e5801c500 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4b09d687b9a0ad3c server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:22:01.478 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:01.476+0000 7f5e60bcc640 1 -- 172.21.15.143:0/3808312409 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5e580ab5f0 con 0x7f5e5c15d6e0 2023-11-26T07:22:01.478 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:01.476+0000 7f5e60bcc640 1 -- 172.21.15.143:0/3808312409 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5e580ab750 con 0x7f5e5c15d6e0 2023-11-26T07:22:01.478 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:01.476+0000 7f5e60bcc640 1 -- 172.21.15.143:0/3808312409 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5e580abae0 con 0x7f5e5c15d6e0 2023-11-26T07:22:01.479 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:01.476+0000 7f5e63e32640 1 -- 172.21.15.143:0/3808312409 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f5e5c15d6e0 msgr2=0x7f5e5c0d0df0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:22:01.479 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:01.476+0000 7f5e63e32640 1 --2- 172.21.15.143:0/3808312409 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f5e5c15d6e0 0x7f5e5c0d0df0 secure :-1 s=READY pgs=809 cs=0 l=1 rev1=1 crypto rx=0x7f5e5c0ca2a0 tx=0x7f5e5801c500 comp rx=0 tx=0).stop 2023-11-26T07:22:01.479 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:01.476+0000 7f5e63e32640 1 -- 172.21.15.143:0/3808312409 shutdown_connections 2023-11-26T07:22:01.480 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:01.476+0000 7f5e63e32640 1 --2- 172.21.15.143:0/3808312409 >> v2:172.21.15.188:3300/0 conn(0x7f5e5c0b2160 0x7f5e5c0d14b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:22:01.480 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:01.476+0000 7f5e63e32640 1 --2- 172.21.15.143:0/3808312409 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f5e5c15d6e0 0x7f5e5c0d0df0 unknown :-1 s=CLOSED pgs=809 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:22:01.480 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:01.476+0000 7f5e63e32640 1 -- 172.21.15.143:0/3808312409 >> 172.21.15.143:0/3808312409 conn(0x7f5e5c09c550 msgr2=0x7f5e5c0d3220 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:22:01.480 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:01.476+0000 7f5e63e32640 1 -- 172.21.15.143:0/3808312409 shutdown_connections 2023-11-26T07:22:01.480 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:01.476+0000 7f5e63e32640 1 -- 172.21.15.143:0/3808312409 wait complete. 2023-11-26T07:22:01.481 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:01.480+0000 7f5e63e32640 1 Processor -- start 2023-11-26T07:22:01.481 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:01.480+0000 7f5e63e32640 1 -- start start 2023-11-26T07:22:01.482 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:01.480+0000 7f5e63e32640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f5e5c0b2160 0x7f5e5c0ca2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:22:01.482 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:01.480+0000 7f5e63e32640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f5e5c0d19f0 0x7f5e5c15b360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:22:01.483 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:01.480+0000 7f5e63e32640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f5e5c15d6e0 0x7f5e5c0b3990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:22:01.483 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:01.480+0000 7f5e63e32640 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5e5c15af40 con 0x7f5e5c0d19f0 2023-11-26T07:22:01.483 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:01.480+0000 7f5e63e32640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5e5c0ca830 con 0x7f5e5c0b2160 2023-11-26T07:22:01.483 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:01.480+0000 7f5e63e32640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5e5c15f9b0 con 0x7f5e5c15d6e0 2023-11-26T07:22:01.483 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:01.480+0000 7f5e613cd640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f5e5c0d19f0 0x7f5e5c15b360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:22:01.483 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:01.480+0000 7f5e613cd640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f5e5c0d19f0 0x7f5e5c15b360 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:57274/0 (socket says 172.21.15.143:57274) 2023-11-26T07:22:01.483 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:01.480+0000 7f5e613cd640 1 -- 172.21.15.143:0/3719662884 learned_addr learned my addr 172.21.15.143:0/3719662884 (peer_addr_for_me v2:172.21.15.143:0/0) 2023-11-26T07:22:01.483 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:01.480+0000 7f5e61bce640 1 --2- 172.21.15.143:0/3719662884 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f5e5c0b2160 0x7f5e5c0ca2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:22:01.483 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:01.480+0000 7f5e623cf640 1 --2- 172.21.15.143:0/3719662884 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f5e5c15d6e0 0x7f5e5c0b3990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:22:01.483 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:01.480+0000 7f5e613cd640 1 -- 172.21.15.143:0/3719662884 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f5e5c15d6e0 msgr2=0x7f5e5c0b3990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:22:01.483 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:01.480+0000 7f5e613cd640 1 --2- 172.21.15.143:0/3719662884 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f5e5c15d6e0 0x7f5e5c0b3990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:22:01.483 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:01.480+0000 7f5e613cd640 1 -- 172.21.15.143:0/3719662884 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f5e5c0b2160 msgr2=0x7f5e5c0ca2f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:22:01.483 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:01.480+0000 7f5e613cd640 1 --2- 172.21.15.143:0/3719662884 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f5e5c0b2160 0x7f5e5c0ca2f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:22:01.484 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:01.480+0000 7f5e613cd640 1 -- 172.21.15.143:0/3719662884 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5e580094d0 con 0x7f5e5c0d19f0 2023-11-26T07:22:01.484 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:01.480+0000 7f5e613cd640 1 --2- 172.21.15.143:0/3719662884 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f5e5c0d19f0 0x7f5e5c15b360 secure :-1 s=READY pgs=810 cs=0 l=1 rev1=1 crypto rx=0x7f5e4c00d800 tx=0x7f5e4c00dcc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:22:01.484 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:01.480+0000 7f5e527fc640 1 -- 172.21.15.143:0/3719662884 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5e4c00b6c0 con 0x7f5e5c0d19f0 2023-11-26T07:22:01.485 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:01.480+0000 7f5e527fc640 1 -- 172.21.15.143:0/3719662884 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5e4c00b820 con 0x7f5e5c0d19f0 2023-11-26T07:22:01.485 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:01.480+0000 7f5e527fc640 1 -- 172.21.15.143:0/3719662884 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5e4c00bb90 con 0x7f5e5c0d19f0 2023-11-26T07:22:01.485 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:01.480+0000 7f5e63e32640 1 -- 172.21.15.143:0/3719662884 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5e5c15fc90 con 0x7f5e5c0d19f0 2023-11-26T07:22:01.485 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:01.480+0000 7f5e63e32640 1 -- 172.21.15.143:0/3719662884 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5e5c15ab60 con 0x7f5e5c0d19f0 2023-11-26T07:22:01.485 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:01.484+0000 7f5e63e32640 1 -- 172.21.15.143:0/3719662884 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5e5c067680 con 0x7f5e5c0d19f0 2023-11-26T07:22:01.485 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:01.484+0000 7f5e527fc640 1 -- 172.21.15.143:0/3719662884 <== mon.0 v2:172.21.15.143:3300/0 4 ==== mgrmap(e 9) v1 ==== 86005+0+0 (secure 0 0 0) 0x7f5e4c00bcf0 con 0x7f5e5c0d19f0 2023-11-26T07:22:01.486 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:01.484+0000 7f5e527fc640 1 --2- 172.21.15.143:0/3719662884 >> [v2:172.21.15.143:6832/2150557544,v1:172.21.15.143:6833/2150557544] conn(0x7f5e40066620 0x7f5e40068ae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:22:01.486 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:01.484+0000 7f5e527fc640 1 -- 172.21.15.143:0/3719662884 <== mon.0 v2:172.21.15.143:3300/0 5 ==== osd_map(59..59 src has 1..59) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f5e4c084530 con 0x7f5e5c0d19f0 2023-11-26T07:22:01.486 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:01.484+0000 7f5e61bce640 1 --2- 172.21.15.143:0/3719662884 >> [v2:172.21.15.143:6832/2150557544,v1:172.21.15.143:6833/2150557544] conn(0x7f5e40066620 0x7f5e40068ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:22:01.487 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:01.484+0000 7f5e61bce640 1 --2- 172.21.15.143:0/3719662884 >> [v2:172.21.15.143:6832/2150557544,v1:172.21.15.143:6833/2150557544] conn(0x7f5e40066620 0x7f5e40068ae0 secure :-1 s=READY pgs=387 cs=0 l=1 rev1=1 crypto rx=0x7f5e580af000 tx=0x7f5e580b6040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:22:01.490 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:01.488+0000 7f5e527fc640 1 -- 172.21.15.143:0/3719662884 <== mon.0 v2:172.21.15.143:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172649 (secure 0 0 0) 0x7f5e4c051870 con 0x7f5e5c0d19f0 2023-11-26T07:22:01.671 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:01.668+0000 7f5e63e32640 1 -- 172.21.15.143:0/3719662884 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f5e5c155c50 con 0x7f5e5c0d19f0 2023-11-26T07:22:01.672 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:01.668+0000 7f5e527fc640 1 -- 172.21.15.143:0/3719662884 <== mon.0 v2:172.21.15.143:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 34 v34) v1 ==== 94+0+4925 (secure 0 0 0) 0x7f5e4c054e90 con 0x7f5e5c0d19f0 2023-11-26T07:22:01.672 INFO:teuthology.orchestra.run.smithi143.stdout: 2023-11-26T07:22:01.672 INFO:teuthology.orchestra.run.smithi143.stdout:{"epoch":34,"default_fscid":4,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5612,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.143:6837/1372238588","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6836","nonce":1372238588},{"type":"v1","addr":"172.21.15.143:6837","nonce":1372238588}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":29},{"gid":6103,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.143:6835/1078738477","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6834","nonce":1078738477},{"type":"v1","addr":"172.21.15.143:6835","nonce":1078738477}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":29},{"gid":6118,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.188:6834/1634083777","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6832","nonce":1634083777},{"type":"v1","addr":"172.21.15.188:6834","nonce":1634083777}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":29}],"filesystems":[{"mdsmap":{"epoch":34,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-11-26T07:21:55.656168+0000","modified":"2023-11-26T07:21:58.674916+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":6121},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6121":{"gid":6121,"name":"b","rank":0,"incarnation":30,"state":"up:active","state_seq":3,"addr":"172.21.15.188:6835/1958716738","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6833","nonce":1958716738},{"type":"v1","addr":"172.21.15.188:6835","nonce":1958716738}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[9],"metadata_pool":8,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":4}]} 2023-11-26T07:22:01.673 INFO:teuthology.orchestra.run.smithi143.stderr:dumped fsmap epoch 34 2023-11-26T07:22:01.674 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:01.672+0000 7f5e63e32640 1 -- 172.21.15.143:0/3719662884 >> [v2:172.21.15.143:6832/2150557544,v1:172.21.15.143:6833/2150557544] conn(0x7f5e40066620 msgr2=0x7f5e40068ae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:22:01.674 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:01.672+0000 7f5e63e32640 1 --2- 172.21.15.143:0/3719662884 >> [v2:172.21.15.143:6832/2150557544,v1:172.21.15.143:6833/2150557544] conn(0x7f5e40066620 0x7f5e40068ae0 secure :-1 s=READY pgs=387 cs=0 l=1 rev1=1 crypto rx=0x7f5e580af000 tx=0x7f5e580b6040 comp rx=0 tx=0).stop 2023-11-26T07:22:01.675 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:01.672+0000 7f5e63e32640 1 -- 172.21.15.143:0/3719662884 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f5e5c0d19f0 msgr2=0x7f5e5c15b360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:22:01.675 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:01.672+0000 7f5e63e32640 1 --2- 172.21.15.143:0/3719662884 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f5e5c0d19f0 0x7f5e5c15b360 secure :-1 s=READY pgs=810 cs=0 l=1 rev1=1 crypto rx=0x7f5e4c00d800 tx=0x7f5e4c00dcc0 comp rx=0 tx=0).stop 2023-11-26T07:22:01.675 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:01.672+0000 7f5e63e32640 1 -- 172.21.15.143:0/3719662884 shutdown_connections 2023-11-26T07:22:01.675 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:01.672+0000 7f5e63e32640 1 --2- 172.21.15.143:0/3719662884 >> [v2:172.21.15.143:6832/2150557544,v1:172.21.15.143:6833/2150557544] conn(0x7f5e40066620 0x7f5e40068ae0 unknown :-1 s=CLOSED pgs=387 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:22:01.675 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:01.672+0000 7f5e63e32640 1 --2- 172.21.15.143:0/3719662884 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f5e5c15d6e0 0x7f5e5c0b3990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:22:01.675 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:01.672+0000 7f5e63e32640 1 --2- 172.21.15.143:0/3719662884 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f5e5c0d19f0 0x7f5e5c15b360 unknown :-1 s=CLOSED pgs=810 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:22:01.675 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:01.672+0000 7f5e63e32640 1 --2- 172.21.15.143:0/3719662884 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f5e5c0b2160 0x7f5e5c0ca2f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:22:01.675 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:01.672+0000 7f5e63e32640 1 -- 172.21.15.143:0/3719662884 >> 172.21.15.143:0/3719662884 conn(0x7f5e5c09c550 msgr2=0x7f5e5c0d3220 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:22:01.675 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:01.672+0000 7f5e63e32640 1 -- 172.21.15.143:0/3719662884 shutdown_connections 2023-11-26T07:22:01.675 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:01.672+0000 7f5e63e32640 1 -- 172.21.15.143:0/3719662884 wait complete. 2023-11-26T07:22:02.096 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 34, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2023-11-26T07:21:55.656168+0000', 'modified': '2023-11-26T07:21:58.674916+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 6121}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_6121': {'gid': 6121, 'name': 'b', 'rank': 0, 'incarnation': 30, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.188:6835/1958716738', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.188:6833', 'nonce': 1958716738}, {'type': 'v1', 'addr': '172.21.15.188:6835', 'nonce': 1958716738}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138322906710015, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}}}, 'data_pools': [9], 'metadata_pool': 8, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0} 2023-11-26T07:22:02.096 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2023-11-26T07:22:02.097 DEBUG:teuthology.orchestra.run.smithi143:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.b status 2023-11-26T07:22:02.236 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:02.236+0000 7fc82726b640 1 Processor -- start 2023-11-26T07:22:02.237 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:02.236+0000 7fc82726b640 1 -- start start 2023-11-26T07:22:02.238 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:02.236+0000 7fc82726b640 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7fc8201557e0 0x7fc82006e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:22:02.238 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:02.236+0000 7fc82726b640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fc8200abef0 0x7fc820071bf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:22:02.239 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:02.236+0000 7fc82726b640 1 -- --> v1:172.21.15.188:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc82015f4f0 con 0x7fc82015f980 2023-11-26T07:22:02.239 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:02.236+0000 7fc82726b640 1 -- --> v2:172.21.15.143:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc82006e830 con 0x7fc8201557e0 2023-11-26T07:22:02.239 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:02.236+0000 7fc82726b640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc8200ac6b0 con 0x7fc8200abef0 2023-11-26T07:22:02.239 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:02.236+0000 7fc825007640 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7fc8201557e0 0x7fc82006e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:22:02.239 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:02.236+0000 7fc825007640 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7fc8201557e0 0x7fc82006e2f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:57276/0 (socket says 172.21.15.143:57276) 2023-11-26T07:22:02.239 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:02.236+0000 7fc825007640 1 -- 172.21.15.143:0/1746205201 learned_addr learned my addr 172.21.15.143:0/1746205201 (peer_addr_for_me v2:172.21.15.143:0/0) 2023-11-26T07:22:02.239 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:02.236+0000 7fc825007640 1 -- 172.21.15.143:0/1746205201 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fc8200abef0 msgr2=0x7fc820071bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-26T07:22:02.240 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:02.236+0000 7fc824806640 1 --2- 172.21.15.143:0/1746205201 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fc8200abef0 0x7fc820071bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:22:02.240 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:02.236+0000 7fc825007640 1 --2- 172.21.15.143:0/1746205201 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fc8200abef0 0x7fc820071bf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:22:02.240 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:02.236+0000 7fc825007640 1 -- 172.21.15.143:0/1746205201 >> v1:172.21.15.188:6789/0 conn(0x7fc82015f980 legacy=0x7fc8200add10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-26T07:22:02.240 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:02.236+0000 7fc825007640 1 -- 172.21.15.143:0/1746205201 --> v2:172.21.15.143:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc820155bc0 con 0x7fc8201557e0 2023-11-26T07:22:02.240 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:02.236+0000 7fc824806640 1 --2- 172.21.15.143:0/1746205201 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fc8200abef0 0x7fc820071bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-26T07:22:02.241 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:02.240+0000 7fc825007640 1 --2- 172.21.15.143:0/1746205201 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fc8201557e0 0x7fc82006e2f0 secure :-1 s=READY pgs=811 cs=0 l=1 rev1=1 crypto rx=0x7fc81c009870 tx=0x7fc81c01c4d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=481fa6680b4ea2a4 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:22:02.242 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:02.240+0000 7fc817fff640 1 -- 172.21.15.143:0/1746205201 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc81c0ab600 con 0x7fc8201557e0 2023-11-26T07:22:02.242 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:02.240+0000 7fc817fff640 1 -- 172.21.15.143:0/1746205201 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc81c0a7050 con 0x7fc8201557e0 2023-11-26T07:22:02.242 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:02.240+0000 7fc817fff640 1 -- 172.21.15.143:0/1746205201 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc81c0ab980 con 0x7fc8201557e0 2023-11-26T07:22:02.243 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:02.240+0000 7fc82726b640 1 -- 172.21.15.143:0/1746205201 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fc8201557e0 msgr2=0x7fc82006e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:22:02.243 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:02.240+0000 7fc82726b640 1 --2- 172.21.15.143:0/1746205201 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fc8201557e0 0x7fc82006e2f0 secure :-1 s=READY pgs=811 cs=0 l=1 rev1=1 crypto rx=0x7fc81c009870 tx=0x7fc81c01c4d0 comp rx=0 tx=0).stop 2023-11-26T07:22:02.243 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:02.240+0000 7fc82726b640 1 -- 172.21.15.143:0/1746205201 shutdown_connections 2023-11-26T07:22:02.243 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:02.240+0000 7fc82726b640 1 --2- 172.21.15.143:0/1746205201 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fc8200abef0 0x7fc820071bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:22:02.243 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:02.240+0000 7fc82726b640 1 --2- 172.21.15.143:0/1746205201 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fc8201557e0 0x7fc82006e2f0 unknown :-1 s=CLOSED pgs=811 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:22:02.243 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:02.240+0000 7fc82726b640 1 -- 172.21.15.143:0/1746205201 >> 172.21.15.143:0/1746205201 conn(0x7fc8200b0c10 msgr2=0x7fc8200b1020 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:22:02.243 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:02.240+0000 7fc82726b640 1 -- 172.21.15.143:0/1746205201 shutdown_connections 2023-11-26T07:22:02.243 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:02.240+0000 7fc82726b640 1 -- 172.21.15.143:0/1746205201 wait complete. 2023-11-26T07:22:02.244 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:02.244+0000 7fc82726b640 1 Processor -- start 2023-11-26T07:22:02.245 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:02.244+0000 7fc82726b640 1 -- start start 2023-11-26T07:22:02.247 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:02.244+0000 7fc82726b640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fc8200abef0 0x7fc82015d6d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:22:02.247 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:02.244+0000 7fc82726b640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fc8201557e0 0x7fc820099a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:22:02.247 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:02.244+0000 7fc82726b640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fc82015f980 0x7fc82009b2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:22:02.247 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:02.244+0000 7fc82726b640 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc82009ae60 con 0x7fc8200abef0 2023-11-26T07:22:02.247 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:02.244+0000 7fc82726b640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc820099fb0 con 0x7fc8201557e0 2023-11-26T07:22:02.247 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:02.244+0000 7fc82726b640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc82009a250 con 0x7fc82015f980 2023-11-26T07:22:02.247 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:02.244+0000 7fc825007640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fc8200abef0 0x7fc82015d6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:22:02.247 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:02.244+0000 7fc825007640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fc8200abef0 0x7fc82015d6d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:57290/0 (socket says 172.21.15.143:57290) 2023-11-26T07:22:02.247 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:02.244+0000 7fc825007640 1 -- 172.21.15.143:0/1497609075 learned_addr learned my addr 172.21.15.143:0/1497609075 (peer_addr_for_me v2:172.21.15.143:0/0) 2023-11-26T07:22:02.247 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:02.244+0000 7fc824806640 1 --2- 172.21.15.143:0/1497609075 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fc8201557e0 0x7fc820099a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:22:02.247 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:02.244+0000 7fc825808640 1 --2- 172.21.15.143:0/1497609075 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fc82015f980 0x7fc82009b2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:22:02.248 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:02.244+0000 7fc825007640 1 -- 172.21.15.143:0/1497609075 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fc82015f980 msgr2=0x7fc82009b2f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:22:02.248 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:02.244+0000 7fc825007640 1 --2- 172.21.15.143:0/1497609075 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fc82015f980 0x7fc82009b2f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:22:02.248 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:02.244+0000 7fc825007640 1 -- 172.21.15.143:0/1497609075 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fc8201557e0 msgr2=0x7fc820099a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:22:02.248 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:02.244+0000 7fc825007640 1 --2- 172.21.15.143:0/1497609075 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fc8201557e0 0x7fc820099a70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:22:02.248 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:02.244+0000 7fc825007640 1 -- 172.21.15.143:0/1497609075 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc81c0094d0 con 0x7fc8200abef0 2023-11-26T07:22:02.248 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:02.244+0000 7fc825808640 1 --2- 172.21.15.143:0/1497609075 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fc82015f980 0x7fc82009b2f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-26T07:22:02.248 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:02.244+0000 7fc825007640 1 --2- 172.21.15.143:0/1497609075 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fc8200abef0 0x7fc82015d6d0 secure :-1 s=READY pgs=812 cs=0 l=1 rev1=1 crypto rx=0x7fc81c0ae040 tx=0x7fc81c01ad10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:22:02.248 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:02.244+0000 7fc815ffb640 1 -- 172.21.15.143:0/1497609075 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc81c0bc070 con 0x7fc8200abef0 2023-11-26T07:22:02.248 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:02.244+0000 7fc815ffb640 1 -- 172.21.15.143:0/1497609075 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc81c0b6420 con 0x7fc8200abef0 2023-11-26T07:22:02.248 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:02.244+0000 7fc82726b640 1 -- 172.21.15.143:0/1497609075 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc82009a530 con 0x7fc8200abef0 2023-11-26T07:22:02.248 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:02.244+0000 7fc815ffb640 1 -- 172.21.15.143:0/1497609075 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc81c0b67f0 con 0x7fc8200abef0 2023-11-26T07:22:02.249 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:02.244+0000 7fc82726b640 1 -- 172.21.15.143:0/1497609075 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc82009aaf0 con 0x7fc8200abef0 2023-11-26T07:22:02.249 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:02.248+0000 7fc815ffb640 1 -- 172.21.15.143:0/1497609075 <== mon.0 v2:172.21.15.143:3300/0 4 ==== mgrmap(e 9) v1 ==== 86005+0+0 (secure 0 0 0) 0x7fc81c0b2040 con 0x7fc8200abef0 2023-11-26T07:22:02.250 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:02.248+0000 7fc815ffb640 1 --2- 172.21.15.143:0/1497609075 >> [v2:172.21.15.143:6832/2150557544,v1:172.21.15.143:6833/2150557544] conn(0x7fc804066650 0x7fc804068b10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:22:02.250 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:02.248+0000 7fc815ffb640 1 -- 172.21.15.143:0/1497609075 <== mon.0 v2:172.21.15.143:3300/0 5 ==== osd_map(59..59 src has 1..59) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fc81c11dbf0 con 0x7fc8200abef0 2023-11-26T07:22:02.250 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:02.248+0000 7fc824806640 1 --2- 172.21.15.143:0/1497609075 >> [v2:172.21.15.143:6832/2150557544,v1:172.21.15.143:6833/2150557544] conn(0x7fc804066650 0x7fc804068b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:22:02.251 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:02.248+0000 7fc82a5e0000 1 Processor -- start 2023-11-26T07:22:02.251 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:02.248+0000 7fc82a5e0000 1 -- start start 2023-11-26T07:22:02.252 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:02.248+0000 7fc824806640 1 --2- 172.21.15.143:0/1497609075 >> [v2:172.21.15.143:6832/2150557544,v1:172.21.15.143:6833/2150557544] conn(0x7fc804066650 0x7fc804068b10 secure :-1 s=READY pgs=388 cs=0 l=1 rev1=1 crypto rx=0x7fc82015ac70 tx=0x7fc810008040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:22:02.252 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:02.248+0000 7fc82a5e0000 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x5562c26edc90 0x5562c26ee070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:22:02.252 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:02.248+0000 7fc82a5e0000 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x5562c26ee5b0 0x5562c26a3de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:22:02.252 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:02.248+0000 7fc82a5e0000 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5562c26a4320 0x5562c26a4700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:22:02.252 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:02.248+0000 7fc82a5e0000 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x5562c250e130 con 0x5562c26ee5b0 2023-11-26T07:22:02.252 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:02.248+0000 7fc82a5e0000 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x5562c24fb9a0 con 0x5562c26a4320 2023-11-26T07:22:02.252 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:02.248+0000 7fc82a5e0000 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x5562c24f8ed0 con 0x5562c26edc90 2023-11-26T07:22:02.252 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:02.248+0000 7fc824806640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x5562c26ee5b0 0x5562c26a3de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:22:02.253 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:02.248+0000 7fc824806640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x5562c26ee5b0 0x5562c26a3de0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:57302/0 (socket says 172.21.15.143:57302) 2023-11-26T07:22:02.253 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:02.248+0000 7fc824806640 1 -- 172.21.15.143:0/4177826951 learned_addr learned my addr 172.21.15.143:0/4177826951 (peer_addr_for_me v2:172.21.15.143:0/0) 2023-11-26T07:22:02.253 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:02.248+0000 7fc825808640 1 --2- 172.21.15.143:0/4177826951 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5562c26a4320 0x5562c26a4700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:22:02.253 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:02.248+0000 7fc825007640 1 --2- 172.21.15.143:0/4177826951 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x5562c26edc90 0x5562c26ee070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:22:02.253 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:02.252+0000 7fc824806640 1 -- 172.21.15.143:0/4177826951 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x5562c26edc90 msgr2=0x5562c26ee070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-26T07:22:02.253 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:02.252+0000 7fc824806640 1 --2- 172.21.15.143:0/4177826951 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x5562c26edc90 0x5562c26ee070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:22:02.253 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:02.252+0000 7fc824806640 1 -- 172.21.15.143:0/4177826951 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5562c26a4320 msgr2=0x5562c26a4700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-26T07:22:02.253 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:02.252+0000 7fc824806640 1 --2- 172.21.15.143:0/4177826951 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5562c26a4320 0x5562c26a4700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:22:02.253 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:02.252+0000 7fc824806640 1 -- 172.21.15.143:0/4177826951 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5562c21ec890 con 0x5562c26ee5b0 2023-11-26T07:22:02.254 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:02.252+0000 7fc824806640 1 --2- 172.21.15.143:0/4177826951 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x5562c26ee5b0 0x5562c26a3de0 secure :-1 s=READY pgs=813 cs=0 l=1 rev1=1 crypto rx=0x7fc81000fc70 tx=0x7fc81000af90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8a622b493c893797 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:22:02.255 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:02.252+0000 7fc80affd640 1 -- 172.21.15.143:0/4177826951 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc810013620 con 0x5562c26ee5b0 2023-11-26T07:22:02.255 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:02.252+0000 7fc80affd640 1 -- 172.21.15.143:0/4177826951 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc81000c040 con 0x5562c26ee5b0 2023-11-26T07:22:02.255 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:02.252+0000 7fc80affd640 1 -- 172.21.15.143:0/4177826951 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc810013980 con 0x5562c26ee5b0 2023-11-26T07:22:02.255 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:02.252+0000 7fc82a5e0000 1 -- 172.21.15.143:0/4177826951 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x5562c26ee5b0 msgr2=0x5562c26a3de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:22:02.255 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:02.252+0000 7fc82a5e0000 1 --2- 172.21.15.143:0/4177826951 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x5562c26ee5b0 0x5562c26a3de0 secure :-1 s=READY pgs=813 cs=0 l=1 rev1=1 crypto rx=0x7fc81000fc70 tx=0x7fc81000af90 comp rx=0 tx=0).stop 2023-11-26T07:22:02.255 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:02.252+0000 7fc82a5e0000 1 -- 172.21.15.143:0/4177826951 shutdown_connections 2023-11-26T07:22:02.255 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:02.252+0000 7fc82a5e0000 1 --2- 172.21.15.143:0/4177826951 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5562c26a4320 0x5562c26a4700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:22:02.255 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:02.252+0000 7fc82a5e0000 1 --2- 172.21.15.143:0/4177826951 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x5562c26ee5b0 0x5562c26a3de0 unknown :-1 s=CLOSED pgs=813 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:22:02.255 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:02.252+0000 7fc82a5e0000 1 --2- 172.21.15.143:0/4177826951 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x5562c26edc90 0x5562c26ee070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:22:02.255 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:02.252+0000 7fc82a5e0000 1 -- 172.21.15.143:0/4177826951 >> 172.21.15.143:0/4177826951 conn(0x5562c26a02d0 msgr2=0x5562c26a26c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:22:02.255 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:02.252+0000 7fc82a5e0000 1 -- 172.21.15.143:0/4177826951 shutdown_connections 2023-11-26T07:22:02.255 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:02.252+0000 7fc82a5e0000 1 -- 172.21.15.143:0/4177826951 wait complete. 2023-11-26T07:22:02.255 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:02.252+0000 7fc82a5e0000 1 -- start start 2023-11-26T07:22:02.256 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:02.252+0000 7fc82a5e0000 1 Processor -- start 2023-11-26T07:22:02.258 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:02.252+0000 7fc82a5e0000 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x5562c26a4320 0x5562c26a8d40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:22:02.258 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:02.252+0000 7fc82a5e0000 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x5562c26edc90 0x5562c26a9280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:22:02.259 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:02.252+0000 7fc82a5e0000 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5562c26ee5b0 0x5562c26bc910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:22:02.259 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:02.252+0000 7fc82a5e0000 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x5562c24f47c0 con 0x5562c26edc90 2023-11-26T07:22:02.259 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:02.252+0000 7fc82a5e0000 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x5562c24acbf0 con 0x5562c26ee5b0 2023-11-26T07:22:02.259 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:02.252+0000 7fc82a5e0000 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x5562c24b2a90 con 0x5562c26a4320 2023-11-26T07:22:02.259 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:02.252+0000 7fc82a5e0000 10 client.0 _ll_register_callbacks cb 0x5562c268f3c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-11-26T07:22:02.259 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:02.252+0000 7fc824806640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x5562c26edc90 0x5562c26a9280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:22:02.259 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:02.252+0000 7fc824806640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x5562c26edc90 0x5562c26a9280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:57316/0 (socket says 172.21.15.143:57316) 2023-11-26T07:22:02.259 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:02.252+0000 7fc824806640 1 -- 172.21.15.143:0/254760995 learned_addr learned my addr 172.21.15.143:0/254760995 (peer_addr_for_me v2:172.21.15.143:0/0) 2023-11-26T07:22:02.259 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:02.252+0000 7fc825007640 1 --2- 172.21.15.143:0/254760995 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x5562c26a4320 0x5562c26a8d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:22:02.259 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:02.252+0000 7fc825808640 1 --2- 172.21.15.143:0/254760995 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5562c26ee5b0 0x5562c26bc910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:22:02.259 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:02.256+0000 7fc824806640 1 -- 172.21.15.143:0/254760995 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x5562c26a4320 msgr2=0x5562c26a8d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-26T07:22:02.259 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:02.256+0000 7fc824806640 1 --2- 172.21.15.143:0/254760995 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x5562c26a4320 0x5562c26a8d40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:22:02.260 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:02.256+0000 7fc824806640 1 -- 172.21.15.143:0/254760995 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5562c26ee5b0 msgr2=0x5562c26bc910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-26T07:22:02.260 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:02.256+0000 7fc824806640 1 --2- 172.21.15.143:0/254760995 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5562c26ee5b0 0x5562c26bc910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:22:02.260 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:02.256+0000 7fc824806640 1 -- 172.21.15.143:0/254760995 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc81000f950 con 0x5562c26edc90 2023-11-26T07:22:02.260 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:02.256+0000 7fc825808640 1 --2- 172.21.15.143:0/254760995 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5562c26ee5b0 0x5562c26bc910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-26T07:22:02.260 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:02.256+0000 7fc824806640 1 --2- 172.21.15.143:0/254760995 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x5562c26edc90 0x5562c26a9280 secure :-1 s=READY pgs=814 cs=0 l=1 rev1=1 crypto rx=0x7fc810020000 tx=0x7fc810004740 comp rx=0 tx=0).ready entity=mon.0 client_cookie=75a8db5aeb10ba3d server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:22:02.260 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:02.256+0000 7fc7fffff640 10 client.0 ms_handle_connect on v2:172.21.15.143:3300/0 2023-11-26T07:22:02.260 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:02.256+0000 7fc7fffff640 1 -- 172.21.15.143:0/254760995 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc810033450 con 0x5562c26edc90 2023-11-26T07:22:02.260 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:02.256+0000 7fc7fffff640 1 -- 172.21.15.143:0/254760995 <== mon.0 v2:172.21.15.143:3300/0 2 ==== osd_map(59..59 src has 1..59) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fc81001a040 con 0x5562c26edc90 2023-11-26T07:22:02.260 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:02.256+0000 7fc82a5e0000 1 -- 172.21.15.143:0/254760995 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x5562c24b6470 con 0x5562c26edc90 2023-11-26T07:22:02.260 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:02.256+0000 7fc7fffff640 1 -- 172.21.15.143:0/254760995 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_get_version_reply(handle=1 version=34) v2 ==== 24+0+0 (secure 0 0 0) 0x7fc8100273f0 con 0x5562c26edc90 2023-11-26T07:22:02.260 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:02.256+0000 7fc82a5e0000 10 client.6555 fetch_fsmap learned FSMap version 34 2023-11-26T07:22:02.260 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:02.256+0000 7fc82a5e0000 1 -- 172.21.15.143:0/254760995 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,fsmap=34,monmap=2+}) v3 -- 0x5562c24b5190 con 0x5562c26edc90 2023-11-26T07:22:02.261 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:02.256+0000 7fc7fffff640 1 -- 172.21.15.143:0/254760995 <== mon.0 v2:172.21.15.143:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x5562c24b6470 con 0x5562c26edc90 2023-11-26T07:22:02.261 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:02.256+0000 7fc7fffff640 1 -- 172.21.15.143:0/254760995 <== mon.0 v2:172.21.15.143:3300/0 5 ==== fsmap(e 34) v1 ==== 3294+0+0 (secure 0 0 0) 0x7fc810011040 con 0x5562c26edc90 2023-11-26T07:22:02.261 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:02.256+0000 7fc82a5e0000 10 client.6555 fetch_fsmap finished waiting for FSMap version 34 2023-11-26T07:22:02.261 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:02.256+0000 7fc82a5e0000 10 client.6555 resolve_mds: resolved name 'b' to daemon mds.b 2023-11-26T07:22:02.261 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:02.256+0000 7fc82a5e0000 20 client.6555 populate_metadata read hostname 'smithi143' 2023-11-26T07:22:02.261 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:02.256+0000 7fc82a5e0000 1 --2- 172.21.15.143:0/254760995 >> [v2:172.21.15.188:6833/1958716738,v1:172.21.15.188:6835/1958716738] conn(0x5562c26bd320 0x5562c26fff50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:22:02.261 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:02.256+0000 7fc82a5e0000 4 client.6555 mds_command: new command op to 6121 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2023-11-26T07:22:02.261 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:02.256+0000 7fc82a5e0000 1 -- 172.21.15.143:0/254760995 --> [v2:172.21.15.188:6833/1958716738,v1:172.21.15.188:6835/1958716738] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x5562c24b9b50 con 0x5562c26bd320 2023-11-26T07:22:02.261 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:02.256+0000 7fc825007640 1 --2- 172.21.15.143:0/254760995 >> [v2:172.21.15.188:6833/1958716738,v1:172.21.15.188:6835/1958716738] conn(0x5562c26bd320 0x5562c26fff50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:22:02.261 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:02.256+0000 7fc7fffff640 10 client.6555 ms_handle_connect on v2:172.21.15.188:6833/1958716738 2023-11-26T07:22:02.261 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:02.256+0000 7fc825007640 1 --2- 172.21.15.143:0/254760995 >> [v2:172.21.15.188:6833/1958716738,v1:172.21.15.188:6835/1958716738] conn(0x5562c26bd320 0x5562c26fff50 crc :-1 s=READY pgs=11 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=73724e415d17e219 server_cookie=1af8afd50df2fd in_seq=0 out_seq=0 2023-11-26T07:22:02.262 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:02.256+0000 7fc7fffff640 1 -- 172.21.15.143:0/254760995 <== mds.0 v2:172.21.15.188:6833/1958716738 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x5562c24b9b50 con 0x5562c26bd320 2023-11-26T07:22:02.262 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:02.256+0000 7fc7fffff640 10 client.6555 handle_command_reply: tid=0 2023-11-26T07:22:02.262 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:02.256+0000 7fc82a5e0000 1 client.6555 shutdown 2023-11-26T07:22:02.263 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:02.256+0000 7fc82a5e0000 1 -- 172.21.15.143:0/254760995 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x5562c26edc90 msgr2=0x5562c26a9280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:22:02.263 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:02.256+0000 7fc82a5e0000 1 --2- 172.21.15.143:0/254760995 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x5562c26edc90 0x5562c26a9280 secure :-1 s=READY pgs=814 cs=0 l=1 rev1=1 crypto rx=0x7fc810020000 tx=0x7fc810004740 comp rx=0 tx=0).stop 2023-11-26T07:22:02.263 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:02.256+0000 7fc82a5e0000 1 -- 172.21.15.143:0/254760995 shutdown_connections 2023-11-26T07:22:02.263 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:02.256+0000 7fc82a5e0000 1 --2- 172.21.15.143:0/254760995 >> [v2:172.21.15.188:6833/1958716738,v1:172.21.15.188:6835/1958716738] conn(0x5562c26bd320 0x5562c26fff50 crc :-1 s=READY pgs=11 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:22:02.263 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:02.256+0000 7fc82a5e0000 1 --2- 172.21.15.143:0/254760995 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5562c26ee5b0 0x5562c26bc910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:22:02.263 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:02.256+0000 7fc82a5e0000 1 --2- 172.21.15.143:0/254760995 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x5562c26edc90 0x5562c26a9280 unknown :-1 s=CLOSED pgs=814 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:22:02.263 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:02.256+0000 7fc7fffff640 0 client.6555 ms_handle_reset on v2:172.21.15.188:6833/1958716738 2023-11-26T07:22:02.263 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:02.256+0000 7fc82a5e0000 1 --2- 172.21.15.143:0/254760995 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x5562c26a4320 0x5562c26a8d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:22:02.263 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:02.256+0000 7fc82a5e0000 1 -- 172.21.15.143:0/254760995 >> 172.21.15.143:0/254760995 conn(0x5562c26a02d0 msgr2=0x5562c26a26c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:22:02.263 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:02.256+0000 7fc82a5e0000 1 -- 172.21.15.143:0/254760995 shutdown_connections 2023-11-26T07:22:02.263 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:02.256+0000 7fc82a5e0000 1 -- 172.21.15.143:0/254760995 wait complete. 2023-11-26T07:22:02.263 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:02.256+0000 7fc82a5e0000 20 client.6555 trim_cache size 0 max 16384 2023-11-26T07:22:02.275 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:02.272+0000 7fc82a5e0000 1 Processor -- start 2023-11-26T07:22:02.275 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:02.272+0000 7fc82a5e0000 1 -- start start 2023-11-26T07:22:02.277 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:02.276+0000 7fc82a5e0000 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x5562c26a02d0 0x5562c26ebb30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:22:02.277 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:02.276+0000 7fc82a5e0000 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5562c26a4320 0x5562c26ec070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:22:02.277 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:02.276+0000 7fc82a5e0000 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x5562c26edc90 0x5562c26d07a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:22:02.277 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:02.276+0000 7fc82a5e0000 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x5562c21ad940 con 0x5562c26a02d0 2023-11-26T07:22:02.277 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:02.276+0000 7fc82a5e0000 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x5562c24a9760 con 0x5562c26a4320 2023-11-26T07:22:02.278 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:02.276+0000 7fc82a5e0000 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x5562c269fc70 con 0x5562c26edc90 2023-11-26T07:22:02.278 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:02.276+0000 7fc825007640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x5562c26a02d0 0x5562c26ebb30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:22:02.278 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:02.276+0000 7fc825007640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x5562c26a02d0 0x5562c26ebb30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:57332/0 (socket says 172.21.15.143:57332) 2023-11-26T07:22:02.278 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:02.276+0000 7fc825007640 1 -- 172.21.15.143:0/329958787 learned_addr learned my addr 172.21.15.143:0/329958787 (peer_addr_for_me v2:172.21.15.143:0/0) 2023-11-26T07:22:02.278 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:02.276+0000 7fc825007640 1 -- 172.21.15.143:0/329958787 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x5562c26edc90 msgr2=0x5562c26d07a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-26T07:22:02.278 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:02.276+0000 7fc825007640 1 --2- 172.21.15.143:0/329958787 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x5562c26edc90 0x5562c26d07a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:22:02.278 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:02.276+0000 7fc825007640 1 -- 172.21.15.143:0/329958787 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5562c26a4320 msgr2=0x5562c26ec070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-26T07:22:02.278 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:02.276+0000 7fc825007640 1 --2- 172.21.15.143:0/329958787 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5562c26a4320 0x5562c26ec070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:22:02.278 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:02.276+0000 7fc825007640 1 -- 172.21.15.143:0/329958787 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5562c249b490 con 0x5562c26a02d0 2023-11-26T07:22:02.278 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:02.276+0000 7fc825007640 1 --2- 172.21.15.143:0/329958787 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x5562c26a02d0 0x5562c26ebb30 secure :-1 s=READY pgs=815 cs=0 l=1 rev1=1 crypto rx=0x7fc81c0eae00 tx=0x7fc81c0091c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a6d23ce8cdf34d03 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:22:02.278 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:02.276+0000 7fc80b7fe640 1 -- 172.21.15.143:0/329958787 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc81c0f6c00 con 0x5562c26a02d0 2023-11-26T07:22:02.278 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:02.276+0000 7fc80b7fe640 1 -- 172.21.15.143:0/329958787 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc81c0ecca0 con 0x5562c26a02d0 2023-11-26T07:22:02.279 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:02.276+0000 7fc80b7fe640 1 -- 172.21.15.143:0/329958787 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc81c0eb010 con 0x5562c26a02d0 2023-11-26T07:22:02.279 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:02.276+0000 7fc82a5e0000 1 -- 172.21.15.143:0/329958787 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x5562c26a02d0 msgr2=0x5562c26ebb30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:22:02.279 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:02.276+0000 7fc82a5e0000 1 --2- 172.21.15.143:0/329958787 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x5562c26a02d0 0x5562c26ebb30 secure :-1 s=READY pgs=815 cs=0 l=1 rev1=1 crypto rx=0x7fc81c0eae00 tx=0x7fc81c0091c0 comp rx=0 tx=0).stop 2023-11-26T07:22:02.280 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:02.276+0000 7fc82a5e0000 1 -- 172.21.15.143:0/329958787 shutdown_connections 2023-11-26T07:22:02.280 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:02.276+0000 7fc82a5e0000 1 --2- 172.21.15.143:0/329958787 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x5562c26edc90 0x5562c26d07a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:22:02.280 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:02.276+0000 7fc82a5e0000 1 --2- 172.21.15.143:0/329958787 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5562c26a4320 0x5562c26ec070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:22:02.280 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:02.276+0000 7fc82a5e0000 1 --2- 172.21.15.143:0/329958787 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x5562c26a02d0 0x5562c26ebb30 unknown :-1 s=CLOSED pgs=815 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:22:02.280 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:02.276+0000 7fc82a5e0000 1 -- 172.21.15.143:0/329958787 >> 172.21.15.143:0/329958787 conn(0x5562c26a8f10 msgr2=0x5562c26a26c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:22:02.280 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:02.276+0000 7fc82a5e0000 1 -- 172.21.15.143:0/329958787 shutdown_connections 2023-11-26T07:22:02.280 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:02.276+0000 7fc82a5e0000 1 -- 172.21.15.143:0/329958787 wait complete. 2023-11-26T07:22:02.280 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:02.276+0000 7fc82a5e0000 1 -- start start 2023-11-26T07:22:02.280 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:02.276+0000 7fc82a5e0000 1 Processor -- start 2023-11-26T07:22:02.283 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:02.276+0000 7fc82a5e0000 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x5562c26a02d0 0x5562c26d0bd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:22:02.283 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:02.276+0000 7fc82a5e0000 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5562c26a4320 0x5562c26fff50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:22:02.283 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:02.276+0000 7fc82a5e0000 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x5562c26edc90 0x5562c2700490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:22:02.283 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:02.276+0000 7fc82a5e0000 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x5562c24f47c0 con 0x5562c26a02d0 2023-11-26T07:22:02.284 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:02.276+0000 7fc82a5e0000 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x5562c24f1bc0 con 0x5562c26a4320 2023-11-26T07:22:02.284 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:02.276+0000 7fc82a5e0000 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x5562c24b3380 con 0x5562c26edc90 2023-11-26T07:22:02.284 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:02.276+0000 7fc82a5e0000 10 client.0 _ll_register_callbacks cb 0x5562c2542c00 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-11-26T07:22:02.284 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:02.276+0000 7fc825007640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x5562c26a02d0 0x5562c26d0bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:22:02.284 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:02.280+0000 7fc825007640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x5562c26a02d0 0x5562c26d0bd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:57340/0 (socket says 172.21.15.143:57340) 2023-11-26T07:22:02.284 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:02.280+0000 7fc825007640 1 -- 172.21.15.143:0/4260055303 learned_addr learned my addr 172.21.15.143:0/4260055303 (peer_addr_for_me v2:172.21.15.143:0/0) 2023-11-26T07:22:02.284 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:02.280+0000 7fc824806640 1 --2- 172.21.15.143:0/4260055303 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5562c26a4320 0x5562c26fff50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:22:02.284 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:02.280+0000 7fc825808640 1 --2- 172.21.15.143:0/4260055303 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x5562c26edc90 0x5562c2700490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:22:02.284 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:02.280+0000 7fc825007640 1 -- 172.21.15.143:0/4260055303 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x5562c26edc90 msgr2=0x5562c2700490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-26T07:22:02.284 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:02.280+0000 7fc825007640 1 --2- 172.21.15.143:0/4260055303 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x5562c26edc90 0x5562c2700490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:22:02.284 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:02.280+0000 7fc825007640 1 -- 172.21.15.143:0/4260055303 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5562c26a4320 msgr2=0x5562c26fff50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-26T07:22:02.284 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:02.280+0000 7fc825007640 1 --2- 172.21.15.143:0/4260055303 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5562c26a4320 0x5562c26fff50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:22:02.285 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:02.280+0000 7fc825007640 1 -- 172.21.15.143:0/4260055303 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc82009aaf0 con 0x5562c26a02d0 2023-11-26T07:22:02.285 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:02.280+0000 7fc824806640 1 --2- 172.21.15.143:0/4260055303 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5562c26a4320 0x5562c26fff50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-26T07:22:02.285 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:02.280+0000 7fc825007640 1 --2- 172.21.15.143:0/4260055303 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x5562c26a02d0 0x5562c26d0bd0 secure :-1 s=READY pgs=816 cs=0 l=1 rev1=1 crypto rx=0x7fc81c0eae00 tx=0x7fc81c0f1040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8c8b053943de092e server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:22:02.285 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:02.280+0000 7fc7fffff640 10 client.0 ms_handle_connect on v2:172.21.15.143:3300/0 2023-11-26T07:22:02.285 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:02.280+0000 7fc7fffff640 1 -- 172.21.15.143:0/4260055303 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc81c0eeb60 con 0x5562c26a02d0 2023-11-26T07:22:02.285 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:02.280+0000 7fc7fffff640 1 -- 172.21.15.143:0/4260055303 <== mon.0 v2:172.21.15.143:3300/0 2 ==== osd_map(59..59 src has 1..59) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fc81c0eecc0 con 0x5562c26a02d0 2023-11-26T07:22:02.285 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:02.280+0000 7fc82a5e0000 1 -- 172.21.15.143:0/4260055303 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x5562c24f3110 con 0x5562c26a02d0 2023-11-26T07:22:02.285 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:02.280+0000 7fc7fffff640 1 -- 172.21.15.143:0/4260055303 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_get_version_reply(handle=1 version=34) v2 ==== 24+0+0 (secure 0 0 0) 0x7fc81c0eddf0 con 0x5562c26a02d0 2023-11-26T07:22:02.285 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:02.280+0000 7fc82a5e0000 10 client.6561 fetch_fsmap learned FSMap version 34 2023-11-26T07:22:02.285 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:02.280+0000 7fc82a5e0000 1 -- 172.21.15.143:0/4260055303 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,fsmap=34,monmap=2+}) v3 -- 0x5562c21eccc0 con 0x5562c26a02d0 2023-11-26T07:22:02.285 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:02.280+0000 7fc7fffff640 1 -- 172.21.15.143:0/4260055303 <== mon.0 v2:172.21.15.143:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc81c110020 con 0x5562c26a02d0 2023-11-26T07:22:02.285 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:02.280+0000 7fc7fffff640 1 -- 172.21.15.143:0/4260055303 <== mon.0 v2:172.21.15.143:3300/0 5 ==== fsmap(e 34) v1 ==== 3294+0+0 (secure 0 0 0) 0x7fc81c0f6770 con 0x5562c26a02d0 2023-11-26T07:22:02.285 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:02.280+0000 7fc82a5e0000 10 client.6561 fetch_fsmap finished waiting for FSMap version 34 2023-11-26T07:22:02.286 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:02.280+0000 7fc82a5e0000 10 client.6561 resolve_mds: resolved name 'b' to daemon mds.b 2023-11-26T07:22:02.286 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:02.280+0000 7fc82a5e0000 20 client.6561 populate_metadata read hostname 'smithi143' 2023-11-26T07:22:02.286 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:02.280+0000 7fc82a5e0000 1 --2- 172.21.15.143:0/4260055303 >> [v2:172.21.15.188:6833/1958716738,v1:172.21.15.188:6835/1958716738] conn(0x5562c26ee5b0 0x5562c275f650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:22:02.286 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:02.280+0000 7fc82a5e0000 4 client.6561 mds_command: new command op to 6121 tid=0 multi_id=0 [{"prefix": "status"}] 2023-11-26T07:22:02.286 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:02.280+0000 7fc82a5e0000 1 -- 172.21.15.143:0/4260055303 --> [v2:172.21.15.188:6833/1958716738,v1:172.21.15.188:6835/1958716738] -- command(tid 0: {"prefix": "status"}) v1 -- 0x5562c25229a0 con 0x5562c26ee5b0 2023-11-26T07:22:02.286 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:02.280+0000 7fc824806640 1 --2- 172.21.15.143:0/4260055303 >> [v2:172.21.15.188:6833/1958716738,v1:172.21.15.188:6835/1958716738] conn(0x5562c26ee5b0 0x5562c275f650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:22:02.286 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:02.280+0000 7fc7fffff640 10 client.6561 ms_handle_connect on v2:172.21.15.188:6833/1958716738 2023-11-26T07:22:02.286 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:02.280+0000 7fc824806640 1 --2- 172.21.15.143:0/4260055303 >> [v2:172.21.15.188:6833/1958716738,v1:172.21.15.188:6835/1958716738] conn(0x5562c26ee5b0 0x5562c275f650 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=6842bf114b12a11a server_cookie=7bd76e6489c51dd3 in_seq=0 out_seq=0 2023-11-26T07:22:02.286 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:02.280+0000 7fc7fffff640 1 -- 172.21.15.143:0/4260055303 <== mds.0 v2:172.21.15.188:6833/1958716738 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+331 (crc 0 0 0) 0x5562c25229a0 con 0x5562c26ee5b0 2023-11-26T07:22:02.286 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:02.280+0000 7fc7fffff640 10 client.6561 handle_command_reply: tid=0 2023-11-26T07:22:02.286 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:02.280+0000 7fc82a5e0000 1 client.6561 shutdown 2023-11-26T07:22:02.286 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:02.280+0000 7fc82a5e0000 1 -- 172.21.15.143:0/4260055303 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x5562c26a02d0 msgr2=0x5562c26d0bd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:22:02.287 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:02.280+0000 7fc82a5e0000 1 --2- 172.21.15.143:0/4260055303 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x5562c26a02d0 0x5562c26d0bd0 secure :-1 s=READY pgs=816 cs=0 l=1 rev1=1 crypto rx=0x7fc81c0eae00 tx=0x7fc81c0f1040 comp rx=0 tx=0).stop 2023-11-26T07:22:02.287 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:02.280+0000 7fc82a5e0000 1 -- 172.21.15.143:0/4260055303 shutdown_connections 2023-11-26T07:22:02.287 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:02.280+0000 7fc82a5e0000 1 --2- 172.21.15.143:0/4260055303 >> [v2:172.21.15.188:6833/1958716738,v1:172.21.15.188:6835/1958716738] conn(0x5562c26ee5b0 0x5562c275f650 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:22:02.287 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:02.280+0000 7fc82a5e0000 1 --2- 172.21.15.143:0/4260055303 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x5562c26edc90 0x5562c2700490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:22:02.287 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:02.280+0000 7fc82a5e0000 1 --2- 172.21.15.143:0/4260055303 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5562c26a4320 0x5562c26fff50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:22:02.287 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:02.280+0000 7fc7fffff640 0 client.6561 ms_handle_reset on v2:172.21.15.188:6833/1958716738 2023-11-26T07:22:02.287 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:02.280+0000 7fc82a5e0000 1 --2- 172.21.15.143:0/4260055303 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x5562c26a02d0 0x5562c26d0bd0 unknown :-1 s=CLOSED pgs=816 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:22:02.287 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:02.280+0000 7fc82a5e0000 1 -- 172.21.15.143:0/4260055303 >> 172.21.15.143:0/4260055303 conn(0x5562c26a8f10 msgr2=0x5562c26a2f50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:22:02.287 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:02.280+0000 7fc82a5e0000 1 -- 172.21.15.143:0/4260055303 shutdown_connections 2023-11-26T07:22:02.287 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:02.280+0000 7fc82a5e0000 1 -- 172.21.15.143:0/4260055303 wait complete. 2023-11-26T07:22:02.288 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:02.280+0000 7fc82a5e0000 20 client.6561 trim_cache size 0 max 16384 2023-11-26T07:22:02.288 INFO:teuthology.orchestra.run.smithi143.stdout:{ 2023-11-26T07:22:02.288 INFO:teuthology.orchestra.run.smithi143.stdout: "cluster_fsid": "f5033060-2469-4a07-95fe-2b76f5404bc1", 2023-11-26T07:22:02.288 INFO:teuthology.orchestra.run.smithi143.stdout: "whoami": 0, 2023-11-26T07:22:02.288 INFO:teuthology.orchestra.run.smithi143.stdout: "id": 6121, 2023-11-26T07:22:02.288 INFO:teuthology.orchestra.run.smithi143.stdout: "want_state": "up:active", 2023-11-26T07:22:02.288 INFO:teuthology.orchestra.run.smithi143.stdout: "state": "up:active", 2023-11-26T07:22:02.288 INFO:teuthology.orchestra.run.smithi143.stdout: "fs_name": "cephfs", 2023-11-26T07:22:02.289 INFO:teuthology.orchestra.run.smithi143.stdout: "rank_uptime": 5.9913013880000001, 2023-11-26T07:22:02.289 INFO:teuthology.orchestra.run.smithi143.stdout: "mdsmap_epoch": 34, 2023-11-26T07:22:02.289 INFO:teuthology.orchestra.run.smithi143.stdout: "osdmap_epoch": 59, 2023-11-26T07:22:02.289 INFO:teuthology.orchestra.run.smithi143.stdout: "osdmap_epoch_barrier": 59, 2023-11-26T07:22:02.289 INFO:teuthology.orchestra.run.smithi143.stdout: "uptime": 11.879899999999999 2023-11-26T07:22:02.289 INFO:teuthology.orchestra.run.smithi143.stdout:} 2023-11-26T07:22:02.290 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:02.284+0000 7fc82726b640 1 -- 172.21.15.143:0/1497609075 >> [v2:172.21.15.143:6832/2150557544,v1:172.21.15.143:6833/2150557544] conn(0x7fc804066650 msgr2=0x7fc804068b10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:22:02.290 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:02.284+0000 7fc82726b640 1 --2- 172.21.15.143:0/1497609075 >> [v2:172.21.15.143:6832/2150557544,v1:172.21.15.143:6833/2150557544] conn(0x7fc804066650 0x7fc804068b10 secure :-1 s=READY pgs=388 cs=0 l=1 rev1=1 crypto rx=0x7fc82015ac70 tx=0x7fc810008040 comp rx=0 tx=0).stop 2023-11-26T07:22:02.290 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:02.284+0000 7fc82726b640 1 -- 172.21.15.143:0/1497609075 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fc8200abef0 msgr2=0x7fc82015d6d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:22:02.290 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:02.284+0000 7fc82726b640 1 --2- 172.21.15.143:0/1497609075 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fc8200abef0 0x7fc82015d6d0 secure :-1 s=READY pgs=812 cs=0 l=1 rev1=1 crypto rx=0x7fc81c0ae040 tx=0x7fc81c01ad10 comp rx=0 tx=0).stop 2023-11-26T07:22:02.290 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:02.284+0000 7fc82726b640 1 -- 172.21.15.143:0/1497609075 shutdown_connections 2023-11-26T07:22:02.290 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:02.284+0000 7fc82726b640 1 --2- 172.21.15.143:0/1497609075 >> [v2:172.21.15.143:6832/2150557544,v1:172.21.15.143:6833/2150557544] conn(0x7fc804066650 0x7fc804068b10 unknown :-1 s=CLOSED pgs=388 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:22:02.290 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:02.284+0000 7fc82726b640 1 --2- 172.21.15.143:0/1497609075 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fc82015f980 0x7fc82009b2f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:22:02.290 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:02.284+0000 7fc82726b640 1 --2- 172.21.15.143:0/1497609075 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fc8201557e0 0x7fc820099a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:22:02.290 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:02.284+0000 7fc82726b640 1 --2- 172.21.15.143:0/1497609075 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fc8200abef0 0x7fc82015d6d0 unknown :-1 s=CLOSED pgs=812 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:22:02.290 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:02.284+0000 7fc82726b640 1 -- 172.21.15.143:0/1497609075 >> 172.21.15.143:0/1497609075 conn(0x7fc8200b0c10 msgr2=0x7fc82006c520 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:22:02.290 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:02.284+0000 7fc82726b640 1 -- 172.21.15.143:0/1497609075 shutdown_connections 2023-11-26T07:22:02.291 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:02.284+0000 7fc82726b640 1 -- 172.21.15.143:0/1497609075 wait complete. 2023-11-26T07:22:02.303 DEBUG:teuthology.orchestra.run.smithi143:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2023-11-26T07:22:02.441 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:02.440+0000 7f6ed2ba1640 1 Processor -- start 2023-11-26T07:22:02.441 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:02.440+0000 7f6ed2ba1640 1 -- start start 2023-11-26T07:22:02.443 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:02.440+0000 7f6ed2ba1640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f6ecc0d1bc0 0x7f6ecc0d0df0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:22:02.443 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:02.440+0000 7f6ed2ba1640 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7f6ecc0b2160 0x7f6ecc0b0750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:22:02.443 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:02.440+0000 7f6ed2ba1640 1 -- --> v1:172.21.15.188:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6ecc0cf0e0 con 0x7f6ecc15d6e0 2023-11-26T07:22:02.444 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:02.440+0000 7f6ed2ba1640 1 -- --> v2:172.21.15.143:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6ecc15dac0 con 0x7f6ecc0b2160 2023-11-26T07:22:02.444 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:02.440+0000 7f6ed2ba1640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6ecc0ceb20 con 0x7f6ecc0d1bc0 2023-11-26T07:22:02.444 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:02.440+0000 7f6ed113e640 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7f6ecc0b2160 0x7f6ecc0b0750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:22:02.444 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:02.440+0000 7f6ed113e640 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7f6ecc0b2160 0x7f6ecc0b0750 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:57352/0 (socket says 172.21.15.143:57352) 2023-11-26T07:22:02.444 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:02.440+0000 7f6ed113e640 1 -- 172.21.15.143:0/3374591509 learned_addr learned my addr 172.21.15.143:0/3374591509 (peer_addr_for_me v2:172.21.15.143:0/0) 2023-11-26T07:22:02.444 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:02.440+0000 7f6ec3fff640 1 --2- 172.21.15.143:0/3374591509 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f6ecc0d1bc0 0x7f6ecc0d0df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:22:02.444 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:02.440+0000 7f6ed113e640 1 -- 172.21.15.143:0/3374591509 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f6ecc0d1bc0 msgr2=0x7f6ecc0d0df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-26T07:22:02.444 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:02.440+0000 7f6ed113e640 1 --2- 172.21.15.143:0/3374591509 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f6ecc0d1bc0 0x7f6ecc0d0df0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:22:02.444 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:02.440+0000 7f6ed113e640 1 -- 172.21.15.143:0/3374591509 >> v1:172.21.15.188:6789/0 conn(0x7f6ecc15d6e0 legacy=0x7f6ecc0d14b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:22:02.444 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:02.440+0000 7f6ed113e640 1 -- 172.21.15.143:0/3374591509 --> v2:172.21.15.143:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6ecc0cee00 con 0x7f6ecc0b2160 2023-11-26T07:22:02.444 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:02.440+0000 7f6ec3fff640 1 --2- 172.21.15.143:0/3374591509 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f6ecc0d1bc0 0x7f6ecc0d0df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-26T07:22:02.446 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:02.444+0000 7f6ed113e640 1 --2- 172.21.15.143:0/3374591509 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f6ecc0b2160 0x7f6ecc0b0750 secure :-1 s=READY pgs=817 cs=0 l=1 rev1=1 crypto rx=0x7f6ec4009e50 tx=0x7f6ec401ca10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=646a94d590f9f576 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:22:02.446 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:02.444+0000 7f6ec37fe640 1 -- 172.21.15.143:0/3374591509 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6ec40a57d0 con 0x7f6ecc0b2160 2023-11-26T07:22:02.446 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:02.444+0000 7f6ec37fe640 1 -- 172.21.15.143:0/3374591509 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6ec40a5930 con 0x7f6ecc0b2160 2023-11-26T07:22:02.447 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:02.444+0000 7f6ec37fe640 1 -- 172.21.15.143:0/3374591509 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6ec40a5cc0 con 0x7f6ecc0b2160 2023-11-26T07:22:02.447 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:02.444+0000 7f6ed2ba1640 1 -- 172.21.15.143:0/3374591509 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f6ecc0b2160 msgr2=0x7f6ecc0b0750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:22:02.447 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:02.444+0000 7f6ed2ba1640 1 --2- 172.21.15.143:0/3374591509 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f6ecc0b2160 0x7f6ecc0b0750 secure :-1 s=READY pgs=817 cs=0 l=1 rev1=1 crypto rx=0x7f6ec4009e50 tx=0x7f6ec401ca10 comp rx=0 tx=0).stop 2023-11-26T07:22:02.447 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:02.444+0000 7f6ed2ba1640 1 -- 172.21.15.143:0/3374591509 shutdown_connections 2023-11-26T07:22:02.448 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:02.444+0000 7f6ed2ba1640 1 --2- 172.21.15.143:0/3374591509 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f6ecc0b2160 0x7f6ecc0b0750 unknown :-1 s=CLOSED pgs=817 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:22:02.448 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:02.444+0000 7f6ed2ba1640 1 --2- 172.21.15.143:0/3374591509 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f6ecc0d1bc0 0x7f6ecc0d0df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:22:02.448 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:02.444+0000 7f6ed2ba1640 1 -- 172.21.15.143:0/3374591509 >> 172.21.15.143:0/3374591509 conn(0x7f6ecc09c550 msgr2=0x7f6ecc0d3220 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:22:02.448 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:02.444+0000 7f6ed2ba1640 1 -- 172.21.15.143:0/3374591509 shutdown_connections 2023-11-26T07:22:02.448 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:02.444+0000 7f6ed2ba1640 1 -- 172.21.15.143:0/3374591509 wait complete. 2023-11-26T07:22:02.449 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:02.448+0000 7f6ed2ba1640 1 Processor -- start 2023-11-26T07:22:02.449 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:02.448+0000 7f6ed2ba1640 1 -- start start 2023-11-26T07:22:02.451 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:02.448+0000 7f6ed2ba1640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f6ecc0b2160 0x7f6ecc153260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:22:02.451 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:02.448+0000 7f6ed2ba1640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f6ecc0d1bc0 0x7f6ecc0c6d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:22:02.451 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:02.448+0000 7f6ed2ba1640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f6ecc15d6e0 0x7f6ecc0b3a40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:22:02.451 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:02.448+0000 7f6ed2ba1640 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6ecc0b7030 con 0x7f6ecc0d1bc0 2023-11-26T07:22:02.452 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:02.448+0000 7f6ed2ba1640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6ecc0b71a0 con 0x7f6ecc0b2160 2023-11-26T07:22:02.452 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:02.448+0000 7f6ed2ba1640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6ecc0c7730 con 0x7f6ecc15d6e0 2023-11-26T07:22:02.452 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:02.448+0000 7f6ec3fff640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f6ecc0d1bc0 0x7f6ecc0c6d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:22:02.452 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:02.448+0000 7f6ec3fff640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f6ecc0d1bc0 0x7f6ecc0c6d50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:57358/0 (socket says 172.21.15.143:57358) 2023-11-26T07:22:02.452 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:02.448+0000 7f6ec3fff640 1 -- 172.21.15.143:0/1982895627 learned_addr learned my addr 172.21.15.143:0/1982895627 (peer_addr_for_me v2:172.21.15.143:0/0) 2023-11-26T07:22:02.452 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:02.448+0000 7f6ed093d640 1 --2- 172.21.15.143:0/1982895627 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f6ecc0b2160 0x7f6ecc153260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:22:02.452 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:02.448+0000 7f6ed113e640 1 --2- 172.21.15.143:0/1982895627 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f6ecc15d6e0 0x7f6ecc0b3a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:22:02.452 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:02.448+0000 7f6ec3fff640 1 -- 172.21.15.143:0/1982895627 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f6ecc15d6e0 msgr2=0x7f6ecc0b3a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:22:02.452 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:02.448+0000 7f6ec3fff640 1 --2- 172.21.15.143:0/1982895627 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f6ecc15d6e0 0x7f6ecc0b3a40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:22:02.452 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:02.448+0000 7f6ec3fff640 1 -- 172.21.15.143:0/1982895627 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f6ecc0b2160 msgr2=0x7f6ecc153260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:22:02.452 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:02.448+0000 7f6ec3fff640 1 --2- 172.21.15.143:0/1982895627 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f6ecc0b2160 0x7f6ecc153260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:22:02.452 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:02.448+0000 7f6ec3fff640 1 -- 172.21.15.143:0/1982895627 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6ec4009b30 con 0x7f6ecc0d1bc0 2023-11-26T07:22:02.453 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:02.448+0000 7f6ec3fff640 1 --2- 172.21.15.143:0/1982895627 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f6ecc0d1bc0 0x7f6ecc0c6d50 secure :-1 s=READY pgs=818 cs=0 l=1 rev1=1 crypto rx=0x7f6ebc00c7c0 tx=0x7f6ebc00cc80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:22:02.453 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:02.448+0000 7f6ec17fa640 1 -- 172.21.15.143:0/1982895627 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6ebc007a30 con 0x7f6ecc0d1bc0 2023-11-26T07:22:02.453 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:02.448+0000 7f6ec17fa640 1 -- 172.21.15.143:0/1982895627 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6ebc007b90 con 0x7f6ecc0d1bc0 2023-11-26T07:22:02.453 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:02.448+0000 7f6ec17fa640 1 -- 172.21.15.143:0/1982895627 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6ebc00ce70 con 0x7f6ecc0d1bc0 2023-11-26T07:22:02.453 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:02.448+0000 7f6ed2ba1640 1 -- 172.21.15.143:0/1982895627 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6ecc0c7a10 con 0x7f6ecc0d1bc0 2023-11-26T07:22:02.453 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:02.448+0000 7f6ed2ba1640 1 -- 172.21.15.143:0/1982895627 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6ecc0c7fd0 con 0x7f6ecc0d1bc0 2023-11-26T07:22:02.453 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:02.452+0000 7f6ed2ba1640 1 -- 172.21.15.143:0/1982895627 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6ecc0b7330 con 0x7f6ecc0d1bc0 2023-11-26T07:22:02.453 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:02.452+0000 7f6ec17fa640 1 -- 172.21.15.143:0/1982895627 <== mon.0 v2:172.21.15.143:3300/0 4 ==== mgrmap(e 9) v1 ==== 86005+0+0 (secure 0 0 0) 0x7f6ebc0113f0 con 0x7f6ecc0d1bc0 2023-11-26T07:22:02.454 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:02.452+0000 7f6ec17fa640 1 --2- 172.21.15.143:0/1982895627 >> [v2:172.21.15.143:6832/2150557544,v1:172.21.15.143:6833/2150557544] conn(0x7f6eb8066620 0x7f6eb8068ae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:22:02.454 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:02.452+0000 7f6ec17fa640 1 -- 172.21.15.143:0/1982895627 <== mon.0 v2:172.21.15.143:3300/0 5 ==== osd_map(59..59 src has 1..59) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f6ebc0845d0 con 0x7f6ecc0d1bc0 2023-11-26T07:22:02.454 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:02.452+0000 7f6ed093d640 1 --2- 172.21.15.143:0/1982895627 >> [v2:172.21.15.143:6832/2150557544,v1:172.21.15.143:6833/2150557544] conn(0x7f6eb8066620 0x7f6eb8068ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:22:02.455 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:02.452+0000 7f6ed093d640 1 --2- 172.21.15.143:0/1982895627 >> [v2:172.21.15.143:6832/2150557544,v1:172.21.15.143:6833/2150557544] conn(0x7f6eb8066620 0x7f6eb8068ae0 secure :-1 s=READY pgs=389 cs=0 l=1 rev1=1 crypto rx=0x7f6ec8007fd0 tx=0x7f6ec8009040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:22:02.458 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:02.456+0000 7f6ec17fa640 1 -- 172.21.15.143:0/1982895627 <== mon.0 v2:172.21.15.143:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172649 (secure 0 0 0) 0x7f6ebc051820 con 0x7f6ecc0d1bc0 2023-11-26T07:22:02.640 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:02.636+0000 7f6ed2ba1640 1 -- 172.21.15.143:0/1982895627 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "fs ls"} v 0) v1 -- 0x7f6ecc067680 con 0x7f6ecc0d1bc0 2023-11-26T07:22:02.640 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:02.640+0000 7f6ec17fa640 1 -- 172.21.15.143:0/1982895627 <== mon.0 v2:172.21.15.143:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v34) v1 ==== 53+0+73 (secure 0 0 0) 0x7f6ebc054e40 con 0x7f6ecc0d1bc0 2023-11-26T07:22:02.640 INFO:teuthology.orchestra.run.smithi143.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2023-11-26T07:22:02.642 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:02.640+0000 7f6ed2ba1640 1 -- 172.21.15.143:0/1982895627 >> [v2:172.21.15.143:6832/2150557544,v1:172.21.15.143:6833/2150557544] conn(0x7f6eb8066620 msgr2=0x7f6eb8068ae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:22:02.643 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:02.640+0000 7f6ed2ba1640 1 --2- 172.21.15.143:0/1982895627 >> [v2:172.21.15.143:6832/2150557544,v1:172.21.15.143:6833/2150557544] conn(0x7f6eb8066620 0x7f6eb8068ae0 secure :-1 s=READY pgs=389 cs=0 l=1 rev1=1 crypto rx=0x7f6ec8007fd0 tx=0x7f6ec8009040 comp rx=0 tx=0).stop 2023-11-26T07:22:02.643 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:02.640+0000 7f6ed2ba1640 1 -- 172.21.15.143:0/1982895627 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f6ecc0d1bc0 msgr2=0x7f6ecc0c6d50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:22:02.643 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:02.640+0000 7f6ed2ba1640 1 --2- 172.21.15.143:0/1982895627 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f6ecc0d1bc0 0x7f6ecc0c6d50 secure :-1 s=READY pgs=818 cs=0 l=1 rev1=1 crypto rx=0x7f6ebc00c7c0 tx=0x7f6ebc00cc80 comp rx=0 tx=0).stop 2023-11-26T07:22:02.643 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:02.640+0000 7f6ed2ba1640 1 -- 172.21.15.143:0/1982895627 shutdown_connections 2023-11-26T07:22:02.643 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:02.640+0000 7f6ed2ba1640 1 --2- 172.21.15.143:0/1982895627 >> [v2:172.21.15.143:6832/2150557544,v1:172.21.15.143:6833/2150557544] conn(0x7f6eb8066620 0x7f6eb8068ae0 unknown :-1 s=CLOSED pgs=389 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:22:02.643 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:02.640+0000 7f6ed2ba1640 1 --2- 172.21.15.143:0/1982895627 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f6ecc15d6e0 0x7f6ecc0b3a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:22:02.643 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:02.640+0000 7f6ed2ba1640 1 --2- 172.21.15.143:0/1982895627 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f6ecc0d1bc0 0x7f6ecc0c6d50 unknown :-1 s=CLOSED pgs=818 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:22:02.643 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:02.640+0000 7f6ed2ba1640 1 --2- 172.21.15.143:0/1982895627 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f6ecc0b2160 0x7f6ecc153260 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:22:02.643 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:02.640+0000 7f6ed2ba1640 1 -- 172.21.15.143:0/1982895627 >> 172.21.15.143:0/1982895627 conn(0x7f6ecc09c550 msgr2=0x7f6ecc0b3230 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:22:02.643 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:02.640+0000 7f6ed2ba1640 1 -- 172.21.15.143:0/1982895627 shutdown_connections 2023-11-26T07:22:02.643 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:02.640+0000 7f6ed2ba1640 1 -- 172.21.15.143:0/1982895627 wait complete. 2023-11-26T07:22:02.663 INFO:tasks.cephfs.mount:Mounting default Ceph FS; just confirmed its presence on cluster 2023-11-26T07:22:02.663 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2023-11-26T07:22:02.663 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi143.front.sepia.ceph.com 2023-11-26T07:22:02.663 INFO:tasks.cephfs.mount:self.client.name = client.0 2023-11-26T07:22:02.663 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2023-11-26T07:22:02.663 INFO:tasks.cephfs.mount:self.cephfs_name = None 2023-11-26T07:22:02.663 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2023-11-26T07:22:02.663 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2023-11-26T07:22:02.663 INFO:tasks.cephfs.mount:Setting the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2023-11-26T07:22:02.663 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-26T07:22:02.663 DEBUG:teuthology.orchestra.run.smithi143:> ip addr 2023-11-26T07:22:02.672 INFO:teuthology.orchestra.run.smithi143.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2023-11-26T07:22:02.673 INFO:teuthology.orchestra.run.smithi143.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2023-11-26T07:22:02.673 INFO:teuthology.orchestra.run.smithi143.stdout: inet 127.0.0.1/8 scope host lo 2023-11-26T07:22:02.673 INFO:teuthology.orchestra.run.smithi143.stdout: valid_lft forever preferred_lft forever 2023-11-26T07:22:02.673 INFO:teuthology.orchestra.run.smithi143.stdout: inet6 ::1/128 scope host 2023-11-26T07:22:02.673 INFO:teuthology.orchestra.run.smithi143.stdout: valid_lft forever preferred_lft forever 2023-11-26T07:22:02.673 INFO:teuthology.orchestra.run.smithi143.stdout:2: eno1: mtu 1500 qdisc noop state DOWN group default qlen 1000 2023-11-26T07:22:02.673 INFO:teuthology.orchestra.run.smithi143.stdout: link/ether 0c:c4:7a:d9:3b:94 brd ff:ff:ff:ff:ff:ff 2023-11-26T07:22:02.673 INFO:teuthology.orchestra.run.smithi143.stdout: altname enp7s0f0 2023-11-26T07:22:02.673 INFO:teuthology.orchestra.run.smithi143.stdout:3: eno2: mtu 1500 qdisc noop state DOWN group default qlen 1000 2023-11-26T07:22:02.673 INFO:teuthology.orchestra.run.smithi143.stdout: link/ether 0c:c4:7a:d9:3b:95 brd ff:ff:ff:ff:ff:ff 2023-11-26T07:22:02.673 INFO:teuthology.orchestra.run.smithi143.stdout: altname enp7s0f1 2023-11-26T07:22:02.673 INFO:teuthology.orchestra.run.smithi143.stdout:4: enp3s0f0: mtu 1500 qdisc noop state DOWN group default qlen 1000 2023-11-26T07:22:02.674 INFO:teuthology.orchestra.run.smithi143.stdout: link/ether 0c:c4:7a:88:77:82 brd ff:ff:ff:ff:ff:ff 2023-11-26T07:22:02.674 INFO:teuthology.orchestra.run.smithi143.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2023-11-26T07:22:02.674 INFO:teuthology.orchestra.run.smithi143.stdout: link/ether 0c:c4:7a:88:77:83 brd ff:ff:ff:ff:ff:ff 2023-11-26T07:22:02.674 INFO:teuthology.orchestra.run.smithi143.stdout: inet 172.21.15.143/20 metric 100 brd 172.21.15.255 scope global dynamic enp3s0f1 2023-11-26T07:22:02.674 INFO:teuthology.orchestra.run.smithi143.stdout: valid_lft 42151sec preferred_lft 42151sec 2023-11-26T07:22:02.674 INFO:teuthology.orchestra.run.smithi143.stdout: inet6 fe80::ec4:7aff:fe88:7783/64 scope link 2023-11-26T07:22:02.674 INFO:teuthology.orchestra.run.smithi143.stdout: valid_lft forever preferred_lft forever 2023-11-26T07:22:02.674 INFO:teuthology.orchestra.run.smithi143.stdout:6: docker0: mtu 1500 qdisc noqueue state DOWN group default 2023-11-26T07:22:02.674 INFO:teuthology.orchestra.run.smithi143.stdout: link/ether 02:42:c4:b4:2f:eb brd ff:ff:ff:ff:ff:ff 2023-11-26T07:22:02.674 INFO:teuthology.orchestra.run.smithi143.stdout: inet 172.17.0.1/16 brd 172.17.255.255 scope global docker0 2023-11-26T07:22:02.674 INFO:teuthology.orchestra.run.smithi143.stdout: valid_lft forever preferred_lft forever 2023-11-26T07:22:02.674 INFO:teuthology.orchestra.run.smithi143.stdout:7: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2023-11-26T07:22:02.675 INFO:teuthology.orchestra.run.smithi143.stdout: link/ether 5e:1f:84:6d:79:ae brd ff:ff:ff:ff:ff:ff 2023-11-26T07:22:02.675 INFO:teuthology.orchestra.run.smithi143.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2023-11-26T07:22:02.675 INFO:teuthology.orchestra.run.smithi143.stdout: valid_lft forever preferred_lft forever 2023-11-26T07:22:02.675 INFO:teuthology.orchestra.run.smithi143.stdout: inet6 fe80::5c1f:84ff:fe6d:79ae/64 scope link 2023-11-26T07:22:02.675 INFO:teuthology.orchestra.run.smithi143.stdout: valid_lft forever preferred_lft forever 2023-11-26T07:22:02.675 INFO:teuthology.orchestra.run.smithi143.stdout:8: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2023-11-26T07:22:02.675 INFO:teuthology.orchestra.run.smithi143.stdout: link/ether 0e:5c:f6:1f:82:2f brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.0 2023-11-26T07:22:02.675 INFO:teuthology.orchestra.run.smithi143.stdout: inet6 fe80::c5c:f6ff:fe1f:822f/64 scope link 2023-11-26T07:22:02.675 INFO:teuthology.orchestra.run.smithi143.stdout: valid_lft forever preferred_lft forever 2023-11-26T07:22:02.675 INFO:teuthology.orchestra.run.smithi143.stdout:9: brx.1@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2023-11-26T07:22:02.675 INFO:teuthology.orchestra.run.smithi143.stdout: link/ether c6:a6:3d:69:d0:db brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.2 2023-11-26T07:22:02.675 INFO:teuthology.orchestra.run.smithi143.stdout: inet6 fe80::c4a6:3dff:fe69:d0db/64 scope link 2023-11-26T07:22:02.675 INFO:teuthology.orchestra.run.smithi143.stdout: valid_lft forever preferred_lft forever 2023-11-26T07:22:02.676 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-26T07:22:02.676 DEBUG:teuthology.orchestra.run.smithi143:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2023-11-26T07:22:02.738 INFO:teuthology.orchestra.run.smithi143.stdout:1 2023-11-26T07:22:02.740 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-26T07:22:02.741 DEBUG:teuthology.orchestra.run.smithi143:> route 2023-11-26T07:22:02.798 INFO:teuthology.orchestra.run.smithi143.stdout:Kernel IP routing table 2023-11-26T07:22:02.798 INFO:teuthology.orchestra.run.smithi143.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2023-11-26T07:22:02.798 INFO:teuthology.orchestra.run.smithi143.stdout:default _gateway 0.0.0.0 UG 0 0 0 enp3s0f1 2023-11-26T07:22:02.798 INFO:teuthology.orchestra.run.smithi143.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2023-11-26T07:22:02.798 INFO:teuthology.orchestra.run.smithi143.stdout:172.17.0.0 0.0.0.0 255.255.0.0 U 0 0 0 docker0 2023-11-26T07:22:02.799 INFO:teuthology.orchestra.run.smithi143.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 enp3s0f1 2023-11-26T07:22:02.799 INFO:teuthology.orchestra.run.smithi143.stdout:ns1.front.sepia 0.0.0.0 255.255.255.255 UH 100 0 0 enp3s0f1 2023-11-26T07:22:02.799 INFO:teuthology.orchestra.run.smithi143.stdout:172.21.0.2 0.0.0.0 255.255.255.255 UH 100 0 0 enp3s0f1 2023-11-26T07:22:02.799 INFO:teuthology.orchestra.run.smithi143.stdout:_gateway 0.0.0.0 255.255.255.255 UH 100 0 0 enp3s0f1 2023-11-26T07:22:02.799 INFO:teuthology.orchestra.run.smithi143.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2023-11-26T07:22:02.799 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-26T07:22:02.799 DEBUG:teuthology.orchestra.run.smithi143:> (cd / && exec bash -c ' 2023-11-26T07:22:02.799 DEBUG:teuthology.orchestra.run.smithi143:> set -e 2023-11-26T07:22:02.799 DEBUG:teuthology.orchestra.run.smithi143:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2023-11-26T07:22:02.799 DEBUG:teuthology.orchestra.run.smithi143:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2023-11-26T07:22:02.800 DEBUG:teuthology.orchestra.run.smithi143:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2023-11-26T07:22:02.800 DEBUG:teuthology.orchestra.run.smithi143:> ') 2023-11-26T07:22:02.890 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-26T07:22:02.891 DEBUG:teuthology.orchestra.run.smithi143:> ip netns list 2023-11-26T07:22:02.900 INFO:teuthology.orchestra.run.smithi143.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2023-11-26T07:22:02.900 INFO:teuthology.orchestra.run.smithi143.stdout:ceph-ns--home-ubuntu-cephtest-mnt.2 (id: 1) 2023-11-26T07:22:02.900 INFO:tasks.cephfs.mount:Resuming the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2023-11-26T07:22:02.901 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-26T07:22:02.901 DEBUG:teuthology.orchestra.run.smithi143:> sudo ip link set brx.0 up 2023-11-26T07:22:02.968 INFO:tasks.cephfs.fuse_mount:Client client.0 config is {} 2023-11-26T07:22:02.968 INFO:teuthology.orchestra.run:Running command with timeout 60 2023-11-26T07:22:02.968 DEBUG:teuthology.orchestra.run.smithi143:> mkdir -p -v /home/ubuntu/cephtest/mnt.0 2023-11-26T07:22:03.020 INFO:teuthology.orchestra.run.smithi143.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.0' 2023-11-26T07:22:03.020 INFO:teuthology.orchestra.run:Running command with timeout 60 2023-11-26T07:22:03.020 DEBUG:teuthology.orchestra.run.smithi143:> chmod 0000 /home/ubuntu/cephtest/mnt.0 2023-11-26T07:22:03.071 DEBUG:teuthology.orchestra.run.smithi143:> sudo modprobe fuse 2023-11-26T07:22:03.132 DEBUG:teuthology.orchestra.run.smithi143:> cat /proc/self/mounts | awk '{print $2}' 2023-11-26T07:22:03.185 INFO:teuthology.orchestra.run.smithi143.stdout:/sys 2023-11-26T07:22:03.185 INFO:teuthology.orchestra.run.smithi143.stdout:/proc 2023-11-26T07:22:03.185 INFO:teuthology.orchestra.run.smithi143.stdout:/dev 2023-11-26T07:22:03.185 INFO:teuthology.orchestra.run.smithi143.stdout:/dev/pts 2023-11-26T07:22:03.186 INFO:teuthology.orchestra.run.smithi143.stdout:/run 2023-11-26T07:22:03.186 INFO:teuthology.orchestra.run.smithi143.stdout:/ 2023-11-26T07:22:03.186 INFO:teuthology.orchestra.run.smithi143.stdout:/sys/kernel/security 2023-11-26T07:22:03.186 INFO:teuthology.orchestra.run.smithi143.stdout:/dev/shm 2023-11-26T07:22:03.186 INFO:teuthology.orchestra.run.smithi143.stdout:/run/lock 2023-11-26T07:22:03.186 INFO:teuthology.orchestra.run.smithi143.stdout:/sys/fs/cgroup 2023-11-26T07:22:03.186 INFO:teuthology.orchestra.run.smithi143.stdout:/sys/fs/pstore 2023-11-26T07:22:03.186 INFO:teuthology.orchestra.run.smithi143.stdout:/sys/fs/bpf 2023-11-26T07:22:03.186 INFO:teuthology.orchestra.run.smithi143.stdout:/proc/sys/fs/binfmt_misc 2023-11-26T07:22:03.186 INFO:teuthology.orchestra.run.smithi143.stdout:/dev/hugepages 2023-11-26T07:22:03.186 INFO:teuthology.orchestra.run.smithi143.stdout:/dev/mqueue 2023-11-26T07:22:03.186 INFO:teuthology.orchestra.run.smithi143.stdout:/sys/kernel/debug 2023-11-26T07:22:03.187 INFO:teuthology.orchestra.run.smithi143.stdout:/sys/kernel/tracing 2023-11-26T07:22:03.187 INFO:teuthology.orchestra.run.smithi143.stdout:/proc/fs/nfsd 2023-11-26T07:22:03.187 INFO:teuthology.orchestra.run.smithi143.stdout:/sys/fs/fuse/connections 2023-11-26T07:22:03.187 INFO:teuthology.orchestra.run.smithi143.stdout:/sys/kernel/config 2023-11-26T07:22:03.187 INFO:teuthology.orchestra.run.smithi143.stdout:/run/credentials/systemd-sysusers.service 2023-11-26T07:22:03.187 INFO:teuthology.orchestra.run.smithi143.stdout:/run/rpc_pipefs 2023-11-26T07:22:03.187 INFO:teuthology.orchestra.run.smithi143.stdout:/run/user/1000 2023-11-26T07:22:03.187 INFO:teuthology.orchestra.run.smithi143.stdout:/var/lib/ceph 2023-11-26T07:22:03.187 INFO:teuthology.orchestra.run.smithi143.stdout:/var/lib/ceph/osd/ceph-0 2023-11-26T07:22:03.187 INFO:teuthology.orchestra.run.smithi143.stdout:/var/lib/ceph/osd/ceph-1 2023-11-26T07:22:03.187 INFO:teuthology.orchestra.run.smithi143.stdout:/var/lib/ceph/osd/ceph-2 2023-11-26T07:22:03.187 INFO:teuthology.orchestra.run.smithi143.stdout:/var/lib/ceph/osd/ceph-3 2023-11-26T07:22:03.188 INFO:teuthology.orchestra.run.smithi143.stdout:/run/netns 2023-11-26T07:22:03.188 INFO:teuthology.orchestra.run.smithi143.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2023-11-26T07:22:03.188 INFO:teuthology.orchestra.run.smithi143.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2023-11-26T07:22:03.188 INFO:teuthology.orchestra.run.smithi143.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 2023-11-26T07:22:03.188 INFO:teuthology.orchestra.run.smithi143.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 2023-11-26T07:22:03.189 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-26T07:22:03.189 DEBUG:teuthology.orchestra.run.smithi143:> ls /sys/fs/fuse/connections 2023-11-26T07:22:03.243 INFO:tasks.cephfs.fuse_mount:Pre-mount connections: [] 2023-11-26T07:22:03.243 DEBUG:teuthology.orchestra.run.smithi143:> (cd /home/ubuntu/cephtest && exec sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-fuse -f --admin-socket '/var/run/ceph/$cluster-$name.$pid.asok' /home/ubuntu/cephtest/mnt.0 --id 0) 2023-11-26T07:22:03.286 DEBUG:teuthology.orchestra.run.smithi143:> sudo modprobe fuse 2023-11-26T07:22:03.309 DEBUG:teuthology.orchestra.run.smithi143:> cat /proc/self/mounts | awk '{print $2}' 2023-11-26T07:22:03.362 INFO:teuthology.orchestra.run.smithi143.stdout:/sys 2023-11-26T07:22:03.362 INFO:teuthology.orchestra.run.smithi143.stdout:/proc 2023-11-26T07:22:03.362 INFO:teuthology.orchestra.run.smithi143.stdout:/dev 2023-11-26T07:22:03.362 INFO:teuthology.orchestra.run.smithi143.stdout:/dev/pts 2023-11-26T07:22:03.362 INFO:teuthology.orchestra.run.smithi143.stdout:/run 2023-11-26T07:22:03.362 INFO:teuthology.orchestra.run.smithi143.stdout:/ 2023-11-26T07:22:03.362 INFO:teuthology.orchestra.run.smithi143.stdout:/sys/kernel/security 2023-11-26T07:22:03.362 INFO:teuthology.orchestra.run.smithi143.stdout:/dev/shm 2023-11-26T07:22:03.362 INFO:teuthology.orchestra.run.smithi143.stdout:/run/lock 2023-11-26T07:22:03.363 INFO:teuthology.orchestra.run.smithi143.stdout:/sys/fs/cgroup 2023-11-26T07:22:03.363 INFO:teuthology.orchestra.run.smithi143.stdout:/sys/fs/pstore 2023-11-26T07:22:03.363 INFO:teuthology.orchestra.run.smithi143.stdout:/sys/fs/bpf 2023-11-26T07:22:03.363 INFO:teuthology.orchestra.run.smithi143.stdout:/proc/sys/fs/binfmt_misc 2023-11-26T07:22:03.363 INFO:teuthology.orchestra.run.smithi143.stdout:/dev/hugepages 2023-11-26T07:22:03.363 INFO:teuthology.orchestra.run.smithi143.stdout:/dev/mqueue 2023-11-26T07:22:03.363 INFO:teuthology.orchestra.run.smithi143.stdout:/sys/kernel/debug 2023-11-26T07:22:03.363 INFO:teuthology.orchestra.run.smithi143.stdout:/sys/kernel/tracing 2023-11-26T07:22:03.363 INFO:teuthology.orchestra.run.smithi143.stdout:/proc/fs/nfsd 2023-11-26T07:22:03.363 INFO:teuthology.orchestra.run.smithi143.stdout:/sys/fs/fuse/connections 2023-11-26T07:22:03.363 INFO:teuthology.orchestra.run.smithi143.stdout:/sys/kernel/config 2023-11-26T07:22:03.363 INFO:teuthology.orchestra.run.smithi143.stdout:/run/credentials/systemd-sysusers.service 2023-11-26T07:22:03.364 INFO:teuthology.orchestra.run.smithi143.stdout:/run/rpc_pipefs 2023-11-26T07:22:03.364 INFO:teuthology.orchestra.run.smithi143.stdout:/run/user/1000 2023-11-26T07:22:03.364 INFO:teuthology.orchestra.run.smithi143.stdout:/var/lib/ceph 2023-11-26T07:22:03.364 INFO:teuthology.orchestra.run.smithi143.stdout:/var/lib/ceph/osd/ceph-0 2023-11-26T07:22:03.364 INFO:teuthology.orchestra.run.smithi143.stdout:/var/lib/ceph/osd/ceph-1 2023-11-26T07:22:03.364 INFO:teuthology.orchestra.run.smithi143.stdout:/var/lib/ceph/osd/ceph-2 2023-11-26T07:22:03.364 INFO:teuthology.orchestra.run.smithi143.stdout:/var/lib/ceph/osd/ceph-3 2023-11-26T07:22:03.364 INFO:teuthology.orchestra.run.smithi143.stdout:/run/netns 2023-11-26T07:22:03.364 INFO:teuthology.orchestra.run.smithi143.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2023-11-26T07:22:03.364 INFO:teuthology.orchestra.run.smithi143.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2023-11-26T07:22:03.364 INFO:teuthology.orchestra.run.smithi143.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 2023-11-26T07:22:03.364 INFO:teuthology.orchestra.run.smithi143.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 2023-11-26T07:22:03.365 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-26T07:22:03.365 DEBUG:teuthology.orchestra.run.smithi143:> ls /sys/fs/fuse/connections 2023-11-26T07:22:03.384 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi143.stderr:2023-11-26T07:22:03.380+0000 7f093c383580 -1 init, newargv = 0x5654ef67a380 newargc=15 2023-11-26T07:22:03.384 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi143.stderr:ceph-fuse[53383]: starting ceph client 2023-11-26T07:22:03.392 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi143.stderr:ceph-fuse[53383]: starting fuse 2023-11-26T07:22:03.393 INFO:teuthology.orchestra.run.smithi143.stdout:43 2023-11-26T07:22:03.393 INFO:tasks.cephfs.fuse_mount:Post-mount connections: [43] 2023-11-26T07:22:03.394 DEBUG:teuthology.orchestra.run.smithi143:> sudo stdin-killer -- python3 -c ' 2023-11-26T07:22:03.394 DEBUG:teuthology.orchestra.run.smithi143:> import glob 2023-11-26T07:22:03.394 DEBUG:teuthology.orchestra.run.smithi143:> import re 2023-11-26T07:22:03.394 DEBUG:teuthology.orchestra.run.smithi143:> import os 2023-11-26T07:22:03.394 DEBUG:teuthology.orchestra.run.smithi143:> import subprocess 2023-11-26T07:22:03.394 DEBUG:teuthology.orchestra.run.smithi143:> 2023-11-26T07:22:03.394 DEBUG:teuthology.orchestra.run.smithi143:> def _find_admin_socket(client_name): 2023-11-26T07:22:03.394 DEBUG:teuthology.orchestra.run.smithi143:> asok_path = "/var/run/ceph/ceph-client.0.*.asok" 2023-11-26T07:22:03.394 DEBUG:teuthology.orchestra.run.smithi143:> files = glob.glob(asok_path) 2023-11-26T07:22:03.394 DEBUG:teuthology.orchestra.run.smithi143:> mountpoint = "/home/ubuntu/cephtest/mnt.0" 2023-11-26T07:22:03.395 DEBUG:teuthology.orchestra.run.smithi143:> 2023-11-26T07:22:03.395 DEBUG:teuthology.orchestra.run.smithi143:> # Given a non-glob path, it better be there 2023-11-26T07:22:03.395 DEBUG:teuthology.orchestra.run.smithi143:> if "*" not in asok_path: 2023-11-26T07:22:03.395 DEBUG:teuthology.orchestra.run.smithi143:> assert(len(files) == 1) 2023-11-26T07:22:03.395 DEBUG:teuthology.orchestra.run.smithi143:> return files[0] 2023-11-26T07:22:03.395 DEBUG:teuthology.orchestra.run.smithi143:> 2023-11-26T07:22:03.395 DEBUG:teuthology.orchestra.run.smithi143:> for f in files: 2023-11-26T07:22:03.395 DEBUG:teuthology.orchestra.run.smithi143:> pid = re.match(".*\.(\d+)\.asok$", f).group(1) 2023-11-26T07:22:03.395 DEBUG:teuthology.orchestra.run.smithi143:> if os.path.exists("/proc/{0}".format(pid)): 2023-11-26T07:22:03.395 DEBUG:teuthology.orchestra.run.smithi143:> with open("/proc/{0}/cmdline".format(pid), '"'"'r'"'"') as proc_f: 2023-11-26T07:22:03.395 DEBUG:teuthology.orchestra.run.smithi143:> contents = proc_f.read() 2023-11-26T07:22:03.395 DEBUG:teuthology.orchestra.run.smithi143:> if mountpoint in contents: 2023-11-26T07:22:03.395 DEBUG:teuthology.orchestra.run.smithi143:> return f 2023-11-26T07:22:03.396 DEBUG:teuthology.orchestra.run.smithi143:> raise RuntimeError("Client socket {0} not found".format(client_name)) 2023-11-26T07:22:03.396 DEBUG:teuthology.orchestra.run.smithi143:> 2023-11-26T07:22:03.396 DEBUG:teuthology.orchestra.run.smithi143:> print(_find_admin_socket("client.0")) 2023-11-26T07:22:03.396 DEBUG:teuthology.orchestra.run.smithi143:> ' 2023-11-26T07:22:03.520 INFO:teuthology.orchestra.run.smithi143.stdout:/var/run/ceph/ceph-client.0.53383.asok 2023-11-26T07:22:03.523 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:03 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2023-11-26T07:22:03.531 INFO:tasks.cephfs.fuse_mount:Found client admin socket at /var/run/ceph/ceph-client.0.53383.asok 2023-11-26T07:22:03.531 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-26T07:22:03.531 DEBUG:teuthology.orchestra.run.smithi143:> sudo ceph --admin-daemon /var/run/ceph/ceph-client.0.53383.asok status 2023-11-26T07:22:03.700 INFO:teuthology.orchestra.run.smithi143.stdout:{ 2023-11-26T07:22:03.700 INFO:teuthology.orchestra.run.smithi143.stdout: "metadata": { 2023-11-26T07:22:03.700 INFO:teuthology.orchestra.run.smithi143.stdout: "ceph_sha1": "44c2237fe77713214148c1fd622c872f346f1b99", 2023-11-26T07:22:03.700 INFO:teuthology.orchestra.run.smithi143.stdout: "ceph_version": "ceph version 18.0.0-7484-g44c2237f (44c2237fe77713214148c1fd622c872f346f1b99) reef (dev)", 2023-11-26T07:22:03.700 INFO:teuthology.orchestra.run.smithi143.stdout: "entity_id": "0", 2023-11-26T07:22:03.700 INFO:teuthology.orchestra.run.smithi143.stdout: "hostname": "smithi143", 2023-11-26T07:22:03.701 INFO:teuthology.orchestra.run.smithi143.stdout: "mount_point": "/home/ubuntu/cephtest/mnt.0", 2023-11-26T07:22:03.701 INFO:teuthology.orchestra.run.smithi143.stdout: "pid": "53383", 2023-11-26T07:22:03.701 INFO:teuthology.orchestra.run.smithi143.stdout: "root": "/" 2023-11-26T07:22:03.701 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2023-11-26T07:22:03.701 INFO:teuthology.orchestra.run.smithi143.stdout: "dentry_count": 0, 2023-11-26T07:22:03.701 INFO:teuthology.orchestra.run.smithi143.stdout: "dentry_pinned_count": 0, 2023-11-26T07:22:03.701 INFO:teuthology.orchestra.run.smithi143.stdout: "id": 6570, 2023-11-26T07:22:03.701 INFO:teuthology.orchestra.run.smithi143.stdout: "inst": { 2023-11-26T07:22:03.701 INFO:teuthology.orchestra.run.smithi143.stdout: "name": { 2023-11-26T07:22:03.701 INFO:teuthology.orchestra.run.smithi143.stdout: "type": "client", 2023-11-26T07:22:03.701 INFO:teuthology.orchestra.run.smithi143.stdout: "num": 6570 2023-11-26T07:22:03.702 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2023-11-26T07:22:03.702 INFO:teuthology.orchestra.run.smithi143.stdout: "addr": { 2023-11-26T07:22:03.702 INFO:teuthology.orchestra.run.smithi143.stdout: "type": "v1", 2023-11-26T07:22:03.702 INFO:teuthology.orchestra.run.smithi143.stdout: "addr": "192.168.0.1:0", 2023-11-26T07:22:03.702 INFO:teuthology.orchestra.run.smithi143.stdout: "nonce": 2705939635 2023-11-26T07:22:03.702 INFO:teuthology.orchestra.run.smithi143.stdout: } 2023-11-26T07:22:03.702 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2023-11-26T07:22:03.702 INFO:teuthology.orchestra.run.smithi143.stdout: "addr": { 2023-11-26T07:22:03.702 INFO:teuthology.orchestra.run.smithi143.stdout: "type": "v1", 2023-11-26T07:22:03.702 INFO:teuthology.orchestra.run.smithi143.stdout: "addr": "192.168.0.1:0", 2023-11-26T07:22:03.702 INFO:teuthology.orchestra.run.smithi143.stdout: "nonce": 2705939635 2023-11-26T07:22:03.703 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2023-11-26T07:22:03.703 INFO:teuthology.orchestra.run.smithi143.stdout: "inst_str": "client.6570 192.168.0.1:0/2705939635", 2023-11-26T07:22:03.703 INFO:teuthology.orchestra.run.smithi143.stdout: "addr_str": "192.168.0.1:0/2705939635", 2023-11-26T07:22:03.703 INFO:teuthology.orchestra.run.smithi143.stdout: "inode_count": 1, 2023-11-26T07:22:03.703 INFO:teuthology.orchestra.run.smithi143.stdout: "mds_epoch": 34, 2023-11-26T07:22:03.703 INFO:teuthology.orchestra.run.smithi143.stdout: "osd_epoch": 59, 2023-11-26T07:22:03.703 INFO:teuthology.orchestra.run.smithi143.stdout: "osd_epoch_barrier": 0, 2023-11-26T07:22:03.703 INFO:teuthology.orchestra.run.smithi143.stdout: "blocklisted": false, 2023-11-26T07:22:03.703 INFO:teuthology.orchestra.run.smithi143.stdout: "fs_name": "cephfs" 2023-11-26T07:22:03.703 INFO:teuthology.orchestra.run.smithi143.stdout:} 2023-11-26T07:22:03.715 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-26T07:22:03.715 DEBUG:teuthology.orchestra.run.smithi143:> stat --file-system '--printf=%T 2023-11-26T07:22:03.715 DEBUG:teuthology.orchestra.run.smithi143:> ' -- /home/ubuntu/cephtest/mnt.0 2023-11-26T07:22:03.725 INFO:teuthology.orchestra.run.smithi143.stdout:fuseblk 2023-11-26T07:22:03.725 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.0 2023-11-26T07:22:03.725 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-26T07:22:03.725 DEBUG:teuthology.orchestra.run.smithi143:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.0 2023-11-26T07:22:03.789 DEBUG:teuthology.orchestra.run.smithi143:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2023-11-26T07:22:03.970 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:03.968+0000 7f1311f01640 1 Processor -- start 2023-11-26T07:22:03.970 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:03.968+0000 7f1311f01640 1 -- start start 2023-11-26T07:22:03.971 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:03.968+0000 7f1311f01640 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7f130c13e290 0x7f130c0cf280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:22:03.971 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:03.968+0000 7f1311f01640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f130c0cf7c0 0x7f130c15efa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:22:03.971 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:03.968+0000 7f1311f01640 1 -- --> v1:172.21.15.188:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f130c06c520 con 0x7f130c15f4e0 2023-11-26T07:22:03.971 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:03.968+0000 7f1311f01640 1 -- --> v2:172.21.15.188:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f130c0cfc40 con 0x7f130c13e290 2023-11-26T07:22:03.971 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:03.968+0000 7f1311f01640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f130c06c800 con 0x7f130c0cf7c0 2023-11-26T07:22:03.972 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:03.968+0000 7f130b7fe640 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7f130c13e290 0x7f130c0cf280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:22:03.973 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:03.968+0000 7f130affd640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f130c0cf7c0 0x7f130c15efa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:22:03.973 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:03.968+0000 7f130bfff640 1 --1- >> v1:172.21.15.188:6789/0 conn(0x7f130c15f4e0 0x7f130c1408e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.188:6789/0 says I am v1:172.21.15.143:41200/0 (socket says 172.21.15.143:41200) 2023-11-26T07:22:03.973 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:03.968+0000 7f130bfff640 1 -- 172.21.15.143:0/2545372381 learned_addr learned my addr 172.21.15.143:0/2545372381 (peer_addr_for_me v1:172.21.15.143:0/0) 2023-11-26T07:22:03.973 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:03.972+0000 7f130a7fc640 1 -- 172.21.15.143:0/2545372381 <== mon.1 v1:172.21.15.188:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1960431515 0 0) 0x7f130c06c520 con 0x7f130c15f4e0 2023-11-26T07:22:03.973 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:03.972+0000 7f130a7fc640 1 -- 172.21.15.143:0/2545372381 --> v1:172.21.15.188:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f12f4003530 con 0x7f130c15f4e0 2023-11-26T07:22:03.973 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:03.972+0000 7f130b7fe640 1 -- 172.21.15.143:0/2545372381 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f130c0cf7c0 msgr2=0x7f130c15efa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-26T07:22:03.973 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:03.972+0000 7f130b7fe640 1 --2- 172.21.15.143:0/2545372381 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f130c0cf7c0 0x7f130c15efa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:22:03.973 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:03.972+0000 7f130b7fe640 1 -- 172.21.15.143:0/2545372381 >> v1:172.21.15.188:6789/0 conn(0x7f130c15f4e0 legacy=0x7f130c1408e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:22:03.973 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:03.972+0000 7f130b7fe640 1 -- 172.21.15.143:0/2545372381 --> v2:172.21.15.188:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f130c06cae0 con 0x7f130c13e290 2023-11-26T07:22:03.973 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:03.972+0000 7f130affd640 1 --2- 172.21.15.143:0/2545372381 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f130c0cf7c0 0x7f130c15efa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-26T07:22:03.975 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:03.972+0000 7f130b7fe640 1 --2- 172.21.15.143:0/2545372381 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f130c13e290 0x7f130c0cf280 secure :-1 s=READY pgs=325 cs=0 l=1 rev1=1 crypto rx=0x7f12fc0097c0 tx=0x7f12fc01c450 comp rx=0 tx=0).ready entity=mon.1 client_cookie=fd6b78373f9a869 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:22:03.975 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:03.972+0000 7f130a7fc640 1 -- 172.21.15.143:0/2545372381 <== mon.1 v2:172.21.15.188:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f12fc0ab650 con 0x7f130c13e290 2023-11-26T07:22:03.976 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:03.972+0000 7f130a7fc640 1 -- 172.21.15.143:0/2545372381 <== mon.1 v2:172.21.15.188:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f12fc0ab7b0 con 0x7f130c13e290 2023-11-26T07:22:03.976 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:03.972+0000 7f130a7fc640 1 -- 172.21.15.143:0/2545372381 <== mon.1 v2:172.21.15.188:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f12fc0abb40 con 0x7f130c13e290 2023-11-26T07:22:03.976 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:03.976+0000 7f1311f01640 1 -- 172.21.15.143:0/2545372381 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f130c13e290 msgr2=0x7f130c0cf280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:22:03.977 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:03.976+0000 7f1311f01640 1 --2- 172.21.15.143:0/2545372381 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f130c13e290 0x7f130c0cf280 secure :-1 s=READY pgs=325 cs=0 l=1 rev1=1 crypto rx=0x7f12fc0097c0 tx=0x7f12fc01c450 comp rx=0 tx=0).stop 2023-11-26T07:22:03.977 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:03.976+0000 7f1311f01640 1 -- 172.21.15.143:0/2545372381 shutdown_connections 2023-11-26T07:22:03.977 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:03.976+0000 7f1311f01640 1 --2- 172.21.15.143:0/2545372381 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f130c0cf7c0 0x7f130c15efa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:22:03.977 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:03.976+0000 7f1311f01640 1 --2- 172.21.15.143:0/2545372381 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f130c13e290 0x7f130c0cf280 unknown :-1 s=CLOSED pgs=325 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:22:03.977 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:03.976+0000 7f1311f01640 1 -- 172.21.15.143:0/2545372381 >> 172.21.15.143:0/2545372381 conn(0x7f130c06a890 msgr2=0x7f130c06aca0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:22:03.977 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:03.976+0000 7f1311f01640 1 -- 172.21.15.143:0/2545372381 shutdown_connections 2023-11-26T07:22:03.977 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:03.976+0000 7f1311f01640 1 -- 172.21.15.143:0/2545372381 wait complete. 2023-11-26T07:22:03.977 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:03.976+0000 7f1311f01640 1 Processor -- start 2023-11-26T07:22:03.978 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:03.976+0000 7f1311f01640 1 -- start start 2023-11-26T07:22:03.980 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:03.976+0000 7f1311f01640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f130c0cf7c0 0x7f130c0b1630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:22:03.980 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:03.976+0000 7f1311f01640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f130c13e290 0x7f130c15dd70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:22:03.980 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:03.976+0000 7f1311f01640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f130c15f4e0 0x7f130c136e70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:22:03.980 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:03.976+0000 7f1311f01640 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f130c15e2b0 con 0x7f130c0cf7c0 2023-11-26T07:22:03.980 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:03.976+0000 7f1311f01640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f130c071510 con 0x7f130c13e290 2023-11-26T07:22:03.980 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:03.976+0000 7f1311f01640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f130c0b0c00 con 0x7f130c15f4e0 2023-11-26T07:22:03.980 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:03.976+0000 7f130b7fe640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f130c0cf7c0 0x7f130c0b1630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:22:03.981 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:03.976+0000 7f130b7fe640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f130c0cf7c0 0x7f130c0b1630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:57362/0 (socket says 172.21.15.143:57362) 2023-11-26T07:22:03.981 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:03.976+0000 7f130b7fe640 1 -- 172.21.15.143:0/2173905727 learned_addr learned my addr 172.21.15.143:0/2173905727 (peer_addr_for_me v2:172.21.15.143:0/0) 2023-11-26T07:22:03.981 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:03.976+0000 7f130bfff640 1 --2- 172.21.15.143:0/2173905727 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f130c15f4e0 0x7f130c136e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:22:03.981 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:03.976+0000 7f130affd640 1 --2- 172.21.15.143:0/2173905727 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f130c13e290 0x7f130c15dd70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:22:03.981 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:03.976+0000 7f130b7fe640 1 -- 172.21.15.143:0/2173905727 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f130c15f4e0 msgr2=0x7f130c136e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:22:03.981 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:03.976+0000 7f130b7fe640 1 --2- 172.21.15.143:0/2173905727 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f130c15f4e0 0x7f130c136e70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:22:03.981 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:03.976+0000 7f130b7fe640 1 -- 172.21.15.143:0/2173905727 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f130c13e290 msgr2=0x7f130c15dd70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:22:03.981 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:03.976+0000 7f130b7fe640 1 --2- 172.21.15.143:0/2173905727 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f130c13e290 0x7f130c15dd70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:22:03.981 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:03.976+0000 7f130b7fe640 1 -- 172.21.15.143:0/2173905727 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1300009420 con 0x7f130c0cf7c0 2023-11-26T07:22:03.981 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:03.976+0000 7f130bfff640 1 --2- 172.21.15.143:0/2173905727 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f130c15f4e0 0x7f130c136e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-26T07:22:03.981 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:03.980+0000 7f130b7fe640 1 --2- 172.21.15.143:0/2173905727 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f130c0cf7c0 0x7f130c0b1630 secure :-1 s=READY pgs=820 cs=0 l=1 rev1=1 crypto rx=0x7f12fc0ad040 tx=0x7f12fc01ac90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:22:03.982 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:03.980+0000 7f12f3fff640 1 -- 172.21.15.143:0/2173905727 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f12fc0bf610 con 0x7f130c0cf7c0 2023-11-26T07:22:03.983 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:03.980+0000 7f12f3fff640 1 -- 172.21.15.143:0/2173905727 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f12fc0bf7b0 con 0x7f130c0cf7c0 2023-11-26T07:22:03.983 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:03.980+0000 7f12f3fff640 1 -- 172.21.15.143:0/2173905727 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f12fc0bfb60 con 0x7f130c0cf7c0 2023-11-26T07:22:03.983 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:03.980+0000 7f1311f01640 1 -- 172.21.15.143:0/2173905727 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f12fc009420 con 0x7f130c0cf7c0 2023-11-26T07:22:03.983 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:03.980+0000 7f1311f01640 1 -- 172.21.15.143:0/2173905727 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f130c071d00 con 0x7f130c0cf7c0 2023-11-26T07:22:03.983 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:03.980+0000 7f1311f01640 1 -- 172.21.15.143:0/2173905727 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f130c067680 con 0x7f130c0cf7c0 2023-11-26T07:22:03.983 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:03.980+0000 7f12f3fff640 1 -- 172.21.15.143:0/2173905727 <== mon.0 v2:172.21.15.143:3300/0 4 ==== mgrmap(e 9) v1 ==== 86005+0+0 (secure 0 0 0) 0x7f12fc0b5400 con 0x7f130c0cf7c0 2023-11-26T07:22:03.984 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:03.980+0000 7f12f3fff640 1 --2- 172.21.15.143:0/2173905727 >> [v2:172.21.15.143:6832/2150557544,v1:172.21.15.143:6833/2150557544] conn(0x7f12f8066650 0x7f12f8068b10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:22:03.984 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:03.980+0000 7f12f3fff640 1 -- 172.21.15.143:0/2173905727 <== mon.0 v2:172.21.15.143:3300/0 5 ==== osd_map(59..59 src has 1..59) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f12fc11ee10 con 0x7f130c0cf7c0 2023-11-26T07:22:03.984 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:03.980+0000 7f130affd640 1 --2- 172.21.15.143:0/2173905727 >> [v2:172.21.15.143:6832/2150557544,v1:172.21.15.143:6833/2150557544] conn(0x7f12f8066650 0x7f12f8068b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:22:03.984 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:03.984+0000 7f130affd640 1 --2- 172.21.15.143:0/2173905727 >> [v2:172.21.15.143:6832/2150557544,v1:172.21.15.143:6833/2150557544] conn(0x7f12f8066650 0x7f12f8068b10 secure :-1 s=READY pgs=390 cs=0 l=1 rev1=1 crypto rx=0x7f130c0b1c00 tx=0x7f1300008040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:22:03.987 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:03.984+0000 7f12f3fff640 1 -- 172.21.15.143:0/2173905727 <== mon.0 v2:172.21.15.143:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172649 (secure 0 0 0) 0x7f12fc122030 con 0x7f130c0cf7c0 2023-11-26T07:22:04.174 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:04.172+0000 7f1311f01640 1 -- 172.21.15.143:0/2173905727 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "fs ls"} v 0) v1 -- 0x7f130c154eb0 con 0x7f130c0cf7c0 2023-11-26T07:22:04.174 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:04.172+0000 7f12f3fff640 1 -- 172.21.15.143:0/2173905727 <== mon.0 v2:172.21.15.143:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v34) v1 ==== 53+0+73 (secure 0 0 0) 0x7f12fc0ec140 con 0x7f130c0cf7c0 2023-11-26T07:22:04.174 INFO:teuthology.orchestra.run.smithi143.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2023-11-26T07:22:04.176 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:04.172+0000 7f1311f01640 1 -- 172.21.15.143:0/2173905727 >> [v2:172.21.15.143:6832/2150557544,v1:172.21.15.143:6833/2150557544] conn(0x7f12f8066650 msgr2=0x7f12f8068b10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:22:04.176 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:04.172+0000 7f1311f01640 1 --2- 172.21.15.143:0/2173905727 >> [v2:172.21.15.143:6832/2150557544,v1:172.21.15.143:6833/2150557544] conn(0x7f12f8066650 0x7f12f8068b10 secure :-1 s=READY pgs=390 cs=0 l=1 rev1=1 crypto rx=0x7f130c0b1c00 tx=0x7f1300008040 comp rx=0 tx=0).stop 2023-11-26T07:22:04.176 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:04.172+0000 7f1311f01640 1 -- 172.21.15.143:0/2173905727 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f130c0cf7c0 msgr2=0x7f130c0b1630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:22:04.176 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:04.172+0000 7f1311f01640 1 --2- 172.21.15.143:0/2173905727 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f130c0cf7c0 0x7f130c0b1630 secure :-1 s=READY pgs=820 cs=0 l=1 rev1=1 crypto rx=0x7f12fc0ad040 tx=0x7f12fc01ac90 comp rx=0 tx=0).stop 2023-11-26T07:22:04.177 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:04.172+0000 7f1311f01640 1 -- 172.21.15.143:0/2173905727 shutdown_connections 2023-11-26T07:22:04.177 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:04.172+0000 7f1311f01640 1 --2- 172.21.15.143:0/2173905727 >> [v2:172.21.15.143:6832/2150557544,v1:172.21.15.143:6833/2150557544] conn(0x7f12f8066650 0x7f12f8068b10 unknown :-1 s=CLOSED pgs=390 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:22:04.177 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:04.172+0000 7f1311f01640 1 --2- 172.21.15.143:0/2173905727 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f130c15f4e0 0x7f130c136e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:22:04.177 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:04.172+0000 7f1311f01640 1 --2- 172.21.15.143:0/2173905727 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f130c13e290 0x7f130c15dd70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:22:04.177 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:04.172+0000 7f1311f01640 1 --2- 172.21.15.143:0/2173905727 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f130c0cf7c0 0x7f130c0b1630 unknown :-1 s=CLOSED pgs=820 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:22:04.177 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:04.172+0000 7f1311f01640 1 -- 172.21.15.143:0/2173905727 >> 172.21.15.143:0/2173905727 conn(0x7f130c06a890 msgr2=0x7f130c0aeb60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:22:04.177 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:04.176+0000 7f1311f01640 1 -- 172.21.15.143:0/2173905727 shutdown_connections 2023-11-26T07:22:04.177 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:04.176+0000 7f1311f01640 1 -- 172.21.15.143:0/2173905727 wait complete. 2023-11-26T07:22:04.195 INFO:tasks.cephfs.mount:Mounting default Ceph FS; just confirmed its presence on cluster 2023-11-26T07:22:04.195 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2023-11-26T07:22:04.196 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi188.front.sepia.ceph.com 2023-11-26T07:22:04.196 INFO:tasks.cephfs.mount:self.client.name = client.1 2023-11-26T07:22:04.196 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.1 2023-11-26T07:22:04.196 INFO:tasks.cephfs.mount:self.cephfs_name = None 2023-11-26T07:22:04.196 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2023-11-26T07:22:04.196 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2023-11-26T07:22:04.196 INFO:tasks.cephfs.mount:Setting the 'ceph-ns--home-ubuntu-cephtest-mnt.1' netns for '/home/ubuntu/cephtest/mnt.1' 2023-11-26T07:22:04.196 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-26T07:22:04.196 DEBUG:teuthology.orchestra.run.smithi188:> ip addr 2023-11-26T07:22:04.206 INFO:teuthology.orchestra.run.smithi188.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2023-11-26T07:22:04.207 INFO:teuthology.orchestra.run.smithi188.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2023-11-26T07:22:04.207 INFO:teuthology.orchestra.run.smithi188.stdout: inet 127.0.0.1/8 scope host lo 2023-11-26T07:22:04.207 INFO:teuthology.orchestra.run.smithi188.stdout: valid_lft forever preferred_lft forever 2023-11-26T07:22:04.207 INFO:teuthology.orchestra.run.smithi188.stdout: inet6 ::1/128 scope host 2023-11-26T07:22:04.207 INFO:teuthology.orchestra.run.smithi188.stdout: valid_lft forever preferred_lft forever 2023-11-26T07:22:04.207 INFO:teuthology.orchestra.run.smithi188.stdout:2: eno1: mtu 1500 qdisc noop state DOWN group default qlen 1000 2023-11-26T07:22:04.207 INFO:teuthology.orchestra.run.smithi188.stdout: link/ether 0c:c4:7a:d9:37:48 brd ff:ff:ff:ff:ff:ff 2023-11-26T07:22:04.207 INFO:teuthology.orchestra.run.smithi188.stdout: altname enp7s0f0 2023-11-26T07:22:04.207 INFO:teuthology.orchestra.run.smithi188.stdout:3: eno2: mtu 1500 qdisc noop state DOWN group default qlen 1000 2023-11-26T07:22:04.207 INFO:teuthology.orchestra.run.smithi188.stdout: link/ether 0c:c4:7a:d9:37:49 brd ff:ff:ff:ff:ff:ff 2023-11-26T07:22:04.207 INFO:teuthology.orchestra.run.smithi188.stdout: altname enp7s0f1 2023-11-26T07:22:04.208 INFO:teuthology.orchestra.run.smithi188.stdout:4: enp3s0f0: mtu 1500 qdisc noop state DOWN group default qlen 1000 2023-11-26T07:22:04.208 INFO:teuthology.orchestra.run.smithi188.stdout: link/ether 0c:c4:7a:88:80:38 brd ff:ff:ff:ff:ff:ff 2023-11-26T07:22:04.208 INFO:teuthology.orchestra.run.smithi188.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2023-11-26T07:22:04.208 INFO:teuthology.orchestra.run.smithi188.stdout: link/ether 0c:c4:7a:88:80:39 brd ff:ff:ff:ff:ff:ff 2023-11-26T07:22:04.208 INFO:teuthology.orchestra.run.smithi188.stdout: inet 172.21.15.188/20 metric 100 brd 172.21.15.255 scope global dynamic enp3s0f1 2023-11-26T07:22:04.208 INFO:teuthology.orchestra.run.smithi188.stdout: valid_lft 42144sec preferred_lft 42144sec 2023-11-26T07:22:04.208 INFO:teuthology.orchestra.run.smithi188.stdout: inet6 fe80::ec4:7aff:fe88:8039/64 scope link 2023-11-26T07:22:04.208 INFO:teuthology.orchestra.run.smithi188.stdout: valid_lft forever preferred_lft forever 2023-11-26T07:22:04.208 INFO:teuthology.orchestra.run.smithi188.stdout:6: docker0: mtu 1500 qdisc noqueue state DOWN group default 2023-11-26T07:22:04.208 INFO:teuthology.orchestra.run.smithi188.stdout: link/ether 02:42:d8:0d:a0:fa brd ff:ff:ff:ff:ff:ff 2023-11-26T07:22:04.208 INFO:teuthology.orchestra.run.smithi188.stdout: inet 172.17.0.1/16 brd 172.17.255.255 scope global docker0 2023-11-26T07:22:04.208 INFO:teuthology.orchestra.run.smithi188.stdout: valid_lft forever preferred_lft forever 2023-11-26T07:22:04.209 INFO:teuthology.orchestra.run.smithi188.stdout:7: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2023-11-26T07:22:04.209 INFO:teuthology.orchestra.run.smithi188.stdout: link/ether 5e:1f:84:6d:79:ae brd ff:ff:ff:ff:ff:ff 2023-11-26T07:22:04.209 INFO:teuthology.orchestra.run.smithi188.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2023-11-26T07:22:04.209 INFO:teuthology.orchestra.run.smithi188.stdout: valid_lft forever preferred_lft forever 2023-11-26T07:22:04.209 INFO:teuthology.orchestra.run.smithi188.stdout: inet6 fe80::5c1f:84ff:fe6d:79ae/64 scope link 2023-11-26T07:22:04.209 INFO:teuthology.orchestra.run.smithi188.stdout: valid_lft forever preferred_lft forever 2023-11-26T07:22:04.209 INFO:teuthology.orchestra.run.smithi188.stdout:8: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2023-11-26T07:22:04.209 INFO:teuthology.orchestra.run.smithi188.stdout: link/ether 0e:5c:f6:1f:82:2f brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.1 2023-11-26T07:22:04.209 INFO:teuthology.orchestra.run.smithi188.stdout: inet6 fe80::c5c:f6ff:fe1f:822f/64 scope link 2023-11-26T07:22:04.209 INFO:teuthology.orchestra.run.smithi188.stdout: valid_lft forever preferred_lft forever 2023-11-26T07:22:04.209 INFO:teuthology.orchestra.run.smithi188.stdout:9: brx.1@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2023-11-26T07:22:04.210 INFO:teuthology.orchestra.run.smithi188.stdout: link/ether c6:a6:3d:69:d0:db brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.3 2023-11-26T07:22:04.210 INFO:teuthology.orchestra.run.smithi188.stdout: inet6 fe80::c4a6:3dff:fe69:d0db/64 scope link 2023-11-26T07:22:04.210 INFO:teuthology.orchestra.run.smithi188.stdout: valid_lft forever preferred_lft forever 2023-11-26T07:22:04.211 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-26T07:22:04.211 DEBUG:teuthology.orchestra.run.smithi188:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2023-11-26T07:22:04.276 INFO:teuthology.orchestra.run.smithi188.stdout:1 2023-11-26T07:22:04.279 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-26T07:22:04.279 DEBUG:teuthology.orchestra.run.smithi188:> route 2023-11-26T07:22:04.331 INFO:teuthology.orchestra.run.smithi188.stdout:Kernel IP routing table 2023-11-26T07:22:04.332 INFO:teuthology.orchestra.run.smithi188.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2023-11-26T07:22:04.332 INFO:teuthology.orchestra.run.smithi188.stdout:default _gateway 0.0.0.0 UG 0 0 0 enp3s0f1 2023-11-26T07:22:04.332 INFO:teuthology.orchestra.run.smithi188.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2023-11-26T07:22:04.332 INFO:teuthology.orchestra.run.smithi188.stdout:172.17.0.0 0.0.0.0 255.255.0.0 U 0 0 0 docker0 2023-11-26T07:22:04.332 INFO:teuthology.orchestra.run.smithi188.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 enp3s0f1 2023-11-26T07:22:04.332 INFO:teuthology.orchestra.run.smithi188.stdout:ns1.front.sepia 0.0.0.0 255.255.255.255 UH 100 0 0 enp3s0f1 2023-11-26T07:22:04.332 INFO:teuthology.orchestra.run.smithi188.stdout:172.21.0.2 0.0.0.0 255.255.255.255 UH 100 0 0 enp3s0f1 2023-11-26T07:22:04.332 INFO:teuthology.orchestra.run.smithi188.stdout:_gateway 0.0.0.0 255.255.255.255 UH 100 0 0 enp3s0f1 2023-11-26T07:22:04.332 INFO:teuthology.orchestra.run.smithi188.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2023-11-26T07:22:04.332 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-26T07:22:04.333 DEBUG:teuthology.orchestra.run.smithi188:> (cd / && exec bash -c ' 2023-11-26T07:22:04.333 DEBUG:teuthology.orchestra.run.smithi188:> set -e 2023-11-26T07:22:04.333 DEBUG:teuthology.orchestra.run.smithi188:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2023-11-26T07:22:04.333 DEBUG:teuthology.orchestra.run.smithi188:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2023-11-26T07:22:04.333 DEBUG:teuthology.orchestra.run.smithi188:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2023-11-26T07:22:04.333 DEBUG:teuthology.orchestra.run.smithi188:> ') 2023-11-26T07:22:04.426 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-26T07:22:04.426 DEBUG:teuthology.orchestra.run.smithi188:> ip netns list 2023-11-26T07:22:04.435 INFO:teuthology.orchestra.run.smithi188.stdout:ceph-ns--home-ubuntu-cephtest-mnt.1 (id: 0) 2023-11-26T07:22:04.435 INFO:teuthology.orchestra.run.smithi188.stdout:ceph-ns--home-ubuntu-cephtest-mnt.3 (id: 1) 2023-11-26T07:22:04.436 INFO:tasks.cephfs.mount:Resuming the 'ceph-ns--home-ubuntu-cephtest-mnt.1' netns for '/home/ubuntu/cephtest/mnt.1' 2023-11-26T07:22:04.436 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-26T07:22:04.436 DEBUG:teuthology.orchestra.run.smithi188:> sudo ip link set brx.0 up 2023-11-26T07:22:04.496 INFO:tasks.cephfs.fuse_mount:Client client.1 config is {} 2023-11-26T07:22:04.497 INFO:teuthology.orchestra.run:Running command with timeout 60 2023-11-26T07:22:04.497 DEBUG:teuthology.orchestra.run.smithi188:> mkdir -p -v /home/ubuntu/cephtest/mnt.1 2023-11-26T07:22:04.552 INFO:teuthology.orchestra.run.smithi188.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.1' 2023-11-26T07:22:04.552 INFO:teuthology.orchestra.run:Running command with timeout 60 2023-11-26T07:22:04.553 DEBUG:teuthology.orchestra.run.smithi188:> chmod 0000 /home/ubuntu/cephtest/mnt.1 2023-11-26T07:22:04.603 DEBUG:teuthology.orchestra.run.smithi188:> sudo modprobe fuse 2023-11-26T07:22:04.665 DEBUG:teuthology.orchestra.run.smithi188:> cat /proc/self/mounts | awk '{print $2}' 2023-11-26T07:22:04.718 INFO:teuthology.orchestra.run.smithi188.stdout:/sys 2023-11-26T07:22:04.718 INFO:teuthology.orchestra.run.smithi188.stdout:/proc 2023-11-26T07:22:04.718 INFO:teuthology.orchestra.run.smithi188.stdout:/dev 2023-11-26T07:22:04.718 INFO:teuthology.orchestra.run.smithi188.stdout:/dev/pts 2023-11-26T07:22:04.718 INFO:teuthology.orchestra.run.smithi188.stdout:/run 2023-11-26T07:22:04.718 INFO:teuthology.orchestra.run.smithi188.stdout:/ 2023-11-26T07:22:04.719 INFO:teuthology.orchestra.run.smithi188.stdout:/sys/kernel/security 2023-11-26T07:22:04.719 INFO:teuthology.orchestra.run.smithi188.stdout:/dev/shm 2023-11-26T07:22:04.719 INFO:teuthology.orchestra.run.smithi188.stdout:/run/lock 2023-11-26T07:22:04.719 INFO:teuthology.orchestra.run.smithi188.stdout:/sys/fs/cgroup 2023-11-26T07:22:04.719 INFO:teuthology.orchestra.run.smithi188.stdout:/sys/fs/pstore 2023-11-26T07:22:04.719 INFO:teuthology.orchestra.run.smithi188.stdout:/sys/fs/bpf 2023-11-26T07:22:04.719 INFO:teuthology.orchestra.run.smithi188.stdout:/proc/sys/fs/binfmt_misc 2023-11-26T07:22:04.719 INFO:teuthology.orchestra.run.smithi188.stdout:/dev/hugepages 2023-11-26T07:22:04.719 INFO:teuthology.orchestra.run.smithi188.stdout:/dev/mqueue 2023-11-26T07:22:04.719 INFO:teuthology.orchestra.run.smithi188.stdout:/sys/kernel/debug 2023-11-26T07:22:04.719 INFO:teuthology.orchestra.run.smithi188.stdout:/sys/kernel/tracing 2023-11-26T07:22:04.720 INFO:teuthology.orchestra.run.smithi188.stdout:/proc/fs/nfsd 2023-11-26T07:22:04.720 INFO:teuthology.orchestra.run.smithi188.stdout:/sys/fs/fuse/connections 2023-11-26T07:22:04.720 INFO:teuthology.orchestra.run.smithi188.stdout:/sys/kernel/config 2023-11-26T07:22:04.720 INFO:teuthology.orchestra.run.smithi188.stdout:/run/credentials/systemd-sysusers.service 2023-11-26T07:22:04.720 INFO:teuthology.orchestra.run.smithi188.stdout:/run/rpc_pipefs 2023-11-26T07:22:04.720 INFO:teuthology.orchestra.run.smithi188.stdout:/run/user/1000 2023-11-26T07:22:04.720 INFO:teuthology.orchestra.run.smithi188.stdout:/var/lib/ceph 2023-11-26T07:22:04.720 INFO:teuthology.orchestra.run.smithi188.stdout:/var/lib/ceph/osd/ceph-4 2023-11-26T07:22:04.720 INFO:teuthology.orchestra.run.smithi188.stdout:/var/lib/ceph/osd/ceph-5 2023-11-26T07:22:04.720 INFO:teuthology.orchestra.run.smithi188.stdout:/var/lib/ceph/osd/ceph-6 2023-11-26T07:22:04.720 INFO:teuthology.orchestra.run.smithi188.stdout:/var/lib/ceph/osd/ceph-7 2023-11-26T07:22:04.720 INFO:teuthology.orchestra.run.smithi188.stdout:/run/netns 2023-11-26T07:22:04.720 INFO:teuthology.orchestra.run.smithi188.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 2023-11-26T07:22:04.721 INFO:teuthology.orchestra.run.smithi188.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 2023-11-26T07:22:04.721 INFO:teuthology.orchestra.run.smithi188.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 2023-11-26T07:22:04.721 INFO:teuthology.orchestra.run.smithi188.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 2023-11-26T07:22:04.722 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-26T07:22:04.722 DEBUG:teuthology.orchestra.run.smithi188:> ls /sys/fs/fuse/connections 2023-11-26T07:22:04.772 INFO:tasks.cephfs.fuse_mount:Pre-mount connections: [] 2023-11-26T07:22:04.772 DEBUG:teuthology.orchestra.run.smithi188:> (cd /home/ubuntu/cephtest && exec sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-fuse -f --admin-socket '/var/run/ceph/$cluster-$name.$pid.asok' /home/ubuntu/cephtest/mnt.1 --id 1) 2023-11-26T07:22:04.815 DEBUG:teuthology.orchestra.run.smithi188:> sudo modprobe fuse 2023-11-26T07:22:04.836 DEBUG:teuthology.orchestra.run.smithi188:> cat /proc/self/mounts | awk '{print $2}' 2023-11-26T07:22:04.891 INFO:teuthology.orchestra.run.smithi188.stdout:/sys 2023-11-26T07:22:04.891 INFO:teuthology.orchestra.run.smithi188.stdout:/proc 2023-11-26T07:22:04.891 INFO:teuthology.orchestra.run.smithi188.stdout:/dev 2023-11-26T07:22:04.892 INFO:teuthology.orchestra.run.smithi188.stdout:/dev/pts 2023-11-26T07:22:04.892 INFO:teuthology.orchestra.run.smithi188.stdout:/run 2023-11-26T07:22:04.892 INFO:teuthology.orchestra.run.smithi188.stdout:/ 2023-11-26T07:22:04.892 INFO:teuthology.orchestra.run.smithi188.stdout:/sys/kernel/security 2023-11-26T07:22:04.892 INFO:teuthology.orchestra.run.smithi188.stdout:/dev/shm 2023-11-26T07:22:04.892 INFO:teuthology.orchestra.run.smithi188.stdout:/run/lock 2023-11-26T07:22:04.892 INFO:teuthology.orchestra.run.smithi188.stdout:/sys/fs/cgroup 2023-11-26T07:22:04.892 INFO:teuthology.orchestra.run.smithi188.stdout:/sys/fs/pstore 2023-11-26T07:22:04.892 INFO:teuthology.orchestra.run.smithi188.stdout:/sys/fs/bpf 2023-11-26T07:22:04.892 INFO:teuthology.orchestra.run.smithi188.stdout:/proc/sys/fs/binfmt_misc 2023-11-26T07:22:04.892 INFO:teuthology.orchestra.run.smithi188.stdout:/dev/hugepages 2023-11-26T07:22:04.892 INFO:teuthology.orchestra.run.smithi188.stdout:/dev/mqueue 2023-11-26T07:22:04.893 INFO:teuthology.orchestra.run.smithi188.stdout:/sys/kernel/debug 2023-11-26T07:22:04.893 INFO:teuthology.orchestra.run.smithi188.stdout:/sys/kernel/tracing 2023-11-26T07:22:04.893 INFO:teuthology.orchestra.run.smithi188.stdout:/proc/fs/nfsd 2023-11-26T07:22:04.893 INFO:teuthology.orchestra.run.smithi188.stdout:/sys/fs/fuse/connections 2023-11-26T07:22:04.893 INFO:teuthology.orchestra.run.smithi188.stdout:/sys/kernel/config 2023-11-26T07:22:04.893 INFO:teuthology.orchestra.run.smithi188.stdout:/run/credentials/systemd-sysusers.service 2023-11-26T07:22:04.893 INFO:teuthology.orchestra.run.smithi188.stdout:/run/rpc_pipefs 2023-11-26T07:22:04.893 INFO:teuthology.orchestra.run.smithi188.stdout:/run/user/1000 2023-11-26T07:22:04.893 INFO:teuthology.orchestra.run.smithi188.stdout:/var/lib/ceph 2023-11-26T07:22:04.893 INFO:teuthology.orchestra.run.smithi188.stdout:/var/lib/ceph/osd/ceph-4 2023-11-26T07:22:04.893 INFO:teuthology.orchestra.run.smithi188.stdout:/var/lib/ceph/osd/ceph-5 2023-11-26T07:22:04.893 INFO:teuthology.orchestra.run.smithi188.stdout:/var/lib/ceph/osd/ceph-6 2023-11-26T07:22:04.894 INFO:teuthology.orchestra.run.smithi188.stdout:/var/lib/ceph/osd/ceph-7 2023-11-26T07:22:04.894 INFO:teuthology.orchestra.run.smithi188.stdout:/run/netns 2023-11-26T07:22:04.894 INFO:teuthology.orchestra.run.smithi188.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 2023-11-26T07:22:04.894 INFO:teuthology.orchestra.run.smithi188.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 2023-11-26T07:22:04.894 INFO:teuthology.orchestra.run.smithi188.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 2023-11-26T07:22:04.894 INFO:teuthology.orchestra.run.smithi188.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 2023-11-26T07:22:04.895 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-26T07:22:04.895 DEBUG:teuthology.orchestra.run.smithi188:> ls /sys/fs/fuse/connections 2023-11-26T07:22:04.912 INFO:tasks.cephfs.fuse_mount.ceph-fuse.1.smithi188.stderr:2023-11-26T07:22:04.906+0000 7f7297a0d580 -1 init, newargv = 0x562d1b4970e0 newargc=15 2023-11-26T07:22:04.913 INFO:tasks.cephfs.fuse_mount.ceph-fuse.1.smithi188.stderr:ceph-fuse[44712]: starting ceph client 2023-11-26T07:22:04.920 INFO:tasks.cephfs.fuse_mount.ceph-fuse.1.smithi188.stderr:ceph-fuse[44712]: starting fuse 2023-11-26T07:22:04.923 INFO:teuthology.orchestra.run.smithi188.stdout:43 2023-11-26T07:22:04.923 INFO:tasks.cephfs.fuse_mount:Post-mount connections: [43] 2023-11-26T07:22:04.923 DEBUG:teuthology.orchestra.run.smithi188:> sudo stdin-killer -- python3 -c ' 2023-11-26T07:22:04.923 DEBUG:teuthology.orchestra.run.smithi188:> import glob 2023-11-26T07:22:04.923 DEBUG:teuthology.orchestra.run.smithi188:> import re 2023-11-26T07:22:04.923 DEBUG:teuthology.orchestra.run.smithi188:> import os 2023-11-26T07:22:04.923 DEBUG:teuthology.orchestra.run.smithi188:> import subprocess 2023-11-26T07:22:04.923 DEBUG:teuthology.orchestra.run.smithi188:> 2023-11-26T07:22:04.924 DEBUG:teuthology.orchestra.run.smithi188:> def _find_admin_socket(client_name): 2023-11-26T07:22:04.924 DEBUG:teuthology.orchestra.run.smithi188:> asok_path = "/var/run/ceph/ceph-client.1.*.asok" 2023-11-26T07:22:04.924 DEBUG:teuthology.orchestra.run.smithi188:> files = glob.glob(asok_path) 2023-11-26T07:22:04.924 DEBUG:teuthology.orchestra.run.smithi188:> mountpoint = "/home/ubuntu/cephtest/mnt.1" 2023-11-26T07:22:04.924 DEBUG:teuthology.orchestra.run.smithi188:> 2023-11-26T07:22:04.924 DEBUG:teuthology.orchestra.run.smithi188:> # Given a non-glob path, it better be there 2023-11-26T07:22:04.924 DEBUG:teuthology.orchestra.run.smithi188:> if "*" not in asok_path: 2023-11-26T07:22:04.924 DEBUG:teuthology.orchestra.run.smithi188:> assert(len(files) == 1) 2023-11-26T07:22:04.924 DEBUG:teuthology.orchestra.run.smithi188:> return files[0] 2023-11-26T07:22:04.924 DEBUG:teuthology.orchestra.run.smithi188:> 2023-11-26T07:22:04.924 DEBUG:teuthology.orchestra.run.smithi188:> for f in files: 2023-11-26T07:22:04.924 DEBUG:teuthology.orchestra.run.smithi188:> pid = re.match(".*\.(\d+)\.asok$", f).group(1) 2023-11-26T07:22:04.924 DEBUG:teuthology.orchestra.run.smithi188:> if os.path.exists("/proc/{0}".format(pid)): 2023-11-26T07:22:04.924 DEBUG:teuthology.orchestra.run.smithi188:> with open("/proc/{0}/cmdline".format(pid), '"'"'r'"'"') as proc_f: 2023-11-26T07:22:04.925 DEBUG:teuthology.orchestra.run.smithi188:> contents = proc_f.read() 2023-11-26T07:22:04.925 DEBUG:teuthology.orchestra.run.smithi188:> if mountpoint in contents: 2023-11-26T07:22:04.925 DEBUG:teuthology.orchestra.run.smithi188:> return f 2023-11-26T07:22:04.925 DEBUG:teuthology.orchestra.run.smithi188:> raise RuntimeError("Client socket {0} not found".format(client_name)) 2023-11-26T07:22:04.925 DEBUG:teuthology.orchestra.run.smithi188:> 2023-11-26T07:22:04.925 DEBUG:teuthology.orchestra.run.smithi188:> print(_find_admin_socket("client.1")) 2023-11-26T07:22:04.925 DEBUG:teuthology.orchestra.run.smithi188:> ' 2023-11-26T07:22:05.049 INFO:teuthology.orchestra.run.smithi188.stdout:/var/run/ceph/ceph-client.1.44712.asok 2023-11-26T07:22:05.052 INFO:teuthology.orchestra.run.smithi188.stderr:2023-11-26T07:22:05 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2023-11-26T07:22:05.060 INFO:tasks.cephfs.fuse_mount:Found client admin socket at /var/run/ceph/ceph-client.1.44712.asok 2023-11-26T07:22:05.060 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-26T07:22:05.061 DEBUG:teuthology.orchestra.run.smithi188:> sudo ceph --admin-daemon /var/run/ceph/ceph-client.1.44712.asok status 2023-11-26T07:22:05.235 INFO:teuthology.orchestra.run.smithi188.stdout:{ 2023-11-26T07:22:05.236 INFO:teuthology.orchestra.run.smithi188.stdout: "metadata": { 2023-11-26T07:22:05.236 INFO:teuthology.orchestra.run.smithi188.stdout: "ceph_sha1": "44c2237fe77713214148c1fd622c872f346f1b99", 2023-11-26T07:22:05.236 INFO:teuthology.orchestra.run.smithi188.stdout: "ceph_version": "ceph version 18.0.0-7484-g44c2237f (44c2237fe77713214148c1fd622c872f346f1b99) reef (dev)", 2023-11-26T07:22:05.236 INFO:teuthology.orchestra.run.smithi188.stdout: "entity_id": "1", 2023-11-26T07:22:05.236 INFO:teuthology.orchestra.run.smithi188.stdout: "hostname": "smithi188", 2023-11-26T07:22:05.236 INFO:teuthology.orchestra.run.smithi188.stdout: "mount_point": "/home/ubuntu/cephtest/mnt.1", 2023-11-26T07:22:05.236 INFO:teuthology.orchestra.run.smithi188.stdout: "pid": "44712", 2023-11-26T07:22:05.236 INFO:teuthology.orchestra.run.smithi188.stdout: "root": "/" 2023-11-26T07:22:05.236 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2023-11-26T07:22:05.236 INFO:teuthology.orchestra.run.smithi188.stdout: "dentry_count": 0, 2023-11-26T07:22:05.236 INFO:teuthology.orchestra.run.smithi188.stdout: "dentry_pinned_count": 0, 2023-11-26T07:22:05.236 INFO:teuthology.orchestra.run.smithi188.stdout: "id": 6256, 2023-11-26T07:22:05.237 INFO:teuthology.orchestra.run.smithi188.stdout: "inst": { 2023-11-26T07:22:05.237 INFO:teuthology.orchestra.run.smithi188.stdout: "name": { 2023-11-26T07:22:05.237 INFO:teuthology.orchestra.run.smithi188.stdout: "type": "client", 2023-11-26T07:22:05.237 INFO:teuthology.orchestra.run.smithi188.stdout: "num": 6256 2023-11-26T07:22:05.237 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2023-11-26T07:22:05.237 INFO:teuthology.orchestra.run.smithi188.stdout: "addr": { 2023-11-26T07:22:05.237 INFO:teuthology.orchestra.run.smithi188.stdout: "type": "v1", 2023-11-26T07:22:05.237 INFO:teuthology.orchestra.run.smithi188.stdout: "addr": "192.168.0.1:0", 2023-11-26T07:22:05.237 INFO:teuthology.orchestra.run.smithi188.stdout: "nonce": 1364884385 2023-11-26T07:22:05.237 INFO:teuthology.orchestra.run.smithi188.stdout: } 2023-11-26T07:22:05.237 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2023-11-26T07:22:05.237 INFO:teuthology.orchestra.run.smithi188.stdout: "addr": { 2023-11-26T07:22:05.237 INFO:teuthology.orchestra.run.smithi188.stdout: "type": "v1", 2023-11-26T07:22:05.238 INFO:teuthology.orchestra.run.smithi188.stdout: "addr": "192.168.0.1:0", 2023-11-26T07:22:05.238 INFO:teuthology.orchestra.run.smithi188.stdout: "nonce": 1364884385 2023-11-26T07:22:05.238 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2023-11-26T07:22:05.238 INFO:teuthology.orchestra.run.smithi188.stdout: "inst_str": "client.6256 192.168.0.1:0/1364884385", 2023-11-26T07:22:05.238 INFO:teuthology.orchestra.run.smithi188.stdout: "addr_str": "192.168.0.1:0/1364884385", 2023-11-26T07:22:05.238 INFO:teuthology.orchestra.run.smithi188.stdout: "inode_count": 1, 2023-11-26T07:22:05.238 INFO:teuthology.orchestra.run.smithi188.stdout: "mds_epoch": 34, 2023-11-26T07:22:05.238 INFO:teuthology.orchestra.run.smithi188.stdout: "osd_epoch": 59, 2023-11-26T07:22:05.238 INFO:teuthology.orchestra.run.smithi188.stdout: "osd_epoch_barrier": 0, 2023-11-26T07:22:05.238 INFO:teuthology.orchestra.run.smithi188.stdout: "blocklisted": false, 2023-11-26T07:22:05.238 INFO:teuthology.orchestra.run.smithi188.stdout: "fs_name": "cephfs" 2023-11-26T07:22:05.238 INFO:teuthology.orchestra.run.smithi188.stdout:} 2023-11-26T07:22:05.250 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-26T07:22:05.251 DEBUG:teuthology.orchestra.run.smithi188:> stat --file-system '--printf=%T 2023-11-26T07:22:05.251 DEBUG:teuthology.orchestra.run.smithi188:> ' -- /home/ubuntu/cephtest/mnt.1 2023-11-26T07:22:05.260 INFO:teuthology.orchestra.run.smithi188.stdout:fuseblk 2023-11-26T07:22:05.260 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.1 2023-11-26T07:22:05.260 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-26T07:22:05.260 DEBUG:teuthology.orchestra.run.smithi188:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.1 2023-11-26T07:22:05.324 DEBUG:teuthology.orchestra.run.smithi143:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs flag set enable_multiple true --yes-i-really-mean-it 2023-11-26T07:22:05.474 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:05.472+0000 7f82a8817640 1 Processor -- start 2023-11-26T07:22:05.474 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:05.472+0000 7f82a8817640 1 -- start start 2023-11-26T07:22:05.475 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:05.472+0000 7f82a8817640 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7f82a4143ac0 0x7f82a4070370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:22:05.476 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:05.472+0000 7f82a8817640 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7f82a40708b0 0x7f82a40cf280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:22:05.476 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:05.472+0000 7f82a8817640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f82a40cf7c0 0x7f82a40b1ef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:22:05.476 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:05.472+0000 7f82a8817640 1 -- --> v2:172.21.15.143:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f82a4143ea0 con 0x7f82a4143ac0 2023-11-26T07:22:05.476 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:05.472+0000 7f82a8817640 1 -- --> v2:172.21.15.188:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f82a40cc730 con 0x7f82a40708b0 2023-11-26T07:22:05.476 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:05.472+0000 7f82a8817640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f82a40cfbd0 con 0x7f82a40cf7c0 2023-11-26T07:22:05.476 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:05.472+0000 7f82a259c640 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7f82a4143ac0 0x7f82a4070370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:22:05.476 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:05.472+0000 7f82a259c640 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7f82a4143ac0 0x7f82a4070370 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:35224/0 (socket says 172.21.15.143:35224) 2023-11-26T07:22:05.476 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:05.472+0000 7f82a259c640 1 -- 172.21.15.143:0/1358062188 learned_addr learned my addr 172.21.15.143:0/1358062188 (peer_addr_for_me v2:172.21.15.143:0/0) 2023-11-26T07:22:05.476 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:05.472+0000 7f8299d9b640 1 --2- 172.21.15.143:0/1358062188 >> v2:172.21.15.188:3300/0 conn(0x7f82a40708b0 0x7f82a40cf280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:22:05.476 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:05.472+0000 7f82a2d9d640 1 --2- 172.21.15.143:0/1358062188 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f82a40cf7c0 0x7f82a40b1ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:22:05.476 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:05.472+0000 7f82a259c640 1 -- 172.21.15.143:0/1358062188 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f82a40cf7c0 msgr2=0x7f82a40b1ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-26T07:22:05.477 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:05.472+0000 7f82a259c640 1 --2- 172.21.15.143:0/1358062188 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f82a40cf7c0 0x7f82a40b1ef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:22:05.477 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:05.472+0000 7f82a259c640 1 -- 172.21.15.143:0/1358062188 >> v2:172.21.15.188:3300/0 conn(0x7f82a40708b0 msgr2=0x7f82a40cf280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-26T07:22:05.477 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:05.472+0000 7f82a259c640 1 --2- 172.21.15.143:0/1358062188 >> v2:172.21.15.188:3300/0 conn(0x7f82a40708b0 0x7f82a40cf280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:22:05.477 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:05.472+0000 7f82a259c640 1 -- 172.21.15.143:0/1358062188 --> v2:172.21.15.143:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f82a4070ee0 con 0x7f82a4143ac0 2023-11-26T07:22:05.478 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:05.476+0000 7f82a259c640 1 --2- 172.21.15.143:0/1358062188 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f82a4143ac0 0x7f82a4070370 secure :-1 s=READY pgs=823 cs=0 l=1 rev1=1 crypto rx=0x7f82a40aeb10 tx=0x7f828c01c240 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8f3e1e36b16cfdba server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:22:05.479 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:05.476+0000 7f82a1d9b640 1 -- 172.21.15.143:0/1358062188 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f828c0ab600 con 0x7f82a4143ac0 2023-11-26T07:22:05.479 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:05.476+0000 7f82a1d9b640 1 -- 172.21.15.143:0/1358062188 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f828c0a7050 con 0x7f82a4143ac0 2023-11-26T07:22:05.479 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:05.476+0000 7f82a1d9b640 1 -- 172.21.15.143:0/1358062188 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f828c0ab980 con 0x7f82a4143ac0 2023-11-26T07:22:05.479 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:05.476+0000 7f82a8817640 1 -- 172.21.15.143:0/1358062188 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f82a4143ac0 msgr2=0x7f82a4070370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:22:05.479 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:05.476+0000 7f82a8817640 1 --2- 172.21.15.143:0/1358062188 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f82a4143ac0 0x7f82a4070370 secure :-1 s=READY pgs=823 cs=0 l=1 rev1=1 crypto rx=0x7f82a40aeb10 tx=0x7f828c01c240 comp rx=0 tx=0).stop 2023-11-26T07:22:05.480 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:05.476+0000 7f82a8817640 1 -- 172.21.15.143:0/1358062188 shutdown_connections 2023-11-26T07:22:05.480 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:05.476+0000 7f82a8817640 1 --2- 172.21.15.143:0/1358062188 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f82a40cf7c0 0x7f82a40b1ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:22:05.480 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:05.476+0000 7f82a8817640 1 --2- 172.21.15.143:0/1358062188 >> v2:172.21.15.188:3300/0 conn(0x7f82a40708b0 0x7f82a40cf280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:22:05.480 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:05.476+0000 7f82a8817640 1 --2- 172.21.15.143:0/1358062188 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f82a4143ac0 0x7f82a4070370 unknown :-1 s=CLOSED pgs=823 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:22:05.480 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:05.476+0000 7f82a8817640 1 -- 172.21.15.143:0/1358062188 >> 172.21.15.143:0/1358062188 conn(0x7f82a406a890 msgr2=0x7f82a406aca0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:22:05.480 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:05.476+0000 7f82a8817640 1 -- 172.21.15.143:0/1358062188 shutdown_connections 2023-11-26T07:22:05.481 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:05.476+0000 7f82a8817640 1 -- 172.21.15.143:0/1358062188 wait complete. 2023-11-26T07:22:05.481 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:05.480+0000 7f82a8817640 1 Processor -- start 2023-11-26T07:22:05.481 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:05.480+0000 7f82a8817640 1 -- start start 2023-11-26T07:22:05.483 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:05.480+0000 7f82a8817640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f82a40708b0 0x7f82a413c060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:22:05.483 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:05.480+0000 7f82a8817640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f82a40cf7c0 0x7f82a4139c40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:22:05.484 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:05.480+0000 7f82a8817640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f82a4143ac0 0x7f82a415b540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:22:05.484 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:05.480+0000 7f82a8817640 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f82a413c6f0 con 0x7f82a4143ac0 2023-11-26T07:22:05.484 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:05.480+0000 7f82a8817640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f82a415a0d0 con 0x7f82a40708b0 2023-11-26T07:22:05.484 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:05.480+0000 7f82a8817640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f82a415a780 con 0x7f82a40cf7c0 2023-11-26T07:22:05.484 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:05.480+0000 7f82a2d9d640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f82a4143ac0 0x7f82a415b540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:22:05.484 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:05.480+0000 7f82a2d9d640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f82a4143ac0 0x7f82a415b540 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:35240/0 (socket says 172.21.15.143:35240) 2023-11-26T07:22:05.484 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:05.480+0000 7f82a2d9d640 1 -- 172.21.15.143:0/2014516978 learned_addr learned my addr 172.21.15.143:0/2014516978 (peer_addr_for_me v2:172.21.15.143:0/0) 2023-11-26T07:22:05.484 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:05.480+0000 7f8299d9b640 1 --2- 172.21.15.143:0/2014516978 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f82a40cf7c0 0x7f82a4139c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:22:05.484 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:05.480+0000 7f82a259c640 1 --2- 172.21.15.143:0/2014516978 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f82a40708b0 0x7f82a413c060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:22:05.484 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:05.480+0000 7f82a2d9d640 1 -- 172.21.15.143:0/2014516978 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f82a40cf7c0 msgr2=0x7f82a4139c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:22:05.484 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:05.480+0000 7f82a2d9d640 1 --2- 172.21.15.143:0/2014516978 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f82a40cf7c0 0x7f82a4139c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:22:05.485 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:05.480+0000 7f82a2d9d640 1 -- 172.21.15.143:0/2014516978 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f82a40708b0 msgr2=0x7f82a413c060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:22:05.485 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:05.480+0000 7f82a2d9d640 1 --2- 172.21.15.143:0/2014516978 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f82a40708b0 0x7f82a413c060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:22:05.485 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:05.480+0000 7f82a2d9d640 1 -- 172.21.15.143:0/2014516978 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f828c0094d0 con 0x7f82a4143ac0 2023-11-26T07:22:05.485 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:05.480+0000 7f82a2d9d640 1 --2- 172.21.15.143:0/2014516978 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f82a4143ac0 0x7f82a415b540 secure :-1 s=READY pgs=824 cs=0 l=1 rev1=1 crypto rx=0x7f829c007e40 tx=0x7f829c00e4b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:22:05.486 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:05.480+0000 7f829b7fe640 1 -- 172.21.15.143:0/2014516978 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f829c014620 con 0x7f82a4143ac0 2023-11-26T07:22:05.486 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:05.480+0000 7f829b7fe640 1 -- 172.21.15.143:0/2014516978 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f829c00ee90 con 0x7f82a4143ac0 2023-11-26T07:22:05.486 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:05.480+0000 7f82a8817640 1 -- 172.21.15.143:0/2014516978 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f82a415aa60 con 0x7f82a4143ac0 2023-11-26T07:22:05.486 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:05.480+0000 7f829b7fe640 1 -- 172.21.15.143:0/2014516978 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f829c0149e0 con 0x7f82a4143ac0 2023-11-26T07:22:05.486 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:05.484+0000 7f82a8817640 1 -- 172.21.15.143:0/2014516978 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f82a415b020 con 0x7f82a4143ac0 2023-11-26T07:22:05.486 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:05.484+0000 7f82a8817640 1 -- 172.21.15.143:0/2014516978 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f82a415b240 con 0x7f82a4143ac0 2023-11-26T07:22:05.486 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:05.484+0000 7f829b7fe640 1 -- 172.21.15.143:0/2014516978 <== mon.0 v2:172.21.15.143:3300/0 4 ==== mgrmap(e 9) v1 ==== 86005+0+0 (secure 0 0 0) 0x7f829c014b40 con 0x7f82a4143ac0 2023-11-26T07:22:05.487 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:05.484+0000 7f829b7fe640 1 --2- 172.21.15.143:0/2014516978 >> [v2:172.21.15.143:6832/2150557544,v1:172.21.15.143:6833/2150557544] conn(0x7f8284066620 0x7f8284068ae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:22:05.487 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:05.484+0000 7f829b7fe640 1 -- 172.21.15.143:0/2014516978 <== mon.0 v2:172.21.15.143:3300/0 5 ==== osd_map(59..59 src has 1..59) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f829c084b10 con 0x7f82a4143ac0 2023-11-26T07:22:05.487 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:05.484+0000 7f82a259c640 1 --2- 172.21.15.143:0/2014516978 >> [v2:172.21.15.143:6832/2150557544,v1:172.21.15.143:6833/2150557544] conn(0x7f8284066620 0x7f8284068ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:22:05.488 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:05.484+0000 7f82a259c640 1 --2- 172.21.15.143:0/2014516978 >> [v2:172.21.15.143:6832/2150557544,v1:172.21.15.143:6833/2150557544] conn(0x7f8284066620 0x7f8284068ae0 secure :-1 s=READY pgs=391 cs=0 l=1 rev1=1 crypto rx=0x7f828c0ad000 tx=0x7f828c008040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:22:05.491 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:05.488+0000 7f829b7fe640 1 -- 172.21.15.143:0/2014516978 <== mon.0 v2:172.21.15.143:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172649 (secure 0 0 0) 0x7f829c051c90 con 0x7f82a4143ac0 2023-11-26T07:22:05.674 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:05.672+0000 7f82a8817640 1 -- 172.21.15.143:0/2014516978 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "fs flag set", "flag_name": "enable_multiple", "val": "true", "yes_i_really_mean_it": true} v 0) v1 -- 0x7f82a4067680 con 0x7f82a4143ac0 2023-11-26T07:22:06.520 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:06.516+0000 7f829b7fe640 1 -- 172.21.15.143:0/2014516978 <== mon.0 v2:172.21.15.143:3300/0 7 ==== mon_command_ack([{"prefix": "fs flag set", "flag_name": "enable_multiple", "val": "true", "yes_i_really_mean_it": true}]=0 v35) v1 ==== 136+0+0 (secure 0 0 0) 0x7f829c0552b0 con 0x7f82a4143ac0 2023-11-26T07:22:06.523 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:06.520+0000 7f82a8817640 1 -- 172.21.15.143:0/2014516978 >> [v2:172.21.15.143:6832/2150557544,v1:172.21.15.143:6833/2150557544] conn(0x7f8284066620 msgr2=0x7f8284068ae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:22:06.523 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:06.520+0000 7f82a8817640 1 --2- 172.21.15.143:0/2014516978 >> [v2:172.21.15.143:6832/2150557544,v1:172.21.15.143:6833/2150557544] conn(0x7f8284066620 0x7f8284068ae0 secure :-1 s=READY pgs=391 cs=0 l=1 rev1=1 crypto rx=0x7f828c0ad000 tx=0x7f828c008040 comp rx=0 tx=0).stop 2023-11-26T07:22:06.523 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:06.520+0000 7f82a8817640 1 -- 172.21.15.143:0/2014516978 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f82a4143ac0 msgr2=0x7f82a415b540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:22:06.523 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:06.520+0000 7f82a8817640 1 --2- 172.21.15.143:0/2014516978 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f82a4143ac0 0x7f82a415b540 secure :-1 s=READY pgs=824 cs=0 l=1 rev1=1 crypto rx=0x7f829c007e40 tx=0x7f829c00e4b0 comp rx=0 tx=0).stop 2023-11-26T07:22:06.523 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:06.520+0000 7f82a8817640 1 -- 172.21.15.143:0/2014516978 shutdown_connections 2023-11-26T07:22:06.523 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:06.520+0000 7f82a8817640 1 --2- 172.21.15.143:0/2014516978 >> [v2:172.21.15.143:6832/2150557544,v1:172.21.15.143:6833/2150557544] conn(0x7f8284066620 0x7f8284068ae0 unknown :-1 s=CLOSED pgs=391 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:22:06.523 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:06.520+0000 7f82a8817640 1 --2- 172.21.15.143:0/2014516978 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f82a4143ac0 0x7f82a415b540 unknown :-1 s=CLOSED pgs=824 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:22:06.524 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:06.520+0000 7f82a8817640 1 --2- 172.21.15.143:0/2014516978 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f82a40cf7c0 0x7f82a4139c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:22:06.524 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:06.520+0000 7f82a8817640 1 --2- 172.21.15.143:0/2014516978 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f82a40708b0 0x7f82a413c060 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:22:06.524 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:06.520+0000 7f82a8817640 1 -- 172.21.15.143:0/2014516978 >> 172.21.15.143:0/2014516978 conn(0x7f82a406a890 msgr2=0x7f82a40ade90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:22:06.524 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:06.520+0000 7f82a8817640 1 -- 172.21.15.143:0/2014516978 shutdown_connections 2023-11-26T07:22:06.524 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:06.520+0000 7f82a8817640 1 -- 172.21.15.143:0/2014516978 wait complete. 2023-11-26T07:22:06.546 DEBUG:teuthology.orchestra.run.smithi143:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs volume ls 2023-11-26T07:22:06.688 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:06.688+0000 7f77fba5a640 1 Processor -- start 2023-11-26T07:22:06.688 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:06.688+0000 7f77fba5a640 1 -- start start 2023-11-26T07:22:06.690 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:06.688+0000 7f77fba5a640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f77f4143ac0 0x7f77f4070370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:22:06.690 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:06.688+0000 7f77fba5a640 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7f77f40708b0 0x7f77f40cf280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:22:06.691 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:06.688+0000 7f77fba5a640 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7f77f40cf7c0 0x7f77f40b1ef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:22:06.691 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:06.688+0000 7f77fba5a640 1 -- --> v2:172.21.15.143:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f77f4143ea0 con 0x7f77f40cf7c0 2023-11-26T07:22:06.691 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:06.688+0000 7f77fba5a640 1 -- --> v2:172.21.15.188:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f77f40cfbf0 con 0x7f77f40708b0 2023-11-26T07:22:06.691 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:06.688+0000 7f77fba5a640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f77f40cc730 con 0x7f77f4143ac0 2023-11-26T07:22:06.691 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:06.688+0000 7f77f9ff7640 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7f77f40cf7c0 0x7f77f40b1ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:22:06.691 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:06.688+0000 7f77f9ff7640 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7f77f40cf7c0 0x7f77f40b1ef0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:35252/0 (socket says 172.21.15.143:35252) 2023-11-26T07:22:06.691 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:06.688+0000 7f77f9ff7640 1 -- 172.21.15.143:0/2941234894 learned_addr learned my addr 172.21.15.143:0/2941234894 (peer_addr_for_me v2:172.21.15.143:0/0) 2023-11-26T07:22:06.691 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:06.688+0000 7f77f97f6640 1 --2- 172.21.15.143:0/2941234894 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f77f4143ac0 0x7f77f4070370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:22:06.691 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:06.688+0000 7f77f8ff5640 1 --2- 172.21.15.143:0/2941234894 >> v2:172.21.15.188:3300/0 conn(0x7f77f40708b0 0x7f77f40cf280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:22:06.692 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:06.688+0000 7f77f9ff7640 1 -- 172.21.15.143:0/2941234894 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f77f4143ac0 msgr2=0x7f77f4070370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-26T07:22:06.692 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:06.688+0000 7f77f9ff7640 1 --2- 172.21.15.143:0/2941234894 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f77f4143ac0 0x7f77f4070370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:22:06.692 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:06.688+0000 7f77f9ff7640 1 -- 172.21.15.143:0/2941234894 >> v2:172.21.15.188:3300/0 conn(0x7f77f40708b0 msgr2=0x7f77f40cf280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-26T07:22:06.692 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:06.688+0000 7f77f9ff7640 1 --2- 172.21.15.143:0/2941234894 >> v2:172.21.15.188:3300/0 conn(0x7f77f40708b0 0x7f77f40cf280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:22:06.692 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:06.688+0000 7f77f9ff7640 1 -- 172.21.15.143:0/2941234894 --> v2:172.21.15.143:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f77f4070ee0 con 0x7f77f40cf7c0 2023-11-26T07:22:06.693 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:06.692+0000 7f77f9ff7640 1 --2- 172.21.15.143:0/2941234894 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f77f40cf7c0 0x7f77f40b1ef0 secure :-1 s=READY pgs=825 cs=0 l=1 rev1=1 crypto rx=0x7f77f40cfec0 tx=0x7f77ec01c710 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b181c8dd394fea26 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:22:06.694 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:06.692+0000 7f77ebfff640 1 -- 172.21.15.143:0/2941234894 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f77ec0a64b0 con 0x7f77f40cf7c0 2023-11-26T07:22:06.694 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:06.692+0000 7f77ebfff640 1 -- 172.21.15.143:0/2941234894 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f77ec0a6610 con 0x7f77f40cf7c0 2023-11-26T07:22:06.694 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:06.692+0000 7f77ebfff640 1 -- 172.21.15.143:0/2941234894 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f77ec0a69a0 con 0x7f77f40cf7c0 2023-11-26T07:22:06.695 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:06.692+0000 7f77fba5a640 1 -- 172.21.15.143:0/2941234894 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f77f40cf7c0 msgr2=0x7f77f40b1ef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:22:06.695 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:06.692+0000 7f77fba5a640 1 --2- 172.21.15.143:0/2941234894 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f77f40cf7c0 0x7f77f40b1ef0 secure :-1 s=READY pgs=825 cs=0 l=1 rev1=1 crypto rx=0x7f77f40cfec0 tx=0x7f77ec01c710 comp rx=0 tx=0).stop 2023-11-26T07:22:06.695 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:06.692+0000 7f77fba5a640 1 -- 172.21.15.143:0/2941234894 shutdown_connections 2023-11-26T07:22:06.695 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:06.692+0000 7f77fba5a640 1 --2- 172.21.15.143:0/2941234894 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f77f40cf7c0 0x7f77f40b1ef0 unknown :-1 s=CLOSED pgs=825 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:22:06.695 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:06.692+0000 7f77fba5a640 1 --2- 172.21.15.143:0/2941234894 >> v2:172.21.15.188:3300/0 conn(0x7f77f40708b0 0x7f77f40cf280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:22:06.695 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:06.692+0000 7f77fba5a640 1 --2- 172.21.15.143:0/2941234894 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f77f4143ac0 0x7f77f4070370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:22:06.695 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:06.692+0000 7f77fba5a640 1 -- 172.21.15.143:0/2941234894 >> 172.21.15.143:0/2941234894 conn(0x7f77f406a890 msgr2=0x7f77f406aca0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:22:06.695 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:06.692+0000 7f77fba5a640 1 -- 172.21.15.143:0/2941234894 shutdown_connections 2023-11-26T07:22:06.695 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:06.692+0000 7f77fba5a640 1 -- 172.21.15.143:0/2941234894 wait complete. 2023-11-26T07:22:06.696 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:06.696+0000 7f77fba5a640 1 Processor -- start 2023-11-26T07:22:06.696 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:06.696+0000 7f77fba5a640 1 -- start start 2023-11-26T07:22:06.699 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:06.696+0000 7f77fba5a640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f77f40708b0 0x7f77f4136030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:22:06.699 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:06.696+0000 7f77fba5a640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f77f40cf7c0 0x7f77f40adb90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:22:06.699 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:06.696+0000 7f77fba5a640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f77f4143ac0 0x7f77f4140140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:22:06.699 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:06.696+0000 7f77fba5a640 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f77f4159f20 con 0x7f77f4143ac0 2023-11-26T07:22:06.699 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:06.696+0000 7f77fba5a640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f77f4136570 con 0x7f77f40cf7c0 2023-11-26T07:22:06.699 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:06.696+0000 7f77fba5a640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f77f41f5510 con 0x7f77f40708b0 2023-11-26T07:22:06.699 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:06.696+0000 7f77f9ff7640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f77f4143ac0 0x7f77f4140140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:22:06.699 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:06.696+0000 7f77f9ff7640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f77f4143ac0 0x7f77f4140140 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:35268/0 (socket says 172.21.15.143:35268) 2023-11-26T07:22:06.700 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:06.696+0000 7f77f9ff7640 1 -- 172.21.15.143:0/3575303730 learned_addr learned my addr 172.21.15.143:0/3575303730 (peer_addr_for_me v2:172.21.15.143:0/0) 2023-11-26T07:22:06.700 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:06.696+0000 7f77f8ff5640 1 --2- 172.21.15.143:0/3575303730 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f77f40cf7c0 0x7f77f40adb90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:22:06.700 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:06.696+0000 7f77f97f6640 1 --2- 172.21.15.143:0/3575303730 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f77f40708b0 0x7f77f4136030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:22:06.700 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:06.696+0000 7f77f9ff7640 1 -- 172.21.15.143:0/3575303730 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f77f40708b0 msgr2=0x7f77f4136030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:22:06.700 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:06.696+0000 7f77f9ff7640 1 --2- 172.21.15.143:0/3575303730 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f77f40708b0 0x7f77f4136030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:22:06.700 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:06.696+0000 7f77f9ff7640 1 -- 172.21.15.143:0/3575303730 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f77f40cf7c0 msgr2=0x7f77f40adb90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:22:06.700 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:06.696+0000 7f77f9ff7640 1 --2- 172.21.15.143:0/3575303730 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f77f40cf7c0 0x7f77f40adb90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:22:06.700 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:06.696+0000 7f77f9ff7640 1 -- 172.21.15.143:0/3575303730 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f77ec009b30 con 0x7f77f4143ac0 2023-11-26T07:22:06.700 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:06.696+0000 7f77f9ff7640 1 --2- 172.21.15.143:0/3575303730 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f77f4143ac0 0x7f77f4140140 secure :-1 s=READY pgs=826 cs=0 l=1 rev1=1 crypto rx=0x7f77ec01adc0 tx=0x7f77ec0194b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:22:06.700 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:06.696+0000 7f77e9ffb640 1 -- 172.21.15.143:0/3575303730 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f77ec0b4070 con 0x7f77f4143ac0 2023-11-26T07:22:06.700 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:06.696+0000 7f77e9ffb640 1 -- 172.21.15.143:0/3575303730 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f77ec009150 con 0x7f77f4143ac0 2023-11-26T07:22:06.700 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:06.696+0000 7f77fba5a640 1 -- 172.21.15.143:0/3575303730 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f77f41f57f0 con 0x7f77f4143ac0 2023-11-26T07:22:06.701 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:06.696+0000 7f77e9ffb640 1 -- 172.21.15.143:0/3575303730 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f77ec0a69a0 con 0x7f77f4143ac0 2023-11-26T07:22:06.701 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:06.696+0000 7f77fba5a640 1 -- 172.21.15.143:0/3575303730 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f77f41f5db0 con 0x7f77f4143ac0 2023-11-26T07:22:06.701 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:06.696+0000 7f77fba5a640 1 -- 172.21.15.143:0/3575303730 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f77f4067680 con 0x7f77f4143ac0 2023-11-26T07:22:06.701 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:06.700+0000 7f77e9ffb640 1 -- 172.21.15.143:0/3575303730 <== mon.0 v2:172.21.15.143:3300/0 4 ==== mgrmap(e 9) v1 ==== 86005+0+0 (secure 0 0 0) 0x7f77ec0b8070 con 0x7f77f4143ac0 2023-11-26T07:22:06.701 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:06.700+0000 7f77e9ffb640 1 --2- 172.21.15.143:0/3575303730 >> [v2:172.21.15.143:6832/2150557544,v1:172.21.15.143:6833/2150557544] conn(0x7f77d8066620 0x7f77d8068ae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:22:06.702 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:06.700+0000 7f77e9ffb640 1 -- 172.21.15.143:0/3575303730 <== mon.0 v2:172.21.15.143:3300/0 5 ==== osd_map(59..59 src has 1..59) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f77ec0aa070 con 0x7f77f4143ac0 2023-11-26T07:22:06.702 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:06.700+0000 7f77f97f6640 1 --2- 172.21.15.143:0/3575303730 >> [v2:172.21.15.143:6832/2150557544,v1:172.21.15.143:6833/2150557544] conn(0x7f77d8066620 0x7f77d8068ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:22:06.702 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:06.700+0000 7f77f97f6640 1 --2- 172.21.15.143:0/3575303730 >> [v2:172.21.15.143:6832/2150557544,v1:172.21.15.143:6833/2150557544] conn(0x7f77d8066620 0x7f77d8068ae0 secure :-1 s=READY pgs=392 cs=0 l=1 rev1=1 crypto rx=0x7f77f00079c0 tx=0x7f77f0008040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:22:06.705 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:06.704+0000 7f77e9ffb640 1 -- 172.21.15.143:0/3575303730 <== mon.0 v2:172.21.15.143:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172649 (secure 0 0 0) 0x7f77ec0eaf00 con 0x7f77f4143ac0 2023-11-26T07:22:06.877 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:06.876+0000 7f77fba5a640 1 -- 172.21.15.143:0/3575303730 --> [v2:172.21.15.143:6832/2150557544,v1:172.21.15.143:6833/2150557544] -- mgr_command(tid 0: {"prefix": "fs volume ls", "target": ["mon-mgr", ""]}) v1 -- 0x7f77f4071c30 con 0x7f77d8066620 2023-11-26T07:22:06.878 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:06.876+0000 7f77e9ffb640 1 -- 172.21.15.143:0/3575303730 <== mgr.4100 v2:172.21.15.143:6832/2150557544 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+41 (secure 0 0 0) 0x7f77f00108d0 con 0x7f77d8066620 2023-11-26T07:22:06.879 INFO:teuthology.orchestra.run.smithi143.stdout:[ 2023-11-26T07:22:06.879 INFO:teuthology.orchestra.run.smithi143.stdout: { 2023-11-26T07:22:06.879 INFO:teuthology.orchestra.run.smithi143.stdout: "name": "cephfs" 2023-11-26T07:22:06.879 INFO:teuthology.orchestra.run.smithi143.stdout: } 2023-11-26T07:22:06.879 INFO:teuthology.orchestra.run.smithi143.stdout:] 2023-11-26T07:22:06.880 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:06.880+0000 7f77fba5a640 1 -- 172.21.15.143:0/3575303730 >> [v2:172.21.15.143:6832/2150557544,v1:172.21.15.143:6833/2150557544] conn(0x7f77d8066620 msgr2=0x7f77d8068ae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:22:06.881 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:06.880+0000 7f77fba5a640 1 --2- 172.21.15.143:0/3575303730 >> [v2:172.21.15.143:6832/2150557544,v1:172.21.15.143:6833/2150557544] conn(0x7f77d8066620 0x7f77d8068ae0 secure :-1 s=READY pgs=392 cs=0 l=1 rev1=1 crypto rx=0x7f77f00079c0 tx=0x7f77f0008040 comp rx=0 tx=0).stop 2023-11-26T07:22:06.881 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:06.880+0000 7f77fba5a640 1 -- 172.21.15.143:0/3575303730 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f77f4143ac0 msgr2=0x7f77f4140140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:22:06.881 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:06.880+0000 7f77fba5a640 1 --2- 172.21.15.143:0/3575303730 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f77f4143ac0 0x7f77f4140140 secure :-1 s=READY pgs=826 cs=0 l=1 rev1=1 crypto rx=0x7f77ec01adc0 tx=0x7f77ec0194b0 comp rx=0 tx=0).stop 2023-11-26T07:22:06.881 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:06.880+0000 7f77fba5a640 1 -- 172.21.15.143:0/3575303730 shutdown_connections 2023-11-26T07:22:06.881 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:06.880+0000 7f77fba5a640 1 --2- 172.21.15.143:0/3575303730 >> [v2:172.21.15.143:6832/2150557544,v1:172.21.15.143:6833/2150557544] conn(0x7f77d8066620 0x7f77d8068ae0 unknown :-1 s=CLOSED pgs=392 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:22:06.881 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:06.880+0000 7f77fba5a640 1 --2- 172.21.15.143:0/3575303730 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f77f4143ac0 0x7f77f4140140 unknown :-1 s=CLOSED pgs=826 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:22:06.881 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:06.880+0000 7f77fba5a640 1 --2- 172.21.15.143:0/3575303730 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f77f40cf7c0 0x7f77f40adb90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:22:06.881 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:06.880+0000 7f77fba5a640 1 --2- 172.21.15.143:0/3575303730 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f77f40708b0 0x7f77f4136030 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:22:06.881 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:06.880+0000 7f77fba5a640 1 -- 172.21.15.143:0/3575303730 >> 172.21.15.143:0/3575303730 conn(0x7f77f406a890 msgr2=0x7f77f40ce200 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:22:06.882 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:06.880+0000 7f77fba5a640 1 -- 172.21.15.143:0/3575303730 shutdown_connections 2023-11-26T07:22:06.882 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:06.880+0000 7f77fba5a640 1 -- 172.21.15.143:0/3575303730 wait complete. 2023-11-26T07:22:06.899 DEBUG:teuthology.orchestra.run.smithi143:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mon mon_allow_pool_delete True 2023-11-26T07:22:07.039 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:07.036+0000 7f25a59b8640 1 Processor -- start 2023-11-26T07:22:07.039 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:07.036+0000 7f25a59b8640 1 -- start start 2023-11-26T07:22:07.041 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:07.040+0000 7f25a59b8640 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7f25a01596d0 0x7f25a0071850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:22:07.041 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:07.040+0000 7f25a59b8640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f25a00b2f80 0x7f25a006cbe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:22:07.041 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:07.040+0000 7f25a59b8640 1 -- --> v1:172.21.15.188:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f25a006c750 con 0x7f25a00ce680 2023-11-26T07:22:07.041 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:07.040+0000 7f25a59b8640 1 -- --> v2:172.21.15.188:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f25a00cbe00 con 0x7f25a01596d0 2023-11-26T07:22:07.041 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:07.040+0000 7f25a59b8640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f25a00b3550 con 0x7f25a00b2f80 2023-11-26T07:22:07.042 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:07.040+0000 7f259effd640 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7f25a01596d0 0x7f25a0071850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:22:07.042 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:07.040+0000 7f259f7fe640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f25a00b2f80 0x7f25a006cbe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:22:07.042 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:07.040+0000 7f259effd640 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7f25a01596d0 0x7f25a0071850 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.188:3300/0 says I am v2:172.21.15.143:37792/0 (socket says 172.21.15.143:37792) 2023-11-26T07:22:07.042 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:07.040+0000 7f2597fff640 1 --1- >> v1:172.21.15.188:6789/0 conn(0x7f25a00ce680 0x7f25a00b2870 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.188:6789/0 says I am v1:172.21.15.143:38458/0 (socket says 172.21.15.143:38458) 2023-11-26T07:22:07.042 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:07.040+0000 7f259effd640 1 -- 172.21.15.143:0/4252367326 learned_addr learned my addr 172.21.15.143:0/4252367326 (peer_addr_for_me v2:172.21.15.143:0/0) 2023-11-26T07:22:07.042 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:07.040+0000 7f259f7fe640 1 -- 172.21.15.143:0/4252367326 >> v2:172.21.15.188:3300/0 conn(0x7f25a01596d0 msgr2=0x7f25a0071850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-26T07:22:07.043 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:07.040+0000 7f259f7fe640 1 --2- 172.21.15.143:0/4252367326 >> v2:172.21.15.188:3300/0 conn(0x7f25a01596d0 0x7f25a0071850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:22:07.043 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:07.040+0000 7f259f7fe640 1 -- 172.21.15.143:0/4252367326 >> v1:172.21.15.188:6789/0 conn(0x7f25a00ce680 legacy=0x7f25a00b2870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:22:07.043 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:07.040+0000 7f259f7fe640 1 -- 172.21.15.143:0/4252367326 --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f25a0071e80 con 0x7f25a00b2f80 2023-11-26T07:22:07.045 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:07.044+0000 7f259f7fe640 1 --2- 172.21.15.143:0/4252367326 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f25a00b2f80 0x7f25a006cbe0 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7f2598009d90 tx=0x7f259801c950 comp rx=0 tx=0).ready entity=mon.2 client_cookie=69d2463db0bb7b11 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:22:07.045 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:07.044+0000 7f259e7fc640 1 -- 172.21.15.143:0/4252367326 <== mon.2 v2:172.21.15.188:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f25980a5880 con 0x7f25a00b2f80 2023-11-26T07:22:07.045 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:07.044+0000 7f259e7fc640 1 -- 172.21.15.143:0/4252367326 <== mon.2 v2:172.21.15.188:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f25980a59e0 con 0x7f25a00b2f80 2023-11-26T07:22:07.045 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:07.044+0000 7f259e7fc640 1 -- 172.21.15.143:0/4252367326 <== mon.2 v2:172.21.15.188:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f25980a5d70 con 0x7f25a00b2f80 2023-11-26T07:22:07.046 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:07.044+0000 7f25a59b8640 1 -- 172.21.15.143:0/4252367326 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f25a00b2f80 msgr2=0x7f25a006cbe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:22:07.046 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:07.044+0000 7f25a59b8640 1 --2- 172.21.15.143:0/4252367326 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f25a00b2f80 0x7f25a006cbe0 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7f2598009d90 tx=0x7f259801c950 comp rx=0 tx=0).stop 2023-11-26T07:22:07.046 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:07.044+0000 7f25a59b8640 1 -- 172.21.15.143:0/4252367326 shutdown_connections 2023-11-26T07:22:07.046 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:07.044+0000 7f25a59b8640 1 --2- 172.21.15.143:0/4252367326 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f25a00b2f80 0x7f25a006cbe0 unknown :-1 s=CLOSED pgs=108 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:22:07.046 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:07.044+0000 7f25a59b8640 1 --2- 172.21.15.143:0/4252367326 >> v2:172.21.15.188:3300/0 conn(0x7f25a01596d0 0x7f25a0071850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:22:07.047 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:07.044+0000 7f25a59b8640 1 -- 172.21.15.143:0/4252367326 >> 172.21.15.143:0/4252367326 conn(0x7f25a006a890 msgr2=0x7f25a006aca0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:22:07.047 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:07.044+0000 7f25a59b8640 1 -- 172.21.15.143:0/4252367326 shutdown_connections 2023-11-26T07:22:07.047 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:07.044+0000 7f25a59b8640 1 -- 172.21.15.143:0/4252367326 wait complete. 2023-11-26T07:22:07.047 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:07.044+0000 7f25a59b8640 1 Processor -- start 2023-11-26T07:22:07.047 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:07.044+0000 7f25a59b8640 1 -- start start 2023-11-26T07:22:07.050 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:07.048+0000 7f25a59b8640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f25a00b2f80 0x7f25a015aaa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:22:07.050 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:07.048+0000 7f25a59b8640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f25a00ce680 0x7f25a014ea10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:22:07.050 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:07.048+0000 7f25a59b8640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f25a01596d0 0x7f25a01f6a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:22:07.050 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:07.048+0000 7f25a59b8640 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f25a015bce0 con 0x7f25a00ce680 2023-11-26T07:22:07.050 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:07.048+0000 7f25a59b8640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f25a014ef50 con 0x7f25a00b2f80 2023-11-26T07:22:07.050 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:07.048+0000 7f25a59b8640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f25a015b160 con 0x7f25a01596d0 2023-11-26T07:22:07.050 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:07.048+0000 7f2597fff640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f25a00ce680 0x7f25a014ea10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:22:07.050 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:07.048+0000 7f259effd640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f25a00b2f80 0x7f25a015aaa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:22:07.051 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:07.048+0000 7f259f7fe640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f25a01596d0 0x7f25a01f6a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:22:07.051 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:07.048+0000 7f259effd640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f25a00b2f80 0x7f25a015aaa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.188:3300/0 says I am v2:172.21.15.143:37798/0 (socket says 172.21.15.143:37798) 2023-11-26T07:22:07.051 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:07.048+0000 7f259effd640 1 -- 172.21.15.143:0/2977696595 learned_addr learned my addr 172.21.15.143:0/2977696595 (peer_addr_for_me v2:172.21.15.143:0/0) 2023-11-26T07:22:07.051 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:07.048+0000 7f2597fff640 1 -- 172.21.15.143:0/2977696595 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f25a01596d0 msgr2=0x7f25a01f6a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:22:07.051 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:07.048+0000 7f2597fff640 1 --2- 172.21.15.143:0/2977696595 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f25a01596d0 0x7f25a01f6a70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:22:07.051 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:07.048+0000 7f2597fff640 1 -- 172.21.15.143:0/2977696595 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f25a00b2f80 msgr2=0x7f25a015aaa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:22:07.051 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:07.048+0000 7f2597fff640 1 --2- 172.21.15.143:0/2977696595 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f25a00b2f80 0x7f25a015aaa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:22:07.051 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:07.048+0000 7f2597fff640 1 -- 172.21.15.143:0/2977696595 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2598009a70 con 0x7f25a00ce680 2023-11-26T07:22:07.051 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:07.048+0000 7f2597fff640 1 --2- 172.21.15.143:0/2977696595 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f25a00ce680 0x7f25a014ea10 secure :-1 s=READY pgs=827 cs=0 l=1 rev1=1 crypto rx=0x7f258800c430 tx=0x7f258800c8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:22:07.051 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:07.048+0000 7f259cff9640 1 -- 172.21.15.143:0/2977696595 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f25880074e0 con 0x7f25a00ce680 2023-11-26T07:22:07.051 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:07.048+0000 7f259cff9640 1 -- 172.21.15.143:0/2977696595 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2588007640 con 0x7f25a00ce680 2023-11-26T07:22:07.051 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:07.048+0000 7f25a59b8640 1 -- 172.21.15.143:0/2977696595 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f25a015b440 con 0x7f25a00ce680 2023-11-26T07:22:07.052 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:07.048+0000 7f259cff9640 1 -- 172.21.15.143:0/2977696595 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2588004a10 con 0x7f25a00ce680 2023-11-26T07:22:07.052 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:07.048+0000 7f25a59b8640 1 -- 172.21.15.143:0/2977696595 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f25a015ba00 con 0x7f25a00ce680 2023-11-26T07:22:07.052 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:07.048+0000 7f25a59b8640 1 -- 172.21.15.143:0/2977696595 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f25a0067680 con 0x7f25a00ce680 2023-11-26T07:22:07.052 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:07.052+0000 7f259cff9640 1 -- 172.21.15.143:0/2977696595 <== mon.0 v2:172.21.15.143:3300/0 4 ==== mgrmap(e 9) v1 ==== 86005+0+0 (secure 0 0 0) 0x7f2588004b70 con 0x7f25a00ce680 2023-11-26T07:22:07.053 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:07.052+0000 7f259cff9640 1 --2- 172.21.15.143:0/2977696595 >> [v2:172.21.15.143:6832/2150557544,v1:172.21.15.143:6833/2150557544] conn(0x7f2580066650 0x7f2580068b10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:22:07.053 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:07.052+0000 7f259cff9640 1 -- 172.21.15.143:0/2977696595 <== mon.0 v2:172.21.15.143:3300/0 5 ==== osd_map(59..59 src has 1..59) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f25880839c0 con 0x7f25a00ce680 2023-11-26T07:22:07.053 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:07.052+0000 7f259effd640 1 --2- 172.21.15.143:0/2977696595 >> [v2:172.21.15.143:6832/2150557544,v1:172.21.15.143:6833/2150557544] conn(0x7f2580066650 0x7f2580068b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:22:07.053 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:07.052+0000 7f259effd640 1 --2- 172.21.15.143:0/2977696595 >> [v2:172.21.15.143:6832/2150557544,v1:172.21.15.143:6833/2150557544] conn(0x7f2580066650 0x7f2580068b10 secure :-1 s=READY pgs=393 cs=0 l=1 rev1=1 crypto rx=0x7f25a014faf0 tx=0x7f259000a350 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:22:07.057 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:07.056+0000 7f259cff9640 1 -- 172.21.15.143:0/2977696595 <== mon.0 v2:172.21.15.143:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172649 (secure 0 0 0) 0x7f2588050c70 con 0x7f25a00ce680 2023-11-26T07:22:07.205 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:07.204+0000 7f25a59b8640 1 -- 172.21.15.143:0/2977696595 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command([{prefix=config set, name=mon_allow_pool_delete}] v 0) v1 -- 0x7f25a00711b0 con 0x7f25a00ce680 2023-11-26T07:22:07.206 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:07.204+0000 7f259cff9640 1 -- 172.21.15.143:0/2977696595 <== mon.0 v2:172.21.15.143:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mon_allow_pool_delete}]=0 v14)=0 v14) v1 ==== 122+0+0 (secure 0 0 0) 0x7f2588054290 con 0x7f25a00ce680 2023-11-26T07:22:07.208 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:07.204+0000 7f25a59b8640 1 -- 172.21.15.143:0/2977696595 >> [v2:172.21.15.143:6832/2150557544,v1:172.21.15.143:6833/2150557544] conn(0x7f2580066650 msgr2=0x7f2580068b10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:22:07.208 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:07.204+0000 7f25a59b8640 1 --2- 172.21.15.143:0/2977696595 >> [v2:172.21.15.143:6832/2150557544,v1:172.21.15.143:6833/2150557544] conn(0x7f2580066650 0x7f2580068b10 secure :-1 s=READY pgs=393 cs=0 l=1 rev1=1 crypto rx=0x7f25a014faf0 tx=0x7f259000a350 comp rx=0 tx=0).stop 2023-11-26T07:22:07.208 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:07.204+0000 7f25a59b8640 1 -- 172.21.15.143:0/2977696595 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f25a00ce680 msgr2=0x7f25a014ea10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:22:07.208 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:07.204+0000 7f25a59b8640 1 --2- 172.21.15.143:0/2977696595 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f25a00ce680 0x7f25a014ea10 secure :-1 s=READY pgs=827 cs=0 l=1 rev1=1 crypto rx=0x7f258800c430 tx=0x7f258800c8f0 comp rx=0 tx=0).stop 2023-11-26T07:22:07.208 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:07.204+0000 7f25a59b8640 1 -- 172.21.15.143:0/2977696595 shutdown_connections 2023-11-26T07:22:07.209 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:07.204+0000 7f25a59b8640 1 --2- 172.21.15.143:0/2977696595 >> [v2:172.21.15.143:6832/2150557544,v1:172.21.15.143:6833/2150557544] conn(0x7f2580066650 0x7f2580068b10 unknown :-1 s=CLOSED pgs=393 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:22:07.209 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:07.204+0000 7f25a59b8640 1 --2- 172.21.15.143:0/2977696595 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f25a01596d0 0x7f25a01f6a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:22:07.209 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:07.204+0000 7f25a59b8640 1 --2- 172.21.15.143:0/2977696595 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f25a00ce680 0x7f25a014ea10 unknown :-1 s=CLOSED pgs=827 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:22:07.209 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:07.204+0000 7f25a59b8640 1 --2- 172.21.15.143:0/2977696595 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f25a00b2f80 0x7f25a015aaa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:22:07.209 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:07.204+0000 7f25a59b8640 1 -- 172.21.15.143:0/2977696595 >> 172.21.15.143:0/2977696595 conn(0x7f25a006a890 msgr2=0x7f25a00cbf70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:22:07.209 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:07.208+0000 7f25a59b8640 1 -- 172.21.15.143:0/2977696595 shutdown_connections 2023-11-26T07:22:07.209 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:07.208+0000 7f25a59b8640 1 -- 172.21.15.143:0/2977696595 wait complete. 2023-11-26T07:22:07.227 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-26T07:22:07.227 DEBUG:teuthology.orchestra.run.smithi143:> (cd /home/ubuntu/cephtest/mnt.0 && exec sudo mkdir -p ./volumes/_nogroup/subvol970) 2023-11-26T07:22:07.260 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-26T07:22:07.260 DEBUG:teuthology.orchestra.run.smithi143:> (cd /home/ubuntu/cephtest/mnt.0 && exec getfattr --only-values -n ceph.dir.layout.pool .) 2023-11-26T07:22:07.316 INFO:teuthology.orchestra.run.smithi143.stdout:cephfs_data 2023-11-26T07:22:07.316 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-26T07:22:07.316 DEBUG:teuthology.orchestra.run.smithi143:> (cd /home/ubuntu/cephtest/mnt.0 && exec sudo setfattr -n ceph.dir.layout.pool -v cephfs_data ./volumes/_nogroup/subvol970) 2023-11-26T07:22:07.386 DEBUG:teuthology.orchestra.run.smithi143:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume create cephfs subvol15241 2023-11-26T07:22:07.573 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:07.572+0000 7f8f65d01640 1 Processor -- start 2023-11-26T07:22:07.573 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:07.572+0000 7f8f65d01640 1 -- start start 2023-11-26T07:22:07.574 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:07.572+0000 7f8f65d01640 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7f8f600abef0 0x7f8f6006e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:22:07.574 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:07.572+0000 7f8f65d01640 1 -- --> v1:172.21.15.143:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8f6015f4f0 con 0x7f8f60072300 2023-11-26T07:22:07.574 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:07.572+0000 7f8f65d01640 1 -- --> v1:172.21.15.188:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8f600ac6b0 con 0x7f8f601557e0 2023-11-26T07:22:07.574 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:07.572+0000 7f8f65d01640 1 -- --> v2:172.21.15.188:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8f6006e830 con 0x7f8f600abef0 2023-11-26T07:22:07.574 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:07.572+0000 7f8f5effd640 1 --1- >> v1:172.21.15.143:6789/0 conn(0x7f8f60072300 0x7f8f600add10 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.143:6789/0 says I am v1:172.21.15.143:42812/0 (socket says 172.21.15.143:42812) 2023-11-26T07:22:07.575 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:07.572+0000 7f8f5effd640 1 -- 172.21.15.143:0/3514867277 learned_addr learned my addr 172.21.15.143:0/3514867277 (peer_addr_for_me v1:172.21.15.143:0/0) 2023-11-26T07:22:07.576 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:07.572+0000 7f8f5ffff640 1 --2- 172.21.15.143:0/3514867277 >> v2:172.21.15.188:3300/0 conn(0x7f8f600abef0 0x7f8f6006e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:22:07.576 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:07.572+0000 7f8f5e7fc640 1 -- 172.21.15.143:0/3514867277 <== mon.0 v1:172.21.15.143:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3840474215 0 0) 0x7f8f6015f4f0 con 0x7f8f60072300 2023-11-26T07:22:07.576 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:07.572+0000 7f8f5e7fc640 1 -- 172.21.15.143:0/3514867277 --> v1:172.21.15.143:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f8f48003530 con 0x7f8f60072300 2023-11-26T07:22:07.576 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:07.572+0000 7f8f5e7fc640 1 -- 172.21.15.143:0/3514867277 <== mon.0 v1:172.21.15.143:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1337200063 0 0) 0x7f8f48003530 con 0x7f8f60072300 2023-11-26T07:22:07.577 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:07.572+0000 7f8f5e7fc640 1 -- 172.21.15.143:0/3514867277 >> v2:172.21.15.188:3300/0 conn(0x7f8f600abef0 msgr2=0x7f8f6006e2f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-26T07:22:07.577 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:07.572+0000 7f8f5e7fc640 1 --2- 172.21.15.143:0/3514867277 >> v2:172.21.15.188:3300/0 conn(0x7f8f600abef0 0x7f8f6006e2f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:22:07.577 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:07.572+0000 7f8f5e7fc640 1 -- 172.21.15.143:0/3514867277 >> v1:172.21.15.188:6789/0 conn(0x7f8f601557e0 legacy=0x7f8f60071bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:22:07.577 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:07.572+0000 7f8f5e7fc640 1 -- 172.21.15.143:0/3514867277 --> v1:172.21.15.143:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8f60155bc0 con 0x7f8f60072300 2023-11-26T07:22:07.577 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:07.572+0000 7f8f5e7fc640 1 -- 172.21.15.143:0/3514867277 <== mon.0 v1:172.21.15.143:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2137882523 0 0) 0x7f8f54001680 con 0x7f8f60072300 2023-11-26T07:22:07.577 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:07.572+0000 7f8f5e7fc640 1 -- 172.21.15.143:0/3514867277 >> v1:172.21.15.143:6789/0 conn(0x7f8f60072300 legacy=0x7f8f600add10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:22:07.577 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:07.572+0000 7f8f5e7fc640 1 --2- 172.21.15.143:0/3514867277 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f8f48003980 0x7f8f48003d60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:22:07.577 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:07.572+0000 7f8f5e7fc640 1 --2- 172.21.15.143:0/3514867277 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f8f48004380 0x7f8f48004850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:22:07.577 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:07.572+0000 7f8f5e7fc640 1 --2- 172.21.15.143:0/3514867277 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f8f48005050 0x7f8f4800b990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:22:07.577 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:07.572+0000 7f8f5e7fc640 1 -- 172.21.15.143:0/3514867277 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8f4800bf80 con 0x7f8f48003980 2023-11-26T07:22:07.577 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:07.572+0000 7f8f5e7fc640 1 -- 172.21.15.143:0/3514867277 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8f4800c0f0 con 0x7f8f48004380 2023-11-26T07:22:07.577 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:07.572+0000 7f8f5e7fc640 1 -- 172.21.15.143:0/3514867277 --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8f4800c2b0 con 0x7f8f48005050 2023-11-26T07:22:07.577 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:07.572+0000 7f8f5f7fe640 1 --2- 172.21.15.143:0/3514867277 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f8f48003980 0x7f8f48003d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:22:07.578 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:07.572+0000 7f8f5effd640 1 --2- 172.21.15.143:0/3514867277 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f8f48004380 0x7f8f48004850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:22:07.578 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:07.572+0000 7f8f5ffff640 1 --2- 172.21.15.143:0/3514867277 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f8f48005050 0x7f8f4800b990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:22:07.578 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:07.572+0000 7f8f5f7fe640 1 -- 172.21.15.143:0/3514867277 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f8f48005050 msgr2=0x7f8f4800b990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-26T07:22:07.578 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:07.572+0000 7f8f5f7fe640 1 --2- 172.21.15.143:0/3514867277 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f8f48005050 0x7f8f4800b990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:22:07.578 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:07.572+0000 7f8f5f7fe640 1 -- 172.21.15.143:0/3514867277 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f8f48004380 msgr2=0x7f8f48004850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-26T07:22:07.578 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:07.572+0000 7f8f5f7fe640 1 --2- 172.21.15.143:0/3514867277 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f8f48004380 0x7f8f48004850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:22:07.578 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:07.572+0000 7f8f5f7fe640 1 -- 172.21.15.143:0/3514867277 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8f48003530 con 0x7f8f48003980 2023-11-26T07:22:07.578 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:07.576+0000 7f8f5ffff640 1 -- 172.21.15.143:0/3514867277 reap_dead start 2023-11-26T07:22:07.579 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:07.576+0000 7f8f5f7fe640 1 --2- 172.21.15.143:0/3514867277 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f8f48003980 0x7f8f48003d60 secure :-1 s=READY pgs=829 cs=0 l=1 rev1=1 crypto rx=0x7f8f50001860 tx=0x7f8f5001c4d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e8fec8a6ba56f4c2 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:22:07.579 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:07.576+0000 7f8f5e7fc640 1 -- 172.21.15.143:0/3514867277 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8f500ab600 con 0x7f8f48003980 2023-11-26T07:22:07.579 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:07.576+0000 7f8f5e7fc640 1 -- 172.21.15.143:0/3514867277 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8f500a7050 con 0x7f8f48003980 2023-11-26T07:22:07.579 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:07.576+0000 7f8f5e7fc640 1 -- 172.21.15.143:0/3514867277 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8f500ab9e0 con 0x7f8f48003980 2023-11-26T07:22:07.580 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:07.576+0000 7f8f65d01640 1 -- 172.21.15.143:0/3514867277 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f8f48003980 msgr2=0x7f8f48003d60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:22:07.580 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:07.576+0000 7f8f65d01640 1 --2- 172.21.15.143:0/3514867277 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f8f48003980 0x7f8f48003d60 secure :-1 s=READY pgs=829 cs=0 l=1 rev1=1 crypto rx=0x7f8f50001860 tx=0x7f8f5001c4d0 comp rx=0 tx=0).stop 2023-11-26T07:22:07.580 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:07.576+0000 7f8f65d01640 1 -- 172.21.15.143:0/3514867277 shutdown_connections 2023-11-26T07:22:07.580 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:07.576+0000 7f8f65d01640 1 --2- 172.21.15.143:0/3514867277 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f8f48003980 0x7f8f48003d60 unknown :-1 s=CLOSED pgs=829 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:22:07.580 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:07.576+0000 7f8f65d01640 1 -- 172.21.15.143:0/3514867277 >> 172.21.15.143:0/3514867277 conn(0x7f8f600b0c10 msgr2=0x7f8f600b1020 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:22:07.580 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:07.576+0000 7f8f65d01640 1 -- 172.21.15.143:0/3514867277 shutdown_connections 2023-11-26T07:22:07.580 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:07.576+0000 7f8f65d01640 1 -- 172.21.15.143:0/3514867277 wait complete. 2023-11-26T07:22:07.581 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:07.580+0000 7f8f65d01640 1 Processor -- start 2023-11-26T07:22:07.582 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:07.580+0000 7f8f65d01640 1 -- start start 2023-11-26T07:22:07.584 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:07.580+0000 7f8f65d01640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f8f48003980 0x7f8f60099a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:22:07.584 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:07.580+0000 7f8f65d01640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f8f6009a400 0x7f8f60152580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:22:07.584 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:07.580+0000 7f8f65d01640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f8f6015b580 0x7f8f6015ba00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:22:07.584 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:07.580+0000 7f8f65d01640 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8f600b2010 con 0x7f8f6015b580 2023-11-26T07:22:07.584 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:07.580+0000 7f8f65d01640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8f6015bf40 con 0x7f8f48003980 2023-11-26T07:22:07.584 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:07.580+0000 7f8f65d01640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8f60099fb0 con 0x7f8f6009a400 2023-11-26T07:22:07.584 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:07.580+0000 7f8f5ffff640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f8f6015b580 0x7f8f6015ba00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:22:07.584 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:07.580+0000 7f8f5ffff640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f8f6015b580 0x7f8f6015ba00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:35298/0 (socket says 172.21.15.143:35298) 2023-11-26T07:22:07.584 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:07.580+0000 7f8f5ffff640 1 -- 172.21.15.143:0/4094510022 learned_addr learned my addr 172.21.15.143:0/4094510022 (peer_addr_for_me v2:172.21.15.143:0/0) 2023-11-26T07:22:07.584 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:07.580+0000 7f8f5effd640 1 --2- 172.21.15.143:0/4094510022 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f8f6009a400 0x7f8f60152580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:22:07.584 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:07.580+0000 7f8f5f7fe640 1 --2- 172.21.15.143:0/4094510022 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f8f48003980 0x7f8f60099a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:22:07.585 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:07.580+0000 7f8f5ffff640 1 -- 172.21.15.143:0/4094510022 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f8f6009a400 msgr2=0x7f8f60152580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:22:07.585 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:07.580+0000 7f8f5ffff640 1 --2- 172.21.15.143:0/4094510022 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f8f6009a400 0x7f8f60152580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:22:07.585 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:07.580+0000 7f8f5ffff640 1 -- 172.21.15.143:0/4094510022 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f8f48003980 msgr2=0x7f8f60099a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:22:07.585 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:07.580+0000 7f8f5ffff640 1 --2- 172.21.15.143:0/4094510022 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f8f48003980 0x7f8f60099a70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:22:07.585 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:07.580+0000 7f8f5ffff640 1 -- 172.21.15.143:0/4094510022 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8f600b2b50 con 0x7f8f6015b580 2023-11-26T07:22:07.585 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:07.580+0000 7f8f5ffff640 1 --2- 172.21.15.143:0/4094510022 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f8f6015b580 0x7f8f6015ba00 secure :-1 s=READY pgs=830 cs=0 l=1 rev1=1 crypto rx=0x7f8f600ac300 tx=0x7f8f5800f1b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:22:07.586 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:07.584+0000 7f8f47fff640 1 -- 172.21.15.143:0/4094510022 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8f5800e040 con 0x7f8f6015b580 2023-11-26T07:22:07.586 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:07.584+0000 7f8f47fff640 1 -- 172.21.15.143:0/4094510022 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8f5800f040 con 0x7f8f6015b580 2023-11-26T07:22:07.586 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:07.584+0000 7f8f47fff640 1 -- 172.21.15.143:0/4094510022 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8f5800fb70 con 0x7f8f6015b580 2023-11-26T07:22:07.586 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:07.584+0000 7f8f65d01640 1 -- 172.21.15.143:0/4094510022 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8f60152ac0 con 0x7f8f6015b580 2023-11-26T07:22:07.586 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:07.584+0000 7f8f65d01640 1 -- 172.21.15.143:0/4094510022 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8f600b2500 con 0x7f8f6015b580 2023-11-26T07:22:07.586 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:07.584+0000 7f8f65d01640 1 -- 172.21.15.143:0/4094510022 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8f600b2ce0 con 0x7f8f6015b580 2023-11-26T07:22:07.586 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:07.584+0000 7f8f47fff640 1 -- 172.21.15.143:0/4094510022 <== mon.0 v2:172.21.15.143:3300/0 4 ==== mgrmap(e 9) v1 ==== 86005+0+0 (secure 0 0 0) 0x7f8f5800fcd0 con 0x7f8f6015b580 2023-11-26T07:22:07.588 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:07.584+0000 7f8f47fff640 1 --2- 172.21.15.143:0/4094510022 >> [v2:172.21.15.143:6832/2150557544,v1:172.21.15.143:6833/2150557544] conn(0x7f8f4c066650 0x7f8f4c068b10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:22:07.588 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:07.584+0000 7f8f47fff640 1 -- 172.21.15.143:0/4094510022 <== mon.0 v2:172.21.15.143:3300/0 5 ==== osd_map(59..59 src has 1..59) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f8f580849a0 con 0x7f8f6015b580 2023-11-26T07:22:07.588 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:07.584+0000 7f8f5f7fe640 1 --2- 172.21.15.143:0/4094510022 >> [v2:172.21.15.143:6832/2150557544,v1:172.21.15.143:6833/2150557544] conn(0x7f8f4c066650 0x7f8f4c068b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:22:07.588 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:07.588+0000 7f8f5f7fe640 1 --2- 172.21.15.143:0/4094510022 >> [v2:172.21.15.143:6832/2150557544,v1:172.21.15.143:6833/2150557544] conn(0x7f8f4c066650 0x7f8f4c068b10 secure :-1 s=READY pgs=394 cs=0 l=1 rev1=1 crypto rx=0x7f8f500ad000 tx=0x7f8f50005040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:22:07.591 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:07.588+0000 7f8f47fff640 1 -- 172.21.15.143:0/4094510022 <== mon.0 v2:172.21.15.143:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172649 (secure 0 0 0) 0x7f8f58051b30 con 0x7f8f6015b580 2023-11-26T07:22:07.770 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:07.768+0000 7f8f65d01640 1 -- 172.21.15.143:0/4094510022 --> [v2:172.21.15.143:6832/2150557544,v1:172.21.15.143:6833/2150557544] -- mgr_command(tid 0: {"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "subvol15241", "target": ["mon-mgr", ""]}) v1 -- 0x7f8f6015f180 con 0x7f8f4c066650 2023-11-26T07:22:07.775 INFO:tasks.ceph.mgr.y.smithi143.stderr:2023-11-26T07:22:07.772+0000 7f5048ac6640 -1 client.0 error registering admin socket command: (17) File exists 2023-11-26T07:22:07.776 INFO:tasks.ceph.mgr.y.smithi143.stderr:2023-11-26T07:22:07.772+0000 7f5048ac6640 -1 client.0 error registering admin socket command: (17) File exists 2023-11-26T07:22:07.776 INFO:tasks.ceph.mgr.y.smithi143.stderr:2023-11-26T07:22:07.772+0000 7f5048ac6640 -1 client.0 error registering admin socket command: (17) File exists 2023-11-26T07:22:07.776 INFO:tasks.ceph.mgr.y.smithi143.stderr:2023-11-26T07:22:07.772+0000 7f5048ac6640 -1 client.0 error registering admin socket command: (17) File exists 2023-11-26T07:22:07.776 INFO:tasks.ceph.mgr.y.smithi143.stderr:2023-11-26T07:22:07.772+0000 7f5048ac6640 -1 client.0 error registering admin socket command: (17) File exists 2023-11-26T07:22:07.810 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:07.808+0000 7f8f47fff640 1 -- 172.21.15.143:0/4094510022 <== mgr.4100 v2:172.21.15.143:6832/2150557544 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+0 (secure 0 0 0) 0x7f8f500052f0 con 0x7f8f4c066650 2023-11-26T07:22:07.813 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:07.812+0000 7f8f65d01640 1 -- 172.21.15.143:0/4094510022 >> [v2:172.21.15.143:6832/2150557544,v1:172.21.15.143:6833/2150557544] conn(0x7f8f4c066650 msgr2=0x7f8f4c068b10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:22:07.813 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:07.812+0000 7f8f65d01640 1 --2- 172.21.15.143:0/4094510022 >> [v2:172.21.15.143:6832/2150557544,v1:172.21.15.143:6833/2150557544] conn(0x7f8f4c066650 0x7f8f4c068b10 secure :-1 s=READY pgs=394 cs=0 l=1 rev1=1 crypto rx=0x7f8f500ad000 tx=0x7f8f50005040 comp rx=0 tx=0).stop 2023-11-26T07:22:07.813 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:07.812+0000 7f8f65d01640 1 -- 172.21.15.143:0/4094510022 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f8f6015b580 msgr2=0x7f8f6015ba00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:22:07.813 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:07.812+0000 7f8f65d01640 1 --2- 172.21.15.143:0/4094510022 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f8f6015b580 0x7f8f6015ba00 secure :-1 s=READY pgs=830 cs=0 l=1 rev1=1 crypto rx=0x7f8f600ac300 tx=0x7f8f5800f1b0 comp rx=0 tx=0).stop 2023-11-26T07:22:07.813 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:07.812+0000 7f8f65d01640 1 -- 172.21.15.143:0/4094510022 shutdown_connections 2023-11-26T07:22:07.813 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:07.812+0000 7f8f65d01640 1 --2- 172.21.15.143:0/4094510022 >> [v2:172.21.15.143:6832/2150557544,v1:172.21.15.143:6833/2150557544] conn(0x7f8f4c066650 0x7f8f4c068b10 unknown :-1 s=CLOSED pgs=394 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:22:07.814 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:07.812+0000 7f8f65d01640 1 --2- 172.21.15.143:0/4094510022 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f8f6015b580 0x7f8f6015ba00 unknown :-1 s=CLOSED pgs=830 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:22:07.814 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:07.812+0000 7f8f65d01640 1 --2- 172.21.15.143:0/4094510022 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f8f6009a400 0x7f8f60152580 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:22:07.814 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:07.812+0000 7f8f65d01640 1 --2- 172.21.15.143:0/4094510022 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f8f48003980 0x7f8f60099a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:22:07.814 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:07.812+0000 7f8f65d01640 1 -- 172.21.15.143:0/4094510022 >> 172.21.15.143:0/4094510022 conn(0x7f8f600b0c10 msgr2=0x7f8f600ce8c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:22:07.814 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:07.812+0000 7f8f65d01640 1 -- 172.21.15.143:0/4094510022 shutdown_connections 2023-11-26T07:22:07.814 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:07.812+0000 7f8f65d01640 1 -- 172.21.15.143:0/4094510022 wait complete. 2023-11-26T07:22:07.832 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-26T07:22:07.832 DEBUG:teuthology.orchestra.run.smithi143:> (cd /home/ubuntu/cephtest/mnt.0 && exec sudo cp ./volumes/_nogroup/subvol15241/.meta ./volumes/_nogroup/subvol970) 2023-11-26T07:22:10.424 DEBUG:teuthology.orchestra.run.smithi143:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume getpath cephfs subvol970 2023-11-26T07:22:10.575 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:10.572+0000 7fa72ee32640 1 Processor -- start 2023-11-26T07:22:10.575 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:10.572+0000 7fa72ee32640 1 -- start start 2023-11-26T07:22:10.576 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:10.576+0000 7fa72ee32640 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7fa728143ac0 0x7fa728070370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:22:10.576 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:10.576+0000 7fa72ee32640 1 -- --> v1:172.21.15.143:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa7280cc730 con 0x7fa7280cf990 2023-11-26T07:22:10.577 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:10.576+0000 7fa72ee32640 1 -- --> v1:172.21.15.188:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa728070ee0 con 0x7fa7280708b0 2023-11-26T07:22:10.577 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:10.576+0000 7fa72ee32640 1 -- --> v2:172.21.15.188:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa728143ea0 con 0x7fa728143ac0 2023-11-26T07:22:10.577 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:10.576+0000 7fa72d3cf640 1 --1- >> v1:172.21.15.143:6789/0 conn(0x7fa7280cf990 0x7fa7280cd1b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.143:6789/0 says I am v1:172.21.15.143:42826/0 (socket says 172.21.15.143:42826) 2023-11-26T07:22:10.577 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:10.576+0000 7fa72d3cf640 1 -- 172.21.15.143:0/2533239461 learned_addr learned my addr 172.21.15.143:0/2533239461 (peer_addr_for_me v1:172.21.15.143:0/0) 2023-11-26T07:22:10.577 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:10.576+0000 7fa72cbce640 1 --2- 172.21.15.143:0/2533239461 >> v2:172.21.15.188:3300/0 conn(0x7fa728143ac0 0x7fa728070370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:22:10.578 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:10.576+0000 7fa7277fe640 1 -- 172.21.15.143:0/2533239461 <== mon.0 v1:172.21.15.143:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3960042605 0 0) 0x7fa7280cc730 con 0x7fa7280cf990 2023-11-26T07:22:10.578 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:10.576+0000 7fa7277fe640 1 -- 172.21.15.143:0/2533239461 --> v1:172.21.15.143:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fa718003530 con 0x7fa7280cf990 2023-11-26T07:22:10.578 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:10.576+0000 7fa7277fe640 1 -- 172.21.15.143:0/2533239461 <== mon.1 v1:172.21.15.188:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1362936436 0 0) 0x7fa728070ee0 con 0x7fa7280708b0 2023-11-26T07:22:10.578 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:10.576+0000 7fa7277fe640 1 -- 172.21.15.143:0/2533239461 --> v1:172.21.15.188:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fa7280cc730 con 0x7fa7280708b0 2023-11-26T07:22:10.578 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:10.576+0000 7fa72cbce640 1 -- 172.21.15.143:0/2533239461 >> v1:172.21.15.188:6789/0 conn(0x7fa7280708b0 legacy=0x7fa7280cf280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:22:10.578 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:10.576+0000 7fa72cbce640 1 -- 172.21.15.143:0/2533239461 >> v1:172.21.15.143:6789/0 conn(0x7fa7280cf990 legacy=0x7fa7280cd1b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:22:10.578 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:10.576+0000 7fa72cbce640 1 -- 172.21.15.143:0/2533239461 --> v2:172.21.15.188:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa7280711c0 con 0x7fa728143ac0 2023-11-26T07:22:10.580 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:10.580+0000 7fa72cbce640 1 --2- 172.21.15.143:0/2533239461 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fa728143ac0 0x7fa728070370 secure :-1 s=READY pgs=331 cs=0 l=1 rev1=1 crypto rx=0x7fa71c0097c0 tx=0x7fa71c01c480 comp rx=0 tx=0).ready entity=mon.1 client_cookie=7d8d09d9f6d1300b server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:22:10.581 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:10.580+0000 7fa7277fe640 1 -- 172.21.15.143:0/2533239461 <== mon.1 v2:172.21.15.188:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa71c0a8050 con 0x7fa728143ac0 2023-11-26T07:22:10.581 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:10.580+0000 7fa7277fe640 1 -- 172.21.15.143:0/2533239461 <== mon.1 v2:172.21.15.188:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa71c0ac5f0 con 0x7fa728143ac0 2023-11-26T07:22:10.581 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:10.580+0000 7fa7277fe640 1 -- 172.21.15.143:0/2533239461 <== mon.1 v2:172.21.15.188:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa71c0ac980 con 0x7fa728143ac0 2023-11-26T07:22:10.581 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:10.580+0000 7fa72ee32640 1 -- 172.21.15.143:0/2533239461 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fa728143ac0 msgr2=0x7fa728070370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:22:10.581 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:10.580+0000 7fa72ee32640 1 --2- 172.21.15.143:0/2533239461 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fa728143ac0 0x7fa728070370 secure :-1 s=READY pgs=331 cs=0 l=1 rev1=1 crypto rx=0x7fa71c0097c0 tx=0x7fa71c01c480 comp rx=0 tx=0).stop 2023-11-26T07:22:10.582 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:10.580+0000 7fa72ee32640 1 -- 172.21.15.143:0/2533239461 shutdown_connections 2023-11-26T07:22:10.582 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:10.580+0000 7fa72ee32640 1 --2- 172.21.15.143:0/2533239461 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fa728143ac0 0x7fa728070370 unknown :-1 s=CLOSED pgs=331 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:22:10.582 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:10.580+0000 7fa72ee32640 1 -- 172.21.15.143:0/2533239461 >> 172.21.15.143:0/2533239461 conn(0x7fa72806a890 msgr2=0x7fa72806aca0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:22:10.582 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:10.580+0000 7fa72ee32640 1 -- 172.21.15.143:0/2533239461 shutdown_connections 2023-11-26T07:22:10.582 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:10.580+0000 7fa72ee32640 1 -- 172.21.15.143:0/2533239461 wait complete. 2023-11-26T07:22:10.583 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:10.580+0000 7fa72ee32640 1 Processor -- start 2023-11-26T07:22:10.583 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:10.580+0000 7fa72ee32640 1 -- start start 2023-11-26T07:22:10.585 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:10.584+0000 7fa72ee32640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fa7280708b0 0x7fa7281368f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:22:10.585 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:10.584+0000 7fa72ee32640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fa7280cf990 0x7fa728136fb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:22:10.585 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:10.584+0000 7fa72ee32640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fa728143ac0 0x7fa728138950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:22:10.585 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:10.584+0000 7fa72ee32640 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa728136e30 con 0x7fa7280cf990 2023-11-26T07:22:10.585 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:10.584+0000 7fa72ee32640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa7281378b0 con 0x7fa728143ac0 2023-11-26T07:22:10.586 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:10.584+0000 7fa72ee32640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa728137b90 con 0x7fa7280708b0 2023-11-26T07:22:10.586 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:10.584+0000 7fa727fff640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fa7280cf990 0x7fa728136fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:22:10.586 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:10.584+0000 7fa727fff640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fa7280cf990 0x7fa728136fb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:35330/0 (socket says 172.21.15.143:35330) 2023-11-26T07:22:10.586 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:10.584+0000 7fa727fff640 1 -- 172.21.15.143:0/2174950444 learned_addr learned my addr 172.21.15.143:0/2174950444 (peer_addr_for_me v2:172.21.15.143:0/0) 2023-11-26T07:22:10.586 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:10.584+0000 7fa72cbce640 1 --2- 172.21.15.143:0/2174950444 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fa7280708b0 0x7fa7281368f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:22:10.586 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:10.584+0000 7fa72d3cf640 1 --2- 172.21.15.143:0/2174950444 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fa728143ac0 0x7fa728138950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:22:10.586 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:10.584+0000 7fa727fff640 1 -- 172.21.15.143:0/2174950444 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fa7280708b0 msgr2=0x7fa7281368f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:22:10.586 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:10.584+0000 7fa727fff640 1 --2- 172.21.15.143:0/2174950444 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fa7280708b0 0x7fa7281368f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:22:10.586 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:10.584+0000 7fa727fff640 1 -- 172.21.15.143:0/2174950444 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fa728143ac0 msgr2=0x7fa728138950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:22:10.586 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:10.584+0000 7fa727fff640 1 --2- 172.21.15.143:0/2174950444 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fa728143ac0 0x7fa728138950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:22:10.586 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:10.584+0000 7fa727fff640 1 -- 172.21.15.143:0/2174950444 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa71c009420 con 0x7fa7280cf990 2023-11-26T07:22:10.586 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:10.584+0000 7fa727fff640 1 --2- 172.21.15.143:0/2174950444 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fa7280cf990 0x7fa728136fb0 secure :-1 s=READY pgs=834 cs=0 l=1 rev1=1 crypto rx=0x7fa71400c430 tx=0x7fa71400c8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:22:10.587 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:10.584+0000 7fa7257fa640 1 -- 172.21.15.143:0/2174950444 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa7140052d0 con 0x7fa7280cf990 2023-11-26T07:22:10.587 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:10.584+0000 7fa7257fa640 1 -- 172.21.15.143:0/2174950444 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa714005450 con 0x7fa7280cf990 2023-11-26T07:22:10.587 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:10.584+0000 7fa72ee32640 1 -- 172.21.15.143:0/2174950444 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa728137e70 con 0x7fa7280cf990 2023-11-26T07:22:10.588 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:10.584+0000 7fa7257fa640 1 -- 172.21.15.143:0/2174950444 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa714003840 con 0x7fa7280cf990 2023-11-26T07:22:10.588 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:10.584+0000 7fa72ee32640 1 -- 172.21.15.143:0/2174950444 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa728138430 con 0x7fa7280cf990 2023-11-26T07:22:10.588 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:10.584+0000 7fa72ee32640 1 -- 172.21.15.143:0/2174950444 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa7281385f0 con 0x7fa7280cf990 2023-11-26T07:22:10.588 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:10.584+0000 7fa7257fa640 1 -- 172.21.15.143:0/2174950444 <== mon.0 v2:172.21.15.143:3300/0 4 ==== mgrmap(e 10) v1 ==== 86056+0+0 (secure 0 0 0) 0x7fa7140039a0 con 0x7fa7280cf990 2023-11-26T07:22:10.589 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:10.588+0000 7fa7257fa640 1 --2- 172.21.15.143:0/2174950444 >> [v2:172.21.15.143:6832/2150557544,v1:172.21.15.143:6833/2150557544] conn(0x7fa70c0666b0 0x7fa70c068b70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:22:10.589 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:10.588+0000 7fa7257fa640 1 -- 172.21.15.143:0/2174950444 <== mon.0 v2:172.21.15.143:3300/0 5 ==== osd_map(59..59 src has 1..59) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fa714083890 con 0x7fa7280cf990 2023-11-26T07:22:10.589 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:10.588+0000 7fa72cbce640 1 --2- 172.21.15.143:0/2174950444 >> [v2:172.21.15.143:6832/2150557544,v1:172.21.15.143:6833/2150557544] conn(0x7fa70c0666b0 0x7fa70c068b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:22:10.589 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:10.588+0000 7fa72cbce640 1 --2- 172.21.15.143:0/2174950444 >> [v2:172.21.15.143:6832/2150557544,v1:172.21.15.143:6833/2150557544] conn(0x7fa70c0666b0 0x7fa70c068b70 secure :-1 s=READY pgs=395 cs=0 l=1 rev1=1 crypto rx=0x7fa71c0ae000 tx=0x7fa71c0a7000 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:22:10.592 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:10.592+0000 7fa7257fa640 1 -- 172.21.15.143:0/2174950444 <== mon.0 v2:172.21.15.143:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172649 (secure 0 0 0) 0x7fa714050a00 con 0x7fa7280cf990 2023-11-26T07:22:10.765 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:10.764+0000 7fa72ee32640 1 -- 172.21.15.143:0/2174950444 --> [v2:172.21.15.143:6832/2150557544,v1:172.21.15.143:6833/2150557544] -- mgr_command(tid 0: {"prefix": "fs subvolume getpath", "vol_name": "cephfs", "sub_name": "subvol970", "target": ["mon-mgr", ""]}) v1 -- 0x7fa728067660 con 0x7fa70c0666b0 2023-11-26T07:22:10.795 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:10.792+0000 7fa7257fa640 1 -- 172.21.15.143:0/2174950444 <== mgr.4100 v2:172.21.15.143:6832/2150557544 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+28 (secure 0 0 0) 0x7fa728067660 con 0x7fa70c0666b0 2023-11-26T07:22:10.795 INFO:teuthology.orchestra.run.smithi143.stdout:/volumes/_nogroup/subvol970 2023-11-26T07:22:10.797 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:10.796+0000 7fa72ee32640 1 -- 172.21.15.143:0/2174950444 >> [v2:172.21.15.143:6832/2150557544,v1:172.21.15.143:6833/2150557544] conn(0x7fa70c0666b0 msgr2=0x7fa70c068b70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:22:10.798 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:10.796+0000 7fa72ee32640 1 --2- 172.21.15.143:0/2174950444 >> [v2:172.21.15.143:6832/2150557544,v1:172.21.15.143:6833/2150557544] conn(0x7fa70c0666b0 0x7fa70c068b70 secure :-1 s=READY pgs=395 cs=0 l=1 rev1=1 crypto rx=0x7fa71c0ae000 tx=0x7fa71c0a7000 comp rx=0 tx=0).stop 2023-11-26T07:22:10.798 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:10.796+0000 7fa72ee32640 1 -- 172.21.15.143:0/2174950444 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fa7280cf990 msgr2=0x7fa728136fb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:22:10.798 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:10.796+0000 7fa72ee32640 1 --2- 172.21.15.143:0/2174950444 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fa7280cf990 0x7fa728136fb0 secure :-1 s=READY pgs=834 cs=0 l=1 rev1=1 crypto rx=0x7fa71400c430 tx=0x7fa71400c8f0 comp rx=0 tx=0).stop 2023-11-26T07:22:10.798 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:10.796+0000 7fa72ee32640 1 -- 172.21.15.143:0/2174950444 shutdown_connections 2023-11-26T07:22:10.798 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:10.796+0000 7fa72ee32640 1 --2- 172.21.15.143:0/2174950444 >> [v2:172.21.15.143:6832/2150557544,v1:172.21.15.143:6833/2150557544] conn(0x7fa70c0666b0 0x7fa70c068b70 unknown :-1 s=CLOSED pgs=395 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:22:10.798 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:10.796+0000 7fa72ee32640 1 --2- 172.21.15.143:0/2174950444 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fa728143ac0 0x7fa728138950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:22:10.798 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:10.796+0000 7fa72ee32640 1 --2- 172.21.15.143:0/2174950444 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fa7280cf990 0x7fa728136fb0 unknown :-1 s=CLOSED pgs=834 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:22:10.798 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:10.796+0000 7fa72ee32640 1 --2- 172.21.15.143:0/2174950444 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fa7280708b0 0x7fa7281368f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:22:10.798 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:10.796+0000 7fa72ee32640 1 -- 172.21.15.143:0/2174950444 >> 172.21.15.143:0/2174950444 conn(0x7fa72806a890 msgr2=0x7fa728139ac0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:22:10.798 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:10.796+0000 7fa72ee32640 1 -- 172.21.15.143:0/2174950444 shutdown_connections 2023-11-26T07:22:10.799 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:10.796+0000 7fa72ee32640 1 -- 172.21.15.143:0/2174950444 wait complete. 2023-11-26T07:22:10.816 DEBUG:teuthology.orchestra.run.smithi143:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume getpath cephfs subvol970 2023-11-26T07:22:10.956 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:10.956+0000 7f6dcbd2a640 1 Processor -- start 2023-11-26T07:22:10.956 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:10.956+0000 7f6dcbd2a640 1 -- start start 2023-11-26T07:22:10.958 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:10.956+0000 7f6dcbd2a640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f6dc414b1e0 0x7f6dc40cf280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:22:10.958 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:10.956+0000 7f6dcbd2a640 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7f6dc40cf7c0 0x7f6dc40708c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:22:10.958 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:10.956+0000 7f6dcbd2a640 1 -- --> v1:172.21.15.143:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6dc40cfc20 con 0x7f6dc4070e00 2023-11-26T07:22:10.958 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:10.956+0000 7f6dcbd2a640 1 -- --> v2:172.21.15.188:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6dc4162080 con 0x7f6dc40cf7c0 2023-11-26T07:22:10.958 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:10.956+0000 7f6dcbd2a640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6dc4161d70 con 0x7f6dc414b1e0 2023-11-26T07:22:10.958 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:10.956+0000 7f6dca2c7640 1 --1- >> v1:172.21.15.143:6789/0 conn(0x7f6dc4070e00 0x7f6dc40b1ef0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.143:6789/0 says I am v1:172.21.15.143:42836/0 (socket says 172.21.15.143:42836) 2023-11-26T07:22:10.958 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:10.956+0000 7f6dca2c7640 1 -- 172.21.15.143:0/3551026593 learned_addr learned my addr 172.21.15.143:0/3551026593 (peer_addr_for_me v1:172.21.15.143:0/0) 2023-11-26T07:22:10.960 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:10.956+0000 7f6dc9ac6640 1 --2- 172.21.15.143:0/3551026593 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f6dc414b1e0 0x7f6dc40cf280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:22:10.960 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:10.956+0000 7f6dc92c5640 1 --2- 172.21.15.143:0/3551026593 >> v2:172.21.15.188:3300/0 conn(0x7f6dc40cf7c0 0x7f6dc40708c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:22:10.960 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:10.956+0000 7f6dc8ac4640 1 -- 172.21.15.143:0/3551026593 <== mon.0 v1:172.21.15.143:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1870384823 0 0) 0x7f6dc40cfc20 con 0x7f6dc4070e00 2023-11-26T07:22:10.960 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:10.956+0000 7f6dc8ac4640 1 -- 172.21.15.143:0/3551026593 --> v1:172.21.15.143:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f6db0003530 con 0x7f6dc4070e00 2023-11-26T07:22:10.960 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:10.956+0000 7f6dc8ac4640 1 -- 172.21.15.143:0/3551026593 <== mon.0 v1:172.21.15.143:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 623138488 0 0) 0x7f6db0003530 con 0x7f6dc4070e00 2023-11-26T07:22:10.960 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:10.956+0000 7f6dc8ac4640 1 -- 172.21.15.143:0/3551026593 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f6dc414b1e0 msgr2=0x7f6dc40cf280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-26T07:22:10.960 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:10.956+0000 7f6dc8ac4640 1 --2- 172.21.15.143:0/3551026593 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f6dc414b1e0 0x7f6dc40cf280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:22:10.960 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:10.956+0000 7f6dc8ac4640 1 -- 172.21.15.143:0/3551026593 >> v2:172.21.15.188:3300/0 conn(0x7f6dc40cf7c0 msgr2=0x7f6dc40708c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-26T07:22:10.961 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:10.956+0000 7f6dc8ac4640 1 --2- 172.21.15.143:0/3551026593 >> v2:172.21.15.188:3300/0 conn(0x7f6dc40cf7c0 0x7f6dc40708c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:22:10.961 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:10.956+0000 7f6dc8ac4640 1 -- 172.21.15.143:0/3551026593 --> v1:172.21.15.143:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6dc414b5c0 con 0x7f6dc4070e00 2023-11-26T07:22:10.961 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:10.956+0000 7f6dc8ac4640 1 -- 172.21.15.143:0/3551026593 <== mon.0 v1:172.21.15.143:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2137882523 0 0) 0x7f6dbc001a00 con 0x7f6dc4070e00 2023-11-26T07:22:10.961 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:10.956+0000 7f6dc8ac4640 1 -- 172.21.15.143:0/3551026593 >> v1:172.21.15.143:6789/0 conn(0x7f6dc4070e00 legacy=0x7f6dc40b1ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:22:10.961 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:10.956+0000 7f6dc8ac4640 1 --2- 172.21.15.143:0/3551026593 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f6db0003980 0x7f6db0003da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:22:10.961 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:10.956+0000 7f6dc8ac4640 1 --2- 172.21.15.143:0/3551026593 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f6dc414b1e0 0x7f6db00047d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:22:10.961 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:10.956+0000 7f6dc8ac4640 1 --2- 172.21.15.143:0/3551026593 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f6db0004d10 0x7f6db0005190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:22:10.961 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:10.956+0000 7f6dc8ac4640 1 -- 172.21.15.143:0/3551026593 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6dc4162080 con 0x7f6db0004d10 2023-11-26T07:22:10.961 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:10.956+0000 7f6dc8ac4640 1 -- 172.21.15.143:0/3551026593 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6dc4161d70 con 0x7f6db0003980 2023-11-26T07:22:10.961 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:10.956+0000 7f6dc8ac4640 1 -- 172.21.15.143:0/3551026593 --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6db000b990 con 0x7f6dc414b1e0 2023-11-26T07:22:10.961 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:10.956+0000 7f6dca2c7640 1 --2- 172.21.15.143:0/3551026593 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f6db0004d10 0x7f6db0005190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:22:10.961 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:10.956+0000 7f6dc92c5640 1 --2- 172.21.15.143:0/3551026593 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f6dc414b1e0 0x7f6db00047d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:22:10.962 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:10.956+0000 7f6dc9ac6640 1 --2- 172.21.15.143:0/3551026593 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f6db0003980 0x7f6db0003da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:22:10.962 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:10.956+0000 7f6dca2c7640 1 -- 172.21.15.143:0/3551026593 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f6dc414b1e0 msgr2=0x7f6db00047d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-26T07:22:10.962 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:10.956+0000 7f6dca2c7640 1 --2- 172.21.15.143:0/3551026593 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f6dc414b1e0 0x7f6db00047d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:22:10.962 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:10.956+0000 7f6dca2c7640 1 -- 172.21.15.143:0/3551026593 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f6db0003980 msgr2=0x7f6db0003da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-26T07:22:10.962 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:10.956+0000 7f6dca2c7640 1 --2- 172.21.15.143:0/3551026593 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f6db0003980 0x7f6db0003da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:22:10.962 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:10.956+0000 7f6dca2c7640 1 -- 172.21.15.143:0/3551026593 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6db0003530 con 0x7f6db0004d10 2023-11-26T07:22:10.963 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:10.960+0000 7f6dca2c7640 1 --2- 172.21.15.143:0/3551026593 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f6db0004d10 0x7f6db0005190 secure :-1 s=READY pgs=836 cs=0 l=1 rev1=1 crypto rx=0x7f6dbc00a830 tx=0x7f6dbc01d360 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ea0baca9a0be67b4 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:22:10.963 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:10.960+0000 7f6dc8ac4640 1 -- 172.21.15.143:0/3551026593 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6dbc0ad630 con 0x7f6db0004d10 2023-11-26T07:22:10.963 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:10.960+0000 7f6dc8ac4640 1 -- 172.21.15.143:0/3551026593 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6dbc0a8070 con 0x7f6db0004d10 2023-11-26T07:22:10.963 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:10.960+0000 7f6dc8ac4640 1 -- 172.21.15.143:0/3551026593 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6dbc0ad9f0 con 0x7f6db0004d10 2023-11-26T07:22:10.963 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:10.960+0000 7f6dcbd2a640 1 -- 172.21.15.143:0/3551026593 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f6db0004d10 msgr2=0x7f6db0005190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:22:10.963 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:10.960+0000 7f6dcbd2a640 1 --2- 172.21.15.143:0/3551026593 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f6db0004d10 0x7f6db0005190 secure :-1 s=READY pgs=836 cs=0 l=1 rev1=1 crypto rx=0x7f6dbc00a830 tx=0x7f6dbc01d360 comp rx=0 tx=0).stop 2023-11-26T07:22:10.964 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:10.960+0000 7f6dca2c7640 1 -- 172.21.15.143:0/3551026593 reap_dead start 2023-11-26T07:22:10.964 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:10.960+0000 7f6dcbd2a640 1 -- 172.21.15.143:0/3551026593 shutdown_connections 2023-11-26T07:22:10.964 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:10.960+0000 7f6dcbd2a640 1 -- 172.21.15.143:0/3551026593 >> 172.21.15.143:0/3551026593 conn(0x7f6dc406a890 msgr2=0x7f6dc406aca0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:22:10.964 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:10.960+0000 7f6dcbd2a640 1 -- 172.21.15.143:0/3551026593 shutdown_connections 2023-11-26T07:22:10.964 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:10.960+0000 7f6dcbd2a640 1 -- 172.21.15.143:0/3551026593 wait complete. 2023-11-26T07:22:10.965 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:10.964+0000 7f6dcbd2a640 1 Processor -- start 2023-11-26T07:22:10.965 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:10.964+0000 7f6dcbd2a640 1 -- start start 2023-11-26T07:22:10.967 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:10.964+0000 7f6dcbd2a640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f6dc41463b0 0x7f6dc4144b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:22:10.967 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:10.964+0000 7f6dcbd2a640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f6dc4143b10 0x7f6dc415e580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:22:10.967 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:10.964+0000 7f6dcbd2a640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f6dc415f1c0 0x7f6dc415eac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:22:10.967 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:10.964+0000 7f6dcbd2a640 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6dc415f000 con 0x7f6dc41463b0 2023-11-26T07:22:10.967 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:10.964+0000 7f6dcbd2a640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6dc4142340 con 0x7f6dc4143b10 2023-11-26T07:22:10.967 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:10.964+0000 7f6dcbd2a640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6dc41457c0 con 0x7f6dc415f1c0 2023-11-26T07:22:10.967 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:10.964+0000 7f6dc9ac6640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f6dc41463b0 0x7f6dc4144b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:22:10.967 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:10.964+0000 7f6dc9ac6640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f6dc41463b0 0x7f6dc4144b90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:35340/0 (socket says 172.21.15.143:35340) 2023-11-26T07:22:10.967 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:10.964+0000 7f6dc9ac6640 1 -- 172.21.15.143:0/1369361994 learned_addr learned my addr 172.21.15.143:0/1369361994 (peer_addr_for_me v2:172.21.15.143:0/0) 2023-11-26T07:22:10.967 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:10.964+0000 7f6dc9ac6640 1 -- 172.21.15.143:0/1369361994 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f6dc415f1c0 msgr2=0x7f6dc415eac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:22:10.967 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:10.964+0000 7f6dc9ac6640 1 --2- 172.21.15.143:0/1369361994 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f6dc415f1c0 0x7f6dc415eac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:22:10.967 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:10.964+0000 7f6dc9ac6640 1 -- 172.21.15.143:0/1369361994 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f6dc4143b10 msgr2=0x7f6dc415e580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:22:10.968 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:10.964+0000 7f6dc9ac6640 1 --2- 172.21.15.143:0/1369361994 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f6dc4143b10 0x7f6dc415e580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:22:10.968 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:10.964+0000 7f6dc9ac6640 1 -- 172.21.15.143:0/1369361994 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6dc41454e0 con 0x7f6dc41463b0 2023-11-26T07:22:10.968 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:10.964+0000 7f6dc9ac6640 1 --2- 172.21.15.143:0/1369361994 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f6dc41463b0 0x7f6dc4144b90 secure :-1 s=READY pgs=837 cs=0 l=1 rev1=1 crypto rx=0x7f6dc00119d0 tx=0x7f6dc0011e90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:22:10.968 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:10.964+0000 7f6dba7fc640 1 -- 172.21.15.143:0/1369361994 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6dc000fad0 con 0x7f6dc41463b0 2023-11-26T07:22:10.969 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:10.964+0000 7f6dba7fc640 1 -- 172.21.15.143:0/1369361994 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6dc000fc30 con 0x7f6dc41463b0 2023-11-26T07:22:10.969 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:10.964+0000 7f6dba7fc640 1 -- 172.21.15.143:0/1369361994 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6dc0014440 con 0x7f6dc41463b0 2023-11-26T07:22:10.969 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:10.964+0000 7f6dcbd2a640 1 -- 172.21.15.143:0/1369361994 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6dc4141cf0 con 0x7f6dc41463b0 2023-11-26T07:22:10.969 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:10.964+0000 7f6dcbd2a640 1 -- 172.21.15.143:0/1369361994 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6dc41450d0 con 0x7f6dc41463b0 2023-11-26T07:22:10.969 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:10.968+0000 7f6dcbd2a640 1 -- 172.21.15.143:0/1369361994 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6dc4067680 con 0x7f6dc41463b0 2023-11-26T07:22:10.969 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:10.968+0000 7f6dba7fc640 1 -- 172.21.15.143:0/1369361994 <== mon.0 v2:172.21.15.143:3300/0 4 ==== mgrmap(e 10) v1 ==== 86056+0+0 (secure 0 0 0) 0x7f6dc00145a0 con 0x7f6dc41463b0 2023-11-26T07:22:10.970 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:10.968+0000 7f6dba7fc640 1 --2- 172.21.15.143:0/1369361994 >> [v2:172.21.15.143:6832/2150557544,v1:172.21.15.143:6833/2150557544] conn(0x7f6da80666b0 0x7f6da8068b70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:22:10.970 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:10.968+0000 7f6dba7fc640 1 -- 172.21.15.143:0/1369361994 <== mon.0 v2:172.21.15.143:3300/0 5 ==== osd_map(59..59 src has 1..59) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f6dc0084b80 con 0x7f6dc41463b0 2023-11-26T07:22:10.970 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:10.968+0000 7f6dc92c5640 1 --2- 172.21.15.143:0/1369361994 >> [v2:172.21.15.143:6832/2150557544,v1:172.21.15.143:6833/2150557544] conn(0x7f6da80666b0 0x7f6da8068b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:22:10.970 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:10.968+0000 7f6dc92c5640 1 --2- 172.21.15.143:0/1369361994 >> [v2:172.21.15.143:6832/2150557544,v1:172.21.15.143:6833/2150557544] conn(0x7f6da80666b0 0x7f6da8068b70 secure :-1 s=READY pgs=396 cs=0 l=1 rev1=1 crypto rx=0x7f6db4006e90 tx=0x7f6db4006e00 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:22:10.974 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:10.972+0000 7f6dba7fc640 1 -- 172.21.15.143:0/1369361994 <== mon.0 v2:172.21.15.143:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172649 (secure 0 0 0) 0x7f6dc0051d50 con 0x7f6dc41463b0 2023-11-26T07:22:11.148 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:11.144+0000 7f6dcbd2a640 1 -- 172.21.15.143:0/1369361994 --> [v2:172.21.15.143:6832/2150557544,v1:172.21.15.143:6833/2150557544] -- mgr_command(tid 0: {"prefix": "fs subvolume getpath", "vol_name": "cephfs", "sub_name": "subvol970", "target": ["mon-mgr", ""]}) v1 -- 0x7f6dc4140a70 con 0x7f6da80666b0 2023-11-26T07:22:11.152 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:11.152+0000 7f6dba7fc640 1 -- 172.21.15.143:0/1369361994 <== mgr.4100 v2:172.21.15.143:6832/2150557544 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+28 (secure 0 0 0) 0x7f6dc4140a70 con 0x7f6da80666b0 2023-11-26T07:22:11.152 INFO:teuthology.orchestra.run.smithi143.stdout:/volumes/_nogroup/subvol970 2023-11-26T07:22:11.154 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:11.152+0000 7f6dcbd2a640 1 -- 172.21.15.143:0/1369361994 >> [v2:172.21.15.143:6832/2150557544,v1:172.21.15.143:6833/2150557544] conn(0x7f6da80666b0 msgr2=0x7f6da8068b70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:22:11.154 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:11.152+0000 7f6dcbd2a640 1 --2- 172.21.15.143:0/1369361994 >> [v2:172.21.15.143:6832/2150557544,v1:172.21.15.143:6833/2150557544] conn(0x7f6da80666b0 0x7f6da8068b70 secure :-1 s=READY pgs=396 cs=0 l=1 rev1=1 crypto rx=0x7f6db4006e90 tx=0x7f6db4006e00 comp rx=0 tx=0).stop 2023-11-26T07:22:11.154 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:11.152+0000 7f6dcbd2a640 1 -- 172.21.15.143:0/1369361994 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f6dc41463b0 msgr2=0x7f6dc4144b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:22:11.154 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:11.152+0000 7f6dcbd2a640 1 --2- 172.21.15.143:0/1369361994 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f6dc41463b0 0x7f6dc4144b90 secure :-1 s=READY pgs=837 cs=0 l=1 rev1=1 crypto rx=0x7f6dc00119d0 tx=0x7f6dc0011e90 comp rx=0 tx=0).stop 2023-11-26T07:22:11.155 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:11.152+0000 7f6dcbd2a640 1 -- 172.21.15.143:0/1369361994 shutdown_connections 2023-11-26T07:22:11.155 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:11.152+0000 7f6dcbd2a640 1 --2- 172.21.15.143:0/1369361994 >> [v2:172.21.15.143:6832/2150557544,v1:172.21.15.143:6833/2150557544] conn(0x7f6da80666b0 0x7f6da8068b70 unknown :-1 s=CLOSED pgs=396 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:22:11.155 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:11.152+0000 7f6dcbd2a640 1 --2- 172.21.15.143:0/1369361994 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f6dc415f1c0 0x7f6dc415eac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:22:11.155 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:11.152+0000 7f6dcbd2a640 1 --2- 172.21.15.143:0/1369361994 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f6dc4143b10 0x7f6dc415e580 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:22:11.155 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:11.152+0000 7f6dcbd2a640 1 --2- 172.21.15.143:0/1369361994 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f6dc41463b0 0x7f6dc4144b90 unknown :-1 s=CLOSED pgs=837 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:22:11.155 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:11.152+0000 7f6dcbd2a640 1 -- 172.21.15.143:0/1369361994 >> 172.21.15.143:0/1369361994 conn(0x7f6dc406a890 msgr2=0x7f6dc40af270 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:22:11.155 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:11.152+0000 7f6dcbd2a640 1 -- 172.21.15.143:0/1369361994 shutdown_connections 2023-11-26T07:22:11.155 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:11.152+0000 7f6dcbd2a640 1 -- 172.21.15.143:0/1369361994 wait complete. 2023-11-26T07:22:11.173 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-26T07:22:11.173 DEBUG:teuthology.orchestra.run.smithi143:> (cd /home/ubuntu/cephtest/mnt.0 && exec sudo cat ./volumes/_legacy/61d0d3156b618c00fb4e550230ec1179.meta) 2023-11-26T07:22:15.415 INFO:teuthology.orchestra.run.smithi143.stdout:[GLOBAL] 2023-11-26T07:22:15.416 INFO:teuthology.orchestra.run.smithi143.stdout:version = 1 2023-11-26T07:22:15.416 INFO:teuthology.orchestra.run.smithi143.stdout:type = subvolume 2023-11-26T07:22:15.416 INFO:teuthology.orchestra.run.smithi143.stdout:path = /volumes/_nogroup/subvol970 2023-11-26T07:22:15.416 INFO:teuthology.orchestra.run.smithi143.stdout:state = complete 2023-11-26T07:22:15.416 INFO:teuthology.orchestra.run.smithi143.stdout: 2023-11-26T07:22:15.419 DEBUG:teuthology.orchestra.run.smithi143:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume authorize cephfs subvol970 alice 2023-11-26T07:22:15.569 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:15.568+0000 7fb60cb8d640 1 Processor -- start 2023-11-26T07:22:15.570 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:15.568+0000 7fb60cb8d640 1 -- start start 2023-11-26T07:22:15.571 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:15.568+0000 7fb60cb8d640 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7fb608143ac0 0x7fb608070370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:22:15.571 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:15.568+0000 7fb60cb8d640 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7fb6080708b0 0x7fb6080cf280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:22:15.571 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:15.568+0000 7fb60cb8d640 1 -- --> v1:172.21.15.143:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb6080cc730 con 0x7fb6080cf7c0 2023-11-26T07:22:15.571 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:15.568+0000 7fb60cb8d640 1 -- --> v2:172.21.15.143:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb608143ea0 con 0x7fb6080708b0 2023-11-26T07:22:15.571 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:15.568+0000 7fb60cb8d640 1 -- --> v2:172.21.15.188:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb6080cfbd0 con 0x7fb608143ac0 2023-11-26T07:22:15.571 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:15.568+0000 7fb606d9d640 1 --1- >> v1:172.21.15.143:6789/0 conn(0x7fb6080cf7c0 0x7fb6080b1ef0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.143:6789/0 says I am v1:172.21.15.143:59114/0 (socket says 172.21.15.143:59114) 2023-11-26T07:22:15.572 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:15.568+0000 7fb606d9d640 1 -- 172.21.15.143:0/2046588783 learned_addr learned my addr 172.21.15.143:0/2046588783 (peer_addr_for_me v1:172.21.15.143:0/0) 2023-11-26T07:22:15.572 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:15.568+0000 7fb5fffff640 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7fb6080708b0 0x7fb6080cf280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:22:15.572 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:15.568+0000 7fb60659c640 1 --2- 172.21.15.143:0/2046588783 >> v2:172.21.15.188:3300/0 conn(0x7fb608143ac0 0x7fb608070370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:22:15.572 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:15.568+0000 7fb5fffff640 1 -- 172.21.15.143:0/2046588783 >> v2:172.21.15.188:3300/0 conn(0x7fb608143ac0 msgr2=0x7fb608070370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-26T07:22:15.572 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:15.568+0000 7fb5fffff640 1 --2- 172.21.15.143:0/2046588783 >> v2:172.21.15.188:3300/0 conn(0x7fb608143ac0 0x7fb608070370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:22:15.572 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:15.568+0000 7fb5fffff640 1 -- 172.21.15.143:0/2046588783 >> v1:172.21.15.143:6789/0 conn(0x7fb6080cf7c0 legacy=0x7fb6080b1ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:22:15.573 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:15.568+0000 7fb5fffff640 1 -- 172.21.15.143:0/2046588783 --> v2:172.21.15.143:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb608070ee0 con 0x7fb6080708b0 2023-11-26T07:22:15.573 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:15.568+0000 7fb605d9b640 1 -- 172.21.15.143:0/2046588783 <== mon.0 v1:172.21.15.143:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 785539605 0 0) 0x7fb6080cc730 con 0x7fb6080cf7c0 2023-11-26T07:22:15.574 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:15.572+0000 7fb5fffff640 1 --2- 172.21.15.143:0/2046588783 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fb6080708b0 0x7fb6080cf280 secure :-1 s=READY pgs=839 cs=0 l=1 rev1=1 crypto rx=0x7fb60806dca0 tx=0x7fb5f001c460 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b7cf3229a892c7e0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:22:15.574 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:15.572+0000 7fb605d9b640 1 -- 172.21.15.143:0/2046588783 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb5f00ab660 con 0x7fb6080708b0 2023-11-26T07:22:15.575 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:15.572+0000 7fb605d9b640 1 -- 172.21.15.143:0/2046588783 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb5f00ab7c0 con 0x7fb6080708b0 2023-11-26T07:22:15.575 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:15.572+0000 7fb605d9b640 1 -- 172.21.15.143:0/2046588783 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb5f00abb50 con 0x7fb6080708b0 2023-11-26T07:22:15.576 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:15.572+0000 7fb60cb8d640 1 -- 172.21.15.143:0/2046588783 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fb6080708b0 msgr2=0x7fb6080cf280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:22:15.576 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:15.572+0000 7fb60cb8d640 1 --2- 172.21.15.143:0/2046588783 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fb6080708b0 0x7fb6080cf280 secure :-1 s=READY pgs=839 cs=0 l=1 rev1=1 crypto rx=0x7fb60806dca0 tx=0x7fb5f001c460 comp rx=0 tx=0).stop 2023-11-26T07:22:15.576 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:15.572+0000 7fb60cb8d640 1 -- 172.21.15.143:0/2046588783 shutdown_connections 2023-11-26T07:22:15.576 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:15.572+0000 7fb60cb8d640 1 --2- 172.21.15.143:0/2046588783 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fb6080708b0 0x7fb6080cf280 unknown :-1 s=CLOSED pgs=839 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:22:15.576 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:15.572+0000 7fb60cb8d640 1 --2- 172.21.15.143:0/2046588783 >> v2:172.21.15.188:3300/0 conn(0x7fb608143ac0 0x7fb608070370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:22:15.576 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:15.572+0000 7fb60cb8d640 1 -- 172.21.15.143:0/2046588783 >> 172.21.15.143:0/2046588783 conn(0x7fb60806a890 msgr2=0x7fb60806aca0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:22:15.576 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:15.572+0000 7fb60cb8d640 1 -- 172.21.15.143:0/2046588783 shutdown_connections 2023-11-26T07:22:15.576 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:15.572+0000 7fb60cb8d640 1 -- 172.21.15.143:0/2046588783 wait complete. 2023-11-26T07:22:15.577 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:15.576+0000 7fb60cb8d640 1 Processor -- start 2023-11-26T07:22:15.577 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:15.576+0000 7fb60cb8d640 1 -- start start 2023-11-26T07:22:15.580 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:15.576+0000 7fb60cb8d640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fb6080708b0 0x7fb6081575f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:22:15.580 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:15.576+0000 7fb60cb8d640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fb6080cf7c0 0x7fb608158550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:22:15.580 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:15.576+0000 7fb60cb8d640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fb608143ac0 0x7fb608158a90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:22:15.580 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:15.576+0000 7fb60cb8d640 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb608165310 con 0x7fb6080cf7c0 2023-11-26T07:22:15.580 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:15.576+0000 7fb60cb8d640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb608165560 con 0x7fb6080708b0 2023-11-26T07:22:15.580 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:15.576+0000 7fb60cb8d640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb608165840 con 0x7fb608143ac0 2023-11-26T07:22:15.580 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:15.576+0000 7fb5fffff640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fb6080cf7c0 0x7fb608158550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:22:15.581 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:15.576+0000 7fb5fffff640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fb6080cf7c0 0x7fb608158550 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:34066/0 (socket says 172.21.15.143:34066) 2023-11-26T07:22:15.581 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:15.576+0000 7fb5fffff640 1 -- 172.21.15.143:0/527903043 learned_addr learned my addr 172.21.15.143:0/527903043 (peer_addr_for_me v2:172.21.15.143:0/0) 2023-11-26T07:22:15.581 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:15.576+0000 7fb60659c640 1 --2- 172.21.15.143:0/527903043 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fb6080708b0 0x7fb6081575f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:22:15.581 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:15.576+0000 7fb606d9d640 1 --2- 172.21.15.143:0/527903043 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fb608143ac0 0x7fb608158a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:22:15.581 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:15.576+0000 7fb5fffff640 1 -- 172.21.15.143:0/527903043 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fb608143ac0 msgr2=0x7fb608158a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:22:15.581 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:15.576+0000 7fb5fffff640 1 --2- 172.21.15.143:0/527903043 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fb608143ac0 0x7fb608158a90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:22:15.581 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:15.576+0000 7fb5fffff640 1 -- 172.21.15.143:0/527903043 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fb6080708b0 msgr2=0x7fb6081575f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:22:15.581 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:15.576+0000 7fb5fffff640 1 --2- 172.21.15.143:0/527903043 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fb6080708b0 0x7fb6081575f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:22:15.581 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:15.576+0000 7fb5fffff640 1 -- 172.21.15.143:0/527903043 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb5f0009420 con 0x7fb6080cf7c0 2023-11-26T07:22:15.581 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:15.576+0000 7fb5fffff640 1 --2- 172.21.15.143:0/527903043 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fb6080cf7c0 0x7fb608158550 secure :-1 s=READY pgs=840 cs=0 l=1 rev1=1 crypto rx=0x7fb5f0019070 tx=0x7fb5f001aca0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:22:15.581 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:15.576+0000 7fb5feffd640 1 -- 172.21.15.143:0/527903043 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb5f00b5810 con 0x7fb6080cf7c0 2023-11-26T07:22:15.582 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:15.576+0000 7fb5feffd640 1 -- 172.21.15.143:0/527903043 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb5f00b5990 con 0x7fb6080cf7c0 2023-11-26T07:22:15.582 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:15.576+0000 7fb5feffd640 1 -- 172.21.15.143:0/527903043 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb5f00b5d40 con 0x7fb6080cf7c0 2023-11-26T07:22:15.582 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:15.576+0000 7fb60cb8d640 1 -- 172.21.15.143:0/527903043 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb608165b20 con 0x7fb6080cf7c0 2023-11-26T07:22:15.582 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:15.580+0000 7fb60cb8d640 1 -- 172.21.15.143:0/527903043 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb608157b30 con 0x7fb6080cf7c0 2023-11-26T07:22:15.582 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:15.580+0000 7fb60cb8d640 1 -- 172.21.15.143:0/527903043 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb608157de0 con 0x7fb6080cf7c0 2023-11-26T07:22:15.583 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:15.580+0000 7fb5feffd640 1 -- 172.21.15.143:0/527903043 <== mon.0 v2:172.21.15.143:3300/0 4 ==== mgrmap(e 10) v1 ==== 86056+0+0 (secure 0 0 0) 0x7fb5f00a9070 con 0x7fb6080cf7c0 2023-11-26T07:22:15.583 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:15.580+0000 7fb5feffd640 1 --2- 172.21.15.143:0/527903043 >> [v2:172.21.15.143:6832/2150557544,v1:172.21.15.143:6833/2150557544] conn(0x7fb5e80666b0 0x7fb5e8068b70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:22:15.583 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:15.580+0000 7fb5feffd640 1 -- 172.21.15.143:0/527903043 <== mon.0 v2:172.21.15.143:3300/0 5 ==== osd_map(59..59 src has 1..59) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fb5f011ea70 con 0x7fb6080cf7c0 2023-11-26T07:22:15.583 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:15.580+0000 7fb60659c640 1 --2- 172.21.15.143:0/527903043 >> [v2:172.21.15.143:6832/2150557544,v1:172.21.15.143:6833/2150557544] conn(0x7fb5e80666b0 0x7fb5e8068b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:22:15.584 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:15.584+0000 7fb60659c640 1 --2- 172.21.15.143:0/527903043 >> [v2:172.21.15.143:6832/2150557544,v1:172.21.15.143:6833/2150557544] conn(0x7fb5e80666b0 0x7fb5e8068b70 secure :-1 s=READY pgs=397 cs=0 l=1 rev1=1 crypto rx=0x7fb5f80079c0 tx=0x7fb5f8008040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:22:15.587 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:15.584+0000 7fb5feffd640 1 -- 172.21.15.143:0/527903043 <== mon.0 v2:172.21.15.143:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172649 (secure 0 0 0) 0x7fb5f00ebbb0 con 0x7fb6080cf7c0 2023-11-26T07:22:15.760 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:15.760+0000 7fb60cb8d640 1 -- 172.21.15.143:0/527903043 --> [v2:172.21.15.143:6832/2150557544,v1:172.21.15.143:6833/2150557544] -- mgr_command(tid 0: {"prefix": "fs subvolume authorize", "vol_name": "cephfs", "sub_name": "subvol970", "auth_id": "alice", "target": ["mon-mgr", ""]}) v1 -- 0x7fb608071c30 con 0x7fb5e80666b0 2023-11-26T07:22:15.771 INFO:tasks.ceph.mgr.y.smithi143.stderr:Creating meta for ID alice with tenant None 2023-11-26T07:22:15.811 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:15.808+0000 7fb5feffd640 1 -- 172.21.15.143:0/527903043 <== mgr.4100 v2:172.21.15.143:6832/2150557544 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+0 (secure 0 0 0) 0x7fb5f80108d0 con 0x7fb5e80666b0 2023-11-26T07:22:15.813 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:15.812+0000 7fb60cb8d640 1 -- 172.21.15.143:0/527903043 >> [v2:172.21.15.143:6832/2150557544,v1:172.21.15.143:6833/2150557544] conn(0x7fb5e80666b0 msgr2=0x7fb5e8068b70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:22:15.814 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:15.812+0000 7fb60cb8d640 1 --2- 172.21.15.143:0/527903043 >> [v2:172.21.15.143:6832/2150557544,v1:172.21.15.143:6833/2150557544] conn(0x7fb5e80666b0 0x7fb5e8068b70 secure :-1 s=READY pgs=397 cs=0 l=1 rev1=1 crypto rx=0x7fb5f80079c0 tx=0x7fb5f8008040 comp rx=0 tx=0).stop 2023-11-26T07:22:15.814 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:15.812+0000 7fb60cb8d640 1 -- 172.21.15.143:0/527903043 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fb6080cf7c0 msgr2=0x7fb608158550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:22:15.814 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:15.812+0000 7fb60cb8d640 1 --2- 172.21.15.143:0/527903043 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fb6080cf7c0 0x7fb608158550 secure :-1 s=READY pgs=840 cs=0 l=1 rev1=1 crypto rx=0x7fb5f0019070 tx=0x7fb5f001aca0 comp rx=0 tx=0).stop 2023-11-26T07:22:15.814 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:15.812+0000 7fb60cb8d640 1 -- 172.21.15.143:0/527903043 shutdown_connections 2023-11-26T07:22:15.814 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:15.812+0000 7fb60cb8d640 1 --2- 172.21.15.143:0/527903043 >> [v2:172.21.15.143:6832/2150557544,v1:172.21.15.143:6833/2150557544] conn(0x7fb5e80666b0 0x7fb5e8068b70 unknown :-1 s=CLOSED pgs=397 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:22:15.814 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:15.812+0000 7fb60cb8d640 1 --2- 172.21.15.143:0/527903043 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fb608143ac0 0x7fb608158a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:22:15.814 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:15.812+0000 7fb60cb8d640 1 --2- 172.21.15.143:0/527903043 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fb6080cf7c0 0x7fb608158550 unknown :-1 s=CLOSED pgs=840 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:22:15.814 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:15.812+0000 7fb60cb8d640 1 --2- 172.21.15.143:0/527903043 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fb6080708b0 0x7fb6081575f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:22:15.815 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:15.812+0000 7fb60cb8d640 1 -- 172.21.15.143:0/527903043 >> 172.21.15.143:0/527903043 conn(0x7fb60806a890 msgr2=0x7fb6080acc50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:22:15.815 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:15.812+0000 7fb60cb8d640 1 -- 172.21.15.143:0/527903043 shutdown_connections 2023-11-26T07:22:15.815 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:15.812+0000 7fb60cb8d640 1 -- 172.21.15.143:0/527903043 wait complete. 2023-11-26T07:22:15.832 DEBUG:teuthology.orchestra.run.smithi143:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth get client.alice --format=json-pretty 2023-11-26T07:22:15.970 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:15.968+0000 7ffb243f8640 1 Processor -- start 2023-11-26T07:22:15.970 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:15.968+0000 7ffb243f8640 1 -- start start 2023-11-26T07:22:15.971 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:15.968+0000 7ffb243f8640 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7ffb1c143ac0 0x7ffb1c070370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:22:15.971 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:15.968+0000 7ffb243f8640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7ffb1c0708b0 0x7ffb1c0cf280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:22:15.971 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:15.968+0000 7ffb243f8640 1 -- --> v1:172.21.15.188:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ffb1c0cfbf0 con 0x7ffb1c0cf7c0 2023-11-26T07:22:15.971 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:15.968+0000 7ffb243f8640 1 -- --> v2:172.21.15.188:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ffb1c143ea0 con 0x7ffb1c143ac0 2023-11-26T07:22:15.971 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:15.968+0000 7ffb243f8640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ffb1c0cc730 con 0x7ffb1c0708b0 2023-11-26T07:22:15.972 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:15.968+0000 7ffb21993640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7ffb1c0708b0 0x7ffb1c0cf280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:22:15.972 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:15.968+0000 7ffb22194640 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7ffb1c143ac0 0x7ffb1c070370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:22:15.972 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:15.968+0000 7ffb22995640 1 --1- >> v1:172.21.15.188:6789/0 conn(0x7ffb1c0cf7c0 0x7ffb1c0b1ef0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.188:6789/0 says I am v1:172.21.15.143:49030/0 (socket says 172.21.15.143:49030) 2023-11-26T07:22:15.972 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:15.972+0000 7ffb22995640 1 -- 172.21.15.143:0/53630672 learned_addr learned my addr 172.21.15.143:0/53630672 (peer_addr_for_me v1:172.21.15.143:0/0) 2023-11-26T07:22:15.972 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:15.972+0000 7ffb22194640 1 -- 172.21.15.143:0/53630672 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7ffb1c0708b0 msgr2=0x7ffb1c0cf280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-26T07:22:15.973 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:15.972+0000 7ffb22194640 1 --2- 172.21.15.143:0/53630672 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7ffb1c0708b0 0x7ffb1c0cf280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:22:15.973 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:15.972+0000 7ffb22194640 1 -- 172.21.15.143:0/53630672 >> v1:172.21.15.188:6789/0 conn(0x7ffb1c0cf7c0 legacy=0x7ffb1c0b1ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:22:15.973 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:15.972+0000 7ffb22194640 1 -- 172.21.15.143:0/53630672 --> v2:172.21.15.188:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ffb1c070ee0 con 0x7ffb1c143ac0 2023-11-26T07:22:15.975 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:15.972+0000 7ffb22194640 1 --2- 172.21.15.143:0/53630672 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7ffb1c143ac0 0x7ffb1c070370 secure :-1 s=READY pgs=333 cs=0 l=1 rev1=1 crypto rx=0x7ffb180097c0 tx=0x7ffb1801c480 comp rx=0 tx=0).ready entity=mon.1 client_cookie=3e696519f936db6 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:22:15.975 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:15.972+0000 7ffb21192640 1 -- 172.21.15.143:0/53630672 <== mon.1 v2:172.21.15.188:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffb180ad640 con 0x7ffb1c143ac0 2023-11-26T07:22:15.975 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:15.972+0000 7ffb21192640 1 -- 172.21.15.143:0/53630672 <== mon.1 v2:172.21.15.188:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ffb180ad7a0 con 0x7ffb1c143ac0 2023-11-26T07:22:15.975 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:15.972+0000 7ffb21192640 1 -- 172.21.15.143:0/53630672 <== mon.1 v2:172.21.15.188:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffb180adb30 con 0x7ffb1c143ac0 2023-11-26T07:22:15.976 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:15.976+0000 7ffb243f8640 1 -- 172.21.15.143:0/53630672 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7ffb1c143ac0 msgr2=0x7ffb1c070370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:22:15.976 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:15.976+0000 7ffb243f8640 1 --2- 172.21.15.143:0/53630672 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7ffb1c143ac0 0x7ffb1c070370 secure :-1 s=READY pgs=333 cs=0 l=1 rev1=1 crypto rx=0x7ffb180097c0 tx=0x7ffb1801c480 comp rx=0 tx=0).stop 2023-11-26T07:22:15.976 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:15.976+0000 7ffb243f8640 1 -- 172.21.15.143:0/53630672 shutdown_connections 2023-11-26T07:22:15.976 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:15.976+0000 7ffb243f8640 1 --2- 172.21.15.143:0/53630672 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7ffb1c0708b0 0x7ffb1c0cf280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:22:15.977 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:15.976+0000 7ffb243f8640 1 --2- 172.21.15.143:0/53630672 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7ffb1c143ac0 0x7ffb1c070370 unknown :-1 s=CLOSED pgs=333 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:22:15.977 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:15.976+0000 7ffb243f8640 1 -- 172.21.15.143:0/53630672 >> 172.21.15.143:0/53630672 conn(0x7ffb1c06a890 msgr2=0x7ffb1c06aca0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:22:15.977 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:15.976+0000 7ffb243f8640 1 -- 172.21.15.143:0/53630672 shutdown_connections 2023-11-26T07:22:15.977 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:15.976+0000 7ffb243f8640 1 -- 172.21.15.143:0/53630672 wait complete. 2023-11-26T07:22:15.978 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:15.976+0000 7ffb243f8640 1 Processor -- start 2023-11-26T07:22:15.978 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:15.976+0000 7ffb243f8640 1 -- start start 2023-11-26T07:22:15.980 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:15.976+0000 7ffb243f8640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7ffb1c0708b0 0x7ffb1c0b12e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:22:15.980 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:15.976+0000 7ffb243f8640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7ffb1c0cf7c0 0x7ffb1c1593a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:22:15.980 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:15.976+0000 7ffb243f8640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7ffb1c143ac0 0x7ffb1c0af520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:22:15.980 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:15.976+0000 7ffb243f8640 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffb1c0b0ec0 con 0x7ffb1c0708b0 2023-11-26T07:22:15.980 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:15.976+0000 7ffb243f8640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffb1c0b02d0 con 0x7ffb1c0cf7c0 2023-11-26T07:22:15.980 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:15.976+0000 7ffb243f8640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ffb1c0b0520 con 0x7ffb1c143ac0 2023-11-26T07:22:15.980 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:15.976+0000 7ffb22194640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7ffb1c0708b0 0x7ffb1c0b12e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:22:15.980 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:15.976+0000 7ffb22194640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7ffb1c0708b0 0x7ffb1c0b12e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:34074/0 (socket says 172.21.15.143:34074) 2023-11-26T07:22:15.981 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:15.976+0000 7ffb22194640 1 -- 172.21.15.143:0/897797501 learned_addr learned my addr 172.21.15.143:0/897797501 (peer_addr_for_me v2:172.21.15.143:0/0) 2023-11-26T07:22:15.981 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:15.976+0000 7ffb21993640 1 --2- 172.21.15.143:0/897797501 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7ffb1c0cf7c0 0x7ffb1c1593a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:22:15.981 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:15.976+0000 7ffb22995640 1 --2- 172.21.15.143:0/897797501 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7ffb1c143ac0 0x7ffb1c0af520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:22:15.981 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:15.976+0000 7ffb22194640 1 -- 172.21.15.143:0/897797501 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7ffb1c143ac0 msgr2=0x7ffb1c0af520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:22:15.981 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:15.976+0000 7ffb22194640 1 --2- 172.21.15.143:0/897797501 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7ffb1c143ac0 0x7ffb1c0af520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:22:15.981 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:15.976+0000 7ffb22194640 1 -- 172.21.15.143:0/897797501 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7ffb1c0cf7c0 msgr2=0x7ffb1c1593a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:22:15.981 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:15.976+0000 7ffb22194640 1 --2- 172.21.15.143:0/897797501 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7ffb1c0cf7c0 0x7ffb1c1593a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:22:15.981 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:15.976+0000 7ffb22194640 1 -- 172.21.15.143:0/897797501 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ffb18009420 con 0x7ffb1c0708b0 2023-11-26T07:22:15.981 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:15.980+0000 7ffb22194640 1 --2- 172.21.15.143:0/897797501 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7ffb1c0708b0 0x7ffb1c0b12e0 secure :-1 s=READY pgs=841 cs=0 l=1 rev1=1 crypto rx=0x7ffb180097c0 tx=0x7ffb18019220 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:22:15.982 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:15.980+0000 7ffb12ffd640 1 -- 172.21.15.143:0/897797501 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffb180b6610 con 0x7ffb1c0708b0 2023-11-26T07:22:15.982 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:15.980+0000 7ffb12ffd640 1 -- 172.21.15.143:0/897797501 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ffb180a6040 con 0x7ffb1c0708b0 2023-11-26T07:22:15.983 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:15.980+0000 7ffb243f8640 1 -- 172.21.15.143:0/897797501 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ffb1c0b0800 con 0x7ffb1c0708b0 2023-11-26T07:22:15.983 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:15.980+0000 7ffb12ffd640 1 -- 172.21.15.143:0/897797501 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffb180b6990 con 0x7ffb1c0708b0 2023-11-26T07:22:15.983 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:15.980+0000 7ffb243f8640 1 -- 172.21.15.143:0/897797501 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ffb1c159e40 con 0x7ffb1c0708b0 2023-11-26T07:22:15.983 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:15.980+0000 7ffb243f8640 1 -- 172.21.15.143:0/897797501 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ffb1c0b1000 con 0x7ffb1c0708b0 2023-11-26T07:22:15.983 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:15.980+0000 7ffb12ffd640 1 -- 172.21.15.143:0/897797501 <== mon.0 v2:172.21.15.143:3300/0 4 ==== mgrmap(e 10) v1 ==== 86056+0+0 (secure 0 0 0) 0x7ffb180b4040 con 0x7ffb1c0708b0 2023-11-26T07:22:15.984 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:15.980+0000 7ffb12ffd640 1 --2- 172.21.15.143:0/897797501 >> [v2:172.21.15.143:6832/2150557544,v1:172.21.15.143:6833/2150557544] conn(0x7ffb000666b0 0x7ffb00068b70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:22:15.984 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:15.980+0000 7ffb12ffd640 1 -- 172.21.15.143:0/897797501 <== mon.0 v2:172.21.15.143:3300/0 5 ==== osd_map(59..59 src has 1..59) v4 ==== 5709+0+0 (secure 0 0 0) 0x7ffb1811e470 con 0x7ffb1c0708b0 2023-11-26T07:22:15.984 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:15.980+0000 7ffb21993640 1 --2- 172.21.15.143:0/897797501 >> [v2:172.21.15.143:6832/2150557544,v1:172.21.15.143:6833/2150557544] conn(0x7ffb000666b0 0x7ffb00068b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:22:15.984 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:15.984+0000 7ffb21993640 1 --2- 172.21.15.143:0/897797501 >> [v2:172.21.15.143:6832/2150557544,v1:172.21.15.143:6833/2150557544] conn(0x7ffb000666b0 0x7ffb00068b70 secure :-1 s=READY pgs=398 cs=0 l=1 rev1=1 crypto rx=0x7ffb1c0b0090 tx=0x7ffb0c009190 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:22:15.987 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:15.984+0000 7ffb12ffd640 1 -- 172.21.15.143:0/897797501 <== mon.0 v2:172.21.15.143:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172649 (secure 0 0 0) 0x7ffb1c0b1000 con 0x7ffb1c0708b0 2023-11-26T07:22:16.188 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:16.188+0000 7ffb243f8640 1 -- 172.21.15.143:0/897797501 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "auth get", "entity": "client.alice", "format": "json-pretty"} v 0) v1 -- 0x7ffb1c067680 con 0x7ffb1c0708b0 2023-11-26T07:22:16.188 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:16.188+0000 7ffb12ffd640 1 -- 172.21.15.143:0/897797501 <== mon.0 v2:172.21.15.143:3300/0 7 ==== mon_command_ack([{"prefix": "auth get", "entity": "client.alice", "format": "json-pretty"}]=0 v16) v1 ==== 107+0+278 (secure 0 0 0) 0x7ffb180eb630 con 0x7ffb1c0708b0 2023-11-26T07:22:16.189 INFO:teuthology.orchestra.run.smithi143.stdout: 2023-11-26T07:22:16.189 INFO:teuthology.orchestra.run.smithi143.stdout:[ 2023-11-26T07:22:16.189 INFO:teuthology.orchestra.run.smithi143.stdout: { 2023-11-26T07:22:16.189 INFO:teuthology.orchestra.run.smithi143.stdout: "entity": "client.alice", 2023-11-26T07:22:16.189 INFO:teuthology.orchestra.run.smithi143.stdout: "key": "AQAn8mJlNQDyLhAAMdZWyzLXsQAbpvtzb5EN9w==", 2023-11-26T07:22:16.189 INFO:teuthology.orchestra.run.smithi143.stdout: "caps": { 2023-11-26T07:22:16.189 INFO:teuthology.orchestra.run.smithi143.stdout: "mds": "allow rw path=/volumes/_nogroup/subvol970", 2023-11-26T07:22:16.189 INFO:teuthology.orchestra.run.smithi143.stdout: "mon": "allow r", 2023-11-26T07:22:16.189 INFO:teuthology.orchestra.run.smithi143.stdout: "osd": "allow rw pool=cephfs_data" 2023-11-26T07:22:16.190 INFO:teuthology.orchestra.run.smithi143.stdout: } 2023-11-26T07:22:16.190 INFO:teuthology.orchestra.run.smithi143.stdout: } 2023-11-26T07:22:16.190 INFO:teuthology.orchestra.run.smithi143.stdout:] 2023-11-26T07:22:16.191 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:16.188+0000 7ffb243f8640 1 -- 172.21.15.143:0/897797501 >> [v2:172.21.15.143:6832/2150557544,v1:172.21.15.143:6833/2150557544] conn(0x7ffb000666b0 msgr2=0x7ffb00068b70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:22:16.191 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:16.188+0000 7ffb243f8640 1 --2- 172.21.15.143:0/897797501 >> [v2:172.21.15.143:6832/2150557544,v1:172.21.15.143:6833/2150557544] conn(0x7ffb000666b0 0x7ffb00068b70 secure :-1 s=READY pgs=398 cs=0 l=1 rev1=1 crypto rx=0x7ffb1c0b0090 tx=0x7ffb0c009190 comp rx=0 tx=0).stop 2023-11-26T07:22:16.191 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:16.188+0000 7ffb243f8640 1 -- 172.21.15.143:0/897797501 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7ffb1c0708b0 msgr2=0x7ffb1c0b12e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:22:16.191 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:16.188+0000 7ffb243f8640 1 --2- 172.21.15.143:0/897797501 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7ffb1c0708b0 0x7ffb1c0b12e0 secure :-1 s=READY pgs=841 cs=0 l=1 rev1=1 crypto rx=0x7ffb180097c0 tx=0x7ffb18019220 comp rx=0 tx=0).stop 2023-11-26T07:22:16.191 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:16.188+0000 7ffb243f8640 1 -- 172.21.15.143:0/897797501 shutdown_connections 2023-11-26T07:22:16.191 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:16.188+0000 7ffb243f8640 1 --2- 172.21.15.143:0/897797501 >> [v2:172.21.15.143:6832/2150557544,v1:172.21.15.143:6833/2150557544] conn(0x7ffb000666b0 0x7ffb00068b70 unknown :-1 s=CLOSED pgs=398 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:22:16.191 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:16.188+0000 7ffb243f8640 1 --2- 172.21.15.143:0/897797501 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7ffb1c143ac0 0x7ffb1c0af520 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:22:16.191 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:16.188+0000 7ffb243f8640 1 --2- 172.21.15.143:0/897797501 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7ffb1c0cf7c0 0x7ffb1c1593a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:22:16.191 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:16.188+0000 7ffb243f8640 1 --2- 172.21.15.143:0/897797501 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7ffb1c0708b0 0x7ffb1c0b12e0 unknown :-1 s=CLOSED pgs=841 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:22:16.191 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:16.188+0000 7ffb243f8640 1 -- 172.21.15.143:0/897797501 >> 172.21.15.143:0/897797501 conn(0x7ffb1c06a890 msgr2=0x7ffb1c06ac70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:22:16.192 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:16.188+0000 7ffb243f8640 1 -- 172.21.15.143:0/897797501 shutdown_connections 2023-11-26T07:22:16.192 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:16.188+0000 7ffb243f8640 1 -- 172.21.15.143:0/897797501 wait complete. 2023-11-26T07:22:16.209 DEBUG:teuthology.orchestra.run.smithi143:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume rm cephfs subvol970 2023-11-26T07:22:16.349 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:16.348+0000 7f06c5017640 1 Processor -- start 2023-11-26T07:22:16.350 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:16.348+0000 7f06c5017640 1 -- start start 2023-11-26T07:22:16.352 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:16.348+0000 7f06c5017640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f06c0071b70 0x7f06c006dec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:22:16.352 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:16.348+0000 7f06c5017640 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7f06c015e350 0x7f06c006e400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:22:16.352 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:16.348+0000 7f06c5017640 1 -- --> v1:172.21.15.143:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f06c015aeb0 con 0x7f06c015b340 2023-11-26T07:22:16.352 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:16.348+0000 7f06c5017640 1 -- --> v2:172.21.15.188:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f06c015ebe0 con 0x7f06c015e350 2023-11-26T07:22:16.353 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:16.348+0000 7f06c5017640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f06c015a650 con 0x7f06c0071b70 2023-11-26T07:22:16.353 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:16.348+0000 7f06b659c640 1 --1- >> v1:172.21.15.143:6789/0 conn(0x7f06c015b340 0x7f06c015dc40 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.143:6789/0 says I am v1:172.21.15.143:59116/0 (socket says 172.21.15.143:59116) 2023-11-26T07:22:16.353 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:16.348+0000 7f06b659c640 1 -- 172.21.15.143:0/1682944513 learned_addr learned my addr 172.21.15.143:0/1682944513 (peer_addr_for_me v1:172.21.15.143:0/0) 2023-11-26T07:22:16.353 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:16.348+0000 7f06bed9d640 1 --2- 172.21.15.143:0/1682944513 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f06c0071b70 0x7f06c006dec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:22:16.353 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:16.348+0000 7f06bf59e640 1 --2- 172.21.15.143:0/1682944513 >> v2:172.21.15.188:3300/0 conn(0x7f06c015e350 0x7f06c006e400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:22:16.353 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:16.348+0000 7f06be59c640 1 -- 172.21.15.143:0/1682944513 <== mon.0 v1:172.21.15.143:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1811188056 0 0) 0x7f06c015aeb0 con 0x7f06c015b340 2023-11-26T07:22:16.353 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:16.348+0000 7f06be59c640 1 -- 172.21.15.143:0/1682944513 --> v1:172.21.15.143:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f06ac003530 con 0x7f06c015b340 2023-11-26T07:22:16.353 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:16.348+0000 7f06be59c640 1 -- 172.21.15.143:0/1682944513 <== mon.0 v1:172.21.15.143:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 272096915 0 0) 0x7f06ac003530 con 0x7f06c015b340 2023-11-26T07:22:16.353 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:16.348+0000 7f06be59c640 1 -- 172.21.15.143:0/1682944513 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f06c0071b70 msgr2=0x7f06c006dec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-26T07:22:16.353 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:16.348+0000 7f06be59c640 1 --2- 172.21.15.143:0/1682944513 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f06c0071b70 0x7f06c006dec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:22:16.353 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:16.348+0000 7f06be59c640 1 -- 172.21.15.143:0/1682944513 >> v2:172.21.15.188:3300/0 conn(0x7f06c015e350 msgr2=0x7f06c006e400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-26T07:22:16.353 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:16.348+0000 7f06be59c640 1 --2- 172.21.15.143:0/1682944513 >> v2:172.21.15.188:3300/0 conn(0x7f06c015e350 0x7f06c006e400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:22:16.353 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:16.348+0000 7f06be59c640 1 -- 172.21.15.143:0/1682944513 --> v1:172.21.15.143:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f06c015e900 con 0x7f06c015b340 2023-11-26T07:22:16.354 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:16.352+0000 7f06be59c640 1 -- 172.21.15.143:0/1682944513 <== mon.0 v1:172.21.15.143:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2137882523 0 0) 0x7f06a8001a00 con 0x7f06c015b340 2023-11-26T07:22:16.354 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:16.352+0000 7f06be59c640 1 -- 172.21.15.143:0/1682944513 >> v1:172.21.15.143:6789/0 conn(0x7f06c015b340 legacy=0x7f06c015dc40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:22:16.354 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:16.352+0000 7f06be59c640 1 --2- 172.21.15.143:0/1682944513 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f06ac003980 0x7f06ac003da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:22:16.354 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:16.352+0000 7f06be59c640 1 --2- 172.21.15.143:0/1682944513 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f06ac0047d0 0x7f06ac004c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:22:16.354 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:16.352+0000 7f06be59c640 1 --2- 172.21.15.143:0/1682944513 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f06c0071b70 0x7f06ac005190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:22:16.354 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:16.352+0000 7f06be59c640 1 -- 172.21.15.143:0/1682944513 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f06c015ebe0 con 0x7f06ac003980 2023-11-26T07:22:16.354 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:16.352+0000 7f06be59c640 1 -- 172.21.15.143:0/1682944513 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f06c015a650 con 0x7f06ac0047d0 2023-11-26T07:22:16.354 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:16.352+0000 7f06be59c640 1 -- 172.21.15.143:0/1682944513 --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f06ac00b990 con 0x7f06c0071b70 2023-11-26T07:22:16.354 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:16.352+0000 7f06bed9d640 1 --2- 172.21.15.143:0/1682944513 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f06ac003980 0x7f06ac003da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:22:16.354 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:16.352+0000 7f06b659c640 1 --2- 172.21.15.143:0/1682944513 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f06ac0047d0 0x7f06ac004c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:22:16.354 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:16.352+0000 7f06bf59e640 1 --2- 172.21.15.143:0/1682944513 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f06c0071b70 0x7f06ac005190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:22:16.355 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:16.352+0000 7f06bed9d640 1 -- 172.21.15.143:0/1682944513 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f06c0071b70 msgr2=0x7f06ac005190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-26T07:22:16.355 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:16.352+0000 7f06bed9d640 1 --2- 172.21.15.143:0/1682944513 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f06c0071b70 0x7f06ac005190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:22:16.355 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:16.352+0000 7f06bed9d640 1 -- 172.21.15.143:0/1682944513 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f06ac0047d0 msgr2=0x7f06ac004c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-26T07:22:16.355 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:16.352+0000 7f06bed9d640 1 --2- 172.21.15.143:0/1682944513 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f06ac0047d0 0x7f06ac004c50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:22:16.355 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:16.352+0000 7f06bed9d640 1 -- 172.21.15.143:0/1682944513 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f06ac003530 con 0x7f06ac003980 2023-11-26T07:22:16.355 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:16.352+0000 7f06bed9d640 1 --2- 172.21.15.143:0/1682944513 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f06ac003980 0x7f06ac003da0 secure :-1 s=READY pgs=843 cs=0 l=1 rev1=1 crypto rx=0x7f06b0009ae0 tx=0x7f06b001c9c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7d0841245dba4a8c server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:22:16.356 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:16.352+0000 7f06be59c640 1 -- 172.21.15.143:0/1682944513 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f06b00a9420 con 0x7f06ac003980 2023-11-26T07:22:16.356 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:16.352+0000 7f06be59c640 1 -- 172.21.15.143:0/1682944513 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f06b00a9580 con 0x7f06ac003980 2023-11-26T07:22:16.357 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:16.352+0000 7f06be59c640 1 -- 172.21.15.143:0/1682944513 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f06b00a9930 con 0x7f06ac003980 2023-11-26T07:22:16.357 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:16.356+0000 7f06c5017640 1 -- 172.21.15.143:0/1682944513 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f06ac003980 msgr2=0x7f06ac003da0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:22:16.357 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:16.356+0000 7f06c5017640 1 --2- 172.21.15.143:0/1682944513 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f06ac003980 0x7f06ac003da0 secure :-1 s=READY pgs=843 cs=0 l=1 rev1=1 crypto rx=0x7f06b0009ae0 tx=0x7f06b001c9c0 comp rx=0 tx=0).stop 2023-11-26T07:22:16.357 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:16.356+0000 7f06bf59e640 1 -- 172.21.15.143:0/1682944513 reap_dead start 2023-11-26T07:22:16.357 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:16.356+0000 7f06c5017640 1 -- 172.21.15.143:0/1682944513 shutdown_connections 2023-11-26T07:22:16.357 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:16.356+0000 7f06c5017640 1 -- 172.21.15.143:0/1682944513 >> 172.21.15.143:0/1682944513 conn(0x7f06c00b0c10 msgr2=0x7f06c00b1020 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:22:16.357 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:16.356+0000 7f06c5017640 1 -- 172.21.15.143:0/1682944513 shutdown_connections 2023-11-26T07:22:16.357 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:16.356+0000 7f06c5017640 1 -- 172.21.15.143:0/1682944513 wait complete. 2023-11-26T07:22:16.358 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:16.356+0000 7f06c5017640 1 Processor -- start 2023-11-26T07:22:16.358 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:16.356+0000 7f06c5017640 1 -- start start 2023-11-26T07:22:16.360 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:16.356+0000 7f06c5017640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f06ac003980 0x7f06c0152440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:22:16.360 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:16.356+0000 7f06c5017640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f06c01408c0 0x7f06c00b2240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:22:16.360 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:16.356+0000 7f06c5017640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f06c00b2780 0x7f06c00aeb80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:22:16.360 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:16.356+0000 7f06c5017640 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f06c00b2c00 con 0x7f06c01408c0 2023-11-26T07:22:16.360 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:16.356+0000 7f06c5017640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f06c0099da0 con 0x7f06ac003980 2023-11-26T07:22:16.360 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:16.356+0000 7f06c5017640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f06c0158d20 con 0x7f06c00b2780 2023-11-26T07:22:16.360 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:16.356+0000 7f06b659c640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f06c01408c0 0x7f06c00b2240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:22:16.361 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:16.356+0000 7f06b659c640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f06c01408c0 0x7f06c00b2240 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:34106/0 (socket says 172.21.15.143:34106) 2023-11-26T07:22:16.361 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:16.356+0000 7f06b659c640 1 -- 172.21.15.143:0/3586447465 learned_addr learned my addr 172.21.15.143:0/3586447465 (peer_addr_for_me v2:172.21.15.143:0/0) 2023-11-26T07:22:16.361 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:16.356+0000 7f06bed9d640 1 --2- 172.21.15.143:0/3586447465 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f06ac003980 0x7f06c0152440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:22:16.361 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:16.356+0000 7f06bf59e640 1 --2- 172.21.15.143:0/3586447465 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f06c00b2780 0x7f06c00aeb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:22:16.361 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:16.360+0000 7f06b659c640 1 -- 172.21.15.143:0/3586447465 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f06c00b2780 msgr2=0x7f06c00aeb80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:22:16.361 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:16.360+0000 7f06b659c640 1 --2- 172.21.15.143:0/3586447465 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f06c00b2780 0x7f06c00aeb80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:22:16.361 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:16.360+0000 7f06b659c640 1 -- 172.21.15.143:0/3586447465 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f06ac003980 msgr2=0x7f06c0152440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:22:16.361 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:16.360+0000 7f06b659c640 1 --2- 172.21.15.143:0/3586447465 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f06ac003980 0x7f06c0152440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:22:16.361 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:16.360+0000 7f06b659c640 1 -- 172.21.15.143:0/3586447465 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f06c0158fc0 con 0x7f06c01408c0 2023-11-26T07:22:16.361 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:16.360+0000 7f06bf59e640 1 --2- 172.21.15.143:0/3586447465 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f06c00b2780 0x7f06c00aeb80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-26T07:22:16.361 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:16.360+0000 7f06b659c640 1 --2- 172.21.15.143:0/3586447465 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f06c01408c0 0x7f06c00b2240 secure :-1 s=READY pgs=844 cs=0 l=1 rev1=1 crypto rx=0x7f06a80079f0 tx=0x7f06a800a640 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:22:16.362 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:16.360+0000 7f06b7fff640 1 -- 172.21.15.143:0/3586447465 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f06a80017e0 con 0x7f06c01408c0 2023-11-26T07:22:16.362 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:16.360+0000 7f06b7fff640 1 -- 172.21.15.143:0/3586447465 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f06a8001a60 con 0x7f06c01408c0 2023-11-26T07:22:16.362 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:16.360+0000 7f06b7fff640 1 -- 172.21.15.143:0/3586447465 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f06a8013610 con 0x7f06c01408c0 2023-11-26T07:22:16.362 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:16.360+0000 7f06c5017640 1 -- 172.21.15.143:0/3586447465 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f06c01592a0 con 0x7f06c01408c0 2023-11-26T07:22:16.362 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:16.360+0000 7f06c5017640 1 -- 172.21.15.143:0/3586447465 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f06c0159860 con 0x7f06c01408c0 2023-11-26T07:22:16.363 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:16.360+0000 7f06c5017640 1 -- 172.21.15.143:0/3586447465 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f06c00676f0 con 0x7f06c01408c0 2023-11-26T07:22:16.363 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:16.360+0000 7f06b7fff640 1 -- 172.21.15.143:0/3586447465 <== mon.0 v2:172.21.15.143:3300/0 4 ==== mgrmap(e 10) v1 ==== 86056+0+0 (secure 0 0 0) 0x7f06a8013770 con 0x7f06c01408c0 2023-11-26T07:22:16.363 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:16.360+0000 7f06b7fff640 1 --2- 172.21.15.143:0/3586447465 >> [v2:172.21.15.143:6832/2150557544,v1:172.21.15.143:6833/2150557544] conn(0x7f06a00666e0 0x7f06a0068ba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:22:16.363 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:16.360+0000 7f06b7fff640 1 -- 172.21.15.143:0/3586447465 <== mon.0 v2:172.21.15.143:3300/0 5 ==== osd_map(59..59 src has 1..59) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f06a8084e10 con 0x7f06c01408c0 2023-11-26T07:22:16.364 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:16.360+0000 7f06bed9d640 1 --2- 172.21.15.143:0/3586447465 >> [v2:172.21.15.143:6832/2150557544,v1:172.21.15.143:6833/2150557544] conn(0x7f06a00666e0 0x7f06a0068ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:22:16.364 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:16.364+0000 7f06bed9d640 1 --2- 172.21.15.143:0/3586447465 >> [v2:172.21.15.143:6832/2150557544,v1:172.21.15.143:6833/2150557544] conn(0x7f06a00666e0 0x7f06a0068ba0 secure :-1 s=READY pgs=399 cs=0 l=1 rev1=1 crypto rx=0x7f06b00ae000 tx=0x7f06b00b3040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:22:16.367 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:16.364+0000 7f06b7fff640 1 -- 172.21.15.143:0/3586447465 <== mon.0 v2:172.21.15.143:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172649 (secure 0 0 0) 0x7f06a8051f30 con 0x7f06c01408c0 2023-11-26T07:22:16.540 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:16.536+0000 7f06c5017640 1 -- 172.21.15.143:0/3586447465 --> [v2:172.21.15.143:6832/2150557544,v1:172.21.15.143:6833/2150557544] -- mgr_command(tid 0: {"prefix": "fs subvolume rm", "vol_name": "cephfs", "sub_name": "subvol970", "target": ["mon-mgr", ""]}) v1 -- 0x7f06c00ae0e0 con 0x7f06a00666e0 2023-11-26T07:22:16.556 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:16.556+0000 7f06b7fff640 1 -- 172.21.15.143:0/3586447465 <== mgr.4100 v2:172.21.15.143:6832/2150557544 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+0 (secure 0 0 0) 0x7f06c00ae0e0 con 0x7f06a00666e0 2023-11-26T07:22:16.559 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:16.556+0000 7f06c5017640 1 -- 172.21.15.143:0/3586447465 >> [v2:172.21.15.143:6832/2150557544,v1:172.21.15.143:6833/2150557544] conn(0x7f06a00666e0 msgr2=0x7f06a0068ba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:22:16.559 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:16.556+0000 7f06c5017640 1 --2- 172.21.15.143:0/3586447465 >> [v2:172.21.15.143:6832/2150557544,v1:172.21.15.143:6833/2150557544] conn(0x7f06a00666e0 0x7f06a0068ba0 secure :-1 s=READY pgs=399 cs=0 l=1 rev1=1 crypto rx=0x7f06b00ae000 tx=0x7f06b00b3040 comp rx=0 tx=0).stop 2023-11-26T07:22:16.559 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:16.556+0000 7f06c5017640 1 -- 172.21.15.143:0/3586447465 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f06c01408c0 msgr2=0x7f06c00b2240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:22:16.559 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:16.556+0000 7f06c5017640 1 --2- 172.21.15.143:0/3586447465 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f06c01408c0 0x7f06c00b2240 secure :-1 s=READY pgs=844 cs=0 l=1 rev1=1 crypto rx=0x7f06a80079f0 tx=0x7f06a800a640 comp rx=0 tx=0).stop 2023-11-26T07:22:16.559 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:16.556+0000 7f06c5017640 1 -- 172.21.15.143:0/3586447465 shutdown_connections 2023-11-26T07:22:16.559 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:16.556+0000 7f06c5017640 1 --2- 172.21.15.143:0/3586447465 >> [v2:172.21.15.143:6832/2150557544,v1:172.21.15.143:6833/2150557544] conn(0x7f06a00666e0 0x7f06a0068ba0 unknown :-1 s=CLOSED pgs=399 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:22:16.560 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:16.556+0000 7f06c5017640 1 --2- 172.21.15.143:0/3586447465 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f06c00b2780 0x7f06c00aeb80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:22:16.560 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:16.556+0000 7f06c5017640 1 --2- 172.21.15.143:0/3586447465 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f06c01408c0 0x7f06c00b2240 unknown :-1 s=CLOSED pgs=844 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:22:16.560 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:16.556+0000 7f06c5017640 1 --2- 172.21.15.143:0/3586447465 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f06ac003980 0x7f06c0152440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:22:16.560 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:16.556+0000 7f06c5017640 1 -- 172.21.15.143:0/3586447465 >> 172.21.15.143:0/3586447465 conn(0x7f06c00b0c10 msgr2=0x7f06c015dc20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:22:16.560 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:16.556+0000 7f06c5017640 1 -- 172.21.15.143:0/3586447465 shutdown_connections 2023-11-26T07:22:16.560 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:16.556+0000 7f06c5017640 1 -- 172.21.15.143:0/3586447465 wait complete. 2023-11-26T07:22:16.565 INFO:tasks.ceph.mgr.y.smithi143.stderr:2023-11-26T07:22:16.564+0000 7f504a2c9640 -1 client.0 error registering admin socket command: (17) File exists 2023-11-26T07:22:16.565 INFO:tasks.ceph.mgr.y.smithi143.stderr:2023-11-26T07:22:16.564+0000 7f504a2c9640 -1 client.0 error registering admin socket command: (17) File exists 2023-11-26T07:22:16.565 INFO:tasks.ceph.mgr.y.smithi143.stderr:2023-11-26T07:22:16.564+0000 7f504a2c9640 -1 client.0 error registering admin socket command: (17) File exists 2023-11-26T07:22:16.565 INFO:tasks.ceph.mgr.y.smithi143.stderr:2023-11-26T07:22:16.564+0000 7f504a2c9640 -1 client.0 error registering admin socket command: (17) File exists 2023-11-26T07:22:16.565 INFO:tasks.ceph.mgr.y.smithi143.stderr:2023-11-26T07:22:16.564+0000 7f504a2c9640 -1 client.0 error registering admin socket command: (17) File exists 2023-11-26T07:22:16.582 DEBUG:teuthology.orchestra.run.smithi143:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume rm cephfs subvol15241 2023-11-26T07:22:16.583 INFO:tasks.ceph.mgr.y.smithi143.stderr:2023-11-26T07:22:16.580+0000 7f504bacc640 -1 client.0 error registering admin socket command: (17) File exists 2023-11-26T07:22:16.583 INFO:tasks.ceph.mgr.y.smithi143.stderr:2023-11-26T07:22:16.580+0000 7f504bacc640 -1 client.0 error registering admin socket command: (17) File exists 2023-11-26T07:22:16.583 INFO:tasks.ceph.mgr.y.smithi143.stderr:2023-11-26T07:22:16.580+0000 7f504bacc640 -1 client.0 error registering admin socket command: (17) File exists 2023-11-26T07:22:16.583 INFO:tasks.ceph.mgr.y.smithi143.stderr:2023-11-26T07:22:16.580+0000 7f504bacc640 -1 client.0 error registering admin socket command: (17) File exists 2023-11-26T07:22:16.583 INFO:tasks.ceph.mgr.y.smithi143.stderr:2023-11-26T07:22:16.580+0000 7f504bacc640 -1 client.0 error registering admin socket command: (17) File exists 2023-11-26T07:22:16.725 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:16.724+0000 7f2188380640 1 Processor -- start 2023-11-26T07:22:16.725 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:16.724+0000 7f2188380640 1 -- start start 2023-11-26T07:22:16.727 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:16.724+0000 7f2188380640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f218015d580 0x7f21800d3920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:22:16.727 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:16.724+0000 7f2188380640 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7f21800d3e60 0x7f21800d17f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:22:16.727 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:16.724+0000 7f2188380640 1 -- --> v1:172.21.15.188:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f21800d21e0 con 0x7f21800d1d30 2023-11-26T07:22:16.727 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:16.724+0000 7f2188380640 1 -- --> v2:172.21.15.143:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f21800d0bc0 con 0x7f21800d3e60 2023-11-26T07:22:16.727 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:16.724+0000 7f2188380640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f21800d0320 con 0x7f218015d580 2023-11-26T07:22:16.728 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:16.724+0000 7f218591b640 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7f21800d3e60 0x7f21800d17f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:22:16.728 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:16.724+0000 7f218591b640 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7f21800d3e60 0x7f21800d17f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:34156/0 (socket says 172.21.15.143:34156) 2023-11-26T07:22:16.728 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:16.724+0000 7f218591b640 1 -- 172.21.15.143:0/2528859300 learned_addr learned my addr 172.21.15.143:0/2528859300 (peer_addr_for_me v2:172.21.15.143:0/0) 2023-11-26T07:22:16.728 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:16.724+0000 7f218611c640 1 --2- 172.21.15.143:0/2528859300 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f218015d580 0x7f21800d3920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:22:16.728 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:16.724+0000 7f218591b640 1 -- 172.21.15.143:0/2528859300 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f218015d580 msgr2=0x7f21800d3920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-26T07:22:16.728 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:16.724+0000 7f218591b640 1 --2- 172.21.15.143:0/2528859300 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f218015d580 0x7f21800d3920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:22:16.728 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:16.724+0000 7f218591b640 1 -- 172.21.15.143:0/2528859300 >> v1:172.21.15.188:6789/0 conn(0x7f21800d1d30 legacy=0x7f21800cdce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:22:16.728 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:16.724+0000 7f218591b640 1 -- 172.21.15.143:0/2528859300 --> v2:172.21.15.143:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f21800d0600 con 0x7f21800d3e60 2023-11-26T07:22:16.728 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:16.724+0000 7f218611c640 1 --2- 172.21.15.143:0/2528859300 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f218015d580 0x7f21800d3920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-26T07:22:16.730 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:16.728+0000 7f218591b640 1 --2- 172.21.15.143:0/2528859300 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f21800d3e60 0x7f21800d17f0 secure :-1 s=READY pgs=849 cs=0 l=1 rev1=1 crypto rx=0x7f2170009870 tx=0x7f217001c510 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4e37ca09a8849a13 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:22:16.730 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:16.728+0000 7f218511a640 1 -- 172.21.15.143:0/2528859300 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f21700aa670 con 0x7f21800d3e60 2023-11-26T07:22:16.730 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:16.728+0000 7f218511a640 1 -- 172.21.15.143:0/2528859300 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f21700aa7d0 con 0x7f21800d3e60 2023-11-26T07:22:16.731 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:16.728+0000 7f218511a640 1 -- 172.21.15.143:0/2528859300 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f21700aab60 con 0x7f21800d3e60 2023-11-26T07:22:16.731 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:16.728+0000 7f2188380640 1 -- 172.21.15.143:0/2528859300 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f21800d3e60 msgr2=0x7f21800d17f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:22:16.731 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:16.728+0000 7f2188380640 1 --2- 172.21.15.143:0/2528859300 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f21800d3e60 0x7f21800d17f0 secure :-1 s=READY pgs=849 cs=0 l=1 rev1=1 crypto rx=0x7f2170009870 tx=0x7f217001c510 comp rx=0 tx=0).stop 2023-11-26T07:22:16.731 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:16.728+0000 7f2188380640 1 -- 172.21.15.143:0/2528859300 shutdown_connections 2023-11-26T07:22:16.732 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:16.728+0000 7f2188380640 1 --2- 172.21.15.143:0/2528859300 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f21800d3e60 0x7f21800d17f0 unknown :-1 s=CLOSED pgs=849 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:22:16.732 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:16.728+0000 7f2188380640 1 --2- 172.21.15.143:0/2528859300 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f218015d580 0x7f21800d3920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:22:16.732 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:16.728+0000 7f2188380640 1 -- 172.21.15.143:0/2528859300 >> 172.21.15.143:0/2528859300 conn(0x7f218009c550 msgr2=0x7f21800d3210 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:22:16.732 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:16.728+0000 7f2188380640 1 -- 172.21.15.143:0/2528859300 shutdown_connections 2023-11-26T07:22:16.732 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:16.728+0000 7f2188380640 1 -- 172.21.15.143:0/2528859300 wait complete. 2023-11-26T07:22:16.733 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:16.732+0000 7f2188380640 1 Processor -- start 2023-11-26T07:22:16.733 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:16.732+0000 7f2188380640 1 -- start start 2023-11-26T07:22:16.735 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:16.732+0000 7f2188380640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f21800d1d30 0x7f2180160360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:22:16.735 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:16.732+0000 7f2188380640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f21800d3e60 0x7f218015ea00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:22:16.735 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:16.732+0000 7f2188380640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f218015d580 0x7f21801594b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:22:16.735 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:16.732+0000 7f2188380640 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f218015e570 con 0x7f218015d580 2023-11-26T07:22:16.735 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:16.732+0000 7f2188380640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f21800ce160 con 0x7f21800d1d30 2023-11-26T07:22:16.735 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:16.732+0000 7f2188380640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f21801609f0 con 0x7f21800d3e60 2023-11-26T07:22:16.735 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:16.732+0000 7f218691d640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f218015d580 0x7f21801594b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:22:16.736 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:16.732+0000 7f218691d640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f218015d580 0x7f21801594b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:34162/0 (socket says 172.21.15.143:34162) 2023-11-26T07:22:16.736 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:16.732+0000 7f218691d640 1 -- 172.21.15.143:0/3545915437 learned_addr learned my addr 172.21.15.143:0/3545915437 (peer_addr_for_me v2:172.21.15.143:0/0) 2023-11-26T07:22:16.736 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:16.732+0000 7f218611c640 1 --2- 172.21.15.143:0/3545915437 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f21800d1d30 0x7f2180160360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:22:16.736 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:16.732+0000 7f218591b640 1 --2- 172.21.15.143:0/3545915437 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f21800d3e60 0x7f218015ea00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:22:16.736 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:16.732+0000 7f218691d640 1 -- 172.21.15.143:0/3545915437 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f21800d3e60 msgr2=0x7f218015ea00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:22:16.736 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:16.732+0000 7f218691d640 1 --2- 172.21.15.143:0/3545915437 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f21800d3e60 0x7f218015ea00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:22:16.736 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:16.732+0000 7f218691d640 1 -- 172.21.15.143:0/3545915437 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f21800d1d30 msgr2=0x7f2180160360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:22:16.736 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:16.732+0000 7f218691d640 1 --2- 172.21.15.143:0/3545915437 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f21800d1d30 0x7f2180160360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:22:16.736 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:16.732+0000 7f218691d640 1 -- 172.21.15.143:0/3545915437 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f21700094d0 con 0x7f218015d580 2023-11-26T07:22:16.736 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:16.732+0000 7f218691d640 1 --2- 172.21.15.143:0/3545915437 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f218015d580 0x7f21801594b0 secure :-1 s=READY pgs=850 cs=0 l=1 rev1=1 crypto rx=0x7f217800c770 tx=0x7f217800cc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:22:16.736 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:16.732+0000 7f2176ffd640 1 -- 172.21.15.143:0/3545915437 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2178003b80 con 0x7f218015d580 2023-11-26T07:22:16.737 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:16.732+0000 7f2176ffd640 1 -- 172.21.15.143:0/3545915437 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2178003ce0 con 0x7f218015d580 2023-11-26T07:22:16.737 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:16.732+0000 7f2176ffd640 1 -- 172.21.15.143:0/3545915437 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f217800aa90 con 0x7f218015d580 2023-11-26T07:22:16.737 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:16.732+0000 7f2188380640 1 -- 172.21.15.143:0/3545915437 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f218015dc40 con 0x7f218015d580 2023-11-26T07:22:16.737 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:16.732+0000 7f2188380640 1 -- 172.21.15.143:0/3545915437 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f218015e200 con 0x7f218015d580 2023-11-26T07:22:16.737 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:16.736+0000 7f2188380640 1 -- 172.21.15.143:0/3545915437 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f218015e6e0 con 0x7f218015d580 2023-11-26T07:22:16.737 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:16.736+0000 7f2176ffd640 1 -- 172.21.15.143:0/3545915437 <== mon.0 v2:172.21.15.143:3300/0 4 ==== mgrmap(e 10) v1 ==== 86056+0+0 (secure 0 0 0) 0x7f217800abf0 con 0x7f218015d580 2023-11-26T07:22:16.738 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:16.736+0000 7f2176ffd640 1 --2- 172.21.15.143:0/3545915437 >> [v2:172.21.15.143:6832/2150557544,v1:172.21.15.143:6833/2150557544] conn(0x7f21640666b0 0x7f2164068b70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:22:16.738 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:16.736+0000 7f2176ffd640 1 -- 172.21.15.143:0/3545915437 <== mon.0 v2:172.21.15.143:3300/0 5 ==== osd_map(59..59 src has 1..59) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f2178083fe0 con 0x7f218015d580 2023-11-26T07:22:16.738 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:16.736+0000 7f218611c640 1 --2- 172.21.15.143:0/3545915437 >> [v2:172.21.15.143:6832/2150557544,v1:172.21.15.143:6833/2150557544] conn(0x7f21640666b0 0x7f2164068b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:22:16.739 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:16.736+0000 7f218611c640 1 --2- 172.21.15.143:0/3545915437 >> [v2:172.21.15.143:6832/2150557544,v1:172.21.15.143:6833/2150557544] conn(0x7f21640666b0 0x7f2164068b70 secure :-1 s=READY pgs=400 cs=0 l=1 rev1=1 crypto rx=0x7f21800b9030 tx=0x7f217c008040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:22:16.742 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:16.740+0000 7f2176ffd640 1 -- 172.21.15.143:0/3545915437 <== mon.0 v2:172.21.15.143:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172649 (secure 0 0 0) 0x7f2178051170 con 0x7f218015d580 2023-11-26T07:22:16.912 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:16.912+0000 7f2188380640 1 -- 172.21.15.143:0/3545915437 --> [v2:172.21.15.143:6832/2150557544,v1:172.21.15.143:6833/2150557544] -- mgr_command(tid 0: {"prefix": "fs subvolume rm", "vol_name": "cephfs", "sub_name": "subvol15241", "target": ["mon-mgr", ""]}) v1 -- 0x7f218016a1b0 con 0x7f21640666b0 2023-11-26T07:22:16.931 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:16.928+0000 7f2176ffd640 1 -- 172.21.15.143:0/3545915437 <== mgr.4100 v2:172.21.15.143:6832/2150557544 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+0 (secure 0 0 0) 0x7f217c007650 con 0x7f21640666b0 2023-11-26T07:22:16.933 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:16.932+0000 7f2188380640 1 -- 172.21.15.143:0/3545915437 >> [v2:172.21.15.143:6832/2150557544,v1:172.21.15.143:6833/2150557544] conn(0x7f21640666b0 msgr2=0x7f2164068b70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:22:16.933 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:16.932+0000 7f2188380640 1 --2- 172.21.15.143:0/3545915437 >> [v2:172.21.15.143:6832/2150557544,v1:172.21.15.143:6833/2150557544] conn(0x7f21640666b0 0x7f2164068b70 secure :-1 s=READY pgs=400 cs=0 l=1 rev1=1 crypto rx=0x7f21800b9030 tx=0x7f217c008040 comp rx=0 tx=0).stop 2023-11-26T07:22:16.933 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:16.932+0000 7f2188380640 1 -- 172.21.15.143:0/3545915437 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f218015d580 msgr2=0x7f21801594b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:22:16.933 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:16.932+0000 7f2188380640 1 --2- 172.21.15.143:0/3545915437 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f218015d580 0x7f21801594b0 secure :-1 s=READY pgs=850 cs=0 l=1 rev1=1 crypto rx=0x7f217800c770 tx=0x7f217800cc30 comp rx=0 tx=0).stop 2023-11-26T07:22:16.933 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:16.932+0000 7f2188380640 1 -- 172.21.15.143:0/3545915437 shutdown_connections 2023-11-26T07:22:16.933 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:16.932+0000 7f2188380640 1 --2- 172.21.15.143:0/3545915437 >> [v2:172.21.15.143:6832/2150557544,v1:172.21.15.143:6833/2150557544] conn(0x7f21640666b0 0x7f2164068b70 unknown :-1 s=CLOSED pgs=400 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:22:16.933 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:16.932+0000 7f2188380640 1 --2- 172.21.15.143:0/3545915437 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f218015d580 0x7f21801594b0 unknown :-1 s=CLOSED pgs=850 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:22:16.933 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:16.932+0000 7f2188380640 1 --2- 172.21.15.143:0/3545915437 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f21800d3e60 0x7f218015ea00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:22:16.934 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:16.932+0000 7f2188380640 1 --2- 172.21.15.143:0/3545915437 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f21800d1d30 0x7f2180160360 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:22:16.934 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:16.932+0000 7f2188380640 1 -- 172.21.15.143:0/3545915437 >> 172.21.15.143:0/3545915437 conn(0x7f218009c550 msgr2=0x7f218006c990 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:22:16.934 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:16.932+0000 7f2188380640 1 -- 172.21.15.143:0/3545915437 shutdown_connections 2023-11-26T07:22:16.934 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:16.932+0000 7f2188380640 1 -- 172.21.15.143:0/3545915437 wait complete. 2023-11-26T07:22:16.954 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-26T07:22:16.954 DEBUG:teuthology.orchestra.run.smithi143:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c 'stat -c %h /home/ubuntu/cephtest/mnt.0/./volumes/_deleting') 2023-11-26T07:22:16.969 INFO:teuthology.orchestra.run.smithi143.stdout:2 2023-11-26T07:22:16.970 INFO:teuthology.nuke.actions:Clearing teuthology firewall rules... 2023-11-26T07:22:16.971 DEBUG:teuthology.orchestra.run.smithi143:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2023-11-26T07:22:17.102 DEBUG:teuthology.orchestra.run.smithi188:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2023-11-26T07:22:17.179 INFO:teuthology.nuke.actions:Cleared teuthology firewall rules. 2023-11-26T07:22:17.179 DEBUG:teuthology.orchestra.run.smithi143:> set -ex 2023-11-26T07:22:17.179 DEBUG:teuthology.orchestra.run.smithi143:> dd if=/proc/self/mounts of=/dev/stdout 2023-11-26T07:22:17.188 DEBUG:teuthology.orchestra.run.smithi143:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-11-26T07:22:17.372 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:17.368+0000 7f6b853a1640 1 Processor -- start 2023-11-26T07:22:17.372 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:17.372+0000 7f6b853a1640 1 -- start start 2023-11-26T07:22:17.373 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:17.372+0000 7f6b853a1640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f6b801596d0 0x7f6b80071850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:22:17.373 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:17.372+0000 7f6b853a1640 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7f6b800b2f80 0x7f6b8006cbe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:22:17.373 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:17.372+0000 7f6b853a1640 1 -- --> v1:172.21.15.143:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6b8006c750 con 0x7f6b800ce680 2023-11-26T07:22:17.374 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:17.372+0000 7f6b853a1640 1 -- --> v2:172.21.15.143:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6b800cbe00 con 0x7f6b800b2f80 2023-11-26T07:22:17.374 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:17.372+0000 7f6b853a1640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6b800b3550 con 0x7f6b801596d0 2023-11-26T07:22:17.374 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:17.372+0000 7f6b7f7fe640 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7f6b800b2f80 0x7f6b8006cbe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:22:17.374 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:17.372+0000 7f6b77fff640 1 --1- >> v1:172.21.15.143:6789/0 conn(0x7f6b800ce680 0x7f6b800b2870 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.143:6789/0 says I am v1:172.21.15.143:59130/0 (socket says 172.21.15.143:59130) 2023-11-26T07:22:17.374 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:17.372+0000 7f6b77fff640 1 -- 172.21.15.143:0/2838116069 learned_addr learned my addr 172.21.15.143:0/2838116069 (peer_addr_for_me v1:172.21.15.143:0/0) 2023-11-26T07:22:17.375 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:17.372+0000 7f6b7effd640 1 --2- 172.21.15.143:0/2838116069 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f6b801596d0 0x7f6b80071850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:22:17.375 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:17.372+0000 7f6b7e7fc640 1 -- 172.21.15.143:0/2838116069 <== mon.0 v1:172.21.15.143:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3395447645 0 0) 0x7f6b8006c750 con 0x7f6b800ce680 2023-11-26T07:22:17.375 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:17.372+0000 7f6b7e7fc640 1 -- 172.21.15.143:0/2838116069 --> v1:172.21.15.143:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f6b6c003450 con 0x7f6b800ce680 2023-11-26T07:22:17.375 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:17.372+0000 7f6b7f7fe640 1 -- 172.21.15.143:0/2838116069 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f6b801596d0 msgr2=0x7f6b80071850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-26T07:22:17.375 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:17.372+0000 7f6b7f7fe640 1 --2- 172.21.15.143:0/2838116069 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f6b801596d0 0x7f6b80071850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:22:17.375 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:17.372+0000 7f6b7f7fe640 1 -- 172.21.15.143:0/2838116069 >> v1:172.21.15.143:6789/0 conn(0x7f6b800ce680 legacy=0x7f6b800b2870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:22:17.375 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:17.372+0000 7f6b7f7fe640 1 -- 172.21.15.143:0/2838116069 --> v2:172.21.15.143:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6b80071e80 con 0x7f6b800b2f80 2023-11-26T07:22:17.377 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:17.376+0000 7f6b7f7fe640 1 --2- 172.21.15.143:0/2838116069 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f6b800b2f80 0x7f6b8006cbe0 secure :-1 s=READY pgs=852 cs=0 l=1 rev1=1 crypto rx=0x7f6b78009d90 tx=0x7f6b7801c950 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4e56bf6e890a3865 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:22:17.377 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:17.376+0000 7f6b7e7fc640 1 -- 172.21.15.143:0/2838116069 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6b780a5880 con 0x7f6b800b2f80 2023-11-26T07:22:17.377 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:17.376+0000 7f6b7e7fc640 1 -- 172.21.15.143:0/2838116069 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6b780a59e0 con 0x7f6b800b2f80 2023-11-26T07:22:17.377 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:17.376+0000 7f6b7e7fc640 1 -- 172.21.15.143:0/2838116069 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6b780a5d70 con 0x7f6b800b2f80 2023-11-26T07:22:17.378 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:17.376+0000 7f6b853a1640 1 -- 172.21.15.143:0/2838116069 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f6b800b2f80 msgr2=0x7f6b8006cbe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:22:17.378 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:17.376+0000 7f6b853a1640 1 --2- 172.21.15.143:0/2838116069 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f6b800b2f80 0x7f6b8006cbe0 secure :-1 s=READY pgs=852 cs=0 l=1 rev1=1 crypto rx=0x7f6b78009d90 tx=0x7f6b7801c950 comp rx=0 tx=0).stop 2023-11-26T07:22:17.378 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:17.376+0000 7f6b853a1640 1 -- 172.21.15.143:0/2838116069 shutdown_connections 2023-11-26T07:22:17.378 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:17.376+0000 7f6b853a1640 1 --2- 172.21.15.143:0/2838116069 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f6b800b2f80 0x7f6b8006cbe0 unknown :-1 s=CLOSED pgs=852 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:22:17.378 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:17.376+0000 7f6b853a1640 1 --2- 172.21.15.143:0/2838116069 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f6b801596d0 0x7f6b80071850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:22:17.378 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:17.376+0000 7f6b853a1640 1 -- 172.21.15.143:0/2838116069 >> 172.21.15.143:0/2838116069 conn(0x7f6b8006a890 msgr2=0x7f6b8006aca0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:22:17.378 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:17.376+0000 7f6b853a1640 1 -- 172.21.15.143:0/2838116069 shutdown_connections 2023-11-26T07:22:17.379 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:17.376+0000 7f6b853a1640 1 -- 172.21.15.143:0/2838116069 wait complete. 2023-11-26T07:22:17.379 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:17.376+0000 7f6b853a1640 1 Processor -- start 2023-11-26T07:22:17.380 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:17.376+0000 7f6b853a1640 1 -- start start 2023-11-26T07:22:17.381 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:17.380+0000 7f6b853a1640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f6b800b2f80 0x7f6b800cbf40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:22:17.381 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:17.380+0000 7f6b853a1640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f6b800ce680 0x7f6b801f9ee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:22:17.381 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:17.380+0000 7f6b853a1640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f6b801596d0 0x7f6b80147020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:22:17.381 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:17.380+0000 7f6b853a1640 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6b8015b5e0 con 0x7f6b800ce680 2023-11-26T07:22:17.381 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:17.380+0000 7f6b853a1640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6b801fa420 con 0x7f6b801596d0 2023-11-26T07:22:17.382 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:17.380+0000 7f6b853a1640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6b800cc5d0 con 0x7f6b800b2f80 2023-11-26T07:22:17.382 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:17.380+0000 7f6b77fff640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f6b800ce680 0x7f6b801f9ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:22:17.382 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:17.380+0000 7f6b77fff640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f6b800ce680 0x7f6b801f9ee0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:34194/0 (socket says 172.21.15.143:34194) 2023-11-26T07:22:17.382 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:17.380+0000 7f6b77fff640 1 -- 172.21.15.143:0/3158263339 learned_addr learned my addr 172.21.15.143:0/3158263339 (peer_addr_for_me v2:172.21.15.143:0/0) 2023-11-26T07:22:17.382 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:17.380+0000 7f6b7effd640 1 --2- 172.21.15.143:0/3158263339 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f6b800b2f80 0x7f6b800cbf40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:22:17.382 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:17.380+0000 7f6b7f7fe640 1 --2- 172.21.15.143:0/3158263339 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f6b801596d0 0x7f6b80147020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:22:17.382 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:17.380+0000 7f6b77fff640 1 -- 172.21.15.143:0/3158263339 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f6b800b2f80 msgr2=0x7f6b800cbf40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:22:17.382 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:17.380+0000 7f6b77fff640 1 --2- 172.21.15.143:0/3158263339 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f6b800b2f80 0x7f6b800cbf40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:22:17.382 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:17.380+0000 7f6b77fff640 1 -- 172.21.15.143:0/3158263339 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f6b801596d0 msgr2=0x7f6b80147020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:22:17.382 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:17.380+0000 7f6b77fff640 1 --2- 172.21.15.143:0/3158263339 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f6b801596d0 0x7f6b80147020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:22:17.382 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:17.380+0000 7f6b77fff640 1 -- 172.21.15.143:0/3158263339 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6b78009a70 con 0x7f6b800ce680 2023-11-26T07:22:17.383 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:17.380+0000 7f6b77fff640 1 --2- 172.21.15.143:0/3158263339 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f6b800ce680 0x7f6b801f9ee0 secure :-1 s=READY pgs=853 cs=0 l=1 rev1=1 crypto rx=0x7f6b6800c4d0 tx=0x7f6b6800c990 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:22:17.383 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:17.380+0000 7f6b777fe640 1 -- 172.21.15.143:0/3158263339 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6b68005cf0 con 0x7f6b800ce680 2023-11-26T07:22:17.384 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:17.380+0000 7f6b777fe640 1 -- 172.21.15.143:0/3158263339 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6b68005e50 con 0x7f6b800ce680 2023-11-26T07:22:17.384 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:17.380+0000 7f6b777fe640 1 -- 172.21.15.143:0/3158263339 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6b68007270 con 0x7f6b800ce680 2023-11-26T07:22:17.384 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:17.380+0000 7f6b853a1640 1 -- 172.21.15.143:0/3158263339 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6b800cc8b0 con 0x7f6b800ce680 2023-11-26T07:22:17.384 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:17.380+0000 7f6b853a1640 1 -- 172.21.15.143:0/3158263339 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6b800cce70 con 0x7f6b800ce680 2023-11-26T07:22:17.384 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:17.380+0000 7f6b853a1640 1 -- 172.21.15.143:0/3158263339 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6b80067680 con 0x7f6b800ce680 2023-11-26T07:22:17.384 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:17.384+0000 7f6b777fe640 1 -- 172.21.15.143:0/3158263339 <== mon.0 v2:172.21.15.143:3300/0 4 ==== mgrmap(e 10) v1 ==== 86056+0+0 (secure 0 0 0) 0x7f6b680073d0 con 0x7f6b800ce680 2023-11-26T07:22:17.385 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:17.384+0000 7f6b777fe640 1 --2- 172.21.15.143:0/3158263339 >> [v2:172.21.15.143:6832/2150557544,v1:172.21.15.143:6833/2150557544] conn(0x7f6b600666b0 0x7f6b60068b70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:22:17.385 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:17.384+0000 7f6b777fe640 1 -- 172.21.15.143:0/3158263339 <== mon.0 v2:172.21.15.143:3300/0 5 ==== osd_map(59..59 src has 1..59) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f6b68083800 con 0x7f6b800ce680 2023-11-26T07:22:17.385 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:17.384+0000 7f6b7effd640 1 --2- 172.21.15.143:0/3158263339 >> [v2:172.21.15.143:6832/2150557544,v1:172.21.15.143:6833/2150557544] conn(0x7f6b600666b0 0x7f6b60068b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:22:17.386 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:17.384+0000 7f6b7effd640 1 --2- 172.21.15.143:0/3158263339 >> [v2:172.21.15.143:6832/2150557544,v1:172.21.15.143:6833/2150557544] conn(0x7f6b600666b0 0x7f6b60068b70 secure :-1 s=READY pgs=401 cs=0 l=1 rev1=1 crypto rx=0x7f6b700079c0 tx=0x7f6b70008040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:22:17.389 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:17.388+0000 7f6b777fe640 1 -- 172.21.15.143:0/3158263339 <== mon.0 v2:172.21.15.143:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172649 (secure 0 0 0) 0x7f6b68050930 con 0x7f6b800ce680 2023-11-26T07:22:17.538 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:17.536+0000 7f6b853a1640 1 -- 172.21.15.143:0/3158263339 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f6b800709f0 con 0x7f6b800ce680 2023-11-26T07:22:17.539 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:17.536+0000 7f6b777fe640 1 -- 172.21.15.143:0/3158263339 <== mon.0 v2:172.21.15.143:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v59) v1 ==== 74+0+16601 (secure 0 0 0) 0x7f6b68017090 con 0x7f6b800ce680 2023-11-26T07:22:17.539 INFO:teuthology.orchestra.run.smithi143.stdout: 2023-11-26T07:22:17.540 INFO:teuthology.orchestra.run.smithi143.stdout:{"epoch":59,"fsid":"f5033060-2469-4a07-95fe-2b76f5404bc1","created":"2023-11-26T07:17:28.613663+0000","modified":"2023-11-26T07:21:57.286555+0000","last_up_change":"2023-11-26T07:17:41.157813+0000","last_in_change":"2023-11-26T07:17:31.181658+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":9,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-11-26T07:17:41.758597+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":8,"pool_name":"cephfs_metadata","create_time":"2023-11-26T07:21:53.382407+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"58","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":9,"pool_name":"cephfs_data","create_time":"2023-11-26T07:21:54.656298+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"59","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.6299999952316284,"score_stable":1.6299999952316284,"optimal_score":1,"raw_score_acting":1.6299999952316284,"raw_score_stable":1.6299999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"ebbee2c6-023e-44b9-9f1d-6f7bfd309730","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":57,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6817","nonce":639807369},{"type":"v1","addr":"172.21.15.143:6819","nonce":639807369}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6821","nonce":639807369},{"type":"v1","addr":"172.21.15.143:6823","nonce":639807369}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6830","nonce":639807369},{"type":"v1","addr":"172.21.15.143:6831","nonce":639807369}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6825","nonce":639807369},{"type":"v1","addr":"172.21.15.143:6828","nonce":639807369}]},"public_addr":"172.21.15.143:6819/639807369","cluster_addr":"172.21.15.143:6823/639807369","heartbeat_back_addr":"172.21.15.143:6831/639807369","heartbeat_front_addr":"172.21.15.143:6828/639807369","state":["exists","up"]},{"osd":1,"uuid":"fec4b8be-7122-49e9-9a19-5850a0700001","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":57,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6816","nonce":1144942530},{"type":"v1","addr":"172.21.15.143:6818","nonce":1144942530}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6820","nonce":1144942530},{"type":"v1","addr":"172.21.15.143:6822","nonce":1144942530}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6827","nonce":1144942530},{"type":"v1","addr":"172.21.15.143:6829","nonce":1144942530}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6824","nonce":1144942530},{"type":"v1","addr":"172.21.15.143:6826","nonce":1144942530}]},"public_addr":"172.21.15.143:6818/1144942530","cluster_addr":"172.21.15.143:6822/1144942530","heartbeat_back_addr":"172.21.15.143:6829/1144942530","heartbeat_front_addr":"172.21.15.143:6826/1144942530","state":["exists","up"]},{"osd":2,"uuid":"2888af5b-f933-46ca-8f95-6dd494ff43f6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":57,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6808","nonce":3909095186},{"type":"v1","addr":"172.21.15.143:6809","nonce":3909095186}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6810","nonce":3909095186},{"type":"v1","addr":"172.21.15.143:6811","nonce":3909095186}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6814","nonce":3909095186},{"type":"v1","addr":"172.21.15.143:6815","nonce":3909095186}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6812","nonce":3909095186},{"type":"v1","addr":"172.21.15.143:6813","nonce":3909095186}]},"public_addr":"172.21.15.143:6809/3909095186","cluster_addr":"172.21.15.143:6811/3909095186","heartbeat_back_addr":"172.21.15.143:6815/3909095186","heartbeat_front_addr":"172.21.15.143:6813/3909095186","state":["exists","up"]},{"osd":3,"uuid":"d1f52435-7bc1-4081-9550-9cd1ba55f549","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":57,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6800","nonce":3854210991},{"type":"v1","addr":"172.21.15.143:6801","nonce":3854210991}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6802","nonce":3854210991},{"type":"v1","addr":"172.21.15.143:6803","nonce":3854210991}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6806","nonce":3854210991},{"type":"v1","addr":"172.21.15.143:6807","nonce":3854210991}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6804","nonce":3854210991},{"type":"v1","addr":"172.21.15.143:6805","nonce":3854210991}]},"public_addr":"172.21.15.143:6801/3854210991","cluster_addr":"172.21.15.143:6803/3854210991","heartbeat_back_addr":"172.21.15.143:6807/3854210991","heartbeat_front_addr":"172.21.15.143:6805/3854210991","state":["exists","up"]},{"osd":4,"uuid":"e2ae8b6e-6e25-49b2-bcbf-aacce68f06a3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":57,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6800","nonce":3924606902},{"type":"v1","addr":"172.21.15.188:6801","nonce":3924606902}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6802","nonce":3924606902},{"type":"v1","addr":"172.21.15.188:6803","nonce":3924606902}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6806","nonce":3924606902},{"type":"v1","addr":"172.21.15.188:6807","nonce":3924606902}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6804","nonce":3924606902},{"type":"v1","addr":"172.21.15.188:6805","nonce":3924606902}]},"public_addr":"172.21.15.188:6801/3924606902","cluster_addr":"172.21.15.188:6803/3924606902","heartbeat_back_addr":"172.21.15.188:6807/3924606902","heartbeat_front_addr":"172.21.15.188:6805/3924606902","state":["exists","up"]},{"osd":5,"uuid":"3cec8a32-7698-4c36-8e89-e4ba11934f33","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":57,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6813","nonce":186841544},{"type":"v1","addr":"172.21.15.188:6815","nonce":186841544}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6817","nonce":186841544},{"type":"v1","addr":"172.21.15.188:6818","nonce":186841544}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6822","nonce":186841544},{"type":"v1","addr":"172.21.15.188:6823","nonce":186841544}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6820","nonce":186841544},{"type":"v1","addr":"172.21.15.188:6821","nonce":186841544}]},"public_addr":"172.21.15.188:6815/186841544","cluster_addr":"172.21.15.188:6818/186841544","heartbeat_back_addr":"172.21.15.188:6823/186841544","heartbeat_front_addr":"172.21.15.188:6821/186841544","state":["exists","up"]},{"osd":6,"uuid":"58a65d5d-e090-473a-81b4-ad38e85029f9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":57,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6808","nonce":2153499405},{"type":"v1","addr":"172.21.15.188:6809","nonce":2153499405}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6810","nonce":2153499405},{"type":"v1","addr":"172.21.15.188:6811","nonce":2153499405}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6816","nonce":2153499405},{"type":"v1","addr":"172.21.15.188:6819","nonce":2153499405}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6812","nonce":2153499405},{"type":"v1","addr":"172.21.15.188:6814","nonce":2153499405}]},"public_addr":"172.21.15.188:6809/2153499405","cluster_addr":"172.21.15.188:6811/2153499405","heartbeat_back_addr":"172.21.15.188:6819/2153499405","heartbeat_front_addr":"172.21.15.188:6814/2153499405","state":["exists","up"]},{"osd":7,"uuid":"ea5b7f70-3554-4cf8-ac30-f7c59b80678d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":57,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6824","nonce":2722096729},{"type":"v1","addr":"172.21.15.188:6825","nonce":2722096729}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6826","nonce":2722096729},{"type":"v1","addr":"172.21.15.188:6827","nonce":2722096729}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6830","nonce":2722096729},{"type":"v1","addr":"172.21.15.188:6831","nonce":2722096729}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6828","nonce":2722096729},{"type":"v1","addr":"172.21.15.188:6829","nonce":2722096729}]},"public_addr":"172.21.15.188:6825/2722096729","cluster_addr":"172.21.15.188:6827/2722096729","heartbeat_back_addr":"172.21.15.188:6831/2722096729","heartbeat_front_addr":"172.21.15.188:6829/2722096729","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-26T07:17:36.995032+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-26T07:17:36.932266+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-26T07:17:36.829304+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-26T07:17:36.804297+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-26T07:17:36.667892+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-26T07:17:36.964868+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-26T07:17:36.827332+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-26T07:17:36.893179+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-11-26T07:22:17.541 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:17.540+0000 7f6b853a1640 1 -- 172.21.15.143:0/3158263339 >> [v2:172.21.15.143:6832/2150557544,v1:172.21.15.143:6833/2150557544] conn(0x7f6b600666b0 msgr2=0x7f6b60068b70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:22:17.542 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:17.540+0000 7f6b853a1640 1 --2- 172.21.15.143:0/3158263339 >> [v2:172.21.15.143:6832/2150557544,v1:172.21.15.143:6833/2150557544] conn(0x7f6b600666b0 0x7f6b60068b70 secure :-1 s=READY pgs=401 cs=0 l=1 rev1=1 crypto rx=0x7f6b700079c0 tx=0x7f6b70008040 comp rx=0 tx=0).stop 2023-11-26T07:22:17.542 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:17.540+0000 7f6b853a1640 1 -- 172.21.15.143:0/3158263339 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f6b800ce680 msgr2=0x7f6b801f9ee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:22:17.542 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:17.540+0000 7f6b853a1640 1 --2- 172.21.15.143:0/3158263339 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f6b800ce680 0x7f6b801f9ee0 secure :-1 s=READY pgs=853 cs=0 l=1 rev1=1 crypto rx=0x7f6b6800c4d0 tx=0x7f6b6800c990 comp rx=0 tx=0).stop 2023-11-26T07:22:17.542 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:17.540+0000 7f6b853a1640 1 -- 172.21.15.143:0/3158263339 shutdown_connections 2023-11-26T07:22:17.542 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:17.540+0000 7f6b853a1640 1 --2- 172.21.15.143:0/3158263339 >> [v2:172.21.15.143:6832/2150557544,v1:172.21.15.143:6833/2150557544] conn(0x7f6b600666b0 0x7f6b60068b70 unknown :-1 s=CLOSED pgs=401 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:22:17.542 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:17.540+0000 7f6b853a1640 1 --2- 172.21.15.143:0/3158263339 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f6b801596d0 0x7f6b80147020 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:22:17.542 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:17.540+0000 7f6b853a1640 1 --2- 172.21.15.143:0/3158263339 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f6b800ce680 0x7f6b801f9ee0 unknown :-1 s=CLOSED pgs=853 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:22:17.542 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:17.540+0000 7f6b853a1640 1 --2- 172.21.15.143:0/3158263339 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f6b800b2f80 0x7f6b800cbf40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:22:17.542 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:17.540+0000 7f6b853a1640 1 -- 172.21.15.143:0/3158263339 >> 172.21.15.143:0/3158263339 conn(0x7f6b8006a890 msgr2=0x7f6b800b00d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:22:17.542 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:17.540+0000 7f6b853a1640 1 -- 172.21.15.143:0/3158263339 shutdown_connections 2023-11-26T07:22:17.542 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:17.540+0000 7f6b853a1640 1 -- 172.21.15.143:0/3158263339 wait complete. 2023-11-26T07:22:17.561 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-11-26T07:17:41.758597+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 8, 'pool_name': 'cephfs_metadata', 'create_time': '2023-11-26T07:21:53.382407+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '58', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 9, 'pool_name': 'cephfs_data', 'create_time': '2023-11-26T07:21:54.656298+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '59', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.6299999952316284, 'score_stable': 1.6299999952316284, 'optimal_score': 1, 'raw_score_acting': 1.6299999952316284, 'raw_score_stable': 1.6299999952316284, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2023-11-26T07:22:17.561 DEBUG:teuthology.orchestra.run.smithi143:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-11-26T07:22:17.702 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:17.700+0000 7fe0775aa640 1 Processor -- start 2023-11-26T07:22:17.702 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:17.700+0000 7fe0775aa640 1 -- start start 2023-11-26T07:22:17.704 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:17.700+0000 7fe0775aa640 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7fe0700712b0 0x7fe070071970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:22:17.704 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:17.700+0000 7fe0775aa640 1 -- --> v1:172.21.15.143:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe070071eb0 con 0x7fe0700cd610 2023-11-26T07:22:17.704 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:17.700+0000 7fe0775aa640 1 -- --> v1:172.21.15.188:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe07015d970 con 0x7fe0701614b0 2023-11-26T07:22:17.704 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:17.700+0000 7fe0775aa640 1 -- --> v2:172.21.15.188:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe07015dc30 con 0x7fe0700712b0 2023-11-26T07:22:17.704 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:17.700+0000 7fe074b45640 1 --1- >> v1:172.21.15.143:6789/0 conn(0x7fe0700cd610 0x7fe070070ba0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.143:6789/0 says I am v1:172.21.15.143:59142/0 (socket says 172.21.15.143:59142) 2023-11-26T07:22:17.704 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:17.704+0000 7fe074b45640 1 -- 172.21.15.143:0/3033041584 learned_addr learned my addr 172.21.15.143:0/3033041584 (peer_addr_for_me v1:172.21.15.143:0/0) 2023-11-26T07:22:17.706 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:17.704+0000 7fe075b47640 1 --2- 172.21.15.143:0/3033041584 >> v2:172.21.15.188:3300/0 conn(0x7fe0700712b0 0x7fe070071970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:22:17.706 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:17.704+0000 7fe067fff640 1 -- 172.21.15.143:0/3033041584 <== mon.0 v1:172.21.15.143:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2091281754 0 0) 0x7fe070071eb0 con 0x7fe0700cd610 2023-11-26T07:22:17.706 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:17.704+0000 7fe067fff640 1 -- 172.21.15.143:0/3033041584 --> v1:172.21.15.143:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fe05c003530 con 0x7fe0700cd610 2023-11-26T07:22:17.706 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:17.704+0000 7fe067fff640 1 -- 172.21.15.143:0/3033041584 <== mon.0 v1:172.21.15.143:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3956851786 0 0) 0x7fe05c003530 con 0x7fe0700cd610 2023-11-26T07:22:17.706 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:17.704+0000 7fe067fff640 1 -- 172.21.15.143:0/3033041584 >> v2:172.21.15.188:3300/0 conn(0x7fe0700712b0 msgr2=0x7fe070071970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-26T07:22:17.706 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:17.704+0000 7fe067fff640 1 --2- 172.21.15.143:0/3033041584 >> v2:172.21.15.188:3300/0 conn(0x7fe0700712b0 0x7fe070071970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:22:17.706 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:17.704+0000 7fe067fff640 1 -- 172.21.15.143:0/3033041584 >> v1:172.21.15.188:6789/0 conn(0x7fe0701614b0 legacy=0x7fe070161890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:22:17.706 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:17.704+0000 7fe067fff640 1 -- 172.21.15.143:0/3033041584 --> v1:172.21.15.143:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe07015df10 con 0x7fe0700cd610 2023-11-26T07:22:17.706 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:17.704+0000 7fe067fff640 1 -- 172.21.15.143:0/3033041584 <== mon.0 v1:172.21.15.143:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2137882523 0 0) 0x7fe0600016a0 con 0x7fe0700cd610 2023-11-26T07:22:17.706 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:17.704+0000 7fe067fff640 1 -- 172.21.15.143:0/3033041584 >> v1:172.21.15.143:6789/0 conn(0x7fe0700cd610 legacy=0x7fe070070ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:22:17.707 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:17.704+0000 7fe067fff640 1 --2- 172.21.15.143:0/3033041584 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fe05c003980 0x7fe05c003da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:22:17.707 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:17.704+0000 7fe067fff640 1 --2- 172.21.15.143:0/3033041584 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fe05c0047d0 0x7fe05c004c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:22:17.707 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:17.704+0000 7fe067fff640 1 --2- 172.21.15.143:0/3033041584 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fe05c005190 0x7fe05c00b990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:22:17.707 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:17.704+0000 7fe067fff640 1 -- 172.21.15.143:0/3033041584 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe07015dc30 con 0x7fe05c003980 2023-11-26T07:22:17.707 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:17.704+0000 7fe067fff640 1 -- 172.21.15.143:0/3033041584 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe05c00c030 con 0x7fe05c0047d0 2023-11-26T07:22:17.707 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:17.704+0000 7fe067fff640 1 -- 172.21.15.143:0/3033041584 --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe05c00c1f0 con 0x7fe05c005190 2023-11-26T07:22:17.707 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:17.704+0000 7fe075346640 1 --2- 172.21.15.143:0/3033041584 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fe05c003980 0x7fe05c003da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:22:17.707 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:17.704+0000 7fe074b45640 1 --2- 172.21.15.143:0/3033041584 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fe05c0047d0 0x7fe05c004c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:22:17.707 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:17.704+0000 7fe075b47640 1 --2- 172.21.15.143:0/3033041584 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fe05c005190 0x7fe05c00b990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:22:17.707 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:17.704+0000 7fe075346640 1 -- 172.21.15.143:0/3033041584 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fe05c005190 msgr2=0x7fe05c00b990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-26T07:22:17.707 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:17.704+0000 7fe075346640 1 --2- 172.21.15.143:0/3033041584 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fe05c005190 0x7fe05c00b990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:22:17.707 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:17.704+0000 7fe075346640 1 -- 172.21.15.143:0/3033041584 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fe05c0047d0 msgr2=0x7fe05c004c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-26T07:22:17.708 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:17.704+0000 7fe075346640 1 --2- 172.21.15.143:0/3033041584 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fe05c0047d0 0x7fe05c004c50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:22:17.708 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:17.704+0000 7fe075346640 1 -- 172.21.15.143:0/3033041584 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe05c003530 con 0x7fe05c003980 2023-11-26T07:22:17.708 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:17.704+0000 7fe075b47640 1 -- 172.21.15.143:0/3033041584 reap_dead start 2023-11-26T07:22:17.708 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:17.708+0000 7fe075346640 1 --2- 172.21.15.143:0/3033041584 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fe05c003980 0x7fe05c003da0 secure :-1 s=READY pgs=855 cs=0 l=1 rev1=1 crypto rx=0x7fe06c001780 tx=0x7fe06c01c3d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=730ded70e28933ed server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:22:17.708 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:17.708+0000 7fe067fff640 1 -- 172.21.15.143:0/3033041584 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe06c0052d0 con 0x7fe05c003980 2023-11-26T07:22:17.708 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:17.708+0000 7fe067fff640 1 -- 172.21.15.143:0/3033041584 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe06c0a7050 con 0x7fe05c003980 2023-11-26T07:22:17.709 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:17.708+0000 7fe067fff640 1 -- 172.21.15.143:0/3033041584 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe06c003600 con 0x7fe05c003980 2023-11-26T07:22:17.709 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:17.708+0000 7fe0775aa640 1 -- 172.21.15.143:0/3033041584 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fe05c003980 msgr2=0x7fe05c003da0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:22:17.709 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:17.708+0000 7fe0775aa640 1 --2- 172.21.15.143:0/3033041584 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fe05c003980 0x7fe05c003da0 secure :-1 s=READY pgs=855 cs=0 l=1 rev1=1 crypto rx=0x7fe06c001780 tx=0x7fe06c01c3d0 comp rx=0 tx=0).stop 2023-11-26T07:22:17.709 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:17.708+0000 7fe0775aa640 1 -- 172.21.15.143:0/3033041584 shutdown_connections 2023-11-26T07:22:17.709 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:17.708+0000 7fe0775aa640 1 --2- 172.21.15.143:0/3033041584 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fe05c003980 0x7fe05c003da0 unknown :-1 s=CLOSED pgs=855 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:22:17.710 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:17.708+0000 7fe0775aa640 1 -- 172.21.15.143:0/3033041584 >> 172.21.15.143:0/3033041584 conn(0x7fe0700cf710 msgr2=0x7fe0700cfb20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:22:17.710 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:17.708+0000 7fe0775aa640 1 -- 172.21.15.143:0/3033041584 shutdown_connections 2023-11-26T07:22:17.710 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:17.708+0000 7fe0775aa640 1 -- 172.21.15.143:0/3033041584 wait complete. 2023-11-26T07:22:17.711 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:17.708+0000 7fe0775aa640 1 Processor -- start 2023-11-26T07:22:17.711 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:17.708+0000 7fe0775aa640 1 -- start start 2023-11-26T07:22:17.712 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:17.712+0000 7fe0775aa640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fe05c003980 0x7fe0700b28d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:22:17.713 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:17.712+0000 7fe0775aa640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fe070158c50 0x7fe0700cbc10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:22:17.713 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:17.712+0000 7fe0775aa640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fe0701f6510 0x7fe07015eaa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:22:17.713 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:17.712+0000 7fe0775aa640 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe0700b2eb0 con 0x7fe070158c50 2023-11-26T07:22:17.713 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:17.712+0000 7fe0775aa640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe0700b3190 con 0x7fe05c003980 2023-11-26T07:22:17.713 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:17.712+0000 7fe0775aa640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe0700b3470 con 0x7fe0701f6510 2023-11-26T07:22:17.713 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:17.712+0000 7fe074b45640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fe070158c50 0x7fe0700cbc10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:22:17.713 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:17.712+0000 7fe074b45640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fe070158c50 0x7fe0700cbc10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:34224/0 (socket says 172.21.15.143:34224) 2023-11-26T07:22:17.713 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:17.712+0000 7fe074b45640 1 -- 172.21.15.143:0/3813119219 learned_addr learned my addr 172.21.15.143:0/3813119219 (peer_addr_for_me v2:172.21.15.143:0/0) 2023-11-26T07:22:17.713 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:17.712+0000 7fe075346640 1 --2- 172.21.15.143:0/3813119219 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fe05c003980 0x7fe0700b28d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:22:17.713 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:17.712+0000 7fe075b47640 1 --2- 172.21.15.143:0/3813119219 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fe0701f6510 0x7fe07015eaa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:22:17.713 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:17.712+0000 7fe074b45640 1 -- 172.21.15.143:0/3813119219 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fe0701f6510 msgr2=0x7fe07015eaa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:22:17.713 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:17.712+0000 7fe074b45640 1 --2- 172.21.15.143:0/3813119219 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fe0701f6510 0x7fe07015eaa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:22:17.714 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:17.712+0000 7fe074b45640 1 -- 172.21.15.143:0/3813119219 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fe05c003980 msgr2=0x7fe0700b28d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:22:17.714 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:17.712+0000 7fe074b45640 1 --2- 172.21.15.143:0/3813119219 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fe05c003980 0x7fe0700b28d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:22:17.714 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:17.712+0000 7fe074b45640 1 -- 172.21.15.143:0/3813119219 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe0700b3750 con 0x7fe070158c50 2023-11-26T07:22:17.714 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:17.712+0000 7fe074b45640 1 --2- 172.21.15.143:0/3813119219 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fe070158c50 0x7fe0700cbc10 secure :-1 s=READY pgs=856 cs=0 l=1 rev1=1 crypto rx=0x7fe060001e70 tx=0x7fe060007720 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:22:17.715 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:17.712+0000 7fe065ffb640 1 -- 172.21.15.143:0/3813119219 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe060001690 con 0x7fe070158c50 2023-11-26T07:22:17.715 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:17.712+0000 7fe065ffb640 1 -- 172.21.15.143:0/3813119219 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe06000f9c0 con 0x7fe070158c50 2023-11-26T07:22:17.715 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:17.712+0000 7fe065ffb640 1 -- 172.21.15.143:0/3813119219 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe06000fd70 con 0x7fe070158c50 2023-11-26T07:22:17.715 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:17.712+0000 7fe0775aa640 1 -- 172.21.15.143:0/3813119219 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe0700b3a30 con 0x7fe070158c50 2023-11-26T07:22:17.715 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:17.712+0000 7fe0775aa640 1 -- 172.21.15.143:0/3813119219 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe0701590d0 con 0x7fe070158c50 2023-11-26T07:22:17.715 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:17.712+0000 7fe0775aa640 1 -- 172.21.15.143:0/3813119219 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe0701592f0 con 0x7fe070158c50 2023-11-26T07:22:17.715 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:17.712+0000 7fe065ffb640 1 -- 172.21.15.143:0/3813119219 <== mon.0 v2:172.21.15.143:3300/0 4 ==== mgrmap(e 10) v1 ==== 86056+0+0 (secure 0 0 0) 0x7fe06001d460 con 0x7fe070158c50 2023-11-26T07:22:17.716 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:17.716+0000 7fe065ffb640 1 --2- 172.21.15.143:0/3813119219 >> [v2:172.21.15.143:6832/2150557544,v1:172.21.15.143:6833/2150557544] conn(0x7fe0540666e0 0x7fe054068ba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:22:17.716 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:17.716+0000 7fe065ffb640 1 -- 172.21.15.143:0/3813119219 <== mon.0 v2:172.21.15.143:3300/0 5 ==== osd_map(59..59 src has 1..59) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fe060084360 con 0x7fe070158c50 2023-11-26T07:22:17.716 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:17.716+0000 7fe075346640 1 --2- 172.21.15.143:0/3813119219 >> [v2:172.21.15.143:6832/2150557544,v1:172.21.15.143:6833/2150557544] conn(0x7fe0540666e0 0x7fe054068ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:22:17.716 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:17.716+0000 7fe075346640 1 --2- 172.21.15.143:0/3813119219 >> [v2:172.21.15.143:6832/2150557544,v1:172.21.15.143:6833/2150557544] conn(0x7fe0540666e0 0x7fe054068ba0 secure :-1 s=READY pgs=402 cs=0 l=1 rev1=1 crypto rx=0x7fe06c001750 tx=0x7fe06c0011a0 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:22:17.719 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:17.716+0000 7fe065ffb640 1 -- 172.21.15.143:0/3813119219 <== mon.0 v2:172.21.15.143:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172649 (secure 0 0 0) 0x7fe0600513e0 con 0x7fe070158c50 2023-11-26T07:22:17.871 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:17.868+0000 7fe0775aa640 1 -- 172.21.15.143:0/3813119219 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7fe0700b3c70 con 0x7fe070158c50 2023-11-26T07:22:17.872 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:17.872+0000 7fe065ffb640 1 -- 172.21.15.143:0/3813119219 <== mon.0 v2:172.21.15.143:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v59) v1 ==== 93+0+10 (secure 0 0 0) 0x7fe0700b3c70 con 0x7fe070158c50 2023-11-26T07:22:17.872 INFO:teuthology.orchestra.run.smithi143.stdout:pg_num: 1 2023-11-26T07:22:17.874 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:17.872+0000 7fe0775aa640 1 -- 172.21.15.143:0/3813119219 >> [v2:172.21.15.143:6832/2150557544,v1:172.21.15.143:6833/2150557544] conn(0x7fe0540666e0 msgr2=0x7fe054068ba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:22:17.875 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:17.872+0000 7fe0775aa640 1 --2- 172.21.15.143:0/3813119219 >> [v2:172.21.15.143:6832/2150557544,v1:172.21.15.143:6833/2150557544] conn(0x7fe0540666e0 0x7fe054068ba0 secure :-1 s=READY pgs=402 cs=0 l=1 rev1=1 crypto rx=0x7fe06c001750 tx=0x7fe06c0011a0 comp rx=0 tx=0).stop 2023-11-26T07:22:17.875 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:17.872+0000 7fe0775aa640 1 -- 172.21.15.143:0/3813119219 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fe070158c50 msgr2=0x7fe0700cbc10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:22:17.875 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:17.872+0000 7fe0775aa640 1 --2- 172.21.15.143:0/3813119219 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fe070158c50 0x7fe0700cbc10 secure :-1 s=READY pgs=856 cs=0 l=1 rev1=1 crypto rx=0x7fe060001e70 tx=0x7fe060007720 comp rx=0 tx=0).stop 2023-11-26T07:22:17.875 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:17.872+0000 7fe0775aa640 1 -- 172.21.15.143:0/3813119219 shutdown_connections 2023-11-26T07:22:17.875 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:17.872+0000 7fe0775aa640 1 --2- 172.21.15.143:0/3813119219 >> [v2:172.21.15.143:6832/2150557544,v1:172.21.15.143:6833/2150557544] conn(0x7fe0540666e0 0x7fe054068ba0 unknown :-1 s=CLOSED pgs=402 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:22:17.875 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:17.872+0000 7fe0775aa640 1 --2- 172.21.15.143:0/3813119219 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fe0701f6510 0x7fe07015eaa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:22:17.875 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:17.872+0000 7fe0775aa640 1 --2- 172.21.15.143:0/3813119219 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fe070158c50 0x7fe0700cbc10 unknown :-1 s=CLOSED pgs=856 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:22:17.875 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:17.872+0000 7fe0775aa640 1 --2- 172.21.15.143:0/3813119219 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fe05c003980 0x7fe0700b28d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:22:17.875 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:17.872+0000 7fe0775aa640 1 -- 172.21.15.143:0/3813119219 >> 172.21.15.143:0/3813119219 conn(0x7fe0700cf710 msgr2=0x7fe0700af4a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:22:17.875 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:17.872+0000 7fe0775aa640 1 -- 172.21.15.143:0/3813119219 shutdown_connections 2023-11-26T07:22:17.875 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:17.872+0000 7fe0775aa640 1 -- 172.21.15.143:0/3813119219 wait complete. 2023-11-26T07:22:17.893 DEBUG:teuthology.orchestra.run.smithi143:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-11-26T07:22:18.034 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:18.032+0000 7fb5ca165640 1 Processor -- start 2023-11-26T07:22:18.034 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:18.032+0000 7fb5ca165640 1 -- start start 2023-11-26T07:22:18.036 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:18.032+0000 7fb5ca165640 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7fb5c4143ac0 0x7fb5c4070370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:22:18.036 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:18.032+0000 7fb5ca165640 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7fb5c40708b0 0x7fb5c40cf280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:22:18.036 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:18.032+0000 7fb5ca165640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fb5c40cf7c0 0x7fb5c40b1ef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:22:18.036 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:18.032+0000 7fb5ca165640 1 -- --> v2:172.21.15.143:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb5c4143ea0 con 0x7fb5c4143ac0 2023-11-26T07:22:18.036 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:18.032+0000 7fb5ca165640 1 -- --> v2:172.21.15.188:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb5c40cc730 con 0x7fb5c40708b0 2023-11-26T07:22:18.036 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:18.032+0000 7fb5ca165640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb5c40cfbd0 con 0x7fb5c40cf7c0 2023-11-26T07:22:18.037 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:18.032+0000 7fb5c37fe640 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7fb5c4143ac0 0x7fb5c4070370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:22:18.037 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:18.032+0000 7fb5c37fe640 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7fb5c4143ac0 0x7fb5c4070370 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:34236/0 (socket says 172.21.15.143:34236) 2023-11-26T07:22:18.037 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:18.032+0000 7fb5c37fe640 1 -- 172.21.15.143:0/2223150844 learned_addr learned my addr 172.21.15.143:0/2223150844 (peer_addr_for_me v2:172.21.15.143:0/0) 2023-11-26T07:22:18.037 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:18.036+0000 7fb5c3fff640 1 --2- 172.21.15.143:0/2223150844 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fb5c40cf7c0 0x7fb5c40b1ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:22:18.037 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:18.036+0000 7fb5bbfff640 1 --2- 172.21.15.143:0/2223150844 >> v2:172.21.15.188:3300/0 conn(0x7fb5c40708b0 0x7fb5c40cf280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:22:18.037 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:18.036+0000 7fb5c37fe640 1 -- 172.21.15.143:0/2223150844 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fb5c40cf7c0 msgr2=0x7fb5c40b1ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-26T07:22:18.037 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:18.036+0000 7fb5c37fe640 1 --2- 172.21.15.143:0/2223150844 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fb5c40cf7c0 0x7fb5c40b1ef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:22:18.037 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:18.036+0000 7fb5c37fe640 1 -- 172.21.15.143:0/2223150844 >> v2:172.21.15.188:3300/0 conn(0x7fb5c40708b0 msgr2=0x7fb5c40cf280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-26T07:22:18.037 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:18.036+0000 7fb5c37fe640 1 --2- 172.21.15.143:0/2223150844 >> v2:172.21.15.188:3300/0 conn(0x7fb5c40708b0 0x7fb5c40cf280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:22:18.037 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:18.036+0000 7fb5c37fe640 1 -- 172.21.15.143:0/2223150844 --> v2:172.21.15.143:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb5c4070ee0 con 0x7fb5c4143ac0 2023-11-26T07:22:18.037 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:18.036+0000 7fb5bbfff640 1 --2- 172.21.15.143:0/2223150844 >> v2:172.21.15.188:3300/0 conn(0x7fb5c40708b0 0x7fb5c40cf280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-26T07:22:18.039 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:18.036+0000 7fb5c37fe640 1 --2- 172.21.15.143:0/2223150844 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fb5c4143ac0 0x7fb5c4070370 secure :-1 s=READY pgs=857 cs=0 l=1 rev1=1 crypto rx=0x7fb5c40aeb10 tx=0x7fb5b401c240 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9670986e9a8f014c server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:22:18.039 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:18.036+0000 7fb5c2ffd640 1 -- 172.21.15.143:0/2223150844 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb5b40ab600 con 0x7fb5c4143ac0 2023-11-26T07:22:18.039 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:18.036+0000 7fb5c2ffd640 1 -- 172.21.15.143:0/2223150844 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb5b40a7050 con 0x7fb5c4143ac0 2023-11-26T07:22:18.040 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:18.036+0000 7fb5c2ffd640 1 -- 172.21.15.143:0/2223150844 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb5b40ab980 con 0x7fb5c4143ac0 2023-11-26T07:22:18.040 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:18.040+0000 7fb5ca165640 1 -- 172.21.15.143:0/2223150844 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fb5c4143ac0 msgr2=0x7fb5c4070370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:22:18.040 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:18.040+0000 7fb5ca165640 1 --2- 172.21.15.143:0/2223150844 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fb5c4143ac0 0x7fb5c4070370 secure :-1 s=READY pgs=857 cs=0 l=1 rev1=1 crypto rx=0x7fb5c40aeb10 tx=0x7fb5b401c240 comp rx=0 tx=0).stop 2023-11-26T07:22:18.040 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:18.040+0000 7fb5ca165640 1 -- 172.21.15.143:0/2223150844 shutdown_connections 2023-11-26T07:22:18.041 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:18.040+0000 7fb5ca165640 1 --2- 172.21.15.143:0/2223150844 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fb5c40cf7c0 0x7fb5c40b1ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:22:18.041 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:18.040+0000 7fb5ca165640 1 --2- 172.21.15.143:0/2223150844 >> v2:172.21.15.188:3300/0 conn(0x7fb5c40708b0 0x7fb5c40cf280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:22:18.041 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:18.040+0000 7fb5ca165640 1 --2- 172.21.15.143:0/2223150844 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fb5c4143ac0 0x7fb5c4070370 unknown :-1 s=CLOSED pgs=857 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:22:18.041 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:18.040+0000 7fb5ca165640 1 -- 172.21.15.143:0/2223150844 >> 172.21.15.143:0/2223150844 conn(0x7fb5c406a890 msgr2=0x7fb5c406aca0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:22:18.041 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:18.040+0000 7fb5ca165640 1 -- 172.21.15.143:0/2223150844 shutdown_connections 2023-11-26T07:22:18.041 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:18.040+0000 7fb5ca165640 1 -- 172.21.15.143:0/2223150844 wait complete. 2023-11-26T07:22:18.042 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:18.040+0000 7fb5ca165640 1 Processor -- start 2023-11-26T07:22:18.042 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:18.040+0000 7fb5ca165640 1 -- start start 2023-11-26T07:22:18.043 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:18.040+0000 7fb5ca165640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fb5c40708b0 0x7fb5c40b1470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:22:18.043 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:18.040+0000 7fb5ca165640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fb5c40cf7c0 0x7fb5c4159140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:22:18.044 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:18.040+0000 7fb5ca165640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fb5c4143ac0 0x7fb5c40b0170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:22:18.044 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:18.040+0000 7fb5ca165640 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb5c41419e0 con 0x7fb5c4143ac0 2023-11-26T07:22:18.044 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:18.040+0000 7fb5ca165640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb5c4140ac0 con 0x7fb5c40cf7c0 2023-11-26T07:22:18.044 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:18.040+0000 7fb5ca165640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb5c4140d60 con 0x7fb5c40708b0 2023-11-26T07:22:18.044 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:18.040+0000 7fb5c3fff640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fb5c4143ac0 0x7fb5c40b0170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:22:18.044 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:18.040+0000 7fb5c3fff640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fb5c4143ac0 0x7fb5c40b0170 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:34242/0 (socket says 172.21.15.143:34242) 2023-11-26T07:22:18.044 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:18.040+0000 7fb5c3fff640 1 -- 172.21.15.143:0/553126099 learned_addr learned my addr 172.21.15.143:0/553126099 (peer_addr_for_me v2:172.21.15.143:0/0) 2023-11-26T07:22:18.044 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:18.040+0000 7fb5bbfff640 1 --2- 172.21.15.143:0/553126099 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fb5c40cf7c0 0x7fb5c4159140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:22:18.044 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:18.040+0000 7fb5c37fe640 1 --2- 172.21.15.143:0/553126099 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fb5c40708b0 0x7fb5c40b1470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:22:18.044 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:18.040+0000 7fb5c3fff640 1 -- 172.21.15.143:0/553126099 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fb5c40708b0 msgr2=0x7fb5c40b1470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:22:18.044 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:18.040+0000 7fb5c3fff640 1 --2- 172.21.15.143:0/553126099 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fb5c40708b0 0x7fb5c40b1470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:22:18.045 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:18.040+0000 7fb5c3fff640 1 -- 172.21.15.143:0/553126099 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fb5c40cf7c0 msgr2=0x7fb5c4159140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:22:18.045 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:18.040+0000 7fb5c3fff640 1 --2- 172.21.15.143:0/553126099 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fb5c40cf7c0 0x7fb5c4159140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:22:18.045 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:18.040+0000 7fb5c3fff640 1 -- 172.21.15.143:0/553126099 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb5b40094d0 con 0x7fb5c4143ac0 2023-11-26T07:22:18.045 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:18.040+0000 7fb5c37fe640 1 --2- 172.21.15.143:0/553126099 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fb5c40708b0 0x7fb5c40b1470 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-26T07:22:18.046 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:18.044+0000 7fb5c3fff640 1 --2- 172.21.15.143:0/553126099 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fb5c4143ac0 0x7fb5c40b0170 secure :-1 s=READY pgs=858 cs=0 l=1 rev1=1 crypto rx=0x7fb5bc00efc0 tx=0x7fb5bc00c490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:22:18.046 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:18.044+0000 7fb5c0ff9640 1 -- 172.21.15.143:0/553126099 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb5bc014480 con 0x7fb5c4143ac0 2023-11-26T07:22:18.046 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:18.044+0000 7fb5c0ff9640 1 -- 172.21.15.143:0/553126099 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb5bc00f040 con 0x7fb5c4143ac0 2023-11-26T07:22:18.046 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:18.044+0000 7fb5ca165640 1 -- 172.21.15.143:0/553126099 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb5c4141040 con 0x7fb5c4143ac0 2023-11-26T07:22:18.046 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:18.044+0000 7fb5c0ff9640 1 -- 172.21.15.143:0/553126099 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb5bc014820 con 0x7fb5c4143ac0 2023-11-26T07:22:18.046 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:18.044+0000 7fb5ca165640 1 -- 172.21.15.143:0/553126099 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb5c4141600 con 0x7fb5c4143ac0 2023-11-26T07:22:18.046 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:18.044+0000 7fb5ca165640 1 -- 172.21.15.143:0/553126099 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb5c4067680 con 0x7fb5c4143ac0 2023-11-26T07:22:18.047 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:18.044+0000 7fb5c0ff9640 1 -- 172.21.15.143:0/553126099 <== mon.0 v2:172.21.15.143:3300/0 4 ==== mgrmap(e 10) v1 ==== 86056+0+0 (secure 0 0 0) 0x7fb5bc014980 con 0x7fb5c4143ac0 2023-11-26T07:22:18.047 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:18.044+0000 7fb5c0ff9640 1 --2- 172.21.15.143:0/553126099 >> [v2:172.21.15.143:6832/2150557544,v1:172.21.15.143:6833/2150557544] conn(0x7fb5a40666b0 0x7fb5a4068b70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:22:18.047 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:18.044+0000 7fb5c0ff9640 1 -- 172.21.15.143:0/553126099 <== mon.0 v2:172.21.15.143:3300/0 5 ==== osd_map(59..59 src has 1..59) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fb5bc085120 con 0x7fb5c4143ac0 2023-11-26T07:22:18.047 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:18.044+0000 7fb5c37fe640 1 --2- 172.21.15.143:0/553126099 >> [v2:172.21.15.143:6832/2150557544,v1:172.21.15.143:6833/2150557544] conn(0x7fb5a40666b0 0x7fb5a4068b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:22:18.048 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:18.048+0000 7fb5c37fe640 1 --2- 172.21.15.143:0/553126099 >> [v2:172.21.15.143:6832/2150557544,v1:172.21.15.143:6833/2150557544] conn(0x7fb5a40666b0 0x7fb5a4068b70 secure :-1 s=READY pgs=403 cs=0 l=1 rev1=1 crypto rx=0x7fb5b40ad000 tx=0x7fb5b4008040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:22:18.051 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:18.048+0000 7fb5c0ff9640 1 -- 172.21.15.143:0/553126099 <== mon.0 v2:172.21.15.143:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172649 (secure 0 0 0) 0x7fb5bc0521c0 con 0x7fb5c4143ac0 2023-11-26T07:22:18.199 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:18.196+0000 7fb5ca165640 1 -- 172.21.15.143:0/553126099 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7fb5c40aec00 con 0x7fb5c4143ac0 2023-11-26T07:22:18.199 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:18.196+0000 7fb5c0ff9640 1 -- 172.21.15.143:0/553126099 <== mon.0 v2:172.21.15.143:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v59) v1 ==== 104+0+11 (secure 0 0 0) 0x7fb5bc0557e0 con 0x7fb5c4143ac0 2023-11-26T07:22:18.199 INFO:teuthology.orchestra.run.smithi143.stdout:pg_num: 32 2023-11-26T07:22:18.201 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:18.200+0000 7fb5ca165640 1 -- 172.21.15.143:0/553126099 >> [v2:172.21.15.143:6832/2150557544,v1:172.21.15.143:6833/2150557544] conn(0x7fb5a40666b0 msgr2=0x7fb5a4068b70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:22:18.201 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:18.200+0000 7fb5ca165640 1 --2- 172.21.15.143:0/553126099 >> [v2:172.21.15.143:6832/2150557544,v1:172.21.15.143:6833/2150557544] conn(0x7fb5a40666b0 0x7fb5a4068b70 secure :-1 s=READY pgs=403 cs=0 l=1 rev1=1 crypto rx=0x7fb5b40ad000 tx=0x7fb5b4008040 comp rx=0 tx=0).stop 2023-11-26T07:22:18.201 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:18.200+0000 7fb5ca165640 1 -- 172.21.15.143:0/553126099 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fb5c4143ac0 msgr2=0x7fb5c40b0170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:22:18.202 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:18.200+0000 7fb5ca165640 1 --2- 172.21.15.143:0/553126099 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fb5c4143ac0 0x7fb5c40b0170 secure :-1 s=READY pgs=858 cs=0 l=1 rev1=1 crypto rx=0x7fb5bc00efc0 tx=0x7fb5bc00c490 comp rx=0 tx=0).stop 2023-11-26T07:22:18.202 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:18.200+0000 7fb5ca165640 1 -- 172.21.15.143:0/553126099 shutdown_connections 2023-11-26T07:22:18.202 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:18.200+0000 7fb5ca165640 1 --2- 172.21.15.143:0/553126099 >> [v2:172.21.15.143:6832/2150557544,v1:172.21.15.143:6833/2150557544] conn(0x7fb5a40666b0 0x7fb5a4068b70 unknown :-1 s=CLOSED pgs=403 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:22:18.202 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:18.200+0000 7fb5ca165640 1 --2- 172.21.15.143:0/553126099 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fb5c4143ac0 0x7fb5c40b0170 unknown :-1 s=CLOSED pgs=858 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:22:18.202 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:18.200+0000 7fb5ca165640 1 --2- 172.21.15.143:0/553126099 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fb5c40cf7c0 0x7fb5c4159140 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:22:18.202 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:18.200+0000 7fb5ca165640 1 --2- 172.21.15.143:0/553126099 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fb5c40708b0 0x7fb5c40b1470 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:22:18.202 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:18.200+0000 7fb5ca165640 1 -- 172.21.15.143:0/553126099 >> 172.21.15.143:0/553126099 conn(0x7fb5c406a890 msgr2=0x7fb5c40cd800 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:22:18.202 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:18.200+0000 7fb5ca165640 1 -- 172.21.15.143:0/553126099 shutdown_connections 2023-11-26T07:22:18.202 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:18.200+0000 7fb5ca165640 1 -- 172.21.15.143:0/553126099 wait complete. 2023-11-26T07:22:18.220 DEBUG:teuthology.orchestra.run.smithi143:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-11-26T07:22:18.360 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:18.356+0000 7f26cd0ea640 1 Processor -- start 2023-11-26T07:22:18.360 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:18.360+0000 7f26cd0ea640 1 -- start start 2023-11-26T07:22:18.361 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:18.360+0000 7f26cd0ea640 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7f26c806cd50 0x7f26c80b2180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:22:18.361 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:18.360+0000 7f26cd0ea640 1 -- --> v1:172.21.15.143:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f26c815b2f0 con 0x7f26c8070d10 2023-11-26T07:22:18.361 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:18.360+0000 7f26cd0ea640 1 -- --> v1:172.21.15.188:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f26c806c5c0 con 0x7f26c80cf280 2023-11-26T07:22:18.361 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:18.360+0000 7f26cd0ea640 1 -- --> v2:172.21.15.143:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f26c815e1a0 con 0x7f26c806cd50 2023-11-26T07:22:18.362 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:18.360+0000 7f26c759e640 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7f26c806cd50 0x7f26c80b2180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:22:18.362 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:18.360+0000 7f26c659c640 1 --1- >> v1:172.21.15.143:6789/0 conn(0x7f26c8070d10 0x7f26c815d900 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.143:6789/0 says I am v1:172.21.15.143:59154/0 (socket says 172.21.15.143:59154) 2023-11-26T07:22:18.362 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:18.360+0000 7f26c659c640 1 -- 172.21.15.143:0/3646361096 learned_addr learned my addr 172.21.15.143:0/3646361096 (peer_addr_for_me v1:172.21.15.143:0/0) 2023-11-26T07:22:18.362 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:18.360+0000 7f26c759e640 1 -- 172.21.15.143:0/3646361096 >> v1:172.21.15.188:6789/0 conn(0x7f26c80cf280 legacy=0x7f26c80cf660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:22:18.362 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:18.360+0000 7f26c759e640 1 -- 172.21.15.143:0/3646361096 >> v1:172.21.15.143:6789/0 conn(0x7f26c8070d10 legacy=0x7f26c815d900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:22:18.363 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:18.360+0000 7f26c5d9b640 1 -- 172.21.15.143:0/3646361096 <== mon.0 v1:172.21.15.143:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4203416493 0 0) 0x7f26bc001120 con 0x7f26c8070d10 2023-11-26T07:22:18.363 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:18.360+0000 7f26c759e640 1 -- 172.21.15.143:0/3646361096 --> v2:172.21.15.143:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f26c815e480 con 0x7f26c806cd50 2023-11-26T07:22:18.365 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:18.364+0000 7f26c759e640 1 --2- 172.21.15.143:0/3646361096 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f26c806cd50 0x7f26c80b2180 secure :-1 s=READY pgs=860 cs=0 l=1 rev1=1 crypto rx=0x7f26c0009d90 tx=0x7f26c001f130 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f2af16813f249101 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:22:18.365 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:18.364+0000 7f26c5d9b640 1 -- 172.21.15.143:0/3646361096 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f26c00af600 con 0x7f26c806cd50 2023-11-26T07:22:18.365 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:18.364+0000 7f26c5d9b640 1 -- 172.21.15.143:0/3646361096 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f26c00ab050 con 0x7f26c806cd50 2023-11-26T07:22:18.365 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:18.364+0000 7f26c5d9b640 1 -- 172.21.15.143:0/3646361096 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f26c00af980 con 0x7f26c806cd50 2023-11-26T07:22:18.366 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:18.364+0000 7f26cd0ea640 1 -- 172.21.15.143:0/3646361096 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f26c806cd50 msgr2=0x7f26c80b2180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:22:18.366 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:18.364+0000 7f26cd0ea640 1 --2- 172.21.15.143:0/3646361096 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f26c806cd50 0x7f26c80b2180 secure :-1 s=READY pgs=860 cs=0 l=1 rev1=1 crypto rx=0x7f26c0009d90 tx=0x7f26c001f130 comp rx=0 tx=0).stop 2023-11-26T07:22:18.366 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:18.364+0000 7f26cd0ea640 1 -- 172.21.15.143:0/3646361096 shutdown_connections 2023-11-26T07:22:18.366 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:18.364+0000 7f26cd0ea640 1 --2- 172.21.15.143:0/3646361096 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f26c806cd50 0x7f26c80b2180 unknown :-1 s=CLOSED pgs=860 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:22:18.366 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:18.364+0000 7f26cd0ea640 1 -- 172.21.15.143:0/3646361096 >> 172.21.15.143:0/3646361096 conn(0x7f26c806a890 msgr2=0x7f26c806aca0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:22:18.366 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:18.364+0000 7f26cd0ea640 1 -- 172.21.15.143:0/3646361096 shutdown_connections 2023-11-26T07:22:18.366 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:18.364+0000 7f26cd0ea640 1 -- 172.21.15.143:0/3646361096 wait complete. 2023-11-26T07:22:18.367 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:18.364+0000 7f26cd0ea640 1 Processor -- start 2023-11-26T07:22:18.367 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:18.364+0000 7f26cd0ea640 1 -- start start 2023-11-26T07:22:18.369 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:18.368+0000 7f26cd0ea640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f26c806cd50 0x7f26c8146480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:22:18.369 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:18.368+0000 7f26cd0ea640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f26c8070d10 0x7f26c8146bd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:22:18.369 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:18.368+0000 7f26cd0ea640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f26c80cf280 0x7f26c80b0480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:22:18.369 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:18.368+0000 7f26cd0ea640 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f26c8147620 con 0x7f26c8070d10 2023-11-26T07:22:18.370 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:18.368+0000 7f26cd0ea640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f26c8147900 con 0x7f26c806cd50 2023-11-26T07:22:18.370 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:18.368+0000 7f26cd0ea640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f26c8147be0 con 0x7f26c80cf280 2023-11-26T07:22:18.370 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:18.368+0000 7f26c659c640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f26c8070d10 0x7f26c8146bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:22:18.370 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:18.368+0000 7f26c659c640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f26c8070d10 0x7f26c8146bd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:34250/0 (socket says 172.21.15.143:34250) 2023-11-26T07:22:18.370 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:18.368+0000 7f26c659c640 1 -- 172.21.15.143:0/1011155442 learned_addr learned my addr 172.21.15.143:0/1011155442 (peer_addr_for_me v2:172.21.15.143:0/0) 2023-11-26T07:22:18.370 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:18.368+0000 7f26c6d9d640 1 --2- 172.21.15.143:0/1011155442 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f26c806cd50 0x7f26c8146480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:22:18.370 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:18.368+0000 7f26c759e640 1 --2- 172.21.15.143:0/1011155442 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f26c80cf280 0x7f26c80b0480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:22:18.370 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:18.368+0000 7f26c659c640 1 -- 172.21.15.143:0/1011155442 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f26c80cf280 msgr2=0x7f26c80b0480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:22:18.370 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:18.368+0000 7f26c659c640 1 --2- 172.21.15.143:0/1011155442 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f26c80cf280 0x7f26c80b0480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:22:18.370 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:18.368+0000 7f26c659c640 1 -- 172.21.15.143:0/1011155442 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f26c806cd50 msgr2=0x7f26c8146480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:22:18.370 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:18.368+0000 7f26c659c640 1 --2- 172.21.15.143:0/1011155442 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f26c806cd50 0x7f26c8146480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:22:18.370 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:18.368+0000 7f26c659c640 1 -- 172.21.15.143:0/1011155442 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f26c0009a70 con 0x7f26c8070d10 2023-11-26T07:22:18.371 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:18.368+0000 7f26c6d9d640 1 --2- 172.21.15.143:0/1011155442 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f26c806cd50 0x7f26c8146480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-26T07:22:18.371 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:18.368+0000 7f26c659c640 1 --2- 172.21.15.143:0/1011155442 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f26c8070d10 0x7f26c8146bd0 secure :-1 s=READY pgs=861 cs=0 l=1 rev1=1 crypto rx=0x7f26bc00c780 tx=0x7f26bc00cc40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:22:18.371 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:18.368+0000 7f26af7fe640 1 -- 172.21.15.143:0/1011155442 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f26bc0072d0 con 0x7f26c8070d10 2023-11-26T07:22:18.372 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:18.368+0000 7f26af7fe640 1 -- 172.21.15.143:0/1011155442 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f26bc007430 con 0x7f26c8070d10 2023-11-26T07:22:18.372 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:18.368+0000 7f26cd0ea640 1 -- 172.21.15.143:0/1011155442 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f26c8147ec0 con 0x7f26c8070d10 2023-11-26T07:22:18.372 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:18.368+0000 7f26af7fe640 1 -- 172.21.15.143:0/1011155442 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f26bc001d20 con 0x7f26c8070d10 2023-11-26T07:22:18.372 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:18.368+0000 7f26cd0ea640 1 -- 172.21.15.143:0/1011155442 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f26c8159e80 con 0x7f26c8070d10 2023-11-26T07:22:18.372 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:18.368+0000 7f26cd0ea640 1 -- 172.21.15.143:0/1011155442 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f26c8067680 con 0x7f26c8070d10 2023-11-26T07:22:18.372 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:18.372+0000 7f26af7fe640 1 -- 172.21.15.143:0/1011155442 <== mon.0 v2:172.21.15.143:3300/0 4 ==== mgrmap(e 10) v1 ==== 86056+0+0 (secure 0 0 0) 0x7f26bc0047f0 con 0x7f26c8070d10 2023-11-26T07:22:18.373 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:18.372+0000 7f26af7fe640 1 --2- 172.21.15.143:0/1011155442 >> [v2:172.21.15.143:6832/2150557544,v1:172.21.15.143:6833/2150557544] conn(0x7f26b40666b0 0x7f26b4068b70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:22:18.373 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:18.372+0000 7f26af7fe640 1 -- 172.21.15.143:0/1011155442 <== mon.0 v2:172.21.15.143:3300/0 5 ==== osd_map(59..59 src has 1..59) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f26bc083240 con 0x7f26c8070d10 2023-11-26T07:22:18.373 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:18.372+0000 7f26c6d9d640 1 --2- 172.21.15.143:0/1011155442 >> [v2:172.21.15.143:6832/2150557544,v1:172.21.15.143:6833/2150557544] conn(0x7f26b40666b0 0x7f26b4068b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:22:18.373 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:18.372+0000 7f26c6d9d640 1 --2- 172.21.15.143:0/1011155442 >> [v2:172.21.15.143:6832/2150557544,v1:172.21.15.143:6833/2150557544] conn(0x7f26b40666b0 0x7f26b4068b70 secure :-1 s=READY pgs=404 cs=0 l=1 rev1=1 crypto rx=0x7f26b8000f70 tx=0x7f26b8009040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:22:18.376 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:18.376+0000 7f26af7fe640 1 -- 172.21.15.143:0/1011155442 <== mon.0 v2:172.21.15.143:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172649 (secure 0 0 0) 0x7f26bc054000 con 0x7f26c8070d10 2023-11-26T07:22:18.528 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:18.528+0000 7f26cd0ea640 1 -- 172.21.15.143:0/1011155442 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f26c815bab0 con 0x7f26c8070d10 2023-11-26T07:22:18.528 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:18.528+0000 7f26af7fe640 1 -- 172.21.15.143:0/1011155442 <== mon.0 v2:172.21.15.143:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v59) v1 ==== 100+0+11 (secure 0 0 0) 0x7f26c815bab0 con 0x7f26c8070d10 2023-11-26T07:22:18.528 INFO:teuthology.orchestra.run.smithi143.stdout:pg_num: 64 2023-11-26T07:22:18.530 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:18.528+0000 7f26cd0ea640 1 -- 172.21.15.143:0/1011155442 >> [v2:172.21.15.143:6832/2150557544,v1:172.21.15.143:6833/2150557544] conn(0x7f26b40666b0 msgr2=0x7f26b4068b70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:22:18.530 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:18.528+0000 7f26cd0ea640 1 --2- 172.21.15.143:0/1011155442 >> [v2:172.21.15.143:6832/2150557544,v1:172.21.15.143:6833/2150557544] conn(0x7f26b40666b0 0x7f26b4068b70 secure :-1 s=READY pgs=404 cs=0 l=1 rev1=1 crypto rx=0x7f26b8000f70 tx=0x7f26b8009040 comp rx=0 tx=0).stop 2023-11-26T07:22:18.531 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:18.528+0000 7f26cd0ea640 1 -- 172.21.15.143:0/1011155442 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f26c8070d10 msgr2=0x7f26c8146bd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:22:18.531 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:18.528+0000 7f26cd0ea640 1 --2- 172.21.15.143:0/1011155442 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f26c8070d10 0x7f26c8146bd0 secure :-1 s=READY pgs=861 cs=0 l=1 rev1=1 crypto rx=0x7f26bc00c780 tx=0x7f26bc00cc40 comp rx=0 tx=0).stop 2023-11-26T07:22:18.531 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:18.528+0000 7f26cd0ea640 1 -- 172.21.15.143:0/1011155442 shutdown_connections 2023-11-26T07:22:18.531 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:18.528+0000 7f26cd0ea640 1 --2- 172.21.15.143:0/1011155442 >> [v2:172.21.15.143:6832/2150557544,v1:172.21.15.143:6833/2150557544] conn(0x7f26b40666b0 0x7f26b4068b70 unknown :-1 s=CLOSED pgs=404 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:22:18.531 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:18.528+0000 7f26cd0ea640 1 --2- 172.21.15.143:0/1011155442 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f26c80cf280 0x7f26c80b0480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:22:18.531 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:18.528+0000 7f26cd0ea640 1 --2- 172.21.15.143:0/1011155442 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f26c8070d10 0x7f26c8146bd0 unknown :-1 s=CLOSED pgs=861 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:22:18.531 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:18.528+0000 7f26cd0ea640 1 --2- 172.21.15.143:0/1011155442 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f26c806cd50 0x7f26c8146480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:22:18.531 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:18.528+0000 7f26cd0ea640 1 -- 172.21.15.143:0/1011155442 >> 172.21.15.143:0/1011155442 conn(0x7f26c806a890 msgr2=0x7f26c806ac70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:22:18.531 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:18.528+0000 7f26cd0ea640 1 -- 172.21.15.143:0/1011155442 shutdown_connections 2023-11-26T07:22:18.531 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:18.528+0000 7f26cd0ea640 1 -- 172.21.15.143:0/1011155442 wait complete. 2023-11-26T07:22:18.549 DEBUG:teuthology.orchestra.run.smithi143:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2023-11-26T07:22:18.691 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:18.688+0000 7f20b1962640 1 Processor -- start 2023-11-26T07:22:18.692 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:18.688+0000 7f20b1962640 1 -- start start 2023-11-26T07:22:18.693 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:18.692+0000 7f20b1962640 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7f20ac160850 0x7f20ac0d3d30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:22:18.693 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:18.692+0000 7f20b1962640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f20ac0cf330 0x7f20ac0cda90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:22:18.693 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:18.692+0000 7f20b1962640 1 -- --> v1:172.21.15.188:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f20ac0cdfd0 con 0x7f20ac160f60 2023-11-26T07:22:18.693 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:18.692+0000 7f20b1962640 1 -- --> v2:172.21.15.188:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f20ac0ce290 con 0x7f20ac160850 2023-11-26T07:22:18.693 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:18.692+0000 7f20b1962640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f20ac0ce570 con 0x7f20ac0cf330 2023-11-26T07:22:18.693 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:18.692+0000 7f20b0960640 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7f20ac160850 0x7f20ac0d3d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:22:18.694 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:18.692+0000 7f20abfff640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f20ac0cf330 0x7f20ac0cda90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:22:18.694 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:18.692+0000 7f20b0960640 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7f20ac160850 0x7f20ac0d3d30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.188:3300/0 says I am v2:172.21.15.143:37254/0 (socket says 172.21.15.143:37254) 2023-11-26T07:22:18.694 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:18.692+0000 7f20b0960640 1 -- 172.21.15.143:0/3062485962 learned_addr learned my addr 172.21.15.143:0/3062485962 (peer_addr_for_me v2:172.21.15.143:0/0) 2023-11-26T07:22:18.694 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:18.692+0000 7f20b0960640 1 -- 172.21.15.143:0/3062485962 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f20ac0cf330 msgr2=0x7f20ac0cda90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-26T07:22:18.694 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:18.692+0000 7f20b0960640 1 --2- 172.21.15.143:0/3062485962 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f20ac0cf330 0x7f20ac0cda90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:22:18.694 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:18.692+0000 7f20b0960640 1 -- 172.21.15.143:0/3062485962 >> v1:172.21.15.188:6789/0 conn(0x7f20ac160f60 legacy=0x7f20ac0b27e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:22:18.695 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:18.692+0000 7f20b0960640 1 -- 172.21.15.143:0/3062485962 --> v2:172.21.15.188:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f20ac0ce850 con 0x7f20ac160850 2023-11-26T07:22:18.696 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:18.696+0000 7f20b0960640 1 --2- 172.21.15.143:0/3062485962 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f20ac160850 0x7f20ac0d3d30 secure :-1 s=READY pgs=338 cs=0 l=1 rev1=1 crypto rx=0x7f20a0009870 tx=0x7f20a001c540 comp rx=0 tx=0).ready entity=mon.1 client_cookie=d46d5e345ed58310 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:22:18.697 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:18.696+0000 7f20ab7fe640 1 -- 172.21.15.143:0/3062485962 <== mon.1 v2:172.21.15.188:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f20a00a9620 con 0x7f20ac160850 2023-11-26T07:22:18.698 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:18.696+0000 7f20ab7fe640 1 -- 172.21.15.143:0/3062485962 <== mon.1 v2:172.21.15.188:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f20a00a9780 con 0x7f20ac160850 2023-11-26T07:22:18.698 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:18.696+0000 7f20ab7fe640 1 -- 172.21.15.143:0/3062485962 <== mon.1 v2:172.21.15.188:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f20a00a9b10 con 0x7f20ac160850 2023-11-26T07:22:18.698 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:18.696+0000 7f20b1962640 1 -- 172.21.15.143:0/3062485962 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f20ac160850 msgr2=0x7f20ac0d3d30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:22:18.698 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:18.696+0000 7f20b1962640 1 --2- 172.21.15.143:0/3062485962 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f20ac160850 0x7f20ac0d3d30 secure :-1 s=READY pgs=338 cs=0 l=1 rev1=1 crypto rx=0x7f20a0009870 tx=0x7f20a001c540 comp rx=0 tx=0).stop 2023-11-26T07:22:18.698 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:18.696+0000 7f20b1962640 1 -- 172.21.15.143:0/3062485962 shutdown_connections 2023-11-26T07:22:18.698 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:18.696+0000 7f20b1962640 1 --2- 172.21.15.143:0/3062485962 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f20ac0cf330 0x7f20ac0cda90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:22:18.698 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:18.696+0000 7f20b1962640 1 --2- 172.21.15.143:0/3062485962 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f20ac160850 0x7f20ac0d3d30 unknown :-1 s=CLOSED pgs=338 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:22:18.698 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:18.696+0000 7f20b1962640 1 -- 172.21.15.143:0/3062485962 >> 172.21.15.143:0/3062485962 conn(0x7f20ac09c5a0 msgr2=0x7f20ac0d3260 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:22:18.698 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:18.696+0000 7f20b1962640 1 -- 172.21.15.143:0/3062485962 shutdown_connections 2023-11-26T07:22:18.698 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:18.696+0000 7f20b1962640 1 -- 172.21.15.143:0/3062485962 wait complete. 2023-11-26T07:22:18.699 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:18.696+0000 7f20b1962640 1 Processor -- start 2023-11-26T07:22:18.699 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:18.696+0000 7f20b1962640 1 -- start start 2023-11-26T07:22:18.701 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:18.700+0000 7f20b1962640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f20ac0cf330 0x7f20ac15f3f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:22:18.701 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:18.700+0000 7f20b1962640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f20ac160850 0x7f20ac15b660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:22:18.701 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:18.700+0000 7f20b1962640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f20ac160f60 0x7f20ac14f290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:22:18.701 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:18.700+0000 7f20b1962640 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f20ac15d670 con 0x7f20ac160850 2023-11-26T07:22:18.701 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:18.700+0000 7f20b1962640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f20ac15ed50 con 0x7f20ac160f60 2023-11-26T07:22:18.701 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:18.700+0000 7f20b1962640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f20ac15d8c0 con 0x7f20ac0cf330 2023-11-26T07:22:18.702 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:18.700+0000 7f20abfff640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f20ac160850 0x7f20ac15b660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:22:18.702 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:18.700+0000 7f20abfff640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f20ac160850 0x7f20ac15b660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:34256/0 (socket says 172.21.15.143:34256) 2023-11-26T07:22:18.702 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:18.700+0000 7f20abfff640 1 -- 172.21.15.143:0/1552883532 learned_addr learned my addr 172.21.15.143:0/1552883532 (peer_addr_for_me v2:172.21.15.143:0/0) 2023-11-26T07:22:18.702 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:18.700+0000 7f20b0960640 1 --2- 172.21.15.143:0/1552883532 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f20ac0cf330 0x7f20ac15f3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:22:18.702 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:18.700+0000 7f20b1161640 1 --2- 172.21.15.143:0/1552883532 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f20ac160f60 0x7f20ac14f290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:22:18.702 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:18.700+0000 7f20abfff640 1 -- 172.21.15.143:0/1552883532 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f20ac0cf330 msgr2=0x7f20ac15f3f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:22:18.702 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:18.700+0000 7f20abfff640 1 --2- 172.21.15.143:0/1552883532 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f20ac0cf330 0x7f20ac15f3f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:22:18.702 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:18.700+0000 7f20abfff640 1 -- 172.21.15.143:0/1552883532 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f20ac160f60 msgr2=0x7f20ac14f290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:22:18.702 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:18.700+0000 7f20abfff640 1 --2- 172.21.15.143:0/1552883532 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f20ac160f60 0x7f20ac14f290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:22:18.702 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:18.700+0000 7f20abfff640 1 -- 172.21.15.143:0/1552883532 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f20a00094d0 con 0x7f20ac160850 2023-11-26T07:22:18.702 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:18.700+0000 7f20abfff640 1 --2- 172.21.15.143:0/1552883532 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f20ac160850 0x7f20ac15b660 secure :-1 s=READY pgs=862 cs=0 l=1 rev1=1 crypto rx=0x7f209800e8a0 tx=0x7f209800eca0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:22:18.703 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:18.700+0000 7f20a97fa640 1 -- 172.21.15.143:0/1552883532 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2098009510 con 0x7f20ac160850 2023-11-26T07:22:18.703 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:18.700+0000 7f20a97fa640 1 -- 172.21.15.143:0/1552883532 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f209800c940 con 0x7f20ac160850 2023-11-26T07:22:18.703 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:18.700+0000 7f20a97fa640 1 -- 172.21.15.143:0/1552883532 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f209800ccb0 con 0x7f20ac160850 2023-11-26T07:22:18.703 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:18.700+0000 7f20b1962640 1 -- 172.21.15.143:0/1552883532 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f20ac161900 con 0x7f20ac160850 2023-11-26T07:22:18.703 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:18.700+0000 7f20b1962640 1 -- 172.21.15.143:0/1552883532 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f20ac15de80 con 0x7f20ac160850 2023-11-26T07:22:18.703 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:18.700+0000 7f20b1962640 1 -- 172.21.15.143:0/1552883532 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f20ac067500 con 0x7f20ac160850 2023-11-26T07:22:18.704 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:18.700+0000 7f20a97fa640 1 -- 172.21.15.143:0/1552883532 <== mon.0 v2:172.21.15.143:3300/0 4 ==== mgrmap(e 11) v1 ==== 86158+0+0 (secure 0 0 0) 0x7f2098013470 con 0x7f20ac160850 2023-11-26T07:22:18.704 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:18.704+0000 7f20a97fa640 1 --2- 172.21.15.143:0/1552883532 >> [v2:172.21.15.143:6832/2150557544,v1:172.21.15.143:6833/2150557544] conn(0x7f20900667d0 0x7f2090068c90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:22:18.704 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:18.704+0000 7f20a97fa640 1 -- 172.21.15.143:0/1552883532 <== mon.0 v2:172.21.15.143:3300/0 5 ==== osd_map(59..59 src has 1..59) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f2098084410 con 0x7f20ac160850 2023-11-26T07:22:18.705 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:18.704+0000 7f20b0960640 1 --2- 172.21.15.143:0/1552883532 >> [v2:172.21.15.143:6832/2150557544,v1:172.21.15.143:6833/2150557544] conn(0x7f20900667d0 0x7f2090068c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:22:18.705 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:18.704+0000 7f20b0960640 1 --2- 172.21.15.143:0/1552883532 >> [v2:172.21.15.143:6832/2150557544,v1:172.21.15.143:6833/2150557544] conn(0x7f20900667d0 0x7f2090068c90 secure :-1 s=READY pgs=405 cs=0 l=1 rev1=1 crypto rx=0x7f20a00ae000 tx=0x7f20a00b3040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:22:18.709 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:18.708+0000 7f20a97fa640 1 -- 172.21.15.143:0/1552883532 <== mon.0 v2:172.21.15.143:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172649 (secure 0 0 0) 0x7f2098051500 con 0x7f20ac160850 2023-11-26T07:22:18.870 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:18.868+0000 7f20b1962640 1 -- 172.21.15.143:0/1552883532 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) v1 -- 0x7f20ac09ca80 con 0x7f20ac160850 2023-11-26T07:22:18.871 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:18.868+0000 7f20a97fa640 1 -- 172.21.15.143:0/1552883532 <== mon.0 v2:172.21.15.143:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v59) v1 ==== 80+0+0 (secure 0 0 0) 0x7f2098054b20 con 0x7f20ac160850 2023-11-26T07:22:18.871 INFO:teuthology.orchestra.run.smithi143.stderr:listed 0 entries 2023-11-26T07:22:18.873 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:18.872+0000 7f20b1962640 1 -- 172.21.15.143:0/1552883532 >> [v2:172.21.15.143:6832/2150557544,v1:172.21.15.143:6833/2150557544] conn(0x7f20900667d0 msgr2=0x7f2090068c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:22:18.874 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:18.872+0000 7f20b1962640 1 --2- 172.21.15.143:0/1552883532 >> [v2:172.21.15.143:6832/2150557544,v1:172.21.15.143:6833/2150557544] conn(0x7f20900667d0 0x7f2090068c90 secure :-1 s=READY pgs=405 cs=0 l=1 rev1=1 crypto rx=0x7f20a00ae000 tx=0x7f20a00b3040 comp rx=0 tx=0).stop 2023-11-26T07:22:18.874 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:18.872+0000 7f20b1962640 1 -- 172.21.15.143:0/1552883532 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f20ac160850 msgr2=0x7f20ac15b660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:22:18.874 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:18.872+0000 7f20b1962640 1 --2- 172.21.15.143:0/1552883532 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f20ac160850 0x7f20ac15b660 secure :-1 s=READY pgs=862 cs=0 l=1 rev1=1 crypto rx=0x7f209800e8a0 tx=0x7f209800eca0 comp rx=0 tx=0).stop 2023-11-26T07:22:18.874 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:18.872+0000 7f20b1962640 1 -- 172.21.15.143:0/1552883532 shutdown_connections 2023-11-26T07:22:18.874 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:18.872+0000 7f20b1962640 1 --2- 172.21.15.143:0/1552883532 >> [v2:172.21.15.143:6832/2150557544,v1:172.21.15.143:6833/2150557544] conn(0x7f20900667d0 0x7f2090068c90 unknown :-1 s=CLOSED pgs=405 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:22:18.874 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:18.872+0000 7f20b1962640 1 --2- 172.21.15.143:0/1552883532 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f20ac160f60 0x7f20ac14f290 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:22:18.874 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:18.872+0000 7f20b1962640 1 --2- 172.21.15.143:0/1552883532 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f20ac160850 0x7f20ac15b660 unknown :-1 s=CLOSED pgs=862 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:22:18.874 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:18.872+0000 7f20b1962640 1 --2- 172.21.15.143:0/1552883532 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f20ac0cf330 0x7f20ac15f3f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:22:18.874 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:18.872+0000 7f20b1962640 1 -- 172.21.15.143:0/1552883532 >> 172.21.15.143:0/1552883532 conn(0x7f20ac09c5a0 msgr2=0x7f20ac15cde0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:22:18.874 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:18.872+0000 7f20b1962640 1 -- 172.21.15.143:0/1552883532 shutdown_connections 2023-11-26T07:22:18.874 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:18.872+0000 7f20b1962640 1 -- 172.21.15.143:0/1552883532 wait complete. 2023-11-26T07:22:18.892 INFO:tasks.cephfs.fuse_mount:Running fusermount -u on ubuntu@smithi143.front.sepia.ceph.com... 2023-11-26T07:22:18.892 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-26T07:22:18.892 DEBUG:teuthology.orchestra.run.smithi143:> sudo fusermount -u /home/ubuntu/cephtest/mnt.0 2023-11-26T07:22:18.911 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi143.stderr:ceph-fuse[53383]: fuse finished with error 0 and tester_r 0 2023-11-26T07:22:18.914 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi143.front.sepia.ceph.com 2023-11-26T07:22:18.914 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-26T07:22:18.914 DEBUG:teuthology.orchestra.run.smithi143:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2023-11-26T07:22:18.985 DEBUG:teuthology.orchestra.run.smithi188:> set -ex 2023-11-26T07:22:18.985 DEBUG:teuthology.orchestra.run.smithi188:> dd if=/proc/self/mounts of=/dev/stdout 2023-11-26T07:22:18.995 DEBUG:teuthology.orchestra.run.smithi143:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-11-26T07:22:19.168 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:19.164+0000 7fd3d0878640 1 Processor -- start 2023-11-26T07:22:19.168 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:19.168+0000 7fd3d0878640 1 -- start start 2023-11-26T07:22:19.169 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:19.168+0000 7fd3d0878640 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7fd3cc0b1e10 0x7fd3cc15d9e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:22:19.169 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:19.168+0000 7fd3d0878640 1 -- --> v1:172.21.15.143:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd3cc15e860 con 0x7fd3cc143810 2023-11-26T07:22:19.169 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:19.168+0000 7fd3d0878640 1 -- --> v1:172.21.15.188:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd3cc0cdba0 con 0x7fd3cc15df20 2023-11-26T07:22:19.169 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:19.168+0000 7fd3d0878640 1 -- --> v2:172.21.15.143:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd3cc15e3b0 con 0x7fd3cc0b1e10 2023-11-26T07:22:19.170 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:19.168+0000 7fd3ca59c640 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7fd3cc0b1e10 0x7fd3cc15d9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:22:19.170 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:19.168+0000 7fd3cad9d640 1 --1- >> v1:172.21.15.143:6789/0 conn(0x7fd3cc143810 0x7fd3cc072140 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.143:6789/0 says I am v1:172.21.15.143:59166/0 (socket says 172.21.15.143:59166) 2023-11-26T07:22:19.170 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:19.168+0000 7fd3cad9d640 1 -- 172.21.15.143:0/1468971513 learned_addr learned my addr 172.21.15.143:0/1468971513 (peer_addr_for_me v1:172.21.15.143:0/0) 2023-11-26T07:22:19.170 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:19.168+0000 7fd3ca59c640 1 -- 172.21.15.143:0/1468971513 >> v1:172.21.15.188:6789/0 conn(0x7fd3cc15df20 legacy=0x7fd3cc0ce030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-26T07:22:19.170 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:19.168+0000 7fd3c9d9b640 1 -- 172.21.15.143:0/1468971513 <== mon.0 v1:172.21.15.143:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1015009641 0 0) 0x7fd3cc15e860 con 0x7fd3cc143810 2023-11-26T07:22:19.171 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:19.168+0000 7fd3ca59c640 1 -- 172.21.15.143:0/1468971513 >> v1:172.21.15.143:6789/0 conn(0x7fd3cc143810 legacy=0x7fd3cc072140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:22:19.171 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:19.168+0000 7fd3ca59c640 1 -- 172.21.15.143:0/1468971513 --> v2:172.21.15.143:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd3cc143bf0 con 0x7fd3cc0b1e10 2023-11-26T07:22:19.173 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:19.172+0000 7fd3ca59c640 1 --2- 172.21.15.143:0/1468971513 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fd3cc0b1e10 0x7fd3cc15d9e0 secure :-1 s=READY pgs=864 cs=0 l=1 rev1=1 crypto rx=0x7fd3c00097c0 tx=0x7fd3c001c410 comp rx=0 tx=0).ready entity=mon.0 client_cookie=75257ee60cc820e4 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:22:19.173 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:19.172+0000 7fd3c9d9b640 1 -- 172.21.15.143:0/1468971513 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd3c00a7070 con 0x7fd3cc0b1e10 2023-11-26T07:22:19.173 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:19.172+0000 7fd3c9d9b640 1 -- 172.21.15.143:0/1468971513 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd3c00ab430 con 0x7fd3cc0b1e10 2023-11-26T07:22:19.173 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:19.172+0000 7fd3c9d9b640 1 -- 172.21.15.143:0/1468971513 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd3c00ab7c0 con 0x7fd3cc0b1e10 2023-11-26T07:22:19.174 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:19.172+0000 7fd3d0878640 1 -- 172.21.15.143:0/1468971513 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fd3cc0b1e10 msgr2=0x7fd3cc15d9e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:22:19.174 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:19.172+0000 7fd3d0878640 1 --2- 172.21.15.143:0/1468971513 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fd3cc0b1e10 0x7fd3cc15d9e0 secure :-1 s=READY pgs=864 cs=0 l=1 rev1=1 crypto rx=0x7fd3c00097c0 tx=0x7fd3c001c410 comp rx=0 tx=0).stop 2023-11-26T07:22:19.174 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:19.172+0000 7fd3d0878640 1 -- 172.21.15.143:0/1468971513 shutdown_connections 2023-11-26T07:22:19.174 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:19.172+0000 7fd3d0878640 1 --2- 172.21.15.143:0/1468971513 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fd3cc0b1e10 0x7fd3cc15d9e0 unknown :-1 s=CLOSED pgs=864 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:22:19.174 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:19.172+0000 7fd3d0878640 1 -- 172.21.15.143:0/1468971513 >> 172.21.15.143:0/1468971513 conn(0x7fd3cc06a890 msgr2=0x7fd3cc06aca0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:22:19.174 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:19.172+0000 7fd3d0878640 1 -- 172.21.15.143:0/1468971513 shutdown_connections 2023-11-26T07:22:19.174 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:19.172+0000 7fd3d0878640 1 -- 172.21.15.143:0/1468971513 wait complete. 2023-11-26T07:22:19.175 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:19.172+0000 7fd3d0878640 1 Processor -- start 2023-11-26T07:22:19.175 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:19.172+0000 7fd3d0878640 1 -- start start 2023-11-26T07:22:19.177 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:19.176+0000 7fd3d0878640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fd3cc0b1e10 0x7fd3cc1515f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:22:19.177 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:19.176+0000 7fd3d0878640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fd3cc143810 0x7fd3cc14fd20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:22:19.177 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:19.176+0000 7fd3d0878640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fd3cc15df20 0x7fd3cc13cfb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:22:19.177 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:19.176+0000 7fd3d0878640 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd3cc150260 con 0x7fd3cc143810 2023-11-26T07:22:19.178 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:19.176+0000 7fd3d0878640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd3cc13d4f0 con 0x7fd3cc15df20 2023-11-26T07:22:19.178 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:19.176+0000 7fd3d0878640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd3cc13b250 con 0x7fd3cc0b1e10 2023-11-26T07:22:19.178 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:19.176+0000 7fd3ca59c640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fd3cc143810 0x7fd3cc14fd20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:22:19.178 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:19.176+0000 7fd3ca59c640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fd3cc143810 0x7fd3cc14fd20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:34280/0 (socket says 172.21.15.143:34280) 2023-11-26T07:22:19.178 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:19.176+0000 7fd3ca59c640 1 -- 172.21.15.143:0/3200850260 learned_addr learned my addr 172.21.15.143:0/3200850260 (peer_addr_for_me v2:172.21.15.143:0/0) 2023-11-26T07:22:19.178 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:19.176+0000 7fd3cad9d640 1 --2- 172.21.15.143:0/3200850260 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fd3cc0b1e10 0x7fd3cc1515f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:22:19.178 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:19.176+0000 7fd3cb59e640 1 --2- 172.21.15.143:0/3200850260 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fd3cc15df20 0x7fd3cc13cfb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:22:19.178 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:19.176+0000 7fd3ca59c640 1 -- 172.21.15.143:0/3200850260 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fd3cc0b1e10 msgr2=0x7fd3cc1515f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:22:19.178 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:19.176+0000 7fd3ca59c640 1 --2- 172.21.15.143:0/3200850260 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fd3cc0b1e10 0x7fd3cc1515f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:22:19.178 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:19.176+0000 7fd3ca59c640 1 -- 172.21.15.143:0/3200850260 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fd3cc15df20 msgr2=0x7fd3cc13cfb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:22:19.178 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:19.176+0000 7fd3ca59c640 1 --2- 172.21.15.143:0/3200850260 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fd3cc15df20 0x7fd3cc13cfb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:22:19.179 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:19.176+0000 7fd3ca59c640 1 -- 172.21.15.143:0/3200850260 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd3c0009420 con 0x7fd3cc143810 2023-11-26T07:22:19.179 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:19.176+0000 7fd3cad9d640 1 --2- 172.21.15.143:0/3200850260 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fd3cc0b1e10 0x7fd3cc1515f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-26T07:22:19.179 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:19.176+0000 7fd3ca59c640 1 --2- 172.21.15.143:0/3200850260 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fd3cc143810 0x7fd3cc14fd20 secure :-1 s=READY pgs=865 cs=0 l=1 rev1=1 crypto rx=0x7fd3c00ad040 tx=0x7fd3c001ac50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:22:19.180 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:19.176+0000 7fd3b37fe640 1 -- 172.21.15.143:0/3200850260 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd3c00b5810 con 0x7fd3cc143810 2023-11-26T07:22:19.180 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:19.176+0000 7fd3b37fe640 1 -- 172.21.15.143:0/3200850260 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd3c00b59b0 con 0x7fd3cc143810 2023-11-26T07:22:19.180 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:19.176+0000 7fd3d0878640 1 -- 172.21.15.143:0/3200850260 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd3cc13b4a0 con 0x7fd3cc143810 2023-11-26T07:22:19.180 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:19.176+0000 7fd3b37fe640 1 -- 172.21.15.143:0/3200850260 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd3c00b5d60 con 0x7fd3cc143810 2023-11-26T07:22:19.180 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:19.176+0000 7fd3d0878640 1 -- 172.21.15.143:0/3200850260 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd3cc138a60 con 0x7fd3cc143810 2023-11-26T07:22:19.180 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:19.176+0000 7fd3d0878640 1 -- 172.21.15.143:0/3200850260 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd3cc067500 con 0x7fd3cc143810 2023-11-26T07:22:19.181 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:19.180+0000 7fd3b37fe640 1 -- 172.21.15.143:0/3200850260 <== mon.0 v2:172.21.15.143:3300/0 4 ==== mgrmap(e 11) v1 ==== 86158+0+0 (secure 0 0 0) 0x7fd3c00c7390 con 0x7fd3cc143810 2023-11-26T07:22:19.181 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:19.180+0000 7fd3b37fe640 1 --2- 172.21.15.143:0/3200850260 >> [v2:172.21.15.143:6832/2150557544,v1:172.21.15.143:6833/2150557544] conn(0x7fd3b80667d0 0x7fd3b8068c90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:22:19.181 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:19.180+0000 7fd3b37fe640 1 -- 172.21.15.143:0/3200850260 <== mon.0 v2:172.21.15.143:3300/0 5 ==== osd_map(59..59 src has 1..59) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fd3c011f050 con 0x7fd3cc143810 2023-11-26T07:22:19.181 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:19.180+0000 7fd3cad9d640 1 --2- 172.21.15.143:0/3200850260 >> [v2:172.21.15.143:6832/2150557544,v1:172.21.15.143:6833/2150557544] conn(0x7fd3b80667d0 0x7fd3b8068c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:22:19.181 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:19.180+0000 7fd3cad9d640 1 --2- 172.21.15.143:0/3200850260 >> [v2:172.21.15.143:6832/2150557544,v1:172.21.15.143:6833/2150557544] conn(0x7fd3b80667d0 0x7fd3b8068c90 secure :-1 s=READY pgs=406 cs=0 l=1 rev1=1 crypto rx=0x7fd3bc000f70 tx=0x7fd3bc007c40 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:22:19.185 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:19.184+0000 7fd3b37fe640 1 -- 172.21.15.143:0/3200850260 <== mon.0 v2:172.21.15.143:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172649 (secure 0 0 0) 0x7fd3c00ebf90 con 0x7fd3cc143810 2023-11-26T07:22:19.341 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:19.340+0000 7fd3d0878640 1 -- 172.21.15.143:0/3200850260 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fd3cc06ae10 con 0x7fd3cc143810 2023-11-26T07:22:19.342 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:19.340+0000 7fd3b37fe640 1 -- 172.21.15.143:0/3200850260 <== mon.0 v2:172.21.15.143:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v59) v1 ==== 74+0+16601 (secure 0 0 0) 0x7fd3c00ef5b0 con 0x7fd3cc143810 2023-11-26T07:22:19.342 INFO:teuthology.orchestra.run.smithi143.stdout: 2023-11-26T07:22:19.343 INFO:teuthology.orchestra.run.smithi143.stdout:{"epoch":59,"fsid":"f5033060-2469-4a07-95fe-2b76f5404bc1","created":"2023-11-26T07:17:28.613663+0000","modified":"2023-11-26T07:21:57.286555+0000","last_up_change":"2023-11-26T07:17:41.157813+0000","last_in_change":"2023-11-26T07:17:31.181658+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":9,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-11-26T07:17:41.758597+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":8,"pool_name":"cephfs_metadata","create_time":"2023-11-26T07:21:53.382407+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"58","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":9,"pool_name":"cephfs_data","create_time":"2023-11-26T07:21:54.656298+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"59","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.6299999952316284,"score_stable":1.6299999952316284,"optimal_score":1,"raw_score_acting":1.6299999952316284,"raw_score_stable":1.6299999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"ebbee2c6-023e-44b9-9f1d-6f7bfd309730","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":57,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6817","nonce":639807369},{"type":"v1","addr":"172.21.15.143:6819","nonce":639807369}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6821","nonce":639807369},{"type":"v1","addr":"172.21.15.143:6823","nonce":639807369}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6830","nonce":639807369},{"type":"v1","addr":"172.21.15.143:6831","nonce":639807369}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6825","nonce":639807369},{"type":"v1","addr":"172.21.15.143:6828","nonce":639807369}]},"public_addr":"172.21.15.143:6819/639807369","cluster_addr":"172.21.15.143:6823/639807369","heartbeat_back_addr":"172.21.15.143:6831/639807369","heartbeat_front_addr":"172.21.15.143:6828/639807369","state":["exists","up"]},{"osd":1,"uuid":"fec4b8be-7122-49e9-9a19-5850a0700001","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":57,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6816","nonce":1144942530},{"type":"v1","addr":"172.21.15.143:6818","nonce":1144942530}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6820","nonce":1144942530},{"type":"v1","addr":"172.21.15.143:6822","nonce":1144942530}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6827","nonce":1144942530},{"type":"v1","addr":"172.21.15.143:6829","nonce":1144942530}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6824","nonce":1144942530},{"type":"v1","addr":"172.21.15.143:6826","nonce":1144942530}]},"public_addr":"172.21.15.143:6818/1144942530","cluster_addr":"172.21.15.143:6822/1144942530","heartbeat_back_addr":"172.21.15.143:6829/1144942530","heartbeat_front_addr":"172.21.15.143:6826/1144942530","state":["exists","up"]},{"osd":2,"uuid":"2888af5b-f933-46ca-8f95-6dd494ff43f6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":57,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6808","nonce":3909095186},{"type":"v1","addr":"172.21.15.143:6809","nonce":3909095186}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6810","nonce":3909095186},{"type":"v1","addr":"172.21.15.143:6811","nonce":3909095186}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6814","nonce":3909095186},{"type":"v1","addr":"172.21.15.143:6815","nonce":3909095186}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6812","nonce":3909095186},{"type":"v1","addr":"172.21.15.143:6813","nonce":3909095186}]},"public_addr":"172.21.15.143:6809/3909095186","cluster_addr":"172.21.15.143:6811/3909095186","heartbeat_back_addr":"172.21.15.143:6815/3909095186","heartbeat_front_addr":"172.21.15.143:6813/3909095186","state":["exists","up"]},{"osd":3,"uuid":"d1f52435-7bc1-4081-9550-9cd1ba55f549","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":57,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6800","nonce":3854210991},{"type":"v1","addr":"172.21.15.143:6801","nonce":3854210991}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6802","nonce":3854210991},{"type":"v1","addr":"172.21.15.143:6803","nonce":3854210991}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6806","nonce":3854210991},{"type":"v1","addr":"172.21.15.143:6807","nonce":3854210991}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6804","nonce":3854210991},{"type":"v1","addr":"172.21.15.143:6805","nonce":3854210991}]},"public_addr":"172.21.15.143:6801/3854210991","cluster_addr":"172.21.15.143:6803/3854210991","heartbeat_back_addr":"172.21.15.143:6807/3854210991","heartbeat_front_addr":"172.21.15.143:6805/3854210991","state":["exists","up"]},{"osd":4,"uuid":"e2ae8b6e-6e25-49b2-bcbf-aacce68f06a3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":57,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6800","nonce":3924606902},{"type":"v1","addr":"172.21.15.188:6801","nonce":3924606902}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6802","nonce":3924606902},{"type":"v1","addr":"172.21.15.188:6803","nonce":3924606902}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6806","nonce":3924606902},{"type":"v1","addr":"172.21.15.188:6807","nonce":3924606902}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6804","nonce":3924606902},{"type":"v1","addr":"172.21.15.188:6805","nonce":3924606902}]},"public_addr":"172.21.15.188:6801/3924606902","cluster_addr":"172.21.15.188:6803/3924606902","heartbeat_back_addr":"172.21.15.188:6807/3924606902","heartbeat_front_addr":"172.21.15.188:6805/3924606902","state":["exists","up"]},{"osd":5,"uuid":"3cec8a32-7698-4c36-8e89-e4ba11934f33","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":57,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6813","nonce":186841544},{"type":"v1","addr":"172.21.15.188:6815","nonce":186841544}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6817","nonce":186841544},{"type":"v1","addr":"172.21.15.188:6818","nonce":186841544}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6822","nonce":186841544},{"type":"v1","addr":"172.21.15.188:6823","nonce":186841544}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6820","nonce":186841544},{"type":"v1","addr":"172.21.15.188:6821","nonce":186841544}]},"public_addr":"172.21.15.188:6815/186841544","cluster_addr":"172.21.15.188:6818/186841544","heartbeat_back_addr":"172.21.15.188:6823/186841544","heartbeat_front_addr":"172.21.15.188:6821/186841544","state":["exists","up"]},{"osd":6,"uuid":"58a65d5d-e090-473a-81b4-ad38e85029f9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":57,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6808","nonce":2153499405},{"type":"v1","addr":"172.21.15.188:6809","nonce":2153499405}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6810","nonce":2153499405},{"type":"v1","addr":"172.21.15.188:6811","nonce":2153499405}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6816","nonce":2153499405},{"type":"v1","addr":"172.21.15.188:6819","nonce":2153499405}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6812","nonce":2153499405},{"type":"v1","addr":"172.21.15.188:6814","nonce":2153499405}]},"public_addr":"172.21.15.188:6809/2153499405","cluster_addr":"172.21.15.188:6811/2153499405","heartbeat_back_addr":"172.21.15.188:6819/2153499405","heartbeat_front_addr":"172.21.15.188:6814/2153499405","state":["exists","up"]},{"osd":7,"uuid":"ea5b7f70-3554-4cf8-ac30-f7c59b80678d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":57,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6824","nonce":2722096729},{"type":"v1","addr":"172.21.15.188:6825","nonce":2722096729}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6826","nonce":2722096729},{"type":"v1","addr":"172.21.15.188:6827","nonce":2722096729}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6830","nonce":2722096729},{"type":"v1","addr":"172.21.15.188:6831","nonce":2722096729}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6828","nonce":2722096729},{"type":"v1","addr":"172.21.15.188:6829","nonce":2722096729}]},"public_addr":"172.21.15.188:6825/2722096729","cluster_addr":"172.21.15.188:6827/2722096729","heartbeat_back_addr":"172.21.15.188:6831/2722096729","heartbeat_front_addr":"172.21.15.188:6829/2722096729","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-26T07:17:36.995032+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-26T07:17:36.932266+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-26T07:17:36.829304+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-26T07:17:36.804297+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-26T07:17:36.667892+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-26T07:17:36.964868+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-26T07:17:36.827332+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-26T07:17:36.893179+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-11-26T07:22:19.344 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:19.344+0000 7fd3d0878640 1 -- 172.21.15.143:0/3200850260 >> [v2:172.21.15.143:6832/2150557544,v1:172.21.15.143:6833/2150557544] conn(0x7fd3b80667d0 msgr2=0x7fd3b8068c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:22:19.344 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:19.344+0000 7fd3d0878640 1 --2- 172.21.15.143:0/3200850260 >> [v2:172.21.15.143:6832/2150557544,v1:172.21.15.143:6833/2150557544] conn(0x7fd3b80667d0 0x7fd3b8068c90 secure :-1 s=READY pgs=406 cs=0 l=1 rev1=1 crypto rx=0x7fd3bc000f70 tx=0x7fd3bc007c40 comp rx=0 tx=0).stop 2023-11-26T07:22:19.345 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:19.344+0000 7fd3d0878640 1 -- 172.21.15.143:0/3200850260 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fd3cc143810 msgr2=0x7fd3cc14fd20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:22:19.345 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:19.344+0000 7fd3d0878640 1 --2- 172.21.15.143:0/3200850260 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fd3cc143810 0x7fd3cc14fd20 secure :-1 s=READY pgs=865 cs=0 l=1 rev1=1 crypto rx=0x7fd3c00ad040 tx=0x7fd3c001ac50 comp rx=0 tx=0).stop 2023-11-26T07:22:19.345 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:19.344+0000 7fd3d0878640 1 -- 172.21.15.143:0/3200850260 shutdown_connections 2023-11-26T07:22:19.345 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:19.344+0000 7fd3d0878640 1 --2- 172.21.15.143:0/3200850260 >> [v2:172.21.15.143:6832/2150557544,v1:172.21.15.143:6833/2150557544] conn(0x7fd3b80667d0 0x7fd3b8068c90 unknown :-1 s=CLOSED pgs=406 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:22:19.345 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:19.344+0000 7fd3d0878640 1 --2- 172.21.15.143:0/3200850260 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fd3cc15df20 0x7fd3cc13cfb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:22:19.345 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:19.344+0000 7fd3d0878640 1 --2- 172.21.15.143:0/3200850260 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fd3cc143810 0x7fd3cc14fd20 unknown :-1 s=CLOSED pgs=865 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:22:19.345 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:19.344+0000 7fd3d0878640 1 --2- 172.21.15.143:0/3200850260 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fd3cc0b1e10 0x7fd3cc1515f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:22:19.345 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:19.344+0000 7fd3d0878640 1 -- 172.21.15.143:0/3200850260 >> 172.21.15.143:0/3200850260 conn(0x7fd3cc06a890 msgr2=0x7fd3cc0b2470 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:22:19.345 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:19.344+0000 7fd3d0878640 1 -- 172.21.15.143:0/3200850260 shutdown_connections 2023-11-26T07:22:19.345 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:19.344+0000 7fd3d0878640 1 -- 172.21.15.143:0/3200850260 wait complete. 2023-11-26T07:22:19.363 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-11-26T07:17:41.758597+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 8, 'pool_name': 'cephfs_metadata', 'create_time': '2023-11-26T07:21:53.382407+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '58', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 9, 'pool_name': 'cephfs_data', 'create_time': '2023-11-26T07:21:54.656298+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '59', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.6299999952316284, 'score_stable': 1.6299999952316284, 'optimal_score': 1, 'raw_score_acting': 1.6299999952316284, 'raw_score_stable': 1.6299999952316284, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2023-11-26T07:22:19.364 DEBUG:teuthology.orchestra.run.smithi143:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-11-26T07:22:19.504 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:19.504+0000 7f1937ff5640 1 Processor -- start 2023-11-26T07:22:19.504 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:19.504+0000 7f1937ff5640 1 -- start start 2023-11-26T07:22:19.506 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:19.504+0000 7f1937ff5640 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7f193014b1e0 0x7f19300cf280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:22:19.506 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:19.504+0000 7f1937ff5640 1 -- --> v1:172.21.15.143:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f19300cfc20 con 0x7f19300cf7c0 2023-11-26T07:22:19.506 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:19.504+0000 7f1937ff5640 1 -- --> v1:172.21.15.188:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1930162080 con 0x7f1930070fd0 2023-11-26T07:22:19.506 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:19.504+0000 7f1937ff5640 1 -- --> v2:172.21.15.188:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1930161d70 con 0x7f193014b1e0 2023-11-26T07:22:19.506 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:19.504+0000 7f1935590640 1 --1- >> v1:172.21.15.143:6789/0 conn(0x7f19300cf7c0 0x7f19300708c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.143:6789/0 says I am v1:172.21.15.143:59180/0 (socket says 172.21.15.143:59180) 2023-11-26T07:22:19.506 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:19.504+0000 7f1935590640 1 -- 172.21.15.143:0/1323720790 learned_addr learned my addr 172.21.15.143:0/1323720790 (peer_addr_for_me v1:172.21.15.143:0/0) 2023-11-26T07:22:19.507 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:19.504+0000 7f1935d91640 1 --2- 172.21.15.143:0/1323720790 >> v2:172.21.15.188:3300/0 conn(0x7f193014b1e0 0x7f19300cf280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:22:19.507 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:19.504+0000 7f1934d8f640 1 -- 172.21.15.143:0/1323720790 <== mon.0 v1:172.21.15.143:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3300952794 0 0) 0x7f19300cfc20 con 0x7f19300cf7c0 2023-11-26T07:22:19.507 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:19.504+0000 7f1934d8f640 1 -- 172.21.15.143:0/1323720790 --> v1:172.21.15.143:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f191c003530 con 0x7f19300cf7c0 2023-11-26T07:22:19.507 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:19.504+0000 7f1935d91640 1 -- 172.21.15.143:0/1323720790 >> v1:172.21.15.188:6789/0 conn(0x7f1930070fd0 legacy=0x7f1930162510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:22:19.507 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:19.504+0000 7f1935d91640 1 -- 172.21.15.143:0/1323720790 >> v1:172.21.15.143:6789/0 conn(0x7f19300cf7c0 legacy=0x7f19300708c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:22:19.507 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:19.504+0000 7f1935d91640 1 -- 172.21.15.143:0/1323720790 --> v2:172.21.15.188:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f193014b5c0 con 0x7f193014b1e0 2023-11-26T07:22:19.509 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:19.508+0000 7f1935d91640 1 --2- 172.21.15.143:0/1323720790 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f193014b1e0 0x7f19300cf280 secure :-1 s=READY pgs=341 cs=0 l=1 rev1=1 crypto rx=0x7f192c0097c0 tx=0x7f192c01c450 comp rx=0 tx=0).ready entity=mon.1 client_cookie=9ef439816c085286 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:22:19.510 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:19.508+0000 7f1934d8f640 1 -- 172.21.15.143:0/1323720790 <== mon.1 v2:172.21.15.188:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f192c0ab650 con 0x7f193014b1e0 2023-11-26T07:22:19.510 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:19.508+0000 7f1934d8f640 1 -- 172.21.15.143:0/1323720790 <== mon.1 v2:172.21.15.188:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f192c0ab7b0 con 0x7f193014b1e0 2023-11-26T07:22:19.510 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:19.508+0000 7f1934d8f640 1 -- 172.21.15.143:0/1323720790 <== mon.1 v2:172.21.15.188:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f192c0abb20 con 0x7f193014b1e0 2023-11-26T07:22:19.511 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:19.508+0000 7f1937ff5640 1 -- 172.21.15.143:0/1323720790 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f193014b1e0 msgr2=0x7f19300cf280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:22:19.511 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:19.508+0000 7f1937ff5640 1 --2- 172.21.15.143:0/1323720790 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f193014b1e0 0x7f19300cf280 secure :-1 s=READY pgs=341 cs=0 l=1 rev1=1 crypto rx=0x7f192c0097c0 tx=0x7f192c01c450 comp rx=0 tx=0).stop 2023-11-26T07:22:19.511 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:19.508+0000 7f1937ff5640 1 -- 172.21.15.143:0/1323720790 shutdown_connections 2023-11-26T07:22:19.511 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:19.508+0000 7f1937ff5640 1 --2- 172.21.15.143:0/1323720790 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f193014b1e0 0x7f19300cf280 unknown :-1 s=CLOSED pgs=341 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:22:19.511 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:19.508+0000 7f1937ff5640 1 -- 172.21.15.143:0/1323720790 >> 172.21.15.143:0/1323720790 conn(0x7f193006a890 msgr2=0x7f193006aca0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:22:19.511 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:19.508+0000 7f1937ff5640 1 -- 172.21.15.143:0/1323720790 shutdown_connections 2023-11-26T07:22:19.511 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:19.508+0000 7f1937ff5640 1 -- 172.21.15.143:0/1323720790 wait complete. 2023-11-26T07:22:19.512 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:19.512+0000 7f1937ff5640 1 Processor -- start 2023-11-26T07:22:19.512 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:19.512+0000 7f1937ff5640 1 -- start start 2023-11-26T07:22:19.514 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:19.512+0000 7f1937ff5640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f1930070fd0 0x7f19300aa970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:22:19.514 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:19.512+0000 7f1937ff5640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f19300cf7c0 0x7f19300a9080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:22:19.514 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:19.512+0000 7f1937ff5640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f193014b1e0 0x7f19300abb00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:22:19.514 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:19.512+0000 7f1937ff5640 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f19300aa550 con 0x7f19300cf7c0 2023-11-26T07:22:19.514 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:19.512+0000 7f1937ff5640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f19300a98d0 con 0x7f193014b1e0 2023-11-26T07:22:19.515 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:19.512+0000 7f1937ff5640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f19300a9bb0 con 0x7f1930070fd0 2023-11-26T07:22:19.515 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:19.512+0000 7f1935590640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f19300cf7c0 0x7f19300a9080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:22:19.515 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:19.512+0000 7f1935590640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f19300cf7c0 0x7f19300a9080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:34290/0 (socket says 172.21.15.143:34290) 2023-11-26T07:22:19.515 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:19.512+0000 7f1935590640 1 -- 172.21.15.143:0/3201000444 learned_addr learned my addr 172.21.15.143:0/3201000444 (peer_addr_for_me v2:172.21.15.143:0/0) 2023-11-26T07:22:19.515 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:19.512+0000 7f1936592640 1 --2- 172.21.15.143:0/3201000444 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f193014b1e0 0x7f19300abb00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:22:19.515 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:19.512+0000 7f1935d91640 1 --2- 172.21.15.143:0/3201000444 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f1930070fd0 0x7f19300aa970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:22:19.515 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:19.512+0000 7f1935590640 1 -- 172.21.15.143:0/3201000444 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f1930070fd0 msgr2=0x7f19300aa970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:22:19.515 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:19.512+0000 7f1935590640 1 --2- 172.21.15.143:0/3201000444 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f1930070fd0 0x7f19300aa970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:22:19.515 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:19.512+0000 7f1935590640 1 -- 172.21.15.143:0/3201000444 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f193014b1e0 msgr2=0x7f19300abb00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:22:19.515 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:19.512+0000 7f1935590640 1 --2- 172.21.15.143:0/3201000444 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f193014b1e0 0x7f19300abb00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:22:19.515 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:19.512+0000 7f1935590640 1 -- 172.21.15.143:0/3201000444 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f192c009420 con 0x7f19300cf7c0 2023-11-26T07:22:19.516 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:19.512+0000 7f1936592640 1 --2- 172.21.15.143:0/3201000444 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f193014b1e0 0x7f19300abb00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-26T07:22:19.516 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:19.512+0000 7f1935d91640 1 --2- 172.21.15.143:0/3201000444 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f1930070fd0 0x7f19300aa970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-26T07:22:19.516 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:19.512+0000 7f1935590640 1 --2- 172.21.15.143:0/3201000444 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f19300cf7c0 0x7f19300a9080 secure :-1 s=READY pgs=867 cs=0 l=1 rev1=1 crypto rx=0x7f192000c4d0 tx=0x7f192000c990 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:22:19.517 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:19.512+0000 7f19267fc640 1 -- 172.21.15.143:0/3201000444 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1920005cf0 con 0x7f19300cf7c0 2023-11-26T07:22:19.517 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:19.512+0000 7f19267fc640 1 -- 172.21.15.143:0/3201000444 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1920005e50 con 0x7f19300cf7c0 2023-11-26T07:22:19.517 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:19.512+0000 7f1937ff5640 1 -- 172.21.15.143:0/3201000444 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f19300a9e90 con 0x7f19300cf7c0 2023-11-26T07:22:19.517 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:19.512+0000 7f19267fc640 1 -- 172.21.15.143:0/3201000444 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1920007270 con 0x7f19300cf7c0 2023-11-26T07:22:19.517 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:19.512+0000 7f1937ff5640 1 -- 172.21.15.143:0/3201000444 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f19300a95c0 con 0x7f19300cf7c0 2023-11-26T07:22:19.517 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:19.516+0000 7f1937ff5640 1 -- 172.21.15.143:0/3201000444 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f19300aa690 con 0x7f19300cf7c0 2023-11-26T07:22:19.517 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:19.516+0000 7f19267fc640 1 -- 172.21.15.143:0/3201000444 <== mon.0 v2:172.21.15.143:3300/0 4 ==== mgrmap(e 11) v1 ==== 86158+0+0 (secure 0 0 0) 0x7f19200073d0 con 0x7f19300cf7c0 2023-11-26T07:22:19.518 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:19.516+0000 7f19267fc640 1 --2- 172.21.15.143:0/3201000444 >> [v2:172.21.15.143:6832/2150557544,v1:172.21.15.143:6833/2150557544] conn(0x7f19140667d0 0x7f1914068c90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:22:19.518 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:19.516+0000 7f19267fc640 1 -- 172.21.15.143:0/3201000444 <== mon.0 v2:172.21.15.143:3300/0 5 ==== osd_map(59..59 src has 1..59) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f1920083980 con 0x7f19300cf7c0 2023-11-26T07:22:19.518 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:19.516+0000 7f1935d91640 1 --2- 172.21.15.143:0/3201000444 >> [v2:172.21.15.143:6832/2150557544,v1:172.21.15.143:6833/2150557544] conn(0x7f19140667d0 0x7f1914068c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:22:19.518 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:19.516+0000 7f1935d91640 1 --2- 172.21.15.143:0/3201000444 >> [v2:172.21.15.143:6832/2150557544,v1:172.21.15.143:6833/2150557544] conn(0x7f19140667d0 0x7f1914068c90 secure :-1 s=READY pgs=407 cs=0 l=1 rev1=1 crypto rx=0x7f192c0af000 tx=0x7f192c0a7000 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:22:19.521 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:19.520+0000 7f19267fc640 1 -- 172.21.15.143:0/3201000444 <== mon.0 v2:172.21.15.143:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172649 (secure 0 0 0) 0x7f19300aa690 con 0x7f19300cf7c0 2023-11-26T07:22:19.673 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:19.672+0000 7f1937ff5640 1 -- 172.21.15.143:0/3201000444 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f19300aa690 con 0x7f19300cf7c0 2023-11-26T07:22:19.673 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:19.672+0000 7f19267fc640 1 -- 172.21.15.143:0/3201000444 <== mon.0 v2:172.21.15.143:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v59) v1 ==== 93+0+10 (secure 0 0 0) 0x7f19300aa690 con 0x7f19300cf7c0 2023-11-26T07:22:19.673 INFO:teuthology.orchestra.run.smithi143.stdout:pg_num: 1 2023-11-26T07:22:19.675 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:19.672+0000 7f1937ff5640 1 -- 172.21.15.143:0/3201000444 >> [v2:172.21.15.143:6832/2150557544,v1:172.21.15.143:6833/2150557544] conn(0x7f19140667d0 msgr2=0x7f1914068c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:22:19.675 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:19.672+0000 7f1937ff5640 1 --2- 172.21.15.143:0/3201000444 >> [v2:172.21.15.143:6832/2150557544,v1:172.21.15.143:6833/2150557544] conn(0x7f19140667d0 0x7f1914068c90 secure :-1 s=READY pgs=407 cs=0 l=1 rev1=1 crypto rx=0x7f192c0af000 tx=0x7f192c0a7000 comp rx=0 tx=0).stop 2023-11-26T07:22:19.676 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:19.672+0000 7f1937ff5640 1 -- 172.21.15.143:0/3201000444 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f19300cf7c0 msgr2=0x7f19300a9080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:22:19.676 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:19.672+0000 7f1937ff5640 1 --2- 172.21.15.143:0/3201000444 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f19300cf7c0 0x7f19300a9080 secure :-1 s=READY pgs=867 cs=0 l=1 rev1=1 crypto rx=0x7f192000c4d0 tx=0x7f192000c990 comp rx=0 tx=0).stop 2023-11-26T07:22:19.676 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:19.672+0000 7f1937ff5640 1 -- 172.21.15.143:0/3201000444 shutdown_connections 2023-11-26T07:22:19.676 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:19.672+0000 7f1937ff5640 1 --2- 172.21.15.143:0/3201000444 >> [v2:172.21.15.143:6832/2150557544,v1:172.21.15.143:6833/2150557544] conn(0x7f19140667d0 0x7f1914068c90 unknown :-1 s=CLOSED pgs=407 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:22:19.676 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:19.672+0000 7f1937ff5640 1 --2- 172.21.15.143:0/3201000444 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f193014b1e0 0x7f19300abb00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:22:19.676 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:19.672+0000 7f1937ff5640 1 --2- 172.21.15.143:0/3201000444 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f19300cf7c0 0x7f19300a9080 unknown :-1 s=CLOSED pgs=867 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:22:19.676 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:19.672+0000 7f1937ff5640 1 --2- 172.21.15.143:0/3201000444 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f1930070fd0 0x7f19300aa970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:22:19.676 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:19.672+0000 7f1937ff5640 1 -- 172.21.15.143:0/3201000444 >> 172.21.15.143:0/3201000444 conn(0x7f193006a890 msgr2=0x7f193016cc70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:22:19.676 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:19.672+0000 7f1937ff5640 1 -- 172.21.15.143:0/3201000444 shutdown_connections 2023-11-26T07:22:19.676 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:19.672+0000 7f1937ff5640 1 -- 172.21.15.143:0/3201000444 wait complete. 2023-11-26T07:22:19.694 DEBUG:teuthology.orchestra.run.smithi143:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-11-26T07:22:19.835 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:19.832+0000 7fd2d5fd0640 1 Processor -- start 2023-11-26T07:22:19.835 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:19.832+0000 7fd2d5fd0640 1 -- start start 2023-11-26T07:22:19.838 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:19.836+0000 7fd2d5fd0640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fd2d0165590 0x7fd2d0070990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:22:19.838 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:19.836+0000 7fd2d5fd0640 1 -- --> v1:172.21.15.143:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd2d015f530 con 0x7fd2d015f150 2023-11-26T07:22:19.838 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:19.836+0000 7fd2d5fd0640 1 -- --> v1:172.21.15.188:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd2d0165a20 con 0x7fd2d01650e0 2023-11-26T07:22:19.838 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:19.836+0000 7fd2d5fd0640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd2d006e050 con 0x7fd2d0165590 2023-11-26T07:22:19.838 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:19.836+0000 7fd2d4fce640 1 --1- >> v1:172.21.15.143:6789/0 conn(0x7fd2d015f150 0x7fd2d006c880 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.143:6789/0 says I am v1:172.21.15.143:59196/0 (socket says 172.21.15.143:59196) 2023-11-26T07:22:19.838 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:19.836+0000 7fd2d4fce640 1 -- 172.21.15.143:0/2207725677 learned_addr learned my addr 172.21.15.143:0/2207725677 (peer_addr_for_me v1:172.21.15.143:0/0) 2023-11-26T07:22:19.838 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:19.836+0000 7fd2d57cf640 1 --2- 172.21.15.143:0/2207725677 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fd2d0165590 0x7fd2d0070990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:22:19.839 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:19.836+0000 7fd2cf7fe640 1 -- 172.21.15.143:0/2207725677 <== mon.0 v1:172.21.15.143:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1129113492 0 0) 0x7fd2d015f530 con 0x7fd2d015f150 2023-11-26T07:22:19.839 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:19.836+0000 7fd2cf7fe640 1 -- 172.21.15.143:0/2207725677 --> v1:172.21.15.143:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fd2c0003530 con 0x7fd2d015f150 2023-11-26T07:22:19.839 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:19.836+0000 7fd2cf7fe640 1 -- 172.21.15.143:0/2207725677 <== mon.1 v1:172.21.15.188:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4035373810 0 0) 0x7fd2d0165a20 con 0x7fd2d01650e0 2023-11-26T07:22:19.839 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:19.836+0000 7fd2cf7fe640 1 -- 172.21.15.143:0/2207725677 --> v1:172.21.15.188:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fd2d015f530 con 0x7fd2d01650e0 2023-11-26T07:22:19.839 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:19.836+0000 7fd2cf7fe640 1 -- 172.21.15.143:0/2207725677 <== mon.0 v1:172.21.15.143:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 462828342 0 0) 0x7fd2c0003530 con 0x7fd2d015f150 2023-11-26T07:22:19.839 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:19.836+0000 7fd2d57cf640 1 -- 172.21.15.143:0/2207725677 >> v1:172.21.15.188:6789/0 conn(0x7fd2d01650e0 legacy=0x7fd2d0070280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:22:19.839 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:19.836+0000 7fd2d57cf640 1 -- 172.21.15.143:0/2207725677 >> v1:172.21.15.143:6789/0 conn(0x7fd2d015f150 legacy=0x7fd2d006c880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:22:19.839 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:19.836+0000 7fd2d57cf640 1 -- 172.21.15.143:0/2207725677 --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd2d0165ec0 con 0x7fd2d0165590 2023-11-26T07:22:19.841 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:19.840+0000 7fd2d57cf640 1 --2- 172.21.15.143:0/2207725677 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fd2d0165590 0x7fd2d0070990 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7fd2c8009dc0 tx=0x7fd2c801c860 comp rx=0 tx=0).ready entity=mon.2 client_cookie=993a7a9eb7a937ae server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:22:19.841 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:19.840+0000 7fd2cf7fe640 1 -- 172.21.15.143:0/2207725677 <== mon.2 v2:172.21.15.188:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd2c4001680 con 0x7fd2d0165590 2023-11-26T07:22:19.841 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:19.840+0000 7fd2cf7fe640 1 -- 172.21.15.143:0/2207725677 <== mon.2 v2:172.21.15.188:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd2c80a56a0 con 0x7fd2d0165590 2023-11-26T07:22:19.841 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:19.840+0000 7fd2cf7fe640 1 -- 172.21.15.143:0/2207725677 <== mon.2 v2:172.21.15.188:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd2c80a5a30 con 0x7fd2d0165590 2023-11-26T07:22:19.842 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:19.840+0000 7fd2d5fd0640 1 -- 172.21.15.143:0/2207725677 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fd2d0165590 msgr2=0x7fd2d0070990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:22:19.842 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:19.840+0000 7fd2d5fd0640 1 --2- 172.21.15.143:0/2207725677 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fd2d0165590 0x7fd2d0070990 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7fd2c8009dc0 tx=0x7fd2c801c860 comp rx=0 tx=0).stop 2023-11-26T07:22:19.843 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:19.840+0000 7fd2d5fd0640 1 -- 172.21.15.143:0/2207725677 shutdown_connections 2023-11-26T07:22:19.843 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:19.840+0000 7fd2d5fd0640 1 --2- 172.21.15.143:0/2207725677 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fd2d0165590 0x7fd2d0070990 unknown :-1 s=CLOSED pgs=109 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:22:19.843 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:19.840+0000 7fd2d5fd0640 1 -- 172.21.15.143:0/2207725677 >> 172.21.15.143:0/2207725677 conn(0x7fd2d006a890 msgr2=0x7fd2d006aca0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:22:19.843 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:19.840+0000 7fd2d5fd0640 1 -- 172.21.15.143:0/2207725677 shutdown_connections 2023-11-26T07:22:19.843 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:19.840+0000 7fd2d5fd0640 1 -- 172.21.15.143:0/2207725677 wait complete. 2023-11-26T07:22:19.843 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:19.840+0000 7fd2d5fd0640 1 Processor -- start 2023-11-26T07:22:19.844 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:19.844+0000 7fd2d5fd0640 1 -- start start 2023-11-26T07:22:19.845 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:19.844+0000 7fd2d5fd0640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fd2d015f150 0x7fd2d00cccd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:22:19.845 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:19.844+0000 7fd2d5fd0640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fd2d01650e0 0x7fd2d0159ba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:22:19.845 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:19.844+0000 7fd2d5fd0640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fd2d0165590 0x7fd2d015d110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:22:19.845 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:19.844+0000 7fd2d5fd0640 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd2d0162540 con 0x7fd2d0165590 2023-11-26T07:22:19.846 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:19.844+0000 7fd2d5fd0640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd2d00ce2d0 con 0x7fd2d015f150 2023-11-26T07:22:19.846 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:19.844+0000 7fd2d5fd0640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd2d0161060 con 0x7fd2d01650e0 2023-11-26T07:22:19.846 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:19.844+0000 7fd2d57cf640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fd2d0165590 0x7fd2d015d110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:22:19.846 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:19.844+0000 7fd2d57cf640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fd2d0165590 0x7fd2d015d110 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:34302/0 (socket says 172.21.15.143:34302) 2023-11-26T07:22:19.846 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:19.844+0000 7fd2d57cf640 1 -- 172.21.15.143:0/4148465185 learned_addr learned my addr 172.21.15.143:0/4148465185 (peer_addr_for_me v2:172.21.15.143:0/0) 2023-11-26T07:22:19.846 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:19.844+0000 7fd2cffff640 1 --2- 172.21.15.143:0/4148465185 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fd2d01650e0 0x7fd2d0159ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:22:19.846 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:19.844+0000 7fd2d4fce640 1 --2- 172.21.15.143:0/4148465185 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fd2d015f150 0x7fd2d00cccd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:22:19.847 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:19.844+0000 7fd2d57cf640 1 -- 172.21.15.143:0/4148465185 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fd2d01650e0 msgr2=0x7fd2d0159ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:22:19.848 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:19.844+0000 7fd2d57cf640 1 --2- 172.21.15.143:0/4148465185 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fd2d01650e0 0x7fd2d0159ba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:22:19.848 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:19.844+0000 7fd2d57cf640 1 -- 172.21.15.143:0/4148465185 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fd2d015f150 msgr2=0x7fd2d00cccd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:22:19.848 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:19.844+0000 7fd2d57cf640 1 --2- 172.21.15.143:0/4148465185 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fd2d015f150 0x7fd2d00cccd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:22:19.848 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:19.844+0000 7fd2d57cf640 1 -- 172.21.15.143:0/4148465185 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd2c8009a70 con 0x7fd2d0165590 2023-11-26T07:22:19.848 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:19.844+0000 7fd2d57cf640 1 --2- 172.21.15.143:0/4148465185 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fd2d0165590 0x7fd2d015d110 secure :-1 s=READY pgs=869 cs=0 l=1 rev1=1 crypto rx=0x7fd2c801af10 tx=0x7fd2d00716e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:22:19.848 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:19.844+0000 7fd2cd7fa640 1 -- 172.21.15.143:0/4148465185 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd2c8003ba0 con 0x7fd2d0165590 2023-11-26T07:22:19.848 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:19.844+0000 7fd2cd7fa640 1 -- 172.21.15.143:0/4148465185 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd2c8003d00 con 0x7fd2d0165590 2023-11-26T07:22:19.848 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:19.844+0000 7fd2d5fd0640 1 -- 172.21.15.143:0/4148465185 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd2d00cd210 con 0x7fd2d0165590 2023-11-26T07:22:19.848 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:19.844+0000 7fd2cd7fa640 1 -- 172.21.15.143:0/4148465185 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd2c80ad890 con 0x7fd2d0165590 2023-11-26T07:22:19.848 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:19.844+0000 7fd2d5fd0640 1 -- 172.21.15.143:0/4148465185 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd2d00cd980 con 0x7fd2d0165590 2023-11-26T07:22:19.849 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:19.844+0000 7fd2d5fd0640 1 -- 172.21.15.143:0/4148465185 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd2d01611f0 con 0x7fd2d0165590 2023-11-26T07:22:19.849 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:19.848+0000 7fd2cd7fa640 1 -- 172.21.15.143:0/4148465185 <== mon.0 v2:172.21.15.143:3300/0 4 ==== mgrmap(e 11) v1 ==== 86158+0+0 (secure 0 0 0) 0x7fd2c80b4040 con 0x7fd2d0165590 2023-11-26T07:22:19.849 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:19.848+0000 7fd2cd7fa640 1 --2- 172.21.15.143:0/4148465185 >> [v2:172.21.15.143:6832/2150557544,v1:172.21.15.143:6833/2150557544] conn(0x7fd2b4066800 0x7fd2b4068cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:22:19.849 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:19.848+0000 7fd2cd7fa640 1 -- 172.21.15.143:0/4148465185 <== mon.0 v2:172.21.15.143:3300/0 5 ==== osd_map(59..59 src has 1..59) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fd2c811dd30 con 0x7fd2d0165590 2023-11-26T07:22:19.850 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:19.848+0000 7fd2d4fce640 1 --2- 172.21.15.143:0/4148465185 >> [v2:172.21.15.143:6832/2150557544,v1:172.21.15.143:6833/2150557544] conn(0x7fd2b4066800 0x7fd2b4068cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:22:19.850 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:19.848+0000 7fd2d4fce640 1 --2- 172.21.15.143:0/4148465185 >> [v2:172.21.15.143:6832/2150557544,v1:172.21.15.143:6833/2150557544] conn(0x7fd2b4066800 0x7fd2b4068cc0 secure :-1 s=READY pgs=408 cs=0 l=1 rev1=1 crypto rx=0x7fd2c4000f70 tx=0x7fd2c4001700 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:22:19.853 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:19.852+0000 7fd2cd7fa640 1 -- 172.21.15.143:0/4148465185 <== mon.0 v2:172.21.15.143:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172649 (secure 0 0 0) 0x7fd2c80eaed0 con 0x7fd2d0165590 2023-11-26T07:22:20.005 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:20.004+0000 7fd2d5fd0640 1 -- 172.21.15.143:0/4148465185 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7fd2d0067500 con 0x7fd2d0165590 2023-11-26T07:22:20.006 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:20.004+0000 7fd2cd7fa640 1 -- 172.21.15.143:0/4148465185 <== mon.0 v2:172.21.15.143:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v59) v1 ==== 104+0+11 (secure 0 0 0) 0x7fd2c8009450 con 0x7fd2d0165590 2023-11-26T07:22:20.006 INFO:teuthology.orchestra.run.smithi143.stdout:pg_num: 32 2023-11-26T07:22:20.008 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:20.008+0000 7fd2d5fd0640 1 -- 172.21.15.143:0/4148465185 >> [v2:172.21.15.143:6832/2150557544,v1:172.21.15.143:6833/2150557544] conn(0x7fd2b4066800 msgr2=0x7fd2b4068cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:22:20.008 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:20.008+0000 7fd2d5fd0640 1 --2- 172.21.15.143:0/4148465185 >> [v2:172.21.15.143:6832/2150557544,v1:172.21.15.143:6833/2150557544] conn(0x7fd2b4066800 0x7fd2b4068cc0 secure :-1 s=READY pgs=408 cs=0 l=1 rev1=1 crypto rx=0x7fd2c4000f70 tx=0x7fd2c4001700 comp rx=0 tx=0).stop 2023-11-26T07:22:20.008 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:20.008+0000 7fd2d5fd0640 1 -- 172.21.15.143:0/4148465185 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fd2d0165590 msgr2=0x7fd2d015d110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:22:20.009 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:20.008+0000 7fd2d5fd0640 1 --2- 172.21.15.143:0/4148465185 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fd2d0165590 0x7fd2d015d110 secure :-1 s=READY pgs=869 cs=0 l=1 rev1=1 crypto rx=0x7fd2c801af10 tx=0x7fd2d00716e0 comp rx=0 tx=0).stop 2023-11-26T07:22:20.009 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:20.008+0000 7fd2d5fd0640 1 -- 172.21.15.143:0/4148465185 shutdown_connections 2023-11-26T07:22:20.009 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:20.008+0000 7fd2d5fd0640 1 --2- 172.21.15.143:0/4148465185 >> [v2:172.21.15.143:6832/2150557544,v1:172.21.15.143:6833/2150557544] conn(0x7fd2b4066800 0x7fd2b4068cc0 unknown :-1 s=CLOSED pgs=408 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:22:20.009 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:20.008+0000 7fd2d5fd0640 1 --2- 172.21.15.143:0/4148465185 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fd2d0165590 0x7fd2d015d110 unknown :-1 s=CLOSED pgs=869 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:22:20.009 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:20.008+0000 7fd2d5fd0640 1 --2- 172.21.15.143:0/4148465185 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fd2d01650e0 0x7fd2d0159ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:22:20.009 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:20.008+0000 7fd2d5fd0640 1 --2- 172.21.15.143:0/4148465185 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fd2d015f150 0x7fd2d00cccd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:22:20.009 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:20.008+0000 7fd2d5fd0640 1 -- 172.21.15.143:0/4148465185 >> 172.21.15.143:0/4148465185 conn(0x7fd2d006a890 msgr2=0x7fd2d01549e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:22:20.009 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:20.008+0000 7fd2d5fd0640 1 -- 172.21.15.143:0/4148465185 shutdown_connections 2023-11-26T07:22:20.009 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:20.008+0000 7fd2d5fd0640 1 -- 172.21.15.143:0/4148465185 wait complete. 2023-11-26T07:22:20.027 DEBUG:teuthology.orchestra.run.smithi143:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-11-26T07:22:20.166 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:20.164+0000 7faebc3f8640 1 Processor -- start 2023-11-26T07:22:20.166 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:20.164+0000 7faebc3f8640 1 -- start start 2023-11-26T07:22:20.167 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:20.164+0000 7faebc3f8640 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7faeb40b2f80 0x7faeb4071850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:22:20.167 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:20.164+0000 7faebc3f8640 1 -- --> v1:172.21.15.143:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7faeb406c750 con 0x7faeb41596d0 2023-11-26T07:22:20.168 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:20.164+0000 7faebc3f8640 1 -- --> v1:172.21.15.188:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7faeb40b3550 con 0x7faeb40ce680 2023-11-26T07:22:20.168 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:20.164+0000 7faebc3f8640 1 -- --> v2:172.21.15.188:3300/0 -- mon_getmap magic: 0 v1 -- 0x7faeb40cbe00 con 0x7faeb40b2f80 2023-11-26T07:22:20.168 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:20.164+0000 7faeba194640 1 --1- >> v1:172.21.15.143:6789/0 conn(0x7faeb41596d0 0x7faeb40b2870 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.143:6789/0 says I am v1:172.21.15.143:59204/0 (socket says 172.21.15.143:59204) 2023-11-26T07:22:20.168 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:20.164+0000 7faeba194640 1 -- 172.21.15.143:0/1428129487 learned_addr learned my addr 172.21.15.143:0/1428129487 (peer_addr_for_me v1:172.21.15.143:0/0) 2023-11-26T07:22:20.168 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:20.164+0000 7faeb9993640 1 --2- 172.21.15.143:0/1428129487 >> v2:172.21.15.188:3300/0 conn(0x7faeb40b2f80 0x7faeb4071850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:22:20.168 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:20.168+0000 7faeb9192640 1 -- 172.21.15.143:0/1428129487 <== mon.0 v1:172.21.15.143:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4161233751 0 0) 0x7faeb406c750 con 0x7faeb41596d0 2023-11-26T07:22:20.169 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:20.168+0000 7faeb9192640 1 -- 172.21.15.143:0/1428129487 --> v1:172.21.15.143:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7faea0003530 con 0x7faeb41596d0 2023-11-26T07:22:20.169 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:20.168+0000 7faeb9993640 1 -- 172.21.15.143:0/1428129487 >> v1:172.21.15.188:6789/0 conn(0x7faeb40ce680 legacy=0x7faeb406cbe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:22:20.169 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:20.168+0000 7faeb9993640 1 -- 172.21.15.143:0/1428129487 >> v1:172.21.15.143:6789/0 conn(0x7faeb41596d0 legacy=0x7faeb40b2870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:22:20.169 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:20.168+0000 7faeb9993640 1 -- 172.21.15.143:0/1428129487 --> v2:172.21.15.188:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7faeb4071e80 con 0x7faeb40b2f80 2023-11-26T07:22:20.171 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:20.168+0000 7faeb9993640 1 --2- 172.21.15.143:0/1428129487 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7faeb40b2f80 0x7faeb4071850 secure :-1 s=READY pgs=344 cs=0 l=1 rev1=1 crypto rx=0x7faeb40cbf40 tx=0x7faea401c3f0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=c9bc24d1173cf6d5 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:22:20.171 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:20.168+0000 7faeb9192640 1 -- 172.21.15.143:0/1428129487 <== mon.1 v2:172.21.15.188:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faea40ab600 con 0x7faeb40b2f80 2023-11-26T07:22:20.171 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:20.168+0000 7faeb9192640 1 -- 172.21.15.143:0/1428129487 <== mon.1 v2:172.21.15.188:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7faea40a7050 con 0x7faeb40b2f80 2023-11-26T07:22:20.171 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:20.168+0000 7faeb9192640 1 -- 172.21.15.143:0/1428129487 <== mon.1 v2:172.21.15.188:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faea40ab980 con 0x7faeb40b2f80 2023-11-26T07:22:20.172 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:20.172+0000 7faebc3f8640 1 -- 172.21.15.143:0/1428129487 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7faeb40b2f80 msgr2=0x7faeb4071850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:22:20.172 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:20.172+0000 7faebc3f8640 1 --2- 172.21.15.143:0/1428129487 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7faeb40b2f80 0x7faeb4071850 secure :-1 s=READY pgs=344 cs=0 l=1 rev1=1 crypto rx=0x7faeb40cbf40 tx=0x7faea401c3f0 comp rx=0 tx=0).stop 2023-11-26T07:22:20.172 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:20.172+0000 7faebc3f8640 1 -- 172.21.15.143:0/1428129487 shutdown_connections 2023-11-26T07:22:20.172 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:20.172+0000 7faebc3f8640 1 --2- 172.21.15.143:0/1428129487 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7faeb40b2f80 0x7faeb4071850 unknown :-1 s=CLOSED pgs=344 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:22:20.172 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:20.172+0000 7faebc3f8640 1 -- 172.21.15.143:0/1428129487 >> 172.21.15.143:0/1428129487 conn(0x7faeb406a890 msgr2=0x7faeb406aca0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:22:20.173 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:20.172+0000 7faebc3f8640 1 -- 172.21.15.143:0/1428129487 shutdown_connections 2023-11-26T07:22:20.173 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:20.172+0000 7faebc3f8640 1 -- 172.21.15.143:0/1428129487 wait complete. 2023-11-26T07:22:20.174 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:20.172+0000 7faebc3f8640 1 Processor -- start 2023-11-26T07:22:20.174 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:20.172+0000 7faebc3f8640 1 -- start start 2023-11-26T07:22:20.175 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:20.172+0000 7faebc3f8640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7faeb40b2f80 0x7faeb415c6e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:22:20.175 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:20.172+0000 7faebc3f8640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7faeb40ce680 0x7faeb4154840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:22:20.175 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:20.172+0000 7faebc3f8640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7faeb41596d0 0x7faeb4154d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:22:20.176 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:20.172+0000 7faebc3f8640 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faeb4158fb0 con 0x7faeb40ce680 2023-11-26T07:22:20.176 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:20.172+0000 7faebc3f8640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faeb415cc20 con 0x7faeb41596d0 2023-11-26T07:22:20.176 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:20.172+0000 7faebc3f8640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7faeb4158cd0 con 0x7faeb40b2f80 2023-11-26T07:22:20.176 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:20.172+0000 7faeb9993640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7faeb40ce680 0x7faeb4154840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:22:20.176 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:20.172+0000 7faeb9993640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7faeb40ce680 0x7faeb4154840 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:34306/0 (socket says 172.21.15.143:34306) 2023-11-26T07:22:20.176 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:20.172+0000 7faeb9993640 1 -- 172.21.15.143:0/1298818076 learned_addr learned my addr 172.21.15.143:0/1298818076 (peer_addr_for_me v2:172.21.15.143:0/0) 2023-11-26T07:22:20.176 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:20.172+0000 7faeba194640 1 --2- 172.21.15.143:0/1298818076 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7faeb40b2f80 0x7faeb415c6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:22:20.176 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:20.172+0000 7faeba995640 1 --2- 172.21.15.143:0/1298818076 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7faeb41596d0 0x7faeb4154d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:22:20.176 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:20.172+0000 7faeb9993640 1 -- 172.21.15.143:0/1298818076 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7faeb40b2f80 msgr2=0x7faeb415c6e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:22:20.176 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:20.172+0000 7faeb9993640 1 --2- 172.21.15.143:0/1298818076 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7faeb40b2f80 0x7faeb415c6e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:22:20.176 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:20.172+0000 7faeb9993640 1 -- 172.21.15.143:0/1298818076 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7faeb41596d0 msgr2=0x7faeb4154d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:22:20.176 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:20.172+0000 7faeb9993640 1 --2- 172.21.15.143:0/1298818076 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7faeb41596d0 0x7faeb4154d80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:22:20.177 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:20.172+0000 7faeb9993640 1 -- 172.21.15.143:0/1298818076 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7faea4009420 con 0x7faeb40ce680 2023-11-26T07:22:20.177 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:20.172+0000 7faeba995640 1 --2- 172.21.15.143:0/1298818076 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7faeb41596d0 0x7faeb4154d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-26T07:22:20.177 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:20.176+0000 7faeb9993640 1 --2- 172.21.15.143:0/1298818076 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7faeb40ce680 0x7faeb4154840 secure :-1 s=READY pgs=871 cs=0 l=1 rev1=1 crypto rx=0x7faea40ae040 tx=0x7faea4019190 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:22:20.178 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:20.176+0000 7faeaaffd640 1 -- 172.21.15.143:0/1298818076 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faea40b4430 con 0x7faeb40ce680 2023-11-26T07:22:20.178 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:20.176+0000 7faeaaffd640 1 -- 172.21.15.143:0/1298818076 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7faea40b45b0 con 0x7faeb40ce680 2023-11-26T07:22:20.178 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:20.176+0000 7faeaaffd640 1 -- 172.21.15.143:0/1298818076 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faea40b4960 con 0x7faeb40ce680 2023-11-26T07:22:20.178 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:20.176+0000 7faebc3f8640 1 -- 172.21.15.143:0/1298818076 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7faeb4159ee0 con 0x7faeb40ce680 2023-11-26T07:22:20.178 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:20.176+0000 7faebc3f8640 1 -- 172.21.15.143:0/1298818076 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7faeb415a4a0 con 0x7faeb40ce680 2023-11-26T07:22:20.178 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:20.176+0000 7faebc3f8640 1 -- 172.21.15.143:0/1298818076 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7faeb4159d40 con 0x7faeb40ce680 2023-11-26T07:22:20.178 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:20.176+0000 7faeaaffd640 1 -- 172.21.15.143:0/1298818076 <== mon.0 v2:172.21.15.143:3300/0 4 ==== mgrmap(e 11) v1 ==== 86158+0+0 (secure 0 0 0) 0x7faea40b2040 con 0x7faeb40ce680 2023-11-26T07:22:20.179 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:20.176+0000 7faeaaffd640 1 --2- 172.21.15.143:0/1298818076 >> [v2:172.21.15.143:6832/2150557544,v1:172.21.15.143:6833/2150557544] conn(0x7fae980667d0 0x7fae98068c90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:22:20.179 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:20.176+0000 7faeaaffd640 1 -- 172.21.15.143:0/1298818076 <== mon.0 v2:172.21.15.143:3300/0 5 ==== osd_map(59..59 src has 1..59) v4 ==== 5709+0+0 (secure 0 0 0) 0x7faea411e2d0 con 0x7faeb40ce680 2023-11-26T07:22:20.179 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:20.176+0000 7faeba194640 1 --2- 172.21.15.143:0/1298818076 >> [v2:172.21.15.143:6832/2150557544,v1:172.21.15.143:6833/2150557544] conn(0x7fae980667d0 0x7fae98068c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:22:20.180 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:20.180+0000 7faeba194640 1 --2- 172.21.15.143:0/1298818076 >> [v2:172.21.15.143:6832/2150557544,v1:172.21.15.143:6833/2150557544] conn(0x7fae980667d0 0x7fae98068c90 secure :-1 s=READY pgs=409 cs=0 l=1 rev1=1 crypto rx=0x7faeb00011f0 tx=0x7faeb0007d40 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:22:20.183 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:20.180+0000 7faeaaffd640 1 -- 172.21.15.143:0/1298818076 <== mon.0 v2:172.21.15.143:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172649 (secure 0 0 0) 0x7faeb4159d40 con 0x7faeb40ce680 2023-11-26T07:22:20.338 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:20.336+0000 7faebc3f8640 1 -- 172.21.15.143:0/1298818076 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7faeb4159d40 con 0x7faeb40ce680 2023-11-26T07:22:20.338 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:20.336+0000 7faeaaffd640 1 -- 172.21.15.143:0/1298818076 <== mon.0 v2:172.21.15.143:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v59) v1 ==== 100+0+11 (secure 0 0 0) 0x7faeb4159d40 con 0x7faeb40ce680 2023-11-26T07:22:20.338 INFO:teuthology.orchestra.run.smithi143.stdout:pg_num: 64 2023-11-26T07:22:20.340 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:20.340+0000 7faebc3f8640 1 -- 172.21.15.143:0/1298818076 >> [v2:172.21.15.143:6832/2150557544,v1:172.21.15.143:6833/2150557544] conn(0x7fae980667d0 msgr2=0x7fae98068c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:22:20.340 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:20.340+0000 7faebc3f8640 1 --2- 172.21.15.143:0/1298818076 >> [v2:172.21.15.143:6832/2150557544,v1:172.21.15.143:6833/2150557544] conn(0x7fae980667d0 0x7fae98068c90 secure :-1 s=READY pgs=409 cs=0 l=1 rev1=1 crypto rx=0x7faeb00011f0 tx=0x7faeb0007d40 comp rx=0 tx=0).stop 2023-11-26T07:22:20.341 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:20.340+0000 7faebc3f8640 1 -- 172.21.15.143:0/1298818076 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7faeb40ce680 msgr2=0x7faeb4154840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:22:20.341 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:20.340+0000 7faebc3f8640 1 --2- 172.21.15.143:0/1298818076 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7faeb40ce680 0x7faeb4154840 secure :-1 s=READY pgs=871 cs=0 l=1 rev1=1 crypto rx=0x7faea40ae040 tx=0x7faea4019190 comp rx=0 tx=0).stop 2023-11-26T07:22:20.341 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:20.340+0000 7faebc3f8640 1 -- 172.21.15.143:0/1298818076 shutdown_connections 2023-11-26T07:22:20.341 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:20.340+0000 7faebc3f8640 1 --2- 172.21.15.143:0/1298818076 >> [v2:172.21.15.143:6832/2150557544,v1:172.21.15.143:6833/2150557544] conn(0x7fae980667d0 0x7fae98068c90 unknown :-1 s=CLOSED pgs=409 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:22:20.341 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:20.340+0000 7faebc3f8640 1 --2- 172.21.15.143:0/1298818076 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7faeb41596d0 0x7faeb4154d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:22:20.341 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:20.340+0000 7faebc3f8640 1 --2- 172.21.15.143:0/1298818076 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7faeb40ce680 0x7faeb4154840 unknown :-1 s=CLOSED pgs=871 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:22:20.341 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:20.340+0000 7faebc3f8640 1 --2- 172.21.15.143:0/1298818076 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7faeb40b2f80 0x7faeb415c6e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:22:20.341 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:20.340+0000 7faebc3f8640 1 -- 172.21.15.143:0/1298818076 >> 172.21.15.143:0/1298818076 conn(0x7faeb406a890 msgr2=0x7faeb4150ac0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:22:20.341 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:20.340+0000 7faebc3f8640 1 -- 172.21.15.143:0/1298818076 shutdown_connections 2023-11-26T07:22:20.341 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:20.340+0000 7faebc3f8640 1 -- 172.21.15.143:0/1298818076 wait complete. 2023-11-26T07:22:20.360 DEBUG:teuthology.orchestra.run.smithi143:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2023-11-26T07:22:20.500 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:20.496+0000 7f5f2b59e640 1 Processor -- start 2023-11-26T07:22:20.500 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:20.500+0000 7f5f2b59e640 1 -- start start 2023-11-26T07:22:20.501 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:20.500+0000 7f5f2b59e640 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7f5f2c0cfa70 0x7f5f2c0d0e40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:22:20.501 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:20.500+0000 7f5f2b59e640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f5f2c0ced30 0x7f5f2c0b0f60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:22:20.501 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:20.500+0000 7f5f2b59e640 1 -- --> v1:172.21.15.143:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5f2c0d3d20 con 0x7f5f2c0cfe50 2023-11-26T07:22:20.501 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:20.500+0000 7f5f2b59e640 1 -- --> v2:172.21.15.188:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5f2c0d4000 con 0x7f5f2c0cfa70 2023-11-26T07:22:20.501 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:20.500+0000 7f5f2b59e640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5f2c0afd70 con 0x7f5f2c0ced30 2023-11-26T07:22:20.502 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:20.500+0000 7f5f29d9b640 1 --1- >> v1:172.21.15.143:6789/0 conn(0x7f5f2c0cfe50 0x7f5f2c0b0850 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.143:6789/0 says I am v1:172.21.15.143:59214/0 (socket says 172.21.15.143:59214) 2023-11-26T07:22:20.502 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:20.500+0000 7f5f29d9b640 1 -- 172.21.15.143:0/4115288497 learned_addr learned my addr 172.21.15.143:0/4115288497 (peer_addr_for_me v1:172.21.15.143:0/0) 2023-11-26T07:22:20.502 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:20.500+0000 7f5f2ad9d640 1 --2- 172.21.15.143:0/4115288497 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f5f2c0ced30 0x7f5f2c0b0f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:22:20.503 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:20.500+0000 7f5f2a59c640 1 --2- 172.21.15.143:0/4115288497 >> v2:172.21.15.188:3300/0 conn(0x7f5f2c0cfa70 0x7f5f2c0d0e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:22:20.503 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:20.500+0000 7f5f2959a640 1 -- 172.21.15.143:0/4115288497 <== mon.0 v1:172.21.15.143:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 749646534 0 0) 0x7f5f2c0d3d20 con 0x7f5f2c0cfe50 2023-11-26T07:22:20.503 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:20.500+0000 7f5f2959a640 1 -- 172.21.15.143:0/4115288497 --> v1:172.21.15.143:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f5f14003530 con 0x7f5f2c0cfe50 2023-11-26T07:22:20.503 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:20.500+0000 7f5f2ad9d640 1 -- 172.21.15.143:0/4115288497 >> v2:172.21.15.188:3300/0 conn(0x7f5f2c0cfa70 msgr2=0x7f5f2c0d0e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-26T07:22:20.503 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:20.500+0000 7f5f2ad9d640 1 --2- 172.21.15.143:0/4115288497 >> v2:172.21.15.188:3300/0 conn(0x7f5f2c0cfa70 0x7f5f2c0d0e40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:22:20.503 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:20.500+0000 7f5f2ad9d640 1 -- 172.21.15.143:0/4115288497 >> v1:172.21.15.143:6789/0 conn(0x7f5f2c0cfe50 legacy=0x7f5f2c0b0850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:22:20.503 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:20.500+0000 7f5f2ad9d640 1 -- 172.21.15.143:0/4115288497 --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5f2c0b0050 con 0x7f5f2c0ced30 2023-11-26T07:22:20.505 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:20.504+0000 7f5f2ad9d640 1 --2- 172.21.15.143:0/4115288497 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f5f2c0ced30 0x7f5f2c0b0f60 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7f5f24009d90 tx=0x7f5f2401c920 comp rx=0 tx=0).ready entity=mon.2 client_cookie=63959615d50c111b server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:22:20.505 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:20.504+0000 7f5f2959a640 1 -- 172.21.15.143:0/4115288497 <== mon.2 v2:172.21.15.188:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5f240a5a00 con 0x7f5f2c0ced30 2023-11-26T07:22:20.505 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:20.504+0000 7f5f2959a640 1 -- 172.21.15.143:0/4115288497 <== mon.2 v2:172.21.15.188:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5f240a5b60 con 0x7f5f2c0ced30 2023-11-26T07:22:20.506 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:20.504+0000 7f5f2959a640 1 -- 172.21.15.143:0/4115288497 <== mon.2 v2:172.21.15.188:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5f240ad440 con 0x7f5f2c0ced30 2023-11-26T07:22:20.506 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:20.504+0000 7f5f2b59e640 1 -- 172.21.15.143:0/4115288497 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f5f2c0ced30 msgr2=0x7f5f2c0b0f60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:22:20.506 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:20.504+0000 7f5f2b59e640 1 --2- 172.21.15.143:0/4115288497 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f5f2c0ced30 0x7f5f2c0b0f60 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7f5f24009d90 tx=0x7f5f2401c920 comp rx=0 tx=0).stop 2023-11-26T07:22:20.507 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:20.504+0000 7f5f2b59e640 1 -- 172.21.15.143:0/4115288497 shutdown_connections 2023-11-26T07:22:20.507 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:20.504+0000 7f5f2b59e640 1 --2- 172.21.15.143:0/4115288497 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f5f2c0ced30 0x7f5f2c0b0f60 unknown :-1 s=CLOSED pgs=110 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:22:20.507 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:20.504+0000 7f5f2b59e640 1 --2- 172.21.15.143:0/4115288497 >> v2:172.21.15.188:3300/0 conn(0x7f5f2c0cfa70 0x7f5f2c0d0e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:22:20.507 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:20.504+0000 7f5f2b59e640 1 -- 172.21.15.143:0/4115288497 >> 172.21.15.143:0/4115288497 conn(0x7f5f2c0d3060 msgr2=0x7f5f2c0d3470 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:22:20.507 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:20.504+0000 7f5f2b59e640 1 -- 172.21.15.143:0/4115288497 shutdown_connections 2023-11-26T07:22:20.507 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:20.504+0000 7f5f2b59e640 1 -- 172.21.15.143:0/4115288497 wait complete. 2023-11-26T07:22:20.508 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:20.508+0000 7f5f2b59e640 1 Processor -- start 2023-11-26T07:22:20.508 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:20.508+0000 7f5f2b59e640 1 -- start start 2023-11-26T07:22:20.510 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:20.508+0000 7f5f2b59e640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f5f2c0ced30 0x7f5f2c14a3f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:22:20.510 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:20.508+0000 7f5f2b59e640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f5f2c0cfa70 0x7f5f2c15b080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:22:20.510 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:20.508+0000 7f5f2b59e640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f5f2c0cfe50 0x7f5f2c15b5c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:22:20.510 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:20.508+0000 7f5f2b59e640 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5f2c0ccd90 con 0x7f5f2c0ced30 2023-11-26T07:22:20.510 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:20.508+0000 7f5f2b59e640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5f2c15c860 con 0x7f5f2c0cfa70 2023-11-26T07:22:20.510 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:20.508+0000 7f5f2b59e640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5f2c14a930 con 0x7f5f2c0cfe50 2023-11-26T07:22:20.510 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:20.508+0000 7f5f2a59c640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f5f2c0ced30 0x7f5f2c14a3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:22:20.510 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:20.508+0000 7f5f2a59c640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f5f2c0ced30 0x7f5f2c14a3f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:34320/0 (socket says 172.21.15.143:34320) 2023-11-26T07:22:20.510 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:20.508+0000 7f5f2a59c640 1 -- 172.21.15.143:0/788382475 learned_addr learned my addr 172.21.15.143:0/788382475 (peer_addr_for_me v2:172.21.15.143:0/0) 2023-11-26T07:22:20.510 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:20.508+0000 7f5f2ad9d640 1 --2- 172.21.15.143:0/788382475 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f5f2c0cfe50 0x7f5f2c15b5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:22:20.511 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:20.508+0000 7f5f29d9b640 1 --2- 172.21.15.143:0/788382475 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f5f2c0cfa70 0x7f5f2c15b080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:22:20.511 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:20.508+0000 7f5f2a59c640 1 -- 172.21.15.143:0/788382475 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f5f2c0cfe50 msgr2=0x7f5f2c15b5c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:22:20.511 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:20.508+0000 7f5f2a59c640 1 --2- 172.21.15.143:0/788382475 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f5f2c0cfe50 0x7f5f2c15b5c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:22:20.511 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:20.508+0000 7f5f2a59c640 1 -- 172.21.15.143:0/788382475 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f5f2c0cfa70 msgr2=0x7f5f2c15b080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:22:20.511 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:20.508+0000 7f5f2a59c640 1 --2- 172.21.15.143:0/788382475 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f5f2c0cfa70 0x7f5f2c15b080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:22:20.511 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:20.508+0000 7f5f2a59c640 1 -- 172.21.15.143:0/788382475 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5f24009a70 con 0x7f5f2c0ced30 2023-11-26T07:22:20.511 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:20.508+0000 7f5f2a59c640 1 --2- 172.21.15.143:0/788382475 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f5f2c0ced30 0x7f5f2c14a3f0 secure :-1 s=READY pgs=873 cs=0 l=1 rev1=1 crypto rx=0x7f5f1c00e960 tx=0x7f5f1c00ee20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:22:20.512 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:20.508+0000 7f5f1affd640 1 -- 172.21.15.143:0/788382475 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5f1c009690 con 0x7f5f2c0ced30 2023-11-26T07:22:20.512 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:20.508+0000 7f5f1affd640 1 -- 172.21.15.143:0/788382475 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5f1c00ca00 con 0x7f5f2c0ced30 2023-11-26T07:22:20.512 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:20.508+0000 7f5f1affd640 1 -- 172.21.15.143:0/788382475 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5f1c00cd70 con 0x7f5f2c0ced30 2023-11-26T07:22:20.512 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:20.508+0000 7f5f2b59e640 1 -- 172.21.15.143:0/788382475 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5f2c15cab0 con 0x7f5f2c0ced30 2023-11-26T07:22:20.512 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:20.508+0000 7f5f2b59e640 1 -- 172.21.15.143:0/788382475 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5f2c0cc9b0 con 0x7f5f2c0ced30 2023-11-26T07:22:20.512 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:20.508+0000 7f5f2b59e640 1 -- 172.21.15.143:0/788382475 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5f2c0b8cb0 con 0x7f5f2c0ced30 2023-11-26T07:22:20.513 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:20.512+0000 7f5f1affd640 1 -- 172.21.15.143:0/788382475 <== mon.0 v2:172.21.15.143:3300/0 4 ==== mgrmap(e 11) v1 ==== 86158+0+0 (secure 0 0 0) 0x7f5f1c013470 con 0x7f5f2c0ced30 2023-11-26T07:22:20.513 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:20.512+0000 7f5f1affd640 1 --2- 172.21.15.143:0/788382475 >> [v2:172.21.15.143:6832/2150557544,v1:172.21.15.143:6833/2150557544] conn(0x7f5f0c066800 0x7f5f0c068cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:22:20.513 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:20.512+0000 7f5f1affd640 1 -- 172.21.15.143:0/788382475 <== mon.0 v2:172.21.15.143:3300/0 5 ==== osd_map(59..59 src has 1..59) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f5f1c084480 con 0x7f5f2c0ced30 2023-11-26T07:22:20.514 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:20.512+0000 7f5f29d9b640 1 --2- 172.21.15.143:0/788382475 >> [v2:172.21.15.143:6832/2150557544,v1:172.21.15.143:6833/2150557544] conn(0x7f5f0c066800 0x7f5f0c068cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:22:20.514 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:20.512+0000 7f5f29d9b640 1 --2- 172.21.15.143:0/788382475 >> [v2:172.21.15.143:6832/2150557544,v1:172.21.15.143:6833/2150557544] conn(0x7f5f0c066800 0x7f5f0c068cc0 secure :-1 s=READY pgs=410 cs=0 l=1 rev1=1 crypto rx=0x7f5f2c160c60 tx=0x7f5f20009040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:22:20.517 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:20.516+0000 7f5f1affd640 1 -- 172.21.15.143:0/788382475 <== mon.0 v2:172.21.15.143:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172649 (secure 0 0 0) 0x7f5f1c051630 con 0x7f5f2c0ced30 2023-11-26T07:22:20.681 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:20.680+0000 7f5f2b59e640 1 -- 172.21.15.143:0/788382475 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) v1 -- 0x7f5f2c067500 con 0x7f5f2c0ced30 2023-11-26T07:22:20.681 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:20.680+0000 7f5f1affd640 1 -- 172.21.15.143:0/788382475 <== mon.0 v2:172.21.15.143:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v59) v1 ==== 80+0+0 (secure 0 0 0) 0x7f5f1c054c50 con 0x7f5f2c0ced30 2023-11-26T07:22:20.681 INFO:teuthology.orchestra.run.smithi143.stderr:listed 0 entries 2023-11-26T07:22:20.683 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:20.680+0000 7f5f2b59e640 1 -- 172.21.15.143:0/788382475 >> [v2:172.21.15.143:6832/2150557544,v1:172.21.15.143:6833/2150557544] conn(0x7f5f0c066800 msgr2=0x7f5f0c068cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:22:20.683 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:20.680+0000 7f5f2b59e640 1 --2- 172.21.15.143:0/788382475 >> [v2:172.21.15.143:6832/2150557544,v1:172.21.15.143:6833/2150557544] conn(0x7f5f0c066800 0x7f5f0c068cc0 secure :-1 s=READY pgs=410 cs=0 l=1 rev1=1 crypto rx=0x7f5f2c160c60 tx=0x7f5f20009040 comp rx=0 tx=0).stop 2023-11-26T07:22:20.683 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:20.680+0000 7f5f2b59e640 1 -- 172.21.15.143:0/788382475 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f5f2c0ced30 msgr2=0x7f5f2c14a3f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:22:20.683 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:20.680+0000 7f5f2b59e640 1 --2- 172.21.15.143:0/788382475 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f5f2c0ced30 0x7f5f2c14a3f0 secure :-1 s=READY pgs=873 cs=0 l=1 rev1=1 crypto rx=0x7f5f1c00e960 tx=0x7f5f1c00ee20 comp rx=0 tx=0).stop 2023-11-26T07:22:20.684 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:20.680+0000 7f5f2b59e640 1 -- 172.21.15.143:0/788382475 shutdown_connections 2023-11-26T07:22:20.684 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:20.680+0000 7f5f2b59e640 1 --2- 172.21.15.143:0/788382475 >> [v2:172.21.15.143:6832/2150557544,v1:172.21.15.143:6833/2150557544] conn(0x7f5f0c066800 0x7f5f0c068cc0 unknown :-1 s=CLOSED pgs=410 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:22:20.684 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:20.680+0000 7f5f2b59e640 1 --2- 172.21.15.143:0/788382475 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f5f2c0cfe50 0x7f5f2c15b5c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:22:20.684 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:20.680+0000 7f5f2b59e640 1 --2- 172.21.15.143:0/788382475 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f5f2c0cfa70 0x7f5f2c15b080 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:22:20.684 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:20.680+0000 7f5f2b59e640 1 --2- 172.21.15.143:0/788382475 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f5f2c0ced30 0x7f5f2c14a3f0 unknown :-1 s=CLOSED pgs=873 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:22:20.684 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:20.680+0000 7f5f2b59e640 1 -- 172.21.15.143:0/788382475 >> 172.21.15.143:0/788382475 conn(0x7f5f2c0d3060 msgr2=0x7f5f2c0b38e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:22:20.684 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:20.680+0000 7f5f2b59e640 1 -- 172.21.15.143:0/788382475 shutdown_connections 2023-11-26T07:22:20.684 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:20.680+0000 7f5f2b59e640 1 -- 172.21.15.143:0/788382475 wait complete. 2023-11-26T07:22:20.702 INFO:tasks.cephfs.fuse_mount:Running fusermount -u on ubuntu@smithi188.front.sepia.ceph.com... 2023-11-26T07:22:20.702 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-26T07:22:20.702 DEBUG:teuthology.orchestra.run.smithi188:> sudo fusermount -u /home/ubuntu/cephtest/mnt.1 2023-11-26T07:22:20.722 INFO:tasks.cephfs.fuse_mount.ceph-fuse.1.smithi188.stderr:ceph-fuse[44712]: fuse finished with error 0 and tester_r 0 2023-11-26T07:22:20.725 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi188.front.sepia.ceph.com 2023-11-26T07:22:20.725 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-26T07:22:20.725 DEBUG:teuthology.orchestra.run.smithi188:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.1) 2023-11-26T07:22:20.894 DEBUG:teuthology.orchestra.run.smithi143:> set -ex 2023-11-26T07:22:20.894 DEBUG:teuthology.orchestra.run.smithi143:> dd if=/proc/self/mounts of=/dev/stdout 2023-11-26T07:22:20.904 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,relatime,size=16354672k,nr_inodes=4088668,mode=755 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3275384k,mode=755 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=3455 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysusers.service ramfs ro,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3275380k,nr_inodes=818845,mode=700,uid=1000,gid=1271 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3275384k,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2023-11-26T07:22:20.904 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi143.front.sepia.ceph.com 2023-11-26T07:22:20.904 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-26T07:22:20.904 DEBUG:teuthology.orchestra.run.smithi143:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.2) 2023-11-26T07:22:20.955 INFO:teuthology.orchestra.run.smithi143.stderr:rmdir: failed to remove '/home/ubuntu/cephtest/mnt.2': No such file or directory 2023-11-26T07:22:20.956 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-11-26T07:22:20.957 DEBUG:teuthology.orchestra.run.smithi188:> set -ex 2023-11-26T07:22:20.957 DEBUG:teuthology.orchestra.run.smithi188:> dd if=/proc/self/mounts of=/dev/stdout 2023-11-26T07:22:20.966 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,relatime,size=16354628k,nr_inodes=4088657,mode=755 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3275376k,mode=755 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=787 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysusers.service ramfs ro,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3275372k,nr_inodes=818843,mode=700,uid=1000,gid=1271 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3275376k,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2023-11-26T07:22:20.966 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi188.front.sepia.ceph.com 2023-11-26T07:22:20.966 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-26T07:22:20.966 DEBUG:teuthology.orchestra.run.smithi188:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.3) 2023-11-26T07:22:21.020 INFO:teuthology.orchestra.run.smithi188.stderr:rmdir: failed to remove '/home/ubuntu/cephtest/mnt.3': No such file or directory 2023-11-26T07:22:21.021 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-11-26T07:22:21.021 DEBUG:teuthology.orchestra.run.smithi143:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-11-26T07:22:21.159 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:21.156+0000 7f7db25e7640 1 Processor -- start 2023-11-26T07:22:21.159 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:21.156+0000 7f7db25e7640 1 -- start start 2023-11-26T07:22:21.161 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:21.160+0000 7f7db25e7640 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7f7dac070ba0 0x7f7dac06bae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:22:21.161 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:21.160+0000 7f7db25e7640 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7f7dac161cf0 0x7f7dac070f80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:22:21.161 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:21.160+0000 7f7db25e7640 1 -- --> v1:172.21.15.143:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7dac0714c0 con 0x7f7dac0cdd10 2023-11-26T07:22:21.161 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:21.160+0000 7f7db25e7640 1 -- --> v2:172.21.15.143:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7dac0cd610 con 0x7f7dac161cf0 2023-11-26T07:22:21.162 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:21.160+0000 7f7db25e7640 1 -- --> v2:172.21.15.188:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7dac0cd8f0 con 0x7f7dac070ba0 2023-11-26T07:22:21.162 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:21.160+0000 7f7dab7fe640 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7f7dac161cf0 0x7f7dac070f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:22:21.162 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:21.160+0000 7f7db0b84640 1 --1- >> v1:172.21.15.143:6789/0 conn(0x7f7dac0cdd10 0x7f7dac06a490 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.143:6789/0 says I am v1:172.21.15.143:59226/0 (socket says 172.21.15.143:59226) 2023-11-26T07:22:21.162 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:21.160+0000 7f7db0b84640 1 -- 172.21.15.143:0/311339092 learned_addr learned my addr 172.21.15.143:0/311339092 (peer_addr_for_me v1:172.21.15.143:0/0) 2023-11-26T07:22:21.162 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:21.160+0000 7f7dabfff640 1 --2- 172.21.15.143:0/311339092 >> v2:172.21.15.188:3300/0 conn(0x7f7dac070ba0 0x7f7dac06bae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:22:21.162 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:21.160+0000 7f7dab7fe640 1 -- 172.21.15.143:0/311339092 >> v2:172.21.15.188:3300/0 conn(0x7f7dac070ba0 msgr2=0x7f7dac06bae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-26T07:22:21.162 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:21.160+0000 7f7dab7fe640 1 --2- 172.21.15.143:0/311339092 >> v2:172.21.15.188:3300/0 conn(0x7f7dac070ba0 0x7f7dac06bae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:22:21.162 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:21.160+0000 7f7dab7fe640 1 -- 172.21.15.143:0/311339092 >> v1:172.21.15.143:6789/0 conn(0x7f7dac0cdd10 legacy=0x7f7dac06a490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:22:21.162 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:21.160+0000 7f7dab7fe640 1 -- 172.21.15.143:0/311339092 --> v2:172.21.15.143:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7dac161210 con 0x7f7dac161cf0 2023-11-26T07:22:21.162 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:21.160+0000 7f7daaffd640 1 -- 172.21.15.143:0/311339092 <== mon.0 v1:172.21.15.143:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3468666274 0 0) 0x7f7dac0714c0 con 0x7f7dac0cdd10 2023-11-26T07:22:21.164 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:21.164+0000 7f7dab7fe640 1 --2- 172.21.15.143:0/311339092 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f7dac161cf0 0x7f7dac070f80 secure :-1 s=READY pgs=875 cs=0 l=1 rev1=1 crypto rx=0x7f7dac073fd0 tx=0x7f7da001c460 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3ab39c32e0dc0c8 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:22:21.164 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:21.164+0000 7f7daaffd640 1 -- 172.21.15.143:0/311339092 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7da00ab600 con 0x7f7dac161cf0 2023-11-26T07:22:21.164 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:21.164+0000 7f7daaffd640 1 -- 172.21.15.143:0/311339092 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7da00a7050 con 0x7f7dac161cf0 2023-11-26T07:22:21.164 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:21.164+0000 7f7daaffd640 1 -- 172.21.15.143:0/311339092 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7da00ab980 con 0x7f7dac161cf0 2023-11-26T07:22:21.165 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:21.164+0000 7f7db25e7640 1 -- 172.21.15.143:0/311339092 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f7dac161cf0 msgr2=0x7f7dac070f80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:22:21.165 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:21.164+0000 7f7db25e7640 1 --2- 172.21.15.143:0/311339092 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f7dac161cf0 0x7f7dac070f80 secure :-1 s=READY pgs=875 cs=0 l=1 rev1=1 crypto rx=0x7f7dac073fd0 tx=0x7f7da001c460 comp rx=0 tx=0).stop 2023-11-26T07:22:21.165 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:21.164+0000 7f7db25e7640 1 -- 172.21.15.143:0/311339092 shutdown_connections 2023-11-26T07:22:21.166 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:21.164+0000 7f7db25e7640 1 --2- 172.21.15.143:0/311339092 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f7dac161cf0 0x7f7dac070f80 unknown :-1 s=CLOSED pgs=875 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:22:21.166 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:21.164+0000 7f7db25e7640 1 --2- 172.21.15.143:0/311339092 >> v2:172.21.15.188:3300/0 conn(0x7f7dac070ba0 0x7f7dac06bae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:22:21.166 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:21.164+0000 7f7db25e7640 1 -- 172.21.15.143:0/311339092 >> 172.21.15.143:0/311339092 conn(0x7f7dac0cf710 msgr2=0x7f7dac0cfb20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:22:21.166 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:21.164+0000 7f7db25e7640 1 -- 172.21.15.143:0/311339092 shutdown_connections 2023-11-26T07:22:21.166 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:21.164+0000 7f7db25e7640 1 -- 172.21.15.143:0/311339092 wait complete. 2023-11-26T07:22:21.166 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:21.164+0000 7f7db25e7640 1 Processor -- start 2023-11-26T07:22:21.167 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:21.164+0000 7f7db25e7640 1 -- start start 2023-11-26T07:22:21.169 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:21.164+0000 7f7db25e7640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f7dac070ba0 0x7f7dac15db40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:22:21.169 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:21.164+0000 7f7db25e7640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f7dac0cdd10 0x7f7dac1604e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:22:21.169 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:21.168+0000 7f7db25e7640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f7dac161cf0 0x7f7dac14dd70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:22:21.169 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:21.168+0000 7f7db25e7640 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7dac14d5d0 con 0x7f7dac0cdd10 2023-11-26T07:22:21.170 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:21.168+0000 7f7db25e7640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7dac15e080 con 0x7f7dac161cf0 2023-11-26T07:22:21.170 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:21.168+0000 7f7db25e7640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7dac14f080 con 0x7f7dac070ba0 2023-11-26T07:22:21.170 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:21.168+0000 7f7dab7fe640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f7dac0cdd10 0x7f7dac1604e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:22:21.170 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:21.168+0000 7f7dab7fe640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f7dac0cdd10 0x7f7dac1604e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:34326/0 (socket says 172.21.15.143:34326) 2023-11-26T07:22:21.170 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:21.168+0000 7f7dab7fe640 1 -- 172.21.15.143:0/835236203 learned_addr learned my addr 172.21.15.143:0/835236203 (peer_addr_for_me v2:172.21.15.143:0/0) 2023-11-26T07:22:21.170 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:21.168+0000 7f7db0b84640 1 --2- 172.21.15.143:0/835236203 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f7dac161cf0 0x7f7dac14dd70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:22:21.170 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:21.168+0000 7f7dabfff640 1 --2- 172.21.15.143:0/835236203 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f7dac070ba0 0x7f7dac15db40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:22:21.170 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:21.168+0000 7f7dab7fe640 1 -- 172.21.15.143:0/835236203 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f7dac070ba0 msgr2=0x7f7dac15db40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:22:21.170 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:21.168+0000 7f7dab7fe640 1 --2- 172.21.15.143:0/835236203 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f7dac070ba0 0x7f7dac15db40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:22:21.170 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:21.168+0000 7f7dab7fe640 1 -- 172.21.15.143:0/835236203 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f7dac161cf0 msgr2=0x7f7dac14dd70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:22:21.170 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:21.168+0000 7f7dab7fe640 1 --2- 172.21.15.143:0/835236203 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f7dac161cf0 0x7f7dac14dd70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:22:21.170 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:21.168+0000 7f7dab7fe640 1 -- 172.21.15.143:0/835236203 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7da0009420 con 0x7f7dac0cdd10 2023-11-26T07:22:21.171 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:21.168+0000 7f7dabfff640 1 --2- 172.21.15.143:0/835236203 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f7dac070ba0 0x7f7dac15db40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-26T07:22:21.171 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:21.168+0000 7f7dab7fe640 1 --2- 172.21.15.143:0/835236203 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f7dac0cdd10 0x7f7dac1604e0 secure :-1 s=READY pgs=876 cs=0 l=1 rev1=1 crypto rx=0x7f7da0019070 tx=0x7f7da0019200 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:22:21.171 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:21.168+0000 7f7da8ff9640 1 -- 172.21.15.143:0/835236203 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7da00b5670 con 0x7f7dac0cdd10 2023-11-26T07:22:21.171 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:21.168+0000 7f7da8ff9640 1 -- 172.21.15.143:0/835236203 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7da00b57d0 con 0x7f7dac0cdd10 2023-11-26T07:22:21.171 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:21.168+0000 7f7db25e7640 1 -- 172.21.15.143:0/835236203 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7dac160a20 con 0x7f7dac0cdd10 2023-11-26T07:22:21.171 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:21.168+0000 7f7da8ff9640 1 -- 172.21.15.143:0/835236203 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7da00b5b40 con 0x7f7dac0cdd10 2023-11-26T07:22:21.171 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:21.168+0000 7f7db25e7640 1 -- 172.21.15.143:0/835236203 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7dac14f600 con 0x7f7dac0cdd10 2023-11-26T07:22:21.172 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:21.168+0000 7f7db25e7640 1 -- 172.21.15.143:0/835236203 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7dac067680 con 0x7f7dac0cdd10 2023-11-26T07:22:21.172 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:21.168+0000 7f7da8ff9640 1 -- 172.21.15.143:0/835236203 <== mon.0 v2:172.21.15.143:3300/0 4 ==== mgrmap(e 11) v1 ==== 86158+0+0 (secure 0 0 0) 0x7f7da00b2040 con 0x7f7dac0cdd10 2023-11-26T07:22:21.172 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:21.172+0000 7f7da8ff9640 1 --2- 172.21.15.143:0/835236203 >> [v2:172.21.15.143:6832/2150557544,v1:172.21.15.143:6833/2150557544] conn(0x7f7d980667d0 0x7f7d98068c90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:22:21.172 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:21.172+0000 7f7da8ff9640 1 -- 172.21.15.143:0/835236203 <== mon.0 v2:172.21.15.143:3300/0 5 ==== osd_map(59..59 src has 1..59) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f7da011e840 con 0x7f7dac0cdd10 2023-11-26T07:22:21.172 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:21.172+0000 7f7dabfff640 1 --2- 172.21.15.143:0/835236203 >> [v2:172.21.15.143:6832/2150557544,v1:172.21.15.143:6833/2150557544] conn(0x7f7d980667d0 0x7f7d98068c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:22:21.173 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:21.172+0000 7f7dabfff640 1 --2- 172.21.15.143:0/835236203 >> [v2:172.21.15.143:6832/2150557544,v1:172.21.15.143:6833/2150557544] conn(0x7f7d980667d0 0x7f7d98068c90 secure :-1 s=READY pgs=411 cs=0 l=1 rev1=1 crypto rx=0x7f7dac0acc00 tx=0x7f7d9c008040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:22:21.176 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:21.176+0000 7f7da8ff9640 1 -- 172.21.15.143:0/835236203 <== mon.0 v2:172.21.15.143:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172649 (secure 0 0 0) 0x7f7da00eb830 con 0x7f7dac0cdd10 2023-11-26T07:22:21.357 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:21.356+0000 7f7db25e7640 1 -- 172.21.15.143:0/835236203 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f7dac071c90 con 0x7f7dac0cdd10 2023-11-26T07:22:21.357 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:21.356+0000 7f7da8ff9640 1 -- 172.21.15.143:0/835236203 <== mon.0 v2:172.21.15.143:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 35 v35) v1 ==== 94+0+4925 (secure 0 0 0) 0x7f7da00b0090 con 0x7f7dac0cdd10 2023-11-26T07:22:21.358 INFO:teuthology.orchestra.run.smithi143.stdout: 2023-11-26T07:22:21.358 INFO:teuthology.orchestra.run.smithi143.stdout:{"epoch":35,"default_fscid":4,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5612,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.143:6837/1372238588","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6836","nonce":1372238588},{"type":"v1","addr":"172.21.15.143:6837","nonce":1372238588}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":29},{"gid":6103,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.143:6835/1078738477","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6834","nonce":1078738477},{"type":"v1","addr":"172.21.15.143:6835","nonce":1078738477}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":29},{"gid":6118,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.188:6834/1634083777","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6832","nonce":1634083777},{"type":"v1","addr":"172.21.15.188:6834","nonce":1634083777}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":29}],"filesystems":[{"mdsmap":{"epoch":34,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-11-26T07:21:55.656168+0000","modified":"2023-11-26T07:21:58.674916+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":6121},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6121":{"gid":6121,"name":"b","rank":0,"incarnation":30,"state":"up:active","state_seq":3,"addr":"172.21.15.188:6835/1958716738","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6833","nonce":1958716738},{"type":"v1","addr":"172.21.15.188:6835","nonce":1958716738}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[9],"metadata_pool":8,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":4}]} 2023-11-26T07:22:21.358 INFO:teuthology.orchestra.run.smithi143.stderr:dumped fsmap epoch 35 2023-11-26T07:22:21.359 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:21.356+0000 7f7db25e7640 1 -- 172.21.15.143:0/835236203 >> [v2:172.21.15.143:6832/2150557544,v1:172.21.15.143:6833/2150557544] conn(0x7f7d980667d0 msgr2=0x7f7d98068c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:22:21.360 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:21.356+0000 7f7db25e7640 1 --2- 172.21.15.143:0/835236203 >> [v2:172.21.15.143:6832/2150557544,v1:172.21.15.143:6833/2150557544] conn(0x7f7d980667d0 0x7f7d98068c90 secure :-1 s=READY pgs=411 cs=0 l=1 rev1=1 crypto rx=0x7f7dac0acc00 tx=0x7f7d9c008040 comp rx=0 tx=0).stop 2023-11-26T07:22:21.360 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:21.356+0000 7f7db25e7640 1 -- 172.21.15.143:0/835236203 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f7dac0cdd10 msgr2=0x7f7dac1604e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:22:21.360 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:21.356+0000 7f7db25e7640 1 --2- 172.21.15.143:0/835236203 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f7dac0cdd10 0x7f7dac1604e0 secure :-1 s=READY pgs=876 cs=0 l=1 rev1=1 crypto rx=0x7f7da0019070 tx=0x7f7da0019200 comp rx=0 tx=0).stop 2023-11-26T07:22:21.360 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:21.356+0000 7f7db25e7640 1 -- 172.21.15.143:0/835236203 shutdown_connections 2023-11-26T07:22:21.360 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:21.356+0000 7f7db25e7640 1 --2- 172.21.15.143:0/835236203 >> [v2:172.21.15.143:6832/2150557544,v1:172.21.15.143:6833/2150557544] conn(0x7f7d980667d0 0x7f7d98068c90 unknown :-1 s=CLOSED pgs=411 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:22:21.360 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:21.356+0000 7f7db25e7640 1 --2- 172.21.15.143:0/835236203 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f7dac161cf0 0x7f7dac14dd70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:22:21.360 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:21.356+0000 7f7db25e7640 1 --2- 172.21.15.143:0/835236203 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f7dac0cdd10 0x7f7dac1604e0 unknown :-1 s=CLOSED pgs=876 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:22:21.360 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:21.356+0000 7f7db25e7640 1 --2- 172.21.15.143:0/835236203 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f7dac070ba0 0x7f7dac15db40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:22:21.360 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:21.356+0000 7f7db25e7640 1 -- 172.21.15.143:0/835236203 >> 172.21.15.143:0/835236203 conn(0x7f7dac0cf710 msgr2=0x7f7dac0cebf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:22:21.360 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:21.360+0000 7f7db25e7640 1 -- 172.21.15.143:0/835236203 shutdown_connections 2023-11-26T07:22:21.361 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:21.360+0000 7f7db25e7640 1 -- 172.21.15.143:0/835236203 wait complete. 2023-11-26T07:22:21.380 DEBUG:teuthology.orchestra.run.smithi143:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-11-26T07:22:21.517 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:21.516+0000 7ff73532a640 1 Processor -- start 2023-11-26T07:22:21.517 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:21.516+0000 7ff73532a640 1 -- start start 2023-11-26T07:22:21.518 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:21.516+0000 7ff73532a640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7ff73015d6e0 0x7ff7300d0df0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:22:21.518 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:21.516+0000 7ff73532a640 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7ff7300b2160 0x7ff7300d14b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:22:21.518 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:21.516+0000 7ff73532a640 1 -- --> v1:172.21.15.188:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff7300cf0e0 con 0x7ff7300d19f0 2023-11-26T07:22:21.518 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:21.516+0000 7ff73532a640 1 -- --> v2:172.21.15.188:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff73015dac0 con 0x7ff7300b2160 2023-11-26T07:22:21.518 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:21.516+0000 7ff73532a640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff7300d1e80 con 0x7ff73015d6e0 2023-11-26T07:22:21.519 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:21.516+0000 7ff72effd640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7ff73015d6e0 0x7ff7300d0df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:22:21.519 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:21.516+0000 7ff7267fc640 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7ff7300b2160 0x7ff7300d14b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:22:21.519 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:21.516+0000 7ff72effd640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7ff73015d6e0 0x7ff7300d0df0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.188:3301/0 says I am v2:172.21.15.143:35144/0 (socket says 172.21.15.143:35144) 2023-11-26T07:22:21.519 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:21.516+0000 7ff7267fc640 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7ff7300b2160 0x7ff7300d14b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.188:3300/0 says I am v2:172.21.15.143:37336/0 (socket says 172.21.15.143:37336) 2023-11-26T07:22:21.519 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:21.516+0000 7ff72effd640 1 -- 172.21.15.143:0/3296130561 learned_addr learned my addr 172.21.15.143:0/3296130561 (peer_addr_for_me v2:172.21.15.143:0/0) 2023-11-26T07:22:21.520 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:21.516+0000 7ff72effd640 1 -- 172.21.15.143:0/3296130561 >> v2:172.21.15.188:3300/0 conn(0x7ff7300b2160 msgr2=0x7ff7300d14b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-26T07:22:21.520 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:21.516+0000 7ff72effd640 1 --2- 172.21.15.143:0/3296130561 >> v2:172.21.15.188:3300/0 conn(0x7ff7300b2160 0x7ff7300d14b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:22:21.520 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:21.516+0000 7ff72effd640 1 -- 172.21.15.143:0/3296130561 >> v1:172.21.15.188:6789/0 conn(0x7ff7300d19f0 legacy=0x7ff7300cfbf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:22:21.520 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:21.516+0000 7ff72effd640 1 -- 172.21.15.143:0/3296130561 --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff7300ceb20 con 0x7ff73015d6e0 2023-11-26T07:22:21.520 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:21.516+0000 7ff7267fc640 1 --2- 172.21.15.143:0/3296130561 >> v2:172.21.15.188:3300/0 conn(0x7ff7300b2160 0x7ff7300d14b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-26T07:22:21.522 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:21.520+0000 7ff72effd640 1 --2- 172.21.15.143:0/3296130561 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7ff73015d6e0 0x7ff7300d0df0 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7ff718009870 tx=0x7ff71801c540 comp rx=0 tx=0).ready entity=mon.2 client_cookie=e488233cb965e18a server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:22:21.523 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:21.520+0000 7ff72e7fc640 1 -- 172.21.15.143:0/3296130561 <== mon.2 v2:172.21.15.188:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff7180a7070 con 0x7ff73015d6e0 2023-11-26T07:22:21.523 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:21.520+0000 7ff72e7fc640 1 -- 172.21.15.143:0/3296130561 <== mon.2 v2:172.21.15.188:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff7180ab430 con 0x7ff73015d6e0 2023-11-26T07:22:21.523 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:21.520+0000 7ff72e7fc640 1 -- 172.21.15.143:0/3296130561 <== mon.2 v2:172.21.15.188:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff7180ab7c0 con 0x7ff73015d6e0 2023-11-26T07:22:21.523 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:21.520+0000 7ff73532a640 1 -- 172.21.15.143:0/3296130561 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7ff73015d6e0 msgr2=0x7ff7300d0df0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:22:21.523 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:21.520+0000 7ff73532a640 1 --2- 172.21.15.143:0/3296130561 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7ff73015d6e0 0x7ff7300d0df0 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7ff718009870 tx=0x7ff71801c540 comp rx=0 tx=0).stop 2023-11-26T07:22:21.523 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:21.520+0000 7ff73532a640 1 -- 172.21.15.143:0/3296130561 shutdown_connections 2023-11-26T07:22:21.523 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:21.520+0000 7ff73532a640 1 --2- 172.21.15.143:0/3296130561 >> v2:172.21.15.188:3300/0 conn(0x7ff7300b2160 0x7ff7300d14b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:22:21.523 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:21.520+0000 7ff73532a640 1 --2- 172.21.15.143:0/3296130561 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7ff73015d6e0 0x7ff7300d0df0 unknown :-1 s=CLOSED pgs=111 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:22:21.524 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:21.520+0000 7ff73532a640 1 -- 172.21.15.143:0/3296130561 >> 172.21.15.143:0/3296130561 conn(0x7ff73009c550 msgr2=0x7ff7300d3220 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:22:21.524 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:21.520+0000 7ff73532a640 1 -- 172.21.15.143:0/3296130561 shutdown_connections 2023-11-26T07:22:21.524 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:21.520+0000 7ff73532a640 1 -- 172.21.15.143:0/3296130561 wait complete. 2023-11-26T07:22:21.524 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:21.524+0000 7ff73532a640 1 Processor -- start 2023-11-26T07:22:21.524 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:21.524+0000 7ff73532a640 1 -- start start 2023-11-26T07:22:21.526 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:21.524+0000 7ff73532a640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7ff7300b2160 0x7ff730155160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:22:21.527 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:21.524+0000 7ff73532a640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7ff7300d19f0 0x7ff73015e5c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:22:21.527 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:21.524+0000 7ff73532a640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7ff73015d6e0 0x7ff7300ca2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:22:21.527 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:21.524+0000 7ff73532a640 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff73015ef80 con 0x7ff7300d19f0 2023-11-26T07:22:21.527 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:21.524+0000 7ff73532a640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff73015f260 con 0x7ff73015d6e0 2023-11-26T07:22:21.527 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:21.524+0000 7ff73532a640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff73015f540 con 0x7ff7300b2160 2023-11-26T07:22:21.527 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:21.524+0000 7ff7267fc640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7ff7300d19f0 0x7ff73015e5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:22:21.527 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:21.524+0000 7ff7267fc640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7ff7300d19f0 0x7ff73015e5c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:34340/0 (socket says 172.21.15.143:34340) 2023-11-26T07:22:21.527 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:21.524+0000 7ff7267fc640 1 -- 172.21.15.143:0/445003267 learned_addr learned my addr 172.21.15.143:0/445003267 (peer_addr_for_me v2:172.21.15.143:0/0) 2023-11-26T07:22:21.527 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:21.524+0000 7ff72effd640 1 --2- 172.21.15.143:0/445003267 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7ff7300b2160 0x7ff730155160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:22:21.527 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:21.524+0000 7ff72f7fe640 1 --2- 172.21.15.143:0/445003267 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7ff73015d6e0 0x7ff7300ca2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:22:21.527 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:21.524+0000 7ff7267fc640 1 -- 172.21.15.143:0/445003267 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7ff7300b2160 msgr2=0x7ff730155160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:22:21.528 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:21.524+0000 7ff7267fc640 1 --2- 172.21.15.143:0/445003267 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7ff7300b2160 0x7ff730155160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:22:21.528 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:21.524+0000 7ff7267fc640 1 -- 172.21.15.143:0/445003267 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7ff73015d6e0 msgr2=0x7ff7300ca2f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:22:21.528 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:21.524+0000 7ff7267fc640 1 --2- 172.21.15.143:0/445003267 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7ff73015d6e0 0x7ff7300ca2f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:22:21.528 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:21.524+0000 7ff7267fc640 1 -- 172.21.15.143:0/445003267 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff720009420 con 0x7ff7300d19f0 2023-11-26T07:22:21.528 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:21.524+0000 7ff7267fc640 1 --2- 172.21.15.143:0/445003267 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7ff7300d19f0 0x7ff73015e5c0 secure :-1 s=READY pgs=877 cs=0 l=1 rev1=1 crypto rx=0x7ff72000ebe0 tx=0x7ff72000c520 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:22:21.529 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:21.524+0000 7ff727fff640 1 -- 172.21.15.143:0/445003267 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff72000eee0 con 0x7ff7300d19f0 2023-11-26T07:22:21.529 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:21.524+0000 7ff727fff640 1 -- 172.21.15.143:0/445003267 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff72000cc30 con 0x7ff7300d19f0 2023-11-26T07:22:21.529 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:21.524+0000 7ff727fff640 1 -- 172.21.15.143:0/445003267 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff720013470 con 0x7ff7300d19f0 2023-11-26T07:22:21.529 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:21.524+0000 7ff73532a640 1 -- 172.21.15.143:0/445003267 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff7180094d0 con 0x7ff7300d19f0 2023-11-26T07:22:21.529 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:21.524+0000 7ff73532a640 1 -- 172.21.15.143:0/445003267 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff73015a8e0 con 0x7ff7300d19f0 2023-11-26T07:22:21.529 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:21.528+0000 7ff73532a640 1 -- 172.21.15.143:0/445003267 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff73015aaa0 con 0x7ff7300d19f0 2023-11-26T07:22:21.530 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:21.528+0000 7ff727fff640 1 -- 172.21.15.143:0/445003267 <== mon.0 v2:172.21.15.143:3300/0 4 ==== mgrmap(e 11) v1 ==== 86158+0+0 (secure 0 0 0) 0x7ff7200135d0 con 0x7ff7300d19f0 2023-11-26T07:22:21.530 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:21.528+0000 7ff727fff640 1 --2- 172.21.15.143:0/445003267 >> [v2:172.21.15.143:6832/2150557544,v1:172.21.15.143:6833/2150557544] conn(0x7ff7100667d0 0x7ff710068c90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:22:21.530 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:21.528+0000 7ff727fff640 1 -- 172.21.15.143:0/445003267 <== mon.0 v2:172.21.15.143:3300/0 5 ==== osd_map(59..59 src has 1..59) v4 ==== 5709+0+0 (secure 0 0 0) 0x7ff720084650 con 0x7ff7300d19f0 2023-11-26T07:22:21.530 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:21.528+0000 7ff72effd640 1 --2- 172.21.15.143:0/445003267 >> [v2:172.21.15.143:6832/2150557544,v1:172.21.15.143:6833/2150557544] conn(0x7ff7100667d0 0x7ff710068c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:22:21.530 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:21.528+0000 7ff72effd640 1 --2- 172.21.15.143:0/445003267 >> [v2:172.21.15.143:6832/2150557544,v1:172.21.15.143:6833/2150557544] conn(0x7ff7100667d0 0x7ff710068c90 secure :-1 s=READY pgs=412 cs=0 l=1 rev1=1 crypto rx=0x7ff7180af000 tx=0x7ff718008040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:22:21.533 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:21.532+0000 7ff727fff640 1 -- 172.21.15.143:0/445003267 <== mon.0 v2:172.21.15.143:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172649 (secure 0 0 0) 0x7ff720051730 con 0x7ff7300d19f0 2023-11-26T07:22:21.684 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:21.684+0000 7ff73532a640 1 -- 172.21.15.143:0/445003267 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7ff730067680 con 0x7ff7300d19f0 2023-11-26T07:22:21.685 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:21.684+0000 7ff727fff640 1 -- 172.21.15.143:0/445003267 <== mon.0 v2:172.21.15.143:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v59) v1 ==== 74+0+16601 (secure 0 0 0) 0x7ff720054d50 con 0x7ff7300d19f0 2023-11-26T07:22:21.686 INFO:teuthology.orchestra.run.smithi143.stdout: 2023-11-26T07:22:21.686 INFO:teuthology.orchestra.run.smithi143.stdout:{"epoch":59,"fsid":"f5033060-2469-4a07-95fe-2b76f5404bc1","created":"2023-11-26T07:17:28.613663+0000","modified":"2023-11-26T07:21:57.286555+0000","last_up_change":"2023-11-26T07:17:41.157813+0000","last_in_change":"2023-11-26T07:17:31.181658+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":9,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-11-26T07:17:41.758597+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":8,"pool_name":"cephfs_metadata","create_time":"2023-11-26T07:21:53.382407+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"58","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":9,"pool_name":"cephfs_data","create_time":"2023-11-26T07:21:54.656298+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"59","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.6299999952316284,"score_stable":1.6299999952316284,"optimal_score":1,"raw_score_acting":1.6299999952316284,"raw_score_stable":1.6299999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"ebbee2c6-023e-44b9-9f1d-6f7bfd309730","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":57,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6817","nonce":639807369},{"type":"v1","addr":"172.21.15.143:6819","nonce":639807369}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6821","nonce":639807369},{"type":"v1","addr":"172.21.15.143:6823","nonce":639807369}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6830","nonce":639807369},{"type":"v1","addr":"172.21.15.143:6831","nonce":639807369}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6825","nonce":639807369},{"type":"v1","addr":"172.21.15.143:6828","nonce":639807369}]},"public_addr":"172.21.15.143:6819/639807369","cluster_addr":"172.21.15.143:6823/639807369","heartbeat_back_addr":"172.21.15.143:6831/639807369","heartbeat_front_addr":"172.21.15.143:6828/639807369","state":["exists","up"]},{"osd":1,"uuid":"fec4b8be-7122-49e9-9a19-5850a0700001","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":57,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6816","nonce":1144942530},{"type":"v1","addr":"172.21.15.143:6818","nonce":1144942530}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6820","nonce":1144942530},{"type":"v1","addr":"172.21.15.143:6822","nonce":1144942530}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6827","nonce":1144942530},{"type":"v1","addr":"172.21.15.143:6829","nonce":1144942530}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6824","nonce":1144942530},{"type":"v1","addr":"172.21.15.143:6826","nonce":1144942530}]},"public_addr":"172.21.15.143:6818/1144942530","cluster_addr":"172.21.15.143:6822/1144942530","heartbeat_back_addr":"172.21.15.143:6829/1144942530","heartbeat_front_addr":"172.21.15.143:6826/1144942530","state":["exists","up"]},{"osd":2,"uuid":"2888af5b-f933-46ca-8f95-6dd494ff43f6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":57,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6808","nonce":3909095186},{"type":"v1","addr":"172.21.15.143:6809","nonce":3909095186}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6810","nonce":3909095186},{"type":"v1","addr":"172.21.15.143:6811","nonce":3909095186}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6814","nonce":3909095186},{"type":"v1","addr":"172.21.15.143:6815","nonce":3909095186}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6812","nonce":3909095186},{"type":"v1","addr":"172.21.15.143:6813","nonce":3909095186}]},"public_addr":"172.21.15.143:6809/3909095186","cluster_addr":"172.21.15.143:6811/3909095186","heartbeat_back_addr":"172.21.15.143:6815/3909095186","heartbeat_front_addr":"172.21.15.143:6813/3909095186","state":["exists","up"]},{"osd":3,"uuid":"d1f52435-7bc1-4081-9550-9cd1ba55f549","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":57,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6800","nonce":3854210991},{"type":"v1","addr":"172.21.15.143:6801","nonce":3854210991}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6802","nonce":3854210991},{"type":"v1","addr":"172.21.15.143:6803","nonce":3854210991}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6806","nonce":3854210991},{"type":"v1","addr":"172.21.15.143:6807","nonce":3854210991}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6804","nonce":3854210991},{"type":"v1","addr":"172.21.15.143:6805","nonce":3854210991}]},"public_addr":"172.21.15.143:6801/3854210991","cluster_addr":"172.21.15.143:6803/3854210991","heartbeat_back_addr":"172.21.15.143:6807/3854210991","heartbeat_front_addr":"172.21.15.143:6805/3854210991","state":["exists","up"]},{"osd":4,"uuid":"e2ae8b6e-6e25-49b2-bcbf-aacce68f06a3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":57,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6800","nonce":3924606902},{"type":"v1","addr":"172.21.15.188:6801","nonce":3924606902}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6802","nonce":3924606902},{"type":"v1","addr":"172.21.15.188:6803","nonce":3924606902}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6806","nonce":3924606902},{"type":"v1","addr":"172.21.15.188:6807","nonce":3924606902}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6804","nonce":3924606902},{"type":"v1","addr":"172.21.15.188:6805","nonce":3924606902}]},"public_addr":"172.21.15.188:6801/3924606902","cluster_addr":"172.21.15.188:6803/3924606902","heartbeat_back_addr":"172.21.15.188:6807/3924606902","heartbeat_front_addr":"172.21.15.188:6805/3924606902","state":["exists","up"]},{"osd":5,"uuid":"3cec8a32-7698-4c36-8e89-e4ba11934f33","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":57,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6813","nonce":186841544},{"type":"v1","addr":"172.21.15.188:6815","nonce":186841544}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6817","nonce":186841544},{"type":"v1","addr":"172.21.15.188:6818","nonce":186841544}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6822","nonce":186841544},{"type":"v1","addr":"172.21.15.188:6823","nonce":186841544}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6820","nonce":186841544},{"type":"v1","addr":"172.21.15.188:6821","nonce":186841544}]},"public_addr":"172.21.15.188:6815/186841544","cluster_addr":"172.21.15.188:6818/186841544","heartbeat_back_addr":"172.21.15.188:6823/186841544","heartbeat_front_addr":"172.21.15.188:6821/186841544","state":["exists","up"]},{"osd":6,"uuid":"58a65d5d-e090-473a-81b4-ad38e85029f9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":57,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6808","nonce":2153499405},{"type":"v1","addr":"172.21.15.188:6809","nonce":2153499405}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6810","nonce":2153499405},{"type":"v1","addr":"172.21.15.188:6811","nonce":2153499405}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6816","nonce":2153499405},{"type":"v1","addr":"172.21.15.188:6819","nonce":2153499405}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6812","nonce":2153499405},{"type":"v1","addr":"172.21.15.188:6814","nonce":2153499405}]},"public_addr":"172.21.15.188:6809/2153499405","cluster_addr":"172.21.15.188:6811/2153499405","heartbeat_back_addr":"172.21.15.188:6819/2153499405","heartbeat_front_addr":"172.21.15.188:6814/2153499405","state":["exists","up"]},{"osd":7,"uuid":"ea5b7f70-3554-4cf8-ac30-f7c59b80678d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":57,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6824","nonce":2722096729},{"type":"v1","addr":"172.21.15.188:6825","nonce":2722096729}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6826","nonce":2722096729},{"type":"v1","addr":"172.21.15.188:6827","nonce":2722096729}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6830","nonce":2722096729},{"type":"v1","addr":"172.21.15.188:6831","nonce":2722096729}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6828","nonce":2722096729},{"type":"v1","addr":"172.21.15.188:6829","nonce":2722096729}]},"public_addr":"172.21.15.188:6825/2722096729","cluster_addr":"172.21.15.188:6827/2722096729","heartbeat_back_addr":"172.21.15.188:6831/2722096729","heartbeat_front_addr":"172.21.15.188:6829/2722096729","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-26T07:17:36.995032+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-26T07:17:36.932266+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-26T07:17:36.829304+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-26T07:17:36.804297+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-26T07:17:36.667892+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-26T07:17:36.964868+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-26T07:17:36.827332+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-26T07:17:36.893179+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-11-26T07:22:21.688 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:21.684+0000 7ff73532a640 1 -- 172.21.15.143:0/445003267 >> [v2:172.21.15.143:6832/2150557544,v1:172.21.15.143:6833/2150557544] conn(0x7ff7100667d0 msgr2=0x7ff710068c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:22:21.688 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:21.684+0000 7ff73532a640 1 --2- 172.21.15.143:0/445003267 >> [v2:172.21.15.143:6832/2150557544,v1:172.21.15.143:6833/2150557544] conn(0x7ff7100667d0 0x7ff710068c90 secure :-1 s=READY pgs=412 cs=0 l=1 rev1=1 crypto rx=0x7ff7180af000 tx=0x7ff718008040 comp rx=0 tx=0).stop 2023-11-26T07:22:21.688 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:21.684+0000 7ff73532a640 1 -- 172.21.15.143:0/445003267 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7ff7300d19f0 msgr2=0x7ff73015e5c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:22:21.688 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:21.684+0000 7ff73532a640 1 --2- 172.21.15.143:0/445003267 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7ff7300d19f0 0x7ff73015e5c0 secure :-1 s=READY pgs=877 cs=0 l=1 rev1=1 crypto rx=0x7ff72000ebe0 tx=0x7ff72000c520 comp rx=0 tx=0).stop 2023-11-26T07:22:21.688 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:21.684+0000 7ff73532a640 1 -- 172.21.15.143:0/445003267 shutdown_connections 2023-11-26T07:22:21.688 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:21.684+0000 7ff73532a640 1 --2- 172.21.15.143:0/445003267 >> [v2:172.21.15.143:6832/2150557544,v1:172.21.15.143:6833/2150557544] conn(0x7ff7100667d0 0x7ff710068c90 unknown :-1 s=CLOSED pgs=412 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:22:21.688 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:21.684+0000 7ff73532a640 1 --2- 172.21.15.143:0/445003267 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7ff73015d6e0 0x7ff7300ca2f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:22:21.688 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:21.684+0000 7ff73532a640 1 --2- 172.21.15.143:0/445003267 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7ff7300d19f0 0x7ff73015e5c0 unknown :-1 s=CLOSED pgs=877 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:22:21.688 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:21.684+0000 7ff73532a640 1 --2- 172.21.15.143:0/445003267 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7ff7300b2160 0x7ff730155160 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:22:21.688 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:21.684+0000 7ff73532a640 1 -- 172.21.15.143:0/445003267 >> 172.21.15.143:0/445003267 conn(0x7ff73009c550 msgr2=0x7ff7300b28a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:22:21.688 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:21.688+0000 7ff73532a640 1 -- 172.21.15.143:0/445003267 shutdown_connections 2023-11-26T07:22:21.689 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:21.688+0000 7ff73532a640 1 -- 172.21.15.143:0/445003267 wait complete. 2023-11-26T07:22:21.707 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-11-26T07:17:41.758597+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 8, 'pool_name': 'cephfs_metadata', 'create_time': '2023-11-26T07:21:53.382407+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '58', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 9, 'pool_name': 'cephfs_data', 'create_time': '2023-11-26T07:21:54.656298+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '59', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.6299999952316284, 'score_stable': 1.6299999952316284, 'optimal_score': 1, 'raw_score_acting': 1.6299999952316284, 'raw_score_stable': 1.6299999952316284, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2023-11-26T07:22:21.707 DEBUG:teuthology.orchestra.run.smithi143:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-11-26T07:22:21.849 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:21.848+0000 7febc1b35640 1 Processor -- start 2023-11-26T07:22:21.850 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:21.848+0000 7febc1b35640 1 -- start start 2023-11-26T07:22:21.852 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:21.848+0000 7febc1b35640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7febbc0d1bc0 0x7febbc0d0df0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:22:21.852 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:21.848+0000 7febc1b35640 1 -- --> v1:172.21.15.143:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7febbc0cf0e0 con 0x7febbc0b2160 2023-11-26T07:22:21.853 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:21.848+0000 7febc1b35640 1 -- --> v1:172.21.15.188:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7febbc0ceb20 con 0x7febbc15d6e0 2023-11-26T07:22:21.853 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:21.848+0000 7febc1b35640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7febbc15dac0 con 0x7febbc0d1bc0 2023-11-26T07:22:21.853 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:21.848+0000 7febbbfff640 1 --1- >> v1:172.21.15.143:6789/0 conn(0x7febbc0b2160 0x7febbc0b0750 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.143:6789/0 says I am v1:172.21.15.143:59236/0 (socket says 172.21.15.143:59236) 2023-11-26T07:22:21.853 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:21.848+0000 7febbbfff640 1 -- 172.21.15.143:0/2833112277 learned_addr learned my addr 172.21.15.143:0/2833112277 (peer_addr_for_me v1:172.21.15.143:0/0) 2023-11-26T07:22:21.853 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:21.848+0000 7febbaffd640 1 --2- 172.21.15.143:0/2833112277 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7febbc0d1bc0 0x7febbc0d0df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:22:21.853 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:21.848+0000 7febba7fc640 1 -- 172.21.15.143:0/2833112277 <== mon.0 v1:172.21.15.143:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2049967585 0 0) 0x7febbc0cf0e0 con 0x7febbc0b2160 2023-11-26T07:22:21.853 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:21.848+0000 7febba7fc640 1 -- 172.21.15.143:0/2833112277 --> v1:172.21.15.143:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7feba4003530 con 0x7febbc0b2160 2023-11-26T07:22:21.853 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:21.848+0000 7febba7fc640 1 -- 172.21.15.143:0/2833112277 <== mon.1 v1:172.21.15.188:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2379061903 0 0) 0x7febbc0ceb20 con 0x7febbc15d6e0 2023-11-26T07:22:21.853 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:21.848+0000 7febba7fc640 1 -- 172.21.15.143:0/2833112277 --> v1:172.21.15.188:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7febbc0cf0e0 con 0x7febbc15d6e0 2023-11-26T07:22:21.853 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:21.848+0000 7febba7fc640 1 -- 172.21.15.143:0/2833112277 <== mon.0 v1:172.21.15.143:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 365201878 0 0) 0x7feba4003530 con 0x7febbc0b2160 2023-11-26T07:22:21.853 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:21.848+0000 7febba7fc640 1 -- 172.21.15.143:0/2833112277 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7febbc0d1bc0 msgr2=0x7febbc0d0df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-26T07:22:21.853 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:21.848+0000 7febba7fc640 1 --2- 172.21.15.143:0/2833112277 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7febbc0d1bc0 0x7febbc0d0df0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:22:21.853 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:21.848+0000 7febba7fc640 1 -- 172.21.15.143:0/2833112277 >> v1:172.21.15.188:6789/0 conn(0x7febbc15d6e0 legacy=0x7febbc0d14b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:22:21.853 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:21.848+0000 7febba7fc640 1 -- 172.21.15.143:0/2833112277 --> v1:172.21.15.143:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7febbc0cee00 con 0x7febbc0b2160 2023-11-26T07:22:21.854 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:21.848+0000 7febba7fc640 1 -- 172.21.15.143:0/2833112277 <== mon.0 v1:172.21.15.143:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2137882523 0 0) 0x7febb4001a00 con 0x7febbc0b2160 2023-11-26T07:22:21.854 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:21.852+0000 7febba7fc640 1 -- 172.21.15.143:0/2833112277 >> v1:172.21.15.143:6789/0 conn(0x7febbc0b2160 legacy=0x7febbc0b0750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:22:21.854 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:21.852+0000 7febba7fc640 1 --2- 172.21.15.143:0/2833112277 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7feba4003b40 0x7feba4003f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:22:21.854 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:21.852+0000 7febba7fc640 1 --2- 172.21.15.143:0/2833112277 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7feba40047d0 0x7feba4004c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:22:21.854 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:21.852+0000 7febba7fc640 1 --2- 172.21.15.143:0/2833112277 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7feba4005190 0x7feba400b990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:22:21.854 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:21.852+0000 7febba7fc640 1 -- 172.21.15.143:0/2833112277 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x7febbc15dac0 con 0x7feba4005190 2023-11-26T07:22:21.854 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:21.852+0000 7febba7fc640 1 -- 172.21.15.143:0/2833112277 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7feba400bfb0 con 0x7feba40047d0 2023-11-26T07:22:21.854 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:21.852+0000 7febba7fc640 1 -- 172.21.15.143:0/2833112277 --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7feba400c170 con 0x7feba4003b40 2023-11-26T07:22:21.854 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:21.852+0000 7febbbfff640 1 --2- 172.21.15.143:0/2833112277 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7feba4005190 0x7feba400b990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:22:21.854 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:21.852+0000 7febbb7fe640 1 --2- 172.21.15.143:0/2833112277 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7feba4003b40 0x7feba4003f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:22:21.854 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:21.852+0000 7febbaffd640 1 --2- 172.21.15.143:0/2833112277 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7feba40047d0 0x7feba4004c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:22:21.854 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:21.852+0000 7febbbfff640 1 -- 172.21.15.143:0/2833112277 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7feba4003b40 msgr2=0x7feba4003f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-26T07:22:21.854 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:21.852+0000 7febbbfff640 1 --2- 172.21.15.143:0/2833112277 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7feba4003b40 0x7feba4003f40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:22:21.855 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:21.852+0000 7febbbfff640 1 -- 172.21.15.143:0/2833112277 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7feba40047d0 msgr2=0x7feba4004c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-26T07:22:21.855 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:21.852+0000 7febbbfff640 1 --2- 172.21.15.143:0/2833112277 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7feba40047d0 0x7feba4004c50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:22:21.855 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:21.852+0000 7febbbfff640 1 -- 172.21.15.143:0/2833112277 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7feba4003530 con 0x7feba4005190 2023-11-26T07:22:21.855 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:21.852+0000 7febbbfff640 1 --2- 172.21.15.143:0/2833112277 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7feba4005190 0x7feba400b990 secure :-1 s=READY pgs=879 cs=0 l=1 rev1=1 crypto rx=0x7febb400a6e0 tx=0x7febb401f920 comp rx=0 tx=0).ready entity=mon.0 client_cookie=403216b8301c01a2 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:22:21.855 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:21.852+0000 7febba7fc640 1 -- 172.21.15.143:0/2833112277 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7febb40ad660 con 0x7feba4005190 2023-11-26T07:22:21.855 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:21.852+0000 7febba7fc640 1 -- 172.21.15.143:0/2833112277 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7febb40ad7c0 con 0x7feba4005190 2023-11-26T07:22:21.855 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:21.852+0000 7febba7fc640 1 -- 172.21.15.143:0/2833112277 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7febb40adbf0 con 0x7feba4005190 2023-11-26T07:22:21.856 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:21.856+0000 7febc1b35640 1 -- 172.21.15.143:0/2833112277 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7feba4005190 msgr2=0x7feba400b990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:22:21.856 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:21.856+0000 7febc1b35640 1 --2- 172.21.15.143:0/2833112277 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7feba4005190 0x7feba400b990 secure :-1 s=READY pgs=879 cs=0 l=1 rev1=1 crypto rx=0x7febb400a6e0 tx=0x7febb401f920 comp rx=0 tx=0).stop 2023-11-26T07:22:21.856 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:21.856+0000 7febbbfff640 1 -- 172.21.15.143:0/2833112277 reap_dead start 2023-11-26T07:22:21.856 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:21.856+0000 7febc1b35640 1 -- 172.21.15.143:0/2833112277 shutdown_connections 2023-11-26T07:22:21.857 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:21.856+0000 7febc1b35640 1 -- 172.21.15.143:0/2833112277 >> 172.21.15.143:0/2833112277 conn(0x7febbc09c550 msgr2=0x7febbc0d3220 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:22:21.857 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:21.856+0000 7febc1b35640 1 -- 172.21.15.143:0/2833112277 shutdown_connections 2023-11-26T07:22:21.857 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:21.856+0000 7febc1b35640 1 -- 172.21.15.143:0/2833112277 wait complete. 2023-11-26T07:22:21.858 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:21.856+0000 7febc1b35640 1 Processor -- start 2023-11-26T07:22:21.858 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:21.856+0000 7febc1b35640 1 -- start start 2023-11-26T07:22:21.860 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:21.856+0000 7febc1b35640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7feba4005190 0x7febbc15dc00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:22:21.861 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:21.856+0000 7febc1b35640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7febbc0ca2f0 0x7febbc151690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:22:21.861 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:21.856+0000 7febc1b35640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7febbc0c85d0 0x7febbc0c8a80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:22:21.861 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:21.856+0000 7febc1b35640 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x7febbc0c8fc0 con 0x7febbc0c85d0 2023-11-26T07:22:21.861 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:21.856+0000 7febc1b35640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7febbc15e1e0 con 0x7febbc0ca2f0 2023-11-26T07:22:21.861 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:21.856+0000 7febc1b35640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7febbc15e4c0 con 0x7feba4005190 2023-11-26T07:22:21.861 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:21.856+0000 7febbbfff640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7febbc0c85d0 0x7febbc0c8a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:22:21.861 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:21.856+0000 7febbbfff640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7febbc0c85d0 0x7febbc0c8a80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:34344/0 (socket says 172.21.15.143:34344) 2023-11-26T07:22:21.861 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:21.856+0000 7febbbfff640 1 -- 172.21.15.143:0/641699714 learned_addr learned my addr 172.21.15.143:0/641699714 (peer_addr_for_me v2:172.21.15.143:0/0) 2023-11-26T07:22:21.861 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:21.856+0000 7febbb7fe640 1 --2- 172.21.15.143:0/641699714 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7feba4005190 0x7febbc15dc00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:22:21.861 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:21.856+0000 7febbaffd640 1 --2- 172.21.15.143:0/641699714 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7febbc0ca2f0 0x7febbc151690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:22:21.861 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:21.860+0000 7febbbfff640 1 -- 172.21.15.143:0/641699714 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7feba4005190 msgr2=0x7febbc15dc00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:22:21.862 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:21.860+0000 7febbbfff640 1 --2- 172.21.15.143:0/641699714 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7feba4005190 0x7febbc15dc00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:22:21.862 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:21.860+0000 7febbbfff640 1 -- 172.21.15.143:0/641699714 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7febbc0ca2f0 msgr2=0x7febbc151690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:22:21.862 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:21.860+0000 7febbbfff640 1 --2- 172.21.15.143:0/641699714 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7febbc0ca2f0 0x7febbc151690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:22:21.862 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:21.860+0000 7febbbfff640 1 -- 172.21.15.143:0/641699714 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7febbc15e7a0 con 0x7febbc0c85d0 2023-11-26T07:22:21.862 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:21.860+0000 7febbbfff640 1 --2- 172.21.15.143:0/641699714 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7febbc0c85d0 0x7febbc0c8a80 secure :-1 s=READY pgs=880 cs=0 l=1 rev1=1 crypto rx=0x7febb400a6e0 tx=0x7feba4005800 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:22:21.862 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:21.860+0000 7feba3fff640 1 -- 172.21.15.143:0/641699714 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7febb4001210 con 0x7febbc0c85d0 2023-11-26T07:22:21.862 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:21.860+0000 7feba3fff640 1 -- 172.21.15.143:0/641699714 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7febb4001800 con 0x7febbc0c85d0 2023-11-26T07:22:21.862 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:21.860+0000 7febc1b35640 1 -- 172.21.15.143:0/641699714 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7febbc15ea80 con 0x7febbc0c85d0 2023-11-26T07:22:21.862 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:21.860+0000 7feba3fff640 1 -- 172.21.15.143:0/641699714 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7febb40ad660 con 0x7febbc0c85d0 2023-11-26T07:22:21.862 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:21.860+0000 7febc1b35640 1 -- 172.21.15.143:0/641699714 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7febbc0ca770 con 0x7febbc0c85d0 2023-11-26T07:22:21.863 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:21.860+0000 7febc1b35640 1 -- 172.21.15.143:0/641699714 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7febbc067680 con 0x7febbc0c85d0 2023-11-26T07:22:21.863 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:21.860+0000 7feba3fff640 1 -- 172.21.15.143:0/641699714 <== mon.0 v2:172.21.15.143:3300/0 4 ==== mgrmap(e 11) v1 ==== 86158+0+0 (secure 0 0 0) 0x7febb40bf070 con 0x7febbc0c85d0 2023-11-26T07:22:21.864 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:21.864+0000 7feba3fff640 1 --2- 172.21.15.143:0/641699714 >> [v2:172.21.15.143:6832/2150557544,v1:172.21.15.143:6833/2150557544] conn(0x7feba80667d0 0x7feba8068c90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:22:21.864 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:21.864+0000 7feba3fff640 1 -- 172.21.15.143:0/641699714 <== mon.0 v2:172.21.15.143:3300/0 5 ==== osd_map(59..59 src has 1..59) v4 ==== 5709+0+0 (secure 0 0 0) 0x7febb4122f10 con 0x7febbc0c85d0 2023-11-26T07:22:21.864 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:21.864+0000 7febbb7fe640 1 --2- 172.21.15.143:0/641699714 >> [v2:172.21.15.143:6832/2150557544,v1:172.21.15.143:6833/2150557544] conn(0x7feba80667d0 0x7feba8068c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:22:21.864 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:21.864+0000 7febbb7fe640 1 --2- 172.21.15.143:0/641699714 >> [v2:172.21.15.143:6832/2150557544,v1:172.21.15.143:6833/2150557544] conn(0x7feba80667d0 0x7feba8068c90 secure :-1 s=READY pgs=413 cs=0 l=1 rev1=1 crypto rx=0x7febac007fd0 tx=0x7febac009040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:22:21.868 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:21.868+0000 7feba3fff640 1 -- 172.21.15.143:0/641699714 <== mon.0 v2:172.21.15.143:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172649 (secure 0 0 0) 0x7febb40f3b50 con 0x7febbc0c85d0 2023-11-26T07:22:22.018 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:22.016+0000 7febc1b35640 1 -- 172.21.15.143:0/641699714 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7febbc15ec10 con 0x7febbc0c85d0 2023-11-26T07:22:22.018 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:22.016+0000 7feba3fff640 1 -- 172.21.15.143:0/641699714 <== mon.0 v2:172.21.15.143:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v59) v1 ==== 93+0+10 (secure 0 0 0) 0x7febb40f36e0 con 0x7febbc0c85d0 2023-11-26T07:22:22.018 INFO:teuthology.orchestra.run.smithi143.stdout:pg_num: 1 2023-11-26T07:22:22.020 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:22.020+0000 7febc1b35640 1 -- 172.21.15.143:0/641699714 >> [v2:172.21.15.143:6832/2150557544,v1:172.21.15.143:6833/2150557544] conn(0x7feba80667d0 msgr2=0x7feba8068c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:22:22.020 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:22.020+0000 7febc1b35640 1 --2- 172.21.15.143:0/641699714 >> [v2:172.21.15.143:6832/2150557544,v1:172.21.15.143:6833/2150557544] conn(0x7feba80667d0 0x7feba8068c90 secure :-1 s=READY pgs=413 cs=0 l=1 rev1=1 crypto rx=0x7febac007fd0 tx=0x7febac009040 comp rx=0 tx=0).stop 2023-11-26T07:22:22.020 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:22.020+0000 7febc1b35640 1 -- 172.21.15.143:0/641699714 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7febbc0c85d0 msgr2=0x7febbc0c8a80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:22:22.020 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:22.020+0000 7febc1b35640 1 --2- 172.21.15.143:0/641699714 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7febbc0c85d0 0x7febbc0c8a80 secure :-1 s=READY pgs=880 cs=0 l=1 rev1=1 crypto rx=0x7febb400a6e0 tx=0x7feba4005800 comp rx=0 tx=0).stop 2023-11-26T07:22:22.020 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:22.020+0000 7febc1b35640 1 -- 172.21.15.143:0/641699714 shutdown_connections 2023-11-26T07:22:22.021 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:22.020+0000 7febc1b35640 1 --2- 172.21.15.143:0/641699714 >> [v2:172.21.15.143:6832/2150557544,v1:172.21.15.143:6833/2150557544] conn(0x7feba80667d0 0x7feba8068c90 unknown :-1 s=CLOSED pgs=413 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:22:22.021 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:22.020+0000 7febc1b35640 1 --2- 172.21.15.143:0/641699714 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7febbc0c85d0 0x7febbc0c8a80 unknown :-1 s=CLOSED pgs=880 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:22:22.021 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:22.020+0000 7febc1b35640 1 --2- 172.21.15.143:0/641699714 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7febbc0ca2f0 0x7febbc151690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:22:22.021 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:22.020+0000 7febc1b35640 1 --2- 172.21.15.143:0/641699714 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7feba4005190 0x7febbc15dc00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:22:22.021 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:22.020+0000 7febc1b35640 1 -- 172.21.15.143:0/641699714 >> 172.21.15.143:0/641699714 conn(0x7febbc09c550 msgr2=0x7febbc0cf730 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:22:22.021 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:22.020+0000 7febc1b35640 1 -- 172.21.15.143:0/641699714 shutdown_connections 2023-11-26T07:22:22.021 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:22.020+0000 7febc1b35640 1 -- 172.21.15.143:0/641699714 wait complete. 2023-11-26T07:22:22.039 DEBUG:teuthology.orchestra.run.smithi143:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-11-26T07:22:22.180 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:22.180+0000 7fe434679640 1 Processor -- start 2023-11-26T07:22:22.180 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:22.180+0000 7fe434679640 1 -- start start 2023-11-26T07:22:22.181 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:22.180+0000 7fe434679640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fe42c13e290 0x7fe42c0cf280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:22:22.181 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:22.180+0000 7fe434679640 1 -- --> v1:172.21.15.143:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe42c06c520 con 0x7fe42c15f6b0 2023-11-26T07:22:22.182 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:22.180+0000 7fe434679640 1 -- --> v1:172.21.15.188:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe42c06c800 con 0x7fe42c0cf7c0 2023-11-26T07:22:22.182 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:22.180+0000 7fe434679640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe42c0cfc40 con 0x7fe42c13e290 2023-11-26T07:22:22.182 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:22.180+0000 7fe432c16640 1 --1- >> v1:172.21.15.143:6789/0 conn(0x7fe42c15f6b0 0x7fe42c0cd7b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.143:6789/0 says I am v1:172.21.15.143:59246/0 (socket says 172.21.15.143:59246) 2023-11-26T07:22:22.182 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:22.180+0000 7fe432c16640 1 -- 172.21.15.143:0/1191437941 learned_addr learned my addr 172.21.15.143:0/1191437941 (peer_addr_for_me v1:172.21.15.143:0/0) 2023-11-26T07:22:22.183 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:22.180+0000 7fe432415640 1 --2- 172.21.15.143:0/1191437941 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fe42c13e290 0x7fe42c0cf280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:22:22.183 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:22.180+0000 7fe431413640 1 -- 172.21.15.143:0/1191437941 <== mon.0 v1:172.21.15.143:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 149348697 0 0) 0x7fe42c06c520 con 0x7fe42c15f6b0 2023-11-26T07:22:22.183 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:22.180+0000 7fe431413640 1 -- 172.21.15.143:0/1191437941 --> v1:172.21.15.143:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fe418003530 con 0x7fe42c15f6b0 2023-11-26T07:22:22.183 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:22.180+0000 7fe431413640 1 -- 172.21.15.143:0/1191437941 <== mon.1 v1:172.21.15.188:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4026217903 0 0) 0x7fe42c06c800 con 0x7fe42c0cf7c0 2023-11-26T07:22:22.183 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:22.180+0000 7fe431413640 1 -- 172.21.15.143:0/1191437941 --> v1:172.21.15.188:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fe42c06c520 con 0x7fe42c0cf7c0 2023-11-26T07:22:22.183 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:22.180+0000 7fe432415640 1 -- 172.21.15.143:0/1191437941 >> v1:172.21.15.188:6789/0 conn(0x7fe42c0cf7c0 legacy=0x7fe42c15efa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:22:22.183 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:22.180+0000 7fe431413640 1 -- 172.21.15.143:0/1191437941 <== mon.0 v1:172.21.15.143:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1256602472 0 0) 0x7fe418003530 con 0x7fe42c15f6b0 2023-11-26T07:22:22.183 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:22.180+0000 7fe432415640 1 -- 172.21.15.143:0/1191437941 >> v1:172.21.15.143:6789/0 conn(0x7fe42c15f6b0 legacy=0x7fe42c0cd7b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:22:22.183 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:22.180+0000 7fe432415640 1 -- 172.21.15.143:0/1191437941 --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe42c06cae0 con 0x7fe42c13e290 2023-11-26T07:22:22.185 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:22.184+0000 7fe432415640 1 --2- 172.21.15.143:0/1191437941 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fe42c13e290 0x7fe42c0cf280 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7fe4280097f0 tx=0x7fe42801c3e0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=ffb31a4d903f5cf8 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:22:22.186 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:22.184+0000 7fe431413640 1 -- 172.21.15.143:0/1191437941 <== mon.2 v2:172.21.15.188:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe424001a00 con 0x7fe42c13e290 2023-11-26T07:22:22.186 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:22.184+0000 7fe431413640 1 -- 172.21.15.143:0/1191437941 <== mon.2 v2:172.21.15.188:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe4280a7050 con 0x7fe42c13e290 2023-11-26T07:22:22.186 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:22.184+0000 7fe431413640 1 -- 172.21.15.143:0/1191437941 <== mon.2 v2:172.21.15.188:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe4280ab600 con 0x7fe42c13e290 2023-11-26T07:22:22.187 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:22.184+0000 7fe434679640 1 -- 172.21.15.143:0/1191437941 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fe42c13e290 msgr2=0x7fe42c0cf280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:22:22.187 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:22.184+0000 7fe434679640 1 --2- 172.21.15.143:0/1191437941 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fe42c13e290 0x7fe42c0cf280 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7fe4280097f0 tx=0x7fe42801c3e0 comp rx=0 tx=0).stop 2023-11-26T07:22:22.187 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:22.184+0000 7fe434679640 1 -- 172.21.15.143:0/1191437941 shutdown_connections 2023-11-26T07:22:22.187 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:22.184+0000 7fe434679640 1 --2- 172.21.15.143:0/1191437941 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fe42c13e290 0x7fe42c0cf280 unknown :-1 s=CLOSED pgs=112 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:22:22.187 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:22.184+0000 7fe434679640 1 -- 172.21.15.143:0/1191437941 >> 172.21.15.143:0/1191437941 conn(0x7fe42c06a890 msgr2=0x7fe42c06aca0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:22:22.187 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:22.184+0000 7fe434679640 1 -- 172.21.15.143:0/1191437941 shutdown_connections 2023-11-26T07:22:22.187 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:22.184+0000 7fe434679640 1 -- 172.21.15.143:0/1191437941 wait complete. 2023-11-26T07:22:22.188 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:22.188+0000 7fe434679640 1 Processor -- start 2023-11-26T07:22:22.188 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:22.188+0000 7fe434679640 1 -- start start 2023-11-26T07:22:22.190 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:22.188+0000 7fe434679640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fe42c0cf7c0 0x7fe42c1573d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:22:22.190 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:22.188+0000 7fe434679640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fe42c13e290 0x7fe42c139bb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:22:22.190 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:22.188+0000 7fe434679640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fe42c15f6b0 0x7fe42c1503d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:22:22.191 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:22.188+0000 7fe434679640 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe42c15b170 con 0x7fe42c15f6b0 2023-11-26T07:22:22.191 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:22.188+0000 7fe434679640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe42c156700 con 0x7fe42c13e290 2023-11-26T07:22:22.191 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:22.188+0000 7fe434679640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe42c156f40 con 0x7fe42c0cf7c0 2023-11-26T07:22:22.191 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:22.188+0000 7fe432c16640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fe42c15f6b0 0x7fe42c1503d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:22:22.191 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:22.188+0000 7fe432c16640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fe42c15f6b0 0x7fe42c1503d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:34352/0 (socket says 172.21.15.143:34352) 2023-11-26T07:22:22.191 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:22.188+0000 7fe432c16640 1 -- 172.21.15.143:0/1612889190 learned_addr learned my addr 172.21.15.143:0/1612889190 (peer_addr_for_me v2:172.21.15.143:0/0) 2023-11-26T07:22:22.191 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:22.188+0000 7fe432415640 1 --2- 172.21.15.143:0/1612889190 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fe42c0cf7c0 0x7fe42c1573d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:22:22.191 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:22.188+0000 7fe431c14640 1 --2- 172.21.15.143:0/1612889190 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fe42c13e290 0x7fe42c139bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:22:22.191 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:22.188+0000 7fe432c16640 1 -- 172.21.15.143:0/1612889190 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fe42c0cf7c0 msgr2=0x7fe42c1573d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:22:22.191 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:22.188+0000 7fe432c16640 1 --2- 172.21.15.143:0/1612889190 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fe42c0cf7c0 0x7fe42c1573d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:22:22.191 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:22.188+0000 7fe432c16640 1 -- 172.21.15.143:0/1612889190 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fe42c13e290 msgr2=0x7fe42c139bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:22:22.191 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:22.188+0000 7fe432c16640 1 --2- 172.21.15.143:0/1612889190 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fe42c13e290 0x7fe42c139bb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:22:22.191 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:22.188+0000 7fe432c16640 1 -- 172.21.15.143:0/1612889190 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe428009420 con 0x7fe42c15f6b0 2023-11-26T07:22:22.192 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:22.188+0000 7fe432c16640 1 --2- 172.21.15.143:0/1612889190 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fe42c15f6b0 0x7fe42c1503d0 secure :-1 s=READY pgs=882 cs=0 l=1 rev1=1 crypto rx=0x7fe42400cc60 tx=0x7fe42400a430 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:22:22.192 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:22.188+0000 7fe422ffd640 1 -- 172.21.15.143:0/1612889190 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe42400ce70 con 0x7fe42c15f6b0 2023-11-26T07:22:22.192 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:22.188+0000 7fe422ffd640 1 -- 172.21.15.143:0/1612889190 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe424012400 con 0x7fe42c15f6b0 2023-11-26T07:22:22.193 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:22.188+0000 7fe434679640 1 -- 172.21.15.143:0/1612889190 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe42c13a0f0 con 0x7fe42c15f6b0 2023-11-26T07:22:22.193 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:22.188+0000 7fe422ffd640 1 -- 172.21.15.143:0/1612889190 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe4240127b0 con 0x7fe42c15f6b0 2023-11-26T07:22:22.193 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:22.188+0000 7fe434679640 1 -- 172.21.15.143:0/1612889190 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe42c1568f0 con 0x7fe42c15f6b0 2023-11-26T07:22:22.193 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:22.192+0000 7fe434679640 1 -- 172.21.15.143:0/1612889190 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe42c1570f0 con 0x7fe42c15f6b0 2023-11-26T07:22:22.193 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:22.192+0000 7fe422ffd640 1 -- 172.21.15.143:0/1612889190 <== mon.0 v2:172.21.15.143:3300/0 4 ==== mgrmap(e 11) v1 ==== 86158+0+0 (secure 0 0 0) 0x7fe424012910 con 0x7fe42c15f6b0 2023-11-26T07:22:22.194 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:22.192+0000 7fe422ffd640 1 --2- 172.21.15.143:0/1612889190 >> [v2:172.21.15.143:6832/2150557544,v1:172.21.15.143:6833/2150557544] conn(0x7fe4100667d0 0x7fe410068c90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:22:22.194 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:22.192+0000 7fe422ffd640 1 -- 172.21.15.143:0/1612889190 <== mon.0 v2:172.21.15.143:3300/0 5 ==== osd_map(59..59 src has 1..59) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fe4240849c0 con 0x7fe42c15f6b0 2023-11-26T07:22:22.194 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:22.192+0000 7fe432415640 1 --2- 172.21.15.143:0/1612889190 >> [v2:172.21.15.143:6832/2150557544,v1:172.21.15.143:6833/2150557544] conn(0x7fe4100667d0 0x7fe410068c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:22:22.195 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:22.192+0000 7fe432415640 1 --2- 172.21.15.143:0/1612889190 >> [v2:172.21.15.143:6832/2150557544,v1:172.21.15.143:6833/2150557544] conn(0x7fe4100667d0 0x7fe410068c90 secure :-1 s=READY pgs=414 cs=0 l=1 rev1=1 crypto rx=0x7fe4280ad000 tx=0x7fe4280a6000 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:22:22.198 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:22.196+0000 7fe422ffd640 1 -- 172.21.15.143:0/1612889190 <== mon.0 v2:172.21.15.143:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172649 (secure 0 0 0) 0x7fe424051a70 con 0x7fe42c15f6b0 2023-11-26T07:22:22.265 DEBUG:teuthology.orchestra.run.smithi143:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-26T07:22:22.268 DEBUG:teuthology.orchestra.run.smithi188:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-26T07:22:22.350 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:22.348+0000 7fe434679640 1 -- 172.21.15.143:0/1612889190 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7fe42c067680 con 0x7fe42c15f6b0 2023-11-26T07:22:22.350 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:22.348+0000 7fe422ffd640 1 -- 172.21.15.143:0/1612889190 <== mon.0 v2:172.21.15.143:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v59) v1 ==== 104+0+11 (secure 0 0 0) 0x7fe424055090 con 0x7fe42c15f6b0 2023-11-26T07:22:22.350 INFO:teuthology.orchestra.run.smithi143.stdout:pg_num: 32 2023-11-26T07:22:22.353 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:22.352+0000 7fe434679640 1 -- 172.21.15.143:0/1612889190 >> [v2:172.21.15.143:6832/2150557544,v1:172.21.15.143:6833/2150557544] conn(0x7fe4100667d0 msgr2=0x7fe410068c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:22:22.353 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:22.352+0000 7fe434679640 1 --2- 172.21.15.143:0/1612889190 >> [v2:172.21.15.143:6832/2150557544,v1:172.21.15.143:6833/2150557544] conn(0x7fe4100667d0 0x7fe410068c90 secure :-1 s=READY pgs=414 cs=0 l=1 rev1=1 crypto rx=0x7fe4280ad000 tx=0x7fe4280a6000 comp rx=0 tx=0).stop 2023-11-26T07:22:22.353 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:22.352+0000 7fe434679640 1 -- 172.21.15.143:0/1612889190 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fe42c15f6b0 msgr2=0x7fe42c1503d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:22:22.353 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:22.352+0000 7fe434679640 1 --2- 172.21.15.143:0/1612889190 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fe42c15f6b0 0x7fe42c1503d0 secure :-1 s=READY pgs=882 cs=0 l=1 rev1=1 crypto rx=0x7fe42400cc60 tx=0x7fe42400a430 comp rx=0 tx=0).stop 2023-11-26T07:22:22.353 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:22.352+0000 7fe434679640 1 -- 172.21.15.143:0/1612889190 shutdown_connections 2023-11-26T07:22:22.353 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:22.352+0000 7fe434679640 1 --2- 172.21.15.143:0/1612889190 >> [v2:172.21.15.143:6832/2150557544,v1:172.21.15.143:6833/2150557544] conn(0x7fe4100667d0 0x7fe410068c90 unknown :-1 s=CLOSED pgs=414 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:22:22.353 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:22.352+0000 7fe434679640 1 --2- 172.21.15.143:0/1612889190 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fe42c15f6b0 0x7fe42c1503d0 unknown :-1 s=CLOSED pgs=882 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:22:22.353 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:22.352+0000 7fe434679640 1 --2- 172.21.15.143:0/1612889190 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fe42c13e290 0x7fe42c139bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:22:22.353 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:22.352+0000 7fe434679640 1 --2- 172.21.15.143:0/1612889190 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fe42c0cf7c0 0x7fe42c1573d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:22:22.354 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:22.352+0000 7fe434679640 1 -- 172.21.15.143:0/1612889190 >> 172.21.15.143:0/1612889190 conn(0x7fe42c06a890 msgr2=0x7fe42c15c270 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:22:22.354 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:22.352+0000 7fe434679640 1 -- 172.21.15.143:0/1612889190 shutdown_connections 2023-11-26T07:22:22.354 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:22.352+0000 7fe434679640 1 -- 172.21.15.143:0/1612889190 wait complete. 2023-11-26T07:22:22.371 DEBUG:teuthology.orchestra.run.smithi143:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-11-26T07:22:22.512 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:22.512+0000 7fe940f7d640 1 Processor -- start 2023-11-26T07:22:22.512 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:22.512+0000 7fe940f7d640 1 -- start start 2023-11-26T07:22:22.513 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:22.512+0000 7fe940f7d640 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7fe93c0b23f0 0x7fe93c0ada60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:22:22.513 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:22.512+0000 7fe940f7d640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fe93c0d0200 0x7fe93c06be10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:22:22.513 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:22.512+0000 7fe940f7d640 1 -- --> v1:172.21.15.188:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe93c0cc050 con 0x7fe93c06b9d0 2023-11-26T07:22:22.513 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:22.512+0000 7fe940f7d640 1 -- --> v2:172.21.15.188:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe93c06c350 con 0x7fe93c0b23f0 2023-11-26T07:22:22.513 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:22.512+0000 7fe940f7d640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe93c0cb440 con 0x7fe93c0d0200 2023-11-26T07:22:22.514 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:22.512+0000 7fe93bfff640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fe93c0d0200 0x7fe93c06be10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:22:22.515 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:22.512+0000 7fe93b7fe640 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7fe93c0b23f0 0x7fe93c0ada60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:22:22.515 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:22.512+0000 7fe93bfff640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fe93c0d0200 0x7fe93c06be10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.188:3301/0 says I am v2:172.21.15.143:35206/0 (socket says 172.21.15.143:35206) 2023-11-26T07:22:22.515 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:22.512+0000 7fe93bfff640 1 -- 172.21.15.143:0/1147824883 learned_addr learned my addr 172.21.15.143:0/1147824883 (peer_addr_for_me v2:172.21.15.143:0/0) 2023-11-26T07:22:22.515 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:22.512+0000 7fe93affd640 1 --1- >> v1:172.21.15.188:6789/0 conn(0x7fe93c06b9d0 0x7fe93c0cfaf0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.188:6789/0 says I am v1:172.21.15.143:49146/0 (socket says 172.21.15.143:49146) 2023-11-26T07:22:22.515 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:22.512+0000 7fe93a7fc640 1 -- 172.21.15.143:0/1147824883 <== mon.1 v1:172.21.15.188:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2391860699 0 0) 0x7fe93c0cc050 con 0x7fe93c06b9d0 2023-11-26T07:22:22.515 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:22.512+0000 7fe93a7fc640 1 -- 172.21.15.143:0/1147824883 --> v1:172.21.15.188:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fe924003530 con 0x7fe93c06b9d0 2023-11-26T07:22:22.515 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:22.512+0000 7fe93bfff640 1 -- 172.21.15.143:0/1147824883 >> v2:172.21.15.188:3300/0 conn(0x7fe93c0b23f0 msgr2=0x7fe93c0ada60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-26T07:22:22.515 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:22.512+0000 7fe93bfff640 1 --2- 172.21.15.143:0/1147824883 >> v2:172.21.15.188:3300/0 conn(0x7fe93c0b23f0 0x7fe93c0ada60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:22:22.515 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:22.512+0000 7fe93bfff640 1 -- 172.21.15.143:0/1147824883 >> v1:172.21.15.188:6789/0 conn(0x7fe93c06b9d0 legacy=0x7fe93c0cfaf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:22:22.515 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:22.512+0000 7fe93bfff640 1 -- 172.21.15.143:0/1147824883 --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe93c0cb720 con 0x7fe93c0d0200 2023-11-26T07:22:22.515 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:22.512+0000 7fe93b7fe640 1 --2- 172.21.15.143:0/1147824883 >> v2:172.21.15.188:3300/0 conn(0x7fe93c0b23f0 0x7fe93c0ada60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-26T07:22:22.517 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:22.516+0000 7fe93bfff640 1 --2- 172.21.15.143:0/1147824883 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fe93c0d0200 0x7fe93c06be10 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7fe934009e50 tx=0x7fe93401ca10 comp rx=0 tx=0).ready entity=mon.2 client_cookie=4b77d88724730008 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:22:22.517 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:22.516+0000 7fe93a7fc640 1 -- 172.21.15.143:0/1147824883 <== mon.2 v2:172.21.15.188:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe9340a57d0 con 0x7fe93c0d0200 2023-11-26T07:22:22.518 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:22.516+0000 7fe93a7fc640 1 -- 172.21.15.143:0/1147824883 <== mon.2 v2:172.21.15.188:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe9340a5930 con 0x7fe93c0d0200 2023-11-26T07:22:22.518 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:22.516+0000 7fe93a7fc640 1 -- 172.21.15.143:0/1147824883 <== mon.2 v2:172.21.15.188:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe9340a5ca0 con 0x7fe93c0d0200 2023-11-26T07:22:22.518 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:22.516+0000 7fe940f7d640 1 -- 172.21.15.143:0/1147824883 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fe93c0d0200 msgr2=0x7fe93c06be10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:22:22.518 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:22.516+0000 7fe940f7d640 1 --2- 172.21.15.143:0/1147824883 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fe93c0d0200 0x7fe93c06be10 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7fe934009e50 tx=0x7fe93401ca10 comp rx=0 tx=0).stop 2023-11-26T07:22:22.519 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:22.516+0000 7fe940f7d640 1 -- 172.21.15.143:0/1147824883 shutdown_connections 2023-11-26T07:22:22.519 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:22.516+0000 7fe940f7d640 1 --2- 172.21.15.143:0/1147824883 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fe93c0d0200 0x7fe93c06be10 unknown :-1 s=CLOSED pgs=113 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:22:22.519 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:22.516+0000 7fe940f7d640 1 --2- 172.21.15.143:0/1147824883 >> v2:172.21.15.188:3300/0 conn(0x7fe93c0b23f0 0x7fe93c0ada60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:22:22.519 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:22.516+0000 7fe940f7d640 1 -- 172.21.15.143:0/1147824883 >> 172.21.15.143:0/1147824883 conn(0x7fe93c0d0df0 msgr2=0x7fe93c0d1200 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:22:22.519 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:22.516+0000 7fe940f7d640 1 -- 172.21.15.143:0/1147824883 shutdown_connections 2023-11-26T07:22:22.519 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:22.516+0000 7fe940f7d640 1 -- 172.21.15.143:0/1147824883 wait complete. 2023-11-26T07:22:22.520 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:22.520+0000 7fe940f7d640 1 Processor -- start 2023-11-26T07:22:22.520 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:22.520+0000 7fe940f7d640 1 -- start start 2023-11-26T07:22:22.521 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:22.520+0000 7fe940f7d640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fe93c06b9d0 0x7fe93c15af00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:22:22.521 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:22.520+0000 7fe940f7d640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fe93c0b23f0 0x7fe93c143530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:22:22.521 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:22.520+0000 7fe940f7d640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fe93c0d0200 0x7fe93c143a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:22:22.522 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:22.520+0000 7fe940f7d640 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe93c15d990 con 0x7fe93c0d0200 2023-11-26T07:22:22.522 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:22.520+0000 7fe940f7d640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe93c144050 con 0x7fe93c06b9d0 2023-11-26T07:22:22.522 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:22.520+0000 7fe940f7d640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe93c0c6170 con 0x7fe93c0b23f0 2023-11-26T07:22:22.522 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:22.520+0000 7fe93bfff640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fe93c0d0200 0x7fe93c143a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:22:22.522 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:22.520+0000 7fe93bfff640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fe93c0d0200 0x7fe93c143a70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:34354/0 (socket says 172.21.15.143:34354) 2023-11-26T07:22:22.522 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:22.520+0000 7fe93bfff640 1 -- 172.21.15.143:0/2055888749 learned_addr learned my addr 172.21.15.143:0/2055888749 (peer_addr_for_me v2:172.21.15.143:0/0) 2023-11-26T07:22:22.523 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:22.520+0000 7fe93affd640 1 --2- 172.21.15.143:0/2055888749 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fe93c0b23f0 0x7fe93c143530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:22:22.523 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:22.520+0000 7fe93b7fe640 1 --2- 172.21.15.143:0/2055888749 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fe93c06b9d0 0x7fe93c15af00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:22:22.523 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:22.520+0000 7fe93bfff640 1 -- 172.21.15.143:0/2055888749 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fe93c0b23f0 msgr2=0x7fe93c143530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:22:22.523 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:22.520+0000 7fe93bfff640 1 --2- 172.21.15.143:0/2055888749 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fe93c0b23f0 0x7fe93c143530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:22:22.524 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:22.520+0000 7fe93bfff640 1 -- 172.21.15.143:0/2055888749 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fe93c06b9d0 msgr2=0x7fe93c15af00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:22:22.524 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:22.520+0000 7fe93bfff640 1 --2- 172.21.15.143:0/2055888749 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fe93c06b9d0 0x7fe93c15af00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:22:22.524 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:22.520+0000 7fe93bfff640 1 -- 172.21.15.143:0/2055888749 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe92c009420 con 0x7fe93c0d0200 2023-11-26T07:22:22.524 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:22.520+0000 7fe93bfff640 1 --2- 172.21.15.143:0/2055888749 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fe93c0d0200 0x7fe93c143a70 secure :-1 s=READY pgs=883 cs=0 l=1 rev1=1 crypto rx=0x7fe9340af040 tx=0x7fe9340197b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:22:22.524 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:22.520+0000 7fe923fff640 1 -- 172.21.15.143:0/2055888749 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe9340b8640 con 0x7fe93c0d0200 2023-11-26T07:22:22.524 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:22.520+0000 7fe923fff640 1 -- 172.21.15.143:0/2055888749 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe9340b87a0 con 0x7fe93c0d0200 2023-11-26T07:22:22.524 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:22.520+0000 7fe923fff640 1 -- 172.21.15.143:0/2055888749 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe9340b8b50 con 0x7fe93c0d0200 2023-11-26T07:22:22.524 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:22.520+0000 7fe940f7d640 1 -- 172.21.15.143:0/2055888749 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe934009b30 con 0x7fe93c0d0200 2023-11-26T07:22:22.524 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:22.520+0000 7fe940f7d640 1 -- 172.21.15.143:0/2055888749 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe93c0c6730 con 0x7fe93c0d0200 2023-11-26T07:22:22.525 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:22.524+0000 7fe940f7d640 1 -- 172.21.15.143:0/2055888749 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe93c06d4f0 con 0x7fe93c0d0200 2023-11-26T07:22:22.525 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:22.524+0000 7fe923fff640 1 -- 172.21.15.143:0/2055888749 <== mon.0 v2:172.21.15.143:3300/0 4 ==== mgrmap(e 11) v1 ==== 86158+0+0 (secure 0 0 0) 0x7fe9340b8cb0 con 0x7fe93c0d0200 2023-11-26T07:22:22.525 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:22.524+0000 7fe923fff640 1 --2- 172.21.15.143:0/2055888749 >> [v2:172.21.15.143:6832/2150557544,v1:172.21.15.143:6833/2150557544] conn(0x7fe9280667d0 0x7fe928068c90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:22:22.525 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:22.524+0000 7fe923fff640 1 -- 172.21.15.143:0/2055888749 <== mon.0 v2:172.21.15.143:3300/0 5 ==== osd_map(59..59 src has 1..59) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fe93411f2c0 con 0x7fe93c0d0200 2023-11-26T07:22:22.526 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:22.524+0000 7fe93b7fe640 1 --2- 172.21.15.143:0/2055888749 >> [v2:172.21.15.143:6832/2150557544,v1:172.21.15.143:6833/2150557544] conn(0x7fe9280667d0 0x7fe928068c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:22:22.526 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:22.524+0000 7fe93b7fe640 1 --2- 172.21.15.143:0/2055888749 >> [v2:172.21.15.143:6832/2150557544,v1:172.21.15.143:6833/2150557544] conn(0x7fe9280667d0 0x7fe928068c90 secure :-1 s=READY pgs=415 cs=0 l=1 rev1=1 crypto rx=0x7fe93c1572e0 tx=0x7fe92c008040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:22:22.529 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:22.528+0000 7fe923fff640 1 -- 172.21.15.143:0/2055888749 <== mon.0 v2:172.21.15.143:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172649 (secure 0 0 0) 0x7fe9340ec420 con 0x7fe93c0d0200 2023-11-26T07:22:22.682 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:22.680+0000 7fe940f7d640 1 -- 172.21.15.143:0/2055888749 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7fe93c067500 con 0x7fe93c0d0200 2023-11-26T07:22:22.683 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:22.680+0000 7fe923fff640 1 -- 172.21.15.143:0/2055888749 <== mon.0 v2:172.21.15.143:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v59) v1 ==== 100+0+11 (secure 0 0 0) 0x7fe9340efa40 con 0x7fe93c0d0200 2023-11-26T07:22:22.683 INFO:teuthology.orchestra.run.smithi143.stdout:pg_num: 64 2023-11-26T07:22:22.685 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:22.684+0000 7fe940f7d640 1 -- 172.21.15.143:0/2055888749 >> [v2:172.21.15.143:6832/2150557544,v1:172.21.15.143:6833/2150557544] conn(0x7fe9280667d0 msgr2=0x7fe928068c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:22:22.685 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:22.684+0000 7fe940f7d640 1 --2- 172.21.15.143:0/2055888749 >> [v2:172.21.15.143:6832/2150557544,v1:172.21.15.143:6833/2150557544] conn(0x7fe9280667d0 0x7fe928068c90 secure :-1 s=READY pgs=415 cs=0 l=1 rev1=1 crypto rx=0x7fe93c1572e0 tx=0x7fe92c008040 comp rx=0 tx=0).stop 2023-11-26T07:22:22.685 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:22.684+0000 7fe940f7d640 1 -- 172.21.15.143:0/2055888749 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fe93c0d0200 msgr2=0x7fe93c143a70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:22:22.685 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:22.684+0000 7fe940f7d640 1 --2- 172.21.15.143:0/2055888749 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fe93c0d0200 0x7fe93c143a70 secure :-1 s=READY pgs=883 cs=0 l=1 rev1=1 crypto rx=0x7fe9340af040 tx=0x7fe9340197b0 comp rx=0 tx=0).stop 2023-11-26T07:22:22.685 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:22.684+0000 7fe940f7d640 1 -- 172.21.15.143:0/2055888749 shutdown_connections 2023-11-26T07:22:22.685 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:22.684+0000 7fe940f7d640 1 --2- 172.21.15.143:0/2055888749 >> [v2:172.21.15.143:6832/2150557544,v1:172.21.15.143:6833/2150557544] conn(0x7fe9280667d0 0x7fe928068c90 unknown :-1 s=CLOSED pgs=415 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:22:22.685 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:22.684+0000 7fe940f7d640 1 --2- 172.21.15.143:0/2055888749 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fe93c0d0200 0x7fe93c143a70 unknown :-1 s=CLOSED pgs=883 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:22:22.686 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:22.684+0000 7fe940f7d640 1 --2- 172.21.15.143:0/2055888749 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fe93c0b23f0 0x7fe93c143530 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:22:22.686 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:22.684+0000 7fe940f7d640 1 --2- 172.21.15.143:0/2055888749 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fe93c06b9d0 0x7fe93c15af00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:22:22.686 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:22.684+0000 7fe940f7d640 1 -- 172.21.15.143:0/2055888749 >> 172.21.15.143:0/2055888749 conn(0x7fe93c0d0df0 msgr2=0x7fe93c0d11d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:22:22.686 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:22.684+0000 7fe940f7d640 1 -- 172.21.15.143:0/2055888749 shutdown_connections 2023-11-26T07:22:22.686 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:22.684+0000 7fe940f7d640 1 -- 172.21.15.143:0/2055888749 wait complete. 2023-11-26T07:22:22.704 DEBUG:teuthology.orchestra.run.smithi143:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-11-26T07:22:22.841 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:22.840+0000 7fe28ec9a640 1 Processor -- start 2023-11-26T07:22:22.842 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:22.840+0000 7fe28ec9a640 1 -- start start 2023-11-26T07:22:22.843 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:22.840+0000 7fe28ec9a640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fe2880d1bc0 0x7fe2880d0df0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:22:22.843 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:22.840+0000 7fe28ec9a640 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7fe2880b2160 0x7fe2880b0750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:22:22.843 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:22.840+0000 7fe28ec9a640 1 -- --> v1:172.21.15.143:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe2880cf0e0 con 0x7fe28815d6e0 2023-11-26T07:22:22.843 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:22.840+0000 7fe28ec9a640 1 -- --> v2:172.21.15.188:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe28815dac0 con 0x7fe2880b2160 2023-11-26T07:22:22.843 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:22.840+0000 7fe28ec9a640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe2880ceb20 con 0x7fe2880d1bc0 2023-11-26T07:22:22.844 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:22.840+0000 7fe28ca36640 1 --1- >> v1:172.21.15.143:6789/0 conn(0x7fe28815d6e0 0x7fe2880d14b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.143:6789/0 says I am v1:172.21.15.143:59260/0 (socket says 172.21.15.143:59260) 2023-11-26T07:22:22.844 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:22.840+0000 7fe28ca36640 1 -- 172.21.15.143:0/4151230671 learned_addr learned my addr 172.21.15.143:0/4151230671 (peer_addr_for_me v1:172.21.15.143:0/0) 2023-11-26T07:22:22.844 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:22.840+0000 7fe28d237640 1 --2- 172.21.15.143:0/4151230671 >> v2:172.21.15.188:3300/0 conn(0x7fe2880b2160 0x7fe2880b0750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:22:22.844 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:22.840+0000 7fe27ffff640 1 --2- 172.21.15.143:0/4151230671 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fe2880d1bc0 0x7fe2880d0df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:22:22.844 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:22.840+0000 7fe27f7fe640 1 -- 172.21.15.143:0/4151230671 <== mon.0 v1:172.21.15.143:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1316320392 0 0) 0x7fe2880cf0e0 con 0x7fe28815d6e0 2023-11-26T07:22:22.844 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:22.840+0000 7fe27f7fe640 1 -- 172.21.15.143:0/4151230671 --> v1:172.21.15.143:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fe270003530 con 0x7fe28815d6e0 2023-11-26T07:22:22.845 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:22.840+0000 7fe27f7fe640 1 -- 172.21.15.143:0/4151230671 <== mon.0 v1:172.21.15.143:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3387618808 0 0) 0x7fe270003530 con 0x7fe28815d6e0 2023-11-26T07:22:22.845 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:22.840+0000 7fe27f7fe640 1 -- 172.21.15.143:0/4151230671 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fe2880d1bc0 msgr2=0x7fe2880d0df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-26T07:22:22.845 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:22.840+0000 7fe27f7fe640 1 --2- 172.21.15.143:0/4151230671 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fe2880d1bc0 0x7fe2880d0df0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:22:22.846 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:22.840+0000 7fe27f7fe640 1 -- 172.21.15.143:0/4151230671 >> v2:172.21.15.188:3300/0 conn(0x7fe2880b2160 msgr2=0x7fe2880b0750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-26T07:22:22.846 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:22.840+0000 7fe27f7fe640 1 --2- 172.21.15.143:0/4151230671 >> v2:172.21.15.188:3300/0 conn(0x7fe2880b2160 0x7fe2880b0750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:22:22.846 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:22.840+0000 7fe27f7fe640 1 -- 172.21.15.143:0/4151230671 --> v1:172.21.15.143:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe2880cee00 con 0x7fe28815d6e0 2023-11-26T07:22:22.846 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:22.840+0000 7fe27f7fe640 1 -- 172.21.15.143:0/4151230671 <== mon.0 v1:172.21.15.143:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2137882523 0 0) 0x7fe284001680 con 0x7fe28815d6e0 2023-11-26T07:22:22.846 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:22.840+0000 7fe27f7fe640 1 -- 172.21.15.143:0/4151230671 >> v1:172.21.15.143:6789/0 conn(0x7fe28815d6e0 legacy=0x7fe2880d14b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:22:22.846 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:22.840+0000 7fe27f7fe640 1 --2- 172.21.15.143:0/4151230671 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fe270003980 0x7fe270003da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:22:22.846 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:22.844+0000 7fe27f7fe640 1 --2- 172.21.15.143:0/4151230671 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fe2700047d0 0x7fe270004c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:22:22.846 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:22.844+0000 7fe27f7fe640 1 --2- 172.21.15.143:0/4151230671 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fe2880d1bc0 0x7fe270005190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:22:22.846 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:22.844+0000 7fe27f7fe640 1 -- 172.21.15.143:0/4151230671 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe28815dac0 con 0x7fe270003980 2023-11-26T07:22:22.846 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:22.844+0000 7fe27f7fe640 1 -- 172.21.15.143:0/4151230671 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe2880ceb20 con 0x7fe2700047d0 2023-11-26T07:22:22.846 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:22.844+0000 7fe27f7fe640 1 -- 172.21.15.143:0/4151230671 --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe27000b990 con 0x7fe2880d1bc0 2023-11-26T07:22:22.846 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:22.844+0000 7fe28ca36640 1 --2- 172.21.15.143:0/4151230671 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fe270003980 0x7fe270003da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:22:22.847 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:22.844+0000 7fe27ffff640 1 --2- 172.21.15.143:0/4151230671 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fe2700047d0 0x7fe270004c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:22:22.847 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:22.844+0000 7fe28d237640 1 --2- 172.21.15.143:0/4151230671 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fe2880d1bc0 0x7fe270005190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:22:22.847 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:22.844+0000 7fe28ca36640 1 -- 172.21.15.143:0/4151230671 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fe2880d1bc0 msgr2=0x7fe270005190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-26T07:22:22.847 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:22.844+0000 7fe28ca36640 1 --2- 172.21.15.143:0/4151230671 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fe2880d1bc0 0x7fe270005190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:22:22.847 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:22.844+0000 7fe28ca36640 1 -- 172.21.15.143:0/4151230671 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fe2700047d0 msgr2=0x7fe270004c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-26T07:22:22.847 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:22.844+0000 7fe28ca36640 1 --2- 172.21.15.143:0/4151230671 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fe2700047d0 0x7fe270004c50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:22:22.847 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:22.844+0000 7fe28ca36640 1 -- 172.21.15.143:0/4151230671 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe270003530 con 0x7fe270003980 2023-11-26T07:22:22.848 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:22.844+0000 7fe28ca36640 1 --2- 172.21.15.143:0/4151230671 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fe270003980 0x7fe270003da0 secure :-1 s=READY pgs=885 cs=0 l=1 rev1=1 crypto rx=0x7fe28400c240 tx=0x7fe28401cc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2696ff9a6c729d0b server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:22:22.848 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:22.844+0000 7fe27f7fe640 1 -- 172.21.15.143:0/4151230671 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe2840058e0 con 0x7fe270003980 2023-11-26T07:22:22.848 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:22.844+0000 7fe27f7fe640 1 -- 172.21.15.143:0/4151230671 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe284005a40 con 0x7fe270003980 2023-11-26T07:22:22.848 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:22.844+0000 7fe27f7fe640 1 -- 172.21.15.143:0/4151230671 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe284005e10 con 0x7fe270003980 2023-11-26T07:22:22.848 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:22.848+0000 7fe28ec9a640 1 -- 172.21.15.143:0/4151230671 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fe270003980 msgr2=0x7fe270003da0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:22:22.848 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:22.848+0000 7fe28ec9a640 1 --2- 172.21.15.143:0/4151230671 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fe270003980 0x7fe270003da0 secure :-1 s=READY pgs=885 cs=0 l=1 rev1=1 crypto rx=0x7fe28400c240 tx=0x7fe28401cc30 comp rx=0 tx=0).stop 2023-11-26T07:22:22.848 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:22.848+0000 7fe28d237640 1 -- 172.21.15.143:0/4151230671 reap_dead start 2023-11-26T07:22:22.849 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:22.848+0000 7fe28ec9a640 1 -- 172.21.15.143:0/4151230671 shutdown_connections 2023-11-26T07:22:22.849 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:22.848+0000 7fe28ec9a640 1 -- 172.21.15.143:0/4151230671 >> 172.21.15.143:0/4151230671 conn(0x7fe28809c550 msgr2=0x7fe2880d3220 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:22:22.849 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:22.848+0000 7fe28ec9a640 1 -- 172.21.15.143:0/4151230671 shutdown_connections 2023-11-26T07:22:22.849 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:22.848+0000 7fe28ec9a640 1 -- 172.21.15.143:0/4151230671 wait complete. 2023-11-26T07:22:22.850 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:22.848+0000 7fe28ec9a640 1 Processor -- start 2023-11-26T07:22:22.850 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:22.848+0000 7fe28ec9a640 1 -- start start 2023-11-26T07:22:22.852 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:22.848+0000 7fe28ec9a640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fe270003980 0x7fe2881630c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:22:22.852 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:22.848+0000 7fe28ec9a640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fe2880b71c0 0x7fe28814c220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:22:22.852 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:22.848+0000 7fe28ec9a640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fe2880b7640 0x7fe28814c760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:22:22.852 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:22.848+0000 7fe28ec9a640 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe2880b5e40 con 0x7fe2880b7640 2023-11-26T07:22:22.852 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:22.848+0000 7fe28ec9a640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe2880b6120 con 0x7fe2880b71c0 2023-11-26T07:22:22.853 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:22.848+0000 7fe28ec9a640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe2880b6400 con 0x7fe270003980 2023-11-26T07:22:22.853 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:22.848+0000 7fe28d237640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fe2880b7640 0x7fe28814c760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:22:22.853 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:22.848+0000 7fe28d237640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fe2880b7640 0x7fe28814c760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:34374/0 (socket says 172.21.15.143:34374) 2023-11-26T07:22:22.853 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:22.848+0000 7fe28d237640 1 -- 172.21.15.143:0/1827581232 learned_addr learned my addr 172.21.15.143:0/1827581232 (peer_addr_for_me v2:172.21.15.143:0/0) 2023-11-26T07:22:22.853 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:22.848+0000 7fe28ca36640 1 --2- 172.21.15.143:0/1827581232 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fe270003980 0x7fe2881630c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:22:22.853 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:22.848+0000 7fe27ffff640 1 --2- 172.21.15.143:0/1827581232 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fe2880b71c0 0x7fe28814c220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:22:22.853 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:22.852+0000 7fe28d237640 1 -- 172.21.15.143:0/1827581232 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fe270003980 msgr2=0x7fe2881630c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:22:22.853 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:22.852+0000 7fe28d237640 1 --2- 172.21.15.143:0/1827581232 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fe270003980 0x7fe2881630c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:22:22.853 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:22.852+0000 7fe28d237640 1 -- 172.21.15.143:0/1827581232 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fe2880b71c0 msgr2=0x7fe28814c220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:22:22.853 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:22.852+0000 7fe28d237640 1 --2- 172.21.15.143:0/1827581232 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fe2880b71c0 0x7fe28814c220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:22:22.853 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:22.852+0000 7fe28d237640 1 -- 172.21.15.143:0/1827581232 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe2880b66e0 con 0x7fe2880b7640 2023-11-26T07:22:22.853 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:22.852+0000 7fe27ffff640 1 --2- 172.21.15.143:0/1827581232 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fe2880b71c0 0x7fe28814c220 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-26T07:22:22.854 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:22.852+0000 7fe28d237640 1 --2- 172.21.15.143:0/1827581232 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fe2880b7640 0x7fe28814c760 secure :-1 s=READY pgs=886 cs=0 l=1 rev1=1 crypto rx=0x7fe28808ca40 tx=0x7fe28000f580 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:22:22.854 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:22.852+0000 7fe27d7fa640 1 -- 172.21.15.143:0/1827581232 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe28000e040 con 0x7fe2880b7640 2023-11-26T07:22:22.854 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:22.852+0000 7fe27d7fa640 1 -- 172.21.15.143:0/1827581232 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe28000f040 con 0x7fe2880b7640 2023-11-26T07:22:22.854 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:22.852+0000 7fe28ec9a640 1 -- 172.21.15.143:0/1827581232 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe2880b69c0 con 0x7fe2880b7640 2023-11-26T07:22:22.854 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:22.852+0000 7fe27d7fa640 1 -- 172.21.15.143:0/1827581232 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe280015670 con 0x7fe2880b7640 2023-11-26T07:22:22.854 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:22.852+0000 7fe28ec9a640 1 -- 172.21.15.143:0/1827581232 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe288163600 con 0x7fe2880b7640 2023-11-26T07:22:22.854 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:22.852+0000 7fe28ec9a640 1 -- 172.21.15.143:0/1827581232 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe288067680 con 0x7fe2880b7640 2023-11-26T07:22:22.855 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:22.852+0000 7fe27d7fa640 1 -- 172.21.15.143:0/1827581232 <== mon.0 v2:172.21.15.143:3300/0 4 ==== mgrmap(e 11) v1 ==== 86158+0+0 (secure 0 0 0) 0x7fe2800157d0 con 0x7fe2880b7640 2023-11-26T07:22:22.856 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:22.852+0000 7fe27d7fa640 1 --2- 172.21.15.143:0/1827581232 >> [v2:172.21.15.143:6832/2150557544,v1:172.21.15.143:6833/2150557544] conn(0x7fe2740667d0 0x7fe274068c90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:22:22.856 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:22.856+0000 7fe27d7fa640 1 -- 172.21.15.143:0/1827581232 <== mon.0 v2:172.21.15.143:3300/0 5 ==== osd_map(59..59 src has 1..59) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fe280084b30 con 0x7fe2880b7640 2023-11-26T07:22:22.856 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:22.856+0000 7fe28ca36640 1 --2- 172.21.15.143:0/1827581232 >> [v2:172.21.15.143:6832/2150557544,v1:172.21.15.143:6833/2150557544] conn(0x7fe2740667d0 0x7fe274068c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:22:22.856 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:22.856+0000 7fe28ca36640 1 --2- 172.21.15.143:0/1827581232 >> [v2:172.21.15.143:6832/2150557544,v1:172.21.15.143:6833/2150557544] conn(0x7fe2740667d0 0x7fe274068c90 secure :-1 s=READY pgs=416 cs=0 l=1 rev1=1 crypto rx=0x7fe2840aa000 tx=0x7fe284001a80 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:22:22.859 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:22.856+0000 7fe27d7fa640 1 -- 172.21.15.143:0/1827581232 <== mon.0 v2:172.21.15.143:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172649 (secure 0 0 0) 0x7fe280051a90 con 0x7fe2880b7640 2023-11-26T07:22:23.039 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:23.036+0000 7fe28ec9a640 1 -- 172.21.15.143:0/1827581232 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fe2880b1b10 con 0x7fe2880b7640 2023-11-26T07:22:23.040 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:23.040+0000 7fe27d7fa640 1 -- 172.21.15.143:0/1827581232 <== mon.0 v2:172.21.15.143:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 35 v35) v1 ==== 94+0+4925 (secure 0 0 0) 0x7fe2800550b0 con 0x7fe2880b7640 2023-11-26T07:22:23.040 INFO:teuthology.orchestra.run.smithi143.stdout: 2023-11-26T07:22:23.040 INFO:teuthology.orchestra.run.smithi143.stdout:{"epoch":35,"default_fscid":4,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5612,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.143:6837/1372238588","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6836","nonce":1372238588},{"type":"v1","addr":"172.21.15.143:6837","nonce":1372238588}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":29},{"gid":6103,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.143:6835/1078738477","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6834","nonce":1078738477},{"type":"v1","addr":"172.21.15.143:6835","nonce":1078738477}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":29},{"gid":6118,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.188:6834/1634083777","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6832","nonce":1634083777},{"type":"v1","addr":"172.21.15.188:6834","nonce":1634083777}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":29}],"filesystems":[{"mdsmap":{"epoch":34,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-11-26T07:21:55.656168+0000","modified":"2023-11-26T07:21:58.674916+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":6121},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6121":{"gid":6121,"name":"b","rank":0,"incarnation":30,"state":"up:active","state_seq":3,"addr":"172.21.15.188:6835/1958716738","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6833","nonce":1958716738},{"type":"v1","addr":"172.21.15.188:6835","nonce":1958716738}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[9],"metadata_pool":8,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":4}]} 2023-11-26T07:22:23.040 INFO:teuthology.orchestra.run.smithi143.stderr:dumped fsmap epoch 35 2023-11-26T07:22:23.042 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:23.040+0000 7fe28ec9a640 1 -- 172.21.15.143:0/1827581232 >> [v2:172.21.15.143:6832/2150557544,v1:172.21.15.143:6833/2150557544] conn(0x7fe2740667d0 msgr2=0x7fe274068c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:22:23.042 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:23.040+0000 7fe28ec9a640 1 --2- 172.21.15.143:0/1827581232 >> [v2:172.21.15.143:6832/2150557544,v1:172.21.15.143:6833/2150557544] conn(0x7fe2740667d0 0x7fe274068c90 secure :-1 s=READY pgs=416 cs=0 l=1 rev1=1 crypto rx=0x7fe2840aa000 tx=0x7fe284001a80 comp rx=0 tx=0).stop 2023-11-26T07:22:23.042 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:23.040+0000 7fe28ec9a640 1 -- 172.21.15.143:0/1827581232 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fe2880b7640 msgr2=0x7fe28814c760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:22:23.042 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:23.040+0000 7fe28ec9a640 1 --2- 172.21.15.143:0/1827581232 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fe2880b7640 0x7fe28814c760 secure :-1 s=READY pgs=886 cs=0 l=1 rev1=1 crypto rx=0x7fe28808ca40 tx=0x7fe28000f580 comp rx=0 tx=0).stop 2023-11-26T07:22:23.042 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:23.040+0000 7fe28ec9a640 1 -- 172.21.15.143:0/1827581232 shutdown_connections 2023-11-26T07:22:23.043 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:23.040+0000 7fe28ec9a640 1 --2- 172.21.15.143:0/1827581232 >> [v2:172.21.15.143:6832/2150557544,v1:172.21.15.143:6833/2150557544] conn(0x7fe2740667d0 0x7fe274068c90 unknown :-1 s=CLOSED pgs=416 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:22:23.043 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:23.040+0000 7fe28ec9a640 1 --2- 172.21.15.143:0/1827581232 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fe2880b7640 0x7fe28814c760 unknown :-1 s=CLOSED pgs=886 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:22:23.043 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:23.040+0000 7fe28ec9a640 1 --2- 172.21.15.143:0/1827581232 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fe2880b71c0 0x7fe28814c220 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:22:23.043 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:23.040+0000 7fe28ec9a640 1 --2- 172.21.15.143:0/1827581232 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fe270003980 0x7fe2881630c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:22:23.043 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:23.040+0000 7fe28ec9a640 1 -- 172.21.15.143:0/1827581232 >> 172.21.15.143:0/1827581232 conn(0x7fe28809c550 msgr2=0x7fe28806c290 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:22:23.043 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:23.040+0000 7fe28ec9a640 1 -- 172.21.15.143:0/1827581232 shutdown_connections 2023-11-26T07:22:23.043 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:23.040+0000 7fe28ec9a640 1 -- 172.21.15.143:0/1827581232 wait complete. 2023-11-26T07:22:23.061 DEBUG:teuthology.orchestra.run.smithi143:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-11-26T07:22:23.198 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:23.196+0000 7fe1ef6fe640 1 Processor -- start 2023-11-26T07:22:23.198 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:23.196+0000 7fe1ef6fe640 1 -- start start 2023-11-26T07:22:23.199 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:23.196+0000 7fe1ef6fe640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fe1e80cf280 0x7fe1e80cf6e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:22:23.200 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:23.196+0000 7fe1ef6fe640 1 -- --> v1:172.21.15.143:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe1e80cfc20 con 0x7fe1e80cd1b0 2023-11-26T07:22:23.200 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:23.196+0000 7fe1ef6fe640 1 -- --> v1:172.21.15.188:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe1e80cc730 con 0x7fe1e8143ac0 2023-11-26T07:22:23.200 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:23.196+0000 7fe1ef6fe640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe1e8143ea0 con 0x7fe1e80cf280 2023-11-26T07:22:23.200 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:23.196+0000 7fe1ed49a640 1 --1- >> v1:172.21.15.188:6789/0 conn(0x7fe1e8143ac0 0x7fe1e8070370 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.188:6789/0 says I am v1:172.21.15.143:49162/0 (socket says 172.21.15.143:49162) 2023-11-26T07:22:23.200 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:23.196+0000 7fe1ed49a640 1 -- 172.21.15.143:0/1790890923 learned_addr learned my addr 172.21.15.143:0/1790890923 (peer_addr_for_me v1:172.21.15.143:0/0) 2023-11-26T07:22:23.200 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:23.196+0000 7fe1edc9b640 1 --1- >> v1:172.21.15.143:6789/0 conn(0x7fe1e80cd1b0 0x7fe1e80cd5e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.143:6789/0 says I am v1:172.21.15.143:59264/0 (socket says 172.21.15.143:59264) 2023-11-26T07:22:23.200 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:23.196+0000 7fe1ecc99640 1 --2- 172.21.15.143:0/1790890923 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fe1e80cf280 0x7fe1e80cf6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:22:23.200 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:23.200+0000 7fe1ecc99640 1 -- 172.21.15.143:0/1790890923 >> v1:172.21.15.188:6789/0 conn(0x7fe1e8143ac0 legacy=0x7fe1e8070370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:22:23.201 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:23.200+0000 7fe1dffff640 1 -- 172.21.15.143:0/1790890923 <== mon.1 v1:172.21.15.188:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 176812648 0 0) 0x7fe1e80cc730 con 0x7fe1e8143ac0 2023-11-26T07:22:23.201 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:23.200+0000 7fe1ecc99640 1 -- 172.21.15.143:0/1790890923 >> v1:172.21.15.143:6789/0 conn(0x7fe1e80cd1b0 legacy=0x7fe1e80cd5e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:22:23.201 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:23.200+0000 7fe1ecc99640 1 -- 172.21.15.143:0/1790890923 --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe1e8070ee0 con 0x7fe1e80cf280 2023-11-26T07:22:23.203 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:23.200+0000 7fe1ecc99640 1 --2- 172.21.15.143:0/1790890923 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fe1e80cf280 0x7fe1e80cf6e0 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7fe1d80097f0 tx=0x7fe1d801c350 comp rx=0 tx=0).ready entity=mon.2 client_cookie=19d37f36b13c470b server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:22:23.203 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:23.200+0000 7fe1dffff640 1 -- 172.21.15.143:0/1790890923 <== mon.2 v2:172.21.15.188:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe1d80ab600 con 0x7fe1e80cf280 2023-11-26T07:22:23.203 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:23.200+0000 7fe1dffff640 1 -- 172.21.15.143:0/1790890923 <== mon.2 v2:172.21.15.188:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe1d80a7050 con 0x7fe1e80cf280 2023-11-26T07:22:23.204 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:23.200+0000 7fe1dffff640 1 -- 172.21.15.143:0/1790890923 <== mon.2 v2:172.21.15.188:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe1d80ab980 con 0x7fe1e80cf280 2023-11-26T07:22:23.204 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:23.204+0000 7fe1ef6fe640 1 -- 172.21.15.143:0/1790890923 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fe1e80cf280 msgr2=0x7fe1e80cf6e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:22:23.204 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:23.204+0000 7fe1ef6fe640 1 --2- 172.21.15.143:0/1790890923 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fe1e80cf280 0x7fe1e80cf6e0 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7fe1d80097f0 tx=0x7fe1d801c350 comp rx=0 tx=0).stop 2023-11-26T07:22:23.204 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:23.204+0000 7fe1ef6fe640 1 -- 172.21.15.143:0/1790890923 shutdown_connections 2023-11-26T07:22:23.205 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:23.204+0000 7fe1ef6fe640 1 --2- 172.21.15.143:0/1790890923 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fe1e80cf280 0x7fe1e80cf6e0 unknown :-1 s=CLOSED pgs=114 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:22:23.205 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:23.204+0000 7fe1ef6fe640 1 -- 172.21.15.143:0/1790890923 >> 172.21.15.143:0/1790890923 conn(0x7fe1e806a890 msgr2=0x7fe1e806aca0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:22:23.205 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:23.204+0000 7fe1ef6fe640 1 -- 172.21.15.143:0/1790890923 shutdown_connections 2023-11-26T07:22:23.205 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:23.204+0000 7fe1ef6fe640 1 -- 172.21.15.143:0/1790890923 wait complete. 2023-11-26T07:22:23.206 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:23.204+0000 7fe1ef6fe640 1 Processor -- start 2023-11-26T07:22:23.206 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:23.204+0000 7fe1ef6fe640 1 -- start start 2023-11-26T07:22:23.208 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:23.204+0000 7fe1ef6fe640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fe1e80cd1b0 0x7fe1e81599b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:22:23.208 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:23.204+0000 7fe1ef6fe640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fe1e80cf280 0x7fe1e80ad2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:22:23.208 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:23.204+0000 7fe1ef6fe640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fe1e8143ac0 0x7fe1e806c550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:22:23.208 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:23.204+0000 7fe1ef6fe640 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe1e8138ae0 con 0x7fe1e80cf280 2023-11-26T07:22:23.208 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:23.204+0000 7fe1ef6fe640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe1e8138cc0 con 0x7fe1e8143ac0 2023-11-26T07:22:23.208 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:23.204+0000 7fe1ef6fe640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe1e8138fa0 con 0x7fe1e80cd1b0 2023-11-26T07:22:23.209 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:23.204+0000 7fe1ecc99640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fe1e80cf280 0x7fe1e80ad2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:22:23.209 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:23.204+0000 7fe1ecc99640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fe1e80cf280 0x7fe1e80ad2f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:34386/0 (socket says 172.21.15.143:34386) 2023-11-26T07:22:23.209 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:23.204+0000 7fe1ecc99640 1 -- 172.21.15.143:0/43964647 learned_addr learned my addr 172.21.15.143:0/43964647 (peer_addr_for_me v2:172.21.15.143:0/0) 2023-11-26T07:22:23.209 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:23.204+0000 7fe1ed49a640 1 --2- 172.21.15.143:0/43964647 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fe1e80cd1b0 0x7fe1e81599b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:22:23.209 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:23.204+0000 7fe1edc9b640 1 --2- 172.21.15.143:0/43964647 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fe1e8143ac0 0x7fe1e806c550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:22:23.209 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:23.204+0000 7fe1ecc99640 1 -- 172.21.15.143:0/43964647 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fe1e80cd1b0 msgr2=0x7fe1e81599b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:22:23.209 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:23.204+0000 7fe1ecc99640 1 --2- 172.21.15.143:0/43964647 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fe1e80cd1b0 0x7fe1e81599b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:22:23.209 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:23.204+0000 7fe1ecc99640 1 -- 172.21.15.143:0/43964647 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fe1e8143ac0 msgr2=0x7fe1e806c550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:22:23.209 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:23.204+0000 7fe1ecc99640 1 --2- 172.21.15.143:0/43964647 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fe1e8143ac0 0x7fe1e806c550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:22:23.209 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:23.204+0000 7fe1ecc99640 1 -- 172.21.15.143:0/43964647 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe1d8009420 con 0x7fe1e80cf280 2023-11-26T07:22:23.209 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:23.208+0000 7fe1ecc99640 1 --2- 172.21.15.143:0/43964647 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fe1e80cf280 0x7fe1e80ad2f0 secure :-1 s=READY pgs=888 cs=0 l=1 rev1=1 crypto rx=0x7fe1d8009740 tx=0x7fe1d801ab90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:22:23.209 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:23.208+0000 7fe1ddffb640 1 -- 172.21.15.143:0/43964647 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe1d8004220 con 0x7fe1e80cf280 2023-11-26T07:22:23.210 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:23.208+0000 7fe1ddffb640 1 -- 172.21.15.143:0/43964647 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe1d80043a0 con 0x7fe1e80cf280 2023-11-26T07:22:23.210 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:23.208+0000 7fe1ddffb640 1 -- 172.21.15.143:0/43964647 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe1d80b6870 con 0x7fe1e80cf280 2023-11-26T07:22:23.210 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:23.208+0000 7fe1ef6fe640 1 -- 172.21.15.143:0/43964647 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe1e8139280 con 0x7fe1e80cf280 2023-11-26T07:22:23.210 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:23.208+0000 7fe1ef6fe640 1 -- 172.21.15.143:0/43964647 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe1e8139840 con 0x7fe1e80cf280 2023-11-26T07:22:23.210 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:23.208+0000 7fe1ef6fe640 1 -- 172.21.15.143:0/43964647 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe1e8139a30 con 0x7fe1e80cf280 2023-11-26T07:22:23.211 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:23.208+0000 7fe1ddffb640 1 -- 172.21.15.143:0/43964647 <== mon.0 v2:172.21.15.143:3300/0 4 ==== mgrmap(e 11) v1 ==== 86158+0+0 (secure 0 0 0) 0x7fe1d80b69d0 con 0x7fe1e80cf280 2023-11-26T07:22:23.212 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:23.212+0000 7fe1ddffb640 1 --2- 172.21.15.143:0/43964647 >> [v2:172.21.15.143:6832/2150557544,v1:172.21.15.143:6833/2150557544] conn(0x7fe1cc0667d0 0x7fe1cc068c90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:22:23.212 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:23.212+0000 7fe1ddffb640 1 -- 172.21.15.143:0/43964647 <== mon.0 v2:172.21.15.143:3300/0 5 ==== osd_map(59..59 src has 1..59) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fe1d811eed0 con 0x7fe1e80cf280 2023-11-26T07:22:23.212 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:23.212+0000 7fe1ed49a640 1 --2- 172.21.15.143:0/43964647 >> [v2:172.21.15.143:6832/2150557544,v1:172.21.15.143:6833/2150557544] conn(0x7fe1cc0667d0 0x7fe1cc068c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:22:23.212 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:23.212+0000 7fe1ed49a640 1 --2- 172.21.15.143:0/43964647 >> [v2:172.21.15.143:6832/2150557544,v1:172.21.15.143:6833/2150557544] conn(0x7fe1cc0667d0 0x7fe1cc068c90 secure :-1 s=READY pgs=417 cs=0 l=1 rev1=1 crypto rx=0x7fe1e4000f70 tx=0x7fe1e4007220 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:22:23.215 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:23.216+0000 7fe1ddffb640 1 -- 172.21.15.143:0/43964647 <== mon.0 v2:172.21.15.143:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172649 (secure 0 0 0) 0x7fe1d80ebe60 con 0x7fe1e80cf280 2023-11-26T07:22:23.366 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:23.364+0000 7fe1ef6fe640 1 -- 172.21.15.143:0/43964647 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fe1e8067680 con 0x7fe1e80cf280 2023-11-26T07:22:23.367 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:23.364+0000 7fe1ddffb640 1 -- 172.21.15.143:0/43964647 <== mon.0 v2:172.21.15.143:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v59) v1 ==== 74+0+16601 (secure 0 0 0) 0x7fe1d80ba090 con 0x7fe1e80cf280 2023-11-26T07:22:23.368 INFO:teuthology.orchestra.run.smithi143.stdout: 2023-11-26T07:22:23.368 INFO:teuthology.orchestra.run.smithi143.stdout:{"epoch":59,"fsid":"f5033060-2469-4a07-95fe-2b76f5404bc1","created":"2023-11-26T07:17:28.613663+0000","modified":"2023-11-26T07:21:57.286555+0000","last_up_change":"2023-11-26T07:17:41.157813+0000","last_in_change":"2023-11-26T07:17:31.181658+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":9,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-11-26T07:17:41.758597+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":8,"pool_name":"cephfs_metadata","create_time":"2023-11-26T07:21:53.382407+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"58","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":9,"pool_name":"cephfs_data","create_time":"2023-11-26T07:21:54.656298+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"59","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.6299999952316284,"score_stable":1.6299999952316284,"optimal_score":1,"raw_score_acting":1.6299999952316284,"raw_score_stable":1.6299999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"ebbee2c6-023e-44b9-9f1d-6f7bfd309730","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":57,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6817","nonce":639807369},{"type":"v1","addr":"172.21.15.143:6819","nonce":639807369}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6821","nonce":639807369},{"type":"v1","addr":"172.21.15.143:6823","nonce":639807369}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6830","nonce":639807369},{"type":"v1","addr":"172.21.15.143:6831","nonce":639807369}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6825","nonce":639807369},{"type":"v1","addr":"172.21.15.143:6828","nonce":639807369}]},"public_addr":"172.21.15.143:6819/639807369","cluster_addr":"172.21.15.143:6823/639807369","heartbeat_back_addr":"172.21.15.143:6831/639807369","heartbeat_front_addr":"172.21.15.143:6828/639807369","state":["exists","up"]},{"osd":1,"uuid":"fec4b8be-7122-49e9-9a19-5850a0700001","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":57,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6816","nonce":1144942530},{"type":"v1","addr":"172.21.15.143:6818","nonce":1144942530}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6820","nonce":1144942530},{"type":"v1","addr":"172.21.15.143:6822","nonce":1144942530}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6827","nonce":1144942530},{"type":"v1","addr":"172.21.15.143:6829","nonce":1144942530}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6824","nonce":1144942530},{"type":"v1","addr":"172.21.15.143:6826","nonce":1144942530}]},"public_addr":"172.21.15.143:6818/1144942530","cluster_addr":"172.21.15.143:6822/1144942530","heartbeat_back_addr":"172.21.15.143:6829/1144942530","heartbeat_front_addr":"172.21.15.143:6826/1144942530","state":["exists","up"]},{"osd":2,"uuid":"2888af5b-f933-46ca-8f95-6dd494ff43f6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":57,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6808","nonce":3909095186},{"type":"v1","addr":"172.21.15.143:6809","nonce":3909095186}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6810","nonce":3909095186},{"type":"v1","addr":"172.21.15.143:6811","nonce":3909095186}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6814","nonce":3909095186},{"type":"v1","addr":"172.21.15.143:6815","nonce":3909095186}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6812","nonce":3909095186},{"type":"v1","addr":"172.21.15.143:6813","nonce":3909095186}]},"public_addr":"172.21.15.143:6809/3909095186","cluster_addr":"172.21.15.143:6811/3909095186","heartbeat_back_addr":"172.21.15.143:6815/3909095186","heartbeat_front_addr":"172.21.15.143:6813/3909095186","state":["exists","up"]},{"osd":3,"uuid":"d1f52435-7bc1-4081-9550-9cd1ba55f549","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":57,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6800","nonce":3854210991},{"type":"v1","addr":"172.21.15.143:6801","nonce":3854210991}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6802","nonce":3854210991},{"type":"v1","addr":"172.21.15.143:6803","nonce":3854210991}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6806","nonce":3854210991},{"type":"v1","addr":"172.21.15.143:6807","nonce":3854210991}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6804","nonce":3854210991},{"type":"v1","addr":"172.21.15.143:6805","nonce":3854210991}]},"public_addr":"172.21.15.143:6801/3854210991","cluster_addr":"172.21.15.143:6803/3854210991","heartbeat_back_addr":"172.21.15.143:6807/3854210991","heartbeat_front_addr":"172.21.15.143:6805/3854210991","state":["exists","up"]},{"osd":4,"uuid":"e2ae8b6e-6e25-49b2-bcbf-aacce68f06a3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":57,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6800","nonce":3924606902},{"type":"v1","addr":"172.21.15.188:6801","nonce":3924606902}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6802","nonce":3924606902},{"type":"v1","addr":"172.21.15.188:6803","nonce":3924606902}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6806","nonce":3924606902},{"type":"v1","addr":"172.21.15.188:6807","nonce":3924606902}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6804","nonce":3924606902},{"type":"v1","addr":"172.21.15.188:6805","nonce":3924606902}]},"public_addr":"172.21.15.188:6801/3924606902","cluster_addr":"172.21.15.188:6803/3924606902","heartbeat_back_addr":"172.21.15.188:6807/3924606902","heartbeat_front_addr":"172.21.15.188:6805/3924606902","state":["exists","up"]},{"osd":5,"uuid":"3cec8a32-7698-4c36-8e89-e4ba11934f33","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":57,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6813","nonce":186841544},{"type":"v1","addr":"172.21.15.188:6815","nonce":186841544}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6817","nonce":186841544},{"type":"v1","addr":"172.21.15.188:6818","nonce":186841544}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6822","nonce":186841544},{"type":"v1","addr":"172.21.15.188:6823","nonce":186841544}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6820","nonce":186841544},{"type":"v1","addr":"172.21.15.188:6821","nonce":186841544}]},"public_addr":"172.21.15.188:6815/186841544","cluster_addr":"172.21.15.188:6818/186841544","heartbeat_back_addr":"172.21.15.188:6823/186841544","heartbeat_front_addr":"172.21.15.188:6821/186841544","state":["exists","up"]},{"osd":6,"uuid":"58a65d5d-e090-473a-81b4-ad38e85029f9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":57,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6808","nonce":2153499405},{"type":"v1","addr":"172.21.15.188:6809","nonce":2153499405}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6810","nonce":2153499405},{"type":"v1","addr":"172.21.15.188:6811","nonce":2153499405}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6816","nonce":2153499405},{"type":"v1","addr":"172.21.15.188:6819","nonce":2153499405}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6812","nonce":2153499405},{"type":"v1","addr":"172.21.15.188:6814","nonce":2153499405}]},"public_addr":"172.21.15.188:6809/2153499405","cluster_addr":"172.21.15.188:6811/2153499405","heartbeat_back_addr":"172.21.15.188:6819/2153499405","heartbeat_front_addr":"172.21.15.188:6814/2153499405","state":["exists","up"]},{"osd":7,"uuid":"ea5b7f70-3554-4cf8-ac30-f7c59b80678d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":57,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6824","nonce":2722096729},{"type":"v1","addr":"172.21.15.188:6825","nonce":2722096729}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6826","nonce":2722096729},{"type":"v1","addr":"172.21.15.188:6827","nonce":2722096729}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6830","nonce":2722096729},{"type":"v1","addr":"172.21.15.188:6831","nonce":2722096729}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6828","nonce":2722096729},{"type":"v1","addr":"172.21.15.188:6829","nonce":2722096729}]},"public_addr":"172.21.15.188:6825/2722096729","cluster_addr":"172.21.15.188:6827/2722096729","heartbeat_back_addr":"172.21.15.188:6831/2722096729","heartbeat_front_addr":"172.21.15.188:6829/2722096729","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-26T07:17:36.995032+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-26T07:17:36.932266+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-26T07:17:36.829304+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-26T07:17:36.804297+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-26T07:17:36.667892+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-26T07:17:36.964868+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-26T07:17:36.827332+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-26T07:17:36.893179+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-11-26T07:22:23.370 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:23.368+0000 7fe1ef6fe640 1 -- 172.21.15.143:0/43964647 >> [v2:172.21.15.143:6832/2150557544,v1:172.21.15.143:6833/2150557544] conn(0x7fe1cc0667d0 msgr2=0x7fe1cc068c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:22:23.370 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:23.368+0000 7fe1ef6fe640 1 --2- 172.21.15.143:0/43964647 >> [v2:172.21.15.143:6832/2150557544,v1:172.21.15.143:6833/2150557544] conn(0x7fe1cc0667d0 0x7fe1cc068c90 secure :-1 s=READY pgs=417 cs=0 l=1 rev1=1 crypto rx=0x7fe1e4000f70 tx=0x7fe1e4007220 comp rx=0 tx=0).stop 2023-11-26T07:22:23.370 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:23.368+0000 7fe1ef6fe640 1 -- 172.21.15.143:0/43964647 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fe1e80cf280 msgr2=0x7fe1e80ad2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:22:23.370 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:23.368+0000 7fe1ef6fe640 1 --2- 172.21.15.143:0/43964647 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fe1e80cf280 0x7fe1e80ad2f0 secure :-1 s=READY pgs=888 cs=0 l=1 rev1=1 crypto rx=0x7fe1d8009740 tx=0x7fe1d801ab90 comp rx=0 tx=0).stop 2023-11-26T07:22:23.370 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:23.368+0000 7fe1ef6fe640 1 -- 172.21.15.143:0/43964647 shutdown_connections 2023-11-26T07:22:23.370 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:23.368+0000 7fe1ef6fe640 1 --2- 172.21.15.143:0/43964647 >> [v2:172.21.15.143:6832/2150557544,v1:172.21.15.143:6833/2150557544] conn(0x7fe1cc0667d0 0x7fe1cc068c90 unknown :-1 s=CLOSED pgs=417 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:22:23.370 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:23.368+0000 7fe1ef6fe640 1 --2- 172.21.15.143:0/43964647 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fe1e8143ac0 0x7fe1e806c550 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:22:23.370 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:23.368+0000 7fe1ef6fe640 1 --2- 172.21.15.143:0/43964647 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fe1e80cf280 0x7fe1e80ad2f0 unknown :-1 s=CLOSED pgs=888 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:22:23.370 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:23.368+0000 7fe1ef6fe640 1 --2- 172.21.15.143:0/43964647 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fe1e80cd1b0 0x7fe1e81599b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:22:23.371 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:23.368+0000 7fe1ef6fe640 1 -- 172.21.15.143:0/43964647 >> 172.21.15.143:0/43964647 conn(0x7fe1e806a890 msgr2=0x7fe1e80ce710 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:22:23.371 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:23.368+0000 7fe1ef6fe640 1 -- 172.21.15.143:0/43964647 shutdown_connections 2023-11-26T07:22:23.371 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:23.368+0000 7fe1ef6fe640 1 -- 172.21.15.143:0/43964647 wait complete. 2023-11-26T07:22:23.389 INFO:tasks.cephfs.filesystem:Destroying file system cephfs and related pools 2023-11-26T07:22:23.389 DEBUG:teuthology.orchestra.run.smithi143:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-11-26T07:22:23.528 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:23.528+0000 7f1866ebb640 1 Processor -- start 2023-11-26T07:22:23.529 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:23.528+0000 7f1866ebb640 1 -- start start 2023-11-26T07:22:23.530 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:23.528+0000 7f1866ebb640 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7f186015d6e0 0x7f18600d0df0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:22:23.530 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:23.528+0000 7f1866ebb640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f18600b2160 0x7f18600d14b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:22:23.530 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:23.528+0000 7f1866ebb640 1 -- --> v1:172.21.15.143:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f18600cf0e0 con 0x7f18600d19f0 2023-11-26T07:22:23.530 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:23.528+0000 7f1866ebb640 1 -- --> v2:172.21.15.143:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f186015dac0 con 0x7f186015d6e0 2023-11-26T07:22:23.530 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:23.528+0000 7f1866ebb640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f18600d1e80 con 0x7f18600b2160 2023-11-26T07:22:23.530 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:23.528+0000 7f1864c57640 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7f186015d6e0 0x7f18600d0df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:22:23.530 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:23.528+0000 7f1865458640 1 --1- >> v1:172.21.15.143:6789/0 conn(0x7f18600d19f0 0x7f18600cfbf0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.143:6789/0 says I am v1:172.21.15.143:59280/0 (socket says 172.21.15.143:59280) 2023-11-26T07:22:23.530 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:23.528+0000 7f1865458640 1 -- 172.21.15.143:0/468725891 learned_addr learned my addr 172.21.15.143:0/468725891 (peer_addr_for_me v1:172.21.15.143:0/0) 2023-11-26T07:22:23.531 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:23.528+0000 7f185ffff640 1 --2- 172.21.15.143:0/468725891 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f18600b2160 0x7f18600d14b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:22:23.531 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:23.528+0000 7f1864c57640 1 -- 172.21.15.143:0/468725891 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f18600b2160 msgr2=0x7f18600d14b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-26T07:22:23.531 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:23.528+0000 7f1864c57640 1 --2- 172.21.15.143:0/468725891 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f18600b2160 0x7f18600d14b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:22:23.531 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:23.528+0000 7f1864c57640 1 -- 172.21.15.143:0/468725891 >> v1:172.21.15.143:6789/0 conn(0x7f18600d19f0 legacy=0x7f18600cfbf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:22:23.531 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:23.528+0000 7f1864c57640 1 -- 172.21.15.143:0/468725891 --> v2:172.21.15.143:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f18600ceb20 con 0x7f186015d6e0 2023-11-26T07:22:23.531 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:23.528+0000 7f185f7fe640 1 -- 172.21.15.143:0/468725891 <== mon.0 v1:172.21.15.143:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3309281677 0 0) 0x7f18600cf0e0 con 0x7f18600d19f0 2023-11-26T07:22:23.533 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:23.532+0000 7f1864c57640 1 --2- 172.21.15.143:0/468725891 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f186015d6e0 0x7f18600d0df0 secure :-1 s=READY pgs=890 cs=0 l=1 rev1=1 crypto rx=0x7f18540097c0 tx=0x7f185401c480 comp rx=0 tx=0).ready entity=mon.0 client_cookie=88a1860e264c41bd server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:22:23.533 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:23.532+0000 7f185f7fe640 1 -- 172.21.15.143:0/468725891 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f18540ab600 con 0x7f186015d6e0 2023-11-26T07:22:23.533 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:23.532+0000 7f185f7fe640 1 -- 172.21.15.143:0/468725891 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f18540a7050 con 0x7f186015d6e0 2023-11-26T07:22:23.534 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:23.532+0000 7f185f7fe640 1 -- 172.21.15.143:0/468725891 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f18540ab980 con 0x7f186015d6e0 2023-11-26T07:22:23.534 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:23.532+0000 7f1866ebb640 1 -- 172.21.15.143:0/468725891 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f186015d6e0 msgr2=0x7f18600d0df0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:22:23.535 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:23.532+0000 7f1866ebb640 1 --2- 172.21.15.143:0/468725891 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f186015d6e0 0x7f18600d0df0 secure :-1 s=READY pgs=890 cs=0 l=1 rev1=1 crypto rx=0x7f18540097c0 tx=0x7f185401c480 comp rx=0 tx=0).stop 2023-11-26T07:22:23.535 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:23.532+0000 7f1866ebb640 1 -- 172.21.15.143:0/468725891 shutdown_connections 2023-11-26T07:22:23.535 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:23.532+0000 7f1866ebb640 1 --2- 172.21.15.143:0/468725891 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f18600b2160 0x7f18600d14b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:22:23.535 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:23.532+0000 7f1866ebb640 1 --2- 172.21.15.143:0/468725891 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f186015d6e0 0x7f18600d0df0 unknown :-1 s=CLOSED pgs=890 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:22:23.535 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:23.532+0000 7f1866ebb640 1 -- 172.21.15.143:0/468725891 >> 172.21.15.143:0/468725891 conn(0x7f186009c550 msgr2=0x7f18600d3220 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:22:23.535 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:23.532+0000 7f1866ebb640 1 -- 172.21.15.143:0/468725891 shutdown_connections 2023-11-26T07:22:23.535 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:23.532+0000 7f1866ebb640 1 -- 172.21.15.143:0/468725891 wait complete. 2023-11-26T07:22:23.536 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:23.536+0000 7f1866ebb640 1 Processor -- start 2023-11-26T07:22:23.536 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:23.536+0000 7f1866ebb640 1 -- start start 2023-11-26T07:22:23.537 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:23.536+0000 7f1866ebb640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f18600b2160 0x7f18601607c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:22:23.537 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:23.536+0000 7f1866ebb640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f18600d19f0 0x7f186014c0f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:22:23.537 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:23.536+0000 7f1866ebb640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f186015d6e0 0x7f1860159d70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:22:23.537 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:23.536+0000 7f1866ebb640 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f18600ccd40 con 0x7f18600d19f0 2023-11-26T07:22:23.537 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:23.536+0000 7f1866ebb640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1860160330 con 0x7f186015d6e0 2023-11-26T07:22:23.538 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:23.536+0000 7f1866ebb640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f186014c840 con 0x7f18600b2160 2023-11-26T07:22:23.538 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:23.536+0000 7f185ffff640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f18600d19f0 0x7f186014c0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:22:23.538 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:23.536+0000 7f185ffff640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f18600d19f0 0x7f186014c0f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:34408/0 (socket says 172.21.15.143:34408) 2023-11-26T07:22:23.538 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:23.536+0000 7f185ffff640 1 -- 172.21.15.143:0/765743475 learned_addr learned my addr 172.21.15.143:0/765743475 (peer_addr_for_me v2:172.21.15.143:0/0) 2023-11-26T07:22:23.538 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:23.536+0000 7f1864c57640 1 --2- 172.21.15.143:0/765743475 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f18600b2160 0x7f18601607c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:22:23.538 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:23.536+0000 7f1865458640 1 --2- 172.21.15.143:0/765743475 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f186015d6e0 0x7f1860159d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:22:23.539 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:23.536+0000 7f185ffff640 1 -- 172.21.15.143:0/765743475 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f18600b2160 msgr2=0x7f18601607c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:22:23.539 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:23.536+0000 7f185ffff640 1 --2- 172.21.15.143:0/765743475 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f18600b2160 0x7f18601607c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:22:23.539 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:23.536+0000 7f185ffff640 1 -- 172.21.15.143:0/765743475 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f186015d6e0 msgr2=0x7f1860159d70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:22:23.539 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:23.536+0000 7f185ffff640 1 --2- 172.21.15.143:0/765743475 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f186015d6e0 0x7f1860159d70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:22:23.540 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:23.536+0000 7f185ffff640 1 -- 172.21.15.143:0/765743475 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1854009420 con 0x7f18600d19f0 2023-11-26T07:22:23.540 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:23.536+0000 7f1865458640 1 --2- 172.21.15.143:0/765743475 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f186015d6e0 0x7f1860159d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-26T07:22:23.540 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:23.536+0000 7f185ffff640 1 --2- 172.21.15.143:0/765743475 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f18600d19f0 0x7f186014c0f0 secure :-1 s=READY pgs=891 cs=0 l=1 rev1=1 crypto rx=0x7f184c00d830 tx=0x7f184c00dcf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:22:23.540 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:23.536+0000 7f185d7fa640 1 -- 172.21.15.143:0/765743475 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f184c00b6c0 con 0x7f18600d19f0 2023-11-26T07:22:23.540 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:23.536+0000 7f185d7fa640 1 -- 172.21.15.143:0/765743475 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f184c00b820 con 0x7f18600d19f0 2023-11-26T07:22:23.540 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:23.536+0000 7f1866ebb640 1 -- 172.21.15.143:0/765743475 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f186015fd40 con 0x7f18600d19f0 2023-11-26T07:22:23.540 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:23.536+0000 7f185d7fa640 1 -- 172.21.15.143:0/765743475 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f184c00bb90 con 0x7f18600d19f0 2023-11-26T07:22:23.540 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:23.536+0000 7f1866ebb640 1 -- 172.21.15.143:0/765743475 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f18600cd270 con 0x7f18600d19f0 2023-11-26T07:22:23.540 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:23.536+0000 7f1866ebb640 1 -- 172.21.15.143:0/765743475 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f18600cd490 con 0x7f18600d19f0 2023-11-26T07:22:23.541 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:23.540+0000 7f185d7fa640 1 -- 172.21.15.143:0/765743475 <== mon.0 v2:172.21.15.143:3300/0 4 ==== mgrmap(e 11) v1 ==== 86158+0+0 (secure 0 0 0) 0x7f184c01e460 con 0x7f18600d19f0 2023-11-26T07:22:23.541 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:23.540+0000 7f185d7fa640 1 --2- 172.21.15.143:0/765743475 >> [v2:172.21.15.143:6832/2150557544,v1:172.21.15.143:6833/2150557544] conn(0x7f1844066800 0x7f1844068cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:22:23.541 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:23.540+0000 7f185d7fa640 1 -- 172.21.15.143:0/765743475 <== mon.0 v2:172.21.15.143:3300/0 5 ==== osd_map(59..59 src has 1..59) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f184c084b60 con 0x7f18600d19f0 2023-11-26T07:22:23.541 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:23.540+0000 7f1864c57640 1 --2- 172.21.15.143:0/765743475 >> [v2:172.21.15.143:6832/2150557544,v1:172.21.15.143:6833/2150557544] conn(0x7f1844066800 0x7f1844068cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:22:23.542 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:23.540+0000 7f1864c57640 1 --2- 172.21.15.143:0/765743475 >> [v2:172.21.15.143:6832/2150557544,v1:172.21.15.143:6833/2150557544] conn(0x7f1844066800 0x7f1844068cc0 secure :-1 s=READY pgs=418 cs=0 l=1 rev1=1 crypto rx=0x7f18540ad000 tx=0x7f1854008040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:22:23.545 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:23.544+0000 7f185d7fa640 1 -- 172.21.15.143:0/765743475 <== mon.0 v2:172.21.15.143:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172649 (secure 0 0 0) 0x7f184c051b90 con 0x7f18600d19f0 2023-11-26T07:22:23.729 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:23.728+0000 7f1866ebb640 1 -- 172.21.15.143:0/765743475 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f1860067680 con 0x7f18600d19f0 2023-11-26T07:22:23.730 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:23.728+0000 7f185d7fa640 1 -- 172.21.15.143:0/765743475 <== mon.0 v2:172.21.15.143:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 35 v35) v1 ==== 94+0+4925 (secure 0 0 0) 0x7f184c0551b0 con 0x7f18600d19f0 2023-11-26T07:22:23.730 INFO:teuthology.orchestra.run.smithi143.stdout: 2023-11-26T07:22:23.730 INFO:teuthology.orchestra.run.smithi143.stdout:{"epoch":35,"default_fscid":4,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5612,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.143:6837/1372238588","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6836","nonce":1372238588},{"type":"v1","addr":"172.21.15.143:6837","nonce":1372238588}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":29},{"gid":6103,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.143:6835/1078738477","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6834","nonce":1078738477},{"type":"v1","addr":"172.21.15.143:6835","nonce":1078738477}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":29},{"gid":6118,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.188:6834/1634083777","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6832","nonce":1634083777},{"type":"v1","addr":"172.21.15.188:6834","nonce":1634083777}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":29}],"filesystems":[{"mdsmap":{"epoch":34,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-11-26T07:21:55.656168+0000","modified":"2023-11-26T07:21:58.674916+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":6121},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6121":{"gid":6121,"name":"b","rank":0,"incarnation":30,"state":"up:active","state_seq":3,"addr":"172.21.15.188:6835/1958716738","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6833","nonce":1958716738},{"type":"v1","addr":"172.21.15.188:6835","nonce":1958716738}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[9],"metadata_pool":8,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":4}]} 2023-11-26T07:22:23.730 INFO:teuthology.orchestra.run.smithi143.stderr:dumped fsmap epoch 35 2023-11-26T07:22:23.731 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:23.728+0000 7f1866ebb640 1 -- 172.21.15.143:0/765743475 >> [v2:172.21.15.143:6832/2150557544,v1:172.21.15.143:6833/2150557544] conn(0x7f1844066800 msgr2=0x7f1844068cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:22:23.731 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:23.728+0000 7f1866ebb640 1 --2- 172.21.15.143:0/765743475 >> [v2:172.21.15.143:6832/2150557544,v1:172.21.15.143:6833/2150557544] conn(0x7f1844066800 0x7f1844068cc0 secure :-1 s=READY pgs=418 cs=0 l=1 rev1=1 crypto rx=0x7f18540ad000 tx=0x7f1854008040 comp rx=0 tx=0).stop 2023-11-26T07:22:23.731 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:23.728+0000 7f1866ebb640 1 -- 172.21.15.143:0/765743475 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f18600d19f0 msgr2=0x7f186014c0f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:22:23.731 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:23.728+0000 7f1866ebb640 1 --2- 172.21.15.143:0/765743475 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f18600d19f0 0x7f186014c0f0 secure :-1 s=READY pgs=891 cs=0 l=1 rev1=1 crypto rx=0x7f184c00d830 tx=0x7f184c00dcf0 comp rx=0 tx=0).stop 2023-11-26T07:22:23.732 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:23.728+0000 7f1866ebb640 1 -- 172.21.15.143:0/765743475 shutdown_connections 2023-11-26T07:22:23.732 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:23.728+0000 7f1866ebb640 1 --2- 172.21.15.143:0/765743475 >> [v2:172.21.15.143:6832/2150557544,v1:172.21.15.143:6833/2150557544] conn(0x7f1844066800 0x7f1844068cc0 unknown :-1 s=CLOSED pgs=418 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:22:23.732 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:23.728+0000 7f1866ebb640 1 --2- 172.21.15.143:0/765743475 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f186015d6e0 0x7f1860159d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:22:23.732 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:23.728+0000 7f1866ebb640 1 --2- 172.21.15.143:0/765743475 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f18600d19f0 0x7f186014c0f0 unknown :-1 s=CLOSED pgs=891 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:22:23.732 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:23.728+0000 7f1866ebb640 1 --2- 172.21.15.143:0/765743475 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f18600b2160 0x7f18601607c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:22:23.733 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:23.728+0000 7f1866ebb640 1 -- 172.21.15.143:0/765743475 >> 172.21.15.143:0/765743475 conn(0x7f186009c550 msgr2=0x7f18601560f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:22:23.733 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:23.732+0000 7f1866ebb640 1 -- 172.21.15.143:0/765743475 shutdown_connections 2023-11-26T07:22:23.733 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:23.732+0000 7f1866ebb640 1 -- 172.21.15.143:0/765743475 wait complete. 2023-11-26T07:22:23.752 DEBUG:teuthology.orchestra.run.smithi143:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-11-26T07:22:23.894 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:23.892+0000 7fb6cdd80640 1 Processor -- start 2023-11-26T07:22:23.894 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:23.892+0000 7fb6cdd80640 1 -- start start 2023-11-26T07:22:23.896 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:23.892+0000 7fb6cdd80640 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7fb6c815d6e0 0x7fb6c80d0df0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:22:23.896 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:23.892+0000 7fb6cdd80640 1 -- --> v1:172.21.15.143:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb6c80cf0e0 con 0x7fb6c80d1bc0 2023-11-26T07:22:23.896 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:23.892+0000 7fb6cdd80640 1 -- --> v1:172.21.15.188:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb6c80ceb20 con 0x7fb6c80b2160 2023-11-26T07:22:23.896 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:23.892+0000 7fb6cdd80640 1 -- --> v2:172.21.15.188:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb6c815dac0 con 0x7fb6c815d6e0 2023-11-26T07:22:23.897 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:23.892+0000 7fb6c7fff640 1 --1- >> v1:172.21.15.143:6789/0 conn(0x7fb6c80d1bc0 0x7fb6c80b0750 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.143:6789/0 says I am v1:172.21.15.143:59288/0 (socket says 172.21.15.143:59288) 2023-11-26T07:22:23.897 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:23.892+0000 7fb6c7fff640 1 -- 172.21.15.143:0/3790859246 learned_addr learned my addr 172.21.15.143:0/3790859246 (peer_addr_for_me v1:172.21.15.143:0/0) 2023-11-26T07:22:23.897 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:23.892+0000 7fb6c77fe640 1 --2- 172.21.15.143:0/3790859246 >> v2:172.21.15.188:3300/0 conn(0x7fb6c815d6e0 0x7fb6c80d0df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:22:23.897 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:23.892+0000 7fb6c67fc640 1 -- 172.21.15.143:0/3790859246 <== mon.0 v1:172.21.15.143:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2132637956 0 0) 0x7fb6c80cf0e0 con 0x7fb6c80d1bc0 2023-11-26T07:22:23.898 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:23.892+0000 7fb6c67fc640 1 -- 172.21.15.143:0/3790859246 --> v1:172.21.15.143:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fb6b0003530 con 0x7fb6c80d1bc0 2023-11-26T07:22:23.898 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:23.892+0000 7fb6c67fc640 1 -- 172.21.15.143:0/3790859246 <== mon.0 v1:172.21.15.143:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 4088754743 0 0) 0x7fb6b0003530 con 0x7fb6c80d1bc0 2023-11-26T07:22:23.898 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:23.892+0000 7fb6c67fc640 1 -- 172.21.15.143:0/3790859246 >> v2:172.21.15.188:3300/0 conn(0x7fb6c815d6e0 msgr2=0x7fb6c80d0df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-26T07:22:23.898 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:23.892+0000 7fb6c67fc640 1 --2- 172.21.15.143:0/3790859246 >> v2:172.21.15.188:3300/0 conn(0x7fb6c815d6e0 0x7fb6c80d0df0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:22:23.899 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:23.892+0000 7fb6c67fc640 1 -- 172.21.15.143:0/3790859246 >> v1:172.21.15.188:6789/0 conn(0x7fb6c80b2160 legacy=0x7fb6c80d14b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:22:23.899 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:23.892+0000 7fb6c67fc640 1 -- 172.21.15.143:0/3790859246 --> v1:172.21.15.143:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb6c80cee00 con 0x7fb6c80d1bc0 2023-11-26T07:22:23.900 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:23.892+0000 7fb6c67fc640 1 -- 172.21.15.143:0/3790859246 <== mon.0 v1:172.21.15.143:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2137882523 0 0) 0x7fb6c0001a00 con 0x7fb6c80d1bc0 2023-11-26T07:22:23.900 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:23.892+0000 7fb6c67fc640 1 -- 172.21.15.143:0/3790859246 >> v1:172.21.15.143:6789/0 conn(0x7fb6c80d1bc0 legacy=0x7fb6c80b0750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:22:23.900 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:23.892+0000 7fb6c67fc640 1 --2- 172.21.15.143:0/3790859246 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fb6b0003980 0x7fb6b0003d60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:22:23.900 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:23.892+0000 7fb6c67fc640 1 --2- 172.21.15.143:0/3790859246 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fb6b0004380 0x7fb6b0004850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:22:23.900 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:23.892+0000 7fb6c67fc640 1 --2- 172.21.15.143:0/3790859246 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fb6b0005050 0x7fb6b000b990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:22:23.900 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:23.892+0000 7fb6c67fc640 1 -- 172.21.15.143:0/3790859246 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb6c815dac0 con 0x7fb6b0003980 2023-11-26T07:22:23.900 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:23.892+0000 7fb6c67fc640 1 -- 172.21.15.143:0/3790859246 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb6b000bfb0 con 0x7fb6b0004380 2023-11-26T07:22:23.900 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:23.892+0000 7fb6c67fc640 1 -- 172.21.15.143:0/3790859246 --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb6b000c170 con 0x7fb6b0005050 2023-11-26T07:22:23.900 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:23.892+0000 7fb6c77fe640 1 --2- 172.21.15.143:0/3790859246 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fb6b0003980 0x7fb6b0003d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:22:23.900 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:23.892+0000 7fb6c6ffd640 1 --2- 172.21.15.143:0/3790859246 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fb6b0004380 0x7fb6b0004850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:22:23.900 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:23.892+0000 7fb6c7fff640 1 --2- 172.21.15.143:0/3790859246 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fb6b0005050 0x7fb6b000b990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:22:23.900 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:23.896+0000 7fb6c77fe640 1 -- 172.21.15.143:0/3790859246 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fb6b0005050 msgr2=0x7fb6b000b990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-26T07:22:23.900 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:23.896+0000 7fb6c77fe640 1 --2- 172.21.15.143:0/3790859246 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fb6b0005050 0x7fb6b000b990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:22:23.901 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:23.896+0000 7fb6c77fe640 1 -- 172.21.15.143:0/3790859246 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fb6b0004380 msgr2=0x7fb6b0004850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-26T07:22:23.901 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:23.896+0000 7fb6c77fe640 1 --2- 172.21.15.143:0/3790859246 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fb6b0004380 0x7fb6b0004850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:22:23.901 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:23.896+0000 7fb6c77fe640 1 -- 172.21.15.143:0/3790859246 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb6b0003530 con 0x7fb6b0003980 2023-11-26T07:22:23.901 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:23.896+0000 7fb6c7fff640 1 --2- 172.21.15.143:0/3790859246 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fb6b0005050 0x7fb6b000b990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-26T07:22:23.901 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:23.896+0000 7fb6c7fff640 1 -- 172.21.15.143:0/3790859246 reap_dead start 2023-11-26T07:22:23.903 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:23.896+0000 7fb6c77fe640 1 --2- 172.21.15.143:0/3790859246 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fb6b0003980 0x7fb6b0003d60 secure :-1 s=READY pgs=893 cs=0 l=1 rev1=1 crypto rx=0x7fb6b8009ae0 tx=0x7fb6b801ca50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=db7cc7c04c4cb0c6 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:22:23.903 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:23.896+0000 7fb6c67fc640 1 -- 172.21.15.143:0/3790859246 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb6b80a5640 con 0x7fb6b0003980 2023-11-26T07:22:23.903 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:23.896+0000 7fb6c67fc640 1 -- 172.21.15.143:0/3790859246 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb6b80a57a0 con 0x7fb6b0003980 2023-11-26T07:22:23.903 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:23.896+0000 7fb6c67fc640 1 -- 172.21.15.143:0/3790859246 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb6b80a5b50 con 0x7fb6b0003980 2023-11-26T07:22:23.904 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:23.896+0000 7fb6cdd80640 1 -- 172.21.15.143:0/3790859246 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fb6b0003980 msgr2=0x7fb6b0003d60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:22:23.904 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:23.896+0000 7fb6cdd80640 1 --2- 172.21.15.143:0/3790859246 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fb6b0003980 0x7fb6b0003d60 secure :-1 s=READY pgs=893 cs=0 l=1 rev1=1 crypto rx=0x7fb6b8009ae0 tx=0x7fb6b801ca50 comp rx=0 tx=0).stop 2023-11-26T07:22:23.904 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:23.900+0000 7fb6cdd80640 1 -- 172.21.15.143:0/3790859246 shutdown_connections 2023-11-26T07:22:23.904 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:23.900+0000 7fb6cdd80640 1 --2- 172.21.15.143:0/3790859246 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fb6b0003980 0x7fb6b0003d60 unknown :-1 s=CLOSED pgs=893 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:22:23.904 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:23.900+0000 7fb6cdd80640 1 -- 172.21.15.143:0/3790859246 >> 172.21.15.143:0/3790859246 conn(0x7fb6c809c550 msgr2=0x7fb6c80d3220 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:22:23.905 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:23.900+0000 7fb6cdd80640 1 -- 172.21.15.143:0/3790859246 shutdown_connections 2023-11-26T07:22:23.905 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:23.900+0000 7fb6cdd80640 1 -- 172.21.15.143:0/3790859246 wait complete. 2023-11-26T07:22:23.905 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:23.900+0000 7fb6cdd80640 1 Processor -- start 2023-11-26T07:22:23.906 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:23.900+0000 7fb6cdd80640 1 -- start start 2023-11-26T07:22:23.907 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:23.900+0000 7fb6cdd80640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fb6b0003980 0x7fb6c8151540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:22:23.907 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:23.900+0000 7fb6cdd80640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fb6c81f2f00 0x7fb6c80ccf50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:22:23.907 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:23.900+0000 7fb6cdd80640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fb6c80b37e0 0x7fb6c815f270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:22:23.907 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:23.900+0000 7fb6cdd80640 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb6c81f34c0 con 0x7fb6c81f2f00 2023-11-26T07:22:23.907 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:23.900+0000 7fb6cdd80640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb6c81f37a0 con 0x7fb6b0003980 2023-11-26T07:22:23.907 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:23.900+0000 7fb6cdd80640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb6c81f3a80 con 0x7fb6c80b37e0 2023-11-26T07:22:23.908 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:23.900+0000 7fb6c6ffd640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fb6c81f2f00 0x7fb6c80ccf50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:22:23.908 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:23.900+0000 7fb6c6ffd640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fb6c81f2f00 0x7fb6c80ccf50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:34428/0 (socket says 172.21.15.143:34428) 2023-11-26T07:22:23.909 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:23.900+0000 7fb6c6ffd640 1 -- 172.21.15.143:0/1938290529 learned_addr learned my addr 172.21.15.143:0/1938290529 (peer_addr_for_me v2:172.21.15.143:0/0) 2023-11-26T07:22:23.909 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:23.900+0000 7fb6c77fe640 1 --2- 172.21.15.143:0/1938290529 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fb6b0003980 0x7fb6c8151540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:22:23.909 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:23.900+0000 7fb6c7fff640 1 --2- 172.21.15.143:0/1938290529 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fb6c80b37e0 0x7fb6c815f270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:22:23.909 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:23.900+0000 7fb6c6ffd640 1 -- 172.21.15.143:0/1938290529 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fb6c80b37e0 msgr2=0x7fb6c815f270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:22:23.909 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:23.900+0000 7fb6c6ffd640 1 --2- 172.21.15.143:0/1938290529 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fb6c80b37e0 0x7fb6c815f270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:22:23.909 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:23.900+0000 7fb6c6ffd640 1 -- 172.21.15.143:0/1938290529 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fb6b0003980 msgr2=0x7fb6c8151540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:22:23.909 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:23.900+0000 7fb6c6ffd640 1 --2- 172.21.15.143:0/1938290529 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fb6b0003980 0x7fb6c8151540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:22:23.909 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:23.900+0000 7fb6c6ffd640 1 -- 172.21.15.143:0/1938290529 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb6c81f3d60 con 0x7fb6c81f2f00 2023-11-26T07:22:23.909 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:23.900+0000 7fb6c7fff640 1 --2- 172.21.15.143:0/1938290529 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fb6c80b37e0 0x7fb6c815f270 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-26T07:22:23.909 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:23.904+0000 7fb6c6ffd640 1 --2- 172.21.15.143:0/1938290529 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fb6c81f2f00 0x7fb6c80ccf50 secure :-1 s=READY pgs=894 cs=0 l=1 rev1=1 crypto rx=0x7fb6bc011f30 tx=0x7fb6bc00f600 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:22:23.909 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:23.904+0000 7fb6affff640 1 -- 172.21.15.143:0/1938290529 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb6bc015610 con 0x7fb6c81f2f00 2023-11-26T07:22:23.909 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:23.904+0000 7fb6affff640 1 -- 172.21.15.143:0/1938290529 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb6bc015790 con 0x7fb6c81f2f00 2023-11-26T07:22:23.909 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:23.904+0000 7fb6affff640 1 -- 172.21.15.143:0/1938290529 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb6bc015b40 con 0x7fb6c81f2f00 2023-11-26T07:22:23.910 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:23.904+0000 7fb6cdd80640 1 -- 172.21.15.143:0/1938290529 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb6c81f6520 con 0x7fb6c81f2f00 2023-11-26T07:22:23.910 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:23.904+0000 7fb6cdd80640 1 -- 172.21.15.143:0/1938290529 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb6c80cd490 con 0x7fb6c81f2f00 2023-11-26T07:22:23.910 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:23.904+0000 7fb6cdd80640 1 -- 172.21.15.143:0/1938290529 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb6c81f6760 con 0x7fb6c81f2f00 2023-11-26T07:22:23.910 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:23.904+0000 7fb6affff640 1 -- 172.21.15.143:0/1938290529 <== mon.0 v2:172.21.15.143:3300/0 4 ==== mgrmap(e 11) v1 ==== 86158+0+0 (secure 0 0 0) 0x7fb6bc015ca0 con 0x7fb6c81f2f00 2023-11-26T07:22:23.913 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:23.904+0000 7fb6affff640 1 --2- 172.21.15.143:0/1938290529 >> [v2:172.21.15.143:6832/2150557544,v1:172.21.15.143:6833/2150557544] conn(0x7fb6b40667d0 0x7fb6b4068c90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:22:23.913 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:23.904+0000 7fb6affff640 1 -- 172.21.15.143:0/1938290529 <== mon.0 v2:172.21.15.143:3300/0 5 ==== osd_map(59..59 src has 1..59) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fb6bc087da0 con 0x7fb6c81f2f00 2023-11-26T07:22:23.913 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:23.904+0000 7fb6c77fe640 1 --2- 172.21.15.143:0/1938290529 >> [v2:172.21.15.143:6832/2150557544,v1:172.21.15.143:6833/2150557544] conn(0x7fb6b40667d0 0x7fb6b4068c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:22:23.914 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:23.908+0000 7fb6c77fe640 1 --2- 172.21.15.143:0/1938290529 >> [v2:172.21.15.143:6832/2150557544,v1:172.21.15.143:6833/2150557544] conn(0x7fb6b40667d0 0x7fb6b4068c90 secure :-1 s=READY pgs=419 cs=0 l=1 rev1=1 crypto rx=0x7fb6b80af000 tx=0x7fb6b80046d0 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:22:23.914 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:23.908+0000 7fb6affff640 1 -- 172.21.15.143:0/1938290529 <== mon.0 v2:172.21.15.143:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172649 (secure 0 0 0) 0x7fb6c81f6760 con 0x7fb6c81f2f00 2023-11-26T07:22:24.093 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:24.092+0000 7fb6cdd80640 1 -- 172.21.15.143:0/1938290529 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fb6c81f6760 con 0x7fb6c81f2f00 2023-11-26T07:22:24.094 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:24.092+0000 7fb6affff640 1 -- 172.21.15.143:0/1938290529 <== mon.0 v2:172.21.15.143:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 35 v35) v1 ==== 94+0+4925 (secure 0 0 0) 0x7fb6c81f6760 con 0x7fb6c81f2f00 2023-11-26T07:22:24.094 INFO:teuthology.orchestra.run.smithi143.stdout: 2023-11-26T07:22:24.094 INFO:teuthology.orchestra.run.smithi143.stdout:{"epoch":35,"default_fscid":4,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5612,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.143:6837/1372238588","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6836","nonce":1372238588},{"type":"v1","addr":"172.21.15.143:6837","nonce":1372238588}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":29},{"gid":6103,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.143:6835/1078738477","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6834","nonce":1078738477},{"type":"v1","addr":"172.21.15.143:6835","nonce":1078738477}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":29},{"gid":6118,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.188:6834/1634083777","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6832","nonce":1634083777},{"type":"v1","addr":"172.21.15.188:6834","nonce":1634083777}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":29}],"filesystems":[{"mdsmap":{"epoch":34,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-11-26T07:21:55.656168+0000","modified":"2023-11-26T07:21:58.674916+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":6121},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6121":{"gid":6121,"name":"b","rank":0,"incarnation":30,"state":"up:active","state_seq":3,"addr":"172.21.15.188:6835/1958716738","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6833","nonce":1958716738},{"type":"v1","addr":"172.21.15.188:6835","nonce":1958716738}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[9],"metadata_pool":8,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":4}]} 2023-11-26T07:22:24.096 INFO:teuthology.orchestra.run.smithi143.stderr:dumped fsmap epoch 35 2023-11-26T07:22:24.096 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:24.092+0000 7fb6cdd80640 1 -- 172.21.15.143:0/1938290529 >> [v2:172.21.15.143:6832/2150557544,v1:172.21.15.143:6833/2150557544] conn(0x7fb6b40667d0 msgr2=0x7fb6b4068c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:22:24.096 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:24.092+0000 7fb6cdd80640 1 --2- 172.21.15.143:0/1938290529 >> [v2:172.21.15.143:6832/2150557544,v1:172.21.15.143:6833/2150557544] conn(0x7fb6b40667d0 0x7fb6b4068c90 secure :-1 s=READY pgs=419 cs=0 l=1 rev1=1 crypto rx=0x7fb6b80af000 tx=0x7fb6b80046d0 comp rx=0 tx=0).stop 2023-11-26T07:22:24.096 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:24.092+0000 7fb6cdd80640 1 -- 172.21.15.143:0/1938290529 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fb6c81f2f00 msgr2=0x7fb6c80ccf50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:22:24.096 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:24.092+0000 7fb6cdd80640 1 --2- 172.21.15.143:0/1938290529 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fb6c81f2f00 0x7fb6c80ccf50 secure :-1 s=READY pgs=894 cs=0 l=1 rev1=1 crypto rx=0x7fb6bc011f30 tx=0x7fb6bc00f600 comp rx=0 tx=0).stop 2023-11-26T07:22:24.096 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:24.092+0000 7fb6cdd80640 1 -- 172.21.15.143:0/1938290529 shutdown_connections 2023-11-26T07:22:24.096 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:24.092+0000 7fb6cdd80640 1 --2- 172.21.15.143:0/1938290529 >> [v2:172.21.15.143:6832/2150557544,v1:172.21.15.143:6833/2150557544] conn(0x7fb6b40667d0 0x7fb6b4068c90 unknown :-1 s=CLOSED pgs=419 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:22:24.096 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:24.092+0000 7fb6cdd80640 1 --2- 172.21.15.143:0/1938290529 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fb6c80b37e0 0x7fb6c815f270 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:22:24.096 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:24.092+0000 7fb6cdd80640 1 --2- 172.21.15.143:0/1938290529 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fb6c81f2f00 0x7fb6c80ccf50 unknown :-1 s=CLOSED pgs=894 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:22:24.097 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:24.092+0000 7fb6cdd80640 1 --2- 172.21.15.143:0/1938290529 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fb6b0003980 0x7fb6c8151540 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:22:24.097 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:24.092+0000 7fb6cdd80640 1 -- 172.21.15.143:0/1938290529 >> 172.21.15.143:0/1938290529 conn(0x7fb6c809c550 msgr2=0x7fb6c80b0650 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:22:24.097 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:24.092+0000 7fb6cdd80640 1 -- 172.21.15.143:0/1938290529 shutdown_connections 2023-11-26T07:22:24.097 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:24.092+0000 7fb6cdd80640 1 -- 172.21.15.143:0/1938290529 wait complete. 2023-11-26T07:22:24.118 DEBUG:teuthology.orchestra.run.smithi143:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-11-26T07:22:24.257 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:24.256+0000 7f4193e54640 1 Processor -- start 2023-11-26T07:22:24.258 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:24.256+0000 7f4193e54640 1 -- start start 2023-11-26T07:22:24.258 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:24.256+0000 7f4193e54640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f418c0cf990 0x7f418c15efa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:22:24.258 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:24.256+0000 7f4193e54640 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7f418c15f4e0 0x7f418c0cd7b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:22:24.259 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:24.256+0000 7f4193e54640 1 -- --> v1:172.21.15.143:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f418c06c520 con 0x7f418c13e290 2023-11-26T07:22:24.259 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:24.256+0000 7f4193e54640 1 -- --> v2:172.21.15.188:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f418c06c800 con 0x7f418c15f4e0 2023-11-26T07:22:24.259 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:24.256+0000 7f4193e54640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f418c06cae0 con 0x7f418c0cf990 2023-11-26T07:22:24.259 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:24.256+0000 7f4191bf0640 1 --1- >> v1:172.21.15.143:6789/0 conn(0x7f418c13e290 0x7f418c0cf280 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.143:6789/0 says I am v1:172.21.15.143:59292/0 (socket says 172.21.15.143:59292) 2023-11-26T07:22:24.259 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:24.256+0000 7f4191bf0640 1 -- 172.21.15.143:0/3044057328 learned_addr learned my addr 172.21.15.143:0/3044057328 (peer_addr_for_me v1:172.21.15.143:0/0) 2023-11-26T07:22:24.259 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:24.256+0000 7f41923f1640 1 --2- 172.21.15.143:0/3044057328 >> v2:172.21.15.188:3300/0 conn(0x7f418c15f4e0 0x7f418c0cd7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:22:24.259 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:24.256+0000 7f41913ef640 1 --2- 172.21.15.143:0/3044057328 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f418c0cf990 0x7f418c15efa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:22:24.260 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:24.256+0000 7f4190bee640 1 -- 172.21.15.143:0/3044057328 <== mon.0 v1:172.21.15.143:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 286778572 0 0) 0x7f418c06c520 con 0x7f418c13e290 2023-11-26T07:22:24.260 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:24.256+0000 7f4190bee640 1 -- 172.21.15.143:0/3044057328 --> v1:172.21.15.143:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f4178003530 con 0x7f418c13e290 2023-11-26T07:22:24.260 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:24.256+0000 7f41923f1640 1 -- 172.21.15.143:0/3044057328 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f418c0cf990 msgr2=0x7f418c15efa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-26T07:22:24.260 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:24.256+0000 7f41923f1640 1 --2- 172.21.15.143:0/3044057328 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f418c0cf990 0x7f418c15efa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:22:24.260 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:24.256+0000 7f41923f1640 1 -- 172.21.15.143:0/3044057328 >> v1:172.21.15.143:6789/0 conn(0x7f418c13e290 legacy=0x7f418c0cf280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:22:24.260 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:24.256+0000 7f41923f1640 1 -- 172.21.15.143:0/3044057328 --> v2:172.21.15.188:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f418c06cdc0 con 0x7f418c15f4e0 2023-11-26T07:22:24.262 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:24.260+0000 7f41923f1640 1 --2- 172.21.15.143:0/3044057328 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f418c15f4e0 0x7f418c0cd7b0 secure :-1 s=READY pgs=351 cs=0 l=1 rev1=1 crypto rx=0x7f4184009d90 tx=0x7f418401c920 comp rx=0 tx=0).ready entity=mon.1 client_cookie=a6db343ee71c0f8e server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:22:24.262 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:24.260+0000 7f4190bee640 1 -- 172.21.15.143:0/3044057328 <== mon.1 v2:172.21.15.188:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f41840a5a00 con 0x7f418c15f4e0 2023-11-26T07:22:24.262 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:24.260+0000 7f4190bee640 1 -- 172.21.15.143:0/3044057328 <== mon.1 v2:172.21.15.188:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f41840a5b60 con 0x7f418c15f4e0 2023-11-26T07:22:24.263 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:24.260+0000 7f4190bee640 1 -- 172.21.15.143:0/3044057328 <== mon.1 v2:172.21.15.188:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f41840ad440 con 0x7f418c15f4e0 2023-11-26T07:22:24.263 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:24.260+0000 7f4193e54640 1 -- 172.21.15.143:0/3044057328 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f418c15f4e0 msgr2=0x7f418c0cd7b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:22:24.263 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:24.260+0000 7f4193e54640 1 --2- 172.21.15.143:0/3044057328 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f418c15f4e0 0x7f418c0cd7b0 secure :-1 s=READY pgs=351 cs=0 l=1 rev1=1 crypto rx=0x7f4184009d90 tx=0x7f418401c920 comp rx=0 tx=0).stop 2023-11-26T07:22:24.264 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:24.260+0000 7f4193e54640 1 -- 172.21.15.143:0/3044057328 shutdown_connections 2023-11-26T07:22:24.264 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:24.260+0000 7f4193e54640 1 --2- 172.21.15.143:0/3044057328 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f418c15f4e0 0x7f418c0cd7b0 unknown :-1 s=CLOSED pgs=351 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:22:24.264 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:24.260+0000 7f4193e54640 1 --2- 172.21.15.143:0/3044057328 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f418c0cf990 0x7f418c15efa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:22:24.264 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:24.260+0000 7f4193e54640 1 -- 172.21.15.143:0/3044057328 >> 172.21.15.143:0/3044057328 conn(0x7f418c06a890 msgr2=0x7f418c06aca0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:22:24.264 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:24.260+0000 7f4193e54640 1 -- 172.21.15.143:0/3044057328 shutdown_connections 2023-11-26T07:22:24.264 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:24.260+0000 7f4193e54640 1 -- 172.21.15.143:0/3044057328 wait complete. 2023-11-26T07:22:24.265 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:24.264+0000 7f4193e54640 1 Processor -- start 2023-11-26T07:22:24.266 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:24.264+0000 7f4193e54640 1 -- start start 2023-11-26T07:22:24.266 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:24.264+0000 7f4193e54640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f418c0cf990 0x7f418c137be0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:22:24.266 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:24.264+0000 7f4193e54640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f418c13e290 0x7f418c15b510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:22:24.267 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:24.264+0000 7f4193e54640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f418c15f4e0 0x7f418c15ba50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:22:24.267 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:24.264+0000 7f4193e54640 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f418c138120 con 0x7f418c15f4e0 2023-11-26T07:22:24.267 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:24.264+0000 7f4193e54640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f418c15e1d0 con 0x7f418c0cf990 2023-11-26T07:22:24.267 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:24.264+0000 7f4193e54640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f418c15c080 con 0x7f418c13e290 2023-11-26T07:22:24.268 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:24.264+0000 7f41923f1640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f418c15f4e0 0x7f418c15ba50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:22:24.268 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:24.264+0000 7f4191bf0640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f418c0cf990 0x7f418c137be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:22:24.268 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:24.264+0000 7f41923f1640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f418c15f4e0 0x7f418c15ba50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:34436/0 (socket says 172.21.15.143:34436) 2023-11-26T07:22:24.268 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:24.264+0000 7f41923f1640 1 -- 172.21.15.143:0/3896297024 learned_addr learned my addr 172.21.15.143:0/3896297024 (peer_addr_for_me v2:172.21.15.143:0/0) 2023-11-26T07:22:24.269 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:24.264+0000 7f41913ef640 1 --2- 172.21.15.143:0/3896297024 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f418c13e290 0x7f418c15b510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:22:24.269 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:24.264+0000 7f41923f1640 1 -- 172.21.15.143:0/3896297024 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f418c13e290 msgr2=0x7f418c15b510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:22:24.269 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:24.264+0000 7f41923f1640 1 --2- 172.21.15.143:0/3896297024 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f418c13e290 0x7f418c15b510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:22:24.269 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:24.264+0000 7f41923f1640 1 -- 172.21.15.143:0/3896297024 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f418c0cf990 msgr2=0x7f418c137be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:22:24.269 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:24.264+0000 7f41923f1640 1 --2- 172.21.15.143:0/3896297024 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f418c0cf990 0x7f418c137be0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:22:24.269 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:24.264+0000 7f41923f1640 1 -- 172.21.15.143:0/3896297024 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4184009a70 con 0x7f418c15f4e0 2023-11-26T07:22:24.269 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:24.264+0000 7f4191bf0640 1 --2- 172.21.15.143:0/3896297024 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f418c0cf990 0x7f418c137be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-26T07:22:24.269 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:24.264+0000 7f41923f1640 1 --2- 172.21.15.143:0/3896297024 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f418c15f4e0 0x7f418c15ba50 secure :-1 s=READY pgs=896 cs=0 l=1 rev1=1 crypto rx=0x7f418401a550 tx=0x7f41840196c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:22:24.269 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:24.264+0000 7f41827fc640 1 -- 172.21.15.143:0/3896297024 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4184007000 con 0x7f418c15f4e0 2023-11-26T07:22:24.269 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:24.264+0000 7f41827fc640 1 -- 172.21.15.143:0/3896297024 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f41840aed30 con 0x7f418c15f4e0 2023-11-26T07:22:24.269 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:24.264+0000 7f4193e54640 1 -- 172.21.15.143:0/3896297024 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f418c15c360 con 0x7f418c15f4e0 2023-11-26T07:22:24.269 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:24.264+0000 7f41827fc640 1 -- 172.21.15.143:0/3896297024 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f41840aeea0 con 0x7f418c15f4e0 2023-11-26T07:22:24.270 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:24.264+0000 7f4193e54640 1 -- 172.21.15.143:0/3896297024 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f418c070720 con 0x7f418c15f4e0 2023-11-26T07:22:24.270 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:24.268+0000 7f4193e54640 1 -- 172.21.15.143:0/3896297024 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f418c067680 con 0x7f418c15f4e0 2023-11-26T07:22:24.270 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:24.268+0000 7f41827fc640 1 -- 172.21.15.143:0/3896297024 <== mon.0 v2:172.21.15.143:3300/0 4 ==== mgrmap(e 11) v1 ==== 86158+0+0 (secure 0 0 0) 0x7f41840b4050 con 0x7f418c15f4e0 2023-11-26T07:22:24.271 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:24.268+0000 7f41827fc640 1 --2- 172.21.15.143:0/3896297024 >> [v2:172.21.15.143:6832/2150557544,v1:172.21.15.143:6833/2150557544] conn(0x7f41700667d0 0x7f4170068c90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:22:24.271 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:24.268+0000 7f41827fc640 1 -- 172.21.15.143:0/3896297024 <== mon.0 v2:172.21.15.143:3300/0 5 ==== osd_map(59..59 src has 1..59) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f418411f470 con 0x7f418c15f4e0 2023-11-26T07:22:24.271 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:24.268+0000 7f4191bf0640 1 --2- 172.21.15.143:0/3896297024 >> [v2:172.21.15.143:6832/2150557544,v1:172.21.15.143:6833/2150557544] conn(0x7f41700667d0 0x7f4170068c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:22:24.271 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:24.268+0000 7f4191bf0640 1 --2- 172.21.15.143:0/3896297024 >> [v2:172.21.15.143:6832/2150557544,v1:172.21.15.143:6833/2150557544] conn(0x7f41700667d0 0x7f4170068c90 secure :-1 s=READY pgs=420 cs=0 l=1 rev1=1 crypto rx=0x7f4188005890 tx=0x7f4188009040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:22:24.274 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:24.272+0000 7f41827fc640 1 -- 172.21.15.143:0/3896297024 <== mon.0 v2:172.21.15.143:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172649 (secure 0 0 0) 0x7f41840ec4e0 con 0x7f418c15f4e0 2023-11-26T07:22:24.424 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:24.424+0000 7f4193e54640 1 -- 172.21.15.143:0/3896297024 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f418c0cbf70 con 0x7f418c15f4e0 2023-11-26T07:22:24.425 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:24.424+0000 7f41827fc640 1 -- 172.21.15.143:0/3896297024 <== mon.0 v2:172.21.15.143:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v59) v1 ==== 74+0+16601 (secure 0 0 0) 0x7f4184009450 con 0x7f418c15f4e0 2023-11-26T07:22:24.426 INFO:teuthology.orchestra.run.smithi143.stdout: 2023-11-26T07:22:24.426 INFO:teuthology.orchestra.run.smithi143.stdout:{"epoch":59,"fsid":"f5033060-2469-4a07-95fe-2b76f5404bc1","created":"2023-11-26T07:17:28.613663+0000","modified":"2023-11-26T07:21:57.286555+0000","last_up_change":"2023-11-26T07:17:41.157813+0000","last_in_change":"2023-11-26T07:17:31.181658+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":9,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-11-26T07:17:41.758597+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":8,"pool_name":"cephfs_metadata","create_time":"2023-11-26T07:21:53.382407+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"58","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":9,"pool_name":"cephfs_data","create_time":"2023-11-26T07:21:54.656298+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"59","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.6299999952316284,"score_stable":1.6299999952316284,"optimal_score":1,"raw_score_acting":1.6299999952316284,"raw_score_stable":1.6299999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"ebbee2c6-023e-44b9-9f1d-6f7bfd309730","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":57,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6817","nonce":639807369},{"type":"v1","addr":"172.21.15.143:6819","nonce":639807369}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6821","nonce":639807369},{"type":"v1","addr":"172.21.15.143:6823","nonce":639807369}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6830","nonce":639807369},{"type":"v1","addr":"172.21.15.143:6831","nonce":639807369}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6825","nonce":639807369},{"type":"v1","addr":"172.21.15.143:6828","nonce":639807369}]},"public_addr":"172.21.15.143:6819/639807369","cluster_addr":"172.21.15.143:6823/639807369","heartbeat_back_addr":"172.21.15.143:6831/639807369","heartbeat_front_addr":"172.21.15.143:6828/639807369","state":["exists","up"]},{"osd":1,"uuid":"fec4b8be-7122-49e9-9a19-5850a0700001","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":57,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6816","nonce":1144942530},{"type":"v1","addr":"172.21.15.143:6818","nonce":1144942530}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6820","nonce":1144942530},{"type":"v1","addr":"172.21.15.143:6822","nonce":1144942530}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6827","nonce":1144942530},{"type":"v1","addr":"172.21.15.143:6829","nonce":1144942530}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6824","nonce":1144942530},{"type":"v1","addr":"172.21.15.143:6826","nonce":1144942530}]},"public_addr":"172.21.15.143:6818/1144942530","cluster_addr":"172.21.15.143:6822/1144942530","heartbeat_back_addr":"172.21.15.143:6829/1144942530","heartbeat_front_addr":"172.21.15.143:6826/1144942530","state":["exists","up"]},{"osd":2,"uuid":"2888af5b-f933-46ca-8f95-6dd494ff43f6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":57,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6808","nonce":3909095186},{"type":"v1","addr":"172.21.15.143:6809","nonce":3909095186}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6810","nonce":3909095186},{"type":"v1","addr":"172.21.15.143:6811","nonce":3909095186}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6814","nonce":3909095186},{"type":"v1","addr":"172.21.15.143:6815","nonce":3909095186}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6812","nonce":3909095186},{"type":"v1","addr":"172.21.15.143:6813","nonce":3909095186}]},"public_addr":"172.21.15.143:6809/3909095186","cluster_addr":"172.21.15.143:6811/3909095186","heartbeat_back_addr":"172.21.15.143:6815/3909095186","heartbeat_front_addr":"172.21.15.143:6813/3909095186","state":["exists","up"]},{"osd":3,"uuid":"d1f52435-7bc1-4081-9550-9cd1ba55f549","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":57,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6800","nonce":3854210991},{"type":"v1","addr":"172.21.15.143:6801","nonce":3854210991}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6802","nonce":3854210991},{"type":"v1","addr":"172.21.15.143:6803","nonce":3854210991}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6806","nonce":3854210991},{"type":"v1","addr":"172.21.15.143:6807","nonce":3854210991}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6804","nonce":3854210991},{"type":"v1","addr":"172.21.15.143:6805","nonce":3854210991}]},"public_addr":"172.21.15.143:6801/3854210991","cluster_addr":"172.21.15.143:6803/3854210991","heartbeat_back_addr":"172.21.15.143:6807/3854210991","heartbeat_front_addr":"172.21.15.143:6805/3854210991","state":["exists","up"]},{"osd":4,"uuid":"e2ae8b6e-6e25-49b2-bcbf-aacce68f06a3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":57,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6800","nonce":3924606902},{"type":"v1","addr":"172.21.15.188:6801","nonce":3924606902}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6802","nonce":3924606902},{"type":"v1","addr":"172.21.15.188:6803","nonce":3924606902}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6806","nonce":3924606902},{"type":"v1","addr":"172.21.15.188:6807","nonce":3924606902}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6804","nonce":3924606902},{"type":"v1","addr":"172.21.15.188:6805","nonce":3924606902}]},"public_addr":"172.21.15.188:6801/3924606902","cluster_addr":"172.21.15.188:6803/3924606902","heartbeat_back_addr":"172.21.15.188:6807/3924606902","heartbeat_front_addr":"172.21.15.188:6805/3924606902","state":["exists","up"]},{"osd":5,"uuid":"3cec8a32-7698-4c36-8e89-e4ba11934f33","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":57,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6813","nonce":186841544},{"type":"v1","addr":"172.21.15.188:6815","nonce":186841544}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6817","nonce":186841544},{"type":"v1","addr":"172.21.15.188:6818","nonce":186841544}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6822","nonce":186841544},{"type":"v1","addr":"172.21.15.188:6823","nonce":186841544}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6820","nonce":186841544},{"type":"v1","addr":"172.21.15.188:6821","nonce":186841544}]},"public_addr":"172.21.15.188:6815/186841544","cluster_addr":"172.21.15.188:6818/186841544","heartbeat_back_addr":"172.21.15.188:6823/186841544","heartbeat_front_addr":"172.21.15.188:6821/186841544","state":["exists","up"]},{"osd":6,"uuid":"58a65d5d-e090-473a-81b4-ad38e85029f9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":57,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6808","nonce":2153499405},{"type":"v1","addr":"172.21.15.188:6809","nonce":2153499405}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6810","nonce":2153499405},{"type":"v1","addr":"172.21.15.188:6811","nonce":2153499405}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6816","nonce":2153499405},{"type":"v1","addr":"172.21.15.188:6819","nonce":2153499405}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6812","nonce":2153499405},{"type":"v1","addr":"172.21.15.188:6814","nonce":2153499405}]},"public_addr":"172.21.15.188:6809/2153499405","cluster_addr":"172.21.15.188:6811/2153499405","heartbeat_back_addr":"172.21.15.188:6819/2153499405","heartbeat_front_addr":"172.21.15.188:6814/2153499405","state":["exists","up"]},{"osd":7,"uuid":"ea5b7f70-3554-4cf8-ac30-f7c59b80678d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":57,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6824","nonce":2722096729},{"type":"v1","addr":"172.21.15.188:6825","nonce":2722096729}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6826","nonce":2722096729},{"type":"v1","addr":"172.21.15.188:6827","nonce":2722096729}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6830","nonce":2722096729},{"type":"v1","addr":"172.21.15.188:6831","nonce":2722096729}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6828","nonce":2722096729},{"type":"v1","addr":"172.21.15.188:6829","nonce":2722096729}]},"public_addr":"172.21.15.188:6825/2722096729","cluster_addr":"172.21.15.188:6827/2722096729","heartbeat_back_addr":"172.21.15.188:6831/2722096729","heartbeat_front_addr":"172.21.15.188:6829/2722096729","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-26T07:17:36.995032+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-26T07:17:36.932266+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-26T07:17:36.829304+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-26T07:17:36.804297+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-26T07:17:36.667892+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-26T07:17:36.964868+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-26T07:17:36.827332+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-26T07:17:36.893179+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-11-26T07:22:24.428 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:24.424+0000 7f4193e54640 1 -- 172.21.15.143:0/3896297024 >> [v2:172.21.15.143:6832/2150557544,v1:172.21.15.143:6833/2150557544] conn(0x7f41700667d0 msgr2=0x7f4170068c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:22:24.428 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:24.424+0000 7f4193e54640 1 --2- 172.21.15.143:0/3896297024 >> [v2:172.21.15.143:6832/2150557544,v1:172.21.15.143:6833/2150557544] conn(0x7f41700667d0 0x7f4170068c90 secure :-1 s=READY pgs=420 cs=0 l=1 rev1=1 crypto rx=0x7f4188005890 tx=0x7f4188009040 comp rx=0 tx=0).stop 2023-11-26T07:22:24.428 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:24.424+0000 7f4193e54640 1 -- 172.21.15.143:0/3896297024 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f418c15f4e0 msgr2=0x7f418c15ba50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:22:24.428 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:24.424+0000 7f4193e54640 1 --2- 172.21.15.143:0/3896297024 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f418c15f4e0 0x7f418c15ba50 secure :-1 s=READY pgs=896 cs=0 l=1 rev1=1 crypto rx=0x7f418401a550 tx=0x7f41840196c0 comp rx=0 tx=0).stop 2023-11-26T07:22:24.428 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:24.424+0000 7f4193e54640 1 -- 172.21.15.143:0/3896297024 shutdown_connections 2023-11-26T07:22:24.428 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:24.424+0000 7f4193e54640 1 --2- 172.21.15.143:0/3896297024 >> [v2:172.21.15.143:6832/2150557544,v1:172.21.15.143:6833/2150557544] conn(0x7f41700667d0 0x7f4170068c90 unknown :-1 s=CLOSED pgs=420 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:22:24.428 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:24.424+0000 7f4193e54640 1 --2- 172.21.15.143:0/3896297024 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f418c15f4e0 0x7f418c15ba50 unknown :-1 s=CLOSED pgs=896 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:22:24.428 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:24.424+0000 7f4193e54640 1 --2- 172.21.15.143:0/3896297024 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f418c13e290 0x7f418c15b510 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:22:24.428 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:24.424+0000 7f4193e54640 1 --2- 172.21.15.143:0/3896297024 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f418c0cf990 0x7f418c137be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:22:24.429 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:24.424+0000 7f4193e54640 1 -- 172.21.15.143:0/3896297024 >> 172.21.15.143:0/3896297024 conn(0x7f418c06a890 msgr2=0x7f418c139b20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:22:24.429 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:24.428+0000 7f4193e54640 1 -- 172.21.15.143:0/3896297024 shutdown_connections 2023-11-26T07:22:24.429 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:24.428+0000 7f4193e54640 1 -- 172.21.15.143:0/3896297024 wait complete. 2023-11-26T07:22:24.447 DEBUG:teuthology.orchestra.run.smithi143:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs fail cephfs 2023-11-26T07:22:24.587 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:24.584+0000 7f1b06e59640 1 Processor -- start 2023-11-26T07:22:24.587 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:24.584+0000 7f1b06e59640 1 -- start start 2023-11-26T07:22:24.588 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:24.588+0000 7f1b06e59640 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7f1b0015d2a0 0x7f1b000d00e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:22:24.588 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:24.588+0000 7f1b06e59640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f1b0015d680 0x7f1b000cec30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:22:24.589 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:24.588+0000 7f1b06e59640 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7f1b000d0e80 0x7f1b000d1680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:22:24.589 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:24.588+0000 7f1b06e59640 1 -- --> v2:172.21.15.143:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1b000cdc40 con 0x7f1b0015d2a0 2023-11-26T07:22:24.589 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:24.588+0000 7f1b06e59640 1 -- --> v2:172.21.15.188:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1b000ce7a0 con 0x7f1b000d0e80 2023-11-26T07:22:24.589 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:24.588+0000 7f1b06e59640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1b000d1440 con 0x7f1b0015d680 2023-11-26T07:22:24.590 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:24.588+0000 7f1b05e57640 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7f1b0015d2a0 0x7f1b000d00e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:22:24.590 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:24.588+0000 7f1b05e57640 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7f1b0015d2a0 0x7f1b000d00e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:47402/0 (socket says 172.21.15.143:47402) 2023-11-26T07:22:24.590 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:24.588+0000 7f1b05e57640 1 -- 172.21.15.143:0/2368822103 learned_addr learned my addr 172.21.15.143:0/2368822103 (peer_addr_for_me v2:172.21.15.143:0/0) 2023-11-26T07:22:24.590 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:24.588+0000 7f1b06658640 1 --2- 172.21.15.143:0/2368822103 >> v2:172.21.15.188:3300/0 conn(0x7f1b000d0e80 0x7f1b000d1680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:22:24.590 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:24.588+0000 7f1b05656640 1 --2- 172.21.15.143:0/2368822103 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f1b0015d680 0x7f1b000cec30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:22:24.590 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:24.588+0000 7f1b05e57640 1 -- 172.21.15.143:0/2368822103 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f1b0015d680 msgr2=0x7f1b000cec30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-26T07:22:24.590 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:24.588+0000 7f1b05e57640 1 --2- 172.21.15.143:0/2368822103 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f1b0015d680 0x7f1b000cec30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:22:24.590 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:24.588+0000 7f1b05e57640 1 -- 172.21.15.143:0/2368822103 >> v2:172.21.15.188:3300/0 conn(0x7f1b000d0e80 msgr2=0x7f1b000d1680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-26T07:22:24.590 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:24.588+0000 7f1b05e57640 1 --2- 172.21.15.143:0/2368822103 >> v2:172.21.15.188:3300/0 conn(0x7f1b000d0e80 0x7f1b000d1680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:22:24.591 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:24.588+0000 7f1b05e57640 1 -- 172.21.15.143:0/2368822103 --> v2:172.21.15.143:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1b000cde70 con 0x7f1b0015d2a0 2023-11-26T07:22:24.591 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:24.588+0000 7f1b06658640 1 --2- 172.21.15.143:0/2368822103 >> v2:172.21.15.188:3300/0 conn(0x7f1b000d0e80 0x7f1b000d1680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-26T07:22:24.592 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:24.592+0000 7f1b05e57640 1 --2- 172.21.15.143:0/2368822103 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f1b0015d2a0 0x7f1b000d00e0 secure :-1 s=READY pgs=897 cs=0 l=1 rev1=1 crypto rx=0x7f1b0008c4a0 tx=0x7f1afc01c440 comp rx=0 tx=0).ready entity=mon.0 client_cookie=20086fc193a0f697 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:22:24.593 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:24.592+0000 7f1b04e55640 1 -- 172.21.15.143:0/2368822103 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1afc0ac640 con 0x7f1b0015d2a0 2023-11-26T07:22:24.593 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:24.592+0000 7f1b04e55640 1 -- 172.21.15.143:0/2368822103 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1afc0ac7a0 con 0x7f1b0015d2a0 2023-11-26T07:22:24.593 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:24.592+0000 7f1b04e55640 1 -- 172.21.15.143:0/2368822103 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1afc0acb30 con 0x7f1b0015d2a0 2023-11-26T07:22:24.594 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:24.592+0000 7f1b06e59640 1 -- 172.21.15.143:0/2368822103 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f1b0015d2a0 msgr2=0x7f1b000d00e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:22:24.594 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:24.592+0000 7f1b06e59640 1 --2- 172.21.15.143:0/2368822103 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f1b0015d2a0 0x7f1b000d00e0 secure :-1 s=READY pgs=897 cs=0 l=1 rev1=1 crypto rx=0x7f1b0008c4a0 tx=0x7f1afc01c440 comp rx=0 tx=0).stop 2023-11-26T07:22:24.594 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:24.592+0000 7f1b06e59640 1 -- 172.21.15.143:0/2368822103 shutdown_connections 2023-11-26T07:22:24.594 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:24.592+0000 7f1b06e59640 1 --2- 172.21.15.143:0/2368822103 >> v2:172.21.15.188:3300/0 conn(0x7f1b000d0e80 0x7f1b000d1680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:22:24.594 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:24.592+0000 7f1b06e59640 1 --2- 172.21.15.143:0/2368822103 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f1b0015d680 0x7f1b000cec30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:22:24.594 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:24.592+0000 7f1b06e59640 1 --2- 172.21.15.143:0/2368822103 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f1b0015d2a0 0x7f1b000d00e0 unknown :-1 s=CLOSED pgs=897 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:22:24.594 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:24.592+0000 7f1b06e59640 1 -- 172.21.15.143:0/2368822103 >> 172.21.15.143:0/2368822103 conn(0x7f1b000d30a0 msgr2=0x7f1b000d3480 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:22:24.594 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:24.592+0000 7f1b06e59640 1 -- 172.21.15.143:0/2368822103 shutdown_connections 2023-11-26T07:22:24.594 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:24.592+0000 7f1b06e59640 1 -- 172.21.15.143:0/2368822103 wait complete. 2023-11-26T07:22:24.595 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:24.592+0000 7f1b06e59640 1 Processor -- start 2023-11-26T07:22:24.595 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:24.596+0000 7f1b06e59640 1 -- start start 2023-11-26T07:22:24.597 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:24.596+0000 7f1b06e59640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f1b000d0e80 0x7f1b00156180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:22:24.598 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:24.596+0000 7f1b06e59640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f1b0015d2a0 0x7f1b000cbeb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:22:24.598 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:24.596+0000 7f1b06e59640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f1b0015d680 0x7f1b001558c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:22:24.598 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:24.596+0000 7f1b06e59640 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1b00155430 con 0x7f1b000d0e80 2023-11-26T07:22:24.598 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:24.596+0000 7f1b06e59640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1b001566c0 con 0x7f1b0015d680 2023-11-26T07:22:24.598 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:24.596+0000 7f1b06e59640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1b00156bb0 con 0x7f1b0015d2a0 2023-11-26T07:22:24.598 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:24.596+0000 7f1b05e57640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f1b000d0e80 0x7f1b00156180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:22:24.598 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:24.596+0000 7f1b05e57640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f1b000d0e80 0x7f1b00156180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:47408/0 (socket says 172.21.15.143:47408) 2023-11-26T07:22:24.598 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:24.596+0000 7f1b05e57640 1 -- 172.21.15.143:0/3922162538 learned_addr learned my addr 172.21.15.143:0/3922162538 (peer_addr_for_me v2:172.21.15.143:0/0) 2023-11-26T07:22:24.598 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:24.596+0000 7f1b05656640 1 --2- 172.21.15.143:0/3922162538 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f1b0015d2a0 0x7f1b000cbeb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:22:24.598 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:24.596+0000 7f1b06658640 1 --2- 172.21.15.143:0/3922162538 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f1b0015d680 0x7f1b001558c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:22:24.598 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:24.596+0000 7f1b05e57640 1 -- 172.21.15.143:0/3922162538 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f1b0015d2a0 msgr2=0x7f1b000cbeb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:22:24.598 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:24.596+0000 7f1b05e57640 1 --2- 172.21.15.143:0/3922162538 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f1b0015d2a0 0x7f1b000cbeb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:22:24.599 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:24.596+0000 7f1b05e57640 1 -- 172.21.15.143:0/3922162538 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f1b0015d680 msgr2=0x7f1b001558c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:22:24.599 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:24.596+0000 7f1b05e57640 1 --2- 172.21.15.143:0/3922162538 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f1b0015d680 0x7f1b001558c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:22:24.599 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:24.596+0000 7f1b05e57640 1 -- 172.21.15.143:0/3922162538 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1afc0094d0 con 0x7f1b000d0e80 2023-11-26T07:22:24.599 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:24.596+0000 7f1b06658640 1 --2- 172.21.15.143:0/3922162538 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f1b0015d680 0x7f1b001558c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-26T07:22:24.599 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:24.596+0000 7f1b05e57640 1 --2- 172.21.15.143:0/3922162538 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f1b000d0e80 0x7f1b00156180 secure :-1 s=READY pgs=898 cs=0 l=1 rev1=1 crypto rx=0x7f1afc0af040 tx=0x7f1afc01ac80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:22:24.600 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:24.600+0000 7f1af67fc640 1 -- 172.21.15.143:0/3922162538 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1afc0b7830 con 0x7f1b000d0e80 2023-11-26T07:22:24.600 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:24.600+0000 7f1af67fc640 1 -- 172.21.15.143:0/3922162538 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1afc0b79b0 con 0x7f1b000d0e80 2023-11-26T07:22:24.600 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:24.600+0000 7f1af67fc640 1 -- 172.21.15.143:0/3922162538 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1afc0b7d60 con 0x7f1b000d0e80 2023-11-26T07:22:24.600 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:24.600+0000 7f1b06e59640 1 -- 172.21.15.143:0/3922162538 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1b00156e90 con 0x7f1b000d0e80 2023-11-26T07:22:24.600 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:24.600+0000 7f1b06e59640 1 -- 172.21.15.143:0/3922162538 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1b001550c0 con 0x7f1b000d0e80 2023-11-26T07:22:24.601 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:24.600+0000 7f1b06e59640 1 -- 172.21.15.143:0/3922162538 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1b00067500 con 0x7f1b000d0e80 2023-11-26T07:22:24.602 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:24.600+0000 7f1af67fc640 1 -- 172.21.15.143:0/3922162538 <== mon.0 v2:172.21.15.143:3300/0 4 ==== mgrmap(e 11) v1 ==== 86158+0+0 (secure 0 0 0) 0x7f1afc0b3040 con 0x7f1b000d0e80 2023-11-26T07:22:24.602 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:24.600+0000 7f1af67fc640 1 --2- 172.21.15.143:0/3922162538 >> [v2:172.21.15.143:6832/2150557544,v1:172.21.15.143:6833/2150557544] conn(0x7f1ae4066800 0x7f1ae4068cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:22:24.603 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:24.600+0000 7f1af67fc640 1 -- 172.21.15.143:0/3922162538 <== mon.0 v2:172.21.15.143:3300/0 5 ==== osd_map(59..59 src has 1..59) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f1afc11eda0 con 0x7f1b000d0e80 2023-11-26T07:22:24.603 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:24.600+0000 7f1b05656640 1 --2- 172.21.15.143:0/3922162538 >> [v2:172.21.15.143:6832/2150557544,v1:172.21.15.143:6833/2150557544] conn(0x7f1ae4066800 0x7f1ae4068cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:22:24.603 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:24.604+0000 7f1b05656640 1 --2- 172.21.15.143:0/3922162538 >> [v2:172.21.15.143:6832/2150557544,v1:172.21.15.143:6833/2150557544] conn(0x7f1ae4066800 0x7f1ae4068cc0 secure :-1 s=READY pgs=421 cs=0 l=1 rev1=1 crypto rx=0x7f1b001f9720 tx=0x7f1af0008040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:22:24.607 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:24.604+0000 7f1af67fc640 1 -- 172.21.15.143:0/3922162538 <== mon.0 v2:172.21.15.143:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172649 (secure 0 0 0) 0x7f1afc0ebd80 con 0x7f1b000d0e80 2023-11-26T07:22:24.790 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:24.788+0000 7f1b06e59640 1 -- 172.21.15.143:0/3922162538 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs"} v 0) v1 -- 0x7f1b001568f0 con 0x7f1b000d0e80 2023-11-26T07:22:24.790 INFO:tasks.ceph.mon.a.smithi143.stderr:2023-11-26T07:22:24.788+0000 7f6946f73640 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2023-11-26T07:22:24.793 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:24.792+0000 7f1af67fc640 1 -- 172.21.15.143:0/3922162538 <== mon.0 v2:172.21.15.143:3300/0 7 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs"}]=0 cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v36) v1 ==== 161+0+0 (secure 0 0 0) 0x7f1afc0ef3a0 con 0x7f1b000d0e80 2023-11-26T07:22:24.794 INFO:teuthology.orchestra.run.smithi143.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2023-11-26T07:22:24.796 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:24.792+0000 7f1b06e59640 1 -- 172.21.15.143:0/3922162538 >> [v2:172.21.15.143:6832/2150557544,v1:172.21.15.143:6833/2150557544] conn(0x7f1ae4066800 msgr2=0x7f1ae4068cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:22:24.796 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:24.792+0000 7f1b06e59640 1 --2- 172.21.15.143:0/3922162538 >> [v2:172.21.15.143:6832/2150557544,v1:172.21.15.143:6833/2150557544] conn(0x7f1ae4066800 0x7f1ae4068cc0 secure :-1 s=READY pgs=421 cs=0 l=1 rev1=1 crypto rx=0x7f1b001f9720 tx=0x7f1af0008040 comp rx=0 tx=0).stop 2023-11-26T07:22:24.796 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:24.796+0000 7f1b06e59640 1 -- 172.21.15.143:0/3922162538 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f1b000d0e80 msgr2=0x7f1b00156180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:22:24.796 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:24.796+0000 7f1b06e59640 1 --2- 172.21.15.143:0/3922162538 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f1b000d0e80 0x7f1b00156180 secure :-1 s=READY pgs=898 cs=0 l=1 rev1=1 crypto rx=0x7f1afc0af040 tx=0x7f1afc01ac80 comp rx=0 tx=0).stop 2023-11-26T07:22:24.796 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:24.796+0000 7f1b06e59640 1 -- 172.21.15.143:0/3922162538 shutdown_connections 2023-11-26T07:22:24.796 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:24.796+0000 7f1b06e59640 1 --2- 172.21.15.143:0/3922162538 >> [v2:172.21.15.143:6832/2150557544,v1:172.21.15.143:6833/2150557544] conn(0x7f1ae4066800 0x7f1ae4068cc0 unknown :-1 s=CLOSED pgs=421 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:22:24.796 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:24.796+0000 7f1b06e59640 1 --2- 172.21.15.143:0/3922162538 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f1b0015d680 0x7f1b001558c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:22:24.796 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:24.796+0000 7f1b06e59640 1 --2- 172.21.15.143:0/3922162538 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f1b0015d2a0 0x7f1b000cbeb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:22:24.797 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:24.796+0000 7f1b06e59640 1 --2- 172.21.15.143:0/3922162538 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f1b000d0e80 0x7f1b00156180 unknown :-1 s=CLOSED pgs=898 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:22:24.797 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:24.796+0000 7f1b06e59640 1 -- 172.21.15.143:0/3922162538 >> 172.21.15.143:0/3922162538 conn(0x7f1b000d30a0 msgr2=0x7f1b000b1790 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:22:24.797 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:24.796+0000 7f1b06e59640 1 -- 172.21.15.143:0/3922162538 shutdown_connections 2023-11-26T07:22:24.797 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:24.796+0000 7f1b06e59640 1 -- 172.21.15.143:0/3922162538 wait complete. 2023-11-26T07:22:24.816 DEBUG:teuthology.orchestra.run.smithi143:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs rm cephfs --yes-i-really-mean-it 2023-11-26T07:22:24.867 INFO:tasks.ceph.mds.b.smithi188.stdout:starting mds.b at 2023-11-26T07:22:24.959 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:24.956+0000 7f95e85aa640 1 Processor -- start 2023-11-26T07:22:24.959 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:24.956+0000 7f95e85aa640 1 -- start start 2023-11-26T07:22:24.961 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:24.960+0000 7f95e85aa640 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7f95e01614b0 0x7f95e00cd610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:22:24.961 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:24.960+0000 7f95e85aa640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f95e0071970 0x7f95e0161890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:22:24.961 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:24.960+0000 7f95e85aa640 1 -- --> v1:172.21.15.188:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f95e015d970 con 0x7f95e0070ba0 2023-11-26T07:22:24.961 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:24.960+0000 7f95e85aa640 1 -- --> v2:172.21.15.188:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f95e0161dd0 con 0x7f95e01614b0 2023-11-26T07:22:24.961 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:24.960+0000 7f95e85aa640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f95e015dc30 con 0x7f95e0071970 2023-11-26T07:22:24.962 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:24.960+0000 7f95e6346640 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7f95e01614b0 0x7f95e00cd610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:22:24.962 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:24.960+0000 7f95e5b45640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f95e0071970 0x7f95e0161890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:22:24.962 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:24.960+0000 7f95e6b47640 1 --1- >> v1:172.21.15.188:6789/0 conn(0x7f95e0070ba0 0x7f95e006dd10 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.188:6789/0 says I am v1:172.21.15.143:34946/0 (socket says 172.21.15.143:34946) 2023-11-26T07:22:24.962 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:24.960+0000 7f95e6b47640 1 -- 172.21.15.143:0/4076010454 learned_addr learned my addr 172.21.15.143:0/4076010454 (peer_addr_for_me v1:172.21.15.143:0/0) 2023-11-26T07:22:24.962 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:24.960+0000 7f95e6346640 1 -- 172.21.15.143:0/4076010454 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f95e0071970 msgr2=0x7f95e0161890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-26T07:22:24.962 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:24.960+0000 7f95e6346640 1 --2- 172.21.15.143:0/4076010454 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f95e0071970 0x7f95e0161890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:22:24.963 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:24.960+0000 7f95e6346640 1 -- 172.21.15.143:0/4076010454 >> v1:172.21.15.188:6789/0 conn(0x7f95e0070ba0 legacy=0x7f95e006dd10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:22:24.963 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:24.960+0000 7f95e6346640 1 -- 172.21.15.143:0/4076010454 --> v2:172.21.15.188:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f95e015df10 con 0x7f95e01614b0 2023-11-26T07:22:24.963 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:24.960+0000 7f95e5b45640 1 --2- 172.21.15.143:0/4076010454 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f95e0071970 0x7f95e0161890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-26T07:22:24.964 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:24.964+0000 7f95e6346640 1 --2- 172.21.15.143:0/4076010454 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f95e01614b0 0x7f95e00cd610 secure :-1 s=READY pgs=355 cs=0 l=1 rev1=1 crypto rx=0x7f95dc0097c0 tx=0x7f95dc01c450 comp rx=0 tx=0).ready entity=mon.1 client_cookie=c1fe01260e061e76 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:22:24.965 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:24.964+0000 7f95e5344640 1 -- 172.21.15.143:0/4076010454 <== mon.1 v2:172.21.15.188:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f95dc0ab620 con 0x7f95e01614b0 2023-11-26T07:22:24.965 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:24.964+0000 7f95e5344640 1 -- 172.21.15.143:0/4076010454 <== mon.1 v2:172.21.15.188:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f95dc0ab780 con 0x7f95e01614b0 2023-11-26T07:22:24.965 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:24.964+0000 7f95e5344640 1 -- 172.21.15.143:0/4076010454 <== mon.1 v2:172.21.15.188:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f95dc0abb10 con 0x7f95e01614b0 2023-11-26T07:22:24.966 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:24.964+0000 7f95e85aa640 1 -- 172.21.15.143:0/4076010454 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f95e01614b0 msgr2=0x7f95e00cd610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:22:24.966 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:24.964+0000 7f95e85aa640 1 --2- 172.21.15.143:0/4076010454 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f95e01614b0 0x7f95e00cd610 secure :-1 s=READY pgs=355 cs=0 l=1 rev1=1 crypto rx=0x7f95dc0097c0 tx=0x7f95dc01c450 comp rx=0 tx=0).stop 2023-11-26T07:22:24.966 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:24.964+0000 7f95e85aa640 1 -- 172.21.15.143:0/4076010454 shutdown_connections 2023-11-26T07:22:24.966 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:24.964+0000 7f95e85aa640 1 --2- 172.21.15.143:0/4076010454 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f95e0071970 0x7f95e0161890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:22:24.966 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:24.964+0000 7f95e85aa640 1 --2- 172.21.15.143:0/4076010454 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f95e01614b0 0x7f95e00cd610 unknown :-1 s=CLOSED pgs=355 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:22:24.966 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:24.964+0000 7f95e85aa640 1 -- 172.21.15.143:0/4076010454 >> 172.21.15.143:0/4076010454 conn(0x7f95e00cf710 msgr2=0x7f95e00cfb20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:22:24.966 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:24.964+0000 7f95e85aa640 1 -- 172.21.15.143:0/4076010454 shutdown_connections 2023-11-26T07:22:24.966 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:24.964+0000 7f95e85aa640 1 -- 172.21.15.143:0/4076010454 wait complete. 2023-11-26T07:22:24.967 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:24.964+0000 7f95e85aa640 1 Processor -- start 2023-11-26T07:22:24.967 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:24.964+0000 7f95e85aa640 1 -- start start 2023-11-26T07:22:24.968 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:24.968+0000 7f95e85aa640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f95e0070ba0 0x7f95e0147070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:22:24.969 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:24.968+0000 7f95e85aa640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f95e0071970 0x7f95e0155070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:22:24.969 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:24.968+0000 7f95e85aa640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f95e01614b0 0x7f95e01555b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:22:24.969 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:24.968+0000 7f95e85aa640 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f95e00b2be0 con 0x7f95e0071970 2023-11-26T07:22:24.969 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:24.968+0000 7f95e85aa640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f95e0155be0 con 0x7f95e0070ba0 2023-11-26T07:22:24.969 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:24.968+0000 7f95e85aa640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f95e0155ec0 con 0x7f95e01614b0 2023-11-26T07:22:24.969 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:24.968+0000 7f95e6346640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f95e0070ba0 0x7f95e0147070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:22:24.969 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:24.968+0000 7f95e5b45640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f95e0071970 0x7f95e0155070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:22:24.969 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:24.968+0000 7f95e6346640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f95e0070ba0 0x7f95e0147070 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.188:3300/0 says I am v2:172.21.15.143:54648/0 (socket says 172.21.15.143:54648) 2023-11-26T07:22:24.969 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:24.968+0000 7f95e6346640 1 -- 172.21.15.143:0/66386945 learned_addr learned my addr 172.21.15.143:0/66386945 (peer_addr_for_me v2:172.21.15.143:0/0) 2023-11-26T07:22:24.969 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:24.968+0000 7f95e6b47640 1 --2- 172.21.15.143:0/66386945 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f95e01614b0 0x7f95e01555b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:22:24.970 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:24.968+0000 7f95e6346640 1 -- 172.21.15.143:0/66386945 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f95e01614b0 msgr2=0x7f95e01555b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:22:24.970 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:24.968+0000 7f95e6346640 1 --2- 172.21.15.143:0/66386945 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f95e01614b0 0x7f95e01555b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:22:24.970 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:24.968+0000 7f95e6346640 1 -- 172.21.15.143:0/66386945 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f95e0071970 msgr2=0x7f95e0155070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:22:24.970 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:24.968+0000 7f95e6346640 1 --2- 172.21.15.143:0/66386945 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f95e0071970 0x7f95e0155070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:22:24.970 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:24.968+0000 7f95e6346640 1 -- 172.21.15.143:0/66386945 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f95d0009420 con 0x7f95e0070ba0 2023-11-26T07:22:24.971 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:24.968+0000 7f95e5b45640 1 --2- 172.21.15.143:0/66386945 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f95e0071970 0x7f95e0155070 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-26T07:22:24.971 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:24.968+0000 7f95e6346640 1 --2- 172.21.15.143:0/66386945 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f95e0070ba0 0x7f95e0147070 secure :-1 s=READY pgs=356 cs=0 l=1 rev1=1 crypto rx=0x7f95dc0ad040 tx=0x7f95dc01ac90 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:22:24.971 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:24.968+0000 7f95d6ffd640 1 -- 172.21.15.143:0/66386945 <== mon.1 v2:172.21.15.188:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f95dc0abe40 con 0x7f95e0070ba0 2023-11-26T07:22:24.971 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:24.968+0000 7f95d6ffd640 1 -- 172.21.15.143:0/66386945 <== mon.1 v2:172.21.15.188:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f95dc0bf410 con 0x7f95e0070ba0 2023-11-26T07:22:24.971 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:24.968+0000 7f95e85aa640 1 -- 172.21.15.143:0/66386945 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f95dc009420 con 0x7f95e0070ba0 2023-11-26T07:22:24.971 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:24.968+0000 7f95d6ffd640 1 -- 172.21.15.143:0/66386945 <== mon.1 v2:172.21.15.188:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f95dc0bf7a0 con 0x7f95e0070ba0 2023-11-26T07:22:24.971 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:24.968+0000 7f95e85aa640 1 -- 172.21.15.143:0/66386945 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f95e0071540 con 0x7f95e0070ba0 2023-11-26T07:22:24.971 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:24.968+0000 7f95e85aa640 1 -- 172.21.15.143:0/66386945 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f95e01563e0 con 0x7f95e0070ba0 2023-11-26T07:22:24.972 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:24.972+0000 7f95d6ffd640 1 -- 172.21.15.143:0/66386945 <== mon.1 v2:172.21.15.188:3300/0 4 ==== mgrmap(e 11) v1 ==== 86158+0+0 (secure 0 0 0) 0x7f95dc0b5400 con 0x7f95e0070ba0 2023-11-26T07:22:24.973 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:24.972+0000 7f95d6ffd640 1 --2- 172.21.15.143:0/66386945 >> [v2:172.21.15.143:6832/2150557544,v1:172.21.15.143:6833/2150557544] conn(0x7f95c40667d0 0x7f95c4068c90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:22:24.973 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:24.972+0000 7f95d6ffd640 1 -- 172.21.15.143:0/66386945 <== mon.1 v2:172.21.15.188:3300/0 5 ==== osd_map(60..60 src has 1..60) v4 ==== 5795+0+0 (secure 0 0 0) 0x7f95dc11f420 con 0x7f95e0070ba0 2023-11-26T07:22:24.973 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:24.972+0000 7f95e5b45640 1 --2- 172.21.15.143:0/66386945 >> [v2:172.21.15.143:6832/2150557544,v1:172.21.15.143:6833/2150557544] conn(0x7f95c40667d0 0x7f95c4068c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:22:24.973 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:24.972+0000 7f95e5b45640 1 --2- 172.21.15.143:0/66386945 >> [v2:172.21.15.143:6832/2150557544,v1:172.21.15.143:6833/2150557544] conn(0x7f95c40667d0 0x7f95c4068c90 secure :-1 s=READY pgs=422 cs=0 l=1 rev1=1 crypto rx=0x7f95e015af80 tx=0x7f95d0009190 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:22:24.977 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:24.976+0000 7f95d6ffd640 1 -- 172.21.15.143:0/66386945 <== mon.1 v2:172.21.15.188:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172649 (secure 0 0 0) 0x7f95dc0ec4a0 con 0x7f95e0070ba0 2023-11-26T07:22:25.160 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:25.156+0000 7f95e85aa640 1 -- 172.21.15.143:0/66386945 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_command({"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true} v 0) v1 -- 0x7f95e0067680 con 0x7f95e0070ba0 2023-11-26T07:22:25.800 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:25.800+0000 7f95d6ffd640 1 -- 172.21.15.143:0/66386945 <== mon.1 v2:172.21.15.188:3300/0 7 ==== mon_command_ack([{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]=0 v37) v1 ==== 104+0+0 (secure 0 0 0) 0x7f95dc0efac0 con 0x7f95e0070ba0 2023-11-26T07:22:25.802 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:25.800+0000 7f95e85aa640 1 -- 172.21.15.143:0/66386945 >> [v2:172.21.15.143:6832/2150557544,v1:172.21.15.143:6833/2150557544] conn(0x7f95c40667d0 msgr2=0x7f95c4068c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:22:25.802 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:25.800+0000 7f95e85aa640 1 --2- 172.21.15.143:0/66386945 >> [v2:172.21.15.143:6832/2150557544,v1:172.21.15.143:6833/2150557544] conn(0x7f95c40667d0 0x7f95c4068c90 secure :-1 s=READY pgs=422 cs=0 l=1 rev1=1 crypto rx=0x7f95e015af80 tx=0x7f95d0009190 comp rx=0 tx=0).stop 2023-11-26T07:22:25.802 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:25.800+0000 7f95e85aa640 1 -- 172.21.15.143:0/66386945 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f95e0070ba0 msgr2=0x7f95e0147070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:22:25.802 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:25.800+0000 7f95e85aa640 1 --2- 172.21.15.143:0/66386945 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f95e0070ba0 0x7f95e0147070 secure :-1 s=READY pgs=356 cs=0 l=1 rev1=1 crypto rx=0x7f95dc0ad040 tx=0x7f95dc01ac90 comp rx=0 tx=0).stop 2023-11-26T07:22:25.802 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:25.800+0000 7f95e85aa640 1 -- 172.21.15.143:0/66386945 shutdown_connections 2023-11-26T07:22:25.803 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:25.800+0000 7f95e85aa640 1 --2- 172.21.15.143:0/66386945 >> [v2:172.21.15.143:6832/2150557544,v1:172.21.15.143:6833/2150557544] conn(0x7f95c40667d0 0x7f95c4068c90 unknown :-1 s=CLOSED pgs=422 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:22:25.803 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:25.800+0000 7f95e85aa640 1 --2- 172.21.15.143:0/66386945 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f95e01614b0 0x7f95e01555b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:22:25.803 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:25.800+0000 7f95e85aa640 1 --2- 172.21.15.143:0/66386945 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f95e0071970 0x7f95e0155070 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:22:25.803 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:25.800+0000 7f95e85aa640 1 --2- 172.21.15.143:0/66386945 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f95e0070ba0 0x7f95e0147070 unknown :-1 s=CLOSED pgs=356 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:22:25.803 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:25.800+0000 7f95e85aa640 1 -- 172.21.15.143:0/66386945 >> 172.21.15.143:0/66386945 conn(0x7f95e00cf710 msgr2=0x7f95e00afb00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:22:25.803 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:25.800+0000 7f95e85aa640 1 -- 172.21.15.143:0/66386945 shutdown_connections 2023-11-26T07:22:25.803 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:25.800+0000 7f95e85aa640 1 -- 172.21.15.143:0/66386945 wait complete. 2023-11-26T07:22:25.822 DEBUG:teuthology.orchestra.run.smithi143:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_metadata cephfs_metadata --yes-i-really-really-mean-it 2023-11-26T07:22:25.964 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:25.964+0000 7fc5f5616640 1 Processor -- start 2023-11-26T07:22:25.964 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:25.964+0000 7fc5f5616640 1 -- start start 2023-11-26T07:22:25.965 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:25.964+0000 7fc5f5616640 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7fc5f00d3850 0x7fc5f00afed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:22:25.966 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:25.964+0000 7fc5f5616640 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7fc5f015d530 0x7fc5f00b0410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:22:25.966 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:25.964+0000 7fc5f5616640 1 -- --> v1:172.21.15.188:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc5f00b0b20 con 0x7fc5f015d150 2023-11-26T07:22:25.966 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:25.964+0000 7fc5f5616640 1 -- --> v2:172.21.15.143:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc5f00d0bf0 con 0x7fc5f015d530 2023-11-26T07:22:25.966 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:25.964+0000 7fc5f5616640 1 -- --> v2:172.21.15.188:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc5f00d1a20 con 0x7fc5f00d3850 2023-11-26T07:22:25.966 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:25.964+0000 7fc5ef7fe640 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7fc5f015d530 0x7fc5f00b0410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:22:25.966 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:25.964+0000 7fc5ef7fe640 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7fc5f015d530 0x7fc5f00b0410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:47418/0 (socket says 172.21.15.143:47418) 2023-11-26T07:22:25.966 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:25.964+0000 7fc5ef7fe640 1 -- 172.21.15.143:0/4270556960 learned_addr learned my addr 172.21.15.143:0/4270556960 (peer_addr_for_me v2:172.21.15.143:0/0) 2023-11-26T07:22:25.967 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:25.964+0000 7fc5ef7fe640 1 -- 172.21.15.143:0/4270556960 >> v2:172.21.15.188:3300/0 conn(0x7fc5f00d3850 msgr2=0x7fc5f00afed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-26T07:22:25.967 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:25.964+0000 7fc5ee7fc640 1 --2- 172.21.15.143:0/4270556960 >> v2:172.21.15.188:3300/0 conn(0x7fc5f00d3850 0x7fc5f00afed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:22:25.967 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:25.964+0000 7fc5ef7fe640 1 --2- 172.21.15.143:0/4270556960 >> v2:172.21.15.188:3300/0 conn(0x7fc5f00d3850 0x7fc5f00afed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:22:25.967 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:25.964+0000 7fc5ef7fe640 1 -- 172.21.15.143:0/4270556960 >> v1:172.21.15.188:6789/0 conn(0x7fc5f015d150 legacy=0x7fc5f00d3140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:22:25.967 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:25.964+0000 7fc5ef7fe640 1 -- 172.21.15.143:0/4270556960 --> v2:172.21.15.143:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc5f00b0e00 con 0x7fc5f015d530 2023-11-26T07:22:25.967 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:25.964+0000 7fc5ee7fc640 1 --2- 172.21.15.143:0/4270556960 >> v2:172.21.15.188:3300/0 conn(0x7fc5f00d3850 0x7fc5f00afed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-26T07:22:25.969 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:25.968+0000 7fc5ef7fe640 1 --2- 172.21.15.143:0/4270556960 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fc5f015d530 0x7fc5f00b0410 secure :-1 s=READY pgs=901 cs=0 l=1 rev1=1 crypto rx=0x7fc5e8009e80 tx=0x7fc5e801f1a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e57a77680085c1e server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:22:25.969 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:25.968+0000 7fc5edffb640 1 -- 172.21.15.143:0/4270556960 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc5e80af400 con 0x7fc5f015d530 2023-11-26T07:22:25.969 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:25.968+0000 7fc5edffb640 1 -- 172.21.15.143:0/4270556960 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc5e80ab050 con 0x7fc5f015d530 2023-11-26T07:22:25.969 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:25.968+0000 7fc5edffb640 1 -- 172.21.15.143:0/4270556960 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc5e80af780 con 0x7fc5f015d530 2023-11-26T07:22:25.970 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:25.968+0000 7fc5f5616640 1 -- 172.21.15.143:0/4270556960 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fc5f015d530 msgr2=0x7fc5f00b0410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:22:25.970 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:25.968+0000 7fc5f5616640 1 --2- 172.21.15.143:0/4270556960 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fc5f015d530 0x7fc5f00b0410 secure :-1 s=READY pgs=901 cs=0 l=1 rev1=1 crypto rx=0x7fc5e8009e80 tx=0x7fc5e801f1a0 comp rx=0 tx=0).stop 2023-11-26T07:22:25.970 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:25.968+0000 7fc5f5616640 1 -- 172.21.15.143:0/4270556960 shutdown_connections 2023-11-26T07:22:25.970 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:25.968+0000 7fc5f5616640 1 --2- 172.21.15.143:0/4270556960 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fc5f015d530 0x7fc5f00b0410 unknown :-1 s=CLOSED pgs=901 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:22:25.971 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:25.968+0000 7fc5f5616640 1 --2- 172.21.15.143:0/4270556960 >> v2:172.21.15.188:3300/0 conn(0x7fc5f00d3850 0x7fc5f00afed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:22:25.971 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:25.968+0000 7fc5f5616640 1 -- 172.21.15.143:0/4270556960 >> 172.21.15.143:0/4270556960 conn(0x7fc5f008ccc0 msgr2=0x7fc5f009c5e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:22:25.971 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:25.968+0000 7fc5f5616640 1 -- 172.21.15.143:0/4270556960 shutdown_connections 2023-11-26T07:22:25.971 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:25.968+0000 7fc5f5616640 1 -- 172.21.15.143:0/4270556960 wait complete. 2023-11-26T07:22:25.971 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:25.972+0000 7fc5f5616640 1 Processor -- start 2023-11-26T07:22:25.972 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:25.972+0000 7fc5f5616640 1 -- start start 2023-11-26T07:22:25.973 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:25.972+0000 7fc5f5616640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fc5f00d3850 0x7fc5f00cd1b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:22:25.973 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:25.972+0000 7fc5f5616640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fc5f015d150 0x7fc5f00ce110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:22:25.973 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:25.972+0000 7fc5f5616640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fc5f015d530 0x7fc5f00cb670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:22:25.974 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:25.972+0000 7fc5f5616640 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc5f00cc130 con 0x7fc5f015d530 2023-11-26T07:22:25.974 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:25.972+0000 7fc5f5616640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc5f00cc410 con 0x7fc5f00d3850 2023-11-26T07:22:25.974 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:25.972+0000 7fc5f5616640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc5f00cc6f0 con 0x7fc5f015d150 2023-11-26T07:22:25.974 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:25.972+0000 7fc5ef7fe640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fc5f015d530 0x7fc5f00cb670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:22:25.974 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:25.972+0000 7fc5ef7fe640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fc5f015d530 0x7fc5f00cb670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:47434/0 (socket says 172.21.15.143:47434) 2023-11-26T07:22:25.974 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:25.972+0000 7fc5ef7fe640 1 -- 172.21.15.143:0/131299293 learned_addr learned my addr 172.21.15.143:0/131299293 (peer_addr_for_me v2:172.21.15.143:0/0) 2023-11-26T07:22:25.974 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:25.972+0000 7fc5eeffd640 1 --2- 172.21.15.143:0/131299293 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fc5f00d3850 0x7fc5f00cd1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:22:25.974 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:25.972+0000 7fc5ee7fc640 1 --2- 172.21.15.143:0/131299293 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fc5f015d150 0x7fc5f00ce110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:22:25.974 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:25.972+0000 7fc5ef7fe640 1 -- 172.21.15.143:0/131299293 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fc5f015d150 msgr2=0x7fc5f00ce110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:22:25.974 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:25.972+0000 7fc5ef7fe640 1 --2- 172.21.15.143:0/131299293 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fc5f015d150 0x7fc5f00ce110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:22:25.974 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:25.972+0000 7fc5ef7fe640 1 -- 172.21.15.143:0/131299293 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fc5f00d3850 msgr2=0x7fc5f00cd1b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:22:25.974 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:25.972+0000 7fc5ef7fe640 1 --2- 172.21.15.143:0/131299293 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fc5f00d3850 0x7fc5f00cd1b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:22:25.975 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:25.972+0000 7fc5ef7fe640 1 -- 172.21.15.143:0/131299293 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc5e8009b30 con 0x7fc5f015d530 2023-11-26T07:22:25.975 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:25.972+0000 7fc5ef7fe640 1 --2- 172.21.15.143:0/131299293 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fc5f015d530 0x7fc5f00cb670 secure :-1 s=READY pgs=902 cs=0 l=1 rev1=1 crypto rx=0x7fc5e80b2040 tx=0x7fc5e801bf40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:22:25.975 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:25.972+0000 7fc5d77fe640 1 -- 172.21.15.143:0/131299293 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc5e80afcf0 con 0x7fc5f015d530 2023-11-26T07:22:25.975 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:25.972+0000 7fc5d77fe640 1 -- 172.21.15.143:0/131299293 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc5e80afe50 con 0x7fc5f015d530 2023-11-26T07:22:25.976 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:25.972+0000 7fc5d77fe640 1 -- 172.21.15.143:0/131299293 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc5e80ba660 con 0x7fc5f015d530 2023-11-26T07:22:25.976 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:25.972+0000 7fc5f5616640 1 -- 172.21.15.143:0/131299293 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc5f00cc9d0 con 0x7fc5f015d530 2023-11-26T07:22:25.976 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:25.972+0000 7fc5f5616640 1 -- 172.21.15.143:0/131299293 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc5f00b89d0 con 0x7fc5f015d530 2023-11-26T07:22:25.976 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:25.972+0000 7fc5f5616640 1 -- 172.21.15.143:0/131299293 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc5f0067500 con 0x7fc5f015d530 2023-11-26T07:22:25.976 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:25.976+0000 7fc5d77fe640 1 -- 172.21.15.143:0/131299293 <== mon.0 v2:172.21.15.143:3300/0 4 ==== mgrmap(e 11) v1 ==== 86158+0+0 (secure 0 0 0) 0x7fc5e80b6040 con 0x7fc5f015d530 2023-11-26T07:22:25.977 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:25.976+0000 7fc5d77fe640 1 --2- 172.21.15.143:0/131299293 >> [v2:172.21.15.143:6832/2150557544,v1:172.21.15.143:6833/2150557544] conn(0x7fc5dc0667d0 0x7fc5dc068c90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:22:25.977 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:25.976+0000 7fc5d77fe640 1 -- 172.21.15.143:0/131299293 <== mon.0 v2:172.21.15.143:3300/0 5 ==== osd_map(60..60 src has 1..60) v4 ==== 5795+0+0 (secure 0 0 0) 0x7fc5e81223b0 con 0x7fc5f015d530 2023-11-26T07:22:25.977 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:25.976+0000 7fc5eeffd640 1 --2- 172.21.15.143:0/131299293 >> [v2:172.21.15.143:6832/2150557544,v1:172.21.15.143:6833/2150557544] conn(0x7fc5dc0667d0 0x7fc5dc068c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:22:25.977 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:25.976+0000 7fc5eeffd640 1 --2- 172.21.15.143:0/131299293 >> [v2:172.21.15.143:6832/2150557544,v1:172.21.15.143:6833/2150557544] conn(0x7fc5dc0667d0 0x7fc5dc068c90 secure :-1 s=READY pgs=424 cs=0 l=1 rev1=1 crypto rx=0x7fc5e0007fd0 tx=0x7fc5e0009040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:22:25.981 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:25.980+0000 7fc5d77fe640 1 -- 172.21.15.143:0/131299293 <== mon.0 v2:172.21.15.143:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172649 (secure 0 0 0) 0x7fc5e80ef2b0 con 0x7fc5f015d530 2023-11-26T07:22:26.133 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:26.132+0000 7fc5f5616640 1 -- 172.21.15.143:0/131299293 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7fc5f00d00d0 con 0x7fc5f015d530 2023-11-26T07:22:26.799 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:26.796+0000 7fc5d77fe640 1 -- 172.21.15.143:0/131299293 <== mon.0 v2:172.21.15.143:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_metadata' removed v61) v1 ==== 181+0+0 (secure 0 0 0) 0x7fc5e8009510 con 0x7fc5f015d530 2023-11-26T07:22:26.799 INFO:teuthology.orchestra.run.smithi143.stderr:pool 'cephfs_metadata' removed 2023-11-26T07:22:26.801 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:26.800+0000 7fc5f5616640 1 -- 172.21.15.143:0/131299293 >> [v2:172.21.15.143:6832/2150557544,v1:172.21.15.143:6833/2150557544] conn(0x7fc5dc0667d0 msgr2=0x7fc5dc068c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:22:26.802 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:26.800+0000 7fc5f5616640 1 --2- 172.21.15.143:0/131299293 >> [v2:172.21.15.143:6832/2150557544,v1:172.21.15.143:6833/2150557544] conn(0x7fc5dc0667d0 0x7fc5dc068c90 secure :-1 s=READY pgs=424 cs=0 l=1 rev1=1 crypto rx=0x7fc5e0007fd0 tx=0x7fc5e0009040 comp rx=0 tx=0).stop 2023-11-26T07:22:26.802 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:26.800+0000 7fc5f5616640 1 -- 172.21.15.143:0/131299293 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fc5f015d530 msgr2=0x7fc5f00cb670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:22:26.802 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:26.800+0000 7fc5f5616640 1 --2- 172.21.15.143:0/131299293 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fc5f015d530 0x7fc5f00cb670 secure :-1 s=READY pgs=902 cs=0 l=1 rev1=1 crypto rx=0x7fc5e80b2040 tx=0x7fc5e801bf40 comp rx=0 tx=0).stop 2023-11-26T07:22:26.802 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:26.800+0000 7fc5f5616640 1 -- 172.21.15.143:0/131299293 shutdown_connections 2023-11-26T07:22:26.802 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:26.800+0000 7fc5f5616640 1 --2- 172.21.15.143:0/131299293 >> [v2:172.21.15.143:6832/2150557544,v1:172.21.15.143:6833/2150557544] conn(0x7fc5dc0667d0 0x7fc5dc068c90 unknown :-1 s=CLOSED pgs=424 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:22:26.802 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:26.800+0000 7fc5f5616640 1 --2- 172.21.15.143:0/131299293 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fc5f015d530 0x7fc5f00cb670 unknown :-1 s=CLOSED pgs=902 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:22:26.802 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:26.800+0000 7fc5f5616640 1 --2- 172.21.15.143:0/131299293 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fc5f015d150 0x7fc5f00ce110 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:22:26.803 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:26.800+0000 7fc5f5616640 1 --2- 172.21.15.143:0/131299293 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fc5f00d3850 0x7fc5f00cd1b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:22:26.803 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:26.800+0000 7fc5f5616640 1 -- 172.21.15.143:0/131299293 >> 172.21.15.143:0/131299293 conn(0x7fc5f008ccc0 msgr2=0x7fc5f00b31d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:22:26.803 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:26.800+0000 7fc5f5616640 1 -- 172.21.15.143:0/131299293 shutdown_connections 2023-11-26T07:22:26.803 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:26.800+0000 7fc5f5616640 1 -- 172.21.15.143:0/131299293 wait complete. 2023-11-26T07:22:26.827 DEBUG:teuthology.orchestra.run.smithi143:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_data cephfs_data --yes-i-really-really-mean-it 2023-11-26T07:22:26.972 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:26.968+0000 7f1ff359e640 1 Processor -- start 2023-11-26T07:22:26.972 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:26.972+0000 7f1ff359e640 1 -- start start 2023-11-26T07:22:26.974 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:26.972+0000 7f1ff359e640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f1ff4143810 0x7f1ff4072140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:22:26.974 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:26.972+0000 7f1ff359e640 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7f1ff40b1e10 0x7f1ff415d9e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:22:26.974 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:26.972+0000 7f1ff359e640 1 -- --> v1:172.21.15.143:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1ff415e860 con 0x7f1ff415df20 2023-11-26T07:22:26.974 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:26.972+0000 7f1ff359e640 1 -- --> v2:172.21.15.143:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1ff415e3b0 con 0x7f1ff40b1e10 2023-11-26T07:22:26.974 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:26.972+0000 7f1ff359e640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1ff40cdba0 con 0x7f1ff4143810 2023-11-26T07:22:26.974 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:26.972+0000 7f1ff1d9b640 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7f1ff40b1e10 0x7f1ff415d9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:22:26.974 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:26.972+0000 7f1ff2d9d640 1 --1- >> v1:172.21.15.143:6789/0 conn(0x7f1ff415df20 0x7f1ff40ce030 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.143:6789/0 says I am v1:172.21.15.143:60212/0 (socket says 172.21.15.143:60212) 2023-11-26T07:22:26.974 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:26.972+0000 7f1ff2d9d640 1 -- 172.21.15.143:0/2083062877 learned_addr learned my addr 172.21.15.143:0/2083062877 (peer_addr_for_me v1:172.21.15.143:0/0) 2023-11-26T07:22:26.974 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:26.972+0000 7f1ff259c640 1 --2- 172.21.15.143:0/2083062877 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f1ff4143810 0x7f1ff4072140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:22:26.974 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:26.972+0000 7f1ff1d9b640 1 -- 172.21.15.143:0/2083062877 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f1ff4143810 msgr2=0x7f1ff4072140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-26T07:22:26.975 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:26.972+0000 7f1ff1d9b640 1 --2- 172.21.15.143:0/2083062877 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f1ff4143810 0x7f1ff4072140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:22:26.975 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:26.972+0000 7f1ff1d9b640 1 -- 172.21.15.143:0/2083062877 >> v1:172.21.15.143:6789/0 conn(0x7f1ff415df20 legacy=0x7f1ff40ce030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:22:26.975 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:26.972+0000 7f1ff1d9b640 1 -- 172.21.15.143:0/2083062877 --> v2:172.21.15.143:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1ff4143bf0 con 0x7f1ff40b1e10 2023-11-26T07:22:26.977 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:26.976+0000 7f1ff1d9b640 1 --2- 172.21.15.143:0/2083062877 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f1ff40b1e10 0x7f1ff415d9e0 secure :-1 s=READY pgs=904 cs=0 l=1 rev1=1 crypto rx=0x7f1fe80097c0 tx=0x7f1fe801c450 comp rx=0 tx=0).ready entity=mon.0 client_cookie=473658856503787 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:22:26.977 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:26.976+0000 7f1ff159a640 1 -- 172.21.15.143:0/2083062877 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1fe80ab670 con 0x7f1ff40b1e10 2023-11-26T07:22:26.977 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:26.976+0000 7f1ff159a640 1 -- 172.21.15.143:0/2083062877 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1fe80ab7d0 con 0x7f1ff40b1e10 2023-11-26T07:22:26.977 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:26.976+0000 7f1ff159a640 1 -- 172.21.15.143:0/2083062877 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1fe80abb60 con 0x7f1ff40b1e10 2023-11-26T07:22:26.978 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:26.976+0000 7f1ff359e640 1 -- 172.21.15.143:0/2083062877 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f1ff40b1e10 msgr2=0x7f1ff415d9e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:22:26.978 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:26.976+0000 7f1ff359e640 1 --2- 172.21.15.143:0/2083062877 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f1ff40b1e10 0x7f1ff415d9e0 secure :-1 s=READY pgs=904 cs=0 l=1 rev1=1 crypto rx=0x7f1fe80097c0 tx=0x7f1fe801c450 comp rx=0 tx=0).stop 2023-11-26T07:22:26.978 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:26.976+0000 7f1ff359e640 1 -- 172.21.15.143:0/2083062877 shutdown_connections 2023-11-26T07:22:26.978 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:26.976+0000 7f1ff359e640 1 --2- 172.21.15.143:0/2083062877 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f1ff40b1e10 0x7f1ff415d9e0 unknown :-1 s=CLOSED pgs=904 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:22:26.978 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:26.976+0000 7f1ff359e640 1 --2- 172.21.15.143:0/2083062877 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f1ff4143810 0x7f1ff4072140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:22:26.978 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:26.976+0000 7f1ff359e640 1 -- 172.21.15.143:0/2083062877 >> 172.21.15.143:0/2083062877 conn(0x7f1ff406a890 msgr2=0x7f1ff406aca0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:22:26.978 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:26.976+0000 7f1ff359e640 1 -- 172.21.15.143:0/2083062877 shutdown_connections 2023-11-26T07:22:26.979 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:26.976+0000 7f1ff359e640 1 -- 172.21.15.143:0/2083062877 wait complete. 2023-11-26T07:22:26.979 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:26.976+0000 7f1ff359e640 1 Processor -- start 2023-11-26T07:22:26.979 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:26.980+0000 7f1ff359e640 1 -- start start 2023-11-26T07:22:26.982 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:26.980+0000 7f1ff359e640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f1ff40b1e10 0x7f1ff41f5ba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:22:26.982 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:26.980+0000 7f1ff359e640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f1ff4143810 0x7f1ff40b0af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:22:26.982 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:26.980+0000 7f1ff359e640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f1ff415df20 0x7f1ff40b1030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:22:26.982 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:26.980+0000 7f1ff359e640 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1ff4152440 con 0x7f1ff415df20 2023-11-26T07:22:26.982 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:26.980+0000 7f1ff359e640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1ff4152720 con 0x7f1ff4143810 2023-11-26T07:22:26.982 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:26.980+0000 7f1ff359e640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1ff4152a00 con 0x7f1ff40b1e10 2023-11-26T07:22:26.982 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:26.980+0000 7f1ff2d9d640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f1ff415df20 0x7f1ff40b1030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:22:26.982 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:26.980+0000 7f1ff2d9d640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f1ff415df20 0x7f1ff40b1030 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:47448/0 (socket says 172.21.15.143:47448) 2023-11-26T07:22:26.982 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:26.980+0000 7f1ff2d9d640 1 -- 172.21.15.143:0/3901735612 learned_addr learned my addr 172.21.15.143:0/3901735612 (peer_addr_for_me v2:172.21.15.143:0/0) 2023-11-26T07:22:26.983 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:26.980+0000 7f1ff1d9b640 1 --2- 172.21.15.143:0/3901735612 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f1ff4143810 0x7f1ff40b0af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:22:26.983 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:26.980+0000 7f1ff259c640 1 --2- 172.21.15.143:0/3901735612 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f1ff40b1e10 0x7f1ff41f5ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:22:26.983 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:26.980+0000 7f1ff2d9d640 1 -- 172.21.15.143:0/3901735612 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f1ff40b1e10 msgr2=0x7f1ff41f5ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:22:26.983 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:26.980+0000 7f1ff2d9d640 1 --2- 172.21.15.143:0/3901735612 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f1ff40b1e10 0x7f1ff41f5ba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:22:26.983 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:26.980+0000 7f1ff2d9d640 1 -- 172.21.15.143:0/3901735612 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f1ff4143810 msgr2=0x7f1ff40b0af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:22:26.983 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:26.980+0000 7f1ff2d9d640 1 --2- 172.21.15.143:0/3901735612 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f1ff4143810 0x7f1ff40b0af0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:22:26.983 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:26.980+0000 7f1ff2d9d640 1 -- 172.21.15.143:0/3901735612 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1fe8009420 con 0x7f1ff415df20 2023-11-26T07:22:26.983 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:26.980+0000 7f1ff1d9b640 1 --2- 172.21.15.143:0/3901735612 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f1ff4143810 0x7f1ff40b0af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-26T07:22:26.983 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:26.980+0000 7f1ff2d9d640 1 --2- 172.21.15.143:0/3901735612 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f1ff415df20 0x7f1ff40b1030 secure :-1 s=READY pgs=905 cs=0 l=1 rev1=1 crypto rx=0x7f1fec00cbc0 tx=0x7f1fec00a430 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:22:26.983 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:26.980+0000 7f1fe2ffd640 1 -- 172.21.15.143:0/3901735612 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1fec00ad40 con 0x7f1ff415df20 2023-11-26T07:22:26.983 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:26.980+0000 7f1fe2ffd640 1 -- 172.21.15.143:0/3901735612 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1fec012400 con 0x7f1ff415df20 2023-11-26T07:22:26.983 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:26.980+0000 7f1fe2ffd640 1 -- 172.21.15.143:0/3901735612 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1fec00aea0 con 0x7f1ff415df20 2023-11-26T07:22:26.984 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:26.980+0000 7f1ff359e640 1 -- 172.21.15.143:0/3901735612 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1ff413cc80 con 0x7f1ff415df20 2023-11-26T07:22:26.984 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:26.980+0000 7f1ff359e640 1 -- 172.21.15.143:0/3901735612 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1ff41516d0 con 0x7f1ff415df20 2023-11-26T07:22:26.984 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:26.980+0000 7f1ff359e640 1 -- 172.21.15.143:0/3901735612 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1ff4067500 con 0x7f1ff415df20 2023-11-26T07:22:26.984 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:26.984+0000 7f1fe2ffd640 1 -- 172.21.15.143:0/3901735612 <== mon.0 v2:172.21.15.143:3300/0 4 ==== mgrmap(e 11) v1 ==== 86158+0+0 (secure 0 0 0) 0x7f1fec007070 con 0x7f1ff415df20 2023-11-26T07:22:26.985 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:26.984+0000 7f1fe2ffd640 1 --2- 172.21.15.143:0/3901735612 >> [v2:172.21.15.143:6832/2150557544,v1:172.21.15.143:6833/2150557544] conn(0x7f1fd4066800 0x7f1fd4068cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:22:26.985 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:26.984+0000 7f1fe2ffd640 1 -- 172.21.15.143:0/3901735612 <== mon.0 v2:172.21.15.143:3300/0 5 ==== osd_map(61..61 src has 1..61) v4 ==== 5357+0+0 (secure 0 0 0) 0x7f1fec0841b0 con 0x7f1ff415df20 2023-11-26T07:22:26.985 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:26.984+0000 7f1ff259c640 1 --2- 172.21.15.143:0/3901735612 >> [v2:172.21.15.143:6832/2150557544,v1:172.21.15.143:6833/2150557544] conn(0x7f1fd4066800 0x7f1fd4068cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:22:26.985 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:26.984+0000 7f1ff259c640 1 --2- 172.21.15.143:0/3901735612 >> [v2:172.21.15.143:6832/2150557544,v1:172.21.15.143:6833/2150557544] conn(0x7f1fd4066800 0x7f1fd4068cc0 secure :-1 s=READY pgs=425 cs=0 l=1 rev1=1 crypto rx=0x7f1fe400b930 tx=0x7f1fe4008040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:22:26.989 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:26.988+0000 7f1fe2ffd640 1 -- 172.21.15.143:0/3901735612 <== mon.0 v2:172.21.15.143:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172649 (secure 0 0 0) 0x7f1fec0512f0 con 0x7f1ff415df20 2023-11-26T07:22:27.138 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:27.136+0000 7f1ff359e640 1 -- 172.21.15.143:0/3901735612 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7f1ff4152580 con 0x7f1ff415df20 2023-11-26T07:22:27.801 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:27.800+0000 7f1fe2ffd640 1 -- 172.21.15.143:0/3901735612 <== mon.0 v2:172.21.15.143:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_data' removed v62) v1 ==== 169+0+0 (secure 0 0 0) 0x7f1ff4152580 con 0x7f1ff415df20 2023-11-26T07:22:27.801 INFO:teuthology.orchestra.run.smithi143.stderr:pool 'cephfs_data' removed 2023-11-26T07:22:27.805 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:27.804+0000 7f1ff359e640 1 -- 172.21.15.143:0/3901735612 >> [v2:172.21.15.143:6832/2150557544,v1:172.21.15.143:6833/2150557544] conn(0x7f1fd4066800 msgr2=0x7f1fd4068cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:22:27.805 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:27.804+0000 7f1ff359e640 1 --2- 172.21.15.143:0/3901735612 >> [v2:172.21.15.143:6832/2150557544,v1:172.21.15.143:6833/2150557544] conn(0x7f1fd4066800 0x7f1fd4068cc0 secure :-1 s=READY pgs=425 cs=0 l=1 rev1=1 crypto rx=0x7f1fe400b930 tx=0x7f1fe4008040 comp rx=0 tx=0).stop 2023-11-26T07:22:27.805 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:27.804+0000 7f1ff359e640 1 -- 172.21.15.143:0/3901735612 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f1ff415df20 msgr2=0x7f1ff40b1030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:22:27.805 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:27.804+0000 7f1ff359e640 1 --2- 172.21.15.143:0/3901735612 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f1ff415df20 0x7f1ff40b1030 secure :-1 s=READY pgs=905 cs=0 l=1 rev1=1 crypto rx=0x7f1fec00cbc0 tx=0x7f1fec00a430 comp rx=0 tx=0).stop 2023-11-26T07:22:27.805 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:27.804+0000 7f1ff359e640 1 -- 172.21.15.143:0/3901735612 shutdown_connections 2023-11-26T07:22:27.806 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:27.804+0000 7f1ff359e640 1 --2- 172.21.15.143:0/3901735612 >> [v2:172.21.15.143:6832/2150557544,v1:172.21.15.143:6833/2150557544] conn(0x7f1fd4066800 0x7f1fd4068cc0 unknown :-1 s=CLOSED pgs=425 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:22:27.806 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:27.804+0000 7f1ff359e640 1 --2- 172.21.15.143:0/3901735612 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f1ff415df20 0x7f1ff40b1030 unknown :-1 s=CLOSED pgs=905 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:22:27.806 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:27.804+0000 7f1ff359e640 1 --2- 172.21.15.143:0/3901735612 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f1ff4143810 0x7f1ff40b0af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:22:27.806 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:27.804+0000 7f1ff359e640 1 --2- 172.21.15.143:0/3901735612 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f1ff40b1e10 0x7f1ff41f5ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:22:27.806 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:27.804+0000 7f1ff359e640 1 -- 172.21.15.143:0/3901735612 >> 172.21.15.143:0/3901735612 conn(0x7f1ff406a890 msgr2=0x7f1ff40706e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:22:27.806 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:27.804+0000 7f1ff359e640 1 -- 172.21.15.143:0/3901735612 shutdown_connections 2023-11-26T07:22:27.806 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:27.804+0000 7f1ff359e640 1 -- 172.21.15.143:0/3901735612 wait complete. 2023-11-26T07:22:27.828 DEBUG:teuthology.orchestra.run.smithi143:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config rm mon mon_allow_pool_delete 2023-11-26T07:22:27.968 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:27.968+0000 7f12b8daa640 1 Processor -- start 2023-11-26T07:22:27.968 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:27.968+0000 7f12b8daa640 1 -- start start 2023-11-26T07:22:27.970 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:27.968+0000 7f12b8daa640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f12b40cf990 0x7f12b415efa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:22:27.970 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:27.968+0000 7f12b8daa640 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7f12b415f4e0 0x7f12b40cd7b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:22:27.970 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:27.968+0000 7f12b8daa640 1 -- --> v1:172.21.15.188:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f12b406c520 con 0x7f12b413e290 2023-11-26T07:22:27.970 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:27.968+0000 7f12b8daa640 1 -- --> v2:172.21.15.188:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f12b406c800 con 0x7f12b415f4e0 2023-11-26T07:22:27.970 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:27.968+0000 7f12b8daa640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f12b406cae0 con 0x7f12b40cf990 2023-11-26T07:22:27.971 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:27.968+0000 7f12b2d9d640 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7f12b415f4e0 0x7f12b40cd7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:22:27.971 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:27.968+0000 7f12b259c640 1 --1- >> v1:172.21.15.188:6789/0 conn(0x7f12b413e290 0x7f12b40cf280 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.188:6789/0 says I am v1:172.21.15.143:34962/0 (socket says 172.21.15.143:34962) 2023-11-26T07:22:27.971 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:27.968+0000 7f12b259c640 1 -- 172.21.15.143:0/1024736074 learned_addr learned my addr 172.21.15.143:0/1024736074 (peer_addr_for_me v1:172.21.15.143:0/0) 2023-11-26T07:22:27.971 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:27.968+0000 7f12b1d9b640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f12b40cf990 0x7f12b415efa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:22:27.971 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:27.968+0000 7f12b1d9b640 1 -- 172.21.15.143:0/1024736074 >> v2:172.21.15.188:3300/0 conn(0x7f12b415f4e0 msgr2=0x7f12b40cd7b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-26T07:22:27.971 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:27.968+0000 7f12b1d9b640 1 --2- 172.21.15.143:0/1024736074 >> v2:172.21.15.188:3300/0 conn(0x7f12b415f4e0 0x7f12b40cd7b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:22:27.971 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:27.968+0000 7f12b159a640 1 -- 172.21.15.143:0/1024736074 <== mon.1 v1:172.21.15.188:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1508611277 0 0) 0x7f12b406c520 con 0x7f12b413e290 2023-11-26T07:22:27.971 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:27.968+0000 7f12b1d9b640 1 -- 172.21.15.143:0/1024736074 >> v1:172.21.15.188:6789/0 conn(0x7f12b413e290 legacy=0x7f12b40cf280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:22:27.971 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:27.968+0000 7f12b1d9b640 1 -- 172.21.15.143:0/1024736074 --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f12b406cdc0 con 0x7f12b40cf990 2023-11-26T07:22:27.973 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:27.972+0000 7f12b1d9b640 1 --2- 172.21.15.143:0/1024736074 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f12b40cf990 0x7f12b415efa0 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7f12a80097c0 tx=0x7f12a801c450 comp rx=0 tx=0).ready entity=mon.2 client_cookie=ef0326c3b3c30650 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:22:27.974 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:27.972+0000 7f12b159a640 1 -- 172.21.15.143:0/1024736074 <== mon.2 v2:172.21.15.188:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f12a80ab670 con 0x7f12b40cf990 2023-11-26T07:22:27.974 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:27.972+0000 7f12b159a640 1 -- 172.21.15.143:0/1024736074 <== mon.2 v2:172.21.15.188:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f12a80ab7d0 con 0x7f12b40cf990 2023-11-26T07:22:27.974 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:27.972+0000 7f12b159a640 1 -- 172.21.15.143:0/1024736074 <== mon.2 v2:172.21.15.188:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f12a80abb60 con 0x7f12b40cf990 2023-11-26T07:22:27.975 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:27.972+0000 7f12b8daa640 1 -- 172.21.15.143:0/1024736074 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f12b40cf990 msgr2=0x7f12b415efa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:22:27.975 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:27.972+0000 7f12b8daa640 1 --2- 172.21.15.143:0/1024736074 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f12b40cf990 0x7f12b415efa0 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7f12a80097c0 tx=0x7f12a801c450 comp rx=0 tx=0).stop 2023-11-26T07:22:27.975 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:27.972+0000 7f12b8daa640 1 -- 172.21.15.143:0/1024736074 shutdown_connections 2023-11-26T07:22:27.975 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:27.972+0000 7f12b8daa640 1 --2- 172.21.15.143:0/1024736074 >> v2:172.21.15.188:3300/0 conn(0x7f12b415f4e0 0x7f12b40cd7b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:22:27.975 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:27.972+0000 7f12b8daa640 1 --2- 172.21.15.143:0/1024736074 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f12b40cf990 0x7f12b415efa0 unknown :-1 s=CLOSED pgs=115 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:22:27.976 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:27.972+0000 7f12b8daa640 1 -- 172.21.15.143:0/1024736074 >> 172.21.15.143:0/1024736074 conn(0x7f12b406a890 msgr2=0x7f12b406aca0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:22:27.976 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:27.972+0000 7f12b8daa640 1 -- 172.21.15.143:0/1024736074 shutdown_connections 2023-11-26T07:22:27.976 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:27.972+0000 7f12b8daa640 1 -- 172.21.15.143:0/1024736074 wait complete. 2023-11-26T07:22:27.977 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:27.976+0000 7f12b8daa640 1 Processor -- start 2023-11-26T07:22:27.977 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:27.976+0000 7f12b8daa640 1 -- start start 2023-11-26T07:22:27.979 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:27.976+0000 7f12b8daa640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f12b40cf990 0x7f12b4136db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:22:27.979 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:27.976+0000 7f12b8daa640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f12b413e290 0x7f12b4149310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:22:27.979 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:27.976+0000 7f12b8daa640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f12b415f4e0 0x7f12b4149850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:22:27.979 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:27.976+0000 7f12b8daa640 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f12b4070d10 con 0x7f12b413e290 2023-11-26T07:22:27.979 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:27.976+0000 7f12b8daa640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f12b40aeb60 con 0x7f12b415f4e0 2023-11-26T07:22:27.979 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:27.976+0000 7f12b8daa640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f12b41372f0 con 0x7f12b40cf990 2023-11-26T07:22:27.980 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:27.976+0000 7f12b1d9b640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f12b413e290 0x7f12b4149310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:22:27.980 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:27.976+0000 7f12b1d9b640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f12b413e290 0x7f12b4149310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:47464/0 (socket says 172.21.15.143:47464) 2023-11-26T07:22:27.980 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:27.976+0000 7f12b1d9b640 1 -- 172.21.15.143:0/3969479155 learned_addr learned my addr 172.21.15.143:0/3969479155 (peer_addr_for_me v2:172.21.15.143:0/0) 2023-11-26T07:22:27.980 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:27.976+0000 7f12b259c640 1 --2- 172.21.15.143:0/3969479155 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f12b40cf990 0x7f12b4136db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:22:27.980 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:27.976+0000 7f12b2d9d640 1 --2- 172.21.15.143:0/3969479155 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f12b415f4e0 0x7f12b4149850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:22:27.980 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:27.976+0000 7f12b1d9b640 1 -- 172.21.15.143:0/3969479155 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f12b40cf990 msgr2=0x7f12b4136db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:22:27.980 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:27.976+0000 7f12b1d9b640 1 --2- 172.21.15.143:0/3969479155 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f12b40cf990 0x7f12b4136db0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:22:27.980 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:27.976+0000 7f12b1d9b640 1 -- 172.21.15.143:0/3969479155 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f12b415f4e0 msgr2=0x7f12b4149850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:22:27.980 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:27.976+0000 7f12b1d9b640 1 --2- 172.21.15.143:0/3969479155 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f12b415f4e0 0x7f12b4149850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:22:27.980 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:27.976+0000 7f12b1d9b640 1 -- 172.21.15.143:0/3969479155 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f12a8009420 con 0x7f12b413e290 2023-11-26T07:22:27.980 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:27.976+0000 7f12b1d9b640 1 --2- 172.21.15.143:0/3969479155 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f12b413e290 0x7f12b4149310 secure :-1 s=READY pgs=906 cs=0 l=1 rev1=1 crypto rx=0x7f12a80ad040 tx=0x7f12a801ac90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:22:27.980 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:27.976+0000 7f12a2ffd640 1 -- 172.21.15.143:0/3969479155 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f12a80b5810 con 0x7f12b413e290 2023-11-26T07:22:27.980 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:27.976+0000 7f12a2ffd640 1 -- 172.21.15.143:0/3969479155 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f12a80b5990 con 0x7f12b413e290 2023-11-26T07:22:27.980 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:27.976+0000 7f12a2ffd640 1 -- 172.21.15.143:0/3969479155 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f12a80b5d40 con 0x7f12b413e290 2023-11-26T07:22:27.981 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:27.976+0000 7f12b8daa640 1 -- 172.21.15.143:0/3969479155 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f12b4150800 con 0x7f12b413e290 2023-11-26T07:22:27.981 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:27.976+0000 7f12b8daa640 1 -- 172.21.15.143:0/3969479155 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f12b4070650 con 0x7f12b413e290 2023-11-26T07:22:27.981 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:27.980+0000 7f12b8daa640 1 -- 172.21.15.143:0/3969479155 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f12b4067680 con 0x7f12b413e290 2023-11-26T07:22:27.981 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:27.980+0000 7f12a2ffd640 1 -- 172.21.15.143:0/3969479155 <== mon.0 v2:172.21.15.143:3300/0 4 ==== mgrmap(e 11) v1 ==== 86158+0+0 (secure 0 0 0) 0x7f12a80b4390 con 0x7f12b413e290 2023-11-26T07:22:27.982 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:27.980+0000 7f12a2ffd640 1 --2- 172.21.15.143:0/3969479155 >> [v2:172.21.15.143:6832/2150557544,v1:172.21.15.143:6833/2150557544] conn(0x7f12940667d0 0x7f1294068c90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:22:27.982 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:27.980+0000 7f12a2ffd640 1 -- 172.21.15.143:0/3969479155 <== mon.0 v2:172.21.15.143:3300/0 5 ==== osd_map(62..62 src has 1..62) v4 ==== 4943+0+0 (secure 0 0 0) 0x7f12b415fb00 con 0x7f12b413e290 2023-11-26T07:22:27.982 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:27.980+0000 7f12b259c640 1 --2- 172.21.15.143:0/3969479155 >> [v2:172.21.15.143:6832/2150557544,v1:172.21.15.143:6833/2150557544] conn(0x7f12940667d0 0x7f1294068c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:22:27.983 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:27.980+0000 7f12b259c640 1 --2- 172.21.15.143:0/3969479155 >> [v2:172.21.15.143:6832/2150557544,v1:172.21.15.143:6833/2150557544] conn(0x7f12940667d0 0x7f1294068c90 secure :-1 s=READY pgs=426 cs=0 l=1 rev1=1 crypto rx=0x7f12a4000f70 tx=0x7f12a4007b60 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:22:27.986 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:27.984+0000 7f12a2ffd640 1 -- 172.21.15.143:0/3969479155 <== mon.0 v2:172.21.15.143:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172649 (secure 0 0 0) 0x7f12a80ebb00 con 0x7f12b413e290 2023-11-26T07:22:28.142 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:28.140+0000 7f12b8daa640 1 -- 172.21.15.143:0/3969479155 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "config rm", "who": "mon", "name": "mon_allow_pool_delete"} v 0) v1 -- 0x7f12b406df50 con 0x7f12b413e290 2023-11-26T07:22:28.146 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:28.144+0000 7f12a2ffd640 1 -- 172.21.15.143:0/3969479155 <== mon.0 v2:172.21.15.143:3300/0 7 ==== mon_command_ack([{"prefix": "config rm", "who": "mon", "name": "mon_allow_pool_delete"}]=0 v15) v1 ==== 104+0+0 (secure 0 0 0) 0x7f12a80ab630 con 0x7f12b413e290 2023-11-26T07:22:28.149 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:28.148+0000 7f12b8daa640 1 -- 172.21.15.143:0/3969479155 >> [v2:172.21.15.143:6832/2150557544,v1:172.21.15.143:6833/2150557544] conn(0x7f12940667d0 msgr2=0x7f1294068c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:22:28.149 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:28.148+0000 7f12b8daa640 1 --2- 172.21.15.143:0/3969479155 >> [v2:172.21.15.143:6832/2150557544,v1:172.21.15.143:6833/2150557544] conn(0x7f12940667d0 0x7f1294068c90 secure :-1 s=READY pgs=426 cs=0 l=1 rev1=1 crypto rx=0x7f12a4000f70 tx=0x7f12a4007b60 comp rx=0 tx=0).stop 2023-11-26T07:22:28.149 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:28.148+0000 7f12b8daa640 1 -- 172.21.15.143:0/3969479155 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f12b413e290 msgr2=0x7f12b4149310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:22:28.149 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:28.148+0000 7f12b8daa640 1 --2- 172.21.15.143:0/3969479155 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f12b413e290 0x7f12b4149310 secure :-1 s=READY pgs=906 cs=0 l=1 rev1=1 crypto rx=0x7f12a80ad040 tx=0x7f12a801ac90 comp rx=0 tx=0).stop 2023-11-26T07:22:28.149 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:28.148+0000 7f12b8daa640 1 -- 172.21.15.143:0/3969479155 shutdown_connections 2023-11-26T07:22:28.149 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:28.148+0000 7f12b8daa640 1 --2- 172.21.15.143:0/3969479155 >> [v2:172.21.15.143:6832/2150557544,v1:172.21.15.143:6833/2150557544] conn(0x7f12940667d0 0x7f1294068c90 unknown :-1 s=CLOSED pgs=426 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:22:28.149 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:28.148+0000 7f12b8daa640 1 --2- 172.21.15.143:0/3969479155 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f12b415f4e0 0x7f12b4149850 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:22:28.150 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:28.148+0000 7f12b8daa640 1 --2- 172.21.15.143:0/3969479155 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f12b413e290 0x7f12b4149310 unknown :-1 s=CLOSED pgs=906 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:22:28.150 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:28.148+0000 7f12b8daa640 1 --2- 172.21.15.143:0/3969479155 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f12b40cf990 0x7f12b4136db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:22:28.150 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:28.148+0000 7f12b8daa640 1 -- 172.21.15.143:0/3969479155 >> 172.21.15.143:0/3969479155 conn(0x7f12b406a890 msgr2=0x7f12b40cf250 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:22:28.150 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:28.148+0000 7f12b8daa640 1 -- 172.21.15.143:0/3969479155 shutdown_connections 2023-11-26T07:22:28.150 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:28.148+0000 7f12b8daa640 1 -- 172.21.15.143:0/3969479155 wait complete. 2023-11-26T07:22:28.168 DEBUG:teuthology.orchestra.run.smithi143:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Ended test tasks.cephfs.test_volumes.TestMisc.test_malicious_metafile_on_legacy_to_v1_upgrade' 2023-11-26T07:22:28.307 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:28.304+0000 7f46d56d3640 1 Processor -- start 2023-11-26T07:22:28.307 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:28.304+0000 7f46d56d3640 1 -- start start 2023-11-26T07:22:28.309 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:28.308+0000 7f46d56d3640 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7f46d015d2a0 0x7f46d00d00e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:22:28.309 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:28.308+0000 7f46d56d3640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f46d015d680 0x7f46d00cec30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:22:28.309 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:28.308+0000 7f46d56d3640 1 -- --> v1:172.21.15.143:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f46d00cdc40 con 0x7f46d00d0e80 2023-11-26T07:22:28.309 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:28.308+0000 7f46d56d3640 1 -- --> v2:172.21.15.143:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f46d00ce7a0 con 0x7f46d015d2a0 2023-11-26T07:22:28.309 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:28.308+0000 7f46d56d3640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f46d00d1440 con 0x7f46d015d680 2023-11-26T07:22:28.309 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:28.308+0000 7f46cffff640 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7f46d015d2a0 0x7f46d00d00e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:22:28.309 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:28.308+0000 7f46cffff640 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7f46d015d2a0 0x7f46d00d00e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:47480/0 (socket says 172.21.15.143:47480) 2023-11-26T07:22:28.309 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:28.308+0000 7f46cffff640 1 -- 172.21.15.143:0/4288847193 learned_addr learned my addr 172.21.15.143:0/4288847193 (peer_addr_for_me v2:172.21.15.143:0/0) 2023-11-26T07:22:28.309 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:28.308+0000 7f46cf7fe640 1 --2- 172.21.15.143:0/4288847193 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f46d015d680 0x7f46d00cec30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:22:28.309 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:28.308+0000 7f46cffff640 1 -- 172.21.15.143:0/4288847193 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f46d015d680 msgr2=0x7f46d00cec30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-26T07:22:28.309 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:28.308+0000 7f46cffff640 1 --2- 172.21.15.143:0/4288847193 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f46d015d680 0x7f46d00cec30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:22:28.309 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:28.308+0000 7f46cffff640 1 -- 172.21.15.143:0/4288847193 >> v1:172.21.15.143:6789/0 conn(0x7f46d00d0e80 legacy=0x7f46d00d1680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:22:28.310 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:28.308+0000 7f46cffff640 1 -- 172.21.15.143:0/4288847193 --> v2:172.21.15.143:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f46d00cde70 con 0x7f46d015d2a0 2023-11-26T07:22:28.312 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:28.312+0000 7f46cffff640 1 --2- 172.21.15.143:0/4288847193 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f46d015d2a0 0x7f46d00d00e0 secure :-1 s=READY pgs=908 cs=0 l=1 rev1=1 crypto rx=0x7f46c0009870 tx=0x7f46c001c480 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1a99befe5d2dce01 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:22:28.312 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:28.312+0000 7f46ceffd640 1 -- 172.21.15.143:0/4288847193 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f46c00ab680 con 0x7f46d015d2a0 2023-11-26T07:22:28.312 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:28.312+0000 7f46ceffd640 1 -- 172.21.15.143:0/4288847193 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f46c00ab7e0 con 0x7f46d015d2a0 2023-11-26T07:22:28.312 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:28.312+0000 7f46ceffd640 1 -- 172.21.15.143:0/4288847193 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f46c00abb70 con 0x7f46d015d2a0 2023-11-26T07:22:28.313 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:28.312+0000 7f46d56d3640 1 -- 172.21.15.143:0/4288847193 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f46d015d2a0 msgr2=0x7f46d00d00e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:22:28.313 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:28.312+0000 7f46d56d3640 1 --2- 172.21.15.143:0/4288847193 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f46d015d2a0 0x7f46d00d00e0 secure :-1 s=READY pgs=908 cs=0 l=1 rev1=1 crypto rx=0x7f46c0009870 tx=0x7f46c001c480 comp rx=0 tx=0).stop 2023-11-26T07:22:28.313 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:28.312+0000 7f46d56d3640 1 -- 172.21.15.143:0/4288847193 shutdown_connections 2023-11-26T07:22:28.313 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:28.312+0000 7f46d56d3640 1 --2- 172.21.15.143:0/4288847193 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f46d015d680 0x7f46d00cec30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:22:28.313 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:28.312+0000 7f46d56d3640 1 --2- 172.21.15.143:0/4288847193 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f46d015d2a0 0x7f46d00d00e0 unknown :-1 s=CLOSED pgs=908 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:22:28.313 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:28.312+0000 7f46d56d3640 1 -- 172.21.15.143:0/4288847193 >> 172.21.15.143:0/4288847193 conn(0x7f46d00d30a0 msgr2=0x7f46d00d3480 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:22:28.313 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:28.312+0000 7f46d56d3640 1 -- 172.21.15.143:0/4288847193 shutdown_connections 2023-11-26T07:22:28.313 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:28.312+0000 7f46d56d3640 1 -- 172.21.15.143:0/4288847193 wait complete. 2023-11-26T07:22:28.314 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:28.312+0000 7f46d56d3640 1 Processor -- start 2023-11-26T07:22:28.314 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:28.312+0000 7f46d56d3640 1 -- start start 2023-11-26T07:22:28.316 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:28.312+0000 7f46d56d3640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f46d00d0e80 0x7f46d00cc920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:22:28.316 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:28.312+0000 7f46d56d3640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f46d015d2a0 0x7f46d0150210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:22:28.316 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:28.312+0000 7f46d56d3640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f46d015d680 0x7f46d0150750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:22:28.316 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:28.312+0000 7f46d56d3640 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f46d00c9a50 con 0x7f46d00d0e80 2023-11-26T07:22:28.317 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:28.312+0000 7f46d56d3640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f46d00c91b0 con 0x7f46d015d680 2023-11-26T07:22:28.317 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:28.312+0000 7f46d56d3640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f46d00c9490 con 0x7f46d015d2a0 2023-11-26T07:22:28.317 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:28.316+0000 7f46cffff640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f46d00d0e80 0x7f46d00cc920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:22:28.317 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:28.316+0000 7f46cffff640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f46d00d0e80 0x7f46d00cc920 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:47488/0 (socket says 172.21.15.143:47488) 2023-11-26T07:22:28.317 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:28.316+0000 7f46cffff640 1 -- 172.21.15.143:0/372096103 learned_addr learned my addr 172.21.15.143:0/372096103 (peer_addr_for_me v2:172.21.15.143:0/0) 2023-11-26T07:22:28.317 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:28.316+0000 7f46cf7fe640 1 --2- 172.21.15.143:0/372096103 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f46d015d2a0 0x7f46d0150210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:22:28.317 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:28.316+0000 7f46d4ed2640 1 --2- 172.21.15.143:0/372096103 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f46d015d680 0x7f46d0150750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:22:28.317 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:28.316+0000 7f46cffff640 1 -- 172.21.15.143:0/372096103 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f46d015d2a0 msgr2=0x7f46d0150210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:22:28.317 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:28.316+0000 7f46cffff640 1 --2- 172.21.15.143:0/372096103 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f46d015d2a0 0x7f46d0150210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:22:28.317 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:28.316+0000 7f46cffff640 1 -- 172.21.15.143:0/372096103 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f46d015d680 msgr2=0x7f46d0150750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:22:28.317 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:28.316+0000 7f46cffff640 1 --2- 172.21.15.143:0/372096103 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f46d015d680 0x7f46d0150750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:22:28.317 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:28.316+0000 7f46cffff640 1 -- 172.21.15.143:0/372096103 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f46c00094d0 con 0x7f46d00d0e80 2023-11-26T07:22:28.317 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:28.316+0000 7f46cffff640 1 --2- 172.21.15.143:0/372096103 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f46d00d0e80 0x7f46d00cc920 secure :-1 s=READY pgs=909 cs=0 l=1 rev1=1 crypto rx=0x7f46c00ad040 tx=0x7f46c001acc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:22:28.318 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:28.316+0000 7f46ccff9640 1 -- 172.21.15.143:0/372096103 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f46c00b4800 con 0x7f46d00d0e80 2023-11-26T07:22:28.318 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:28.316+0000 7f46ccff9640 1 -- 172.21.15.143:0/372096103 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f46c00b4980 con 0x7f46d00d0e80 2023-11-26T07:22:28.318 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:28.316+0000 7f46ccff9640 1 -- 172.21.15.143:0/372096103 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f46c00b4d30 con 0x7f46d00d0e80 2023-11-26T07:22:28.319 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:28.316+0000 7f46d56d3640 1 -- 172.21.15.143:0/372096103 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f46d00c9770 con 0x7f46d00d0e80 2023-11-26T07:22:28.319 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:28.316+0000 7f46d56d3640 1 -- 172.21.15.143:0/372096103 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f46d00c6ca0 con 0x7f46d00d0e80 2023-11-26T07:22:28.319 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:28.316+0000 7f46d56d3640 1 -- 172.21.15.143:0/372096103 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f46d00ca390 con 0x7f46d00d0e80 2023-11-26T07:22:28.319 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:28.316+0000 7f46ccff9640 1 -- 172.21.15.143:0/372096103 <== mon.0 v2:172.21.15.143:3300/0 4 ==== mgrmap(e 11) v1 ==== 86158+0+0 (secure 0 0 0) 0x7f46c00c7440 con 0x7f46d00d0e80 2023-11-26T07:22:28.320 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:28.320+0000 7f46ccff9640 1 --2- 172.21.15.143:0/372096103 >> [v2:172.21.15.143:6832/2150557544,v1:172.21.15.143:6833/2150557544] conn(0x7f46bc0667d0 0x7f46bc068c90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:22:28.320 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:28.320+0000 7f46ccff9640 1 -- 172.21.15.143:0/372096103 <== mon.0 v2:172.21.15.143:3300/0 5 ==== osd_map(62..62 src has 1..62) v4 ==== 4943+0+0 (secure 0 0 0) 0x7f46c011f040 con 0x7f46d00d0e80 2023-11-26T07:22:28.320 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:28.320+0000 7f46cf7fe640 1 --2- 172.21.15.143:0/372096103 >> [v2:172.21.15.143:6832/2150557544,v1:172.21.15.143:6833/2150557544] conn(0x7f46bc0667d0 0x7f46bc068c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:22:28.320 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:28.320+0000 7f46cf7fe640 1 --2- 172.21.15.143:0/372096103 >> [v2:172.21.15.143:6832/2150557544,v1:172.21.15.143:6833/2150557544] conn(0x7f46bc0667d0 0x7f46bc068c90 secure :-1 s=READY pgs=427 cs=0 l=1 rev1=1 crypto rx=0x7f46d0160e50 tx=0x7f46c4008040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:22:28.324 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:28.324+0000 7f46ccff9640 1 -- 172.21.15.143:0/372096103 <== mon.0 v2:172.21.15.143:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172649 (secure 0 0 0) 0x7f46d00ca390 con 0x7f46d00d0e80 2023-11-26T07:22:28.539 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:28.536+0000 7f46d56d3640 1 -- 172.21.15.143:0/372096103 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_volumes.TestMisc.test_malicious_metafile_on_legacy_to_v1_upgrade"]} v 0) v1 -- 0x7f46d00ca390 con 0x7f46d00d0e80 2023-11-26T07:22:28.804 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:28.804+0000 7f46ccff9640 1 -- 172.21.15.143:0/372096103 <== mon.0 v2:172.21.15.143:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_volumes.TestMisc.test_malicious_metafile_on_legacy_to_v1_upgrade"]}]=0 v242) v1 ==== 161+0+0 (secure 0 0 0) 0x7f46d00ca390 con 0x7f46d00d0e80 2023-11-26T07:22:28.807 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:28.804+0000 7f46d56d3640 1 -- 172.21.15.143:0/372096103 >> [v2:172.21.15.143:6832/2150557544,v1:172.21.15.143:6833/2150557544] conn(0x7f46bc0667d0 msgr2=0x7f46bc068c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:22:28.807 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:28.804+0000 7f46d56d3640 1 --2- 172.21.15.143:0/372096103 >> [v2:172.21.15.143:6832/2150557544,v1:172.21.15.143:6833/2150557544] conn(0x7f46bc0667d0 0x7f46bc068c90 secure :-1 s=READY pgs=427 cs=0 l=1 rev1=1 crypto rx=0x7f46d0160e50 tx=0x7f46c4008040 comp rx=0 tx=0).stop 2023-11-26T07:22:28.807 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:28.804+0000 7f46d56d3640 1 -- 172.21.15.143:0/372096103 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f46d00d0e80 msgr2=0x7f46d00cc920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:22:28.807 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:28.804+0000 7f46d56d3640 1 --2- 172.21.15.143:0/372096103 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f46d00d0e80 0x7f46d00cc920 secure :-1 s=READY pgs=909 cs=0 l=1 rev1=1 crypto rx=0x7f46c00ad040 tx=0x7f46c001acc0 comp rx=0 tx=0).stop 2023-11-26T07:22:28.807 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:28.804+0000 7f46d56d3640 1 -- 172.21.15.143:0/372096103 shutdown_connections 2023-11-26T07:22:28.807 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:28.804+0000 7f46d56d3640 1 --2- 172.21.15.143:0/372096103 >> [v2:172.21.15.143:6832/2150557544,v1:172.21.15.143:6833/2150557544] conn(0x7f46bc0667d0 0x7f46bc068c90 unknown :-1 s=CLOSED pgs=427 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:22:28.807 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:28.804+0000 7f46d56d3640 1 --2- 172.21.15.143:0/372096103 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f46d015d680 0x7f46d0150750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:22:28.807 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:28.804+0000 7f46d56d3640 1 --2- 172.21.15.143:0/372096103 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f46d015d2a0 0x7f46d0150210 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:22:28.807 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:28.804+0000 7f46d56d3640 1 --2- 172.21.15.143:0/372096103 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f46d00d0e80 0x7f46d00cc920 unknown :-1 s=CLOSED pgs=909 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:22:28.807 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:28.804+0000 7f46d56d3640 1 -- 172.21.15.143:0/372096103 >> 172.21.15.143:0/372096103 conn(0x7f46d00d30a0 msgr2=0x7f46d00d1680 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:22:28.807 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:28.804+0000 7f46d56d3640 1 -- 172.21.15.143:0/372096103 shutdown_connections 2023-11-26T07:22:28.808 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:28.804+0000 7f46d56d3640 1 -- 172.21.15.143:0/372096103 wait complete. 2023-11-26T07:22:28.827 INFO:tasks.cephfs_test_runner:Validate handcrafted .meta file on legacy subvol root doesn't break the system ... ok 2023-11-26T07:22:28.827 INFO:tasks.cephfs_test_runner:Starting test: test_mgr_eviction (tasks.cephfs.test_volumes.TestMisc) 2023-11-26T07:22:28.828 DEBUG:teuthology.orchestra.run.smithi143:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-11-26T07:22:28.966 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:28.964+0000 7f13926e4640 1 Processor -- start 2023-11-26T07:22:28.966 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:28.964+0000 7f13926e4640 1 -- start start 2023-11-26T07:22:28.967 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:28.964+0000 7f13926e4640 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7f138c0cf280 0x7f138c0cf710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:22:28.967 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:28.964+0000 7f13926e4640 1 -- --> v1:172.21.15.143:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f138c0cc730 con 0x7f138c0cd1b0 2023-11-26T07:22:28.967 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:28.964+0000 7f13926e4640 1 -- --> v1:172.21.15.188:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f138c070ee0 con 0x7f138c143ac0 2023-11-26T07:22:28.967 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:28.964+0000 7f13926e4640 1 -- --> v2:172.21.15.188:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f138c143ea0 con 0x7f138c0cf280 2023-11-26T07:22:28.967 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:28.964+0000 7f1390c81640 1 --1- >> v1:172.21.15.143:6789/0 conn(0x7f138c0cd1b0 0x7f138c0cd5c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.143:6789/0 says I am v1:172.21.15.143:60230/0 (socket says 172.21.15.143:60230) 2023-11-26T07:22:28.968 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:28.964+0000 7f1390c81640 1 -- 172.21.15.143:0/2104798205 learned_addr learned my addr 172.21.15.143:0/2104798205 (peer_addr_for_me v1:172.21.15.143:0/0) 2023-11-26T07:22:28.969 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:28.968+0000 7f138b7fe640 1 --2- 172.21.15.143:0/2104798205 >> v2:172.21.15.188:3300/0 conn(0x7f138c0cf280 0x7f138c0cf710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:22:28.970 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:28.968+0000 7f138affd640 1 -- 172.21.15.143:0/2104798205 <== mon.0 v1:172.21.15.143:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1885837498 0 0) 0x7f138c0cc730 con 0x7f138c0cd1b0 2023-11-26T07:22:28.970 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:28.968+0000 7f138affd640 1 -- 172.21.15.143:0/2104798205 --> v1:172.21.15.143:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f1374005740 con 0x7f138c0cd1b0 2023-11-26T07:22:28.970 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:28.968+0000 7f138affd640 1 -- 172.21.15.143:0/2104798205 <== mon.1 v1:172.21.15.188:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3622586645 0 0) 0x7f138c070ee0 con 0x7f138c143ac0 2023-11-26T07:22:28.970 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:28.968+0000 7f138affd640 1 -- 172.21.15.143:0/2104798205 --> v1:172.21.15.188:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f138c0cc730 con 0x7f138c143ac0 2023-11-26T07:22:28.970 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:28.968+0000 7f138affd640 1 -- 172.21.15.143:0/2104798205 <== mon.0 v1:172.21.15.143:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3221899720 0 0) 0x7f1374005740 con 0x7f138c0cd1b0 2023-11-26T07:22:28.970 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:28.968+0000 7f138affd640 1 -- 172.21.15.143:0/2104798205 >> v2:172.21.15.188:3300/0 conn(0x7f138c0cf280 msgr2=0x7f138c0cf710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-26T07:22:28.970 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:28.968+0000 7f138affd640 1 --2- 172.21.15.143:0/2104798205 >> v2:172.21.15.188:3300/0 conn(0x7f138c0cf280 0x7f138c0cf710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:22:28.970 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:28.968+0000 7f138affd640 1 -- 172.21.15.143:0/2104798205 >> v1:172.21.15.188:6789/0 conn(0x7f138c143ac0 legacy=0x7f138c070370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:22:28.970 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:28.968+0000 7f138affd640 1 -- 172.21.15.143:0/2104798205 --> v1:172.21.15.143:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f138c0711c0 con 0x7f138c0cd1b0 2023-11-26T07:22:28.970 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:28.968+0000 7f138affd640 1 -- 172.21.15.143:0/2104798205 <== mon.0 v1:172.21.15.143:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2137882523 0 0) 0x7f1384001a00 con 0x7f138c0cd1b0 2023-11-26T07:22:28.970 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:28.968+0000 7f138affd640 1 -- 172.21.15.143:0/2104798205 >> v1:172.21.15.143:6789/0 conn(0x7f138c0cd1b0 legacy=0x7f138c0cd5c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:22:28.970 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:28.968+0000 7f138affd640 1 --2- 172.21.15.143:0/2104798205 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f1374005d70 0x7f1374006170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:22:28.971 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:28.968+0000 7f138affd640 1 --2- 172.21.15.143:0/2104798205 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f13740069e0 0x7f1374006e60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:22:28.971 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:28.968+0000 7f138affd640 1 --2- 172.21.15.143:0/2104798205 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f13740073a0 0x7f137400dba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:22:28.971 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:28.968+0000 7f138affd640 1 -- 172.21.15.143:0/2104798205 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f137400e270 con 0x7f1374005d70 2023-11-26T07:22:28.971 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:28.968+0000 7f138affd640 1 -- 172.21.15.143:0/2104798205 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f137400e3b0 con 0x7f13740069e0 2023-11-26T07:22:28.971 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:28.968+0000 7f138affd640 1 -- 172.21.15.143:0/2104798205 --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f137400e570 con 0x7f13740073a0 2023-11-26T07:22:28.971 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:28.968+0000 7f138bfff640 1 --2- 172.21.15.143:0/2104798205 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f1374005d70 0x7f1374006170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:22:28.971 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:28.968+0000 7f1390c81640 1 --2- 172.21.15.143:0/2104798205 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f13740073a0 0x7f137400dba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:22:28.971 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:28.968+0000 7f138b7fe640 1 --2- 172.21.15.143:0/2104798205 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f13740069e0 0x7f1374006e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:22:28.971 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:28.968+0000 7f138bfff640 1 -- 172.21.15.143:0/2104798205 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f13740073a0 msgr2=0x7f137400dba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-26T07:22:28.971 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:28.968+0000 7f138bfff640 1 --2- 172.21.15.143:0/2104798205 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f13740073a0 0x7f137400dba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:22:28.971 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:28.968+0000 7f138bfff640 1 -- 172.21.15.143:0/2104798205 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f13740069e0 msgr2=0x7f1374006e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-26T07:22:28.971 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:28.968+0000 7f138bfff640 1 --2- 172.21.15.143:0/2104798205 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f13740069e0 0x7f1374006e60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:22:28.971 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:28.968+0000 7f138bfff640 1 -- 172.21.15.143:0/2104798205 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1374005740 con 0x7f1374005d70 2023-11-26T07:22:28.972 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:28.968+0000 7f138b7fe640 1 --2- 172.21.15.143:0/2104798205 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f13740069e0 0x7f1374006e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-26T07:22:28.972 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:28.968+0000 7f1390c81640 1 -- 172.21.15.143:0/2104798205 reap_dead start 2023-11-26T07:22:28.973 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:28.972+0000 7f138bfff640 1 --2- 172.21.15.143:0/2104798205 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f1374005d70 0x7f1374006170 secure :-1 s=READY pgs=911 cs=0 l=1 rev1=1 crypto rx=0x7f137c0018d0 tx=0x7f137c01c560 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ebc1533e937ecf87 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:22:28.973 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:28.972+0000 7f138affd640 1 -- 172.21.15.143:0/2104798205 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f137c0052d0 con 0x7f1374005d70 2023-11-26T07:22:28.973 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:28.972+0000 7f138affd640 1 -- 172.21.15.143:0/2104798205 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f137c005450 con 0x7f1374005d70 2023-11-26T07:22:28.973 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:28.972+0000 7f138affd640 1 -- 172.21.15.143:0/2104798205 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f137c003840 con 0x7f1374005d70 2023-11-26T07:22:28.973 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:28.972+0000 7f13926e4640 1 -- 172.21.15.143:0/2104798205 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f1374005d70 msgr2=0x7f1374006170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:22:28.973 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:28.972+0000 7f13926e4640 1 --2- 172.21.15.143:0/2104798205 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f1374005d70 0x7f1374006170 secure :-1 s=READY pgs=911 cs=0 l=1 rev1=1 crypto rx=0x7f137c0018d0 tx=0x7f137c01c560 comp rx=0 tx=0).stop 2023-11-26T07:22:28.973 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:28.972+0000 7f13926e4640 1 -- 172.21.15.143:0/2104798205 shutdown_connections 2023-11-26T07:22:28.973 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:28.972+0000 7f13926e4640 1 --2- 172.21.15.143:0/2104798205 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f1374005d70 0x7f1374006170 unknown :-1 s=CLOSED pgs=911 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:22:28.973 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:28.972+0000 7f13926e4640 1 -- 172.21.15.143:0/2104798205 >> 172.21.15.143:0/2104798205 conn(0x7f138c06a890 msgr2=0x7f138c06aca0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:22:28.973 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:28.972+0000 7f13926e4640 1 -- 172.21.15.143:0/2104798205 shutdown_connections 2023-11-26T07:22:28.973 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:28.972+0000 7f13926e4640 1 -- 172.21.15.143:0/2104798205 wait complete. 2023-11-26T07:22:28.974 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:28.972+0000 7f13926e4640 1 Processor -- start 2023-11-26T07:22:28.975 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:28.972+0000 7f13926e4640 1 -- start start 2023-11-26T07:22:28.977 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:28.972+0000 7f13926e4640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f1374005d70 0x7f138c153e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:22:28.977 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:28.972+0000 7f13926e4640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f138c13dce0 0x7f138c1fa2d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:22:28.977 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:28.972+0000 7f13926e4640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f138c13bf30 0x7f138c0ce2d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:22:28.977 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:28.976+0000 7f13926e4640 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f138c13e160 con 0x7f138c13dce0 2023-11-26T07:22:28.977 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:28.976+0000 7f13926e4640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f138c1fa810 con 0x7f1374005d70 2023-11-26T07:22:28.977 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:28.976+0000 7f13926e4640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f138c13e8b0 con 0x7f138c13bf30 2023-11-26T07:22:28.977 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:28.976+0000 7f138b7fe640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f138c13dce0 0x7f138c1fa2d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:22:28.977 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:28.976+0000 7f138b7fe640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f138c13dce0 0x7f138c1fa2d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:47506/0 (socket says 172.21.15.143:47506) 2023-11-26T07:22:28.978 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:28.976+0000 7f138b7fe640 1 -- 172.21.15.143:0/619547698 learned_addr learned my addr 172.21.15.143:0/619547698 (peer_addr_for_me v2:172.21.15.143:0/0) 2023-11-26T07:22:28.978 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:28.976+0000 7f138bfff640 1 --2- 172.21.15.143:0/619547698 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f1374005d70 0x7f138c153e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:22:28.978 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:28.976+0000 7f1390c81640 1 --2- 172.21.15.143:0/619547698 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f138c13bf30 0x7f138c0ce2d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:22:28.978 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:28.976+0000 7f138b7fe640 1 -- 172.21.15.143:0/619547698 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f138c13bf30 msgr2=0x7f138c0ce2d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:22:28.978 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:28.976+0000 7f138b7fe640 1 --2- 172.21.15.143:0/619547698 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f138c13bf30 0x7f138c0ce2d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:22:28.978 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:28.976+0000 7f138b7fe640 1 -- 172.21.15.143:0/619547698 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f1374005d70 msgr2=0x7f138c153e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:22:28.978 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:28.976+0000 7f138b7fe640 1 --2- 172.21.15.143:0/619547698 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f1374005d70 0x7f138c153e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:22:28.978 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:28.976+0000 7f138b7fe640 1 -- 172.21.15.143:0/619547698 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f138c0cfc20 con 0x7f138c13dce0 2023-11-26T07:22:28.978 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:28.976+0000 7f138b7fe640 1 --2- 172.21.15.143:0/619547698 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f138c13dce0 0x7f138c1fa2d0 secure :-1 s=READY pgs=912 cs=0 l=1 rev1=1 crypto rx=0x7f1380011510 tx=0x7f13800119d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:22:28.978 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:28.976+0000 7f1388ff9640 1 -- 172.21.15.143:0/619547698 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1380004220 con 0x7f138c13dce0 2023-11-26T07:22:28.978 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:28.976+0000 7f1388ff9640 1 -- 172.21.15.143:0/619547698 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1380004380 con 0x7f138c13dce0 2023-11-26T07:22:28.978 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:28.976+0000 7f1388ff9640 1 -- 172.21.15.143:0/619547698 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1380015890 con 0x7f138c13dce0 2023-11-26T07:22:28.979 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:28.976+0000 7f13926e4640 1 -- 172.21.15.143:0/619547698 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f138c13eb90 con 0x7f138c13dce0 2023-11-26T07:22:28.979 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:28.976+0000 7f13926e4640 1 -- 172.21.15.143:0/619547698 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f138c0b09c0 con 0x7f138c13dce0 2023-11-26T07:22:28.979 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:28.976+0000 7f13926e4640 1 -- 172.21.15.143:0/619547698 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f138c13ed80 con 0x7f138c13dce0 2023-11-26T07:22:28.979 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:28.976+0000 7f1388ff9640 1 -- 172.21.15.143:0/619547698 <== mon.0 v2:172.21.15.143:3300/0 4 ==== mgrmap(e 11) v1 ==== 86158+0+0 (secure 0 0 0) 0x7f13800159f0 con 0x7f138c13dce0 2023-11-26T07:22:28.979 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:28.980+0000 7f1388ff9640 1 --2- 172.21.15.143:0/619547698 >> [v2:172.21.15.143:6832/2150557544,v1:172.21.15.143:6833/2150557544] conn(0x7f1378066800 0x7f1378068cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:22:28.980 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:28.980+0000 7f1388ff9640 1 -- 172.21.15.143:0/619547698 <== mon.0 v2:172.21.15.143:3300/0 5 ==== osd_map(62..62 src has 1..62) v4 ==== 4943+0+0 (secure 0 0 0) 0x7f13800844f0 con 0x7f138c13dce0 2023-11-26T07:22:28.980 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:28.980+0000 7f138bfff640 1 --2- 172.21.15.143:0/619547698 >> [v2:172.21.15.143:6832/2150557544,v1:172.21.15.143:6833/2150557544] conn(0x7f1378066800 0x7f1378068cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:22:28.980 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:28.980+0000 7f138bfff640 1 --2- 172.21.15.143:0/619547698 >> [v2:172.21.15.143:6832/2150557544,v1:172.21.15.143:6833/2150557544] conn(0x7f1378066800 0x7f1378068cc0 secure :-1 s=READY pgs=428 cs=0 l=1 rev1=1 crypto rx=0x7f137c0ad000 tx=0x7f137c0a7000 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:22:28.984 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:28.984+0000 7f1388ff9640 1 -- 172.21.15.143:0/619547698 <== mon.0 v2:172.21.15.143:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172649 (secure 0 0 0) 0x7f13800518f0 con 0x7f138c13dce0 2023-11-26T07:22:29.138 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:29.136+0000 7f13926e4640 1 -- 172.21.15.143:0/619547698 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f138c067680 con 0x7f138c13dce0 2023-11-26T07:22:29.138 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:29.136+0000 7f1388ff9640 1 -- 172.21.15.143:0/619547698 <== mon.0 v2:172.21.15.143:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v62) v1 ==== 74+0+13024 (secure 0 0 0) 0x7f1380022090 con 0x7f138c13dce0 2023-11-26T07:22:29.139 INFO:teuthology.orchestra.run.smithi143.stdout: 2023-11-26T07:22:29.139 INFO:teuthology.orchestra.run.smithi143.stdout:{"epoch":62,"fsid":"f5033060-2469-4a07-95fe-2b76f5404bc1","created":"2023-11-26T07:17:28.613663+0000","modified":"2023-11-26T07:22:27.799683+0000","last_up_change":"2023-11-26T07:17:41.157813+0000","last_in_change":"2023-11-26T07:17:31.181658+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":9,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-11-26T07:17:41.758597+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"ebbee2c6-023e-44b9-9f1d-6f7bfd309730","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":57,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6817","nonce":639807369},{"type":"v1","addr":"172.21.15.143:6819","nonce":639807369}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6821","nonce":639807369},{"type":"v1","addr":"172.21.15.143:6823","nonce":639807369}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6830","nonce":639807369},{"type":"v1","addr":"172.21.15.143:6831","nonce":639807369}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6825","nonce":639807369},{"type":"v1","addr":"172.21.15.143:6828","nonce":639807369}]},"public_addr":"172.21.15.143:6819/639807369","cluster_addr":"172.21.15.143:6823/639807369","heartbeat_back_addr":"172.21.15.143:6831/639807369","heartbeat_front_addr":"172.21.15.143:6828/639807369","state":["exists","up"]},{"osd":1,"uuid":"fec4b8be-7122-49e9-9a19-5850a0700001","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":57,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6816","nonce":1144942530},{"type":"v1","addr":"172.21.15.143:6818","nonce":1144942530}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6820","nonce":1144942530},{"type":"v1","addr":"172.21.15.143:6822","nonce":1144942530}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6827","nonce":1144942530},{"type":"v1","addr":"172.21.15.143:6829","nonce":1144942530}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6824","nonce":1144942530},{"type":"v1","addr":"172.21.15.143:6826","nonce":1144942530}]},"public_addr":"172.21.15.143:6818/1144942530","cluster_addr":"172.21.15.143:6822/1144942530","heartbeat_back_addr":"172.21.15.143:6829/1144942530","heartbeat_front_addr":"172.21.15.143:6826/1144942530","state":["exists","up"]},{"osd":2,"uuid":"2888af5b-f933-46ca-8f95-6dd494ff43f6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":57,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6808","nonce":3909095186},{"type":"v1","addr":"172.21.15.143:6809","nonce":3909095186}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6810","nonce":3909095186},{"type":"v1","addr":"172.21.15.143:6811","nonce":3909095186}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6814","nonce":3909095186},{"type":"v1","addr":"172.21.15.143:6815","nonce":3909095186}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6812","nonce":3909095186},{"type":"v1","addr":"172.21.15.143:6813","nonce":3909095186}]},"public_addr":"172.21.15.143:6809/3909095186","cluster_addr":"172.21.15.143:6811/3909095186","heartbeat_back_addr":"172.21.15.143:6815/3909095186","heartbeat_front_addr":"172.21.15.143:6813/3909095186","state":["exists","up"]},{"osd":3,"uuid":"d1f52435-7bc1-4081-9550-9cd1ba55f549","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":57,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6800","nonce":3854210991},{"type":"v1","addr":"172.21.15.143:6801","nonce":3854210991}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6802","nonce":3854210991},{"type":"v1","addr":"172.21.15.143:6803","nonce":3854210991}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6806","nonce":3854210991},{"type":"v1","addr":"172.21.15.143:6807","nonce":3854210991}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6804","nonce":3854210991},{"type":"v1","addr":"172.21.15.143:6805","nonce":3854210991}]},"public_addr":"172.21.15.143:6801/3854210991","cluster_addr":"172.21.15.143:6803/3854210991","heartbeat_back_addr":"172.21.15.143:6807/3854210991","heartbeat_front_addr":"172.21.15.143:6805/3854210991","state":["exists","up"]},{"osd":4,"uuid":"e2ae8b6e-6e25-49b2-bcbf-aacce68f06a3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":57,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6800","nonce":3924606902},{"type":"v1","addr":"172.21.15.188:6801","nonce":3924606902}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6802","nonce":3924606902},{"type":"v1","addr":"172.21.15.188:6803","nonce":3924606902}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6806","nonce":3924606902},{"type":"v1","addr":"172.21.15.188:6807","nonce":3924606902}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6804","nonce":3924606902},{"type":"v1","addr":"172.21.15.188:6805","nonce":3924606902}]},"public_addr":"172.21.15.188:6801/3924606902","cluster_addr":"172.21.15.188:6803/3924606902","heartbeat_back_addr":"172.21.15.188:6807/3924606902","heartbeat_front_addr":"172.21.15.188:6805/3924606902","state":["exists","up"]},{"osd":5,"uuid":"3cec8a32-7698-4c36-8e89-e4ba11934f33","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":57,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6813","nonce":186841544},{"type":"v1","addr":"172.21.15.188:6815","nonce":186841544}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6817","nonce":186841544},{"type":"v1","addr":"172.21.15.188:6818","nonce":186841544}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6822","nonce":186841544},{"type":"v1","addr":"172.21.15.188:6823","nonce":186841544}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6820","nonce":186841544},{"type":"v1","addr":"172.21.15.188:6821","nonce":186841544}]},"public_addr":"172.21.15.188:6815/186841544","cluster_addr":"172.21.15.188:6818/186841544","heartbeat_back_addr":"172.21.15.188:6823/186841544","heartbeat_front_addr":"172.21.15.188:6821/186841544","state":["exists","up"]},{"osd":6,"uuid":"58a65d5d-e090-473a-81b4-ad38e85029f9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":57,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6808","nonce":2153499405},{"type":"v1","addr":"172.21.15.188:6809","nonce":2153499405}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6810","nonce":2153499405},{"type":"v1","addr":"172.21.15.188:6811","nonce":2153499405}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6816","nonce":2153499405},{"type":"v1","addr":"172.21.15.188:6819","nonce":2153499405}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6812","nonce":2153499405},{"type":"v1","addr":"172.21.15.188:6814","nonce":2153499405}]},"public_addr":"172.21.15.188:6809/2153499405","cluster_addr":"172.21.15.188:6811/2153499405","heartbeat_back_addr":"172.21.15.188:6819/2153499405","heartbeat_front_addr":"172.21.15.188:6814/2153499405","state":["exists","up"]},{"osd":7,"uuid":"ea5b7f70-3554-4cf8-ac30-f7c59b80678d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":57,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6824","nonce":2722096729},{"type":"v1","addr":"172.21.15.188:6825","nonce":2722096729}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6826","nonce":2722096729},{"type":"v1","addr":"172.21.15.188:6827","nonce":2722096729}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6830","nonce":2722096729},{"type":"v1","addr":"172.21.15.188:6831","nonce":2722096729}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6828","nonce":2722096729},{"type":"v1","addr":"172.21.15.188:6829","nonce":2722096729}]},"public_addr":"172.21.15.188:6825/2722096729","cluster_addr":"172.21.15.188:6827/2722096729","heartbeat_back_addr":"172.21.15.188:6831/2722096729","heartbeat_front_addr":"172.21.15.188:6829/2722096729","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-26T07:17:36.995032+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-26T07:17:36.932266+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-26T07:17:36.829304+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-26T07:17:36.804297+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-26T07:17:36.667892+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-26T07:17:36.964868+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-26T07:17:36.827332+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-26T07:17:36.893179+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.188:6835/1958716738":"2023-11-27T07:22:24.791113+0000","172.21.15.188:6833/1958716738":"2023-11-27T07:22:24.791113+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-11-26T07:22:29.141 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:29.140+0000 7f13926e4640 1 -- 172.21.15.143:0/619547698 >> [v2:172.21.15.143:6832/2150557544,v1:172.21.15.143:6833/2150557544] conn(0x7f1378066800 msgr2=0x7f1378068cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:22:29.141 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:29.140+0000 7f13926e4640 1 --2- 172.21.15.143:0/619547698 >> [v2:172.21.15.143:6832/2150557544,v1:172.21.15.143:6833/2150557544] conn(0x7f1378066800 0x7f1378068cc0 secure :-1 s=READY pgs=428 cs=0 l=1 rev1=1 crypto rx=0x7f137c0ad000 tx=0x7f137c0a7000 comp rx=0 tx=0).stop 2023-11-26T07:22:29.141 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:29.140+0000 7f13926e4640 1 -- 172.21.15.143:0/619547698 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f138c13dce0 msgr2=0x7f138c1fa2d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:22:29.141 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:29.140+0000 7f13926e4640 1 --2- 172.21.15.143:0/619547698 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f138c13dce0 0x7f138c1fa2d0 secure :-1 s=READY pgs=912 cs=0 l=1 rev1=1 crypto rx=0x7f1380011510 tx=0x7f13800119d0 comp rx=0 tx=0).stop 2023-11-26T07:22:29.141 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:29.140+0000 7f13926e4640 1 -- 172.21.15.143:0/619547698 shutdown_connections 2023-11-26T07:22:29.141 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:29.140+0000 7f13926e4640 1 --2- 172.21.15.143:0/619547698 >> [v2:172.21.15.143:6832/2150557544,v1:172.21.15.143:6833/2150557544] conn(0x7f1378066800 0x7f1378068cc0 unknown :-1 s=CLOSED pgs=428 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:22:29.141 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:29.140+0000 7f13926e4640 1 --2- 172.21.15.143:0/619547698 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f138c13bf30 0x7f138c0ce2d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:22:29.142 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:29.140+0000 7f13926e4640 1 --2- 172.21.15.143:0/619547698 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f138c13dce0 0x7f138c1fa2d0 unknown :-1 s=CLOSED pgs=912 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:22:29.142 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:29.140+0000 7f13926e4640 1 --2- 172.21.15.143:0/619547698 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f1374005d70 0x7f138c153e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:22:29.142 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:29.140+0000 7f13926e4640 1 -- 172.21.15.143:0/619547698 >> 172.21.15.143:0/619547698 conn(0x7f138c06a890 msgr2=0x7f138c06cd60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:22:29.142 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:29.140+0000 7f13926e4640 1 -- 172.21.15.143:0/619547698 shutdown_connections 2023-11-26T07:22:29.142 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:29.140+0000 7f13926e4640 1 -- 172.21.15.143:0/619547698 wait complete. 2023-11-26T07:22:29.160 INFO:tasks.ceph_test_case.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-11-26T07:17:41.758597+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2023-11-26T07:22:29.160 DEBUG:teuthology.orchestra.run.smithi143:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-11-26T07:22:29.301 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:29.300+0000 7f573ae9a640 1 Processor -- start 2023-11-26T07:22:29.301 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:29.300+0000 7f573ae9a640 1 -- start start 2023-11-26T07:22:29.302 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:29.300+0000 7f573ae9a640 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7f57340cf280 0x7f57340cf710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:22:29.302 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:29.300+0000 7f573ae9a640 1 -- --> v1:172.21.15.143:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f57340cc730 con 0x7f5734143ac0 2023-11-26T07:22:29.302 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:29.300+0000 7f573ae9a640 1 -- --> v1:172.21.15.188:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5734070ee0 con 0x7f57340cd1b0 2023-11-26T07:22:29.303 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:29.300+0000 7f573ae9a640 1 -- --> v2:172.21.15.188:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5734143ea0 con 0x7f57340cf280 2023-11-26T07:22:29.303 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:29.300+0000 7f5738c36640 1 --1- >> v1:172.21.15.143:6789/0 conn(0x7f5734143ac0 0x7f5734070370 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.143:6789/0 says I am v1:172.21.15.143:60246/0 (socket says 172.21.15.143:60246) 2023-11-26T07:22:29.303 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:29.300+0000 7f5738c36640 1 -- 172.21.15.143:0/2251693796 learned_addr learned my addr 172.21.15.143:0/2251693796 (peer_addr_for_me v1:172.21.15.143:0/0) 2023-11-26T07:22:29.303 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:29.300+0000 7f572bfff640 1 --2- 172.21.15.143:0/2251693796 >> v2:172.21.15.188:3300/0 conn(0x7f57340cf280 0x7f57340cf710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:22:29.303 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:29.300+0000 7f572b7fe640 1 -- 172.21.15.143:0/2251693796 <== mon.0 v1:172.21.15.143:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 548385994 0 0) 0x7f57340cc730 con 0x7f5734143ac0 2023-11-26T07:22:29.303 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:29.300+0000 7f572b7fe640 1 -- 172.21.15.143:0/2251693796 --> v1:172.21.15.143:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f571c003530 con 0x7f5734143ac0 2023-11-26T07:22:29.305 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:29.300+0000 7f572b7fe640 1 -- 172.21.15.143:0/2251693796 <== mon.1 v1:172.21.15.188:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3075341769 0 0) 0x7f5734070ee0 con 0x7f57340cd1b0 2023-11-26T07:22:29.305 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:29.300+0000 7f572b7fe640 1 -- 172.21.15.143:0/2251693796 --> v1:172.21.15.188:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f57340cc730 con 0x7f57340cd1b0 2023-11-26T07:22:29.305 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:29.300+0000 7f572b7fe640 1 -- 172.21.15.143:0/2251693796 <== mon.0 v1:172.21.15.143:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1450741540 0 0) 0x7f571c003530 con 0x7f5734143ac0 2023-11-26T07:22:29.305 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:29.300+0000 7f572b7fe640 1 -- 172.21.15.143:0/2251693796 >> v2:172.21.15.188:3300/0 conn(0x7f57340cf280 msgr2=0x7f57340cf710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-26T07:22:29.305 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:29.300+0000 7f572b7fe640 1 --2- 172.21.15.143:0/2251693796 >> v2:172.21.15.188:3300/0 conn(0x7f57340cf280 0x7f57340cf710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:22:29.305 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:29.300+0000 7f572b7fe640 1 -- 172.21.15.143:0/2251693796 >> v1:172.21.15.188:6789/0 conn(0x7f57340cd1b0 legacy=0x7f57340cd5c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:22:29.305 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:29.300+0000 7f572b7fe640 1 -- 172.21.15.143:0/2251693796 --> v1:172.21.15.143:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f57340711c0 con 0x7f5734143ac0 2023-11-26T07:22:29.305 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:29.300+0000 7f572b7fe640 1 -- 172.21.15.143:0/2251693796 <== mon.0 v1:172.21.15.143:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2137882523 0 0) 0x7f5730001680 con 0x7f5734143ac0 2023-11-26T07:22:29.305 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:29.300+0000 7f572b7fe640 1 -- 172.21.15.143:0/2251693796 >> v1:172.21.15.143:6789/0 conn(0x7f5734143ac0 legacy=0x7f5734070370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:22:29.306 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:29.300+0000 7f572b7fe640 1 --2- 172.21.15.143:0/2251693796 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f571c003b10 0x7f571c003f10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:22:29.306 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:29.300+0000 7f572b7fe640 1 --2- 172.21.15.143:0/2251693796 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f571c0047d0 0x7f571c004c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:22:29.306 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:29.300+0000 7f572b7fe640 1 --2- 172.21.15.143:0/2251693796 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f571c005190 0x7f571c00b990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:22:29.306 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:29.300+0000 7f572b7fe640 1 -- 172.21.15.143:0/2251693796 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f571c00bfe0 con 0x7f571c0047d0 2023-11-26T07:22:29.306 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:29.300+0000 7f572b7fe640 1 -- 172.21.15.143:0/2251693796 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f571c00c150 con 0x7f571c005190 2023-11-26T07:22:29.306 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:29.300+0000 7f572b7fe640 1 -- 172.21.15.143:0/2251693796 --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f571c00c310 con 0x7f571c003b10 2023-11-26T07:22:29.306 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:29.300+0000 7f572bfff640 1 --2- 172.21.15.143:0/2251693796 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f571c0047d0 0x7f571c004c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:22:29.306 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:29.300+0000 7f5739437640 1 --2- 172.21.15.143:0/2251693796 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f571c005190 0x7f571c00b990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:22:29.306 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:29.300+0000 7f5738c36640 1 --2- 172.21.15.143:0/2251693796 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f571c003b10 0x7f571c003f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:22:29.306 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:29.304+0000 7f572bfff640 1 -- 172.21.15.143:0/2251693796 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f571c003b10 msgr2=0x7f571c003f10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-26T07:22:29.306 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:29.304+0000 7f572bfff640 1 --2- 172.21.15.143:0/2251693796 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f571c003b10 0x7f571c003f10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:22:29.306 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:29.304+0000 7f572bfff640 1 -- 172.21.15.143:0/2251693796 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f571c005190 msgr2=0x7f571c00b990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-26T07:22:29.307 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:29.304+0000 7f572bfff640 1 --2- 172.21.15.143:0/2251693796 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f571c005190 0x7f571c00b990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:22:29.307 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:29.304+0000 7f572bfff640 1 -- 172.21.15.143:0/2251693796 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f571c003530 con 0x7f571c0047d0 2023-11-26T07:22:29.307 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:29.304+0000 7f5739437640 1 -- 172.21.15.143:0/2251693796 reap_dead start 2023-11-26T07:22:29.307 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:29.304+0000 7f5738c36640 1 --2- 172.21.15.143:0/2251693796 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f571c003b10 0x7f571c003f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-26T07:22:29.308 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:29.304+0000 7f572bfff640 1 --2- 172.21.15.143:0/2251693796 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f571c0047d0 0x7f571c004c50 secure :-1 s=READY pgs=914 cs=0 l=1 rev1=1 crypto rx=0x7f5724009670 tx=0x7f572401ec80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a144d57ed72824ad server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:22:29.308 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:29.304+0000 7f572b7fe640 1 -- 172.21.15.143:0/2251693796 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f57240a7770 con 0x7f571c0047d0 2023-11-26T07:22:29.308 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:29.304+0000 7f572b7fe640 1 -- 172.21.15.143:0/2251693796 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f57240a78d0 con 0x7f571c0047d0 2023-11-26T07:22:29.308 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:29.304+0000 7f572b7fe640 1 -- 172.21.15.143:0/2251693796 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f57240a7c80 con 0x7f571c0047d0 2023-11-26T07:22:29.308 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:29.304+0000 7f573ae9a640 1 -- 172.21.15.143:0/2251693796 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f571c0047d0 msgr2=0x7f571c004c50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:22:29.308 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:29.304+0000 7f573ae9a640 1 --2- 172.21.15.143:0/2251693796 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f571c0047d0 0x7f571c004c50 secure :-1 s=READY pgs=914 cs=0 l=1 rev1=1 crypto rx=0x7f5724009670 tx=0x7f572401ec80 comp rx=0 tx=0).stop 2023-11-26T07:22:29.308 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:29.304+0000 7f573ae9a640 1 -- 172.21.15.143:0/2251693796 shutdown_connections 2023-11-26T07:22:29.308 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:29.304+0000 7f573ae9a640 1 --2- 172.21.15.143:0/2251693796 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f571c0047d0 0x7f571c004c50 unknown :-1 s=CLOSED pgs=914 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:22:29.308 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:29.304+0000 7f573ae9a640 1 -- 172.21.15.143:0/2251693796 >> 172.21.15.143:0/2251693796 conn(0x7f573406a890 msgr2=0x7f573406aca0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:22:29.309 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:29.308+0000 7f573ae9a640 1 -- 172.21.15.143:0/2251693796 shutdown_connections 2023-11-26T07:22:29.309 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:29.308+0000 7f573ae9a640 1 -- 172.21.15.143:0/2251693796 wait complete. 2023-11-26T07:22:29.309 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:29.308+0000 7f573ae9a640 1 Processor -- start 2023-11-26T07:22:29.309 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:29.308+0000 7f573ae9a640 1 -- start start 2023-11-26T07:22:29.311 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:29.308+0000 7f573ae9a640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f571c0047d0 0x7f57341ef070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:22:29.311 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:29.308+0000 7f573ae9a640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f573413e170 0x7f573413f980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:22:29.311 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:29.308+0000 7f573ae9a640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f57340b0f70 0x7f57341593d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:22:29.312 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:29.308+0000 7f573ae9a640 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f573413e930 con 0x7f571c0047d0 2023-11-26T07:22:29.312 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:29.308+0000 7f573ae9a640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f573413ec10 con 0x7f573413e170 2023-11-26T07:22:29.312 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:29.308+0000 7f573ae9a640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f573413eef0 con 0x7f57340b0f70 2023-11-26T07:22:29.312 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:29.308+0000 7f5738c36640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f571c0047d0 0x7f57341ef070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:22:29.312 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:29.308+0000 7f5738c36640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f571c0047d0 0x7f57341ef070 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:47530/0 (socket says 172.21.15.143:47530) 2023-11-26T07:22:29.312 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:29.308+0000 7f5738c36640 1 -- 172.21.15.143:0/4127249186 learned_addr learned my addr 172.21.15.143:0/4127249186 (peer_addr_for_me v2:172.21.15.143:0/0) 2023-11-26T07:22:29.312 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:29.308+0000 7f572bfff640 1 --2- 172.21.15.143:0/4127249186 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f573413e170 0x7f573413f980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:22:29.312 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:29.308+0000 7f5739437640 1 --2- 172.21.15.143:0/4127249186 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f57340b0f70 0x7f57341593d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:22:29.313 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:29.308+0000 7f5738c36640 1 -- 172.21.15.143:0/4127249186 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f57340b0f70 msgr2=0x7f57341593d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:22:29.313 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:29.308+0000 7f5738c36640 1 --2- 172.21.15.143:0/4127249186 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f57340b0f70 0x7f57341593d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:22:29.313 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:29.308+0000 7f5738c36640 1 -- 172.21.15.143:0/4127249186 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f573413e170 msgr2=0x7f573413f980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:22:29.313 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:29.308+0000 7f5738c36640 1 --2- 172.21.15.143:0/4127249186 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f573413e170 0x7f573413f980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:22:29.313 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:29.308+0000 7f5738c36640 1 -- 172.21.15.143:0/4127249186 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f573413b360 con 0x7f571c0047d0 2023-11-26T07:22:29.313 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:29.308+0000 7f5739437640 1 --2- 172.21.15.143:0/4127249186 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f57340b0f70 0x7f57341593d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-26T07:22:29.313 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:29.308+0000 7f5738c36640 1 --2- 172.21.15.143:0/4127249186 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f571c0047d0 0x7f57341ef070 secure :-1 s=READY pgs=915 cs=0 l=1 rev1=1 crypto rx=0x7f573000e780 tx=0x7f573000ec40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:22:29.314 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:29.312+0000 7f57297fa640 1 -- 172.21.15.143:0/4127249186 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f573000c6c0 con 0x7f571c0047d0 2023-11-26T07:22:29.314 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:29.312+0000 7f57297fa640 1 -- 172.21.15.143:0/4127249186 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f573000c820 con 0x7f571c0047d0 2023-11-26T07:22:29.315 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:29.312+0000 7f57297fa640 1 -- 172.21.15.143:0/4127249186 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f573000cb90 con 0x7f571c0047d0 2023-11-26T07:22:29.315 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:29.312+0000 7f573ae9a640 1 -- 172.21.15.143:0/4127249186 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f573413e5f0 con 0x7f571c0047d0 2023-11-26T07:22:29.315 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:29.312+0000 7f573ae9a640 1 -- 172.21.15.143:0/4127249186 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f57340b13f0 con 0x7f571c0047d0 2023-11-26T07:22:29.315 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:29.312+0000 7f573ae9a640 1 -- 172.21.15.143:0/4127249186 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f573413b5a0 con 0x7f571c0047d0 2023-11-26T07:22:29.315 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:29.312+0000 7f57297fa640 1 -- 172.21.15.143:0/4127249186 <== mon.0 v2:172.21.15.143:3300/0 4 ==== mgrmap(e 11) v1 ==== 86158+0+0 (secure 0 0 0) 0x7f573000ccf0 con 0x7f571c0047d0 2023-11-26T07:22:29.315 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:29.312+0000 7f57297fa640 1 --2- 172.21.15.143:0/4127249186 >> [v2:172.21.15.143:6832/2150557544,v1:172.21.15.143:6833/2150557544] conn(0x7f57200667c0 0x7f5720068c80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:22:29.315 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:29.312+0000 7f57297fa640 1 -- 172.21.15.143:0/4127249186 <== mon.0 v2:172.21.15.143:3300/0 5 ==== osd_map(62..62 src has 1..62) v4 ==== 4943+0+0 (secure 0 0 0) 0x7f5730083f50 con 0x7f571c0047d0 2023-11-26T07:22:29.315 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:29.312+0000 7f572bfff640 1 --2- 172.21.15.143:0/4127249186 >> [v2:172.21.15.143:6832/2150557544,v1:172.21.15.143:6833/2150557544] conn(0x7f57200667c0 0x7f5720068c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:22:29.315 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:29.312+0000 7f572bfff640 1 --2- 172.21.15.143:0/4127249186 >> [v2:172.21.15.143:6832/2150557544,v1:172.21.15.143:6833/2150557544] conn(0x7f57200667c0 0x7f5720068c80 secure :-1 s=READY pgs=429 cs=0 l=1 rev1=1 crypto rx=0x7f57240b1000 tx=0x7f57240a9040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:22:29.319 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:29.316+0000 7f57297fa640 1 -- 172.21.15.143:0/4127249186 <== mon.0 v2:172.21.15.143:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172649 (secure 0 0 0) 0x7f573413b5a0 con 0x7f571c0047d0 2023-11-26T07:22:29.469 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:29.468+0000 7f573ae9a640 1 -- 172.21.15.143:0/4127249186 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f573413b5a0 con 0x7f571c0047d0 2023-11-26T07:22:29.469 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:29.468+0000 7f57297fa640 1 -- 172.21.15.143:0/4127249186 <== mon.0 v2:172.21.15.143:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v62) v1 ==== 93+0+10 (secure 0 0 0) 0x7f573413b5a0 con 0x7f571c0047d0 2023-11-26T07:22:29.470 INFO:teuthology.orchestra.run.smithi143.stdout:pg_num: 1 2023-11-26T07:22:29.471 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:29.468+0000 7f573ae9a640 1 -- 172.21.15.143:0/4127249186 >> [v2:172.21.15.143:6832/2150557544,v1:172.21.15.143:6833/2150557544] conn(0x7f57200667c0 msgr2=0x7f5720068c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:22:29.471 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:29.468+0000 7f573ae9a640 1 --2- 172.21.15.143:0/4127249186 >> [v2:172.21.15.143:6832/2150557544,v1:172.21.15.143:6833/2150557544] conn(0x7f57200667c0 0x7f5720068c80 secure :-1 s=READY pgs=429 cs=0 l=1 rev1=1 crypto rx=0x7f57240b1000 tx=0x7f57240a9040 comp rx=0 tx=0).stop 2023-11-26T07:22:29.472 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:29.468+0000 7f573ae9a640 1 -- 172.21.15.143:0/4127249186 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f571c0047d0 msgr2=0x7f57341ef070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:22:29.472 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:29.468+0000 7f573ae9a640 1 --2- 172.21.15.143:0/4127249186 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f571c0047d0 0x7f57341ef070 secure :-1 s=READY pgs=915 cs=0 l=1 rev1=1 crypto rx=0x7f573000e780 tx=0x7f573000ec40 comp rx=0 tx=0).stop 2023-11-26T07:22:29.472 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:29.468+0000 7f573ae9a640 1 -- 172.21.15.143:0/4127249186 shutdown_connections 2023-11-26T07:22:29.472 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:29.468+0000 7f573ae9a640 1 --2- 172.21.15.143:0/4127249186 >> [v2:172.21.15.143:6832/2150557544,v1:172.21.15.143:6833/2150557544] conn(0x7f57200667c0 0x7f5720068c80 unknown :-1 s=CLOSED pgs=429 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:22:29.472 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:29.468+0000 7f573ae9a640 1 --2- 172.21.15.143:0/4127249186 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f57340b0f70 0x7f57341593d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:22:29.472 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:29.468+0000 7f573ae9a640 1 --2- 172.21.15.143:0/4127249186 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f573413e170 0x7f573413f980 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:22:29.472 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:29.468+0000 7f573ae9a640 1 --2- 172.21.15.143:0/4127249186 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f571c0047d0 0x7f57341ef070 unknown :-1 s=CLOSED pgs=915 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:22:29.472 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:29.468+0000 7f573ae9a640 1 -- 172.21.15.143:0/4127249186 >> 172.21.15.143:0/4127249186 conn(0x7f573406a890 msgr2=0x7f57340b2540 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:22:29.472 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:29.468+0000 7f573ae9a640 1 -- 172.21.15.143:0/4127249186 shutdown_connections 2023-11-26T07:22:29.472 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:29.468+0000 7f573ae9a640 1 -- 172.21.15.143:0/4127249186 wait complete. 2023-11-26T07:22:29.491 DEBUG:teuthology.orchestra.run.smithi143:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Starting test tasks.cephfs.test_volumes.TestMisc.test_mgr_eviction' 2023-11-26T07:22:29.627 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:29.624+0000 7f55d3a32640 1 Processor -- start 2023-11-26T07:22:29.627 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:29.624+0000 7f55d3a32640 1 -- start start 2023-11-26T07:22:29.629 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:29.628+0000 7f55d3a32640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f55cc0ce680 0x7f55cc071850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:22:29.629 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:29.628+0000 7f55d3a32640 1 -- --> v1:172.21.15.143:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f55cc06c750 con 0x7f55cc1596d0 2023-11-26T07:22:29.629 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:29.628+0000 7f55d3a32640 1 -- --> v1:172.21.15.188:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f55cc0b3550 con 0x7f55cc0b2f80 2023-11-26T07:22:29.629 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:29.628+0000 7f55d3a32640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f55cc0cbe00 con 0x7f55cc0ce680 2023-11-26T07:22:29.629 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:29.628+0000 7f55d17ce640 1 --1- >> v1:172.21.15.143:6789/0 conn(0x7f55cc1596d0 0x7f55cc0b2870 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.143:6789/0 says I am v1:172.21.15.143:60258/0 (socket says 172.21.15.143:60258) 2023-11-26T07:22:29.629 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:29.628+0000 7f55d17ce640 1 -- 172.21.15.143:0/1844371942 learned_addr learned my addr 172.21.15.143:0/1844371942 (peer_addr_for_me v1:172.21.15.143:0/0) 2023-11-26T07:22:29.631 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:29.628+0000 7f55d1fcf640 1 --2- 172.21.15.143:0/1844371942 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f55cc0ce680 0x7f55cc071850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:22:29.631 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:29.628+0000 7f55c3fff640 1 -- 172.21.15.143:0/1844371942 <== mon.0 v1:172.21.15.143:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1791305532 0 0) 0x7f55cc06c750 con 0x7f55cc1596d0 2023-11-26T07:22:29.632 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:29.628+0000 7f55c3fff640 1 -- 172.21.15.143:0/1844371942 --> v1:172.21.15.143:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f55b8003530 con 0x7f55cc1596d0 2023-11-26T07:22:29.632 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:29.628+0000 7f55c3fff640 1 -- 172.21.15.143:0/1844371942 <== mon.1 v1:172.21.15.188:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2780746284 0 0) 0x7f55cc0b3550 con 0x7f55cc0b2f80 2023-11-26T07:22:29.632 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:29.628+0000 7f55c3fff640 1 -- 172.21.15.143:0/1844371942 --> v1:172.21.15.188:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f55cc06c750 con 0x7f55cc0b2f80 2023-11-26T07:22:29.632 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:29.628+0000 7f55c3fff640 1 -- 172.21.15.143:0/1844371942 <== mon.0 v1:172.21.15.143:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1517531484 0 0) 0x7f55b8003530 con 0x7f55cc1596d0 2023-11-26T07:22:29.632 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:29.628+0000 7f55c3fff640 1 -- 172.21.15.143:0/1844371942 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f55cc0ce680 msgr2=0x7f55cc071850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-26T07:22:29.632 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:29.628+0000 7f55c3fff640 1 --2- 172.21.15.143:0/1844371942 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f55cc0ce680 0x7f55cc071850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:22:29.632 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:29.628+0000 7f55c3fff640 1 -- 172.21.15.143:0/1844371942 >> v1:172.21.15.188:6789/0 conn(0x7f55cc0b2f80 legacy=0x7f55cc06cbe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:22:29.632 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:29.628+0000 7f55c3fff640 1 -- 172.21.15.143:0/1844371942 --> v1:172.21.15.143:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f55cc071e80 con 0x7f55cc1596d0 2023-11-26T07:22:29.632 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:29.628+0000 7f55c3fff640 1 -- 172.21.15.143:0/1844371942 <== mon.0 v1:172.21.15.143:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2137882523 0 0) 0x7f55c8001600 con 0x7f55cc1596d0 2023-11-26T07:22:29.632 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:29.628+0000 7f55c3fff640 1 -- 172.21.15.143:0/1844371942 >> v1:172.21.15.143:6789/0 conn(0x7f55cc1596d0 legacy=0x7f55cc0b2870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:22:29.632 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:29.628+0000 7f55c3fff640 1 --2- 172.21.15.143:0/1844371942 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f55cc0ce680 0x7f55b8003b40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:22:29.633 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:29.628+0000 7f55c3fff640 1 --2- 172.21.15.143:0/1844371942 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f55b80040f0 0x7f55b80047d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:22:29.633 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:29.628+0000 7f55c3fff640 1 --2- 172.21.15.143:0/1844371942 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f55b8004d10 0x7f55b8005190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:22:29.633 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:29.628+0000 7f55c3fff640 1 -- 172.21.15.143:0/1844371942 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f55cc0cbe00 con 0x7f55b8004d10 2023-11-26T07:22:29.633 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:29.628+0000 7f55c3fff640 1 -- 172.21.15.143:0/1844371942 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f55b800cbe0 con 0x7f55b80040f0 2023-11-26T07:22:29.633 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:29.628+0000 7f55c3fff640 1 -- 172.21.15.143:0/1844371942 --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f55b800cda0 con 0x7f55cc0ce680 2023-11-26T07:22:29.633 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:29.628+0000 7f55d1fcf640 1 --2- 172.21.15.143:0/1844371942 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f55b8004d10 0x7f55b8005190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:22:29.633 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:29.628+0000 7f55d17ce640 1 --2- 172.21.15.143:0/1844371942 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f55cc0ce680 0x7f55b8003b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:22:29.633 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:29.628+0000 7f55d0fcd640 1 --2- 172.21.15.143:0/1844371942 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f55b80040f0 0x7f55b80047d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:22:29.633 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:29.628+0000 7f55d1fcf640 1 -- 172.21.15.143:0/1844371942 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f55cc0ce680 msgr2=0x7f55b8003b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-26T07:22:29.633 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:29.628+0000 7f55d1fcf640 1 --2- 172.21.15.143:0/1844371942 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f55cc0ce680 0x7f55b8003b40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:22:29.633 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:29.628+0000 7f55d1fcf640 1 -- 172.21.15.143:0/1844371942 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f55b80040f0 msgr2=0x7f55b80047d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-26T07:22:29.633 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:29.628+0000 7f55d1fcf640 1 --2- 172.21.15.143:0/1844371942 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f55b80040f0 0x7f55b80047d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:22:29.634 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:29.628+0000 7f55d1fcf640 1 -- 172.21.15.143:0/1844371942 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f55b8003530 con 0x7f55b8004d10 2023-11-26T07:22:29.635 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:29.632+0000 7f55d1fcf640 1 --2- 172.21.15.143:0/1844371942 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f55b8004d10 0x7f55b8005190 secure :-1 s=READY pgs=917 cs=0 l=1 rev1=1 crypto rx=0x7f55c4009fc0 tx=0x7f55c401cc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ebcf28ddabceec8b server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:22:29.635 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:29.632+0000 7f55c3fff640 1 -- 172.21.15.143:0/1844371942 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f55c40a5c70 con 0x7f55b8004d10 2023-11-26T07:22:29.635 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:29.632+0000 7f55c3fff640 1 -- 172.21.15.143:0/1844371942 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f55c400f040 con 0x7f55b8004d10 2023-11-26T07:22:29.635 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:29.632+0000 7f55c3fff640 1 -- 172.21.15.143:0/1844371942 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f55c40ad440 con 0x7f55b8004d10 2023-11-26T07:22:29.635 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:29.632+0000 7f55d3a32640 1 -- 172.21.15.143:0/1844371942 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f55b8004d10 msgr2=0x7f55b8005190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:22:29.635 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:29.632+0000 7f55d3a32640 1 --2- 172.21.15.143:0/1844371942 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f55b8004d10 0x7f55b8005190 secure :-1 s=READY pgs=917 cs=0 l=1 rev1=1 crypto rx=0x7f55c4009fc0 tx=0x7f55c401cc60 comp rx=0 tx=0).stop 2023-11-26T07:22:29.635 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:29.632+0000 7f55d1fcf640 1 -- 172.21.15.143:0/1844371942 reap_dead start 2023-11-26T07:22:29.635 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:29.632+0000 7f55d3a32640 1 -- 172.21.15.143:0/1844371942 shutdown_connections 2023-11-26T07:22:29.635 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:29.632+0000 7f55d3a32640 1 -- 172.21.15.143:0/1844371942 >> 172.21.15.143:0/1844371942 conn(0x7f55cc06a890 msgr2=0x7f55cc06aca0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:22:29.635 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:29.632+0000 7f55d3a32640 1 -- 172.21.15.143:0/1844371942 shutdown_connections 2023-11-26T07:22:29.635 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:29.632+0000 7f55d3a32640 1 -- 172.21.15.143:0/1844371942 wait complete. 2023-11-26T07:22:29.636 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:29.636+0000 7f55d3a32640 1 Processor -- start 2023-11-26T07:22:29.636 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:29.636+0000 7f55d3a32640 1 -- start start 2023-11-26T07:22:29.638 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:29.636+0000 7f55d3a32640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f55b8004d10 0x7f55cc152dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:22:29.638 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:29.636+0000 7f55d3a32640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f55cc156900 0x7f55cc156d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:22:29.638 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:29.636+0000 7f55d3a32640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f55cc15c050 0x7f55cc15c4d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:22:29.638 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:29.636+0000 7f55d3a32640 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f55cc1572c0 con 0x7f55b8004d10 2023-11-26T07:22:29.638 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:29.636+0000 7f55d3a32640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f55cc15ca10 con 0x7f55cc156900 2023-11-26T07:22:29.638 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:29.636+0000 7f55d3a32640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f55cc15a280 con 0x7f55cc15c050 2023-11-26T07:22:29.638 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:29.636+0000 7f55d17ce640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f55b8004d10 0x7f55cc152dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:22:29.638 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:29.636+0000 7f55d17ce640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f55b8004d10 0x7f55cc152dd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:47546/0 (socket says 172.21.15.143:47546) 2023-11-26T07:22:29.639 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:29.636+0000 7f55d17ce640 1 -- 172.21.15.143:0/2514124808 learned_addr learned my addr 172.21.15.143:0/2514124808 (peer_addr_for_me v2:172.21.15.143:0/0) 2023-11-26T07:22:29.639 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:29.636+0000 7f55d0fcd640 1 --2- 172.21.15.143:0/2514124808 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f55cc156900 0x7f55cc156d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:22:29.639 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:29.636+0000 7f55d1fcf640 1 --2- 172.21.15.143:0/2514124808 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f55cc15c050 0x7f55cc15c4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:22:29.639 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:29.636+0000 7f55d17ce640 1 -- 172.21.15.143:0/2514124808 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f55cc15c050 msgr2=0x7f55cc15c4d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:22:29.639 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:29.636+0000 7f55d17ce640 1 --2- 172.21.15.143:0/2514124808 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f55cc15c050 0x7f55cc15c4d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:22:29.639 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:29.636+0000 7f55d17ce640 1 -- 172.21.15.143:0/2514124808 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f55cc156900 msgr2=0x7f55cc156d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:22:29.639 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:29.636+0000 7f55d17ce640 1 --2- 172.21.15.143:0/2514124808 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f55cc156900 0x7f55cc156d80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:22:29.639 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:29.636+0000 7f55d17ce640 1 -- 172.21.15.143:0/2514124808 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f55cc155cc0 con 0x7f55b8004d10 2023-11-26T07:22:29.639 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:29.636+0000 7f55d17ce640 1 --2- 172.21.15.143:0/2514124808 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f55b8004d10 0x7f55cc152dd0 secure :-1 s=READY pgs=918 cs=0 l=1 rev1=1 crypto rx=0x7f55c8012800 tx=0x7f55c8012cc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:22:29.640 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:29.636+0000 7f55c1ffb640 1 -- 172.21.15.143:0/2514124808 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f55c80109c0 con 0x7f55b8004d10 2023-11-26T07:22:29.640 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:29.636+0000 7f55c1ffb640 1 -- 172.21.15.143:0/2514124808 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f55c8010b60 con 0x7f55b8004d10 2023-11-26T07:22:29.640 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:29.636+0000 7f55c1ffb640 1 -- 172.21.15.143:0/2514124808 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f55c8012eb0 con 0x7f55b8004d10 2023-11-26T07:22:29.640 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:29.636+0000 7f55d3a32640 1 -- 172.21.15.143:0/2514124808 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f55cc1574a0 con 0x7f55b8004d10 2023-11-26T07:22:29.640 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:29.636+0000 7f55d3a32640 1 -- 172.21.15.143:0/2514124808 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f55cc159c30 con 0x7f55b8004d10 2023-11-26T07:22:29.640 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:29.640+0000 7f55d3a32640 1 -- 172.21.15.143:0/2514124808 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f55cc067680 con 0x7f55b8004d10 2023-11-26T07:22:29.641 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:29.640+0000 7f55c1ffb640 1 -- 172.21.15.143:0/2514124808 <== mon.0 v2:172.21.15.143:3300/0 4 ==== mgrmap(e 11) v1 ==== 86158+0+0 (secure 0 0 0) 0x7f55c8016480 con 0x7f55b8004d10 2023-11-26T07:22:29.642 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:29.640+0000 7f55c1ffb640 1 --2- 172.21.15.143:0/2514124808 >> [v2:172.21.15.143:6832/2150557544,v1:172.21.15.143:6833/2150557544] conn(0x7f55b0066800 0x7f55b0068cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:22:29.642 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:29.640+0000 7f55c1ffb640 1 -- 172.21.15.143:0/2514124808 <== mon.0 v2:172.21.15.143:3300/0 5 ==== osd_map(62..62 src has 1..62) v4 ==== 4943+0+0 (secure 0 0 0) 0x7f55c80875d0 con 0x7f55b8004d10 2023-11-26T07:22:29.642 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:29.640+0000 7f55d0fcd640 1 --2- 172.21.15.143:0/2514124808 >> [v2:172.21.15.143:6832/2150557544,v1:172.21.15.143:6833/2150557544] conn(0x7f55b0066800 0x7f55b0068cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:22:29.642 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:29.640+0000 7f55d0fcd640 1 --2- 172.21.15.143:0/2514124808 >> [v2:172.21.15.143:6832/2150557544,v1:172.21.15.143:6833/2150557544] conn(0x7f55b0066800 0x7f55b0068cc0 secure :-1 s=READY pgs=430 cs=0 l=1 rev1=1 crypto rx=0x7f55cc0b0590 tx=0x7f55bc009040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:22:29.645 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:29.644+0000 7f55c1ffb640 1 -- 172.21.15.143:0/2514124808 <== mon.0 v2:172.21.15.143:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172649 (secure 0 0 0) 0x7f55c8054990 con 0x7f55b8004d10 2023-11-26T07:22:29.848 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:29.848+0000 7f55d3a32640 1 -- 172.21.15.143:0/2514124808 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_volumes.TestMisc.test_mgr_eviction"]} v 0) v1 -- 0x7f55cc15be30 con 0x7f55b8004d10 2023-11-26T07:22:30.815 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:30.812+0000 7f55c1ffb640 1 -- 172.21.15.143:0/2514124808 <== mon.0 v2:172.21.15.143:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_volumes.TestMisc.test_mgr_eviction"]}]=0 v244) v1 ==== 134+0+0 (secure 0 0 0) 0x7f55c8057fb0 con 0x7f55b8004d10 2023-11-26T07:22:30.818 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:30.816+0000 7f55d3a32640 1 -- 172.21.15.143:0/2514124808 >> [v2:172.21.15.143:6832/2150557544,v1:172.21.15.143:6833/2150557544] conn(0x7f55b0066800 msgr2=0x7f55b0068cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:22:30.818 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:30.816+0000 7f55d3a32640 1 --2- 172.21.15.143:0/2514124808 >> [v2:172.21.15.143:6832/2150557544,v1:172.21.15.143:6833/2150557544] conn(0x7f55b0066800 0x7f55b0068cc0 secure :-1 s=READY pgs=430 cs=0 l=1 rev1=1 crypto rx=0x7f55cc0b0590 tx=0x7f55bc009040 comp rx=0 tx=0).stop 2023-11-26T07:22:30.818 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:30.816+0000 7f55d3a32640 1 -- 172.21.15.143:0/2514124808 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f55b8004d10 msgr2=0x7f55cc152dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:22:30.818 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:30.816+0000 7f55d3a32640 1 --2- 172.21.15.143:0/2514124808 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f55b8004d10 0x7f55cc152dd0 secure :-1 s=READY pgs=918 cs=0 l=1 rev1=1 crypto rx=0x7f55c8012800 tx=0x7f55c8012cc0 comp rx=0 tx=0).stop 2023-11-26T07:22:30.818 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:30.816+0000 7f55d3a32640 1 -- 172.21.15.143:0/2514124808 shutdown_connections 2023-11-26T07:22:30.819 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:30.816+0000 7f55d3a32640 1 --2- 172.21.15.143:0/2514124808 >> [v2:172.21.15.143:6832/2150557544,v1:172.21.15.143:6833/2150557544] conn(0x7f55b0066800 0x7f55b0068cc0 unknown :-1 s=CLOSED pgs=430 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:22:30.819 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:30.816+0000 7f55d3a32640 1 --2- 172.21.15.143:0/2514124808 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f55cc15c050 0x7f55cc15c4d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:22:30.819 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:30.816+0000 7f55d3a32640 1 --2- 172.21.15.143:0/2514124808 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f55cc156900 0x7f55cc156d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:22:30.819 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:30.816+0000 7f55d3a32640 1 --2- 172.21.15.143:0/2514124808 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f55b8004d10 0x7f55cc152dd0 unknown :-1 s=CLOSED pgs=918 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:22:30.819 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:30.816+0000 7f55d3a32640 1 -- 172.21.15.143:0/2514124808 >> 172.21.15.143:0/2514124808 conn(0x7f55cc06a890 msgr2=0x7f55cc06d9e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:22:30.819 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:30.816+0000 7f55d3a32640 1 -- 172.21.15.143:0/2514124808 shutdown_connections 2023-11-26T07:22:30.819 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:30.816+0000 7f55d3a32640 1 -- 172.21.15.143:0/2514124808 wait complete. 2023-11-26T07:22:30.841 DEBUG:teuthology.orchestra.run.smithi143:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mon mon_allow_pool_delete True 2023-11-26T07:22:30.984 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:30.984+0000 7f22e8c5a640 1 Processor -- start 2023-11-26T07:22:30.984 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:30.984+0000 7f22e8c5a640 1 -- start start 2023-11-26T07:22:30.986 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:30.984+0000 7f22e8c5a640 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7f22e40ce7c0 0x7f22e415f6b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:22:30.986 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:30.984+0000 7f22e8c5a640 1 -- --> v1:172.21.15.143:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f22e406c520 con 0x7f22e40cf990 2023-11-26T07:22:30.986 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:30.984+0000 7f22e8c5a640 1 -- --> v1:172.21.15.188:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f22e406c800 con 0x7f22e413e290 2023-11-26T07:22:30.986 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:30.984+0000 7f22e8c5a640 1 -- --> v2:172.21.15.188:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f22e406cae0 con 0x7f22e40ce7c0 2023-11-26T07:22:30.986 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:30.984+0000 7f22d9d9b640 1 --1- >> v1:172.21.15.143:6789/0 conn(0x7f22e40cf990 0x7f22e415efa0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.143:6789/0 says I am v1:172.21.15.143:60270/0 (socket says 172.21.15.143:60270) 2023-11-26T07:22:30.986 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:30.984+0000 7f22d9d9b640 1 -- 172.21.15.143:0/876579348 learned_addr learned my addr 172.21.15.143:0/876579348 (peer_addr_for_me v1:172.21.15.143:0/0) 2023-11-26T07:22:30.986 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:30.984+0000 7f22e2d9d640 1 --2- 172.21.15.143:0/876579348 >> v2:172.21.15.188:3300/0 conn(0x7f22e40ce7c0 0x7f22e415f6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:22:30.988 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:30.984+0000 7f22e1d9b640 1 -- 172.21.15.143:0/876579348 <== mon.0 v1:172.21.15.143:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2711225571 0 0) 0x7f22e406c520 con 0x7f22e40cf990 2023-11-26T07:22:30.988 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:30.984+0000 7f22e1d9b640 1 -- 172.21.15.143:0/876579348 --> v1:172.21.15.143:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f22d0003530 con 0x7f22e40cf990 2023-11-26T07:22:30.988 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:30.984+0000 7f22e1d9b640 1 -- 172.21.15.143:0/876579348 <== mon.0 v1:172.21.15.143:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1378987625 0 0) 0x7f22d0003530 con 0x7f22e40cf990 2023-11-26T07:22:30.988 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:30.984+0000 7f22e1d9b640 1 -- 172.21.15.143:0/876579348 >> v2:172.21.15.188:3300/0 conn(0x7f22e40ce7c0 msgr2=0x7f22e415f6b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-26T07:22:30.989 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:30.984+0000 7f22e1d9b640 1 --2- 172.21.15.143:0/876579348 >> v2:172.21.15.188:3300/0 conn(0x7f22e40ce7c0 0x7f22e415f6b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:22:30.989 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:30.984+0000 7f22e1d9b640 1 -- 172.21.15.143:0/876579348 >> v1:172.21.15.188:6789/0 conn(0x7f22e413e290 legacy=0x7f22e40cf280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:22:30.989 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:30.984+0000 7f22e1d9b640 1 -- 172.21.15.143:0/876579348 --> v1:172.21.15.143:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f22e406cdc0 con 0x7f22e40cf990 2023-11-26T07:22:30.989 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:30.984+0000 7f22e1d9b640 1 -- 172.21.15.143:0/876579348 <== mon.0 v1:172.21.15.143:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2137882523 0 0) 0x7f22cc001680 con 0x7f22e40cf990 2023-11-26T07:22:30.989 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:30.984+0000 7f22e1d9b640 1 -- 172.21.15.143:0/876579348 >> v1:172.21.15.143:6789/0 conn(0x7f22e40cf990 legacy=0x7f22e415efa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:22:30.989 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:30.984+0000 7f22e1d9b640 1 --2- 172.21.15.143:0/876579348 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f22d0003980 0x7f22d0003da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:22:30.989 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:30.984+0000 7f22e1d9b640 1 --2- 172.21.15.143:0/876579348 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f22d00047d0 0x7f22d0004c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:22:30.989 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:30.984+0000 7f22e1d9b640 1 --2- 172.21.15.143:0/876579348 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f22d0005190 0x7f22d000b990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:22:30.989 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:30.984+0000 7f22e1d9b640 1 -- 172.21.15.143:0/876579348 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f22e406cae0 con 0x7f22d00047d0 2023-11-26T07:22:30.989 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:30.984+0000 7f22e1d9b640 1 -- 172.21.15.143:0/876579348 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f22d000c130 con 0x7f22d0005190 2023-11-26T07:22:30.989 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:30.984+0000 7f22e1d9b640 1 -- 172.21.15.143:0/876579348 --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f22d000c2f0 con 0x7f22d0003980 2023-11-26T07:22:30.989 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:30.984+0000 7f22d9d9b640 1 --2- 172.21.15.143:0/876579348 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f22d00047d0 0x7f22d0004c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:22:30.990 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:30.984+0000 7f22e259c640 1 --2- 172.21.15.143:0/876579348 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f22d0003980 0x7f22d0003da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:22:30.990 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:30.984+0000 7f22e2d9d640 1 --2- 172.21.15.143:0/876579348 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f22d0005190 0x7f22d000b990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:22:30.990 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:30.984+0000 7f22d9d9b640 1 -- 172.21.15.143:0/876579348 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f22d0003980 msgr2=0x7f22d0003da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-26T07:22:30.990 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:30.984+0000 7f22d9d9b640 1 --2- 172.21.15.143:0/876579348 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f22d0003980 0x7f22d0003da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:22:30.990 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:30.984+0000 7f22d9d9b640 1 -- 172.21.15.143:0/876579348 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f22d0005190 msgr2=0x7f22d000b990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-26T07:22:30.990 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:30.984+0000 7f22d9d9b640 1 --2- 172.21.15.143:0/876579348 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f22d0005190 0x7f22d000b990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:22:30.990 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:30.988+0000 7f22d9d9b640 1 -- 172.21.15.143:0/876579348 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f22d0003530 con 0x7f22d00047d0 2023-11-26T07:22:30.990 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:30.988+0000 7f22e2d9d640 1 -- 172.21.15.143:0/876579348 reap_dead start 2023-11-26T07:22:30.990 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:30.988+0000 7f22e259c640 1 --2- 172.21.15.143:0/876579348 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f22d0003980 0x7f22d0003da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-26T07:22:30.991 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:30.988+0000 7f22d9d9b640 1 --2- 172.21.15.143:0/876579348 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f22d00047d0 0x7f22d0004c50 secure :-1 s=READY pgs=920 cs=0 l=1 rev1=1 crypto rx=0x7f22cc00c1c0 tx=0x7f22cc01cbb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2fd79870275414bf server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:22:30.992 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:30.988+0000 7f22e1d9b640 1 -- 172.21.15.143:0/876579348 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f22cc005ae0 con 0x7f22d00047d0 2023-11-26T07:22:30.992 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:30.988+0000 7f22e1d9b640 1 -- 172.21.15.143:0/876579348 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f22cc005c40 con 0x7f22d00047d0 2023-11-26T07:22:30.992 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:30.988+0000 7f22e1d9b640 1 -- 172.21.15.143:0/876579348 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f22cc0a5a00 con 0x7f22d00047d0 2023-11-26T07:22:30.992 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:30.988+0000 7f22e8c5a640 1 -- 172.21.15.143:0/876579348 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f22d00047d0 msgr2=0x7f22d0004c50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:22:30.992 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:30.988+0000 7f22e8c5a640 1 --2- 172.21.15.143:0/876579348 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f22d00047d0 0x7f22d0004c50 secure :-1 s=READY pgs=920 cs=0 l=1 rev1=1 crypto rx=0x7f22cc00c1c0 tx=0x7f22cc01cbb0 comp rx=0 tx=0).stop 2023-11-26T07:22:30.992 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:30.988+0000 7f22e8c5a640 1 -- 172.21.15.143:0/876579348 shutdown_connections 2023-11-26T07:22:30.992 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:30.988+0000 7f22e8c5a640 1 --2- 172.21.15.143:0/876579348 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f22d00047d0 0x7f22d0004c50 unknown :-1 s=CLOSED pgs=920 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:22:30.992 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:30.988+0000 7f22e8c5a640 1 -- 172.21.15.143:0/876579348 >> 172.21.15.143:0/876579348 conn(0x7f22e406a890 msgr2=0x7f22e406aca0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:22:30.992 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:30.988+0000 7f22e8c5a640 1 -- 172.21.15.143:0/876579348 shutdown_connections 2023-11-26T07:22:30.992 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:30.988+0000 7f22e8c5a640 1 -- 172.21.15.143:0/876579348 wait complete. 2023-11-26T07:22:30.993 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:30.992+0000 7f22e8c5a640 1 Processor -- start 2023-11-26T07:22:30.993 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:30.992+0000 7f22e8c5a640 1 -- start start 2023-11-26T07:22:30.995 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:30.992+0000 7f22e8c5a640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f22d00047d0 0x7f22e41f6ae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:22:30.995 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:30.992+0000 7f22e8c5a640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f22e415d500 0x7f22e413b900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:22:30.995 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:30.992+0000 7f22e8c5a640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f22e413f500 0x7f22e413f980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:22:30.995 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:30.992+0000 7f22e8c5a640 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f22e413fec0 con 0x7f22d00047d0 2023-11-26T07:22:30.995 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:30.992+0000 7f22e8c5a640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f22e4141e60 con 0x7f22e415d500 2023-11-26T07:22:30.995 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:30.992+0000 7f22e8c5a640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f22e413be40 con 0x7f22e413f500 2023-11-26T07:22:30.996 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:30.992+0000 7f22e259c640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f22d00047d0 0x7f22e41f6ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:22:30.996 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:30.992+0000 7f22e259c640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f22d00047d0 0x7f22e41f6ae0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:47562/0 (socket says 172.21.15.143:47562) 2023-11-26T07:22:30.996 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:30.992+0000 7f22e259c640 1 -- 172.21.15.143:0/1320728088 learned_addr learned my addr 172.21.15.143:0/1320728088 (peer_addr_for_me v2:172.21.15.143:0/0) 2023-11-26T07:22:30.996 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:30.992+0000 7f22d9d9b640 1 --2- 172.21.15.143:0/1320728088 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f22e415d500 0x7f22e413b900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:22:30.996 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:30.992+0000 7f22e2d9d640 1 --2- 172.21.15.143:0/1320728088 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f22e413f500 0x7f22e413f980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:22:30.996 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:30.992+0000 7f22e259c640 1 -- 172.21.15.143:0/1320728088 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f22e413f500 msgr2=0x7f22e413f980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:22:30.996 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:30.992+0000 7f22e259c640 1 --2- 172.21.15.143:0/1320728088 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f22e413f500 0x7f22e413f980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:22:30.996 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:30.992+0000 7f22e259c640 1 -- 172.21.15.143:0/1320728088 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f22e415d500 msgr2=0x7f22e413b900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:22:30.996 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:30.992+0000 7f22e259c640 1 --2- 172.21.15.143:0/1320728088 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f22e415d500 0x7f22e413b900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:22:30.996 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:30.992+0000 7f22e259c640 1 -- 172.21.15.143:0/1320728088 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f22e415da70 con 0x7f22d00047d0 2023-11-26T07:22:30.996 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:30.992+0000 7f22e2d9d640 1 --2- 172.21.15.143:0/1320728088 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f22e413f500 0x7f22e413f980 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-26T07:22:30.996 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:30.992+0000 7f22e259c640 1 --2- 172.21.15.143:0/1320728088 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f22d00047d0 0x7f22e41f6ae0 secure :-1 s=READY pgs=921 cs=0 l=1 rev1=1 crypto rx=0x7f22d400e800 tx=0x7f22d400ecc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:22:30.997 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:30.996+0000 7f22db7fe640 1 -- 172.21.15.143:0/1320728088 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f22d4003b90 con 0x7f22d00047d0 2023-11-26T07:22:30.997 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:30.996+0000 7f22db7fe640 1 -- 172.21.15.143:0/1320728088 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f22d4003cf0 con 0x7f22d00047d0 2023-11-26T07:22:30.997 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:30.996+0000 7f22db7fe640 1 -- 172.21.15.143:0/1320728088 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f22d40052f0 con 0x7f22d00047d0 2023-11-26T07:22:30.997 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:30.996+0000 7f22e8c5a640 1 -- 172.21.15.143:0/1320728088 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f22e415dd50 con 0x7f22d00047d0 2023-11-26T07:22:30.997 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:30.996+0000 7f22e8c5a640 1 -- 172.21.15.143:0/1320728088 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f22e415e310 con 0x7f22d00047d0 2023-11-26T07:22:30.998 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:30.996+0000 7f22e8c5a640 1 -- 172.21.15.143:0/1320728088 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f22e4067680 con 0x7f22d00047d0 2023-11-26T07:22:30.998 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:30.996+0000 7f22db7fe640 1 -- 172.21.15.143:0/1320728088 <== mon.0 v2:172.21.15.143:3300/0 4 ==== mgrmap(e 11) v1 ==== 86158+0+0 (secure 0 0 0) 0x7f22d4005450 con 0x7f22d00047d0 2023-11-26T07:22:30.999 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:30.996+0000 7f22db7fe640 1 --2- 172.21.15.143:0/1320728088 >> [v2:172.21.15.143:6832/2150557544,v1:172.21.15.143:6833/2150557544] conn(0x7f22c40667d0 0x7f22c4068c90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:22:30.999 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:30.996+0000 7f22db7fe640 1 -- 172.21.15.143:0/1320728088 <== mon.0 v2:172.21.15.143:3300/0 5 ==== osd_map(62..62 src has 1..62) v4 ==== 4943+0+0 (secure 0 0 0) 0x7f22d4084140 con 0x7f22d00047d0 2023-11-26T07:22:30.999 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:30.996+0000 7f22d9d9b640 1 --2- 172.21.15.143:0/1320728088 >> [v2:172.21.15.143:6832/2150557544,v1:172.21.15.143:6833/2150557544] conn(0x7f22c40667d0 0x7f22c4068c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:22:30.999 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:31.000+0000 7f22d9d9b640 1 --2- 172.21.15.143:0/1320728088 >> [v2:172.21.15.143:6832/2150557544,v1:172.21.15.143:6833/2150557544] conn(0x7f22c40667d0 0x7f22c4068c90 secure :-1 s=READY pgs=431 cs=0 l=1 rev1=1 crypto rx=0x7f22cc0ab000 tx=0x7f22cc0a7070 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:22:31.002 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:31.000+0000 7f22db7fe640 1 -- 172.21.15.143:0/1320728088 <== mon.0 v2:172.21.15.143:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172649 (secure 0 0 0) 0x7f22d4051470 con 0x7f22d00047d0 2023-11-26T07:22:31.151 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:31.148+0000 7f22e8c5a640 1 -- 172.21.15.143:0/1320728088 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command([{prefix=config set, name=mon_allow_pool_delete}] v 0) v1 -- 0x7f22e406df50 con 0x7f22d00047d0 2023-11-26T07:22:31.157 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:31.156+0000 7f22db7fe640 1 -- 172.21.15.143:0/1320728088 <== mon.0 v2:172.21.15.143:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mon_allow_pool_delete}]=0 v16)=0 v16) v1 ==== 122+0+0 (secure 0 0 0) 0x7f22d4003e60 con 0x7f22d00047d0 2023-11-26T07:22:31.160 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:31.156+0000 7f22e8c5a640 1 -- 172.21.15.143:0/1320728088 >> [v2:172.21.15.143:6832/2150557544,v1:172.21.15.143:6833/2150557544] conn(0x7f22c40667d0 msgr2=0x7f22c4068c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:22:31.160 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:31.156+0000 7f22e8c5a640 1 --2- 172.21.15.143:0/1320728088 >> [v2:172.21.15.143:6832/2150557544,v1:172.21.15.143:6833/2150557544] conn(0x7f22c40667d0 0x7f22c4068c90 secure :-1 s=READY pgs=431 cs=0 l=1 rev1=1 crypto rx=0x7f22cc0ab000 tx=0x7f22cc0a7070 comp rx=0 tx=0).stop 2023-11-26T07:22:31.160 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:31.156+0000 7f22e8c5a640 1 -- 172.21.15.143:0/1320728088 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f22d00047d0 msgr2=0x7f22e41f6ae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:22:31.160 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:31.156+0000 7f22e8c5a640 1 --2- 172.21.15.143:0/1320728088 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f22d00047d0 0x7f22e41f6ae0 secure :-1 s=READY pgs=921 cs=0 l=1 rev1=1 crypto rx=0x7f22d400e800 tx=0x7f22d400ecc0 comp rx=0 tx=0).stop 2023-11-26T07:22:31.160 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:31.156+0000 7f22e8c5a640 1 -- 172.21.15.143:0/1320728088 shutdown_connections 2023-11-26T07:22:31.161 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:31.156+0000 7f22e8c5a640 1 --2- 172.21.15.143:0/1320728088 >> [v2:172.21.15.143:6832/2150557544,v1:172.21.15.143:6833/2150557544] conn(0x7f22c40667d0 0x7f22c4068c90 unknown :-1 s=CLOSED pgs=431 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:22:31.161 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:31.156+0000 7f22e8c5a640 1 --2- 172.21.15.143:0/1320728088 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f22e413f500 0x7f22e413f980 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:22:31.161 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:31.156+0000 7f22e8c5a640 1 --2- 172.21.15.143:0/1320728088 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f22e415d500 0x7f22e413b900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:22:31.161 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:31.156+0000 7f22e8c5a640 1 --2- 172.21.15.143:0/1320728088 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f22d00047d0 0x7f22e41f6ae0 unknown :-1 s=CLOSED pgs=921 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:22:31.161 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:31.156+0000 7f22e8c5a640 1 -- 172.21.15.143:0/1320728088 >> 172.21.15.143:0/1320728088 conn(0x7f22e406a890 msgr2=0x7f22e40b1460 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:22:31.161 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:31.156+0000 7f22e8c5a640 1 -- 172.21.15.143:0/1320728088 shutdown_connections 2023-11-26T07:22:31.161 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:31.156+0000 7f22e8c5a640 1 -- 172.21.15.143:0/1320728088 wait complete. 2023-11-26T07:22:31.178 INFO:teuthology.nuke.actions:Clearing teuthology firewall rules... 2023-11-26T07:22:31.179 DEBUG:teuthology.orchestra.run.smithi143:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2023-11-26T07:22:31.242 DEBUG:teuthology.orchestra.run.smithi188:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2023-11-26T07:22:31.299 INFO:teuthology.nuke.actions:Cleared teuthology firewall rules. 2023-11-26T07:22:31.300 DEBUG:teuthology.orchestra.run.smithi143:> set -ex 2023-11-26T07:22:31.300 DEBUG:teuthology.orchestra.run.smithi143:> dd if=/proc/self/mounts of=/dev/stdout 2023-11-26T07:22:31.309 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,relatime,size=16354672k,nr_inodes=4088668,mode=755 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3275384k,mode=755 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=3455 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysusers.service ramfs ro,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3275380k,nr_inodes=818845,mode=700,uid=1000,gid=1271 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3275384k,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2023-11-26T07:22:31.309 DEBUG:teuthology.orchestra.run.smithi188:> set -ex 2023-11-26T07:22:31.309 DEBUG:teuthology.orchestra.run.smithi188:> dd if=/proc/self/mounts of=/dev/stdout 2023-11-26T07:22:31.319 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,relatime,size=16354628k,nr_inodes=4088657,mode=755 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3275376k,mode=755 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=787 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysusers.service ramfs ro,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3275372k,nr_inodes=818843,mode=700,uid=1000,gid=1271 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3275376k,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2023-11-26T07:22:31.319 DEBUG:teuthology.orchestra.run.smithi143:> set -ex 2023-11-26T07:22:31.319 DEBUG:teuthology.orchestra.run.smithi143:> dd if=/proc/self/mounts of=/dev/stdout 2023-11-26T07:22:31.359 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,relatime,size=16354672k,nr_inodes=4088668,mode=755 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3275384k,mode=755 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=3455 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysusers.service ramfs ro,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3275380k,nr_inodes=818845,mode=700,uid=1000,gid=1271 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3275384k,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2023-11-26T07:22:31.359 DEBUG:teuthology.orchestra.run.smithi188:> set -ex 2023-11-26T07:22:31.359 DEBUG:teuthology.orchestra.run.smithi188:> dd if=/proc/self/mounts of=/dev/stdout 2023-11-26T07:22:31.372 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,relatime,size=16354628k,nr_inodes=4088657,mode=755 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3275376k,mode=755 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=787 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysusers.service ramfs ro,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3275372k,nr_inodes=818843,mode=700,uid=1000,gid=1271 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3275376k,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2023-11-26T07:22:31.372 INFO:tasks.cephfs.cephfs_test_case:[, , , ] 2023-11-26T07:22:31.373 DEBUG:teuthology.orchestra.run.smithi143:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-11-26T07:22:31.539 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:31.536+0000 7f8586ddf640 1 Processor -- start 2023-11-26T07:22:31.540 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:31.536+0000 7f8586ddf640 1 -- start start 2023-11-26T07:22:31.541 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:31.540+0000 7f8586ddf640 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7f858013e290 0x7f85800cf280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:22:31.541 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:31.540+0000 7f8586ddf640 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7f85800cf7c0 0x7f858015efa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:22:31.542 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:31.540+0000 7f8586ddf640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f858015f4e0 0x7f85801408e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:22:31.542 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:31.540+0000 7f8586ddf640 1 -- --> v2:172.21.15.143:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f85800cfc40 con 0x7f858013e290 2023-11-26T07:22:31.542 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:31.540+0000 7f8586ddf640 1 -- --> v2:172.21.15.188:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f858006c520 con 0x7f85800cf7c0 2023-11-26T07:22:31.542 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:31.540+0000 7f8586ddf640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f858006c800 con 0x7f858015f4e0 2023-11-26T07:22:31.542 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:31.540+0000 7f8584b7b640 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7f858013e290 0x7f85800cf280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:22:31.542 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:31.540+0000 7f8584b7b640 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7f858013e290 0x7f85800cf280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:47570/0 (socket says 172.21.15.143:47570) 2023-11-26T07:22:31.542 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:31.540+0000 7f8584b7b640 1 -- 172.21.15.143:0/2384975832 learned_addr learned my addr 172.21.15.143:0/2384975832 (peer_addr_for_me v2:172.21.15.143:0/0) 2023-11-26T07:22:31.542 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:31.540+0000 7f8584b7b640 1 -- 172.21.15.143:0/2384975832 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f858015f4e0 msgr2=0x7f85801408e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-26T07:22:31.542 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:31.540+0000 7f8584b7b640 1 --2- 172.21.15.143:0/2384975832 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f858015f4e0 0x7f85801408e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:22:31.542 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:31.540+0000 7f8584b7b640 1 -- 172.21.15.143:0/2384975832 >> v2:172.21.15.188:3300/0 conn(0x7f85800cf7c0 msgr2=0x7f858015efa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-26T07:22:31.542 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:31.540+0000 7f8584b7b640 1 --2- 172.21.15.143:0/2384975832 >> v2:172.21.15.188:3300/0 conn(0x7f85800cf7c0 0x7f858015efa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:22:31.542 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:31.540+0000 7f8584b7b640 1 -- 172.21.15.143:0/2384975832 --> v2:172.21.15.143:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f858006cae0 con 0x7f858013e290 2023-11-26T07:22:31.544 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:31.544+0000 7f8584b7b640 1 --2- 172.21.15.143:0/2384975832 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f858013e290 0x7f85800cf280 secure :-1 s=READY pgs=922 cs=0 l=1 rev1=1 crypto rx=0x7f85800cfe40 tx=0x7f857c01c210 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fd16ecda63d81c67 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:22:31.544 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:31.544+0000 7f85777fe640 1 -- 172.21.15.143:0/2384975832 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f857c0ab600 con 0x7f858013e290 2023-11-26T07:22:31.545 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:31.544+0000 7f85777fe640 1 -- 172.21.15.143:0/2384975832 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f857c0a7050 con 0x7f858013e290 2023-11-26T07:22:31.545 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:31.544+0000 7f85777fe640 1 -- 172.21.15.143:0/2384975832 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f857c0ab980 con 0x7f858013e290 2023-11-26T07:22:31.546 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:31.544+0000 7f8586ddf640 1 -- 172.21.15.143:0/2384975832 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f858013e290 msgr2=0x7f85800cf280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:22:31.546 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:31.544+0000 7f8586ddf640 1 --2- 172.21.15.143:0/2384975832 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f858013e290 0x7f85800cf280 secure :-1 s=READY pgs=922 cs=0 l=1 rev1=1 crypto rx=0x7f85800cfe40 tx=0x7f857c01c210 comp rx=0 tx=0).stop 2023-11-26T07:22:31.546 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:31.544+0000 7f8586ddf640 1 -- 172.21.15.143:0/2384975832 shutdown_connections 2023-11-26T07:22:31.546 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:31.544+0000 7f8586ddf640 1 --2- 172.21.15.143:0/2384975832 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f858015f4e0 0x7f85801408e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:22:31.546 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:31.544+0000 7f8586ddf640 1 --2- 172.21.15.143:0/2384975832 >> v2:172.21.15.188:3300/0 conn(0x7f85800cf7c0 0x7f858015efa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:22:31.546 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:31.544+0000 7f8586ddf640 1 --2- 172.21.15.143:0/2384975832 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f858013e290 0x7f85800cf280 unknown :-1 s=CLOSED pgs=922 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:22:31.546 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:31.544+0000 7f8586ddf640 1 -- 172.21.15.143:0/2384975832 >> 172.21.15.143:0/2384975832 conn(0x7f858006a890 msgr2=0x7f858006aca0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:22:31.546 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:31.544+0000 7f8586ddf640 1 -- 172.21.15.143:0/2384975832 shutdown_connections 2023-11-26T07:22:31.546 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:31.544+0000 7f8586ddf640 1 -- 172.21.15.143:0/2384975832 wait complete. 2023-11-26T07:22:31.547 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:31.544+0000 7f8586ddf640 1 Processor -- start 2023-11-26T07:22:31.547 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:31.544+0000 7f8586ddf640 1 -- start start 2023-11-26T07:22:31.549 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:31.548+0000 7f8586ddf640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f85800cf7c0 0x7f8580154450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:22:31.549 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:31.548+0000 7f8586ddf640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f858013e290 0x7f858015b6b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:22:31.549 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:31.548+0000 7f8586ddf640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f858015f4e0 0x7f858013cc40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:22:31.550 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:31.548+0000 7f8586ddf640 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8580141bd0 con 0x7f858015f4e0 2023-11-26T07:22:31.550 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:31.548+0000 7f8586ddf640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f858015df70 con 0x7f85800cf7c0 2023-11-26T07:22:31.550 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:31.548+0000 7f8586ddf640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f858015e250 con 0x7f858013e290 2023-11-26T07:22:31.550 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:31.548+0000 7f858537c640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f858015f4e0 0x7f858013cc40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:22:31.550 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:31.548+0000 7f858537c640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f858015f4e0 0x7f858013cc40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:47586/0 (socket says 172.21.15.143:47586) 2023-11-26T07:22:31.550 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:31.548+0000 7f858537c640 1 -- 172.21.15.143:0/1452751609 learned_addr learned my addr 172.21.15.143:0/1452751609 (peer_addr_for_me v2:172.21.15.143:0/0) 2023-11-26T07:22:31.550 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:31.548+0000 7f8584b7b640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f85800cf7c0 0x7f8580154450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:22:31.550 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:31.548+0000 7f8577fff640 1 --2- 172.21.15.143:0/1452751609 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f858013e290 0x7f858015b6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:22:31.550 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:31.548+0000 7f858537c640 1 -- 172.21.15.143:0/1452751609 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f858013e290 msgr2=0x7f858015b6b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:22:31.550 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:31.548+0000 7f858537c640 1 --2- 172.21.15.143:0/1452751609 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f858013e290 0x7f858015b6b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:22:31.550 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:31.548+0000 7f858537c640 1 -- 172.21.15.143:0/1452751609 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f85800cf7c0 msgr2=0x7f8580154450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:22:31.551 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:31.548+0000 7f858537c640 1 --2- 172.21.15.143:0/1452751609 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f85800cf7c0 0x7f8580154450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:22:31.551 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:31.548+0000 7f858537c640 1 -- 172.21.15.143:0/1452751609 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f857c0094d0 con 0x7f858015f4e0 2023-11-26T07:22:31.551 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:31.548+0000 7f858537c640 1 --2- 172.21.15.143:0/1452751609 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f858015f4e0 0x7f858013cc40 secure :-1 s=READY pgs=923 cs=0 l=1 rev1=1 crypto rx=0x7f857800b970 tx=0x7f857800be30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:22:31.551 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:31.548+0000 7f85757fa640 1 -- 172.21.15.143:0/1452751609 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f857800cb40 con 0x7f858015f4e0 2023-11-26T07:22:31.551 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:31.548+0000 7f85757fa640 1 -- 172.21.15.143:0/1452751609 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f857800ccc0 con 0x7f858015f4e0 2023-11-26T07:22:31.551 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:31.548+0000 7f85757fa640 1 -- 172.21.15.143:0/1452751609 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8578014680 con 0x7f858015f4e0 2023-11-26T07:22:31.551 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:31.548+0000 7f8586ddf640 1 -- 172.21.15.143:0/1452751609 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f858015e530 con 0x7f858015f4e0 2023-11-26T07:22:31.551 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:31.548+0000 7f8586ddf640 1 -- 172.21.15.143:0/1452751609 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f858015db90 con 0x7f858015f4e0 2023-11-26T07:22:31.552 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:31.548+0000 7f8586ddf640 1 -- 172.21.15.143:0/1452751609 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f85801419a0 con 0x7f858015f4e0 2023-11-26T07:22:31.552 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:31.548+0000 7f85757fa640 1 -- 172.21.15.143:0/1452751609 <== mon.0 v2:172.21.15.143:3300/0 4 ==== mgrmap(e 11) v1 ==== 86158+0+0 (secure 0 0 0) 0x7f857801d3b0 con 0x7f858015f4e0 2023-11-26T07:22:31.552 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:31.552+0000 7f85757fa640 1 --2- 172.21.15.143:0/1452751609 >> [v2:172.21.15.143:6832/2150557544,v1:172.21.15.143:6833/2150557544] conn(0x7f856c0667d0 0x7f856c068c90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:22:31.552 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:31.552+0000 7f85757fa640 1 -- 172.21.15.143:0/1452751609 <== mon.0 v2:172.21.15.143:3300/0 5 ==== osd_map(62..62 src has 1..62) v4 ==== 4943+0+0 (secure 0 0 0) 0x7f8578083f20 con 0x7f858015f4e0 2023-11-26T07:22:31.552 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:31.552+0000 7f8584b7b640 1 --2- 172.21.15.143:0/1452751609 >> [v2:172.21.15.143:6832/2150557544,v1:172.21.15.143:6833/2150557544] conn(0x7f856c0667d0 0x7f856c068c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:22:31.553 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:31.552+0000 7f8584b7b640 1 --2- 172.21.15.143:0/1452751609 >> [v2:172.21.15.143:6832/2150557544,v1:172.21.15.143:6833/2150557544] conn(0x7f856c0667d0 0x7f856c068c90 secure :-1 s=READY pgs=432 cs=0 l=1 rev1=1 crypto rx=0x7f857c0ad000 tx=0x7f857c0a6000 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:22:31.556 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:31.556+0000 7f85757fa640 1 -- 172.21.15.143:0/1452751609 <== mon.0 v2:172.21.15.143:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172649 (secure 0 0 0) 0x7f85801419a0 con 0x7f858015f4e0 2023-11-26T07:22:31.736 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:31.736+0000 7f8586ddf640 1 -- 172.21.15.143:0/1452751609 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f85801419a0 con 0x7f858015f4e0 2023-11-26T07:22:31.737 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:31.736+0000 7f85757fa640 1 -- 172.21.15.143:0/1452751609 <== mon.0 v2:172.21.15.143:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 37 v37) v1 ==== 94+0+3711 (secure 0 0 0) 0x7f85801419a0 con 0x7f858015f4e0 2023-11-26T07:22:31.737 INFO:teuthology.orchestra.run.smithi143.stdout: 2023-11-26T07:22:31.737 INFO:teuthology.orchestra.run.smithi143.stdout:{"epoch":37,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5612,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.143:6837/1372238588","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6836","nonce":1372238588},{"type":"v1","addr":"172.21.15.143:6837","nonce":1372238588}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":29},{"gid":6103,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.143:6835/1078738477","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6834","nonce":1078738477},{"type":"v1","addr":"172.21.15.143:6835","nonce":1078738477}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":29},{"gid":6118,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.188:6834/1634083777","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6832","nonce":1634083777},{"type":"v1","addr":"172.21.15.188:6834","nonce":1634083777}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":29},{"gid":6478,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.188:6835/1066289932","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6833","nonce":1066289932},{"type":"v1","addr":"172.21.15.188:6835","nonce":1066289932}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":37}],"filesystems":[]} 2023-11-26T07:22:31.737 INFO:teuthology.orchestra.run.smithi143.stderr:dumped fsmap epoch 37 2023-11-26T07:22:31.739 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:31.736+0000 7f8586ddf640 1 -- 172.21.15.143:0/1452751609 >> [v2:172.21.15.143:6832/2150557544,v1:172.21.15.143:6833/2150557544] conn(0x7f856c0667d0 msgr2=0x7f856c068c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:22:31.739 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:31.736+0000 7f8586ddf640 1 --2- 172.21.15.143:0/1452751609 >> [v2:172.21.15.143:6832/2150557544,v1:172.21.15.143:6833/2150557544] conn(0x7f856c0667d0 0x7f856c068c90 secure :-1 s=READY pgs=432 cs=0 l=1 rev1=1 crypto rx=0x7f857c0ad000 tx=0x7f857c0a6000 comp rx=0 tx=0).stop 2023-11-26T07:22:31.739 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:31.736+0000 7f8586ddf640 1 -- 172.21.15.143:0/1452751609 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f858015f4e0 msgr2=0x7f858013cc40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:22:31.739 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:31.736+0000 7f8586ddf640 1 --2- 172.21.15.143:0/1452751609 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f858015f4e0 0x7f858013cc40 secure :-1 s=READY pgs=923 cs=0 l=1 rev1=1 crypto rx=0x7f857800b970 tx=0x7f857800be30 comp rx=0 tx=0).stop 2023-11-26T07:22:31.739 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:31.736+0000 7f8586ddf640 1 -- 172.21.15.143:0/1452751609 shutdown_connections 2023-11-26T07:22:31.739 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:31.736+0000 7f8586ddf640 1 --2- 172.21.15.143:0/1452751609 >> [v2:172.21.15.143:6832/2150557544,v1:172.21.15.143:6833/2150557544] conn(0x7f856c0667d0 0x7f856c068c90 unknown :-1 s=CLOSED pgs=432 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:22:31.740 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:31.736+0000 7f8586ddf640 1 --2- 172.21.15.143:0/1452751609 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f858015f4e0 0x7f858013cc40 unknown :-1 s=CLOSED pgs=923 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:22:31.740 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:31.736+0000 7f8586ddf640 1 --2- 172.21.15.143:0/1452751609 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f858013e290 0x7f858015b6b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:22:31.740 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:31.736+0000 7f8586ddf640 1 --2- 172.21.15.143:0/1452751609 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f85800cf7c0 0x7f8580154450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:22:31.740 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:31.736+0000 7f8586ddf640 1 -- 172.21.15.143:0/1452751609 >> 172.21.15.143:0/1452751609 conn(0x7f858006a890 msgr2=0x7f85800ce660 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:22:31.740 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:31.736+0000 7f8586ddf640 1 -- 172.21.15.143:0/1452751609 shutdown_connections 2023-11-26T07:22:31.740 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:31.736+0000 7f8586ddf640 1 -- 172.21.15.143:0/1452751609 wait complete. 2023-11-26T07:22:31.759 INFO:tasks.ceph.mds.a:Restarting daemon 2023-11-26T07:22:31.759 INFO:tasks.ceph.mds.a:Stopping old one... 2023-11-26T07:22:31.759 DEBUG:tasks.ceph.mds.a:waiting for process to exit 2023-11-26T07:22:31.759 INFO:teuthology.orchestra.run:waiting for 300 2023-11-26T07:22:31.759 INFO:tasks.ceph.mds.c:Restarting daemon 2023-11-26T07:22:31.759 INFO:tasks.ceph.mds.c:Stopping old one... 2023-11-26T07:22:31.760 DEBUG:tasks.ceph.mds.c:waiting for process to exit 2023-11-26T07:22:31.760 INFO:teuthology.orchestra.run:waiting for 300 2023-11-26T07:22:31.760 INFO:tasks.ceph.mds.b:Restarting daemon 2023-11-26T07:22:31.760 INFO:tasks.ceph.mds.b:Stopping old one... 2023-11-26T07:22:31.760 DEBUG:tasks.ceph.mds.b:waiting for process to exit 2023-11-26T07:22:31.760 INFO:teuthology.orchestra.run:waiting for 300 2023-11-26T07:22:31.760 INFO:tasks.ceph.mds.d:Restarting daemon 2023-11-26T07:22:31.761 INFO:tasks.ceph.mds.d:Stopping old one... 2023-11-26T07:22:31.761 DEBUG:tasks.ceph.mds.d:waiting for process to exit 2023-11-26T07:22:31.761 INFO:teuthology.orchestra.run:waiting for 300 2023-11-26T07:22:31.774 INFO:tasks.ceph.mds.a:Stopped 2023-11-26T07:22:31.775 DEBUG:teuthology.orchestra.run.smithi143:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i a 2023-11-26T07:22:31.775 INFO:tasks.ceph.mds.c:Stopped 2023-11-26T07:22:31.775 DEBUG:teuthology.orchestra.run.smithi143:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i c 2023-11-26T07:22:31.777 INFO:tasks.ceph.mds.b:Stopped 2023-11-26T07:22:31.777 DEBUG:teuthology.orchestra.run.smithi188:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i b 2023-11-26T07:22:31.778 INFO:tasks.ceph.mds.d:Stopped 2023-11-26T07:22:31.778 DEBUG:teuthology.orchestra.run.smithi188:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i d 2023-11-26T07:22:31.822 INFO:tasks.ceph.mds.a:Started 2023-11-26T07:22:31.822 DEBUG:teuthology.parallel:result is None 2023-11-26T07:22:31.824 INFO:tasks.ceph.mds.c:Started 2023-11-26T07:22:31.824 DEBUG:teuthology.parallel:result is None 2023-11-26T07:22:31.827 INFO:tasks.ceph.mds.b:Started 2023-11-26T07:22:31.827 DEBUG:teuthology.parallel:result is None 2023-11-26T07:22:31.829 INFO:tasks.ceph.mds.d:Started 2023-11-26T07:22:31.830 DEBUG:teuthology.parallel:result is None 2023-11-26T07:22:31.830 DEBUG:teuthology.orchestra.run.smithi143:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist clear 2023-11-26T07:22:31.910 INFO:tasks.ceph.mds.a.smithi143.stdout:starting mds.a at 2023-11-26T07:22:31.911 INFO:tasks.ceph.mds.c.smithi143.stdout:starting mds.c at 2023-11-26T07:22:31.920 INFO:tasks.ceph.mds.b.smithi188.stdout:starting mds.b at 2023-11-26T07:22:31.921 INFO:tasks.ceph.mds.d.smithi188.stdout:starting mds.d at 2023-11-26T07:22:31.976 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:31.976+0000 7ffb23557640 1 Processor -- start 2023-11-26T07:22:31.976 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:31.976+0000 7ffb23557640 1 -- start start 2023-11-26T07:22:31.977 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:31.976+0000 7ffb23557640 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7ffb1c0d1bc0 0x7ffb1c0d0df0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:22:31.977 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:31.976+0000 7ffb23557640 1 -- --> v1:172.21.15.143:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ffb1c0cf0e0 con 0x7ffb1c0b2160 2023-11-26T07:22:31.977 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:31.976+0000 7ffb23557640 1 -- --> v1:172.21.15.188:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ffb1c0ceb20 con 0x7ffb1c15d6e0 2023-11-26T07:22:31.977 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:31.976+0000 7ffb23557640 1 -- --> v2:172.21.15.188:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ffb1c15dac0 con 0x7ffb1c0d1bc0 2023-11-26T07:22:31.977 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:31.976+0000 7ffb21af4640 1 --1- >> v1:172.21.15.143:6789/0 conn(0x7ffb1c0b2160 0x7ffb1c0b0750 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.143:6789/0 says I am v1:172.21.15.143:60306/0 (socket says 172.21.15.143:60306) 2023-11-26T07:22:31.977 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:31.976+0000 7ffb21af4640 1 -- 172.21.15.143:0/1951131448 learned_addr learned my addr 172.21.15.143:0/1951131448 (peer_addr_for_me v1:172.21.15.143:0/0) 2023-11-26T07:22:31.978 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:31.976+0000 7ffb20af2640 1 --2- 172.21.15.143:0/1951131448 >> v2:172.21.15.188:3300/0 conn(0x7ffb1c0d1bc0 0x7ffb1c0d0df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:22:31.978 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:31.976+0000 7ffb13fff640 1 -- 172.21.15.143:0/1951131448 <== mon.0 v1:172.21.15.143:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 418937152 0 0) 0x7ffb1c0cf0e0 con 0x7ffb1c0b2160 2023-11-26T07:22:31.978 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:31.976+0000 7ffb13fff640 1 -- 172.21.15.143:0/1951131448 --> v1:172.21.15.143:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7ffb08003530 con 0x7ffb1c0b2160 2023-11-26T07:22:31.978 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:31.976+0000 7ffb20af2640 1 -- 172.21.15.143:0/1951131448 >> v1:172.21.15.188:6789/0 conn(0x7ffb1c15d6e0 legacy=0x7ffb1c0d14b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:22:31.979 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:31.976+0000 7ffb20af2640 1 -- 172.21.15.143:0/1951131448 >> v1:172.21.15.143:6789/0 conn(0x7ffb1c0b2160 legacy=0x7ffb1c0b0750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:22:31.979 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:31.976+0000 7ffb20af2640 1 -- 172.21.15.143:0/1951131448 --> v2:172.21.15.188:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ffb1c0cee00 con 0x7ffb1c0d1bc0 2023-11-26T07:22:31.980 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:31.980+0000 7ffb20af2640 1 --2- 172.21.15.143:0/1951131448 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7ffb1c0d1bc0 0x7ffb1c0d0df0 secure :-1 s=READY pgs=369 cs=0 l=1 rev1=1 crypto rx=0x7ffb0c0097c0 tx=0x7ffb0c01c480 comp rx=0 tx=0).ready entity=mon.1 client_cookie=11dacf4e21552a83 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:22:31.981 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:31.980+0000 7ffb13fff640 1 -- 172.21.15.143:0/1951131448 <== mon.1 v2:172.21.15.188:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffb0c0ab600 con 0x7ffb1c0d1bc0 2023-11-26T07:22:31.981 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:31.980+0000 7ffb13fff640 1 -- 172.21.15.143:0/1951131448 <== mon.1 v2:172.21.15.188:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ffb0c0a7050 con 0x7ffb1c0d1bc0 2023-11-26T07:22:31.981 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:31.980+0000 7ffb13fff640 1 -- 172.21.15.143:0/1951131448 <== mon.1 v2:172.21.15.188:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffb0c0ab980 con 0x7ffb1c0d1bc0 2023-11-26T07:22:31.982 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:31.980+0000 7ffb23557640 1 -- 172.21.15.143:0/1951131448 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7ffb1c0d1bc0 msgr2=0x7ffb1c0d0df0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:22:31.982 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:31.980+0000 7ffb23557640 1 --2- 172.21.15.143:0/1951131448 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7ffb1c0d1bc0 0x7ffb1c0d0df0 secure :-1 s=READY pgs=369 cs=0 l=1 rev1=1 crypto rx=0x7ffb0c0097c0 tx=0x7ffb0c01c480 comp rx=0 tx=0).stop 2023-11-26T07:22:31.982 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:31.980+0000 7ffb23557640 1 -- 172.21.15.143:0/1951131448 shutdown_connections 2023-11-26T07:22:31.982 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:31.980+0000 7ffb23557640 1 --2- 172.21.15.143:0/1951131448 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7ffb1c0d1bc0 0x7ffb1c0d0df0 unknown :-1 s=CLOSED pgs=369 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:22:31.982 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:31.980+0000 7ffb23557640 1 -- 172.21.15.143:0/1951131448 >> 172.21.15.143:0/1951131448 conn(0x7ffb1c09c550 msgr2=0x7ffb1c0d3220 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:22:31.982 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:31.980+0000 7ffb23557640 1 -- 172.21.15.143:0/1951131448 shutdown_connections 2023-11-26T07:22:31.982 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:31.980+0000 7ffb23557640 1 -- 172.21.15.143:0/1951131448 wait complete. 2023-11-26T07:22:31.983 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:31.980+0000 7ffb23557640 1 Processor -- start 2023-11-26T07:22:31.983 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:31.984+0000 7ffb23557640 1 -- start start 2023-11-26T07:22:31.985 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:31.984+0000 7ffb23557640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7ffb1c0b2160 0x7ffb1c153d20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:22:31.985 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:31.984+0000 7ffb23557640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7ffb1c0d1bc0 0x7ffb1c151fc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:22:31.985 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:31.984+0000 7ffb23557640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7ffb1c15d6e0 0x7ffb1c15fbb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:22:31.985 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:31.984+0000 7ffb23557640 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffb1c15f700 con 0x7ffb1c0d1bc0 2023-11-26T07:22:31.985 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:31.984+0000 7ffb23557640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffb1c0b8a30 con 0x7ffb1c15d6e0 2023-11-26T07:22:31.985 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:31.984+0000 7ffb23557640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ffb1c0b8d10 con 0x7ffb1c0b2160 2023-11-26T07:22:31.985 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:31.984+0000 7ffb20af2640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7ffb1c0d1bc0 0x7ffb1c151fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:22:31.986 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:31.984+0000 7ffb20af2640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7ffb1c0d1bc0 0x7ffb1c151fc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:47632/0 (socket says 172.21.15.143:47632) 2023-11-26T07:22:31.986 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:31.984+0000 7ffb20af2640 1 -- 172.21.15.143:0/1932071884 learned_addr learned my addr 172.21.15.143:0/1932071884 (peer_addr_for_me v2:172.21.15.143:0/0) 2023-11-26T07:22:31.986 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:31.984+0000 7ffb20af2640 1 -- 172.21.15.143:0/1932071884 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7ffb1c0b2160 msgr2=0x7ffb1c153d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:22:31.986 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:31.984+0000 7ffb20af2640 1 --2- 172.21.15.143:0/1932071884 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7ffb1c0b2160 0x7ffb1c153d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:22:31.986 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:31.984+0000 7ffb20af2640 1 -- 172.21.15.143:0/1932071884 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7ffb1c15d6e0 msgr2=0x7ffb1c15fbb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:22:31.986 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:31.984+0000 7ffb20af2640 1 --2- 172.21.15.143:0/1932071884 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7ffb1c15d6e0 0x7ffb1c15fbb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:22:31.986 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:31.984+0000 7ffb20af2640 1 -- 172.21.15.143:0/1932071884 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ffb0c009420 con 0x7ffb1c0d1bc0 2023-11-26T07:22:31.986 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:31.984+0000 7ffb20af2640 1 --2- 172.21.15.143:0/1932071884 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7ffb1c0d1bc0 0x7ffb1c151fc0 secure :-1 s=READY pgs=930 cs=0 l=1 rev1=1 crypto rx=0x7ffb0c0ae040 tx=0x7ffb0c019220 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:22:31.987 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:31.984+0000 7ffb11ffb640 1 -- 172.21.15.143:0/1932071884 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffb0c0b4430 con 0x7ffb1c0d1bc0 2023-11-26T07:22:31.987 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:31.984+0000 7ffb11ffb640 1 -- 172.21.15.143:0/1932071884 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ffb0c0b4590 con 0x7ffb1c0d1bc0 2023-11-26T07:22:31.987 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:31.984+0000 7ffb11ffb640 1 -- 172.21.15.143:0/1932071884 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffb0c0b4940 con 0x7ffb1c0d1bc0 2023-11-26T07:22:31.987 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:31.984+0000 7ffb23557640 1 -- 172.21.15.143:0/1932071884 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ffb1c0b8ff0 con 0x7ffb1c0d1bc0 2023-11-26T07:22:31.987 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:31.984+0000 7ffb23557640 1 -- 172.21.15.143:0/1932071884 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ffb1c0b6290 con 0x7ffb1c0d1bc0 2023-11-26T07:22:31.987 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:31.984+0000 7ffb23557640 1 -- 172.21.15.143:0/1932071884 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ffb1c15f870 con 0x7ffb1c0d1bc0 2023-11-26T07:22:31.988 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:31.988+0000 7ffb11ffb640 1 -- 172.21.15.143:0/1932071884 <== mon.0 v2:172.21.15.143:3300/0 4 ==== mgrmap(e 11) v1 ==== 86158+0+0 (secure 0 0 0) 0x7ffb0c0b2040 con 0x7ffb1c0d1bc0 2023-11-26T07:22:31.988 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:31.988+0000 7ffb11ffb640 1 --2- 172.21.15.143:0/1932071884 >> [v2:172.21.15.143:6832/2150557544,v1:172.21.15.143:6833/2150557544] conn(0x7ffb000667d0 0x7ffb00068c90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:22:31.989 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:31.988+0000 7ffb11ffb640 1 -- 172.21.15.143:0/1932071884 <== mon.0 v2:172.21.15.143:3300/0 5 ==== osd_map(62..62 src has 1..62) v4 ==== 4943+0+0 (secure 0 0 0) 0x7ffb0c11e090 con 0x7ffb1c0d1bc0 2023-11-26T07:22:31.989 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:31.988+0000 7ffb212f3640 1 --2- 172.21.15.143:0/1932071884 >> [v2:172.21.15.143:6832/2150557544,v1:172.21.15.143:6833/2150557544] conn(0x7ffb000667d0 0x7ffb00068c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:22:31.989 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:31.988+0000 7ffb212f3640 1 --2- 172.21.15.143:0/1932071884 >> [v2:172.21.15.143:6832/2150557544,v1:172.21.15.143:6833/2150557544] conn(0x7ffb000667d0 0x7ffb00068c90 secure :-1 s=READY pgs=433 cs=0 l=1 rev1=1 crypto rx=0x7ffb18000f70 tx=0x7ffb180079c0 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:22:31.992 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:31.992+0000 7ffb11ffb640 1 -- 172.21.15.143:0/1932071884 <== mon.0 v2:172.21.15.143:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172649 (secure 0 0 0) 0x7ffb0c0eb2b0 con 0x7ffb1c0d1bc0 2023-11-26T07:22:32.153 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:32.152+0000 7ffb23557640 1 -- 172.21.15.143:0/1932071884 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "osd blocklist clear"} v 0) v1 -- 0x7ffb1c067680 con 0x7ffb1c0d1bc0 2023-11-26T07:22:33.176 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:33.176+0000 7ffb11ffb640 1 -- 172.21.15.143:0/1932071884 <== mon.0 v2:172.21.15.143:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist clear"}]=0 removed all blocklist entries v67) v1 ==== 97+0+0 (secure 0 0 0) 0x7ffb0c0ee8d0 con 0x7ffb1c0d1bc0 2023-11-26T07:22:33.176 INFO:teuthology.orchestra.run.smithi143.stderr: removed all blocklist entries 2023-11-26T07:22:33.180 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:33.180+0000 7ffb23557640 1 -- 172.21.15.143:0/1932071884 >> [v2:172.21.15.143:6832/2150557544,v1:172.21.15.143:6833/2150557544] conn(0x7ffb000667d0 msgr2=0x7ffb00068c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:22:33.180 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:33.180+0000 7ffb23557640 1 --2- 172.21.15.143:0/1932071884 >> [v2:172.21.15.143:6832/2150557544,v1:172.21.15.143:6833/2150557544] conn(0x7ffb000667d0 0x7ffb00068c90 secure :-1 s=READY pgs=433 cs=0 l=1 rev1=1 crypto rx=0x7ffb18000f70 tx=0x7ffb180079c0 comp rx=0 tx=0).stop 2023-11-26T07:22:33.180 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:33.180+0000 7ffb23557640 1 -- 172.21.15.143:0/1932071884 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7ffb1c0d1bc0 msgr2=0x7ffb1c151fc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:22:33.180 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:33.180+0000 7ffb23557640 1 --2- 172.21.15.143:0/1932071884 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7ffb1c0d1bc0 0x7ffb1c151fc0 secure :-1 s=READY pgs=930 cs=0 l=1 rev1=1 crypto rx=0x7ffb0c0ae040 tx=0x7ffb0c019220 comp rx=0 tx=0).stop 2023-11-26T07:22:33.180 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:33.180+0000 7ffb23557640 1 -- 172.21.15.143:0/1932071884 shutdown_connections 2023-11-26T07:22:33.180 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:33.180+0000 7ffb23557640 1 --2- 172.21.15.143:0/1932071884 >> [v2:172.21.15.143:6832/2150557544,v1:172.21.15.143:6833/2150557544] conn(0x7ffb000667d0 0x7ffb00068c90 unknown :-1 s=CLOSED pgs=433 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:22:33.180 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:33.180+0000 7ffb23557640 1 --2- 172.21.15.143:0/1932071884 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7ffb1c15d6e0 0x7ffb1c15fbb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:22:33.180 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:33.180+0000 7ffb23557640 1 --2- 172.21.15.143:0/1932071884 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7ffb1c0d1bc0 0x7ffb1c151fc0 unknown :-1 s=CLOSED pgs=930 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:22:33.181 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:33.180+0000 7ffb23557640 1 --2- 172.21.15.143:0/1932071884 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7ffb1c0b2160 0x7ffb1c153d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:22:33.181 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:33.180+0000 7ffb23557640 1 -- 172.21.15.143:0/1932071884 >> 172.21.15.143:0/1932071884 conn(0x7ffb1c09c550 msgr2=0x7ffb1c06c920 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:22:33.181 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:33.180+0000 7ffb23557640 1 -- 172.21.15.143:0/1932071884 shutdown_connections 2023-11-26T07:22:33.181 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:33.180+0000 7ffb23557640 1 -- 172.21.15.143:0/1932071884 wait complete. 2023-11-26T07:22:33.203 DEBUG:teuthology.orchestra.run.smithi143:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth ls --format=json-pretty 2023-11-26T07:22:33.344 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:33.344+0000 7f8b58e9c640 1 Processor -- start 2023-11-26T07:22:33.344 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:33.344+0000 7f8b58e9c640 1 -- start start 2023-11-26T07:22:33.345 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:33.344+0000 7f8b58e9c640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f8b540b0aa0 0x7f8b540b2040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:22:33.345 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:33.344+0000 7f8b58e9c640 1 -- --> v1:172.21.15.143:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8b5406b5a0 con 0x7f8b54164e90 2023-11-26T07:22:33.346 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:33.344+0000 7f8b58e9c640 1 -- --> v1:172.21.15.188:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8b540ce160 con 0x7f8b5409a670 2023-11-26T07:22:33.346 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:33.344+0000 7f8b58e9c640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8b540ce3d0 con 0x7f8b540b0aa0 2023-11-26T07:22:33.346 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:33.344+0000 7f8b53fff640 1 --1- >> v1:172.21.15.143:6789/0 conn(0x7f8b54164e90 0x7f8b54167280 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.143:6789/0 says I am v1:172.21.15.143:60314/0 (socket says 172.21.15.143:60314) 2023-11-26T07:22:33.346 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:33.344+0000 7f8b53fff640 1 -- 172.21.15.143:0/3747284820 learned_addr learned my addr 172.21.15.143:0/3747284820 (peer_addr_for_me v1:172.21.15.143:0/0) 2023-11-26T07:22:33.348 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:33.344+0000 7f8b537fe640 1 --2- 172.21.15.143:0/3747284820 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f8b540b0aa0 0x7f8b540b2040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:22:33.348 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:33.344+0000 7f8b527fc640 1 -- 172.21.15.143:0/3747284820 <== mon.0 v1:172.21.15.143:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1681574629 0 0) 0x7f8b5406b5a0 con 0x7f8b54164e90 2023-11-26T07:22:33.348 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:33.344+0000 7f8b527fc640 1 -- 172.21.15.143:0/3747284820 --> v1:172.21.15.143:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f8b3c003530 con 0x7f8b54164e90 2023-11-26T07:22:33.348 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:33.344+0000 7f8b527fc640 1 -- 172.21.15.143:0/3747284820 <== mon.1 v1:172.21.15.188:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1655482022 0 0) 0x7f8b540ce160 con 0x7f8b5409a670 2023-11-26T07:22:33.348 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:33.344+0000 7f8b527fc640 1 -- 172.21.15.143:0/3747284820 --> v1:172.21.15.188:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f8b5406b5a0 con 0x7f8b5409a670 2023-11-26T07:22:33.348 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:33.344+0000 7f8b527fc640 1 -- 172.21.15.143:0/3747284820 <== mon.0 v1:172.21.15.143:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2457353492 0 0) 0x7f8b3c003530 con 0x7f8b54164e90 2023-11-26T07:22:33.348 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:33.344+0000 7f8b527fc640 1 -- 172.21.15.143:0/3747284820 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f8b540b0aa0 msgr2=0x7f8b540b2040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-26T07:22:33.348 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:33.344+0000 7f8b527fc640 1 --2- 172.21.15.143:0/3747284820 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f8b540b0aa0 0x7f8b540b2040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:22:33.348 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:33.344+0000 7f8b527fc640 1 -- 172.21.15.143:0/3747284820 >> v1:172.21.15.188:6789/0 conn(0x7f8b5409a670 legacy=0x7f8b540b0e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:22:33.348 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:33.344+0000 7f8b527fc640 1 -- 172.21.15.143:0/3747284820 --> v1:172.21.15.143:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8b540ce6b0 con 0x7f8b54164e90 2023-11-26T07:22:33.349 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:33.344+0000 7f8b527fc640 1 -- 172.21.15.143:0/3747284820 <== mon.0 v1:172.21.15.143:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2137882523 0 0) 0x7f8b4c001980 con 0x7f8b54164e90 2023-11-26T07:22:33.349 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:33.344+0000 7f8b527fc640 1 -- 172.21.15.143:0/3747284820 >> v1:172.21.15.143:6789/0 conn(0x7f8b54164e90 legacy=0x7f8b54167280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:22:33.349 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:33.344+0000 7f8b527fc640 1 --2- 172.21.15.143:0/3747284820 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f8b3c003b40 0x7f8b3c003f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:22:33.349 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:33.344+0000 7f8b527fc640 1 --2- 172.21.15.143:0/3747284820 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f8b3c0047d0 0x7f8b3c004c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:22:33.349 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:33.344+0000 7f8b527fc640 1 --2- 172.21.15.143:0/3747284820 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f8b540b0aa0 0x7f8b3c005190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:22:33.349 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:33.344+0000 7f8b527fc640 1 -- 172.21.15.143:0/3747284820 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8b540ce3d0 con 0x7f8b3c003b40 2023-11-26T07:22:33.349 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:33.344+0000 7f8b527fc640 1 -- 172.21.15.143:0/3747284820 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8b3c00ba40 con 0x7f8b3c0047d0 2023-11-26T07:22:33.349 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:33.344+0000 7f8b527fc640 1 -- 172.21.15.143:0/3747284820 --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8b3c00bc00 con 0x7f8b540b0aa0 2023-11-26T07:22:33.349 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:33.344+0000 7f8b537fe640 1 --2- 172.21.15.143:0/3747284820 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f8b3c003b40 0x7f8b3c003f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:22:33.349 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:33.344+0000 7f8b52ffd640 1 --2- 172.21.15.143:0/3747284820 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f8b3c0047d0 0x7f8b3c004c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:22:33.349 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:33.344+0000 7f8b53fff640 1 --2- 172.21.15.143:0/3747284820 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f8b540b0aa0 0x7f8b3c005190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:22:33.349 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:33.344+0000 7f8b537fe640 1 -- 172.21.15.143:0/3747284820 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f8b540b0aa0 msgr2=0x7f8b3c005190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-26T07:22:33.350 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:33.344+0000 7f8b537fe640 1 --2- 172.21.15.143:0/3747284820 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f8b540b0aa0 0x7f8b3c005190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:22:33.350 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:33.344+0000 7f8b537fe640 1 -- 172.21.15.143:0/3747284820 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f8b3c0047d0 msgr2=0x7f8b3c004c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-26T07:22:33.350 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:33.344+0000 7f8b537fe640 1 --2- 172.21.15.143:0/3747284820 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f8b3c0047d0 0x7f8b3c004c50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:22:33.350 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:33.344+0000 7f8b537fe640 1 -- 172.21.15.143:0/3747284820 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8b3c003530 con 0x7f8b3c003b40 2023-11-26T07:22:33.351 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:33.348+0000 7f8b537fe640 1 --2- 172.21.15.143:0/3747284820 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f8b3c003b40 0x7f8b3c003f40 secure :-1 s=READY pgs=932 cs=0 l=1 rev1=1 crypto rx=0x7f8b44009ae0 tx=0x7f8b4401ca50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cff30d2f7c048406 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:22:33.351 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:33.348+0000 7f8b527fc640 1 -- 172.21.15.143:0/3747284820 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8b440a55b0 con 0x7f8b3c003b40 2023-11-26T07:22:33.351 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:33.348+0000 7f8b527fc640 1 -- 172.21.15.143:0/3747284820 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8b440a5710 con 0x7f8b3c003b40 2023-11-26T07:22:33.351 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:33.348+0000 7f8b527fc640 1 -- 172.21.15.143:0/3747284820 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8b440a5ac0 con 0x7f8b3c003b40 2023-11-26T07:22:33.351 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:33.348+0000 7f8b58e9c640 1 -- 172.21.15.143:0/3747284820 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f8b3c003b40 msgr2=0x7f8b3c003f40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:22:33.351 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:33.348+0000 7f8b58e9c640 1 --2- 172.21.15.143:0/3747284820 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f8b3c003b40 0x7f8b3c003f40 secure :-1 s=READY pgs=932 cs=0 l=1 rev1=1 crypto rx=0x7f8b44009ae0 tx=0x7f8b4401ca50 comp rx=0 tx=0).stop 2023-11-26T07:22:33.351 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:33.348+0000 7f8b53fff640 1 -- 172.21.15.143:0/3747284820 reap_dead start 2023-11-26T07:22:33.351 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:33.348+0000 7f8b58e9c640 1 -- 172.21.15.143:0/3747284820 shutdown_connections 2023-11-26T07:22:33.351 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:33.348+0000 7f8b58e9c640 1 -- 172.21.15.143:0/3747284820 >> 172.21.15.143:0/3747284820 conn(0x7f8b54099a70 msgr2=0x7f8b54099e80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:22:33.352 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:33.348+0000 7f8b58e9c640 1 -- 172.21.15.143:0/3747284820 shutdown_connections 2023-11-26T07:22:33.352 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:33.348+0000 7f8b58e9c640 1 -- 172.21.15.143:0/3747284820 wait complete. 2023-11-26T07:22:33.352 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:33.352+0000 7f8b58e9c640 1 Processor -- start 2023-11-26T07:22:33.353 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:33.352+0000 7f8b58e9c640 1 -- start start 2023-11-26T07:22:33.355 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:33.352+0000 7f8b58e9c640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f8b3c003b40 0x7f8b5414db70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:22:33.355 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:33.352+0000 7f8b58e9c640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f8b5414d620 0x7f8b5414aa10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:22:33.355 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:33.352+0000 7f8b58e9c640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f8b5406f4f0 0x7f8b5406f9a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:22:33.355 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:33.352+0000 7f8b58e9c640 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8b5414af50 con 0x7f8b3c003b40 2023-11-26T07:22:33.355 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:33.352+0000 7f8b58e9c640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8b5406fee0 con 0x7f8b5414d620 2023-11-26T07:22:33.356 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:33.352+0000 7f8b58e9c640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8b541fe8b0 con 0x7f8b5406f4f0 2023-11-26T07:22:33.356 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:33.352+0000 7f8b537fe640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f8b3c003b40 0x7f8b5414db70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:22:33.356 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:33.352+0000 7f8b537fe640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f8b3c003b40 0x7f8b5414db70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:47652/0 (socket says 172.21.15.143:47652) 2023-11-26T07:22:33.356 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:33.352+0000 7f8b537fe640 1 -- 172.21.15.143:0/3185619854 learned_addr learned my addr 172.21.15.143:0/3185619854 (peer_addr_for_me v2:172.21.15.143:0/0) 2023-11-26T07:22:33.356 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:33.352+0000 7f8b52ffd640 1 --2- 172.21.15.143:0/3185619854 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f8b5414d620 0x7f8b5414aa10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:22:33.356 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:33.352+0000 7f8b53fff640 1 --2- 172.21.15.143:0/3185619854 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f8b5406f4f0 0x7f8b5406f9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:22:33.356 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:33.352+0000 7f8b537fe640 1 -- 172.21.15.143:0/3185619854 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f8b5406f4f0 msgr2=0x7f8b5406f9a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:22:33.356 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:33.352+0000 7f8b537fe640 1 --2- 172.21.15.143:0/3185619854 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f8b5406f4f0 0x7f8b5406f9a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:22:33.356 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:33.352+0000 7f8b537fe640 1 -- 172.21.15.143:0/3185619854 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f8b5414d620 msgr2=0x7f8b5414aa10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:22:33.356 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:33.352+0000 7f8b537fe640 1 --2- 172.21.15.143:0/3185619854 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f8b5414d620 0x7f8b5414aa10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:22:33.356 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:33.352+0000 7f8b537fe640 1 -- 172.21.15.143:0/3185619854 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8b540b2cc0 con 0x7f8b3c003b40 2023-11-26T07:22:33.356 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:33.352+0000 7f8b537fe640 1 --2- 172.21.15.143:0/3185619854 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f8b3c003b40 0x7f8b5414db70 secure :-1 s=READY pgs=933 cs=0 l=1 rev1=1 crypto rx=0x7f8b440b0040 tx=0x7f8b3c005800 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:22:33.357 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:33.352+0000 7f8b3bfff640 1 -- 172.21.15.143:0/3185619854 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8b440a55b0 con 0x7f8b3c003b40 2023-11-26T07:22:33.357 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:33.352+0000 7f8b3bfff640 1 -- 172.21.15.143:0/3185619854 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8b440a5e80 con 0x7f8b3c003b40 2023-11-26T07:22:33.357 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:33.352+0000 7f8b58e9c640 1 -- 172.21.15.143:0/3185619854 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8b540b4100 con 0x7f8b3c003b40 2023-11-26T07:22:33.357 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:33.352+0000 7f8b3bfff640 1 -- 172.21.15.143:0/3185619854 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8b44004240 con 0x7f8b3c003b40 2023-11-26T07:22:33.357 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:33.352+0000 7f8b58e9c640 1 -- 172.21.15.143:0/3185619854 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8b54160210 con 0x7f8b3c003b40 2023-11-26T07:22:33.357 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:33.356+0000 7f8b58e9c640 1 -- 172.21.15.143:0/3185619854 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8b54160060 con 0x7f8b3c003b40 2023-11-26T07:22:33.357 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:33.356+0000 7f8b3bfff640 1 -- 172.21.15.143:0/3185619854 <== mon.0 v2:172.21.15.143:3300/0 4 ==== mgrmap(e 11) v1 ==== 86158+0+0 (secure 0 0 0) 0x7f8b440b4040 con 0x7f8b3c003b40 2023-11-26T07:22:33.358 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:33.356+0000 7f8b3bfff640 1 --2- 172.21.15.143:0/3185619854 >> [v2:172.21.15.143:6832/2150557544,v1:172.21.15.143:6833/2150557544] conn(0x7f8b400667d0 0x7f8b40068c90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:22:33.358 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:33.356+0000 7f8b3bfff640 1 -- 172.21.15.143:0/3185619854 <== mon.0 v2:172.21.15.143:3300/0 5 ==== osd_map(67..67 src has 1..67) v4 ==== 4857+0+0 (secure 0 0 0) 0x7f8b4411e090 con 0x7f8b3c003b40 2023-11-26T07:22:33.358 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:33.356+0000 7f8b52ffd640 1 --2- 172.21.15.143:0/3185619854 >> [v2:172.21.15.143:6832/2150557544,v1:172.21.15.143:6833/2150557544] conn(0x7f8b400667d0 0x7f8b40068c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:22:33.359 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:33.356+0000 7f8b52ffd640 1 --2- 172.21.15.143:0/3185619854 >> [v2:172.21.15.143:6832/2150557544,v1:172.21.15.143:6833/2150557544] conn(0x7f8b400667d0 0x7f8b40068c90 secure :-1 s=READY pgs=438 cs=0 l=1 rev1=1 crypto rx=0x7f8b5414db20 tx=0x7f8b48009040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:22:33.362 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:33.360+0000 7f8b3bfff640 1 -- 172.21.15.143:0/3185619854 <== mon.0 v2:172.21.15.143:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172649 (secure 0 0 0) 0x7f8b440eb490 con 0x7f8b3c003b40 2023-11-26T07:22:33.563 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:33.560+0000 7f8b58e9c640 1 -- 172.21.15.143:0/3185619854 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "auth ls", "format": "json-pretty"} v 0) v1 -- 0x7f8b54067500 con 0x7f8b3c003b40 2023-11-26T07:22:33.563 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:33.560+0000 7f8b3bfff640 1 -- 172.21.15.143:0/3185619854 <== mon.0 v2:172.21.15.143:3300/0 7 ==== mon_command_ack([{"prefix": "auth ls", "format": "json-pretty"}]=0 v16) v1 ==== 80+0+5615 (secure 0 0 0) 0x7f8b440eeab0 con 0x7f8b3c003b40 2023-11-26T07:22:33.563 INFO:teuthology.orchestra.run.smithi143.stdout: 2023-11-26T07:22:33.564 INFO:teuthology.orchestra.run.smithi143.stdout:{ 2023-11-26T07:22:33.564 INFO:teuthology.orchestra.run.smithi143.stdout: "auth_dump": [ 2023-11-26T07:22:33.564 INFO:teuthology.orchestra.run.smithi143.stdout: { 2023-11-26T07:22:33.564 INFO:teuthology.orchestra.run.smithi143.stdout: "entity": "mds.a", 2023-11-26T07:22:33.564 INFO:teuthology.orchestra.run.smithi143.stdout: "key": "AQDl8GJldeBsOBAAhk75XZnQRURKcr0MrW17Lg==", 2023-11-26T07:22:33.564 INFO:teuthology.orchestra.run.smithi143.stdout: "caps": { 2023-11-26T07:22:33.564 INFO:teuthology.orchestra.run.smithi143.stdout: "mds": "allow", 2023-11-26T07:22:33.564 INFO:teuthology.orchestra.run.smithi143.stdout: "mgr": "allow *", 2023-11-26T07:22:33.564 INFO:teuthology.orchestra.run.smithi143.stdout: "mon": "allow *", 2023-11-26T07:22:33.565 INFO:teuthology.orchestra.run.smithi143.stdout: "osd": "allow *" 2023-11-26T07:22:33.565 INFO:teuthology.orchestra.run.smithi143.stdout: } 2023-11-26T07:22:33.565 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2023-11-26T07:22:33.565 INFO:teuthology.orchestra.run.smithi143.stdout: { 2023-11-26T07:22:33.565 INFO:teuthology.orchestra.run.smithi143.stdout: "entity": "mds.b", 2023-11-26T07:22:33.565 INFO:teuthology.orchestra.run.smithi143.stdout: "key": "AQDm8GJl0lvhCBAAXuvHKoB8TAhp0bJ4DcV94g==", 2023-11-26T07:22:33.565 INFO:teuthology.orchestra.run.smithi143.stdout: "caps": { 2023-11-26T07:22:33.565 INFO:teuthology.orchestra.run.smithi143.stdout: "mds": "allow", 2023-11-26T07:22:33.565 INFO:teuthology.orchestra.run.smithi143.stdout: "mgr": "allow *", 2023-11-26T07:22:33.565 INFO:teuthology.orchestra.run.smithi143.stdout: "mon": "allow *", 2023-11-26T07:22:33.566 INFO:teuthology.orchestra.run.smithi143.stdout: "osd": "allow *" 2023-11-26T07:22:33.566 INFO:teuthology.orchestra.run.smithi143.stdout: } 2023-11-26T07:22:33.566 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2023-11-26T07:22:33.566 INFO:teuthology.orchestra.run.smithi143.stdout: { 2023-11-26T07:22:33.566 INFO:teuthology.orchestra.run.smithi143.stdout: "entity": "mds.c", 2023-11-26T07:22:33.566 INFO:teuthology.orchestra.run.smithi143.stdout: "key": "AQDm8GJluQ0PBBAA23EmVdD3bVk52h6izUXDRQ==", 2023-11-26T07:22:33.566 INFO:teuthology.orchestra.run.smithi143.stdout: "caps": { 2023-11-26T07:22:33.566 INFO:teuthology.orchestra.run.smithi143.stdout: "mds": "allow", 2023-11-26T07:22:33.566 INFO:teuthology.orchestra.run.smithi143.stdout: "mgr": "allow *", 2023-11-26T07:22:33.566 INFO:teuthology.orchestra.run.smithi143.stdout: "mon": "allow *", 2023-11-26T07:22:33.567 INFO:teuthology.orchestra.run.smithi143.stdout: "osd": "allow *" 2023-11-26T07:22:33.567 INFO:teuthology.orchestra.run.smithi143.stdout: } 2023-11-26T07:22:33.567 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2023-11-26T07:22:33.567 INFO:teuthology.orchestra.run.smithi143.stdout: { 2023-11-26T07:22:33.567 INFO:teuthology.orchestra.run.smithi143.stdout: "entity": "mds.d", 2023-11-26T07:22:33.567 INFO:teuthology.orchestra.run.smithi143.stdout: "key": "AQDm8GJlkt9nEBAAg+LYtKVNHAN/fwYjEcNJHQ==", 2023-11-26T07:22:33.567 INFO:teuthology.orchestra.run.smithi143.stdout: "caps": { 2023-11-26T07:22:33.567 INFO:teuthology.orchestra.run.smithi143.stdout: "mds": "allow", 2023-11-26T07:22:33.567 INFO:teuthology.orchestra.run.smithi143.stdout: "mgr": "allow *", 2023-11-26T07:22:33.567 INFO:teuthology.orchestra.run.smithi143.stdout: "mon": "allow *", 2023-11-26T07:22:33.567 INFO:teuthology.orchestra.run.smithi143.stdout: "osd": "allow *" 2023-11-26T07:22:33.568 INFO:teuthology.orchestra.run.smithi143.stdout: } 2023-11-26T07:22:33.568 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2023-11-26T07:22:33.568 INFO:teuthology.orchestra.run.smithi143.stdout: { 2023-11-26T07:22:33.568 INFO:teuthology.orchestra.run.smithi143.stdout: "entity": "osd.0", 2023-11-26T07:22:33.568 INFO:teuthology.orchestra.run.smithi143.stdout: "key": "AQDp8GJlNscZLhAA31zK4vTcnxwiRIRgxzewpA==", 2023-11-26T07:22:33.568 INFO:teuthology.orchestra.run.smithi143.stdout: "caps": { 2023-11-26T07:22:33.568 INFO:teuthology.orchestra.run.smithi143.stdout: "mgr": "allow profile osd", 2023-11-26T07:22:33.568 INFO:teuthology.orchestra.run.smithi143.stdout: "mon": "allow profile osd", 2023-11-26T07:22:33.568 INFO:teuthology.orchestra.run.smithi143.stdout: "osd": "allow *" 2023-11-26T07:22:33.568 INFO:teuthology.orchestra.run.smithi143.stdout: } 2023-11-26T07:22:33.568 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2023-11-26T07:22:33.569 INFO:teuthology.orchestra.run.smithi143.stdout: { 2023-11-26T07:22:33.569 INFO:teuthology.orchestra.run.smithi143.stdout: "entity": "osd.1", 2023-11-26T07:22:33.569 INFO:teuthology.orchestra.run.smithi143.stdout: "key": "AQDs8GJlsyKUNBAA+dQ1ElGPrdP1GV7lk4JQjQ==", 2023-11-26T07:22:33.569 INFO:teuthology.orchestra.run.smithi143.stdout: "caps": { 2023-11-26T07:22:33.569 INFO:teuthology.orchestra.run.smithi143.stdout: "mgr": "allow profile osd", 2023-11-26T07:22:33.569 INFO:teuthology.orchestra.run.smithi143.stdout: "mon": "allow profile osd", 2023-11-26T07:22:33.569 INFO:teuthology.orchestra.run.smithi143.stdout: "osd": "allow *" 2023-11-26T07:22:33.569 INFO:teuthology.orchestra.run.smithi143.stdout: } 2023-11-26T07:22:33.569 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2023-11-26T07:22:33.569 INFO:teuthology.orchestra.run.smithi143.stdout: { 2023-11-26T07:22:33.570 INFO:teuthology.orchestra.run.smithi143.stdout: "entity": "osd.2", 2023-11-26T07:22:33.570 INFO:teuthology.orchestra.run.smithi143.stdout: "key": "AQDv8GJlXsy7OhAAJJdldsodgGZVtyV2HuyC4A==", 2023-11-26T07:22:33.570 INFO:teuthology.orchestra.run.smithi143.stdout: "caps": { 2023-11-26T07:22:33.570 INFO:teuthology.orchestra.run.smithi143.stdout: "mgr": "allow profile osd", 2023-11-26T07:22:33.570 INFO:teuthology.orchestra.run.smithi143.stdout: "mon": "allow profile osd", 2023-11-26T07:22:33.570 INFO:teuthology.orchestra.run.smithi143.stdout: "osd": "allow *" 2023-11-26T07:22:33.570 INFO:teuthology.orchestra.run.smithi143.stdout: } 2023-11-26T07:22:33.570 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2023-11-26T07:22:33.570 INFO:teuthology.orchestra.run.smithi143.stdout: { 2023-11-26T07:22:33.570 INFO:teuthology.orchestra.run.smithi143.stdout: "entity": "osd.3", 2023-11-26T07:22:33.570 INFO:teuthology.orchestra.run.smithi143.stdout: "key": "AQDz8GJloLhMBxAAOWD9ec2y4c8nkdJmQnne8Q==", 2023-11-26T07:22:33.571 INFO:teuthology.orchestra.run.smithi143.stdout: "caps": { 2023-11-26T07:22:33.571 INFO:teuthology.orchestra.run.smithi143.stdout: "mgr": "allow profile osd", 2023-11-26T07:22:33.571 INFO:teuthology.orchestra.run.smithi143.stdout: "mon": "allow profile osd", 2023-11-26T07:22:33.571 INFO:teuthology.orchestra.run.smithi143.stdout: "osd": "allow *" 2023-11-26T07:22:33.571 INFO:teuthology.orchestra.run.smithi143.stdout: } 2023-11-26T07:22:33.571 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2023-11-26T07:22:33.571 INFO:teuthology.orchestra.run.smithi143.stdout: { 2023-11-26T07:22:33.571 INFO:teuthology.orchestra.run.smithi143.stdout: "entity": "osd.4", 2023-11-26T07:22:33.571 INFO:teuthology.orchestra.run.smithi143.stdout: "key": "AQD58GJlDlZsDBAAZNauAPPPfAZ+KIqSmgSQhg==", 2023-11-26T07:22:33.571 INFO:teuthology.orchestra.run.smithi143.stdout: "caps": { 2023-11-26T07:22:33.572 INFO:teuthology.orchestra.run.smithi143.stdout: "mgr": "allow profile osd", 2023-11-26T07:22:33.572 INFO:teuthology.orchestra.run.smithi143.stdout: "mon": "allow profile osd", 2023-11-26T07:22:33.572 INFO:teuthology.orchestra.run.smithi143.stdout: "osd": "allow *" 2023-11-26T07:22:33.572 INFO:teuthology.orchestra.run.smithi143.stdout: } 2023-11-26T07:22:33.572 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2023-11-26T07:22:33.572 INFO:teuthology.orchestra.run.smithi143.stdout: { 2023-11-26T07:22:33.572 INFO:teuthology.orchestra.run.smithi143.stdout: "entity": "osd.5", 2023-11-26T07:22:33.572 INFO:teuthology.orchestra.run.smithi143.stdout: "key": "AQD88GJlNE4mExAA5jYkCa0tE3oR6rgK1Dt5cA==", 2023-11-26T07:22:33.572 INFO:teuthology.orchestra.run.smithi143.stdout: "caps": { 2023-11-26T07:22:33.572 INFO:teuthology.orchestra.run.smithi143.stdout: "mgr": "allow profile osd", 2023-11-26T07:22:33.572 INFO:teuthology.orchestra.run.smithi143.stdout: "mon": "allow profile osd", 2023-11-26T07:22:33.573 INFO:teuthology.orchestra.run.smithi143.stdout: "osd": "allow *" 2023-11-26T07:22:33.573 INFO:teuthology.orchestra.run.smithi143.stdout: } 2023-11-26T07:22:33.573 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2023-11-26T07:22:33.573 INFO:teuthology.orchestra.run.smithi143.stdout: { 2023-11-26T07:22:33.573 INFO:teuthology.orchestra.run.smithi143.stdout: "entity": "osd.6", 2023-11-26T07:22:33.573 INFO:teuthology.orchestra.run.smithi143.stdout: "key": "AQD/8GJlh0lYGRAArHalCyZ36UvBOVMZor+N/g==", 2023-11-26T07:22:33.573 INFO:teuthology.orchestra.run.smithi143.stdout: "caps": { 2023-11-26T07:22:33.573 INFO:teuthology.orchestra.run.smithi143.stdout: "mgr": "allow profile osd", 2023-11-26T07:22:33.573 INFO:teuthology.orchestra.run.smithi143.stdout: "mon": "allow profile osd", 2023-11-26T07:22:33.573 INFO:teuthology.orchestra.run.smithi143.stdout: "osd": "allow *" 2023-11-26T07:22:33.574 INFO:teuthology.orchestra.run.smithi143.stdout: } 2023-11-26T07:22:33.574 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2023-11-26T07:22:33.574 INFO:teuthology.orchestra.run.smithi143.stdout: { 2023-11-26T07:22:33.574 INFO:teuthology.orchestra.run.smithi143.stdout: "entity": "osd.7", 2023-11-26T07:22:33.574 INFO:teuthology.orchestra.run.smithi143.stdout: "key": "AQAC8WJlJavtIBAAKMvnR4AnnrDVOXlzN7j05w==", 2023-11-26T07:22:33.574 INFO:teuthology.orchestra.run.smithi143.stdout: "caps": { 2023-11-26T07:22:33.574 INFO:teuthology.orchestra.run.smithi143.stdout: "mgr": "allow profile osd", 2023-11-26T07:22:33.574 INFO:teuthology.orchestra.run.smithi143.stdout: "mon": "allow profile osd", 2023-11-26T07:22:33.574 INFO:teuthology.orchestra.run.smithi143.stdout: "osd": "allow *" 2023-11-26T07:22:33.574 INFO:teuthology.orchestra.run.smithi143.stdout: } 2023-11-26T07:22:33.574 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2023-11-26T07:22:33.575 INFO:teuthology.orchestra.run.smithi143.stdout: { 2023-11-26T07:22:33.575 INFO:teuthology.orchestra.run.smithi143.stdout: "entity": "client.0", 2023-11-26T07:22:33.575 INFO:teuthology.orchestra.run.smithi143.stdout: "key": "AQDm8GJlWMh9FBAAslVzi6fm4/wt7vr/rHCHHQ==", 2023-11-26T07:22:33.575 INFO:teuthology.orchestra.run.smithi143.stdout: "caps": { 2023-11-26T07:22:33.575 INFO:teuthology.orchestra.run.smithi143.stdout: "mds": "allow", 2023-11-26T07:22:33.575 INFO:teuthology.orchestra.run.smithi143.stdout: "mon": "allow r", 2023-11-26T07:22:33.575 INFO:teuthology.orchestra.run.smithi143.stdout: "osd": "allow rw tag cephfs data=cephfs" 2023-11-26T07:22:33.575 INFO:teuthology.orchestra.run.smithi143.stdout: } 2023-11-26T07:22:33.575 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2023-11-26T07:22:33.575 INFO:teuthology.orchestra.run.smithi143.stdout: { 2023-11-26T07:22:33.575 INFO:teuthology.orchestra.run.smithi143.stdout: "entity": "client.1", 2023-11-26T07:22:33.576 INFO:teuthology.orchestra.run.smithi143.stdout: "key": "AQDm8GJl5vkYHhAAgP+gKH2+lpl3aTlgqZNIXg==", 2023-11-26T07:22:33.576 INFO:teuthology.orchestra.run.smithi143.stdout: "caps": { 2023-11-26T07:22:33.576 INFO:teuthology.orchestra.run.smithi143.stdout: "mds": "allow", 2023-11-26T07:22:33.576 INFO:teuthology.orchestra.run.smithi143.stdout: "mgr": "allow r", 2023-11-26T07:22:33.576 INFO:teuthology.orchestra.run.smithi143.stdout: "mon": "allow rw", 2023-11-26T07:22:33.576 INFO:teuthology.orchestra.run.smithi143.stdout: "osd": "allow rwx" 2023-11-26T07:22:33.576 INFO:teuthology.orchestra.run.smithi143.stdout: } 2023-11-26T07:22:33.576 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2023-11-26T07:22:33.576 INFO:teuthology.orchestra.run.smithi143.stdout: { 2023-11-26T07:22:33.576 INFO:teuthology.orchestra.run.smithi143.stdout: "entity": "client.2", 2023-11-26T07:22:33.577 INFO:teuthology.orchestra.run.smithi143.stdout: "key": "AQDm8GJlW4KlGhAANFKsyVJems0ZoE6bo3WmfQ==", 2023-11-26T07:22:33.577 INFO:teuthology.orchestra.run.smithi143.stdout: "caps": { 2023-11-26T07:22:33.577 INFO:teuthology.orchestra.run.smithi143.stdout: "mds": "allow", 2023-11-26T07:22:33.577 INFO:teuthology.orchestra.run.smithi143.stdout: "mgr": "allow r", 2023-11-26T07:22:33.577 INFO:teuthology.orchestra.run.smithi143.stdout: "mon": "allow rw", 2023-11-26T07:22:33.577 INFO:teuthology.orchestra.run.smithi143.stdout: "osd": "allow rwx" 2023-11-26T07:22:33.577 INFO:teuthology.orchestra.run.smithi143.stdout: } 2023-11-26T07:22:33.577 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2023-11-26T07:22:33.577 INFO:teuthology.orchestra.run.smithi143.stdout: { 2023-11-26T07:22:33.577 INFO:teuthology.orchestra.run.smithi143.stdout: "entity": "client.3", 2023-11-26T07:22:33.578 INFO:teuthology.orchestra.run.smithi143.stdout: "key": "AQDm8GJlAJZSJBAAUSUNcwRBY01hOuilBJjlFQ==", 2023-11-26T07:22:33.578 INFO:teuthology.orchestra.run.smithi143.stdout: "caps": { 2023-11-26T07:22:33.578 INFO:teuthology.orchestra.run.smithi143.stdout: "mds": "allow", 2023-11-26T07:22:33.578 INFO:teuthology.orchestra.run.smithi143.stdout: "mgr": "allow r", 2023-11-26T07:22:33.578 INFO:teuthology.orchestra.run.smithi143.stdout: "mon": "allow rw", 2023-11-26T07:22:33.578 INFO:teuthology.orchestra.run.smithi143.stdout: "osd": "allow rwx" 2023-11-26T07:22:33.578 INFO:teuthology.orchestra.run.smithi143.stdout: } 2023-11-26T07:22:33.578 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2023-11-26T07:22:33.578 INFO:teuthology.orchestra.run.smithi143.stdout: { 2023-11-26T07:22:33.578 INFO:teuthology.orchestra.run.smithi143.stdout: "entity": "client.admin", 2023-11-26T07:22:33.579 INFO:teuthology.orchestra.run.smithi143.stdout: "key": "AQDl8GJlbQv7GxAA0Midh8tAO2KO+dH10OV8Xg==", 2023-11-26T07:22:33.579 INFO:teuthology.orchestra.run.smithi143.stdout: "caps": { 2023-11-26T07:22:33.579 INFO:teuthology.orchestra.run.smithi143.stdout: "mds": "allow *", 2023-11-26T07:22:33.579 INFO:teuthology.orchestra.run.smithi143.stdout: "mgr": "allow *", 2023-11-26T07:22:33.579 INFO:teuthology.orchestra.run.smithi143.stdout: "mon": "allow *", 2023-11-26T07:22:33.579 INFO:teuthology.orchestra.run.smithi143.stdout: "osd": "allow *" 2023-11-26T07:22:33.579 INFO:teuthology.orchestra.run.smithi143.stdout: } 2023-11-26T07:22:33.579 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2023-11-26T07:22:33.579 INFO:teuthology.orchestra.run.smithi143.stdout: { 2023-11-26T07:22:33.579 INFO:teuthology.orchestra.run.smithi143.stdout: "entity": "client.alice", 2023-11-26T07:22:33.579 INFO:teuthology.orchestra.run.smithi143.stdout: "key": "AQAn8mJlNQDyLhAAMdZWyzLXsQAbpvtzb5EN9w==", 2023-11-26T07:22:33.580 INFO:teuthology.orchestra.run.smithi143.stdout: "caps": { 2023-11-26T07:22:33.580 INFO:teuthology.orchestra.run.smithi143.stdout: "mds": "allow rw path=/volumes/_nogroup/subvol970", 2023-11-26T07:22:33.580 INFO:teuthology.orchestra.run.smithi143.stdout: "mon": "allow r", 2023-11-26T07:22:33.580 INFO:teuthology.orchestra.run.smithi143.stdout: "osd": "allow rw pool=cephfs_data" 2023-11-26T07:22:33.580 INFO:teuthology.orchestra.run.smithi143.stdout: } 2023-11-26T07:22:33.580 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2023-11-26T07:22:33.580 INFO:teuthology.orchestra.run.smithi143.stdout: { 2023-11-26T07:22:33.580 INFO:teuthology.orchestra.run.smithi143.stdout: "entity": "mgr.x", 2023-11-26T07:22:33.580 INFO:teuthology.orchestra.run.smithi143.stdout: "key": "AQDl8GJlXy8cNBAA4f5RDC1MASCqZPcJQvnUDA==", 2023-11-26T07:22:33.580 INFO:teuthology.orchestra.run.smithi143.stdout: "caps": { 2023-11-26T07:22:33.581 INFO:teuthology.orchestra.run.smithi143.stdout: "mds": "allow *", 2023-11-26T07:22:33.581 INFO:teuthology.orchestra.run.smithi143.stdout: "mon": "allow profile mgr", 2023-11-26T07:22:33.581 INFO:teuthology.orchestra.run.smithi143.stdout: "osd": "allow *" 2023-11-26T07:22:33.581 INFO:teuthology.orchestra.run.smithi143.stdout: } 2023-11-26T07:22:33.581 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2023-11-26T07:22:33.581 INFO:teuthology.orchestra.run.smithi143.stdout: { 2023-11-26T07:22:33.581 INFO:teuthology.orchestra.run.smithi143.stdout: "entity": "mgr.y", 2023-11-26T07:22:33.581 INFO:teuthology.orchestra.run.smithi143.stdout: "key": "AQDl8GJlfuFPMBAA3TYEFvdfk4DyGsBkbpv0hg==", 2023-11-26T07:22:33.581 INFO:teuthology.orchestra.run.smithi143.stdout: "caps": { 2023-11-26T07:22:33.581 INFO:teuthology.orchestra.run.smithi143.stdout: "mds": "allow *", 2023-11-26T07:22:33.582 INFO:teuthology.orchestra.run.smithi143.stdout: "mon": "allow profile mgr", 2023-11-26T07:22:33.582 INFO:teuthology.orchestra.run.smithi143.stdout: "osd": "allow *" 2023-11-26T07:22:33.582 INFO:teuthology.orchestra.run.smithi143.stdout: } 2023-11-26T07:22:33.582 INFO:teuthology.orchestra.run.smithi143.stdout: } 2023-11-26T07:22:33.582 INFO:teuthology.orchestra.run.smithi143.stdout: ] 2023-11-26T07:22:33.582 INFO:teuthology.orchestra.run.smithi143.stdout:} 2023-11-26T07:22:33.584 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:33.564+0000 7f8b58e9c640 1 -- 172.21.15.143:0/3185619854 >> [v2:172.21.15.143:6832/2150557544,v1:172.21.15.143:6833/2150557544] conn(0x7f8b400667d0 msgr2=0x7f8b40068c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:22:33.584 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:33.564+0000 7f8b58e9c640 1 --2- 172.21.15.143:0/3185619854 >> [v2:172.21.15.143:6832/2150557544,v1:172.21.15.143:6833/2150557544] conn(0x7f8b400667d0 0x7f8b40068c90 secure :-1 s=READY pgs=438 cs=0 l=1 rev1=1 crypto rx=0x7f8b5414db20 tx=0x7f8b48009040 comp rx=0 tx=0).stop 2023-11-26T07:22:33.584 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:33.564+0000 7f8b58e9c640 1 -- 172.21.15.143:0/3185619854 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f8b3c003b40 msgr2=0x7f8b5414db70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:22:33.584 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:33.564+0000 7f8b58e9c640 1 --2- 172.21.15.143:0/3185619854 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f8b3c003b40 0x7f8b5414db70 secure :-1 s=READY pgs=933 cs=0 l=1 rev1=1 crypto rx=0x7f8b440b0040 tx=0x7f8b3c005800 comp rx=0 tx=0).stop 2023-11-26T07:22:33.584 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:33.564+0000 7f8b58e9c640 1 -- 172.21.15.143:0/3185619854 shutdown_connections 2023-11-26T07:22:33.584 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:33.564+0000 7f8b58e9c640 1 --2- 172.21.15.143:0/3185619854 >> [v2:172.21.15.143:6832/2150557544,v1:172.21.15.143:6833/2150557544] conn(0x7f8b400667d0 0x7f8b40068c90 unknown :-1 s=CLOSED pgs=438 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:22:33.584 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:33.564+0000 7f8b58e9c640 1 --2- 172.21.15.143:0/3185619854 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f8b5406f4f0 0x7f8b5406f9a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:22:33.584 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:33.564+0000 7f8b58e9c640 1 --2- 172.21.15.143:0/3185619854 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f8b5414d620 0x7f8b5414aa10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:22:33.584 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:33.564+0000 7f8b58e9c640 1 --2- 172.21.15.143:0/3185619854 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f8b3c003b40 0x7f8b5414db70 unknown :-1 s=CLOSED pgs=933 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:22:33.585 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:33.564+0000 7f8b58e9c640 1 -- 172.21.15.143:0/3185619854 >> 172.21.15.143:0/3185619854 conn(0x7f8b54099a70 msgr2=0x7f8b54157e00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:22:33.585 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:33.564+0000 7f8b58e9c640 1 -- 172.21.15.143:0/3185619854 shutdown_connections 2023-11-26T07:22:33.585 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:33.564+0000 7f8b58e9c640 1 -- 172.21.15.143:0/3185619854 wait complete. 2023-11-26T07:22:33.585 DEBUG:teuthology.orchestra.run.smithi143:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth del client.alice 2023-11-26T07:22:33.726 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:33.724+0000 7f38b9380640 1 Processor -- start 2023-11-26T07:22:33.726 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:33.724+0000 7f38b9380640 1 -- start start 2023-11-26T07:22:33.727 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:33.724+0000 7f38b9380640 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7f38b415d580 0x7f38b40d3920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:22:33.727 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:33.724+0000 7f38b9380640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f38b40d3e60 0x7f38b40d17f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:22:33.727 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:33.724+0000 7f38b9380640 1 -- --> v1:172.21.15.188:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f38b40d21e0 con 0x7f38b40d1d30 2023-11-26T07:22:33.727 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:33.724+0000 7f38b9380640 1 -- --> v2:172.21.15.188:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f38b40d0bc0 con 0x7f38b415d580 2023-11-26T07:22:33.727 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:33.724+0000 7f38b9380640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f38b40d0320 con 0x7f38b40d3e60 2023-11-26T07:22:33.728 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:33.728+0000 7f38b27fc640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f38b40d3e60 0x7f38b40d17f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:22:33.728 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:33.728+0000 7f38b2ffd640 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7f38b415d580 0x7f38b40d3920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:22:33.729 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:33.728+0000 7f38b27fc640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f38b40d3e60 0x7f38b40d17f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.188:3301/0 says I am v2:172.21.15.143:34596/0 (socket says 172.21.15.143:34596) 2023-11-26T07:22:33.729 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:33.728+0000 7f38b27fc640 1 -- 172.21.15.143:0/3373810799 learned_addr learned my addr 172.21.15.143:0/3373810799 (peer_addr_for_me v2:172.21.15.143:0/0) 2023-11-26T07:22:33.729 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:33.728+0000 7f38b27fc640 1 -- 172.21.15.143:0/3373810799 >> v2:172.21.15.188:3300/0 conn(0x7f38b415d580 msgr2=0x7f38b40d3920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-26T07:22:33.729 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:33.728+0000 7f38b27fc640 1 --2- 172.21.15.143:0/3373810799 >> v2:172.21.15.188:3300/0 conn(0x7f38b415d580 0x7f38b40d3920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:22:33.729 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:33.728+0000 7f38b27fc640 1 -- 172.21.15.143:0/3373810799 >> v1:172.21.15.188:6789/0 conn(0x7f38b40d1d30 legacy=0x7f38b40cdce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:22:33.729 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:33.728+0000 7f38b27fc640 1 -- 172.21.15.143:0/3373810799 --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f38b40d0600 con 0x7f38b40d3e60 2023-11-26T07:22:33.729 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:33.728+0000 7f38b2ffd640 1 --2- 172.21.15.143:0/3373810799 >> v2:172.21.15.188:3300/0 conn(0x7f38b415d580 0x7f38b40d3920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-26T07:22:33.731 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:33.728+0000 7f38b27fc640 1 --2- 172.21.15.143:0/3373810799 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f38b40d3e60 0x7f38b40d17f0 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7f38a8009870 tx=0x7f38a801c510 comp rx=0 tx=0).ready entity=mon.2 client_cookie=295531a38b07b175 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:22:33.731 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:33.728+0000 7f38b1ffb640 1 -- 172.21.15.143:0/3373810799 <== mon.2 v2:172.21.15.188:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f38a80aa670 con 0x7f38b40d3e60 2023-11-26T07:22:33.731 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:33.728+0000 7f38b1ffb640 1 -- 172.21.15.143:0/3373810799 <== mon.2 v2:172.21.15.188:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f38a80aa7d0 con 0x7f38b40d3e60 2023-11-26T07:22:33.732 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:33.728+0000 7f38b1ffb640 1 -- 172.21.15.143:0/3373810799 <== mon.2 v2:172.21.15.188:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f38a80aab60 con 0x7f38b40d3e60 2023-11-26T07:22:33.732 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:33.732+0000 7f38b9380640 1 -- 172.21.15.143:0/3373810799 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f38b40d3e60 msgr2=0x7f38b40d17f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:22:33.732 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:33.732+0000 7f38b9380640 1 --2- 172.21.15.143:0/3373810799 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f38b40d3e60 0x7f38b40d17f0 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7f38a8009870 tx=0x7f38a801c510 comp rx=0 tx=0).stop 2023-11-26T07:22:33.733 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:33.732+0000 7f38b9380640 1 -- 172.21.15.143:0/3373810799 shutdown_connections 2023-11-26T07:22:33.733 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:33.732+0000 7f38b9380640 1 --2- 172.21.15.143:0/3373810799 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f38b40d3e60 0x7f38b40d17f0 unknown :-1 s=CLOSED pgs=120 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:22:33.733 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:33.732+0000 7f38b9380640 1 --2- 172.21.15.143:0/3373810799 >> v2:172.21.15.188:3300/0 conn(0x7f38b415d580 0x7f38b40d3920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:22:33.733 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:33.732+0000 7f38b9380640 1 -- 172.21.15.143:0/3373810799 >> 172.21.15.143:0/3373810799 conn(0x7f38b409c550 msgr2=0x7f38b40d3210 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:22:33.733 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:33.732+0000 7f38b9380640 1 -- 172.21.15.143:0/3373810799 shutdown_connections 2023-11-26T07:22:33.733 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:33.732+0000 7f38b9380640 1 -- 172.21.15.143:0/3373810799 wait complete. 2023-11-26T07:22:33.734 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:33.732+0000 7f38b9380640 1 Processor -- start 2023-11-26T07:22:33.734 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:33.732+0000 7f38b9380640 1 -- start start 2023-11-26T07:22:33.735 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:33.732+0000 7f38b9380640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f38b40d1d30 0x7f38b40c9940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:22:33.735 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:33.732+0000 7f38b9380640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f38b40d3e60 0x7f38b4160440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:22:33.736 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:33.732+0000 7f38b9380640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f38b415d580 0x7f38b4158b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:22:33.736 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:33.732+0000 7f38b9380640 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f38b41586a0 con 0x7f38b40d1d30 2023-11-26T07:22:33.736 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:33.732+0000 7f38b9380640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f38b40c9e80 con 0x7f38b415d580 2023-11-26T07:22:33.736 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:33.732+0000 7f38b9380640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f38b4157a90 con 0x7f38b40d3e60 2023-11-26T07:22:33.736 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:33.732+0000 7f38b2ffd640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f38b40d1d30 0x7f38b40c9940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:22:33.736 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:33.732+0000 7f38b2ffd640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f38b40d1d30 0x7f38b40c9940 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:47664/0 (socket says 172.21.15.143:47664) 2023-11-26T07:22:33.736 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:33.732+0000 7f38b2ffd640 1 -- 172.21.15.143:0/3299955490 learned_addr learned my addr 172.21.15.143:0/3299955490 (peer_addr_for_me v2:172.21.15.143:0/0) 2023-11-26T07:22:33.736 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:33.736+0000 7f38b27fc640 1 --2- 172.21.15.143:0/3299955490 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f38b40d3e60 0x7f38b4160440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:22:33.736 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:33.736+0000 7f38b37fe640 1 --2- 172.21.15.143:0/3299955490 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f38b415d580 0x7f38b4158b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:22:33.737 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:33.736+0000 7f38b2ffd640 1 -- 172.21.15.143:0/3299955490 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f38b40d3e60 msgr2=0x7f38b4160440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:22:33.738 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:33.736+0000 7f38b2ffd640 1 --2- 172.21.15.143:0/3299955490 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f38b40d3e60 0x7f38b4160440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:22:33.738 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:33.736+0000 7f38b2ffd640 1 -- 172.21.15.143:0/3299955490 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f38b415d580 msgr2=0x7f38b4158b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:22:33.738 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:33.736+0000 7f38b2ffd640 1 --2- 172.21.15.143:0/3299955490 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f38b415d580 0x7f38b4158b30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:22:33.738 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:33.736+0000 7f38b2ffd640 1 -- 172.21.15.143:0/3299955490 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f38a4009420 con 0x7f38b40d1d30 2023-11-26T07:22:33.738 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:33.736+0000 7f38b37fe640 1 --2- 172.21.15.143:0/3299955490 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f38b415d580 0x7f38b4158b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-26T07:22:33.738 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:33.736+0000 7f38b2ffd640 1 --2- 172.21.15.143:0/3299955490 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f38b40d1d30 0x7f38b40c9940 secure :-1 s=READY pgs=934 cs=0 l=1 rev1=1 crypto rx=0x7f38a400e9b0 tx=0x7f38a400ee70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:22:33.738 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:33.736+0000 7f389b7fe640 1 -- 172.21.15.143:0/3299955490 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f38a400ca00 con 0x7f38b40d1d30 2023-11-26T07:22:33.738 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:33.736+0000 7f389b7fe640 1 -- 172.21.15.143:0/3299955490 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f38a400cb60 con 0x7f38b40d1d30 2023-11-26T07:22:33.738 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:33.736+0000 7f389b7fe640 1 -- 172.21.15.143:0/3299955490 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f38a4013470 con 0x7f38b40d1d30 2023-11-26T07:22:33.738 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:33.736+0000 7f38b9380640 1 -- 172.21.15.143:0/3299955490 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f38a80094d0 con 0x7f38b40d1d30 2023-11-26T07:22:33.738 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:33.736+0000 7f38b9380640 1 -- 172.21.15.143:0/3299955490 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f38b4158050 con 0x7f38b40d1d30 2023-11-26T07:22:33.739 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:33.736+0000 7f38b9380640 1 -- 172.21.15.143:0/3299955490 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f38b4067680 con 0x7f38b40d1d30 2023-11-26T07:22:33.739 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:33.736+0000 7f389b7fe640 1 -- 172.21.15.143:0/3299955490 <== mon.0 v2:172.21.15.143:3300/0 4 ==== mgrmap(e 11) v1 ==== 86158+0+0 (secure 0 0 0) 0x7f38a40135d0 con 0x7f38b40d1d30 2023-11-26T07:22:33.740 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:33.740+0000 7f389b7fe640 1 --2- 172.21.15.143:0/3299955490 >> [v2:172.21.15.143:6832/2150557544,v1:172.21.15.143:6833/2150557544] conn(0x7f38a00667d0 0x7f38a0068c90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:22:33.740 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:33.740+0000 7f389b7fe640 1 -- 172.21.15.143:0/3299955490 <== mon.0 v2:172.21.15.143:3300/0 5 ==== osd_map(67..67 src has 1..67) v4 ==== 4857+0+0 (secure 0 0 0) 0x7f38a4084670 con 0x7f38b40d1d30 2023-11-26T07:22:33.740 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:33.740+0000 7f38b27fc640 1 --2- 172.21.15.143:0/3299955490 >> [v2:172.21.15.143:6832/2150557544,v1:172.21.15.143:6833/2150557544] conn(0x7f38a00667d0 0x7f38a0068c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:22:33.741 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:33.740+0000 7f38b27fc640 1 --2- 172.21.15.143:0/3299955490 >> [v2:172.21.15.143:6832/2150557544,v1:172.21.15.143:6833/2150557544] conn(0x7f38a00667d0 0x7f38a0068c90 secure :-1 s=READY pgs=439 cs=0 l=1 rev1=1 crypto rx=0x7f38a80ae000 tx=0x7f38a80b5040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:22:33.744 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:33.744+0000 7f389b7fe640 1 -- 172.21.15.143:0/3299955490 <== mon.0 v2:172.21.15.143:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172649 (secure 0 0 0) 0x7f38a4051a50 con 0x7f38b40d1d30 2023-11-26T07:22:33.951 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:33.948+0000 7f38b9380640 1 -- 172.21.15.143:0/3299955490 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "auth del", "entity": "client.alice"} v 0) v1 -- 0x7f38b40b33d0 con 0x7f38b40d1d30 2023-11-26T07:22:33.954 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:33.952+0000 7f389b7fe640 1 -- 172.21.15.143:0/3299955490 <== mon.0 v2:172.21.15.143:3300/0 7 ==== mon_command_ack([{"prefix": "auth del", "entity": "client.alice"}]=0 v17) v1 ==== 82+0+0 (secure 0 0 0) 0x7f38a4055070 con 0x7f38b40d1d30 2023-11-26T07:22:33.956 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:33.956+0000 7f38b9380640 1 -- 172.21.15.143:0/3299955490 >> [v2:172.21.15.143:6832/2150557544,v1:172.21.15.143:6833/2150557544] conn(0x7f38a00667d0 msgr2=0x7f38a0068c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:22:33.956 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:33.956+0000 7f38b9380640 1 --2- 172.21.15.143:0/3299955490 >> [v2:172.21.15.143:6832/2150557544,v1:172.21.15.143:6833/2150557544] conn(0x7f38a00667d0 0x7f38a0068c90 secure :-1 s=READY pgs=439 cs=0 l=1 rev1=1 crypto rx=0x7f38a80ae000 tx=0x7f38a80b5040 comp rx=0 tx=0).stop 2023-11-26T07:22:33.956 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:33.956+0000 7f38b9380640 1 -- 172.21.15.143:0/3299955490 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f38b40d1d30 msgr2=0x7f38b40c9940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:22:33.956 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:33.956+0000 7f38b9380640 1 --2- 172.21.15.143:0/3299955490 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f38b40d1d30 0x7f38b40c9940 secure :-1 s=READY pgs=934 cs=0 l=1 rev1=1 crypto rx=0x7f38a400e9b0 tx=0x7f38a400ee70 comp rx=0 tx=0).stop 2023-11-26T07:22:33.956 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:33.956+0000 7f38b9380640 1 -- 172.21.15.143:0/3299955490 shutdown_connections 2023-11-26T07:22:33.957 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:33.956+0000 7f38b9380640 1 --2- 172.21.15.143:0/3299955490 >> [v2:172.21.15.143:6832/2150557544,v1:172.21.15.143:6833/2150557544] conn(0x7f38a00667d0 0x7f38a0068c90 unknown :-1 s=CLOSED pgs=439 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:22:33.957 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:33.956+0000 7f38b9380640 1 --2- 172.21.15.143:0/3299955490 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f38b415d580 0x7f38b4158b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:22:33.957 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:33.956+0000 7f38b9380640 1 --2- 172.21.15.143:0/3299955490 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f38b40d3e60 0x7f38b4160440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:22:33.957 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:33.956+0000 7f38b9380640 1 --2- 172.21.15.143:0/3299955490 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f38b40d1d30 0x7f38b40c9940 unknown :-1 s=CLOSED pgs=934 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:22:33.957 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:33.956+0000 7f38b9380640 1 -- 172.21.15.143:0/3299955490 >> 172.21.15.143:0/3299955490 conn(0x7f38b409c550 msgr2=0x7f38b41564f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:22:33.957 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:33.956+0000 7f38b9380640 1 -- 172.21.15.143:0/3299955490 shutdown_connections 2023-11-26T07:22:33.957 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:33.956+0000 7f38b9380640 1 -- 172.21.15.143:0/3299955490 wait complete. 2023-11-26T07:22:33.975 DEBUG:teuthology.orchestra.run.smithi143:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-11-26T07:22:34.112 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:34.112+0000 7ffb5e101640 1 Processor -- start 2023-11-26T07:22:34.113 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:34.112+0000 7ffb5e101640 1 -- start start 2023-11-26T07:22:34.113 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:34.112+0000 7ffb5e101640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7ffb580b2160 0x7ffb580d0df0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:22:34.114 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:34.112+0000 7ffb5e101640 1 -- --> v1:172.21.15.143:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ffb580cf0e0 con 0x7ffb580d1bc0 2023-11-26T07:22:34.114 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:34.112+0000 7ffb5e101640 1 -- --> v1:172.21.15.188:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ffb580ceb20 con 0x7ffb5815d6e0 2023-11-26T07:22:34.114 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:34.112+0000 7ffb5e101640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ffb5815dac0 con 0x7ffb580b2160 2023-11-26T07:22:34.114 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:34.112+0000 7ffb56ffd640 1 --1- >> v1:172.21.15.143:6789/0 conn(0x7ffb580d1bc0 0x7ffb580b0750 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.143:6789/0 says I am v1:172.21.15.143:60320/0 (socket says 172.21.15.143:60320) 2023-11-26T07:22:34.114 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:34.112+0000 7ffb56ffd640 1 -- 172.21.15.143:0/856447311 learned_addr learned my addr 172.21.15.143:0/856447311 (peer_addr_for_me v1:172.21.15.143:0/0) 2023-11-26T07:22:34.115 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:34.112+0000 7ffb57fff640 1 --2- 172.21.15.143:0/856447311 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7ffb580b2160 0x7ffb580d0df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:22:34.115 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:34.112+0000 7ffb567fc640 1 -- 172.21.15.143:0/856447311 <== mon.0 v1:172.21.15.143:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 503841636 0 0) 0x7ffb580cf0e0 con 0x7ffb580d1bc0 2023-11-26T07:22:34.115 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:34.112+0000 7ffb567fc640 1 -- 172.21.15.143:0/856447311 --> v1:172.21.15.143:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7ffb40003530 con 0x7ffb580d1bc0 2023-11-26T07:22:34.115 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:34.112+0000 7ffb567fc640 1 -- 172.21.15.143:0/856447311 <== mon.1 v1:172.21.15.188:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3219456579 0 0) 0x7ffb580ceb20 con 0x7ffb5815d6e0 2023-11-26T07:22:34.115 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:34.112+0000 7ffb567fc640 1 -- 172.21.15.143:0/856447311 --> v1:172.21.15.188:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7ffb580cf0e0 con 0x7ffb5815d6e0 2023-11-26T07:22:34.115 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:34.112+0000 7ffb57fff640 1 -- 172.21.15.143:0/856447311 >> v1:172.21.15.188:6789/0 conn(0x7ffb5815d6e0 legacy=0x7ffb580d14b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:22:34.116 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:34.112+0000 7ffb57fff640 1 -- 172.21.15.143:0/856447311 >> v1:172.21.15.143:6789/0 conn(0x7ffb580d1bc0 legacy=0x7ffb580b0750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:22:34.116 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:34.112+0000 7ffb57fff640 1 -- 172.21.15.143:0/856447311 --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ffb580cee00 con 0x7ffb580b2160 2023-11-26T07:22:34.117 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:34.116+0000 7ffb57fff640 1 --2- 172.21.15.143:0/856447311 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7ffb580b2160 0x7ffb580d0df0 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7ffb50009dc0 tx=0x7ffb5001c970 comp rx=0 tx=0).ready entity=mon.2 client_cookie=6f5bb0659bc6a84f server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:22:34.118 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:34.116+0000 7ffb567fc640 1 -- 172.21.15.143:0/856447311 <== mon.2 v2:172.21.15.188:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffb500a5920 con 0x7ffb580b2160 2023-11-26T07:22:34.118 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:34.116+0000 7ffb567fc640 1 -- 172.21.15.143:0/856447311 <== mon.2 v2:172.21.15.188:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ffb500a5a80 con 0x7ffb580b2160 2023-11-26T07:22:34.118 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:34.116+0000 7ffb567fc640 1 -- 172.21.15.143:0/856447311 <== mon.2 v2:172.21.15.188:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffb500a5e10 con 0x7ffb580b2160 2023-11-26T07:22:34.118 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:34.116+0000 7ffb5e101640 1 -- 172.21.15.143:0/856447311 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7ffb580b2160 msgr2=0x7ffb580d0df0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:22:34.119 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:34.116+0000 7ffb5e101640 1 --2- 172.21.15.143:0/856447311 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7ffb580b2160 0x7ffb580d0df0 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7ffb50009dc0 tx=0x7ffb5001c970 comp rx=0 tx=0).stop 2023-11-26T07:22:34.119 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:34.116+0000 7ffb5e101640 1 -- 172.21.15.143:0/856447311 shutdown_connections 2023-11-26T07:22:34.119 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:34.116+0000 7ffb5e101640 1 --2- 172.21.15.143:0/856447311 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7ffb580b2160 0x7ffb580d0df0 unknown :-1 s=CLOSED pgs=121 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:22:34.119 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:34.116+0000 7ffb5e101640 1 -- 172.21.15.143:0/856447311 >> 172.21.15.143:0/856447311 conn(0x7ffb5809c550 msgr2=0x7ffb580d3220 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:22:34.119 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:34.116+0000 7ffb5e101640 1 -- 172.21.15.143:0/856447311 shutdown_connections 2023-11-26T07:22:34.119 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:34.116+0000 7ffb5e101640 1 -- 172.21.15.143:0/856447311 wait complete. 2023-11-26T07:22:34.120 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:34.120+0000 7ffb5e101640 1 Processor -- start 2023-11-26T07:22:34.120 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:34.120+0000 7ffb5e101640 1 -- start start 2023-11-26T07:22:34.122 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:34.120+0000 7ffb5e101640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7ffb580b2160 0x7ffb581546a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:22:34.123 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:34.120+0000 7ffb5e101640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7ffb580d1bc0 0x7ffb580b5e40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:22:34.123 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:34.120+0000 7ffb5e101640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7ffb5815d6e0 0x7ffb580b6380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:22:34.123 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:34.120+0000 7ffb5e101640 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffb581585e0 con 0x7ffb580b2160 2023-11-26T07:22:34.123 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:34.120+0000 7ffb5e101640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffb58154be0 con 0x7ffb5815d6e0 2023-11-26T07:22:34.123 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:34.120+0000 7ffb5e101640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ffb5815e0f0 con 0x7ffb580d1bc0 2023-11-26T07:22:34.123 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:34.120+0000 7ffb577fe640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7ffb580b2160 0x7ffb581546a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:22:34.123 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:34.120+0000 7ffb577fe640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7ffb580b2160 0x7ffb581546a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:47668/0 (socket says 172.21.15.143:47668) 2023-11-26T07:22:34.123 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:34.120+0000 7ffb577fe640 1 -- 172.21.15.143:0/2227251921 learned_addr learned my addr 172.21.15.143:0/2227251921 (peer_addr_for_me v2:172.21.15.143:0/0) 2023-11-26T07:22:34.123 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:34.120+0000 7ffb57fff640 1 --2- 172.21.15.143:0/2227251921 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7ffb5815d6e0 0x7ffb580b6380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:22:34.123 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:34.120+0000 7ffb56ffd640 1 --2- 172.21.15.143:0/2227251921 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7ffb580d1bc0 0x7ffb580b5e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:22:34.123 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:34.120+0000 7ffb577fe640 1 -- 172.21.15.143:0/2227251921 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7ffb580d1bc0 msgr2=0x7ffb580b5e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:22:34.124 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:34.120+0000 7ffb577fe640 1 --2- 172.21.15.143:0/2227251921 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7ffb580d1bc0 0x7ffb580b5e40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:22:34.124 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:34.120+0000 7ffb577fe640 1 -- 172.21.15.143:0/2227251921 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7ffb5815d6e0 msgr2=0x7ffb580b6380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:22:34.124 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:34.120+0000 7ffb577fe640 1 --2- 172.21.15.143:0/2227251921 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7ffb5815d6e0 0x7ffb580b6380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:22:34.124 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:34.120+0000 7ffb577fe640 1 -- 172.21.15.143:0/2227251921 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ffb50009a70 con 0x7ffb580b2160 2023-11-26T07:22:34.124 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:34.120+0000 7ffb56ffd640 1 --2- 172.21.15.143:0/2227251921 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7ffb580d1bc0 0x7ffb580b5e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-26T07:22:34.124 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:34.120+0000 7ffb577fe640 1 --2- 172.21.15.143:0/2227251921 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7ffb580b2160 0x7ffb581546a0 secure :-1 s=READY pgs=936 cs=0 l=1 rev1=1 crypto rx=0x7ffb4800c490 tx=0x7ffb4800c950 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:22:34.124 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:34.120+0000 7ffb3ffff640 1 -- 172.21.15.143:0/2227251921 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffb480052d0 con 0x7ffb580b2160 2023-11-26T07:22:34.124 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:34.120+0000 7ffb3ffff640 1 -- 172.21.15.143:0/2227251921 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ffb48005430 con 0x7ffb580b2160 2023-11-26T07:22:34.124 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:34.120+0000 7ffb3ffff640 1 -- 172.21.15.143:0/2227251921 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffb48012870 con 0x7ffb580b2160 2023-11-26T07:22:34.124 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:34.120+0000 7ffb5e101640 1 -- 172.21.15.143:0/2227251921 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ffb5815e3d0 con 0x7ffb580b2160 2023-11-26T07:22:34.124 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:34.120+0000 7ffb5e101640 1 -- 172.21.15.143:0/2227251921 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ffb5815e990 con 0x7ffb580b2160 2023-11-26T07:22:34.125 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:34.124+0000 7ffb5e101640 1 -- 172.21.15.143:0/2227251921 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ffb580d2690 con 0x7ffb580b2160 2023-11-26T07:22:34.125 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:34.124+0000 7ffb3ffff640 1 -- 172.21.15.143:0/2227251921 <== mon.0 v2:172.21.15.143:3300/0 4 ==== mgrmap(e 11) v1 ==== 86158+0+0 (secure 0 0 0) 0x7ffb480129d0 con 0x7ffb580b2160 2023-11-26T07:22:34.125 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:34.124+0000 7ffb3ffff640 1 --2- 172.21.15.143:0/2227251921 >> [v2:172.21.15.143:6832/2150557544,v1:172.21.15.143:6833/2150557544] conn(0x7ffb440667d0 0x7ffb44068c90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:22:34.125 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:34.124+0000 7ffb3ffff640 1 -- 172.21.15.143:0/2227251921 <== mon.0 v2:172.21.15.143:3300/0 5 ==== osd_map(67..67 src has 1..67) v4 ==== 4857+0+0 (secure 0 0 0) 0x7ffb48083c10 con 0x7ffb580b2160 2023-11-26T07:22:34.125 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:34.124+0000 7ffb56ffd640 1 --2- 172.21.15.143:0/2227251921 >> [v2:172.21.15.143:6832/2150557544,v1:172.21.15.143:6833/2150557544] conn(0x7ffb440667d0 0x7ffb44068c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:22:34.126 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:34.124+0000 7ffb56ffd640 1 --2- 172.21.15.143:0/2227251921 >> [v2:172.21.15.143:6832/2150557544,v1:172.21.15.143:6833/2150557544] conn(0x7ffb440667d0 0x7ffb44068c90 secure :-1 s=READY pgs=440 cs=0 l=1 rev1=1 crypto rx=0x7ffb5815cdc0 tx=0x7ffb4c009040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:22:34.129 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:34.128+0000 7ffb3ffff640 1 -- 172.21.15.143:0/2227251921 <== mon.0 v2:172.21.15.143:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172649 (secure 0 0 0) 0x7ffb48050f20 con 0x7ffb580b2160 2023-11-26T07:22:34.278 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:34.276+0000 7ffb5e101640 1 -- 172.21.15.143:0/2227251921 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7ffb580b1c20 con 0x7ffb580b2160 2023-11-26T07:22:34.279 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:34.276+0000 7ffb3ffff640 1 -- 172.21.15.143:0/2227251921 <== mon.0 v2:172.21.15.143:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v67) v1 ==== 74+0+12893 (secure 0 0 0) 0x7ffb48054540 con 0x7ffb580b2160 2023-11-26T07:22:34.279 INFO:teuthology.orchestra.run.smithi143.stdout: 2023-11-26T07:22:34.279 INFO:teuthology.orchestra.run.smithi143.stdout:{"epoch":67,"fsid":"f5033060-2469-4a07-95fe-2b76f5404bc1","created":"2023-11-26T07:17:28.613663+0000","modified":"2023-11-26T07:22:33.174316+0000","last_up_change":"2023-11-26T07:17:41.157813+0000","last_in_change":"2023-11-26T07:17:31.181658+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":9,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-11-26T07:17:41.758597+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"ebbee2c6-023e-44b9-9f1d-6f7bfd309730","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":57,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6817","nonce":639807369},{"type":"v1","addr":"172.21.15.143:6819","nonce":639807369}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6821","nonce":639807369},{"type":"v1","addr":"172.21.15.143:6823","nonce":639807369}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6830","nonce":639807369},{"type":"v1","addr":"172.21.15.143:6831","nonce":639807369}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6825","nonce":639807369},{"type":"v1","addr":"172.21.15.143:6828","nonce":639807369}]},"public_addr":"172.21.15.143:6819/639807369","cluster_addr":"172.21.15.143:6823/639807369","heartbeat_back_addr":"172.21.15.143:6831/639807369","heartbeat_front_addr":"172.21.15.143:6828/639807369","state":["exists","up"]},{"osd":1,"uuid":"fec4b8be-7122-49e9-9a19-5850a0700001","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":57,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6816","nonce":1144942530},{"type":"v1","addr":"172.21.15.143:6818","nonce":1144942530}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6820","nonce":1144942530},{"type":"v1","addr":"172.21.15.143:6822","nonce":1144942530}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6827","nonce":1144942530},{"type":"v1","addr":"172.21.15.143:6829","nonce":1144942530}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6824","nonce":1144942530},{"type":"v1","addr":"172.21.15.143:6826","nonce":1144942530}]},"public_addr":"172.21.15.143:6818/1144942530","cluster_addr":"172.21.15.143:6822/1144942530","heartbeat_back_addr":"172.21.15.143:6829/1144942530","heartbeat_front_addr":"172.21.15.143:6826/1144942530","state":["exists","up"]},{"osd":2,"uuid":"2888af5b-f933-46ca-8f95-6dd494ff43f6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":57,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6808","nonce":3909095186},{"type":"v1","addr":"172.21.15.143:6809","nonce":3909095186}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6810","nonce":3909095186},{"type":"v1","addr":"172.21.15.143:6811","nonce":3909095186}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6814","nonce":3909095186},{"type":"v1","addr":"172.21.15.143:6815","nonce":3909095186}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6812","nonce":3909095186},{"type":"v1","addr":"172.21.15.143:6813","nonce":3909095186}]},"public_addr":"172.21.15.143:6809/3909095186","cluster_addr":"172.21.15.143:6811/3909095186","heartbeat_back_addr":"172.21.15.143:6815/3909095186","heartbeat_front_addr":"172.21.15.143:6813/3909095186","state":["exists","up"]},{"osd":3,"uuid":"d1f52435-7bc1-4081-9550-9cd1ba55f549","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":57,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6800","nonce":3854210991},{"type":"v1","addr":"172.21.15.143:6801","nonce":3854210991}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6802","nonce":3854210991},{"type":"v1","addr":"172.21.15.143:6803","nonce":3854210991}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6806","nonce":3854210991},{"type":"v1","addr":"172.21.15.143:6807","nonce":3854210991}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6804","nonce":3854210991},{"type":"v1","addr":"172.21.15.143:6805","nonce":3854210991}]},"public_addr":"172.21.15.143:6801/3854210991","cluster_addr":"172.21.15.143:6803/3854210991","heartbeat_back_addr":"172.21.15.143:6807/3854210991","heartbeat_front_addr":"172.21.15.143:6805/3854210991","state":["exists","up"]},{"osd":4,"uuid":"e2ae8b6e-6e25-49b2-bcbf-aacce68f06a3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":57,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6800","nonce":3924606902},{"type":"v1","addr":"172.21.15.188:6801","nonce":3924606902}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6802","nonce":3924606902},{"type":"v1","addr":"172.21.15.188:6803","nonce":3924606902}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6806","nonce":3924606902},{"type":"v1","addr":"172.21.15.188:6807","nonce":3924606902}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6804","nonce":3924606902},{"type":"v1","addr":"172.21.15.188:6805","nonce":3924606902}]},"public_addr":"172.21.15.188:6801/3924606902","cluster_addr":"172.21.15.188:6803/3924606902","heartbeat_back_addr":"172.21.15.188:6807/3924606902","heartbeat_front_addr":"172.21.15.188:6805/3924606902","state":["exists","up"]},{"osd":5,"uuid":"3cec8a32-7698-4c36-8e89-e4ba11934f33","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":57,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6813","nonce":186841544},{"type":"v1","addr":"172.21.15.188:6815","nonce":186841544}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6817","nonce":186841544},{"type":"v1","addr":"172.21.15.188:6818","nonce":186841544}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6822","nonce":186841544},{"type":"v1","addr":"172.21.15.188:6823","nonce":186841544}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6820","nonce":186841544},{"type":"v1","addr":"172.21.15.188:6821","nonce":186841544}]},"public_addr":"172.21.15.188:6815/186841544","cluster_addr":"172.21.15.188:6818/186841544","heartbeat_back_addr":"172.21.15.188:6823/186841544","heartbeat_front_addr":"172.21.15.188:6821/186841544","state":["exists","up"]},{"osd":6,"uuid":"58a65d5d-e090-473a-81b4-ad38e85029f9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":57,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6808","nonce":2153499405},{"type":"v1","addr":"172.21.15.188:6809","nonce":2153499405}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6810","nonce":2153499405},{"type":"v1","addr":"172.21.15.188:6811","nonce":2153499405}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6816","nonce":2153499405},{"type":"v1","addr":"172.21.15.188:6819","nonce":2153499405}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6812","nonce":2153499405},{"type":"v1","addr":"172.21.15.188:6814","nonce":2153499405}]},"public_addr":"172.21.15.188:6809/2153499405","cluster_addr":"172.21.15.188:6811/2153499405","heartbeat_back_addr":"172.21.15.188:6819/2153499405","heartbeat_front_addr":"172.21.15.188:6814/2153499405","state":["exists","up"]},{"osd":7,"uuid":"ea5b7f70-3554-4cf8-ac30-f7c59b80678d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":57,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6824","nonce":2722096729},{"type":"v1","addr":"172.21.15.188:6825","nonce":2722096729}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6826","nonce":2722096729},{"type":"v1","addr":"172.21.15.188:6827","nonce":2722096729}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6830","nonce":2722096729},{"type":"v1","addr":"172.21.15.188:6831","nonce":2722096729}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6828","nonce":2722096729},{"type":"v1","addr":"172.21.15.188:6829","nonce":2722096729}]},"public_addr":"172.21.15.188:6825/2722096729","cluster_addr":"172.21.15.188:6827/2722096729","heartbeat_back_addr":"172.21.15.188:6831/2722096729","heartbeat_front_addr":"172.21.15.188:6829/2722096729","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-26T07:17:36.995032+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-26T07:17:36.932266+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-26T07:17:36.829304+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-26T07:17:36.804297+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-26T07:17:36.667892+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-26T07:17:36.964868+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-26T07:17:36.827332+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-26T07:17:36.893179+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-11-26T07:22:34.281 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:34.280+0000 7ffb5e101640 1 -- 172.21.15.143:0/2227251921 >> [v2:172.21.15.143:6832/2150557544,v1:172.21.15.143:6833/2150557544] conn(0x7ffb440667d0 msgr2=0x7ffb44068c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:22:34.281 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:34.280+0000 7ffb5e101640 1 --2- 172.21.15.143:0/2227251921 >> [v2:172.21.15.143:6832/2150557544,v1:172.21.15.143:6833/2150557544] conn(0x7ffb440667d0 0x7ffb44068c90 secure :-1 s=READY pgs=440 cs=0 l=1 rev1=1 crypto rx=0x7ffb5815cdc0 tx=0x7ffb4c009040 comp rx=0 tx=0).stop 2023-11-26T07:22:34.282 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:34.280+0000 7ffb5e101640 1 -- 172.21.15.143:0/2227251921 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7ffb580b2160 msgr2=0x7ffb581546a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:22:34.282 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:34.280+0000 7ffb5e101640 1 --2- 172.21.15.143:0/2227251921 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7ffb580b2160 0x7ffb581546a0 secure :-1 s=READY pgs=936 cs=0 l=1 rev1=1 crypto rx=0x7ffb4800c490 tx=0x7ffb4800c950 comp rx=0 tx=0).stop 2023-11-26T07:22:34.282 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:34.280+0000 7ffb5e101640 1 -- 172.21.15.143:0/2227251921 shutdown_connections 2023-11-26T07:22:34.282 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:34.280+0000 7ffb5e101640 1 --2- 172.21.15.143:0/2227251921 >> [v2:172.21.15.143:6832/2150557544,v1:172.21.15.143:6833/2150557544] conn(0x7ffb440667d0 0x7ffb44068c90 unknown :-1 s=CLOSED pgs=440 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:22:34.282 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:34.280+0000 7ffb5e101640 1 --2- 172.21.15.143:0/2227251921 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7ffb5815d6e0 0x7ffb580b6380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:22:34.282 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:34.280+0000 7ffb5e101640 1 --2- 172.21.15.143:0/2227251921 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7ffb580d1bc0 0x7ffb580b5e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:22:34.282 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:34.280+0000 7ffb5e101640 1 --2- 172.21.15.143:0/2227251921 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7ffb580b2160 0x7ffb581546a0 unknown :-1 s=CLOSED pgs=936 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:22:34.282 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:34.280+0000 7ffb5e101640 1 -- 172.21.15.143:0/2227251921 >> 172.21.15.143:0/2227251921 conn(0x7ffb5809c550 msgr2=0x7ffb580734a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:22:34.282 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:34.280+0000 7ffb5e101640 1 -- 172.21.15.143:0/2227251921 shutdown_connections 2023-11-26T07:22:34.282 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:34.280+0000 7ffb5e101640 1 -- 172.21.15.143:0/2227251921 wait complete. 2023-11-26T07:22:34.301 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-11-26T07:17:41.758597+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2023-11-26T07:22:34.301 DEBUG:teuthology.orchestra.run.smithi143:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-11-26T07:22:34.441 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:34.440+0000 7f7477d8d640 1 Processor -- start 2023-11-26T07:22:34.441 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:34.440+0000 7f7477d8d640 1 -- start start 2023-11-26T07:22:34.442 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:34.440+0000 7f7477d8d640 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7f7470070ba0 0x7f74700aead0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:22:34.443 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:34.440+0000 7f7477d8d640 1 -- --> v1:172.21.15.143:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f74700b3690 con 0x7f747006a490 2023-11-26T07:22:34.443 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:34.440+0000 7f7477d8d640 1 -- --> v1:172.21.15.188:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f74700cf080 con 0x7f74700cdb60 2023-11-26T07:22:34.443 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:34.440+0000 7f7477d8d640 1 -- --> v2:172.21.15.143:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f74700b3ca0 con 0x7f7470070ba0 2023-11-26T07:22:34.443 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:34.440+0000 7f7475b29640 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7f7470070ba0 0x7f74700aead0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:22:34.443 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:34.440+0000 7f747632a640 1 --1- >> v1:172.21.15.143:6789/0 conn(0x7f747006a490 0x7f74700b40c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.143:6789/0 says I am v1:172.21.15.143:35722/0 (socket says 172.21.15.143:35722) 2023-11-26T07:22:34.443 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:34.440+0000 7f7475b29640 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7f7470070ba0 0x7f74700aead0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:47098/0 (socket says 172.21.15.143:47098) 2023-11-26T07:22:34.443 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:34.440+0000 7f747632a640 1 -- 172.21.15.143:0/1495153182 learned_addr learned my addr 172.21.15.143:0/1495153182 (peer_addr_for_me v1:172.21.15.143:0/0) 2023-11-26T07:22:34.443 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:34.440+0000 7f7475b29640 1 -- 172.21.15.143:0/1495153182 >> v1:172.21.15.188:6789/0 conn(0x7f74700cdb60 legacy=0x7f7470070f80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:22:34.443 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:34.440+0000 7f7475b29640 1 -- 172.21.15.143:0/1495153182 >> v1:172.21.15.143:6789/0 conn(0x7f747006a490 legacy=0x7f74700b40c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:22:34.443 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:34.440+0000 7f7475b29640 1 -- 172.21.15.143:0/1495153182 --> v2:172.21.15.143:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f74700cd8c0 con 0x7f7470070ba0 2023-11-26T07:22:34.445 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:34.444+0000 7f7475b29640 1 --2- 172.21.15.143:0/1495153182 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f7470070ba0 0x7f74700aead0 secure :-1 s=READY pgs=938 cs=0 l=1 rev1=1 crypto rx=0x7f746c0097c0 tx=0x7f746c01c3f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=52acdc9cdadd77c9 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:22:34.446 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:34.444+0000 7f7474b27640 1 -- 172.21.15.143:0/1495153182 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f746c0ab600 con 0x7f7470070ba0 2023-11-26T07:22:34.446 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:34.444+0000 7f7474b27640 1 -- 172.21.15.143:0/1495153182 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f746c0a7050 con 0x7f7470070ba0 2023-11-26T07:22:34.446 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:34.444+0000 7f7474b27640 1 -- 172.21.15.143:0/1495153182 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f746c0ab980 con 0x7f7470070ba0 2023-11-26T07:22:34.447 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:34.444+0000 7f7477d8d640 1 -- 172.21.15.143:0/1495153182 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f7470070ba0 msgr2=0x7f74700aead0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:22:34.447 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:34.444+0000 7f7477d8d640 1 --2- 172.21.15.143:0/1495153182 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f7470070ba0 0x7f74700aead0 secure :-1 s=READY pgs=938 cs=0 l=1 rev1=1 crypto rx=0x7f746c0097c0 tx=0x7f746c01c3f0 comp rx=0 tx=0).stop 2023-11-26T07:22:34.447 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:34.444+0000 7f7477d8d640 1 -- 172.21.15.143:0/1495153182 shutdown_connections 2023-11-26T07:22:34.447 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:34.444+0000 7f7477d8d640 1 --2- 172.21.15.143:0/1495153182 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f7470070ba0 0x7f74700aead0 unknown :-1 s=CLOSED pgs=938 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:22:34.447 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:34.444+0000 7f7477d8d640 1 -- 172.21.15.143:0/1495153182 >> 172.21.15.143:0/1495153182 conn(0x7f74700cf710 msgr2=0x7f74700cfb20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:22:34.447 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:34.444+0000 7f7477d8d640 1 -- 172.21.15.143:0/1495153182 shutdown_connections 2023-11-26T07:22:34.447 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:34.444+0000 7f7477d8d640 1 -- 172.21.15.143:0/1495153182 wait complete. 2023-11-26T07:22:34.448 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:34.448+0000 7f7477d8d640 1 Processor -- start 2023-11-26T07:22:34.448 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:34.448+0000 7f7477d8d640 1 -- start start 2023-11-26T07:22:34.451 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:34.448+0000 7f7477d8d640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f747006a490 0x7f747014f420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:22:34.451 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:34.448+0000 7f7477d8d640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f7470070ba0 0x7f7470159e30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:22:34.451 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:34.448+0000 7f7477d8d640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f74700cdb60 0x7f74700a9a90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:22:34.451 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:34.448+0000 7f7477d8d640 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7470158b40 con 0x7f747006a490 2023-11-26T07:22:34.451 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:34.448+0000 7f7477d8d640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f74701599a0 con 0x7f7470070ba0 2023-11-26T07:22:34.451 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:34.448+0000 7f7477d8d640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7470158d90 con 0x7f74700cdb60 2023-11-26T07:22:34.451 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:34.448+0000 7f7475b29640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f747006a490 0x7f747014f420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:22:34.451 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:34.448+0000 7f7475b29640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f747006a490 0x7f747014f420 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:47110/0 (socket says 172.21.15.143:47110) 2023-11-26T07:22:34.451 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:34.448+0000 7f7475b29640 1 -- 172.21.15.143:0/606464902 learned_addr learned my addr 172.21.15.143:0/606464902 (peer_addr_for_me v2:172.21.15.143:0/0) 2023-11-26T07:22:34.452 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:34.448+0000 7f7475328640 1 --2- 172.21.15.143:0/606464902 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f7470070ba0 0x7f7470159e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:22:34.452 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:34.448+0000 7f747632a640 1 --2- 172.21.15.143:0/606464902 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f74700cdb60 0x7f74700a9a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:22:34.452 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:34.448+0000 7f7475b29640 1 -- 172.21.15.143:0/606464902 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f74700cdb60 msgr2=0x7f74700a9a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:22:34.452 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:34.448+0000 7f7475b29640 1 --2- 172.21.15.143:0/606464902 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f74700cdb60 0x7f74700a9a90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:22:34.452 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:34.448+0000 7f7475b29640 1 -- 172.21.15.143:0/606464902 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f7470070ba0 msgr2=0x7f7470159e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:22:34.452 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:34.448+0000 7f7475b29640 1 --2- 172.21.15.143:0/606464902 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f7470070ba0 0x7f7470159e30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:22:34.452 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:34.448+0000 7f7475b29640 1 -- 172.21.15.143:0/606464902 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f746c009420 con 0x7f747006a490 2023-11-26T07:22:34.452 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:34.448+0000 7f7475328640 1 --2- 172.21.15.143:0/606464902 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f7470070ba0 0x7f7470159e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-26T07:22:34.452 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:34.448+0000 7f7475b29640 1 --2- 172.21.15.143:0/606464902 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f747006a490 0x7f747014f420 secure :-1 s=READY pgs=939 cs=0 l=1 rev1=1 crypto rx=0x7f746c0ae040 tx=0x7f746c01ac30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:22:34.452 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:34.448+0000 7f74667fc640 1 -- 172.21.15.143:0/606464902 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f746c0b7650 con 0x7f747006a490 2023-11-26T07:22:34.453 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:34.448+0000 7f74667fc640 1 -- 172.21.15.143:0/606464902 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f746c0b77b0 con 0x7f747006a490 2023-11-26T07:22:34.453 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:34.448+0000 7f74667fc640 1 -- 172.21.15.143:0/606464902 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f746c0b7b60 con 0x7f747006a490 2023-11-26T07:22:34.453 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:34.448+0000 7f7477d8d640 1 -- 172.21.15.143:0/606464902 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7470159070 con 0x7f747006a490 2023-11-26T07:22:34.453 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:34.448+0000 7f7477d8d640 1 -- 172.21.15.143:0/606464902 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7470159630 con 0x7f747006a490 2023-11-26T07:22:34.454 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:34.452+0000 7f7477d8d640 1 -- 172.21.15.143:0/606464902 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7470067680 con 0x7f747006a490 2023-11-26T07:22:34.454 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:34.452+0000 7f74667fc640 1 -- 172.21.15.143:0/606464902 <== mon.0 v2:172.21.15.143:3300/0 4 ==== mgrmap(e 11) v1 ==== 86158+0+0 (secure 0 0 0) 0x7f746c0b7cc0 con 0x7f747006a490 2023-11-26T07:22:34.454 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:34.452+0000 7f74667fc640 1 --2- 172.21.15.143:0/606464902 >> [v2:172.21.15.143:6832/2150557544,v1:172.21.15.143:6833/2150557544] conn(0x7f74540667d0 0x7f7454068c90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:22:34.454 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:34.452+0000 7f74667fc640 1 -- 172.21.15.143:0/606464902 <== mon.0 v2:172.21.15.143:3300/0 5 ==== osd_map(67..67 src has 1..67) v4 ==== 4857+0+0 (secure 0 0 0) 0x7f746c11e960 con 0x7f747006a490 2023-11-26T07:22:34.454 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:34.452+0000 7f7475328640 1 --2- 172.21.15.143:0/606464902 >> [v2:172.21.15.143:6832/2150557544,v1:172.21.15.143:6833/2150557544] conn(0x7f74540667d0 0x7f7454068c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:22:34.455 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:34.452+0000 7f7475328640 1 --2- 172.21.15.143:0/606464902 >> [v2:172.21.15.143:6832/2150557544,v1:172.21.15.143:6833/2150557544] conn(0x7f74540667d0 0x7f7454068c90 secure :-1 s=READY pgs=441 cs=0 l=1 rev1=1 crypto rx=0x7f747015cda0 tx=0x7f7460009040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:22:34.457 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:34.456+0000 7f74667fc640 1 -- 172.21.15.143:0/606464902 <== mon.0 v2:172.21.15.143:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172649 (secure 0 0 0) 0x7f746c0ebca0 con 0x7f747006a490 2023-11-26T07:22:34.609 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:34.608+0000 7f7477d8d640 1 -- 172.21.15.143:0/606464902 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f74700665d0 con 0x7f747006a490 2023-11-26T07:22:34.609 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:34.608+0000 7f74667fc640 1 -- 172.21.15.143:0/606464902 <== mon.0 v2:172.21.15.143:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v67) v1 ==== 93+0+10 (secure 0 0 0) 0x7f746c0b00a0 con 0x7f747006a490 2023-11-26T07:22:34.610 INFO:teuthology.orchestra.run.smithi143.stdout:pg_num: 1 2023-11-26T07:22:34.612 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:34.608+0000 7f7477d8d640 1 -- 172.21.15.143:0/606464902 >> [v2:172.21.15.143:6832/2150557544,v1:172.21.15.143:6833/2150557544] conn(0x7f74540667d0 msgr2=0x7f7454068c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:22:34.612 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:34.608+0000 7f7477d8d640 1 --2- 172.21.15.143:0/606464902 >> [v2:172.21.15.143:6832/2150557544,v1:172.21.15.143:6833/2150557544] conn(0x7f74540667d0 0x7f7454068c90 secure :-1 s=READY pgs=441 cs=0 l=1 rev1=1 crypto rx=0x7f747015cda0 tx=0x7f7460009040 comp rx=0 tx=0).stop 2023-11-26T07:22:34.612 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:34.608+0000 7f7477d8d640 1 -- 172.21.15.143:0/606464902 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f747006a490 msgr2=0x7f747014f420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:22:34.612 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:34.608+0000 7f7477d8d640 1 --2- 172.21.15.143:0/606464902 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f747006a490 0x7f747014f420 secure :-1 s=READY pgs=939 cs=0 l=1 rev1=1 crypto rx=0x7f746c0ae040 tx=0x7f746c01ac30 comp rx=0 tx=0).stop 2023-11-26T07:22:34.612 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:34.608+0000 7f7477d8d640 1 -- 172.21.15.143:0/606464902 shutdown_connections 2023-11-26T07:22:34.612 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:34.608+0000 7f7477d8d640 1 --2- 172.21.15.143:0/606464902 >> [v2:172.21.15.143:6832/2150557544,v1:172.21.15.143:6833/2150557544] conn(0x7f74540667d0 0x7f7454068c90 unknown :-1 s=CLOSED pgs=441 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:22:34.612 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:34.608+0000 7f7477d8d640 1 --2- 172.21.15.143:0/606464902 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f74700cdb60 0x7f74700a9a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:22:34.612 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:34.608+0000 7f7477d8d640 1 --2- 172.21.15.143:0/606464902 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f7470070ba0 0x7f7470159e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:22:34.612 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:34.608+0000 7f7477d8d640 1 --2- 172.21.15.143:0/606464902 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f747006a490 0x7f747014f420 unknown :-1 s=CLOSED pgs=939 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:22:34.612 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:34.608+0000 7f7477d8d640 1 -- 172.21.15.143:0/606464902 >> 172.21.15.143:0/606464902 conn(0x7f74700cf710 msgr2=0x7f7470152cb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:22:34.612 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:34.612+0000 7f7477d8d640 1 -- 172.21.15.143:0/606464902 shutdown_connections 2023-11-26T07:22:34.613 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:34.612+0000 7f7477d8d640 1 -- 172.21.15.143:0/606464902 wait complete. 2023-11-26T07:22:34.630 DEBUG:teuthology.orchestra.run.smithi143:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --format=json-pretty osd lspools 2023-11-26T07:22:34.770 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:34.768+0000 7f394c157640 1 Processor -- start 2023-11-26T07:22:34.770 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:34.768+0000 7f394c157640 1 -- start start 2023-11-26T07:22:34.771 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:34.768+0000 7f394c157640 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7f394414b1e0 0x7f39440cf280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:22:34.771 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:34.768+0000 7f394c157640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f39440cf7c0 0x7f39440708c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:22:34.771 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:34.768+0000 7f394c157640 1 -- --> v1:172.21.15.188:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f39440cfc20 con 0x7f3944070e00 2023-11-26T07:22:34.771 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:34.768+0000 7f394c157640 1 -- --> v2:172.21.15.188:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3944162080 con 0x7f394414b1e0 2023-11-26T07:22:34.771 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:34.768+0000 7f394c157640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3944161d70 con 0x7f39440cf7c0 2023-11-26T07:22:34.772 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:34.772+0000 7f39496f2640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f39440cf7c0 0x7f39440708c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:22:34.772 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:34.772+0000 7f3949ef3640 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7f394414b1e0 0x7f39440cf280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:22:34.772 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:34.772+0000 7f3949ef3640 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7f394414b1e0 0x7f39440cf280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.188:3300/0 says I am v2:172.21.15.143:43246/0 (socket says 172.21.15.143:43246) 2023-11-26T07:22:34.772 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:34.772+0000 7f39496f2640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f39440cf7c0 0x7f39440708c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.188:3301/0 says I am v2:172.21.15.143:42304/0 (socket says 172.21.15.143:42304) 2023-11-26T07:22:34.773 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:34.772+0000 7f3949ef3640 1 -- 172.21.15.143:0/689866856 learned_addr learned my addr 172.21.15.143:0/689866856 (peer_addr_for_me v2:172.21.15.143:0/0) 2023-11-26T07:22:34.773 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:34.772+0000 7f394a6f4640 1 --1- 172.21.15.143:0/689866856 >> v1:172.21.15.188:6789/0 conn(0x7f3944070e00 0x7f39440b1ef0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.188:6789/0 says I am v1:172.21.15.143:44186/0 (socket says 172.21.15.143:44186) 2023-11-26T07:22:34.773 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:34.772+0000 7f3949ef3640 1 -- 172.21.15.143:0/689866856 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f39440cf7c0 msgr2=0x7f39440708c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-26T07:22:34.773 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:34.772+0000 7f3949ef3640 1 --2- 172.21.15.143:0/689866856 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f39440cf7c0 0x7f39440708c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:22:34.773 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:34.772+0000 7f3949ef3640 1 -- 172.21.15.143:0/689866856 >> v1:172.21.15.188:6789/0 conn(0x7f3944070e00 legacy=0x7f39440b1ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:22:34.773 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:34.772+0000 7f3949ef3640 1 -- 172.21.15.143:0/689866856 --> v2:172.21.15.188:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f394414b5c0 con 0x7f394414b1e0 2023-11-26T07:22:34.775 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:34.772+0000 7f3949ef3640 1 --2- 172.21.15.143:0/689866856 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f394414b1e0 0x7f39440cf280 secure :-1 s=READY pgs=375 cs=0 l=1 rev1=1 crypto rx=0x7f3940009870 tx=0x7f394001c510 comp rx=0 tx=0).ready entity=mon.1 client_cookie=d243c484f52a459d server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:22:34.775 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:34.772+0000 7f3948ef1640 1 -- 172.21.15.143:0/689866856 <== mon.1 v2:172.21.15.188:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f39400aa670 con 0x7f394414b1e0 2023-11-26T07:22:34.775 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:34.772+0000 7f3948ef1640 1 -- 172.21.15.143:0/689866856 <== mon.1 v2:172.21.15.188:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f39400aa7d0 con 0x7f394414b1e0 2023-11-26T07:22:34.775 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:34.772+0000 7f3948ef1640 1 -- 172.21.15.143:0/689866856 <== mon.1 v2:172.21.15.188:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f39400aab60 con 0x7f394414b1e0 2023-11-26T07:22:34.776 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:34.776+0000 7f394c157640 1 -- 172.21.15.143:0/689866856 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f394414b1e0 msgr2=0x7f39440cf280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:22:34.776 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:34.776+0000 7f394c157640 1 --2- 172.21.15.143:0/689866856 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f394414b1e0 0x7f39440cf280 secure :-1 s=READY pgs=375 cs=0 l=1 rev1=1 crypto rx=0x7f3940009870 tx=0x7f394001c510 comp rx=0 tx=0).stop 2023-11-26T07:22:34.776 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:34.776+0000 7f394c157640 1 -- 172.21.15.143:0/689866856 shutdown_connections 2023-11-26T07:22:34.776 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:34.776+0000 7f394c157640 1 --2- 172.21.15.143:0/689866856 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f39440cf7c0 0x7f39440708c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:22:34.776 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:34.776+0000 7f394c157640 1 --2- 172.21.15.143:0/689866856 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f394414b1e0 0x7f39440cf280 unknown :-1 s=CLOSED pgs=375 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:22:34.776 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:34.776+0000 7f394c157640 1 -- 172.21.15.143:0/689866856 >> 172.21.15.143:0/689866856 conn(0x7f394406a890 msgr2=0x7f394406aca0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:22:34.776 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:34.776+0000 7f394c157640 1 -- 172.21.15.143:0/689866856 shutdown_connections 2023-11-26T07:22:34.777 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:34.776+0000 7f394c157640 1 -- 172.21.15.143:0/689866856 wait complete. 2023-11-26T07:22:34.777 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:34.776+0000 7f394c157640 1 Processor -- start 2023-11-26T07:22:34.778 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:34.776+0000 7f394c157640 1 -- start start 2023-11-26T07:22:34.779 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:34.776+0000 7f394c157640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f3944070e00 0x7f394415f9f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:22:34.779 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:34.776+0000 7f394c157640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f39440cf7c0 0x7f394416b5a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:22:34.779 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:34.776+0000 7f394c157640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f394414b1e0 0x7f3944147400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:22:34.779 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:34.776+0000 7f394c157640 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3944160cc0 con 0x7f394414b1e0 2023-11-26T07:22:34.779 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:34.776+0000 7f394c157640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f39441645a0 con 0x7f39440cf7c0 2023-11-26T07:22:34.779 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:34.776+0000 7f394c157640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3944160040 con 0x7f3944070e00 2023-11-26T07:22:34.780 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:34.776+0000 7f394a6f4640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f394414b1e0 0x7f3944147400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:22:34.780 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:34.776+0000 7f3949ef3640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f3944070e00 0x7f394415f9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:22:34.780 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:34.776+0000 7f394a6f4640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f394414b1e0 0x7f3944147400 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:47122/0 (socket says 172.21.15.143:47122) 2023-11-26T07:22:34.780 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:34.776+0000 7f394a6f4640 1 -- 172.21.15.143:0/427578370 learned_addr learned my addr 172.21.15.143:0/427578370 (peer_addr_for_me v2:172.21.15.143:0/0) 2023-11-26T07:22:34.780 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:34.776+0000 7f39496f2640 1 --2- 172.21.15.143:0/427578370 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f39440cf7c0 0x7f394416b5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:22:34.781 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:34.780+0000 7f394a6f4640 1 -- 172.21.15.143:0/427578370 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f3944070e00 msgr2=0x7f394415f9f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:22:34.781 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:34.780+0000 7f394a6f4640 1 --2- 172.21.15.143:0/427578370 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f3944070e00 0x7f394415f9f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:22:34.781 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:34.780+0000 7f394a6f4640 1 -- 172.21.15.143:0/427578370 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f39440cf7c0 msgr2=0x7f394416b5a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:22:34.781 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:34.780+0000 7f394a6f4640 1 --2- 172.21.15.143:0/427578370 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f39440cf7c0 0x7f394416b5a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:22:34.781 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:34.780+0000 7f394a6f4640 1 -- 172.21.15.143:0/427578370 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f39400094d0 con 0x7f394414b1e0 2023-11-26T07:22:34.781 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:34.780+0000 7f394a6f4640 1 --2- 172.21.15.143:0/427578370 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f394414b1e0 0x7f3944147400 secure :-1 s=READY pgs=940 cs=0 l=1 rev1=1 crypto rx=0x7f393c00c7a0 tx=0x7f393c00cc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:22:34.782 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:34.780+0000 7f393a7fc640 1 -- 172.21.15.143:0/427578370 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f393c00a640 con 0x7f394414b1e0 2023-11-26T07:22:34.782 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:34.780+0000 7f393a7fc640 1 -- 172.21.15.143:0/427578370 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f393c00a7a0 con 0x7f394414b1e0 2023-11-26T07:22:34.782 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:34.780+0000 7f394c157640 1 -- 172.21.15.143:0/427578370 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3944160320 con 0x7f394414b1e0 2023-11-26T07:22:34.782 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:34.780+0000 7f393a7fc640 1 -- 172.21.15.143:0/427578370 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f393c00ab10 con 0x7f394414b1e0 2023-11-26T07:22:34.782 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:34.780+0000 7f394c157640 1 -- 172.21.15.143:0/427578370 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f39441608e0 con 0x7f394414b1e0 2023-11-26T07:22:34.782 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:34.780+0000 7f394c157640 1 -- 172.21.15.143:0/427578370 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3944067680 con 0x7f394414b1e0 2023-11-26T07:22:34.782 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:34.780+0000 7f393a7fc640 1 -- 172.21.15.143:0/427578370 <== mon.0 v2:172.21.15.143:3300/0 4 ==== mgrmap(e 11) v1 ==== 86158+0+0 (secure 0 0 0) 0x7f393c00ac70 con 0x7f394414b1e0 2023-11-26T07:22:34.783 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:34.780+0000 7f393a7fc640 1 --2- 172.21.15.143:0/427578370 >> [v2:172.21.15.143:6832/2150557544,v1:172.21.15.143:6833/2150557544] conn(0x7f39280667d0 0x7f3928068c90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:22:34.783 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:34.780+0000 7f393a7fc640 1 -- 172.21.15.143:0/427578370 <== mon.0 v2:172.21.15.143:3300/0 5 ==== osd_map(67..67 src has 1..67) v4 ==== 4857+0+0 (secure 0 0 0) 0x7f393c083860 con 0x7f394414b1e0 2023-11-26T07:22:34.783 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:34.780+0000 7f3949ef3640 1 --2- 172.21.15.143:0/427578370 >> [v2:172.21.15.143:6832/2150557544,v1:172.21.15.143:6833/2150557544] conn(0x7f39280667d0 0x7f3928068c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:22:34.784 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:34.784+0000 7f3949ef3640 1 --2- 172.21.15.143:0/427578370 >> [v2:172.21.15.143:6832/2150557544,v1:172.21.15.143:6833/2150557544] conn(0x7f39280667d0 0x7f3928068c90 secure :-1 s=READY pgs=442 cs=0 l=1 rev1=1 crypto rx=0x7f39400ae000 tx=0x7f39400b5040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:22:34.787 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:34.784+0000 7f393a7fc640 1 -- 172.21.15.143:0/427578370 <== mon.0 v2:172.21.15.143:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172649 (secure 0 0 0) 0x7f393c050c20 con 0x7f394414b1e0 2023-11-26T07:22:34.940 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:34.936+0000 7f394c157640 1 -- 172.21.15.143:0/427578370 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "osd lspools", "format": "json-pretty"} v 0) v1 -- 0x7f39440709f0 con 0x7f394414b1e0 2023-11-26T07:22:34.940 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:34.940+0000 7f393a7fc640 1 -- 172.21.15.143:0/427578370 <== mon.0 v2:172.21.15.143:3300/0 7 ==== mon_command_ack([{"prefix": "osd lspools", "format": "json-pretty"}]=0 v67) v1 ==== 84+0+65 (secure 0 0 0) 0x7f393c017090 con 0x7f394414b1e0 2023-11-26T07:22:34.940 INFO:teuthology.orchestra.run.smithi143.stdout: 2023-11-26T07:22:34.940 INFO:teuthology.orchestra.run.smithi143.stdout:[ 2023-11-26T07:22:34.940 INFO:teuthology.orchestra.run.smithi143.stdout: { 2023-11-26T07:22:34.940 INFO:teuthology.orchestra.run.smithi143.stdout: "poolnum": 1, 2023-11-26T07:22:34.941 INFO:teuthology.orchestra.run.smithi143.stdout: "poolname": ".mgr" 2023-11-26T07:22:34.941 INFO:teuthology.orchestra.run.smithi143.stdout: } 2023-11-26T07:22:34.941 INFO:teuthology.orchestra.run.smithi143.stdout:] 2023-11-26T07:22:34.942 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:34.940+0000 7f394c157640 1 -- 172.21.15.143:0/427578370 >> [v2:172.21.15.143:6832/2150557544,v1:172.21.15.143:6833/2150557544] conn(0x7f39280667d0 msgr2=0x7f3928068c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:22:34.942 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:34.940+0000 7f394c157640 1 --2- 172.21.15.143:0/427578370 >> [v2:172.21.15.143:6832/2150557544,v1:172.21.15.143:6833/2150557544] conn(0x7f39280667d0 0x7f3928068c90 secure :-1 s=READY pgs=442 cs=0 l=1 rev1=1 crypto rx=0x7f39400ae000 tx=0x7f39400b5040 comp rx=0 tx=0).stop 2023-11-26T07:22:34.942 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:34.940+0000 7f394c157640 1 -- 172.21.15.143:0/427578370 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f394414b1e0 msgr2=0x7f3944147400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:22:34.942 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:34.940+0000 7f394c157640 1 --2- 172.21.15.143:0/427578370 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f394414b1e0 0x7f3944147400 secure :-1 s=READY pgs=940 cs=0 l=1 rev1=1 crypto rx=0x7f393c00c7a0 tx=0x7f393c00cc60 comp rx=0 tx=0).stop 2023-11-26T07:22:34.943 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:34.940+0000 7f394c157640 1 -- 172.21.15.143:0/427578370 shutdown_connections 2023-11-26T07:22:34.943 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:34.940+0000 7f394c157640 1 --2- 172.21.15.143:0/427578370 >> [v2:172.21.15.143:6832/2150557544,v1:172.21.15.143:6833/2150557544] conn(0x7f39280667d0 0x7f3928068c90 unknown :-1 s=CLOSED pgs=442 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:22:34.943 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:34.940+0000 7f394c157640 1 --2- 172.21.15.143:0/427578370 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f394414b1e0 0x7f3944147400 unknown :-1 s=CLOSED pgs=940 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:22:34.943 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:34.940+0000 7f394c157640 1 --2- 172.21.15.143:0/427578370 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f39440cf7c0 0x7f394416b5a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:22:34.943 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:34.940+0000 7f394c157640 1 --2- 172.21.15.143:0/427578370 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f3944070e00 0x7f394415f9f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:22:34.943 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:34.940+0000 7f394c157640 1 -- 172.21.15.143:0/427578370 >> 172.21.15.143:0/427578370 conn(0x7f394406a890 msgr2=0x7f39440b1330 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:22:34.943 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:34.940+0000 7f394c157640 1 -- 172.21.15.143:0/427578370 shutdown_connections 2023-11-26T07:22:34.943 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:34.940+0000 7f394c157640 1 -- 172.21.15.143:0/427578370 wait complete. 2023-11-26T07:22:34.960 DEBUG:tasks.cephfs.filesystem:Creating filesystem 'cephfs' 2023-11-26T07:22:34.961 DEBUG:teuthology.orchestra.run.smithi143:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_metadata --pg_num_min 64 2023-11-26T07:22:35.102 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:35.100+0000 7fcc92e79640 1 Processor -- start 2023-11-26T07:22:35.102 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:35.100+0000 7fcc92e79640 1 -- start start 2023-11-26T07:22:35.103 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:35.100+0000 7fcc92e79640 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7fcc8c0cf280 0x7fcc8c0cf710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:22:35.103 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:35.100+0000 7fcc92e79640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fcc8c0cd1b0 0x7fcc8c0cd5c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:22:35.103 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:35.100+0000 7fcc92e79640 1 -- --> v1:172.21.15.188:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fcc8c0cc730 con 0x7fcc8c143ac0 2023-11-26T07:22:35.103 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:35.100+0000 7fcc92e79640 1 -- --> v2:172.21.15.143:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fcc8c143ea0 con 0x7fcc8c0cf280 2023-11-26T07:22:35.103 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:35.100+0000 7fcc92e79640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcc8c0cfc50 con 0x7fcc8c0cd1b0 2023-11-26T07:22:35.104 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:35.100+0000 7fcc8bfff640 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7fcc8c0cf280 0x7fcc8c0cf710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:22:35.104 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:35.100+0000 7fcc8bfff640 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7fcc8c0cf280 0x7fcc8c0cf710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:47124/0 (socket says 172.21.15.143:47124) 2023-11-26T07:22:35.104 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:35.100+0000 7fcc8bfff640 1 -- 172.21.15.143:0/327130191 learned_addr learned my addr 172.21.15.143:0/327130191 (peer_addr_for_me v2:172.21.15.143:0/0) 2023-11-26T07:22:35.105 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:35.104+0000 7fcc91416640 1 --2- 172.21.15.143:0/327130191 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fcc8c0cd1b0 0x7fcc8c0cd5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:22:35.105 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:35.104+0000 7fcc8bfff640 1 -- 172.21.15.143:0/327130191 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fcc8c0cd1b0 msgr2=0x7fcc8c0cd5c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-26T07:22:35.105 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:35.104+0000 7fcc8bfff640 1 --2- 172.21.15.143:0/327130191 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fcc8c0cd1b0 0x7fcc8c0cd5c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:22:35.105 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:35.104+0000 7fcc8bfff640 1 -- 172.21.15.143:0/327130191 >> v1:172.21.15.188:6789/0 conn(0x7fcc8c143ac0 legacy=0x7fcc8c070370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:22:35.105 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:35.104+0000 7fcc8bfff640 1 -- 172.21.15.143:0/327130191 --> v2:172.21.15.143:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcc8c070ee0 con 0x7fcc8c0cf280 2023-11-26T07:22:35.105 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:35.104+0000 7fcc91416640 1 --2- 172.21.15.143:0/327130191 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fcc8c0cd1b0 0x7fcc8c0cd5c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-26T07:22:35.107 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:35.104+0000 7fcc8bfff640 1 --2- 172.21.15.143:0/327130191 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fcc8c0cf280 0x7fcc8c0cf710 secure :-1 s=READY pgs=941 cs=0 l=1 rev1=1 crypto rx=0x7fcc78009870 tx=0x7fcc7801c540 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7e05c60b6204eb56 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:22:35.107 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:35.104+0000 7fcc8b7fe640 1 -- 172.21.15.143:0/327130191 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcc780a9400 con 0x7fcc8c0cf280 2023-11-26T07:22:35.107 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:35.104+0000 7fcc8b7fe640 1 -- 172.21.15.143:0/327130191 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcc780a9560 con 0x7fcc8c0cf280 2023-11-26T07:22:35.107 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:35.104+0000 7fcc8b7fe640 1 -- 172.21.15.143:0/327130191 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcc780a98f0 con 0x7fcc8c0cf280 2023-11-26T07:22:35.108 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:35.104+0000 7fcc92e79640 1 -- 172.21.15.143:0/327130191 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fcc8c0cf280 msgr2=0x7fcc8c0cf710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:22:35.108 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:35.104+0000 7fcc92e79640 1 --2- 172.21.15.143:0/327130191 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fcc8c0cf280 0x7fcc8c0cf710 secure :-1 s=READY pgs=941 cs=0 l=1 rev1=1 crypto rx=0x7fcc78009870 tx=0x7fcc7801c540 comp rx=0 tx=0).stop 2023-11-26T07:22:35.108 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:35.108+0000 7fcc92e79640 1 -- 172.21.15.143:0/327130191 shutdown_connections 2023-11-26T07:22:35.108 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:35.108+0000 7fcc92e79640 1 --2- 172.21.15.143:0/327130191 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fcc8c0cf280 0x7fcc8c0cf710 unknown :-1 s=CLOSED pgs=941 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:22:35.108 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:35.108+0000 7fcc92e79640 1 --2- 172.21.15.143:0/327130191 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fcc8c0cd1b0 0x7fcc8c0cd5c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:22:35.108 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:35.108+0000 7fcc92e79640 1 -- 172.21.15.143:0/327130191 >> 172.21.15.143:0/327130191 conn(0x7fcc8c06a890 msgr2=0x7fcc8c06aca0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:22:35.109 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:35.108+0000 7fcc92e79640 1 -- 172.21.15.143:0/327130191 shutdown_connections 2023-11-26T07:22:35.109 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:35.108+0000 7fcc92e79640 1 -- 172.21.15.143:0/327130191 wait complete. 2023-11-26T07:22:35.109 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:35.108+0000 7fcc92e79640 1 Processor -- start 2023-11-26T07:22:35.109 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:35.108+0000 7fcc92e79640 1 -- start start 2023-11-26T07:22:35.111 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:35.108+0000 7fcc92e79640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fcc8c0cd1b0 0x7fcc8c13d5b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:22:35.111 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:35.108+0000 7fcc92e79640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fcc8c0cf280 0x7fcc8c0aec40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:22:35.111 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:35.108+0000 7fcc92e79640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fcc8c143ac0 0x7fcc8c15c070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:22:35.111 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:35.108+0000 7fcc92e79640 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcc8c0af180 con 0x7fcc8c143ac0 2023-11-26T07:22:35.111 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:35.108+0000 7fcc92e79640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcc8c13ba90 con 0x7fcc8c0cf280 2023-11-26T07:22:35.111 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:35.108+0000 7fcc92e79640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcc8c13dc40 con 0x7fcc8c0cd1b0 2023-11-26T07:22:35.112 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:35.108+0000 7fcc91416640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fcc8c143ac0 0x7fcc8c15c070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:22:35.112 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:35.108+0000 7fcc91416640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fcc8c143ac0 0x7fcc8c15c070 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:47138/0 (socket says 172.21.15.143:47138) 2023-11-26T07:22:35.112 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:35.108+0000 7fcc91416640 1 -- 172.21.15.143:0/1271849669 learned_addr learned my addr 172.21.15.143:0/1271849669 (peer_addr_for_me v2:172.21.15.143:0/0) 2023-11-26T07:22:35.112 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:35.108+0000 7fcc8bfff640 1 --2- 172.21.15.143:0/1271849669 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fcc8c0cf280 0x7fcc8c0aec40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:22:35.112 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:35.108+0000 7fcc90c15640 1 --2- 172.21.15.143:0/1271849669 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fcc8c0cd1b0 0x7fcc8c13d5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:22:35.112 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:35.108+0000 7fcc91416640 1 -- 172.21.15.143:0/1271849669 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fcc8c0cd1b0 msgr2=0x7fcc8c13d5b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:22:35.112 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:35.108+0000 7fcc91416640 1 --2- 172.21.15.143:0/1271849669 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fcc8c0cd1b0 0x7fcc8c13d5b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:22:35.112 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:35.108+0000 7fcc91416640 1 -- 172.21.15.143:0/1271849669 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fcc8c0cf280 msgr2=0x7fcc8c0aec40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:22:35.112 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:35.108+0000 7fcc91416640 1 --2- 172.21.15.143:0/1271849669 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fcc8c0cf280 0x7fcc8c0aec40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:22:35.112 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:35.108+0000 7fcc91416640 1 -- 172.21.15.143:0/1271849669 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcc780094d0 con 0x7fcc8c143ac0 2023-11-26T07:22:35.113 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:35.108+0000 7fcc91416640 1 --2- 172.21.15.143:0/1271849669 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fcc8c143ac0 0x7fcc8c15c070 secure :-1 s=READY pgs=942 cs=0 l=1 rev1=1 crypto rx=0x7fcc84007df0 tx=0x7fcc8400d4a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:22:35.113 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:35.112+0000 7fcc897fa640 1 -- 172.21.15.143:0/1271849669 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcc840039a0 con 0x7fcc8c143ac0 2023-11-26T07:22:35.114 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:35.112+0000 7fcc897fa640 1 -- 172.21.15.143:0/1271849669 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcc8400f040 con 0x7fcc8c143ac0 2023-11-26T07:22:35.114 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:35.112+0000 7fcc92e79640 1 -- 172.21.15.143:0/1271849669 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcc8c0b1270 con 0x7fcc8c143ac0 2023-11-26T07:22:35.114 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:35.112+0000 7fcc897fa640 1 -- 172.21.15.143:0/1271849669 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcc84003d40 con 0x7fcc8c143ac0 2023-11-26T07:22:35.114 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:35.112+0000 7fcc92e79640 1 -- 172.21.15.143:0/1271849669 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcc8c0b1830 con 0x7fcc8c143ac0 2023-11-26T07:22:35.114 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:35.112+0000 7fcc92e79640 1 -- 172.21.15.143:0/1271849669 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fcc8c0b1a20 con 0x7fcc8c143ac0 2023-11-26T07:22:35.115 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:35.112+0000 7fcc897fa640 1 -- 172.21.15.143:0/1271849669 <== mon.0 v2:172.21.15.143:3300/0 4 ==== mgrmap(e 11) v1 ==== 86158+0+0 (secure 0 0 0) 0x7fcc84004030 con 0x7fcc8c143ac0 2023-11-26T07:22:35.115 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:35.112+0000 7fcc897fa640 1 --2- 172.21.15.143:0/1271849669 >> [v2:172.21.15.143:6832/2150557544,v1:172.21.15.143:6833/2150557544] conn(0x7fcc700667d0 0x7fcc70068c90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:22:35.115 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:35.112+0000 7fcc897fa640 1 -- 172.21.15.143:0/1271849669 <== mon.0 v2:172.21.15.143:3300/0 5 ==== osd_map(67..67 src has 1..67) v4 ==== 4857+0+0 (secure 0 0 0) 0x7fcc84084330 con 0x7fcc8c143ac0 2023-11-26T07:22:35.115 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:35.112+0000 7fcc90c15640 1 --2- 172.21.15.143:0/1271849669 >> [v2:172.21.15.143:6832/2150557544,v1:172.21.15.143:6833/2150557544] conn(0x7fcc700667d0 0x7fcc70068c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:22:35.115 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:35.116+0000 7fcc90c15640 1 --2- 172.21.15.143:0/1271849669 >> [v2:172.21.15.143:6832/2150557544,v1:172.21.15.143:6833/2150557544] conn(0x7fcc700667d0 0x7fcc70068c90 secure :-1 s=READY pgs=443 cs=0 l=1 rev1=1 crypto rx=0x7fcc80007fd0 tx=0x7fcc80009040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:22:35.118 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:35.116+0000 7fcc897fa640 1 -- 172.21.15.143:0/1271849669 <== mon.0 v2:172.21.15.143:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172649 (secure 0 0 0) 0x7fcc84051720 con 0x7fcc8c143ac0 2023-11-26T07:22:35.269 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:35.268+0000 7fcc92e79640 1 -- 172.21.15.143:0/1271849669 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64} v 0) v1 -- 0x7fcc8c13bc00 con 0x7fcc8c143ac0 2023-11-26T07:22:35.835 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:35.832+0000 7fcc897fa640 1 -- 172.21.15.143:0/1271849669 <== mon.0 v2:172.21.15.143:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]=0 pool 'cephfs_metadata' created v68) v1 ==== 138+0+0 (secure 0 0 0) 0x7fcc84054d40 con 0x7fcc8c143ac0 2023-11-26T07:22:35.835 INFO:teuthology.orchestra.run.smithi143.stderr:pool 'cephfs_metadata' created 2023-11-26T07:22:35.838 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:35.836+0000 7fcc92e79640 1 -- 172.21.15.143:0/1271849669 >> [v2:172.21.15.143:6832/2150557544,v1:172.21.15.143:6833/2150557544] conn(0x7fcc700667d0 msgr2=0x7fcc70068c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:22:35.838 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:35.836+0000 7fcc92e79640 1 --2- 172.21.15.143:0/1271849669 >> [v2:172.21.15.143:6832/2150557544,v1:172.21.15.143:6833/2150557544] conn(0x7fcc700667d0 0x7fcc70068c90 secure :-1 s=READY pgs=443 cs=0 l=1 rev1=1 crypto rx=0x7fcc80007fd0 tx=0x7fcc80009040 comp rx=0 tx=0).stop 2023-11-26T07:22:35.838 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:35.836+0000 7fcc92e79640 1 -- 172.21.15.143:0/1271849669 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fcc8c143ac0 msgr2=0x7fcc8c15c070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:22:35.838 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:35.836+0000 7fcc92e79640 1 --2- 172.21.15.143:0/1271849669 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fcc8c143ac0 0x7fcc8c15c070 secure :-1 s=READY pgs=942 cs=0 l=1 rev1=1 crypto rx=0x7fcc84007df0 tx=0x7fcc8400d4a0 comp rx=0 tx=0).stop 2023-11-26T07:22:35.838 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:35.836+0000 7fcc92e79640 1 -- 172.21.15.143:0/1271849669 shutdown_connections 2023-11-26T07:22:35.838 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:35.836+0000 7fcc92e79640 1 --2- 172.21.15.143:0/1271849669 >> [v2:172.21.15.143:6832/2150557544,v1:172.21.15.143:6833/2150557544] conn(0x7fcc700667d0 0x7fcc70068c90 unknown :-1 s=CLOSED pgs=443 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:22:35.838 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:35.836+0000 7fcc92e79640 1 --2- 172.21.15.143:0/1271849669 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fcc8c143ac0 0x7fcc8c15c070 unknown :-1 s=CLOSED pgs=942 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:22:35.838 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:35.836+0000 7fcc92e79640 1 --2- 172.21.15.143:0/1271849669 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fcc8c0cf280 0x7fcc8c0aec40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:22:35.838 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:35.836+0000 7fcc92e79640 1 --2- 172.21.15.143:0/1271849669 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fcc8c0cd1b0 0x7fcc8c13d5b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:22:35.838 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:35.836+0000 7fcc92e79640 1 -- 172.21.15.143:0/1271849669 >> 172.21.15.143:0/1271849669 conn(0x7fcc8c06a890 msgr2=0x7fcc8c13a960 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:22:35.839 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:35.836+0000 7fcc92e79640 1 -- 172.21.15.143:0/1271849669 shutdown_connections 2023-11-26T07:22:35.839 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:35.836+0000 7fcc92e79640 1 -- 172.21.15.143:0/1271849669 wait complete. 2023-11-26T07:22:35.863 DEBUG:teuthology.orchestra.run.smithi143:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_data 64 --pg_num_min 64 --target_size_ratio 0.9 2023-11-26T07:22:36.007 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:36.004+0000 7f88fcfd1640 1 Processor -- start 2023-11-26T07:22:36.007 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:36.004+0000 7f88fcfd1640 1 -- start start 2023-11-26T07:22:36.008 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:36.008+0000 7f88fcfd1640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f88f8161ae0 0x7f88f8070bc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:22:36.009 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:36.008+0000 7f88fcfd1640 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7f88f8071100 0x7f88f80cd620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:22:36.009 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:36.008+0000 7f88fcfd1640 1 -- --> v1:172.21.15.188:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f88f806ddb0 con 0x7f88f815d280 2023-11-26T07:22:36.009 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:36.008+0000 7f88fcfd1640 1 -- --> v2:172.21.15.143:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f88f8071590 con 0x7f88f8071100 2023-11-26T07:22:36.009 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:36.008+0000 7f88fcfd1640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f88f806e090 con 0x7f88f8161ae0 2023-11-26T07:22:36.010 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:36.008+0000 7f88f6d9d640 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7f88f8071100 0x7f88f80cd620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:22:36.010 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:36.008+0000 7f88f6d9d640 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7f88f8071100 0x7f88f80cd620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:47140/0 (socket says 172.21.15.143:47140) 2023-11-26T07:22:36.010 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:36.008+0000 7f88f6d9d640 1 -- 172.21.15.143:0/1781041098 learned_addr learned my addr 172.21.15.143:0/1781041098 (peer_addr_for_me v2:172.21.15.143:0/0) 2023-11-26T07:22:36.010 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:36.008+0000 7f88edd9b640 1 --2- 172.21.15.143:0/1781041098 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f88f8161ae0 0x7f88f8070bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:22:36.010 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:36.008+0000 7f88f6d9d640 1 -- 172.21.15.143:0/1781041098 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f88f8161ae0 msgr2=0x7f88f8070bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-26T07:22:36.010 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:36.008+0000 7f88f6d9d640 1 --2- 172.21.15.143:0/1781041098 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f88f8161ae0 0x7f88f8070bc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:22:36.010 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:36.008+0000 7f88f6d9d640 1 -- 172.21.15.143:0/1781041098 >> v1:172.21.15.188:6789/0 conn(0x7f88f815d280 legacy=0x7f88f80cfe30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-26T07:22:36.010 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:36.008+0000 7f88f6d9d640 1 -- 172.21.15.143:0/1781041098 --> v2:172.21.15.143:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f88f806e370 con 0x7f88f8071100 2023-11-26T07:22:36.012 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:36.012+0000 7f88f6d9d640 1 --2- 172.21.15.143:0/1781041098 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f88f8071100 0x7f88f80cd620 secure :-1 s=READY pgs=943 cs=0 l=1 rev1=1 crypto rx=0x7f88f0009e50 tx=0x7f88f001c9a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=805358ffbea7581 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:22:36.012 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:36.012+0000 7f88f5d9b640 1 -- 172.21.15.143:0/1781041098 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f88f00a5730 con 0x7f88f8071100 2023-11-26T07:22:36.013 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:36.012+0000 7f88f5d9b640 1 -- 172.21.15.143:0/1781041098 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f88f00a5890 con 0x7f88f8071100 2023-11-26T07:22:36.013 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:36.012+0000 7f88f5d9b640 1 -- 172.21.15.143:0/1781041098 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f88f00a5c20 con 0x7f88f8071100 2023-11-26T07:22:36.013 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:36.012+0000 7f88fcfd1640 1 -- 172.21.15.143:0/1781041098 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f88f8071100 msgr2=0x7f88f80cd620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:22:36.014 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:36.012+0000 7f88fcfd1640 1 --2- 172.21.15.143:0/1781041098 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f88f8071100 0x7f88f80cd620 secure :-1 s=READY pgs=943 cs=0 l=1 rev1=1 crypto rx=0x7f88f0009e50 tx=0x7f88f001c9a0 comp rx=0 tx=0).stop 2023-11-26T07:22:36.014 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:36.012+0000 7f88fcfd1640 1 -- 172.21.15.143:0/1781041098 shutdown_connections 2023-11-26T07:22:36.014 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:36.012+0000 7f88fcfd1640 1 --2- 172.21.15.143:0/1781041098 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f88f8071100 0x7f88f80cd620 unknown :-1 s=CLOSED pgs=943 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:22:36.014 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:36.012+0000 7f88fcfd1640 1 --2- 172.21.15.143:0/1781041098 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f88f8161ae0 0x7f88f8070bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:22:36.014 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:36.012+0000 7f88fcfd1640 1 -- 172.21.15.143:0/1781041098 >> 172.21.15.143:0/1781041098 conn(0x7f88f80b1a40 msgr2=0x7f88f80cf720 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:22:36.014 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:36.012+0000 7f88fcfd1640 1 -- 172.21.15.143:0/1781041098 shutdown_connections 2023-11-26T07:22:36.015 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:36.012+0000 7f88fcfd1640 1 -- 172.21.15.143:0/1781041098 wait complete. 2023-11-26T07:22:36.015 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:36.012+0000 7f88fcfd1640 1 Processor -- start 2023-11-26T07:22:36.015 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:36.012+0000 7f88fcfd1640 1 -- start start 2023-11-26T07:22:36.019 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:36.016+0000 7f88fcfd1640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f88f8071100 0x7f88f8156480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:22:36.019 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:36.016+0000 7f88fcfd1640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f88f815d280 0x7f88f815b7e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:22:36.019 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:36.016+0000 7f88fcfd1640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f88f8161ae0 0x7f88f80b0670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:22:36.019 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:36.016+0000 7f88fcfd1640 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f88f815ad90 con 0x7f88f815d280 2023-11-26T07:22:36.019 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:36.016+0000 7f88fcfd1640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f88f815a110 con 0x7f88f8071100 2023-11-26T07:22:36.020 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:36.016+0000 7f88fcfd1640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f88f815a3f0 con 0x7f88f8161ae0 2023-11-26T07:22:36.020 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:36.016+0000 7f88edd9b640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f88f815d280 0x7f88f815b7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:22:36.020 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:36.016+0000 7f88edd9b640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f88f815d280 0x7f88f815b7e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:47144/0 (socket says 172.21.15.143:47144) 2023-11-26T07:22:36.020 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:36.016+0000 7f88edd9b640 1 -- 172.21.15.143:0/2926603435 learned_addr learned my addr 172.21.15.143:0/2926603435 (peer_addr_for_me v2:172.21.15.143:0/0) 2023-11-26T07:22:36.020 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:36.016+0000 7f88f6d9d640 1 --2- 172.21.15.143:0/2926603435 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f88f8161ae0 0x7f88f80b0670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:22:36.020 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:36.016+0000 7f88f659c640 1 --2- 172.21.15.143:0/2926603435 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f88f8071100 0x7f88f8156480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:22:36.020 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:36.016+0000 7f88edd9b640 1 -- 172.21.15.143:0/2926603435 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f88f8161ae0 msgr2=0x7f88f80b0670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:22:36.021 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:36.016+0000 7f88edd9b640 1 --2- 172.21.15.143:0/2926603435 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f88f8161ae0 0x7f88f80b0670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:22:36.021 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:36.016+0000 7f88edd9b640 1 -- 172.21.15.143:0/2926603435 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f88f8071100 msgr2=0x7f88f8156480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:22:36.021 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:36.016+0000 7f88edd9b640 1 --2- 172.21.15.143:0/2926603435 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f88f8071100 0x7f88f8156480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:22:36.021 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:36.016+0000 7f88edd9b640 1 -- 172.21.15.143:0/2926603435 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f88f0009b30 con 0x7f88f815d280 2023-11-26T07:22:36.021 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:36.016+0000 7f88edd9b640 1 --2- 172.21.15.143:0/2926603435 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f88f815d280 0x7f88f815b7e0 secure :-1 s=READY pgs=944 cs=0 l=1 rev1=1 crypto rx=0x7f88e800d5f0 tx=0x7f88e800dab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:22:36.021 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:36.016+0000 7f88ef7fe640 1 -- 172.21.15.143:0/2926603435 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f88e80125f0 con 0x7f88f815d280 2023-11-26T07:22:36.021 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:36.016+0000 7f88ef7fe640 1 -- 172.21.15.143:0/2926603435 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f88e8012750 con 0x7f88f815d280 2023-11-26T07:22:36.021 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:36.016+0000 7f88fcfd1640 1 -- 172.21.15.143:0/2926603435 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f88f815a6d0 con 0x7f88f815d280 2023-11-26T07:22:36.021 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:36.016+0000 7f88ef7fe640 1 -- 172.21.15.143:0/2926603435 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f88e8012ae0 con 0x7f88f815d280 2023-11-26T07:22:36.021 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:36.016+0000 7f88fcfd1640 1 -- 172.21.15.143:0/2926603435 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f88f80cb9d0 con 0x7f88f815d280 2023-11-26T07:22:36.021 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:36.016+0000 7f88fcfd1640 1 -- 172.21.15.143:0/2926603435 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f88f8067720 con 0x7f88f815d280 2023-11-26T07:22:36.022 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:36.020+0000 7f88ef7fe640 1 -- 172.21.15.143:0/2926603435 <== mon.0 v2:172.21.15.143:3300/0 4 ==== mgrmap(e 11) v1 ==== 86158+0+0 (secure 0 0 0) 0x7f88e8012c40 con 0x7f88f815d280 2023-11-26T07:22:36.022 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:36.020+0000 7f88ef7fe640 1 --2- 172.21.15.143:0/2926603435 >> [v2:172.21.15.143:6832/2150557544,v1:172.21.15.143:6833/2150557544] conn(0x7f88d8066740 0x7f88d8068c00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:22:36.022 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:36.020+0000 7f88ef7fe640 1 -- 172.21.15.143:0/2926603435 <== mon.0 v2:172.21.15.143:3300/0 5 ==== osd_map(68..68 src has 1..68) v4 ==== 5227+0+0 (secure 0 0 0) 0x7f88e8084550 con 0x7f88f815d280 2023-11-26T07:22:36.022 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:36.020+0000 7f88f659c640 1 --2- 172.21.15.143:0/2926603435 >> [v2:172.21.15.143:6832/2150557544,v1:172.21.15.143:6833/2150557544] conn(0x7f88d8066740 0x7f88d8068c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:22:36.022 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:36.020+0000 7f88f659c640 1 --2- 172.21.15.143:0/2926603435 >> [v2:172.21.15.143:6832/2150557544,v1:172.21.15.143:6833/2150557544] conn(0x7f88d8066740 0x7f88d8068c00 secure :-1 s=READY pgs=444 cs=0 l=1 rev1=1 crypto rx=0x7f88e0007fd0 tx=0x7f88e0007e40 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:22:36.025 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:36.024+0000 7f88ef7fe640 1 -- 172.21.15.143:0/2926603435 <== mon.0 v2:172.21.15.143:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172649 (secure 0 0 0) 0x7f88e8051830 con 0x7f88f815d280 2023-11-26T07:22:36.175 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:36.172+0000 7f88fcfd1640 1 -- 172.21.15.143:0/2926603435 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9} v 0) v1 -- 0x7f88f80b4bb0 con 0x7f88f815d280 2023-11-26T07:22:36.842 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:36.840+0000 7f88ef7fe640 1 -- 172.21.15.143:0/2926603435 <== mon.0 v2:172.21.15.143:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]=0 pool 'cephfs_data' created v69) v1 ==== 170+0+0 (secure 0 0 0) 0x7f88e8054e50 con 0x7f88f815d280 2023-11-26T07:22:36.842 INFO:teuthology.orchestra.run.smithi143.stderr:pool 'cephfs_data' created 2023-11-26T07:22:36.844 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:36.844+0000 7f88fcfd1640 1 -- 172.21.15.143:0/2926603435 >> [v2:172.21.15.143:6832/2150557544,v1:172.21.15.143:6833/2150557544] conn(0x7f88d8066740 msgr2=0x7f88d8068c00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:22:36.844 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:36.844+0000 7f88fcfd1640 1 --2- 172.21.15.143:0/2926603435 >> [v2:172.21.15.143:6832/2150557544,v1:172.21.15.143:6833/2150557544] conn(0x7f88d8066740 0x7f88d8068c00 secure :-1 s=READY pgs=444 cs=0 l=1 rev1=1 crypto rx=0x7f88e0007fd0 tx=0x7f88e0007e40 comp rx=0 tx=0).stop 2023-11-26T07:22:36.844 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:36.844+0000 7f88fcfd1640 1 -- 172.21.15.143:0/2926603435 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f88f815d280 msgr2=0x7f88f815b7e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:22:36.844 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:36.844+0000 7f88fcfd1640 1 --2- 172.21.15.143:0/2926603435 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f88f815d280 0x7f88f815b7e0 secure :-1 s=READY pgs=944 cs=0 l=1 rev1=1 crypto rx=0x7f88e800d5f0 tx=0x7f88e800dab0 comp rx=0 tx=0).stop 2023-11-26T07:22:36.845 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:36.844+0000 7f88fcfd1640 1 -- 172.21.15.143:0/2926603435 shutdown_connections 2023-11-26T07:22:36.845 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:36.844+0000 7f88fcfd1640 1 --2- 172.21.15.143:0/2926603435 >> [v2:172.21.15.143:6832/2150557544,v1:172.21.15.143:6833/2150557544] conn(0x7f88d8066740 0x7f88d8068c00 unknown :-1 s=CLOSED pgs=444 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:22:36.845 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:36.844+0000 7f88fcfd1640 1 --2- 172.21.15.143:0/2926603435 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f88f8161ae0 0x7f88f80b0670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:22:36.845 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:36.844+0000 7f88fcfd1640 1 --2- 172.21.15.143:0/2926603435 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f88f815d280 0x7f88f815b7e0 unknown :-1 s=CLOSED pgs=944 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:22:36.845 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:36.844+0000 7f88fcfd1640 1 --2- 172.21.15.143:0/2926603435 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f88f8071100 0x7f88f8156480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:22:36.845 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:36.844+0000 7f88fcfd1640 1 -- 172.21.15.143:0/2926603435 >> 172.21.15.143:0/2926603435 conn(0x7f88f80b1a40 msgr2=0x7f88f80acfa0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:22:36.845 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:36.844+0000 7f88fcfd1640 1 -- 172.21.15.143:0/2926603435 shutdown_connections 2023-11-26T07:22:36.846 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:36.844+0000 7f88fcfd1640 1 -- 172.21.15.143:0/2926603435 wait complete. 2023-11-26T07:22:36.880 DEBUG:teuthology.orchestra.run.smithi143:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs new cephfs cephfs_metadata cephfs_data 2023-11-26T07:22:37.020 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:37.020+0000 7f710d72a640 1 Processor -- start 2023-11-26T07:22:37.021 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:37.020+0000 7f710d72a640 1 -- start start 2023-11-26T07:22:37.022 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:37.020+0000 7f710d72a640 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7f710815d6e0 0x7f71080d0df0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:22:37.022 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:37.020+0000 7f710d72a640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f71080d1bc0 0x7f71080b0750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:22:37.022 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:37.020+0000 7f710d72a640 1 -- --> v1:172.21.15.143:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f71080cf0e0 con 0x7f71080b2160 2023-11-26T07:22:37.022 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:37.020+0000 7f710d72a640 1 -- --> v2:172.21.15.143:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f710815dac0 con 0x7f710815d6e0 2023-11-26T07:22:37.023 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:37.020+0000 7f710d72a640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f71080ceb20 con 0x7f71080d1bc0 2023-11-26T07:22:37.023 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:37.020+0000 7f7106ffd640 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7f710815d6e0 0x7f71080d0df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:22:37.023 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:37.020+0000 7f7106ffd640 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7f710815d6e0 0x7f71080d0df0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:47152/0 (socket says 172.21.15.143:47152) 2023-11-26T07:22:37.023 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:37.020+0000 7f7106ffd640 1 -- 172.21.15.143:0/4127360848 learned_addr learned my addr 172.21.15.143:0/4127360848 (peer_addr_for_me v2:172.21.15.143:0/0) 2023-11-26T07:22:37.023 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:37.020+0000 7f70fe7fc640 1 --1- 172.21.15.143:0/4127360848 >> v1:172.21.15.143:6789/0 conn(0x7f71080b2160 0x7f71080d14b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.143:6789/0 says I am v1:172.21.15.143:35738/0 (socket says 172.21.15.143:35738) 2023-11-26T07:22:37.023 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:37.020+0000 7f71077fe640 1 --2- 172.21.15.143:0/4127360848 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f71080d1bc0 0x7f71080b0750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:22:37.023 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:37.020+0000 7f7106ffd640 1 -- 172.21.15.143:0/4127360848 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f71080d1bc0 msgr2=0x7f71080b0750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-26T07:22:37.023 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:37.020+0000 7f7106ffd640 1 --2- 172.21.15.143:0/4127360848 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f71080d1bc0 0x7f71080b0750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:22:37.023 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:37.020+0000 7f7106ffd640 1 -- 172.21.15.143:0/4127360848 >> v1:172.21.15.143:6789/0 conn(0x7f71080b2160 legacy=0x7f71080d14b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:22:37.023 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:37.020+0000 7f7106ffd640 1 -- 172.21.15.143:0/4127360848 --> v2:172.21.15.143:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f71080cee00 con 0x7f710815d6e0 2023-11-26T07:22:37.025 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:37.024+0000 7f7106ffd640 1 --2- 172.21.15.143:0/4127360848 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f710815d6e0 0x7f71080d0df0 secure :-1 s=READY pgs=946 cs=0 l=1 rev1=1 crypto rx=0x7f70f8009870 tx=0x7f70f801c540 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d1fff8cd30523a27 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:22:37.025 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:37.024+0000 7f71067fc640 1 -- 172.21.15.143:0/4127360848 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f70f80a7070 con 0x7f710815d6e0 2023-11-26T07:22:37.026 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:37.024+0000 7f71067fc640 1 -- 172.21.15.143:0/4127360848 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f70f80ab430 con 0x7f710815d6e0 2023-11-26T07:22:37.026 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:37.024+0000 7f71067fc640 1 -- 172.21.15.143:0/4127360848 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f70f80ab7c0 con 0x7f710815d6e0 2023-11-26T07:22:37.026 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:37.024+0000 7f710d72a640 1 -- 172.21.15.143:0/4127360848 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f710815d6e0 msgr2=0x7f71080d0df0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:22:37.026 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:37.024+0000 7f710d72a640 1 --2- 172.21.15.143:0/4127360848 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f710815d6e0 0x7f71080d0df0 secure :-1 s=READY pgs=946 cs=0 l=1 rev1=1 crypto rx=0x7f70f8009870 tx=0x7f70f801c540 comp rx=0 tx=0).stop 2023-11-26T07:22:37.027 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:37.024+0000 7f710d72a640 1 -- 172.21.15.143:0/4127360848 shutdown_connections 2023-11-26T07:22:37.027 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:37.024+0000 7f710d72a640 1 --2- 172.21.15.143:0/4127360848 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f71080d1bc0 0x7f71080b0750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:22:37.027 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:37.024+0000 7f710d72a640 1 --2- 172.21.15.143:0/4127360848 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f710815d6e0 0x7f71080d0df0 unknown :-1 s=CLOSED pgs=946 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:22:37.027 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:37.024+0000 7f710d72a640 1 -- 172.21.15.143:0/4127360848 >> 172.21.15.143:0/4127360848 conn(0x7f710809c550 msgr2=0x7f71080d3220 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:22:37.027 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:37.024+0000 7f710d72a640 1 -- 172.21.15.143:0/4127360848 shutdown_connections 2023-11-26T07:22:37.027 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:37.024+0000 7f710d72a640 1 -- 172.21.15.143:0/4127360848 wait complete. 2023-11-26T07:22:37.027 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:37.028+0000 7f710d72a640 1 Processor -- start 2023-11-26T07:22:37.028 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:37.028+0000 7f710d72a640 1 -- start start 2023-11-26T07:22:37.029 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:37.028+0000 7f710d72a640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f71080b2160 0x7f7108153410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:22:37.029 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:37.028+0000 7f710d72a640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f71080d1bc0 0x7f710815b570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:22:37.029 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:37.028+0000 7f710d72a640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f710815d6e0 0x7f710815f2e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:22:37.029 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:37.028+0000 7f710d72a640 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f71081541e0 con 0x7f710815d6e0 2023-11-26T07:22:37.029 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:37.028+0000 7f710d72a640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7108153950 con 0x7f71080b2160 2023-11-26T07:22:37.029 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:37.028+0000 7f710d72a640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f710815bfc0 con 0x7f71080d1bc0 2023-11-26T07:22:37.029 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:37.028+0000 7f71077fe640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f710815d6e0 0x7f710815f2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:22:37.030 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:37.028+0000 7f71077fe640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f710815d6e0 0x7f710815f2e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:47166/0 (socket says 172.21.15.143:47166) 2023-11-26T07:22:37.030 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:37.028+0000 7f71077fe640 1 -- 172.21.15.143:0/2745601677 learned_addr learned my addr 172.21.15.143:0/2745601677 (peer_addr_for_me v2:172.21.15.143:0/0) 2023-11-26T07:22:37.032 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:37.028+0000 7f70fe7fc640 1 --2- 172.21.15.143:0/2745601677 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f71080d1bc0 0x7f710815b570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:22:37.032 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:37.028+0000 7f7106ffd640 1 --2- 172.21.15.143:0/2745601677 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f71080b2160 0x7f7108153410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:22:37.032 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:37.028+0000 7f71077fe640 1 -- 172.21.15.143:0/2745601677 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f71080d1bc0 msgr2=0x7f710815b570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:22:37.032 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:37.028+0000 7f71077fe640 1 --2- 172.21.15.143:0/2745601677 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f71080d1bc0 0x7f710815b570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:22:37.032 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:37.028+0000 7f71077fe640 1 -- 172.21.15.143:0/2745601677 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f71080b2160 msgr2=0x7f7108153410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:22:37.032 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:37.028+0000 7f71077fe640 1 --2- 172.21.15.143:0/2745601677 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f71080b2160 0x7f7108153410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:22:37.032 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:37.028+0000 7f71077fe640 1 -- 172.21.15.143:0/2745601677 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f70f80094d0 con 0x7f710815d6e0 2023-11-26T07:22:37.032 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:37.028+0000 7f7106ffd640 1 --2- 172.21.15.143:0/2745601677 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f71080b2160 0x7f7108153410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-26T07:22:37.032 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:37.028+0000 7f70fe7fc640 1 --2- 172.21.15.143:0/2745601677 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f71080d1bc0 0x7f710815b570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-26T07:22:37.032 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:37.028+0000 7f71077fe640 1 --2- 172.21.15.143:0/2745601677 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f710815d6e0 0x7f710815f2e0 secure :-1 s=READY pgs=947 cs=0 l=1 rev1=1 crypto rx=0x7f710000e930 tx=0x7f710000edf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:22:37.032 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:37.028+0000 7f70fffff640 1 -- 172.21.15.143:0/2745601677 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f71000096b0 con 0x7f710815d6e0 2023-11-26T07:22:37.032 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:37.028+0000 7f70fffff640 1 -- 172.21.15.143:0/2745601677 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f710000c9a0 con 0x7f710815d6e0 2023-11-26T07:22:37.033 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:37.028+0000 7f710d72a640 1 -- 172.21.15.143:0/2745601677 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f710815c2a0 con 0x7f710815d6e0 2023-11-26T07:22:37.033 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:37.028+0000 7f70fffff640 1 -- 172.21.15.143:0/2745601677 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f710000cd30 con 0x7f710815d6e0 2023-11-26T07:22:37.033 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:37.028+0000 7f710d72a640 1 -- 172.21.15.143:0/2745601677 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7108153e70 con 0x7f710815d6e0 2023-11-26T07:22:37.033 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:37.028+0000 7f710d72a640 1 -- 172.21.15.143:0/2745601677 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7108154350 con 0x7f710815d6e0 2023-11-26T07:22:37.034 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:37.032+0000 7f70fffff640 1 -- 172.21.15.143:0/2745601677 <== mon.0 v2:172.21.15.143:3300/0 4 ==== mgrmap(e 11) v1 ==== 86158+0+0 (secure 0 0 0) 0x7f7100013470 con 0x7f710815d6e0 2023-11-26T07:22:37.034 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:37.032+0000 7f70fffff640 1 --2- 172.21.15.143:0/2745601677 >> [v2:172.21.15.143:6832/2150557544,v1:172.21.15.143:6833/2150557544] conn(0x7f70e80667d0 0x7f70e8068c90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:22:37.034 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:37.032+0000 7f70fffff640 1 -- 172.21.15.143:0/2745601677 <== mon.0 v2:172.21.15.143:3300/0 5 ==== osd_map(69..69 src has 1..69) v4 ==== 5609+0+0 (secure 0 0 0) 0x7f7100084c90 con 0x7f710815d6e0 2023-11-26T07:22:37.034 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:37.032+0000 7f7106ffd640 1 --2- 172.21.15.143:0/2745601677 >> [v2:172.21.15.143:6832/2150557544,v1:172.21.15.143:6833/2150557544] conn(0x7f70e80667d0 0x7f70e8068c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:22:37.034 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:37.032+0000 7f7106ffd640 1 --2- 172.21.15.143:0/2745601677 >> [v2:172.21.15.143:6832/2150557544,v1:172.21.15.143:6833/2150557544] conn(0x7f70e80667d0 0x7f70e8068c90 secure :-1 s=READY pgs=445 cs=0 l=1 rev1=1 crypto rx=0x7f70f80af000 tx=0x7f70f8008040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:22:37.036 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:37.036+0000 7f70fffff640 1 -- 172.21.15.143:0/2745601677 <== mon.0 v2:172.21.15.143:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172649 (secure 0 0 0) 0x7f7100051f20 con 0x7f710815d6e0 2023-11-26T07:22:37.218 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:37.216+0000 7f710d72a640 1 -- 172.21.15.143:0/2745601677 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"} v 0) v1 -- 0x7f7108067680 con 0x7f710815d6e0 2023-11-26T07:22:37.858 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:37.856+0000 7f70fffff640 1 -- 172.21.15.143:0/2745601677 <== mon.0 v2:172.21.15.143:3300/0 7 ==== mon_command_ack([{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]=0 new fs with metadata pool 10 and data pool 11 v39) v1 ==== 174+0+0 (secure 0 0 0) 0x7f71000210c0 con 0x7f710815d6e0 2023-11-26T07:22:37.859 INFO:teuthology.orchestra.run.smithi143.stderr:new fs with metadata pool 10 and data pool 11 2023-11-26T07:22:37.862 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:37.860+0000 7f710d72a640 1 -- 172.21.15.143:0/2745601677 >> [v2:172.21.15.143:6832/2150557544,v1:172.21.15.143:6833/2150557544] conn(0x7f70e80667d0 msgr2=0x7f70e8068c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:22:37.862 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:37.860+0000 7f710d72a640 1 --2- 172.21.15.143:0/2745601677 >> [v2:172.21.15.143:6832/2150557544,v1:172.21.15.143:6833/2150557544] conn(0x7f70e80667d0 0x7f70e8068c90 secure :-1 s=READY pgs=445 cs=0 l=1 rev1=1 crypto rx=0x7f70f80af000 tx=0x7f70f8008040 comp rx=0 tx=0).stop 2023-11-26T07:22:37.862 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:37.860+0000 7f710d72a640 1 -- 172.21.15.143:0/2745601677 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f710815d6e0 msgr2=0x7f710815f2e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:22:37.862 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:37.860+0000 7f710d72a640 1 --2- 172.21.15.143:0/2745601677 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f710815d6e0 0x7f710815f2e0 secure :-1 s=READY pgs=947 cs=0 l=1 rev1=1 crypto rx=0x7f710000e930 tx=0x7f710000edf0 comp rx=0 tx=0).stop 2023-11-26T07:22:37.864 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:37.864+0000 7f710d72a640 1 -- 172.21.15.143:0/2745601677 shutdown_connections 2023-11-26T07:22:37.865 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:37.864+0000 7f710d72a640 1 --2- 172.21.15.143:0/2745601677 >> [v2:172.21.15.143:6832/2150557544,v1:172.21.15.143:6833/2150557544] conn(0x7f70e80667d0 0x7f70e8068c90 secure :-1 s=CLOSED pgs=445 cs=0 l=1 rev1=1 crypto rx=0x7f70f80af000 tx=0x7f70f8008040 comp rx=0 tx=0).stop 2023-11-26T07:22:37.865 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:37.864+0000 7f710d72a640 1 --2- 172.21.15.143:0/2745601677 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f710815d6e0 0x7f710815f2e0 unknown :-1 s=CLOSED pgs=947 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:22:37.865 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:37.864+0000 7f710d72a640 1 --2- 172.21.15.143:0/2745601677 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f71080d1bc0 0x7f710815b570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:22:37.865 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:37.864+0000 7f710d72a640 1 --2- 172.21.15.143:0/2745601677 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f71080b2160 0x7f7108153410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:22:37.865 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:37.864+0000 7f710d72a640 1 -- 172.21.15.143:0/2745601677 >> 172.21.15.143:0/2745601677 conn(0x7f710809c550 msgr2=0x7f71080734a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:22:37.868 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:37.868+0000 7f710d72a640 1 -- 172.21.15.143:0/2745601677 shutdown_connections 2023-11-26T07:22:37.868 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:37.868+0000 7f710d72a640 1 -- 172.21.15.143:0/2745601677 wait complete. 2023-11-26T07:22:37.898 DEBUG:teuthology.orchestra.run.smithi143:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-11-26T07:22:38.036 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:38.036+0000 7f81e2b01640 1 Processor -- start 2023-11-26T07:22:38.036 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:38.036+0000 7f81e2b01640 1 -- start start 2023-11-26T07:22:38.038 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:38.036+0000 7f81e2b01640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f81dc0cf280 0x7f81dc0cf6e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:22:38.038 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:38.036+0000 7f81e2b01640 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7f81dc0cd1b0 0x7f81dc0cd5e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:22:38.038 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:38.036+0000 7f81e2b01640 1 -- --> v1:172.21.15.188:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f81dc0cfc20 con 0x7f81dc143ac0 2023-11-26T07:22:38.038 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:38.036+0000 7f81e2b01640 1 -- --> v2:172.21.15.188:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f81dc143ea0 con 0x7f81dc0cd1b0 2023-11-26T07:22:38.038 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:38.036+0000 7f81e2b01640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f81dc0cc730 con 0x7f81dc0cf280 2023-11-26T07:22:38.038 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:38.036+0000 7f81dbfff640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f81dc0cf280 0x7f81dc0cf6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:22:38.038 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:38.036+0000 7f81e109e640 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7f81dc0cd1b0 0x7f81dc0cd5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:22:38.038 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:38.036+0000 7f81dbfff640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f81dc0cf280 0x7f81dc0cf6e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.188:3301/0 says I am v2:172.21.15.143:42348/0 (socket says 172.21.15.143:42348) 2023-11-26T07:22:38.038 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:38.036+0000 7f81dbfff640 1 -- 172.21.15.143:0/1556908592 learned_addr learned my addr 172.21.15.143:0/1556908592 (peer_addr_for_me v2:172.21.15.143:0/0) 2023-11-26T07:22:38.039 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:38.036+0000 7f81e089d640 1 --1- 172.21.15.143:0/1556908592 >> v1:172.21.15.188:6789/0 conn(0x7f81dc143ac0 0x7f81dc070370 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.188:6789/0 says I am v1:172.21.15.143:44214/0 (socket says 172.21.15.143:44214) 2023-11-26T07:22:38.039 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:38.036+0000 7f81dbfff640 1 -- 172.21.15.143:0/1556908592 >> v2:172.21.15.188:3300/0 conn(0x7f81dc0cd1b0 msgr2=0x7f81dc0cd5e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-26T07:22:38.039 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:38.036+0000 7f81dbfff640 1 --2- 172.21.15.143:0/1556908592 >> v2:172.21.15.188:3300/0 conn(0x7f81dc0cd1b0 0x7f81dc0cd5e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:22:38.040 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:38.036+0000 7f81dbfff640 1 -- 172.21.15.143:0/1556908592 >> v1:172.21.15.188:6789/0 conn(0x7f81dc143ac0 legacy=0x7f81dc070370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:22:38.040 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:38.036+0000 7f81dbfff640 1 -- 172.21.15.143:0/1556908592 --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f81dc070ee0 con 0x7f81dc0cf280 2023-11-26T07:22:38.040 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:38.036+0000 7f81e109e640 1 --2- 172.21.15.143:0/1556908592 >> v2:172.21.15.188:3300/0 conn(0x7f81dc0cd1b0 0x7f81dc0cd5e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-26T07:22:38.041 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:38.040+0000 7f81dbfff640 1 --2- 172.21.15.143:0/1556908592 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f81dc0cf280 0x7f81dc0cf6e0 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7f81c8009870 tx=0x7f81c801c540 comp rx=0 tx=0).ready entity=mon.2 client_cookie=c7610698486f0cf0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:22:38.042 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:38.040+0000 7f81db7fe640 1 -- 172.21.15.143:0/1556908592 <== mon.2 v2:172.21.15.188:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f81c80a9400 con 0x7f81dc0cf280 2023-11-26T07:22:38.042 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:38.040+0000 7f81db7fe640 1 -- 172.21.15.143:0/1556908592 <== mon.2 v2:172.21.15.188:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f81c80a9560 con 0x7f81dc0cf280 2023-11-26T07:22:38.042 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:38.040+0000 7f81db7fe640 1 -- 172.21.15.143:0/1556908592 <== mon.2 v2:172.21.15.188:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f81c80a98f0 con 0x7f81dc0cf280 2023-11-26T07:22:38.042 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:38.040+0000 7f81e2b01640 1 -- 172.21.15.143:0/1556908592 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f81dc0cf280 msgr2=0x7f81dc0cf6e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:22:38.043 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:38.040+0000 7f81e2b01640 1 --2- 172.21.15.143:0/1556908592 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f81dc0cf280 0x7f81dc0cf6e0 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7f81c8009870 tx=0x7f81c801c540 comp rx=0 tx=0).stop 2023-11-26T07:22:38.043 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:38.040+0000 7f81e2b01640 1 -- 172.21.15.143:0/1556908592 shutdown_connections 2023-11-26T07:22:38.043 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:38.040+0000 7f81e2b01640 1 --2- 172.21.15.143:0/1556908592 >> v2:172.21.15.188:3300/0 conn(0x7f81dc0cd1b0 0x7f81dc0cd5e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:22:38.043 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:38.040+0000 7f81e2b01640 1 --2- 172.21.15.143:0/1556908592 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f81dc0cf280 0x7f81dc0cf6e0 unknown :-1 s=CLOSED pgs=122 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:22:38.043 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:38.040+0000 7f81e2b01640 1 -- 172.21.15.143:0/1556908592 >> 172.21.15.143:0/1556908592 conn(0x7f81dc06a890 msgr2=0x7f81dc06aca0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:22:38.043 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:38.040+0000 7f81e2b01640 1 -- 172.21.15.143:0/1556908592 shutdown_connections 2023-11-26T07:22:38.043 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:38.040+0000 7f81e2b01640 1 -- 172.21.15.143:0/1556908592 wait complete. 2023-11-26T07:22:38.044 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:38.044+0000 7f81e2b01640 1 Processor -- start 2023-11-26T07:22:38.044 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:38.044+0000 7f81e2b01640 1 -- start start 2023-11-26T07:22:38.045 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:38.044+0000 7f81e2b01640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f81dc0cd1b0 0x7f81dc154c60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:22:38.045 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:38.044+0000 7f81e2b01640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f81dc0cf280 0x7f81dc12e0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:22:38.045 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:38.044+0000 7f81e2b01640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f81dc143ac0 0x7f81dc0b3370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:22:38.045 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:38.044+0000 7f81e2b01640 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f81dc130fb0 con 0x7f81dc0cd1b0 2023-11-26T07:22:38.045 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:38.044+0000 7f81e2b01640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f81dc1551a0 con 0x7f81dc0cf280 2023-11-26T07:22:38.046 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:38.044+0000 7f81e2b01640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f81dc0b0610 con 0x7f81dc143ac0 2023-11-26T07:22:38.046 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:38.044+0000 7f81e089d640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f81dc0cd1b0 0x7f81dc154c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:22:38.046 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:38.044+0000 7f81e089d640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f81dc0cd1b0 0x7f81dc154c60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:47182/0 (socket says 172.21.15.143:47182) 2023-11-26T07:22:38.046 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:38.044+0000 7f81e089d640 1 -- 172.21.15.143:0/1664653920 learned_addr learned my addr 172.21.15.143:0/1664653920 (peer_addr_for_me v2:172.21.15.143:0/0) 2023-11-26T07:22:38.046 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:38.044+0000 7f81e109e640 1 --2- 172.21.15.143:0/1664653920 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f81dc143ac0 0x7f81dc0b3370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:22:38.046 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:38.044+0000 7f81dbfff640 1 --2- 172.21.15.143:0/1664653920 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f81dc0cf280 0x7f81dc12e0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:22:38.047 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:38.044+0000 7f81e089d640 1 -- 172.21.15.143:0/1664653920 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f81dc143ac0 msgr2=0x7f81dc0b3370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:22:38.048 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:38.044+0000 7f81e089d640 1 --2- 172.21.15.143:0/1664653920 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f81dc143ac0 0x7f81dc0b3370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:22:38.048 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:38.044+0000 7f81e089d640 1 -- 172.21.15.143:0/1664653920 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f81dc0cf280 msgr2=0x7f81dc12e0a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:22:38.048 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:38.044+0000 7f81e089d640 1 --2- 172.21.15.143:0/1664653920 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f81dc0cf280 0x7f81dc12e0a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:22:38.048 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:38.044+0000 7f81e089d640 1 -- 172.21.15.143:0/1664653920 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f81d4009a70 con 0x7f81dc0cd1b0 2023-11-26T07:22:38.048 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:38.044+0000 7f81e089d640 1 --2- 172.21.15.143:0/1664653920 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f81dc0cd1b0 0x7f81dc154c60 secure :-1 s=READY pgs=948 cs=0 l=1 rev1=1 crypto rx=0x7f81d000c460 tx=0x7f81d000c860 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:22:38.048 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:38.044+0000 7f81d97fa640 1 -- 172.21.15.143:0/1664653920 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f81d00074e0 con 0x7f81dc0cd1b0 2023-11-26T07:22:38.048 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:38.044+0000 7f81d97fa640 1 -- 172.21.15.143:0/1664653920 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f81d0007640 con 0x7f81dc0cd1b0 2023-11-26T07:22:38.048 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:38.044+0000 7f81d97fa640 1 -- 172.21.15.143:0/1664653920 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f81d0004a10 con 0x7f81dc0cd1b0 2023-11-26T07:22:38.048 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:38.044+0000 7f81e2b01640 1 -- 172.21.15.143:0/1664653920 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f81c80094d0 con 0x7f81dc0cd1b0 2023-11-26T07:22:38.049 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:38.044+0000 7f81e2b01640 1 -- 172.21.15.143:0/1664653920 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f81dc158da0 con 0x7f81dc0cd1b0 2023-11-26T07:22:38.049 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:38.044+0000 7f81e2b01640 1 -- 172.21.15.143:0/1664653920 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f81dc067680 con 0x7f81dc0cd1b0 2023-11-26T07:22:38.049 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:38.048+0000 7f81d97fa640 1 -- 172.21.15.143:0/1664653920 <== mon.0 v2:172.21.15.143:3300/0 4 ==== mgrmap(e 11) v1 ==== 86158+0+0 (secure 0 0 0) 0x7f81d0004b70 con 0x7f81dc0cd1b0 2023-11-26T07:22:38.050 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:38.048+0000 7f81d97fa640 1 --2- 172.21.15.143:0/1664653920 >> [v2:172.21.15.143:6832/2150557544,v1:172.21.15.143:6833/2150557544] conn(0x7f81c00667d0 0x7f81c0068c90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:22:38.050 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:38.048+0000 7f81d97fa640 1 -- 172.21.15.143:0/1664653920 <== mon.0 v2:172.21.15.143:3300/0 5 ==== osd_map(70..70 src has 1..70) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f81d0083cb0 con 0x7f81dc0cd1b0 2023-11-26T07:22:38.050 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:38.048+0000 7f81dbfff640 1 --2- 172.21.15.143:0/1664653920 >> [v2:172.21.15.143:6832/2150557544,v1:172.21.15.143:6833/2150557544] conn(0x7f81c00667d0 0x7f81c0068c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:22:38.050 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:38.048+0000 7f81dbfff640 1 --2- 172.21.15.143:0/1664653920 >> [v2:172.21.15.143:6832/2150557544,v1:172.21.15.143:6833/2150557544] conn(0x7f81c00667d0 0x7f81c0068c90 secure :-1 s=READY pgs=446 cs=0 l=1 rev1=1 crypto rx=0x7f81c80ae000 tx=0x7f81c80b3040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:22:38.053 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:38.052+0000 7f81d97fa640 1 -- 172.21.15.143:0/1664653920 <== mon.0 v2:172.21.15.143:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172649 (secure 0 0 0) 0x7f81d0050ce0 con 0x7f81dc0cd1b0 2023-11-26T07:22:38.206 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:38.204+0000 7f81e2b01640 1 -- 172.21.15.143:0/1664653920 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f81dc071c00 con 0x7f81dc0cd1b0 2023-11-26T07:22:38.207 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:38.204+0000 7f81d97fa640 1 -- 172.21.15.143:0/1664653920 <== mon.0 v2:172.21.15.143:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v70) v1 ==== 74+0+16617 (secure 0 0 0) 0x7f81d0054300 con 0x7f81dc0cd1b0 2023-11-26T07:22:38.207 INFO:teuthology.orchestra.run.smithi143.stdout: 2023-11-26T07:22:38.208 INFO:teuthology.orchestra.run.smithi143.stdout:{"epoch":70,"fsid":"f5033060-2469-4a07-95fe-2b76f5404bc1","created":"2023-11-26T07:17:28.613663+0000","modified":"2023-11-26T07:22:37.219794+0000","last_up_change":"2023-11-26T07:17:41.157813+0000","last_in_change":"2023-11-26T07:17:31.181658+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":11,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-11-26T07:17:41.758597+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":10,"pool_name":"cephfs_metadata","create_time":"2023-11-26T07:22:35.314819+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"70","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":2.5,"score_stable":2.5,"optimal_score":1,"raw_score_acting":2.5,"raw_score_stable":2.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":11,"pool_name":"cephfs_data","create_time":"2023-11-26T07:22:36.219536+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"70","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"ebbee2c6-023e-44b9-9f1d-6f7bfd309730","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":69,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6817","nonce":639807369},{"type":"v1","addr":"172.21.15.143:6819","nonce":639807369}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6821","nonce":639807369},{"type":"v1","addr":"172.21.15.143:6823","nonce":639807369}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6830","nonce":639807369},{"type":"v1","addr":"172.21.15.143:6831","nonce":639807369}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6825","nonce":639807369},{"type":"v1","addr":"172.21.15.143:6828","nonce":639807369}]},"public_addr":"172.21.15.143:6819/639807369","cluster_addr":"172.21.15.143:6823/639807369","heartbeat_back_addr":"172.21.15.143:6831/639807369","heartbeat_front_addr":"172.21.15.143:6828/639807369","state":["exists","up"]},{"osd":1,"uuid":"fec4b8be-7122-49e9-9a19-5850a0700001","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":69,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6816","nonce":1144942530},{"type":"v1","addr":"172.21.15.143:6818","nonce":1144942530}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6820","nonce":1144942530},{"type":"v1","addr":"172.21.15.143:6822","nonce":1144942530}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6827","nonce":1144942530},{"type":"v1","addr":"172.21.15.143:6829","nonce":1144942530}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6824","nonce":1144942530},{"type":"v1","addr":"172.21.15.143:6826","nonce":1144942530}]},"public_addr":"172.21.15.143:6818/1144942530","cluster_addr":"172.21.15.143:6822/1144942530","heartbeat_back_addr":"172.21.15.143:6829/1144942530","heartbeat_front_addr":"172.21.15.143:6826/1144942530","state":["exists","up"]},{"osd":2,"uuid":"2888af5b-f933-46ca-8f95-6dd494ff43f6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":69,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6808","nonce":3909095186},{"type":"v1","addr":"172.21.15.143:6809","nonce":3909095186}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6810","nonce":3909095186},{"type":"v1","addr":"172.21.15.143:6811","nonce":3909095186}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6814","nonce":3909095186},{"type":"v1","addr":"172.21.15.143:6815","nonce":3909095186}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6812","nonce":3909095186},{"type":"v1","addr":"172.21.15.143:6813","nonce":3909095186}]},"public_addr":"172.21.15.143:6809/3909095186","cluster_addr":"172.21.15.143:6811/3909095186","heartbeat_back_addr":"172.21.15.143:6815/3909095186","heartbeat_front_addr":"172.21.15.143:6813/3909095186","state":["exists","up"]},{"osd":3,"uuid":"d1f52435-7bc1-4081-9550-9cd1ba55f549","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":69,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6800","nonce":3854210991},{"type":"v1","addr":"172.21.15.143:6801","nonce":3854210991}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6802","nonce":3854210991},{"type":"v1","addr":"172.21.15.143:6803","nonce":3854210991}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6806","nonce":3854210991},{"type":"v1","addr":"172.21.15.143:6807","nonce":3854210991}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6804","nonce":3854210991},{"type":"v1","addr":"172.21.15.143:6805","nonce":3854210991}]},"public_addr":"172.21.15.143:6801/3854210991","cluster_addr":"172.21.15.143:6803/3854210991","heartbeat_back_addr":"172.21.15.143:6807/3854210991","heartbeat_front_addr":"172.21.15.143:6805/3854210991","state":["exists","up"]},{"osd":4,"uuid":"e2ae8b6e-6e25-49b2-bcbf-aacce68f06a3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":69,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6800","nonce":3924606902},{"type":"v1","addr":"172.21.15.188:6801","nonce":3924606902}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6802","nonce":3924606902},{"type":"v1","addr":"172.21.15.188:6803","nonce":3924606902}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6806","nonce":3924606902},{"type":"v1","addr":"172.21.15.188:6807","nonce":3924606902}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6804","nonce":3924606902},{"type":"v1","addr":"172.21.15.188:6805","nonce":3924606902}]},"public_addr":"172.21.15.188:6801/3924606902","cluster_addr":"172.21.15.188:6803/3924606902","heartbeat_back_addr":"172.21.15.188:6807/3924606902","heartbeat_front_addr":"172.21.15.188:6805/3924606902","state":["exists","up"]},{"osd":5,"uuid":"3cec8a32-7698-4c36-8e89-e4ba11934f33","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":69,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6813","nonce":186841544},{"type":"v1","addr":"172.21.15.188:6815","nonce":186841544}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6817","nonce":186841544},{"type":"v1","addr":"172.21.15.188:6818","nonce":186841544}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6822","nonce":186841544},{"type":"v1","addr":"172.21.15.188:6823","nonce":186841544}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6820","nonce":186841544},{"type":"v1","addr":"172.21.15.188:6821","nonce":186841544}]},"public_addr":"172.21.15.188:6815/186841544","cluster_addr":"172.21.15.188:6818/186841544","heartbeat_back_addr":"172.21.15.188:6823/186841544","heartbeat_front_addr":"172.21.15.188:6821/186841544","state":["exists","up"]},{"osd":6,"uuid":"58a65d5d-e090-473a-81b4-ad38e85029f9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":69,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6808","nonce":2153499405},{"type":"v1","addr":"172.21.15.188:6809","nonce":2153499405}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6810","nonce":2153499405},{"type":"v1","addr":"172.21.15.188:6811","nonce":2153499405}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6816","nonce":2153499405},{"type":"v1","addr":"172.21.15.188:6819","nonce":2153499405}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6812","nonce":2153499405},{"type":"v1","addr":"172.21.15.188:6814","nonce":2153499405}]},"public_addr":"172.21.15.188:6809/2153499405","cluster_addr":"172.21.15.188:6811/2153499405","heartbeat_back_addr":"172.21.15.188:6819/2153499405","heartbeat_front_addr":"172.21.15.188:6814/2153499405","state":["exists","up"]},{"osd":7,"uuid":"ea5b7f70-3554-4cf8-ac30-f7c59b80678d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":69,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6824","nonce":2722096729},{"type":"v1","addr":"172.21.15.188:6825","nonce":2722096729}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6826","nonce":2722096729},{"type":"v1","addr":"172.21.15.188:6827","nonce":2722096729}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6830","nonce":2722096729},{"type":"v1","addr":"172.21.15.188:6831","nonce":2722096729}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6828","nonce":2722096729},{"type":"v1","addr":"172.21.15.188:6829","nonce":2722096729}]},"public_addr":"172.21.15.188:6825/2722096729","cluster_addr":"172.21.15.188:6827/2722096729","heartbeat_back_addr":"172.21.15.188:6831/2722096729","heartbeat_front_addr":"172.21.15.188:6829/2722096729","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-26T07:17:36.995032+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-26T07:17:36.932266+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-26T07:17:36.829304+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-26T07:17:36.804297+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-26T07:17:36.667892+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-26T07:17:36.964868+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-26T07:17:36.827332+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-26T07:17:36.893179+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-11-26T07:22:38.209 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:38.208+0000 7f81e2b01640 1 -- 172.21.15.143:0/1664653920 >> [v2:172.21.15.143:6832/2150557544,v1:172.21.15.143:6833/2150557544] conn(0x7f81c00667d0 msgr2=0x7f81c0068c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:22:38.210 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:38.208+0000 7f81e2b01640 1 --2- 172.21.15.143:0/1664653920 >> [v2:172.21.15.143:6832/2150557544,v1:172.21.15.143:6833/2150557544] conn(0x7f81c00667d0 0x7f81c0068c90 secure :-1 s=READY pgs=446 cs=0 l=1 rev1=1 crypto rx=0x7f81c80ae000 tx=0x7f81c80b3040 comp rx=0 tx=0).stop 2023-11-26T07:22:38.210 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:38.208+0000 7f81e2b01640 1 -- 172.21.15.143:0/1664653920 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f81dc0cd1b0 msgr2=0x7f81dc154c60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:22:38.210 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:38.208+0000 7f81e2b01640 1 --2- 172.21.15.143:0/1664653920 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f81dc0cd1b0 0x7f81dc154c60 secure :-1 s=READY pgs=948 cs=0 l=1 rev1=1 crypto rx=0x7f81d000c460 tx=0x7f81d000c860 comp rx=0 tx=0).stop 2023-11-26T07:22:38.210 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:38.208+0000 7f81e2b01640 1 -- 172.21.15.143:0/1664653920 shutdown_connections 2023-11-26T07:22:38.210 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:38.208+0000 7f81e2b01640 1 --2- 172.21.15.143:0/1664653920 >> [v2:172.21.15.143:6832/2150557544,v1:172.21.15.143:6833/2150557544] conn(0x7f81c00667d0 0x7f81c0068c90 unknown :-1 s=CLOSED pgs=446 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:22:38.210 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:38.208+0000 7f81e2b01640 1 --2- 172.21.15.143:0/1664653920 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f81dc143ac0 0x7f81dc0b3370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:22:38.210 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:38.208+0000 7f81e2b01640 1 --2- 172.21.15.143:0/1664653920 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f81dc0cf280 0x7f81dc12e0a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:22:38.210 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:38.208+0000 7f81e2b01640 1 --2- 172.21.15.143:0/1664653920 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f81dc0cd1b0 0x7f81dc154c60 unknown :-1 s=CLOSED pgs=948 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:22:38.210 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:38.208+0000 7f81e2b01640 1 -- 172.21.15.143:0/1664653920 >> 172.21.15.143:0/1664653920 conn(0x7f81dc06a890 msgr2=0x7f81dc15bee0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:22:38.210 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:38.208+0000 7f81e2b01640 1 -- 172.21.15.143:0/1664653920 shutdown_connections 2023-11-26T07:22:38.211 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:38.208+0000 7f81e2b01640 1 -- 172.21.15.143:0/1664653920 wait complete. 2023-11-26T07:22:38.229 DEBUG:teuthology.orchestra.run.smithi143:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-11-26T07:22:38.366 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:38.364+0000 7fc0179c4640 1 Processor -- start 2023-11-26T07:22:38.366 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:38.364+0000 7fc0179c4640 1 -- start start 2023-11-26T07:22:38.368 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:38.368+0000 7fc0179c4640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fc010143ac0 0x7fc010070370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:22:38.368 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:38.368+0000 7fc0179c4640 1 -- --> v1:172.21.15.143:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc0100cc730 con 0x7fc0100cf990 2023-11-26T07:22:38.368 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:38.368+0000 7fc0179c4640 1 -- --> v1:172.21.15.188:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc010070ee0 con 0x7fc0100708b0 2023-11-26T07:22:38.368 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:38.368+0000 7fc0179c4640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc010143ea0 con 0x7fc010143ac0 2023-11-26T07:22:38.368 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:38.368+0000 7fc015f61640 1 --1- >> v1:172.21.15.143:6789/0 conn(0x7fc0100cf990 0x7fc0100cd1b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.143:6789/0 says I am v1:172.21.15.143:35754/0 (socket says 172.21.15.143:35754) 2023-11-26T07:22:38.368 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:38.368+0000 7fc015f61640 1 -- 172.21.15.143:0/2523134494 learned_addr learned my addr 172.21.15.143:0/2523134494 (peer_addr_for_me v1:172.21.15.143:0/0) 2023-11-26T07:22:38.369 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:38.368+0000 7fc015760640 1 --2- 172.21.15.143:0/2523134494 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fc010143ac0 0x7fc010070370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:22:38.369 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:38.368+0000 7fc007fff640 1 -- 172.21.15.143:0/2523134494 <== mon.0 v1:172.21.15.143:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 544022676 0 0) 0x7fc0100cc730 con 0x7fc0100cf990 2023-11-26T07:22:38.369 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:38.368+0000 7fc007fff640 1 -- 172.21.15.143:0/2523134494 --> v1:172.21.15.143:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fbffc005740 con 0x7fc0100cf990 2023-11-26T07:22:38.370 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:38.368+0000 7fc007fff640 1 -- 172.21.15.143:0/2523134494 <== mon.1 v1:172.21.15.188:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2949530 0 0) 0x7fc010070ee0 con 0x7fc0100708b0 2023-11-26T07:22:38.370 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:38.368+0000 7fc007fff640 1 -- 172.21.15.143:0/2523134494 --> v1:172.21.15.188:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fc0100cc730 con 0x7fc0100708b0 2023-11-26T07:22:38.370 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:38.368+0000 7fc015760640 1 -- 172.21.15.143:0/2523134494 >> v1:172.21.15.188:6789/0 conn(0x7fc0100708b0 legacy=0x7fc0100cf280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:22:38.370 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:38.368+0000 7fc015760640 1 -- 172.21.15.143:0/2523134494 >> v1:172.21.15.143:6789/0 conn(0x7fc0100cf990 legacy=0x7fc0100cd1b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:22:38.370 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:38.368+0000 7fc015760640 1 -- 172.21.15.143:0/2523134494 --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc0100711c0 con 0x7fc010143ac0 2023-11-26T07:22:38.372 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:38.372+0000 7fc015760640 1 --2- 172.21.15.143:0/2523134494 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fc010143ac0 0x7fc010070370 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7fc00c0097f0 tx=0x7fc00c01c4c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=c7b3e3d2c798316 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:22:38.372 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:38.372+0000 7fc007fff640 1 -- 172.21.15.143:0/2523134494 <== mon.2 v2:172.21.15.188:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc00c0ac630 con 0x7fc010143ac0 2023-11-26T07:22:38.372 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:38.372+0000 7fc007fff640 1 -- 172.21.15.143:0/2523134494 <== mon.2 v2:172.21.15.188:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc00c0ac790 con 0x7fc010143ac0 2023-11-26T07:22:38.372 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:38.372+0000 7fc007fff640 1 -- 172.21.15.143:0/2523134494 <== mon.2 v2:172.21.15.188:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc00c0acb20 con 0x7fc010143ac0 2023-11-26T07:22:38.373 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:38.372+0000 7fc0179c4640 1 -- 172.21.15.143:0/2523134494 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fc010143ac0 msgr2=0x7fc010070370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:22:38.373 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:38.372+0000 7fc0179c4640 1 --2- 172.21.15.143:0/2523134494 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fc010143ac0 0x7fc010070370 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7fc00c0097f0 tx=0x7fc00c01c4c0 comp rx=0 tx=0).stop 2023-11-26T07:22:38.373 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:38.372+0000 7fc0179c4640 1 -- 172.21.15.143:0/2523134494 shutdown_connections 2023-11-26T07:22:38.374 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:38.372+0000 7fc0179c4640 1 --2- 172.21.15.143:0/2523134494 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fc010143ac0 0x7fc010070370 unknown :-1 s=CLOSED pgs=123 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:22:38.374 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:38.372+0000 7fc0179c4640 1 -- 172.21.15.143:0/2523134494 >> 172.21.15.143:0/2523134494 conn(0x7fc01006a890 msgr2=0x7fc01006aca0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:22:38.374 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:38.372+0000 7fc0179c4640 1 -- 172.21.15.143:0/2523134494 shutdown_connections 2023-11-26T07:22:38.374 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:38.372+0000 7fc0179c4640 1 -- 172.21.15.143:0/2523134494 wait complete. 2023-11-26T07:22:38.375 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:38.372+0000 7fc0179c4640 1 Processor -- start 2023-11-26T07:22:38.375 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:38.372+0000 7fc0179c4640 1 -- start start 2023-11-26T07:22:38.377 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:38.376+0000 7fc0179c4640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fc0100708b0 0x7fc01015b7d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:22:38.377 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:38.376+0000 7fc0179c4640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fc0100cf990 0x7fc0100c4ec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:22:38.377 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:38.376+0000 7fc0179c4640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fc010143ac0 0x7fc01015cec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:22:38.378 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:38.376+0000 7fc0179c4640 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc01015be20 con 0x7fc010143ac0 2023-11-26T07:22:38.378 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:38.376+0000 7fc0179c4640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc01015c100 con 0x7fc0100cf990 2023-11-26T07:22:38.378 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:38.376+0000 7fc0179c4640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc01015c3e0 con 0x7fc0100708b0 2023-11-26T07:22:38.378 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:38.376+0000 7fc015f61640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fc010143ac0 0x7fc01015cec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:22:38.378 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:38.376+0000 7fc015f61640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fc010143ac0 0x7fc01015cec0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:47190/0 (socket says 172.21.15.143:47190) 2023-11-26T07:22:38.378 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:38.376+0000 7fc015760640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fc0100708b0 0x7fc01015b7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:22:38.378 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:38.376+0000 7fc015f61640 1 -- 172.21.15.143:0/1651546968 learned_addr learned my addr 172.21.15.143:0/1651546968 (peer_addr_for_me v2:172.21.15.143:0/0) 2023-11-26T07:22:38.378 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:38.376+0000 7fc014f5f640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fc0100cf990 0x7fc0100c4ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:22:38.378 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:38.376+0000 7fc015f61640 1 -- 172.21.15.143:0/1651546968 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fc0100708b0 msgr2=0x7fc01015b7d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:22:38.379 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:38.376+0000 7fc015f61640 1 --2- 172.21.15.143:0/1651546968 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fc0100708b0 0x7fc01015b7d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:22:38.379 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:38.376+0000 7fc015f61640 1 -- 172.21.15.143:0/1651546968 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fc0100cf990 msgr2=0x7fc0100c4ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:22:38.379 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:38.376+0000 7fc015f61640 1 --2- 172.21.15.143:0/1651546968 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fc0100cf990 0x7fc0100c4ec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:22:38.379 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:38.376+0000 7fc015f61640 1 -- 172.21.15.143:0/1651546968 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc00c009420 con 0x7fc010143ac0 2023-11-26T07:22:38.379 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:38.376+0000 7fc015f61640 1 --2- 172.21.15.143:0/1651546968 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fc010143ac0 0x7fc01015cec0 secure :-1 s=READY pgs=950 cs=0 l=1 rev1=1 crypto rx=0x7fc00800cbb0 tx=0x7fc00800a060 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:22:38.379 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:38.376+0000 7fc005ffb640 1 -- 172.21.15.143:0/1651546968 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc00800aca0 con 0x7fc010143ac0 2023-11-26T07:22:38.379 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:38.376+0000 7fc005ffb640 1 -- 172.21.15.143:0/1651546968 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc00800ae00 con 0x7fc010143ac0 2023-11-26T07:22:38.379 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:38.376+0000 7fc005ffb640 1 -- 172.21.15.143:0/1651546968 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc008012640 con 0x7fc010143ac0 2023-11-26T07:22:38.379 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:38.376+0000 7fc0179c4640 1 -- 172.21.15.143:0/1651546968 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc01015c6c0 con 0x7fc010143ac0 2023-11-26T07:22:38.379 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:38.376+0000 7fc0179c4640 1 -- 172.21.15.143:0/1651546968 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc0100c5400 con 0x7fc010143ac0 2023-11-26T07:22:38.380 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:38.376+0000 7fc0179c4640 1 -- 172.21.15.143:0/1651546968 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc010067680 con 0x7fc010143ac0 2023-11-26T07:22:38.380 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:38.376+0000 7fc005ffb640 1 -- 172.21.15.143:0/1651546968 <== mon.0 v2:172.21.15.143:3300/0 4 ==== mgrmap(e 11) v1 ==== 86158+0+0 (secure 0 0 0) 0x7fc008007070 con 0x7fc010143ac0 2023-11-26T07:22:38.380 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:38.380+0000 7fc005ffb640 1 --2- 172.21.15.143:0/1651546968 >> [v2:172.21.15.143:6832/2150557544,v1:172.21.15.143:6833/2150557544] conn(0x7fbff40667d0 0x7fbff4068c90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:22:38.380 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:38.380+0000 7fc005ffb640 1 -- 172.21.15.143:0/1651546968 <== mon.0 v2:172.21.15.143:3300/0 5 ==== osd_map(70..70 src has 1..70) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fc008084330 con 0x7fc010143ac0 2023-11-26T07:22:38.380 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:38.380+0000 7fc015760640 1 --2- 172.21.15.143:0/1651546968 >> [v2:172.21.15.143:6832/2150557544,v1:172.21.15.143:6833/2150557544] conn(0x7fbff40667d0 0x7fbff4068c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:22:38.381 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:38.380+0000 7fc015760640 1 --2- 172.21.15.143:0/1651546968 >> [v2:172.21.15.143:6832/2150557544,v1:172.21.15.143:6833/2150557544] conn(0x7fbff40667d0 0x7fbff4068c90 secure :-1 s=READY pgs=447 cs=0 l=1 rev1=1 crypto rx=0x7fc00c0ae000 tx=0x7fc00c0a7000 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:22:38.384 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:38.384+0000 7fc005ffb640 1 -- 172.21.15.143:0/1651546968 <== mon.0 v2:172.21.15.143:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172649 (secure 0 0 0) 0x7fc008051410 con 0x7fc010143ac0 2023-11-26T07:22:38.541 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:38.540+0000 7fc0179c4640 1 -- 172.21.15.143:0/1651546968 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fc0100cd1b0 con 0x7fc010143ac0 2023-11-26T07:22:38.542 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:38.540+0000 7fc005ffb640 1 -- 172.21.15.143:0/1651546968 <== mon.0 v2:172.21.15.143:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v70) v1 ==== 74+0+16617 (secure 0 0 0) 0x7fc008001bd0 con 0x7fc010143ac0 2023-11-26T07:22:38.542 INFO:teuthology.orchestra.run.smithi143.stdout: 2023-11-26T07:22:38.543 INFO:teuthology.orchestra.run.smithi143.stdout:{"epoch":70,"fsid":"f5033060-2469-4a07-95fe-2b76f5404bc1","created":"2023-11-26T07:17:28.613663+0000","modified":"2023-11-26T07:22:37.219794+0000","last_up_change":"2023-11-26T07:17:41.157813+0000","last_in_change":"2023-11-26T07:17:31.181658+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":11,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-11-26T07:17:41.758597+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":10,"pool_name":"cephfs_metadata","create_time":"2023-11-26T07:22:35.314819+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"70","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":2.5,"score_stable":2.5,"optimal_score":1,"raw_score_acting":2.5,"raw_score_stable":2.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":11,"pool_name":"cephfs_data","create_time":"2023-11-26T07:22:36.219536+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"70","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"ebbee2c6-023e-44b9-9f1d-6f7bfd309730","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":69,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6817","nonce":639807369},{"type":"v1","addr":"172.21.15.143:6819","nonce":639807369}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6821","nonce":639807369},{"type":"v1","addr":"172.21.15.143:6823","nonce":639807369}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6830","nonce":639807369},{"type":"v1","addr":"172.21.15.143:6831","nonce":639807369}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6825","nonce":639807369},{"type":"v1","addr":"172.21.15.143:6828","nonce":639807369}]},"public_addr":"172.21.15.143:6819/639807369","cluster_addr":"172.21.15.143:6823/639807369","heartbeat_back_addr":"172.21.15.143:6831/639807369","heartbeat_front_addr":"172.21.15.143:6828/639807369","state":["exists","up"]},{"osd":1,"uuid":"fec4b8be-7122-49e9-9a19-5850a0700001","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":69,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6816","nonce":1144942530},{"type":"v1","addr":"172.21.15.143:6818","nonce":1144942530}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6820","nonce":1144942530},{"type":"v1","addr":"172.21.15.143:6822","nonce":1144942530}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6827","nonce":1144942530},{"type":"v1","addr":"172.21.15.143:6829","nonce":1144942530}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6824","nonce":1144942530},{"type":"v1","addr":"172.21.15.143:6826","nonce":1144942530}]},"public_addr":"172.21.15.143:6818/1144942530","cluster_addr":"172.21.15.143:6822/1144942530","heartbeat_back_addr":"172.21.15.143:6829/1144942530","heartbeat_front_addr":"172.21.15.143:6826/1144942530","state":["exists","up"]},{"osd":2,"uuid":"2888af5b-f933-46ca-8f95-6dd494ff43f6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":69,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6808","nonce":3909095186},{"type":"v1","addr":"172.21.15.143:6809","nonce":3909095186}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6810","nonce":3909095186},{"type":"v1","addr":"172.21.15.143:6811","nonce":3909095186}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6814","nonce":3909095186},{"type":"v1","addr":"172.21.15.143:6815","nonce":3909095186}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6812","nonce":3909095186},{"type":"v1","addr":"172.21.15.143:6813","nonce":3909095186}]},"public_addr":"172.21.15.143:6809/3909095186","cluster_addr":"172.21.15.143:6811/3909095186","heartbeat_back_addr":"172.21.15.143:6815/3909095186","heartbeat_front_addr":"172.21.15.143:6813/3909095186","state":["exists","up"]},{"osd":3,"uuid":"d1f52435-7bc1-4081-9550-9cd1ba55f549","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":69,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6800","nonce":3854210991},{"type":"v1","addr":"172.21.15.143:6801","nonce":3854210991}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6802","nonce":3854210991},{"type":"v1","addr":"172.21.15.143:6803","nonce":3854210991}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6806","nonce":3854210991},{"type":"v1","addr":"172.21.15.143:6807","nonce":3854210991}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6804","nonce":3854210991},{"type":"v1","addr":"172.21.15.143:6805","nonce":3854210991}]},"public_addr":"172.21.15.143:6801/3854210991","cluster_addr":"172.21.15.143:6803/3854210991","heartbeat_back_addr":"172.21.15.143:6807/3854210991","heartbeat_front_addr":"172.21.15.143:6805/3854210991","state":["exists","up"]},{"osd":4,"uuid":"e2ae8b6e-6e25-49b2-bcbf-aacce68f06a3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":69,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6800","nonce":3924606902},{"type":"v1","addr":"172.21.15.188:6801","nonce":3924606902}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6802","nonce":3924606902},{"type":"v1","addr":"172.21.15.188:6803","nonce":3924606902}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6806","nonce":3924606902},{"type":"v1","addr":"172.21.15.188:6807","nonce":3924606902}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6804","nonce":3924606902},{"type":"v1","addr":"172.21.15.188:6805","nonce":3924606902}]},"public_addr":"172.21.15.188:6801/3924606902","cluster_addr":"172.21.15.188:6803/3924606902","heartbeat_back_addr":"172.21.15.188:6807/3924606902","heartbeat_front_addr":"172.21.15.188:6805/3924606902","state":["exists","up"]},{"osd":5,"uuid":"3cec8a32-7698-4c36-8e89-e4ba11934f33","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":69,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6813","nonce":186841544},{"type":"v1","addr":"172.21.15.188:6815","nonce":186841544}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6817","nonce":186841544},{"type":"v1","addr":"172.21.15.188:6818","nonce":186841544}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6822","nonce":186841544},{"type":"v1","addr":"172.21.15.188:6823","nonce":186841544}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6820","nonce":186841544},{"type":"v1","addr":"172.21.15.188:6821","nonce":186841544}]},"public_addr":"172.21.15.188:6815/186841544","cluster_addr":"172.21.15.188:6818/186841544","heartbeat_back_addr":"172.21.15.188:6823/186841544","heartbeat_front_addr":"172.21.15.188:6821/186841544","state":["exists","up"]},{"osd":6,"uuid":"58a65d5d-e090-473a-81b4-ad38e85029f9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":69,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6808","nonce":2153499405},{"type":"v1","addr":"172.21.15.188:6809","nonce":2153499405}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6810","nonce":2153499405},{"type":"v1","addr":"172.21.15.188:6811","nonce":2153499405}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6816","nonce":2153499405},{"type":"v1","addr":"172.21.15.188:6819","nonce":2153499405}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6812","nonce":2153499405},{"type":"v1","addr":"172.21.15.188:6814","nonce":2153499405}]},"public_addr":"172.21.15.188:6809/2153499405","cluster_addr":"172.21.15.188:6811/2153499405","heartbeat_back_addr":"172.21.15.188:6819/2153499405","heartbeat_front_addr":"172.21.15.188:6814/2153499405","state":["exists","up"]},{"osd":7,"uuid":"ea5b7f70-3554-4cf8-ac30-f7c59b80678d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":69,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6824","nonce":2722096729},{"type":"v1","addr":"172.21.15.188:6825","nonce":2722096729}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6826","nonce":2722096729},{"type":"v1","addr":"172.21.15.188:6827","nonce":2722096729}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6830","nonce":2722096729},{"type":"v1","addr":"172.21.15.188:6831","nonce":2722096729}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6828","nonce":2722096729},{"type":"v1","addr":"172.21.15.188:6829","nonce":2722096729}]},"public_addr":"172.21.15.188:6825/2722096729","cluster_addr":"172.21.15.188:6827/2722096729","heartbeat_back_addr":"172.21.15.188:6831/2722096729","heartbeat_front_addr":"172.21.15.188:6829/2722096729","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-26T07:17:36.995032+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-26T07:17:36.932266+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-26T07:17:36.829304+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-26T07:17:36.804297+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-26T07:17:36.667892+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-26T07:17:36.964868+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-26T07:17:36.827332+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-26T07:17:36.893179+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-11-26T07:22:38.545 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:38.544+0000 7fc0179c4640 1 -- 172.21.15.143:0/1651546968 >> [v2:172.21.15.143:6832/2150557544,v1:172.21.15.143:6833/2150557544] conn(0x7fbff40667d0 msgr2=0x7fbff4068c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:22:38.545 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:38.544+0000 7fc0179c4640 1 --2- 172.21.15.143:0/1651546968 >> [v2:172.21.15.143:6832/2150557544,v1:172.21.15.143:6833/2150557544] conn(0x7fbff40667d0 0x7fbff4068c90 secure :-1 s=READY pgs=447 cs=0 l=1 rev1=1 crypto rx=0x7fc00c0ae000 tx=0x7fc00c0a7000 comp rx=0 tx=0).stop 2023-11-26T07:22:38.545 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:38.544+0000 7fc0179c4640 1 -- 172.21.15.143:0/1651546968 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fc010143ac0 msgr2=0x7fc01015cec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:22:38.545 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:38.544+0000 7fc0179c4640 1 --2- 172.21.15.143:0/1651546968 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fc010143ac0 0x7fc01015cec0 secure :-1 s=READY pgs=950 cs=0 l=1 rev1=1 crypto rx=0x7fc00800cbb0 tx=0x7fc00800a060 comp rx=0 tx=0).stop 2023-11-26T07:22:38.545 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:38.544+0000 7fc0179c4640 1 -- 172.21.15.143:0/1651546968 shutdown_connections 2023-11-26T07:22:38.545 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:38.544+0000 7fc0179c4640 1 --2- 172.21.15.143:0/1651546968 >> [v2:172.21.15.143:6832/2150557544,v1:172.21.15.143:6833/2150557544] conn(0x7fbff40667d0 0x7fbff4068c90 unknown :-1 s=CLOSED pgs=447 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:22:38.545 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:38.544+0000 7fc0179c4640 1 --2- 172.21.15.143:0/1651546968 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fc010143ac0 0x7fc01015cec0 unknown :-1 s=CLOSED pgs=950 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:22:38.545 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:38.544+0000 7fc0179c4640 1 --2- 172.21.15.143:0/1651546968 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fc0100cf990 0x7fc0100c4ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:22:38.545 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:38.544+0000 7fc0179c4640 1 --2- 172.21.15.143:0/1651546968 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fc0100708b0 0x7fc01015b7d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:22:38.545 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:38.544+0000 7fc0179c4640 1 -- 172.21.15.143:0/1651546968 >> 172.21.15.143:0/1651546968 conn(0x7fc01006a890 msgr2=0x7fc01006ce40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:22:38.545 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:38.544+0000 7fc0179c4640 1 -- 172.21.15.143:0/1651546968 shutdown_connections 2023-11-26T07:22:38.546 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:38.544+0000 7fc0179c4640 1 -- 172.21.15.143:0/1651546968 wait complete. 2023-11-26T07:22:38.564 DEBUG:teuthology.orchestra.run.smithi143:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs standby_count_wanted 0 2023-11-26T07:22:38.704 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:38.704+0000 7f418e101640 1 Processor -- start 2023-11-26T07:22:38.704 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:38.704+0000 7f418e101640 1 -- start start 2023-11-26T07:22:38.705 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:38.704+0000 7f418e101640 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7f4188143ac0 0x7f4188070370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:22:38.706 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:38.704+0000 7f418e101640 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7f41880708b0 0x7f41880cf280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:22:38.706 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:38.704+0000 7f418e101640 1 -- --> v1:172.21.15.188:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f41880cc730 con 0x7f41880cf7c0 2023-11-26T07:22:38.706 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:38.704+0000 7f418e101640 1 -- --> v2:172.21.15.143:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4188143ea0 con 0x7f4188143ac0 2023-11-26T07:22:38.706 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:38.704+0000 7f418e101640 1 -- --> v2:172.21.15.188:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f41880cfbd0 con 0x7f41880708b0 2023-11-26T07:22:38.707 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:38.704+0000 7f41877fe640 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7f4188143ac0 0x7f4188070370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:22:38.707 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:38.704+0000 7f41877fe640 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7f4188143ac0 0x7f4188070370 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:47200/0 (socket says 172.21.15.143:47200) 2023-11-26T07:22:38.707 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:38.704+0000 7f41877fe640 1 -- 172.21.15.143:0/1815875932 learned_addr learned my addr 172.21.15.143:0/1815875932 (peer_addr_for_me v2:172.21.15.143:0/0) 2023-11-26T07:22:38.707 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:38.704+0000 7f417ffff640 1 --2- 172.21.15.143:0/1815875932 >> v2:172.21.15.188:3300/0 conn(0x7f41880708b0 0x7f41880cf280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:22:38.707 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:38.704+0000 7f41877fe640 1 -- 172.21.15.143:0/1815875932 >> v2:172.21.15.188:3300/0 conn(0x7f41880708b0 msgr2=0x7f41880cf280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-26T07:22:38.707 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:38.704+0000 7f41877fe640 1 --2- 172.21.15.143:0/1815875932 >> v2:172.21.15.188:3300/0 conn(0x7f41880708b0 0x7f41880cf280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:22:38.707 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:38.704+0000 7f41877fe640 1 -- 172.21.15.143:0/1815875932 >> v1:172.21.15.188:6789/0 conn(0x7f41880cf7c0 legacy=0x7f41880b1ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-26T07:22:38.707 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:38.704+0000 7f41877fe640 1 -- 172.21.15.143:0/1815875932 --> v2:172.21.15.143:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4188070ee0 con 0x7f4188143ac0 2023-11-26T07:22:38.709 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:38.708+0000 7f41877fe640 1 --2- 172.21.15.143:0/1815875932 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f4188143ac0 0x7f4188070370 secure :-1 s=READY pgs=951 cs=0 l=1 rev1=1 crypto rx=0x7f41880aeb10 tx=0x7f417801c4c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3bbb96b04256a117 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:22:38.710 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:38.708+0000 7f4186ffd640 1 -- 172.21.15.143:0/1815875932 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f41780ac630 con 0x7f4188143ac0 2023-11-26T07:22:38.710 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:38.708+0000 7f4186ffd640 1 -- 172.21.15.143:0/1815875932 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f41780ac790 con 0x7f4188143ac0 2023-11-26T07:22:38.710 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:38.708+0000 7f4186ffd640 1 -- 172.21.15.143:0/1815875932 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f41780acb20 con 0x7f4188143ac0 2023-11-26T07:22:38.711 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:38.708+0000 7f418e101640 1 -- 172.21.15.143:0/1815875932 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f4188143ac0 msgr2=0x7f4188070370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:22:38.711 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:38.708+0000 7f418e101640 1 --2- 172.21.15.143:0/1815875932 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f4188143ac0 0x7f4188070370 secure :-1 s=READY pgs=951 cs=0 l=1 rev1=1 crypto rx=0x7f41880aeb10 tx=0x7f417801c4c0 comp rx=0 tx=0).stop 2023-11-26T07:22:38.711 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:38.708+0000 7f418e101640 1 -- 172.21.15.143:0/1815875932 shutdown_connections 2023-11-26T07:22:38.711 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:38.708+0000 7f418e101640 1 --2- 172.21.15.143:0/1815875932 >> v2:172.21.15.188:3300/0 conn(0x7f41880708b0 0x7f41880cf280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:22:38.711 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:38.708+0000 7f418e101640 1 --2- 172.21.15.143:0/1815875932 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f4188143ac0 0x7f4188070370 unknown :-1 s=CLOSED pgs=951 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:22:38.711 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:38.708+0000 7f418e101640 1 -- 172.21.15.143:0/1815875932 >> 172.21.15.143:0/1815875932 conn(0x7f418806a890 msgr2=0x7f418806aca0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:22:38.711 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:38.708+0000 7f418e101640 1 -- 172.21.15.143:0/1815875932 shutdown_connections 2023-11-26T07:22:38.711 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:38.708+0000 7f418e101640 1 -- 172.21.15.143:0/1815875932 wait complete. 2023-11-26T07:22:38.712 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:38.712+0000 7f418e101640 1 Processor -- start 2023-11-26T07:22:38.713 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:38.712+0000 7f418e101640 1 -- start start 2023-11-26T07:22:38.715 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:38.712+0000 7f418e101640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f41880708b0 0x7f418813a330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:22:38.715 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:38.712+0000 7f418e101640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f41880cf7c0 0x7f41881574e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:22:38.715 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:38.712+0000 7f418e101640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f4188143ac0 0x7f41881594c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:22:38.715 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:38.712+0000 7f418e101640 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4188153760 con 0x7f4188143ac0 2023-11-26T07:22:38.715 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:38.712+0000 7f418e101640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f41881531a0 con 0x7f41880cf7c0 2023-11-26T07:22:38.715 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:38.712+0000 7f418e101640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4188153480 con 0x7f41880708b0 2023-11-26T07:22:38.716 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:38.712+0000 7f4187fff640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f4188143ac0 0x7f41881594c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:22:38.716 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:38.712+0000 7f4187fff640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f4188143ac0 0x7f41881594c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:47206/0 (socket says 172.21.15.143:47206) 2023-11-26T07:22:38.716 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:38.712+0000 7f4187fff640 1 -- 172.21.15.143:0/2381587347 learned_addr learned my addr 172.21.15.143:0/2381587347 (peer_addr_for_me v2:172.21.15.143:0/0) 2023-11-26T07:22:38.716 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:38.712+0000 7f41877fe640 1 --2- 172.21.15.143:0/2381587347 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f41880708b0 0x7f418813a330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:22:38.716 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:38.712+0000 7f417ffff640 1 --2- 172.21.15.143:0/2381587347 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f41880cf7c0 0x7f41881574e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:22:38.716 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:38.712+0000 7f4187fff640 1 -- 172.21.15.143:0/2381587347 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f41880708b0 msgr2=0x7f418813a330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:22:38.716 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:38.712+0000 7f4187fff640 1 --2- 172.21.15.143:0/2381587347 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f41880708b0 0x7f418813a330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:22:38.716 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:38.712+0000 7f4187fff640 1 -- 172.21.15.143:0/2381587347 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f41880cf7c0 msgr2=0x7f41881574e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:22:38.716 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:38.712+0000 7f4187fff640 1 --2- 172.21.15.143:0/2381587347 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f41880cf7c0 0x7f41881574e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:22:38.716 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:38.712+0000 7f4187fff640 1 -- 172.21.15.143:0/2381587347 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f41780094d0 con 0x7f4188143ac0 2023-11-26T07:22:38.716 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:38.712+0000 7f4187fff640 1 --2- 172.21.15.143:0/2381587347 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f4188143ac0 0x7f41881594c0 secure :-1 s=READY pgs=952 cs=0 l=1 rev1=1 crypto rx=0x7f418000c780 tx=0x7f418000cc40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:22:38.716 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:38.712+0000 7f4184ff9640 1 -- 172.21.15.143:0/2381587347 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4180004b70 con 0x7f4188143ac0 2023-11-26T07:22:38.717 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:38.712+0000 7f4184ff9640 1 -- 172.21.15.143:0/2381587347 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f418000ce90 con 0x7f4188143ac0 2023-11-26T07:22:38.717 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:38.712+0000 7f4184ff9640 1 -- 172.21.15.143:0/2381587347 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f418000a850 con 0x7f4188143ac0 2023-11-26T07:22:38.717 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:38.712+0000 7f418e101640 1 -- 172.21.15.143:0/2381587347 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f41880afdb0 con 0x7f4188143ac0 2023-11-26T07:22:38.717 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:38.712+0000 7f418e101640 1 -- 172.21.15.143:0/2381587347 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f418813a870 con 0x7f4188143ac0 2023-11-26T07:22:38.717 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:38.716+0000 7f418e101640 1 -- 172.21.15.143:0/2381587347 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4188165550 con 0x7f4188143ac0 2023-11-26T07:22:38.717 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:38.716+0000 7f4184ff9640 1 -- 172.21.15.143:0/2381587347 <== mon.0 v2:172.21.15.143:3300/0 4 ==== mgrmap(e 11) v1 ==== 86158+0+0 (secure 0 0 0) 0x7f418000a9b0 con 0x7f4188143ac0 2023-11-26T07:22:38.718 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:38.716+0000 7f4184ff9640 1 --2- 172.21.15.143:0/2381587347 >> [v2:172.21.15.143:6832/2150557544,v1:172.21.15.143:6833/2150557544] conn(0x7f4168066800 0x7f4168068cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:22:38.718 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:38.716+0000 7f4184ff9640 1 -- 172.21.15.143:0/2381587347 <== mon.0 v2:172.21.15.143:3300/0 5 ==== osd_map(70..70 src has 1..70) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f41800844b0 con 0x7f4188143ac0 2023-11-26T07:22:38.718 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:38.716+0000 7f41877fe640 1 --2- 172.21.15.143:0/2381587347 >> [v2:172.21.15.143:6832/2150557544,v1:172.21.15.143:6833/2150557544] conn(0x7f4168066800 0x7f4168068cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:22:38.719 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:38.716+0000 7f41877fe640 1 --2- 172.21.15.143:0/2381587347 >> [v2:172.21.15.143:6832/2150557544,v1:172.21.15.143:6833/2150557544] conn(0x7f4168066800 0x7f4168068cc0 secure :-1 s=READY pgs=448 cs=0 l=1 rev1=1 crypto rx=0x7f41780ae000 tx=0x7f41780a7000 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:22:38.722 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:38.720+0000 7f4184ff9640 1 -- 172.21.15.143:0/2381587347 <== mon.0 v2:172.21.15.143:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172649 (secure 0 0 0) 0x7f41800514d0 con 0x7f4188143ac0 2023-11-26T07:22:38.909 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:38.908+0000 7f418e101640 1 -- 172.21.15.143:0/2381587347 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"} v 0) v1 -- 0x7f4188067680 con 0x7f4188143ac0 2023-11-26T07:22:39.870 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:39.868+0000 7f4184ff9640 1 -- 172.21.15.143:0/2381587347 <== mon.0 v2:172.21.15.143:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]=0 v41) v1 ==== 118+0+0 (secure 0 0 0) 0x7f4180054af0 con 0x7f4188143ac0 2023-11-26T07:22:39.873 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:39.872+0000 7f418e101640 1 -- 172.21.15.143:0/2381587347 >> [v2:172.21.15.143:6832/2150557544,v1:172.21.15.143:6833/2150557544] conn(0x7f4168066800 msgr2=0x7f4168068cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:22:39.874 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:39.872+0000 7f418e101640 1 --2- 172.21.15.143:0/2381587347 >> [v2:172.21.15.143:6832/2150557544,v1:172.21.15.143:6833/2150557544] conn(0x7f4168066800 0x7f4168068cc0 secure :-1 s=READY pgs=448 cs=0 l=1 rev1=1 crypto rx=0x7f41780ae000 tx=0x7f41780a7000 comp rx=0 tx=0).stop 2023-11-26T07:22:39.874 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:39.872+0000 7f418e101640 1 -- 172.21.15.143:0/2381587347 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f4188143ac0 msgr2=0x7f41881594c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:22:39.874 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:39.872+0000 7f418e101640 1 --2- 172.21.15.143:0/2381587347 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f4188143ac0 0x7f41881594c0 secure :-1 s=READY pgs=952 cs=0 l=1 rev1=1 crypto rx=0x7f418000c780 tx=0x7f418000cc40 comp rx=0 tx=0).stop 2023-11-26T07:22:39.874 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:39.872+0000 7f418e101640 1 -- 172.21.15.143:0/2381587347 shutdown_connections 2023-11-26T07:22:39.874 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:39.872+0000 7f418e101640 1 --2- 172.21.15.143:0/2381587347 >> [v2:172.21.15.143:6832/2150557544,v1:172.21.15.143:6833/2150557544] conn(0x7f4168066800 0x7f4168068cc0 unknown :-1 s=CLOSED pgs=448 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:22:39.874 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:39.872+0000 7f418e101640 1 --2- 172.21.15.143:0/2381587347 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f4188143ac0 0x7f41881594c0 unknown :-1 s=CLOSED pgs=952 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:22:39.874 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:39.872+0000 7f418e101640 1 --2- 172.21.15.143:0/2381587347 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f41880cf7c0 0x7f41881574e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:22:39.874 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:39.872+0000 7f418e101640 1 --2- 172.21.15.143:0/2381587347 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f41880708b0 0x7f418813a330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:22:39.874 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:39.872+0000 7f418e101640 1 -- 172.21.15.143:0/2381587347 >> 172.21.15.143:0/2381587347 conn(0x7f418806a890 msgr2=0x7f41880b1cb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:22:39.875 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:39.872+0000 7f418e101640 1 -- 172.21.15.143:0/2381587347 shutdown_connections 2023-11-26T07:22:39.875 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:39.872+0000 7f418e101640 1 -- 172.21.15.143:0/2381587347 wait complete. 2023-11-26T07:22:39.896 DEBUG:tasks.cephfs.filesystem:fs_config: {} 2023-11-26T07:22:39.897 DEBUG:teuthology.orchestra.run.smithi143:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs allow_standby_replay false 2023-11-26T07:22:40.043 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:40.044+0000 7f72f478d640 1 Processor -- start 2023-11-26T07:22:40.044 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:40.044+0000 7f72f478d640 1 -- start start 2023-11-26T07:22:40.047 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:40.044+0000 7f72f478d640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f72ec0cf280 0x7f72ec0cf6e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:22:40.047 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:40.044+0000 7f72f478d640 1 -- --> v1:172.21.15.143:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f72ec0cfc20 con 0x7f72ec143ac0 2023-11-26T07:22:40.047 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:40.044+0000 7f72f478d640 1 -- --> v1:172.21.15.188:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f72ec0cc730 con 0x7f72ec0cd1b0 2023-11-26T07:22:40.047 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:40.044+0000 7f72f478d640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f72ec143ea0 con 0x7f72ec0cf280 2023-11-26T07:22:40.047 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:40.044+0000 7f72f2529640 1 --1- >> v1:172.21.15.143:6789/0 conn(0x7f72ec143ac0 0x7f72ec070370 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.143:6789/0 says I am v1:172.21.15.143:35758/0 (socket says 172.21.15.143:35758) 2023-11-26T07:22:40.048 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:40.044+0000 7f72f2529640 1 -- 172.21.15.143:0/176394699 learned_addr learned my addr 172.21.15.143:0/176394699 (peer_addr_for_me v1:172.21.15.143:0/0) 2023-11-26T07:22:40.048 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:40.044+0000 7f72f1527640 1 -- 172.21.15.143:0/176394699 <== mon.0 v1:172.21.15.143:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3441081881 0 0) 0x7f72ec0cfc20 con 0x7f72ec143ac0 2023-11-26T07:22:40.048 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:40.044+0000 7f72f1527640 1 -- 172.21.15.143:0/176394699 --> v1:172.21.15.143:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f72d8003530 con 0x7f72ec143ac0 2023-11-26T07:22:40.048 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:40.044+0000 7f72f1d28640 1 --2- 172.21.15.143:0/176394699 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f72ec0cf280 0x7f72ec0cf6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:22:40.048 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:40.044+0000 7f72f1527640 1 -- 172.21.15.143:0/176394699 <== mon.0 v1:172.21.15.143:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3069966258 0 0) 0x7f72d8003530 con 0x7f72ec143ac0 2023-11-26T07:22:40.048 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:40.044+0000 7f72f1527640 1 -- 172.21.15.143:0/176394699 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f72ec0cf280 msgr2=0x7f72ec0cf6e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-26T07:22:40.048 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:40.044+0000 7f72f1527640 1 --2- 172.21.15.143:0/176394699 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f72ec0cf280 0x7f72ec0cf6e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:22:40.048 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:40.044+0000 7f72f1527640 1 -- 172.21.15.143:0/176394699 >> v1:172.21.15.188:6789/0 conn(0x7f72ec0cd1b0 legacy=0x7f72ec0cd5e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:22:40.048 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:40.044+0000 7f72f1527640 1 -- 172.21.15.143:0/176394699 --> v1:172.21.15.143:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f72ec070ee0 con 0x7f72ec143ac0 2023-11-26T07:22:40.048 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:40.044+0000 7f72f1527640 1 -- 172.21.15.143:0/176394699 <== mon.0 v1:172.21.15.143:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2137882523 0 0) 0x7f72e8001680 con 0x7f72ec143ac0 2023-11-26T07:22:40.048 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:40.044+0000 7f72f1d28640 1 --2- 172.21.15.143:0/176394699 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f72ec0cf280 0x7f72ec0cf6e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-26T07:22:40.049 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:40.044+0000 7f72f1527640 1 -- 172.21.15.143:0/176394699 >> v1:172.21.15.143:6789/0 conn(0x7f72ec143ac0 legacy=0x7f72ec070370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:22:40.049 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:40.044+0000 7f72f1527640 1 --2- 172.21.15.143:0/176394699 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f72ec0cf280 0x7f72d80039b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:22:40.049 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:40.044+0000 7f72f1527640 1 --2- 172.21.15.143:0/176394699 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f72d8003fa0 0x7f72d80047d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:22:40.049 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:40.044+0000 7f72f1527640 1 --2- 172.21.15.143:0/176394699 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f72d8004d10 0x7f72d8005190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:22:40.049 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:40.044+0000 7f72f1527640 1 -- 172.21.15.143:0/176394699 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f72d800bac0 con 0x7f72d8003fa0 2023-11-26T07:22:40.049 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:40.044+0000 7f72f1527640 1 -- 172.21.15.143:0/176394699 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f72d800bc30 con 0x7f72d8004d10 2023-11-26T07:22:40.049 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:40.044+0000 7f72f1527640 1 -- 172.21.15.143:0/176394699 --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f72d800bdf0 con 0x7f72ec0cf280 2023-11-26T07:22:40.049 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:40.044+0000 7f72f1d28640 1 --2- 172.21.15.143:0/176394699 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f72d8003fa0 0x7f72d80047d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:22:40.049 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:40.044+0000 7f72f2529640 1 --2- 172.21.15.143:0/176394699 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f72ec0cf280 0x7f72d80039b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:22:40.049 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:40.044+0000 7f72f2d2a640 1 --2- 172.21.15.143:0/176394699 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f72d8004d10 0x7f72d8005190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:22:40.050 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:40.044+0000 7f72f1d28640 1 -- 172.21.15.143:0/176394699 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f72ec0cf280 msgr2=0x7f72d80039b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-26T07:22:40.050 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:40.044+0000 7f72f1d28640 1 --2- 172.21.15.143:0/176394699 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f72ec0cf280 0x7f72d80039b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:22:40.050 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:40.044+0000 7f72f1d28640 1 -- 172.21.15.143:0/176394699 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f72d8004d10 msgr2=0x7f72d8005190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-26T07:22:40.050 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:40.044+0000 7f72f1d28640 1 --2- 172.21.15.143:0/176394699 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f72d8004d10 0x7f72d8005190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:22:40.050 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:40.044+0000 7f72f1d28640 1 -- 172.21.15.143:0/176394699 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f72d8003530 con 0x7f72d8003fa0 2023-11-26T07:22:40.051 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:40.048+0000 7f72f1d28640 1 --2- 172.21.15.143:0/176394699 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f72d8003fa0 0x7f72d80047d0 secure :-1 s=READY pgs=954 cs=0 l=1 rev1=1 crypto rx=0x7f72dc009600 tx=0x7f72dc01e700 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ce81062137720cd9 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:22:40.051 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:40.048+0000 7f72f1527640 1 -- 172.21.15.143:0/176394699 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f72dc0a8620 con 0x7f72d8003fa0 2023-11-26T07:22:40.051 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:40.048+0000 7f72f1527640 1 -- 172.21.15.143:0/176394699 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f72dc0a8780 con 0x7f72d8003fa0 2023-11-26T07:22:40.051 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:40.048+0000 7f72f1527640 1 -- 172.21.15.143:0/176394699 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f72dc0a8af0 con 0x7f72d8003fa0 2023-11-26T07:22:40.051 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:40.048+0000 7f72f478d640 1 -- 172.21.15.143:0/176394699 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f72d8003fa0 msgr2=0x7f72d80047d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:22:40.051 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:40.048+0000 7f72f478d640 1 --2- 172.21.15.143:0/176394699 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f72d8003fa0 0x7f72d80047d0 secure :-1 s=READY pgs=954 cs=0 l=1 rev1=1 crypto rx=0x7f72dc009600 tx=0x7f72dc01e700 comp rx=0 tx=0).stop 2023-11-26T07:22:40.051 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:40.048+0000 7f72f2d2a640 1 -- 172.21.15.143:0/176394699 reap_dead start 2023-11-26T07:22:40.052 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:40.048+0000 7f72f478d640 1 -- 172.21.15.143:0/176394699 shutdown_connections 2023-11-26T07:22:40.052 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:40.048+0000 7f72f478d640 1 -- 172.21.15.143:0/176394699 >> 172.21.15.143:0/176394699 conn(0x7f72ec06a890 msgr2=0x7f72ec06aca0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:22:40.052 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:40.048+0000 7f72f478d640 1 -- 172.21.15.143:0/176394699 shutdown_connections 2023-11-26T07:22:40.052 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:40.048+0000 7f72f478d640 1 -- 172.21.15.143:0/176394699 wait complete. 2023-11-26T07:22:40.052 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:40.052+0000 7f72f478d640 1 Processor -- start 2023-11-26T07:22:40.053 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:40.052+0000 7f72f478d640 1 -- start start 2023-11-26T07:22:40.055 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:40.052+0000 7f72f478d640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f72d8003fa0 0x7f72ec159a60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:22:40.055 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:40.052+0000 7f72f478d640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f72ec0ca750 0x7f72ec0cabd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:22:40.055 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:40.052+0000 7f72f478d640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f72ec13bdc0 0x7f72ec13c240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:22:40.055 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:40.052+0000 7f72f478d640 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f72ec13cc20 con 0x7f72ec13bdc0 2023-11-26T07:22:40.056 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:40.052+0000 7f72f478d640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f72ec13c780 con 0x7f72ec0ca750 2023-11-26T07:22:40.056 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:40.052+0000 7f72f478d640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f72ec13ecd0 con 0x7f72d8003fa0 2023-11-26T07:22:40.056 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:40.052+0000 7f72f2d2a640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f72ec13bdc0 0x7f72ec13c240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:22:40.056 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:40.052+0000 7f72f2d2a640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f72ec13bdc0 0x7f72ec13c240 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:47234/0 (socket says 172.21.15.143:47234) 2023-11-26T07:22:40.056 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:40.052+0000 7f72f2d2a640 1 -- 172.21.15.143:0/3994706694 learned_addr learned my addr 172.21.15.143:0/3994706694 (peer_addr_for_me v2:172.21.15.143:0/0) 2023-11-26T07:22:40.056 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:40.052+0000 7f72f1d28640 1 --2- 172.21.15.143:0/3994706694 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f72ec0ca750 0x7f72ec0cabd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:22:40.056 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:40.052+0000 7f72f2529640 1 --2- 172.21.15.143:0/3994706694 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f72d8003fa0 0x7f72ec159a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:22:40.056 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:40.052+0000 7f72f2d2a640 1 -- 172.21.15.143:0/3994706694 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f72d8003fa0 msgr2=0x7f72ec159a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:22:40.056 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:40.052+0000 7f72f2d2a640 1 --2- 172.21.15.143:0/3994706694 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f72d8003fa0 0x7f72ec159a60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:22:40.056 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:40.052+0000 7f72f2d2a640 1 -- 172.21.15.143:0/3994706694 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f72ec0ca750 msgr2=0x7f72ec0cabd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:22:40.056 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:40.052+0000 7f72f2d2a640 1 --2- 172.21.15.143:0/3994706694 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f72ec0ca750 0x7f72ec0cabd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:22:40.056 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:40.052+0000 7f72f2d2a640 1 -- 172.21.15.143:0/3994706694 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f72ec13f160 con 0x7f72ec13bdc0 2023-11-26T07:22:40.057 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:40.052+0000 7f72f2d2a640 1 --2- 172.21.15.143:0/3994706694 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f72ec13bdc0 0x7f72ec13c240 secure :-1 s=READY pgs=955 cs=0 l=1 rev1=1 crypto rx=0x7f72e4000f90 tx=0x7f72e400ec10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:22:40.057 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:40.052+0000 7f72e2ffd640 1 -- 172.21.15.143:0/3994706694 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f72e4006b60 con 0x7f72ec13bdc0 2023-11-26T07:22:40.057 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:40.052+0000 7f72e2ffd640 1 -- 172.21.15.143:0/3994706694 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f72e4006ce0 con 0x7f72ec13bdc0 2023-11-26T07:22:40.057 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:40.052+0000 7f72f478d640 1 -- 172.21.15.143:0/3994706694 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f72ec13e3a0 con 0x7f72ec13bdc0 2023-11-26T07:22:40.057 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:40.052+0000 7f72e2ffd640 1 -- 172.21.15.143:0/3994706694 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f72e4003bc0 con 0x7f72ec13bdc0 2023-11-26T07:22:40.057 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:40.052+0000 7f72f478d640 1 -- 172.21.15.143:0/3994706694 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f72ec13e960 con 0x7f72ec13bdc0 2023-11-26T07:22:40.058 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:40.056+0000 7f72f478d640 1 -- 172.21.15.143:0/3994706694 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f72ec13ee40 con 0x7f72ec13bdc0 2023-11-26T07:22:40.058 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:40.056+0000 7f72e2ffd640 1 -- 172.21.15.143:0/3994706694 <== mon.0 v2:172.21.15.143:3300/0 4 ==== mgrmap(e 11) v1 ==== 86158+0+0 (secure 0 0 0) 0x7f72e4003d20 con 0x7f72ec13bdc0 2023-11-26T07:22:40.058 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:40.056+0000 7f72e2ffd640 1 --2- 172.21.15.143:0/3994706694 >> [v2:172.21.15.143:6832/2150557544,v1:172.21.15.143:6833/2150557544] conn(0x7f72d0066800 0x7f72d0068cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:22:40.058 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:40.056+0000 7f72e2ffd640 1 -- 172.21.15.143:0/3994706694 <== mon.0 v2:172.21.15.143:3300/0 5 ==== osd_map(71..71 src has 1..71) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f72e4083af0 con 0x7f72ec13bdc0 2023-11-26T07:22:40.058 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:40.056+0000 7f72f2529640 1 --2- 172.21.15.143:0/3994706694 >> [v2:172.21.15.143:6832/2150557544,v1:172.21.15.143:6833/2150557544] conn(0x7f72d0066800 0x7f72d0068cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:22:40.058 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:40.056+0000 7f72f2529640 1 --2- 172.21.15.143:0/3994706694 >> [v2:172.21.15.143:6832/2150557544,v1:172.21.15.143:6833/2150557544] conn(0x7f72d0066800 0x7f72d0068cc0 secure :-1 s=READY pgs=449 cs=0 l=1 rev1=1 crypto rx=0x7f72ec15aa80 tx=0x7f72e8000f90 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:22:40.062 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:40.060+0000 7f72e2ffd640 1 -- 172.21.15.143:0/3994706694 <== mon.0 v2:172.21.15.143:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172649 (secure 0 0 0) 0x7f72e4050bd0 con 0x7f72ec13bdc0 2023-11-26T07:22:40.241 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:40.240+0000 7f72f478d640 1 -- 172.21.15.143:0/3994706694 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"} v 0) v1 -- 0x7f72ec067680 con 0x7f72ec13bdc0 2023-11-26T07:22:40.878 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:40.876+0000 7f72e2ffd640 1 -- 172.21.15.143:0/3994706694 <== mon.0 v2:172.21.15.143:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"}]=0 v42) v1 ==== 122+0+0 (secure 0 0 0) 0x7f72e40190c0 con 0x7f72ec13bdc0 2023-11-26T07:22:40.881 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:40.880+0000 7f72f478d640 1 -- 172.21.15.143:0/3994706694 >> [v2:172.21.15.143:6832/2150557544,v1:172.21.15.143:6833/2150557544] conn(0x7f72d0066800 msgr2=0x7f72d0068cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:22:40.881 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:40.880+0000 7f72f478d640 1 --2- 172.21.15.143:0/3994706694 >> [v2:172.21.15.143:6832/2150557544,v1:172.21.15.143:6833/2150557544] conn(0x7f72d0066800 0x7f72d0068cc0 secure :-1 s=READY pgs=449 cs=0 l=1 rev1=1 crypto rx=0x7f72ec15aa80 tx=0x7f72e8000f90 comp rx=0 tx=0).stop 2023-11-26T07:22:40.881 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:40.880+0000 7f72f478d640 1 -- 172.21.15.143:0/3994706694 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f72ec13bdc0 msgr2=0x7f72ec13c240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:22:40.881 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:40.880+0000 7f72f478d640 1 --2- 172.21.15.143:0/3994706694 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f72ec13bdc0 0x7f72ec13c240 secure :-1 s=READY pgs=955 cs=0 l=1 rev1=1 crypto rx=0x7f72e4000f90 tx=0x7f72e400ec10 comp rx=0 tx=0).stop 2023-11-26T07:22:40.882 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:40.880+0000 7f72f478d640 1 -- 172.21.15.143:0/3994706694 shutdown_connections 2023-11-26T07:22:40.882 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:40.880+0000 7f72f478d640 1 --2- 172.21.15.143:0/3994706694 >> [v2:172.21.15.143:6832/2150557544,v1:172.21.15.143:6833/2150557544] conn(0x7f72d0066800 0x7f72d0068cc0 unknown :-1 s=CLOSED pgs=449 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:22:40.882 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:40.880+0000 7f72f478d640 1 --2- 172.21.15.143:0/3994706694 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f72ec13bdc0 0x7f72ec13c240 unknown :-1 s=CLOSED pgs=955 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:22:40.882 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:40.880+0000 7f72f478d640 1 --2- 172.21.15.143:0/3994706694 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f72ec0ca750 0x7f72ec0cabd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:22:40.882 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:40.880+0000 7f72f478d640 1 --2- 172.21.15.143:0/3994706694 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f72d8003fa0 0x7f72ec159a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:22:40.882 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:40.880+0000 7f72f478d640 1 -- 172.21.15.143:0/3994706694 >> 172.21.15.143:0/3994706694 conn(0x7f72ec06a890 msgr2=0x7f72ec06aca0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:22:40.882 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:40.880+0000 7f72f478d640 1 -- 172.21.15.143:0/3994706694 shutdown_connections 2023-11-26T07:22:40.882 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:40.880+0000 7f72f478d640 1 -- 172.21.15.143:0/3994706694 wait complete. 2023-11-26T07:22:40.901 DEBUG:tasks.cephfs.filesystem:Not Creating any subvols for filesystem 'cephfs' 2023-11-26T07:22:40.901 DEBUG:teuthology.orchestra.run.smithi143:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-11-26T07:22:41.037 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:41.036+0000 7f9a784e4640 1 Processor -- start 2023-11-26T07:22:41.038 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:41.036+0000 7f9a784e4640 1 -- start start 2023-11-26T07:22:41.039 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:41.036+0000 7f9a784e4640 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7f9a70143ac0 0x7f9a70070370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:22:41.039 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:41.036+0000 7f9a784e4640 1 -- --> v1:172.21.15.143:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9a700cc730 con 0x7f9a700cf990 2023-11-26T07:22:41.039 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:41.036+0000 7f9a784e4640 1 -- --> v1:172.21.15.188:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9a70070ee0 con 0x7f9a700708b0 2023-11-26T07:22:41.039 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:41.036+0000 7f9a784e4640 1 -- --> v2:172.21.15.188:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9a70143ea0 con 0x7f9a70143ac0 2023-11-26T07:22:41.039 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:41.036+0000 7f9a76a81640 1 --1- >> v1:172.21.15.143:6789/0 conn(0x7f9a700cf990 0x7f9a700cd1b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.143:6789/0 says I am v1:172.21.15.143:35772/0 (socket says 172.21.15.143:35772) 2023-11-26T07:22:41.039 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:41.036+0000 7f9a76a81640 1 -- 172.21.15.143:0/1326830608 learned_addr learned my addr 172.21.15.143:0/1326830608 (peer_addr_for_me v1:172.21.15.143:0/0) 2023-11-26T07:22:41.040 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:41.040+0000 7f9a76280640 1 --2- 172.21.15.143:0/1326830608 >> v2:172.21.15.188:3300/0 conn(0x7f9a70143ac0 0x7f9a70070370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:22:41.040 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:41.040+0000 7f9a7527e640 1 -- 172.21.15.143:0/1326830608 <== mon.0 v1:172.21.15.143:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3280417904 0 0) 0x7f9a700cc730 con 0x7f9a700cf990 2023-11-26T07:22:41.040 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:41.040+0000 7f9a7527e640 1 -- 172.21.15.143:0/1326830608 --> v1:172.21.15.143:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f9a5c003530 con 0x7f9a700cf990 2023-11-26T07:22:41.040 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:41.040+0000 7f9a76280640 1 -- 172.21.15.143:0/1326830608 >> v1:172.21.15.188:6789/0 conn(0x7f9a700708b0 legacy=0x7f9a700cf280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:22:41.041 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:41.040+0000 7f9a76280640 1 -- 172.21.15.143:0/1326830608 >> v1:172.21.15.143:6789/0 conn(0x7f9a700cf990 legacy=0x7f9a700cd1b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:22:41.041 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:41.040+0000 7f9a76280640 1 -- 172.21.15.143:0/1326830608 --> v2:172.21.15.188:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9a700711c0 con 0x7f9a70143ac0 2023-11-26T07:22:41.043 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:41.040+0000 7f9a76280640 1 --2- 172.21.15.143:0/1326830608 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f9a70143ac0 0x7f9a70070370 secure :-1 s=READY pgs=383 cs=0 l=1 rev1=1 crypto rx=0x7f9a6c0097c0 tx=0x7f9a6c01c480 comp rx=0 tx=0).ready entity=mon.1 client_cookie=9811410449448054 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:22:41.043 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:41.040+0000 7f9a7527e640 1 -- 172.21.15.143:0/1326830608 <== mon.1 v2:172.21.15.188:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9a6c0a8050 con 0x7f9a70143ac0 2023-11-26T07:22:41.043 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:41.040+0000 7f9a7527e640 1 -- 172.21.15.143:0/1326830608 <== mon.1 v2:172.21.15.188:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9a6c0ac5f0 con 0x7f9a70143ac0 2023-11-26T07:22:41.043 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:41.040+0000 7f9a7527e640 1 -- 172.21.15.143:0/1326830608 <== mon.1 v2:172.21.15.188:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9a6c0ac980 con 0x7f9a70143ac0 2023-11-26T07:22:41.044 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:41.044+0000 7f9a784e4640 1 -- 172.21.15.143:0/1326830608 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f9a70143ac0 msgr2=0x7f9a70070370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:22:41.044 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:41.044+0000 7f9a784e4640 1 --2- 172.21.15.143:0/1326830608 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f9a70143ac0 0x7f9a70070370 secure :-1 s=READY pgs=383 cs=0 l=1 rev1=1 crypto rx=0x7f9a6c0097c0 tx=0x7f9a6c01c480 comp rx=0 tx=0).stop 2023-11-26T07:22:41.044 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:41.044+0000 7f9a784e4640 1 -- 172.21.15.143:0/1326830608 shutdown_connections 2023-11-26T07:22:41.044 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:41.044+0000 7f9a784e4640 1 --2- 172.21.15.143:0/1326830608 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f9a70143ac0 0x7f9a70070370 unknown :-1 s=CLOSED pgs=383 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:22:41.045 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:41.044+0000 7f9a784e4640 1 -- 172.21.15.143:0/1326830608 >> 172.21.15.143:0/1326830608 conn(0x7f9a7006a890 msgr2=0x7f9a7006aca0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:22:41.045 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:41.044+0000 7f9a784e4640 1 -- 172.21.15.143:0/1326830608 shutdown_connections 2023-11-26T07:22:41.045 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:41.044+0000 7f9a784e4640 1 -- 172.21.15.143:0/1326830608 wait complete. 2023-11-26T07:22:41.045 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:41.044+0000 7f9a784e4640 1 Processor -- start 2023-11-26T07:22:41.046 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:41.044+0000 7f9a784e4640 1 -- start start 2023-11-26T07:22:41.048 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:41.044+0000 7f9a784e4640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f9a700708b0 0x7f9a7015c370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:22:41.048 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:41.044+0000 7f9a784e4640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f9a700cf990 0x7f9a701f68f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:22:41.048 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:41.044+0000 7f9a784e4640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f9a70143ac0 0x7f9a701592f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:22:41.048 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:41.044+0000 7f9a784e4640 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9a701f6e30 con 0x7f9a700708b0 2023-11-26T07:22:41.048 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:41.044+0000 7f9a784e4640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9a701f9440 con 0x7f9a70143ac0 2023-11-26T07:22:41.048 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:41.044+0000 7f9a784e4640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9a70159830 con 0x7f9a700cf990 2023-11-26T07:22:41.049 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:41.044+0000 7f9a76280640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f9a700708b0 0x7f9a7015c370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:22:41.049 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:41.044+0000 7f9a76280640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f9a700708b0 0x7f9a7015c370 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:47240/0 (socket says 172.21.15.143:47240) 2023-11-26T07:22:41.049 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:41.044+0000 7f9a76280640 1 -- 172.21.15.143:0/2792730577 learned_addr learned my addr 172.21.15.143:0/2792730577 (peer_addr_for_me v2:172.21.15.143:0/0) 2023-11-26T07:22:41.049 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:41.044+0000 7f9a76a81640 1 --2- 172.21.15.143:0/2792730577 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f9a70143ac0 0x7f9a701592f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:22:41.049 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:41.044+0000 7f9a75a7f640 1 --2- 172.21.15.143:0/2792730577 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f9a700cf990 0x7f9a701f68f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:22:41.049 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:41.048+0000 7f9a76280640 1 -- 172.21.15.143:0/2792730577 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f9a700cf990 msgr2=0x7f9a701f68f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:22:41.049 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:41.048+0000 7f9a76280640 1 --2- 172.21.15.143:0/2792730577 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f9a700cf990 0x7f9a701f68f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:22:41.049 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:41.048+0000 7f9a76280640 1 -- 172.21.15.143:0/2792730577 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f9a70143ac0 msgr2=0x7f9a701592f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:22:41.049 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:41.048+0000 7f9a76280640 1 --2- 172.21.15.143:0/2792730577 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f9a70143ac0 0x7f9a701592f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:22:41.049 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:41.048+0000 7f9a76280640 1 -- 172.21.15.143:0/2792730577 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9a6c009420 con 0x7f9a700708b0 2023-11-26T07:22:41.049 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:41.048+0000 7f9a76a81640 1 --2- 172.21.15.143:0/2792730577 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f9a70143ac0 0x7f9a701592f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-26T07:22:41.049 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:41.048+0000 7f9a76280640 1 --2- 172.21.15.143:0/2792730577 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f9a700708b0 0x7f9a7015c370 secure :-1 s=READY pgs=957 cs=0 l=1 rev1=1 crypto rx=0x7f9a6c0097c0 tx=0x7f9a6c01acc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:22:41.049 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:41.048+0000 7f9a66ffd640 1 -- 172.21.15.143:0/2792730577 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9a6c006e20 con 0x7f9a700708b0 2023-11-26T07:22:41.050 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:41.048+0000 7f9a66ffd640 1 -- 172.21.15.143:0/2792730577 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9a6c0b8660 con 0x7f9a700708b0 2023-11-26T07:22:41.050 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:41.048+0000 7f9a66ffd640 1 -- 172.21.15.143:0/2792730577 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9a6c0b89f0 con 0x7f9a700708b0 2023-11-26T07:22:41.050 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:41.048+0000 7f9a784e4640 1 -- 172.21.15.143:0/2792730577 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9a7015c9a0 con 0x7f9a700708b0 2023-11-26T07:22:41.050 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:41.048+0000 7f9a784e4640 1 -- 172.21.15.143:0/2792730577 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9a701f9970 con 0x7f9a700708b0 2023-11-26T07:22:41.050 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:41.048+0000 7f9a784e4640 1 -- 172.21.15.143:0/2792730577 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9a70067680 con 0x7f9a700708b0 2023-11-26T07:22:41.050 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:41.048+0000 7f9a66ffd640 1 -- 172.21.15.143:0/2792730577 <== mon.0 v2:172.21.15.143:3300/0 4 ==== mgrmap(e 11) v1 ==== 86158+0+0 (secure 0 0 0) 0x7f9a6c0b8b50 con 0x7f9a700708b0 2023-11-26T07:22:41.051 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:41.048+0000 7f9a66ffd640 1 --2- 172.21.15.143:0/2792730577 >> [v2:172.21.15.143:6832/2150557544,v1:172.21.15.143:6833/2150557544] conn(0x7f9a540667d0 0x7f9a54068c90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:22:41.051 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:41.048+0000 7f9a66ffd640 1 -- 172.21.15.143:0/2792730577 <== mon.0 v2:172.21.15.143:3300/0 5 ==== osd_map(71..71 src has 1..71) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f9a6c11edb0 con 0x7f9a700708b0 2023-11-26T07:22:41.051 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:41.048+0000 7f9a75a7f640 1 --2- 172.21.15.143:0/2792730577 >> [v2:172.21.15.143:6832/2150557544,v1:172.21.15.143:6833/2150557544] conn(0x7f9a540667d0 0x7f9a54068c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:22:41.051 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:41.052+0000 7f9a75a7f640 1 --2- 172.21.15.143:0/2792730577 >> [v2:172.21.15.143:6832/2150557544,v1:172.21.15.143:6833/2150557544] conn(0x7f9a540667d0 0x7f9a54068c90 secure :-1 s=READY pgs=450 cs=0 l=1 rev1=1 crypto rx=0x7f9a70141f90 tx=0x7f9a60009040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:22:41.055 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:41.052+0000 7f9a66ffd640 1 -- 172.21.15.143:0/2792730577 <== mon.0 v2:172.21.15.143:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172649 (secure 0 0 0) 0x7f9a6c123070 con 0x7f9a700708b0 2023-11-26T07:22:41.235 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:41.232+0000 7f9a784e4640 1 -- 172.21.15.143:0/2792730577 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f9a700b2db0 con 0x7f9a700708b0 2023-11-26T07:22:41.235 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:41.236+0000 7f9a66ffd640 1 -- 172.21.15.143:0/2792730577 <== mon.0 v2:172.21.15.143:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 42 v42) v1 ==== 94+0+4924 (secure 0 0 0) 0x7f9a6c0ebd60 con 0x7f9a700708b0 2023-11-26T07:22:41.236 INFO:teuthology.orchestra.run.smithi143.stdout: 2023-11-26T07:22:41.236 INFO:teuthology.orchestra.run.smithi143.stdout:{"epoch":42,"default_fscid":5,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5927,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.143:6837/578144260","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6835","nonce":578144260},{"type":"v1","addr":"172.21.15.143:6837","nonce":578144260}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":38},{"gid":5930,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.188:6834/1048736873","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6832","nonce":1048736873},{"type":"v1","addr":"172.21.15.188:6834","nonce":1048736873}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":38},{"gid":5933,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.188:6835/2063768716","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6833","nonce":2063768716},{"type":"v1","addr":"172.21.15.188:6835","nonce":2063768716}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":38}],"filesystems":[{"mdsmap":{"epoch":42,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-11-26T07:22:37.220113+0000","modified":"2023-11-26T07:22:40.243052+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":6553},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6553":{"gid":6553,"name":"a","rank":0,"incarnation":39,"state":"up:active","state_seq":3,"addr":"172.21.15.143:6836/2590663349","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6834","nonce":2590663349},{"type":"v1","addr":"172.21.15.143:6836","nonce":2590663349}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[11],"metadata_pool":10,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":5}]} 2023-11-26T07:22:41.236 INFO:teuthology.orchestra.run.smithi143.stderr:dumped fsmap epoch 42 2023-11-26T07:22:41.238 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:41.236+0000 7f9a784e4640 1 -- 172.21.15.143:0/2792730577 >> [v2:172.21.15.143:6832/2150557544,v1:172.21.15.143:6833/2150557544] conn(0x7f9a540667d0 msgr2=0x7f9a54068c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:22:41.238 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:41.236+0000 7f9a784e4640 1 --2- 172.21.15.143:0/2792730577 >> [v2:172.21.15.143:6832/2150557544,v1:172.21.15.143:6833/2150557544] conn(0x7f9a540667d0 0x7f9a54068c90 secure :-1 s=READY pgs=450 cs=0 l=1 rev1=1 crypto rx=0x7f9a70141f90 tx=0x7f9a60009040 comp rx=0 tx=0).stop 2023-11-26T07:22:41.238 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:41.236+0000 7f9a784e4640 1 -- 172.21.15.143:0/2792730577 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f9a700708b0 msgr2=0x7f9a7015c370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:22:41.238 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:41.236+0000 7f9a784e4640 1 --2- 172.21.15.143:0/2792730577 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f9a700708b0 0x7f9a7015c370 secure :-1 s=READY pgs=957 cs=0 l=1 rev1=1 crypto rx=0x7f9a6c0097c0 tx=0x7f9a6c01acc0 comp rx=0 tx=0).stop 2023-11-26T07:22:41.239 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:41.236+0000 7f9a784e4640 1 -- 172.21.15.143:0/2792730577 shutdown_connections 2023-11-26T07:22:41.239 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:41.236+0000 7f9a784e4640 1 --2- 172.21.15.143:0/2792730577 >> [v2:172.21.15.143:6832/2150557544,v1:172.21.15.143:6833/2150557544] conn(0x7f9a540667d0 0x7f9a54068c90 unknown :-1 s=CLOSED pgs=450 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:22:41.239 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:41.236+0000 7f9a784e4640 1 --2- 172.21.15.143:0/2792730577 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f9a70143ac0 0x7f9a701592f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:22:41.239 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:41.236+0000 7f9a784e4640 1 --2- 172.21.15.143:0/2792730577 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f9a700cf990 0x7f9a701f68f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:22:41.239 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:41.236+0000 7f9a784e4640 1 --2- 172.21.15.143:0/2792730577 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f9a700708b0 0x7f9a7015c370 unknown :-1 s=CLOSED pgs=957 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:22:41.239 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:41.236+0000 7f9a784e4640 1 -- 172.21.15.143:0/2792730577 >> 172.21.15.143:0/2792730577 conn(0x7f9a7006a890 msgr2=0x7f9a700addc0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:22:41.239 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:41.236+0000 7f9a784e4640 1 -- 172.21.15.143:0/2792730577 shutdown_connections 2023-11-26T07:22:41.239 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:41.236+0000 7f9a784e4640 1 -- 172.21.15.143:0/2792730577 wait complete. 2023-11-26T07:22:41.257 DEBUG:teuthology.orchestra.run.smithi143:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-11-26T07:22:41.394 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:41.392+0000 7f1cdaa6b640 1 Processor -- start 2023-11-26T07:22:41.394 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:41.392+0000 7f1cdaa6b640 1 -- start start 2023-11-26T07:22:41.395 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:41.392+0000 7f1cdaa6b640 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7f1cd40cf280 0x7f1cd40cf710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:22:41.395 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:41.392+0000 7f1cdaa6b640 1 -- --> v1:172.21.15.143:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1cd40cc730 con 0x7f1cd4143ac0 2023-11-26T07:22:41.396 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:41.396+0000 7f1cdaa6b640 1 -- --> v1:172.21.15.188:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1cd4070ee0 con 0x7f1cd40cd1b0 2023-11-26T07:22:41.396 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:41.396+0000 7f1cdaa6b640 1 -- --> v2:172.21.15.143:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1cd4143ea0 con 0x7f1cd40cf280 2023-11-26T07:22:41.396 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:41.396+0000 7f1cd8807640 1 --1- >> v1:172.21.15.143:6789/0 conn(0x7f1cd4143ac0 0x7f1cd4070370 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.143:6789/0 says I am v1:172.21.15.143:35788/0 (socket says 172.21.15.143:35788) 2023-11-26T07:22:41.396 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:41.396+0000 7f1cd8807640 1 -- 172.21.15.143:0/3470719760 learned_addr learned my addr 172.21.15.143:0/3470719760 (peer_addr_for_me v1:172.21.15.143:0/0) 2023-11-26T07:22:41.396 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:41.396+0000 7f1ccbfff640 1 --2- 172.21.15.143:0/3470719760 >> v2:172.21.15.143:3300/0 conn(0x7f1cd40cf280 0x7f1cd40cf710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:22:41.396 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:41.396+0000 7f1ccbfff640 1 -- 172.21.15.143:0/3470719760 >> v1:172.21.15.188:6789/0 conn(0x7f1cd40cd1b0 legacy=0x7f1cd40cd5c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:22:41.397 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:41.396+0000 7f1ccbfff640 1 -- 172.21.15.143:0/3470719760 >> v1:172.21.15.143:6789/0 conn(0x7f1cd4143ac0 legacy=0x7f1cd4070370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:22:41.397 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:41.396+0000 7f1ccbfff640 1 -- 172.21.15.143:0/3470719760 --> v2:172.21.15.143:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1cd40711c0 con 0x7f1cd40cf280 2023-11-26T07:22:41.397 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:41.396+0000 7f1ccb7fe640 1 -- 172.21.15.143:0/3470719760 <== mon.0 v1:172.21.15.143:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1294647700 0 0) 0x7f1cd40cc730 con 0x7f1cd4143ac0 2023-11-26T07:22:41.399 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:41.396+0000 7f1ccbfff640 1 --2- 172.21.15.143:0/3470719760 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f1cd40cf280 0x7f1cd40cf710 secure :-1 s=READY pgs=959 cs=0 l=1 rev1=1 crypto rx=0x7f1cc40097c0 tx=0x7f1cc401c480 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ce6b7b282b1a75c9 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:22:41.399 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:41.400+0000 7f1ccb7fe640 1 -- 172.21.15.143:0/3470719760 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1cc40a8050 con 0x7f1cd40cf280 2023-11-26T07:22:41.400 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:41.400+0000 7f1ccb7fe640 1 -- 172.21.15.143:0/3470719760 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1cc40ac5f0 con 0x7f1cd40cf280 2023-11-26T07:22:41.400 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:41.400+0000 7f1ccb7fe640 1 -- 172.21.15.143:0/3470719760 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1cc40ac980 con 0x7f1cd40cf280 2023-11-26T07:22:41.400 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:41.400+0000 7f1cdaa6b640 1 -- 172.21.15.143:0/3470719760 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f1cd40cf280 msgr2=0x7f1cd40cf710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:22:41.400 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:41.400+0000 7f1cdaa6b640 1 --2- 172.21.15.143:0/3470719760 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f1cd40cf280 0x7f1cd40cf710 secure :-1 s=READY pgs=959 cs=0 l=1 rev1=1 crypto rx=0x7f1cc40097c0 tx=0x7f1cc401c480 comp rx=0 tx=0).stop 2023-11-26T07:22:41.401 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:41.400+0000 7f1cdaa6b640 1 -- 172.21.15.143:0/3470719760 shutdown_connections 2023-11-26T07:22:41.401 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:41.400+0000 7f1cdaa6b640 1 --2- 172.21.15.143:0/3470719760 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f1cd40cf280 0x7f1cd40cf710 unknown :-1 s=CLOSED pgs=959 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:22:41.401 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:41.400+0000 7f1cdaa6b640 1 -- 172.21.15.143:0/3470719760 >> 172.21.15.143:0/3470719760 conn(0x7f1cd406a890 msgr2=0x7f1cd406aca0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:22:41.401 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:41.400+0000 7f1cdaa6b640 1 -- 172.21.15.143:0/3470719760 shutdown_connections 2023-11-26T07:22:41.401 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:41.400+0000 7f1cdaa6b640 1 -- 172.21.15.143:0/3470719760 wait complete. 2023-11-26T07:22:41.402 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:41.400+0000 7f1cdaa6b640 1 Processor -- start 2023-11-26T07:22:41.402 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:41.400+0000 7f1cdaa6b640 1 -- start start 2023-11-26T07:22:41.403 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:41.400+0000 7f1cdaa6b640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f1cd40cd1b0 0x7f1cd41f6150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:22:41.403 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:41.400+0000 7f1cdaa6b640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f1cd40cf280 0x7f1cd40af290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:22:41.403 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:41.400+0000 7f1cdaa6b640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f1cd4143ac0 0x7f1cd40b01b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:22:41.403 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:41.400+0000 7f1cdaa6b640 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1cd413b320 con 0x7f1cd40cd1b0 2023-11-26T07:22:41.403 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:41.400+0000 7f1cdaa6b640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1cd40aeb60 con 0x7f1cd40cf280 2023-11-26T07:22:41.404 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:41.400+0000 7f1cdaa6b640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1cd415c110 con 0x7f1cd4143ac0 2023-11-26T07:22:41.404 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:41.400+0000 7f1cd8807640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f1cd40cd1b0 0x7f1cd41f6150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:22:41.404 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:41.404+0000 7f1cd8807640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f1cd40cd1b0 0x7f1cd41f6150 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:47258/0 (socket says 172.21.15.143:47258) 2023-11-26T07:22:41.404 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:41.404+0000 7f1cd8807640 1 -- 172.21.15.143:0/3132375107 learned_addr learned my addr 172.21.15.143:0/3132375107 (peer_addr_for_me v2:172.21.15.143:0/0) 2023-11-26T07:22:41.405 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:41.404+0000 7f1cd8807640 1 -- 172.21.15.143:0/3132375107 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f1cd4143ac0 msgr2=0x7f1cd40b01b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:22:41.405 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:41.404+0000 7f1cd8807640 1 --2- 172.21.15.143:0/3132375107 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f1cd4143ac0 0x7f1cd40b01b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:22:41.405 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:41.404+0000 7f1cd8807640 1 -- 172.21.15.143:0/3132375107 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f1cd40cf280 msgr2=0x7f1cd40af290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:22:41.405 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:41.404+0000 7f1cd8807640 1 --2- 172.21.15.143:0/3132375107 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f1cd40cf280 0x7f1cd40af290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:22:41.405 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:41.404+0000 7f1cd8807640 1 -- 172.21.15.143:0/3132375107 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1cc4009420 con 0x7f1cd40cd1b0 2023-11-26T07:22:41.405 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:41.404+0000 7f1cd8807640 1 --2- 172.21.15.143:0/3132375107 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f1cd40cd1b0 0x7f1cd41f6150 secure :-1 s=READY pgs=960 cs=0 l=1 rev1=1 crypto rx=0x7f1cd000c410 tx=0x7f1cd000c8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:22:41.405 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:41.404+0000 7f1cc97fa640 1 -- 172.21.15.143:0/3132375107 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1cd00074e0 con 0x7f1cd40cd1b0 2023-11-26T07:22:41.406 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:41.404+0000 7f1cc97fa640 1 -- 172.21.15.143:0/3132375107 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1cd0007640 con 0x7f1cd40cd1b0 2023-11-26T07:22:41.406 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:41.404+0000 7f1cc97fa640 1 -- 172.21.15.143:0/3132375107 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1cd0004a30 con 0x7f1cd40cd1b0 2023-11-26T07:22:41.406 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:41.404+0000 7f1cdaa6b640 1 -- 172.21.15.143:0/3132375107 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1cd41f6980 con 0x7f1cd40cd1b0 2023-11-26T07:22:41.406 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:41.404+0000 7f1cdaa6b640 1 -- 172.21.15.143:0/3132375107 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1cd41f6f40 con 0x7f1cd40cd1b0 2023-11-26T07:22:41.406 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:41.404+0000 7f1cdaa6b640 1 -- 172.21.15.143:0/3132375107 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1cd4067680 con 0x7f1cd40cd1b0 2023-11-26T07:22:41.406 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:41.404+0000 7f1cc97fa640 1 -- 172.21.15.143:0/3132375107 <== mon.0 v2:172.21.15.143:3300/0 4 ==== mgrmap(e 11) v1 ==== 86158+0+0 (secure 0 0 0) 0x7f1cd0004b90 con 0x7f1cd40cd1b0 2023-11-26T07:22:41.407 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:41.404+0000 7f1cc97fa640 1 --2- 172.21.15.143:0/3132375107 >> [v2:172.21.15.143:6832/2150557544,v1:172.21.15.143:6833/2150557544] conn(0x7f1cc00667d0 0x7f1cc0068c90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:22:41.407 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:41.408+0000 7f1cc97fa640 1 -- 172.21.15.143:0/3132375107 <== mon.0 v2:172.21.15.143:3300/0 5 ==== osd_map(71..71 src has 1..71) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f1cd0084300 con 0x7f1cd40cd1b0 2023-11-26T07:22:41.407 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:41.408+0000 7f1ccbfff640 1 --2- 172.21.15.143:0/3132375107 >> [v2:172.21.15.143:6832/2150557544,v1:172.21.15.143:6833/2150557544] conn(0x7f1cc00667d0 0x7f1cc0068c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:22:41.408 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:41.408+0000 7f1ccbfff640 1 --2- 172.21.15.143:0/3132375107 >> [v2:172.21.15.143:6832/2150557544,v1:172.21.15.143:6833/2150557544] conn(0x7f1cc00667d0 0x7f1cc0068c90 secure :-1 s=READY pgs=451 cs=0 l=1 rev1=1 crypto rx=0x7f1cc40ae000 tx=0x7f1cc4004740 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:22:41.411 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:41.408+0000 7f1cc97fa640 1 -- 172.21.15.143:0/3132375107 <== mon.0 v2:172.21.15.143:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172649 (secure 0 0 0) 0x7f1cd0051230 con 0x7f1cd40cd1b0 2023-11-26T07:22:41.561 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:41.560+0000 7f1cdaa6b640 1 -- 172.21.15.143:0/3132375107 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f1cd4141a10 con 0x7f1cd40cd1b0 2023-11-26T07:22:41.562 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:41.560+0000 7f1cc97fa640 1 -- 172.21.15.143:0/3132375107 <== mon.0 v2:172.21.15.143:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v71) v1 ==== 74+0+16604 (secure 0 0 0) 0x7f1cd0054850 con 0x7f1cd40cd1b0 2023-11-26T07:22:41.562 INFO:teuthology.orchestra.run.smithi143.stdout: 2023-11-26T07:22:41.563 INFO:teuthology.orchestra.run.smithi143.stdout:{"epoch":71,"fsid":"f5033060-2469-4a07-95fe-2b76f5404bc1","created":"2023-11-26T07:17:28.613663+0000","modified":"2023-11-26T07:22:38.856077+0000","last_up_change":"2023-11-26T07:17:41.157813+0000","last_in_change":"2023-11-26T07:17:31.181658+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":11,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-11-26T07:17:41.758597+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":10,"pool_name":"cephfs_metadata","create_time":"2023-11-26T07:22:35.314819+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"70","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":2.5,"score_stable":2.5,"optimal_score":1,"raw_score_acting":2.5,"raw_score_stable":2.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":11,"pool_name":"cephfs_data","create_time":"2023-11-26T07:22:36.219536+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"71","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"ebbee2c6-023e-44b9-9f1d-6f7bfd309730","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":69,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6817","nonce":639807369},{"type":"v1","addr":"172.21.15.143:6819","nonce":639807369}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6821","nonce":639807369},{"type":"v1","addr":"172.21.15.143:6823","nonce":639807369}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6830","nonce":639807369},{"type":"v1","addr":"172.21.15.143:6831","nonce":639807369}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6825","nonce":639807369},{"type":"v1","addr":"172.21.15.143:6828","nonce":639807369}]},"public_addr":"172.21.15.143:6819/639807369","cluster_addr":"172.21.15.143:6823/639807369","heartbeat_back_addr":"172.21.15.143:6831/639807369","heartbeat_front_addr":"172.21.15.143:6828/639807369","state":["exists","up"]},{"osd":1,"uuid":"fec4b8be-7122-49e9-9a19-5850a0700001","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":69,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6816","nonce":1144942530},{"type":"v1","addr":"172.21.15.143:6818","nonce":1144942530}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6820","nonce":1144942530},{"type":"v1","addr":"172.21.15.143:6822","nonce":1144942530}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6827","nonce":1144942530},{"type":"v1","addr":"172.21.15.143:6829","nonce":1144942530}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6824","nonce":1144942530},{"type":"v1","addr":"172.21.15.143:6826","nonce":1144942530}]},"public_addr":"172.21.15.143:6818/1144942530","cluster_addr":"172.21.15.143:6822/1144942530","heartbeat_back_addr":"172.21.15.143:6829/1144942530","heartbeat_front_addr":"172.21.15.143:6826/1144942530","state":["exists","up"]},{"osd":2,"uuid":"2888af5b-f933-46ca-8f95-6dd494ff43f6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":69,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6808","nonce":3909095186},{"type":"v1","addr":"172.21.15.143:6809","nonce":3909095186}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6810","nonce":3909095186},{"type":"v1","addr":"172.21.15.143:6811","nonce":3909095186}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6814","nonce":3909095186},{"type":"v1","addr":"172.21.15.143:6815","nonce":3909095186}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6812","nonce":3909095186},{"type":"v1","addr":"172.21.15.143:6813","nonce":3909095186}]},"public_addr":"172.21.15.143:6809/3909095186","cluster_addr":"172.21.15.143:6811/3909095186","heartbeat_back_addr":"172.21.15.143:6815/3909095186","heartbeat_front_addr":"172.21.15.143:6813/3909095186","state":["exists","up"]},{"osd":3,"uuid":"d1f52435-7bc1-4081-9550-9cd1ba55f549","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":69,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6800","nonce":3854210991},{"type":"v1","addr":"172.21.15.143:6801","nonce":3854210991}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6802","nonce":3854210991},{"type":"v1","addr":"172.21.15.143:6803","nonce":3854210991}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6806","nonce":3854210991},{"type":"v1","addr":"172.21.15.143:6807","nonce":3854210991}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6804","nonce":3854210991},{"type":"v1","addr":"172.21.15.143:6805","nonce":3854210991}]},"public_addr":"172.21.15.143:6801/3854210991","cluster_addr":"172.21.15.143:6803/3854210991","heartbeat_back_addr":"172.21.15.143:6807/3854210991","heartbeat_front_addr":"172.21.15.143:6805/3854210991","state":["exists","up"]},{"osd":4,"uuid":"e2ae8b6e-6e25-49b2-bcbf-aacce68f06a3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":69,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6800","nonce":3924606902},{"type":"v1","addr":"172.21.15.188:6801","nonce":3924606902}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6802","nonce":3924606902},{"type":"v1","addr":"172.21.15.188:6803","nonce":3924606902}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6806","nonce":3924606902},{"type":"v1","addr":"172.21.15.188:6807","nonce":3924606902}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6804","nonce":3924606902},{"type":"v1","addr":"172.21.15.188:6805","nonce":3924606902}]},"public_addr":"172.21.15.188:6801/3924606902","cluster_addr":"172.21.15.188:6803/3924606902","heartbeat_back_addr":"172.21.15.188:6807/3924606902","heartbeat_front_addr":"172.21.15.188:6805/3924606902","state":["exists","up"]},{"osd":5,"uuid":"3cec8a32-7698-4c36-8e89-e4ba11934f33","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":69,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6813","nonce":186841544},{"type":"v1","addr":"172.21.15.188:6815","nonce":186841544}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6817","nonce":186841544},{"type":"v1","addr":"172.21.15.188:6818","nonce":186841544}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6822","nonce":186841544},{"type":"v1","addr":"172.21.15.188:6823","nonce":186841544}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6820","nonce":186841544},{"type":"v1","addr":"172.21.15.188:6821","nonce":186841544}]},"public_addr":"172.21.15.188:6815/186841544","cluster_addr":"172.21.15.188:6818/186841544","heartbeat_back_addr":"172.21.15.188:6823/186841544","heartbeat_front_addr":"172.21.15.188:6821/186841544","state":["exists","up"]},{"osd":6,"uuid":"58a65d5d-e090-473a-81b4-ad38e85029f9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":69,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6808","nonce":2153499405},{"type":"v1","addr":"172.21.15.188:6809","nonce":2153499405}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6810","nonce":2153499405},{"type":"v1","addr":"172.21.15.188:6811","nonce":2153499405}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6816","nonce":2153499405},{"type":"v1","addr":"172.21.15.188:6819","nonce":2153499405}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6812","nonce":2153499405},{"type":"v1","addr":"172.21.15.188:6814","nonce":2153499405}]},"public_addr":"172.21.15.188:6809/2153499405","cluster_addr":"172.21.15.188:6811/2153499405","heartbeat_back_addr":"172.21.15.188:6819/2153499405","heartbeat_front_addr":"172.21.15.188:6814/2153499405","state":["exists","up"]},{"osd":7,"uuid":"ea5b7f70-3554-4cf8-ac30-f7c59b80678d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":69,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6824","nonce":2722096729},{"type":"v1","addr":"172.21.15.188:6825","nonce":2722096729}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6826","nonce":2722096729},{"type":"v1","addr":"172.21.15.188:6827","nonce":2722096729}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6830","nonce":2722096729},{"type":"v1","addr":"172.21.15.188:6831","nonce":2722096729}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6828","nonce":2722096729},{"type":"v1","addr":"172.21.15.188:6829","nonce":2722096729}]},"public_addr":"172.21.15.188:6825/2722096729","cluster_addr":"172.21.15.188:6827/2722096729","heartbeat_back_addr":"172.21.15.188:6831/2722096729","heartbeat_front_addr":"172.21.15.188:6829/2722096729","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-26T07:17:36.995032+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-26T07:17:36.932266+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-26T07:17:36.829304+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-26T07:17:36.804297+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-26T07:17:36.667892+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-26T07:17:36.964868+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-26T07:17:36.827332+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-26T07:17:36.893179+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-11-26T07:22:41.565 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:41.564+0000 7f1cdaa6b640 1 -- 172.21.15.143:0/3132375107 >> [v2:172.21.15.143:6832/2150557544,v1:172.21.15.143:6833/2150557544] conn(0x7f1cc00667d0 msgr2=0x7f1cc0068c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:22:41.565 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:41.564+0000 7f1cdaa6b640 1 --2- 172.21.15.143:0/3132375107 >> [v2:172.21.15.143:6832/2150557544,v1:172.21.15.143:6833/2150557544] conn(0x7f1cc00667d0 0x7f1cc0068c90 secure :-1 s=READY pgs=451 cs=0 l=1 rev1=1 crypto rx=0x7f1cc40ae000 tx=0x7f1cc4004740 comp rx=0 tx=0).stop 2023-11-26T07:22:41.565 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:41.564+0000 7f1cdaa6b640 1 -- 172.21.15.143:0/3132375107 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f1cd40cd1b0 msgr2=0x7f1cd41f6150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:22:41.565 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:41.564+0000 7f1cdaa6b640 1 --2- 172.21.15.143:0/3132375107 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f1cd40cd1b0 0x7f1cd41f6150 secure :-1 s=READY pgs=960 cs=0 l=1 rev1=1 crypto rx=0x7f1cd000c410 tx=0x7f1cd000c8d0 comp rx=0 tx=0).stop 2023-11-26T07:22:41.565 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:41.564+0000 7f1cdaa6b640 1 -- 172.21.15.143:0/3132375107 shutdown_connections 2023-11-26T07:22:41.565 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:41.564+0000 7f1cdaa6b640 1 --2- 172.21.15.143:0/3132375107 >> [v2:172.21.15.143:6832/2150557544,v1:172.21.15.143:6833/2150557544] conn(0x7f1cc00667d0 0x7f1cc0068c90 unknown :-1 s=CLOSED pgs=451 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:22:41.566 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:41.564+0000 7f1cdaa6b640 1 --2- 172.21.15.143:0/3132375107 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f1cd4143ac0 0x7f1cd40b01b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:22:41.566 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:41.564+0000 7f1cdaa6b640 1 --2- 172.21.15.143:0/3132375107 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f1cd40cf280 0x7f1cd40af290 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:22:41.566 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:41.564+0000 7f1cdaa6b640 1 --2- 172.21.15.143:0/3132375107 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f1cd40cd1b0 0x7f1cd41f6150 unknown :-1 s=CLOSED pgs=960 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:22:41.566 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:41.564+0000 7f1cdaa6b640 1 -- 172.21.15.143:0/3132375107 >> 172.21.15.143:0/3132375107 conn(0x7f1cd406a890 msgr2=0x7f1cd4140da0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:22:41.566 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:41.564+0000 7f1cdaa6b640 1 -- 172.21.15.143:0/3132375107 shutdown_connections 2023-11-26T07:22:41.566 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:41.564+0000 7f1cdaa6b640 1 -- 172.21.15.143:0/3132375107 wait complete. 2023-11-26T07:22:41.584 INFO:tasks.cephfs.filesystem.ceph_manager:waiting for clean 2023-11-26T07:22:41.584 DEBUG:teuthology.orchestra.run.smithi143:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-11-26T07:22:41.721 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:41.720+0000 7f82839df640 1 Processor -- start 2023-11-26T07:22:41.721 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:41.720+0000 7f82839df640 1 -- start start 2023-11-26T07:22:41.723 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:41.720+0000 7f82839df640 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7f827c0cf990 0x7f827c0708c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:22:41.723 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:41.720+0000 7f82839df640 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7f827c070e00 0x7f827c162510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:22:41.723 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:41.720+0000 7f82839df640 1 -- --> v1:172.21.15.143:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f827c162080 con 0x7f827c14b1e0 2023-11-26T07:22:41.724 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:41.720+0000 7f82839df640 1 -- --> v2:172.21.15.143:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f827c161d70 con 0x7f827c070e00 2023-11-26T07:22:41.724 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:41.720+0000 7f82839df640 1 -- --> v2:172.21.15.188:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f827c14b5c0 con 0x7f827c0cf990 2023-11-26T07:22:41.724 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:41.720+0000 7f828177b640 1 --1- >> v1:172.21.15.143:6789/0 conn(0x7f827c14b1e0 0x7f827c0cf280 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.143:6789/0 says I am v1:172.21.15.143:35790/0 (socket says 172.21.15.143:35790) 2023-11-26T07:22:41.724 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:41.720+0000 7f828177b640 1 -- 172.21.15.143:0/1999956368 learned_addr learned my addr 172.21.15.143:0/1999956368 (peer_addr_for_me v1:172.21.15.143:0/0) 2023-11-26T07:22:41.724 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:41.720+0000 7f8281f7c640 1 --2- 172.21.15.143:0/1999956368 >> v2:172.21.15.143:3300/0 conn(0x7f827c070e00 0x7f827c162510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:22:41.724 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:41.720+0000 7f8280f7a640 1 --2- 172.21.15.143:0/1999956368 >> v2:172.21.15.188:3300/0 conn(0x7f827c0cf990 0x7f827c0708c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:22:41.724 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:41.720+0000 7f8273fff640 1 -- 172.21.15.143:0/1999956368 <== mon.0 v1:172.21.15.143:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3653243269 0 0) 0x7f827c162080 con 0x7f827c14b1e0 2023-11-26T07:22:41.724 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:41.724+0000 7f8273fff640 1 -- 172.21.15.143:0/1999956368 --> v1:172.21.15.143:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f8268003530 con 0x7f827c14b1e0 2023-11-26T07:22:41.724 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:41.724+0000 7f8281f7c640 1 -- 172.21.15.143:0/1999956368 >> v2:172.21.15.188:3300/0 conn(0x7f827c0cf990 msgr2=0x7f827c0708c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-26T07:22:41.724 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:41.724+0000 7f8281f7c640 1 --2- 172.21.15.143:0/1999956368 >> v2:172.21.15.188:3300/0 conn(0x7f827c0cf990 0x7f827c0708c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:22:41.724 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:41.724+0000 7f8281f7c640 1 -- 172.21.15.143:0/1999956368 >> v1:172.21.15.143:6789/0 conn(0x7f827c14b1e0 legacy=0x7f827c0cf280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:22:41.725 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:41.724+0000 7f8281f7c640 1 -- 172.21.15.143:0/1999956368 --> v2:172.21.15.143:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f827c071580 con 0x7f827c070e00 2023-11-26T07:22:41.726 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:41.724+0000 7f8281f7c640 1 --2- 172.21.15.143:0/1999956368 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f827c070e00 0x7f827c162510 secure :-1 s=READY pgs=962 cs=0 l=1 rev1=1 crypto rx=0x7f827c06dfa0 tx=0x7f827401c8e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7c034bd188c1e682 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:22:41.726 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:41.724+0000 7f8273fff640 1 -- 172.21.15.143:0/1999956368 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f82740a59a0 con 0x7f827c070e00 2023-11-26T07:22:41.727 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:41.724+0000 7f8273fff640 1 -- 172.21.15.143:0/1999956368 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f82740a5b00 con 0x7f827c070e00 2023-11-26T07:22:41.727 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:41.724+0000 7f8273fff640 1 -- 172.21.15.143:0/1999956368 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f82740a5e70 con 0x7f827c070e00 2023-11-26T07:22:41.727 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:41.724+0000 7f82839df640 1 -- 172.21.15.143:0/1999956368 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f827c070e00 msgr2=0x7f827c162510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:22:41.727 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:41.724+0000 7f82839df640 1 --2- 172.21.15.143:0/1999956368 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f827c070e00 0x7f827c162510 secure :-1 s=READY pgs=962 cs=0 l=1 rev1=1 crypto rx=0x7f827c06dfa0 tx=0x7f827401c8e0 comp rx=0 tx=0).stop 2023-11-26T07:22:41.728 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:41.724+0000 7f82839df640 1 -- 172.21.15.143:0/1999956368 shutdown_connections 2023-11-26T07:22:41.728 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:41.724+0000 7f82839df640 1 --2- 172.21.15.143:0/1999956368 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f827c070e00 0x7f827c162510 unknown :-1 s=CLOSED pgs=962 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:22:41.728 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:41.724+0000 7f82839df640 1 --2- 172.21.15.143:0/1999956368 >> v2:172.21.15.188:3300/0 conn(0x7f827c0cf990 0x7f827c0708c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:22:41.728 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:41.724+0000 7f82839df640 1 -- 172.21.15.143:0/1999956368 >> 172.21.15.143:0/1999956368 conn(0x7f827c06a890 msgr2=0x7f827c06aca0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:22:41.728 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:41.724+0000 7f82839df640 1 -- 172.21.15.143:0/1999956368 shutdown_connections 2023-11-26T07:22:41.728 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:41.728+0000 7f82839df640 1 -- 172.21.15.143:0/1999956368 wait complete. 2023-11-26T07:22:41.729 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:41.728+0000 7f82839df640 1 Processor -- start 2023-11-26T07:22:41.729 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:41.728+0000 7f82839df640 1 -- start start 2023-11-26T07:22:41.731 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:41.728+0000 7f82839df640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f827c070e00 0x7f827c14a380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:22:41.731 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:41.728+0000 7f82839df640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f827c0cf990 0x7f827c160f50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:22:41.731 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:41.728+0000 7f82839df640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f827c14b1e0 0x7f827c15d540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:22:41.731 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:41.728+0000 7f82839df640 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f827c143ce0 con 0x7f827c14b1e0 2023-11-26T07:22:41.732 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:41.728+0000 7f82839df640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f827c14a8c0 con 0x7f827c070e00 2023-11-26T07:22:41.732 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:41.728+0000 7f82839df640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f827c142c40 con 0x7f827c0cf990 2023-11-26T07:22:41.732 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:41.728+0000 7f8281f7c640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f827c14b1e0 0x7f827c15d540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:22:41.732 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:41.728+0000 7f8281f7c640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f827c14b1e0 0x7f827c15d540 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:47276/0 (socket says 172.21.15.143:47276) 2023-11-26T07:22:41.732 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:41.728+0000 7f8281f7c640 1 -- 172.21.15.143:0/3284385513 learned_addr learned my addr 172.21.15.143:0/3284385513 (peer_addr_for_me v2:172.21.15.143:0/0) 2023-11-26T07:22:41.732 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:41.728+0000 7f828177b640 1 --2- 172.21.15.143:0/3284385513 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f827c070e00 0x7f827c14a380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:22:41.732 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:41.728+0000 7f8280f7a640 1 --2- 172.21.15.143:0/3284385513 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f827c0cf990 0x7f827c160f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:22:41.732 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:41.728+0000 7f8281f7c640 1 -- 172.21.15.143:0/3284385513 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f827c0cf990 msgr2=0x7f827c160f50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:22:41.732 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:41.728+0000 7f8281f7c640 1 --2- 172.21.15.143:0/3284385513 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f827c0cf990 0x7f827c160f50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:22:41.732 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:41.728+0000 7f8281f7c640 1 -- 172.21.15.143:0/3284385513 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f827c070e00 msgr2=0x7f827c14a380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:22:41.732 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:41.728+0000 7f8281f7c640 1 --2- 172.21.15.143:0/3284385513 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f827c070e00 0x7f827c14a380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:22:41.733 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:41.728+0000 7f8281f7c640 1 -- 172.21.15.143:0/3284385513 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8274009a70 con 0x7f827c14b1e0 2023-11-26T07:22:41.733 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:41.728+0000 7f8281f7c640 1 --2- 172.21.15.143:0/3284385513 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f827c14b1e0 0x7f827c15d540 secure :-1 s=READY pgs=963 cs=0 l=1 rev1=1 crypto rx=0x7f82740194f0 tx=0x7f8274019680 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:22:41.733 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:41.728+0000 7f8271ffb640 1 -- 172.21.15.143:0/3284385513 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f82740bf070 con 0x7f827c14b1e0 2023-11-26T07:22:41.733 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:41.732+0000 7f8271ffb640 1 -- 172.21.15.143:0/3284385513 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8274009450 con 0x7f827c14b1e0 2023-11-26T07:22:41.733 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:41.732+0000 7f8271ffb640 1 -- 172.21.15.143:0/3284385513 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f82740aee20 con 0x7f827c14b1e0 2023-11-26T07:22:41.733 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:41.732+0000 7f82839df640 1 -- 172.21.15.143:0/3284385513 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f827c142f20 con 0x7f827c14b1e0 2023-11-26T07:22:41.733 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:41.732+0000 7f82839df640 1 -- 172.21.15.143:0/3284385513 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f827c1434e0 con 0x7f827c14b1e0 2023-11-26T07:22:41.734 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:41.732+0000 7f82839df640 1 -- 172.21.15.143:0/3284385513 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f827c067680 con 0x7f827c14b1e0 2023-11-26T07:22:41.734 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:41.732+0000 7f8271ffb640 1 -- 172.21.15.143:0/3284385513 <== mon.0 v2:172.21.15.143:3300/0 4 ==== mgrmap(e 11) v1 ==== 86158+0+0 (secure 0 0 0) 0x7f82740b4050 con 0x7f827c14b1e0 2023-11-26T07:22:41.735 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:41.732+0000 7f8271ffb640 1 --2- 172.21.15.143:0/3284385513 >> [v2:172.21.15.143:6832/2150557544,v1:172.21.15.143:6833/2150557544] conn(0x7f82600667d0 0x7f8260068c90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:22:41.735 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:41.732+0000 7f8271ffb640 1 -- 172.21.15.143:0/3284385513 <== mon.0 v2:172.21.15.143:3300/0 5 ==== osd_map(71..71 src has 1..71) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f827411ebf0 con 0x7f827c14b1e0 2023-11-26T07:22:41.735 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:41.732+0000 7f828177b640 1 --2- 172.21.15.143:0/3284385513 >> [v2:172.21.15.143:6832/2150557544,v1:172.21.15.143:6833/2150557544] conn(0x7f82600667d0 0x7f8260068c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:22:41.735 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:41.736+0000 7f828177b640 1 --2- 172.21.15.143:0/3284385513 >> [v2:172.21.15.143:6832/2150557544,v1:172.21.15.143:6833/2150557544] conn(0x7f82600667d0 0x7f8260068c90 secure :-1 s=READY pgs=452 cs=0 l=1 rev1=1 crypto rx=0x7f8278007fd0 tx=0x7f8278009040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:22:41.738 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:41.736+0000 7f8271ffb640 1 -- 172.21.15.143:0/3284385513 <== mon.0 v2:172.21.15.143:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172649 (secure 0 0 0) 0x7f82740ebcd0 con 0x7f827c14b1e0 2023-11-26T07:22:41.888 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:41.888+0000 7f82839df640 1 -- 172.21.15.143:0/3284385513 --> [v2:172.21.15.143:6832/2150557544,v1:172.21.15.143:6833/2150557544] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f827c0a87e0 con 0x7f82600667d0 2023-11-26T07:22:41.896 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:41.896+0000 7f8271ffb640 1 -- 172.21.15.143:0/3284385513 <== mgr.4100 v2:172.21.15.143:6832/2150557544 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+289343 (secure 0 0 0) 0x7f827c0a87e0 con 0x7f82600667d0 2023-11-26T07:22:41.899 INFO:teuthology.orchestra.run.smithi143.stdout: 2023-11-26T07:22:41.901 INFO:teuthology.orchestra.run.smithi143.stderr:dumped all 2023-11-26T07:22:41.902 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:41.900+0000 7f82839df640 1 -- 172.21.15.143:0/3284385513 >> [v2:172.21.15.143:6832/2150557544,v1:172.21.15.143:6833/2150557544] conn(0x7f82600667d0 msgr2=0x7f8260068c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:22:41.902 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:41.900+0000 7f82839df640 1 --2- 172.21.15.143:0/3284385513 >> [v2:172.21.15.143:6832/2150557544,v1:172.21.15.143:6833/2150557544] conn(0x7f82600667d0 0x7f8260068c90 secure :-1 s=READY pgs=452 cs=0 l=1 rev1=1 crypto rx=0x7f8278007fd0 tx=0x7f8278009040 comp rx=0 tx=0).stop 2023-11-26T07:22:41.902 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:41.900+0000 7f82839df640 1 -- 172.21.15.143:0/3284385513 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f827c14b1e0 msgr2=0x7f827c15d540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:22:41.902 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:41.900+0000 7f82839df640 1 --2- 172.21.15.143:0/3284385513 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f827c14b1e0 0x7f827c15d540 secure :-1 s=READY pgs=963 cs=0 l=1 rev1=1 crypto rx=0x7f82740194f0 tx=0x7f8274019680 comp rx=0 tx=0).stop 2023-11-26T07:22:41.902 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:41.900+0000 7f82839df640 1 -- 172.21.15.143:0/3284385513 shutdown_connections 2023-11-26T07:22:41.903 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:41.900+0000 7f82839df640 1 --2- 172.21.15.143:0/3284385513 >> [v2:172.21.15.143:6832/2150557544,v1:172.21.15.143:6833/2150557544] conn(0x7f82600667d0 0x7f8260068c90 unknown :-1 s=CLOSED pgs=452 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:22:41.903 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:41.900+0000 7f82839df640 1 --2- 172.21.15.143:0/3284385513 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f827c14b1e0 0x7f827c15d540 unknown :-1 s=CLOSED pgs=963 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:22:41.903 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:41.900+0000 7f82839df640 1 --2- 172.21.15.143:0/3284385513 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f827c0cf990 0x7f827c160f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:22:41.903 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:41.900+0000 7f82839df640 1 --2- 172.21.15.143:0/3284385513 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f827c070e00 0x7f827c14a380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:22:41.903 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:41.900+0000 7f82839df640 1 -- 172.21.15.143:0/3284385513 >> 172.21.15.143:0/3284385513 conn(0x7f827c06a890 msgr2=0x7f827c0b12f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:22:41.903 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:41.900+0000 7f82839df640 1 -- 172.21.15.143:0/3284385513 shutdown_connections 2023-11-26T07:22:41.903 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:41.900+0000 7f82839df640 1 -- 172.21.15.143:0/3284385513 wait complete. 2023-11-26T07:22:41.920 INFO:teuthology.orchestra.run.smithi143.stdout:{"pg_ready":true,"pg_map":{"version":214,"stamp":"2023-11-26T07:22:41.757732+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592778,"num_objects":23,"num_object_clones":0,"num_object_copies":46,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":23,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":203,"num_write_kb":2430,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":155,"ondisk_log_size":155,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":177,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":249132,"kb_used_data":7976,"kb_used_omap":12,"kb_used_meta":241011,"kb_avail":754725588,"statfs":{"total":773094113280,"available":772839002112,"internally_reserved":0,"allocated":8167424,"data_stored":4099788,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12749,"internal_metadata":246795827},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":40,"apply_latency_ms":40,"commit_latency_ns":40000000,"apply_latency_ns":40000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2410,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"5.921180"},"pg_stats":[{"pgid":"11.3f","version":"0'0","reported_seq":11,"reported_epoch":71,"state":"active+clean","last_fresh":"2023-11-26T07:22:39.009090+0000","last_change":"2023-11-26T07:22:37.876990+0000","last_active":"2023-11-26T07:22:39.009090+0000","last_peered":"2023-11-26T07:22:39.009090+0000","last_clean":"2023-11-26T07:22:39.009090+0000","last_became_active":"2023-11-26T07:22:37.876609+0000","last_became_peered":"2023-11-26T07:22:37.876609+0000","last_unstale":"2023-11-26T07:22:39.009090+0000","last_undegraded":"2023-11-26T07:22:39.009090+0000","last_fullsized":"2023-11-26T07:22:39.009090+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:22:36.836338+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:22:36.836338+0000","last_clean_scrub_stamp":"2023-11-26T07:22:36.836338+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T11:59:24.349605+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"11.3e","version":"0'0","reported_seq":11,"reported_epoch":71,"state":"active+clean","last_fresh":"2023-11-26T07:22:40.017080+0000","last_change":"2023-11-26T07:22:37.875094+0000","last_active":"2023-11-26T07:22:40.017080+0000","last_peered":"2023-11-26T07:22:40.017080+0000","last_clean":"2023-11-26T07:22:40.017080+0000","last_became_active":"2023-11-26T07:22:37.874754+0000","last_became_peered":"2023-11-26T07:22:37.874754+0000","last_unstale":"2023-11-26T07:22:40.017080+0000","last_undegraded":"2023-11-26T07:22:40.017080+0000","last_fullsized":"2023-11-26T07:22:40.017080+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:22:36.836338+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:22:36.836338+0000","last_clean_scrub_stamp":"2023-11-26T07:22:36.836338+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T08:37:52.234781+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.3d","version":"0'0","reported_seq":11,"reported_epoch":71,"state":"active+clean","last_fresh":"2023-11-26T07:22:40.017971+0000","last_change":"2023-11-26T07:22:37.857076+0000","last_active":"2023-11-26T07:22:40.017971+0000","last_peered":"2023-11-26T07:22:40.017971+0000","last_clean":"2023-11-26T07:22:40.017971+0000","last_became_active":"2023-11-26T07:22:37.856806+0000","last_became_peered":"2023-11-26T07:22:37.856806+0000","last_unstale":"2023-11-26T07:22:40.017971+0000","last_undegraded":"2023-11-26T07:22:40.017971+0000","last_fullsized":"2023-11-26T07:22:40.017971+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:22:36.836338+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:22:36.836338+0000","last_clean_scrub_stamp":"2023-11-26T07:22:36.836338+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T19:09:21.623652+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"11.3c","version":"0'0","reported_seq":11,"reported_epoch":71,"state":"active+clean","last_fresh":"2023-11-26T07:22:38.862976+0000","last_change":"2023-11-26T07:22:37.881413+0000","last_active":"2023-11-26T07:22:38.862976+0000","last_peered":"2023-11-26T07:22:38.862976+0000","last_clean":"2023-11-26T07:22:38.862976+0000","last_became_active":"2023-11-26T07:22:37.880403+0000","last_became_peered":"2023-11-26T07:22:37.880403+0000","last_unstale":"2023-11-26T07:22:38.862976+0000","last_undegraded":"2023-11-26T07:22:38.862976+0000","last_fullsized":"2023-11-26T07:22:38.862976+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:22:36.836338+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:22:36.836338+0000","last_clean_scrub_stamp":"2023-11-26T07:22:36.836338+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T17:20:27.435514+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.3b","version":"0'0","reported_seq":11,"reported_epoch":71,"state":"active+clean","last_fresh":"2023-11-26T07:22:38.861471+0000","last_change":"2023-11-26T07:22:37.869512+0000","last_active":"2023-11-26T07:22:38.861471+0000","last_peered":"2023-11-26T07:22:38.861471+0000","last_clean":"2023-11-26T07:22:38.861471+0000","last_became_active":"2023-11-26T07:22:37.869236+0000","last_became_peered":"2023-11-26T07:22:37.869236+0000","last_unstale":"2023-11-26T07:22:38.861471+0000","last_undegraded":"2023-11-26T07:22:38.861471+0000","last_fullsized":"2023-11-26T07:22:38.861471+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:22:36.836338+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:22:36.836338+0000","last_clean_scrub_stamp":"2023-11-26T07:22:36.836338+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T13:03:21.054144+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"11.3a","version":"0'0","reported_seq":11,"reported_epoch":71,"state":"active+clean","last_fresh":"2023-11-26T07:22:40.016528+0000","last_change":"2023-11-26T07:22:37.867618+0000","last_active":"2023-11-26T07:22:40.016528+0000","last_peered":"2023-11-26T07:22:40.016528+0000","last_clean":"2023-11-26T07:22:40.016528+0000","last_became_active":"2023-11-26T07:22:37.867007+0000","last_became_peered":"2023-11-26T07:22:37.867007+0000","last_unstale":"2023-11-26T07:22:40.016528+0000","last_undegraded":"2023-11-26T07:22:40.016528+0000","last_fullsized":"2023-11-26T07:22:40.016528+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:22:36.836338+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:22:36.836338+0000","last_clean_scrub_stamp":"2023-11-26T07:22:36.836338+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T08:05:50.197932+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.39","version":"0'0","reported_seq":11,"reported_epoch":71,"state":"active+clean","last_fresh":"2023-11-26T07:22:39.479830+0000","last_change":"2023-11-26T07:22:37.868725+0000","last_active":"2023-11-26T07:22:39.479830+0000","last_peered":"2023-11-26T07:22:39.479830+0000","last_clean":"2023-11-26T07:22:39.479830+0000","last_became_active":"2023-11-26T07:22:37.868369+0000","last_became_peered":"2023-11-26T07:22:37.868369+0000","last_unstale":"2023-11-26T07:22:39.479830+0000","last_undegraded":"2023-11-26T07:22:39.479830+0000","last_fullsized":"2023-11-26T07:22:39.479830+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:22:36.836338+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:22:36.836338+0000","last_clean_scrub_stamp":"2023-11-26T07:22:36.836338+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T12:02:44.081433+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"11.38","version":"0'0","reported_seq":11,"reported_epoch":71,"state":"active+clean","last_fresh":"2023-11-26T07:22:40.016667+0000","last_change":"2023-11-26T07:22:37.871042+0000","last_active":"2023-11-26T07:22:40.016667+0000","last_peered":"2023-11-26T07:22:40.016667+0000","last_clean":"2023-11-26T07:22:40.016667+0000","last_became_active":"2023-11-26T07:22:37.869494+0000","last_became_peered":"2023-11-26T07:22:37.869494+0000","last_unstale":"2023-11-26T07:22:40.016667+0000","last_undegraded":"2023-11-26T07:22:40.016667+0000","last_fullsized":"2023-11-26T07:22:40.016667+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:22:36.836338+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:22:36.836338+0000","last_clean_scrub_stamp":"2023-11-26T07:22:36.836338+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T07:55:11.708496+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.37","version":"0'0","reported_seq":11,"reported_epoch":71,"state":"active+clean","last_fresh":"2023-11-26T07:22:38.861863+0000","last_change":"2023-11-26T07:22:37.863093+0000","last_active":"2023-11-26T07:22:38.861863+0000","last_peered":"2023-11-26T07:22:38.861863+0000","last_clean":"2023-11-26T07:22:38.861863+0000","last_became_active":"2023-11-26T07:22:37.862823+0000","last_became_peered":"2023-11-26T07:22:37.862823+0000","last_unstale":"2023-11-26T07:22:38.861863+0000","last_undegraded":"2023-11-26T07:22:38.861863+0000","last_fullsized":"2023-11-26T07:22:38.861863+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:22:36.836338+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:22:36.836338+0000","last_clean_scrub_stamp":"2023-11-26T07:22:36.836338+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T13:38:50.756292+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"11.36","version":"0'0","reported_seq":11,"reported_epoch":71,"state":"active+clean","last_fresh":"2023-11-26T07:22:39.658097+0000","last_change":"2023-11-26T07:22:37.857561+0000","last_active":"2023-11-26T07:22:39.658097+0000","last_peered":"2023-11-26T07:22:39.658097+0000","last_clean":"2023-11-26T07:22:39.658097+0000","last_became_active":"2023-11-26T07:22:37.857181+0000","last_became_peered":"2023-11-26T07:22:37.857181+0000","last_unstale":"2023-11-26T07:22:39.658097+0000","last_undegraded":"2023-11-26T07:22:39.658097+0000","last_fullsized":"2023-11-26T07:22:39.658097+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:22:36.836338+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:22:36.836338+0000","last_clean_scrub_stamp":"2023-11-26T07:22:36.836338+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T11:09:49.677547+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.35","version":"0'0","reported_seq":11,"reported_epoch":71,"state":"active+clean","last_fresh":"2023-11-26T07:22:39.008999+0000","last_change":"2023-11-26T07:22:37.879093+0000","last_active":"2023-11-26T07:22:39.008999+0000","last_peered":"2023-11-26T07:22:39.008999+0000","last_clean":"2023-11-26T07:22:39.008999+0000","last_became_active":"2023-11-26T07:22:37.878330+0000","last_became_peered":"2023-11-26T07:22:37.878330+0000","last_unstale":"2023-11-26T07:22:39.008999+0000","last_undegraded":"2023-11-26T07:22:39.008999+0000","last_fullsized":"2023-11-26T07:22:39.008999+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:22:36.836338+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:22:36.836338+0000","last_clean_scrub_stamp":"2023-11-26T07:22:36.836338+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T14:21:24.831517+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"11.34","version":"0'0","reported_seq":11,"reported_epoch":71,"state":"active+clean","last_fresh":"2023-11-26T07:22:38.862846+0000","last_change":"2023-11-26T07:22:37.868734+0000","last_active":"2023-11-26T07:22:38.862846+0000","last_peered":"2023-11-26T07:22:38.862846+0000","last_clean":"2023-11-26T07:22:38.862846+0000","last_became_active":"2023-11-26T07:22:37.868461+0000","last_became_peered":"2023-11-26T07:22:37.868461+0000","last_unstale":"2023-11-26T07:22:38.862846+0000","last_undegraded":"2023-11-26T07:22:38.862846+0000","last_fullsized":"2023-11-26T07:22:38.862846+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:22:36.836338+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:22:36.836338+0000","last_clean_scrub_stamp":"2023-11-26T07:22:36.836338+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T09:40:51.476908+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.33","version":"0'0","reported_seq":11,"reported_epoch":71,"state":"active+clean","last_fresh":"2023-11-26T07:22:39.864177+0000","last_change":"2023-11-26T07:22:37.865247+0000","last_active":"2023-11-26T07:22:39.864177+0000","last_peered":"2023-11-26T07:22:39.864177+0000","last_clean":"2023-11-26T07:22:39.864177+0000","last_became_active":"2023-11-26T07:22:37.864308+0000","last_became_peered":"2023-11-26T07:22:37.864308+0000","last_unstale":"2023-11-26T07:22:39.864177+0000","last_undegraded":"2023-11-26T07:22:39.864177+0000","last_fullsized":"2023-11-26T07:22:39.864177+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:22:36.836338+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:22:36.836338+0000","last_clean_scrub_stamp":"2023-11-26T07:22:36.836338+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T17:06:11.853975+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.32","version":"0'0","reported_seq":11,"reported_epoch":71,"state":"active+clean","last_fresh":"2023-11-26T07:22:38.861454+0000","last_change":"2023-11-26T07:22:37.863027+0000","last_active":"2023-11-26T07:22:38.861454+0000","last_peered":"2023-11-26T07:22:38.861454+0000","last_clean":"2023-11-26T07:22:38.861454+0000","last_became_active":"2023-11-26T07:22:37.862707+0000","last_became_peered":"2023-11-26T07:22:37.862707+0000","last_unstale":"2023-11-26T07:22:38.861454+0000","last_undegraded":"2023-11-26T07:22:38.861454+0000","last_fullsized":"2023-11-26T07:22:38.861454+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:22:36.836338+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:22:36.836338+0000","last_clean_scrub_stamp":"2023-11-26T07:22:36.836338+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T07:39:30.457576+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"11.31","version":"0'0","reported_seq":11,"reported_epoch":71,"state":"active+clean","last_fresh":"2023-11-26T07:22:39.863999+0000","last_change":"2023-11-26T07:22:37.857331+0000","last_active":"2023-11-26T07:22:39.863999+0000","last_peered":"2023-11-26T07:22:39.863999+0000","last_clean":"2023-11-26T07:22:39.863999+0000","last_became_active":"2023-11-26T07:22:37.857080+0000","last_became_peered":"2023-11-26T07:22:37.857080+0000","last_unstale":"2023-11-26T07:22:39.863999+0000","last_undegraded":"2023-11-26T07:22:39.863999+0000","last_fullsized":"2023-11-26T07:22:39.863999+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:22:36.836338+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:22:36.836338+0000","last_clean_scrub_stamp":"2023-11-26T07:22:36.836338+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T10:28:26.783912+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.30","version":"0'0","reported_seq":11,"reported_epoch":71,"state":"active+clean","last_fresh":"2023-11-26T07:22:39.657680+0000","last_change":"2023-11-26T07:22:37.868553+0000","last_active":"2023-11-26T07:22:39.657680+0000","last_peered":"2023-11-26T07:22:39.657680+0000","last_clean":"2023-11-26T07:22:39.657680+0000","last_became_active":"2023-11-26T07:22:37.868283+0000","last_became_peered":"2023-11-26T07:22:37.868283+0000","last_unstale":"2023-11-26T07:22:39.657680+0000","last_undegraded":"2023-11-26T07:22:39.657680+0000","last_fullsized":"2023-11-26T07:22:39.657680+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:22:36.836338+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:22:36.836338+0000","last_clean_scrub_stamp":"2023-11-26T07:22:36.836338+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T13:57:20.044140+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.2f","version":"0'0","reported_seq":11,"reported_epoch":71,"state":"active+clean","last_fresh":"2023-11-26T07:22:38.862041+0000","last_change":"2023-11-26T07:22:37.862866+0000","last_active":"2023-11-26T07:22:38.862041+0000","last_peered":"2023-11-26T07:22:38.862041+0000","last_clean":"2023-11-26T07:22:38.862041+0000","last_became_active":"2023-11-26T07:22:37.862120+0000","last_became_peered":"2023-11-26T07:22:37.862120+0000","last_unstale":"2023-11-26T07:22:38.862041+0000","last_undegraded":"2023-11-26T07:22:38.862041+0000","last_fullsized":"2023-11-26T07:22:38.862041+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:22:36.836338+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:22:36.836338+0000","last_clean_scrub_stamp":"2023-11-26T07:22:36.836338+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T09:36:10.823109+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"11.2e","version":"0'0","reported_seq":11,"reported_epoch":71,"state":"active+clean","last_fresh":"2023-11-26T07:22:38.862990+0000","last_change":"2023-11-26T07:22:37.879070+0000","last_active":"2023-11-26T07:22:38.862990+0000","last_peered":"2023-11-26T07:22:38.862990+0000","last_clean":"2023-11-26T07:22:38.862990+0000","last_became_active":"2023-11-26T07:22:37.878806+0000","last_became_peered":"2023-11-26T07:22:37.878806+0000","last_unstale":"2023-11-26T07:22:38.862990+0000","last_undegraded":"2023-11-26T07:22:38.862990+0000","last_fullsized":"2023-11-26T07:22:38.862990+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:22:36.836338+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:22:36.836338+0000","last_clean_scrub_stamp":"2023-11-26T07:22:36.836338+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T16:40:17.858078+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.2d","version":"0'0","reported_seq":11,"reported_epoch":71,"state":"active+clean","last_fresh":"2023-11-26T07:22:40.016859+0000","last_change":"2023-11-26T07:22:37.868375+0000","last_active":"2023-11-26T07:22:40.016859+0000","last_peered":"2023-11-26T07:22:40.016859+0000","last_clean":"2023-11-26T07:22:40.016859+0000","last_became_active":"2023-11-26T07:22:37.868104+0000","last_became_peered":"2023-11-26T07:22:37.868104+0000","last_unstale":"2023-11-26T07:22:40.016859+0000","last_undegraded":"2023-11-26T07:22:40.016859+0000","last_fullsized":"2023-11-26T07:22:40.016859+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:22:36.836338+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:22:36.836338+0000","last_clean_scrub_stamp":"2023-11-26T07:22:36.836338+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T12:02:44.081433+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.2c","version":"0'0","reported_seq":11,"reported_epoch":71,"state":"active+clean","last_fresh":"2023-11-26T07:22:39.864201+0000","last_change":"2023-11-26T07:22:37.867875+0000","last_active":"2023-11-26T07:22:39.864201+0000","last_peered":"2023-11-26T07:22:39.864201+0000","last_clean":"2023-11-26T07:22:39.864201+0000","last_became_active":"2023-11-26T07:22:37.867362+0000","last_became_peered":"2023-11-26T07:22:37.867362+0000","last_unstale":"2023-11-26T07:22:39.864201+0000","last_undegraded":"2023-11-26T07:22:39.864201+0000","last_fullsized":"2023-11-26T07:22:39.864201+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:22:36.836338+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:22:36.836338+0000","last_clean_scrub_stamp":"2023-11-26T07:22:36.836338+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T14:29:02.649975+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.2b","version":"0'0","reported_seq":11,"reported_epoch":71,"state":"active+clean","last_fresh":"2023-11-26T07:22:39.657916+0000","last_change":"2023-11-26T07:22:37.853932+0000","last_active":"2023-11-26T07:22:39.657916+0000","last_peered":"2023-11-26T07:22:39.657916+0000","last_clean":"2023-11-26T07:22:39.657916+0000","last_became_active":"2023-11-26T07:22:37.853624+0000","last_became_peered":"2023-11-26T07:22:37.853624+0000","last_unstale":"2023-11-26T07:22:39.657916+0000","last_undegraded":"2023-11-26T07:22:39.657916+0000","last_fullsized":"2023-11-26T07:22:39.657916+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:22:36.836338+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:22:36.836338+0000","last_clean_scrub_stamp":"2023-11-26T07:22:36.836338+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T08:17:38.785700+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.2a","version":"0'0","reported_seq":11,"reported_epoch":71,"state":"active+clean","last_fresh":"2023-11-26T07:22:39.657855+0000","last_change":"2023-11-26T07:22:37.857910+0000","last_active":"2023-11-26T07:22:39.657855+0000","last_peered":"2023-11-26T07:22:39.657855+0000","last_clean":"2023-11-26T07:22:39.657855+0000","last_became_active":"2023-11-26T07:22:37.857615+0000","last_became_peered":"2023-11-26T07:22:37.857615+0000","last_unstale":"2023-11-26T07:22:39.657855+0000","last_undegraded":"2023-11-26T07:22:39.657855+0000","last_fullsized":"2023-11-26T07:22:39.657855+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:22:36.836338+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:22:36.836338+0000","last_clean_scrub_stamp":"2023-11-26T07:22:36.836338+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T10:25:42.181123+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.29","version":"0'0","reported_seq":11,"reported_epoch":71,"state":"active+clean","last_fresh":"2023-11-26T07:22:39.008659+0000","last_change":"2023-11-26T07:22:37.870429+0000","last_active":"2023-11-26T07:22:39.008659+0000","last_peered":"2023-11-26T07:22:39.008659+0000","last_clean":"2023-11-26T07:22:39.008659+0000","last_became_active":"2023-11-26T07:22:37.870047+0000","last_became_peered":"2023-11-26T07:22:37.870047+0000","last_unstale":"2023-11-26T07:22:39.008659+0000","last_undegraded":"2023-11-26T07:22:39.008659+0000","last_fullsized":"2023-11-26T07:22:39.008659+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:22:36.836338+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:22:36.836338+0000","last_clean_scrub_stamp":"2023-11-26T07:22:36.836338+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T17:06:11.853975+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"11.28","version":"0'0","reported_seq":11,"reported_epoch":71,"state":"active+clean","last_fresh":"2023-11-26T07:22:39.008836+0000","last_change":"2023-11-26T07:22:37.881994+0000","last_active":"2023-11-26T07:22:39.008836+0000","last_peered":"2023-11-26T07:22:39.008836+0000","last_clean":"2023-11-26T07:22:39.008836+0000","last_became_active":"2023-11-26T07:22:37.881669+0000","last_became_peered":"2023-11-26T07:22:37.881669+0000","last_unstale":"2023-11-26T07:22:39.008836+0000","last_undegraded":"2023-11-26T07:22:39.008836+0000","last_fullsized":"2023-11-26T07:22:39.008836+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:22:36.836338+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:22:36.836338+0000","last_clean_scrub_stamp":"2023-11-26T07:22:36.836338+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T15:34:25.091425+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"11.27","version":"0'0","reported_seq":11,"reported_epoch":71,"state":"active+clean","last_fresh":"2023-11-26T07:22:39.658257+0000","last_change":"2023-11-26T07:22:37.872517+0000","last_active":"2023-11-26T07:22:39.658257+0000","last_peered":"2023-11-26T07:22:39.658257+0000","last_clean":"2023-11-26T07:22:39.658257+0000","last_became_active":"2023-11-26T07:22:37.872228+0000","last_became_peered":"2023-11-26T07:22:37.872228+0000","last_unstale":"2023-11-26T07:22:39.658257+0000","last_undegraded":"2023-11-26T07:22:39.658257+0000","last_fullsized":"2023-11-26T07:22:39.658257+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:22:36.836338+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:22:36.836338+0000","last_clean_scrub_stamp":"2023-11-26T07:22:36.836338+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T16:05:00.044056+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.26","version":"0'0","reported_seq":11,"reported_epoch":71,"state":"active+clean","last_fresh":"2023-11-26T07:22:39.480219+0000","last_change":"2023-11-26T07:22:37.878497+0000","last_active":"2023-11-26T07:22:39.480219+0000","last_peered":"2023-11-26T07:22:39.480219+0000","last_clean":"2023-11-26T07:22:39.480219+0000","last_became_active":"2023-11-26T07:22:37.878209+0000","last_became_peered":"2023-11-26T07:22:37.878209+0000","last_unstale":"2023-11-26T07:22:39.480219+0000","last_undegraded":"2023-11-26T07:22:39.480219+0000","last_fullsized":"2023-11-26T07:22:39.480219+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:22:36.836338+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:22:36.836338+0000","last_clean_scrub_stamp":"2023-11-26T07:22:36.836338+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T13:34:41.619362+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"11.25","version":"0'0","reported_seq":11,"reported_epoch":71,"state":"active+clean","last_fresh":"2023-11-26T07:22:40.017975+0000","last_change":"2023-11-26T07:22:37.873393+0000","last_active":"2023-11-26T07:22:40.017975+0000","last_peered":"2023-11-26T07:22:40.017975+0000","last_clean":"2023-11-26T07:22:40.017975+0000","last_became_active":"2023-11-26T07:22:37.873080+0000","last_became_peered":"2023-11-26T07:22:37.873080+0000","last_unstale":"2023-11-26T07:22:40.017975+0000","last_undegraded":"2023-11-26T07:22:40.017975+0000","last_fullsized":"2023-11-26T07:22:40.017975+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:22:36.836338+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:22:36.836338+0000","last_clean_scrub_stamp":"2023-11-26T07:22:36.836338+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T17:33:46.796820+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"11.24","version":"0'0","reported_seq":11,"reported_epoch":71,"state":"active+clean","last_fresh":"2023-11-26T07:22:39.658115+0000","last_change":"2023-11-26T07:22:37.857135+0000","last_active":"2023-11-26T07:22:39.658115+0000","last_peered":"2023-11-26T07:22:39.658115+0000","last_clean":"2023-11-26T07:22:39.658115+0000","last_became_active":"2023-11-26T07:22:37.856836+0000","last_became_peered":"2023-11-26T07:22:37.856836+0000","last_unstale":"2023-11-26T07:22:39.658115+0000","last_undegraded":"2023-11-26T07:22:39.658115+0000","last_fullsized":"2023-11-26T07:22:39.658115+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:22:36.836338+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:22:36.836338+0000","last_clean_scrub_stamp":"2023-11-26T07:22:36.836338+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T09:24:31.774376+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.23","version":"0'0","reported_seq":11,"reported_epoch":71,"state":"active+clean","last_fresh":"2023-11-26T07:22:40.017775+0000","last_change":"2023-11-26T07:22:37.869946+0000","last_active":"2023-11-26T07:22:40.017775+0000","last_peered":"2023-11-26T07:22:40.017775+0000","last_clean":"2023-11-26T07:22:40.017775+0000","last_became_active":"2023-11-26T07:22:37.869701+0000","last_became_peered":"2023-11-26T07:22:37.869701+0000","last_unstale":"2023-11-26T07:22:40.017775+0000","last_undegraded":"2023-11-26T07:22:40.017775+0000","last_fullsized":"2023-11-26T07:22:40.017775+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:22:36.836338+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:22:36.836338+0000","last_clean_scrub_stamp":"2023-11-26T07:22:36.836338+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T13:15:12.583356+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"11.22","version":"0'0","reported_seq":11,"reported_epoch":71,"state":"active+clean","last_fresh":"2023-11-26T07:22:39.479919+0000","last_change":"2023-11-26T07:22:37.878295+0000","last_active":"2023-11-26T07:22:39.479919+0000","last_peered":"2023-11-26T07:22:39.479919+0000","last_clean":"2023-11-26T07:22:39.479919+0000","last_became_active":"2023-11-26T07:22:37.877997+0000","last_became_peered":"2023-11-26T07:22:37.877997+0000","last_unstale":"2023-11-26T07:22:39.479919+0000","last_undegraded":"2023-11-26T07:22:39.479919+0000","last_fullsized":"2023-11-26T07:22:39.479919+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:22:36.836338+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:22:36.836338+0000","last_clean_scrub_stamp":"2023-11-26T07:22:36.836338+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T08:44:15.104642+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"11.21","version":"0'0","reported_seq":11,"reported_epoch":71,"state":"active+clean","last_fresh":"2023-11-26T07:22:39.480040+0000","last_change":"2023-11-26T07:22:37.864587+0000","last_active":"2023-11-26T07:22:39.480040+0000","last_peered":"2023-11-26T07:22:39.480040+0000","last_clean":"2023-11-26T07:22:39.480040+0000","last_became_active":"2023-11-26T07:22:37.864015+0000","last_became_peered":"2023-11-26T07:22:37.864015+0000","last_unstale":"2023-11-26T07:22:39.480040+0000","last_undegraded":"2023-11-26T07:22:39.480040+0000","last_fullsized":"2023-11-26T07:22:39.480040+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:22:36.836338+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:22:36.836338+0000","last_clean_scrub_stamp":"2023-11-26T07:22:36.836338+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T14:52:44.124161+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"11.20","version":"0'0","reported_seq":11,"reported_epoch":71,"state":"active+clean","last_fresh":"2023-11-26T07:22:40.016651+0000","last_change":"2023-11-26T07:22:37.878621+0000","last_active":"2023-11-26T07:22:40.016651+0000","last_peered":"2023-11-26T07:22:40.016651+0000","last_clean":"2023-11-26T07:22:40.016651+0000","last_became_active":"2023-11-26T07:22:37.878299+0000","last_became_peered":"2023-11-26T07:22:37.878299+0000","last_unstale":"2023-11-26T07:22:40.016651+0000","last_undegraded":"2023-11-26T07:22:40.016651+0000","last_fullsized":"2023-11-26T07:22:40.016651+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:22:36.836338+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:22:36.836338+0000","last_clean_scrub_stamp":"2023-11-26T07:22:36.836338+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T09:24:31.774376+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.1e","version":"0'0","reported_seq":11,"reported_epoch":71,"state":"active+clean","last_fresh":"2023-11-26T07:22:39.009057+0000","last_change":"2023-11-26T07:22:37.880169+0000","last_active":"2023-11-26T07:22:39.009057+0000","last_peered":"2023-11-26T07:22:39.009057+0000","last_clean":"2023-11-26T07:22:39.009057+0000","last_became_active":"2023-11-26T07:22:37.878729+0000","last_became_peered":"2023-11-26T07:22:37.878729+0000","last_unstale":"2023-11-26T07:22:39.009057+0000","last_undegraded":"2023-11-26T07:22:39.009057+0000","last_fullsized":"2023-11-26T07:22:39.009057+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:22:36.836338+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:22:36.836338+0000","last_clean_scrub_stamp":"2023-11-26T07:22:36.836338+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T14:15:53.054036+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.1f","version":"70'4","reported_seq":16,"reported_epoch":70,"state":"active+clean","last_fresh":"2023-11-26T07:22:37.890541+0000","last_change":"2023-11-26T07:22:36.875269+0000","last_active":"2023-11-26T07:22:37.890541+0000","last_peered":"2023-11-26T07:22:37.890541+0000","last_clean":"2023-11-26T07:22:37.890541+0000","last_became_active":"2023-11-26T07:22:36.874786+0000","last_became_peered":"2023-11-26T07:22:36.874786+0000","last_unstale":"2023-11-26T07:22:37.890541+0000","last_undegraded":"2023-11-26T07:22:37.890541+0000","last_fullsized":"2023-11-26T07:22:37.890541+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:22:35.829702+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:22:35.829702+0000","last_clean_scrub_stamp":"2023-11-26T07:22:35.829702+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T18:23:53.779522+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"11.1f","version":"0'0","reported_seq":11,"reported_epoch":71,"state":"active+clean","last_fresh":"2023-11-26T07:22:38.862159+0000","last_change":"2023-11-26T07:22:37.863028+0000","last_active":"2023-11-26T07:22:38.862159+0000","last_peered":"2023-11-26T07:22:38.862159+0000","last_clean":"2023-11-26T07:22:38.862159+0000","last_became_active":"2023-11-26T07:22:37.862462+0000","last_became_peered":"2023-11-26T07:22:37.862462+0000","last_unstale":"2023-11-26T07:22:38.862159+0000","last_undegraded":"2023-11-26T07:22:38.862159+0000","last_fullsized":"2023-11-26T07:22:38.862159+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:22:36.836338+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:22:36.836338+0000","last_clean_scrub_stamp":"2023-11-26T07:22:36.836338+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T13:31:06.990149+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"10.1e","version":"70'1","reported_seq":13,"reported_epoch":70,"state":"active+clean","last_fresh":"2023-11-26T07:22:37.881672+0000","last_change":"2023-11-26T07:22:36.865454+0000","last_active":"2023-11-26T07:22:37.881672+0000","last_peered":"2023-11-26T07:22:37.881672+0000","last_clean":"2023-11-26T07:22:37.881672+0000","last_became_active":"2023-11-26T07:22:36.865055+0000","last_became_peered":"2023-11-26T07:22:36.865055+0000","last_unstale":"2023-11-26T07:22:37.881672+0000","last_undegraded":"2023-11-26T07:22:37.881672+0000","last_fullsized":"2023-11-26T07:22:37.881672+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:22:35.829702+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:22:35.829702+0000","last_clean_scrub_stamp":"2023-11-26T07:22:35.829702+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T11:49:26.867950+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.1c","version":"0'0","reported_seq":11,"reported_epoch":71,"state":"active+clean","last_fresh":"2023-11-26T07:22:40.016774+0000","last_change":"2023-11-26T07:22:37.879395+0000","last_active":"2023-11-26T07:22:40.016774+0000","last_peered":"2023-11-26T07:22:40.016774+0000","last_clean":"2023-11-26T07:22:40.016774+0000","last_became_active":"2023-11-26T07:22:37.879033+0000","last_became_peered":"2023-11-26T07:22:37.879033+0000","last_unstale":"2023-11-26T07:22:40.016774+0000","last_undegraded":"2023-11-26T07:22:40.016774+0000","last_fullsized":"2023-11-26T07:22:40.016774+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:22:36.836338+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:22:36.836338+0000","last_clean_scrub_stamp":"2023-11-26T07:22:36.836338+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T16:05:00.044056+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"10.1d","version":"70'2","reported_seq":14,"reported_epoch":70,"state":"active+clean","last_fresh":"2023-11-26T07:22:37.888967+0000","last_change":"2023-11-26T07:22:36.862194+0000","last_active":"2023-11-26T07:22:37.888967+0000","last_peered":"2023-11-26T07:22:37.888967+0000","last_clean":"2023-11-26T07:22:37.888967+0000","last_became_active":"2023-11-26T07:22:36.860756+0000","last_became_peered":"2023-11-26T07:22:36.860756+0000","last_unstale":"2023-11-26T07:22:37.888967+0000","last_undegraded":"2023-11-26T07:22:37.888967+0000","last_fullsized":"2023-11-26T07:22:37.888967+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:22:35.829702+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:22:35.829702+0000","last_clean_scrub_stamp":"2023-11-26T07:22:35.829702+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T16:57:10.472680+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.1d","version":"0'0","reported_seq":11,"reported_epoch":71,"state":"active+clean","last_fresh":"2023-11-26T07:22:39.009101+0000","last_change":"2023-11-26T07:22:37.874358+0000","last_active":"2023-11-26T07:22:39.009101+0000","last_peered":"2023-11-26T07:22:39.009101+0000","last_clean":"2023-11-26T07:22:39.009101+0000","last_became_active":"2023-11-26T07:22:37.874067+0000","last_became_peered":"2023-11-26T07:22:37.874067+0000","last_unstale":"2023-11-26T07:22:39.009101+0000","last_undegraded":"2023-11-26T07:22:39.009101+0000","last_fullsized":"2023-11-26T07:22:39.009101+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:22:36.836338+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:22:36.836338+0000","last_clean_scrub_stamp":"2023-11-26T07:22:36.836338+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T17:24:30.176302+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.1c","version":"70'2","reported_seq":14,"reported_epoch":70,"state":"active+clean","last_fresh":"2023-11-26T07:22:37.892791+0000","last_change":"2023-11-26T07:22:36.862999+0000","last_active":"2023-11-26T07:22:37.892791+0000","last_peered":"2023-11-26T07:22:37.892791+0000","last_clean":"2023-11-26T07:22:37.892791+0000","last_became_active":"2023-11-26T07:22:36.862710+0000","last_became_peered":"2023-11-26T07:22:36.862710+0000","last_unstale":"2023-11-26T07:22:37.892791+0000","last_undegraded":"2023-11-26T07:22:37.892791+0000","last_fullsized":"2023-11-26T07:22:37.892791+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:22:35.829702+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:22:35.829702+0000","last_clean_scrub_stamp":"2023-11-26T07:22:35.829702+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T16:57:10.472680+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.1a","version":"0'0","reported_seq":11,"reported_epoch":71,"state":"active+clean","last_fresh":"2023-11-26T07:22:39.480018+0000","last_change":"2023-11-26T07:22:37.874585+0000","last_active":"2023-11-26T07:22:39.480018+0000","last_peered":"2023-11-26T07:22:39.480018+0000","last_clean":"2023-11-26T07:22:39.480018+0000","last_became_active":"2023-11-26T07:22:37.873823+0000","last_became_peered":"2023-11-26T07:22:37.873823+0000","last_unstale":"2023-11-26T07:22:39.480018+0000","last_undegraded":"2023-11-26T07:22:39.480018+0000","last_fullsized":"2023-11-26T07:22:39.480018+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:22:36.836338+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:22:36.836338+0000","last_clean_scrub_stamp":"2023-11-26T07:22:36.836338+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T08:37:52.234781+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.1b","version":"70'1","reported_seq":13,"reported_epoch":70,"state":"active+clean","last_fresh":"2023-11-26T07:22:37.887545+0000","last_change":"2023-11-26T07:22:36.860364+0000","last_active":"2023-11-26T07:22:37.887545+0000","last_peered":"2023-11-26T07:22:37.887545+0000","last_clean":"2023-11-26T07:22:37.887545+0000","last_became_active":"2023-11-26T07:22:36.860002+0000","last_became_peered":"2023-11-26T07:22:36.860002+0000","last_unstale":"2023-11-26T07:22:37.887545+0000","last_undegraded":"2023-11-26T07:22:37.887545+0000","last_fullsized":"2023-11-26T07:22:37.887545+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:22:35.829702+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:22:35.829702+0000","last_clean_scrub_stamp":"2023-11-26T07:22:35.829702+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T09:11:42.010855+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"11.1b","version":"0'0","reported_seq":11,"reported_epoch":71,"state":"active+clean","last_fresh":"2023-11-26T07:22:39.658028+0000","last_change":"2023-11-26T07:22:37.874997+0000","last_active":"2023-11-26T07:22:39.658028+0000","last_peered":"2023-11-26T07:22:39.658028+0000","last_clean":"2023-11-26T07:22:39.658028+0000","last_became_active":"2023-11-26T07:22:37.874544+0000","last_became_peered":"2023-11-26T07:22:37.874544+0000","last_unstale":"2023-11-26T07:22:39.658028+0000","last_undegraded":"2023-11-26T07:22:39.658028+0000","last_fullsized":"2023-11-26T07:22:39.658028+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:22:36.836338+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:22:36.836338+0000","last_clean_scrub_stamp":"2023-11-26T07:22:36.836338+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T08:05:50.197932+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"10.1a","version":"70'1","reported_seq":13,"reported_epoch":70,"state":"active+clean","last_fresh":"2023-11-26T07:22:37.892156+0000","last_change":"2023-11-26T07:22:36.866439+0000","last_active":"2023-11-26T07:22:37.892156+0000","last_peered":"2023-11-26T07:22:37.892156+0000","last_clean":"2023-11-26T07:22:37.892156+0000","last_became_active":"2023-11-26T07:22:36.865935+0000","last_became_peered":"2023-11-26T07:22:36.865935+0000","last_unstale":"2023-11-26T07:22:37.892156+0000","last_undegraded":"2023-11-26T07:22:37.892156+0000","last_fullsized":"2023-11-26T07:22:37.892156+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:22:35.829702+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:22:35.829702+0000","last_clean_scrub_stamp":"2023-11-26T07:22:35.829702+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T17:20:26.428878+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.18","version":"0'0","reported_seq":11,"reported_epoch":71,"state":"active+clean","last_fresh":"2023-11-26T07:22:38.861163+0000","last_change":"2023-11-26T07:22:37.865874+0000","last_active":"2023-11-26T07:22:38.861163+0000","last_peered":"2023-11-26T07:22:38.861163+0000","last_clean":"2023-11-26T07:22:38.861163+0000","last_became_active":"2023-11-26T07:22:37.865490+0000","last_became_peered":"2023-11-26T07:22:37.865490+0000","last_unstale":"2023-11-26T07:22:38.861163+0000","last_undegraded":"2023-11-26T07:22:38.861163+0000","last_fullsized":"2023-11-26T07:22:38.861163+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:22:36.836338+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:22:36.836338+0000","last_clean_scrub_stamp":"2023-11-26T07:22:36.836338+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T15:21:22.997055+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"10.19","version":"0'0","reported_seq":12,"reported_epoch":70,"state":"active+clean","last_fresh":"2023-11-26T07:22:37.861366+0000","last_change":"2023-11-26T07:22:36.861455+0000","last_active":"2023-11-26T07:22:37.861366+0000","last_peered":"2023-11-26T07:22:37.861366+0000","last_clean":"2023-11-26T07:22:37.861366+0000","last_became_active":"2023-11-26T07:22:36.861128+0000","last_became_peered":"2023-11-26T07:22:36.861128+0000","last_unstale":"2023-11-26T07:22:37.861366+0000","last_undegraded":"2023-11-26T07:22:37.861366+0000","last_fullsized":"2023-11-26T07:22:37.861366+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:22:35.829702+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:22:35.829702+0000","last_clean_scrub_stamp":"2023-11-26T07:22:35.829702+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T14:55:45.237720+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.19","version":"0'0","reported_seq":11,"reported_epoch":71,"state":"active+clean","last_fresh":"2023-11-26T07:22:39.864073+0000","last_change":"2023-11-26T07:22:37.873372+0000","last_active":"2023-11-26T07:22:39.864073+0000","last_peered":"2023-11-26T07:22:39.864073+0000","last_clean":"2023-11-26T07:22:39.864073+0000","last_became_active":"2023-11-26T07:22:37.873033+0000","last_became_peered":"2023-11-26T07:22:37.873033+0000","last_unstale":"2023-11-26T07:22:39.864073+0000","last_undegraded":"2023-11-26T07:22:39.864073+0000","last_fullsized":"2023-11-26T07:22:39.864073+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:22:36.836338+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:22:36.836338+0000","last_clean_scrub_stamp":"2023-11-26T07:22:36.836338+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T14:34:41.099035+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"10.18","version":"0'0","reported_seq":12,"reported_epoch":70,"state":"active+clean","last_fresh":"2023-11-26T07:22:37.855379+0000","last_change":"2023-11-26T07:22:36.859199+0000","last_active":"2023-11-26T07:22:37.855379+0000","last_peered":"2023-11-26T07:22:37.855379+0000","last_clean":"2023-11-26T07:22:37.855379+0000","last_became_active":"2023-11-26T07:22:36.858925+0000","last_became_peered":"2023-11-26T07:22:36.858925+0000","last_unstale":"2023-11-26T07:22:37.855379+0000","last_undegraded":"2023-11-26T07:22:37.855379+0000","last_fullsized":"2023-11-26T07:22:37.855379+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:22:35.829702+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:22:35.829702+0000","last_clean_scrub_stamp":"2023-11-26T07:22:35.829702+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T14:22:02.210852+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"11.16","version":"0'0","reported_seq":11,"reported_epoch":71,"state":"active+clean","last_fresh":"2023-11-26T07:22:39.480277+0000","last_change":"2023-11-26T07:22:37.873279+0000","last_active":"2023-11-26T07:22:39.480277+0000","last_peered":"2023-11-26T07:22:39.480277+0000","last_clean":"2023-11-26T07:22:39.480277+0000","last_became_active":"2023-11-26T07:22:37.872909+0000","last_became_peered":"2023-11-26T07:22:37.872909+0000","last_unstale":"2023-11-26T07:22:39.480277+0000","last_undegraded":"2023-11-26T07:22:39.480277+0000","last_fullsized":"2023-11-26T07:22:39.480277+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:22:36.836338+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:22:36.836338+0000","last_clean_scrub_stamp":"2023-11-26T07:22:36.836338+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T10:53:10.511974+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.17","version":"0'0","reported_seq":12,"reported_epoch":70,"state":"active+clean","last_fresh":"2023-11-26T07:22:37.850457+0000","last_change":"2023-11-26T07:22:36.854554+0000","last_active":"2023-11-26T07:22:37.850457+0000","last_peered":"2023-11-26T07:22:37.850457+0000","last_clean":"2023-11-26T07:22:37.850457+0000","last_became_active":"2023-11-26T07:22:36.854192+0000","last_became_peered":"2023-11-26T07:22:36.854192+0000","last_unstale":"2023-11-26T07:22:37.850457+0000","last_undegraded":"2023-11-26T07:22:37.850457+0000","last_fullsized":"2023-11-26T07:22:37.850457+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:22:35.829702+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:22:35.829702+0000","last_clean_scrub_stamp":"2023-11-26T07:22:35.829702+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T07:41:05.117550+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.17","version":"0'0","reported_seq":11,"reported_epoch":71,"state":"active+clean","last_fresh":"2023-11-26T07:22:39.009202+0000","last_change":"2023-11-26T07:22:37.879717+0000","last_active":"2023-11-26T07:22:39.009202+0000","last_peered":"2023-11-26T07:22:39.009202+0000","last_clean":"2023-11-26T07:22:39.009202+0000","last_became_active":"2023-11-26T07:22:37.879423+0000","last_became_peered":"2023-11-26T07:22:37.879423+0000","last_unstale":"2023-11-26T07:22:39.009202+0000","last_undegraded":"2023-11-26T07:22:39.009202+0000","last_fullsized":"2023-11-26T07:22:39.009202+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:22:36.836338+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:22:36.836338+0000","last_clean_scrub_stamp":"2023-11-26T07:22:36.836338+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T12:42:37.711199+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.16","version":"0'0","reported_seq":12,"reported_epoch":70,"state":"active+clean","last_fresh":"2023-11-26T07:22:37.867466+0000","last_change":"2023-11-26T07:22:36.867429+0000","last_active":"2023-11-26T07:22:37.867466+0000","last_peered":"2023-11-26T07:22:37.867466+0000","last_clean":"2023-11-26T07:22:37.867466+0000","last_became_active":"2023-11-26T07:22:36.866921+0000","last_became_peered":"2023-11-26T07:22:36.866921+0000","last_unstale":"2023-11-26T07:22:37.867466+0000","last_undegraded":"2023-11-26T07:22:37.867466+0000","last_fullsized":"2023-11-26T07:22:37.867466+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:22:35.829702+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:22:35.829702+0000","last_clean_scrub_stamp":"2023-11-26T07:22:35.829702+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T10:25:41.174487+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.14","version":"0'0","reported_seq":11,"reported_epoch":71,"state":"active+clean","last_fresh":"2023-11-26T07:22:40.016991+0000","last_change":"2023-11-26T07:22:37.866132+0000","last_active":"2023-11-26T07:22:40.016991+0000","last_peered":"2023-11-26T07:22:40.016991+0000","last_clean":"2023-11-26T07:22:40.016991+0000","last_became_active":"2023-11-26T07:22:37.865208+0000","last_became_peered":"2023-11-26T07:22:37.865208+0000","last_unstale":"2023-11-26T07:22:40.016991+0000","last_undegraded":"2023-11-26T07:22:40.016991+0000","last_fullsized":"2023-11-26T07:22:40.016991+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:22:36.836338+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:22:36.836338+0000","last_clean_scrub_stamp":"2023-11-26T07:22:36.836338+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T08:44:15.104642+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"10.15","version":"70'3","reported_seq":15,"reported_epoch":70,"state":"active+clean","last_fresh":"2023-11-26T07:22:37.891482+0000","last_change":"2023-11-26T07:22:36.867479+0000","last_active":"2023-11-26T07:22:37.891482+0000","last_peered":"2023-11-26T07:22:37.891482+0000","last_clean":"2023-11-26T07:22:37.891482+0000","last_became_active":"2023-11-26T07:22:36.867227+0000","last_became_peered":"2023-11-26T07:22:36.867227+0000","last_unstale":"2023-11-26T07:22:37.891482+0000","last_undegraded":"2023-11-26T07:22:37.891482+0000","last_fullsized":"2023-11-26T07:22:37.891482+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:22:35.829702+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:22:35.829702+0000","last_clean_scrub_stamp":"2023-11-26T07:22:35.829702+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T11:59:23.342969+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"11.15","version":"0'0","reported_seq":11,"reported_epoch":71,"state":"active+clean","last_fresh":"2023-11-26T07:22:38.861785+0000","last_change":"2023-11-26T07:22:37.869962+0000","last_active":"2023-11-26T07:22:38.861785+0000","last_peered":"2023-11-26T07:22:38.861785+0000","last_clean":"2023-11-26T07:22:38.861785+0000","last_became_active":"2023-11-26T07:22:37.869699+0000","last_became_peered":"2023-11-26T07:22:37.869699+0000","last_unstale":"2023-11-26T07:22:38.861785+0000","last_undegraded":"2023-11-26T07:22:38.861785+0000","last_fullsized":"2023-11-26T07:22:38.861785+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:22:36.836338+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:22:36.836338+0000","last_clean_scrub_stamp":"2023-11-26T07:22:36.836338+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T08:29:30.155972+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"10.14","version":"70'6","reported_seq":18,"reported_epoch":70,"state":"active+clean","last_fresh":"2023-11-26T07:22:37.894032+0000","last_change":"2023-11-26T07:22:36.864408+0000","last_active":"2023-11-26T07:22:37.894032+0000","last_peered":"2023-11-26T07:22:37.894032+0000","last_clean":"2023-11-26T07:22:37.894032+0000","last_became_active":"2023-11-26T07:22:36.864120+0000","last_became_peered":"2023-11-26T07:22:36.864120+0000","last_unstale":"2023-11-26T07:22:37.894032+0000","last_undegraded":"2023-11-26T07:22:37.894032+0000","last_fullsized":"2023-11-26T07:22:37.894032+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:22:35.829702+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:22:35.829702+0000","last_clean_scrub_stamp":"2023-11-26T07:22:35.829702+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T19:06:39.025778+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.12","version":"0'0","reported_seq":11,"reported_epoch":71,"state":"active+clean","last_fresh":"2023-11-26T07:22:39.008663+0000","last_change":"2023-11-26T07:22:37.868503+0000","last_active":"2023-11-26T07:22:39.008663+0000","last_peered":"2023-11-26T07:22:39.008663+0000","last_clean":"2023-11-26T07:22:39.008663+0000","last_became_active":"2023-11-26T07:22:37.868221+0000","last_became_peered":"2023-11-26T07:22:37.868221+0000","last_unstale":"2023-11-26T07:22:39.008663+0000","last_undegraded":"2023-11-26T07:22:39.008663+0000","last_fullsized":"2023-11-26T07:22:39.008663+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:22:36.836338+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:22:36.836338+0000","last_clean_scrub_stamp":"2023-11-26T07:22:36.836338+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T14:29:02.649975+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.13","version":"70'2","reported_seq":14,"reported_epoch":70,"state":"active+clean","last_fresh":"2023-11-26T07:22:37.891499+0000","last_change":"2023-11-26T07:22:36.871188+0000","last_active":"2023-11-26T07:22:37.891499+0000","last_peered":"2023-11-26T07:22:37.891499+0000","last_clean":"2023-11-26T07:22:37.891499+0000","last_became_active":"2023-11-26T07:22:36.870926+0000","last_became_peered":"2023-11-26T07:22:36.870926+0000","last_unstale":"2023-11-26T07:22:37.891499+0000","last_undegraded":"2023-11-26T07:22:37.891499+0000","last_fullsized":"2023-11-26T07:22:37.891499+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:22:35.829702+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:22:35.829702+0000","last_clean_scrub_stamp":"2023-11-26T07:22:35.829702+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T10:56:59.018095+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.13","version":"0'0","reported_seq":11,"reported_epoch":71,"state":"active+clean","last_fresh":"2023-11-26T07:22:39.480015+0000","last_change":"2023-11-26T07:22:37.871038+0000","last_active":"2023-11-26T07:22:39.480015+0000","last_peered":"2023-11-26T07:22:39.480015+0000","last_clean":"2023-11-26T07:22:39.480015+0000","last_became_active":"2023-11-26T07:22:37.870441+0000","last_became_peered":"2023-11-26T07:22:37.870441+0000","last_unstale":"2023-11-26T07:22:39.480015+0000","last_undegraded":"2023-11-26T07:22:39.480015+0000","last_fullsized":"2023-11-26T07:22:39.480015+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:22:36.836338+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:22:36.836338+0000","last_clean_scrub_stamp":"2023-11-26T07:22:36.836338+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T15:03:26.446628+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.12","version":"0'0","reported_seq":12,"reported_epoch":70,"state":"active+clean","last_fresh":"2023-11-26T07:22:37.865979+0000","last_change":"2023-11-26T07:22:36.866255+0000","last_active":"2023-11-26T07:22:37.865979+0000","last_peered":"2023-11-26T07:22:37.865979+0000","last_clean":"2023-11-26T07:22:37.865979+0000","last_became_active":"2023-11-26T07:22:36.865530+0000","last_became_peered":"2023-11-26T07:22:36.865530+0000","last_unstale":"2023-11-26T07:22:37.865979+0000","last_undegraded":"2023-11-26T07:22:37.865979+0000","last_fullsized":"2023-11-26T07:22:37.865979+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:22:35.829702+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:22:35.829702+0000","last_clean_scrub_stamp":"2023-11-26T07:22:35.829702+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T19:16:57.380642+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.10","version":"0'0","reported_seq":11,"reported_epoch":71,"state":"active+clean","last_fresh":"2023-11-26T07:22:39.864074+0000","last_change":"2023-11-26T07:22:37.870669+0000","last_active":"2023-11-26T07:22:39.864074+0000","last_peered":"2023-11-26T07:22:39.864074+0000","last_clean":"2023-11-26T07:22:39.864074+0000","last_became_active":"2023-11-26T07:22:37.870306+0000","last_became_peered":"2023-11-26T07:22:37.870306+0000","last_unstale":"2023-11-26T07:22:39.864074+0000","last_undegraded":"2023-11-26T07:22:39.864074+0000","last_fullsized":"2023-11-26T07:22:39.864074+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:22:36.836338+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:22:36.836338+0000","last_clean_scrub_stamp":"2023-11-26T07:22:36.836338+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T17:59:50.448468+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"10.11","version":"0'0","reported_seq":12,"reported_epoch":70,"state":"active+clean","last_fresh":"2023-11-26T07:22:37.854375+0000","last_change":"2023-11-26T07:22:36.866279+0000","last_active":"2023-11-26T07:22:37.854375+0000","last_peered":"2023-11-26T07:22:37.854375+0000","last_clean":"2023-11-26T07:22:37.854375+0000","last_became_active":"2023-11-26T07:22:36.865955+0000","last_became_peered":"2023-11-26T07:22:36.865955+0000","last_unstale":"2023-11-26T07:22:37.854375+0000","last_undegraded":"2023-11-26T07:22:37.854375+0000","last_fullsized":"2023-11-26T07:22:37.854375+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:22:35.829702+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:22:35.829702+0000","last_clean_scrub_stamp":"2023-11-26T07:22:35.829702+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T08:17:37.779064+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"11.11","version":"0'0","reported_seq":11,"reported_epoch":71,"state":"active+clean","last_fresh":"2023-11-26T07:22:38.861373+0000","last_change":"2023-11-26T07:22:37.863518+0000","last_active":"2023-11-26T07:22:38.861373+0000","last_peered":"2023-11-26T07:22:38.861373+0000","last_clean":"2023-11-26T07:22:38.861373+0000","last_became_active":"2023-11-26T07:22:37.863275+0000","last_became_peered":"2023-11-26T07:22:37.863275+0000","last_unstale":"2023-11-26T07:22:38.861373+0000","last_undegraded":"2023-11-26T07:22:38.861373+0000","last_fullsized":"2023-11-26T07:22:38.861373+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:22:36.836338+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:22:36.836338+0000","last_clean_scrub_stamp":"2023-11-26T07:22:36.836338+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T09:19:07.788102+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"10.10","version":"70'8","reported_seq":20,"reported_epoch":70,"state":"active+clean","last_fresh":"2023-11-26T07:22:37.891639+0000","last_change":"2023-11-26T07:22:36.869929+0000","last_active":"2023-11-26T07:22:37.891639+0000","last_peered":"2023-11-26T07:22:37.891639+0000","last_clean":"2023-11-26T07:22:37.891639+0000","last_became_active":"2023-11-26T07:22:36.869372+0000","last_became_peered":"2023-11-26T07:22:36.869372+0000","last_unstale":"2023-11-26T07:22:37.891639+0000","last_undegraded":"2023-11-26T07:22:37.891639+0000","last_fullsized":"2023-11-26T07:22:37.891639+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:22:35.829702+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:22:35.829702+0000","last_clean_scrub_stamp":"2023-11-26T07:22:35.829702+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T16:40:16.851442+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.e","version":"0'0","reported_seq":11,"reported_epoch":71,"state":"active+clean","last_fresh":"2023-11-26T07:22:39.658192+0000","last_change":"2023-11-26T07:22:37.868826+0000","last_active":"2023-11-26T07:22:39.658192+0000","last_peered":"2023-11-26T07:22:39.658192+0000","last_clean":"2023-11-26T07:22:39.658192+0000","last_became_active":"2023-11-26T07:22:37.867718+0000","last_became_peered":"2023-11-26T07:22:37.867718+0000","last_unstale":"2023-11-26T07:22:39.658192+0000","last_undegraded":"2023-11-26T07:22:39.658192+0000","last_fullsized":"2023-11-26T07:22:39.658192+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:22:36.836338+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:22:36.836338+0000","last_clean_scrub_stamp":"2023-11-26T07:22:36.836338+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T07:55:11.708496+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"10.f","version":"70'1","reported_seq":13,"reported_epoch":70,"state":"active+clean","last_fresh":"2023-11-26T07:22:37.892228+0000","last_change":"2023-11-26T07:22:36.868080+0000","last_active":"2023-11-26T07:22:37.892228+0000","last_peered":"2023-11-26T07:22:37.892228+0000","last_clean":"2023-11-26T07:22:37.892228+0000","last_became_active":"2023-11-26T07:22:36.865257+0000","last_became_peered":"2023-11-26T07:22:36.865257+0000","last_unstale":"2023-11-26T07:22:37.892228+0000","last_undegraded":"2023-11-26T07:22:37.892228+0000","last_fullsized":"2023-11-26T07:22:37.892228+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:22:35.829702+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:22:35.829702+0000","last_clean_scrub_stamp":"2023-11-26T07:22:35.829702+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T18:04:03.792313+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.f","version":"0'0","reported_seq":11,"reported_epoch":71,"state":"active+clean","last_fresh":"2023-11-26T07:22:39.009243+0000","last_change":"2023-11-26T07:22:37.871386+0000","last_active":"2023-11-26T07:22:39.009243+0000","last_peered":"2023-11-26T07:22:39.009243+0000","last_clean":"2023-11-26T07:22:39.009243+0000","last_became_active":"2023-11-26T07:22:37.871041+0000","last_became_peered":"2023-11-26T07:22:37.871041+0000","last_unstale":"2023-11-26T07:22:39.009243+0000","last_undegraded":"2023-11-26T07:22:39.009243+0000","last_fullsized":"2023-11-26T07:22:39.009243+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:22:36.836338+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:22:36.836338+0000","last_clean_scrub_stamp":"2023-11-26T07:22:36.836338+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T09:11:43.017491+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.e","version":"70'2","reported_seq":14,"reported_epoch":70,"state":"active+clean","last_fresh":"2023-11-26T07:22:37.891182+0000","last_change":"2023-11-26T07:22:36.866137+0000","last_active":"2023-11-26T07:22:37.891182+0000","last_peered":"2023-11-26T07:22:37.891182+0000","last_clean":"2023-11-26T07:22:37.891182+0000","last_became_active":"2023-11-26T07:22:36.865873+0000","last_became_peered":"2023-11-26T07:22:36.865873+0000","last_unstale":"2023-11-26T07:22:37.891182+0000","last_undegraded":"2023-11-26T07:22:37.891182+0000","last_fullsized":"2023-11-26T07:22:37.891182+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:22:35.829702+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:22:35.829702+0000","last_clean_scrub_stamp":"2023-11-26T07:22:35.829702+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T19:14:58.011264+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.c","version":"0'0","reported_seq":11,"reported_epoch":71,"state":"active+clean","last_fresh":"2023-11-26T07:22:38.862889+0000","last_change":"2023-11-26T07:22:37.865642+0000","last_active":"2023-11-26T07:22:38.862889+0000","last_peered":"2023-11-26T07:22:38.862889+0000","last_clean":"2023-11-26T07:22:38.862889+0000","last_became_active":"2023-11-26T07:22:37.865314+0000","last_became_peered":"2023-11-26T07:22:37.865314+0000","last_unstale":"2023-11-26T07:22:38.862889+0000","last_undegraded":"2023-11-26T07:22:38.862889+0000","last_fullsized":"2023-11-26T07:22:38.862889+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:22:36.836338+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:22:36.836338+0000","last_clean_scrub_stamp":"2023-11-26T07:22:36.836338+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T18:01:35.745684+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"10.d","version":"70'1","reported_seq":13,"reported_epoch":70,"state":"active+clean","last_fresh":"2023-11-26T07:22:37.892203+0000","last_change":"2023-11-26T07:22:36.861086+0000","last_active":"2023-11-26T07:22:37.892203+0000","last_peered":"2023-11-26T07:22:37.892203+0000","last_clean":"2023-11-26T07:22:37.892203+0000","last_became_active":"2023-11-26T07:22:36.860779+0000","last_became_peered":"2023-11-26T07:22:36.860779+0000","last_unstale":"2023-11-26T07:22:37.892203+0000","last_undegraded":"2023-11-26T07:22:37.892203+0000","last_fullsized":"2023-11-26T07:22:37.892203+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:22:35.829702+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:22:35.829702+0000","last_clean_scrub_stamp":"2023-11-26T07:22:35.829702+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T19:14:58.011264+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.d","version":"0'0","reported_seq":11,"reported_epoch":71,"state":"active+clean","last_fresh":"2023-11-26T07:22:38.862029+0000","last_change":"2023-11-26T07:22:37.865825+0000","last_active":"2023-11-26T07:22:38.862029+0000","last_peered":"2023-11-26T07:22:38.862029+0000","last_clean":"2023-11-26T07:22:38.862029+0000","last_became_active":"2023-11-26T07:22:37.865388+0000","last_became_peered":"2023-11-26T07:22:37.865388+0000","last_unstale":"2023-11-26T07:22:38.862029+0000","last_undegraded":"2023-11-26T07:22:38.862029+0000","last_fullsized":"2023-11-26T07:22:38.862029+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:22:36.836338+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:22:36.836338+0000","last_clean_scrub_stamp":"2023-11-26T07:22:36.836338+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T10:33:19.999582+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"10.c","version":"0'0","reported_seq":12,"reported_epoch":70,"state":"active+clean","last_fresh":"2023-11-26T07:22:37.855671+0000","last_change":"2023-11-26T07:22:36.875047+0000","last_active":"2023-11-26T07:22:37.855671+0000","last_peered":"2023-11-26T07:22:37.855671+0000","last_clean":"2023-11-26T07:22:37.855671+0000","last_became_active":"2023-11-26T07:22:36.874745+0000","last_became_peered":"2023-11-26T07:22:36.874745+0000","last_unstale":"2023-11-26T07:22:37.855671+0000","last_undegraded":"2023-11-26T07:22:37.855671+0000","last_fullsized":"2023-11-26T07:22:37.855671+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:22:35.829702+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:22:35.829702+0000","last_clean_scrub_stamp":"2023-11-26T07:22:35.829702+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T13:34:40.612725+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"11.b","version":"0'0","reported_seq":11,"reported_epoch":71,"state":"active+clean","last_fresh":"2023-11-26T07:22:40.017951+0000","last_change":"2023-11-26T07:22:37.871637+0000","last_active":"2023-11-26T07:22:40.017951+0000","last_peered":"2023-11-26T07:22:40.017951+0000","last_clean":"2023-11-26T07:22:40.017951+0000","last_became_active":"2023-11-26T07:22:37.871390+0000","last_became_peered":"2023-11-26T07:22:37.871390+0000","last_unstale":"2023-11-26T07:22:40.017951+0000","last_undegraded":"2023-11-26T07:22:40.017951+0000","last_fullsized":"2023-11-26T07:22:40.017951+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:22:36.836338+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:22:36.836338+0000","last_clean_scrub_stamp":"2023-11-26T07:22:36.836338+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T14:36:50.851660+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.a","version":"0'0","reported_seq":12,"reported_epoch":70,"state":"active+clean","last_fresh":"2023-11-26T07:22:37.866111+0000","last_change":"2023-11-26T07:22:36.862939+0000","last_active":"2023-11-26T07:22:37.866111+0000","last_peered":"2023-11-26T07:22:37.866111+0000","last_clean":"2023-11-26T07:22:37.866111+0000","last_became_active":"2023-11-26T07:22:36.862625+0000","last_became_peered":"2023-11-26T07:22:36.862625+0000","last_unstale":"2023-11-26T07:22:37.866111+0000","last_undegraded":"2023-11-26T07:22:37.866111+0000","last_fullsized":"2023-11-26T07:22:37.866111+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:22:35.829702+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:22:35.829702+0000","last_clean_scrub_stamp":"2023-11-26T07:22:35.829702+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T07:41:05.117550+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.8","version":"0'0","reported_seq":11,"reported_epoch":71,"state":"active+clean","last_fresh":"2023-11-26T07:22:40.017765+0000","last_change":"2023-11-26T07:22:37.872438+0000","last_active":"2023-11-26T07:22:40.017765+0000","last_peered":"2023-11-26T07:22:40.017765+0000","last_clean":"2023-11-26T07:22:40.017765+0000","last_became_active":"2023-11-26T07:22:37.872114+0000","last_became_peered":"2023-11-26T07:22:37.872114+0000","last_unstale":"2023-11-26T07:22:40.017765+0000","last_undegraded":"2023-11-26T07:22:40.017765+0000","last_fullsized":"2023-11-26T07:22:40.017765+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:22:36.836338+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:22:36.836338+0000","last_clean_scrub_stamp":"2023-11-26T07:22:36.836338+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T17:12:49.941456+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.9","version":"0'0","reported_seq":12,"reported_epoch":70,"state":"active+clean","last_fresh":"2023-11-26T07:22:37.855701+0000","last_change":"2023-11-26T07:22:36.865706+0000","last_active":"2023-11-26T07:22:37.855701+0000","last_peered":"2023-11-26T07:22:37.855701+0000","last_clean":"2023-11-26T07:22:37.855701+0000","last_became_active":"2023-11-26T07:22:36.865421+0000","last_became_peered":"2023-11-26T07:22:36.865421+0000","last_unstale":"2023-11-26T07:22:37.855701+0000","last_undegraded":"2023-11-26T07:22:37.855701+0000","last_fullsized":"2023-11-26T07:22:37.855701+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:22:35.829702+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:22:35.829702+0000","last_clean_scrub_stamp":"2023-11-26T07:22:35.829702+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T09:24:30.767740+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"11.9","version":"0'0","reported_seq":11,"reported_epoch":71,"state":"active+clean","last_fresh":"2023-11-26T07:22:40.017950+0000","last_change":"2023-11-26T07:22:37.879871+0000","last_active":"2023-11-26T07:22:40.017950+0000","last_peered":"2023-11-26T07:22:40.017950+0000","last_clean":"2023-11-26T07:22:40.017950+0000","last_became_active":"2023-11-26T07:22:37.879560+0000","last_became_peered":"2023-11-26T07:22:37.879560+0000","last_unstale":"2023-11-26T07:22:40.017950+0000","last_undegraded":"2023-11-26T07:22:40.017950+0000","last_fullsized":"2023-11-26T07:22:40.017950+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:22:36.836338+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:22:36.836338+0000","last_clean_scrub_stamp":"2023-11-26T07:22:36.836338+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T09:52:57.186530+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.8","version":"0'0","reported_seq":12,"reported_epoch":70,"state":"active+clean","last_fresh":"2023-11-26T07:22:37.848351+0000","last_change":"2023-11-26T07:22:36.863944+0000","last_active":"2023-11-26T07:22:37.848351+0000","last_peered":"2023-11-26T07:22:37.848351+0000","last_clean":"2023-11-26T07:22:37.848351+0000","last_became_active":"2023-11-26T07:22:36.863619+0000","last_became_peered":"2023-11-26T07:22:36.863619+0000","last_unstale":"2023-11-26T07:22:37.848351+0000","last_undegraded":"2023-11-26T07:22:37.848351+0000","last_fullsized":"2023-11-26T07:22:37.848351+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:22:35.829702+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:22:35.829702+0000","last_clean_scrub_stamp":"2023-11-26T07:22:35.829702+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T18:23:53.779522+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.6","version":"0'0","reported_seq":11,"reported_epoch":71,"state":"active+clean","last_fresh":"2023-11-26T07:22:39.864365+0000","last_change":"2023-11-26T07:22:37.858075+0000","last_active":"2023-11-26T07:22:39.864365+0000","last_peered":"2023-11-26T07:22:39.864365+0000","last_clean":"2023-11-26T07:22:39.864365+0000","last_became_active":"2023-11-26T07:22:37.857651+0000","last_became_peered":"2023-11-26T07:22:37.857651+0000","last_unstale":"2023-11-26T07:22:39.864365+0000","last_undegraded":"2023-11-26T07:22:39.864365+0000","last_fullsized":"2023-11-26T07:22:39.864365+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:22:36.836338+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:22:36.836338+0000","last_clean_scrub_stamp":"2023-11-26T07:22:36.836338+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T15:34:25.091425+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"10.7","version":"70'2","reported_seq":14,"reported_epoch":70,"state":"active+clean","last_fresh":"2023-11-26T07:22:37.892279+0000","last_change":"2023-11-26T07:22:36.865572+0000","last_active":"2023-11-26T07:22:37.892279+0000","last_peered":"2023-11-26T07:22:37.892279+0000","last_clean":"2023-11-26T07:22:37.892279+0000","last_became_active":"2023-11-26T07:22:36.864928+0000","last_became_peered":"2023-11-26T07:22:36.864928+0000","last_unstale":"2023-11-26T07:22:37.892279+0000","last_undegraded":"2023-11-26T07:22:37.892279+0000","last_fullsized":"2023-11-26T07:22:37.892279+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:22:35.829702+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:22:35.829702+0000","last_clean_scrub_stamp":"2023-11-26T07:22:35.829702+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T13:57:19.037504+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.7","version":"0'0","reported_seq":11,"reported_epoch":71,"state":"active+clean","last_fresh":"2023-11-26T07:22:39.480277+0000","last_change":"2023-11-26T07:22:37.871391+0000","last_active":"2023-11-26T07:22:39.480277+0000","last_peered":"2023-11-26T07:22:39.480277+0000","last_clean":"2023-11-26T07:22:39.480277+0000","last_became_active":"2023-11-26T07:22:37.869546+0000","last_became_peered":"2023-11-26T07:22:37.869546+0000","last_unstale":"2023-11-26T07:22:39.480277+0000","last_undegraded":"2023-11-26T07:22:39.480277+0000","last_fullsized":"2023-11-26T07:22:39.480277+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:22:36.836338+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:22:36.836338+0000","last_clean_scrub_stamp":"2023-11-26T07:22:36.836338+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T18:39:51.590102+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.6","version":"0'0","reported_seq":12,"reported_epoch":70,"state":"active+clean","last_fresh":"2023-11-26T07:22:37.860114+0000","last_change":"2023-11-26T07:22:36.866276+0000","last_active":"2023-11-26T07:22:37.860114+0000","last_peered":"2023-11-26T07:22:37.860114+0000","last_clean":"2023-11-26T07:22:37.860114+0000","last_became_active":"2023-11-26T07:22:36.865972+0000","last_became_peered":"2023-11-26T07:22:36.865972+0000","last_unstale":"2023-11-26T07:22:37.860114+0000","last_undegraded":"2023-11-26T07:22:37.860114+0000","last_fullsized":"2023-11-26T07:22:37.860114+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:22:35.829702+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:22:35.829702+0000","last_clean_scrub_stamp":"2023-11-26T07:22:35.829702+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T07:55:10.701860+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"11.4","version":"0'0","reported_seq":11,"reported_epoch":71,"state":"active+clean","last_fresh":"2023-11-26T07:22:39.009019+0000","last_change":"2023-11-26T07:22:37.881934+0000","last_active":"2023-11-26T07:22:39.009019+0000","last_peered":"2023-11-26T07:22:39.009019+0000","last_clean":"2023-11-26T07:22:39.009019+0000","last_became_active":"2023-11-26T07:22:37.879130+0000","last_became_peered":"2023-11-26T07:22:37.879130+0000","last_unstale":"2023-11-26T07:22:39.009019+0000","last_undegraded":"2023-11-26T07:22:39.009019+0000","last_fullsized":"2023-11-26T07:22:39.009019+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:22:36.836338+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:22:36.836338+0000","last_clean_scrub_stamp":"2023-11-26T07:22:36.836338+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T17:24:41.141823+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.5","version":"0'0","reported_seq":12,"reported_epoch":70,"state":"active+clean","last_fresh":"2023-11-26T07:22:37.855189+0000","last_change":"2023-11-26T07:22:36.862659+0000","last_active":"2023-11-26T07:22:37.855189+0000","last_peered":"2023-11-26T07:22:37.855189+0000","last_clean":"2023-11-26T07:22:37.855189+0000","last_became_active":"2023-11-26T07:22:36.862342+0000","last_became_peered":"2023-11-26T07:22:36.862342+0000","last_unstale":"2023-11-26T07:22:37.855189+0000","last_undegraded":"2023-11-26T07:22:37.855189+0000","last_fullsized":"2023-11-26T07:22:37.855189+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:22:35.829702+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:22:35.829702+0000","last_clean_scrub_stamp":"2023-11-26T07:22:35.829702+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T15:11:09.629347+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"11.5","version":"0'0","reported_seq":11,"reported_epoch":71,"state":"active+clean","last_fresh":"2023-11-26T07:22:39.009179+0000","last_change":"2023-11-26T07:22:37.877378+0000","last_active":"2023-11-26T07:22:39.009179+0000","last_peered":"2023-11-26T07:22:39.009179+0000","last_clean":"2023-11-26T07:22:39.009179+0000","last_became_active":"2023-11-26T07:22:37.877099+0000","last_became_peered":"2023-11-26T07:22:37.877099+0000","last_unstale":"2023-11-26T07:22:39.009179+0000","last_undegraded":"2023-11-26T07:22:39.009179+0000","last_fullsized":"2023-11-26T07:22:39.009179+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:22:36.836338+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:22:36.836338+0000","last_clean_scrub_stamp":"2023-11-26T07:22:36.836338+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T15:10:55.003398+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.4","version":"0'0","reported_seq":12,"reported_epoch":70,"state":"active+clean","last_fresh":"2023-11-26T07:22:37.848299+0000","last_change":"2023-11-26T07:22:36.861249+0000","last_active":"2023-11-26T07:22:37.848299+0000","last_peered":"2023-11-26T07:22:37.848299+0000","last_clean":"2023-11-26T07:22:37.848299+0000","last_became_active":"2023-11-26T07:22:36.860803+0000","last_became_peered":"2023-11-26T07:22:36.860803+0000","last_unstale":"2023-11-26T07:22:37.848299+0000","last_undegraded":"2023-11-26T07:22:37.848299+0000","last_fullsized":"2023-11-26T07:22:37.848299+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:22:35.829702+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:22:35.829702+0000","last_clean_scrub_stamp":"2023-11-26T07:22:35.829702+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T14:27:32.693731+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.2","version":"0'0","reported_seq":11,"reported_epoch":71,"state":"active+clean","last_fresh":"2023-11-26T07:22:38.861539+0000","last_change":"2023-11-26T07:22:37.862679+0000","last_active":"2023-11-26T07:22:38.861539+0000","last_peered":"2023-11-26T07:22:38.861539+0000","last_clean":"2023-11-26T07:22:38.861539+0000","last_became_active":"2023-11-26T07:22:37.862423+0000","last_became_peered":"2023-11-26T07:22:37.862423+0000","last_unstale":"2023-11-26T07:22:38.861539+0000","last_undegraded":"2023-11-26T07:22:38.861539+0000","last_fullsized":"2023-11-26T07:22:38.861539+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:22:36.836338+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:22:36.836338+0000","last_clean_scrub_stamp":"2023-11-26T07:22:36.836338+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T19:20:37.466959+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"10.3","version":"0'0","reported_seq":12,"reported_epoch":70,"state":"active+clean","last_fresh":"2023-11-26T07:22:37.854850+0000","last_change":"2023-11-26T07:22:36.870180+0000","last_active":"2023-11-26T07:22:37.854850+0000","last_peered":"2023-11-26T07:22:37.854850+0000","last_clean":"2023-11-26T07:22:37.854850+0000","last_became_active":"2023-11-26T07:22:36.869919+0000","last_became_peered":"2023-11-26T07:22:36.869919+0000","last_unstale":"2023-11-26T07:22:37.854850+0000","last_undegraded":"2023-11-26T07:22:37.854850+0000","last_fullsized":"2023-11-26T07:22:37.854850+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:22:35.829702+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:22:35.829702+0000","last_clean_scrub_stamp":"2023-11-26T07:22:35.829702+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T10:28:25.331030+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"11.3","version":"0'0","reported_seq":11,"reported_epoch":71,"state":"active+clean","last_fresh":"2023-11-26T07:22:40.016709+0000","last_change":"2023-11-26T07:22:37.878023+0000","last_active":"2023-11-26T07:22:40.016709+0000","last_peered":"2023-11-26T07:22:40.016709+0000","last_clean":"2023-11-26T07:22:40.016709+0000","last_became_active":"2023-11-26T07:22:37.877444+0000","last_became_peered":"2023-11-26T07:22:37.877444+0000","last_unstale":"2023-11-26T07:22:40.016709+0000","last_undegraded":"2023-11-26T07:22:40.016709+0000","last_fullsized":"2023-11-26T07:22:40.016709+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:22:36.836338+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:22:36.836338+0000","last_clean_scrub_stamp":"2023-11-26T07:22:36.836338+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T11:09:49.677547+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"10.2","version":"0'0","reported_seq":12,"reported_epoch":70,"state":"active+clean","last_fresh":"2023-11-26T07:22:37.860639+0000","last_change":"2023-11-26T07:22:36.865453+0000","last_active":"2023-11-26T07:22:37.860639+0000","last_peered":"2023-11-26T07:22:37.860639+0000","last_clean":"2023-11-26T07:22:37.860639+0000","last_became_active":"2023-11-26T07:22:36.864886+0000","last_became_peered":"2023-11-26T07:22:36.864886+0000","last_unstale":"2023-11-26T07:22:37.860639+0000","last_undegraded":"2023-11-26T07:22:37.860639+0000","last_fullsized":"2023-11-26T07:22:37.860639+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:22:35.829702+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:22:35.829702+0000","last_clean_scrub_stamp":"2023-11-26T07:22:35.829702+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T10:04:12.676909+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.0","version":"0'0","reported_seq":11,"reported_epoch":71,"state":"active+clean","last_fresh":"2023-11-26T07:22:38.862787+0000","last_change":"2023-11-26T07:22:37.874876+0000","last_active":"2023-11-26T07:22:38.862787+0000","last_peered":"2023-11-26T07:22:38.862787+0000","last_clean":"2023-11-26T07:22:38.862787+0000","last_became_active":"2023-11-26T07:22:37.874581+0000","last_became_peered":"2023-11-26T07:22:37.874581+0000","last_unstale":"2023-11-26T07:22:38.862787+0000","last_undegraded":"2023-11-26T07:22:38.862787+0000","last_fullsized":"2023-11-26T07:22:38.862787+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:22:36.836338+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:22:36.836338+0000","last_clean_scrub_stamp":"2023-11-26T07:22:36.836338+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T09:31:53.635850+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"10.1","version":"0'0","reported_seq":12,"reported_epoch":70,"state":"active+clean","last_fresh":"2023-11-26T07:22:37.849474+0000","last_change":"2023-11-26T07:22:36.869677+0000","last_active":"2023-11-26T07:22:37.849474+0000","last_peered":"2023-11-26T07:22:37.849474+0000","last_clean":"2023-11-26T07:22:37.849474+0000","last_became_active":"2023-11-26T07:22:36.869395+0000","last_became_peered":"2023-11-26T07:22:36.869395+0000","last_unstale":"2023-11-26T07:22:37.849474+0000","last_undegraded":"2023-11-26T07:22:37.849474+0000","last_fullsized":"2023-11-26T07:22:37.849474+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:22:35.829702+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:22:35.829702+0000","last_clean_scrub_stamp":"2023-11-26T07:22:35.829702+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T14:00:20.899765+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.1","version":"0'0","reported_seq":11,"reported_epoch":71,"state":"active+clean","last_fresh":"2023-11-26T07:22:40.016659+0000","last_change":"2023-11-26T07:22:37.877486+0000","last_active":"2023-11-26T07:22:40.016659+0000","last_peered":"2023-11-26T07:22:40.016659+0000","last_clean":"2023-11-26T07:22:40.016659+0000","last_became_active":"2023-11-26T07:22:37.876471+0000","last_became_peered":"2023-11-26T07:22:37.876471+0000","last_unstale":"2023-11-26T07:22:40.016659+0000","last_undegraded":"2023-11-26T07:22:40.016659+0000","last_fullsized":"2023-11-26T07:22:40.016659+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:22:36.836338+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:22:36.836338+0000","last_clean_scrub_stamp":"2023-11-26T07:22:36.836338+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T08:17:38.785700+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"10.0","version":"0'0","reported_seq":12,"reported_epoch":70,"state":"active+clean","last_fresh":"2023-11-26T07:22:37.865899+0000","last_change":"2023-11-26T07:22:36.862071+0000","last_active":"2023-11-26T07:22:37.865899+0000","last_peered":"2023-11-26T07:22:37.865899+0000","last_clean":"2023-11-26T07:22:37.865899+0000","last_became_active":"2023-11-26T07:22:36.861791+0000","last_became_peered":"2023-11-26T07:22:36.861791+0000","last_unstale":"2023-11-26T07:22:37.865899+0000","last_undegraded":"2023-11-26T07:22:37.865899+0000","last_fullsized":"2023-11-26T07:22:37.865899+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:22:35.829702+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:22:35.829702+0000","last_clean_scrub_stamp":"2023-11-26T07:22:35.829702+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T14:00:20.899765+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.a","version":"0'0","reported_seq":11,"reported_epoch":71,"state":"active+clean","last_fresh":"2023-11-26T07:22:40.017829+0000","last_change":"2023-11-26T07:22:37.879289+0000","last_active":"2023-11-26T07:22:40.017829+0000","last_peered":"2023-11-26T07:22:40.017829+0000","last_clean":"2023-11-26T07:22:40.017829+0000","last_became_active":"2023-11-26T07:22:37.879002+0000","last_became_peered":"2023-11-26T07:22:37.879002+0000","last_unstale":"2023-11-26T07:22:40.017829+0000","last_undegraded":"2023-11-26T07:22:40.017829+0000","last_fullsized":"2023-11-26T07:22:40.017829+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:22:36.836338+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:22:36.836338+0000","last_clean_scrub_stamp":"2023-11-26T07:22:36.836338+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T17:54:06.225209+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.b","version":"70'2","reported_seq":14,"reported_epoch":70,"state":"active+clean","last_fresh":"2023-11-26T07:22:37.892777+0000","last_change":"2023-11-26T07:22:36.868634+0000","last_active":"2023-11-26T07:22:37.892777+0000","last_peered":"2023-11-26T07:22:37.892777+0000","last_clean":"2023-11-26T07:22:37.892777+0000","last_became_active":"2023-11-26T07:22:36.867494+0000","last_became_peered":"2023-11-26T07:22:36.867494+0000","last_unstale":"2023-11-26T07:22:37.892777+0000","last_undegraded":"2023-11-26T07:22:37.892777+0000","last_fullsized":"2023-11-26T07:22:37.892777+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:22:35.829702+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:22:35.829702+0000","last_clean_scrub_stamp":"2023-11-26T07:22:35.829702+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T16:40:16.851442+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"1.0","version":"15'117","reported_seq":205,"reported_epoch":70,"state":"active+clean","last_fresh":"2023-11-26T07:22:37.855521+0000","last_change":"2023-11-26T07:17:43.827802+0000","last_active":"2023-11-26T07:22:37.855521+0000","last_peered":"2023-11-26T07:22:37.855521+0000","last_clean":"2023-11-26T07:22:37.855521+0000","last_became_active":"2023-11-26T07:17:43.827308+0000","last_became_peered":"2023-11-26T07:17:43.827308+0000","last_unstale":"2023-11-26T07:22:37.855521+0000","last_undegraded":"2023-11-26T07:22:37.855521+0000","last_fullsized":"2023-11-26T07:22:37.855521+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:17:42.160945+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:17:42.160945+0000","last_clean_scrub_stamp":"2023-11-26T07:17:42.160945+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T08:17:54.167366+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":11,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":10,"num_pg":32,"stat_sum":{"num_bytes":2410,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":38,"ondisk_log_size":38,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574910,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32620,"kb_used_data":1432,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339220,"statfs":{"total":96636764160,"available":96603361280,"internally_reserved":0,"allocated":1466368,"data_stored":955263,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":31914442},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Nov 26 07:21:51 2023","interfaces":[{"interface":"back","average":{"1min":0.76600000000000001,"5min":0.71899999999999997,"15min":0.71599999999999997},"min":{"1min":0.42999999999999999,"5min":0.42999999999999999,"15min":0.42999999999999999},"max":{"1min":0.95999999999999996,"5min":0.98199999999999998,"15min":0.98199999999999998},"last":0.72599999999999998},{"interface":"front","average":{"1min":0.76400000000000001,"5min":0.72799999999999998,"15min":0.71499999999999997},"min":{"1min":0.53300000000000003,"5min":0.52900000000000003,"15min":0.52900000000000003},"max":{"1min":0.98599999999999999,"5min":1.0980000000000001,"15min":1.0980000000000001},"last":0.75900000000000001}]},{"osd":1,"last update":"Sun Nov 26 07:21:51 2023","interfaces":[{"interface":"back","average":{"1min":0.70599999999999996,"5min":0.68700000000000006,"15min":0.68300000000000005},"min":{"1min":0.441,"5min":0.441,"15min":0.441},"max":{"1min":1.012,"5min":1.0669999999999999,"15min":1.0669999999999999},"last":0.61299999999999999},{"interface":"front","average":{"1min":0.73599999999999999,"5min":0.74399999999999999,"15min":0.80700000000000005},"min":{"1min":0.58099999999999996,"5min":0.47599999999999998,"15min":0.47599999999999998},"max":{"1min":0.94199999999999995,"5min":4.5309999999999997,"15min":4.5309999999999997},"last":0.80600000000000005}]},{"osd":2,"last update":"Sun Nov 26 07:21:51 2023","interfaces":[{"interface":"back","average":{"1min":0.73399999999999999,"5min":0.69299999999999995,"15min":0.68100000000000005},"min":{"1min":0.39700000000000002,"5min":0.39700000000000002,"15min":0.39700000000000002},"max":{"1min":1.0800000000000001,"5min":1.0800000000000001,"15min":1.0800000000000001},"last":0.67600000000000005},{"interface":"front","average":{"1min":0.76800000000000002,"5min":0.71299999999999997,"15min":0.71299999999999997},"min":{"1min":0.497,"5min":0.497,"15min":0.497},"max":{"1min":0.96299999999999997,"5min":1.1339999999999999,"15min":1.1339999999999999},"last":0.83099999999999996}]},{"osd":3,"last update":"Sun Nov 26 07:21:51 2023","interfaces":[{"interface":"back","average":{"1min":0.73799999999999999,"5min":0.71899999999999997,"15min":0.70399999999999996},"min":{"1min":0.53000000000000003,"5min":0.47799999999999998,"15min":0.47799999999999998},"max":{"1min":1.1240000000000001,"5min":1.1240000000000001,"15min":1.1240000000000001},"last":0.71299999999999997},{"interface":"front","average":{"1min":0.81599999999999995,"5min":0.748,"15min":0.73799999999999999},"min":{"1min":0.54600000000000004,"5min":0.433,"15min":0.433},"max":{"1min":1.036,"5min":1.4490000000000001,"15min":1.4490000000000001},"last":0.55500000000000005}]},{"osd":4,"last update":"Sun Nov 26 07:21:51 2023","interfaces":[{"interface":"back","average":{"1min":0.59999999999999998,"5min":0.61199999999999999,"15min":0.60399999999999998},"min":{"1min":0.47099999999999997,"5min":0.39400000000000002,"15min":0.39400000000000002},"max":{"1min":0.96199999999999997,"5min":0.96299999999999997,"15min":0.96299999999999997},"last":0.63400000000000001},{"interface":"front","average":{"1min":0.66000000000000003,"5min":0.66100000000000003,"15min":0.66900000000000004},"min":{"1min":0.44600000000000001,"5min":0.436,"15min":0.436},"max":{"1min":1.004,"5min":1.5509999999999999,"15min":1.5509999999999999},"last":0.58699999999999997}]},{"osd":5,"last update":"Sun Nov 26 07:21:51 2023","interfaces":[{"interface":"back","average":{"1min":0.69299999999999995,"5min":0.68200000000000005,"15min":0.67100000000000004},"min":{"1min":0.46300000000000002,"5min":0.443,"15min":0.443},"max":{"1min":0.93300000000000005,"5min":1.415,"15min":1.415},"last":0.66000000000000003},{"interface":"front","average":{"1min":0.69299999999999995,"5min":0.69399999999999995,"15min":0.70699999999999996},"min":{"1min":0.48699999999999999,"5min":0.48599999999999999,"15min":0.48599999999999999},"max":{"1min":1.0840000000000001,"5min":1.3859999999999999,"15min":1.3859999999999999},"last":0.69199999999999995}]},{"osd":6,"last update":"Sun Nov 26 07:21:51 2023","interfaces":[{"interface":"back","average":{"1min":0.79200000000000004,"5min":0.77800000000000002,"15min":0.77300000000000002},"min":{"1min":0.56399999999999995,"5min":0.51700000000000002,"15min":0.51700000000000002},"max":{"1min":1.123,"5min":1.3069999999999999,"15min":1.3069999999999999},"last":0.88400000000000001},{"interface":"front","average":{"1min":0.75600000000000001,"5min":0.76200000000000001,"15min":0.76800000000000002},"min":{"1min":0.58399999999999996,"5min":0.46899999999999997,"15min":0.46899999999999997},"max":{"1min":0.90800000000000003,"5min":1.6759999999999999,"15min":1.6759999999999999},"last":0.74099999999999999}]}]},{"osd":6,"up_from":13,"seq":55834574912,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32032,"kb_used_data":852,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339808,"statfs":{"total":96636764160,"available":96603963392,"internally_reserved":0,"allocated":872448,"data_stored":364371,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Nov 26 07:21:53 2023","interfaces":[{"interface":"back","average":{"1min":0.69799999999999995,"5min":0.69799999999999995,"15min":0.68799999999999994},"min":{"1min":0.48999999999999999,"5min":0.36699999999999999,"15min":0.36699999999999999},"max":{"1min":0.91800000000000004,"5min":1.7549999999999999,"15min":1.7549999999999999},"last":0.77600000000000002},{"interface":"front","average":{"1min":0.72899999999999998,"5min":0.81799999999999995,"15min":0.91100000000000003},"min":{"1min":0.46600000000000003,"5min":0.38300000000000001,"15min":0.38300000000000001},"max":{"1min":0.874,"5min":5.7489999999999997,"15min":5.7489999999999997},"last":0.58299999999999996}]},{"osd":1,"last update":"Sun Nov 26 07:21:53 2023","interfaces":[{"interface":"back","average":{"1min":0.74199999999999999,"5min":0.80300000000000005,"15min":0.877},"min":{"1min":0.48699999999999999,"5min":0.41199999999999998,"15min":0.41199999999999998},"max":{"1min":0.97799999999999998,"5min":5.4240000000000004,"15min":5.4240000000000004},"last":0.67800000000000005},{"interface":"front","average":{"1min":0.72999999999999998,"5min":0.79500000000000004,"15min":0.86299999999999999},"min":{"1min":0.52200000000000002,"5min":0.499,"15min":0.499},"max":{"1min":0.96499999999999997,"5min":5.6020000000000003,"15min":5.6020000000000003},"last":0.70999999999999996}]},{"osd":2,"last update":"Sun Nov 26 07:21:53 2023","interfaces":[{"interface":"back","average":{"1min":0.70199999999999996,"5min":0.68999999999999995,"15min":0.69399999999999995},"min":{"1min":0.499,"5min":0.45100000000000001,"15min":0.45100000000000001},"max":{"1min":0.93600000000000005,"5min":1.8520000000000001,"15min":1.8520000000000001},"last":0.74399999999999999},{"interface":"front","average":{"1min":0.752,"5min":0.73399999999999999,"15min":0.73399999999999999},"min":{"1min":0.54300000000000004,"5min":0.437,"15min":0.437},"max":{"1min":0.95199999999999996,"5min":1.9159999999999999,"15min":1.9159999999999999},"last":0.60899999999999999}]},{"osd":3,"last update":"Sun Nov 26 07:21:53 2023","interfaces":[{"interface":"back","average":{"1min":0.72699999999999998,"5min":0.80000000000000004,"15min":0.86799999999999999},"min":{"1min":0.52400000000000002,"5min":0.46899999999999997,"15min":0.46899999999999997},"max":{"1min":0.91500000000000004,"5min":5.4390000000000001,"15min":5.4390000000000001},"last":0.64200000000000002},{"interface":"front","average":{"1min":0.74299999999999999,"5min":0.81499999999999995,"15min":0.90100000000000002},"min":{"1min":0.52300000000000002,"5min":0.48399999999999999,"15min":0.48399999999999999},"max":{"1min":0.89800000000000002,"5min":5.5490000000000004,"15min":5.5490000000000004},"last":0.79900000000000004}]},{"osd":4,"last update":"Sun Nov 26 07:21:53 2023","interfaces":[{"interface":"back","average":{"1min":0.56399999999999995,"5min":0.626,"15min":0.63600000000000001},"min":{"1min":0.40600000000000003,"5min":0.371,"15min":0.371},"max":{"1min":0.82999999999999996,"5min":1.0129999999999999,"15min":1.0129999999999999},"last":0.83399999999999996},{"interface":"front","average":{"1min":0.58899999999999997,"5min":0.65300000000000002,"15min":0.66700000000000004},"min":{"1min":0.38600000000000001,"5min":0.38600000000000001,"15min":0.38600000000000001},"max":{"1min":0.86399999999999999,"5min":1.0349999999999999,"15min":1.0349999999999999},"last":0.81999999999999995}]},{"osd":5,"last update":"Sun Nov 26 07:21:53 2023","interfaces":[{"interface":"back","average":{"1min":0.62,"5min":0.63800000000000001,"15min":0.65200000000000002},"min":{"1min":0.40699999999999997,"5min":0.38200000000000001,"15min":0.38200000000000001},"max":{"1min":0.84499999999999997,"5min":0.873,"15min":0.873},"last":0.85499999999999998},{"interface":"front","average":{"1min":0.65700000000000003,"5min":0.69899999999999995,"15min":0.70799999999999996},"min":{"1min":0.44,"5min":0.436,"15min":0.436},"max":{"1min":0.92900000000000005,"5min":1.0960000000000001,"15min":1.0960000000000001},"last":0.86699999999999999}]},{"osd":7,"last update":"Sun Nov 26 07:21:53 2023","interfaces":[{"interface":"back","average":{"1min":0.67900000000000005,"5min":0.71999999999999997,"15min":0.73399999999999999},"min":{"1min":0.42599999999999999,"5min":0.42599999999999999,"15min":0.42599999999999999},"max":{"1min":0.93999999999999995,"5min":0.998,"15min":0.998},"last":0.76400000000000001},{"interface":"front","average":{"1min":0.64800000000000002,"5min":0.69799999999999995,"15min":0.70999999999999996},"min":{"1min":0.46400000000000002,"5min":0.46400000000000002,"15min":0.46400000000000002},"max":{"1min":0.85299999999999998,"5min":0.98799999999999999,"15min":0.98799999999999999},"last":0.88900000000000001}]}]},{"osd":5,"up_from":13,"seq":55834574910,"num_pgs":19,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27876,"kb_used_data":848,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343964,"statfs":{"total":96636764160,"available":96608219136,"internally_reserved":0,"allocated":868352,"data_stored":364369,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Nov 26 07:21:48 2023","interfaces":[{"interface":"back","average":{"1min":0.72199999999999998,"5min":0.68799999999999994,"15min":0.67300000000000004},"min":{"1min":0.56200000000000006,"5min":0.38100000000000001,"15min":0.38100000000000001},"max":{"1min":0.872,"5min":1.0289999999999999,"15min":1.0289999999999999},"last":0.82899999999999996},{"interface":"front","average":{"1min":0.71399999999999997,"5min":0.69599999999999995,"15min":0.67200000000000004},"min":{"1min":0.50600000000000001,"5min":0.438,"15min":0.438},"max":{"1min":1.028,"5min":1.05,"15min":1.05},"last":0.95199999999999996}]},{"osd":1,"last update":"Sun Nov 26 07:21:48 2023","interfaces":[{"interface":"back","average":{"1min":0.71199999999999997,"5min":0.69499999999999995,"15min":0.67700000000000005},"min":{"1min":0.45700000000000002,"5min":0.41599999999999998,"15min":0.41599999999999998},"max":{"1min":0.95399999999999996,"5min":1.0389999999999999,"15min":1.0389999999999999},"last":0.93300000000000005},{"interface":"front","average":{"1min":0.72199999999999998,"5min":0.70899999999999996,"15min":0.69299999999999995},"min":{"1min":0.53500000000000003,"5min":0.443,"15min":0.443},"max":{"1min":0.99299999999999999,"5min":1.006,"15min":1.006},"last":0.76800000000000002}]},{"osd":2,"last update":"Sun Nov 26 07:21:48 2023","interfaces":[{"interface":"back","average":{"1min":0.72199999999999998,"5min":0.69799999999999995,"15min":0.68200000000000005},"min":{"1min":0.51300000000000001,"5min":0.45900000000000002,"15min":0.45900000000000002},"max":{"1min":0.92900000000000005,"5min":0.95699999999999996,"15min":0.95699999999999996},"last":0.84599999999999997},{"interface":"front","average":{"1min":0.78700000000000003,"5min":0.72999999999999998,"15min":0.68999999999999995},"min":{"1min":0.55100000000000005,"5min":0.51200000000000001,"15min":0.51200000000000001},"max":{"1min":1.1379999999999999,"5min":1.1379999999999999,"15min":1.1379999999999999},"last":0.80400000000000005}]},{"osd":3,"last update":"Sun Nov 26 07:21:48 2023","interfaces":[{"interface":"back","average":{"1min":0.75700000000000001,"5min":0.73299999999999998,"15min":0.72399999999999998},"min":{"1min":0.44700000000000001,"5min":0.43099999999999999,"15min":0.43099999999999999},"max":{"1min":1.1739999999999999,"5min":1.1739999999999999,"15min":1.1739999999999999},"last":0.73599999999999999},{"interface":"front","average":{"1min":0.77800000000000002,"5min":0.72299999999999998,"15min":0.69099999999999995},"min":{"1min":0.55600000000000005,"5min":0.44600000000000001,"15min":0.44600000000000001},"max":{"1min":1.0029999999999999,"5min":1.0209999999999999,"15min":1.0209999999999999},"last":0.69799999999999995}]},{"osd":4,"last update":"Sun Nov 26 07:21:48 2023","interfaces":[{"interface":"back","average":{"1min":0.56799999999999995,"5min":0.59999999999999998,"15min":0.623},"min":{"1min":0.41199999999999998,"5min":0.35299999999999998,"15min":0.35299999999999998},"max":{"1min":0.92700000000000005,"5min":0.95099999999999996,"15min":0.95099999999999996},"last":0.504},{"interface":"front","average":{"1min":0.66200000000000003,"5min":0.67000000000000004,"15min":0.69599999999999995},"min":{"1min":0.45000000000000001,"5min":0.41299999999999998,"15min":0.41299999999999998},"max":{"1min":0.94699999999999995,"5min":1.254,"15min":1.254},"last":0.60699999999999998}]},{"osd":6,"last update":"Sun Nov 26 07:21:48 2023","interfaces":[{"interface":"back","average":{"1min":0.61399999999999999,"5min":0.66200000000000003,"15min":0.66800000000000004},"min":{"1min":0.44400000000000001,"5min":0.439,"15min":0.439},"max":{"1min":1.0880000000000001,"5min":1.0880000000000001,"15min":1.0880000000000001},"last":0.66300000000000003},{"interface":"front","average":{"1min":0.66600000000000004,"5min":0.70299999999999996,"15min":0.71899999999999997},"min":{"1min":0.47699999999999998,"5min":0.436,"15min":0.436},"max":{"1min":1.052,"5min":1.0649999999999999,"15min":1.0649999999999999},"last":0.63700000000000001}]},{"osd":7,"last update":"Sun Nov 26 07:21:48 2023","interfaces":[{"interface":"back","average":{"1min":0.73099999999999998,"5min":0.75900000000000001,"15min":0.78900000000000003},"min":{"1min":0.51800000000000002,"5min":0.48199999999999998,"15min":0.48199999999999998},"max":{"1min":1.2010000000000001,"5min":1.4079999999999999,"15min":1.4079999999999999},"last":0.873},{"interface":"front","average":{"1min":0.72899999999999998,"5min":0.751,"15min":0.76700000000000002},"min":{"1min":0.52400000000000002,"5min":0.45000000000000001,"15min":0.45000000000000001},"max":{"1min":1.115,"5min":1.115,"15min":1.115},"last":0.89400000000000002}]}]},{"osd":4,"up_from":13,"seq":55834574912,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32028,"kb_used_data":848,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339812,"statfs":{"total":96636764160,"available":96603967488,"internally_reserved":0,"allocated":868352,"data_stored":364349,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Nov 26 07:21:53 2023","interfaces":[{"interface":"back","average":{"1min":0.67300000000000004,"5min":0.70899999999999996,"15min":0.76500000000000001},"min":{"1min":0.46100000000000002,"5min":0.32100000000000001,"15min":0.32100000000000001},"max":{"1min":0.94499999999999995,"5min":4.1689999999999996,"15min":4.1689999999999996},"last":0.63300000000000001},{"interface":"front","average":{"1min":1.609,"5min":0.98299999999999998,"15min":0.995},"min":{"1min":0.52800000000000002,"5min":0.46200000000000002,"15min":0.46200000000000002},"max":{"1min":18.021000000000001,"5min":18.021000000000001,"15min":18.021000000000001},"last":0.81699999999999995}]},{"osd":1,"last update":"Sun Nov 26 07:21:53 2023","interfaces":[{"interface":"back","average":{"1min":0.90500000000000003,"5min":0.71199999999999997,"15min":0.66800000000000004},"min":{"1min":0.47499999999999998,"5min":0.43099999999999999,"15min":0.43099999999999999},"max":{"1min":4.3159999999999998,"5min":4.3159999999999998,"15min":4.3159999999999998},"last":0.70599999999999996},{"interface":"front","average":{"1min":1.2150000000000001,"5min":0.82799999999999996,"15min":0.82199999999999995},"min":{"1min":0.51900000000000002,"5min":0.39800000000000002,"15min":0.39800000000000002},"max":{"1min":11.206,"5min":11.206,"15min":11.206},"last":0.65400000000000003}]},{"osd":2,"last update":"Sun Nov 26 07:21:53 2023","interfaces":[{"interface":"back","average":{"1min":1.492,"5min":0.89800000000000002,"15min":0.83699999999999997},"min":{"1min":0.47199999999999998,"5min":0.433,"15min":0.433},"max":{"1min":15.441000000000001,"5min":15.441000000000001,"15min":15.441000000000001},"last":0.748},{"interface":"front","average":{"1min":1.2350000000000001,"5min":0.97399999999999998,"15min":1.0349999999999999},"min":{"1min":0.59399999999999997,"5min":0.53400000000000003,"15min":0.53400000000000003},"max":{"1min":9.4800000000000004,"5min":9.4800000000000004,"15min":9.4800000000000004},"last":0.76900000000000002}]},{"osd":3,"last update":"Sun Nov 26 07:21:53 2023","interfaces":[{"interface":"back","average":{"1min":1.3700000000000001,"5min":0.83599999999999997,"15min":0.76500000000000001},"min":{"1min":0.57199999999999995,"5min":0.441,"15min":0.441},"max":{"1min":12.715,"5min":12.715,"15min":12.715},"last":0.498},{"interface":"front","average":{"1min":1.145,"5min":0.87,"15min":0.876},"min":{"1min":0.50700000000000001,"5min":0.50700000000000001,"15min":0.50700000000000001},"max":{"1min":8.3239999999999998,"5min":8.3239999999999998,"15min":8.3239999999999998},"last":0.79000000000000004}]},{"osd":5,"last update":"Sun Nov 26 07:21:53 2023","interfaces":[{"interface":"back","average":{"1min":0.63800000000000001,"5min":0.61299999999999999,"15min":0.61099999999999999},"min":{"1min":0.47099999999999997,"5min":0.38200000000000001,"15min":0.38200000000000001},"max":{"1min":0.93999999999999995,"5min":0.96199999999999997,"15min":0.96199999999999997},"last":0.86299999999999999},{"interface":"front","average":{"1min":1.165,"5min":0.72799999999999998,"15min":0.67000000000000004},"min":{"1min":0.442,"5min":0.35599999999999998,"15min":0.35599999999999998},"max":{"1min":11.148999999999999,"5min":11.148999999999999,"15min":11.148999999999999},"last":0.80800000000000005}]},{"osd":6,"last update":"Sun Nov 26 07:21:53 2023","interfaces":[{"interface":"back","average":{"1min":0.624,"5min":0.67500000000000004,"15min":0.754},"min":{"1min":0.41699999999999998,"5min":0.372,"15min":0.372},"max":{"1min":0.81799999999999995,"5min":2.8330000000000002,"15min":2.8330000000000002},"last":0.54400000000000004},{"interface":"front","average":{"1min":0.79200000000000004,"5min":0.83299999999999996,"15min":0.94999999999999996},"min":{"1min":0.54000000000000004,"5min":0.41299999999999998,"15min":0.41299999999999998},"max":{"1min":2.472,"5min":7.6050000000000004,"15min":7.6050000000000004},"last":0.60499999999999998}]},{"osd":7,"last update":"Sun Nov 26 07:21:53 2023","interfaces":[{"interface":"back","average":{"1min":1.339,"5min":0.88100000000000001,"15min":0.86699999999999999},"min":{"1min":0.57999999999999996,"5min":0.44700000000000001,"15min":0.44700000000000001},"max":{"1min":12.653,"5min":12.653,"15min":12.653},"last":0.72999999999999998},{"interface":"front","average":{"1min":0.90100000000000002,"5min":0.82199999999999995,"15min":0.86799999999999999},"min":{"1min":0.47499999999999998,"5min":0.33300000000000002,"15min":0.33300000000000002},"max":{"1min":3.8319999999999999,"5min":3.8319999999999999,"15min":3.8319999999999999},"last":0.93700000000000006}]}]},{"osd":3,"up_from":13,"seq":55834574913,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32032,"kb_used_data":852,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339808,"statfs":{"total":96636764160,"available":96603963392,"internally_reserved":0,"allocated":872448,"data_stored":364391,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1592,"internal_metadata":31914440},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Nov 26 07:21:48 2023","interfaces":[{"interface":"back","average":{"1min":0.46000000000000002,"5min":0.41299999999999998,"15min":0.39000000000000001},"min":{"1min":0.25800000000000001,"5min":0.25800000000000001,"15min":0.25800000000000001},"max":{"1min":0.72799999999999998,"5min":0.72799999999999998,"15min":0.72799999999999998},"last":0.51600000000000001},{"interface":"front","average":{"1min":0.48599999999999999,"5min":0.45900000000000002,"15min":0.441},"min":{"1min":0.316,"5min":0.26900000000000002,"15min":0.26900000000000002},"max":{"1min":0.71299999999999997,"5min":0.77300000000000002,"15min":0.77300000000000002},"last":0.5}]},{"osd":1,"last update":"Sun Nov 26 07:21:48 2023","interfaces":[{"interface":"back","average":{"1min":0.443,"5min":0.438,"15min":0.42599999999999999},"min":{"1min":0.34100000000000003,"5min":0.28799999999999998,"15min":0.28799999999999998},"max":{"1min":0.59299999999999997,"5min":0.752,"15min":0.752},"last":0.42999999999999999},{"interface":"front","average":{"1min":0.503,"5min":0.498,"15min":0.48799999999999999},"min":{"1min":0.374,"5min":0.34599999999999997,"15min":0.34599999999999997},"max":{"1min":0.76200000000000001,"5min":0.872,"15min":0.872},"last":0.44500000000000001}]},{"osd":2,"last update":"Sun Nov 26 07:21:48 2023","interfaces":[{"interface":"back","average":{"1min":0.58399999999999996,"5min":0.56299999999999994,"15min":0.54300000000000004},"min":{"1min":0.376,"5min":0.376,"15min":0.376},"max":{"1min":0.97199999999999998,"5min":0.97199999999999998,"15min":0.97199999999999998},"last":0.53500000000000003},{"interface":"front","average":{"1min":0.56799999999999995,"5min":0.53700000000000003,"15min":0.51500000000000001},"min":{"1min":0.40899999999999997,"5min":0.33000000000000002,"15min":0.33000000000000002},"max":{"1min":0.81999999999999995,"5min":0.89700000000000002,"15min":0.89700000000000002},"last":0.48199999999999998}]},{"osd":4,"last update":"Sun Nov 26 07:21:48 2023","interfaces":[{"interface":"back","average":{"1min":0.71099999999999997,"5min":0.71799999999999997,"15min":0.70999999999999996},"min":{"1min":0.45800000000000002,"5min":0.45800000000000002,"15min":0.45800000000000002},"max":{"1min":0.93000000000000005,"5min":1.091,"15min":1.091},"last":0.879},{"interface":"front","average":{"1min":0.73999999999999999,"5min":0.73899999999999999,"15min":0.73099999999999998},"min":{"1min":0.39900000000000002,"5min":0.39900000000000002,"15min":0.39900000000000002},"max":{"1min":1.0289999999999999,"5min":1.242,"15min":1.242},"last":0.75700000000000001}]},{"osd":5,"last update":"Sun Nov 26 07:21:48 2023","interfaces":[{"interface":"back","average":{"1min":0.70999999999999996,"5min":0.73299999999999998,"15min":0.73699999999999999},"min":{"1min":0.47099999999999997,"5min":0.47099999999999997,"15min":0.47099999999999997},"max":{"1min":0.94699999999999995,"5min":1.105,"15min":1.105},"last":0.73999999999999999},{"interface":"front","average":{"1min":0.73499999999999999,"5min":0.75,"15min":0.74199999999999999},"min":{"1min":0.48799999999999999,"5min":0.48799999999999999,"15min":0.48799999999999999},"max":{"1min":0.97399999999999998,"5min":1.1319999999999999,"15min":1.1319999999999999},"last":0.77400000000000002}]},{"osd":6,"last update":"Sun Nov 26 07:21:48 2023","interfaces":[{"interface":"back","average":{"1min":0.76100000000000001,"5min":0.79400000000000004,"15min":0.80700000000000005},"min":{"1min":0.503,"5min":0.503,"15min":0.503},"max":{"1min":0.89100000000000001,"5min":1.2729999999999999,"15min":1.2729999999999999},"last":0.81699999999999995},{"interface":"front","average":{"1min":0.77500000000000002,"5min":0.77900000000000003,"15min":0.78800000000000003},"min":{"1min":0.51100000000000001,"5min":0.47299999999999998,"15min":0.47299999999999998},"max":{"1min":1.002,"5min":1.204,"15min":1.204},"last":0.82799999999999996}]},{"osd":7,"last update":"Sun Nov 26 07:21:48 2023","interfaces":[{"interface":"back","average":{"1min":0.78900000000000003,"5min":0.77700000000000002,"15min":0.79300000000000004},"min":{"1min":0.54500000000000004,"5min":0.5,"15min":0.5},"max":{"1min":0.999,"5min":1.3600000000000001,"15min":1.3600000000000001},"last":0.80600000000000005},{"interface":"front","average":{"1min":0.77600000000000002,"5min":0.81699999999999995,"15min":0.82299999999999995},"min":{"1min":0.47299999999999998,"5min":0.47299999999999998,"15min":0.47299999999999998},"max":{"1min":1.0349999999999999,"5min":1.617,"15min":1.617},"last":0.84999999999999998}]}]},{"osd":2,"up_from":13,"seq":55834574913,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27884,"kb_used_data":856,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343956,"statfs":{"total":96636764160,"available":96608210944,"internally_reserved":0,"allocated":876544,"data_stored":365891,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Nov 26 07:21:47 2023","interfaces":[{"interface":"back","average":{"1min":0.42299999999999999,"5min":0.55200000000000005,"15min":0.66000000000000003},"min":{"1min":0.29799999999999999,"5min":0.28699999999999998,"15min":0.28699999999999998},"max":{"1min":0.72899999999999998,"5min":4.7030000000000003,"15min":4.7030000000000003},"last":0.44700000000000001},{"interface":"front","average":{"1min":0.48299999999999998,"5min":0.51300000000000001,"15min":0.54500000000000004},"min":{"1min":0.27300000000000002,"5min":0.27300000000000002,"15min":0.27300000000000002},"max":{"1min":0.71899999999999997,"5min":2.4340000000000002,"15min":2.4340000000000002},"last":0.49299999999999999}]},{"osd":1,"last update":"Sun Nov 26 07:21:47 2023","interfaces":[{"interface":"back","average":{"1min":0.52400000000000002,"5min":0.64900000000000002,"15min":0.79200000000000004},"min":{"1min":0.36099999999999999,"5min":0.34599999999999997,"15min":0.34599999999999997},"max":{"1min":0.86099999999999999,"5min":4.2690000000000001,"15min":4.2690000000000001},"last":0.54600000000000004},{"interface":"front","average":{"1min":0.52700000000000002,"5min":0.60799999999999998,"15min":0.69699999999999995},"min":{"1min":0.376,"5min":0.31900000000000001,"15min":0.31900000000000001},"max":{"1min":0.70899999999999996,"5min":3.9940000000000002,"15min":3.9940000000000002},"last":0.42099999999999999}]},{"osd":3,"last update":"Sun Nov 26 07:21:47 2023","interfaces":[{"interface":"back","average":{"1min":0.497,"5min":0.55200000000000005,"15min":0.61199999999999999},"min":{"1min":0.32200000000000001,"5min":0.32000000000000001,"15min":0.32000000000000001},"max":{"1min":0.82699999999999996,"5min":4.0300000000000002,"15min":4.0300000000000002},"last":0.39000000000000001},{"interface":"front","average":{"1min":0.56399999999999995,"5min":0.70299999999999996,"15min":0.84299999999999997},"min":{"1min":0.36099999999999999,"5min":0.36099999999999999,"15min":0.36099999999999999},"max":{"1min":0.86699999999999999,"5min":4.5810000000000004,"15min":4.5810000000000004},"last":0.52300000000000002}]},{"osd":4,"last update":"Sun Nov 26 07:21:47 2023","interfaces":[{"interface":"back","average":{"1min":0.78500000000000003,"5min":0.85599999999999998,"15min":0.94199999999999995},"min":{"1min":0.58899999999999997,"5min":0.51400000000000001,"15min":0.51400000000000001},"max":{"1min":0.98699999999999999,"5min":4.4269999999999996,"15min":4.4269999999999996},"last":0.72199999999999998},{"interface":"front","average":{"1min":0.83799999999999997,"5min":0.88100000000000001,"15min":0.96999999999999997},"min":{"1min":0.55500000000000005,"5min":0.55500000000000005,"15min":0.55500000000000005},"max":{"1min":1.0780000000000001,"5min":4.085,"15min":4.085},"last":0.67600000000000005}]},{"osd":5,"last update":"Sun Nov 26 07:21:47 2023","interfaces":[{"interface":"back","average":{"1min":0.81200000000000006,"5min":0.81000000000000005,"15min":0.85199999999999998},"min":{"1min":0.60699999999999998,"5min":0.49199999999999999,"15min":0.49199999999999999},"max":{"1min":1,"5min":3.9590000000000001,"15min":3.9590000000000001},"last":0.82099999999999995},{"interface":"front","average":{"1min":0.81200000000000006,"5min":0.874,"15min":0.94999999999999996},"min":{"1min":0.499,"5min":0.499,"15min":0.499},"max":{"1min":1.024,"5min":4.3849999999999998,"15min":4.3849999999999998},"last":0.83499999999999996}]},{"osd":6,"last update":"Sun Nov 26 07:21:47 2023","interfaces":[{"interface":"back","average":{"1min":0.79100000000000004,"5min":0.86199999999999999,"15min":0.95099999999999996},"min":{"1min":0.51800000000000002,"5min":0.51800000000000002,"15min":0.51800000000000002},"max":{"1min":0.94999999999999996,"5min":4.0449999999999999,"15min":4.0449999999999999},"last":0.76700000000000002},{"interface":"front","average":{"1min":0.77000000000000002,"5min":0.83499999999999996,"15min":0.89000000000000001},"min":{"1min":0.48099999999999998,"5min":0.48099999999999998,"15min":0.48099999999999998},"max":{"1min":1,"5min":3.992,"15min":3.992},"last":0.74299999999999999}]},{"osd":7,"last update":"Sun Nov 26 07:21:47 2023","interfaces":[{"interface":"back","average":{"1min":0.81599999999999995,"5min":0.88600000000000001,"15min":0.98499999999999999},"min":{"1min":0.63900000000000001,"5min":0.52200000000000002,"15min":0.52200000000000002},"max":{"1min":1.0149999999999999,"5min":4.1219999999999999,"15min":4.1219999999999999},"last":0.81100000000000005},{"interface":"front","average":{"1min":0.80100000000000005,"5min":0.85699999999999998,"15min":0.91100000000000003},"min":{"1min":0.50700000000000001,"5min":0.50700000000000001,"15min":0.50700000000000001},"max":{"1min":0.96599999999999997,"5min":3.9159999999999999,"15min":3.9159999999999999},"last":0.85199999999999998}]}]},{"osd":1,"up_from":13,"seq":55834574912,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32052,"kb_used_data":864,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339788,"statfs":{"total":96636764160,"available":96603942912,"internally_reserved":0,"allocated":884736,"data_stored":366507,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":31914439},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Nov 26 07:21:45 2023","interfaces":[{"interface":"back","average":{"1min":0.39400000000000002,"5min":0.40999999999999998,"15min":0.41399999999999998},"min":{"1min":0.29899999999999999,"5min":0.23599999999999999,"15min":0.23599999999999999},"max":{"1min":0.57499999999999996,"5min":0.61199999999999999,"15min":0.61199999999999999},"last":0.45200000000000001},{"interface":"front","average":{"1min":0.435,"5min":0.46200000000000002,"15min":0.47799999999999998},"min":{"1min":0.28899999999999998,"5min":0.26700000000000002,"15min":0.26700000000000002},"max":{"1min":0.61699999999999999,"5min":0.78700000000000003,"15min":0.78700000000000003},"last":0.499}]},{"osd":2,"last update":"Sun Nov 26 07:21:45 2023","interfaces":[{"interface":"back","average":{"1min":0.42999999999999999,"5min":0.443,"15min":0.45100000000000001},"min":{"1min":0.35499999999999998,"5min":0.27800000000000002,"15min":0.27800000000000002},"max":{"1min":0.53600000000000003,"5min":0.67600000000000005,"15min":0.67600000000000005},"last":0.38300000000000001},{"interface":"front","average":{"1min":0.47399999999999998,"5min":0.498,"15min":0.504},"min":{"1min":0.35999999999999999,"5min":0.27700000000000002,"15min":0.27700000000000002},"max":{"1min":0.64200000000000002,"5min":0.80600000000000005,"15min":0.80600000000000005},"last":0.48199999999999998}]},{"osd":3,"last update":"Sun Nov 26 07:21:45 2023","interfaces":[{"interface":"back","average":{"1min":0.51200000000000001,"5min":0.52800000000000002,"15min":0.52900000000000003},"min":{"1min":0.42099999999999999,"5min":0.38400000000000001,"15min":0.38400000000000001},"max":{"1min":0.60799999999999998,"5min":0.748,"15min":0.748},"last":0.51700000000000002},{"interface":"front","average":{"1min":0.496,"5min":0.499,"15min":0.497},"min":{"1min":0.35499999999999998,"5min":0.31,"15min":0.31},"max":{"1min":0.68600000000000005,"5min":0.80700000000000005,"15min":0.80700000000000005},"last":0.46400000000000002}]},{"osd":4,"last update":"Sun Nov 26 07:21:45 2023","interfaces":[{"interface":"back","average":{"1min":0.749,"5min":0.71999999999999997,"15min":0.70199999999999996},"min":{"1min":0.51500000000000001,"5min":0.41299999999999998,"15min":0.41299999999999998},"max":{"1min":1.0589999999999999,"5min":1.1000000000000001,"15min":1.1000000000000001},"last":0.97999999999999998},{"interface":"front","average":{"1min":0.71999999999999997,"5min":0.71399999999999997,"15min":0.69099999999999995},"min":{"1min":0.44500000000000001,"5min":0.44500000000000001,"15min":0.44500000000000001},"max":{"1min":0.93300000000000005,"5min":1.131,"15min":1.131},"last":0.66600000000000004}]},{"osd":5,"last update":"Sun Nov 26 07:21:45 2023","interfaces":[{"interface":"back","average":{"1min":0.73399999999999999,"5min":0.73099999999999998,"15min":0.72299999999999998},"min":{"1min":0.52000000000000002,"5min":0.49299999999999999,"15min":0.49299999999999999},"max":{"1min":0.89900000000000002,"5min":1.1919999999999999,"15min":1.1919999999999999},"last":0.82899999999999996},{"interface":"front","average":{"1min":0.70399999999999996,"5min":0.72699999999999998,"15min":0.71999999999999997},"min":{"1min":0.54700000000000004,"5min":0.48799999999999999,"15min":0.48799999999999999},"max":{"1min":0.93400000000000005,"5min":1.0609999999999999,"15min":1.0609999999999999},"last":0.93899999999999995}]},{"osd":6,"last update":"Sun Nov 26 07:21:45 2023","interfaces":[{"interface":"back","average":{"1min":0.76000000000000001,"5min":0.73199999999999998,"15min":0.71299999999999997},"min":{"1min":0.55000000000000004,"5min":0.50600000000000001,"15min":0.50600000000000001},"max":{"1min":0.97799999999999998,"5min":1.0369999999999999,"15min":1.0369999999999999},"last":0.90400000000000003},{"interface":"front","average":{"1min":0.74399999999999999,"5min":0.73399999999999999,"15min":0.71199999999999997},"min":{"1min":0.59999999999999998,"5min":0.504,"15min":0.504},"max":{"1min":0.94999999999999996,"5min":1.1339999999999999,"15min":1.1339999999999999},"last":0.79100000000000004}]},{"osd":7,"last update":"Sun Nov 26 07:21:45 2023","interfaces":[{"interface":"back","average":{"1min":0.75,"5min":0.74099999999999999,"15min":0.72299999999999998},"min":{"1min":0.50600000000000001,"5min":0.50600000000000001,"15min":0.50600000000000001},"max":{"1min":0.92400000000000004,"5min":1.002,"15min":1.002},"last":0.84099999999999997},{"interface":"front","average":{"1min":0.73699999999999999,"5min":0.747,"15min":0.751},"min":{"1min":0.56999999999999995,"5min":0.48499999999999999,"15min":0.48499999999999999},"max":{"1min":1.018,"5min":1.024,"15min":1.024},"last":0.95299999999999996}]}]},{"osd":0,"up_from":13,"seq":55834574912,"num_pgs":15,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32608,"kb_used_data":1424,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339232,"statfs":{"total":96636764160,"available":96603373568,"internally_reserved":0,"allocated":1458176,"data_stored":954647,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Sun Nov 26 07:21:43 2023","interfaces":[{"interface":"back","average":{"1min":0.438,"5min":0.42599999999999999,"15min":0.42899999999999999},"min":{"1min":0.27000000000000002,"5min":0.27000000000000002,"15min":0.27000000000000002},"max":{"1min":0.624,"5min":0.64300000000000002,"15min":0.64300000000000002},"last":0.33700000000000002},{"interface":"front","average":{"1min":0.47299999999999998,"5min":0.46800000000000003,"15min":0.47799999999999998},"min":{"1min":0.32200000000000001,"5min":0.29899999999999999,"15min":0.29899999999999999},"max":{"1min":0.77200000000000002,"5min":1.133,"15min":1.133},"last":0.36899999999999999}]},{"osd":2,"last update":"Sun Nov 26 07:21:43 2023","interfaces":[{"interface":"back","average":{"1min":0.496,"5min":0.51800000000000002,"15min":0.53000000000000003},"min":{"1min":0.34699999999999998,"5min":0.34699999999999998,"15min":0.34699999999999998},"max":{"1min":0.84199999999999997,"5min":0.84199999999999997,"15min":0.84199999999999997},"last":0.69399999999999995},{"interface":"front","average":{"1min":0.56299999999999994,"5min":0.55700000000000005,"15min":0.56100000000000005},"min":{"1min":0.41399999999999998,"5min":0.36399999999999999,"15min":0.36399999999999999},"max":{"1min":0.77300000000000002,"5min":0.84199999999999997,"15min":0.84199999999999997},"last":0.40100000000000002}]},{"osd":3,"last update":"Sun Nov 26 07:21:43 2023","interfaces":[{"interface":"back","average":{"1min":0.52200000000000002,"5min":0.52200000000000002,"15min":0.52800000000000002},"min":{"1min":0.39000000000000001,"5min":0.34999999999999998,"15min":0.34999999999999998},"max":{"1min":0.75700000000000001,"5min":1.3149999999999999,"15min":1.3149999999999999},"last":0.873},{"interface":"front","average":{"1min":0.55900000000000005,"5min":0.54400000000000004,"15min":0.55600000000000005},"min":{"1min":0.40000000000000002,"5min":0.371,"15min":0.371},"max":{"1min":0.72799999999999998,"5min":1.2509999999999999,"15min":1.2509999999999999},"last":0.42099999999999999}]},{"osd":4,"last update":"Sun Nov 26 07:21:43 2023","interfaces":[{"interface":"back","average":{"1min":0.747,"5min":0.749,"15min":0.755},"min":{"1min":0.44700000000000001,"5min":0.44700000000000001,"15min":0.44700000000000001},"max":{"1min":1.0900000000000001,"5min":1.2010000000000001,"15min":1.2010000000000001},"last":0.81200000000000006},{"interface":"front","average":{"1min":0.78000000000000003,"5min":0.78800000000000003,"15min":0.78800000000000003},"min":{"1min":0.51100000000000001,"5min":0.51100000000000001,"15min":0.51100000000000001},"max":{"1min":0.99199999999999999,"5min":1.381,"15min":1.381},"last":0.83599999999999997}]},{"osd":5,"last update":"Sun Nov 26 07:21:43 2023","interfaces":[{"interface":"back","average":{"1min":0.75900000000000001,"5min":0.76100000000000001,"15min":0.76000000000000001},"min":{"1min":0.54800000000000004,"5min":0.42199999999999999,"15min":0.42199999999999999},"max":{"1min":0.97999999999999998,"5min":1.1579999999999999,"15min":1.1579999999999999},"last":0.88700000000000001},{"interface":"front","average":{"1min":0.81499999999999995,"5min":0.80600000000000005,"15min":0.80000000000000004},"min":{"1min":0.53600000000000003,"5min":0.53600000000000003,"15min":0.53600000000000003},"max":{"1min":1.0680000000000001,"5min":1.399,"15min":1.399},"last":0.78900000000000003}]},{"osd":6,"last update":"Sun Nov 26 07:21:43 2023","interfaces":[{"interface":"back","average":{"1min":0.72299999999999998,"5min":0.72199999999999998,"15min":0.75},"min":{"1min":0.52400000000000002,"5min":0.50800000000000001,"15min":0.50800000000000001},"max":{"1min":0.89300000000000002,"5min":1.3029999999999999,"15min":1.3029999999999999},"last":0.71299999999999997},{"interface":"front","average":{"1min":0.747,"5min":0.75700000000000001,"15min":0.76500000000000001},"min":{"1min":0.47199999999999998,"5min":0.47199999999999998,"15min":0.47199999999999998},"max":{"1min":1.0489999999999999,"5min":1.3600000000000001,"15min":1.3600000000000001},"last":0.73199999999999998}]},{"osd":7,"last update":"Sun Nov 26 07:21:43 2023","interfaces":[{"interface":"back","average":{"1min":0.77200000000000002,"5min":0.78800000000000003,"15min":0.80300000000000005},"min":{"1min":0.56899999999999995,"5min":0.56899999999999995,"15min":0.56899999999999995},"max":{"1min":0.98899999999999999,"5min":1.177,"15min":1.177},"last":0.82599999999999996},{"interface":"front","average":{"1min":0.78600000000000003,"5min":0.76700000000000002,"15min":0.78800000000000003},"min":{"1min":0.51200000000000001,"5min":0.51200000000000001,"15min":0.51200000000000001},"max":{"1min":1.0629999999999999,"5min":1.2390000000000001,"15min":1.2390000000000001},"last":0.85499999999999998}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":2228,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":112,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":70,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":92,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":616,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-11-26T07:22:41.925 DEBUG:teuthology.orchestra.run.smithi143:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-11-26T07:22:42.104 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:42.104+0000 7f2ee99f8640 1 Processor -- start 2023-11-26T07:22:42.104 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:42.104+0000 7f2ee99f8640 1 -- start start 2023-11-26T07:22:42.105 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:42.104+0000 7f2ee99f8640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f2ee415d2a0 0x7f2ee40d00e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:22:42.105 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:42.104+0000 7f2ee99f8640 1 -- --> v1:172.21.15.143:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2ee40cdc40 con 0x7f2ee40d1d90 2023-11-26T07:22:42.105 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:42.104+0000 7f2ee99f8640 1 -- --> v1:172.21.15.188:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2ee40ce7a0 con 0x7f2ee415d680 2023-11-26T07:22:42.105 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:42.104+0000 7f2ee99f8640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2ee40d1440 con 0x7f2ee415d2a0 2023-11-26T07:22:42.107 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:42.104+0000 7f2ee91f7640 1 --1- >> v1:172.21.15.143:6789/0 conn(0x7f2ee40d1d90 0x7f2ee40b2480 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.143:6789/0 says I am v1:172.21.15.143:35792/0 (socket says 172.21.15.143:35792) 2023-11-26T07:22:42.107 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:42.104+0000 7f2ee91f7640 1 -- 172.21.15.143:0/1581110776 learned_addr learned my addr 172.21.15.143:0/1581110776 (peer_addr_for_me v1:172.21.15.143:0/0) 2023-11-26T07:22:42.107 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:42.104+0000 7f2ee89f6640 1 --2- 172.21.15.143:0/1581110776 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f2ee415d2a0 0x7f2ee40d00e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:22:42.107 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:42.104+0000 7f2ee37fe640 1 -- 172.21.15.143:0/1581110776 <== mon.0 v1:172.21.15.143:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1753391693 0 0) 0x7f2ee40cdc40 con 0x7f2ee40d1d90 2023-11-26T07:22:42.107 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:42.104+0000 7f2ee37fe640 1 -- 172.21.15.143:0/1581110776 --> v1:172.21.15.143:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f2ed4003530 con 0x7f2ee40d1d90 2023-11-26T07:22:42.107 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:42.104+0000 7f2ee37fe640 1 -- 172.21.15.143:0/1581110776 <== mon.1 v1:172.21.15.188:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2831540139 0 0) 0x7f2ee40ce7a0 con 0x7f2ee415d680 2023-11-26T07:22:42.107 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:42.104+0000 7f2ee37fe640 1 -- 172.21.15.143:0/1581110776 --> v1:172.21.15.188:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f2ee40cdc40 con 0x7f2ee415d680 2023-11-26T07:22:42.107 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:42.104+0000 7f2ee37fe640 1 -- 172.21.15.143:0/1581110776 <== mon.0 v1:172.21.15.143:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2850326430 0 0) 0x7f2ed4003530 con 0x7f2ee40d1d90 2023-11-26T07:22:42.107 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:42.104+0000 7f2ee37fe640 1 -- 172.21.15.143:0/1581110776 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f2ee415d2a0 msgr2=0x7f2ee40d00e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-26T07:22:42.107 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:42.104+0000 7f2ee37fe640 1 --2- 172.21.15.143:0/1581110776 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f2ee415d2a0 0x7f2ee40d00e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:22:42.107 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:42.104+0000 7f2ee37fe640 1 -- 172.21.15.143:0/1581110776 >> v1:172.21.15.188:6789/0 conn(0x7f2ee415d680 legacy=0x7f2ee40d1680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:22:42.108 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:42.104+0000 7f2ee37fe640 1 -- 172.21.15.143:0/1581110776 --> v1:172.21.15.143:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2ee40cde70 con 0x7f2ee40d1d90 2023-11-26T07:22:42.108 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:42.104+0000 7f2ee37fe640 1 -- 172.21.15.143:0/1581110776 <== mon.0 v1:172.21.15.143:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2137882523 0 0) 0x7f2edc001a00 con 0x7f2ee40d1d90 2023-11-26T07:22:42.108 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:42.104+0000 7f2ee37fe640 1 -- 172.21.15.143:0/1581110776 >> v1:172.21.15.143:6789/0 conn(0x7f2ee40d1d90 legacy=0x7f2ee40b2480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:22:42.108 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:42.104+0000 7f2ee37fe640 1 --2- 172.21.15.143:0/1581110776 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f2ed4003b40 0x7f2ed4003f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:22:42.108 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:42.104+0000 7f2ee37fe640 1 --2- 172.21.15.143:0/1581110776 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f2ee415d2a0 0x7f2ed40047d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:22:42.108 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:42.104+0000 7f2ee37fe640 1 --2- 172.21.15.143:0/1581110776 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f2ed4004d10 0x7f2ed4005190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:22:42.108 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:42.104+0000 7f2ee37fe640 1 -- 172.21.15.143:0/1581110776 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2ee40d1440 con 0x7f2ed4004d10 2023-11-26T07:22:42.108 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:42.104+0000 7f2ee37fe640 1 -- 172.21.15.143:0/1581110776 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2ed400ba40 con 0x7f2ed4003b40 2023-11-26T07:22:42.108 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:42.104+0000 7f2ee37fe640 1 -- 172.21.15.143:0/1581110776 --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2ed400bc00 con 0x7f2ee415d2a0 2023-11-26T07:22:42.108 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:42.104+0000 7f2ee91f7640 1 --2- 172.21.15.143:0/1581110776 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f2ed4004d10 0x7f2ed4005190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:22:42.108 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:42.104+0000 7f2ee3fff640 1 --2- 172.21.15.143:0/1581110776 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f2ee415d2a0 0x7f2ed40047d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:22:42.108 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:42.104+0000 7f2ee89f6640 1 --2- 172.21.15.143:0/1581110776 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f2ed4003b40 0x7f2ed4003f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:22:42.109 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:42.104+0000 7f2ee91f7640 1 -- 172.21.15.143:0/1581110776 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f2ee415d2a0 msgr2=0x7f2ed40047d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-26T07:22:42.109 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:42.104+0000 7f2ee91f7640 1 --2- 172.21.15.143:0/1581110776 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f2ee415d2a0 0x7f2ed40047d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:22:42.109 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:42.104+0000 7f2ee91f7640 1 -- 172.21.15.143:0/1581110776 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f2ed4003b40 msgr2=0x7f2ed4003f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-26T07:22:42.109 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:42.104+0000 7f2ee91f7640 1 --2- 172.21.15.143:0/1581110776 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f2ed4003b40 0x7f2ed4003f40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:22:42.109 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:42.104+0000 7f2ee91f7640 1 -- 172.21.15.143:0/1581110776 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2ed4003530 con 0x7f2ed4004d10 2023-11-26T07:22:42.109 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:42.108+0000 7f2ee91f7640 1 --2- 172.21.15.143:0/1581110776 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f2ed4004d10 0x7f2ed4005190 secure :-1 s=READY pgs=965 cs=0 l=1 rev1=1 crypto rx=0x7f2edc00a6f0 tx=0x7f2edc01f940 comp rx=0 tx=0).ready entity=mon.0 client_cookie=78528c2585957701 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:22:42.110 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:42.108+0000 7f2ee37fe640 1 -- 172.21.15.143:0/1581110776 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2edc0ac6b0 con 0x7f2ed4004d10 2023-11-26T07:22:42.110 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:42.108+0000 7f2ee37fe640 1 -- 172.21.15.143:0/1581110776 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2edc0ac810 con 0x7f2ed4004d10 2023-11-26T07:22:42.110 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:42.108+0000 7f2ee37fe640 1 -- 172.21.15.143:0/1581110776 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2edc0acc00 con 0x7f2ed4004d10 2023-11-26T07:22:42.111 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:42.108+0000 7f2ee99f8640 1 -- 172.21.15.143:0/1581110776 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f2ed4004d10 msgr2=0x7f2ed4005190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:22:42.111 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:42.108+0000 7f2ee99f8640 1 --2- 172.21.15.143:0/1581110776 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f2ed4004d10 0x7f2ed4005190 secure :-1 s=READY pgs=965 cs=0 l=1 rev1=1 crypto rx=0x7f2edc00a6f0 tx=0x7f2edc01f940 comp rx=0 tx=0).stop 2023-11-26T07:22:42.111 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:42.108+0000 7f2ee91f7640 1 -- 172.21.15.143:0/1581110776 reap_dead start 2023-11-26T07:22:42.111 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:42.108+0000 7f2ee99f8640 1 -- 172.21.15.143:0/1581110776 shutdown_connections 2023-11-26T07:22:42.111 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:42.108+0000 7f2ee99f8640 1 -- 172.21.15.143:0/1581110776 >> 172.21.15.143:0/1581110776 conn(0x7f2ee40d30a0 msgr2=0x7f2ee40d3480 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:22:42.111 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:42.108+0000 7f2ee99f8640 1 -- 172.21.15.143:0/1581110776 shutdown_connections 2023-11-26T07:22:42.111 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:42.108+0000 7f2ee99f8640 1 -- 172.21.15.143:0/1581110776 wait complete. 2023-11-26T07:22:42.112 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:42.112+0000 7f2ee99f8640 1 Processor -- start 2023-11-26T07:22:42.112 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:42.112+0000 7f2ee99f8640 1 -- start start 2023-11-26T07:22:42.114 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:42.112+0000 7f2ee99f8640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f2ee41f24e0 0x7f2ee40b62f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:22:42.115 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:42.112+0000 7f2ee99f8640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f2ee40b2460 0x7f2ee40b8980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:22:42.115 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:42.112+0000 7f2ee99f8640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f2ee41f28c0 0x7f2ee41f62c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:22:42.115 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:42.112+0000 7f2ee99f8640 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2ee40b2970 con 0x7f2ee40b2460 2023-11-26T07:22:42.115 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:42.112+0000 7f2ee99f8640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2ee40cb1c0 con 0x7f2ee41f24e0 2023-11-26T07:22:42.115 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:42.112+0000 7f2ee99f8640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2ee40cb4a0 con 0x7f2ee41f28c0 2023-11-26T07:22:42.115 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:42.112+0000 7f2ee3fff640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f2ee40b2460 0x7f2ee40b8980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:22:42.115 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:42.112+0000 7f2ee3fff640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f2ee40b2460 0x7f2ee40b8980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:47300/0 (socket says 172.21.15.143:47300) 2023-11-26T07:22:42.115 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:42.112+0000 7f2ee3fff640 1 -- 172.21.15.143:0/1748713903 learned_addr learned my addr 172.21.15.143:0/1748713903 (peer_addr_for_me v2:172.21.15.143:0/0) 2023-11-26T07:22:42.115 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:42.112+0000 7f2ee89f6640 1 --2- 172.21.15.143:0/1748713903 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f2ee41f24e0 0x7f2ee40b62f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:22:42.115 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:42.112+0000 7f2ee91f7640 1 --2- 172.21.15.143:0/1748713903 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f2ee41f28c0 0x7f2ee41f62c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:22:42.115 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:42.112+0000 7f2ee3fff640 1 -- 172.21.15.143:0/1748713903 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f2ee41f28c0 msgr2=0x7f2ee41f62c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:22:42.115 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:42.112+0000 7f2ee3fff640 1 --2- 172.21.15.143:0/1748713903 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f2ee41f28c0 0x7f2ee41f62c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:22:42.115 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:42.112+0000 7f2ee3fff640 1 -- 172.21.15.143:0/1748713903 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f2ee41f24e0 msgr2=0x7f2ee40b62f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:22:42.116 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:42.112+0000 7f2ee3fff640 1 --2- 172.21.15.143:0/1748713903 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f2ee41f24e0 0x7f2ee40b62f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:22:42.116 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:42.112+0000 7f2ee3fff640 1 -- 172.21.15.143:0/1748713903 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2ee40cb780 con 0x7f2ee40b2460 2023-11-26T07:22:42.116 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:42.112+0000 7f2ee3fff640 1 --2- 172.21.15.143:0/1748713903 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f2ee40b2460 0x7f2ee40b8980 secure :-1 s=READY pgs=966 cs=0 l=1 rev1=1 crypto rx=0x7f2ed000f770 tx=0x7f2ed000fb70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:22:42.116 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:42.112+0000 7f2ee17fa640 1 -- 172.21.15.143:0/1748713903 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2ed0005110 con 0x7f2ee40b2460 2023-11-26T07:22:42.117 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:42.112+0000 7f2ee17fa640 1 -- 172.21.15.143:0/1748713903 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2ed0005290 con 0x7f2ee40b2460 2023-11-26T07:22:42.117 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:42.112+0000 7f2ee17fa640 1 -- 172.21.15.143:0/1748713903 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2ed00067e0 con 0x7f2ee40b2460 2023-11-26T07:22:42.117 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:42.112+0000 7f2ee99f8640 1 -- 172.21.15.143:0/1748713903 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2ee40cba60 con 0x7f2ee40b2460 2023-11-26T07:22:42.117 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:42.112+0000 7f2ee99f8640 1 -- 172.21.15.143:0/1748713903 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2ee40b8ec0 con 0x7f2ee40b2460 2023-11-26T07:22:42.117 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:42.116+0000 7f2ee99f8640 1 -- 172.21.15.143:0/1748713903 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2ee4067500 con 0x7f2ee40b2460 2023-11-26T07:22:42.117 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:42.116+0000 7f2ee17fa640 1 -- 172.21.15.143:0/1748713903 <== mon.0 v2:172.21.15.143:3300/0 4 ==== mgrmap(e 11) v1 ==== 86158+0+0 (secure 0 0 0) 0x7f2ed0006940 con 0x7f2ee40b2460 2023-11-26T07:22:42.118 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:42.116+0000 7f2ee17fa640 1 --2- 172.21.15.143:0/1748713903 >> [v2:172.21.15.143:6832/2150557544,v1:172.21.15.143:6833/2150557544] conn(0x7f2ec8066800 0x7f2ec8068cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:22:42.118 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:42.116+0000 7f2ee17fa640 1 -- 172.21.15.143:0/1748713903 <== mon.0 v2:172.21.15.143:3300/0 5 ==== osd_map(71..71 src has 1..71) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f2ed0083950 con 0x7f2ee40b2460 2023-11-26T07:22:42.118 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:42.116+0000 7f2ee89f6640 1 --2- 172.21.15.143:0/1748713903 >> [v2:172.21.15.143:6832/2150557544,v1:172.21.15.143:6833/2150557544] conn(0x7f2ec8066800 0x7f2ec8068cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:22:42.118 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:42.116+0000 7f2ee89f6640 1 --2- 172.21.15.143:0/1748713903 >> [v2:172.21.15.143:6832/2150557544,v1:172.21.15.143:6833/2150557544] conn(0x7f2ec8066800 0x7f2ec8068cc0 secure :-1 s=READY pgs=453 cs=0 l=1 rev1=1 crypto rx=0x7f2ed8009be0 tx=0x7f2ed8009190 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:22:42.121 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:42.120+0000 7f2ee17fa640 1 -- 172.21.15.143:0/1748713903 <== mon.0 v2:172.21.15.143:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172649 (secure 0 0 0) 0x7f2ed0054500 con 0x7f2ee40b2460 2023-11-26T07:22:42.268 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:42.268+0000 7f2ee99f8640 1 -- 172.21.15.143:0/1748713903 --> [v2:172.21.15.143:6832/2150557544,v1:172.21.15.143:6833/2150557544] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f2ee40d3e50 con 0x7f2ec8066800 2023-11-26T07:22:42.275 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:42.276+0000 7f2ee17fa640 1 -- 172.21.15.143:0/1748713903 <== mgr.4100 v2:172.21.15.143:6832/2150557544 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+289343 (secure 0 0 0) 0x7f2ed8019070 con 0x7f2ec8066800 2023-11-26T07:22:42.278 INFO:teuthology.orchestra.run.smithi143.stdout: 2023-11-26T07:22:42.279 INFO:teuthology.orchestra.run.smithi143.stderr:dumped all 2023-11-26T07:22:42.281 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:42.276+0000 7f2ee99f8640 1 -- 172.21.15.143:0/1748713903 >> [v2:172.21.15.143:6832/2150557544,v1:172.21.15.143:6833/2150557544] conn(0x7f2ec8066800 msgr2=0x7f2ec8068cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:22:42.281 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:42.276+0000 7f2ee99f8640 1 --2- 172.21.15.143:0/1748713903 >> [v2:172.21.15.143:6832/2150557544,v1:172.21.15.143:6833/2150557544] conn(0x7f2ec8066800 0x7f2ec8068cc0 secure :-1 s=READY pgs=453 cs=0 l=1 rev1=1 crypto rx=0x7f2ed8009be0 tx=0x7f2ed8009190 comp rx=0 tx=0).stop 2023-11-26T07:22:42.281 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:42.280+0000 7f2ee99f8640 1 -- 172.21.15.143:0/1748713903 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f2ee40b2460 msgr2=0x7f2ee40b8980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:22:42.281 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:42.280+0000 7f2ee99f8640 1 --2- 172.21.15.143:0/1748713903 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f2ee40b2460 0x7f2ee40b8980 secure :-1 s=READY pgs=966 cs=0 l=1 rev1=1 crypto rx=0x7f2ed000f770 tx=0x7f2ed000fb70 comp rx=0 tx=0).stop 2023-11-26T07:22:42.281 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:42.280+0000 7f2ee99f8640 1 -- 172.21.15.143:0/1748713903 shutdown_connections 2023-11-26T07:22:42.281 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:42.280+0000 7f2ee99f8640 1 --2- 172.21.15.143:0/1748713903 >> [v2:172.21.15.143:6832/2150557544,v1:172.21.15.143:6833/2150557544] conn(0x7f2ec8066800 0x7f2ec8068cc0 unknown :-1 s=CLOSED pgs=453 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:22:42.281 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:42.280+0000 7f2ee99f8640 1 --2- 172.21.15.143:0/1748713903 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f2ee41f28c0 0x7f2ee41f62c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:22:42.281 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:42.280+0000 7f2ee99f8640 1 --2- 172.21.15.143:0/1748713903 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f2ee40b2460 0x7f2ee40b8980 unknown :-1 s=CLOSED pgs=966 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:22:42.281 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:42.280+0000 7f2ee99f8640 1 --2- 172.21.15.143:0/1748713903 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f2ee41f24e0 0x7f2ee40b62f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:22:42.281 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:42.280+0000 7f2ee99f8640 1 -- 172.21.15.143:0/1748713903 >> 172.21.15.143:0/1748713903 conn(0x7f2ee40d30a0 msgr2=0x7f2ee40d1680 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:22:42.281 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:42.280+0000 7f2ee99f8640 1 -- 172.21.15.143:0/1748713903 shutdown_connections 2023-11-26T07:22:42.281 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:42.280+0000 7f2ee99f8640 1 -- 172.21.15.143:0/1748713903 wait complete. 2023-11-26T07:22:42.299 INFO:teuthology.orchestra.run.smithi143.stdout:{"pg_ready":true,"pg_map":{"version":214,"stamp":"2023-11-26T07:22:41.757732+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592778,"num_objects":23,"num_object_clones":0,"num_object_copies":46,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":23,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":203,"num_write_kb":2430,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":155,"ondisk_log_size":155,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":177,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":249132,"kb_used_data":7976,"kb_used_omap":12,"kb_used_meta":241011,"kb_avail":754725588,"statfs":{"total":773094113280,"available":772839002112,"internally_reserved":0,"allocated":8167424,"data_stored":4099788,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12749,"internal_metadata":246795827},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":40,"apply_latency_ms":40,"commit_latency_ns":40000000,"apply_latency_ns":40000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2410,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"5.921180"},"pg_stats":[{"pgid":"11.3f","version":"0'0","reported_seq":11,"reported_epoch":71,"state":"active+clean","last_fresh":"2023-11-26T07:22:39.009090+0000","last_change":"2023-11-26T07:22:37.876990+0000","last_active":"2023-11-26T07:22:39.009090+0000","last_peered":"2023-11-26T07:22:39.009090+0000","last_clean":"2023-11-26T07:22:39.009090+0000","last_became_active":"2023-11-26T07:22:37.876609+0000","last_became_peered":"2023-11-26T07:22:37.876609+0000","last_unstale":"2023-11-26T07:22:39.009090+0000","last_undegraded":"2023-11-26T07:22:39.009090+0000","last_fullsized":"2023-11-26T07:22:39.009090+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:22:36.836338+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:22:36.836338+0000","last_clean_scrub_stamp":"2023-11-26T07:22:36.836338+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T11:59:24.349605+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"11.3e","version":"0'0","reported_seq":11,"reported_epoch":71,"state":"active+clean","last_fresh":"2023-11-26T07:22:40.017080+0000","last_change":"2023-11-26T07:22:37.875094+0000","last_active":"2023-11-26T07:22:40.017080+0000","last_peered":"2023-11-26T07:22:40.017080+0000","last_clean":"2023-11-26T07:22:40.017080+0000","last_became_active":"2023-11-26T07:22:37.874754+0000","last_became_peered":"2023-11-26T07:22:37.874754+0000","last_unstale":"2023-11-26T07:22:40.017080+0000","last_undegraded":"2023-11-26T07:22:40.017080+0000","last_fullsized":"2023-11-26T07:22:40.017080+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:22:36.836338+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:22:36.836338+0000","last_clean_scrub_stamp":"2023-11-26T07:22:36.836338+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T08:37:52.234781+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.3d","version":"0'0","reported_seq":11,"reported_epoch":71,"state":"active+clean","last_fresh":"2023-11-26T07:22:40.017971+0000","last_change":"2023-11-26T07:22:37.857076+0000","last_active":"2023-11-26T07:22:40.017971+0000","last_peered":"2023-11-26T07:22:40.017971+0000","last_clean":"2023-11-26T07:22:40.017971+0000","last_became_active":"2023-11-26T07:22:37.856806+0000","last_became_peered":"2023-11-26T07:22:37.856806+0000","last_unstale":"2023-11-26T07:22:40.017971+0000","last_undegraded":"2023-11-26T07:22:40.017971+0000","last_fullsized":"2023-11-26T07:22:40.017971+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:22:36.836338+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:22:36.836338+0000","last_clean_scrub_stamp":"2023-11-26T07:22:36.836338+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T19:09:21.623652+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"11.3c","version":"0'0","reported_seq":11,"reported_epoch":71,"state":"active+clean","last_fresh":"2023-11-26T07:22:38.862976+0000","last_change":"2023-11-26T07:22:37.881413+0000","last_active":"2023-11-26T07:22:38.862976+0000","last_peered":"2023-11-26T07:22:38.862976+0000","last_clean":"2023-11-26T07:22:38.862976+0000","last_became_active":"2023-11-26T07:22:37.880403+0000","last_became_peered":"2023-11-26T07:22:37.880403+0000","last_unstale":"2023-11-26T07:22:38.862976+0000","last_undegraded":"2023-11-26T07:22:38.862976+0000","last_fullsized":"2023-11-26T07:22:38.862976+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:22:36.836338+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:22:36.836338+0000","last_clean_scrub_stamp":"2023-11-26T07:22:36.836338+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T17:20:27.435514+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.3b","version":"0'0","reported_seq":11,"reported_epoch":71,"state":"active+clean","last_fresh":"2023-11-26T07:22:38.861471+0000","last_change":"2023-11-26T07:22:37.869512+0000","last_active":"2023-11-26T07:22:38.861471+0000","last_peered":"2023-11-26T07:22:38.861471+0000","last_clean":"2023-11-26T07:22:38.861471+0000","last_became_active":"2023-11-26T07:22:37.869236+0000","last_became_peered":"2023-11-26T07:22:37.869236+0000","last_unstale":"2023-11-26T07:22:38.861471+0000","last_undegraded":"2023-11-26T07:22:38.861471+0000","last_fullsized":"2023-11-26T07:22:38.861471+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:22:36.836338+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:22:36.836338+0000","last_clean_scrub_stamp":"2023-11-26T07:22:36.836338+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T13:03:21.054144+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"11.3a","version":"0'0","reported_seq":11,"reported_epoch":71,"state":"active+clean","last_fresh":"2023-11-26T07:22:40.016528+0000","last_change":"2023-11-26T07:22:37.867618+0000","last_active":"2023-11-26T07:22:40.016528+0000","last_peered":"2023-11-26T07:22:40.016528+0000","last_clean":"2023-11-26T07:22:40.016528+0000","last_became_active":"2023-11-26T07:22:37.867007+0000","last_became_peered":"2023-11-26T07:22:37.867007+0000","last_unstale":"2023-11-26T07:22:40.016528+0000","last_undegraded":"2023-11-26T07:22:40.016528+0000","last_fullsized":"2023-11-26T07:22:40.016528+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:22:36.836338+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:22:36.836338+0000","last_clean_scrub_stamp":"2023-11-26T07:22:36.836338+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T08:05:50.197932+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.39","version":"0'0","reported_seq":11,"reported_epoch":71,"state":"active+clean","last_fresh":"2023-11-26T07:22:39.479830+0000","last_change":"2023-11-26T07:22:37.868725+0000","last_active":"2023-11-26T07:22:39.479830+0000","last_peered":"2023-11-26T07:22:39.479830+0000","last_clean":"2023-11-26T07:22:39.479830+0000","last_became_active":"2023-11-26T07:22:37.868369+0000","last_became_peered":"2023-11-26T07:22:37.868369+0000","last_unstale":"2023-11-26T07:22:39.479830+0000","last_undegraded":"2023-11-26T07:22:39.479830+0000","last_fullsized":"2023-11-26T07:22:39.479830+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:22:36.836338+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:22:36.836338+0000","last_clean_scrub_stamp":"2023-11-26T07:22:36.836338+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T12:02:44.081433+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"11.38","version":"0'0","reported_seq":11,"reported_epoch":71,"state":"active+clean","last_fresh":"2023-11-26T07:22:40.016667+0000","last_change":"2023-11-26T07:22:37.871042+0000","last_active":"2023-11-26T07:22:40.016667+0000","last_peered":"2023-11-26T07:22:40.016667+0000","last_clean":"2023-11-26T07:22:40.016667+0000","last_became_active":"2023-11-26T07:22:37.869494+0000","last_became_peered":"2023-11-26T07:22:37.869494+0000","last_unstale":"2023-11-26T07:22:40.016667+0000","last_undegraded":"2023-11-26T07:22:40.016667+0000","last_fullsized":"2023-11-26T07:22:40.016667+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:22:36.836338+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:22:36.836338+0000","last_clean_scrub_stamp":"2023-11-26T07:22:36.836338+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T07:55:11.708496+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.37","version":"0'0","reported_seq":11,"reported_epoch":71,"state":"active+clean","last_fresh":"2023-11-26T07:22:38.861863+0000","last_change":"2023-11-26T07:22:37.863093+0000","last_active":"2023-11-26T07:22:38.861863+0000","last_peered":"2023-11-26T07:22:38.861863+0000","last_clean":"2023-11-26T07:22:38.861863+0000","last_became_active":"2023-11-26T07:22:37.862823+0000","last_became_peered":"2023-11-26T07:22:37.862823+0000","last_unstale":"2023-11-26T07:22:38.861863+0000","last_undegraded":"2023-11-26T07:22:38.861863+0000","last_fullsized":"2023-11-26T07:22:38.861863+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:22:36.836338+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:22:36.836338+0000","last_clean_scrub_stamp":"2023-11-26T07:22:36.836338+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T13:38:50.756292+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"11.36","version":"0'0","reported_seq":11,"reported_epoch":71,"state":"active+clean","last_fresh":"2023-11-26T07:22:39.658097+0000","last_change":"2023-11-26T07:22:37.857561+0000","last_active":"2023-11-26T07:22:39.658097+0000","last_peered":"2023-11-26T07:22:39.658097+0000","last_clean":"2023-11-26T07:22:39.658097+0000","last_became_active":"2023-11-26T07:22:37.857181+0000","last_became_peered":"2023-11-26T07:22:37.857181+0000","last_unstale":"2023-11-26T07:22:39.658097+0000","last_undegraded":"2023-11-26T07:22:39.658097+0000","last_fullsized":"2023-11-26T07:22:39.658097+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:22:36.836338+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:22:36.836338+0000","last_clean_scrub_stamp":"2023-11-26T07:22:36.836338+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T11:09:49.677547+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.35","version":"0'0","reported_seq":11,"reported_epoch":71,"state":"active+clean","last_fresh":"2023-11-26T07:22:39.008999+0000","last_change":"2023-11-26T07:22:37.879093+0000","last_active":"2023-11-26T07:22:39.008999+0000","last_peered":"2023-11-26T07:22:39.008999+0000","last_clean":"2023-11-26T07:22:39.008999+0000","last_became_active":"2023-11-26T07:22:37.878330+0000","last_became_peered":"2023-11-26T07:22:37.878330+0000","last_unstale":"2023-11-26T07:22:39.008999+0000","last_undegraded":"2023-11-26T07:22:39.008999+0000","last_fullsized":"2023-11-26T07:22:39.008999+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:22:36.836338+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:22:36.836338+0000","last_clean_scrub_stamp":"2023-11-26T07:22:36.836338+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T14:21:24.831517+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"11.34","version":"0'0","reported_seq":11,"reported_epoch":71,"state":"active+clean","last_fresh":"2023-11-26T07:22:38.862846+0000","last_change":"2023-11-26T07:22:37.868734+0000","last_active":"2023-11-26T07:22:38.862846+0000","last_peered":"2023-11-26T07:22:38.862846+0000","last_clean":"2023-11-26T07:22:38.862846+0000","last_became_active":"2023-11-26T07:22:37.868461+0000","last_became_peered":"2023-11-26T07:22:37.868461+0000","last_unstale":"2023-11-26T07:22:38.862846+0000","last_undegraded":"2023-11-26T07:22:38.862846+0000","last_fullsized":"2023-11-26T07:22:38.862846+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:22:36.836338+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:22:36.836338+0000","last_clean_scrub_stamp":"2023-11-26T07:22:36.836338+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T09:40:51.476908+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.33","version":"0'0","reported_seq":11,"reported_epoch":71,"state":"active+clean","last_fresh":"2023-11-26T07:22:39.864177+0000","last_change":"2023-11-26T07:22:37.865247+0000","last_active":"2023-11-26T07:22:39.864177+0000","last_peered":"2023-11-26T07:22:39.864177+0000","last_clean":"2023-11-26T07:22:39.864177+0000","last_became_active":"2023-11-26T07:22:37.864308+0000","last_became_peered":"2023-11-26T07:22:37.864308+0000","last_unstale":"2023-11-26T07:22:39.864177+0000","last_undegraded":"2023-11-26T07:22:39.864177+0000","last_fullsized":"2023-11-26T07:22:39.864177+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:22:36.836338+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:22:36.836338+0000","last_clean_scrub_stamp":"2023-11-26T07:22:36.836338+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T17:06:11.853975+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.32","version":"0'0","reported_seq":11,"reported_epoch":71,"state":"active+clean","last_fresh":"2023-11-26T07:22:38.861454+0000","last_change":"2023-11-26T07:22:37.863027+0000","last_active":"2023-11-26T07:22:38.861454+0000","last_peered":"2023-11-26T07:22:38.861454+0000","last_clean":"2023-11-26T07:22:38.861454+0000","last_became_active":"2023-11-26T07:22:37.862707+0000","last_became_peered":"2023-11-26T07:22:37.862707+0000","last_unstale":"2023-11-26T07:22:38.861454+0000","last_undegraded":"2023-11-26T07:22:38.861454+0000","last_fullsized":"2023-11-26T07:22:38.861454+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:22:36.836338+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:22:36.836338+0000","last_clean_scrub_stamp":"2023-11-26T07:22:36.836338+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T07:39:30.457576+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"11.31","version":"0'0","reported_seq":11,"reported_epoch":71,"state":"active+clean","last_fresh":"2023-11-26T07:22:39.863999+0000","last_change":"2023-11-26T07:22:37.857331+0000","last_active":"2023-11-26T07:22:39.863999+0000","last_peered":"2023-11-26T07:22:39.863999+0000","last_clean":"2023-11-26T07:22:39.863999+0000","last_became_active":"2023-11-26T07:22:37.857080+0000","last_became_peered":"2023-11-26T07:22:37.857080+0000","last_unstale":"2023-11-26T07:22:39.863999+0000","last_undegraded":"2023-11-26T07:22:39.863999+0000","last_fullsized":"2023-11-26T07:22:39.863999+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:22:36.836338+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:22:36.836338+0000","last_clean_scrub_stamp":"2023-11-26T07:22:36.836338+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T10:28:26.783912+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.30","version":"0'0","reported_seq":11,"reported_epoch":71,"state":"active+clean","last_fresh":"2023-11-26T07:22:39.657680+0000","last_change":"2023-11-26T07:22:37.868553+0000","last_active":"2023-11-26T07:22:39.657680+0000","last_peered":"2023-11-26T07:22:39.657680+0000","last_clean":"2023-11-26T07:22:39.657680+0000","last_became_active":"2023-11-26T07:22:37.868283+0000","last_became_peered":"2023-11-26T07:22:37.868283+0000","last_unstale":"2023-11-26T07:22:39.657680+0000","last_undegraded":"2023-11-26T07:22:39.657680+0000","last_fullsized":"2023-11-26T07:22:39.657680+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:22:36.836338+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:22:36.836338+0000","last_clean_scrub_stamp":"2023-11-26T07:22:36.836338+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T13:57:20.044140+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.2f","version":"0'0","reported_seq":11,"reported_epoch":71,"state":"active+clean","last_fresh":"2023-11-26T07:22:38.862041+0000","last_change":"2023-11-26T07:22:37.862866+0000","last_active":"2023-11-26T07:22:38.862041+0000","last_peered":"2023-11-26T07:22:38.862041+0000","last_clean":"2023-11-26T07:22:38.862041+0000","last_became_active":"2023-11-26T07:22:37.862120+0000","last_became_peered":"2023-11-26T07:22:37.862120+0000","last_unstale":"2023-11-26T07:22:38.862041+0000","last_undegraded":"2023-11-26T07:22:38.862041+0000","last_fullsized":"2023-11-26T07:22:38.862041+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:22:36.836338+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:22:36.836338+0000","last_clean_scrub_stamp":"2023-11-26T07:22:36.836338+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T09:36:10.823109+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"11.2e","version":"0'0","reported_seq":11,"reported_epoch":71,"state":"active+clean","last_fresh":"2023-11-26T07:22:38.862990+0000","last_change":"2023-11-26T07:22:37.879070+0000","last_active":"2023-11-26T07:22:38.862990+0000","last_peered":"2023-11-26T07:22:38.862990+0000","last_clean":"2023-11-26T07:22:38.862990+0000","last_became_active":"2023-11-26T07:22:37.878806+0000","last_became_peered":"2023-11-26T07:22:37.878806+0000","last_unstale":"2023-11-26T07:22:38.862990+0000","last_undegraded":"2023-11-26T07:22:38.862990+0000","last_fullsized":"2023-11-26T07:22:38.862990+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:22:36.836338+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:22:36.836338+0000","last_clean_scrub_stamp":"2023-11-26T07:22:36.836338+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T16:40:17.858078+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.2d","version":"0'0","reported_seq":11,"reported_epoch":71,"state":"active+clean","last_fresh":"2023-11-26T07:22:40.016859+0000","last_change":"2023-11-26T07:22:37.868375+0000","last_active":"2023-11-26T07:22:40.016859+0000","last_peered":"2023-11-26T07:22:40.016859+0000","last_clean":"2023-11-26T07:22:40.016859+0000","last_became_active":"2023-11-26T07:22:37.868104+0000","last_became_peered":"2023-11-26T07:22:37.868104+0000","last_unstale":"2023-11-26T07:22:40.016859+0000","last_undegraded":"2023-11-26T07:22:40.016859+0000","last_fullsized":"2023-11-26T07:22:40.016859+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:22:36.836338+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:22:36.836338+0000","last_clean_scrub_stamp":"2023-11-26T07:22:36.836338+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T12:02:44.081433+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.2c","version":"0'0","reported_seq":11,"reported_epoch":71,"state":"active+clean","last_fresh":"2023-11-26T07:22:39.864201+0000","last_change":"2023-11-26T07:22:37.867875+0000","last_active":"2023-11-26T07:22:39.864201+0000","last_peered":"2023-11-26T07:22:39.864201+0000","last_clean":"2023-11-26T07:22:39.864201+0000","last_became_active":"2023-11-26T07:22:37.867362+0000","last_became_peered":"2023-11-26T07:22:37.867362+0000","last_unstale":"2023-11-26T07:22:39.864201+0000","last_undegraded":"2023-11-26T07:22:39.864201+0000","last_fullsized":"2023-11-26T07:22:39.864201+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:22:36.836338+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:22:36.836338+0000","last_clean_scrub_stamp":"2023-11-26T07:22:36.836338+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T14:29:02.649975+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.2b","version":"0'0","reported_seq":11,"reported_epoch":71,"state":"active+clean","last_fresh":"2023-11-26T07:22:39.657916+0000","last_change":"2023-11-26T07:22:37.853932+0000","last_active":"2023-11-26T07:22:39.657916+0000","last_peered":"2023-11-26T07:22:39.657916+0000","last_clean":"2023-11-26T07:22:39.657916+0000","last_became_active":"2023-11-26T07:22:37.853624+0000","last_became_peered":"2023-11-26T07:22:37.853624+0000","last_unstale":"2023-11-26T07:22:39.657916+0000","last_undegraded":"2023-11-26T07:22:39.657916+0000","last_fullsized":"2023-11-26T07:22:39.657916+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:22:36.836338+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:22:36.836338+0000","last_clean_scrub_stamp":"2023-11-26T07:22:36.836338+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T08:17:38.785700+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.2a","version":"0'0","reported_seq":11,"reported_epoch":71,"state":"active+clean","last_fresh":"2023-11-26T07:22:39.657855+0000","last_change":"2023-11-26T07:22:37.857910+0000","last_active":"2023-11-26T07:22:39.657855+0000","last_peered":"2023-11-26T07:22:39.657855+0000","last_clean":"2023-11-26T07:22:39.657855+0000","last_became_active":"2023-11-26T07:22:37.857615+0000","last_became_peered":"2023-11-26T07:22:37.857615+0000","last_unstale":"2023-11-26T07:22:39.657855+0000","last_undegraded":"2023-11-26T07:22:39.657855+0000","last_fullsized":"2023-11-26T07:22:39.657855+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:22:36.836338+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:22:36.836338+0000","last_clean_scrub_stamp":"2023-11-26T07:22:36.836338+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T10:25:42.181123+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.29","version":"0'0","reported_seq":11,"reported_epoch":71,"state":"active+clean","last_fresh":"2023-11-26T07:22:39.008659+0000","last_change":"2023-11-26T07:22:37.870429+0000","last_active":"2023-11-26T07:22:39.008659+0000","last_peered":"2023-11-26T07:22:39.008659+0000","last_clean":"2023-11-26T07:22:39.008659+0000","last_became_active":"2023-11-26T07:22:37.870047+0000","last_became_peered":"2023-11-26T07:22:37.870047+0000","last_unstale":"2023-11-26T07:22:39.008659+0000","last_undegraded":"2023-11-26T07:22:39.008659+0000","last_fullsized":"2023-11-26T07:22:39.008659+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:22:36.836338+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:22:36.836338+0000","last_clean_scrub_stamp":"2023-11-26T07:22:36.836338+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T17:06:11.853975+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"11.28","version":"0'0","reported_seq":11,"reported_epoch":71,"state":"active+clean","last_fresh":"2023-11-26T07:22:39.008836+0000","last_change":"2023-11-26T07:22:37.881994+0000","last_active":"2023-11-26T07:22:39.008836+0000","last_peered":"2023-11-26T07:22:39.008836+0000","last_clean":"2023-11-26T07:22:39.008836+0000","last_became_active":"2023-11-26T07:22:37.881669+0000","last_became_peered":"2023-11-26T07:22:37.881669+0000","last_unstale":"2023-11-26T07:22:39.008836+0000","last_undegraded":"2023-11-26T07:22:39.008836+0000","last_fullsized":"2023-11-26T07:22:39.008836+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:22:36.836338+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:22:36.836338+0000","last_clean_scrub_stamp":"2023-11-26T07:22:36.836338+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T15:34:25.091425+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"11.27","version":"0'0","reported_seq":11,"reported_epoch":71,"state":"active+clean","last_fresh":"2023-11-26T07:22:39.658257+0000","last_change":"2023-11-26T07:22:37.872517+0000","last_active":"2023-11-26T07:22:39.658257+0000","last_peered":"2023-11-26T07:22:39.658257+0000","last_clean":"2023-11-26T07:22:39.658257+0000","last_became_active":"2023-11-26T07:22:37.872228+0000","last_became_peered":"2023-11-26T07:22:37.872228+0000","last_unstale":"2023-11-26T07:22:39.658257+0000","last_undegraded":"2023-11-26T07:22:39.658257+0000","last_fullsized":"2023-11-26T07:22:39.658257+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:22:36.836338+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:22:36.836338+0000","last_clean_scrub_stamp":"2023-11-26T07:22:36.836338+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T16:05:00.044056+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.26","version":"0'0","reported_seq":11,"reported_epoch":71,"state":"active+clean","last_fresh":"2023-11-26T07:22:39.480219+0000","last_change":"2023-11-26T07:22:37.878497+0000","last_active":"2023-11-26T07:22:39.480219+0000","last_peered":"2023-11-26T07:22:39.480219+0000","last_clean":"2023-11-26T07:22:39.480219+0000","last_became_active":"2023-11-26T07:22:37.878209+0000","last_became_peered":"2023-11-26T07:22:37.878209+0000","last_unstale":"2023-11-26T07:22:39.480219+0000","last_undegraded":"2023-11-26T07:22:39.480219+0000","last_fullsized":"2023-11-26T07:22:39.480219+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:22:36.836338+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:22:36.836338+0000","last_clean_scrub_stamp":"2023-11-26T07:22:36.836338+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T13:34:41.619362+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"11.25","version":"0'0","reported_seq":11,"reported_epoch":71,"state":"active+clean","last_fresh":"2023-11-26T07:22:40.017975+0000","last_change":"2023-11-26T07:22:37.873393+0000","last_active":"2023-11-26T07:22:40.017975+0000","last_peered":"2023-11-26T07:22:40.017975+0000","last_clean":"2023-11-26T07:22:40.017975+0000","last_became_active":"2023-11-26T07:22:37.873080+0000","last_became_peered":"2023-11-26T07:22:37.873080+0000","last_unstale":"2023-11-26T07:22:40.017975+0000","last_undegraded":"2023-11-26T07:22:40.017975+0000","last_fullsized":"2023-11-26T07:22:40.017975+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:22:36.836338+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:22:36.836338+0000","last_clean_scrub_stamp":"2023-11-26T07:22:36.836338+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T17:33:46.796820+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"11.24","version":"0'0","reported_seq":11,"reported_epoch":71,"state":"active+clean","last_fresh":"2023-11-26T07:22:39.658115+0000","last_change":"2023-11-26T07:22:37.857135+0000","last_active":"2023-11-26T07:22:39.658115+0000","last_peered":"2023-11-26T07:22:39.658115+0000","last_clean":"2023-11-26T07:22:39.658115+0000","last_became_active":"2023-11-26T07:22:37.856836+0000","last_became_peered":"2023-11-26T07:22:37.856836+0000","last_unstale":"2023-11-26T07:22:39.658115+0000","last_undegraded":"2023-11-26T07:22:39.658115+0000","last_fullsized":"2023-11-26T07:22:39.658115+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:22:36.836338+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:22:36.836338+0000","last_clean_scrub_stamp":"2023-11-26T07:22:36.836338+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T09:24:31.774376+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.23","version":"0'0","reported_seq":11,"reported_epoch":71,"state":"active+clean","last_fresh":"2023-11-26T07:22:40.017775+0000","last_change":"2023-11-26T07:22:37.869946+0000","last_active":"2023-11-26T07:22:40.017775+0000","last_peered":"2023-11-26T07:22:40.017775+0000","last_clean":"2023-11-26T07:22:40.017775+0000","last_became_active":"2023-11-26T07:22:37.869701+0000","last_became_peered":"2023-11-26T07:22:37.869701+0000","last_unstale":"2023-11-26T07:22:40.017775+0000","last_undegraded":"2023-11-26T07:22:40.017775+0000","last_fullsized":"2023-11-26T07:22:40.017775+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:22:36.836338+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:22:36.836338+0000","last_clean_scrub_stamp":"2023-11-26T07:22:36.836338+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T13:15:12.583356+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"11.22","version":"0'0","reported_seq":11,"reported_epoch":71,"state":"active+clean","last_fresh":"2023-11-26T07:22:39.479919+0000","last_change":"2023-11-26T07:22:37.878295+0000","last_active":"2023-11-26T07:22:39.479919+0000","last_peered":"2023-11-26T07:22:39.479919+0000","last_clean":"2023-11-26T07:22:39.479919+0000","last_became_active":"2023-11-26T07:22:37.877997+0000","last_became_peered":"2023-11-26T07:22:37.877997+0000","last_unstale":"2023-11-26T07:22:39.479919+0000","last_undegraded":"2023-11-26T07:22:39.479919+0000","last_fullsized":"2023-11-26T07:22:39.479919+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:22:36.836338+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:22:36.836338+0000","last_clean_scrub_stamp":"2023-11-26T07:22:36.836338+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T08:44:15.104642+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"11.21","version":"0'0","reported_seq":11,"reported_epoch":71,"state":"active+clean","last_fresh":"2023-11-26T07:22:39.480040+0000","last_change":"2023-11-26T07:22:37.864587+0000","last_active":"2023-11-26T07:22:39.480040+0000","last_peered":"2023-11-26T07:22:39.480040+0000","last_clean":"2023-11-26T07:22:39.480040+0000","last_became_active":"2023-11-26T07:22:37.864015+0000","last_became_peered":"2023-11-26T07:22:37.864015+0000","last_unstale":"2023-11-26T07:22:39.480040+0000","last_undegraded":"2023-11-26T07:22:39.480040+0000","last_fullsized":"2023-11-26T07:22:39.480040+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:22:36.836338+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:22:36.836338+0000","last_clean_scrub_stamp":"2023-11-26T07:22:36.836338+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T14:52:44.124161+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"11.20","version":"0'0","reported_seq":11,"reported_epoch":71,"state":"active+clean","last_fresh":"2023-11-26T07:22:40.016651+0000","last_change":"2023-11-26T07:22:37.878621+0000","last_active":"2023-11-26T07:22:40.016651+0000","last_peered":"2023-11-26T07:22:40.016651+0000","last_clean":"2023-11-26T07:22:40.016651+0000","last_became_active":"2023-11-26T07:22:37.878299+0000","last_became_peered":"2023-11-26T07:22:37.878299+0000","last_unstale":"2023-11-26T07:22:40.016651+0000","last_undegraded":"2023-11-26T07:22:40.016651+0000","last_fullsized":"2023-11-26T07:22:40.016651+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:22:36.836338+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:22:36.836338+0000","last_clean_scrub_stamp":"2023-11-26T07:22:36.836338+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T09:24:31.774376+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.1e","version":"0'0","reported_seq":11,"reported_epoch":71,"state":"active+clean","last_fresh":"2023-11-26T07:22:39.009057+0000","last_change":"2023-11-26T07:22:37.880169+0000","last_active":"2023-11-26T07:22:39.009057+0000","last_peered":"2023-11-26T07:22:39.009057+0000","last_clean":"2023-11-26T07:22:39.009057+0000","last_became_active":"2023-11-26T07:22:37.878729+0000","last_became_peered":"2023-11-26T07:22:37.878729+0000","last_unstale":"2023-11-26T07:22:39.009057+0000","last_undegraded":"2023-11-26T07:22:39.009057+0000","last_fullsized":"2023-11-26T07:22:39.009057+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:22:36.836338+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:22:36.836338+0000","last_clean_scrub_stamp":"2023-11-26T07:22:36.836338+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T14:15:53.054036+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.1f","version":"70'4","reported_seq":16,"reported_epoch":70,"state":"active+clean","last_fresh":"2023-11-26T07:22:37.890541+0000","last_change":"2023-11-26T07:22:36.875269+0000","last_active":"2023-11-26T07:22:37.890541+0000","last_peered":"2023-11-26T07:22:37.890541+0000","last_clean":"2023-11-26T07:22:37.890541+0000","last_became_active":"2023-11-26T07:22:36.874786+0000","last_became_peered":"2023-11-26T07:22:36.874786+0000","last_unstale":"2023-11-26T07:22:37.890541+0000","last_undegraded":"2023-11-26T07:22:37.890541+0000","last_fullsized":"2023-11-26T07:22:37.890541+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:22:35.829702+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:22:35.829702+0000","last_clean_scrub_stamp":"2023-11-26T07:22:35.829702+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T18:23:53.779522+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"11.1f","version":"0'0","reported_seq":11,"reported_epoch":71,"state":"active+clean","last_fresh":"2023-11-26T07:22:38.862159+0000","last_change":"2023-11-26T07:22:37.863028+0000","last_active":"2023-11-26T07:22:38.862159+0000","last_peered":"2023-11-26T07:22:38.862159+0000","last_clean":"2023-11-26T07:22:38.862159+0000","last_became_active":"2023-11-26T07:22:37.862462+0000","last_became_peered":"2023-11-26T07:22:37.862462+0000","last_unstale":"2023-11-26T07:22:38.862159+0000","last_undegraded":"2023-11-26T07:22:38.862159+0000","last_fullsized":"2023-11-26T07:22:38.862159+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:22:36.836338+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:22:36.836338+0000","last_clean_scrub_stamp":"2023-11-26T07:22:36.836338+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T13:31:06.990149+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"10.1e","version":"70'1","reported_seq":13,"reported_epoch":70,"state":"active+clean","last_fresh":"2023-11-26T07:22:37.881672+0000","last_change":"2023-11-26T07:22:36.865454+0000","last_active":"2023-11-26T07:22:37.881672+0000","last_peered":"2023-11-26T07:22:37.881672+0000","last_clean":"2023-11-26T07:22:37.881672+0000","last_became_active":"2023-11-26T07:22:36.865055+0000","last_became_peered":"2023-11-26T07:22:36.865055+0000","last_unstale":"2023-11-26T07:22:37.881672+0000","last_undegraded":"2023-11-26T07:22:37.881672+0000","last_fullsized":"2023-11-26T07:22:37.881672+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:22:35.829702+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:22:35.829702+0000","last_clean_scrub_stamp":"2023-11-26T07:22:35.829702+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T11:49:26.867950+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.1c","version":"0'0","reported_seq":11,"reported_epoch":71,"state":"active+clean","last_fresh":"2023-11-26T07:22:40.016774+0000","last_change":"2023-11-26T07:22:37.879395+0000","last_active":"2023-11-26T07:22:40.016774+0000","last_peered":"2023-11-26T07:22:40.016774+0000","last_clean":"2023-11-26T07:22:40.016774+0000","last_became_active":"2023-11-26T07:22:37.879033+0000","last_became_peered":"2023-11-26T07:22:37.879033+0000","last_unstale":"2023-11-26T07:22:40.016774+0000","last_undegraded":"2023-11-26T07:22:40.016774+0000","last_fullsized":"2023-11-26T07:22:40.016774+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:22:36.836338+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:22:36.836338+0000","last_clean_scrub_stamp":"2023-11-26T07:22:36.836338+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T16:05:00.044056+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"10.1d","version":"70'2","reported_seq":14,"reported_epoch":70,"state":"active+clean","last_fresh":"2023-11-26T07:22:37.888967+0000","last_change":"2023-11-26T07:22:36.862194+0000","last_active":"2023-11-26T07:22:37.888967+0000","last_peered":"2023-11-26T07:22:37.888967+0000","last_clean":"2023-11-26T07:22:37.888967+0000","last_became_active":"2023-11-26T07:22:36.860756+0000","last_became_peered":"2023-11-26T07:22:36.860756+0000","last_unstale":"2023-11-26T07:22:37.888967+0000","last_undegraded":"2023-11-26T07:22:37.888967+0000","last_fullsized":"2023-11-26T07:22:37.888967+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:22:35.829702+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:22:35.829702+0000","last_clean_scrub_stamp":"2023-11-26T07:22:35.829702+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T16:57:10.472680+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.1d","version":"0'0","reported_seq":11,"reported_epoch":71,"state":"active+clean","last_fresh":"2023-11-26T07:22:39.009101+0000","last_change":"2023-11-26T07:22:37.874358+0000","last_active":"2023-11-26T07:22:39.009101+0000","last_peered":"2023-11-26T07:22:39.009101+0000","last_clean":"2023-11-26T07:22:39.009101+0000","last_became_active":"2023-11-26T07:22:37.874067+0000","last_became_peered":"2023-11-26T07:22:37.874067+0000","last_unstale":"2023-11-26T07:22:39.009101+0000","last_undegraded":"2023-11-26T07:22:39.009101+0000","last_fullsized":"2023-11-26T07:22:39.009101+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:22:36.836338+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:22:36.836338+0000","last_clean_scrub_stamp":"2023-11-26T07:22:36.836338+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T17:24:30.176302+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.1c","version":"70'2","reported_seq":14,"reported_epoch":70,"state":"active+clean","last_fresh":"2023-11-26T07:22:37.892791+0000","last_change":"2023-11-26T07:22:36.862999+0000","last_active":"2023-11-26T07:22:37.892791+0000","last_peered":"2023-11-26T07:22:37.892791+0000","last_clean":"2023-11-26T07:22:37.892791+0000","last_became_active":"2023-11-26T07:22:36.862710+0000","last_became_peered":"2023-11-26T07:22:36.862710+0000","last_unstale":"2023-11-26T07:22:37.892791+0000","last_undegraded":"2023-11-26T07:22:37.892791+0000","last_fullsized":"2023-11-26T07:22:37.892791+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:22:35.829702+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:22:35.829702+0000","last_clean_scrub_stamp":"2023-11-26T07:22:35.829702+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T16:57:10.472680+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.1a","version":"0'0","reported_seq":11,"reported_epoch":71,"state":"active+clean","last_fresh":"2023-11-26T07:22:39.480018+0000","last_change":"2023-11-26T07:22:37.874585+0000","last_active":"2023-11-26T07:22:39.480018+0000","last_peered":"2023-11-26T07:22:39.480018+0000","last_clean":"2023-11-26T07:22:39.480018+0000","last_became_active":"2023-11-26T07:22:37.873823+0000","last_became_peered":"2023-11-26T07:22:37.873823+0000","last_unstale":"2023-11-26T07:22:39.480018+0000","last_undegraded":"2023-11-26T07:22:39.480018+0000","last_fullsized":"2023-11-26T07:22:39.480018+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:22:36.836338+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:22:36.836338+0000","last_clean_scrub_stamp":"2023-11-26T07:22:36.836338+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T08:37:52.234781+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.1b","version":"70'1","reported_seq":13,"reported_epoch":70,"state":"active+clean","last_fresh":"2023-11-26T07:22:37.887545+0000","last_change":"2023-11-26T07:22:36.860364+0000","last_active":"2023-11-26T07:22:37.887545+0000","last_peered":"2023-11-26T07:22:37.887545+0000","last_clean":"2023-11-26T07:22:37.887545+0000","last_became_active":"2023-11-26T07:22:36.860002+0000","last_became_peered":"2023-11-26T07:22:36.860002+0000","last_unstale":"2023-11-26T07:22:37.887545+0000","last_undegraded":"2023-11-26T07:22:37.887545+0000","last_fullsized":"2023-11-26T07:22:37.887545+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:22:35.829702+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:22:35.829702+0000","last_clean_scrub_stamp":"2023-11-26T07:22:35.829702+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T09:11:42.010855+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"11.1b","version":"0'0","reported_seq":11,"reported_epoch":71,"state":"active+clean","last_fresh":"2023-11-26T07:22:39.658028+0000","last_change":"2023-11-26T07:22:37.874997+0000","last_active":"2023-11-26T07:22:39.658028+0000","last_peered":"2023-11-26T07:22:39.658028+0000","last_clean":"2023-11-26T07:22:39.658028+0000","last_became_active":"2023-11-26T07:22:37.874544+0000","last_became_peered":"2023-11-26T07:22:37.874544+0000","last_unstale":"2023-11-26T07:22:39.658028+0000","last_undegraded":"2023-11-26T07:22:39.658028+0000","last_fullsized":"2023-11-26T07:22:39.658028+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:22:36.836338+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:22:36.836338+0000","last_clean_scrub_stamp":"2023-11-26T07:22:36.836338+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T08:05:50.197932+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"10.1a","version":"70'1","reported_seq":13,"reported_epoch":70,"state":"active+clean","last_fresh":"2023-11-26T07:22:37.892156+0000","last_change":"2023-11-26T07:22:36.866439+0000","last_active":"2023-11-26T07:22:37.892156+0000","last_peered":"2023-11-26T07:22:37.892156+0000","last_clean":"2023-11-26T07:22:37.892156+0000","last_became_active":"2023-11-26T07:22:36.865935+0000","last_became_peered":"2023-11-26T07:22:36.865935+0000","last_unstale":"2023-11-26T07:22:37.892156+0000","last_undegraded":"2023-11-26T07:22:37.892156+0000","last_fullsized":"2023-11-26T07:22:37.892156+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:22:35.829702+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:22:35.829702+0000","last_clean_scrub_stamp":"2023-11-26T07:22:35.829702+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T17:20:26.428878+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.18","version":"0'0","reported_seq":11,"reported_epoch":71,"state":"active+clean","last_fresh":"2023-11-26T07:22:38.861163+0000","last_change":"2023-11-26T07:22:37.865874+0000","last_active":"2023-11-26T07:22:38.861163+0000","last_peered":"2023-11-26T07:22:38.861163+0000","last_clean":"2023-11-26T07:22:38.861163+0000","last_became_active":"2023-11-26T07:22:37.865490+0000","last_became_peered":"2023-11-26T07:22:37.865490+0000","last_unstale":"2023-11-26T07:22:38.861163+0000","last_undegraded":"2023-11-26T07:22:38.861163+0000","last_fullsized":"2023-11-26T07:22:38.861163+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:22:36.836338+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:22:36.836338+0000","last_clean_scrub_stamp":"2023-11-26T07:22:36.836338+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T15:21:22.997055+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"10.19","version":"0'0","reported_seq":12,"reported_epoch":70,"state":"active+clean","last_fresh":"2023-11-26T07:22:37.861366+0000","last_change":"2023-11-26T07:22:36.861455+0000","last_active":"2023-11-26T07:22:37.861366+0000","last_peered":"2023-11-26T07:22:37.861366+0000","last_clean":"2023-11-26T07:22:37.861366+0000","last_became_active":"2023-11-26T07:22:36.861128+0000","last_became_peered":"2023-11-26T07:22:36.861128+0000","last_unstale":"2023-11-26T07:22:37.861366+0000","last_undegraded":"2023-11-26T07:22:37.861366+0000","last_fullsized":"2023-11-26T07:22:37.861366+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:22:35.829702+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:22:35.829702+0000","last_clean_scrub_stamp":"2023-11-26T07:22:35.829702+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T14:55:45.237720+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.19","version":"0'0","reported_seq":11,"reported_epoch":71,"state":"active+clean","last_fresh":"2023-11-26T07:22:39.864073+0000","last_change":"2023-11-26T07:22:37.873372+0000","last_active":"2023-11-26T07:22:39.864073+0000","last_peered":"2023-11-26T07:22:39.864073+0000","last_clean":"2023-11-26T07:22:39.864073+0000","last_became_active":"2023-11-26T07:22:37.873033+0000","last_became_peered":"2023-11-26T07:22:37.873033+0000","last_unstale":"2023-11-26T07:22:39.864073+0000","last_undegraded":"2023-11-26T07:22:39.864073+0000","last_fullsized":"2023-11-26T07:22:39.864073+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:22:36.836338+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:22:36.836338+0000","last_clean_scrub_stamp":"2023-11-26T07:22:36.836338+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T14:34:41.099035+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"10.18","version":"0'0","reported_seq":12,"reported_epoch":70,"state":"active+clean","last_fresh":"2023-11-26T07:22:37.855379+0000","last_change":"2023-11-26T07:22:36.859199+0000","last_active":"2023-11-26T07:22:37.855379+0000","last_peered":"2023-11-26T07:22:37.855379+0000","last_clean":"2023-11-26T07:22:37.855379+0000","last_became_active":"2023-11-26T07:22:36.858925+0000","last_became_peered":"2023-11-26T07:22:36.858925+0000","last_unstale":"2023-11-26T07:22:37.855379+0000","last_undegraded":"2023-11-26T07:22:37.855379+0000","last_fullsized":"2023-11-26T07:22:37.855379+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:22:35.829702+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:22:35.829702+0000","last_clean_scrub_stamp":"2023-11-26T07:22:35.829702+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T14:22:02.210852+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"11.16","version":"0'0","reported_seq":11,"reported_epoch":71,"state":"active+clean","last_fresh":"2023-11-26T07:22:39.480277+0000","last_change":"2023-11-26T07:22:37.873279+0000","last_active":"2023-11-26T07:22:39.480277+0000","last_peered":"2023-11-26T07:22:39.480277+0000","last_clean":"2023-11-26T07:22:39.480277+0000","last_became_active":"2023-11-26T07:22:37.872909+0000","last_became_peered":"2023-11-26T07:22:37.872909+0000","last_unstale":"2023-11-26T07:22:39.480277+0000","last_undegraded":"2023-11-26T07:22:39.480277+0000","last_fullsized":"2023-11-26T07:22:39.480277+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:22:36.836338+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:22:36.836338+0000","last_clean_scrub_stamp":"2023-11-26T07:22:36.836338+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T10:53:10.511974+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.17","version":"0'0","reported_seq":12,"reported_epoch":70,"state":"active+clean","last_fresh":"2023-11-26T07:22:37.850457+0000","last_change":"2023-11-26T07:22:36.854554+0000","last_active":"2023-11-26T07:22:37.850457+0000","last_peered":"2023-11-26T07:22:37.850457+0000","last_clean":"2023-11-26T07:22:37.850457+0000","last_became_active":"2023-11-26T07:22:36.854192+0000","last_became_peered":"2023-11-26T07:22:36.854192+0000","last_unstale":"2023-11-26T07:22:37.850457+0000","last_undegraded":"2023-11-26T07:22:37.850457+0000","last_fullsized":"2023-11-26T07:22:37.850457+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:22:35.829702+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:22:35.829702+0000","last_clean_scrub_stamp":"2023-11-26T07:22:35.829702+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T07:41:05.117550+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.17","version":"0'0","reported_seq":11,"reported_epoch":71,"state":"active+clean","last_fresh":"2023-11-26T07:22:39.009202+0000","last_change":"2023-11-26T07:22:37.879717+0000","last_active":"2023-11-26T07:22:39.009202+0000","last_peered":"2023-11-26T07:22:39.009202+0000","last_clean":"2023-11-26T07:22:39.009202+0000","last_became_active":"2023-11-26T07:22:37.879423+0000","last_became_peered":"2023-11-26T07:22:37.879423+0000","last_unstale":"2023-11-26T07:22:39.009202+0000","last_undegraded":"2023-11-26T07:22:39.009202+0000","last_fullsized":"2023-11-26T07:22:39.009202+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:22:36.836338+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:22:36.836338+0000","last_clean_scrub_stamp":"2023-11-26T07:22:36.836338+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T12:42:37.711199+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.16","version":"0'0","reported_seq":12,"reported_epoch":70,"state":"active+clean","last_fresh":"2023-11-26T07:22:37.867466+0000","last_change":"2023-11-26T07:22:36.867429+0000","last_active":"2023-11-26T07:22:37.867466+0000","last_peered":"2023-11-26T07:22:37.867466+0000","last_clean":"2023-11-26T07:22:37.867466+0000","last_became_active":"2023-11-26T07:22:36.866921+0000","last_became_peered":"2023-11-26T07:22:36.866921+0000","last_unstale":"2023-11-26T07:22:37.867466+0000","last_undegraded":"2023-11-26T07:22:37.867466+0000","last_fullsized":"2023-11-26T07:22:37.867466+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:22:35.829702+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:22:35.829702+0000","last_clean_scrub_stamp":"2023-11-26T07:22:35.829702+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T10:25:41.174487+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.14","version":"0'0","reported_seq":11,"reported_epoch":71,"state":"active+clean","last_fresh":"2023-11-26T07:22:40.016991+0000","last_change":"2023-11-26T07:22:37.866132+0000","last_active":"2023-11-26T07:22:40.016991+0000","last_peered":"2023-11-26T07:22:40.016991+0000","last_clean":"2023-11-26T07:22:40.016991+0000","last_became_active":"2023-11-26T07:22:37.865208+0000","last_became_peered":"2023-11-26T07:22:37.865208+0000","last_unstale":"2023-11-26T07:22:40.016991+0000","last_undegraded":"2023-11-26T07:22:40.016991+0000","last_fullsized":"2023-11-26T07:22:40.016991+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:22:36.836338+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:22:36.836338+0000","last_clean_scrub_stamp":"2023-11-26T07:22:36.836338+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T08:44:15.104642+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"10.15","version":"70'3","reported_seq":15,"reported_epoch":70,"state":"active+clean","last_fresh":"2023-11-26T07:22:37.891482+0000","last_change":"2023-11-26T07:22:36.867479+0000","last_active":"2023-11-26T07:22:37.891482+0000","last_peered":"2023-11-26T07:22:37.891482+0000","last_clean":"2023-11-26T07:22:37.891482+0000","last_became_active":"2023-11-26T07:22:36.867227+0000","last_became_peered":"2023-11-26T07:22:36.867227+0000","last_unstale":"2023-11-26T07:22:37.891482+0000","last_undegraded":"2023-11-26T07:22:37.891482+0000","last_fullsized":"2023-11-26T07:22:37.891482+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:22:35.829702+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:22:35.829702+0000","last_clean_scrub_stamp":"2023-11-26T07:22:35.829702+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T11:59:23.342969+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"11.15","version":"0'0","reported_seq":11,"reported_epoch":71,"state":"active+clean","last_fresh":"2023-11-26T07:22:38.861785+0000","last_change":"2023-11-26T07:22:37.869962+0000","last_active":"2023-11-26T07:22:38.861785+0000","last_peered":"2023-11-26T07:22:38.861785+0000","last_clean":"2023-11-26T07:22:38.861785+0000","last_became_active":"2023-11-26T07:22:37.869699+0000","last_became_peered":"2023-11-26T07:22:37.869699+0000","last_unstale":"2023-11-26T07:22:38.861785+0000","last_undegraded":"2023-11-26T07:22:38.861785+0000","last_fullsized":"2023-11-26T07:22:38.861785+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:22:36.836338+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:22:36.836338+0000","last_clean_scrub_stamp":"2023-11-26T07:22:36.836338+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T08:29:30.155972+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"10.14","version":"70'6","reported_seq":18,"reported_epoch":70,"state":"active+clean","last_fresh":"2023-11-26T07:22:37.894032+0000","last_change":"2023-11-26T07:22:36.864408+0000","last_active":"2023-11-26T07:22:37.894032+0000","last_peered":"2023-11-26T07:22:37.894032+0000","last_clean":"2023-11-26T07:22:37.894032+0000","last_became_active":"2023-11-26T07:22:36.864120+0000","last_became_peered":"2023-11-26T07:22:36.864120+0000","last_unstale":"2023-11-26T07:22:37.894032+0000","last_undegraded":"2023-11-26T07:22:37.894032+0000","last_fullsized":"2023-11-26T07:22:37.894032+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:22:35.829702+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:22:35.829702+0000","last_clean_scrub_stamp":"2023-11-26T07:22:35.829702+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T19:06:39.025778+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.12","version":"0'0","reported_seq":11,"reported_epoch":71,"state":"active+clean","last_fresh":"2023-11-26T07:22:39.008663+0000","last_change":"2023-11-26T07:22:37.868503+0000","last_active":"2023-11-26T07:22:39.008663+0000","last_peered":"2023-11-26T07:22:39.008663+0000","last_clean":"2023-11-26T07:22:39.008663+0000","last_became_active":"2023-11-26T07:22:37.868221+0000","last_became_peered":"2023-11-26T07:22:37.868221+0000","last_unstale":"2023-11-26T07:22:39.008663+0000","last_undegraded":"2023-11-26T07:22:39.008663+0000","last_fullsized":"2023-11-26T07:22:39.008663+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:22:36.836338+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:22:36.836338+0000","last_clean_scrub_stamp":"2023-11-26T07:22:36.836338+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T14:29:02.649975+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.13","version":"70'2","reported_seq":14,"reported_epoch":70,"state":"active+clean","last_fresh":"2023-11-26T07:22:37.891499+0000","last_change":"2023-11-26T07:22:36.871188+0000","last_active":"2023-11-26T07:22:37.891499+0000","last_peered":"2023-11-26T07:22:37.891499+0000","last_clean":"2023-11-26T07:22:37.891499+0000","last_became_active":"2023-11-26T07:22:36.870926+0000","last_became_peered":"2023-11-26T07:22:36.870926+0000","last_unstale":"2023-11-26T07:22:37.891499+0000","last_undegraded":"2023-11-26T07:22:37.891499+0000","last_fullsized":"2023-11-26T07:22:37.891499+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:22:35.829702+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:22:35.829702+0000","last_clean_scrub_stamp":"2023-11-26T07:22:35.829702+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T10:56:59.018095+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.13","version":"0'0","reported_seq":11,"reported_epoch":71,"state":"active+clean","last_fresh":"2023-11-26T07:22:39.480015+0000","last_change":"2023-11-26T07:22:37.871038+0000","last_active":"2023-11-26T07:22:39.480015+0000","last_peered":"2023-11-26T07:22:39.480015+0000","last_clean":"2023-11-26T07:22:39.480015+0000","last_became_active":"2023-11-26T07:22:37.870441+0000","last_became_peered":"2023-11-26T07:22:37.870441+0000","last_unstale":"2023-11-26T07:22:39.480015+0000","last_undegraded":"2023-11-26T07:22:39.480015+0000","last_fullsized":"2023-11-26T07:22:39.480015+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:22:36.836338+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:22:36.836338+0000","last_clean_scrub_stamp":"2023-11-26T07:22:36.836338+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T15:03:26.446628+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.12","version":"0'0","reported_seq":12,"reported_epoch":70,"state":"active+clean","last_fresh":"2023-11-26T07:22:37.865979+0000","last_change":"2023-11-26T07:22:36.866255+0000","last_active":"2023-11-26T07:22:37.865979+0000","last_peered":"2023-11-26T07:22:37.865979+0000","last_clean":"2023-11-26T07:22:37.865979+0000","last_became_active":"2023-11-26T07:22:36.865530+0000","last_became_peered":"2023-11-26T07:22:36.865530+0000","last_unstale":"2023-11-26T07:22:37.865979+0000","last_undegraded":"2023-11-26T07:22:37.865979+0000","last_fullsized":"2023-11-26T07:22:37.865979+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:22:35.829702+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:22:35.829702+0000","last_clean_scrub_stamp":"2023-11-26T07:22:35.829702+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T19:16:57.380642+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.10","version":"0'0","reported_seq":11,"reported_epoch":71,"state":"active+clean","last_fresh":"2023-11-26T07:22:39.864074+0000","last_change":"2023-11-26T07:22:37.870669+0000","last_active":"2023-11-26T07:22:39.864074+0000","last_peered":"2023-11-26T07:22:39.864074+0000","last_clean":"2023-11-26T07:22:39.864074+0000","last_became_active":"2023-11-26T07:22:37.870306+0000","last_became_peered":"2023-11-26T07:22:37.870306+0000","last_unstale":"2023-11-26T07:22:39.864074+0000","last_undegraded":"2023-11-26T07:22:39.864074+0000","last_fullsized":"2023-11-26T07:22:39.864074+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:22:36.836338+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:22:36.836338+0000","last_clean_scrub_stamp":"2023-11-26T07:22:36.836338+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T17:59:50.448468+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"10.11","version":"0'0","reported_seq":12,"reported_epoch":70,"state":"active+clean","last_fresh":"2023-11-26T07:22:37.854375+0000","last_change":"2023-11-26T07:22:36.866279+0000","last_active":"2023-11-26T07:22:37.854375+0000","last_peered":"2023-11-26T07:22:37.854375+0000","last_clean":"2023-11-26T07:22:37.854375+0000","last_became_active":"2023-11-26T07:22:36.865955+0000","last_became_peered":"2023-11-26T07:22:36.865955+0000","last_unstale":"2023-11-26T07:22:37.854375+0000","last_undegraded":"2023-11-26T07:22:37.854375+0000","last_fullsized":"2023-11-26T07:22:37.854375+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:22:35.829702+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:22:35.829702+0000","last_clean_scrub_stamp":"2023-11-26T07:22:35.829702+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T08:17:37.779064+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"11.11","version":"0'0","reported_seq":11,"reported_epoch":71,"state":"active+clean","last_fresh":"2023-11-26T07:22:38.861373+0000","last_change":"2023-11-26T07:22:37.863518+0000","last_active":"2023-11-26T07:22:38.861373+0000","last_peered":"2023-11-26T07:22:38.861373+0000","last_clean":"2023-11-26T07:22:38.861373+0000","last_became_active":"2023-11-26T07:22:37.863275+0000","last_became_peered":"2023-11-26T07:22:37.863275+0000","last_unstale":"2023-11-26T07:22:38.861373+0000","last_undegraded":"2023-11-26T07:22:38.861373+0000","last_fullsized":"2023-11-26T07:22:38.861373+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:22:36.836338+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:22:36.836338+0000","last_clean_scrub_stamp":"2023-11-26T07:22:36.836338+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T09:19:07.788102+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"10.10","version":"70'8","reported_seq":20,"reported_epoch":70,"state":"active+clean","last_fresh":"2023-11-26T07:22:37.891639+0000","last_change":"2023-11-26T07:22:36.869929+0000","last_active":"2023-11-26T07:22:37.891639+0000","last_peered":"2023-11-26T07:22:37.891639+0000","last_clean":"2023-11-26T07:22:37.891639+0000","last_became_active":"2023-11-26T07:22:36.869372+0000","last_became_peered":"2023-11-26T07:22:36.869372+0000","last_unstale":"2023-11-26T07:22:37.891639+0000","last_undegraded":"2023-11-26T07:22:37.891639+0000","last_fullsized":"2023-11-26T07:22:37.891639+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:22:35.829702+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:22:35.829702+0000","last_clean_scrub_stamp":"2023-11-26T07:22:35.829702+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T16:40:16.851442+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.e","version":"0'0","reported_seq":11,"reported_epoch":71,"state":"active+clean","last_fresh":"2023-11-26T07:22:39.658192+0000","last_change":"2023-11-26T07:22:37.868826+0000","last_active":"2023-11-26T07:22:39.658192+0000","last_peered":"2023-11-26T07:22:39.658192+0000","last_clean":"2023-11-26T07:22:39.658192+0000","last_became_active":"2023-11-26T07:22:37.867718+0000","last_became_peered":"2023-11-26T07:22:37.867718+0000","last_unstale":"2023-11-26T07:22:39.658192+0000","last_undegraded":"2023-11-26T07:22:39.658192+0000","last_fullsized":"2023-11-26T07:22:39.658192+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:22:36.836338+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:22:36.836338+0000","last_clean_scrub_stamp":"2023-11-26T07:22:36.836338+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T07:55:11.708496+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"10.f","version":"70'1","reported_seq":13,"reported_epoch":70,"state":"active+clean","last_fresh":"2023-11-26T07:22:37.892228+0000","last_change":"2023-11-26T07:22:36.868080+0000","last_active":"2023-11-26T07:22:37.892228+0000","last_peered":"2023-11-26T07:22:37.892228+0000","last_clean":"2023-11-26T07:22:37.892228+0000","last_became_active":"2023-11-26T07:22:36.865257+0000","last_became_peered":"2023-11-26T07:22:36.865257+0000","last_unstale":"2023-11-26T07:22:37.892228+0000","last_undegraded":"2023-11-26T07:22:37.892228+0000","last_fullsized":"2023-11-26T07:22:37.892228+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:22:35.829702+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:22:35.829702+0000","last_clean_scrub_stamp":"2023-11-26T07:22:35.829702+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T18:04:03.792313+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.f","version":"0'0","reported_seq":11,"reported_epoch":71,"state":"active+clean","last_fresh":"2023-11-26T07:22:39.009243+0000","last_change":"2023-11-26T07:22:37.871386+0000","last_active":"2023-11-26T07:22:39.009243+0000","last_peered":"2023-11-26T07:22:39.009243+0000","last_clean":"2023-11-26T07:22:39.009243+0000","last_became_active":"2023-11-26T07:22:37.871041+0000","last_became_peered":"2023-11-26T07:22:37.871041+0000","last_unstale":"2023-11-26T07:22:39.009243+0000","last_undegraded":"2023-11-26T07:22:39.009243+0000","last_fullsized":"2023-11-26T07:22:39.009243+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:22:36.836338+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:22:36.836338+0000","last_clean_scrub_stamp":"2023-11-26T07:22:36.836338+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T09:11:43.017491+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.e","version":"70'2","reported_seq":14,"reported_epoch":70,"state":"active+clean","last_fresh":"2023-11-26T07:22:37.891182+0000","last_change":"2023-11-26T07:22:36.866137+0000","last_active":"2023-11-26T07:22:37.891182+0000","last_peered":"2023-11-26T07:22:37.891182+0000","last_clean":"2023-11-26T07:22:37.891182+0000","last_became_active":"2023-11-26T07:22:36.865873+0000","last_became_peered":"2023-11-26T07:22:36.865873+0000","last_unstale":"2023-11-26T07:22:37.891182+0000","last_undegraded":"2023-11-26T07:22:37.891182+0000","last_fullsized":"2023-11-26T07:22:37.891182+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:22:35.829702+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:22:35.829702+0000","last_clean_scrub_stamp":"2023-11-26T07:22:35.829702+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T19:14:58.011264+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.c","version":"0'0","reported_seq":11,"reported_epoch":71,"state":"active+clean","last_fresh":"2023-11-26T07:22:38.862889+0000","last_change":"2023-11-26T07:22:37.865642+0000","last_active":"2023-11-26T07:22:38.862889+0000","last_peered":"2023-11-26T07:22:38.862889+0000","last_clean":"2023-11-26T07:22:38.862889+0000","last_became_active":"2023-11-26T07:22:37.865314+0000","last_became_peered":"2023-11-26T07:22:37.865314+0000","last_unstale":"2023-11-26T07:22:38.862889+0000","last_undegraded":"2023-11-26T07:22:38.862889+0000","last_fullsized":"2023-11-26T07:22:38.862889+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:22:36.836338+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:22:36.836338+0000","last_clean_scrub_stamp":"2023-11-26T07:22:36.836338+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T18:01:35.745684+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"10.d","version":"70'1","reported_seq":13,"reported_epoch":70,"state":"active+clean","last_fresh":"2023-11-26T07:22:37.892203+0000","last_change":"2023-11-26T07:22:36.861086+0000","last_active":"2023-11-26T07:22:37.892203+0000","last_peered":"2023-11-26T07:22:37.892203+0000","last_clean":"2023-11-26T07:22:37.892203+0000","last_became_active":"2023-11-26T07:22:36.860779+0000","last_became_peered":"2023-11-26T07:22:36.860779+0000","last_unstale":"2023-11-26T07:22:37.892203+0000","last_undegraded":"2023-11-26T07:22:37.892203+0000","last_fullsized":"2023-11-26T07:22:37.892203+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:22:35.829702+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:22:35.829702+0000","last_clean_scrub_stamp":"2023-11-26T07:22:35.829702+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T19:14:58.011264+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.d","version":"0'0","reported_seq":11,"reported_epoch":71,"state":"active+clean","last_fresh":"2023-11-26T07:22:38.862029+0000","last_change":"2023-11-26T07:22:37.865825+0000","last_active":"2023-11-26T07:22:38.862029+0000","last_peered":"2023-11-26T07:22:38.862029+0000","last_clean":"2023-11-26T07:22:38.862029+0000","last_became_active":"2023-11-26T07:22:37.865388+0000","last_became_peered":"2023-11-26T07:22:37.865388+0000","last_unstale":"2023-11-26T07:22:38.862029+0000","last_undegraded":"2023-11-26T07:22:38.862029+0000","last_fullsized":"2023-11-26T07:22:38.862029+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:22:36.836338+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:22:36.836338+0000","last_clean_scrub_stamp":"2023-11-26T07:22:36.836338+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T10:33:19.999582+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"10.c","version":"0'0","reported_seq":12,"reported_epoch":70,"state":"active+clean","last_fresh":"2023-11-26T07:22:37.855671+0000","last_change":"2023-11-26T07:22:36.875047+0000","last_active":"2023-11-26T07:22:37.855671+0000","last_peered":"2023-11-26T07:22:37.855671+0000","last_clean":"2023-11-26T07:22:37.855671+0000","last_became_active":"2023-11-26T07:22:36.874745+0000","last_became_peered":"2023-11-26T07:22:36.874745+0000","last_unstale":"2023-11-26T07:22:37.855671+0000","last_undegraded":"2023-11-26T07:22:37.855671+0000","last_fullsized":"2023-11-26T07:22:37.855671+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:22:35.829702+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:22:35.829702+0000","last_clean_scrub_stamp":"2023-11-26T07:22:35.829702+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T13:34:40.612725+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"11.b","version":"0'0","reported_seq":11,"reported_epoch":71,"state":"active+clean","last_fresh":"2023-11-26T07:22:40.017951+0000","last_change":"2023-11-26T07:22:37.871637+0000","last_active":"2023-11-26T07:22:40.017951+0000","last_peered":"2023-11-26T07:22:40.017951+0000","last_clean":"2023-11-26T07:22:40.017951+0000","last_became_active":"2023-11-26T07:22:37.871390+0000","last_became_peered":"2023-11-26T07:22:37.871390+0000","last_unstale":"2023-11-26T07:22:40.017951+0000","last_undegraded":"2023-11-26T07:22:40.017951+0000","last_fullsized":"2023-11-26T07:22:40.017951+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:22:36.836338+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:22:36.836338+0000","last_clean_scrub_stamp":"2023-11-26T07:22:36.836338+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T14:36:50.851660+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.a","version":"0'0","reported_seq":12,"reported_epoch":70,"state":"active+clean","last_fresh":"2023-11-26T07:22:37.866111+0000","last_change":"2023-11-26T07:22:36.862939+0000","last_active":"2023-11-26T07:22:37.866111+0000","last_peered":"2023-11-26T07:22:37.866111+0000","last_clean":"2023-11-26T07:22:37.866111+0000","last_became_active":"2023-11-26T07:22:36.862625+0000","last_became_peered":"2023-11-26T07:22:36.862625+0000","last_unstale":"2023-11-26T07:22:37.866111+0000","last_undegraded":"2023-11-26T07:22:37.866111+0000","last_fullsized":"2023-11-26T07:22:37.866111+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:22:35.829702+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:22:35.829702+0000","last_clean_scrub_stamp":"2023-11-26T07:22:35.829702+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T07:41:05.117550+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.8","version":"0'0","reported_seq":11,"reported_epoch":71,"state":"active+clean","last_fresh":"2023-11-26T07:22:40.017765+0000","last_change":"2023-11-26T07:22:37.872438+0000","last_active":"2023-11-26T07:22:40.017765+0000","last_peered":"2023-11-26T07:22:40.017765+0000","last_clean":"2023-11-26T07:22:40.017765+0000","last_became_active":"2023-11-26T07:22:37.872114+0000","last_became_peered":"2023-11-26T07:22:37.872114+0000","last_unstale":"2023-11-26T07:22:40.017765+0000","last_undegraded":"2023-11-26T07:22:40.017765+0000","last_fullsized":"2023-11-26T07:22:40.017765+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:22:36.836338+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:22:36.836338+0000","last_clean_scrub_stamp":"2023-11-26T07:22:36.836338+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T17:12:49.941456+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.9","version":"0'0","reported_seq":12,"reported_epoch":70,"state":"active+clean","last_fresh":"2023-11-26T07:22:37.855701+0000","last_change":"2023-11-26T07:22:36.865706+0000","last_active":"2023-11-26T07:22:37.855701+0000","last_peered":"2023-11-26T07:22:37.855701+0000","last_clean":"2023-11-26T07:22:37.855701+0000","last_became_active":"2023-11-26T07:22:36.865421+0000","last_became_peered":"2023-11-26T07:22:36.865421+0000","last_unstale":"2023-11-26T07:22:37.855701+0000","last_undegraded":"2023-11-26T07:22:37.855701+0000","last_fullsized":"2023-11-26T07:22:37.855701+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:22:35.829702+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:22:35.829702+0000","last_clean_scrub_stamp":"2023-11-26T07:22:35.829702+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T09:24:30.767740+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"11.9","version":"0'0","reported_seq":11,"reported_epoch":71,"state":"active+clean","last_fresh":"2023-11-26T07:22:40.017950+0000","last_change":"2023-11-26T07:22:37.879871+0000","last_active":"2023-11-26T07:22:40.017950+0000","last_peered":"2023-11-26T07:22:40.017950+0000","last_clean":"2023-11-26T07:22:40.017950+0000","last_became_active":"2023-11-26T07:22:37.879560+0000","last_became_peered":"2023-11-26T07:22:37.879560+0000","last_unstale":"2023-11-26T07:22:40.017950+0000","last_undegraded":"2023-11-26T07:22:40.017950+0000","last_fullsized":"2023-11-26T07:22:40.017950+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:22:36.836338+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:22:36.836338+0000","last_clean_scrub_stamp":"2023-11-26T07:22:36.836338+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T09:52:57.186530+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.8","version":"0'0","reported_seq":12,"reported_epoch":70,"state":"active+clean","last_fresh":"2023-11-26T07:22:37.848351+0000","last_change":"2023-11-26T07:22:36.863944+0000","last_active":"2023-11-26T07:22:37.848351+0000","last_peered":"2023-11-26T07:22:37.848351+0000","last_clean":"2023-11-26T07:22:37.848351+0000","last_became_active":"2023-11-26T07:22:36.863619+0000","last_became_peered":"2023-11-26T07:22:36.863619+0000","last_unstale":"2023-11-26T07:22:37.848351+0000","last_undegraded":"2023-11-26T07:22:37.848351+0000","last_fullsized":"2023-11-26T07:22:37.848351+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:22:35.829702+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:22:35.829702+0000","last_clean_scrub_stamp":"2023-11-26T07:22:35.829702+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T18:23:53.779522+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.6","version":"0'0","reported_seq":11,"reported_epoch":71,"state":"active+clean","last_fresh":"2023-11-26T07:22:39.864365+0000","last_change":"2023-11-26T07:22:37.858075+0000","last_active":"2023-11-26T07:22:39.864365+0000","last_peered":"2023-11-26T07:22:39.864365+0000","last_clean":"2023-11-26T07:22:39.864365+0000","last_became_active":"2023-11-26T07:22:37.857651+0000","last_became_peered":"2023-11-26T07:22:37.857651+0000","last_unstale":"2023-11-26T07:22:39.864365+0000","last_undegraded":"2023-11-26T07:22:39.864365+0000","last_fullsized":"2023-11-26T07:22:39.864365+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:22:36.836338+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:22:36.836338+0000","last_clean_scrub_stamp":"2023-11-26T07:22:36.836338+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T15:34:25.091425+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"10.7","version":"70'2","reported_seq":14,"reported_epoch":70,"state":"active+clean","last_fresh":"2023-11-26T07:22:37.892279+0000","last_change":"2023-11-26T07:22:36.865572+0000","last_active":"2023-11-26T07:22:37.892279+0000","last_peered":"2023-11-26T07:22:37.892279+0000","last_clean":"2023-11-26T07:22:37.892279+0000","last_became_active":"2023-11-26T07:22:36.864928+0000","last_became_peered":"2023-11-26T07:22:36.864928+0000","last_unstale":"2023-11-26T07:22:37.892279+0000","last_undegraded":"2023-11-26T07:22:37.892279+0000","last_fullsized":"2023-11-26T07:22:37.892279+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:22:35.829702+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:22:35.829702+0000","last_clean_scrub_stamp":"2023-11-26T07:22:35.829702+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T13:57:19.037504+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.7","version":"0'0","reported_seq":11,"reported_epoch":71,"state":"active+clean","last_fresh":"2023-11-26T07:22:39.480277+0000","last_change":"2023-11-26T07:22:37.871391+0000","last_active":"2023-11-26T07:22:39.480277+0000","last_peered":"2023-11-26T07:22:39.480277+0000","last_clean":"2023-11-26T07:22:39.480277+0000","last_became_active":"2023-11-26T07:22:37.869546+0000","last_became_peered":"2023-11-26T07:22:37.869546+0000","last_unstale":"2023-11-26T07:22:39.480277+0000","last_undegraded":"2023-11-26T07:22:39.480277+0000","last_fullsized":"2023-11-26T07:22:39.480277+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:22:36.836338+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:22:36.836338+0000","last_clean_scrub_stamp":"2023-11-26T07:22:36.836338+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T18:39:51.590102+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.6","version":"0'0","reported_seq":12,"reported_epoch":70,"state":"active+clean","last_fresh":"2023-11-26T07:22:37.860114+0000","last_change":"2023-11-26T07:22:36.866276+0000","last_active":"2023-11-26T07:22:37.860114+0000","last_peered":"2023-11-26T07:22:37.860114+0000","last_clean":"2023-11-26T07:22:37.860114+0000","last_became_active":"2023-11-26T07:22:36.865972+0000","last_became_peered":"2023-11-26T07:22:36.865972+0000","last_unstale":"2023-11-26T07:22:37.860114+0000","last_undegraded":"2023-11-26T07:22:37.860114+0000","last_fullsized":"2023-11-26T07:22:37.860114+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:22:35.829702+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:22:35.829702+0000","last_clean_scrub_stamp":"2023-11-26T07:22:35.829702+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T07:55:10.701860+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"11.4","version":"0'0","reported_seq":11,"reported_epoch":71,"state":"active+clean","last_fresh":"2023-11-26T07:22:39.009019+0000","last_change":"2023-11-26T07:22:37.881934+0000","last_active":"2023-11-26T07:22:39.009019+0000","last_peered":"2023-11-26T07:22:39.009019+0000","last_clean":"2023-11-26T07:22:39.009019+0000","last_became_active":"2023-11-26T07:22:37.879130+0000","last_became_peered":"2023-11-26T07:22:37.879130+0000","last_unstale":"2023-11-26T07:22:39.009019+0000","last_undegraded":"2023-11-26T07:22:39.009019+0000","last_fullsized":"2023-11-26T07:22:39.009019+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:22:36.836338+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:22:36.836338+0000","last_clean_scrub_stamp":"2023-11-26T07:22:36.836338+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T17:24:41.141823+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.5","version":"0'0","reported_seq":12,"reported_epoch":70,"state":"active+clean","last_fresh":"2023-11-26T07:22:37.855189+0000","last_change":"2023-11-26T07:22:36.862659+0000","last_active":"2023-11-26T07:22:37.855189+0000","last_peered":"2023-11-26T07:22:37.855189+0000","last_clean":"2023-11-26T07:22:37.855189+0000","last_became_active":"2023-11-26T07:22:36.862342+0000","last_became_peered":"2023-11-26T07:22:36.862342+0000","last_unstale":"2023-11-26T07:22:37.855189+0000","last_undegraded":"2023-11-26T07:22:37.855189+0000","last_fullsized":"2023-11-26T07:22:37.855189+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:22:35.829702+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:22:35.829702+0000","last_clean_scrub_stamp":"2023-11-26T07:22:35.829702+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T15:11:09.629347+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"11.5","version":"0'0","reported_seq":11,"reported_epoch":71,"state":"active+clean","last_fresh":"2023-11-26T07:22:39.009179+0000","last_change":"2023-11-26T07:22:37.877378+0000","last_active":"2023-11-26T07:22:39.009179+0000","last_peered":"2023-11-26T07:22:39.009179+0000","last_clean":"2023-11-26T07:22:39.009179+0000","last_became_active":"2023-11-26T07:22:37.877099+0000","last_became_peered":"2023-11-26T07:22:37.877099+0000","last_unstale":"2023-11-26T07:22:39.009179+0000","last_undegraded":"2023-11-26T07:22:39.009179+0000","last_fullsized":"2023-11-26T07:22:39.009179+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:22:36.836338+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:22:36.836338+0000","last_clean_scrub_stamp":"2023-11-26T07:22:36.836338+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T15:10:55.003398+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.4","version":"0'0","reported_seq":12,"reported_epoch":70,"state":"active+clean","last_fresh":"2023-11-26T07:22:37.848299+0000","last_change":"2023-11-26T07:22:36.861249+0000","last_active":"2023-11-26T07:22:37.848299+0000","last_peered":"2023-11-26T07:22:37.848299+0000","last_clean":"2023-11-26T07:22:37.848299+0000","last_became_active":"2023-11-26T07:22:36.860803+0000","last_became_peered":"2023-11-26T07:22:36.860803+0000","last_unstale":"2023-11-26T07:22:37.848299+0000","last_undegraded":"2023-11-26T07:22:37.848299+0000","last_fullsized":"2023-11-26T07:22:37.848299+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:22:35.829702+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:22:35.829702+0000","last_clean_scrub_stamp":"2023-11-26T07:22:35.829702+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T14:27:32.693731+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.2","version":"0'0","reported_seq":11,"reported_epoch":71,"state":"active+clean","last_fresh":"2023-11-26T07:22:38.861539+0000","last_change":"2023-11-26T07:22:37.862679+0000","last_active":"2023-11-26T07:22:38.861539+0000","last_peered":"2023-11-26T07:22:38.861539+0000","last_clean":"2023-11-26T07:22:38.861539+0000","last_became_active":"2023-11-26T07:22:37.862423+0000","last_became_peered":"2023-11-26T07:22:37.862423+0000","last_unstale":"2023-11-26T07:22:38.861539+0000","last_undegraded":"2023-11-26T07:22:38.861539+0000","last_fullsized":"2023-11-26T07:22:38.861539+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:22:36.836338+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:22:36.836338+0000","last_clean_scrub_stamp":"2023-11-26T07:22:36.836338+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T19:20:37.466959+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"10.3","version":"0'0","reported_seq":12,"reported_epoch":70,"state":"active+clean","last_fresh":"2023-11-26T07:22:37.854850+0000","last_change":"2023-11-26T07:22:36.870180+0000","last_active":"2023-11-26T07:22:37.854850+0000","last_peered":"2023-11-26T07:22:37.854850+0000","last_clean":"2023-11-26T07:22:37.854850+0000","last_became_active":"2023-11-26T07:22:36.869919+0000","last_became_peered":"2023-11-26T07:22:36.869919+0000","last_unstale":"2023-11-26T07:22:37.854850+0000","last_undegraded":"2023-11-26T07:22:37.854850+0000","last_fullsized":"2023-11-26T07:22:37.854850+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:22:35.829702+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:22:35.829702+0000","last_clean_scrub_stamp":"2023-11-26T07:22:35.829702+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T10:28:25.331030+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"11.3","version":"0'0","reported_seq":11,"reported_epoch":71,"state":"active+clean","last_fresh":"2023-11-26T07:22:40.016709+0000","last_change":"2023-11-26T07:22:37.878023+0000","last_active":"2023-11-26T07:22:40.016709+0000","last_peered":"2023-11-26T07:22:40.016709+0000","last_clean":"2023-11-26T07:22:40.016709+0000","last_became_active":"2023-11-26T07:22:37.877444+0000","last_became_peered":"2023-11-26T07:22:37.877444+0000","last_unstale":"2023-11-26T07:22:40.016709+0000","last_undegraded":"2023-11-26T07:22:40.016709+0000","last_fullsized":"2023-11-26T07:22:40.016709+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:22:36.836338+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:22:36.836338+0000","last_clean_scrub_stamp":"2023-11-26T07:22:36.836338+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T11:09:49.677547+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"10.2","version":"0'0","reported_seq":12,"reported_epoch":70,"state":"active+clean","last_fresh":"2023-11-26T07:22:37.860639+0000","last_change":"2023-11-26T07:22:36.865453+0000","last_active":"2023-11-26T07:22:37.860639+0000","last_peered":"2023-11-26T07:22:37.860639+0000","last_clean":"2023-11-26T07:22:37.860639+0000","last_became_active":"2023-11-26T07:22:36.864886+0000","last_became_peered":"2023-11-26T07:22:36.864886+0000","last_unstale":"2023-11-26T07:22:37.860639+0000","last_undegraded":"2023-11-26T07:22:37.860639+0000","last_fullsized":"2023-11-26T07:22:37.860639+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:22:35.829702+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:22:35.829702+0000","last_clean_scrub_stamp":"2023-11-26T07:22:35.829702+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T10:04:12.676909+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.0","version":"0'0","reported_seq":11,"reported_epoch":71,"state":"active+clean","last_fresh":"2023-11-26T07:22:38.862787+0000","last_change":"2023-11-26T07:22:37.874876+0000","last_active":"2023-11-26T07:22:38.862787+0000","last_peered":"2023-11-26T07:22:38.862787+0000","last_clean":"2023-11-26T07:22:38.862787+0000","last_became_active":"2023-11-26T07:22:37.874581+0000","last_became_peered":"2023-11-26T07:22:37.874581+0000","last_unstale":"2023-11-26T07:22:38.862787+0000","last_undegraded":"2023-11-26T07:22:38.862787+0000","last_fullsized":"2023-11-26T07:22:38.862787+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:22:36.836338+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:22:36.836338+0000","last_clean_scrub_stamp":"2023-11-26T07:22:36.836338+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T09:31:53.635850+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"10.1","version":"0'0","reported_seq":12,"reported_epoch":70,"state":"active+clean","last_fresh":"2023-11-26T07:22:37.849474+0000","last_change":"2023-11-26T07:22:36.869677+0000","last_active":"2023-11-26T07:22:37.849474+0000","last_peered":"2023-11-26T07:22:37.849474+0000","last_clean":"2023-11-26T07:22:37.849474+0000","last_became_active":"2023-11-26T07:22:36.869395+0000","last_became_peered":"2023-11-26T07:22:36.869395+0000","last_unstale":"2023-11-26T07:22:37.849474+0000","last_undegraded":"2023-11-26T07:22:37.849474+0000","last_fullsized":"2023-11-26T07:22:37.849474+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:22:35.829702+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:22:35.829702+0000","last_clean_scrub_stamp":"2023-11-26T07:22:35.829702+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T14:00:20.899765+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.1","version":"0'0","reported_seq":11,"reported_epoch":71,"state":"active+clean","last_fresh":"2023-11-26T07:22:40.016659+0000","last_change":"2023-11-26T07:22:37.877486+0000","last_active":"2023-11-26T07:22:40.016659+0000","last_peered":"2023-11-26T07:22:40.016659+0000","last_clean":"2023-11-26T07:22:40.016659+0000","last_became_active":"2023-11-26T07:22:37.876471+0000","last_became_peered":"2023-11-26T07:22:37.876471+0000","last_unstale":"2023-11-26T07:22:40.016659+0000","last_undegraded":"2023-11-26T07:22:40.016659+0000","last_fullsized":"2023-11-26T07:22:40.016659+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:22:36.836338+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:22:36.836338+0000","last_clean_scrub_stamp":"2023-11-26T07:22:36.836338+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T08:17:38.785700+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"10.0","version":"0'0","reported_seq":12,"reported_epoch":70,"state":"active+clean","last_fresh":"2023-11-26T07:22:37.865899+0000","last_change":"2023-11-26T07:22:36.862071+0000","last_active":"2023-11-26T07:22:37.865899+0000","last_peered":"2023-11-26T07:22:37.865899+0000","last_clean":"2023-11-26T07:22:37.865899+0000","last_became_active":"2023-11-26T07:22:36.861791+0000","last_became_peered":"2023-11-26T07:22:36.861791+0000","last_unstale":"2023-11-26T07:22:37.865899+0000","last_undegraded":"2023-11-26T07:22:37.865899+0000","last_fullsized":"2023-11-26T07:22:37.865899+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:22:35.829702+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:22:35.829702+0000","last_clean_scrub_stamp":"2023-11-26T07:22:35.829702+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T14:00:20.899765+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.a","version":"0'0","reported_seq":11,"reported_epoch":71,"state":"active+clean","last_fresh":"2023-11-26T07:22:40.017829+0000","last_change":"2023-11-26T07:22:37.879289+0000","last_active":"2023-11-26T07:22:40.017829+0000","last_peered":"2023-11-26T07:22:40.017829+0000","last_clean":"2023-11-26T07:22:40.017829+0000","last_became_active":"2023-11-26T07:22:37.879002+0000","last_became_peered":"2023-11-26T07:22:37.879002+0000","last_unstale":"2023-11-26T07:22:40.017829+0000","last_undegraded":"2023-11-26T07:22:40.017829+0000","last_fullsized":"2023-11-26T07:22:40.017829+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:22:36.836338+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:22:36.836338+0000","last_clean_scrub_stamp":"2023-11-26T07:22:36.836338+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T17:54:06.225209+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.b","version":"70'2","reported_seq":14,"reported_epoch":70,"state":"active+clean","last_fresh":"2023-11-26T07:22:37.892777+0000","last_change":"2023-11-26T07:22:36.868634+0000","last_active":"2023-11-26T07:22:37.892777+0000","last_peered":"2023-11-26T07:22:37.892777+0000","last_clean":"2023-11-26T07:22:37.892777+0000","last_became_active":"2023-11-26T07:22:36.867494+0000","last_became_peered":"2023-11-26T07:22:36.867494+0000","last_unstale":"2023-11-26T07:22:37.892777+0000","last_undegraded":"2023-11-26T07:22:37.892777+0000","last_fullsized":"2023-11-26T07:22:37.892777+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:22:35.829702+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:22:35.829702+0000","last_clean_scrub_stamp":"2023-11-26T07:22:35.829702+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T16:40:16.851442+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"1.0","version":"15'117","reported_seq":205,"reported_epoch":70,"state":"active+clean","last_fresh":"2023-11-26T07:22:37.855521+0000","last_change":"2023-11-26T07:17:43.827802+0000","last_active":"2023-11-26T07:22:37.855521+0000","last_peered":"2023-11-26T07:22:37.855521+0000","last_clean":"2023-11-26T07:22:37.855521+0000","last_became_active":"2023-11-26T07:17:43.827308+0000","last_became_peered":"2023-11-26T07:17:43.827308+0000","last_unstale":"2023-11-26T07:22:37.855521+0000","last_undegraded":"2023-11-26T07:22:37.855521+0000","last_fullsized":"2023-11-26T07:22:37.855521+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:17:42.160945+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:17:42.160945+0000","last_clean_scrub_stamp":"2023-11-26T07:17:42.160945+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T08:17:54.167366+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":11,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":10,"num_pg":32,"stat_sum":{"num_bytes":2410,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":38,"ondisk_log_size":38,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574910,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32620,"kb_used_data":1432,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339220,"statfs":{"total":96636764160,"available":96603361280,"internally_reserved":0,"allocated":1466368,"data_stored":955263,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":31914442},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Nov 26 07:21:51 2023","interfaces":[{"interface":"back","average":{"1min":0.76600000000000001,"5min":0.71899999999999997,"15min":0.71599999999999997},"min":{"1min":0.42999999999999999,"5min":0.42999999999999999,"15min":0.42999999999999999},"max":{"1min":0.95999999999999996,"5min":0.98199999999999998,"15min":0.98199999999999998},"last":0.72599999999999998},{"interface":"front","average":{"1min":0.76400000000000001,"5min":0.72799999999999998,"15min":0.71499999999999997},"min":{"1min":0.53300000000000003,"5min":0.52900000000000003,"15min":0.52900000000000003},"max":{"1min":0.98599999999999999,"5min":1.0980000000000001,"15min":1.0980000000000001},"last":0.75900000000000001}]},{"osd":1,"last update":"Sun Nov 26 07:21:51 2023","interfaces":[{"interface":"back","average":{"1min":0.70599999999999996,"5min":0.68700000000000006,"15min":0.68300000000000005},"min":{"1min":0.441,"5min":0.441,"15min":0.441},"max":{"1min":1.012,"5min":1.0669999999999999,"15min":1.0669999999999999},"last":0.61299999999999999},{"interface":"front","average":{"1min":0.73599999999999999,"5min":0.74399999999999999,"15min":0.80700000000000005},"min":{"1min":0.58099999999999996,"5min":0.47599999999999998,"15min":0.47599999999999998},"max":{"1min":0.94199999999999995,"5min":4.5309999999999997,"15min":4.5309999999999997},"last":0.80600000000000005}]},{"osd":2,"last update":"Sun Nov 26 07:21:51 2023","interfaces":[{"interface":"back","average":{"1min":0.73399999999999999,"5min":0.69299999999999995,"15min":0.68100000000000005},"min":{"1min":0.39700000000000002,"5min":0.39700000000000002,"15min":0.39700000000000002},"max":{"1min":1.0800000000000001,"5min":1.0800000000000001,"15min":1.0800000000000001},"last":0.67600000000000005},{"interface":"front","average":{"1min":0.76800000000000002,"5min":0.71299999999999997,"15min":0.71299999999999997},"min":{"1min":0.497,"5min":0.497,"15min":0.497},"max":{"1min":0.96299999999999997,"5min":1.1339999999999999,"15min":1.1339999999999999},"last":0.83099999999999996}]},{"osd":3,"last update":"Sun Nov 26 07:21:51 2023","interfaces":[{"interface":"back","average":{"1min":0.73799999999999999,"5min":0.71899999999999997,"15min":0.70399999999999996},"min":{"1min":0.53000000000000003,"5min":0.47799999999999998,"15min":0.47799999999999998},"max":{"1min":1.1240000000000001,"5min":1.1240000000000001,"15min":1.1240000000000001},"last":0.71299999999999997},{"interface":"front","average":{"1min":0.81599999999999995,"5min":0.748,"15min":0.73799999999999999},"min":{"1min":0.54600000000000004,"5min":0.433,"15min":0.433},"max":{"1min":1.036,"5min":1.4490000000000001,"15min":1.4490000000000001},"last":0.55500000000000005}]},{"osd":4,"last update":"Sun Nov 26 07:21:51 2023","interfaces":[{"interface":"back","average":{"1min":0.59999999999999998,"5min":0.61199999999999999,"15min":0.60399999999999998},"min":{"1min":0.47099999999999997,"5min":0.39400000000000002,"15min":0.39400000000000002},"max":{"1min":0.96199999999999997,"5min":0.96299999999999997,"15min":0.96299999999999997},"last":0.63400000000000001},{"interface":"front","average":{"1min":0.66000000000000003,"5min":0.66100000000000003,"15min":0.66900000000000004},"min":{"1min":0.44600000000000001,"5min":0.436,"15min":0.436},"max":{"1min":1.004,"5min":1.5509999999999999,"15min":1.5509999999999999},"last":0.58699999999999997}]},{"osd":5,"last update":"Sun Nov 26 07:21:51 2023","interfaces":[{"interface":"back","average":{"1min":0.69299999999999995,"5min":0.68200000000000005,"15min":0.67100000000000004},"min":{"1min":0.46300000000000002,"5min":0.443,"15min":0.443},"max":{"1min":0.93300000000000005,"5min":1.415,"15min":1.415},"last":0.66000000000000003},{"interface":"front","average":{"1min":0.69299999999999995,"5min":0.69399999999999995,"15min":0.70699999999999996},"min":{"1min":0.48699999999999999,"5min":0.48599999999999999,"15min":0.48599999999999999},"max":{"1min":1.0840000000000001,"5min":1.3859999999999999,"15min":1.3859999999999999},"last":0.69199999999999995}]},{"osd":6,"last update":"Sun Nov 26 07:21:51 2023","interfaces":[{"interface":"back","average":{"1min":0.79200000000000004,"5min":0.77800000000000002,"15min":0.77300000000000002},"min":{"1min":0.56399999999999995,"5min":0.51700000000000002,"15min":0.51700000000000002},"max":{"1min":1.123,"5min":1.3069999999999999,"15min":1.3069999999999999},"last":0.88400000000000001},{"interface":"front","average":{"1min":0.75600000000000001,"5min":0.76200000000000001,"15min":0.76800000000000002},"min":{"1min":0.58399999999999996,"5min":0.46899999999999997,"15min":0.46899999999999997},"max":{"1min":0.90800000000000003,"5min":1.6759999999999999,"15min":1.6759999999999999},"last":0.74099999999999999}]}]},{"osd":6,"up_from":13,"seq":55834574912,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32032,"kb_used_data":852,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339808,"statfs":{"total":96636764160,"available":96603963392,"internally_reserved":0,"allocated":872448,"data_stored":364371,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Nov 26 07:21:53 2023","interfaces":[{"interface":"back","average":{"1min":0.69799999999999995,"5min":0.69799999999999995,"15min":0.68799999999999994},"min":{"1min":0.48999999999999999,"5min":0.36699999999999999,"15min":0.36699999999999999},"max":{"1min":0.91800000000000004,"5min":1.7549999999999999,"15min":1.7549999999999999},"last":0.77600000000000002},{"interface":"front","average":{"1min":0.72899999999999998,"5min":0.81799999999999995,"15min":0.91100000000000003},"min":{"1min":0.46600000000000003,"5min":0.38300000000000001,"15min":0.38300000000000001},"max":{"1min":0.874,"5min":5.7489999999999997,"15min":5.7489999999999997},"last":0.58299999999999996}]},{"osd":1,"last update":"Sun Nov 26 07:21:53 2023","interfaces":[{"interface":"back","average":{"1min":0.74199999999999999,"5min":0.80300000000000005,"15min":0.877},"min":{"1min":0.48699999999999999,"5min":0.41199999999999998,"15min":0.41199999999999998},"max":{"1min":0.97799999999999998,"5min":5.4240000000000004,"15min":5.4240000000000004},"last":0.67800000000000005},{"interface":"front","average":{"1min":0.72999999999999998,"5min":0.79500000000000004,"15min":0.86299999999999999},"min":{"1min":0.52200000000000002,"5min":0.499,"15min":0.499},"max":{"1min":0.96499999999999997,"5min":5.6020000000000003,"15min":5.6020000000000003},"last":0.70999999999999996}]},{"osd":2,"last update":"Sun Nov 26 07:21:53 2023","interfaces":[{"interface":"back","average":{"1min":0.70199999999999996,"5min":0.68999999999999995,"15min":0.69399999999999995},"min":{"1min":0.499,"5min":0.45100000000000001,"15min":0.45100000000000001},"max":{"1min":0.93600000000000005,"5min":1.8520000000000001,"15min":1.8520000000000001},"last":0.74399999999999999},{"interface":"front","average":{"1min":0.752,"5min":0.73399999999999999,"15min":0.73399999999999999},"min":{"1min":0.54300000000000004,"5min":0.437,"15min":0.437},"max":{"1min":0.95199999999999996,"5min":1.9159999999999999,"15min":1.9159999999999999},"last":0.60899999999999999}]},{"osd":3,"last update":"Sun Nov 26 07:21:53 2023","interfaces":[{"interface":"back","average":{"1min":0.72699999999999998,"5min":0.80000000000000004,"15min":0.86799999999999999},"min":{"1min":0.52400000000000002,"5min":0.46899999999999997,"15min":0.46899999999999997},"max":{"1min":0.91500000000000004,"5min":5.4390000000000001,"15min":5.4390000000000001},"last":0.64200000000000002},{"interface":"front","average":{"1min":0.74299999999999999,"5min":0.81499999999999995,"15min":0.90100000000000002},"min":{"1min":0.52300000000000002,"5min":0.48399999999999999,"15min":0.48399999999999999},"max":{"1min":0.89800000000000002,"5min":5.5490000000000004,"15min":5.5490000000000004},"last":0.79900000000000004}]},{"osd":4,"last update":"Sun Nov 26 07:21:53 2023","interfaces":[{"interface":"back","average":{"1min":0.56399999999999995,"5min":0.626,"15min":0.63600000000000001},"min":{"1min":0.40600000000000003,"5min":0.371,"15min":0.371},"max":{"1min":0.82999999999999996,"5min":1.0129999999999999,"15min":1.0129999999999999},"last":0.83399999999999996},{"interface":"front","average":{"1min":0.58899999999999997,"5min":0.65300000000000002,"15min":0.66700000000000004},"min":{"1min":0.38600000000000001,"5min":0.38600000000000001,"15min":0.38600000000000001},"max":{"1min":0.86399999999999999,"5min":1.0349999999999999,"15min":1.0349999999999999},"last":0.81999999999999995}]},{"osd":5,"last update":"Sun Nov 26 07:21:53 2023","interfaces":[{"interface":"back","average":{"1min":0.62,"5min":0.63800000000000001,"15min":0.65200000000000002},"min":{"1min":0.40699999999999997,"5min":0.38200000000000001,"15min":0.38200000000000001},"max":{"1min":0.84499999999999997,"5min":0.873,"15min":0.873},"last":0.85499999999999998},{"interface":"front","average":{"1min":0.65700000000000003,"5min":0.69899999999999995,"15min":0.70799999999999996},"min":{"1min":0.44,"5min":0.436,"15min":0.436},"max":{"1min":0.92900000000000005,"5min":1.0960000000000001,"15min":1.0960000000000001},"last":0.86699999999999999}]},{"osd":7,"last update":"Sun Nov 26 07:21:53 2023","interfaces":[{"interface":"back","average":{"1min":0.67900000000000005,"5min":0.71999999999999997,"15min":0.73399999999999999},"min":{"1min":0.42599999999999999,"5min":0.42599999999999999,"15min":0.42599999999999999},"max":{"1min":0.93999999999999995,"5min":0.998,"15min":0.998},"last":0.76400000000000001},{"interface":"front","average":{"1min":0.64800000000000002,"5min":0.69799999999999995,"15min":0.70999999999999996},"min":{"1min":0.46400000000000002,"5min":0.46400000000000002,"15min":0.46400000000000002},"max":{"1min":0.85299999999999998,"5min":0.98799999999999999,"15min":0.98799999999999999},"last":0.88900000000000001}]}]},{"osd":5,"up_from":13,"seq":55834574910,"num_pgs":19,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27876,"kb_used_data":848,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343964,"statfs":{"total":96636764160,"available":96608219136,"internally_reserved":0,"allocated":868352,"data_stored":364369,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Nov 26 07:21:48 2023","interfaces":[{"interface":"back","average":{"1min":0.72199999999999998,"5min":0.68799999999999994,"15min":0.67300000000000004},"min":{"1min":0.56200000000000006,"5min":0.38100000000000001,"15min":0.38100000000000001},"max":{"1min":0.872,"5min":1.0289999999999999,"15min":1.0289999999999999},"last":0.82899999999999996},{"interface":"front","average":{"1min":0.71399999999999997,"5min":0.69599999999999995,"15min":0.67200000000000004},"min":{"1min":0.50600000000000001,"5min":0.438,"15min":0.438},"max":{"1min":1.028,"5min":1.05,"15min":1.05},"last":0.95199999999999996}]},{"osd":1,"last update":"Sun Nov 26 07:21:48 2023","interfaces":[{"interface":"back","average":{"1min":0.71199999999999997,"5min":0.69499999999999995,"15min":0.67700000000000005},"min":{"1min":0.45700000000000002,"5min":0.41599999999999998,"15min":0.41599999999999998},"max":{"1min":0.95399999999999996,"5min":1.0389999999999999,"15min":1.0389999999999999},"last":0.93300000000000005},{"interface":"front","average":{"1min":0.72199999999999998,"5min":0.70899999999999996,"15min":0.69299999999999995},"min":{"1min":0.53500000000000003,"5min":0.443,"15min":0.443},"max":{"1min":0.99299999999999999,"5min":1.006,"15min":1.006},"last":0.76800000000000002}]},{"osd":2,"last update":"Sun Nov 26 07:21:48 2023","interfaces":[{"interface":"back","average":{"1min":0.72199999999999998,"5min":0.69799999999999995,"15min":0.68200000000000005},"min":{"1min":0.51300000000000001,"5min":0.45900000000000002,"15min":0.45900000000000002},"max":{"1min":0.92900000000000005,"5min":0.95699999999999996,"15min":0.95699999999999996},"last":0.84599999999999997},{"interface":"front","average":{"1min":0.78700000000000003,"5min":0.72999999999999998,"15min":0.68999999999999995},"min":{"1min":0.55100000000000005,"5min":0.51200000000000001,"15min":0.51200000000000001},"max":{"1min":1.1379999999999999,"5min":1.1379999999999999,"15min":1.1379999999999999},"last":0.80400000000000005}]},{"osd":3,"last update":"Sun Nov 26 07:21:48 2023","interfaces":[{"interface":"back","average":{"1min":0.75700000000000001,"5min":0.73299999999999998,"15min":0.72399999999999998},"min":{"1min":0.44700000000000001,"5min":0.43099999999999999,"15min":0.43099999999999999},"max":{"1min":1.1739999999999999,"5min":1.1739999999999999,"15min":1.1739999999999999},"last":0.73599999999999999},{"interface":"front","average":{"1min":0.77800000000000002,"5min":0.72299999999999998,"15min":0.69099999999999995},"min":{"1min":0.55600000000000005,"5min":0.44600000000000001,"15min":0.44600000000000001},"max":{"1min":1.0029999999999999,"5min":1.0209999999999999,"15min":1.0209999999999999},"last":0.69799999999999995}]},{"osd":4,"last update":"Sun Nov 26 07:21:48 2023","interfaces":[{"interface":"back","average":{"1min":0.56799999999999995,"5min":0.59999999999999998,"15min":0.623},"min":{"1min":0.41199999999999998,"5min":0.35299999999999998,"15min":0.35299999999999998},"max":{"1min":0.92700000000000005,"5min":0.95099999999999996,"15min":0.95099999999999996},"last":0.504},{"interface":"front","average":{"1min":0.66200000000000003,"5min":0.67000000000000004,"15min":0.69599999999999995},"min":{"1min":0.45000000000000001,"5min":0.41299999999999998,"15min":0.41299999999999998},"max":{"1min":0.94699999999999995,"5min":1.254,"15min":1.254},"last":0.60699999999999998}]},{"osd":6,"last update":"Sun Nov 26 07:21:48 2023","interfaces":[{"interface":"back","average":{"1min":0.61399999999999999,"5min":0.66200000000000003,"15min":0.66800000000000004},"min":{"1min":0.44400000000000001,"5min":0.439,"15min":0.439},"max":{"1min":1.0880000000000001,"5min":1.0880000000000001,"15min":1.0880000000000001},"last":0.66300000000000003},{"interface":"front","average":{"1min":0.66600000000000004,"5min":0.70299999999999996,"15min":0.71899999999999997},"min":{"1min":0.47699999999999998,"5min":0.436,"15min":0.436},"max":{"1min":1.052,"5min":1.0649999999999999,"15min":1.0649999999999999},"last":0.63700000000000001}]},{"osd":7,"last update":"Sun Nov 26 07:21:48 2023","interfaces":[{"interface":"back","average":{"1min":0.73099999999999998,"5min":0.75900000000000001,"15min":0.78900000000000003},"min":{"1min":0.51800000000000002,"5min":0.48199999999999998,"15min":0.48199999999999998},"max":{"1min":1.2010000000000001,"5min":1.4079999999999999,"15min":1.4079999999999999},"last":0.873},{"interface":"front","average":{"1min":0.72899999999999998,"5min":0.751,"15min":0.76700000000000002},"min":{"1min":0.52400000000000002,"5min":0.45000000000000001,"15min":0.45000000000000001},"max":{"1min":1.115,"5min":1.115,"15min":1.115},"last":0.89400000000000002}]}]},{"osd":4,"up_from":13,"seq":55834574912,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32028,"kb_used_data":848,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339812,"statfs":{"total":96636764160,"available":96603967488,"internally_reserved":0,"allocated":868352,"data_stored":364349,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Nov 26 07:21:53 2023","interfaces":[{"interface":"back","average":{"1min":0.67300000000000004,"5min":0.70899999999999996,"15min":0.76500000000000001},"min":{"1min":0.46100000000000002,"5min":0.32100000000000001,"15min":0.32100000000000001},"max":{"1min":0.94499999999999995,"5min":4.1689999999999996,"15min":4.1689999999999996},"last":0.63300000000000001},{"interface":"front","average":{"1min":1.609,"5min":0.98299999999999998,"15min":0.995},"min":{"1min":0.52800000000000002,"5min":0.46200000000000002,"15min":0.46200000000000002},"max":{"1min":18.021000000000001,"5min":18.021000000000001,"15min":18.021000000000001},"last":0.81699999999999995}]},{"osd":1,"last update":"Sun Nov 26 07:21:53 2023","interfaces":[{"interface":"back","average":{"1min":0.90500000000000003,"5min":0.71199999999999997,"15min":0.66800000000000004},"min":{"1min":0.47499999999999998,"5min":0.43099999999999999,"15min":0.43099999999999999},"max":{"1min":4.3159999999999998,"5min":4.3159999999999998,"15min":4.3159999999999998},"last":0.70599999999999996},{"interface":"front","average":{"1min":1.2150000000000001,"5min":0.82799999999999996,"15min":0.82199999999999995},"min":{"1min":0.51900000000000002,"5min":0.39800000000000002,"15min":0.39800000000000002},"max":{"1min":11.206,"5min":11.206,"15min":11.206},"last":0.65400000000000003}]},{"osd":2,"last update":"Sun Nov 26 07:21:53 2023","interfaces":[{"interface":"back","average":{"1min":1.492,"5min":0.89800000000000002,"15min":0.83699999999999997},"min":{"1min":0.47199999999999998,"5min":0.433,"15min":0.433},"max":{"1min":15.441000000000001,"5min":15.441000000000001,"15min":15.441000000000001},"last":0.748},{"interface":"front","average":{"1min":1.2350000000000001,"5min":0.97399999999999998,"15min":1.0349999999999999},"min":{"1min":0.59399999999999997,"5min":0.53400000000000003,"15min":0.53400000000000003},"max":{"1min":9.4800000000000004,"5min":9.4800000000000004,"15min":9.4800000000000004},"last":0.76900000000000002}]},{"osd":3,"last update":"Sun Nov 26 07:21:53 2023","interfaces":[{"interface":"back","average":{"1min":1.3700000000000001,"5min":0.83599999999999997,"15min":0.76500000000000001},"min":{"1min":0.57199999999999995,"5min":0.441,"15min":0.441},"max":{"1min":12.715,"5min":12.715,"15min":12.715},"last":0.498},{"interface":"front","average":{"1min":1.145,"5min":0.87,"15min":0.876},"min":{"1min":0.50700000000000001,"5min":0.50700000000000001,"15min":0.50700000000000001},"max":{"1min":8.3239999999999998,"5min":8.3239999999999998,"15min":8.3239999999999998},"last":0.79000000000000004}]},{"osd":5,"last update":"Sun Nov 26 07:21:53 2023","interfaces":[{"interface":"back","average":{"1min":0.63800000000000001,"5min":0.61299999999999999,"15min":0.61099999999999999},"min":{"1min":0.47099999999999997,"5min":0.38200000000000001,"15min":0.38200000000000001},"max":{"1min":0.93999999999999995,"5min":0.96199999999999997,"15min":0.96199999999999997},"last":0.86299999999999999},{"interface":"front","average":{"1min":1.165,"5min":0.72799999999999998,"15min":0.67000000000000004},"min":{"1min":0.442,"5min":0.35599999999999998,"15min":0.35599999999999998},"max":{"1min":11.148999999999999,"5min":11.148999999999999,"15min":11.148999999999999},"last":0.80800000000000005}]},{"osd":6,"last update":"Sun Nov 26 07:21:53 2023","interfaces":[{"interface":"back","average":{"1min":0.624,"5min":0.67500000000000004,"15min":0.754},"min":{"1min":0.41699999999999998,"5min":0.372,"15min":0.372},"max":{"1min":0.81799999999999995,"5min":2.8330000000000002,"15min":2.8330000000000002},"last":0.54400000000000004},{"interface":"front","average":{"1min":0.79200000000000004,"5min":0.83299999999999996,"15min":0.94999999999999996},"min":{"1min":0.54000000000000004,"5min":0.41299999999999998,"15min":0.41299999999999998},"max":{"1min":2.472,"5min":7.6050000000000004,"15min":7.6050000000000004},"last":0.60499999999999998}]},{"osd":7,"last update":"Sun Nov 26 07:21:53 2023","interfaces":[{"interface":"back","average":{"1min":1.339,"5min":0.88100000000000001,"15min":0.86699999999999999},"min":{"1min":0.57999999999999996,"5min":0.44700000000000001,"15min":0.44700000000000001},"max":{"1min":12.653,"5min":12.653,"15min":12.653},"last":0.72999999999999998},{"interface":"front","average":{"1min":0.90100000000000002,"5min":0.82199999999999995,"15min":0.86799999999999999},"min":{"1min":0.47499999999999998,"5min":0.33300000000000002,"15min":0.33300000000000002},"max":{"1min":3.8319999999999999,"5min":3.8319999999999999,"15min":3.8319999999999999},"last":0.93700000000000006}]}]},{"osd":3,"up_from":13,"seq":55834574913,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32032,"kb_used_data":852,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339808,"statfs":{"total":96636764160,"available":96603963392,"internally_reserved":0,"allocated":872448,"data_stored":364391,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1592,"internal_metadata":31914440},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Nov 26 07:21:48 2023","interfaces":[{"interface":"back","average":{"1min":0.46000000000000002,"5min":0.41299999999999998,"15min":0.39000000000000001},"min":{"1min":0.25800000000000001,"5min":0.25800000000000001,"15min":0.25800000000000001},"max":{"1min":0.72799999999999998,"5min":0.72799999999999998,"15min":0.72799999999999998},"last":0.51600000000000001},{"interface":"front","average":{"1min":0.48599999999999999,"5min":0.45900000000000002,"15min":0.441},"min":{"1min":0.316,"5min":0.26900000000000002,"15min":0.26900000000000002},"max":{"1min":0.71299999999999997,"5min":0.77300000000000002,"15min":0.77300000000000002},"last":0.5}]},{"osd":1,"last update":"Sun Nov 26 07:21:48 2023","interfaces":[{"interface":"back","average":{"1min":0.443,"5min":0.438,"15min":0.42599999999999999},"min":{"1min":0.34100000000000003,"5min":0.28799999999999998,"15min":0.28799999999999998},"max":{"1min":0.59299999999999997,"5min":0.752,"15min":0.752},"last":0.42999999999999999},{"interface":"front","average":{"1min":0.503,"5min":0.498,"15min":0.48799999999999999},"min":{"1min":0.374,"5min":0.34599999999999997,"15min":0.34599999999999997},"max":{"1min":0.76200000000000001,"5min":0.872,"15min":0.872},"last":0.44500000000000001}]},{"osd":2,"last update":"Sun Nov 26 07:21:48 2023","interfaces":[{"interface":"back","average":{"1min":0.58399999999999996,"5min":0.56299999999999994,"15min":0.54300000000000004},"min":{"1min":0.376,"5min":0.376,"15min":0.376},"max":{"1min":0.97199999999999998,"5min":0.97199999999999998,"15min":0.97199999999999998},"last":0.53500000000000003},{"interface":"front","average":{"1min":0.56799999999999995,"5min":0.53700000000000003,"15min":0.51500000000000001},"min":{"1min":0.40899999999999997,"5min":0.33000000000000002,"15min":0.33000000000000002},"max":{"1min":0.81999999999999995,"5min":0.89700000000000002,"15min":0.89700000000000002},"last":0.48199999999999998}]},{"osd":4,"last update":"Sun Nov 26 07:21:48 2023","interfaces":[{"interface":"back","average":{"1min":0.71099999999999997,"5min":0.71799999999999997,"15min":0.70999999999999996},"min":{"1min":0.45800000000000002,"5min":0.45800000000000002,"15min":0.45800000000000002},"max":{"1min":0.93000000000000005,"5min":1.091,"15min":1.091},"last":0.879},{"interface":"front","average":{"1min":0.73999999999999999,"5min":0.73899999999999999,"15min":0.73099999999999998},"min":{"1min":0.39900000000000002,"5min":0.39900000000000002,"15min":0.39900000000000002},"max":{"1min":1.0289999999999999,"5min":1.242,"15min":1.242},"last":0.75700000000000001}]},{"osd":5,"last update":"Sun Nov 26 07:21:48 2023","interfaces":[{"interface":"back","average":{"1min":0.70999999999999996,"5min":0.73299999999999998,"15min":0.73699999999999999},"min":{"1min":0.47099999999999997,"5min":0.47099999999999997,"15min":0.47099999999999997},"max":{"1min":0.94699999999999995,"5min":1.105,"15min":1.105},"last":0.73999999999999999},{"interface":"front","average":{"1min":0.73499999999999999,"5min":0.75,"15min":0.74199999999999999},"min":{"1min":0.48799999999999999,"5min":0.48799999999999999,"15min":0.48799999999999999},"max":{"1min":0.97399999999999998,"5min":1.1319999999999999,"15min":1.1319999999999999},"last":0.77400000000000002}]},{"osd":6,"last update":"Sun Nov 26 07:21:48 2023","interfaces":[{"interface":"back","average":{"1min":0.76100000000000001,"5min":0.79400000000000004,"15min":0.80700000000000005},"min":{"1min":0.503,"5min":0.503,"15min":0.503},"max":{"1min":0.89100000000000001,"5min":1.2729999999999999,"15min":1.2729999999999999},"last":0.81699999999999995},{"interface":"front","average":{"1min":0.77500000000000002,"5min":0.77900000000000003,"15min":0.78800000000000003},"min":{"1min":0.51100000000000001,"5min":0.47299999999999998,"15min":0.47299999999999998},"max":{"1min":1.002,"5min":1.204,"15min":1.204},"last":0.82799999999999996}]},{"osd":7,"last update":"Sun Nov 26 07:21:48 2023","interfaces":[{"interface":"back","average":{"1min":0.78900000000000003,"5min":0.77700000000000002,"15min":0.79300000000000004},"min":{"1min":0.54500000000000004,"5min":0.5,"15min":0.5},"max":{"1min":0.999,"5min":1.3600000000000001,"15min":1.3600000000000001},"last":0.80600000000000005},{"interface":"front","average":{"1min":0.77600000000000002,"5min":0.81699999999999995,"15min":0.82299999999999995},"min":{"1min":0.47299999999999998,"5min":0.47299999999999998,"15min":0.47299999999999998},"max":{"1min":1.0349999999999999,"5min":1.617,"15min":1.617},"last":0.84999999999999998}]}]},{"osd":2,"up_from":13,"seq":55834574913,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27884,"kb_used_data":856,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343956,"statfs":{"total":96636764160,"available":96608210944,"internally_reserved":0,"allocated":876544,"data_stored":365891,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Nov 26 07:21:47 2023","interfaces":[{"interface":"back","average":{"1min":0.42299999999999999,"5min":0.55200000000000005,"15min":0.66000000000000003},"min":{"1min":0.29799999999999999,"5min":0.28699999999999998,"15min":0.28699999999999998},"max":{"1min":0.72899999999999998,"5min":4.7030000000000003,"15min":4.7030000000000003},"last":0.44700000000000001},{"interface":"front","average":{"1min":0.48299999999999998,"5min":0.51300000000000001,"15min":0.54500000000000004},"min":{"1min":0.27300000000000002,"5min":0.27300000000000002,"15min":0.27300000000000002},"max":{"1min":0.71899999999999997,"5min":2.4340000000000002,"15min":2.4340000000000002},"last":0.49299999999999999}]},{"osd":1,"last update":"Sun Nov 26 07:21:47 2023","interfaces":[{"interface":"back","average":{"1min":0.52400000000000002,"5min":0.64900000000000002,"15min":0.79200000000000004},"min":{"1min":0.36099999999999999,"5min":0.34599999999999997,"15min":0.34599999999999997},"max":{"1min":0.86099999999999999,"5min":4.2690000000000001,"15min":4.2690000000000001},"last":0.54600000000000004},{"interface":"front","average":{"1min":0.52700000000000002,"5min":0.60799999999999998,"15min":0.69699999999999995},"min":{"1min":0.376,"5min":0.31900000000000001,"15min":0.31900000000000001},"max":{"1min":0.70899999999999996,"5min":3.9940000000000002,"15min":3.9940000000000002},"last":0.42099999999999999}]},{"osd":3,"last update":"Sun Nov 26 07:21:47 2023","interfaces":[{"interface":"back","average":{"1min":0.497,"5min":0.55200000000000005,"15min":0.61199999999999999},"min":{"1min":0.32200000000000001,"5min":0.32000000000000001,"15min":0.32000000000000001},"max":{"1min":0.82699999999999996,"5min":4.0300000000000002,"15min":4.0300000000000002},"last":0.39000000000000001},{"interface":"front","average":{"1min":0.56399999999999995,"5min":0.70299999999999996,"15min":0.84299999999999997},"min":{"1min":0.36099999999999999,"5min":0.36099999999999999,"15min":0.36099999999999999},"max":{"1min":0.86699999999999999,"5min":4.5810000000000004,"15min":4.5810000000000004},"last":0.52300000000000002}]},{"osd":4,"last update":"Sun Nov 26 07:21:47 2023","interfaces":[{"interface":"back","average":{"1min":0.78500000000000003,"5min":0.85599999999999998,"15min":0.94199999999999995},"min":{"1min":0.58899999999999997,"5min":0.51400000000000001,"15min":0.51400000000000001},"max":{"1min":0.98699999999999999,"5min":4.4269999999999996,"15min":4.4269999999999996},"last":0.72199999999999998},{"interface":"front","average":{"1min":0.83799999999999997,"5min":0.88100000000000001,"15min":0.96999999999999997},"min":{"1min":0.55500000000000005,"5min":0.55500000000000005,"15min":0.55500000000000005},"max":{"1min":1.0780000000000001,"5min":4.085,"15min":4.085},"last":0.67600000000000005}]},{"osd":5,"last update":"Sun Nov 26 07:21:47 2023","interfaces":[{"interface":"back","average":{"1min":0.81200000000000006,"5min":0.81000000000000005,"15min":0.85199999999999998},"min":{"1min":0.60699999999999998,"5min":0.49199999999999999,"15min":0.49199999999999999},"max":{"1min":1,"5min":3.9590000000000001,"15min":3.9590000000000001},"last":0.82099999999999995},{"interface":"front","average":{"1min":0.81200000000000006,"5min":0.874,"15min":0.94999999999999996},"min":{"1min":0.499,"5min":0.499,"15min":0.499},"max":{"1min":1.024,"5min":4.3849999999999998,"15min":4.3849999999999998},"last":0.83499999999999996}]},{"osd":6,"last update":"Sun Nov 26 07:21:47 2023","interfaces":[{"interface":"back","average":{"1min":0.79100000000000004,"5min":0.86199999999999999,"15min":0.95099999999999996},"min":{"1min":0.51800000000000002,"5min":0.51800000000000002,"15min":0.51800000000000002},"max":{"1min":0.94999999999999996,"5min":4.0449999999999999,"15min":4.0449999999999999},"last":0.76700000000000002},{"interface":"front","average":{"1min":0.77000000000000002,"5min":0.83499999999999996,"15min":0.89000000000000001},"min":{"1min":0.48099999999999998,"5min":0.48099999999999998,"15min":0.48099999999999998},"max":{"1min":1,"5min":3.992,"15min":3.992},"last":0.74299999999999999}]},{"osd":7,"last update":"Sun Nov 26 07:21:47 2023","interfaces":[{"interface":"back","average":{"1min":0.81599999999999995,"5min":0.88600000000000001,"15min":0.98499999999999999},"min":{"1min":0.63900000000000001,"5min":0.52200000000000002,"15min":0.52200000000000002},"max":{"1min":1.0149999999999999,"5min":4.1219999999999999,"15min":4.1219999999999999},"last":0.81100000000000005},{"interface":"front","average":{"1min":0.80100000000000005,"5min":0.85699999999999998,"15min":0.91100000000000003},"min":{"1min":0.50700000000000001,"5min":0.50700000000000001,"15min":0.50700000000000001},"max":{"1min":0.96599999999999997,"5min":3.9159999999999999,"15min":3.9159999999999999},"last":0.85199999999999998}]}]},{"osd":1,"up_from":13,"seq":55834574912,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32052,"kb_used_data":864,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339788,"statfs":{"total":96636764160,"available":96603942912,"internally_reserved":0,"allocated":884736,"data_stored":366507,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":31914439},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Nov 26 07:21:45 2023","interfaces":[{"interface":"back","average":{"1min":0.39400000000000002,"5min":0.40999999999999998,"15min":0.41399999999999998},"min":{"1min":0.29899999999999999,"5min":0.23599999999999999,"15min":0.23599999999999999},"max":{"1min":0.57499999999999996,"5min":0.61199999999999999,"15min":0.61199999999999999},"last":0.45200000000000001},{"interface":"front","average":{"1min":0.435,"5min":0.46200000000000002,"15min":0.47799999999999998},"min":{"1min":0.28899999999999998,"5min":0.26700000000000002,"15min":0.26700000000000002},"max":{"1min":0.61699999999999999,"5min":0.78700000000000003,"15min":0.78700000000000003},"last":0.499}]},{"osd":2,"last update":"Sun Nov 26 07:21:45 2023","interfaces":[{"interface":"back","average":{"1min":0.42999999999999999,"5min":0.443,"15min":0.45100000000000001},"min":{"1min":0.35499999999999998,"5min":0.27800000000000002,"15min":0.27800000000000002},"max":{"1min":0.53600000000000003,"5min":0.67600000000000005,"15min":0.67600000000000005},"last":0.38300000000000001},{"interface":"front","average":{"1min":0.47399999999999998,"5min":0.498,"15min":0.504},"min":{"1min":0.35999999999999999,"5min":0.27700000000000002,"15min":0.27700000000000002},"max":{"1min":0.64200000000000002,"5min":0.80600000000000005,"15min":0.80600000000000005},"last":0.48199999999999998}]},{"osd":3,"last update":"Sun Nov 26 07:21:45 2023","interfaces":[{"interface":"back","average":{"1min":0.51200000000000001,"5min":0.52800000000000002,"15min":0.52900000000000003},"min":{"1min":0.42099999999999999,"5min":0.38400000000000001,"15min":0.38400000000000001},"max":{"1min":0.60799999999999998,"5min":0.748,"15min":0.748},"last":0.51700000000000002},{"interface":"front","average":{"1min":0.496,"5min":0.499,"15min":0.497},"min":{"1min":0.35499999999999998,"5min":0.31,"15min":0.31},"max":{"1min":0.68600000000000005,"5min":0.80700000000000005,"15min":0.80700000000000005},"last":0.46400000000000002}]},{"osd":4,"last update":"Sun Nov 26 07:21:45 2023","interfaces":[{"interface":"back","average":{"1min":0.749,"5min":0.71999999999999997,"15min":0.70199999999999996},"min":{"1min":0.51500000000000001,"5min":0.41299999999999998,"15min":0.41299999999999998},"max":{"1min":1.0589999999999999,"5min":1.1000000000000001,"15min":1.1000000000000001},"last":0.97999999999999998},{"interface":"front","average":{"1min":0.71999999999999997,"5min":0.71399999999999997,"15min":0.69099999999999995},"min":{"1min":0.44500000000000001,"5min":0.44500000000000001,"15min":0.44500000000000001},"max":{"1min":0.93300000000000005,"5min":1.131,"15min":1.131},"last":0.66600000000000004}]},{"osd":5,"last update":"Sun Nov 26 07:21:45 2023","interfaces":[{"interface":"back","average":{"1min":0.73399999999999999,"5min":0.73099999999999998,"15min":0.72299999999999998},"min":{"1min":0.52000000000000002,"5min":0.49299999999999999,"15min":0.49299999999999999},"max":{"1min":0.89900000000000002,"5min":1.1919999999999999,"15min":1.1919999999999999},"last":0.82899999999999996},{"interface":"front","average":{"1min":0.70399999999999996,"5min":0.72699999999999998,"15min":0.71999999999999997},"min":{"1min":0.54700000000000004,"5min":0.48799999999999999,"15min":0.48799999999999999},"max":{"1min":0.93400000000000005,"5min":1.0609999999999999,"15min":1.0609999999999999},"last":0.93899999999999995}]},{"osd":6,"last update":"Sun Nov 26 07:21:45 2023","interfaces":[{"interface":"back","average":{"1min":0.76000000000000001,"5min":0.73199999999999998,"15min":0.71299999999999997},"min":{"1min":0.55000000000000004,"5min":0.50600000000000001,"15min":0.50600000000000001},"max":{"1min":0.97799999999999998,"5min":1.0369999999999999,"15min":1.0369999999999999},"last":0.90400000000000003},{"interface":"front","average":{"1min":0.74399999999999999,"5min":0.73399999999999999,"15min":0.71199999999999997},"min":{"1min":0.59999999999999998,"5min":0.504,"15min":0.504},"max":{"1min":0.94999999999999996,"5min":1.1339999999999999,"15min":1.1339999999999999},"last":0.79100000000000004}]},{"osd":7,"last update":"Sun Nov 26 07:21:45 2023","interfaces":[{"interface":"back","average":{"1min":0.75,"5min":0.74099999999999999,"15min":0.72299999999999998},"min":{"1min":0.50600000000000001,"5min":0.50600000000000001,"15min":0.50600000000000001},"max":{"1min":0.92400000000000004,"5min":1.002,"15min":1.002},"last":0.84099999999999997},{"interface":"front","average":{"1min":0.73699999999999999,"5min":0.747,"15min":0.751},"min":{"1min":0.56999999999999995,"5min":0.48499999999999999,"15min":0.48499999999999999},"max":{"1min":1.018,"5min":1.024,"15min":1.024},"last":0.95299999999999996}]}]},{"osd":0,"up_from":13,"seq":55834574912,"num_pgs":15,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32608,"kb_used_data":1424,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339232,"statfs":{"total":96636764160,"available":96603373568,"internally_reserved":0,"allocated":1458176,"data_stored":954647,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Sun Nov 26 07:21:43 2023","interfaces":[{"interface":"back","average":{"1min":0.438,"5min":0.42599999999999999,"15min":0.42899999999999999},"min":{"1min":0.27000000000000002,"5min":0.27000000000000002,"15min":0.27000000000000002},"max":{"1min":0.624,"5min":0.64300000000000002,"15min":0.64300000000000002},"last":0.33700000000000002},{"interface":"front","average":{"1min":0.47299999999999998,"5min":0.46800000000000003,"15min":0.47799999999999998},"min":{"1min":0.32200000000000001,"5min":0.29899999999999999,"15min":0.29899999999999999},"max":{"1min":0.77200000000000002,"5min":1.133,"15min":1.133},"last":0.36899999999999999}]},{"osd":2,"last update":"Sun Nov 26 07:21:43 2023","interfaces":[{"interface":"back","average":{"1min":0.496,"5min":0.51800000000000002,"15min":0.53000000000000003},"min":{"1min":0.34699999999999998,"5min":0.34699999999999998,"15min":0.34699999999999998},"max":{"1min":0.84199999999999997,"5min":0.84199999999999997,"15min":0.84199999999999997},"last":0.69399999999999995},{"interface":"front","average":{"1min":0.56299999999999994,"5min":0.55700000000000005,"15min":0.56100000000000005},"min":{"1min":0.41399999999999998,"5min":0.36399999999999999,"15min":0.36399999999999999},"max":{"1min":0.77300000000000002,"5min":0.84199999999999997,"15min":0.84199999999999997},"last":0.40100000000000002}]},{"osd":3,"last update":"Sun Nov 26 07:21:43 2023","interfaces":[{"interface":"back","average":{"1min":0.52200000000000002,"5min":0.52200000000000002,"15min":0.52800000000000002},"min":{"1min":0.39000000000000001,"5min":0.34999999999999998,"15min":0.34999999999999998},"max":{"1min":0.75700000000000001,"5min":1.3149999999999999,"15min":1.3149999999999999},"last":0.873},{"interface":"front","average":{"1min":0.55900000000000005,"5min":0.54400000000000004,"15min":0.55600000000000005},"min":{"1min":0.40000000000000002,"5min":0.371,"15min":0.371},"max":{"1min":0.72799999999999998,"5min":1.2509999999999999,"15min":1.2509999999999999},"last":0.42099999999999999}]},{"osd":4,"last update":"Sun Nov 26 07:21:43 2023","interfaces":[{"interface":"back","average":{"1min":0.747,"5min":0.749,"15min":0.755},"min":{"1min":0.44700000000000001,"5min":0.44700000000000001,"15min":0.44700000000000001},"max":{"1min":1.0900000000000001,"5min":1.2010000000000001,"15min":1.2010000000000001},"last":0.81200000000000006},{"interface":"front","average":{"1min":0.78000000000000003,"5min":0.78800000000000003,"15min":0.78800000000000003},"min":{"1min":0.51100000000000001,"5min":0.51100000000000001,"15min":0.51100000000000001},"max":{"1min":0.99199999999999999,"5min":1.381,"15min":1.381},"last":0.83599999999999997}]},{"osd":5,"last update":"Sun Nov 26 07:21:43 2023","interfaces":[{"interface":"back","average":{"1min":0.75900000000000001,"5min":0.76100000000000001,"15min":0.76000000000000001},"min":{"1min":0.54800000000000004,"5min":0.42199999999999999,"15min":0.42199999999999999},"max":{"1min":0.97999999999999998,"5min":1.1579999999999999,"15min":1.1579999999999999},"last":0.88700000000000001},{"interface":"front","average":{"1min":0.81499999999999995,"5min":0.80600000000000005,"15min":0.80000000000000004},"min":{"1min":0.53600000000000003,"5min":0.53600000000000003,"15min":0.53600000000000003},"max":{"1min":1.0680000000000001,"5min":1.399,"15min":1.399},"last":0.78900000000000003}]},{"osd":6,"last update":"Sun Nov 26 07:21:43 2023","interfaces":[{"interface":"back","average":{"1min":0.72299999999999998,"5min":0.72199999999999998,"15min":0.75},"min":{"1min":0.52400000000000002,"5min":0.50800000000000001,"15min":0.50800000000000001},"max":{"1min":0.89300000000000002,"5min":1.3029999999999999,"15min":1.3029999999999999},"last":0.71299999999999997},{"interface":"front","average":{"1min":0.747,"5min":0.75700000000000001,"15min":0.76500000000000001},"min":{"1min":0.47199999999999998,"5min":0.47199999999999998,"15min":0.47199999999999998},"max":{"1min":1.0489999999999999,"5min":1.3600000000000001,"15min":1.3600000000000001},"last":0.73199999999999998}]},{"osd":7,"last update":"Sun Nov 26 07:21:43 2023","interfaces":[{"interface":"back","average":{"1min":0.77200000000000002,"5min":0.78800000000000003,"15min":0.80300000000000005},"min":{"1min":0.56899999999999995,"5min":0.56899999999999995,"15min":0.56899999999999995},"max":{"1min":0.98899999999999999,"5min":1.177,"15min":1.177},"last":0.82599999999999996},{"interface":"front","average":{"1min":0.78600000000000003,"5min":0.76700000000000002,"15min":0.78800000000000003},"min":{"1min":0.51200000000000001,"5min":0.51200000000000001,"15min":0.51200000000000001},"max":{"1min":1.0629999999999999,"5min":1.2390000000000001,"15min":1.2390000000000001},"last":0.85499999999999998}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":2228,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":112,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":70,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":92,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":616,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-11-26T07:22:42.304 INFO:tasks.cephfs.filesystem.ceph_manager:clean! 2023-11-26T07:22:42.304 DEBUG:teuthology.orchestra.run.smithi143:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth caps client.0 mon 'allow r' osd 'allow rw tag cephfs data=cephfs' mds allow 2023-11-26T07:22:42.486 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:42.484+0000 7f587afa4640 1 Processor -- start 2023-11-26T07:22:42.486 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:42.484+0000 7f587afa4640 1 -- start start 2023-11-26T07:22:42.488 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:42.488+0000 7f587afa4640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f5874153b90 0x7f587415a600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:22:42.488 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:42.488+0000 7f587afa4640 1 -- --> v1:172.21.15.143:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f58740b1e10 con 0x7f587406d820 2023-11-26T07:22:42.488 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:42.488+0000 7f587afa4640 1 -- --> v1:172.21.15.188:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f587415ab40 con 0x7f58740b2b40 2023-11-26T07:22:42.488 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:42.488+0000 7f587afa4640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f58740b4160 con 0x7f5874153b90 2023-11-26T07:22:42.488 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:42.488+0000 7f587a7a3640 1 --1- >> v1:172.21.15.143:6789/0 conn(0x7f587406d820 0x7f58740ccc80 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.143:6789/0 says I am v1:172.21.15.143:35800/0 (socket says 172.21.15.143:35800) 2023-11-26T07:22:42.488 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:42.488+0000 7f587a7a3640 1 -- 172.21.15.143:0/2803933173 learned_addr learned my addr 172.21.15.143:0/2803933173 (peer_addr_for_me v1:172.21.15.143:0/0) 2023-11-26T07:22:42.489 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:42.488+0000 7f5879fa2640 1 --2- 172.21.15.143:0/2803933173 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f5874153b90 0x7f587415a600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:22:42.489 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:42.488+0000 7f5878fa0640 1 -- 172.21.15.143:0/2803933173 <== mon.0 v1:172.21.15.143:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2941693054 0 0) 0x7f58740b1e10 con 0x7f587406d820 2023-11-26T07:22:42.490 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:42.488+0000 7f5878fa0640 1 -- 172.21.15.143:0/2803933173 --> v1:172.21.15.143:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f5860003530 con 0x7f587406d820 2023-11-26T07:22:42.490 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:42.488+0000 7f5878fa0640 1 -- 172.21.15.143:0/2803933173 <== mon.1 v1:172.21.15.188:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1429507444 0 0) 0x7f587415ab40 con 0x7f58740b2b40 2023-11-26T07:22:42.490 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:42.488+0000 7f5878fa0640 1 -- 172.21.15.143:0/2803933173 --> v1:172.21.15.188:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f58740b1e10 con 0x7f58740b2b40 2023-11-26T07:22:42.490 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:42.488+0000 7f5879fa2640 1 -- 172.21.15.143:0/2803933173 >> v1:172.21.15.188:6789/0 conn(0x7f58740b2b40 legacy=0x7f587406d110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:22:42.490 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:42.488+0000 7f5879fa2640 1 -- 172.21.15.143:0/2803933173 >> v1:172.21.15.143:6789/0 conn(0x7f587406d820 legacy=0x7f58740ccc80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:22:42.490 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:42.488+0000 7f5878fa0640 1 -- 172.21.15.143:0/2803933173 <== mon.0 v1:172.21.15.143:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1697917218 0 0) 0x7f5860003530 con 0x7f587406d820 2023-11-26T07:22:42.490 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:42.488+0000 7f5879fa2640 1 -- 172.21.15.143:0/2803933173 --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f58740cea00 con 0x7f5874153b90 2023-11-26T07:22:42.492 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:42.492+0000 7f5879fa2640 1 --2- 172.21.15.143:0/2803933173 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f5874153b90 0x7f587415a600 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7f58700097f0 tx=0x7f587001c3b0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=550917d3bbd3bd1b server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:22:42.492 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:42.492+0000 7f5878fa0640 1 -- 172.21.15.143:0/2803933173 <== mon.2 v2:172.21.15.188:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f586c001a00 con 0x7f5874153b90 2023-11-26T07:22:42.492 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:42.492+0000 7f5878fa0640 1 -- 172.21.15.143:0/2803933173 <== mon.2 v2:172.21.15.188:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f58700a7050 con 0x7f5874153b90 2023-11-26T07:22:42.492 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:42.492+0000 7f5878fa0640 1 -- 172.21.15.143:0/2803933173 <== mon.2 v2:172.21.15.188:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f58700ab600 con 0x7f5874153b90 2023-11-26T07:22:42.493 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:42.492+0000 7f587afa4640 1 -- 172.21.15.143:0/2803933173 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f5874153b90 msgr2=0x7f587415a600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:22:42.493 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:42.492+0000 7f587afa4640 1 --2- 172.21.15.143:0/2803933173 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f5874153b90 0x7f587415a600 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7f58700097f0 tx=0x7f587001c3b0 comp rx=0 tx=0).stop 2023-11-26T07:22:42.493 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:42.492+0000 7f587afa4640 1 -- 172.21.15.143:0/2803933173 shutdown_connections 2023-11-26T07:22:42.493 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:42.492+0000 7f587afa4640 1 --2- 172.21.15.143:0/2803933173 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f5874153b90 0x7f587415a600 unknown :-1 s=CLOSED pgs=124 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:22:42.493 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:42.492+0000 7f587afa4640 1 -- 172.21.15.143:0/2803933173 >> 172.21.15.143:0/2803933173 conn(0x7f587406a890 msgr2=0x7f587406aca0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:22:42.494 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:42.492+0000 7f587afa4640 1 -- 172.21.15.143:0/2803933173 shutdown_connections 2023-11-26T07:22:42.494 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:42.492+0000 7f587afa4640 1 -- 172.21.15.143:0/2803933173 wait complete. 2023-11-26T07:22:42.495 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:42.492+0000 7f587afa4640 1 Processor -- start 2023-11-26T07:22:42.495 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:42.492+0000 7f587afa4640 1 -- start start 2023-11-26T07:22:42.496 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:42.496+0000 7f587afa4640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f587406d820 0x7f5874140200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:22:42.496 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:42.496+0000 7f587afa4640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f58740b2b40 0x7f58741505b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:22:42.497 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:42.496+0000 7f587afa4640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f5874153b90 0x7f5874150af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:22:42.497 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:42.496+0000 7f587afa4640 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5874159380 con 0x7f587406d820 2023-11-26T07:22:42.497 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:42.496+0000 7f587afa4640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5874158db0 con 0x7f5874153b90 2023-11-26T07:22:42.497 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:42.496+0000 7f587afa4640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5874159590 con 0x7f58740b2b40 2023-11-26T07:22:42.497 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:42.496+0000 7f5879fa2640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f587406d820 0x7f5874140200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:22:42.497 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:42.496+0000 7f5879fa2640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f587406d820 0x7f5874140200 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:47310/0 (socket says 172.21.15.143:47310) 2023-11-26T07:22:42.497 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:42.496+0000 7f5879fa2640 1 -- 172.21.15.143:0/3383544224 learned_addr learned my addr 172.21.15.143:0/3383544224 (peer_addr_for_me v2:172.21.15.143:0/0) 2023-11-26T07:22:42.497 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:42.496+0000 7f58797a1640 1 --2- 172.21.15.143:0/3383544224 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f58740b2b40 0x7f58741505b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:22:42.497 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:42.496+0000 7f587a7a3640 1 --2- 172.21.15.143:0/3383544224 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f5874153b90 0x7f5874150af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:22:42.497 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:42.496+0000 7f5879fa2640 1 -- 172.21.15.143:0/3383544224 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f58740b2b40 msgr2=0x7f58741505b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:22:42.497 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:42.496+0000 7f5879fa2640 1 --2- 172.21.15.143:0/3383544224 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f58740b2b40 0x7f58741505b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:22:42.497 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:42.496+0000 7f5879fa2640 1 -- 172.21.15.143:0/3383544224 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f5874153b90 msgr2=0x7f5874150af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:22:42.498 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:42.496+0000 7f5879fa2640 1 --2- 172.21.15.143:0/3383544224 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f5874153b90 0x7f5874150af0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:22:42.498 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:42.496+0000 7f5879fa2640 1 -- 172.21.15.143:0/3383544224 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5870009420 con 0x7f587406d820 2023-11-26T07:22:42.498 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:42.496+0000 7f5879fa2640 1 --2- 172.21.15.143:0/3383544224 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f587406d820 0x7f5874140200 secure :-1 s=READY pgs=968 cs=0 l=1 rev1=1 crypto rx=0x7f587001aa60 tx=0x7f5870019150 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:22:42.499 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:42.496+0000 7f586a7fc640 1 -- 172.21.15.143:0/3383544224 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5870003a30 con 0x7f587406d820 2023-11-26T07:22:42.499 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:42.496+0000 7f586a7fc640 1 -- 172.21.15.143:0/3383544224 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5870003b90 con 0x7f587406d820 2023-11-26T07:22:42.499 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:42.496+0000 7f586a7fc640 1 -- 172.21.15.143:0/3383544224 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f58700b6420 con 0x7f587406d820 2023-11-26T07:22:42.499 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:42.496+0000 7f587afa4640 1 -- 172.21.15.143:0/3383544224 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5874159870 con 0x7f587406d820 2023-11-26T07:22:42.499 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:42.496+0000 7f587afa4640 1 -- 172.21.15.143:0/3383544224 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f58741589f0 con 0x7f587406d820 2023-11-26T07:22:42.499 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:42.496+0000 7f587afa4640 1 -- 172.21.15.143:0/3383544224 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5874067500 con 0x7f587406d820 2023-11-26T07:22:42.499 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:42.496+0000 7f586a7fc640 1 -- 172.21.15.143:0/3383544224 <== mon.0 v2:172.21.15.143:3300/0 4 ==== mgrmap(e 11) v1 ==== 86158+0+0 (secure 0 0 0) 0x7f58700ab400 con 0x7f587406d820 2023-11-26T07:22:42.500 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:42.500+0000 7f586a7fc640 1 --2- 172.21.15.143:0/3383544224 >> [v2:172.21.15.143:6832/2150557544,v1:172.21.15.143:6833/2150557544] conn(0x7f58580667d0 0x7f5858068c90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:22:42.500 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:42.500+0000 7f586a7fc640 1 -- 172.21.15.143:0/3383544224 <== mon.0 v2:172.21.15.143:3300/0 5 ==== osd_map(71..71 src has 1..71) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f587011e680 con 0x7f587406d820 2023-11-26T07:22:42.500 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:42.500+0000 7f58797a1640 1 --2- 172.21.15.143:0/3383544224 >> [v2:172.21.15.143:6832/2150557544,v1:172.21.15.143:6833/2150557544] conn(0x7f58580667d0 0x7f5858068c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:22:42.501 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:42.500+0000 7f58797a1640 1 --2- 172.21.15.143:0/3383544224 >> [v2:172.21.15.143:6832/2150557544,v1:172.21.15.143:6833/2150557544] conn(0x7f58580667d0 0x7f5858068c90 secure :-1 s=READY pgs=454 cs=0 l=1 rev1=1 crypto rx=0x7f58640011f0 tx=0x7f5864009040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:22:42.503 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:42.504+0000 7f586a7fc640 1 -- 172.21.15.143:0/3383544224 <== mon.0 v2:172.21.15.143:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172649 (secure 0 0 0) 0x7f58700eb650 con 0x7f587406d820 2023-11-26T07:22:42.703 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:42.700+0000 7f587afa4640 1 -- 172.21.15.143:0/3383544224 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]} v 0) v1 -- 0x7f58740b3790 con 0x7f587406d820 2023-11-26T07:22:42.706 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:42.704+0000 7f586a7fc640 1 -- 172.21.15.143:0/3383544224 <== mon.0 v2:172.21.15.143:3300/0 7 ==== mon_command_ack([{"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]}]=0 updated caps for client.0 v18) v1 ==== 190+0+147 (secure 0 0 0) 0x7f58700eec70 con 0x7f587406d820 2023-11-26T07:22:42.706 INFO:teuthology.orchestra.run.smithi143.stdout:[client.0] 2023-11-26T07:22:42.706 INFO:teuthology.orchestra.run.smithi143.stdout: key = AQDm8GJlWMh9FBAAslVzi6fm4/wt7vr/rHCHHQ== 2023-11-26T07:22:42.706 INFO:teuthology.orchestra.run.smithi143.stdout: caps mds = "allow" 2023-11-26T07:22:42.706 INFO:teuthology.orchestra.run.smithi143.stdout: caps mon = "allow r" 2023-11-26T07:22:42.706 INFO:teuthology.orchestra.run.smithi143.stdout: caps osd = "allow rw tag cephfs data=cephfs" 2023-11-26T07:22:42.706 INFO:teuthology.orchestra.run.smithi143.stderr:updated caps for client.0 2023-11-26T07:22:42.708 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:42.708+0000 7f587afa4640 1 -- 172.21.15.143:0/3383544224 >> [v2:172.21.15.143:6832/2150557544,v1:172.21.15.143:6833/2150557544] conn(0x7f58580667d0 msgr2=0x7f5858068c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:22:42.708 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:42.708+0000 7f587afa4640 1 --2- 172.21.15.143:0/3383544224 >> [v2:172.21.15.143:6832/2150557544,v1:172.21.15.143:6833/2150557544] conn(0x7f58580667d0 0x7f5858068c90 secure :-1 s=READY pgs=454 cs=0 l=1 rev1=1 crypto rx=0x7f58640011f0 tx=0x7f5864009040 comp rx=0 tx=0).stop 2023-11-26T07:22:42.708 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:42.708+0000 7f587afa4640 1 -- 172.21.15.143:0/3383544224 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f587406d820 msgr2=0x7f5874140200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:22:42.708 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:42.708+0000 7f587afa4640 1 --2- 172.21.15.143:0/3383544224 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f587406d820 0x7f5874140200 secure :-1 s=READY pgs=968 cs=0 l=1 rev1=1 crypto rx=0x7f587001aa60 tx=0x7f5870019150 comp rx=0 tx=0).stop 2023-11-26T07:22:42.709 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:42.708+0000 7f587afa4640 1 -- 172.21.15.143:0/3383544224 shutdown_connections 2023-11-26T07:22:42.709 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:42.708+0000 7f587afa4640 1 --2- 172.21.15.143:0/3383544224 >> [v2:172.21.15.143:6832/2150557544,v1:172.21.15.143:6833/2150557544] conn(0x7f58580667d0 0x7f5858068c90 unknown :-1 s=CLOSED pgs=454 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:22:42.709 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:42.708+0000 7f587afa4640 1 --2- 172.21.15.143:0/3383544224 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f5874153b90 0x7f5874150af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:22:42.709 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:42.708+0000 7f587afa4640 1 --2- 172.21.15.143:0/3383544224 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f58740b2b40 0x7f58741505b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:22:42.709 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:42.708+0000 7f587afa4640 1 --2- 172.21.15.143:0/3383544224 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f587406d820 0x7f5874140200 unknown :-1 s=CLOSED pgs=968 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:22:42.709 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:42.708+0000 7f587afa4640 1 -- 172.21.15.143:0/3383544224 >> 172.21.15.143:0/3383544224 conn(0x7f587406a890 msgr2=0x7f58740af9c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:22:42.709 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:42.708+0000 7f587afa4640 1 -- 172.21.15.143:0/3383544224 shutdown_connections 2023-11-26T07:22:42.709 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:42.708+0000 7f587afa4640 1 -- 172.21.15.143:0/3383544224 wait complete. 2023-11-26T07:22:42.727 DEBUG:teuthology.orchestra.run.smithi143:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-11-26T07:22:42.832 INFO:tasks.ceph.mds.a.smithi143.stderr:2023-11-26T07:22:42.832+0000 7fa827c91640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2023-11-26T07:22:42.865 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:42.864+0000 7f97e759e640 1 Processor -- start 2023-11-26T07:22:42.865 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:42.864+0000 7f97e759e640 1 -- start start 2023-11-26T07:22:42.867 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:42.864+0000 7f97e759e640 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7f97e815e0f0 0x7f97e80ce030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:22:42.867 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:42.864+0000 7f97e759e640 1 -- --> v1:172.21.15.143:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f97e815e860 con 0x7f97e80b1e10 2023-11-26T07:22:42.867 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:42.864+0000 7f97e759e640 1 -- --> v1:172.21.15.188:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f97e80cdba0 con 0x7f97e8143810 2023-11-26T07:22:42.867 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:42.864+0000 7f97e759e640 1 -- --> v2:172.21.15.143:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f97e8143bf0 con 0x7f97e815e0f0 2023-11-26T07:22:42.867 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:42.864+0000 7f97e6d9d640 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7f97e815e0f0 0x7f97e80ce030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:22:42.867 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:42.864+0000 7f97e5d9b640 1 --1- >> v1:172.21.15.143:6789/0 conn(0x7f97e80b1e10 0x7f97e815d9e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.143:6789/0 says I am v1:172.21.15.143:35816/0 (socket says 172.21.15.143:35816) 2023-11-26T07:22:42.867 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:42.864+0000 7f97e5d9b640 1 -- 172.21.15.143:0/3626054167 learned_addr learned my addr 172.21.15.143:0/3626054167 (peer_addr_for_me v1:172.21.15.143:0/0) 2023-11-26T07:22:42.868 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:42.868+0000 7f97e6d9d640 1 -- 172.21.15.143:0/3626054167 >> v1:172.21.15.188:6789/0 conn(0x7f97e8143810 legacy=0x7f97e8072140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:22:42.868 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:42.868+0000 7f97e6d9d640 1 -- 172.21.15.143:0/3626054167 >> v1:172.21.15.143:6789/0 conn(0x7f97e80b1e10 legacy=0x7f97e815d9e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:22:42.868 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:42.868+0000 7f97e6d9d640 1 -- 172.21.15.143:0/3626054167 --> v2:172.21.15.143:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f97e80714f0 con 0x7f97e815e0f0 2023-11-26T07:22:42.868 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:42.868+0000 7f97e559a640 1 -- 172.21.15.143:0/3626054167 <== mon.0 v1:172.21.15.143:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1930186048 0 0) 0x7f97e815e860 con 0x7f97e80b1e10 2023-11-26T07:22:42.870 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:42.868+0000 7f97e6d9d640 1 --2- 172.21.15.143:0/3626054167 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f97e815e0f0 0x7f97e80ce030 secure :-1 s=READY pgs=970 cs=0 l=1 rev1=1 crypto rx=0x7f97e0009d90 tx=0x7f97e001c8e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=868b9d07d5b89038 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:22:42.870 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:42.868+0000 7f97e559a640 1 -- 172.21.15.143:0/3626054167 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f97e00a59a0 con 0x7f97e815e0f0 2023-11-26T07:22:42.871 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:42.868+0000 7f97e559a640 1 -- 172.21.15.143:0/3626054167 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f97e00a5b00 con 0x7f97e815e0f0 2023-11-26T07:22:42.871 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:42.868+0000 7f97e559a640 1 -- 172.21.15.143:0/3626054167 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f97e00a5e90 con 0x7f97e815e0f0 2023-11-26T07:22:42.871 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:42.868+0000 7f97e759e640 1 -- 172.21.15.143:0/3626054167 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f97e815e0f0 msgr2=0x7f97e80ce030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:22:42.871 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:42.868+0000 7f97e759e640 1 --2- 172.21.15.143:0/3626054167 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f97e815e0f0 0x7f97e80ce030 secure :-1 s=READY pgs=970 cs=0 l=1 rev1=1 crypto rx=0x7f97e0009d90 tx=0x7f97e001c8e0 comp rx=0 tx=0).stop 2023-11-26T07:22:42.871 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:42.868+0000 7f97e759e640 1 -- 172.21.15.143:0/3626054167 shutdown_connections 2023-11-26T07:22:42.872 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:42.868+0000 7f97e759e640 1 --2- 172.21.15.143:0/3626054167 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f97e815e0f0 0x7f97e80ce030 unknown :-1 s=CLOSED pgs=970 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:22:42.872 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:42.868+0000 7f97e759e640 1 -- 172.21.15.143:0/3626054167 >> 172.21.15.143:0/3626054167 conn(0x7f97e806a890 msgr2=0x7f97e806aca0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:22:42.872 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:42.872+0000 7f97e759e640 1 -- 172.21.15.143:0/3626054167 shutdown_connections 2023-11-26T07:22:42.872 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:42.872+0000 7f97e759e640 1 -- 172.21.15.143:0/3626054167 wait complete. 2023-11-26T07:22:42.873 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:42.872+0000 7f97e759e640 1 Processor -- start 2023-11-26T07:22:42.873 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:42.872+0000 7f97e759e640 1 -- start start 2023-11-26T07:22:42.875 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:42.872+0000 7f97e759e640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f97e80b1e10 0x7f97e813b690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:22:42.875 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:42.872+0000 7f97e759e640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f97e8143810 0x7f97e8158ec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:22:42.875 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:42.872+0000 7f97e759e640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f97e815e0f0 0x7f97e8159400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:22:42.875 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:42.872+0000 7f97e759e640 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f97e813bbd0 con 0x7f97e815e0f0 2023-11-26T07:22:42.876 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:42.872+0000 7f97e759e640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f97e813e490 con 0x7f97e8143810 2023-11-26T07:22:42.876 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:42.872+0000 7f97e759e640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f97e81f9830 con 0x7f97e80b1e10 2023-11-26T07:22:42.876 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:42.872+0000 7f97e6d9d640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f97e815e0f0 0x7f97e8159400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:22:42.876 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:42.872+0000 7f97e6d9d640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f97e815e0f0 0x7f97e8159400 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:47326/0 (socket says 172.21.15.143:47326) 2023-11-26T07:22:42.876 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:42.872+0000 7f97e6d9d640 1 -- 172.21.15.143:0/1854224495 learned_addr learned my addr 172.21.15.143:0/1854224495 (peer_addr_for_me v2:172.21.15.143:0/0) 2023-11-26T07:22:42.876 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:42.872+0000 7f97e5d9b640 1 --2- 172.21.15.143:0/1854224495 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f97e8143810 0x7f97e8158ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:22:42.876 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:42.872+0000 7f97e659c640 1 --2- 172.21.15.143:0/1854224495 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f97e80b1e10 0x7f97e813b690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:22:42.876 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:42.872+0000 7f97e6d9d640 1 -- 172.21.15.143:0/1854224495 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f97e80b1e10 msgr2=0x7f97e813b690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:22:42.876 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:42.872+0000 7f97e6d9d640 1 --2- 172.21.15.143:0/1854224495 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f97e80b1e10 0x7f97e813b690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:22:42.876 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:42.872+0000 7f97e6d9d640 1 -- 172.21.15.143:0/1854224495 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f97e8143810 msgr2=0x7f97e8158ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:22:42.876 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:42.872+0000 7f97e6d9d640 1 --2- 172.21.15.143:0/1854224495 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f97e8143810 0x7f97e8158ec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:22:42.876 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:42.872+0000 7f97e6d9d640 1 -- 172.21.15.143:0/1854224495 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f97e0009a70 con 0x7f97e815e0f0 2023-11-26T07:22:42.877 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:42.876+0000 7f97e6d9d640 1 --2- 172.21.15.143:0/1854224495 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f97e815e0f0 0x7f97e8159400 secure :-1 s=READY pgs=971 cs=0 l=1 rev1=1 crypto rx=0x7f97e00194f0 tx=0x7f97e0019680 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:22:42.877 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:42.876+0000 7f97d6ffd640 1 -- 172.21.15.143:0/1854224495 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f97e00ade80 con 0x7f97e815e0f0 2023-11-26T07:22:42.878 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:42.876+0000 7f97d6ffd640 1 -- 172.21.15.143:0/1854224495 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f97e00039a0 con 0x7f97e815e0f0 2023-11-26T07:22:42.878 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:42.876+0000 7f97e759e640 1 -- 172.21.15.143:0/1854224495 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f97e81f9b10 con 0x7f97e815e0f0 2023-11-26T07:22:42.878 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:42.876+0000 7f97d6ffd640 1 -- 172.21.15.143:0/1854224495 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f97e0003d50 con 0x7f97e815e0f0 2023-11-26T07:22:42.878 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:42.876+0000 7f97e759e640 1 -- 172.21.15.143:0/1854224495 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f97e81fa0d0 con 0x7f97e815e0f0 2023-11-26T07:22:42.878 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:42.876+0000 7f97e759e640 1 -- 172.21.15.143:0/1854224495 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f97e80cc010 con 0x7f97e815e0f0 2023-11-26T07:22:42.878 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:42.876+0000 7f97d6ffd640 1 -- 172.21.15.143:0/1854224495 <== mon.0 v2:172.21.15.143:3300/0 4 ==== mgrmap(e 11) v1 ==== 86158+0+0 (secure 0 0 0) 0x7f97e0004030 con 0x7f97e815e0f0 2023-11-26T07:22:42.879 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:42.876+0000 7f97d6ffd640 1 --2- 172.21.15.143:0/1854224495 >> [v2:172.21.15.143:6832/2150557544,v1:172.21.15.143:6833/2150557544] conn(0x7f97c80667d0 0x7f97c8068c90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:22:42.879 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:42.876+0000 7f97d6ffd640 1 -- 172.21.15.143:0/1854224495 <== mon.0 v2:172.21.15.143:3300/0 5 ==== osd_map(71..71 src has 1..71) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f97e011f8c0 con 0x7f97e815e0f0 2023-11-26T07:22:42.879 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:42.876+0000 7f97e659c640 1 --2- 172.21.15.143:0/1854224495 >> [v2:172.21.15.143:6832/2150557544,v1:172.21.15.143:6833/2150557544] conn(0x7f97c80667d0 0x7f97c8068c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:22:42.879 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:42.880+0000 7f97e659c640 1 --2- 172.21.15.143:0/1854224495 >> [v2:172.21.15.143:6832/2150557544,v1:172.21.15.143:6833/2150557544] conn(0x7f97c80667d0 0x7f97c8068c90 secure :-1 s=READY pgs=455 cs=0 l=1 rev1=1 crypto rx=0x7f97d8007fd0 tx=0x7f97d8009040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:22:42.882 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:42.880+0000 7f97d6ffd640 1 -- 172.21.15.143:0/1854224495 <== mon.0 v2:172.21.15.143:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172649 (secure 0 0 0) 0x7f97e00ecaf0 con 0x7f97e815e0f0 2023-11-26T07:22:43.062 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:43.060+0000 7f97e759e640 1 -- 172.21.15.143:0/1854224495 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f97e8067500 con 0x7f97e815e0f0 2023-11-26T07:22:43.062 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:43.060+0000 7f97d6ffd640 1 -- 172.21.15.143:0/1854224495 <== mon.0 v2:172.21.15.143:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 42 v42) v1 ==== 94+0+4924 (secure 0 0 0) 0x7f97e00b2020 con 0x7f97e815e0f0 2023-11-26T07:22:43.063 INFO:teuthology.orchestra.run.smithi143.stdout: 2023-11-26T07:22:43.063 INFO:teuthology.orchestra.run.smithi143.stdout:{"epoch":42,"default_fscid":5,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5927,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.143:6837/578144260","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6835","nonce":578144260},{"type":"v1","addr":"172.21.15.143:6837","nonce":578144260}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":38},{"gid":5930,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.188:6834/1048736873","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6832","nonce":1048736873},{"type":"v1","addr":"172.21.15.188:6834","nonce":1048736873}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":38},{"gid":5933,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.188:6835/2063768716","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6833","nonce":2063768716},{"type":"v1","addr":"172.21.15.188:6835","nonce":2063768716}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":38}],"filesystems":[{"mdsmap":{"epoch":42,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-11-26T07:22:37.220113+0000","modified":"2023-11-26T07:22:40.243052+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":6553},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6553":{"gid":6553,"name":"a","rank":0,"incarnation":39,"state":"up:active","state_seq":3,"addr":"172.21.15.143:6836/2590663349","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6834","nonce":2590663349},{"type":"v1","addr":"172.21.15.143:6836","nonce":2590663349}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[11],"metadata_pool":10,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":5}]} 2023-11-26T07:22:43.063 INFO:teuthology.orchestra.run.smithi143.stderr:dumped fsmap epoch 42 2023-11-26T07:22:43.065 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:43.064+0000 7f97e759e640 1 -- 172.21.15.143:0/1854224495 >> [v2:172.21.15.143:6832/2150557544,v1:172.21.15.143:6833/2150557544] conn(0x7f97c80667d0 msgr2=0x7f97c8068c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:22:43.065 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:43.064+0000 7f97e759e640 1 --2- 172.21.15.143:0/1854224495 >> [v2:172.21.15.143:6832/2150557544,v1:172.21.15.143:6833/2150557544] conn(0x7f97c80667d0 0x7f97c8068c90 secure :-1 s=READY pgs=455 cs=0 l=1 rev1=1 crypto rx=0x7f97d8007fd0 tx=0x7f97d8009040 comp rx=0 tx=0).stop 2023-11-26T07:22:43.065 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:43.064+0000 7f97e759e640 1 -- 172.21.15.143:0/1854224495 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f97e815e0f0 msgr2=0x7f97e8159400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:22:43.065 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:43.064+0000 7f97e759e640 1 --2- 172.21.15.143:0/1854224495 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f97e815e0f0 0x7f97e8159400 secure :-1 s=READY pgs=971 cs=0 l=1 rev1=1 crypto rx=0x7f97e00194f0 tx=0x7f97e0019680 comp rx=0 tx=0).stop 2023-11-26T07:22:43.065 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:43.064+0000 7f97e759e640 1 -- 172.21.15.143:0/1854224495 shutdown_connections 2023-11-26T07:22:43.065 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:43.064+0000 7f97e759e640 1 --2- 172.21.15.143:0/1854224495 >> [v2:172.21.15.143:6832/2150557544,v1:172.21.15.143:6833/2150557544] conn(0x7f97c80667d0 0x7f97c8068c90 unknown :-1 s=CLOSED pgs=455 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:22:43.065 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:43.064+0000 7f97e759e640 1 --2- 172.21.15.143:0/1854224495 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f97e815e0f0 0x7f97e8159400 unknown :-1 s=CLOSED pgs=971 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:22:43.065 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:43.064+0000 7f97e759e640 1 --2- 172.21.15.143:0/1854224495 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f97e8143810 0x7f97e8158ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:22:43.065 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:43.064+0000 7f97e759e640 1 --2- 172.21.15.143:0/1854224495 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f97e80b1e10 0x7f97e813b690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:22:43.065 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:43.064+0000 7f97e759e640 1 -- 172.21.15.143:0/1854224495 >> 172.21.15.143:0/1854224495 conn(0x7f97e806a890 msgr2=0x7f97e81426b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:22:43.065 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:43.064+0000 7f97e759e640 1 -- 172.21.15.143:0/1854224495 shutdown_connections 2023-11-26T07:22:43.066 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:43.064+0000 7f97e759e640 1 -- 172.21.15.143:0/1854224495 wait complete. 2023-11-26T07:22:43.486 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 42, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2023-11-26T07:22:37.220113+0000', 'modified': '2023-11-26T07:22:40.243052+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 6553}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_6553': {'gid': 6553, 'name': 'a', 'rank': 0, 'incarnation': 39, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.143:6836/2590663349', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.143:6834', 'nonce': 2590663349}, {'type': 'v1', 'addr': '172.21.15.143:6836', 'nonce': 2590663349}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138322906710015, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}}}, 'data_pools': [11], 'metadata_pool': 10, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0} 2023-11-26T07:22:43.486 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2023-11-26T07:22:43.486 DEBUG:teuthology.orchestra.run.smithi143:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.a status 2023-11-26T07:22:43.625 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:43.624+0000 7f43ae5f8640 1 Processor -- start 2023-11-26T07:22:43.625 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:43.624+0000 7f43ae5f8640 1 -- start start 2023-11-26T07:22:43.627 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:43.624+0000 7f43ae5f8640 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7f43a81614b0 0x7f43a80cd610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:22:43.627 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:43.624+0000 7f43ae5f8640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f43a8071970 0x7f43a8161890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:22:43.628 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:43.624+0000 7f43ae5f8640 1 -- --> v1:172.21.15.143:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f43a815d970 con 0x7f43a8070ba0 2023-11-26T07:22:43.628 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:43.624+0000 7f43ae5f8640 1 -- --> v2:172.21.15.143:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f43a8161dd0 con 0x7f43a81614b0 2023-11-26T07:22:43.628 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:43.624+0000 7f43ae5f8640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f43a815dc30 con 0x7f43a8071970 2023-11-26T07:22:43.628 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:43.624+0000 7f43a7fff640 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7f43a81614b0 0x7f43a80cd610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:22:43.628 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:43.624+0000 7f43a7fff640 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7f43a81614b0 0x7f43a80cd610 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:47334/0 (socket says 172.21.15.143:47334) 2023-11-26T07:22:43.628 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:43.624+0000 7f43a7fff640 1 -- 172.21.15.143:0/1140564909 learned_addr learned my addr 172.21.15.143:0/1140564909 (peer_addr_for_me v2:172.21.15.143:0/0) 2023-11-26T07:22:43.628 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:43.628+0000 7f43a77fe640 1 --2- 172.21.15.143:0/1140564909 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f43a8071970 0x7f43a8161890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:22:43.628 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:43.628+0000 7f43a7fff640 1 -- 172.21.15.143:0/1140564909 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f43a8071970 msgr2=0x7f43a8161890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-26T07:22:43.628 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:43.628+0000 7f43a7fff640 1 --2- 172.21.15.143:0/1140564909 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f43a8071970 0x7f43a8161890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:22:43.628 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:43.628+0000 7f43a7fff640 1 -- 172.21.15.143:0/1140564909 >> v1:172.21.15.143:6789/0 conn(0x7f43a8070ba0 legacy=0x7f43a806dd10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:22:43.629 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:43.628+0000 7f43a7fff640 1 -- 172.21.15.143:0/1140564909 --> v2:172.21.15.143:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f43a815df10 con 0x7f43a81614b0 2023-11-26T07:22:43.629 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:43.628+0000 7f43a6ffd640 1 -- 172.21.15.143:0/1140564909 <== mon.0 v1:172.21.15.143:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1051126035 0 0) 0x7f43a815d970 con 0x7f43a8070ba0 2023-11-26T07:22:43.630 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:43.628+0000 7f43a7fff640 1 --2- 172.21.15.143:0/1140564909 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f43a81614b0 0x7f43a80cd610 secure :-1 s=READY pgs=973 cs=0 l=1 rev1=1 crypto rx=0x7f4398009870 tx=0x7f439801c510 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ff66ddd47743b50a server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:22:43.631 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:43.628+0000 7f43a6ffd640 1 -- 172.21.15.143:0/1140564909 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f43980aa640 con 0x7f43a81614b0 2023-11-26T07:22:43.631 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:43.628+0000 7f43a6ffd640 1 -- 172.21.15.143:0/1140564909 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f43980aa7a0 con 0x7f43a81614b0 2023-11-26T07:22:43.631 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:43.628+0000 7f43a6ffd640 1 -- 172.21.15.143:0/1140564909 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f43980aab30 con 0x7f43a81614b0 2023-11-26T07:22:43.631 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:43.632+0000 7f43ae5f8640 1 -- 172.21.15.143:0/1140564909 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f43a81614b0 msgr2=0x7f43a80cd610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:22:43.631 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:43.632+0000 7f43ae5f8640 1 --2- 172.21.15.143:0/1140564909 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f43a81614b0 0x7f43a80cd610 secure :-1 s=READY pgs=973 cs=0 l=1 rev1=1 crypto rx=0x7f4398009870 tx=0x7f439801c510 comp rx=0 tx=0).stop 2023-11-26T07:22:43.632 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:43.632+0000 7f43ae5f8640 1 -- 172.21.15.143:0/1140564909 shutdown_connections 2023-11-26T07:22:43.632 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:43.632+0000 7f43ae5f8640 1 --2- 172.21.15.143:0/1140564909 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f43a8071970 0x7f43a8161890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:22:43.632 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:43.632+0000 7f43ae5f8640 1 --2- 172.21.15.143:0/1140564909 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f43a81614b0 0x7f43a80cd610 unknown :-1 s=CLOSED pgs=973 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:22:43.632 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:43.632+0000 7f43ae5f8640 1 -- 172.21.15.143:0/1140564909 >> 172.21.15.143:0/1140564909 conn(0x7f43a80cf710 msgr2=0x7f43a80cfb20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:22:43.632 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:43.632+0000 7f43ae5f8640 1 -- 172.21.15.143:0/1140564909 shutdown_connections 2023-11-26T07:22:43.633 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:43.632+0000 7f43ae5f8640 1 -- 172.21.15.143:0/1140564909 wait complete. 2023-11-26T07:22:43.633 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:43.632+0000 7f43ae5f8640 1 Processor -- start 2023-11-26T07:22:43.633 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:43.632+0000 7f43ae5f8640 1 -- start start 2023-11-26T07:22:43.635 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:43.632+0000 7f43ae5f8640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f43a8070ba0 0x7f43a80ade70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:22:43.636 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:43.632+0000 7f43ae5f8640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f43a8071970 0x7f43a8151dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:22:43.636 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:43.632+0000 7f43ae5f8640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f43a81614b0 0x7f43a8152300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:22:43.636 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:43.632+0000 7f43ae5f8640 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f43a8154fd0 con 0x7f43a81614b0 2023-11-26T07:22:43.636 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:43.632+0000 7f43ae5f8640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f43a8154cf0 con 0x7f43a8070ba0 2023-11-26T07:22:43.636 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:43.632+0000 7f43ae5f8640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f43a80ad390 con 0x7f43a8071970 2023-11-26T07:22:43.636 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:43.632+0000 7f43acb95640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f43a81614b0 0x7f43a8152300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:22:43.636 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:43.632+0000 7f43acb95640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f43a81614b0 0x7f43a8152300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:47348/0 (socket says 172.21.15.143:47348) 2023-11-26T07:22:43.636 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:43.632+0000 7f43acb95640 1 -- 172.21.15.143:0/4280451448 learned_addr learned my addr 172.21.15.143:0/4280451448 (peer_addr_for_me v2:172.21.15.143:0/0) 2023-11-26T07:22:43.636 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:43.632+0000 7f43a7fff640 1 --2- 172.21.15.143:0/4280451448 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f43a8070ba0 0x7f43a80ade70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:22:43.636 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:43.632+0000 7f43a77fe640 1 --2- 172.21.15.143:0/4280451448 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f43a8071970 0x7f43a8151dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:22:43.636 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:43.632+0000 7f43acb95640 1 -- 172.21.15.143:0/4280451448 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f43a8071970 msgr2=0x7f43a8151dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:22:43.636 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:43.632+0000 7f43acb95640 1 --2- 172.21.15.143:0/4280451448 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f43a8071970 0x7f43a8151dc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:22:43.637 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:43.632+0000 7f43acb95640 1 -- 172.21.15.143:0/4280451448 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f43a8070ba0 msgr2=0x7f43a80ade70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:22:43.637 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:43.632+0000 7f43acb95640 1 --2- 172.21.15.143:0/4280451448 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f43a8070ba0 0x7f43a80ade70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:22:43.637 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:43.632+0000 7f43acb95640 1 -- 172.21.15.143:0/4280451448 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f43980094d0 con 0x7f43a81614b0 2023-11-26T07:22:43.637 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:43.632+0000 7f43a77fe640 1 --2- 172.21.15.143:0/4280451448 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f43a8071970 0x7f43a8151dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-26T07:22:43.637 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:43.636+0000 7f43acb95640 1 --2- 172.21.15.143:0/4280451448 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f43a81614b0 0x7f43a8152300 secure :-1 s=READY pgs=974 cs=0 l=1 rev1=1 crypto rx=0x7f43a000c780 tx=0x7f43a000cc40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:22:43.637 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:43.636+0000 7f43a4ff9640 1 -- 172.21.15.143:0/4280451448 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f43a0003db0 con 0x7f43a81614b0 2023-11-26T07:22:43.637 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:43.636+0000 7f43a4ff9640 1 -- 172.21.15.143:0/4280451448 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f43a000ce30 con 0x7f43a81614b0 2023-11-26T07:22:43.637 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:43.636+0000 7f43a4ff9640 1 -- 172.21.15.143:0/4280451448 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f43a00052b0 con 0x7f43a81614b0 2023-11-26T07:22:43.637 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:43.636+0000 7f43ae5f8640 1 -- 172.21.15.143:0/4280451448 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f43a80ad670 con 0x7f43a81614b0 2023-11-26T07:22:43.637 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:43.636+0000 7f43ae5f8640 1 -- 172.21.15.143:0/4280451448 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f43a8154910 con 0x7f43a81614b0 2023-11-26T07:22:43.638 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:43.636+0000 7f43a4ff9640 1 -- 172.21.15.143:0/4280451448 <== mon.0 v2:172.21.15.143:3300/0 4 ==== mgrmap(e 11) v1 ==== 86158+0+0 (secure 0 0 0) 0x7f43a0005410 con 0x7f43a81614b0 2023-11-26T07:22:43.639 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:43.640+0000 7f43a4ff9640 1 --2- 172.21.15.143:0/4280451448 >> [v2:172.21.15.143:6832/2150557544,v1:172.21.15.143:6833/2150557544] conn(0x7f43940667d0 0x7f4394068c90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:22:43.640 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:43.640+0000 7f43a4ff9640 1 -- 172.21.15.143:0/4280451448 <== mon.0 v2:172.21.15.143:3300/0 5 ==== osd_map(71..71 src has 1..71) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f43a00842e0 con 0x7f43a81614b0 2023-11-26T07:22:43.640 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:43.640+0000 7f43a7fff640 1 --2- 172.21.15.143:0/4280451448 >> [v2:172.21.15.143:6832/2150557544,v1:172.21.15.143:6833/2150557544] conn(0x7f43940667d0 0x7f4394068c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:22:43.640 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:43.640+0000 7f43b1969000 1 Processor -- start 2023-11-26T07:22:43.640 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:43.640+0000 7f43b1969000 1 -- start start 2023-11-26T07:22:43.641 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:43.640+0000 7f43a7fff640 1 --2- 172.21.15.143:0/4280451448 >> [v2:172.21.15.143:6832/2150557544,v1:172.21.15.143:6833/2150557544] conn(0x7f43940667d0 0x7f4394068c90 secure :-1 s=READY pgs=456 cs=0 l=1 rev1=1 crypto rx=0x7f43980ae000 tx=0x7f43980a7000 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:22:43.642 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:43.640+0000 7f43b1969000 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55e1212b7c90 0x55e1212b8070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:22:43.643 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:43.640+0000 7f43b1969000 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55e1212b85b0 0x55e12126dde0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:22:43.643 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:43.640+0000 7f43b1969000 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x55e12126e320 0x55e12126e700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:22:43.643 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:43.640+0000 7f43b1969000 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e1210d8130 con 0x55e1212b85b0 2023-11-26T07:22:43.643 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:43.640+0000 7f43b1969000 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e1210c59a0 con 0x55e1212b7c90 2023-11-26T07:22:43.643 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:43.640+0000 7f43b1969000 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x55e1210c2ed0 con 0x55e12126e320 2023-11-26T07:22:43.643 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:43.640+0000 7f43a77fe640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55e1212b85b0 0x55e12126dde0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:22:43.643 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:43.640+0000 7f43a77fe640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55e1212b85b0 0x55e12126dde0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:47362/0 (socket says 172.21.15.143:47362) 2023-11-26T07:22:43.643 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:43.640+0000 7f43a77fe640 1 -- 172.21.15.143:0/4157032374 learned_addr learned my addr 172.21.15.143:0/4157032374 (peer_addr_for_me v2:172.21.15.143:0/0) 2023-11-26T07:22:43.643 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:43.640+0000 7f43a7fff640 1 --2- 172.21.15.143:0/4157032374 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55e1212b7c90 0x55e1212b8070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:22:43.644 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:43.640+0000 7f43acb95640 1 --2- 172.21.15.143:0/4157032374 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x55e12126e320 0x55e12126e700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:22:43.644 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:43.640+0000 7f43a77fe640 1 -- 172.21.15.143:0/4157032374 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x55e12126e320 msgr2=0x55e12126e700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-26T07:22:43.644 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:43.640+0000 7f43a77fe640 1 --2- 172.21.15.143:0/4157032374 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x55e12126e320 0x55e12126e700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:22:43.644 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:43.640+0000 7f43a77fe640 1 -- 172.21.15.143:0/4157032374 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55e1212b7c90 msgr2=0x55e1212b8070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-26T07:22:43.644 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:43.640+0000 7f43a77fe640 1 --2- 172.21.15.143:0/4157032374 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55e1212b7c90 0x55e1212b8070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:22:43.644 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:43.640+0000 7f43a77fe640 1 -- 172.21.15.143:0/4157032374 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55e120db6890 con 0x55e1212b85b0 2023-11-26T07:22:43.644 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:43.640+0000 7f43a77fe640 1 --2- 172.21.15.143:0/4157032374 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55e1212b85b0 0x55e12126dde0 secure :-1 s=READY pgs=975 cs=0 l=1 rev1=1 crypto rx=0x7f439c0099d0 tx=0x7f439c009e90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5c390da1da793c06 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:22:43.644 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:43.640+0000 7f438e7fc640 1 -- 172.21.15.143:0/4157032374 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f439c00c070 con 0x55e1212b85b0 2023-11-26T07:22:43.644 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:43.640+0000 7f438e7fc640 1 -- 172.21.15.143:0/4157032374 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f439c00ea70 con 0x55e1212b85b0 2023-11-26T07:22:43.644 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:43.640+0000 7f438e7fc640 1 -- 172.21.15.143:0/4157032374 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f439c00ede0 con 0x55e1212b85b0 2023-11-26T07:22:43.649 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:43.640+0000 7f43b1969000 1 -- 172.21.15.143:0/4157032374 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55e1212b85b0 msgr2=0x55e12126dde0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:22:43.649 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:43.640+0000 7f43b1969000 1 --2- 172.21.15.143:0/4157032374 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55e1212b85b0 0x55e12126dde0 secure :-1 s=READY pgs=975 cs=0 l=1 rev1=1 crypto rx=0x7f439c0099d0 tx=0x7f439c009e90 comp rx=0 tx=0).stop 2023-11-26T07:22:43.649 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:43.640+0000 7f43b1969000 1 -- 172.21.15.143:0/4157032374 shutdown_connections 2023-11-26T07:22:43.649 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:43.640+0000 7f43b1969000 1 --2- 172.21.15.143:0/4157032374 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x55e12126e320 0x55e12126e700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:22:43.649 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:43.640+0000 7f43b1969000 1 --2- 172.21.15.143:0/4157032374 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55e1212b85b0 0x55e12126dde0 unknown :-1 s=CLOSED pgs=975 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:22:43.649 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:43.640+0000 7f43b1969000 1 --2- 172.21.15.143:0/4157032374 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55e1212b7c90 0x55e1212b8070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:22:43.649 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:43.640+0000 7f43b1969000 1 -- 172.21.15.143:0/4157032374 >> 172.21.15.143:0/4157032374 conn(0x55e12126a2d0 msgr2=0x55e12126c6c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:22:43.649 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:43.640+0000 7f43b1969000 1 -- 172.21.15.143:0/4157032374 shutdown_connections 2023-11-26T07:22:43.649 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:43.644+0000 7f43b1969000 1 -- 172.21.15.143:0/4157032374 wait complete. 2023-11-26T07:22:43.649 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:43.644+0000 7f43b1969000 1 -- start start 2023-11-26T07:22:43.650 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:43.644+0000 7f43b1969000 1 Processor -- start 2023-11-26T07:22:43.650 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:43.644+0000 7f43b1969000 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x55e12126e320 0x55e121272d40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:22:43.650 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:43.644+0000 7f43b1969000 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55e1212b7c90 0x55e121273280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:22:43.650 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:43.644+0000 7f43b1969000 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55e1212b85b0 0x55e121286910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:22:43.650 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:43.644+0000 7f43b1969000 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e1210be7c0 con 0x55e1212b85b0 2023-11-26T07:22:43.650 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:43.644+0000 7f43b1969000 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e121076bf0 con 0x55e1212b7c90 2023-11-26T07:22:43.650 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:43.644+0000 7f43b1969000 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x55e12107ca90 con 0x55e12126e320 2023-11-26T07:22:43.650 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:43.644+0000 7f43b1969000 10 client.0 _ll_register_callbacks cb 0x55e1212593c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-11-26T07:22:43.650 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:43.644+0000 7f43acb95640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55e1212b85b0 0x55e121286910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:22:43.650 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:43.644+0000 7f43acb95640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55e1212b85b0 0x55e121286910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:47364/0 (socket says 172.21.15.143:47364) 2023-11-26T07:22:43.650 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:43.644+0000 7f43acb95640 1 -- 172.21.15.143:0/2189993790 learned_addr learned my addr 172.21.15.143:0/2189993790 (peer_addr_for_me v2:172.21.15.143:0/0) 2023-11-26T07:22:43.651 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:43.644+0000 7f43a7fff640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x55e12126e320 0x55e121272d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:22:43.651 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:43.644+0000 7f43a77fe640 1 --2- 172.21.15.143:0/2189993790 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55e1212b7c90 0x55e121273280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:22:43.651 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:43.644+0000 7f43acb95640 1 -- 172.21.15.143:0/2189993790 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x55e12126e320 msgr2=0x55e121272d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-26T07:22:43.651 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:43.644+0000 7f43acb95640 1 --2- 172.21.15.143:0/2189993790 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x55e12126e320 0x55e121272d40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:22:43.651 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:43.644+0000 7f43acb95640 1 -- 172.21.15.143:0/2189993790 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55e1212b7c90 msgr2=0x55e121273280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-26T07:22:43.651 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:43.644+0000 7f43acb95640 1 --2- 172.21.15.143:0/2189993790 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55e1212b7c90 0x55e121273280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:22:43.651 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:43.644+0000 7f43acb95640 1 -- 172.21.15.143:0/2189993790 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f439c009630 con 0x55e1212b85b0 2023-11-26T07:22:43.651 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:43.644+0000 7f43a77fe640 1 --2- 172.21.15.143:0/2189993790 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55e1212b7c90 0x55e121273280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-26T07:22:43.651 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:43.644+0000 7f43acb95640 1 --2- 172.21.15.143:0/2189993790 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55e1212b85b0 0x55e121286910 secure :-1 s=READY pgs=976 cs=0 l=1 rev1=1 crypto rx=0x7f43a0054ea0 tx=0x7f43a0055610 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1d40fb04b53c7e1b server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:22:43.651 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:43.644+0000 7f4386ffd640 10 client.0 ms_handle_connect on v2:172.21.15.143:3300/0 2023-11-26T07:22:43.651 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:43.644+0000 7f4386ffd640 1 -- 172.21.15.143:0/2189993790 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f43a005d210 con 0x55e1212b85b0 2023-11-26T07:22:43.651 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:43.644+0000 7f4386ffd640 1 -- 172.21.15.143:0/2189993790 <== mon.0 v2:172.21.15.143:3300/0 2 ==== osd_map(71..71 src has 1..71) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f43a00056c0 con 0x55e1212b85b0 2023-11-26T07:22:43.651 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:43.644+0000 7f43b1969000 1 -- 172.21.15.143:0/2189993790 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55e121080470 con 0x55e1212b85b0 2023-11-26T07:22:43.652 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:43.644+0000 7f4386ffd640 1 -- 172.21.15.143:0/2189993790 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_get_version_reply(handle=1 version=42) v2 ==== 24+0+0 (secure 0 0 0) 0x7f43a0075050 con 0x55e1212b85b0 2023-11-26T07:22:43.652 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:43.644+0000 7f43b1969000 10 client.7023 fetch_fsmap learned FSMap version 42 2023-11-26T07:22:43.652 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:43.644+0000 7f43b1969000 1 -- 172.21.15.143:0/2189993790 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,fsmap=42,monmap=2+}) v3 -- 0x55e12107f190 con 0x55e1212b85b0 2023-11-26T07:22:43.652 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:43.644+0000 7f4386ffd640 1 -- 172.21.15.143:0/2189993790 <== mon.0 v2:172.21.15.143:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x55e121080470 con 0x55e1212b85b0 2023-11-26T07:22:43.652 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:43.644+0000 7f4386ffd640 1 -- 172.21.15.143:0/2189993790 <== mon.0 v2:172.21.15.143:3300/0 5 ==== fsmap(e 42) v1 ==== 3294+0+0 (secure 0 0 0) 0x7f43a0058070 con 0x55e1212b85b0 2023-11-26T07:22:43.652 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:43.644+0000 7f43b1969000 10 client.7023 fetch_fsmap finished waiting for FSMap version 42 2023-11-26T07:22:43.652 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:43.644+0000 7f43b1969000 10 client.7023 resolve_mds: resolved name 'a' to daemon mds.a 2023-11-26T07:22:43.652 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:43.644+0000 7f43b1969000 20 client.7023 populate_metadata read hostname 'smithi143' 2023-11-26T07:22:43.652 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:43.644+0000 7f43b1969000 1 --2- 172.21.15.143:0/2189993790 >> [v2:172.21.15.143:6834/2590663349,v1:172.21.15.143:6836/2590663349] conn(0x55e121287320 0x55e1212c9f50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:22:43.652 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:43.644+0000 7f43b1969000 4 client.7023 mds_command: new command op to 6553 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2023-11-26T07:22:43.652 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:43.644+0000 7f43b1969000 1 -- 172.21.15.143:0/2189993790 --> [v2:172.21.15.143:6834/2590663349,v1:172.21.15.143:6836/2590663349] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55e121083b50 con 0x55e121287320 2023-11-26T07:22:43.653 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:43.644+0000 7f43a7fff640 1 --2- 172.21.15.143:0/2189993790 >> [v2:172.21.15.143:6834/2590663349,v1:172.21.15.143:6836/2590663349] conn(0x55e121287320 0x55e1212c9f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:22:43.653 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:43.648+0000 7f43a7fff640 1 --2- 172.21.15.143:0/2189993790 >> [v2:172.21.15.143:6834/2590663349,v1:172.21.15.143:6836/2590663349] conn(0x55e121287320 0x55e1212c9f50 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=657ba08298e0dd90 server_cookie=fa5afeea6ec38239 in_seq=0 out_seq=0 2023-11-26T07:22:43.653 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:43.648+0000 7f4386ffd640 10 client.7023 ms_handle_connect on v2:172.21.15.143:6834/2590663349 2023-11-26T07:22:43.653 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:43.648+0000 7f4386ffd640 1 -- 172.21.15.143:0/2189993790 <== mds.0 v2:172.21.15.143:6834/2590663349 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x55e121083b50 con 0x55e121287320 2023-11-26T07:22:43.653 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:43.648+0000 7f4386ffd640 10 client.7023 handle_command_reply: tid=0 2023-11-26T07:22:43.653 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:43.648+0000 7f43b1969000 1 client.7023 shutdown 2023-11-26T07:22:43.653 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:43.648+0000 7f43b1969000 1 -- 172.21.15.143:0/2189993790 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55e1212b85b0 msgr2=0x55e121286910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:22:43.653 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:43.648+0000 7f43b1969000 1 --2- 172.21.15.143:0/2189993790 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55e1212b85b0 0x55e121286910 secure :-1 s=READY pgs=976 cs=0 l=1 rev1=1 crypto rx=0x7f43a0054ea0 tx=0x7f43a0055610 comp rx=0 tx=0).stop 2023-11-26T07:22:43.653 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:43.648+0000 7f43b1969000 1 -- 172.21.15.143:0/2189993790 shutdown_connections 2023-11-26T07:22:43.653 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:43.648+0000 7f43b1969000 1 --2- 172.21.15.143:0/2189993790 >> [v2:172.21.15.143:6834/2590663349,v1:172.21.15.143:6836/2590663349] conn(0x55e121287320 0x55e1212c9f50 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:22:43.653 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:43.648+0000 7f43b1969000 1 --2- 172.21.15.143:0/2189993790 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55e1212b85b0 0x55e121286910 unknown :-1 s=CLOSED pgs=976 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:22:43.653 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:43.648+0000 7f43b1969000 1 --2- 172.21.15.143:0/2189993790 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55e1212b7c90 0x55e121273280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:22:43.653 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:43.648+0000 7f43b1969000 1 --2- 172.21.15.143:0/2189993790 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x55e12126e320 0x55e121272d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:22:43.654 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:43.648+0000 7f43b1969000 1 -- 172.21.15.143:0/2189993790 >> 172.21.15.143:0/2189993790 conn(0x55e12126a2d0 msgr2=0x55e12126c6c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:22:43.654 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:43.648+0000 7f4386ffd640 0 client.7023 ms_handle_reset on v2:172.21.15.143:6834/2590663349 2023-11-26T07:22:43.654 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:43.648+0000 7f43b1969000 1 -- 172.21.15.143:0/2189993790 shutdown_connections 2023-11-26T07:22:43.654 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:43.648+0000 7f43b1969000 1 -- 172.21.15.143:0/2189993790 wait complete. 2023-11-26T07:22:43.654 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:43.648+0000 7f43b1969000 20 client.7023 trim_cache size 0 max 16384 2023-11-26T07:22:43.664 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:43.664+0000 7f43b1969000 1 Processor -- start 2023-11-26T07:22:43.664 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:43.664+0000 7f43b1969000 1 -- start start 2023-11-26T07:22:43.666 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:43.664+0000 7f43b1969000 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55e12126e320 0x55e1212b6a80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:22:43.666 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:43.664+0000 7f43b1969000 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55e1212b7c90 0x55e1212b5b30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:22:43.666 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:43.664+0000 7f43b1969000 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x55e1212b85b0 0x55e1212b6070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:22:43.666 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:43.664+0000 7f43b1969000 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e120d77940 con 0x55e1212b7c90 2023-11-26T07:22:43.666 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:43.664+0000 7f43b1969000 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e1212868c0 con 0x55e12126e320 2023-11-26T07:22:43.666 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:43.664+0000 7f43b1969000 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x55e1210bbbc0 con 0x55e1212b85b0 2023-11-26T07:22:43.666 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:43.664+0000 7f43a77fe640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55e1212b7c90 0x55e1212b5b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:22:43.666 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:43.664+0000 7f43a77fe640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55e1212b7c90 0x55e1212b5b30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:47380/0 (socket says 172.21.15.143:47380) 2023-11-26T07:22:43.666 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:43.664+0000 7f43a77fe640 1 -- 172.21.15.143:0/3740956714 learned_addr learned my addr 172.21.15.143:0/3740956714 (peer_addr_for_me v2:172.21.15.143:0/0) 2023-11-26T07:22:43.666 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:43.664+0000 7f43a77fe640 1 -- 172.21.15.143:0/3740956714 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x55e1212b85b0 msgr2=0x55e1212b6070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-26T07:22:43.666 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:43.664+0000 7f43a7fff640 1 --2- 172.21.15.143:0/3740956714 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55e12126e320 0x55e1212b6a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:22:43.667 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:43.664+0000 7f43acb95640 1 --2- 172.21.15.143:0/3740956714 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x55e1212b85b0 0x55e1212b6070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:22:43.667 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:43.664+0000 7f43a77fe640 1 --2- 172.21.15.143:0/3740956714 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x55e1212b85b0 0x55e1212b6070 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:22:43.667 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:43.664+0000 7f43a77fe640 1 -- 172.21.15.143:0/3740956714 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55e12126e320 msgr2=0x55e1212b6a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-26T07:22:43.667 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:43.664+0000 7f43a77fe640 1 --2- 172.21.15.143:0/3740956714 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55e12126e320 0x55e1212b6a80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:22:43.667 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:43.664+0000 7f43a77fe640 1 -- 172.21.15.143:0/3740956714 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55e121065490 con 0x55e1212b7c90 2023-11-26T07:22:43.667 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:43.664+0000 7f43a7fff640 1 --2- 172.21.15.143:0/3740956714 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55e12126e320 0x55e1212b6a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-26T07:22:43.667 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:43.664+0000 7f43a77fe640 1 --2- 172.21.15.143:0/3740956714 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55e1212b7c90 0x55e1212b5b30 secure :-1 s=READY pgs=977 cs=0 l=1 rev1=1 crypto rx=0x7f439c000c10 tx=0x7f439c00c000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4c65a72599ebd64d server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:22:43.670 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:43.664+0000 7f438effd640 1 -- 172.21.15.143:0/3740956714 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f439c021040 con 0x55e1212b7c90 2023-11-26T07:22:43.670 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:43.664+0000 7f438effd640 1 -- 172.21.15.143:0/3740956714 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f439c004260 con 0x55e1212b7c90 2023-11-26T07:22:43.670 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:43.664+0000 7f438effd640 1 -- 172.21.15.143:0/3740956714 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f439c009120 con 0x55e1212b7c90 2023-11-26T07:22:43.670 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:43.664+0000 7f43b1969000 1 -- 172.21.15.143:0/3740956714 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55e1212b7c90 msgr2=0x55e1212b5b30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:22:43.671 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:43.664+0000 7f43b1969000 1 --2- 172.21.15.143:0/3740956714 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55e1212b7c90 0x55e1212b5b30 secure :-1 s=READY pgs=977 cs=0 l=1 rev1=1 crypto rx=0x7f439c000c10 tx=0x7f439c00c000 comp rx=0 tx=0).stop 2023-11-26T07:22:43.671 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:43.664+0000 7f43b1969000 1 -- 172.21.15.143:0/3740956714 shutdown_connections 2023-11-26T07:22:43.671 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:43.664+0000 7f43b1969000 1 --2- 172.21.15.143:0/3740956714 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x55e1212b85b0 0x55e1212b6070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:22:43.671 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:43.664+0000 7f43b1969000 1 --2- 172.21.15.143:0/3740956714 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55e1212b7c90 0x55e1212b5b30 unknown :-1 s=CLOSED pgs=977 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:22:43.671 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:43.664+0000 7f43b1969000 1 --2- 172.21.15.143:0/3740956714 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55e12126e320 0x55e1212b6a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:22:43.671 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:43.664+0000 7f43b1969000 1 -- 172.21.15.143:0/3740956714 >> 172.21.15.143:0/3740956714 conn(0x55e12126a2d0 msgr2=0x55e12126c6c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:22:43.671 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:43.664+0000 7f43b1969000 1 -- 172.21.15.143:0/3740956714 shutdown_connections 2023-11-26T07:22:43.671 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:43.664+0000 7f43b1969000 1 -- 172.21.15.143:0/3740956714 wait complete. 2023-11-26T07:22:43.671 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:43.668+0000 7f43b1969000 1 -- start start 2023-11-26T07:22:43.671 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:43.668+0000 7f43b1969000 1 Processor -- start 2023-11-26T07:22:43.671 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:43.668+0000 7f43b1969000 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55e12126e320 0x55e12129a7e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:22:43.671 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:43.668+0000 7f43b1969000 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x55e1212b7c90 0x55e12129ad20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:22:43.671 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:43.668+0000 7f43b1969000 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55e1212b85b0 0x55e1212c9f50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:22:43.672 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:43.668+0000 7f43b1969000 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e121275890 con 0x55e12126e320 2023-11-26T07:22:43.672 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:43.668+0000 7f43b1969000 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e121086a60 con 0x55e1212b85b0 2023-11-26T07:22:43.672 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:43.668+0000 7f43b1969000 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x55e12107d380 con 0x55e1212b7c90 2023-11-26T07:22:43.672 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:43.668+0000 7f43b1969000 10 client.0 _ll_register_callbacks cb 0x55e12110cc00 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-11-26T07:22:43.672 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:43.668+0000 7f43a7fff640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55e12126e320 0x55e12129a7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:22:43.672 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:43.668+0000 7f43a7fff640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55e12126e320 0x55e12129a7e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:47388/0 (socket says 172.21.15.143:47388) 2023-11-26T07:22:43.672 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:43.668+0000 7f43a7fff640 1 -- 172.21.15.143:0/284281534 learned_addr learned my addr 172.21.15.143:0/284281534 (peer_addr_for_me v2:172.21.15.143:0/0) 2023-11-26T07:22:43.672 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:43.668+0000 7f43a77fe640 1 --2- 172.21.15.143:0/284281534 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x55e1212b7c90 0x55e12129ad20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:22:43.672 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:43.668+0000 7f43acb95640 1 --2- 172.21.15.143:0/284281534 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55e1212b85b0 0x55e1212c9f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:22:43.672 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:43.668+0000 7f43a7fff640 1 -- 172.21.15.143:0/284281534 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x55e1212b7c90 msgr2=0x55e12129ad20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-26T07:22:43.672 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:43.668+0000 7f43a7fff640 1 --2- 172.21.15.143:0/284281534 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x55e1212b7c90 0x55e12129ad20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:22:43.672 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:43.668+0000 7f43a7fff640 1 -- 172.21.15.143:0/284281534 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55e1212b85b0 msgr2=0x55e1212c9f50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-26T07:22:43.673 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:43.668+0000 7f43a7fff640 1 --2- 172.21.15.143:0/284281534 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55e1212b85b0 0x55e1212c9f50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:22:43.673 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:43.668+0000 7f43a7fff640 1 -- 172.21.15.143:0/284281534 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55e120db6890 con 0x55e12126e320 2023-11-26T07:22:43.673 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:43.668+0000 7f43a7fff640 1 --2- 172.21.15.143:0/284281534 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55e12126e320 0x55e12129a7e0 secure :-1 s=READY pgs=978 cs=0 l=1 rev1=1 crypto rx=0x7f4398000c10 tx=0x7f439801ad50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f49d243e9efc429d server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:22:43.673 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:43.668+0000 7f4386ffd640 10 client.0 ms_handle_connect on v2:172.21.15.143:3300/0 2023-11-26T07:22:43.673 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:43.668+0000 7f4386ffd640 1 -- 172.21.15.143:0/284281534 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f43980c5040 con 0x55e12126e320 2023-11-26T07:22:43.673 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:43.668+0000 7f4386ffd640 1 -- 172.21.15.143:0/284281534 <== mon.0 v2:172.21.15.143:3300/0 2 ==== osd_map(71..71 src has 1..71) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f43980aad90 con 0x55e12126e320 2023-11-26T07:22:43.673 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:43.668+0000 7f43b1969000 1 -- 172.21.15.143:0/284281534 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55e1210bd110 con 0x55e12126e320 2023-11-26T07:22:43.673 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:43.668+0000 7f4386ffd640 1 -- 172.21.15.143:0/284281534 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_get_version_reply(handle=1 version=42) v2 ==== 24+0+0 (secure 0 0 0) 0x7f43980cd070 con 0x55e12126e320 2023-11-26T07:22:43.673 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:43.668+0000 7f43b1969000 10 client.7029 fetch_fsmap learned FSMap version 42 2023-11-26T07:22:43.673 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:43.668+0000 7f43b1969000 1 -- 172.21.15.143:0/284281534 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,fsmap=42,monmap=2+}) v3 -- 0x55e120db6cc0 con 0x55e12126e320 2023-11-26T07:22:43.673 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:43.668+0000 7f4386ffd640 1 -- 172.21.15.143:0/284281534 <== mon.0 v2:172.21.15.143:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x55e1210bd110 con 0x55e12126e320 2023-11-26T07:22:43.674 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:43.668+0000 7f4386ffd640 1 -- 172.21.15.143:0/284281534 <== mon.0 v2:172.21.15.143:3300/0 5 ==== fsmap(e 42) v1 ==== 3294+0+0 (secure 0 0 0) 0x7f43980cd1d0 con 0x55e12126e320 2023-11-26T07:22:43.674 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:43.668+0000 7f43b1969000 10 client.7029 fetch_fsmap finished waiting for FSMap version 42 2023-11-26T07:22:43.674 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:43.668+0000 7f43b1969000 10 client.7029 resolve_mds: resolved name 'a' to daemon mds.a 2023-11-26T07:22:43.674 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:43.668+0000 7f43b1969000 20 client.7029 populate_metadata read hostname 'smithi143' 2023-11-26T07:22:43.674 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:43.668+0000 7f43b1969000 1 --2- 172.21.15.143:0/284281534 >> [v2:172.21.15.143:6834/2590663349,v1:172.21.15.143:6836/2590663349] conn(0x55e1212ca490 0x55e1212ca870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:22:43.674 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:43.668+0000 7f43b1969000 4 client.7029 mds_command: new command op to 6553 tid=0 multi_id=0 [{"prefix": "status"}] 2023-11-26T07:22:43.674 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:43.668+0000 7f43b1969000 1 -- 172.21.15.143:0/284281534 --> [v2:172.21.15.143:6834/2590663349,v1:172.21.15.143:6836/2590663349] -- command(tid 0: {"prefix": "status"}) v1 -- 0x55e1210ec9a0 con 0x55e1212ca490 2023-11-26T07:22:43.674 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:43.668+0000 7f43a77fe640 1 --2- 172.21.15.143:0/284281534 >> [v2:172.21.15.143:6834/2590663349,v1:172.21.15.143:6836/2590663349] conn(0x55e1212ca490 0x55e1212ca870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:22:43.675 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:43.668+0000 7f4386ffd640 10 client.7029 ms_handle_connect on v2:172.21.15.143:6834/2590663349 2023-11-26T07:22:43.676 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:43.668+0000 7f43a77fe640 1 --2- 172.21.15.143:0/284281534 >> [v2:172.21.15.143:6834/2590663349,v1:172.21.15.143:6836/2590663349] conn(0x55e1212ca490 0x55e1212ca870 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=8d698dc0640ed7be server_cookie=1526d8cffe2d7379 in_seq=0 out_seq=0 2023-11-26T07:22:43.676 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:43.672+0000 7f4386ffd640 1 -- 172.21.15.143:0/284281534 <== mds.0 v2:172.21.15.143:6834/2590663349 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+331 (crc 0 0 0) 0x55e1210ec9a0 con 0x55e1212ca490 2023-11-26T07:22:43.676 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:43.672+0000 7f4386ffd640 10 client.7029 handle_command_reply: tid=0 2023-11-26T07:22:43.676 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:43.672+0000 7f43b1969000 1 client.7029 shutdown 2023-11-26T07:22:43.676 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:43.672+0000 7f43b1969000 1 -- 172.21.15.143:0/284281534 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55e12126e320 msgr2=0x55e12129a7e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:22:43.676 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:43.672+0000 7f43b1969000 1 --2- 172.21.15.143:0/284281534 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55e12126e320 0x55e12129a7e0 secure :-1 s=READY pgs=978 cs=0 l=1 rev1=1 crypto rx=0x7f4398000c10 tx=0x7f439801ad50 comp rx=0 tx=0).stop 2023-11-26T07:22:43.676 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:43.672+0000 7f43b1969000 1 -- 172.21.15.143:0/284281534 shutdown_connections 2023-11-26T07:22:43.676 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:43.672+0000 7f43b1969000 1 --2- 172.21.15.143:0/284281534 >> [v2:172.21.15.143:6834/2590663349,v1:172.21.15.143:6836/2590663349] conn(0x55e1212ca490 0x55e1212ca870 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:22:43.676 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:43.672+0000 7f43b1969000 1 --2- 172.21.15.143:0/284281534 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55e1212b85b0 0x55e1212c9f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:22:43.676 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:43.672+0000 7f43b1969000 1 --2- 172.21.15.143:0/284281534 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x55e1212b7c90 0x55e12129ad20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:22:43.676 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:43.672+0000 7f4386ffd640 0 client.7029 ms_handle_reset on v2:172.21.15.143:6834/2590663349 2023-11-26T07:22:43.677 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:43.672+0000 7f43b1969000 1 --2- 172.21.15.143:0/284281534 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55e12126e320 0x55e12129a7e0 unknown :-1 s=CLOSED pgs=978 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:22:43.677 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:43.672+0000 7f43b1969000 1 -- 172.21.15.143:0/284281534 >> 172.21.15.143:0/284281534 conn(0x55e12126a2d0 msgr2=0x55e1212c50f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:22:43.677 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:43.672+0000 7f43b1969000 1 -- 172.21.15.143:0/284281534 shutdown_connections 2023-11-26T07:22:43.677 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:43.672+0000 7f43b1969000 1 -- 172.21.15.143:0/284281534 wait complete. 2023-11-26T07:22:43.677 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:43.672+0000 7f43b1969000 20 client.7029 trim_cache size 0 max 16384 2023-11-26T07:22:43.677 INFO:teuthology.orchestra.run.smithi143.stdout:{ 2023-11-26T07:22:43.677 INFO:teuthology.orchestra.run.smithi143.stdout: "cluster_fsid": "f5033060-2469-4a07-95fe-2b76f5404bc1", 2023-11-26T07:22:43.677 INFO:teuthology.orchestra.run.smithi143.stdout: "whoami": 0, 2023-11-26T07:22:43.677 INFO:teuthology.orchestra.run.smithi143.stdout: "id": 6553, 2023-11-26T07:22:43.678 INFO:teuthology.orchestra.run.smithi143.stdout: "want_state": "up:active", 2023-11-26T07:22:43.678 INFO:teuthology.orchestra.run.smithi143.stdout: "state": "up:active", 2023-11-26T07:22:43.678 INFO:teuthology.orchestra.run.smithi143.stdout: "fs_name": "cephfs", 2023-11-26T07:22:43.678 INFO:teuthology.orchestra.run.smithi143.stdout: "rank_uptime": 5.8086180140000003, 2023-11-26T07:22:43.678 INFO:teuthology.orchestra.run.smithi143.stdout: "mdsmap_epoch": 42, 2023-11-26T07:22:43.678 INFO:teuthology.orchestra.run.smithi143.stdout: "osdmap_epoch": 71, 2023-11-26T07:22:43.678 INFO:teuthology.orchestra.run.smithi143.stdout: "osdmap_epoch_barrier": 71, 2023-11-26T07:22:43.678 INFO:teuthology.orchestra.run.smithi143.stdout: "uptime": 11.757969105999999 2023-11-26T07:22:43.678 INFO:teuthology.orchestra.run.smithi143.stdout:} 2023-11-26T07:22:43.679 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:43.672+0000 7f43ae5f8640 1 -- 172.21.15.143:0/4280451448 >> [v2:172.21.15.143:6832/2150557544,v1:172.21.15.143:6833/2150557544] conn(0x7f43940667d0 msgr2=0x7f4394068c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:22:43.679 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:43.672+0000 7f43ae5f8640 1 --2- 172.21.15.143:0/4280451448 >> [v2:172.21.15.143:6832/2150557544,v1:172.21.15.143:6833/2150557544] conn(0x7f43940667d0 0x7f4394068c90 secure :-1 s=READY pgs=456 cs=0 l=1 rev1=1 crypto rx=0x7f43980ae000 tx=0x7f43980a7000 comp rx=0 tx=0).stop 2023-11-26T07:22:43.679 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:43.672+0000 7f43ae5f8640 1 -- 172.21.15.143:0/4280451448 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f43a81614b0 msgr2=0x7f43a8152300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:22:43.679 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:43.672+0000 7f43ae5f8640 1 --2- 172.21.15.143:0/4280451448 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f43a81614b0 0x7f43a8152300 secure :-1 s=READY pgs=974 cs=0 l=1 rev1=1 crypto rx=0x7f43a000c780 tx=0x7f43a000cc40 comp rx=0 tx=0).stop 2023-11-26T07:22:43.679 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:43.672+0000 7f43ae5f8640 1 -- 172.21.15.143:0/4280451448 shutdown_connections 2023-11-26T07:22:43.679 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:43.672+0000 7f43ae5f8640 1 --2- 172.21.15.143:0/4280451448 >> [v2:172.21.15.143:6832/2150557544,v1:172.21.15.143:6833/2150557544] conn(0x7f43940667d0 0x7f4394068c90 unknown :-1 s=CLOSED pgs=456 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:22:43.679 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:43.672+0000 7f43ae5f8640 1 --2- 172.21.15.143:0/4280451448 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f43a81614b0 0x7f43a8152300 unknown :-1 s=CLOSED pgs=974 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:22:43.679 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:43.672+0000 7f43ae5f8640 1 --2- 172.21.15.143:0/4280451448 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f43a8071970 0x7f43a8151dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:22:43.680 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:43.672+0000 7f43ae5f8640 1 --2- 172.21.15.143:0/4280451448 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f43a8070ba0 0x7f43a80ade70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:22:43.680 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:43.672+0000 7f43ae5f8640 1 -- 172.21.15.143:0/4280451448 >> 172.21.15.143:0/4280451448 conn(0x7f43a80cf710 msgr2=0x7f43a806b940 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:22:43.680 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:43.672+0000 7f43ae5f8640 1 -- 172.21.15.143:0/4280451448 shutdown_connections 2023-11-26T07:22:43.680 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:43.672+0000 7f43ae5f8640 1 -- 172.21.15.143:0/4280451448 wait complete. 2023-11-26T07:22:43.691 DEBUG:teuthology.orchestra.run.smithi143:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2023-11-26T07:22:43.829 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:43.828+0000 7fe2f7c06640 1 Processor -- start 2023-11-26T07:22:43.832 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:43.828+0000 7fe2f7c06640 1 -- start start 2023-11-26T07:22:43.832 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:43.828+0000 7fe2f7c06640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fe2f015d6e0 0x7fe2f00d0df0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:22:43.832 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:43.828+0000 7fe2f7c06640 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7fe2f00d1bc0 0x7fe2f00b0750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:22:43.832 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:43.828+0000 7fe2f7c06640 1 -- --> v1:172.21.15.143:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe2f00cf0e0 con 0x7fe2f00b2160 2023-11-26T07:22:43.833 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:43.828+0000 7fe2f7c06640 1 -- --> v2:172.21.15.143:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe2f015dac0 con 0x7fe2f00d1bc0 2023-11-26T07:22:43.833 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:43.828+0000 7fe2f7c06640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe2f00ceb20 con 0x7fe2f015d6e0 2023-11-26T07:22:43.833 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:43.828+0000 7fe2f61a3640 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7fe2f00d1bc0 0x7fe2f00b0750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:22:43.833 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:43.828+0000 7fe2f61a3640 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7fe2f00d1bc0 0x7fe2f00b0750 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:47390/0 (socket says 172.21.15.143:47390) 2023-11-26T07:22:43.833 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:43.828+0000 7fe2f59a2640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fe2f015d6e0 0x7fe2f00d0df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:22:43.833 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:43.828+0000 7fe2f61a3640 1 -- 172.21.15.143:0/3561378406 learned_addr learned my addr 172.21.15.143:0/3561378406 (peer_addr_for_me v2:172.21.15.143:0/0) 2023-11-26T07:22:43.833 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:43.828+0000 7fe2f51a1640 1 --1- 172.21.15.143:0/3561378406 >> v1:172.21.15.143:6789/0 conn(0x7fe2f00b2160 0x7fe2f00d14b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.143:6789/0 says I am v1:172.21.15.143:35830/0 (socket says 172.21.15.143:35830) 2023-11-26T07:22:43.833 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:43.828+0000 7fe2f61a3640 1 -- 172.21.15.143:0/3561378406 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fe2f015d6e0 msgr2=0x7fe2f00d0df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-26T07:22:43.833 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:43.828+0000 7fe2f61a3640 1 --2- 172.21.15.143:0/3561378406 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fe2f015d6e0 0x7fe2f00d0df0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:22:43.834 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:43.828+0000 7fe2f61a3640 1 -- 172.21.15.143:0/3561378406 >> v1:172.21.15.143:6789/0 conn(0x7fe2f00b2160 legacy=0x7fe2f00d14b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:22:43.834 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:43.828+0000 7fe2f61a3640 1 -- 172.21.15.143:0/3561378406 --> v2:172.21.15.143:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe2f00cee00 con 0x7fe2f00d1bc0 2023-11-26T07:22:43.835 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:43.832+0000 7fe2f61a3640 1 --2- 172.21.15.143:0/3561378406 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fe2f00d1bc0 0x7fe2f00b0750 secure :-1 s=READY pgs=980 cs=0 l=1 rev1=1 crypto rx=0x7fe2e8009e50 tx=0x7fe2e801ca10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=30e89801290528c1 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:22:43.835 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:43.832+0000 7fe2f49a0640 1 -- 172.21.15.143:0/3561378406 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe2e80a57d0 con 0x7fe2f00d1bc0 2023-11-26T07:22:43.835 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:43.832+0000 7fe2f49a0640 1 -- 172.21.15.143:0/3561378406 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe2e80a5930 con 0x7fe2f00d1bc0 2023-11-26T07:22:43.835 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:43.832+0000 7fe2f49a0640 1 -- 172.21.15.143:0/3561378406 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe2e80a5cc0 con 0x7fe2f00d1bc0 2023-11-26T07:22:43.835 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:43.832+0000 7fe2f7c06640 1 -- 172.21.15.143:0/3561378406 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fe2f00d1bc0 msgr2=0x7fe2f00b0750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:22:43.835 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:43.832+0000 7fe2f7c06640 1 --2- 172.21.15.143:0/3561378406 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fe2f00d1bc0 0x7fe2f00b0750 secure :-1 s=READY pgs=980 cs=0 l=1 rev1=1 crypto rx=0x7fe2e8009e50 tx=0x7fe2e801ca10 comp rx=0 tx=0).stop 2023-11-26T07:22:43.835 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:43.832+0000 7fe2f7c06640 1 -- 172.21.15.143:0/3561378406 shutdown_connections 2023-11-26T07:22:43.835 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:43.832+0000 7fe2f7c06640 1 --2- 172.21.15.143:0/3561378406 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fe2f00d1bc0 0x7fe2f00b0750 unknown :-1 s=CLOSED pgs=980 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:22:43.835 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:43.832+0000 7fe2f7c06640 1 --2- 172.21.15.143:0/3561378406 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fe2f015d6e0 0x7fe2f00d0df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:22:43.835 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:43.832+0000 7fe2f7c06640 1 -- 172.21.15.143:0/3561378406 >> 172.21.15.143:0/3561378406 conn(0x7fe2f009c550 msgr2=0x7fe2f00d3220 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:22:43.836 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:43.832+0000 7fe2f7c06640 1 -- 172.21.15.143:0/3561378406 shutdown_connections 2023-11-26T07:22:43.836 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:43.832+0000 7fe2f7c06640 1 -- 172.21.15.143:0/3561378406 wait complete. 2023-11-26T07:22:43.836 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:43.836+0000 7fe2f7c06640 1 Processor -- start 2023-11-26T07:22:43.837 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:43.836+0000 7fe2f7c06640 1 -- start start 2023-11-26T07:22:43.838 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:43.836+0000 7fe2f7c06640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fe2f00b2160 0x7fe2f00cc870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:22:43.838 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:43.836+0000 7fe2f7c06640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fe2f00d1bc0 0x7fe2f015dcb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:22:43.838 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:43.836+0000 7fe2f7c06640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fe2f015d6e0 0x7fe2f015e1f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:22:43.838 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:43.836+0000 7fe2f7c06640 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe2f00cb7d0 con 0x7fe2f00d1bc0 2023-11-26T07:22:43.838 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:43.836+0000 7fe2f7c06640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe2f00cbab0 con 0x7fe2f015d6e0 2023-11-26T07:22:43.838 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:43.836+0000 7fe2f7c06640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe2f00cbd90 con 0x7fe2f00b2160 2023-11-26T07:22:43.838 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:43.836+0000 7fe2f51a1640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fe2f00d1bc0 0x7fe2f015dcb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:22:43.838 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:43.836+0000 7fe2f51a1640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fe2f00d1bc0 0x7fe2f015dcb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:47396/0 (socket says 172.21.15.143:47396) 2023-11-26T07:22:43.839 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:43.836+0000 7fe2f51a1640 1 -- 172.21.15.143:0/742147469 learned_addr learned my addr 172.21.15.143:0/742147469 (peer_addr_for_me v2:172.21.15.143:0/0) 2023-11-26T07:22:43.840 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:43.836+0000 7fe2f61a3640 1 --2- 172.21.15.143:0/742147469 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fe2f015d6e0 0x7fe2f015e1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:22:43.840 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:43.836+0000 7fe2f51a1640 1 -- 172.21.15.143:0/742147469 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fe2f00b2160 msgr2=0x7fe2f00cc870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:22:43.840 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:43.836+0000 7fe2f59a2640 1 --2- 172.21.15.143:0/742147469 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fe2f00b2160 0x7fe2f00cc870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:22:43.840 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:43.836+0000 7fe2f51a1640 1 --2- 172.21.15.143:0/742147469 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fe2f00b2160 0x7fe2f00cc870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:22:43.840 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:43.836+0000 7fe2f51a1640 1 -- 172.21.15.143:0/742147469 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fe2f015d6e0 msgr2=0x7fe2f015e1f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:22:43.840 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:43.836+0000 7fe2f51a1640 1 --2- 172.21.15.143:0/742147469 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fe2f015d6e0 0x7fe2f015e1f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:22:43.840 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:43.836+0000 7fe2f51a1640 1 -- 172.21.15.143:0/742147469 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe2e8009b30 con 0x7fe2f00d1bc0 2023-11-26T07:22:43.840 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:43.836+0000 7fe2f61a3640 1 --2- 172.21.15.143:0/742147469 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fe2f015d6e0 0x7fe2f015e1f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-26T07:22:43.840 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:43.836+0000 7fe2f59a2640 1 --2- 172.21.15.143:0/742147469 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fe2f00b2160 0x7fe2f00cc870 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-26T07:22:43.840 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:43.836+0000 7fe2f51a1640 1 --2- 172.21.15.143:0/742147469 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fe2f00d1bc0 0x7fe2f015dcb0 secure :-1 s=READY pgs=981 cs=0 l=1 rev1=1 crypto rx=0x7fe2e000c490 tx=0x7fe2e000c950 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:22:43.841 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:43.836+0000 7fe2e67fc640 1 -- 172.21.15.143:0/742147469 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe2e0007480 con 0x7fe2f00d1bc0 2023-11-26T07:22:43.841 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:43.836+0000 7fe2e67fc640 1 -- 172.21.15.143:0/742147469 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe2e00075e0 con 0x7fe2f00d1bc0 2023-11-26T07:22:43.841 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:43.836+0000 7fe2e67fc640 1 -- 172.21.15.143:0/742147469 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe2e0004a10 con 0x7fe2f00d1bc0 2023-11-26T07:22:43.841 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:43.836+0000 7fe2f7c06640 1 -- 172.21.15.143:0/742147469 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe2f00cc070 con 0x7fe2f00d1bc0 2023-11-26T07:22:43.841 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:43.836+0000 7fe2f7c06640 1 -- 172.21.15.143:0/742147469 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe2f0155730 con 0x7fe2f00d1bc0 2023-11-26T07:22:43.841 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:43.840+0000 7fe2f7c06640 1 -- 172.21.15.143:0/742147469 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe2f0067680 con 0x7fe2f00d1bc0 2023-11-26T07:22:43.842 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:43.840+0000 7fe2e67fc640 1 -- 172.21.15.143:0/742147469 <== mon.0 v2:172.21.15.143:3300/0 4 ==== mgrmap(e 11) v1 ==== 86158+0+0 (secure 0 0 0) 0x7fe2e0004b70 con 0x7fe2f00d1bc0 2023-11-26T07:22:43.842 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:43.840+0000 7fe2e67fc640 1 --2- 172.21.15.143:0/742147469 >> [v2:172.21.15.143:6832/2150557544,v1:172.21.15.143:6833/2150557544] conn(0x7fe2d40667d0 0x7fe2d4068c90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:22:43.842 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:43.840+0000 7fe2e67fc640 1 -- 172.21.15.143:0/742147469 <== mon.0 v2:172.21.15.143:3300/0 5 ==== osd_map(71..71 src has 1..71) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fe2e0083570 con 0x7fe2f00d1bc0 2023-11-26T07:22:43.842 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:43.840+0000 7fe2f59a2640 1 --2- 172.21.15.143:0/742147469 >> [v2:172.21.15.143:6832/2150557544,v1:172.21.15.143:6833/2150557544] conn(0x7fe2d40667d0 0x7fe2d4068c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:22:43.842 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:43.840+0000 7fe2f59a2640 1 --2- 172.21.15.143:0/742147469 >> [v2:172.21.15.143:6832/2150557544,v1:172.21.15.143:6833/2150557544] conn(0x7fe2d40667d0 0x7fe2d4068c90 secure :-1 s=READY pgs=457 cs=0 l=1 rev1=1 crypto rx=0x7fe2f00cd3f0 tx=0x7fe2ec008040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:22:43.846 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:43.844+0000 7fe2e67fc640 1 -- 172.21.15.143:0/742147469 <== mon.0 v2:172.21.15.143:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172649 (secure 0 0 0) 0x7fe2e0050630 con 0x7fe2f00d1bc0 2023-11-26T07:22:44.030 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:44.028+0000 7fe2f7c06640 1 -- 172.21.15.143:0/742147469 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "fs ls"} v 0) v1 -- 0x7fe2f00cc520 con 0x7fe2f00d1bc0 2023-11-26T07:22:44.030 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:44.028+0000 7fe2e67fc640 1 -- 172.21.15.143:0/742147469 <== mon.0 v2:172.21.15.143:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v42) v1 ==== 53+0+73 (secure 0 0 0) 0x7fe2e0018060 con 0x7fe2f00d1bc0 2023-11-26T07:22:44.031 INFO:teuthology.orchestra.run.smithi143.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2023-11-26T07:22:44.033 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:44.032+0000 7fe2f7c06640 1 -- 172.21.15.143:0/742147469 >> [v2:172.21.15.143:6832/2150557544,v1:172.21.15.143:6833/2150557544] conn(0x7fe2d40667d0 msgr2=0x7fe2d4068c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:22:44.033 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:44.032+0000 7fe2f7c06640 1 --2- 172.21.15.143:0/742147469 >> [v2:172.21.15.143:6832/2150557544,v1:172.21.15.143:6833/2150557544] conn(0x7fe2d40667d0 0x7fe2d4068c90 secure :-1 s=READY pgs=457 cs=0 l=1 rev1=1 crypto rx=0x7fe2f00cd3f0 tx=0x7fe2ec008040 comp rx=0 tx=0).stop 2023-11-26T07:22:44.033 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:44.032+0000 7fe2f7c06640 1 -- 172.21.15.143:0/742147469 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fe2f00d1bc0 msgr2=0x7fe2f015dcb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:22:44.033 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:44.032+0000 7fe2f7c06640 1 --2- 172.21.15.143:0/742147469 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fe2f00d1bc0 0x7fe2f015dcb0 secure :-1 s=READY pgs=981 cs=0 l=1 rev1=1 crypto rx=0x7fe2e000c490 tx=0x7fe2e000c950 comp rx=0 tx=0).stop 2023-11-26T07:22:44.033 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:44.032+0000 7fe2f7c06640 1 -- 172.21.15.143:0/742147469 shutdown_connections 2023-11-26T07:22:44.033 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:44.032+0000 7fe2f7c06640 1 --2- 172.21.15.143:0/742147469 >> [v2:172.21.15.143:6832/2150557544,v1:172.21.15.143:6833/2150557544] conn(0x7fe2d40667d0 0x7fe2d4068c90 unknown :-1 s=CLOSED pgs=457 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:22:44.033 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:44.032+0000 7fe2f7c06640 1 --2- 172.21.15.143:0/742147469 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fe2f015d6e0 0x7fe2f015e1f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:22:44.033 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:44.032+0000 7fe2f7c06640 1 --2- 172.21.15.143:0/742147469 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fe2f00d1bc0 0x7fe2f015dcb0 unknown :-1 s=CLOSED pgs=981 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:22:44.033 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:44.032+0000 7fe2f7c06640 1 --2- 172.21.15.143:0/742147469 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fe2f00b2160 0x7fe2f00cc870 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:22:44.033 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:44.032+0000 7fe2f7c06640 1 -- 172.21.15.143:0/742147469 >> 172.21.15.143:0/742147469 conn(0x7fe2f009c550 msgr2=0x7fe2f00734a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:22:44.034 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:44.032+0000 7fe2f7c06640 1 -- 172.21.15.143:0/742147469 shutdown_connections 2023-11-26T07:22:44.034 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:44.032+0000 7fe2f7c06640 1 -- 172.21.15.143:0/742147469 wait complete. 2023-11-26T07:22:44.051 INFO:tasks.cephfs.mount:Mounting default Ceph FS; just confirmed its presence on cluster 2023-11-26T07:22:44.051 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2023-11-26T07:22:44.052 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi143.front.sepia.ceph.com 2023-11-26T07:22:44.052 INFO:tasks.cephfs.mount:self.client.name = client.0 2023-11-26T07:22:44.052 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2023-11-26T07:22:44.052 INFO:tasks.cephfs.mount:self.cephfs_name = None 2023-11-26T07:22:44.052 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2023-11-26T07:22:44.052 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2023-11-26T07:22:44.052 INFO:tasks.cephfs.mount:Setting the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2023-11-26T07:22:44.052 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-26T07:22:44.052 DEBUG:teuthology.orchestra.run.smithi143:> ip addr 2023-11-26T07:22:44.062 INFO:teuthology.orchestra.run.smithi143.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2023-11-26T07:22:44.063 INFO:teuthology.orchestra.run.smithi143.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2023-11-26T07:22:44.063 INFO:teuthology.orchestra.run.smithi143.stdout: inet 127.0.0.1/8 scope host lo 2023-11-26T07:22:44.063 INFO:teuthology.orchestra.run.smithi143.stdout: valid_lft forever preferred_lft forever 2023-11-26T07:22:44.063 INFO:teuthology.orchestra.run.smithi143.stdout: inet6 ::1/128 scope host 2023-11-26T07:22:44.063 INFO:teuthology.orchestra.run.smithi143.stdout: valid_lft forever preferred_lft forever 2023-11-26T07:22:44.063 INFO:teuthology.orchestra.run.smithi143.stdout:2: eno1: mtu 1500 qdisc noop state DOWN group default qlen 1000 2023-11-26T07:22:44.063 INFO:teuthology.orchestra.run.smithi143.stdout: link/ether 0c:c4:7a:d9:3b:94 brd ff:ff:ff:ff:ff:ff 2023-11-26T07:22:44.063 INFO:teuthology.orchestra.run.smithi143.stdout: altname enp7s0f0 2023-11-26T07:22:44.063 INFO:teuthology.orchestra.run.smithi143.stdout:3: eno2: mtu 1500 qdisc noop state DOWN group default qlen 1000 2023-11-26T07:22:44.063 INFO:teuthology.orchestra.run.smithi143.stdout: link/ether 0c:c4:7a:d9:3b:95 brd ff:ff:ff:ff:ff:ff 2023-11-26T07:22:44.063 INFO:teuthology.orchestra.run.smithi143.stdout: altname enp7s0f1 2023-11-26T07:22:44.064 INFO:teuthology.orchestra.run.smithi143.stdout:4: enp3s0f0: mtu 1500 qdisc noop state DOWN group default qlen 1000 2023-11-26T07:22:44.064 INFO:teuthology.orchestra.run.smithi143.stdout: link/ether 0c:c4:7a:88:77:82 brd ff:ff:ff:ff:ff:ff 2023-11-26T07:22:44.064 INFO:teuthology.orchestra.run.smithi143.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2023-11-26T07:22:44.064 INFO:teuthology.orchestra.run.smithi143.stdout: link/ether 0c:c4:7a:88:77:83 brd ff:ff:ff:ff:ff:ff 2023-11-26T07:22:44.064 INFO:teuthology.orchestra.run.smithi143.stdout: inet 172.21.15.143/20 metric 100 brd 172.21.15.255 scope global dynamic enp3s0f1 2023-11-26T07:22:44.064 INFO:teuthology.orchestra.run.smithi143.stdout: valid_lft 42110sec preferred_lft 42110sec 2023-11-26T07:22:44.064 INFO:teuthology.orchestra.run.smithi143.stdout: inet6 fe80::ec4:7aff:fe88:7783/64 scope link 2023-11-26T07:22:44.064 INFO:teuthology.orchestra.run.smithi143.stdout: valid_lft forever preferred_lft forever 2023-11-26T07:22:44.064 INFO:teuthology.orchestra.run.smithi143.stdout:6: docker0: mtu 1500 qdisc noqueue state DOWN group default 2023-11-26T07:22:44.064 INFO:teuthology.orchestra.run.smithi143.stdout: link/ether 02:42:c4:b4:2f:eb brd ff:ff:ff:ff:ff:ff 2023-11-26T07:22:44.064 INFO:teuthology.orchestra.run.smithi143.stdout: inet 172.17.0.1/16 brd 172.17.255.255 scope global docker0 2023-11-26T07:22:44.064 INFO:teuthology.orchestra.run.smithi143.stdout: valid_lft forever preferred_lft forever 2023-11-26T07:22:44.065 INFO:teuthology.orchestra.run.smithi143.stdout:7: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2023-11-26T07:22:44.065 INFO:teuthology.orchestra.run.smithi143.stdout: link/ether 5e:1f:84:6d:79:ae brd ff:ff:ff:ff:ff:ff 2023-11-26T07:22:44.065 INFO:teuthology.orchestra.run.smithi143.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2023-11-26T07:22:44.065 INFO:teuthology.orchestra.run.smithi143.stdout: valid_lft forever preferred_lft forever 2023-11-26T07:22:44.065 INFO:teuthology.orchestra.run.smithi143.stdout: inet6 fe80::5c1f:84ff:fe6d:79ae/64 scope link 2023-11-26T07:22:44.065 INFO:teuthology.orchestra.run.smithi143.stdout: valid_lft forever preferred_lft forever 2023-11-26T07:22:44.065 INFO:teuthology.orchestra.run.smithi143.stdout:8: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2023-11-26T07:22:44.065 INFO:teuthology.orchestra.run.smithi143.stdout: link/ether 0e:5c:f6:1f:82:2f brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.0 2023-11-26T07:22:44.065 INFO:teuthology.orchestra.run.smithi143.stdout: inet6 fe80::c5c:f6ff:fe1f:822f/64 scope link 2023-11-26T07:22:44.065 INFO:teuthology.orchestra.run.smithi143.stdout: valid_lft forever preferred_lft forever 2023-11-26T07:22:44.065 INFO:teuthology.orchestra.run.smithi143.stdout:9: brx.1@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2023-11-26T07:22:44.065 INFO:teuthology.orchestra.run.smithi143.stdout: link/ether c6:a6:3d:69:d0:db brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.2 2023-11-26T07:22:44.065 INFO:teuthology.orchestra.run.smithi143.stdout: inet6 fe80::c4a6:3dff:fe69:d0db/64 scope link 2023-11-26T07:22:44.066 INFO:teuthology.orchestra.run.smithi143.stdout: valid_lft forever preferred_lft forever 2023-11-26T07:22:44.066 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-26T07:22:44.066 DEBUG:teuthology.orchestra.run.smithi143:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2023-11-26T07:22:44.121 INFO:teuthology.orchestra.run.smithi143.stdout:1 2023-11-26T07:22:44.123 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-26T07:22:44.124 DEBUG:teuthology.orchestra.run.smithi143:> route 2023-11-26T07:22:44.177 INFO:teuthology.orchestra.run.smithi143.stdout:Kernel IP routing table 2023-11-26T07:22:44.178 INFO:teuthology.orchestra.run.smithi143.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2023-11-26T07:22:44.178 INFO:teuthology.orchestra.run.smithi143.stdout:default _gateway 0.0.0.0 UG 0 0 0 enp3s0f1 2023-11-26T07:22:44.178 INFO:teuthology.orchestra.run.smithi143.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2023-11-26T07:22:44.178 INFO:teuthology.orchestra.run.smithi143.stdout:172.17.0.0 0.0.0.0 255.255.0.0 U 0 0 0 docker0 2023-11-26T07:22:44.178 INFO:teuthology.orchestra.run.smithi143.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 enp3s0f1 2023-11-26T07:22:44.178 INFO:teuthology.orchestra.run.smithi143.stdout:ns1.front.sepia 0.0.0.0 255.255.255.255 UH 100 0 0 enp3s0f1 2023-11-26T07:22:44.178 INFO:teuthology.orchestra.run.smithi143.stdout:172.21.0.2 0.0.0.0 255.255.255.255 UH 100 0 0 enp3s0f1 2023-11-26T07:22:44.178 INFO:teuthology.orchestra.run.smithi143.stdout:_gateway 0.0.0.0 255.255.255.255 UH 100 0 0 enp3s0f1 2023-11-26T07:22:44.178 INFO:teuthology.orchestra.run.smithi143.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2023-11-26T07:22:44.179 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-26T07:22:44.179 DEBUG:teuthology.orchestra.run.smithi143:> (cd / && exec bash -c ' 2023-11-26T07:22:44.179 DEBUG:teuthology.orchestra.run.smithi143:> set -e 2023-11-26T07:22:44.179 DEBUG:teuthology.orchestra.run.smithi143:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2023-11-26T07:22:44.179 DEBUG:teuthology.orchestra.run.smithi143:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2023-11-26T07:22:44.179 DEBUG:teuthology.orchestra.run.smithi143:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2023-11-26T07:22:44.179 DEBUG:teuthology.orchestra.run.smithi143:> ') 2023-11-26T07:22:44.269 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-26T07:22:44.269 DEBUG:teuthology.orchestra.run.smithi143:> ip netns list 2023-11-26T07:22:44.278 INFO:teuthology.orchestra.run.smithi143.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2023-11-26T07:22:44.279 INFO:teuthology.orchestra.run.smithi143.stdout:ceph-ns--home-ubuntu-cephtest-mnt.2 (id: 1) 2023-11-26T07:22:44.279 INFO:tasks.cephfs.mount:Resuming the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2023-11-26T07:22:44.279 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-26T07:22:44.279 DEBUG:teuthology.orchestra.run.smithi143:> sudo ip link set brx.0 up 2023-11-26T07:22:44.339 INFO:tasks.cephfs.fuse_mount:Client client.0 config is {} 2023-11-26T07:22:44.339 INFO:teuthology.orchestra.run:Running command with timeout 60 2023-11-26T07:22:44.339 DEBUG:teuthology.orchestra.run.smithi143:> mkdir -p -v /home/ubuntu/cephtest/mnt.0 2023-11-26T07:22:44.391 INFO:teuthology.orchestra.run.smithi143.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.0' 2023-11-26T07:22:44.392 INFO:teuthology.orchestra.run:Running command with timeout 60 2023-11-26T07:22:44.392 DEBUG:teuthology.orchestra.run.smithi143:> chmod 0000 /home/ubuntu/cephtest/mnt.0 2023-11-26T07:22:44.443 DEBUG:teuthology.orchestra.run.smithi143:> sudo modprobe fuse 2023-11-26T07:22:44.508 DEBUG:teuthology.orchestra.run.smithi143:> cat /proc/self/mounts | awk '{print $2}' 2023-11-26T07:22:44.561 INFO:teuthology.orchestra.run.smithi143.stdout:/sys 2023-11-26T07:22:44.561 INFO:teuthology.orchestra.run.smithi143.stdout:/proc 2023-11-26T07:22:44.561 INFO:teuthology.orchestra.run.smithi143.stdout:/dev 2023-11-26T07:22:44.561 INFO:teuthology.orchestra.run.smithi143.stdout:/dev/pts 2023-11-26T07:22:44.561 INFO:teuthology.orchestra.run.smithi143.stdout:/run 2023-11-26T07:22:44.561 INFO:teuthology.orchestra.run.smithi143.stdout:/ 2023-11-26T07:22:44.561 INFO:teuthology.orchestra.run.smithi143.stdout:/sys/kernel/security 2023-11-26T07:22:44.561 INFO:teuthology.orchestra.run.smithi143.stdout:/dev/shm 2023-11-26T07:22:44.561 INFO:teuthology.orchestra.run.smithi143.stdout:/run/lock 2023-11-26T07:22:44.561 INFO:teuthology.orchestra.run.smithi143.stdout:/sys/fs/cgroup 2023-11-26T07:22:44.562 INFO:teuthology.orchestra.run.smithi143.stdout:/sys/fs/pstore 2023-11-26T07:22:44.562 INFO:teuthology.orchestra.run.smithi143.stdout:/sys/fs/bpf 2023-11-26T07:22:44.562 INFO:teuthology.orchestra.run.smithi143.stdout:/proc/sys/fs/binfmt_misc 2023-11-26T07:22:44.562 INFO:teuthology.orchestra.run.smithi143.stdout:/dev/hugepages 2023-11-26T07:22:44.562 INFO:teuthology.orchestra.run.smithi143.stdout:/dev/mqueue 2023-11-26T07:22:44.562 INFO:teuthology.orchestra.run.smithi143.stdout:/sys/kernel/debug 2023-11-26T07:22:44.562 INFO:teuthology.orchestra.run.smithi143.stdout:/sys/kernel/tracing 2023-11-26T07:22:44.562 INFO:teuthology.orchestra.run.smithi143.stdout:/proc/fs/nfsd 2023-11-26T07:22:44.562 INFO:teuthology.orchestra.run.smithi143.stdout:/sys/fs/fuse/connections 2023-11-26T07:22:44.562 INFO:teuthology.orchestra.run.smithi143.stdout:/sys/kernel/config 2023-11-26T07:22:44.562 INFO:teuthology.orchestra.run.smithi143.stdout:/run/credentials/systemd-sysusers.service 2023-11-26T07:22:44.562 INFO:teuthology.orchestra.run.smithi143.stdout:/run/rpc_pipefs 2023-11-26T07:22:44.563 INFO:teuthology.orchestra.run.smithi143.stdout:/run/user/1000 2023-11-26T07:22:44.563 INFO:teuthology.orchestra.run.smithi143.stdout:/var/lib/ceph 2023-11-26T07:22:44.563 INFO:teuthology.orchestra.run.smithi143.stdout:/var/lib/ceph/osd/ceph-0 2023-11-26T07:22:44.563 INFO:teuthology.orchestra.run.smithi143.stdout:/var/lib/ceph/osd/ceph-1 2023-11-26T07:22:44.563 INFO:teuthology.orchestra.run.smithi143.stdout:/var/lib/ceph/osd/ceph-2 2023-11-26T07:22:44.563 INFO:teuthology.orchestra.run.smithi143.stdout:/var/lib/ceph/osd/ceph-3 2023-11-26T07:22:44.563 INFO:teuthology.orchestra.run.smithi143.stdout:/run/netns 2023-11-26T07:22:44.563 INFO:teuthology.orchestra.run.smithi143.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2023-11-26T07:22:44.563 INFO:teuthology.orchestra.run.smithi143.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2023-11-26T07:22:44.563 INFO:teuthology.orchestra.run.smithi143.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 2023-11-26T07:22:44.563 INFO:teuthology.orchestra.run.smithi143.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 2023-11-26T07:22:44.564 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-26T07:22:44.564 DEBUG:teuthology.orchestra.run.smithi143:> ls /sys/fs/fuse/connections 2023-11-26T07:22:44.615 INFO:tasks.cephfs.fuse_mount:Pre-mount connections: [] 2023-11-26T07:22:44.615 DEBUG:teuthology.orchestra.run.smithi143:> (cd /home/ubuntu/cephtest && exec sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-fuse -f --admin-socket '/var/run/ceph/$cluster-$name.$pid.asok' /home/ubuntu/cephtest/mnt.0 --id 0) 2023-11-26T07:22:44.662 DEBUG:teuthology.orchestra.run.smithi143:> sudo modprobe fuse 2023-11-26T07:22:44.683 DEBUG:teuthology.orchestra.run.smithi143:> cat /proc/self/mounts | awk '{print $2}' 2023-11-26T07:22:44.737 INFO:teuthology.orchestra.run.smithi143.stdout:/sys 2023-11-26T07:22:44.737 INFO:teuthology.orchestra.run.smithi143.stdout:/proc 2023-11-26T07:22:44.737 INFO:teuthology.orchestra.run.smithi143.stdout:/dev 2023-11-26T07:22:44.737 INFO:teuthology.orchestra.run.smithi143.stdout:/dev/pts 2023-11-26T07:22:44.737 INFO:teuthology.orchestra.run.smithi143.stdout:/run 2023-11-26T07:22:44.737 INFO:teuthology.orchestra.run.smithi143.stdout:/ 2023-11-26T07:22:44.737 INFO:teuthology.orchestra.run.smithi143.stdout:/sys/kernel/security 2023-11-26T07:22:44.737 INFO:teuthology.orchestra.run.smithi143.stdout:/dev/shm 2023-11-26T07:22:44.737 INFO:teuthology.orchestra.run.smithi143.stdout:/run/lock 2023-11-26T07:22:44.737 INFO:teuthology.orchestra.run.smithi143.stdout:/sys/fs/cgroup 2023-11-26T07:22:44.738 INFO:teuthology.orchestra.run.smithi143.stdout:/sys/fs/pstore 2023-11-26T07:22:44.738 INFO:teuthology.orchestra.run.smithi143.stdout:/sys/fs/bpf 2023-11-26T07:22:44.738 INFO:teuthology.orchestra.run.smithi143.stdout:/proc/sys/fs/binfmt_misc 2023-11-26T07:22:44.738 INFO:teuthology.orchestra.run.smithi143.stdout:/dev/hugepages 2023-11-26T07:22:44.738 INFO:teuthology.orchestra.run.smithi143.stdout:/dev/mqueue 2023-11-26T07:22:44.738 INFO:teuthology.orchestra.run.smithi143.stdout:/sys/kernel/debug 2023-11-26T07:22:44.738 INFO:teuthology.orchestra.run.smithi143.stdout:/sys/kernel/tracing 2023-11-26T07:22:44.738 INFO:teuthology.orchestra.run.smithi143.stdout:/proc/fs/nfsd 2023-11-26T07:22:44.738 INFO:teuthology.orchestra.run.smithi143.stdout:/sys/fs/fuse/connections 2023-11-26T07:22:44.738 INFO:teuthology.orchestra.run.smithi143.stdout:/sys/kernel/config 2023-11-26T07:22:44.738 INFO:teuthology.orchestra.run.smithi143.stdout:/run/credentials/systemd-sysusers.service 2023-11-26T07:22:44.738 INFO:teuthology.orchestra.run.smithi143.stdout:/run/rpc_pipefs 2023-11-26T07:22:44.739 INFO:teuthology.orchestra.run.smithi143.stdout:/run/user/1000 2023-11-26T07:22:44.739 INFO:teuthology.orchestra.run.smithi143.stdout:/var/lib/ceph 2023-11-26T07:22:44.739 INFO:teuthology.orchestra.run.smithi143.stdout:/var/lib/ceph/osd/ceph-0 2023-11-26T07:22:44.739 INFO:teuthology.orchestra.run.smithi143.stdout:/var/lib/ceph/osd/ceph-1 2023-11-26T07:22:44.739 INFO:teuthology.orchestra.run.smithi143.stdout:/var/lib/ceph/osd/ceph-2 2023-11-26T07:22:44.739 INFO:teuthology.orchestra.run.smithi143.stdout:/var/lib/ceph/osd/ceph-3 2023-11-26T07:22:44.739 INFO:teuthology.orchestra.run.smithi143.stdout:/run/netns 2023-11-26T07:22:44.739 INFO:teuthology.orchestra.run.smithi143.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2023-11-26T07:22:44.739 INFO:teuthology.orchestra.run.smithi143.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2023-11-26T07:22:44.739 INFO:teuthology.orchestra.run.smithi143.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 2023-11-26T07:22:44.739 INFO:teuthology.orchestra.run.smithi143.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 2023-11-26T07:22:44.740 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-26T07:22:44.740 DEBUG:teuthology.orchestra.run.smithi143:> ls /sys/fs/fuse/connections 2023-11-26T07:22:44.759 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi143.stderr:2023-11-26T07:22:44.756+0000 7fcbf7dc2580 -1 init, newargv = 0x55773b30c260 newargc=15 2023-11-26T07:22:44.760 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi143.stderr:ceph-fuse[55259]: starting ceph client 2023-11-26T07:22:44.766 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi143.stderr:ceph-fuse[55259]: starting fuse 2023-11-26T07:22:44.769 INFO:teuthology.orchestra.run.smithi143.stdout:43 2023-11-26T07:22:44.769 INFO:tasks.cephfs.fuse_mount:Post-mount connections: [43] 2023-11-26T07:22:44.770 DEBUG:teuthology.orchestra.run.smithi143:> sudo stdin-killer -- python3 -c ' 2023-11-26T07:22:44.770 DEBUG:teuthology.orchestra.run.smithi143:> import glob 2023-11-26T07:22:44.770 DEBUG:teuthology.orchestra.run.smithi143:> import re 2023-11-26T07:22:44.770 DEBUG:teuthology.orchestra.run.smithi143:> import os 2023-11-26T07:22:44.770 DEBUG:teuthology.orchestra.run.smithi143:> import subprocess 2023-11-26T07:22:44.770 DEBUG:teuthology.orchestra.run.smithi143:> 2023-11-26T07:22:44.770 DEBUG:teuthology.orchestra.run.smithi143:> def _find_admin_socket(client_name): 2023-11-26T07:22:44.770 DEBUG:teuthology.orchestra.run.smithi143:> asok_path = "/var/run/ceph/ceph-client.0.*.asok" 2023-11-26T07:22:44.770 DEBUG:teuthology.orchestra.run.smithi143:> files = glob.glob(asok_path) 2023-11-26T07:22:44.770 DEBUG:teuthology.orchestra.run.smithi143:> mountpoint = "/home/ubuntu/cephtest/mnt.0" 2023-11-26T07:22:44.771 DEBUG:teuthology.orchestra.run.smithi143:> 2023-11-26T07:22:44.771 DEBUG:teuthology.orchestra.run.smithi143:> # Given a non-glob path, it better be there 2023-11-26T07:22:44.771 DEBUG:teuthology.orchestra.run.smithi143:> if "*" not in asok_path: 2023-11-26T07:22:44.771 DEBUG:teuthology.orchestra.run.smithi143:> assert(len(files) == 1) 2023-11-26T07:22:44.771 DEBUG:teuthology.orchestra.run.smithi143:> return files[0] 2023-11-26T07:22:44.771 DEBUG:teuthology.orchestra.run.smithi143:> 2023-11-26T07:22:44.771 DEBUG:teuthology.orchestra.run.smithi143:> for f in files: 2023-11-26T07:22:44.771 DEBUG:teuthology.orchestra.run.smithi143:> pid = re.match(".*\.(\d+)\.asok$", f).group(1) 2023-11-26T07:22:44.771 DEBUG:teuthology.orchestra.run.smithi143:> if os.path.exists("/proc/{0}".format(pid)): 2023-11-26T07:22:44.771 DEBUG:teuthology.orchestra.run.smithi143:> with open("/proc/{0}/cmdline".format(pid), '"'"'r'"'"') as proc_f: 2023-11-26T07:22:44.771 DEBUG:teuthology.orchestra.run.smithi143:> contents = proc_f.read() 2023-11-26T07:22:44.771 DEBUG:teuthology.orchestra.run.smithi143:> if mountpoint in contents: 2023-11-26T07:22:44.771 DEBUG:teuthology.orchestra.run.smithi143:> return f 2023-11-26T07:22:44.771 DEBUG:teuthology.orchestra.run.smithi143:> raise RuntimeError("Client socket {0} not found".format(client_name)) 2023-11-26T07:22:44.772 DEBUG:teuthology.orchestra.run.smithi143:> 2023-11-26T07:22:44.772 DEBUG:teuthology.orchestra.run.smithi143:> print(_find_admin_socket("client.0")) 2023-11-26T07:22:44.772 DEBUG:teuthology.orchestra.run.smithi143:> ' 2023-11-26T07:22:44.894 INFO:teuthology.orchestra.run.smithi143.stdout:/var/run/ceph/ceph-client.0.55259.asok 2023-11-26T07:22:44.898 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:44 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2023-11-26T07:22:44.905 INFO:tasks.cephfs.fuse_mount:Found client admin socket at /var/run/ceph/ceph-client.0.55259.asok 2023-11-26T07:22:44.905 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-26T07:22:44.906 DEBUG:teuthology.orchestra.run.smithi143:> sudo ceph --admin-daemon /var/run/ceph/ceph-client.0.55259.asok status 2023-11-26T07:22:45.080 INFO:teuthology.orchestra.run.smithi143.stdout:{ 2023-11-26T07:22:45.080 INFO:teuthology.orchestra.run.smithi143.stdout: "metadata": { 2023-11-26T07:22:45.080 INFO:teuthology.orchestra.run.smithi143.stdout: "ceph_sha1": "44c2237fe77713214148c1fd622c872f346f1b99", 2023-11-26T07:22:45.080 INFO:teuthology.orchestra.run.smithi143.stdout: "ceph_version": "ceph version 18.0.0-7484-g44c2237f (44c2237fe77713214148c1fd622c872f346f1b99) reef (dev)", 2023-11-26T07:22:45.080 INFO:teuthology.orchestra.run.smithi143.stdout: "entity_id": "0", 2023-11-26T07:22:45.080 INFO:teuthology.orchestra.run.smithi143.stdout: "hostname": "smithi143", 2023-11-26T07:22:45.080 INFO:teuthology.orchestra.run.smithi143.stdout: "mount_point": "/home/ubuntu/cephtest/mnt.0", 2023-11-26T07:22:45.080 INFO:teuthology.orchestra.run.smithi143.stdout: "pid": "55259", 2023-11-26T07:22:45.080 INFO:teuthology.orchestra.run.smithi143.stdout: "root": "/" 2023-11-26T07:22:45.080 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2023-11-26T07:22:45.081 INFO:teuthology.orchestra.run.smithi143.stdout: "dentry_count": 0, 2023-11-26T07:22:45.081 INFO:teuthology.orchestra.run.smithi143.stdout: "dentry_pinned_count": 0, 2023-11-26T07:22:45.081 INFO:teuthology.orchestra.run.smithi143.stdout: "id": 7041, 2023-11-26T07:22:45.081 INFO:teuthology.orchestra.run.smithi143.stdout: "inst": { 2023-11-26T07:22:45.081 INFO:teuthology.orchestra.run.smithi143.stdout: "name": { 2023-11-26T07:22:45.081 INFO:teuthology.orchestra.run.smithi143.stdout: "type": "client", 2023-11-26T07:22:45.081 INFO:teuthology.orchestra.run.smithi143.stdout: "num": 7041 2023-11-26T07:22:45.081 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2023-11-26T07:22:45.081 INFO:teuthology.orchestra.run.smithi143.stdout: "addr": { 2023-11-26T07:22:45.081 INFO:teuthology.orchestra.run.smithi143.stdout: "type": "v1", 2023-11-26T07:22:45.081 INFO:teuthology.orchestra.run.smithi143.stdout: "addr": "192.168.0.1:0", 2023-11-26T07:22:45.081 INFO:teuthology.orchestra.run.smithi143.stdout: "nonce": 3891126989 2023-11-26T07:22:45.082 INFO:teuthology.orchestra.run.smithi143.stdout: } 2023-11-26T07:22:45.082 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2023-11-26T07:22:45.082 INFO:teuthology.orchestra.run.smithi143.stdout: "addr": { 2023-11-26T07:22:45.082 INFO:teuthology.orchestra.run.smithi143.stdout: "type": "v1", 2023-11-26T07:22:45.082 INFO:teuthology.orchestra.run.smithi143.stdout: "addr": "192.168.0.1:0", 2023-11-26T07:22:45.082 INFO:teuthology.orchestra.run.smithi143.stdout: "nonce": 3891126989 2023-11-26T07:22:45.082 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2023-11-26T07:22:45.082 INFO:teuthology.orchestra.run.smithi143.stdout: "inst_str": "client.7041 192.168.0.1:0/3891126989", 2023-11-26T07:22:45.082 INFO:teuthology.orchestra.run.smithi143.stdout: "addr_str": "192.168.0.1:0/3891126989", 2023-11-26T07:22:45.082 INFO:teuthology.orchestra.run.smithi143.stdout: "inode_count": 1, 2023-11-26T07:22:45.082 INFO:teuthology.orchestra.run.smithi143.stdout: "mds_epoch": 42, 2023-11-26T07:22:45.082 INFO:teuthology.orchestra.run.smithi143.stdout: "osd_epoch": 71, 2023-11-26T07:22:45.083 INFO:teuthology.orchestra.run.smithi143.stdout: "osd_epoch_barrier": 0, 2023-11-26T07:22:45.083 INFO:teuthology.orchestra.run.smithi143.stdout: "blocklisted": false, 2023-11-26T07:22:45.083 INFO:teuthology.orchestra.run.smithi143.stdout: "fs_name": "cephfs" 2023-11-26T07:22:45.083 INFO:teuthology.orchestra.run.smithi143.stdout:} 2023-11-26T07:22:45.094 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-26T07:22:45.094 DEBUG:teuthology.orchestra.run.smithi143:> stat --file-system '--printf=%T 2023-11-26T07:22:45.094 DEBUG:teuthology.orchestra.run.smithi143:> ' -- /home/ubuntu/cephtest/mnt.0 2023-11-26T07:22:45.103 INFO:teuthology.orchestra.run.smithi143.stdout:fuseblk 2023-11-26T07:22:45.103 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.0 2023-11-26T07:22:45.104 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-26T07:22:45.104 DEBUG:teuthology.orchestra.run.smithi143:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.0 2023-11-26T07:22:45.171 DEBUG:teuthology.orchestra.run.smithi143:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2023-11-26T07:22:45.355 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:45.352+0000 7f6cc0a6b640 1 Processor -- start 2023-11-26T07:22:45.355 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:45.352+0000 7f6cc0a6b640 1 -- start start 2023-11-26T07:22:45.356 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:45.356+0000 7f6cc0a6b640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f6cbc0cf280 0x7f6cbc0cf6e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:22:45.356 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:45.356+0000 7f6cc0a6b640 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7f6cbc0cd1b0 0x7f6cbc0cd5e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:22:45.357 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:45.356+0000 7f6cc0a6b640 1 -- --> v1:172.21.15.143:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6cbc0cfc20 con 0x7f6cbc143ac0 2023-11-26T07:22:45.357 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:45.356+0000 7f6cc0a6b640 1 -- --> v2:172.21.15.188:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6cbc143ea0 con 0x7f6cbc0cd1b0 2023-11-26T07:22:45.357 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:45.356+0000 7f6cc0a6b640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6cbc0cc730 con 0x7f6cbc0cf280 2023-11-26T07:22:45.357 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:45.356+0000 7f6cba59c640 1 --1- >> v1:172.21.15.143:6789/0 conn(0x7f6cbc143ac0 0x7f6cbc070370 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.143:6789/0 says I am v1:172.21.15.143:47964/0 (socket says 172.21.15.143:47964) 2023-11-26T07:22:45.357 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:45.356+0000 7f6cba59c640 1 -- 172.21.15.143:0/3261020485 learned_addr learned my addr 172.21.15.143:0/3261020485 (peer_addr_for_me v1:172.21.15.143:0/0) 2023-11-26T07:22:45.359 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:45.356+0000 7f6cbad9d640 1 --2- 172.21.15.143:0/3261020485 >> v2:172.21.15.188:3300/0 conn(0x7f6cbc0cd1b0 0x7f6cbc0cd5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:22:45.359 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:45.356+0000 7f6cb1d9b640 1 --2- 172.21.15.143:0/3261020485 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f6cbc0cf280 0x7f6cbc0cf6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:22:45.359 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:45.356+0000 7f6cb9d9b640 1 -- 172.21.15.143:0/3261020485 <== mon.0 v1:172.21.15.143:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1010115347 0 0) 0x7f6cbc0cfc20 con 0x7f6cbc143ac0 2023-11-26T07:22:45.359 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:45.356+0000 7f6cb9d9b640 1 -- 172.21.15.143:0/3261020485 --> v1:172.21.15.143:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f6ca8003530 con 0x7f6cbc143ac0 2023-11-26T07:22:45.359 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:45.356+0000 7f6cb9d9b640 1 -- 172.21.15.143:0/3261020485 <== mon.0 v1:172.21.15.143:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2332471360 0 0) 0x7f6ca8003530 con 0x7f6cbc143ac0 2023-11-26T07:22:45.359 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:45.356+0000 7f6cb9d9b640 1 -- 172.21.15.143:0/3261020485 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f6cbc0cf280 msgr2=0x7f6cbc0cf6e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-26T07:22:45.359 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:45.356+0000 7f6cb9d9b640 1 --2- 172.21.15.143:0/3261020485 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f6cbc0cf280 0x7f6cbc0cf6e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:22:45.359 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:45.356+0000 7f6cb9d9b640 1 -- 172.21.15.143:0/3261020485 >> v2:172.21.15.188:3300/0 conn(0x7f6cbc0cd1b0 msgr2=0x7f6cbc0cd5e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-26T07:22:45.359 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:45.356+0000 7f6cb9d9b640 1 --2- 172.21.15.143:0/3261020485 >> v2:172.21.15.188:3300/0 conn(0x7f6cbc0cd1b0 0x7f6cbc0cd5e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:22:45.359 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:45.356+0000 7f6cb9d9b640 1 -- 172.21.15.143:0/3261020485 --> v1:172.21.15.143:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6cbc070ee0 con 0x7f6cbc143ac0 2023-11-26T07:22:45.359 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:45.356+0000 7f6cb9d9b640 1 -- 172.21.15.143:0/3261020485 <== mon.0 v1:172.21.15.143:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2137882523 0 0) 0x7f6cac001680 con 0x7f6cbc143ac0 2023-11-26T07:22:45.360 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:45.356+0000 7f6cb9d9b640 1 -- 172.21.15.143:0/3261020485 >> v1:172.21.15.143:6789/0 conn(0x7f6cbc143ac0 legacy=0x7f6cbc070370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:22:45.360 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:45.356+0000 7f6cb9d9b640 1 --2- 172.21.15.143:0/3261020485 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f6ca8003980 0x7f6ca8003da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:22:45.360 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:45.356+0000 7f6cb9d9b640 1 --2- 172.21.15.143:0/3261020485 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f6cbc0cf280 0x7f6ca80047d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:22:45.360 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:45.356+0000 7f6cb9d9b640 1 --2- 172.21.15.143:0/3261020485 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f6ca8004d10 0x7f6ca8005190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:22:45.360 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:45.356+0000 7f6cb9d9b640 1 -- 172.21.15.143:0/3261020485 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6cbc0cc730 con 0x7f6ca8004d10 2023-11-26T07:22:45.360 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:45.356+0000 7f6cb9d9b640 1 -- 172.21.15.143:0/3261020485 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6ca800b990 con 0x7f6ca8003980 2023-11-26T07:22:45.360 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:45.356+0000 7f6cb9d9b640 1 -- 172.21.15.143:0/3261020485 --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6ca800bad0 con 0x7f6cbc0cf280 2023-11-26T07:22:45.360 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:45.356+0000 7f6cba59c640 1 --2- 172.21.15.143:0/3261020485 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f6ca8003980 0x7f6ca8003da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:22:45.360 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:45.356+0000 7f6cb1d9b640 1 --2- 172.21.15.143:0/3261020485 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f6cbc0cf280 0x7f6ca80047d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:22:45.360 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:45.356+0000 7f6cbad9d640 1 --2- 172.21.15.143:0/3261020485 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f6ca8004d10 0x7f6ca8005190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:22:45.360 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:45.356+0000 7f6cba59c640 1 -- 172.21.15.143:0/3261020485 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f6cbc0cf280 msgr2=0x7f6ca80047d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-26T07:22:45.360 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:45.356+0000 7f6cba59c640 1 --2- 172.21.15.143:0/3261020485 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f6cbc0cf280 0x7f6ca80047d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:22:45.361 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:45.356+0000 7f6cba59c640 1 -- 172.21.15.143:0/3261020485 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f6ca8004d10 msgr2=0x7f6ca8005190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-26T07:22:45.361 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:45.356+0000 7f6cba59c640 1 --2- 172.21.15.143:0/3261020485 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f6ca8004d10 0x7f6ca8005190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:22:45.361 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:45.356+0000 7f6cba59c640 1 -- 172.21.15.143:0/3261020485 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6ca8003530 con 0x7f6ca8003980 2023-11-26T07:22:45.361 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:45.356+0000 7f6cb1d9b640 1 --2- 172.21.15.143:0/3261020485 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f6cbc0cf280 0x7f6ca80047d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-26T07:22:45.361 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:45.356+0000 7f6cbad9d640 1 --2- 172.21.15.143:0/3261020485 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f6ca8004d10 0x7f6ca8005190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-26T07:22:45.362 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:45.360+0000 7f6cba59c640 1 --2- 172.21.15.143:0/3261020485 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f6ca8003980 0x7f6ca8003da0 secure :-1 s=READY pgs=389 cs=0 l=1 rev1=1 crypto rx=0x7f6cac00c110 tx=0x7f6cac01cb60 comp rx=0 tx=0).ready entity=mon.1 client_cookie=8450cf491970995e server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:22:45.362 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:45.360+0000 7f6cb9d9b640 1 -- 172.21.15.143:0/3261020485 <== mon.1 v2:172.21.15.188:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6cac0058e0 con 0x7f6ca8003980 2023-11-26T07:22:45.362 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:45.360+0000 7f6cb9d9b640 1 -- 172.21.15.143:0/3261020485 <== mon.1 v2:172.21.15.188:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6cac005a40 con 0x7f6ca8003980 2023-11-26T07:22:45.362 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:45.360+0000 7f6cb9d9b640 1 -- 172.21.15.143:0/3261020485 <== mon.1 v2:172.21.15.188:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6cac005df0 con 0x7f6ca8003980 2023-11-26T07:22:45.362 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:45.360+0000 7f6cc0a6b640 1 -- 172.21.15.143:0/3261020485 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f6ca8003980 msgr2=0x7f6ca8003da0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:22:45.362 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:45.360+0000 7f6cc0a6b640 1 --2- 172.21.15.143:0/3261020485 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f6ca8003980 0x7f6ca8003da0 secure :-1 s=READY pgs=389 cs=0 l=1 rev1=1 crypto rx=0x7f6cac00c110 tx=0x7f6cac01cb60 comp rx=0 tx=0).stop 2023-11-26T07:22:45.362 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:45.360+0000 7f6cbad9d640 1 -- 172.21.15.143:0/3261020485 reap_dead start 2023-11-26T07:22:45.362 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:45.360+0000 7f6cc0a6b640 1 -- 172.21.15.143:0/3261020485 shutdown_connections 2023-11-26T07:22:45.363 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:45.360+0000 7f6cc0a6b640 1 -- 172.21.15.143:0/3261020485 >> 172.21.15.143:0/3261020485 conn(0x7f6cbc06a890 msgr2=0x7f6cbc06aca0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:22:45.363 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:45.360+0000 7f6cc0a6b640 1 -- 172.21.15.143:0/3261020485 shutdown_connections 2023-11-26T07:22:45.363 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:45.360+0000 7f6cc0a6b640 1 -- 172.21.15.143:0/3261020485 wait complete. 2023-11-26T07:22:45.364 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:45.364+0000 7f6cc0a6b640 1 Processor -- start 2023-11-26T07:22:45.364 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:45.364+0000 7f6cc0a6b640 1 -- start start 2023-11-26T07:22:45.366 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:45.364+0000 7f6cc0a6b640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f6ca8003980 0x7f6cbc0b0e20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:22:45.366 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:45.364+0000 7f6cc0a6b640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f6cbc0aeb30 0x7f6cbc136b10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:22:45.367 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:45.364+0000 7f6cc0a6b640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f6cbc140880 0x7f6cbc140d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:22:45.367 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:45.364+0000 7f6cc0a6b640 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6cbc137ad0 con 0x7f6ca8003980 2023-11-26T07:22:45.367 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:45.364+0000 7f6cc0a6b640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6cbc141250 con 0x7f6cbc140880 2023-11-26T07:22:45.367 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:45.364+0000 7f6cc0a6b640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6cbc0aefb0 con 0x7f6cbc0aeb30 2023-11-26T07:22:45.367 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:45.364+0000 7f6cba59c640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f6ca8003980 0x7f6cbc0b0e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:22:45.367 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:45.364+0000 7f6cba59c640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f6ca8003980 0x7f6cbc0b0e20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:50328/0 (socket says 172.21.15.143:50328) 2023-11-26T07:22:45.367 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:45.364+0000 7f6cba59c640 1 -- 172.21.15.143:0/3692376059 learned_addr learned my addr 172.21.15.143:0/3692376059 (peer_addr_for_me v2:172.21.15.143:0/0) 2023-11-26T07:22:45.367 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:45.364+0000 7f6cb1d9b640 1 --2- 172.21.15.143:0/3692376059 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f6cbc0aeb30 0x7f6cbc136b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:22:45.367 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:45.364+0000 7f6cbad9d640 1 --2- 172.21.15.143:0/3692376059 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f6cbc140880 0x7f6cbc140d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:22:45.367 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:45.364+0000 7f6cba59c640 1 -- 172.21.15.143:0/3692376059 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f6cbc0aeb30 msgr2=0x7f6cbc136b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:22:45.367 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:45.364+0000 7f6cba59c640 1 --2- 172.21.15.143:0/3692376059 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f6cbc0aeb30 0x7f6cbc136b10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:22:45.367 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:45.364+0000 7f6cba59c640 1 -- 172.21.15.143:0/3692376059 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f6cbc140880 msgr2=0x7f6cbc140d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:22:45.367 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:45.364+0000 7f6cba59c640 1 --2- 172.21.15.143:0/3692376059 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f6cbc140880 0x7f6cbc140d10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:22:45.368 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:45.364+0000 7f6cba59c640 1 -- 172.21.15.143:0/3692376059 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6cbc159ba0 con 0x7f6ca8003980 2023-11-26T07:22:45.368 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:45.364+0000 7f6cb1d9b640 1 --2- 172.21.15.143:0/3692376059 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f6cbc0aeb30 0x7f6cbc136b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-26T07:22:45.368 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:45.364+0000 7f6cbad9d640 1 --2- 172.21.15.143:0/3692376059 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f6cbc140880 0x7f6cbc140d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-26T07:22:45.368 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:45.364+0000 7f6cba59c640 1 --2- 172.21.15.143:0/3692376059 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f6ca8003980 0x7f6cbc0b0e20 secure :-1 s=READY pgs=984 cs=0 l=1 rev1=1 crypto rx=0x7f6cac007040 tx=0x7f6ca8005800 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:22:45.368 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:45.364+0000 7f6cb37fe640 1 -- 172.21.15.143:0/3692376059 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6cac0a6bc0 con 0x7f6ca8003980 2023-11-26T07:22:45.368 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:45.364+0000 7f6cb37fe640 1 -- 172.21.15.143:0/3692376059 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6cac0a6d40 con 0x7f6ca8003980 2023-11-26T07:22:45.368 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:45.364+0000 7f6cb37fe640 1 -- 172.21.15.143:0/3692376059 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6cac004a30 con 0x7f6ca8003980 2023-11-26T07:22:45.368 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:45.364+0000 7f6cc0a6b640 1 -- 172.21.15.143:0/3692376059 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6cbc0af1c0 con 0x7f6ca8003980 2023-11-26T07:22:45.368 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:45.364+0000 7f6cc0a6b640 1 -- 172.21.15.143:0/3692376059 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6cbc0af780 con 0x7f6ca8003980 2023-11-26T07:22:45.369 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:45.368+0000 7f6cc0a6b640 1 -- 172.21.15.143:0/3692376059 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6cbc067680 con 0x7f6ca8003980 2023-11-26T07:22:45.369 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:45.368+0000 7f6cb37fe640 1 -- 172.21.15.143:0/3692376059 <== mon.0 v2:172.21.15.143:3300/0 4 ==== mgrmap(e 11) v1 ==== 86158+0+0 (secure 0 0 0) 0x7f6cac004b90 con 0x7f6ca8003980 2023-11-26T07:22:45.370 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:45.368+0000 7f6cb37fe640 1 --2- 172.21.15.143:0/3692376059 >> [v2:172.21.15.143:6832/2150557544,v1:172.21.15.143:6833/2150557544] conn(0x7f6c9c0667d0 0x7f6c9c068c90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:22:45.370 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:45.368+0000 7f6cb37fe640 1 -- 172.21.15.143:0/3692376059 <== mon.0 v2:172.21.15.143:3300/0 5 ==== osd_map(71..71 src has 1..71) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f6cac11dce0 con 0x7f6ca8003980 2023-11-26T07:22:45.370 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:45.368+0000 7f6cb1d9b640 1 --2- 172.21.15.143:0/3692376059 >> [v2:172.21.15.143:6832/2150557544,v1:172.21.15.143:6833/2150557544] conn(0x7f6c9c0667d0 0x7f6c9c068c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:22:45.371 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:45.368+0000 7f6cb1d9b640 1 --2- 172.21.15.143:0/3692376059 >> [v2:172.21.15.143:6832/2150557544,v1:172.21.15.143:6833/2150557544] conn(0x7f6c9c0667d0 0x7f6c9c068c90 secure :-1 s=READY pgs=458 cs=0 l=1 rev1=1 crypto rx=0x7f6ca4009fd0 tx=0x7f6ca400b040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:22:45.374 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:45.372+0000 7f6cb37fe640 1 -- 172.21.15.143:0/3692376059 <== mon.0 v2:172.21.15.143:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172649 (secure 0 0 0) 0x7f6cac0eada0 con 0x7f6ca8003980 2023-11-26T07:22:45.556 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:45.556+0000 7f6cc0a6b640 1 -- 172.21.15.143:0/3692376059 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "fs ls"} v 0) v1 -- 0x7f6cbc0cb270 con 0x7f6ca8003980 2023-11-26T07:22:45.556 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:45.556+0000 7f6cb37fe640 1 -- 172.21.15.143:0/3692376059 <== mon.0 v2:172.21.15.143:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v42) v1 ==== 53+0+73 (secure 0 0 0) 0x7f6cac0bc070 con 0x7f6ca8003980 2023-11-26T07:22:45.557 INFO:teuthology.orchestra.run.smithi143.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2023-11-26T07:22:45.558 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:45.556+0000 7f6cc0a6b640 1 -- 172.21.15.143:0/3692376059 >> [v2:172.21.15.143:6832/2150557544,v1:172.21.15.143:6833/2150557544] conn(0x7f6c9c0667d0 msgr2=0x7f6c9c068c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:22:45.559 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:45.556+0000 7f6cc0a6b640 1 --2- 172.21.15.143:0/3692376059 >> [v2:172.21.15.143:6832/2150557544,v1:172.21.15.143:6833/2150557544] conn(0x7f6c9c0667d0 0x7f6c9c068c90 secure :-1 s=READY pgs=458 cs=0 l=1 rev1=1 crypto rx=0x7f6ca4009fd0 tx=0x7f6ca400b040 comp rx=0 tx=0).stop 2023-11-26T07:22:45.559 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:45.556+0000 7f6cc0a6b640 1 -- 172.21.15.143:0/3692376059 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f6ca8003980 msgr2=0x7f6cbc0b0e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:22:45.559 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:45.556+0000 7f6cc0a6b640 1 --2- 172.21.15.143:0/3692376059 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f6ca8003980 0x7f6cbc0b0e20 secure :-1 s=READY pgs=984 cs=0 l=1 rev1=1 crypto rx=0x7f6cac007040 tx=0x7f6ca8005800 comp rx=0 tx=0).stop 2023-11-26T07:22:45.559 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:45.556+0000 7f6cc0a6b640 1 -- 172.21.15.143:0/3692376059 shutdown_connections 2023-11-26T07:22:45.559 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:45.556+0000 7f6cc0a6b640 1 --2- 172.21.15.143:0/3692376059 >> [v2:172.21.15.143:6832/2150557544,v1:172.21.15.143:6833/2150557544] conn(0x7f6c9c0667d0 0x7f6c9c068c90 unknown :-1 s=CLOSED pgs=458 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:22:45.559 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:45.556+0000 7f6cc0a6b640 1 --2- 172.21.15.143:0/3692376059 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f6cbc140880 0x7f6cbc140d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:22:45.559 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:45.556+0000 7f6cc0a6b640 1 --2- 172.21.15.143:0/3692376059 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f6cbc0aeb30 0x7f6cbc136b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:22:45.559 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:45.556+0000 7f6cc0a6b640 1 --2- 172.21.15.143:0/3692376059 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f6ca8003980 0x7f6cbc0b0e20 unknown :-1 s=CLOSED pgs=984 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:22:45.559 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:45.556+0000 7f6cc0a6b640 1 -- 172.21.15.143:0/3692376059 >> 172.21.15.143:0/3692376059 conn(0x7f6cbc06a890 msgr2=0x7f6cbc0cd590 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:22:45.559 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:45.556+0000 7f6cc0a6b640 1 -- 172.21.15.143:0/3692376059 shutdown_connections 2023-11-26T07:22:45.559 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:45.556+0000 7f6cc0a6b640 1 -- 172.21.15.143:0/3692376059 wait complete. 2023-11-26T07:22:45.577 INFO:tasks.cephfs.mount:Mounting default Ceph FS; just confirmed its presence on cluster 2023-11-26T07:22:45.578 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2023-11-26T07:22:45.578 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi188.front.sepia.ceph.com 2023-11-26T07:22:45.578 INFO:tasks.cephfs.mount:self.client.name = client.1 2023-11-26T07:22:45.578 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.1 2023-11-26T07:22:45.578 INFO:tasks.cephfs.mount:self.cephfs_name = None 2023-11-26T07:22:45.578 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2023-11-26T07:22:45.578 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2023-11-26T07:22:45.578 INFO:tasks.cephfs.mount:Setting the 'ceph-ns--home-ubuntu-cephtest-mnt.1' netns for '/home/ubuntu/cephtest/mnt.1' 2023-11-26T07:22:45.578 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-26T07:22:45.578 DEBUG:teuthology.orchestra.run.smithi188:> ip addr 2023-11-26T07:22:45.589 INFO:teuthology.orchestra.run.smithi188.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2023-11-26T07:22:45.589 INFO:teuthology.orchestra.run.smithi188.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2023-11-26T07:22:45.589 INFO:teuthology.orchestra.run.smithi188.stdout: inet 127.0.0.1/8 scope host lo 2023-11-26T07:22:45.589 INFO:teuthology.orchestra.run.smithi188.stdout: valid_lft forever preferred_lft forever 2023-11-26T07:22:45.589 INFO:teuthology.orchestra.run.smithi188.stdout: inet6 ::1/128 scope host 2023-11-26T07:22:45.589 INFO:teuthology.orchestra.run.smithi188.stdout: valid_lft forever preferred_lft forever 2023-11-26T07:22:45.589 INFO:teuthology.orchestra.run.smithi188.stdout:2: eno1: mtu 1500 qdisc noop state DOWN group default qlen 1000 2023-11-26T07:22:45.589 INFO:teuthology.orchestra.run.smithi188.stdout: link/ether 0c:c4:7a:d9:37:48 brd ff:ff:ff:ff:ff:ff 2023-11-26T07:22:45.589 INFO:teuthology.orchestra.run.smithi188.stdout: altname enp7s0f0 2023-11-26T07:22:45.590 INFO:teuthology.orchestra.run.smithi188.stdout:3: eno2: mtu 1500 qdisc noop state DOWN group default qlen 1000 2023-11-26T07:22:45.590 INFO:teuthology.orchestra.run.smithi188.stdout: link/ether 0c:c4:7a:d9:37:49 brd ff:ff:ff:ff:ff:ff 2023-11-26T07:22:45.590 INFO:teuthology.orchestra.run.smithi188.stdout: altname enp7s0f1 2023-11-26T07:22:45.590 INFO:teuthology.orchestra.run.smithi188.stdout:4: enp3s0f0: mtu 1500 qdisc noop state DOWN group default qlen 1000 2023-11-26T07:22:45.590 INFO:teuthology.orchestra.run.smithi188.stdout: link/ether 0c:c4:7a:88:80:38 brd ff:ff:ff:ff:ff:ff 2023-11-26T07:22:45.590 INFO:teuthology.orchestra.run.smithi188.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2023-11-26T07:22:45.590 INFO:teuthology.orchestra.run.smithi188.stdout: link/ether 0c:c4:7a:88:80:39 brd ff:ff:ff:ff:ff:ff 2023-11-26T07:22:45.590 INFO:teuthology.orchestra.run.smithi188.stdout: inet 172.21.15.188/20 metric 100 brd 172.21.15.255 scope global dynamic enp3s0f1 2023-11-26T07:22:45.590 INFO:teuthology.orchestra.run.smithi188.stdout: valid_lft 42102sec preferred_lft 42102sec 2023-11-26T07:22:45.590 INFO:teuthology.orchestra.run.smithi188.stdout: inet6 fe80::ec4:7aff:fe88:8039/64 scope link 2023-11-26T07:22:45.590 INFO:teuthology.orchestra.run.smithi188.stdout: valid_lft forever preferred_lft forever 2023-11-26T07:22:45.590 INFO:teuthology.orchestra.run.smithi188.stdout:6: docker0: mtu 1500 qdisc noqueue state DOWN group default 2023-11-26T07:22:45.591 INFO:teuthology.orchestra.run.smithi188.stdout: link/ether 02:42:d8:0d:a0:fa brd ff:ff:ff:ff:ff:ff 2023-11-26T07:22:45.591 INFO:teuthology.orchestra.run.smithi188.stdout: inet 172.17.0.1/16 brd 172.17.255.255 scope global docker0 2023-11-26T07:22:45.591 INFO:teuthology.orchestra.run.smithi188.stdout: valid_lft forever preferred_lft forever 2023-11-26T07:22:45.591 INFO:teuthology.orchestra.run.smithi188.stdout:7: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2023-11-26T07:22:45.591 INFO:teuthology.orchestra.run.smithi188.stdout: link/ether 5e:1f:84:6d:79:ae brd ff:ff:ff:ff:ff:ff 2023-11-26T07:22:45.591 INFO:teuthology.orchestra.run.smithi188.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2023-11-26T07:22:45.591 INFO:teuthology.orchestra.run.smithi188.stdout: valid_lft forever preferred_lft forever 2023-11-26T07:22:45.591 INFO:teuthology.orchestra.run.smithi188.stdout: inet6 fe80::5c1f:84ff:fe6d:79ae/64 scope link 2023-11-26T07:22:45.591 INFO:teuthology.orchestra.run.smithi188.stdout: valid_lft forever preferred_lft forever 2023-11-26T07:22:45.591 INFO:teuthology.orchestra.run.smithi188.stdout:8: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2023-11-26T07:22:45.591 INFO:teuthology.orchestra.run.smithi188.stdout: link/ether 0e:5c:f6:1f:82:2f brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.1 2023-11-26T07:22:45.591 INFO:teuthology.orchestra.run.smithi188.stdout: inet6 fe80::c5c:f6ff:fe1f:822f/64 scope link 2023-11-26T07:22:45.592 INFO:teuthology.orchestra.run.smithi188.stdout: valid_lft forever preferred_lft forever 2023-11-26T07:22:45.592 INFO:teuthology.orchestra.run.smithi188.stdout:9: brx.1@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2023-11-26T07:22:45.592 INFO:teuthology.orchestra.run.smithi188.stdout: link/ether c6:a6:3d:69:d0:db brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.3 2023-11-26T07:22:45.592 INFO:teuthology.orchestra.run.smithi188.stdout: inet6 fe80::c4a6:3dff:fe69:d0db/64 scope link 2023-11-26T07:22:45.592 INFO:teuthology.orchestra.run.smithi188.stdout: valid_lft forever preferred_lft forever 2023-11-26T07:22:45.593 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-26T07:22:45.593 DEBUG:teuthology.orchestra.run.smithi188:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2023-11-26T07:22:45.655 INFO:teuthology.orchestra.run.smithi188.stdout:1 2023-11-26T07:22:45.658 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-26T07:22:45.658 DEBUG:teuthology.orchestra.run.smithi188:> route 2023-11-26T07:22:45.710 INFO:teuthology.orchestra.run.smithi188.stdout:Kernel IP routing table 2023-11-26T07:22:45.711 INFO:teuthology.orchestra.run.smithi188.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2023-11-26T07:22:45.711 INFO:teuthology.orchestra.run.smithi188.stdout:default _gateway 0.0.0.0 UG 0 0 0 enp3s0f1 2023-11-26T07:22:45.711 INFO:teuthology.orchestra.run.smithi188.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2023-11-26T07:22:45.711 INFO:teuthology.orchestra.run.smithi188.stdout:172.17.0.0 0.0.0.0 255.255.0.0 U 0 0 0 docker0 2023-11-26T07:22:45.711 INFO:teuthology.orchestra.run.smithi188.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 enp3s0f1 2023-11-26T07:22:45.711 INFO:teuthology.orchestra.run.smithi188.stdout:ns1.front.sepia 0.0.0.0 255.255.255.255 UH 100 0 0 enp3s0f1 2023-11-26T07:22:45.711 INFO:teuthology.orchestra.run.smithi188.stdout:172.21.0.2 0.0.0.0 255.255.255.255 UH 100 0 0 enp3s0f1 2023-11-26T07:22:45.712 INFO:teuthology.orchestra.run.smithi188.stdout:_gateway 0.0.0.0 255.255.255.255 UH 100 0 0 enp3s0f1 2023-11-26T07:22:45.712 INFO:teuthology.orchestra.run.smithi188.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2023-11-26T07:22:45.712 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-26T07:22:45.712 DEBUG:teuthology.orchestra.run.smithi188:> (cd / && exec bash -c ' 2023-11-26T07:22:45.712 DEBUG:teuthology.orchestra.run.smithi188:> set -e 2023-11-26T07:22:45.712 DEBUG:teuthology.orchestra.run.smithi188:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2023-11-26T07:22:45.712 DEBUG:teuthology.orchestra.run.smithi188:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2023-11-26T07:22:45.712 DEBUG:teuthology.orchestra.run.smithi188:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2023-11-26T07:22:45.712 DEBUG:teuthology.orchestra.run.smithi188:> ') 2023-11-26T07:22:45.805 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-26T07:22:45.805 DEBUG:teuthology.orchestra.run.smithi188:> ip netns list 2023-11-26T07:22:45.814 INFO:teuthology.orchestra.run.smithi188.stdout:ceph-ns--home-ubuntu-cephtest-mnt.1 (id: 0) 2023-11-26T07:22:45.814 INFO:teuthology.orchestra.run.smithi188.stdout:ceph-ns--home-ubuntu-cephtest-mnt.3 (id: 1) 2023-11-26T07:22:45.814 INFO:tasks.cephfs.mount:Resuming the 'ceph-ns--home-ubuntu-cephtest-mnt.1' netns for '/home/ubuntu/cephtest/mnt.1' 2023-11-26T07:22:45.815 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-26T07:22:45.815 DEBUG:teuthology.orchestra.run.smithi188:> sudo ip link set brx.0 up 2023-11-26T07:22:45.879 INFO:tasks.cephfs.fuse_mount:Client client.1 config is {} 2023-11-26T07:22:45.879 INFO:teuthology.orchestra.run:Running command with timeout 60 2023-11-26T07:22:45.879 DEBUG:teuthology.orchestra.run.smithi188:> mkdir -p -v /home/ubuntu/cephtest/mnt.1 2023-11-26T07:22:45.932 INFO:teuthology.orchestra.run.smithi188.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.1' 2023-11-26T07:22:45.933 INFO:teuthology.orchestra.run:Running command with timeout 60 2023-11-26T07:22:45.933 DEBUG:teuthology.orchestra.run.smithi188:> chmod 0000 /home/ubuntu/cephtest/mnt.1 2023-11-26T07:22:45.987 DEBUG:teuthology.orchestra.run.smithi188:> sudo modprobe fuse 2023-11-26T07:22:46.052 DEBUG:teuthology.orchestra.run.smithi188:> cat /proc/self/mounts | awk '{print $2}' 2023-11-26T07:22:46.105 INFO:teuthology.orchestra.run.smithi188.stdout:/sys 2023-11-26T07:22:46.106 INFO:teuthology.orchestra.run.smithi188.stdout:/proc 2023-11-26T07:22:46.106 INFO:teuthology.orchestra.run.smithi188.stdout:/dev 2023-11-26T07:22:46.106 INFO:teuthology.orchestra.run.smithi188.stdout:/dev/pts 2023-11-26T07:22:46.106 INFO:teuthology.orchestra.run.smithi188.stdout:/run 2023-11-26T07:22:46.106 INFO:teuthology.orchestra.run.smithi188.stdout:/ 2023-11-26T07:22:46.106 INFO:teuthology.orchestra.run.smithi188.stdout:/sys/kernel/security 2023-11-26T07:22:46.106 INFO:teuthology.orchestra.run.smithi188.stdout:/dev/shm 2023-11-26T07:22:46.106 INFO:teuthology.orchestra.run.smithi188.stdout:/run/lock 2023-11-26T07:22:46.106 INFO:teuthology.orchestra.run.smithi188.stdout:/sys/fs/cgroup 2023-11-26T07:22:46.106 INFO:teuthology.orchestra.run.smithi188.stdout:/sys/fs/pstore 2023-11-26T07:22:46.106 INFO:teuthology.orchestra.run.smithi188.stdout:/sys/fs/bpf 2023-11-26T07:22:46.106 INFO:teuthology.orchestra.run.smithi188.stdout:/proc/sys/fs/binfmt_misc 2023-11-26T07:22:46.107 INFO:teuthology.orchestra.run.smithi188.stdout:/dev/hugepages 2023-11-26T07:22:46.107 INFO:teuthology.orchestra.run.smithi188.stdout:/dev/mqueue 2023-11-26T07:22:46.107 INFO:teuthology.orchestra.run.smithi188.stdout:/sys/kernel/debug 2023-11-26T07:22:46.107 INFO:teuthology.orchestra.run.smithi188.stdout:/sys/kernel/tracing 2023-11-26T07:22:46.107 INFO:teuthology.orchestra.run.smithi188.stdout:/proc/fs/nfsd 2023-11-26T07:22:46.107 INFO:teuthology.orchestra.run.smithi188.stdout:/sys/fs/fuse/connections 2023-11-26T07:22:46.107 INFO:teuthology.orchestra.run.smithi188.stdout:/sys/kernel/config 2023-11-26T07:22:46.107 INFO:teuthology.orchestra.run.smithi188.stdout:/run/credentials/systemd-sysusers.service 2023-11-26T07:22:46.107 INFO:teuthology.orchestra.run.smithi188.stdout:/run/rpc_pipefs 2023-11-26T07:22:46.107 INFO:teuthology.orchestra.run.smithi188.stdout:/run/user/1000 2023-11-26T07:22:46.107 INFO:teuthology.orchestra.run.smithi188.stdout:/var/lib/ceph 2023-11-26T07:22:46.107 INFO:teuthology.orchestra.run.smithi188.stdout:/var/lib/ceph/osd/ceph-4 2023-11-26T07:22:46.107 INFO:teuthology.orchestra.run.smithi188.stdout:/var/lib/ceph/osd/ceph-5 2023-11-26T07:22:46.108 INFO:teuthology.orchestra.run.smithi188.stdout:/var/lib/ceph/osd/ceph-6 2023-11-26T07:22:46.108 INFO:teuthology.orchestra.run.smithi188.stdout:/var/lib/ceph/osd/ceph-7 2023-11-26T07:22:46.108 INFO:teuthology.orchestra.run.smithi188.stdout:/run/netns 2023-11-26T07:22:46.108 INFO:teuthology.orchestra.run.smithi188.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 2023-11-26T07:22:46.108 INFO:teuthology.orchestra.run.smithi188.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 2023-11-26T07:22:46.108 INFO:teuthology.orchestra.run.smithi188.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 2023-11-26T07:22:46.108 INFO:teuthology.orchestra.run.smithi188.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 2023-11-26T07:22:46.109 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-26T07:22:46.109 DEBUG:teuthology.orchestra.run.smithi188:> ls /sys/fs/fuse/connections 2023-11-26T07:22:46.159 INFO:tasks.cephfs.fuse_mount:Pre-mount connections: [] 2023-11-26T07:22:46.160 DEBUG:teuthology.orchestra.run.smithi188:> (cd /home/ubuntu/cephtest && exec sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-fuse -f --admin-socket '/var/run/ceph/$cluster-$name.$pid.asok' /home/ubuntu/cephtest/mnt.1 --id 1) 2023-11-26T07:22:46.203 DEBUG:teuthology.orchestra.run.smithi188:> sudo modprobe fuse 2023-11-26T07:22:46.224 DEBUG:teuthology.orchestra.run.smithi188:> cat /proc/self/mounts | awk '{print $2}' 2023-11-26T07:22:46.281 INFO:teuthology.orchestra.run.smithi188.stdout:/sys 2023-11-26T07:22:46.281 INFO:teuthology.orchestra.run.smithi188.stdout:/proc 2023-11-26T07:22:46.282 INFO:teuthology.orchestra.run.smithi188.stdout:/dev 2023-11-26T07:22:46.282 INFO:teuthology.orchestra.run.smithi188.stdout:/dev/pts 2023-11-26T07:22:46.282 INFO:teuthology.orchestra.run.smithi188.stdout:/run 2023-11-26T07:22:46.282 INFO:teuthology.orchestra.run.smithi188.stdout:/ 2023-11-26T07:22:46.282 INFO:teuthology.orchestra.run.smithi188.stdout:/sys/kernel/security 2023-11-26T07:22:46.282 INFO:teuthology.orchestra.run.smithi188.stdout:/dev/shm 2023-11-26T07:22:46.282 INFO:teuthology.orchestra.run.smithi188.stdout:/run/lock 2023-11-26T07:22:46.282 INFO:teuthology.orchestra.run.smithi188.stdout:/sys/fs/cgroup 2023-11-26T07:22:46.282 INFO:teuthology.orchestra.run.smithi188.stdout:/sys/fs/pstore 2023-11-26T07:22:46.282 INFO:teuthology.orchestra.run.smithi188.stdout:/sys/fs/bpf 2023-11-26T07:22:46.282 INFO:teuthology.orchestra.run.smithi188.stdout:/proc/sys/fs/binfmt_misc 2023-11-26T07:22:46.283 INFO:teuthology.orchestra.run.smithi188.stdout:/dev/hugepages 2023-11-26T07:22:46.283 INFO:teuthology.orchestra.run.smithi188.stdout:/dev/mqueue 2023-11-26T07:22:46.283 INFO:teuthology.orchestra.run.smithi188.stdout:/sys/kernel/debug 2023-11-26T07:22:46.283 INFO:teuthology.orchestra.run.smithi188.stdout:/sys/kernel/tracing 2023-11-26T07:22:46.283 INFO:teuthology.orchestra.run.smithi188.stdout:/proc/fs/nfsd 2023-11-26T07:22:46.283 INFO:teuthology.orchestra.run.smithi188.stdout:/sys/fs/fuse/connections 2023-11-26T07:22:46.283 INFO:teuthology.orchestra.run.smithi188.stdout:/sys/kernel/config 2023-11-26T07:22:46.283 INFO:teuthology.orchestra.run.smithi188.stdout:/run/credentials/systemd-sysusers.service 2023-11-26T07:22:46.283 INFO:teuthology.orchestra.run.smithi188.stdout:/run/rpc_pipefs 2023-11-26T07:22:46.283 INFO:teuthology.orchestra.run.smithi188.stdout:/run/user/1000 2023-11-26T07:22:46.283 INFO:teuthology.orchestra.run.smithi188.stdout:/var/lib/ceph 2023-11-26T07:22:46.283 INFO:teuthology.orchestra.run.smithi188.stdout:/var/lib/ceph/osd/ceph-4 2023-11-26T07:22:46.284 INFO:teuthology.orchestra.run.smithi188.stdout:/var/lib/ceph/osd/ceph-5 2023-11-26T07:22:46.284 INFO:teuthology.orchestra.run.smithi188.stdout:/var/lib/ceph/osd/ceph-6 2023-11-26T07:22:46.284 INFO:teuthology.orchestra.run.smithi188.stdout:/var/lib/ceph/osd/ceph-7 2023-11-26T07:22:46.284 INFO:teuthology.orchestra.run.smithi188.stdout:/run/netns 2023-11-26T07:22:46.284 INFO:teuthology.orchestra.run.smithi188.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 2023-11-26T07:22:46.284 INFO:teuthology.orchestra.run.smithi188.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 2023-11-26T07:22:46.284 INFO:teuthology.orchestra.run.smithi188.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 2023-11-26T07:22:46.284 INFO:teuthology.orchestra.run.smithi188.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 2023-11-26T07:22:46.285 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-26T07:22:46.285 DEBUG:teuthology.orchestra.run.smithi188:> ls /sys/fs/fuse/connections 2023-11-26T07:22:46.298 INFO:tasks.cephfs.fuse_mount.ceph-fuse.1.smithi188.stderr:2023-11-26T07:22:46.294+0000 7f3225593580 -1 init, newargv = 0x55646ed243c0 newargc=15 2023-11-26T07:22:46.298 INFO:tasks.cephfs.fuse_mount.ceph-fuse.1.smithi188.stderr:ceph-fuse[44878]: starting ceph client 2023-11-26T07:22:46.305 INFO:tasks.cephfs.fuse_mount.ceph-fuse.1.smithi188.stderr:ceph-fuse[44878]: starting fuse 2023-11-26T07:22:46.307 INFO:teuthology.orchestra.run.smithi188.stdout:43 2023-11-26T07:22:46.307 INFO:tasks.cephfs.fuse_mount:Post-mount connections: [43] 2023-11-26T07:22:46.308 DEBUG:teuthology.orchestra.run.smithi188:> sudo stdin-killer -- python3 -c ' 2023-11-26T07:22:46.308 DEBUG:teuthology.orchestra.run.smithi188:> import glob 2023-11-26T07:22:46.308 DEBUG:teuthology.orchestra.run.smithi188:> import re 2023-11-26T07:22:46.308 DEBUG:teuthology.orchestra.run.smithi188:> import os 2023-11-26T07:22:46.308 DEBUG:teuthology.orchestra.run.smithi188:> import subprocess 2023-11-26T07:22:46.308 DEBUG:teuthology.orchestra.run.smithi188:> 2023-11-26T07:22:46.308 DEBUG:teuthology.orchestra.run.smithi188:> def _find_admin_socket(client_name): 2023-11-26T07:22:46.308 DEBUG:teuthology.orchestra.run.smithi188:> asok_path = "/var/run/ceph/ceph-client.1.*.asok" 2023-11-26T07:22:46.308 DEBUG:teuthology.orchestra.run.smithi188:> files = glob.glob(asok_path) 2023-11-26T07:22:46.308 DEBUG:teuthology.orchestra.run.smithi188:> mountpoint = "/home/ubuntu/cephtest/mnt.1" 2023-11-26T07:22:46.308 DEBUG:teuthology.orchestra.run.smithi188:> 2023-11-26T07:22:46.309 DEBUG:teuthology.orchestra.run.smithi188:> # Given a non-glob path, it better be there 2023-11-26T07:22:46.309 DEBUG:teuthology.orchestra.run.smithi188:> if "*" not in asok_path: 2023-11-26T07:22:46.309 DEBUG:teuthology.orchestra.run.smithi188:> assert(len(files) == 1) 2023-11-26T07:22:46.309 DEBUG:teuthology.orchestra.run.smithi188:> return files[0] 2023-11-26T07:22:46.309 DEBUG:teuthology.orchestra.run.smithi188:> 2023-11-26T07:22:46.309 DEBUG:teuthology.orchestra.run.smithi188:> for f in files: 2023-11-26T07:22:46.309 DEBUG:teuthology.orchestra.run.smithi188:> pid = re.match(".*\.(\d+)\.asok$", f).group(1) 2023-11-26T07:22:46.309 DEBUG:teuthology.orchestra.run.smithi188:> if os.path.exists("/proc/{0}".format(pid)): 2023-11-26T07:22:46.309 DEBUG:teuthology.orchestra.run.smithi188:> with open("/proc/{0}/cmdline".format(pid), '"'"'r'"'"') as proc_f: 2023-11-26T07:22:46.309 DEBUG:teuthology.orchestra.run.smithi188:> contents = proc_f.read() 2023-11-26T07:22:46.309 DEBUG:teuthology.orchestra.run.smithi188:> if mountpoint in contents: 2023-11-26T07:22:46.309 DEBUG:teuthology.orchestra.run.smithi188:> return f 2023-11-26T07:22:46.309 DEBUG:teuthology.orchestra.run.smithi188:> raise RuntimeError("Client socket {0} not found".format(client_name)) 2023-11-26T07:22:46.309 DEBUG:teuthology.orchestra.run.smithi188:> 2023-11-26T07:22:46.310 DEBUG:teuthology.orchestra.run.smithi188:> print(_find_admin_socket("client.1")) 2023-11-26T07:22:46.310 DEBUG:teuthology.orchestra.run.smithi188:> ' 2023-11-26T07:22:46.428 INFO:teuthology.orchestra.run.smithi188.stdout:/var/run/ceph/ceph-client.1.44878.asok 2023-11-26T07:22:46.431 INFO:teuthology.orchestra.run.smithi188.stderr:2023-11-26T07:22:46 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2023-11-26T07:22:46.439 INFO:tasks.cephfs.fuse_mount:Found client admin socket at /var/run/ceph/ceph-client.1.44878.asok 2023-11-26T07:22:46.439 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-26T07:22:46.439 DEBUG:teuthology.orchestra.run.smithi188:> sudo ceph --admin-daemon /var/run/ceph/ceph-client.1.44878.asok status 2023-11-26T07:22:46.611 INFO:teuthology.orchestra.run.smithi188.stdout:{ 2023-11-26T07:22:46.611 INFO:teuthology.orchestra.run.smithi188.stdout: "metadata": { 2023-11-26T07:22:46.611 INFO:teuthology.orchestra.run.smithi188.stdout: "ceph_sha1": "44c2237fe77713214148c1fd622c872f346f1b99", 2023-11-26T07:22:46.611 INFO:teuthology.orchestra.run.smithi188.stdout: "ceph_version": "ceph version 18.0.0-7484-g44c2237f (44c2237fe77713214148c1fd622c872f346f1b99) reef (dev)", 2023-11-26T07:22:46.611 INFO:teuthology.orchestra.run.smithi188.stdout: "entity_id": "1", 2023-11-26T07:22:46.611 INFO:teuthology.orchestra.run.smithi188.stdout: "hostname": "smithi188", 2023-11-26T07:22:46.611 INFO:teuthology.orchestra.run.smithi188.stdout: "mount_point": "/home/ubuntu/cephtest/mnt.1", 2023-11-26T07:22:46.611 INFO:teuthology.orchestra.run.smithi188.stdout: "pid": "44878", 2023-11-26T07:22:46.612 INFO:teuthology.orchestra.run.smithi188.stdout: "root": "/" 2023-11-26T07:22:46.612 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2023-11-26T07:22:46.612 INFO:teuthology.orchestra.run.smithi188.stdout: "dentry_count": 0, 2023-11-26T07:22:46.612 INFO:teuthology.orchestra.run.smithi188.stdout: "dentry_pinned_count": 0, 2023-11-26T07:22:46.612 INFO:teuthology.orchestra.run.smithi188.stdout: "id": 6709, 2023-11-26T07:22:46.612 INFO:teuthology.orchestra.run.smithi188.stdout: "inst": { 2023-11-26T07:22:46.612 INFO:teuthology.orchestra.run.smithi188.stdout: "name": { 2023-11-26T07:22:46.612 INFO:teuthology.orchestra.run.smithi188.stdout: "type": "client", 2023-11-26T07:22:46.612 INFO:teuthology.orchestra.run.smithi188.stdout: "num": 6709 2023-11-26T07:22:46.612 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2023-11-26T07:22:46.612 INFO:teuthology.orchestra.run.smithi188.stdout: "addr": { 2023-11-26T07:22:46.612 INFO:teuthology.orchestra.run.smithi188.stdout: "type": "v1", 2023-11-26T07:22:46.613 INFO:teuthology.orchestra.run.smithi188.stdout: "addr": "192.168.0.1:0", 2023-11-26T07:22:46.613 INFO:teuthology.orchestra.run.smithi188.stdout: "nonce": 2943348720 2023-11-26T07:22:46.613 INFO:teuthology.orchestra.run.smithi188.stdout: } 2023-11-26T07:22:46.613 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2023-11-26T07:22:46.613 INFO:teuthology.orchestra.run.smithi188.stdout: "addr": { 2023-11-26T07:22:46.613 INFO:teuthology.orchestra.run.smithi188.stdout: "type": "v1", 2023-11-26T07:22:46.613 INFO:teuthology.orchestra.run.smithi188.stdout: "addr": "192.168.0.1:0", 2023-11-26T07:22:46.613 INFO:teuthology.orchestra.run.smithi188.stdout: "nonce": 2943348720 2023-11-26T07:22:46.613 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2023-11-26T07:22:46.613 INFO:teuthology.orchestra.run.smithi188.stdout: "inst_str": "client.6709 192.168.0.1:0/2943348720", 2023-11-26T07:22:46.613 INFO:teuthology.orchestra.run.smithi188.stdout: "addr_str": "192.168.0.1:0/2943348720", 2023-11-26T07:22:46.613 INFO:teuthology.orchestra.run.smithi188.stdout: "inode_count": 1, 2023-11-26T07:22:46.614 INFO:teuthology.orchestra.run.smithi188.stdout: "mds_epoch": 42, 2023-11-26T07:22:46.614 INFO:teuthology.orchestra.run.smithi188.stdout: "osd_epoch": 71, 2023-11-26T07:22:46.614 INFO:teuthology.orchestra.run.smithi188.stdout: "osd_epoch_barrier": 0, 2023-11-26T07:22:46.614 INFO:teuthology.orchestra.run.smithi188.stdout: "blocklisted": false, 2023-11-26T07:22:46.614 INFO:teuthology.orchestra.run.smithi188.stdout: "fs_name": "cephfs" 2023-11-26T07:22:46.614 INFO:teuthology.orchestra.run.smithi188.stdout:} 2023-11-26T07:22:46.625 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-26T07:22:46.626 DEBUG:teuthology.orchestra.run.smithi188:> stat --file-system '--printf=%T 2023-11-26T07:22:46.626 DEBUG:teuthology.orchestra.run.smithi188:> ' -- /home/ubuntu/cephtest/mnt.1 2023-11-26T07:22:46.635 INFO:teuthology.orchestra.run.smithi188.stdout:fuseblk 2023-11-26T07:22:46.635 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.1 2023-11-26T07:22:46.636 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-26T07:22:46.636 DEBUG:teuthology.orchestra.run.smithi188:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.1 2023-11-26T07:22:46.695 DEBUG:teuthology.orchestra.run.smithi143:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs flag set enable_multiple true --yes-i-really-mean-it 2023-11-26T07:22:46.839 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:46.836+0000 7f0a22d2a640 1 Processor -- start 2023-11-26T07:22:46.839 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:46.836+0000 7f0a22d2a640 1 -- start start 2023-11-26T07:22:46.840 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:46.840+0000 7f0a22d2a640 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7f0a1c0cf280 0x7f0a1c0cf710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:22:46.841 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:46.840+0000 7f0a22d2a640 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7f0a1c0cd1b0 0x7f0a1c0cd5c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:22:46.841 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:46.840+0000 7f0a22d2a640 1 -- --> v1:172.21.15.188:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0a1c0cc730 con 0x7f0a1c143ac0 2023-11-26T07:22:46.841 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:46.840+0000 7f0a22d2a640 1 -- --> v2:172.21.15.143:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0a1c143ea0 con 0x7f0a1c0cf280 2023-11-26T07:22:46.841 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:46.840+0000 7f0a22d2a640 1 -- --> v2:172.21.15.188:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0a1c0cfc50 con 0x7f0a1c0cd1b0 2023-11-26T07:22:46.841 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:46.840+0000 7f0a13fff640 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7f0a1c0cf280 0x7f0a1c0cf710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:22:46.841 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:46.840+0000 7f0a20ac6640 1 --1- >> v1:172.21.15.188:6789/0 conn(0x7f0a1c143ac0 0x7f0a1c070370 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.188:6789/0 says I am v1:172.21.15.143:50356/0 (socket says 172.21.15.143:50356) 2023-11-26T07:22:46.841 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:46.840+0000 7f0a20ac6640 1 -- 172.21.15.143:0/730125261 learned_addr learned my addr 172.21.15.143:0/730125261 (peer_addr_for_me v1:172.21.15.143:0/0) 2023-11-26T07:22:46.841 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:46.840+0000 7f0a13fff640 1 --2- 172.21.15.143:0/730125261 >> v2:172.21.15.143:3300/0 conn(0x7f0a1c0cf280 0x7f0a1c0cf710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:50342/0 (socket says 172.21.15.143:50342) 2023-11-26T07:22:46.841 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:46.840+0000 7f0a212c7640 1 --2- 172.21.15.143:0/730125261 >> v2:172.21.15.188:3300/0 conn(0x7f0a1c0cd1b0 0x7f0a1c0cd5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:22:46.842 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:46.840+0000 7f0a13fff640 1 -- 172.21.15.143:0/730125261 >> v2:172.21.15.188:3300/0 conn(0x7f0a1c0cd1b0 msgr2=0x7f0a1c0cd5c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-26T07:22:46.842 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:46.840+0000 7f0a13fff640 1 --2- 172.21.15.143:0/730125261 >> v2:172.21.15.188:3300/0 conn(0x7f0a1c0cd1b0 0x7f0a1c0cd5c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:22:46.842 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:46.840+0000 7f0a13fff640 1 -- 172.21.15.143:0/730125261 >> v1:172.21.15.188:6789/0 conn(0x7f0a1c143ac0 legacy=0x7f0a1c070370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:22:46.842 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:46.840+0000 7f0a137fe640 1 -- 172.21.15.143:0/730125261 <== mon.1 v1:172.21.15.188:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3340517771 0 0) 0x7f0a1c0cc730 con 0x7f0a1c143ac0 2023-11-26T07:22:46.842 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:46.840+0000 7f0a13fff640 1 -- 172.21.15.143:0/730125261 --> v2:172.21.15.143:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0a1c070ee0 con 0x7f0a1c0cf280 2023-11-26T07:22:46.844 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:46.844+0000 7f0a13fff640 1 --2- 172.21.15.143:0/730125261 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f0a1c0cf280 0x7f0a1c0cf710 secure :-1 s=READY pgs=986 cs=0 l=1 rev1=1 crypto rx=0x7f0a1c06e570 tx=0x7f0a0c01c460 comp rx=0 tx=0).ready entity=mon.0 client_cookie=de3aea2862df25f server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:22:46.844 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:46.844+0000 7f0a137fe640 1 -- 172.21.15.143:0/730125261 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0a0c0ab660 con 0x7f0a1c0cf280 2023-11-26T07:22:46.844 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:46.844+0000 7f0a137fe640 1 -- 172.21.15.143:0/730125261 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0a0c0ab7c0 con 0x7f0a1c0cf280 2023-11-26T07:22:46.844 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:46.844+0000 7f0a137fe640 1 -- 172.21.15.143:0/730125261 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0a0c0abb50 con 0x7f0a1c0cf280 2023-11-26T07:22:46.845 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:46.844+0000 7f0a22d2a640 1 -- 172.21.15.143:0/730125261 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f0a1c0cf280 msgr2=0x7f0a1c0cf710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:22:46.845 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:46.844+0000 7f0a22d2a640 1 --2- 172.21.15.143:0/730125261 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f0a1c0cf280 0x7f0a1c0cf710 secure :-1 s=READY pgs=986 cs=0 l=1 rev1=1 crypto rx=0x7f0a1c06e570 tx=0x7f0a0c01c460 comp rx=0 tx=0).stop 2023-11-26T07:22:46.845 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:46.844+0000 7f0a22d2a640 1 -- 172.21.15.143:0/730125261 shutdown_connections 2023-11-26T07:22:46.845 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:46.844+0000 7f0a22d2a640 1 --2- 172.21.15.143:0/730125261 >> v2:172.21.15.188:3300/0 conn(0x7f0a1c0cd1b0 0x7f0a1c0cd5c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:22:46.845 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:46.844+0000 7f0a22d2a640 1 --2- 172.21.15.143:0/730125261 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f0a1c0cf280 0x7f0a1c0cf710 unknown :-1 s=CLOSED pgs=986 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:22:46.845 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:46.844+0000 7f0a22d2a640 1 -- 172.21.15.143:0/730125261 >> 172.21.15.143:0/730125261 conn(0x7f0a1c06a890 msgr2=0x7f0a1c06aca0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:22:46.846 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:46.844+0000 7f0a22d2a640 1 -- 172.21.15.143:0/730125261 shutdown_connections 2023-11-26T07:22:46.846 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:46.844+0000 7f0a22d2a640 1 -- 172.21.15.143:0/730125261 wait complete. 2023-11-26T07:22:46.846 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:46.844+0000 7f0a22d2a640 1 Processor -- start 2023-11-26T07:22:46.846 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:46.844+0000 7f0a22d2a640 1 -- start start 2023-11-26T07:22:46.848 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:46.848+0000 7f0a22d2a640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f0a1c0cd1b0 0x7f0a1c1582a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:22:46.849 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:46.848+0000 7f0a22d2a640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f0a1c0cf280 0x7f0a1c13bc70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:22:46.849 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:46.848+0000 7f0a22d2a640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f0a1c143ac0 0x7f0a1c0ab9d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:22:46.849 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:46.848+0000 7f0a22d2a640 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0a1c13c1b0 con 0x7f0a1c0cd1b0 2023-11-26T07:22:46.849 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:46.848+0000 7f0a22d2a640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0a1c13d200 con 0x7f0a1c0cf280 2023-11-26T07:22:46.849 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:46.848+0000 7f0a22d2a640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0a1c13f9e0 con 0x7f0a1c143ac0 2023-11-26T07:22:46.849 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:46.848+0000 7f0a20ac6640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f0a1c0cd1b0 0x7f0a1c1582a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:22:46.849 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:46.848+0000 7f0a20ac6640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f0a1c0cd1b0 0x7f0a1c1582a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:50350/0 (socket says 172.21.15.143:50350) 2023-11-26T07:22:46.849 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:46.848+0000 7f0a20ac6640 1 -- 172.21.15.143:0/3322169661 learned_addr learned my addr 172.21.15.143:0/3322169661 (peer_addr_for_me v2:172.21.15.143:0/0) 2023-11-26T07:22:46.849 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:46.848+0000 7f0a13fff640 1 --2- 172.21.15.143:0/3322169661 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f0a1c0cf280 0x7f0a1c13bc70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:22:46.849 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:46.848+0000 7f0a212c7640 1 --2- 172.21.15.143:0/3322169661 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f0a1c143ac0 0x7f0a1c0ab9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:22:46.849 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:46.848+0000 7f0a20ac6640 1 -- 172.21.15.143:0/3322169661 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f0a1c143ac0 msgr2=0x7f0a1c0ab9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:22:46.850 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:46.848+0000 7f0a20ac6640 1 --2- 172.21.15.143:0/3322169661 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f0a1c143ac0 0x7f0a1c0ab9d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:22:46.850 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:46.848+0000 7f0a20ac6640 1 -- 172.21.15.143:0/3322169661 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f0a1c0cf280 msgr2=0x7f0a1c13bc70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:22:46.850 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:46.848+0000 7f0a20ac6640 1 --2- 172.21.15.143:0/3322169661 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f0a1c0cf280 0x7f0a1c13bc70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:22:46.850 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:46.848+0000 7f0a20ac6640 1 -- 172.21.15.143:0/3322169661 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0a0c009420 con 0x7f0a1c0cd1b0 2023-11-26T07:22:46.850 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:46.848+0000 7f0a20ac6640 1 --2- 172.21.15.143:0/3322169661 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f0a1c0cd1b0 0x7f0a1c1582a0 secure :-1 s=READY pgs=987 cs=0 l=1 rev1=1 crypto rx=0x7f0a180011f0 tx=0x7f0a1800c9b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:22:46.851 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:46.848+0000 7f0a117fa640 1 -- 172.21.15.143:0/3322169661 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0a180074e0 con 0x7f0a1c0cd1b0 2023-11-26T07:22:46.851 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:46.848+0000 7f0a117fa640 1 -- 172.21.15.143:0/3322169661 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0a18007640 con 0x7f0a1c0cd1b0 2023-11-26T07:22:46.851 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:46.848+0000 7f0a22d2a640 1 -- 172.21.15.143:0/3322169661 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0a1c13e6e0 con 0x7f0a1c0cd1b0 2023-11-26T07:22:46.851 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:46.848+0000 7f0a117fa640 1 -- 172.21.15.143:0/3322169661 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0a18004a10 con 0x7f0a1c0cd1b0 2023-11-26T07:22:46.851 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:46.848+0000 7f0a22d2a640 1 -- 172.21.15.143:0/3322169661 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0a1c13eca0 con 0x7f0a1c0cd1b0 2023-11-26T07:22:46.851 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:46.848+0000 7f0a22d2a640 1 -- 172.21.15.143:0/3322169661 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0a1c067680 con 0x7f0a1c0cd1b0 2023-11-26T07:22:46.851 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:46.848+0000 7f0a117fa640 1 -- 172.21.15.143:0/3322169661 <== mon.0 v2:172.21.15.143:3300/0 4 ==== mgrmap(e 11) v1 ==== 86158+0+0 (secure 0 0 0) 0x7f0a18004b70 con 0x7f0a1c0cd1b0 2023-11-26T07:22:46.852 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:46.852+0000 7f0a117fa640 1 --2- 172.21.15.143:0/3322169661 >> [v2:172.21.15.143:6832/2150557544,v1:172.21.15.143:6833/2150557544] conn(0x7f0a080667d0 0x7f0a08068c90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:22:46.852 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:46.852+0000 7f0a117fa640 1 -- 172.21.15.143:0/3322169661 <== mon.0 v2:172.21.15.143:3300/0 5 ==== osd_map(71..71 src has 1..71) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f0a18083950 con 0x7f0a1c0cd1b0 2023-11-26T07:22:46.852 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:46.852+0000 7f0a13fff640 1 --2- 172.21.15.143:0/3322169661 >> [v2:172.21.15.143:6832/2150557544,v1:172.21.15.143:6833/2150557544] conn(0x7f0a080667d0 0x7f0a08068c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:22:46.853 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:46.852+0000 7f0a13fff640 1 --2- 172.21.15.143:0/3322169661 >> [v2:172.21.15.143:6832/2150557544,v1:172.21.15.143:6833/2150557544] conn(0x7f0a080667d0 0x7f0a08068c90 secure :-1 s=READY pgs=459 cs=0 l=1 rev1=1 crypto rx=0x7f0a0c0af000 tx=0x7f0a0c0a7000 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:22:46.856 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:46.856+0000 7f0a117fa640 1 -- 172.21.15.143:0/3322169661 <== mon.0 v2:172.21.15.143:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172649 (secure 0 0 0) 0x7f0a180509c0 con 0x7f0a1c0cd1b0 2023-11-26T07:22:47.042 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:47.040+0000 7f0a22d2a640 1 -- 172.21.15.143:0/3322169661 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "fs flag set", "flag_name": "enable_multiple", "val": "true", "yes_i_really_mean_it": true} v 0) v1 -- 0x7f0a1c071be0 con 0x7f0a1c0cd1b0 2023-11-26T07:22:47.913 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:47.911+0000 7f0a117fa640 1 -- 172.21.15.143:0/3322169661 <== mon.0 v2:172.21.15.143:3300/0 7 ==== mon_command_ack([{"prefix": "fs flag set", "flag_name": "enable_multiple", "val": "true", "yes_i_really_mean_it": true}]=0 v43) v1 ==== 136+0+0 (secure 0 0 0) 0x7f0a180180c0 con 0x7f0a1c0cd1b0 2023-11-26T07:22:47.916 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:47.915+0000 7f0a22d2a640 1 -- 172.21.15.143:0/3322169661 >> [v2:172.21.15.143:6832/2150557544,v1:172.21.15.143:6833/2150557544] conn(0x7f0a080667d0 msgr2=0x7f0a08068c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:22:47.916 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:47.915+0000 7f0a22d2a640 1 --2- 172.21.15.143:0/3322169661 >> [v2:172.21.15.143:6832/2150557544,v1:172.21.15.143:6833/2150557544] conn(0x7f0a080667d0 0x7f0a08068c90 secure :-1 s=READY pgs=459 cs=0 l=1 rev1=1 crypto rx=0x7f0a0c0af000 tx=0x7f0a0c0a7000 comp rx=0 tx=0).stop 2023-11-26T07:22:47.917 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:47.915+0000 7f0a22d2a640 1 -- 172.21.15.143:0/3322169661 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f0a1c0cd1b0 msgr2=0x7f0a1c1582a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:22:47.917 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:47.915+0000 7f0a22d2a640 1 --2- 172.21.15.143:0/3322169661 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f0a1c0cd1b0 0x7f0a1c1582a0 secure :-1 s=READY pgs=987 cs=0 l=1 rev1=1 crypto rx=0x7f0a180011f0 tx=0x7f0a1800c9b0 comp rx=0 tx=0).stop 2023-11-26T07:22:47.917 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:47.915+0000 7f0a22d2a640 1 -- 172.21.15.143:0/3322169661 shutdown_connections 2023-11-26T07:22:47.917 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:47.915+0000 7f0a22d2a640 1 --2- 172.21.15.143:0/3322169661 >> [v2:172.21.15.143:6832/2150557544,v1:172.21.15.143:6833/2150557544] conn(0x7f0a080667d0 0x7f0a08068c90 unknown :-1 s=CLOSED pgs=459 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:22:47.917 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:47.915+0000 7f0a22d2a640 1 --2- 172.21.15.143:0/3322169661 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f0a1c143ac0 0x7f0a1c0ab9d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:22:47.917 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:47.915+0000 7f0a22d2a640 1 --2- 172.21.15.143:0/3322169661 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f0a1c0cf280 0x7f0a1c13bc70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:22:47.917 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:47.915+0000 7f0a22d2a640 1 --2- 172.21.15.143:0/3322169661 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f0a1c0cd1b0 0x7f0a1c1582a0 unknown :-1 s=CLOSED pgs=987 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:22:47.917 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:47.915+0000 7f0a22d2a640 1 -- 172.21.15.143:0/3322169661 >> 172.21.15.143:0/3322169661 conn(0x7f0a1c06a890 msgr2=0x7f0a1c140610 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:22:47.917 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:47.915+0000 7f0a22d2a640 1 -- 172.21.15.143:0/3322169661 shutdown_connections 2023-11-26T07:22:47.917 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:47.915+0000 7f0a22d2a640 1 -- 172.21.15.143:0/3322169661 wait complete. 2023-11-26T07:22:47.937 DEBUG:teuthology.orchestra.run.smithi143:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs volume ls 2023-11-26T07:22:48.078 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:48.075+0000 7f048b53e640 1 Processor -- start 2023-11-26T07:22:48.078 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:48.075+0000 7f048b53e640 1 -- start start 2023-11-26T07:22:48.079 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:48.075+0000 7f048b53e640 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7f04840cf280 0x7f04840cf710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:22:48.079 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:48.079+0000 7f048b53e640 1 -- --> v1:172.21.15.143:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f04840cc730 con 0x7f04840cd1b0 2023-11-26T07:22:48.079 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:48.079+0000 7f048b53e640 1 -- --> v1:172.21.15.188:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0484070ee0 con 0x7f0484143ac0 2023-11-26T07:22:48.079 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:48.079+0000 7f048b53e640 1 -- --> v2:172.21.15.188:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0484143ea0 con 0x7f04840cf280 2023-11-26T07:22:48.080 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:48.079+0000 7f0489adb640 1 --1- >> v1:172.21.15.143:6789/0 conn(0x7f04840cd1b0 0x7f04840cd5c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.143:6789/0 says I am v1:172.21.15.143:47972/0 (socket says 172.21.15.143:47972) 2023-11-26T07:22:48.080 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:48.079+0000 7f0489adb640 1 -- 172.21.15.143:0/3214286227 learned_addr learned my addr 172.21.15.143:0/3214286227 (peer_addr_for_me v1:172.21.15.143:0/0) 2023-11-26T07:22:48.080 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:48.079+0000 7f0488ad9640 1 --2- 172.21.15.143:0/3214286227 >> v2:172.21.15.188:3300/0 conn(0x7f04840cf280 0x7f04840cf710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:22:48.080 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:48.079+0000 7f047bfff640 1 -- 172.21.15.143:0/3214286227 <== mon.0 v1:172.21.15.143:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3369976737 0 0) 0x7f04840cc730 con 0x7f04840cd1b0 2023-11-26T07:22:48.081 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:48.079+0000 7f047bfff640 1 -- 172.21.15.143:0/3214286227 --> v1:172.21.15.143:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f0470003530 con 0x7f04840cd1b0 2023-11-26T07:22:48.081 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:48.079+0000 7f0488ad9640 1 -- 172.21.15.143:0/3214286227 >> v1:172.21.15.188:6789/0 conn(0x7f0484143ac0 legacy=0x7f0484070370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:22:48.081 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:48.079+0000 7f0488ad9640 1 -- 172.21.15.143:0/3214286227 >> v1:172.21.15.143:6789/0 conn(0x7f04840cd1b0 legacy=0x7f04840cd5c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:22:48.081 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:48.079+0000 7f0488ad9640 1 -- 172.21.15.143:0/3214286227 --> v2:172.21.15.188:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f04840711c0 con 0x7f04840cf280 2023-11-26T07:22:48.083 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:48.079+0000 7f0488ad9640 1 --2- 172.21.15.143:0/3214286227 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f04840cf280 0x7f04840cf710 secure :-1 s=READY pgs=394 cs=0 l=1 rev1=1 crypto rx=0x7f04740097c0 tx=0x7f047401c480 comp rx=0 tx=0).ready entity=mon.1 client_cookie=970aef6ac1af5c68 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:22:48.083 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:48.083+0000 7f047bfff640 1 -- 172.21.15.143:0/3214286227 <== mon.1 v2:172.21.15.188:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f04740a8050 con 0x7f04840cf280 2023-11-26T07:22:48.083 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:48.083+0000 7f047bfff640 1 -- 172.21.15.143:0/3214286227 <== mon.1 v2:172.21.15.188:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f04740ac5f0 con 0x7f04840cf280 2023-11-26T07:22:48.083 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:48.083+0000 7f047bfff640 1 -- 172.21.15.143:0/3214286227 <== mon.1 v2:172.21.15.188:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f04740ac980 con 0x7f04840cf280 2023-11-26T07:22:48.084 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:48.083+0000 7f048b53e640 1 -- 172.21.15.143:0/3214286227 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f04840cf280 msgr2=0x7f04840cf710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:22:48.084 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:48.083+0000 7f048b53e640 1 --2- 172.21.15.143:0/3214286227 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f04840cf280 0x7f04840cf710 secure :-1 s=READY pgs=394 cs=0 l=1 rev1=1 crypto rx=0x7f04740097c0 tx=0x7f047401c480 comp rx=0 tx=0).stop 2023-11-26T07:22:48.084 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:48.083+0000 7f048b53e640 1 -- 172.21.15.143:0/3214286227 shutdown_connections 2023-11-26T07:22:48.084 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:48.083+0000 7f048b53e640 1 --2- 172.21.15.143:0/3214286227 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f04840cf280 0x7f04840cf710 unknown :-1 s=CLOSED pgs=394 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:22:48.085 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:48.083+0000 7f048b53e640 1 -- 172.21.15.143:0/3214286227 >> 172.21.15.143:0/3214286227 conn(0x7f048406a890 msgr2=0x7f048406aca0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:22:48.085 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:48.083+0000 7f048b53e640 1 -- 172.21.15.143:0/3214286227 shutdown_connections 2023-11-26T07:22:48.085 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:48.083+0000 7f048b53e640 1 -- 172.21.15.143:0/3214286227 wait complete. 2023-11-26T07:22:48.086 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:48.083+0000 7f048b53e640 1 Processor -- start 2023-11-26T07:22:48.086 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:48.083+0000 7f048b53e640 1 -- start start 2023-11-26T07:22:48.087 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:48.083+0000 7f048b53e640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f04840cd1b0 0x7f048413afa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:22:48.088 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:48.083+0000 7f048b53e640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f04840cf280 0x7f0484157160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:22:48.088 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:48.083+0000 7f048b53e640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f0484143ac0 0x7f0484158130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:22:48.088 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:48.083+0000 7f048b53e640 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f04841576a0 con 0x7f04840cd1b0 2023-11-26T07:22:48.088 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:48.083+0000 7f048b53e640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0484139540 con 0x7f0484143ac0 2023-11-26T07:22:48.088 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:48.083+0000 7f048b53e640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0484139820 con 0x7f04840cf280 2023-11-26T07:22:48.088 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:48.083+0000 7f04892da640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f04840cd1b0 0x7f048413afa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:22:48.088 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:48.087+0000 7f04892da640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f04840cd1b0 0x7f048413afa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:50362/0 (socket says 172.21.15.143:50362) 2023-11-26T07:22:48.088 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:48.087+0000 7f04892da640 1 -- 172.21.15.143:0/1434910069 learned_addr learned my addr 172.21.15.143:0/1434910069 (peer_addr_for_me v2:172.21.15.143:0/0) 2023-11-26T07:22:48.088 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:48.087+0000 7f0489adb640 1 --2- 172.21.15.143:0/1434910069 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f0484143ac0 0x7f0484158130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:22:48.088 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:48.087+0000 7f0488ad9640 1 --2- 172.21.15.143:0/1434910069 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f04840cf280 0x7f0484157160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:22:48.088 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:48.087+0000 7f04892da640 1 -- 172.21.15.143:0/1434910069 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f04840cf280 msgr2=0x7f0484157160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:22:48.088 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:48.087+0000 7f04892da640 1 --2- 172.21.15.143:0/1434910069 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f04840cf280 0x7f0484157160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:22:48.089 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:48.087+0000 7f04892da640 1 -- 172.21.15.143:0/1434910069 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f0484143ac0 msgr2=0x7f0484158130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:22:48.089 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:48.087+0000 7f04892da640 1 --2- 172.21.15.143:0/1434910069 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f0484143ac0 0x7f0484158130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:22:48.089 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:48.087+0000 7f04892da640 1 -- 172.21.15.143:0/1434910069 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0474009420 con 0x7f04840cd1b0 2023-11-26T07:22:48.090 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:48.087+0000 7f04892da640 1 --2- 172.21.15.143:0/1434910069 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f04840cd1b0 0x7f048413afa0 secure :-1 s=READY pgs=989 cs=0 l=1 rev1=1 crypto rx=0x7f048000c3c0 tx=0x7f048000c880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:22:48.090 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:48.087+0000 7f0479ffb640 1 -- 172.21.15.143:0/1434910069 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f04800052d0 con 0x7f04840cd1b0 2023-11-26T07:22:48.090 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:48.087+0000 7f0479ffb640 1 -- 172.21.15.143:0/1434910069 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0480005430 con 0x7f04840cd1b0 2023-11-26T07:22:48.090 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:48.087+0000 7f048b53e640 1 -- 172.21.15.143:0/1434910069 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0484139b00 con 0x7f04840cd1b0 2023-11-26T07:22:48.090 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:48.087+0000 7f0479ffb640 1 -- 172.21.15.143:0/1434910069 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0480003840 con 0x7f04840cd1b0 2023-11-26T07:22:48.090 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:48.087+0000 7f048b53e640 1 -- 172.21.15.143:0/1434910069 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f048413a0c0 con 0x7f04840cd1b0 2023-11-26T07:22:48.090 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:48.087+0000 7f048b53e640 1 -- 172.21.15.143:0/1434910069 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f04840af030 con 0x7f04840cd1b0 2023-11-26T07:22:48.090 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:48.087+0000 7f0479ffb640 1 -- 172.21.15.143:0/1434910069 <== mon.0 v2:172.21.15.143:3300/0 4 ==== mgrmap(e 11) v1 ==== 86158+0+0 (secure 0 0 0) 0x7f04800039a0 con 0x7f04840cd1b0 2023-11-26T07:22:48.091 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:48.087+0000 7f0479ffb640 1 --2- 172.21.15.143:0/1434910069 >> [v2:172.21.15.143:6832/2150557544,v1:172.21.15.143:6833/2150557544] conn(0x7f04680667d0 0x7f0468068c90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:22:48.091 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:48.087+0000 7f0479ffb640 1 -- 172.21.15.143:0/1434910069 <== mon.0 v2:172.21.15.143:3300/0 5 ==== osd_map(71..71 src has 1..71) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f04800838a0 con 0x7f04840cd1b0 2023-11-26T07:22:48.091 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:48.091+0000 7f0488ad9640 1 --2- 172.21.15.143:0/1434910069 >> [v2:172.21.15.143:6832/2150557544,v1:172.21.15.143:6833/2150557544] conn(0x7f04680667d0 0x7f0468068c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:22:48.092 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:48.091+0000 7f0488ad9640 1 --2- 172.21.15.143:0/1434910069 >> [v2:172.21.15.143:6832/2150557544,v1:172.21.15.143:6833/2150557544] conn(0x7f04680667d0 0x7f0468068c90 secure :-1 s=READY pgs=460 cs=0 l=1 rev1=1 crypto rx=0x7f04740ae000 tx=0x7f0474008040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:22:48.095 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:48.091+0000 7f0479ffb640 1 -- 172.21.15.143:0/1434910069 <== mon.0 v2:172.21.15.143:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172649 (secure 0 0 0) 0x7f0480050980 con 0x7f04840cd1b0 2023-11-26T07:22:48.263 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:48.263+0000 7f048b53e640 1 -- 172.21.15.143:0/1434910069 --> [v2:172.21.15.143:6832/2150557544,v1:172.21.15.143:6833/2150557544] -- mgr_command(tid 0: {"prefix": "fs volume ls", "target": ["mon-mgr", ""]}) v1 -- 0x7f0484167be0 con 0x7f04680667d0 2023-11-26T07:22:48.265 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:48.263+0000 7f0479ffb640 1 -- 172.21.15.143:0/1434910069 <== mgr.4100 v2:172.21.15.143:6832/2150557544 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+41 (secure 0 0 0) 0x7f04740bd080 con 0x7f04680667d0 2023-11-26T07:22:48.265 INFO:teuthology.orchestra.run.smithi143.stdout:[ 2023-11-26T07:22:48.265 INFO:teuthology.orchestra.run.smithi143.stdout: { 2023-11-26T07:22:48.265 INFO:teuthology.orchestra.run.smithi143.stdout: "name": "cephfs" 2023-11-26T07:22:48.266 INFO:teuthology.orchestra.run.smithi143.stdout: } 2023-11-26T07:22:48.266 INFO:teuthology.orchestra.run.smithi143.stdout:] 2023-11-26T07:22:48.267 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:48.263+0000 7f048b53e640 1 -- 172.21.15.143:0/1434910069 >> [v2:172.21.15.143:6832/2150557544,v1:172.21.15.143:6833/2150557544] conn(0x7f04680667d0 msgr2=0x7f0468068c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:22:48.267 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:48.263+0000 7f048b53e640 1 --2- 172.21.15.143:0/1434910069 >> [v2:172.21.15.143:6832/2150557544,v1:172.21.15.143:6833/2150557544] conn(0x7f04680667d0 0x7f0468068c90 secure :-1 s=READY pgs=460 cs=0 l=1 rev1=1 crypto rx=0x7f04740ae000 tx=0x7f0474008040 comp rx=0 tx=0).stop 2023-11-26T07:22:48.268 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:48.263+0000 7f048b53e640 1 -- 172.21.15.143:0/1434910069 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f04840cd1b0 msgr2=0x7f048413afa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:22:48.268 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:48.263+0000 7f048b53e640 1 --2- 172.21.15.143:0/1434910069 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f04840cd1b0 0x7f048413afa0 secure :-1 s=READY pgs=989 cs=0 l=1 rev1=1 crypto rx=0x7f048000c3c0 tx=0x7f048000c880 comp rx=0 tx=0).stop 2023-11-26T07:22:48.268 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:48.263+0000 7f048b53e640 1 -- 172.21.15.143:0/1434910069 shutdown_connections 2023-11-26T07:22:48.268 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:48.263+0000 7f048b53e640 1 --2- 172.21.15.143:0/1434910069 >> [v2:172.21.15.143:6832/2150557544,v1:172.21.15.143:6833/2150557544] conn(0x7f04680667d0 0x7f0468068c90 unknown :-1 s=CLOSED pgs=460 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:22:48.268 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:48.263+0000 7f048b53e640 1 --2- 172.21.15.143:0/1434910069 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f0484143ac0 0x7f0484158130 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:22:48.268 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:48.263+0000 7f048b53e640 1 --2- 172.21.15.143:0/1434910069 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f04840cf280 0x7f0484157160 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:22:48.268 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:48.263+0000 7f048b53e640 1 --2- 172.21.15.143:0/1434910069 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f04840cd1b0 0x7f048413afa0 unknown :-1 s=CLOSED pgs=989 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:22:48.268 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:48.263+0000 7f048b53e640 1 -- 172.21.15.143:0/1434910069 >> 172.21.15.143:0/1434910069 conn(0x7f048406a890 msgr2=0x7f0484072130 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:22:48.268 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:48.267+0000 7f048b53e640 1 -- 172.21.15.143:0/1434910069 shutdown_connections 2023-11-26T07:22:48.268 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:48.267+0000 7f048b53e640 1 -- 172.21.15.143:0/1434910069 wait complete. 2023-11-26T07:22:48.286 DEBUG:teuthology.orchestra.run.smithi143:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mon mon_allow_pool_delete True 2023-11-26T07:22:48.425 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:48.423+0000 7fc9dd9f8640 1 Processor -- start 2023-11-26T07:22:48.425 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:48.423+0000 7fc9dd9f8640 1 -- start start 2023-11-26T07:22:48.427 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:48.423+0000 7fc9dd9f8640 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7fc9d815d580 0x7fc9d80d3920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:22:48.427 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:48.423+0000 7fc9dd9f8640 1 -- --> v1:172.21.15.143:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc9d80d0bc0 con 0x7fc9d80d1f00 2023-11-26T07:22:48.427 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:48.423+0000 7fc9dd9f8640 1 -- --> v1:172.21.15.188:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc9d80d0320 con 0x7fc9d80d3e60 2023-11-26T07:22:48.427 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:48.423+0000 7fc9dd9f8640 1 -- --> v2:172.21.15.188:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc9d80d0600 con 0x7fc9d815d580 2023-11-26T07:22:48.428 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:48.423+0000 7fc9d77fe640 1 --1- >> v1:172.21.15.143:6789/0 conn(0x7fc9d80d1f00 0x7fc9d80b0490 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.143:6789/0 says I am v1:172.21.15.143:47984/0 (socket says 172.21.15.143:47984) 2023-11-26T07:22:48.428 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:48.423+0000 7fc9d77fe640 1 -- 172.21.15.143:0/117419454 learned_addr learned my addr 172.21.15.143:0/117419454 (peer_addr_for_me v1:172.21.15.143:0/0) 2023-11-26T07:22:48.428 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:48.423+0000 7fc9d6ffd640 1 --2- 172.21.15.143:0/117419454 >> v2:172.21.15.188:3300/0 conn(0x7fc9d815d580 0x7fc9d80d3920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:22:48.429 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:48.427+0000 7fc9d5ffb640 1 -- 172.21.15.143:0/117419454 <== mon.0 v1:172.21.15.143:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1676147973 0 0) 0x7fc9d80d0bc0 con 0x7fc9d80d1f00 2023-11-26T07:22:48.429 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:48.427+0000 7fc9d5ffb640 1 -- 172.21.15.143:0/117419454 --> v1:172.21.15.143:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fc9c0003530 con 0x7fc9d80d1f00 2023-11-26T07:22:48.429 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:48.427+0000 7fc9d6ffd640 1 -- 172.21.15.143:0/117419454 >> v1:172.21.15.188:6789/0 conn(0x7fc9d80d3e60 legacy=0x7fc9d80d17f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:22:48.429 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:48.427+0000 7fc9d6ffd640 1 -- 172.21.15.143:0/117419454 >> v1:172.21.15.143:6789/0 conn(0x7fc9d80d1f00 legacy=0x7fc9d80b0490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:22:48.429 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:48.427+0000 7fc9d6ffd640 1 -- 172.21.15.143:0/117419454 --> v2:172.21.15.188:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc9d80d08e0 con 0x7fc9d815d580 2023-11-26T07:22:48.430 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:48.427+0000 7fc9d6ffd640 1 --2- 172.21.15.143:0/117419454 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fc9d815d580 0x7fc9d80d3920 secure :-1 s=READY pgs=396 cs=0 l=1 rev1=1 crypto rx=0x7fc9c80097c0 tx=0x7fc9c801c450 comp rx=0 tx=0).ready entity=mon.1 client_cookie=79ea3ea71bc79838 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:22:48.431 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:48.427+0000 7fc9d5ffb640 1 -- 172.21.15.143:0/117419454 <== mon.1 v2:172.21.15.188:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc9c80ad680 con 0x7fc9d815d580 2023-11-26T07:22:48.431 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:48.427+0000 7fc9d5ffb640 1 -- 172.21.15.143:0/117419454 <== mon.1 v2:172.21.15.188:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc9c80ad7e0 con 0x7fc9d815d580 2023-11-26T07:22:48.431 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:48.427+0000 7fc9d5ffb640 1 -- 172.21.15.143:0/117419454 <== mon.1 v2:172.21.15.188:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc9c80adb70 con 0x7fc9d815d580 2023-11-26T07:22:48.432 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:48.431+0000 7fc9dd9f8640 1 -- 172.21.15.143:0/117419454 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fc9d815d580 msgr2=0x7fc9d80d3920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:22:48.432 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:48.431+0000 7fc9dd9f8640 1 --2- 172.21.15.143:0/117419454 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fc9d815d580 0x7fc9d80d3920 secure :-1 s=READY pgs=396 cs=0 l=1 rev1=1 crypto rx=0x7fc9c80097c0 tx=0x7fc9c801c450 comp rx=0 tx=0).stop 2023-11-26T07:22:48.432 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:48.431+0000 7fc9dd9f8640 1 -- 172.21.15.143:0/117419454 shutdown_connections 2023-11-26T07:22:48.432 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:48.431+0000 7fc9dd9f8640 1 --2- 172.21.15.143:0/117419454 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fc9d815d580 0x7fc9d80d3920 unknown :-1 s=CLOSED pgs=396 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:22:48.432 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:48.431+0000 7fc9dd9f8640 1 -- 172.21.15.143:0/117419454 >> 172.21.15.143:0/117419454 conn(0x7fc9d809c550 msgr2=0x7fc9d80d3210 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:22:48.432 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:48.431+0000 7fc9dd9f8640 1 -- 172.21.15.143:0/117419454 shutdown_connections 2023-11-26T07:22:48.433 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:48.431+0000 7fc9dd9f8640 1 -- 172.21.15.143:0/117419454 wait complete. 2023-11-26T07:22:48.433 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:48.431+0000 7fc9dd9f8640 1 Processor -- start 2023-11-26T07:22:48.433 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:48.431+0000 7fc9dd9f8640 1 -- start start 2023-11-26T07:22:48.435 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:48.431+0000 7fc9dd9f8640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fc9d80d1f00 0x7fc9d80b3c80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:22:48.435 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:48.431+0000 7fc9dd9f8640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fc9d80d3e60 0x7fc9d8155440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:22:48.435 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:48.431+0000 7fc9dd9f8640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fc9d815d580 0x7fc9d8155980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:22:48.435 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:48.431+0000 7fc9dd9f8640 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc9d80b4c40 con 0x7fc9d815d580 2023-11-26T07:22:48.435 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:48.431+0000 7fc9dd9f8640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc9d8155ec0 con 0x7fc9d80d1f00 2023-11-26T07:22:48.435 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:48.431+0000 7fc9dd9f8640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc9d80b2090 con 0x7fc9d80d3e60 2023-11-26T07:22:48.435 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:48.431+0000 7fc9d77fe640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fc9d815d580 0x7fc9d8155980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:22:48.435 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:48.431+0000 7fc9d77fe640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fc9d815d580 0x7fc9d8155980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:50374/0 (socket says 172.21.15.143:50374) 2023-11-26T07:22:48.436 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:48.431+0000 7fc9d77fe640 1 -- 172.21.15.143:0/1178465370 learned_addr learned my addr 172.21.15.143:0/1178465370 (peer_addr_for_me v2:172.21.15.143:0/0) 2023-11-26T07:22:48.436 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:48.431+0000 7fc9d6ffd640 1 --2- 172.21.15.143:0/1178465370 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fc9d80d1f00 0x7fc9d80b3c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:22:48.436 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:48.431+0000 7fc9d67fc640 1 --2- 172.21.15.143:0/1178465370 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fc9d80d3e60 0x7fc9d8155440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:22:48.437 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:48.435+0000 7fc9d77fe640 1 -- 172.21.15.143:0/1178465370 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fc9d80d3e60 msgr2=0x7fc9d8155440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:22:48.437 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:48.435+0000 7fc9d77fe640 1 --2- 172.21.15.143:0/1178465370 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fc9d80d3e60 0x7fc9d8155440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:22:48.437 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:48.435+0000 7fc9d77fe640 1 -- 172.21.15.143:0/1178465370 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fc9d80d1f00 msgr2=0x7fc9d80b3c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:22:48.437 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:48.435+0000 7fc9d77fe640 1 --2- 172.21.15.143:0/1178465370 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fc9d80d1f00 0x7fc9d80b3c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:22:48.437 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:48.435+0000 7fc9d77fe640 1 -- 172.21.15.143:0/1178465370 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc9c8009420 con 0x7fc9d815d580 2023-11-26T07:22:48.437 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:48.435+0000 7fc9d77fe640 1 --2- 172.21.15.143:0/1178465370 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fc9d815d580 0x7fc9d8155980 secure :-1 s=READY pgs=991 cs=0 l=1 rev1=1 crypto rx=0x7fc9d000cbb0 tx=0x7fc9d000a430 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:22:48.437 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:48.435+0000 7fc9bffff640 1 -- 172.21.15.143:0/1178465370 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc9d000ad40 con 0x7fc9d815d580 2023-11-26T07:22:48.438 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:48.435+0000 7fc9bffff640 1 -- 172.21.15.143:0/1178465370 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc9d0012400 con 0x7fc9d815d580 2023-11-26T07:22:48.438 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:48.435+0000 7fc9bffff640 1 -- 172.21.15.143:0/1178465370 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc9d000aea0 con 0x7fc9d815d580 2023-11-26T07:22:48.438 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:48.435+0000 7fc9dd9f8640 1 -- 172.21.15.143:0/1178465370 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc9d80b45f0 con 0x7fc9d815d580 2023-11-26T07:22:48.438 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:48.435+0000 7fc9dd9f8640 1 -- 172.21.15.143:0/1178465370 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc9d80b1d20 con 0x7fc9d815d580 2023-11-26T07:22:48.438 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:48.435+0000 7fc9dd9f8640 1 -- 172.21.15.143:0/1178465370 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc9d8067680 con 0x7fc9d815d580 2023-11-26T07:22:48.438 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:48.435+0000 7fc9bffff640 1 -- 172.21.15.143:0/1178465370 <== mon.0 v2:172.21.15.143:3300/0 4 ==== mgrmap(e 11) v1 ==== 86158+0+0 (secure 0 0 0) 0x7fc9d0007070 con 0x7fc9d815d580 2023-11-26T07:22:48.439 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:48.435+0000 7fc9bffff640 1 --2- 172.21.15.143:0/1178465370 >> [v2:172.21.15.143:6832/2150557544,v1:172.21.15.143:6833/2150557544] conn(0x7fc9c4066800 0x7fc9c4068cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:22:48.439 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:48.439+0000 7fc9bffff640 1 -- 172.21.15.143:0/1178465370 <== mon.0 v2:172.21.15.143:3300/0 5 ==== osd_map(71..71 src has 1..71) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fc9d0084220 con 0x7fc9d815d580 2023-11-26T07:22:48.439 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:48.439+0000 7fc9d6ffd640 1 --2- 172.21.15.143:0/1178465370 >> [v2:172.21.15.143:6832/2150557544,v1:172.21.15.143:6833/2150557544] conn(0x7fc9c4066800 0x7fc9c4068cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:22:48.440 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:48.439+0000 7fc9d6ffd640 1 --2- 172.21.15.143:0/1178465370 >> [v2:172.21.15.143:6832/2150557544,v1:172.21.15.143:6833/2150557544] conn(0x7fc9c4066800 0x7fc9c4068cc0 secure :-1 s=READY pgs=461 cs=0 l=1 rev1=1 crypto rx=0x7fc9c80af000 tx=0x7fc9c80a7000 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:22:48.443 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:48.443+0000 7fc9bffff640 1 -- 172.21.15.143:0/1178465370 <== mon.0 v2:172.21.15.143:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172649 (secure 0 0 0) 0x7fc9d00512d0 con 0x7fc9d815d580 2023-11-26T07:22:48.594 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:48.591+0000 7fc9dd9f8640 1 -- 172.21.15.143:0/1178465370 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command([{prefix=config set, name=mon_allow_pool_delete}] v 0) v1 -- 0x7fc9d80b4d80 con 0x7fc9d815d580 2023-11-26T07:22:48.594 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:48.591+0000 7fc9bffff640 1 -- 172.21.15.143:0/1178465370 <== mon.0 v2:172.21.15.143:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mon_allow_pool_delete}]=0 v16)=0 v16) v1 ==== 122+0+0 (secure 0 0 0) 0x7fc9d80b4d80 con 0x7fc9d815d580 2023-11-26T07:22:48.596 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:48.595+0000 7fc9dd9f8640 1 -- 172.21.15.143:0/1178465370 >> [v2:172.21.15.143:6832/2150557544,v1:172.21.15.143:6833/2150557544] conn(0x7fc9c4066800 msgr2=0x7fc9c4068cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:22:48.596 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:48.595+0000 7fc9dd9f8640 1 --2- 172.21.15.143:0/1178465370 >> [v2:172.21.15.143:6832/2150557544,v1:172.21.15.143:6833/2150557544] conn(0x7fc9c4066800 0x7fc9c4068cc0 secure :-1 s=READY pgs=461 cs=0 l=1 rev1=1 crypto rx=0x7fc9c80af000 tx=0x7fc9c80a7000 comp rx=0 tx=0).stop 2023-11-26T07:22:48.597 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:48.595+0000 7fc9dd9f8640 1 -- 172.21.15.143:0/1178465370 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fc9d815d580 msgr2=0x7fc9d8155980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:22:48.597 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:48.595+0000 7fc9dd9f8640 1 --2- 172.21.15.143:0/1178465370 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fc9d815d580 0x7fc9d8155980 secure :-1 s=READY pgs=991 cs=0 l=1 rev1=1 crypto rx=0x7fc9d000cbb0 tx=0x7fc9d000a430 comp rx=0 tx=0).stop 2023-11-26T07:22:48.597 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:48.595+0000 7fc9dd9f8640 1 -- 172.21.15.143:0/1178465370 shutdown_connections 2023-11-26T07:22:48.597 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:48.595+0000 7fc9dd9f8640 1 --2- 172.21.15.143:0/1178465370 >> [v2:172.21.15.143:6832/2150557544,v1:172.21.15.143:6833/2150557544] conn(0x7fc9c4066800 0x7fc9c4068cc0 unknown :-1 s=CLOSED pgs=461 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:22:48.597 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:48.595+0000 7fc9dd9f8640 1 --2- 172.21.15.143:0/1178465370 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fc9d815d580 0x7fc9d8155980 unknown :-1 s=CLOSED pgs=991 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:22:48.597 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:48.595+0000 7fc9dd9f8640 1 --2- 172.21.15.143:0/1178465370 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fc9d80d3e60 0x7fc9d8155440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:22:48.597 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:48.595+0000 7fc9dd9f8640 1 --2- 172.21.15.143:0/1178465370 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fc9d80d1f00 0x7fc9d80b3c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:22:48.597 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:48.595+0000 7fc9dd9f8640 1 -- 172.21.15.143:0/1178465370 >> 172.21.15.143:0/1178465370 conn(0x7fc9d809c550 msgr2=0x7fc9d80d0b20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:22:48.597 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:48.595+0000 7fc9dd9f8640 1 -- 172.21.15.143:0/1178465370 shutdown_connections 2023-11-26T07:22:48.598 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:48.595+0000 7fc9dd9f8640 1 -- 172.21.15.143:0/1178465370 wait complete. 2023-11-26T07:22:48.615 DEBUG:teuthology.orchestra.run.smithi143:> set -ex 2023-11-26T07:22:48.615 DEBUG:teuthology.orchestra.run.smithi143:> dd if=/proc/self/mounts of=/dev/stdout 2023-11-26T07:22:48.625 DEBUG:teuthology.orchestra.run.smithi143:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-11-26T07:22:48.804 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:48.803+0000 7fe7a873e640 1 Processor -- start 2023-11-26T07:22:48.805 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:48.803+0000 7fe7a873e640 1 -- start start 2023-11-26T07:22:48.806 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:48.803+0000 7fe7a873e640 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7fe7a015d6e0 0x7fe7a00d0df0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:22:48.806 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:48.803+0000 7fe7a873e640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fe7a00b2160 0x7fe7a00d14b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:22:48.806 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:48.803+0000 7fe7a873e640 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7fe7a00d19f0 0x7fe7a00cfbf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:22:48.806 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:48.803+0000 7fe7a873e640 1 -- --> v2:172.21.15.143:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe7a015dac0 con 0x7fe7a015d6e0 2023-11-26T07:22:48.806 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:48.803+0000 7fe7a873e640 1 -- --> v2:172.21.15.188:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe7a00d1e80 con 0x7fe7a00d19f0 2023-11-26T07:22:48.806 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:48.803+0000 7fe7a873e640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe7a00cf0e0 con 0x7fe7a00b2160 2023-11-26T07:22:48.806 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:48.803+0000 7fe7a64da640 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7fe7a015d6e0 0x7fe7a00d0df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:22:48.807 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:48.803+0000 7fe7a64da640 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7fe7a015d6e0 0x7fe7a00d0df0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:50378/0 (socket says 172.21.15.143:50378) 2023-11-26T07:22:48.807 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:48.803+0000 7fe7a64da640 1 -- 172.21.15.143:0/2120771056 learned_addr learned my addr 172.21.15.143:0/2120771056 (peer_addr_for_me v2:172.21.15.143:0/0) 2023-11-26T07:22:48.807 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:48.803+0000 7fe7a64da640 1 -- 172.21.15.143:0/2120771056 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fe7a00b2160 msgr2=0x7fe7a00d14b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-26T07:22:48.807 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:48.803+0000 7fe7a64da640 1 --2- 172.21.15.143:0/2120771056 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fe7a00b2160 0x7fe7a00d14b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:22:48.807 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:48.803+0000 7fe7a64da640 1 -- 172.21.15.143:0/2120771056 >> v2:172.21.15.188:3300/0 conn(0x7fe7a00d19f0 msgr2=0x7fe7a00cfbf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-26T07:22:48.808 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:48.803+0000 7fe7a6cdb640 1 --2- 172.21.15.143:0/2120771056 >> v2:172.21.15.188:3300/0 conn(0x7fe7a00d19f0 0x7fe7a00cfbf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:22:48.808 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:48.803+0000 7fe7a64da640 1 --2- 172.21.15.143:0/2120771056 >> v2:172.21.15.188:3300/0 conn(0x7fe7a00d19f0 0x7fe7a00cfbf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:22:48.808 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:48.803+0000 7fe7a64da640 1 -- 172.21.15.143:0/2120771056 --> v2:172.21.15.143:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe7a00ceb20 con 0x7fe7a015d6e0 2023-11-26T07:22:48.808 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:48.803+0000 7fe7a6cdb640 1 --2- 172.21.15.143:0/2120771056 >> v2:172.21.15.188:3300/0 conn(0x7fe7a00d19f0 0x7fe7a00cfbf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-26T07:22:48.809 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:48.807+0000 7fe7a64da640 1 --2- 172.21.15.143:0/2120771056 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fe7a015d6e0 0x7fe7a00d0df0 secure :-1 s=READY pgs=992 cs=0 l=1 rev1=1 crypto rx=0x7fe7a008d820 tx=0x7fe79c01c710 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ebba159a5cdc1218 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:22:48.810 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:48.807+0000 7fe7a54d8640 1 -- 172.21.15.143:0/2120771056 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe79c0a6490 con 0x7fe7a015d6e0 2023-11-26T07:22:48.810 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:48.807+0000 7fe7a54d8640 1 -- 172.21.15.143:0/2120771056 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe79c0a65f0 con 0x7fe7a015d6e0 2023-11-26T07:22:48.810 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:48.807+0000 7fe7a54d8640 1 -- 172.21.15.143:0/2120771056 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe79c0a6980 con 0x7fe7a015d6e0 2023-11-26T07:22:48.810 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:48.807+0000 7fe7a873e640 1 -- 172.21.15.143:0/2120771056 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fe7a015d6e0 msgr2=0x7fe7a00d0df0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:22:48.810 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:48.807+0000 7fe7a873e640 1 --2- 172.21.15.143:0/2120771056 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fe7a015d6e0 0x7fe7a00d0df0 secure :-1 s=READY pgs=992 cs=0 l=1 rev1=1 crypto rx=0x7fe7a008d820 tx=0x7fe79c01c710 comp rx=0 tx=0).stop 2023-11-26T07:22:48.811 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:48.807+0000 7fe7a873e640 1 -- 172.21.15.143:0/2120771056 shutdown_connections 2023-11-26T07:22:48.811 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:48.807+0000 7fe7a873e640 1 --2- 172.21.15.143:0/2120771056 >> v2:172.21.15.188:3300/0 conn(0x7fe7a00d19f0 0x7fe7a00cfbf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:22:48.811 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:48.807+0000 7fe7a873e640 1 --2- 172.21.15.143:0/2120771056 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fe7a00b2160 0x7fe7a00d14b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:22:48.811 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:48.807+0000 7fe7a873e640 1 --2- 172.21.15.143:0/2120771056 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fe7a015d6e0 0x7fe7a00d0df0 unknown :-1 s=CLOSED pgs=992 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:22:48.811 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:48.807+0000 7fe7a873e640 1 -- 172.21.15.143:0/2120771056 >> 172.21.15.143:0/2120771056 conn(0x7fe7a009c550 msgr2=0x7fe7a00d3220 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:22:48.811 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:48.807+0000 7fe7a873e640 1 -- 172.21.15.143:0/2120771056 shutdown_connections 2023-11-26T07:22:48.811 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:48.807+0000 7fe7a873e640 1 -- 172.21.15.143:0/2120771056 wait complete. 2023-11-26T07:22:48.812 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:48.811+0000 7fe7a873e640 1 Processor -- start 2023-11-26T07:22:48.812 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:48.811+0000 7fe7a873e640 1 -- start start 2023-11-26T07:22:48.814 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:48.811+0000 7fe7a873e640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fe7a00b2160 0x7fe7a00b80e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:22:48.814 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:48.811+0000 7fe7a873e640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fe7a00d19f0 0x7fe7a00c8d00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:22:48.814 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:48.811+0000 7fe7a873e640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fe7a015d6e0 0x7fe7a015ec50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:22:48.814 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:48.811+0000 7fe7a873e640 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe7a0156ae0 con 0x7fe7a015d6e0 2023-11-26T07:22:48.814 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:48.811+0000 7fe7a873e640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe7a00b8620 con 0x7fe7a00b2160 2023-11-26T07:22:48.814 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:48.811+0000 7fe7a873e640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe7a0155e60 con 0x7fe7a00d19f0 2023-11-26T07:22:48.814 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:48.811+0000 7fe7a6cdb640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fe7a015d6e0 0x7fe7a015ec50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:22:48.814 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:48.811+0000 7fe7a6cdb640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fe7a015d6e0 0x7fe7a015ec50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:50380/0 (socket says 172.21.15.143:50380) 2023-11-26T07:22:48.814 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:48.811+0000 7fe7a6cdb640 1 -- 172.21.15.143:0/1582053412 learned_addr learned my addr 172.21.15.143:0/1582053412 (peer_addr_for_me v2:172.21.15.143:0/0) 2023-11-26T07:22:48.815 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:48.811+0000 7fe7a64da640 1 --2- 172.21.15.143:0/1582053412 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fe7a00b2160 0x7fe7a00b80e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:22:48.815 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:48.811+0000 7fe7a5cd9640 1 --2- 172.21.15.143:0/1582053412 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fe7a00d19f0 0x7fe7a00c8d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:22:48.815 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:48.811+0000 7fe7a6cdb640 1 -- 172.21.15.143:0/1582053412 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fe7a00d19f0 msgr2=0x7fe7a00c8d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:22:48.815 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:48.811+0000 7fe7a6cdb640 1 --2- 172.21.15.143:0/1582053412 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fe7a00d19f0 0x7fe7a00c8d00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:22:48.815 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:48.811+0000 7fe7a6cdb640 1 -- 172.21.15.143:0/1582053412 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fe7a00b2160 msgr2=0x7fe7a00b80e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:22:48.815 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:48.811+0000 7fe7a6cdb640 1 --2- 172.21.15.143:0/1582053412 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fe7a00b2160 0x7fe7a00b80e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:22:48.815 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:48.811+0000 7fe7a6cdb640 1 -- 172.21.15.143:0/1582053412 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe79c009b30 con 0x7fe7a015d6e0 2023-11-26T07:22:48.815 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:48.811+0000 7fe7a6cdb640 1 --2- 172.21.15.143:0/1582053412 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fe7a015d6e0 0x7fe7a015ec50 secure :-1 s=READY pgs=993 cs=0 l=1 rev1=1 crypto rx=0x7fe79800c7f0 tx=0x7fe79800ccb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:22:48.816 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:48.811+0000 7fe796ffd640 1 -- 172.21.15.143:0/1582053412 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe798007820 con 0x7fe7a015d6e0 2023-11-26T07:22:48.816 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:48.811+0000 7fe796ffd640 1 -- 172.21.15.143:0/1582053412 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe7980079c0 con 0x7fe7a015d6e0 2023-11-26T07:22:48.816 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:48.811+0000 7fe7a873e640 1 -- 172.21.15.143:0/1582053412 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe7a0156140 con 0x7fe7a015d6e0 2023-11-26T07:22:48.816 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:48.811+0000 7fe796ffd640 1 -- 172.21.15.143:0/1582053412 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe798007d70 con 0x7fe7a015d6e0 2023-11-26T07:22:48.816 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:48.811+0000 7fe7a873e640 1 -- 172.21.15.143:0/1582053412 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe7a0156700 con 0x7fe7a015d6e0 2023-11-26T07:22:48.816 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:48.815+0000 7fe7a873e640 1 -- 172.21.15.143:0/1582053412 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe7a0156c20 con 0x7fe7a015d6e0 2023-11-26T07:22:48.816 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:48.815+0000 7fe796ffd640 1 -- 172.21.15.143:0/1582053412 <== mon.0 v2:172.21.15.143:3300/0 4 ==== mgrmap(e 11) v1 ==== 86158+0+0 (secure 0 0 0) 0x7fe7980113f0 con 0x7fe7a015d6e0 2023-11-26T07:22:48.817 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:48.815+0000 7fe796ffd640 1 --2- 172.21.15.143:0/1582053412 >> [v2:172.21.15.143:6832/2150557544,v1:172.21.15.143:6833/2150557544] conn(0x7fe784066740 0x7fe784068c00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:22:48.817 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:48.815+0000 7fe796ffd640 1 -- 172.21.15.143:0/1582053412 <== mon.0 v2:172.21.15.143:3300/0 5 ==== osd_map(71..71 src has 1..71) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fe798084730 con 0x7fe7a015d6e0 2023-11-26T07:22:48.817 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:48.815+0000 7fe7a64da640 1 --2- 172.21.15.143:0/1582053412 >> [v2:172.21.15.143:6832/2150557544,v1:172.21.15.143:6833/2150557544] conn(0x7fe784066740 0x7fe784068c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:22:48.818 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:48.815+0000 7fe7a64da640 1 --2- 172.21.15.143:0/1582053412 >> [v2:172.21.15.143:6832/2150557544,v1:172.21.15.143:6833/2150557544] conn(0x7fe784066740 0x7fe784068c00 secure :-1 s=READY pgs=462 cs=0 l=1 rev1=1 crypto rx=0x7fe79c0ad000 tx=0x7fe79c0b3040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:22:48.821 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:48.819+0000 7fe796ffd640 1 -- 172.21.15.143:0/1582053412 <== mon.0 v2:172.21.15.143:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172649 (secure 0 0 0) 0x7fe7a0156c20 con 0x7fe7a015d6e0 2023-11-26T07:22:48.974 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:48.971+0000 7fe7a873e640 1 -- 172.21.15.143:0/1582053412 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fe7a0156c20 con 0x7fe7a015d6e0 2023-11-26T07:22:48.975 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:48.971+0000 7fe796ffd640 1 -- 172.21.15.143:0/1582053412 <== mon.0 v2:172.21.15.143:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v71) v1 ==== 74+0+16604 (secure 0 0 0) 0x7fe7a0156c20 con 0x7fe7a015d6e0 2023-11-26T07:22:48.975 INFO:teuthology.orchestra.run.smithi143.stdout: 2023-11-26T07:22:48.975 INFO:teuthology.orchestra.run.smithi143.stdout:{"epoch":71,"fsid":"f5033060-2469-4a07-95fe-2b76f5404bc1","created":"2023-11-26T07:17:28.613663+0000","modified":"2023-11-26T07:22:38.856077+0000","last_up_change":"2023-11-26T07:17:41.157813+0000","last_in_change":"2023-11-26T07:17:31.181658+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":11,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-11-26T07:17:41.758597+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":10,"pool_name":"cephfs_metadata","create_time":"2023-11-26T07:22:35.314819+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"70","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":2.5,"score_stable":2.5,"optimal_score":1,"raw_score_acting":2.5,"raw_score_stable":2.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":11,"pool_name":"cephfs_data","create_time":"2023-11-26T07:22:36.219536+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"71","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"ebbee2c6-023e-44b9-9f1d-6f7bfd309730","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":69,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6817","nonce":639807369},{"type":"v1","addr":"172.21.15.143:6819","nonce":639807369}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6821","nonce":639807369},{"type":"v1","addr":"172.21.15.143:6823","nonce":639807369}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6830","nonce":639807369},{"type":"v1","addr":"172.21.15.143:6831","nonce":639807369}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6825","nonce":639807369},{"type":"v1","addr":"172.21.15.143:6828","nonce":639807369}]},"public_addr":"172.21.15.143:6819/639807369","cluster_addr":"172.21.15.143:6823/639807369","heartbeat_back_addr":"172.21.15.143:6831/639807369","heartbeat_front_addr":"172.21.15.143:6828/639807369","state":["exists","up"]},{"osd":1,"uuid":"fec4b8be-7122-49e9-9a19-5850a0700001","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":69,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6816","nonce":1144942530},{"type":"v1","addr":"172.21.15.143:6818","nonce":1144942530}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6820","nonce":1144942530},{"type":"v1","addr":"172.21.15.143:6822","nonce":1144942530}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6827","nonce":1144942530},{"type":"v1","addr":"172.21.15.143:6829","nonce":1144942530}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6824","nonce":1144942530},{"type":"v1","addr":"172.21.15.143:6826","nonce":1144942530}]},"public_addr":"172.21.15.143:6818/1144942530","cluster_addr":"172.21.15.143:6822/1144942530","heartbeat_back_addr":"172.21.15.143:6829/1144942530","heartbeat_front_addr":"172.21.15.143:6826/1144942530","state":["exists","up"]},{"osd":2,"uuid":"2888af5b-f933-46ca-8f95-6dd494ff43f6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":69,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6808","nonce":3909095186},{"type":"v1","addr":"172.21.15.143:6809","nonce":3909095186}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6810","nonce":3909095186},{"type":"v1","addr":"172.21.15.143:6811","nonce":3909095186}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6814","nonce":3909095186},{"type":"v1","addr":"172.21.15.143:6815","nonce":3909095186}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6812","nonce":3909095186},{"type":"v1","addr":"172.21.15.143:6813","nonce":3909095186}]},"public_addr":"172.21.15.143:6809/3909095186","cluster_addr":"172.21.15.143:6811/3909095186","heartbeat_back_addr":"172.21.15.143:6815/3909095186","heartbeat_front_addr":"172.21.15.143:6813/3909095186","state":["exists","up"]},{"osd":3,"uuid":"d1f52435-7bc1-4081-9550-9cd1ba55f549","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":69,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6800","nonce":3854210991},{"type":"v1","addr":"172.21.15.143:6801","nonce":3854210991}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6802","nonce":3854210991},{"type":"v1","addr":"172.21.15.143:6803","nonce":3854210991}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6806","nonce":3854210991},{"type":"v1","addr":"172.21.15.143:6807","nonce":3854210991}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6804","nonce":3854210991},{"type":"v1","addr":"172.21.15.143:6805","nonce":3854210991}]},"public_addr":"172.21.15.143:6801/3854210991","cluster_addr":"172.21.15.143:6803/3854210991","heartbeat_back_addr":"172.21.15.143:6807/3854210991","heartbeat_front_addr":"172.21.15.143:6805/3854210991","state":["exists","up"]},{"osd":4,"uuid":"e2ae8b6e-6e25-49b2-bcbf-aacce68f06a3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":69,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6800","nonce":3924606902},{"type":"v1","addr":"172.21.15.188:6801","nonce":3924606902}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6802","nonce":3924606902},{"type":"v1","addr":"172.21.15.188:6803","nonce":3924606902}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6806","nonce":3924606902},{"type":"v1","addr":"172.21.15.188:6807","nonce":3924606902}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6804","nonce":3924606902},{"type":"v1","addr":"172.21.15.188:6805","nonce":3924606902}]},"public_addr":"172.21.15.188:6801/3924606902","cluster_addr":"172.21.15.188:6803/3924606902","heartbeat_back_addr":"172.21.15.188:6807/3924606902","heartbeat_front_addr":"172.21.15.188:6805/3924606902","state":["exists","up"]},{"osd":5,"uuid":"3cec8a32-7698-4c36-8e89-e4ba11934f33","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":69,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6813","nonce":186841544},{"type":"v1","addr":"172.21.15.188:6815","nonce":186841544}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6817","nonce":186841544},{"type":"v1","addr":"172.21.15.188:6818","nonce":186841544}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6822","nonce":186841544},{"type":"v1","addr":"172.21.15.188:6823","nonce":186841544}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6820","nonce":186841544},{"type":"v1","addr":"172.21.15.188:6821","nonce":186841544}]},"public_addr":"172.21.15.188:6815/186841544","cluster_addr":"172.21.15.188:6818/186841544","heartbeat_back_addr":"172.21.15.188:6823/186841544","heartbeat_front_addr":"172.21.15.188:6821/186841544","state":["exists","up"]},{"osd":6,"uuid":"58a65d5d-e090-473a-81b4-ad38e85029f9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":69,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6808","nonce":2153499405},{"type":"v1","addr":"172.21.15.188:6809","nonce":2153499405}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6810","nonce":2153499405},{"type":"v1","addr":"172.21.15.188:6811","nonce":2153499405}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6816","nonce":2153499405},{"type":"v1","addr":"172.21.15.188:6819","nonce":2153499405}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6812","nonce":2153499405},{"type":"v1","addr":"172.21.15.188:6814","nonce":2153499405}]},"public_addr":"172.21.15.188:6809/2153499405","cluster_addr":"172.21.15.188:6811/2153499405","heartbeat_back_addr":"172.21.15.188:6819/2153499405","heartbeat_front_addr":"172.21.15.188:6814/2153499405","state":["exists","up"]},{"osd":7,"uuid":"ea5b7f70-3554-4cf8-ac30-f7c59b80678d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":69,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6824","nonce":2722096729},{"type":"v1","addr":"172.21.15.188:6825","nonce":2722096729}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6826","nonce":2722096729},{"type":"v1","addr":"172.21.15.188:6827","nonce":2722096729}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6830","nonce":2722096729},{"type":"v1","addr":"172.21.15.188:6831","nonce":2722096729}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6828","nonce":2722096729},{"type":"v1","addr":"172.21.15.188:6829","nonce":2722096729}]},"public_addr":"172.21.15.188:6825/2722096729","cluster_addr":"172.21.15.188:6827/2722096729","heartbeat_back_addr":"172.21.15.188:6831/2722096729","heartbeat_front_addr":"172.21.15.188:6829/2722096729","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-26T07:17:36.995032+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-26T07:17:36.932266+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-26T07:17:36.829304+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-26T07:17:36.804297+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-26T07:17:36.667892+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-26T07:17:36.964868+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-26T07:17:36.827332+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-26T07:17:36.893179+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-11-26T07:22:48.977 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:48.975+0000 7fe7a873e640 1 -- 172.21.15.143:0/1582053412 >> [v2:172.21.15.143:6832/2150557544,v1:172.21.15.143:6833/2150557544] conn(0x7fe784066740 msgr2=0x7fe784068c00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:22:48.977 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:48.975+0000 7fe7a873e640 1 --2- 172.21.15.143:0/1582053412 >> [v2:172.21.15.143:6832/2150557544,v1:172.21.15.143:6833/2150557544] conn(0x7fe784066740 0x7fe784068c00 secure :-1 s=READY pgs=462 cs=0 l=1 rev1=1 crypto rx=0x7fe79c0ad000 tx=0x7fe79c0b3040 comp rx=0 tx=0).stop 2023-11-26T07:22:48.977 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:48.975+0000 7fe7a873e640 1 -- 172.21.15.143:0/1582053412 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fe7a015d6e0 msgr2=0x7fe7a015ec50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:22:48.977 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:48.975+0000 7fe7a873e640 1 --2- 172.21.15.143:0/1582053412 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fe7a015d6e0 0x7fe7a015ec50 secure :-1 s=READY pgs=993 cs=0 l=1 rev1=1 crypto rx=0x7fe79800c7f0 tx=0x7fe79800ccb0 comp rx=0 tx=0).stop 2023-11-26T07:22:48.977 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:48.975+0000 7fe7a873e640 1 -- 172.21.15.143:0/1582053412 shutdown_connections 2023-11-26T07:22:48.978 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:48.975+0000 7fe7a873e640 1 --2- 172.21.15.143:0/1582053412 >> [v2:172.21.15.143:6832/2150557544,v1:172.21.15.143:6833/2150557544] conn(0x7fe784066740 0x7fe784068c00 unknown :-1 s=CLOSED pgs=462 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:22:48.978 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:48.975+0000 7fe7a873e640 1 --2- 172.21.15.143:0/1582053412 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fe7a015d6e0 0x7fe7a015ec50 unknown :-1 s=CLOSED pgs=993 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:22:48.978 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:48.975+0000 7fe7a873e640 1 --2- 172.21.15.143:0/1582053412 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fe7a00d19f0 0x7fe7a00c8d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:22:48.978 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:48.975+0000 7fe7a873e640 1 --2- 172.21.15.143:0/1582053412 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fe7a00b2160 0x7fe7a00b80e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:22:48.978 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:48.975+0000 7fe7a873e640 1 -- 172.21.15.143:0/1582053412 >> 172.21.15.143:0/1582053412 conn(0x7fe7a009c550 msgr2=0x7fe7a00b0ce0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:22:48.978 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:48.975+0000 7fe7a873e640 1 -- 172.21.15.143:0/1582053412 shutdown_connections 2023-11-26T07:22:48.978 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:48.975+0000 7fe7a873e640 1 -- 172.21.15.143:0/1582053412 wait complete. 2023-11-26T07:22:48.996 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-11-26T07:17:41.758597+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 10, 'pool_name': 'cephfs_metadata', 'create_time': '2023-11-26T07:22:35.314819+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '70', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 2.5, 'score_stable': 2.5, 'optimal_score': 1, 'raw_score_acting': 2.5, 'raw_score_stable': 2.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 11, 'pool_name': 'cephfs_data', 'create_time': '2023-11-26T07:22:36.219536+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '71', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.3799999952316284, 'score_stable': 1.3799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.3799999952316284, 'raw_score_stable': 1.3799999952316284, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2023-11-26T07:22:48.996 DEBUG:teuthology.orchestra.run.smithi143:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-11-26T07:22:49.137 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:49.135+0000 7f5130a9c640 1 Processor -- start 2023-11-26T07:22:49.137 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:49.135+0000 7f5130a9c640 1 -- start start 2023-11-26T07:22:49.138 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:49.135+0000 7f5130a9c640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f512c15d580 0x7f512c0d3920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:22:49.138 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:49.135+0000 7f5130a9c640 1 -- --> v1:172.21.15.143:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f512c0d0bc0 con 0x7f512c0d3e60 2023-11-26T07:22:49.138 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:49.135+0000 7f5130a9c640 1 -- --> v1:172.21.15.188:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f512c0d0320 con 0x7f512c0d1f00 2023-11-26T07:22:49.138 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:49.135+0000 7f5130a9c640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f512c0d0600 con 0x7f512c15d580 2023-11-26T07:22:49.140 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:49.135+0000 7f5123fff640 1 --1- >> v1:172.21.15.143:6789/0 conn(0x7f512c0d3e60 0x7f512c0d17f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.143:6789/0 says I am v1:172.21.15.143:47998/0 (socket says 172.21.15.143:47998) 2023-11-26T07:22:49.140 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:49.135+0000 7f5123fff640 1 -- 172.21.15.143:0/896960812 learned_addr learned my addr 172.21.15.143:0/896960812 (peer_addr_for_me v1:172.21.15.143:0/0) 2023-11-26T07:22:49.140 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:49.135+0000 7f512a59c640 1 --2- 172.21.15.143:0/896960812 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f512c15d580 0x7f512c0d3920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:22:49.140 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:49.135+0000 7f5129d9b640 1 -- 172.21.15.143:0/896960812 <== mon.0 v1:172.21.15.143:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 137337638 0 0) 0x7f512c0d0bc0 con 0x7f512c0d3e60 2023-11-26T07:22:49.140 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:49.135+0000 7f5129d9b640 1 -- 172.21.15.143:0/896960812 --> v1:172.21.15.143:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f5118003530 con 0x7f512c0d3e60 2023-11-26T07:22:49.140 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:49.135+0000 7f5129d9b640 1 -- 172.21.15.143:0/896960812 <== mon.1 v1:172.21.15.188:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 745270802 0 0) 0x7f512c0d0320 con 0x7f512c0d1f00 2023-11-26T07:22:49.141 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:49.135+0000 7f5129d9b640 1 -- 172.21.15.143:0/896960812 --> v1:172.21.15.188:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f512c0d0bc0 con 0x7f512c0d1f00 2023-11-26T07:22:49.141 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:49.139+0000 7f5129d9b640 1 -- 172.21.15.143:0/896960812 <== mon.0 v1:172.21.15.143:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1409468535 0 0) 0x7f5118003530 con 0x7f512c0d3e60 2023-11-26T07:22:49.141 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:49.139+0000 7f5129d9b640 1 -- 172.21.15.143:0/896960812 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f512c15d580 msgr2=0x7f512c0d3920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-26T07:22:49.141 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:49.139+0000 7f5129d9b640 1 --2- 172.21.15.143:0/896960812 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f512c15d580 0x7f512c0d3920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:22:49.141 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:49.139+0000 7f5129d9b640 1 -- 172.21.15.143:0/896960812 >> v1:172.21.15.188:6789/0 conn(0x7f512c0d1f00 legacy=0x7f512c0b0490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:22:49.141 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:49.139+0000 7f5129d9b640 1 -- 172.21.15.143:0/896960812 --> v1:172.21.15.143:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f512c0d08e0 con 0x7f512c0d3e60 2023-11-26T07:22:49.141 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:49.139+0000 7f5129d9b640 1 -- 172.21.15.143:0/896960812 <== mon.0 v1:172.21.15.143:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2137882523 0 0) 0x7f5114001680 con 0x7f512c0d3e60 2023-11-26T07:22:49.141 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:49.139+0000 7f5129d9b640 1 -- 172.21.15.143:0/896960812 >> v1:172.21.15.143:6789/0 conn(0x7f512c0d3e60 legacy=0x7f512c0d17f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:22:49.141 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:49.139+0000 7f5129d9b640 1 --2- 172.21.15.143:0/896960812 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f512c15d580 0x7f5118003b40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:22:49.141 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:49.139+0000 7f5129d9b640 1 --2- 172.21.15.143:0/896960812 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f51180040a0 0x7f51180047d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:22:49.141 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:49.139+0000 7f5129d9b640 1 --2- 172.21.15.143:0/896960812 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f5118004d10 0x7f5118005190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:22:49.142 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:49.139+0000 7f5129d9b640 1 -- 172.21.15.143:0/896960812 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f512c0d0600 con 0x7f5118004d10 2023-11-26T07:22:49.142 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:49.139+0000 7f5129d9b640 1 -- 172.21.15.143:0/896960812 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f511800ba70 con 0x7f51180040a0 2023-11-26T07:22:49.142 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:49.139+0000 7f5129d9b640 1 -- 172.21.15.143:0/896960812 --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f511800bc30 con 0x7f512c15d580 2023-11-26T07:22:49.142 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:49.139+0000 7f512ad9d640 1 --2- 172.21.15.143:0/896960812 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f5118004d10 0x7f5118005190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:22:49.142 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:49.139+0000 7f5123fff640 1 --2- 172.21.15.143:0/896960812 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f51180040a0 0x7f51180047d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:22:49.142 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:49.139+0000 7f512a59c640 1 --2- 172.21.15.143:0/896960812 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f512c15d580 0x7f5118003b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:22:49.142 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:49.139+0000 7f512ad9d640 1 -- 172.21.15.143:0/896960812 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f512c15d580 msgr2=0x7f5118003b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-26T07:22:49.142 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:49.139+0000 7f512ad9d640 1 --2- 172.21.15.143:0/896960812 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f512c15d580 0x7f5118003b40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:22:49.142 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:49.139+0000 7f512ad9d640 1 -- 172.21.15.143:0/896960812 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f51180040a0 msgr2=0x7f51180047d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-26T07:22:49.142 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:49.139+0000 7f512ad9d640 1 --2- 172.21.15.143:0/896960812 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f51180040a0 0x7f51180047d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:22:49.142 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:49.139+0000 7f512ad9d640 1 -- 172.21.15.143:0/896960812 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5118003530 con 0x7f5118004d10 2023-11-26T07:22:49.143 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:49.139+0000 7f512ad9d640 1 --2- 172.21.15.143:0/896960812 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f5118004d10 0x7f5118005190 secure :-1 s=READY pgs=995 cs=0 l=1 rev1=1 crypto rx=0x7f512c0d2390 tx=0x7f512401c8c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5ced52c5cdf6a527 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:22:49.143 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:49.139+0000 7f5129d9b640 1 -- 172.21.15.143:0/896960812 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5114007ea0 con 0x7f5118004d10 2023-11-26T07:22:49.143 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:49.139+0000 7f5129d9b640 1 -- 172.21.15.143:0/896960812 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5124003ba0 con 0x7f5118004d10 2023-11-26T07:22:49.143 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:49.139+0000 7f5129d9b640 1 -- 172.21.15.143:0/896960812 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f51240050d0 con 0x7f5118004d10 2023-11-26T07:22:49.144 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:49.143+0000 7f5130a9c640 1 -- 172.21.15.143:0/896960812 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f5118004d10 msgr2=0x7f5118005190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:22:49.145 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:49.143+0000 7f5130a9c640 1 --2- 172.21.15.143:0/896960812 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f5118004d10 0x7f5118005190 secure :-1 s=READY pgs=995 cs=0 l=1 rev1=1 crypto rx=0x7f512c0d2390 tx=0x7f512401c8c0 comp rx=0 tx=0).stop 2023-11-26T07:22:49.145 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:49.143+0000 7f512ad9d640 1 -- 172.21.15.143:0/896960812 reap_dead start 2023-11-26T07:22:49.145 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:49.143+0000 7f5130a9c640 1 -- 172.21.15.143:0/896960812 shutdown_connections 2023-11-26T07:22:49.145 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:49.143+0000 7f5130a9c640 1 -- 172.21.15.143:0/896960812 >> 172.21.15.143:0/896960812 conn(0x7f512c09c550 msgr2=0x7f512c0d3210 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:22:49.145 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:49.143+0000 7f5130a9c640 1 -- 172.21.15.143:0/896960812 shutdown_connections 2023-11-26T07:22:49.145 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:49.143+0000 7f5130a9c640 1 -- 172.21.15.143:0/896960812 wait complete. 2023-11-26T07:22:49.145 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:49.143+0000 7f5130a9c640 1 Processor -- start 2023-11-26T07:22:49.146 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:49.143+0000 7f5130a9c640 1 -- start start 2023-11-26T07:22:49.147 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:49.143+0000 7f5130a9c640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f5118004d10 0x7f512c0b6060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:22:49.147 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:49.143+0000 7f5130a9c640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f512c0b32d0 0x7f512c14c4e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:22:49.147 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:49.143+0000 7f5130a9c640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f512c0b7790 0x7f512c0b7bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:22:49.148 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:49.143+0000 7f5130a9c640 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f512c0b7300 con 0x7f5118004d10 2023-11-26T07:22:49.148 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:49.143+0000 7f5130a9c640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f512c0b66f0 con 0x7f512c0b32d0 2023-11-26T07:22:49.148 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:49.143+0000 7f5130a9c640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f512c0b69d0 con 0x7f512c0b7790 2023-11-26T07:22:49.148 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:49.143+0000 7f512a59c640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f5118004d10 0x7f512c0b6060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:22:49.148 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:49.143+0000 7f512a59c640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f5118004d10 0x7f512c0b6060 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:50396/0 (socket says 172.21.15.143:50396) 2023-11-26T07:22:49.148 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:49.143+0000 7f512a59c640 1 -- 172.21.15.143:0/3943570226 learned_addr learned my addr 172.21.15.143:0/3943570226 (peer_addr_for_me v2:172.21.15.143:0/0) 2023-11-26T07:22:49.148 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:49.143+0000 7f512ad9d640 1 --2- 172.21.15.143:0/3943570226 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f512c0b7790 0x7f512c0b7bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:22:49.148 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:49.143+0000 7f5123fff640 1 --2- 172.21.15.143:0/3943570226 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f512c0b32d0 0x7f512c14c4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:22:49.148 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:49.147+0000 7f512a59c640 1 -- 172.21.15.143:0/3943570226 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f512c0b7790 msgr2=0x7f512c0b7bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:22:49.148 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:49.147+0000 7f512a59c640 1 --2- 172.21.15.143:0/3943570226 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f512c0b7790 0x7f512c0b7bf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:22:49.148 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:49.147+0000 7f512a59c640 1 -- 172.21.15.143:0/3943570226 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f512c0b32d0 msgr2=0x7f512c14c4e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:22:49.148 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:49.147+0000 7f512a59c640 1 --2- 172.21.15.143:0/3943570226 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f512c0b32d0 0x7f512c14c4e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:22:49.149 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:49.147+0000 7f512a59c640 1 -- 172.21.15.143:0/3943570226 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f512c0b6cb0 con 0x7f5118004d10 2023-11-26T07:22:49.149 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:49.147+0000 7f512a59c640 1 --2- 172.21.15.143:0/3943570226 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f5118004d10 0x7f512c0b6060 secure :-1 s=READY pgs=996 cs=0 l=1 rev1=1 crypto rx=0x7f511c0119d0 tx=0x7f511c011e90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:22:49.149 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:49.147+0000 7f5122ffd640 1 -- 172.21.15.143:0/3943570226 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f511c00fad0 con 0x7f5118004d10 2023-11-26T07:22:49.150 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:49.147+0000 7f5122ffd640 1 -- 172.21.15.143:0/3943570226 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f511c00fc30 con 0x7f5118004d10 2023-11-26T07:22:49.150 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:49.147+0000 7f5130a9c640 1 -- 172.21.15.143:0/3943570226 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f512c0b6f90 con 0x7f5118004d10 2023-11-26T07:22:49.150 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:49.147+0000 7f5122ffd640 1 -- 172.21.15.143:0/3943570226 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f511c014440 con 0x7f5118004d10 2023-11-26T07:22:49.150 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:49.147+0000 7f5130a9c640 1 -- 172.21.15.143:0/3943570226 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f512c14ca20 con 0x7f5118004d10 2023-11-26T07:22:49.150 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:49.147+0000 7f5130a9c640 1 -- 172.21.15.143:0/3943570226 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f512c067680 con 0x7f5118004d10 2023-11-26T07:22:49.150 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:49.147+0000 7f5122ffd640 1 -- 172.21.15.143:0/3943570226 <== mon.0 v2:172.21.15.143:3300/0 4 ==== mgrmap(e 11) v1 ==== 86158+0+0 (secure 0 0 0) 0x7f511c01d400 con 0x7f5118004d10 2023-11-26T07:22:49.151 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:49.147+0000 7f5122ffd640 1 --2- 172.21.15.143:0/3943570226 >> [v2:172.21.15.143:6832/2150557544,v1:172.21.15.143:6833/2150557544] conn(0x7f510c0667d0 0x7f510c068c90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:22:49.151 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:49.147+0000 7f5122ffd640 1 -- 172.21.15.143:0/3943570226 <== mon.0 v2:172.21.15.143:3300/0 5 ==== osd_map(71..71 src has 1..71) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f511c084c90 con 0x7f5118004d10 2023-11-26T07:22:49.151 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:49.147+0000 7f5123fff640 1 --2- 172.21.15.143:0/3943570226 >> [v2:172.21.15.143:6832/2150557544,v1:172.21.15.143:6833/2150557544] conn(0x7f510c0667d0 0x7f510c068c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:22:49.151 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:49.151+0000 7f5123fff640 1 --2- 172.21.15.143:0/3943570226 >> [v2:172.21.15.143:6832/2150557544,v1:172.21.15.143:6833/2150557544] conn(0x7f510c0667d0 0x7f510c068c90 secure :-1 s=READY pgs=463 cs=0 l=1 rev1=1 crypto rx=0x7f512c1f6160 tx=0x7f5114009040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:22:49.154 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:49.151+0000 7f5122ffd640 1 -- 172.21.15.143:0/3943570226 <== mon.0 v2:172.21.15.143:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172649 (secure 0 0 0) 0x7f511c051d80 con 0x7f5118004d10 2023-11-26T07:22:49.306 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:49.303+0000 7f5130a9c640 1 -- 172.21.15.143:0/3943570226 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f512c0c9a00 con 0x7f5118004d10 2023-11-26T07:22:49.307 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:49.303+0000 7f5122ffd640 1 -- 172.21.15.143:0/3943570226 <== mon.0 v2:172.21.15.143:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v71) v1 ==== 93+0+10 (secure 0 0 0) 0x7f511c0553a0 con 0x7f5118004d10 2023-11-26T07:22:49.307 INFO:teuthology.orchestra.run.smithi143.stdout:pg_num: 1 2023-11-26T07:22:49.309 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:49.307+0000 7f5130a9c640 1 -- 172.21.15.143:0/3943570226 >> [v2:172.21.15.143:6832/2150557544,v1:172.21.15.143:6833/2150557544] conn(0x7f510c0667d0 msgr2=0x7f510c068c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:22:49.309 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:49.307+0000 7f5130a9c640 1 --2- 172.21.15.143:0/3943570226 >> [v2:172.21.15.143:6832/2150557544,v1:172.21.15.143:6833/2150557544] conn(0x7f510c0667d0 0x7f510c068c90 secure :-1 s=READY pgs=463 cs=0 l=1 rev1=1 crypto rx=0x7f512c1f6160 tx=0x7f5114009040 comp rx=0 tx=0).stop 2023-11-26T07:22:49.309 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:49.307+0000 7f5130a9c640 1 -- 172.21.15.143:0/3943570226 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f5118004d10 msgr2=0x7f512c0b6060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:22:49.309 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:49.307+0000 7f5130a9c640 1 --2- 172.21.15.143:0/3943570226 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f5118004d10 0x7f512c0b6060 secure :-1 s=READY pgs=996 cs=0 l=1 rev1=1 crypto rx=0x7f511c0119d0 tx=0x7f511c011e90 comp rx=0 tx=0).stop 2023-11-26T07:22:49.309 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:49.307+0000 7f5130a9c640 1 -- 172.21.15.143:0/3943570226 shutdown_connections 2023-11-26T07:22:49.309 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:49.307+0000 7f5130a9c640 1 --2- 172.21.15.143:0/3943570226 >> [v2:172.21.15.143:6832/2150557544,v1:172.21.15.143:6833/2150557544] conn(0x7f510c0667d0 0x7f510c068c90 unknown :-1 s=CLOSED pgs=463 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:22:49.309 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:49.307+0000 7f5130a9c640 1 --2- 172.21.15.143:0/3943570226 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f512c0b7790 0x7f512c0b7bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:22:49.309 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:49.307+0000 7f5130a9c640 1 --2- 172.21.15.143:0/3943570226 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f512c0b32d0 0x7f512c14c4e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:22:49.309 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:49.307+0000 7f5130a9c640 1 --2- 172.21.15.143:0/3943570226 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f5118004d10 0x7f512c0b6060 unknown :-1 s=CLOSED pgs=996 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:22:49.309 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:49.307+0000 7f5130a9c640 1 -- 172.21.15.143:0/3943570226 >> 172.21.15.143:0/3943570226 conn(0x7f512c09c550 msgr2=0x7f512c0b5420 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:22:49.310 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:49.307+0000 7f5130a9c640 1 -- 172.21.15.143:0/3943570226 shutdown_connections 2023-11-26T07:22:49.310 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:49.307+0000 7f5130a9c640 1 -- 172.21.15.143:0/3943570226 wait complete. 2023-11-26T07:22:49.328 DEBUG:teuthology.orchestra.run.smithi143:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-11-26T07:22:49.468 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:49.467+0000 7fa87d7c1640 1 Processor -- start 2023-11-26T07:22:49.468 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:49.467+0000 7fa87d7c1640 1 -- start start 2023-11-26T07:22:49.470 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:49.467+0000 7fa87d7c1640 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7fa878143ac0 0x7fa878070370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:22:49.470 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:49.467+0000 7fa87d7c1640 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7fa8780708b0 0x7fa8780cf280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:22:49.470 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:49.467+0000 7fa87d7c1640 1 -- --> v1:172.21.15.143:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa8780cc730 con 0x7fa8780cf7c0 2023-11-26T07:22:49.470 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:49.467+0000 7fa87d7c1640 1 -- --> v2:172.21.15.143:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa878143ea0 con 0x7fa8780708b0 2023-11-26T07:22:49.470 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:49.467+0000 7fa87d7c1640 1 -- --> v2:172.21.15.188:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa8780cfbd0 con 0x7fa878143ac0 2023-11-26T07:22:49.470 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:49.467+0000 7fa86e7fc640 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7fa8780708b0 0x7fa8780cf280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:22:49.471 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:49.467+0000 7fa8777fe640 1 --1- >> v1:172.21.15.143:6789/0 conn(0x7fa8780cf7c0 0x7fa8780b1ef0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.143:6789/0 says I am v1:172.21.15.143:48002/0 (socket says 172.21.15.143:48002) 2023-11-26T07:22:49.471 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:49.467+0000 7fa8777fe640 1 -- 172.21.15.143:0/389198687 learned_addr learned my addr 172.21.15.143:0/389198687 (peer_addr_for_me v1:172.21.15.143:0/0) 2023-11-26T07:22:49.471 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:49.467+0000 7fa876ffd640 1 --2- 172.21.15.143:0/389198687 >> v2:172.21.15.188:3300/0 conn(0x7fa878143ac0 0x7fa878070370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:22:49.471 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:49.467+0000 7fa86e7fc640 1 -- 172.21.15.143:0/389198687 >> v2:172.21.15.188:3300/0 conn(0x7fa878143ac0 msgr2=0x7fa878070370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-26T07:22:49.471 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:49.467+0000 7fa86e7fc640 1 --2- 172.21.15.143:0/389198687 >> v2:172.21.15.188:3300/0 conn(0x7fa878143ac0 0x7fa878070370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:22:49.471 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:49.467+0000 7fa86e7fc640 1 -- 172.21.15.143:0/389198687 >> v1:172.21.15.143:6789/0 conn(0x7fa8780cf7c0 legacy=0x7fa8780b1ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:22:49.471 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:49.467+0000 7fa86e7fc640 1 -- 172.21.15.143:0/389198687 --> v2:172.21.15.143:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa878070ee0 con 0x7fa8780708b0 2023-11-26T07:22:49.473 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:49.471+0000 7fa86e7fc640 1 --2- 172.21.15.143:0/389198687 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fa8780708b0 0x7fa8780cf280 secure :-1 s=READY pgs=998 cs=0 l=1 rev1=1 crypto rx=0x7fa87806dca0 tx=0x7fa86001c460 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a2d3647d3d89b757 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:22:49.474 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:49.471+0000 7fa8767fc640 1 -- 172.21.15.143:0/389198687 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa8600ab660 con 0x7fa8780708b0 2023-11-26T07:22:49.474 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:49.471+0000 7fa8767fc640 1 -- 172.21.15.143:0/389198687 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa8600ab7c0 con 0x7fa8780708b0 2023-11-26T07:22:49.474 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:49.471+0000 7fa8767fc640 1 -- 172.21.15.143:0/389198687 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa8600abb50 con 0x7fa8780708b0 2023-11-26T07:22:49.474 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:49.471+0000 7fa87d7c1640 1 -- 172.21.15.143:0/389198687 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fa8780708b0 msgr2=0x7fa8780cf280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:22:49.474 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:49.471+0000 7fa87d7c1640 1 --2- 172.21.15.143:0/389198687 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fa8780708b0 0x7fa8780cf280 secure :-1 s=READY pgs=998 cs=0 l=1 rev1=1 crypto rx=0x7fa87806dca0 tx=0x7fa86001c460 comp rx=0 tx=0).stop 2023-11-26T07:22:49.474 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:49.471+0000 7fa87d7c1640 1 -- 172.21.15.143:0/389198687 shutdown_connections 2023-11-26T07:22:49.474 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:49.471+0000 7fa87d7c1640 1 --2- 172.21.15.143:0/389198687 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fa8780708b0 0x7fa8780cf280 unknown :-1 s=CLOSED pgs=998 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:22:49.474 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:49.471+0000 7fa87d7c1640 1 --2- 172.21.15.143:0/389198687 >> v2:172.21.15.188:3300/0 conn(0x7fa878143ac0 0x7fa878070370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:22:49.474 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:49.471+0000 7fa87d7c1640 1 -- 172.21.15.143:0/389198687 >> 172.21.15.143:0/389198687 conn(0x7fa87806a890 msgr2=0x7fa87806aca0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:22:49.475 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:49.471+0000 7fa87d7c1640 1 -- 172.21.15.143:0/389198687 shutdown_connections 2023-11-26T07:22:49.475 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:49.471+0000 7fa87d7c1640 1 -- 172.21.15.143:0/389198687 wait complete. 2023-11-26T07:22:49.475 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:49.475+0000 7fa87d7c1640 1 Processor -- start 2023-11-26T07:22:49.476 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:49.475+0000 7fa87d7c1640 1 -- start start 2023-11-26T07:22:49.478 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:49.475+0000 7fa87d7c1640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fa8780708b0 0x7fa878159b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:22:49.478 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:49.475+0000 7fa87d7c1640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fa8780cf7c0 0x7fa87813b3f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:22:49.478 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:49.475+0000 7fa87d7c1640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fa878143ac0 0x7fa87813b930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:22:49.478 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:49.475+0000 7fa87d7c1640 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa8780a9cd0 con 0x7fa878143ac0 2023-11-26T07:22:49.478 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:49.475+0000 7fa87d7c1640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa87815a060 con 0x7fa8780708b0 2023-11-26T07:22:49.478 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:49.475+0000 7fa87d7c1640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa8780b1730 con 0x7fa8780cf7c0 2023-11-26T07:22:49.478 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:49.475+0000 7fa8777fe640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fa878143ac0 0x7fa87813b930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:22:49.479 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:49.475+0000 7fa8777fe640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fa878143ac0 0x7fa87813b930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:50418/0 (socket says 172.21.15.143:50418) 2023-11-26T07:22:49.479 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:49.475+0000 7fa8777fe640 1 -- 172.21.15.143:0/3291906827 learned_addr learned my addr 172.21.15.143:0/3291906827 (peer_addr_for_me v2:172.21.15.143:0/0) 2023-11-26T07:22:49.479 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:49.475+0000 7fa876ffd640 1 --2- 172.21.15.143:0/3291906827 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fa8780708b0 0x7fa878159b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:22:49.479 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:49.475+0000 7fa86e7fc640 1 --2- 172.21.15.143:0/3291906827 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fa8780cf7c0 0x7fa87813b3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:22:49.479 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:49.475+0000 7fa8777fe640 1 -- 172.21.15.143:0/3291906827 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fa8780cf7c0 msgr2=0x7fa87813b3f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:22:49.479 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:49.475+0000 7fa8777fe640 1 --2- 172.21.15.143:0/3291906827 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fa8780cf7c0 0x7fa87813b3f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:22:49.479 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:49.475+0000 7fa8777fe640 1 -- 172.21.15.143:0/3291906827 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fa8780708b0 msgr2=0x7fa878159b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:22:49.479 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:49.475+0000 7fa8777fe640 1 --2- 172.21.15.143:0/3291906827 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fa8780708b0 0x7fa878159b20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:22:49.479 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:49.475+0000 7fa8777fe640 1 -- 172.21.15.143:0/3291906827 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa860009420 con 0x7fa878143ac0 2023-11-26T07:22:49.479 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:49.475+0000 7fa8777fe640 1 --2- 172.21.15.143:0/3291906827 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fa878143ac0 0x7fa87813b930 secure :-1 s=READY pgs=999 cs=0 l=1 rev1=1 crypto rx=0x7fa87000cbf0 tx=0x7fa87000a430 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:22:49.480 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:49.475+0000 7fa86ffff640 1 -- 172.21.15.143:0/3291906827 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa87000ad40 con 0x7fa878143ac0 2023-11-26T07:22:49.480 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:49.475+0000 7fa86ffff640 1 -- 172.21.15.143:0/3291906827 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa870012400 con 0x7fa878143ac0 2023-11-26T07:22:49.480 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:49.475+0000 7fa87d7c1640 1 -- 172.21.15.143:0/3291906827 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa8780b0b20 con 0x7fa878143ac0 2023-11-26T07:22:49.480 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:49.475+0000 7fa86ffff640 1 -- 172.21.15.143:0/3291906827 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa87000aea0 con 0x7fa878143ac0 2023-11-26T07:22:49.481 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:49.475+0000 7fa87d7c1640 1 -- 172.21.15.143:0/3291906827 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa8780b10e0 con 0x7fa878143ac0 2023-11-26T07:22:49.481 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:49.479+0000 7fa87d7c1640 1 -- 172.21.15.143:0/3291906827 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa8780b1b20 con 0x7fa878143ac0 2023-11-26T07:22:49.481 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:49.479+0000 7fa86ffff640 1 -- 172.21.15.143:0/3291906827 <== mon.0 v2:172.21.15.143:3300/0 4 ==== mgrmap(e 11) v1 ==== 86158+0+0 (secure 0 0 0) 0x7fa870007070 con 0x7fa878143ac0 2023-11-26T07:22:49.481 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:49.479+0000 7fa86ffff640 1 --2- 172.21.15.143:0/3291906827 >> [v2:172.21.15.143:6832/2150557544,v1:172.21.15.143:6833/2150557544] conn(0x7fa858066800 0x7fa858068cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:22:49.482 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:49.479+0000 7fa86ffff640 1 -- 172.21.15.143:0/3291906827 <== mon.0 v2:172.21.15.143:3300/0 5 ==== osd_map(71..71 src has 1..71) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fa870084360 con 0x7fa878143ac0 2023-11-26T07:22:49.482 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:49.479+0000 7fa876ffd640 1 --2- 172.21.15.143:0/3291906827 >> [v2:172.21.15.143:6832/2150557544,v1:172.21.15.143:6833/2150557544] conn(0x7fa858066800 0x7fa858068cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:22:49.482 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:49.479+0000 7fa876ffd640 1 --2- 172.21.15.143:0/3291906827 >> [v2:172.21.15.143:6832/2150557544,v1:172.21.15.143:6833/2150557544] conn(0x7fa858066800 0x7fa858068cc0 secure :-1 s=READY pgs=464 cs=0 l=1 rev1=1 crypto rx=0x7fa868007a60 tx=0x7fa868008040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:22:49.485 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:49.483+0000 7fa86ffff640 1 -- 172.21.15.143:0/3291906827 <== mon.0 v2:172.21.15.143:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172649 (secure 0 0 0) 0x7fa870051340 con 0x7fa878143ac0 2023-11-26T07:22:49.635 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:49.631+0000 7fa87d7c1640 1 -- 172.21.15.143:0/3291906827 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7fa878071be0 con 0x7fa878143ac0 2023-11-26T07:22:49.687 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:49.631+0000 7fa86ffff640 1 -- 172.21.15.143:0/3291906827 <== mon.0 v2:172.21.15.143:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v71) v1 ==== 104+0+11 (secure 0 0 0) 0x7fa870054960 con 0x7fa878143ac0 2023-11-26T07:22:49.688 INFO:teuthology.orchestra.run.smithi143.stdout:pg_num: 32 2023-11-26T07:22:49.689 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:49.635+0000 7fa87d7c1640 1 -- 172.21.15.143:0/3291906827 >> [v2:172.21.15.143:6832/2150557544,v1:172.21.15.143:6833/2150557544] conn(0x7fa858066800 msgr2=0x7fa858068cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:22:49.689 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:49.635+0000 7fa87d7c1640 1 --2- 172.21.15.143:0/3291906827 >> [v2:172.21.15.143:6832/2150557544,v1:172.21.15.143:6833/2150557544] conn(0x7fa858066800 0x7fa858068cc0 secure :-1 s=READY pgs=464 cs=0 l=1 rev1=1 crypto rx=0x7fa868007a60 tx=0x7fa868008040 comp rx=0 tx=0).stop 2023-11-26T07:22:49.689 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:49.635+0000 7fa87d7c1640 1 -- 172.21.15.143:0/3291906827 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fa878143ac0 msgr2=0x7fa87813b930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:22:49.689 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:49.635+0000 7fa87d7c1640 1 --2- 172.21.15.143:0/3291906827 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fa878143ac0 0x7fa87813b930 secure :-1 s=READY pgs=999 cs=0 l=1 rev1=1 crypto rx=0x7fa87000cbf0 tx=0x7fa87000a430 comp rx=0 tx=0).stop 2023-11-26T07:22:49.689 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:49.635+0000 7fa87d7c1640 1 -- 172.21.15.143:0/3291906827 shutdown_connections 2023-11-26T07:22:49.689 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:49.635+0000 7fa87d7c1640 1 --2- 172.21.15.143:0/3291906827 >> [v2:172.21.15.143:6832/2150557544,v1:172.21.15.143:6833/2150557544] conn(0x7fa858066800 0x7fa858068cc0 unknown :-1 s=CLOSED pgs=464 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:22:49.690 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:49.635+0000 7fa87d7c1640 1 --2- 172.21.15.143:0/3291906827 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fa878143ac0 0x7fa87813b930 unknown :-1 s=CLOSED pgs=999 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:22:49.690 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:49.635+0000 7fa87d7c1640 1 --2- 172.21.15.143:0/3291906827 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fa8780cf7c0 0x7fa87813b3f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:22:49.690 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:49.635+0000 7fa87d7c1640 1 --2- 172.21.15.143:0/3291906827 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fa8780708b0 0x7fa878159b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:22:49.690 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:49.635+0000 7fa87d7c1640 1 -- 172.21.15.143:0/3291906827 >> 172.21.15.143:0/3291906827 conn(0x7fa87806a890 msgr2=0x7fa87806ac70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:22:49.690 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:49.635+0000 7fa87d7c1640 1 -- 172.21.15.143:0/3291906827 shutdown_connections 2023-11-26T07:22:49.690 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:49.635+0000 7fa87d7c1640 1 -- 172.21.15.143:0/3291906827 wait complete. 2023-11-26T07:22:49.690 DEBUG:teuthology.orchestra.run.smithi143:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-11-26T07:22:49.838 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:49.835+0000 7f16eba54640 1 Processor -- start 2023-11-26T07:22:49.838 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:49.835+0000 7f16eba54640 1 -- start start 2023-11-26T07:22:49.839 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:49.835+0000 7f16eba54640 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7f16e413e290 0x7f16e40cf280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:22:49.839 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:49.835+0000 7f16eba54640 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7f16e415f6b0 0x7f16e40cd7b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:22:49.839 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:49.835+0000 7f16eba54640 1 -- --> v1:172.21.15.143:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f16e415fb20 con 0x7f16e40cf7c0 2023-11-26T07:22:49.840 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:49.835+0000 7f16eba54640 1 -- --> v2:172.21.15.143:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f16e40cfc40 con 0x7f16e415f6b0 2023-11-26T07:22:49.840 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:49.835+0000 7f16eba54640 1 -- --> v2:172.21.15.188:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f16e406c520 con 0x7f16e413e290 2023-11-26T07:22:49.840 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:49.839+0000 7f16e9ff1640 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7f16e415f6b0 0x7f16e40cd7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:22:49.840 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:49.839+0000 7f16e9ff1640 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7f16e415f6b0 0x7f16e40cd7b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:50420/0 (socket says 172.21.15.143:50420) 2023-11-26T07:22:49.840 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:49.839+0000 7f16e9ff1640 1 -- 172.21.15.143:0/2613648920 learned_addr learned my addr 172.21.15.143:0/2613648920 (peer_addr_for_me v2:172.21.15.143:0/0) 2023-11-26T07:22:49.840 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:49.839+0000 7f16e97f0640 1 --2- 172.21.15.143:0/2613648920 >> v2:172.21.15.188:3300/0 conn(0x7f16e413e290 0x7f16e40cf280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:22:49.841 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:49.839+0000 7f16e9ff1640 1 -- 172.21.15.143:0/2613648920 >> v2:172.21.15.188:3300/0 conn(0x7f16e413e290 msgr2=0x7f16e40cf280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-26T07:22:49.841 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:49.839+0000 7f16e9ff1640 1 --2- 172.21.15.143:0/2613648920 >> v2:172.21.15.188:3300/0 conn(0x7f16e413e290 0x7f16e40cf280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:22:49.841 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:49.839+0000 7f16e9ff1640 1 -- 172.21.15.143:0/2613648920 >> v1:172.21.15.143:6789/0 conn(0x7f16e40cf7c0 legacy=0x7f16e415efa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:22:49.841 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:49.839+0000 7f16e9ff1640 1 -- 172.21.15.143:0/2613648920 --> v2:172.21.15.143:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f16e406c800 con 0x7f16e415f6b0 2023-11-26T07:22:49.843 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:49.839+0000 7f16e9ff1640 1 --2- 172.21.15.143:0/2613648920 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f16e415f6b0 0x7f16e40cd7b0 secure :-1 s=READY pgs=1001 cs=0 l=1 rev1=1 crypto rx=0x7f16e40725a0 tx=0x7f16dc01c9a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6006f3f050d7fe1e server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:22:49.843 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:49.839+0000 7f16dbfff640 1 -- 172.21.15.143:0/2613648920 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f16dc0a5860 con 0x7f16e415f6b0 2023-11-26T07:22:49.843 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:49.839+0000 7f16dbfff640 1 -- 172.21.15.143:0/2613648920 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f16dc0a59c0 con 0x7f16e415f6b0 2023-11-26T07:22:49.843 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:49.839+0000 7f16dbfff640 1 -- 172.21.15.143:0/2613648920 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f16dc0a5d50 con 0x7f16e415f6b0 2023-11-26T07:22:49.844 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:49.843+0000 7f16eba54640 1 -- 172.21.15.143:0/2613648920 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f16e415f6b0 msgr2=0x7f16e40cd7b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:22:49.844 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:49.843+0000 7f16eba54640 1 --2- 172.21.15.143:0/2613648920 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f16e415f6b0 0x7f16e40cd7b0 secure :-1 s=READY pgs=1001 cs=0 l=1 rev1=1 crypto rx=0x7f16e40725a0 tx=0x7f16dc01c9a0 comp rx=0 tx=0).stop 2023-11-26T07:22:49.844 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:49.843+0000 7f16eba54640 1 -- 172.21.15.143:0/2613648920 shutdown_connections 2023-11-26T07:22:49.844 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:49.843+0000 7f16eba54640 1 --2- 172.21.15.143:0/2613648920 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f16e415f6b0 0x7f16e40cd7b0 unknown :-1 s=CLOSED pgs=1001 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:22:49.844 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:49.843+0000 7f16eba54640 1 --2- 172.21.15.143:0/2613648920 >> v2:172.21.15.188:3300/0 conn(0x7f16e413e290 0x7f16e40cf280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:22:49.844 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:49.843+0000 7f16eba54640 1 -- 172.21.15.143:0/2613648920 >> 172.21.15.143:0/2613648920 conn(0x7f16e406a890 msgr2=0x7f16e406aca0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:22:49.844 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:49.843+0000 7f16eba54640 1 -- 172.21.15.143:0/2613648920 shutdown_connections 2023-11-26T07:22:49.845 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:49.843+0000 7f16eba54640 1 -- 172.21.15.143:0/2613648920 wait complete. 2023-11-26T07:22:49.845 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:49.843+0000 7f16eba54640 1 Processor -- start 2023-11-26T07:22:49.846 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:49.843+0000 7f16eba54640 1 -- start start 2023-11-26T07:22:49.847 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:49.843+0000 7f16eba54640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f16e40cf7c0 0x7f16e4153a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:22:49.847 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:49.843+0000 7f16eba54640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f16e413e290 0x7f16e4133cd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:22:49.847 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:49.843+0000 7f16eba54640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f16e415f6b0 0x7f16e40afe80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:22:49.847 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:49.843+0000 7f16eba54640 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f16e40b2050 con 0x7f16e415f6b0 2023-11-26T07:22:49.847 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:49.843+0000 7f16eba54640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f16e40b1d50 con 0x7f16e413e290 2023-11-26T07:22:49.847 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:49.843+0000 7f16eba54640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f16e4153f60 con 0x7f16e40cf7c0 2023-11-26T07:22:49.848 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:49.843+0000 7f16e9ff1640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f16e415f6b0 0x7f16e40afe80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:22:49.848 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:49.843+0000 7f16e9ff1640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f16e415f6b0 0x7f16e40afe80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:50424/0 (socket says 172.21.15.143:50424) 2023-11-26T07:22:49.848 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:49.843+0000 7f16e9ff1640 1 -- 172.21.15.143:0/2316143696 learned_addr learned my addr 172.21.15.143:0/2316143696 (peer_addr_for_me v2:172.21.15.143:0/0) 2023-11-26T07:22:49.848 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:49.847+0000 7f16e97f0640 1 --2- 172.21.15.143:0/2316143696 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f16e40cf7c0 0x7f16e4153a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:22:49.848 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:49.847+0000 7f16e9ff1640 1 -- 172.21.15.143:0/2316143696 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f16e40cf7c0 msgr2=0x7f16e4153a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:22:49.849 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:49.847+0000 7f16e9ff1640 1 --2- 172.21.15.143:0/2316143696 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f16e40cf7c0 0x7f16e4153a20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:22:49.849 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:49.847+0000 7f16e8fef640 1 --2- 172.21.15.143:0/2316143696 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f16e413e290 0x7f16e4133cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:22:49.849 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:49.847+0000 7f16e9ff1640 1 -- 172.21.15.143:0/2316143696 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f16e413e290 msgr2=0x7f16e4133cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:22:49.849 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:49.847+0000 7f16e9ff1640 1 --2- 172.21.15.143:0/2316143696 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f16e413e290 0x7f16e4133cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:22:49.849 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:49.847+0000 7f16e9ff1640 1 -- 172.21.15.143:0/2316143696 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f16dc009b30 con 0x7f16e415f6b0 2023-11-26T07:22:49.850 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:49.847+0000 7f16e8fef640 1 --2- 172.21.15.143:0/2316143696 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f16e413e290 0x7f16e4133cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-26T07:22:49.850 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:49.847+0000 7f16e9ff1640 1 --2- 172.21.15.143:0/2316143696 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f16e415f6b0 0x7f16e40afe80 secure :-1 s=READY pgs=1002 cs=0 l=1 rev1=1 crypto rx=0x7f16dc0af040 tx=0x7f16dc019740 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:22:49.850 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:49.847+0000 7f16d9ffb640 1 -- 172.21.15.143:0/2316143696 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f16dc0b8670 con 0x7f16e415f6b0 2023-11-26T07:22:49.850 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:49.847+0000 7f16d9ffb640 1 -- 172.21.15.143:0/2316143696 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f16dc0b8810 con 0x7f16e415f6b0 2023-11-26T07:22:49.850 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:49.847+0000 7f16d9ffb640 1 -- 172.21.15.143:0/2316143696 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f16dc0b8bc0 con 0x7f16e415f6b0 2023-11-26T07:22:49.850 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:49.847+0000 7f16eba54640 1 -- 172.21.15.143:0/2316143696 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f16e40b0be0 con 0x7f16e415f6b0 2023-11-26T07:22:49.850 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:49.847+0000 7f16eba54640 1 -- 172.21.15.143:0/2316143696 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f16e40b11a0 con 0x7f16e415f6b0 2023-11-26T07:22:49.850 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:49.847+0000 7f16eba54640 1 -- 172.21.15.143:0/2316143696 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f16e40b21c0 con 0x7f16e415f6b0 2023-11-26T07:22:49.850 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:49.847+0000 7f16d9ffb640 1 -- 172.21.15.143:0/2316143696 <== mon.0 v2:172.21.15.143:3300/0 4 ==== mgrmap(e 11) v1 ==== 86158+0+0 (secure 0 0 0) 0x7f16dc0b8d20 con 0x7f16e415f6b0 2023-11-26T07:22:49.851 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:49.847+0000 7f16d9ffb640 1 --2- 172.21.15.143:0/2316143696 >> [v2:172.21.15.143:6832/2150557544,v1:172.21.15.143:6833/2150557544] conn(0x7f16c80667d0 0x7f16c8068c90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:22:49.851 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:49.851+0000 7f16d9ffb640 1 -- 172.21.15.143:0/2316143696 <== mon.0 v2:172.21.15.143:3300/0 5 ==== osd_map(71..71 src has 1..71) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f16dc11fa20 con 0x7f16e415f6b0 2023-11-26T07:22:49.852 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:49.851+0000 7f16e97f0640 1 --2- 172.21.15.143:0/2316143696 >> [v2:172.21.15.143:6832/2150557544,v1:172.21.15.143:6833/2150557544] conn(0x7f16c80667d0 0x7f16c8068c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:22:49.852 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:49.851+0000 7f16e97f0640 1 --2- 172.21.15.143:0/2316143696 >> [v2:172.21.15.143:6832/2150557544,v1:172.21.15.143:6833/2150557544] conn(0x7f16c80667d0 0x7f16c8068c90 secure :-1 s=READY pgs=465 cs=0 l=1 rev1=1 crypto rx=0x7f16e415aab0 tx=0x7f16e0008040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:22:49.855 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:49.855+0000 7f16d9ffb640 1 -- 172.21.15.143:0/2316143696 <== mon.0 v2:172.21.15.143:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172649 (secure 0 0 0) 0x7f16dc0ecb80 con 0x7f16e415f6b0 2023-11-26T07:22:50.008 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:50.007+0000 7f16eba54640 1 -- 172.21.15.143:0/2316143696 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f16e415e800 con 0x7f16e415f6b0 2023-11-26T07:22:50.008 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:50.007+0000 7f16d9ffb640 1 -- 172.21.15.143:0/2316143696 <== mon.0 v2:172.21.15.143:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v71) v1 ==== 100+0+11 (secure 0 0 0) 0x7f16dc0f01a0 con 0x7f16e415f6b0 2023-11-26T07:22:50.008 INFO:teuthology.orchestra.run.smithi143.stdout:pg_num: 64 2023-11-26T07:22:50.014 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:50.011+0000 7f16eba54640 1 -- 172.21.15.143:0/2316143696 >> [v2:172.21.15.143:6832/2150557544,v1:172.21.15.143:6833/2150557544] conn(0x7f16c80667d0 msgr2=0x7f16c8068c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:22:50.014 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:50.011+0000 7f16eba54640 1 --2- 172.21.15.143:0/2316143696 >> [v2:172.21.15.143:6832/2150557544,v1:172.21.15.143:6833/2150557544] conn(0x7f16c80667d0 0x7f16c8068c90 secure :-1 s=READY pgs=465 cs=0 l=1 rev1=1 crypto rx=0x7f16e415aab0 tx=0x7f16e0008040 comp rx=0 tx=0).stop 2023-11-26T07:22:50.014 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:50.011+0000 7f16eba54640 1 -- 172.21.15.143:0/2316143696 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f16e415f6b0 msgr2=0x7f16e40afe80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:22:50.014 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:50.011+0000 7f16eba54640 1 --2- 172.21.15.143:0/2316143696 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f16e415f6b0 0x7f16e40afe80 secure :-1 s=READY pgs=1002 cs=0 l=1 rev1=1 crypto rx=0x7f16dc0af040 tx=0x7f16dc019740 comp rx=0 tx=0).stop 2023-11-26T07:22:50.014 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:50.011+0000 7f16eba54640 1 -- 172.21.15.143:0/2316143696 shutdown_connections 2023-11-26T07:22:50.014 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:50.011+0000 7f16eba54640 1 --2- 172.21.15.143:0/2316143696 >> [v2:172.21.15.143:6832/2150557544,v1:172.21.15.143:6833/2150557544] conn(0x7f16c80667d0 0x7f16c8068c90 unknown :-1 s=CLOSED pgs=465 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:22:50.015 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:50.011+0000 7f16eba54640 1 --2- 172.21.15.143:0/2316143696 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f16e415f6b0 0x7f16e40afe80 unknown :-1 s=CLOSED pgs=1002 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:22:50.015 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:50.011+0000 7f16eba54640 1 --2- 172.21.15.143:0/2316143696 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f16e413e290 0x7f16e4133cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:22:50.015 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:50.011+0000 7f16eba54640 1 --2- 172.21.15.143:0/2316143696 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f16e40cf7c0 0x7f16e4153a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:22:50.015 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:50.011+0000 7f16eba54640 1 -- 172.21.15.143:0/2316143696 >> 172.21.15.143:0/2316143696 conn(0x7f16e406a890 msgr2=0x7f16e4071360 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:22:50.015 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:50.011+0000 7f16eba54640 1 -- 172.21.15.143:0/2316143696 shutdown_connections 2023-11-26T07:22:50.015 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:50.011+0000 7f16eba54640 1 -- 172.21.15.143:0/2316143696 wait complete. 2023-11-26T07:22:50.039 DEBUG:teuthology.orchestra.run.smithi143:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2023-11-26T07:22:50.178 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:50.175+0000 7fa1fcd72640 1 Processor -- start 2023-11-26T07:22:50.178 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:50.175+0000 7fa1fcd72640 1 -- start start 2023-11-26T07:22:50.179 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:50.179+0000 7fa1fcd72640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fa1f80cf280 0x7fa1f80cf6e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:22:50.180 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:50.179+0000 7fa1fcd72640 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7fa1f80cd1b0 0x7fa1f80cd5e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:22:50.180 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:50.179+0000 7fa1fcd72640 1 -- --> v1:172.21.15.188:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa1f80cfc20 con 0x7fa1f8143ac0 2023-11-26T07:22:50.180 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:50.179+0000 7fa1fcd72640 1 -- --> v2:172.21.15.188:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa1f8143ea0 con 0x7fa1f80cd1b0 2023-11-26T07:22:50.180 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:50.179+0000 7fa1fcd72640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa1f80cc730 con 0x7fa1f80cf280 2023-11-26T07:22:50.181 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:50.179+0000 7fa1f5d9b640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fa1f80cf280 0x7fa1f80cf6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:22:50.181 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:50.179+0000 7fa1f5d9b640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fa1f80cf280 0x7fa1f80cf6e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.188:3301/0 says I am v2:172.21.15.143:60060/0 (socket says 172.21.15.143:60060) 2023-11-26T07:22:50.181 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:50.179+0000 7fa1f5d9b640 1 -- 172.21.15.143:0/1961762855 learned_addr learned my addr 172.21.15.143:0/1961762855 (peer_addr_for_me v2:172.21.15.143:0/0) 2023-11-26T07:22:50.181 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:50.179+0000 7fa1f6d9d640 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7fa1f80cd1b0 0x7fa1f80cd5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:22:50.181 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:50.179+0000 7fa1f5d9b640 1 -- 172.21.15.143:0/1961762855 >> v2:172.21.15.188:3300/0 conn(0x7fa1f80cd1b0 msgr2=0x7fa1f80cd5e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-26T07:22:50.181 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:50.179+0000 7fa1f5d9b640 1 --2- 172.21.15.143:0/1961762855 >> v2:172.21.15.188:3300/0 conn(0x7fa1f80cd1b0 0x7fa1f80cd5e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:22:50.181 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:50.179+0000 7fa1f5d9b640 1 -- 172.21.15.143:0/1961762855 >> v1:172.21.15.188:6789/0 conn(0x7fa1f8143ac0 legacy=0x7fa1f8070370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:22:50.181 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:50.179+0000 7fa1f5d9b640 1 -- 172.21.15.143:0/1961762855 --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa1f8070ee0 con 0x7fa1f80cf280 2023-11-26T07:22:50.183 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:50.183+0000 7fa1f5d9b640 1 --2- 172.21.15.143:0/1961762855 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fa1f80cf280 0x7fa1f80cf6e0 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7fa1ec009870 tx=0x7fa1ec01e750 comp rx=0 tx=0).ready entity=mon.2 client_cookie=348b80691309928c server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:22:50.184 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:50.183+0000 7fa1f559a640 1 -- 172.21.15.143:0/1961762855 <== mon.2 v2:172.21.15.188:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa1ec0a7710 con 0x7fa1f80cf280 2023-11-26T07:22:50.184 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:50.183+0000 7fa1f559a640 1 -- 172.21.15.143:0/1961762855 <== mon.2 v2:172.21.15.188:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa1ec0a7870 con 0x7fa1f80cf280 2023-11-26T07:22:50.184 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:50.183+0000 7fa1f559a640 1 -- 172.21.15.143:0/1961762855 <== mon.2 v2:172.21.15.188:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa1ec0a7c00 con 0x7fa1f80cf280 2023-11-26T07:22:50.184 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:50.183+0000 7fa1fcd72640 1 -- 172.21.15.143:0/1961762855 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fa1f80cf280 msgr2=0x7fa1f80cf6e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:22:50.184 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:50.183+0000 7fa1fcd72640 1 --2- 172.21.15.143:0/1961762855 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fa1f80cf280 0x7fa1f80cf6e0 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7fa1ec009870 tx=0x7fa1ec01e750 comp rx=0 tx=0).stop 2023-11-26T07:22:50.185 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:50.183+0000 7fa1fcd72640 1 -- 172.21.15.143:0/1961762855 shutdown_connections 2023-11-26T07:22:50.185 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:50.183+0000 7fa1fcd72640 1 --2- 172.21.15.143:0/1961762855 >> v2:172.21.15.188:3300/0 conn(0x7fa1f80cd1b0 0x7fa1f80cd5e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:22:50.185 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:50.183+0000 7fa1fcd72640 1 --2- 172.21.15.143:0/1961762855 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fa1f80cf280 0x7fa1f80cf6e0 unknown :-1 s=CLOSED pgs=126 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:22:50.185 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:50.183+0000 7fa1fcd72640 1 -- 172.21.15.143:0/1961762855 >> 172.21.15.143:0/1961762855 conn(0x7fa1f806a890 msgr2=0x7fa1f806aca0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:22:50.185 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:50.183+0000 7fa1fcd72640 1 -- 172.21.15.143:0/1961762855 shutdown_connections 2023-11-26T07:22:50.185 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:50.183+0000 7fa1fcd72640 1 -- 172.21.15.143:0/1961762855 wait complete. 2023-11-26T07:22:50.186 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:50.183+0000 7fa1fcd72640 1 Processor -- start 2023-11-26T07:22:50.187 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:50.183+0000 7fa1fcd72640 1 -- start start 2023-11-26T07:22:50.188 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:50.187+0000 7fa1fcd72640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fa1f80cd1b0 0x7fa1f8156b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:22:50.188 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:50.187+0000 7fa1fcd72640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fa1f80cf280 0x7fa1f813b970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:22:50.188 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:50.187+0000 7fa1fcd72640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fa1f8143ac0 0x7fa1f8142420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:22:50.188 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:50.187+0000 7fa1fcd72640 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa1f81570c0 con 0x7fa1f8143ac0 2023-11-26T07:22:50.188 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:50.187+0000 7fa1fcd72640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa1f813c750 con 0x7fa1f80cf280 2023-11-26T07:22:50.188 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:50.187+0000 7fa1fcd72640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa1f8158660 con 0x7fa1f80cd1b0 2023-11-26T07:22:50.189 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:50.187+0000 7fa1f659c640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fa1f80cd1b0 0x7fa1f8156b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:22:50.190 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:50.187+0000 7fa1f6d9d640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fa1f8143ac0 0x7fa1f8142420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:22:50.190 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:50.187+0000 7fa1f659c640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fa1f80cd1b0 0x7fa1f8156b80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.188:3301/0 says I am v2:172.21.15.143:60068/0 (socket says 172.21.15.143:60068) 2023-11-26T07:22:50.190 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:50.187+0000 7fa1f6d9d640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fa1f8143ac0 0x7fa1f8142420 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:50436/0 (socket says 172.21.15.143:50436) 2023-11-26T07:22:50.190 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:50.187+0000 7fa1f659c640 1 -- 172.21.15.143:0/1216485128 learned_addr learned my addr 172.21.15.143:0/1216485128 (peer_addr_for_me v2:172.21.15.143:0/0) 2023-11-26T07:22:50.190 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:50.187+0000 7fa1f5d9b640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fa1f80cf280 0x7fa1f813b970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:22:50.190 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:50.187+0000 7fa1f659c640 1 -- 172.21.15.143:0/1216485128 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fa1f80cf280 msgr2=0x7fa1f813b970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:22:50.190 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:50.187+0000 7fa1f659c640 1 --2- 172.21.15.143:0/1216485128 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fa1f80cf280 0x7fa1f813b970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:22:50.190 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:50.187+0000 7fa1f659c640 1 -- 172.21.15.143:0/1216485128 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fa1f8143ac0 msgr2=0x7fa1f8142420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:22:50.190 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:50.187+0000 7fa1f659c640 1 --2- 172.21.15.143:0/1216485128 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fa1f8143ac0 0x7fa1f8142420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:22:50.190 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:50.187+0000 7fa1f659c640 1 -- 172.21.15.143:0/1216485128 --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa1ec0094d0 con 0x7fa1f80cd1b0 2023-11-26T07:22:50.190 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:50.187+0000 7fa1f6d9d640 1 --2- 172.21.15.143:0/1216485128 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fa1f8143ac0 0x7fa1f8142420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-26T07:22:50.191 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:50.187+0000 7fa1f659c640 1 --2- 172.21.15.143:0/1216485128 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fa1f80cd1b0 0x7fa1f8156b80 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7fa1e800c3c0 tx=0x7fa1e800c880 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:22:50.191 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:50.187+0000 7fa1e6ffd640 1 -- 172.21.15.143:0/1216485128 <== mon.2 v2:172.21.15.188:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa1e80052d0 con 0x7fa1f80cd1b0 2023-11-26T07:22:50.191 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:50.187+0000 7fa1e6ffd640 1 -- 172.21.15.143:0/1216485128 <== mon.2 v2:172.21.15.188:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa1e8005430 con 0x7fa1f80cd1b0 2023-11-26T07:22:50.191 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:50.187+0000 7fa1e6ffd640 1 -- 172.21.15.143:0/1216485128 <== mon.2 v2:172.21.15.188:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa1e8003840 con 0x7fa1f80cd1b0 2023-11-26T07:22:50.191 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:50.187+0000 7fa1fcd72640 1 -- 172.21.15.143:0/1216485128 --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa1f813c470 con 0x7fa1f80cd1b0 2023-11-26T07:22:50.191 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:50.187+0000 7fa1fcd72640 1 -- 172.21.15.143:0/1216485128 --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa1f8158b90 con 0x7fa1f80cd1b0 2023-11-26T07:22:50.191 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:50.187+0000 7fa1fcd72640 1 -- 172.21.15.143:0/1216485128 --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa1f8067680 con 0x7fa1f80cd1b0 2023-11-26T07:22:50.192 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:50.191+0000 7fa1e6ffd640 1 -- 172.21.15.143:0/1216485128 <== mon.2 v2:172.21.15.188:3301/0 4 ==== mgrmap(e 11) v1 ==== 86158+0+0 (secure 0 0 0) 0x7fa1e80039a0 con 0x7fa1f80cd1b0 2023-11-26T07:22:50.193 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:50.191+0000 7fa1e6ffd640 1 --2- 172.21.15.143:0/1216485128 >> [v2:172.21.15.143:6832/2150557544,v1:172.21.15.143:6833/2150557544] conn(0x7fa1d80667d0 0x7fa1d8068c90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:22:50.193 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:50.191+0000 7fa1e6ffd640 1 -- 172.21.15.143:0/1216485128 <== mon.2 v2:172.21.15.188:3301/0 5 ==== osd_map(71..71 src has 1..71) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fa1e8083920 con 0x7fa1f80cd1b0 2023-11-26T07:22:50.193 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:50.191+0000 7fa1f5d9b640 1 --2- 172.21.15.143:0/1216485128 >> [v2:172.21.15.143:6832/2150557544,v1:172.21.15.143:6833/2150557544] conn(0x7fa1d80667d0 0x7fa1d8068c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:22:50.193 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:50.191+0000 7fa1f5d9b640 1 --2- 172.21.15.143:0/1216485128 >> [v2:172.21.15.143:6832/2150557544,v1:172.21.15.143:6833/2150557544] conn(0x7fa1d80667d0 0x7fa1d8068c90 secure :-1 s=READY pgs=466 cs=0 l=1 rev1=1 crypto rx=0x7fa1ec0b1000 tx=0x7fa1ec009040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:22:50.198 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:50.195+0000 7fa1e6ffd640 1 -- 172.21.15.143:0/1216485128 <== mon.2 v2:172.21.15.188:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172649 (secure 0 0 0) 0x7fa1e8050a00 con 0x7fa1f80cd1b0 2023-11-26T07:22:50.364 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:50.363+0000 7fa1fcd72640 1 -- 172.21.15.143:0/1216485128 --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) v1 -- 0x7fa1f813b510 con 0x7fa1f80cd1b0 2023-11-26T07:22:50.365 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:50.363+0000 7fa1e6ffd640 1 -- 172.21.15.143:0/1216485128 <== mon.2 v2:172.21.15.188:3301/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v71) v1 ==== 80+0+0 (secure 0 0 0) 0x7fa1e8017080 con 0x7fa1f80cd1b0 2023-11-26T07:22:50.365 INFO:teuthology.orchestra.run.smithi143.stderr:listed 0 entries 2023-11-26T07:22:50.367 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:50.363+0000 7fa1fcd72640 1 -- 172.21.15.143:0/1216485128 >> [v2:172.21.15.143:6832/2150557544,v1:172.21.15.143:6833/2150557544] conn(0x7fa1d80667d0 msgr2=0x7fa1d8068c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:22:50.367 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:50.363+0000 7fa1fcd72640 1 --2- 172.21.15.143:0/1216485128 >> [v2:172.21.15.143:6832/2150557544,v1:172.21.15.143:6833/2150557544] conn(0x7fa1d80667d0 0x7fa1d8068c90 secure :-1 s=READY pgs=466 cs=0 l=1 rev1=1 crypto rx=0x7fa1ec0b1000 tx=0x7fa1ec009040 comp rx=0 tx=0).stop 2023-11-26T07:22:50.367 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:50.363+0000 7fa1fcd72640 1 -- 172.21.15.143:0/1216485128 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fa1f80cd1b0 msgr2=0x7fa1f8156b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:22:50.367 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:50.363+0000 7fa1fcd72640 1 --2- 172.21.15.143:0/1216485128 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fa1f80cd1b0 0x7fa1f8156b80 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7fa1e800c3c0 tx=0x7fa1e800c880 comp rx=0 tx=0).stop 2023-11-26T07:22:50.367 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:50.363+0000 7fa1fcd72640 1 -- 172.21.15.143:0/1216485128 shutdown_connections 2023-11-26T07:22:50.367 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:50.363+0000 7fa1fcd72640 1 --2- 172.21.15.143:0/1216485128 >> [v2:172.21.15.143:6832/2150557544,v1:172.21.15.143:6833/2150557544] conn(0x7fa1d80667d0 0x7fa1d8068c90 unknown :-1 s=CLOSED pgs=466 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:22:50.367 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:50.363+0000 7fa1fcd72640 1 --2- 172.21.15.143:0/1216485128 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fa1f8143ac0 0x7fa1f8142420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:22:50.368 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:50.363+0000 7fa1fcd72640 1 --2- 172.21.15.143:0/1216485128 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fa1f80cf280 0x7fa1f813b970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:22:50.368 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:50.363+0000 7fa1fcd72640 1 --2- 172.21.15.143:0/1216485128 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fa1f80cd1b0 0x7fa1f8156b80 unknown :-1 s=CLOSED pgs=127 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:22:50.368 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:50.363+0000 7fa1fcd72640 1 -- 172.21.15.143:0/1216485128 >> 172.21.15.143:0/1216485128 conn(0x7fa1f806a890 msgr2=0x7fa1f806ce40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:22:50.368 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:50.363+0000 7fa1fcd72640 1 -- 172.21.15.143:0/1216485128 shutdown_connections 2023-11-26T07:22:50.368 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:50.363+0000 7fa1fcd72640 1 -- 172.21.15.143:0/1216485128 wait complete. 2023-11-26T07:22:50.386 DEBUG:teuthology.orchestra.run.smithi143:> set -ex 2023-11-26T07:22:50.386 DEBUG:teuthology.orchestra.run.smithi143:> dd if=/proc/self/mounts of=/dev/stdout 2023-11-26T07:22:50.396 DEBUG:teuthology.orchestra.run.smithi143:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-11-26T07:22:50.577 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:50.575+0000 7fe27bbf8640 1 Processor -- start 2023-11-26T07:22:50.577 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:50.575+0000 7fe27bbf8640 1 -- start start 2023-11-26T07:22:50.579 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:50.575+0000 7fe27bbf8640 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7fe27415d6e0 0x7fe2740d0df0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:22:50.579 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:50.575+0000 7fe27bbf8640 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7fe2740b2160 0x7fe2740d14b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:22:50.579 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:50.575+0000 7fe27bbf8640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fe2740d19f0 0x7fe2740cfbf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:22:50.579 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:50.575+0000 7fe27bbf8640 1 -- --> v2:172.21.15.143:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe27415dac0 con 0x7fe27415d6e0 2023-11-26T07:22:50.579 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:50.575+0000 7fe27bbf8640 1 -- --> v2:172.21.15.188:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe2740d1e80 con 0x7fe2740b2160 2023-11-26T07:22:50.579 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:50.575+0000 7fe27bbf8640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe2740cf0e0 con 0x7fe2740d19f0 2023-11-26T07:22:50.579 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:50.575+0000 7fe279994640 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7fe27415d6e0 0x7fe2740d0df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:22:50.579 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:50.575+0000 7fe279994640 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7fe27415d6e0 0x7fe2740d0df0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:50446/0 (socket says 172.21.15.143:50446) 2023-11-26T07:22:50.579 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:50.575+0000 7fe279994640 1 -- 172.21.15.143:0/4238670293 learned_addr learned my addr 172.21.15.143:0/4238670293 (peer_addr_for_me v2:172.21.15.143:0/0) 2023-11-26T07:22:50.580 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:50.575+0000 7fe27a195640 1 --2- 172.21.15.143:0/4238670293 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fe2740d19f0 0x7fe2740cfbf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:22:50.580 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:50.575+0000 7fe279193640 1 --2- 172.21.15.143:0/4238670293 >> v2:172.21.15.188:3300/0 conn(0x7fe2740b2160 0x7fe2740d14b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:22:50.580 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:50.579+0000 7fe279994640 1 -- 172.21.15.143:0/4238670293 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fe2740d19f0 msgr2=0x7fe2740cfbf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-26T07:22:50.580 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:50.579+0000 7fe279994640 1 --2- 172.21.15.143:0/4238670293 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fe2740d19f0 0x7fe2740cfbf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:22:50.580 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:50.579+0000 7fe279994640 1 -- 172.21.15.143:0/4238670293 >> v2:172.21.15.188:3300/0 conn(0x7fe2740b2160 msgr2=0x7fe2740d14b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-26T07:22:50.580 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:50.579+0000 7fe279994640 1 --2- 172.21.15.143:0/4238670293 >> v2:172.21.15.188:3300/0 conn(0x7fe2740b2160 0x7fe2740d14b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:22:50.581 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:50.579+0000 7fe279994640 1 -- 172.21.15.143:0/4238670293 --> v2:172.21.15.143:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe2740ceb20 con 0x7fe27415d6e0 2023-11-26T07:22:50.582 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:50.579+0000 7fe279994640 1 --2- 172.21.15.143:0/4238670293 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fe27415d6e0 0x7fe2740d0df0 secure :-1 s=READY pgs=1003 cs=0 l=1 rev1=1 crypto rx=0x7fe2740ca2a0 tx=0x7fe27001c240 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2b3782b5e5ef9a3d server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:22:50.582 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:50.579+0000 7fe278992640 1 -- 172.21.15.143:0/4238670293 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe2700ab600 con 0x7fe27415d6e0 2023-11-26T07:22:50.583 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:50.579+0000 7fe278992640 1 -- 172.21.15.143:0/4238670293 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe2700a7050 con 0x7fe27415d6e0 2023-11-26T07:22:50.583 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:50.579+0000 7fe278992640 1 -- 172.21.15.143:0/4238670293 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe2700ab980 con 0x7fe27415d6e0 2023-11-26T07:22:50.583 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:50.579+0000 7fe27bbf8640 1 -- 172.21.15.143:0/4238670293 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fe27415d6e0 msgr2=0x7fe2740d0df0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:22:50.583 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:50.579+0000 7fe27bbf8640 1 --2- 172.21.15.143:0/4238670293 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fe27415d6e0 0x7fe2740d0df0 secure :-1 s=READY pgs=1003 cs=0 l=1 rev1=1 crypto rx=0x7fe2740ca2a0 tx=0x7fe27001c240 comp rx=0 tx=0).stop 2023-11-26T07:22:50.584 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:50.583+0000 7fe27bbf8640 1 -- 172.21.15.143:0/4238670293 shutdown_connections 2023-11-26T07:22:50.584 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:50.583+0000 7fe27bbf8640 1 --2- 172.21.15.143:0/4238670293 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fe2740d19f0 0x7fe2740cfbf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:22:50.584 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:50.583+0000 7fe27bbf8640 1 --2- 172.21.15.143:0/4238670293 >> v2:172.21.15.188:3300/0 conn(0x7fe2740b2160 0x7fe2740d14b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:22:50.584 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:50.583+0000 7fe27bbf8640 1 --2- 172.21.15.143:0/4238670293 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fe27415d6e0 0x7fe2740d0df0 unknown :-1 s=CLOSED pgs=1003 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:22:50.584 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:50.583+0000 7fe27bbf8640 1 -- 172.21.15.143:0/4238670293 >> 172.21.15.143:0/4238670293 conn(0x7fe27409c550 msgr2=0x7fe2740d3220 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:22:50.584 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:50.583+0000 7fe27bbf8640 1 -- 172.21.15.143:0/4238670293 shutdown_connections 2023-11-26T07:22:50.584 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:50.583+0000 7fe27bbf8640 1 -- 172.21.15.143:0/4238670293 wait complete. 2023-11-26T07:22:50.585 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:50.583+0000 7fe27bbf8640 1 Processor -- start 2023-11-26T07:22:50.585 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:50.583+0000 7fe27bbf8640 1 -- start start 2023-11-26T07:22:50.587 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:50.583+0000 7fe27bbf8640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fe2740b2160 0x7fe2740ccd30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:22:50.587 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:50.583+0000 7fe27bbf8640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fe2740d19f0 0x7fe2740b0f10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:22:50.587 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:50.583+0000 7fe27bbf8640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fe27415d6e0 0x7fe27415e460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:22:50.587 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:50.583+0000 7fe27bbf8640 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe27415dca0 con 0x7fe2740b2160 2023-11-26T07:22:50.587 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:50.583+0000 7fe27bbf8640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe2740cc2e0 con 0x7fe2740d19f0 2023-11-26T07:22:50.587 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:50.583+0000 7fe27bbf8640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe2740cc5c0 con 0x7fe27415d6e0 2023-11-26T07:22:50.587 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:50.583+0000 7fe279994640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fe2740b2160 0x7fe2740ccd30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:22:50.588 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:50.583+0000 7fe279994640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fe2740b2160 0x7fe2740ccd30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:50458/0 (socket says 172.21.15.143:50458) 2023-11-26T07:22:50.588 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:50.583+0000 7fe279994640 1 -- 172.21.15.143:0/855470737 learned_addr learned my addr 172.21.15.143:0/855470737 (peer_addr_for_me v2:172.21.15.143:0/0) 2023-11-26T07:22:50.588 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:50.583+0000 7fe279193640 1 --2- 172.21.15.143:0/855470737 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fe2740d19f0 0x7fe2740b0f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:22:50.588 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:50.583+0000 7fe27a195640 1 --2- 172.21.15.143:0/855470737 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fe27415d6e0 0x7fe27415e460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:22:50.588 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:50.583+0000 7fe279994640 1 -- 172.21.15.143:0/855470737 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fe27415d6e0 msgr2=0x7fe27415e460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:22:50.588 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:50.583+0000 7fe279994640 1 --2- 172.21.15.143:0/855470737 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fe27415d6e0 0x7fe27415e460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:22:50.588 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:50.583+0000 7fe279994640 1 -- 172.21.15.143:0/855470737 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fe2740d19f0 msgr2=0x7fe2740b0f10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:22:50.588 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:50.583+0000 7fe279994640 1 --2- 172.21.15.143:0/855470737 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fe2740d19f0 0x7fe2740b0f10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:22:50.588 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:50.583+0000 7fe279994640 1 -- 172.21.15.143:0/855470737 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe2700094d0 con 0x7fe2740b2160 2023-11-26T07:22:50.588 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:50.587+0000 7fe279994640 1 --2- 172.21.15.143:0/855470737 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fe2740b2160 0x7fe2740ccd30 secure :-1 s=READY pgs=1004 cs=0 l=1 rev1=1 crypto rx=0x7fe27001a8f0 tx=0x7fe270018fe0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:22:50.589 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:50.587+0000 7fe26a7fc640 1 -- 172.21.15.143:0/855470737 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe2700b4620 con 0x7fe2740b2160 2023-11-26T07:22:50.589 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:50.587+0000 7fe26a7fc640 1 -- 172.21.15.143:0/855470737 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe2700b4780 con 0x7fe2740b2160 2023-11-26T07:22:50.589 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:50.587+0000 7fe27bbf8640 1 -- 172.21.15.143:0/855470737 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe27415df40 con 0x7fe2740b2160 2023-11-26T07:22:50.589 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:50.587+0000 7fe26a7fc640 1 -- 172.21.15.143:0/855470737 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe2700b4b30 con 0x7fe2740b2160 2023-11-26T07:22:50.589 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:50.587+0000 7fe27bbf8640 1 -- 172.21.15.143:0/855470737 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe2741514b0 con 0x7fe2740b2160 2023-11-26T07:22:50.589 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:50.587+0000 7fe27bbf8640 1 -- 172.21.15.143:0/855470737 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe274067680 con 0x7fe2740b2160 2023-11-26T07:22:50.590 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:50.587+0000 7fe26a7fc640 1 -- 172.21.15.143:0/855470737 <== mon.0 v2:172.21.15.143:3300/0 4 ==== mgrmap(e 11) v1 ==== 86158+0+0 (secure 0 0 0) 0x7fe2700b2070 con 0x7fe2740b2160 2023-11-26T07:22:50.591 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:50.587+0000 7fe26a7fc640 1 --2- 172.21.15.143:0/855470737 >> [v2:172.21.15.143:6832/2150557544,v1:172.21.15.143:6833/2150557544] conn(0x7fe258066820 0x7fe258068ce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:22:50.591 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:50.591+0000 7fe26a7fc640 1 -- 172.21.15.143:0/855470737 <== mon.0 v2:172.21.15.143:3300/0 5 ==== osd_map(71..71 src has 1..71) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fe27011d980 con 0x7fe2740b2160 2023-11-26T07:22:50.591 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:50.591+0000 7fe279193640 1 --2- 172.21.15.143:0/855470737 >> [v2:172.21.15.143:6832/2150557544,v1:172.21.15.143:6833/2150557544] conn(0x7fe258066820 0x7fe258068ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:22:50.592 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:50.591+0000 7fe279193640 1 --2- 172.21.15.143:0/855470737 >> [v2:172.21.15.143:6832/2150557544,v1:172.21.15.143:6833/2150557544] conn(0x7fe258066820 0x7fe258068ce0 secure :-1 s=READY pgs=467 cs=0 l=1 rev1=1 crypto rx=0x7fe2740c6970 tx=0x7fe264008040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:22:50.594 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:50.591+0000 7fe26a7fc640 1 -- 172.21.15.143:0/855470737 <== mon.0 v2:172.21.15.143:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172649 (secure 0 0 0) 0x7fe2700ea9d0 con 0x7fe2740b2160 2023-11-26T07:22:50.746 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:50.743+0000 7fe27bbf8640 1 -- 172.21.15.143:0/855470737 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fe2740cc700 con 0x7fe2740b2160 2023-11-26T07:22:50.747 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:50.743+0000 7fe26a7fc640 1 -- 172.21.15.143:0/855470737 <== mon.0 v2:172.21.15.143:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v71) v1 ==== 74+0+16604 (secure 0 0 0) 0x7fe2740cc700 con 0x7fe2740b2160 2023-11-26T07:22:50.747 INFO:teuthology.orchestra.run.smithi143.stdout: 2023-11-26T07:22:50.747 INFO:teuthology.orchestra.run.smithi143.stdout:{"epoch":71,"fsid":"f5033060-2469-4a07-95fe-2b76f5404bc1","created":"2023-11-26T07:17:28.613663+0000","modified":"2023-11-26T07:22:38.856077+0000","last_up_change":"2023-11-26T07:17:41.157813+0000","last_in_change":"2023-11-26T07:17:31.181658+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":11,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-11-26T07:17:41.758597+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":10,"pool_name":"cephfs_metadata","create_time":"2023-11-26T07:22:35.314819+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"70","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":2.5,"score_stable":2.5,"optimal_score":1,"raw_score_acting":2.5,"raw_score_stable":2.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":11,"pool_name":"cephfs_data","create_time":"2023-11-26T07:22:36.219536+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"71","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"ebbee2c6-023e-44b9-9f1d-6f7bfd309730","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":69,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6817","nonce":639807369},{"type":"v1","addr":"172.21.15.143:6819","nonce":639807369}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6821","nonce":639807369},{"type":"v1","addr":"172.21.15.143:6823","nonce":639807369}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6830","nonce":639807369},{"type":"v1","addr":"172.21.15.143:6831","nonce":639807369}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6825","nonce":639807369},{"type":"v1","addr":"172.21.15.143:6828","nonce":639807369}]},"public_addr":"172.21.15.143:6819/639807369","cluster_addr":"172.21.15.143:6823/639807369","heartbeat_back_addr":"172.21.15.143:6831/639807369","heartbeat_front_addr":"172.21.15.143:6828/639807369","state":["exists","up"]},{"osd":1,"uuid":"fec4b8be-7122-49e9-9a19-5850a0700001","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":69,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6816","nonce":1144942530},{"type":"v1","addr":"172.21.15.143:6818","nonce":1144942530}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6820","nonce":1144942530},{"type":"v1","addr":"172.21.15.143:6822","nonce":1144942530}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6827","nonce":1144942530},{"type":"v1","addr":"172.21.15.143:6829","nonce":1144942530}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6824","nonce":1144942530},{"type":"v1","addr":"172.21.15.143:6826","nonce":1144942530}]},"public_addr":"172.21.15.143:6818/1144942530","cluster_addr":"172.21.15.143:6822/1144942530","heartbeat_back_addr":"172.21.15.143:6829/1144942530","heartbeat_front_addr":"172.21.15.143:6826/1144942530","state":["exists","up"]},{"osd":2,"uuid":"2888af5b-f933-46ca-8f95-6dd494ff43f6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":69,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6808","nonce":3909095186},{"type":"v1","addr":"172.21.15.143:6809","nonce":3909095186}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6810","nonce":3909095186},{"type":"v1","addr":"172.21.15.143:6811","nonce":3909095186}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6814","nonce":3909095186},{"type":"v1","addr":"172.21.15.143:6815","nonce":3909095186}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6812","nonce":3909095186},{"type":"v1","addr":"172.21.15.143:6813","nonce":3909095186}]},"public_addr":"172.21.15.143:6809/3909095186","cluster_addr":"172.21.15.143:6811/3909095186","heartbeat_back_addr":"172.21.15.143:6815/3909095186","heartbeat_front_addr":"172.21.15.143:6813/3909095186","state":["exists","up"]},{"osd":3,"uuid":"d1f52435-7bc1-4081-9550-9cd1ba55f549","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":69,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6800","nonce":3854210991},{"type":"v1","addr":"172.21.15.143:6801","nonce":3854210991}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6802","nonce":3854210991},{"type":"v1","addr":"172.21.15.143:6803","nonce":3854210991}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6806","nonce":3854210991},{"type":"v1","addr":"172.21.15.143:6807","nonce":3854210991}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6804","nonce":3854210991},{"type":"v1","addr":"172.21.15.143:6805","nonce":3854210991}]},"public_addr":"172.21.15.143:6801/3854210991","cluster_addr":"172.21.15.143:6803/3854210991","heartbeat_back_addr":"172.21.15.143:6807/3854210991","heartbeat_front_addr":"172.21.15.143:6805/3854210991","state":["exists","up"]},{"osd":4,"uuid":"e2ae8b6e-6e25-49b2-bcbf-aacce68f06a3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":69,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6800","nonce":3924606902},{"type":"v1","addr":"172.21.15.188:6801","nonce":3924606902}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6802","nonce":3924606902},{"type":"v1","addr":"172.21.15.188:6803","nonce":3924606902}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6806","nonce":3924606902},{"type":"v1","addr":"172.21.15.188:6807","nonce":3924606902}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6804","nonce":3924606902},{"type":"v1","addr":"172.21.15.188:6805","nonce":3924606902}]},"public_addr":"172.21.15.188:6801/3924606902","cluster_addr":"172.21.15.188:6803/3924606902","heartbeat_back_addr":"172.21.15.188:6807/3924606902","heartbeat_front_addr":"172.21.15.188:6805/3924606902","state":["exists","up"]},{"osd":5,"uuid":"3cec8a32-7698-4c36-8e89-e4ba11934f33","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":69,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6813","nonce":186841544},{"type":"v1","addr":"172.21.15.188:6815","nonce":186841544}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6817","nonce":186841544},{"type":"v1","addr":"172.21.15.188:6818","nonce":186841544}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6822","nonce":186841544},{"type":"v1","addr":"172.21.15.188:6823","nonce":186841544}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6820","nonce":186841544},{"type":"v1","addr":"172.21.15.188:6821","nonce":186841544}]},"public_addr":"172.21.15.188:6815/186841544","cluster_addr":"172.21.15.188:6818/186841544","heartbeat_back_addr":"172.21.15.188:6823/186841544","heartbeat_front_addr":"172.21.15.188:6821/186841544","state":["exists","up"]},{"osd":6,"uuid":"58a65d5d-e090-473a-81b4-ad38e85029f9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":69,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6808","nonce":2153499405},{"type":"v1","addr":"172.21.15.188:6809","nonce":2153499405}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6810","nonce":2153499405},{"type":"v1","addr":"172.21.15.188:6811","nonce":2153499405}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6816","nonce":2153499405},{"type":"v1","addr":"172.21.15.188:6819","nonce":2153499405}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6812","nonce":2153499405},{"type":"v1","addr":"172.21.15.188:6814","nonce":2153499405}]},"public_addr":"172.21.15.188:6809/2153499405","cluster_addr":"172.21.15.188:6811/2153499405","heartbeat_back_addr":"172.21.15.188:6819/2153499405","heartbeat_front_addr":"172.21.15.188:6814/2153499405","state":["exists","up"]},{"osd":7,"uuid":"ea5b7f70-3554-4cf8-ac30-f7c59b80678d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":69,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6824","nonce":2722096729},{"type":"v1","addr":"172.21.15.188:6825","nonce":2722096729}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6826","nonce":2722096729},{"type":"v1","addr":"172.21.15.188:6827","nonce":2722096729}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6830","nonce":2722096729},{"type":"v1","addr":"172.21.15.188:6831","nonce":2722096729}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6828","nonce":2722096729},{"type":"v1","addr":"172.21.15.188:6829","nonce":2722096729}]},"public_addr":"172.21.15.188:6825/2722096729","cluster_addr":"172.21.15.188:6827/2722096729","heartbeat_back_addr":"172.21.15.188:6831/2722096729","heartbeat_front_addr":"172.21.15.188:6829/2722096729","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-26T07:17:36.995032+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-26T07:17:36.932266+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-26T07:17:36.829304+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-26T07:17:36.804297+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-26T07:17:36.667892+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-26T07:17:36.964868+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-26T07:17:36.827332+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-26T07:17:36.893179+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-11-26T07:22:50.749 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:50.747+0000 7fe27bbf8640 1 -- 172.21.15.143:0/855470737 >> [v2:172.21.15.143:6832/2150557544,v1:172.21.15.143:6833/2150557544] conn(0x7fe258066820 msgr2=0x7fe258068ce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:22:50.749 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:50.747+0000 7fe27bbf8640 1 --2- 172.21.15.143:0/855470737 >> [v2:172.21.15.143:6832/2150557544,v1:172.21.15.143:6833/2150557544] conn(0x7fe258066820 0x7fe258068ce0 secure :-1 s=READY pgs=467 cs=0 l=1 rev1=1 crypto rx=0x7fe2740c6970 tx=0x7fe264008040 comp rx=0 tx=0).stop 2023-11-26T07:22:50.749 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:50.747+0000 7fe27bbf8640 1 -- 172.21.15.143:0/855470737 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fe2740b2160 msgr2=0x7fe2740ccd30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:22:50.749 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:50.747+0000 7fe27bbf8640 1 --2- 172.21.15.143:0/855470737 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fe2740b2160 0x7fe2740ccd30 secure :-1 s=READY pgs=1004 cs=0 l=1 rev1=1 crypto rx=0x7fe27001a8f0 tx=0x7fe270018fe0 comp rx=0 tx=0).stop 2023-11-26T07:22:50.749 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:50.747+0000 7fe27bbf8640 1 -- 172.21.15.143:0/855470737 shutdown_connections 2023-11-26T07:22:50.750 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:50.747+0000 7fe27bbf8640 1 --2- 172.21.15.143:0/855470737 >> [v2:172.21.15.143:6832/2150557544,v1:172.21.15.143:6833/2150557544] conn(0x7fe258066820 0x7fe258068ce0 unknown :-1 s=CLOSED pgs=467 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:22:50.750 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:50.747+0000 7fe27bbf8640 1 --2- 172.21.15.143:0/855470737 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fe27415d6e0 0x7fe27415e460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:22:50.750 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:50.747+0000 7fe27bbf8640 1 --2- 172.21.15.143:0/855470737 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fe2740d19f0 0x7fe2740b0f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:22:50.750 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:50.747+0000 7fe27bbf8640 1 --2- 172.21.15.143:0/855470737 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fe2740b2160 0x7fe2740ccd30 unknown :-1 s=CLOSED pgs=1004 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:22:50.750 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:50.747+0000 7fe27bbf8640 1 -- 172.21.15.143:0/855470737 >> 172.21.15.143:0/855470737 conn(0x7fe27409c550 msgr2=0x7fe2740b2c70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:22:50.750 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:50.747+0000 7fe27bbf8640 1 -- 172.21.15.143:0/855470737 shutdown_connections 2023-11-26T07:22:50.750 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:50.747+0000 7fe27bbf8640 1 -- 172.21.15.143:0/855470737 wait complete. 2023-11-26T07:22:50.768 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-11-26T07:17:41.758597+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 10, 'pool_name': 'cephfs_metadata', 'create_time': '2023-11-26T07:22:35.314819+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '70', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 2.5, 'score_stable': 2.5, 'optimal_score': 1, 'raw_score_acting': 2.5, 'raw_score_stable': 2.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 11, 'pool_name': 'cephfs_data', 'create_time': '2023-11-26T07:22:36.219536+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '71', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.3799999952316284, 'score_stable': 1.3799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.3799999952316284, 'raw_score_stable': 1.3799999952316284, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2023-11-26T07:22:50.769 DEBUG:teuthology.orchestra.run.smithi143:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-11-26T07:22:50.910 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:50.907+0000 7eff95957640 1 Processor -- start 2023-11-26T07:22:50.910 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:50.907+0000 7eff95957640 1 -- start start 2023-11-26T07:22:50.911 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:50.907+0000 7eff95957640 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7eff900b14a0 0x7eff901681a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:22:50.911 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:50.907+0000 7eff95957640 1 -- --> v1:172.21.15.143:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7eff900d0bc0 con 0x7eff9015d580 2023-11-26T07:22:50.912 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:50.907+0000 7eff95957640 1 -- --> v1:172.21.15.188:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7eff900d0320 con 0x7eff900d17f0 2023-11-26T07:22:50.912 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:50.907+0000 7eff95957640 1 -- --> v2:172.21.15.143:3300/0 -- mon_getmap magic: 0 v1 -- 0x7eff900d0600 con 0x7eff900b14a0 2023-11-26T07:22:50.912 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:50.911+0000 7eff8f7fe640 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7eff900b14a0 0x7eff901681a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:22:50.912 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:50.911+0000 7eff8effd640 1 --1- >> v1:172.21.15.143:6789/0 conn(0x7eff9015d580 0x7eff900d3920 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.143:6789/0 says I am v1:172.21.15.143:48034/0 (socket says 172.21.15.143:48034) 2023-11-26T07:22:50.912 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:50.911+0000 7eff8effd640 1 -- 172.21.15.143:0/4211399694 learned_addr learned my addr 172.21.15.143:0/4211399694 (peer_addr_for_me v1:172.21.15.143:0/0) 2023-11-26T07:22:50.912 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:50.911+0000 7eff8f7fe640 1 -- 172.21.15.143:0/4211399694 >> v1:172.21.15.188:6789/0 conn(0x7eff900d17f0 legacy=0x7eff900d1c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-26T07:22:50.912 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:50.911+0000 7eff8f7fe640 1 -- 172.21.15.143:0/4211399694 >> v1:172.21.15.143:6789/0 conn(0x7eff9015d580 legacy=0x7eff900d3920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:22:50.912 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:50.911+0000 7eff8f7fe640 1 -- 172.21.15.143:0/4211399694 --> v2:172.21.15.143:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7eff900d08e0 con 0x7eff900b14a0 2023-11-26T07:22:50.913 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:50.911+0000 7eff8e7fc640 1 -- 172.21.15.143:0/4211399694 <== mon.0 v1:172.21.15.143:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 13233192 0 0) 0x7eff900d0bc0 con 0x7eff9015d580 2023-11-26T07:22:50.915 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:50.911+0000 7eff8f7fe640 1 --2- 172.21.15.143:0/4211399694 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7eff900b14a0 0x7eff901681a0 secure :-1 s=READY pgs=1006 cs=0 l=1 rev1=1 crypto rx=0x7eff88009d90 tx=0x7eff8801c8e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7e3e2dd2cf6e01bb server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:22:50.915 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:50.911+0000 7eff8e7fc640 1 -- 172.21.15.143:0/4211399694 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7eff880a5940 con 0x7eff900b14a0 2023-11-26T07:22:50.915 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:50.911+0000 7eff8e7fc640 1 -- 172.21.15.143:0/4211399694 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7eff880a5aa0 con 0x7eff900b14a0 2023-11-26T07:22:50.915 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:50.911+0000 7eff8e7fc640 1 -- 172.21.15.143:0/4211399694 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7eff880a5e30 con 0x7eff900b14a0 2023-11-26T07:22:50.916 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:50.915+0000 7eff95957640 1 -- 172.21.15.143:0/4211399694 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7eff900b14a0 msgr2=0x7eff901681a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:22:50.916 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:50.915+0000 7eff95957640 1 --2- 172.21.15.143:0/4211399694 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7eff900b14a0 0x7eff901681a0 secure :-1 s=READY pgs=1006 cs=0 l=1 rev1=1 crypto rx=0x7eff88009d90 tx=0x7eff8801c8e0 comp rx=0 tx=0).stop 2023-11-26T07:22:50.916 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:50.915+0000 7eff95957640 1 -- 172.21.15.143:0/4211399694 shutdown_connections 2023-11-26T07:22:50.916 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:50.915+0000 7eff95957640 1 --2- 172.21.15.143:0/4211399694 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7eff900b14a0 0x7eff901681a0 unknown :-1 s=CLOSED pgs=1006 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:22:50.916 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:50.915+0000 7eff95957640 1 -- 172.21.15.143:0/4211399694 >> 172.21.15.143:0/4211399694 conn(0x7eff9009c550 msgr2=0x7eff900d3210 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:22:50.916 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:50.915+0000 7eff95957640 1 -- 172.21.15.143:0/4211399694 shutdown_connections 2023-11-26T07:22:50.916 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:50.915+0000 7eff95957640 1 -- 172.21.15.143:0/4211399694 wait complete. 2023-11-26T07:22:50.917 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:50.915+0000 7eff95957640 1 Processor -- start 2023-11-26T07:22:50.917 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:50.915+0000 7eff95957640 1 -- start start 2023-11-26T07:22:50.919 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:50.915+0000 7eff95957640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7eff900b14a0 0x7eff900ccc60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:22:50.919 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:50.915+0000 7eff95957640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7eff900d17f0 0x7eff900c6b70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:22:50.919 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:50.915+0000 7eff95957640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7eff9015d580 0x7eff900c70b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:22:50.919 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:50.915+0000 7eff95957640 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x7eff901536c0 con 0x7eff900d17f0 2023-11-26T07:22:50.919 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:50.915+0000 7eff95957640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7eff90153f60 con 0x7eff9015d580 2023-11-26T07:22:50.920 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:50.915+0000 7eff95957640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7eff900b81f0 con 0x7eff900b14a0 2023-11-26T07:22:50.920 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:50.915+0000 7eff867fc640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7eff900d17f0 0x7eff900c6b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:22:50.920 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:50.915+0000 7eff867fc640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7eff900d17f0 0x7eff900c6b70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:50474/0 (socket says 172.21.15.143:50474) 2023-11-26T07:22:50.920 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:50.915+0000 7eff867fc640 1 -- 172.21.15.143:0/26914630 learned_addr learned my addr 172.21.15.143:0/26914630 (peer_addr_for_me v2:172.21.15.143:0/0) 2023-11-26T07:22:50.920 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:50.915+0000 7eff8effd640 1 --2- 172.21.15.143:0/26914630 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7eff900b14a0 0x7eff900ccc60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:22:50.920 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:50.915+0000 7eff8f7fe640 1 --2- 172.21.15.143:0/26914630 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7eff9015d580 0x7eff900c70b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:22:50.920 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:50.915+0000 7eff867fc640 1 -- 172.21.15.143:0/26914630 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7eff900b14a0 msgr2=0x7eff900ccc60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:22:50.920 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:50.915+0000 7eff867fc640 1 --2- 172.21.15.143:0/26914630 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7eff900b14a0 0x7eff900ccc60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:22:50.920 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:50.915+0000 7eff867fc640 1 -- 172.21.15.143:0/26914630 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7eff9015d580 msgr2=0x7eff900c70b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:22:50.920 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:50.915+0000 7eff867fc640 1 --2- 172.21.15.143:0/26914630 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7eff9015d580 0x7eff900c70b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:22:50.920 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:50.915+0000 7eff867fc640 1 -- 172.21.15.143:0/26914630 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7eff88009a70 con 0x7eff900d17f0 2023-11-26T07:22:50.920 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:50.919+0000 7eff867fc640 1 --2- 172.21.15.143:0/26914630 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7eff900d17f0 0x7eff900c6b70 secure :-1 s=READY pgs=1007 cs=0 l=1 rev1=1 crypto rx=0x7eff8000c450 tx=0x7eff8000c910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:22:50.920 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:50.919+0000 7eff8cff9640 1 -- 172.21.15.143:0/26914630 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7eff80012830 con 0x7eff900d17f0 2023-11-26T07:22:50.921 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:50.919+0000 7eff8cff9640 1 -- 172.21.15.143:0/26914630 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7eff800129b0 con 0x7eff900d17f0 2023-11-26T07:22:50.921 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:50.919+0000 7eff8cff9640 1 -- 172.21.15.143:0/26914630 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7eff80012d60 con 0x7eff900d17f0 2023-11-26T07:22:50.921 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:50.919+0000 7eff95957640 1 -- 172.21.15.143:0/26914630 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7eff90153910 con 0x7eff900d17f0 2023-11-26T07:22:50.921 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:50.919+0000 7eff95957640 1 -- 172.21.15.143:0/26914630 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7eff900cd1a0 con 0x7eff900d17f0 2023-11-26T07:22:50.922 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:50.919+0000 7eff95957640 1 -- 172.21.15.143:0/26914630 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7eff900cff50 con 0x7eff900d17f0 2023-11-26T07:22:50.922 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:50.919+0000 7eff8cff9640 1 -- 172.21.15.143:0/26914630 <== mon.0 v2:172.21.15.143:3300/0 4 ==== mgrmap(e 11) v1 ==== 86158+0+0 (secure 0 0 0) 0x7eff8001c460 con 0x7eff900d17f0 2023-11-26T07:22:50.922 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:50.919+0000 7eff8cff9640 1 --2- 172.21.15.143:0/26914630 >> [v2:172.21.15.143:6832/2150557544,v1:172.21.15.143:6833/2150557544] conn(0x7eff700667d0 0x7eff70068c90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:22:50.922 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:50.919+0000 7eff8cff9640 1 -- 172.21.15.143:0/26914630 <== mon.0 v2:172.21.15.143:3300/0 5 ==== osd_map(71..71 src has 1..71) v4 ==== 5709+0+0 (secure 0 0 0) 0x7eff800841e0 con 0x7eff900d17f0 2023-11-26T07:22:50.922 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:50.919+0000 7eff8effd640 1 --2- 172.21.15.143:0/26914630 >> [v2:172.21.15.143:6832/2150557544,v1:172.21.15.143:6833/2150557544] conn(0x7eff700667d0 0x7eff70068c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:22:50.923 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:50.919+0000 7eff8effd640 1 --2- 172.21.15.143:0/26914630 >> [v2:172.21.15.143:6832/2150557544,v1:172.21.15.143:6833/2150557544] conn(0x7eff700667d0 0x7eff70068c90 secure :-1 s=READY pgs=468 cs=0 l=1 rev1=1 crypto rx=0x7eff78000f70 tx=0x7eff78007b80 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:22:50.926 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:50.923+0000 7eff8cff9640 1 -- 172.21.15.143:0/26914630 <== mon.0 v2:172.21.15.143:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172649 (secure 0 0 0) 0x7eff800512a0 con 0x7eff900d17f0 2023-11-26T07:22:51.083 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:51.083+0000 7eff95957640 1 -- 172.21.15.143:0/26914630 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7eff901540d0 con 0x7eff900d17f0 2023-11-26T07:22:51.544 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:51.083+0000 7eff8cff9640 1 -- 172.21.15.143:0/26914630 <== mon.0 v2:172.21.15.143:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v71) v1 ==== 93+0+10 (secure 0 0 0) 0x7eff800548c0 con 0x7eff900d17f0 2023-11-26T07:22:51.545 INFO:teuthology.orchestra.run.smithi143.stdout:pg_num: 1 2023-11-26T07:22:51.546 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:51.083+0000 7eff95957640 1 -- 172.21.15.143:0/26914630 >> [v2:172.21.15.143:6832/2150557544,v1:172.21.15.143:6833/2150557544] conn(0x7eff700667d0 msgr2=0x7eff70068c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:22:51.546 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:51.083+0000 7eff95957640 1 --2- 172.21.15.143:0/26914630 >> [v2:172.21.15.143:6832/2150557544,v1:172.21.15.143:6833/2150557544] conn(0x7eff700667d0 0x7eff70068c90 secure :-1 s=READY pgs=468 cs=0 l=1 rev1=1 crypto rx=0x7eff78000f70 tx=0x7eff78007b80 comp rx=0 tx=0).stop 2023-11-26T07:22:51.546 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:51.083+0000 7eff95957640 1 -- 172.21.15.143:0/26914630 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7eff900d17f0 msgr2=0x7eff900c6b70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:22:51.546 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:51.083+0000 7eff95957640 1 --2- 172.21.15.143:0/26914630 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7eff900d17f0 0x7eff900c6b70 secure :-1 s=READY pgs=1007 cs=0 l=1 rev1=1 crypto rx=0x7eff8000c450 tx=0x7eff8000c910 comp rx=0 tx=0).stop 2023-11-26T07:22:51.546 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:51.083+0000 7eff95957640 1 -- 172.21.15.143:0/26914630 shutdown_connections 2023-11-26T07:22:51.546 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:51.083+0000 7eff95957640 1 --2- 172.21.15.143:0/26914630 >> [v2:172.21.15.143:6832/2150557544,v1:172.21.15.143:6833/2150557544] conn(0x7eff700667d0 0x7eff70068c90 unknown :-1 s=CLOSED pgs=468 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:22:51.546 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:51.083+0000 7eff95957640 1 --2- 172.21.15.143:0/26914630 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7eff9015d580 0x7eff900c70b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:22:51.547 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:51.083+0000 7eff95957640 1 --2- 172.21.15.143:0/26914630 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7eff900d17f0 0x7eff900c6b70 unknown :-1 s=CLOSED pgs=1007 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:22:51.547 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:51.083+0000 7eff95957640 1 --2- 172.21.15.143:0/26914630 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7eff900b14a0 0x7eff900ccc60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:22:51.547 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:51.083+0000 7eff95957640 1 -- 172.21.15.143:0/26914630 >> 172.21.15.143:0/26914630 conn(0x7eff9009c550 msgr2=0x7eff9006cac0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:22:51.547 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:51.083+0000 7eff95957640 1 -- 172.21.15.143:0/26914630 shutdown_connections 2023-11-26T07:22:51.547 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:51.083+0000 7eff95957640 1 -- 172.21.15.143:0/26914630 wait complete. 2023-11-26T07:22:51.547 DEBUG:teuthology.orchestra.run.smithi143:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-11-26T07:22:51.690 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:51.687+0000 7fbc56e79640 1 Processor -- start 2023-11-26T07:22:51.691 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:51.687+0000 7fbc56e79640 1 -- start start 2023-11-26T07:22:51.692 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:51.691+0000 7fbc56e79640 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7fbc5013e290 0x7fbc500cf280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:22:51.692 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:51.691+0000 7fbc56e79640 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7fbc5015f6b0 0x7fbc500cd7b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:22:51.692 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:51.691+0000 7fbc56e79640 1 -- --> v1:172.21.15.188:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbc5015fb20 con 0x7fbc500cf7c0 2023-11-26T07:22:51.692 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:51.691+0000 7fbc56e79640 1 -- --> v2:172.21.15.143:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fbc500cfc40 con 0x7fbc5015f6b0 2023-11-26T07:22:51.692 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:51.691+0000 7fbc56e79640 1 -- --> v2:172.21.15.188:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fbc5006c520 con 0x7fbc5013e290 2023-11-26T07:22:51.692 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:51.691+0000 7fbc55416640 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7fbc5015f6b0 0x7fbc500cd7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:22:51.692 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:51.691+0000 7fbc55416640 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7fbc5015f6b0 0x7fbc500cd7b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:50488/0 (socket says 172.21.15.143:50488) 2023-11-26T07:22:51.692 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:51.691+0000 7fbc55416640 1 -- 172.21.15.143:0/3569547721 learned_addr learned my addr 172.21.15.143:0/3569547721 (peer_addr_for_me v2:172.21.15.143:0/0) 2023-11-26T07:22:51.693 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:51.691+0000 7fbc54c15640 1 --2- 172.21.15.143:0/3569547721 >> v2:172.21.15.188:3300/0 conn(0x7fbc5013e290 0x7fbc500cf280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:22:51.693 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:51.691+0000 7fbc55416640 1 -- 172.21.15.143:0/3569547721 >> v2:172.21.15.188:3300/0 conn(0x7fbc5013e290 msgr2=0x7fbc500cf280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-26T07:22:51.693 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:51.691+0000 7fbc55416640 1 --2- 172.21.15.143:0/3569547721 >> v2:172.21.15.188:3300/0 conn(0x7fbc5013e290 0x7fbc500cf280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:22:51.693 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:51.691+0000 7fbc55416640 1 -- 172.21.15.143:0/3569547721 >> v1:172.21.15.188:6789/0 conn(0x7fbc500cf7c0 legacy=0x7fbc5015efa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:22:51.693 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:51.691+0000 7fbc55416640 1 -- 172.21.15.143:0/3569547721 --> v2:172.21.15.143:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbc5006c800 con 0x7fbc5015f6b0 2023-11-26T07:22:51.695 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:51.695+0000 7fbc55416640 1 --2- 172.21.15.143:0/3569547721 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fbc5015f6b0 0x7fbc500cd7b0 secure :-1 s=READY pgs=1008 cs=0 l=1 rev1=1 crypto rx=0x7fbc500725a0 tx=0x7fbc4801c9a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=65db0d21090b5f43 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:22:51.696 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:51.695+0000 7fbc4f7fe640 1 -- 172.21.15.143:0/3569547721 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbc480a5860 con 0x7fbc5015f6b0 2023-11-26T07:22:51.696 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:51.695+0000 7fbc4f7fe640 1 -- 172.21.15.143:0/3569547721 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbc480a59c0 con 0x7fbc5015f6b0 2023-11-26T07:22:51.696 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:51.695+0000 7fbc4f7fe640 1 -- 172.21.15.143:0/3569547721 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbc480a5d50 con 0x7fbc5015f6b0 2023-11-26T07:22:51.696 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:51.695+0000 7fbc56e79640 1 -- 172.21.15.143:0/3569547721 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fbc5015f6b0 msgr2=0x7fbc500cd7b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:22:51.696 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:51.695+0000 7fbc56e79640 1 --2- 172.21.15.143:0/3569547721 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fbc5015f6b0 0x7fbc500cd7b0 secure :-1 s=READY pgs=1008 cs=0 l=1 rev1=1 crypto rx=0x7fbc500725a0 tx=0x7fbc4801c9a0 comp rx=0 tx=0).stop 2023-11-26T07:22:51.697 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:51.695+0000 7fbc56e79640 1 -- 172.21.15.143:0/3569547721 shutdown_connections 2023-11-26T07:22:51.697 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:51.695+0000 7fbc56e79640 1 --2- 172.21.15.143:0/3569547721 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fbc5015f6b0 0x7fbc500cd7b0 unknown :-1 s=CLOSED pgs=1008 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:22:51.697 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:51.695+0000 7fbc56e79640 1 --2- 172.21.15.143:0/3569547721 >> v2:172.21.15.188:3300/0 conn(0x7fbc5013e290 0x7fbc500cf280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:22:51.697 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:51.695+0000 7fbc56e79640 1 -- 172.21.15.143:0/3569547721 >> 172.21.15.143:0/3569547721 conn(0x7fbc5006a890 msgr2=0x7fbc5006aca0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:22:51.697 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:51.695+0000 7fbc56e79640 1 -- 172.21.15.143:0/3569547721 shutdown_connections 2023-11-26T07:22:51.697 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:51.695+0000 7fbc56e79640 1 -- 172.21.15.143:0/3569547721 wait complete. 2023-11-26T07:22:51.698 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:51.695+0000 7fbc56e79640 1 Processor -- start 2023-11-26T07:22:51.698 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:51.695+0000 7fbc56e79640 1 -- start start 2023-11-26T07:22:51.700 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:51.699+0000 7fbc56e79640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fbc500cf7c0 0x7fbc5015df00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:22:51.700 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:51.699+0000 7fbc56e79640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fbc5013e290 0x7fbc5015c750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:22:51.700 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:51.699+0000 7fbc56e79640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fbc5015f6b0 0x7fbc500b3bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:22:51.700 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:51.699+0000 7fbc56e79640 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbc5015d180 con 0x7fbc5015f6b0 2023-11-26T07:22:51.700 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:51.699+0000 7fbc56e79640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbc50157490 con 0x7fbc500cf7c0 2023-11-26T07:22:51.700 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:51.699+0000 7fbc56e79640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbc5015d420 con 0x7fbc5013e290 2023-11-26T07:22:51.700 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:51.699+0000 7fbc55416640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fbc5015f6b0 0x7fbc500b3bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:22:51.700 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:51.699+0000 7fbc55416640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fbc5015f6b0 0x7fbc500b3bf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:50498/0 (socket says 172.21.15.143:50498) 2023-11-26T07:22:51.700 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:51.699+0000 7fbc55416640 1 -- 172.21.15.143:0/3280312026 learned_addr learned my addr 172.21.15.143:0/3280312026 (peer_addr_for_me v2:172.21.15.143:0/0) 2023-11-26T07:22:51.700 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:51.699+0000 7fbc54c15640 1 --2- 172.21.15.143:0/3280312026 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fbc500cf7c0 0x7fbc5015df00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:22:51.701 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:51.699+0000 7fbc4ffff640 1 --2- 172.21.15.143:0/3280312026 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fbc5013e290 0x7fbc5015c750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:22:51.701 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:51.699+0000 7fbc55416640 1 -- 172.21.15.143:0/3280312026 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fbc5013e290 msgr2=0x7fbc5015c750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:22:51.701 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:51.699+0000 7fbc55416640 1 --2- 172.21.15.143:0/3280312026 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fbc5013e290 0x7fbc5015c750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:22:51.701 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:51.699+0000 7fbc55416640 1 -- 172.21.15.143:0/3280312026 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fbc500cf7c0 msgr2=0x7fbc5015df00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:22:51.701 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:51.699+0000 7fbc55416640 1 --2- 172.21.15.143:0/3280312026 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fbc500cf7c0 0x7fbc5015df00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:22:51.701 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:51.699+0000 7fbc55416640 1 -- 172.21.15.143:0/3280312026 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbc48009b30 con 0x7fbc5015f6b0 2023-11-26T07:22:51.702 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:51.699+0000 7fbc55416640 1 --2- 172.21.15.143:0/3280312026 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fbc5015f6b0 0x7fbc500b3bf0 secure :-1 s=READY pgs=1009 cs=0 l=1 rev1=1 crypto rx=0x7fbc480af040 tx=0x7fbc48019740 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:22:51.702 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:51.699+0000 7fbc4d7fa640 1 -- 172.21.15.143:0/3280312026 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbc480b8670 con 0x7fbc5015f6b0 2023-11-26T07:22:51.702 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:51.699+0000 7fbc4d7fa640 1 -- 172.21.15.143:0/3280312026 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbc480b8810 con 0x7fbc5015f6b0 2023-11-26T07:22:51.702 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:51.699+0000 7fbc56e79640 1 -- 172.21.15.143:0/3280312026 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbc5015d700 con 0x7fbc5015f6b0 2023-11-26T07:22:51.702 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:51.699+0000 7fbc4d7fa640 1 -- 172.21.15.143:0/3280312026 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbc480b8bc0 con 0x7fbc5015f6b0 2023-11-26T07:22:51.702 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:51.699+0000 7fbc56e79640 1 -- 172.21.15.143:0/3280312026 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbc5015ce50 con 0x7fbc5015f6b0 2023-11-26T07:22:51.702 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:51.699+0000 7fbc56e79640 1 -- 172.21.15.143:0/3280312026 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbc50067d30 con 0x7fbc5015f6b0 2023-11-26T07:22:51.703 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:51.699+0000 7fbc4d7fa640 1 -- 172.21.15.143:0/3280312026 <== mon.0 v2:172.21.15.143:3300/0 4 ==== mgrmap(e 11) v1 ==== 86158+0+0 (secure 0 0 0) 0x7fbc480b8d20 con 0x7fbc5015f6b0 2023-11-26T07:22:51.703 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:51.703+0000 7fbc4d7fa640 1 --2- 172.21.15.143:0/3280312026 >> [v2:172.21.15.143:6832/2150557544,v1:172.21.15.143:6833/2150557544] conn(0x7fbc34066800 0x7fbc34068cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:22:51.703 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:51.703+0000 7fbc4d7fa640 1 -- 172.21.15.143:0/3280312026 <== mon.0 v2:172.21.15.143:3300/0 5 ==== osd_map(71..71 src has 1..71) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fbc4811f7f0 con 0x7fbc5015f6b0 2023-11-26T07:22:51.704 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:51.703+0000 7fbc54c15640 1 --2- 172.21.15.143:0/3280312026 >> [v2:172.21.15.143:6832/2150557544,v1:172.21.15.143:6833/2150557544] conn(0x7fbc34066800 0x7fbc34068cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:22:51.704 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:51.703+0000 7fbc54c15640 1 --2- 172.21.15.143:0/3280312026 >> [v2:172.21.15.143:6832/2150557544,v1:172.21.15.143:6833/2150557544] conn(0x7fbc34066800 0x7fbc34068cc0 secure :-1 s=READY pgs=469 cs=0 l=1 rev1=1 crypto rx=0x7fbc44009b60 tx=0x7fbc44009190 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:22:51.707 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:51.707+0000 7fbc4d7fa640 1 -- 172.21.15.143:0/3280312026 <== mon.0 v2:172.21.15.143:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172649 (secure 0 0 0) 0x7fbc480ec9a0 con 0x7fbc5015f6b0 2023-11-26T07:22:51.860 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:51.859+0000 7fbc56e79640 1 -- 172.21.15.143:0/3280312026 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7fbc50067680 con 0x7fbc5015f6b0 2023-11-26T07:22:51.860 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:51.859+0000 7fbc4d7fa640 1 -- 172.21.15.143:0/3280312026 <== mon.0 v2:172.21.15.143:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v71) v1 ==== 104+0+11 (secure 0 0 0) 0x7fbc480effc0 con 0x7fbc5015f6b0 2023-11-26T07:22:51.861 INFO:teuthology.orchestra.run.smithi143.stdout:pg_num: 32 2023-11-26T07:22:51.863 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:51.859+0000 7fbc56e79640 1 -- 172.21.15.143:0/3280312026 >> [v2:172.21.15.143:6832/2150557544,v1:172.21.15.143:6833/2150557544] conn(0x7fbc34066800 msgr2=0x7fbc34068cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:22:51.863 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:51.859+0000 7fbc56e79640 1 --2- 172.21.15.143:0/3280312026 >> [v2:172.21.15.143:6832/2150557544,v1:172.21.15.143:6833/2150557544] conn(0x7fbc34066800 0x7fbc34068cc0 secure :-1 s=READY pgs=469 cs=0 l=1 rev1=1 crypto rx=0x7fbc44009b60 tx=0x7fbc44009190 comp rx=0 tx=0).stop 2023-11-26T07:22:51.863 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:51.859+0000 7fbc56e79640 1 -- 172.21.15.143:0/3280312026 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fbc5015f6b0 msgr2=0x7fbc500b3bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:22:51.863 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:51.859+0000 7fbc56e79640 1 --2- 172.21.15.143:0/3280312026 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fbc5015f6b0 0x7fbc500b3bf0 secure :-1 s=READY pgs=1009 cs=0 l=1 rev1=1 crypto rx=0x7fbc480af040 tx=0x7fbc48019740 comp rx=0 tx=0).stop 2023-11-26T07:22:51.863 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:51.859+0000 7fbc56e79640 1 -- 172.21.15.143:0/3280312026 shutdown_connections 2023-11-26T07:22:51.863 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:51.859+0000 7fbc56e79640 1 --2- 172.21.15.143:0/3280312026 >> [v2:172.21.15.143:6832/2150557544,v1:172.21.15.143:6833/2150557544] conn(0x7fbc34066800 0x7fbc34068cc0 unknown :-1 s=CLOSED pgs=469 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:22:51.863 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:51.859+0000 7fbc56e79640 1 --2- 172.21.15.143:0/3280312026 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fbc5015f6b0 0x7fbc500b3bf0 unknown :-1 s=CLOSED pgs=1009 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:22:51.863 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:51.859+0000 7fbc56e79640 1 --2- 172.21.15.143:0/3280312026 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fbc5013e290 0x7fbc5015c750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:22:51.863 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:51.859+0000 7fbc56e79640 1 --2- 172.21.15.143:0/3280312026 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fbc500cf7c0 0x7fbc5015df00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:22:51.863 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:51.859+0000 7fbc56e79640 1 -- 172.21.15.143:0/3280312026 >> 172.21.15.143:0/3280312026 conn(0x7fbc5006a890 msgr2=0x7fbc5013c720 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:22:51.863 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:51.859+0000 7fbc56e79640 1 -- 172.21.15.143:0/3280312026 shutdown_connections 2023-11-26T07:22:51.863 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:51.859+0000 7fbc56e79640 1 -- 172.21.15.143:0/3280312026 wait complete. 2023-11-26T07:22:51.881 DEBUG:teuthology.orchestra.run.smithi143:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-11-26T07:22:52.024 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:52.023+0000 7f958689c640 1 Processor -- start 2023-11-26T07:22:52.025 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:52.023+0000 7f958689c640 1 -- start start 2023-11-26T07:22:52.026 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:52.023+0000 7f958689c640 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7f95800cf710 0x7f95800cd8c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:22:52.026 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:52.023+0000 7f958689c640 1 -- --> v1:172.21.15.143:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f95800cc730 con 0x7f9580143ac0 2023-11-26T07:22:52.026 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:52.023+0000 7f958689c640 1 -- --> v1:172.21.15.188:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9580070ee0 con 0x7f95800cf280 2023-11-26T07:22:52.026 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:52.023+0000 7f958689c640 1 -- --> v2:172.21.15.143:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9580143ea0 con 0x7f95800cf710 2023-11-26T07:22:52.026 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:52.023+0000 7f957ffff640 1 --1- >> v1:172.21.15.143:6789/0 conn(0x7f9580143ac0 0x7f9580070370 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.143:6789/0 says I am v1:172.21.15.143:48036/0 (socket says 172.21.15.143:48036) 2023-11-26T07:22:52.026 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:52.023+0000 7f957ffff640 1 -- 172.21.15.143:0/2017010550 learned_addr learned my addr 172.21.15.143:0/2017010550 (peer_addr_for_me v1:172.21.15.143:0/0) 2023-11-26T07:22:52.026 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:52.023+0000 7f9584e39640 1 --2- 172.21.15.143:0/2017010550 >> v2:172.21.15.143:3300/0 conn(0x7f95800cf710 0x7f95800cd8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:22:52.027 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:52.023+0000 7f957effd640 1 -- 172.21.15.143:0/2017010550 <== mon.0 v1:172.21.15.143:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2231287671 0 0) 0x7f95800cc730 con 0x7f9580143ac0 2023-11-26T07:22:52.027 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:52.023+0000 7f957effd640 1 -- 172.21.15.143:0/2017010550 --> v1:172.21.15.143:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f9568003530 con 0x7f9580143ac0 2023-11-26T07:22:52.027 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:52.023+0000 7f9584e39640 1 -- 172.21.15.143:0/2017010550 >> v1:172.21.15.188:6789/0 conn(0x7f95800cf280 legacy=0x7f95800cd1b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:22:52.027 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:52.023+0000 7f9584e39640 1 -- 172.21.15.143:0/2017010550 >> v1:172.21.15.143:6789/0 conn(0x7f9580143ac0 legacy=0x7f9580070370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:22:52.027 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:52.023+0000 7f9584e39640 1 -- 172.21.15.143:0/2017010550 --> v2:172.21.15.143:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f95800711c0 con 0x7f95800cf710 2023-11-26T07:22:52.030 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:52.027+0000 7f9584e39640 1 --2- 172.21.15.143:0/2017010550 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f95800cf710 0x7f95800cd8c0 secure :-1 s=READY pgs=1011 cs=0 l=1 rev1=1 crypto rx=0x7f9578009d90 tx=0x7f957801c950 comp rx=0 tx=0).ready entity=mon.0 client_cookie=60660b441544b8c8 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:22:52.030 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:52.027+0000 7f957effd640 1 -- 172.21.15.143:0/2017010550 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f95780a5a10 con 0x7f95800cf710 2023-11-26T07:22:52.030 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:52.027+0000 7f957effd640 1 -- 172.21.15.143:0/2017010550 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f95780a5b70 con 0x7f95800cf710 2023-11-26T07:22:52.030 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:52.027+0000 7f957effd640 1 -- 172.21.15.143:0/2017010550 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f95780ad440 con 0x7f95800cf710 2023-11-26T07:22:52.031 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:52.027+0000 7f958689c640 1 -- 172.21.15.143:0/2017010550 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f95800cf710 msgr2=0x7f95800cd8c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:22:52.031 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:52.027+0000 7f958689c640 1 --2- 172.21.15.143:0/2017010550 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f95800cf710 0x7f95800cd8c0 secure :-1 s=READY pgs=1011 cs=0 l=1 rev1=1 crypto rx=0x7f9578009d90 tx=0x7f957801c950 comp rx=0 tx=0).stop 2023-11-26T07:22:52.031 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:52.027+0000 7f958689c640 1 -- 172.21.15.143:0/2017010550 shutdown_connections 2023-11-26T07:22:52.031 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:52.027+0000 7f958689c640 1 --2- 172.21.15.143:0/2017010550 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f95800cf710 0x7f95800cd8c0 unknown :-1 s=CLOSED pgs=1011 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:22:52.031 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:52.027+0000 7f958689c640 1 -- 172.21.15.143:0/2017010550 >> 172.21.15.143:0/2017010550 conn(0x7f958006a890 msgr2=0x7f958006aca0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:22:52.031 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:52.027+0000 7f958689c640 1 -- 172.21.15.143:0/2017010550 shutdown_connections 2023-11-26T07:22:52.031 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:52.027+0000 7f958689c640 1 -- 172.21.15.143:0/2017010550 wait complete. 2023-11-26T07:22:52.032 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:52.031+0000 7f958689c640 1 Processor -- start 2023-11-26T07:22:52.032 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:52.031+0000 7f958689c640 1 -- start start 2023-11-26T07:22:52.034 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:52.031+0000 7f958689c640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f95800cf280 0x7f9580159c40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:22:52.035 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:52.031+0000 7f958689c640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f95800cf710 0x7f958015a660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:22:52.035 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:52.031+0000 7f958689c640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f9580143ac0 0x7f958015aba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:22:52.035 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:52.031+0000 7f958689c640 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f958015a180 con 0x7f95800cf710 2023-11-26T07:22:52.035 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:52.031+0000 7f958689c640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f95801305a0 con 0x7f95800cf280 2023-11-26T07:22:52.035 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:52.031+0000 7f958689c640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9580130880 con 0x7f9580143ac0 2023-11-26T07:22:52.035 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:52.031+0000 7f957f7fe640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f95800cf710 0x7f958015a660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:22:52.035 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:52.031+0000 7f957f7fe640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f95800cf710 0x7f958015a660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:50512/0 (socket says 172.21.15.143:50512) 2023-11-26T07:22:52.035 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:52.031+0000 7f957f7fe640 1 -- 172.21.15.143:0/460192694 learned_addr learned my addr 172.21.15.143:0/460192694 (peer_addr_for_me v2:172.21.15.143:0/0) 2023-11-26T07:22:52.035 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:52.031+0000 7f957ffff640 1 --2- 172.21.15.143:0/460192694 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f95800cf280 0x7f9580159c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:22:52.035 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:52.031+0000 7f9584e39640 1 --2- 172.21.15.143:0/460192694 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f9580143ac0 0x7f958015aba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:22:52.035 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:52.031+0000 7f957f7fe640 1 -- 172.21.15.143:0/460192694 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f9580143ac0 msgr2=0x7f958015aba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:22:52.035 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:52.031+0000 7f957f7fe640 1 --2- 172.21.15.143:0/460192694 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f9580143ac0 0x7f958015aba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:22:52.036 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:52.031+0000 7f957f7fe640 1 -- 172.21.15.143:0/460192694 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f95800cf280 msgr2=0x7f9580159c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:22:52.036 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:52.031+0000 7f957f7fe640 1 --2- 172.21.15.143:0/460192694 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f95800cf280 0x7f9580159c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:22:52.036 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:52.031+0000 7f957f7fe640 1 -- 172.21.15.143:0/460192694 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9578009a70 con 0x7f95800cf710 2023-11-26T07:22:52.036 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:52.031+0000 7f957f7fe640 1 --2- 172.21.15.143:0/460192694 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f95800cf710 0x7f958015a660 secure :-1 s=READY pgs=1012 cs=0 l=1 rev1=1 crypto rx=0x7f957400c3c0 tx=0x7f957400c880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:22:52.036 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:52.031+0000 7f957cff9640 1 -- 172.21.15.143:0/460192694 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f95740052d0 con 0x7f95800cf710 2023-11-26T07:22:52.036 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:52.031+0000 7f957cff9640 1 -- 172.21.15.143:0/460192694 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9574005430 con 0x7f95800cf710 2023-11-26T07:22:52.036 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:52.031+0000 7f957cff9640 1 -- 172.21.15.143:0/460192694 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9574003840 con 0x7f95800cf710 2023-11-26T07:22:52.036 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:52.031+0000 7f958689c640 1 -- 172.21.15.143:0/460192694 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9580130b60 con 0x7f95800cf710 2023-11-26T07:22:52.036 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:52.031+0000 7f958689c640 1 -- 172.21.15.143:0/460192694 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9580131120 con 0x7f95800cf710 2023-11-26T07:22:52.037 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:52.035+0000 7f958689c640 1 -- 172.21.15.143:0/460192694 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9580131340 con 0x7f95800cf710 2023-11-26T07:22:52.037 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:52.035+0000 7f957cff9640 1 -- 172.21.15.143:0/460192694 <== mon.0 v2:172.21.15.143:3300/0 4 ==== mgrmap(e 11) v1 ==== 86158+0+0 (secure 0 0 0) 0x7f95740039a0 con 0x7f95800cf710 2023-11-26T07:22:52.037 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:52.035+0000 7f957cff9640 1 --2- 172.21.15.143:0/460192694 >> [v2:172.21.15.143:6832/2150557544,v1:172.21.15.143:6833/2150557544] conn(0x7f956c0667d0 0x7f956c068c90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:22:52.037 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:52.035+0000 7f957cff9640 1 -- 172.21.15.143:0/460192694 <== mon.0 v2:172.21.15.143:3300/0 5 ==== osd_map(71..71 src has 1..71) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f95740838d0 con 0x7f95800cf710 2023-11-26T07:22:52.038 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:52.035+0000 7f957ffff640 1 --2- 172.21.15.143:0/460192694 >> [v2:172.21.15.143:6832/2150557544,v1:172.21.15.143:6833/2150557544] conn(0x7f956c0667d0 0x7f956c068c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:22:52.038 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:52.035+0000 7f957ffff640 1 --2- 172.21.15.143:0/460192694 >> [v2:172.21.15.143:6832/2150557544,v1:172.21.15.143:6833/2150557544] conn(0x7f956c0667d0 0x7f956c068c90 secure :-1 s=READY pgs=470 cs=0 l=1 rev1=1 crypto rx=0x7f95700011f0 tx=0x7f9570007d40 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:22:52.041 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:52.039+0000 7f957cff9640 1 -- 172.21.15.143:0/460192694 <== mon.0 v2:172.21.15.143:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172649 (secure 0 0 0) 0x7f95740509b0 con 0x7f95800cf710 2023-11-26T07:22:52.189 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:52.187+0000 7f958689c640 1 -- 172.21.15.143:0/460192694 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f9580067680 con 0x7f95800cf710 2023-11-26T07:22:52.190 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:52.187+0000 7f957cff9640 1 -- 172.21.15.143:0/460192694 <== mon.0 v2:172.21.15.143:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v71) v1 ==== 100+0+11 (secure 0 0 0) 0x7f95740170a0 con 0x7f95800cf710 2023-11-26T07:22:52.190 INFO:teuthology.orchestra.run.smithi143.stdout:pg_num: 64 2023-11-26T07:22:52.192 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:52.191+0000 7f958689c640 1 -- 172.21.15.143:0/460192694 >> [v2:172.21.15.143:6832/2150557544,v1:172.21.15.143:6833/2150557544] conn(0x7f956c0667d0 msgr2=0x7f956c068c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:22:52.192 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:52.191+0000 7f958689c640 1 --2- 172.21.15.143:0/460192694 >> [v2:172.21.15.143:6832/2150557544,v1:172.21.15.143:6833/2150557544] conn(0x7f956c0667d0 0x7f956c068c90 secure :-1 s=READY pgs=470 cs=0 l=1 rev1=1 crypto rx=0x7f95700011f0 tx=0x7f9570007d40 comp rx=0 tx=0).stop 2023-11-26T07:22:52.192 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:52.191+0000 7f958689c640 1 -- 172.21.15.143:0/460192694 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f95800cf710 msgr2=0x7f958015a660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:22:52.192 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:52.191+0000 7f958689c640 1 --2- 172.21.15.143:0/460192694 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f95800cf710 0x7f958015a660 secure :-1 s=READY pgs=1012 cs=0 l=1 rev1=1 crypto rx=0x7f957400c3c0 tx=0x7f957400c880 comp rx=0 tx=0).stop 2023-11-26T07:22:52.192 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:52.191+0000 7f958689c640 1 -- 172.21.15.143:0/460192694 shutdown_connections 2023-11-26T07:22:52.192 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:52.191+0000 7f958689c640 1 --2- 172.21.15.143:0/460192694 >> [v2:172.21.15.143:6832/2150557544,v1:172.21.15.143:6833/2150557544] conn(0x7f956c0667d0 0x7f956c068c90 unknown :-1 s=CLOSED pgs=470 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:22:52.192 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:52.191+0000 7f958689c640 1 --2- 172.21.15.143:0/460192694 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f9580143ac0 0x7f958015aba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:22:52.192 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:52.191+0000 7f958689c640 1 --2- 172.21.15.143:0/460192694 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f95800cf710 0x7f958015a660 unknown :-1 s=CLOSED pgs=1012 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:22:52.192 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:52.191+0000 7f958689c640 1 --2- 172.21.15.143:0/460192694 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f95800cf280 0x7f9580159c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:22:52.192 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:52.191+0000 7f958689c640 1 -- 172.21.15.143:0/460192694 >> 172.21.15.143:0/460192694 conn(0x7f958006a890 msgr2=0x7f958006d030 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:22:52.192 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:52.191+0000 7f958689c640 1 -- 172.21.15.143:0/460192694 shutdown_connections 2023-11-26T07:22:52.193 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:52.191+0000 7f958689c640 1 -- 172.21.15.143:0/460192694 wait complete. 2023-11-26T07:22:52.211 DEBUG:teuthology.orchestra.run.smithi143:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2023-11-26T07:22:52.348 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:52.347+0000 7febbb59e640 1 Processor -- start 2023-11-26T07:22:52.348 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:52.347+0000 7febbb59e640 1 -- start start 2023-11-26T07:22:52.350 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:52.347+0000 7febbb59e640 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7febbc15d230 0x7febbc0d3be0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:22:52.351 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:52.347+0000 7febbb59e640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7febbc15d610 0x7febbc0cdc40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:22:52.351 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:52.347+0000 7febbb59e640 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7febbc0cfaa0 0x7febbc0cfee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:22:52.351 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:52.347+0000 7febbb59e640 1 -- --> v2:172.21.15.143:3300/0 -- mon_getmap magic: 0 v1 -- 0x7febbc0ce180 con 0x7febbc0cfaa0 2023-11-26T07:22:52.351 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:52.347+0000 7febbb59e640 1 -- --> v2:172.21.15.188:3300/0 -- mon_getmap magic: 0 v1 -- 0x7febbc0d0420 con 0x7febbc15d230 2023-11-26T07:22:52.351 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:52.347+0000 7febbb59e640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7febbc0ce3b0 con 0x7febbc15d610 2023-11-26T07:22:52.351 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:52.347+0000 7febbad9d640 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7febbc0cfaa0 0x7febbc0cfee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:22:52.351 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:52.347+0000 7febbad9d640 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7febbc0cfaa0 0x7febbc0cfee0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:50524/0 (socket says 172.21.15.143:50524) 2023-11-26T07:22:52.351 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:52.347+0000 7febbad9d640 1 -- 172.21.15.143:0/1256941382 learned_addr learned my addr 172.21.15.143:0/1256941382 (peer_addr_for_me v2:172.21.15.143:0/0) 2023-11-26T07:22:52.351 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:52.347+0000 7febb9d9b640 1 --2- 172.21.15.143:0/1256941382 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7febbc15d610 0x7febbc0cdc40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:22:52.351 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:52.347+0000 7febba59c640 1 --2- 172.21.15.143:0/1256941382 >> v2:172.21.15.188:3300/0 conn(0x7febbc15d230 0x7febbc0d3be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:22:52.351 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:52.347+0000 7febbad9d640 1 -- 172.21.15.143:0/1256941382 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7febbc15d610 msgr2=0x7febbc0cdc40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-26T07:22:52.352 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:52.347+0000 7febbad9d640 1 --2- 172.21.15.143:0/1256941382 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7febbc15d610 0x7febbc0cdc40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:22:52.352 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:52.347+0000 7febbad9d640 1 -- 172.21.15.143:0/1256941382 >> v2:172.21.15.188:3300/0 conn(0x7febbc15d230 msgr2=0x7febbc0d3be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-26T07:22:52.352 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:52.347+0000 7febbad9d640 1 --2- 172.21.15.143:0/1256941382 >> v2:172.21.15.188:3300/0 conn(0x7febbc15d230 0x7febbc0d3be0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:22:52.352 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:52.347+0000 7febbad9d640 1 -- 172.21.15.143:0/1256941382 --> v2:172.21.15.143:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7febbc0ce690 con 0x7febbc0cfaa0 2023-11-26T07:22:52.352 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:52.347+0000 7febb9d9b640 1 --2- 172.21.15.143:0/1256941382 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7febbc15d610 0x7febbc0cdc40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-26T07:22:52.352 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:52.347+0000 7febba59c640 1 --2- 172.21.15.143:0/1256941382 >> v2:172.21.15.188:3300/0 conn(0x7febbc15d230 0x7febbc0d3be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-26T07:22:52.353 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:52.351+0000 7febbad9d640 1 --2- 172.21.15.143:0/1256941382 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7febbc0cfaa0 0x7febbc0cfee0 secure :-1 s=READY pgs=1013 cs=0 l=1 rev1=1 crypto rx=0x7febbc08b3a0 tx=0x7febb401c940 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dccead04164ac3c9 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:22:52.354 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:52.351+0000 7febb959a640 1 -- 172.21.15.143:0/1256941382 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7febb40a57c0 con 0x7febbc0cfaa0 2023-11-26T07:22:52.354 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:52.351+0000 7febb959a640 1 -- 172.21.15.143:0/1256941382 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7febb40a5920 con 0x7febbc0cfaa0 2023-11-26T07:22:52.354 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:52.351+0000 7febb959a640 1 -- 172.21.15.143:0/1256941382 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7febb40a5cb0 con 0x7febbc0cfaa0 2023-11-26T07:22:52.355 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:52.351+0000 7febbb59e640 1 -- 172.21.15.143:0/1256941382 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7febbc0cfaa0 msgr2=0x7febbc0cfee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:22:52.355 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:52.351+0000 7febbb59e640 1 --2- 172.21.15.143:0/1256941382 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7febbc0cfaa0 0x7febbc0cfee0 secure :-1 s=READY pgs=1013 cs=0 l=1 rev1=1 crypto rx=0x7febbc08b3a0 tx=0x7febb401c940 comp rx=0 tx=0).stop 2023-11-26T07:22:52.355 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:52.351+0000 7febbb59e640 1 -- 172.21.15.143:0/1256941382 shutdown_connections 2023-11-26T07:22:52.355 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:52.351+0000 7febbb59e640 1 --2- 172.21.15.143:0/1256941382 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7febbc0cfaa0 0x7febbc0cfee0 unknown :-1 s=CLOSED pgs=1013 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:22:52.355 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:52.351+0000 7febbb59e640 1 --2- 172.21.15.143:0/1256941382 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7febbc15d610 0x7febbc0cdc40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:22:52.355 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:52.351+0000 7febbb59e640 1 --2- 172.21.15.143:0/1256941382 >> v2:172.21.15.188:3300/0 conn(0x7febbc15d230 0x7febbc0d3be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:22:52.355 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:52.351+0000 7febbb59e640 1 -- 172.21.15.143:0/1256941382 >> 172.21.15.143:0/1256941382 conn(0x7febbc0d30c0 msgr2=0x7febbc0d34d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:22:52.355 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:52.351+0000 7febbb59e640 1 -- 172.21.15.143:0/1256941382 shutdown_connections 2023-11-26T07:22:52.355 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:52.351+0000 7febbb59e640 1 -- 172.21.15.143:0/1256941382 wait complete. 2023-11-26T07:22:52.356 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:52.355+0000 7febbb59e640 1 Processor -- start 2023-11-26T07:22:52.356 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:52.355+0000 7febbb59e640 1 -- start start 2023-11-26T07:22:52.358 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:52.355+0000 7febbb59e640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7febbc0cfaa0 0x7febbc14e170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:22:52.358 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:52.355+0000 7febbb59e640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7febbc15d230 0x7febbc15e230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:22:52.359 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:52.355+0000 7febbb59e640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7febbc15d610 0x7febbc0cbc70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:22:52.359 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:52.355+0000 7febbb59e640 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x7febbc0c9160 con 0x7febbc0cfaa0 2023-11-26T07:22:52.359 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:52.355+0000 7febbb59e640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7febbc15dc40 con 0x7febbc15d230 2023-11-26T07:22:52.359 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:52.355+0000 7febbb59e640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7febbc15df20 con 0x7febbc15d610 2023-11-26T07:22:52.359 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:52.355+0000 7febba59c640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7febbc0cfaa0 0x7febbc14e170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:22:52.359 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:52.355+0000 7febba59c640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7febbc0cfaa0 0x7febbc14e170 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:50526/0 (socket says 172.21.15.143:50526) 2023-11-26T07:22:52.359 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:52.355+0000 7febba59c640 1 -- 172.21.15.143:0/3515738008 learned_addr learned my addr 172.21.15.143:0/3515738008 (peer_addr_for_me v2:172.21.15.143:0/0) 2023-11-26T07:22:52.359 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:52.355+0000 7febbad9d640 1 --2- 172.21.15.143:0/3515738008 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7febbc15d610 0x7febbc0cbc70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:22:52.359 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:52.355+0000 7febb9d9b640 1 --2- 172.21.15.143:0/3515738008 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7febbc15d230 0x7febbc15e230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:22:52.359 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:52.355+0000 7febba59c640 1 -- 172.21.15.143:0/3515738008 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7febbc15d610 msgr2=0x7febbc0cbc70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:22:52.359 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:52.355+0000 7febba59c640 1 --2- 172.21.15.143:0/3515738008 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7febbc15d610 0x7febbc0cbc70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:22:52.359 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:52.355+0000 7febba59c640 1 -- 172.21.15.143:0/3515738008 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7febbc15d230 msgr2=0x7febbc15e230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:22:52.359 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:52.355+0000 7febba59c640 1 --2- 172.21.15.143:0/3515738008 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7febbc15d230 0x7febbc15e230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:22:52.360 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:52.355+0000 7febba59c640 1 -- 172.21.15.143:0/3515738008 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7febb4009b30 con 0x7febbc0cfaa0 2023-11-26T07:22:52.360 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:52.355+0000 7febba59c640 1 --2- 172.21.15.143:0/3515738008 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7febbc0cfaa0 0x7febbc14e170 secure :-1 s=READY pgs=1014 cs=0 l=1 rev1=1 crypto rx=0x7febac00e810 tx=0x7febac00ecd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:22:52.360 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:52.355+0000 7febaaffd640 1 -- 172.21.15.143:0/3515738008 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7febac004030 con 0x7febbc0cfaa0 2023-11-26T07:22:52.360 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:52.355+0000 7febaaffd640 1 -- 172.21.15.143:0/3515738008 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7febac004190 con 0x7febbc0cfaa0 2023-11-26T07:22:52.360 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:52.355+0000 7febbb59e640 1 -- 172.21.15.143:0/3515738008 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7febbc160390 con 0x7febbc0cfaa0 2023-11-26T07:22:52.360 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:52.355+0000 7febaaffd640 1 -- 172.21.15.143:0/3515738008 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7febac004520 con 0x7febbc0cfaa0 2023-11-26T07:22:52.360 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:52.355+0000 7febbb59e640 1 -- 172.21.15.143:0/3515738008 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7febbc15e770 con 0x7febbc0cfaa0 2023-11-26T07:22:52.360 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:52.359+0000 7febbb59e640 1 -- 172.21.15.143:0/3515738008 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7febbc067c10 con 0x7febbc0cfaa0 2023-11-26T07:22:52.360 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:52.359+0000 7febaaffd640 1 -- 172.21.15.143:0/3515738008 <== mon.0 v2:172.21.15.143:3300/0 4 ==== mgrmap(e 11) v1 ==== 86158+0+0 (secure 0 0 0) 0x7febac00c6d0 con 0x7febbc0cfaa0 2023-11-26T07:22:52.361 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:52.359+0000 7febaaffd640 1 --2- 172.21.15.143:0/3515738008 >> [v2:172.21.15.143:6832/2150557544,v1:172.21.15.143:6833/2150557544] conn(0x7feb9c0667d0 0x7feb9c068c90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:22:52.362 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:52.359+0000 7febaaffd640 1 -- 172.21.15.143:0/3515738008 <== mon.0 v2:172.21.15.143:3300/0 5 ==== osd_map(71..71 src has 1..71) v4 ==== 5709+0+0 (secure 0 0 0) 0x7febac084020 con 0x7febbc0cfaa0 2023-11-26T07:22:52.362 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:52.359+0000 7febb9d9b640 1 --2- 172.21.15.143:0/3515738008 >> [v2:172.21.15.143:6832/2150557544,v1:172.21.15.143:6833/2150557544] conn(0x7feb9c0667d0 0x7feb9c068c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:22:52.362 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:52.359+0000 7febb9d9b640 1 --2- 172.21.15.143:0/3515738008 >> [v2:172.21.15.143:6832/2150557544,v1:172.21.15.143:6833/2150557544] conn(0x7feb9c0667d0 0x7feb9c068c90 secure :-1 s=READY pgs=471 cs=0 l=1 rev1=1 crypto rx=0x7febbc0d3fa0 tx=0x7febb0008040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:22:52.365 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:52.363+0000 7febaaffd640 1 -- 172.21.15.143:0/3515738008 <== mon.0 v2:172.21.15.143:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172649 (secure 0 0 0) 0x7febac050fd0 con 0x7febbc0cfaa0 2023-11-26T07:22:52.403 DEBUG:teuthology.orchestra.run.smithi143:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-26T07:22:52.406 DEBUG:teuthology.orchestra.run.smithi188:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-26T07:22:52.528 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:52.527+0000 7febbb59e640 1 -- 172.21.15.143:0/3515738008 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) v1 -- 0x7febbc0d1480 con 0x7febbc0cfaa0 2023-11-26T07:22:52.528 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:52.527+0000 7febaaffd640 1 -- 172.21.15.143:0/3515738008 <== mon.0 v2:172.21.15.143:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v71) v1 ==== 80+0+0 (secure 0 0 0) 0x7febac0545f0 con 0x7febbc0cfaa0 2023-11-26T07:22:52.529 INFO:teuthology.orchestra.run.smithi143.stderr:listed 0 entries 2023-11-26T07:22:52.531 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:52.527+0000 7febbb59e640 1 -- 172.21.15.143:0/3515738008 >> [v2:172.21.15.143:6832/2150557544,v1:172.21.15.143:6833/2150557544] conn(0x7feb9c0667d0 msgr2=0x7feb9c068c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:22:52.531 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:52.527+0000 7febbb59e640 1 --2- 172.21.15.143:0/3515738008 >> [v2:172.21.15.143:6832/2150557544,v1:172.21.15.143:6833/2150557544] conn(0x7feb9c0667d0 0x7feb9c068c90 secure :-1 s=READY pgs=471 cs=0 l=1 rev1=1 crypto rx=0x7febbc0d3fa0 tx=0x7febb0008040 comp rx=0 tx=0).stop 2023-11-26T07:22:52.531 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:52.527+0000 7febbb59e640 1 -- 172.21.15.143:0/3515738008 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7febbc0cfaa0 msgr2=0x7febbc14e170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:22:52.531 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:52.527+0000 7febbb59e640 1 --2- 172.21.15.143:0/3515738008 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7febbc0cfaa0 0x7febbc14e170 secure :-1 s=READY pgs=1014 cs=0 l=1 rev1=1 crypto rx=0x7febac00e810 tx=0x7febac00ecd0 comp rx=0 tx=0).stop 2023-11-26T07:22:52.531 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:52.527+0000 7febbb59e640 1 -- 172.21.15.143:0/3515738008 shutdown_connections 2023-11-26T07:22:52.531 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:52.527+0000 7febbb59e640 1 --2- 172.21.15.143:0/3515738008 >> [v2:172.21.15.143:6832/2150557544,v1:172.21.15.143:6833/2150557544] conn(0x7feb9c0667d0 0x7feb9c068c90 unknown :-1 s=CLOSED pgs=471 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:22:52.531 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:52.527+0000 7febbb59e640 1 --2- 172.21.15.143:0/3515738008 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7febbc15d610 0x7febbc0cbc70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:22:52.531 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:52.527+0000 7febbb59e640 1 --2- 172.21.15.143:0/3515738008 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7febbc15d230 0x7febbc15e230 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:22:52.531 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:52.527+0000 7febbb59e640 1 --2- 172.21.15.143:0/3515738008 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7febbc0cfaa0 0x7febbc14e170 unknown :-1 s=CLOSED pgs=1014 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:22:52.531 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:52.527+0000 7febbb59e640 1 -- 172.21.15.143:0/3515738008 >> 172.21.15.143:0/3515738008 conn(0x7febbc0d30c0 msgr2=0x7febbc06af60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:22:52.531 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:52.527+0000 7febbb59e640 1 -- 172.21.15.143:0/3515738008 shutdown_connections 2023-11-26T07:22:52.531 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:52.527+0000 7febbb59e640 1 -- 172.21.15.143:0/3515738008 wait complete. 2023-11-26T07:22:52.549 INFO:tasks.cephfs.fuse_mount:Running fusermount -u on ubuntu@smithi143.front.sepia.ceph.com... 2023-11-26T07:22:52.549 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-26T07:22:52.550 DEBUG:teuthology.orchestra.run.smithi143:> sudo fusermount -u /home/ubuntu/cephtest/mnt.0 2023-11-26T07:22:52.568 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi143.stderr:ceph-fuse[55259]: fuse finished with error 0 and tester_r 0 2023-11-26T07:22:52.570 INFO:teuthology.orchestra.run:waiting for 300 2023-11-26T07:22:58.672 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi143.front.sepia.ceph.com 2023-11-26T07:22:58.673 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-26T07:22:58.673 DEBUG:teuthology.orchestra.run.smithi143:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2023-11-26T07:22:58.683 DEBUG:teuthology.orchestra.run.smithi188:> set -ex 2023-11-26T07:22:58.683 DEBUG:teuthology.orchestra.run.smithi188:> dd if=/proc/self/mounts of=/dev/stdout 2023-11-26T07:22:58.693 DEBUG:teuthology.orchestra.run.smithi143:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-11-26T07:22:58.873 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:58.871+0000 7f7fa3232640 1 Processor -- start 2023-11-26T07:22:58.873 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:58.871+0000 7f7fa3232640 1 -- start start 2023-11-26T07:22:58.874 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:58.871+0000 7f7fa3232640 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7f7f9c0d0330 0x7f7f9c0d3de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:22:58.874 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:58.871+0000 7f7fa3232640 1 -- --> v1:172.21.15.143:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7f9c0ce9d0 con 0x7f7f9c160d30 2023-11-26T07:22:58.875 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:58.871+0000 7f7fa3232640 1 -- --> v1:172.21.15.188:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7f9c0cecb0 con 0x7f7f9c0b20e0 2023-11-26T07:22:58.875 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:58.871+0000 7f7fa3232640 1 -- --> v2:172.21.15.188:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7f9c0cf550 con 0x7f7f9c0d0330 2023-11-26T07:22:58.875 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:58.871+0000 7f7fa0fce640 1 --1- >> v1:172.21.15.143:6789/0 conn(0x7f7f9c160d30 0x7f7f9c0cfc20 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.143:6789/0 says I am v1:172.21.15.143:57854/0 (socket says 172.21.15.143:57854) 2023-11-26T07:22:58.875 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:58.871+0000 7f7fa0fce640 1 -- 172.21.15.143:0/171516758 learned_addr learned my addr 172.21.15.143:0/171516758 (peer_addr_for_me v1:172.21.15.143:0/0) 2023-11-26T07:22:58.875 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:58.871+0000 7f7f93fff640 1 --2- 172.21.15.143:0/171516758 >> v2:172.21.15.188:3300/0 conn(0x7f7f9c0d0330 0x7f7f9c0d3de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:22:58.876 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:58.871+0000 7f7f937fe640 1 -- 172.21.15.143:0/171516758 <== mon.0 v1:172.21.15.143:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3085699502 0 0) 0x7f7f9c0ce9d0 con 0x7f7f9c160d30 2023-11-26T07:22:58.876 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:58.871+0000 7f7f937fe640 1 -- 172.21.15.143:0/171516758 --> v1:172.21.15.143:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f7f84003530 con 0x7f7f9c160d30 2023-11-26T07:22:58.876 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:58.875+0000 7f7f937fe640 1 -- 172.21.15.143:0/171516758 <== mon.0 v1:172.21.15.143:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1113364475 0 0) 0x7f7f84003530 con 0x7f7f9c160d30 2023-11-26T07:22:58.876 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:58.875+0000 7f7f93fff640 1 -- 172.21.15.143:0/171516758 >> v1:172.21.15.188:6789/0 conn(0x7f7f9c0b20e0 legacy=0x7f7f9c0b2dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:22:58.876 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:58.875+0000 7f7f93fff640 1 -- 172.21.15.143:0/171516758 >> v1:172.21.15.143:6789/0 conn(0x7f7f9c160d30 legacy=0x7f7f9c0cfc20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:22:58.876 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:58.875+0000 7f7f93fff640 1 -- 172.21.15.143:0/171516758 --> v2:172.21.15.188:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7f9c0cef90 con 0x7f7f9c0d0330 2023-11-26T07:22:58.878 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:58.875+0000 7f7f93fff640 1 --2- 172.21.15.143:0/171516758 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f7f9c0d0330 0x7f7f9c0d3de0 secure :-1 s=READY pgs=403 cs=0 l=1 rev1=1 crypto rx=0x7f7f8c0097c0 tx=0x7f7f8c01c3a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=da8aedb943d214c8 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:22:58.878 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:58.875+0000 7f7f937fe640 1 -- 172.21.15.143:0/171516758 <== mon.1 v2:172.21.15.188:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7f9c0b4ab0 con 0x7f7f9c0d0330 2023-11-26T07:22:58.878 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:58.875+0000 7f7f937fe640 1 -- 172.21.15.143:0/171516758 <== mon.1 v2:172.21.15.188:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7f8c0a7050 con 0x7f7f9c0d0330 2023-11-26T07:22:58.879 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:58.875+0000 7f7f937fe640 1 -- 172.21.15.143:0/171516758 <== mon.1 v2:172.21.15.188:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7f98001680 con 0x7f7f9c0d0330 2023-11-26T07:22:58.879 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:58.875+0000 7f7fa3232640 1 -- 172.21.15.143:0/171516758 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f7f9c0d0330 msgr2=0x7f7f9c0d3de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:22:58.879 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:58.875+0000 7f7fa3232640 1 --2- 172.21.15.143:0/171516758 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f7f9c0d0330 0x7f7f9c0d3de0 secure :-1 s=READY pgs=403 cs=0 l=1 rev1=1 crypto rx=0x7f7f8c0097c0 tx=0x7f7f8c01c3a0 comp rx=0 tx=0).stop 2023-11-26T07:22:58.879 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:58.879+0000 7f7fa3232640 1 -- 172.21.15.143:0/171516758 shutdown_connections 2023-11-26T07:22:58.880 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:58.879+0000 7f7fa3232640 1 --2- 172.21.15.143:0/171516758 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f7f9c0d0330 0x7f7f9c0d3de0 unknown :-1 s=CLOSED pgs=403 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:22:58.880 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:58.879+0000 7f7fa3232640 1 -- 172.21.15.143:0/171516758 >> 172.21.15.143:0/171516758 conn(0x7f7f9c09c560 msgr2=0x7f7f9c0d3210 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:22:58.880 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:58.879+0000 7f7fa3232640 1 -- 172.21.15.143:0/171516758 shutdown_connections 2023-11-26T07:22:58.880 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:58.879+0000 7f7fa3232640 1 -- 172.21.15.143:0/171516758 wait complete. 2023-11-26T07:22:58.881 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:58.879+0000 7f7fa3232640 1 Processor -- start 2023-11-26T07:22:58.881 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:58.879+0000 7f7fa3232640 1 -- start start 2023-11-26T07:22:58.883 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:58.879+0000 7f7fa3232640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f7f9c0b20e0 0x7f7f9c1555c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:22:58.883 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:58.879+0000 7f7fa3232640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f7f9c0d0330 0x7f7f9c0cbf10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:22:58.883 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:58.879+0000 7f7fa3232640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f7f9c160d30 0x7f7f9c14ba20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:22:58.883 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:58.879+0000 7f7fa3232640 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7f9c0cc450 con 0x7f7f9c160d30 2023-11-26T07:22:58.884 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:58.879+0000 7f7fa3232640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7f9c153b20 con 0x7f7f9c0b20e0 2023-11-26T07:22:58.884 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:58.879+0000 7f7fa3232640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7f9c153e00 con 0x7f7f9c0d0330 2023-11-26T07:22:58.884 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:58.879+0000 7f7fa17cf640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f7f9c160d30 0x7f7f9c14ba20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:22:58.884 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:58.879+0000 7f7fa17cf640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f7f9c160d30 0x7f7f9c14ba20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:42796/0 (socket says 172.21.15.143:42796) 2023-11-26T07:22:58.884 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:58.879+0000 7f7fa17cf640 1 -- 172.21.15.143:0/1870050776 learned_addr learned my addr 172.21.15.143:0/1870050776 (peer_addr_for_me v2:172.21.15.143:0/0) 2023-11-26T07:22:58.884 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:58.879+0000 7f7fa0fce640 1 --2- 172.21.15.143:0/1870050776 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f7f9c0b20e0 0x7f7f9c1555c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:22:58.884 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:58.879+0000 7f7f93fff640 1 --2- 172.21.15.143:0/1870050776 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f7f9c0d0330 0x7f7f9c0cbf10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:22:58.884 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:58.879+0000 7f7fa17cf640 1 -- 172.21.15.143:0/1870050776 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f7f9c0d0330 msgr2=0x7f7f9c0cbf10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:22:58.884 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:58.879+0000 7f7fa17cf640 1 --2- 172.21.15.143:0/1870050776 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f7f9c0d0330 0x7f7f9c0cbf10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:22:58.884 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:58.879+0000 7f7fa17cf640 1 -- 172.21.15.143:0/1870050776 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f7f9c0b20e0 msgr2=0x7f7f9c1555c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:22:58.884 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:58.879+0000 7f7fa17cf640 1 --2- 172.21.15.143:0/1870050776 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f7f9c0b20e0 0x7f7f9c1555c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:22:58.884 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:58.879+0000 7f7fa17cf640 1 -- 172.21.15.143:0/1870050776 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7f8c009420 con 0x7f7f9c160d30 2023-11-26T07:22:58.884 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:58.879+0000 7f7f93fff640 1 --2- 172.21.15.143:0/1870050776 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f7f9c0d0330 0x7f7f9c0cbf10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-26T07:22:58.885 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:58.883+0000 7f7fa17cf640 1 --2- 172.21.15.143:0/1870050776 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f7f9c160d30 0x7f7f9c14ba20 secure :-1 s=READY pgs=1016 cs=0 l=1 rev1=1 crypto rx=0x7f7f9400c770 tx=0x7f7f9400cc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:22:58.885 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:58.883+0000 7f7f917fa640 1 -- 172.21.15.143:0/1870050776 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7f94003b80 con 0x7f7f9c160d30 2023-11-26T07:22:58.885 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:58.883+0000 7f7f917fa640 1 -- 172.21.15.143:0/1870050776 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7f94003ce0 con 0x7f7f9c160d30 2023-11-26T07:22:58.885 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:58.883+0000 7f7fa3232640 1 -- 172.21.15.143:0/1870050776 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7f9c1540e0 con 0x7f7f9c160d30 2023-11-26T07:22:58.885 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:58.883+0000 7f7f917fa640 1 -- 172.21.15.143:0/1870050776 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7f9400aa90 con 0x7f7f9c160d30 2023-11-26T07:22:58.885 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:58.883+0000 7f7fa3232640 1 -- 172.21.15.143:0/1870050776 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7f9c155ba0 con 0x7f7f9c160d30 2023-11-26T07:22:58.885 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:58.883+0000 7f7fa3232640 1 -- 172.21.15.143:0/1870050776 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7f9c067680 con 0x7f7f9c160d30 2023-11-26T07:22:58.886 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:58.883+0000 7f7f917fa640 1 -- 172.21.15.143:0/1870050776 <== mon.0 v2:172.21.15.143:3300/0 4 ==== mgrmap(e 11) v1 ==== 86158+0+0 (secure 0 0 0) 0x7f7f9400abf0 con 0x7f7f9c160d30 2023-11-26T07:22:58.886 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:58.883+0000 7f7f917fa640 1 --2- 172.21.15.143:0/1870050776 >> [v2:172.21.15.143:6832/2150557544,v1:172.21.15.143:6833/2150557544] conn(0x7f7f880667d0 0x7f7f88068c90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:22:58.886 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:58.883+0000 7f7f917fa640 1 -- 172.21.15.143:0/1870050776 <== mon.0 v2:172.21.15.143:3300/0 5 ==== osd_map(71..71 src has 1..71) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f7f94084250 con 0x7f7f9c160d30 2023-11-26T07:22:58.887 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:58.883+0000 7f7fa0fce640 1 --2- 172.21.15.143:0/1870050776 >> [v2:172.21.15.143:6832/2150557544,v1:172.21.15.143:6833/2150557544] conn(0x7f7f880667d0 0x7f7f88068c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:22:58.887 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:58.887+0000 7f7fa0fce640 1 --2- 172.21.15.143:0/1870050776 >> [v2:172.21.15.143:6832/2150557544,v1:172.21.15.143:6833/2150557544] conn(0x7f7f880667d0 0x7f7f88068c90 secure :-1 s=READY pgs=472 cs=0 l=1 rev1=1 crypto rx=0x7f7f98000f70 tx=0x7f7f98001d70 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:22:58.890 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:58.887+0000 7f7f917fa640 1 -- 172.21.15.143:0/1870050776 <== mon.0 v2:172.21.15.143:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172649 (secure 0 0 0) 0x7f7f940511d0 con 0x7f7f9c160d30 2023-11-26T07:22:59.041 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:59.039+0000 7f7fa3232640 1 -- 172.21.15.143:0/1870050776 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f7f9c0cfda0 con 0x7f7f9c160d30 2023-11-26T07:22:59.042 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:59.039+0000 7f7f917fa640 1 -- 172.21.15.143:0/1870050776 <== mon.0 v2:172.21.15.143:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v71) v1 ==== 74+0+16604 (secure 0 0 0) 0x7f7f940547f0 con 0x7f7f9c160d30 2023-11-26T07:22:59.042 INFO:teuthology.orchestra.run.smithi143.stdout: 2023-11-26T07:22:59.043 INFO:teuthology.orchestra.run.smithi143.stdout:{"epoch":71,"fsid":"f5033060-2469-4a07-95fe-2b76f5404bc1","created":"2023-11-26T07:17:28.613663+0000","modified":"2023-11-26T07:22:38.856077+0000","last_up_change":"2023-11-26T07:17:41.157813+0000","last_in_change":"2023-11-26T07:17:31.181658+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":11,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-11-26T07:17:41.758597+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":10,"pool_name":"cephfs_metadata","create_time":"2023-11-26T07:22:35.314819+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"70","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":2.5,"score_stable":2.5,"optimal_score":1,"raw_score_acting":2.5,"raw_score_stable":2.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":11,"pool_name":"cephfs_data","create_time":"2023-11-26T07:22:36.219536+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"71","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"ebbee2c6-023e-44b9-9f1d-6f7bfd309730","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":69,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6817","nonce":639807369},{"type":"v1","addr":"172.21.15.143:6819","nonce":639807369}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6821","nonce":639807369},{"type":"v1","addr":"172.21.15.143:6823","nonce":639807369}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6830","nonce":639807369},{"type":"v1","addr":"172.21.15.143:6831","nonce":639807369}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6825","nonce":639807369},{"type":"v1","addr":"172.21.15.143:6828","nonce":639807369}]},"public_addr":"172.21.15.143:6819/639807369","cluster_addr":"172.21.15.143:6823/639807369","heartbeat_back_addr":"172.21.15.143:6831/639807369","heartbeat_front_addr":"172.21.15.143:6828/639807369","state":["exists","up"]},{"osd":1,"uuid":"fec4b8be-7122-49e9-9a19-5850a0700001","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":69,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6816","nonce":1144942530},{"type":"v1","addr":"172.21.15.143:6818","nonce":1144942530}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6820","nonce":1144942530},{"type":"v1","addr":"172.21.15.143:6822","nonce":1144942530}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6827","nonce":1144942530},{"type":"v1","addr":"172.21.15.143:6829","nonce":1144942530}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6824","nonce":1144942530},{"type":"v1","addr":"172.21.15.143:6826","nonce":1144942530}]},"public_addr":"172.21.15.143:6818/1144942530","cluster_addr":"172.21.15.143:6822/1144942530","heartbeat_back_addr":"172.21.15.143:6829/1144942530","heartbeat_front_addr":"172.21.15.143:6826/1144942530","state":["exists","up"]},{"osd":2,"uuid":"2888af5b-f933-46ca-8f95-6dd494ff43f6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":69,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6808","nonce":3909095186},{"type":"v1","addr":"172.21.15.143:6809","nonce":3909095186}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6810","nonce":3909095186},{"type":"v1","addr":"172.21.15.143:6811","nonce":3909095186}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6814","nonce":3909095186},{"type":"v1","addr":"172.21.15.143:6815","nonce":3909095186}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6812","nonce":3909095186},{"type":"v1","addr":"172.21.15.143:6813","nonce":3909095186}]},"public_addr":"172.21.15.143:6809/3909095186","cluster_addr":"172.21.15.143:6811/3909095186","heartbeat_back_addr":"172.21.15.143:6815/3909095186","heartbeat_front_addr":"172.21.15.143:6813/3909095186","state":["exists","up"]},{"osd":3,"uuid":"d1f52435-7bc1-4081-9550-9cd1ba55f549","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":69,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6800","nonce":3854210991},{"type":"v1","addr":"172.21.15.143:6801","nonce":3854210991}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6802","nonce":3854210991},{"type":"v1","addr":"172.21.15.143:6803","nonce":3854210991}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6806","nonce":3854210991},{"type":"v1","addr":"172.21.15.143:6807","nonce":3854210991}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6804","nonce":3854210991},{"type":"v1","addr":"172.21.15.143:6805","nonce":3854210991}]},"public_addr":"172.21.15.143:6801/3854210991","cluster_addr":"172.21.15.143:6803/3854210991","heartbeat_back_addr":"172.21.15.143:6807/3854210991","heartbeat_front_addr":"172.21.15.143:6805/3854210991","state":["exists","up"]},{"osd":4,"uuid":"e2ae8b6e-6e25-49b2-bcbf-aacce68f06a3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":69,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6800","nonce":3924606902},{"type":"v1","addr":"172.21.15.188:6801","nonce":3924606902}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6802","nonce":3924606902},{"type":"v1","addr":"172.21.15.188:6803","nonce":3924606902}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6806","nonce":3924606902},{"type":"v1","addr":"172.21.15.188:6807","nonce":3924606902}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6804","nonce":3924606902},{"type":"v1","addr":"172.21.15.188:6805","nonce":3924606902}]},"public_addr":"172.21.15.188:6801/3924606902","cluster_addr":"172.21.15.188:6803/3924606902","heartbeat_back_addr":"172.21.15.188:6807/3924606902","heartbeat_front_addr":"172.21.15.188:6805/3924606902","state":["exists","up"]},{"osd":5,"uuid":"3cec8a32-7698-4c36-8e89-e4ba11934f33","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":69,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6813","nonce":186841544},{"type":"v1","addr":"172.21.15.188:6815","nonce":186841544}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6817","nonce":186841544},{"type":"v1","addr":"172.21.15.188:6818","nonce":186841544}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6822","nonce":186841544},{"type":"v1","addr":"172.21.15.188:6823","nonce":186841544}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6820","nonce":186841544},{"type":"v1","addr":"172.21.15.188:6821","nonce":186841544}]},"public_addr":"172.21.15.188:6815/186841544","cluster_addr":"172.21.15.188:6818/186841544","heartbeat_back_addr":"172.21.15.188:6823/186841544","heartbeat_front_addr":"172.21.15.188:6821/186841544","state":["exists","up"]},{"osd":6,"uuid":"58a65d5d-e090-473a-81b4-ad38e85029f9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":69,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6808","nonce":2153499405},{"type":"v1","addr":"172.21.15.188:6809","nonce":2153499405}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6810","nonce":2153499405},{"type":"v1","addr":"172.21.15.188:6811","nonce":2153499405}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6816","nonce":2153499405},{"type":"v1","addr":"172.21.15.188:6819","nonce":2153499405}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6812","nonce":2153499405},{"type":"v1","addr":"172.21.15.188:6814","nonce":2153499405}]},"public_addr":"172.21.15.188:6809/2153499405","cluster_addr":"172.21.15.188:6811/2153499405","heartbeat_back_addr":"172.21.15.188:6819/2153499405","heartbeat_front_addr":"172.21.15.188:6814/2153499405","state":["exists","up"]},{"osd":7,"uuid":"ea5b7f70-3554-4cf8-ac30-f7c59b80678d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":69,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6824","nonce":2722096729},{"type":"v1","addr":"172.21.15.188:6825","nonce":2722096729}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6826","nonce":2722096729},{"type":"v1","addr":"172.21.15.188:6827","nonce":2722096729}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6830","nonce":2722096729},{"type":"v1","addr":"172.21.15.188:6831","nonce":2722096729}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6828","nonce":2722096729},{"type":"v1","addr":"172.21.15.188:6829","nonce":2722096729}]},"public_addr":"172.21.15.188:6825/2722096729","cluster_addr":"172.21.15.188:6827/2722096729","heartbeat_back_addr":"172.21.15.188:6831/2722096729","heartbeat_front_addr":"172.21.15.188:6829/2722096729","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-26T07:17:36.995032+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-26T07:17:36.932266+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-26T07:17:36.829304+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-26T07:17:36.804297+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-26T07:17:36.667892+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-26T07:17:36.964868+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-26T07:17:36.827332+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-26T07:17:36.893179+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-11-26T07:22:59.044 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:59.043+0000 7f7fa3232640 1 -- 172.21.15.143:0/1870050776 >> [v2:172.21.15.143:6832/2150557544,v1:172.21.15.143:6833/2150557544] conn(0x7f7f880667d0 msgr2=0x7f7f88068c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:22:59.044 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:59.043+0000 7f7fa3232640 1 --2- 172.21.15.143:0/1870050776 >> [v2:172.21.15.143:6832/2150557544,v1:172.21.15.143:6833/2150557544] conn(0x7f7f880667d0 0x7f7f88068c90 secure :-1 s=READY pgs=472 cs=0 l=1 rev1=1 crypto rx=0x7f7f98000f70 tx=0x7f7f98001d70 comp rx=0 tx=0).stop 2023-11-26T07:22:59.044 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:59.043+0000 7f7fa3232640 1 -- 172.21.15.143:0/1870050776 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f7f9c160d30 msgr2=0x7f7f9c14ba20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:22:59.045 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:59.043+0000 7f7fa3232640 1 --2- 172.21.15.143:0/1870050776 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f7f9c160d30 0x7f7f9c14ba20 secure :-1 s=READY pgs=1016 cs=0 l=1 rev1=1 crypto rx=0x7f7f9400c770 tx=0x7f7f9400cc30 comp rx=0 tx=0).stop 2023-11-26T07:22:59.045 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:59.043+0000 7f7fa3232640 1 -- 172.21.15.143:0/1870050776 shutdown_connections 2023-11-26T07:22:59.045 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:59.043+0000 7f7fa3232640 1 --2- 172.21.15.143:0/1870050776 >> [v2:172.21.15.143:6832/2150557544,v1:172.21.15.143:6833/2150557544] conn(0x7f7f880667d0 0x7f7f88068c90 unknown :-1 s=CLOSED pgs=472 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:22:59.045 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:59.043+0000 7f7fa3232640 1 --2- 172.21.15.143:0/1870050776 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f7f9c160d30 0x7f7f9c14ba20 unknown :-1 s=CLOSED pgs=1016 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:22:59.045 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:59.043+0000 7f7fa3232640 1 --2- 172.21.15.143:0/1870050776 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f7f9c0d0330 0x7f7f9c0cbf10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:22:59.045 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:59.043+0000 7f7fa3232640 1 --2- 172.21.15.143:0/1870050776 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f7f9c0b20e0 0x7f7f9c1555c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:22:59.045 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:59.043+0000 7f7fa3232640 1 -- 172.21.15.143:0/1870050776 >> 172.21.15.143:0/1870050776 conn(0x7f7f9c09c560 msgr2=0x7f7f9c156320 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:22:59.045 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:59.043+0000 7f7fa3232640 1 -- 172.21.15.143:0/1870050776 shutdown_connections 2023-11-26T07:22:59.045 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:59.043+0000 7f7fa3232640 1 -- 172.21.15.143:0/1870050776 wait complete. 2023-11-26T07:22:59.064 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-11-26T07:17:41.758597+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 10, 'pool_name': 'cephfs_metadata', 'create_time': '2023-11-26T07:22:35.314819+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '70', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 2.5, 'score_stable': 2.5, 'optimal_score': 1, 'raw_score_acting': 2.5, 'raw_score_stable': 2.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 11, 'pool_name': 'cephfs_data', 'create_time': '2023-11-26T07:22:36.219536+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '71', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.3799999952316284, 'score_stable': 1.3799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.3799999952316284, 'raw_score_stable': 1.3799999952316284, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2023-11-26T07:22:59.064 DEBUG:teuthology.orchestra.run.smithi143:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-11-26T07:22:59.204 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:59.203+0000 7f1bc2a79640 1 Processor -- start 2023-11-26T07:22:59.205 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:59.203+0000 7f1bc2a79640 1 -- start start 2023-11-26T07:22:59.206 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:59.203+0000 7f1bc2a79640 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7f1bbc06c970 0x7f1bbc06cde0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:22:59.206 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:59.203+0000 7f1bc2a79640 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7f1bbc0b2a00 0x7f1bbc0b2e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:22:59.206 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:59.203+0000 7f1bc2a79640 1 -- --> v1:172.21.15.143:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1bbc143dc0 con 0x7f1bbc1439e0 2023-11-26T07:22:59.206 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:59.203+0000 7f1bc2a79640 1 -- --> v2:172.21.15.143:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1bbc0ad700 con 0x7f1bbc06c970 2023-11-26T07:22:59.207 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:59.203+0000 7f1bc2a79640 1 -- --> v2:172.21.15.188:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1bbc06d320 con 0x7f1bbc0b2a00 2023-11-26T07:22:59.207 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:59.203+0000 7f1bbbfff640 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7f1bbc06c970 0x7f1bbc06cde0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:22:59.207 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:59.203+0000 7f1bc0815640 1 --1- >> v1:172.21.15.143:6789/0 conn(0x7f1bbc1439e0 0x7f1bbc0adb20 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.143:6789/0 says I am v1:172.21.15.143:57864/0 (socket says 172.21.15.143:57864) 2023-11-26T07:22:59.207 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:59.203+0000 7f1bc0815640 1 -- 172.21.15.143:0/1571039595 learned_addr learned my addr 172.21.15.143:0/1571039595 (peer_addr_for_me v1:172.21.15.143:0/0) 2023-11-26T07:22:59.207 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:59.203+0000 7f1bbbfff640 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7f1bbc06c970 0x7f1bbc06cde0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:42812/0 (socket says 172.21.15.143:42812) 2023-11-26T07:22:59.208 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:59.203+0000 7f1bbbfff640 1 -- 172.21.15.143:0/1571039595 >> v2:172.21.15.188:3300/0 conn(0x7f1bbc0b2a00 msgr2=0x7f1bbc0b2e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-26T07:22:59.208 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:59.203+0000 7f1bc1016640 1 --2- 172.21.15.143:0/1571039595 >> v2:172.21.15.188:3300/0 conn(0x7f1bbc0b2a00 0x7f1bbc0b2e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:22:59.208 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:59.203+0000 7f1bbbfff640 1 --2- 172.21.15.143:0/1571039595 >> v2:172.21.15.188:3300/0 conn(0x7f1bbc0b2a00 0x7f1bbc0b2e70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:22:59.208 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:59.203+0000 7f1bbbfff640 1 -- 172.21.15.143:0/1571039595 >> v1:172.21.15.143:6789/0 conn(0x7f1bbc1439e0 legacy=0x7f1bbc0adb20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:22:59.208 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:59.203+0000 7f1bbb7fe640 1 -- 172.21.15.143:0/1571039595 <== mon.0 v1:172.21.15.143:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1865752830 0 0) 0x7f1bbc143dc0 con 0x7f1bbc1439e0 2023-11-26T07:22:59.208 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:59.203+0000 7f1bbbfff640 1 -- 172.21.15.143:0/1571039595 --> v2:172.21.15.143:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1bbc0aca80 con 0x7f1bbc06c970 2023-11-26T07:22:59.208 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:59.203+0000 7f1bc1016640 1 --2- 172.21.15.143:0/1571039595 >> v2:172.21.15.188:3300/0 conn(0x7f1bbc0b2a00 0x7f1bbc0b2e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-26T07:22:59.209 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:59.207+0000 7f1bbbfff640 1 --2- 172.21.15.143:0/1571039595 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f1bbc06c970 0x7f1bbc06cde0 secure :-1 s=READY pgs=1018 cs=0 l=1 rev1=1 crypto rx=0x7f1bbc0b1920 tx=0x7f1ba801c440 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e17a1b7f2961a921 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:22:59.210 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:59.207+0000 7f1bbb7fe640 1 -- 172.21.15.143:0/1571039595 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1ba80ab600 con 0x7f1bbc06c970 2023-11-26T07:22:59.210 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:59.207+0000 7f1bbb7fe640 1 -- 172.21.15.143:0/1571039595 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1ba80a7050 con 0x7f1bbc06c970 2023-11-26T07:22:59.210 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:59.207+0000 7f1bbb7fe640 1 -- 172.21.15.143:0/1571039595 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1ba80ab980 con 0x7f1bbc06c970 2023-11-26T07:22:59.211 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:59.207+0000 7f1bc2a79640 1 -- 172.21.15.143:0/1571039595 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f1bbc06c970 msgr2=0x7f1bbc06cde0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:22:59.211 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:59.207+0000 7f1bc2a79640 1 --2- 172.21.15.143:0/1571039595 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f1bbc06c970 0x7f1bbc06cde0 secure :-1 s=READY pgs=1018 cs=0 l=1 rev1=1 crypto rx=0x7f1bbc0b1920 tx=0x7f1ba801c440 comp rx=0 tx=0).stop 2023-11-26T07:22:59.211 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:59.207+0000 7f1bc2a79640 1 -- 172.21.15.143:0/1571039595 shutdown_connections 2023-11-26T07:22:59.211 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:59.207+0000 7f1bc2a79640 1 --2- 172.21.15.143:0/1571039595 >> v2:172.21.15.188:3300/0 conn(0x7f1bbc0b2a00 0x7f1bbc0b2e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:22:59.211 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:59.207+0000 7f1bc2a79640 1 --2- 172.21.15.143:0/1571039595 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f1bbc06c970 0x7f1bbc06cde0 unknown :-1 s=CLOSED pgs=1018 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:22:59.211 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:59.207+0000 7f1bc2a79640 1 -- 172.21.15.143:0/1571039595 >> 172.21.15.143:0/1571039595 conn(0x7f1bbc0b0bf0 msgr2=0x7f1bbc0b1000 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:22:59.211 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:59.207+0000 7f1bc2a79640 1 -- 172.21.15.143:0/1571039595 shutdown_connections 2023-11-26T07:22:59.211 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:59.207+0000 7f1bc2a79640 1 -- 172.21.15.143:0/1571039595 wait complete. 2023-11-26T07:22:59.212 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:59.211+0000 7f1bc2a79640 1 Processor -- start 2023-11-26T07:22:59.212 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:59.211+0000 7f1bc2a79640 1 -- start start 2023-11-26T07:22:59.215 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:59.211+0000 7f1bc2a79640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f1bbc06c970 0x7f1bbc0cb1f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:22:59.215 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:59.211+0000 7f1bc2a79640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f1bbc0b2a00 0x7f1bbc141ca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:22:59.215 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:59.211+0000 7f1bc2a79640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f1bbc1439e0 0x7f1bbc13a7f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:22:59.215 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:59.211+0000 7f1bc2a79640 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1bbc142980 con 0x7f1bbc0b2a00 2023-11-26T07:22:59.215 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:59.211+0000 7f1bc2a79640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1bbc140a30 con 0x7f1bbc06c970 2023-11-26T07:22:59.215 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:59.211+0000 7f1bc2a79640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1bbc141a40 con 0x7f1bbc1439e0 2023-11-26T07:22:59.215 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:59.211+0000 7f1bbbfff640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f1bbc0b2a00 0x7f1bbc141ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:22:59.215 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:59.211+0000 7f1bbbfff640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f1bbc0b2a00 0x7f1bbc141ca0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:42820/0 (socket says 172.21.15.143:42820) 2023-11-26T07:22:59.215 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:59.211+0000 7f1bbbfff640 1 -- 172.21.15.143:0/3972670528 learned_addr learned my addr 172.21.15.143:0/3972670528 (peer_addr_for_me v2:172.21.15.143:0/0) 2023-11-26T07:22:59.216 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:59.211+0000 7f1bc0815640 1 --2- 172.21.15.143:0/3972670528 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f1bbc06c970 0x7f1bbc0cb1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:22:59.216 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:59.211+0000 7f1bbbfff640 1 -- 172.21.15.143:0/3972670528 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f1bbc1439e0 msgr2=0x7f1bbc13a7f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:22:59.216 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:59.211+0000 7f1bbbfff640 1 --2- 172.21.15.143:0/3972670528 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f1bbc1439e0 0x7f1bbc13a7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:22:59.216 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:59.211+0000 7f1bbbfff640 1 -- 172.21.15.143:0/3972670528 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f1bbc06c970 msgr2=0x7f1bbc0cb1f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:22:59.216 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:59.211+0000 7f1bbbfff640 1 --2- 172.21.15.143:0/3972670528 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f1bbc06c970 0x7f1bbc0cb1f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:22:59.216 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:59.211+0000 7f1bbbfff640 1 -- 172.21.15.143:0/3972670528 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1ba8009420 con 0x7f1bbc0b2a00 2023-11-26T07:22:59.216 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:59.211+0000 7f1bbbfff640 1 --2- 172.21.15.143:0/3972670528 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f1bbc0b2a00 0x7f1bbc141ca0 secure :-1 s=READY pgs=1019 cs=0 l=1 rev1=1 crypto rx=0x7f1ba801c920 tx=0x7f1ba80191e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:22:59.216 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:59.211+0000 7f1bb97fa640 1 -- 172.21.15.143:0/3972670528 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1ba8004030 con 0x7f1bbc0b2a00 2023-11-26T07:22:59.216 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:59.211+0000 7f1bb97fa640 1 -- 172.21.15.143:0/3972670528 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1ba8004190 con 0x7f1bbc0b2a00 2023-11-26T07:22:59.216 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:59.211+0000 7f1bc2a79640 1 -- 172.21.15.143:0/3972670528 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1bbc140c80 con 0x7f1bbc0b2a00 2023-11-26T07:22:59.216 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:59.211+0000 7f1bb97fa640 1 -- 172.21.15.143:0/3972670528 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1ba8004500 con 0x7f1bbc0b2a00 2023-11-26T07:22:59.216 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:59.215+0000 7f1bc2a79640 1 -- 172.21.15.143:0/3972670528 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1bbc141240 con 0x7f1bbc0b2a00 2023-11-26T07:22:59.217 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:59.215+0000 7f1bc2a79640 1 -- 172.21.15.143:0/3972670528 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1bbc067680 con 0x7f1bbc0b2a00 2023-11-26T07:22:59.217 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:59.215+0000 7f1bb97fa640 1 -- 172.21.15.143:0/3972670528 <== mon.0 v2:172.21.15.143:3300/0 4 ==== mgrmap(e 11) v1 ==== 86158+0+0 (secure 0 0 0) 0x7f1ba80b2040 con 0x7f1bbc0b2a00 2023-11-26T07:22:59.218 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:59.215+0000 7f1bb97fa640 1 --2- 172.21.15.143:0/3972670528 >> [v2:172.21.15.143:6832/2150557544,v1:172.21.15.143:6833/2150557544] conn(0x7f1ba00667d0 0x7f1ba0068c90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:22:59.218 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:59.215+0000 7f1bb97fa640 1 -- 172.21.15.143:0/3972670528 <== mon.0 v2:172.21.15.143:3300/0 5 ==== osd_map(71..71 src has 1..71) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f1ba811dbd0 con 0x7f1bbc0b2a00 2023-11-26T07:22:59.218 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:59.215+0000 7f1bc0815640 1 --2- 172.21.15.143:0/3972670528 >> [v2:172.21.15.143:6832/2150557544,v1:172.21.15.143:6833/2150557544] conn(0x7f1ba00667d0 0x7f1ba0068c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:22:59.219 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:59.215+0000 7f1bc0815640 1 --2- 172.21.15.143:0/3972670528 >> [v2:172.21.15.143:6832/2150557544,v1:172.21.15.143:6833/2150557544] conn(0x7f1ba00667d0 0x7f1ba0068c90 secure :-1 s=READY pgs=473 cs=0 l=1 rev1=1 crypto rx=0x7f1bb0000f70 tx=0x7f1bb0007b60 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:22:59.222 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:59.219+0000 7f1bb97fa640 1 -- 172.21.15.143:0/3972670528 <== mon.0 v2:172.21.15.143:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172649 (secure 0 0 0) 0x7f1ba80eaaf0 con 0x7f1bbc0b2a00 2023-11-26T07:22:59.374 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:59.371+0000 7f1bc2a79640 1 -- 172.21.15.143:0/3972670528 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f1bbc0ce750 con 0x7f1bbc0b2a00 2023-11-26T07:22:59.375 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:59.375+0000 7f1bb97fa640 1 -- 172.21.15.143:0/3972670528 <== mon.0 v2:172.21.15.143:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v71) v1 ==== 93+0+10 (secure 0 0 0) 0x7f1ba80bc020 con 0x7f1bbc0b2a00 2023-11-26T07:22:59.375 INFO:teuthology.orchestra.run.smithi143.stdout:pg_num: 1 2023-11-26T07:22:59.377 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:59.375+0000 7f1bc2a79640 1 -- 172.21.15.143:0/3972670528 >> [v2:172.21.15.143:6832/2150557544,v1:172.21.15.143:6833/2150557544] conn(0x7f1ba00667d0 msgr2=0x7f1ba0068c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:22:59.377 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:59.375+0000 7f1bc2a79640 1 --2- 172.21.15.143:0/3972670528 >> [v2:172.21.15.143:6832/2150557544,v1:172.21.15.143:6833/2150557544] conn(0x7f1ba00667d0 0x7f1ba0068c90 secure :-1 s=READY pgs=473 cs=0 l=1 rev1=1 crypto rx=0x7f1bb0000f70 tx=0x7f1bb0007b60 comp rx=0 tx=0).stop 2023-11-26T07:22:59.377 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:59.375+0000 7f1bc2a79640 1 -- 172.21.15.143:0/3972670528 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f1bbc0b2a00 msgr2=0x7f1bbc141ca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:22:59.377 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:59.375+0000 7f1bc2a79640 1 --2- 172.21.15.143:0/3972670528 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f1bbc0b2a00 0x7f1bbc141ca0 secure :-1 s=READY pgs=1019 cs=0 l=1 rev1=1 crypto rx=0x7f1ba801c920 tx=0x7f1ba80191e0 comp rx=0 tx=0).stop 2023-11-26T07:22:59.377 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:59.375+0000 7f1bc2a79640 1 -- 172.21.15.143:0/3972670528 shutdown_connections 2023-11-26T07:22:59.378 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:59.375+0000 7f1bc2a79640 1 --2- 172.21.15.143:0/3972670528 >> [v2:172.21.15.143:6832/2150557544,v1:172.21.15.143:6833/2150557544] conn(0x7f1ba00667d0 0x7f1ba0068c90 unknown :-1 s=CLOSED pgs=473 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:22:59.378 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:59.375+0000 7f1bc2a79640 1 --2- 172.21.15.143:0/3972670528 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f1bbc1439e0 0x7f1bbc13a7f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:22:59.378 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:59.375+0000 7f1bc2a79640 1 --2- 172.21.15.143:0/3972670528 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f1bbc0b2a00 0x7f1bbc141ca0 unknown :-1 s=CLOSED pgs=1019 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:22:59.378 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:59.375+0000 7f1bc2a79640 1 --2- 172.21.15.143:0/3972670528 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f1bbc06c970 0x7f1bbc0cb1f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:22:59.378 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:59.375+0000 7f1bc2a79640 1 -- 172.21.15.143:0/3972670528 >> 172.21.15.143:0/3972670528 conn(0x7f1bbc0b0bf0 msgr2=0x7f1bbc0cee50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:22:59.378 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:59.375+0000 7f1bc2a79640 1 -- 172.21.15.143:0/3972670528 shutdown_connections 2023-11-26T07:22:59.378 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:59.375+0000 7f1bc2a79640 1 -- 172.21.15.143:0/3972670528 wait complete. 2023-11-26T07:22:59.397 DEBUG:teuthology.orchestra.run.smithi143:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-11-26T07:22:59.537 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:59.535+0000 7fefacde7640 1 Processor -- start 2023-11-26T07:22:59.537 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:59.535+0000 7fefacde7640 1 -- start start 2023-11-26T07:22:59.538 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:59.535+0000 7fefacde7640 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7fefa815d6e0 0x7fefa80d0df0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:22:59.539 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:59.535+0000 7fefacde7640 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7fefa80b2160 0x7fefa80d14b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:22:59.539 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:59.535+0000 7fefacde7640 1 -- --> v1:172.21.15.143:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fefa80cf0e0 con 0x7fefa80d19f0 2023-11-26T07:22:59.539 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:59.535+0000 7fefacde7640 1 -- --> v2:172.21.15.143:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fefa815dac0 con 0x7fefa80b2160 2023-11-26T07:22:59.539 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:59.535+0000 7fefacde7640 1 -- --> v2:172.21.15.188:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fefa80d1e80 con 0x7fefa815d6e0 2023-11-26T07:22:59.539 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:59.535+0000 7fef9ffff640 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7fefa80b2160 0x7fefa80d14b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:22:59.539 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:59.535+0000 7fef9ffff640 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7fefa80b2160 0x7fefa80d14b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:42824/0 (socket says 172.21.15.143:42824) 2023-11-26T07:22:59.539 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:59.535+0000 7fef9ffff640 1 -- 172.21.15.143:0/3762079013 learned_addr learned my addr 172.21.15.143:0/3762079013 (peer_addr_for_me v2:172.21.15.143:0/0) 2023-11-26T07:22:59.539 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:59.535+0000 7fefa659c640 1 --2- 172.21.15.143:0/3762079013 >> v2:172.21.15.188:3300/0 conn(0x7fefa815d6e0 0x7fefa80d0df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:22:59.539 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:59.535+0000 7fef9ffff640 1 -- 172.21.15.143:0/3762079013 >> v2:172.21.15.188:3300/0 conn(0x7fefa815d6e0 msgr2=0x7fefa80d0df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-26T07:22:59.540 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:59.535+0000 7fef9ffff640 1 --2- 172.21.15.143:0/3762079013 >> v2:172.21.15.188:3300/0 conn(0x7fefa815d6e0 0x7fefa80d0df0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:22:59.540 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:59.535+0000 7fef9ffff640 1 -- 172.21.15.143:0/3762079013 >> v1:172.21.15.143:6789/0 conn(0x7fefa80d19f0 legacy=0x7fefa80cfbf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:22:59.540 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:59.535+0000 7fef9ffff640 1 -- 172.21.15.143:0/3762079013 --> v2:172.21.15.143:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fefa80ceb20 con 0x7fefa80b2160 2023-11-26T07:22:59.542 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:59.539+0000 7fef9ffff640 1 --2- 172.21.15.143:0/3762079013 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fefa80b2160 0x7fefa80d14b0 secure :-1 s=READY pgs=1021 cs=0 l=1 rev1=1 crypto rx=0x7fefa80b5db0 tx=0x7fef9001c500 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c72cd90606082a87 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:22:59.542 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:59.539+0000 7fefa5d9b640 1 -- 172.21.15.143:0/3762079013 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fef900ab5f0 con 0x7fefa80b2160 2023-11-26T07:22:59.542 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:59.539+0000 7fefa5d9b640 1 -- 172.21.15.143:0/3762079013 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fef900ab750 con 0x7fefa80b2160 2023-11-26T07:22:59.542 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:59.539+0000 7fefa5d9b640 1 -- 172.21.15.143:0/3762079013 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fef900abae0 con 0x7fefa80b2160 2023-11-26T07:22:59.543 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:59.539+0000 7fefacde7640 1 -- 172.21.15.143:0/3762079013 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fefa80b2160 msgr2=0x7fefa80d14b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:22:59.543 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:59.539+0000 7fefacde7640 1 --2- 172.21.15.143:0/3762079013 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fefa80b2160 0x7fefa80d14b0 secure :-1 s=READY pgs=1021 cs=0 l=1 rev1=1 crypto rx=0x7fefa80b5db0 tx=0x7fef9001c500 comp rx=0 tx=0).stop 2023-11-26T07:22:59.543 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:59.539+0000 7fefacde7640 1 -- 172.21.15.143:0/3762079013 shutdown_connections 2023-11-26T07:22:59.543 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:59.539+0000 7fefacde7640 1 --2- 172.21.15.143:0/3762079013 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fefa80b2160 0x7fefa80d14b0 unknown :-1 s=CLOSED pgs=1021 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:22:59.543 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:59.539+0000 7fefacde7640 1 --2- 172.21.15.143:0/3762079013 >> v2:172.21.15.188:3300/0 conn(0x7fefa815d6e0 0x7fefa80d0df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:22:59.543 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:59.539+0000 7fefacde7640 1 -- 172.21.15.143:0/3762079013 >> 172.21.15.143:0/3762079013 conn(0x7fefa809c550 msgr2=0x7fefa80d3220 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:22:59.543 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:59.539+0000 7fefacde7640 1 -- 172.21.15.143:0/3762079013 shutdown_connections 2023-11-26T07:22:59.544 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:59.539+0000 7fefacde7640 1 -- 172.21.15.143:0/3762079013 wait complete. 2023-11-26T07:22:59.544 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:59.543+0000 7fefacde7640 1 Processor -- start 2023-11-26T07:22:59.544 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:59.543+0000 7fefacde7640 1 -- start start 2023-11-26T07:22:59.547 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:59.543+0000 7fefacde7640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fefa80b2160 0x7fefa8155020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:22:59.547 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:59.543+0000 7fefacde7640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fefa80d19f0 0x7fefa80cc2b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:22:59.547 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:59.543+0000 7fefacde7640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fefa815d6e0 0x7fefa80cc7f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:22:59.547 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:59.543+0000 7fefacde7640 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fefa81565a0 con 0x7fefa815d6e0 2023-11-26T07:22:59.547 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:59.543+0000 7fefacde7640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fefa8156880 con 0x7fefa80b2160 2023-11-26T07:22:59.547 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:59.543+0000 7fefacde7640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fefa8156b60 con 0x7fefa80d19f0 2023-11-26T07:22:59.547 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:59.543+0000 7fefa6d9d640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fefa815d6e0 0x7fefa80cc7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:22:59.548 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:59.543+0000 7fefa6d9d640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fefa815d6e0 0x7fefa80cc7f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:42832/0 (socket says 172.21.15.143:42832) 2023-11-26T07:22:59.548 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:59.543+0000 7fefa6d9d640 1 -- 172.21.15.143:0/1018383199 learned_addr learned my addr 172.21.15.143:0/1018383199 (peer_addr_for_me v2:172.21.15.143:0/0) 2023-11-26T07:22:59.548 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:59.543+0000 7fefa659c640 1 --2- 172.21.15.143:0/1018383199 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fefa80b2160 0x7fefa8155020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:22:59.548 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:59.543+0000 7fef9ffff640 1 --2- 172.21.15.143:0/1018383199 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fefa80d19f0 0x7fefa80cc2b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:22:59.548 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:59.543+0000 7fefa6d9d640 1 -- 172.21.15.143:0/1018383199 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fefa80d19f0 msgr2=0x7fefa80cc2b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:22:59.548 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:59.543+0000 7fefa6d9d640 1 --2- 172.21.15.143:0/1018383199 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fefa80d19f0 0x7fefa80cc2b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:22:59.548 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:59.543+0000 7fefa6d9d640 1 -- 172.21.15.143:0/1018383199 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fefa80b2160 msgr2=0x7fefa8155020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:22:59.548 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:59.543+0000 7fefa6d9d640 1 --2- 172.21.15.143:0/1018383199 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fefa80b2160 0x7fefa8155020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:22:59.548 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:59.543+0000 7fefa6d9d640 1 -- 172.21.15.143:0/1018383199 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fef900094d0 con 0x7fefa815d6e0 2023-11-26T07:22:59.548 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:59.543+0000 7fefa6d9d640 1 --2- 172.21.15.143:0/1018383199 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fefa815d6e0 0x7fefa80cc7f0 secure :-1 s=READY pgs=1022 cs=0 l=1 rev1=1 crypto rx=0x7fefa000c770 tx=0x7fefa000cc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:22:59.549 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:59.543+0000 7fef9effd640 1 -- 172.21.15.143:0/1018383199 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fefa0003b80 con 0x7fefa815d6e0 2023-11-26T07:22:59.549 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:59.543+0000 7fef9effd640 1 -- 172.21.15.143:0/1018383199 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fefa0003ce0 con 0x7fefa815d6e0 2023-11-26T07:22:59.549 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:59.543+0000 7fef9effd640 1 -- 172.21.15.143:0/1018383199 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fefa000aa90 con 0x7fefa815d6e0 2023-11-26T07:22:59.549 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:59.543+0000 7fefacde7640 1 -- 172.21.15.143:0/1018383199 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fefa8158310 con 0x7fefa815d6e0 2023-11-26T07:22:59.549 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:59.543+0000 7fefacde7640 1 -- 172.21.15.143:0/1018383199 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fefa8155560 con 0x7fefa815d6e0 2023-11-26T07:22:59.549 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:59.547+0000 7fefacde7640 1 -- 172.21.15.143:0/1018383199 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fefa8155750 con 0x7fefa815d6e0 2023-11-26T07:22:59.550 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:59.547+0000 7fef9effd640 1 -- 172.21.15.143:0/1018383199 <== mon.0 v2:172.21.15.143:3300/0 4 ==== mgrmap(e 11) v1 ==== 86158+0+0 (secure 0 0 0) 0x7fefa000abf0 con 0x7fefa815d6e0 2023-11-26T07:22:59.550 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:59.547+0000 7fef9effd640 1 --2- 172.21.15.143:0/1018383199 >> [v2:172.21.15.143:6832/2150557544,v1:172.21.15.143:6833/2150557544] conn(0x7fef880667d0 0x7fef88068c90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:22:59.550 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:59.547+0000 7fef9effd640 1 -- 172.21.15.143:0/1018383199 <== mon.0 v2:172.21.15.143:3300/0 5 ==== osd_map(71..71 src has 1..71) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fefa00841f0 con 0x7fefa815d6e0 2023-11-26T07:22:59.550 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:59.547+0000 7fefa659c640 1 --2- 172.21.15.143:0/1018383199 >> [v2:172.21.15.143:6832/2150557544,v1:172.21.15.143:6833/2150557544] conn(0x7fef880667d0 0x7fef88068c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:22:59.551 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:59.547+0000 7fefa659c640 1 --2- 172.21.15.143:0/1018383199 >> [v2:172.21.15.143:6832/2150557544,v1:172.21.15.143:6833/2150557544] conn(0x7fef880667d0 0x7fef88068c90 secure :-1 s=READY pgs=474 cs=0 l=1 rev1=1 crypto rx=0x7fef980079c0 tx=0x7fef98008040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:22:59.554 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:59.551+0000 7fef9effd640 1 -- 172.21.15.143:0/1018383199 <== mon.0 v2:172.21.15.143:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172649 (secure 0 0 0) 0x7fefa00511d0 con 0x7fefa815d6e0 2023-11-26T07:22:59.703 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:59.699+0000 7fefacde7640 1 -- 172.21.15.143:0/1018383199 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7fefa8158120 con 0x7fefa815d6e0 2023-11-26T07:22:59.703 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:59.699+0000 7fef9effd640 1 -- 172.21.15.143:0/1018383199 <== mon.0 v2:172.21.15.143:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v71) v1 ==== 104+0+11 (secure 0 0 0) 0x7fefa00547f0 con 0x7fefa815d6e0 2023-11-26T07:22:59.703 INFO:teuthology.orchestra.run.smithi143.stdout:pg_num: 32 2023-11-26T07:22:59.705 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:59.703+0000 7fefacde7640 1 -- 172.21.15.143:0/1018383199 >> [v2:172.21.15.143:6832/2150557544,v1:172.21.15.143:6833/2150557544] conn(0x7fef880667d0 msgr2=0x7fef88068c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:22:59.705 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:59.703+0000 7fefacde7640 1 --2- 172.21.15.143:0/1018383199 >> [v2:172.21.15.143:6832/2150557544,v1:172.21.15.143:6833/2150557544] conn(0x7fef880667d0 0x7fef88068c90 secure :-1 s=READY pgs=474 cs=0 l=1 rev1=1 crypto rx=0x7fef980079c0 tx=0x7fef98008040 comp rx=0 tx=0).stop 2023-11-26T07:22:59.705 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:59.703+0000 7fefacde7640 1 -- 172.21.15.143:0/1018383199 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fefa815d6e0 msgr2=0x7fefa80cc7f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:22:59.705 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:59.703+0000 7fefacde7640 1 --2- 172.21.15.143:0/1018383199 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fefa815d6e0 0x7fefa80cc7f0 secure :-1 s=READY pgs=1022 cs=0 l=1 rev1=1 crypto rx=0x7fefa000c770 tx=0x7fefa000cc30 comp rx=0 tx=0).stop 2023-11-26T07:22:59.705 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:59.703+0000 7fefacde7640 1 -- 172.21.15.143:0/1018383199 shutdown_connections 2023-11-26T07:22:59.705 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:59.703+0000 7fefacde7640 1 --2- 172.21.15.143:0/1018383199 >> [v2:172.21.15.143:6832/2150557544,v1:172.21.15.143:6833/2150557544] conn(0x7fef880667d0 0x7fef88068c90 unknown :-1 s=CLOSED pgs=474 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:22:59.706 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:59.703+0000 7fefacde7640 1 --2- 172.21.15.143:0/1018383199 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fefa815d6e0 0x7fefa80cc7f0 unknown :-1 s=CLOSED pgs=1022 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:22:59.706 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:59.703+0000 7fefacde7640 1 --2- 172.21.15.143:0/1018383199 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fefa80d19f0 0x7fefa80cc2b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:22:59.706 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:59.703+0000 7fefacde7640 1 --2- 172.21.15.143:0/1018383199 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fefa80b2160 0x7fefa8155020 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:22:59.706 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:59.703+0000 7fefacde7640 1 -- 172.21.15.143:0/1018383199 >> 172.21.15.143:0/1018383199 conn(0x7fefa809c550 msgr2=0x7fefa80cfbe0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:22:59.706 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:59.703+0000 7fefacde7640 1 -- 172.21.15.143:0/1018383199 shutdown_connections 2023-11-26T07:22:59.706 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:59.703+0000 7fefacde7640 1 -- 172.21.15.143:0/1018383199 wait complete. 2023-11-26T07:22:59.724 DEBUG:teuthology.orchestra.run.smithi143:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-11-26T07:22:59.865 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:59.863+0000 7ff29e69a640 1 Processor -- start 2023-11-26T07:22:59.866 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:59.863+0000 7ff29e69a640 1 -- start start 2023-11-26T07:22:59.868 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:59.863+0000 7ff29e69a640 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7ff2980d1bc0 0x7ff2980d0df0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:22:59.868 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:59.863+0000 7ff29e69a640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7ff2980b2160 0x7ff2980b0750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:22:59.868 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:59.863+0000 7ff29e69a640 1 -- --> v1:172.21.15.143:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff2980cf0e0 con 0x7ff29815d6e0 2023-11-26T07:22:59.868 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:59.863+0000 7ff29e69a640 1 -- --> v2:172.21.15.143:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff29815dac0 con 0x7ff2980d1bc0 2023-11-26T07:22:59.868 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:59.863+0000 7ff29e69a640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff2980ceb20 con 0x7ff2980b2160 2023-11-26T07:22:59.868 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:59.867+0000 7ff2977fe640 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7ff2980d1bc0 0x7ff2980d0df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:22:59.868 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:59.867+0000 7ff297fff640 1 --1- >> v1:172.21.15.143:6789/0 conn(0x7ff29815d6e0 0x7ff2980d14b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.143:6789/0 says I am v1:172.21.15.143:57886/0 (socket says 172.21.15.143:57886) 2023-11-26T07:22:59.868 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:59.867+0000 7ff297fff640 1 -- 172.21.15.143:0/188870170 learned_addr learned my addr 172.21.15.143:0/188870170 (peer_addr_for_me v1:172.21.15.143:0/0) 2023-11-26T07:22:59.869 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:59.867+0000 7ff29cc37640 1 --2- 172.21.15.143:0/188870170 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7ff2980b2160 0x7ff2980b0750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:22:59.869 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:59.867+0000 7ff2977fe640 1 -- 172.21.15.143:0/188870170 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7ff2980b2160 msgr2=0x7ff2980b0750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-26T07:22:59.869 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:59.867+0000 7ff2977fe640 1 --2- 172.21.15.143:0/188870170 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7ff2980b2160 0x7ff2980b0750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:22:59.869 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:59.867+0000 7ff2977fe640 1 -- 172.21.15.143:0/188870170 >> v1:172.21.15.143:6789/0 conn(0x7ff29815d6e0 legacy=0x7ff2980d14b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:22:59.869 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:59.867+0000 7ff2977fe640 1 -- 172.21.15.143:0/188870170 --> v2:172.21.15.143:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff2980cee00 con 0x7ff2980d1bc0 2023-11-26T07:22:59.869 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:59.867+0000 7ff296ffd640 1 -- 172.21.15.143:0/188870170 <== mon.0 v1:172.21.15.143:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3397718110 0 0) 0x7ff2980cf0e0 con 0x7ff29815d6e0 2023-11-26T07:22:59.870 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:59.867+0000 7ff2977fe640 1 --2- 172.21.15.143:0/188870170 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7ff2980d1bc0 0x7ff2980d0df0 secure :-1 s=READY pgs=1024 cs=0 l=1 rev1=1 crypto rx=0x7ff28c0097c0 tx=0x7ff28c01c480 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f790f48d3d0ee97b server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:22:59.871 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:59.867+0000 7ff296ffd640 1 -- 172.21.15.143:0/188870170 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff28c0ab600 con 0x7ff2980d1bc0 2023-11-26T07:22:59.871 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:59.867+0000 7ff296ffd640 1 -- 172.21.15.143:0/188870170 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff28c0a7050 con 0x7ff2980d1bc0 2023-11-26T07:22:59.871 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:59.867+0000 7ff296ffd640 1 -- 172.21.15.143:0/188870170 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff28c0ab980 con 0x7ff2980d1bc0 2023-11-26T07:22:59.872 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:59.871+0000 7ff29e69a640 1 -- 172.21.15.143:0/188870170 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7ff2980d1bc0 msgr2=0x7ff2980d0df0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:22:59.872 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:59.871+0000 7ff29e69a640 1 --2- 172.21.15.143:0/188870170 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7ff2980d1bc0 0x7ff2980d0df0 secure :-1 s=READY pgs=1024 cs=0 l=1 rev1=1 crypto rx=0x7ff28c0097c0 tx=0x7ff28c01c480 comp rx=0 tx=0).stop 2023-11-26T07:22:59.872 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:59.871+0000 7ff29e69a640 1 -- 172.21.15.143:0/188870170 shutdown_connections 2023-11-26T07:22:59.872 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:59.871+0000 7ff29e69a640 1 --2- 172.21.15.143:0/188870170 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7ff2980b2160 0x7ff2980b0750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:22:59.872 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:59.871+0000 7ff29e69a640 1 --2- 172.21.15.143:0/188870170 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7ff2980d1bc0 0x7ff2980d0df0 unknown :-1 s=CLOSED pgs=1024 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:22:59.872 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:59.871+0000 7ff29e69a640 1 -- 172.21.15.143:0/188870170 >> 172.21.15.143:0/188870170 conn(0x7ff29809c550 msgr2=0x7ff2980d3220 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:22:59.872 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:59.871+0000 7ff29e69a640 1 -- 172.21.15.143:0/188870170 shutdown_connections 2023-11-26T07:22:59.872 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:59.871+0000 7ff29e69a640 1 -- 172.21.15.143:0/188870170 wait complete. 2023-11-26T07:22:59.873 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:59.871+0000 7ff29e69a640 1 Processor -- start 2023-11-26T07:22:59.873 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:59.871+0000 7ff29e69a640 1 -- start start 2023-11-26T07:22:59.875 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:59.871+0000 7ff29e69a640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7ff2980b2160 0x7ff298158800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:22:59.875 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:59.871+0000 7ff29e69a640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7ff2980d1bc0 0x7ff2980b3cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:22:59.876 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:59.871+0000 7ff29e69a640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7ff29815d6e0 0x7ff2980b4200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:22:59.876 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:59.871+0000 7ff29e69a640 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff298158d40 con 0x7ff2980b2160 2023-11-26T07:22:59.876 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:59.871+0000 7ff29e69a640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff29815f520 con 0x7ff29815d6e0 2023-11-26T07:22:59.876 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:59.871+0000 7ff29e69a640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff2980ca6b0 con 0x7ff2980d1bc0 2023-11-26T07:22:59.876 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:59.871+0000 7ff297fff640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7ff2980b2160 0x7ff298158800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:22:59.876 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:59.871+0000 7ff297fff640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7ff2980b2160 0x7ff298158800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:42856/0 (socket says 172.21.15.143:42856) 2023-11-26T07:22:59.876 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:59.871+0000 7ff297fff640 1 -- 172.21.15.143:0/2458160225 learned_addr learned my addr 172.21.15.143:0/2458160225 (peer_addr_for_me v2:172.21.15.143:0/0) 2023-11-26T07:22:59.876 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:59.871+0000 7ff297fff640 1 -- 172.21.15.143:0/2458160225 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7ff2980d1bc0 msgr2=0x7ff2980b3cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:22:59.876 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:59.871+0000 7ff297fff640 1 --2- 172.21.15.143:0/2458160225 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7ff2980d1bc0 0x7ff2980b3cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:22:59.876 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:59.871+0000 7ff297fff640 1 -- 172.21.15.143:0/2458160225 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7ff29815d6e0 msgr2=0x7ff2980b4200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:22:59.877 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:59.871+0000 7ff29cc37640 1 --2- 172.21.15.143:0/2458160225 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7ff29815d6e0 0x7ff2980b4200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:22:59.877 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:59.871+0000 7ff297fff640 1 --2- 172.21.15.143:0/2458160225 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7ff29815d6e0 0x7ff2980b4200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:22:59.877 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:59.875+0000 7ff297fff640 1 -- 172.21.15.143:0/2458160225 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff28c009420 con 0x7ff2980b2160 2023-11-26T07:22:59.877 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:59.875+0000 7ff29cc37640 1 --2- 172.21.15.143:0/2458160225 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7ff29815d6e0 0x7ff2980b4200 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-26T07:22:59.877 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:59.875+0000 7ff297fff640 1 --2- 172.21.15.143:0/2458160225 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7ff2980b2160 0x7ff298158800 secure :-1 s=READY pgs=1025 cs=0 l=1 rev1=1 crypto rx=0x7ff28800c4b0 tx=0x7ff28800c970 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:22:59.877 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:59.875+0000 7ff294ff9640 1 -- 172.21.15.143:0/2458160225 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff2880074e0 con 0x7ff2980b2160 2023-11-26T07:22:59.877 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:59.875+0000 7ff294ff9640 1 -- 172.21.15.143:0/2458160225 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff288007660 con 0x7ff2980b2160 2023-11-26T07:22:59.877 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:59.875+0000 7ff294ff9640 1 -- 172.21.15.143:0/2458160225 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff288004a30 con 0x7ff2980b2160 2023-11-26T07:22:59.877 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:59.875+0000 7ff29e69a640 1 -- 172.21.15.143:0/2458160225 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff2980ca990 con 0x7ff2980b2160 2023-11-26T07:22:59.878 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:59.875+0000 7ff29e69a640 1 -- 172.21.15.143:0/2458160225 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff2980caf50 con 0x7ff2980b2160 2023-11-26T07:22:59.878 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:59.875+0000 7ff29e69a640 1 -- 172.21.15.143:0/2458160225 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff298067680 con 0x7ff2980b2160 2023-11-26T07:22:59.878 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:59.875+0000 7ff294ff9640 1 -- 172.21.15.143:0/2458160225 <== mon.0 v2:172.21.15.143:3300/0 4 ==== mgrmap(e 11) v1 ==== 86158+0+0 (secure 0 0 0) 0x7ff288004b90 con 0x7ff2980b2160 2023-11-26T07:22:59.879 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:59.875+0000 7ff294ff9640 1 --2- 172.21.15.143:0/2458160225 >> [v2:172.21.15.143:6832/2150557544,v1:172.21.15.143:6833/2150557544] conn(0x7ff2840667d0 0x7ff284068c90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:22:59.879 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:59.875+0000 7ff294ff9640 1 -- 172.21.15.143:0/2458160225 <== mon.0 v2:172.21.15.143:3300/0 5 ==== osd_map(71..71 src has 1..71) v4 ==== 5709+0+0 (secure 0 0 0) 0x7ff2880835f0 con 0x7ff2980b2160 2023-11-26T07:22:59.879 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:59.879+0000 7ff2977fe640 1 --2- 172.21.15.143:0/2458160225 >> [v2:172.21.15.143:6832/2150557544,v1:172.21.15.143:6833/2150557544] conn(0x7ff2840667d0 0x7ff284068c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:22:59.880 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:59.879+0000 7ff2977fe640 1 --2- 172.21.15.143:0/2458160225 >> [v2:172.21.15.143:6832/2150557544,v1:172.21.15.143:6833/2150557544] conn(0x7ff2840667d0 0x7ff284068c90 secure :-1 s=READY pgs=475 cs=0 l=1 rev1=1 crypto rx=0x7ff28c0ad000 tx=0x7ff28c004740 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:22:59.882 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:22:59.879+0000 7ff294ff9640 1 -- 172.21.15.143:0/2458160225 <== mon.0 v2:172.21.15.143:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172649 (secure 0 0 0) 0x7ff288054040 con 0x7ff2980b2160 2023-11-26T07:23:00.037 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:00.035+0000 7ff29e69a640 1 -- 172.21.15.143:0/2458160225 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7ff29806c270 con 0x7ff2980b2160 2023-11-26T07:23:00.037 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:00.035+0000 7ff294ff9640 1 -- 172.21.15.143:0/2458160225 <== mon.0 v2:172.21.15.143:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v71) v1 ==== 100+0+11 (secure 0 0 0) 0x7ff288018020 con 0x7ff2980b2160 2023-11-26T07:23:00.038 INFO:teuthology.orchestra.run.smithi143.stdout:pg_num: 64 2023-11-26T07:23:00.040 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:00.039+0000 7ff29e69a640 1 -- 172.21.15.143:0/2458160225 >> [v2:172.21.15.143:6832/2150557544,v1:172.21.15.143:6833/2150557544] conn(0x7ff2840667d0 msgr2=0x7ff284068c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:23:00.040 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:00.039+0000 7ff29e69a640 1 --2- 172.21.15.143:0/2458160225 >> [v2:172.21.15.143:6832/2150557544,v1:172.21.15.143:6833/2150557544] conn(0x7ff2840667d0 0x7ff284068c90 secure :-1 s=READY pgs=475 cs=0 l=1 rev1=1 crypto rx=0x7ff28c0ad000 tx=0x7ff28c004740 comp rx=0 tx=0).stop 2023-11-26T07:23:00.040 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:00.039+0000 7ff29e69a640 1 -- 172.21.15.143:0/2458160225 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7ff2980b2160 msgr2=0x7ff298158800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:23:00.040 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:00.039+0000 7ff29e69a640 1 --2- 172.21.15.143:0/2458160225 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7ff2980b2160 0x7ff298158800 secure :-1 s=READY pgs=1025 cs=0 l=1 rev1=1 crypto rx=0x7ff28800c4b0 tx=0x7ff28800c970 comp rx=0 tx=0).stop 2023-11-26T07:23:00.040 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:00.039+0000 7ff29e69a640 1 -- 172.21.15.143:0/2458160225 shutdown_connections 2023-11-26T07:23:00.041 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:00.039+0000 7ff29e69a640 1 --2- 172.21.15.143:0/2458160225 >> [v2:172.21.15.143:6832/2150557544,v1:172.21.15.143:6833/2150557544] conn(0x7ff2840667d0 0x7ff284068c90 unknown :-1 s=CLOSED pgs=475 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:23:00.041 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:00.039+0000 7ff29e69a640 1 --2- 172.21.15.143:0/2458160225 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7ff29815d6e0 0x7ff2980b4200 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:23:00.041 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:00.039+0000 7ff29e69a640 1 --2- 172.21.15.143:0/2458160225 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7ff2980d1bc0 0x7ff2980b3cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:23:00.041 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:00.039+0000 7ff29e69a640 1 --2- 172.21.15.143:0/2458160225 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7ff2980b2160 0x7ff298158800 unknown :-1 s=CLOSED pgs=1025 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:23:00.041 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:00.039+0000 7ff29e69a640 1 -- 172.21.15.143:0/2458160225 >> 172.21.15.143:0/2458160225 conn(0x7ff29809c550 msgr2=0x7ff298159c10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:23:00.041 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:00.039+0000 7ff29e69a640 1 -- 172.21.15.143:0/2458160225 shutdown_connections 2023-11-26T07:23:00.041 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:00.039+0000 7ff29e69a640 1 -- 172.21.15.143:0/2458160225 wait complete. 2023-11-26T07:23:00.059 DEBUG:teuthology.orchestra.run.smithi143:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2023-11-26T07:23:00.197 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:00.195+0000 7f13ca0e3640 1 Processor -- start 2023-11-26T07:23:00.197 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:00.195+0000 7f13ca0e3640 1 -- start start 2023-11-26T07:23:00.199 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:00.195+0000 7f13ca0e3640 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7f13c40cf990 0x7f13c415efa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:23:00.199 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:00.195+0000 7f13ca0e3640 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7f13c415f4e0 0x7f13c40cd7b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:23:00.199 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:00.195+0000 7f13ca0e3640 1 -- --> v1:172.21.15.143:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f13c406c520 con 0x7f13c413e290 2023-11-26T07:23:00.199 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:00.195+0000 7f13ca0e3640 1 -- --> v2:172.21.15.143:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f13c406c800 con 0x7f13c40cf990 2023-11-26T07:23:00.199 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:00.195+0000 7f13ca0e3640 1 -- --> v2:172.21.15.188:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f13c406cae0 con 0x7f13c415f4e0 2023-11-26T07:23:00.200 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:00.195+0000 7f13c2ffd640 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7f13c40cf990 0x7f13c415efa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:23:00.200 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:00.195+0000 7f13c37fe640 1 --1- >> v1:172.21.15.143:6789/0 conn(0x7f13c413e290 0x7f13c40cf280 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.143:6789/0 says I am v1:172.21.15.143:57896/0 (socket says 172.21.15.143:57896) 2023-11-26T07:23:00.200 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:00.195+0000 7f13c2ffd640 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7f13c40cf990 0x7f13c415efa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:42870/0 (socket says 172.21.15.143:42870) 2023-11-26T07:23:00.200 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:00.195+0000 7f13c37fe640 1 -- 172.21.15.143:0/3455806671 learned_addr learned my addr 172.21.15.143:0/3455806671 (peer_addr_for_me v1:172.21.15.143:0/0) 2023-11-26T07:23:00.200 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:00.199+0000 7f13c3fff640 1 --2- 172.21.15.143:0/3455806671 >> v2:172.21.15.188:3300/0 conn(0x7f13c415f4e0 0x7f13c40cd7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:23:00.200 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:00.199+0000 7f13c2ffd640 1 -- 172.21.15.143:0/3455806671 >> v2:172.21.15.188:3300/0 conn(0x7f13c415f4e0 msgr2=0x7f13c40cd7b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-26T07:23:00.200 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:00.199+0000 7f13c2ffd640 1 --2- 172.21.15.143:0/3455806671 >> v2:172.21.15.188:3300/0 conn(0x7f13c415f4e0 0x7f13c40cd7b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:23:00.200 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:00.199+0000 7f13c2ffd640 1 -- 172.21.15.143:0/3455806671 >> v1:172.21.15.143:6789/0 conn(0x7f13c413e290 legacy=0x7f13c40cf280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:23:00.200 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:00.199+0000 7f13c2ffd640 1 -- 172.21.15.143:0/3455806671 --> v2:172.21.15.143:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f13c406cdc0 con 0x7f13c40cf990 2023-11-26T07:23:00.202 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:00.199+0000 7f13c2ffd640 1 --2- 172.21.15.143:0/3455806671 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f13c40cf990 0x7f13c415efa0 secure :-1 s=READY pgs=1027 cs=0 l=1 rev1=1 crypto rx=0x7f13c4155cf0 tx=0x7f13b801c430 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d707db7ead9adcc5 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:23:00.202 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:00.199+0000 7f13c27fc640 1 -- 172.21.15.143:0/3455806671 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f13b80ab650 con 0x7f13c40cf990 2023-11-26T07:23:00.202 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:00.199+0000 7f13c27fc640 1 -- 172.21.15.143:0/3455806671 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f13b80ab7b0 con 0x7f13c40cf990 2023-11-26T07:23:00.202 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:00.199+0000 7f13c27fc640 1 -- 172.21.15.143:0/3455806671 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f13b80abb40 con 0x7f13c40cf990 2023-11-26T07:23:00.203 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:00.199+0000 7f13ca0e3640 1 -- 172.21.15.143:0/3455806671 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f13c40cf990 msgr2=0x7f13c415efa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:23:00.203 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:00.199+0000 7f13ca0e3640 1 --2- 172.21.15.143:0/3455806671 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f13c40cf990 0x7f13c415efa0 secure :-1 s=READY pgs=1027 cs=0 l=1 rev1=1 crypto rx=0x7f13c4155cf0 tx=0x7f13b801c430 comp rx=0 tx=0).stop 2023-11-26T07:23:00.203 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:00.203+0000 7f13ca0e3640 1 -- 172.21.15.143:0/3455806671 shutdown_connections 2023-11-26T07:23:00.204 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:00.203+0000 7f13ca0e3640 1 --2- 172.21.15.143:0/3455806671 >> v2:172.21.15.188:3300/0 conn(0x7f13c415f4e0 0x7f13c40cd7b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:23:00.204 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:00.203+0000 7f13ca0e3640 1 --2- 172.21.15.143:0/3455806671 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f13c40cf990 0x7f13c415efa0 unknown :-1 s=CLOSED pgs=1027 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:23:00.204 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:00.203+0000 7f13ca0e3640 1 -- 172.21.15.143:0/3455806671 >> 172.21.15.143:0/3455806671 conn(0x7f13c406a890 msgr2=0x7f13c406aca0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:23:00.204 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:00.203+0000 7f13ca0e3640 1 -- 172.21.15.143:0/3455806671 shutdown_connections 2023-11-26T07:23:00.204 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:00.203+0000 7f13ca0e3640 1 -- 172.21.15.143:0/3455806671 wait complete. 2023-11-26T07:23:00.204 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:00.203+0000 7f13ca0e3640 1 Processor -- start 2023-11-26T07:23:00.205 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:00.203+0000 7f13ca0e3640 1 -- start start 2023-11-26T07:23:00.206 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:00.203+0000 7f13ca0e3640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f13c40cf990 0x7f13c412a5b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:23:00.207 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:00.203+0000 7f13ca0e3640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f13c413e290 0x7f13c413c720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:23:00.207 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:00.203+0000 7f13ca0e3640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f13c415f4e0 0x7f13c413cc60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:23:00.207 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:00.203+0000 7f13ca0e3640 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f13c415d1d0 con 0x7f13c40cf990 2023-11-26T07:23:00.207 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:00.203+0000 7f13ca0e3640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f13c415c5c0 con 0x7f13c415f4e0 2023-11-26T07:23:00.207 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:00.203+0000 7f13ca0e3640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f13c415c8a0 con 0x7f13c413e290 2023-11-26T07:23:00.207 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:00.203+0000 7f13c37fe640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f13c40cf990 0x7f13c412a5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:23:00.207 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:00.203+0000 7f13c37fe640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f13c40cf990 0x7f13c412a5b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:42872/0 (socket says 172.21.15.143:42872) 2023-11-26T07:23:00.207 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:00.203+0000 7f13c37fe640 1 -- 172.21.15.143:0/3855727294 learned_addr learned my addr 172.21.15.143:0/3855727294 (peer_addr_for_me v2:172.21.15.143:0/0) 2023-11-26T07:23:00.207 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:00.203+0000 7f13c3fff640 1 --2- 172.21.15.143:0/3855727294 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f13c415f4e0 0x7f13c413cc60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:23:00.207 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:00.203+0000 7f13c2ffd640 1 --2- 172.21.15.143:0/3855727294 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f13c413e290 0x7f13c413c720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:23:00.207 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:00.203+0000 7f13c37fe640 1 -- 172.21.15.143:0/3855727294 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f13c413e290 msgr2=0x7f13c413c720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:23:00.208 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:00.203+0000 7f13c37fe640 1 --2- 172.21.15.143:0/3855727294 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f13c413e290 0x7f13c413c720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:23:00.208 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:00.203+0000 7f13c37fe640 1 -- 172.21.15.143:0/3855727294 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f13c415f4e0 msgr2=0x7f13c413cc60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:23:00.208 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:00.203+0000 7f13c37fe640 1 --2- 172.21.15.143:0/3855727294 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f13c415f4e0 0x7f13c413cc60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:23:00.208 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:00.203+0000 7f13c37fe640 1 -- 172.21.15.143:0/3855727294 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f13b8009420 con 0x7f13c40cf990 2023-11-26T07:23:00.208 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:00.203+0000 7f13c37fe640 1 --2- 172.21.15.143:0/3855727294 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f13c40cf990 0x7f13c412a5b0 secure :-1 s=READY pgs=1028 cs=0 l=1 rev1=1 crypto rx=0x7f13b400c500 tx=0x7f13b400c9c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:23:00.209 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:00.207+0000 7f13abfff640 1 -- 172.21.15.143:0/3855727294 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f13b4005cf0 con 0x7f13c40cf990 2023-11-26T07:23:00.209 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:00.207+0000 7f13ca0e3640 1 -- 172.21.15.143:0/3855727294 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f13c415cb80 con 0x7f13c40cf990 2023-11-26T07:23:00.209 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:00.207+0000 7f13abfff640 1 -- 172.21.15.143:0/3855727294 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f13b4005e50 con 0x7f13c40cf990 2023-11-26T07:23:00.209 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:00.207+0000 7f13abfff640 1 -- 172.21.15.143:0/3855727294 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f13b40072f0 con 0x7f13c40cf990 2023-11-26T07:23:00.209 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:00.207+0000 7f13ca0e3640 1 -- 172.21.15.143:0/3855727294 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f13c415d660 con 0x7f13c40cf990 2023-11-26T07:23:00.209 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:00.207+0000 7f13ca0e3640 1 -- 172.21.15.143:0/3855727294 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f13c4067680 con 0x7f13c40cf990 2023-11-26T07:23:00.209 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:00.207+0000 7f13abfff640 1 -- 172.21.15.143:0/3855727294 <== mon.0 v2:172.21.15.143:3300/0 4 ==== mgrmap(e 11) v1 ==== 86158+0+0 (secure 0 0 0) 0x7f13b4007450 con 0x7f13c40cf990 2023-11-26T07:23:00.210 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:00.207+0000 7f13abfff640 1 --2- 172.21.15.143:0/3855727294 >> [v2:172.21.15.143:6832/2150557544,v1:172.21.15.143:6833/2150557544] conn(0x7f13b00667d0 0x7f13b0068c90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:23:00.210 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:00.207+0000 7f13abfff640 1 -- 172.21.15.143:0/3855727294 <== mon.0 v2:172.21.15.143:3300/0 5 ==== osd_map(71..71 src has 1..71) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f13b40839a0 con 0x7f13c40cf990 2023-11-26T07:23:00.210 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:00.207+0000 7f13c2ffd640 1 --2- 172.21.15.143:0/3855727294 >> [v2:172.21.15.143:6832/2150557544,v1:172.21.15.143:6833/2150557544] conn(0x7f13b00667d0 0x7f13b0068c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:23:00.211 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:00.207+0000 7f13c2ffd640 1 --2- 172.21.15.143:0/3855727294 >> [v2:172.21.15.143:6832/2150557544,v1:172.21.15.143:6833/2150557544] conn(0x7f13b00667d0 0x7f13b0068c90 secure :-1 s=READY pgs=476 cs=0 l=1 rev1=1 crypto rx=0x7f13b80af000 tx=0x7f13b80a7000 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:23:00.214 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:00.211+0000 7f13abfff640 1 -- 172.21.15.143:0/3855727294 <== mon.0 v2:172.21.15.143:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172649 (secure 0 0 0) 0x7f13b40509d0 con 0x7f13c40cf990 2023-11-26T07:23:00.376 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:00.375+0000 7f13ca0e3640 1 -- 172.21.15.143:0/3855727294 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) v1 -- 0x7f13c406df50 con 0x7f13c40cf990 2023-11-26T07:23:00.376 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:00.375+0000 7f13abfff640 1 -- 172.21.15.143:0/3855727294 <== mon.0 v2:172.21.15.143:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v71) v1 ==== 80+0+0 (secure 0 0 0) 0x7f13b4017080 con 0x7f13c40cf990 2023-11-26T07:23:00.376 INFO:teuthology.orchestra.run.smithi143.stderr:listed 0 entries 2023-11-26T07:23:00.378 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:00.375+0000 7f13ca0e3640 1 -- 172.21.15.143:0/3855727294 >> [v2:172.21.15.143:6832/2150557544,v1:172.21.15.143:6833/2150557544] conn(0x7f13b00667d0 msgr2=0x7f13b0068c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:23:00.378 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:00.375+0000 7f13ca0e3640 1 --2- 172.21.15.143:0/3855727294 >> [v2:172.21.15.143:6832/2150557544,v1:172.21.15.143:6833/2150557544] conn(0x7f13b00667d0 0x7f13b0068c90 secure :-1 s=READY pgs=476 cs=0 l=1 rev1=1 crypto rx=0x7f13b80af000 tx=0x7f13b80a7000 comp rx=0 tx=0).stop 2023-11-26T07:23:00.378 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:00.375+0000 7f13ca0e3640 1 -- 172.21.15.143:0/3855727294 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f13c40cf990 msgr2=0x7f13c412a5b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:23:00.379 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:00.375+0000 7f13ca0e3640 1 --2- 172.21.15.143:0/3855727294 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f13c40cf990 0x7f13c412a5b0 secure :-1 s=READY pgs=1028 cs=0 l=1 rev1=1 crypto rx=0x7f13b400c500 tx=0x7f13b400c9c0 comp rx=0 tx=0).stop 2023-11-26T07:23:00.379 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:00.375+0000 7f13ca0e3640 1 -- 172.21.15.143:0/3855727294 shutdown_connections 2023-11-26T07:23:00.379 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:00.375+0000 7f13ca0e3640 1 --2- 172.21.15.143:0/3855727294 >> [v2:172.21.15.143:6832/2150557544,v1:172.21.15.143:6833/2150557544] conn(0x7f13b00667d0 0x7f13b0068c90 unknown :-1 s=CLOSED pgs=476 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:23:00.379 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:00.375+0000 7f13ca0e3640 1 --2- 172.21.15.143:0/3855727294 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f13c415f4e0 0x7f13c413cc60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:23:00.379 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:00.375+0000 7f13ca0e3640 1 --2- 172.21.15.143:0/3855727294 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f13c413e290 0x7f13c413c720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:23:00.379 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:00.375+0000 7f13ca0e3640 1 --2- 172.21.15.143:0/3855727294 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f13c40cf990 0x7f13c412a5b0 unknown :-1 s=CLOSED pgs=1028 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:23:00.379 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:00.375+0000 7f13ca0e3640 1 -- 172.21.15.143:0/3855727294 >> 172.21.15.143:0/3855727294 conn(0x7f13c406a890 msgr2=0x7f13c406ac70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:23:00.379 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:00.375+0000 7f13ca0e3640 1 -- 172.21.15.143:0/3855727294 shutdown_connections 2023-11-26T07:23:00.379 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:00.375+0000 7f13ca0e3640 1 -- 172.21.15.143:0/3855727294 wait complete. 2023-11-26T07:23:00.398 DEBUG:teuthology.orchestra.run.smithi188:> set -ex 2023-11-26T07:23:00.398 DEBUG:teuthology.orchestra.run.smithi188:> dd if=/proc/self/mounts of=/dev/stdout 2023-11-26T07:23:00.407 DEBUG:teuthology.orchestra.run.smithi143:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-11-26T07:23:00.547 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:00.543+0000 7f74b345a640 1 Processor -- start 2023-11-26T07:23:00.547 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:00.543+0000 7f74b345a640 1 -- start start 2023-11-26T07:23:00.549 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:00.547+0000 7f74b345a640 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7f74ac0ce7c0 0x7f74ac15f6b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:23:00.549 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:00.547+0000 7f74b345a640 1 -- --> v1:172.21.15.143:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f74ac06c520 con 0x7f74ac0cf990 2023-11-26T07:23:00.549 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:00.547+0000 7f74b345a640 1 -- --> v1:172.21.15.188:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f74ac06c800 con 0x7f74ac13e290 2023-11-26T07:23:00.550 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:00.547+0000 7f74b345a640 1 -- --> v2:172.21.15.188:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f74ac06cae0 con 0x7f74ac0ce7c0 2023-11-26T07:23:00.550 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:00.547+0000 7f74b09f5640 1 --1- >> v1:172.21.15.143:6789/0 conn(0x7f74ac0cf990 0x7f74ac15efa0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.143:6789/0 says I am v1:172.21.15.143:57898/0 (socket says 172.21.15.143:57898) 2023-11-26T07:23:00.550 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:00.547+0000 7f74b09f5640 1 -- 172.21.15.143:0/3613986534 learned_addr learned my addr 172.21.15.143:0/3613986534 (peer_addr_for_me v1:172.21.15.143:0/0) 2023-11-26T07:23:00.550 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:00.547+0000 7f74b19f7640 1 --2- 172.21.15.143:0/3613986534 >> v2:172.21.15.188:3300/0 conn(0x7f74ac0ce7c0 0x7f74ac15f6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:23:00.550 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:00.547+0000 7f74a3fff640 1 -- 172.21.15.143:0/3613986534 <== mon.0 v1:172.21.15.143:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1947349969 0 0) 0x7f74ac06c520 con 0x7f74ac0cf990 2023-11-26T07:23:00.551 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:00.547+0000 7f74a3fff640 1 -- 172.21.15.143:0/3613986534 --> v1:172.21.15.143:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f7498003530 con 0x7f74ac0cf990 2023-11-26T07:23:00.551 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:00.547+0000 7f74b19f7640 1 -- 172.21.15.143:0/3613986534 >> v1:172.21.15.188:6789/0 conn(0x7f74ac13e290 legacy=0x7f74ac0cf280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:23:00.551 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:00.547+0000 7f74b19f7640 1 -- 172.21.15.143:0/3613986534 >> v1:172.21.15.143:6789/0 conn(0x7f74ac0cf990 legacy=0x7f74ac15efa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:23:00.551 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:00.547+0000 7f74b19f7640 1 -- 172.21.15.143:0/3613986534 --> v2:172.21.15.188:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f74ac06cdc0 con 0x7f74ac0ce7c0 2023-11-26T07:23:00.552 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:00.551+0000 7f74b19f7640 1 --2- 172.21.15.143:0/3613986534 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f74ac0ce7c0 0x7f74ac15f6b0 secure :-1 s=READY pgs=405 cs=0 l=1 rev1=1 crypto rx=0x7f74a4009d90 tx=0x7f74a401c890 comp rx=0 tx=0).ready entity=mon.1 client_cookie=8332d04aed48a727 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:23:00.553 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:00.551+0000 7f74a3fff640 1 -- 172.21.15.143:0/3613986534 <== mon.1 v2:172.21.15.188:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f74a40a5970 con 0x7f74ac0ce7c0 2023-11-26T07:23:00.553 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:00.551+0000 7f74a3fff640 1 -- 172.21.15.143:0/3613986534 <== mon.1 v2:172.21.15.188:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f74a40a5ad0 con 0x7f74ac0ce7c0 2023-11-26T07:23:00.553 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:00.551+0000 7f74a3fff640 1 -- 172.21.15.143:0/3613986534 <== mon.1 v2:172.21.15.188:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f74a40a5e60 con 0x7f74ac0ce7c0 2023-11-26T07:23:00.554 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:00.551+0000 7f74b345a640 1 -- 172.21.15.143:0/3613986534 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f74ac0ce7c0 msgr2=0x7f74ac15f6b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:23:00.554 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:00.551+0000 7f74b345a640 1 --2- 172.21.15.143:0/3613986534 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f74ac0ce7c0 0x7f74ac15f6b0 secure :-1 s=READY pgs=405 cs=0 l=1 rev1=1 crypto rx=0x7f74a4009d90 tx=0x7f74a401c890 comp rx=0 tx=0).stop 2023-11-26T07:23:00.554 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:00.551+0000 7f74b345a640 1 -- 172.21.15.143:0/3613986534 shutdown_connections 2023-11-26T07:23:00.554 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:00.551+0000 7f74b345a640 1 --2- 172.21.15.143:0/3613986534 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f74ac0ce7c0 0x7f74ac15f6b0 unknown :-1 s=CLOSED pgs=405 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:23:00.554 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:00.551+0000 7f74b345a640 1 -- 172.21.15.143:0/3613986534 >> 172.21.15.143:0/3613986534 conn(0x7f74ac06a890 msgr2=0x7f74ac06aca0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:23:00.555 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:00.551+0000 7f74b345a640 1 -- 172.21.15.143:0/3613986534 shutdown_connections 2023-11-26T07:23:00.555 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:00.551+0000 7f74b345a640 1 -- 172.21.15.143:0/3613986534 wait complete. 2023-11-26T07:23:00.555 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:00.555+0000 7f74b345a640 1 Processor -- start 2023-11-26T07:23:00.555 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:00.555+0000 7f74b345a640 1 -- start start 2023-11-26T07:23:00.557 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:00.555+0000 7f74b345a640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f74ac0ce7c0 0x7f74ac151d60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:23:00.557 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:00.555+0000 7f74b345a640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f74ac0cf990 0x7f74ac0b0d70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:23:00.557 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:00.555+0000 7f74b345a640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f74ac13e290 0x7f74ac137790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:23:00.557 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:00.555+0000 7f74b345a640 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f74ac15cdb0 con 0x7f74ac0ce7c0 2023-11-26T07:23:00.558 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:00.555+0000 7f74b345a640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f74ac15c130 con 0x7f74ac13e290 2023-11-26T07:23:00.558 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:00.555+0000 7f74b345a640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f74ac15c410 con 0x7f74ac0cf990 2023-11-26T07:23:00.558 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:00.555+0000 7f74b11f6640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f74ac0ce7c0 0x7f74ac151d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:23:00.558 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:00.555+0000 7f74b11f6640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f74ac0ce7c0 0x7f74ac151d60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:42876/0 (socket says 172.21.15.143:42876) 2023-11-26T07:23:00.558 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:00.555+0000 7f74b11f6640 1 -- 172.21.15.143:0/1880182461 learned_addr learned my addr 172.21.15.143:0/1880182461 (peer_addr_for_me v2:172.21.15.143:0/0) 2023-11-26T07:23:00.558 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:00.555+0000 7f74b09f5640 1 --2- 172.21.15.143:0/1880182461 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f74ac0cf990 0x7f74ac0b0d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:23:00.559 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:00.555+0000 7f74b19f7640 1 --2- 172.21.15.143:0/1880182461 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f74ac13e290 0x7f74ac137790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:23:00.559 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:00.555+0000 7f74b11f6640 1 -- 172.21.15.143:0/1880182461 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f74ac0cf990 msgr2=0x7f74ac0b0d70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:23:00.559 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:00.555+0000 7f74b11f6640 1 --2- 172.21.15.143:0/1880182461 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f74ac0cf990 0x7f74ac0b0d70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:23:00.559 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:00.555+0000 7f74b11f6640 1 -- 172.21.15.143:0/1880182461 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f74ac13e290 msgr2=0x7f74ac137790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:23:00.559 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:00.555+0000 7f74b11f6640 1 --2- 172.21.15.143:0/1880182461 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f74ac13e290 0x7f74ac137790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:23:00.559 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:00.555+0000 7f74b11f6640 1 -- 172.21.15.143:0/1880182461 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f74a4009a70 con 0x7f74ac0ce7c0 2023-11-26T07:23:00.559 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:00.555+0000 7f74b19f7640 1 --2- 172.21.15.143:0/1880182461 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f74ac13e290 0x7f74ac137790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-26T07:23:00.559 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:00.555+0000 7f74b11f6640 1 --2- 172.21.15.143:0/1880182461 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f74ac0ce7c0 0x7f74ac151d60 secure :-1 s=READY pgs=1030 cs=0 l=1 rev1=1 crypto rx=0x7f74a800c3c0 tx=0x7f74a800c880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:23:00.561 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:00.555+0000 7f74a1ffb640 1 -- 172.21.15.143:0/1880182461 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f74a80052d0 con 0x7f74ac0ce7c0 2023-11-26T07:23:00.561 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:00.555+0000 7f74a1ffb640 1 -- 172.21.15.143:0/1880182461 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f74a8005430 con 0x7f74ac0ce7c0 2023-11-26T07:23:00.561 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:00.555+0000 7f74b345a640 1 -- 172.21.15.143:0/1880182461 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f74ac15c6f0 con 0x7f74ac0ce7c0 2023-11-26T07:23:00.561 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:00.555+0000 7f74a1ffb640 1 -- 172.21.15.143:0/1880182461 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f74a8003840 con 0x7f74ac0ce7c0 2023-11-26T07:23:00.561 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:00.555+0000 7f74b345a640 1 -- 172.21.15.143:0/1880182461 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f74ac15bd70 con 0x7f74ac0ce7c0 2023-11-26T07:23:00.561 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:00.555+0000 7f74b345a640 1 -- 172.21.15.143:0/1880182461 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f74ac067680 con 0x7f74ac0ce7c0 2023-11-26T07:23:00.562 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:00.559+0000 7f74a1ffb640 1 -- 172.21.15.143:0/1880182461 <== mon.0 v2:172.21.15.143:3300/0 4 ==== mgrmap(e 11) v1 ==== 86158+0+0 (secure 0 0 0) 0x7f74a80039a0 con 0x7f74ac0ce7c0 2023-11-26T07:23:00.562 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:00.559+0000 7f74a1ffb640 1 --2- 172.21.15.143:0/1880182461 >> [v2:172.21.15.143:6832/2150557544,v1:172.21.15.143:6833/2150557544] conn(0x7f74900667d0 0x7f7490068c90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:23:00.562 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:00.559+0000 7f74a1ffb640 1 -- 172.21.15.143:0/1880182461 <== mon.0 v2:172.21.15.143:3300/0 5 ==== osd_map(71..71 src has 1..71) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f74a8083880 con 0x7f74ac0ce7c0 2023-11-26T07:23:00.562 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:00.559+0000 7f74b09f5640 1 --2- 172.21.15.143:0/1880182461 >> [v2:172.21.15.143:6832/2150557544,v1:172.21.15.143:6833/2150557544] conn(0x7f74900667d0 0x7f7490068c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:23:00.562 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:00.559+0000 7f74b09f5640 1 --2- 172.21.15.143:0/1880182461 >> [v2:172.21.15.143:6832/2150557544,v1:172.21.15.143:6833/2150557544] conn(0x7f74900667d0 0x7f7490068c90 secure :-1 s=READY pgs=477 cs=0 l=1 rev1=1 crypto rx=0x7f74ac14d2c0 tx=0x7f749c007d40 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:23:00.564 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:00.563+0000 7f74a1ffb640 1 -- 172.21.15.143:0/1880182461 <== mon.0 v2:172.21.15.143:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172649 (secure 0 0 0) 0x7f74a8050970 con 0x7f74ac0ce7c0 2023-11-26T07:23:00.716 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:00.715+0000 7f74b345a640 1 -- 172.21.15.143:0/1880182461 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f74ac067ff0 con 0x7f74ac0ce7c0 2023-11-26T07:23:00.717 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:00.715+0000 7f74a1ffb640 1 -- 172.21.15.143:0/1880182461 <== mon.0 v2:172.21.15.143:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v71) v1 ==== 74+0+16604 (secure 0 0 0) 0x7f74a8053f90 con 0x7f74ac0ce7c0 2023-11-26T07:23:00.717 INFO:teuthology.orchestra.run.smithi143.stdout: 2023-11-26T07:23:00.717 INFO:teuthology.orchestra.run.smithi143.stdout:{"epoch":71,"fsid":"f5033060-2469-4a07-95fe-2b76f5404bc1","created":"2023-11-26T07:17:28.613663+0000","modified":"2023-11-26T07:22:38.856077+0000","last_up_change":"2023-11-26T07:17:41.157813+0000","last_in_change":"2023-11-26T07:17:31.181658+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":11,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-11-26T07:17:41.758597+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":10,"pool_name":"cephfs_metadata","create_time":"2023-11-26T07:22:35.314819+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"70","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":2.5,"score_stable":2.5,"optimal_score":1,"raw_score_acting":2.5,"raw_score_stable":2.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":11,"pool_name":"cephfs_data","create_time":"2023-11-26T07:22:36.219536+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"71","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"ebbee2c6-023e-44b9-9f1d-6f7bfd309730","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":69,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6817","nonce":639807369},{"type":"v1","addr":"172.21.15.143:6819","nonce":639807369}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6821","nonce":639807369},{"type":"v1","addr":"172.21.15.143:6823","nonce":639807369}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6830","nonce":639807369},{"type":"v1","addr":"172.21.15.143:6831","nonce":639807369}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6825","nonce":639807369},{"type":"v1","addr":"172.21.15.143:6828","nonce":639807369}]},"public_addr":"172.21.15.143:6819/639807369","cluster_addr":"172.21.15.143:6823/639807369","heartbeat_back_addr":"172.21.15.143:6831/639807369","heartbeat_front_addr":"172.21.15.143:6828/639807369","state":["exists","up"]},{"osd":1,"uuid":"fec4b8be-7122-49e9-9a19-5850a0700001","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":69,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6816","nonce":1144942530},{"type":"v1","addr":"172.21.15.143:6818","nonce":1144942530}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6820","nonce":1144942530},{"type":"v1","addr":"172.21.15.143:6822","nonce":1144942530}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6827","nonce":1144942530},{"type":"v1","addr":"172.21.15.143:6829","nonce":1144942530}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6824","nonce":1144942530},{"type":"v1","addr":"172.21.15.143:6826","nonce":1144942530}]},"public_addr":"172.21.15.143:6818/1144942530","cluster_addr":"172.21.15.143:6822/1144942530","heartbeat_back_addr":"172.21.15.143:6829/1144942530","heartbeat_front_addr":"172.21.15.143:6826/1144942530","state":["exists","up"]},{"osd":2,"uuid":"2888af5b-f933-46ca-8f95-6dd494ff43f6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":69,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6808","nonce":3909095186},{"type":"v1","addr":"172.21.15.143:6809","nonce":3909095186}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6810","nonce":3909095186},{"type":"v1","addr":"172.21.15.143:6811","nonce":3909095186}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6814","nonce":3909095186},{"type":"v1","addr":"172.21.15.143:6815","nonce":3909095186}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6812","nonce":3909095186},{"type":"v1","addr":"172.21.15.143:6813","nonce":3909095186}]},"public_addr":"172.21.15.143:6809/3909095186","cluster_addr":"172.21.15.143:6811/3909095186","heartbeat_back_addr":"172.21.15.143:6815/3909095186","heartbeat_front_addr":"172.21.15.143:6813/3909095186","state":["exists","up"]},{"osd":3,"uuid":"d1f52435-7bc1-4081-9550-9cd1ba55f549","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":69,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6800","nonce":3854210991},{"type":"v1","addr":"172.21.15.143:6801","nonce":3854210991}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6802","nonce":3854210991},{"type":"v1","addr":"172.21.15.143:6803","nonce":3854210991}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6806","nonce":3854210991},{"type":"v1","addr":"172.21.15.143:6807","nonce":3854210991}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6804","nonce":3854210991},{"type":"v1","addr":"172.21.15.143:6805","nonce":3854210991}]},"public_addr":"172.21.15.143:6801/3854210991","cluster_addr":"172.21.15.143:6803/3854210991","heartbeat_back_addr":"172.21.15.143:6807/3854210991","heartbeat_front_addr":"172.21.15.143:6805/3854210991","state":["exists","up"]},{"osd":4,"uuid":"e2ae8b6e-6e25-49b2-bcbf-aacce68f06a3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":69,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6800","nonce":3924606902},{"type":"v1","addr":"172.21.15.188:6801","nonce":3924606902}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6802","nonce":3924606902},{"type":"v1","addr":"172.21.15.188:6803","nonce":3924606902}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6806","nonce":3924606902},{"type":"v1","addr":"172.21.15.188:6807","nonce":3924606902}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6804","nonce":3924606902},{"type":"v1","addr":"172.21.15.188:6805","nonce":3924606902}]},"public_addr":"172.21.15.188:6801/3924606902","cluster_addr":"172.21.15.188:6803/3924606902","heartbeat_back_addr":"172.21.15.188:6807/3924606902","heartbeat_front_addr":"172.21.15.188:6805/3924606902","state":["exists","up"]},{"osd":5,"uuid":"3cec8a32-7698-4c36-8e89-e4ba11934f33","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":69,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6813","nonce":186841544},{"type":"v1","addr":"172.21.15.188:6815","nonce":186841544}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6817","nonce":186841544},{"type":"v1","addr":"172.21.15.188:6818","nonce":186841544}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6822","nonce":186841544},{"type":"v1","addr":"172.21.15.188:6823","nonce":186841544}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6820","nonce":186841544},{"type":"v1","addr":"172.21.15.188:6821","nonce":186841544}]},"public_addr":"172.21.15.188:6815/186841544","cluster_addr":"172.21.15.188:6818/186841544","heartbeat_back_addr":"172.21.15.188:6823/186841544","heartbeat_front_addr":"172.21.15.188:6821/186841544","state":["exists","up"]},{"osd":6,"uuid":"58a65d5d-e090-473a-81b4-ad38e85029f9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":69,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6808","nonce":2153499405},{"type":"v1","addr":"172.21.15.188:6809","nonce":2153499405}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6810","nonce":2153499405},{"type":"v1","addr":"172.21.15.188:6811","nonce":2153499405}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6816","nonce":2153499405},{"type":"v1","addr":"172.21.15.188:6819","nonce":2153499405}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6812","nonce":2153499405},{"type":"v1","addr":"172.21.15.188:6814","nonce":2153499405}]},"public_addr":"172.21.15.188:6809/2153499405","cluster_addr":"172.21.15.188:6811/2153499405","heartbeat_back_addr":"172.21.15.188:6819/2153499405","heartbeat_front_addr":"172.21.15.188:6814/2153499405","state":["exists","up"]},{"osd":7,"uuid":"ea5b7f70-3554-4cf8-ac30-f7c59b80678d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":69,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6824","nonce":2722096729},{"type":"v1","addr":"172.21.15.188:6825","nonce":2722096729}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6826","nonce":2722096729},{"type":"v1","addr":"172.21.15.188:6827","nonce":2722096729}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6830","nonce":2722096729},{"type":"v1","addr":"172.21.15.188:6831","nonce":2722096729}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6828","nonce":2722096729},{"type":"v1","addr":"172.21.15.188:6829","nonce":2722096729}]},"public_addr":"172.21.15.188:6825/2722096729","cluster_addr":"172.21.15.188:6827/2722096729","heartbeat_back_addr":"172.21.15.188:6831/2722096729","heartbeat_front_addr":"172.21.15.188:6829/2722096729","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-26T07:17:36.995032+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-26T07:17:36.932266+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-26T07:17:36.829304+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-26T07:17:36.804297+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-26T07:17:36.667892+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-26T07:17:36.964868+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-26T07:17:36.827332+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-26T07:17:36.893179+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-11-26T07:23:00.719 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:00.719+0000 7f74b345a640 1 -- 172.21.15.143:0/1880182461 >> [v2:172.21.15.143:6832/2150557544,v1:172.21.15.143:6833/2150557544] conn(0x7f74900667d0 msgr2=0x7f7490068c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:23:00.719 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:00.719+0000 7f74b345a640 1 --2- 172.21.15.143:0/1880182461 >> [v2:172.21.15.143:6832/2150557544,v1:172.21.15.143:6833/2150557544] conn(0x7f74900667d0 0x7f7490068c90 secure :-1 s=READY pgs=477 cs=0 l=1 rev1=1 crypto rx=0x7f74ac14d2c0 tx=0x7f749c007d40 comp rx=0 tx=0).stop 2023-11-26T07:23:00.720 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:00.719+0000 7f74b345a640 1 -- 172.21.15.143:0/1880182461 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f74ac0ce7c0 msgr2=0x7f74ac151d60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:23:00.720 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:00.719+0000 7f74b345a640 1 --2- 172.21.15.143:0/1880182461 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f74ac0ce7c0 0x7f74ac151d60 secure :-1 s=READY pgs=1030 cs=0 l=1 rev1=1 crypto rx=0x7f74a800c3c0 tx=0x7f74a800c880 comp rx=0 tx=0).stop 2023-11-26T07:23:00.720 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:00.719+0000 7f74b345a640 1 -- 172.21.15.143:0/1880182461 shutdown_connections 2023-11-26T07:23:00.720 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:00.719+0000 7f74b345a640 1 --2- 172.21.15.143:0/1880182461 >> [v2:172.21.15.143:6832/2150557544,v1:172.21.15.143:6833/2150557544] conn(0x7f74900667d0 0x7f7490068c90 unknown :-1 s=CLOSED pgs=477 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:23:00.720 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:00.719+0000 7f74b345a640 1 --2- 172.21.15.143:0/1880182461 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f74ac13e290 0x7f74ac137790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:23:00.720 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:00.719+0000 7f74b345a640 1 --2- 172.21.15.143:0/1880182461 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f74ac0cf990 0x7f74ac0b0d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:23:00.720 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:00.719+0000 7f74b345a640 1 --2- 172.21.15.143:0/1880182461 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f74ac0ce7c0 0x7f74ac151d60 unknown :-1 s=CLOSED pgs=1030 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:23:00.721 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:00.719+0000 7f74b345a640 1 -- 172.21.15.143:0/1880182461 >> 172.21.15.143:0/1880182461 conn(0x7f74ac06a890 msgr2=0x7f74ac1511b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:23:00.721 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:00.719+0000 7f74b345a640 1 -- 172.21.15.143:0/1880182461 shutdown_connections 2023-11-26T07:23:00.721 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:00.719+0000 7f74b345a640 1 -- 172.21.15.143:0/1880182461 wait complete. 2023-11-26T07:23:00.738 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-11-26T07:17:41.758597+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 10, 'pool_name': 'cephfs_metadata', 'create_time': '2023-11-26T07:22:35.314819+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '70', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 2.5, 'score_stable': 2.5, 'optimal_score': 1, 'raw_score_acting': 2.5, 'raw_score_stable': 2.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 11, 'pool_name': 'cephfs_data', 'create_time': '2023-11-26T07:22:36.219536+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '71', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.3799999952316284, 'score_stable': 1.3799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.3799999952316284, 'raw_score_stable': 1.3799999952316284, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2023-11-26T07:23:00.739 DEBUG:teuthology.orchestra.run.smithi143:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-11-26T07:23:00.882 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:00.879+0000 7efc04ad6640 1 Processor -- start 2023-11-26T07:23:00.882 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:00.879+0000 7efc04ad6640 1 -- start start 2023-11-26T07:23:00.883 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:00.879+0000 7efc04ad6640 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7efc00145fa0 0x7efc00072420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:23:00.883 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:00.879+0000 7efc04ad6640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7efc0006b190 0x7efc00146530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:23:00.883 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:00.879+0000 7efc04ad6640 1 -- --> v1:172.21.15.143:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7efc00146380 con 0x7efc0006a490 2023-11-26T07:23:00.883 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:00.879+0000 7efc04ad6640 1 -- --> v2:172.21.15.188:3300/0 -- mon_getmap magic: 0 v1 -- 0x7efc00071480 con 0x7efc00145fa0 2023-11-26T07:23:00.883 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:00.879+0000 7efc04ad6640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7efc00072000 con 0x7efc0006b190 2023-11-26T07:23:00.883 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:00.883+0000 7efbfed9d640 1 --1- >> v1:172.21.15.143:6789/0 conn(0x7efc0006a490 0x7efc0006e890 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.143:6789/0 says I am v1:172.21.15.143:57914/0 (socket says 172.21.15.143:57914) 2023-11-26T07:23:00.883 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:00.883+0000 7efbfed9d640 1 -- 172.21.15.143:0/692205038 learned_addr learned my addr 172.21.15.143:0/692205038 (peer_addr_for_me v1:172.21.15.143:0/0) 2023-11-26T07:23:00.884 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:00.883+0000 7efbf5d9b640 1 --2- 172.21.15.143:0/692205038 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7efc0006b190 0x7efc00146530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:23:00.884 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:00.883+0000 7efbfe59c640 1 --2- 172.21.15.143:0/692205038 >> v2:172.21.15.188:3300/0 conn(0x7efc00145fa0 0x7efc00072420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:23:00.885 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:00.883+0000 7efbfdd9b640 1 -- 172.21.15.143:0/692205038 <== mon.0 v1:172.21.15.143:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2899408749 0 0) 0x7efc00146380 con 0x7efc0006a490 2023-11-26T07:23:00.885 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:00.883+0000 7efbfdd9b640 1 -- 172.21.15.143:0/692205038 --> v1:172.21.15.143:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7efbec003530 con 0x7efc0006a490 2023-11-26T07:23:00.885 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:00.883+0000 7efbf5d9b640 1 -- 172.21.15.143:0/692205038 >> v2:172.21.15.188:3300/0 conn(0x7efc00145fa0 msgr2=0x7efc00072420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-26T07:23:00.885 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:00.883+0000 7efbf5d9b640 1 --2- 172.21.15.143:0/692205038 >> v2:172.21.15.188:3300/0 conn(0x7efc00145fa0 0x7efc00072420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:23:00.885 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:00.883+0000 7efbf5d9b640 1 -- 172.21.15.143:0/692205038 >> v1:172.21.15.143:6789/0 conn(0x7efc0006a490 legacy=0x7efc0006e890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:23:00.885 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:00.883+0000 7efbfdd9b640 1 -- 172.21.15.143:0/692205038 <== mon.0 v1:172.21.15.143:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1395929189 0 0) 0x7efbec003530 con 0x7efc0006a490 2023-11-26T07:23:00.885 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:00.883+0000 7efbf5d9b640 1 -- 172.21.15.143:0/692205038 --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efc00072960 con 0x7efc0006b190 2023-11-26T07:23:00.885 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:00.883+0000 7efbfe59c640 1 --2- 172.21.15.143:0/692205038 >> v2:172.21.15.188:3300/0 conn(0x7efc00145fa0 0x7efc00072420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-26T07:23:00.886 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:00.883+0000 7efbf5d9b640 1 --2- 172.21.15.143:0/692205038 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7efc0006b190 0x7efc00146530 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7efbe80097c0 tx=0x7efbe801c370 comp rx=0 tx=0).ready entity=mon.2 client_cookie=56b1895be89d281e server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:23:00.887 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:00.887+0000 7efbfdd9b640 1 -- 172.21.15.143:0/692205038 <== mon.2 v2:172.21.15.188:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efbf8001680 con 0x7efc0006b190 2023-11-26T07:23:00.887 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:00.887+0000 7efbfdd9b640 1 -- 172.21.15.143:0/692205038 <== mon.2 v2:172.21.15.188:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7efbe80a7050 con 0x7efc0006b190 2023-11-26T07:23:00.887 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:00.887+0000 7efbfdd9b640 1 -- 172.21.15.143:0/692205038 <== mon.2 v2:172.21.15.188:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efbe80ab620 con 0x7efc0006b190 2023-11-26T07:23:00.888 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:00.887+0000 7efc04ad6640 1 -- 172.21.15.143:0/692205038 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7efc0006b190 msgr2=0x7efc00146530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:23:00.888 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:00.887+0000 7efc04ad6640 1 --2- 172.21.15.143:0/692205038 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7efc0006b190 0x7efc00146530 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7efbe80097c0 tx=0x7efbe801c370 comp rx=0 tx=0).stop 2023-11-26T07:23:00.888 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:00.887+0000 7efc04ad6640 1 -- 172.21.15.143:0/692205038 shutdown_connections 2023-11-26T07:23:00.888 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:00.887+0000 7efc04ad6640 1 --2- 172.21.15.143:0/692205038 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7efc0006b190 0x7efc00146530 unknown :-1 s=CLOSED pgs=128 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:23:00.888 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:00.887+0000 7efc04ad6640 1 --2- 172.21.15.143:0/692205038 >> v2:172.21.15.188:3300/0 conn(0x7efc00145fa0 0x7efc00072420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:23:00.888 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:00.887+0000 7efc04ad6640 1 -- 172.21.15.143:0/692205038 >> 172.21.15.143:0/692205038 conn(0x7efc000cff00 msgr2=0x7efc000d0310 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:23:00.888 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:00.887+0000 7efc04ad6640 1 -- 172.21.15.143:0/692205038 shutdown_connections 2023-11-26T07:23:00.888 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:00.887+0000 7efc04ad6640 1 -- 172.21.15.143:0/692205038 wait complete. 2023-11-26T07:23:00.890 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:00.887+0000 7efc04ad6640 1 Processor -- start 2023-11-26T07:23:00.890 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:00.887+0000 7efc04ad6640 1 -- start start 2023-11-26T07:23:00.891 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:00.887+0000 7efc04ad6640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7efc0006a490 0x7efc000ad420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:23:00.891 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:00.887+0000 7efc04ad6640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7efc0006b190 0x7efc00143910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:23:00.892 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:00.887+0000 7efc04ad6640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7efc00145fa0 0x7efc00158250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:23:00.892 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:00.887+0000 7efc04ad6640 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efc00158ef0 con 0x7efc00145fa0 2023-11-26T07:23:00.892 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:00.887+0000 7efc04ad6640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efc0015a700 con 0x7efc0006b190 2023-11-26T07:23:00.892 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:00.887+0000 7efc04ad6640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7efc00159140 con 0x7efc0006a490 2023-11-26T07:23:00.892 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:00.887+0000 7efbfed9d640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7efc00145fa0 0x7efc00158250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:23:00.892 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:00.891+0000 7efbfed9d640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7efc00145fa0 0x7efc00158250 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:42886/0 (socket says 172.21.15.143:42886) 2023-11-26T07:23:00.892 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:00.891+0000 7efbfed9d640 1 -- 172.21.15.143:0/492596855 learned_addr learned my addr 172.21.15.143:0/492596855 (peer_addr_for_me v2:172.21.15.143:0/0) 2023-11-26T07:23:00.892 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:00.891+0000 7efbfe59c640 1 --2- 172.21.15.143:0/492596855 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7efc0006a490 0x7efc000ad420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:23:00.892 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:00.891+0000 7efbf5d9b640 1 --2- 172.21.15.143:0/492596855 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7efc0006b190 0x7efc00143910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:23:00.892 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:00.891+0000 7efbfed9d640 1 -- 172.21.15.143:0/492596855 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7efc0006a490 msgr2=0x7efc000ad420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:23:00.892 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:00.891+0000 7efbfed9d640 1 --2- 172.21.15.143:0/492596855 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7efc0006a490 0x7efc000ad420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:23:00.892 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:00.891+0000 7efbfed9d640 1 -- 172.21.15.143:0/492596855 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7efc0006b190 msgr2=0x7efc00143910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:23:00.893 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:00.891+0000 7efbfed9d640 1 --2- 172.21.15.143:0/492596855 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7efc0006b190 0x7efc00143910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:23:00.893 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:00.891+0000 7efbfed9d640 1 -- 172.21.15.143:0/492596855 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efbf0009a70 con 0x7efc00145fa0 2023-11-26T07:23:00.893 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:00.891+0000 7efbfed9d640 1 --2- 172.21.15.143:0/492596855 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7efc00145fa0 0x7efc00158250 secure :-1 s=READY pgs=1032 cs=0 l=1 rev1=1 crypto rx=0x7efbf800c750 tx=0x7efbf800cc10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:23:00.894 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:00.891+0000 7efbf77fe640 1 -- 172.21.15.143:0/492596855 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efbf8005ae0 con 0x7efc00145fa0 2023-11-26T07:23:00.894 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:00.891+0000 7efbf77fe640 1 -- 172.21.15.143:0/492596855 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7efbf8005c40 con 0x7efc00145fa0 2023-11-26T07:23:00.894 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:00.891+0000 7efbf77fe640 1 -- 172.21.15.143:0/492596855 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efbf8007070 con 0x7efc00145fa0 2023-11-26T07:23:00.894 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:00.891+0000 7efc04ad6640 1 -- 172.21.15.143:0/492596855 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7efbe8009420 con 0x7efc00145fa0 2023-11-26T07:23:00.894 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:00.891+0000 7efc04ad6640 1 -- 172.21.15.143:0/492596855 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7efc00159700 con 0x7efc00145fa0 2023-11-26T07:23:00.894 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:00.891+0000 7efc04ad6640 1 -- 172.21.15.143:0/492596855 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7efc00067680 con 0x7efc00145fa0 2023-11-26T07:23:00.894 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:00.891+0000 7efbf77fe640 1 -- 172.21.15.143:0/492596855 <== mon.0 v2:172.21.15.143:3300/0 4 ==== mgrmap(e 11) v1 ==== 86158+0+0 (secure 0 0 0) 0x7efbf80071d0 con 0x7efc00145fa0 2023-11-26T07:23:00.895 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:00.891+0000 7efbf77fe640 1 --2- 172.21.15.143:0/492596855 >> [v2:172.21.15.143:6832/2150557544,v1:172.21.15.143:6833/2150557544] conn(0x7efbe00667d0 0x7efbe0068c90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:23:00.895 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:00.891+0000 7efbf77fe640 1 -- 172.21.15.143:0/492596855 <== mon.0 v2:172.21.15.143:3300/0 5 ==== osd_map(71..71 src has 1..71) v4 ==== 5709+0+0 (secure 0 0 0) 0x7efbf8084150 con 0x7efc00145fa0 2023-11-26T07:23:00.895 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:00.895+0000 7efbfe59c640 1 --2- 172.21.15.143:0/492596855 >> [v2:172.21.15.143:6832/2150557544,v1:172.21.15.143:6833/2150557544] conn(0x7efbe00667d0 0x7efbe0068c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:23:00.895 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:00.895+0000 7efbfe59c640 1 --2- 172.21.15.143:0/492596855 >> [v2:172.21.15.143:6832/2150557544,v1:172.21.15.143:6833/2150557544] conn(0x7efbe00667d0 0x7efbe0068c90 secure :-1 s=READY pgs=478 cs=0 l=1 rev1=1 crypto rx=0x7efbf0007600 tx=0x7efbf000b040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:23:00.898 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:00.895+0000 7efbf77fe640 1 -- 172.21.15.143:0/492596855 <== mon.0 v2:172.21.15.143:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172649 (secure 0 0 0) 0x7efbf8051110 con 0x7efc00145fa0 2023-11-26T07:23:01.049 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:01.047+0000 7efc04ad6640 1 -- 172.21.15.143:0/492596855 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7efc0015a1e0 con 0x7efc00145fa0 2023-11-26T07:23:01.049 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:01.047+0000 7efbf77fe640 1 -- 172.21.15.143:0/492596855 <== mon.0 v2:172.21.15.143:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v71) v1 ==== 93+0+10 (secure 0 0 0) 0x7efbf8054730 con 0x7efc00145fa0 2023-11-26T07:23:01.049 INFO:teuthology.orchestra.run.smithi143.stdout:pg_num: 1 2023-11-26T07:23:01.051 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:01.051+0000 7efc04ad6640 1 -- 172.21.15.143:0/492596855 >> [v2:172.21.15.143:6832/2150557544,v1:172.21.15.143:6833/2150557544] conn(0x7efbe00667d0 msgr2=0x7efbe0068c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:23:01.051 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:01.051+0000 7efc04ad6640 1 --2- 172.21.15.143:0/492596855 >> [v2:172.21.15.143:6832/2150557544,v1:172.21.15.143:6833/2150557544] conn(0x7efbe00667d0 0x7efbe0068c90 secure :-1 s=READY pgs=478 cs=0 l=1 rev1=1 crypto rx=0x7efbf0007600 tx=0x7efbf000b040 comp rx=0 tx=0).stop 2023-11-26T07:23:01.052 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:01.051+0000 7efc04ad6640 1 -- 172.21.15.143:0/492596855 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7efc00145fa0 msgr2=0x7efc00158250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:23:01.052 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:01.051+0000 7efc04ad6640 1 --2- 172.21.15.143:0/492596855 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7efc00145fa0 0x7efc00158250 secure :-1 s=READY pgs=1032 cs=0 l=1 rev1=1 crypto rx=0x7efbf800c750 tx=0x7efbf800cc10 comp rx=0 tx=0).stop 2023-11-26T07:23:01.052 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:01.051+0000 7efc04ad6640 1 -- 172.21.15.143:0/492596855 shutdown_connections 2023-11-26T07:23:01.052 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:01.051+0000 7efc04ad6640 1 --2- 172.21.15.143:0/492596855 >> [v2:172.21.15.143:6832/2150557544,v1:172.21.15.143:6833/2150557544] conn(0x7efbe00667d0 0x7efbe0068c90 unknown :-1 s=CLOSED pgs=478 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:23:01.052 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:01.051+0000 7efc04ad6640 1 --2- 172.21.15.143:0/492596855 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7efc00145fa0 0x7efc00158250 unknown :-1 s=CLOSED pgs=1032 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:23:01.052 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:01.051+0000 7efc04ad6640 1 --2- 172.21.15.143:0/492596855 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7efc0006b190 0x7efc00143910 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:23:01.052 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:01.051+0000 7efc04ad6640 1 --2- 172.21.15.143:0/492596855 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7efc0006a490 0x7efc000ad420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:23:01.052 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:01.051+0000 7efc04ad6640 1 -- 172.21.15.143:0/492596855 >> 172.21.15.143:0/492596855 conn(0x7efc000cff00 msgr2=0x7efc000aef70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:23:01.052 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:01.051+0000 7efc04ad6640 1 -- 172.21.15.143:0/492596855 shutdown_connections 2023-11-26T07:23:01.052 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:01.051+0000 7efc04ad6640 1 -- 172.21.15.143:0/492596855 wait complete. 2023-11-26T07:23:01.071 DEBUG:teuthology.orchestra.run.smithi143:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-11-26T07:23:01.212 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:01.211+0000 7f1e64616640 1 Processor -- start 2023-11-26T07:23:01.212 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:01.211+0000 7f1e64616640 1 -- start start 2023-11-26T07:23:01.214 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:01.211+0000 7f1e64616640 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7f1e5c0b2160 0x7f1e5c0d0df0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:23:01.214 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:01.211+0000 7f1e64616640 1 -- --> v1:172.21.15.143:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1e5c0cf0e0 con 0x7f1e5c0d1bc0 2023-11-26T07:23:01.214 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:01.211+0000 7f1e64616640 1 -- --> v1:172.21.15.188:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1e5c0ceb20 con 0x7f1e5c15d6e0 2023-11-26T07:23:01.214 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:01.211+0000 7f1e64616640 1 -- --> v2:172.21.15.188:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1e5c15dac0 con 0x7f1e5c0b2160 2023-11-26T07:23:01.214 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:01.211+0000 7f1e61bb1640 1 --1- >> v1:172.21.15.143:6789/0 conn(0x7f1e5c0d1bc0 0x7f1e5c0b0750 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.143:6789/0 says I am v1:172.21.15.143:57916/0 (socket says 172.21.15.143:57916) 2023-11-26T07:23:01.214 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:01.211+0000 7f1e61bb1640 1 -- 172.21.15.143:0/429260153 learned_addr learned my addr 172.21.15.143:0/429260153 (peer_addr_for_me v1:172.21.15.143:0/0) 2023-11-26T07:23:01.215 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:01.211+0000 7f1e62bb3640 1 --2- 172.21.15.143:0/429260153 >> v2:172.21.15.188:3300/0 conn(0x7f1e5c0b2160 0x7f1e5c0d0df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:23:01.215 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:01.211+0000 7f1e613b0640 1 -- 172.21.15.143:0/429260153 <== mon.0 v1:172.21.15.143:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3643247025 0 0) 0x7f1e5c0cf0e0 con 0x7f1e5c0d1bc0 2023-11-26T07:23:01.215 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:01.211+0000 7f1e613b0640 1 -- 172.21.15.143:0/429260153 --> v1:172.21.15.143:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f1e48003530 con 0x7f1e5c0d1bc0 2023-11-26T07:23:01.215 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:01.211+0000 7f1e613b0640 1 -- 172.21.15.143:0/429260153 <== mon.0 v1:172.21.15.143:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2806379683 0 0) 0x7f1e48003530 con 0x7f1e5c0d1bc0 2023-11-26T07:23:01.216 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:01.211+0000 7f1e62bb3640 1 -- 172.21.15.143:0/429260153 >> v1:172.21.15.188:6789/0 conn(0x7f1e5c15d6e0 legacy=0x7f1e5c0d14b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:23:01.216 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:01.211+0000 7f1e62bb3640 1 -- 172.21.15.143:0/429260153 >> v1:172.21.15.143:6789/0 conn(0x7f1e5c0d1bc0 legacy=0x7f1e5c0b0750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:23:01.216 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:01.211+0000 7f1e62bb3640 1 -- 172.21.15.143:0/429260153 --> v2:172.21.15.188:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1e5c0cee00 con 0x7f1e5c0b2160 2023-11-26T07:23:01.217 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:01.215+0000 7f1e62bb3640 1 --2- 172.21.15.143:0/429260153 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f1e5c0b2160 0x7f1e5c0d0df0 secure :-1 s=READY pgs=407 cs=0 l=1 rev1=1 crypto rx=0x7f1e54009d90 tx=0x7f1e5401c870 comp rx=0 tx=0).ready entity=mon.1 client_cookie=5eb9e1ac8038ae2 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:23:01.218 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:01.215+0000 7f1e613b0640 1 -- 172.21.15.143:0/429260153 <== mon.1 v2:172.21.15.188:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1e4c001680 con 0x7f1e5c0b2160 2023-11-26T07:23:01.218 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:01.215+0000 7f1e613b0640 1 -- 172.21.15.143:0/429260153 <== mon.1 v2:172.21.15.188:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1e540a55a0 con 0x7f1e5c0b2160 2023-11-26T07:23:01.218 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:01.215+0000 7f1e613b0640 1 -- 172.21.15.143:0/429260153 <== mon.1 v2:172.21.15.188:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1e540a5930 con 0x7f1e5c0b2160 2023-11-26T07:23:01.219 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:01.215+0000 7f1e64616640 1 -- 172.21.15.143:0/429260153 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f1e5c0b2160 msgr2=0x7f1e5c0d0df0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:23:01.219 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:01.215+0000 7f1e64616640 1 --2- 172.21.15.143:0/429260153 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f1e5c0b2160 0x7f1e5c0d0df0 secure :-1 s=READY pgs=407 cs=0 l=1 rev1=1 crypto rx=0x7f1e54009d90 tx=0x7f1e5401c870 comp rx=0 tx=0).stop 2023-11-26T07:23:01.219 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:01.215+0000 7f1e64616640 1 -- 172.21.15.143:0/429260153 shutdown_connections 2023-11-26T07:23:01.219 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:01.219+0000 7f1e64616640 1 --2- 172.21.15.143:0/429260153 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f1e5c0b2160 0x7f1e5c0d0df0 unknown :-1 s=CLOSED pgs=407 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:23:01.219 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:01.219+0000 7f1e64616640 1 -- 172.21.15.143:0/429260153 >> 172.21.15.143:0/429260153 conn(0x7f1e5c09c550 msgr2=0x7f1e5c0d3220 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:23:01.219 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:01.219+0000 7f1e64616640 1 -- 172.21.15.143:0/429260153 shutdown_connections 2023-11-26T07:23:01.219 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:01.219+0000 7f1e64616640 1 -- 172.21.15.143:0/429260153 wait complete. 2023-11-26T07:23:01.221 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:01.219+0000 7f1e64616640 1 Processor -- start 2023-11-26T07:23:01.221 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:01.219+0000 7f1e64616640 1 -- start start 2023-11-26T07:23:01.222 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:01.219+0000 7f1e64616640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f1e5c0b2160 0x7f1e5c154050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:23:01.222 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:01.219+0000 7f1e64616640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f1e5c0d1bc0 0x7f1e5c1514d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:23:01.222 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:01.219+0000 7f1e64616640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f1e5c15d6e0 0x7f1e5c151b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:23:01.222 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:01.219+0000 7f1e64616640 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1e5c0c9ba0 con 0x7f1e5c0d1bc0 2023-11-26T07:23:01.222 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:01.219+0000 7f1e64616640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1e5c0c98c0 con 0x7f1e5c0b2160 2023-11-26T07:23:01.222 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:01.219+0000 7f1e64616640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1e5c0c7800 con 0x7f1e5c15d6e0 2023-11-26T07:23:01.222 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:01.219+0000 7f1e61bb1640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f1e5c0d1bc0 0x7f1e5c1514d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:23:01.223 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:01.219+0000 7f1e61bb1640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f1e5c0d1bc0 0x7f1e5c1514d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:42902/0 (socket says 172.21.15.143:42902) 2023-11-26T07:23:01.223 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:01.219+0000 7f1e61bb1640 1 -- 172.21.15.143:0/982364122 learned_addr learned my addr 172.21.15.143:0/982364122 (peer_addr_for_me v2:172.21.15.143:0/0) 2023-11-26T07:23:01.224 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:01.219+0000 7f1e62bb3640 1 --2- 172.21.15.143:0/982364122 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f1e5c15d6e0 0x7f1e5c151b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:23:01.224 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:01.219+0000 7f1e623b2640 1 --2- 172.21.15.143:0/982364122 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f1e5c0b2160 0x7f1e5c154050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:23:01.224 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:01.219+0000 7f1e61bb1640 1 -- 172.21.15.143:0/982364122 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f1e5c15d6e0 msgr2=0x7f1e5c151b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:23:01.224 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:01.219+0000 7f1e61bb1640 1 --2- 172.21.15.143:0/982364122 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f1e5c15d6e0 0x7f1e5c151b50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:23:01.224 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:01.219+0000 7f1e61bb1640 1 -- 172.21.15.143:0/982364122 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f1e5c0b2160 msgr2=0x7f1e5c154050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:23:01.224 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:01.219+0000 7f1e61bb1640 1 --2- 172.21.15.143:0/982364122 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f1e5c0b2160 0x7f1e5c154050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:23:01.224 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:01.219+0000 7f1e61bb1640 1 -- 172.21.15.143:0/982364122 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1e54009a70 con 0x7f1e5c0d1bc0 2023-11-26T07:23:01.224 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:01.219+0000 7f1e62bb3640 1 --2- 172.21.15.143:0/982364122 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f1e5c15d6e0 0x7f1e5c151b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-26T07:23:01.224 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:01.219+0000 7f1e623b2640 1 --2- 172.21.15.143:0/982364122 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f1e5c0b2160 0x7f1e5c154050 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-26T07:23:01.225 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:01.219+0000 7f1e61bb1640 1 --2- 172.21.15.143:0/982364122 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f1e5c0d1bc0 0x7f1e5c1514d0 secure :-1 s=READY pgs=1034 cs=0 l=1 rev1=1 crypto rx=0x7f1e4c00c730 tx=0x7f1e4c00cbf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:23:01.225 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:01.219+0000 7f1e52ffd640 1 -- 172.21.15.143:0/982364122 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1e4c005ae0 con 0x7f1e5c0d1bc0 2023-11-26T07:23:01.225 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:01.223+0000 7f1e52ffd640 1 -- 172.21.15.143:0/982364122 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1e4c005c40 con 0x7f1e5c0d1bc0 2023-11-26T07:23:01.225 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:01.223+0000 7f1e64616640 1 -- 172.21.15.143:0/982364122 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1e5c0c7ae0 con 0x7f1e5c0d1bc0 2023-11-26T07:23:01.225 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:01.223+0000 7f1e52ffd640 1 -- 172.21.15.143:0/982364122 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1e4c0072b0 con 0x7f1e5c0d1bc0 2023-11-26T07:23:01.225 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:01.223+0000 7f1e64616640 1 -- 172.21.15.143:0/982364122 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1e5c0c9500 con 0x7f1e5c0d1bc0 2023-11-26T07:23:01.225 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:01.223+0000 7f1e64616640 1 -- 172.21.15.143:0/982364122 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1e5c067680 con 0x7f1e5c0d1bc0 2023-11-26T07:23:01.226 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:01.223+0000 7f1e52ffd640 1 -- 172.21.15.143:0/982364122 <== mon.0 v2:172.21.15.143:3300/0 4 ==== mgrmap(e 11) v1 ==== 86158+0+0 (secure 0 0 0) 0x7f1e4c007410 con 0x7f1e5c0d1bc0 2023-11-26T07:23:01.226 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:01.223+0000 7f1e52ffd640 1 --2- 172.21.15.143:0/982364122 >> [v2:172.21.15.143:6832/2150557544,v1:172.21.15.143:6833/2150557544] conn(0x7f1e400667d0 0x7f1e40068c90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:23:01.226 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:01.223+0000 7f1e52ffd640 1 -- 172.21.15.143:0/982364122 <== mon.0 v2:172.21.15.143:3300/0 5 ==== osd_map(71..71 src has 1..71) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f1e4c083bd0 con 0x7f1e5c0d1bc0 2023-11-26T07:23:01.226 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:01.223+0000 7f1e623b2640 1 --2- 172.21.15.143:0/982364122 >> [v2:172.21.15.143:6832/2150557544,v1:172.21.15.143:6833/2150557544] conn(0x7f1e400667d0 0x7f1e40068c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:23:01.227 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:01.223+0000 7f1e623b2640 1 --2- 172.21.15.143:0/982364122 >> [v2:172.21.15.143:6832/2150557544,v1:172.21.15.143:6833/2150557544] conn(0x7f1e400667d0 0x7f1e40068c90 secure :-1 s=READY pgs=479 cs=0 l=1 rev1=1 crypto rx=0x7f1e5c0ccf60 tx=0x7f1e58009040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:23:01.229 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:01.227+0000 7f1e52ffd640 1 -- 172.21.15.143:0/982364122 <== mon.0 v2:172.21.15.143:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172649 (secure 0 0 0) 0x7f1e4c050c90 con 0x7f1e5c0d1bc0 2023-11-26T07:23:01.378 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:01.375+0000 7f1e64616640 1 -- 172.21.15.143:0/982364122 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f1e5c0c7f90 con 0x7f1e5c0d1bc0 2023-11-26T07:23:01.379 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:01.375+0000 7f1e52ffd640 1 -- 172.21.15.143:0/982364122 <== mon.0 v2:172.21.15.143:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v71) v1 ==== 104+0+11 (secure 0 0 0) 0x7f1e4c0170a0 con 0x7f1e5c0d1bc0 2023-11-26T07:23:01.379 INFO:teuthology.orchestra.run.smithi143.stdout:pg_num: 32 2023-11-26T07:23:01.381 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:01.379+0000 7f1e64616640 1 -- 172.21.15.143:0/982364122 >> [v2:172.21.15.143:6832/2150557544,v1:172.21.15.143:6833/2150557544] conn(0x7f1e400667d0 msgr2=0x7f1e40068c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:23:01.381 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:01.379+0000 7f1e64616640 1 --2- 172.21.15.143:0/982364122 >> [v2:172.21.15.143:6832/2150557544,v1:172.21.15.143:6833/2150557544] conn(0x7f1e400667d0 0x7f1e40068c90 secure :-1 s=READY pgs=479 cs=0 l=1 rev1=1 crypto rx=0x7f1e5c0ccf60 tx=0x7f1e58009040 comp rx=0 tx=0).stop 2023-11-26T07:23:01.381 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:01.379+0000 7f1e64616640 1 -- 172.21.15.143:0/982364122 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f1e5c0d1bc0 msgr2=0x7f1e5c1514d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:23:01.381 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:01.379+0000 7f1e64616640 1 --2- 172.21.15.143:0/982364122 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f1e5c0d1bc0 0x7f1e5c1514d0 secure :-1 s=READY pgs=1034 cs=0 l=1 rev1=1 crypto rx=0x7f1e4c00c730 tx=0x7f1e4c00cbf0 comp rx=0 tx=0).stop 2023-11-26T07:23:01.381 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:01.379+0000 7f1e64616640 1 -- 172.21.15.143:0/982364122 shutdown_connections 2023-11-26T07:23:01.381 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:01.379+0000 7f1e64616640 1 --2- 172.21.15.143:0/982364122 >> [v2:172.21.15.143:6832/2150557544,v1:172.21.15.143:6833/2150557544] conn(0x7f1e400667d0 0x7f1e40068c90 unknown :-1 s=CLOSED pgs=479 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:23:01.382 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:01.379+0000 7f1e64616640 1 --2- 172.21.15.143:0/982364122 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f1e5c15d6e0 0x7f1e5c151b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:23:01.382 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:01.379+0000 7f1e64616640 1 --2- 172.21.15.143:0/982364122 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f1e5c0d1bc0 0x7f1e5c1514d0 unknown :-1 s=CLOSED pgs=1034 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:23:01.382 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:01.379+0000 7f1e64616640 1 --2- 172.21.15.143:0/982364122 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f1e5c0b2160 0x7f1e5c154050 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:23:01.382 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:01.379+0000 7f1e64616640 1 -- 172.21.15.143:0/982364122 >> 172.21.15.143:0/982364122 conn(0x7f1e5c09c550 msgr2=0x7f1e5c0734a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:23:01.382 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:01.379+0000 7f1e64616640 1 -- 172.21.15.143:0/982364122 shutdown_connections 2023-11-26T07:23:01.382 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:01.379+0000 7f1e64616640 1 -- 172.21.15.143:0/982364122 wait complete. 2023-11-26T07:23:01.400 DEBUG:teuthology.orchestra.run.smithi143:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-11-26T07:23:01.543 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:01.543+0000 7f8264254640 1 Processor -- start 2023-11-26T07:23:01.543 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:01.543+0000 7f8264254640 1 -- start start 2023-11-26T07:23:01.545 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:01.543+0000 7f8264254640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f825c13e290 0x7f825c0cf280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:23:01.545 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:01.543+0000 7f8264254640 1 -- --> v1:172.21.15.143:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f825c06c520 con 0x7f825c15f6b0 2023-11-26T07:23:01.545 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:01.543+0000 7f8264254640 1 -- --> v1:172.21.15.188:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f825c06c800 con 0x7f825c0cf7c0 2023-11-26T07:23:01.545 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:01.543+0000 7f8264254640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f825c0cfc40 con 0x7f825c13e290 2023-11-26T07:23:01.545 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:01.543+0000 7f82627f1640 1 --1- >> v1:172.21.15.143:6789/0 conn(0x7f825c15f6b0 0x7f825c0cd7b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.143:6789/0 says I am v1:172.21.15.143:57922/0 (socket says 172.21.15.143:57922) 2023-11-26T07:23:01.545 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:01.543+0000 7f82627f1640 1 -- 172.21.15.143:0/2349893555 learned_addr learned my addr 172.21.15.143:0/2349893555 (peer_addr_for_me v1:172.21.15.143:0/0) 2023-11-26T07:23:01.545 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:01.543+0000 7f8261ff0640 1 --2- 172.21.15.143:0/2349893555 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f825c13e290 0x7f825c0cf280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:23:01.547 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:01.543+0000 7f8260fee640 1 -- 172.21.15.143:0/2349893555 <== mon.0 v1:172.21.15.143:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 616171076 0 0) 0x7f825c06c520 con 0x7f825c15f6b0 2023-11-26T07:23:01.547 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:01.543+0000 7f8260fee640 1 -- 172.21.15.143:0/2349893555 --> v1:172.21.15.143:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f8248003530 con 0x7f825c15f6b0 2023-11-26T07:23:01.547 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:01.543+0000 7f8260fee640 1 -- 172.21.15.143:0/2349893555 <== mon.1 v1:172.21.15.188:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3042180074 0 0) 0x7f825c06c800 con 0x7f825c0cf7c0 2023-11-26T07:23:01.547 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:01.543+0000 7f8260fee640 1 -- 172.21.15.143:0/2349893555 --> v1:172.21.15.188:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f825c06c520 con 0x7f825c0cf7c0 2023-11-26T07:23:01.548 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:01.543+0000 7f8260fee640 1 -- 172.21.15.143:0/2349893555 <== mon.0 v1:172.21.15.143:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2782158015 0 0) 0x7f8248003530 con 0x7f825c15f6b0 2023-11-26T07:23:01.548 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:01.543+0000 7f8260fee640 1 -- 172.21.15.143:0/2349893555 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f825c13e290 msgr2=0x7f825c0cf280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-26T07:23:01.548 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:01.543+0000 7f8260fee640 1 --2- 172.21.15.143:0/2349893555 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f825c13e290 0x7f825c0cf280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:23:01.548 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:01.543+0000 7f8260fee640 1 -- 172.21.15.143:0/2349893555 >> v1:172.21.15.188:6789/0 conn(0x7f825c0cf7c0 legacy=0x7f825c15efa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:23:01.548 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:01.543+0000 7f8260fee640 1 -- 172.21.15.143:0/2349893555 --> v1:172.21.15.143:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f825c06cae0 con 0x7f825c15f6b0 2023-11-26T07:23:01.548 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:01.543+0000 7f8260fee640 1 -- 172.21.15.143:0/2349893555 <== mon.0 v1:172.21.15.143:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2137882523 0 0) 0x7f8254001a00 con 0x7f825c15f6b0 2023-11-26T07:23:01.548 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:01.543+0000 7f8260fee640 1 -- 172.21.15.143:0/2349893555 >> v1:172.21.15.143:6789/0 conn(0x7f825c15f6b0 legacy=0x7f825c0cd7b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:23:01.548 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:01.543+0000 7f8260fee640 1 --2- 172.21.15.143:0/2349893555 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f8248003b40 0x7f8248003f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:23:01.548 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:01.543+0000 7f8260fee640 1 --2- 172.21.15.143:0/2349893555 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f825c13e290 0x7f82480047d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:23:01.548 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:01.543+0000 7f8260fee640 1 --2- 172.21.15.143:0/2349893555 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f8248004d10 0x7f8248005190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:23:01.548 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:01.543+0000 7f8260fee640 1 -- 172.21.15.143:0/2349893555 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f825c0cfc40 con 0x7f8248003b40 2023-11-26T07:23:01.548 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:01.543+0000 7f8260fee640 1 -- 172.21.15.143:0/2349893555 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f824800ba40 con 0x7f8248004d10 2023-11-26T07:23:01.549 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:01.543+0000 7f8260fee640 1 -- 172.21.15.143:0/2349893555 --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f824800bbb0 con 0x7f825c13e290 2023-11-26T07:23:01.549 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:01.543+0000 7f8261ff0640 1 --2- 172.21.15.143:0/2349893555 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f8248003b40 0x7f8248003f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:23:01.549 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:01.543+0000 7f82627f1640 1 --2- 172.21.15.143:0/2349893555 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f8248004d10 0x7f8248005190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:23:01.549 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:01.543+0000 7f82617ef640 1 --2- 172.21.15.143:0/2349893555 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f825c13e290 0x7f82480047d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:23:01.549 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:01.543+0000 7f8261ff0640 1 -- 172.21.15.143:0/2349893555 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f825c13e290 msgr2=0x7f82480047d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-26T07:23:01.549 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:01.543+0000 7f8261ff0640 1 --2- 172.21.15.143:0/2349893555 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f825c13e290 0x7f82480047d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:23:01.549 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:01.543+0000 7f8261ff0640 1 -- 172.21.15.143:0/2349893555 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f8248004d10 msgr2=0x7f8248005190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-26T07:23:01.549 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:01.543+0000 7f8261ff0640 1 --2- 172.21.15.143:0/2349893555 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f8248004d10 0x7f8248005190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:23:01.549 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:01.543+0000 7f8261ff0640 1 -- 172.21.15.143:0/2349893555 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8248003530 con 0x7f8248003b40 2023-11-26T07:23:01.550 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:01.547+0000 7f8261ff0640 1 --2- 172.21.15.143:0/2349893555 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f8248003b40 0x7f8248003f40 secure :-1 s=READY pgs=1036 cs=0 l=1 rev1=1 crypto rx=0x7f8258009ae0 tx=0x7f825801ca50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5afb1f03cd776714 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:23:01.550 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:01.547+0000 7f8260fee640 1 -- 172.21.15.143:0/2349893555 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f82580a6430 con 0x7f8248003b40 2023-11-26T07:23:01.551 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:01.547+0000 7f8260fee640 1 -- 172.21.15.143:0/2349893555 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f82580a6590 con 0x7f8248003b40 2023-11-26T07:23:01.551 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:01.547+0000 7f8260fee640 1 -- 172.21.15.143:0/2349893555 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f82580a6940 con 0x7f8248003b40 2023-11-26T07:23:01.551 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:01.547+0000 7f8264254640 1 -- 172.21.15.143:0/2349893555 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f8248003b40 msgr2=0x7f8248003f40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:23:01.551 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:01.547+0000 7f8264254640 1 --2- 172.21.15.143:0/2349893555 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f8248003b40 0x7f8248003f40 secure :-1 s=READY pgs=1036 cs=0 l=1 rev1=1 crypto rx=0x7f8258009ae0 tx=0x7f825801ca50 comp rx=0 tx=0).stop 2023-11-26T07:23:01.551 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:01.547+0000 7f82627f1640 1 -- 172.21.15.143:0/2349893555 reap_dead start 2023-11-26T07:23:01.551 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:01.547+0000 7f8264254640 1 -- 172.21.15.143:0/2349893555 shutdown_connections 2023-11-26T07:23:01.551 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:01.547+0000 7f8264254640 1 -- 172.21.15.143:0/2349893555 >> 172.21.15.143:0/2349893555 conn(0x7f825c06a890 msgr2=0x7f825c06aca0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:23:01.551 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:01.547+0000 7f8264254640 1 -- 172.21.15.143:0/2349893555 shutdown_connections 2023-11-26T07:23:01.551 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:01.547+0000 7f8264254640 1 -- 172.21.15.143:0/2349893555 wait complete. 2023-11-26T07:23:01.552 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:01.551+0000 7f8264254640 1 Processor -- start 2023-11-26T07:23:01.552 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:01.551+0000 7f8264254640 1 -- start start 2023-11-26T07:23:01.554 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:01.551+0000 7f8264254640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f8248003b40 0x7f825c15e190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:23:01.555 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:01.551+0000 7f8264254640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f825c15e6d0 0x7f825c151fa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:23:01.555 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:01.551+0000 7f8264254640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f825c1524e0 0x7f825c15b7e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:23:01.555 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:01.551+0000 7f8264254640 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f825c153400 con 0x7f825c1524e0 2023-11-26T07:23:01.555 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:01.551+0000 7f8264254640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f825c1541f0 con 0x7f825c15e6d0 2023-11-26T07:23:01.555 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:01.551+0000 7f8264254640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f825c1535e0 con 0x7f8248003b40 2023-11-26T07:23:01.555 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:01.551+0000 7f8261ff0640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f8248003b40 0x7f825c15e190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:23:01.555 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:01.551+0000 7f82627f1640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f825c1524e0 0x7f825c15b7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:23:01.555 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:01.551+0000 7f8261ff0640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f8248003b40 0x7f825c15e190 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.188:3301/0 says I am v2:172.21.15.143:43858/0 (socket says 172.21.15.143:43858) 2023-11-26T07:23:01.555 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:01.551+0000 7f8261ff0640 1 -- 172.21.15.143:0/2659865497 learned_addr learned my addr 172.21.15.143:0/2659865497 (peer_addr_for_me v2:172.21.15.143:0/0) 2023-11-26T07:23:01.555 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:01.551+0000 7f82617ef640 1 --2- 172.21.15.143:0/2659865497 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f825c15e6d0 0x7f825c151fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:23:01.555 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:01.551+0000 7f82627f1640 1 -- 172.21.15.143:0/2659865497 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f8248003b40 msgr2=0x7f825c15e190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:23:01.556 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:01.551+0000 7f82627f1640 1 --2- 172.21.15.143:0/2659865497 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f8248003b40 0x7f825c15e190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:23:01.556 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:01.551+0000 7f82627f1640 1 -- 172.21.15.143:0/2659865497 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f825c15e6d0 msgr2=0x7f825c151fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:23:01.556 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:01.551+0000 7f82627f1640 1 --2- 172.21.15.143:0/2659865497 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f825c15e6d0 0x7f825c151fa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:23:01.556 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:01.551+0000 7f82627f1640 1 -- 172.21.15.143:0/2659865497 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f825c1538c0 con 0x7f825c1524e0 2023-11-26T07:23:01.556 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:01.551+0000 7f82627f1640 1 --2- 172.21.15.143:0/2659865497 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f825c1524e0 0x7f825c15b7e0 secure :-1 s=READY pgs=1037 cs=0 l=1 rev1=1 crypto rx=0x7f825c0cfec0 tx=0x7f825400e160 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:23:01.556 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:01.551+0000 7f82527fc640 1 -- 172.21.15.143:0/2659865497 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f825400eb70 con 0x7f825c1524e0 2023-11-26T07:23:01.556 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:01.551+0000 7f82527fc640 1 -- 172.21.15.143:0/2659865497 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f825400ecf0 con 0x7f825c1524e0 2023-11-26T07:23:01.556 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:01.551+0000 7f8264254640 1 -- 172.21.15.143:0/2659865497 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f825c153ba0 con 0x7f825c1524e0 2023-11-26T07:23:01.556 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:01.551+0000 7f82527fc640 1 -- 172.21.15.143:0/2659865497 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8254013660 con 0x7f825c1524e0 2023-11-26T07:23:01.556 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:01.551+0000 7f8264254640 1 -- 172.21.15.143:0/2659865497 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f825c137ad0 con 0x7f825c1524e0 2023-11-26T07:23:01.557 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:01.555+0000 7f8264254640 1 -- 172.21.15.143:0/2659865497 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f825c154380 con 0x7f825c1524e0 2023-11-26T07:23:01.557 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:01.555+0000 7f82527fc640 1 -- 172.21.15.143:0/2659865497 <== mon.0 v2:172.21.15.143:3300/0 4 ==== mgrmap(e 11) v1 ==== 86158+0+0 (secure 0 0 0) 0x7f82540137c0 con 0x7f825c1524e0 2023-11-26T07:23:01.558 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:01.555+0000 7f82527fc640 1 --2- 172.21.15.143:0/2659865497 >> [v2:172.21.15.143:6832/2150557544,v1:172.21.15.143:6833/2150557544] conn(0x7f8240066800 0x7f8240068cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:23:01.558 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:01.555+0000 7f82527fc640 1 -- 172.21.15.143:0/2659865497 <== mon.0 v2:172.21.15.143:3300/0 5 ==== osd_map(71..71 src has 1..71) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f82540846a0 con 0x7f825c1524e0 2023-11-26T07:23:01.558 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:01.555+0000 7f8261ff0640 1 --2- 172.21.15.143:0/2659865497 >> [v2:172.21.15.143:6832/2150557544,v1:172.21.15.143:6833/2150557544] conn(0x7f8240066800 0x7f8240068cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:23:01.558 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:01.555+0000 7f8261ff0640 1 --2- 172.21.15.143:0/2659865497 >> [v2:172.21.15.143:6832/2150557544,v1:172.21.15.143:6833/2150557544] conn(0x7f8240066800 0x7f8240068cc0 secure :-1 s=READY pgs=480 cs=0 l=1 rev1=1 crypto rx=0x7f825801ca90 tx=0x7f82580b3040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:23:01.561 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:01.559+0000 7f82527fc640 1 -- 172.21.15.143:0/2659865497 <== mon.0 v2:172.21.15.143:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172649 (secure 0 0 0) 0x7f8254051780 con 0x7f825c1524e0 2023-11-26T07:23:01.711 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:01.711+0000 7f8264254640 1 -- 172.21.15.143:0/2659865497 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f825c067680 con 0x7f825c1524e0 2023-11-26T07:23:01.712 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:01.711+0000 7f82527fc640 1 -- 172.21.15.143:0/2659865497 <== mon.0 v2:172.21.15.143:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v71) v1 ==== 100+0+11 (secure 0 0 0) 0x7f8254054da0 con 0x7f825c1524e0 2023-11-26T07:23:01.712 INFO:teuthology.orchestra.run.smithi143.stdout:pg_num: 64 2023-11-26T07:23:01.714 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:01.711+0000 7f8264254640 1 -- 172.21.15.143:0/2659865497 >> [v2:172.21.15.143:6832/2150557544,v1:172.21.15.143:6833/2150557544] conn(0x7f8240066800 msgr2=0x7f8240068cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:23:01.714 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:01.711+0000 7f8264254640 1 --2- 172.21.15.143:0/2659865497 >> [v2:172.21.15.143:6832/2150557544,v1:172.21.15.143:6833/2150557544] conn(0x7f8240066800 0x7f8240068cc0 secure :-1 s=READY pgs=480 cs=0 l=1 rev1=1 crypto rx=0x7f825801ca90 tx=0x7f82580b3040 comp rx=0 tx=0).stop 2023-11-26T07:23:01.714 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:01.711+0000 7f8264254640 1 -- 172.21.15.143:0/2659865497 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f825c1524e0 msgr2=0x7f825c15b7e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:23:01.714 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:01.711+0000 7f8264254640 1 --2- 172.21.15.143:0/2659865497 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f825c1524e0 0x7f825c15b7e0 secure :-1 s=READY pgs=1037 cs=0 l=1 rev1=1 crypto rx=0x7f825c0cfec0 tx=0x7f825400e160 comp rx=0 tx=0).stop 2023-11-26T07:23:01.714 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:01.711+0000 7f8264254640 1 -- 172.21.15.143:0/2659865497 shutdown_connections 2023-11-26T07:23:01.714 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:01.711+0000 7f8264254640 1 --2- 172.21.15.143:0/2659865497 >> [v2:172.21.15.143:6832/2150557544,v1:172.21.15.143:6833/2150557544] conn(0x7f8240066800 0x7f8240068cc0 unknown :-1 s=CLOSED pgs=480 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:23:01.714 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:01.711+0000 7f8264254640 1 --2- 172.21.15.143:0/2659865497 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f825c1524e0 0x7f825c15b7e0 unknown :-1 s=CLOSED pgs=1037 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:23:01.714 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:01.711+0000 7f8264254640 1 --2- 172.21.15.143:0/2659865497 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f825c15e6d0 0x7f825c151fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:23:01.715 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:01.711+0000 7f8264254640 1 --2- 172.21.15.143:0/2659865497 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f8248003b40 0x7f825c15e190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:23:01.715 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:01.711+0000 7f8264254640 1 -- 172.21.15.143:0/2659865497 >> 172.21.15.143:0/2659865497 conn(0x7f825c06a890 msgr2=0x7f825c13cdf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:23:01.715 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:01.711+0000 7f8264254640 1 -- 172.21.15.143:0/2659865497 shutdown_connections 2023-11-26T07:23:01.715 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:01.711+0000 7f8264254640 1 -- 172.21.15.143:0/2659865497 wait complete. 2023-11-26T07:23:01.733 DEBUG:teuthology.orchestra.run.smithi143:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2023-11-26T07:23:01.876 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:01.875+0000 7f8fbd274640 1 Processor -- start 2023-11-26T07:23:01.876 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:01.875+0000 7f8fbd274640 1 -- start start 2023-11-26T07:23:01.878 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:01.875+0000 7f8fbd274640 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7f8fb8144720 0x7f8fb806d300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:23:01.878 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:01.875+0000 7f8fbd274640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f8fb806d840 0x7f8fb80714e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:23:01.878 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:01.875+0000 7f8fbd274640 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7f8fb8071a20 0x7f8fb80cc170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:23:01.878 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:01.875+0000 7f8fbd274640 1 -- --> v2:172.21.15.143:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8fb8071e90 con 0x7f8fb8144720 2023-11-26T07:23:01.878 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:01.875+0000 7f8fbd274640 1 -- --> v2:172.21.15.188:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8fb806e3f0 con 0x7f8fb8071a20 2023-11-26T07:23:01.879 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:01.875+0000 7f8fbd274640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8fb80722f0 con 0x7f8fb806d840 2023-11-26T07:23:01.879 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:01.875+0000 7f8fb7fff640 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7f8fb8144720 0x7f8fb806d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:23:01.879 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:01.875+0000 7f8fb7fff640 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7f8fb8144720 0x7f8fb806d300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:42930/0 (socket says 172.21.15.143:42930) 2023-11-26T07:23:01.879 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:01.875+0000 7f8fb7fff640 1 -- 172.21.15.143:0/421698095 learned_addr learned my addr 172.21.15.143:0/421698095 (peer_addr_for_me v2:172.21.15.143:0/0) 2023-11-26T07:23:01.879 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:01.875+0000 7f8fbca73640 1 --2- 172.21.15.143:0/421698095 >> v2:172.21.15.188:3300/0 conn(0x7f8fb8071a20 0x7f8fb80cc170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:23:01.879 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:01.875+0000 7f8fb77fe640 1 --2- 172.21.15.143:0/421698095 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f8fb806d840 0x7f8fb80714e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:23:01.879 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:01.875+0000 7f8fb7fff640 1 -- 172.21.15.143:0/421698095 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f8fb806d840 msgr2=0x7f8fb80714e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-26T07:23:01.879 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:01.875+0000 7f8fb7fff640 1 --2- 172.21.15.143:0/421698095 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f8fb806d840 0x7f8fb80714e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:23:01.879 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:01.875+0000 7f8fb7fff640 1 -- 172.21.15.143:0/421698095 >> v2:172.21.15.188:3300/0 conn(0x7f8fb8071a20 msgr2=0x7f8fb80cc170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-26T07:23:01.879 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:01.875+0000 7f8fb7fff640 1 --2- 172.21.15.143:0/421698095 >> v2:172.21.15.188:3300/0 conn(0x7f8fb8071a20 0x7f8fb80cc170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:23:01.879 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:01.875+0000 7f8fb7fff640 1 -- 172.21.15.143:0/421698095 --> v2:172.21.15.143:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8fb806de30 con 0x7f8fb8144720 2023-11-26T07:23:01.881 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:01.879+0000 7f8fb7fff640 1 --2- 172.21.15.143:0/421698095 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f8fb8144720 0x7f8fb806d300 secure :-1 s=READY pgs=1038 cs=0 l=1 rev1=1 crypto rx=0x7f8fb80b1ec0 tx=0x7f8fa801c210 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2552c64e2567e621 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:23:01.881 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:01.879+0000 7f8fb6ffd640 1 -- 172.21.15.143:0/421698095 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8fa80ab600 con 0x7f8fb8144720 2023-11-26T07:23:01.881 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:01.879+0000 7f8fb6ffd640 1 -- 172.21.15.143:0/421698095 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8fa80a7050 con 0x7f8fb8144720 2023-11-26T07:23:01.882 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:01.879+0000 7f8fb6ffd640 1 -- 172.21.15.143:0/421698095 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8fa80ab980 con 0x7f8fb8144720 2023-11-26T07:23:01.882 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:01.879+0000 7f8fbd274640 1 -- 172.21.15.143:0/421698095 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f8fb8144720 msgr2=0x7f8fb806d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:23:01.882 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:01.879+0000 7f8fbd274640 1 --2- 172.21.15.143:0/421698095 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f8fb8144720 0x7f8fb806d300 secure :-1 s=READY pgs=1038 cs=0 l=1 rev1=1 crypto rx=0x7f8fb80b1ec0 tx=0x7f8fa801c210 comp rx=0 tx=0).stop 2023-11-26T07:23:01.883 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:01.879+0000 7f8fbd274640 1 -- 172.21.15.143:0/421698095 shutdown_connections 2023-11-26T07:23:01.883 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:01.879+0000 7f8fbd274640 1 --2- 172.21.15.143:0/421698095 >> v2:172.21.15.188:3300/0 conn(0x7f8fb8071a20 0x7f8fb80cc170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:23:01.883 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:01.879+0000 7f8fbd274640 1 --2- 172.21.15.143:0/421698095 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f8fb806d840 0x7f8fb80714e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:23:01.883 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:01.879+0000 7f8fbd274640 1 --2- 172.21.15.143:0/421698095 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f8fb8144720 0x7f8fb806d300 unknown :-1 s=CLOSED pgs=1038 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:23:01.883 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:01.879+0000 7f8fbd274640 1 -- 172.21.15.143:0/421698095 >> 172.21.15.143:0/421698095 conn(0x7f8fb806a890 msgr2=0x7f8fb806aca0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:23:01.883 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:01.879+0000 7f8fbd274640 1 -- 172.21.15.143:0/421698095 shutdown_connections 2023-11-26T07:23:01.883 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:01.879+0000 7f8fbd274640 1 -- 172.21.15.143:0/421698095 wait complete. 2023-11-26T07:23:01.883 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:01.883+0000 7f8fbd274640 1 Processor -- start 2023-11-26T07:23:01.884 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:01.883+0000 7f8fbd274640 1 -- start start 2023-11-26T07:23:01.886 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:01.883+0000 7f8fbd274640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f8fb806d840 0x7f8fb815a770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:23:01.886 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:01.883+0000 7f8fbd274640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f8fb8071a20 0x7f8fb813be00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:23:01.886 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:01.883+0000 7f8fbd274640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f8fb8144720 0x7f8fb813c340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:23:01.886 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:01.883+0000 7f8fbd274640 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8fb8143b70 con 0x7f8fb806d840 2023-11-26T07:23:01.886 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:01.883+0000 7f8fbd274640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8fb8142f60 con 0x7f8fb8071a20 2023-11-26T07:23:01.886 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:01.883+0000 7f8fbd274640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8fb8143240 con 0x7f8fb8144720 2023-11-26T07:23:01.886 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:01.883+0000 7f8fb7fff640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f8fb806d840 0x7f8fb815a770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:23:01.886 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:01.883+0000 7f8fb7fff640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f8fb806d840 0x7f8fb815a770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:42938/0 (socket says 172.21.15.143:42938) 2023-11-26T07:23:01.886 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:01.883+0000 7f8fb7fff640 1 -- 172.21.15.143:0/4173775527 learned_addr learned my addr 172.21.15.143:0/4173775527 (peer_addr_for_me v2:172.21.15.143:0/0) 2023-11-26T07:23:01.887 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:01.883+0000 7f8fb77fe640 1 --2- 172.21.15.143:0/4173775527 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f8fb8071a20 0x7f8fb813be00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:23:01.887 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:01.883+0000 7f8fbca73640 1 --2- 172.21.15.143:0/4173775527 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f8fb8144720 0x7f8fb813c340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:23:01.887 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:01.883+0000 7f8fb7fff640 1 -- 172.21.15.143:0/4173775527 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f8fb8144720 msgr2=0x7f8fb813c340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:23:01.887 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:01.883+0000 7f8fb7fff640 1 --2- 172.21.15.143:0/4173775527 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f8fb8144720 0x7f8fb813c340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:23:01.887 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:01.883+0000 7f8fb7fff640 1 -- 172.21.15.143:0/4173775527 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f8fb8071a20 msgr2=0x7f8fb813be00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:23:01.887 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:01.883+0000 7f8fb7fff640 1 --2- 172.21.15.143:0/4173775527 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f8fb8071a20 0x7f8fb813be00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:23:01.887 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:01.883+0000 7f8fb7fff640 1 -- 172.21.15.143:0/4173775527 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8fa80094d0 con 0x7f8fb806d840 2023-11-26T07:23:01.887 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:01.883+0000 7f8fb7fff640 1 --2- 172.21.15.143:0/4173775527 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f8fb806d840 0x7f8fb815a770 secure :-1 s=READY pgs=1039 cs=0 l=1 rev1=1 crypto rx=0x7f8fa80ae040 tx=0x7f8fa801aa50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:23:01.888 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:01.883+0000 7f8fb4ff9640 1 -- 172.21.15.143:0/4173775527 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8fa80b6420 con 0x7f8fb806d840 2023-11-26T07:23:01.888 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:01.883+0000 7f8fb4ff9640 1 -- 172.21.15.143:0/4173775527 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8fa80b65a0 con 0x7f8fb806d840 2023-11-26T07:23:01.888 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:01.883+0000 7f8fbd274640 1 -- 172.21.15.143:0/4173775527 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8fb8143520 con 0x7f8fb806d840 2023-11-26T07:23:01.888 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:01.883+0000 7f8fb4ff9640 1 -- 172.21.15.143:0/4173775527 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8fa80b6950 con 0x7f8fb806d840 2023-11-26T07:23:01.888 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:01.883+0000 7f8fbd274640 1 -- 172.21.15.143:0/4173775527 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8fb815acb0 con 0x7f8fb806d840 2023-11-26T07:23:01.888 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:01.887+0000 7f8fbd274640 1 -- 172.21.15.143:0/4173775527 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8fb8067500 con 0x7f8fb806d840 2023-11-26T07:23:01.889 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:01.887+0000 7f8fb4ff9640 1 -- 172.21.15.143:0/4173775527 <== mon.0 v2:172.21.15.143:3300/0 4 ==== mgrmap(e 11) v1 ==== 86158+0+0 (secure 0 0 0) 0x7f8fa80b2040 con 0x7f8fb806d840 2023-11-26T07:23:01.889 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:01.887+0000 7f8fb4ff9640 1 --2- 172.21.15.143:0/4173775527 >> [v2:172.21.15.143:6832/2150557544,v1:172.21.15.143:6833/2150557544] conn(0x7f8fa4066800 0x7f8fa4068cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:23:01.889 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:01.887+0000 7f8fb4ff9640 1 -- 172.21.15.143:0/4173775527 <== mon.0 v2:172.21.15.143:3300/0 5 ==== osd_map(71..71 src has 1..71) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f8fa811e580 con 0x7f8fb806d840 2023-11-26T07:23:01.889 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:01.887+0000 7f8fb77fe640 1 --2- 172.21.15.143:0/4173775527 >> [v2:172.21.15.143:6832/2150557544,v1:172.21.15.143:6833/2150557544] conn(0x7f8fa4066800 0x7f8fa4068cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:23:01.890 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:01.887+0000 7f8fb77fe640 1 --2- 172.21.15.143:0/4173775527 >> [v2:172.21.15.143:6832/2150557544,v1:172.21.15.143:6833/2150557544] conn(0x7f8fa4066800 0x7f8fa4068cc0 secure :-1 s=READY pgs=481 cs=0 l=1 rev1=1 crypto rx=0x7f8fb80ca330 tx=0x7f8fac009190 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:23:01.893 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:01.891+0000 7f8fb4ff9640 1 -- 172.21.15.143:0/4173775527 <== mon.0 v2:172.21.15.143:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172649 (secure 0 0 0) 0x7f8fa80eb4e0 con 0x7f8fb806d840 2023-11-26T07:23:02.055 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:02.051+0000 7f8fbd274640 1 -- 172.21.15.143:0/4173775527 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) v1 -- 0x7f8fb80cb5b0 con 0x7f8fb806d840 2023-11-26T07:23:02.055 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:02.055+0000 7f8fb4ff9640 1 -- 172.21.15.143:0/4173775527 <== mon.0 v2:172.21.15.143:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v71) v1 ==== 80+0+0 (secure 0 0 0) 0x7f8fa80eeb00 con 0x7f8fb806d840 2023-11-26T07:23:02.055 INFO:teuthology.orchestra.run.smithi143.stderr:listed 0 entries 2023-11-26T07:23:02.057 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:02.055+0000 7f8fbd274640 1 -- 172.21.15.143:0/4173775527 >> [v2:172.21.15.143:6832/2150557544,v1:172.21.15.143:6833/2150557544] conn(0x7f8fa4066800 msgr2=0x7f8fa4068cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:23:02.057 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:02.055+0000 7f8fbd274640 1 --2- 172.21.15.143:0/4173775527 >> [v2:172.21.15.143:6832/2150557544,v1:172.21.15.143:6833/2150557544] conn(0x7f8fa4066800 0x7f8fa4068cc0 secure :-1 s=READY pgs=481 cs=0 l=1 rev1=1 crypto rx=0x7f8fb80ca330 tx=0x7f8fac009190 comp rx=0 tx=0).stop 2023-11-26T07:23:02.057 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:02.055+0000 7f8fbd274640 1 -- 172.21.15.143:0/4173775527 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f8fb806d840 msgr2=0x7f8fb815a770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:23:02.057 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:02.055+0000 7f8fbd274640 1 --2- 172.21.15.143:0/4173775527 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f8fb806d840 0x7f8fb815a770 secure :-1 s=READY pgs=1039 cs=0 l=1 rev1=1 crypto rx=0x7f8fa80ae040 tx=0x7f8fa801aa50 comp rx=0 tx=0).stop 2023-11-26T07:23:02.058 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:02.055+0000 7f8fbd274640 1 -- 172.21.15.143:0/4173775527 shutdown_connections 2023-11-26T07:23:02.058 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:02.055+0000 7f8fbd274640 1 --2- 172.21.15.143:0/4173775527 >> [v2:172.21.15.143:6832/2150557544,v1:172.21.15.143:6833/2150557544] conn(0x7f8fa4066800 0x7f8fa4068cc0 unknown :-1 s=CLOSED pgs=481 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:23:02.058 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:02.055+0000 7f8fbd274640 1 --2- 172.21.15.143:0/4173775527 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f8fb8144720 0x7f8fb813c340 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:23:02.058 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:02.055+0000 7f8fbd274640 1 --2- 172.21.15.143:0/4173775527 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f8fb8071a20 0x7f8fb813be00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:23:02.058 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:02.055+0000 7f8fbd274640 1 --2- 172.21.15.143:0/4173775527 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f8fb806d840 0x7f8fb815a770 unknown :-1 s=CLOSED pgs=1039 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:23:02.058 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:02.055+0000 7f8fbd274640 1 -- 172.21.15.143:0/4173775527 >> 172.21.15.143:0/4173775527 conn(0x7f8fb806a890 msgr2=0x7f8fb80b0af0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:23:02.058 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:02.055+0000 7f8fbd274640 1 -- 172.21.15.143:0/4173775527 shutdown_connections 2023-11-26T07:23:02.058 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:02.055+0000 7f8fbd274640 1 -- 172.21.15.143:0/4173775527 wait complete. 2023-11-26T07:23:02.076 INFO:tasks.cephfs.fuse_mount:Running fusermount -u on ubuntu@smithi188.front.sepia.ceph.com... 2023-11-26T07:23:02.076 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-26T07:23:02.076 DEBUG:teuthology.orchestra.run.smithi188:> sudo fusermount -u /home/ubuntu/cephtest/mnt.1 2023-11-26T07:23:02.097 INFO:tasks.cephfs.fuse_mount.ceph-fuse.1.smithi188.stderr:ceph-fuse[44878]: fuse finished with error 0 and tester_r 0 2023-11-26T07:23:02.099 INFO:teuthology.orchestra.run:waiting for 300 2023-11-26T07:23:08.201 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi188.front.sepia.ceph.com 2023-11-26T07:23:08.201 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-26T07:23:08.202 DEBUG:teuthology.orchestra.run.smithi188:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.1) 2023-11-26T07:23:08.213 DEBUG:teuthology.orchestra.run.smithi143:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-11-26T07:23:08.360 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:08.359+0000 7fb7be7df640 1 Processor -- start 2023-11-26T07:23:08.360 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:08.359+0000 7fb7be7df640 1 -- start start 2023-11-26T07:23:08.361 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:08.359+0000 7fb7be7df640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fb7b80d17f0 0x7fb7b80d1c30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:23:08.361 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:08.359+0000 7fb7be7df640 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7fb7b80b0490 0x7fb7b80b0920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:23:08.361 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:08.359+0000 7fb7be7df640 1 -- --> v1:172.21.15.143:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb7b80d0bc0 con 0x7fb7b815d580 2023-11-26T07:23:08.362 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:08.359+0000 7fb7be7df640 1 -- --> v2:172.21.15.143:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb7b80d0320 con 0x7fb7b80b0490 2023-11-26T07:23:08.362 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:08.359+0000 7fb7be7df640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb7b80d0600 con 0x7fb7b80d17f0 2023-11-26T07:23:08.362 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:08.359+0000 7fb7b7fff640 1 --1- >> v1:172.21.15.143:6789/0 conn(0x7fb7b815d580 0x7fb7b80d3920 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.143:6789/0 says I am v1:172.21.15.143:54452/0 (socket says 172.21.15.143:54452) 2023-11-26T07:23:08.362 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:08.359+0000 7fb7b7fff640 1 -- 172.21.15.143:0/3763277001 learned_addr learned my addr 172.21.15.143:0/3763277001 (peer_addr_for_me v1:172.21.15.143:0/0) 2023-11-26T07:23:08.362 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:08.359+0000 7fb7bcd7c640 1 --2- 172.21.15.143:0/3763277001 >> v2:172.21.15.143:3300/0 conn(0x7fb7b80b0490 0x7fb7b80b0920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:23:08.362 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:08.359+0000 7fb7b77fe640 1 --2- 172.21.15.143:0/3763277001 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fb7b80d17f0 0x7fb7b80d1c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:23:08.362 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:08.359+0000 7fb7bcd7c640 1 -- 172.21.15.143:0/3763277001 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fb7b80d17f0 msgr2=0x7fb7b80d1c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-26T07:23:08.362 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:08.359+0000 7fb7bcd7c640 1 --2- 172.21.15.143:0/3763277001 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fb7b80d17f0 0x7fb7b80d1c30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:23:08.362 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:08.359+0000 7fb7bcd7c640 1 -- 172.21.15.143:0/3763277001 >> v1:172.21.15.143:6789/0 conn(0x7fb7b815d580 legacy=0x7fb7b80d3920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:23:08.362 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:08.359+0000 7fb7bcd7c640 1 -- 172.21.15.143:0/3763277001 --> v2:172.21.15.143:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb7b80d08e0 con 0x7fb7b80b0490 2023-11-26T07:23:08.362 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:08.359+0000 7fb7b6ffd640 1 -- 172.21.15.143:0/3763277001 <== mon.0 v1:172.21.15.143:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1123741399 0 0) 0x7fb7b80d0bc0 con 0x7fb7b815d580 2023-11-26T07:23:08.363 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:08.359+0000 7fb7b77fe640 1 --2- 172.21.15.143:0/3763277001 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fb7b80d17f0 0x7fb7b80d1c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-26T07:23:08.364 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:08.363+0000 7fb7bcd7c640 1 --2- 172.21.15.143:0/3763277001 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fb7b80b0490 0x7fb7b80b0920 secure :-1 s=READY pgs=1041 cs=0 l=1 rev1=1 crypto rx=0x7fb7b0009d90 tx=0x7fb7b001c920 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a294e9b932a9786 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:23:08.365 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:08.363+0000 7fb7b6ffd640 1 -- 172.21.15.143:0/3763277001 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb7b00a59c0 con 0x7fb7b80b0490 2023-11-26T07:23:08.365 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:08.363+0000 7fb7b6ffd640 1 -- 172.21.15.143:0/3763277001 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb7b00a5b20 con 0x7fb7b80b0490 2023-11-26T07:23:08.365 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:08.363+0000 7fb7b6ffd640 1 -- 172.21.15.143:0/3763277001 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb7b00ad440 con 0x7fb7b80b0490 2023-11-26T07:23:08.366 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:08.363+0000 7fb7be7df640 1 -- 172.21.15.143:0/3763277001 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fb7b80b0490 msgr2=0x7fb7b80b0920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:23:08.366 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:08.363+0000 7fb7be7df640 1 --2- 172.21.15.143:0/3763277001 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fb7b80b0490 0x7fb7b80b0920 secure :-1 s=READY pgs=1041 cs=0 l=1 rev1=1 crypto rx=0x7fb7b0009d90 tx=0x7fb7b001c920 comp rx=0 tx=0).stop 2023-11-26T07:23:08.366 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:08.363+0000 7fb7be7df640 1 -- 172.21.15.143:0/3763277001 shutdown_connections 2023-11-26T07:23:08.366 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:08.363+0000 7fb7be7df640 1 --2- 172.21.15.143:0/3763277001 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fb7b80b0490 0x7fb7b80b0920 unknown :-1 s=CLOSED pgs=1041 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:23:08.366 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:08.363+0000 7fb7be7df640 1 --2- 172.21.15.143:0/3763277001 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fb7b80d17f0 0x7fb7b80d1c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:23:08.366 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:08.363+0000 7fb7be7df640 1 -- 172.21.15.143:0/3763277001 >> 172.21.15.143:0/3763277001 conn(0x7fb7b809c550 msgr2=0x7fb7b80d3210 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:23:08.366 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:08.363+0000 7fb7be7df640 1 -- 172.21.15.143:0/3763277001 shutdown_connections 2023-11-26T07:23:08.366 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:08.363+0000 7fb7be7df640 1 -- 172.21.15.143:0/3763277001 wait complete. 2023-11-26T07:23:08.367 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:08.367+0000 7fb7be7df640 1 Processor -- start 2023-11-26T07:23:08.367 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:08.367+0000 7fb7be7df640 1 -- start start 2023-11-26T07:23:08.369 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:08.367+0000 7fb7be7df640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fb7b80b0490 0x7fb7b815f310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:23:08.369 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:08.367+0000 7fb7be7df640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fb7b80d17f0 0x7fb7b8159770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:23:08.370 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:08.367+0000 7fb7be7df640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fb7b815d580 0x7fb7b8157790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:23:08.370 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:08.367+0000 7fb7be7df640 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb7b815eef0 con 0x7fb7b80d17f0 2023-11-26T07:23:08.370 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:08.367+0000 7fb7be7df640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb7b8153780 con 0x7fb7b80b0490 2023-11-26T07:23:08.370 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:08.367+0000 7fb7be7df640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb7b815e890 con 0x7fb7b815d580 2023-11-26T07:23:08.370 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:08.367+0000 7fb7b77fe640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fb7b80d17f0 0x7fb7b8159770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:23:08.370 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:08.367+0000 7fb7b77fe640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fb7b80d17f0 0x7fb7b8159770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:46474/0 (socket says 172.21.15.143:46474) 2023-11-26T07:23:08.370 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:08.367+0000 7fb7b77fe640 1 -- 172.21.15.143:0/3351204336 learned_addr learned my addr 172.21.15.143:0/3351204336 (peer_addr_for_me v2:172.21.15.143:0/0) 2023-11-26T07:23:08.370 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:08.367+0000 7fb7bcd7c640 1 --2- 172.21.15.143:0/3351204336 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fb7b815d580 0x7fb7b8157790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:23:08.370 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:08.367+0000 7fb7b7fff640 1 --2- 172.21.15.143:0/3351204336 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fb7b80b0490 0x7fb7b815f310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:23:08.370 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:08.367+0000 7fb7b77fe640 1 -- 172.21.15.143:0/3351204336 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fb7b815d580 msgr2=0x7fb7b8157790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:23:08.371 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:08.367+0000 7fb7b77fe640 1 --2- 172.21.15.143:0/3351204336 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fb7b815d580 0x7fb7b8157790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:23:08.371 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:08.367+0000 7fb7b77fe640 1 -- 172.21.15.143:0/3351204336 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fb7b80b0490 msgr2=0x7fb7b815f310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:23:08.371 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:08.367+0000 7fb7b77fe640 1 --2- 172.21.15.143:0/3351204336 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fb7b80b0490 0x7fb7b815f310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:23:08.371 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:08.367+0000 7fb7b77fe640 1 -- 172.21.15.143:0/3351204336 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb7b0009a70 con 0x7fb7b80d17f0 2023-11-26T07:23:08.371 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:08.367+0000 7fb7bcd7c640 1 --2- 172.21.15.143:0/3351204336 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fb7b815d580 0x7fb7b8157790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-26T07:23:08.371 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:08.367+0000 7fb7b7fff640 1 --2- 172.21.15.143:0/3351204336 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fb7b80b0490 0x7fb7b815f310 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-26T07:23:08.371 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:08.367+0000 7fb7b77fe640 1 --2- 172.21.15.143:0/3351204336 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fb7b80d17f0 0x7fb7b8159770 secure :-1 s=READY pgs=1042 cs=0 l=1 rev1=1 crypto rx=0x7fb7ac00e840 tx=0x7fb7ac00ed00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:23:08.371 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:08.367+0000 7fb7b4ff9640 1 -- 172.21.15.143:0/3351204336 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb7ac00c8d0 con 0x7fb7b80d17f0 2023-11-26T07:23:08.371 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:08.367+0000 7fb7b4ff9640 1 -- 172.21.15.143:0/3351204336 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb7ac00ca30 con 0x7fb7b80d17f0 2023-11-26T07:23:08.371 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:08.367+0000 7fb7b4ff9640 1 -- 172.21.15.143:0/3351204336 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb7ac00cde0 con 0x7fb7b80d17f0 2023-11-26T07:23:08.371 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:08.367+0000 7fb7be7df640 1 -- 172.21.15.143:0/3351204336 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb7b80b8ba0 con 0x7fb7b80d17f0 2023-11-26T07:23:08.371 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:08.367+0000 7fb7be7df640 1 -- 172.21.15.143:0/3351204336 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb7b815eb10 con 0x7fb7b80d17f0 2023-11-26T07:23:08.372 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:08.367+0000 7fb7be7df640 1 -- 172.21.15.143:0/3351204336 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb7b8067680 con 0x7fb7b80d17f0 2023-11-26T07:23:08.372 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:08.371+0000 7fb7b4ff9640 1 -- 172.21.15.143:0/3351204336 <== mon.0 v2:172.21.15.143:3300/0 4 ==== mgrmap(e 11) v1 ==== 86158+0+0 (secure 0 0 0) 0x7fb7ac013410 con 0x7fb7b80d17f0 2023-11-26T07:23:08.372 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:08.371+0000 7fb7b4ff9640 1 --2- 172.21.15.143:0/3351204336 >> [v2:172.21.15.143:6832/2150557544,v1:172.21.15.143:6833/2150557544] conn(0x7fb7a40667d0 0x7fb7a4068c90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:23:08.373 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:08.371+0000 7fb7b4ff9640 1 -- 172.21.15.143:0/3351204336 <== mon.0 v2:172.21.15.143:3300/0 5 ==== osd_map(71..71 src has 1..71) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fb7ac084330 con 0x7fb7b80d17f0 2023-11-26T07:23:08.373 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:08.371+0000 7fb7b7fff640 1 --2- 172.21.15.143:0/3351204336 >> [v2:172.21.15.143:6832/2150557544,v1:172.21.15.143:6833/2150557544] conn(0x7fb7a40667d0 0x7fb7a4068c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:23:08.373 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:08.371+0000 7fb7b7fff640 1 --2- 172.21.15.143:0/3351204336 >> [v2:172.21.15.143:6832/2150557544,v1:172.21.15.143:6833/2150557544] conn(0x7fb7a40667d0 0x7fb7a4068c90 secure :-1 s=READY pgs=482 cs=0 l=1 rev1=1 crypto rx=0x7fb7a8007fd0 tx=0x7fb7a8007c20 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:23:08.376 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:08.375+0000 7fb7b4ff9640 1 -- 172.21.15.143:0/3351204336 <== mon.0 v2:172.21.15.143:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172649 (secure 0 0 0) 0x7fb7ac051250 con 0x7fb7b80d17f0 2023-11-26T07:23:08.559 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:08.555+0000 7fb7be7df640 1 -- 172.21.15.143:0/3351204336 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fb7b815a120 con 0x7fb7b80d17f0 2023-11-26T07:23:08.559 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:08.559+0000 7fb7b4ff9640 1 -- 172.21.15.143:0/3351204336 <== mon.0 v2:172.21.15.143:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 43 v43) v1 ==== 94+0+4924 (secure 0 0 0) 0x7fb7ac054870 con 0x7fb7b80d17f0 2023-11-26T07:23:08.559 INFO:teuthology.orchestra.run.smithi143.stdout: 2023-11-26T07:23:08.560 INFO:teuthology.orchestra.run.smithi143.stdout:{"epoch":43,"default_fscid":5,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5927,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.143:6837/578144260","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6835","nonce":578144260},{"type":"v1","addr":"172.21.15.143:6837","nonce":578144260}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":38},{"gid":5930,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.188:6834/1048736873","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6832","nonce":1048736873},{"type":"v1","addr":"172.21.15.188:6834","nonce":1048736873}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":38},{"gid":5933,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.188:6835/2063768716","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6833","nonce":2063768716},{"type":"v1","addr":"172.21.15.188:6835","nonce":2063768716}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":38}],"filesystems":[{"mdsmap":{"epoch":42,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-11-26T07:22:37.220113+0000","modified":"2023-11-26T07:22:40.243052+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":6553},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6553":{"gid":6553,"name":"a","rank":0,"incarnation":39,"state":"up:active","state_seq":3,"addr":"172.21.15.143:6836/2590663349","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6834","nonce":2590663349},{"type":"v1","addr":"172.21.15.143:6836","nonce":2590663349}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[11],"metadata_pool":10,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":5}]} 2023-11-26T07:23:08.560 INFO:teuthology.orchestra.run.smithi143.stderr:dumped fsmap epoch 43 2023-11-26T07:23:08.561 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:08.559+0000 7fb7be7df640 1 -- 172.21.15.143:0/3351204336 >> [v2:172.21.15.143:6832/2150557544,v1:172.21.15.143:6833/2150557544] conn(0x7fb7a40667d0 msgr2=0x7fb7a4068c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:23:08.561 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:08.559+0000 7fb7be7df640 1 --2- 172.21.15.143:0/3351204336 >> [v2:172.21.15.143:6832/2150557544,v1:172.21.15.143:6833/2150557544] conn(0x7fb7a40667d0 0x7fb7a4068c90 secure :-1 s=READY pgs=482 cs=0 l=1 rev1=1 crypto rx=0x7fb7a8007fd0 tx=0x7fb7a8007c20 comp rx=0 tx=0).stop 2023-11-26T07:23:08.561 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:08.559+0000 7fb7be7df640 1 -- 172.21.15.143:0/3351204336 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fb7b80d17f0 msgr2=0x7fb7b8159770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:23:08.562 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:08.559+0000 7fb7be7df640 1 --2- 172.21.15.143:0/3351204336 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fb7b80d17f0 0x7fb7b8159770 secure :-1 s=READY pgs=1042 cs=0 l=1 rev1=1 crypto rx=0x7fb7ac00e840 tx=0x7fb7ac00ed00 comp rx=0 tx=0).stop 2023-11-26T07:23:08.562 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:08.559+0000 7fb7be7df640 1 -- 172.21.15.143:0/3351204336 shutdown_connections 2023-11-26T07:23:08.562 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:08.559+0000 7fb7be7df640 1 --2- 172.21.15.143:0/3351204336 >> [v2:172.21.15.143:6832/2150557544,v1:172.21.15.143:6833/2150557544] conn(0x7fb7a40667d0 0x7fb7a4068c90 unknown :-1 s=CLOSED pgs=482 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:23:08.562 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:08.559+0000 7fb7be7df640 1 --2- 172.21.15.143:0/3351204336 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fb7b815d580 0x7fb7b8157790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:23:08.562 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:08.559+0000 7fb7be7df640 1 --2- 172.21.15.143:0/3351204336 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fb7b80d17f0 0x7fb7b8159770 unknown :-1 s=CLOSED pgs=1042 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:23:08.562 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:08.559+0000 7fb7be7df640 1 --2- 172.21.15.143:0/3351204336 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fb7b80b0490 0x7fb7b815f310 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:23:08.562 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:08.559+0000 7fb7be7df640 1 -- 172.21.15.143:0/3351204336 >> 172.21.15.143:0/3351204336 conn(0x7fb7b809c550 msgr2=0x7fb7b815c850 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:23:08.562 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:08.559+0000 7fb7be7df640 1 -- 172.21.15.143:0/3351204336 shutdown_connections 2023-11-26T07:23:08.562 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:08.559+0000 7fb7be7df640 1 -- 172.21.15.143:0/3351204336 wait complete. 2023-11-26T07:23:08.580 DEBUG:teuthology.orchestra.run.smithi143:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.a.asok --format=json session ls 2023-11-26T07:23:08.738 INFO:teuthology.orchestra.run.smithi143.stdout:[] 2023-11-26T07:23:08.738 DEBUG:tasks.cephfs.filesystem:_json_asok output [] 2023-11-26T07:23:08.738 DEBUG:teuthology.orchestra.run.smithi143:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume create cephfs subvol31960 2023-11-26T07:23:08.881 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:08.879+0000 7f61fb557640 1 Processor -- start 2023-11-26T07:23:08.881 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:08.879+0000 7f61fb557640 1 -- start start 2023-11-26T07:23:08.883 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:08.879+0000 7f61fb557640 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7f61f413e290 0x7f61f40cf280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:23:08.883 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:08.879+0000 7f61fb557640 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7f61f40cf7c0 0x7f61f415efa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:23:08.883 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:08.879+0000 7f61fb557640 1 -- --> v1:172.21.15.188:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f61f406c520 con 0x7f61f415f4e0 2023-11-26T07:23:08.884 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:08.879+0000 7f61fb557640 1 -- --> v2:172.21.15.143:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f61f40cfc40 con 0x7f61f40cf7c0 2023-11-26T07:23:08.884 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:08.879+0000 7f61fb557640 1 -- --> v2:172.21.15.188:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f61f406c800 con 0x7f61f413e290 2023-11-26T07:23:08.884 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:08.879+0000 7f61f8af2640 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7f61f40cf7c0 0x7f61f415efa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:23:08.884 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:08.883+0000 7f61f8af2640 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7f61f40cf7c0 0x7f61f415efa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:46484/0 (socket says 172.21.15.143:46484) 2023-11-26T07:23:08.884 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:08.883+0000 7f61f8af2640 1 -- 172.21.15.143:0/1673271187 learned_addr learned my addr 172.21.15.143:0/1673271187 (peer_addr_for_me v2:172.21.15.143:0/0) 2023-11-26T07:23:08.884 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:08.883+0000 7f61f92f3640 1 --2- 172.21.15.143:0/1673271187 >> v2:172.21.15.188:3300/0 conn(0x7f61f413e290 0x7f61f40cf280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:23:08.884 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:08.883+0000 7f61f8af2640 1 -- 172.21.15.143:0/1673271187 >> v2:172.21.15.188:3300/0 conn(0x7f61f413e290 msgr2=0x7f61f40cf280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-26T07:23:08.884 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:08.883+0000 7f61f8af2640 1 --2- 172.21.15.143:0/1673271187 >> v2:172.21.15.188:3300/0 conn(0x7f61f413e290 0x7f61f40cf280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:23:08.884 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:08.883+0000 7f61f8af2640 1 -- 172.21.15.143:0/1673271187 >> v1:172.21.15.188:6789/0 conn(0x7f61f415f4e0 legacy=0x7f61f41408e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-26T07:23:08.884 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:08.883+0000 7f61f8af2640 1 -- 172.21.15.143:0/1673271187 --> v2:172.21.15.143:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f61f406cae0 con 0x7f61f40cf7c0 2023-11-26T07:23:08.886 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:08.883+0000 7f61f8af2640 1 --2- 172.21.15.143:0/1673271187 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f61f40cf7c0 0x7f61f415efa0 secure :-1 s=READY pgs=1043 cs=0 l=1 rev1=1 crypto rx=0x7f61f40cfe40 tx=0x7f61e401c470 comp rx=0 tx=0).ready entity=mon.0 client_cookie=283d37b5f534e92f server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:23:08.886 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:08.883+0000 7f61ebfff640 1 -- 172.21.15.143:0/1673271187 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f61e40ab670 con 0x7f61f40cf7c0 2023-11-26T07:23:08.887 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:08.883+0000 7f61ebfff640 1 -- 172.21.15.143:0/1673271187 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f61e40ab7d0 con 0x7f61f40cf7c0 2023-11-26T07:23:08.887 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:08.883+0000 7f61ebfff640 1 -- 172.21.15.143:0/1673271187 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f61e40abb60 con 0x7f61f40cf7c0 2023-11-26T07:23:08.887 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:08.887+0000 7f61fb557640 1 -- 172.21.15.143:0/1673271187 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f61f40cf7c0 msgr2=0x7f61f415efa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:23:08.888 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:08.887+0000 7f61fb557640 1 --2- 172.21.15.143:0/1673271187 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f61f40cf7c0 0x7f61f415efa0 secure :-1 s=READY pgs=1043 cs=0 l=1 rev1=1 crypto rx=0x7f61f40cfe40 tx=0x7f61e401c470 comp rx=0 tx=0).stop 2023-11-26T07:23:08.888 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:08.887+0000 7f61fb557640 1 -- 172.21.15.143:0/1673271187 shutdown_connections 2023-11-26T07:23:08.888 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:08.887+0000 7f61fb557640 1 --2- 172.21.15.143:0/1673271187 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f61f40cf7c0 0x7f61f415efa0 unknown :-1 s=CLOSED pgs=1043 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:23:08.888 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:08.887+0000 7f61fb557640 1 --2- 172.21.15.143:0/1673271187 >> v2:172.21.15.188:3300/0 conn(0x7f61f413e290 0x7f61f40cf280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:23:08.888 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:08.887+0000 7f61fb557640 1 -- 172.21.15.143:0/1673271187 >> 172.21.15.143:0/1673271187 conn(0x7f61f406a890 msgr2=0x7f61f406aca0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:23:08.888 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:08.887+0000 7f61fb557640 1 -- 172.21.15.143:0/1673271187 shutdown_connections 2023-11-26T07:23:08.888 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:08.887+0000 7f61fb557640 1 -- 172.21.15.143:0/1673271187 wait complete. 2023-11-26T07:23:08.889 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:08.887+0000 7f61fb557640 1 Processor -- start 2023-11-26T07:23:08.889 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:08.887+0000 7f61fb557640 1 -- start start 2023-11-26T07:23:08.891 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:08.887+0000 7f61fb557640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f61f40cf7c0 0x7f61f415c350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:23:08.891 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:08.887+0000 7f61fb557640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f61f413e290 0x7f61f4137400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:23:08.891 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:08.887+0000 7f61fb557640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f61f415f4e0 0x7f61f4152960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:23:08.891 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:08.887+0000 7f61fb557640 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f61f4137940 con 0x7f61f413e290 2023-11-26T07:23:08.891 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:08.887+0000 7f61fb557640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f61f415dea0 con 0x7f61f415f4e0 2023-11-26T07:23:08.891 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:08.887+0000 7f61fb557640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f61f415e180 con 0x7f61f40cf7c0 2023-11-26T07:23:08.891 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:08.887+0000 7f61f8af2640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f61f413e290 0x7f61f4137400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:23:08.891 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:08.887+0000 7f61f8af2640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f61f413e290 0x7f61f4137400 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:46492/0 (socket says 172.21.15.143:46492) 2023-11-26T07:23:08.891 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:08.887+0000 7f61f92f3640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f61f40cf7c0 0x7f61f415c350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:23:08.892 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:08.887+0000 7f61f8af2640 1 -- 172.21.15.143:0/3376601928 learned_addr learned my addr 172.21.15.143:0/3376601928 (peer_addr_for_me v2:172.21.15.143:0/0) 2023-11-26T07:23:08.892 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:08.887+0000 7f61f9af4640 1 --2- 172.21.15.143:0/3376601928 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f61f415f4e0 0x7f61f4152960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:23:08.892 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:08.887+0000 7f61f8af2640 1 -- 172.21.15.143:0/3376601928 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f61f40cf7c0 msgr2=0x7f61f415c350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:23:08.892 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:08.887+0000 7f61f8af2640 1 --2- 172.21.15.143:0/3376601928 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f61f40cf7c0 0x7f61f415c350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:23:08.892 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:08.887+0000 7f61f8af2640 1 -- 172.21.15.143:0/3376601928 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f61f415f4e0 msgr2=0x7f61f4152960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:23:08.892 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:08.887+0000 7f61f8af2640 1 --2- 172.21.15.143:0/3376601928 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f61f415f4e0 0x7f61f4152960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:23:08.892 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:08.887+0000 7f61f8af2640 1 -- 172.21.15.143:0/3376601928 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f61e40094d0 con 0x7f61f413e290 2023-11-26T07:23:08.892 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:08.891+0000 7f61f8af2640 1 --2- 172.21.15.143:0/3376601928 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f61f413e290 0x7f61f4137400 secure :-1 s=READY pgs=1044 cs=0 l=1 rev1=1 crypto rx=0x7f61e401a0a0 tx=0x7f61e401acb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:23:08.893 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:08.891+0000 7f61e9ffb640 1 -- 172.21.15.143:0/3376601928 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f61e40b4800 con 0x7f61f413e290 2023-11-26T07:23:08.893 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:08.891+0000 7f61e9ffb640 1 -- 172.21.15.143:0/3376601928 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f61e40b4960 con 0x7f61f413e290 2023-11-26T07:23:08.893 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:08.891+0000 7f61fb557640 1 -- 172.21.15.143:0/3376601928 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f61f415e460 con 0x7f61f413e290 2023-11-26T07:23:08.893 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:08.891+0000 7f61e9ffb640 1 -- 172.21.15.143:0/3376601928 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f61e40b4cd0 con 0x7f61f413e290 2023-11-26T07:23:08.893 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:08.891+0000 7f61fb557640 1 -- 172.21.15.143:0/3376601928 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f61f40aecd0 con 0x7f61f413e290 2023-11-26T07:23:08.894 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:08.891+0000 7f61fb557640 1 -- 172.21.15.143:0/3376601928 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f61f4067680 con 0x7f61f413e290 2023-11-26T07:23:08.894 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:08.891+0000 7f61e9ffb640 1 -- 172.21.15.143:0/3376601928 <== mon.0 v2:172.21.15.143:3300/0 4 ==== mgrmap(e 11) v1 ==== 86158+0+0 (secure 0 0 0) 0x7f61e40b3380 con 0x7f61f413e290 2023-11-26T07:23:08.894 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:08.891+0000 7f61e9ffb640 1 --2- 172.21.15.143:0/3376601928 >> [v2:172.21.15.143:6832/2150557544,v1:172.21.15.143:6833/2150557544] conn(0x7f61d80667d0 0x7f61d8068c90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:23:08.894 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:08.891+0000 7f61e9ffb640 1 -- 172.21.15.143:0/3376601928 <== mon.0 v2:172.21.15.143:3300/0 5 ==== osd_map(71..71 src has 1..71) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f61e411ec20 con 0x7f61f413e290 2023-11-26T07:23:08.894 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:08.891+0000 7f61f92f3640 1 --2- 172.21.15.143:0/3376601928 >> [v2:172.21.15.143:6832/2150557544,v1:172.21.15.143:6833/2150557544] conn(0x7f61d80667d0 0x7f61d8068c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:23:08.895 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:08.895+0000 7f61f92f3640 1 --2- 172.21.15.143:0/3376601928 >> [v2:172.21.15.143:6832/2150557544,v1:172.21.15.143:6833/2150557544] conn(0x7f61d80667d0 0x7f61d8068c90 secure :-1 s=READY pgs=483 cs=0 l=1 rev1=1 crypto rx=0x7f61f41551f0 tx=0x7f61f0008040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:23:08.898 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:08.895+0000 7f61e9ffb640 1 -- 172.21.15.143:0/3376601928 <== mon.0 v2:172.21.15.143:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172649 (secure 0 0 0) 0x7f61e40ebbf0 con 0x7f61f413e290 2023-11-26T07:23:09.069 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:09.067+0000 7f61fb557640 1 -- 172.21.15.143:0/3376601928 --> [v2:172.21.15.143:6832/2150557544,v1:172.21.15.143:6833/2150557544] -- mgr_command(tid 0: {"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "subvol31960", "target": ["mon-mgr", ""]}) v1 -- 0x7f61f406df50 con 0x7f61d80667d0 2023-11-26T07:23:09.076 INFO:tasks.ceph.mgr.y.smithi143.stderr:2023-11-26T07:23:09.075+0000 7f5048ac6640 -1 client.0 error registering admin socket command: (17) File exists 2023-11-26T07:23:09.076 INFO:tasks.ceph.mgr.y.smithi143.stderr:2023-11-26T07:23:09.075+0000 7f5048ac6640 -1 client.0 error registering admin socket command: (17) File exists 2023-11-26T07:23:09.076 INFO:tasks.ceph.mgr.y.smithi143.stderr:2023-11-26T07:23:09.075+0000 7f5048ac6640 -1 client.0 error registering admin socket command: (17) File exists 2023-11-26T07:23:09.076 INFO:tasks.ceph.mgr.y.smithi143.stderr:2023-11-26T07:23:09.075+0000 7f5048ac6640 -1 client.0 error registering admin socket command: (17) File exists 2023-11-26T07:23:09.076 INFO:tasks.ceph.mgr.y.smithi143.stderr:2023-11-26T07:23:09.075+0000 7f5048ac6640 -1 client.0 error registering admin socket command: (17) File exists 2023-11-26T07:23:09.106 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:09.103+0000 7f61e9ffb640 1 -- 172.21.15.143:0/3376601928 <== mgr.4100 v2:172.21.15.143:6832/2150557544 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+0 (secure 0 0 0) 0x7f61f406df50 con 0x7f61d80667d0 2023-11-26T07:23:09.108 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:09.107+0000 7f61fb557640 1 -- 172.21.15.143:0/3376601928 >> [v2:172.21.15.143:6832/2150557544,v1:172.21.15.143:6833/2150557544] conn(0x7f61d80667d0 msgr2=0x7f61d8068c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:23:09.108 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:09.107+0000 7f61fb557640 1 --2- 172.21.15.143:0/3376601928 >> [v2:172.21.15.143:6832/2150557544,v1:172.21.15.143:6833/2150557544] conn(0x7f61d80667d0 0x7f61d8068c90 secure :-1 s=READY pgs=483 cs=0 l=1 rev1=1 crypto rx=0x7f61f41551f0 tx=0x7f61f0008040 comp rx=0 tx=0).stop 2023-11-26T07:23:09.108 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:09.107+0000 7f61fb557640 1 -- 172.21.15.143:0/3376601928 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f61f413e290 msgr2=0x7f61f4137400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:23:09.109 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:09.107+0000 7f61fb557640 1 --2- 172.21.15.143:0/3376601928 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f61f413e290 0x7f61f4137400 secure :-1 s=READY pgs=1044 cs=0 l=1 rev1=1 crypto rx=0x7f61e401a0a0 tx=0x7f61e401acb0 comp rx=0 tx=0).stop 2023-11-26T07:23:09.109 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:09.107+0000 7f61fb557640 1 -- 172.21.15.143:0/3376601928 shutdown_connections 2023-11-26T07:23:09.109 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:09.107+0000 7f61fb557640 1 --2- 172.21.15.143:0/3376601928 >> [v2:172.21.15.143:6832/2150557544,v1:172.21.15.143:6833/2150557544] conn(0x7f61d80667d0 0x7f61d8068c90 unknown :-1 s=CLOSED pgs=483 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:23:09.109 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:09.107+0000 7f61fb557640 1 --2- 172.21.15.143:0/3376601928 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f61f415f4e0 0x7f61f4152960 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:23:09.109 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:09.107+0000 7f61fb557640 1 --2- 172.21.15.143:0/3376601928 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f61f413e290 0x7f61f4137400 unknown :-1 s=CLOSED pgs=1044 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:23:09.109 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:09.107+0000 7f61fb557640 1 --2- 172.21.15.143:0/3376601928 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f61f40cf7c0 0x7f61f415c350 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:23:09.109 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:09.107+0000 7f61fb557640 1 -- 172.21.15.143:0/3376601928 >> 172.21.15.143:0/3376601928 conn(0x7f61f406a890 msgr2=0x7f61f413e700 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:23:09.109 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:09.107+0000 7f61fb557640 1 -- 172.21.15.143:0/3376601928 shutdown_connections 2023-11-26T07:23:09.109 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:09.107+0000 7f61fb557640 1 -- 172.21.15.143:0/3376601928 wait complete. 2023-11-26T07:23:09.127 DEBUG:teuthology.orchestra.run.smithi143:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-11-26T07:23:09.265 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:09.263+0000 7f3847054640 1 Processor -- start 2023-11-26T07:23:09.265 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:09.263+0000 7f3847054640 1 -- start start 2023-11-26T07:23:09.266 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:09.263+0000 7f3847054640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f384013e290 0x7f38400cf280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:23:09.266 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:09.263+0000 7f3847054640 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7f38400cf7c0 0x7f384015efa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:23:09.267 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:09.263+0000 7f3847054640 1 -- --> v1:172.21.15.143:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f384006c520 con 0x7f384015f4e0 2023-11-26T07:23:09.267 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:09.263+0000 7f3847054640 1 -- --> v2:172.21.15.143:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f38400cfc40 con 0x7f38400cf7c0 2023-11-26T07:23:09.267 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:09.263+0000 7f3847054640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f384006c800 con 0x7f384013e290 2023-11-26T07:23:09.267 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:09.263+0000 7f3837fff640 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7f38400cf7c0 0x7f384015efa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:23:09.267 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:09.263+0000 7f3837fff640 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7f38400cf7c0 0x7f384015efa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:46526/0 (socket says 172.21.15.143:46526) 2023-11-26T07:23:09.267 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:09.263+0000 7f3837fff640 1 -- 172.21.15.143:0/2848940860 learned_addr learned my addr 172.21.15.143:0/2848940860 (peer_addr_for_me v2:172.21.15.143:0/0) 2023-11-26T07:23:09.267 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:09.267+0000 7f3844df0640 1 --2- 172.21.15.143:0/2848940860 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f384013e290 0x7f38400cf280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:23:09.268 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:09.267+0000 7f3837fff640 1 -- 172.21.15.143:0/2848940860 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f384013e290 msgr2=0x7f38400cf280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-26T07:23:09.268 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:09.267+0000 7f3837fff640 1 --2- 172.21.15.143:0/2848940860 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f384013e290 0x7f38400cf280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:23:09.268 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:09.267+0000 7f3837fff640 1 -- 172.21.15.143:0/2848940860 >> v1:172.21.15.143:6789/0 conn(0x7f384015f4e0 legacy=0x7f38401408e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:23:09.268 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:09.267+0000 7f38377fe640 1 -- 172.21.15.143:0/2848940860 <== mon.0 v1:172.21.15.143:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 884052813 0 0) 0x7f384006c520 con 0x7f384015f4e0 2023-11-26T07:23:09.268 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:09.267+0000 7f3837fff640 1 -- 172.21.15.143:0/2848940860 --> v2:172.21.15.143:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f384006cae0 con 0x7f38400cf7c0 2023-11-26T07:23:09.270 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:09.267+0000 7f3837fff640 1 --2- 172.21.15.143:0/2848940860 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f38400cf7c0 0x7f384015efa0 secure :-1 s=READY pgs=1048 cs=0 l=1 rev1=1 crypto rx=0x7f3830009870 tx=0x7f383001c510 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e9366bb7c2ddcae5 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:23:09.270 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:09.267+0000 7f38377fe640 1 -- 172.21.15.143:0/2848940860 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f38300aa670 con 0x7f38400cf7c0 2023-11-26T07:23:09.270 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:09.267+0000 7f38377fe640 1 -- 172.21.15.143:0/2848940860 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f38300aa7d0 con 0x7f38400cf7c0 2023-11-26T07:23:09.270 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:09.267+0000 7f38377fe640 1 -- 172.21.15.143:0/2848940860 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f38300aab60 con 0x7f38400cf7c0 2023-11-26T07:23:09.271 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:09.271+0000 7f3847054640 1 -- 172.21.15.143:0/2848940860 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f38400cf7c0 msgr2=0x7f384015efa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:23:09.271 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:09.271+0000 7f3847054640 1 --2- 172.21.15.143:0/2848940860 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f38400cf7c0 0x7f384015efa0 secure :-1 s=READY pgs=1048 cs=0 l=1 rev1=1 crypto rx=0x7f3830009870 tx=0x7f383001c510 comp rx=0 tx=0).stop 2023-11-26T07:23:09.271 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:09.271+0000 7f3847054640 1 -- 172.21.15.143:0/2848940860 shutdown_connections 2023-11-26T07:23:09.271 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:09.271+0000 7f3847054640 1 --2- 172.21.15.143:0/2848940860 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f38400cf7c0 0x7f384015efa0 unknown :-1 s=CLOSED pgs=1048 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:23:09.271 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:09.271+0000 7f3847054640 1 --2- 172.21.15.143:0/2848940860 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f384013e290 0x7f38400cf280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:23:09.271 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:09.271+0000 7f3847054640 1 -- 172.21.15.143:0/2848940860 >> 172.21.15.143:0/2848940860 conn(0x7f384006a890 msgr2=0x7f384006aca0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:23:09.272 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:09.271+0000 7f3847054640 1 -- 172.21.15.143:0/2848940860 shutdown_connections 2023-11-26T07:23:09.272 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:09.271+0000 7f3847054640 1 -- 172.21.15.143:0/2848940860 wait complete. 2023-11-26T07:23:09.273 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:09.271+0000 7f3847054640 1 Processor -- start 2023-11-26T07:23:09.273 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:09.271+0000 7f3847054640 1 -- start start 2023-11-26T07:23:09.275 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:09.271+0000 7f3847054640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f38400cf7c0 0x7f3840137540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:23:09.275 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:09.271+0000 7f3847054640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f384013e290 0x7f384015ca30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:23:09.275 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:09.271+0000 7f3847054640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f384015f4e0 0x7f384015cf70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:23:09.275 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:09.271+0000 7f3847054640 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f384015e1b0 con 0x7f384013e290 2023-11-26T07:23:09.275 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:09.271+0000 7f3847054640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f384015d5a0 con 0x7f384015f4e0 2023-11-26T07:23:09.276 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:09.271+0000 7f3847054640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f384015d880 con 0x7f38400cf7c0 2023-11-26T07:23:09.276 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:09.271+0000 7f3837fff640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f384013e290 0x7f384015ca30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:23:09.276 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:09.271+0000 7f3837fff640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f384013e290 0x7f384015ca30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:46528/0 (socket says 172.21.15.143:46528) 2023-11-26T07:23:09.276 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:09.271+0000 7f3837fff640 1 -- 172.21.15.143:0/3151035145 learned_addr learned my addr 172.21.15.143:0/3151035145 (peer_addr_for_me v2:172.21.15.143:0/0) 2023-11-26T07:23:09.276 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:09.271+0000 7f3844df0640 1 --2- 172.21.15.143:0/3151035145 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f38400cf7c0 0x7f3840137540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:23:09.276 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:09.271+0000 7f38455f1640 1 --2- 172.21.15.143:0/3151035145 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f384015f4e0 0x7f384015cf70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:23:09.276 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:09.271+0000 7f3837fff640 1 -- 172.21.15.143:0/3151035145 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f38400cf7c0 msgr2=0x7f3840137540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:23:09.276 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:09.271+0000 7f3837fff640 1 --2- 172.21.15.143:0/3151035145 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f38400cf7c0 0x7f3840137540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:23:09.276 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:09.271+0000 7f3837fff640 1 -- 172.21.15.143:0/3151035145 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f384015f4e0 msgr2=0x7f384015cf70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:23:09.276 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:09.271+0000 7f3837fff640 1 --2- 172.21.15.143:0/3151035145 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f384015f4e0 0x7f384015cf70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:23:09.276 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:09.271+0000 7f3837fff640 1 -- 172.21.15.143:0/3151035145 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f38300094d0 con 0x7f384013e290 2023-11-26T07:23:09.276 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:09.271+0000 7f38455f1640 1 --2- 172.21.15.143:0/3151035145 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f384015f4e0 0x7f384015cf70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-26T07:23:09.276 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:09.275+0000 7f3837fff640 1 --2- 172.21.15.143:0/3151035145 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f384013e290 0x7f384015ca30 secure :-1 s=READY pgs=1049 cs=0 l=1 rev1=1 crypto rx=0x7f383001a140 tx=0x7f383001ad50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:23:09.277 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:09.275+0000 7f38357fa640 1 -- 172.21.15.143:0/3151035145 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f38300aa670 con 0x7f384013e290 2023-11-26T07:23:09.277 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:09.275+0000 7f38357fa640 1 -- 172.21.15.143:0/3151035145 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f38300b3800 con 0x7f384013e290 2023-11-26T07:23:09.277 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:09.275+0000 7f3847054640 1 -- 172.21.15.143:0/3151035145 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f384015db60 con 0x7f384013e290 2023-11-26T07:23:09.277 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:09.275+0000 7f38357fa640 1 -- 172.21.15.143:0/3151035145 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f38300b3bb0 con 0x7f384013e290 2023-11-26T07:23:09.277 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:09.275+0000 7f3847054640 1 -- 172.21.15.143:0/3151035145 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3840137d80 con 0x7f384013e290 2023-11-26T07:23:09.277 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:09.275+0000 7f3847054640 1 -- 172.21.15.143:0/3151035145 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f384015e360 con 0x7f384013e290 2023-11-26T07:23:09.277 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:09.275+0000 7f38357fa640 1 -- 172.21.15.143:0/3151035145 <== mon.0 v2:172.21.15.143:3300/0 4 ==== mgrmap(e 11) v1 ==== 86158+0+0 (secure 0 0 0) 0x7f38300b3d10 con 0x7f384013e290 2023-11-26T07:23:09.278 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:09.275+0000 7f38357fa640 1 --2- 172.21.15.143:0/3151035145 >> [v2:172.21.15.143:6832/2150557544,v1:172.21.15.143:6833/2150557544] conn(0x7f382c0667d0 0x7f382c068c90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:23:09.278 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:09.275+0000 7f38357fa640 1 -- 172.21.15.143:0/3151035145 <== mon.0 v2:172.21.15.143:3300/0 5 ==== osd_map(71..71 src has 1..71) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f383011ecb0 con 0x7f384013e290 2023-11-26T07:23:09.278 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:09.275+0000 7f3844df0640 1 --2- 172.21.15.143:0/3151035145 >> [v2:172.21.15.143:6832/2150557544,v1:172.21.15.143:6833/2150557544] conn(0x7f382c0667d0 0x7f382c068c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:23:09.279 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:09.279+0000 7f3844df0640 1 --2- 172.21.15.143:0/3151035145 >> [v2:172.21.15.143:6832/2150557544,v1:172.21.15.143:6833/2150557544] conn(0x7f382c0667d0 0x7f382c068c90 secure :-1 s=READY pgs=484 cs=0 l=1 rev1=1 crypto rx=0x7f384013a150 tx=0x7f383c008040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:23:09.282 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:09.279+0000 7f38357fa640 1 -- 172.21.15.143:0/3151035145 <== mon.0 v2:172.21.15.143:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172649 (secure 0 0 0) 0x7f384015e360 con 0x7f384013e290 2023-11-26T07:23:09.461 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:09.459+0000 7f3847054640 1 -- 172.21.15.143:0/3151035145 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f384015e360 con 0x7f384013e290 2023-11-26T07:23:09.462 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:09.459+0000 7f38357fa640 1 -- 172.21.15.143:0/3151035145 <== mon.0 v2:172.21.15.143:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 43 v43) v1 ==== 94+0+4924 (secure 0 0 0) 0x7f384015e360 con 0x7f384013e290 2023-11-26T07:23:09.462 INFO:teuthology.orchestra.run.smithi143.stdout: 2023-11-26T07:23:09.462 INFO:teuthology.orchestra.run.smithi143.stdout:{"epoch":43,"default_fscid":5,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5927,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.143:6837/578144260","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6835","nonce":578144260},{"type":"v1","addr":"172.21.15.143:6837","nonce":578144260}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":38},{"gid":5930,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.188:6834/1048736873","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6832","nonce":1048736873},{"type":"v1","addr":"172.21.15.188:6834","nonce":1048736873}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":38},{"gid":5933,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.188:6835/2063768716","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6833","nonce":2063768716},{"type":"v1","addr":"172.21.15.188:6835","nonce":2063768716}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":38}],"filesystems":[{"mdsmap":{"epoch":42,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-11-26T07:22:37.220113+0000","modified":"2023-11-26T07:22:40.243052+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":6553},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6553":{"gid":6553,"name":"a","rank":0,"incarnation":39,"state":"up:active","state_seq":3,"addr":"172.21.15.143:6836/2590663349","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6834","nonce":2590663349},{"type":"v1","addr":"172.21.15.143:6836","nonce":2590663349}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[11],"metadata_pool":10,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":5}]} 2023-11-26T07:23:09.463 INFO:teuthology.orchestra.run.smithi143.stderr:dumped fsmap epoch 43 2023-11-26T07:23:09.464 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:09.463+0000 7f3847054640 1 -- 172.21.15.143:0/3151035145 >> [v2:172.21.15.143:6832/2150557544,v1:172.21.15.143:6833/2150557544] conn(0x7f382c0667d0 msgr2=0x7f382c068c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:23:09.464 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:09.463+0000 7f3847054640 1 --2- 172.21.15.143:0/3151035145 >> [v2:172.21.15.143:6832/2150557544,v1:172.21.15.143:6833/2150557544] conn(0x7f382c0667d0 0x7f382c068c90 secure :-1 s=READY pgs=484 cs=0 l=1 rev1=1 crypto rx=0x7f384013a150 tx=0x7f383c008040 comp rx=0 tx=0).stop 2023-11-26T07:23:09.464 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:09.463+0000 7f3847054640 1 -- 172.21.15.143:0/3151035145 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f384013e290 msgr2=0x7f384015ca30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:23:09.464 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:09.463+0000 7f3847054640 1 --2- 172.21.15.143:0/3151035145 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f384013e290 0x7f384015ca30 secure :-1 s=READY pgs=1049 cs=0 l=1 rev1=1 crypto rx=0x7f383001a140 tx=0x7f383001ad50 comp rx=0 tx=0).stop 2023-11-26T07:23:09.464 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:09.463+0000 7f3847054640 1 -- 172.21.15.143:0/3151035145 shutdown_connections 2023-11-26T07:23:09.465 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:09.463+0000 7f3847054640 1 --2- 172.21.15.143:0/3151035145 >> [v2:172.21.15.143:6832/2150557544,v1:172.21.15.143:6833/2150557544] conn(0x7f382c0667d0 0x7f382c068c90 unknown :-1 s=CLOSED pgs=484 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:23:09.465 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:09.463+0000 7f3847054640 1 --2- 172.21.15.143:0/3151035145 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f384015f4e0 0x7f384015cf70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:23:09.465 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:09.463+0000 7f3847054640 1 --2- 172.21.15.143:0/3151035145 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f384013e290 0x7f384015ca30 unknown :-1 s=CLOSED pgs=1049 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:23:09.465 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:09.463+0000 7f3847054640 1 --2- 172.21.15.143:0/3151035145 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f38400cf7c0 0x7f3840137540 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:23:09.465 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:09.463+0000 7f3847054640 1 -- 172.21.15.143:0/3151035145 >> 172.21.15.143:0/3151035145 conn(0x7f384006a890 msgr2=0x7f38400b3cc0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:23:09.465 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:09.463+0000 7f3847054640 1 -- 172.21.15.143:0/3151035145 shutdown_connections 2023-11-26T07:23:09.465 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:09.463+0000 7f3847054640 1 -- 172.21.15.143:0/3151035145 wait complete. 2023-11-26T07:23:09.483 DEBUG:teuthology.orchestra.run.smithi143:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.a.asok --format=json session ls 2023-11-26T07:23:09.641 INFO:teuthology.orchestra.run.smithi143.stdout:[{"id":7233,"entity":{"name":{"type":"client","num":7233},"addr":{"type":"any","addr":"172.21.15.143:0","nonce":631097144}},"state":"open","num_leases":1,"num_caps":7,"request_load_avg":14,"uptime":0.54799510600000001,"requests_in_flight":2,"num_completed_requests":0,"num_completed_flushes":1,"reconnecting":false,"recall_caps":{"value":0,"halflife":60},"release_caps":{"value":0,"halflife":60},"recall_caps_throttle":{"value":0,"halflife":1.5},"recall_caps_throttle2o":{"value":0,"halflife":0.5},"session_cache_liveness":{"value":21.972781034853341,"halflife":300},"cap_acquisition":{"value":0,"halflife":30},"last_trim_completed_requests_tid":14,"last_trim_completed_flushes_tid":2,"delegated_inos":[{"start":"0x10000000005","length":500}],"inst":"client.7233 172.21.15.143:0/631097144","completed_requests":[],"prealloc_inos":[{"start":"0x10000000005","length":996}],"client_metadata":{"client_features":{"feature_bits":"0x00000000003fffff"},"metric_spec":{"metric_flags":{"feature_bits":"0x000000000000ffff"}},"ceph_sha1":"44c2237fe77713214148c1fd622c872f346f1b99","ceph_version":"ceph version 18.0.0-7484-g44c2237f (44c2237fe77713214148c1fd622c872f346f1b99) reef (dev)","entity_id":"y","hostname":"smithi143","pid":"38097","root":"/"}}] 2023-11-26T07:23:09.641 DEBUG:tasks.cephfs.filesystem:_json_asok output [ { "cap_acquisition": { "halflife": 30, "value": 0 }, "client_metadata": { "ceph_sha1": "44c2237fe77713214148c1fd622c872f346f1b99", "ceph_version": "ceph version 18.0.0-7484-g44c2237f (44c2237fe77713214148c1fd622c872f346f1b99) reef (dev)", "client_features": { "feature_bits": "0x00000000003fffff" }, "entity_id": "y", "hostname": "smithi143", "metric_spec": { "metric_flags": { "feature_bits": "0x000000000000ffff" } }, "pid": "38097", "root": "/" }, "completed_requests": [], "delegated_inos": [ { "length": 500, "start": "0x10000000005" } ], "entity": { "addr": { "addr": "172.21.15.143:0", "nonce": 631097144, "type": "any" }, "name": { "num": 7233, "type": "client" } }, "id": 7233, "inst": "client.7233 172.21.15.143:0/631097144", "last_trim_completed_flushes_tid": 2, "last_trim_completed_requests_tid": 14, "num_caps": 7, "num_completed_flushes": 1, "num_completed_requests": 0, "num_leases": 1, "prealloc_inos": [ { "length": 996, "start": "0x10000000005" } ], "recall_caps": { "halflife": 60, "value": 0 }, "recall_caps_throttle": { "halflife": 1.5, "value": 0 }, "recall_caps_throttle2o": { "halflife": 0.5, "value": 0 }, "reconnecting": false, "release_caps": { "halflife": 60, "value": 0 }, "request_load_avg": 14, "requests_in_flight": 2, "session_cache_liveness": { "halflife": 300, "value": 21.97278103485334 }, "state": "open", "uptime": 0.547995106 } ] 2023-11-26T07:23:09.641 DEBUG:teuthology.orchestra.run.smithi143:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph mgr dump --format=json-pretty 2023-11-26T07:23:09.780 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:09.779+0000 7f6154806640 1 Processor -- start 2023-11-26T07:23:09.780 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:09.779+0000 7f6154806640 1 -- start start 2023-11-26T07:23:09.781 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:09.779+0000 7f6154806640 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7f61500d1f30 0x7f61500b3170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:23:09.781 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:09.779+0000 7f6154806640 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7f61500b36b0 0x7f61500cea20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:23:09.782 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:09.779+0000 7f6154806640 1 -- --> v1:172.21.15.143:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f61500ce590 con 0x7f615015d5b0 2023-11-26T07:23:09.782 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:09.779+0000 7f6154806640 1 -- --> v2:172.21.15.143:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f61500cdf80 con 0x7f61500b36b0 2023-11-26T07:23:09.782 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:09.779+0000 7f6154806640 1 -- --> v2:172.21.15.188:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f61500b3ea0 con 0x7f61500d1f30 2023-11-26T07:23:09.782 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:09.779+0000 7f614ed9d640 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7f61500b36b0 0x7f61500cea20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:23:09.782 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:09.779+0000 7f614ed9d640 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7f61500b36b0 0x7f61500cea20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:46542/0 (socket says 172.21.15.143:46542) 2023-11-26T07:23:09.782 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:09.779+0000 7f614e59c640 1 --1- >> v1:172.21.15.143:6789/0 conn(0x7f615015d5b0 0x7f61500d1820 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.143:6789/0 says I am v1:172.21.15.143:54480/0 (socket says 172.21.15.143:54480) 2023-11-26T07:23:09.782 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:09.779+0000 7f614ed9d640 1 -- 172.21.15.143:0/4224222760 learned_addr learned my addr 172.21.15.143:0/4224222760 (peer_addr_for_me v2:172.21.15.143:0/0) 2023-11-26T07:23:09.782 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:09.779+0000 7f614ed9d640 1 -- 172.21.15.143:0/4224222760 >> v2:172.21.15.188:3300/0 conn(0x7f61500d1f30 msgr2=0x7f61500b3170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-26T07:23:09.783 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:09.779+0000 7f614ed9d640 1 --2- 172.21.15.143:0/4224222760 >> v2:172.21.15.188:3300/0 conn(0x7f61500d1f30 0x7f61500b3170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:23:09.783 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:09.779+0000 7f614ed9d640 1 -- 172.21.15.143:0/4224222760 >> v1:172.21.15.143:6789/0 conn(0x7f615015d5b0 legacy=0x7f61500d1820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:23:09.783 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:09.779+0000 7f614ed9d640 1 -- 172.21.15.143:0/4224222760 --> v2:172.21.15.143:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f61500ce220 con 0x7f61500b36b0 2023-11-26T07:23:09.785 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:09.783+0000 7f614ed9d640 1 --2- 172.21.15.143:0/4224222760 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f61500b36b0 0x7f61500cea20 secure :-1 s=READY pgs=1051 cs=0 l=1 rev1=1 crypto rx=0x7f615006b120 tx=0x7f614801c9d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f1c0340b3c34b7f0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:23:09.785 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:09.783+0000 7f614dd9b640 1 -- 172.21.15.143:0/4224222760 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f61480a58b0 con 0x7f61500b36b0 2023-11-26T07:23:09.785 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:09.783+0000 7f614dd9b640 1 -- 172.21.15.143:0/4224222760 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f61480a5a10 con 0x7f61500b36b0 2023-11-26T07:23:09.785 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:09.783+0000 7f614dd9b640 1 -- 172.21.15.143:0/4224222760 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f61480a5da0 con 0x7f61500b36b0 2023-11-26T07:23:09.786 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:09.783+0000 7f6154806640 1 -- 172.21.15.143:0/4224222760 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f61500b36b0 msgr2=0x7f61500cea20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:23:09.786 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:09.783+0000 7f6154806640 1 --2- 172.21.15.143:0/4224222760 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f61500b36b0 0x7f61500cea20 secure :-1 s=READY pgs=1051 cs=0 l=1 rev1=1 crypto rx=0x7f615006b120 tx=0x7f614801c9d0 comp rx=0 tx=0).stop 2023-11-26T07:23:09.786 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:09.783+0000 7f6154806640 1 -- 172.21.15.143:0/4224222760 shutdown_connections 2023-11-26T07:23:09.786 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:09.783+0000 7f6154806640 1 --2- 172.21.15.143:0/4224222760 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f61500b36b0 0x7f61500cea20 unknown :-1 s=CLOSED pgs=1051 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:23:09.786 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:09.783+0000 7f6154806640 1 --2- 172.21.15.143:0/4224222760 >> v2:172.21.15.188:3300/0 conn(0x7f61500d1f30 0x7f61500b3170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:23:09.786 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:09.783+0000 7f6154806640 1 -- 172.21.15.143:0/4224222760 >> 172.21.15.143:0/4224222760 conn(0x7f615008cc60 msgr2=0x7f615009c580 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:23:09.786 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:09.783+0000 7f6154806640 1 -- 172.21.15.143:0/4224222760 shutdown_connections 2023-11-26T07:23:09.786 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:09.783+0000 7f6154806640 1 -- 172.21.15.143:0/4224222760 wait complete. 2023-11-26T07:23:09.787 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:09.787+0000 7f6154806640 1 Processor -- start 2023-11-26T07:23:09.787 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:09.787+0000 7f6154806640 1 -- start start 2023-11-26T07:23:09.789 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:09.787+0000 7f6154806640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f61500b36b0 0x7f615015be00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:23:09.789 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:09.787+0000 7f6154806640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f61500d1f30 0x7f615014f240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:23:09.789 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:09.787+0000 7f6154806640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f615015d5b0 0x7f615015a390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:23:09.789 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:09.787+0000 7f6154806640 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f61500b6790 con 0x7f61500d1f30 2023-11-26T07:23:09.790 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:09.787+0000 7f6154806640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f615015a8d0 con 0x7f615015d5b0 2023-11-26T07:23:09.790 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:09.787+0000 7f6154806640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f615015ab70 con 0x7f61500b36b0 2023-11-26T07:23:09.790 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:09.787+0000 7f6147fff640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f61500d1f30 0x7f615014f240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:23:09.790 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:09.787+0000 7f6147fff640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f61500d1f30 0x7f615014f240 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:46548/0 (socket says 172.21.15.143:46548) 2023-11-26T07:23:09.790 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:09.787+0000 7f6147fff640 1 -- 172.21.15.143:0/2854946509 learned_addr learned my addr 172.21.15.143:0/2854946509 (peer_addr_for_me v2:172.21.15.143:0/0) 2023-11-26T07:23:09.790 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:09.787+0000 7f614e59c640 1 --2- 172.21.15.143:0/2854946509 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f61500b36b0 0x7f615015be00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:23:09.790 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:09.787+0000 7f614ed9d640 1 --2- 172.21.15.143:0/2854946509 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f615015d5b0 0x7f615015a390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:23:09.790 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:09.787+0000 7f6147fff640 1 -- 172.21.15.143:0/2854946509 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f61500b36b0 msgr2=0x7f615015be00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:23:09.790 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:09.787+0000 7f6147fff640 1 --2- 172.21.15.143:0/2854946509 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f61500b36b0 0x7f615015be00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:23:09.790 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:09.787+0000 7f6147fff640 1 -- 172.21.15.143:0/2854946509 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f615015d5b0 msgr2=0x7f615015a390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:23:09.790 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:09.787+0000 7f6147fff640 1 --2- 172.21.15.143:0/2854946509 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f615015d5b0 0x7f615015a390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:23:09.790 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:09.787+0000 7f6147fff640 1 -- 172.21.15.143:0/2854946509 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6148009b30 con 0x7f61500d1f30 2023-11-26T07:23:09.791 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:09.787+0000 7f6147fff640 1 --2- 172.21.15.143:0/2854946509 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f61500d1f30 0x7f615014f240 secure :-1 s=READY pgs=1052 cs=0 l=1 rev1=1 crypto rx=0x7f613800b550 tx=0x7f613800ba10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:23:09.791 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:09.787+0000 7f6146ffd640 1 -- 172.21.15.143:0/2854946509 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6138010870 con 0x7f61500d1f30 2023-11-26T07:23:09.791 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:09.787+0000 7f6146ffd640 1 -- 172.21.15.143:0/2854946509 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f61380109f0 con 0x7f61500d1f30 2023-11-26T07:23:09.791 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:09.787+0000 7f6146ffd640 1 -- 172.21.15.143:0/2854946509 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6138010da0 con 0x7f61500d1f30 2023-11-26T07:23:09.792 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:09.787+0000 7f6154806640 1 -- 172.21.15.143:0/2854946509 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f615015ae50 con 0x7f61500d1f30 2023-11-26T07:23:09.792 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:09.787+0000 7f6154806640 1 -- 172.21.15.143:0/2854946509 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f615015b410 con 0x7f61500d1f30 2023-11-26T07:23:09.792 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:09.791+0000 7f6154806640 1 -- 172.21.15.143:0/2854946509 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6150067680 con 0x7f61500d1f30 2023-11-26T07:23:09.792 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:09.791+0000 7f6146ffd640 1 -- 172.21.15.143:0/2854946509 <== mon.0 v2:172.21.15.143:3300/0 4 ==== mgrmap(e 11) v1 ==== 86158+0+0 (secure 0 0 0) 0x7f613800f3c0 con 0x7f61500d1f30 2023-11-26T07:23:09.793 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:09.791+0000 7f6146ffd640 1 --2- 172.21.15.143:0/2854946509 >> [v2:172.21.15.143:6832/2150557544,v1:172.21.15.143:6833/2150557544] conn(0x7f6130066800 0x7f6130068cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:23:09.793 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:09.791+0000 7f6146ffd640 1 -- 172.21.15.143:0/2854946509 <== mon.0 v2:172.21.15.143:3300/0 5 ==== osd_map(71..71 src has 1..71) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f6138084420 con 0x7f61500d1f30 2023-11-26T07:23:09.793 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:09.791+0000 7f614e59c640 1 --2- 172.21.15.143:0/2854946509 >> [v2:172.21.15.143:6832/2150557544,v1:172.21.15.143:6833/2150557544] conn(0x7f6130066800 0x7f6130068cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:23:09.793 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:09.791+0000 7f614e59c640 1 --2- 172.21.15.143:0/2854946509 >> [v2:172.21.15.143:6832/2150557544,v1:172.21.15.143:6833/2150557544] conn(0x7f6130066800 0x7f6130068cc0 secure :-1 s=READY pgs=485 cs=0 l=1 rev1=1 crypto rx=0x7f6140000f70 tx=0x7f6140007cb0 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:23:09.796 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:09.795+0000 7f6146ffd640 1 -- 172.21.15.143:0/2854946509 <== mon.0 v2:172.21.15.143:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172649 (secure 0 0 0) 0x7f6138051420 con 0x7f61500d1f30 2023-11-26T07:23:09.985 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:09.983+0000 7f6154806640 1 -- 172.21.15.143:0/2854946509 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "mgr dump", "format": "json-pretty"} v 0) v1 -- 0x7f61500b8e90 con 0x7f61500d1f30 2023-11-26T07:23:09.991 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:09.991+0000 7f6146ffd640 1 -- 172.21.15.143:0/2854946509 <== mon.0 v2:172.21.15.143:3300/0 7 ==== mon_command_ack([{"prefix": "mgr dump", "format": "json-pretty"}]=0 v11) v1 ==== 81+0+448021 (secure 0 0 0) 0x7f6138054a40 con 0x7f61500d1f30 2023-11-26T07:23:09.996 INFO:teuthology.orchestra.run.smithi143.stdout: 2023-11-26T07:23:09.996 INFO:teuthology.orchestra.run.smithi143.stdout:{ 2023-11-26T07:23:09.996 INFO:teuthology.orchestra.run.smithi143.stdout: "epoch": 11, 2023-11-26T07:23:09.996 INFO:teuthology.orchestra.run.smithi143.stdout: "active_gid": 4100, 2023-11-26T07:23:09.996 INFO:teuthology.orchestra.run.smithi143.stdout: "active_name": "y", 2023-11-26T07:23:09.997 INFO:teuthology.orchestra.run.smithi143.stdout: "active_addrs": { 2023-11-26T07:23:09.997 INFO:teuthology.orchestra.run.smithi143.stdout: "addrvec": [ 2023-11-26T07:23:09.997 INFO:teuthology.orchestra.run.smithi143.stdout: { 2023-11-26T07:23:09.997 INFO:teuthology.orchestra.run.smithi143.stdout: "type": "v2", 2023-11-26T07:23:09.997 INFO:teuthology.orchestra.run.smithi143.stdout: "addr": "172.21.15.143:6832", 2023-11-26T07:23:09.997 INFO:teuthology.orchestra.run.smithi143.stdout: "nonce": 2150557544 2023-11-26T07:23:09.997 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2023-11-26T07:23:09.998 INFO:teuthology.orchestra.run.smithi143.stdout: { 2023-11-26T07:23:09.998 INFO:teuthology.orchestra.run.smithi143.stdout: "type": "v1", 2023-11-26T07:23:09.998 INFO:teuthology.orchestra.run.smithi143.stdout: "addr": "172.21.15.143:6833", 2023-11-26T07:23:09.998 INFO:teuthology.orchestra.run.smithi143.stdout: "nonce": 2150557544 2023-11-26T07:23:09.998 INFO:teuthology.orchestra.run.smithi143.stdout: } 2023-11-26T07:23:09.998 INFO:teuthology.orchestra.run.smithi143.stdout: ] 2023-11-26T07:23:09.998 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2023-11-26T07:23:09.999 INFO:teuthology.orchestra.run.smithi143.stdout: "active_addr": "172.21.15.143:6833/2150557544", 2023-11-26T07:23:09.999 INFO:teuthology.orchestra.run.smithi143.stdout: "active_change": "2023-11-26T07:17:33.680133+0000", 2023-11-26T07:23:09.999 INFO:teuthology.orchestra.run.smithi143.stdout: "active_mgr_features": 4540138322906710015, 2023-11-26T07:23:09.999 INFO:teuthology.orchestra.run.smithi143.stdout: "available": true, 2023-11-26T07:23:09.999 INFO:teuthology.orchestra.run.smithi143.stdout: "standbys": [ 2023-11-26T07:23:09.999 INFO:teuthology.orchestra.run.smithi143.stdout: { 2023-11-26T07:23:09.999 INFO:teuthology.orchestra.run.smithi143.stdout: "gid": 4108, 2023-11-26T07:23:09.999 INFO:teuthology.orchestra.run.smithi143.stdout: "name": "x", 2023-11-26T07:23:10.000 INFO:teuthology.orchestra.run.smithi143.stdout: "mgr_features": 4540138322906710015, 2023-11-26T07:23:10.000 INFO:teuthology.orchestra.run.smithi143.stdout: "available_modules": [ 2023-11-26T07:23:10.000 INFO:teuthology.orchestra.run.smithi143.stdout: { 2023-11-26T07:23:10.000 INFO:teuthology.orchestra.run.smithi143.stdout: "name": "alerts", 2023-11-26T07:23:10.000 INFO:teuthology.orchestra.run.smithi143.stdout: "can_run": true, 2023-11-26T07:23:10.000 INFO:teuthology.orchestra.run.smithi143.stdout: "error_string": "", 2023-11-26T07:23:10.000 INFO:teuthology.orchestra.run.smithi143.stdout: "module_options": { 2023-11-26T07:23:10.001 INFO:teuthology.orchestra.run.smithi143.stdout: "interval": { 2023-11-26T07:23:10.001 INFO:teuthology.orchestra.run.smithi143.stdout: "name": "interval", 2023-11-26T07:23:10.001 INFO:teuthology.orchestra.run.smithi143.stdout: "type": "secs", 2023-11-26T07:23:10.001 INFO:teuthology.orchestra.run.smithi143.stdout: "level": "advanced", 2023-11-26T07:23:10.001 INFO:teuthology.orchestra.run.smithi143.stdout: "flags": 1, 2023-11-26T07:23:10.001 INFO:teuthology.orchestra.run.smithi143.stdout: "default_value": "60", 2023-11-26T07:23:10.001 INFO:teuthology.orchestra.run.smithi143.stdout: "min": "", 2023-11-26T07:23:10.002 INFO:teuthology.orchestra.run.smithi143.stdout: "max": "", 2023-11-26T07:23:10.002 INFO:teuthology.orchestra.run.smithi143.stdout: "enum_allowed": [], 2023-11-26T07:23:10.002 INFO:teuthology.orchestra.run.smithi143.stdout: "desc": "How frequently to reexamine health status", 2023-11-26T07:23:10.002 INFO:teuthology.orchestra.run.smithi143.stdout: "long_desc": "", 2023-11-26T07:23:10.002 INFO:teuthology.orchestra.run.smithi143.stdout: "tags": [], 2023-11-26T07:23:10.002 INFO:teuthology.orchestra.run.smithi143.stdout: "see_also": [] 2023-11-26T07:23:10.002 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2023-11-26T07:23:10.002 INFO:teuthology.orchestra.run.smithi143.stdout: "log_level": { 2023-11-26T07:23:10.003 INFO:teuthology.orchestra.run.smithi143.stdout: "name": "log_level", 2023-11-26T07:23:10.003 INFO:teuthology.orchestra.run.smithi143.stdout: "type": "str", 2023-11-26T07:23:10.003 INFO:teuthology.orchestra.run.smithi143.stdout: "level": "advanced", 2023-11-26T07:23:10.003 INFO:teuthology.orchestra.run.smithi143.stdout: "flags": 1, 2023-11-26T07:23:10.003 INFO:teuthology.orchestra.run.smithi143.stdout: "default_value": "", 2023-11-26T07:23:10.003 INFO:teuthology.orchestra.run.smithi143.stdout: "min": "", 2023-11-26T07:23:10.003 INFO:teuthology.orchestra.run.smithi143.stdout: "max": "", 2023-11-26T07:23:10.004 INFO:teuthology.orchestra.run.smithi143.stdout: "enum_allowed": [ 2023-11-26T07:23:10.004 INFO:teuthology.orchestra.run.smithi143.stdout: "", 2023-11-26T07:23:10.004 INFO:teuthology.orchestra.run.smithi143.stdout: "critical", 2023-11-26T07:23:10.004 INFO:teuthology.orchestra.run.smithi143.stdout: "debug", 2023-11-26T07:23:10.004 INFO:teuthology.orchestra.run.smithi143.stdout: "error", 2023-11-26T07:23:10.004 INFO:teuthology.orchestra.run.smithi143.stdout: "info", 2023-11-26T07:23:10.004 INFO:teuthology.orchestra.run.smithi143.stdout: "warning" 2023-11-26T07:23:10.004 INFO:teuthology.orchestra.run.smithi143.stdout: ], 2023-11-26T07:23:10.005 INFO:teuthology.orchestra.run.smithi143.stdout: "desc": "", 2023-11-26T07:23:10.005 INFO:teuthology.orchestra.run.smithi143.stdout: "long_desc": "", 2023-11-26T07:23:10.005 INFO:teuthology.orchestra.run.smithi143.stdout: "tags": [], 2023-11-26T07:23:10.005 INFO:teuthology.orchestra.run.smithi143.stdout: "see_also": [] 2023-11-26T07:23:10.005 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2023-11-26T07:23:10.006 INFO:teuthology.orchestra.run.smithi143.stdout: "log_to_cluster": { 2023-11-26T07:23:10.006 INFO:teuthology.orchestra.run.smithi143.stdout: "name": "log_to_cluster", 2023-11-26T07:23:10.006 INFO:teuthology.orchestra.run.smithi143.stdout: "type": "bool", 2023-11-26T07:23:10.006 INFO:teuthology.orchestra.run.smithi143.stdout: "level": "advanced", 2023-11-26T07:23:10.006 INFO:teuthology.orchestra.run.smithi143.stdout: "flags": 1, 2023-11-26T07:23:10.007 INFO:teuthology.orchestra.run.smithi143.stdout: "default_value": "False", 2023-11-26T07:23:10.007 INFO:teuthology.orchestra.run.smithi143.stdout: "min": "", 2023-11-26T07:23:10.007 INFO:teuthology.orchestra.run.smithi143.stdout: "max": "", 2023-11-26T07:23:10.007 INFO:teuthology.orchestra.run.smithi143.stdout: "enum_allowed": [], 2023-11-26T07:23:10.007 INFO:teuthology.orchestra.run.smithi143.stdout: "desc": "", 2023-11-26T07:23:10.007 INFO:teuthology.orchestra.run.smithi143.stdout: "long_desc": "", 2023-11-26T07:23:10.007 INFO:teuthology.orchestra.run.smithi143.stdout: "tags": [], 2023-11-26T07:23:10.007 INFO:teuthology.orchestra.run.smithi143.stdout: "see_also": [] 2023-11-26T07:23:10.007 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2023-11-26T07:23:10.007 INFO:teuthology.orchestra.run.smithi143.stdout: "log_to_cluster_level": { 2023-11-26T07:23:10.007 INFO:teuthology.orchestra.run.smithi143.stdout: "name": "log_to_cluster_level", 2023-11-26T07:23:10.007 INFO:teuthology.orchestra.run.smithi143.stdout: "type": "str", 2023-11-26T07:23:10.007 INFO:teuthology.orchestra.run.smithi143.stdout: "level": "advanced", 2023-11-26T07:23:10.008 INFO:teuthology.orchestra.run.smithi143.stdout: "flags": 1, 2023-11-26T07:23:10.008 INFO:teuthology.orchestra.run.smithi143.stdout: "default_value": "info", 2023-11-26T07:23:10.008 INFO:teuthology.orchestra.run.smithi143.stdout: "min": "", 2023-11-26T07:23:10.008 INFO:teuthology.orchestra.run.smithi143.stdout: "max": "", 2023-11-26T07:23:10.008 INFO:teuthology.orchestra.run.smithi143.stdout: "enum_allowed": [ 2023-11-26T07:23:10.008 INFO:teuthology.orchestra.run.smithi143.stdout: "", 2023-11-26T07:23:10.008 INFO:teuthology.orchestra.run.smithi143.stdout: "critical", 2023-11-26T07:23:10.008 INFO:teuthology.orchestra.run.smithi143.stdout: "debug", 2023-11-26T07:23:10.008 INFO:teuthology.orchestra.run.smithi143.stdout: "error", 2023-11-26T07:23:10.008 INFO:teuthology.orchestra.run.smithi143.stdout: "info", 2023-11-26T07:23:10.008 INFO:teuthology.orchestra.run.smithi143.stdout: "warning" 2023-11-26T07:23:10.008 INFO:teuthology.orchestra.run.smithi143.stdout: ], 2023-11-26T07:23:10.008 INFO:teuthology.orchestra.run.smithi143.stdout: "desc": "", 2023-11-26T07:23:10.008 INFO:teuthology.orchestra.run.smithi143.stdout: "long_desc": "", 2023-11-26T07:23:10.009 INFO:teuthology.orchestra.run.smithi143.stdout: "tags": [], 2023-11-26T07:23:10.009 INFO:teuthology.orchestra.run.smithi143.stdout: "see_also": [] 2023-11-26T07:23:10.009 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2023-11-26T07:23:10.009 INFO:teuthology.orchestra.run.smithi143.stdout: "log_to_file": { 2023-11-26T07:23:10.009 INFO:teuthology.orchestra.run.smithi143.stdout: "name": "log_to_file", 2023-11-26T07:23:10.009 INFO:teuthology.orchestra.run.smithi143.stdout: "type": "bool", 2023-11-26T07:23:10.009 INFO:teuthology.orchestra.run.smithi143.stdout: "level": "advanced", 2023-11-26T07:23:10.009 INFO:teuthology.orchestra.run.smithi143.stdout: "flags": 1, 2023-11-26T07:23:10.009 INFO:teuthology.orchestra.run.smithi143.stdout: "default_value": "False", 2023-11-26T07:23:10.009 INFO:teuthology.orchestra.run.smithi143.stdout: "min": "", 2023-11-26T07:23:10.009 INFO:teuthology.orchestra.run.smithi143.stdout: "max": "", 2023-11-26T07:23:10.009 INFO:teuthology.orchestra.run.smithi143.stdout: "enum_allowed": [], 2023-11-26T07:23:10.009 INFO:teuthology.orchestra.run.smithi143.stdout: "desc": "", 2023-11-26T07:23:10.009 INFO:teuthology.orchestra.run.smithi143.stdout: "long_desc": "", 2023-11-26T07:23:10.010 INFO:teuthology.orchestra.run.smithi143.stdout: "tags": [], 2023-11-26T07:23:10.010 INFO:teuthology.orchestra.run.smithi143.stdout: "see_also": [] 2023-11-26T07:23:10.010 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2023-11-26T07:23:10.010 INFO:teuthology.orchestra.run.smithi143.stdout: "smtp_destination": { 2023-11-26T07:23:10.010 INFO:teuthology.orchestra.run.smithi143.stdout: "name": "smtp_destination", 2023-11-26T07:23:10.010 INFO:teuthology.orchestra.run.smithi143.stdout: "type": "str", 2023-11-26T07:23:10.010 INFO:teuthology.orchestra.run.smithi143.stdout: "level": "advanced", 2023-11-26T07:23:10.010 INFO:teuthology.orchestra.run.smithi143.stdout: "flags": 1, 2023-11-26T07:23:10.010 INFO:teuthology.orchestra.run.smithi143.stdout: "default_value": "", 2023-11-26T07:23:10.010 INFO:teuthology.orchestra.run.smithi143.stdout: "min": "", 2023-11-26T07:23:10.010 INFO:teuthology.orchestra.run.smithi143.stdout: "max": "", 2023-11-26T07:23:10.010 INFO:teuthology.orchestra.run.smithi143.stdout: "enum_allowed": [], 2023-11-26T07:23:10.010 INFO:teuthology.orchestra.run.smithi143.stdout: "desc": "Email address to send alerts to", 2023-11-26T07:23:10.010 INFO:teuthology.orchestra.run.smithi143.stdout: "long_desc": "", 2023-11-26T07:23:10.011 INFO:teuthology.orchestra.run.smithi143.stdout: "tags": [], 2023-11-26T07:23:10.011 INFO:teuthology.orchestra.run.smithi143.stdout: "see_also": [] 2023-11-26T07:23:10.011 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2023-11-26T07:23:10.011 INFO:teuthology.orchestra.run.smithi143.stdout: "smtp_from_name": { 2023-11-26T07:23:10.011 INFO:teuthology.orchestra.run.smithi143.stdout: "name": "smtp_from_name", 2023-11-26T07:23:10.011 INFO:teuthology.orchestra.run.smithi143.stdout: "type": "str", 2023-11-26T07:23:10.011 INFO:teuthology.orchestra.run.smithi143.stdout: "level": "advanced", 2023-11-26T07:23:10.011 INFO:teuthology.orchestra.run.smithi143.stdout: "flags": 1, 2023-11-26T07:23:10.011 INFO:teuthology.orchestra.run.smithi143.stdout: "default_value": "Ceph", 2023-11-26T07:23:10.011 INFO:teuthology.orchestra.run.smithi143.stdout: "min": "", 2023-11-26T07:23:10.011 INFO:teuthology.orchestra.run.smithi143.stdout: "max": "", 2023-11-26T07:23:10.011 INFO:teuthology.orchestra.run.smithi143.stdout: "enum_allowed": [], 2023-11-26T07:23:10.011 INFO:teuthology.orchestra.run.smithi143.stdout: "desc": "Email From: name", 2023-11-26T07:23:10.011 INFO:teuthology.orchestra.run.smithi143.stdout: "long_desc": "", 2023-11-26T07:23:10.011 INFO:teuthology.orchestra.run.smithi143.stdout: "tags": [], 2023-11-26T07:23:10.012 INFO:teuthology.orchestra.run.smithi143.stdout: "see_also": [] 2023-11-26T07:23:10.012 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2023-11-26T07:23:10.012 INFO:teuthology.orchestra.run.smithi143.stdout: "smtp_host": { 2023-11-26T07:23:10.012 INFO:teuthology.orchestra.run.smithi143.stdout: "name": "smtp_host", 2023-11-26T07:23:10.012 INFO:teuthology.orchestra.run.smithi143.stdout: "type": "str", 2023-11-26T07:23:10.012 INFO:teuthology.orchestra.run.smithi143.stdout: "level": "advanced", 2023-11-26T07:23:10.012 INFO:teuthology.orchestra.run.smithi143.stdout: "flags": 1, 2023-11-26T07:23:10.012 INFO:teuthology.orchestra.run.smithi143.stdout: "default_value": "", 2023-11-26T07:23:10.012 INFO:teuthology.orchestra.run.smithi143.stdout: "min": "", 2023-11-26T07:23:10.012 INFO:teuthology.orchestra.run.smithi143.stdout: "max": "", 2023-11-26T07:23:10.012 INFO:teuthology.orchestra.run.smithi143.stdout: "enum_allowed": [], 2023-11-26T07:23:10.012 INFO:teuthology.orchestra.run.smithi143.stdout: "desc": "SMTP server", 2023-11-26T07:23:10.012 INFO:teuthology.orchestra.run.smithi143.stdout: "long_desc": "", 2023-11-26T07:23:10.012 INFO:teuthology.orchestra.run.smithi143.stdout: "tags": [], 2023-11-26T07:23:10.013 INFO:teuthology.orchestra.run.smithi143.stdout: "see_also": [] 2023-11-26T07:23:10.013 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2023-11-26T07:23:10.013 INFO:teuthology.orchestra.run.smithi143.stdout: "smtp_password": { 2023-11-26T07:23:10.013 INFO:teuthology.orchestra.run.smithi143.stdout: "name": "smtp_password", 2023-11-26T07:23:10.013 INFO:teuthology.orchestra.run.smithi143.stdout: "type": "str", 2023-11-26T07:23:10.013 INFO:teuthology.orchestra.run.smithi143.stdout: "level": "advanced", 2023-11-26T07:23:10.013 INFO:teuthology.orchestra.run.smithi143.stdout: "flags": 1, 2023-11-26T07:23:10.013 INFO:teuthology.orchestra.run.smithi143.stdout: "default_value": "", 2023-11-26T07:23:10.013 INFO:teuthology.orchestra.run.smithi143.stdout: "min": "", 2023-11-26T07:23:10.013 INFO:teuthology.orchestra.run.smithi143.stdout: "max": "", 2023-11-26T07:23:10.013 INFO:teuthology.orchestra.run.smithi143.stdout: "enum_allowed": [], 2023-11-26T07:23:10.013 INFO:teuthology.orchestra.run.smithi143.stdout: "desc": "Password to authenticate with", 2023-11-26T07:23:10.013 INFO:teuthology.orchestra.run.smithi143.stdout: "long_desc": "", 2023-11-26T07:23:10.013 INFO:teuthology.orchestra.run.smithi143.stdout: "tags": [], 2023-11-26T07:23:10.014 INFO:teuthology.orchestra.run.smithi143.stdout: "see_also": [] 2023-11-26T07:23:10.014 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2023-11-26T07:23:10.014 INFO:teuthology.orchestra.run.smithi143.stdout: "smtp_port": { 2023-11-26T07:23:10.014 INFO:teuthology.orchestra.run.smithi143.stdout: "name": "smtp_port", 2023-11-26T07:23:10.014 INFO:teuthology.orchestra.run.smithi143.stdout: "type": "int", 2023-11-26T07:23:10.014 INFO:teuthology.orchestra.run.smithi143.stdout: "level": "advanced", 2023-11-26T07:23:10.014 INFO:teuthology.orchestra.run.smithi143.stdout: "flags": 1, 2023-11-26T07:23:10.014 INFO:teuthology.orchestra.run.smithi143.stdout: "default_value": "465", 2023-11-26T07:23:10.014 INFO:teuthology.orchestra.run.smithi143.stdout: "min": "", 2023-11-26T07:23:10.014 INFO:teuthology.orchestra.run.smithi143.stdout: "max": "", 2023-11-26T07:23:10.014 INFO:teuthology.orchestra.run.smithi143.stdout: "enum_allowed": [], 2023-11-26T07:23:10.014 INFO:teuthology.orchestra.run.smithi143.stdout: "desc": "SMTP port", 2023-11-26T07:23:10.014 INFO:teuthology.orchestra.run.smithi143.stdout: "long_desc": "", 2023-11-26T07:23:10.014 INFO:teuthology.orchestra.run.smithi143.stdout: "tags": [], 2023-11-26T07:23:10.015 INFO:teuthology.orchestra.run.smithi143.stdout: "see_also": [] 2023-11-26T07:23:10.015 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2023-11-26T07:23:10.015 INFO:teuthology.orchestra.run.smithi143.stdout: "smtp_sender": { 2023-11-26T07:23:10.015 INFO:teuthology.orchestra.run.smithi143.stdout: "name": "smtp_sender", 2023-11-26T07:23:10.015 INFO:teuthology.orchestra.run.smithi143.stdout: "type": "str", 2023-11-26T07:23:10.015 INFO:teuthology.orchestra.run.smithi143.stdout: "level": "advanced", 2023-11-26T07:23:10.015 INFO:teuthology.orchestra.run.smithi143.stdout: "flags": 1, 2023-11-26T07:23:10.015 INFO:teuthology.orchestra.run.smithi143.stdout: "default_value": "", 2023-11-26T07:23:10.015 INFO:teuthology.orchestra.run.smithi143.stdout: "min": "", 2023-11-26T07:23:10.015 INFO:teuthology.orchestra.run.smithi143.stdout: "max": "", 2023-11-26T07:23:10.015 INFO:teuthology.orchestra.run.smithi143.stdout: "enum_allowed": [], 2023-11-26T07:23:10.015 INFO:teuthology.orchestra.run.smithi143.stdout: "desc": "SMTP envelope sender", 2023-11-26T07:23:10.015 INFO:teuthology.orchestra.run.smithi143.stdout: "long_desc": "", 2023-11-26T07:23:10.015 INFO:teuthology.orchestra.run.smithi143.stdout: "tags": [], 2023-11-26T07:23:10.016 INFO:teuthology.orchestra.run.smithi143.stdout: "see_also": [] 2023-11-26T07:23:10.016 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2023-11-26T07:23:10.016 INFO:teuthology.orchestra.run.smithi143.stdout: "smtp_ssl": { 2023-11-26T07:23:10.016 INFO:teuthology.orchestra.run.smithi143.stdout: "name": "smtp_ssl", 2023-11-26T07:23:10.016 INFO:teuthology.orchestra.run.smithi143.stdout: "type": "bool", 2023-11-26T07:23:10.016 INFO:teuthology.orchestra.run.smithi143.stdout: "level": "advanced", 2023-11-26T07:23:10.016 INFO:teuthology.orchestra.run.smithi143.stdout: "flags": 1, 2023-11-26T07:23:10.016 INFO:teuthology.orchestra.run.smithi143.stdout: "default_value": "True", 2023-11-26T07:23:10.016 INFO:teuthology.orchestra.run.smithi143.stdout: "min": "", 2023-11-26T07:23:10.016 INFO:teuthology.orchestra.run.smithi143.stdout: "max": "", 2023-11-26T07:23:10.016 INFO:teuthology.orchestra.run.smithi143.stdout: "enum_allowed": [], 2023-11-26T07:23:10.016 INFO:teuthology.orchestra.run.smithi143.stdout: "desc": "Use SSL to connect to SMTP server", 2023-11-26T07:23:10.016 INFO:teuthology.orchestra.run.smithi143.stdout: "long_desc": "", 2023-11-26T07:23:10.016 INFO:teuthology.orchestra.run.smithi143.stdout: "tags": [], 2023-11-26T07:23:10.017 INFO:teuthology.orchestra.run.smithi143.stdout: "see_also": [] 2023-11-26T07:23:10.017 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2023-11-26T07:23:10.017 INFO:teuthology.orchestra.run.smithi143.stdout: "smtp_user": { 2023-11-26T07:23:10.017 INFO:teuthology.orchestra.run.smithi143.stdout: "name": "smtp_user", 2023-11-26T07:23:10.017 INFO:teuthology.orchestra.run.smithi143.stdout: "type": "str", 2023-11-26T07:23:10.017 INFO:teuthology.orchestra.run.smithi143.stdout: "level": "advanced", 2023-11-26T07:23:10.017 INFO:teuthology.orchestra.run.smithi143.stdout: "flags": 1, 2023-11-26T07:23:10.017 INFO:teuthology.orchestra.run.smithi143.stdout: "default_value": "", 2023-11-26T07:23:10.017 INFO:teuthology.orchestra.run.smithi143.stdout: "min": "", 2023-11-26T07:23:10.017 INFO:teuthology.orchestra.run.smithi143.stdout: "max": "", 2023-11-26T07:23:10.017 INFO:teuthology.orchestra.run.smithi143.stdout: "enum_allowed": [], 2023-11-26T07:23:10.017 INFO:teuthology.orchestra.run.smithi143.stdout: "desc": "User to authenticate as", 2023-11-26T07:23:10.017 INFO:teuthology.orchestra.run.smithi143.stdout: "long_desc": "", 2023-11-26T07:23:10.017 INFO:teuthology.orchestra.run.smithi143.stdout: "tags": [], 2023-11-26T07:23:10.018 INFO:teuthology.orchestra.run.smithi143.stdout: "see_also": [] 2023-11-26T07:23:10.018 INFO:teuthology.orchestra.run.smithi143.stdout: } 2023-11-26T07:23:10.018 INFO:teuthology.orchestra.run.smithi143.stdout: } 2023-11-26T07:23:10.018 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2023-11-26T07:23:10.018 INFO:teuthology.orchestra.run.smithi143.stdout: { 2023-11-26T07:23:10.018 INFO:teuthology.orchestra.run.smithi143.stdout: "name": "balancer", 2023-11-26T07:23:10.018 INFO:teuthology.orchestra.run.smithi143.stdout: "can_run": true, 2023-11-26T07:23:10.018 INFO:teuthology.orchestra.run.smithi143.stdout: "error_string": "", 2023-11-26T07:23:10.018 INFO:teuthology.orchestra.run.smithi143.stdout: "module_options": { 2023-11-26T07:23:10.018 INFO:teuthology.orchestra.run.smithi143.stdout: "active": { 2023-11-26T07:23:10.018 INFO:teuthology.orchestra.run.smithi143.stdout: "name": "active", 2023-11-26T07:23:10.018 INFO:teuthology.orchestra.run.smithi143.stdout: "type": "bool", 2023-11-26T07:23:10.018 INFO:teuthology.orchestra.run.smithi143.stdout: "level": "advanced", 2023-11-26T07:23:10.018 INFO:teuthology.orchestra.run.smithi143.stdout: "flags": 1, 2023-11-26T07:23:10.019 INFO:teuthology.orchestra.run.smithi143.stdout: "default_value": "True", 2023-11-26T07:23:10.019 INFO:teuthology.orchestra.run.smithi143.stdout: "min": "", 2023-11-26T07:23:10.019 INFO:teuthology.orchestra.run.smithi143.stdout: "max": "", 2023-11-26T07:23:10.019 INFO:teuthology.orchestra.run.smithi143.stdout: "enum_allowed": [], 2023-11-26T07:23:10.019 INFO:teuthology.orchestra.run.smithi143.stdout: "desc": "automatically balance PGs across cluster", 2023-11-26T07:23:10.019 INFO:teuthology.orchestra.run.smithi143.stdout: "long_desc": "", 2023-11-26T07:23:10.019 INFO:teuthology.orchestra.run.smithi143.stdout: "tags": [], 2023-11-26T07:23:10.019 INFO:teuthology.orchestra.run.smithi143.stdout: "see_also": [] 2023-11-26T07:23:10.019 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2023-11-26T07:23:10.019 INFO:teuthology.orchestra.run.smithi143.stdout: "begin_time": { 2023-11-26T07:23:10.019 INFO:teuthology.orchestra.run.smithi143.stdout: "name": "begin_time", 2023-11-26T07:23:10.019 INFO:teuthology.orchestra.run.smithi143.stdout: "type": "str", 2023-11-26T07:23:10.019 INFO:teuthology.orchestra.run.smithi143.stdout: "level": "advanced", 2023-11-26T07:23:10.019 INFO:teuthology.orchestra.run.smithi143.stdout: "flags": 1, 2023-11-26T07:23:10.020 INFO:teuthology.orchestra.run.smithi143.stdout: "default_value": "0000", 2023-11-26T07:23:10.020 INFO:teuthology.orchestra.run.smithi143.stdout: "min": "", 2023-11-26T07:23:10.020 INFO:teuthology.orchestra.run.smithi143.stdout: "max": "", 2023-11-26T07:23:10.020 INFO:teuthology.orchestra.run.smithi143.stdout: "enum_allowed": [], 2023-11-26T07:23:10.020 INFO:teuthology.orchestra.run.smithi143.stdout: "desc": "beginning time of day to automatically balance", 2023-11-26T07:23:10.020 INFO:teuthology.orchestra.run.smithi143.stdout: "long_desc": "This is a time of day in the format HHMM.", 2023-11-26T07:23:10.020 INFO:teuthology.orchestra.run.smithi143.stdout: "tags": [], 2023-11-26T07:23:10.020 INFO:teuthology.orchestra.run.smithi143.stdout: "see_also": [] 2023-11-26T07:23:10.020 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2023-11-26T07:23:10.020 INFO:teuthology.orchestra.run.smithi143.stdout: "begin_weekday": { 2023-11-26T07:23:10.020 INFO:teuthology.orchestra.run.smithi143.stdout: "name": "begin_weekday", 2023-11-26T07:23:10.020 INFO:teuthology.orchestra.run.smithi143.stdout: "type": "uint", 2023-11-26T07:23:10.021 INFO:teuthology.orchestra.run.smithi143.stdout: "level": "advanced", 2023-11-26T07:23:10.021 INFO:teuthology.orchestra.run.smithi143.stdout: "flags": 1, 2023-11-26T07:23:10.021 INFO:teuthology.orchestra.run.smithi143.stdout: "default_value": "0", 2023-11-26T07:23:10.021 INFO:teuthology.orchestra.run.smithi143.stdout: "min": "0", 2023-11-26T07:23:10.021 INFO:teuthology.orchestra.run.smithi143.stdout: "max": "6", 2023-11-26T07:23:10.021 INFO:teuthology.orchestra.run.smithi143.stdout: "enum_allowed": [], 2023-11-26T07:23:10.021 INFO:teuthology.orchestra.run.smithi143.stdout: "desc": "Restrict automatic balancing to this day of the week or later", 2023-11-26T07:23:10.021 INFO:teuthology.orchestra.run.smithi143.stdout: "long_desc": "0 = Sunday, 1 = Monday, etc.", 2023-11-26T07:23:10.021 INFO:teuthology.orchestra.run.smithi143.stdout: "tags": [], 2023-11-26T07:23:10.021 INFO:teuthology.orchestra.run.smithi143.stdout: "see_also": [] 2023-11-26T07:23:10.021 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2023-11-26T07:23:10.021 INFO:teuthology.orchestra.run.smithi143.stdout: "crush_compat_max_iterations": { 2023-11-26T07:23:10.021 INFO:teuthology.orchestra.run.smithi143.stdout: "name": "crush_compat_max_iterations", 2023-11-26T07:23:10.021 INFO:teuthology.orchestra.run.smithi143.stdout: "type": "uint", 2023-11-26T07:23:10.022 INFO:teuthology.orchestra.run.smithi143.stdout: "level": "advanced", 2023-11-26T07:23:10.022 INFO:teuthology.orchestra.run.smithi143.stdout: "flags": 1, 2023-11-26T07:23:10.022 INFO:teuthology.orchestra.run.smithi143.stdout: "default_value": "25", 2023-11-26T07:23:10.022 INFO:teuthology.orchestra.run.smithi143.stdout: "min": "1", 2023-11-26T07:23:10.022 INFO:teuthology.orchestra.run.smithi143.stdout: "max": "250", 2023-11-26T07:23:10.022 INFO:teuthology.orchestra.run.smithi143.stdout: "enum_allowed": [], 2023-11-26T07:23:10.022 INFO:teuthology.orchestra.run.smithi143.stdout: "desc": "maximum number of iterations to attempt optimization", 2023-11-26T07:23:10.022 INFO:teuthology.orchestra.run.smithi143.stdout: "long_desc": "", 2023-11-26T07:23:10.022 INFO:teuthology.orchestra.run.smithi143.stdout: "tags": [], 2023-11-26T07:23:10.022 INFO:teuthology.orchestra.run.smithi143.stdout: "see_also": [] 2023-11-26T07:23:10.022 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2023-11-26T07:23:10.022 INFO:teuthology.orchestra.run.smithi143.stdout: "crush_compat_metrics": { 2023-11-26T07:23:10.022 INFO:teuthology.orchestra.run.smithi143.stdout: "name": "crush_compat_metrics", 2023-11-26T07:23:10.022 INFO:teuthology.orchestra.run.smithi143.stdout: "type": "str", 2023-11-26T07:23:10.023 INFO:teuthology.orchestra.run.smithi143.stdout: "level": "advanced", 2023-11-26T07:23:10.023 INFO:teuthology.orchestra.run.smithi143.stdout: "flags": 1, 2023-11-26T07:23:10.023 INFO:teuthology.orchestra.run.smithi143.stdout: "default_value": "pgs,objects,bytes", 2023-11-26T07:23:10.023 INFO:teuthology.orchestra.run.smithi143.stdout: "min": "", 2023-11-26T07:23:10.023 INFO:teuthology.orchestra.run.smithi143.stdout: "max": "", 2023-11-26T07:23:10.023 INFO:teuthology.orchestra.run.smithi143.stdout: "enum_allowed": [], 2023-11-26T07:23:10.023 INFO:teuthology.orchestra.run.smithi143.stdout: "desc": "metrics with which to calculate OSD utilization", 2023-11-26T07:23:10.023 INFO:teuthology.orchestra.run.smithi143.stdout: "long_desc": "Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.", 2023-11-26T07:23:10.023 INFO:teuthology.orchestra.run.smithi143.stdout: "tags": [], 2023-11-26T07:23:10.023 INFO:teuthology.orchestra.run.smithi143.stdout: "see_also": [] 2023-11-26T07:23:10.023 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2023-11-26T07:23:10.023 INFO:teuthology.orchestra.run.smithi143.stdout: "crush_compat_step": { 2023-11-26T07:23:10.023 INFO:teuthology.orchestra.run.smithi143.stdout: "name": "crush_compat_step", 2023-11-26T07:23:10.024 INFO:teuthology.orchestra.run.smithi143.stdout: "type": "float", 2023-11-26T07:23:10.024 INFO:teuthology.orchestra.run.smithi143.stdout: "level": "advanced", 2023-11-26T07:23:10.024 INFO:teuthology.orchestra.run.smithi143.stdout: "flags": 1, 2023-11-26T07:23:10.024 INFO:teuthology.orchestra.run.smithi143.stdout: "default_value": "0.5", 2023-11-26T07:23:10.024 INFO:teuthology.orchestra.run.smithi143.stdout: "min": "0.001", 2023-11-26T07:23:10.024 INFO:teuthology.orchestra.run.smithi143.stdout: "max": "0.999", 2023-11-26T07:23:10.024 INFO:teuthology.orchestra.run.smithi143.stdout: "enum_allowed": [], 2023-11-26T07:23:10.024 INFO:teuthology.orchestra.run.smithi143.stdout: "desc": "aggressiveness of optimization", 2023-11-26T07:23:10.024 INFO:teuthology.orchestra.run.smithi143.stdout: "long_desc": ".99 is very aggressive, .01 is less aggressive", 2023-11-26T07:23:10.024 INFO:teuthology.orchestra.run.smithi143.stdout: "tags": [], 2023-11-26T07:23:10.024 INFO:teuthology.orchestra.run.smithi143.stdout: "see_also": [] 2023-11-26T07:23:10.024 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2023-11-26T07:23:10.025 INFO:teuthology.orchestra.run.smithi143.stdout: "end_time": { 2023-11-26T07:23:10.025 INFO:teuthology.orchestra.run.smithi143.stdout: "name": "end_time", 2023-11-26T07:23:10.025 INFO:teuthology.orchestra.run.smithi143.stdout: "type": "str", 2023-11-26T07:23:10.025 INFO:teuthology.orchestra.run.smithi143.stdout: "level": "advanced", 2023-11-26T07:23:10.025 INFO:teuthology.orchestra.run.smithi143.stdout: "flags": 1, 2023-11-26T07:23:10.025 INFO:teuthology.orchestra.run.smithi143.stdout: "default_value": "2359", 2023-11-26T07:23:10.025 INFO:teuthology.orchestra.run.smithi143.stdout: "min": "", 2023-11-26T07:23:10.025 INFO:teuthology.orchestra.run.smithi143.stdout: "max": "", 2023-11-26T07:23:10.025 INFO:teuthology.orchestra.run.smithi143.stdout: "enum_allowed": [], 2023-11-26T07:23:10.025 INFO:teuthology.orchestra.run.smithi143.stdout: "desc": "ending time of day to automatically balance", 2023-11-26T07:23:10.025 INFO:teuthology.orchestra.run.smithi143.stdout: "long_desc": "This is a time of day in the format HHMM.", 2023-11-26T07:23:10.025 INFO:teuthology.orchestra.run.smithi143.stdout: "tags": [], 2023-11-26T07:23:10.025 INFO:teuthology.orchestra.run.smithi143.stdout: "see_also": [] 2023-11-26T07:23:10.026 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2023-11-26T07:23:10.026 INFO:teuthology.orchestra.run.smithi143.stdout: "end_weekday": { 2023-11-26T07:23:10.026 INFO:teuthology.orchestra.run.smithi143.stdout: "name": "end_weekday", 2023-11-26T07:23:10.026 INFO:teuthology.orchestra.run.smithi143.stdout: "type": "uint", 2023-11-26T07:23:10.026 INFO:teuthology.orchestra.run.smithi143.stdout: "level": "advanced", 2023-11-26T07:23:10.026 INFO:teuthology.orchestra.run.smithi143.stdout: "flags": 1, 2023-11-26T07:23:10.026 INFO:teuthology.orchestra.run.smithi143.stdout: "default_value": "0", 2023-11-26T07:23:10.026 INFO:teuthology.orchestra.run.smithi143.stdout: "min": "0", 2023-11-26T07:23:10.026 INFO:teuthology.orchestra.run.smithi143.stdout: "max": "6", 2023-11-26T07:23:10.026 INFO:teuthology.orchestra.run.smithi143.stdout: "enum_allowed": [], 2023-11-26T07:23:10.026 INFO:teuthology.orchestra.run.smithi143.stdout: "desc": "Restrict automatic balancing to days of the week earlier than this", 2023-11-26T07:23:10.026 INFO:teuthology.orchestra.run.smithi143.stdout: "long_desc": "0 = Sunday, 1 = Monday, etc.", 2023-11-26T07:23:10.026 INFO:teuthology.orchestra.run.smithi143.stdout: "tags": [], 2023-11-26T07:23:10.027 INFO:teuthology.orchestra.run.smithi143.stdout: "see_also": [] 2023-11-26T07:23:10.027 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2023-11-26T07:23:10.027 INFO:teuthology.orchestra.run.smithi143.stdout: "log_level": { 2023-11-26T07:23:10.027 INFO:teuthology.orchestra.run.smithi143.stdout: "name": "log_level", 2023-11-26T07:23:10.027 INFO:teuthology.orchestra.run.smithi143.stdout: "type": "str", 2023-11-26T07:23:10.027 INFO:teuthology.orchestra.run.smithi143.stdout: "level": "advanced", 2023-11-26T07:23:10.027 INFO:teuthology.orchestra.run.smithi143.stdout: "flags": 1, 2023-11-26T07:23:10.027 INFO:teuthology.orchestra.run.smithi143.stdout: "default_value": "", 2023-11-26T07:23:10.027 INFO:teuthology.orchestra.run.smithi143.stdout: "min": "", 2023-11-26T07:23:10.027 INFO:teuthology.orchestra.run.smithi143.stdout: "max": "", 2023-11-26T07:23:10.027 INFO:teuthology.orchestra.run.smithi143.stdout: "enum_allowed": [ 2023-11-26T07:23:10.027 INFO:teuthology.orchestra.run.smithi143.stdout: "", 2023-11-26T07:23:10.027 INFO:teuthology.orchestra.run.smithi143.stdout: "critical", 2023-11-26T07:23:10.028 INFO:teuthology.orchestra.run.smithi143.stdout: "debug", 2023-11-26T07:23:10.028 INFO:teuthology.orchestra.run.smithi143.stdout: "error", 2023-11-26T07:23:10.028 INFO:teuthology.orchestra.run.smithi143.stdout: "info", 2023-11-26T07:23:10.028 INFO:teuthology.orchestra.run.smithi143.stdout: "warning" 2023-11-26T07:23:10.028 INFO:teuthology.orchestra.run.smithi143.stdout: ], 2023-11-26T07:23:10.028 INFO:teuthology.orchestra.run.smithi143.stdout: "desc": "", 2023-11-26T07:23:10.028 INFO:teuthology.orchestra.run.smithi143.stdout: "long_desc": "", 2023-11-26T07:23:10.028 INFO:teuthology.orchestra.run.smithi143.stdout: "tags": [], 2023-11-26T07:23:10.028 INFO:teuthology.orchestra.run.smithi143.stdout: "see_also": [] 2023-11-26T07:23:10.028 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2023-11-26T07:23:10.028 INFO:teuthology.orchestra.run.smithi143.stdout: "log_to_cluster": { 2023-11-26T07:23:10.028 INFO:teuthology.orchestra.run.smithi143.stdout: "name": "log_to_cluster", 2023-11-26T07:23:10.029 INFO:teuthology.orchestra.run.smithi143.stdout: "type": "bool", 2023-11-26T07:23:10.029 INFO:teuthology.orchestra.run.smithi143.stdout: "level": "advanced", 2023-11-26T07:23:10.029 INFO:teuthology.orchestra.run.smithi143.stdout: "flags": 1, 2023-11-26T07:23:10.029 INFO:teuthology.orchestra.run.smithi143.stdout: "default_value": "False", 2023-11-26T07:23:10.029 INFO:teuthology.orchestra.run.smithi143.stdout: "min": "", 2023-11-26T07:23:10.029 INFO:teuthology.orchestra.run.smithi143.stdout: "max": "", 2023-11-26T07:23:10.029 INFO:teuthology.orchestra.run.smithi143.stdout: "enum_allowed": [], 2023-11-26T07:23:10.029 INFO:teuthology.orchestra.run.smithi143.stdout: "desc": "", 2023-11-26T07:23:10.029 INFO:teuthology.orchestra.run.smithi143.stdout: "long_desc": "", 2023-11-26T07:23:10.029 INFO:teuthology.orchestra.run.smithi143.stdout: "tags": [], 2023-11-26T07:23:10.029 INFO:teuthology.orchestra.run.smithi143.stdout: "see_also": [] 2023-11-26T07:23:10.029 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2023-11-26T07:23:10.030 INFO:teuthology.orchestra.run.smithi143.stdout: "log_to_cluster_level": { 2023-11-26T07:23:10.030 INFO:teuthology.orchestra.run.smithi143.stdout: "name": "log_to_cluster_level", 2023-11-26T07:23:10.030 INFO:teuthology.orchestra.run.smithi143.stdout: "type": "str", 2023-11-26T07:23:10.030 INFO:teuthology.orchestra.run.smithi143.stdout: "level": "advanced", 2023-11-26T07:23:10.030 INFO:teuthology.orchestra.run.smithi143.stdout: "flags": 1, 2023-11-26T07:23:10.030 INFO:teuthology.orchestra.run.smithi143.stdout: "default_value": "info", 2023-11-26T07:23:10.030 INFO:teuthology.orchestra.run.smithi143.stdout: "min": "", 2023-11-26T07:23:10.030 INFO:teuthology.orchestra.run.smithi143.stdout: "max": "", 2023-11-26T07:23:10.030 INFO:teuthology.orchestra.run.smithi143.stdout: "enum_allowed": [ 2023-11-26T07:23:10.030 INFO:teuthology.orchestra.run.smithi143.stdout: "", 2023-11-26T07:23:10.030 INFO:teuthology.orchestra.run.smithi143.stdout: "critical", 2023-11-26T07:23:10.030 INFO:teuthology.orchestra.run.smithi143.stdout: "debug", 2023-11-26T07:23:10.031 INFO:teuthology.orchestra.run.smithi143.stdout: "error", 2023-11-26T07:23:10.031 INFO:teuthology.orchestra.run.smithi143.stdout: "info", 2023-11-26T07:23:10.031 INFO:teuthology.orchestra.run.smithi143.stdout: "warning" 2023-11-26T07:23:10.031 INFO:teuthology.orchestra.run.smithi143.stdout: ], 2023-11-26T07:23:10.031 INFO:teuthology.orchestra.run.smithi143.stdout: "desc": "", 2023-11-26T07:23:10.031 INFO:teuthology.orchestra.run.smithi143.stdout: "long_desc": "", 2023-11-26T07:23:10.031 INFO:teuthology.orchestra.run.smithi143.stdout: "tags": [], 2023-11-26T07:23:10.031 INFO:teuthology.orchestra.run.smithi143.stdout: "see_also": [] 2023-11-26T07:23:10.031 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2023-11-26T07:23:10.031 INFO:teuthology.orchestra.run.smithi143.stdout: "log_to_file": { 2023-11-26T07:23:10.031 INFO:teuthology.orchestra.run.smithi143.stdout: "name": "log_to_file", 2023-11-26T07:23:10.031 INFO:teuthology.orchestra.run.smithi143.stdout: "type": "bool", 2023-11-26T07:23:10.032 INFO:teuthology.orchestra.run.smithi143.stdout: "level": "advanced", 2023-11-26T07:23:10.032 INFO:teuthology.orchestra.run.smithi143.stdout: "flags": 1, 2023-11-26T07:23:10.032 INFO:teuthology.orchestra.run.smithi143.stdout: "default_value": "False", 2023-11-26T07:23:10.032 INFO:teuthology.orchestra.run.smithi143.stdout: "min": "", 2023-11-26T07:23:10.032 INFO:teuthology.orchestra.run.smithi143.stdout: "max": "", 2023-11-26T07:23:10.032 INFO:teuthology.orchestra.run.smithi143.stdout: "enum_allowed": [], 2023-11-26T07:23:10.032 INFO:teuthology.orchestra.run.smithi143.stdout: "desc": "", 2023-11-26T07:23:10.032 INFO:teuthology.orchestra.run.smithi143.stdout: "long_desc": "", 2023-11-26T07:23:10.032 INFO:teuthology.orchestra.run.smithi143.stdout: "tags": [], 2023-11-26T07:23:10.032 INFO:teuthology.orchestra.run.smithi143.stdout: "see_also": [] 2023-11-26T07:23:10.032 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2023-11-26T07:23:10.032 INFO:teuthology.orchestra.run.smithi143.stdout: "min_score": { 2023-11-26T07:23:10.032 INFO:teuthology.orchestra.run.smithi143.stdout: "name": "min_score", 2023-11-26T07:23:10.033 INFO:teuthology.orchestra.run.smithi143.stdout: "type": "float", 2023-11-26T07:23:10.033 INFO:teuthology.orchestra.run.smithi143.stdout: "level": "advanced", 2023-11-26T07:23:10.033 INFO:teuthology.orchestra.run.smithi143.stdout: "flags": 1, 2023-11-26T07:23:10.033 INFO:teuthology.orchestra.run.smithi143.stdout: "default_value": "0", 2023-11-26T07:23:10.033 INFO:teuthology.orchestra.run.smithi143.stdout: "min": "", 2023-11-26T07:23:10.033 INFO:teuthology.orchestra.run.smithi143.stdout: "max": "", 2023-11-26T07:23:10.033 INFO:teuthology.orchestra.run.smithi143.stdout: "enum_allowed": [], 2023-11-26T07:23:10.033 INFO:teuthology.orchestra.run.smithi143.stdout: "desc": "minimum score, below which no optimization is attempted", 2023-11-26T07:23:10.033 INFO:teuthology.orchestra.run.smithi143.stdout: "long_desc": "", 2023-11-26T07:23:10.033 INFO:teuthology.orchestra.run.smithi143.stdout: "tags": [], 2023-11-26T07:23:10.033 INFO:teuthology.orchestra.run.smithi143.stdout: "see_also": [] 2023-11-26T07:23:10.033 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2023-11-26T07:23:10.033 INFO:teuthology.orchestra.run.smithi143.stdout: "mode": { 2023-11-26T07:23:10.034 INFO:teuthology.orchestra.run.smithi143.stdout: "name": "mode", 2023-11-26T07:23:10.034 INFO:teuthology.orchestra.run.smithi143.stdout: "type": "str", 2023-11-26T07:23:10.034 INFO:teuthology.orchestra.run.smithi143.stdout: "level": "advanced", 2023-11-26T07:23:10.034 INFO:teuthology.orchestra.run.smithi143.stdout: "flags": 1, 2023-11-26T07:23:10.034 INFO:teuthology.orchestra.run.smithi143.stdout: "default_value": "upmap", 2023-11-26T07:23:10.034 INFO:teuthology.orchestra.run.smithi143.stdout: "min": "", 2023-11-26T07:23:10.034 INFO:teuthology.orchestra.run.smithi143.stdout: "max": "", 2023-11-26T07:23:10.034 INFO:teuthology.orchestra.run.smithi143.stdout: "enum_allowed": [ 2023-11-26T07:23:10.034 INFO:teuthology.orchestra.run.smithi143.stdout: "crush-compat", 2023-11-26T07:23:10.034 INFO:teuthology.orchestra.run.smithi143.stdout: "none", 2023-11-26T07:23:10.034 INFO:teuthology.orchestra.run.smithi143.stdout: "upmap" 2023-11-26T07:23:10.034 INFO:teuthology.orchestra.run.smithi143.stdout: ], 2023-11-26T07:23:10.035 INFO:teuthology.orchestra.run.smithi143.stdout: "desc": "Balancer mode", 2023-11-26T07:23:10.035 INFO:teuthology.orchestra.run.smithi143.stdout: "long_desc": "", 2023-11-26T07:23:10.035 INFO:teuthology.orchestra.run.smithi143.stdout: "tags": [], 2023-11-26T07:23:10.035 INFO:teuthology.orchestra.run.smithi143.stdout: "see_also": [] 2023-11-26T07:23:10.035 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2023-11-26T07:23:10.035 INFO:teuthology.orchestra.run.smithi143.stdout: "pool_ids": { 2023-11-26T07:23:10.035 INFO:teuthology.orchestra.run.smithi143.stdout: "name": "pool_ids", 2023-11-26T07:23:10.035 INFO:teuthology.orchestra.run.smithi143.stdout: "type": "str", 2023-11-26T07:23:10.035 INFO:teuthology.orchestra.run.smithi143.stdout: "level": "advanced", 2023-11-26T07:23:10.035 INFO:teuthology.orchestra.run.smithi143.stdout: "flags": 1, 2023-11-26T07:23:10.035 INFO:teuthology.orchestra.run.smithi143.stdout: "default_value": "", 2023-11-26T07:23:10.035 INFO:teuthology.orchestra.run.smithi143.stdout: "min": "", 2023-11-26T07:23:10.036 INFO:teuthology.orchestra.run.smithi143.stdout: "max": "", 2023-11-26T07:23:10.036 INFO:teuthology.orchestra.run.smithi143.stdout: "enum_allowed": [], 2023-11-26T07:23:10.036 INFO:teuthology.orchestra.run.smithi143.stdout: "desc": "pools which the automatic balancing will be limited to", 2023-11-26T07:23:10.036 INFO:teuthology.orchestra.run.smithi143.stdout: "long_desc": "", 2023-11-26T07:23:10.036 INFO:teuthology.orchestra.run.smithi143.stdout: "tags": [], 2023-11-26T07:23:10.036 INFO:teuthology.orchestra.run.smithi143.stdout: "see_also": [] 2023-11-26T07:23:10.036 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2023-11-26T07:23:10.036 INFO:teuthology.orchestra.run.smithi143.stdout: "sleep_interval": { 2023-11-26T07:23:10.036 INFO:teuthology.orchestra.run.smithi143.stdout: "name": "sleep_interval", 2023-11-26T07:23:10.036 INFO:teuthology.orchestra.run.smithi143.stdout: "type": "secs", 2023-11-26T07:23:10.036 INFO:teuthology.orchestra.run.smithi143.stdout: "level": "advanced", 2023-11-26T07:23:10.036 INFO:teuthology.orchestra.run.smithi143.stdout: "flags": 1, 2023-11-26T07:23:10.037 INFO:teuthology.orchestra.run.smithi143.stdout: "default_value": "60", 2023-11-26T07:23:10.037 INFO:teuthology.orchestra.run.smithi143.stdout: "min": "", 2023-11-26T07:23:10.037 INFO:teuthology.orchestra.run.smithi143.stdout: "max": "", 2023-11-26T07:23:10.037 INFO:teuthology.orchestra.run.smithi143.stdout: "enum_allowed": [], 2023-11-26T07:23:10.037 INFO:teuthology.orchestra.run.smithi143.stdout: "desc": "how frequently to wake up and attempt optimization", 2023-11-26T07:23:10.037 INFO:teuthology.orchestra.run.smithi143.stdout: "long_desc": "", 2023-11-26T07:23:10.037 INFO:teuthology.orchestra.run.smithi143.stdout: "tags": [], 2023-11-26T07:23:10.037 INFO:teuthology.orchestra.run.smithi143.stdout: "see_also": [] 2023-11-26T07:23:10.037 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2023-11-26T07:23:10.037 INFO:teuthology.orchestra.run.smithi143.stdout: "upmap_max_deviation": { 2023-11-26T07:23:10.037 INFO:teuthology.orchestra.run.smithi143.stdout: "name": "upmap_max_deviation", 2023-11-26T07:23:10.037 INFO:teuthology.orchestra.run.smithi143.stdout: "type": "int", 2023-11-26T07:23:10.037 INFO:teuthology.orchestra.run.smithi143.stdout: "level": "advanced", 2023-11-26T07:23:10.038 INFO:teuthology.orchestra.run.smithi143.stdout: "flags": 1, 2023-11-26T07:23:10.038 INFO:teuthology.orchestra.run.smithi143.stdout: "default_value": "5", 2023-11-26T07:23:10.038 INFO:teuthology.orchestra.run.smithi143.stdout: "min": "1", 2023-11-26T07:23:10.038 INFO:teuthology.orchestra.run.smithi143.stdout: "max": "", 2023-11-26T07:23:10.038 INFO:teuthology.orchestra.run.smithi143.stdout: "enum_allowed": [], 2023-11-26T07:23:10.038 INFO:teuthology.orchestra.run.smithi143.stdout: "desc": "deviation below which no optimization is attempted", 2023-11-26T07:23:10.038 INFO:teuthology.orchestra.run.smithi143.stdout: "long_desc": "If the number of PGs are within this count then no optimization is attempted", 2023-11-26T07:23:10.038 INFO:teuthology.orchestra.run.smithi143.stdout: "tags": [], 2023-11-26T07:23:10.038 INFO:teuthology.orchestra.run.smithi143.stdout: "see_also": [] 2023-11-26T07:23:10.038 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2023-11-26T07:23:10.038 INFO:teuthology.orchestra.run.smithi143.stdout: "upmap_max_optimizations": { 2023-11-26T07:23:10.038 INFO:teuthology.orchestra.run.smithi143.stdout: "name": "upmap_max_optimizations", 2023-11-26T07:23:10.039 INFO:teuthology.orchestra.run.smithi143.stdout: "type": "uint", 2023-11-26T07:23:10.039 INFO:teuthology.orchestra.run.smithi143.stdout: "level": "advanced", 2023-11-26T07:23:10.039 INFO:teuthology.orchestra.run.smithi143.stdout: "flags": 1, 2023-11-26T07:23:10.039 INFO:teuthology.orchestra.run.smithi143.stdout: "default_value": "10", 2023-11-26T07:23:10.039 INFO:teuthology.orchestra.run.smithi143.stdout: "min": "", 2023-11-26T07:23:10.039 INFO:teuthology.orchestra.run.smithi143.stdout: "max": "", 2023-11-26T07:23:10.039 INFO:teuthology.orchestra.run.smithi143.stdout: "enum_allowed": [], 2023-11-26T07:23:10.039 INFO:teuthology.orchestra.run.smithi143.stdout: "desc": "maximum upmap optimizations to make per attempt", 2023-11-26T07:23:10.039 INFO:teuthology.orchestra.run.smithi143.stdout: "long_desc": "", 2023-11-26T07:23:10.039 INFO:teuthology.orchestra.run.smithi143.stdout: "tags": [], 2023-11-26T07:23:10.039 INFO:teuthology.orchestra.run.smithi143.stdout: "see_also": [] 2023-11-26T07:23:10.039 INFO:teuthology.orchestra.run.smithi143.stdout: } 2023-11-26T07:23:10.039 INFO:teuthology.orchestra.run.smithi143.stdout: } 2023-11-26T07:23:10.040 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2023-11-26T07:23:10.040 INFO:teuthology.orchestra.run.smithi143.stdout: { 2023-11-26T07:23:10.040 INFO:teuthology.orchestra.run.smithi143.stdout: "name": "cephadm", 2023-11-26T07:23:10.040 INFO:teuthology.orchestra.run.smithi143.stdout: "can_run": true, 2023-11-26T07:23:10.040 INFO:teuthology.orchestra.run.smithi143.stdout: "error_string": "", 2023-11-26T07:23:10.040 INFO:teuthology.orchestra.run.smithi143.stdout: "module_options": { 2023-11-26T07:23:10.040 INFO:teuthology.orchestra.run.smithi143.stdout: "agent_down_multiplier": { 2023-11-26T07:23:10.040 INFO:teuthology.orchestra.run.smithi143.stdout: "name": "agent_down_multiplier", 2023-11-26T07:23:10.040 INFO:teuthology.orchestra.run.smithi143.stdout: "type": "float", 2023-11-26T07:23:10.040 INFO:teuthology.orchestra.run.smithi143.stdout: "level": "advanced", 2023-11-26T07:23:10.040 INFO:teuthology.orchestra.run.smithi143.stdout: "flags": 0, 2023-11-26T07:23:10.040 INFO:teuthology.orchestra.run.smithi143.stdout: "default_value": "3.0", 2023-11-26T07:23:10.040 INFO:teuthology.orchestra.run.smithi143.stdout: "min": "", 2023-11-26T07:23:10.041 INFO:teuthology.orchestra.run.smithi143.stdout: "max": "", 2023-11-26T07:23:10.041 INFO:teuthology.orchestra.run.smithi143.stdout: "enum_allowed": [], 2023-11-26T07:23:10.041 INFO:teuthology.orchestra.run.smithi143.stdout: "desc": "Multiplied by agent refresh rate to calculate how long agent must not report before being marked down", 2023-11-26T07:23:10.041 INFO:teuthology.orchestra.run.smithi143.stdout: "long_desc": "", 2023-11-26T07:23:10.041 INFO:teuthology.orchestra.run.smithi143.stdout: "tags": [], 2023-11-26T07:23:10.041 INFO:teuthology.orchestra.run.smithi143.stdout: "see_also": [] 2023-11-26T07:23:10.041 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2023-11-26T07:23:10.041 INFO:teuthology.orchestra.run.smithi143.stdout: "agent_refresh_rate": { 2023-11-26T07:23:10.041 INFO:teuthology.orchestra.run.smithi143.stdout: "name": "agent_refresh_rate", 2023-11-26T07:23:10.041 INFO:teuthology.orchestra.run.smithi143.stdout: "type": "secs", 2023-11-26T07:23:10.041 INFO:teuthology.orchestra.run.smithi143.stdout: "level": "advanced", 2023-11-26T07:23:10.041 INFO:teuthology.orchestra.run.smithi143.stdout: "flags": 0, 2023-11-26T07:23:10.041 INFO:teuthology.orchestra.run.smithi143.stdout: "default_value": "20", 2023-11-26T07:23:10.042 INFO:teuthology.orchestra.run.smithi143.stdout: "min": "", 2023-11-26T07:23:10.042 INFO:teuthology.orchestra.run.smithi143.stdout: "max": "", 2023-11-26T07:23:10.042 INFO:teuthology.orchestra.run.smithi143.stdout: "enum_allowed": [], 2023-11-26T07:23:10.042 INFO:teuthology.orchestra.run.smithi143.stdout: "desc": "How often agent on each host will try to gather and send metadata", 2023-11-26T07:23:10.042 INFO:teuthology.orchestra.run.smithi143.stdout: "long_desc": "", 2023-11-26T07:23:10.042 INFO:teuthology.orchestra.run.smithi143.stdout: "tags": [], 2023-11-26T07:23:10.042 INFO:teuthology.orchestra.run.smithi143.stdout: "see_also": [] 2023-11-26T07:23:10.042 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2023-11-26T07:23:10.042 INFO:teuthology.orchestra.run.smithi143.stdout: "agent_starting_port": { 2023-11-26T07:23:10.042 INFO:teuthology.orchestra.run.smithi143.stdout: "name": "agent_starting_port", 2023-11-26T07:23:10.042 INFO:teuthology.orchestra.run.smithi143.stdout: "type": "int", 2023-11-26T07:23:10.042 INFO:teuthology.orchestra.run.smithi143.stdout: "level": "advanced", 2023-11-26T07:23:10.042 INFO:teuthology.orchestra.run.smithi143.stdout: "flags": 0, 2023-11-26T07:23:10.043 INFO:teuthology.orchestra.run.smithi143.stdout: "default_value": "4721", 2023-11-26T07:23:10.043 INFO:teuthology.orchestra.run.smithi143.stdout: "min": "", 2023-11-26T07:23:10.043 INFO:teuthology.orchestra.run.smithi143.stdout: "max": "", 2023-11-26T07:23:10.043 INFO:teuthology.orchestra.run.smithi143.stdout: "enum_allowed": [], 2023-11-26T07:23:10.043 INFO:teuthology.orchestra.run.smithi143.stdout: "desc": "First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)", 2023-11-26T07:23:10.043 INFO:teuthology.orchestra.run.smithi143.stdout: "long_desc": "", 2023-11-26T07:23:10.043 INFO:teuthology.orchestra.run.smithi143.stdout: "tags": [], 2023-11-26T07:23:10.043 INFO:teuthology.orchestra.run.smithi143.stdout: "see_also": [] 2023-11-26T07:23:10.043 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2023-11-26T07:23:10.043 INFO:teuthology.orchestra.run.smithi143.stdout: "allow_ptrace": { 2023-11-26T07:23:10.043 INFO:teuthology.orchestra.run.smithi143.stdout: "name": "allow_ptrace", 2023-11-26T07:23:10.043 INFO:teuthology.orchestra.run.smithi143.stdout: "type": "bool", 2023-11-26T07:23:10.044 INFO:teuthology.orchestra.run.smithi143.stdout: "level": "advanced", 2023-11-26T07:23:10.044 INFO:teuthology.orchestra.run.smithi143.stdout: "flags": 0, 2023-11-26T07:23:10.044 INFO:teuthology.orchestra.run.smithi143.stdout: "default_value": "False", 2023-11-26T07:23:10.044 INFO:teuthology.orchestra.run.smithi143.stdout: "min": "", 2023-11-26T07:23:10.044 INFO:teuthology.orchestra.run.smithi143.stdout: "max": "", 2023-11-26T07:23:10.044 INFO:teuthology.orchestra.run.smithi143.stdout: "enum_allowed": [], 2023-11-26T07:23:10.044 INFO:teuthology.orchestra.run.smithi143.stdout: "desc": "allow SYS_PTRACE capability on ceph containers", 2023-11-26T07:23:10.044 INFO:teuthology.orchestra.run.smithi143.stdout: "long_desc": "The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.", 2023-11-26T07:23:10.044 INFO:teuthology.orchestra.run.smithi143.stdout: "tags": [], 2023-11-26T07:23:10.044 INFO:teuthology.orchestra.run.smithi143.stdout: "see_also": [] 2023-11-26T07:23:10.044 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2023-11-26T07:23:10.044 INFO:teuthology.orchestra.run.smithi143.stdout: "autotune_interval": { 2023-11-26T07:23:10.044 INFO:teuthology.orchestra.run.smithi143.stdout: "name": "autotune_interval", 2023-11-26T07:23:10.045 INFO:teuthology.orchestra.run.smithi143.stdout: "type": "secs", 2023-11-26T07:23:10.045 INFO:teuthology.orchestra.run.smithi143.stdout: "level": "advanced", 2023-11-26T07:23:10.045 INFO:teuthology.orchestra.run.smithi143.stdout: "flags": 0, 2023-11-26T07:23:10.045 INFO:teuthology.orchestra.run.smithi143.stdout: "default_value": "600", 2023-11-26T07:23:10.045 INFO:teuthology.orchestra.run.smithi143.stdout: "min": "", 2023-11-26T07:23:10.045 INFO:teuthology.orchestra.run.smithi143.stdout: "max": "", 2023-11-26T07:23:10.045 INFO:teuthology.orchestra.run.smithi143.stdout: "enum_allowed": [], 2023-11-26T07:23:10.045 INFO:teuthology.orchestra.run.smithi143.stdout: "desc": "how frequently to autotune daemon memory", 2023-11-26T07:23:10.045 INFO:teuthology.orchestra.run.smithi143.stdout: "long_desc": "", 2023-11-26T07:23:10.045 INFO:teuthology.orchestra.run.smithi143.stdout: "tags": [], 2023-11-26T07:23:10.045 INFO:teuthology.orchestra.run.smithi143.stdout: "see_also": [] 2023-11-26T07:23:10.045 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2023-11-26T07:23:10.045 INFO:teuthology.orchestra.run.smithi143.stdout: "autotune_memory_target_ratio": { 2023-11-26T07:23:10.046 INFO:teuthology.orchestra.run.smithi143.stdout: "name": "autotune_memory_target_ratio", 2023-11-26T07:23:10.046 INFO:teuthology.orchestra.run.smithi143.stdout: "type": "float", 2023-11-26T07:23:10.046 INFO:teuthology.orchestra.run.smithi143.stdout: "level": "advanced", 2023-11-26T07:23:10.046 INFO:teuthology.orchestra.run.smithi143.stdout: "flags": 0, 2023-11-26T07:23:10.046 INFO:teuthology.orchestra.run.smithi143.stdout: "default_value": "0.7", 2023-11-26T07:23:10.046 INFO:teuthology.orchestra.run.smithi143.stdout: "min": "", 2023-11-26T07:23:10.046 INFO:teuthology.orchestra.run.smithi143.stdout: "max": "", 2023-11-26T07:23:10.046 INFO:teuthology.orchestra.run.smithi143.stdout: "enum_allowed": [], 2023-11-26T07:23:10.046 INFO:teuthology.orchestra.run.smithi143.stdout: "desc": "ratio of total system memory to divide amongst autotuned daemons", 2023-11-26T07:23:10.046 INFO:teuthology.orchestra.run.smithi143.stdout: "long_desc": "", 2023-11-26T07:23:10.046 INFO:teuthology.orchestra.run.smithi143.stdout: "tags": [], 2023-11-26T07:23:10.046 INFO:teuthology.orchestra.run.smithi143.stdout: "see_also": [] 2023-11-26T07:23:10.047 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2023-11-26T07:23:10.047 INFO:teuthology.orchestra.run.smithi143.stdout: "cephadm_log_destination": { 2023-11-26T07:23:10.047 INFO:teuthology.orchestra.run.smithi143.stdout: "name": "cephadm_log_destination", 2023-11-26T07:23:10.047 INFO:teuthology.orchestra.run.smithi143.stdout: "type": "str", 2023-11-26T07:23:10.047 INFO:teuthology.orchestra.run.smithi143.stdout: "level": "advanced", 2023-11-26T07:23:10.047 INFO:teuthology.orchestra.run.smithi143.stdout: "flags": 0, 2023-11-26T07:23:10.047 INFO:teuthology.orchestra.run.smithi143.stdout: "default_value": "", 2023-11-26T07:23:10.047 INFO:teuthology.orchestra.run.smithi143.stdout: "min": "", 2023-11-26T07:23:10.047 INFO:teuthology.orchestra.run.smithi143.stdout: "max": "", 2023-11-26T07:23:10.047 INFO:teuthology.orchestra.run.smithi143.stdout: "enum_allowed": [ 2023-11-26T07:23:10.047 INFO:teuthology.orchestra.run.smithi143.stdout: "file", 2023-11-26T07:23:10.047 INFO:teuthology.orchestra.run.smithi143.stdout: "file,syslog", 2023-11-26T07:23:10.047 INFO:teuthology.orchestra.run.smithi143.stdout: "syslog" 2023-11-26T07:23:10.048 INFO:teuthology.orchestra.run.smithi143.stdout: ], 2023-11-26T07:23:10.048 INFO:teuthology.orchestra.run.smithi143.stdout: "desc": "Destination for cephadm command's persistent logging", 2023-11-26T07:23:10.048 INFO:teuthology.orchestra.run.smithi143.stdout: "long_desc": "", 2023-11-26T07:23:10.048 INFO:teuthology.orchestra.run.smithi143.stdout: "tags": [], 2023-11-26T07:23:10.048 INFO:teuthology.orchestra.run.smithi143.stdout: "see_also": [] 2023-11-26T07:23:10.048 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2023-11-26T07:23:10.048 INFO:teuthology.orchestra.run.smithi143.stdout: "cgroups_split": { 2023-11-26T07:23:10.048 INFO:teuthology.orchestra.run.smithi143.stdout: "name": "cgroups_split", 2023-11-26T07:23:10.048 INFO:teuthology.orchestra.run.smithi143.stdout: "type": "bool", 2023-11-26T07:23:10.048 INFO:teuthology.orchestra.run.smithi143.stdout: "level": "advanced", 2023-11-26T07:23:10.048 INFO:teuthology.orchestra.run.smithi143.stdout: "flags": 0, 2023-11-26T07:23:10.048 INFO:teuthology.orchestra.run.smithi143.stdout: "default_value": "True", 2023-11-26T07:23:10.048 INFO:teuthology.orchestra.run.smithi143.stdout: "min": "", 2023-11-26T07:23:10.049 INFO:teuthology.orchestra.run.smithi143.stdout: "max": "", 2023-11-26T07:23:10.049 INFO:teuthology.orchestra.run.smithi143.stdout: "enum_allowed": [], 2023-11-26T07:23:10.049 INFO:teuthology.orchestra.run.smithi143.stdout: "desc": "Pass --cgroups=split when cephadm creates containers (currently podman only)", 2023-11-26T07:23:10.049 INFO:teuthology.orchestra.run.smithi143.stdout: "long_desc": "", 2023-11-26T07:23:10.049 INFO:teuthology.orchestra.run.smithi143.stdout: "tags": [], 2023-11-26T07:23:10.049 INFO:teuthology.orchestra.run.smithi143.stdout: "see_also": [] 2023-11-26T07:23:10.049 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2023-11-26T07:23:10.049 INFO:teuthology.orchestra.run.smithi143.stdout: "config_checks_enabled": { 2023-11-26T07:23:10.049 INFO:teuthology.orchestra.run.smithi143.stdout: "name": "config_checks_enabled", 2023-11-26T07:23:10.049 INFO:teuthology.orchestra.run.smithi143.stdout: "type": "bool", 2023-11-26T07:23:10.049 INFO:teuthology.orchestra.run.smithi143.stdout: "level": "advanced", 2023-11-26T07:23:10.049 INFO:teuthology.orchestra.run.smithi143.stdout: "flags": 0, 2023-11-26T07:23:10.049 INFO:teuthology.orchestra.run.smithi143.stdout: "default_value": "False", 2023-11-26T07:23:10.050 INFO:teuthology.orchestra.run.smithi143.stdout: "min": "", 2023-11-26T07:23:10.050 INFO:teuthology.orchestra.run.smithi143.stdout: "max": "", 2023-11-26T07:23:10.050 INFO:teuthology.orchestra.run.smithi143.stdout: "enum_allowed": [], 2023-11-26T07:23:10.050 INFO:teuthology.orchestra.run.smithi143.stdout: "desc": "Enable or disable the cephadm configuration analysis", 2023-11-26T07:23:10.050 INFO:teuthology.orchestra.run.smithi143.stdout: "long_desc": "", 2023-11-26T07:23:10.050 INFO:teuthology.orchestra.run.smithi143.stdout: "tags": [], 2023-11-26T07:23:10.050 INFO:teuthology.orchestra.run.smithi143.stdout: "see_also": [] 2023-11-26T07:23:10.050 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2023-11-26T07:23:10.050 INFO:teuthology.orchestra.run.smithi143.stdout: "config_dashboard": { 2023-11-26T07:23:10.050 INFO:teuthology.orchestra.run.smithi143.stdout: "name": "config_dashboard", 2023-11-26T07:23:10.050 INFO:teuthology.orchestra.run.smithi143.stdout: "type": "bool", 2023-11-26T07:23:10.050 INFO:teuthology.orchestra.run.smithi143.stdout: "level": "advanced", 2023-11-26T07:23:10.050 INFO:teuthology.orchestra.run.smithi143.stdout: "flags": 0, 2023-11-26T07:23:10.051 INFO:teuthology.orchestra.run.smithi143.stdout: "default_value": "True", 2023-11-26T07:23:10.051 INFO:teuthology.orchestra.run.smithi143.stdout: "min": "", 2023-11-26T07:23:10.051 INFO:teuthology.orchestra.run.smithi143.stdout: "max": "", 2023-11-26T07:23:10.051 INFO:teuthology.orchestra.run.smithi143.stdout: "enum_allowed": [], 2023-11-26T07:23:10.051 INFO:teuthology.orchestra.run.smithi143.stdout: "desc": "manage configs like API endpoints in Dashboard.", 2023-11-26T07:23:10.051 INFO:teuthology.orchestra.run.smithi143.stdout: "long_desc": "", 2023-11-26T07:23:10.051 INFO:teuthology.orchestra.run.smithi143.stdout: "tags": [], 2023-11-26T07:23:10.051 INFO:teuthology.orchestra.run.smithi143.stdout: "see_also": [] 2023-11-26T07:23:10.051 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2023-11-26T07:23:10.051 INFO:teuthology.orchestra.run.smithi143.stdout: "container_image_alertmanager": { 2023-11-26T07:23:10.051 INFO:teuthology.orchestra.run.smithi143.stdout: "name": "container_image_alertmanager", 2023-11-26T07:23:10.051 INFO:teuthology.orchestra.run.smithi143.stdout: "type": "str", 2023-11-26T07:23:10.052 INFO:teuthology.orchestra.run.smithi143.stdout: "level": "advanced", 2023-11-26T07:23:10.052 INFO:teuthology.orchestra.run.smithi143.stdout: "flags": 0, 2023-11-26T07:23:10.052 INFO:teuthology.orchestra.run.smithi143.stdout: "default_value": "quay.io/prometheus/alertmanager:v0.25.0", 2023-11-26T07:23:10.052 INFO:teuthology.orchestra.run.smithi143.stdout: "min": "", 2023-11-26T07:23:10.052 INFO:teuthology.orchestra.run.smithi143.stdout: "max": "", 2023-11-26T07:23:10.052 INFO:teuthology.orchestra.run.smithi143.stdout: "enum_allowed": [], 2023-11-26T07:23:10.052 INFO:teuthology.orchestra.run.smithi143.stdout: "desc": "Prometheus container image", 2023-11-26T07:23:10.052 INFO:teuthology.orchestra.run.smithi143.stdout: "long_desc": "", 2023-11-26T07:23:10.052 INFO:teuthology.orchestra.run.smithi143.stdout: "tags": [], 2023-11-26T07:23:10.052 INFO:teuthology.orchestra.run.smithi143.stdout: "see_also": [] 2023-11-26T07:23:10.052 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2023-11-26T07:23:10.052 INFO:teuthology.orchestra.run.smithi143.stdout: "container_image_base": { 2023-11-26T07:23:10.052 INFO:teuthology.orchestra.run.smithi143.stdout: "name": "container_image_base", 2023-11-26T07:23:10.053 INFO:teuthology.orchestra.run.smithi143.stdout: "type": "str", 2023-11-26T07:23:10.053 INFO:teuthology.orchestra.run.smithi143.stdout: "level": "advanced", 2023-11-26T07:23:10.053 INFO:teuthology.orchestra.run.smithi143.stdout: "flags": 1, 2023-11-26T07:23:10.053 INFO:teuthology.orchestra.run.smithi143.stdout: "default_value": "quay.io/ceph/ceph", 2023-11-26T07:23:10.053 INFO:teuthology.orchestra.run.smithi143.stdout: "min": "", 2023-11-26T07:23:10.053 INFO:teuthology.orchestra.run.smithi143.stdout: "max": "", 2023-11-26T07:23:10.053 INFO:teuthology.orchestra.run.smithi143.stdout: "enum_allowed": [], 2023-11-26T07:23:10.053 INFO:teuthology.orchestra.run.smithi143.stdout: "desc": "Container image name, without the tag", 2023-11-26T07:23:10.053 INFO:teuthology.orchestra.run.smithi143.stdout: "long_desc": "", 2023-11-26T07:23:10.053 INFO:teuthology.orchestra.run.smithi143.stdout: "tags": [], 2023-11-26T07:23:10.053 INFO:teuthology.orchestra.run.smithi143.stdout: "see_also": [] 2023-11-26T07:23:10.053 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2023-11-26T07:23:10.054 INFO:teuthology.orchestra.run.smithi143.stdout: "container_image_elasticsearch": { 2023-11-26T07:23:10.054 INFO:teuthology.orchestra.run.smithi143.stdout: "name": "container_image_elasticsearch", 2023-11-26T07:23:10.054 INFO:teuthology.orchestra.run.smithi143.stdout: "type": "str", 2023-11-26T07:23:10.054 INFO:teuthology.orchestra.run.smithi143.stdout: "level": "advanced", 2023-11-26T07:23:10.054 INFO:teuthology.orchestra.run.smithi143.stdout: "flags": 0, 2023-11-26T07:23:10.054 INFO:teuthology.orchestra.run.smithi143.stdout: "default_value": "quay.io/omrizeneva/elasticsearch:6.8.23", 2023-11-26T07:23:10.054 INFO:teuthology.orchestra.run.smithi143.stdout: "min": "", 2023-11-26T07:23:10.054 INFO:teuthology.orchestra.run.smithi143.stdout: "max": "", 2023-11-26T07:23:10.054 INFO:teuthology.orchestra.run.smithi143.stdout: "enum_allowed": [], 2023-11-26T07:23:10.054 INFO:teuthology.orchestra.run.smithi143.stdout: "desc": "elasticsearch container image", 2023-11-26T07:23:10.054 INFO:teuthology.orchestra.run.smithi143.stdout: "long_desc": "", 2023-11-26T07:23:10.054 INFO:teuthology.orchestra.run.smithi143.stdout: "tags": [], 2023-11-26T07:23:10.055 INFO:teuthology.orchestra.run.smithi143.stdout: "see_also": [] 2023-11-26T07:23:10.055 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2023-11-26T07:23:10.055 INFO:teuthology.orchestra.run.smithi143.stdout: "container_image_grafana": { 2023-11-26T07:23:10.055 INFO:teuthology.orchestra.run.smithi143.stdout: "name": "container_image_grafana", 2023-11-26T07:23:10.055 INFO:teuthology.orchestra.run.smithi143.stdout: "type": "str", 2023-11-26T07:23:10.055 INFO:teuthology.orchestra.run.smithi143.stdout: "level": "advanced", 2023-11-26T07:23:10.055 INFO:teuthology.orchestra.run.smithi143.stdout: "flags": 0, 2023-11-26T07:23:10.055 INFO:teuthology.orchestra.run.smithi143.stdout: "default_value": "quay.io/ceph/ceph-grafana:9.4.7", 2023-11-26T07:23:10.055 INFO:teuthology.orchestra.run.smithi143.stdout: "min": "", 2023-11-26T07:23:10.055 INFO:teuthology.orchestra.run.smithi143.stdout: "max": "", 2023-11-26T07:23:10.055 INFO:teuthology.orchestra.run.smithi143.stdout: "enum_allowed": [], 2023-11-26T07:23:10.055 INFO:teuthology.orchestra.run.smithi143.stdout: "desc": "Prometheus container image", 2023-11-26T07:23:10.055 INFO:teuthology.orchestra.run.smithi143.stdout: "long_desc": "", 2023-11-26T07:23:10.055 INFO:teuthology.orchestra.run.smithi143.stdout: "tags": [], 2023-11-26T07:23:10.056 INFO:teuthology.orchestra.run.smithi143.stdout: "see_also": [] 2023-11-26T07:23:10.056 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2023-11-26T07:23:10.056 INFO:teuthology.orchestra.run.smithi143.stdout: "container_image_haproxy": { 2023-11-26T07:23:10.056 INFO:teuthology.orchestra.run.smithi143.stdout: "name": "container_image_haproxy", 2023-11-26T07:23:10.056 INFO:teuthology.orchestra.run.smithi143.stdout: "type": "str", 2023-11-26T07:23:10.056 INFO:teuthology.orchestra.run.smithi143.stdout: "level": "advanced", 2023-11-26T07:23:10.056 INFO:teuthology.orchestra.run.smithi143.stdout: "flags": 0, 2023-11-26T07:23:10.056 INFO:teuthology.orchestra.run.smithi143.stdout: "default_value": "quay.io/ceph/haproxy:2.3", 2023-11-26T07:23:10.056 INFO:teuthology.orchestra.run.smithi143.stdout: "min": "", 2023-11-26T07:23:10.056 INFO:teuthology.orchestra.run.smithi143.stdout: "max": "", 2023-11-26T07:23:10.056 INFO:teuthology.orchestra.run.smithi143.stdout: "enum_allowed": [], 2023-11-26T07:23:10.057 INFO:teuthology.orchestra.run.smithi143.stdout: "desc": "HAproxy container image", 2023-11-26T07:23:10.057 INFO:teuthology.orchestra.run.smithi143.stdout: "long_desc": "", 2023-11-26T07:23:10.057 INFO:teuthology.orchestra.run.smithi143.stdout: "tags": [], 2023-11-26T07:23:10.057 INFO:teuthology.orchestra.run.smithi143.stdout: "see_also": [] 2023-11-26T07:23:10.057 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2023-11-26T07:23:10.057 INFO:teuthology.orchestra.run.smithi143.stdout: "container_image_jaeger_agent": { 2023-11-26T07:23:10.057 INFO:teuthology.orchestra.run.smithi143.stdout: "name": "container_image_jaeger_agent", 2023-11-26T07:23:10.057 INFO:teuthology.orchestra.run.smithi143.stdout: "type": "str", 2023-11-26T07:23:10.057 INFO:teuthology.orchestra.run.smithi143.stdout: "level": "advanced", 2023-11-26T07:23:10.057 INFO:teuthology.orchestra.run.smithi143.stdout: "flags": 0, 2023-11-26T07:23:10.057 INFO:teuthology.orchestra.run.smithi143.stdout: "default_value": "quay.io/jaegertracing/jaeger-agent:1.29", 2023-11-26T07:23:10.057 INFO:teuthology.orchestra.run.smithi143.stdout: "min": "", 2023-11-26T07:23:10.057 INFO:teuthology.orchestra.run.smithi143.stdout: "max": "", 2023-11-26T07:23:10.058 INFO:teuthology.orchestra.run.smithi143.stdout: "enum_allowed": [], 2023-11-26T07:23:10.058 INFO:teuthology.orchestra.run.smithi143.stdout: "desc": "Jaeger agent container image", 2023-11-26T07:23:10.058 INFO:teuthology.orchestra.run.smithi143.stdout: "long_desc": "", 2023-11-26T07:23:10.058 INFO:teuthology.orchestra.run.smithi143.stdout: "tags": [], 2023-11-26T07:23:10.058 INFO:teuthology.orchestra.run.smithi143.stdout: "see_also": [] 2023-11-26T07:23:10.058 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2023-11-26T07:23:10.058 INFO:teuthology.orchestra.run.smithi143.stdout: "container_image_jaeger_collector": { 2023-11-26T07:23:10.058 INFO:teuthology.orchestra.run.smithi143.stdout: "name": "container_image_jaeger_collector", 2023-11-26T07:23:10.058 INFO:teuthology.orchestra.run.smithi143.stdout: "type": "str", 2023-11-26T07:23:10.058 INFO:teuthology.orchestra.run.smithi143.stdout: "level": "advanced", 2023-11-26T07:23:10.058 INFO:teuthology.orchestra.run.smithi143.stdout: "flags": 0, 2023-11-26T07:23:10.058 INFO:teuthology.orchestra.run.smithi143.stdout: "default_value": "quay.io/jaegertracing/jaeger-collector:1.29", 2023-11-26T07:23:10.059 INFO:teuthology.orchestra.run.smithi143.stdout: "min": "", 2023-11-26T07:23:10.059 INFO:teuthology.orchestra.run.smithi143.stdout: "max": "", 2023-11-26T07:23:10.059 INFO:teuthology.orchestra.run.smithi143.stdout: "enum_allowed": [], 2023-11-26T07:23:10.059 INFO:teuthology.orchestra.run.smithi143.stdout: "desc": "Jaeger collector container image", 2023-11-26T07:23:10.059 INFO:teuthology.orchestra.run.smithi143.stdout: "long_desc": "", 2023-11-26T07:23:10.059 INFO:teuthology.orchestra.run.smithi143.stdout: "tags": [], 2023-11-26T07:23:10.059 INFO:teuthology.orchestra.run.smithi143.stdout: "see_also": [] 2023-11-26T07:23:10.059 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2023-11-26T07:23:10.059 INFO:teuthology.orchestra.run.smithi143.stdout: "container_image_jaeger_query": { 2023-11-26T07:23:10.059 INFO:teuthology.orchestra.run.smithi143.stdout: "name": "container_image_jaeger_query", 2023-11-26T07:23:10.059 INFO:teuthology.orchestra.run.smithi143.stdout: "type": "str", 2023-11-26T07:23:10.059 INFO:teuthology.orchestra.run.smithi143.stdout: "level": "advanced", 2023-11-26T07:23:10.059 INFO:teuthology.orchestra.run.smithi143.stdout: "flags": 0, 2023-11-26T07:23:10.060 INFO:teuthology.orchestra.run.smithi143.stdout: "default_value": "quay.io/jaegertracing/jaeger-query:1.29", 2023-11-26T07:23:10.060 INFO:teuthology.orchestra.run.smithi143.stdout: "min": "", 2023-11-26T07:23:10.060 INFO:teuthology.orchestra.run.smithi143.stdout: "max": "", 2023-11-26T07:23:10.060 INFO:teuthology.orchestra.run.smithi143.stdout: "enum_allowed": [], 2023-11-26T07:23:10.060 INFO:teuthology.orchestra.run.smithi143.stdout: "desc": "Jaeger query container image", 2023-11-26T07:23:10.060 INFO:teuthology.orchestra.run.smithi143.stdout: "long_desc": "", 2023-11-26T07:23:10.060 INFO:teuthology.orchestra.run.smithi143.stdout: "tags": [], 2023-11-26T07:23:10.060 INFO:teuthology.orchestra.run.smithi143.stdout: "see_also": [] 2023-11-26T07:23:10.060 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2023-11-26T07:23:10.060 INFO:teuthology.orchestra.run.smithi143.stdout: "container_image_keepalived": { 2023-11-26T07:23:10.060 INFO:teuthology.orchestra.run.smithi143.stdout: "name": "container_image_keepalived", 2023-11-26T07:23:10.060 INFO:teuthology.orchestra.run.smithi143.stdout: "type": "str", 2023-11-26T07:23:10.061 INFO:teuthology.orchestra.run.smithi143.stdout: "level": "advanced", 2023-11-26T07:23:10.061 INFO:teuthology.orchestra.run.smithi143.stdout: "flags": 0, 2023-11-26T07:23:10.061 INFO:teuthology.orchestra.run.smithi143.stdout: "default_value": "quay.io/ceph/keepalived:2.2.4", 2023-11-26T07:23:10.061 INFO:teuthology.orchestra.run.smithi143.stdout: "min": "", 2023-11-26T07:23:10.061 INFO:teuthology.orchestra.run.smithi143.stdout: "max": "", 2023-11-26T07:23:10.061 INFO:teuthology.orchestra.run.smithi143.stdout: "enum_allowed": [], 2023-11-26T07:23:10.061 INFO:teuthology.orchestra.run.smithi143.stdout: "desc": "Keepalived container image", 2023-11-26T07:23:10.061 INFO:teuthology.orchestra.run.smithi143.stdout: "long_desc": "", 2023-11-26T07:23:10.061 INFO:teuthology.orchestra.run.smithi143.stdout: "tags": [], 2023-11-26T07:23:10.061 INFO:teuthology.orchestra.run.smithi143.stdout: "see_also": [] 2023-11-26T07:23:10.061 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2023-11-26T07:23:10.061 INFO:teuthology.orchestra.run.smithi143.stdout: "container_image_loki": { 2023-11-26T07:23:10.062 INFO:teuthology.orchestra.run.smithi143.stdout: "name": "container_image_loki", 2023-11-26T07:23:10.062 INFO:teuthology.orchestra.run.smithi143.stdout: "type": "str", 2023-11-26T07:23:10.062 INFO:teuthology.orchestra.run.smithi143.stdout: "level": "advanced", 2023-11-26T07:23:10.062 INFO:teuthology.orchestra.run.smithi143.stdout: "flags": 0, 2023-11-26T07:23:10.062 INFO:teuthology.orchestra.run.smithi143.stdout: "default_value": "docker.io/grafana/loki:2.4.0", 2023-11-26T07:23:10.062 INFO:teuthology.orchestra.run.smithi143.stdout: "min": "", 2023-11-26T07:23:10.062 INFO:teuthology.orchestra.run.smithi143.stdout: "max": "", 2023-11-26T07:23:10.062 INFO:teuthology.orchestra.run.smithi143.stdout: "enum_allowed": [], 2023-11-26T07:23:10.062 INFO:teuthology.orchestra.run.smithi143.stdout: "desc": "Loki container image", 2023-11-26T07:23:10.062 INFO:teuthology.orchestra.run.smithi143.stdout: "long_desc": "", 2023-11-26T07:23:10.062 INFO:teuthology.orchestra.run.smithi143.stdout: "tags": [], 2023-11-26T07:23:10.062 INFO:teuthology.orchestra.run.smithi143.stdout: "see_also": [] 2023-11-26T07:23:10.062 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2023-11-26T07:23:10.063 INFO:teuthology.orchestra.run.smithi143.stdout: "container_image_node_exporter": { 2023-11-26T07:23:10.063 INFO:teuthology.orchestra.run.smithi143.stdout: "name": "container_image_node_exporter", 2023-11-26T07:23:10.063 INFO:teuthology.orchestra.run.smithi143.stdout: "type": "str", 2023-11-26T07:23:10.063 INFO:teuthology.orchestra.run.smithi143.stdout: "level": "advanced", 2023-11-26T07:23:10.063 INFO:teuthology.orchestra.run.smithi143.stdout: "flags": 0, 2023-11-26T07:23:10.063 INFO:teuthology.orchestra.run.smithi143.stdout: "default_value": "quay.io/prometheus/node-exporter:v1.5.0", 2023-11-26T07:23:10.063 INFO:teuthology.orchestra.run.smithi143.stdout: "min": "", 2023-11-26T07:23:10.063 INFO:teuthology.orchestra.run.smithi143.stdout: "max": "", 2023-11-26T07:23:10.063 INFO:teuthology.orchestra.run.smithi143.stdout: "enum_allowed": [], 2023-11-26T07:23:10.063 INFO:teuthology.orchestra.run.smithi143.stdout: "desc": "Prometheus container image", 2023-11-26T07:23:10.063 INFO:teuthology.orchestra.run.smithi143.stdout: "long_desc": "", 2023-11-26T07:23:10.063 INFO:teuthology.orchestra.run.smithi143.stdout: "tags": [], 2023-11-26T07:23:10.063 INFO:teuthology.orchestra.run.smithi143.stdout: "see_also": [] 2023-11-26T07:23:10.064 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2023-11-26T07:23:10.064 INFO:teuthology.orchestra.run.smithi143.stdout: "container_image_nvmeof": { 2023-11-26T07:23:10.064 INFO:teuthology.orchestra.run.smithi143.stdout: "name": "container_image_nvmeof", 2023-11-26T07:23:10.064 INFO:teuthology.orchestra.run.smithi143.stdout: "type": "str", 2023-11-26T07:23:10.064 INFO:teuthology.orchestra.run.smithi143.stdout: "level": "advanced", 2023-11-26T07:23:10.064 INFO:teuthology.orchestra.run.smithi143.stdout: "flags": 0, 2023-11-26T07:23:10.064 INFO:teuthology.orchestra.run.smithi143.stdout: "default_value": "quay.io/ceph/nvmeof:latest", 2023-11-26T07:23:10.064 INFO:teuthology.orchestra.run.smithi143.stdout: "min": "", 2023-11-26T07:23:10.064 INFO:teuthology.orchestra.run.smithi143.stdout: "max": "", 2023-11-26T07:23:10.064 INFO:teuthology.orchestra.run.smithi143.stdout: "enum_allowed": [], 2023-11-26T07:23:10.064 INFO:teuthology.orchestra.run.smithi143.stdout: "desc": "Nvme-of container image", 2023-11-26T07:23:10.064 INFO:teuthology.orchestra.run.smithi143.stdout: "long_desc": "", 2023-11-26T07:23:10.064 INFO:teuthology.orchestra.run.smithi143.stdout: "tags": [], 2023-11-26T07:23:10.065 INFO:teuthology.orchestra.run.smithi143.stdout: "see_also": [] 2023-11-26T07:23:10.065 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2023-11-26T07:23:10.065 INFO:teuthology.orchestra.run.smithi143.stdout: "container_image_prometheus": { 2023-11-26T07:23:10.065 INFO:teuthology.orchestra.run.smithi143.stdout: "name": "container_image_prometheus", 2023-11-26T07:23:10.065 INFO:teuthology.orchestra.run.smithi143.stdout: "type": "str", 2023-11-26T07:23:10.065 INFO:teuthology.orchestra.run.smithi143.stdout: "level": "advanced", 2023-11-26T07:23:10.065 INFO:teuthology.orchestra.run.smithi143.stdout: "flags": 0, 2023-11-26T07:23:10.065 INFO:teuthology.orchestra.run.smithi143.stdout: "default_value": "quay.io/prometheus/prometheus:v2.43.0", 2023-11-26T07:23:10.065 INFO:teuthology.orchestra.run.smithi143.stdout: "min": "", 2023-11-26T07:23:10.065 INFO:teuthology.orchestra.run.smithi143.stdout: "max": "", 2023-11-26T07:23:10.065 INFO:teuthology.orchestra.run.smithi143.stdout: "enum_allowed": [], 2023-11-26T07:23:10.065 INFO:teuthology.orchestra.run.smithi143.stdout: "desc": "Prometheus container image", 2023-11-26T07:23:10.065 INFO:teuthology.orchestra.run.smithi143.stdout: "long_desc": "", 2023-11-26T07:23:10.066 INFO:teuthology.orchestra.run.smithi143.stdout: "tags": [], 2023-11-26T07:23:10.066 INFO:teuthology.orchestra.run.smithi143.stdout: "see_also": [] 2023-11-26T07:23:10.066 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2023-11-26T07:23:10.066 INFO:teuthology.orchestra.run.smithi143.stdout: "container_image_promtail": { 2023-11-26T07:23:10.066 INFO:teuthology.orchestra.run.smithi143.stdout: "name": "container_image_promtail", 2023-11-26T07:23:10.066 INFO:teuthology.orchestra.run.smithi143.stdout: "type": "str", 2023-11-26T07:23:10.066 INFO:teuthology.orchestra.run.smithi143.stdout: "level": "advanced", 2023-11-26T07:23:10.066 INFO:teuthology.orchestra.run.smithi143.stdout: "flags": 0, 2023-11-26T07:23:10.066 INFO:teuthology.orchestra.run.smithi143.stdout: "default_value": "docker.io/grafana/promtail:2.4.0", 2023-11-26T07:23:10.066 INFO:teuthology.orchestra.run.smithi143.stdout: "min": "", 2023-11-26T07:23:10.066 INFO:teuthology.orchestra.run.smithi143.stdout: "max": "", 2023-11-26T07:23:10.066 INFO:teuthology.orchestra.run.smithi143.stdout: "enum_allowed": [], 2023-11-26T07:23:10.067 INFO:teuthology.orchestra.run.smithi143.stdout: "desc": "Promtail container image", 2023-11-26T07:23:10.067 INFO:teuthology.orchestra.run.smithi143.stdout: "long_desc": "", 2023-11-26T07:23:10.067 INFO:teuthology.orchestra.run.smithi143.stdout: "tags": [], 2023-11-26T07:23:10.067 INFO:teuthology.orchestra.run.smithi143.stdout: "see_also": [] 2023-11-26T07:23:10.067 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2023-11-26T07:23:10.067 INFO:teuthology.orchestra.run.smithi143.stdout: "container_image_snmp_gateway": { 2023-11-26T07:23:10.067 INFO:teuthology.orchestra.run.smithi143.stdout: "name": "container_image_snmp_gateway", 2023-11-26T07:23:10.067 INFO:teuthology.orchestra.run.smithi143.stdout: "type": "str", 2023-11-26T07:23:10.067 INFO:teuthology.orchestra.run.smithi143.stdout: "level": "advanced", 2023-11-26T07:23:10.067 INFO:teuthology.orchestra.run.smithi143.stdout: "flags": 0, 2023-11-26T07:23:10.067 INFO:teuthology.orchestra.run.smithi143.stdout: "default_value": "docker.io/maxwo/snmp-notifier:v1.2.1", 2023-11-26T07:23:10.067 INFO:teuthology.orchestra.run.smithi143.stdout: "min": "", 2023-11-26T07:23:10.067 INFO:teuthology.orchestra.run.smithi143.stdout: "max": "", 2023-11-26T07:23:10.068 INFO:teuthology.orchestra.run.smithi143.stdout: "enum_allowed": [], 2023-11-26T07:23:10.068 INFO:teuthology.orchestra.run.smithi143.stdout: "desc": "SNMP Gateway container image", 2023-11-26T07:23:10.068 INFO:teuthology.orchestra.run.smithi143.stdout: "long_desc": "", 2023-11-26T07:23:10.068 INFO:teuthology.orchestra.run.smithi143.stdout: "tags": [], 2023-11-26T07:23:10.068 INFO:teuthology.orchestra.run.smithi143.stdout: "see_also": [] 2023-11-26T07:23:10.068 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2023-11-26T07:23:10.068 INFO:teuthology.orchestra.run.smithi143.stdout: "container_init": { 2023-11-26T07:23:10.068 INFO:teuthology.orchestra.run.smithi143.stdout: "name": "container_init", 2023-11-26T07:23:10.068 INFO:teuthology.orchestra.run.smithi143.stdout: "type": "bool", 2023-11-26T07:23:10.068 INFO:teuthology.orchestra.run.smithi143.stdout: "level": "advanced", 2023-11-26T07:23:10.068 INFO:teuthology.orchestra.run.smithi143.stdout: "flags": 0, 2023-11-26T07:23:10.068 INFO:teuthology.orchestra.run.smithi143.stdout: "default_value": "True", 2023-11-26T07:23:10.068 INFO:teuthology.orchestra.run.smithi143.stdout: "min": "", 2023-11-26T07:23:10.069 INFO:teuthology.orchestra.run.smithi143.stdout: "max": "", 2023-11-26T07:23:10.069 INFO:teuthology.orchestra.run.smithi143.stdout: "enum_allowed": [], 2023-11-26T07:23:10.069 INFO:teuthology.orchestra.run.smithi143.stdout: "desc": "Run podman/docker with `--init`", 2023-11-26T07:23:10.069 INFO:teuthology.orchestra.run.smithi143.stdout: "long_desc": "", 2023-11-26T07:23:10.069 INFO:teuthology.orchestra.run.smithi143.stdout: "tags": [], 2023-11-26T07:23:10.069 INFO:teuthology.orchestra.run.smithi143.stdout: "see_also": [] 2023-11-26T07:23:10.069 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2023-11-26T07:23:10.069 INFO:teuthology.orchestra.run.smithi143.stdout: "daemon_cache_timeout": { 2023-11-26T07:23:10.069 INFO:teuthology.orchestra.run.smithi143.stdout: "name": "daemon_cache_timeout", 2023-11-26T07:23:10.069 INFO:teuthology.orchestra.run.smithi143.stdout: "type": "secs", 2023-11-26T07:23:10.069 INFO:teuthology.orchestra.run.smithi143.stdout: "level": "advanced", 2023-11-26T07:23:10.069 INFO:teuthology.orchestra.run.smithi143.stdout: "flags": 0, 2023-11-26T07:23:10.069 INFO:teuthology.orchestra.run.smithi143.stdout: "default_value": "600", 2023-11-26T07:23:10.070 INFO:teuthology.orchestra.run.smithi143.stdout: "min": "", 2023-11-26T07:23:10.070 INFO:teuthology.orchestra.run.smithi143.stdout: "max": "", 2023-11-26T07:23:10.070 INFO:teuthology.orchestra.run.smithi143.stdout: "enum_allowed": [], 2023-11-26T07:23:10.070 INFO:teuthology.orchestra.run.smithi143.stdout: "desc": "seconds to cache service (daemon) inventory", 2023-11-26T07:23:10.070 INFO:teuthology.orchestra.run.smithi143.stdout: "long_desc": "", 2023-11-26T07:23:10.070 INFO:teuthology.orchestra.run.smithi143.stdout: "tags": [], 2023-11-26T07:23:10.070 INFO:teuthology.orchestra.run.smithi143.stdout: "see_also": [] 2023-11-26T07:23:10.070 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2023-11-26T07:23:10.070 INFO:teuthology.orchestra.run.smithi143.stdout: "default_cephadm_command_timeout": { 2023-11-26T07:23:10.070 INFO:teuthology.orchestra.run.smithi143.stdout: "name": "default_cephadm_command_timeout", 2023-11-26T07:23:10.070 INFO:teuthology.orchestra.run.smithi143.stdout: "type": "secs", 2023-11-26T07:23:10.071 INFO:teuthology.orchestra.run.smithi143.stdout: "level": "advanced", 2023-11-26T07:23:10.071 INFO:teuthology.orchestra.run.smithi143.stdout: "flags": 0, 2023-11-26T07:23:10.071 INFO:teuthology.orchestra.run.smithi143.stdout: "default_value": "900", 2023-11-26T07:23:10.071 INFO:teuthology.orchestra.run.smithi143.stdout: "min": "", 2023-11-26T07:23:10.071 INFO:teuthology.orchestra.run.smithi143.stdout: "max": "", 2023-11-26T07:23:10.071 INFO:teuthology.orchestra.run.smithi143.stdout: "enum_allowed": [], 2023-11-26T07:23:10.071 INFO:teuthology.orchestra.run.smithi143.stdout: "desc": "Default timeout applied to cephadm commands run directly on the host (in seconds)", 2023-11-26T07:23:10.071 INFO:teuthology.orchestra.run.smithi143.stdout: "long_desc": "", 2023-11-26T07:23:10.071 INFO:teuthology.orchestra.run.smithi143.stdout: "tags": [], 2023-11-26T07:23:10.071 INFO:teuthology.orchestra.run.smithi143.stdout: "see_also": [] 2023-11-26T07:23:10.071 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2023-11-26T07:23:10.071 INFO:teuthology.orchestra.run.smithi143.stdout: "default_registry": { 2023-11-26T07:23:10.071 INFO:teuthology.orchestra.run.smithi143.stdout: "name": "default_registry", 2023-11-26T07:23:10.072 INFO:teuthology.orchestra.run.smithi143.stdout: "type": "str", 2023-11-26T07:23:10.072 INFO:teuthology.orchestra.run.smithi143.stdout: "level": "advanced", 2023-11-26T07:23:10.072 INFO:teuthology.orchestra.run.smithi143.stdout: "flags": 0, 2023-11-26T07:23:10.072 INFO:teuthology.orchestra.run.smithi143.stdout: "default_value": "docker.io", 2023-11-26T07:23:10.072 INFO:teuthology.orchestra.run.smithi143.stdout: "min": "", 2023-11-26T07:23:10.072 INFO:teuthology.orchestra.run.smithi143.stdout: "max": "", 2023-11-26T07:23:10.072 INFO:teuthology.orchestra.run.smithi143.stdout: "enum_allowed": [], 2023-11-26T07:23:10.072 INFO:teuthology.orchestra.run.smithi143.stdout: "desc": "Search-registry to which we should normalize unqualified image names. This is not the default registry", 2023-11-26T07:23:10.072 INFO:teuthology.orchestra.run.smithi143.stdout: "long_desc": "", 2023-11-26T07:23:10.072 INFO:teuthology.orchestra.run.smithi143.stdout: "tags": [], 2023-11-26T07:23:10.072 INFO:teuthology.orchestra.run.smithi143.stdout: "see_also": [] 2023-11-26T07:23:10.072 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2023-11-26T07:23:10.072 INFO:teuthology.orchestra.run.smithi143.stdout: "device_cache_timeout": { 2023-11-26T07:23:10.073 INFO:teuthology.orchestra.run.smithi143.stdout: "name": "device_cache_timeout", 2023-11-26T07:23:10.073 INFO:teuthology.orchestra.run.smithi143.stdout: "type": "secs", 2023-11-26T07:23:10.073 INFO:teuthology.orchestra.run.smithi143.stdout: "level": "advanced", 2023-11-26T07:23:10.073 INFO:teuthology.orchestra.run.smithi143.stdout: "flags": 0, 2023-11-26T07:23:10.073 INFO:teuthology.orchestra.run.smithi143.stdout: "default_value": "1800", 2023-11-26T07:23:10.073 INFO:teuthology.orchestra.run.smithi143.stdout: "min": "", 2023-11-26T07:23:10.073 INFO:teuthology.orchestra.run.smithi143.stdout: "max": "", 2023-11-26T07:23:10.073 INFO:teuthology.orchestra.run.smithi143.stdout: "enum_allowed": [], 2023-11-26T07:23:10.073 INFO:teuthology.orchestra.run.smithi143.stdout: "desc": "seconds to cache device inventory", 2023-11-26T07:23:10.073 INFO:teuthology.orchestra.run.smithi143.stdout: "long_desc": "", 2023-11-26T07:23:10.073 INFO:teuthology.orchestra.run.smithi143.stdout: "tags": [], 2023-11-26T07:23:10.073 INFO:teuthology.orchestra.run.smithi143.stdout: "see_also": [] 2023-11-26T07:23:10.074 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2023-11-26T07:23:10.074 INFO:teuthology.orchestra.run.smithi143.stdout: "device_enhanced_scan": { 2023-11-26T07:23:10.074 INFO:teuthology.orchestra.run.smithi143.stdout: "name": "device_enhanced_scan", 2023-11-26T07:23:10.074 INFO:teuthology.orchestra.run.smithi143.stdout: "type": "bool", 2023-11-26T07:23:10.074 INFO:teuthology.orchestra.run.smithi143.stdout: "level": "advanced", 2023-11-26T07:23:10.074 INFO:teuthology.orchestra.run.smithi143.stdout: "flags": 0, 2023-11-26T07:23:10.074 INFO:teuthology.orchestra.run.smithi143.stdout: "default_value": "False", 2023-11-26T07:23:10.074 INFO:teuthology.orchestra.run.smithi143.stdout: "min": "", 2023-11-26T07:23:10.074 INFO:teuthology.orchestra.run.smithi143.stdout: "max": "", 2023-11-26T07:23:10.074 INFO:teuthology.orchestra.run.smithi143.stdout: "enum_allowed": [], 2023-11-26T07:23:10.074 INFO:teuthology.orchestra.run.smithi143.stdout: "desc": "Use libstoragemgmt during device scans", 2023-11-26T07:23:10.074 INFO:teuthology.orchestra.run.smithi143.stdout: "long_desc": "", 2023-11-26T07:23:10.075 INFO:teuthology.orchestra.run.smithi143.stdout: "tags": [], 2023-11-26T07:23:10.075 INFO:teuthology.orchestra.run.smithi143.stdout: "see_also": [] 2023-11-26T07:23:10.075 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2023-11-26T07:23:10.075 INFO:teuthology.orchestra.run.smithi143.stdout: "facts_cache_timeout": { 2023-11-26T07:23:10.075 INFO:teuthology.orchestra.run.smithi143.stdout: "name": "facts_cache_timeout", 2023-11-26T07:23:10.075 INFO:teuthology.orchestra.run.smithi143.stdout: "type": "secs", 2023-11-26T07:23:10.075 INFO:teuthology.orchestra.run.smithi143.stdout: "level": "advanced", 2023-11-26T07:23:10.075 INFO:teuthology.orchestra.run.smithi143.stdout: "flags": 0, 2023-11-26T07:23:10.075 INFO:teuthology.orchestra.run.smithi143.stdout: "default_value": "60", 2023-11-26T07:23:10.075 INFO:teuthology.orchestra.run.smithi143.stdout: "min": "", 2023-11-26T07:23:10.075 INFO:teuthology.orchestra.run.smithi143.stdout: "max": "", 2023-11-26T07:23:10.075 INFO:teuthology.orchestra.run.smithi143.stdout: "enum_allowed": [], 2023-11-26T07:23:10.076 INFO:teuthology.orchestra.run.smithi143.stdout: "desc": "seconds to cache host facts data", 2023-11-26T07:23:10.076 INFO:teuthology.orchestra.run.smithi143.stdout: "long_desc": "", 2023-11-26T07:23:10.076 INFO:teuthology.orchestra.run.smithi143.stdout: "tags": [], 2023-11-26T07:23:10.076 INFO:teuthology.orchestra.run.smithi143.stdout: "see_also": [] 2023-11-26T07:23:10.076 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2023-11-26T07:23:10.076 INFO:teuthology.orchestra.run.smithi143.stdout: "host_check_interval": { 2023-11-26T07:23:10.076 INFO:teuthology.orchestra.run.smithi143.stdout: "name": "host_check_interval", 2023-11-26T07:23:10.076 INFO:teuthology.orchestra.run.smithi143.stdout: "type": "secs", 2023-11-26T07:23:10.076 INFO:teuthology.orchestra.run.smithi143.stdout: "level": "advanced", 2023-11-26T07:23:10.076 INFO:teuthology.orchestra.run.smithi143.stdout: "flags": 0, 2023-11-26T07:23:10.076 INFO:teuthology.orchestra.run.smithi143.stdout: "default_value": "600", 2023-11-26T07:23:10.076 INFO:teuthology.orchestra.run.smithi143.stdout: "min": "", 2023-11-26T07:23:10.076 INFO:teuthology.orchestra.run.smithi143.stdout: "max": "", 2023-11-26T07:23:10.077 INFO:teuthology.orchestra.run.smithi143.stdout: "enum_allowed": [], 2023-11-26T07:23:10.077 INFO:teuthology.orchestra.run.smithi143.stdout: "desc": "how frequently to perform a host check", 2023-11-26T07:23:10.077 INFO:teuthology.orchestra.run.smithi143.stdout: "long_desc": "", 2023-11-26T07:23:10.077 INFO:teuthology.orchestra.run.smithi143.stdout: "tags": [], 2023-11-26T07:23:10.077 INFO:teuthology.orchestra.run.smithi143.stdout: "see_also": [] 2023-11-26T07:23:10.077 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2023-11-26T07:23:10.077 INFO:teuthology.orchestra.run.smithi143.stdout: "inventory_list_all": { 2023-11-26T07:23:10.077 INFO:teuthology.orchestra.run.smithi143.stdout: "name": "inventory_list_all", 2023-11-26T07:23:10.077 INFO:teuthology.orchestra.run.smithi143.stdout: "type": "bool", 2023-11-26T07:23:10.077 INFO:teuthology.orchestra.run.smithi143.stdout: "level": "advanced", 2023-11-26T07:23:10.077 INFO:teuthology.orchestra.run.smithi143.stdout: "flags": 0, 2023-11-26T07:23:10.077 INFO:teuthology.orchestra.run.smithi143.stdout: "default_value": "False", 2023-11-26T07:23:10.077 INFO:teuthology.orchestra.run.smithi143.stdout: "min": "", 2023-11-26T07:23:10.078 INFO:teuthology.orchestra.run.smithi143.stdout: "max": "", 2023-11-26T07:23:10.078 INFO:teuthology.orchestra.run.smithi143.stdout: "enum_allowed": [], 2023-11-26T07:23:10.078 INFO:teuthology.orchestra.run.smithi143.stdout: "desc": "Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)", 2023-11-26T07:23:10.078 INFO:teuthology.orchestra.run.smithi143.stdout: "long_desc": "", 2023-11-26T07:23:10.078 INFO:teuthology.orchestra.run.smithi143.stdout: "tags": [], 2023-11-26T07:23:10.078 INFO:teuthology.orchestra.run.smithi143.stdout: "see_also": [] 2023-11-26T07:23:10.078 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2023-11-26T07:23:10.078 INFO:teuthology.orchestra.run.smithi143.stdout: "log_level": { 2023-11-26T07:23:10.078 INFO:teuthology.orchestra.run.smithi143.stdout: "name": "log_level", 2023-11-26T07:23:10.078 INFO:teuthology.orchestra.run.smithi143.stdout: "type": "str", 2023-11-26T07:23:10.078 INFO:teuthology.orchestra.run.smithi143.stdout: "level": "advanced", 2023-11-26T07:23:10.078 INFO:teuthology.orchestra.run.smithi143.stdout: "flags": 1, 2023-11-26T07:23:10.079 INFO:teuthology.orchestra.run.smithi143.stdout: "default_value": "", 2023-11-26T07:23:10.079 INFO:teuthology.orchestra.run.smithi143.stdout: "min": "", 2023-11-26T07:23:10.079 INFO:teuthology.orchestra.run.smithi143.stdout: "max": "", 2023-11-26T07:23:10.079 INFO:teuthology.orchestra.run.smithi143.stdout: "enum_allowed": [ 2023-11-26T07:23:10.079 INFO:teuthology.orchestra.run.smithi143.stdout: "", 2023-11-26T07:23:10.079 INFO:teuthology.orchestra.run.smithi143.stdout: "critical", 2023-11-26T07:23:10.079 INFO:teuthology.orchestra.run.smithi143.stdout: "debug", 2023-11-26T07:23:10.079 INFO:teuthology.orchestra.run.smithi143.stdout: "error", 2023-11-26T07:23:10.079 INFO:teuthology.orchestra.run.smithi143.stdout: "info", 2023-11-26T07:23:10.079 INFO:teuthology.orchestra.run.smithi143.stdout: "warning" 2023-11-26T07:23:10.079 INFO:teuthology.orchestra.run.smithi143.stdout: ], 2023-11-26T07:23:10.079 INFO:teuthology.orchestra.run.smithi143.stdout: "desc": "", 2023-11-26T07:23:10.079 INFO:teuthology.orchestra.run.smithi143.stdout: "long_desc": "", 2023-11-26T07:23:10.080 INFO:teuthology.orchestra.run.smithi143.stdout: "tags": [], 2023-11-26T07:23:10.080 INFO:teuthology.orchestra.run.smithi143.stdout: "see_also": [] 2023-11-26T07:23:10.080 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2023-11-26T07:23:10.080 INFO:teuthology.orchestra.run.smithi143.stdout: "log_refresh_metadata": { 2023-11-26T07:23:10.080 INFO:teuthology.orchestra.run.smithi143.stdout: "name": "log_refresh_metadata", 2023-11-26T07:23:10.080 INFO:teuthology.orchestra.run.smithi143.stdout: "type": "bool", 2023-11-26T07:23:10.080 INFO:teuthology.orchestra.run.smithi143.stdout: "level": "advanced", 2023-11-26T07:23:10.080 INFO:teuthology.orchestra.run.smithi143.stdout: "flags": 0, 2023-11-26T07:23:10.080 INFO:teuthology.orchestra.run.smithi143.stdout: "default_value": "False", 2023-11-26T07:23:10.080 INFO:teuthology.orchestra.run.smithi143.stdout: "min": "", 2023-11-26T07:23:10.080 INFO:teuthology.orchestra.run.smithi143.stdout: "max": "", 2023-11-26T07:23:10.080 INFO:teuthology.orchestra.run.smithi143.stdout: "enum_allowed": [], 2023-11-26T07:23:10.081 INFO:teuthology.orchestra.run.smithi143.stdout: "desc": "Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level", 2023-11-26T07:23:10.081 INFO:teuthology.orchestra.run.smithi143.stdout: "long_desc": "", 2023-11-26T07:23:10.081 INFO:teuthology.orchestra.run.smithi143.stdout: "tags": [], 2023-11-26T07:23:10.081 INFO:teuthology.orchestra.run.smithi143.stdout: "see_also": [] 2023-11-26T07:23:10.081 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2023-11-26T07:23:10.081 INFO:teuthology.orchestra.run.smithi143.stdout: "log_to_cluster": { 2023-11-26T07:23:10.081 INFO:teuthology.orchestra.run.smithi143.stdout: "name": "log_to_cluster", 2023-11-26T07:23:10.081 INFO:teuthology.orchestra.run.smithi143.stdout: "type": "bool", 2023-11-26T07:23:10.081 INFO:teuthology.orchestra.run.smithi143.stdout: "level": "advanced", 2023-11-26T07:23:10.081 INFO:teuthology.orchestra.run.smithi143.stdout: "flags": 0, 2023-11-26T07:23:10.081 INFO:teuthology.orchestra.run.smithi143.stdout: "default_value": "True", 2023-11-26T07:23:10.081 INFO:teuthology.orchestra.run.smithi143.stdout: "min": "", 2023-11-26T07:23:10.081 INFO:teuthology.orchestra.run.smithi143.stdout: "max": "", 2023-11-26T07:23:10.082 INFO:teuthology.orchestra.run.smithi143.stdout: "enum_allowed": [], 2023-11-26T07:23:10.082 INFO:teuthology.orchestra.run.smithi143.stdout: "desc": "log to the \"cephadm\" cluster log channel\"", 2023-11-26T07:23:10.082 INFO:teuthology.orchestra.run.smithi143.stdout: "long_desc": "", 2023-11-26T07:23:10.082 INFO:teuthology.orchestra.run.smithi143.stdout: "tags": [], 2023-11-26T07:23:10.082 INFO:teuthology.orchestra.run.smithi143.stdout: "see_also": [] 2023-11-26T07:23:10.082 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2023-11-26T07:23:10.082 INFO:teuthology.orchestra.run.smithi143.stdout: "log_to_cluster_level": { 2023-11-26T07:23:10.082 INFO:teuthology.orchestra.run.smithi143.stdout: "name": "log_to_cluster_level", 2023-11-26T07:23:10.082 INFO:teuthology.orchestra.run.smithi143.stdout: "type": "str", 2023-11-26T07:23:10.082 INFO:teuthology.orchestra.run.smithi143.stdout: "level": "advanced", 2023-11-26T07:23:10.082 INFO:teuthology.orchestra.run.smithi143.stdout: "flags": 1, 2023-11-26T07:23:10.082 INFO:teuthology.orchestra.run.smithi143.stdout: "default_value": "info", 2023-11-26T07:23:10.082 INFO:teuthology.orchestra.run.smithi143.stdout: "min": "", 2023-11-26T07:23:10.083 INFO:teuthology.orchestra.run.smithi143.stdout: "max": "", 2023-11-26T07:23:10.083 INFO:teuthology.orchestra.run.smithi143.stdout: "enum_allowed": [ 2023-11-26T07:23:10.083 INFO:teuthology.orchestra.run.smithi143.stdout: "", 2023-11-26T07:23:10.083 INFO:teuthology.orchestra.run.smithi143.stdout: "critical", 2023-11-26T07:23:10.083 INFO:teuthology.orchestra.run.smithi143.stdout: "debug", 2023-11-26T07:23:10.083 INFO:teuthology.orchestra.run.smithi143.stdout: "error", 2023-11-26T07:23:10.083 INFO:teuthology.orchestra.run.smithi143.stdout: "info", 2023-11-26T07:23:10.083 INFO:teuthology.orchestra.run.smithi143.stdout: "warning" 2023-11-26T07:23:10.083 INFO:teuthology.orchestra.run.smithi143.stdout: ], 2023-11-26T07:23:10.083 INFO:teuthology.orchestra.run.smithi143.stdout: "desc": "", 2023-11-26T07:23:10.083 INFO:teuthology.orchestra.run.smithi143.stdout: "long_desc": "", 2023-11-26T07:23:10.083 INFO:teuthology.orchestra.run.smithi143.stdout: "tags": [], 2023-11-26T07:23:10.084 INFO:teuthology.orchestra.run.smithi143.stdout: "see_also": [] 2023-11-26T07:23:10.084 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2023-11-26T07:23:10.084 INFO:teuthology.orchestra.run.smithi143.stdout: "log_to_file": { 2023-11-26T07:23:10.084 INFO:teuthology.orchestra.run.smithi143.stdout: "name": "log_to_file", 2023-11-26T07:23:10.084 INFO:teuthology.orchestra.run.smithi143.stdout: "type": "bool", 2023-11-26T07:23:10.084 INFO:teuthology.orchestra.run.smithi143.stdout: "level": "advanced", 2023-11-26T07:23:10.084 INFO:teuthology.orchestra.run.smithi143.stdout: "flags": 1, 2023-11-26T07:23:10.084 INFO:teuthology.orchestra.run.smithi143.stdout: "default_value": "False", 2023-11-26T07:23:10.084 INFO:teuthology.orchestra.run.smithi143.stdout: "min": "", 2023-11-26T07:23:10.084 INFO:teuthology.orchestra.run.smithi143.stdout: "max": "", 2023-11-26T07:23:10.084 INFO:teuthology.orchestra.run.smithi143.stdout: "enum_allowed": [], 2023-11-26T07:23:10.084 INFO:teuthology.orchestra.run.smithi143.stdout: "desc": "", 2023-11-26T07:23:10.084 INFO:teuthology.orchestra.run.smithi143.stdout: "long_desc": "", 2023-11-26T07:23:10.085 INFO:teuthology.orchestra.run.smithi143.stdout: "tags": [], 2023-11-26T07:23:10.085 INFO:teuthology.orchestra.run.smithi143.stdout: "see_also": [] 2023-11-26T07:23:10.085 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2023-11-26T07:23:10.085 INFO:teuthology.orchestra.run.smithi143.stdout: "manage_etc_ceph_ceph_conf": { 2023-11-26T07:23:10.085 INFO:teuthology.orchestra.run.smithi143.stdout: "name": "manage_etc_ceph_ceph_conf", 2023-11-26T07:23:10.085 INFO:teuthology.orchestra.run.smithi143.stdout: "type": "bool", 2023-11-26T07:23:10.085 INFO:teuthology.orchestra.run.smithi143.stdout: "level": "advanced", 2023-11-26T07:23:10.085 INFO:teuthology.orchestra.run.smithi143.stdout: "flags": 0, 2023-11-26T07:23:10.085 INFO:teuthology.orchestra.run.smithi143.stdout: "default_value": "False", 2023-11-26T07:23:10.085 INFO:teuthology.orchestra.run.smithi143.stdout: "min": "", 2023-11-26T07:23:10.085 INFO:teuthology.orchestra.run.smithi143.stdout: "max": "", 2023-11-26T07:23:10.085 INFO:teuthology.orchestra.run.smithi143.stdout: "enum_allowed": [], 2023-11-26T07:23:10.085 INFO:teuthology.orchestra.run.smithi143.stdout: "desc": "Manage and own /etc/ceph/ceph.conf on the hosts.", 2023-11-26T07:23:10.086 INFO:teuthology.orchestra.run.smithi143.stdout: "long_desc": "", 2023-11-26T07:23:10.086 INFO:teuthology.orchestra.run.smithi143.stdout: "tags": [], 2023-11-26T07:23:10.086 INFO:teuthology.orchestra.run.smithi143.stdout: "see_also": [] 2023-11-26T07:23:10.086 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2023-11-26T07:23:10.086 INFO:teuthology.orchestra.run.smithi143.stdout: "manage_etc_ceph_ceph_conf_hosts": { 2023-11-26T07:23:10.086 INFO:teuthology.orchestra.run.smithi143.stdout: "name": "manage_etc_ceph_ceph_conf_hosts", 2023-11-26T07:23:10.086 INFO:teuthology.orchestra.run.smithi143.stdout: "type": "str", 2023-11-26T07:23:10.086 INFO:teuthology.orchestra.run.smithi143.stdout: "level": "advanced", 2023-11-26T07:23:10.086 INFO:teuthology.orchestra.run.smithi143.stdout: "flags": 0, 2023-11-26T07:23:10.086 INFO:teuthology.orchestra.run.smithi143.stdout: "default_value": "*", 2023-11-26T07:23:10.086 INFO:teuthology.orchestra.run.smithi143.stdout: "min": "", 2023-11-26T07:23:10.086 INFO:teuthology.orchestra.run.smithi143.stdout: "max": "", 2023-11-26T07:23:10.087 INFO:teuthology.orchestra.run.smithi143.stdout: "enum_allowed": [], 2023-11-26T07:23:10.087 INFO:teuthology.orchestra.run.smithi143.stdout: "desc": "PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf", 2023-11-26T07:23:10.087 INFO:teuthology.orchestra.run.smithi143.stdout: "long_desc": "", 2023-11-26T07:23:10.087 INFO:teuthology.orchestra.run.smithi143.stdout: "tags": [], 2023-11-26T07:23:10.087 INFO:teuthology.orchestra.run.smithi143.stdout: "see_also": [] 2023-11-26T07:23:10.087 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2023-11-26T07:23:10.087 INFO:teuthology.orchestra.run.smithi143.stdout: "max_count_per_host": { 2023-11-26T07:23:10.087 INFO:teuthology.orchestra.run.smithi143.stdout: "name": "max_count_per_host", 2023-11-26T07:23:10.087 INFO:teuthology.orchestra.run.smithi143.stdout: "type": "int", 2023-11-26T07:23:10.087 INFO:teuthology.orchestra.run.smithi143.stdout: "level": "advanced", 2023-11-26T07:23:10.087 INFO:teuthology.orchestra.run.smithi143.stdout: "flags": 0, 2023-11-26T07:23:10.087 INFO:teuthology.orchestra.run.smithi143.stdout: "default_value": "10", 2023-11-26T07:23:10.087 INFO:teuthology.orchestra.run.smithi143.stdout: "min": "", 2023-11-26T07:23:10.088 INFO:teuthology.orchestra.run.smithi143.stdout: "max": "", 2023-11-26T07:23:10.088 INFO:teuthology.orchestra.run.smithi143.stdout: "enum_allowed": [], 2023-11-26T07:23:10.088 INFO:teuthology.orchestra.run.smithi143.stdout: "desc": "max number of daemons per service per host", 2023-11-26T07:23:10.088 INFO:teuthology.orchestra.run.smithi143.stdout: "long_desc": "", 2023-11-26T07:23:10.088 INFO:teuthology.orchestra.run.smithi143.stdout: "tags": [], 2023-11-26T07:23:10.088 INFO:teuthology.orchestra.run.smithi143.stdout: "see_also": [] 2023-11-26T07:23:10.088 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2023-11-26T07:23:10.088 INFO:teuthology.orchestra.run.smithi143.stdout: "max_osd_draining_count": { 2023-11-26T07:23:10.088 INFO:teuthology.orchestra.run.smithi143.stdout: "name": "max_osd_draining_count", 2023-11-26T07:23:10.088 INFO:teuthology.orchestra.run.smithi143.stdout: "type": "int", 2023-11-26T07:23:10.088 INFO:teuthology.orchestra.run.smithi143.stdout: "level": "advanced", 2023-11-26T07:23:10.088 INFO:teuthology.orchestra.run.smithi143.stdout: "flags": 0, 2023-11-26T07:23:10.088 INFO:teuthology.orchestra.run.smithi143.stdout: "default_value": "10", 2023-11-26T07:23:10.089 INFO:teuthology.orchestra.run.smithi143.stdout: "min": "", 2023-11-26T07:23:10.089 INFO:teuthology.orchestra.run.smithi143.stdout: "max": "", 2023-11-26T07:23:10.089 INFO:teuthology.orchestra.run.smithi143.stdout: "enum_allowed": [], 2023-11-26T07:23:10.089 INFO:teuthology.orchestra.run.smithi143.stdout: "desc": "max number of osds that will be drained simultaneously when osds are removed", 2023-11-26T07:23:10.089 INFO:teuthology.orchestra.run.smithi143.stdout: "long_desc": "", 2023-11-26T07:23:10.089 INFO:teuthology.orchestra.run.smithi143.stdout: "tags": [], 2023-11-26T07:23:10.089 INFO:teuthology.orchestra.run.smithi143.stdout: "see_also": [] 2023-11-26T07:23:10.089 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2023-11-26T07:23:10.089 INFO:teuthology.orchestra.run.smithi143.stdout: "migration_current": { 2023-11-26T07:23:10.089 INFO:teuthology.orchestra.run.smithi143.stdout: "name": "migration_current", 2023-11-26T07:23:10.089 INFO:teuthology.orchestra.run.smithi143.stdout: "type": "int", 2023-11-26T07:23:10.089 INFO:teuthology.orchestra.run.smithi143.stdout: "level": "advanced", 2023-11-26T07:23:10.089 INFO:teuthology.orchestra.run.smithi143.stdout: "flags": 0, 2023-11-26T07:23:10.090 INFO:teuthology.orchestra.run.smithi143.stdout: "default_value": "", 2023-11-26T07:23:10.090 INFO:teuthology.orchestra.run.smithi143.stdout: "min": "", 2023-11-26T07:23:10.090 INFO:teuthology.orchestra.run.smithi143.stdout: "max": "", 2023-11-26T07:23:10.090 INFO:teuthology.orchestra.run.smithi143.stdout: "enum_allowed": [], 2023-11-26T07:23:10.090 INFO:teuthology.orchestra.run.smithi143.stdout: "desc": "internal - do not modify", 2023-11-26T07:23:10.090 INFO:teuthology.orchestra.run.smithi143.stdout: "long_desc": "", 2023-11-26T07:23:10.090 INFO:teuthology.orchestra.run.smithi143.stdout: "tags": [], 2023-11-26T07:23:10.090 INFO:teuthology.orchestra.run.smithi143.stdout: "see_also": [] 2023-11-26T07:23:10.090 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2023-11-26T07:23:10.090 INFO:teuthology.orchestra.run.smithi143.stdout: "mode": { 2023-11-26T07:23:10.090 INFO:teuthology.orchestra.run.smithi143.stdout: "name": "mode", 2023-11-26T07:23:10.090 INFO:teuthology.orchestra.run.smithi143.stdout: "type": "str", 2023-11-26T07:23:10.091 INFO:teuthology.orchestra.run.smithi143.stdout: "level": "advanced", 2023-11-26T07:23:10.091 INFO:teuthology.orchestra.run.smithi143.stdout: "flags": 0, 2023-11-26T07:23:10.091 INFO:teuthology.orchestra.run.smithi143.stdout: "default_value": "root", 2023-11-26T07:23:10.091 INFO:teuthology.orchestra.run.smithi143.stdout: "min": "", 2023-11-26T07:23:10.091 INFO:teuthology.orchestra.run.smithi143.stdout: "max": "", 2023-11-26T07:23:10.091 INFO:teuthology.orchestra.run.smithi143.stdout: "enum_allowed": [ 2023-11-26T07:23:10.091 INFO:teuthology.orchestra.run.smithi143.stdout: "cephadm-package", 2023-11-26T07:23:10.091 INFO:teuthology.orchestra.run.smithi143.stdout: "root" 2023-11-26T07:23:10.091 INFO:teuthology.orchestra.run.smithi143.stdout: ], 2023-11-26T07:23:10.091 INFO:teuthology.orchestra.run.smithi143.stdout: "desc": "mode for remote execution of cephadm", 2023-11-26T07:23:10.091 INFO:teuthology.orchestra.run.smithi143.stdout: "long_desc": "", 2023-11-26T07:23:10.091 INFO:teuthology.orchestra.run.smithi143.stdout: "tags": [], 2023-11-26T07:23:10.092 INFO:teuthology.orchestra.run.smithi143.stdout: "see_also": [] 2023-11-26T07:23:10.092 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2023-11-26T07:23:10.092 INFO:teuthology.orchestra.run.smithi143.stdout: "prometheus_alerts_path": { 2023-11-26T07:23:10.092 INFO:teuthology.orchestra.run.smithi143.stdout: "name": "prometheus_alerts_path", 2023-11-26T07:23:10.092 INFO:teuthology.orchestra.run.smithi143.stdout: "type": "str", 2023-11-26T07:23:10.092 INFO:teuthology.orchestra.run.smithi143.stdout: "level": "advanced", 2023-11-26T07:23:10.092 INFO:teuthology.orchestra.run.smithi143.stdout: "flags": 0, 2023-11-26T07:23:10.092 INFO:teuthology.orchestra.run.smithi143.stdout: "default_value": "/etc/prometheus/ceph/ceph_default_alerts.yml", 2023-11-26T07:23:10.092 INFO:teuthology.orchestra.run.smithi143.stdout: "min": "", 2023-11-26T07:23:10.092 INFO:teuthology.orchestra.run.smithi143.stdout: "max": "", 2023-11-26T07:23:10.092 INFO:teuthology.orchestra.run.smithi143.stdout: "enum_allowed": [], 2023-11-26T07:23:10.092 INFO:teuthology.orchestra.run.smithi143.stdout: "desc": "location of alerts to include in prometheus deployments", 2023-11-26T07:23:10.092 INFO:teuthology.orchestra.run.smithi143.stdout: "long_desc": "", 2023-11-26T07:23:10.093 INFO:teuthology.orchestra.run.smithi143.stdout: "tags": [], 2023-11-26T07:23:10.093 INFO:teuthology.orchestra.run.smithi143.stdout: "see_also": [] 2023-11-26T07:23:10.093 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2023-11-26T07:23:10.093 INFO:teuthology.orchestra.run.smithi143.stdout: "registry_insecure": { 2023-11-26T07:23:10.093 INFO:teuthology.orchestra.run.smithi143.stdout: "name": "registry_insecure", 2023-11-26T07:23:10.093 INFO:teuthology.orchestra.run.smithi143.stdout: "type": "bool", 2023-11-26T07:23:10.093 INFO:teuthology.orchestra.run.smithi143.stdout: "level": "advanced", 2023-11-26T07:23:10.093 INFO:teuthology.orchestra.run.smithi143.stdout: "flags": 0, 2023-11-26T07:23:10.093 INFO:teuthology.orchestra.run.smithi143.stdout: "default_value": "False", 2023-11-26T07:23:10.093 INFO:teuthology.orchestra.run.smithi143.stdout: "min": "", 2023-11-26T07:23:10.093 INFO:teuthology.orchestra.run.smithi143.stdout: "max": "", 2023-11-26T07:23:10.093 INFO:teuthology.orchestra.run.smithi143.stdout: "enum_allowed": [], 2023-11-26T07:23:10.093 INFO:teuthology.orchestra.run.smithi143.stdout: "desc": "Registry is to be considered insecure (no TLS available). Only for development purposes.", 2023-11-26T07:23:10.094 INFO:teuthology.orchestra.run.smithi143.stdout: "long_desc": "", 2023-11-26T07:23:10.094 INFO:teuthology.orchestra.run.smithi143.stdout: "tags": [], 2023-11-26T07:23:10.094 INFO:teuthology.orchestra.run.smithi143.stdout: "see_also": [] 2023-11-26T07:23:10.094 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2023-11-26T07:23:10.094 INFO:teuthology.orchestra.run.smithi143.stdout: "registry_password": { 2023-11-26T07:23:10.094 INFO:teuthology.orchestra.run.smithi143.stdout: "name": "registry_password", 2023-11-26T07:23:10.094 INFO:teuthology.orchestra.run.smithi143.stdout: "type": "str", 2023-11-26T07:23:10.094 INFO:teuthology.orchestra.run.smithi143.stdout: "level": "advanced", 2023-11-26T07:23:10.094 INFO:teuthology.orchestra.run.smithi143.stdout: "flags": 0, 2023-11-26T07:23:10.094 INFO:teuthology.orchestra.run.smithi143.stdout: "default_value": "", 2023-11-26T07:23:10.094 INFO:teuthology.orchestra.run.smithi143.stdout: "min": "", 2023-11-26T07:23:10.094 INFO:teuthology.orchestra.run.smithi143.stdout: "max": "", 2023-11-26T07:23:10.094 INFO:teuthology.orchestra.run.smithi143.stdout: "enum_allowed": [], 2023-11-26T07:23:10.095 INFO:teuthology.orchestra.run.smithi143.stdout: "desc": "Custom repository password. Only used for logging into a registry.", 2023-11-26T07:23:10.095 INFO:teuthology.orchestra.run.smithi143.stdout: "long_desc": "", 2023-11-26T07:23:10.095 INFO:teuthology.orchestra.run.smithi143.stdout: "tags": [], 2023-11-26T07:23:10.095 INFO:teuthology.orchestra.run.smithi143.stdout: "see_also": [] 2023-11-26T07:23:10.095 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2023-11-26T07:23:10.095 INFO:teuthology.orchestra.run.smithi143.stdout: "registry_url": { 2023-11-26T07:23:10.095 INFO:teuthology.orchestra.run.smithi143.stdout: "name": "registry_url", 2023-11-26T07:23:10.095 INFO:teuthology.orchestra.run.smithi143.stdout: "type": "str", 2023-11-26T07:23:10.095 INFO:teuthology.orchestra.run.smithi143.stdout: "level": "advanced", 2023-11-26T07:23:10.095 INFO:teuthology.orchestra.run.smithi143.stdout: "flags": 0, 2023-11-26T07:23:10.095 INFO:teuthology.orchestra.run.smithi143.stdout: "default_value": "", 2023-11-26T07:23:10.095 INFO:teuthology.orchestra.run.smithi143.stdout: "min": "", 2023-11-26T07:23:10.095 INFO:teuthology.orchestra.run.smithi143.stdout: "max": "", 2023-11-26T07:23:10.096 INFO:teuthology.orchestra.run.smithi143.stdout: "enum_allowed": [], 2023-11-26T07:23:10.096 INFO:teuthology.orchestra.run.smithi143.stdout: "desc": "Registry url for login purposes. This is not the default registry", 2023-11-26T07:23:10.096 INFO:teuthology.orchestra.run.smithi143.stdout: "long_desc": "", 2023-11-26T07:23:10.096 INFO:teuthology.orchestra.run.smithi143.stdout: "tags": [], 2023-11-26T07:23:10.096 INFO:teuthology.orchestra.run.smithi143.stdout: "see_also": [] 2023-11-26T07:23:10.096 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2023-11-26T07:23:10.096 INFO:teuthology.orchestra.run.smithi143.stdout: "registry_username": { 2023-11-26T07:23:10.096 INFO:teuthology.orchestra.run.smithi143.stdout: "name": "registry_username", 2023-11-26T07:23:10.096 INFO:teuthology.orchestra.run.smithi143.stdout: "type": "str", 2023-11-26T07:23:10.096 INFO:teuthology.orchestra.run.smithi143.stdout: "level": "advanced", 2023-11-26T07:23:10.096 INFO:teuthology.orchestra.run.smithi143.stdout: "flags": 0, 2023-11-26T07:23:10.096 INFO:teuthology.orchestra.run.smithi143.stdout: "default_value": "", 2023-11-26T07:23:10.097 INFO:teuthology.orchestra.run.smithi143.stdout: "min": "", 2023-11-26T07:23:10.097 INFO:teuthology.orchestra.run.smithi143.stdout: "max": "", 2023-11-26T07:23:10.097 INFO:teuthology.orchestra.run.smithi143.stdout: "enum_allowed": [], 2023-11-26T07:23:10.097 INFO:teuthology.orchestra.run.smithi143.stdout: "desc": "Custom repository username. Only used for logging into a registry.", 2023-11-26T07:23:10.097 INFO:teuthology.orchestra.run.smithi143.stdout: "long_desc": "", 2023-11-26T07:23:10.097 INFO:teuthology.orchestra.run.smithi143.stdout: "tags": [], 2023-11-26T07:23:10.097 INFO:teuthology.orchestra.run.smithi143.stdout: "see_also": [] 2023-11-26T07:23:10.097 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2023-11-26T07:23:10.097 INFO:teuthology.orchestra.run.smithi143.stdout: "secure_monitoring_stack": { 2023-11-26T07:23:10.097 INFO:teuthology.orchestra.run.smithi143.stdout: "name": "secure_monitoring_stack", 2023-11-26T07:23:10.097 INFO:teuthology.orchestra.run.smithi143.stdout: "type": "bool", 2023-11-26T07:23:10.097 INFO:teuthology.orchestra.run.smithi143.stdout: "level": "advanced", 2023-11-26T07:23:10.097 INFO:teuthology.orchestra.run.smithi143.stdout: "flags": 0, 2023-11-26T07:23:10.098 INFO:teuthology.orchestra.run.smithi143.stdout: "default_value": "False", 2023-11-26T07:23:10.098 INFO:teuthology.orchestra.run.smithi143.stdout: "min": "", 2023-11-26T07:23:10.098 INFO:teuthology.orchestra.run.smithi143.stdout: "max": "", 2023-11-26T07:23:10.098 INFO:teuthology.orchestra.run.smithi143.stdout: "enum_allowed": [], 2023-11-26T07:23:10.098 INFO:teuthology.orchestra.run.smithi143.stdout: "desc": "Enable TLS security for all the monitoring stack daemons", 2023-11-26T07:23:10.098 INFO:teuthology.orchestra.run.smithi143.stdout: "long_desc": "", 2023-11-26T07:23:10.098 INFO:teuthology.orchestra.run.smithi143.stdout: "tags": [], 2023-11-26T07:23:10.098 INFO:teuthology.orchestra.run.smithi143.stdout: "see_also": [] 2023-11-26T07:23:10.098 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2023-11-26T07:23:10.098 INFO:teuthology.orchestra.run.smithi143.stdout: "service_discovery_port": { 2023-11-26T07:23:10.098 INFO:teuthology.orchestra.run.smithi143.stdout: "name": "service_discovery_port", 2023-11-26T07:23:10.098 INFO:teuthology.orchestra.run.smithi143.stdout: "type": "int", 2023-11-26T07:23:10.099 INFO:teuthology.orchestra.run.smithi143.stdout: "level": "advanced", 2023-11-26T07:23:10.099 INFO:teuthology.orchestra.run.smithi143.stdout: "flags": 0, 2023-11-26T07:23:10.099 INFO:teuthology.orchestra.run.smithi143.stdout: "default_value": "8765", 2023-11-26T07:23:10.099 INFO:teuthology.orchestra.run.smithi143.stdout: "min": "", 2023-11-26T07:23:10.099 INFO:teuthology.orchestra.run.smithi143.stdout: "max": "", 2023-11-26T07:23:10.099 INFO:teuthology.orchestra.run.smithi143.stdout: "enum_allowed": [], 2023-11-26T07:23:10.099 INFO:teuthology.orchestra.run.smithi143.stdout: "desc": "cephadm service discovery port", 2023-11-26T07:23:10.099 INFO:teuthology.orchestra.run.smithi143.stdout: "long_desc": "", 2023-11-26T07:23:10.099 INFO:teuthology.orchestra.run.smithi143.stdout: "tags": [], 2023-11-26T07:23:10.099 INFO:teuthology.orchestra.run.smithi143.stdout: "see_also": [] 2023-11-26T07:23:10.099 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2023-11-26T07:23:10.099 INFO:teuthology.orchestra.run.smithi143.stdout: "ssh_config_file": { 2023-11-26T07:23:10.099 INFO:teuthology.orchestra.run.smithi143.stdout: "name": "ssh_config_file", 2023-11-26T07:23:10.100 INFO:teuthology.orchestra.run.smithi143.stdout: "type": "str", 2023-11-26T07:23:10.100 INFO:teuthology.orchestra.run.smithi143.stdout: "level": "advanced", 2023-11-26T07:23:10.100 INFO:teuthology.orchestra.run.smithi143.stdout: "flags": 0, 2023-11-26T07:23:10.100 INFO:teuthology.orchestra.run.smithi143.stdout: "default_value": "", 2023-11-26T07:23:10.100 INFO:teuthology.orchestra.run.smithi143.stdout: "min": "", 2023-11-26T07:23:10.100 INFO:teuthology.orchestra.run.smithi143.stdout: "max": "", 2023-11-26T07:23:10.100 INFO:teuthology.orchestra.run.smithi143.stdout: "enum_allowed": [], 2023-11-26T07:23:10.100 INFO:teuthology.orchestra.run.smithi143.stdout: "desc": "customized SSH config file to connect to managed hosts", 2023-11-26T07:23:10.100 INFO:teuthology.orchestra.run.smithi143.stdout: "long_desc": "", 2023-11-26T07:23:10.100 INFO:teuthology.orchestra.run.smithi143.stdout: "tags": [], 2023-11-26T07:23:10.100 INFO:teuthology.orchestra.run.smithi143.stdout: "see_also": [] 2023-11-26T07:23:10.100 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2023-11-26T07:23:10.100 INFO:teuthology.orchestra.run.smithi143.stdout: "use_agent": { 2023-11-26T07:23:10.101 INFO:teuthology.orchestra.run.smithi143.stdout: "name": "use_agent", 2023-11-26T07:23:10.101 INFO:teuthology.orchestra.run.smithi143.stdout: "type": "bool", 2023-11-26T07:23:10.101 INFO:teuthology.orchestra.run.smithi143.stdout: "level": "advanced", 2023-11-26T07:23:10.101 INFO:teuthology.orchestra.run.smithi143.stdout: "flags": 0, 2023-11-26T07:23:10.101 INFO:teuthology.orchestra.run.smithi143.stdout: "default_value": "False", 2023-11-26T07:23:10.101 INFO:teuthology.orchestra.run.smithi143.stdout: "min": "", 2023-11-26T07:23:10.101 INFO:teuthology.orchestra.run.smithi143.stdout: "max": "", 2023-11-26T07:23:10.101 INFO:teuthology.orchestra.run.smithi143.stdout: "enum_allowed": [], 2023-11-26T07:23:10.101 INFO:teuthology.orchestra.run.smithi143.stdout: "desc": "Use cephadm agent on each host to gather and send metadata", 2023-11-26T07:23:10.101 INFO:teuthology.orchestra.run.smithi143.stdout: "long_desc": "", 2023-11-26T07:23:10.101 INFO:teuthology.orchestra.run.smithi143.stdout: "tags": [], 2023-11-26T07:23:10.101 INFO:teuthology.orchestra.run.smithi143.stdout: "see_also": [] 2023-11-26T07:23:10.101 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2023-11-26T07:23:10.102 INFO:teuthology.orchestra.run.smithi143.stdout: "use_repo_digest": { 2023-11-26T07:23:10.102 INFO:teuthology.orchestra.run.smithi143.stdout: "name": "use_repo_digest", 2023-11-26T07:23:10.102 INFO:teuthology.orchestra.run.smithi143.stdout: "type": "bool", 2023-11-26T07:23:10.102 INFO:teuthology.orchestra.run.smithi143.stdout: "level": "advanced", 2023-11-26T07:23:10.102 INFO:teuthology.orchestra.run.smithi143.stdout: "flags": 0, 2023-11-26T07:23:10.102 INFO:teuthology.orchestra.run.smithi143.stdout: "default_value": "True", 2023-11-26T07:23:10.102 INFO:teuthology.orchestra.run.smithi143.stdout: "min": "", 2023-11-26T07:23:10.102 INFO:teuthology.orchestra.run.smithi143.stdout: "max": "", 2023-11-26T07:23:10.102 INFO:teuthology.orchestra.run.smithi143.stdout: "enum_allowed": [], 2023-11-26T07:23:10.102 INFO:teuthology.orchestra.run.smithi143.stdout: "desc": "Automatically convert image tags to image digest. Make sure all daemons use the same image", 2023-11-26T07:23:10.102 INFO:teuthology.orchestra.run.smithi143.stdout: "long_desc": "", 2023-11-26T07:23:10.102 INFO:teuthology.orchestra.run.smithi143.stdout: "tags": [], 2023-11-26T07:23:10.102 INFO:teuthology.orchestra.run.smithi143.stdout: "see_also": [] 2023-11-26T07:23:10.103 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2023-11-26T07:23:10.103 INFO:teuthology.orchestra.run.smithi143.stdout: "warn_on_failed_host_check": { 2023-11-26T07:23:10.103 INFO:teuthology.orchestra.run.smithi143.stdout: "name": "warn_on_failed_host_check", 2023-11-26T07:23:10.103 INFO:teuthology.orchestra.run.smithi143.stdout: "type": "bool", 2023-11-26T07:23:10.103 INFO:teuthology.orchestra.run.smithi143.stdout: "level": "advanced", 2023-11-26T07:23:10.103 INFO:teuthology.orchestra.run.smithi143.stdout: "flags": 0, 2023-11-26T07:23:10.103 INFO:teuthology.orchestra.run.smithi143.stdout: "default_value": "True", 2023-11-26T07:23:10.103 INFO:teuthology.orchestra.run.smithi143.stdout: "min": "", 2023-11-26T07:23:10.103 INFO:teuthology.orchestra.run.smithi143.stdout: "max": "", 2023-11-26T07:23:10.103 INFO:teuthology.orchestra.run.smithi143.stdout: "enum_allowed": [], 2023-11-26T07:23:10.103 INFO:teuthology.orchestra.run.smithi143.stdout: "desc": "raise a health warning if the host check fails", 2023-11-26T07:23:10.103 INFO:teuthology.orchestra.run.smithi143.stdout: "long_desc": "", 2023-11-26T07:23:10.103 INFO:teuthology.orchestra.run.smithi143.stdout: "tags": [], 2023-11-26T07:23:10.104 INFO:teuthology.orchestra.run.smithi143.stdout: "see_also": [] 2023-11-26T07:23:10.104 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2023-11-26T07:23:10.104 INFO:teuthology.orchestra.run.smithi143.stdout: "warn_on_stray_daemons": { 2023-11-26T07:23:10.104 INFO:teuthology.orchestra.run.smithi143.stdout: "name": "warn_on_stray_daemons", 2023-11-26T07:23:10.104 INFO:teuthology.orchestra.run.smithi143.stdout: "type": "bool", 2023-11-26T07:23:10.104 INFO:teuthology.orchestra.run.smithi143.stdout: "level": "advanced", 2023-11-26T07:23:10.104 INFO:teuthology.orchestra.run.smithi143.stdout: "flags": 0, 2023-11-26T07:23:10.104 INFO:teuthology.orchestra.run.smithi143.stdout: "default_value": "True", 2023-11-26T07:23:10.104 INFO:teuthology.orchestra.run.smithi143.stdout: "min": "", 2023-11-26T07:23:10.104 INFO:teuthology.orchestra.run.smithi143.stdout: "max": "", 2023-11-26T07:23:10.104 INFO:teuthology.orchestra.run.smithi143.stdout: "enum_allowed": [], 2023-11-26T07:23:10.104 INFO:teuthology.orchestra.run.smithi143.stdout: "desc": "raise a health warning if daemons are detected that are not managed by cephadm", 2023-11-26T07:23:10.104 INFO:teuthology.orchestra.run.smithi143.stdout: "long_desc": "", 2023-11-26T07:23:10.105 INFO:teuthology.orchestra.run.smithi143.stdout: "tags": [], 2023-11-26T07:23:10.105 INFO:teuthology.orchestra.run.smithi143.stdout: "see_also": [] 2023-11-26T07:23:10.105 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2023-11-26T07:23:10.105 INFO:teuthology.orchestra.run.smithi143.stdout: "warn_on_stray_hosts": { 2023-11-26T07:23:10.105 INFO:teuthology.orchestra.run.smithi143.stdout: "name": "warn_on_stray_hosts", 2023-11-26T07:23:10.105 INFO:teuthology.orchestra.run.smithi143.stdout: "type": "bool", 2023-11-26T07:23:10.105 INFO:teuthology.orchestra.run.smithi143.stdout: "level": "advanced", 2023-11-26T07:23:10.105 INFO:teuthology.orchestra.run.smithi143.stdout: "flags": 0, 2023-11-26T07:23:10.105 INFO:teuthology.orchestra.run.smithi143.stdout: "default_value": "True", 2023-11-26T07:23:10.105 INFO:teuthology.orchestra.run.smithi143.stdout: "min": "", 2023-11-26T07:23:10.105 INFO:teuthology.orchestra.run.smithi143.stdout: "max": "", 2023-11-26T07:23:10.105 INFO:teuthology.orchestra.run.smithi143.stdout: "enum_allowed": [], 2023-11-26T07:23:10.105 INFO:teuthology.orchestra.run.smithi143.stdout: "desc": "raise a health warning if daemons are detected on a host that is not managed by cephadm", 2023-11-26T07:23:10.106 INFO:teuthology.orchestra.run.smithi143.stdout: "long_desc": "", 2023-11-26T07:23:10.106 INFO:teuthology.orchestra.run.smithi143.stdout: "tags": [], 2023-11-26T07:23:10.106 INFO:teuthology.orchestra.run.smithi143.stdout: "see_also": [] 2023-11-26T07:23:10.106 INFO:teuthology.orchestra.run.smithi143.stdout: } 2023-11-26T07:23:10.106 INFO:teuthology.orchestra.run.smithi143.stdout: } 2023-11-26T07:23:10.106 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2023-11-26T07:23:10.106 INFO:teuthology.orchestra.run.smithi143.stdout: { 2023-11-26T07:23:10.106 INFO:teuthology.orchestra.run.smithi143.stdout: "name": "crash", 2023-11-26T07:23:10.106 INFO:teuthology.orchestra.run.smithi143.stdout: "can_run": true, 2023-11-26T07:23:10.106 INFO:teuthology.orchestra.run.smithi143.stdout: "error_string": "", 2023-11-26T07:23:10.106 INFO:teuthology.orchestra.run.smithi143.stdout: "module_options": { 2023-11-26T07:23:10.106 INFO:teuthology.orchestra.run.smithi143.stdout: "log_level": { 2023-11-26T07:23:10.107 INFO:teuthology.orchestra.run.smithi143.stdout: "name": "log_level", 2023-11-26T07:23:10.107 INFO:teuthology.orchestra.run.smithi143.stdout: "type": "str", 2023-11-26T07:23:10.107 INFO:teuthology.orchestra.run.smithi143.stdout: "level": "advanced", 2023-11-26T07:23:10.107 INFO:teuthology.orchestra.run.smithi143.stdout: "flags": 1, 2023-11-26T07:23:10.107 INFO:teuthology.orchestra.run.smithi143.stdout: "default_value": "", 2023-11-26T07:23:10.107 INFO:teuthology.orchestra.run.smithi143.stdout: "min": "", 2023-11-26T07:23:10.107 INFO:teuthology.orchestra.run.smithi143.stdout: "max": "", 2023-11-26T07:23:10.107 INFO:teuthology.orchestra.run.smithi143.stdout: "enum_allowed": [ 2023-11-26T07:23:10.107 INFO:teuthology.orchestra.run.smithi143.stdout: "", 2023-11-26T07:23:10.107 INFO:teuthology.orchestra.run.smithi143.stdout: "critical", 2023-11-26T07:23:10.107 INFO:teuthology.orchestra.run.smithi143.stdout: "debug", 2023-11-26T07:23:10.107 INFO:teuthology.orchestra.run.smithi143.stdout: "error", 2023-11-26T07:23:10.107 INFO:teuthology.orchestra.run.smithi143.stdout: "info", 2023-11-26T07:23:10.108 INFO:teuthology.orchestra.run.smithi143.stdout: "warning" 2023-11-26T07:23:10.108 INFO:teuthology.orchestra.run.smithi143.stdout: ], 2023-11-26T07:23:10.108 INFO:teuthology.orchestra.run.smithi143.stdout: "desc": "", 2023-11-26T07:23:10.108 INFO:teuthology.orchestra.run.smithi143.stdout: "long_desc": "", 2023-11-26T07:23:10.108 INFO:teuthology.orchestra.run.smithi143.stdout: "tags": [], 2023-11-26T07:23:10.108 INFO:teuthology.orchestra.run.smithi143.stdout: "see_also": [] 2023-11-26T07:23:10.108 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2023-11-26T07:23:10.108 INFO:teuthology.orchestra.run.smithi143.stdout: "log_to_cluster": { 2023-11-26T07:23:10.108 INFO:teuthology.orchestra.run.smithi143.stdout: "name": "log_to_cluster", 2023-11-26T07:23:10.108 INFO:teuthology.orchestra.run.smithi143.stdout: "type": "bool", 2023-11-26T07:23:10.108 INFO:teuthology.orchestra.run.smithi143.stdout: "level": "advanced", 2023-11-26T07:23:10.108 INFO:teuthology.orchestra.run.smithi143.stdout: "flags": 1, 2023-11-26T07:23:10.108 INFO:teuthology.orchestra.run.smithi143.stdout: "default_value": "False", 2023-11-26T07:23:10.109 INFO:teuthology.orchestra.run.smithi143.stdout: "min": "", 2023-11-26T07:23:10.109 INFO:teuthology.orchestra.run.smithi143.stdout: "max": "", 2023-11-26T07:23:10.109 INFO:teuthology.orchestra.run.smithi143.stdout: "enum_allowed": [], 2023-11-26T07:23:10.109 INFO:teuthology.orchestra.run.smithi143.stdout: "desc": "", 2023-11-26T07:23:10.109 INFO:teuthology.orchestra.run.smithi143.stdout: "long_desc": "", 2023-11-26T07:23:10.109 INFO:teuthology.orchestra.run.smithi143.stdout: "tags": [], 2023-11-26T07:23:10.109 INFO:teuthology.orchestra.run.smithi143.stdout: "see_also": [] 2023-11-26T07:23:10.109 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2023-11-26T07:23:10.109 INFO:teuthology.orchestra.run.smithi143.stdout: "log_to_cluster_level": { 2023-11-26T07:23:10.109 INFO:teuthology.orchestra.run.smithi143.stdout: "name": "log_to_cluster_level", 2023-11-26T07:23:10.109 INFO:teuthology.orchestra.run.smithi143.stdout: "type": "str", 2023-11-26T07:23:10.109 INFO:teuthology.orchestra.run.smithi143.stdout: "level": "advanced", 2023-11-26T07:23:10.109 INFO:teuthology.orchestra.run.smithi143.stdout: "flags": 1, 2023-11-26T07:23:10.110 INFO:teuthology.orchestra.run.smithi143.stdout: "default_value": "info", 2023-11-26T07:23:10.110 INFO:teuthology.orchestra.run.smithi143.stdout: "min": "", 2023-11-26T07:23:10.110 INFO:teuthology.orchestra.run.smithi143.stdout: "max": "", 2023-11-26T07:23:10.110 INFO:teuthology.orchestra.run.smithi143.stdout: "enum_allowed": [ 2023-11-26T07:23:10.110 INFO:teuthology.orchestra.run.smithi143.stdout: "", 2023-11-26T07:23:10.110 INFO:teuthology.orchestra.run.smithi143.stdout: "critical", 2023-11-26T07:23:10.110 INFO:teuthology.orchestra.run.smithi143.stdout: "debug", 2023-11-26T07:23:10.110 INFO:teuthology.orchestra.run.smithi143.stdout: "error", 2023-11-26T07:23:10.110 INFO:teuthology.orchestra.run.smithi143.stdout: "info", 2023-11-26T07:23:10.110 INFO:teuthology.orchestra.run.smithi143.stdout: "warning" 2023-11-26T07:23:10.110 INFO:teuthology.orchestra.run.smithi143.stdout: ], 2023-11-26T07:23:10.110 INFO:teuthology.orchestra.run.smithi143.stdout: "desc": "", 2023-11-26T07:23:10.111 INFO:teuthology.orchestra.run.smithi143.stdout: "long_desc": "", 2023-11-26T07:23:10.111 INFO:teuthology.orchestra.run.smithi143.stdout: "tags": [], 2023-11-26T07:23:10.111 INFO:teuthology.orchestra.run.smithi143.stdout: "see_also": [] 2023-11-26T07:23:10.111 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2023-11-26T07:23:10.111 INFO:teuthology.orchestra.run.smithi143.stdout: "log_to_file": { 2023-11-26T07:23:10.111 INFO:teuthology.orchestra.run.smithi143.stdout: "name": "log_to_file", 2023-11-26T07:23:10.111 INFO:teuthology.orchestra.run.smithi143.stdout: "type": "bool", 2023-11-26T07:23:10.111 INFO:teuthology.orchestra.run.smithi143.stdout: "level": "advanced", 2023-11-26T07:23:10.111 INFO:teuthology.orchestra.run.smithi143.stdout: "flags": 1, 2023-11-26T07:23:10.111 INFO:teuthology.orchestra.run.smithi143.stdout: "default_value": "False", 2023-11-26T07:23:10.111 INFO:teuthology.orchestra.run.smithi143.stdout: "min": "", 2023-11-26T07:23:10.111 INFO:teuthology.orchestra.run.smithi143.stdout: "max": "", 2023-11-26T07:23:10.111 INFO:teuthology.orchestra.run.smithi143.stdout: "enum_allowed": [], 2023-11-26T07:23:10.112 INFO:teuthology.orchestra.run.smithi143.stdout: "desc": "", 2023-11-26T07:23:10.112 INFO:teuthology.orchestra.run.smithi143.stdout: "long_desc": "", 2023-11-26T07:23:10.112 INFO:teuthology.orchestra.run.smithi143.stdout: "tags": [], 2023-11-26T07:23:10.112 INFO:teuthology.orchestra.run.smithi143.stdout: "see_also": [] 2023-11-26T07:23:10.112 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2023-11-26T07:23:10.112 INFO:teuthology.orchestra.run.smithi143.stdout: "retain_interval": { 2023-11-26T07:23:10.112 INFO:teuthology.orchestra.run.smithi143.stdout: "name": "retain_interval", 2023-11-26T07:23:10.112 INFO:teuthology.orchestra.run.smithi143.stdout: "type": "secs", 2023-11-26T07:23:10.112 INFO:teuthology.orchestra.run.smithi143.stdout: "level": "advanced", 2023-11-26T07:23:10.112 INFO:teuthology.orchestra.run.smithi143.stdout: "flags": 1, 2023-11-26T07:23:10.112 INFO:teuthology.orchestra.run.smithi143.stdout: "default_value": "31536000", 2023-11-26T07:23:10.112 INFO:teuthology.orchestra.run.smithi143.stdout: "min": "", 2023-11-26T07:23:10.112 INFO:teuthology.orchestra.run.smithi143.stdout: "max": "", 2023-11-26T07:23:10.113 INFO:teuthology.orchestra.run.smithi143.stdout: "enum_allowed": [], 2023-11-26T07:23:10.113 INFO:teuthology.orchestra.run.smithi143.stdout: "desc": "how long to retain crashes before pruning them", 2023-11-26T07:23:10.113 INFO:teuthology.orchestra.run.smithi143.stdout: "long_desc": "", 2023-11-26T07:23:10.113 INFO:teuthology.orchestra.run.smithi143.stdout: "tags": [], 2023-11-26T07:23:10.113 INFO:teuthology.orchestra.run.smithi143.stdout: "see_also": [] 2023-11-26T07:23:10.113 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2023-11-26T07:23:10.113 INFO:teuthology.orchestra.run.smithi143.stdout: "warn_recent_interval": { 2023-11-26T07:23:10.113 INFO:teuthology.orchestra.run.smithi143.stdout: "name": "warn_recent_interval", 2023-11-26T07:23:10.113 INFO:teuthology.orchestra.run.smithi143.stdout: "type": "secs", 2023-11-26T07:23:10.113 INFO:teuthology.orchestra.run.smithi143.stdout: "level": "advanced", 2023-11-26T07:23:10.113 INFO:teuthology.orchestra.run.smithi143.stdout: "flags": 1, 2023-11-26T07:23:10.113 INFO:teuthology.orchestra.run.smithi143.stdout: "default_value": "1209600", 2023-11-26T07:23:10.113 INFO:teuthology.orchestra.run.smithi143.stdout: "min": "", 2023-11-26T07:23:10.114 INFO:teuthology.orchestra.run.smithi143.stdout: "max": "", 2023-11-26T07:23:10.114 INFO:teuthology.orchestra.run.smithi143.stdout: "enum_allowed": [], 2023-11-26T07:23:10.114 INFO:teuthology.orchestra.run.smithi143.stdout: "desc": "time interval in which to warn about recent crashes", 2023-11-26T07:23:10.114 INFO:teuthology.orchestra.run.smithi143.stdout: "long_desc": "", 2023-11-26T07:23:10.114 INFO:teuthology.orchestra.run.smithi143.stdout: "tags": [], 2023-11-26T07:23:10.114 INFO:teuthology.orchestra.run.smithi143.stdout: "see_also": [] 2023-11-26T07:23:10.114 INFO:teuthology.orchestra.run.smithi143.stdout: } 2023-11-26T07:23:10.114 INFO:teuthology.orchestra.run.smithi143.stdout: } 2023-11-26T07:23:10.114 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2023-11-26T07:23:10.114 INFO:teuthology.orchestra.run.smithi143.stdout: { 2023-11-26T07:23:10.114 INFO:teuthology.orchestra.run.smithi143.stdout: "name": "dashboard", 2023-11-26T07:23:10.114 INFO:teuthology.orchestra.run.smithi143.stdout: "can_run": true, 2023-11-26T07:23:10.114 INFO:teuthology.orchestra.run.smithi143.stdout: "error_string": "", 2023-11-26T07:23:10.115 INFO:teuthology.orchestra.run.smithi143.stdout: "module_options": { 2023-11-26T07:23:10.115 INFO:teuthology.orchestra.run.smithi143.stdout: "ACCOUNT_LOCKOUT_ATTEMPTS": { 2023-11-26T07:23:10.115 INFO:teuthology.orchestra.run.smithi143.stdout: "name": "ACCOUNT_LOCKOUT_ATTEMPTS", 2023-11-26T07:23:10.115 INFO:teuthology.orchestra.run.smithi143.stdout: "type": "int", 2023-11-26T07:23:10.115 INFO:teuthology.orchestra.run.smithi143.stdout: "level": "advanced", 2023-11-26T07:23:10.115 INFO:teuthology.orchestra.run.smithi143.stdout: "flags": 0, 2023-11-26T07:23:10.115 INFO:teuthology.orchestra.run.smithi143.stdout: "default_value": "10", 2023-11-26T07:23:10.115 INFO:teuthology.orchestra.run.smithi143.stdout: "min": "", 2023-11-26T07:23:10.115 INFO:teuthology.orchestra.run.smithi143.stdout: "max": "", 2023-11-26T07:23:10.115 INFO:teuthology.orchestra.run.smithi143.stdout: "enum_allowed": [], 2023-11-26T07:23:10.115 INFO:teuthology.orchestra.run.smithi143.stdout: "desc": "", 2023-11-26T07:23:10.115 INFO:teuthology.orchestra.run.smithi143.stdout: "long_desc": "", 2023-11-26T07:23:10.115 INFO:teuthology.orchestra.run.smithi143.stdout: "tags": [], 2023-11-26T07:23:10.116 INFO:teuthology.orchestra.run.smithi143.stdout: "see_also": [] 2023-11-26T07:23:10.116 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2023-11-26T07:23:10.116 INFO:teuthology.orchestra.run.smithi143.stdout: "ALERTMANAGER_API_HOST": { 2023-11-26T07:23:10.116 INFO:teuthology.orchestra.run.smithi143.stdout: "name": "ALERTMANAGER_API_HOST", 2023-11-26T07:23:10.116 INFO:teuthology.orchestra.run.smithi143.stdout: "type": "str", 2023-11-26T07:23:10.116 INFO:teuthology.orchestra.run.smithi143.stdout: "level": "advanced", 2023-11-26T07:23:10.116 INFO:teuthology.orchestra.run.smithi143.stdout: "flags": 0, 2023-11-26T07:23:10.116 INFO:teuthology.orchestra.run.smithi143.stdout: "default_value": "", 2023-11-26T07:23:10.116 INFO:teuthology.orchestra.run.smithi143.stdout: "min": "", 2023-11-26T07:23:10.116 INFO:teuthology.orchestra.run.smithi143.stdout: "max": "", 2023-11-26T07:23:10.116 INFO:teuthology.orchestra.run.smithi143.stdout: "enum_allowed": [], 2023-11-26T07:23:10.116 INFO:teuthology.orchestra.run.smithi143.stdout: "desc": "", 2023-11-26T07:23:10.116 INFO:teuthology.orchestra.run.smithi143.stdout: "long_desc": "", 2023-11-26T07:23:10.117 INFO:teuthology.orchestra.run.smithi143.stdout: "tags": [], 2023-11-26T07:23:10.117 INFO:teuthology.orchestra.run.smithi143.stdout: "see_also": [] 2023-11-26T07:23:10.117 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2023-11-26T07:23:10.117 INFO:teuthology.orchestra.run.smithi143.stdout: "ALERTMANAGER_API_SSL_VERIFY": { 2023-11-26T07:23:10.117 INFO:teuthology.orchestra.run.smithi143.stdout: "name": "ALERTMANAGER_API_SSL_VERIFY", 2023-11-26T07:23:10.117 INFO:teuthology.orchestra.run.smithi143.stdout: "type": "bool", 2023-11-26T07:23:10.117 INFO:teuthology.orchestra.run.smithi143.stdout: "level": "advanced", 2023-11-26T07:23:10.117 INFO:teuthology.orchestra.run.smithi143.stdout: "flags": 0, 2023-11-26T07:23:10.117 INFO:teuthology.orchestra.run.smithi143.stdout: "default_value": "True", 2023-11-26T07:23:10.117 INFO:teuthology.orchestra.run.smithi143.stdout: "min": "", 2023-11-26T07:23:10.117 INFO:teuthology.orchestra.run.smithi143.stdout: "max": "", 2023-11-26T07:23:10.117 INFO:teuthology.orchestra.run.smithi143.stdout: "enum_allowed": [], 2023-11-26T07:23:10.117 INFO:teuthology.orchestra.run.smithi143.stdout: "desc": "", 2023-11-26T07:23:10.118 INFO:teuthology.orchestra.run.smithi143.stdout: "long_desc": "", 2023-11-26T07:23:10.118 INFO:teuthology.orchestra.run.smithi143.stdout: "tags": [], 2023-11-26T07:23:10.118 INFO:teuthology.orchestra.run.smithi143.stdout: "see_also": [] 2023-11-26T07:23:10.118 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2023-11-26T07:23:10.118 INFO:teuthology.orchestra.run.smithi143.stdout: "AUDIT_API_ENABLED": { 2023-11-26T07:23:10.118 INFO:teuthology.orchestra.run.smithi143.stdout: "name": "AUDIT_API_ENABLED", 2023-11-26T07:23:10.118 INFO:teuthology.orchestra.run.smithi143.stdout: "type": "bool", 2023-11-26T07:23:10.118 INFO:teuthology.orchestra.run.smithi143.stdout: "level": "advanced", 2023-11-26T07:23:10.118 INFO:teuthology.orchestra.run.smithi143.stdout: "flags": 0, 2023-11-26T07:23:10.118 INFO:teuthology.orchestra.run.smithi143.stdout: "default_value": "False", 2023-11-26T07:23:10.118 INFO:teuthology.orchestra.run.smithi143.stdout: "min": "", 2023-11-26T07:23:10.118 INFO:teuthology.orchestra.run.smithi143.stdout: "max": "", 2023-11-26T07:23:10.119 INFO:teuthology.orchestra.run.smithi143.stdout: "enum_allowed": [], 2023-11-26T07:23:10.119 INFO:teuthology.orchestra.run.smithi143.stdout: "desc": "", 2023-11-26T07:23:10.119 INFO:teuthology.orchestra.run.smithi143.stdout: "long_desc": "", 2023-11-26T07:23:10.119 INFO:teuthology.orchestra.run.smithi143.stdout: "tags": [], 2023-11-26T07:23:10.119 INFO:teuthology.orchestra.run.smithi143.stdout: "see_also": [] 2023-11-26T07:23:10.119 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2023-11-26T07:23:10.119 INFO:teuthology.orchestra.run.smithi143.stdout: "AUDIT_API_LOG_PAYLOAD": { 2023-11-26T07:23:10.119 INFO:teuthology.orchestra.run.smithi143.stdout: "name": "AUDIT_API_LOG_PAYLOAD", 2023-11-26T07:23:10.119 INFO:teuthology.orchestra.run.smithi143.stdout: "type": "bool", 2023-11-26T07:23:10.119 INFO:teuthology.orchestra.run.smithi143.stdout: "level": "advanced", 2023-11-26T07:23:10.119 INFO:teuthology.orchestra.run.smithi143.stdout: "flags": 0, 2023-11-26T07:23:10.119 INFO:teuthology.orchestra.run.smithi143.stdout: "default_value": "True", 2023-11-26T07:23:10.119 INFO:teuthology.orchestra.run.smithi143.stdout: "min": "", 2023-11-26T07:23:10.120 INFO:teuthology.orchestra.run.smithi143.stdout: "max": "", 2023-11-26T07:23:10.120 INFO:teuthology.orchestra.run.smithi143.stdout: "enum_allowed": [], 2023-11-26T07:23:10.120 INFO:teuthology.orchestra.run.smithi143.stdout: "desc": "", 2023-11-26T07:23:10.120 INFO:teuthology.orchestra.run.smithi143.stdout: "long_desc": "", 2023-11-26T07:23:10.120 INFO:teuthology.orchestra.run.smithi143.stdout: "tags": [], 2023-11-26T07:23:10.120 INFO:teuthology.orchestra.run.smithi143.stdout: "see_also": [] 2023-11-26T07:23:10.120 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2023-11-26T07:23:10.120 INFO:teuthology.orchestra.run.smithi143.stdout: "ENABLE_BROWSABLE_API": { 2023-11-26T07:23:10.120 INFO:teuthology.orchestra.run.smithi143.stdout: "name": "ENABLE_BROWSABLE_API", 2023-11-26T07:23:10.120 INFO:teuthology.orchestra.run.smithi143.stdout: "type": "bool", 2023-11-26T07:23:10.120 INFO:teuthology.orchestra.run.smithi143.stdout: "level": "advanced", 2023-11-26T07:23:10.120 INFO:teuthology.orchestra.run.smithi143.stdout: "flags": 0, 2023-11-26T07:23:10.120 INFO:teuthology.orchestra.run.smithi143.stdout: "default_value": "True", 2023-11-26T07:23:10.121 INFO:teuthology.orchestra.run.smithi143.stdout: "min": "", 2023-11-26T07:23:10.121 INFO:teuthology.orchestra.run.smithi143.stdout: "max": "", 2023-11-26T07:23:10.121 INFO:teuthology.orchestra.run.smithi143.stdout: "enum_allowed": [], 2023-11-26T07:23:10.121 INFO:teuthology.orchestra.run.smithi143.stdout: "desc": "", 2023-11-26T07:23:10.121 INFO:teuthology.orchestra.run.smithi143.stdout: "long_desc": "", 2023-11-26T07:23:10.121 INFO:teuthology.orchestra.run.smithi143.stdout: "tags": [], 2023-11-26T07:23:10.121 INFO:teuthology.orchestra.run.smithi143.stdout: "see_also": [] 2023-11-26T07:23:10.121 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2023-11-26T07:23:10.121 INFO:teuthology.orchestra.run.smithi143.stdout: "FEATURE_TOGGLE_CEPHFS": { 2023-11-26T07:23:10.121 INFO:teuthology.orchestra.run.smithi143.stdout: "name": "FEATURE_TOGGLE_CEPHFS", 2023-11-26T07:23:10.121 INFO:teuthology.orchestra.run.smithi143.stdout: "type": "bool", 2023-11-26T07:23:10.121 INFO:teuthology.orchestra.run.smithi143.stdout: "level": "advanced", 2023-11-26T07:23:10.121 INFO:teuthology.orchestra.run.smithi143.stdout: "flags": 0, 2023-11-26T07:23:10.122 INFO:teuthology.orchestra.run.smithi143.stdout: "default_value": "True", 2023-11-26T07:23:10.122 INFO:teuthology.orchestra.run.smithi143.stdout: "min": "", 2023-11-26T07:23:10.122 INFO:teuthology.orchestra.run.smithi143.stdout: "max": "", 2023-11-26T07:23:10.122 INFO:teuthology.orchestra.run.smithi143.stdout: "enum_allowed": [], 2023-11-26T07:23:10.122 INFO:teuthology.orchestra.run.smithi143.stdout: "desc": "", 2023-11-26T07:23:10.122 INFO:teuthology.orchestra.run.smithi143.stdout: "long_desc": "", 2023-11-26T07:23:10.122 INFO:teuthology.orchestra.run.smithi143.stdout: "tags": [], 2023-11-26T07:23:10.122 INFO:teuthology.orchestra.run.smithi143.stdout: "see_also": [] 2023-11-26T07:23:10.122 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2023-11-26T07:23:10.122 INFO:teuthology.orchestra.run.smithi143.stdout: "FEATURE_TOGGLE_DASHBOARD": { 2023-11-26T07:23:10.122 INFO:teuthology.orchestra.run.smithi143.stdout: "name": "FEATURE_TOGGLE_DASHBOARD", 2023-11-26T07:23:10.122 INFO:teuthology.orchestra.run.smithi143.stdout: "type": "bool", 2023-11-26T07:23:10.123 INFO:teuthology.orchestra.run.smithi143.stdout: "level": "advanced", 2023-11-26T07:23:10.123 INFO:teuthology.orchestra.run.smithi143.stdout: "flags": 0, 2023-11-26T07:23:10.123 INFO:teuthology.orchestra.run.smithi143.stdout: "default_value": "True", 2023-11-26T07:23:10.123 INFO:teuthology.orchestra.run.smithi143.stdout: "min": "", 2023-11-26T07:23:10.123 INFO:teuthology.orchestra.run.smithi143.stdout: "max": "", 2023-11-26T07:23:10.123 INFO:teuthology.orchestra.run.smithi143.stdout: "enum_allowed": [], 2023-11-26T07:23:10.123 INFO:teuthology.orchestra.run.smithi143.stdout: "desc": "", 2023-11-26T07:23:10.123 INFO:teuthology.orchestra.run.smithi143.stdout: "long_desc": "", 2023-11-26T07:23:10.123 INFO:teuthology.orchestra.run.smithi143.stdout: "tags": [], 2023-11-26T07:23:10.123 INFO:teuthology.orchestra.run.smithi143.stdout: "see_also": [] 2023-11-26T07:23:10.123 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2023-11-26T07:23:10.123 INFO:teuthology.orchestra.run.smithi143.stdout: "FEATURE_TOGGLE_ISCSI": { 2023-11-26T07:23:10.123 INFO:teuthology.orchestra.run.smithi143.stdout: "name": "FEATURE_TOGGLE_ISCSI", 2023-11-26T07:23:10.123 INFO:teuthology.orchestra.run.smithi143.stdout: "type": "bool", 2023-11-26T07:23:10.124 INFO:teuthology.orchestra.run.smithi143.stdout: "level": "advanced", 2023-11-26T07:23:10.124 INFO:teuthology.orchestra.run.smithi143.stdout: "flags": 0, 2023-11-26T07:23:10.124 INFO:teuthology.orchestra.run.smithi143.stdout: "default_value": "True", 2023-11-26T07:23:10.124 INFO:teuthology.orchestra.run.smithi143.stdout: "min": "", 2023-11-26T07:23:10.124 INFO:teuthology.orchestra.run.smithi143.stdout: "max": "", 2023-11-26T07:23:10.124 INFO:teuthology.orchestra.run.smithi143.stdout: "enum_allowed": [], 2023-11-26T07:23:10.124 INFO:teuthology.orchestra.run.smithi143.stdout: "desc": "", 2023-11-26T07:23:10.124 INFO:teuthology.orchestra.run.smithi143.stdout: "long_desc": "", 2023-11-26T07:23:10.124 INFO:teuthology.orchestra.run.smithi143.stdout: "tags": [], 2023-11-26T07:23:10.124 INFO:teuthology.orchestra.run.smithi143.stdout: "see_also": [] 2023-11-26T07:23:10.124 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2023-11-26T07:23:10.124 INFO:teuthology.orchestra.run.smithi143.stdout: "FEATURE_TOGGLE_MIRRORING": { 2023-11-26T07:23:10.125 INFO:teuthology.orchestra.run.smithi143.stdout: "name": "FEATURE_TOGGLE_MIRRORING", 2023-11-26T07:23:10.125 INFO:teuthology.orchestra.run.smithi143.stdout: "type": "bool", 2023-11-26T07:23:10.125 INFO:teuthology.orchestra.run.smithi143.stdout: "level": "advanced", 2023-11-26T07:23:10.125 INFO:teuthology.orchestra.run.smithi143.stdout: "flags": 0, 2023-11-26T07:23:10.125 INFO:teuthology.orchestra.run.smithi143.stdout: "default_value": "True", 2023-11-26T07:23:10.125 INFO:teuthology.orchestra.run.smithi143.stdout: "min": "", 2023-11-26T07:23:10.125 INFO:teuthology.orchestra.run.smithi143.stdout: "max": "", 2023-11-26T07:23:10.125 INFO:teuthology.orchestra.run.smithi143.stdout: "enum_allowed": [], 2023-11-26T07:23:10.125 INFO:teuthology.orchestra.run.smithi143.stdout: "desc": "", 2023-11-26T07:23:10.125 INFO:teuthology.orchestra.run.smithi143.stdout: "long_desc": "", 2023-11-26T07:23:10.125 INFO:teuthology.orchestra.run.smithi143.stdout: "tags": [], 2023-11-26T07:23:10.125 INFO:teuthology.orchestra.run.smithi143.stdout: "see_also": [] 2023-11-26T07:23:10.125 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2023-11-26T07:23:10.126 INFO:teuthology.orchestra.run.smithi143.stdout: "FEATURE_TOGGLE_NFS": { 2023-11-26T07:23:10.126 INFO:teuthology.orchestra.run.smithi143.stdout: "name": "FEATURE_TOGGLE_NFS", 2023-11-26T07:23:10.126 INFO:teuthology.orchestra.run.smithi143.stdout: "type": "bool", 2023-11-26T07:23:10.126 INFO:teuthology.orchestra.run.smithi143.stdout: "level": "advanced", 2023-11-26T07:23:10.126 INFO:teuthology.orchestra.run.smithi143.stdout: "flags": 0, 2023-11-26T07:23:10.126 INFO:teuthology.orchestra.run.smithi143.stdout: "default_value": "True", 2023-11-26T07:23:10.126 INFO:teuthology.orchestra.run.smithi143.stdout: "min": "", 2023-11-26T07:23:10.126 INFO:teuthology.orchestra.run.smithi143.stdout: "max": "", 2023-11-26T07:23:10.126 INFO:teuthology.orchestra.run.smithi143.stdout: "enum_allowed": [], 2023-11-26T07:23:10.126 INFO:teuthology.orchestra.run.smithi143.stdout: "desc": "", 2023-11-26T07:23:10.126 INFO:teuthology.orchestra.run.smithi143.stdout: "long_desc": "", 2023-11-26T07:23:10.126 INFO:teuthology.orchestra.run.smithi143.stdout: "tags": [], 2023-11-26T07:23:10.127 INFO:teuthology.orchestra.run.smithi143.stdout: "see_also": [] 2023-11-26T07:23:10.127 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2023-11-26T07:23:10.127 INFO:teuthology.orchestra.run.smithi143.stdout: "FEATURE_TOGGLE_RBD": { 2023-11-26T07:23:10.127 INFO:teuthology.orchestra.run.smithi143.stdout: "name": "FEATURE_TOGGLE_RBD", 2023-11-26T07:23:10.127 INFO:teuthology.orchestra.run.smithi143.stdout: "type": "bool", 2023-11-26T07:23:10.127 INFO:teuthology.orchestra.run.smithi143.stdout: "level": "advanced", 2023-11-26T07:23:10.127 INFO:teuthology.orchestra.run.smithi143.stdout: "flags": 0, 2023-11-26T07:23:10.127 INFO:teuthology.orchestra.run.smithi143.stdout: "default_value": "True", 2023-11-26T07:23:10.127 INFO:teuthology.orchestra.run.smithi143.stdout: "min": "", 2023-11-26T07:23:10.127 INFO:teuthology.orchestra.run.smithi143.stdout: "max": "", 2023-11-26T07:23:10.127 INFO:teuthology.orchestra.run.smithi143.stdout: "enum_allowed": [], 2023-11-26T07:23:10.127 INFO:teuthology.orchestra.run.smithi143.stdout: "desc": "", 2023-11-26T07:23:10.128 INFO:teuthology.orchestra.run.smithi143.stdout: "long_desc": "", 2023-11-26T07:23:10.128 INFO:teuthology.orchestra.run.smithi143.stdout: "tags": [], 2023-11-26T07:23:10.128 INFO:teuthology.orchestra.run.smithi143.stdout: "see_also": [] 2023-11-26T07:23:10.128 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2023-11-26T07:23:10.128 INFO:teuthology.orchestra.run.smithi143.stdout: "FEATURE_TOGGLE_RGW": { 2023-11-26T07:23:10.128 INFO:teuthology.orchestra.run.smithi143.stdout: "name": "FEATURE_TOGGLE_RGW", 2023-11-26T07:23:10.128 INFO:teuthology.orchestra.run.smithi143.stdout: "type": "bool", 2023-11-26T07:23:10.128 INFO:teuthology.orchestra.run.smithi143.stdout: "level": "advanced", 2023-11-26T07:23:10.128 INFO:teuthology.orchestra.run.smithi143.stdout: "flags": 0, 2023-11-26T07:23:10.128 INFO:teuthology.orchestra.run.smithi143.stdout: "default_value": "True", 2023-11-26T07:23:10.128 INFO:teuthology.orchestra.run.smithi143.stdout: "min": "", 2023-11-26T07:23:10.128 INFO:teuthology.orchestra.run.smithi143.stdout: "max": "", 2023-11-26T07:23:10.128 INFO:teuthology.orchestra.run.smithi143.stdout: "enum_allowed": [], 2023-11-26T07:23:10.129 INFO:teuthology.orchestra.run.smithi143.stdout: "desc": "", 2023-11-26T07:23:10.129 INFO:teuthology.orchestra.run.smithi143.stdout: "long_desc": "", 2023-11-26T07:23:10.129 INFO:teuthology.orchestra.run.smithi143.stdout: "tags": [], 2023-11-26T07:23:10.129 INFO:teuthology.orchestra.run.smithi143.stdout: "see_also": [] 2023-11-26T07:23:10.129 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2023-11-26T07:23:10.129 INFO:teuthology.orchestra.run.smithi143.stdout: "GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE": { 2023-11-26T07:23:10.129 INFO:teuthology.orchestra.run.smithi143.stdout: "name": "GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE", 2023-11-26T07:23:10.129 INFO:teuthology.orchestra.run.smithi143.stdout: "type": "str", 2023-11-26T07:23:10.129 INFO:teuthology.orchestra.run.smithi143.stdout: "level": "advanced", 2023-11-26T07:23:10.129 INFO:teuthology.orchestra.run.smithi143.stdout: "flags": 0, 2023-11-26T07:23:10.129 INFO:teuthology.orchestra.run.smithi143.stdout: "default_value": "", 2023-11-26T07:23:10.129 INFO:teuthology.orchestra.run.smithi143.stdout: "min": "", 2023-11-26T07:23:10.129 INFO:teuthology.orchestra.run.smithi143.stdout: "max": "", 2023-11-26T07:23:10.130 INFO:teuthology.orchestra.run.smithi143.stdout: "enum_allowed": [], 2023-11-26T07:23:10.130 INFO:teuthology.orchestra.run.smithi143.stdout: "desc": "", 2023-11-26T07:23:10.130 INFO:teuthology.orchestra.run.smithi143.stdout: "long_desc": "", 2023-11-26T07:23:10.130 INFO:teuthology.orchestra.run.smithi143.stdout: "tags": [], 2023-11-26T07:23:10.130 INFO:teuthology.orchestra.run.smithi143.stdout: "see_also": [] 2023-11-26T07:23:10.130 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2023-11-26T07:23:10.130 INFO:teuthology.orchestra.run.smithi143.stdout: "GRAFANA_API_PASSWORD": { 2023-11-26T07:23:10.130 INFO:teuthology.orchestra.run.smithi143.stdout: "name": "GRAFANA_API_PASSWORD", 2023-11-26T07:23:10.130 INFO:teuthology.orchestra.run.smithi143.stdout: "type": "str", 2023-11-26T07:23:10.130 INFO:teuthology.orchestra.run.smithi143.stdout: "level": "advanced", 2023-11-26T07:23:10.130 INFO:teuthology.orchestra.run.smithi143.stdout: "flags": 0, 2023-11-26T07:23:10.130 INFO:teuthology.orchestra.run.smithi143.stdout: "default_value": "admin", 2023-11-26T07:23:10.131 INFO:teuthology.orchestra.run.smithi143.stdout: "min": "", 2023-11-26T07:23:10.131 INFO:teuthology.orchestra.run.smithi143.stdout: "max": "", 2023-11-26T07:23:10.131 INFO:teuthology.orchestra.run.smithi143.stdout: "enum_allowed": [], 2023-11-26T07:23:10.131 INFO:teuthology.orchestra.run.smithi143.stdout: "desc": "", 2023-11-26T07:23:10.131 INFO:teuthology.orchestra.run.smithi143.stdout: "long_desc": "", 2023-11-26T07:23:10.131 INFO:teuthology.orchestra.run.smithi143.stdout: "tags": [], 2023-11-26T07:23:10.131 INFO:teuthology.orchestra.run.smithi143.stdout: "see_also": [] 2023-11-26T07:23:10.131 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2023-11-26T07:23:10.131 INFO:teuthology.orchestra.run.smithi143.stdout: "GRAFANA_API_SSL_VERIFY": { 2023-11-26T07:23:10.131 INFO:teuthology.orchestra.run.smithi143.stdout: "name": "GRAFANA_API_SSL_VERIFY", 2023-11-26T07:23:10.131 INFO:teuthology.orchestra.run.smithi143.stdout: "type": "bool", 2023-11-26T07:23:10.131 INFO:teuthology.orchestra.run.smithi143.stdout: "level": "advanced", 2023-11-26T07:23:10.131 INFO:teuthology.orchestra.run.smithi143.stdout: "flags": 0, 2023-11-26T07:23:10.131 INFO:teuthology.orchestra.run.smithi143.stdout: "default_value": "True", 2023-11-26T07:23:10.132 INFO:teuthology.orchestra.run.smithi143.stdout: "min": "", 2023-11-26T07:23:10.132 INFO:teuthology.orchestra.run.smithi143.stdout: "max": "", 2023-11-26T07:23:10.132 INFO:teuthology.orchestra.run.smithi143.stdout: "enum_allowed": [], 2023-11-26T07:23:10.132 INFO:teuthology.orchestra.run.smithi143.stdout: "desc": "", 2023-11-26T07:23:10.132 INFO:teuthology.orchestra.run.smithi143.stdout: "long_desc": "", 2023-11-26T07:23:10.132 INFO:teuthology.orchestra.run.smithi143.stdout: "tags": [], 2023-11-26T07:23:10.132 INFO:teuthology.orchestra.run.smithi143.stdout: "see_also": [] 2023-11-26T07:23:10.132 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2023-11-26T07:23:10.132 INFO:teuthology.orchestra.run.smithi143.stdout: "GRAFANA_API_URL": { 2023-11-26T07:23:10.132 INFO:teuthology.orchestra.run.smithi143.stdout: "name": "GRAFANA_API_URL", 2023-11-26T07:23:10.132 INFO:teuthology.orchestra.run.smithi143.stdout: "type": "str", 2023-11-26T07:23:10.132 INFO:teuthology.orchestra.run.smithi143.stdout: "level": "advanced", 2023-11-26T07:23:10.133 INFO:teuthology.orchestra.run.smithi143.stdout: "flags": 0, 2023-11-26T07:23:10.133 INFO:teuthology.orchestra.run.smithi143.stdout: "default_value": "", 2023-11-26T07:23:10.133 INFO:teuthology.orchestra.run.smithi143.stdout: "min": "", 2023-11-26T07:23:10.133 INFO:teuthology.orchestra.run.smithi143.stdout: "max": "", 2023-11-26T07:23:10.133 INFO:teuthology.orchestra.run.smithi143.stdout: "enum_allowed": [], 2023-11-26T07:23:10.133 INFO:teuthology.orchestra.run.smithi143.stdout: "desc": "", 2023-11-26T07:23:10.133 INFO:teuthology.orchestra.run.smithi143.stdout: "long_desc": "", 2023-11-26T07:23:10.133 INFO:teuthology.orchestra.run.smithi143.stdout: "tags": [], 2023-11-26T07:23:10.133 INFO:teuthology.orchestra.run.smithi143.stdout: "see_also": [] 2023-11-26T07:23:10.133 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2023-11-26T07:23:10.133 INFO:teuthology.orchestra.run.smithi143.stdout: "GRAFANA_API_USERNAME": { 2023-11-26T07:23:10.133 INFO:teuthology.orchestra.run.smithi143.stdout: "name": "GRAFANA_API_USERNAME", 2023-11-26T07:23:10.133 INFO:teuthology.orchestra.run.smithi143.stdout: "type": "str", 2023-11-26T07:23:10.134 INFO:teuthology.orchestra.run.smithi143.stdout: "level": "advanced", 2023-11-26T07:23:10.134 INFO:teuthology.orchestra.run.smithi143.stdout: "flags": 0, 2023-11-26T07:23:10.134 INFO:teuthology.orchestra.run.smithi143.stdout: "default_value": "admin", 2023-11-26T07:23:10.134 INFO:teuthology.orchestra.run.smithi143.stdout: "min": "", 2023-11-26T07:23:10.134 INFO:teuthology.orchestra.run.smithi143.stdout: "max": "", 2023-11-26T07:23:10.134 INFO:teuthology.orchestra.run.smithi143.stdout: "enum_allowed": [], 2023-11-26T07:23:10.134 INFO:teuthology.orchestra.run.smithi143.stdout: "desc": "", 2023-11-26T07:23:10.134 INFO:teuthology.orchestra.run.smithi143.stdout: "long_desc": "", 2023-11-26T07:23:10.134 INFO:teuthology.orchestra.run.smithi143.stdout: "tags": [], 2023-11-26T07:23:10.134 INFO:teuthology.orchestra.run.smithi143.stdout: "see_also": [] 2023-11-26T07:23:10.134 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2023-11-26T07:23:10.134 INFO:teuthology.orchestra.run.smithi143.stdout: "GRAFANA_FRONTEND_API_URL": { 2023-11-26T07:23:10.135 INFO:teuthology.orchestra.run.smithi143.stdout: "name": "GRAFANA_FRONTEND_API_URL", 2023-11-26T07:23:10.135 INFO:teuthology.orchestra.run.smithi143.stdout: "type": "str", 2023-11-26T07:23:10.135 INFO:teuthology.orchestra.run.smithi143.stdout: "level": "advanced", 2023-11-26T07:23:10.135 INFO:teuthology.orchestra.run.smithi143.stdout: "flags": 0, 2023-11-26T07:23:10.135 INFO:teuthology.orchestra.run.smithi143.stdout: "default_value": "", 2023-11-26T07:23:10.135 INFO:teuthology.orchestra.run.smithi143.stdout: "min": "", 2023-11-26T07:23:10.135 INFO:teuthology.orchestra.run.smithi143.stdout: "max": "", 2023-11-26T07:23:10.135 INFO:teuthology.orchestra.run.smithi143.stdout: "enum_allowed": [], 2023-11-26T07:23:10.135 INFO:teuthology.orchestra.run.smithi143.stdout: "desc": "", 2023-11-26T07:23:10.135 INFO:teuthology.orchestra.run.smithi143.stdout: "long_desc": "", 2023-11-26T07:23:10.135 INFO:teuthology.orchestra.run.smithi143.stdout: "tags": [], 2023-11-26T07:23:10.135 INFO:teuthology.orchestra.run.smithi143.stdout: "see_also": [] 2023-11-26T07:23:10.135 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2023-11-26T07:23:10.136 INFO:teuthology.orchestra.run.smithi143.stdout: "GRAFANA_UPDATE_DASHBOARDS": { 2023-11-26T07:23:10.136 INFO:teuthology.orchestra.run.smithi143.stdout: "name": "GRAFANA_UPDATE_DASHBOARDS", 2023-11-26T07:23:10.136 INFO:teuthology.orchestra.run.smithi143.stdout: "type": "bool", 2023-11-26T07:23:10.136 INFO:teuthology.orchestra.run.smithi143.stdout: "level": "advanced", 2023-11-26T07:23:10.136 INFO:teuthology.orchestra.run.smithi143.stdout: "flags": 0, 2023-11-26T07:23:10.136 INFO:teuthology.orchestra.run.smithi143.stdout: "default_value": "False", 2023-11-26T07:23:10.136 INFO:teuthology.orchestra.run.smithi143.stdout: "min": "", 2023-11-26T07:23:10.136 INFO:teuthology.orchestra.run.smithi143.stdout: "max": "", 2023-11-26T07:23:10.136 INFO:teuthology.orchestra.run.smithi143.stdout: "enum_allowed": [], 2023-11-26T07:23:10.136 INFO:teuthology.orchestra.run.smithi143.stdout: "desc": "", 2023-11-26T07:23:10.136 INFO:teuthology.orchestra.run.smithi143.stdout: "long_desc": "", 2023-11-26T07:23:10.136 INFO:teuthology.orchestra.run.smithi143.stdout: "tags": [], 2023-11-26T07:23:10.137 INFO:teuthology.orchestra.run.smithi143.stdout: "see_also": [] 2023-11-26T07:23:10.137 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2023-11-26T07:23:10.137 INFO:teuthology.orchestra.run.smithi143.stdout: "ISCSI_API_SSL_VERIFICATION": { 2023-11-26T07:23:10.137 INFO:teuthology.orchestra.run.smithi143.stdout: "name": "ISCSI_API_SSL_VERIFICATION", 2023-11-26T07:23:10.137 INFO:teuthology.orchestra.run.smithi143.stdout: "type": "bool", 2023-11-26T07:23:10.137 INFO:teuthology.orchestra.run.smithi143.stdout: "level": "advanced", 2023-11-26T07:23:10.137 INFO:teuthology.orchestra.run.smithi143.stdout: "flags": 0, 2023-11-26T07:23:10.137 INFO:teuthology.orchestra.run.smithi143.stdout: "default_value": "True", 2023-11-26T07:23:10.137 INFO:teuthology.orchestra.run.smithi143.stdout: "min": "", 2023-11-26T07:23:10.137 INFO:teuthology.orchestra.run.smithi143.stdout: "max": "", 2023-11-26T07:23:10.137 INFO:teuthology.orchestra.run.smithi143.stdout: "enum_allowed": [], 2023-11-26T07:23:10.137 INFO:teuthology.orchestra.run.smithi143.stdout: "desc": "", 2023-11-26T07:23:10.137 INFO:teuthology.orchestra.run.smithi143.stdout: "long_desc": "", 2023-11-26T07:23:10.138 INFO:teuthology.orchestra.run.smithi143.stdout: "tags": [], 2023-11-26T07:23:10.138 INFO:teuthology.orchestra.run.smithi143.stdout: "see_also": [] 2023-11-26T07:23:10.138 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2023-11-26T07:23:10.138 INFO:teuthology.orchestra.run.smithi143.stdout: "ISSUE_TRACKER_API_KEY": { 2023-11-26T07:23:10.138 INFO:teuthology.orchestra.run.smithi143.stdout: "name": "ISSUE_TRACKER_API_KEY", 2023-11-26T07:23:10.138 INFO:teuthology.orchestra.run.smithi143.stdout: "type": "str", 2023-11-26T07:23:10.138 INFO:teuthology.orchestra.run.smithi143.stdout: "level": "advanced", 2023-11-26T07:23:10.138 INFO:teuthology.orchestra.run.smithi143.stdout: "flags": 0, 2023-11-26T07:23:10.138 INFO:teuthology.orchestra.run.smithi143.stdout: "default_value": "", 2023-11-26T07:23:10.138 INFO:teuthology.orchestra.run.smithi143.stdout: "min": "", 2023-11-26T07:23:10.138 INFO:teuthology.orchestra.run.smithi143.stdout: "max": "", 2023-11-26T07:23:10.138 INFO:teuthology.orchestra.run.smithi143.stdout: "enum_allowed": [], 2023-11-26T07:23:10.139 INFO:teuthology.orchestra.run.smithi143.stdout: "desc": "", 2023-11-26T07:23:10.139 INFO:teuthology.orchestra.run.smithi143.stdout: "long_desc": "", 2023-11-26T07:23:10.139 INFO:teuthology.orchestra.run.smithi143.stdout: "tags": [], 2023-11-26T07:23:10.139 INFO:teuthology.orchestra.run.smithi143.stdout: "see_also": [] 2023-11-26T07:23:10.139 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2023-11-26T07:23:10.139 INFO:teuthology.orchestra.run.smithi143.stdout: "PROMETHEUS_API_HOST": { 2023-11-26T07:23:10.139 INFO:teuthology.orchestra.run.smithi143.stdout: "name": "PROMETHEUS_API_HOST", 2023-11-26T07:23:10.139 INFO:teuthology.orchestra.run.smithi143.stdout: "type": "str", 2023-11-26T07:23:10.139 INFO:teuthology.orchestra.run.smithi143.stdout: "level": "advanced", 2023-11-26T07:23:10.139 INFO:teuthology.orchestra.run.smithi143.stdout: "flags": 0, 2023-11-26T07:23:10.139 INFO:teuthology.orchestra.run.smithi143.stdout: "default_value": "", 2023-11-26T07:23:10.139 INFO:teuthology.orchestra.run.smithi143.stdout: "min": "", 2023-11-26T07:23:10.140 INFO:teuthology.orchestra.run.smithi143.stdout: "max": "", 2023-11-26T07:23:10.140 INFO:teuthology.orchestra.run.smithi143.stdout: "enum_allowed": [], 2023-11-26T07:23:10.140 INFO:teuthology.orchestra.run.smithi143.stdout: "desc": "", 2023-11-26T07:23:10.140 INFO:teuthology.orchestra.run.smithi143.stdout: "long_desc": "", 2023-11-26T07:23:10.140 INFO:teuthology.orchestra.run.smithi143.stdout: "tags": [], 2023-11-26T07:23:10.140 INFO:teuthology.orchestra.run.smithi143.stdout: "see_also": [] 2023-11-26T07:23:10.140 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2023-11-26T07:23:10.140 INFO:teuthology.orchestra.run.smithi143.stdout: "PROMETHEUS_API_SSL_VERIFY": { 2023-11-26T07:23:10.140 INFO:teuthology.orchestra.run.smithi143.stdout: "name": "PROMETHEUS_API_SSL_VERIFY", 2023-11-26T07:23:10.140 INFO:teuthology.orchestra.run.smithi143.stdout: "type": "bool", 2023-11-26T07:23:10.140 INFO:teuthology.orchestra.run.smithi143.stdout: "level": "advanced", 2023-11-26T07:23:10.140 INFO:teuthology.orchestra.run.smithi143.stdout: "flags": 0, 2023-11-26T07:23:10.140 INFO:teuthology.orchestra.run.smithi143.stdout: "default_value": "True", 2023-11-26T07:23:10.141 INFO:teuthology.orchestra.run.smithi143.stdout: "min": "", 2023-11-26T07:23:10.141 INFO:teuthology.orchestra.run.smithi143.stdout: "max": "", 2023-11-26T07:23:10.141 INFO:teuthology.orchestra.run.smithi143.stdout: "enum_allowed": [], 2023-11-26T07:23:10.141 INFO:teuthology.orchestra.run.smithi143.stdout: "desc": "", 2023-11-26T07:23:10.141 INFO:teuthology.orchestra.run.smithi143.stdout: "long_desc": "", 2023-11-26T07:23:10.141 INFO:teuthology.orchestra.run.smithi143.stdout: "tags": [], 2023-11-26T07:23:10.141 INFO:teuthology.orchestra.run.smithi143.stdout: "see_also": [] 2023-11-26T07:23:10.141 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2023-11-26T07:23:10.141 INFO:teuthology.orchestra.run.smithi143.stdout: "PWD_POLICY_CHECK_COMPLEXITY_ENABLED": { 2023-11-26T07:23:10.141 INFO:teuthology.orchestra.run.smithi143.stdout: "name": "PWD_POLICY_CHECK_COMPLEXITY_ENABLED", 2023-11-26T07:23:10.141 INFO:teuthology.orchestra.run.smithi143.stdout: "type": "bool", 2023-11-26T07:23:10.141 INFO:teuthology.orchestra.run.smithi143.stdout: "level": "advanced", 2023-11-26T07:23:10.142 INFO:teuthology.orchestra.run.smithi143.stdout: "flags": 0, 2023-11-26T07:23:10.142 INFO:teuthology.orchestra.run.smithi143.stdout: "default_value": "False", 2023-11-26T07:23:10.142 INFO:teuthology.orchestra.run.smithi143.stdout: "min": "", 2023-11-26T07:23:10.142 INFO:teuthology.orchestra.run.smithi143.stdout: "max": "", 2023-11-26T07:23:10.142 INFO:teuthology.orchestra.run.smithi143.stdout: "enum_allowed": [], 2023-11-26T07:23:10.142 INFO:teuthology.orchestra.run.smithi143.stdout: "desc": "", 2023-11-26T07:23:10.142 INFO:teuthology.orchestra.run.smithi143.stdout: "long_desc": "", 2023-11-26T07:23:10.142 INFO:teuthology.orchestra.run.smithi143.stdout: "tags": [], 2023-11-26T07:23:10.142 INFO:teuthology.orchestra.run.smithi143.stdout: "see_also": [] 2023-11-26T07:23:10.142 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2023-11-26T07:23:10.142 INFO:teuthology.orchestra.run.smithi143.stdout: "PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED": { 2023-11-26T07:23:10.142 INFO:teuthology.orchestra.run.smithi143.stdout: "name": "PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED", 2023-11-26T07:23:10.143 INFO:teuthology.orchestra.run.smithi143.stdout: "type": "bool", 2023-11-26T07:23:10.143 INFO:teuthology.orchestra.run.smithi143.stdout: "level": "advanced", 2023-11-26T07:23:10.143 INFO:teuthology.orchestra.run.smithi143.stdout: "flags": 0, 2023-11-26T07:23:10.143 INFO:teuthology.orchestra.run.smithi143.stdout: "default_value": "False", 2023-11-26T07:23:10.143 INFO:teuthology.orchestra.run.smithi143.stdout: "min": "", 2023-11-26T07:23:10.143 INFO:teuthology.orchestra.run.smithi143.stdout: "max": "", 2023-11-26T07:23:10.143 INFO:teuthology.orchestra.run.smithi143.stdout: "enum_allowed": [], 2023-11-26T07:23:10.143 INFO:teuthology.orchestra.run.smithi143.stdout: "desc": "", 2023-11-26T07:23:10.143 INFO:teuthology.orchestra.run.smithi143.stdout: "long_desc": "", 2023-11-26T07:23:10.143 INFO:teuthology.orchestra.run.smithi143.stdout: "tags": [], 2023-11-26T07:23:10.143 INFO:teuthology.orchestra.run.smithi143.stdout: "see_also": [] 2023-11-26T07:23:10.143 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2023-11-26T07:23:10.144 INFO:teuthology.orchestra.run.smithi143.stdout: "PWD_POLICY_CHECK_LENGTH_ENABLED": { 2023-11-26T07:23:10.144 INFO:teuthology.orchestra.run.smithi143.stdout: "name": "PWD_POLICY_CHECK_LENGTH_ENABLED", 2023-11-26T07:23:10.144 INFO:teuthology.orchestra.run.smithi143.stdout: "type": "bool", 2023-11-26T07:23:10.144 INFO:teuthology.orchestra.run.smithi143.stdout: "level": "advanced", 2023-11-26T07:23:10.144 INFO:teuthology.orchestra.run.smithi143.stdout: "flags": 0, 2023-11-26T07:23:10.144 INFO:teuthology.orchestra.run.smithi143.stdout: "default_value": "True", 2023-11-26T07:23:10.144 INFO:teuthology.orchestra.run.smithi143.stdout: "min": "", 2023-11-26T07:23:10.144 INFO:teuthology.orchestra.run.smithi143.stdout: "max": "", 2023-11-26T07:23:10.144 INFO:teuthology.orchestra.run.smithi143.stdout: "enum_allowed": [], 2023-11-26T07:23:10.144 INFO:teuthology.orchestra.run.smithi143.stdout: "desc": "", 2023-11-26T07:23:10.144 INFO:teuthology.orchestra.run.smithi143.stdout: "long_desc": "", 2023-11-26T07:23:10.145 INFO:teuthology.orchestra.run.smithi143.stdout: "tags": [], 2023-11-26T07:23:10.145 INFO:teuthology.orchestra.run.smithi143.stdout: "see_also": [] 2023-11-26T07:23:10.145 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2023-11-26T07:23:10.145 INFO:teuthology.orchestra.run.smithi143.stdout: "PWD_POLICY_CHECK_OLDPWD_ENABLED": { 2023-11-26T07:23:10.145 INFO:teuthology.orchestra.run.smithi143.stdout: "name": "PWD_POLICY_CHECK_OLDPWD_ENABLED", 2023-11-26T07:23:10.145 INFO:teuthology.orchestra.run.smithi143.stdout: "type": "bool", 2023-11-26T07:23:10.145 INFO:teuthology.orchestra.run.smithi143.stdout: "level": "advanced", 2023-11-26T07:23:10.145 INFO:teuthology.orchestra.run.smithi143.stdout: "flags": 0, 2023-11-26T07:23:10.145 INFO:teuthology.orchestra.run.smithi143.stdout: "default_value": "True", 2023-11-26T07:23:10.145 INFO:teuthology.orchestra.run.smithi143.stdout: "min": "", 2023-11-26T07:23:10.145 INFO:teuthology.orchestra.run.smithi143.stdout: "max": "", 2023-11-26T07:23:10.145 INFO:teuthology.orchestra.run.smithi143.stdout: "enum_allowed": [], 2023-11-26T07:23:10.145 INFO:teuthology.orchestra.run.smithi143.stdout: "desc": "", 2023-11-26T07:23:10.146 INFO:teuthology.orchestra.run.smithi143.stdout: "long_desc": "", 2023-11-26T07:23:10.146 INFO:teuthology.orchestra.run.smithi143.stdout: "tags": [], 2023-11-26T07:23:10.146 INFO:teuthology.orchestra.run.smithi143.stdout: "see_also": [] 2023-11-26T07:23:10.146 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2023-11-26T07:23:10.146 INFO:teuthology.orchestra.run.smithi143.stdout: "PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED": { 2023-11-26T07:23:10.146 INFO:teuthology.orchestra.run.smithi143.stdout: "name": "PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED", 2023-11-26T07:23:10.146 INFO:teuthology.orchestra.run.smithi143.stdout: "type": "bool", 2023-11-26T07:23:10.146 INFO:teuthology.orchestra.run.smithi143.stdout: "level": "advanced", 2023-11-26T07:23:10.146 INFO:teuthology.orchestra.run.smithi143.stdout: "flags": 0, 2023-11-26T07:23:10.146 INFO:teuthology.orchestra.run.smithi143.stdout: "default_value": "False", 2023-11-26T07:23:10.146 INFO:teuthology.orchestra.run.smithi143.stdout: "min": "", 2023-11-26T07:23:10.146 INFO:teuthology.orchestra.run.smithi143.stdout: "max": "", 2023-11-26T07:23:10.146 INFO:teuthology.orchestra.run.smithi143.stdout: "enum_allowed": [], 2023-11-26T07:23:10.147 INFO:teuthology.orchestra.run.smithi143.stdout: "desc": "", 2023-11-26T07:23:10.147 INFO:teuthology.orchestra.run.smithi143.stdout: "long_desc": "", 2023-11-26T07:23:10.147 INFO:teuthology.orchestra.run.smithi143.stdout: "tags": [], 2023-11-26T07:23:10.147 INFO:teuthology.orchestra.run.smithi143.stdout: "see_also": [] 2023-11-26T07:23:10.147 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2023-11-26T07:23:10.147 INFO:teuthology.orchestra.run.smithi143.stdout: "PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED": { 2023-11-26T07:23:10.147 INFO:teuthology.orchestra.run.smithi143.stdout: "name": "PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED", 2023-11-26T07:23:10.147 INFO:teuthology.orchestra.run.smithi143.stdout: "type": "bool", 2023-11-26T07:23:10.147 INFO:teuthology.orchestra.run.smithi143.stdout: "level": "advanced", 2023-11-26T07:23:10.147 INFO:teuthology.orchestra.run.smithi143.stdout: "flags": 0, 2023-11-26T07:23:10.147 INFO:teuthology.orchestra.run.smithi143.stdout: "default_value": "False", 2023-11-26T07:23:10.147 INFO:teuthology.orchestra.run.smithi143.stdout: "min": "", 2023-11-26T07:23:10.148 INFO:teuthology.orchestra.run.smithi143.stdout: "max": "", 2023-11-26T07:23:10.148 INFO:teuthology.orchestra.run.smithi143.stdout: "enum_allowed": [], 2023-11-26T07:23:10.148 INFO:teuthology.orchestra.run.smithi143.stdout: "desc": "", 2023-11-26T07:23:10.148 INFO:teuthology.orchestra.run.smithi143.stdout: "long_desc": "", 2023-11-26T07:23:10.148 INFO:teuthology.orchestra.run.smithi143.stdout: "tags": [], 2023-11-26T07:23:10.148 INFO:teuthology.orchestra.run.smithi143.stdout: "see_also": [] 2023-11-26T07:23:10.148 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2023-11-26T07:23:10.148 INFO:teuthology.orchestra.run.smithi143.stdout: "PWD_POLICY_CHECK_USERNAME_ENABLED": { 2023-11-26T07:23:10.148 INFO:teuthology.orchestra.run.smithi143.stdout: "name": "PWD_POLICY_CHECK_USERNAME_ENABLED", 2023-11-26T07:23:10.148 INFO:teuthology.orchestra.run.smithi143.stdout: "type": "bool", 2023-11-26T07:23:10.148 INFO:teuthology.orchestra.run.smithi143.stdout: "level": "advanced", 2023-11-26T07:23:10.148 INFO:teuthology.orchestra.run.smithi143.stdout: "flags": 0, 2023-11-26T07:23:10.148 INFO:teuthology.orchestra.run.smithi143.stdout: "default_value": "False", 2023-11-26T07:23:10.148 INFO:teuthology.orchestra.run.smithi143.stdout: "min": "", 2023-11-26T07:23:10.149 INFO:teuthology.orchestra.run.smithi143.stdout: "max": "", 2023-11-26T07:23:10.149 INFO:teuthology.orchestra.run.smithi143.stdout: "enum_allowed": [], 2023-11-26T07:23:10.149 INFO:teuthology.orchestra.run.smithi143.stdout: "desc": "", 2023-11-26T07:23:10.149 INFO:teuthology.orchestra.run.smithi143.stdout: "long_desc": "", 2023-11-26T07:23:10.149 INFO:teuthology.orchestra.run.smithi143.stdout: "tags": [], 2023-11-26T07:23:10.149 INFO:teuthology.orchestra.run.smithi143.stdout: "see_also": [] 2023-11-26T07:23:10.149 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2023-11-26T07:23:10.149 INFO:teuthology.orchestra.run.smithi143.stdout: "PWD_POLICY_ENABLED": { 2023-11-26T07:23:10.149 INFO:teuthology.orchestra.run.smithi143.stdout: "name": "PWD_POLICY_ENABLED", 2023-11-26T07:23:10.149 INFO:teuthology.orchestra.run.smithi143.stdout: "type": "bool", 2023-11-26T07:23:10.149 INFO:teuthology.orchestra.run.smithi143.stdout: "level": "advanced", 2023-11-26T07:23:10.149 INFO:teuthology.orchestra.run.smithi143.stdout: "flags": 0, 2023-11-26T07:23:10.149 INFO:teuthology.orchestra.run.smithi143.stdout: "default_value": "True", 2023-11-26T07:23:10.150 INFO:teuthology.orchestra.run.smithi143.stdout: "min": "", 2023-11-26T07:23:10.150 INFO:teuthology.orchestra.run.smithi143.stdout: "max": "", 2023-11-26T07:23:10.150 INFO:teuthology.orchestra.run.smithi143.stdout: "enum_allowed": [], 2023-11-26T07:23:10.150 INFO:teuthology.orchestra.run.smithi143.stdout: "desc": "", 2023-11-26T07:23:10.150 INFO:teuthology.orchestra.run.smithi143.stdout: "long_desc": "", 2023-11-26T07:23:10.150 INFO:teuthology.orchestra.run.smithi143.stdout: "tags": [], 2023-11-26T07:23:10.150 INFO:teuthology.orchestra.run.smithi143.stdout: "see_also": [] 2023-11-26T07:23:10.150 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2023-11-26T07:23:10.150 INFO:teuthology.orchestra.run.smithi143.stdout: "PWD_POLICY_EXCLUSION_LIST": { 2023-11-26T07:23:10.150 INFO:teuthology.orchestra.run.smithi143.stdout: "name": "PWD_POLICY_EXCLUSION_LIST", 2023-11-26T07:23:10.150 INFO:teuthology.orchestra.run.smithi143.stdout: "type": "str", 2023-11-26T07:23:10.150 INFO:teuthology.orchestra.run.smithi143.stdout: "level": "advanced", 2023-11-26T07:23:10.151 INFO:teuthology.orchestra.run.smithi143.stdout: "flags": 0, 2023-11-26T07:23:10.151 INFO:teuthology.orchestra.run.smithi143.stdout: "default_value": "osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps", 2023-11-26T07:23:10.151 INFO:teuthology.orchestra.run.smithi143.stdout: "min": "", 2023-11-26T07:23:10.151 INFO:teuthology.orchestra.run.smithi143.stdout: "max": "", 2023-11-26T07:23:10.151 INFO:teuthology.orchestra.run.smithi143.stdout: "enum_allowed": [], 2023-11-26T07:23:10.151 INFO:teuthology.orchestra.run.smithi143.stdout: "desc": "", 2023-11-26T07:23:10.151 INFO:teuthology.orchestra.run.smithi143.stdout: "long_desc": "", 2023-11-26T07:23:10.151 INFO:teuthology.orchestra.run.smithi143.stdout: "tags": [], 2023-11-26T07:23:10.151 INFO:teuthology.orchestra.run.smithi143.stdout: "see_also": [] 2023-11-26T07:23:10.151 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2023-11-26T07:23:10.151 INFO:teuthology.orchestra.run.smithi143.stdout: "PWD_POLICY_MIN_COMPLEXITY": { 2023-11-26T07:23:10.151 INFO:teuthology.orchestra.run.smithi143.stdout: "name": "PWD_POLICY_MIN_COMPLEXITY", 2023-11-26T07:23:10.151 INFO:teuthology.orchestra.run.smithi143.stdout: "type": "int", 2023-11-26T07:23:10.152 INFO:teuthology.orchestra.run.smithi143.stdout: "level": "advanced", 2023-11-26T07:23:10.152 INFO:teuthology.orchestra.run.smithi143.stdout: "flags": 0, 2023-11-26T07:23:10.152 INFO:teuthology.orchestra.run.smithi143.stdout: "default_value": "10", 2023-11-26T07:23:10.152 INFO:teuthology.orchestra.run.smithi143.stdout: "min": "", 2023-11-26T07:23:10.152 INFO:teuthology.orchestra.run.smithi143.stdout: "max": "", 2023-11-26T07:23:10.152 INFO:teuthology.orchestra.run.smithi143.stdout: "enum_allowed": [], 2023-11-26T07:23:10.152 INFO:teuthology.orchestra.run.smithi143.stdout: "desc": "", 2023-11-26T07:23:10.152 INFO:teuthology.orchestra.run.smithi143.stdout: "long_desc": "", 2023-11-26T07:23:10.152 INFO:teuthology.orchestra.run.smithi143.stdout: "tags": [], 2023-11-26T07:23:10.152 INFO:teuthology.orchestra.run.smithi143.stdout: "see_also": [] 2023-11-26T07:23:10.152 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2023-11-26T07:23:10.152 INFO:teuthology.orchestra.run.smithi143.stdout: "PWD_POLICY_MIN_LENGTH": { 2023-11-26T07:23:10.153 INFO:teuthology.orchestra.run.smithi143.stdout: "name": "PWD_POLICY_MIN_LENGTH", 2023-11-26T07:23:10.153 INFO:teuthology.orchestra.run.smithi143.stdout: "type": "int", 2023-11-26T07:23:10.153 INFO:teuthology.orchestra.run.smithi143.stdout: "level": "advanced", 2023-11-26T07:23:10.153 INFO:teuthology.orchestra.run.smithi143.stdout: "flags": 0, 2023-11-26T07:23:10.153 INFO:teuthology.orchestra.run.smithi143.stdout: "default_value": "8", 2023-11-26T07:23:10.153 INFO:teuthology.orchestra.run.smithi143.stdout: "min": "", 2023-11-26T07:23:10.153 INFO:teuthology.orchestra.run.smithi143.stdout: "max": "", 2023-11-26T07:23:10.153 INFO:teuthology.orchestra.run.smithi143.stdout: "enum_allowed": [], 2023-11-26T07:23:10.153 INFO:teuthology.orchestra.run.smithi143.stdout: "desc": "", 2023-11-26T07:23:10.153 INFO:teuthology.orchestra.run.smithi143.stdout: "long_desc": "", 2023-11-26T07:23:10.153 INFO:teuthology.orchestra.run.smithi143.stdout: "tags": [], 2023-11-26T07:23:10.153 INFO:teuthology.orchestra.run.smithi143.stdout: "see_also": [] 2023-11-26T07:23:10.153 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2023-11-26T07:23:10.154 INFO:teuthology.orchestra.run.smithi143.stdout: "REST_REQUESTS_TIMEOUT": { 2023-11-26T07:23:10.154 INFO:teuthology.orchestra.run.smithi143.stdout: "name": "REST_REQUESTS_TIMEOUT", 2023-11-26T07:23:10.154 INFO:teuthology.orchestra.run.smithi143.stdout: "type": "int", 2023-11-26T07:23:10.154 INFO:teuthology.orchestra.run.smithi143.stdout: "level": "advanced", 2023-11-26T07:23:10.154 INFO:teuthology.orchestra.run.smithi143.stdout: "flags": 0, 2023-11-26T07:23:10.154 INFO:teuthology.orchestra.run.smithi143.stdout: "default_value": "45", 2023-11-26T07:23:10.154 INFO:teuthology.orchestra.run.smithi143.stdout: "min": "", 2023-11-26T07:23:10.154 INFO:teuthology.orchestra.run.smithi143.stdout: "max": "", 2023-11-26T07:23:10.154 INFO:teuthology.orchestra.run.smithi143.stdout: "enum_allowed": [], 2023-11-26T07:23:10.154 INFO:teuthology.orchestra.run.smithi143.stdout: "desc": "", 2023-11-26T07:23:10.154 INFO:teuthology.orchestra.run.smithi143.stdout: "long_desc": "", 2023-11-26T07:23:10.154 INFO:teuthology.orchestra.run.smithi143.stdout: "tags": [], 2023-11-26T07:23:10.155 INFO:teuthology.orchestra.run.smithi143.stdout: "see_also": [] 2023-11-26T07:23:10.155 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2023-11-26T07:23:10.155 INFO:teuthology.orchestra.run.smithi143.stdout: "RGW_API_ACCESS_KEY": { 2023-11-26T07:23:10.155 INFO:teuthology.orchestra.run.smithi143.stdout: "name": "RGW_API_ACCESS_KEY", 2023-11-26T07:23:10.155 INFO:teuthology.orchestra.run.smithi143.stdout: "type": "str", 2023-11-26T07:23:10.155 INFO:teuthology.orchestra.run.smithi143.stdout: "level": "advanced", 2023-11-26T07:23:10.155 INFO:teuthology.orchestra.run.smithi143.stdout: "flags": 0, 2023-11-26T07:23:10.155 INFO:teuthology.orchestra.run.smithi143.stdout: "default_value": "", 2023-11-26T07:23:10.155 INFO:teuthology.orchestra.run.smithi143.stdout: "min": "", 2023-11-26T07:23:10.155 INFO:teuthology.orchestra.run.smithi143.stdout: "max": "", 2023-11-26T07:23:10.155 INFO:teuthology.orchestra.run.smithi143.stdout: "enum_allowed": [], 2023-11-26T07:23:10.155 INFO:teuthology.orchestra.run.smithi143.stdout: "desc": "", 2023-11-26T07:23:10.156 INFO:teuthology.orchestra.run.smithi143.stdout: "long_desc": "", 2023-11-26T07:23:10.156 INFO:teuthology.orchestra.run.smithi143.stdout: "tags": [], 2023-11-26T07:23:10.156 INFO:teuthology.orchestra.run.smithi143.stdout: "see_also": [] 2023-11-26T07:23:10.156 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2023-11-26T07:23:10.156 INFO:teuthology.orchestra.run.smithi143.stdout: "RGW_API_ADMIN_RESOURCE": { 2023-11-26T07:23:10.156 INFO:teuthology.orchestra.run.smithi143.stdout: "name": "RGW_API_ADMIN_RESOURCE", 2023-11-26T07:23:10.156 INFO:teuthology.orchestra.run.smithi143.stdout: "type": "str", 2023-11-26T07:23:10.156 INFO:teuthology.orchestra.run.smithi143.stdout: "level": "advanced", 2023-11-26T07:23:10.156 INFO:teuthology.orchestra.run.smithi143.stdout: "flags": 0, 2023-11-26T07:23:10.156 INFO:teuthology.orchestra.run.smithi143.stdout: "default_value": "admin", 2023-11-26T07:23:10.156 INFO:teuthology.orchestra.run.smithi143.stdout: "min": "", 2023-11-26T07:23:10.156 INFO:teuthology.orchestra.run.smithi143.stdout: "max": "", 2023-11-26T07:23:10.156 INFO:teuthology.orchestra.run.smithi143.stdout: "enum_allowed": [], 2023-11-26T07:23:10.157 INFO:teuthology.orchestra.run.smithi143.stdout: "desc": "", 2023-11-26T07:23:10.157 INFO:teuthology.orchestra.run.smithi143.stdout: "long_desc": "", 2023-11-26T07:23:10.157 INFO:teuthology.orchestra.run.smithi143.stdout: "tags": [], 2023-11-26T07:23:10.157 INFO:teuthology.orchestra.run.smithi143.stdout: "see_also": [] 2023-11-26T07:23:10.157 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2023-11-26T07:23:10.157 INFO:teuthology.orchestra.run.smithi143.stdout: "RGW_API_SECRET_KEY": { 2023-11-26T07:23:10.157 INFO:teuthology.orchestra.run.smithi143.stdout: "name": "RGW_API_SECRET_KEY", 2023-11-26T07:23:10.157 INFO:teuthology.orchestra.run.smithi143.stdout: "type": "str", 2023-11-26T07:23:10.157 INFO:teuthology.orchestra.run.smithi143.stdout: "level": "advanced", 2023-11-26T07:23:10.157 INFO:teuthology.orchestra.run.smithi143.stdout: "flags": 0, 2023-11-26T07:23:10.157 INFO:teuthology.orchestra.run.smithi143.stdout: "default_value": "", 2023-11-26T07:23:10.157 INFO:teuthology.orchestra.run.smithi143.stdout: "min": "", 2023-11-26T07:23:10.158 INFO:teuthology.orchestra.run.smithi143.stdout: "max": "", 2023-11-26T07:23:10.158 INFO:teuthology.orchestra.run.smithi143.stdout: "enum_allowed": [], 2023-11-26T07:23:10.158 INFO:teuthology.orchestra.run.smithi143.stdout: "desc": "", 2023-11-26T07:23:10.158 INFO:teuthology.orchestra.run.smithi143.stdout: "long_desc": "", 2023-11-26T07:23:10.158 INFO:teuthology.orchestra.run.smithi143.stdout: "tags": [], 2023-11-26T07:23:10.158 INFO:teuthology.orchestra.run.smithi143.stdout: "see_also": [] 2023-11-26T07:23:10.158 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2023-11-26T07:23:10.158 INFO:teuthology.orchestra.run.smithi143.stdout: "RGW_API_SSL_VERIFY": { 2023-11-26T07:23:10.158 INFO:teuthology.orchestra.run.smithi143.stdout: "name": "RGW_API_SSL_VERIFY", 2023-11-26T07:23:10.158 INFO:teuthology.orchestra.run.smithi143.stdout: "type": "bool", 2023-11-26T07:23:10.158 INFO:teuthology.orchestra.run.smithi143.stdout: "level": "advanced", 2023-11-26T07:23:10.158 INFO:teuthology.orchestra.run.smithi143.stdout: "flags": 0, 2023-11-26T07:23:10.159 INFO:teuthology.orchestra.run.smithi143.stdout: "default_value": "True", 2023-11-26T07:23:10.159 INFO:teuthology.orchestra.run.smithi143.stdout: "min": "", 2023-11-26T07:23:10.159 INFO:teuthology.orchestra.run.smithi143.stdout: "max": "", 2023-11-26T07:23:10.159 INFO:teuthology.orchestra.run.smithi143.stdout: "enum_allowed": [], 2023-11-26T07:23:10.159 INFO:teuthology.orchestra.run.smithi143.stdout: "desc": "", 2023-11-26T07:23:10.159 INFO:teuthology.orchestra.run.smithi143.stdout: "long_desc": "", 2023-11-26T07:23:10.159 INFO:teuthology.orchestra.run.smithi143.stdout: "tags": [], 2023-11-26T07:23:10.159 INFO:teuthology.orchestra.run.smithi143.stdout: "see_also": [] 2023-11-26T07:23:10.159 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2023-11-26T07:23:10.159 INFO:teuthology.orchestra.run.smithi143.stdout: "UNSAFE_TLS_v1_2": { 2023-11-26T07:23:10.159 INFO:teuthology.orchestra.run.smithi143.stdout: "name": "UNSAFE_TLS_v1_2", 2023-11-26T07:23:10.159 INFO:teuthology.orchestra.run.smithi143.stdout: "type": "bool", 2023-11-26T07:23:10.159 INFO:teuthology.orchestra.run.smithi143.stdout: "level": "advanced", 2023-11-26T07:23:10.160 INFO:teuthology.orchestra.run.smithi143.stdout: "flags": 0, 2023-11-26T07:23:10.160 INFO:teuthology.orchestra.run.smithi143.stdout: "default_value": "False", 2023-11-26T07:23:10.160 INFO:teuthology.orchestra.run.smithi143.stdout: "min": "", 2023-11-26T07:23:10.160 INFO:teuthology.orchestra.run.smithi143.stdout: "max": "", 2023-11-26T07:23:10.160 INFO:teuthology.orchestra.run.smithi143.stdout: "enum_allowed": [], 2023-11-26T07:23:10.160 INFO:teuthology.orchestra.run.smithi143.stdout: "desc": "", 2023-11-26T07:23:10.160 INFO:teuthology.orchestra.run.smithi143.stdout: "long_desc": "", 2023-11-26T07:23:10.160 INFO:teuthology.orchestra.run.smithi143.stdout: "tags": [], 2023-11-26T07:23:10.160 INFO:teuthology.orchestra.run.smithi143.stdout: "see_also": [] 2023-11-26T07:23:10.160 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2023-11-26T07:23:10.160 INFO:teuthology.orchestra.run.smithi143.stdout: "USER_PWD_EXPIRATION_SPAN": { 2023-11-26T07:23:10.160 INFO:teuthology.orchestra.run.smithi143.stdout: "name": "USER_PWD_EXPIRATION_SPAN", 2023-11-26T07:23:10.160 INFO:teuthology.orchestra.run.smithi143.stdout: "type": "int", 2023-11-26T07:23:10.161 INFO:teuthology.orchestra.run.smithi143.stdout: "level": "advanced", 2023-11-26T07:23:10.161 INFO:teuthology.orchestra.run.smithi143.stdout: "flags": 0, 2023-11-26T07:23:10.161 INFO:teuthology.orchestra.run.smithi143.stdout: "default_value": "0", 2023-11-26T07:23:10.161 INFO:teuthology.orchestra.run.smithi143.stdout: "min": "", 2023-11-26T07:23:10.161 INFO:teuthology.orchestra.run.smithi143.stdout: "max": "", 2023-11-26T07:23:10.161 INFO:teuthology.orchestra.run.smithi143.stdout: "enum_allowed": [], 2023-11-26T07:23:10.161 INFO:teuthology.orchestra.run.smithi143.stdout: "desc": "", 2023-11-26T07:23:10.161 INFO:teuthology.orchestra.run.smithi143.stdout: "long_desc": "", 2023-11-26T07:23:10.161 INFO:teuthology.orchestra.run.smithi143.stdout: "tags": [], 2023-11-26T07:23:10.161 INFO:teuthology.orchestra.run.smithi143.stdout: "see_also": [] 2023-11-26T07:23:10.161 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2023-11-26T07:23:10.161 INFO:teuthology.orchestra.run.smithi143.stdout: "USER_PWD_EXPIRATION_WARNING_1": { 2023-11-26T07:23:10.162 INFO:teuthology.orchestra.run.smithi143.stdout: "name": "USER_PWD_EXPIRATION_WARNING_1", 2023-11-26T07:23:10.162 INFO:teuthology.orchestra.run.smithi143.stdout: "type": "int", 2023-11-26T07:23:10.162 INFO:teuthology.orchestra.run.smithi143.stdout: "level": "advanced", 2023-11-26T07:23:10.162 INFO:teuthology.orchestra.run.smithi143.stdout: "flags": 0, 2023-11-26T07:23:10.162 INFO:teuthology.orchestra.run.smithi143.stdout: "default_value": "10", 2023-11-26T07:23:10.162 INFO:teuthology.orchestra.run.smithi143.stdout: "min": "", 2023-11-26T07:23:10.162 INFO:teuthology.orchestra.run.smithi143.stdout: "max": "", 2023-11-26T07:23:10.162 INFO:teuthology.orchestra.run.smithi143.stdout: "enum_allowed": [], 2023-11-26T07:23:10.162 INFO:teuthology.orchestra.run.smithi143.stdout: "desc": "", 2023-11-26T07:23:10.162 INFO:teuthology.orchestra.run.smithi143.stdout: "long_desc": "", 2023-11-26T07:23:10.162 INFO:teuthology.orchestra.run.smithi143.stdout: "tags": [], 2023-11-26T07:23:10.162 INFO:teuthology.orchestra.run.smithi143.stdout: "see_also": [] 2023-11-26T07:23:10.162 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2023-11-26T07:23:10.163 INFO:teuthology.orchestra.run.smithi143.stdout: "USER_PWD_EXPIRATION_WARNING_2": { 2023-11-26T07:23:10.163 INFO:teuthology.orchestra.run.smithi143.stdout: "name": "USER_PWD_EXPIRATION_WARNING_2", 2023-11-26T07:23:10.163 INFO:teuthology.orchestra.run.smithi143.stdout: "type": "int", 2023-11-26T07:23:10.163 INFO:teuthology.orchestra.run.smithi143.stdout: "level": "advanced", 2023-11-26T07:23:10.163 INFO:teuthology.orchestra.run.smithi143.stdout: "flags": 0, 2023-11-26T07:23:10.163 INFO:teuthology.orchestra.run.smithi143.stdout: "default_value": "5", 2023-11-26T07:23:10.163 INFO:teuthology.orchestra.run.smithi143.stdout: "min": "", 2023-11-26T07:23:10.163 INFO:teuthology.orchestra.run.smithi143.stdout: "max": "", 2023-11-26T07:23:10.163 INFO:teuthology.orchestra.run.smithi143.stdout: "enum_allowed": [], 2023-11-26T07:23:10.163 INFO:teuthology.orchestra.run.smithi143.stdout: "desc": "", 2023-11-26T07:23:10.163 INFO:teuthology.orchestra.run.smithi143.stdout: "long_desc": "", 2023-11-26T07:23:10.163 INFO:teuthology.orchestra.run.smithi143.stdout: "tags": [], 2023-11-26T07:23:10.163 INFO:teuthology.orchestra.run.smithi143.stdout: "see_also": [] 2023-11-26T07:23:10.164 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2023-11-26T07:23:10.164 INFO:teuthology.orchestra.run.smithi143.stdout: "cross_origin_url": { 2023-11-26T07:23:10.164 INFO:teuthology.orchestra.run.smithi143.stdout: "name": "cross_origin_url", 2023-11-26T07:23:10.164 INFO:teuthology.orchestra.run.smithi143.stdout: "type": "str", 2023-11-26T07:23:10.164 INFO:teuthology.orchestra.run.smithi143.stdout: "level": "advanced", 2023-11-26T07:23:10.164 INFO:teuthology.orchestra.run.smithi143.stdout: "flags": 0, 2023-11-26T07:23:10.164 INFO:teuthology.orchestra.run.smithi143.stdout: "default_value": "", 2023-11-26T07:23:10.164 INFO:teuthology.orchestra.run.smithi143.stdout: "min": "", 2023-11-26T07:23:10.164 INFO:teuthology.orchestra.run.smithi143.stdout: "max": "", 2023-11-26T07:23:10.164 INFO:teuthology.orchestra.run.smithi143.stdout: "enum_allowed": [], 2023-11-26T07:23:10.164 INFO:teuthology.orchestra.run.smithi143.stdout: "desc": "", 2023-11-26T07:23:10.164 INFO:teuthology.orchestra.run.smithi143.stdout: "long_desc": "", 2023-11-26T07:23:10.165 INFO:teuthology.orchestra.run.smithi143.stdout: "tags": [], 2023-11-26T07:23:10.165 INFO:teuthology.orchestra.run.smithi143.stdout: "see_also": [] 2023-11-26T07:23:10.165 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2023-11-26T07:23:10.165 INFO:teuthology.orchestra.run.smithi143.stdout: "crt_file": { 2023-11-26T07:23:10.165 INFO:teuthology.orchestra.run.smithi143.stdout: "name": "crt_file", 2023-11-26T07:23:10.165 INFO:teuthology.orchestra.run.smithi143.stdout: "type": "str", 2023-11-26T07:23:10.165 INFO:teuthology.orchestra.run.smithi143.stdout: "level": "advanced", 2023-11-26T07:23:10.165 INFO:teuthology.orchestra.run.smithi143.stdout: "flags": 0, 2023-11-26T07:23:10.165 INFO:teuthology.orchestra.run.smithi143.stdout: "default_value": "", 2023-11-26T07:23:10.165 INFO:teuthology.orchestra.run.smithi143.stdout: "min": "", 2023-11-26T07:23:10.165 INFO:teuthology.orchestra.run.smithi143.stdout: "max": "", 2023-11-26T07:23:10.165 INFO:teuthology.orchestra.run.smithi143.stdout: "enum_allowed": [], 2023-11-26T07:23:10.165 INFO:teuthology.orchestra.run.smithi143.stdout: "desc": "", 2023-11-26T07:23:10.165 INFO:teuthology.orchestra.run.smithi143.stdout: "long_desc": "", 2023-11-26T07:23:10.166 INFO:teuthology.orchestra.run.smithi143.stdout: "tags": [], 2023-11-26T07:23:10.166 INFO:teuthology.orchestra.run.smithi143.stdout: "see_also": [] 2023-11-26T07:23:10.166 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2023-11-26T07:23:10.166 INFO:teuthology.orchestra.run.smithi143.stdout: "debug": { 2023-11-26T07:23:10.166 INFO:teuthology.orchestra.run.smithi143.stdout: "name": "debug", 2023-11-26T07:23:10.166 INFO:teuthology.orchestra.run.smithi143.stdout: "type": "bool", 2023-11-26T07:23:10.166 INFO:teuthology.orchestra.run.smithi143.stdout: "level": "advanced", 2023-11-26T07:23:10.166 INFO:teuthology.orchestra.run.smithi143.stdout: "flags": 0, 2023-11-26T07:23:10.166 INFO:teuthology.orchestra.run.smithi143.stdout: "default_value": "False", 2023-11-26T07:23:10.166 INFO:teuthology.orchestra.run.smithi143.stdout: "min": "", 2023-11-26T07:23:10.166 INFO:teuthology.orchestra.run.smithi143.stdout: "max": "", 2023-11-26T07:23:10.166 INFO:teuthology.orchestra.run.smithi143.stdout: "enum_allowed": [], 2023-11-26T07:23:10.167 INFO:teuthology.orchestra.run.smithi143.stdout: "desc": "Enable/disable debug options", 2023-11-26T07:23:10.167 INFO:teuthology.orchestra.run.smithi143.stdout: "long_desc": "", 2023-11-26T07:23:10.167 INFO:teuthology.orchestra.run.smithi143.stdout: "tags": [], 2023-11-26T07:23:10.167 INFO:teuthology.orchestra.run.smithi143.stdout: "see_also": [] 2023-11-26T07:23:10.167 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2023-11-26T07:23:10.167 INFO:teuthology.orchestra.run.smithi143.stdout: "jwt_token_ttl": { 2023-11-26T07:23:10.167 INFO:teuthology.orchestra.run.smithi143.stdout: "name": "jwt_token_ttl", 2023-11-26T07:23:10.167 INFO:teuthology.orchestra.run.smithi143.stdout: "type": "int", 2023-11-26T07:23:10.167 INFO:teuthology.orchestra.run.smithi143.stdout: "level": "advanced", 2023-11-26T07:23:10.167 INFO:teuthology.orchestra.run.smithi143.stdout: "flags": 0, 2023-11-26T07:23:10.167 INFO:teuthology.orchestra.run.smithi143.stdout: "default_value": "28800", 2023-11-26T07:23:10.167 INFO:teuthology.orchestra.run.smithi143.stdout: "min": "", 2023-11-26T07:23:10.167 INFO:teuthology.orchestra.run.smithi143.stdout: "max": "", 2023-11-26T07:23:10.168 INFO:teuthology.orchestra.run.smithi143.stdout: "enum_allowed": [], 2023-11-26T07:23:10.168 INFO:teuthology.orchestra.run.smithi143.stdout: "desc": "", 2023-11-26T07:23:10.168 INFO:teuthology.orchestra.run.smithi143.stdout: "long_desc": "", 2023-11-26T07:23:10.168 INFO:teuthology.orchestra.run.smithi143.stdout: "tags": [], 2023-11-26T07:23:10.168 INFO:teuthology.orchestra.run.smithi143.stdout: "see_also": [] 2023-11-26T07:23:10.168 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2023-11-26T07:23:10.168 INFO:teuthology.orchestra.run.smithi143.stdout: "key_file": { 2023-11-26T07:23:10.168 INFO:teuthology.orchestra.run.smithi143.stdout: "name": "key_file", 2023-11-26T07:23:10.168 INFO:teuthology.orchestra.run.smithi143.stdout: "type": "str", 2023-11-26T07:23:10.168 INFO:teuthology.orchestra.run.smithi143.stdout: "level": "advanced", 2023-11-26T07:23:10.168 INFO:teuthology.orchestra.run.smithi143.stdout: "flags": 0, 2023-11-26T07:23:10.168 INFO:teuthology.orchestra.run.smithi143.stdout: "default_value": "", 2023-11-26T07:23:10.168 INFO:teuthology.orchestra.run.smithi143.stdout: "min": "", 2023-11-26T07:23:10.169 INFO:teuthology.orchestra.run.smithi143.stdout: "max": "", 2023-11-26T07:23:10.169 INFO:teuthology.orchestra.run.smithi143.stdout: "enum_allowed": [], 2023-11-26T07:23:10.169 INFO:teuthology.orchestra.run.smithi143.stdout: "desc": "", 2023-11-26T07:23:10.169 INFO:teuthology.orchestra.run.smithi143.stdout: "long_desc": "", 2023-11-26T07:23:10.169 INFO:teuthology.orchestra.run.smithi143.stdout: "tags": [], 2023-11-26T07:23:10.169 INFO:teuthology.orchestra.run.smithi143.stdout: "see_also": [] 2023-11-26T07:23:10.169 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2023-11-26T07:23:10.169 INFO:teuthology.orchestra.run.smithi143.stdout: "log_level": { 2023-11-26T07:23:10.169 INFO:teuthology.orchestra.run.smithi143.stdout: "name": "log_level", 2023-11-26T07:23:10.169 INFO:teuthology.orchestra.run.smithi143.stdout: "type": "str", 2023-11-26T07:23:10.169 INFO:teuthology.orchestra.run.smithi143.stdout: "level": "advanced", 2023-11-26T07:23:10.169 INFO:teuthology.orchestra.run.smithi143.stdout: "flags": 1, 2023-11-26T07:23:10.169 INFO:teuthology.orchestra.run.smithi143.stdout: "default_value": "", 2023-11-26T07:23:10.170 INFO:teuthology.orchestra.run.smithi143.stdout: "min": "", 2023-11-26T07:23:10.170 INFO:teuthology.orchestra.run.smithi143.stdout: "max": "", 2023-11-26T07:23:10.170 INFO:teuthology.orchestra.run.smithi143.stdout: "enum_allowed": [ 2023-11-26T07:23:10.170 INFO:teuthology.orchestra.run.smithi143.stdout: "", 2023-11-26T07:23:10.170 INFO:teuthology.orchestra.run.smithi143.stdout: "critical", 2023-11-26T07:23:10.170 INFO:teuthology.orchestra.run.smithi143.stdout: "debug", 2023-11-26T07:23:10.170 INFO:teuthology.orchestra.run.smithi143.stdout: "error", 2023-11-26T07:23:10.170 INFO:teuthology.orchestra.run.smithi143.stdout: "info", 2023-11-26T07:23:10.170 INFO:teuthology.orchestra.run.smithi143.stdout: "warning" 2023-11-26T07:23:10.170 INFO:teuthology.orchestra.run.smithi143.stdout: ], 2023-11-26T07:23:10.170 INFO:teuthology.orchestra.run.smithi143.stdout: "desc": "", 2023-11-26T07:23:10.170 INFO:teuthology.orchestra.run.smithi143.stdout: "long_desc": "", 2023-11-26T07:23:10.171 INFO:teuthology.orchestra.run.smithi143.stdout: "tags": [], 2023-11-26T07:23:10.171 INFO:teuthology.orchestra.run.smithi143.stdout: "see_also": [] 2023-11-26T07:23:10.171 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2023-11-26T07:23:10.171 INFO:teuthology.orchestra.run.smithi143.stdout: "log_to_cluster": { 2023-11-26T07:23:10.171 INFO:teuthology.orchestra.run.smithi143.stdout: "name": "log_to_cluster", 2023-11-26T07:23:10.171 INFO:teuthology.orchestra.run.smithi143.stdout: "type": "bool", 2023-11-26T07:23:10.171 INFO:teuthology.orchestra.run.smithi143.stdout: "level": "advanced", 2023-11-26T07:23:10.171 INFO:teuthology.orchestra.run.smithi143.stdout: "flags": 1, 2023-11-26T07:23:10.171 INFO:teuthology.orchestra.run.smithi143.stdout: "default_value": "False", 2023-11-26T07:23:10.171 INFO:teuthology.orchestra.run.smithi143.stdout: "min": "", 2023-11-26T07:23:10.171 INFO:teuthology.orchestra.run.smithi143.stdout: "max": "", 2023-11-26T07:23:10.171 INFO:teuthology.orchestra.run.smithi143.stdout: "enum_allowed": [], 2023-11-26T07:23:10.171 INFO:teuthology.orchestra.run.smithi143.stdout: "desc": "", 2023-11-26T07:23:10.172 INFO:teuthology.orchestra.run.smithi143.stdout: "long_desc": "", 2023-11-26T07:23:10.172 INFO:teuthology.orchestra.run.smithi143.stdout: "tags": [], 2023-11-26T07:23:10.172 INFO:teuthology.orchestra.run.smithi143.stdout: "see_also": [] 2023-11-26T07:23:10.172 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2023-11-26T07:23:10.172 INFO:teuthology.orchestra.run.smithi143.stdout: "log_to_cluster_level": { 2023-11-26T07:23:10.172 INFO:teuthology.orchestra.run.smithi143.stdout: "name": "log_to_cluster_level", 2023-11-26T07:23:10.172 INFO:teuthology.orchestra.run.smithi143.stdout: "type": "str", 2023-11-26T07:23:10.172 INFO:teuthology.orchestra.run.smithi143.stdout: "level": "advanced", 2023-11-26T07:23:10.172 INFO:teuthology.orchestra.run.smithi143.stdout: "flags": 1, 2023-11-26T07:23:10.172 INFO:teuthology.orchestra.run.smithi143.stdout: "default_value": "info", 2023-11-26T07:23:10.172 INFO:teuthology.orchestra.run.smithi143.stdout: "min": "", 2023-11-26T07:23:10.172 INFO:teuthology.orchestra.run.smithi143.stdout: "max": "", 2023-11-26T07:23:10.172 INFO:teuthology.orchestra.run.smithi143.stdout: "enum_allowed": [ 2023-11-26T07:23:10.173 INFO:teuthology.orchestra.run.smithi143.stdout: "", 2023-11-26T07:23:10.173 INFO:teuthology.orchestra.run.smithi143.stdout: "critical", 2023-11-26T07:23:10.173 INFO:teuthology.orchestra.run.smithi143.stdout: "debug", 2023-11-26T07:23:10.173 INFO:teuthology.orchestra.run.smithi143.stdout: "error", 2023-11-26T07:23:10.173 INFO:teuthology.orchestra.run.smithi143.stdout: "info", 2023-11-26T07:23:10.173 INFO:teuthology.orchestra.run.smithi143.stdout: "warning" 2023-11-26T07:23:10.173 INFO:teuthology.orchestra.run.smithi143.stdout: ], 2023-11-26T07:23:10.173 INFO:teuthology.orchestra.run.smithi143.stdout: "desc": "", 2023-11-26T07:23:10.173 INFO:teuthology.orchestra.run.smithi143.stdout: "long_desc": "", 2023-11-26T07:23:10.173 INFO:teuthology.orchestra.run.smithi143.stdout: "tags": [], 2023-11-26T07:23:10.173 INFO:teuthology.orchestra.run.smithi143.stdout: "see_also": [] 2023-11-26T07:23:10.173 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2023-11-26T07:23:10.173 INFO:teuthology.orchestra.run.smithi143.stdout: "log_to_file": { 2023-11-26T07:23:10.174 INFO:teuthology.orchestra.run.smithi143.stdout: "name": "log_to_file", 2023-11-26T07:23:10.174 INFO:teuthology.orchestra.run.smithi143.stdout: "type": "bool", 2023-11-26T07:23:10.174 INFO:teuthology.orchestra.run.smithi143.stdout: "level": "advanced", 2023-11-26T07:23:10.174 INFO:teuthology.orchestra.run.smithi143.stdout: "flags": 1, 2023-11-26T07:23:10.174 INFO:teuthology.orchestra.run.smithi143.stdout: "default_value": "False", 2023-11-26T07:23:10.174 INFO:teuthology.orchestra.run.smithi143.stdout: "min": "", 2023-11-26T07:23:10.174 INFO:teuthology.orchestra.run.smithi143.stdout: "max": "", 2023-11-26T07:23:10.174 INFO:teuthology.orchestra.run.smithi143.stdout: "enum_allowed": [], 2023-11-26T07:23:10.174 INFO:teuthology.orchestra.run.smithi143.stdout: "desc": "", 2023-11-26T07:23:10.174 INFO:teuthology.orchestra.run.smithi143.stdout: "long_desc": "", 2023-11-26T07:23:10.174 INFO:teuthology.orchestra.run.smithi143.stdout: "tags": [], 2023-11-26T07:23:10.174 INFO:teuthology.orchestra.run.smithi143.stdout: "see_also": [] 2023-11-26T07:23:10.175 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2023-11-26T07:23:10.175 INFO:teuthology.orchestra.run.smithi143.stdout: "motd": { 2023-11-26T07:23:10.175 INFO:teuthology.orchestra.run.smithi143.stdout: "name": "motd", 2023-11-26T07:23:10.175 INFO:teuthology.orchestra.run.smithi143.stdout: "type": "str", 2023-11-26T07:23:10.175 INFO:teuthology.orchestra.run.smithi143.stdout: "level": "advanced", 2023-11-26T07:23:10.175 INFO:teuthology.orchestra.run.smithi143.stdout: "flags": 0, 2023-11-26T07:23:10.175 INFO:teuthology.orchestra.run.smithi143.stdout: "default_value": "", 2023-11-26T07:23:10.175 INFO:teuthology.orchestra.run.smithi143.stdout: "min": "", 2023-11-26T07:23:10.175 INFO:teuthology.orchestra.run.smithi143.stdout: "max": "", 2023-11-26T07:23:10.175 INFO:teuthology.orchestra.run.smithi143.stdout: "enum_allowed": [], 2023-11-26T07:23:10.175 INFO:teuthology.orchestra.run.smithi143.stdout: "desc": "The message of the day", 2023-11-26T07:23:10.175 INFO:teuthology.orchestra.run.smithi143.stdout: "long_desc": "", 2023-11-26T07:23:10.175 INFO:teuthology.orchestra.run.smithi143.stdout: "tags": [], 2023-11-26T07:23:10.176 INFO:teuthology.orchestra.run.smithi143.stdout: "see_also": [] 2023-11-26T07:23:10.176 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2023-11-26T07:23:10.176 INFO:teuthology.orchestra.run.smithi143.stdout: "redirect_resolve_ip_addr": { 2023-11-26T07:23:10.176 INFO:teuthology.orchestra.run.smithi143.stdout: "name": "redirect_resolve_ip_addr", 2023-11-26T07:23:10.176 INFO:teuthology.orchestra.run.smithi143.stdout: "type": "bool", 2023-11-26T07:23:10.176 INFO:teuthology.orchestra.run.smithi143.stdout: "level": "advanced", 2023-11-26T07:23:10.176 INFO:teuthology.orchestra.run.smithi143.stdout: "flags": 0, 2023-11-26T07:23:10.176 INFO:teuthology.orchestra.run.smithi143.stdout: "default_value": "False", 2023-11-26T07:23:10.176 INFO:teuthology.orchestra.run.smithi143.stdout: "min": "", 2023-11-26T07:23:10.176 INFO:teuthology.orchestra.run.smithi143.stdout: "max": "", 2023-11-26T07:23:10.176 INFO:teuthology.orchestra.run.smithi143.stdout: "enum_allowed": [], 2023-11-26T07:23:10.176 INFO:teuthology.orchestra.run.smithi143.stdout: "desc": "", 2023-11-26T07:23:10.176 INFO:teuthology.orchestra.run.smithi143.stdout: "long_desc": "", 2023-11-26T07:23:10.177 INFO:teuthology.orchestra.run.smithi143.stdout: "tags": [], 2023-11-26T07:23:10.177 INFO:teuthology.orchestra.run.smithi143.stdout: "see_also": [] 2023-11-26T07:23:10.177 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2023-11-26T07:23:10.177 INFO:teuthology.orchestra.run.smithi143.stdout: "server_addr": { 2023-11-26T07:23:10.177 INFO:teuthology.orchestra.run.smithi143.stdout: "name": "server_addr", 2023-11-26T07:23:10.177 INFO:teuthology.orchestra.run.smithi143.stdout: "type": "str", 2023-11-26T07:23:10.177 INFO:teuthology.orchestra.run.smithi143.stdout: "level": "advanced", 2023-11-26T07:23:10.177 INFO:teuthology.orchestra.run.smithi143.stdout: "flags": 0, 2023-11-26T07:23:10.177 INFO:teuthology.orchestra.run.smithi143.stdout: "default_value": "::", 2023-11-26T07:23:10.177 INFO:teuthology.orchestra.run.smithi143.stdout: "min": "", 2023-11-26T07:23:10.177 INFO:teuthology.orchestra.run.smithi143.stdout: "max": "", 2023-11-26T07:23:10.177 INFO:teuthology.orchestra.run.smithi143.stdout: "enum_allowed": [], 2023-11-26T07:23:10.177 INFO:teuthology.orchestra.run.smithi143.stdout: "desc": "", 2023-11-26T07:23:10.178 INFO:teuthology.orchestra.run.smithi143.stdout: "long_desc": "", 2023-11-26T07:23:10.178 INFO:teuthology.orchestra.run.smithi143.stdout: "tags": [], 2023-11-26T07:23:10.178 INFO:teuthology.orchestra.run.smithi143.stdout: "see_also": [] 2023-11-26T07:23:10.178 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2023-11-26T07:23:10.178 INFO:teuthology.orchestra.run.smithi143.stdout: "server_port": { 2023-11-26T07:23:10.178 INFO:teuthology.orchestra.run.smithi143.stdout: "name": "server_port", 2023-11-26T07:23:10.178 INFO:teuthology.orchestra.run.smithi143.stdout: "type": "int", 2023-11-26T07:23:10.178 INFO:teuthology.orchestra.run.smithi143.stdout: "level": "advanced", 2023-11-26T07:23:10.178 INFO:teuthology.orchestra.run.smithi143.stdout: "flags": 0, 2023-11-26T07:23:10.178 INFO:teuthology.orchestra.run.smithi143.stdout: "default_value": "8080", 2023-11-26T07:23:10.178 INFO:teuthology.orchestra.run.smithi143.stdout: "min": "", 2023-11-26T07:23:10.178 INFO:teuthology.orchestra.run.smithi143.stdout: "max": "", 2023-11-26T07:23:10.179 INFO:teuthology.orchestra.run.smithi143.stdout: "enum_allowed": [], 2023-11-26T07:23:10.179 INFO:teuthology.orchestra.run.smithi143.stdout: "desc": "", 2023-11-26T07:23:10.179 INFO:teuthology.orchestra.run.smithi143.stdout: "long_desc": "", 2023-11-26T07:23:10.179 INFO:teuthology.orchestra.run.smithi143.stdout: "tags": [], 2023-11-26T07:23:10.179 INFO:teuthology.orchestra.run.smithi143.stdout: "see_also": [] 2023-11-26T07:23:10.179 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2023-11-26T07:23:10.179 INFO:teuthology.orchestra.run.smithi143.stdout: "ssl": { 2023-11-26T07:23:10.179 INFO:teuthology.orchestra.run.smithi143.stdout: "name": "ssl", 2023-11-26T07:23:10.179 INFO:teuthology.orchestra.run.smithi143.stdout: "type": "bool", 2023-11-26T07:23:10.179 INFO:teuthology.orchestra.run.smithi143.stdout: "level": "advanced", 2023-11-26T07:23:10.179 INFO:teuthology.orchestra.run.smithi143.stdout: "flags": 0, 2023-11-26T07:23:10.179 INFO:teuthology.orchestra.run.smithi143.stdout: "default_value": "True", 2023-11-26T07:23:10.179 INFO:teuthology.orchestra.run.smithi143.stdout: "min": "", 2023-11-26T07:23:10.180 INFO:teuthology.orchestra.run.smithi143.stdout: "max": "", 2023-11-26T07:23:10.180 INFO:teuthology.orchestra.run.smithi143.stdout: "enum_allowed": [], 2023-11-26T07:23:10.180 INFO:teuthology.orchestra.run.smithi143.stdout: "desc": "", 2023-11-26T07:23:10.180 INFO:teuthology.orchestra.run.smithi143.stdout: "long_desc": "", 2023-11-26T07:23:10.180 INFO:teuthology.orchestra.run.smithi143.stdout: "tags": [], 2023-11-26T07:23:10.180 INFO:teuthology.orchestra.run.smithi143.stdout: "see_also": [] 2023-11-26T07:23:10.180 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2023-11-26T07:23:10.180 INFO:teuthology.orchestra.run.smithi143.stdout: "ssl_server_port": { 2023-11-26T07:23:10.180 INFO:teuthology.orchestra.run.smithi143.stdout: "name": "ssl_server_port", 2023-11-26T07:23:10.180 INFO:teuthology.orchestra.run.smithi143.stdout: "type": "int", 2023-11-26T07:23:10.180 INFO:teuthology.orchestra.run.smithi143.stdout: "level": "advanced", 2023-11-26T07:23:10.180 INFO:teuthology.orchestra.run.smithi143.stdout: "flags": 0, 2023-11-26T07:23:10.180 INFO:teuthology.orchestra.run.smithi143.stdout: "default_value": "8443", 2023-11-26T07:23:10.181 INFO:teuthology.orchestra.run.smithi143.stdout: "min": "", 2023-11-26T07:23:10.181 INFO:teuthology.orchestra.run.smithi143.stdout: "max": "", 2023-11-26T07:23:10.181 INFO:teuthology.orchestra.run.smithi143.stdout: "enum_allowed": [], 2023-11-26T07:23:10.181 INFO:teuthology.orchestra.run.smithi143.stdout: "desc": "", 2023-11-26T07:23:10.181 INFO:teuthology.orchestra.run.smithi143.stdout: "long_desc": "", 2023-11-26T07:23:10.181 INFO:teuthology.orchestra.run.smithi143.stdout: "tags": [], 2023-11-26T07:23:10.181 INFO:teuthology.orchestra.run.smithi143.stdout: "see_also": [] 2023-11-26T07:23:10.181 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2023-11-26T07:23:10.181 INFO:teuthology.orchestra.run.smithi143.stdout: "standby_behaviour": { 2023-11-26T07:23:10.181 INFO:teuthology.orchestra.run.smithi143.stdout: "name": "standby_behaviour", 2023-11-26T07:23:10.181 INFO:teuthology.orchestra.run.smithi143.stdout: "type": "str", 2023-11-26T07:23:10.181 INFO:teuthology.orchestra.run.smithi143.stdout: "level": "advanced", 2023-11-26T07:23:10.181 INFO:teuthology.orchestra.run.smithi143.stdout: "flags": 0, 2023-11-26T07:23:10.182 INFO:teuthology.orchestra.run.smithi143.stdout: "default_value": "redirect", 2023-11-26T07:23:10.182 INFO:teuthology.orchestra.run.smithi143.stdout: "min": "", 2023-11-26T07:23:10.182 INFO:teuthology.orchestra.run.smithi143.stdout: "max": "", 2023-11-26T07:23:10.182 INFO:teuthology.orchestra.run.smithi143.stdout: "enum_allowed": [ 2023-11-26T07:23:10.182 INFO:teuthology.orchestra.run.smithi143.stdout: "error", 2023-11-26T07:23:10.182 INFO:teuthology.orchestra.run.smithi143.stdout: "redirect" 2023-11-26T07:23:10.182 INFO:teuthology.orchestra.run.smithi143.stdout: ], 2023-11-26T07:23:10.182 INFO:teuthology.orchestra.run.smithi143.stdout: "desc": "", 2023-11-26T07:23:10.182 INFO:teuthology.orchestra.run.smithi143.stdout: "long_desc": "", 2023-11-26T07:23:10.182 INFO:teuthology.orchestra.run.smithi143.stdout: "tags": [], 2023-11-26T07:23:10.182 INFO:teuthology.orchestra.run.smithi143.stdout: "see_also": [] 2023-11-26T07:23:10.182 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2023-11-26T07:23:10.183 INFO:teuthology.orchestra.run.smithi143.stdout: "standby_error_status_code": { 2023-11-26T07:23:10.183 INFO:teuthology.orchestra.run.smithi143.stdout: "name": "standby_error_status_code", 2023-11-26T07:23:10.183 INFO:teuthology.orchestra.run.smithi143.stdout: "type": "int", 2023-11-26T07:23:10.183 INFO:teuthology.orchestra.run.smithi143.stdout: "level": "advanced", 2023-11-26T07:23:10.183 INFO:teuthology.orchestra.run.smithi143.stdout: "flags": 0, 2023-11-26T07:23:10.183 INFO:teuthology.orchestra.run.smithi143.stdout: "default_value": "500", 2023-11-26T07:23:10.183 INFO:teuthology.orchestra.run.smithi143.stdout: "min": "400", 2023-11-26T07:23:10.183 INFO:teuthology.orchestra.run.smithi143.stdout: "max": "599", 2023-11-26T07:23:10.183 INFO:teuthology.orchestra.run.smithi143.stdout: "enum_allowed": [], 2023-11-26T07:23:10.183 INFO:teuthology.orchestra.run.smithi143.stdout: "desc": "", 2023-11-26T07:23:10.183 INFO:teuthology.orchestra.run.smithi143.stdout: "long_desc": "", 2023-11-26T07:23:10.183 INFO:teuthology.orchestra.run.smithi143.stdout: "tags": [], 2023-11-26T07:23:10.183 INFO:teuthology.orchestra.run.smithi143.stdout: "see_also": [] 2023-11-26T07:23:10.184 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2023-11-26T07:23:10.184 INFO:teuthology.orchestra.run.smithi143.stdout: "url_prefix": { 2023-11-26T07:23:10.184 INFO:teuthology.orchestra.run.smithi143.stdout: "name": "url_prefix", 2023-11-26T07:23:10.184 INFO:teuthology.orchestra.run.smithi143.stdout: "type": "str", 2023-11-26T07:23:10.184 INFO:teuthology.orchestra.run.smithi143.stdout: "level": "advanced", 2023-11-26T07:23:10.184 INFO:teuthology.orchestra.run.smithi143.stdout: "flags": 0, 2023-11-26T07:23:10.184 INFO:teuthology.orchestra.run.smithi143.stdout: "default_value": "", 2023-11-26T07:23:10.184 INFO:teuthology.orchestra.run.smithi143.stdout: "min": "", 2023-11-26T07:23:10.184 INFO:teuthology.orchestra.run.smithi143.stdout: "max": "", 2023-11-26T07:23:10.184 INFO:teuthology.orchestra.run.smithi143.stdout: "enum_allowed": [], 2023-11-26T07:23:10.184 INFO:teuthology.orchestra.run.smithi143.stdout: "desc": "", 2023-11-26T07:23:10.184 INFO:teuthology.orchestra.run.smithi143.stdout: "long_desc": "", 2023-11-26T07:23:10.184 INFO:teuthology.orchestra.run.smithi143.stdout: "tags": [], 2023-11-26T07:23:10.185 INFO:teuthology.orchestra.run.smithi143.stdout: "see_also": [] 2023-11-26T07:23:10.185 INFO:teuthology.orchestra.run.smithi143.stdout: } 2023-11-26T07:23:10.185 INFO:teuthology.orchestra.run.smithi143.stdout: } 2023-11-26T07:23:10.185 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2023-11-26T07:23:10.185 INFO:teuthology.orchestra.run.smithi143.stdout: { 2023-11-26T07:23:10.185 INFO:teuthology.orchestra.run.smithi143.stdout: "name": "devicehealth", 2023-11-26T07:23:10.185 INFO:teuthology.orchestra.run.smithi143.stdout: "can_run": true, 2023-11-26T07:23:10.185 INFO:teuthology.orchestra.run.smithi143.stdout: "error_string": "", 2023-11-26T07:23:10.185 INFO:teuthology.orchestra.run.smithi143.stdout: "module_options": { 2023-11-26T07:23:10.185 INFO:teuthology.orchestra.run.smithi143.stdout: "enable_monitoring": { 2023-11-26T07:23:10.185 INFO:teuthology.orchestra.run.smithi143.stdout: "name": "enable_monitoring", 2023-11-26T07:23:10.185 INFO:teuthology.orchestra.run.smithi143.stdout: "type": "bool", 2023-11-26T07:23:10.186 INFO:teuthology.orchestra.run.smithi143.stdout: "level": "advanced", 2023-11-26T07:23:10.186 INFO:teuthology.orchestra.run.smithi143.stdout: "flags": 1, 2023-11-26T07:23:10.186 INFO:teuthology.orchestra.run.smithi143.stdout: "default_value": "True", 2023-11-26T07:23:10.186 INFO:teuthology.orchestra.run.smithi143.stdout: "min": "", 2023-11-26T07:23:10.186 INFO:teuthology.orchestra.run.smithi143.stdout: "max": "", 2023-11-26T07:23:10.186 INFO:teuthology.orchestra.run.smithi143.stdout: "enum_allowed": [], 2023-11-26T07:23:10.186 INFO:teuthology.orchestra.run.smithi143.stdout: "desc": "monitor device health metrics", 2023-11-26T07:23:10.186 INFO:teuthology.orchestra.run.smithi143.stdout: "long_desc": "", 2023-11-26T07:23:10.186 INFO:teuthology.orchestra.run.smithi143.stdout: "tags": [], 2023-11-26T07:23:10.186 INFO:teuthology.orchestra.run.smithi143.stdout: "see_also": [] 2023-11-26T07:23:10.186 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2023-11-26T07:23:10.186 INFO:teuthology.orchestra.run.smithi143.stdout: "log_level": { 2023-11-26T07:23:10.186 INFO:teuthology.orchestra.run.smithi143.stdout: "name": "log_level", 2023-11-26T07:23:10.187 INFO:teuthology.orchestra.run.smithi143.stdout: "type": "str", 2023-11-26T07:23:10.187 INFO:teuthology.orchestra.run.smithi143.stdout: "level": "advanced", 2023-11-26T07:23:10.187 INFO:teuthology.orchestra.run.smithi143.stdout: "flags": 1, 2023-11-26T07:23:10.187 INFO:teuthology.orchestra.run.smithi143.stdout: "default_value": "", 2023-11-26T07:23:10.187 INFO:teuthology.orchestra.run.smithi143.stdout: "min": "", 2023-11-26T07:23:10.187 INFO:teuthology.orchestra.run.smithi143.stdout: "max": "", 2023-11-26T07:23:10.187 INFO:teuthology.orchestra.run.smithi143.stdout: "enum_allowed": [ 2023-11-26T07:23:10.187 INFO:teuthology.orchestra.run.smithi143.stdout: "", 2023-11-26T07:23:10.187 INFO:teuthology.orchestra.run.smithi143.stdout: "critical", 2023-11-26T07:23:10.187 INFO:teuthology.orchestra.run.smithi143.stdout: "debug", 2023-11-26T07:23:10.187 INFO:teuthology.orchestra.run.smithi143.stdout: "error", 2023-11-26T07:23:10.187 INFO:teuthology.orchestra.run.smithi143.stdout: "info", 2023-11-26T07:23:10.187 INFO:teuthology.orchestra.run.smithi143.stdout: "warning" 2023-11-26T07:23:10.188 INFO:teuthology.orchestra.run.smithi143.stdout: ], 2023-11-26T07:23:10.188 INFO:teuthology.orchestra.run.smithi143.stdout: "desc": "", 2023-11-26T07:23:10.188 INFO:teuthology.orchestra.run.smithi143.stdout: "long_desc": "", 2023-11-26T07:23:10.188 INFO:teuthology.orchestra.run.smithi143.stdout: "tags": [], 2023-11-26T07:23:10.188 INFO:teuthology.orchestra.run.smithi143.stdout: "see_also": [] 2023-11-26T07:23:10.188 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2023-11-26T07:23:10.188 INFO:teuthology.orchestra.run.smithi143.stdout: "log_to_cluster": { 2023-11-26T07:23:10.188 INFO:teuthology.orchestra.run.smithi143.stdout: "name": "log_to_cluster", 2023-11-26T07:23:10.188 INFO:teuthology.orchestra.run.smithi143.stdout: "type": "bool", 2023-11-26T07:23:10.188 INFO:teuthology.orchestra.run.smithi143.stdout: "level": "advanced", 2023-11-26T07:23:10.188 INFO:teuthology.orchestra.run.smithi143.stdout: "flags": 1, 2023-11-26T07:23:10.188 INFO:teuthology.orchestra.run.smithi143.stdout: "default_value": "False", 2023-11-26T07:23:10.189 INFO:teuthology.orchestra.run.smithi143.stdout: "min": "", 2023-11-26T07:23:10.189 INFO:teuthology.orchestra.run.smithi143.stdout: "max": "", 2023-11-26T07:23:10.189 INFO:teuthology.orchestra.run.smithi143.stdout: "enum_allowed": [], 2023-11-26T07:23:10.189 INFO:teuthology.orchestra.run.smithi143.stdout: "desc": "", 2023-11-26T07:23:10.189 INFO:teuthology.orchestra.run.smithi143.stdout: "long_desc": "", 2023-11-26T07:23:10.189 INFO:teuthology.orchestra.run.smithi143.stdout: "tags": [], 2023-11-26T07:23:10.189 INFO:teuthology.orchestra.run.smithi143.stdout: "see_also": [] 2023-11-26T07:23:10.189 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2023-11-26T07:23:10.189 INFO:teuthology.orchestra.run.smithi143.stdout: "log_to_cluster_level": { 2023-11-26T07:23:10.189 INFO:teuthology.orchestra.run.smithi143.stdout: "name": "log_to_cluster_level", 2023-11-26T07:23:10.189 INFO:teuthology.orchestra.run.smithi143.stdout: "type": "str", 2023-11-26T07:23:10.189 INFO:teuthology.orchestra.run.smithi143.stdout: "level": "advanced", 2023-11-26T07:23:10.189 INFO:teuthology.orchestra.run.smithi143.stdout: "flags": 1, 2023-11-26T07:23:10.190 INFO:teuthology.orchestra.run.smithi143.stdout: "default_value": "info", 2023-11-26T07:23:10.190 INFO:teuthology.orchestra.run.smithi143.stdout: "min": "", 2023-11-26T07:23:10.190 INFO:teuthology.orchestra.run.smithi143.stdout: "max": "", 2023-11-26T07:23:10.190 INFO:teuthology.orchestra.run.smithi143.stdout: "enum_allowed": [ 2023-11-26T07:23:10.190 INFO:teuthology.orchestra.run.smithi143.stdout: "", 2023-11-26T07:23:10.190 INFO:teuthology.orchestra.run.smithi143.stdout: "critical", 2023-11-26T07:23:10.190 INFO:teuthology.orchestra.run.smithi143.stdout: "debug", 2023-11-26T07:23:10.190 INFO:teuthology.orchestra.run.smithi143.stdout: "error", 2023-11-26T07:23:10.190 INFO:teuthology.orchestra.run.smithi143.stdout: "info", 2023-11-26T07:23:10.190 INFO:teuthology.orchestra.run.smithi143.stdout: "warning" 2023-11-26T07:23:10.190 INFO:teuthology.orchestra.run.smithi143.stdout: ], 2023-11-26T07:23:10.190 INFO:teuthology.orchestra.run.smithi143.stdout: "desc": "", 2023-11-26T07:23:10.190 INFO:teuthology.orchestra.run.smithi143.stdout: "long_desc": "", 2023-11-26T07:23:10.191 INFO:teuthology.orchestra.run.smithi143.stdout: "tags": [], 2023-11-26T07:23:10.191 INFO:teuthology.orchestra.run.smithi143.stdout: "see_also": [] 2023-11-26T07:23:10.191 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2023-11-26T07:23:10.191 INFO:teuthology.orchestra.run.smithi143.stdout: "log_to_file": { 2023-11-26T07:23:10.191 INFO:teuthology.orchestra.run.smithi143.stdout: "name": "log_to_file", 2023-11-26T07:23:10.191 INFO:teuthology.orchestra.run.smithi143.stdout: "type": "bool", 2023-11-26T07:23:10.191 INFO:teuthology.orchestra.run.smithi143.stdout: "level": "advanced", 2023-11-26T07:23:10.191 INFO:teuthology.orchestra.run.smithi143.stdout: "flags": 1, 2023-11-26T07:23:10.191 INFO:teuthology.orchestra.run.smithi143.stdout: "default_value": "False", 2023-11-26T07:23:10.191 INFO:teuthology.orchestra.run.smithi143.stdout: "min": "", 2023-11-26T07:23:10.191 INFO:teuthology.orchestra.run.smithi143.stdout: "max": "", 2023-11-26T07:23:10.191 INFO:teuthology.orchestra.run.smithi143.stdout: "enum_allowed": [], 2023-11-26T07:23:10.191 INFO:teuthology.orchestra.run.smithi143.stdout: "desc": "", 2023-11-26T07:23:10.192 INFO:teuthology.orchestra.run.smithi143.stdout: "long_desc": "", 2023-11-26T07:23:10.192 INFO:teuthology.orchestra.run.smithi143.stdout: "tags": [], 2023-11-26T07:23:10.192 INFO:teuthology.orchestra.run.smithi143.stdout: "see_also": [] 2023-11-26T07:23:10.192 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2023-11-26T07:23:10.192 INFO:teuthology.orchestra.run.smithi143.stdout: "mark_out_threshold": { 2023-11-26T07:23:10.192 INFO:teuthology.orchestra.run.smithi143.stdout: "name": "mark_out_threshold", 2023-11-26T07:23:10.192 INFO:teuthology.orchestra.run.smithi143.stdout: "type": "secs", 2023-11-26T07:23:10.192 INFO:teuthology.orchestra.run.smithi143.stdout: "level": "advanced", 2023-11-26T07:23:10.192 INFO:teuthology.orchestra.run.smithi143.stdout: "flags": 1, 2023-11-26T07:23:10.192 INFO:teuthology.orchestra.run.smithi143.stdout: "default_value": "2419200", 2023-11-26T07:23:10.192 INFO:teuthology.orchestra.run.smithi143.stdout: "min": "", 2023-11-26T07:23:10.192 INFO:teuthology.orchestra.run.smithi143.stdout: "max": "", 2023-11-26T07:23:10.193 INFO:teuthology.orchestra.run.smithi143.stdout: "enum_allowed": [], 2023-11-26T07:23:10.193 INFO:teuthology.orchestra.run.smithi143.stdout: "desc": "automatically mark OSD if it may fail before this long", 2023-11-26T07:23:10.193 INFO:teuthology.orchestra.run.smithi143.stdout: "long_desc": "", 2023-11-26T07:23:10.193 INFO:teuthology.orchestra.run.smithi143.stdout: "tags": [], 2023-11-26T07:23:10.193 INFO:teuthology.orchestra.run.smithi143.stdout: "see_also": [] 2023-11-26T07:23:10.193 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2023-11-26T07:23:10.193 INFO:teuthology.orchestra.run.smithi143.stdout: "pool_name": { 2023-11-26T07:23:10.193 INFO:teuthology.orchestra.run.smithi143.stdout: "name": "pool_name", 2023-11-26T07:23:10.193 INFO:teuthology.orchestra.run.smithi143.stdout: "type": "str", 2023-11-26T07:23:10.193 INFO:teuthology.orchestra.run.smithi143.stdout: "level": "advanced", 2023-11-26T07:23:10.193 INFO:teuthology.orchestra.run.smithi143.stdout: "flags": 1, 2023-11-26T07:23:10.193 INFO:teuthology.orchestra.run.smithi143.stdout: "default_value": "device_health_metrics", 2023-11-26T07:23:10.193 INFO:teuthology.orchestra.run.smithi143.stdout: "min": "", 2023-11-26T07:23:10.194 INFO:teuthology.orchestra.run.smithi143.stdout: "max": "", 2023-11-26T07:23:10.194 INFO:teuthology.orchestra.run.smithi143.stdout: "enum_allowed": [], 2023-11-26T07:23:10.194 INFO:teuthology.orchestra.run.smithi143.stdout: "desc": "name of pool in which to store device health metrics", 2023-11-26T07:23:10.194 INFO:teuthology.orchestra.run.smithi143.stdout: "long_desc": "", 2023-11-26T07:23:10.194 INFO:teuthology.orchestra.run.smithi143.stdout: "tags": [], 2023-11-26T07:23:10.194 INFO:teuthology.orchestra.run.smithi143.stdout: "see_also": [] 2023-11-26T07:23:10.194 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2023-11-26T07:23:10.194 INFO:teuthology.orchestra.run.smithi143.stdout: "retention_period": { 2023-11-26T07:23:10.194 INFO:teuthology.orchestra.run.smithi143.stdout: "name": "retention_period", 2023-11-26T07:23:10.194 INFO:teuthology.orchestra.run.smithi143.stdout: "type": "secs", 2023-11-26T07:23:10.194 INFO:teuthology.orchestra.run.smithi143.stdout: "level": "advanced", 2023-11-26T07:23:10.194 INFO:teuthology.orchestra.run.smithi143.stdout: "flags": 1, 2023-11-26T07:23:10.195 INFO:teuthology.orchestra.run.smithi143.stdout: "default_value": "15552000", 2023-11-26T07:23:10.195 INFO:teuthology.orchestra.run.smithi143.stdout: "min": "", 2023-11-26T07:23:10.195 INFO:teuthology.orchestra.run.smithi143.stdout: "max": "", 2023-11-26T07:23:10.195 INFO:teuthology.orchestra.run.smithi143.stdout: "enum_allowed": [], 2023-11-26T07:23:10.195 INFO:teuthology.orchestra.run.smithi143.stdout: "desc": "how long to retain device health metrics", 2023-11-26T07:23:10.195 INFO:teuthology.orchestra.run.smithi143.stdout: "long_desc": "", 2023-11-26T07:23:10.195 INFO:teuthology.orchestra.run.smithi143.stdout: "tags": [], 2023-11-26T07:23:10.195 INFO:teuthology.orchestra.run.smithi143.stdout: "see_also": [] 2023-11-26T07:23:10.195 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2023-11-26T07:23:10.195 INFO:teuthology.orchestra.run.smithi143.stdout: "scrape_frequency": { 2023-11-26T07:23:10.195 INFO:teuthology.orchestra.run.smithi143.stdout: "name": "scrape_frequency", 2023-11-26T07:23:10.195 INFO:teuthology.orchestra.run.smithi143.stdout: "type": "secs", 2023-11-26T07:23:10.195 INFO:teuthology.orchestra.run.smithi143.stdout: "level": "advanced", 2023-11-26T07:23:10.195 INFO:teuthology.orchestra.run.smithi143.stdout: "flags": 1, 2023-11-26T07:23:10.196 INFO:teuthology.orchestra.run.smithi143.stdout: "default_value": "86400", 2023-11-26T07:23:10.196 INFO:teuthology.orchestra.run.smithi143.stdout: "min": "", 2023-11-26T07:23:10.196 INFO:teuthology.orchestra.run.smithi143.stdout: "max": "", 2023-11-26T07:23:10.196 INFO:teuthology.orchestra.run.smithi143.stdout: "enum_allowed": [], 2023-11-26T07:23:10.196 INFO:teuthology.orchestra.run.smithi143.stdout: "desc": "how frequently to scrape device health metrics", 2023-11-26T07:23:10.196 INFO:teuthology.orchestra.run.smithi143.stdout: "long_desc": "", 2023-11-26T07:23:10.196 INFO:teuthology.orchestra.run.smithi143.stdout: "tags": [], 2023-11-26T07:23:10.196 INFO:teuthology.orchestra.run.smithi143.stdout: "see_also": [] 2023-11-26T07:23:10.196 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2023-11-26T07:23:10.196 INFO:teuthology.orchestra.run.smithi143.stdout: "self_heal": { 2023-11-26T07:23:10.196 INFO:teuthology.orchestra.run.smithi143.stdout: "name": "self_heal", 2023-11-26T07:23:10.196 INFO:teuthology.orchestra.run.smithi143.stdout: "type": "bool", 2023-11-26T07:23:10.196 INFO:teuthology.orchestra.run.smithi143.stdout: "level": "advanced", 2023-11-26T07:23:10.197 INFO:teuthology.orchestra.run.smithi143.stdout: "flags": 1, 2023-11-26T07:23:10.197 INFO:teuthology.orchestra.run.smithi143.stdout: "default_value": "True", 2023-11-26T07:23:10.197 INFO:teuthology.orchestra.run.smithi143.stdout: "min": "", 2023-11-26T07:23:10.197 INFO:teuthology.orchestra.run.smithi143.stdout: "max": "", 2023-11-26T07:23:10.197 INFO:teuthology.orchestra.run.smithi143.stdout: "enum_allowed": [], 2023-11-26T07:23:10.197 INFO:teuthology.orchestra.run.smithi143.stdout: "desc": "preemptively heal cluster around devices that may fail", 2023-11-26T07:23:10.197 INFO:teuthology.orchestra.run.smithi143.stdout: "long_desc": "", 2023-11-26T07:23:10.197 INFO:teuthology.orchestra.run.smithi143.stdout: "tags": [], 2023-11-26T07:23:10.197 INFO:teuthology.orchestra.run.smithi143.stdout: "see_also": [] 2023-11-26T07:23:10.197 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2023-11-26T07:23:10.197 INFO:teuthology.orchestra.run.smithi143.stdout: "sleep_interval": { 2023-11-26T07:23:10.197 INFO:teuthology.orchestra.run.smithi143.stdout: "name": "sleep_interval", 2023-11-26T07:23:10.197 INFO:teuthology.orchestra.run.smithi143.stdout: "type": "secs", 2023-11-26T07:23:10.198 INFO:teuthology.orchestra.run.smithi143.stdout: "level": "advanced", 2023-11-26T07:23:10.198 INFO:teuthology.orchestra.run.smithi143.stdout: "flags": 1, 2023-11-26T07:23:10.198 INFO:teuthology.orchestra.run.smithi143.stdout: "default_value": "600", 2023-11-26T07:23:10.198 INFO:teuthology.orchestra.run.smithi143.stdout: "min": "", 2023-11-26T07:23:10.198 INFO:teuthology.orchestra.run.smithi143.stdout: "max": "", 2023-11-26T07:23:10.198 INFO:teuthology.orchestra.run.smithi143.stdout: "enum_allowed": [], 2023-11-26T07:23:10.198 INFO:teuthology.orchestra.run.smithi143.stdout: "desc": "how frequently to wake up and check device health", 2023-11-26T07:23:10.198 INFO:teuthology.orchestra.run.smithi143.stdout: "long_desc": "", 2023-11-26T07:23:10.198 INFO:teuthology.orchestra.run.smithi143.stdout: "tags": [], 2023-11-26T07:23:10.198 INFO:teuthology.orchestra.run.smithi143.stdout: "see_also": [] 2023-11-26T07:23:10.198 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2023-11-26T07:23:10.198 INFO:teuthology.orchestra.run.smithi143.stdout: "warn_threshold": { 2023-11-26T07:23:10.198 INFO:teuthology.orchestra.run.smithi143.stdout: "name": "warn_threshold", 2023-11-26T07:23:10.199 INFO:teuthology.orchestra.run.smithi143.stdout: "type": "secs", 2023-11-26T07:23:10.199 INFO:teuthology.orchestra.run.smithi143.stdout: "level": "advanced", 2023-11-26T07:23:10.199 INFO:teuthology.orchestra.run.smithi143.stdout: "flags": 1, 2023-11-26T07:23:10.199 INFO:teuthology.orchestra.run.smithi143.stdout: "default_value": "7257600", 2023-11-26T07:23:10.199 INFO:teuthology.orchestra.run.smithi143.stdout: "min": "", 2023-11-26T07:23:10.199 INFO:teuthology.orchestra.run.smithi143.stdout: "max": "", 2023-11-26T07:23:10.199 INFO:teuthology.orchestra.run.smithi143.stdout: "enum_allowed": [], 2023-11-26T07:23:10.199 INFO:teuthology.orchestra.run.smithi143.stdout: "desc": "raise health warning if OSD may fail before this long", 2023-11-26T07:23:10.199 INFO:teuthology.orchestra.run.smithi143.stdout: "long_desc": "", 2023-11-26T07:23:10.199 INFO:teuthology.orchestra.run.smithi143.stdout: "tags": [], 2023-11-26T07:23:10.199 INFO:teuthology.orchestra.run.smithi143.stdout: "see_also": [] 2023-11-26T07:23:10.199 INFO:teuthology.orchestra.run.smithi143.stdout: } 2023-11-26T07:23:10.200 INFO:teuthology.orchestra.run.smithi143.stdout: } 2023-11-26T07:23:10.200 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2023-11-26T07:23:10.200 INFO:teuthology.orchestra.run.smithi143.stdout: { 2023-11-26T07:23:10.200 INFO:teuthology.orchestra.run.smithi143.stdout: "name": "diskprediction_local", 2023-11-26T07:23:10.200 INFO:teuthology.orchestra.run.smithi143.stdout: "can_run": true, 2023-11-26T07:23:10.200 INFO:teuthology.orchestra.run.smithi143.stdout: "error_string": "", 2023-11-26T07:23:10.200 INFO:teuthology.orchestra.run.smithi143.stdout: "module_options": { 2023-11-26T07:23:10.200 INFO:teuthology.orchestra.run.smithi143.stdout: "log_level": { 2023-11-26T07:23:10.200 INFO:teuthology.orchestra.run.smithi143.stdout: "name": "log_level", 2023-11-26T07:23:10.200 INFO:teuthology.orchestra.run.smithi143.stdout: "type": "str", 2023-11-26T07:23:10.200 INFO:teuthology.orchestra.run.smithi143.stdout: "level": "advanced", 2023-11-26T07:23:10.200 INFO:teuthology.orchestra.run.smithi143.stdout: "flags": 1, 2023-11-26T07:23:10.200 INFO:teuthology.orchestra.run.smithi143.stdout: "default_value": "", 2023-11-26T07:23:10.201 INFO:teuthology.orchestra.run.smithi143.stdout: "min": "", 2023-11-26T07:23:10.201 INFO:teuthology.orchestra.run.smithi143.stdout: "max": "", 2023-11-26T07:23:10.201 INFO:teuthology.orchestra.run.smithi143.stdout: "enum_allowed": [ 2023-11-26T07:23:10.201 INFO:teuthology.orchestra.run.smithi143.stdout: "", 2023-11-26T07:23:10.201 INFO:teuthology.orchestra.run.smithi143.stdout: "critical", 2023-11-26T07:23:10.201 INFO:teuthology.orchestra.run.smithi143.stdout: "debug", 2023-11-26T07:23:10.201 INFO:teuthology.orchestra.run.smithi143.stdout: "error", 2023-11-26T07:23:10.201 INFO:teuthology.orchestra.run.smithi143.stdout: "info", 2023-11-26T07:23:10.201 INFO:teuthology.orchestra.run.smithi143.stdout: "warning" 2023-11-26T07:23:10.201 INFO:teuthology.orchestra.run.smithi143.stdout: ], 2023-11-26T07:23:10.201 INFO:teuthology.orchestra.run.smithi143.stdout: "desc": "", 2023-11-26T07:23:10.201 INFO:teuthology.orchestra.run.smithi143.stdout: "long_desc": "", 2023-11-26T07:23:10.201 INFO:teuthology.orchestra.run.smithi143.stdout: "tags": [], 2023-11-26T07:23:10.202 INFO:teuthology.orchestra.run.smithi143.stdout: "see_also": [] 2023-11-26T07:23:10.202 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2023-11-26T07:23:10.202 INFO:teuthology.orchestra.run.smithi143.stdout: "log_to_cluster": { 2023-11-26T07:23:10.202 INFO:teuthology.orchestra.run.smithi143.stdout: "name": "log_to_cluster", 2023-11-26T07:23:10.202 INFO:teuthology.orchestra.run.smithi143.stdout: "type": "bool", 2023-11-26T07:23:10.202 INFO:teuthology.orchestra.run.smithi143.stdout: "level": "advanced", 2023-11-26T07:23:10.202 INFO:teuthology.orchestra.run.smithi143.stdout: "flags": 1, 2023-11-26T07:23:10.202 INFO:teuthology.orchestra.run.smithi143.stdout: "default_value": "False", 2023-11-26T07:23:10.202 INFO:teuthology.orchestra.run.smithi143.stdout: "min": "", 2023-11-26T07:23:10.202 INFO:teuthology.orchestra.run.smithi143.stdout: "max": "", 2023-11-26T07:23:10.202 INFO:teuthology.orchestra.run.smithi143.stdout: "enum_allowed": [], 2023-11-26T07:23:10.202 INFO:teuthology.orchestra.run.smithi143.stdout: "desc": "", 2023-11-26T07:23:10.202 INFO:teuthology.orchestra.run.smithi143.stdout: "long_desc": "", 2023-11-26T07:23:10.203 INFO:teuthology.orchestra.run.smithi143.stdout: "tags": [], 2023-11-26T07:23:10.203 INFO:teuthology.orchestra.run.smithi143.stdout: "see_also": [] 2023-11-26T07:23:10.203 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2023-11-26T07:23:10.203 INFO:teuthology.orchestra.run.smithi143.stdout: "log_to_cluster_level": { 2023-11-26T07:23:10.203 INFO:teuthology.orchestra.run.smithi143.stdout: "name": "log_to_cluster_level", 2023-11-26T07:23:10.203 INFO:teuthology.orchestra.run.smithi143.stdout: "type": "str", 2023-11-26T07:23:10.203 INFO:teuthology.orchestra.run.smithi143.stdout: "level": "advanced", 2023-11-26T07:23:10.203 INFO:teuthology.orchestra.run.smithi143.stdout: "flags": 1, 2023-11-26T07:23:10.203 INFO:teuthology.orchestra.run.smithi143.stdout: "default_value": "info", 2023-11-26T07:23:10.203 INFO:teuthology.orchestra.run.smithi143.stdout: "min": "", 2023-11-26T07:23:10.203 INFO:teuthology.orchestra.run.smithi143.stdout: "max": "", 2023-11-26T07:23:10.203 INFO:teuthology.orchestra.run.smithi143.stdout: "enum_allowed": [ 2023-11-26T07:23:10.204 INFO:teuthology.orchestra.run.smithi143.stdout: "", 2023-11-26T07:23:10.204 INFO:teuthology.orchestra.run.smithi143.stdout: "critical", 2023-11-26T07:23:10.204 INFO:teuthology.orchestra.run.smithi143.stdout: "debug", 2023-11-26T07:23:10.204 INFO:teuthology.orchestra.run.smithi143.stdout: "error", 2023-11-26T07:23:10.204 INFO:teuthology.orchestra.run.smithi143.stdout: "info", 2023-11-26T07:23:10.204 INFO:teuthology.orchestra.run.smithi143.stdout: "warning" 2023-11-26T07:23:10.204 INFO:teuthology.orchestra.run.smithi143.stdout: ], 2023-11-26T07:23:10.204 INFO:teuthology.orchestra.run.smithi143.stdout: "desc": "", 2023-11-26T07:23:10.204 INFO:teuthology.orchestra.run.smithi143.stdout: "long_desc": "", 2023-11-26T07:23:10.204 INFO:teuthology.orchestra.run.smithi143.stdout: "tags": [], 2023-11-26T07:23:10.204 INFO:teuthology.orchestra.run.smithi143.stdout: "see_also": [] 2023-11-26T07:23:10.204 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2023-11-26T07:23:10.204 INFO:teuthology.orchestra.run.smithi143.stdout: "log_to_file": { 2023-11-26T07:23:10.204 INFO:teuthology.orchestra.run.smithi143.stdout: "name": "log_to_file", 2023-11-26T07:23:10.205 INFO:teuthology.orchestra.run.smithi143.stdout: "type": "bool", 2023-11-26T07:23:10.205 INFO:teuthology.orchestra.run.smithi143.stdout: "level": "advanced", 2023-11-26T07:23:10.205 INFO:teuthology.orchestra.run.smithi143.stdout: "flags": 1, 2023-11-26T07:23:10.205 INFO:teuthology.orchestra.run.smithi143.stdout: "default_value": "False", 2023-11-26T07:23:10.205 INFO:teuthology.orchestra.run.smithi143.stdout: "min": "", 2023-11-26T07:23:10.205 INFO:teuthology.orchestra.run.smithi143.stdout: "max": "", 2023-11-26T07:23:10.205 INFO:teuthology.orchestra.run.smithi143.stdout: "enum_allowed": [], 2023-11-26T07:23:10.205 INFO:teuthology.orchestra.run.smithi143.stdout: "desc": "", 2023-11-26T07:23:10.205 INFO:teuthology.orchestra.run.smithi143.stdout: "long_desc": "", 2023-11-26T07:23:10.205 INFO:teuthology.orchestra.run.smithi143.stdout: "tags": [], 2023-11-26T07:23:10.205 INFO:teuthology.orchestra.run.smithi143.stdout: "see_also": [] 2023-11-26T07:23:10.205 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2023-11-26T07:23:10.205 INFO:teuthology.orchestra.run.smithi143.stdout: "predict_interval": { 2023-11-26T07:23:10.206 INFO:teuthology.orchestra.run.smithi143.stdout: "name": "predict_interval", 2023-11-26T07:23:10.206 INFO:teuthology.orchestra.run.smithi143.stdout: "type": "str", 2023-11-26T07:23:10.206 INFO:teuthology.orchestra.run.smithi143.stdout: "level": "advanced", 2023-11-26T07:23:10.206 INFO:teuthology.orchestra.run.smithi143.stdout: "flags": 0, 2023-11-26T07:23:10.206 INFO:teuthology.orchestra.run.smithi143.stdout: "default_value": "86400", 2023-11-26T07:23:10.206 INFO:teuthology.orchestra.run.smithi143.stdout: "min": "", 2023-11-26T07:23:10.206 INFO:teuthology.orchestra.run.smithi143.stdout: "max": "", 2023-11-26T07:23:10.209 INFO:teuthology.orchestra.run.smithi143.stdout: "enum_allowed": [], 2023-11-26T07:23:10.209 INFO:teuthology.orchestra.run.smithi143.stdout: "desc": "", 2023-11-26T07:23:10.209 INFO:teuthology.orchestra.run.smithi143.stdout: "long_desc": "", 2023-11-26T07:23:10.209 INFO:teuthology.orchestra.run.smithi143.stdout: "tags": [], 2023-11-26T07:23:10.209 INFO:teuthology.orchestra.run.smithi143.stdout: "see_also": [] 2023-11-26T07:23:10.209 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2023-11-26T07:23:10.209 INFO:teuthology.orchestra.run.smithi143.stdout: "predictor_model": { 2023-11-26T07:23:10.209 INFO:teuthology.orchestra.run.smithi143.stdout: "name": "predictor_model", 2023-11-26T07:23:10.209 INFO:teuthology.orchestra.run.smithi143.stdout: "type": "str", 2023-11-26T07:23:10.209 INFO:teuthology.orchestra.run.smithi143.stdout: "level": "advanced", 2023-11-26T07:23:10.210 INFO:teuthology.orchestra.run.smithi143.stdout: "flags": 0, 2023-11-26T07:23:10.210 INFO:teuthology.orchestra.run.smithi143.stdout: "default_value": "prophetstor", 2023-11-26T07:23:10.210 INFO:teuthology.orchestra.run.smithi143.stdout: "min": "", 2023-11-26T07:23:10.210 INFO:teuthology.orchestra.run.smithi143.stdout: "max": "", 2023-11-26T07:23:10.210 INFO:teuthology.orchestra.run.smithi143.stdout: "enum_allowed": [], 2023-11-26T07:23:10.210 INFO:teuthology.orchestra.run.smithi143.stdout: "desc": "", 2023-11-26T07:23:10.210 INFO:teuthology.orchestra.run.smithi143.stdout: "long_desc": "", 2023-11-26T07:23:10.210 INFO:teuthology.orchestra.run.smithi143.stdout: "tags": [], 2023-11-26T07:23:10.210 INFO:teuthology.orchestra.run.smithi143.stdout: "see_also": [] 2023-11-26T07:23:10.210 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2023-11-26T07:23:10.210 INFO:teuthology.orchestra.run.smithi143.stdout: "sleep_interval": { 2023-11-26T07:23:10.210 INFO:teuthology.orchestra.run.smithi143.stdout: "name": "sleep_interval", 2023-11-26T07:23:10.210 INFO:teuthology.orchestra.run.smithi143.stdout: "type": "str", 2023-11-26T07:23:10.211 INFO:teuthology.orchestra.run.smithi143.stdout: "level": "advanced", 2023-11-26T07:23:10.211 INFO:teuthology.orchestra.run.smithi143.stdout: "flags": 0, 2023-11-26T07:23:10.211 INFO:teuthology.orchestra.run.smithi143.stdout: "default_value": "600", 2023-11-26T07:23:10.211 INFO:teuthology.orchestra.run.smithi143.stdout: "min": "", 2023-11-26T07:23:10.211 INFO:teuthology.orchestra.run.smithi143.stdout: "max": "", 2023-11-26T07:23:10.211 INFO:teuthology.orchestra.run.smithi143.stdout: "enum_allowed": [], 2023-11-26T07:23:10.211 INFO:teuthology.orchestra.run.smithi143.stdout: "desc": "", 2023-11-26T07:23:10.211 INFO:teuthology.orchestra.run.smithi143.stdout: "long_desc": "", 2023-11-26T07:23:10.211 INFO:teuthology.orchestra.run.smithi143.stdout: "tags": [], 2023-11-26T07:23:10.211 INFO:teuthology.orchestra.run.smithi143.stdout: "see_also": [] 2023-11-26T07:23:10.211 INFO:teuthology.orchestra.run.smithi143.stdout: } 2023-11-26T07:23:10.211 INFO:teuthology.orchestra.run.smithi143.stdout: } 2023-11-26T07:23:10.211 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2023-11-26T07:23:10.212 INFO:teuthology.orchestra.run.smithi143.stdout: { 2023-11-26T07:23:10.212 INFO:teuthology.orchestra.run.smithi143.stdout: "name": "influx", 2023-11-26T07:23:10.212 INFO:teuthology.orchestra.run.smithi143.stdout: "can_run": false, 2023-11-26T07:23:10.212 INFO:teuthology.orchestra.run.smithi143.stdout: "error_string": "influxdb python module not found", 2023-11-26T07:23:10.212 INFO:teuthology.orchestra.run.smithi143.stdout: "module_options": { 2023-11-26T07:23:10.212 INFO:teuthology.orchestra.run.smithi143.stdout: "batch_size": { 2023-11-26T07:23:10.212 INFO:teuthology.orchestra.run.smithi143.stdout: "name": "batch_size", 2023-11-26T07:23:10.212 INFO:teuthology.orchestra.run.smithi143.stdout: "type": "int", 2023-11-26T07:23:10.212 INFO:teuthology.orchestra.run.smithi143.stdout: "level": "advanced", 2023-11-26T07:23:10.212 INFO:teuthology.orchestra.run.smithi143.stdout: "flags": 0, 2023-11-26T07:23:10.212 INFO:teuthology.orchestra.run.smithi143.stdout: "default_value": "5000", 2023-11-26T07:23:10.212 INFO:teuthology.orchestra.run.smithi143.stdout: "min": "", 2023-11-26T07:23:10.212 INFO:teuthology.orchestra.run.smithi143.stdout: "max": "", 2023-11-26T07:23:10.213 INFO:teuthology.orchestra.run.smithi143.stdout: "enum_allowed": [], 2023-11-26T07:23:10.213 INFO:teuthology.orchestra.run.smithi143.stdout: "desc": "How big batches of data points should be when sending to InfluxDB.", 2023-11-26T07:23:10.213 INFO:teuthology.orchestra.run.smithi143.stdout: "long_desc": "", 2023-11-26T07:23:10.213 INFO:teuthology.orchestra.run.smithi143.stdout: "tags": [], 2023-11-26T07:23:10.213 INFO:teuthology.orchestra.run.smithi143.stdout: "see_also": [] 2023-11-26T07:23:10.213 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2023-11-26T07:23:10.213 INFO:teuthology.orchestra.run.smithi143.stdout: "database": { 2023-11-26T07:23:10.213 INFO:teuthology.orchestra.run.smithi143.stdout: "name": "database", 2023-11-26T07:23:10.213 INFO:teuthology.orchestra.run.smithi143.stdout: "type": "str", 2023-11-26T07:23:10.213 INFO:teuthology.orchestra.run.smithi143.stdout: "level": "advanced", 2023-11-26T07:23:10.213 INFO:teuthology.orchestra.run.smithi143.stdout: "flags": 0, 2023-11-26T07:23:10.213 INFO:teuthology.orchestra.run.smithi143.stdout: "default_value": "ceph", 2023-11-26T07:23:10.213 INFO:teuthology.orchestra.run.smithi143.stdout: "min": "", 2023-11-26T07:23:10.214 INFO:teuthology.orchestra.run.smithi143.stdout: "max": "", 2023-11-26T07:23:10.214 INFO:teuthology.orchestra.run.smithi143.stdout: "enum_allowed": [], 2023-11-26T07:23:10.214 INFO:teuthology.orchestra.run.smithi143.stdout: "desc": "InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.", 2023-11-26T07:23:10.214 INFO:teuthology.orchestra.run.smithi143.stdout: "long_desc": "", 2023-11-26T07:23:10.214 INFO:teuthology.orchestra.run.smithi143.stdout: "tags": [], 2023-11-26T07:23:10.214 INFO:teuthology.orchestra.run.smithi143.stdout: "see_also": [] 2023-11-26T07:23:10.214 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2023-11-26T07:23:10.214 INFO:teuthology.orchestra.run.smithi143.stdout: "hostname": { 2023-11-26T07:23:10.214 INFO:teuthology.orchestra.run.smithi143.stdout: "name": "hostname", 2023-11-26T07:23:10.214 INFO:teuthology.orchestra.run.smithi143.stdout: "type": "str", 2023-11-26T07:23:10.214 INFO:teuthology.orchestra.run.smithi143.stdout: "level": "advanced", 2023-11-26T07:23:10.214 INFO:teuthology.orchestra.run.smithi143.stdout: "flags": 0, 2023-11-26T07:23:10.215 INFO:teuthology.orchestra.run.smithi143.stdout: "default_value": "", 2023-11-26T07:23:10.215 INFO:teuthology.orchestra.run.smithi143.stdout: "min": "", 2023-11-26T07:23:10.215 INFO:teuthology.orchestra.run.smithi143.stdout: "max": "", 2023-11-26T07:23:10.215 INFO:teuthology.orchestra.run.smithi143.stdout: "enum_allowed": [], 2023-11-26T07:23:10.215 INFO:teuthology.orchestra.run.smithi143.stdout: "desc": "InfluxDB server hostname", 2023-11-26T07:23:10.215 INFO:teuthology.orchestra.run.smithi143.stdout: "long_desc": "", 2023-11-26T07:23:10.215 INFO:teuthology.orchestra.run.smithi143.stdout: "tags": [], 2023-11-26T07:23:10.215 INFO:teuthology.orchestra.run.smithi143.stdout: "see_also": [] 2023-11-26T07:23:10.215 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2023-11-26T07:23:10.215 INFO:teuthology.orchestra.run.smithi143.stdout: "interval": { 2023-11-26T07:23:10.216 INFO:teuthology.orchestra.run.smithi143.stdout: "name": "interval", 2023-11-26T07:23:10.216 INFO:teuthology.orchestra.run.smithi143.stdout: "type": "secs", 2023-11-26T07:23:10.216 INFO:teuthology.orchestra.run.smithi143.stdout: "level": "advanced", 2023-11-26T07:23:10.216 INFO:teuthology.orchestra.run.smithi143.stdout: "flags": 0, 2023-11-26T07:23:10.216 INFO:teuthology.orchestra.run.smithi143.stdout: "default_value": "30", 2023-11-26T07:23:10.216 INFO:teuthology.orchestra.run.smithi143.stdout: "min": "5", 2023-11-26T07:23:10.216 INFO:teuthology.orchestra.run.smithi143.stdout: "max": "", 2023-11-26T07:23:10.216 INFO:teuthology.orchestra.run.smithi143.stdout: "enum_allowed": [], 2023-11-26T07:23:10.216 INFO:teuthology.orchestra.run.smithi143.stdout: "desc": "Time between reports to InfluxDB. Default 30 seconds.", 2023-11-26T07:23:10.216 INFO:teuthology.orchestra.run.smithi143.stdout: "long_desc": "", 2023-11-26T07:23:10.216 INFO:teuthology.orchestra.run.smithi143.stdout: "tags": [], 2023-11-26T07:23:10.216 INFO:teuthology.orchestra.run.smithi143.stdout: "see_also": [] 2023-11-26T07:23:10.216 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2023-11-26T07:23:10.217 INFO:teuthology.orchestra.run.smithi143.stdout: "log_level": { 2023-11-26T07:23:10.217 INFO:teuthology.orchestra.run.smithi143.stdout: "name": "log_level", 2023-11-26T07:23:10.217 INFO:teuthology.orchestra.run.smithi143.stdout: "type": "str", 2023-11-26T07:23:10.217 INFO:teuthology.orchestra.run.smithi143.stdout: "level": "advanced", 2023-11-26T07:23:10.217 INFO:teuthology.orchestra.run.smithi143.stdout: "flags": 1, 2023-11-26T07:23:10.217 INFO:teuthology.orchestra.run.smithi143.stdout: "default_value": "", 2023-11-26T07:23:10.217 INFO:teuthology.orchestra.run.smithi143.stdout: "min": "", 2023-11-26T07:23:10.217 INFO:teuthology.orchestra.run.smithi143.stdout: "max": "", 2023-11-26T07:23:10.217 INFO:teuthology.orchestra.run.smithi143.stdout: "enum_allowed": [ 2023-11-26T07:23:10.217 INFO:teuthology.orchestra.run.smithi143.stdout: "", 2023-11-26T07:23:10.217 INFO:teuthology.orchestra.run.smithi143.stdout: "critical", 2023-11-26T07:23:10.217 INFO:teuthology.orchestra.run.smithi143.stdout: "debug", 2023-11-26T07:23:10.218 INFO:teuthology.orchestra.run.smithi143.stdout: "error", 2023-11-26T07:23:10.218 INFO:teuthology.orchestra.run.smithi143.stdout: "info", 2023-11-26T07:23:10.218 INFO:teuthology.orchestra.run.smithi143.stdout: "warning" 2023-11-26T07:23:10.218 INFO:teuthology.orchestra.run.smithi143.stdout: ], 2023-11-26T07:23:10.218 INFO:teuthology.orchestra.run.smithi143.stdout: "desc": "", 2023-11-26T07:23:10.218 INFO:teuthology.orchestra.run.smithi143.stdout: "long_desc": "", 2023-11-26T07:23:10.218 INFO:teuthology.orchestra.run.smithi143.stdout: "tags": [], 2023-11-26T07:23:10.218 INFO:teuthology.orchestra.run.smithi143.stdout: "see_also": [] 2023-11-26T07:23:10.218 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2023-11-26T07:23:10.218 INFO:teuthology.orchestra.run.smithi143.stdout: "log_to_cluster": { 2023-11-26T07:23:10.218 INFO:teuthology.orchestra.run.smithi143.stdout: "name": "log_to_cluster", 2023-11-26T07:23:10.219 INFO:teuthology.orchestra.run.smithi143.stdout: "type": "bool", 2023-11-26T07:23:10.219 INFO:teuthology.orchestra.run.smithi143.stdout: "level": "advanced", 2023-11-26T07:23:10.219 INFO:teuthology.orchestra.run.smithi143.stdout: "flags": 1, 2023-11-26T07:23:10.219 INFO:teuthology.orchestra.run.smithi143.stdout: "default_value": "False", 2023-11-26T07:23:10.219 INFO:teuthology.orchestra.run.smithi143.stdout: "min": "", 2023-11-26T07:23:10.219 INFO:teuthology.orchestra.run.smithi143.stdout: "max": "", 2023-11-26T07:23:10.219 INFO:teuthology.orchestra.run.smithi143.stdout: "enum_allowed": [], 2023-11-26T07:23:10.219 INFO:teuthology.orchestra.run.smithi143.stdout: "desc": "", 2023-11-26T07:23:10.219 INFO:teuthology.orchestra.run.smithi143.stdout: "long_desc": "", 2023-11-26T07:23:10.219 INFO:teuthology.orchestra.run.smithi143.stdout: "tags": [], 2023-11-26T07:23:10.219 INFO:teuthology.orchestra.run.smithi143.stdout: "see_also": [] 2023-11-26T07:23:10.219 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2023-11-26T07:23:10.220 INFO:teuthology.orchestra.run.smithi143.stdout: "log_to_cluster_level": { 2023-11-26T07:23:10.220 INFO:teuthology.orchestra.run.smithi143.stdout: "name": "log_to_cluster_level", 2023-11-26T07:23:10.220 INFO:teuthology.orchestra.run.smithi143.stdout: "type": "str", 2023-11-26T07:23:10.220 INFO:teuthology.orchestra.run.smithi143.stdout: "level": "advanced", 2023-11-26T07:23:10.220 INFO:teuthology.orchestra.run.smithi143.stdout: "flags": 1, 2023-11-26T07:23:10.220 INFO:teuthology.orchestra.run.smithi143.stdout: "default_value": "info", 2023-11-26T07:23:10.220 INFO:teuthology.orchestra.run.smithi143.stdout: "min": "", 2023-11-26T07:23:10.220 INFO:teuthology.orchestra.run.smithi143.stdout: "max": "", 2023-11-26T07:23:10.220 INFO:teuthology.orchestra.run.smithi143.stdout: "enum_allowed": [ 2023-11-26T07:23:10.220 INFO:teuthology.orchestra.run.smithi143.stdout: "", 2023-11-26T07:23:10.220 INFO:teuthology.orchestra.run.smithi143.stdout: "critical", 2023-11-26T07:23:10.221 INFO:teuthology.orchestra.run.smithi143.stdout: "debug", 2023-11-26T07:23:10.221 INFO:teuthology.orchestra.run.smithi143.stdout: "error", 2023-11-26T07:23:10.221 INFO:teuthology.orchestra.run.smithi143.stdout: "info", 2023-11-26T07:23:10.221 INFO:teuthology.orchestra.run.smithi143.stdout: "warning" 2023-11-26T07:23:10.221 INFO:teuthology.orchestra.run.smithi143.stdout: ], 2023-11-26T07:23:10.221 INFO:teuthology.orchestra.run.smithi143.stdout: "desc": "", 2023-11-26T07:23:10.221 INFO:teuthology.orchestra.run.smithi143.stdout: "long_desc": "", 2023-11-26T07:23:10.221 INFO:teuthology.orchestra.run.smithi143.stdout: "tags": [], 2023-11-26T07:23:10.221 INFO:teuthology.orchestra.run.smithi143.stdout: "see_also": [] 2023-11-26T07:23:10.221 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2023-11-26T07:23:10.221 INFO:teuthology.orchestra.run.smithi143.stdout: "log_to_file": { 2023-11-26T07:23:10.221 INFO:teuthology.orchestra.run.smithi143.stdout: "name": "log_to_file", 2023-11-26T07:23:10.221 INFO:teuthology.orchestra.run.smithi143.stdout: "type": "bool", 2023-11-26T07:23:10.222 INFO:teuthology.orchestra.run.smithi143.stdout: "level": "advanced", 2023-11-26T07:23:10.222 INFO:teuthology.orchestra.run.smithi143.stdout: "flags": 1, 2023-11-26T07:23:10.222 INFO:teuthology.orchestra.run.smithi143.stdout: "default_value": "False", 2023-11-26T07:23:10.222 INFO:teuthology.orchestra.run.smithi143.stdout: "min": "", 2023-11-26T07:23:10.222 INFO:teuthology.orchestra.run.smithi143.stdout: "max": "", 2023-11-26T07:23:10.222 INFO:teuthology.orchestra.run.smithi143.stdout: "enum_allowed": [], 2023-11-26T07:23:10.222 INFO:teuthology.orchestra.run.smithi143.stdout: "desc": "", 2023-11-26T07:23:10.222 INFO:teuthology.orchestra.run.smithi143.stdout: "long_desc": "", 2023-11-26T07:23:10.222 INFO:teuthology.orchestra.run.smithi143.stdout: "tags": [], 2023-11-26T07:23:10.222 INFO:teuthology.orchestra.run.smithi143.stdout: "see_also": [] 2023-11-26T07:23:10.222 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2023-11-26T07:23:10.222 INFO:teuthology.orchestra.run.smithi143.stdout: "password": { 2023-11-26T07:23:10.223 INFO:teuthology.orchestra.run.smithi143.stdout: "name": "password", 2023-11-26T07:23:10.223 INFO:teuthology.orchestra.run.smithi143.stdout: "type": "str", 2023-11-26T07:23:10.223 INFO:teuthology.orchestra.run.smithi143.stdout: "level": "advanced", 2023-11-26T07:23:10.223 INFO:teuthology.orchestra.run.smithi143.stdout: "flags": 0, 2023-11-26T07:23:10.223 INFO:teuthology.orchestra.run.smithi143.stdout: "default_value": "", 2023-11-26T07:23:10.223 INFO:teuthology.orchestra.run.smithi143.stdout: "min": "", 2023-11-26T07:23:10.223 INFO:teuthology.orchestra.run.smithi143.stdout: "max": "", 2023-11-26T07:23:10.223 INFO:teuthology.orchestra.run.smithi143.stdout: "enum_allowed": [], 2023-11-26T07:23:10.223 INFO:teuthology.orchestra.run.smithi143.stdout: "desc": "password of InfluxDB server user", 2023-11-26T07:23:10.223 INFO:teuthology.orchestra.run.smithi143.stdout: "long_desc": "", 2023-11-26T07:23:10.223 INFO:teuthology.orchestra.run.smithi143.stdout: "tags": [], 2023-11-26T07:23:10.223 INFO:teuthology.orchestra.run.smithi143.stdout: "see_also": [] 2023-11-26T07:23:10.224 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2023-11-26T07:23:10.224 INFO:teuthology.orchestra.run.smithi143.stdout: "port": { 2023-11-26T07:23:10.224 INFO:teuthology.orchestra.run.smithi143.stdout: "name": "port", 2023-11-26T07:23:10.224 INFO:teuthology.orchestra.run.smithi143.stdout: "type": "int", 2023-11-26T07:23:10.224 INFO:teuthology.orchestra.run.smithi143.stdout: "level": "advanced", 2023-11-26T07:23:10.224 INFO:teuthology.orchestra.run.smithi143.stdout: "flags": 0, 2023-11-26T07:23:10.224 INFO:teuthology.orchestra.run.smithi143.stdout: "default_value": "8086", 2023-11-26T07:23:10.224 INFO:teuthology.orchestra.run.smithi143.stdout: "min": "", 2023-11-26T07:23:10.224 INFO:teuthology.orchestra.run.smithi143.stdout: "max": "", 2023-11-26T07:23:10.224 INFO:teuthology.orchestra.run.smithi143.stdout: "enum_allowed": [], 2023-11-26T07:23:10.224 INFO:teuthology.orchestra.run.smithi143.stdout: "desc": "InfluxDB server port", 2023-11-26T07:23:10.224 INFO:teuthology.orchestra.run.smithi143.stdout: "long_desc": "", 2023-11-26T07:23:10.224 INFO:teuthology.orchestra.run.smithi143.stdout: "tags": [], 2023-11-26T07:23:10.225 INFO:teuthology.orchestra.run.smithi143.stdout: "see_also": [] 2023-11-26T07:23:10.225 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2023-11-26T07:23:10.225 INFO:teuthology.orchestra.run.smithi143.stdout: "ssl": { 2023-11-26T07:23:10.225 INFO:teuthology.orchestra.run.smithi143.stdout: "name": "ssl", 2023-11-26T07:23:10.225 INFO:teuthology.orchestra.run.smithi143.stdout: "type": "str", 2023-11-26T07:23:10.225 INFO:teuthology.orchestra.run.smithi143.stdout: "level": "advanced", 2023-11-26T07:23:10.225 INFO:teuthology.orchestra.run.smithi143.stdout: "flags": 0, 2023-11-26T07:23:10.225 INFO:teuthology.orchestra.run.smithi143.stdout: "default_value": "false", 2023-11-26T07:23:10.225 INFO:teuthology.orchestra.run.smithi143.stdout: "min": "", 2023-11-26T07:23:10.225 INFO:teuthology.orchestra.run.smithi143.stdout: "max": "", 2023-11-26T07:23:10.225 INFO:teuthology.orchestra.run.smithi143.stdout: "enum_allowed": [], 2023-11-26T07:23:10.225 INFO:teuthology.orchestra.run.smithi143.stdout: "desc": "Use https connection for InfluxDB server. Use \"true\" or \"false\".", 2023-11-26T07:23:10.225 INFO:teuthology.orchestra.run.smithi143.stdout: "long_desc": "", 2023-11-26T07:23:10.226 INFO:teuthology.orchestra.run.smithi143.stdout: "tags": [], 2023-11-26T07:23:10.226 INFO:teuthology.orchestra.run.smithi143.stdout: "see_also": [] 2023-11-26T07:23:10.226 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2023-11-26T07:23:10.226 INFO:teuthology.orchestra.run.smithi143.stdout: "threads": { 2023-11-26T07:23:10.226 INFO:teuthology.orchestra.run.smithi143.stdout: "name": "threads", 2023-11-26T07:23:10.226 INFO:teuthology.orchestra.run.smithi143.stdout: "type": "int", 2023-11-26T07:23:10.226 INFO:teuthology.orchestra.run.smithi143.stdout: "level": "advanced", 2023-11-26T07:23:10.226 INFO:teuthology.orchestra.run.smithi143.stdout: "flags": 0, 2023-11-26T07:23:10.226 INFO:teuthology.orchestra.run.smithi143.stdout: "default_value": "5", 2023-11-26T07:23:10.226 INFO:teuthology.orchestra.run.smithi143.stdout: "min": "1", 2023-11-26T07:23:10.226 INFO:teuthology.orchestra.run.smithi143.stdout: "max": "32", 2023-11-26T07:23:10.226 INFO:teuthology.orchestra.run.smithi143.stdout: "enum_allowed": [], 2023-11-26T07:23:10.227 INFO:teuthology.orchestra.run.smithi143.stdout: "desc": "How many worker threads should be spawned for sending data to InfluxDB.", 2023-11-26T07:23:10.227 INFO:teuthology.orchestra.run.smithi143.stdout: "long_desc": "", 2023-11-26T07:23:10.227 INFO:teuthology.orchestra.run.smithi143.stdout: "tags": [], 2023-11-26T07:23:10.227 INFO:teuthology.orchestra.run.smithi143.stdout: "see_also": [] 2023-11-26T07:23:10.227 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2023-11-26T07:23:10.227 INFO:teuthology.orchestra.run.smithi143.stdout: "username": { 2023-11-26T07:23:10.227 INFO:teuthology.orchestra.run.smithi143.stdout: "name": "username", 2023-11-26T07:23:10.227 INFO:teuthology.orchestra.run.smithi143.stdout: "type": "str", 2023-11-26T07:23:10.227 INFO:teuthology.orchestra.run.smithi143.stdout: "level": "advanced", 2023-11-26T07:23:10.227 INFO:teuthology.orchestra.run.smithi143.stdout: "flags": 0, 2023-11-26T07:23:10.227 INFO:teuthology.orchestra.run.smithi143.stdout: "default_value": "", 2023-11-26T07:23:10.228 INFO:teuthology.orchestra.run.smithi143.stdout: "min": "", 2023-11-26T07:23:10.228 INFO:teuthology.orchestra.run.smithi143.stdout: "max": "", 2023-11-26T07:23:10.228 INFO:teuthology.orchestra.run.smithi143.stdout: "enum_allowed": [], 2023-11-26T07:23:10.228 INFO:teuthology.orchestra.run.smithi143.stdout: "desc": "username of InfluxDB server user", 2023-11-26T07:23:10.228 INFO:teuthology.orchestra.run.smithi143.stdout: "long_desc": "", 2023-11-26T07:23:10.228 INFO:teuthology.orchestra.run.smithi143.stdout: "tags": [], 2023-11-26T07:23:10.228 INFO:teuthology.orchestra.run.smithi143.stdout: "see_also": [] 2023-11-26T07:23:10.228 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2023-11-26T07:23:10.228 INFO:teuthology.orchestra.run.smithi143.stdout: "verify_ssl": { 2023-11-26T07:23:10.228 INFO:teuthology.orchestra.run.smithi143.stdout: "name": "verify_ssl", 2023-11-26T07:23:10.228 INFO:teuthology.orchestra.run.smithi143.stdout: "type": "str", 2023-11-26T07:23:10.229 INFO:teuthology.orchestra.run.smithi143.stdout: "level": "advanced", 2023-11-26T07:23:10.229 INFO:teuthology.orchestra.run.smithi143.stdout: "flags": 0, 2023-11-26T07:23:10.229 INFO:teuthology.orchestra.run.smithi143.stdout: "default_value": "true", 2023-11-26T07:23:10.229 INFO:teuthology.orchestra.run.smithi143.stdout: "min": "", 2023-11-26T07:23:10.229 INFO:teuthology.orchestra.run.smithi143.stdout: "max": "", 2023-11-26T07:23:10.229 INFO:teuthology.orchestra.run.smithi143.stdout: "enum_allowed": [], 2023-11-26T07:23:10.229 INFO:teuthology.orchestra.run.smithi143.stdout: "desc": "Verify https cert for InfluxDB server. Use \"true\" or \"false\".", 2023-11-26T07:23:10.229 INFO:teuthology.orchestra.run.smithi143.stdout: "long_desc": "", 2023-11-26T07:23:10.229 INFO:teuthology.orchestra.run.smithi143.stdout: "tags": [], 2023-11-26T07:23:10.229 INFO:teuthology.orchestra.run.smithi143.stdout: "see_also": [] 2023-11-26T07:23:10.229 INFO:teuthology.orchestra.run.smithi143.stdout: } 2023-11-26T07:23:10.229 INFO:teuthology.orchestra.run.smithi143.stdout: } 2023-11-26T07:23:10.230 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2023-11-26T07:23:10.230 INFO:teuthology.orchestra.run.smithi143.stdout: { 2023-11-26T07:23:10.230 INFO:teuthology.orchestra.run.smithi143.stdout: "name": "insights", 2023-11-26T07:23:10.230 INFO:teuthology.orchestra.run.smithi143.stdout: "can_run": true, 2023-11-26T07:23:10.230 INFO:teuthology.orchestra.run.smithi143.stdout: "error_string": "", 2023-11-26T07:23:10.230 INFO:teuthology.orchestra.run.smithi143.stdout: "module_options": { 2023-11-26T07:23:10.230 INFO:teuthology.orchestra.run.smithi143.stdout: "log_level": { 2023-11-26T07:23:10.230 INFO:teuthology.orchestra.run.smithi143.stdout: "name": "log_level", 2023-11-26T07:23:10.230 INFO:teuthology.orchestra.run.smithi143.stdout: "type": "str", 2023-11-26T07:23:10.230 INFO:teuthology.orchestra.run.smithi143.stdout: "level": "advanced", 2023-11-26T07:23:10.230 INFO:teuthology.orchestra.run.smithi143.stdout: "flags": 1, 2023-11-26T07:23:10.230 INFO:teuthology.orchestra.run.smithi143.stdout: "default_value": "", 2023-11-26T07:23:10.230 INFO:teuthology.orchestra.run.smithi143.stdout: "min": "", 2023-11-26T07:23:10.231 INFO:teuthology.orchestra.run.smithi143.stdout: "max": "", 2023-11-26T07:23:10.231 INFO:teuthology.orchestra.run.smithi143.stdout: "enum_allowed": [ 2023-11-26T07:23:10.231 INFO:teuthology.orchestra.run.smithi143.stdout: "", 2023-11-26T07:23:10.231 INFO:teuthology.orchestra.run.smithi143.stdout: "critical", 2023-11-26T07:23:10.231 INFO:teuthology.orchestra.run.smithi143.stdout: "debug", 2023-11-26T07:23:10.231 INFO:teuthology.orchestra.run.smithi143.stdout: "error", 2023-11-26T07:23:10.231 INFO:teuthology.orchestra.run.smithi143.stdout: "info", 2023-11-26T07:23:10.231 INFO:teuthology.orchestra.run.smithi143.stdout: "warning" 2023-11-26T07:23:10.231 INFO:teuthology.orchestra.run.smithi143.stdout: ], 2023-11-26T07:23:10.231 INFO:teuthology.orchestra.run.smithi143.stdout: "desc": "", 2023-11-26T07:23:10.231 INFO:teuthology.orchestra.run.smithi143.stdout: "long_desc": "", 2023-11-26T07:23:10.231 INFO:teuthology.orchestra.run.smithi143.stdout: "tags": [], 2023-11-26T07:23:10.232 INFO:teuthology.orchestra.run.smithi143.stdout: "see_also": [] 2023-11-26T07:23:10.232 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2023-11-26T07:23:10.232 INFO:teuthology.orchestra.run.smithi143.stdout: "log_to_cluster": { 2023-11-26T07:23:10.232 INFO:teuthology.orchestra.run.smithi143.stdout: "name": "log_to_cluster", 2023-11-26T07:23:10.232 INFO:teuthology.orchestra.run.smithi143.stdout: "type": "bool", 2023-11-26T07:23:10.232 INFO:teuthology.orchestra.run.smithi143.stdout: "level": "advanced", 2023-11-26T07:23:10.232 INFO:teuthology.orchestra.run.smithi143.stdout: "flags": 1, 2023-11-26T07:23:10.232 INFO:teuthology.orchestra.run.smithi143.stdout: "default_value": "False", 2023-11-26T07:23:10.232 INFO:teuthology.orchestra.run.smithi143.stdout: "min": "", 2023-11-26T07:23:10.232 INFO:teuthology.orchestra.run.smithi143.stdout: "max": "", 2023-11-26T07:23:10.232 INFO:teuthology.orchestra.run.smithi143.stdout: "enum_allowed": [], 2023-11-26T07:23:10.232 INFO:teuthology.orchestra.run.smithi143.stdout: "desc": "", 2023-11-26T07:23:10.233 INFO:teuthology.orchestra.run.smithi143.stdout: "long_desc": "", 2023-11-26T07:23:10.233 INFO:teuthology.orchestra.run.smithi143.stdout: "tags": [], 2023-11-26T07:23:10.233 INFO:teuthology.orchestra.run.smithi143.stdout: "see_also": [] 2023-11-26T07:23:10.233 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2023-11-26T07:23:10.233 INFO:teuthology.orchestra.run.smithi143.stdout: "log_to_cluster_level": { 2023-11-26T07:23:10.233 INFO:teuthology.orchestra.run.smithi143.stdout: "name": "log_to_cluster_level", 2023-11-26T07:23:10.233 INFO:teuthology.orchestra.run.smithi143.stdout: "type": "str", 2023-11-26T07:23:10.233 INFO:teuthology.orchestra.run.smithi143.stdout: "level": "advanced", 2023-11-26T07:23:10.233 INFO:teuthology.orchestra.run.smithi143.stdout: "flags": 1, 2023-11-26T07:23:10.233 INFO:teuthology.orchestra.run.smithi143.stdout: "default_value": "info", 2023-11-26T07:23:10.233 INFO:teuthology.orchestra.run.smithi143.stdout: "min": "", 2023-11-26T07:23:10.233 INFO:teuthology.orchestra.run.smithi143.stdout: "max": "", 2023-11-26T07:23:10.234 INFO:teuthology.orchestra.run.smithi143.stdout: "enum_allowed": [ 2023-11-26T07:23:10.234 INFO:teuthology.orchestra.run.smithi143.stdout: "", 2023-11-26T07:23:10.234 INFO:teuthology.orchestra.run.smithi143.stdout: "critical", 2023-11-26T07:23:10.234 INFO:teuthology.orchestra.run.smithi143.stdout: "debug", 2023-11-26T07:23:10.234 INFO:teuthology.orchestra.run.smithi143.stdout: "error", 2023-11-26T07:23:10.234 INFO:teuthology.orchestra.run.smithi143.stdout: "info", 2023-11-26T07:23:10.234 INFO:teuthology.orchestra.run.smithi143.stdout: "warning" 2023-11-26T07:23:10.234 INFO:teuthology.orchestra.run.smithi143.stdout: ], 2023-11-26T07:23:10.234 INFO:teuthology.orchestra.run.smithi143.stdout: "desc": "", 2023-11-26T07:23:10.234 INFO:teuthology.orchestra.run.smithi143.stdout: "long_desc": "", 2023-11-26T07:23:10.234 INFO:teuthology.orchestra.run.smithi143.stdout: "tags": [], 2023-11-26T07:23:10.234 INFO:teuthology.orchestra.run.smithi143.stdout: "see_also": [] 2023-11-26T07:23:10.234 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2023-11-26T07:23:10.235 INFO:teuthology.orchestra.run.smithi143.stdout: "log_to_file": { 2023-11-26T07:23:10.235 INFO:teuthology.orchestra.run.smithi143.stdout: "name": "log_to_file", 2023-11-26T07:23:10.235 INFO:teuthology.orchestra.run.smithi143.stdout: "type": "bool", 2023-11-26T07:23:10.235 INFO:teuthology.orchestra.run.smithi143.stdout: "level": "advanced", 2023-11-26T07:23:10.235 INFO:teuthology.orchestra.run.smithi143.stdout: "flags": 1, 2023-11-26T07:23:10.235 INFO:teuthology.orchestra.run.smithi143.stdout: "default_value": "False", 2023-11-26T07:23:10.235 INFO:teuthology.orchestra.run.smithi143.stdout: "min": "", 2023-11-26T07:23:10.235 INFO:teuthology.orchestra.run.smithi143.stdout: "max": "", 2023-11-26T07:23:10.235 INFO:teuthology.orchestra.run.smithi143.stdout: "enum_allowed": [], 2023-11-26T07:23:10.235 INFO:teuthology.orchestra.run.smithi143.stdout: "desc": "", 2023-11-26T07:23:10.235 INFO:teuthology.orchestra.run.smithi143.stdout: "long_desc": "", 2023-11-26T07:23:10.235 INFO:teuthology.orchestra.run.smithi143.stdout: "tags": [], 2023-11-26T07:23:10.236 INFO:teuthology.orchestra.run.smithi143.stdout: "see_also": [] 2023-11-26T07:23:10.236 INFO:teuthology.orchestra.run.smithi143.stdout: } 2023-11-26T07:23:10.236 INFO:teuthology.orchestra.run.smithi143.stdout: } 2023-11-26T07:23:10.236 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2023-11-26T07:23:10.236 INFO:teuthology.orchestra.run.smithi143.stdout: { 2023-11-26T07:23:10.236 INFO:teuthology.orchestra.run.smithi143.stdout: "name": "iostat", 2023-11-26T07:23:10.236 INFO:teuthology.orchestra.run.smithi143.stdout: "can_run": true, 2023-11-26T07:23:10.236 INFO:teuthology.orchestra.run.smithi143.stdout: "error_string": "", 2023-11-26T07:23:10.236 INFO:teuthology.orchestra.run.smithi143.stdout: "module_options": { 2023-11-26T07:23:10.236 INFO:teuthology.orchestra.run.smithi143.stdout: "log_level": { 2023-11-26T07:23:10.236 INFO:teuthology.orchestra.run.smithi143.stdout: "name": "log_level", 2023-11-26T07:23:10.236 INFO:teuthology.orchestra.run.smithi143.stdout: "type": "str", 2023-11-26T07:23:10.236 INFO:teuthology.orchestra.run.smithi143.stdout: "level": "advanced", 2023-11-26T07:23:10.237 INFO:teuthology.orchestra.run.smithi143.stdout: "flags": 1, 2023-11-26T07:23:10.237 INFO:teuthology.orchestra.run.smithi143.stdout: "default_value": "", 2023-11-26T07:23:10.237 INFO:teuthology.orchestra.run.smithi143.stdout: "min": "", 2023-11-26T07:23:10.237 INFO:teuthology.orchestra.run.smithi143.stdout: "max": "", 2023-11-26T07:23:10.237 INFO:teuthology.orchestra.run.smithi143.stdout: "enum_allowed": [ 2023-11-26T07:23:10.237 INFO:teuthology.orchestra.run.smithi143.stdout: "", 2023-11-26T07:23:10.237 INFO:teuthology.orchestra.run.smithi143.stdout: "critical", 2023-11-26T07:23:10.237 INFO:teuthology.orchestra.run.smithi143.stdout: "debug", 2023-11-26T07:23:10.237 INFO:teuthology.orchestra.run.smithi143.stdout: "error", 2023-11-26T07:23:10.237 INFO:teuthology.orchestra.run.smithi143.stdout: "info", 2023-11-26T07:23:10.237 INFO:teuthology.orchestra.run.smithi143.stdout: "warning" 2023-11-26T07:23:10.237 INFO:teuthology.orchestra.run.smithi143.stdout: ], 2023-11-26T07:23:10.237 INFO:teuthology.orchestra.run.smithi143.stdout: "desc": "", 2023-11-26T07:23:10.238 INFO:teuthology.orchestra.run.smithi143.stdout: "long_desc": "", 2023-11-26T07:23:10.238 INFO:teuthology.orchestra.run.smithi143.stdout: "tags": [], 2023-11-26T07:23:10.238 INFO:teuthology.orchestra.run.smithi143.stdout: "see_also": [] 2023-11-26T07:23:10.238 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2023-11-26T07:23:10.238 INFO:teuthology.orchestra.run.smithi143.stdout: "log_to_cluster": { 2023-11-26T07:23:10.238 INFO:teuthology.orchestra.run.smithi143.stdout: "name": "log_to_cluster", 2023-11-26T07:23:10.238 INFO:teuthology.orchestra.run.smithi143.stdout: "type": "bool", 2023-11-26T07:23:10.238 INFO:teuthology.orchestra.run.smithi143.stdout: "level": "advanced", 2023-11-26T07:23:10.238 INFO:teuthology.orchestra.run.smithi143.stdout: "flags": 1, 2023-11-26T07:23:10.238 INFO:teuthology.orchestra.run.smithi143.stdout: "default_value": "False", 2023-11-26T07:23:10.238 INFO:teuthology.orchestra.run.smithi143.stdout: "min": "", 2023-11-26T07:23:10.238 INFO:teuthology.orchestra.run.smithi143.stdout: "max": "", 2023-11-26T07:23:10.239 INFO:teuthology.orchestra.run.smithi143.stdout: "enum_allowed": [], 2023-11-26T07:23:10.239 INFO:teuthology.orchestra.run.smithi143.stdout: "desc": "", 2023-11-26T07:23:10.239 INFO:teuthology.orchestra.run.smithi143.stdout: "long_desc": "", 2023-11-26T07:23:10.239 INFO:teuthology.orchestra.run.smithi143.stdout: "tags": [], 2023-11-26T07:23:10.239 INFO:teuthology.orchestra.run.smithi143.stdout: "see_also": [] 2023-11-26T07:23:10.239 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2023-11-26T07:23:10.239 INFO:teuthology.orchestra.run.smithi143.stdout: "log_to_cluster_level": { 2023-11-26T07:23:10.239 INFO:teuthology.orchestra.run.smithi143.stdout: "name": "log_to_cluster_level", 2023-11-26T07:23:10.239 INFO:teuthology.orchestra.run.smithi143.stdout: "type": "str", 2023-11-26T07:23:10.239 INFO:teuthology.orchestra.run.smithi143.stdout: "level": "advanced", 2023-11-26T07:23:10.239 INFO:teuthology.orchestra.run.smithi143.stdout: "flags": 1, 2023-11-26T07:23:10.239 INFO:teuthology.orchestra.run.smithi143.stdout: "default_value": "info", 2023-11-26T07:23:10.239 INFO:teuthology.orchestra.run.smithi143.stdout: "min": "", 2023-11-26T07:23:10.240 INFO:teuthology.orchestra.run.smithi143.stdout: "max": "", 2023-11-26T07:23:10.240 INFO:teuthology.orchestra.run.smithi143.stdout: "enum_allowed": [ 2023-11-26T07:23:10.240 INFO:teuthology.orchestra.run.smithi143.stdout: "", 2023-11-26T07:23:10.240 INFO:teuthology.orchestra.run.smithi143.stdout: "critical", 2023-11-26T07:23:10.240 INFO:teuthology.orchestra.run.smithi143.stdout: "debug", 2023-11-26T07:23:10.240 INFO:teuthology.orchestra.run.smithi143.stdout: "error", 2023-11-26T07:23:10.240 INFO:teuthology.orchestra.run.smithi143.stdout: "info", 2023-11-26T07:23:10.240 INFO:teuthology.orchestra.run.smithi143.stdout: "warning" 2023-11-26T07:23:10.240 INFO:teuthology.orchestra.run.smithi143.stdout: ], 2023-11-26T07:23:10.240 INFO:teuthology.orchestra.run.smithi143.stdout: "desc": "", 2023-11-26T07:23:10.240 INFO:teuthology.orchestra.run.smithi143.stdout: "long_desc": "", 2023-11-26T07:23:10.240 INFO:teuthology.orchestra.run.smithi143.stdout: "tags": [], 2023-11-26T07:23:10.240 INFO:teuthology.orchestra.run.smithi143.stdout: "see_also": [] 2023-11-26T07:23:10.241 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2023-11-26T07:23:10.241 INFO:teuthology.orchestra.run.smithi143.stdout: "log_to_file": { 2023-11-26T07:23:10.241 INFO:teuthology.orchestra.run.smithi143.stdout: "name": "log_to_file", 2023-11-26T07:23:10.241 INFO:teuthology.orchestra.run.smithi143.stdout: "type": "bool", 2023-11-26T07:23:10.241 INFO:teuthology.orchestra.run.smithi143.stdout: "level": "advanced", 2023-11-26T07:23:10.241 INFO:teuthology.orchestra.run.smithi143.stdout: "flags": 1, 2023-11-26T07:23:10.241 INFO:teuthology.orchestra.run.smithi143.stdout: "default_value": "False", 2023-11-26T07:23:10.241 INFO:teuthology.orchestra.run.smithi143.stdout: "min": "", 2023-11-26T07:23:10.241 INFO:teuthology.orchestra.run.smithi143.stdout: "max": "", 2023-11-26T07:23:10.241 INFO:teuthology.orchestra.run.smithi143.stdout: "enum_allowed": [], 2023-11-26T07:23:10.241 INFO:teuthology.orchestra.run.smithi143.stdout: "desc": "", 2023-11-26T07:23:10.241 INFO:teuthology.orchestra.run.smithi143.stdout: "long_desc": "", 2023-11-26T07:23:10.242 INFO:teuthology.orchestra.run.smithi143.stdout: "tags": [], 2023-11-26T07:23:10.242 INFO:teuthology.orchestra.run.smithi143.stdout: "see_also": [] 2023-11-26T07:23:10.242 INFO:teuthology.orchestra.run.smithi143.stdout: } 2023-11-26T07:23:10.242 INFO:teuthology.orchestra.run.smithi143.stdout: } 2023-11-26T07:23:10.242 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2023-11-26T07:23:10.242 INFO:teuthology.orchestra.run.smithi143.stdout: { 2023-11-26T07:23:10.242 INFO:teuthology.orchestra.run.smithi143.stdout: "name": "localpool", 2023-11-26T07:23:10.242 INFO:teuthology.orchestra.run.smithi143.stdout: "can_run": true, 2023-11-26T07:23:10.242 INFO:teuthology.orchestra.run.smithi143.stdout: "error_string": "", 2023-11-26T07:23:10.242 INFO:teuthology.orchestra.run.smithi143.stdout: "module_options": { 2023-11-26T07:23:10.242 INFO:teuthology.orchestra.run.smithi143.stdout: "failure_domain": { 2023-11-26T07:23:10.242 INFO:teuthology.orchestra.run.smithi143.stdout: "name": "failure_domain", 2023-11-26T07:23:10.242 INFO:teuthology.orchestra.run.smithi143.stdout: "type": "str", 2023-11-26T07:23:10.243 INFO:teuthology.orchestra.run.smithi143.stdout: "level": "advanced", 2023-11-26T07:23:10.243 INFO:teuthology.orchestra.run.smithi143.stdout: "flags": 1, 2023-11-26T07:23:10.243 INFO:teuthology.orchestra.run.smithi143.stdout: "default_value": "host", 2023-11-26T07:23:10.243 INFO:teuthology.orchestra.run.smithi143.stdout: "min": "", 2023-11-26T07:23:10.243 INFO:teuthology.orchestra.run.smithi143.stdout: "max": "", 2023-11-26T07:23:10.243 INFO:teuthology.orchestra.run.smithi143.stdout: "enum_allowed": [], 2023-11-26T07:23:10.243 INFO:teuthology.orchestra.run.smithi143.stdout: "desc": "failure domain for any created local pool", 2023-11-26T07:23:10.243 INFO:teuthology.orchestra.run.smithi143.stdout: "long_desc": "what failure domain we should separate data replicas across.", 2023-11-26T07:23:10.243 INFO:teuthology.orchestra.run.smithi143.stdout: "tags": [], 2023-11-26T07:23:10.243 INFO:teuthology.orchestra.run.smithi143.stdout: "see_also": [] 2023-11-26T07:23:10.243 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2023-11-26T07:23:10.243 INFO:teuthology.orchestra.run.smithi143.stdout: "log_level": { 2023-11-26T07:23:10.243 INFO:teuthology.orchestra.run.smithi143.stdout: "name": "log_level", 2023-11-26T07:23:10.244 INFO:teuthology.orchestra.run.smithi143.stdout: "type": "str", 2023-11-26T07:23:10.244 INFO:teuthology.orchestra.run.smithi143.stdout: "level": "advanced", 2023-11-26T07:23:10.244 INFO:teuthology.orchestra.run.smithi143.stdout: "flags": 1, 2023-11-26T07:23:10.244 INFO:teuthology.orchestra.run.smithi143.stdout: "default_value": "", 2023-11-26T07:23:10.244 INFO:teuthology.orchestra.run.smithi143.stdout: "min": "", 2023-11-26T07:23:10.244 INFO:teuthology.orchestra.run.smithi143.stdout: "max": "", 2023-11-26T07:23:10.244 INFO:teuthology.orchestra.run.smithi143.stdout: "enum_allowed": [ 2023-11-26T07:23:10.244 INFO:teuthology.orchestra.run.smithi143.stdout: "", 2023-11-26T07:23:10.244 INFO:teuthology.orchestra.run.smithi143.stdout: "critical", 2023-11-26T07:23:10.244 INFO:teuthology.orchestra.run.smithi143.stdout: "debug", 2023-11-26T07:23:10.244 INFO:teuthology.orchestra.run.smithi143.stdout: "error", 2023-11-26T07:23:10.245 INFO:teuthology.orchestra.run.smithi143.stdout: "info", 2023-11-26T07:23:10.245 INFO:teuthology.orchestra.run.smithi143.stdout: "warning" 2023-11-26T07:23:10.245 INFO:teuthology.orchestra.run.smithi143.stdout: ], 2023-11-26T07:23:10.245 INFO:teuthology.orchestra.run.smithi143.stdout: "desc": "", 2023-11-26T07:23:10.245 INFO:teuthology.orchestra.run.smithi143.stdout: "long_desc": "", 2023-11-26T07:23:10.245 INFO:teuthology.orchestra.run.smithi143.stdout: "tags": [], 2023-11-26T07:23:10.245 INFO:teuthology.orchestra.run.smithi143.stdout: "see_also": [] 2023-11-26T07:23:10.245 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2023-11-26T07:23:10.245 INFO:teuthology.orchestra.run.smithi143.stdout: "log_to_cluster": { 2023-11-26T07:23:10.245 INFO:teuthology.orchestra.run.smithi143.stdout: "name": "log_to_cluster", 2023-11-26T07:23:10.245 INFO:teuthology.orchestra.run.smithi143.stdout: "type": "bool", 2023-11-26T07:23:10.245 INFO:teuthology.orchestra.run.smithi143.stdout: "level": "advanced", 2023-11-26T07:23:10.245 INFO:teuthology.orchestra.run.smithi143.stdout: "flags": 1, 2023-11-26T07:23:10.246 INFO:teuthology.orchestra.run.smithi143.stdout: "default_value": "False", 2023-11-26T07:23:10.246 INFO:teuthology.orchestra.run.smithi143.stdout: "min": "", 2023-11-26T07:23:10.246 INFO:teuthology.orchestra.run.smithi143.stdout: "max": "", 2023-11-26T07:23:10.246 INFO:teuthology.orchestra.run.smithi143.stdout: "enum_allowed": [], 2023-11-26T07:23:10.246 INFO:teuthology.orchestra.run.smithi143.stdout: "desc": "", 2023-11-26T07:23:10.246 INFO:teuthology.orchestra.run.smithi143.stdout: "long_desc": "", 2023-11-26T07:23:10.246 INFO:teuthology.orchestra.run.smithi143.stdout: "tags": [], 2023-11-26T07:23:10.246 INFO:teuthology.orchestra.run.smithi143.stdout: "see_also": [] 2023-11-26T07:23:10.246 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2023-11-26T07:23:10.246 INFO:teuthology.orchestra.run.smithi143.stdout: "log_to_cluster_level": { 2023-11-26T07:23:10.246 INFO:teuthology.orchestra.run.smithi143.stdout: "name": "log_to_cluster_level", 2023-11-26T07:23:10.247 INFO:teuthology.orchestra.run.smithi143.stdout: "type": "str", 2023-11-26T07:23:10.247 INFO:teuthology.orchestra.run.smithi143.stdout: "level": "advanced", 2023-11-26T07:23:10.247 INFO:teuthology.orchestra.run.smithi143.stdout: "flags": 1, 2023-11-26T07:23:10.247 INFO:teuthology.orchestra.run.smithi143.stdout: "default_value": "info", 2023-11-26T07:23:10.247 INFO:teuthology.orchestra.run.smithi143.stdout: "min": "", 2023-11-26T07:23:10.247 INFO:teuthology.orchestra.run.smithi143.stdout: "max": "", 2023-11-26T07:23:10.247 INFO:teuthology.orchestra.run.smithi143.stdout: "enum_allowed": [ 2023-11-26T07:23:10.247 INFO:teuthology.orchestra.run.smithi143.stdout: "", 2023-11-26T07:23:10.247 INFO:teuthology.orchestra.run.smithi143.stdout: "critical", 2023-11-26T07:23:10.247 INFO:teuthology.orchestra.run.smithi143.stdout: "debug", 2023-11-26T07:23:10.247 INFO:teuthology.orchestra.run.smithi143.stdout: "error", 2023-11-26T07:23:10.247 INFO:teuthology.orchestra.run.smithi143.stdout: "info", 2023-11-26T07:23:10.247 INFO:teuthology.orchestra.run.smithi143.stdout: "warning" 2023-11-26T07:23:10.248 INFO:teuthology.orchestra.run.smithi143.stdout: ], 2023-11-26T07:23:10.248 INFO:teuthology.orchestra.run.smithi143.stdout: "desc": "", 2023-11-26T07:23:10.248 INFO:teuthology.orchestra.run.smithi143.stdout: "long_desc": "", 2023-11-26T07:23:10.248 INFO:teuthology.orchestra.run.smithi143.stdout: "tags": [], 2023-11-26T07:23:10.248 INFO:teuthology.orchestra.run.smithi143.stdout: "see_also": [] 2023-11-26T07:23:10.248 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2023-11-26T07:23:10.248 INFO:teuthology.orchestra.run.smithi143.stdout: "log_to_file": { 2023-11-26T07:23:10.248 INFO:teuthology.orchestra.run.smithi143.stdout: "name": "log_to_file", 2023-11-26T07:23:10.248 INFO:teuthology.orchestra.run.smithi143.stdout: "type": "bool", 2023-11-26T07:23:10.248 INFO:teuthology.orchestra.run.smithi143.stdout: "level": "advanced", 2023-11-26T07:23:10.248 INFO:teuthology.orchestra.run.smithi143.stdout: "flags": 1, 2023-11-26T07:23:10.248 INFO:teuthology.orchestra.run.smithi143.stdout: "default_value": "False", 2023-11-26T07:23:10.249 INFO:teuthology.orchestra.run.smithi143.stdout: "min": "", 2023-11-26T07:23:10.249 INFO:teuthology.orchestra.run.smithi143.stdout: "max": "", 2023-11-26T07:23:10.249 INFO:teuthology.orchestra.run.smithi143.stdout: "enum_allowed": [], 2023-11-26T07:23:10.249 INFO:teuthology.orchestra.run.smithi143.stdout: "desc": "", 2023-11-26T07:23:10.249 INFO:teuthology.orchestra.run.smithi143.stdout: "long_desc": "", 2023-11-26T07:23:10.249 INFO:teuthology.orchestra.run.smithi143.stdout: "tags": [], 2023-11-26T07:23:10.249 INFO:teuthology.orchestra.run.smithi143.stdout: "see_also": [] 2023-11-26T07:23:10.249 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2023-11-26T07:23:10.249 INFO:teuthology.orchestra.run.smithi143.stdout: "min_size": { 2023-11-26T07:23:10.249 INFO:teuthology.orchestra.run.smithi143.stdout: "name": "min_size", 2023-11-26T07:23:10.249 INFO:teuthology.orchestra.run.smithi143.stdout: "type": "int", 2023-11-26T07:23:10.249 INFO:teuthology.orchestra.run.smithi143.stdout: "level": "advanced", 2023-11-26T07:23:10.249 INFO:teuthology.orchestra.run.smithi143.stdout: "flags": 1, 2023-11-26T07:23:10.250 INFO:teuthology.orchestra.run.smithi143.stdout: "default_value": "", 2023-11-26T07:23:10.250 INFO:teuthology.orchestra.run.smithi143.stdout: "min": "", 2023-11-26T07:23:10.250 INFO:teuthology.orchestra.run.smithi143.stdout: "max": "", 2023-11-26T07:23:10.250 INFO:teuthology.orchestra.run.smithi143.stdout: "enum_allowed": [], 2023-11-26T07:23:10.250 INFO:teuthology.orchestra.run.smithi143.stdout: "desc": "default min_size for any created local pool", 2023-11-26T07:23:10.250 INFO:teuthology.orchestra.run.smithi143.stdout: "long_desc": "value to set min_size to (unchanged from Ceph's default if this option is not set)", 2023-11-26T07:23:10.250 INFO:teuthology.orchestra.run.smithi143.stdout: "tags": [], 2023-11-26T07:23:10.250 INFO:teuthology.orchestra.run.smithi143.stdout: "see_also": [] 2023-11-26T07:23:10.250 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2023-11-26T07:23:10.250 INFO:teuthology.orchestra.run.smithi143.stdout: "num_rep": { 2023-11-26T07:23:10.250 INFO:teuthology.orchestra.run.smithi143.stdout: "name": "num_rep", 2023-11-26T07:23:10.250 INFO:teuthology.orchestra.run.smithi143.stdout: "type": "int", 2023-11-26T07:23:10.251 INFO:teuthology.orchestra.run.smithi143.stdout: "level": "advanced", 2023-11-26T07:23:10.251 INFO:teuthology.orchestra.run.smithi143.stdout: "flags": 1, 2023-11-26T07:23:10.251 INFO:teuthology.orchestra.run.smithi143.stdout: "default_value": "3", 2023-11-26T07:23:10.251 INFO:teuthology.orchestra.run.smithi143.stdout: "min": "", 2023-11-26T07:23:10.251 INFO:teuthology.orchestra.run.smithi143.stdout: "max": "", 2023-11-26T07:23:10.251 INFO:teuthology.orchestra.run.smithi143.stdout: "enum_allowed": [], 2023-11-26T07:23:10.251 INFO:teuthology.orchestra.run.smithi143.stdout: "desc": "default replica count for any created local pool", 2023-11-26T07:23:10.251 INFO:teuthology.orchestra.run.smithi143.stdout: "long_desc": "", 2023-11-26T07:23:10.251 INFO:teuthology.orchestra.run.smithi143.stdout: "tags": [], 2023-11-26T07:23:10.251 INFO:teuthology.orchestra.run.smithi143.stdout: "see_also": [] 2023-11-26T07:23:10.251 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2023-11-26T07:23:10.251 INFO:teuthology.orchestra.run.smithi143.stdout: "pg_num": { 2023-11-26T07:23:10.252 INFO:teuthology.orchestra.run.smithi143.stdout: "name": "pg_num", 2023-11-26T07:23:10.252 INFO:teuthology.orchestra.run.smithi143.stdout: "type": "int", 2023-11-26T07:23:10.252 INFO:teuthology.orchestra.run.smithi143.stdout: "level": "advanced", 2023-11-26T07:23:10.252 INFO:teuthology.orchestra.run.smithi143.stdout: "flags": 1, 2023-11-26T07:23:10.252 INFO:teuthology.orchestra.run.smithi143.stdout: "default_value": "128", 2023-11-26T07:23:10.252 INFO:teuthology.orchestra.run.smithi143.stdout: "min": "", 2023-11-26T07:23:10.252 INFO:teuthology.orchestra.run.smithi143.stdout: "max": "", 2023-11-26T07:23:10.252 INFO:teuthology.orchestra.run.smithi143.stdout: "enum_allowed": [], 2023-11-26T07:23:10.252 INFO:teuthology.orchestra.run.smithi143.stdout: "desc": "default pg_num for any created local pool", 2023-11-26T07:23:10.252 INFO:teuthology.orchestra.run.smithi143.stdout: "long_desc": "", 2023-11-26T07:23:10.252 INFO:teuthology.orchestra.run.smithi143.stdout: "tags": [], 2023-11-26T07:23:10.252 INFO:teuthology.orchestra.run.smithi143.stdout: "see_also": [] 2023-11-26T07:23:10.252 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2023-11-26T07:23:10.253 INFO:teuthology.orchestra.run.smithi143.stdout: "prefix": { 2023-11-26T07:23:10.253 INFO:teuthology.orchestra.run.smithi143.stdout: "name": "prefix", 2023-11-26T07:23:10.253 INFO:teuthology.orchestra.run.smithi143.stdout: "type": "str", 2023-11-26T07:23:10.253 INFO:teuthology.orchestra.run.smithi143.stdout: "level": "advanced", 2023-11-26T07:23:10.253 INFO:teuthology.orchestra.run.smithi143.stdout: "flags": 1, 2023-11-26T07:23:10.253 INFO:teuthology.orchestra.run.smithi143.stdout: "default_value": "", 2023-11-26T07:23:10.253 INFO:teuthology.orchestra.run.smithi143.stdout: "min": "", 2023-11-26T07:23:10.253 INFO:teuthology.orchestra.run.smithi143.stdout: "max": "", 2023-11-26T07:23:10.253 INFO:teuthology.orchestra.run.smithi143.stdout: "enum_allowed": [], 2023-11-26T07:23:10.253 INFO:teuthology.orchestra.run.smithi143.stdout: "desc": "name prefix for any created local pool", 2023-11-26T07:23:10.253 INFO:teuthology.orchestra.run.smithi143.stdout: "long_desc": "", 2023-11-26T07:23:10.253 INFO:teuthology.orchestra.run.smithi143.stdout: "tags": [], 2023-11-26T07:23:10.253 INFO:teuthology.orchestra.run.smithi143.stdout: "see_also": [] 2023-11-26T07:23:10.254 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2023-11-26T07:23:10.254 INFO:teuthology.orchestra.run.smithi143.stdout: "subtree": { 2023-11-26T07:23:10.254 INFO:teuthology.orchestra.run.smithi143.stdout: "name": "subtree", 2023-11-26T07:23:10.254 INFO:teuthology.orchestra.run.smithi143.stdout: "type": "str", 2023-11-26T07:23:10.254 INFO:teuthology.orchestra.run.smithi143.stdout: "level": "advanced", 2023-11-26T07:23:10.254 INFO:teuthology.orchestra.run.smithi143.stdout: "flags": 1, 2023-11-26T07:23:10.254 INFO:teuthology.orchestra.run.smithi143.stdout: "default_value": "rack", 2023-11-26T07:23:10.254 INFO:teuthology.orchestra.run.smithi143.stdout: "min": "", 2023-11-26T07:23:10.254 INFO:teuthology.orchestra.run.smithi143.stdout: "max": "", 2023-11-26T07:23:10.254 INFO:teuthology.orchestra.run.smithi143.stdout: "enum_allowed": [], 2023-11-26T07:23:10.254 INFO:teuthology.orchestra.run.smithi143.stdout: "desc": "CRUSH level for which to create a local pool", 2023-11-26T07:23:10.254 INFO:teuthology.orchestra.run.smithi143.stdout: "long_desc": "which CRUSH subtree type the module should create a pool for.", 2023-11-26T07:23:10.254 INFO:teuthology.orchestra.run.smithi143.stdout: "tags": [], 2023-11-26T07:23:10.255 INFO:teuthology.orchestra.run.smithi143.stdout: "see_also": [] 2023-11-26T07:23:10.255 INFO:teuthology.orchestra.run.smithi143.stdout: } 2023-11-26T07:23:10.255 INFO:teuthology.orchestra.run.smithi143.stdout: } 2023-11-26T07:23:10.255 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2023-11-26T07:23:10.255 INFO:teuthology.orchestra.run.smithi143.stdout: { 2023-11-26T07:23:10.255 INFO:teuthology.orchestra.run.smithi143.stdout: "name": "mirroring", 2023-11-26T07:23:10.255 INFO:teuthology.orchestra.run.smithi143.stdout: "can_run": true, 2023-11-26T07:23:10.255 INFO:teuthology.orchestra.run.smithi143.stdout: "error_string": "", 2023-11-26T07:23:10.255 INFO:teuthology.orchestra.run.smithi143.stdout: "module_options": { 2023-11-26T07:23:10.255 INFO:teuthology.orchestra.run.smithi143.stdout: "log_level": { 2023-11-26T07:23:10.255 INFO:teuthology.orchestra.run.smithi143.stdout: "name": "log_level", 2023-11-26T07:23:10.255 INFO:teuthology.orchestra.run.smithi143.stdout: "type": "str", 2023-11-26T07:23:10.255 INFO:teuthology.orchestra.run.smithi143.stdout: "level": "advanced", 2023-11-26T07:23:10.256 INFO:teuthology.orchestra.run.smithi143.stdout: "flags": 1, 2023-11-26T07:23:10.256 INFO:teuthology.orchestra.run.smithi143.stdout: "default_value": "", 2023-11-26T07:23:10.256 INFO:teuthology.orchestra.run.smithi143.stdout: "min": "", 2023-11-26T07:23:10.256 INFO:teuthology.orchestra.run.smithi143.stdout: "max": "", 2023-11-26T07:23:10.256 INFO:teuthology.orchestra.run.smithi143.stdout: "enum_allowed": [ 2023-11-26T07:23:10.256 INFO:teuthology.orchestra.run.smithi143.stdout: "", 2023-11-26T07:23:10.256 INFO:teuthology.orchestra.run.smithi143.stdout: "critical", 2023-11-26T07:23:10.256 INFO:teuthology.orchestra.run.smithi143.stdout: "debug", 2023-11-26T07:23:10.256 INFO:teuthology.orchestra.run.smithi143.stdout: "error", 2023-11-26T07:23:10.256 INFO:teuthology.orchestra.run.smithi143.stdout: "info", 2023-11-26T07:23:10.256 INFO:teuthology.orchestra.run.smithi143.stdout: "warning" 2023-11-26T07:23:10.256 INFO:teuthology.orchestra.run.smithi143.stdout: ], 2023-11-26T07:23:10.256 INFO:teuthology.orchestra.run.smithi143.stdout: "desc": "", 2023-11-26T07:23:10.257 INFO:teuthology.orchestra.run.smithi143.stdout: "long_desc": "", 2023-11-26T07:23:10.257 INFO:teuthology.orchestra.run.smithi143.stdout: "tags": [], 2023-11-26T07:23:10.257 INFO:teuthology.orchestra.run.smithi143.stdout: "see_also": [] 2023-11-26T07:23:10.257 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2023-11-26T07:23:10.257 INFO:teuthology.orchestra.run.smithi143.stdout: "log_to_cluster": { 2023-11-26T07:23:10.257 INFO:teuthology.orchestra.run.smithi143.stdout: "name": "log_to_cluster", 2023-11-26T07:23:10.257 INFO:teuthology.orchestra.run.smithi143.stdout: "type": "bool", 2023-11-26T07:23:10.257 INFO:teuthology.orchestra.run.smithi143.stdout: "level": "advanced", 2023-11-26T07:23:10.257 INFO:teuthology.orchestra.run.smithi143.stdout: "flags": 1, 2023-11-26T07:23:10.257 INFO:teuthology.orchestra.run.smithi143.stdout: "default_value": "False", 2023-11-26T07:23:10.257 INFO:teuthology.orchestra.run.smithi143.stdout: "min": "", 2023-11-26T07:23:10.257 INFO:teuthology.orchestra.run.smithi143.stdout: "max": "", 2023-11-26T07:23:10.257 INFO:teuthology.orchestra.run.smithi143.stdout: "enum_allowed": [], 2023-11-26T07:23:10.258 INFO:teuthology.orchestra.run.smithi143.stdout: "desc": "", 2023-11-26T07:23:10.258 INFO:teuthology.orchestra.run.smithi143.stdout: "long_desc": "", 2023-11-26T07:23:10.258 INFO:teuthology.orchestra.run.smithi143.stdout: "tags": [], 2023-11-26T07:23:10.258 INFO:teuthology.orchestra.run.smithi143.stdout: "see_also": [] 2023-11-26T07:23:10.258 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2023-11-26T07:23:10.258 INFO:teuthology.orchestra.run.smithi143.stdout: "log_to_cluster_level": { 2023-11-26T07:23:10.258 INFO:teuthology.orchestra.run.smithi143.stdout: "name": "log_to_cluster_level", 2023-11-26T07:23:10.258 INFO:teuthology.orchestra.run.smithi143.stdout: "type": "str", 2023-11-26T07:23:10.258 INFO:teuthology.orchestra.run.smithi143.stdout: "level": "advanced", 2023-11-26T07:23:10.258 INFO:teuthology.orchestra.run.smithi143.stdout: "flags": 1, 2023-11-26T07:23:10.258 INFO:teuthology.orchestra.run.smithi143.stdout: "default_value": "info", 2023-11-26T07:23:10.258 INFO:teuthology.orchestra.run.smithi143.stdout: "min": "", 2023-11-26T07:23:10.258 INFO:teuthology.orchestra.run.smithi143.stdout: "max": "", 2023-11-26T07:23:10.259 INFO:teuthology.orchestra.run.smithi143.stdout: "enum_allowed": [ 2023-11-26T07:23:10.259 INFO:teuthology.orchestra.run.smithi143.stdout: "", 2023-11-26T07:23:10.259 INFO:teuthology.orchestra.run.smithi143.stdout: "critical", 2023-11-26T07:23:10.259 INFO:teuthology.orchestra.run.smithi143.stdout: "debug", 2023-11-26T07:23:10.259 INFO:teuthology.orchestra.run.smithi143.stdout: "error", 2023-11-26T07:23:10.259 INFO:teuthology.orchestra.run.smithi143.stdout: "info", 2023-11-26T07:23:10.259 INFO:teuthology.orchestra.run.smithi143.stdout: "warning" 2023-11-26T07:23:10.259 INFO:teuthology.orchestra.run.smithi143.stdout: ], 2023-11-26T07:23:10.259 INFO:teuthology.orchestra.run.smithi143.stdout: "desc": "", 2023-11-26T07:23:10.259 INFO:teuthology.orchestra.run.smithi143.stdout: "long_desc": "", 2023-11-26T07:23:10.259 INFO:teuthology.orchestra.run.smithi143.stdout: "tags": [], 2023-11-26T07:23:10.259 INFO:teuthology.orchestra.run.smithi143.stdout: "see_also": [] 2023-11-26T07:23:10.259 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2023-11-26T07:23:10.260 INFO:teuthology.orchestra.run.smithi143.stdout: "log_to_file": { 2023-11-26T07:23:10.260 INFO:teuthology.orchestra.run.smithi143.stdout: "name": "log_to_file", 2023-11-26T07:23:10.260 INFO:teuthology.orchestra.run.smithi143.stdout: "type": "bool", 2023-11-26T07:23:10.260 INFO:teuthology.orchestra.run.smithi143.stdout: "level": "advanced", 2023-11-26T07:23:10.260 INFO:teuthology.orchestra.run.smithi143.stdout: "flags": 1, 2023-11-26T07:23:10.260 INFO:teuthology.orchestra.run.smithi143.stdout: "default_value": "False", 2023-11-26T07:23:10.260 INFO:teuthology.orchestra.run.smithi143.stdout: "min": "", 2023-11-26T07:23:10.260 INFO:teuthology.orchestra.run.smithi143.stdout: "max": "", 2023-11-26T07:23:10.260 INFO:teuthology.orchestra.run.smithi143.stdout: "enum_allowed": [], 2023-11-26T07:23:10.260 INFO:teuthology.orchestra.run.smithi143.stdout: "desc": "", 2023-11-26T07:23:10.260 INFO:teuthology.orchestra.run.smithi143.stdout: "long_desc": "", 2023-11-26T07:23:10.260 INFO:teuthology.orchestra.run.smithi143.stdout: "tags": [], 2023-11-26T07:23:10.260 INFO:teuthology.orchestra.run.smithi143.stdout: "see_also": [] 2023-11-26T07:23:10.261 INFO:teuthology.orchestra.run.smithi143.stdout: } 2023-11-26T07:23:10.261 INFO:teuthology.orchestra.run.smithi143.stdout: } 2023-11-26T07:23:10.261 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2023-11-26T07:23:10.261 INFO:teuthology.orchestra.run.smithi143.stdout: { 2023-11-26T07:23:10.261 INFO:teuthology.orchestra.run.smithi143.stdout: "name": "nfs", 2023-11-26T07:23:10.261 INFO:teuthology.orchestra.run.smithi143.stdout: "can_run": true, 2023-11-26T07:23:10.261 INFO:teuthology.orchestra.run.smithi143.stdout: "error_string": "", 2023-11-26T07:23:10.261 INFO:teuthology.orchestra.run.smithi143.stdout: "module_options": { 2023-11-26T07:23:10.261 INFO:teuthology.orchestra.run.smithi143.stdout: "log_level": { 2023-11-26T07:23:10.261 INFO:teuthology.orchestra.run.smithi143.stdout: "name": "log_level", 2023-11-26T07:23:10.261 INFO:teuthology.orchestra.run.smithi143.stdout: "type": "str", 2023-11-26T07:23:10.261 INFO:teuthology.orchestra.run.smithi143.stdout: "level": "advanced", 2023-11-26T07:23:10.261 INFO:teuthology.orchestra.run.smithi143.stdout: "flags": 1, 2023-11-26T07:23:10.262 INFO:teuthology.orchestra.run.smithi143.stdout: "default_value": "", 2023-11-26T07:23:10.262 INFO:teuthology.orchestra.run.smithi143.stdout: "min": "", 2023-11-26T07:23:10.262 INFO:teuthology.orchestra.run.smithi143.stdout: "max": "", 2023-11-26T07:23:10.262 INFO:teuthology.orchestra.run.smithi143.stdout: "enum_allowed": [ 2023-11-26T07:23:10.262 INFO:teuthology.orchestra.run.smithi143.stdout: "", 2023-11-26T07:23:10.262 INFO:teuthology.orchestra.run.smithi143.stdout: "critical", 2023-11-26T07:23:10.262 INFO:teuthology.orchestra.run.smithi143.stdout: "debug", 2023-11-26T07:23:10.262 INFO:teuthology.orchestra.run.smithi143.stdout: "error", 2023-11-26T07:23:10.262 INFO:teuthology.orchestra.run.smithi143.stdout: "info", 2023-11-26T07:23:10.262 INFO:teuthology.orchestra.run.smithi143.stdout: "warning" 2023-11-26T07:23:10.262 INFO:teuthology.orchestra.run.smithi143.stdout: ], 2023-11-26T07:23:10.262 INFO:teuthology.orchestra.run.smithi143.stdout: "desc": "", 2023-11-26T07:23:10.262 INFO:teuthology.orchestra.run.smithi143.stdout: "long_desc": "", 2023-11-26T07:23:10.263 INFO:teuthology.orchestra.run.smithi143.stdout: "tags": [], 2023-11-26T07:23:10.263 INFO:teuthology.orchestra.run.smithi143.stdout: "see_also": [] 2023-11-26T07:23:10.263 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2023-11-26T07:23:10.263 INFO:teuthology.orchestra.run.smithi143.stdout: "log_to_cluster": { 2023-11-26T07:23:10.263 INFO:teuthology.orchestra.run.smithi143.stdout: "name": "log_to_cluster", 2023-11-26T07:23:10.263 INFO:teuthology.orchestra.run.smithi143.stdout: "type": "bool", 2023-11-26T07:23:10.263 INFO:teuthology.orchestra.run.smithi143.stdout: "level": "advanced", 2023-11-26T07:23:10.263 INFO:teuthology.orchestra.run.smithi143.stdout: "flags": 1, 2023-11-26T07:23:10.263 INFO:teuthology.orchestra.run.smithi143.stdout: "default_value": "False", 2023-11-26T07:23:10.263 INFO:teuthology.orchestra.run.smithi143.stdout: "min": "", 2023-11-26T07:23:10.263 INFO:teuthology.orchestra.run.smithi143.stdout: "max": "", 2023-11-26T07:23:10.263 INFO:teuthology.orchestra.run.smithi143.stdout: "enum_allowed": [], 2023-11-26T07:23:10.264 INFO:teuthology.orchestra.run.smithi143.stdout: "desc": "", 2023-11-26T07:23:10.264 INFO:teuthology.orchestra.run.smithi143.stdout: "long_desc": "", 2023-11-26T07:23:10.264 INFO:teuthology.orchestra.run.smithi143.stdout: "tags": [], 2023-11-26T07:23:10.264 INFO:teuthology.orchestra.run.smithi143.stdout: "see_also": [] 2023-11-26T07:23:10.264 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2023-11-26T07:23:10.264 INFO:teuthology.orchestra.run.smithi143.stdout: "log_to_cluster_level": { 2023-11-26T07:23:10.264 INFO:teuthology.orchestra.run.smithi143.stdout: "name": "log_to_cluster_level", 2023-11-26T07:23:10.264 INFO:teuthology.orchestra.run.smithi143.stdout: "type": "str", 2023-11-26T07:23:10.264 INFO:teuthology.orchestra.run.smithi143.stdout: "level": "advanced", 2023-11-26T07:23:10.264 INFO:teuthology.orchestra.run.smithi143.stdout: "flags": 1, 2023-11-26T07:23:10.264 INFO:teuthology.orchestra.run.smithi143.stdout: "default_value": "info", 2023-11-26T07:23:10.264 INFO:teuthology.orchestra.run.smithi143.stdout: "min": "", 2023-11-26T07:23:10.264 INFO:teuthology.orchestra.run.smithi143.stdout: "max": "", 2023-11-26T07:23:10.265 INFO:teuthology.orchestra.run.smithi143.stdout: "enum_allowed": [ 2023-11-26T07:23:10.265 INFO:teuthology.orchestra.run.smithi143.stdout: "", 2023-11-26T07:23:10.265 INFO:teuthology.orchestra.run.smithi143.stdout: "critical", 2023-11-26T07:23:10.265 INFO:teuthology.orchestra.run.smithi143.stdout: "debug", 2023-11-26T07:23:10.265 INFO:teuthology.orchestra.run.smithi143.stdout: "error", 2023-11-26T07:23:10.265 INFO:teuthology.orchestra.run.smithi143.stdout: "info", 2023-11-26T07:23:10.265 INFO:teuthology.orchestra.run.smithi143.stdout: "warning" 2023-11-26T07:23:10.265 INFO:teuthology.orchestra.run.smithi143.stdout: ], 2023-11-26T07:23:10.265 INFO:teuthology.orchestra.run.smithi143.stdout: "desc": "", 2023-11-26T07:23:10.265 INFO:teuthology.orchestra.run.smithi143.stdout: "long_desc": "", 2023-11-26T07:23:10.265 INFO:teuthology.orchestra.run.smithi143.stdout: "tags": [], 2023-11-26T07:23:10.265 INFO:teuthology.orchestra.run.smithi143.stdout: "see_also": [] 2023-11-26T07:23:10.265 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2023-11-26T07:23:10.266 INFO:teuthology.orchestra.run.smithi143.stdout: "log_to_file": { 2023-11-26T07:23:10.266 INFO:teuthology.orchestra.run.smithi143.stdout: "name": "log_to_file", 2023-11-26T07:23:10.266 INFO:teuthology.orchestra.run.smithi143.stdout: "type": "bool", 2023-11-26T07:23:10.266 INFO:teuthology.orchestra.run.smithi143.stdout: "level": "advanced", 2023-11-26T07:23:10.266 INFO:teuthology.orchestra.run.smithi143.stdout: "flags": 1, 2023-11-26T07:23:10.266 INFO:teuthology.orchestra.run.smithi143.stdout: "default_value": "False", 2023-11-26T07:23:10.266 INFO:teuthology.orchestra.run.smithi143.stdout: "min": "", 2023-11-26T07:23:10.266 INFO:teuthology.orchestra.run.smithi143.stdout: "max": "", 2023-11-26T07:23:10.266 INFO:teuthology.orchestra.run.smithi143.stdout: "enum_allowed": [], 2023-11-26T07:23:10.266 INFO:teuthology.orchestra.run.smithi143.stdout: "desc": "", 2023-11-26T07:23:10.266 INFO:teuthology.orchestra.run.smithi143.stdout: "long_desc": "", 2023-11-26T07:23:10.266 INFO:teuthology.orchestra.run.smithi143.stdout: "tags": [], 2023-11-26T07:23:10.266 INFO:teuthology.orchestra.run.smithi143.stdout: "see_also": [] 2023-11-26T07:23:10.267 INFO:teuthology.orchestra.run.smithi143.stdout: } 2023-11-26T07:23:10.267 INFO:teuthology.orchestra.run.smithi143.stdout: } 2023-11-26T07:23:10.267 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2023-11-26T07:23:10.267 INFO:teuthology.orchestra.run.smithi143.stdout: { 2023-11-26T07:23:10.267 INFO:teuthology.orchestra.run.smithi143.stdout: "name": "orchestrator", 2023-11-26T07:23:10.267 INFO:teuthology.orchestra.run.smithi143.stdout: "can_run": true, 2023-11-26T07:23:10.267 INFO:teuthology.orchestra.run.smithi143.stdout: "error_string": "", 2023-11-26T07:23:10.267 INFO:teuthology.orchestra.run.smithi143.stdout: "module_options": { 2023-11-26T07:23:10.267 INFO:teuthology.orchestra.run.smithi143.stdout: "fail_fs": { 2023-11-26T07:23:10.267 INFO:teuthology.orchestra.run.smithi143.stdout: "name": "fail_fs", 2023-11-26T07:23:10.267 INFO:teuthology.orchestra.run.smithi143.stdout: "type": "bool", 2023-11-26T07:23:10.267 INFO:teuthology.orchestra.run.smithi143.stdout: "level": "advanced", 2023-11-26T07:23:10.267 INFO:teuthology.orchestra.run.smithi143.stdout: "flags": 0, 2023-11-26T07:23:10.268 INFO:teuthology.orchestra.run.smithi143.stdout: "default_value": "False", 2023-11-26T07:23:10.268 INFO:teuthology.orchestra.run.smithi143.stdout: "min": "", 2023-11-26T07:23:10.268 INFO:teuthology.orchestra.run.smithi143.stdout: "max": "", 2023-11-26T07:23:10.268 INFO:teuthology.orchestra.run.smithi143.stdout: "enum_allowed": [], 2023-11-26T07:23:10.268 INFO:teuthology.orchestra.run.smithi143.stdout: "desc": "Fail filesystem for rapid multi-rank mds upgrade", 2023-11-26T07:23:10.268 INFO:teuthology.orchestra.run.smithi143.stdout: "long_desc": "", 2023-11-26T07:23:10.268 INFO:teuthology.orchestra.run.smithi143.stdout: "tags": [], 2023-11-26T07:23:10.268 INFO:teuthology.orchestra.run.smithi143.stdout: "see_also": [] 2023-11-26T07:23:10.268 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2023-11-26T07:23:10.268 INFO:teuthology.orchestra.run.smithi143.stdout: "log_level": { 2023-11-26T07:23:10.268 INFO:teuthology.orchestra.run.smithi143.stdout: "name": "log_level", 2023-11-26T07:23:10.268 INFO:teuthology.orchestra.run.smithi143.stdout: "type": "str", 2023-11-26T07:23:10.269 INFO:teuthology.orchestra.run.smithi143.stdout: "level": "advanced", 2023-11-26T07:23:10.269 INFO:teuthology.orchestra.run.smithi143.stdout: "flags": 1, 2023-11-26T07:23:10.269 INFO:teuthology.orchestra.run.smithi143.stdout: "default_value": "", 2023-11-26T07:23:10.269 INFO:teuthology.orchestra.run.smithi143.stdout: "min": "", 2023-11-26T07:23:10.269 INFO:teuthology.orchestra.run.smithi143.stdout: "max": "", 2023-11-26T07:23:10.269 INFO:teuthology.orchestra.run.smithi143.stdout: "enum_allowed": [ 2023-11-26T07:23:10.269 INFO:teuthology.orchestra.run.smithi143.stdout: "", 2023-11-26T07:23:10.269 INFO:teuthology.orchestra.run.smithi143.stdout: "critical", 2023-11-26T07:23:10.269 INFO:teuthology.orchestra.run.smithi143.stdout: "debug", 2023-11-26T07:23:10.269 INFO:teuthology.orchestra.run.smithi143.stdout: "error", 2023-11-26T07:23:10.269 INFO:teuthology.orchestra.run.smithi143.stdout: "info", 2023-11-26T07:23:10.269 INFO:teuthology.orchestra.run.smithi143.stdout: "warning" 2023-11-26T07:23:10.269 INFO:teuthology.orchestra.run.smithi143.stdout: ], 2023-11-26T07:23:10.270 INFO:teuthology.orchestra.run.smithi143.stdout: "desc": "", 2023-11-26T07:23:10.270 INFO:teuthology.orchestra.run.smithi143.stdout: "long_desc": "", 2023-11-26T07:23:10.270 INFO:teuthology.orchestra.run.smithi143.stdout: "tags": [], 2023-11-26T07:23:10.270 INFO:teuthology.orchestra.run.smithi143.stdout: "see_also": [] 2023-11-26T07:23:10.270 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2023-11-26T07:23:10.270 INFO:teuthology.orchestra.run.smithi143.stdout: "log_to_cluster": { 2023-11-26T07:23:10.270 INFO:teuthology.orchestra.run.smithi143.stdout: "name": "log_to_cluster", 2023-11-26T07:23:10.270 INFO:teuthology.orchestra.run.smithi143.stdout: "type": "bool", 2023-11-26T07:23:10.270 INFO:teuthology.orchestra.run.smithi143.stdout: "level": "advanced", 2023-11-26T07:23:10.270 INFO:teuthology.orchestra.run.smithi143.stdout: "flags": 1, 2023-11-26T07:23:10.270 INFO:teuthology.orchestra.run.smithi143.stdout: "default_value": "False", 2023-11-26T07:23:10.270 INFO:teuthology.orchestra.run.smithi143.stdout: "min": "", 2023-11-26T07:23:10.270 INFO:teuthology.orchestra.run.smithi143.stdout: "max": "", 2023-11-26T07:23:10.271 INFO:teuthology.orchestra.run.smithi143.stdout: "enum_allowed": [], 2023-11-26T07:23:10.271 INFO:teuthology.orchestra.run.smithi143.stdout: "desc": "", 2023-11-26T07:23:10.271 INFO:teuthology.orchestra.run.smithi143.stdout: "long_desc": "", 2023-11-26T07:23:10.271 INFO:teuthology.orchestra.run.smithi143.stdout: "tags": [], 2023-11-26T07:23:10.271 INFO:teuthology.orchestra.run.smithi143.stdout: "see_also": [] 2023-11-26T07:23:10.271 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2023-11-26T07:23:10.271 INFO:teuthology.orchestra.run.smithi143.stdout: "log_to_cluster_level": { 2023-11-26T07:23:10.271 INFO:teuthology.orchestra.run.smithi143.stdout: "name": "log_to_cluster_level", 2023-11-26T07:23:10.271 INFO:teuthology.orchestra.run.smithi143.stdout: "type": "str", 2023-11-26T07:23:10.271 INFO:teuthology.orchestra.run.smithi143.stdout: "level": "advanced", 2023-11-26T07:23:10.271 INFO:teuthology.orchestra.run.smithi143.stdout: "flags": 1, 2023-11-26T07:23:10.271 INFO:teuthology.orchestra.run.smithi143.stdout: "default_value": "info", 2023-11-26T07:23:10.271 INFO:teuthology.orchestra.run.smithi143.stdout: "min": "", 2023-11-26T07:23:10.272 INFO:teuthology.orchestra.run.smithi143.stdout: "max": "", 2023-11-26T07:23:10.272 INFO:teuthology.orchestra.run.smithi143.stdout: "enum_allowed": [ 2023-11-26T07:23:10.272 INFO:teuthology.orchestra.run.smithi143.stdout: "", 2023-11-26T07:23:10.272 INFO:teuthology.orchestra.run.smithi143.stdout: "critical", 2023-11-26T07:23:10.272 INFO:teuthology.orchestra.run.smithi143.stdout: "debug", 2023-11-26T07:23:10.272 INFO:teuthology.orchestra.run.smithi143.stdout: "error", 2023-11-26T07:23:10.272 INFO:teuthology.orchestra.run.smithi143.stdout: "info", 2023-11-26T07:23:10.272 INFO:teuthology.orchestra.run.smithi143.stdout: "warning" 2023-11-26T07:23:10.272 INFO:teuthology.orchestra.run.smithi143.stdout: ], 2023-11-26T07:23:10.272 INFO:teuthology.orchestra.run.smithi143.stdout: "desc": "", 2023-11-26T07:23:10.272 INFO:teuthology.orchestra.run.smithi143.stdout: "long_desc": "", 2023-11-26T07:23:10.272 INFO:teuthology.orchestra.run.smithi143.stdout: "tags": [], 2023-11-26T07:23:10.273 INFO:teuthology.orchestra.run.smithi143.stdout: "see_also": [] 2023-11-26T07:23:10.273 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2023-11-26T07:23:10.273 INFO:teuthology.orchestra.run.smithi143.stdout: "log_to_file": { 2023-11-26T07:23:10.273 INFO:teuthology.orchestra.run.smithi143.stdout: "name": "log_to_file", 2023-11-26T07:23:10.273 INFO:teuthology.orchestra.run.smithi143.stdout: "type": "bool", 2023-11-26T07:23:10.273 INFO:teuthology.orchestra.run.smithi143.stdout: "level": "advanced", 2023-11-26T07:23:10.273 INFO:teuthology.orchestra.run.smithi143.stdout: "flags": 1, 2023-11-26T07:23:10.273 INFO:teuthology.orchestra.run.smithi143.stdout: "default_value": "False", 2023-11-26T07:23:10.273 INFO:teuthology.orchestra.run.smithi143.stdout: "min": "", 2023-11-26T07:23:10.273 INFO:teuthology.orchestra.run.smithi143.stdout: "max": "", 2023-11-26T07:23:10.273 INFO:teuthology.orchestra.run.smithi143.stdout: "enum_allowed": [], 2023-11-26T07:23:10.273 INFO:teuthology.orchestra.run.smithi143.stdout: "desc": "", 2023-11-26T07:23:10.273 INFO:teuthology.orchestra.run.smithi143.stdout: "long_desc": "", 2023-11-26T07:23:10.274 INFO:teuthology.orchestra.run.smithi143.stdout: "tags": [], 2023-11-26T07:23:10.274 INFO:teuthology.orchestra.run.smithi143.stdout: "see_also": [] 2023-11-26T07:23:10.274 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2023-11-26T07:23:10.274 INFO:teuthology.orchestra.run.smithi143.stdout: "orchestrator": { 2023-11-26T07:23:10.274 INFO:teuthology.orchestra.run.smithi143.stdout: "name": "orchestrator", 2023-11-26T07:23:10.274 INFO:teuthology.orchestra.run.smithi143.stdout: "type": "str", 2023-11-26T07:23:10.274 INFO:teuthology.orchestra.run.smithi143.stdout: "level": "advanced", 2023-11-26T07:23:10.274 INFO:teuthology.orchestra.run.smithi143.stdout: "flags": 1, 2023-11-26T07:23:10.274 INFO:teuthology.orchestra.run.smithi143.stdout: "default_value": "", 2023-11-26T07:23:10.274 INFO:teuthology.orchestra.run.smithi143.stdout: "min": "", 2023-11-26T07:23:10.274 INFO:teuthology.orchestra.run.smithi143.stdout: "max": "", 2023-11-26T07:23:10.274 INFO:teuthology.orchestra.run.smithi143.stdout: "enum_allowed": [ 2023-11-26T07:23:10.275 INFO:teuthology.orchestra.run.smithi143.stdout: "cephadm", 2023-11-26T07:23:10.275 INFO:teuthology.orchestra.run.smithi143.stdout: "rook", 2023-11-26T07:23:10.275 INFO:teuthology.orchestra.run.smithi143.stdout: "test_orchestrator" 2023-11-26T07:23:10.275 INFO:teuthology.orchestra.run.smithi143.stdout: ], 2023-11-26T07:23:10.275 INFO:teuthology.orchestra.run.smithi143.stdout: "desc": "Orchestrator backend", 2023-11-26T07:23:10.275 INFO:teuthology.orchestra.run.smithi143.stdout: "long_desc": "", 2023-11-26T07:23:10.275 INFO:teuthology.orchestra.run.smithi143.stdout: "tags": [], 2023-11-26T07:23:10.275 INFO:teuthology.orchestra.run.smithi143.stdout: "see_also": [] 2023-11-26T07:23:10.275 INFO:teuthology.orchestra.run.smithi143.stdout: } 2023-11-26T07:23:10.275 INFO:teuthology.orchestra.run.smithi143.stdout: } 2023-11-26T07:23:10.275 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2023-11-26T07:23:10.275 INFO:teuthology.orchestra.run.smithi143.stdout: { 2023-11-26T07:23:10.275 INFO:teuthology.orchestra.run.smithi143.stdout: "name": "osd_perf_query", 2023-11-26T07:23:10.276 INFO:teuthology.orchestra.run.smithi143.stdout: "can_run": true, 2023-11-26T07:23:10.276 INFO:teuthology.orchestra.run.smithi143.stdout: "error_string": "", 2023-11-26T07:23:10.276 INFO:teuthology.orchestra.run.smithi143.stdout: "module_options": { 2023-11-26T07:23:10.276 INFO:teuthology.orchestra.run.smithi143.stdout: "log_level": { 2023-11-26T07:23:10.276 INFO:teuthology.orchestra.run.smithi143.stdout: "name": "log_level", 2023-11-26T07:23:10.276 INFO:teuthology.orchestra.run.smithi143.stdout: "type": "str", 2023-11-26T07:23:10.276 INFO:teuthology.orchestra.run.smithi143.stdout: "level": "advanced", 2023-11-26T07:23:10.276 INFO:teuthology.orchestra.run.smithi143.stdout: "flags": 1, 2023-11-26T07:23:10.276 INFO:teuthology.orchestra.run.smithi143.stdout: "default_value": "", 2023-11-26T07:23:10.276 INFO:teuthology.orchestra.run.smithi143.stdout: "min": "", 2023-11-26T07:23:10.276 INFO:teuthology.orchestra.run.smithi143.stdout: "max": "", 2023-11-26T07:23:10.276 INFO:teuthology.orchestra.run.smithi143.stdout: "enum_allowed": [ 2023-11-26T07:23:10.277 INFO:teuthology.orchestra.run.smithi143.stdout: "", 2023-11-26T07:23:10.277 INFO:teuthology.orchestra.run.smithi143.stdout: "critical", 2023-11-26T07:23:10.277 INFO:teuthology.orchestra.run.smithi143.stdout: "debug", 2023-11-26T07:23:10.277 INFO:teuthology.orchestra.run.smithi143.stdout: "error", 2023-11-26T07:23:10.277 INFO:teuthology.orchestra.run.smithi143.stdout: "info", 2023-11-26T07:23:10.277 INFO:teuthology.orchestra.run.smithi143.stdout: "warning" 2023-11-26T07:23:10.277 INFO:teuthology.orchestra.run.smithi143.stdout: ], 2023-11-26T07:23:10.277 INFO:teuthology.orchestra.run.smithi143.stdout: "desc": "", 2023-11-26T07:23:10.277 INFO:teuthology.orchestra.run.smithi143.stdout: "long_desc": "", 2023-11-26T07:23:10.277 INFO:teuthology.orchestra.run.smithi143.stdout: "tags": [], 2023-11-26T07:23:10.277 INFO:teuthology.orchestra.run.smithi143.stdout: "see_also": [] 2023-11-26T07:23:10.277 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2023-11-26T07:23:10.277 INFO:teuthology.orchestra.run.smithi143.stdout: "log_to_cluster": { 2023-11-26T07:23:10.278 INFO:teuthology.orchestra.run.smithi143.stdout: "name": "log_to_cluster", 2023-11-26T07:23:10.278 INFO:teuthology.orchestra.run.smithi143.stdout: "type": "bool", 2023-11-26T07:23:10.278 INFO:teuthology.orchestra.run.smithi143.stdout: "level": "advanced", 2023-11-26T07:23:10.278 INFO:teuthology.orchestra.run.smithi143.stdout: "flags": 1, 2023-11-26T07:23:10.278 INFO:teuthology.orchestra.run.smithi143.stdout: "default_value": "False", 2023-11-26T07:23:10.278 INFO:teuthology.orchestra.run.smithi143.stdout: "min": "", 2023-11-26T07:23:10.278 INFO:teuthology.orchestra.run.smithi143.stdout: "max": "", 2023-11-26T07:23:10.278 INFO:teuthology.orchestra.run.smithi143.stdout: "enum_allowed": [], 2023-11-26T07:23:10.278 INFO:teuthology.orchestra.run.smithi143.stdout: "desc": "", 2023-11-26T07:23:10.278 INFO:teuthology.orchestra.run.smithi143.stdout: "long_desc": "", 2023-11-26T07:23:10.278 INFO:teuthology.orchestra.run.smithi143.stdout: "tags": [], 2023-11-26T07:23:10.278 INFO:teuthology.orchestra.run.smithi143.stdout: "see_also": [] 2023-11-26T07:23:10.278 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2023-11-26T07:23:10.279 INFO:teuthology.orchestra.run.smithi143.stdout: "log_to_cluster_level": { 2023-11-26T07:23:10.279 INFO:teuthology.orchestra.run.smithi143.stdout: "name": "log_to_cluster_level", 2023-11-26T07:23:10.279 INFO:teuthology.orchestra.run.smithi143.stdout: "type": "str", 2023-11-26T07:23:10.279 INFO:teuthology.orchestra.run.smithi143.stdout: "level": "advanced", 2023-11-26T07:23:10.279 INFO:teuthology.orchestra.run.smithi143.stdout: "flags": 1, 2023-11-26T07:23:10.279 INFO:teuthology.orchestra.run.smithi143.stdout: "default_value": "info", 2023-11-26T07:23:10.279 INFO:teuthology.orchestra.run.smithi143.stdout: "min": "", 2023-11-26T07:23:10.279 INFO:teuthology.orchestra.run.smithi143.stdout: "max": "", 2023-11-26T07:23:10.279 INFO:teuthology.orchestra.run.smithi143.stdout: "enum_allowed": [ 2023-11-26T07:23:10.279 INFO:teuthology.orchestra.run.smithi143.stdout: "", 2023-11-26T07:23:10.279 INFO:teuthology.orchestra.run.smithi143.stdout: "critical", 2023-11-26T07:23:10.279 INFO:teuthology.orchestra.run.smithi143.stdout: "debug", 2023-11-26T07:23:10.280 INFO:teuthology.orchestra.run.smithi143.stdout: "error", 2023-11-26T07:23:10.280 INFO:teuthology.orchestra.run.smithi143.stdout: "info", 2023-11-26T07:23:10.280 INFO:teuthology.orchestra.run.smithi143.stdout: "warning" 2023-11-26T07:23:10.280 INFO:teuthology.orchestra.run.smithi143.stdout: ], 2023-11-26T07:23:10.280 INFO:teuthology.orchestra.run.smithi143.stdout: "desc": "", 2023-11-26T07:23:10.280 INFO:teuthology.orchestra.run.smithi143.stdout: "long_desc": "", 2023-11-26T07:23:10.280 INFO:teuthology.orchestra.run.smithi143.stdout: "tags": [], 2023-11-26T07:23:10.280 INFO:teuthology.orchestra.run.smithi143.stdout: "see_also": [] 2023-11-26T07:23:10.280 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2023-11-26T07:23:10.280 INFO:teuthology.orchestra.run.smithi143.stdout: "log_to_file": { 2023-11-26T07:23:10.280 INFO:teuthology.orchestra.run.smithi143.stdout: "name": "log_to_file", 2023-11-26T07:23:10.280 INFO:teuthology.orchestra.run.smithi143.stdout: "type": "bool", 2023-11-26T07:23:10.280 INFO:teuthology.orchestra.run.smithi143.stdout: "level": "advanced", 2023-11-26T07:23:10.281 INFO:teuthology.orchestra.run.smithi143.stdout: "flags": 1, 2023-11-26T07:23:10.281 INFO:teuthology.orchestra.run.smithi143.stdout: "default_value": "False", 2023-11-26T07:23:10.281 INFO:teuthology.orchestra.run.smithi143.stdout: "min": "", 2023-11-26T07:23:10.281 INFO:teuthology.orchestra.run.smithi143.stdout: "max": "", 2023-11-26T07:23:10.281 INFO:teuthology.orchestra.run.smithi143.stdout: "enum_allowed": [], 2023-11-26T07:23:10.281 INFO:teuthology.orchestra.run.smithi143.stdout: "desc": "", 2023-11-26T07:23:10.281 INFO:teuthology.orchestra.run.smithi143.stdout: "long_desc": "", 2023-11-26T07:23:10.281 INFO:teuthology.orchestra.run.smithi143.stdout: "tags": [], 2023-11-26T07:23:10.281 INFO:teuthology.orchestra.run.smithi143.stdout: "see_also": [] 2023-11-26T07:23:10.281 INFO:teuthology.orchestra.run.smithi143.stdout: } 2023-11-26T07:23:10.281 INFO:teuthology.orchestra.run.smithi143.stdout: } 2023-11-26T07:23:10.281 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2023-11-26T07:23:10.281 INFO:teuthology.orchestra.run.smithi143.stdout: { 2023-11-26T07:23:10.282 INFO:teuthology.orchestra.run.smithi143.stdout: "name": "osd_support", 2023-11-26T07:23:10.282 INFO:teuthology.orchestra.run.smithi143.stdout: "can_run": true, 2023-11-26T07:23:10.282 INFO:teuthology.orchestra.run.smithi143.stdout: "error_string": "", 2023-11-26T07:23:10.282 INFO:teuthology.orchestra.run.smithi143.stdout: "module_options": { 2023-11-26T07:23:10.282 INFO:teuthology.orchestra.run.smithi143.stdout: "log_level": { 2023-11-26T07:23:10.282 INFO:teuthology.orchestra.run.smithi143.stdout: "name": "log_level", 2023-11-26T07:23:10.282 INFO:teuthology.orchestra.run.smithi143.stdout: "type": "str", 2023-11-26T07:23:10.282 INFO:teuthology.orchestra.run.smithi143.stdout: "level": "advanced", 2023-11-26T07:23:10.282 INFO:teuthology.orchestra.run.smithi143.stdout: "flags": 1, 2023-11-26T07:23:10.282 INFO:teuthology.orchestra.run.smithi143.stdout: "default_value": "", 2023-11-26T07:23:10.282 INFO:teuthology.orchestra.run.smithi143.stdout: "min": "", 2023-11-26T07:23:10.282 INFO:teuthology.orchestra.run.smithi143.stdout: "max": "", 2023-11-26T07:23:10.283 INFO:teuthology.orchestra.run.smithi143.stdout: "enum_allowed": [ 2023-11-26T07:23:10.283 INFO:teuthology.orchestra.run.smithi143.stdout: "", 2023-11-26T07:23:10.283 INFO:teuthology.orchestra.run.smithi143.stdout: "critical", 2023-11-26T07:23:10.283 INFO:teuthology.orchestra.run.smithi143.stdout: "debug", 2023-11-26T07:23:10.283 INFO:teuthology.orchestra.run.smithi143.stdout: "error", 2023-11-26T07:23:10.283 INFO:teuthology.orchestra.run.smithi143.stdout: "info", 2023-11-26T07:23:10.283 INFO:teuthology.orchestra.run.smithi143.stdout: "warning" 2023-11-26T07:23:10.283 INFO:teuthology.orchestra.run.smithi143.stdout: ], 2023-11-26T07:23:10.283 INFO:teuthology.orchestra.run.smithi143.stdout: "desc": "", 2023-11-26T07:23:10.283 INFO:teuthology.orchestra.run.smithi143.stdout: "long_desc": "", 2023-11-26T07:23:10.283 INFO:teuthology.orchestra.run.smithi143.stdout: "tags": [], 2023-11-26T07:23:10.283 INFO:teuthology.orchestra.run.smithi143.stdout: "see_also": [] 2023-11-26T07:23:10.283 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2023-11-26T07:23:10.284 INFO:teuthology.orchestra.run.smithi143.stdout: "log_to_cluster": { 2023-11-26T07:23:10.284 INFO:teuthology.orchestra.run.smithi143.stdout: "name": "log_to_cluster", 2023-11-26T07:23:10.284 INFO:teuthology.orchestra.run.smithi143.stdout: "type": "bool", 2023-11-26T07:23:10.284 INFO:teuthology.orchestra.run.smithi143.stdout: "level": "advanced", 2023-11-26T07:23:10.284 INFO:teuthology.orchestra.run.smithi143.stdout: "flags": 1, 2023-11-26T07:23:10.284 INFO:teuthology.orchestra.run.smithi143.stdout: "default_value": "False", 2023-11-26T07:23:10.284 INFO:teuthology.orchestra.run.smithi143.stdout: "min": "", 2023-11-26T07:23:10.284 INFO:teuthology.orchestra.run.smithi143.stdout: "max": "", 2023-11-26T07:23:10.284 INFO:teuthology.orchestra.run.smithi143.stdout: "enum_allowed": [], 2023-11-26T07:23:10.284 INFO:teuthology.orchestra.run.smithi143.stdout: "desc": "", 2023-11-26T07:23:10.284 INFO:teuthology.orchestra.run.smithi143.stdout: "long_desc": "", 2023-11-26T07:23:10.284 INFO:teuthology.orchestra.run.smithi143.stdout: "tags": [], 2023-11-26T07:23:10.284 INFO:teuthology.orchestra.run.smithi143.stdout: "see_also": [] 2023-11-26T07:23:10.285 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2023-11-26T07:23:10.285 INFO:teuthology.orchestra.run.smithi143.stdout: "log_to_cluster_level": { 2023-11-26T07:23:10.285 INFO:teuthology.orchestra.run.smithi143.stdout: "name": "log_to_cluster_level", 2023-11-26T07:23:10.285 INFO:teuthology.orchestra.run.smithi143.stdout: "type": "str", 2023-11-26T07:23:10.285 INFO:teuthology.orchestra.run.smithi143.stdout: "level": "advanced", 2023-11-26T07:23:10.285 INFO:teuthology.orchestra.run.smithi143.stdout: "flags": 1, 2023-11-26T07:23:10.285 INFO:teuthology.orchestra.run.smithi143.stdout: "default_value": "info", 2023-11-26T07:23:10.285 INFO:teuthology.orchestra.run.smithi143.stdout: "min": "", 2023-11-26T07:23:10.285 INFO:teuthology.orchestra.run.smithi143.stdout: "max": "", 2023-11-26T07:23:10.285 INFO:teuthology.orchestra.run.smithi143.stdout: "enum_allowed": [ 2023-11-26T07:23:10.285 INFO:teuthology.orchestra.run.smithi143.stdout: "", 2023-11-26T07:23:10.285 INFO:teuthology.orchestra.run.smithi143.stdout: "critical", 2023-11-26T07:23:10.285 INFO:teuthology.orchestra.run.smithi143.stdout: "debug", 2023-11-26T07:23:10.286 INFO:teuthology.orchestra.run.smithi143.stdout: "error", 2023-11-26T07:23:10.286 INFO:teuthology.orchestra.run.smithi143.stdout: "info", 2023-11-26T07:23:10.286 INFO:teuthology.orchestra.run.smithi143.stdout: "warning" 2023-11-26T07:23:10.286 INFO:teuthology.orchestra.run.smithi143.stdout: ], 2023-11-26T07:23:10.286 INFO:teuthology.orchestra.run.smithi143.stdout: "desc": "", 2023-11-26T07:23:10.286 INFO:teuthology.orchestra.run.smithi143.stdout: "long_desc": "", 2023-11-26T07:23:10.286 INFO:teuthology.orchestra.run.smithi143.stdout: "tags": [], 2023-11-26T07:23:10.286 INFO:teuthology.orchestra.run.smithi143.stdout: "see_also": [] 2023-11-26T07:23:10.286 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2023-11-26T07:23:10.286 INFO:teuthology.orchestra.run.smithi143.stdout: "log_to_file": { 2023-11-26T07:23:10.286 INFO:teuthology.orchestra.run.smithi143.stdout: "name": "log_to_file", 2023-11-26T07:23:10.286 INFO:teuthology.orchestra.run.smithi143.stdout: "type": "bool", 2023-11-26T07:23:10.287 INFO:teuthology.orchestra.run.smithi143.stdout: "level": "advanced", 2023-11-26T07:23:10.287 INFO:teuthology.orchestra.run.smithi143.stdout: "flags": 1, 2023-11-26T07:23:10.287 INFO:teuthology.orchestra.run.smithi143.stdout: "default_value": "False", 2023-11-26T07:23:10.287 INFO:teuthology.orchestra.run.smithi143.stdout: "min": "", 2023-11-26T07:23:10.287 INFO:teuthology.orchestra.run.smithi143.stdout: "max": "", 2023-11-26T07:23:10.287 INFO:teuthology.orchestra.run.smithi143.stdout: "enum_allowed": [], 2023-11-26T07:23:10.287 INFO:teuthology.orchestra.run.smithi143.stdout: "desc": "", 2023-11-26T07:23:10.287 INFO:teuthology.orchestra.run.smithi143.stdout: "long_desc": "", 2023-11-26T07:23:10.287 INFO:teuthology.orchestra.run.smithi143.stdout: "tags": [], 2023-11-26T07:23:10.287 INFO:teuthology.orchestra.run.smithi143.stdout: "see_also": [] 2023-11-26T07:23:10.287 INFO:teuthology.orchestra.run.smithi143.stdout: } 2023-11-26T07:23:10.287 INFO:teuthology.orchestra.run.smithi143.stdout: } 2023-11-26T07:23:10.287 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2023-11-26T07:23:10.288 INFO:teuthology.orchestra.run.smithi143.stdout: { 2023-11-26T07:23:10.288 INFO:teuthology.orchestra.run.smithi143.stdout: "name": "pg_autoscaler", 2023-11-26T07:23:10.288 INFO:teuthology.orchestra.run.smithi143.stdout: "can_run": true, 2023-11-26T07:23:10.288 INFO:teuthology.orchestra.run.smithi143.stdout: "error_string": "", 2023-11-26T07:23:10.288 INFO:teuthology.orchestra.run.smithi143.stdout: "module_options": { 2023-11-26T07:23:10.288 INFO:teuthology.orchestra.run.smithi143.stdout: "log_level": { 2023-11-26T07:23:10.288 INFO:teuthology.orchestra.run.smithi143.stdout: "name": "log_level", 2023-11-26T07:23:10.288 INFO:teuthology.orchestra.run.smithi143.stdout: "type": "str", 2023-11-26T07:23:10.288 INFO:teuthology.orchestra.run.smithi143.stdout: "level": "advanced", 2023-11-26T07:23:10.288 INFO:teuthology.orchestra.run.smithi143.stdout: "flags": 1, 2023-11-26T07:23:10.288 INFO:teuthology.orchestra.run.smithi143.stdout: "default_value": "", 2023-11-26T07:23:10.288 INFO:teuthology.orchestra.run.smithi143.stdout: "min": "", 2023-11-26T07:23:10.288 INFO:teuthology.orchestra.run.smithi143.stdout: "max": "", 2023-11-26T07:23:10.289 INFO:teuthology.orchestra.run.smithi143.stdout: "enum_allowed": [ 2023-11-26T07:23:10.289 INFO:teuthology.orchestra.run.smithi143.stdout: "", 2023-11-26T07:23:10.289 INFO:teuthology.orchestra.run.smithi143.stdout: "critical", 2023-11-26T07:23:10.289 INFO:teuthology.orchestra.run.smithi143.stdout: "debug", 2023-11-26T07:23:10.289 INFO:teuthology.orchestra.run.smithi143.stdout: "error", 2023-11-26T07:23:10.289 INFO:teuthology.orchestra.run.smithi143.stdout: "info", 2023-11-26T07:23:10.289 INFO:teuthology.orchestra.run.smithi143.stdout: "warning" 2023-11-26T07:23:10.289 INFO:teuthology.orchestra.run.smithi143.stdout: ], 2023-11-26T07:23:10.289 INFO:teuthology.orchestra.run.smithi143.stdout: "desc": "", 2023-11-26T07:23:10.289 INFO:teuthology.orchestra.run.smithi143.stdout: "long_desc": "", 2023-11-26T07:23:10.289 INFO:teuthology.orchestra.run.smithi143.stdout: "tags": [], 2023-11-26T07:23:10.289 INFO:teuthology.orchestra.run.smithi143.stdout: "see_also": [] 2023-11-26T07:23:10.289 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2023-11-26T07:23:10.290 INFO:teuthology.orchestra.run.smithi143.stdout: "log_to_cluster": { 2023-11-26T07:23:10.290 INFO:teuthology.orchestra.run.smithi143.stdout: "name": "log_to_cluster", 2023-11-26T07:23:10.290 INFO:teuthology.orchestra.run.smithi143.stdout: "type": "bool", 2023-11-26T07:23:10.290 INFO:teuthology.orchestra.run.smithi143.stdout: "level": "advanced", 2023-11-26T07:23:10.290 INFO:teuthology.orchestra.run.smithi143.stdout: "flags": 1, 2023-11-26T07:23:10.290 INFO:teuthology.orchestra.run.smithi143.stdout: "default_value": "False", 2023-11-26T07:23:10.290 INFO:teuthology.orchestra.run.smithi143.stdout: "min": "", 2023-11-26T07:23:10.290 INFO:teuthology.orchestra.run.smithi143.stdout: "max": "", 2023-11-26T07:23:10.290 INFO:teuthology.orchestra.run.smithi143.stdout: "enum_allowed": [], 2023-11-26T07:23:10.290 INFO:teuthology.orchestra.run.smithi143.stdout: "desc": "", 2023-11-26T07:23:10.290 INFO:teuthology.orchestra.run.smithi143.stdout: "long_desc": "", 2023-11-26T07:23:10.290 INFO:teuthology.orchestra.run.smithi143.stdout: "tags": [], 2023-11-26T07:23:10.291 INFO:teuthology.orchestra.run.smithi143.stdout: "see_also": [] 2023-11-26T07:23:10.291 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2023-11-26T07:23:10.291 INFO:teuthology.orchestra.run.smithi143.stdout: "log_to_cluster_level": { 2023-11-26T07:23:10.291 INFO:teuthology.orchestra.run.smithi143.stdout: "name": "log_to_cluster_level", 2023-11-26T07:23:10.291 INFO:teuthology.orchestra.run.smithi143.stdout: "type": "str", 2023-11-26T07:23:10.291 INFO:teuthology.orchestra.run.smithi143.stdout: "level": "advanced", 2023-11-26T07:23:10.291 INFO:teuthology.orchestra.run.smithi143.stdout: "flags": 1, 2023-11-26T07:23:10.291 INFO:teuthology.orchestra.run.smithi143.stdout: "default_value": "info", 2023-11-26T07:23:10.291 INFO:teuthology.orchestra.run.smithi143.stdout: "min": "", 2023-11-26T07:23:10.291 INFO:teuthology.orchestra.run.smithi143.stdout: "max": "", 2023-11-26T07:23:10.291 INFO:teuthology.orchestra.run.smithi143.stdout: "enum_allowed": [ 2023-11-26T07:23:10.291 INFO:teuthology.orchestra.run.smithi143.stdout: "", 2023-11-26T07:23:10.291 INFO:teuthology.orchestra.run.smithi143.stdout: "critical", 2023-11-26T07:23:10.292 INFO:teuthology.orchestra.run.smithi143.stdout: "debug", 2023-11-26T07:23:10.292 INFO:teuthology.orchestra.run.smithi143.stdout: "error", 2023-11-26T07:23:10.292 INFO:teuthology.orchestra.run.smithi143.stdout: "info", 2023-11-26T07:23:10.292 INFO:teuthology.orchestra.run.smithi143.stdout: "warning" 2023-11-26T07:23:10.292 INFO:teuthology.orchestra.run.smithi143.stdout: ], 2023-11-26T07:23:10.292 INFO:teuthology.orchestra.run.smithi143.stdout: "desc": "", 2023-11-26T07:23:10.292 INFO:teuthology.orchestra.run.smithi143.stdout: "long_desc": "", 2023-11-26T07:23:10.292 INFO:teuthology.orchestra.run.smithi143.stdout: "tags": [], 2023-11-26T07:23:10.292 INFO:teuthology.orchestra.run.smithi143.stdout: "see_also": [] 2023-11-26T07:23:10.292 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2023-11-26T07:23:10.292 INFO:teuthology.orchestra.run.smithi143.stdout: "log_to_file": { 2023-11-26T07:23:10.292 INFO:teuthology.orchestra.run.smithi143.stdout: "name": "log_to_file", 2023-11-26T07:23:10.293 INFO:teuthology.orchestra.run.smithi143.stdout: "type": "bool", 2023-11-26T07:23:10.293 INFO:teuthology.orchestra.run.smithi143.stdout: "level": "advanced", 2023-11-26T07:23:10.293 INFO:teuthology.orchestra.run.smithi143.stdout: "flags": 1, 2023-11-26T07:23:10.293 INFO:teuthology.orchestra.run.smithi143.stdout: "default_value": "False", 2023-11-26T07:23:10.293 INFO:teuthology.orchestra.run.smithi143.stdout: "min": "", 2023-11-26T07:23:10.293 INFO:teuthology.orchestra.run.smithi143.stdout: "max": "", 2023-11-26T07:23:10.293 INFO:teuthology.orchestra.run.smithi143.stdout: "enum_allowed": [], 2023-11-26T07:23:10.293 INFO:teuthology.orchestra.run.smithi143.stdout: "desc": "", 2023-11-26T07:23:10.293 INFO:teuthology.orchestra.run.smithi143.stdout: "long_desc": "", 2023-11-26T07:23:10.293 INFO:teuthology.orchestra.run.smithi143.stdout: "tags": [], 2023-11-26T07:23:10.293 INFO:teuthology.orchestra.run.smithi143.stdout: "see_also": [] 2023-11-26T07:23:10.293 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2023-11-26T07:23:10.293 INFO:teuthology.orchestra.run.smithi143.stdout: "sleep_interval": { 2023-11-26T07:23:10.294 INFO:teuthology.orchestra.run.smithi143.stdout: "name": "sleep_interval", 2023-11-26T07:23:10.294 INFO:teuthology.orchestra.run.smithi143.stdout: "type": "secs", 2023-11-26T07:23:10.294 INFO:teuthology.orchestra.run.smithi143.stdout: "level": "advanced", 2023-11-26T07:23:10.294 INFO:teuthology.orchestra.run.smithi143.stdout: "flags": 0, 2023-11-26T07:23:10.294 INFO:teuthology.orchestra.run.smithi143.stdout: "default_value": "60", 2023-11-26T07:23:10.294 INFO:teuthology.orchestra.run.smithi143.stdout: "min": "", 2023-11-26T07:23:10.294 INFO:teuthology.orchestra.run.smithi143.stdout: "max": "", 2023-11-26T07:23:10.294 INFO:teuthology.orchestra.run.smithi143.stdout: "enum_allowed": [], 2023-11-26T07:23:10.294 INFO:teuthology.orchestra.run.smithi143.stdout: "desc": "", 2023-11-26T07:23:10.294 INFO:teuthology.orchestra.run.smithi143.stdout: "long_desc": "", 2023-11-26T07:23:10.294 INFO:teuthology.orchestra.run.smithi143.stdout: "tags": [], 2023-11-26T07:23:10.295 INFO:teuthology.orchestra.run.smithi143.stdout: "see_also": [] 2023-11-26T07:23:10.295 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2023-11-26T07:23:10.295 INFO:teuthology.orchestra.run.smithi143.stdout: "threshold": { 2023-11-26T07:23:10.295 INFO:teuthology.orchestra.run.smithi143.stdout: "name": "threshold", 2023-11-26T07:23:10.295 INFO:teuthology.orchestra.run.smithi143.stdout: "type": "float", 2023-11-26T07:23:10.295 INFO:teuthology.orchestra.run.smithi143.stdout: "level": "advanced", 2023-11-26T07:23:10.295 INFO:teuthology.orchestra.run.smithi143.stdout: "flags": 0, 2023-11-26T07:23:10.295 INFO:teuthology.orchestra.run.smithi143.stdout: "default_value": "3.0", 2023-11-26T07:23:10.295 INFO:teuthology.orchestra.run.smithi143.stdout: "min": "1.0", 2023-11-26T07:23:10.295 INFO:teuthology.orchestra.run.smithi143.stdout: "max": "", 2023-11-26T07:23:10.295 INFO:teuthology.orchestra.run.smithi143.stdout: "enum_allowed": [], 2023-11-26T07:23:10.295 INFO:teuthology.orchestra.run.smithi143.stdout: "desc": "scaling threshold", 2023-11-26T07:23:10.295 INFO:teuthology.orchestra.run.smithi143.stdout: "long_desc": "The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0", 2023-11-26T07:23:10.296 INFO:teuthology.orchestra.run.smithi143.stdout: "tags": [], 2023-11-26T07:23:10.296 INFO:teuthology.orchestra.run.smithi143.stdout: "see_also": [] 2023-11-26T07:23:10.296 INFO:teuthology.orchestra.run.smithi143.stdout: } 2023-11-26T07:23:10.296 INFO:teuthology.orchestra.run.smithi143.stdout: } 2023-11-26T07:23:10.296 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2023-11-26T07:23:10.296 INFO:teuthology.orchestra.run.smithi143.stdout: { 2023-11-26T07:23:10.296 INFO:teuthology.orchestra.run.smithi143.stdout: "name": "progress", 2023-11-26T07:23:10.296 INFO:teuthology.orchestra.run.smithi143.stdout: "can_run": true, 2023-11-26T07:23:10.296 INFO:teuthology.orchestra.run.smithi143.stdout: "error_string": "", 2023-11-26T07:23:10.296 INFO:teuthology.orchestra.run.smithi143.stdout: "module_options": { 2023-11-26T07:23:10.296 INFO:teuthology.orchestra.run.smithi143.stdout: "allow_pg_recovery_event": { 2023-11-26T07:23:10.296 INFO:teuthology.orchestra.run.smithi143.stdout: "name": "allow_pg_recovery_event", 2023-11-26T07:23:10.297 INFO:teuthology.orchestra.run.smithi143.stdout: "type": "bool", 2023-11-26T07:23:10.297 INFO:teuthology.orchestra.run.smithi143.stdout: "level": "advanced", 2023-11-26T07:23:10.297 INFO:teuthology.orchestra.run.smithi143.stdout: "flags": 1, 2023-11-26T07:23:10.297 INFO:teuthology.orchestra.run.smithi143.stdout: "default_value": "False", 2023-11-26T07:23:10.297 INFO:teuthology.orchestra.run.smithi143.stdout: "min": "", 2023-11-26T07:23:10.297 INFO:teuthology.orchestra.run.smithi143.stdout: "max": "", 2023-11-26T07:23:10.297 INFO:teuthology.orchestra.run.smithi143.stdout: "enum_allowed": [], 2023-11-26T07:23:10.297 INFO:teuthology.orchestra.run.smithi143.stdout: "desc": "allow the module to show pg recovery progress", 2023-11-26T07:23:10.297 INFO:teuthology.orchestra.run.smithi143.stdout: "long_desc": "", 2023-11-26T07:23:10.297 INFO:teuthology.orchestra.run.smithi143.stdout: "tags": [], 2023-11-26T07:23:10.297 INFO:teuthology.orchestra.run.smithi143.stdout: "see_also": [] 2023-11-26T07:23:10.298 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2023-11-26T07:23:10.298 INFO:teuthology.orchestra.run.smithi143.stdout: "enabled": { 2023-11-26T07:23:10.298 INFO:teuthology.orchestra.run.smithi143.stdout: "name": "enabled", 2023-11-26T07:23:10.298 INFO:teuthology.orchestra.run.smithi143.stdout: "type": "bool", 2023-11-26T07:23:10.298 INFO:teuthology.orchestra.run.smithi143.stdout: "level": "advanced", 2023-11-26T07:23:10.298 INFO:teuthology.orchestra.run.smithi143.stdout: "flags": 0, 2023-11-26T07:23:10.298 INFO:teuthology.orchestra.run.smithi143.stdout: "default_value": "True", 2023-11-26T07:23:10.298 INFO:teuthology.orchestra.run.smithi143.stdout: "min": "", 2023-11-26T07:23:10.298 INFO:teuthology.orchestra.run.smithi143.stdout: "max": "", 2023-11-26T07:23:10.298 INFO:teuthology.orchestra.run.smithi143.stdout: "enum_allowed": [], 2023-11-26T07:23:10.298 INFO:teuthology.orchestra.run.smithi143.stdout: "desc": "", 2023-11-26T07:23:10.298 INFO:teuthology.orchestra.run.smithi143.stdout: "long_desc": "", 2023-11-26T07:23:10.299 INFO:teuthology.orchestra.run.smithi143.stdout: "tags": [], 2023-11-26T07:23:10.299 INFO:teuthology.orchestra.run.smithi143.stdout: "see_also": [] 2023-11-26T07:23:10.299 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2023-11-26T07:23:10.299 INFO:teuthology.orchestra.run.smithi143.stdout: "log_level": { 2023-11-26T07:23:10.299 INFO:teuthology.orchestra.run.smithi143.stdout: "name": "log_level", 2023-11-26T07:23:10.299 INFO:teuthology.orchestra.run.smithi143.stdout: "type": "str", 2023-11-26T07:23:10.299 INFO:teuthology.orchestra.run.smithi143.stdout: "level": "advanced", 2023-11-26T07:23:10.299 INFO:teuthology.orchestra.run.smithi143.stdout: "flags": 1, 2023-11-26T07:23:10.299 INFO:teuthology.orchestra.run.smithi143.stdout: "default_value": "", 2023-11-26T07:23:10.299 INFO:teuthology.orchestra.run.smithi143.stdout: "min": "", 2023-11-26T07:23:10.299 INFO:teuthology.orchestra.run.smithi143.stdout: "max": "", 2023-11-26T07:23:10.299 INFO:teuthology.orchestra.run.smithi143.stdout: "enum_allowed": [ 2023-11-26T07:23:10.299 INFO:teuthology.orchestra.run.smithi143.stdout: "", 2023-11-26T07:23:10.300 INFO:teuthology.orchestra.run.smithi143.stdout: "critical", 2023-11-26T07:23:10.300 INFO:teuthology.orchestra.run.smithi143.stdout: "debug", 2023-11-26T07:23:10.300 INFO:teuthology.orchestra.run.smithi143.stdout: "error", 2023-11-26T07:23:10.300 INFO:teuthology.orchestra.run.smithi143.stdout: "info", 2023-11-26T07:23:10.300 INFO:teuthology.orchestra.run.smithi143.stdout: "warning" 2023-11-26T07:23:10.300 INFO:teuthology.orchestra.run.smithi143.stdout: ], 2023-11-26T07:23:10.300 INFO:teuthology.orchestra.run.smithi143.stdout: "desc": "", 2023-11-26T07:23:10.300 INFO:teuthology.orchestra.run.smithi143.stdout: "long_desc": "", 2023-11-26T07:23:10.300 INFO:teuthology.orchestra.run.smithi143.stdout: "tags": [], 2023-11-26T07:23:10.300 INFO:teuthology.orchestra.run.smithi143.stdout: "see_also": [] 2023-11-26T07:23:10.300 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2023-11-26T07:23:10.300 INFO:teuthology.orchestra.run.smithi143.stdout: "log_to_cluster": { 2023-11-26T07:23:10.301 INFO:teuthology.orchestra.run.smithi143.stdout: "name": "log_to_cluster", 2023-11-26T07:23:10.301 INFO:teuthology.orchestra.run.smithi143.stdout: "type": "bool", 2023-11-26T07:23:10.301 INFO:teuthology.orchestra.run.smithi143.stdout: "level": "advanced", 2023-11-26T07:23:10.301 INFO:teuthology.orchestra.run.smithi143.stdout: "flags": 1, 2023-11-26T07:23:10.301 INFO:teuthology.orchestra.run.smithi143.stdout: "default_value": "False", 2023-11-26T07:23:10.301 INFO:teuthology.orchestra.run.smithi143.stdout: "min": "", 2023-11-26T07:23:10.301 INFO:teuthology.orchestra.run.smithi143.stdout: "max": "", 2023-11-26T07:23:10.301 INFO:teuthology.orchestra.run.smithi143.stdout: "enum_allowed": [], 2023-11-26T07:23:10.301 INFO:teuthology.orchestra.run.smithi143.stdout: "desc": "", 2023-11-26T07:23:10.301 INFO:teuthology.orchestra.run.smithi143.stdout: "long_desc": "", 2023-11-26T07:23:10.301 INFO:teuthology.orchestra.run.smithi143.stdout: "tags": [], 2023-11-26T07:23:10.301 INFO:teuthology.orchestra.run.smithi143.stdout: "see_also": [] 2023-11-26T07:23:10.302 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2023-11-26T07:23:10.302 INFO:teuthology.orchestra.run.smithi143.stdout: "log_to_cluster_level": { 2023-11-26T07:23:10.302 INFO:teuthology.orchestra.run.smithi143.stdout: "name": "log_to_cluster_level", 2023-11-26T07:23:10.302 INFO:teuthology.orchestra.run.smithi143.stdout: "type": "str", 2023-11-26T07:23:10.302 INFO:teuthology.orchestra.run.smithi143.stdout: "level": "advanced", 2023-11-26T07:23:10.302 INFO:teuthology.orchestra.run.smithi143.stdout: "flags": 1, 2023-11-26T07:23:10.302 INFO:teuthology.orchestra.run.smithi143.stdout: "default_value": "info", 2023-11-26T07:23:10.302 INFO:teuthology.orchestra.run.smithi143.stdout: "min": "", 2023-11-26T07:23:10.302 INFO:teuthology.orchestra.run.smithi143.stdout: "max": "", 2023-11-26T07:23:10.302 INFO:teuthology.orchestra.run.smithi143.stdout: "enum_allowed": [ 2023-11-26T07:23:10.302 INFO:teuthology.orchestra.run.smithi143.stdout: "", 2023-11-26T07:23:10.302 INFO:teuthology.orchestra.run.smithi143.stdout: "critical", 2023-11-26T07:23:10.302 INFO:teuthology.orchestra.run.smithi143.stdout: "debug", 2023-11-26T07:23:10.303 INFO:teuthology.orchestra.run.smithi143.stdout: "error", 2023-11-26T07:23:10.303 INFO:teuthology.orchestra.run.smithi143.stdout: "info", 2023-11-26T07:23:10.303 INFO:teuthology.orchestra.run.smithi143.stdout: "warning" 2023-11-26T07:23:10.303 INFO:teuthology.orchestra.run.smithi143.stdout: ], 2023-11-26T07:23:10.303 INFO:teuthology.orchestra.run.smithi143.stdout: "desc": "", 2023-11-26T07:23:10.303 INFO:teuthology.orchestra.run.smithi143.stdout: "long_desc": "", 2023-11-26T07:23:10.303 INFO:teuthology.orchestra.run.smithi143.stdout: "tags": [], 2023-11-26T07:23:10.303 INFO:teuthology.orchestra.run.smithi143.stdout: "see_also": [] 2023-11-26T07:23:10.303 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2023-11-26T07:23:10.303 INFO:teuthology.orchestra.run.smithi143.stdout: "log_to_file": { 2023-11-26T07:23:10.303 INFO:teuthology.orchestra.run.smithi143.stdout: "name": "log_to_file", 2023-11-26T07:23:10.303 INFO:teuthology.orchestra.run.smithi143.stdout: "type": "bool", 2023-11-26T07:23:10.304 INFO:teuthology.orchestra.run.smithi143.stdout: "level": "advanced", 2023-11-26T07:23:10.304 INFO:teuthology.orchestra.run.smithi143.stdout: "flags": 1, 2023-11-26T07:23:10.304 INFO:teuthology.orchestra.run.smithi143.stdout: "default_value": "False", 2023-11-26T07:23:10.304 INFO:teuthology.orchestra.run.smithi143.stdout: "min": "", 2023-11-26T07:23:10.304 INFO:teuthology.orchestra.run.smithi143.stdout: "max": "", 2023-11-26T07:23:10.304 INFO:teuthology.orchestra.run.smithi143.stdout: "enum_allowed": [], 2023-11-26T07:23:10.304 INFO:teuthology.orchestra.run.smithi143.stdout: "desc": "", 2023-11-26T07:23:10.304 INFO:teuthology.orchestra.run.smithi143.stdout: "long_desc": "", 2023-11-26T07:23:10.304 INFO:teuthology.orchestra.run.smithi143.stdout: "tags": [], 2023-11-26T07:23:10.304 INFO:teuthology.orchestra.run.smithi143.stdout: "see_also": [] 2023-11-26T07:23:10.304 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2023-11-26T07:23:10.304 INFO:teuthology.orchestra.run.smithi143.stdout: "max_completed_events": { 2023-11-26T07:23:10.304 INFO:teuthology.orchestra.run.smithi143.stdout: "name": "max_completed_events", 2023-11-26T07:23:10.305 INFO:teuthology.orchestra.run.smithi143.stdout: "type": "int", 2023-11-26T07:23:10.305 INFO:teuthology.orchestra.run.smithi143.stdout: "level": "advanced", 2023-11-26T07:23:10.305 INFO:teuthology.orchestra.run.smithi143.stdout: "flags": 1, 2023-11-26T07:23:10.305 INFO:teuthology.orchestra.run.smithi143.stdout: "default_value": "50", 2023-11-26T07:23:10.305 INFO:teuthology.orchestra.run.smithi143.stdout: "min": "", 2023-11-26T07:23:10.305 INFO:teuthology.orchestra.run.smithi143.stdout: "max": "", 2023-11-26T07:23:10.305 INFO:teuthology.orchestra.run.smithi143.stdout: "enum_allowed": [], 2023-11-26T07:23:10.305 INFO:teuthology.orchestra.run.smithi143.stdout: "desc": "number of past completed events to remember", 2023-11-26T07:23:10.305 INFO:teuthology.orchestra.run.smithi143.stdout: "long_desc": "", 2023-11-26T07:23:10.305 INFO:teuthology.orchestra.run.smithi143.stdout: "tags": [], 2023-11-26T07:23:10.305 INFO:teuthology.orchestra.run.smithi143.stdout: "see_also": [] 2023-11-26T07:23:10.305 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2023-11-26T07:23:10.305 INFO:teuthology.orchestra.run.smithi143.stdout: "sleep_interval": { 2023-11-26T07:23:10.306 INFO:teuthology.orchestra.run.smithi143.stdout: "name": "sleep_interval", 2023-11-26T07:23:10.306 INFO:teuthology.orchestra.run.smithi143.stdout: "type": "secs", 2023-11-26T07:23:10.306 INFO:teuthology.orchestra.run.smithi143.stdout: "level": "advanced", 2023-11-26T07:23:10.306 INFO:teuthology.orchestra.run.smithi143.stdout: "flags": 1, 2023-11-26T07:23:10.306 INFO:teuthology.orchestra.run.smithi143.stdout: "default_value": "5", 2023-11-26T07:23:10.306 INFO:teuthology.orchestra.run.smithi143.stdout: "min": "", 2023-11-26T07:23:10.306 INFO:teuthology.orchestra.run.smithi143.stdout: "max": "", 2023-11-26T07:23:10.306 INFO:teuthology.orchestra.run.smithi143.stdout: "enum_allowed": [], 2023-11-26T07:23:10.306 INFO:teuthology.orchestra.run.smithi143.stdout: "desc": "how long the module is going to sleep", 2023-11-26T07:23:10.306 INFO:teuthology.orchestra.run.smithi143.stdout: "long_desc": "", 2023-11-26T07:23:10.306 INFO:teuthology.orchestra.run.smithi143.stdout: "tags": [], 2023-11-26T07:23:10.306 INFO:teuthology.orchestra.run.smithi143.stdout: "see_also": [] 2023-11-26T07:23:10.307 INFO:teuthology.orchestra.run.smithi143.stdout: } 2023-11-26T07:23:10.307 INFO:teuthology.orchestra.run.smithi143.stdout: } 2023-11-26T07:23:10.307 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2023-11-26T07:23:10.307 INFO:teuthology.orchestra.run.smithi143.stdout: { 2023-11-26T07:23:10.307 INFO:teuthology.orchestra.run.smithi143.stdout: "name": "prometheus", 2023-11-26T07:23:10.307 INFO:teuthology.orchestra.run.smithi143.stdout: "can_run": true, 2023-11-26T07:23:10.307 INFO:teuthology.orchestra.run.smithi143.stdout: "error_string": "", 2023-11-26T07:23:10.307 INFO:teuthology.orchestra.run.smithi143.stdout: "module_options": { 2023-11-26T07:23:10.307 INFO:teuthology.orchestra.run.smithi143.stdout: "cache": { 2023-11-26T07:23:10.307 INFO:teuthology.orchestra.run.smithi143.stdout: "name": "cache", 2023-11-26T07:23:10.307 INFO:teuthology.orchestra.run.smithi143.stdout: "type": "bool", 2023-11-26T07:23:10.307 INFO:teuthology.orchestra.run.smithi143.stdout: "level": "advanced", 2023-11-26T07:23:10.307 INFO:teuthology.orchestra.run.smithi143.stdout: "flags": 0, 2023-11-26T07:23:10.308 INFO:teuthology.orchestra.run.smithi143.stdout: "default_value": "True", 2023-11-26T07:23:10.308 INFO:teuthology.orchestra.run.smithi143.stdout: "min": "", 2023-11-26T07:23:10.308 INFO:teuthology.orchestra.run.smithi143.stdout: "max": "", 2023-11-26T07:23:10.308 INFO:teuthology.orchestra.run.smithi143.stdout: "enum_allowed": [], 2023-11-26T07:23:10.308 INFO:teuthology.orchestra.run.smithi143.stdout: "desc": "", 2023-11-26T07:23:10.308 INFO:teuthology.orchestra.run.smithi143.stdout: "long_desc": "", 2023-11-26T07:23:10.308 INFO:teuthology.orchestra.run.smithi143.stdout: "tags": [], 2023-11-26T07:23:10.308 INFO:teuthology.orchestra.run.smithi143.stdout: "see_also": [] 2023-11-26T07:23:10.308 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2023-11-26T07:23:10.308 INFO:teuthology.orchestra.run.smithi143.stdout: "exclude_perf_counters": { 2023-11-26T07:23:10.308 INFO:teuthology.orchestra.run.smithi143.stdout: "name": "exclude_perf_counters", 2023-11-26T07:23:10.308 INFO:teuthology.orchestra.run.smithi143.stdout: "type": "bool", 2023-11-26T07:23:10.308 INFO:teuthology.orchestra.run.smithi143.stdout: "level": "advanced", 2023-11-26T07:23:10.309 INFO:teuthology.orchestra.run.smithi143.stdout: "flags": 1, 2023-11-26T07:23:10.309 INFO:teuthology.orchestra.run.smithi143.stdout: "default_value": "True", 2023-11-26T07:23:10.309 INFO:teuthology.orchestra.run.smithi143.stdout: "min": "", 2023-11-26T07:23:10.309 INFO:teuthology.orchestra.run.smithi143.stdout: "max": "", 2023-11-26T07:23:10.309 INFO:teuthology.orchestra.run.smithi143.stdout: "enum_allowed": [], 2023-11-26T07:23:10.309 INFO:teuthology.orchestra.run.smithi143.stdout: "desc": "Do not include perf-counters in the metrics output", 2023-11-26T07:23:10.309 INFO:teuthology.orchestra.run.smithi143.stdout: "long_desc": "Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.", 2023-11-26T07:23:10.309 INFO:teuthology.orchestra.run.smithi143.stdout: "tags": [], 2023-11-26T07:23:10.309 INFO:teuthology.orchestra.run.smithi143.stdout: "see_also": [] 2023-11-26T07:23:10.309 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2023-11-26T07:23:10.309 INFO:teuthology.orchestra.run.smithi143.stdout: "log_level": { 2023-11-26T07:23:10.309 INFO:teuthology.orchestra.run.smithi143.stdout: "name": "log_level", 2023-11-26T07:23:10.310 INFO:teuthology.orchestra.run.smithi143.stdout: "type": "str", 2023-11-26T07:23:10.310 INFO:teuthology.orchestra.run.smithi143.stdout: "level": "advanced", 2023-11-26T07:23:10.310 INFO:teuthology.orchestra.run.smithi143.stdout: "flags": 1, 2023-11-26T07:23:10.310 INFO:teuthology.orchestra.run.smithi143.stdout: "default_value": "", 2023-11-26T07:23:10.310 INFO:teuthology.orchestra.run.smithi143.stdout: "min": "", 2023-11-26T07:23:10.310 INFO:teuthology.orchestra.run.smithi143.stdout: "max": "", 2023-11-26T07:23:10.310 INFO:teuthology.orchestra.run.smithi143.stdout: "enum_allowed": [ 2023-11-26T07:23:10.310 INFO:teuthology.orchestra.run.smithi143.stdout: "", 2023-11-26T07:23:10.310 INFO:teuthology.orchestra.run.smithi143.stdout: "critical", 2023-11-26T07:23:10.311 INFO:teuthology.orchestra.run.smithi143.stdout: "debug", 2023-11-26T07:23:10.311 INFO:teuthology.orchestra.run.smithi143.stdout: "error", 2023-11-26T07:23:10.311 INFO:teuthology.orchestra.run.smithi143.stdout: "info", 2023-11-26T07:23:10.311 INFO:teuthology.orchestra.run.smithi143.stdout: "warning" 2023-11-26T07:23:10.311 INFO:teuthology.orchestra.run.smithi143.stdout: ], 2023-11-26T07:23:10.311 INFO:teuthology.orchestra.run.smithi143.stdout: "desc": "", 2023-11-26T07:23:10.311 INFO:teuthology.orchestra.run.smithi143.stdout: "long_desc": "", 2023-11-26T07:23:10.311 INFO:teuthology.orchestra.run.smithi143.stdout: "tags": [], 2023-11-26T07:23:10.311 INFO:teuthology.orchestra.run.smithi143.stdout: "see_also": [] 2023-11-26T07:23:10.311 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2023-11-26T07:23:10.311 INFO:teuthology.orchestra.run.smithi143.stdout: "log_to_cluster": { 2023-11-26T07:23:10.311 INFO:teuthology.orchestra.run.smithi143.stdout: "name": "log_to_cluster", 2023-11-26T07:23:10.311 INFO:teuthology.orchestra.run.smithi143.stdout: "type": "bool", 2023-11-26T07:23:10.312 INFO:teuthology.orchestra.run.smithi143.stdout: "level": "advanced", 2023-11-26T07:23:10.312 INFO:teuthology.orchestra.run.smithi143.stdout: "flags": 1, 2023-11-26T07:23:10.312 INFO:teuthology.orchestra.run.smithi143.stdout: "default_value": "False", 2023-11-26T07:23:10.312 INFO:teuthology.orchestra.run.smithi143.stdout: "min": "", 2023-11-26T07:23:10.312 INFO:teuthology.orchestra.run.smithi143.stdout: "max": "", 2023-11-26T07:23:10.312 INFO:teuthology.orchestra.run.smithi143.stdout: "enum_allowed": [], 2023-11-26T07:23:10.312 INFO:teuthology.orchestra.run.smithi143.stdout: "desc": "", 2023-11-26T07:23:10.312 INFO:teuthology.orchestra.run.smithi143.stdout: "long_desc": "", 2023-11-26T07:23:10.312 INFO:teuthology.orchestra.run.smithi143.stdout: "tags": [], 2023-11-26T07:23:10.312 INFO:teuthology.orchestra.run.smithi143.stdout: "see_also": [] 2023-11-26T07:23:10.312 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2023-11-26T07:23:10.312 INFO:teuthology.orchestra.run.smithi143.stdout: "log_to_cluster_level": { 2023-11-26T07:23:10.313 INFO:teuthology.orchestra.run.smithi143.stdout: "name": "log_to_cluster_level", 2023-11-26T07:23:10.313 INFO:teuthology.orchestra.run.smithi143.stdout: "type": "str", 2023-11-26T07:23:10.313 INFO:teuthology.orchestra.run.smithi143.stdout: "level": "advanced", 2023-11-26T07:23:10.313 INFO:teuthology.orchestra.run.smithi143.stdout: "flags": 1, 2023-11-26T07:23:10.313 INFO:teuthology.orchestra.run.smithi143.stdout: "default_value": "info", 2023-11-26T07:23:10.313 INFO:teuthology.orchestra.run.smithi143.stdout: "min": "", 2023-11-26T07:23:10.313 INFO:teuthology.orchestra.run.smithi143.stdout: "max": "", 2023-11-26T07:23:10.313 INFO:teuthology.orchestra.run.smithi143.stdout: "enum_allowed": [ 2023-11-26T07:23:10.313 INFO:teuthology.orchestra.run.smithi143.stdout: "", 2023-11-26T07:23:10.313 INFO:teuthology.orchestra.run.smithi143.stdout: "critical", 2023-11-26T07:23:10.313 INFO:teuthology.orchestra.run.smithi143.stdout: "debug", 2023-11-26T07:23:10.313 INFO:teuthology.orchestra.run.smithi143.stdout: "error", 2023-11-26T07:23:10.313 INFO:teuthology.orchestra.run.smithi143.stdout: "info", 2023-11-26T07:23:10.314 INFO:teuthology.orchestra.run.smithi143.stdout: "warning" 2023-11-26T07:23:10.314 INFO:teuthology.orchestra.run.smithi143.stdout: ], 2023-11-26T07:23:10.314 INFO:teuthology.orchestra.run.smithi143.stdout: "desc": "", 2023-11-26T07:23:10.314 INFO:teuthology.orchestra.run.smithi143.stdout: "long_desc": "", 2023-11-26T07:23:10.314 INFO:teuthology.orchestra.run.smithi143.stdout: "tags": [], 2023-11-26T07:23:10.314 INFO:teuthology.orchestra.run.smithi143.stdout: "see_also": [] 2023-11-26T07:23:10.314 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2023-11-26T07:23:10.314 INFO:teuthology.orchestra.run.smithi143.stdout: "log_to_file": { 2023-11-26T07:23:10.314 INFO:teuthology.orchestra.run.smithi143.stdout: "name": "log_to_file", 2023-11-26T07:23:10.314 INFO:teuthology.orchestra.run.smithi143.stdout: "type": "bool", 2023-11-26T07:23:10.314 INFO:teuthology.orchestra.run.smithi143.stdout: "level": "advanced", 2023-11-26T07:23:10.314 INFO:teuthology.orchestra.run.smithi143.stdout: "flags": 1, 2023-11-26T07:23:10.314 INFO:teuthology.orchestra.run.smithi143.stdout: "default_value": "False", 2023-11-26T07:23:10.315 INFO:teuthology.orchestra.run.smithi143.stdout: "min": "", 2023-11-26T07:23:10.315 INFO:teuthology.orchestra.run.smithi143.stdout: "max": "", 2023-11-26T07:23:10.315 INFO:teuthology.orchestra.run.smithi143.stdout: "enum_allowed": [], 2023-11-26T07:23:10.315 INFO:teuthology.orchestra.run.smithi143.stdout: "desc": "", 2023-11-26T07:23:10.315 INFO:teuthology.orchestra.run.smithi143.stdout: "long_desc": "", 2023-11-26T07:23:10.315 INFO:teuthology.orchestra.run.smithi143.stdout: "tags": [], 2023-11-26T07:23:10.315 INFO:teuthology.orchestra.run.smithi143.stdout: "see_also": [] 2023-11-26T07:23:10.315 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2023-11-26T07:23:10.315 INFO:teuthology.orchestra.run.smithi143.stdout: "rbd_stats_pools": { 2023-11-26T07:23:10.315 INFO:teuthology.orchestra.run.smithi143.stdout: "name": "rbd_stats_pools", 2023-11-26T07:23:10.315 INFO:teuthology.orchestra.run.smithi143.stdout: "type": "str", 2023-11-26T07:23:10.315 INFO:teuthology.orchestra.run.smithi143.stdout: "level": "advanced", 2023-11-26T07:23:10.315 INFO:teuthology.orchestra.run.smithi143.stdout: "flags": 0, 2023-11-26T07:23:10.316 INFO:teuthology.orchestra.run.smithi143.stdout: "default_value": "", 2023-11-26T07:23:10.316 INFO:teuthology.orchestra.run.smithi143.stdout: "min": "", 2023-11-26T07:23:10.316 INFO:teuthology.orchestra.run.smithi143.stdout: "max": "", 2023-11-26T07:23:10.316 INFO:teuthology.orchestra.run.smithi143.stdout: "enum_allowed": [], 2023-11-26T07:23:10.316 INFO:teuthology.orchestra.run.smithi143.stdout: "desc": "", 2023-11-26T07:23:10.316 INFO:teuthology.orchestra.run.smithi143.stdout: "long_desc": "", 2023-11-26T07:23:10.316 INFO:teuthology.orchestra.run.smithi143.stdout: "tags": [], 2023-11-26T07:23:10.316 INFO:teuthology.orchestra.run.smithi143.stdout: "see_also": [] 2023-11-26T07:23:10.316 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2023-11-26T07:23:10.316 INFO:teuthology.orchestra.run.smithi143.stdout: "rbd_stats_pools_refresh_interval": { 2023-11-26T07:23:10.316 INFO:teuthology.orchestra.run.smithi143.stdout: "name": "rbd_stats_pools_refresh_interval", 2023-11-26T07:23:10.316 INFO:teuthology.orchestra.run.smithi143.stdout: "type": "int", 2023-11-26T07:23:10.316 INFO:teuthology.orchestra.run.smithi143.stdout: "level": "advanced", 2023-11-26T07:23:10.316 INFO:teuthology.orchestra.run.smithi143.stdout: "flags": 0, 2023-11-26T07:23:10.317 INFO:teuthology.orchestra.run.smithi143.stdout: "default_value": "300", 2023-11-26T07:23:10.317 INFO:teuthology.orchestra.run.smithi143.stdout: "min": "", 2023-11-26T07:23:10.317 INFO:teuthology.orchestra.run.smithi143.stdout: "max": "", 2023-11-26T07:23:10.317 INFO:teuthology.orchestra.run.smithi143.stdout: "enum_allowed": [], 2023-11-26T07:23:10.317 INFO:teuthology.orchestra.run.smithi143.stdout: "desc": "", 2023-11-26T07:23:10.317 INFO:teuthology.orchestra.run.smithi143.stdout: "long_desc": "", 2023-11-26T07:23:10.317 INFO:teuthology.orchestra.run.smithi143.stdout: "tags": [], 2023-11-26T07:23:10.317 INFO:teuthology.orchestra.run.smithi143.stdout: "see_also": [] 2023-11-26T07:23:10.317 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2023-11-26T07:23:10.317 INFO:teuthology.orchestra.run.smithi143.stdout: "scrape_interval": { 2023-11-26T07:23:10.317 INFO:teuthology.orchestra.run.smithi143.stdout: "name": "scrape_interval", 2023-11-26T07:23:10.317 INFO:teuthology.orchestra.run.smithi143.stdout: "type": "float", 2023-11-26T07:23:10.317 INFO:teuthology.orchestra.run.smithi143.stdout: "level": "advanced", 2023-11-26T07:23:10.318 INFO:teuthology.orchestra.run.smithi143.stdout: "flags": 0, 2023-11-26T07:23:10.318 INFO:teuthology.orchestra.run.smithi143.stdout: "default_value": "15.0", 2023-11-26T07:23:10.318 INFO:teuthology.orchestra.run.smithi143.stdout: "min": "", 2023-11-26T07:23:10.318 INFO:teuthology.orchestra.run.smithi143.stdout: "max": "", 2023-11-26T07:23:10.318 INFO:teuthology.orchestra.run.smithi143.stdout: "enum_allowed": [], 2023-11-26T07:23:10.318 INFO:teuthology.orchestra.run.smithi143.stdout: "desc": "", 2023-11-26T07:23:10.318 INFO:teuthology.orchestra.run.smithi143.stdout: "long_desc": "", 2023-11-26T07:23:10.318 INFO:teuthology.orchestra.run.smithi143.stdout: "tags": [], 2023-11-26T07:23:10.318 INFO:teuthology.orchestra.run.smithi143.stdout: "see_also": [] 2023-11-26T07:23:10.318 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2023-11-26T07:23:10.318 INFO:teuthology.orchestra.run.smithi143.stdout: "server_addr": { 2023-11-26T07:23:10.318 INFO:teuthology.orchestra.run.smithi143.stdout: "name": "server_addr", 2023-11-26T07:23:10.318 INFO:teuthology.orchestra.run.smithi143.stdout: "type": "str", 2023-11-26T07:23:10.319 INFO:teuthology.orchestra.run.smithi143.stdout: "level": "advanced", 2023-11-26T07:23:10.319 INFO:teuthology.orchestra.run.smithi143.stdout: "flags": 0, 2023-11-26T07:23:10.319 INFO:teuthology.orchestra.run.smithi143.stdout: "default_value": "::", 2023-11-26T07:23:10.319 INFO:teuthology.orchestra.run.smithi143.stdout: "min": "", 2023-11-26T07:23:10.319 INFO:teuthology.orchestra.run.smithi143.stdout: "max": "", 2023-11-26T07:23:10.319 INFO:teuthology.orchestra.run.smithi143.stdout: "enum_allowed": [], 2023-11-26T07:23:10.319 INFO:teuthology.orchestra.run.smithi143.stdout: "desc": "the IPv4 or IPv6 address on which the module listens for HTTP requests", 2023-11-26T07:23:10.319 INFO:teuthology.orchestra.run.smithi143.stdout: "long_desc": "", 2023-11-26T07:23:10.319 INFO:teuthology.orchestra.run.smithi143.stdout: "tags": [], 2023-11-26T07:23:10.319 INFO:teuthology.orchestra.run.smithi143.stdout: "see_also": [] 2023-11-26T07:23:10.319 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2023-11-26T07:23:10.319 INFO:teuthology.orchestra.run.smithi143.stdout: "server_port": { 2023-11-26T07:23:10.319 INFO:teuthology.orchestra.run.smithi143.stdout: "name": "server_port", 2023-11-26T07:23:10.320 INFO:teuthology.orchestra.run.smithi143.stdout: "type": "int", 2023-11-26T07:23:10.320 INFO:teuthology.orchestra.run.smithi143.stdout: "level": "advanced", 2023-11-26T07:23:10.320 INFO:teuthology.orchestra.run.smithi143.stdout: "flags": 1, 2023-11-26T07:23:10.320 INFO:teuthology.orchestra.run.smithi143.stdout: "default_value": "9283", 2023-11-26T07:23:10.320 INFO:teuthology.orchestra.run.smithi143.stdout: "min": "", 2023-11-26T07:23:10.320 INFO:teuthology.orchestra.run.smithi143.stdout: "max": "", 2023-11-26T07:23:10.320 INFO:teuthology.orchestra.run.smithi143.stdout: "enum_allowed": [], 2023-11-26T07:23:10.320 INFO:teuthology.orchestra.run.smithi143.stdout: "desc": "the port on which the module listens for HTTP requests", 2023-11-26T07:23:10.320 INFO:teuthology.orchestra.run.smithi143.stdout: "long_desc": "", 2023-11-26T07:23:10.320 INFO:teuthology.orchestra.run.smithi143.stdout: "tags": [], 2023-11-26T07:23:10.320 INFO:teuthology.orchestra.run.smithi143.stdout: "see_also": [] 2023-11-26T07:23:10.320 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2023-11-26T07:23:10.320 INFO:teuthology.orchestra.run.smithi143.stdout: "stale_cache_strategy": { 2023-11-26T07:23:10.321 INFO:teuthology.orchestra.run.smithi143.stdout: "name": "stale_cache_strategy", 2023-11-26T07:23:10.321 INFO:teuthology.orchestra.run.smithi143.stdout: "type": "str", 2023-11-26T07:23:10.321 INFO:teuthology.orchestra.run.smithi143.stdout: "level": "advanced", 2023-11-26T07:23:10.321 INFO:teuthology.orchestra.run.smithi143.stdout: "flags": 0, 2023-11-26T07:23:10.321 INFO:teuthology.orchestra.run.smithi143.stdout: "default_value": "log", 2023-11-26T07:23:10.321 INFO:teuthology.orchestra.run.smithi143.stdout: "min": "", 2023-11-26T07:23:10.321 INFO:teuthology.orchestra.run.smithi143.stdout: "max": "", 2023-11-26T07:23:10.321 INFO:teuthology.orchestra.run.smithi143.stdout: "enum_allowed": [], 2023-11-26T07:23:10.321 INFO:teuthology.orchestra.run.smithi143.stdout: "desc": "", 2023-11-26T07:23:10.321 INFO:teuthology.orchestra.run.smithi143.stdout: "long_desc": "", 2023-11-26T07:23:10.321 INFO:teuthology.orchestra.run.smithi143.stdout: "tags": [], 2023-11-26T07:23:10.321 INFO:teuthology.orchestra.run.smithi143.stdout: "see_also": [] 2023-11-26T07:23:10.321 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2023-11-26T07:23:10.322 INFO:teuthology.orchestra.run.smithi143.stdout: "standby_behaviour": { 2023-11-26T07:23:10.322 INFO:teuthology.orchestra.run.smithi143.stdout: "name": "standby_behaviour", 2023-11-26T07:23:10.322 INFO:teuthology.orchestra.run.smithi143.stdout: "type": "str", 2023-11-26T07:23:10.322 INFO:teuthology.orchestra.run.smithi143.stdout: "level": "advanced", 2023-11-26T07:23:10.322 INFO:teuthology.orchestra.run.smithi143.stdout: "flags": 1, 2023-11-26T07:23:10.322 INFO:teuthology.orchestra.run.smithi143.stdout: "default_value": "default", 2023-11-26T07:23:10.322 INFO:teuthology.orchestra.run.smithi143.stdout: "min": "", 2023-11-26T07:23:10.322 INFO:teuthology.orchestra.run.smithi143.stdout: "max": "", 2023-11-26T07:23:10.322 INFO:teuthology.orchestra.run.smithi143.stdout: "enum_allowed": [ 2023-11-26T07:23:10.322 INFO:teuthology.orchestra.run.smithi143.stdout: "default", 2023-11-26T07:23:10.322 INFO:teuthology.orchestra.run.smithi143.stdout: "error" 2023-11-26T07:23:10.322 INFO:teuthology.orchestra.run.smithi143.stdout: ], 2023-11-26T07:23:10.323 INFO:teuthology.orchestra.run.smithi143.stdout: "desc": "", 2023-11-26T07:23:10.323 INFO:teuthology.orchestra.run.smithi143.stdout: "long_desc": "", 2023-11-26T07:23:10.323 INFO:teuthology.orchestra.run.smithi143.stdout: "tags": [], 2023-11-26T07:23:10.323 INFO:teuthology.orchestra.run.smithi143.stdout: "see_also": [] 2023-11-26T07:23:10.323 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2023-11-26T07:23:10.323 INFO:teuthology.orchestra.run.smithi143.stdout: "standby_error_status_code": { 2023-11-26T07:23:10.323 INFO:teuthology.orchestra.run.smithi143.stdout: "name": "standby_error_status_code", 2023-11-26T07:23:10.323 INFO:teuthology.orchestra.run.smithi143.stdout: "type": "int", 2023-11-26T07:23:10.323 INFO:teuthology.orchestra.run.smithi143.stdout: "level": "advanced", 2023-11-26T07:23:10.323 INFO:teuthology.orchestra.run.smithi143.stdout: "flags": 1, 2023-11-26T07:23:10.323 INFO:teuthology.orchestra.run.smithi143.stdout: "default_value": "500", 2023-11-26T07:23:10.323 INFO:teuthology.orchestra.run.smithi143.stdout: "min": "400", 2023-11-26T07:23:10.323 INFO:teuthology.orchestra.run.smithi143.stdout: "max": "599", 2023-11-26T07:23:10.324 INFO:teuthology.orchestra.run.smithi143.stdout: "enum_allowed": [], 2023-11-26T07:23:10.324 INFO:teuthology.orchestra.run.smithi143.stdout: "desc": "", 2023-11-26T07:23:10.324 INFO:teuthology.orchestra.run.smithi143.stdout: "long_desc": "", 2023-11-26T07:23:10.324 INFO:teuthology.orchestra.run.smithi143.stdout: "tags": [], 2023-11-26T07:23:10.324 INFO:teuthology.orchestra.run.smithi143.stdout: "see_also": [] 2023-11-26T07:23:10.324 INFO:teuthology.orchestra.run.smithi143.stdout: } 2023-11-26T07:23:10.324 INFO:teuthology.orchestra.run.smithi143.stdout: } 2023-11-26T07:23:10.324 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2023-11-26T07:23:10.324 INFO:teuthology.orchestra.run.smithi143.stdout: { 2023-11-26T07:23:10.324 INFO:teuthology.orchestra.run.smithi143.stdout: "name": "rbd_support", 2023-11-26T07:23:10.324 INFO:teuthology.orchestra.run.smithi143.stdout: "can_run": true, 2023-11-26T07:23:10.324 INFO:teuthology.orchestra.run.smithi143.stdout: "error_string": "", 2023-11-26T07:23:10.324 INFO:teuthology.orchestra.run.smithi143.stdout: "module_options": { 2023-11-26T07:23:10.325 INFO:teuthology.orchestra.run.smithi143.stdout: "log_level": { 2023-11-26T07:23:10.325 INFO:teuthology.orchestra.run.smithi143.stdout: "name": "log_level", 2023-11-26T07:23:10.325 INFO:teuthology.orchestra.run.smithi143.stdout: "type": "str", 2023-11-26T07:23:10.325 INFO:teuthology.orchestra.run.smithi143.stdout: "level": "advanced", 2023-11-26T07:23:10.325 INFO:teuthology.orchestra.run.smithi143.stdout: "flags": 1, 2023-11-26T07:23:10.325 INFO:teuthology.orchestra.run.smithi143.stdout: "default_value": "", 2023-11-26T07:23:10.325 INFO:teuthology.orchestra.run.smithi143.stdout: "min": "", 2023-11-26T07:23:10.325 INFO:teuthology.orchestra.run.smithi143.stdout: "max": "", 2023-11-26T07:23:10.325 INFO:teuthology.orchestra.run.smithi143.stdout: "enum_allowed": [ 2023-11-26T07:23:10.325 INFO:teuthology.orchestra.run.smithi143.stdout: "", 2023-11-26T07:23:10.325 INFO:teuthology.orchestra.run.smithi143.stdout: "critical", 2023-11-26T07:23:10.325 INFO:teuthology.orchestra.run.smithi143.stdout: "debug", 2023-11-26T07:23:10.325 INFO:teuthology.orchestra.run.smithi143.stdout: "error", 2023-11-26T07:23:10.326 INFO:teuthology.orchestra.run.smithi143.stdout: "info", 2023-11-26T07:23:10.326 INFO:teuthology.orchestra.run.smithi143.stdout: "warning" 2023-11-26T07:23:10.326 INFO:teuthology.orchestra.run.smithi143.stdout: ], 2023-11-26T07:23:10.326 INFO:teuthology.orchestra.run.smithi143.stdout: "desc": "", 2023-11-26T07:23:10.326 INFO:teuthology.orchestra.run.smithi143.stdout: "long_desc": "", 2023-11-26T07:23:10.326 INFO:teuthology.orchestra.run.smithi143.stdout: "tags": [], 2023-11-26T07:23:10.326 INFO:teuthology.orchestra.run.smithi143.stdout: "see_also": [] 2023-11-26T07:23:10.326 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2023-11-26T07:23:10.326 INFO:teuthology.orchestra.run.smithi143.stdout: "log_to_cluster": { 2023-11-26T07:23:10.326 INFO:teuthology.orchestra.run.smithi143.stdout: "name": "log_to_cluster", 2023-11-26T07:23:10.326 INFO:teuthology.orchestra.run.smithi143.stdout: "type": "bool", 2023-11-26T07:23:10.326 INFO:teuthology.orchestra.run.smithi143.stdout: "level": "advanced", 2023-11-26T07:23:10.327 INFO:teuthology.orchestra.run.smithi143.stdout: "flags": 1, 2023-11-26T07:23:10.327 INFO:teuthology.orchestra.run.smithi143.stdout: "default_value": "False", 2023-11-26T07:23:10.327 INFO:teuthology.orchestra.run.smithi143.stdout: "min": "", 2023-11-26T07:23:10.327 INFO:teuthology.orchestra.run.smithi143.stdout: "max": "", 2023-11-26T07:23:10.327 INFO:teuthology.orchestra.run.smithi143.stdout: "enum_allowed": [], 2023-11-26T07:23:10.327 INFO:teuthology.orchestra.run.smithi143.stdout: "desc": "", 2023-11-26T07:23:10.327 INFO:teuthology.orchestra.run.smithi143.stdout: "long_desc": "", 2023-11-26T07:23:10.327 INFO:teuthology.orchestra.run.smithi143.stdout: "tags": [], 2023-11-26T07:23:10.327 INFO:teuthology.orchestra.run.smithi143.stdout: "see_also": [] 2023-11-26T07:23:10.327 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2023-11-26T07:23:10.327 INFO:teuthology.orchestra.run.smithi143.stdout: "log_to_cluster_level": { 2023-11-26T07:23:10.327 INFO:teuthology.orchestra.run.smithi143.stdout: "name": "log_to_cluster_level", 2023-11-26T07:23:10.327 INFO:teuthology.orchestra.run.smithi143.stdout: "type": "str", 2023-11-26T07:23:10.328 INFO:teuthology.orchestra.run.smithi143.stdout: "level": "advanced", 2023-11-26T07:23:10.328 INFO:teuthology.orchestra.run.smithi143.stdout: "flags": 1, 2023-11-26T07:23:10.328 INFO:teuthology.orchestra.run.smithi143.stdout: "default_value": "info", 2023-11-26T07:23:10.328 INFO:teuthology.orchestra.run.smithi143.stdout: "min": "", 2023-11-26T07:23:10.328 INFO:teuthology.orchestra.run.smithi143.stdout: "max": "", 2023-11-26T07:23:10.328 INFO:teuthology.orchestra.run.smithi143.stdout: "enum_allowed": [ 2023-11-26T07:23:10.328 INFO:teuthology.orchestra.run.smithi143.stdout: "", 2023-11-26T07:23:10.328 INFO:teuthology.orchestra.run.smithi143.stdout: "critical", 2023-11-26T07:23:10.328 INFO:teuthology.orchestra.run.smithi143.stdout: "debug", 2023-11-26T07:23:10.328 INFO:teuthology.orchestra.run.smithi143.stdout: "error", 2023-11-26T07:23:10.328 INFO:teuthology.orchestra.run.smithi143.stdout: "info", 2023-11-26T07:23:10.329 INFO:teuthology.orchestra.run.smithi143.stdout: "warning" 2023-11-26T07:23:10.329 INFO:teuthology.orchestra.run.smithi143.stdout: ], 2023-11-26T07:23:10.329 INFO:teuthology.orchestra.run.smithi143.stdout: "desc": "", 2023-11-26T07:23:10.329 INFO:teuthology.orchestra.run.smithi143.stdout: "long_desc": "", 2023-11-26T07:23:10.329 INFO:teuthology.orchestra.run.smithi143.stdout: "tags": [], 2023-11-26T07:23:10.329 INFO:teuthology.orchestra.run.smithi143.stdout: "see_also": [] 2023-11-26T07:23:10.329 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2023-11-26T07:23:10.329 INFO:teuthology.orchestra.run.smithi143.stdout: "log_to_file": { 2023-11-26T07:23:10.329 INFO:teuthology.orchestra.run.smithi143.stdout: "name": "log_to_file", 2023-11-26T07:23:10.329 INFO:teuthology.orchestra.run.smithi143.stdout: "type": "bool", 2023-11-26T07:23:10.329 INFO:teuthology.orchestra.run.smithi143.stdout: "level": "advanced", 2023-11-26T07:23:10.330 INFO:teuthology.orchestra.run.smithi143.stdout: "flags": 1, 2023-11-26T07:23:10.330 INFO:teuthology.orchestra.run.smithi143.stdout: "default_value": "False", 2023-11-26T07:23:10.330 INFO:teuthology.orchestra.run.smithi143.stdout: "min": "", 2023-11-26T07:23:10.330 INFO:teuthology.orchestra.run.smithi143.stdout: "max": "", 2023-11-26T07:23:10.330 INFO:teuthology.orchestra.run.smithi143.stdout: "enum_allowed": [], 2023-11-26T07:23:10.330 INFO:teuthology.orchestra.run.smithi143.stdout: "desc": "", 2023-11-26T07:23:10.330 INFO:teuthology.orchestra.run.smithi143.stdout: "long_desc": "", 2023-11-26T07:23:10.330 INFO:teuthology.orchestra.run.smithi143.stdout: "tags": [], 2023-11-26T07:23:10.330 INFO:teuthology.orchestra.run.smithi143.stdout: "see_also": [] 2023-11-26T07:23:10.330 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2023-11-26T07:23:10.330 INFO:teuthology.orchestra.run.smithi143.stdout: "max_concurrent_snap_create": { 2023-11-26T07:23:10.331 INFO:teuthology.orchestra.run.smithi143.stdout: "name": "max_concurrent_snap_create", 2023-11-26T07:23:10.331 INFO:teuthology.orchestra.run.smithi143.stdout: "type": "int", 2023-11-26T07:23:10.331 INFO:teuthology.orchestra.run.smithi143.stdout: "level": "advanced", 2023-11-26T07:23:10.331 INFO:teuthology.orchestra.run.smithi143.stdout: "flags": 0, 2023-11-26T07:23:10.331 INFO:teuthology.orchestra.run.smithi143.stdout: "default_value": "10", 2023-11-26T07:23:10.331 INFO:teuthology.orchestra.run.smithi143.stdout: "min": "", 2023-11-26T07:23:10.331 INFO:teuthology.orchestra.run.smithi143.stdout: "max": "", 2023-11-26T07:23:10.331 INFO:teuthology.orchestra.run.smithi143.stdout: "enum_allowed": [], 2023-11-26T07:23:10.331 INFO:teuthology.orchestra.run.smithi143.stdout: "desc": "", 2023-11-26T07:23:10.331 INFO:teuthology.orchestra.run.smithi143.stdout: "long_desc": "", 2023-11-26T07:23:10.331 INFO:teuthology.orchestra.run.smithi143.stdout: "tags": [], 2023-11-26T07:23:10.331 INFO:teuthology.orchestra.run.smithi143.stdout: "see_also": [] 2023-11-26T07:23:10.331 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2023-11-26T07:23:10.332 INFO:teuthology.orchestra.run.smithi143.stdout: "mirror_snapshot_schedule": { 2023-11-26T07:23:10.332 INFO:teuthology.orchestra.run.smithi143.stdout: "name": "mirror_snapshot_schedule", 2023-11-26T07:23:10.332 INFO:teuthology.orchestra.run.smithi143.stdout: "type": "str", 2023-11-26T07:23:10.332 INFO:teuthology.orchestra.run.smithi143.stdout: "level": "advanced", 2023-11-26T07:23:10.332 INFO:teuthology.orchestra.run.smithi143.stdout: "flags": 0, 2023-11-26T07:23:10.332 INFO:teuthology.orchestra.run.smithi143.stdout: "default_value": "", 2023-11-26T07:23:10.332 INFO:teuthology.orchestra.run.smithi143.stdout: "min": "", 2023-11-26T07:23:10.332 INFO:teuthology.orchestra.run.smithi143.stdout: "max": "", 2023-11-26T07:23:10.332 INFO:teuthology.orchestra.run.smithi143.stdout: "enum_allowed": [], 2023-11-26T07:23:10.332 INFO:teuthology.orchestra.run.smithi143.stdout: "desc": "", 2023-11-26T07:23:10.332 INFO:teuthology.orchestra.run.smithi143.stdout: "long_desc": "", 2023-11-26T07:23:10.332 INFO:teuthology.orchestra.run.smithi143.stdout: "tags": [], 2023-11-26T07:23:10.332 INFO:teuthology.orchestra.run.smithi143.stdout: "see_also": [] 2023-11-26T07:23:10.332 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2023-11-26T07:23:10.333 INFO:teuthology.orchestra.run.smithi143.stdout: "trash_purge_schedule": { 2023-11-26T07:23:10.333 INFO:teuthology.orchestra.run.smithi143.stdout: "name": "trash_purge_schedule", 2023-11-26T07:23:10.333 INFO:teuthology.orchestra.run.smithi143.stdout: "type": "str", 2023-11-26T07:23:10.333 INFO:teuthology.orchestra.run.smithi143.stdout: "level": "advanced", 2023-11-26T07:23:10.333 INFO:teuthology.orchestra.run.smithi143.stdout: "flags": 0, 2023-11-26T07:23:10.333 INFO:teuthology.orchestra.run.smithi143.stdout: "default_value": "", 2023-11-26T07:23:10.333 INFO:teuthology.orchestra.run.smithi143.stdout: "min": "", 2023-11-26T07:23:10.333 INFO:teuthology.orchestra.run.smithi143.stdout: "max": "", 2023-11-26T07:23:10.333 INFO:teuthology.orchestra.run.smithi143.stdout: "enum_allowed": [], 2023-11-26T07:23:10.333 INFO:teuthology.orchestra.run.smithi143.stdout: "desc": "", 2023-11-26T07:23:10.333 INFO:teuthology.orchestra.run.smithi143.stdout: "long_desc": "", 2023-11-26T07:23:10.333 INFO:teuthology.orchestra.run.smithi143.stdout: "tags": [], 2023-11-26T07:23:10.333 INFO:teuthology.orchestra.run.smithi143.stdout: "see_also": [] 2023-11-26T07:23:10.333 INFO:teuthology.orchestra.run.smithi143.stdout: } 2023-11-26T07:23:10.334 INFO:teuthology.orchestra.run.smithi143.stdout: } 2023-11-26T07:23:10.334 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2023-11-26T07:23:10.334 INFO:teuthology.orchestra.run.smithi143.stdout: { 2023-11-26T07:23:10.334 INFO:teuthology.orchestra.run.smithi143.stdout: "name": "restful", 2023-11-26T07:23:10.334 INFO:teuthology.orchestra.run.smithi143.stdout: "can_run": true, 2023-11-26T07:23:10.334 INFO:teuthology.orchestra.run.smithi143.stdout: "error_string": "", 2023-11-26T07:23:10.334 INFO:teuthology.orchestra.run.smithi143.stdout: "module_options": { 2023-11-26T07:23:10.334 INFO:teuthology.orchestra.run.smithi143.stdout: "enable_auth": { 2023-11-26T07:23:10.334 INFO:teuthology.orchestra.run.smithi143.stdout: "name": "enable_auth", 2023-11-26T07:23:10.334 INFO:teuthology.orchestra.run.smithi143.stdout: "type": "bool", 2023-11-26T07:23:10.334 INFO:teuthology.orchestra.run.smithi143.stdout: "level": "advanced", 2023-11-26T07:23:10.334 INFO:teuthology.orchestra.run.smithi143.stdout: "flags": 0, 2023-11-26T07:23:10.334 INFO:teuthology.orchestra.run.smithi143.stdout: "default_value": "True", 2023-11-26T07:23:10.335 INFO:teuthology.orchestra.run.smithi143.stdout: "min": "", 2023-11-26T07:23:10.335 INFO:teuthology.orchestra.run.smithi143.stdout: "max": "", 2023-11-26T07:23:10.335 INFO:teuthology.orchestra.run.smithi143.stdout: "enum_allowed": [], 2023-11-26T07:23:10.335 INFO:teuthology.orchestra.run.smithi143.stdout: "desc": "", 2023-11-26T07:23:10.335 INFO:teuthology.orchestra.run.smithi143.stdout: "long_desc": "", 2023-11-26T07:23:10.335 INFO:teuthology.orchestra.run.smithi143.stdout: "tags": [], 2023-11-26T07:23:10.335 INFO:teuthology.orchestra.run.smithi143.stdout: "see_also": [] 2023-11-26T07:23:10.335 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2023-11-26T07:23:10.335 INFO:teuthology.orchestra.run.smithi143.stdout: "key_file": { 2023-11-26T07:23:10.335 INFO:teuthology.orchestra.run.smithi143.stdout: "name": "key_file", 2023-11-26T07:23:10.335 INFO:teuthology.orchestra.run.smithi143.stdout: "type": "str", 2023-11-26T07:23:10.335 INFO:teuthology.orchestra.run.smithi143.stdout: "level": "advanced", 2023-11-26T07:23:10.335 INFO:teuthology.orchestra.run.smithi143.stdout: "flags": 0, 2023-11-26T07:23:10.335 INFO:teuthology.orchestra.run.smithi143.stdout: "default_value": "", 2023-11-26T07:23:10.336 INFO:teuthology.orchestra.run.smithi143.stdout: "min": "", 2023-11-26T07:23:10.336 INFO:teuthology.orchestra.run.smithi143.stdout: "max": "", 2023-11-26T07:23:10.336 INFO:teuthology.orchestra.run.smithi143.stdout: "enum_allowed": [], 2023-11-26T07:23:10.336 INFO:teuthology.orchestra.run.smithi143.stdout: "desc": "", 2023-11-26T07:23:10.336 INFO:teuthology.orchestra.run.smithi143.stdout: "long_desc": "", 2023-11-26T07:23:10.336 INFO:teuthology.orchestra.run.smithi143.stdout: "tags": [], 2023-11-26T07:23:10.336 INFO:teuthology.orchestra.run.smithi143.stdout: "see_also": [] 2023-11-26T07:23:10.336 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2023-11-26T07:23:10.336 INFO:teuthology.orchestra.run.smithi143.stdout: "log_level": { 2023-11-26T07:23:10.336 INFO:teuthology.orchestra.run.smithi143.stdout: "name": "log_level", 2023-11-26T07:23:10.336 INFO:teuthology.orchestra.run.smithi143.stdout: "type": "str", 2023-11-26T07:23:10.336 INFO:teuthology.orchestra.run.smithi143.stdout: "level": "advanced", 2023-11-26T07:23:10.337 INFO:teuthology.orchestra.run.smithi143.stdout: "flags": 1, 2023-11-26T07:23:10.337 INFO:teuthology.orchestra.run.smithi143.stdout: "default_value": "", 2023-11-26T07:23:10.337 INFO:teuthology.orchestra.run.smithi143.stdout: "min": "", 2023-11-26T07:23:10.337 INFO:teuthology.orchestra.run.smithi143.stdout: "max": "", 2023-11-26T07:23:10.337 INFO:teuthology.orchestra.run.smithi143.stdout: "enum_allowed": [ 2023-11-26T07:23:10.337 INFO:teuthology.orchestra.run.smithi143.stdout: "", 2023-11-26T07:23:10.337 INFO:teuthology.orchestra.run.smithi143.stdout: "critical", 2023-11-26T07:23:10.337 INFO:teuthology.orchestra.run.smithi143.stdout: "debug", 2023-11-26T07:23:10.337 INFO:teuthology.orchestra.run.smithi143.stdout: "error", 2023-11-26T07:23:10.337 INFO:teuthology.orchestra.run.smithi143.stdout: "info", 2023-11-26T07:23:10.337 INFO:teuthology.orchestra.run.smithi143.stdout: "warning" 2023-11-26T07:23:10.337 INFO:teuthology.orchestra.run.smithi143.stdout: ], 2023-11-26T07:23:10.338 INFO:teuthology.orchestra.run.smithi143.stdout: "desc": "", 2023-11-26T07:23:10.338 INFO:teuthology.orchestra.run.smithi143.stdout: "long_desc": "", 2023-11-26T07:23:10.338 INFO:teuthology.orchestra.run.smithi143.stdout: "tags": [], 2023-11-26T07:23:10.338 INFO:teuthology.orchestra.run.smithi143.stdout: "see_also": [] 2023-11-26T07:23:10.338 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2023-11-26T07:23:10.338 INFO:teuthology.orchestra.run.smithi143.stdout: "log_to_cluster": { 2023-11-26T07:23:10.338 INFO:teuthology.orchestra.run.smithi143.stdout: "name": "log_to_cluster", 2023-11-26T07:23:10.338 INFO:teuthology.orchestra.run.smithi143.stdout: "type": "bool", 2023-11-26T07:23:10.338 INFO:teuthology.orchestra.run.smithi143.stdout: "level": "advanced", 2023-11-26T07:23:10.338 INFO:teuthology.orchestra.run.smithi143.stdout: "flags": 1, 2023-11-26T07:23:10.338 INFO:teuthology.orchestra.run.smithi143.stdout: "default_value": "False", 2023-11-26T07:23:10.339 INFO:teuthology.orchestra.run.smithi143.stdout: "min": "", 2023-11-26T07:23:10.339 INFO:teuthology.orchestra.run.smithi143.stdout: "max": "", 2023-11-26T07:23:10.339 INFO:teuthology.orchestra.run.smithi143.stdout: "enum_allowed": [], 2023-11-26T07:23:10.339 INFO:teuthology.orchestra.run.smithi143.stdout: "desc": "", 2023-11-26T07:23:10.339 INFO:teuthology.orchestra.run.smithi143.stdout: "long_desc": "", 2023-11-26T07:23:10.339 INFO:teuthology.orchestra.run.smithi143.stdout: "tags": [], 2023-11-26T07:23:10.339 INFO:teuthology.orchestra.run.smithi143.stdout: "see_also": [] 2023-11-26T07:23:10.339 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2023-11-26T07:23:10.339 INFO:teuthology.orchestra.run.smithi143.stdout: "log_to_cluster_level": { 2023-11-26T07:23:10.339 INFO:teuthology.orchestra.run.smithi143.stdout: "name": "log_to_cluster_level", 2023-11-26T07:23:10.339 INFO:teuthology.orchestra.run.smithi143.stdout: "type": "str", 2023-11-26T07:23:10.339 INFO:teuthology.orchestra.run.smithi143.stdout: "level": "advanced", 2023-11-26T07:23:10.340 INFO:teuthology.orchestra.run.smithi143.stdout: "flags": 1, 2023-11-26T07:23:10.340 INFO:teuthology.orchestra.run.smithi143.stdout: "default_value": "info", 2023-11-26T07:23:10.340 INFO:teuthology.orchestra.run.smithi143.stdout: "min": "", 2023-11-26T07:23:10.340 INFO:teuthology.orchestra.run.smithi143.stdout: "max": "", 2023-11-26T07:23:10.340 INFO:teuthology.orchestra.run.smithi143.stdout: "enum_allowed": [ 2023-11-26T07:23:10.340 INFO:teuthology.orchestra.run.smithi143.stdout: "", 2023-11-26T07:23:10.340 INFO:teuthology.orchestra.run.smithi143.stdout: "critical", 2023-11-26T07:23:10.340 INFO:teuthology.orchestra.run.smithi143.stdout: "debug", 2023-11-26T07:23:10.340 INFO:teuthology.orchestra.run.smithi143.stdout: "error", 2023-11-26T07:23:10.340 INFO:teuthology.orchestra.run.smithi143.stdout: "info", 2023-11-26T07:23:10.340 INFO:teuthology.orchestra.run.smithi143.stdout: "warning" 2023-11-26T07:23:10.340 INFO:teuthology.orchestra.run.smithi143.stdout: ], 2023-11-26T07:23:10.340 INFO:teuthology.orchestra.run.smithi143.stdout: "desc": "", 2023-11-26T07:23:10.341 INFO:teuthology.orchestra.run.smithi143.stdout: "long_desc": "", 2023-11-26T07:23:10.341 INFO:teuthology.orchestra.run.smithi143.stdout: "tags": [], 2023-11-26T07:23:10.341 INFO:teuthology.orchestra.run.smithi143.stdout: "see_also": [] 2023-11-26T07:23:10.341 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2023-11-26T07:23:10.341 INFO:teuthology.orchestra.run.smithi143.stdout: "log_to_file": { 2023-11-26T07:23:10.341 INFO:teuthology.orchestra.run.smithi143.stdout: "name": "log_to_file", 2023-11-26T07:23:10.341 INFO:teuthology.orchestra.run.smithi143.stdout: "type": "bool", 2023-11-26T07:23:10.341 INFO:teuthology.orchestra.run.smithi143.stdout: "level": "advanced", 2023-11-26T07:23:10.341 INFO:teuthology.orchestra.run.smithi143.stdout: "flags": 1, 2023-11-26T07:23:10.341 INFO:teuthology.orchestra.run.smithi143.stdout: "default_value": "False", 2023-11-26T07:23:10.341 INFO:teuthology.orchestra.run.smithi143.stdout: "min": "", 2023-11-26T07:23:10.341 INFO:teuthology.orchestra.run.smithi143.stdout: "max": "", 2023-11-26T07:23:10.342 INFO:teuthology.orchestra.run.smithi143.stdout: "enum_allowed": [], 2023-11-26T07:23:10.342 INFO:teuthology.orchestra.run.smithi143.stdout: "desc": "", 2023-11-26T07:23:10.342 INFO:teuthology.orchestra.run.smithi143.stdout: "long_desc": "", 2023-11-26T07:23:10.342 INFO:teuthology.orchestra.run.smithi143.stdout: "tags": [], 2023-11-26T07:23:10.342 INFO:teuthology.orchestra.run.smithi143.stdout: "see_also": [] 2023-11-26T07:23:10.342 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2023-11-26T07:23:10.342 INFO:teuthology.orchestra.run.smithi143.stdout: "server_addr": { 2023-11-26T07:23:10.342 INFO:teuthology.orchestra.run.smithi143.stdout: "name": "server_addr", 2023-11-26T07:23:10.342 INFO:teuthology.orchestra.run.smithi143.stdout: "type": "str", 2023-11-26T07:23:10.342 INFO:teuthology.orchestra.run.smithi143.stdout: "level": "advanced", 2023-11-26T07:23:10.342 INFO:teuthology.orchestra.run.smithi143.stdout: "flags": 0, 2023-11-26T07:23:10.342 INFO:teuthology.orchestra.run.smithi143.stdout: "default_value": "", 2023-11-26T07:23:10.343 INFO:teuthology.orchestra.run.smithi143.stdout: "min": "", 2023-11-26T07:23:10.343 INFO:teuthology.orchestra.run.smithi143.stdout: "max": "", 2023-11-26T07:23:10.343 INFO:teuthology.orchestra.run.smithi143.stdout: "enum_allowed": [], 2023-11-26T07:23:10.343 INFO:teuthology.orchestra.run.smithi143.stdout: "desc": "", 2023-11-26T07:23:10.343 INFO:teuthology.orchestra.run.smithi143.stdout: "long_desc": "", 2023-11-26T07:23:10.343 INFO:teuthology.orchestra.run.smithi143.stdout: "tags": [], 2023-11-26T07:23:10.343 INFO:teuthology.orchestra.run.smithi143.stdout: "see_also": [] 2023-11-26T07:23:10.343 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2023-11-26T07:23:10.343 INFO:teuthology.orchestra.run.smithi143.stdout: "server_port": { 2023-11-26T07:23:10.343 INFO:teuthology.orchestra.run.smithi143.stdout: "name": "server_port", 2023-11-26T07:23:10.343 INFO:teuthology.orchestra.run.smithi143.stdout: "type": "str", 2023-11-26T07:23:10.343 INFO:teuthology.orchestra.run.smithi143.stdout: "level": "advanced", 2023-11-26T07:23:10.343 INFO:teuthology.orchestra.run.smithi143.stdout: "flags": 0, 2023-11-26T07:23:10.343 INFO:teuthology.orchestra.run.smithi143.stdout: "default_value": "", 2023-11-26T07:23:10.344 INFO:teuthology.orchestra.run.smithi143.stdout: "min": "", 2023-11-26T07:23:10.344 INFO:teuthology.orchestra.run.smithi143.stdout: "max": "", 2023-11-26T07:23:10.344 INFO:teuthology.orchestra.run.smithi143.stdout: "enum_allowed": [], 2023-11-26T07:23:10.344 INFO:teuthology.orchestra.run.smithi143.stdout: "desc": "", 2023-11-26T07:23:10.344 INFO:teuthology.orchestra.run.smithi143.stdout: "long_desc": "", 2023-11-26T07:23:10.344 INFO:teuthology.orchestra.run.smithi143.stdout: "tags": [], 2023-11-26T07:23:10.344 INFO:teuthology.orchestra.run.smithi143.stdout: "see_also": [] 2023-11-26T07:23:10.344 INFO:teuthology.orchestra.run.smithi143.stdout: } 2023-11-26T07:23:10.344 INFO:teuthology.orchestra.run.smithi143.stdout: } 2023-11-26T07:23:10.344 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2023-11-26T07:23:10.344 INFO:teuthology.orchestra.run.smithi143.stdout: { 2023-11-26T07:23:10.344 INFO:teuthology.orchestra.run.smithi143.stdout: "name": "selftest", 2023-11-26T07:23:10.345 INFO:teuthology.orchestra.run.smithi143.stdout: "can_run": true, 2023-11-26T07:23:10.345 INFO:teuthology.orchestra.run.smithi143.stdout: "error_string": "", 2023-11-26T07:23:10.345 INFO:teuthology.orchestra.run.smithi143.stdout: "module_options": { 2023-11-26T07:23:10.345 INFO:teuthology.orchestra.run.smithi143.stdout: "log_level": { 2023-11-26T07:23:10.345 INFO:teuthology.orchestra.run.smithi143.stdout: "name": "log_level", 2023-11-26T07:23:10.345 INFO:teuthology.orchestra.run.smithi143.stdout: "type": "str", 2023-11-26T07:23:10.345 INFO:teuthology.orchestra.run.smithi143.stdout: "level": "advanced", 2023-11-26T07:23:10.345 INFO:teuthology.orchestra.run.smithi143.stdout: "flags": 1, 2023-11-26T07:23:10.345 INFO:teuthology.orchestra.run.smithi143.stdout: "default_value": "", 2023-11-26T07:23:10.345 INFO:teuthology.orchestra.run.smithi143.stdout: "min": "", 2023-11-26T07:23:10.345 INFO:teuthology.orchestra.run.smithi143.stdout: "max": "", 2023-11-26T07:23:10.345 INFO:teuthology.orchestra.run.smithi143.stdout: "enum_allowed": [ 2023-11-26T07:23:10.346 INFO:teuthology.orchestra.run.smithi143.stdout: "", 2023-11-26T07:23:10.346 INFO:teuthology.orchestra.run.smithi143.stdout: "critical", 2023-11-26T07:23:10.346 INFO:teuthology.orchestra.run.smithi143.stdout: "debug", 2023-11-26T07:23:10.346 INFO:teuthology.orchestra.run.smithi143.stdout: "error", 2023-11-26T07:23:10.346 INFO:teuthology.orchestra.run.smithi143.stdout: "info", 2023-11-26T07:23:10.346 INFO:teuthology.orchestra.run.smithi143.stdout: "warning" 2023-11-26T07:23:10.346 INFO:teuthology.orchestra.run.smithi143.stdout: ], 2023-11-26T07:23:10.346 INFO:teuthology.orchestra.run.smithi143.stdout: "desc": "", 2023-11-26T07:23:10.346 INFO:teuthology.orchestra.run.smithi143.stdout: "long_desc": "", 2023-11-26T07:23:10.346 INFO:teuthology.orchestra.run.smithi143.stdout: "tags": [], 2023-11-26T07:23:10.347 INFO:teuthology.orchestra.run.smithi143.stdout: "see_also": [] 2023-11-26T07:23:10.347 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2023-11-26T07:23:10.347 INFO:teuthology.orchestra.run.smithi143.stdout: "log_to_cluster": { 2023-11-26T07:23:10.347 INFO:teuthology.orchestra.run.smithi143.stdout: "name": "log_to_cluster", 2023-11-26T07:23:10.347 INFO:teuthology.orchestra.run.smithi143.stdout: "type": "bool", 2023-11-26T07:23:10.347 INFO:teuthology.orchestra.run.smithi143.stdout: "level": "advanced", 2023-11-26T07:23:10.347 INFO:teuthology.orchestra.run.smithi143.stdout: "flags": 1, 2023-11-26T07:23:10.347 INFO:teuthology.orchestra.run.smithi143.stdout: "default_value": "False", 2023-11-26T07:23:10.347 INFO:teuthology.orchestra.run.smithi143.stdout: "min": "", 2023-11-26T07:23:10.347 INFO:teuthology.orchestra.run.smithi143.stdout: "max": "", 2023-11-26T07:23:10.347 INFO:teuthology.orchestra.run.smithi143.stdout: "enum_allowed": [], 2023-11-26T07:23:10.347 INFO:teuthology.orchestra.run.smithi143.stdout: "desc": "", 2023-11-26T07:23:10.347 INFO:teuthology.orchestra.run.smithi143.stdout: "long_desc": "", 2023-11-26T07:23:10.348 INFO:teuthology.orchestra.run.smithi143.stdout: "tags": [], 2023-11-26T07:23:10.348 INFO:teuthology.orchestra.run.smithi143.stdout: "see_also": [] 2023-11-26T07:23:10.348 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2023-11-26T07:23:10.348 INFO:teuthology.orchestra.run.smithi143.stdout: "log_to_cluster_level": { 2023-11-26T07:23:10.348 INFO:teuthology.orchestra.run.smithi143.stdout: "name": "log_to_cluster_level", 2023-11-26T07:23:10.348 INFO:teuthology.orchestra.run.smithi143.stdout: "type": "str", 2023-11-26T07:23:10.348 INFO:teuthology.orchestra.run.smithi143.stdout: "level": "advanced", 2023-11-26T07:23:10.348 INFO:teuthology.orchestra.run.smithi143.stdout: "flags": 1, 2023-11-26T07:23:10.348 INFO:teuthology.orchestra.run.smithi143.stdout: "default_value": "info", 2023-11-26T07:23:10.348 INFO:teuthology.orchestra.run.smithi143.stdout: "min": "", 2023-11-26T07:23:10.348 INFO:teuthology.orchestra.run.smithi143.stdout: "max": "", 2023-11-26T07:23:10.348 INFO:teuthology.orchestra.run.smithi143.stdout: "enum_allowed": [ 2023-11-26T07:23:10.348 INFO:teuthology.orchestra.run.smithi143.stdout: "", 2023-11-26T07:23:10.349 INFO:teuthology.orchestra.run.smithi143.stdout: "critical", 2023-11-26T07:23:10.349 INFO:teuthology.orchestra.run.smithi143.stdout: "debug", 2023-11-26T07:23:10.349 INFO:teuthology.orchestra.run.smithi143.stdout: "error", 2023-11-26T07:23:10.349 INFO:teuthology.orchestra.run.smithi143.stdout: "info", 2023-11-26T07:23:10.349 INFO:teuthology.orchestra.run.smithi143.stdout: "warning" 2023-11-26T07:23:10.349 INFO:teuthology.orchestra.run.smithi143.stdout: ], 2023-11-26T07:23:10.349 INFO:teuthology.orchestra.run.smithi143.stdout: "desc": "", 2023-11-26T07:23:10.349 INFO:teuthology.orchestra.run.smithi143.stdout: "long_desc": "", 2023-11-26T07:23:10.349 INFO:teuthology.orchestra.run.smithi143.stdout: "tags": [], 2023-11-26T07:23:10.349 INFO:teuthology.orchestra.run.smithi143.stdout: "see_also": [] 2023-11-26T07:23:10.349 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2023-11-26T07:23:10.349 INFO:teuthology.orchestra.run.smithi143.stdout: "log_to_file": { 2023-11-26T07:23:10.349 INFO:teuthology.orchestra.run.smithi143.stdout: "name": "log_to_file", 2023-11-26T07:23:10.350 INFO:teuthology.orchestra.run.smithi143.stdout: "type": "bool", 2023-11-26T07:23:10.350 INFO:teuthology.orchestra.run.smithi143.stdout: "level": "advanced", 2023-11-26T07:23:10.350 INFO:teuthology.orchestra.run.smithi143.stdout: "flags": 1, 2023-11-26T07:23:10.350 INFO:teuthology.orchestra.run.smithi143.stdout: "default_value": "False", 2023-11-26T07:23:10.350 INFO:teuthology.orchestra.run.smithi143.stdout: "min": "", 2023-11-26T07:23:10.350 INFO:teuthology.orchestra.run.smithi143.stdout: "max": "", 2023-11-26T07:23:10.350 INFO:teuthology.orchestra.run.smithi143.stdout: "enum_allowed": [], 2023-11-26T07:23:10.350 INFO:teuthology.orchestra.run.smithi143.stdout: "desc": "", 2023-11-26T07:23:10.350 INFO:teuthology.orchestra.run.smithi143.stdout: "long_desc": "", 2023-11-26T07:23:10.350 INFO:teuthology.orchestra.run.smithi143.stdout: "tags": [], 2023-11-26T07:23:10.350 INFO:teuthology.orchestra.run.smithi143.stdout: "see_also": [] 2023-11-26T07:23:10.350 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2023-11-26T07:23:10.351 INFO:teuthology.orchestra.run.smithi143.stdout: "roption1": { 2023-11-26T07:23:10.351 INFO:teuthology.orchestra.run.smithi143.stdout: "name": "roption1", 2023-11-26T07:23:10.351 INFO:teuthology.orchestra.run.smithi143.stdout: "type": "str", 2023-11-26T07:23:10.351 INFO:teuthology.orchestra.run.smithi143.stdout: "level": "advanced", 2023-11-26T07:23:10.351 INFO:teuthology.orchestra.run.smithi143.stdout: "flags": 0, 2023-11-26T07:23:10.351 INFO:teuthology.orchestra.run.smithi143.stdout: "default_value": "", 2023-11-26T07:23:10.351 INFO:teuthology.orchestra.run.smithi143.stdout: "min": "", 2023-11-26T07:23:10.351 INFO:teuthology.orchestra.run.smithi143.stdout: "max": "", 2023-11-26T07:23:10.351 INFO:teuthology.orchestra.run.smithi143.stdout: "enum_allowed": [], 2023-11-26T07:23:10.351 INFO:teuthology.orchestra.run.smithi143.stdout: "desc": "", 2023-11-26T07:23:10.351 INFO:teuthology.orchestra.run.smithi143.stdout: "long_desc": "", 2023-11-26T07:23:10.351 INFO:teuthology.orchestra.run.smithi143.stdout: "tags": [], 2023-11-26T07:23:10.351 INFO:teuthology.orchestra.run.smithi143.stdout: "see_also": [] 2023-11-26T07:23:10.352 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2023-11-26T07:23:10.352 INFO:teuthology.orchestra.run.smithi143.stdout: "roption2": { 2023-11-26T07:23:10.352 INFO:teuthology.orchestra.run.smithi143.stdout: "name": "roption2", 2023-11-26T07:23:10.352 INFO:teuthology.orchestra.run.smithi143.stdout: "type": "str", 2023-11-26T07:23:10.352 INFO:teuthology.orchestra.run.smithi143.stdout: "level": "advanced", 2023-11-26T07:23:10.352 INFO:teuthology.orchestra.run.smithi143.stdout: "flags": 0, 2023-11-26T07:23:10.352 INFO:teuthology.orchestra.run.smithi143.stdout: "default_value": "xyz", 2023-11-26T07:23:10.352 INFO:teuthology.orchestra.run.smithi143.stdout: "min": "", 2023-11-26T07:23:10.352 INFO:teuthology.orchestra.run.smithi143.stdout: "max": "", 2023-11-26T07:23:10.352 INFO:teuthology.orchestra.run.smithi143.stdout: "enum_allowed": [], 2023-11-26T07:23:10.352 INFO:teuthology.orchestra.run.smithi143.stdout: "desc": "", 2023-11-26T07:23:10.352 INFO:teuthology.orchestra.run.smithi143.stdout: "long_desc": "", 2023-11-26T07:23:10.352 INFO:teuthology.orchestra.run.smithi143.stdout: "tags": [], 2023-11-26T07:23:10.352 INFO:teuthology.orchestra.run.smithi143.stdout: "see_also": [] 2023-11-26T07:23:10.353 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2023-11-26T07:23:10.353 INFO:teuthology.orchestra.run.smithi143.stdout: "rwoption1": { 2023-11-26T07:23:10.353 INFO:teuthology.orchestra.run.smithi143.stdout: "name": "rwoption1", 2023-11-26T07:23:10.353 INFO:teuthology.orchestra.run.smithi143.stdout: "type": "str", 2023-11-26T07:23:10.353 INFO:teuthology.orchestra.run.smithi143.stdout: "level": "advanced", 2023-11-26T07:23:10.353 INFO:teuthology.orchestra.run.smithi143.stdout: "flags": 0, 2023-11-26T07:23:10.353 INFO:teuthology.orchestra.run.smithi143.stdout: "default_value": "", 2023-11-26T07:23:10.353 INFO:teuthology.orchestra.run.smithi143.stdout: "min": "", 2023-11-26T07:23:10.353 INFO:teuthology.orchestra.run.smithi143.stdout: "max": "", 2023-11-26T07:23:10.353 INFO:teuthology.orchestra.run.smithi143.stdout: "enum_allowed": [], 2023-11-26T07:23:10.353 INFO:teuthology.orchestra.run.smithi143.stdout: "desc": "", 2023-11-26T07:23:10.353 INFO:teuthology.orchestra.run.smithi143.stdout: "long_desc": "", 2023-11-26T07:23:10.354 INFO:teuthology.orchestra.run.smithi143.stdout: "tags": [], 2023-11-26T07:23:10.354 INFO:teuthology.orchestra.run.smithi143.stdout: "see_also": [] 2023-11-26T07:23:10.354 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2023-11-26T07:23:10.354 INFO:teuthology.orchestra.run.smithi143.stdout: "rwoption2": { 2023-11-26T07:23:10.354 INFO:teuthology.orchestra.run.smithi143.stdout: "name": "rwoption2", 2023-11-26T07:23:10.354 INFO:teuthology.orchestra.run.smithi143.stdout: "type": "int", 2023-11-26T07:23:10.354 INFO:teuthology.orchestra.run.smithi143.stdout: "level": "advanced", 2023-11-26T07:23:10.354 INFO:teuthology.orchestra.run.smithi143.stdout: "flags": 0, 2023-11-26T07:23:10.354 INFO:teuthology.orchestra.run.smithi143.stdout: "default_value": "", 2023-11-26T07:23:10.354 INFO:teuthology.orchestra.run.smithi143.stdout: "min": "", 2023-11-26T07:23:10.354 INFO:teuthology.orchestra.run.smithi143.stdout: "max": "", 2023-11-26T07:23:10.354 INFO:teuthology.orchestra.run.smithi143.stdout: "enum_allowed": [], 2023-11-26T07:23:10.354 INFO:teuthology.orchestra.run.smithi143.stdout: "desc": "", 2023-11-26T07:23:10.355 INFO:teuthology.orchestra.run.smithi143.stdout: "long_desc": "", 2023-11-26T07:23:10.355 INFO:teuthology.orchestra.run.smithi143.stdout: "tags": [], 2023-11-26T07:23:10.355 INFO:teuthology.orchestra.run.smithi143.stdout: "see_also": [] 2023-11-26T07:23:10.355 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2023-11-26T07:23:10.355 INFO:teuthology.orchestra.run.smithi143.stdout: "rwoption3": { 2023-11-26T07:23:10.355 INFO:teuthology.orchestra.run.smithi143.stdout: "name": "rwoption3", 2023-11-26T07:23:10.355 INFO:teuthology.orchestra.run.smithi143.stdout: "type": "float", 2023-11-26T07:23:10.355 INFO:teuthology.orchestra.run.smithi143.stdout: "level": "advanced", 2023-11-26T07:23:10.355 INFO:teuthology.orchestra.run.smithi143.stdout: "flags": 0, 2023-11-26T07:23:10.356 INFO:teuthology.orchestra.run.smithi143.stdout: "default_value": "", 2023-11-26T07:23:10.356 INFO:teuthology.orchestra.run.smithi143.stdout: "min": "", 2023-11-26T07:23:10.356 INFO:teuthology.orchestra.run.smithi143.stdout: "max": "", 2023-11-26T07:23:10.356 INFO:teuthology.orchestra.run.smithi143.stdout: "enum_allowed": [], 2023-11-26T07:23:10.356 INFO:teuthology.orchestra.run.smithi143.stdout: "desc": "", 2023-11-26T07:23:10.356 INFO:teuthology.orchestra.run.smithi143.stdout: "long_desc": "", 2023-11-26T07:23:10.356 INFO:teuthology.orchestra.run.smithi143.stdout: "tags": [], 2023-11-26T07:23:10.356 INFO:teuthology.orchestra.run.smithi143.stdout: "see_also": [] 2023-11-26T07:23:10.356 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2023-11-26T07:23:10.356 INFO:teuthology.orchestra.run.smithi143.stdout: "rwoption4": { 2023-11-26T07:23:10.356 INFO:teuthology.orchestra.run.smithi143.stdout: "name": "rwoption4", 2023-11-26T07:23:10.356 INFO:teuthology.orchestra.run.smithi143.stdout: "type": "str", 2023-11-26T07:23:10.356 INFO:teuthology.orchestra.run.smithi143.stdout: "level": "advanced", 2023-11-26T07:23:10.357 INFO:teuthology.orchestra.run.smithi143.stdout: "flags": 0, 2023-11-26T07:23:10.357 INFO:teuthology.orchestra.run.smithi143.stdout: "default_value": "", 2023-11-26T07:23:10.357 INFO:teuthology.orchestra.run.smithi143.stdout: "min": "", 2023-11-26T07:23:10.357 INFO:teuthology.orchestra.run.smithi143.stdout: "max": "", 2023-11-26T07:23:10.357 INFO:teuthology.orchestra.run.smithi143.stdout: "enum_allowed": [], 2023-11-26T07:23:10.357 INFO:teuthology.orchestra.run.smithi143.stdout: "desc": "", 2023-11-26T07:23:10.357 INFO:teuthology.orchestra.run.smithi143.stdout: "long_desc": "", 2023-11-26T07:23:10.357 INFO:teuthology.orchestra.run.smithi143.stdout: "tags": [], 2023-11-26T07:23:10.357 INFO:teuthology.orchestra.run.smithi143.stdout: "see_also": [] 2023-11-26T07:23:10.357 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2023-11-26T07:23:10.357 INFO:teuthology.orchestra.run.smithi143.stdout: "rwoption5": { 2023-11-26T07:23:10.357 INFO:teuthology.orchestra.run.smithi143.stdout: "name": "rwoption5", 2023-11-26T07:23:10.357 INFO:teuthology.orchestra.run.smithi143.stdout: "type": "bool", 2023-11-26T07:23:10.358 INFO:teuthology.orchestra.run.smithi143.stdout: "level": "advanced", 2023-11-26T07:23:10.358 INFO:teuthology.orchestra.run.smithi143.stdout: "flags": 0, 2023-11-26T07:23:10.358 INFO:teuthology.orchestra.run.smithi143.stdout: "default_value": "", 2023-11-26T07:23:10.358 INFO:teuthology.orchestra.run.smithi143.stdout: "min": "", 2023-11-26T07:23:10.358 INFO:teuthology.orchestra.run.smithi143.stdout: "max": "", 2023-11-26T07:23:10.358 INFO:teuthology.orchestra.run.smithi143.stdout: "enum_allowed": [], 2023-11-26T07:23:10.358 INFO:teuthology.orchestra.run.smithi143.stdout: "desc": "", 2023-11-26T07:23:10.358 INFO:teuthology.orchestra.run.smithi143.stdout: "long_desc": "", 2023-11-26T07:23:10.358 INFO:teuthology.orchestra.run.smithi143.stdout: "tags": [], 2023-11-26T07:23:10.358 INFO:teuthology.orchestra.run.smithi143.stdout: "see_also": [] 2023-11-26T07:23:10.358 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2023-11-26T07:23:10.358 INFO:teuthology.orchestra.run.smithi143.stdout: "rwoption6": { 2023-11-26T07:23:10.358 INFO:teuthology.orchestra.run.smithi143.stdout: "name": "rwoption6", 2023-11-26T07:23:10.359 INFO:teuthology.orchestra.run.smithi143.stdout: "type": "bool", 2023-11-26T07:23:10.359 INFO:teuthology.orchestra.run.smithi143.stdout: "level": "advanced", 2023-11-26T07:23:10.359 INFO:teuthology.orchestra.run.smithi143.stdout: "flags": 0, 2023-11-26T07:23:10.359 INFO:teuthology.orchestra.run.smithi143.stdout: "default_value": "True", 2023-11-26T07:23:10.359 INFO:teuthology.orchestra.run.smithi143.stdout: "min": "", 2023-11-26T07:23:10.359 INFO:teuthology.orchestra.run.smithi143.stdout: "max": "", 2023-11-26T07:23:10.359 INFO:teuthology.orchestra.run.smithi143.stdout: "enum_allowed": [], 2023-11-26T07:23:10.359 INFO:teuthology.orchestra.run.smithi143.stdout: "desc": "", 2023-11-26T07:23:10.359 INFO:teuthology.orchestra.run.smithi143.stdout: "long_desc": "", 2023-11-26T07:23:10.359 INFO:teuthology.orchestra.run.smithi143.stdout: "tags": [], 2023-11-26T07:23:10.359 INFO:teuthology.orchestra.run.smithi143.stdout: "see_also": [] 2023-11-26T07:23:10.359 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2023-11-26T07:23:10.359 INFO:teuthology.orchestra.run.smithi143.stdout: "rwoption7": { 2023-11-26T07:23:10.360 INFO:teuthology.orchestra.run.smithi143.stdout: "name": "rwoption7", 2023-11-26T07:23:10.360 INFO:teuthology.orchestra.run.smithi143.stdout: "type": "int", 2023-11-26T07:23:10.360 INFO:teuthology.orchestra.run.smithi143.stdout: "level": "advanced", 2023-11-26T07:23:10.360 INFO:teuthology.orchestra.run.smithi143.stdout: "flags": 0, 2023-11-26T07:23:10.360 INFO:teuthology.orchestra.run.smithi143.stdout: "default_value": "", 2023-11-26T07:23:10.360 INFO:teuthology.orchestra.run.smithi143.stdout: "min": "1", 2023-11-26T07:23:10.360 INFO:teuthology.orchestra.run.smithi143.stdout: "max": "42", 2023-11-26T07:23:10.360 INFO:teuthology.orchestra.run.smithi143.stdout: "enum_allowed": [], 2023-11-26T07:23:10.360 INFO:teuthology.orchestra.run.smithi143.stdout: "desc": "", 2023-11-26T07:23:10.360 INFO:teuthology.orchestra.run.smithi143.stdout: "long_desc": "", 2023-11-26T07:23:10.360 INFO:teuthology.orchestra.run.smithi143.stdout: "tags": [], 2023-11-26T07:23:10.360 INFO:teuthology.orchestra.run.smithi143.stdout: "see_also": [] 2023-11-26T07:23:10.360 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2023-11-26T07:23:10.361 INFO:teuthology.orchestra.run.smithi143.stdout: "testkey": { 2023-11-26T07:23:10.361 INFO:teuthology.orchestra.run.smithi143.stdout: "name": "testkey", 2023-11-26T07:23:10.361 INFO:teuthology.orchestra.run.smithi143.stdout: "type": "str", 2023-11-26T07:23:10.361 INFO:teuthology.orchestra.run.smithi143.stdout: "level": "advanced", 2023-11-26T07:23:10.361 INFO:teuthology.orchestra.run.smithi143.stdout: "flags": 0, 2023-11-26T07:23:10.361 INFO:teuthology.orchestra.run.smithi143.stdout: "default_value": "", 2023-11-26T07:23:10.361 INFO:teuthology.orchestra.run.smithi143.stdout: "min": "", 2023-11-26T07:23:10.361 INFO:teuthology.orchestra.run.smithi143.stdout: "max": "", 2023-11-26T07:23:10.361 INFO:teuthology.orchestra.run.smithi143.stdout: "enum_allowed": [], 2023-11-26T07:23:10.361 INFO:teuthology.orchestra.run.smithi143.stdout: "desc": "", 2023-11-26T07:23:10.361 INFO:teuthology.orchestra.run.smithi143.stdout: "long_desc": "", 2023-11-26T07:23:10.361 INFO:teuthology.orchestra.run.smithi143.stdout: "tags": [], 2023-11-26T07:23:10.362 INFO:teuthology.orchestra.run.smithi143.stdout: "see_also": [] 2023-11-26T07:23:10.362 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2023-11-26T07:23:10.362 INFO:teuthology.orchestra.run.smithi143.stdout: "testlkey": { 2023-11-26T07:23:10.362 INFO:teuthology.orchestra.run.smithi143.stdout: "name": "testlkey", 2023-11-26T07:23:10.362 INFO:teuthology.orchestra.run.smithi143.stdout: "type": "str", 2023-11-26T07:23:10.362 INFO:teuthology.orchestra.run.smithi143.stdout: "level": "advanced", 2023-11-26T07:23:10.362 INFO:teuthology.orchestra.run.smithi143.stdout: "flags": 0, 2023-11-26T07:23:10.362 INFO:teuthology.orchestra.run.smithi143.stdout: "default_value": "", 2023-11-26T07:23:10.362 INFO:teuthology.orchestra.run.smithi143.stdout: "min": "", 2023-11-26T07:23:10.362 INFO:teuthology.orchestra.run.smithi143.stdout: "max": "", 2023-11-26T07:23:10.362 INFO:teuthology.orchestra.run.smithi143.stdout: "enum_allowed": [], 2023-11-26T07:23:10.362 INFO:teuthology.orchestra.run.smithi143.stdout: "desc": "", 2023-11-26T07:23:10.362 INFO:teuthology.orchestra.run.smithi143.stdout: "long_desc": "", 2023-11-26T07:23:10.363 INFO:teuthology.orchestra.run.smithi143.stdout: "tags": [], 2023-11-26T07:23:10.363 INFO:teuthology.orchestra.run.smithi143.stdout: "see_also": [] 2023-11-26T07:23:10.363 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2023-11-26T07:23:10.363 INFO:teuthology.orchestra.run.smithi143.stdout: "testnewline": { 2023-11-26T07:23:10.363 INFO:teuthology.orchestra.run.smithi143.stdout: "name": "testnewline", 2023-11-26T07:23:10.363 INFO:teuthology.orchestra.run.smithi143.stdout: "type": "str", 2023-11-26T07:23:10.363 INFO:teuthology.orchestra.run.smithi143.stdout: "level": "advanced", 2023-11-26T07:23:10.363 INFO:teuthology.orchestra.run.smithi143.stdout: "flags": 0, 2023-11-26T07:23:10.363 INFO:teuthology.orchestra.run.smithi143.stdout: "default_value": "", 2023-11-26T07:23:10.363 INFO:teuthology.orchestra.run.smithi143.stdout: "min": "", 2023-11-26T07:23:10.363 INFO:teuthology.orchestra.run.smithi143.stdout: "max": "", 2023-11-26T07:23:10.363 INFO:teuthology.orchestra.run.smithi143.stdout: "enum_allowed": [], 2023-11-26T07:23:10.364 INFO:teuthology.orchestra.run.smithi143.stdout: "desc": "", 2023-11-26T07:23:10.364 INFO:teuthology.orchestra.run.smithi143.stdout: "long_desc": "", 2023-11-26T07:23:10.364 INFO:teuthology.orchestra.run.smithi143.stdout: "tags": [], 2023-11-26T07:23:10.364 INFO:teuthology.orchestra.run.smithi143.stdout: "see_also": [] 2023-11-26T07:23:10.364 INFO:teuthology.orchestra.run.smithi143.stdout: } 2023-11-26T07:23:10.364 INFO:teuthology.orchestra.run.smithi143.stdout: } 2023-11-26T07:23:10.364 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2023-11-26T07:23:10.364 INFO:teuthology.orchestra.run.smithi143.stdout: { 2023-11-26T07:23:10.364 INFO:teuthology.orchestra.run.smithi143.stdout: "name": "snap_schedule", 2023-11-26T07:23:10.364 INFO:teuthology.orchestra.run.smithi143.stdout: "can_run": true, 2023-11-26T07:23:10.364 INFO:teuthology.orchestra.run.smithi143.stdout: "error_string": "", 2023-11-26T07:23:10.364 INFO:teuthology.orchestra.run.smithi143.stdout: "module_options": { 2023-11-26T07:23:10.364 INFO:teuthology.orchestra.run.smithi143.stdout: "allow_m_granularity": { 2023-11-26T07:23:10.365 INFO:teuthology.orchestra.run.smithi143.stdout: "name": "allow_m_granularity", 2023-11-26T07:23:10.365 INFO:teuthology.orchestra.run.smithi143.stdout: "type": "bool", 2023-11-26T07:23:10.365 INFO:teuthology.orchestra.run.smithi143.stdout: "level": "advanced", 2023-11-26T07:23:10.365 INFO:teuthology.orchestra.run.smithi143.stdout: "flags": 1, 2023-11-26T07:23:10.365 INFO:teuthology.orchestra.run.smithi143.stdout: "default_value": "False", 2023-11-26T07:23:10.365 INFO:teuthology.orchestra.run.smithi143.stdout: "min": "", 2023-11-26T07:23:10.365 INFO:teuthology.orchestra.run.smithi143.stdout: "max": "", 2023-11-26T07:23:10.365 INFO:teuthology.orchestra.run.smithi143.stdout: "enum_allowed": [], 2023-11-26T07:23:10.365 INFO:teuthology.orchestra.run.smithi143.stdout: "desc": "allow minute scheduled snapshots", 2023-11-26T07:23:10.365 INFO:teuthology.orchestra.run.smithi143.stdout: "long_desc": "", 2023-11-26T07:23:10.365 INFO:teuthology.orchestra.run.smithi143.stdout: "tags": [], 2023-11-26T07:23:10.365 INFO:teuthology.orchestra.run.smithi143.stdout: "see_also": [] 2023-11-26T07:23:10.365 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2023-11-26T07:23:10.365 INFO:teuthology.orchestra.run.smithi143.stdout: "dump_on_update": { 2023-11-26T07:23:10.366 INFO:teuthology.orchestra.run.smithi143.stdout: "name": "dump_on_update", 2023-11-26T07:23:10.366 INFO:teuthology.orchestra.run.smithi143.stdout: "type": "bool", 2023-11-26T07:23:10.366 INFO:teuthology.orchestra.run.smithi143.stdout: "level": "advanced", 2023-11-26T07:23:10.366 INFO:teuthology.orchestra.run.smithi143.stdout: "flags": 1, 2023-11-26T07:23:10.366 INFO:teuthology.orchestra.run.smithi143.stdout: "default_value": "False", 2023-11-26T07:23:10.366 INFO:teuthology.orchestra.run.smithi143.stdout: "min": "", 2023-11-26T07:23:10.366 INFO:teuthology.orchestra.run.smithi143.stdout: "max": "", 2023-11-26T07:23:10.366 INFO:teuthology.orchestra.run.smithi143.stdout: "enum_allowed": [], 2023-11-26T07:23:10.366 INFO:teuthology.orchestra.run.smithi143.stdout: "desc": "dump database to debug log on update", 2023-11-26T07:23:10.366 INFO:teuthology.orchestra.run.smithi143.stdout: "long_desc": "", 2023-11-26T07:23:10.366 INFO:teuthology.orchestra.run.smithi143.stdout: "tags": [], 2023-11-26T07:23:10.366 INFO:teuthology.orchestra.run.smithi143.stdout: "see_also": [] 2023-11-26T07:23:10.366 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2023-11-26T07:23:10.367 INFO:teuthology.orchestra.run.smithi143.stdout: "log_level": { 2023-11-26T07:23:10.367 INFO:teuthology.orchestra.run.smithi143.stdout: "name": "log_level", 2023-11-26T07:23:10.367 INFO:teuthology.orchestra.run.smithi143.stdout: "type": "str", 2023-11-26T07:23:10.367 INFO:teuthology.orchestra.run.smithi143.stdout: "level": "advanced", 2023-11-26T07:23:10.367 INFO:teuthology.orchestra.run.smithi143.stdout: "flags": 1, 2023-11-26T07:23:10.367 INFO:teuthology.orchestra.run.smithi143.stdout: "default_value": "", 2023-11-26T07:23:10.367 INFO:teuthology.orchestra.run.smithi143.stdout: "min": "", 2023-11-26T07:23:10.367 INFO:teuthology.orchestra.run.smithi143.stdout: "max": "", 2023-11-26T07:23:10.367 INFO:teuthology.orchestra.run.smithi143.stdout: "enum_allowed": [ 2023-11-26T07:23:10.367 INFO:teuthology.orchestra.run.smithi143.stdout: "", 2023-11-26T07:23:10.367 INFO:teuthology.orchestra.run.smithi143.stdout: "critical", 2023-11-26T07:23:10.367 INFO:teuthology.orchestra.run.smithi143.stdout: "debug", 2023-11-26T07:23:10.367 INFO:teuthology.orchestra.run.smithi143.stdout: "error", 2023-11-26T07:23:10.368 INFO:teuthology.orchestra.run.smithi143.stdout: "info", 2023-11-26T07:23:10.368 INFO:teuthology.orchestra.run.smithi143.stdout: "warning" 2023-11-26T07:23:10.368 INFO:teuthology.orchestra.run.smithi143.stdout: ], 2023-11-26T07:23:10.368 INFO:teuthology.orchestra.run.smithi143.stdout: "desc": "", 2023-11-26T07:23:10.368 INFO:teuthology.orchestra.run.smithi143.stdout: "long_desc": "", 2023-11-26T07:23:10.368 INFO:teuthology.orchestra.run.smithi143.stdout: "tags": [], 2023-11-26T07:23:10.368 INFO:teuthology.orchestra.run.smithi143.stdout: "see_also": [] 2023-11-26T07:23:10.368 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2023-11-26T07:23:10.368 INFO:teuthology.orchestra.run.smithi143.stdout: "log_to_cluster": { 2023-11-26T07:23:10.368 INFO:teuthology.orchestra.run.smithi143.stdout: "name": "log_to_cluster", 2023-11-26T07:23:10.368 INFO:teuthology.orchestra.run.smithi143.stdout: "type": "bool", 2023-11-26T07:23:10.368 INFO:teuthology.orchestra.run.smithi143.stdout: "level": "advanced", 2023-11-26T07:23:10.368 INFO:teuthology.orchestra.run.smithi143.stdout: "flags": 1, 2023-11-26T07:23:10.369 INFO:teuthology.orchestra.run.smithi143.stdout: "default_value": "False", 2023-11-26T07:23:10.369 INFO:teuthology.orchestra.run.smithi143.stdout: "min": "", 2023-11-26T07:23:10.369 INFO:teuthology.orchestra.run.smithi143.stdout: "max": "", 2023-11-26T07:23:10.369 INFO:teuthology.orchestra.run.smithi143.stdout: "enum_allowed": [], 2023-11-26T07:23:10.369 INFO:teuthology.orchestra.run.smithi143.stdout: "desc": "", 2023-11-26T07:23:10.369 INFO:teuthology.orchestra.run.smithi143.stdout: "long_desc": "", 2023-11-26T07:23:10.369 INFO:teuthology.orchestra.run.smithi143.stdout: "tags": [], 2023-11-26T07:23:10.369 INFO:teuthology.orchestra.run.smithi143.stdout: "see_also": [] 2023-11-26T07:23:10.369 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2023-11-26T07:23:10.369 INFO:teuthology.orchestra.run.smithi143.stdout: "log_to_cluster_level": { 2023-11-26T07:23:10.369 INFO:teuthology.orchestra.run.smithi143.stdout: "name": "log_to_cluster_level", 2023-11-26T07:23:10.369 INFO:teuthology.orchestra.run.smithi143.stdout: "type": "str", 2023-11-26T07:23:10.370 INFO:teuthology.orchestra.run.smithi143.stdout: "level": "advanced", 2023-11-26T07:23:10.370 INFO:teuthology.orchestra.run.smithi143.stdout: "flags": 1, 2023-11-26T07:23:10.370 INFO:teuthology.orchestra.run.smithi143.stdout: "default_value": "info", 2023-11-26T07:23:10.370 INFO:teuthology.orchestra.run.smithi143.stdout: "min": "", 2023-11-26T07:23:10.370 INFO:teuthology.orchestra.run.smithi143.stdout: "max": "", 2023-11-26T07:23:10.370 INFO:teuthology.orchestra.run.smithi143.stdout: "enum_allowed": [ 2023-11-26T07:23:10.370 INFO:teuthology.orchestra.run.smithi143.stdout: "", 2023-11-26T07:23:10.370 INFO:teuthology.orchestra.run.smithi143.stdout: "critical", 2023-11-26T07:23:10.370 INFO:teuthology.orchestra.run.smithi143.stdout: "debug", 2023-11-26T07:23:10.370 INFO:teuthology.orchestra.run.smithi143.stdout: "error", 2023-11-26T07:23:10.370 INFO:teuthology.orchestra.run.smithi143.stdout: "info", 2023-11-26T07:23:10.370 INFO:teuthology.orchestra.run.smithi143.stdout: "warning" 2023-11-26T07:23:10.371 INFO:teuthology.orchestra.run.smithi143.stdout: ], 2023-11-26T07:23:10.371 INFO:teuthology.orchestra.run.smithi143.stdout: "desc": "", 2023-11-26T07:23:10.371 INFO:teuthology.orchestra.run.smithi143.stdout: "long_desc": "", 2023-11-26T07:23:10.371 INFO:teuthology.orchestra.run.smithi143.stdout: "tags": [], 2023-11-26T07:23:10.371 INFO:teuthology.orchestra.run.smithi143.stdout: "see_also": [] 2023-11-26T07:23:10.371 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2023-11-26T07:23:10.371 INFO:teuthology.orchestra.run.smithi143.stdout: "log_to_file": { 2023-11-26T07:23:10.371 INFO:teuthology.orchestra.run.smithi143.stdout: "name": "log_to_file", 2023-11-26T07:23:10.371 INFO:teuthology.orchestra.run.smithi143.stdout: "type": "bool", 2023-11-26T07:23:10.371 INFO:teuthology.orchestra.run.smithi143.stdout: "level": "advanced", 2023-11-26T07:23:10.371 INFO:teuthology.orchestra.run.smithi143.stdout: "flags": 1, 2023-11-26T07:23:10.371 INFO:teuthology.orchestra.run.smithi143.stdout: "default_value": "False", 2023-11-26T07:23:10.372 INFO:teuthology.orchestra.run.smithi143.stdout: "min": "", 2023-11-26T07:23:10.372 INFO:teuthology.orchestra.run.smithi143.stdout: "max": "", 2023-11-26T07:23:10.372 INFO:teuthology.orchestra.run.smithi143.stdout: "enum_allowed": [], 2023-11-26T07:23:10.372 INFO:teuthology.orchestra.run.smithi143.stdout: "desc": "", 2023-11-26T07:23:10.372 INFO:teuthology.orchestra.run.smithi143.stdout: "long_desc": "", 2023-11-26T07:23:10.372 INFO:teuthology.orchestra.run.smithi143.stdout: "tags": [], 2023-11-26T07:23:10.372 INFO:teuthology.orchestra.run.smithi143.stdout: "see_also": [] 2023-11-26T07:23:10.372 INFO:teuthology.orchestra.run.smithi143.stdout: } 2023-11-26T07:23:10.372 INFO:teuthology.orchestra.run.smithi143.stdout: } 2023-11-26T07:23:10.372 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2023-11-26T07:23:10.372 INFO:teuthology.orchestra.run.smithi143.stdout: { 2023-11-26T07:23:10.372 INFO:teuthology.orchestra.run.smithi143.stdout: "name": "stats", 2023-11-26T07:23:10.372 INFO:teuthology.orchestra.run.smithi143.stdout: "can_run": true, 2023-11-26T07:23:10.373 INFO:teuthology.orchestra.run.smithi143.stdout: "error_string": "", 2023-11-26T07:23:10.373 INFO:teuthology.orchestra.run.smithi143.stdout: "module_options": { 2023-11-26T07:23:10.373 INFO:teuthology.orchestra.run.smithi143.stdout: "log_level": { 2023-11-26T07:23:10.373 INFO:teuthology.orchestra.run.smithi143.stdout: "name": "log_level", 2023-11-26T07:23:10.373 INFO:teuthology.orchestra.run.smithi143.stdout: "type": "str", 2023-11-26T07:23:10.373 INFO:teuthology.orchestra.run.smithi143.stdout: "level": "advanced", 2023-11-26T07:23:10.373 INFO:teuthology.orchestra.run.smithi143.stdout: "flags": 1, 2023-11-26T07:23:10.373 INFO:teuthology.orchestra.run.smithi143.stdout: "default_value": "", 2023-11-26T07:23:10.373 INFO:teuthology.orchestra.run.smithi143.stdout: "min": "", 2023-11-26T07:23:10.373 INFO:teuthology.orchestra.run.smithi143.stdout: "max": "", 2023-11-26T07:23:10.373 INFO:teuthology.orchestra.run.smithi143.stdout: "enum_allowed": [ 2023-11-26T07:23:10.373 INFO:teuthology.orchestra.run.smithi143.stdout: "", 2023-11-26T07:23:10.373 INFO:teuthology.orchestra.run.smithi143.stdout: "critical", 2023-11-26T07:23:10.374 INFO:teuthology.orchestra.run.smithi143.stdout: "debug", 2023-11-26T07:23:10.374 INFO:teuthology.orchestra.run.smithi143.stdout: "error", 2023-11-26T07:23:10.374 INFO:teuthology.orchestra.run.smithi143.stdout: "info", 2023-11-26T07:23:10.374 INFO:teuthology.orchestra.run.smithi143.stdout: "warning" 2023-11-26T07:23:10.374 INFO:teuthology.orchestra.run.smithi143.stdout: ], 2023-11-26T07:23:10.374 INFO:teuthology.orchestra.run.smithi143.stdout: "desc": "", 2023-11-26T07:23:10.374 INFO:teuthology.orchestra.run.smithi143.stdout: "long_desc": "", 2023-11-26T07:23:10.374 INFO:teuthology.orchestra.run.smithi143.stdout: "tags": [], 2023-11-26T07:23:10.374 INFO:teuthology.orchestra.run.smithi143.stdout: "see_also": [] 2023-11-26T07:23:10.374 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2023-11-26T07:23:10.374 INFO:teuthology.orchestra.run.smithi143.stdout: "log_to_cluster": { 2023-11-26T07:23:10.374 INFO:teuthology.orchestra.run.smithi143.stdout: "name": "log_to_cluster", 2023-11-26T07:23:10.375 INFO:teuthology.orchestra.run.smithi143.stdout: "type": "bool", 2023-11-26T07:23:10.375 INFO:teuthology.orchestra.run.smithi143.stdout: "level": "advanced", 2023-11-26T07:23:10.375 INFO:teuthology.orchestra.run.smithi143.stdout: "flags": 1, 2023-11-26T07:23:10.375 INFO:teuthology.orchestra.run.smithi143.stdout: "default_value": "False", 2023-11-26T07:23:10.375 INFO:teuthology.orchestra.run.smithi143.stdout: "min": "", 2023-11-26T07:23:10.375 INFO:teuthology.orchestra.run.smithi143.stdout: "max": "", 2023-11-26T07:23:10.375 INFO:teuthology.orchestra.run.smithi143.stdout: "enum_allowed": [], 2023-11-26T07:23:10.375 INFO:teuthology.orchestra.run.smithi143.stdout: "desc": "", 2023-11-26T07:23:10.375 INFO:teuthology.orchestra.run.smithi143.stdout: "long_desc": "", 2023-11-26T07:23:10.375 INFO:teuthology.orchestra.run.smithi143.stdout: "tags": [], 2023-11-26T07:23:10.375 INFO:teuthology.orchestra.run.smithi143.stdout: "see_also": [] 2023-11-26T07:23:10.375 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2023-11-26T07:23:10.375 INFO:teuthology.orchestra.run.smithi143.stdout: "log_to_cluster_level": { 2023-11-26T07:23:10.375 INFO:teuthology.orchestra.run.smithi143.stdout: "name": "log_to_cluster_level", 2023-11-26T07:23:10.376 INFO:teuthology.orchestra.run.smithi143.stdout: "type": "str", 2023-11-26T07:23:10.376 INFO:teuthology.orchestra.run.smithi143.stdout: "level": "advanced", 2023-11-26T07:23:10.376 INFO:teuthology.orchestra.run.smithi143.stdout: "flags": 1, 2023-11-26T07:23:10.376 INFO:teuthology.orchestra.run.smithi143.stdout: "default_value": "info", 2023-11-26T07:23:10.376 INFO:teuthology.orchestra.run.smithi143.stdout: "min": "", 2023-11-26T07:23:10.376 INFO:teuthology.orchestra.run.smithi143.stdout: "max": "", 2023-11-26T07:23:10.376 INFO:teuthology.orchestra.run.smithi143.stdout: "enum_allowed": [ 2023-11-26T07:23:10.376 INFO:teuthology.orchestra.run.smithi143.stdout: "", 2023-11-26T07:23:10.376 INFO:teuthology.orchestra.run.smithi143.stdout: "critical", 2023-11-26T07:23:10.376 INFO:teuthology.orchestra.run.smithi143.stdout: "debug", 2023-11-26T07:23:10.376 INFO:teuthology.orchestra.run.smithi143.stdout: "error", 2023-11-26T07:23:10.376 INFO:teuthology.orchestra.run.smithi143.stdout: "info", 2023-11-26T07:23:10.376 INFO:teuthology.orchestra.run.smithi143.stdout: "warning" 2023-11-26T07:23:10.377 INFO:teuthology.orchestra.run.smithi143.stdout: ], 2023-11-26T07:23:10.377 INFO:teuthology.orchestra.run.smithi143.stdout: "desc": "", 2023-11-26T07:23:10.377 INFO:teuthology.orchestra.run.smithi143.stdout: "long_desc": "", 2023-11-26T07:23:10.377 INFO:teuthology.orchestra.run.smithi143.stdout: "tags": [], 2023-11-26T07:23:10.377 INFO:teuthology.orchestra.run.smithi143.stdout: "see_also": [] 2023-11-26T07:23:10.377 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2023-11-26T07:23:10.377 INFO:teuthology.orchestra.run.smithi143.stdout: "log_to_file": { 2023-11-26T07:23:10.377 INFO:teuthology.orchestra.run.smithi143.stdout: "name": "log_to_file", 2023-11-26T07:23:10.377 INFO:teuthology.orchestra.run.smithi143.stdout: "type": "bool", 2023-11-26T07:23:10.377 INFO:teuthology.orchestra.run.smithi143.stdout: "level": "advanced", 2023-11-26T07:23:10.377 INFO:teuthology.orchestra.run.smithi143.stdout: "flags": 1, 2023-11-26T07:23:10.377 INFO:teuthology.orchestra.run.smithi143.stdout: "default_value": "False", 2023-11-26T07:23:10.377 INFO:teuthology.orchestra.run.smithi143.stdout: "min": "", 2023-11-26T07:23:10.377 INFO:teuthology.orchestra.run.smithi143.stdout: "max": "", 2023-11-26T07:23:10.378 INFO:teuthology.orchestra.run.smithi143.stdout: "enum_allowed": [], 2023-11-26T07:23:10.378 INFO:teuthology.orchestra.run.smithi143.stdout: "desc": "", 2023-11-26T07:23:10.378 INFO:teuthology.orchestra.run.smithi143.stdout: "long_desc": "", 2023-11-26T07:23:10.378 INFO:teuthology.orchestra.run.smithi143.stdout: "tags": [], 2023-11-26T07:23:10.378 INFO:teuthology.orchestra.run.smithi143.stdout: "see_also": [] 2023-11-26T07:23:10.378 INFO:teuthology.orchestra.run.smithi143.stdout: } 2023-11-26T07:23:10.378 INFO:teuthology.orchestra.run.smithi143.stdout: } 2023-11-26T07:23:10.378 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2023-11-26T07:23:10.378 INFO:teuthology.orchestra.run.smithi143.stdout: { 2023-11-26T07:23:10.378 INFO:teuthology.orchestra.run.smithi143.stdout: "name": "status", 2023-11-26T07:23:10.378 INFO:teuthology.orchestra.run.smithi143.stdout: "can_run": true, 2023-11-26T07:23:10.378 INFO:teuthology.orchestra.run.smithi143.stdout: "error_string": "", 2023-11-26T07:23:10.379 INFO:teuthology.orchestra.run.smithi143.stdout: "module_options": { 2023-11-26T07:23:10.379 INFO:teuthology.orchestra.run.smithi143.stdout: "log_level": { 2023-11-26T07:23:10.379 INFO:teuthology.orchestra.run.smithi143.stdout: "name": "log_level", 2023-11-26T07:23:10.379 INFO:teuthology.orchestra.run.smithi143.stdout: "type": "str", 2023-11-26T07:23:10.379 INFO:teuthology.orchestra.run.smithi143.stdout: "level": "advanced", 2023-11-26T07:23:10.379 INFO:teuthology.orchestra.run.smithi143.stdout: "flags": 1, 2023-11-26T07:23:10.379 INFO:teuthology.orchestra.run.smithi143.stdout: "default_value": "", 2023-11-26T07:23:10.379 INFO:teuthology.orchestra.run.smithi143.stdout: "min": "", 2023-11-26T07:23:10.379 INFO:teuthology.orchestra.run.smithi143.stdout: "max": "", 2023-11-26T07:23:10.379 INFO:teuthology.orchestra.run.smithi143.stdout: "enum_allowed": [ 2023-11-26T07:23:10.379 INFO:teuthology.orchestra.run.smithi143.stdout: "", 2023-11-26T07:23:10.379 INFO:teuthology.orchestra.run.smithi143.stdout: "critical", 2023-11-26T07:23:10.380 INFO:teuthology.orchestra.run.smithi143.stdout: "debug", 2023-11-26T07:23:10.380 INFO:teuthology.orchestra.run.smithi143.stdout: "error", 2023-11-26T07:23:10.380 INFO:teuthology.orchestra.run.smithi143.stdout: "info", 2023-11-26T07:23:10.380 INFO:teuthology.orchestra.run.smithi143.stdout: "warning" 2023-11-26T07:23:10.380 INFO:teuthology.orchestra.run.smithi143.stdout: ], 2023-11-26T07:23:10.380 INFO:teuthology.orchestra.run.smithi143.stdout: "desc": "", 2023-11-26T07:23:10.380 INFO:teuthology.orchestra.run.smithi143.stdout: "long_desc": "", 2023-11-26T07:23:10.380 INFO:teuthology.orchestra.run.smithi143.stdout: "tags": [], 2023-11-26T07:23:10.380 INFO:teuthology.orchestra.run.smithi143.stdout: "see_also": [] 2023-11-26T07:23:10.380 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2023-11-26T07:23:10.380 INFO:teuthology.orchestra.run.smithi143.stdout: "log_to_cluster": { 2023-11-26T07:23:10.380 INFO:teuthology.orchestra.run.smithi143.stdout: "name": "log_to_cluster", 2023-11-26T07:23:10.380 INFO:teuthology.orchestra.run.smithi143.stdout: "type": "bool", 2023-11-26T07:23:10.381 INFO:teuthology.orchestra.run.smithi143.stdout: "level": "advanced", 2023-11-26T07:23:10.381 INFO:teuthology.orchestra.run.smithi143.stdout: "flags": 1, 2023-11-26T07:23:10.381 INFO:teuthology.orchestra.run.smithi143.stdout: "default_value": "False", 2023-11-26T07:23:10.381 INFO:teuthology.orchestra.run.smithi143.stdout: "min": "", 2023-11-26T07:23:10.381 INFO:teuthology.orchestra.run.smithi143.stdout: "max": "", 2023-11-26T07:23:10.381 INFO:teuthology.orchestra.run.smithi143.stdout: "enum_allowed": [], 2023-11-26T07:23:10.381 INFO:teuthology.orchestra.run.smithi143.stdout: "desc": "", 2023-11-26T07:23:10.381 INFO:teuthology.orchestra.run.smithi143.stdout: "long_desc": "", 2023-11-26T07:23:10.381 INFO:teuthology.orchestra.run.smithi143.stdout: "tags": [], 2023-11-26T07:23:10.381 INFO:teuthology.orchestra.run.smithi143.stdout: "see_also": [] 2023-11-26T07:23:10.381 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2023-11-26T07:23:10.381 INFO:teuthology.orchestra.run.smithi143.stdout: "log_to_cluster_level": { 2023-11-26T07:23:10.381 INFO:teuthology.orchestra.run.smithi143.stdout: "name": "log_to_cluster_level", 2023-11-26T07:23:10.382 INFO:teuthology.orchestra.run.smithi143.stdout: "type": "str", 2023-11-26T07:23:10.382 INFO:teuthology.orchestra.run.smithi143.stdout: "level": "advanced", 2023-11-26T07:23:10.382 INFO:teuthology.orchestra.run.smithi143.stdout: "flags": 1, 2023-11-26T07:23:10.382 INFO:teuthology.orchestra.run.smithi143.stdout: "default_value": "info", 2023-11-26T07:23:10.382 INFO:teuthology.orchestra.run.smithi143.stdout: "min": "", 2023-11-26T07:23:10.382 INFO:teuthology.orchestra.run.smithi143.stdout: "max": "", 2023-11-26T07:23:10.382 INFO:teuthology.orchestra.run.smithi143.stdout: "enum_allowed": [ 2023-11-26T07:23:10.382 INFO:teuthology.orchestra.run.smithi143.stdout: "", 2023-11-26T07:23:10.382 INFO:teuthology.orchestra.run.smithi143.stdout: "critical", 2023-11-26T07:23:10.382 INFO:teuthology.orchestra.run.smithi143.stdout: "debug", 2023-11-26T07:23:10.382 INFO:teuthology.orchestra.run.smithi143.stdout: "error", 2023-11-26T07:23:10.382 INFO:teuthology.orchestra.run.smithi143.stdout: "info", 2023-11-26T07:23:10.382 INFO:teuthology.orchestra.run.smithi143.stdout: "warning" 2023-11-26T07:23:10.383 INFO:teuthology.orchestra.run.smithi143.stdout: ], 2023-11-26T07:23:10.383 INFO:teuthology.orchestra.run.smithi143.stdout: "desc": "", 2023-11-26T07:23:10.383 INFO:teuthology.orchestra.run.smithi143.stdout: "long_desc": "", 2023-11-26T07:23:10.383 INFO:teuthology.orchestra.run.smithi143.stdout: "tags": [], 2023-11-26T07:23:10.383 INFO:teuthology.orchestra.run.smithi143.stdout: "see_also": [] 2023-11-26T07:23:10.383 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2023-11-26T07:23:10.383 INFO:teuthology.orchestra.run.smithi143.stdout: "log_to_file": { 2023-11-26T07:23:10.383 INFO:teuthology.orchestra.run.smithi143.stdout: "name": "log_to_file", 2023-11-26T07:23:10.383 INFO:teuthology.orchestra.run.smithi143.stdout: "type": "bool", 2023-11-26T07:23:10.383 INFO:teuthology.orchestra.run.smithi143.stdout: "level": "advanced", 2023-11-26T07:23:10.383 INFO:teuthology.orchestra.run.smithi143.stdout: "flags": 1, 2023-11-26T07:23:10.383 INFO:teuthology.orchestra.run.smithi143.stdout: "default_value": "False", 2023-11-26T07:23:10.384 INFO:teuthology.orchestra.run.smithi143.stdout: "min": "", 2023-11-26T07:23:10.384 INFO:teuthology.orchestra.run.smithi143.stdout: "max": "", 2023-11-26T07:23:10.384 INFO:teuthology.orchestra.run.smithi143.stdout: "enum_allowed": [], 2023-11-26T07:23:10.384 INFO:teuthology.orchestra.run.smithi143.stdout: "desc": "", 2023-11-26T07:23:10.384 INFO:teuthology.orchestra.run.smithi143.stdout: "long_desc": "", 2023-11-26T07:23:10.384 INFO:teuthology.orchestra.run.smithi143.stdout: "tags": [], 2023-11-26T07:23:10.384 INFO:teuthology.orchestra.run.smithi143.stdout: "see_also": [] 2023-11-26T07:23:10.384 INFO:teuthology.orchestra.run.smithi143.stdout: } 2023-11-26T07:23:10.384 INFO:teuthology.orchestra.run.smithi143.stdout: } 2023-11-26T07:23:10.384 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2023-11-26T07:23:10.384 INFO:teuthology.orchestra.run.smithi143.stdout: { 2023-11-26T07:23:10.385 INFO:teuthology.orchestra.run.smithi143.stdout: "name": "telegraf", 2023-11-26T07:23:10.385 INFO:teuthology.orchestra.run.smithi143.stdout: "can_run": true, 2023-11-26T07:23:10.385 INFO:teuthology.orchestra.run.smithi143.stdout: "error_string": "", 2023-11-26T07:23:10.385 INFO:teuthology.orchestra.run.smithi143.stdout: "module_options": { 2023-11-26T07:23:10.385 INFO:teuthology.orchestra.run.smithi143.stdout: "address": { 2023-11-26T07:23:10.385 INFO:teuthology.orchestra.run.smithi143.stdout: "name": "address", 2023-11-26T07:23:10.385 INFO:teuthology.orchestra.run.smithi143.stdout: "type": "str", 2023-11-26T07:23:10.385 INFO:teuthology.orchestra.run.smithi143.stdout: "level": "advanced", 2023-11-26T07:23:10.385 INFO:teuthology.orchestra.run.smithi143.stdout: "flags": 0, 2023-11-26T07:23:10.385 INFO:teuthology.orchestra.run.smithi143.stdout: "default_value": "unixgram:///tmp/telegraf.sock", 2023-11-26T07:23:10.385 INFO:teuthology.orchestra.run.smithi143.stdout: "min": "", 2023-11-26T07:23:10.385 INFO:teuthology.orchestra.run.smithi143.stdout: "max": "", 2023-11-26T07:23:10.385 INFO:teuthology.orchestra.run.smithi143.stdout: "enum_allowed": [], 2023-11-26T07:23:10.386 INFO:teuthology.orchestra.run.smithi143.stdout: "desc": "", 2023-11-26T07:23:10.386 INFO:teuthology.orchestra.run.smithi143.stdout: "long_desc": "", 2023-11-26T07:23:10.386 INFO:teuthology.orchestra.run.smithi143.stdout: "tags": [], 2023-11-26T07:23:10.386 INFO:teuthology.orchestra.run.smithi143.stdout: "see_also": [] 2023-11-26T07:23:10.386 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2023-11-26T07:23:10.386 INFO:teuthology.orchestra.run.smithi143.stdout: "interval": { 2023-11-26T07:23:10.386 INFO:teuthology.orchestra.run.smithi143.stdout: "name": "interval", 2023-11-26T07:23:10.386 INFO:teuthology.orchestra.run.smithi143.stdout: "type": "secs", 2023-11-26T07:23:10.386 INFO:teuthology.orchestra.run.smithi143.stdout: "level": "advanced", 2023-11-26T07:23:10.386 INFO:teuthology.orchestra.run.smithi143.stdout: "flags": 0, 2023-11-26T07:23:10.386 INFO:teuthology.orchestra.run.smithi143.stdout: "default_value": "15", 2023-11-26T07:23:10.386 INFO:teuthology.orchestra.run.smithi143.stdout: "min": "", 2023-11-26T07:23:10.387 INFO:teuthology.orchestra.run.smithi143.stdout: "max": "", 2023-11-26T07:23:10.387 INFO:teuthology.orchestra.run.smithi143.stdout: "enum_allowed": [], 2023-11-26T07:23:10.387 INFO:teuthology.orchestra.run.smithi143.stdout: "desc": "", 2023-11-26T07:23:10.387 INFO:teuthology.orchestra.run.smithi143.stdout: "long_desc": "", 2023-11-26T07:23:10.387 INFO:teuthology.orchestra.run.smithi143.stdout: "tags": [], 2023-11-26T07:23:10.387 INFO:teuthology.orchestra.run.smithi143.stdout: "see_also": [] 2023-11-26T07:23:10.387 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2023-11-26T07:23:10.387 INFO:teuthology.orchestra.run.smithi143.stdout: "log_level": { 2023-11-26T07:23:10.387 INFO:teuthology.orchestra.run.smithi143.stdout: "name": "log_level", 2023-11-26T07:23:10.387 INFO:teuthology.orchestra.run.smithi143.stdout: "type": "str", 2023-11-26T07:23:10.387 INFO:teuthology.orchestra.run.smithi143.stdout: "level": "advanced", 2023-11-26T07:23:10.387 INFO:teuthology.orchestra.run.smithi143.stdout: "flags": 1, 2023-11-26T07:23:10.388 INFO:teuthology.orchestra.run.smithi143.stdout: "default_value": "", 2023-11-26T07:23:10.388 INFO:teuthology.orchestra.run.smithi143.stdout: "min": "", 2023-11-26T07:23:10.388 INFO:teuthology.orchestra.run.smithi143.stdout: "max": "", 2023-11-26T07:23:10.388 INFO:teuthology.orchestra.run.smithi143.stdout: "enum_allowed": [ 2023-11-26T07:23:10.388 INFO:teuthology.orchestra.run.smithi143.stdout: "", 2023-11-26T07:23:10.388 INFO:teuthology.orchestra.run.smithi143.stdout: "critical", 2023-11-26T07:23:10.388 INFO:teuthology.orchestra.run.smithi143.stdout: "debug", 2023-11-26T07:23:10.388 INFO:teuthology.orchestra.run.smithi143.stdout: "error", 2023-11-26T07:23:10.388 INFO:teuthology.orchestra.run.smithi143.stdout: "info", 2023-11-26T07:23:10.388 INFO:teuthology.orchestra.run.smithi143.stdout: "warning" 2023-11-26T07:23:10.388 INFO:teuthology.orchestra.run.smithi143.stdout: ], 2023-11-26T07:23:10.388 INFO:teuthology.orchestra.run.smithi143.stdout: "desc": "", 2023-11-26T07:23:10.388 INFO:teuthology.orchestra.run.smithi143.stdout: "long_desc": "", 2023-11-26T07:23:10.389 INFO:teuthology.orchestra.run.smithi143.stdout: "tags": [], 2023-11-26T07:23:10.389 INFO:teuthology.orchestra.run.smithi143.stdout: "see_also": [] 2023-11-26T07:23:10.389 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2023-11-26T07:23:10.389 INFO:teuthology.orchestra.run.smithi143.stdout: "log_to_cluster": { 2023-11-26T07:23:10.389 INFO:teuthology.orchestra.run.smithi143.stdout: "name": "log_to_cluster", 2023-11-26T07:23:10.389 INFO:teuthology.orchestra.run.smithi143.stdout: "type": "bool", 2023-11-26T07:23:10.389 INFO:teuthology.orchestra.run.smithi143.stdout: "level": "advanced", 2023-11-26T07:23:10.389 INFO:teuthology.orchestra.run.smithi143.stdout: "flags": 1, 2023-11-26T07:23:10.389 INFO:teuthology.orchestra.run.smithi143.stdout: "default_value": "False", 2023-11-26T07:23:10.389 INFO:teuthology.orchestra.run.smithi143.stdout: "min": "", 2023-11-26T07:23:10.389 INFO:teuthology.orchestra.run.smithi143.stdout: "max": "", 2023-11-26T07:23:10.389 INFO:teuthology.orchestra.run.smithi143.stdout: "enum_allowed": [], 2023-11-26T07:23:10.389 INFO:teuthology.orchestra.run.smithi143.stdout: "desc": "", 2023-11-26T07:23:10.390 INFO:teuthology.orchestra.run.smithi143.stdout: "long_desc": "", 2023-11-26T07:23:10.390 INFO:teuthology.orchestra.run.smithi143.stdout: "tags": [], 2023-11-26T07:23:10.390 INFO:teuthology.orchestra.run.smithi143.stdout: "see_also": [] 2023-11-26T07:23:10.390 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2023-11-26T07:23:10.390 INFO:teuthology.orchestra.run.smithi143.stdout: "log_to_cluster_level": { 2023-11-26T07:23:10.390 INFO:teuthology.orchestra.run.smithi143.stdout: "name": "log_to_cluster_level", 2023-11-26T07:23:10.390 INFO:teuthology.orchestra.run.smithi143.stdout: "type": "str", 2023-11-26T07:23:10.390 INFO:teuthology.orchestra.run.smithi143.stdout: "level": "advanced", 2023-11-26T07:23:10.390 INFO:teuthology.orchestra.run.smithi143.stdout: "flags": 1, 2023-11-26T07:23:10.390 INFO:teuthology.orchestra.run.smithi143.stdout: "default_value": "info", 2023-11-26T07:23:10.390 INFO:teuthology.orchestra.run.smithi143.stdout: "min": "", 2023-11-26T07:23:10.390 INFO:teuthology.orchestra.run.smithi143.stdout: "max": "", 2023-11-26T07:23:10.391 INFO:teuthology.orchestra.run.smithi143.stdout: "enum_allowed": [ 2023-11-26T07:23:10.391 INFO:teuthology.orchestra.run.smithi143.stdout: "", 2023-11-26T07:23:10.391 INFO:teuthology.orchestra.run.smithi143.stdout: "critical", 2023-11-26T07:23:10.391 INFO:teuthology.orchestra.run.smithi143.stdout: "debug", 2023-11-26T07:23:10.391 INFO:teuthology.orchestra.run.smithi143.stdout: "error", 2023-11-26T07:23:10.391 INFO:teuthology.orchestra.run.smithi143.stdout: "info", 2023-11-26T07:23:10.391 INFO:teuthology.orchestra.run.smithi143.stdout: "warning" 2023-11-26T07:23:10.391 INFO:teuthology.orchestra.run.smithi143.stdout: ], 2023-11-26T07:23:10.391 INFO:teuthology.orchestra.run.smithi143.stdout: "desc": "", 2023-11-26T07:23:10.391 INFO:teuthology.orchestra.run.smithi143.stdout: "long_desc": "", 2023-11-26T07:23:10.391 INFO:teuthology.orchestra.run.smithi143.stdout: "tags": [], 2023-11-26T07:23:10.391 INFO:teuthology.orchestra.run.smithi143.stdout: "see_also": [] 2023-11-26T07:23:10.391 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2023-11-26T07:23:10.392 INFO:teuthology.orchestra.run.smithi143.stdout: "log_to_file": { 2023-11-26T07:23:10.392 INFO:teuthology.orchestra.run.smithi143.stdout: "name": "log_to_file", 2023-11-26T07:23:10.392 INFO:teuthology.orchestra.run.smithi143.stdout: "type": "bool", 2023-11-26T07:23:10.392 INFO:teuthology.orchestra.run.smithi143.stdout: "level": "advanced", 2023-11-26T07:23:10.392 INFO:teuthology.orchestra.run.smithi143.stdout: "flags": 1, 2023-11-26T07:23:10.392 INFO:teuthology.orchestra.run.smithi143.stdout: "default_value": "False", 2023-11-26T07:23:10.392 INFO:teuthology.orchestra.run.smithi143.stdout: "min": "", 2023-11-26T07:23:10.392 INFO:teuthology.orchestra.run.smithi143.stdout: "max": "", 2023-11-26T07:23:10.392 INFO:teuthology.orchestra.run.smithi143.stdout: "enum_allowed": [], 2023-11-26T07:23:10.392 INFO:teuthology.orchestra.run.smithi143.stdout: "desc": "", 2023-11-26T07:23:10.393 INFO:teuthology.orchestra.run.smithi143.stdout: "long_desc": "", 2023-11-26T07:23:10.393 INFO:teuthology.orchestra.run.smithi143.stdout: "tags": [], 2023-11-26T07:23:10.393 INFO:teuthology.orchestra.run.smithi143.stdout: "see_also": [] 2023-11-26T07:23:10.393 INFO:teuthology.orchestra.run.smithi143.stdout: } 2023-11-26T07:23:10.393 INFO:teuthology.orchestra.run.smithi143.stdout: } 2023-11-26T07:23:10.393 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2023-11-26T07:23:10.393 INFO:teuthology.orchestra.run.smithi143.stdout: { 2023-11-26T07:23:10.393 INFO:teuthology.orchestra.run.smithi143.stdout: "name": "telemetry", 2023-11-26T07:23:10.393 INFO:teuthology.orchestra.run.smithi143.stdout: "can_run": true, 2023-11-26T07:23:10.393 INFO:teuthology.orchestra.run.smithi143.stdout: "error_string": "", 2023-11-26T07:23:10.393 INFO:teuthology.orchestra.run.smithi143.stdout: "module_options": { 2023-11-26T07:23:10.393 INFO:teuthology.orchestra.run.smithi143.stdout: "channel_basic": { 2023-11-26T07:23:10.394 INFO:teuthology.orchestra.run.smithi143.stdout: "name": "channel_basic", 2023-11-26T07:23:10.394 INFO:teuthology.orchestra.run.smithi143.stdout: "type": "bool", 2023-11-26T07:23:10.394 INFO:teuthology.orchestra.run.smithi143.stdout: "level": "advanced", 2023-11-26T07:23:10.394 INFO:teuthology.orchestra.run.smithi143.stdout: "flags": 0, 2023-11-26T07:23:10.394 INFO:teuthology.orchestra.run.smithi143.stdout: "default_value": "True", 2023-11-26T07:23:10.394 INFO:teuthology.orchestra.run.smithi143.stdout: "min": "", 2023-11-26T07:23:10.394 INFO:teuthology.orchestra.run.smithi143.stdout: "max": "", 2023-11-26T07:23:10.394 INFO:teuthology.orchestra.run.smithi143.stdout: "enum_allowed": [], 2023-11-26T07:23:10.394 INFO:teuthology.orchestra.run.smithi143.stdout: "desc": "Share basic cluster information (size, version)", 2023-11-26T07:23:10.394 INFO:teuthology.orchestra.run.smithi143.stdout: "long_desc": "", 2023-11-26T07:23:10.394 INFO:teuthology.orchestra.run.smithi143.stdout: "tags": [], 2023-11-26T07:23:10.394 INFO:teuthology.orchestra.run.smithi143.stdout: "see_also": [] 2023-11-26T07:23:10.394 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2023-11-26T07:23:10.395 INFO:teuthology.orchestra.run.smithi143.stdout: "channel_crash": { 2023-11-26T07:23:10.395 INFO:teuthology.orchestra.run.smithi143.stdout: "name": "channel_crash", 2023-11-26T07:23:10.395 INFO:teuthology.orchestra.run.smithi143.stdout: "type": "bool", 2023-11-26T07:23:10.395 INFO:teuthology.orchestra.run.smithi143.stdout: "level": "advanced", 2023-11-26T07:23:10.395 INFO:teuthology.orchestra.run.smithi143.stdout: "flags": 0, 2023-11-26T07:23:10.395 INFO:teuthology.orchestra.run.smithi143.stdout: "default_value": "True", 2023-11-26T07:23:10.395 INFO:teuthology.orchestra.run.smithi143.stdout: "min": "", 2023-11-26T07:23:10.395 INFO:teuthology.orchestra.run.smithi143.stdout: "max": "", 2023-11-26T07:23:10.395 INFO:teuthology.orchestra.run.smithi143.stdout: "enum_allowed": [], 2023-11-26T07:23:10.395 INFO:teuthology.orchestra.run.smithi143.stdout: "desc": "Share metadata about Ceph daemon crashes (version, stack straces, etc)", 2023-11-26T07:23:10.395 INFO:teuthology.orchestra.run.smithi143.stdout: "long_desc": "", 2023-11-26T07:23:10.395 INFO:teuthology.orchestra.run.smithi143.stdout: "tags": [], 2023-11-26T07:23:10.395 INFO:teuthology.orchestra.run.smithi143.stdout: "see_also": [] 2023-11-26T07:23:10.396 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2023-11-26T07:23:10.396 INFO:teuthology.orchestra.run.smithi143.stdout: "channel_device": { 2023-11-26T07:23:10.396 INFO:teuthology.orchestra.run.smithi143.stdout: "name": "channel_device", 2023-11-26T07:23:10.396 INFO:teuthology.orchestra.run.smithi143.stdout: "type": "bool", 2023-11-26T07:23:10.396 INFO:teuthology.orchestra.run.smithi143.stdout: "level": "advanced", 2023-11-26T07:23:10.396 INFO:teuthology.orchestra.run.smithi143.stdout: "flags": 0, 2023-11-26T07:23:10.396 INFO:teuthology.orchestra.run.smithi143.stdout: "default_value": "True", 2023-11-26T07:23:10.396 INFO:teuthology.orchestra.run.smithi143.stdout: "min": "", 2023-11-26T07:23:10.396 INFO:teuthology.orchestra.run.smithi143.stdout: "max": "", 2023-11-26T07:23:10.396 INFO:teuthology.orchestra.run.smithi143.stdout: "enum_allowed": [], 2023-11-26T07:23:10.396 INFO:teuthology.orchestra.run.smithi143.stdout: "desc": "Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)", 2023-11-26T07:23:10.396 INFO:teuthology.orchestra.run.smithi143.stdout: "long_desc": "", 2023-11-26T07:23:10.396 INFO:teuthology.orchestra.run.smithi143.stdout: "tags": [], 2023-11-26T07:23:10.397 INFO:teuthology.orchestra.run.smithi143.stdout: "see_also": [] 2023-11-26T07:23:10.397 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2023-11-26T07:23:10.397 INFO:teuthology.orchestra.run.smithi143.stdout: "channel_ident": { 2023-11-26T07:23:10.397 INFO:teuthology.orchestra.run.smithi143.stdout: "name": "channel_ident", 2023-11-26T07:23:10.397 INFO:teuthology.orchestra.run.smithi143.stdout: "type": "bool", 2023-11-26T07:23:10.397 INFO:teuthology.orchestra.run.smithi143.stdout: "level": "advanced", 2023-11-26T07:23:10.397 INFO:teuthology.orchestra.run.smithi143.stdout: "flags": 0, 2023-11-26T07:23:10.397 INFO:teuthology.orchestra.run.smithi143.stdout: "default_value": "False", 2023-11-26T07:23:10.397 INFO:teuthology.orchestra.run.smithi143.stdout: "min": "", 2023-11-26T07:23:10.397 INFO:teuthology.orchestra.run.smithi143.stdout: "max": "", 2023-11-26T07:23:10.397 INFO:teuthology.orchestra.run.smithi143.stdout: "enum_allowed": [], 2023-11-26T07:23:10.397 INFO:teuthology.orchestra.run.smithi143.stdout: "desc": "Share a user-provided description and/or contact email for the cluster", 2023-11-26T07:23:10.397 INFO:teuthology.orchestra.run.smithi143.stdout: "long_desc": "", 2023-11-26T07:23:10.398 INFO:teuthology.orchestra.run.smithi143.stdout: "tags": [], 2023-11-26T07:23:10.398 INFO:teuthology.orchestra.run.smithi143.stdout: "see_also": [] 2023-11-26T07:23:10.398 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2023-11-26T07:23:10.398 INFO:teuthology.orchestra.run.smithi143.stdout: "channel_perf": { 2023-11-26T07:23:10.398 INFO:teuthology.orchestra.run.smithi143.stdout: "name": "channel_perf", 2023-11-26T07:23:10.398 INFO:teuthology.orchestra.run.smithi143.stdout: "type": "bool", 2023-11-26T07:23:10.398 INFO:teuthology.orchestra.run.smithi143.stdout: "level": "advanced", 2023-11-26T07:23:10.398 INFO:teuthology.orchestra.run.smithi143.stdout: "flags": 0, 2023-11-26T07:23:10.398 INFO:teuthology.orchestra.run.smithi143.stdout: "default_value": "False", 2023-11-26T07:23:10.398 INFO:teuthology.orchestra.run.smithi143.stdout: "min": "", 2023-11-26T07:23:10.398 INFO:teuthology.orchestra.run.smithi143.stdout: "max": "", 2023-11-26T07:23:10.398 INFO:teuthology.orchestra.run.smithi143.stdout: "enum_allowed": [], 2023-11-26T07:23:10.398 INFO:teuthology.orchestra.run.smithi143.stdout: "desc": "Share various performance metrics of a cluster", 2023-11-26T07:23:10.399 INFO:teuthology.orchestra.run.smithi143.stdout: "long_desc": "", 2023-11-26T07:23:10.399 INFO:teuthology.orchestra.run.smithi143.stdout: "tags": [], 2023-11-26T07:23:10.399 INFO:teuthology.orchestra.run.smithi143.stdout: "see_also": [] 2023-11-26T07:23:10.399 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2023-11-26T07:23:10.399 INFO:teuthology.orchestra.run.smithi143.stdout: "contact": { 2023-11-26T07:23:10.399 INFO:teuthology.orchestra.run.smithi143.stdout: "name": "contact", 2023-11-26T07:23:10.399 INFO:teuthology.orchestra.run.smithi143.stdout: "type": "str", 2023-11-26T07:23:10.399 INFO:teuthology.orchestra.run.smithi143.stdout: "level": "advanced", 2023-11-26T07:23:10.399 INFO:teuthology.orchestra.run.smithi143.stdout: "flags": 0, 2023-11-26T07:23:10.399 INFO:teuthology.orchestra.run.smithi143.stdout: "default_value": "", 2023-11-26T07:23:10.399 INFO:teuthology.orchestra.run.smithi143.stdout: "min": "", 2023-11-26T07:23:10.399 INFO:teuthology.orchestra.run.smithi143.stdout: "max": "", 2023-11-26T07:23:10.399 INFO:teuthology.orchestra.run.smithi143.stdout: "enum_allowed": [], 2023-11-26T07:23:10.400 INFO:teuthology.orchestra.run.smithi143.stdout: "desc": "", 2023-11-26T07:23:10.400 INFO:teuthology.orchestra.run.smithi143.stdout: "long_desc": "", 2023-11-26T07:23:10.400 INFO:teuthology.orchestra.run.smithi143.stdout: "tags": [], 2023-11-26T07:23:10.400 INFO:teuthology.orchestra.run.smithi143.stdout: "see_also": [] 2023-11-26T07:23:10.400 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2023-11-26T07:23:10.400 INFO:teuthology.orchestra.run.smithi143.stdout: "description": { 2023-11-26T07:23:10.400 INFO:teuthology.orchestra.run.smithi143.stdout: "name": "description", 2023-11-26T07:23:10.400 INFO:teuthology.orchestra.run.smithi143.stdout: "type": "str", 2023-11-26T07:23:10.400 INFO:teuthology.orchestra.run.smithi143.stdout: "level": "advanced", 2023-11-26T07:23:10.400 INFO:teuthology.orchestra.run.smithi143.stdout: "flags": 0, 2023-11-26T07:23:10.400 INFO:teuthology.orchestra.run.smithi143.stdout: "default_value": "", 2023-11-26T07:23:10.400 INFO:teuthology.orchestra.run.smithi143.stdout: "min": "", 2023-11-26T07:23:10.400 INFO:teuthology.orchestra.run.smithi143.stdout: "max": "", 2023-11-26T07:23:10.401 INFO:teuthology.orchestra.run.smithi143.stdout: "enum_allowed": [], 2023-11-26T07:23:10.401 INFO:teuthology.orchestra.run.smithi143.stdout: "desc": "", 2023-11-26T07:23:10.401 INFO:teuthology.orchestra.run.smithi143.stdout: "long_desc": "", 2023-11-26T07:23:10.401 INFO:teuthology.orchestra.run.smithi143.stdout: "tags": [], 2023-11-26T07:23:10.401 INFO:teuthology.orchestra.run.smithi143.stdout: "see_also": [] 2023-11-26T07:23:10.401 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2023-11-26T07:23:10.401 INFO:teuthology.orchestra.run.smithi143.stdout: "device_url": { 2023-11-26T07:23:10.401 INFO:teuthology.orchestra.run.smithi143.stdout: "name": "device_url", 2023-11-26T07:23:10.401 INFO:teuthology.orchestra.run.smithi143.stdout: "type": "str", 2023-11-26T07:23:10.401 INFO:teuthology.orchestra.run.smithi143.stdout: "level": "advanced", 2023-11-26T07:23:10.401 INFO:teuthology.orchestra.run.smithi143.stdout: "flags": 0, 2023-11-26T07:23:10.401 INFO:teuthology.orchestra.run.smithi143.stdout: "default_value": "https://telemetry.ceph.com/device", 2023-11-26T07:23:10.402 INFO:teuthology.orchestra.run.smithi143.stdout: "min": "", 2023-11-26T07:23:10.402 INFO:teuthology.orchestra.run.smithi143.stdout: "max": "", 2023-11-26T07:23:10.402 INFO:teuthology.orchestra.run.smithi143.stdout: "enum_allowed": [], 2023-11-26T07:23:10.402 INFO:teuthology.orchestra.run.smithi143.stdout: "desc": "", 2023-11-26T07:23:10.402 INFO:teuthology.orchestra.run.smithi143.stdout: "long_desc": "", 2023-11-26T07:23:10.402 INFO:teuthology.orchestra.run.smithi143.stdout: "tags": [], 2023-11-26T07:23:10.402 INFO:teuthology.orchestra.run.smithi143.stdout: "see_also": [] 2023-11-26T07:23:10.402 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2023-11-26T07:23:10.402 INFO:teuthology.orchestra.run.smithi143.stdout: "enabled": { 2023-11-26T07:23:10.402 INFO:teuthology.orchestra.run.smithi143.stdout: "name": "enabled", 2023-11-26T07:23:10.402 INFO:teuthology.orchestra.run.smithi143.stdout: "type": "bool", 2023-11-26T07:23:10.402 INFO:teuthology.orchestra.run.smithi143.stdout: "level": "advanced", 2023-11-26T07:23:10.402 INFO:teuthology.orchestra.run.smithi143.stdout: "flags": 0, 2023-11-26T07:23:10.403 INFO:teuthology.orchestra.run.smithi143.stdout: "default_value": "False", 2023-11-26T07:23:10.403 INFO:teuthology.orchestra.run.smithi143.stdout: "min": "", 2023-11-26T07:23:10.403 INFO:teuthology.orchestra.run.smithi143.stdout: "max": "", 2023-11-26T07:23:10.403 INFO:teuthology.orchestra.run.smithi143.stdout: "enum_allowed": [], 2023-11-26T07:23:10.403 INFO:teuthology.orchestra.run.smithi143.stdout: "desc": "", 2023-11-26T07:23:10.403 INFO:teuthology.orchestra.run.smithi143.stdout: "long_desc": "", 2023-11-26T07:23:10.403 INFO:teuthology.orchestra.run.smithi143.stdout: "tags": [], 2023-11-26T07:23:10.403 INFO:teuthology.orchestra.run.smithi143.stdout: "see_also": [] 2023-11-26T07:23:10.403 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2023-11-26T07:23:10.403 INFO:teuthology.orchestra.run.smithi143.stdout: "interval": { 2023-11-26T07:23:10.403 INFO:teuthology.orchestra.run.smithi143.stdout: "name": "interval", 2023-11-26T07:23:10.403 INFO:teuthology.orchestra.run.smithi143.stdout: "type": "int", 2023-11-26T07:23:10.403 INFO:teuthology.orchestra.run.smithi143.stdout: "level": "advanced", 2023-11-26T07:23:10.404 INFO:teuthology.orchestra.run.smithi143.stdout: "flags": 0, 2023-11-26T07:23:10.404 INFO:teuthology.orchestra.run.smithi143.stdout: "default_value": "24", 2023-11-26T07:23:10.404 INFO:teuthology.orchestra.run.smithi143.stdout: "min": "8", 2023-11-26T07:23:10.404 INFO:teuthology.orchestra.run.smithi143.stdout: "max": "", 2023-11-26T07:23:10.404 INFO:teuthology.orchestra.run.smithi143.stdout: "enum_allowed": [], 2023-11-26T07:23:10.404 INFO:teuthology.orchestra.run.smithi143.stdout: "desc": "", 2023-11-26T07:23:10.404 INFO:teuthology.orchestra.run.smithi143.stdout: "long_desc": "", 2023-11-26T07:23:10.404 INFO:teuthology.orchestra.run.smithi143.stdout: "tags": [], 2023-11-26T07:23:10.404 INFO:teuthology.orchestra.run.smithi143.stdout: "see_also": [] 2023-11-26T07:23:10.404 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2023-11-26T07:23:10.404 INFO:teuthology.orchestra.run.smithi143.stdout: "last_opt_revision": { 2023-11-26T07:23:10.404 INFO:teuthology.orchestra.run.smithi143.stdout: "name": "last_opt_revision", 2023-11-26T07:23:10.404 INFO:teuthology.orchestra.run.smithi143.stdout: "type": "int", 2023-11-26T07:23:10.405 INFO:teuthology.orchestra.run.smithi143.stdout: "level": "advanced", 2023-11-26T07:23:10.405 INFO:teuthology.orchestra.run.smithi143.stdout: "flags": 0, 2023-11-26T07:23:10.405 INFO:teuthology.orchestra.run.smithi143.stdout: "default_value": "1", 2023-11-26T07:23:10.405 INFO:teuthology.orchestra.run.smithi143.stdout: "min": "", 2023-11-26T07:23:10.405 INFO:teuthology.orchestra.run.smithi143.stdout: "max": "", 2023-11-26T07:23:10.405 INFO:teuthology.orchestra.run.smithi143.stdout: "enum_allowed": [], 2023-11-26T07:23:10.405 INFO:teuthology.orchestra.run.smithi143.stdout: "desc": "", 2023-11-26T07:23:10.405 INFO:teuthology.orchestra.run.smithi143.stdout: "long_desc": "", 2023-11-26T07:23:10.405 INFO:teuthology.orchestra.run.smithi143.stdout: "tags": [], 2023-11-26T07:23:10.405 INFO:teuthology.orchestra.run.smithi143.stdout: "see_also": [] 2023-11-26T07:23:10.405 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2023-11-26T07:23:10.405 INFO:teuthology.orchestra.run.smithi143.stdout: "leaderboard": { 2023-11-26T07:23:10.405 INFO:teuthology.orchestra.run.smithi143.stdout: "name": "leaderboard", 2023-11-26T07:23:10.406 INFO:teuthology.orchestra.run.smithi143.stdout: "type": "bool", 2023-11-26T07:23:10.406 INFO:teuthology.orchestra.run.smithi143.stdout: "level": "advanced", 2023-11-26T07:23:10.406 INFO:teuthology.orchestra.run.smithi143.stdout: "flags": 0, 2023-11-26T07:23:10.406 INFO:teuthology.orchestra.run.smithi143.stdout: "default_value": "False", 2023-11-26T07:23:10.406 INFO:teuthology.orchestra.run.smithi143.stdout: "min": "", 2023-11-26T07:23:10.406 INFO:teuthology.orchestra.run.smithi143.stdout: "max": "", 2023-11-26T07:23:10.406 INFO:teuthology.orchestra.run.smithi143.stdout: "enum_allowed": [], 2023-11-26T07:23:10.406 INFO:teuthology.orchestra.run.smithi143.stdout: "desc": "", 2023-11-26T07:23:10.406 INFO:teuthology.orchestra.run.smithi143.stdout: "long_desc": "", 2023-11-26T07:23:10.406 INFO:teuthology.orchestra.run.smithi143.stdout: "tags": [], 2023-11-26T07:23:10.406 INFO:teuthology.orchestra.run.smithi143.stdout: "see_also": [] 2023-11-26T07:23:10.406 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2023-11-26T07:23:10.406 INFO:teuthology.orchestra.run.smithi143.stdout: "leaderboard_description": { 2023-11-26T07:23:10.407 INFO:teuthology.orchestra.run.smithi143.stdout: "name": "leaderboard_description", 2023-11-26T07:23:10.407 INFO:teuthology.orchestra.run.smithi143.stdout: "type": "str", 2023-11-26T07:23:10.407 INFO:teuthology.orchestra.run.smithi143.stdout: "level": "advanced", 2023-11-26T07:23:10.407 INFO:teuthology.orchestra.run.smithi143.stdout: "flags": 0, 2023-11-26T07:23:10.407 INFO:teuthology.orchestra.run.smithi143.stdout: "default_value": "", 2023-11-26T07:23:10.407 INFO:teuthology.orchestra.run.smithi143.stdout: "min": "", 2023-11-26T07:23:10.407 INFO:teuthology.orchestra.run.smithi143.stdout: "max": "", 2023-11-26T07:23:10.407 INFO:teuthology.orchestra.run.smithi143.stdout: "enum_allowed": [], 2023-11-26T07:23:10.407 INFO:teuthology.orchestra.run.smithi143.stdout: "desc": "", 2023-11-26T07:23:10.407 INFO:teuthology.orchestra.run.smithi143.stdout: "long_desc": "", 2023-11-26T07:23:10.407 INFO:teuthology.orchestra.run.smithi143.stdout: "tags": [], 2023-11-26T07:23:10.407 INFO:teuthology.orchestra.run.smithi143.stdout: "see_also": [] 2023-11-26T07:23:10.407 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2023-11-26T07:23:10.407 INFO:teuthology.orchestra.run.smithi143.stdout: "log_level": { 2023-11-26T07:23:10.408 INFO:teuthology.orchestra.run.smithi143.stdout: "name": "log_level", 2023-11-26T07:23:10.408 INFO:teuthology.orchestra.run.smithi143.stdout: "type": "str", 2023-11-26T07:23:10.408 INFO:teuthology.orchestra.run.smithi143.stdout: "level": "advanced", 2023-11-26T07:23:10.408 INFO:teuthology.orchestra.run.smithi143.stdout: "flags": 1, 2023-11-26T07:23:10.408 INFO:teuthology.orchestra.run.smithi143.stdout: "default_value": "", 2023-11-26T07:23:10.408 INFO:teuthology.orchestra.run.smithi143.stdout: "min": "", 2023-11-26T07:23:10.408 INFO:teuthology.orchestra.run.smithi143.stdout: "max": "", 2023-11-26T07:23:10.408 INFO:teuthology.orchestra.run.smithi143.stdout: "enum_allowed": [ 2023-11-26T07:23:10.408 INFO:teuthology.orchestra.run.smithi143.stdout: "", 2023-11-26T07:23:10.408 INFO:teuthology.orchestra.run.smithi143.stdout: "critical", 2023-11-26T07:23:10.408 INFO:teuthology.orchestra.run.smithi143.stdout: "debug", 2023-11-26T07:23:10.408 INFO:teuthology.orchestra.run.smithi143.stdout: "error", 2023-11-26T07:23:10.408 INFO:teuthology.orchestra.run.smithi143.stdout: "info", 2023-11-26T07:23:10.409 INFO:teuthology.orchestra.run.smithi143.stdout: "warning" 2023-11-26T07:23:10.409 INFO:teuthology.orchestra.run.smithi143.stdout: ], 2023-11-26T07:23:10.409 INFO:teuthology.orchestra.run.smithi143.stdout: "desc": "", 2023-11-26T07:23:10.409 INFO:teuthology.orchestra.run.smithi143.stdout: "long_desc": "", 2023-11-26T07:23:10.409 INFO:teuthology.orchestra.run.smithi143.stdout: "tags": [], 2023-11-26T07:23:10.409 INFO:teuthology.orchestra.run.smithi143.stdout: "see_also": [] 2023-11-26T07:23:10.409 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2023-11-26T07:23:10.409 INFO:teuthology.orchestra.run.smithi143.stdout: "log_to_cluster": { 2023-11-26T07:23:10.409 INFO:teuthology.orchestra.run.smithi143.stdout: "name": "log_to_cluster", 2023-11-26T07:23:10.409 INFO:teuthology.orchestra.run.smithi143.stdout: "type": "bool", 2023-11-26T07:23:10.409 INFO:teuthology.orchestra.run.smithi143.stdout: "level": "advanced", 2023-11-26T07:23:10.409 INFO:teuthology.orchestra.run.smithi143.stdout: "flags": 1, 2023-11-26T07:23:10.409 INFO:teuthology.orchestra.run.smithi143.stdout: "default_value": "False", 2023-11-26T07:23:10.410 INFO:teuthology.orchestra.run.smithi143.stdout: "min": "", 2023-11-26T07:23:10.410 INFO:teuthology.orchestra.run.smithi143.stdout: "max": "", 2023-11-26T07:23:10.410 INFO:teuthology.orchestra.run.smithi143.stdout: "enum_allowed": [], 2023-11-26T07:23:10.410 INFO:teuthology.orchestra.run.smithi143.stdout: "desc": "", 2023-11-26T07:23:10.410 INFO:teuthology.orchestra.run.smithi143.stdout: "long_desc": "", 2023-11-26T07:23:10.410 INFO:teuthology.orchestra.run.smithi143.stdout: "tags": [], 2023-11-26T07:23:10.410 INFO:teuthology.orchestra.run.smithi143.stdout: "see_also": [] 2023-11-26T07:23:10.410 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2023-11-26T07:23:10.410 INFO:teuthology.orchestra.run.smithi143.stdout: "log_to_cluster_level": { 2023-11-26T07:23:10.410 INFO:teuthology.orchestra.run.smithi143.stdout: "name": "log_to_cluster_level", 2023-11-26T07:23:10.410 INFO:teuthology.orchestra.run.smithi143.stdout: "type": "str", 2023-11-26T07:23:10.410 INFO:teuthology.orchestra.run.smithi143.stdout: "level": "advanced", 2023-11-26T07:23:10.410 INFO:teuthology.orchestra.run.smithi143.stdout: "flags": 1, 2023-11-26T07:23:10.411 INFO:teuthology.orchestra.run.smithi143.stdout: "default_value": "info", 2023-11-26T07:23:10.411 INFO:teuthology.orchestra.run.smithi143.stdout: "min": "", 2023-11-26T07:23:10.411 INFO:teuthology.orchestra.run.smithi143.stdout: "max": "", 2023-11-26T07:23:10.411 INFO:teuthology.orchestra.run.smithi143.stdout: "enum_allowed": [ 2023-11-26T07:23:10.411 INFO:teuthology.orchestra.run.smithi143.stdout: "", 2023-11-26T07:23:10.411 INFO:teuthology.orchestra.run.smithi143.stdout: "critical", 2023-11-26T07:23:10.411 INFO:teuthology.orchestra.run.smithi143.stdout: "debug", 2023-11-26T07:23:10.411 INFO:teuthology.orchestra.run.smithi143.stdout: "error", 2023-11-26T07:23:10.411 INFO:teuthology.orchestra.run.smithi143.stdout: "info", 2023-11-26T07:23:10.411 INFO:teuthology.orchestra.run.smithi143.stdout: "warning" 2023-11-26T07:23:10.411 INFO:teuthology.orchestra.run.smithi143.stdout: ], 2023-11-26T07:23:10.411 INFO:teuthology.orchestra.run.smithi143.stdout: "desc": "", 2023-11-26T07:23:10.411 INFO:teuthology.orchestra.run.smithi143.stdout: "long_desc": "", 2023-11-26T07:23:10.412 INFO:teuthology.orchestra.run.smithi143.stdout: "tags": [], 2023-11-26T07:23:10.412 INFO:teuthology.orchestra.run.smithi143.stdout: "see_also": [] 2023-11-26T07:23:10.413 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2023-11-26T07:23:10.413 INFO:teuthology.orchestra.run.smithi143.stdout: "log_to_file": { 2023-11-26T07:23:10.413 INFO:teuthology.orchestra.run.smithi143.stdout: "name": "log_to_file", 2023-11-26T07:23:10.413 INFO:teuthology.orchestra.run.smithi143.stdout: "type": "bool", 2023-11-26T07:23:10.413 INFO:teuthology.orchestra.run.smithi143.stdout: "level": "advanced", 2023-11-26T07:23:10.413 INFO:teuthology.orchestra.run.smithi143.stdout: "flags": 1, 2023-11-26T07:23:10.414 INFO:teuthology.orchestra.run.smithi143.stdout: "default_value": "False", 2023-11-26T07:23:10.414 INFO:teuthology.orchestra.run.smithi143.stdout: "min": "", 2023-11-26T07:23:10.414 INFO:teuthology.orchestra.run.smithi143.stdout: "max": "", 2023-11-26T07:23:10.414 INFO:teuthology.orchestra.run.smithi143.stdout: "enum_allowed": [], 2023-11-26T07:23:10.414 INFO:teuthology.orchestra.run.smithi143.stdout: "desc": "", 2023-11-26T07:23:10.414 INFO:teuthology.orchestra.run.smithi143.stdout: "long_desc": "", 2023-11-26T07:23:10.414 INFO:teuthology.orchestra.run.smithi143.stdout: "tags": [], 2023-11-26T07:23:10.414 INFO:teuthology.orchestra.run.smithi143.stdout: "see_also": [] 2023-11-26T07:23:10.414 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2023-11-26T07:23:10.414 INFO:teuthology.orchestra.run.smithi143.stdout: "organization": { 2023-11-26T07:23:10.414 INFO:teuthology.orchestra.run.smithi143.stdout: "name": "organization", 2023-11-26T07:23:10.414 INFO:teuthology.orchestra.run.smithi143.stdout: "type": "str", 2023-11-26T07:23:10.414 INFO:teuthology.orchestra.run.smithi143.stdout: "level": "advanced", 2023-11-26T07:23:10.415 INFO:teuthology.orchestra.run.smithi143.stdout: "flags": 0, 2023-11-26T07:23:10.415 INFO:teuthology.orchestra.run.smithi143.stdout: "default_value": "", 2023-11-26T07:23:10.415 INFO:teuthology.orchestra.run.smithi143.stdout: "min": "", 2023-11-26T07:23:10.415 INFO:teuthology.orchestra.run.smithi143.stdout: "max": "", 2023-11-26T07:23:10.415 INFO:teuthology.orchestra.run.smithi143.stdout: "enum_allowed": [], 2023-11-26T07:23:10.415 INFO:teuthology.orchestra.run.smithi143.stdout: "desc": "", 2023-11-26T07:23:10.415 INFO:teuthology.orchestra.run.smithi143.stdout: "long_desc": "", 2023-11-26T07:23:10.415 INFO:teuthology.orchestra.run.smithi143.stdout: "tags": [], 2023-11-26T07:23:10.415 INFO:teuthology.orchestra.run.smithi143.stdout: "see_also": [] 2023-11-26T07:23:10.415 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2023-11-26T07:23:10.415 INFO:teuthology.orchestra.run.smithi143.stdout: "proxy": { 2023-11-26T07:23:10.415 INFO:teuthology.orchestra.run.smithi143.stdout: "name": "proxy", 2023-11-26T07:23:10.415 INFO:teuthology.orchestra.run.smithi143.stdout: "type": "str", 2023-11-26T07:23:10.416 INFO:teuthology.orchestra.run.smithi143.stdout: "level": "advanced", 2023-11-26T07:23:10.416 INFO:teuthology.orchestra.run.smithi143.stdout: "flags": 0, 2023-11-26T07:23:10.416 INFO:teuthology.orchestra.run.smithi143.stdout: "default_value": "", 2023-11-26T07:23:10.416 INFO:teuthology.orchestra.run.smithi143.stdout: "min": "", 2023-11-26T07:23:10.416 INFO:teuthology.orchestra.run.smithi143.stdout: "max": "", 2023-11-26T07:23:10.416 INFO:teuthology.orchestra.run.smithi143.stdout: "enum_allowed": [], 2023-11-26T07:23:10.416 INFO:teuthology.orchestra.run.smithi143.stdout: "desc": "", 2023-11-26T07:23:10.416 INFO:teuthology.orchestra.run.smithi143.stdout: "long_desc": "", 2023-11-26T07:23:10.416 INFO:teuthology.orchestra.run.smithi143.stdout: "tags": [], 2023-11-26T07:23:10.416 INFO:teuthology.orchestra.run.smithi143.stdout: "see_also": [] 2023-11-26T07:23:10.416 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2023-11-26T07:23:10.417 INFO:teuthology.orchestra.run.smithi143.stdout: "url": { 2023-11-26T07:23:10.417 INFO:teuthology.orchestra.run.smithi143.stdout: "name": "url", 2023-11-26T07:23:10.417 INFO:teuthology.orchestra.run.smithi143.stdout: "type": "str", 2023-11-26T07:23:10.417 INFO:teuthology.orchestra.run.smithi143.stdout: "level": "advanced", 2023-11-26T07:23:10.417 INFO:teuthology.orchestra.run.smithi143.stdout: "flags": 0, 2023-11-26T07:23:10.417 INFO:teuthology.orchestra.run.smithi143.stdout: "default_value": "https://telemetry.ceph.com/report", 2023-11-26T07:23:10.417 INFO:teuthology.orchestra.run.smithi143.stdout: "min": "", 2023-11-26T07:23:10.417 INFO:teuthology.orchestra.run.smithi143.stdout: "max": "", 2023-11-26T07:23:10.417 INFO:teuthology.orchestra.run.smithi143.stdout: "enum_allowed": [], 2023-11-26T07:23:10.417 INFO:teuthology.orchestra.run.smithi143.stdout: "desc": "", 2023-11-26T07:23:10.417 INFO:teuthology.orchestra.run.smithi143.stdout: "long_desc": "", 2023-11-26T07:23:10.417 INFO:teuthology.orchestra.run.smithi143.stdout: "tags": [], 2023-11-26T07:23:10.417 INFO:teuthology.orchestra.run.smithi143.stdout: "see_also": [] 2023-11-26T07:23:10.418 INFO:teuthology.orchestra.run.smithi143.stdout: } 2023-11-26T07:23:10.418 INFO:teuthology.orchestra.run.smithi143.stdout: } 2023-11-26T07:23:10.418 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2023-11-26T07:23:10.418 INFO:teuthology.orchestra.run.smithi143.stdout: { 2023-11-26T07:23:10.418 INFO:teuthology.orchestra.run.smithi143.stdout: "name": "test_orchestrator", 2023-11-26T07:23:10.418 INFO:teuthology.orchestra.run.smithi143.stdout: "can_run": true, 2023-11-26T07:23:10.418 INFO:teuthology.orchestra.run.smithi143.stdout: "error_string": "", 2023-11-26T07:23:10.418 INFO:teuthology.orchestra.run.smithi143.stdout: "module_options": { 2023-11-26T07:23:10.418 INFO:teuthology.orchestra.run.smithi143.stdout: "log_level": { 2023-11-26T07:23:10.418 INFO:teuthology.orchestra.run.smithi143.stdout: "name": "log_level", 2023-11-26T07:23:10.418 INFO:teuthology.orchestra.run.smithi143.stdout: "type": "str", 2023-11-26T07:23:10.418 INFO:teuthology.orchestra.run.smithi143.stdout: "level": "advanced", 2023-11-26T07:23:10.419 INFO:teuthology.orchestra.run.smithi143.stdout: "flags": 1, 2023-11-26T07:23:10.419 INFO:teuthology.orchestra.run.smithi143.stdout: "default_value": "", 2023-11-26T07:23:10.419 INFO:teuthology.orchestra.run.smithi143.stdout: "min": "", 2023-11-26T07:23:10.419 INFO:teuthology.orchestra.run.smithi143.stdout: "max": "", 2023-11-26T07:23:10.419 INFO:teuthology.orchestra.run.smithi143.stdout: "enum_allowed": [ 2023-11-26T07:23:10.419 INFO:teuthology.orchestra.run.smithi143.stdout: "", 2023-11-26T07:23:10.419 INFO:teuthology.orchestra.run.smithi143.stdout: "critical", 2023-11-26T07:23:10.419 INFO:teuthology.orchestra.run.smithi143.stdout: "debug", 2023-11-26T07:23:10.419 INFO:teuthology.orchestra.run.smithi143.stdout: "error", 2023-11-26T07:23:10.419 INFO:teuthology.orchestra.run.smithi143.stdout: "info", 2023-11-26T07:23:10.419 INFO:teuthology.orchestra.run.smithi143.stdout: "warning" 2023-11-26T07:23:10.419 INFO:teuthology.orchestra.run.smithi143.stdout: ], 2023-11-26T07:23:10.419 INFO:teuthology.orchestra.run.smithi143.stdout: "desc": "", 2023-11-26T07:23:10.420 INFO:teuthology.orchestra.run.smithi143.stdout: "long_desc": "", 2023-11-26T07:23:10.420 INFO:teuthology.orchestra.run.smithi143.stdout: "tags": [], 2023-11-26T07:23:10.420 INFO:teuthology.orchestra.run.smithi143.stdout: "see_also": [] 2023-11-26T07:23:10.420 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2023-11-26T07:23:10.420 INFO:teuthology.orchestra.run.smithi143.stdout: "log_to_cluster": { 2023-11-26T07:23:10.420 INFO:teuthology.orchestra.run.smithi143.stdout: "name": "log_to_cluster", 2023-11-26T07:23:10.420 INFO:teuthology.orchestra.run.smithi143.stdout: "type": "bool", 2023-11-26T07:23:10.420 INFO:teuthology.orchestra.run.smithi143.stdout: "level": "advanced", 2023-11-26T07:23:10.420 INFO:teuthology.orchestra.run.smithi143.stdout: "flags": 1, 2023-11-26T07:23:10.420 INFO:teuthology.orchestra.run.smithi143.stdout: "default_value": "False", 2023-11-26T07:23:10.420 INFO:teuthology.orchestra.run.smithi143.stdout: "min": "", 2023-11-26T07:23:10.420 INFO:teuthology.orchestra.run.smithi143.stdout: "max": "", 2023-11-26T07:23:10.420 INFO:teuthology.orchestra.run.smithi143.stdout: "enum_allowed": [], 2023-11-26T07:23:10.421 INFO:teuthology.orchestra.run.smithi143.stdout: "desc": "", 2023-11-26T07:23:10.421 INFO:teuthology.orchestra.run.smithi143.stdout: "long_desc": "", 2023-11-26T07:23:10.421 INFO:teuthology.orchestra.run.smithi143.stdout: "tags": [], 2023-11-26T07:23:10.421 INFO:teuthology.orchestra.run.smithi143.stdout: "see_also": [] 2023-11-26T07:23:10.421 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2023-11-26T07:23:10.421 INFO:teuthology.orchestra.run.smithi143.stdout: "log_to_cluster_level": { 2023-11-26T07:23:10.421 INFO:teuthology.orchestra.run.smithi143.stdout: "name": "log_to_cluster_level", 2023-11-26T07:23:10.421 INFO:teuthology.orchestra.run.smithi143.stdout: "type": "str", 2023-11-26T07:23:10.421 INFO:teuthology.orchestra.run.smithi143.stdout: "level": "advanced", 2023-11-26T07:23:10.421 INFO:teuthology.orchestra.run.smithi143.stdout: "flags": 1, 2023-11-26T07:23:10.421 INFO:teuthology.orchestra.run.smithi143.stdout: "default_value": "info", 2023-11-26T07:23:10.421 INFO:teuthology.orchestra.run.smithi143.stdout: "min": "", 2023-11-26T07:23:10.421 INFO:teuthology.orchestra.run.smithi143.stdout: "max": "", 2023-11-26T07:23:10.422 INFO:teuthology.orchestra.run.smithi143.stdout: "enum_allowed": [ 2023-11-26T07:23:10.422 INFO:teuthology.orchestra.run.smithi143.stdout: "", 2023-11-26T07:23:10.422 INFO:teuthology.orchestra.run.smithi143.stdout: "critical", 2023-11-26T07:23:10.422 INFO:teuthology.orchestra.run.smithi143.stdout: "debug", 2023-11-26T07:23:10.422 INFO:teuthology.orchestra.run.smithi143.stdout: "error", 2023-11-26T07:23:10.422 INFO:teuthology.orchestra.run.smithi143.stdout: "info", 2023-11-26T07:23:10.422 INFO:teuthology.orchestra.run.smithi143.stdout: "warning" 2023-11-26T07:23:10.422 INFO:teuthology.orchestra.run.smithi143.stdout: ], 2023-11-26T07:23:10.422 INFO:teuthology.orchestra.run.smithi143.stdout: "desc": "", 2023-11-26T07:23:10.422 INFO:teuthology.orchestra.run.smithi143.stdout: "long_desc": "", 2023-11-26T07:23:10.422 INFO:teuthology.orchestra.run.smithi143.stdout: "tags": [], 2023-11-26T07:23:10.422 INFO:teuthology.orchestra.run.smithi143.stdout: "see_also": [] 2023-11-26T07:23:10.423 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2023-11-26T07:23:10.423 INFO:teuthology.orchestra.run.smithi143.stdout: "log_to_file": { 2023-11-26T07:23:10.423 INFO:teuthology.orchestra.run.smithi143.stdout: "name": "log_to_file", 2023-11-26T07:23:10.423 INFO:teuthology.orchestra.run.smithi143.stdout: "type": "bool", 2023-11-26T07:23:10.423 INFO:teuthology.orchestra.run.smithi143.stdout: "level": "advanced", 2023-11-26T07:23:10.423 INFO:teuthology.orchestra.run.smithi143.stdout: "flags": 1, 2023-11-26T07:23:10.423 INFO:teuthology.orchestra.run.smithi143.stdout: "default_value": "False", 2023-11-26T07:23:10.423 INFO:teuthology.orchestra.run.smithi143.stdout: "min": "", 2023-11-26T07:23:10.423 INFO:teuthology.orchestra.run.smithi143.stdout: "max": "", 2023-11-26T07:23:10.423 INFO:teuthology.orchestra.run.smithi143.stdout: "enum_allowed": [], 2023-11-26T07:23:10.423 INFO:teuthology.orchestra.run.smithi143.stdout: "desc": "", 2023-11-26T07:23:10.423 INFO:teuthology.orchestra.run.smithi143.stdout: "long_desc": "", 2023-11-26T07:23:10.423 INFO:teuthology.orchestra.run.smithi143.stdout: "tags": [], 2023-11-26T07:23:10.423 INFO:teuthology.orchestra.run.smithi143.stdout: "see_also": [] 2023-11-26T07:23:10.424 INFO:teuthology.orchestra.run.smithi143.stdout: } 2023-11-26T07:23:10.424 INFO:teuthology.orchestra.run.smithi143.stdout: } 2023-11-26T07:23:10.424 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2023-11-26T07:23:10.424 INFO:teuthology.orchestra.run.smithi143.stdout: { 2023-11-26T07:23:10.424 INFO:teuthology.orchestra.run.smithi143.stdout: "name": "volumes", 2023-11-26T07:23:10.424 INFO:teuthology.orchestra.run.smithi143.stdout: "can_run": true, 2023-11-26T07:23:10.424 INFO:teuthology.orchestra.run.smithi143.stdout: "error_string": "", 2023-11-26T07:23:10.424 INFO:teuthology.orchestra.run.smithi143.stdout: "module_options": { 2023-11-26T07:23:10.424 INFO:teuthology.orchestra.run.smithi143.stdout: "log_level": { 2023-11-26T07:23:10.424 INFO:teuthology.orchestra.run.smithi143.stdout: "name": "log_level", 2023-11-26T07:23:10.424 INFO:teuthology.orchestra.run.smithi143.stdout: "type": "str", 2023-11-26T07:23:10.424 INFO:teuthology.orchestra.run.smithi143.stdout: "level": "advanced", 2023-11-26T07:23:10.424 INFO:teuthology.orchestra.run.smithi143.stdout: "flags": 1, 2023-11-26T07:23:10.425 INFO:teuthology.orchestra.run.smithi143.stdout: "default_value": "", 2023-11-26T07:23:10.425 INFO:teuthology.orchestra.run.smithi143.stdout: "min": "", 2023-11-26T07:23:10.425 INFO:teuthology.orchestra.run.smithi143.stdout: "max": "", 2023-11-26T07:23:10.425 INFO:teuthology.orchestra.run.smithi143.stdout: "enum_allowed": [ 2023-11-26T07:23:10.425 INFO:teuthology.orchestra.run.smithi143.stdout: "", 2023-11-26T07:23:10.425 INFO:teuthology.orchestra.run.smithi143.stdout: "critical", 2023-11-26T07:23:10.425 INFO:teuthology.orchestra.run.smithi143.stdout: "debug", 2023-11-26T07:23:10.425 INFO:teuthology.orchestra.run.smithi143.stdout: "error", 2023-11-26T07:23:10.425 INFO:teuthology.orchestra.run.smithi143.stdout: "info", 2023-11-26T07:23:10.425 INFO:teuthology.orchestra.run.smithi143.stdout: "warning" 2023-11-26T07:23:10.425 INFO:teuthology.orchestra.run.smithi143.stdout: ], 2023-11-26T07:23:10.425 INFO:teuthology.orchestra.run.smithi143.stdout: "desc": "", 2023-11-26T07:23:10.425 INFO:teuthology.orchestra.run.smithi143.stdout: "long_desc": "", 2023-11-26T07:23:10.426 INFO:teuthology.orchestra.run.smithi143.stdout: "tags": [], 2023-11-26T07:23:10.426 INFO:teuthology.orchestra.run.smithi143.stdout: "see_also": [] 2023-11-26T07:23:10.426 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2023-11-26T07:23:10.426 INFO:teuthology.orchestra.run.smithi143.stdout: "log_to_cluster": { 2023-11-26T07:23:10.426 INFO:teuthology.orchestra.run.smithi143.stdout: "name": "log_to_cluster", 2023-11-26T07:23:10.426 INFO:teuthology.orchestra.run.smithi143.stdout: "type": "bool", 2023-11-26T07:23:10.426 INFO:teuthology.orchestra.run.smithi143.stdout: "level": "advanced", 2023-11-26T07:23:10.426 INFO:teuthology.orchestra.run.smithi143.stdout: "flags": 1, 2023-11-26T07:23:10.426 INFO:teuthology.orchestra.run.smithi143.stdout: "default_value": "False", 2023-11-26T07:23:10.426 INFO:teuthology.orchestra.run.smithi143.stdout: "min": "", 2023-11-26T07:23:10.426 INFO:teuthology.orchestra.run.smithi143.stdout: "max": "", 2023-11-26T07:23:10.426 INFO:teuthology.orchestra.run.smithi143.stdout: "enum_allowed": [], 2023-11-26T07:23:10.427 INFO:teuthology.orchestra.run.smithi143.stdout: "desc": "", 2023-11-26T07:23:10.427 INFO:teuthology.orchestra.run.smithi143.stdout: "long_desc": "", 2023-11-26T07:23:10.427 INFO:teuthology.orchestra.run.smithi143.stdout: "tags": [], 2023-11-26T07:23:10.427 INFO:teuthology.orchestra.run.smithi143.stdout: "see_also": [] 2023-11-26T07:23:10.427 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2023-11-26T07:23:10.427 INFO:teuthology.orchestra.run.smithi143.stdout: "log_to_cluster_level": { 2023-11-26T07:23:10.427 INFO:teuthology.orchestra.run.smithi143.stdout: "name": "log_to_cluster_level", 2023-11-26T07:23:10.427 INFO:teuthology.orchestra.run.smithi143.stdout: "type": "str", 2023-11-26T07:23:10.427 INFO:teuthology.orchestra.run.smithi143.stdout: "level": "advanced", 2023-11-26T07:23:10.427 INFO:teuthology.orchestra.run.smithi143.stdout: "flags": 1, 2023-11-26T07:23:10.427 INFO:teuthology.orchestra.run.smithi143.stdout: "default_value": "info", 2023-11-26T07:23:10.427 INFO:teuthology.orchestra.run.smithi143.stdout: "min": "", 2023-11-26T07:23:10.427 INFO:teuthology.orchestra.run.smithi143.stdout: "max": "", 2023-11-26T07:23:10.428 INFO:teuthology.orchestra.run.smithi143.stdout: "enum_allowed": [ 2023-11-26T07:23:10.428 INFO:teuthology.orchestra.run.smithi143.stdout: "", 2023-11-26T07:23:10.428 INFO:teuthology.orchestra.run.smithi143.stdout: "critical", 2023-11-26T07:23:10.428 INFO:teuthology.orchestra.run.smithi143.stdout: "debug", 2023-11-26T07:23:10.428 INFO:teuthology.orchestra.run.smithi143.stdout: "error", 2023-11-26T07:23:10.428 INFO:teuthology.orchestra.run.smithi143.stdout: "info", 2023-11-26T07:23:10.428 INFO:teuthology.orchestra.run.smithi143.stdout: "warning" 2023-11-26T07:23:10.428 INFO:teuthology.orchestra.run.smithi143.stdout: ], 2023-11-26T07:23:10.428 INFO:teuthology.orchestra.run.smithi143.stdout: "desc": "", 2023-11-26T07:23:10.428 INFO:teuthology.orchestra.run.smithi143.stdout: "long_desc": "", 2023-11-26T07:23:10.428 INFO:teuthology.orchestra.run.smithi143.stdout: "tags": [], 2023-11-26T07:23:10.428 INFO:teuthology.orchestra.run.smithi143.stdout: "see_also": [] 2023-11-26T07:23:10.429 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2023-11-26T07:23:10.429 INFO:teuthology.orchestra.run.smithi143.stdout: "log_to_file": { 2023-11-26T07:23:10.429 INFO:teuthology.orchestra.run.smithi143.stdout: "name": "log_to_file", 2023-11-26T07:23:10.429 INFO:teuthology.orchestra.run.smithi143.stdout: "type": "bool", 2023-11-26T07:23:10.429 INFO:teuthology.orchestra.run.smithi143.stdout: "level": "advanced", 2023-11-26T07:23:10.429 INFO:teuthology.orchestra.run.smithi143.stdout: "flags": 1, 2023-11-26T07:23:10.429 INFO:teuthology.orchestra.run.smithi143.stdout: "default_value": "False", 2023-11-26T07:23:10.429 INFO:teuthology.orchestra.run.smithi143.stdout: "min": "", 2023-11-26T07:23:10.429 INFO:teuthology.orchestra.run.smithi143.stdout: "max": "", 2023-11-26T07:23:10.429 INFO:teuthology.orchestra.run.smithi143.stdout: "enum_allowed": [], 2023-11-26T07:23:10.429 INFO:teuthology.orchestra.run.smithi143.stdout: "desc": "", 2023-11-26T07:23:10.430 INFO:teuthology.orchestra.run.smithi143.stdout: "long_desc": "", 2023-11-26T07:23:10.430 INFO:teuthology.orchestra.run.smithi143.stdout: "tags": [], 2023-11-26T07:23:10.430 INFO:teuthology.orchestra.run.smithi143.stdout: "see_also": [] 2023-11-26T07:23:10.430 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2023-11-26T07:23:10.430 INFO:teuthology.orchestra.run.smithi143.stdout: "max_concurrent_clones": { 2023-11-26T07:23:10.430 INFO:teuthology.orchestra.run.smithi143.stdout: "name": "max_concurrent_clones", 2023-11-26T07:23:10.430 INFO:teuthology.orchestra.run.smithi143.stdout: "type": "int", 2023-11-26T07:23:10.430 INFO:teuthology.orchestra.run.smithi143.stdout: "level": "advanced", 2023-11-26T07:23:10.430 INFO:teuthology.orchestra.run.smithi143.stdout: "flags": 0, 2023-11-26T07:23:10.430 INFO:teuthology.orchestra.run.smithi143.stdout: "default_value": "4", 2023-11-26T07:23:10.430 INFO:teuthology.orchestra.run.smithi143.stdout: "min": "", 2023-11-26T07:23:10.430 INFO:teuthology.orchestra.run.smithi143.stdout: "max": "", 2023-11-26T07:23:10.431 INFO:teuthology.orchestra.run.smithi143.stdout: "enum_allowed": [], 2023-11-26T07:23:10.431 INFO:teuthology.orchestra.run.smithi143.stdout: "desc": "Number of asynchronous cloner threads", 2023-11-26T07:23:10.431 INFO:teuthology.orchestra.run.smithi143.stdout: "long_desc": "", 2023-11-26T07:23:10.431 INFO:teuthology.orchestra.run.smithi143.stdout: "tags": [], 2023-11-26T07:23:10.431 INFO:teuthology.orchestra.run.smithi143.stdout: "see_also": [] 2023-11-26T07:23:10.431 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2023-11-26T07:23:10.431 INFO:teuthology.orchestra.run.smithi143.stdout: "periodic_async_work": { 2023-11-26T07:23:10.431 INFO:teuthology.orchestra.run.smithi143.stdout: "name": "periodic_async_work", 2023-11-26T07:23:10.431 INFO:teuthology.orchestra.run.smithi143.stdout: "type": "bool", 2023-11-26T07:23:10.431 INFO:teuthology.orchestra.run.smithi143.stdout: "level": "advanced", 2023-11-26T07:23:10.431 INFO:teuthology.orchestra.run.smithi143.stdout: "flags": 0, 2023-11-26T07:23:10.431 INFO:teuthology.orchestra.run.smithi143.stdout: "default_value": "False", 2023-11-26T07:23:10.432 INFO:teuthology.orchestra.run.smithi143.stdout: "min": "", 2023-11-26T07:23:10.432 INFO:teuthology.orchestra.run.smithi143.stdout: "max": "", 2023-11-26T07:23:10.432 INFO:teuthology.orchestra.run.smithi143.stdout: "enum_allowed": [], 2023-11-26T07:23:10.432 INFO:teuthology.orchestra.run.smithi143.stdout: "desc": "Periodically check for async work", 2023-11-26T07:23:10.432 INFO:teuthology.orchestra.run.smithi143.stdout: "long_desc": "", 2023-11-26T07:23:10.432 INFO:teuthology.orchestra.run.smithi143.stdout: "tags": [], 2023-11-26T07:23:10.432 INFO:teuthology.orchestra.run.smithi143.stdout: "see_also": [] 2023-11-26T07:23:10.432 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2023-11-26T07:23:10.432 INFO:teuthology.orchestra.run.smithi143.stdout: "snapshot_clone_delay": { 2023-11-26T07:23:10.432 INFO:teuthology.orchestra.run.smithi143.stdout: "name": "snapshot_clone_delay", 2023-11-26T07:23:10.432 INFO:teuthology.orchestra.run.smithi143.stdout: "type": "int", 2023-11-26T07:23:10.432 INFO:teuthology.orchestra.run.smithi143.stdout: "level": "advanced", 2023-11-26T07:23:10.432 INFO:teuthology.orchestra.run.smithi143.stdout: "flags": 0, 2023-11-26T07:23:10.433 INFO:teuthology.orchestra.run.smithi143.stdout: "default_value": "0", 2023-11-26T07:23:10.433 INFO:teuthology.orchestra.run.smithi143.stdout: "min": "", 2023-11-26T07:23:10.433 INFO:teuthology.orchestra.run.smithi143.stdout: "max": "", 2023-11-26T07:23:10.433 INFO:teuthology.orchestra.run.smithi143.stdout: "enum_allowed": [], 2023-11-26T07:23:10.433 INFO:teuthology.orchestra.run.smithi143.stdout: "desc": "Delay clone begin operation by snapshot_clone_delay seconds", 2023-11-26T07:23:10.433 INFO:teuthology.orchestra.run.smithi143.stdout: "long_desc": "", 2023-11-26T07:23:10.433 INFO:teuthology.orchestra.run.smithi143.stdout: "tags": [], 2023-11-26T07:23:10.433 INFO:teuthology.orchestra.run.smithi143.stdout: "see_also": [] 2023-11-26T07:23:10.433 INFO:teuthology.orchestra.run.smithi143.stdout: } 2023-11-26T07:23:10.433 INFO:teuthology.orchestra.run.smithi143.stdout: } 2023-11-26T07:23:10.433 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2023-11-26T07:23:10.433 INFO:teuthology.orchestra.run.smithi143.stdout: { 2023-11-26T07:23:10.433 INFO:teuthology.orchestra.run.smithi143.stdout: "name": "zabbix", 2023-11-26T07:23:10.434 INFO:teuthology.orchestra.run.smithi143.stdout: "can_run": true, 2023-11-26T07:23:10.434 INFO:teuthology.orchestra.run.smithi143.stdout: "error_string": "", 2023-11-26T07:23:10.434 INFO:teuthology.orchestra.run.smithi143.stdout: "module_options": { 2023-11-26T07:23:10.434 INFO:teuthology.orchestra.run.smithi143.stdout: "discovery_interval": { 2023-11-26T07:23:10.434 INFO:teuthology.orchestra.run.smithi143.stdout: "name": "discovery_interval", 2023-11-26T07:23:10.434 INFO:teuthology.orchestra.run.smithi143.stdout: "type": "uint", 2023-11-26T07:23:10.434 INFO:teuthology.orchestra.run.smithi143.stdout: "level": "advanced", 2023-11-26T07:23:10.434 INFO:teuthology.orchestra.run.smithi143.stdout: "flags": 0, 2023-11-26T07:23:10.434 INFO:teuthology.orchestra.run.smithi143.stdout: "default_value": "100", 2023-11-26T07:23:10.434 INFO:teuthology.orchestra.run.smithi143.stdout: "min": "", 2023-11-26T07:23:10.434 INFO:teuthology.orchestra.run.smithi143.stdout: "max": "", 2023-11-26T07:23:10.434 INFO:teuthology.orchestra.run.smithi143.stdout: "enum_allowed": [], 2023-11-26T07:23:10.435 INFO:teuthology.orchestra.run.smithi143.stdout: "desc": "", 2023-11-26T07:23:10.435 INFO:teuthology.orchestra.run.smithi143.stdout: "long_desc": "", 2023-11-26T07:23:10.435 INFO:teuthology.orchestra.run.smithi143.stdout: "tags": [], 2023-11-26T07:23:10.435 INFO:teuthology.orchestra.run.smithi143.stdout: "see_also": [] 2023-11-26T07:23:10.435 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2023-11-26T07:23:10.435 INFO:teuthology.orchestra.run.smithi143.stdout: "identifier": { 2023-11-26T07:23:10.435 INFO:teuthology.orchestra.run.smithi143.stdout: "name": "identifier", 2023-11-26T07:23:10.435 INFO:teuthology.orchestra.run.smithi143.stdout: "type": "str", 2023-11-26T07:23:10.435 INFO:teuthology.orchestra.run.smithi143.stdout: "level": "advanced", 2023-11-26T07:23:10.435 INFO:teuthology.orchestra.run.smithi143.stdout: "flags": 0, 2023-11-26T07:23:10.435 INFO:teuthology.orchestra.run.smithi143.stdout: "default_value": "", 2023-11-26T07:23:10.435 INFO:teuthology.orchestra.run.smithi143.stdout: "min": "", 2023-11-26T07:23:10.435 INFO:teuthology.orchestra.run.smithi143.stdout: "max": "", 2023-11-26T07:23:10.436 INFO:teuthology.orchestra.run.smithi143.stdout: "enum_allowed": [], 2023-11-26T07:23:10.436 INFO:teuthology.orchestra.run.smithi143.stdout: "desc": "", 2023-11-26T07:23:10.436 INFO:teuthology.orchestra.run.smithi143.stdout: "long_desc": "", 2023-11-26T07:23:10.436 INFO:teuthology.orchestra.run.smithi143.stdout: "tags": [], 2023-11-26T07:23:10.436 INFO:teuthology.orchestra.run.smithi143.stdout: "see_also": [] 2023-11-26T07:23:10.436 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2023-11-26T07:23:10.436 INFO:teuthology.orchestra.run.smithi143.stdout: "interval": { 2023-11-26T07:23:10.436 INFO:teuthology.orchestra.run.smithi143.stdout: "name": "interval", 2023-11-26T07:23:10.436 INFO:teuthology.orchestra.run.smithi143.stdout: "type": "secs", 2023-11-26T07:23:10.436 INFO:teuthology.orchestra.run.smithi143.stdout: "level": "advanced", 2023-11-26T07:23:10.436 INFO:teuthology.orchestra.run.smithi143.stdout: "flags": 0, 2023-11-26T07:23:10.437 INFO:teuthology.orchestra.run.smithi143.stdout: "default_value": "60", 2023-11-26T07:23:10.437 INFO:teuthology.orchestra.run.smithi143.stdout: "min": "", 2023-11-26T07:23:10.437 INFO:teuthology.orchestra.run.smithi143.stdout: "max": "", 2023-11-26T07:23:10.437 INFO:teuthology.orchestra.run.smithi143.stdout: "enum_allowed": [], 2023-11-26T07:23:10.437 INFO:teuthology.orchestra.run.smithi143.stdout: "desc": "", 2023-11-26T07:23:10.437 INFO:teuthology.orchestra.run.smithi143.stdout: "long_desc": "", 2023-11-26T07:23:10.437 INFO:teuthology.orchestra.run.smithi143.stdout: "tags": [], 2023-11-26T07:23:10.437 INFO:teuthology.orchestra.run.smithi143.stdout: "see_also": [] 2023-11-26T07:23:10.437 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2023-11-26T07:23:10.437 INFO:teuthology.orchestra.run.smithi143.stdout: "log_level": { 2023-11-26T07:23:10.437 INFO:teuthology.orchestra.run.smithi143.stdout: "name": "log_level", 2023-11-26T07:23:10.437 INFO:teuthology.orchestra.run.smithi143.stdout: "type": "str", 2023-11-26T07:23:10.437 INFO:teuthology.orchestra.run.smithi143.stdout: "level": "advanced", 2023-11-26T07:23:10.438 INFO:teuthology.orchestra.run.smithi143.stdout: "flags": 1, 2023-11-26T07:23:10.438 INFO:teuthology.orchestra.run.smithi143.stdout: "default_value": "", 2023-11-26T07:23:10.438 INFO:teuthology.orchestra.run.smithi143.stdout: "min": "", 2023-11-26T07:23:10.438 INFO:teuthology.orchestra.run.smithi143.stdout: "max": "", 2023-11-26T07:23:10.438 INFO:teuthology.orchestra.run.smithi143.stdout: "enum_allowed": [ 2023-11-26T07:23:10.438 INFO:teuthology.orchestra.run.smithi143.stdout: "", 2023-11-26T07:23:10.438 INFO:teuthology.orchestra.run.smithi143.stdout: "critical", 2023-11-26T07:23:10.438 INFO:teuthology.orchestra.run.smithi143.stdout: "debug", 2023-11-26T07:23:10.438 INFO:teuthology.orchestra.run.smithi143.stdout: "error", 2023-11-26T07:23:10.438 INFO:teuthology.orchestra.run.smithi143.stdout: "info", 2023-11-26T07:23:10.438 INFO:teuthology.orchestra.run.smithi143.stdout: "warning" 2023-11-26T07:23:10.438 INFO:teuthology.orchestra.run.smithi143.stdout: ], 2023-11-26T07:23:10.439 INFO:teuthology.orchestra.run.smithi143.stdout: "desc": "", 2023-11-26T07:23:10.439 INFO:teuthology.orchestra.run.smithi143.stdout: "long_desc": "", 2023-11-26T07:23:10.439 INFO:teuthology.orchestra.run.smithi143.stdout: "tags": [], 2023-11-26T07:23:10.439 INFO:teuthology.orchestra.run.smithi143.stdout: "see_also": [] 2023-11-26T07:23:10.439 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2023-11-26T07:23:10.439 INFO:teuthology.orchestra.run.smithi143.stdout: "log_to_cluster": { 2023-11-26T07:23:10.439 INFO:teuthology.orchestra.run.smithi143.stdout: "name": "log_to_cluster", 2023-11-26T07:23:10.439 INFO:teuthology.orchestra.run.smithi143.stdout: "type": "bool", 2023-11-26T07:23:10.439 INFO:teuthology.orchestra.run.smithi143.stdout: "level": "advanced", 2023-11-26T07:23:10.439 INFO:teuthology.orchestra.run.smithi143.stdout: "flags": 1, 2023-11-26T07:23:10.439 INFO:teuthology.orchestra.run.smithi143.stdout: "default_value": "False", 2023-11-26T07:23:10.439 INFO:teuthology.orchestra.run.smithi143.stdout: "min": "", 2023-11-26T07:23:10.440 INFO:teuthology.orchestra.run.smithi143.stdout: "max": "", 2023-11-26T07:23:10.440 INFO:teuthology.orchestra.run.smithi143.stdout: "enum_allowed": [], 2023-11-26T07:23:10.440 INFO:teuthology.orchestra.run.smithi143.stdout: "desc": "", 2023-11-26T07:23:10.440 INFO:teuthology.orchestra.run.smithi143.stdout: "long_desc": "", 2023-11-26T07:23:10.440 INFO:teuthology.orchestra.run.smithi143.stdout: "tags": [], 2023-11-26T07:23:10.440 INFO:teuthology.orchestra.run.smithi143.stdout: "see_also": [] 2023-11-26T07:23:10.440 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2023-11-26T07:23:10.440 INFO:teuthology.orchestra.run.smithi143.stdout: "log_to_cluster_level": { 2023-11-26T07:23:10.440 INFO:teuthology.orchestra.run.smithi143.stdout: "name": "log_to_cluster_level", 2023-11-26T07:23:10.440 INFO:teuthology.orchestra.run.smithi143.stdout: "type": "str", 2023-11-26T07:23:10.440 INFO:teuthology.orchestra.run.smithi143.stdout: "level": "advanced", 2023-11-26T07:23:10.441 INFO:teuthology.orchestra.run.smithi143.stdout: "flags": 1, 2023-11-26T07:23:10.441 INFO:teuthology.orchestra.run.smithi143.stdout: "default_value": "info", 2023-11-26T07:23:10.441 INFO:teuthology.orchestra.run.smithi143.stdout: "min": "", 2023-11-26T07:23:10.441 INFO:teuthology.orchestra.run.smithi143.stdout: "max": "", 2023-11-26T07:23:10.441 INFO:teuthology.orchestra.run.smithi143.stdout: "enum_allowed": [ 2023-11-26T07:23:10.441 INFO:teuthology.orchestra.run.smithi143.stdout: "", 2023-11-26T07:23:10.441 INFO:teuthology.orchestra.run.smithi143.stdout: "critical", 2023-11-26T07:23:10.441 INFO:teuthology.orchestra.run.smithi143.stdout: "debug", 2023-11-26T07:23:10.441 INFO:teuthology.orchestra.run.smithi143.stdout: "error", 2023-11-26T07:23:10.441 INFO:teuthology.orchestra.run.smithi143.stdout: "info", 2023-11-26T07:23:10.441 INFO:teuthology.orchestra.run.smithi143.stdout: "warning" 2023-11-26T07:23:10.441 INFO:teuthology.orchestra.run.smithi143.stdout: ], 2023-11-26T07:23:10.442 INFO:teuthology.orchestra.run.smithi143.stdout: "desc": "", 2023-11-26T07:23:10.442 INFO:teuthology.orchestra.run.smithi143.stdout: "long_desc": "", 2023-11-26T07:23:10.442 INFO:teuthology.orchestra.run.smithi143.stdout: "tags": [], 2023-11-26T07:23:10.442 INFO:teuthology.orchestra.run.smithi143.stdout: "see_also": [] 2023-11-26T07:23:10.442 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2023-11-26T07:23:10.442 INFO:teuthology.orchestra.run.smithi143.stdout: "log_to_file": { 2023-11-26T07:23:10.442 INFO:teuthology.orchestra.run.smithi143.stdout: "name": "log_to_file", 2023-11-26T07:23:10.442 INFO:teuthology.orchestra.run.smithi143.stdout: "type": "bool", 2023-11-26T07:23:10.442 INFO:teuthology.orchestra.run.smithi143.stdout: "level": "advanced", 2023-11-26T07:23:10.442 INFO:teuthology.orchestra.run.smithi143.stdout: "flags": 1, 2023-11-26T07:23:10.442 INFO:teuthology.orchestra.run.smithi143.stdout: "default_value": "False", 2023-11-26T07:23:10.442 INFO:teuthology.orchestra.run.smithi143.stdout: "min": "", 2023-11-26T07:23:10.443 INFO:teuthology.orchestra.run.smithi143.stdout: "max": "", 2023-11-26T07:23:10.443 INFO:teuthology.orchestra.run.smithi143.stdout: "enum_allowed": [], 2023-11-26T07:23:10.443 INFO:teuthology.orchestra.run.smithi143.stdout: "desc": "", 2023-11-26T07:23:10.443 INFO:teuthology.orchestra.run.smithi143.stdout: "long_desc": "", 2023-11-26T07:23:10.443 INFO:teuthology.orchestra.run.smithi143.stdout: "tags": [], 2023-11-26T07:23:10.443 INFO:teuthology.orchestra.run.smithi143.stdout: "see_also": [] 2023-11-26T07:23:10.443 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2023-11-26T07:23:10.443 INFO:teuthology.orchestra.run.smithi143.stdout: "zabbix_host": { 2023-11-26T07:23:10.443 INFO:teuthology.orchestra.run.smithi143.stdout: "name": "zabbix_host", 2023-11-26T07:23:10.443 INFO:teuthology.orchestra.run.smithi143.stdout: "type": "str", 2023-11-26T07:23:10.443 INFO:teuthology.orchestra.run.smithi143.stdout: "level": "advanced", 2023-11-26T07:23:10.443 INFO:teuthology.orchestra.run.smithi143.stdout: "flags": 0, 2023-11-26T07:23:10.444 INFO:teuthology.orchestra.run.smithi143.stdout: "default_value": "", 2023-11-26T07:23:10.444 INFO:teuthology.orchestra.run.smithi143.stdout: "min": "", 2023-11-26T07:23:10.444 INFO:teuthology.orchestra.run.smithi143.stdout: "max": "", 2023-11-26T07:23:10.444 INFO:teuthology.orchestra.run.smithi143.stdout: "enum_allowed": [], 2023-11-26T07:23:10.444 INFO:teuthology.orchestra.run.smithi143.stdout: "desc": "", 2023-11-26T07:23:10.444 INFO:teuthology.orchestra.run.smithi143.stdout: "long_desc": "", 2023-11-26T07:23:10.444 INFO:teuthology.orchestra.run.smithi143.stdout: "tags": [], 2023-11-26T07:23:10.444 INFO:teuthology.orchestra.run.smithi143.stdout: "see_also": [] 2023-11-26T07:23:10.444 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2023-11-26T07:23:10.444 INFO:teuthology.orchestra.run.smithi143.stdout: "zabbix_port": { 2023-11-26T07:23:10.444 INFO:teuthology.orchestra.run.smithi143.stdout: "name": "zabbix_port", 2023-11-26T07:23:10.444 INFO:teuthology.orchestra.run.smithi143.stdout: "type": "int", 2023-11-26T07:23:10.444 INFO:teuthology.orchestra.run.smithi143.stdout: "level": "advanced", 2023-11-26T07:23:10.445 INFO:teuthology.orchestra.run.smithi143.stdout: "flags": 0, 2023-11-26T07:23:10.445 INFO:teuthology.orchestra.run.smithi143.stdout: "default_value": "10051", 2023-11-26T07:23:10.445 INFO:teuthology.orchestra.run.smithi143.stdout: "min": "", 2023-11-26T07:23:10.445 INFO:teuthology.orchestra.run.smithi143.stdout: "max": "", 2023-11-26T07:23:10.445 INFO:teuthology.orchestra.run.smithi143.stdout: "enum_allowed": [], 2023-11-26T07:23:10.445 INFO:teuthology.orchestra.run.smithi143.stdout: "desc": "", 2023-11-26T07:23:10.445 INFO:teuthology.orchestra.run.smithi143.stdout: "long_desc": "", 2023-11-26T07:23:10.445 INFO:teuthology.orchestra.run.smithi143.stdout: "tags": [], 2023-11-26T07:23:10.445 INFO:teuthology.orchestra.run.smithi143.stdout: "see_also": [] 2023-11-26T07:23:10.445 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2023-11-26T07:23:10.445 INFO:teuthology.orchestra.run.smithi143.stdout: "zabbix_sender": { 2023-11-26T07:23:10.445 INFO:teuthology.orchestra.run.smithi143.stdout: "name": "zabbix_sender", 2023-11-26T07:23:10.445 INFO:teuthology.orchestra.run.smithi143.stdout: "type": "str", 2023-11-26T07:23:10.446 INFO:teuthology.orchestra.run.smithi143.stdout: "level": "advanced", 2023-11-26T07:23:10.446 INFO:teuthology.orchestra.run.smithi143.stdout: "flags": 0, 2023-11-26T07:23:10.446 INFO:teuthology.orchestra.run.smithi143.stdout: "default_value": "/usr/bin/zabbix_sender", 2023-11-26T07:23:10.446 INFO:teuthology.orchestra.run.smithi143.stdout: "min": "", 2023-11-26T07:23:10.446 INFO:teuthology.orchestra.run.smithi143.stdout: "max": "", 2023-11-26T07:23:10.446 INFO:teuthology.orchestra.run.smithi143.stdout: "enum_allowed": [], 2023-11-26T07:23:10.446 INFO:teuthology.orchestra.run.smithi143.stdout: "desc": "", 2023-11-26T07:23:10.446 INFO:teuthology.orchestra.run.smithi143.stdout: "long_desc": "", 2023-11-26T07:23:10.446 INFO:teuthology.orchestra.run.smithi143.stdout: "tags": [], 2023-11-26T07:23:10.446 INFO:teuthology.orchestra.run.smithi143.stdout: "see_also": [] 2023-11-26T07:23:10.446 INFO:teuthology.orchestra.run.smithi143.stdout: } 2023-11-26T07:23:10.447 INFO:teuthology.orchestra.run.smithi143.stdout: } 2023-11-26T07:23:10.447 INFO:teuthology.orchestra.run.smithi143.stdout: } 2023-11-26T07:23:10.447 INFO:teuthology.orchestra.run.smithi143.stdout: ] 2023-11-26T07:23:10.447 INFO:teuthology.orchestra.run.smithi143.stdout: } 2023-11-26T07:23:10.447 INFO:teuthology.orchestra.run.smithi143.stdout: ], 2023-11-26T07:23:10.447 INFO:teuthology.orchestra.run.smithi143.stdout: "modules": [ 2023-11-26T07:23:10.447 INFO:teuthology.orchestra.run.smithi143.stdout: "iostat", 2023-11-26T07:23:10.447 INFO:teuthology.orchestra.run.smithi143.stdout: "nfs", 2023-11-26T07:23:10.447 INFO:teuthology.orchestra.run.smithi143.stdout: "restful" 2023-11-26T07:23:10.447 INFO:teuthology.orchestra.run.smithi143.stdout: ], 2023-11-26T07:23:10.447 INFO:teuthology.orchestra.run.smithi143.stdout: "available_modules": [ 2023-11-26T07:23:10.448 INFO:teuthology.orchestra.run.smithi143.stdout: { 2023-11-26T07:23:10.448 INFO:teuthology.orchestra.run.smithi143.stdout: "name": "alerts", 2023-11-26T07:23:10.448 INFO:teuthology.orchestra.run.smithi143.stdout: "can_run": true, 2023-11-26T07:23:10.448 INFO:teuthology.orchestra.run.smithi143.stdout: "error_string": "", 2023-11-26T07:23:10.448 INFO:teuthology.orchestra.run.smithi143.stdout: "module_options": { 2023-11-26T07:23:10.448 INFO:teuthology.orchestra.run.smithi143.stdout: "interval": { 2023-11-26T07:23:10.448 INFO:teuthology.orchestra.run.smithi143.stdout: "name": "interval", 2023-11-26T07:23:10.448 INFO:teuthology.orchestra.run.smithi143.stdout: "type": "secs", 2023-11-26T07:23:10.448 INFO:teuthology.orchestra.run.smithi143.stdout: "level": "advanced", 2023-11-26T07:23:10.448 INFO:teuthology.orchestra.run.smithi143.stdout: "flags": 1, 2023-11-26T07:23:10.448 INFO:teuthology.orchestra.run.smithi143.stdout: "default_value": "60", 2023-11-26T07:23:10.448 INFO:teuthology.orchestra.run.smithi143.stdout: "min": "", 2023-11-26T07:23:10.449 INFO:teuthology.orchestra.run.smithi143.stdout: "max": "", 2023-11-26T07:23:10.449 INFO:teuthology.orchestra.run.smithi143.stdout: "enum_allowed": [], 2023-11-26T07:23:10.449 INFO:teuthology.orchestra.run.smithi143.stdout: "desc": "How frequently to reexamine health status", 2023-11-26T07:23:10.449 INFO:teuthology.orchestra.run.smithi143.stdout: "long_desc": "", 2023-11-26T07:23:10.449 INFO:teuthology.orchestra.run.smithi143.stdout: "tags": [], 2023-11-26T07:23:10.449 INFO:teuthology.orchestra.run.smithi143.stdout: "see_also": [] 2023-11-26T07:23:10.449 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2023-11-26T07:23:10.449 INFO:teuthology.orchestra.run.smithi143.stdout: "log_level": { 2023-11-26T07:23:10.449 INFO:teuthology.orchestra.run.smithi143.stdout: "name": "log_level", 2023-11-26T07:23:10.449 INFO:teuthology.orchestra.run.smithi143.stdout: "type": "str", 2023-11-26T07:23:10.449 INFO:teuthology.orchestra.run.smithi143.stdout: "level": "advanced", 2023-11-26T07:23:10.449 INFO:teuthology.orchestra.run.smithi143.stdout: "flags": 1, 2023-11-26T07:23:10.450 INFO:teuthology.orchestra.run.smithi143.stdout: "default_value": "", 2023-11-26T07:23:10.450 INFO:teuthology.orchestra.run.smithi143.stdout: "min": "", 2023-11-26T07:23:10.450 INFO:teuthology.orchestra.run.smithi143.stdout: "max": "", 2023-11-26T07:23:10.450 INFO:teuthology.orchestra.run.smithi143.stdout: "enum_allowed": [ 2023-11-26T07:23:10.450 INFO:teuthology.orchestra.run.smithi143.stdout: "", 2023-11-26T07:23:10.450 INFO:teuthology.orchestra.run.smithi143.stdout: "critical", 2023-11-26T07:23:10.450 INFO:teuthology.orchestra.run.smithi143.stdout: "debug", 2023-11-26T07:23:10.450 INFO:teuthology.orchestra.run.smithi143.stdout: "error", 2023-11-26T07:23:10.450 INFO:teuthology.orchestra.run.smithi143.stdout: "info", 2023-11-26T07:23:10.450 INFO:teuthology.orchestra.run.smithi143.stdout: "warning" 2023-11-26T07:23:10.450 INFO:teuthology.orchestra.run.smithi143.stdout: ], 2023-11-26T07:23:10.450 INFO:teuthology.orchestra.run.smithi143.stdout: "desc": "", 2023-11-26T07:23:10.451 INFO:teuthology.orchestra.run.smithi143.stdout: "long_desc": "", 2023-11-26T07:23:10.451 INFO:teuthology.orchestra.run.smithi143.stdout: "tags": [], 2023-11-26T07:23:10.451 INFO:teuthology.orchestra.run.smithi143.stdout: "see_also": [] 2023-11-26T07:23:10.451 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2023-11-26T07:23:10.451 INFO:teuthology.orchestra.run.smithi143.stdout: "log_to_cluster": { 2023-11-26T07:23:10.451 INFO:teuthology.orchestra.run.smithi143.stdout: "name": "log_to_cluster", 2023-11-26T07:23:10.451 INFO:teuthology.orchestra.run.smithi143.stdout: "type": "bool", 2023-11-26T07:23:10.451 INFO:teuthology.orchestra.run.smithi143.stdout: "level": "advanced", 2023-11-26T07:23:10.451 INFO:teuthology.orchestra.run.smithi143.stdout: "flags": 1, 2023-11-26T07:23:10.451 INFO:teuthology.orchestra.run.smithi143.stdout: "default_value": "False", 2023-11-26T07:23:10.451 INFO:teuthology.orchestra.run.smithi143.stdout: "min": "", 2023-11-26T07:23:10.451 INFO:teuthology.orchestra.run.smithi143.stdout: "max": "", 2023-11-26T07:23:10.451 INFO:teuthology.orchestra.run.smithi143.stdout: "enum_allowed": [], 2023-11-26T07:23:10.452 INFO:teuthology.orchestra.run.smithi143.stdout: "desc": "", 2023-11-26T07:23:10.452 INFO:teuthology.orchestra.run.smithi143.stdout: "long_desc": "", 2023-11-26T07:23:10.452 INFO:teuthology.orchestra.run.smithi143.stdout: "tags": [], 2023-11-26T07:23:10.452 INFO:teuthology.orchestra.run.smithi143.stdout: "see_also": [] 2023-11-26T07:23:10.452 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2023-11-26T07:23:10.452 INFO:teuthology.orchestra.run.smithi143.stdout: "log_to_cluster_level": { 2023-11-26T07:23:10.452 INFO:teuthology.orchestra.run.smithi143.stdout: "name": "log_to_cluster_level", 2023-11-26T07:23:10.452 INFO:teuthology.orchestra.run.smithi143.stdout: "type": "str", 2023-11-26T07:23:10.452 INFO:teuthology.orchestra.run.smithi143.stdout: "level": "advanced", 2023-11-26T07:23:10.452 INFO:teuthology.orchestra.run.smithi143.stdout: "flags": 1, 2023-11-26T07:23:10.452 INFO:teuthology.orchestra.run.smithi143.stdout: "default_value": "info", 2023-11-26T07:23:10.452 INFO:teuthology.orchestra.run.smithi143.stdout: "min": "", 2023-11-26T07:23:10.452 INFO:teuthology.orchestra.run.smithi143.stdout: "max": "", 2023-11-26T07:23:10.453 INFO:teuthology.orchestra.run.smithi143.stdout: "enum_allowed": [ 2023-11-26T07:23:10.453 INFO:teuthology.orchestra.run.smithi143.stdout: "", 2023-11-26T07:23:10.453 INFO:teuthology.orchestra.run.smithi143.stdout: "critical", 2023-11-26T07:23:10.453 INFO:teuthology.orchestra.run.smithi143.stdout: "debug", 2023-11-26T07:23:10.453 INFO:teuthology.orchestra.run.smithi143.stdout: "error", 2023-11-26T07:23:10.453 INFO:teuthology.orchestra.run.smithi143.stdout: "info", 2023-11-26T07:23:10.453 INFO:teuthology.orchestra.run.smithi143.stdout: "warning" 2023-11-26T07:23:10.453 INFO:teuthology.orchestra.run.smithi143.stdout: ], 2023-11-26T07:23:10.453 INFO:teuthology.orchestra.run.smithi143.stdout: "desc": "", 2023-11-26T07:23:10.453 INFO:teuthology.orchestra.run.smithi143.stdout: "long_desc": "", 2023-11-26T07:23:10.453 INFO:teuthology.orchestra.run.smithi143.stdout: "tags": [], 2023-11-26T07:23:10.453 INFO:teuthology.orchestra.run.smithi143.stdout: "see_also": [] 2023-11-26T07:23:10.453 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2023-11-26T07:23:10.454 INFO:teuthology.orchestra.run.smithi143.stdout: "log_to_file": { 2023-11-26T07:23:10.454 INFO:teuthology.orchestra.run.smithi143.stdout: "name": "log_to_file", 2023-11-26T07:23:10.454 INFO:teuthology.orchestra.run.smithi143.stdout: "type": "bool", 2023-11-26T07:23:10.454 INFO:teuthology.orchestra.run.smithi143.stdout: "level": "advanced", 2023-11-26T07:23:10.454 INFO:teuthology.orchestra.run.smithi143.stdout: "flags": 1, 2023-11-26T07:23:10.454 INFO:teuthology.orchestra.run.smithi143.stdout: "default_value": "False", 2023-11-26T07:23:10.454 INFO:teuthology.orchestra.run.smithi143.stdout: "min": "", 2023-11-26T07:23:10.454 INFO:teuthology.orchestra.run.smithi143.stdout: "max": "", 2023-11-26T07:23:10.454 INFO:teuthology.orchestra.run.smithi143.stdout: "enum_allowed": [], 2023-11-26T07:23:10.454 INFO:teuthology.orchestra.run.smithi143.stdout: "desc": "", 2023-11-26T07:23:10.454 INFO:teuthology.orchestra.run.smithi143.stdout: "long_desc": "", 2023-11-26T07:23:10.454 INFO:teuthology.orchestra.run.smithi143.stdout: "tags": [], 2023-11-26T07:23:10.454 INFO:teuthology.orchestra.run.smithi143.stdout: "see_also": [] 2023-11-26T07:23:10.455 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2023-11-26T07:23:10.455 INFO:teuthology.orchestra.run.smithi143.stdout: "smtp_destination": { 2023-11-26T07:23:10.455 INFO:teuthology.orchestra.run.smithi143.stdout: "name": "smtp_destination", 2023-11-26T07:23:10.455 INFO:teuthology.orchestra.run.smithi143.stdout: "type": "str", 2023-11-26T07:23:10.455 INFO:teuthology.orchestra.run.smithi143.stdout: "level": "advanced", 2023-11-26T07:23:10.455 INFO:teuthology.orchestra.run.smithi143.stdout: "flags": 1, 2023-11-26T07:23:10.455 INFO:teuthology.orchestra.run.smithi143.stdout: "default_value": "", 2023-11-26T07:23:10.455 INFO:teuthology.orchestra.run.smithi143.stdout: "min": "", 2023-11-26T07:23:10.455 INFO:teuthology.orchestra.run.smithi143.stdout: "max": "", 2023-11-26T07:23:10.455 INFO:teuthology.orchestra.run.smithi143.stdout: "enum_allowed": [], 2023-11-26T07:23:10.455 INFO:teuthology.orchestra.run.smithi143.stdout: "desc": "Email address to send alerts to", 2023-11-26T07:23:10.455 INFO:teuthology.orchestra.run.smithi143.stdout: "long_desc": "", 2023-11-26T07:23:10.456 INFO:teuthology.orchestra.run.smithi143.stdout: "tags": [], 2023-11-26T07:23:10.456 INFO:teuthology.orchestra.run.smithi143.stdout: "see_also": [] 2023-11-26T07:23:10.456 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2023-11-26T07:23:10.456 INFO:teuthology.orchestra.run.smithi143.stdout: "smtp_from_name": { 2023-11-26T07:23:10.456 INFO:teuthology.orchestra.run.smithi143.stdout: "name": "smtp_from_name", 2023-11-26T07:23:10.456 INFO:teuthology.orchestra.run.smithi143.stdout: "type": "str", 2023-11-26T07:23:10.456 INFO:teuthology.orchestra.run.smithi143.stdout: "level": "advanced", 2023-11-26T07:23:10.456 INFO:teuthology.orchestra.run.smithi143.stdout: "flags": 1, 2023-11-26T07:23:10.456 INFO:teuthology.orchestra.run.smithi143.stdout: "default_value": "Ceph", 2023-11-26T07:23:10.456 INFO:teuthology.orchestra.run.smithi143.stdout: "min": "", 2023-11-26T07:23:10.456 INFO:teuthology.orchestra.run.smithi143.stdout: "max": "", 2023-11-26T07:23:10.456 INFO:teuthology.orchestra.run.smithi143.stdout: "enum_allowed": [], 2023-11-26T07:23:10.457 INFO:teuthology.orchestra.run.smithi143.stdout: "desc": "Email From: name", 2023-11-26T07:23:10.457 INFO:teuthology.orchestra.run.smithi143.stdout: "long_desc": "", 2023-11-26T07:23:10.457 INFO:teuthology.orchestra.run.smithi143.stdout: "tags": [], 2023-11-26T07:23:10.457 INFO:teuthology.orchestra.run.smithi143.stdout: "see_also": [] 2023-11-26T07:23:10.457 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2023-11-26T07:23:10.457 INFO:teuthology.orchestra.run.smithi143.stdout: "smtp_host": { 2023-11-26T07:23:10.457 INFO:teuthology.orchestra.run.smithi143.stdout: "name": "smtp_host", 2023-11-26T07:23:10.457 INFO:teuthology.orchestra.run.smithi143.stdout: "type": "str", 2023-11-26T07:23:10.457 INFO:teuthology.orchestra.run.smithi143.stdout: "level": "advanced", 2023-11-26T07:23:10.457 INFO:teuthology.orchestra.run.smithi143.stdout: "flags": 1, 2023-11-26T07:23:10.457 INFO:teuthology.orchestra.run.smithi143.stdout: "default_value": "", 2023-11-26T07:23:10.457 INFO:teuthology.orchestra.run.smithi143.stdout: "min": "", 2023-11-26T07:23:10.457 INFO:teuthology.orchestra.run.smithi143.stdout: "max": "", 2023-11-26T07:23:10.458 INFO:teuthology.orchestra.run.smithi143.stdout: "enum_allowed": [], 2023-11-26T07:23:10.458 INFO:teuthology.orchestra.run.smithi143.stdout: "desc": "SMTP server", 2023-11-26T07:23:10.458 INFO:teuthology.orchestra.run.smithi143.stdout: "long_desc": "", 2023-11-26T07:23:10.458 INFO:teuthology.orchestra.run.smithi143.stdout: "tags": [], 2023-11-26T07:23:10.458 INFO:teuthology.orchestra.run.smithi143.stdout: "see_also": [] 2023-11-26T07:23:10.458 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2023-11-26T07:23:10.458 INFO:teuthology.orchestra.run.smithi143.stdout: "smtp_password": { 2023-11-26T07:23:10.458 INFO:teuthology.orchestra.run.smithi143.stdout: "name": "smtp_password", 2023-11-26T07:23:10.458 INFO:teuthology.orchestra.run.smithi143.stdout: "type": "str", 2023-11-26T07:23:10.458 INFO:teuthology.orchestra.run.smithi143.stdout: "level": "advanced", 2023-11-26T07:23:10.458 INFO:teuthology.orchestra.run.smithi143.stdout: "flags": 1, 2023-11-26T07:23:10.458 INFO:teuthology.orchestra.run.smithi143.stdout: "default_value": "", 2023-11-26T07:23:10.459 INFO:teuthology.orchestra.run.smithi143.stdout: "min": "", 2023-11-26T07:23:10.459 INFO:teuthology.orchestra.run.smithi143.stdout: "max": "", 2023-11-26T07:23:10.459 INFO:teuthology.orchestra.run.smithi143.stdout: "enum_allowed": [], 2023-11-26T07:23:10.459 INFO:teuthology.orchestra.run.smithi143.stdout: "desc": "Password to authenticate with", 2023-11-26T07:23:10.459 INFO:teuthology.orchestra.run.smithi143.stdout: "long_desc": "", 2023-11-26T07:23:10.459 INFO:teuthology.orchestra.run.smithi143.stdout: "tags": [], 2023-11-26T07:23:10.459 INFO:teuthology.orchestra.run.smithi143.stdout: "see_also": [] 2023-11-26T07:23:10.459 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2023-11-26T07:23:10.459 INFO:teuthology.orchestra.run.smithi143.stdout: "smtp_port": { 2023-11-26T07:23:10.459 INFO:teuthology.orchestra.run.smithi143.stdout: "name": "smtp_port", 2023-11-26T07:23:10.459 INFO:teuthology.orchestra.run.smithi143.stdout: "type": "int", 2023-11-26T07:23:10.459 INFO:teuthology.orchestra.run.smithi143.stdout: "level": "advanced", 2023-11-26T07:23:10.459 INFO:teuthology.orchestra.run.smithi143.stdout: "flags": 1, 2023-11-26T07:23:10.460 INFO:teuthology.orchestra.run.smithi143.stdout: "default_value": "465", 2023-11-26T07:23:10.460 INFO:teuthology.orchestra.run.smithi143.stdout: "min": "", 2023-11-26T07:23:10.460 INFO:teuthology.orchestra.run.smithi143.stdout: "max": "", 2023-11-26T07:23:10.460 INFO:teuthology.orchestra.run.smithi143.stdout: "enum_allowed": [], 2023-11-26T07:23:10.460 INFO:teuthology.orchestra.run.smithi143.stdout: "desc": "SMTP port", 2023-11-26T07:23:10.460 INFO:teuthology.orchestra.run.smithi143.stdout: "long_desc": "", 2023-11-26T07:23:10.460 INFO:teuthology.orchestra.run.smithi143.stdout: "tags": [], 2023-11-26T07:23:10.460 INFO:teuthology.orchestra.run.smithi143.stdout: "see_also": [] 2023-11-26T07:23:10.460 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2023-11-26T07:23:10.460 INFO:teuthology.orchestra.run.smithi143.stdout: "smtp_sender": { 2023-11-26T07:23:10.460 INFO:teuthology.orchestra.run.smithi143.stdout: "name": "smtp_sender", 2023-11-26T07:23:10.460 INFO:teuthology.orchestra.run.smithi143.stdout: "type": "str", 2023-11-26T07:23:10.460 INFO:teuthology.orchestra.run.smithi143.stdout: "level": "advanced", 2023-11-26T07:23:10.461 INFO:teuthology.orchestra.run.smithi143.stdout: "flags": 1, 2023-11-26T07:23:10.461 INFO:teuthology.orchestra.run.smithi143.stdout: "default_value": "", 2023-11-26T07:23:10.461 INFO:teuthology.orchestra.run.smithi143.stdout: "min": "", 2023-11-26T07:23:10.461 INFO:teuthology.orchestra.run.smithi143.stdout: "max": "", 2023-11-26T07:23:10.461 INFO:teuthology.orchestra.run.smithi143.stdout: "enum_allowed": [], 2023-11-26T07:23:10.461 INFO:teuthology.orchestra.run.smithi143.stdout: "desc": "SMTP envelope sender", 2023-11-26T07:23:10.461 INFO:teuthology.orchestra.run.smithi143.stdout: "long_desc": "", 2023-11-26T07:23:10.461 INFO:teuthology.orchestra.run.smithi143.stdout: "tags": [], 2023-11-26T07:23:10.461 INFO:teuthology.orchestra.run.smithi143.stdout: "see_also": [] 2023-11-26T07:23:10.461 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2023-11-26T07:23:10.461 INFO:teuthology.orchestra.run.smithi143.stdout: "smtp_ssl": { 2023-11-26T07:23:10.461 INFO:teuthology.orchestra.run.smithi143.stdout: "name": "smtp_ssl", 2023-11-26T07:23:10.461 INFO:teuthology.orchestra.run.smithi143.stdout: "type": "bool", 2023-11-26T07:23:10.462 INFO:teuthology.orchestra.run.smithi143.stdout: "level": "advanced", 2023-11-26T07:23:10.462 INFO:teuthology.orchestra.run.smithi143.stdout: "flags": 1, 2023-11-26T07:23:10.462 INFO:teuthology.orchestra.run.smithi143.stdout: "default_value": "True", 2023-11-26T07:23:10.462 INFO:teuthology.orchestra.run.smithi143.stdout: "min": "", 2023-11-26T07:23:10.462 INFO:teuthology.orchestra.run.smithi143.stdout: "max": "", 2023-11-26T07:23:10.462 INFO:teuthology.orchestra.run.smithi143.stdout: "enum_allowed": [], 2023-11-26T07:23:10.462 INFO:teuthology.orchestra.run.smithi143.stdout: "desc": "Use SSL to connect to SMTP server", 2023-11-26T07:23:10.462 INFO:teuthology.orchestra.run.smithi143.stdout: "long_desc": "", 2023-11-26T07:23:10.462 INFO:teuthology.orchestra.run.smithi143.stdout: "tags": [], 2023-11-26T07:23:10.462 INFO:teuthology.orchestra.run.smithi143.stdout: "see_also": [] 2023-11-26T07:23:10.462 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2023-11-26T07:23:10.462 INFO:teuthology.orchestra.run.smithi143.stdout: "smtp_user": { 2023-11-26T07:23:10.462 INFO:teuthology.orchestra.run.smithi143.stdout: "name": "smtp_user", 2023-11-26T07:23:10.463 INFO:teuthology.orchestra.run.smithi143.stdout: "type": "str", 2023-11-26T07:23:10.463 INFO:teuthology.orchestra.run.smithi143.stdout: "level": "advanced", 2023-11-26T07:23:10.463 INFO:teuthology.orchestra.run.smithi143.stdout: "flags": 1, 2023-11-26T07:23:10.463 INFO:teuthology.orchestra.run.smithi143.stdout: "default_value": "", 2023-11-26T07:23:10.463 INFO:teuthology.orchestra.run.smithi143.stdout: "min": "", 2023-11-26T07:23:10.463 INFO:teuthology.orchestra.run.smithi143.stdout: "max": "", 2023-11-26T07:23:10.463 INFO:teuthology.orchestra.run.smithi143.stdout: "enum_allowed": [], 2023-11-26T07:23:10.463 INFO:teuthology.orchestra.run.smithi143.stdout: "desc": "User to authenticate as", 2023-11-26T07:23:10.463 INFO:teuthology.orchestra.run.smithi143.stdout: "long_desc": "", 2023-11-26T07:23:10.463 INFO:teuthology.orchestra.run.smithi143.stdout: "tags": [], 2023-11-26T07:23:10.463 INFO:teuthology.orchestra.run.smithi143.stdout: "see_also": [] 2023-11-26T07:23:10.463 INFO:teuthology.orchestra.run.smithi143.stdout: } 2023-11-26T07:23:10.464 INFO:teuthology.orchestra.run.smithi143.stdout: } 2023-11-26T07:23:10.464 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2023-11-26T07:23:10.464 INFO:teuthology.orchestra.run.smithi143.stdout: { 2023-11-26T07:23:10.464 INFO:teuthology.orchestra.run.smithi143.stdout: "name": "balancer", 2023-11-26T07:23:10.464 INFO:teuthology.orchestra.run.smithi143.stdout: "can_run": true, 2023-11-26T07:23:10.464 INFO:teuthology.orchestra.run.smithi143.stdout: "error_string": "", 2023-11-26T07:23:10.464 INFO:teuthology.orchestra.run.smithi143.stdout: "module_options": { 2023-11-26T07:23:10.464 INFO:teuthology.orchestra.run.smithi143.stdout: "active": { 2023-11-26T07:23:10.464 INFO:teuthology.orchestra.run.smithi143.stdout: "name": "active", 2023-11-26T07:23:10.464 INFO:teuthology.orchestra.run.smithi143.stdout: "type": "bool", 2023-11-26T07:23:10.464 INFO:teuthology.orchestra.run.smithi143.stdout: "level": "advanced", 2023-11-26T07:23:10.464 INFO:teuthology.orchestra.run.smithi143.stdout: "flags": 1, 2023-11-26T07:23:10.464 INFO:teuthology.orchestra.run.smithi143.stdout: "default_value": "True", 2023-11-26T07:23:10.464 INFO:teuthology.orchestra.run.smithi143.stdout: "min": "", 2023-11-26T07:23:10.465 INFO:teuthology.orchestra.run.smithi143.stdout: "max": "", 2023-11-26T07:23:10.465 INFO:teuthology.orchestra.run.smithi143.stdout: "enum_allowed": [], 2023-11-26T07:23:10.465 INFO:teuthology.orchestra.run.smithi143.stdout: "desc": "automatically balance PGs across cluster", 2023-11-26T07:23:10.465 INFO:teuthology.orchestra.run.smithi143.stdout: "long_desc": "", 2023-11-26T07:23:10.465 INFO:teuthology.orchestra.run.smithi143.stdout: "tags": [], 2023-11-26T07:23:10.465 INFO:teuthology.orchestra.run.smithi143.stdout: "see_also": [] 2023-11-26T07:23:10.465 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2023-11-26T07:23:10.465 INFO:teuthology.orchestra.run.smithi143.stdout: "begin_time": { 2023-11-26T07:23:10.465 INFO:teuthology.orchestra.run.smithi143.stdout: "name": "begin_time", 2023-11-26T07:23:10.465 INFO:teuthology.orchestra.run.smithi143.stdout: "type": "str", 2023-11-26T07:23:10.465 INFO:teuthology.orchestra.run.smithi143.stdout: "level": "advanced", 2023-11-26T07:23:10.465 INFO:teuthology.orchestra.run.smithi143.stdout: "flags": 1, 2023-11-26T07:23:10.465 INFO:teuthology.orchestra.run.smithi143.stdout: "default_value": "0000", 2023-11-26T07:23:10.465 INFO:teuthology.orchestra.run.smithi143.stdout: "min": "", 2023-11-26T07:23:10.466 INFO:teuthology.orchestra.run.smithi143.stdout: "max": "", 2023-11-26T07:23:10.466 INFO:teuthology.orchestra.run.smithi143.stdout: "enum_allowed": [], 2023-11-26T07:23:10.466 INFO:teuthology.orchestra.run.smithi143.stdout: "desc": "beginning time of day to automatically balance", 2023-11-26T07:23:10.466 INFO:teuthology.orchestra.run.smithi143.stdout: "long_desc": "This is a time of day in the format HHMM.", 2023-11-26T07:23:10.466 INFO:teuthology.orchestra.run.smithi143.stdout: "tags": [], 2023-11-26T07:23:10.466 INFO:teuthology.orchestra.run.smithi143.stdout: "see_also": [] 2023-11-26T07:23:10.466 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2023-11-26T07:23:10.466 INFO:teuthology.orchestra.run.smithi143.stdout: "begin_weekday": { 2023-11-26T07:23:10.466 INFO:teuthology.orchestra.run.smithi143.stdout: "name": "begin_weekday", 2023-11-26T07:23:10.466 INFO:teuthology.orchestra.run.smithi143.stdout: "type": "uint", 2023-11-26T07:23:10.466 INFO:teuthology.orchestra.run.smithi143.stdout: "level": "advanced", 2023-11-26T07:23:10.466 INFO:teuthology.orchestra.run.smithi143.stdout: "flags": 1, 2023-11-26T07:23:10.467 INFO:teuthology.orchestra.run.smithi143.stdout: "default_value": "0", 2023-11-26T07:23:10.467 INFO:teuthology.orchestra.run.smithi143.stdout: "min": "0", 2023-11-26T07:23:10.467 INFO:teuthology.orchestra.run.smithi143.stdout: "max": "6", 2023-11-26T07:23:10.467 INFO:teuthology.orchestra.run.smithi143.stdout: "enum_allowed": [], 2023-11-26T07:23:10.467 INFO:teuthology.orchestra.run.smithi143.stdout: "desc": "Restrict automatic balancing to this day of the week or later", 2023-11-26T07:23:10.467 INFO:teuthology.orchestra.run.smithi143.stdout: "long_desc": "0 = Sunday, 1 = Monday, etc.", 2023-11-26T07:23:10.467 INFO:teuthology.orchestra.run.smithi143.stdout: "tags": [], 2023-11-26T07:23:10.467 INFO:teuthology.orchestra.run.smithi143.stdout: "see_also": [] 2023-11-26T07:23:10.467 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2023-11-26T07:23:10.467 INFO:teuthology.orchestra.run.smithi143.stdout: "crush_compat_max_iterations": { 2023-11-26T07:23:10.467 INFO:teuthology.orchestra.run.smithi143.stdout: "name": "crush_compat_max_iterations", 2023-11-26T07:23:10.467 INFO:teuthology.orchestra.run.smithi143.stdout: "type": "uint", 2023-11-26T07:23:10.467 INFO:teuthology.orchestra.run.smithi143.stdout: "level": "advanced", 2023-11-26T07:23:10.467 INFO:teuthology.orchestra.run.smithi143.stdout: "flags": 1, 2023-11-26T07:23:10.468 INFO:teuthology.orchestra.run.smithi143.stdout: "default_value": "25", 2023-11-26T07:23:10.468 INFO:teuthology.orchestra.run.smithi143.stdout: "min": "1", 2023-11-26T07:23:10.468 INFO:teuthology.orchestra.run.smithi143.stdout: "max": "250", 2023-11-26T07:23:10.468 INFO:teuthology.orchestra.run.smithi143.stdout: "enum_allowed": [], 2023-11-26T07:23:10.468 INFO:teuthology.orchestra.run.smithi143.stdout: "desc": "maximum number of iterations to attempt optimization", 2023-11-26T07:23:10.468 INFO:teuthology.orchestra.run.smithi143.stdout: "long_desc": "", 2023-11-26T07:23:10.468 INFO:teuthology.orchestra.run.smithi143.stdout: "tags": [], 2023-11-26T07:23:10.468 INFO:teuthology.orchestra.run.smithi143.stdout: "see_also": [] 2023-11-26T07:23:10.468 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2023-11-26T07:23:10.468 INFO:teuthology.orchestra.run.smithi143.stdout: "crush_compat_metrics": { 2023-11-26T07:23:10.468 INFO:teuthology.orchestra.run.smithi143.stdout: "name": "crush_compat_metrics", 2023-11-26T07:23:10.468 INFO:teuthology.orchestra.run.smithi143.stdout: "type": "str", 2023-11-26T07:23:10.468 INFO:teuthology.orchestra.run.smithi143.stdout: "level": "advanced", 2023-11-26T07:23:10.468 INFO:teuthology.orchestra.run.smithi143.stdout: "flags": 1, 2023-11-26T07:23:10.469 INFO:teuthology.orchestra.run.smithi143.stdout: "default_value": "pgs,objects,bytes", 2023-11-26T07:23:10.469 INFO:teuthology.orchestra.run.smithi143.stdout: "min": "", 2023-11-26T07:23:10.469 INFO:teuthology.orchestra.run.smithi143.stdout: "max": "", 2023-11-26T07:23:10.469 INFO:teuthology.orchestra.run.smithi143.stdout: "enum_allowed": [], 2023-11-26T07:23:10.469 INFO:teuthology.orchestra.run.smithi143.stdout: "desc": "metrics with which to calculate OSD utilization", 2023-11-26T07:23:10.469 INFO:teuthology.orchestra.run.smithi143.stdout: "long_desc": "Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.", 2023-11-26T07:23:10.469 INFO:teuthology.orchestra.run.smithi143.stdout: "tags": [], 2023-11-26T07:23:10.469 INFO:teuthology.orchestra.run.smithi143.stdout: "see_also": [] 2023-11-26T07:23:10.469 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2023-11-26T07:23:10.469 INFO:teuthology.orchestra.run.smithi143.stdout: "crush_compat_step": { 2023-11-26T07:23:10.469 INFO:teuthology.orchestra.run.smithi143.stdout: "name": "crush_compat_step", 2023-11-26T07:23:10.469 INFO:teuthology.orchestra.run.smithi143.stdout: "type": "float", 2023-11-26T07:23:10.469 INFO:teuthology.orchestra.run.smithi143.stdout: "level": "advanced", 2023-11-26T07:23:10.470 INFO:teuthology.orchestra.run.smithi143.stdout: "flags": 1, 2023-11-26T07:23:10.470 INFO:teuthology.orchestra.run.smithi143.stdout: "default_value": "0.5", 2023-11-26T07:23:10.470 INFO:teuthology.orchestra.run.smithi143.stdout: "min": "0.001", 2023-11-26T07:23:10.470 INFO:teuthology.orchestra.run.smithi143.stdout: "max": "0.999", 2023-11-26T07:23:10.470 INFO:teuthology.orchestra.run.smithi143.stdout: "enum_allowed": [], 2023-11-26T07:23:10.470 INFO:teuthology.orchestra.run.smithi143.stdout: "desc": "aggressiveness of optimization", 2023-11-26T07:23:10.470 INFO:teuthology.orchestra.run.smithi143.stdout: "long_desc": ".99 is very aggressive, .01 is less aggressive", 2023-11-26T07:23:10.470 INFO:teuthology.orchestra.run.smithi143.stdout: "tags": [], 2023-11-26T07:23:10.470 INFO:teuthology.orchestra.run.smithi143.stdout: "see_also": [] 2023-11-26T07:23:10.470 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2023-11-26T07:23:10.470 INFO:teuthology.orchestra.run.smithi143.stdout: "end_time": { 2023-11-26T07:23:10.470 INFO:teuthology.orchestra.run.smithi143.stdout: "name": "end_time", 2023-11-26T07:23:10.471 INFO:teuthology.orchestra.run.smithi143.stdout: "type": "str", 2023-11-26T07:23:10.471 INFO:teuthology.orchestra.run.smithi143.stdout: "level": "advanced", 2023-11-26T07:23:10.471 INFO:teuthology.orchestra.run.smithi143.stdout: "flags": 1, 2023-11-26T07:23:10.471 INFO:teuthology.orchestra.run.smithi143.stdout: "default_value": "2359", 2023-11-26T07:23:10.471 INFO:teuthology.orchestra.run.smithi143.stdout: "min": "", 2023-11-26T07:23:10.471 INFO:teuthology.orchestra.run.smithi143.stdout: "max": "", 2023-11-26T07:23:10.471 INFO:teuthology.orchestra.run.smithi143.stdout: "enum_allowed": [], 2023-11-26T07:23:10.471 INFO:teuthology.orchestra.run.smithi143.stdout: "desc": "ending time of day to automatically balance", 2023-11-26T07:23:10.471 INFO:teuthology.orchestra.run.smithi143.stdout: "long_desc": "This is a time of day in the format HHMM.", 2023-11-26T07:23:10.471 INFO:teuthology.orchestra.run.smithi143.stdout: "tags": [], 2023-11-26T07:23:10.471 INFO:teuthology.orchestra.run.smithi143.stdout: "see_also": [] 2023-11-26T07:23:10.471 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2023-11-26T07:23:10.471 INFO:teuthology.orchestra.run.smithi143.stdout: "end_weekday": { 2023-11-26T07:23:10.472 INFO:teuthology.orchestra.run.smithi143.stdout: "name": "end_weekday", 2023-11-26T07:23:10.472 INFO:teuthology.orchestra.run.smithi143.stdout: "type": "uint", 2023-11-26T07:23:10.472 INFO:teuthology.orchestra.run.smithi143.stdout: "level": "advanced", 2023-11-26T07:23:10.472 INFO:teuthology.orchestra.run.smithi143.stdout: "flags": 1, 2023-11-26T07:23:10.472 INFO:teuthology.orchestra.run.smithi143.stdout: "default_value": "0", 2023-11-26T07:23:10.472 INFO:teuthology.orchestra.run.smithi143.stdout: "min": "0", 2023-11-26T07:23:10.472 INFO:teuthology.orchestra.run.smithi143.stdout: "max": "6", 2023-11-26T07:23:10.472 INFO:teuthology.orchestra.run.smithi143.stdout: "enum_allowed": [], 2023-11-26T07:23:10.472 INFO:teuthology.orchestra.run.smithi143.stdout: "desc": "Restrict automatic balancing to days of the week earlier than this", 2023-11-26T07:23:10.472 INFO:teuthology.orchestra.run.smithi143.stdout: "long_desc": "0 = Sunday, 1 = Monday, etc.", 2023-11-26T07:23:10.472 INFO:teuthology.orchestra.run.smithi143.stdout: "tags": [], 2023-11-26T07:23:10.472 INFO:teuthology.orchestra.run.smithi143.stdout: "see_also": [] 2023-11-26T07:23:10.472 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2023-11-26T07:23:10.473 INFO:teuthology.orchestra.run.smithi143.stdout: "log_level": { 2023-11-26T07:23:10.473 INFO:teuthology.orchestra.run.smithi143.stdout: "name": "log_level", 2023-11-26T07:23:10.473 INFO:teuthology.orchestra.run.smithi143.stdout: "type": "str", 2023-11-26T07:23:10.473 INFO:teuthology.orchestra.run.smithi143.stdout: "level": "advanced", 2023-11-26T07:23:10.473 INFO:teuthology.orchestra.run.smithi143.stdout: "flags": 1, 2023-11-26T07:23:10.473 INFO:teuthology.orchestra.run.smithi143.stdout: "default_value": "", 2023-11-26T07:23:10.473 INFO:teuthology.orchestra.run.smithi143.stdout: "min": "", 2023-11-26T07:23:10.473 INFO:teuthology.orchestra.run.smithi143.stdout: "max": "", 2023-11-26T07:23:10.473 INFO:teuthology.orchestra.run.smithi143.stdout: "enum_allowed": [ 2023-11-26T07:23:10.473 INFO:teuthology.orchestra.run.smithi143.stdout: "", 2023-11-26T07:23:10.473 INFO:teuthology.orchestra.run.smithi143.stdout: "critical", 2023-11-26T07:23:10.473 INFO:teuthology.orchestra.run.smithi143.stdout: "debug", 2023-11-26T07:23:10.473 INFO:teuthology.orchestra.run.smithi143.stdout: "error", 2023-11-26T07:23:10.473 INFO:teuthology.orchestra.run.smithi143.stdout: "info", 2023-11-26T07:23:10.474 INFO:teuthology.orchestra.run.smithi143.stdout: "warning" 2023-11-26T07:23:10.474 INFO:teuthology.orchestra.run.smithi143.stdout: ], 2023-11-26T07:23:10.474 INFO:teuthology.orchestra.run.smithi143.stdout: "desc": "", 2023-11-26T07:23:10.474 INFO:teuthology.orchestra.run.smithi143.stdout: "long_desc": "", 2023-11-26T07:23:10.474 INFO:teuthology.orchestra.run.smithi143.stdout: "tags": [], 2023-11-26T07:23:10.474 INFO:teuthology.orchestra.run.smithi143.stdout: "see_also": [] 2023-11-26T07:23:10.474 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2023-11-26T07:23:10.474 INFO:teuthology.orchestra.run.smithi143.stdout: "log_to_cluster": { 2023-11-26T07:23:10.474 INFO:teuthology.orchestra.run.smithi143.stdout: "name": "log_to_cluster", 2023-11-26T07:23:10.474 INFO:teuthology.orchestra.run.smithi143.stdout: "type": "bool", 2023-11-26T07:23:10.474 INFO:teuthology.orchestra.run.smithi143.stdout: "level": "advanced", 2023-11-26T07:23:10.474 INFO:teuthology.orchestra.run.smithi143.stdout: "flags": 1, 2023-11-26T07:23:10.474 INFO:teuthology.orchestra.run.smithi143.stdout: "default_value": "False", 2023-11-26T07:23:10.475 INFO:teuthology.orchestra.run.smithi143.stdout: "min": "", 2023-11-26T07:23:10.475 INFO:teuthology.orchestra.run.smithi143.stdout: "max": "", 2023-11-26T07:23:10.475 INFO:teuthology.orchestra.run.smithi143.stdout: "enum_allowed": [], 2023-11-26T07:23:10.475 INFO:teuthology.orchestra.run.smithi143.stdout: "desc": "", 2023-11-26T07:23:10.475 INFO:teuthology.orchestra.run.smithi143.stdout: "long_desc": "", 2023-11-26T07:23:10.475 INFO:teuthology.orchestra.run.smithi143.stdout: "tags": [], 2023-11-26T07:23:10.475 INFO:teuthology.orchestra.run.smithi143.stdout: "see_also": [] 2023-11-26T07:23:10.475 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2023-11-26T07:23:10.475 INFO:teuthology.orchestra.run.smithi143.stdout: "log_to_cluster_level": { 2023-11-26T07:23:10.475 INFO:teuthology.orchestra.run.smithi143.stdout: "name": "log_to_cluster_level", 2023-11-26T07:23:10.475 INFO:teuthology.orchestra.run.smithi143.stdout: "type": "str", 2023-11-26T07:23:10.475 INFO:teuthology.orchestra.run.smithi143.stdout: "level": "advanced", 2023-11-26T07:23:10.475 INFO:teuthology.orchestra.run.smithi143.stdout: "flags": 1, 2023-11-26T07:23:10.475 INFO:teuthology.orchestra.run.smithi143.stdout: "default_value": "info", 2023-11-26T07:23:10.476 INFO:teuthology.orchestra.run.smithi143.stdout: "min": "", 2023-11-26T07:23:10.476 INFO:teuthology.orchestra.run.smithi143.stdout: "max": "", 2023-11-26T07:23:10.476 INFO:teuthology.orchestra.run.smithi143.stdout: "enum_allowed": [ 2023-11-26T07:23:10.476 INFO:teuthology.orchestra.run.smithi143.stdout: "", 2023-11-26T07:23:10.476 INFO:teuthology.orchestra.run.smithi143.stdout: "critical", 2023-11-26T07:23:10.476 INFO:teuthology.orchestra.run.smithi143.stdout: "debug", 2023-11-26T07:23:10.476 INFO:teuthology.orchestra.run.smithi143.stdout: "error", 2023-11-26T07:23:10.476 INFO:teuthology.orchestra.run.smithi143.stdout: "info", 2023-11-26T07:23:10.476 INFO:teuthology.orchestra.run.smithi143.stdout: "warning" 2023-11-26T07:23:10.476 INFO:teuthology.orchestra.run.smithi143.stdout: ], 2023-11-26T07:23:10.476 INFO:teuthology.orchestra.run.smithi143.stdout: "desc": "", 2023-11-26T07:23:10.476 INFO:teuthology.orchestra.run.smithi143.stdout: "long_desc": "", 2023-11-26T07:23:10.476 INFO:teuthology.orchestra.run.smithi143.stdout: "tags": [], 2023-11-26T07:23:10.477 INFO:teuthology.orchestra.run.smithi143.stdout: "see_also": [] 2023-11-26T07:23:10.477 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2023-11-26T07:23:10.477 INFO:teuthology.orchestra.run.smithi143.stdout: "log_to_file": { 2023-11-26T07:23:10.477 INFO:teuthology.orchestra.run.smithi143.stdout: "name": "log_to_file", 2023-11-26T07:23:10.477 INFO:teuthology.orchestra.run.smithi143.stdout: "type": "bool", 2023-11-26T07:23:10.477 INFO:teuthology.orchestra.run.smithi143.stdout: "level": "advanced", 2023-11-26T07:23:10.477 INFO:teuthology.orchestra.run.smithi143.stdout: "flags": 1, 2023-11-26T07:23:10.477 INFO:teuthology.orchestra.run.smithi143.stdout: "default_value": "False", 2023-11-26T07:23:10.477 INFO:teuthology.orchestra.run.smithi143.stdout: "min": "", 2023-11-26T07:23:10.477 INFO:teuthology.orchestra.run.smithi143.stdout: "max": "", 2023-11-26T07:23:10.477 INFO:teuthology.orchestra.run.smithi143.stdout: "enum_allowed": [], 2023-11-26T07:23:10.477 INFO:teuthology.orchestra.run.smithi143.stdout: "desc": "", 2023-11-26T07:23:10.478 INFO:teuthology.orchestra.run.smithi143.stdout: "long_desc": "", 2023-11-26T07:23:10.478 INFO:teuthology.orchestra.run.smithi143.stdout: "tags": [], 2023-11-26T07:23:10.478 INFO:teuthology.orchestra.run.smithi143.stdout: "see_also": [] 2023-11-26T07:23:10.478 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2023-11-26T07:23:10.478 INFO:teuthology.orchestra.run.smithi143.stdout: "min_score": { 2023-11-26T07:23:10.478 INFO:teuthology.orchestra.run.smithi143.stdout: "name": "min_score", 2023-11-26T07:23:10.478 INFO:teuthology.orchestra.run.smithi143.stdout: "type": "float", 2023-11-26T07:23:10.478 INFO:teuthology.orchestra.run.smithi143.stdout: "level": "advanced", 2023-11-26T07:23:10.478 INFO:teuthology.orchestra.run.smithi143.stdout: "flags": 1, 2023-11-26T07:23:10.478 INFO:teuthology.orchestra.run.smithi143.stdout: "default_value": "0", 2023-11-26T07:23:10.478 INFO:teuthology.orchestra.run.smithi143.stdout: "min": "", 2023-11-26T07:23:10.478 INFO:teuthology.orchestra.run.smithi143.stdout: "max": "", 2023-11-26T07:23:10.479 INFO:teuthology.orchestra.run.smithi143.stdout: "enum_allowed": [], 2023-11-26T07:23:10.479 INFO:teuthology.orchestra.run.smithi143.stdout: "desc": "minimum score, below which no optimization is attempted", 2023-11-26T07:23:10.479 INFO:teuthology.orchestra.run.smithi143.stdout: "long_desc": "", 2023-11-26T07:23:10.479 INFO:teuthology.orchestra.run.smithi143.stdout: "tags": [], 2023-11-26T07:23:10.479 INFO:teuthology.orchestra.run.smithi143.stdout: "see_also": [] 2023-11-26T07:23:10.479 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2023-11-26T07:23:10.479 INFO:teuthology.orchestra.run.smithi143.stdout: "mode": { 2023-11-26T07:23:10.479 INFO:teuthology.orchestra.run.smithi143.stdout: "name": "mode", 2023-11-26T07:23:10.479 INFO:teuthology.orchestra.run.smithi143.stdout: "type": "str", 2023-11-26T07:23:10.479 INFO:teuthology.orchestra.run.smithi143.stdout: "level": "advanced", 2023-11-26T07:23:10.479 INFO:teuthology.orchestra.run.smithi143.stdout: "flags": 1, 2023-11-26T07:23:10.479 INFO:teuthology.orchestra.run.smithi143.stdout: "default_value": "upmap", 2023-11-26T07:23:10.480 INFO:teuthology.orchestra.run.smithi143.stdout: "min": "", 2023-11-26T07:23:10.480 INFO:teuthology.orchestra.run.smithi143.stdout: "max": "", 2023-11-26T07:23:10.480 INFO:teuthology.orchestra.run.smithi143.stdout: "enum_allowed": [ 2023-11-26T07:23:10.480 INFO:teuthology.orchestra.run.smithi143.stdout: "crush-compat", 2023-11-26T07:23:10.480 INFO:teuthology.orchestra.run.smithi143.stdout: "none", 2023-11-26T07:23:10.480 INFO:teuthology.orchestra.run.smithi143.stdout: "upmap" 2023-11-26T07:23:10.480 INFO:teuthology.orchestra.run.smithi143.stdout: ], 2023-11-26T07:23:10.480 INFO:teuthology.orchestra.run.smithi143.stdout: "desc": "Balancer mode", 2023-11-26T07:23:10.480 INFO:teuthology.orchestra.run.smithi143.stdout: "long_desc": "", 2023-11-26T07:23:10.480 INFO:teuthology.orchestra.run.smithi143.stdout: "tags": [], 2023-11-26T07:23:10.480 INFO:teuthology.orchestra.run.smithi143.stdout: "see_also": [] 2023-11-26T07:23:10.480 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2023-11-26T07:23:10.480 INFO:teuthology.orchestra.run.smithi143.stdout: "pool_ids": { 2023-11-26T07:23:10.481 INFO:teuthology.orchestra.run.smithi143.stdout: "name": "pool_ids", 2023-11-26T07:23:10.481 INFO:teuthology.orchestra.run.smithi143.stdout: "type": "str", 2023-11-26T07:23:10.481 INFO:teuthology.orchestra.run.smithi143.stdout: "level": "advanced", 2023-11-26T07:23:10.481 INFO:teuthology.orchestra.run.smithi143.stdout: "flags": 1, 2023-11-26T07:23:10.481 INFO:teuthology.orchestra.run.smithi143.stdout: "default_value": "", 2023-11-26T07:23:10.481 INFO:teuthology.orchestra.run.smithi143.stdout: "min": "", 2023-11-26T07:23:10.481 INFO:teuthology.orchestra.run.smithi143.stdout: "max": "", 2023-11-26T07:23:10.481 INFO:teuthology.orchestra.run.smithi143.stdout: "enum_allowed": [], 2023-11-26T07:23:10.481 INFO:teuthology.orchestra.run.smithi143.stdout: "desc": "pools which the automatic balancing will be limited to", 2023-11-26T07:23:10.481 INFO:teuthology.orchestra.run.smithi143.stdout: "long_desc": "", 2023-11-26T07:23:10.481 INFO:teuthology.orchestra.run.smithi143.stdout: "tags": [], 2023-11-26T07:23:10.481 INFO:teuthology.orchestra.run.smithi143.stdout: "see_also": [] 2023-11-26T07:23:10.481 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2023-11-26T07:23:10.482 INFO:teuthology.orchestra.run.smithi143.stdout: "sleep_interval": { 2023-11-26T07:23:10.482 INFO:teuthology.orchestra.run.smithi143.stdout: "name": "sleep_interval", 2023-11-26T07:23:10.482 INFO:teuthology.orchestra.run.smithi143.stdout: "type": "secs", 2023-11-26T07:23:10.482 INFO:teuthology.orchestra.run.smithi143.stdout: "level": "advanced", 2023-11-26T07:23:10.482 INFO:teuthology.orchestra.run.smithi143.stdout: "flags": 1, 2023-11-26T07:23:10.482 INFO:teuthology.orchestra.run.smithi143.stdout: "default_value": "60", 2023-11-26T07:23:10.482 INFO:teuthology.orchestra.run.smithi143.stdout: "min": "", 2023-11-26T07:23:10.482 INFO:teuthology.orchestra.run.smithi143.stdout: "max": "", 2023-11-26T07:23:10.482 INFO:teuthology.orchestra.run.smithi143.stdout: "enum_allowed": [], 2023-11-26T07:23:10.482 INFO:teuthology.orchestra.run.smithi143.stdout: "desc": "how frequently to wake up and attempt optimization", 2023-11-26T07:23:10.482 INFO:teuthology.orchestra.run.smithi143.stdout: "long_desc": "", 2023-11-26T07:23:10.482 INFO:teuthology.orchestra.run.smithi143.stdout: "tags": [], 2023-11-26T07:23:10.482 INFO:teuthology.orchestra.run.smithi143.stdout: "see_also": [] 2023-11-26T07:23:10.483 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2023-11-26T07:23:10.483 INFO:teuthology.orchestra.run.smithi143.stdout: "upmap_max_deviation": { 2023-11-26T07:23:10.483 INFO:teuthology.orchestra.run.smithi143.stdout: "name": "upmap_max_deviation", 2023-11-26T07:23:10.483 INFO:teuthology.orchestra.run.smithi143.stdout: "type": "int", 2023-11-26T07:23:10.483 INFO:teuthology.orchestra.run.smithi143.stdout: "level": "advanced", 2023-11-26T07:23:10.483 INFO:teuthology.orchestra.run.smithi143.stdout: "flags": 1, 2023-11-26T07:23:10.483 INFO:teuthology.orchestra.run.smithi143.stdout: "default_value": "5", 2023-11-26T07:23:10.483 INFO:teuthology.orchestra.run.smithi143.stdout: "min": "1", 2023-11-26T07:23:10.483 INFO:teuthology.orchestra.run.smithi143.stdout: "max": "", 2023-11-26T07:23:10.483 INFO:teuthology.orchestra.run.smithi143.stdout: "enum_allowed": [], 2023-11-26T07:23:10.483 INFO:teuthology.orchestra.run.smithi143.stdout: "desc": "deviation below which no optimization is attempted", 2023-11-26T07:23:10.483 INFO:teuthology.orchestra.run.smithi143.stdout: "long_desc": "If the number of PGs are within this count then no optimization is attempted", 2023-11-26T07:23:10.483 INFO:teuthology.orchestra.run.smithi143.stdout: "tags": [], 2023-11-26T07:23:10.484 INFO:teuthology.orchestra.run.smithi143.stdout: "see_also": [] 2023-11-26T07:23:10.484 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2023-11-26T07:23:10.484 INFO:teuthology.orchestra.run.smithi143.stdout: "upmap_max_optimizations": { 2023-11-26T07:23:10.484 INFO:teuthology.orchestra.run.smithi143.stdout: "name": "upmap_max_optimizations", 2023-11-26T07:23:10.484 INFO:teuthology.orchestra.run.smithi143.stdout: "type": "uint", 2023-11-26T07:23:10.484 INFO:teuthology.orchestra.run.smithi143.stdout: "level": "advanced", 2023-11-26T07:23:10.484 INFO:teuthology.orchestra.run.smithi143.stdout: "flags": 1, 2023-11-26T07:23:10.484 INFO:teuthology.orchestra.run.smithi143.stdout: "default_value": "10", 2023-11-26T07:23:10.484 INFO:teuthology.orchestra.run.smithi143.stdout: "min": "", 2023-11-26T07:23:10.484 INFO:teuthology.orchestra.run.smithi143.stdout: "max": "", 2023-11-26T07:23:10.484 INFO:teuthology.orchestra.run.smithi143.stdout: "enum_allowed": [], 2023-11-26T07:23:10.484 INFO:teuthology.orchestra.run.smithi143.stdout: "desc": "maximum upmap optimizations to make per attempt", 2023-11-26T07:23:10.484 INFO:teuthology.orchestra.run.smithi143.stdout: "long_desc": "", 2023-11-26T07:23:10.485 INFO:teuthology.orchestra.run.smithi143.stdout: "tags": [], 2023-11-26T07:23:10.485 INFO:teuthology.orchestra.run.smithi143.stdout: "see_also": [] 2023-11-26T07:23:10.485 INFO:teuthology.orchestra.run.smithi143.stdout: } 2023-11-26T07:23:10.485 INFO:teuthology.orchestra.run.smithi143.stdout: } 2023-11-26T07:23:10.485 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2023-11-26T07:23:10.485 INFO:teuthology.orchestra.run.smithi143.stdout: { 2023-11-26T07:23:10.485 INFO:teuthology.orchestra.run.smithi143.stdout: "name": "cephadm", 2023-11-26T07:23:10.485 INFO:teuthology.orchestra.run.smithi143.stdout: "can_run": true, 2023-11-26T07:23:10.485 INFO:teuthology.orchestra.run.smithi143.stdout: "error_string": "", 2023-11-26T07:23:10.485 INFO:teuthology.orchestra.run.smithi143.stdout: "module_options": { 2023-11-26T07:23:10.485 INFO:teuthology.orchestra.run.smithi143.stdout: "agent_down_multiplier": { 2023-11-26T07:23:10.485 INFO:teuthology.orchestra.run.smithi143.stdout: "name": "agent_down_multiplier", 2023-11-26T07:23:10.485 INFO:teuthology.orchestra.run.smithi143.stdout: "type": "float", 2023-11-26T07:23:10.486 INFO:teuthology.orchestra.run.smithi143.stdout: "level": "advanced", 2023-11-26T07:23:10.486 INFO:teuthology.orchestra.run.smithi143.stdout: "flags": 0, 2023-11-26T07:23:10.486 INFO:teuthology.orchestra.run.smithi143.stdout: "default_value": "3.0", 2023-11-26T07:23:10.486 INFO:teuthology.orchestra.run.smithi143.stdout: "min": "", 2023-11-26T07:23:10.486 INFO:teuthology.orchestra.run.smithi143.stdout: "max": "", 2023-11-26T07:23:10.486 INFO:teuthology.orchestra.run.smithi143.stdout: "enum_allowed": [], 2023-11-26T07:23:10.486 INFO:teuthology.orchestra.run.smithi143.stdout: "desc": "Multiplied by agent refresh rate to calculate how long agent must not report before being marked down", 2023-11-26T07:23:10.486 INFO:teuthology.orchestra.run.smithi143.stdout: "long_desc": "", 2023-11-26T07:23:10.486 INFO:teuthology.orchestra.run.smithi143.stdout: "tags": [], 2023-11-26T07:23:10.486 INFO:teuthology.orchestra.run.smithi143.stdout: "see_also": [] 2023-11-26T07:23:10.486 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2023-11-26T07:23:10.486 INFO:teuthology.orchestra.run.smithi143.stdout: "agent_refresh_rate": { 2023-11-26T07:23:10.487 INFO:teuthology.orchestra.run.smithi143.stdout: "name": "agent_refresh_rate", 2023-11-26T07:23:10.487 INFO:teuthology.orchestra.run.smithi143.stdout: "type": "secs", 2023-11-26T07:23:10.487 INFO:teuthology.orchestra.run.smithi143.stdout: "level": "advanced", 2023-11-26T07:23:10.487 INFO:teuthology.orchestra.run.smithi143.stdout: "flags": 0, 2023-11-26T07:23:10.487 INFO:teuthology.orchestra.run.smithi143.stdout: "default_value": "20", 2023-11-26T07:23:10.487 INFO:teuthology.orchestra.run.smithi143.stdout: "min": "", 2023-11-26T07:23:10.487 INFO:teuthology.orchestra.run.smithi143.stdout: "max": "", 2023-11-26T07:23:10.487 INFO:teuthology.orchestra.run.smithi143.stdout: "enum_allowed": [], 2023-11-26T07:23:10.487 INFO:teuthology.orchestra.run.smithi143.stdout: "desc": "How often agent on each host will try to gather and send metadata", 2023-11-26T07:23:10.487 INFO:teuthology.orchestra.run.smithi143.stdout: "long_desc": "", 2023-11-26T07:23:10.487 INFO:teuthology.orchestra.run.smithi143.stdout: "tags": [], 2023-11-26T07:23:10.487 INFO:teuthology.orchestra.run.smithi143.stdout: "see_also": [] 2023-11-26T07:23:10.487 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2023-11-26T07:23:10.487 INFO:teuthology.orchestra.run.smithi143.stdout: "agent_starting_port": { 2023-11-26T07:23:10.488 INFO:teuthology.orchestra.run.smithi143.stdout: "name": "agent_starting_port", 2023-11-26T07:23:10.488 INFO:teuthology.orchestra.run.smithi143.stdout: "type": "int", 2023-11-26T07:23:10.488 INFO:teuthology.orchestra.run.smithi143.stdout: "level": "advanced", 2023-11-26T07:23:10.488 INFO:teuthology.orchestra.run.smithi143.stdout: "flags": 0, 2023-11-26T07:23:10.488 INFO:teuthology.orchestra.run.smithi143.stdout: "default_value": "4721", 2023-11-26T07:23:10.488 INFO:teuthology.orchestra.run.smithi143.stdout: "min": "", 2023-11-26T07:23:10.488 INFO:teuthology.orchestra.run.smithi143.stdout: "max": "", 2023-11-26T07:23:10.488 INFO:teuthology.orchestra.run.smithi143.stdout: "enum_allowed": [], 2023-11-26T07:23:10.488 INFO:teuthology.orchestra.run.smithi143.stdout: "desc": "First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)", 2023-11-26T07:23:10.488 INFO:teuthology.orchestra.run.smithi143.stdout: "long_desc": "", 2023-11-26T07:23:10.489 INFO:teuthology.orchestra.run.smithi143.stdout: "tags": [], 2023-11-26T07:23:10.489 INFO:teuthology.orchestra.run.smithi143.stdout: "see_also": [] 2023-11-26T07:23:10.489 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2023-11-26T07:23:10.489 INFO:teuthology.orchestra.run.smithi143.stdout: "allow_ptrace": { 2023-11-26T07:23:10.489 INFO:teuthology.orchestra.run.smithi143.stdout: "name": "allow_ptrace", 2023-11-26T07:23:10.489 INFO:teuthology.orchestra.run.smithi143.stdout: "type": "bool", 2023-11-26T07:23:10.489 INFO:teuthology.orchestra.run.smithi143.stdout: "level": "advanced", 2023-11-26T07:23:10.489 INFO:teuthology.orchestra.run.smithi143.stdout: "flags": 0, 2023-11-26T07:23:10.489 INFO:teuthology.orchestra.run.smithi143.stdout: "default_value": "False", 2023-11-26T07:23:10.489 INFO:teuthology.orchestra.run.smithi143.stdout: "min": "", 2023-11-26T07:23:10.489 INFO:teuthology.orchestra.run.smithi143.stdout: "max": "", 2023-11-26T07:23:10.489 INFO:teuthology.orchestra.run.smithi143.stdout: "enum_allowed": [], 2023-11-26T07:23:10.489 INFO:teuthology.orchestra.run.smithi143.stdout: "desc": "allow SYS_PTRACE capability on ceph containers", 2023-11-26T07:23:10.490 INFO:teuthology.orchestra.run.smithi143.stdout: "long_desc": "The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.", 2023-11-26T07:23:10.490 INFO:teuthology.orchestra.run.smithi143.stdout: "tags": [], 2023-11-26T07:23:10.490 INFO:teuthology.orchestra.run.smithi143.stdout: "see_also": [] 2023-11-26T07:23:10.490 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2023-11-26T07:23:10.490 INFO:teuthology.orchestra.run.smithi143.stdout: "autotune_interval": { 2023-11-26T07:23:10.490 INFO:teuthology.orchestra.run.smithi143.stdout: "name": "autotune_interval", 2023-11-26T07:23:10.490 INFO:teuthology.orchestra.run.smithi143.stdout: "type": "secs", 2023-11-26T07:23:10.490 INFO:teuthology.orchestra.run.smithi143.stdout: "level": "advanced", 2023-11-26T07:23:10.490 INFO:teuthology.orchestra.run.smithi143.stdout: "flags": 0, 2023-11-26T07:23:10.490 INFO:teuthology.orchestra.run.smithi143.stdout: "default_value": "600", 2023-11-26T07:23:10.490 INFO:teuthology.orchestra.run.smithi143.stdout: "min": "", 2023-11-26T07:23:10.490 INFO:teuthology.orchestra.run.smithi143.stdout: "max": "", 2023-11-26T07:23:10.491 INFO:teuthology.orchestra.run.smithi143.stdout: "enum_allowed": [], 2023-11-26T07:23:10.491 INFO:teuthology.orchestra.run.smithi143.stdout: "desc": "how frequently to autotune daemon memory", 2023-11-26T07:23:10.491 INFO:teuthology.orchestra.run.smithi143.stdout: "long_desc": "", 2023-11-26T07:23:10.491 INFO:teuthology.orchestra.run.smithi143.stdout: "tags": [], 2023-11-26T07:23:10.491 INFO:teuthology.orchestra.run.smithi143.stdout: "see_also": [] 2023-11-26T07:23:10.491 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2023-11-26T07:23:10.491 INFO:teuthology.orchestra.run.smithi143.stdout: "autotune_memory_target_ratio": { 2023-11-26T07:23:10.491 INFO:teuthology.orchestra.run.smithi143.stdout: "name": "autotune_memory_target_ratio", 2023-11-26T07:23:10.491 INFO:teuthology.orchestra.run.smithi143.stdout: "type": "float", 2023-11-26T07:23:10.491 INFO:teuthology.orchestra.run.smithi143.stdout: "level": "advanced", 2023-11-26T07:23:10.491 INFO:teuthology.orchestra.run.smithi143.stdout: "flags": 0, 2023-11-26T07:23:10.491 INFO:teuthology.orchestra.run.smithi143.stdout: "default_value": "0.7", 2023-11-26T07:23:10.491 INFO:teuthology.orchestra.run.smithi143.stdout: "min": "", 2023-11-26T07:23:10.492 INFO:teuthology.orchestra.run.smithi143.stdout: "max": "", 2023-11-26T07:23:10.492 INFO:teuthology.orchestra.run.smithi143.stdout: "enum_allowed": [], 2023-11-26T07:23:10.492 INFO:teuthology.orchestra.run.smithi143.stdout: "desc": "ratio of total system memory to divide amongst autotuned daemons", 2023-11-26T07:23:10.492 INFO:teuthology.orchestra.run.smithi143.stdout: "long_desc": "", 2023-11-26T07:23:10.492 INFO:teuthology.orchestra.run.smithi143.stdout: "tags": [], 2023-11-26T07:23:10.492 INFO:teuthology.orchestra.run.smithi143.stdout: "see_also": [] 2023-11-26T07:23:10.492 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2023-11-26T07:23:10.492 INFO:teuthology.orchestra.run.smithi143.stdout: "cephadm_log_destination": { 2023-11-26T07:23:10.492 INFO:teuthology.orchestra.run.smithi143.stdout: "name": "cephadm_log_destination", 2023-11-26T07:23:10.492 INFO:teuthology.orchestra.run.smithi143.stdout: "type": "str", 2023-11-26T07:23:10.492 INFO:teuthology.orchestra.run.smithi143.stdout: "level": "advanced", 2023-11-26T07:23:10.492 INFO:teuthology.orchestra.run.smithi143.stdout: "flags": 0, 2023-11-26T07:23:10.492 INFO:teuthology.orchestra.run.smithi143.stdout: "default_value": "", 2023-11-26T07:23:10.493 INFO:teuthology.orchestra.run.smithi143.stdout: "min": "", 2023-11-26T07:23:10.493 INFO:teuthology.orchestra.run.smithi143.stdout: "max": "", 2023-11-26T07:23:10.493 INFO:teuthology.orchestra.run.smithi143.stdout: "enum_allowed": [ 2023-11-26T07:23:10.493 INFO:teuthology.orchestra.run.smithi143.stdout: "file", 2023-11-26T07:23:10.493 INFO:teuthology.orchestra.run.smithi143.stdout: "file,syslog", 2023-11-26T07:23:10.493 INFO:teuthology.orchestra.run.smithi143.stdout: "syslog" 2023-11-26T07:23:10.493 INFO:teuthology.orchestra.run.smithi143.stdout: ], 2023-11-26T07:23:10.493 INFO:teuthology.orchestra.run.smithi143.stdout: "desc": "Destination for cephadm command's persistent logging", 2023-11-26T07:23:10.493 INFO:teuthology.orchestra.run.smithi143.stdout: "long_desc": "", 2023-11-26T07:23:10.493 INFO:teuthology.orchestra.run.smithi143.stdout: "tags": [], 2023-11-26T07:23:10.493 INFO:teuthology.orchestra.run.smithi143.stdout: "see_also": [] 2023-11-26T07:23:10.493 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2023-11-26T07:23:10.493 INFO:teuthology.orchestra.run.smithi143.stdout: "cgroups_split": { 2023-11-26T07:23:10.494 INFO:teuthology.orchestra.run.smithi143.stdout: "name": "cgroups_split", 2023-11-26T07:23:10.494 INFO:teuthology.orchestra.run.smithi143.stdout: "type": "bool", 2023-11-26T07:23:10.494 INFO:teuthology.orchestra.run.smithi143.stdout: "level": "advanced", 2023-11-26T07:23:10.494 INFO:teuthology.orchestra.run.smithi143.stdout: "flags": 0, 2023-11-26T07:23:10.494 INFO:teuthology.orchestra.run.smithi143.stdout: "default_value": "True", 2023-11-26T07:23:10.494 INFO:teuthology.orchestra.run.smithi143.stdout: "min": "", 2023-11-26T07:23:10.494 INFO:teuthology.orchestra.run.smithi143.stdout: "max": "", 2023-11-26T07:23:10.494 INFO:teuthology.orchestra.run.smithi143.stdout: "enum_allowed": [], 2023-11-26T07:23:10.494 INFO:teuthology.orchestra.run.smithi143.stdout: "desc": "Pass --cgroups=split when cephadm creates containers (currently podman only)", 2023-11-26T07:23:10.494 INFO:teuthology.orchestra.run.smithi143.stdout: "long_desc": "", 2023-11-26T07:23:10.494 INFO:teuthology.orchestra.run.smithi143.stdout: "tags": [], 2023-11-26T07:23:10.494 INFO:teuthology.orchestra.run.smithi143.stdout: "see_also": [] 2023-11-26T07:23:10.495 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2023-11-26T07:23:10.495 INFO:teuthology.orchestra.run.smithi143.stdout: "config_checks_enabled": { 2023-11-26T07:23:10.495 INFO:teuthology.orchestra.run.smithi143.stdout: "name": "config_checks_enabled", 2023-11-26T07:23:10.495 INFO:teuthology.orchestra.run.smithi143.stdout: "type": "bool", 2023-11-26T07:23:10.495 INFO:teuthology.orchestra.run.smithi143.stdout: "level": "advanced", 2023-11-26T07:23:10.495 INFO:teuthology.orchestra.run.smithi143.stdout: "flags": 0, 2023-11-26T07:23:10.495 INFO:teuthology.orchestra.run.smithi143.stdout: "default_value": "False", 2023-11-26T07:23:10.495 INFO:teuthology.orchestra.run.smithi143.stdout: "min": "", 2023-11-26T07:23:10.495 INFO:teuthology.orchestra.run.smithi143.stdout: "max": "", 2023-11-26T07:23:10.495 INFO:teuthology.orchestra.run.smithi143.stdout: "enum_allowed": [], 2023-11-26T07:23:10.495 INFO:teuthology.orchestra.run.smithi143.stdout: "desc": "Enable or disable the cephadm configuration analysis", 2023-11-26T07:23:10.495 INFO:teuthology.orchestra.run.smithi143.stdout: "long_desc": "", 2023-11-26T07:23:10.495 INFO:teuthology.orchestra.run.smithi143.stdout: "tags": [], 2023-11-26T07:23:10.496 INFO:teuthology.orchestra.run.smithi143.stdout: "see_also": [] 2023-11-26T07:23:10.496 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2023-11-26T07:23:10.496 INFO:teuthology.orchestra.run.smithi143.stdout: "config_dashboard": { 2023-11-26T07:23:10.496 INFO:teuthology.orchestra.run.smithi143.stdout: "name": "config_dashboard", 2023-11-26T07:23:10.496 INFO:teuthology.orchestra.run.smithi143.stdout: "type": "bool", 2023-11-26T07:23:10.496 INFO:teuthology.orchestra.run.smithi143.stdout: "level": "advanced", 2023-11-26T07:23:10.496 INFO:teuthology.orchestra.run.smithi143.stdout: "flags": 0, 2023-11-26T07:23:10.496 INFO:teuthology.orchestra.run.smithi143.stdout: "default_value": "True", 2023-11-26T07:23:10.496 INFO:teuthology.orchestra.run.smithi143.stdout: "min": "", 2023-11-26T07:23:10.496 INFO:teuthology.orchestra.run.smithi143.stdout: "max": "", 2023-11-26T07:23:10.497 INFO:teuthology.orchestra.run.smithi143.stdout: "enum_allowed": [], 2023-11-26T07:23:10.497 INFO:teuthology.orchestra.run.smithi143.stdout: "desc": "manage configs like API endpoints in Dashboard.", 2023-11-26T07:23:10.497 INFO:teuthology.orchestra.run.smithi143.stdout: "long_desc": "", 2023-11-26T07:23:10.497 INFO:teuthology.orchestra.run.smithi143.stdout: "tags": [], 2023-11-26T07:23:10.497 INFO:teuthology.orchestra.run.smithi143.stdout: "see_also": [] 2023-11-26T07:23:10.497 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2023-11-26T07:23:10.497 INFO:teuthology.orchestra.run.smithi143.stdout: "container_image_alertmanager": { 2023-11-26T07:23:10.497 INFO:teuthology.orchestra.run.smithi143.stdout: "name": "container_image_alertmanager", 2023-11-26T07:23:10.497 INFO:teuthology.orchestra.run.smithi143.stdout: "type": "str", 2023-11-26T07:23:10.497 INFO:teuthology.orchestra.run.smithi143.stdout: "level": "advanced", 2023-11-26T07:23:10.497 INFO:teuthology.orchestra.run.smithi143.stdout: "flags": 0, 2023-11-26T07:23:10.497 INFO:teuthology.orchestra.run.smithi143.stdout: "default_value": "quay.io/prometheus/alertmanager:v0.25.0", 2023-11-26T07:23:10.498 INFO:teuthology.orchestra.run.smithi143.stdout: "min": "", 2023-11-26T07:23:10.498 INFO:teuthology.orchestra.run.smithi143.stdout: "max": "", 2023-11-26T07:23:10.498 INFO:teuthology.orchestra.run.smithi143.stdout: "enum_allowed": [], 2023-11-26T07:23:10.498 INFO:teuthology.orchestra.run.smithi143.stdout: "desc": "Prometheus container image", 2023-11-26T07:23:10.498 INFO:teuthology.orchestra.run.smithi143.stdout: "long_desc": "", 2023-11-26T07:23:10.498 INFO:teuthology.orchestra.run.smithi143.stdout: "tags": [], 2023-11-26T07:23:10.498 INFO:teuthology.orchestra.run.smithi143.stdout: "see_also": [] 2023-11-26T07:23:10.498 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2023-11-26T07:23:10.498 INFO:teuthology.orchestra.run.smithi143.stdout: "container_image_base": { 2023-11-26T07:23:10.498 INFO:teuthology.orchestra.run.smithi143.stdout: "name": "container_image_base", 2023-11-26T07:23:10.498 INFO:teuthology.orchestra.run.smithi143.stdout: "type": "str", 2023-11-26T07:23:10.498 INFO:teuthology.orchestra.run.smithi143.stdout: "level": "advanced", 2023-11-26T07:23:10.499 INFO:teuthology.orchestra.run.smithi143.stdout: "flags": 1, 2023-11-26T07:23:10.499 INFO:teuthology.orchestra.run.smithi143.stdout: "default_value": "quay.io/ceph/ceph", 2023-11-26T07:23:10.499 INFO:teuthology.orchestra.run.smithi143.stdout: "min": "", 2023-11-26T07:23:10.499 INFO:teuthology.orchestra.run.smithi143.stdout: "max": "", 2023-11-26T07:23:10.499 INFO:teuthology.orchestra.run.smithi143.stdout: "enum_allowed": [], 2023-11-26T07:23:10.499 INFO:teuthology.orchestra.run.smithi143.stdout: "desc": "Container image name, without the tag", 2023-11-26T07:23:10.499 INFO:teuthology.orchestra.run.smithi143.stdout: "long_desc": "", 2023-11-26T07:23:10.499 INFO:teuthology.orchestra.run.smithi143.stdout: "tags": [], 2023-11-26T07:23:10.499 INFO:teuthology.orchestra.run.smithi143.stdout: "see_also": [] 2023-11-26T07:23:10.499 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2023-11-26T07:23:10.499 INFO:teuthology.orchestra.run.smithi143.stdout: "container_image_elasticsearch": { 2023-11-26T07:23:10.499 INFO:teuthology.orchestra.run.smithi143.stdout: "name": "container_image_elasticsearch", 2023-11-26T07:23:10.499 INFO:teuthology.orchestra.run.smithi143.stdout: "type": "str", 2023-11-26T07:23:10.500 INFO:teuthology.orchestra.run.smithi143.stdout: "level": "advanced", 2023-11-26T07:23:10.500 INFO:teuthology.orchestra.run.smithi143.stdout: "flags": 0, 2023-11-26T07:23:10.500 INFO:teuthology.orchestra.run.smithi143.stdout: "default_value": "quay.io/omrizeneva/elasticsearch:6.8.23", 2023-11-26T07:23:10.500 INFO:teuthology.orchestra.run.smithi143.stdout: "min": "", 2023-11-26T07:23:10.500 INFO:teuthology.orchestra.run.smithi143.stdout: "max": "", 2023-11-26T07:23:10.500 INFO:teuthology.orchestra.run.smithi143.stdout: "enum_allowed": [], 2023-11-26T07:23:10.500 INFO:teuthology.orchestra.run.smithi143.stdout: "desc": "elasticsearch container image", 2023-11-26T07:23:10.500 INFO:teuthology.orchestra.run.smithi143.stdout: "long_desc": "", 2023-11-26T07:23:10.500 INFO:teuthology.orchestra.run.smithi143.stdout: "tags": [], 2023-11-26T07:23:10.500 INFO:teuthology.orchestra.run.smithi143.stdout: "see_also": [] 2023-11-26T07:23:10.500 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2023-11-26T07:23:10.500 INFO:teuthology.orchestra.run.smithi143.stdout: "container_image_grafana": { 2023-11-26T07:23:10.500 INFO:teuthology.orchestra.run.smithi143.stdout: "name": "container_image_grafana", 2023-11-26T07:23:10.501 INFO:teuthology.orchestra.run.smithi143.stdout: "type": "str", 2023-11-26T07:23:10.501 INFO:teuthology.orchestra.run.smithi143.stdout: "level": "advanced", 2023-11-26T07:23:10.501 INFO:teuthology.orchestra.run.smithi143.stdout: "flags": 0, 2023-11-26T07:23:10.501 INFO:teuthology.orchestra.run.smithi143.stdout: "default_value": "quay.io/ceph/ceph-grafana:9.4.7", 2023-11-26T07:23:10.501 INFO:teuthology.orchestra.run.smithi143.stdout: "min": "", 2023-11-26T07:23:10.501 INFO:teuthology.orchestra.run.smithi143.stdout: "max": "", 2023-11-26T07:23:10.501 INFO:teuthology.orchestra.run.smithi143.stdout: "enum_allowed": [], 2023-11-26T07:23:10.501 INFO:teuthology.orchestra.run.smithi143.stdout: "desc": "Prometheus container image", 2023-11-26T07:23:10.501 INFO:teuthology.orchestra.run.smithi143.stdout: "long_desc": "", 2023-11-26T07:23:10.501 INFO:teuthology.orchestra.run.smithi143.stdout: "tags": [], 2023-11-26T07:23:10.501 INFO:teuthology.orchestra.run.smithi143.stdout: "see_also": [] 2023-11-26T07:23:10.501 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2023-11-26T07:23:10.501 INFO:teuthology.orchestra.run.smithi143.stdout: "container_image_haproxy": { 2023-11-26T07:23:10.502 INFO:teuthology.orchestra.run.smithi143.stdout: "name": "container_image_haproxy", 2023-11-26T07:23:10.502 INFO:teuthology.orchestra.run.smithi143.stdout: "type": "str", 2023-11-26T07:23:10.502 INFO:teuthology.orchestra.run.smithi143.stdout: "level": "advanced", 2023-11-26T07:23:10.502 INFO:teuthology.orchestra.run.smithi143.stdout: "flags": 0, 2023-11-26T07:23:10.502 INFO:teuthology.orchestra.run.smithi143.stdout: "default_value": "quay.io/ceph/haproxy:2.3", 2023-11-26T07:23:10.502 INFO:teuthology.orchestra.run.smithi143.stdout: "min": "", 2023-11-26T07:23:10.502 INFO:teuthology.orchestra.run.smithi143.stdout: "max": "", 2023-11-26T07:23:10.502 INFO:teuthology.orchestra.run.smithi143.stdout: "enum_allowed": [], 2023-11-26T07:23:10.502 INFO:teuthology.orchestra.run.smithi143.stdout: "desc": "HAproxy container image", 2023-11-26T07:23:10.502 INFO:teuthology.orchestra.run.smithi143.stdout: "long_desc": "", 2023-11-26T07:23:10.502 INFO:teuthology.orchestra.run.smithi143.stdout: "tags": [], 2023-11-26T07:23:10.502 INFO:teuthology.orchestra.run.smithi143.stdout: "see_also": [] 2023-11-26T07:23:10.502 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2023-11-26T07:23:10.503 INFO:teuthology.orchestra.run.smithi143.stdout: "container_image_jaeger_agent": { 2023-11-26T07:23:10.503 INFO:teuthology.orchestra.run.smithi143.stdout: "name": "container_image_jaeger_agent", 2023-11-26T07:23:10.503 INFO:teuthology.orchestra.run.smithi143.stdout: "type": "str", 2023-11-26T07:23:10.503 INFO:teuthology.orchestra.run.smithi143.stdout: "level": "advanced", 2023-11-26T07:23:10.503 INFO:teuthology.orchestra.run.smithi143.stdout: "flags": 0, 2023-11-26T07:23:10.503 INFO:teuthology.orchestra.run.smithi143.stdout: "default_value": "quay.io/jaegertracing/jaeger-agent:1.29", 2023-11-26T07:23:10.503 INFO:teuthology.orchestra.run.smithi143.stdout: "min": "", 2023-11-26T07:23:10.503 INFO:teuthology.orchestra.run.smithi143.stdout: "max": "", 2023-11-26T07:23:10.503 INFO:teuthology.orchestra.run.smithi143.stdout: "enum_allowed": [], 2023-11-26T07:23:10.503 INFO:teuthology.orchestra.run.smithi143.stdout: "desc": "Jaeger agent container image", 2023-11-26T07:23:10.503 INFO:teuthology.orchestra.run.smithi143.stdout: "long_desc": "", 2023-11-26T07:23:10.503 INFO:teuthology.orchestra.run.smithi143.stdout: "tags": [], 2023-11-26T07:23:10.503 INFO:teuthology.orchestra.run.smithi143.stdout: "see_also": [] 2023-11-26T07:23:10.504 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2023-11-26T07:23:10.504 INFO:teuthology.orchestra.run.smithi143.stdout: "container_image_jaeger_collector": { 2023-11-26T07:23:10.504 INFO:teuthology.orchestra.run.smithi143.stdout: "name": "container_image_jaeger_collector", 2023-11-26T07:23:10.504 INFO:teuthology.orchestra.run.smithi143.stdout: "type": "str", 2023-11-26T07:23:10.504 INFO:teuthology.orchestra.run.smithi143.stdout: "level": "advanced", 2023-11-26T07:23:10.504 INFO:teuthology.orchestra.run.smithi143.stdout: "flags": 0, 2023-11-26T07:23:10.504 INFO:teuthology.orchestra.run.smithi143.stdout: "default_value": "quay.io/jaegertracing/jaeger-collector:1.29", 2023-11-26T07:23:10.504 INFO:teuthology.orchestra.run.smithi143.stdout: "min": "", 2023-11-26T07:23:10.504 INFO:teuthology.orchestra.run.smithi143.stdout: "max": "", 2023-11-26T07:23:10.504 INFO:teuthology.orchestra.run.smithi143.stdout: "enum_allowed": [], 2023-11-26T07:23:10.504 INFO:teuthology.orchestra.run.smithi143.stdout: "desc": "Jaeger collector container image", 2023-11-26T07:23:10.504 INFO:teuthology.orchestra.run.smithi143.stdout: "long_desc": "", 2023-11-26T07:23:10.504 INFO:teuthology.orchestra.run.smithi143.stdout: "tags": [], 2023-11-26T07:23:10.504 INFO:teuthology.orchestra.run.smithi143.stdout: "see_also": [] 2023-11-26T07:23:10.505 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2023-11-26T07:23:10.505 INFO:teuthology.orchestra.run.smithi143.stdout: "container_image_jaeger_query": { 2023-11-26T07:23:10.505 INFO:teuthology.orchestra.run.smithi143.stdout: "name": "container_image_jaeger_query", 2023-11-26T07:23:10.505 INFO:teuthology.orchestra.run.smithi143.stdout: "type": "str", 2023-11-26T07:23:10.505 INFO:teuthology.orchestra.run.smithi143.stdout: "level": "advanced", 2023-11-26T07:23:10.505 INFO:teuthology.orchestra.run.smithi143.stdout: "flags": 0, 2023-11-26T07:23:10.505 INFO:teuthology.orchestra.run.smithi143.stdout: "default_value": "quay.io/jaegertracing/jaeger-query:1.29", 2023-11-26T07:23:10.505 INFO:teuthology.orchestra.run.smithi143.stdout: "min": "", 2023-11-26T07:23:10.505 INFO:teuthology.orchestra.run.smithi143.stdout: "max": "", 2023-11-26T07:23:10.505 INFO:teuthology.orchestra.run.smithi143.stdout: "enum_allowed": [], 2023-11-26T07:23:10.505 INFO:teuthology.orchestra.run.smithi143.stdout: "desc": "Jaeger query container image", 2023-11-26T07:23:10.505 INFO:teuthology.orchestra.run.smithi143.stdout: "long_desc": "", 2023-11-26T07:23:10.505 INFO:teuthology.orchestra.run.smithi143.stdout: "tags": [], 2023-11-26T07:23:10.506 INFO:teuthology.orchestra.run.smithi143.stdout: "see_also": [] 2023-11-26T07:23:10.506 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2023-11-26T07:23:10.506 INFO:teuthology.orchestra.run.smithi143.stdout: "container_image_keepalived": { 2023-11-26T07:23:10.506 INFO:teuthology.orchestra.run.smithi143.stdout: "name": "container_image_keepalived", 2023-11-26T07:23:10.506 INFO:teuthology.orchestra.run.smithi143.stdout: "type": "str", 2023-11-26T07:23:10.506 INFO:teuthology.orchestra.run.smithi143.stdout: "level": "advanced", 2023-11-26T07:23:10.506 INFO:teuthology.orchestra.run.smithi143.stdout: "flags": 0, 2023-11-26T07:23:10.506 INFO:teuthology.orchestra.run.smithi143.stdout: "default_value": "quay.io/ceph/keepalived:2.2.4", 2023-11-26T07:23:10.506 INFO:teuthology.orchestra.run.smithi143.stdout: "min": "", 2023-11-26T07:23:10.506 INFO:teuthology.orchestra.run.smithi143.stdout: "max": "", 2023-11-26T07:23:10.506 INFO:teuthology.orchestra.run.smithi143.stdout: "enum_allowed": [], 2023-11-26T07:23:10.506 INFO:teuthology.orchestra.run.smithi143.stdout: "desc": "Keepalived container image", 2023-11-26T07:23:10.507 INFO:teuthology.orchestra.run.smithi143.stdout: "long_desc": "", 2023-11-26T07:23:10.507 INFO:teuthology.orchestra.run.smithi143.stdout: "tags": [], 2023-11-26T07:23:10.507 INFO:teuthology.orchestra.run.smithi143.stdout: "see_also": [] 2023-11-26T07:23:10.507 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2023-11-26T07:23:10.507 INFO:teuthology.orchestra.run.smithi143.stdout: "container_image_loki": { 2023-11-26T07:23:10.507 INFO:teuthology.orchestra.run.smithi143.stdout: "name": "container_image_loki", 2023-11-26T07:23:10.507 INFO:teuthology.orchestra.run.smithi143.stdout: "type": "str", 2023-11-26T07:23:10.507 INFO:teuthology.orchestra.run.smithi143.stdout: "level": "advanced", 2023-11-26T07:23:10.507 INFO:teuthology.orchestra.run.smithi143.stdout: "flags": 0, 2023-11-26T07:23:10.507 INFO:teuthology.orchestra.run.smithi143.stdout: "default_value": "docker.io/grafana/loki:2.4.0", 2023-11-26T07:23:10.507 INFO:teuthology.orchestra.run.smithi143.stdout: "min": "", 2023-11-26T07:23:10.507 INFO:teuthology.orchestra.run.smithi143.stdout: "max": "", 2023-11-26T07:23:10.508 INFO:teuthology.orchestra.run.smithi143.stdout: "enum_allowed": [], 2023-11-26T07:23:10.508 INFO:teuthology.orchestra.run.smithi143.stdout: "desc": "Loki container image", 2023-11-26T07:23:10.508 INFO:teuthology.orchestra.run.smithi143.stdout: "long_desc": "", 2023-11-26T07:23:10.508 INFO:teuthology.orchestra.run.smithi143.stdout: "tags": [], 2023-11-26T07:23:10.508 INFO:teuthology.orchestra.run.smithi143.stdout: "see_also": [] 2023-11-26T07:23:10.508 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2023-11-26T07:23:10.508 INFO:teuthology.orchestra.run.smithi143.stdout: "container_image_node_exporter": { 2023-11-26T07:23:10.508 INFO:teuthology.orchestra.run.smithi143.stdout: "name": "container_image_node_exporter", 2023-11-26T07:23:10.508 INFO:teuthology.orchestra.run.smithi143.stdout: "type": "str", 2023-11-26T07:23:10.508 INFO:teuthology.orchestra.run.smithi143.stdout: "level": "advanced", 2023-11-26T07:23:10.508 INFO:teuthology.orchestra.run.smithi143.stdout: "flags": 0, 2023-11-26T07:23:10.508 INFO:teuthology.orchestra.run.smithi143.stdout: "default_value": "quay.io/prometheus/node-exporter:v1.5.0", 2023-11-26T07:23:10.509 INFO:teuthology.orchestra.run.smithi143.stdout: "min": "", 2023-11-26T07:23:10.509 INFO:teuthology.orchestra.run.smithi143.stdout: "max": "", 2023-11-26T07:23:10.509 INFO:teuthology.orchestra.run.smithi143.stdout: "enum_allowed": [], 2023-11-26T07:23:10.509 INFO:teuthology.orchestra.run.smithi143.stdout: "desc": "Prometheus container image", 2023-11-26T07:23:10.509 INFO:teuthology.orchestra.run.smithi143.stdout: "long_desc": "", 2023-11-26T07:23:10.509 INFO:teuthology.orchestra.run.smithi143.stdout: "tags": [], 2023-11-26T07:23:10.509 INFO:teuthology.orchestra.run.smithi143.stdout: "see_also": [] 2023-11-26T07:23:10.509 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2023-11-26T07:23:10.509 INFO:teuthology.orchestra.run.smithi143.stdout: "container_image_nvmeof": { 2023-11-26T07:23:10.509 INFO:teuthology.orchestra.run.smithi143.stdout: "name": "container_image_nvmeof", 2023-11-26T07:23:10.509 INFO:teuthology.orchestra.run.smithi143.stdout: "type": "str", 2023-11-26T07:23:10.509 INFO:teuthology.orchestra.run.smithi143.stdout: "level": "advanced", 2023-11-26T07:23:10.509 INFO:teuthology.orchestra.run.smithi143.stdout: "flags": 0, 2023-11-26T07:23:10.510 INFO:teuthology.orchestra.run.smithi143.stdout: "default_value": "quay.io/ceph/nvmeof:latest", 2023-11-26T07:23:10.510 INFO:teuthology.orchestra.run.smithi143.stdout: "min": "", 2023-11-26T07:23:10.510 INFO:teuthology.orchestra.run.smithi143.stdout: "max": "", 2023-11-26T07:23:10.510 INFO:teuthology.orchestra.run.smithi143.stdout: "enum_allowed": [], 2023-11-26T07:23:10.510 INFO:teuthology.orchestra.run.smithi143.stdout: "desc": "Nvme-of container image", 2023-11-26T07:23:10.510 INFO:teuthology.orchestra.run.smithi143.stdout: "long_desc": "", 2023-11-26T07:23:10.510 INFO:teuthology.orchestra.run.smithi143.stdout: "tags": [], 2023-11-26T07:23:10.510 INFO:teuthology.orchestra.run.smithi143.stdout: "see_also": [] 2023-11-26T07:23:10.510 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2023-11-26T07:23:10.510 INFO:teuthology.orchestra.run.smithi143.stdout: "container_image_prometheus": { 2023-11-26T07:23:10.510 INFO:teuthology.orchestra.run.smithi143.stdout: "name": "container_image_prometheus", 2023-11-26T07:23:10.510 INFO:teuthology.orchestra.run.smithi143.stdout: "type": "str", 2023-11-26T07:23:10.511 INFO:teuthology.orchestra.run.smithi143.stdout: "level": "advanced", 2023-11-26T07:23:10.511 INFO:teuthology.orchestra.run.smithi143.stdout: "flags": 0, 2023-11-26T07:23:10.511 INFO:teuthology.orchestra.run.smithi143.stdout: "default_value": "quay.io/prometheus/prometheus:v2.43.0", 2023-11-26T07:23:10.511 INFO:teuthology.orchestra.run.smithi143.stdout: "min": "", 2023-11-26T07:23:10.511 INFO:teuthology.orchestra.run.smithi143.stdout: "max": "", 2023-11-26T07:23:10.511 INFO:teuthology.orchestra.run.smithi143.stdout: "enum_allowed": [], 2023-11-26T07:23:10.511 INFO:teuthology.orchestra.run.smithi143.stdout: "desc": "Prometheus container image", 2023-11-26T07:23:10.511 INFO:teuthology.orchestra.run.smithi143.stdout: "long_desc": "", 2023-11-26T07:23:10.511 INFO:teuthology.orchestra.run.smithi143.stdout: "tags": [], 2023-11-26T07:23:10.511 INFO:teuthology.orchestra.run.smithi143.stdout: "see_also": [] 2023-11-26T07:23:10.511 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2023-11-26T07:23:10.511 INFO:teuthology.orchestra.run.smithi143.stdout: "container_image_promtail": { 2023-11-26T07:23:10.511 INFO:teuthology.orchestra.run.smithi143.stdout: "name": "container_image_promtail", 2023-11-26T07:23:10.512 INFO:teuthology.orchestra.run.smithi143.stdout: "type": "str", 2023-11-26T07:23:10.512 INFO:teuthology.orchestra.run.smithi143.stdout: "level": "advanced", 2023-11-26T07:23:10.512 INFO:teuthology.orchestra.run.smithi143.stdout: "flags": 0, 2023-11-26T07:23:10.512 INFO:teuthology.orchestra.run.smithi143.stdout: "default_value": "docker.io/grafana/promtail:2.4.0", 2023-11-26T07:23:10.512 INFO:teuthology.orchestra.run.smithi143.stdout: "min": "", 2023-11-26T07:23:10.512 INFO:teuthology.orchestra.run.smithi143.stdout: "max": "", 2023-11-26T07:23:10.512 INFO:teuthology.orchestra.run.smithi143.stdout: "enum_allowed": [], 2023-11-26T07:23:10.512 INFO:teuthology.orchestra.run.smithi143.stdout: "desc": "Promtail container image", 2023-11-26T07:23:10.512 INFO:teuthology.orchestra.run.smithi143.stdout: "long_desc": "", 2023-11-26T07:23:10.512 INFO:teuthology.orchestra.run.smithi143.stdout: "tags": [], 2023-11-26T07:23:10.512 INFO:teuthology.orchestra.run.smithi143.stdout: "see_also": [] 2023-11-26T07:23:10.512 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2023-11-26T07:23:10.512 INFO:teuthology.orchestra.run.smithi143.stdout: "container_image_snmp_gateway": { 2023-11-26T07:23:10.513 INFO:teuthology.orchestra.run.smithi143.stdout: "name": "container_image_snmp_gateway", 2023-11-26T07:23:10.513 INFO:teuthology.orchestra.run.smithi143.stdout: "type": "str", 2023-11-26T07:23:10.513 INFO:teuthology.orchestra.run.smithi143.stdout: "level": "advanced", 2023-11-26T07:23:10.513 INFO:teuthology.orchestra.run.smithi143.stdout: "flags": 0, 2023-11-26T07:23:10.513 INFO:teuthology.orchestra.run.smithi143.stdout: "default_value": "docker.io/maxwo/snmp-notifier:v1.2.1", 2023-11-26T07:23:10.513 INFO:teuthology.orchestra.run.smithi143.stdout: "min": "", 2023-11-26T07:23:10.513 INFO:teuthology.orchestra.run.smithi143.stdout: "max": "", 2023-11-26T07:23:10.513 INFO:teuthology.orchestra.run.smithi143.stdout: "enum_allowed": [], 2023-11-26T07:23:10.513 INFO:teuthology.orchestra.run.smithi143.stdout: "desc": "SNMP Gateway container image", 2023-11-26T07:23:10.513 INFO:teuthology.orchestra.run.smithi143.stdout: "long_desc": "", 2023-11-26T07:23:10.513 INFO:teuthology.orchestra.run.smithi143.stdout: "tags": [], 2023-11-26T07:23:10.513 INFO:teuthology.orchestra.run.smithi143.stdout: "see_also": [] 2023-11-26T07:23:10.513 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2023-11-26T07:23:10.514 INFO:teuthology.orchestra.run.smithi143.stdout: "container_init": { 2023-11-26T07:23:10.514 INFO:teuthology.orchestra.run.smithi143.stdout: "name": "container_init", 2023-11-26T07:23:10.514 INFO:teuthology.orchestra.run.smithi143.stdout: "type": "bool", 2023-11-26T07:23:10.514 INFO:teuthology.orchestra.run.smithi143.stdout: "level": "advanced", 2023-11-26T07:23:10.514 INFO:teuthology.orchestra.run.smithi143.stdout: "flags": 0, 2023-11-26T07:23:10.514 INFO:teuthology.orchestra.run.smithi143.stdout: "default_value": "True", 2023-11-26T07:23:10.514 INFO:teuthology.orchestra.run.smithi143.stdout: "min": "", 2023-11-26T07:23:10.514 INFO:teuthology.orchestra.run.smithi143.stdout: "max": "", 2023-11-26T07:23:10.514 INFO:teuthology.orchestra.run.smithi143.stdout: "enum_allowed": [], 2023-11-26T07:23:10.514 INFO:teuthology.orchestra.run.smithi143.stdout: "desc": "Run podman/docker with `--init`", 2023-11-26T07:23:10.514 INFO:teuthology.orchestra.run.smithi143.stdout: "long_desc": "", 2023-11-26T07:23:10.514 INFO:teuthology.orchestra.run.smithi143.stdout: "tags": [], 2023-11-26T07:23:10.515 INFO:teuthology.orchestra.run.smithi143.stdout: "see_also": [] 2023-11-26T07:23:10.515 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2023-11-26T07:23:10.515 INFO:teuthology.orchestra.run.smithi143.stdout: "daemon_cache_timeout": { 2023-11-26T07:23:10.515 INFO:teuthology.orchestra.run.smithi143.stdout: "name": "daemon_cache_timeout", 2023-11-26T07:23:10.515 INFO:teuthology.orchestra.run.smithi143.stdout: "type": "secs", 2023-11-26T07:23:10.515 INFO:teuthology.orchestra.run.smithi143.stdout: "level": "advanced", 2023-11-26T07:23:10.515 INFO:teuthology.orchestra.run.smithi143.stdout: "flags": 0, 2023-11-26T07:23:10.515 INFO:teuthology.orchestra.run.smithi143.stdout: "default_value": "600", 2023-11-26T07:23:10.515 INFO:teuthology.orchestra.run.smithi143.stdout: "min": "", 2023-11-26T07:23:10.515 INFO:teuthology.orchestra.run.smithi143.stdout: "max": "", 2023-11-26T07:23:10.515 INFO:teuthology.orchestra.run.smithi143.stdout: "enum_allowed": [], 2023-11-26T07:23:10.515 INFO:teuthology.orchestra.run.smithi143.stdout: "desc": "seconds to cache service (daemon) inventory", 2023-11-26T07:23:10.515 INFO:teuthology.orchestra.run.smithi143.stdout: "long_desc": "", 2023-11-26T07:23:10.516 INFO:teuthology.orchestra.run.smithi143.stdout: "tags": [], 2023-11-26T07:23:10.516 INFO:teuthology.orchestra.run.smithi143.stdout: "see_also": [] 2023-11-26T07:23:10.516 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2023-11-26T07:23:10.516 INFO:teuthology.orchestra.run.smithi143.stdout: "default_cephadm_command_timeout": { 2023-11-26T07:23:10.516 INFO:teuthology.orchestra.run.smithi143.stdout: "name": "default_cephadm_command_timeout", 2023-11-26T07:23:10.516 INFO:teuthology.orchestra.run.smithi143.stdout: "type": "secs", 2023-11-26T07:23:10.516 INFO:teuthology.orchestra.run.smithi143.stdout: "level": "advanced", 2023-11-26T07:23:10.516 INFO:teuthology.orchestra.run.smithi143.stdout: "flags": 0, 2023-11-26T07:23:10.516 INFO:teuthology.orchestra.run.smithi143.stdout: "default_value": "900", 2023-11-26T07:23:10.516 INFO:teuthology.orchestra.run.smithi143.stdout: "min": "", 2023-11-26T07:23:10.516 INFO:teuthology.orchestra.run.smithi143.stdout: "max": "", 2023-11-26T07:23:10.516 INFO:teuthology.orchestra.run.smithi143.stdout: "enum_allowed": [], 2023-11-26T07:23:10.516 INFO:teuthology.orchestra.run.smithi143.stdout: "desc": "Default timeout applied to cephadm commands run directly on the host (in seconds)", 2023-11-26T07:23:10.517 INFO:teuthology.orchestra.run.smithi143.stdout: "long_desc": "", 2023-11-26T07:23:10.517 INFO:teuthology.orchestra.run.smithi143.stdout: "tags": [], 2023-11-26T07:23:10.517 INFO:teuthology.orchestra.run.smithi143.stdout: "see_also": [] 2023-11-26T07:23:10.517 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2023-11-26T07:23:10.517 INFO:teuthology.orchestra.run.smithi143.stdout: "default_registry": { 2023-11-26T07:23:10.517 INFO:teuthology.orchestra.run.smithi143.stdout: "name": "default_registry", 2023-11-26T07:23:10.517 INFO:teuthology.orchestra.run.smithi143.stdout: "type": "str", 2023-11-26T07:23:10.517 INFO:teuthology.orchestra.run.smithi143.stdout: "level": "advanced", 2023-11-26T07:23:10.517 INFO:teuthology.orchestra.run.smithi143.stdout: "flags": 0, 2023-11-26T07:23:10.517 INFO:teuthology.orchestra.run.smithi143.stdout: "default_value": "docker.io", 2023-11-26T07:23:10.517 INFO:teuthology.orchestra.run.smithi143.stdout: "min": "", 2023-11-26T07:23:10.517 INFO:teuthology.orchestra.run.smithi143.stdout: "max": "", 2023-11-26T07:23:10.517 INFO:teuthology.orchestra.run.smithi143.stdout: "enum_allowed": [], 2023-11-26T07:23:10.518 INFO:teuthology.orchestra.run.smithi143.stdout: "desc": "Search-registry to which we should normalize unqualified image names. This is not the default registry", 2023-11-26T07:23:10.518 INFO:teuthology.orchestra.run.smithi143.stdout: "long_desc": "", 2023-11-26T07:23:10.518 INFO:teuthology.orchestra.run.smithi143.stdout: "tags": [], 2023-11-26T07:23:10.518 INFO:teuthology.orchestra.run.smithi143.stdout: "see_also": [] 2023-11-26T07:23:10.518 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2023-11-26T07:23:10.518 INFO:teuthology.orchestra.run.smithi143.stdout: "device_cache_timeout": { 2023-11-26T07:23:10.518 INFO:teuthology.orchestra.run.smithi143.stdout: "name": "device_cache_timeout", 2023-11-26T07:23:10.518 INFO:teuthology.orchestra.run.smithi143.stdout: "type": "secs", 2023-11-26T07:23:10.518 INFO:teuthology.orchestra.run.smithi143.stdout: "level": "advanced", 2023-11-26T07:23:10.518 INFO:teuthology.orchestra.run.smithi143.stdout: "flags": 0, 2023-11-26T07:23:10.518 INFO:teuthology.orchestra.run.smithi143.stdout: "default_value": "1800", 2023-11-26T07:23:10.518 INFO:teuthology.orchestra.run.smithi143.stdout: "min": "", 2023-11-26T07:23:10.519 INFO:teuthology.orchestra.run.smithi143.stdout: "max": "", 2023-11-26T07:23:10.519 INFO:teuthology.orchestra.run.smithi143.stdout: "enum_allowed": [], 2023-11-26T07:23:10.519 INFO:teuthology.orchestra.run.smithi143.stdout: "desc": "seconds to cache device inventory", 2023-11-26T07:23:10.519 INFO:teuthology.orchestra.run.smithi143.stdout: "long_desc": "", 2023-11-26T07:23:10.519 INFO:teuthology.orchestra.run.smithi143.stdout: "tags": [], 2023-11-26T07:23:10.519 INFO:teuthology.orchestra.run.smithi143.stdout: "see_also": [] 2023-11-26T07:23:10.519 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2023-11-26T07:23:10.519 INFO:teuthology.orchestra.run.smithi143.stdout: "device_enhanced_scan": { 2023-11-26T07:23:10.519 INFO:teuthology.orchestra.run.smithi143.stdout: "name": "device_enhanced_scan", 2023-11-26T07:23:10.519 INFO:teuthology.orchestra.run.smithi143.stdout: "type": "bool", 2023-11-26T07:23:10.519 INFO:teuthology.orchestra.run.smithi143.stdout: "level": "advanced", 2023-11-26T07:23:10.519 INFO:teuthology.orchestra.run.smithi143.stdout: "flags": 0, 2023-11-26T07:23:10.519 INFO:teuthology.orchestra.run.smithi143.stdout: "default_value": "False", 2023-11-26T07:23:10.519 INFO:teuthology.orchestra.run.smithi143.stdout: "min": "", 2023-11-26T07:23:10.520 INFO:teuthology.orchestra.run.smithi143.stdout: "max": "", 2023-11-26T07:23:10.520 INFO:teuthology.orchestra.run.smithi143.stdout: "enum_allowed": [], 2023-11-26T07:23:10.520 INFO:teuthology.orchestra.run.smithi143.stdout: "desc": "Use libstoragemgmt during device scans", 2023-11-26T07:23:10.520 INFO:teuthology.orchestra.run.smithi143.stdout: "long_desc": "", 2023-11-26T07:23:10.520 INFO:teuthology.orchestra.run.smithi143.stdout: "tags": [], 2023-11-26T07:23:10.520 INFO:teuthology.orchestra.run.smithi143.stdout: "see_also": [] 2023-11-26T07:23:10.520 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2023-11-26T07:23:10.520 INFO:teuthology.orchestra.run.smithi143.stdout: "facts_cache_timeout": { 2023-11-26T07:23:10.520 INFO:teuthology.orchestra.run.smithi143.stdout: "name": "facts_cache_timeout", 2023-11-26T07:23:10.520 INFO:teuthology.orchestra.run.smithi143.stdout: "type": "secs", 2023-11-26T07:23:10.520 INFO:teuthology.orchestra.run.smithi143.stdout: "level": "advanced", 2023-11-26T07:23:10.520 INFO:teuthology.orchestra.run.smithi143.stdout: "flags": 0, 2023-11-26T07:23:10.520 INFO:teuthology.orchestra.run.smithi143.stdout: "default_value": "60", 2023-11-26T07:23:10.521 INFO:teuthology.orchestra.run.smithi143.stdout: "min": "", 2023-11-26T07:23:10.521 INFO:teuthology.orchestra.run.smithi143.stdout: "max": "", 2023-11-26T07:23:10.521 INFO:teuthology.orchestra.run.smithi143.stdout: "enum_allowed": [], 2023-11-26T07:23:10.521 INFO:teuthology.orchestra.run.smithi143.stdout: "desc": "seconds to cache host facts data", 2023-11-26T07:23:10.521 INFO:teuthology.orchestra.run.smithi143.stdout: "long_desc": "", 2023-11-26T07:23:10.521 INFO:teuthology.orchestra.run.smithi143.stdout: "tags": [], 2023-11-26T07:23:10.521 INFO:teuthology.orchestra.run.smithi143.stdout: "see_also": [] 2023-11-26T07:23:10.521 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2023-11-26T07:23:10.521 INFO:teuthology.orchestra.run.smithi143.stdout: "host_check_interval": { 2023-11-26T07:23:10.521 INFO:teuthology.orchestra.run.smithi143.stdout: "name": "host_check_interval", 2023-11-26T07:23:10.521 INFO:teuthology.orchestra.run.smithi143.stdout: "type": "secs", 2023-11-26T07:23:10.521 INFO:teuthology.orchestra.run.smithi143.stdout: "level": "advanced", 2023-11-26T07:23:10.521 INFO:teuthology.orchestra.run.smithi143.stdout: "flags": 0, 2023-11-26T07:23:10.522 INFO:teuthology.orchestra.run.smithi143.stdout: "default_value": "600", 2023-11-26T07:23:10.522 INFO:teuthology.orchestra.run.smithi143.stdout: "min": "", 2023-11-26T07:23:10.522 INFO:teuthology.orchestra.run.smithi143.stdout: "max": "", 2023-11-26T07:23:10.522 INFO:teuthology.orchestra.run.smithi143.stdout: "enum_allowed": [], 2023-11-26T07:23:10.522 INFO:teuthology.orchestra.run.smithi143.stdout: "desc": "how frequently to perform a host check", 2023-11-26T07:23:10.522 INFO:teuthology.orchestra.run.smithi143.stdout: "long_desc": "", 2023-11-26T07:23:10.522 INFO:teuthology.orchestra.run.smithi143.stdout: "tags": [], 2023-11-26T07:23:10.522 INFO:teuthology.orchestra.run.smithi143.stdout: "see_also": [] 2023-11-26T07:23:10.522 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2023-11-26T07:23:10.522 INFO:teuthology.orchestra.run.smithi143.stdout: "inventory_list_all": { 2023-11-26T07:23:10.522 INFO:teuthology.orchestra.run.smithi143.stdout: "name": "inventory_list_all", 2023-11-26T07:23:10.522 INFO:teuthology.orchestra.run.smithi143.stdout: "type": "bool", 2023-11-26T07:23:10.523 INFO:teuthology.orchestra.run.smithi143.stdout: "level": "advanced", 2023-11-26T07:23:10.523 INFO:teuthology.orchestra.run.smithi143.stdout: "flags": 0, 2023-11-26T07:23:10.523 INFO:teuthology.orchestra.run.smithi143.stdout: "default_value": "False", 2023-11-26T07:23:10.523 INFO:teuthology.orchestra.run.smithi143.stdout: "min": "", 2023-11-26T07:23:10.523 INFO:teuthology.orchestra.run.smithi143.stdout: "max": "", 2023-11-26T07:23:10.523 INFO:teuthology.orchestra.run.smithi143.stdout: "enum_allowed": [], 2023-11-26T07:23:10.523 INFO:teuthology.orchestra.run.smithi143.stdout: "desc": "Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)", 2023-11-26T07:23:10.523 INFO:teuthology.orchestra.run.smithi143.stdout: "long_desc": "", 2023-11-26T07:23:10.523 INFO:teuthology.orchestra.run.smithi143.stdout: "tags": [], 2023-11-26T07:23:10.523 INFO:teuthology.orchestra.run.smithi143.stdout: "see_also": [] 2023-11-26T07:23:10.523 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2023-11-26T07:23:10.523 INFO:teuthology.orchestra.run.smithi143.stdout: "log_level": { 2023-11-26T07:23:10.523 INFO:teuthology.orchestra.run.smithi143.stdout: "name": "log_level", 2023-11-26T07:23:10.523 INFO:teuthology.orchestra.run.smithi143.stdout: "type": "str", 2023-11-26T07:23:10.524 INFO:teuthology.orchestra.run.smithi143.stdout: "level": "advanced", 2023-11-26T07:23:10.524 INFO:teuthology.orchestra.run.smithi143.stdout: "flags": 1, 2023-11-26T07:23:10.524 INFO:teuthology.orchestra.run.smithi143.stdout: "default_value": "", 2023-11-26T07:23:10.524 INFO:teuthology.orchestra.run.smithi143.stdout: "min": "", 2023-11-26T07:23:10.524 INFO:teuthology.orchestra.run.smithi143.stdout: "max": "", 2023-11-26T07:23:10.524 INFO:teuthology.orchestra.run.smithi143.stdout: "enum_allowed": [ 2023-11-26T07:23:10.524 INFO:teuthology.orchestra.run.smithi143.stdout: "", 2023-11-26T07:23:10.524 INFO:teuthology.orchestra.run.smithi143.stdout: "critical", 2023-11-26T07:23:10.524 INFO:teuthology.orchestra.run.smithi143.stdout: "debug", 2023-11-26T07:23:10.524 INFO:teuthology.orchestra.run.smithi143.stdout: "error", 2023-11-26T07:23:10.524 INFO:teuthology.orchestra.run.smithi143.stdout: "info", 2023-11-26T07:23:10.524 INFO:teuthology.orchestra.run.smithi143.stdout: "warning" 2023-11-26T07:23:10.524 INFO:teuthology.orchestra.run.smithi143.stdout: ], 2023-11-26T07:23:10.524 INFO:teuthology.orchestra.run.smithi143.stdout: "desc": "", 2023-11-26T07:23:10.525 INFO:teuthology.orchestra.run.smithi143.stdout: "long_desc": "", 2023-11-26T07:23:10.525 INFO:teuthology.orchestra.run.smithi143.stdout: "tags": [], 2023-11-26T07:23:10.525 INFO:teuthology.orchestra.run.smithi143.stdout: "see_also": [] 2023-11-26T07:23:10.525 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2023-11-26T07:23:10.525 INFO:teuthology.orchestra.run.smithi143.stdout: "log_refresh_metadata": { 2023-11-26T07:23:10.525 INFO:teuthology.orchestra.run.smithi143.stdout: "name": "log_refresh_metadata", 2023-11-26T07:23:10.525 INFO:teuthology.orchestra.run.smithi143.stdout: "type": "bool", 2023-11-26T07:23:10.525 INFO:teuthology.orchestra.run.smithi143.stdout: "level": "advanced", 2023-11-26T07:23:10.525 INFO:teuthology.orchestra.run.smithi143.stdout: "flags": 0, 2023-11-26T07:23:10.525 INFO:teuthology.orchestra.run.smithi143.stdout: "default_value": "False", 2023-11-26T07:23:10.525 INFO:teuthology.orchestra.run.smithi143.stdout: "min": "", 2023-11-26T07:23:10.525 INFO:teuthology.orchestra.run.smithi143.stdout: "max": "", 2023-11-26T07:23:10.525 INFO:teuthology.orchestra.run.smithi143.stdout: "enum_allowed": [], 2023-11-26T07:23:10.526 INFO:teuthology.orchestra.run.smithi143.stdout: "desc": "Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level", 2023-11-26T07:23:10.526 INFO:teuthology.orchestra.run.smithi143.stdout: "long_desc": "", 2023-11-26T07:23:10.526 INFO:teuthology.orchestra.run.smithi143.stdout: "tags": [], 2023-11-26T07:23:10.526 INFO:teuthology.orchestra.run.smithi143.stdout: "see_also": [] 2023-11-26T07:23:10.526 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2023-11-26T07:23:10.526 INFO:teuthology.orchestra.run.smithi143.stdout: "log_to_cluster": { 2023-11-26T07:23:10.526 INFO:teuthology.orchestra.run.smithi143.stdout: "name": "log_to_cluster", 2023-11-26T07:23:10.526 INFO:teuthology.orchestra.run.smithi143.stdout: "type": "bool", 2023-11-26T07:23:10.526 INFO:teuthology.orchestra.run.smithi143.stdout: "level": "advanced", 2023-11-26T07:23:10.526 INFO:teuthology.orchestra.run.smithi143.stdout: "flags": 0, 2023-11-26T07:23:10.526 INFO:teuthology.orchestra.run.smithi143.stdout: "default_value": "True", 2023-11-26T07:23:10.526 INFO:teuthology.orchestra.run.smithi143.stdout: "min": "", 2023-11-26T07:23:10.527 INFO:teuthology.orchestra.run.smithi143.stdout: "max": "", 2023-11-26T07:23:10.527 INFO:teuthology.orchestra.run.smithi143.stdout: "enum_allowed": [], 2023-11-26T07:23:10.527 INFO:teuthology.orchestra.run.smithi143.stdout: "desc": "log to the \"cephadm\" cluster log channel\"", 2023-11-26T07:23:10.527 INFO:teuthology.orchestra.run.smithi143.stdout: "long_desc": "", 2023-11-26T07:23:10.527 INFO:teuthology.orchestra.run.smithi143.stdout: "tags": [], 2023-11-26T07:23:10.527 INFO:teuthology.orchestra.run.smithi143.stdout: "see_also": [] 2023-11-26T07:23:10.527 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2023-11-26T07:23:10.527 INFO:teuthology.orchestra.run.smithi143.stdout: "log_to_cluster_level": { 2023-11-26T07:23:10.527 INFO:teuthology.orchestra.run.smithi143.stdout: "name": "log_to_cluster_level", 2023-11-26T07:23:10.527 INFO:teuthology.orchestra.run.smithi143.stdout: "type": "str", 2023-11-26T07:23:10.527 INFO:teuthology.orchestra.run.smithi143.stdout: "level": "advanced", 2023-11-26T07:23:10.527 INFO:teuthology.orchestra.run.smithi143.stdout: "flags": 1, 2023-11-26T07:23:10.528 INFO:teuthology.orchestra.run.smithi143.stdout: "default_value": "info", 2023-11-26T07:23:10.528 INFO:teuthology.orchestra.run.smithi143.stdout: "min": "", 2023-11-26T07:23:10.528 INFO:teuthology.orchestra.run.smithi143.stdout: "max": "", 2023-11-26T07:23:10.528 INFO:teuthology.orchestra.run.smithi143.stdout: "enum_allowed": [ 2023-11-26T07:23:10.528 INFO:teuthology.orchestra.run.smithi143.stdout: "", 2023-11-26T07:23:10.528 INFO:teuthology.orchestra.run.smithi143.stdout: "critical", 2023-11-26T07:23:10.528 INFO:teuthology.orchestra.run.smithi143.stdout: "debug", 2023-11-26T07:23:10.528 INFO:teuthology.orchestra.run.smithi143.stdout: "error", 2023-11-26T07:23:10.528 INFO:teuthology.orchestra.run.smithi143.stdout: "info", 2023-11-26T07:23:10.528 INFO:teuthology.orchestra.run.smithi143.stdout: "warning" 2023-11-26T07:23:10.528 INFO:teuthology.orchestra.run.smithi143.stdout: ], 2023-11-26T07:23:10.528 INFO:teuthology.orchestra.run.smithi143.stdout: "desc": "", 2023-11-26T07:23:10.529 INFO:teuthology.orchestra.run.smithi143.stdout: "long_desc": "", 2023-11-26T07:23:10.529 INFO:teuthology.orchestra.run.smithi143.stdout: "tags": [], 2023-11-26T07:23:10.529 INFO:teuthology.orchestra.run.smithi143.stdout: "see_also": [] 2023-11-26T07:23:10.529 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2023-11-26T07:23:10.529 INFO:teuthology.orchestra.run.smithi143.stdout: "log_to_file": { 2023-11-26T07:23:10.529 INFO:teuthology.orchestra.run.smithi143.stdout: "name": "log_to_file", 2023-11-26T07:23:10.529 INFO:teuthology.orchestra.run.smithi143.stdout: "type": "bool", 2023-11-26T07:23:10.529 INFO:teuthology.orchestra.run.smithi143.stdout: "level": "advanced", 2023-11-26T07:23:10.529 INFO:teuthology.orchestra.run.smithi143.stdout: "flags": 1, 2023-11-26T07:23:10.529 INFO:teuthology.orchestra.run.smithi143.stdout: "default_value": "False", 2023-11-26T07:23:10.529 INFO:teuthology.orchestra.run.smithi143.stdout: "min": "", 2023-11-26T07:23:10.529 INFO:teuthology.orchestra.run.smithi143.stdout: "max": "", 2023-11-26T07:23:10.529 INFO:teuthology.orchestra.run.smithi143.stdout: "enum_allowed": [], 2023-11-26T07:23:10.530 INFO:teuthology.orchestra.run.smithi143.stdout: "desc": "", 2023-11-26T07:23:10.530 INFO:teuthology.orchestra.run.smithi143.stdout: "long_desc": "", 2023-11-26T07:23:10.530 INFO:teuthology.orchestra.run.smithi143.stdout: "tags": [], 2023-11-26T07:23:10.530 INFO:teuthology.orchestra.run.smithi143.stdout: "see_also": [] 2023-11-26T07:23:10.530 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2023-11-26T07:23:10.530 INFO:teuthology.orchestra.run.smithi143.stdout: "manage_etc_ceph_ceph_conf": { 2023-11-26T07:23:10.530 INFO:teuthology.orchestra.run.smithi143.stdout: "name": "manage_etc_ceph_ceph_conf", 2023-11-26T07:23:10.530 INFO:teuthology.orchestra.run.smithi143.stdout: "type": "bool", 2023-11-26T07:23:10.530 INFO:teuthology.orchestra.run.smithi143.stdout: "level": "advanced", 2023-11-26T07:23:10.530 INFO:teuthology.orchestra.run.smithi143.stdout: "flags": 0, 2023-11-26T07:23:10.530 INFO:teuthology.orchestra.run.smithi143.stdout: "default_value": "False", 2023-11-26T07:23:10.530 INFO:teuthology.orchestra.run.smithi143.stdout: "min": "", 2023-11-26T07:23:10.531 INFO:teuthology.orchestra.run.smithi143.stdout: "max": "", 2023-11-26T07:23:10.531 INFO:teuthology.orchestra.run.smithi143.stdout: "enum_allowed": [], 2023-11-26T07:23:10.531 INFO:teuthology.orchestra.run.smithi143.stdout: "desc": "Manage and own /etc/ceph/ceph.conf on the hosts.", 2023-11-26T07:23:10.531 INFO:teuthology.orchestra.run.smithi143.stdout: "long_desc": "", 2023-11-26T07:23:10.531 INFO:teuthology.orchestra.run.smithi143.stdout: "tags": [], 2023-11-26T07:23:10.531 INFO:teuthology.orchestra.run.smithi143.stdout: "see_also": [] 2023-11-26T07:23:10.531 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2023-11-26T07:23:10.531 INFO:teuthology.orchestra.run.smithi143.stdout: "manage_etc_ceph_ceph_conf_hosts": { 2023-11-26T07:23:10.531 INFO:teuthology.orchestra.run.smithi143.stdout: "name": "manage_etc_ceph_ceph_conf_hosts", 2023-11-26T07:23:10.531 INFO:teuthology.orchestra.run.smithi143.stdout: "type": "str", 2023-11-26T07:23:10.531 INFO:teuthology.orchestra.run.smithi143.stdout: "level": "advanced", 2023-11-26T07:23:10.531 INFO:teuthology.orchestra.run.smithi143.stdout: "flags": 0, 2023-11-26T07:23:10.531 INFO:teuthology.orchestra.run.smithi143.stdout: "default_value": "*", 2023-11-26T07:23:10.531 INFO:teuthology.orchestra.run.smithi143.stdout: "min": "", 2023-11-26T07:23:10.531 INFO:teuthology.orchestra.run.smithi143.stdout: "max": "", 2023-11-26T07:23:10.532 INFO:teuthology.orchestra.run.smithi143.stdout: "enum_allowed": [], 2023-11-26T07:23:10.532 INFO:teuthology.orchestra.run.smithi143.stdout: "desc": "PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf", 2023-11-26T07:23:10.532 INFO:teuthology.orchestra.run.smithi143.stdout: "long_desc": "", 2023-11-26T07:23:10.532 INFO:teuthology.orchestra.run.smithi143.stdout: "tags": [], 2023-11-26T07:23:10.532 INFO:teuthology.orchestra.run.smithi143.stdout: "see_also": [] 2023-11-26T07:23:10.532 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2023-11-26T07:23:10.532 INFO:teuthology.orchestra.run.smithi143.stdout: "max_count_per_host": { 2023-11-26T07:23:10.532 INFO:teuthology.orchestra.run.smithi143.stdout: "name": "max_count_per_host", 2023-11-26T07:23:10.532 INFO:teuthology.orchestra.run.smithi143.stdout: "type": "int", 2023-11-26T07:23:10.532 INFO:teuthology.orchestra.run.smithi143.stdout: "level": "advanced", 2023-11-26T07:23:10.532 INFO:teuthology.orchestra.run.smithi143.stdout: "flags": 0, 2023-11-26T07:23:10.532 INFO:teuthology.orchestra.run.smithi143.stdout: "default_value": "10", 2023-11-26T07:23:10.532 INFO:teuthology.orchestra.run.smithi143.stdout: "min": "", 2023-11-26T07:23:10.533 INFO:teuthology.orchestra.run.smithi143.stdout: "max": "", 2023-11-26T07:23:10.533 INFO:teuthology.orchestra.run.smithi143.stdout: "enum_allowed": [], 2023-11-26T07:23:10.533 INFO:teuthology.orchestra.run.smithi143.stdout: "desc": "max number of daemons per service per host", 2023-11-26T07:23:10.533 INFO:teuthology.orchestra.run.smithi143.stdout: "long_desc": "", 2023-11-26T07:23:10.533 INFO:teuthology.orchestra.run.smithi143.stdout: "tags": [], 2023-11-26T07:23:10.533 INFO:teuthology.orchestra.run.smithi143.stdout: "see_also": [] 2023-11-26T07:23:10.533 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2023-11-26T07:23:10.533 INFO:teuthology.orchestra.run.smithi143.stdout: "max_osd_draining_count": { 2023-11-26T07:23:10.533 INFO:teuthology.orchestra.run.smithi143.stdout: "name": "max_osd_draining_count", 2023-11-26T07:23:10.533 INFO:teuthology.orchestra.run.smithi143.stdout: "type": "int", 2023-11-26T07:23:10.533 INFO:teuthology.orchestra.run.smithi143.stdout: "level": "advanced", 2023-11-26T07:23:10.533 INFO:teuthology.orchestra.run.smithi143.stdout: "flags": 0, 2023-11-26T07:23:10.533 INFO:teuthology.orchestra.run.smithi143.stdout: "default_value": "10", 2023-11-26T07:23:10.534 INFO:teuthology.orchestra.run.smithi143.stdout: "min": "", 2023-11-26T07:23:10.534 INFO:teuthology.orchestra.run.smithi143.stdout: "max": "", 2023-11-26T07:23:10.534 INFO:teuthology.orchestra.run.smithi143.stdout: "enum_allowed": [], 2023-11-26T07:23:10.534 INFO:teuthology.orchestra.run.smithi143.stdout: "desc": "max number of osds that will be drained simultaneously when osds are removed", 2023-11-26T07:23:10.534 INFO:teuthology.orchestra.run.smithi143.stdout: "long_desc": "", 2023-11-26T07:23:10.534 INFO:teuthology.orchestra.run.smithi143.stdout: "tags": [], 2023-11-26T07:23:10.534 INFO:teuthology.orchestra.run.smithi143.stdout: "see_also": [] 2023-11-26T07:23:10.534 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2023-11-26T07:23:10.534 INFO:teuthology.orchestra.run.smithi143.stdout: "migration_current": { 2023-11-26T07:23:10.534 INFO:teuthology.orchestra.run.smithi143.stdout: "name": "migration_current", 2023-11-26T07:23:10.534 INFO:teuthology.orchestra.run.smithi143.stdout: "type": "int", 2023-11-26T07:23:10.535 INFO:teuthology.orchestra.run.smithi143.stdout: "level": "advanced", 2023-11-26T07:23:10.535 INFO:teuthology.orchestra.run.smithi143.stdout: "flags": 0, 2023-11-26T07:23:10.535 INFO:teuthology.orchestra.run.smithi143.stdout: "default_value": "", 2023-11-26T07:23:10.535 INFO:teuthology.orchestra.run.smithi143.stdout: "min": "", 2023-11-26T07:23:10.535 INFO:teuthology.orchestra.run.smithi143.stdout: "max": "", 2023-11-26T07:23:10.535 INFO:teuthology.orchestra.run.smithi143.stdout: "enum_allowed": [], 2023-11-26T07:23:10.535 INFO:teuthology.orchestra.run.smithi143.stdout: "desc": "internal - do not modify", 2023-11-26T07:23:10.535 INFO:teuthology.orchestra.run.smithi143.stdout: "long_desc": "", 2023-11-26T07:23:10.535 INFO:teuthology.orchestra.run.smithi143.stdout: "tags": [], 2023-11-26T07:23:10.535 INFO:teuthology.orchestra.run.smithi143.stdout: "see_also": [] 2023-11-26T07:23:10.535 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2023-11-26T07:23:10.535 INFO:teuthology.orchestra.run.smithi143.stdout: "mode": { 2023-11-26T07:23:10.535 INFO:teuthology.orchestra.run.smithi143.stdout: "name": "mode", 2023-11-26T07:23:10.536 INFO:teuthology.orchestra.run.smithi143.stdout: "type": "str", 2023-11-26T07:23:10.536 INFO:teuthology.orchestra.run.smithi143.stdout: "level": "advanced", 2023-11-26T07:23:10.536 INFO:teuthology.orchestra.run.smithi143.stdout: "flags": 0, 2023-11-26T07:23:10.536 INFO:teuthology.orchestra.run.smithi143.stdout: "default_value": "root", 2023-11-26T07:23:10.536 INFO:teuthology.orchestra.run.smithi143.stdout: "min": "", 2023-11-26T07:23:10.536 INFO:teuthology.orchestra.run.smithi143.stdout: "max": "", 2023-11-26T07:23:10.536 INFO:teuthology.orchestra.run.smithi143.stdout: "enum_allowed": [ 2023-11-26T07:23:10.536 INFO:teuthology.orchestra.run.smithi143.stdout: "cephadm-package", 2023-11-26T07:23:10.536 INFO:teuthology.orchestra.run.smithi143.stdout: "root" 2023-11-26T07:23:10.536 INFO:teuthology.orchestra.run.smithi143.stdout: ], 2023-11-26T07:23:10.536 INFO:teuthology.orchestra.run.smithi143.stdout: "desc": "mode for remote execution of cephadm", 2023-11-26T07:23:10.536 INFO:teuthology.orchestra.run.smithi143.stdout: "long_desc": "", 2023-11-26T07:23:10.536 INFO:teuthology.orchestra.run.smithi143.stdout: "tags": [], 2023-11-26T07:23:10.537 INFO:teuthology.orchestra.run.smithi143.stdout: "see_also": [] 2023-11-26T07:23:10.537 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2023-11-26T07:23:10.537 INFO:teuthology.orchestra.run.smithi143.stdout: "prometheus_alerts_path": { 2023-11-26T07:23:10.537 INFO:teuthology.orchestra.run.smithi143.stdout: "name": "prometheus_alerts_path", 2023-11-26T07:23:10.537 INFO:teuthology.orchestra.run.smithi143.stdout: "type": "str", 2023-11-26T07:23:10.537 INFO:teuthology.orchestra.run.smithi143.stdout: "level": "advanced", 2023-11-26T07:23:10.537 INFO:teuthology.orchestra.run.smithi143.stdout: "flags": 0, 2023-11-26T07:23:10.537 INFO:teuthology.orchestra.run.smithi143.stdout: "default_value": "/etc/prometheus/ceph/ceph_default_alerts.yml", 2023-11-26T07:23:10.537 INFO:teuthology.orchestra.run.smithi143.stdout: "min": "", 2023-11-26T07:23:10.537 INFO:teuthology.orchestra.run.smithi143.stdout: "max": "", 2023-11-26T07:23:10.537 INFO:teuthology.orchestra.run.smithi143.stdout: "enum_allowed": [], 2023-11-26T07:23:10.537 INFO:teuthology.orchestra.run.smithi143.stdout: "desc": "location of alerts to include in prometheus deployments", 2023-11-26T07:23:10.537 INFO:teuthology.orchestra.run.smithi143.stdout: "long_desc": "", 2023-11-26T07:23:10.538 INFO:teuthology.orchestra.run.smithi143.stdout: "tags": [], 2023-11-26T07:23:10.538 INFO:teuthology.orchestra.run.smithi143.stdout: "see_also": [] 2023-11-26T07:23:10.538 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2023-11-26T07:23:10.538 INFO:teuthology.orchestra.run.smithi143.stdout: "registry_insecure": { 2023-11-26T07:23:10.538 INFO:teuthology.orchestra.run.smithi143.stdout: "name": "registry_insecure", 2023-11-26T07:23:10.538 INFO:teuthology.orchestra.run.smithi143.stdout: "type": "bool", 2023-11-26T07:23:10.538 INFO:teuthology.orchestra.run.smithi143.stdout: "level": "advanced", 2023-11-26T07:23:10.538 INFO:teuthology.orchestra.run.smithi143.stdout: "flags": 0, 2023-11-26T07:23:10.538 INFO:teuthology.orchestra.run.smithi143.stdout: "default_value": "False", 2023-11-26T07:23:10.538 INFO:teuthology.orchestra.run.smithi143.stdout: "min": "", 2023-11-26T07:23:10.538 INFO:teuthology.orchestra.run.smithi143.stdout: "max": "", 2023-11-26T07:23:10.538 INFO:teuthology.orchestra.run.smithi143.stdout: "enum_allowed": [], 2023-11-26T07:23:10.539 INFO:teuthology.orchestra.run.smithi143.stdout: "desc": "Registry is to be considered insecure (no TLS available). Only for development purposes.", 2023-11-26T07:23:10.539 INFO:teuthology.orchestra.run.smithi143.stdout: "long_desc": "", 2023-11-26T07:23:10.539 INFO:teuthology.orchestra.run.smithi143.stdout: "tags": [], 2023-11-26T07:23:10.539 INFO:teuthology.orchestra.run.smithi143.stdout: "see_also": [] 2023-11-26T07:23:10.539 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2023-11-26T07:23:10.539 INFO:teuthology.orchestra.run.smithi143.stdout: "registry_password": { 2023-11-26T07:23:10.539 INFO:teuthology.orchestra.run.smithi143.stdout: "name": "registry_password", 2023-11-26T07:23:10.539 INFO:teuthology.orchestra.run.smithi143.stdout: "type": "str", 2023-11-26T07:23:10.539 INFO:teuthology.orchestra.run.smithi143.stdout: "level": "advanced", 2023-11-26T07:23:10.539 INFO:teuthology.orchestra.run.smithi143.stdout: "flags": 0, 2023-11-26T07:23:10.539 INFO:teuthology.orchestra.run.smithi143.stdout: "default_value": "", 2023-11-26T07:23:10.539 INFO:teuthology.orchestra.run.smithi143.stdout: "min": "", 2023-11-26T07:23:10.539 INFO:teuthology.orchestra.run.smithi143.stdout: "max": "", 2023-11-26T07:23:10.539 INFO:teuthology.orchestra.run.smithi143.stdout: "enum_allowed": [], 2023-11-26T07:23:10.540 INFO:teuthology.orchestra.run.smithi143.stdout: "desc": "Custom repository password. Only used for logging into a registry.", 2023-11-26T07:23:10.540 INFO:teuthology.orchestra.run.smithi143.stdout: "long_desc": "", 2023-11-26T07:23:10.540 INFO:teuthology.orchestra.run.smithi143.stdout: "tags": [], 2023-11-26T07:23:10.540 INFO:teuthology.orchestra.run.smithi143.stdout: "see_also": [] 2023-11-26T07:23:10.540 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2023-11-26T07:23:10.540 INFO:teuthology.orchestra.run.smithi143.stdout: "registry_url": { 2023-11-26T07:23:10.540 INFO:teuthology.orchestra.run.smithi143.stdout: "name": "registry_url", 2023-11-26T07:23:10.540 INFO:teuthology.orchestra.run.smithi143.stdout: "type": "str", 2023-11-26T07:23:10.540 INFO:teuthology.orchestra.run.smithi143.stdout: "level": "advanced", 2023-11-26T07:23:10.540 INFO:teuthology.orchestra.run.smithi143.stdout: "flags": 0, 2023-11-26T07:23:10.540 INFO:teuthology.orchestra.run.smithi143.stdout: "default_value": "", 2023-11-26T07:23:10.541 INFO:teuthology.orchestra.run.smithi143.stdout: "min": "", 2023-11-26T07:23:10.541 INFO:teuthology.orchestra.run.smithi143.stdout: "max": "", 2023-11-26T07:23:10.541 INFO:teuthology.orchestra.run.smithi143.stdout: "enum_allowed": [], 2023-11-26T07:23:10.541 INFO:teuthology.orchestra.run.smithi143.stdout: "desc": "Registry url for login purposes. This is not the default registry", 2023-11-26T07:23:10.541 INFO:teuthology.orchestra.run.smithi143.stdout: "long_desc": "", 2023-11-26T07:23:10.541 INFO:teuthology.orchestra.run.smithi143.stdout: "tags": [], 2023-11-26T07:23:10.541 INFO:teuthology.orchestra.run.smithi143.stdout: "see_also": [] 2023-11-26T07:23:10.541 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2023-11-26T07:23:10.541 INFO:teuthology.orchestra.run.smithi143.stdout: "registry_username": { 2023-11-26T07:23:10.541 INFO:teuthology.orchestra.run.smithi143.stdout: "name": "registry_username", 2023-11-26T07:23:10.541 INFO:teuthology.orchestra.run.smithi143.stdout: "type": "str", 2023-11-26T07:23:10.541 INFO:teuthology.orchestra.run.smithi143.stdout: "level": "advanced", 2023-11-26T07:23:10.541 INFO:teuthology.orchestra.run.smithi143.stdout: "flags": 0, 2023-11-26T07:23:10.542 INFO:teuthology.orchestra.run.smithi143.stdout: "default_value": "", 2023-11-26T07:23:10.542 INFO:teuthology.orchestra.run.smithi143.stdout: "min": "", 2023-11-26T07:23:10.542 INFO:teuthology.orchestra.run.smithi143.stdout: "max": "", 2023-11-26T07:23:10.542 INFO:teuthology.orchestra.run.smithi143.stdout: "enum_allowed": [], 2023-11-26T07:23:10.542 INFO:teuthology.orchestra.run.smithi143.stdout: "desc": "Custom repository username. Only used for logging into a registry.", 2023-11-26T07:23:10.542 INFO:teuthology.orchestra.run.smithi143.stdout: "long_desc": "", 2023-11-26T07:23:10.542 INFO:teuthology.orchestra.run.smithi143.stdout: "tags": [], 2023-11-26T07:23:10.542 INFO:teuthology.orchestra.run.smithi143.stdout: "see_also": [] 2023-11-26T07:23:10.542 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2023-11-26T07:23:10.542 INFO:teuthology.orchestra.run.smithi143.stdout: "secure_monitoring_stack": { 2023-11-26T07:23:10.542 INFO:teuthology.orchestra.run.smithi143.stdout: "name": "secure_monitoring_stack", 2023-11-26T07:23:10.542 INFO:teuthology.orchestra.run.smithi143.stdout: "type": "bool", 2023-11-26T07:23:10.543 INFO:teuthology.orchestra.run.smithi143.stdout: "level": "advanced", 2023-11-26T07:23:10.543 INFO:teuthology.orchestra.run.smithi143.stdout: "flags": 0, 2023-11-26T07:23:10.543 INFO:teuthology.orchestra.run.smithi143.stdout: "default_value": "False", 2023-11-26T07:23:10.543 INFO:teuthology.orchestra.run.smithi143.stdout: "min": "", 2023-11-26T07:23:10.543 INFO:teuthology.orchestra.run.smithi143.stdout: "max": "", 2023-11-26T07:23:10.543 INFO:teuthology.orchestra.run.smithi143.stdout: "enum_allowed": [], 2023-11-26T07:23:10.543 INFO:teuthology.orchestra.run.smithi143.stdout: "desc": "Enable TLS security for all the monitoring stack daemons", 2023-11-26T07:23:10.543 INFO:teuthology.orchestra.run.smithi143.stdout: "long_desc": "", 2023-11-26T07:23:10.543 INFO:teuthology.orchestra.run.smithi143.stdout: "tags": [], 2023-11-26T07:23:10.543 INFO:teuthology.orchestra.run.smithi143.stdout: "see_also": [] 2023-11-26T07:23:10.543 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2023-11-26T07:23:10.543 INFO:teuthology.orchestra.run.smithi143.stdout: "service_discovery_port": { 2023-11-26T07:23:10.544 INFO:teuthology.orchestra.run.smithi143.stdout: "name": "service_discovery_port", 2023-11-26T07:23:10.544 INFO:teuthology.orchestra.run.smithi143.stdout: "type": "int", 2023-11-26T07:23:10.544 INFO:teuthology.orchestra.run.smithi143.stdout: "level": "advanced", 2023-11-26T07:23:10.544 INFO:teuthology.orchestra.run.smithi143.stdout: "flags": 0, 2023-11-26T07:23:10.544 INFO:teuthology.orchestra.run.smithi143.stdout: "default_value": "8765", 2023-11-26T07:23:10.544 INFO:teuthology.orchestra.run.smithi143.stdout: "min": "", 2023-11-26T07:23:10.544 INFO:teuthology.orchestra.run.smithi143.stdout: "max": "", 2023-11-26T07:23:10.544 INFO:teuthology.orchestra.run.smithi143.stdout: "enum_allowed": [], 2023-11-26T07:23:10.544 INFO:teuthology.orchestra.run.smithi143.stdout: "desc": "cephadm service discovery port", 2023-11-26T07:23:10.544 INFO:teuthology.orchestra.run.smithi143.stdout: "long_desc": "", 2023-11-26T07:23:10.544 INFO:teuthology.orchestra.run.smithi143.stdout: "tags": [], 2023-11-26T07:23:10.544 INFO:teuthology.orchestra.run.smithi143.stdout: "see_also": [] 2023-11-26T07:23:10.544 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2023-11-26T07:23:10.545 INFO:teuthology.orchestra.run.smithi143.stdout: "ssh_config_file": { 2023-11-26T07:23:10.545 INFO:teuthology.orchestra.run.smithi143.stdout: "name": "ssh_config_file", 2023-11-26T07:23:10.545 INFO:teuthology.orchestra.run.smithi143.stdout: "type": "str", 2023-11-26T07:23:10.545 INFO:teuthology.orchestra.run.smithi143.stdout: "level": "advanced", 2023-11-26T07:23:10.545 INFO:teuthology.orchestra.run.smithi143.stdout: "flags": 0, 2023-11-26T07:23:10.545 INFO:teuthology.orchestra.run.smithi143.stdout: "default_value": "", 2023-11-26T07:23:10.545 INFO:teuthology.orchestra.run.smithi143.stdout: "min": "", 2023-11-26T07:23:10.545 INFO:teuthology.orchestra.run.smithi143.stdout: "max": "", 2023-11-26T07:23:10.545 INFO:teuthology.orchestra.run.smithi143.stdout: "enum_allowed": [], 2023-11-26T07:23:10.545 INFO:teuthology.orchestra.run.smithi143.stdout: "desc": "customized SSH config file to connect to managed hosts", 2023-11-26T07:23:10.545 INFO:teuthology.orchestra.run.smithi143.stdout: "long_desc": "", 2023-11-26T07:23:10.545 INFO:teuthology.orchestra.run.smithi143.stdout: "tags": [], 2023-11-26T07:23:10.545 INFO:teuthology.orchestra.run.smithi143.stdout: "see_also": [] 2023-11-26T07:23:10.546 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2023-11-26T07:23:10.546 INFO:teuthology.orchestra.run.smithi143.stdout: "use_agent": { 2023-11-26T07:23:10.546 INFO:teuthology.orchestra.run.smithi143.stdout: "name": "use_agent", 2023-11-26T07:23:10.546 INFO:teuthology.orchestra.run.smithi143.stdout: "type": "bool", 2023-11-26T07:23:10.546 INFO:teuthology.orchestra.run.smithi143.stdout: "level": "advanced", 2023-11-26T07:23:10.546 INFO:teuthology.orchestra.run.smithi143.stdout: "flags": 0, 2023-11-26T07:23:10.546 INFO:teuthology.orchestra.run.smithi143.stdout: "default_value": "False", 2023-11-26T07:23:10.546 INFO:teuthology.orchestra.run.smithi143.stdout: "min": "", 2023-11-26T07:23:10.546 INFO:teuthology.orchestra.run.smithi143.stdout: "max": "", 2023-11-26T07:23:10.546 INFO:teuthology.orchestra.run.smithi143.stdout: "enum_allowed": [], 2023-11-26T07:23:10.546 INFO:teuthology.orchestra.run.smithi143.stdout: "desc": "Use cephadm agent on each host to gather and send metadata", 2023-11-26T07:23:10.546 INFO:teuthology.orchestra.run.smithi143.stdout: "long_desc": "", 2023-11-26T07:23:10.546 INFO:teuthology.orchestra.run.smithi143.stdout: "tags": [], 2023-11-26T07:23:10.547 INFO:teuthology.orchestra.run.smithi143.stdout: "see_also": [] 2023-11-26T07:23:10.547 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2023-11-26T07:23:10.547 INFO:teuthology.orchestra.run.smithi143.stdout: "use_repo_digest": { 2023-11-26T07:23:10.547 INFO:teuthology.orchestra.run.smithi143.stdout: "name": "use_repo_digest", 2023-11-26T07:23:10.547 INFO:teuthology.orchestra.run.smithi143.stdout: "type": "bool", 2023-11-26T07:23:10.547 INFO:teuthology.orchestra.run.smithi143.stdout: "level": "advanced", 2023-11-26T07:23:10.547 INFO:teuthology.orchestra.run.smithi143.stdout: "flags": 0, 2023-11-26T07:23:10.547 INFO:teuthology.orchestra.run.smithi143.stdout: "default_value": "True", 2023-11-26T07:23:10.547 INFO:teuthology.orchestra.run.smithi143.stdout: "min": "", 2023-11-26T07:23:10.547 INFO:teuthology.orchestra.run.smithi143.stdout: "max": "", 2023-11-26T07:23:10.547 INFO:teuthology.orchestra.run.smithi143.stdout: "enum_allowed": [], 2023-11-26T07:23:10.547 INFO:teuthology.orchestra.run.smithi143.stdout: "desc": "Automatically convert image tags to image digest. Make sure all daemons use the same image", 2023-11-26T07:23:10.547 INFO:teuthology.orchestra.run.smithi143.stdout: "long_desc": "", 2023-11-26T07:23:10.548 INFO:teuthology.orchestra.run.smithi143.stdout: "tags": [], 2023-11-26T07:23:10.548 INFO:teuthology.orchestra.run.smithi143.stdout: "see_also": [] 2023-11-26T07:23:10.548 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2023-11-26T07:23:10.548 INFO:teuthology.orchestra.run.smithi143.stdout: "warn_on_failed_host_check": { 2023-11-26T07:23:10.548 INFO:teuthology.orchestra.run.smithi143.stdout: "name": "warn_on_failed_host_check", 2023-11-26T07:23:10.548 INFO:teuthology.orchestra.run.smithi143.stdout: "type": "bool", 2023-11-26T07:23:10.548 INFO:teuthology.orchestra.run.smithi143.stdout: "level": "advanced", 2023-11-26T07:23:10.548 INFO:teuthology.orchestra.run.smithi143.stdout: "flags": 0, 2023-11-26T07:23:10.548 INFO:teuthology.orchestra.run.smithi143.stdout: "default_value": "True", 2023-11-26T07:23:10.548 INFO:teuthology.orchestra.run.smithi143.stdout: "min": "", 2023-11-26T07:23:10.548 INFO:teuthology.orchestra.run.smithi143.stdout: "max": "", 2023-11-26T07:23:10.548 INFO:teuthology.orchestra.run.smithi143.stdout: "enum_allowed": [], 2023-11-26T07:23:10.548 INFO:teuthology.orchestra.run.smithi143.stdout: "desc": "raise a health warning if the host check fails", 2023-11-26T07:23:10.549 INFO:teuthology.orchestra.run.smithi143.stdout: "long_desc": "", 2023-11-26T07:23:10.549 INFO:teuthology.orchestra.run.smithi143.stdout: "tags": [], 2023-11-26T07:23:10.549 INFO:teuthology.orchestra.run.smithi143.stdout: "see_also": [] 2023-11-26T07:23:10.549 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2023-11-26T07:23:10.549 INFO:teuthology.orchestra.run.smithi143.stdout: "warn_on_stray_daemons": { 2023-11-26T07:23:10.549 INFO:teuthology.orchestra.run.smithi143.stdout: "name": "warn_on_stray_daemons", 2023-11-26T07:23:10.549 INFO:teuthology.orchestra.run.smithi143.stdout: "type": "bool", 2023-11-26T07:23:10.549 INFO:teuthology.orchestra.run.smithi143.stdout: "level": "advanced", 2023-11-26T07:23:10.549 INFO:teuthology.orchestra.run.smithi143.stdout: "flags": 0, 2023-11-26T07:23:10.549 INFO:teuthology.orchestra.run.smithi143.stdout: "default_value": "True", 2023-11-26T07:23:10.549 INFO:teuthology.orchestra.run.smithi143.stdout: "min": "", 2023-11-26T07:23:10.549 INFO:teuthology.orchestra.run.smithi143.stdout: "max": "", 2023-11-26T07:23:10.549 INFO:teuthology.orchestra.run.smithi143.stdout: "enum_allowed": [], 2023-11-26T07:23:10.550 INFO:teuthology.orchestra.run.smithi143.stdout: "desc": "raise a health warning if daemons are detected that are not managed by cephadm", 2023-11-26T07:23:10.550 INFO:teuthology.orchestra.run.smithi143.stdout: "long_desc": "", 2023-11-26T07:23:10.550 INFO:teuthology.orchestra.run.smithi143.stdout: "tags": [], 2023-11-26T07:23:10.550 INFO:teuthology.orchestra.run.smithi143.stdout: "see_also": [] 2023-11-26T07:23:10.550 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2023-11-26T07:23:10.550 INFO:teuthology.orchestra.run.smithi143.stdout: "warn_on_stray_hosts": { 2023-11-26T07:23:10.550 INFO:teuthology.orchestra.run.smithi143.stdout: "name": "warn_on_stray_hosts", 2023-11-26T07:23:10.550 INFO:teuthology.orchestra.run.smithi143.stdout: "type": "bool", 2023-11-26T07:23:10.550 INFO:teuthology.orchestra.run.smithi143.stdout: "level": "advanced", 2023-11-26T07:23:10.550 INFO:teuthology.orchestra.run.smithi143.stdout: "flags": 0, 2023-11-26T07:23:10.550 INFO:teuthology.orchestra.run.smithi143.stdout: "default_value": "True", 2023-11-26T07:23:10.550 INFO:teuthology.orchestra.run.smithi143.stdout: "min": "", 2023-11-26T07:23:10.550 INFO:teuthology.orchestra.run.smithi143.stdout: "max": "", 2023-11-26T07:23:10.551 INFO:teuthology.orchestra.run.smithi143.stdout: "enum_allowed": [], 2023-11-26T07:23:10.551 INFO:teuthology.orchestra.run.smithi143.stdout: "desc": "raise a health warning if daemons are detected on a host that is not managed by cephadm", 2023-11-26T07:23:10.551 INFO:teuthology.orchestra.run.smithi143.stdout: "long_desc": "", 2023-11-26T07:23:10.551 INFO:teuthology.orchestra.run.smithi143.stdout: "tags": [], 2023-11-26T07:23:10.551 INFO:teuthology.orchestra.run.smithi143.stdout: "see_also": [] 2023-11-26T07:23:10.551 INFO:teuthology.orchestra.run.smithi143.stdout: } 2023-11-26T07:23:10.551 INFO:teuthology.orchestra.run.smithi143.stdout: } 2023-11-26T07:23:10.551 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2023-11-26T07:23:10.551 INFO:teuthology.orchestra.run.smithi143.stdout: { 2023-11-26T07:23:10.551 INFO:teuthology.orchestra.run.smithi143.stdout: "name": "crash", 2023-11-26T07:23:10.551 INFO:teuthology.orchestra.run.smithi143.stdout: "can_run": true, 2023-11-26T07:23:10.551 INFO:teuthology.orchestra.run.smithi143.stdout: "error_string": "", 2023-11-26T07:23:10.551 INFO:teuthology.orchestra.run.smithi143.stdout: "module_options": { 2023-11-26T07:23:10.552 INFO:teuthology.orchestra.run.smithi143.stdout: "log_level": { 2023-11-26T07:23:10.552 INFO:teuthology.orchestra.run.smithi143.stdout: "name": "log_level", 2023-11-26T07:23:10.552 INFO:teuthology.orchestra.run.smithi143.stdout: "type": "str", 2023-11-26T07:23:10.552 INFO:teuthology.orchestra.run.smithi143.stdout: "level": "advanced", 2023-11-26T07:23:10.552 INFO:teuthology.orchestra.run.smithi143.stdout: "flags": 1, 2023-11-26T07:23:10.552 INFO:teuthology.orchestra.run.smithi143.stdout: "default_value": "", 2023-11-26T07:23:10.552 INFO:teuthology.orchestra.run.smithi143.stdout: "min": "", 2023-11-26T07:23:10.552 INFO:teuthology.orchestra.run.smithi143.stdout: "max": "", 2023-11-26T07:23:10.552 INFO:teuthology.orchestra.run.smithi143.stdout: "enum_allowed": [ 2023-11-26T07:23:10.552 INFO:teuthology.orchestra.run.smithi143.stdout: "", 2023-11-26T07:23:10.552 INFO:teuthology.orchestra.run.smithi143.stdout: "critical", 2023-11-26T07:23:10.552 INFO:teuthology.orchestra.run.smithi143.stdout: "debug", 2023-11-26T07:23:10.552 INFO:teuthology.orchestra.run.smithi143.stdout: "error", 2023-11-26T07:23:10.553 INFO:teuthology.orchestra.run.smithi143.stdout: "info", 2023-11-26T07:23:10.553 INFO:teuthology.orchestra.run.smithi143.stdout: "warning" 2023-11-26T07:23:10.553 INFO:teuthology.orchestra.run.smithi143.stdout: ], 2023-11-26T07:23:10.553 INFO:teuthology.orchestra.run.smithi143.stdout: "desc": "", 2023-11-26T07:23:10.553 INFO:teuthology.orchestra.run.smithi143.stdout: "long_desc": "", 2023-11-26T07:23:10.553 INFO:teuthology.orchestra.run.smithi143.stdout: "tags": [], 2023-11-26T07:23:10.553 INFO:teuthology.orchestra.run.smithi143.stdout: "see_also": [] 2023-11-26T07:23:10.553 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2023-11-26T07:23:10.553 INFO:teuthology.orchestra.run.smithi143.stdout: "log_to_cluster": { 2023-11-26T07:23:10.553 INFO:teuthology.orchestra.run.smithi143.stdout: "name": "log_to_cluster", 2023-11-26T07:23:10.553 INFO:teuthology.orchestra.run.smithi143.stdout: "type": "bool", 2023-11-26T07:23:10.553 INFO:teuthology.orchestra.run.smithi143.stdout: "level": "advanced", 2023-11-26T07:23:10.554 INFO:teuthology.orchestra.run.smithi143.stdout: "flags": 1, 2023-11-26T07:23:10.554 INFO:teuthology.orchestra.run.smithi143.stdout: "default_value": "False", 2023-11-26T07:23:10.554 INFO:teuthology.orchestra.run.smithi143.stdout: "min": "", 2023-11-26T07:23:10.554 INFO:teuthology.orchestra.run.smithi143.stdout: "max": "", 2023-11-26T07:23:10.554 INFO:teuthology.orchestra.run.smithi143.stdout: "enum_allowed": [], 2023-11-26T07:23:10.554 INFO:teuthology.orchestra.run.smithi143.stdout: "desc": "", 2023-11-26T07:23:10.554 INFO:teuthology.orchestra.run.smithi143.stdout: "long_desc": "", 2023-11-26T07:23:10.554 INFO:teuthology.orchestra.run.smithi143.stdout: "tags": [], 2023-11-26T07:23:10.554 INFO:teuthology.orchestra.run.smithi143.stdout: "see_also": [] 2023-11-26T07:23:10.554 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2023-11-26T07:23:10.554 INFO:teuthology.orchestra.run.smithi143.stdout: "log_to_cluster_level": { 2023-11-26T07:23:10.554 INFO:teuthology.orchestra.run.smithi143.stdout: "name": "log_to_cluster_level", 2023-11-26T07:23:10.554 INFO:teuthology.orchestra.run.smithi143.stdout: "type": "str", 2023-11-26T07:23:10.555 INFO:teuthology.orchestra.run.smithi143.stdout: "level": "advanced", 2023-11-26T07:23:10.555 INFO:teuthology.orchestra.run.smithi143.stdout: "flags": 1, 2023-11-26T07:23:10.555 INFO:teuthology.orchestra.run.smithi143.stdout: "default_value": "info", 2023-11-26T07:23:10.555 INFO:teuthology.orchestra.run.smithi143.stdout: "min": "", 2023-11-26T07:23:10.555 INFO:teuthology.orchestra.run.smithi143.stdout: "max": "", 2023-11-26T07:23:10.555 INFO:teuthology.orchestra.run.smithi143.stdout: "enum_allowed": [ 2023-11-26T07:23:10.555 INFO:teuthology.orchestra.run.smithi143.stdout: "", 2023-11-26T07:23:10.555 INFO:teuthology.orchestra.run.smithi143.stdout: "critical", 2023-11-26T07:23:10.555 INFO:teuthology.orchestra.run.smithi143.stdout: "debug", 2023-11-26T07:23:10.555 INFO:teuthology.orchestra.run.smithi143.stdout: "error", 2023-11-26T07:23:10.555 INFO:teuthology.orchestra.run.smithi143.stdout: "info", 2023-11-26T07:23:10.555 INFO:teuthology.orchestra.run.smithi143.stdout: "warning" 2023-11-26T07:23:10.556 INFO:teuthology.orchestra.run.smithi143.stdout: ], 2023-11-26T07:23:10.556 INFO:teuthology.orchestra.run.smithi143.stdout: "desc": "", 2023-11-26T07:23:10.556 INFO:teuthology.orchestra.run.smithi143.stdout: "long_desc": "", 2023-11-26T07:23:10.556 INFO:teuthology.orchestra.run.smithi143.stdout: "tags": [], 2023-11-26T07:23:10.556 INFO:teuthology.orchestra.run.smithi143.stdout: "see_also": [] 2023-11-26T07:23:10.556 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2023-11-26T07:23:10.556 INFO:teuthology.orchestra.run.smithi143.stdout: "log_to_file": { 2023-11-26T07:23:10.556 INFO:teuthology.orchestra.run.smithi143.stdout: "name": "log_to_file", 2023-11-26T07:23:10.556 INFO:teuthology.orchestra.run.smithi143.stdout: "type": "bool", 2023-11-26T07:23:10.556 INFO:teuthology.orchestra.run.smithi143.stdout: "level": "advanced", 2023-11-26T07:23:10.556 INFO:teuthology.orchestra.run.smithi143.stdout: "flags": 1, 2023-11-26T07:23:10.556 INFO:teuthology.orchestra.run.smithi143.stdout: "default_value": "False", 2023-11-26T07:23:10.557 INFO:teuthology.orchestra.run.smithi143.stdout: "min": "", 2023-11-26T07:23:10.557 INFO:teuthology.orchestra.run.smithi143.stdout: "max": "", 2023-11-26T07:23:10.557 INFO:teuthology.orchestra.run.smithi143.stdout: "enum_allowed": [], 2023-11-26T07:23:10.557 INFO:teuthology.orchestra.run.smithi143.stdout: "desc": "", 2023-11-26T07:23:10.557 INFO:teuthology.orchestra.run.smithi143.stdout: "long_desc": "", 2023-11-26T07:23:10.557 INFO:teuthology.orchestra.run.smithi143.stdout: "tags": [], 2023-11-26T07:23:10.557 INFO:teuthology.orchestra.run.smithi143.stdout: "see_also": [] 2023-11-26T07:23:10.557 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2023-11-26T07:23:10.557 INFO:teuthology.orchestra.run.smithi143.stdout: "retain_interval": { 2023-11-26T07:23:10.557 INFO:teuthology.orchestra.run.smithi143.stdout: "name": "retain_interval", 2023-11-26T07:23:10.557 INFO:teuthology.orchestra.run.smithi143.stdout: "type": "secs", 2023-11-26T07:23:10.557 INFO:teuthology.orchestra.run.smithi143.stdout: "level": "advanced", 2023-11-26T07:23:10.557 INFO:teuthology.orchestra.run.smithi143.stdout: "flags": 1, 2023-11-26T07:23:10.558 INFO:teuthology.orchestra.run.smithi143.stdout: "default_value": "31536000", 2023-11-26T07:23:10.558 INFO:teuthology.orchestra.run.smithi143.stdout: "min": "", 2023-11-26T07:23:10.558 INFO:teuthology.orchestra.run.smithi143.stdout: "max": "", 2023-11-26T07:23:10.558 INFO:teuthology.orchestra.run.smithi143.stdout: "enum_allowed": [], 2023-11-26T07:23:10.558 INFO:teuthology.orchestra.run.smithi143.stdout: "desc": "how long to retain crashes before pruning them", 2023-11-26T07:23:10.558 INFO:teuthology.orchestra.run.smithi143.stdout: "long_desc": "", 2023-11-26T07:23:10.558 INFO:teuthology.orchestra.run.smithi143.stdout: "tags": [], 2023-11-26T07:23:10.558 INFO:teuthology.orchestra.run.smithi143.stdout: "see_also": [] 2023-11-26T07:23:10.558 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2023-11-26T07:23:10.558 INFO:teuthology.orchestra.run.smithi143.stdout: "warn_recent_interval": { 2023-11-26T07:23:10.558 INFO:teuthology.orchestra.run.smithi143.stdout: "name": "warn_recent_interval", 2023-11-26T07:23:10.559 INFO:teuthology.orchestra.run.smithi143.stdout: "type": "secs", 2023-11-26T07:23:10.559 INFO:teuthology.orchestra.run.smithi143.stdout: "level": "advanced", 2023-11-26T07:23:10.559 INFO:teuthology.orchestra.run.smithi143.stdout: "flags": 1, 2023-11-26T07:23:10.559 INFO:teuthology.orchestra.run.smithi143.stdout: "default_value": "1209600", 2023-11-26T07:23:10.559 INFO:teuthology.orchestra.run.smithi143.stdout: "min": "", 2023-11-26T07:23:10.559 INFO:teuthology.orchestra.run.smithi143.stdout: "max": "", 2023-11-26T07:23:10.559 INFO:teuthology.orchestra.run.smithi143.stdout: "enum_allowed": [], 2023-11-26T07:23:10.559 INFO:teuthology.orchestra.run.smithi143.stdout: "desc": "time interval in which to warn about recent crashes", 2023-11-26T07:23:10.559 INFO:teuthology.orchestra.run.smithi143.stdout: "long_desc": "", 2023-11-26T07:23:10.559 INFO:teuthology.orchestra.run.smithi143.stdout: "tags": [], 2023-11-26T07:23:10.559 INFO:teuthology.orchestra.run.smithi143.stdout: "see_also": [] 2023-11-26T07:23:10.559 INFO:teuthology.orchestra.run.smithi143.stdout: } 2023-11-26T07:23:10.559 INFO:teuthology.orchestra.run.smithi143.stdout: } 2023-11-26T07:23:10.559 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2023-11-26T07:23:10.560 INFO:teuthology.orchestra.run.smithi143.stdout: { 2023-11-26T07:23:10.560 INFO:teuthology.orchestra.run.smithi143.stdout: "name": "dashboard", 2023-11-26T07:23:10.560 INFO:teuthology.orchestra.run.smithi143.stdout: "can_run": true, 2023-11-26T07:23:10.560 INFO:teuthology.orchestra.run.smithi143.stdout: "error_string": "", 2023-11-26T07:23:10.560 INFO:teuthology.orchestra.run.smithi143.stdout: "module_options": { 2023-11-26T07:23:10.560 INFO:teuthology.orchestra.run.smithi143.stdout: "ACCOUNT_LOCKOUT_ATTEMPTS": { 2023-11-26T07:23:10.560 INFO:teuthology.orchestra.run.smithi143.stdout: "name": "ACCOUNT_LOCKOUT_ATTEMPTS", 2023-11-26T07:23:10.560 INFO:teuthology.orchestra.run.smithi143.stdout: "type": "int", 2023-11-26T07:23:10.560 INFO:teuthology.orchestra.run.smithi143.stdout: "level": "advanced", 2023-11-26T07:23:10.560 INFO:teuthology.orchestra.run.smithi143.stdout: "flags": 0, 2023-11-26T07:23:10.560 INFO:teuthology.orchestra.run.smithi143.stdout: "default_value": "10", 2023-11-26T07:23:10.560 INFO:teuthology.orchestra.run.smithi143.stdout: "min": "", 2023-11-26T07:23:10.560 INFO:teuthology.orchestra.run.smithi143.stdout: "max": "", 2023-11-26T07:23:10.561 INFO:teuthology.orchestra.run.smithi143.stdout: "enum_allowed": [], 2023-11-26T07:23:10.561 INFO:teuthology.orchestra.run.smithi143.stdout: "desc": "", 2023-11-26T07:23:10.561 INFO:teuthology.orchestra.run.smithi143.stdout: "long_desc": "", 2023-11-26T07:23:10.561 INFO:teuthology.orchestra.run.smithi143.stdout: "tags": [], 2023-11-26T07:23:10.561 INFO:teuthology.orchestra.run.smithi143.stdout: "see_also": [] 2023-11-26T07:23:10.561 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2023-11-26T07:23:10.561 INFO:teuthology.orchestra.run.smithi143.stdout: "ALERTMANAGER_API_HOST": { 2023-11-26T07:23:10.561 INFO:teuthology.orchestra.run.smithi143.stdout: "name": "ALERTMANAGER_API_HOST", 2023-11-26T07:23:10.561 INFO:teuthology.orchestra.run.smithi143.stdout: "type": "str", 2023-11-26T07:23:10.561 INFO:teuthology.orchestra.run.smithi143.stdout: "level": "advanced", 2023-11-26T07:23:10.561 INFO:teuthology.orchestra.run.smithi143.stdout: "flags": 0, 2023-11-26T07:23:10.561 INFO:teuthology.orchestra.run.smithi143.stdout: "default_value": "", 2023-11-26T07:23:10.561 INFO:teuthology.orchestra.run.smithi143.stdout: "min": "", 2023-11-26T07:23:10.562 INFO:teuthology.orchestra.run.smithi143.stdout: "max": "", 2023-11-26T07:23:10.562 INFO:teuthology.orchestra.run.smithi143.stdout: "enum_allowed": [], 2023-11-26T07:23:10.562 INFO:teuthology.orchestra.run.smithi143.stdout: "desc": "", 2023-11-26T07:23:10.562 INFO:teuthology.orchestra.run.smithi143.stdout: "long_desc": "", 2023-11-26T07:23:10.562 INFO:teuthology.orchestra.run.smithi143.stdout: "tags": [], 2023-11-26T07:23:10.562 INFO:teuthology.orchestra.run.smithi143.stdout: "see_also": [] 2023-11-26T07:23:10.562 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2023-11-26T07:23:10.562 INFO:teuthology.orchestra.run.smithi143.stdout: "ALERTMANAGER_API_SSL_VERIFY": { 2023-11-26T07:23:10.562 INFO:teuthology.orchestra.run.smithi143.stdout: "name": "ALERTMANAGER_API_SSL_VERIFY", 2023-11-26T07:23:10.562 INFO:teuthology.orchestra.run.smithi143.stdout: "type": "bool", 2023-11-26T07:23:10.562 INFO:teuthology.orchestra.run.smithi143.stdout: "level": "advanced", 2023-11-26T07:23:10.562 INFO:teuthology.orchestra.run.smithi143.stdout: "flags": 0, 2023-11-26T07:23:10.562 INFO:teuthology.orchestra.run.smithi143.stdout: "default_value": "True", 2023-11-26T07:23:10.563 INFO:teuthology.orchestra.run.smithi143.stdout: "min": "", 2023-11-26T07:23:10.563 INFO:teuthology.orchestra.run.smithi143.stdout: "max": "", 2023-11-26T07:23:10.563 INFO:teuthology.orchestra.run.smithi143.stdout: "enum_allowed": [], 2023-11-26T07:23:10.563 INFO:teuthology.orchestra.run.smithi143.stdout: "desc": "", 2023-11-26T07:23:10.563 INFO:teuthology.orchestra.run.smithi143.stdout: "long_desc": "", 2023-11-26T07:23:10.563 INFO:teuthology.orchestra.run.smithi143.stdout: "tags": [], 2023-11-26T07:23:10.563 INFO:teuthology.orchestra.run.smithi143.stdout: "see_also": [] 2023-11-26T07:23:10.563 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2023-11-26T07:23:10.563 INFO:teuthology.orchestra.run.smithi143.stdout: "AUDIT_API_ENABLED": { 2023-11-26T07:23:10.563 INFO:teuthology.orchestra.run.smithi143.stdout: "name": "AUDIT_API_ENABLED", 2023-11-26T07:23:10.563 INFO:teuthology.orchestra.run.smithi143.stdout: "type": "bool", 2023-11-26T07:23:10.563 INFO:teuthology.orchestra.run.smithi143.stdout: "level": "advanced", 2023-11-26T07:23:10.563 INFO:teuthology.orchestra.run.smithi143.stdout: "flags": 0, 2023-11-26T07:23:10.563 INFO:teuthology.orchestra.run.smithi143.stdout: "default_value": "False", 2023-11-26T07:23:10.564 INFO:teuthology.orchestra.run.smithi143.stdout: "min": "", 2023-11-26T07:23:10.564 INFO:teuthology.orchestra.run.smithi143.stdout: "max": "", 2023-11-26T07:23:10.564 INFO:teuthology.orchestra.run.smithi143.stdout: "enum_allowed": [], 2023-11-26T07:23:10.564 INFO:teuthology.orchestra.run.smithi143.stdout: "desc": "", 2023-11-26T07:23:10.564 INFO:teuthology.orchestra.run.smithi143.stdout: "long_desc": "", 2023-11-26T07:23:10.564 INFO:teuthology.orchestra.run.smithi143.stdout: "tags": [], 2023-11-26T07:23:10.564 INFO:teuthology.orchestra.run.smithi143.stdout: "see_also": [] 2023-11-26T07:23:10.564 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2023-11-26T07:23:10.564 INFO:teuthology.orchestra.run.smithi143.stdout: "AUDIT_API_LOG_PAYLOAD": { 2023-11-26T07:23:10.564 INFO:teuthology.orchestra.run.smithi143.stdout: "name": "AUDIT_API_LOG_PAYLOAD", 2023-11-26T07:23:10.564 INFO:teuthology.orchestra.run.smithi143.stdout: "type": "bool", 2023-11-26T07:23:10.564 INFO:teuthology.orchestra.run.smithi143.stdout: "level": "advanced", 2023-11-26T07:23:10.564 INFO:teuthology.orchestra.run.smithi143.stdout: "flags": 0, 2023-11-26T07:23:10.565 INFO:teuthology.orchestra.run.smithi143.stdout: "default_value": "True", 2023-11-26T07:23:10.565 INFO:teuthology.orchestra.run.smithi143.stdout: "min": "", 2023-11-26T07:23:10.565 INFO:teuthology.orchestra.run.smithi143.stdout: "max": "", 2023-11-26T07:23:10.565 INFO:teuthology.orchestra.run.smithi143.stdout: "enum_allowed": [], 2023-11-26T07:23:10.565 INFO:teuthology.orchestra.run.smithi143.stdout: "desc": "", 2023-11-26T07:23:10.565 INFO:teuthology.orchestra.run.smithi143.stdout: "long_desc": "", 2023-11-26T07:23:10.565 INFO:teuthology.orchestra.run.smithi143.stdout: "tags": [], 2023-11-26T07:23:10.565 INFO:teuthology.orchestra.run.smithi143.stdout: "see_also": [] 2023-11-26T07:23:10.565 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2023-11-26T07:23:10.565 INFO:teuthology.orchestra.run.smithi143.stdout: "ENABLE_BROWSABLE_API": { 2023-11-26T07:23:10.565 INFO:teuthology.orchestra.run.smithi143.stdout: "name": "ENABLE_BROWSABLE_API", 2023-11-26T07:23:10.565 INFO:teuthology.orchestra.run.smithi143.stdout: "type": "bool", 2023-11-26T07:23:10.565 INFO:teuthology.orchestra.run.smithi143.stdout: "level": "advanced", 2023-11-26T07:23:10.566 INFO:teuthology.orchestra.run.smithi143.stdout: "flags": 0, 2023-11-26T07:23:10.566 INFO:teuthology.orchestra.run.smithi143.stdout: "default_value": "True", 2023-11-26T07:23:10.566 INFO:teuthology.orchestra.run.smithi143.stdout: "min": "", 2023-11-26T07:23:10.566 INFO:teuthology.orchestra.run.smithi143.stdout: "max": "", 2023-11-26T07:23:10.566 INFO:teuthology.orchestra.run.smithi143.stdout: "enum_allowed": [], 2023-11-26T07:23:10.566 INFO:teuthology.orchestra.run.smithi143.stdout: "desc": "", 2023-11-26T07:23:10.566 INFO:teuthology.orchestra.run.smithi143.stdout: "long_desc": "", 2023-11-26T07:23:10.566 INFO:teuthology.orchestra.run.smithi143.stdout: "tags": [], 2023-11-26T07:23:10.566 INFO:teuthology.orchestra.run.smithi143.stdout: "see_also": [] 2023-11-26T07:23:10.566 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2023-11-26T07:23:10.566 INFO:teuthology.orchestra.run.smithi143.stdout: "FEATURE_TOGGLE_CEPHFS": { 2023-11-26T07:23:10.566 INFO:teuthology.orchestra.run.smithi143.stdout: "name": "FEATURE_TOGGLE_CEPHFS", 2023-11-26T07:23:10.566 INFO:teuthology.orchestra.run.smithi143.stdout: "type": "bool", 2023-11-26T07:23:10.566 INFO:teuthology.orchestra.run.smithi143.stdout: "level": "advanced", 2023-11-26T07:23:10.567 INFO:teuthology.orchestra.run.smithi143.stdout: "flags": 0, 2023-11-26T07:23:10.567 INFO:teuthology.orchestra.run.smithi143.stdout: "default_value": "True", 2023-11-26T07:23:10.567 INFO:teuthology.orchestra.run.smithi143.stdout: "min": "", 2023-11-26T07:23:10.567 INFO:teuthology.orchestra.run.smithi143.stdout: "max": "", 2023-11-26T07:23:10.567 INFO:teuthology.orchestra.run.smithi143.stdout: "enum_allowed": [], 2023-11-26T07:23:10.567 INFO:teuthology.orchestra.run.smithi143.stdout: "desc": "", 2023-11-26T07:23:10.567 INFO:teuthology.orchestra.run.smithi143.stdout: "long_desc": "", 2023-11-26T07:23:10.567 INFO:teuthology.orchestra.run.smithi143.stdout: "tags": [], 2023-11-26T07:23:10.567 INFO:teuthology.orchestra.run.smithi143.stdout: "see_also": [] 2023-11-26T07:23:10.567 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2023-11-26T07:23:10.567 INFO:teuthology.orchestra.run.smithi143.stdout: "FEATURE_TOGGLE_DASHBOARD": { 2023-11-26T07:23:10.567 INFO:teuthology.orchestra.run.smithi143.stdout: "name": "FEATURE_TOGGLE_DASHBOARD", 2023-11-26T07:23:10.567 INFO:teuthology.orchestra.run.smithi143.stdout: "type": "bool", 2023-11-26T07:23:10.568 INFO:teuthology.orchestra.run.smithi143.stdout: "level": "advanced", 2023-11-26T07:23:10.568 INFO:teuthology.orchestra.run.smithi143.stdout: "flags": 0, 2023-11-26T07:23:10.568 INFO:teuthology.orchestra.run.smithi143.stdout: "default_value": "True", 2023-11-26T07:23:10.568 INFO:teuthology.orchestra.run.smithi143.stdout: "min": "", 2023-11-26T07:23:10.568 INFO:teuthology.orchestra.run.smithi143.stdout: "max": "", 2023-11-26T07:23:10.568 INFO:teuthology.orchestra.run.smithi143.stdout: "enum_allowed": [], 2023-11-26T07:23:10.568 INFO:teuthology.orchestra.run.smithi143.stdout: "desc": "", 2023-11-26T07:23:10.568 INFO:teuthology.orchestra.run.smithi143.stdout: "long_desc": "", 2023-11-26T07:23:10.568 INFO:teuthology.orchestra.run.smithi143.stdout: "tags": [], 2023-11-26T07:23:10.568 INFO:teuthology.orchestra.run.smithi143.stdout: "see_also": [] 2023-11-26T07:23:10.568 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2023-11-26T07:23:10.568 INFO:teuthology.orchestra.run.smithi143.stdout: "FEATURE_TOGGLE_ISCSI": { 2023-11-26T07:23:10.568 INFO:teuthology.orchestra.run.smithi143.stdout: "name": "FEATURE_TOGGLE_ISCSI", 2023-11-26T07:23:10.568 INFO:teuthology.orchestra.run.smithi143.stdout: "type": "bool", 2023-11-26T07:23:10.569 INFO:teuthology.orchestra.run.smithi143.stdout: "level": "advanced", 2023-11-26T07:23:10.569 INFO:teuthology.orchestra.run.smithi143.stdout: "flags": 0, 2023-11-26T07:23:10.569 INFO:teuthology.orchestra.run.smithi143.stdout: "default_value": "True", 2023-11-26T07:23:10.569 INFO:teuthology.orchestra.run.smithi143.stdout: "min": "", 2023-11-26T07:23:10.569 INFO:teuthology.orchestra.run.smithi143.stdout: "max": "", 2023-11-26T07:23:10.569 INFO:teuthology.orchestra.run.smithi143.stdout: "enum_allowed": [], 2023-11-26T07:23:10.569 INFO:teuthology.orchestra.run.smithi143.stdout: "desc": "", 2023-11-26T07:23:10.569 INFO:teuthology.orchestra.run.smithi143.stdout: "long_desc": "", 2023-11-26T07:23:10.569 INFO:teuthology.orchestra.run.smithi143.stdout: "tags": [], 2023-11-26T07:23:10.569 INFO:teuthology.orchestra.run.smithi143.stdout: "see_also": [] 2023-11-26T07:23:10.569 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2023-11-26T07:23:10.569 INFO:teuthology.orchestra.run.smithi143.stdout: "FEATURE_TOGGLE_MIRRORING": { 2023-11-26T07:23:10.570 INFO:teuthology.orchestra.run.smithi143.stdout: "name": "FEATURE_TOGGLE_MIRRORING", 2023-11-26T07:23:10.570 INFO:teuthology.orchestra.run.smithi143.stdout: "type": "bool", 2023-11-26T07:23:10.570 INFO:teuthology.orchestra.run.smithi143.stdout: "level": "advanced", 2023-11-26T07:23:10.570 INFO:teuthology.orchestra.run.smithi143.stdout: "flags": 0, 2023-11-26T07:23:10.570 INFO:teuthology.orchestra.run.smithi143.stdout: "default_value": "True", 2023-11-26T07:23:10.570 INFO:teuthology.orchestra.run.smithi143.stdout: "min": "", 2023-11-26T07:23:10.570 INFO:teuthology.orchestra.run.smithi143.stdout: "max": "", 2023-11-26T07:23:10.570 INFO:teuthology.orchestra.run.smithi143.stdout: "enum_allowed": [], 2023-11-26T07:23:10.570 INFO:teuthology.orchestra.run.smithi143.stdout: "desc": "", 2023-11-26T07:23:10.570 INFO:teuthology.orchestra.run.smithi143.stdout: "long_desc": "", 2023-11-26T07:23:10.570 INFO:teuthology.orchestra.run.smithi143.stdout: "tags": [], 2023-11-26T07:23:10.570 INFO:teuthology.orchestra.run.smithi143.stdout: "see_also": [] 2023-11-26T07:23:10.570 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2023-11-26T07:23:10.571 INFO:teuthology.orchestra.run.smithi143.stdout: "FEATURE_TOGGLE_NFS": { 2023-11-26T07:23:10.571 INFO:teuthology.orchestra.run.smithi143.stdout: "name": "FEATURE_TOGGLE_NFS", 2023-11-26T07:23:10.571 INFO:teuthology.orchestra.run.smithi143.stdout: "type": "bool", 2023-11-26T07:23:10.571 INFO:teuthology.orchestra.run.smithi143.stdout: "level": "advanced", 2023-11-26T07:23:10.571 INFO:teuthology.orchestra.run.smithi143.stdout: "flags": 0, 2023-11-26T07:23:10.571 INFO:teuthology.orchestra.run.smithi143.stdout: "default_value": "True", 2023-11-26T07:23:10.571 INFO:teuthology.orchestra.run.smithi143.stdout: "min": "", 2023-11-26T07:23:10.571 INFO:teuthology.orchestra.run.smithi143.stdout: "max": "", 2023-11-26T07:23:10.571 INFO:teuthology.orchestra.run.smithi143.stdout: "enum_allowed": [], 2023-11-26T07:23:10.571 INFO:teuthology.orchestra.run.smithi143.stdout: "desc": "", 2023-11-26T07:23:10.571 INFO:teuthology.orchestra.run.smithi143.stdout: "long_desc": "", 2023-11-26T07:23:10.571 INFO:teuthology.orchestra.run.smithi143.stdout: "tags": [], 2023-11-26T07:23:10.571 INFO:teuthology.orchestra.run.smithi143.stdout: "see_also": [] 2023-11-26T07:23:10.571 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2023-11-26T07:23:10.572 INFO:teuthology.orchestra.run.smithi143.stdout: "FEATURE_TOGGLE_RBD": { 2023-11-26T07:23:10.572 INFO:teuthology.orchestra.run.smithi143.stdout: "name": "FEATURE_TOGGLE_RBD", 2023-11-26T07:23:10.572 INFO:teuthology.orchestra.run.smithi143.stdout: "type": "bool", 2023-11-26T07:23:10.572 INFO:teuthology.orchestra.run.smithi143.stdout: "level": "advanced", 2023-11-26T07:23:10.572 INFO:teuthology.orchestra.run.smithi143.stdout: "flags": 0, 2023-11-26T07:23:10.572 INFO:teuthology.orchestra.run.smithi143.stdout: "default_value": "True", 2023-11-26T07:23:10.572 INFO:teuthology.orchestra.run.smithi143.stdout: "min": "", 2023-11-26T07:23:10.572 INFO:teuthology.orchestra.run.smithi143.stdout: "max": "", 2023-11-26T07:23:10.572 INFO:teuthology.orchestra.run.smithi143.stdout: "enum_allowed": [], 2023-11-26T07:23:10.572 INFO:teuthology.orchestra.run.smithi143.stdout: "desc": "", 2023-11-26T07:23:10.572 INFO:teuthology.orchestra.run.smithi143.stdout: "long_desc": "", 2023-11-26T07:23:10.572 INFO:teuthology.orchestra.run.smithi143.stdout: "tags": [], 2023-11-26T07:23:10.572 INFO:teuthology.orchestra.run.smithi143.stdout: "see_also": [] 2023-11-26T07:23:10.573 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2023-11-26T07:23:10.573 INFO:teuthology.orchestra.run.smithi143.stdout: "FEATURE_TOGGLE_RGW": { 2023-11-26T07:23:10.573 INFO:teuthology.orchestra.run.smithi143.stdout: "name": "FEATURE_TOGGLE_RGW", 2023-11-26T07:23:10.573 INFO:teuthology.orchestra.run.smithi143.stdout: "type": "bool", 2023-11-26T07:23:10.573 INFO:teuthology.orchestra.run.smithi143.stdout: "level": "advanced", 2023-11-26T07:23:10.573 INFO:teuthology.orchestra.run.smithi143.stdout: "flags": 0, 2023-11-26T07:23:10.573 INFO:teuthology.orchestra.run.smithi143.stdout: "default_value": "True", 2023-11-26T07:23:10.573 INFO:teuthology.orchestra.run.smithi143.stdout: "min": "", 2023-11-26T07:23:10.573 INFO:teuthology.orchestra.run.smithi143.stdout: "max": "", 2023-11-26T07:23:10.573 INFO:teuthology.orchestra.run.smithi143.stdout: "enum_allowed": [], 2023-11-26T07:23:10.573 INFO:teuthology.orchestra.run.smithi143.stdout: "desc": "", 2023-11-26T07:23:10.573 INFO:teuthology.orchestra.run.smithi143.stdout: "long_desc": "", 2023-11-26T07:23:10.573 INFO:teuthology.orchestra.run.smithi143.stdout: "tags": [], 2023-11-26T07:23:10.574 INFO:teuthology.orchestra.run.smithi143.stdout: "see_also": [] 2023-11-26T07:23:10.574 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2023-11-26T07:23:10.574 INFO:teuthology.orchestra.run.smithi143.stdout: "GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE": { 2023-11-26T07:23:10.574 INFO:teuthology.orchestra.run.smithi143.stdout: "name": "GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE", 2023-11-26T07:23:10.574 INFO:teuthology.orchestra.run.smithi143.stdout: "type": "str", 2023-11-26T07:23:10.574 INFO:teuthology.orchestra.run.smithi143.stdout: "level": "advanced", 2023-11-26T07:23:10.574 INFO:teuthology.orchestra.run.smithi143.stdout: "flags": 0, 2023-11-26T07:23:10.574 INFO:teuthology.orchestra.run.smithi143.stdout: "default_value": "", 2023-11-26T07:23:10.574 INFO:teuthology.orchestra.run.smithi143.stdout: "min": "", 2023-11-26T07:23:10.574 INFO:teuthology.orchestra.run.smithi143.stdout: "max": "", 2023-11-26T07:23:10.574 INFO:teuthology.orchestra.run.smithi143.stdout: "enum_allowed": [], 2023-11-26T07:23:10.574 INFO:teuthology.orchestra.run.smithi143.stdout: "desc": "", 2023-11-26T07:23:10.574 INFO:teuthology.orchestra.run.smithi143.stdout: "long_desc": "", 2023-11-26T07:23:10.575 INFO:teuthology.orchestra.run.smithi143.stdout: "tags": [], 2023-11-26T07:23:10.575 INFO:teuthology.orchestra.run.smithi143.stdout: "see_also": [] 2023-11-26T07:23:10.575 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2023-11-26T07:23:10.575 INFO:teuthology.orchestra.run.smithi143.stdout: "GRAFANA_API_PASSWORD": { 2023-11-26T07:23:10.575 INFO:teuthology.orchestra.run.smithi143.stdout: "name": "GRAFANA_API_PASSWORD", 2023-11-26T07:23:10.575 INFO:teuthology.orchestra.run.smithi143.stdout: "type": "str", 2023-11-26T07:23:10.575 INFO:teuthology.orchestra.run.smithi143.stdout: "level": "advanced", 2023-11-26T07:23:10.575 INFO:teuthology.orchestra.run.smithi143.stdout: "flags": 0, 2023-11-26T07:23:10.575 INFO:teuthology.orchestra.run.smithi143.stdout: "default_value": "admin", 2023-11-26T07:23:10.575 INFO:teuthology.orchestra.run.smithi143.stdout: "min": "", 2023-11-26T07:23:10.575 INFO:teuthology.orchestra.run.smithi143.stdout: "max": "", 2023-11-26T07:23:10.575 INFO:teuthology.orchestra.run.smithi143.stdout: "enum_allowed": [], 2023-11-26T07:23:10.575 INFO:teuthology.orchestra.run.smithi143.stdout: "desc": "", 2023-11-26T07:23:10.576 INFO:teuthology.orchestra.run.smithi143.stdout: "long_desc": "", 2023-11-26T07:23:10.576 INFO:teuthology.orchestra.run.smithi143.stdout: "tags": [], 2023-11-26T07:23:10.576 INFO:teuthology.orchestra.run.smithi143.stdout: "see_also": [] 2023-11-26T07:23:10.576 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2023-11-26T07:23:10.576 INFO:teuthology.orchestra.run.smithi143.stdout: "GRAFANA_API_SSL_VERIFY": { 2023-11-26T07:23:10.576 INFO:teuthology.orchestra.run.smithi143.stdout: "name": "GRAFANA_API_SSL_VERIFY", 2023-11-26T07:23:10.576 INFO:teuthology.orchestra.run.smithi143.stdout: "type": "bool", 2023-11-26T07:23:10.576 INFO:teuthology.orchestra.run.smithi143.stdout: "level": "advanced", 2023-11-26T07:23:10.576 INFO:teuthology.orchestra.run.smithi143.stdout: "flags": 0, 2023-11-26T07:23:10.576 INFO:teuthology.orchestra.run.smithi143.stdout: "default_value": "True", 2023-11-26T07:23:10.576 INFO:teuthology.orchestra.run.smithi143.stdout: "min": "", 2023-11-26T07:23:10.576 INFO:teuthology.orchestra.run.smithi143.stdout: "max": "", 2023-11-26T07:23:10.576 INFO:teuthology.orchestra.run.smithi143.stdout: "enum_allowed": [], 2023-11-26T07:23:10.577 INFO:teuthology.orchestra.run.smithi143.stdout: "desc": "", 2023-11-26T07:23:10.577 INFO:teuthology.orchestra.run.smithi143.stdout: "long_desc": "", 2023-11-26T07:23:10.577 INFO:teuthology.orchestra.run.smithi143.stdout: "tags": [], 2023-11-26T07:23:10.577 INFO:teuthology.orchestra.run.smithi143.stdout: "see_also": [] 2023-11-26T07:23:10.577 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2023-11-26T07:23:10.577 INFO:teuthology.orchestra.run.smithi143.stdout: "GRAFANA_API_URL": { 2023-11-26T07:23:10.577 INFO:teuthology.orchestra.run.smithi143.stdout: "name": "GRAFANA_API_URL", 2023-11-26T07:23:10.577 INFO:teuthology.orchestra.run.smithi143.stdout: "type": "str", 2023-11-26T07:23:10.577 INFO:teuthology.orchestra.run.smithi143.stdout: "level": "advanced", 2023-11-26T07:23:10.577 INFO:teuthology.orchestra.run.smithi143.stdout: "flags": 0, 2023-11-26T07:23:10.577 INFO:teuthology.orchestra.run.smithi143.stdout: "default_value": "", 2023-11-26T07:23:10.577 INFO:teuthology.orchestra.run.smithi143.stdout: "min": "", 2023-11-26T07:23:10.577 INFO:teuthology.orchestra.run.smithi143.stdout: "max": "", 2023-11-26T07:23:10.578 INFO:teuthology.orchestra.run.smithi143.stdout: "enum_allowed": [], 2023-11-26T07:23:10.578 INFO:teuthology.orchestra.run.smithi143.stdout: "desc": "", 2023-11-26T07:23:10.578 INFO:teuthology.orchestra.run.smithi143.stdout: "long_desc": "", 2023-11-26T07:23:10.578 INFO:teuthology.orchestra.run.smithi143.stdout: "tags": [], 2023-11-26T07:23:10.578 INFO:teuthology.orchestra.run.smithi143.stdout: "see_also": [] 2023-11-26T07:23:10.578 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2023-11-26T07:23:10.578 INFO:teuthology.orchestra.run.smithi143.stdout: "GRAFANA_API_USERNAME": { 2023-11-26T07:23:10.578 INFO:teuthology.orchestra.run.smithi143.stdout: "name": "GRAFANA_API_USERNAME", 2023-11-26T07:23:10.578 INFO:teuthology.orchestra.run.smithi143.stdout: "type": "str", 2023-11-26T07:23:10.578 INFO:teuthology.orchestra.run.smithi143.stdout: "level": "advanced", 2023-11-26T07:23:10.578 INFO:teuthology.orchestra.run.smithi143.stdout: "flags": 0, 2023-11-26T07:23:10.578 INFO:teuthology.orchestra.run.smithi143.stdout: "default_value": "admin", 2023-11-26T07:23:10.578 INFO:teuthology.orchestra.run.smithi143.stdout: "min": "", 2023-11-26T07:23:10.579 INFO:teuthology.orchestra.run.smithi143.stdout: "max": "", 2023-11-26T07:23:10.579 INFO:teuthology.orchestra.run.smithi143.stdout: "enum_allowed": [], 2023-11-26T07:23:10.579 INFO:teuthology.orchestra.run.smithi143.stdout: "desc": "", 2023-11-26T07:23:10.579 INFO:teuthology.orchestra.run.smithi143.stdout: "long_desc": "", 2023-11-26T07:23:10.579 INFO:teuthology.orchestra.run.smithi143.stdout: "tags": [], 2023-11-26T07:23:10.579 INFO:teuthology.orchestra.run.smithi143.stdout: "see_also": [] 2023-11-26T07:23:10.579 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2023-11-26T07:23:10.579 INFO:teuthology.orchestra.run.smithi143.stdout: "GRAFANA_FRONTEND_API_URL": { 2023-11-26T07:23:10.579 INFO:teuthology.orchestra.run.smithi143.stdout: "name": "GRAFANA_FRONTEND_API_URL", 2023-11-26T07:23:10.579 INFO:teuthology.orchestra.run.smithi143.stdout: "type": "str", 2023-11-26T07:23:10.579 INFO:teuthology.orchestra.run.smithi143.stdout: "level": "advanced", 2023-11-26T07:23:10.579 INFO:teuthology.orchestra.run.smithi143.stdout: "flags": 0, 2023-11-26T07:23:10.579 INFO:teuthology.orchestra.run.smithi143.stdout: "default_value": "", 2023-11-26T07:23:10.579 INFO:teuthology.orchestra.run.smithi143.stdout: "min": "", 2023-11-26T07:23:10.580 INFO:teuthology.orchestra.run.smithi143.stdout: "max": "", 2023-11-26T07:23:10.580 INFO:teuthology.orchestra.run.smithi143.stdout: "enum_allowed": [], 2023-11-26T07:23:10.580 INFO:teuthology.orchestra.run.smithi143.stdout: "desc": "", 2023-11-26T07:23:10.580 INFO:teuthology.orchestra.run.smithi143.stdout: "long_desc": "", 2023-11-26T07:23:10.580 INFO:teuthology.orchestra.run.smithi143.stdout: "tags": [], 2023-11-26T07:23:10.580 INFO:teuthology.orchestra.run.smithi143.stdout: "see_also": [] 2023-11-26T07:23:10.580 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2023-11-26T07:23:10.580 INFO:teuthology.orchestra.run.smithi143.stdout: "GRAFANA_UPDATE_DASHBOARDS": { 2023-11-26T07:23:10.580 INFO:teuthology.orchestra.run.smithi143.stdout: "name": "GRAFANA_UPDATE_DASHBOARDS", 2023-11-26T07:23:10.580 INFO:teuthology.orchestra.run.smithi143.stdout: "type": "bool", 2023-11-26T07:23:10.580 INFO:teuthology.orchestra.run.smithi143.stdout: "level": "advanced", 2023-11-26T07:23:10.580 INFO:teuthology.orchestra.run.smithi143.stdout: "flags": 0, 2023-11-26T07:23:10.580 INFO:teuthology.orchestra.run.smithi143.stdout: "default_value": "False", 2023-11-26T07:23:10.581 INFO:teuthology.orchestra.run.smithi143.stdout: "min": "", 2023-11-26T07:23:10.581 INFO:teuthology.orchestra.run.smithi143.stdout: "max": "", 2023-11-26T07:23:10.581 INFO:teuthology.orchestra.run.smithi143.stdout: "enum_allowed": [], 2023-11-26T07:23:10.581 INFO:teuthology.orchestra.run.smithi143.stdout: "desc": "", 2023-11-26T07:23:10.581 INFO:teuthology.orchestra.run.smithi143.stdout: "long_desc": "", 2023-11-26T07:23:10.581 INFO:teuthology.orchestra.run.smithi143.stdout: "tags": [], 2023-11-26T07:23:10.581 INFO:teuthology.orchestra.run.smithi143.stdout: "see_also": [] 2023-11-26T07:23:10.581 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2023-11-26T07:23:10.581 INFO:teuthology.orchestra.run.smithi143.stdout: "ISCSI_API_SSL_VERIFICATION": { 2023-11-26T07:23:10.581 INFO:teuthology.orchestra.run.smithi143.stdout: "name": "ISCSI_API_SSL_VERIFICATION", 2023-11-26T07:23:10.581 INFO:teuthology.orchestra.run.smithi143.stdout: "type": "bool", 2023-11-26T07:23:10.581 INFO:teuthology.orchestra.run.smithi143.stdout: "level": "advanced", 2023-11-26T07:23:10.581 INFO:teuthology.orchestra.run.smithi143.stdout: "flags": 0, 2023-11-26T07:23:10.582 INFO:teuthology.orchestra.run.smithi143.stdout: "default_value": "True", 2023-11-26T07:23:10.582 INFO:teuthology.orchestra.run.smithi143.stdout: "min": "", 2023-11-26T07:23:10.582 INFO:teuthology.orchestra.run.smithi143.stdout: "max": "", 2023-11-26T07:23:10.582 INFO:teuthology.orchestra.run.smithi143.stdout: "enum_allowed": [], 2023-11-26T07:23:10.582 INFO:teuthology.orchestra.run.smithi143.stdout: "desc": "", 2023-11-26T07:23:10.582 INFO:teuthology.orchestra.run.smithi143.stdout: "long_desc": "", 2023-11-26T07:23:10.582 INFO:teuthology.orchestra.run.smithi143.stdout: "tags": [], 2023-11-26T07:23:10.582 INFO:teuthology.orchestra.run.smithi143.stdout: "see_also": [] 2023-11-26T07:23:10.582 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2023-11-26T07:23:10.582 INFO:teuthology.orchestra.run.smithi143.stdout: "ISSUE_TRACKER_API_KEY": { 2023-11-26T07:23:10.582 INFO:teuthology.orchestra.run.smithi143.stdout: "name": "ISSUE_TRACKER_API_KEY", 2023-11-26T07:23:10.582 INFO:teuthology.orchestra.run.smithi143.stdout: "type": "str", 2023-11-26T07:23:10.582 INFO:teuthology.orchestra.run.smithi143.stdout: "level": "advanced", 2023-11-26T07:23:10.583 INFO:teuthology.orchestra.run.smithi143.stdout: "flags": 0, 2023-11-26T07:23:10.583 INFO:teuthology.orchestra.run.smithi143.stdout: "default_value": "", 2023-11-26T07:23:10.583 INFO:teuthology.orchestra.run.smithi143.stdout: "min": "", 2023-11-26T07:23:10.583 INFO:teuthology.orchestra.run.smithi143.stdout: "max": "", 2023-11-26T07:23:10.583 INFO:teuthology.orchestra.run.smithi143.stdout: "enum_allowed": [], 2023-11-26T07:23:10.583 INFO:teuthology.orchestra.run.smithi143.stdout: "desc": "", 2023-11-26T07:23:10.583 INFO:teuthology.orchestra.run.smithi143.stdout: "long_desc": "", 2023-11-26T07:23:10.583 INFO:teuthology.orchestra.run.smithi143.stdout: "tags": [], 2023-11-26T07:23:10.583 INFO:teuthology.orchestra.run.smithi143.stdout: "see_also": [] 2023-11-26T07:23:10.583 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2023-11-26T07:23:10.583 INFO:teuthology.orchestra.run.smithi143.stdout: "PROMETHEUS_API_HOST": { 2023-11-26T07:23:10.583 INFO:teuthology.orchestra.run.smithi143.stdout: "name": "PROMETHEUS_API_HOST", 2023-11-26T07:23:10.583 INFO:teuthology.orchestra.run.smithi143.stdout: "type": "str", 2023-11-26T07:23:10.583 INFO:teuthology.orchestra.run.smithi143.stdout: "level": "advanced", 2023-11-26T07:23:10.584 INFO:teuthology.orchestra.run.smithi143.stdout: "flags": 0, 2023-11-26T07:23:10.584 INFO:teuthology.orchestra.run.smithi143.stdout: "default_value": "", 2023-11-26T07:23:10.584 INFO:teuthology.orchestra.run.smithi143.stdout: "min": "", 2023-11-26T07:23:10.584 INFO:teuthology.orchestra.run.smithi143.stdout: "max": "", 2023-11-26T07:23:10.584 INFO:teuthology.orchestra.run.smithi143.stdout: "enum_allowed": [], 2023-11-26T07:23:10.584 INFO:teuthology.orchestra.run.smithi143.stdout: "desc": "", 2023-11-26T07:23:10.584 INFO:teuthology.orchestra.run.smithi143.stdout: "long_desc": "", 2023-11-26T07:23:10.584 INFO:teuthology.orchestra.run.smithi143.stdout: "tags": [], 2023-11-26T07:23:10.584 INFO:teuthology.orchestra.run.smithi143.stdout: "see_also": [] 2023-11-26T07:23:10.584 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2023-11-26T07:23:10.584 INFO:teuthology.orchestra.run.smithi143.stdout: "PROMETHEUS_API_SSL_VERIFY": { 2023-11-26T07:23:10.584 INFO:teuthology.orchestra.run.smithi143.stdout: "name": "PROMETHEUS_API_SSL_VERIFY", 2023-11-26T07:23:10.585 INFO:teuthology.orchestra.run.smithi143.stdout: "type": "bool", 2023-11-26T07:23:10.585 INFO:teuthology.orchestra.run.smithi143.stdout: "level": "advanced", 2023-11-26T07:23:10.585 INFO:teuthology.orchestra.run.smithi143.stdout: "flags": 0, 2023-11-26T07:23:10.585 INFO:teuthology.orchestra.run.smithi143.stdout: "default_value": "True", 2023-11-26T07:23:10.585 INFO:teuthology.orchestra.run.smithi143.stdout: "min": "", 2023-11-26T07:23:10.585 INFO:teuthology.orchestra.run.smithi143.stdout: "max": "", 2023-11-26T07:23:10.585 INFO:teuthology.orchestra.run.smithi143.stdout: "enum_allowed": [], 2023-11-26T07:23:10.585 INFO:teuthology.orchestra.run.smithi143.stdout: "desc": "", 2023-11-26T07:23:10.585 INFO:teuthology.orchestra.run.smithi143.stdout: "long_desc": "", 2023-11-26T07:23:10.585 INFO:teuthology.orchestra.run.smithi143.stdout: "tags": [], 2023-11-26T07:23:10.585 INFO:teuthology.orchestra.run.smithi143.stdout: "see_also": [] 2023-11-26T07:23:10.585 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2023-11-26T07:23:10.585 INFO:teuthology.orchestra.run.smithi143.stdout: "PWD_POLICY_CHECK_COMPLEXITY_ENABLED": { 2023-11-26T07:23:10.586 INFO:teuthology.orchestra.run.smithi143.stdout: "name": "PWD_POLICY_CHECK_COMPLEXITY_ENABLED", 2023-11-26T07:23:10.586 INFO:teuthology.orchestra.run.smithi143.stdout: "type": "bool", 2023-11-26T07:23:10.586 INFO:teuthology.orchestra.run.smithi143.stdout: "level": "advanced", 2023-11-26T07:23:10.586 INFO:teuthology.orchestra.run.smithi143.stdout: "flags": 0, 2023-11-26T07:23:10.586 INFO:teuthology.orchestra.run.smithi143.stdout: "default_value": "False", 2023-11-26T07:23:10.586 INFO:teuthology.orchestra.run.smithi143.stdout: "min": "", 2023-11-26T07:23:10.586 INFO:teuthology.orchestra.run.smithi143.stdout: "max": "", 2023-11-26T07:23:10.586 INFO:teuthology.orchestra.run.smithi143.stdout: "enum_allowed": [], 2023-11-26T07:23:10.586 INFO:teuthology.orchestra.run.smithi143.stdout: "desc": "", 2023-11-26T07:23:10.586 INFO:teuthology.orchestra.run.smithi143.stdout: "long_desc": "", 2023-11-26T07:23:10.586 INFO:teuthology.orchestra.run.smithi143.stdout: "tags": [], 2023-11-26T07:23:10.586 INFO:teuthology.orchestra.run.smithi143.stdout: "see_also": [] 2023-11-26T07:23:10.586 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2023-11-26T07:23:10.587 INFO:teuthology.orchestra.run.smithi143.stdout: "PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED": { 2023-11-26T07:23:10.587 INFO:teuthology.orchestra.run.smithi143.stdout: "name": "PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED", 2023-11-26T07:23:10.587 INFO:teuthology.orchestra.run.smithi143.stdout: "type": "bool", 2023-11-26T07:23:10.587 INFO:teuthology.orchestra.run.smithi143.stdout: "level": "advanced", 2023-11-26T07:23:10.587 INFO:teuthology.orchestra.run.smithi143.stdout: "flags": 0, 2023-11-26T07:23:10.587 INFO:teuthology.orchestra.run.smithi143.stdout: "default_value": "False", 2023-11-26T07:23:10.587 INFO:teuthology.orchestra.run.smithi143.stdout: "min": "", 2023-11-26T07:23:10.587 INFO:teuthology.orchestra.run.smithi143.stdout: "max": "", 2023-11-26T07:23:10.587 INFO:teuthology.orchestra.run.smithi143.stdout: "enum_allowed": [], 2023-11-26T07:23:10.587 INFO:teuthology.orchestra.run.smithi143.stdout: "desc": "", 2023-11-26T07:23:10.587 INFO:teuthology.orchestra.run.smithi143.stdout: "long_desc": "", 2023-11-26T07:23:10.587 INFO:teuthology.orchestra.run.smithi143.stdout: "tags": [], 2023-11-26T07:23:10.588 INFO:teuthology.orchestra.run.smithi143.stdout: "see_also": [] 2023-11-26T07:23:10.588 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2023-11-26T07:23:10.588 INFO:teuthology.orchestra.run.smithi143.stdout: "PWD_POLICY_CHECK_LENGTH_ENABLED": { 2023-11-26T07:23:10.588 INFO:teuthology.orchestra.run.smithi143.stdout: "name": "PWD_POLICY_CHECK_LENGTH_ENABLED", 2023-11-26T07:23:10.588 INFO:teuthology.orchestra.run.smithi143.stdout: "type": "bool", 2023-11-26T07:23:10.588 INFO:teuthology.orchestra.run.smithi143.stdout: "level": "advanced", 2023-11-26T07:23:10.588 INFO:teuthology.orchestra.run.smithi143.stdout: "flags": 0, 2023-11-26T07:23:10.588 INFO:teuthology.orchestra.run.smithi143.stdout: "default_value": "True", 2023-11-26T07:23:10.588 INFO:teuthology.orchestra.run.smithi143.stdout: "min": "", 2023-11-26T07:23:10.588 INFO:teuthology.orchestra.run.smithi143.stdout: "max": "", 2023-11-26T07:23:10.589 INFO:teuthology.orchestra.run.smithi143.stdout: "enum_allowed": [], 2023-11-26T07:23:10.589 INFO:teuthology.orchestra.run.smithi143.stdout: "desc": "", 2023-11-26T07:23:10.589 INFO:teuthology.orchestra.run.smithi143.stdout: "long_desc": "", 2023-11-26T07:23:10.589 INFO:teuthology.orchestra.run.smithi143.stdout: "tags": [], 2023-11-26T07:23:10.589 INFO:teuthology.orchestra.run.smithi143.stdout: "see_also": [] 2023-11-26T07:23:10.589 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2023-11-26T07:23:10.589 INFO:teuthology.orchestra.run.smithi143.stdout: "PWD_POLICY_CHECK_OLDPWD_ENABLED": { 2023-11-26T07:23:10.589 INFO:teuthology.orchestra.run.smithi143.stdout: "name": "PWD_POLICY_CHECK_OLDPWD_ENABLED", 2023-11-26T07:23:10.589 INFO:teuthology.orchestra.run.smithi143.stdout: "type": "bool", 2023-11-26T07:23:10.589 INFO:teuthology.orchestra.run.smithi143.stdout: "level": "advanced", 2023-11-26T07:23:10.590 INFO:teuthology.orchestra.run.smithi143.stdout: "flags": 0, 2023-11-26T07:23:10.590 INFO:teuthology.orchestra.run.smithi143.stdout: "default_value": "True", 2023-11-26T07:23:10.590 INFO:teuthology.orchestra.run.smithi143.stdout: "min": "", 2023-11-26T07:23:10.590 INFO:teuthology.orchestra.run.smithi143.stdout: "max": "", 2023-11-26T07:23:10.590 INFO:teuthology.orchestra.run.smithi143.stdout: "enum_allowed": [], 2023-11-26T07:23:10.590 INFO:teuthology.orchestra.run.smithi143.stdout: "desc": "", 2023-11-26T07:23:10.590 INFO:teuthology.orchestra.run.smithi143.stdout: "long_desc": "", 2023-11-26T07:23:10.590 INFO:teuthology.orchestra.run.smithi143.stdout: "tags": [], 2023-11-26T07:23:10.591 INFO:teuthology.orchestra.run.smithi143.stdout: "see_also": [] 2023-11-26T07:23:10.591 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2023-11-26T07:23:10.591 INFO:teuthology.orchestra.run.smithi143.stdout: "PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED": { 2023-11-26T07:23:10.591 INFO:teuthology.orchestra.run.smithi143.stdout: "name": "PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED", 2023-11-26T07:23:10.591 INFO:teuthology.orchestra.run.smithi143.stdout: "type": "bool", 2023-11-26T07:23:10.591 INFO:teuthology.orchestra.run.smithi143.stdout: "level": "advanced", 2023-11-26T07:23:10.591 INFO:teuthology.orchestra.run.smithi143.stdout: "flags": 0, 2023-11-26T07:23:10.591 INFO:teuthology.orchestra.run.smithi143.stdout: "default_value": "False", 2023-11-26T07:23:10.591 INFO:teuthology.orchestra.run.smithi143.stdout: "min": "", 2023-11-26T07:23:10.591 INFO:teuthology.orchestra.run.smithi143.stdout: "max": "", 2023-11-26T07:23:10.591 INFO:teuthology.orchestra.run.smithi143.stdout: "enum_allowed": [], 2023-11-26T07:23:10.591 INFO:teuthology.orchestra.run.smithi143.stdout: "desc": "", 2023-11-26T07:23:10.592 INFO:teuthology.orchestra.run.smithi143.stdout: "long_desc": "", 2023-11-26T07:23:10.592 INFO:teuthology.orchestra.run.smithi143.stdout: "tags": [], 2023-11-26T07:23:10.592 INFO:teuthology.orchestra.run.smithi143.stdout: "see_also": [] 2023-11-26T07:23:10.592 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2023-11-26T07:23:10.592 INFO:teuthology.orchestra.run.smithi143.stdout: "PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED": { 2023-11-26T07:23:10.592 INFO:teuthology.orchestra.run.smithi143.stdout: "name": "PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED", 2023-11-26T07:23:10.592 INFO:teuthology.orchestra.run.smithi143.stdout: "type": "bool", 2023-11-26T07:23:10.592 INFO:teuthology.orchestra.run.smithi143.stdout: "level": "advanced", 2023-11-26T07:23:10.592 INFO:teuthology.orchestra.run.smithi143.stdout: "flags": 0, 2023-11-26T07:23:10.592 INFO:teuthology.orchestra.run.smithi143.stdout: "default_value": "False", 2023-11-26T07:23:10.592 INFO:teuthology.orchestra.run.smithi143.stdout: "min": "", 2023-11-26T07:23:10.592 INFO:teuthology.orchestra.run.smithi143.stdout: "max": "", 2023-11-26T07:23:10.592 INFO:teuthology.orchestra.run.smithi143.stdout: "enum_allowed": [], 2023-11-26T07:23:10.593 INFO:teuthology.orchestra.run.smithi143.stdout: "desc": "", 2023-11-26T07:23:10.593 INFO:teuthology.orchestra.run.smithi143.stdout: "long_desc": "", 2023-11-26T07:23:10.593 INFO:teuthology.orchestra.run.smithi143.stdout: "tags": [], 2023-11-26T07:23:10.593 INFO:teuthology.orchestra.run.smithi143.stdout: "see_also": [] 2023-11-26T07:23:10.593 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2023-11-26T07:23:10.593 INFO:teuthology.orchestra.run.smithi143.stdout: "PWD_POLICY_CHECK_USERNAME_ENABLED": { 2023-11-26T07:23:10.593 INFO:teuthology.orchestra.run.smithi143.stdout: "name": "PWD_POLICY_CHECK_USERNAME_ENABLED", 2023-11-26T07:23:10.593 INFO:teuthology.orchestra.run.smithi143.stdout: "type": "bool", 2023-11-26T07:23:10.593 INFO:teuthology.orchestra.run.smithi143.stdout: "level": "advanced", 2023-11-26T07:23:10.593 INFO:teuthology.orchestra.run.smithi143.stdout: "flags": 0, 2023-11-26T07:23:10.593 INFO:teuthology.orchestra.run.smithi143.stdout: "default_value": "False", 2023-11-26T07:23:10.593 INFO:teuthology.orchestra.run.smithi143.stdout: "min": "", 2023-11-26T07:23:10.593 INFO:teuthology.orchestra.run.smithi143.stdout: "max": "", 2023-11-26T07:23:10.594 INFO:teuthology.orchestra.run.smithi143.stdout: "enum_allowed": [], 2023-11-26T07:23:10.594 INFO:teuthology.orchestra.run.smithi143.stdout: "desc": "", 2023-11-26T07:23:10.594 INFO:teuthology.orchestra.run.smithi143.stdout: "long_desc": "", 2023-11-26T07:23:10.594 INFO:teuthology.orchestra.run.smithi143.stdout: "tags": [], 2023-11-26T07:23:10.594 INFO:teuthology.orchestra.run.smithi143.stdout: "see_also": [] 2023-11-26T07:23:10.594 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2023-11-26T07:23:10.594 INFO:teuthology.orchestra.run.smithi143.stdout: "PWD_POLICY_ENABLED": { 2023-11-26T07:23:10.594 INFO:teuthology.orchestra.run.smithi143.stdout: "name": "PWD_POLICY_ENABLED", 2023-11-26T07:23:10.594 INFO:teuthology.orchestra.run.smithi143.stdout: "type": "bool", 2023-11-26T07:23:10.594 INFO:teuthology.orchestra.run.smithi143.stdout: "level": "advanced", 2023-11-26T07:23:10.594 INFO:teuthology.orchestra.run.smithi143.stdout: "flags": 0, 2023-11-26T07:23:10.594 INFO:teuthology.orchestra.run.smithi143.stdout: "default_value": "True", 2023-11-26T07:23:10.594 INFO:teuthology.orchestra.run.smithi143.stdout: "min": "", 2023-11-26T07:23:10.595 INFO:teuthology.orchestra.run.smithi143.stdout: "max": "", 2023-11-26T07:23:10.595 INFO:teuthology.orchestra.run.smithi143.stdout: "enum_allowed": [], 2023-11-26T07:23:10.595 INFO:teuthology.orchestra.run.smithi143.stdout: "desc": "", 2023-11-26T07:23:10.595 INFO:teuthology.orchestra.run.smithi143.stdout: "long_desc": "", 2023-11-26T07:23:10.595 INFO:teuthology.orchestra.run.smithi143.stdout: "tags": [], 2023-11-26T07:23:10.595 INFO:teuthology.orchestra.run.smithi143.stdout: "see_also": [] 2023-11-26T07:23:10.595 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2023-11-26T07:23:10.595 INFO:teuthology.orchestra.run.smithi143.stdout: "PWD_POLICY_EXCLUSION_LIST": { 2023-11-26T07:23:10.595 INFO:teuthology.orchestra.run.smithi143.stdout: "name": "PWD_POLICY_EXCLUSION_LIST", 2023-11-26T07:23:10.595 INFO:teuthology.orchestra.run.smithi143.stdout: "type": "str", 2023-11-26T07:23:10.595 INFO:teuthology.orchestra.run.smithi143.stdout: "level": "advanced", 2023-11-26T07:23:10.595 INFO:teuthology.orchestra.run.smithi143.stdout: "flags": 0, 2023-11-26T07:23:10.595 INFO:teuthology.orchestra.run.smithi143.stdout: "default_value": "osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps", 2023-11-26T07:23:10.596 INFO:teuthology.orchestra.run.smithi143.stdout: "min": "", 2023-11-26T07:23:10.596 INFO:teuthology.orchestra.run.smithi143.stdout: "max": "", 2023-11-26T07:23:10.596 INFO:teuthology.orchestra.run.smithi143.stdout: "enum_allowed": [], 2023-11-26T07:23:10.596 INFO:teuthology.orchestra.run.smithi143.stdout: "desc": "", 2023-11-26T07:23:10.596 INFO:teuthology.orchestra.run.smithi143.stdout: "long_desc": "", 2023-11-26T07:23:10.596 INFO:teuthology.orchestra.run.smithi143.stdout: "tags": [], 2023-11-26T07:23:10.596 INFO:teuthology.orchestra.run.smithi143.stdout: "see_also": [] 2023-11-26T07:23:10.596 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2023-11-26T07:23:10.596 INFO:teuthology.orchestra.run.smithi143.stdout: "PWD_POLICY_MIN_COMPLEXITY": { 2023-11-26T07:23:10.596 INFO:teuthology.orchestra.run.smithi143.stdout: "name": "PWD_POLICY_MIN_COMPLEXITY", 2023-11-26T07:23:10.596 INFO:teuthology.orchestra.run.smithi143.stdout: "type": "int", 2023-11-26T07:23:10.596 INFO:teuthology.orchestra.run.smithi143.stdout: "level": "advanced", 2023-11-26T07:23:10.596 INFO:teuthology.orchestra.run.smithi143.stdout: "flags": 0, 2023-11-26T07:23:10.597 INFO:teuthology.orchestra.run.smithi143.stdout: "default_value": "10", 2023-11-26T07:23:10.597 INFO:teuthology.orchestra.run.smithi143.stdout: "min": "", 2023-11-26T07:23:10.597 INFO:teuthology.orchestra.run.smithi143.stdout: "max": "", 2023-11-26T07:23:10.597 INFO:teuthology.orchestra.run.smithi143.stdout: "enum_allowed": [], 2023-11-26T07:23:10.597 INFO:teuthology.orchestra.run.smithi143.stdout: "desc": "", 2023-11-26T07:23:10.597 INFO:teuthology.orchestra.run.smithi143.stdout: "long_desc": "", 2023-11-26T07:23:10.597 INFO:teuthology.orchestra.run.smithi143.stdout: "tags": [], 2023-11-26T07:23:10.597 INFO:teuthology.orchestra.run.smithi143.stdout: "see_also": [] 2023-11-26T07:23:10.597 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2023-11-26T07:23:10.597 INFO:teuthology.orchestra.run.smithi143.stdout: "PWD_POLICY_MIN_LENGTH": { 2023-11-26T07:23:10.597 INFO:teuthology.orchestra.run.smithi143.stdout: "name": "PWD_POLICY_MIN_LENGTH", 2023-11-26T07:23:10.597 INFO:teuthology.orchestra.run.smithi143.stdout: "type": "int", 2023-11-26T07:23:10.598 INFO:teuthology.orchestra.run.smithi143.stdout: "level": "advanced", 2023-11-26T07:23:10.598 INFO:teuthology.orchestra.run.smithi143.stdout: "flags": 0, 2023-11-26T07:23:10.598 INFO:teuthology.orchestra.run.smithi143.stdout: "default_value": "8", 2023-11-26T07:23:10.598 INFO:teuthology.orchestra.run.smithi143.stdout: "min": "", 2023-11-26T07:23:10.598 INFO:teuthology.orchestra.run.smithi143.stdout: "max": "", 2023-11-26T07:23:10.598 INFO:teuthology.orchestra.run.smithi143.stdout: "enum_allowed": [], 2023-11-26T07:23:10.598 INFO:teuthology.orchestra.run.smithi143.stdout: "desc": "", 2023-11-26T07:23:10.598 INFO:teuthology.orchestra.run.smithi143.stdout: "long_desc": "", 2023-11-26T07:23:10.598 INFO:teuthology.orchestra.run.smithi143.stdout: "tags": [], 2023-11-26T07:23:10.598 INFO:teuthology.orchestra.run.smithi143.stdout: "see_also": [] 2023-11-26T07:23:10.598 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2023-11-26T07:23:10.598 INFO:teuthology.orchestra.run.smithi143.stdout: "REST_REQUESTS_TIMEOUT": { 2023-11-26T07:23:10.598 INFO:teuthology.orchestra.run.smithi143.stdout: "name": "REST_REQUESTS_TIMEOUT", 2023-11-26T07:23:10.599 INFO:teuthology.orchestra.run.smithi143.stdout: "type": "int", 2023-11-26T07:23:10.599 INFO:teuthology.orchestra.run.smithi143.stdout: "level": "advanced", 2023-11-26T07:23:10.599 INFO:teuthology.orchestra.run.smithi143.stdout: "flags": 0, 2023-11-26T07:23:10.599 INFO:teuthology.orchestra.run.smithi143.stdout: "default_value": "45", 2023-11-26T07:23:10.599 INFO:teuthology.orchestra.run.smithi143.stdout: "min": "", 2023-11-26T07:23:10.599 INFO:teuthology.orchestra.run.smithi143.stdout: "max": "", 2023-11-26T07:23:10.599 INFO:teuthology.orchestra.run.smithi143.stdout: "enum_allowed": [], 2023-11-26T07:23:10.599 INFO:teuthology.orchestra.run.smithi143.stdout: "desc": "", 2023-11-26T07:23:10.599 INFO:teuthology.orchestra.run.smithi143.stdout: "long_desc": "", 2023-11-26T07:23:10.599 INFO:teuthology.orchestra.run.smithi143.stdout: "tags": [], 2023-11-26T07:23:10.599 INFO:teuthology.orchestra.run.smithi143.stdout: "see_also": [] 2023-11-26T07:23:10.599 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2023-11-26T07:23:10.599 INFO:teuthology.orchestra.run.smithi143.stdout: "RGW_API_ACCESS_KEY": { 2023-11-26T07:23:10.600 INFO:teuthology.orchestra.run.smithi143.stdout: "name": "RGW_API_ACCESS_KEY", 2023-11-26T07:23:10.600 INFO:teuthology.orchestra.run.smithi143.stdout: "type": "str", 2023-11-26T07:23:10.600 INFO:teuthology.orchestra.run.smithi143.stdout: "level": "advanced", 2023-11-26T07:23:10.600 INFO:teuthology.orchestra.run.smithi143.stdout: "flags": 0, 2023-11-26T07:23:10.600 INFO:teuthology.orchestra.run.smithi143.stdout: "default_value": "", 2023-11-26T07:23:10.600 INFO:teuthology.orchestra.run.smithi143.stdout: "min": "", 2023-11-26T07:23:10.600 INFO:teuthology.orchestra.run.smithi143.stdout: "max": "", 2023-11-26T07:23:10.600 INFO:teuthology.orchestra.run.smithi143.stdout: "enum_allowed": [], 2023-11-26T07:23:10.600 INFO:teuthology.orchestra.run.smithi143.stdout: "desc": "", 2023-11-26T07:23:10.600 INFO:teuthology.orchestra.run.smithi143.stdout: "long_desc": "", 2023-11-26T07:23:10.600 INFO:teuthology.orchestra.run.smithi143.stdout: "tags": [], 2023-11-26T07:23:10.600 INFO:teuthology.orchestra.run.smithi143.stdout: "see_also": [] 2023-11-26T07:23:10.600 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2023-11-26T07:23:10.601 INFO:teuthology.orchestra.run.smithi143.stdout: "RGW_API_ADMIN_RESOURCE": { 2023-11-26T07:23:10.601 INFO:teuthology.orchestra.run.smithi143.stdout: "name": "RGW_API_ADMIN_RESOURCE", 2023-11-26T07:23:10.601 INFO:teuthology.orchestra.run.smithi143.stdout: "type": "str", 2023-11-26T07:23:10.601 INFO:teuthology.orchestra.run.smithi143.stdout: "level": "advanced", 2023-11-26T07:23:10.601 INFO:teuthology.orchestra.run.smithi143.stdout: "flags": 0, 2023-11-26T07:23:10.601 INFO:teuthology.orchestra.run.smithi143.stdout: "default_value": "admin", 2023-11-26T07:23:10.601 INFO:teuthology.orchestra.run.smithi143.stdout: "min": "", 2023-11-26T07:23:10.601 INFO:teuthology.orchestra.run.smithi143.stdout: "max": "", 2023-11-26T07:23:10.601 INFO:teuthology.orchestra.run.smithi143.stdout: "enum_allowed": [], 2023-11-26T07:23:10.601 INFO:teuthology.orchestra.run.smithi143.stdout: "desc": "", 2023-11-26T07:23:10.601 INFO:teuthology.orchestra.run.smithi143.stdout: "long_desc": "", 2023-11-26T07:23:10.601 INFO:teuthology.orchestra.run.smithi143.stdout: "tags": [], 2023-11-26T07:23:10.601 INFO:teuthology.orchestra.run.smithi143.stdout: "see_also": [] 2023-11-26T07:23:10.602 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2023-11-26T07:23:10.602 INFO:teuthology.orchestra.run.smithi143.stdout: "RGW_API_SECRET_KEY": { 2023-11-26T07:23:10.602 INFO:teuthology.orchestra.run.smithi143.stdout: "name": "RGW_API_SECRET_KEY", 2023-11-26T07:23:10.602 INFO:teuthology.orchestra.run.smithi143.stdout: "type": "str", 2023-11-26T07:23:10.602 INFO:teuthology.orchestra.run.smithi143.stdout: "level": "advanced", 2023-11-26T07:23:10.602 INFO:teuthology.orchestra.run.smithi143.stdout: "flags": 0, 2023-11-26T07:23:10.602 INFO:teuthology.orchestra.run.smithi143.stdout: "default_value": "", 2023-11-26T07:23:10.602 INFO:teuthology.orchestra.run.smithi143.stdout: "min": "", 2023-11-26T07:23:10.602 INFO:teuthology.orchestra.run.smithi143.stdout: "max": "", 2023-11-26T07:23:10.602 INFO:teuthology.orchestra.run.smithi143.stdout: "enum_allowed": [], 2023-11-26T07:23:10.602 INFO:teuthology.orchestra.run.smithi143.stdout: "desc": "", 2023-11-26T07:23:10.602 INFO:teuthology.orchestra.run.smithi143.stdout: "long_desc": "", 2023-11-26T07:23:10.602 INFO:teuthology.orchestra.run.smithi143.stdout: "tags": [], 2023-11-26T07:23:10.603 INFO:teuthology.orchestra.run.smithi143.stdout: "see_also": [] 2023-11-26T07:23:10.603 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2023-11-26T07:23:10.603 INFO:teuthology.orchestra.run.smithi143.stdout: "RGW_API_SSL_VERIFY": { 2023-11-26T07:23:10.603 INFO:teuthology.orchestra.run.smithi143.stdout: "name": "RGW_API_SSL_VERIFY", 2023-11-26T07:23:10.603 INFO:teuthology.orchestra.run.smithi143.stdout: "type": "bool", 2023-11-26T07:23:10.603 INFO:teuthology.orchestra.run.smithi143.stdout: "level": "advanced", 2023-11-26T07:23:10.603 INFO:teuthology.orchestra.run.smithi143.stdout: "flags": 0, 2023-11-26T07:23:10.603 INFO:teuthology.orchestra.run.smithi143.stdout: "default_value": "True", 2023-11-26T07:23:10.603 INFO:teuthology.orchestra.run.smithi143.stdout: "min": "", 2023-11-26T07:23:10.603 INFO:teuthology.orchestra.run.smithi143.stdout: "max": "", 2023-11-26T07:23:10.603 INFO:teuthology.orchestra.run.smithi143.stdout: "enum_allowed": [], 2023-11-26T07:23:10.603 INFO:teuthology.orchestra.run.smithi143.stdout: "desc": "", 2023-11-26T07:23:10.603 INFO:teuthology.orchestra.run.smithi143.stdout: "long_desc": "", 2023-11-26T07:23:10.604 INFO:teuthology.orchestra.run.smithi143.stdout: "tags": [], 2023-11-26T07:23:10.604 INFO:teuthology.orchestra.run.smithi143.stdout: "see_also": [] 2023-11-26T07:23:10.604 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2023-11-26T07:23:10.604 INFO:teuthology.orchestra.run.smithi143.stdout: "UNSAFE_TLS_v1_2": { 2023-11-26T07:23:10.604 INFO:teuthology.orchestra.run.smithi143.stdout: "name": "UNSAFE_TLS_v1_2", 2023-11-26T07:23:10.604 INFO:teuthology.orchestra.run.smithi143.stdout: "type": "bool", 2023-11-26T07:23:10.604 INFO:teuthology.orchestra.run.smithi143.stdout: "level": "advanced", 2023-11-26T07:23:10.604 INFO:teuthology.orchestra.run.smithi143.stdout: "flags": 0, 2023-11-26T07:23:10.604 INFO:teuthology.orchestra.run.smithi143.stdout: "default_value": "False", 2023-11-26T07:23:10.604 INFO:teuthology.orchestra.run.smithi143.stdout: "min": "", 2023-11-26T07:23:10.604 INFO:teuthology.orchestra.run.smithi143.stdout: "max": "", 2023-11-26T07:23:10.604 INFO:teuthology.orchestra.run.smithi143.stdout: "enum_allowed": [], 2023-11-26T07:23:10.605 INFO:teuthology.orchestra.run.smithi143.stdout: "desc": "", 2023-11-26T07:23:10.605 INFO:teuthology.orchestra.run.smithi143.stdout: "long_desc": "", 2023-11-26T07:23:10.605 INFO:teuthology.orchestra.run.smithi143.stdout: "tags": [], 2023-11-26T07:23:10.605 INFO:teuthology.orchestra.run.smithi143.stdout: "see_also": [] 2023-11-26T07:23:10.605 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2023-11-26T07:23:10.605 INFO:teuthology.orchestra.run.smithi143.stdout: "USER_PWD_EXPIRATION_SPAN": { 2023-11-26T07:23:10.605 INFO:teuthology.orchestra.run.smithi143.stdout: "name": "USER_PWD_EXPIRATION_SPAN", 2023-11-26T07:23:10.605 INFO:teuthology.orchestra.run.smithi143.stdout: "type": "int", 2023-11-26T07:23:10.605 INFO:teuthology.orchestra.run.smithi143.stdout: "level": "advanced", 2023-11-26T07:23:10.605 INFO:teuthology.orchestra.run.smithi143.stdout: "flags": 0, 2023-11-26T07:23:10.605 INFO:teuthology.orchestra.run.smithi143.stdout: "default_value": "0", 2023-11-26T07:23:10.605 INFO:teuthology.orchestra.run.smithi143.stdout: "min": "", 2023-11-26T07:23:10.605 INFO:teuthology.orchestra.run.smithi143.stdout: "max": "", 2023-11-26T07:23:10.605 INFO:teuthology.orchestra.run.smithi143.stdout: "enum_allowed": [], 2023-11-26T07:23:10.606 INFO:teuthology.orchestra.run.smithi143.stdout: "desc": "", 2023-11-26T07:23:10.606 INFO:teuthology.orchestra.run.smithi143.stdout: "long_desc": "", 2023-11-26T07:23:10.606 INFO:teuthology.orchestra.run.smithi143.stdout: "tags": [], 2023-11-26T07:23:10.606 INFO:teuthology.orchestra.run.smithi143.stdout: "see_also": [] 2023-11-26T07:23:10.606 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2023-11-26T07:23:10.606 INFO:teuthology.orchestra.run.smithi143.stdout: "USER_PWD_EXPIRATION_WARNING_1": { 2023-11-26T07:23:10.606 INFO:teuthology.orchestra.run.smithi143.stdout: "name": "USER_PWD_EXPIRATION_WARNING_1", 2023-11-26T07:23:10.606 INFO:teuthology.orchestra.run.smithi143.stdout: "type": "int", 2023-11-26T07:23:10.606 INFO:teuthology.orchestra.run.smithi143.stdout: "level": "advanced", 2023-11-26T07:23:10.606 INFO:teuthology.orchestra.run.smithi143.stdout: "flags": 0, 2023-11-26T07:23:10.606 INFO:teuthology.orchestra.run.smithi143.stdout: "default_value": "10", 2023-11-26T07:23:10.607 INFO:teuthology.orchestra.run.smithi143.stdout: "min": "", 2023-11-26T07:23:10.607 INFO:teuthology.orchestra.run.smithi143.stdout: "max": "", 2023-11-26T07:23:10.607 INFO:teuthology.orchestra.run.smithi143.stdout: "enum_allowed": [], 2023-11-26T07:23:10.607 INFO:teuthology.orchestra.run.smithi143.stdout: "desc": "", 2023-11-26T07:23:10.607 INFO:teuthology.orchestra.run.smithi143.stdout: "long_desc": "", 2023-11-26T07:23:10.607 INFO:teuthology.orchestra.run.smithi143.stdout: "tags": [], 2023-11-26T07:23:10.607 INFO:teuthology.orchestra.run.smithi143.stdout: "see_also": [] 2023-11-26T07:23:10.607 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2023-11-26T07:23:10.607 INFO:teuthology.orchestra.run.smithi143.stdout: "USER_PWD_EXPIRATION_WARNING_2": { 2023-11-26T07:23:10.607 INFO:teuthology.orchestra.run.smithi143.stdout: "name": "USER_PWD_EXPIRATION_WARNING_2", 2023-11-26T07:23:10.607 INFO:teuthology.orchestra.run.smithi143.stdout: "type": "int", 2023-11-26T07:23:10.607 INFO:teuthology.orchestra.run.smithi143.stdout: "level": "advanced", 2023-11-26T07:23:10.607 INFO:teuthology.orchestra.run.smithi143.stdout: "flags": 0, 2023-11-26T07:23:10.608 INFO:teuthology.orchestra.run.smithi143.stdout: "default_value": "5", 2023-11-26T07:23:10.608 INFO:teuthology.orchestra.run.smithi143.stdout: "min": "", 2023-11-26T07:23:10.608 INFO:teuthology.orchestra.run.smithi143.stdout: "max": "", 2023-11-26T07:23:10.608 INFO:teuthology.orchestra.run.smithi143.stdout: "enum_allowed": [], 2023-11-26T07:23:10.608 INFO:teuthology.orchestra.run.smithi143.stdout: "desc": "", 2023-11-26T07:23:10.608 INFO:teuthology.orchestra.run.smithi143.stdout: "long_desc": "", 2023-11-26T07:23:10.608 INFO:teuthology.orchestra.run.smithi143.stdout: "tags": [], 2023-11-26T07:23:10.608 INFO:teuthology.orchestra.run.smithi143.stdout: "see_also": [] 2023-11-26T07:23:10.608 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2023-11-26T07:23:10.608 INFO:teuthology.orchestra.run.smithi143.stdout: "cross_origin_url": { 2023-11-26T07:23:10.608 INFO:teuthology.orchestra.run.smithi143.stdout: "name": "cross_origin_url", 2023-11-26T07:23:10.608 INFO:teuthology.orchestra.run.smithi143.stdout: "type": "str", 2023-11-26T07:23:10.608 INFO:teuthology.orchestra.run.smithi143.stdout: "level": "advanced", 2023-11-26T07:23:10.609 INFO:teuthology.orchestra.run.smithi143.stdout: "flags": 0, 2023-11-26T07:23:10.609 INFO:teuthology.orchestra.run.smithi143.stdout: "default_value": "", 2023-11-26T07:23:10.609 INFO:teuthology.orchestra.run.smithi143.stdout: "min": "", 2023-11-26T07:23:10.609 INFO:teuthology.orchestra.run.smithi143.stdout: "max": "", 2023-11-26T07:23:10.609 INFO:teuthology.orchestra.run.smithi143.stdout: "enum_allowed": [], 2023-11-26T07:23:10.609 INFO:teuthology.orchestra.run.smithi143.stdout: "desc": "", 2023-11-26T07:23:10.609 INFO:teuthology.orchestra.run.smithi143.stdout: "long_desc": "", 2023-11-26T07:23:10.609 INFO:teuthology.orchestra.run.smithi143.stdout: "tags": [], 2023-11-26T07:23:10.609 INFO:teuthology.orchestra.run.smithi143.stdout: "see_also": [] 2023-11-26T07:23:10.609 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2023-11-26T07:23:10.609 INFO:teuthology.orchestra.run.smithi143.stdout: "crt_file": { 2023-11-26T07:23:10.609 INFO:teuthology.orchestra.run.smithi143.stdout: "name": "crt_file", 2023-11-26T07:23:10.609 INFO:teuthology.orchestra.run.smithi143.stdout: "type": "str", 2023-11-26T07:23:10.610 INFO:teuthology.orchestra.run.smithi143.stdout: "level": "advanced", 2023-11-26T07:23:10.610 INFO:teuthology.orchestra.run.smithi143.stdout: "flags": 0, 2023-11-26T07:23:10.610 INFO:teuthology.orchestra.run.smithi143.stdout: "default_value": "", 2023-11-26T07:23:10.610 INFO:teuthology.orchestra.run.smithi143.stdout: "min": "", 2023-11-26T07:23:10.610 INFO:teuthology.orchestra.run.smithi143.stdout: "max": "", 2023-11-26T07:23:10.610 INFO:teuthology.orchestra.run.smithi143.stdout: "enum_allowed": [], 2023-11-26T07:23:10.610 INFO:teuthology.orchestra.run.smithi143.stdout: "desc": "", 2023-11-26T07:23:10.610 INFO:teuthology.orchestra.run.smithi143.stdout: "long_desc": "", 2023-11-26T07:23:10.610 INFO:teuthology.orchestra.run.smithi143.stdout: "tags": [], 2023-11-26T07:23:10.610 INFO:teuthology.orchestra.run.smithi143.stdout: "see_also": [] 2023-11-26T07:23:10.610 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2023-11-26T07:23:10.610 INFO:teuthology.orchestra.run.smithi143.stdout: "debug": { 2023-11-26T07:23:10.610 INFO:teuthology.orchestra.run.smithi143.stdout: "name": "debug", 2023-11-26T07:23:10.611 INFO:teuthology.orchestra.run.smithi143.stdout: "type": "bool", 2023-11-26T07:23:10.611 INFO:teuthology.orchestra.run.smithi143.stdout: "level": "advanced", 2023-11-26T07:23:10.611 INFO:teuthology.orchestra.run.smithi143.stdout: "flags": 0, 2023-11-26T07:23:10.611 INFO:teuthology.orchestra.run.smithi143.stdout: "default_value": "False", 2023-11-26T07:23:10.611 INFO:teuthology.orchestra.run.smithi143.stdout: "min": "", 2023-11-26T07:23:10.611 INFO:teuthology.orchestra.run.smithi143.stdout: "max": "", 2023-11-26T07:23:10.611 INFO:teuthology.orchestra.run.smithi143.stdout: "enum_allowed": [], 2023-11-26T07:23:10.611 INFO:teuthology.orchestra.run.smithi143.stdout: "desc": "Enable/disable debug options", 2023-11-26T07:23:10.611 INFO:teuthology.orchestra.run.smithi143.stdout: "long_desc": "", 2023-11-26T07:23:10.611 INFO:teuthology.orchestra.run.smithi143.stdout: "tags": [], 2023-11-26T07:23:10.611 INFO:teuthology.orchestra.run.smithi143.stdout: "see_also": [] 2023-11-26T07:23:10.611 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2023-11-26T07:23:10.611 INFO:teuthology.orchestra.run.smithi143.stdout: "jwt_token_ttl": { 2023-11-26T07:23:10.612 INFO:teuthology.orchestra.run.smithi143.stdout: "name": "jwt_token_ttl", 2023-11-26T07:23:10.612 INFO:teuthology.orchestra.run.smithi143.stdout: "type": "int", 2023-11-26T07:23:10.612 INFO:teuthology.orchestra.run.smithi143.stdout: "level": "advanced", 2023-11-26T07:23:10.612 INFO:teuthology.orchestra.run.smithi143.stdout: "flags": 0, 2023-11-26T07:23:10.612 INFO:teuthology.orchestra.run.smithi143.stdout: "default_value": "28800", 2023-11-26T07:23:10.612 INFO:teuthology.orchestra.run.smithi143.stdout: "min": "", 2023-11-26T07:23:10.612 INFO:teuthology.orchestra.run.smithi143.stdout: "max": "", 2023-11-26T07:23:10.612 INFO:teuthology.orchestra.run.smithi143.stdout: "enum_allowed": [], 2023-11-26T07:23:10.612 INFO:teuthology.orchestra.run.smithi143.stdout: "desc": "", 2023-11-26T07:23:10.612 INFO:teuthology.orchestra.run.smithi143.stdout: "long_desc": "", 2023-11-26T07:23:10.612 INFO:teuthology.orchestra.run.smithi143.stdout: "tags": [], 2023-11-26T07:23:10.612 INFO:teuthology.orchestra.run.smithi143.stdout: "see_also": [] 2023-11-26T07:23:10.613 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2023-11-26T07:23:10.613 INFO:teuthology.orchestra.run.smithi143.stdout: "key_file": { 2023-11-26T07:23:10.613 INFO:teuthology.orchestra.run.smithi143.stdout: "name": "key_file", 2023-11-26T07:23:10.613 INFO:teuthology.orchestra.run.smithi143.stdout: "type": "str", 2023-11-26T07:23:10.613 INFO:teuthology.orchestra.run.smithi143.stdout: "level": "advanced", 2023-11-26T07:23:10.613 INFO:teuthology.orchestra.run.smithi143.stdout: "flags": 0, 2023-11-26T07:23:10.613 INFO:teuthology.orchestra.run.smithi143.stdout: "default_value": "", 2023-11-26T07:23:10.613 INFO:teuthology.orchestra.run.smithi143.stdout: "min": "", 2023-11-26T07:23:10.613 INFO:teuthology.orchestra.run.smithi143.stdout: "max": "", 2023-11-26T07:23:10.613 INFO:teuthology.orchestra.run.smithi143.stdout: "enum_allowed": [], 2023-11-26T07:23:10.613 INFO:teuthology.orchestra.run.smithi143.stdout: "desc": "", 2023-11-26T07:23:10.613 INFO:teuthology.orchestra.run.smithi143.stdout: "long_desc": "", 2023-11-26T07:23:10.613 INFO:teuthology.orchestra.run.smithi143.stdout: "tags": [], 2023-11-26T07:23:10.614 INFO:teuthology.orchestra.run.smithi143.stdout: "see_also": [] 2023-11-26T07:23:10.614 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2023-11-26T07:23:10.614 INFO:teuthology.orchestra.run.smithi143.stdout: "log_level": { 2023-11-26T07:23:10.614 INFO:teuthology.orchestra.run.smithi143.stdout: "name": "log_level", 2023-11-26T07:23:10.614 INFO:teuthology.orchestra.run.smithi143.stdout: "type": "str", 2023-11-26T07:23:10.614 INFO:teuthology.orchestra.run.smithi143.stdout: "level": "advanced", 2023-11-26T07:23:10.614 INFO:teuthology.orchestra.run.smithi143.stdout: "flags": 1, 2023-11-26T07:23:10.614 INFO:teuthology.orchestra.run.smithi143.stdout: "default_value": "", 2023-11-26T07:23:10.614 INFO:teuthology.orchestra.run.smithi143.stdout: "min": "", 2023-11-26T07:23:10.614 INFO:teuthology.orchestra.run.smithi143.stdout: "max": "", 2023-11-26T07:23:10.615 INFO:teuthology.orchestra.run.smithi143.stdout: "enum_allowed": [ 2023-11-26T07:23:10.615 INFO:teuthology.orchestra.run.smithi143.stdout: "", 2023-11-26T07:23:10.615 INFO:teuthology.orchestra.run.smithi143.stdout: "critical", 2023-11-26T07:23:10.615 INFO:teuthology.orchestra.run.smithi143.stdout: "debug", 2023-11-26T07:23:10.615 INFO:teuthology.orchestra.run.smithi143.stdout: "error", 2023-11-26T07:23:10.615 INFO:teuthology.orchestra.run.smithi143.stdout: "info", 2023-11-26T07:23:10.615 INFO:teuthology.orchestra.run.smithi143.stdout: "warning" 2023-11-26T07:23:10.615 INFO:teuthology.orchestra.run.smithi143.stdout: ], 2023-11-26T07:23:10.615 INFO:teuthology.orchestra.run.smithi143.stdout: "desc": "", 2023-11-26T07:23:10.615 INFO:teuthology.orchestra.run.smithi143.stdout: "long_desc": "", 2023-11-26T07:23:10.615 INFO:teuthology.orchestra.run.smithi143.stdout: "tags": [], 2023-11-26T07:23:10.615 INFO:teuthology.orchestra.run.smithi143.stdout: "see_also": [] 2023-11-26T07:23:10.615 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2023-11-26T07:23:10.616 INFO:teuthology.orchestra.run.smithi143.stdout: "log_to_cluster": { 2023-11-26T07:23:10.616 INFO:teuthology.orchestra.run.smithi143.stdout: "name": "log_to_cluster", 2023-11-26T07:23:10.616 INFO:teuthology.orchestra.run.smithi143.stdout: "type": "bool", 2023-11-26T07:23:10.616 INFO:teuthology.orchestra.run.smithi143.stdout: "level": "advanced", 2023-11-26T07:23:10.616 INFO:teuthology.orchestra.run.smithi143.stdout: "flags": 1, 2023-11-26T07:23:10.616 INFO:teuthology.orchestra.run.smithi143.stdout: "default_value": "False", 2023-11-26T07:23:10.616 INFO:teuthology.orchestra.run.smithi143.stdout: "min": "", 2023-11-26T07:23:10.616 INFO:teuthology.orchestra.run.smithi143.stdout: "max": "", 2023-11-26T07:23:10.616 INFO:teuthology.orchestra.run.smithi143.stdout: "enum_allowed": [], 2023-11-26T07:23:10.616 INFO:teuthology.orchestra.run.smithi143.stdout: "desc": "", 2023-11-26T07:23:10.616 INFO:teuthology.orchestra.run.smithi143.stdout: "long_desc": "", 2023-11-26T07:23:10.616 INFO:teuthology.orchestra.run.smithi143.stdout: "tags": [], 2023-11-26T07:23:10.617 INFO:teuthology.orchestra.run.smithi143.stdout: "see_also": [] 2023-11-26T07:23:10.617 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2023-11-26T07:23:10.617 INFO:teuthology.orchestra.run.smithi143.stdout: "log_to_cluster_level": { 2023-11-26T07:23:10.617 INFO:teuthology.orchestra.run.smithi143.stdout: "name": "log_to_cluster_level", 2023-11-26T07:23:10.617 INFO:teuthology.orchestra.run.smithi143.stdout: "type": "str", 2023-11-26T07:23:10.617 INFO:teuthology.orchestra.run.smithi143.stdout: "level": "advanced", 2023-11-26T07:23:10.617 INFO:teuthology.orchestra.run.smithi143.stdout: "flags": 1, 2023-11-26T07:23:10.617 INFO:teuthology.orchestra.run.smithi143.stdout: "default_value": "info", 2023-11-26T07:23:10.617 INFO:teuthology.orchestra.run.smithi143.stdout: "min": "", 2023-11-26T07:23:10.617 INFO:teuthology.orchestra.run.smithi143.stdout: "max": "", 2023-11-26T07:23:10.617 INFO:teuthology.orchestra.run.smithi143.stdout: "enum_allowed": [ 2023-11-26T07:23:10.617 INFO:teuthology.orchestra.run.smithi143.stdout: "", 2023-11-26T07:23:10.617 INFO:teuthology.orchestra.run.smithi143.stdout: "critical", 2023-11-26T07:23:10.618 INFO:teuthology.orchestra.run.smithi143.stdout: "debug", 2023-11-26T07:23:10.618 INFO:teuthology.orchestra.run.smithi143.stdout: "error", 2023-11-26T07:23:10.618 INFO:teuthology.orchestra.run.smithi143.stdout: "info", 2023-11-26T07:23:10.618 INFO:teuthology.orchestra.run.smithi143.stdout: "warning" 2023-11-26T07:23:10.618 INFO:teuthology.orchestra.run.smithi143.stdout: ], 2023-11-26T07:23:10.618 INFO:teuthology.orchestra.run.smithi143.stdout: "desc": "", 2023-11-26T07:23:10.618 INFO:teuthology.orchestra.run.smithi143.stdout: "long_desc": "", 2023-11-26T07:23:10.618 INFO:teuthology.orchestra.run.smithi143.stdout: "tags": [], 2023-11-26T07:23:10.618 INFO:teuthology.orchestra.run.smithi143.stdout: "see_also": [] 2023-11-26T07:23:10.618 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2023-11-26T07:23:10.618 INFO:teuthology.orchestra.run.smithi143.stdout: "log_to_file": { 2023-11-26T07:23:10.618 INFO:teuthology.orchestra.run.smithi143.stdout: "name": "log_to_file", 2023-11-26T07:23:10.619 INFO:teuthology.orchestra.run.smithi143.stdout: "type": "bool", 2023-11-26T07:23:10.619 INFO:teuthology.orchestra.run.smithi143.stdout: "level": "advanced", 2023-11-26T07:23:10.619 INFO:teuthology.orchestra.run.smithi143.stdout: "flags": 1, 2023-11-26T07:23:10.619 INFO:teuthology.orchestra.run.smithi143.stdout: "default_value": "False", 2023-11-26T07:23:10.619 INFO:teuthology.orchestra.run.smithi143.stdout: "min": "", 2023-11-26T07:23:10.619 INFO:teuthology.orchestra.run.smithi143.stdout: "max": "", 2023-11-26T07:23:10.619 INFO:teuthology.orchestra.run.smithi143.stdout: "enum_allowed": [], 2023-11-26T07:23:10.619 INFO:teuthology.orchestra.run.smithi143.stdout: "desc": "", 2023-11-26T07:23:10.619 INFO:teuthology.orchestra.run.smithi143.stdout: "long_desc": "", 2023-11-26T07:23:10.619 INFO:teuthology.orchestra.run.smithi143.stdout: "tags": [], 2023-11-26T07:23:10.619 INFO:teuthology.orchestra.run.smithi143.stdout: "see_also": [] 2023-11-26T07:23:10.619 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2023-11-26T07:23:10.619 INFO:teuthology.orchestra.run.smithi143.stdout: "motd": { 2023-11-26T07:23:10.620 INFO:teuthology.orchestra.run.smithi143.stdout: "name": "motd", 2023-11-26T07:23:10.620 INFO:teuthology.orchestra.run.smithi143.stdout: "type": "str", 2023-11-26T07:23:10.620 INFO:teuthology.orchestra.run.smithi143.stdout: "level": "advanced", 2023-11-26T07:23:10.620 INFO:teuthology.orchestra.run.smithi143.stdout: "flags": 0, 2023-11-26T07:23:10.620 INFO:teuthology.orchestra.run.smithi143.stdout: "default_value": "", 2023-11-26T07:23:10.620 INFO:teuthology.orchestra.run.smithi143.stdout: "min": "", 2023-11-26T07:23:10.620 INFO:teuthology.orchestra.run.smithi143.stdout: "max": "", 2023-11-26T07:23:10.620 INFO:teuthology.orchestra.run.smithi143.stdout: "enum_allowed": [], 2023-11-26T07:23:10.620 INFO:teuthology.orchestra.run.smithi143.stdout: "desc": "The message of the day", 2023-11-26T07:23:10.620 INFO:teuthology.orchestra.run.smithi143.stdout: "long_desc": "", 2023-11-26T07:23:10.620 INFO:teuthology.orchestra.run.smithi143.stdout: "tags": [], 2023-11-26T07:23:10.620 INFO:teuthology.orchestra.run.smithi143.stdout: "see_also": [] 2023-11-26T07:23:10.621 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2023-11-26T07:23:10.621 INFO:teuthology.orchestra.run.smithi143.stdout: "redirect_resolve_ip_addr": { 2023-11-26T07:23:10.621 INFO:teuthology.orchestra.run.smithi143.stdout: "name": "redirect_resolve_ip_addr", 2023-11-26T07:23:10.621 INFO:teuthology.orchestra.run.smithi143.stdout: "type": "bool", 2023-11-26T07:23:10.621 INFO:teuthology.orchestra.run.smithi143.stdout: "level": "advanced", 2023-11-26T07:23:10.621 INFO:teuthology.orchestra.run.smithi143.stdout: "flags": 0, 2023-11-26T07:23:10.621 INFO:teuthology.orchestra.run.smithi143.stdout: "default_value": "False", 2023-11-26T07:23:10.621 INFO:teuthology.orchestra.run.smithi143.stdout: "min": "", 2023-11-26T07:23:10.621 INFO:teuthology.orchestra.run.smithi143.stdout: "max": "", 2023-11-26T07:23:10.621 INFO:teuthology.orchestra.run.smithi143.stdout: "enum_allowed": [], 2023-11-26T07:23:10.621 INFO:teuthology.orchestra.run.smithi143.stdout: "desc": "", 2023-11-26T07:23:10.622 INFO:teuthology.orchestra.run.smithi143.stdout: "long_desc": "", 2023-11-26T07:23:10.622 INFO:teuthology.orchestra.run.smithi143.stdout: "tags": [], 2023-11-26T07:23:10.622 INFO:teuthology.orchestra.run.smithi143.stdout: "see_also": [] 2023-11-26T07:23:10.622 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2023-11-26T07:23:10.622 INFO:teuthology.orchestra.run.smithi143.stdout: "server_addr": { 2023-11-26T07:23:10.622 INFO:teuthology.orchestra.run.smithi143.stdout: "name": "server_addr", 2023-11-26T07:23:10.622 INFO:teuthology.orchestra.run.smithi143.stdout: "type": "str", 2023-11-26T07:23:10.622 INFO:teuthology.orchestra.run.smithi143.stdout: "level": "advanced", 2023-11-26T07:23:10.622 INFO:teuthology.orchestra.run.smithi143.stdout: "flags": 0, 2023-11-26T07:23:10.622 INFO:teuthology.orchestra.run.smithi143.stdout: "default_value": "::", 2023-11-26T07:23:10.622 INFO:teuthology.orchestra.run.smithi143.stdout: "min": "", 2023-11-26T07:23:10.622 INFO:teuthology.orchestra.run.smithi143.stdout: "max": "", 2023-11-26T07:23:10.622 INFO:teuthology.orchestra.run.smithi143.stdout: "enum_allowed": [], 2023-11-26T07:23:10.623 INFO:teuthology.orchestra.run.smithi143.stdout: "desc": "", 2023-11-26T07:23:10.623 INFO:teuthology.orchestra.run.smithi143.stdout: "long_desc": "", 2023-11-26T07:23:10.623 INFO:teuthology.orchestra.run.smithi143.stdout: "tags": [], 2023-11-26T07:23:10.623 INFO:teuthology.orchestra.run.smithi143.stdout: "see_also": [] 2023-11-26T07:23:10.623 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2023-11-26T07:23:10.623 INFO:teuthology.orchestra.run.smithi143.stdout: "server_port": { 2023-11-26T07:23:10.623 INFO:teuthology.orchestra.run.smithi143.stdout: "name": "server_port", 2023-11-26T07:23:10.623 INFO:teuthology.orchestra.run.smithi143.stdout: "type": "int", 2023-11-26T07:23:10.623 INFO:teuthology.orchestra.run.smithi143.stdout: "level": "advanced", 2023-11-26T07:23:10.623 INFO:teuthology.orchestra.run.smithi143.stdout: "flags": 0, 2023-11-26T07:23:10.623 INFO:teuthology.orchestra.run.smithi143.stdout: "default_value": "8080", 2023-11-26T07:23:10.623 INFO:teuthology.orchestra.run.smithi143.stdout: "min": "", 2023-11-26T07:23:10.623 INFO:teuthology.orchestra.run.smithi143.stdout: "max": "", 2023-11-26T07:23:10.624 INFO:teuthology.orchestra.run.smithi143.stdout: "enum_allowed": [], 2023-11-26T07:23:10.624 INFO:teuthology.orchestra.run.smithi143.stdout: "desc": "", 2023-11-26T07:23:10.624 INFO:teuthology.orchestra.run.smithi143.stdout: "long_desc": "", 2023-11-26T07:23:10.624 INFO:teuthology.orchestra.run.smithi143.stdout: "tags": [], 2023-11-26T07:23:10.624 INFO:teuthology.orchestra.run.smithi143.stdout: "see_also": [] 2023-11-26T07:23:10.624 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2023-11-26T07:23:10.624 INFO:teuthology.orchestra.run.smithi143.stdout: "ssl": { 2023-11-26T07:23:10.624 INFO:teuthology.orchestra.run.smithi143.stdout: "name": "ssl", 2023-11-26T07:23:10.624 INFO:teuthology.orchestra.run.smithi143.stdout: "type": "bool", 2023-11-26T07:23:10.624 INFO:teuthology.orchestra.run.smithi143.stdout: "level": "advanced", 2023-11-26T07:23:10.624 INFO:teuthology.orchestra.run.smithi143.stdout: "flags": 0, 2023-11-26T07:23:10.624 INFO:teuthology.orchestra.run.smithi143.stdout: "default_value": "True", 2023-11-26T07:23:10.624 INFO:teuthology.orchestra.run.smithi143.stdout: "min": "", 2023-11-26T07:23:10.625 INFO:teuthology.orchestra.run.smithi143.stdout: "max": "", 2023-11-26T07:23:10.625 INFO:teuthology.orchestra.run.smithi143.stdout: "enum_allowed": [], 2023-11-26T07:23:10.625 INFO:teuthology.orchestra.run.smithi143.stdout: "desc": "", 2023-11-26T07:23:10.625 INFO:teuthology.orchestra.run.smithi143.stdout: "long_desc": "", 2023-11-26T07:23:10.625 INFO:teuthology.orchestra.run.smithi143.stdout: "tags": [], 2023-11-26T07:23:10.625 INFO:teuthology.orchestra.run.smithi143.stdout: "see_also": [] 2023-11-26T07:23:10.625 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2023-11-26T07:23:10.625 INFO:teuthology.orchestra.run.smithi143.stdout: "ssl_server_port": { 2023-11-26T07:23:10.625 INFO:teuthology.orchestra.run.smithi143.stdout: "name": "ssl_server_port", 2023-11-26T07:23:10.625 INFO:teuthology.orchestra.run.smithi143.stdout: "type": "int", 2023-11-26T07:23:10.625 INFO:teuthology.orchestra.run.smithi143.stdout: "level": "advanced", 2023-11-26T07:23:10.625 INFO:teuthology.orchestra.run.smithi143.stdout: "flags": 0, 2023-11-26T07:23:10.625 INFO:teuthology.orchestra.run.smithi143.stdout: "default_value": "8443", 2023-11-26T07:23:10.626 INFO:teuthology.orchestra.run.smithi143.stdout: "min": "", 2023-11-26T07:23:10.626 INFO:teuthology.orchestra.run.smithi143.stdout: "max": "", 2023-11-26T07:23:10.626 INFO:teuthology.orchestra.run.smithi143.stdout: "enum_allowed": [], 2023-11-26T07:23:10.626 INFO:teuthology.orchestra.run.smithi143.stdout: "desc": "", 2023-11-26T07:23:10.626 INFO:teuthology.orchestra.run.smithi143.stdout: "long_desc": "", 2023-11-26T07:23:10.626 INFO:teuthology.orchestra.run.smithi143.stdout: "tags": [], 2023-11-26T07:23:10.626 INFO:teuthology.orchestra.run.smithi143.stdout: "see_also": [] 2023-11-26T07:23:10.626 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2023-11-26T07:23:10.626 INFO:teuthology.orchestra.run.smithi143.stdout: "standby_behaviour": { 2023-11-26T07:23:10.626 INFO:teuthology.orchestra.run.smithi143.stdout: "name": "standby_behaviour", 2023-11-26T07:23:10.626 INFO:teuthology.orchestra.run.smithi143.stdout: "type": "str", 2023-11-26T07:23:10.627 INFO:teuthology.orchestra.run.smithi143.stdout: "level": "advanced", 2023-11-26T07:23:10.627 INFO:teuthology.orchestra.run.smithi143.stdout: "flags": 0, 2023-11-26T07:23:10.627 INFO:teuthology.orchestra.run.smithi143.stdout: "default_value": "redirect", 2023-11-26T07:23:10.627 INFO:teuthology.orchestra.run.smithi143.stdout: "min": "", 2023-11-26T07:23:10.627 INFO:teuthology.orchestra.run.smithi143.stdout: "max": "", 2023-11-26T07:23:10.627 INFO:teuthology.orchestra.run.smithi143.stdout: "enum_allowed": [ 2023-11-26T07:23:10.627 INFO:teuthology.orchestra.run.smithi143.stdout: "error", 2023-11-26T07:23:10.627 INFO:teuthology.orchestra.run.smithi143.stdout: "redirect" 2023-11-26T07:23:10.627 INFO:teuthology.orchestra.run.smithi143.stdout: ], 2023-11-26T07:23:10.627 INFO:teuthology.orchestra.run.smithi143.stdout: "desc": "", 2023-11-26T07:23:10.627 INFO:teuthology.orchestra.run.smithi143.stdout: "long_desc": "", 2023-11-26T07:23:10.627 INFO:teuthology.orchestra.run.smithi143.stdout: "tags": [], 2023-11-26T07:23:10.627 INFO:teuthology.orchestra.run.smithi143.stdout: "see_also": [] 2023-11-26T07:23:10.628 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2023-11-26T07:23:10.628 INFO:teuthology.orchestra.run.smithi143.stdout: "standby_error_status_code": { 2023-11-26T07:23:10.628 INFO:teuthology.orchestra.run.smithi143.stdout: "name": "standby_error_status_code", 2023-11-26T07:23:10.628 INFO:teuthology.orchestra.run.smithi143.stdout: "type": "int", 2023-11-26T07:23:10.628 INFO:teuthology.orchestra.run.smithi143.stdout: "level": "advanced", 2023-11-26T07:23:10.628 INFO:teuthology.orchestra.run.smithi143.stdout: "flags": 0, 2023-11-26T07:23:10.628 INFO:teuthology.orchestra.run.smithi143.stdout: "default_value": "500", 2023-11-26T07:23:10.628 INFO:teuthology.orchestra.run.smithi143.stdout: "min": "400", 2023-11-26T07:23:10.628 INFO:teuthology.orchestra.run.smithi143.stdout: "max": "599", 2023-11-26T07:23:10.628 INFO:teuthology.orchestra.run.smithi143.stdout: "enum_allowed": [], 2023-11-26T07:23:10.628 INFO:teuthology.orchestra.run.smithi143.stdout: "desc": "", 2023-11-26T07:23:10.628 INFO:teuthology.orchestra.run.smithi143.stdout: "long_desc": "", 2023-11-26T07:23:10.628 INFO:teuthology.orchestra.run.smithi143.stdout: "tags": [], 2023-11-26T07:23:10.629 INFO:teuthology.orchestra.run.smithi143.stdout: "see_also": [] 2023-11-26T07:23:10.629 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2023-11-26T07:23:10.629 INFO:teuthology.orchestra.run.smithi143.stdout: "url_prefix": { 2023-11-26T07:23:10.629 INFO:teuthology.orchestra.run.smithi143.stdout: "name": "url_prefix", 2023-11-26T07:23:10.629 INFO:teuthology.orchestra.run.smithi143.stdout: "type": "str", 2023-11-26T07:23:10.629 INFO:teuthology.orchestra.run.smithi143.stdout: "level": "advanced", 2023-11-26T07:23:10.629 INFO:teuthology.orchestra.run.smithi143.stdout: "flags": 0, 2023-11-26T07:23:10.629 INFO:teuthology.orchestra.run.smithi143.stdout: "default_value": "", 2023-11-26T07:23:10.629 INFO:teuthology.orchestra.run.smithi143.stdout: "min": "", 2023-11-26T07:23:10.629 INFO:teuthology.orchestra.run.smithi143.stdout: "max": "", 2023-11-26T07:23:10.629 INFO:teuthology.orchestra.run.smithi143.stdout: "enum_allowed": [], 2023-11-26T07:23:10.629 INFO:teuthology.orchestra.run.smithi143.stdout: "desc": "", 2023-11-26T07:23:10.629 INFO:teuthology.orchestra.run.smithi143.stdout: "long_desc": "", 2023-11-26T07:23:10.630 INFO:teuthology.orchestra.run.smithi143.stdout: "tags": [], 2023-11-26T07:23:10.630 INFO:teuthology.orchestra.run.smithi143.stdout: "see_also": [] 2023-11-26T07:23:10.630 INFO:teuthology.orchestra.run.smithi143.stdout: } 2023-11-26T07:23:10.630 INFO:teuthology.orchestra.run.smithi143.stdout: } 2023-11-26T07:23:10.630 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2023-11-26T07:23:10.630 INFO:teuthology.orchestra.run.smithi143.stdout: { 2023-11-26T07:23:10.630 INFO:teuthology.orchestra.run.smithi143.stdout: "name": "devicehealth", 2023-11-26T07:23:10.630 INFO:teuthology.orchestra.run.smithi143.stdout: "can_run": true, 2023-11-26T07:23:10.630 INFO:teuthology.orchestra.run.smithi143.stdout: "error_string": "", 2023-11-26T07:23:10.630 INFO:teuthology.orchestra.run.smithi143.stdout: "module_options": { 2023-11-26T07:23:10.630 INFO:teuthology.orchestra.run.smithi143.stdout: "enable_monitoring": { 2023-11-26T07:23:10.630 INFO:teuthology.orchestra.run.smithi143.stdout: "name": "enable_monitoring", 2023-11-26T07:23:10.631 INFO:teuthology.orchestra.run.smithi143.stdout: "type": "bool", 2023-11-26T07:23:10.631 INFO:teuthology.orchestra.run.smithi143.stdout: "level": "advanced", 2023-11-26T07:23:10.631 INFO:teuthology.orchestra.run.smithi143.stdout: "flags": 1, 2023-11-26T07:23:10.631 INFO:teuthology.orchestra.run.smithi143.stdout: "default_value": "True", 2023-11-26T07:23:10.631 INFO:teuthology.orchestra.run.smithi143.stdout: "min": "", 2023-11-26T07:23:10.631 INFO:teuthology.orchestra.run.smithi143.stdout: "max": "", 2023-11-26T07:23:10.631 INFO:teuthology.orchestra.run.smithi143.stdout: "enum_allowed": [], 2023-11-26T07:23:10.631 INFO:teuthology.orchestra.run.smithi143.stdout: "desc": "monitor device health metrics", 2023-11-26T07:23:10.631 INFO:teuthology.orchestra.run.smithi143.stdout: "long_desc": "", 2023-11-26T07:23:10.631 INFO:teuthology.orchestra.run.smithi143.stdout: "tags": [], 2023-11-26T07:23:10.631 INFO:teuthology.orchestra.run.smithi143.stdout: "see_also": [] 2023-11-26T07:23:10.631 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2023-11-26T07:23:10.631 INFO:teuthology.orchestra.run.smithi143.stdout: "log_level": { 2023-11-26T07:23:10.632 INFO:teuthology.orchestra.run.smithi143.stdout: "name": "log_level", 2023-11-26T07:23:10.632 INFO:teuthology.orchestra.run.smithi143.stdout: "type": "str", 2023-11-26T07:23:10.632 INFO:teuthology.orchestra.run.smithi143.stdout: "level": "advanced", 2023-11-26T07:23:10.632 INFO:teuthology.orchestra.run.smithi143.stdout: "flags": 1, 2023-11-26T07:23:10.632 INFO:teuthology.orchestra.run.smithi143.stdout: "default_value": "", 2023-11-26T07:23:10.632 INFO:teuthology.orchestra.run.smithi143.stdout: "min": "", 2023-11-26T07:23:10.632 INFO:teuthology.orchestra.run.smithi143.stdout: "max": "", 2023-11-26T07:23:10.632 INFO:teuthology.orchestra.run.smithi143.stdout: "enum_allowed": [ 2023-11-26T07:23:10.632 INFO:teuthology.orchestra.run.smithi143.stdout: "", 2023-11-26T07:23:10.632 INFO:teuthology.orchestra.run.smithi143.stdout: "critical", 2023-11-26T07:23:10.632 INFO:teuthology.orchestra.run.smithi143.stdout: "debug", 2023-11-26T07:23:10.632 INFO:teuthology.orchestra.run.smithi143.stdout: "error", 2023-11-26T07:23:10.632 INFO:teuthology.orchestra.run.smithi143.stdout: "info", 2023-11-26T07:23:10.633 INFO:teuthology.orchestra.run.smithi143.stdout: "warning" 2023-11-26T07:23:10.633 INFO:teuthology.orchestra.run.smithi143.stdout: ], 2023-11-26T07:23:10.633 INFO:teuthology.orchestra.run.smithi143.stdout: "desc": "", 2023-11-26T07:23:10.633 INFO:teuthology.orchestra.run.smithi143.stdout: "long_desc": "", 2023-11-26T07:23:10.633 INFO:teuthology.orchestra.run.smithi143.stdout: "tags": [], 2023-11-26T07:23:10.633 INFO:teuthology.orchestra.run.smithi143.stdout: "see_also": [] 2023-11-26T07:23:10.633 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2023-11-26T07:23:10.633 INFO:teuthology.orchestra.run.smithi143.stdout: "log_to_cluster": { 2023-11-26T07:23:10.633 INFO:teuthology.orchestra.run.smithi143.stdout: "name": "log_to_cluster", 2023-11-26T07:23:10.633 INFO:teuthology.orchestra.run.smithi143.stdout: "type": "bool", 2023-11-26T07:23:10.633 INFO:teuthology.orchestra.run.smithi143.stdout: "level": "advanced", 2023-11-26T07:23:10.633 INFO:teuthology.orchestra.run.smithi143.stdout: "flags": 1, 2023-11-26T07:23:10.633 INFO:teuthology.orchestra.run.smithi143.stdout: "default_value": "False", 2023-11-26T07:23:10.633 INFO:teuthology.orchestra.run.smithi143.stdout: "min": "", 2023-11-26T07:23:10.634 INFO:teuthology.orchestra.run.smithi143.stdout: "max": "", 2023-11-26T07:23:10.634 INFO:teuthology.orchestra.run.smithi143.stdout: "enum_allowed": [], 2023-11-26T07:23:10.634 INFO:teuthology.orchestra.run.smithi143.stdout: "desc": "", 2023-11-26T07:23:10.634 INFO:teuthology.orchestra.run.smithi143.stdout: "long_desc": "", 2023-11-26T07:23:10.634 INFO:teuthology.orchestra.run.smithi143.stdout: "tags": [], 2023-11-26T07:23:10.634 INFO:teuthology.orchestra.run.smithi143.stdout: "see_also": [] 2023-11-26T07:23:10.634 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2023-11-26T07:23:10.634 INFO:teuthology.orchestra.run.smithi143.stdout: "log_to_cluster_level": { 2023-11-26T07:23:10.634 INFO:teuthology.orchestra.run.smithi143.stdout: "name": "log_to_cluster_level", 2023-11-26T07:23:10.634 INFO:teuthology.orchestra.run.smithi143.stdout: "type": "str", 2023-11-26T07:23:10.634 INFO:teuthology.orchestra.run.smithi143.stdout: "level": "advanced", 2023-11-26T07:23:10.634 INFO:teuthology.orchestra.run.smithi143.stdout: "flags": 1, 2023-11-26T07:23:10.635 INFO:teuthology.orchestra.run.smithi143.stdout: "default_value": "info", 2023-11-26T07:23:10.635 INFO:teuthology.orchestra.run.smithi143.stdout: "min": "", 2023-11-26T07:23:10.635 INFO:teuthology.orchestra.run.smithi143.stdout: "max": "", 2023-11-26T07:23:10.635 INFO:teuthology.orchestra.run.smithi143.stdout: "enum_allowed": [ 2023-11-26T07:23:10.635 INFO:teuthology.orchestra.run.smithi143.stdout: "", 2023-11-26T07:23:10.635 INFO:teuthology.orchestra.run.smithi143.stdout: "critical", 2023-11-26T07:23:10.635 INFO:teuthology.orchestra.run.smithi143.stdout: "debug", 2023-11-26T07:23:10.635 INFO:teuthology.orchestra.run.smithi143.stdout: "error", 2023-11-26T07:23:10.635 INFO:teuthology.orchestra.run.smithi143.stdout: "info", 2023-11-26T07:23:10.635 INFO:teuthology.orchestra.run.smithi143.stdout: "warning" 2023-11-26T07:23:10.635 INFO:teuthology.orchestra.run.smithi143.stdout: ], 2023-11-26T07:23:10.635 INFO:teuthology.orchestra.run.smithi143.stdout: "desc": "", 2023-11-26T07:23:10.635 INFO:teuthology.orchestra.run.smithi143.stdout: "long_desc": "", 2023-11-26T07:23:10.636 INFO:teuthology.orchestra.run.smithi143.stdout: "tags": [], 2023-11-26T07:23:10.636 INFO:teuthology.orchestra.run.smithi143.stdout: "see_also": [] 2023-11-26T07:23:10.636 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2023-11-26T07:23:10.636 INFO:teuthology.orchestra.run.smithi143.stdout: "log_to_file": { 2023-11-26T07:23:10.636 INFO:teuthology.orchestra.run.smithi143.stdout: "name": "log_to_file", 2023-11-26T07:23:10.636 INFO:teuthology.orchestra.run.smithi143.stdout: "type": "bool", 2023-11-26T07:23:10.636 INFO:teuthology.orchestra.run.smithi143.stdout: "level": "advanced", 2023-11-26T07:23:10.636 INFO:teuthology.orchestra.run.smithi143.stdout: "flags": 1, 2023-11-26T07:23:10.636 INFO:teuthology.orchestra.run.smithi143.stdout: "default_value": "False", 2023-11-26T07:23:10.636 INFO:teuthology.orchestra.run.smithi143.stdout: "min": "", 2023-11-26T07:23:10.636 INFO:teuthology.orchestra.run.smithi143.stdout: "max": "", 2023-11-26T07:23:10.636 INFO:teuthology.orchestra.run.smithi143.stdout: "enum_allowed": [], 2023-11-26T07:23:10.636 INFO:teuthology.orchestra.run.smithi143.stdout: "desc": "", 2023-11-26T07:23:10.637 INFO:teuthology.orchestra.run.smithi143.stdout: "long_desc": "", 2023-11-26T07:23:10.637 INFO:teuthology.orchestra.run.smithi143.stdout: "tags": [], 2023-11-26T07:23:10.637 INFO:teuthology.orchestra.run.smithi143.stdout: "see_also": [] 2023-11-26T07:23:10.637 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2023-11-26T07:23:10.637 INFO:teuthology.orchestra.run.smithi143.stdout: "mark_out_threshold": { 2023-11-26T07:23:10.637 INFO:teuthology.orchestra.run.smithi143.stdout: "name": "mark_out_threshold", 2023-11-26T07:23:10.637 INFO:teuthology.orchestra.run.smithi143.stdout: "type": "secs", 2023-11-26T07:23:10.637 INFO:teuthology.orchestra.run.smithi143.stdout: "level": "advanced", 2023-11-26T07:23:10.637 INFO:teuthology.orchestra.run.smithi143.stdout: "flags": 1, 2023-11-26T07:23:10.637 INFO:teuthology.orchestra.run.smithi143.stdout: "default_value": "2419200", 2023-11-26T07:23:10.637 INFO:teuthology.orchestra.run.smithi143.stdout: "min": "", 2023-11-26T07:23:10.637 INFO:teuthology.orchestra.run.smithi143.stdout: "max": "", 2023-11-26T07:23:10.637 INFO:teuthology.orchestra.run.smithi143.stdout: "enum_allowed": [], 2023-11-26T07:23:10.638 INFO:teuthology.orchestra.run.smithi143.stdout: "desc": "automatically mark OSD if it may fail before this long", 2023-11-26T07:23:10.638 INFO:teuthology.orchestra.run.smithi143.stdout: "long_desc": "", 2023-11-26T07:23:10.638 INFO:teuthology.orchestra.run.smithi143.stdout: "tags": [], 2023-11-26T07:23:10.638 INFO:teuthology.orchestra.run.smithi143.stdout: "see_also": [] 2023-11-26T07:23:10.638 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2023-11-26T07:23:10.638 INFO:teuthology.orchestra.run.smithi143.stdout: "pool_name": { 2023-11-26T07:23:10.638 INFO:teuthology.orchestra.run.smithi143.stdout: "name": "pool_name", 2023-11-26T07:23:10.638 INFO:teuthology.orchestra.run.smithi143.stdout: "type": "str", 2023-11-26T07:23:10.638 INFO:teuthology.orchestra.run.smithi143.stdout: "level": "advanced", 2023-11-26T07:23:10.638 INFO:teuthology.orchestra.run.smithi143.stdout: "flags": 1, 2023-11-26T07:23:10.638 INFO:teuthology.orchestra.run.smithi143.stdout: "default_value": "device_health_metrics", 2023-11-26T07:23:10.638 INFO:teuthology.orchestra.run.smithi143.stdout: "min": "", 2023-11-26T07:23:10.639 INFO:teuthology.orchestra.run.smithi143.stdout: "max": "", 2023-11-26T07:23:10.639 INFO:teuthology.orchestra.run.smithi143.stdout: "enum_allowed": [], 2023-11-26T07:23:10.639 INFO:teuthology.orchestra.run.smithi143.stdout: "desc": "name of pool in which to store device health metrics", 2023-11-26T07:23:10.639 INFO:teuthology.orchestra.run.smithi143.stdout: "long_desc": "", 2023-11-26T07:23:10.639 INFO:teuthology.orchestra.run.smithi143.stdout: "tags": [], 2023-11-26T07:23:10.639 INFO:teuthology.orchestra.run.smithi143.stdout: "see_also": [] 2023-11-26T07:23:10.639 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2023-11-26T07:23:10.639 INFO:teuthology.orchestra.run.smithi143.stdout: "retention_period": { 2023-11-26T07:23:10.639 INFO:teuthology.orchestra.run.smithi143.stdout: "name": "retention_period", 2023-11-26T07:23:10.639 INFO:teuthology.orchestra.run.smithi143.stdout: "type": "secs", 2023-11-26T07:23:10.639 INFO:teuthology.orchestra.run.smithi143.stdout: "level": "advanced", 2023-11-26T07:23:10.639 INFO:teuthology.orchestra.run.smithi143.stdout: "flags": 1, 2023-11-26T07:23:10.639 INFO:teuthology.orchestra.run.smithi143.stdout: "default_value": "15552000", 2023-11-26T07:23:10.640 INFO:teuthology.orchestra.run.smithi143.stdout: "min": "", 2023-11-26T07:23:10.640 INFO:teuthology.orchestra.run.smithi143.stdout: "max": "", 2023-11-26T07:23:10.640 INFO:teuthology.orchestra.run.smithi143.stdout: "enum_allowed": [], 2023-11-26T07:23:10.640 INFO:teuthology.orchestra.run.smithi143.stdout: "desc": "how long to retain device health metrics", 2023-11-26T07:23:10.640 INFO:teuthology.orchestra.run.smithi143.stdout: "long_desc": "", 2023-11-26T07:23:10.640 INFO:teuthology.orchestra.run.smithi143.stdout: "tags": [], 2023-11-26T07:23:10.640 INFO:teuthology.orchestra.run.smithi143.stdout: "see_also": [] 2023-11-26T07:23:10.640 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2023-11-26T07:23:10.640 INFO:teuthology.orchestra.run.smithi143.stdout: "scrape_frequency": { 2023-11-26T07:23:10.640 INFO:teuthology.orchestra.run.smithi143.stdout: "name": "scrape_frequency", 2023-11-26T07:23:10.640 INFO:teuthology.orchestra.run.smithi143.stdout: "type": "secs", 2023-11-26T07:23:10.640 INFO:teuthology.orchestra.run.smithi143.stdout: "level": "advanced", 2023-11-26T07:23:10.640 INFO:teuthology.orchestra.run.smithi143.stdout: "flags": 1, 2023-11-26T07:23:10.641 INFO:teuthology.orchestra.run.smithi143.stdout: "default_value": "86400", 2023-11-26T07:23:10.641 INFO:teuthology.orchestra.run.smithi143.stdout: "min": "", 2023-11-26T07:23:10.641 INFO:teuthology.orchestra.run.smithi143.stdout: "max": "", 2023-11-26T07:23:10.641 INFO:teuthology.orchestra.run.smithi143.stdout: "enum_allowed": [], 2023-11-26T07:23:10.641 INFO:teuthology.orchestra.run.smithi143.stdout: "desc": "how frequently to scrape device health metrics", 2023-11-26T07:23:10.641 INFO:teuthology.orchestra.run.smithi143.stdout: "long_desc": "", 2023-11-26T07:23:10.641 INFO:teuthology.orchestra.run.smithi143.stdout: "tags": [], 2023-11-26T07:23:10.641 INFO:teuthology.orchestra.run.smithi143.stdout: "see_also": [] 2023-11-26T07:23:10.641 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2023-11-26T07:23:10.641 INFO:teuthology.orchestra.run.smithi143.stdout: "self_heal": { 2023-11-26T07:23:10.641 INFO:teuthology.orchestra.run.smithi143.stdout: "name": "self_heal", 2023-11-26T07:23:10.641 INFO:teuthology.orchestra.run.smithi143.stdout: "type": "bool", 2023-11-26T07:23:10.642 INFO:teuthology.orchestra.run.smithi143.stdout: "level": "advanced", 2023-11-26T07:23:10.642 INFO:teuthology.orchestra.run.smithi143.stdout: "flags": 1, 2023-11-26T07:23:10.642 INFO:teuthology.orchestra.run.smithi143.stdout: "default_value": "True", 2023-11-26T07:23:10.642 INFO:teuthology.orchestra.run.smithi143.stdout: "min": "", 2023-11-26T07:23:10.642 INFO:teuthology.orchestra.run.smithi143.stdout: "max": "", 2023-11-26T07:23:10.642 INFO:teuthology.orchestra.run.smithi143.stdout: "enum_allowed": [], 2023-11-26T07:23:10.642 INFO:teuthology.orchestra.run.smithi143.stdout: "desc": "preemptively heal cluster around devices that may fail", 2023-11-26T07:23:10.642 INFO:teuthology.orchestra.run.smithi143.stdout: "long_desc": "", 2023-11-26T07:23:10.642 INFO:teuthology.orchestra.run.smithi143.stdout: "tags": [], 2023-11-26T07:23:10.642 INFO:teuthology.orchestra.run.smithi143.stdout: "see_also": [] 2023-11-26T07:23:10.642 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2023-11-26T07:23:10.642 INFO:teuthology.orchestra.run.smithi143.stdout: "sleep_interval": { 2023-11-26T07:23:10.643 INFO:teuthology.orchestra.run.smithi143.stdout: "name": "sleep_interval", 2023-11-26T07:23:10.643 INFO:teuthology.orchestra.run.smithi143.stdout: "type": "secs", 2023-11-26T07:23:10.643 INFO:teuthology.orchestra.run.smithi143.stdout: "level": "advanced", 2023-11-26T07:23:10.643 INFO:teuthology.orchestra.run.smithi143.stdout: "flags": 1, 2023-11-26T07:23:10.643 INFO:teuthology.orchestra.run.smithi143.stdout: "default_value": "600", 2023-11-26T07:23:10.643 INFO:teuthology.orchestra.run.smithi143.stdout: "min": "", 2023-11-26T07:23:10.643 INFO:teuthology.orchestra.run.smithi143.stdout: "max": "", 2023-11-26T07:23:10.643 INFO:teuthology.orchestra.run.smithi143.stdout: "enum_allowed": [], 2023-11-26T07:23:10.643 INFO:teuthology.orchestra.run.smithi143.stdout: "desc": "how frequently to wake up and check device health", 2023-11-26T07:23:10.643 INFO:teuthology.orchestra.run.smithi143.stdout: "long_desc": "", 2023-11-26T07:23:10.643 INFO:teuthology.orchestra.run.smithi143.stdout: "tags": [], 2023-11-26T07:23:10.643 INFO:teuthology.orchestra.run.smithi143.stdout: "see_also": [] 2023-11-26T07:23:10.643 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2023-11-26T07:23:10.644 INFO:teuthology.orchestra.run.smithi143.stdout: "warn_threshold": { 2023-11-26T07:23:10.644 INFO:teuthology.orchestra.run.smithi143.stdout: "name": "warn_threshold", 2023-11-26T07:23:10.644 INFO:teuthology.orchestra.run.smithi143.stdout: "type": "secs", 2023-11-26T07:23:10.644 INFO:teuthology.orchestra.run.smithi143.stdout: "level": "advanced", 2023-11-26T07:23:10.644 INFO:teuthology.orchestra.run.smithi143.stdout: "flags": 1, 2023-11-26T07:23:10.644 INFO:teuthology.orchestra.run.smithi143.stdout: "default_value": "7257600", 2023-11-26T07:23:10.644 INFO:teuthology.orchestra.run.smithi143.stdout: "min": "", 2023-11-26T07:23:10.644 INFO:teuthology.orchestra.run.smithi143.stdout: "max": "", 2023-11-26T07:23:10.644 INFO:teuthology.orchestra.run.smithi143.stdout: "enum_allowed": [], 2023-11-26T07:23:10.644 INFO:teuthology.orchestra.run.smithi143.stdout: "desc": "raise health warning if OSD may fail before this long", 2023-11-26T07:23:10.644 INFO:teuthology.orchestra.run.smithi143.stdout: "long_desc": "", 2023-11-26T07:23:10.644 INFO:teuthology.orchestra.run.smithi143.stdout: "tags": [], 2023-11-26T07:23:10.645 INFO:teuthology.orchestra.run.smithi143.stdout: "see_also": [] 2023-11-26T07:23:10.645 INFO:teuthology.orchestra.run.smithi143.stdout: } 2023-11-26T07:23:10.645 INFO:teuthology.orchestra.run.smithi143.stdout: } 2023-11-26T07:23:10.645 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2023-11-26T07:23:10.645 INFO:teuthology.orchestra.run.smithi143.stdout: { 2023-11-26T07:23:10.645 INFO:teuthology.orchestra.run.smithi143.stdout: "name": "diskprediction_local", 2023-11-26T07:23:10.645 INFO:teuthology.orchestra.run.smithi143.stdout: "can_run": true, 2023-11-26T07:23:10.645 INFO:teuthology.orchestra.run.smithi143.stdout: "error_string": "", 2023-11-26T07:23:10.645 INFO:teuthology.orchestra.run.smithi143.stdout: "module_options": { 2023-11-26T07:23:10.645 INFO:teuthology.orchestra.run.smithi143.stdout: "log_level": { 2023-11-26T07:23:10.645 INFO:teuthology.orchestra.run.smithi143.stdout: "name": "log_level", 2023-11-26T07:23:10.645 INFO:teuthology.orchestra.run.smithi143.stdout: "type": "str", 2023-11-26T07:23:10.645 INFO:teuthology.orchestra.run.smithi143.stdout: "level": "advanced", 2023-11-26T07:23:10.646 INFO:teuthology.orchestra.run.smithi143.stdout: "flags": 1, 2023-11-26T07:23:10.646 INFO:teuthology.orchestra.run.smithi143.stdout: "default_value": "", 2023-11-26T07:23:10.646 INFO:teuthology.orchestra.run.smithi143.stdout: "min": "", 2023-11-26T07:23:10.646 INFO:teuthology.orchestra.run.smithi143.stdout: "max": "", 2023-11-26T07:23:10.646 INFO:teuthology.orchestra.run.smithi143.stdout: "enum_allowed": [ 2023-11-26T07:23:10.646 INFO:teuthology.orchestra.run.smithi143.stdout: "", 2023-11-26T07:23:10.646 INFO:teuthology.orchestra.run.smithi143.stdout: "critical", 2023-11-26T07:23:10.646 INFO:teuthology.orchestra.run.smithi143.stdout: "debug", 2023-11-26T07:23:10.646 INFO:teuthology.orchestra.run.smithi143.stdout: "error", 2023-11-26T07:23:10.646 INFO:teuthology.orchestra.run.smithi143.stdout: "info", 2023-11-26T07:23:10.646 INFO:teuthology.orchestra.run.smithi143.stdout: "warning" 2023-11-26T07:23:10.646 INFO:teuthology.orchestra.run.smithi143.stdout: ], 2023-11-26T07:23:10.646 INFO:teuthology.orchestra.run.smithi143.stdout: "desc": "", 2023-11-26T07:23:10.647 INFO:teuthology.orchestra.run.smithi143.stdout: "long_desc": "", 2023-11-26T07:23:10.647 INFO:teuthology.orchestra.run.smithi143.stdout: "tags": [], 2023-11-26T07:23:10.647 INFO:teuthology.orchestra.run.smithi143.stdout: "see_also": [] 2023-11-26T07:23:10.647 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2023-11-26T07:23:10.647 INFO:teuthology.orchestra.run.smithi143.stdout: "log_to_cluster": { 2023-11-26T07:23:10.647 INFO:teuthology.orchestra.run.smithi143.stdout: "name": "log_to_cluster", 2023-11-26T07:23:10.647 INFO:teuthology.orchestra.run.smithi143.stdout: "type": "bool", 2023-11-26T07:23:10.647 INFO:teuthology.orchestra.run.smithi143.stdout: "level": "advanced", 2023-11-26T07:23:10.647 INFO:teuthology.orchestra.run.smithi143.stdout: "flags": 1, 2023-11-26T07:23:10.647 INFO:teuthology.orchestra.run.smithi143.stdout: "default_value": "False", 2023-11-26T07:23:10.647 INFO:teuthology.orchestra.run.smithi143.stdout: "min": "", 2023-11-26T07:23:10.647 INFO:teuthology.orchestra.run.smithi143.stdout: "max": "", 2023-11-26T07:23:10.648 INFO:teuthology.orchestra.run.smithi143.stdout: "enum_allowed": [], 2023-11-26T07:23:10.648 INFO:teuthology.orchestra.run.smithi143.stdout: "desc": "", 2023-11-26T07:23:10.648 INFO:teuthology.orchestra.run.smithi143.stdout: "long_desc": "", 2023-11-26T07:23:10.648 INFO:teuthology.orchestra.run.smithi143.stdout: "tags": [], 2023-11-26T07:23:10.648 INFO:teuthology.orchestra.run.smithi143.stdout: "see_also": [] 2023-11-26T07:23:10.648 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2023-11-26T07:23:10.648 INFO:teuthology.orchestra.run.smithi143.stdout: "log_to_cluster_level": { 2023-11-26T07:23:10.648 INFO:teuthology.orchestra.run.smithi143.stdout: "name": "log_to_cluster_level", 2023-11-26T07:23:10.648 INFO:teuthology.orchestra.run.smithi143.stdout: "type": "str", 2023-11-26T07:23:10.648 INFO:teuthology.orchestra.run.smithi143.stdout: "level": "advanced", 2023-11-26T07:23:10.648 INFO:teuthology.orchestra.run.smithi143.stdout: "flags": 1, 2023-11-26T07:23:10.648 INFO:teuthology.orchestra.run.smithi143.stdout: "default_value": "info", 2023-11-26T07:23:10.648 INFO:teuthology.orchestra.run.smithi143.stdout: "min": "", 2023-11-26T07:23:10.649 INFO:teuthology.orchestra.run.smithi143.stdout: "max": "", 2023-11-26T07:23:10.649 INFO:teuthology.orchestra.run.smithi143.stdout: "enum_allowed": [ 2023-11-26T07:23:10.649 INFO:teuthology.orchestra.run.smithi143.stdout: "", 2023-11-26T07:23:10.649 INFO:teuthology.orchestra.run.smithi143.stdout: "critical", 2023-11-26T07:23:10.649 INFO:teuthology.orchestra.run.smithi143.stdout: "debug", 2023-11-26T07:23:10.649 INFO:teuthology.orchestra.run.smithi143.stdout: "error", 2023-11-26T07:23:10.649 INFO:teuthology.orchestra.run.smithi143.stdout: "info", 2023-11-26T07:23:10.649 INFO:teuthology.orchestra.run.smithi143.stdout: "warning" 2023-11-26T07:23:10.649 INFO:teuthology.orchestra.run.smithi143.stdout: ], 2023-11-26T07:23:10.649 INFO:teuthology.orchestra.run.smithi143.stdout: "desc": "", 2023-11-26T07:23:10.649 INFO:teuthology.orchestra.run.smithi143.stdout: "long_desc": "", 2023-11-26T07:23:10.649 INFO:teuthology.orchestra.run.smithi143.stdout: "tags": [], 2023-11-26T07:23:10.649 INFO:teuthology.orchestra.run.smithi143.stdout: "see_also": [] 2023-11-26T07:23:10.650 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2023-11-26T07:23:10.650 INFO:teuthology.orchestra.run.smithi143.stdout: "log_to_file": { 2023-11-26T07:23:10.650 INFO:teuthology.orchestra.run.smithi143.stdout: "name": "log_to_file", 2023-11-26T07:23:10.650 INFO:teuthology.orchestra.run.smithi143.stdout: "type": "bool", 2023-11-26T07:23:10.650 INFO:teuthology.orchestra.run.smithi143.stdout: "level": "advanced", 2023-11-26T07:23:10.650 INFO:teuthology.orchestra.run.smithi143.stdout: "flags": 1, 2023-11-26T07:23:10.650 INFO:teuthology.orchestra.run.smithi143.stdout: "default_value": "False", 2023-11-26T07:23:10.650 INFO:teuthology.orchestra.run.smithi143.stdout: "min": "", 2023-11-26T07:23:10.650 INFO:teuthology.orchestra.run.smithi143.stdout: "max": "", 2023-11-26T07:23:10.650 INFO:teuthology.orchestra.run.smithi143.stdout: "enum_allowed": [], 2023-11-26T07:23:10.650 INFO:teuthology.orchestra.run.smithi143.stdout: "desc": "", 2023-11-26T07:23:10.650 INFO:teuthology.orchestra.run.smithi143.stdout: "long_desc": "", 2023-11-26T07:23:10.650 INFO:teuthology.orchestra.run.smithi143.stdout: "tags": [], 2023-11-26T07:23:10.651 INFO:teuthology.orchestra.run.smithi143.stdout: "see_also": [] 2023-11-26T07:23:10.651 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2023-11-26T07:23:10.651 INFO:teuthology.orchestra.run.smithi143.stdout: "predict_interval": { 2023-11-26T07:23:10.651 INFO:teuthology.orchestra.run.smithi143.stdout: "name": "predict_interval", 2023-11-26T07:23:10.651 INFO:teuthology.orchestra.run.smithi143.stdout: "type": "str", 2023-11-26T07:23:10.651 INFO:teuthology.orchestra.run.smithi143.stdout: "level": "advanced", 2023-11-26T07:23:10.651 INFO:teuthology.orchestra.run.smithi143.stdout: "flags": 0, 2023-11-26T07:23:10.651 INFO:teuthology.orchestra.run.smithi143.stdout: "default_value": "86400", 2023-11-26T07:23:10.651 INFO:teuthology.orchestra.run.smithi143.stdout: "min": "", 2023-11-26T07:23:10.651 INFO:teuthology.orchestra.run.smithi143.stdout: "max": "", 2023-11-26T07:23:10.651 INFO:teuthology.orchestra.run.smithi143.stdout: "enum_allowed": [], 2023-11-26T07:23:10.651 INFO:teuthology.orchestra.run.smithi143.stdout: "desc": "", 2023-11-26T07:23:10.652 INFO:teuthology.orchestra.run.smithi143.stdout: "long_desc": "", 2023-11-26T07:23:10.652 INFO:teuthology.orchestra.run.smithi143.stdout: "tags": [], 2023-11-26T07:23:10.652 INFO:teuthology.orchestra.run.smithi143.stdout: "see_also": [] 2023-11-26T07:23:10.652 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2023-11-26T07:23:10.652 INFO:teuthology.orchestra.run.smithi143.stdout: "predictor_model": { 2023-11-26T07:23:10.652 INFO:teuthology.orchestra.run.smithi143.stdout: "name": "predictor_model", 2023-11-26T07:23:10.652 INFO:teuthology.orchestra.run.smithi143.stdout: "type": "str", 2023-11-26T07:23:10.652 INFO:teuthology.orchestra.run.smithi143.stdout: "level": "advanced", 2023-11-26T07:23:10.652 INFO:teuthology.orchestra.run.smithi143.stdout: "flags": 0, 2023-11-26T07:23:10.652 INFO:teuthology.orchestra.run.smithi143.stdout: "default_value": "prophetstor", 2023-11-26T07:23:10.652 INFO:teuthology.orchestra.run.smithi143.stdout: "min": "", 2023-11-26T07:23:10.652 INFO:teuthology.orchestra.run.smithi143.stdout: "max": "", 2023-11-26T07:23:10.653 INFO:teuthology.orchestra.run.smithi143.stdout: "enum_allowed": [], 2023-11-26T07:23:10.653 INFO:teuthology.orchestra.run.smithi143.stdout: "desc": "", 2023-11-26T07:23:10.653 INFO:teuthology.orchestra.run.smithi143.stdout: "long_desc": "", 2023-11-26T07:23:10.653 INFO:teuthology.orchestra.run.smithi143.stdout: "tags": [], 2023-11-26T07:23:10.653 INFO:teuthology.orchestra.run.smithi143.stdout: "see_also": [] 2023-11-26T07:23:10.653 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2023-11-26T07:23:10.653 INFO:teuthology.orchestra.run.smithi143.stdout: "sleep_interval": { 2023-11-26T07:23:10.653 INFO:teuthology.orchestra.run.smithi143.stdout: "name": "sleep_interval", 2023-11-26T07:23:10.653 INFO:teuthology.orchestra.run.smithi143.stdout: "type": "str", 2023-11-26T07:23:10.653 INFO:teuthology.orchestra.run.smithi143.stdout: "level": "advanced", 2023-11-26T07:23:10.653 INFO:teuthology.orchestra.run.smithi143.stdout: "flags": 0, 2023-11-26T07:23:10.653 INFO:teuthology.orchestra.run.smithi143.stdout: "default_value": "600", 2023-11-26T07:23:10.653 INFO:teuthology.orchestra.run.smithi143.stdout: "min": "", 2023-11-26T07:23:10.654 INFO:teuthology.orchestra.run.smithi143.stdout: "max": "", 2023-11-26T07:23:10.654 INFO:teuthology.orchestra.run.smithi143.stdout: "enum_allowed": [], 2023-11-26T07:23:10.654 INFO:teuthology.orchestra.run.smithi143.stdout: "desc": "", 2023-11-26T07:23:10.654 INFO:teuthology.orchestra.run.smithi143.stdout: "long_desc": "", 2023-11-26T07:23:10.654 INFO:teuthology.orchestra.run.smithi143.stdout: "tags": [], 2023-11-26T07:23:10.654 INFO:teuthology.orchestra.run.smithi143.stdout: "see_also": [] 2023-11-26T07:23:10.654 INFO:teuthology.orchestra.run.smithi143.stdout: } 2023-11-26T07:23:10.654 INFO:teuthology.orchestra.run.smithi143.stdout: } 2023-11-26T07:23:10.654 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2023-11-26T07:23:10.654 INFO:teuthology.orchestra.run.smithi143.stdout: { 2023-11-26T07:23:10.654 INFO:teuthology.orchestra.run.smithi143.stdout: "name": "influx", 2023-11-26T07:23:10.654 INFO:teuthology.orchestra.run.smithi143.stdout: "can_run": false, 2023-11-26T07:23:10.654 INFO:teuthology.orchestra.run.smithi143.stdout: "error_string": "influxdb python module not found", 2023-11-26T07:23:10.655 INFO:teuthology.orchestra.run.smithi143.stdout: "module_options": { 2023-11-26T07:23:10.655 INFO:teuthology.orchestra.run.smithi143.stdout: "batch_size": { 2023-11-26T07:23:10.655 INFO:teuthology.orchestra.run.smithi143.stdout: "name": "batch_size", 2023-11-26T07:23:10.655 INFO:teuthology.orchestra.run.smithi143.stdout: "type": "int", 2023-11-26T07:23:10.655 INFO:teuthology.orchestra.run.smithi143.stdout: "level": "advanced", 2023-11-26T07:23:10.655 INFO:teuthology.orchestra.run.smithi143.stdout: "flags": 0, 2023-11-26T07:23:10.655 INFO:teuthology.orchestra.run.smithi143.stdout: "default_value": "5000", 2023-11-26T07:23:10.655 INFO:teuthology.orchestra.run.smithi143.stdout: "min": "", 2023-11-26T07:23:10.655 INFO:teuthology.orchestra.run.smithi143.stdout: "max": "", 2023-11-26T07:23:10.655 INFO:teuthology.orchestra.run.smithi143.stdout: "enum_allowed": [], 2023-11-26T07:23:10.655 INFO:teuthology.orchestra.run.smithi143.stdout: "desc": "How big batches of data points should be when sending to InfluxDB.", 2023-11-26T07:23:10.655 INFO:teuthology.orchestra.run.smithi143.stdout: "long_desc": "", 2023-11-26T07:23:10.655 INFO:teuthology.orchestra.run.smithi143.stdout: "tags": [], 2023-11-26T07:23:10.656 INFO:teuthology.orchestra.run.smithi143.stdout: "see_also": [] 2023-11-26T07:23:10.656 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2023-11-26T07:23:10.656 INFO:teuthology.orchestra.run.smithi143.stdout: "database": { 2023-11-26T07:23:10.656 INFO:teuthology.orchestra.run.smithi143.stdout: "name": "database", 2023-11-26T07:23:10.656 INFO:teuthology.orchestra.run.smithi143.stdout: "type": "str", 2023-11-26T07:23:10.656 INFO:teuthology.orchestra.run.smithi143.stdout: "level": "advanced", 2023-11-26T07:23:10.656 INFO:teuthology.orchestra.run.smithi143.stdout: "flags": 0, 2023-11-26T07:23:10.656 INFO:teuthology.orchestra.run.smithi143.stdout: "default_value": "ceph", 2023-11-26T07:23:10.656 INFO:teuthology.orchestra.run.smithi143.stdout: "min": "", 2023-11-26T07:23:10.656 INFO:teuthology.orchestra.run.smithi143.stdout: "max": "", 2023-11-26T07:23:10.656 INFO:teuthology.orchestra.run.smithi143.stdout: "enum_allowed": [], 2023-11-26T07:23:10.656 INFO:teuthology.orchestra.run.smithi143.stdout: "desc": "InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.", 2023-11-26T07:23:10.656 INFO:teuthology.orchestra.run.smithi143.stdout: "long_desc": "", 2023-11-26T07:23:10.657 INFO:teuthology.orchestra.run.smithi143.stdout: "tags": [], 2023-11-26T07:23:10.657 INFO:teuthology.orchestra.run.smithi143.stdout: "see_also": [] 2023-11-26T07:23:10.657 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2023-11-26T07:23:10.657 INFO:teuthology.orchestra.run.smithi143.stdout: "hostname": { 2023-11-26T07:23:10.657 INFO:teuthology.orchestra.run.smithi143.stdout: "name": "hostname", 2023-11-26T07:23:10.657 INFO:teuthology.orchestra.run.smithi143.stdout: "type": "str", 2023-11-26T07:23:10.657 INFO:teuthology.orchestra.run.smithi143.stdout: "level": "advanced", 2023-11-26T07:23:10.657 INFO:teuthology.orchestra.run.smithi143.stdout: "flags": 0, 2023-11-26T07:23:10.657 INFO:teuthology.orchestra.run.smithi143.stdout: "default_value": "", 2023-11-26T07:23:10.657 INFO:teuthology.orchestra.run.smithi143.stdout: "min": "", 2023-11-26T07:23:10.657 INFO:teuthology.orchestra.run.smithi143.stdout: "max": "", 2023-11-26T07:23:10.657 INFO:teuthology.orchestra.run.smithi143.stdout: "enum_allowed": [], 2023-11-26T07:23:10.658 INFO:teuthology.orchestra.run.smithi143.stdout: "desc": "InfluxDB server hostname", 2023-11-26T07:23:10.658 INFO:teuthology.orchestra.run.smithi143.stdout: "long_desc": "", 2023-11-26T07:23:10.658 INFO:teuthology.orchestra.run.smithi143.stdout: "tags": [], 2023-11-26T07:23:10.658 INFO:teuthology.orchestra.run.smithi143.stdout: "see_also": [] 2023-11-26T07:23:10.658 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2023-11-26T07:23:10.658 INFO:teuthology.orchestra.run.smithi143.stdout: "interval": { 2023-11-26T07:23:10.658 INFO:teuthology.orchestra.run.smithi143.stdout: "name": "interval", 2023-11-26T07:23:10.658 INFO:teuthology.orchestra.run.smithi143.stdout: "type": "secs", 2023-11-26T07:23:10.658 INFO:teuthology.orchestra.run.smithi143.stdout: "level": "advanced", 2023-11-26T07:23:10.658 INFO:teuthology.orchestra.run.smithi143.stdout: "flags": 0, 2023-11-26T07:23:10.658 INFO:teuthology.orchestra.run.smithi143.stdout: "default_value": "30", 2023-11-26T07:23:10.658 INFO:teuthology.orchestra.run.smithi143.stdout: "min": "5", 2023-11-26T07:23:10.658 INFO:teuthology.orchestra.run.smithi143.stdout: "max": "", 2023-11-26T07:23:10.659 INFO:teuthology.orchestra.run.smithi143.stdout: "enum_allowed": [], 2023-11-26T07:23:10.659 INFO:teuthology.orchestra.run.smithi143.stdout: "desc": "Time between reports to InfluxDB. Default 30 seconds.", 2023-11-26T07:23:10.659 INFO:teuthology.orchestra.run.smithi143.stdout: "long_desc": "", 2023-11-26T07:23:10.659 INFO:teuthology.orchestra.run.smithi143.stdout: "tags": [], 2023-11-26T07:23:10.659 INFO:teuthology.orchestra.run.smithi143.stdout: "see_also": [] 2023-11-26T07:23:10.659 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2023-11-26T07:23:10.659 INFO:teuthology.orchestra.run.smithi143.stdout: "log_level": { 2023-11-26T07:23:10.659 INFO:teuthology.orchestra.run.smithi143.stdout: "name": "log_level", 2023-11-26T07:23:10.659 INFO:teuthology.orchestra.run.smithi143.stdout: "type": "str", 2023-11-26T07:23:10.659 INFO:teuthology.orchestra.run.smithi143.stdout: "level": "advanced", 2023-11-26T07:23:10.659 INFO:teuthology.orchestra.run.smithi143.stdout: "flags": 1, 2023-11-26T07:23:10.659 INFO:teuthology.orchestra.run.smithi143.stdout: "default_value": "", 2023-11-26T07:23:10.659 INFO:teuthology.orchestra.run.smithi143.stdout: "min": "", 2023-11-26T07:23:10.660 INFO:teuthology.orchestra.run.smithi143.stdout: "max": "", 2023-11-26T07:23:10.660 INFO:teuthology.orchestra.run.smithi143.stdout: "enum_allowed": [ 2023-11-26T07:23:10.660 INFO:teuthology.orchestra.run.smithi143.stdout: "", 2023-11-26T07:23:10.660 INFO:teuthology.orchestra.run.smithi143.stdout: "critical", 2023-11-26T07:23:10.660 INFO:teuthology.orchestra.run.smithi143.stdout: "debug", 2023-11-26T07:23:10.660 INFO:teuthology.orchestra.run.smithi143.stdout: "error", 2023-11-26T07:23:10.660 INFO:teuthology.orchestra.run.smithi143.stdout: "info", 2023-11-26T07:23:10.660 INFO:teuthology.orchestra.run.smithi143.stdout: "warning" 2023-11-26T07:23:10.660 INFO:teuthology.orchestra.run.smithi143.stdout: ], 2023-11-26T07:23:10.660 INFO:teuthology.orchestra.run.smithi143.stdout: "desc": "", 2023-11-26T07:23:10.660 INFO:teuthology.orchestra.run.smithi143.stdout: "long_desc": "", 2023-11-26T07:23:10.660 INFO:teuthology.orchestra.run.smithi143.stdout: "tags": [], 2023-11-26T07:23:10.660 INFO:teuthology.orchestra.run.smithi143.stdout: "see_also": [] 2023-11-26T07:23:10.661 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2023-11-26T07:23:10.661 INFO:teuthology.orchestra.run.smithi143.stdout: "log_to_cluster": { 2023-11-26T07:23:10.661 INFO:teuthology.orchestra.run.smithi143.stdout: "name": "log_to_cluster", 2023-11-26T07:23:10.661 INFO:teuthology.orchestra.run.smithi143.stdout: "type": "bool", 2023-11-26T07:23:10.661 INFO:teuthology.orchestra.run.smithi143.stdout: "level": "advanced", 2023-11-26T07:23:10.661 INFO:teuthology.orchestra.run.smithi143.stdout: "flags": 1, 2023-11-26T07:23:10.661 INFO:teuthology.orchestra.run.smithi143.stdout: "default_value": "False", 2023-11-26T07:23:10.661 INFO:teuthology.orchestra.run.smithi143.stdout: "min": "", 2023-11-26T07:23:10.661 INFO:teuthology.orchestra.run.smithi143.stdout: "max": "", 2023-11-26T07:23:10.661 INFO:teuthology.orchestra.run.smithi143.stdout: "enum_allowed": [], 2023-11-26T07:23:10.661 INFO:teuthology.orchestra.run.smithi143.stdout: "desc": "", 2023-11-26T07:23:10.661 INFO:teuthology.orchestra.run.smithi143.stdout: "long_desc": "", 2023-11-26T07:23:10.662 INFO:teuthology.orchestra.run.smithi143.stdout: "tags": [], 2023-11-26T07:23:10.662 INFO:teuthology.orchestra.run.smithi143.stdout: "see_also": [] 2023-11-26T07:23:10.662 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2023-11-26T07:23:10.662 INFO:teuthology.orchestra.run.smithi143.stdout: "log_to_cluster_level": { 2023-11-26T07:23:10.662 INFO:teuthology.orchestra.run.smithi143.stdout: "name": "log_to_cluster_level", 2023-11-26T07:23:10.662 INFO:teuthology.orchestra.run.smithi143.stdout: "type": "str", 2023-11-26T07:23:10.662 INFO:teuthology.orchestra.run.smithi143.stdout: "level": "advanced", 2023-11-26T07:23:10.662 INFO:teuthology.orchestra.run.smithi143.stdout: "flags": 1, 2023-11-26T07:23:10.662 INFO:teuthology.orchestra.run.smithi143.stdout: "default_value": "info", 2023-11-26T07:23:10.662 INFO:teuthology.orchestra.run.smithi143.stdout: "min": "", 2023-11-26T07:23:10.662 INFO:teuthology.orchestra.run.smithi143.stdout: "max": "", 2023-11-26T07:23:10.662 INFO:teuthology.orchestra.run.smithi143.stdout: "enum_allowed": [ 2023-11-26T07:23:10.662 INFO:teuthology.orchestra.run.smithi143.stdout: "", 2023-11-26T07:23:10.663 INFO:teuthology.orchestra.run.smithi143.stdout: "critical", 2023-11-26T07:23:10.663 INFO:teuthology.orchestra.run.smithi143.stdout: "debug", 2023-11-26T07:23:10.663 INFO:teuthology.orchestra.run.smithi143.stdout: "error", 2023-11-26T07:23:10.663 INFO:teuthology.orchestra.run.smithi143.stdout: "info", 2023-11-26T07:23:10.663 INFO:teuthology.orchestra.run.smithi143.stdout: "warning" 2023-11-26T07:23:10.663 INFO:teuthology.orchestra.run.smithi143.stdout: ], 2023-11-26T07:23:10.663 INFO:teuthology.orchestra.run.smithi143.stdout: "desc": "", 2023-11-26T07:23:10.663 INFO:teuthology.orchestra.run.smithi143.stdout: "long_desc": "", 2023-11-26T07:23:10.663 INFO:teuthology.orchestra.run.smithi143.stdout: "tags": [], 2023-11-26T07:23:10.663 INFO:teuthology.orchestra.run.smithi143.stdout: "see_also": [] 2023-11-26T07:23:10.663 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2023-11-26T07:23:10.663 INFO:teuthology.orchestra.run.smithi143.stdout: "log_to_file": { 2023-11-26T07:23:10.663 INFO:teuthology.orchestra.run.smithi143.stdout: "name": "log_to_file", 2023-11-26T07:23:10.664 INFO:teuthology.orchestra.run.smithi143.stdout: "type": "bool", 2023-11-26T07:23:10.664 INFO:teuthology.orchestra.run.smithi143.stdout: "level": "advanced", 2023-11-26T07:23:10.664 INFO:teuthology.orchestra.run.smithi143.stdout: "flags": 1, 2023-11-26T07:23:10.664 INFO:teuthology.orchestra.run.smithi143.stdout: "default_value": "False", 2023-11-26T07:23:10.664 INFO:teuthology.orchestra.run.smithi143.stdout: "min": "", 2023-11-26T07:23:10.664 INFO:teuthology.orchestra.run.smithi143.stdout: "max": "", 2023-11-26T07:23:10.664 INFO:teuthology.orchestra.run.smithi143.stdout: "enum_allowed": [], 2023-11-26T07:23:10.664 INFO:teuthology.orchestra.run.smithi143.stdout: "desc": "", 2023-11-26T07:23:10.664 INFO:teuthology.orchestra.run.smithi143.stdout: "long_desc": "", 2023-11-26T07:23:10.664 INFO:teuthology.orchestra.run.smithi143.stdout: "tags": [], 2023-11-26T07:23:10.664 INFO:teuthology.orchestra.run.smithi143.stdout: "see_also": [] 2023-11-26T07:23:10.664 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2023-11-26T07:23:10.664 INFO:teuthology.orchestra.run.smithi143.stdout: "password": { 2023-11-26T07:23:10.664 INFO:teuthology.orchestra.run.smithi143.stdout: "name": "password", 2023-11-26T07:23:10.665 INFO:teuthology.orchestra.run.smithi143.stdout: "type": "str", 2023-11-26T07:23:10.665 INFO:teuthology.orchestra.run.smithi143.stdout: "level": "advanced", 2023-11-26T07:23:10.665 INFO:teuthology.orchestra.run.smithi143.stdout: "flags": 0, 2023-11-26T07:23:10.665 INFO:teuthology.orchestra.run.smithi143.stdout: "default_value": "", 2023-11-26T07:23:10.665 INFO:teuthology.orchestra.run.smithi143.stdout: "min": "", 2023-11-26T07:23:10.665 INFO:teuthology.orchestra.run.smithi143.stdout: "max": "", 2023-11-26T07:23:10.665 INFO:teuthology.orchestra.run.smithi143.stdout: "enum_allowed": [], 2023-11-26T07:23:10.665 INFO:teuthology.orchestra.run.smithi143.stdout: "desc": "password of InfluxDB server user", 2023-11-26T07:23:10.665 INFO:teuthology.orchestra.run.smithi143.stdout: "long_desc": "", 2023-11-26T07:23:10.665 INFO:teuthology.orchestra.run.smithi143.stdout: "tags": [], 2023-11-26T07:23:10.665 INFO:teuthology.orchestra.run.smithi143.stdout: "see_also": [] 2023-11-26T07:23:10.665 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2023-11-26T07:23:10.666 INFO:teuthology.orchestra.run.smithi143.stdout: "port": { 2023-11-26T07:23:10.666 INFO:teuthology.orchestra.run.smithi143.stdout: "name": "port", 2023-11-26T07:23:10.666 INFO:teuthology.orchestra.run.smithi143.stdout: "type": "int", 2023-11-26T07:23:10.666 INFO:teuthology.orchestra.run.smithi143.stdout: "level": "advanced", 2023-11-26T07:23:10.666 INFO:teuthology.orchestra.run.smithi143.stdout: "flags": 0, 2023-11-26T07:23:10.666 INFO:teuthology.orchestra.run.smithi143.stdout: "default_value": "8086", 2023-11-26T07:23:10.666 INFO:teuthology.orchestra.run.smithi143.stdout: "min": "", 2023-11-26T07:23:10.666 INFO:teuthology.orchestra.run.smithi143.stdout: "max": "", 2023-11-26T07:23:10.666 INFO:teuthology.orchestra.run.smithi143.stdout: "enum_allowed": [], 2023-11-26T07:23:10.666 INFO:teuthology.orchestra.run.smithi143.stdout: "desc": "InfluxDB server port", 2023-11-26T07:23:10.666 INFO:teuthology.orchestra.run.smithi143.stdout: "long_desc": "", 2023-11-26T07:23:10.666 INFO:teuthology.orchestra.run.smithi143.stdout: "tags": [], 2023-11-26T07:23:10.667 INFO:teuthology.orchestra.run.smithi143.stdout: "see_also": [] 2023-11-26T07:23:10.667 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2023-11-26T07:23:10.667 INFO:teuthology.orchestra.run.smithi143.stdout: "ssl": { 2023-11-26T07:23:10.667 INFO:teuthology.orchestra.run.smithi143.stdout: "name": "ssl", 2023-11-26T07:23:10.667 INFO:teuthology.orchestra.run.smithi143.stdout: "type": "str", 2023-11-26T07:23:10.667 INFO:teuthology.orchestra.run.smithi143.stdout: "level": "advanced", 2023-11-26T07:23:10.667 INFO:teuthology.orchestra.run.smithi143.stdout: "flags": 0, 2023-11-26T07:23:10.667 INFO:teuthology.orchestra.run.smithi143.stdout: "default_value": "false", 2023-11-26T07:23:10.667 INFO:teuthology.orchestra.run.smithi143.stdout: "min": "", 2023-11-26T07:23:10.667 INFO:teuthology.orchestra.run.smithi143.stdout: "max": "", 2023-11-26T07:23:10.667 INFO:teuthology.orchestra.run.smithi143.stdout: "enum_allowed": [], 2023-11-26T07:23:10.667 INFO:teuthology.orchestra.run.smithi143.stdout: "desc": "Use https connection for InfluxDB server. Use \"true\" or \"false\".", 2023-11-26T07:23:10.667 INFO:teuthology.orchestra.run.smithi143.stdout: "long_desc": "", 2023-11-26T07:23:10.667 INFO:teuthology.orchestra.run.smithi143.stdout: "tags": [], 2023-11-26T07:23:10.668 INFO:teuthology.orchestra.run.smithi143.stdout: "see_also": [] 2023-11-26T07:23:10.668 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2023-11-26T07:23:10.668 INFO:teuthology.orchestra.run.smithi143.stdout: "threads": { 2023-11-26T07:23:10.668 INFO:teuthology.orchestra.run.smithi143.stdout: "name": "threads", 2023-11-26T07:23:10.668 INFO:teuthology.orchestra.run.smithi143.stdout: "type": "int", 2023-11-26T07:23:10.668 INFO:teuthology.orchestra.run.smithi143.stdout: "level": "advanced", 2023-11-26T07:23:10.668 INFO:teuthology.orchestra.run.smithi143.stdout: "flags": 0, 2023-11-26T07:23:10.668 INFO:teuthology.orchestra.run.smithi143.stdout: "default_value": "5", 2023-11-26T07:23:10.668 INFO:teuthology.orchestra.run.smithi143.stdout: "min": "1", 2023-11-26T07:23:10.668 INFO:teuthology.orchestra.run.smithi143.stdout: "max": "32", 2023-11-26T07:23:10.668 INFO:teuthology.orchestra.run.smithi143.stdout: "enum_allowed": [], 2023-11-26T07:23:10.668 INFO:teuthology.orchestra.run.smithi143.stdout: "desc": "How many worker threads should be spawned for sending data to InfluxDB.", 2023-11-26T07:23:10.668 INFO:teuthology.orchestra.run.smithi143.stdout: "long_desc": "", 2023-11-26T07:23:10.669 INFO:teuthology.orchestra.run.smithi143.stdout: "tags": [], 2023-11-26T07:23:10.669 INFO:teuthology.orchestra.run.smithi143.stdout: "see_also": [] 2023-11-26T07:23:10.669 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2023-11-26T07:23:10.669 INFO:teuthology.orchestra.run.smithi143.stdout: "username": { 2023-11-26T07:23:10.669 INFO:teuthology.orchestra.run.smithi143.stdout: "name": "username", 2023-11-26T07:23:10.669 INFO:teuthology.orchestra.run.smithi143.stdout: "type": "str", 2023-11-26T07:23:10.669 INFO:teuthology.orchestra.run.smithi143.stdout: "level": "advanced", 2023-11-26T07:23:10.669 INFO:teuthology.orchestra.run.smithi143.stdout: "flags": 0, 2023-11-26T07:23:10.669 INFO:teuthology.orchestra.run.smithi143.stdout: "default_value": "", 2023-11-26T07:23:10.669 INFO:teuthology.orchestra.run.smithi143.stdout: "min": "", 2023-11-26T07:23:10.669 INFO:teuthology.orchestra.run.smithi143.stdout: "max": "", 2023-11-26T07:23:10.669 INFO:teuthology.orchestra.run.smithi143.stdout: "enum_allowed": [], 2023-11-26T07:23:10.669 INFO:teuthology.orchestra.run.smithi143.stdout: "desc": "username of InfluxDB server user", 2023-11-26T07:23:10.670 INFO:teuthology.orchestra.run.smithi143.stdout: "long_desc": "", 2023-11-26T07:23:10.670 INFO:teuthology.orchestra.run.smithi143.stdout: "tags": [], 2023-11-26T07:23:10.670 INFO:teuthology.orchestra.run.smithi143.stdout: "see_also": [] 2023-11-26T07:23:10.670 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2023-11-26T07:23:10.670 INFO:teuthology.orchestra.run.smithi143.stdout: "verify_ssl": { 2023-11-26T07:23:10.670 INFO:teuthology.orchestra.run.smithi143.stdout: "name": "verify_ssl", 2023-11-26T07:23:10.670 INFO:teuthology.orchestra.run.smithi143.stdout: "type": "str", 2023-11-26T07:23:10.670 INFO:teuthology.orchestra.run.smithi143.stdout: "level": "advanced", 2023-11-26T07:23:10.670 INFO:teuthology.orchestra.run.smithi143.stdout: "flags": 0, 2023-11-26T07:23:10.670 INFO:teuthology.orchestra.run.smithi143.stdout: "default_value": "true", 2023-11-26T07:23:10.670 INFO:teuthology.orchestra.run.smithi143.stdout: "min": "", 2023-11-26T07:23:10.670 INFO:teuthology.orchestra.run.smithi143.stdout: "max": "", 2023-11-26T07:23:10.670 INFO:teuthology.orchestra.run.smithi143.stdout: "enum_allowed": [], 2023-11-26T07:23:10.671 INFO:teuthology.orchestra.run.smithi143.stdout: "desc": "Verify https cert for InfluxDB server. Use \"true\" or \"false\".", 2023-11-26T07:23:10.671 INFO:teuthology.orchestra.run.smithi143.stdout: "long_desc": "", 2023-11-26T07:23:10.671 INFO:teuthology.orchestra.run.smithi143.stdout: "tags": [], 2023-11-26T07:23:10.671 INFO:teuthology.orchestra.run.smithi143.stdout: "see_also": [] 2023-11-26T07:23:10.671 INFO:teuthology.orchestra.run.smithi143.stdout: } 2023-11-26T07:23:10.671 INFO:teuthology.orchestra.run.smithi143.stdout: } 2023-11-26T07:23:10.671 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2023-11-26T07:23:10.671 INFO:teuthology.orchestra.run.smithi143.stdout: { 2023-11-26T07:23:10.671 INFO:teuthology.orchestra.run.smithi143.stdout: "name": "insights", 2023-11-26T07:23:10.671 INFO:teuthology.orchestra.run.smithi143.stdout: "can_run": true, 2023-11-26T07:23:10.671 INFO:teuthology.orchestra.run.smithi143.stdout: "error_string": "", 2023-11-26T07:23:10.671 INFO:teuthology.orchestra.run.smithi143.stdout: "module_options": { 2023-11-26T07:23:10.671 INFO:teuthology.orchestra.run.smithi143.stdout: "log_level": { 2023-11-26T07:23:10.672 INFO:teuthology.orchestra.run.smithi143.stdout: "name": "log_level", 2023-11-26T07:23:10.672 INFO:teuthology.orchestra.run.smithi143.stdout: "type": "str", 2023-11-26T07:23:10.672 INFO:teuthology.orchestra.run.smithi143.stdout: "level": "advanced", 2023-11-26T07:23:10.672 INFO:teuthology.orchestra.run.smithi143.stdout: "flags": 1, 2023-11-26T07:23:10.672 INFO:teuthology.orchestra.run.smithi143.stdout: "default_value": "", 2023-11-26T07:23:10.672 INFO:teuthology.orchestra.run.smithi143.stdout: "min": "", 2023-11-26T07:23:10.672 INFO:teuthology.orchestra.run.smithi143.stdout: "max": "", 2023-11-26T07:23:10.672 INFO:teuthology.orchestra.run.smithi143.stdout: "enum_allowed": [ 2023-11-26T07:23:10.672 INFO:teuthology.orchestra.run.smithi143.stdout: "", 2023-11-26T07:23:10.672 INFO:teuthology.orchestra.run.smithi143.stdout: "critical", 2023-11-26T07:23:10.672 INFO:teuthology.orchestra.run.smithi143.stdout: "debug", 2023-11-26T07:23:10.672 INFO:teuthology.orchestra.run.smithi143.stdout: "error", 2023-11-26T07:23:10.673 INFO:teuthology.orchestra.run.smithi143.stdout: "info", 2023-11-26T07:23:10.673 INFO:teuthology.orchestra.run.smithi143.stdout: "warning" 2023-11-26T07:23:10.673 INFO:teuthology.orchestra.run.smithi143.stdout: ], 2023-11-26T07:23:10.673 INFO:teuthology.orchestra.run.smithi143.stdout: "desc": "", 2023-11-26T07:23:10.673 INFO:teuthology.orchestra.run.smithi143.stdout: "long_desc": "", 2023-11-26T07:23:10.673 INFO:teuthology.orchestra.run.smithi143.stdout: "tags": [], 2023-11-26T07:23:10.673 INFO:teuthology.orchestra.run.smithi143.stdout: "see_also": [] 2023-11-26T07:23:10.673 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2023-11-26T07:23:10.673 INFO:teuthology.orchestra.run.smithi143.stdout: "log_to_cluster": { 2023-11-26T07:23:10.673 INFO:teuthology.orchestra.run.smithi143.stdout: "name": "log_to_cluster", 2023-11-26T07:23:10.673 INFO:teuthology.orchestra.run.smithi143.stdout: "type": "bool", 2023-11-26T07:23:10.673 INFO:teuthology.orchestra.run.smithi143.stdout: "level": "advanced", 2023-11-26T07:23:10.673 INFO:teuthology.orchestra.run.smithi143.stdout: "flags": 1, 2023-11-26T07:23:10.673 INFO:teuthology.orchestra.run.smithi143.stdout: "default_value": "False", 2023-11-26T07:23:10.674 INFO:teuthology.orchestra.run.smithi143.stdout: "min": "", 2023-11-26T07:23:10.674 INFO:teuthology.orchestra.run.smithi143.stdout: "max": "", 2023-11-26T07:23:10.674 INFO:teuthology.orchestra.run.smithi143.stdout: "enum_allowed": [], 2023-11-26T07:23:10.674 INFO:teuthology.orchestra.run.smithi143.stdout: "desc": "", 2023-11-26T07:23:10.674 INFO:teuthology.orchestra.run.smithi143.stdout: "long_desc": "", 2023-11-26T07:23:10.674 INFO:teuthology.orchestra.run.smithi143.stdout: "tags": [], 2023-11-26T07:23:10.674 INFO:teuthology.orchestra.run.smithi143.stdout: "see_also": [] 2023-11-26T07:23:10.674 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2023-11-26T07:23:10.674 INFO:teuthology.orchestra.run.smithi143.stdout: "log_to_cluster_level": { 2023-11-26T07:23:10.674 INFO:teuthology.orchestra.run.smithi143.stdout: "name": "log_to_cluster_level", 2023-11-26T07:23:10.674 INFO:teuthology.orchestra.run.smithi143.stdout: "type": "str", 2023-11-26T07:23:10.674 INFO:teuthology.orchestra.run.smithi143.stdout: "level": "advanced", 2023-11-26T07:23:10.675 INFO:teuthology.orchestra.run.smithi143.stdout: "flags": 1, 2023-11-26T07:23:10.675 INFO:teuthology.orchestra.run.smithi143.stdout: "default_value": "info", 2023-11-26T07:23:10.675 INFO:teuthology.orchestra.run.smithi143.stdout: "min": "", 2023-11-26T07:23:10.675 INFO:teuthology.orchestra.run.smithi143.stdout: "max": "", 2023-11-26T07:23:10.675 INFO:teuthology.orchestra.run.smithi143.stdout: "enum_allowed": [ 2023-11-26T07:23:10.675 INFO:teuthology.orchestra.run.smithi143.stdout: "", 2023-11-26T07:23:10.675 INFO:teuthology.orchestra.run.smithi143.stdout: "critical", 2023-11-26T07:23:10.675 INFO:teuthology.orchestra.run.smithi143.stdout: "debug", 2023-11-26T07:23:10.675 INFO:teuthology.orchestra.run.smithi143.stdout: "error", 2023-11-26T07:23:10.675 INFO:teuthology.orchestra.run.smithi143.stdout: "info", 2023-11-26T07:23:10.675 INFO:teuthology.orchestra.run.smithi143.stdout: "warning" 2023-11-26T07:23:10.675 INFO:teuthology.orchestra.run.smithi143.stdout: ], 2023-11-26T07:23:10.675 INFO:teuthology.orchestra.run.smithi143.stdout: "desc": "", 2023-11-26T07:23:10.676 INFO:teuthology.orchestra.run.smithi143.stdout: "long_desc": "", 2023-11-26T07:23:10.676 INFO:teuthology.orchestra.run.smithi143.stdout: "tags": [], 2023-11-26T07:23:10.676 INFO:teuthology.orchestra.run.smithi143.stdout: "see_also": [] 2023-11-26T07:23:10.676 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2023-11-26T07:23:10.676 INFO:teuthology.orchestra.run.smithi143.stdout: "log_to_file": { 2023-11-26T07:23:10.676 INFO:teuthology.orchestra.run.smithi143.stdout: "name": "log_to_file", 2023-11-26T07:23:10.676 INFO:teuthology.orchestra.run.smithi143.stdout: "type": "bool", 2023-11-26T07:23:10.676 INFO:teuthology.orchestra.run.smithi143.stdout: "level": "advanced", 2023-11-26T07:23:10.676 INFO:teuthology.orchestra.run.smithi143.stdout: "flags": 1, 2023-11-26T07:23:10.676 INFO:teuthology.orchestra.run.smithi143.stdout: "default_value": "False", 2023-11-26T07:23:10.676 INFO:teuthology.orchestra.run.smithi143.stdout: "min": "", 2023-11-26T07:23:10.677 INFO:teuthology.orchestra.run.smithi143.stdout: "max": "", 2023-11-26T07:23:10.677 INFO:teuthology.orchestra.run.smithi143.stdout: "enum_allowed": [], 2023-11-26T07:23:10.677 INFO:teuthology.orchestra.run.smithi143.stdout: "desc": "", 2023-11-26T07:23:10.677 INFO:teuthology.orchestra.run.smithi143.stdout: "long_desc": "", 2023-11-26T07:23:10.677 INFO:teuthology.orchestra.run.smithi143.stdout: "tags": [], 2023-11-26T07:23:10.677 INFO:teuthology.orchestra.run.smithi143.stdout: "see_also": [] 2023-11-26T07:23:10.677 INFO:teuthology.orchestra.run.smithi143.stdout: } 2023-11-26T07:23:10.677 INFO:teuthology.orchestra.run.smithi143.stdout: } 2023-11-26T07:23:10.677 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2023-11-26T07:23:10.677 INFO:teuthology.orchestra.run.smithi143.stdout: { 2023-11-26T07:23:10.677 INFO:teuthology.orchestra.run.smithi143.stdout: "name": "iostat", 2023-11-26T07:23:10.677 INFO:teuthology.orchestra.run.smithi143.stdout: "can_run": true, 2023-11-26T07:23:10.677 INFO:teuthology.orchestra.run.smithi143.stdout: "error_string": "", 2023-11-26T07:23:10.678 INFO:teuthology.orchestra.run.smithi143.stdout: "module_options": { 2023-11-26T07:23:10.678 INFO:teuthology.orchestra.run.smithi143.stdout: "log_level": { 2023-11-26T07:23:10.678 INFO:teuthology.orchestra.run.smithi143.stdout: "name": "log_level", 2023-11-26T07:23:10.678 INFO:teuthology.orchestra.run.smithi143.stdout: "type": "str", 2023-11-26T07:23:10.678 INFO:teuthology.orchestra.run.smithi143.stdout: "level": "advanced", 2023-11-26T07:23:10.678 INFO:teuthology.orchestra.run.smithi143.stdout: "flags": 1, 2023-11-26T07:23:10.678 INFO:teuthology.orchestra.run.smithi143.stdout: "default_value": "", 2023-11-26T07:23:10.678 INFO:teuthology.orchestra.run.smithi143.stdout: "min": "", 2023-11-26T07:23:10.678 INFO:teuthology.orchestra.run.smithi143.stdout: "max": "", 2023-11-26T07:23:10.678 INFO:teuthology.orchestra.run.smithi143.stdout: "enum_allowed": [ 2023-11-26T07:23:10.678 INFO:teuthology.orchestra.run.smithi143.stdout: "", 2023-11-26T07:23:10.678 INFO:teuthology.orchestra.run.smithi143.stdout: "critical", 2023-11-26T07:23:10.679 INFO:teuthology.orchestra.run.smithi143.stdout: "debug", 2023-11-26T07:23:10.679 INFO:teuthology.orchestra.run.smithi143.stdout: "error", 2023-11-26T07:23:10.679 INFO:teuthology.orchestra.run.smithi143.stdout: "info", 2023-11-26T07:23:10.679 INFO:teuthology.orchestra.run.smithi143.stdout: "warning" 2023-11-26T07:23:10.679 INFO:teuthology.orchestra.run.smithi143.stdout: ], 2023-11-26T07:23:10.679 INFO:teuthology.orchestra.run.smithi143.stdout: "desc": "", 2023-11-26T07:23:10.679 INFO:teuthology.orchestra.run.smithi143.stdout: "long_desc": "", 2023-11-26T07:23:10.679 INFO:teuthology.orchestra.run.smithi143.stdout: "tags": [], 2023-11-26T07:23:10.679 INFO:teuthology.orchestra.run.smithi143.stdout: "see_also": [] 2023-11-26T07:23:10.679 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2023-11-26T07:23:10.679 INFO:teuthology.orchestra.run.smithi143.stdout: "log_to_cluster": { 2023-11-26T07:23:10.679 INFO:teuthology.orchestra.run.smithi143.stdout: "name": "log_to_cluster", 2023-11-26T07:23:10.680 INFO:teuthology.orchestra.run.smithi143.stdout: "type": "bool", 2023-11-26T07:23:10.680 INFO:teuthology.orchestra.run.smithi143.stdout: "level": "advanced", 2023-11-26T07:23:10.680 INFO:teuthology.orchestra.run.smithi143.stdout: "flags": 1, 2023-11-26T07:23:10.680 INFO:teuthology.orchestra.run.smithi143.stdout: "default_value": "False", 2023-11-26T07:23:10.680 INFO:teuthology.orchestra.run.smithi143.stdout: "min": "", 2023-11-26T07:23:10.680 INFO:teuthology.orchestra.run.smithi143.stdout: "max": "", 2023-11-26T07:23:10.680 INFO:teuthology.orchestra.run.smithi143.stdout: "enum_allowed": [], 2023-11-26T07:23:10.680 INFO:teuthology.orchestra.run.smithi143.stdout: "desc": "", 2023-11-26T07:23:10.680 INFO:teuthology.orchestra.run.smithi143.stdout: "long_desc": "", 2023-11-26T07:23:10.680 INFO:teuthology.orchestra.run.smithi143.stdout: "tags": [], 2023-11-26T07:23:10.680 INFO:teuthology.orchestra.run.smithi143.stdout: "see_also": [] 2023-11-26T07:23:10.680 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2023-11-26T07:23:10.680 INFO:teuthology.orchestra.run.smithi143.stdout: "log_to_cluster_level": { 2023-11-26T07:23:10.681 INFO:teuthology.orchestra.run.smithi143.stdout: "name": "log_to_cluster_level", 2023-11-26T07:23:10.681 INFO:teuthology.orchestra.run.smithi143.stdout: "type": "str", 2023-11-26T07:23:10.681 INFO:teuthology.orchestra.run.smithi143.stdout: "level": "advanced", 2023-11-26T07:23:10.681 INFO:teuthology.orchestra.run.smithi143.stdout: "flags": 1, 2023-11-26T07:23:10.681 INFO:teuthology.orchestra.run.smithi143.stdout: "default_value": "info", 2023-11-26T07:23:10.681 INFO:teuthology.orchestra.run.smithi143.stdout: "min": "", 2023-11-26T07:23:10.681 INFO:teuthology.orchestra.run.smithi143.stdout: "max": "", 2023-11-26T07:23:10.681 INFO:teuthology.orchestra.run.smithi143.stdout: "enum_allowed": [ 2023-11-26T07:23:10.681 INFO:teuthology.orchestra.run.smithi143.stdout: "", 2023-11-26T07:23:10.681 INFO:teuthology.orchestra.run.smithi143.stdout: "critical", 2023-11-26T07:23:10.681 INFO:teuthology.orchestra.run.smithi143.stdout: "debug", 2023-11-26T07:23:10.681 INFO:teuthology.orchestra.run.smithi143.stdout: "error", 2023-11-26T07:23:10.682 INFO:teuthology.orchestra.run.smithi143.stdout: "info", 2023-11-26T07:23:10.682 INFO:teuthology.orchestra.run.smithi143.stdout: "warning" 2023-11-26T07:23:10.682 INFO:teuthology.orchestra.run.smithi143.stdout: ], 2023-11-26T07:23:10.682 INFO:teuthology.orchestra.run.smithi143.stdout: "desc": "", 2023-11-26T07:23:10.682 INFO:teuthology.orchestra.run.smithi143.stdout: "long_desc": "", 2023-11-26T07:23:10.682 INFO:teuthology.orchestra.run.smithi143.stdout: "tags": [], 2023-11-26T07:23:10.682 INFO:teuthology.orchestra.run.smithi143.stdout: "see_also": [] 2023-11-26T07:23:10.682 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2023-11-26T07:23:10.682 INFO:teuthology.orchestra.run.smithi143.stdout: "log_to_file": { 2023-11-26T07:23:10.682 INFO:teuthology.orchestra.run.smithi143.stdout: "name": "log_to_file", 2023-11-26T07:23:10.682 INFO:teuthology.orchestra.run.smithi143.stdout: "type": "bool", 2023-11-26T07:23:10.682 INFO:teuthology.orchestra.run.smithi143.stdout: "level": "advanced", 2023-11-26T07:23:10.683 INFO:teuthology.orchestra.run.smithi143.stdout: "flags": 1, 2023-11-26T07:23:10.683 INFO:teuthology.orchestra.run.smithi143.stdout: "default_value": "False", 2023-11-26T07:23:10.683 INFO:teuthology.orchestra.run.smithi143.stdout: "min": "", 2023-11-26T07:23:10.683 INFO:teuthology.orchestra.run.smithi143.stdout: "max": "", 2023-11-26T07:23:10.683 INFO:teuthology.orchestra.run.smithi143.stdout: "enum_allowed": [], 2023-11-26T07:23:10.683 INFO:teuthology.orchestra.run.smithi143.stdout: "desc": "", 2023-11-26T07:23:10.683 INFO:teuthology.orchestra.run.smithi143.stdout: "long_desc": "", 2023-11-26T07:23:10.683 INFO:teuthology.orchestra.run.smithi143.stdout: "tags": [], 2023-11-26T07:23:10.683 INFO:teuthology.orchestra.run.smithi143.stdout: "see_also": [] 2023-11-26T07:23:10.683 INFO:teuthology.orchestra.run.smithi143.stdout: } 2023-11-26T07:23:10.683 INFO:teuthology.orchestra.run.smithi143.stdout: } 2023-11-26T07:23:10.683 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2023-11-26T07:23:10.684 INFO:teuthology.orchestra.run.smithi143.stdout: { 2023-11-26T07:23:10.684 INFO:teuthology.orchestra.run.smithi143.stdout: "name": "localpool", 2023-11-26T07:23:10.684 INFO:teuthology.orchestra.run.smithi143.stdout: "can_run": true, 2023-11-26T07:23:10.684 INFO:teuthology.orchestra.run.smithi143.stdout: "error_string": "", 2023-11-26T07:23:10.684 INFO:teuthology.orchestra.run.smithi143.stdout: "module_options": { 2023-11-26T07:23:10.684 INFO:teuthology.orchestra.run.smithi143.stdout: "failure_domain": { 2023-11-26T07:23:10.684 INFO:teuthology.orchestra.run.smithi143.stdout: "name": "failure_domain", 2023-11-26T07:23:10.684 INFO:teuthology.orchestra.run.smithi143.stdout: "type": "str", 2023-11-26T07:23:10.684 INFO:teuthology.orchestra.run.smithi143.stdout: "level": "advanced", 2023-11-26T07:23:10.684 INFO:teuthology.orchestra.run.smithi143.stdout: "flags": 1, 2023-11-26T07:23:10.684 INFO:teuthology.orchestra.run.smithi143.stdout: "default_value": "host", 2023-11-26T07:23:10.684 INFO:teuthology.orchestra.run.smithi143.stdout: "min": "", 2023-11-26T07:23:10.685 INFO:teuthology.orchestra.run.smithi143.stdout: "max": "", 2023-11-26T07:23:10.685 INFO:teuthology.orchestra.run.smithi143.stdout: "enum_allowed": [], 2023-11-26T07:23:10.685 INFO:teuthology.orchestra.run.smithi143.stdout: "desc": "failure domain for any created local pool", 2023-11-26T07:23:10.685 INFO:teuthology.orchestra.run.smithi143.stdout: "long_desc": "what failure domain we should separate data replicas across.", 2023-11-26T07:23:10.685 INFO:teuthology.orchestra.run.smithi143.stdout: "tags": [], 2023-11-26T07:23:10.685 INFO:teuthology.orchestra.run.smithi143.stdout: "see_also": [] 2023-11-26T07:23:10.685 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2023-11-26T07:23:10.685 INFO:teuthology.orchestra.run.smithi143.stdout: "log_level": { 2023-11-26T07:23:10.685 INFO:teuthology.orchestra.run.smithi143.stdout: "name": "log_level", 2023-11-26T07:23:10.685 INFO:teuthology.orchestra.run.smithi143.stdout: "type": "str", 2023-11-26T07:23:10.685 INFO:teuthology.orchestra.run.smithi143.stdout: "level": "advanced", 2023-11-26T07:23:10.685 INFO:teuthology.orchestra.run.smithi143.stdout: "flags": 1, 2023-11-26T07:23:10.685 INFO:teuthology.orchestra.run.smithi143.stdout: "default_value": "", 2023-11-26T07:23:10.686 INFO:teuthology.orchestra.run.smithi143.stdout: "min": "", 2023-11-26T07:23:10.686 INFO:teuthology.orchestra.run.smithi143.stdout: "max": "", 2023-11-26T07:23:10.686 INFO:teuthology.orchestra.run.smithi143.stdout: "enum_allowed": [ 2023-11-26T07:23:10.686 INFO:teuthology.orchestra.run.smithi143.stdout: "", 2023-11-26T07:23:10.686 INFO:teuthology.orchestra.run.smithi143.stdout: "critical", 2023-11-26T07:23:10.686 INFO:teuthology.orchestra.run.smithi143.stdout: "debug", 2023-11-26T07:23:10.686 INFO:teuthology.orchestra.run.smithi143.stdout: "error", 2023-11-26T07:23:10.686 INFO:teuthology.orchestra.run.smithi143.stdout: "info", 2023-11-26T07:23:10.686 INFO:teuthology.orchestra.run.smithi143.stdout: "warning" 2023-11-26T07:23:10.686 INFO:teuthology.orchestra.run.smithi143.stdout: ], 2023-11-26T07:23:10.686 INFO:teuthology.orchestra.run.smithi143.stdout: "desc": "", 2023-11-26T07:23:10.687 INFO:teuthology.orchestra.run.smithi143.stdout: "long_desc": "", 2023-11-26T07:23:10.687 INFO:teuthology.orchestra.run.smithi143.stdout: "tags": [], 2023-11-26T07:23:10.687 INFO:teuthology.orchestra.run.smithi143.stdout: "see_also": [] 2023-11-26T07:23:10.687 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2023-11-26T07:23:10.687 INFO:teuthology.orchestra.run.smithi143.stdout: "log_to_cluster": { 2023-11-26T07:23:10.687 INFO:teuthology.orchestra.run.smithi143.stdout: "name": "log_to_cluster", 2023-11-26T07:23:10.687 INFO:teuthology.orchestra.run.smithi143.stdout: "type": "bool", 2023-11-26T07:23:10.687 INFO:teuthology.orchestra.run.smithi143.stdout: "level": "advanced", 2023-11-26T07:23:10.687 INFO:teuthology.orchestra.run.smithi143.stdout: "flags": 1, 2023-11-26T07:23:10.687 INFO:teuthology.orchestra.run.smithi143.stdout: "default_value": "False", 2023-11-26T07:23:10.687 INFO:teuthology.orchestra.run.smithi143.stdout: "min": "", 2023-11-26T07:23:10.687 INFO:teuthology.orchestra.run.smithi143.stdout: "max": "", 2023-11-26T07:23:10.687 INFO:teuthology.orchestra.run.smithi143.stdout: "enum_allowed": [], 2023-11-26T07:23:10.688 INFO:teuthology.orchestra.run.smithi143.stdout: "desc": "", 2023-11-26T07:23:10.688 INFO:teuthology.orchestra.run.smithi143.stdout: "long_desc": "", 2023-11-26T07:23:10.688 INFO:teuthology.orchestra.run.smithi143.stdout: "tags": [], 2023-11-26T07:23:10.688 INFO:teuthology.orchestra.run.smithi143.stdout: "see_also": [] 2023-11-26T07:23:10.688 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2023-11-26T07:23:10.688 INFO:teuthology.orchestra.run.smithi143.stdout: "log_to_cluster_level": { 2023-11-26T07:23:10.688 INFO:teuthology.orchestra.run.smithi143.stdout: "name": "log_to_cluster_level", 2023-11-26T07:23:10.688 INFO:teuthology.orchestra.run.smithi143.stdout: "type": "str", 2023-11-26T07:23:10.688 INFO:teuthology.orchestra.run.smithi143.stdout: "level": "advanced", 2023-11-26T07:23:10.688 INFO:teuthology.orchestra.run.smithi143.stdout: "flags": 1, 2023-11-26T07:23:10.688 INFO:teuthology.orchestra.run.smithi143.stdout: "default_value": "info", 2023-11-26T07:23:10.688 INFO:teuthology.orchestra.run.smithi143.stdout: "min": "", 2023-11-26T07:23:10.688 INFO:teuthology.orchestra.run.smithi143.stdout: "max": "", 2023-11-26T07:23:10.689 INFO:teuthology.orchestra.run.smithi143.stdout: "enum_allowed": [ 2023-11-26T07:23:10.689 INFO:teuthology.orchestra.run.smithi143.stdout: "", 2023-11-26T07:23:10.689 INFO:teuthology.orchestra.run.smithi143.stdout: "critical", 2023-11-26T07:23:10.689 INFO:teuthology.orchestra.run.smithi143.stdout: "debug", 2023-11-26T07:23:10.689 INFO:teuthology.orchestra.run.smithi143.stdout: "error", 2023-11-26T07:23:10.689 INFO:teuthology.orchestra.run.smithi143.stdout: "info", 2023-11-26T07:23:10.689 INFO:teuthology.orchestra.run.smithi143.stdout: "warning" 2023-11-26T07:23:10.689 INFO:teuthology.orchestra.run.smithi143.stdout: ], 2023-11-26T07:23:10.689 INFO:teuthology.orchestra.run.smithi143.stdout: "desc": "", 2023-11-26T07:23:10.689 INFO:teuthology.orchestra.run.smithi143.stdout: "long_desc": "", 2023-11-26T07:23:10.689 INFO:teuthology.orchestra.run.smithi143.stdout: "tags": [], 2023-11-26T07:23:10.689 INFO:teuthology.orchestra.run.smithi143.stdout: "see_also": [] 2023-11-26T07:23:10.690 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2023-11-26T07:23:10.690 INFO:teuthology.orchestra.run.smithi143.stdout: "log_to_file": { 2023-11-26T07:23:10.690 INFO:teuthology.orchestra.run.smithi143.stdout: "name": "log_to_file", 2023-11-26T07:23:10.690 INFO:teuthology.orchestra.run.smithi143.stdout: "type": "bool", 2023-11-26T07:23:10.690 INFO:teuthology.orchestra.run.smithi143.stdout: "level": "advanced", 2023-11-26T07:23:10.690 INFO:teuthology.orchestra.run.smithi143.stdout: "flags": 1, 2023-11-26T07:23:10.690 INFO:teuthology.orchestra.run.smithi143.stdout: "default_value": "False", 2023-11-26T07:23:10.690 INFO:teuthology.orchestra.run.smithi143.stdout: "min": "", 2023-11-26T07:23:10.690 INFO:teuthology.orchestra.run.smithi143.stdout: "max": "", 2023-11-26T07:23:10.690 INFO:teuthology.orchestra.run.smithi143.stdout: "enum_allowed": [], 2023-11-26T07:23:10.690 INFO:teuthology.orchestra.run.smithi143.stdout: "desc": "", 2023-11-26T07:23:10.690 INFO:teuthology.orchestra.run.smithi143.stdout: "long_desc": "", 2023-11-26T07:23:10.691 INFO:teuthology.orchestra.run.smithi143.stdout: "tags": [], 2023-11-26T07:23:10.691 INFO:teuthology.orchestra.run.smithi143.stdout: "see_also": [] 2023-11-26T07:23:10.691 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2023-11-26T07:23:10.691 INFO:teuthology.orchestra.run.smithi143.stdout: "min_size": { 2023-11-26T07:23:10.691 INFO:teuthology.orchestra.run.smithi143.stdout: "name": "min_size", 2023-11-26T07:23:10.691 INFO:teuthology.orchestra.run.smithi143.stdout: "type": "int", 2023-11-26T07:23:10.691 INFO:teuthology.orchestra.run.smithi143.stdout: "level": "advanced", 2023-11-26T07:23:10.691 INFO:teuthology.orchestra.run.smithi143.stdout: "flags": 1, 2023-11-26T07:23:10.691 INFO:teuthology.orchestra.run.smithi143.stdout: "default_value": "", 2023-11-26T07:23:10.691 INFO:teuthology.orchestra.run.smithi143.stdout: "min": "", 2023-11-26T07:23:10.691 INFO:teuthology.orchestra.run.smithi143.stdout: "max": "", 2023-11-26T07:23:10.691 INFO:teuthology.orchestra.run.smithi143.stdout: "enum_allowed": [], 2023-11-26T07:23:10.691 INFO:teuthology.orchestra.run.smithi143.stdout: "desc": "default min_size for any created local pool", 2023-11-26T07:23:10.692 INFO:teuthology.orchestra.run.smithi143.stdout: "long_desc": "value to set min_size to (unchanged from Ceph's default if this option is not set)", 2023-11-26T07:23:10.692 INFO:teuthology.orchestra.run.smithi143.stdout: "tags": [], 2023-11-26T07:23:10.692 INFO:teuthology.orchestra.run.smithi143.stdout: "see_also": [] 2023-11-26T07:23:10.692 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2023-11-26T07:23:10.692 INFO:teuthology.orchestra.run.smithi143.stdout: "num_rep": { 2023-11-26T07:23:10.692 INFO:teuthology.orchestra.run.smithi143.stdout: "name": "num_rep", 2023-11-26T07:23:10.692 INFO:teuthology.orchestra.run.smithi143.stdout: "type": "int", 2023-11-26T07:23:10.692 INFO:teuthology.orchestra.run.smithi143.stdout: "level": "advanced", 2023-11-26T07:23:10.692 INFO:teuthology.orchestra.run.smithi143.stdout: "flags": 1, 2023-11-26T07:23:10.692 INFO:teuthology.orchestra.run.smithi143.stdout: "default_value": "3", 2023-11-26T07:23:10.692 INFO:teuthology.orchestra.run.smithi143.stdout: "min": "", 2023-11-26T07:23:10.692 INFO:teuthology.orchestra.run.smithi143.stdout: "max": "", 2023-11-26T07:23:10.693 INFO:teuthology.orchestra.run.smithi143.stdout: "enum_allowed": [], 2023-11-26T07:23:10.693 INFO:teuthology.orchestra.run.smithi143.stdout: "desc": "default replica count for any created local pool", 2023-11-26T07:23:10.693 INFO:teuthology.orchestra.run.smithi143.stdout: "long_desc": "", 2023-11-26T07:23:10.693 INFO:teuthology.orchestra.run.smithi143.stdout: "tags": [], 2023-11-26T07:23:10.693 INFO:teuthology.orchestra.run.smithi143.stdout: "see_also": [] 2023-11-26T07:23:10.693 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2023-11-26T07:23:10.693 INFO:teuthology.orchestra.run.smithi143.stdout: "pg_num": { 2023-11-26T07:23:10.693 INFO:teuthology.orchestra.run.smithi143.stdout: "name": "pg_num", 2023-11-26T07:23:10.693 INFO:teuthology.orchestra.run.smithi143.stdout: "type": "int", 2023-11-26T07:23:10.693 INFO:teuthology.orchestra.run.smithi143.stdout: "level": "advanced", 2023-11-26T07:23:10.693 INFO:teuthology.orchestra.run.smithi143.stdout: "flags": 1, 2023-11-26T07:23:10.693 INFO:teuthology.orchestra.run.smithi143.stdout: "default_value": "128", 2023-11-26T07:23:10.693 INFO:teuthology.orchestra.run.smithi143.stdout: "min": "", 2023-11-26T07:23:10.694 INFO:teuthology.orchestra.run.smithi143.stdout: "max": "", 2023-11-26T07:23:10.694 INFO:teuthology.orchestra.run.smithi143.stdout: "enum_allowed": [], 2023-11-26T07:23:10.694 INFO:teuthology.orchestra.run.smithi143.stdout: "desc": "default pg_num for any created local pool", 2023-11-26T07:23:10.694 INFO:teuthology.orchestra.run.smithi143.stdout: "long_desc": "", 2023-11-26T07:23:10.694 INFO:teuthology.orchestra.run.smithi143.stdout: "tags": [], 2023-11-26T07:23:10.694 INFO:teuthology.orchestra.run.smithi143.stdout: "see_also": [] 2023-11-26T07:23:10.694 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2023-11-26T07:23:10.694 INFO:teuthology.orchestra.run.smithi143.stdout: "prefix": { 2023-11-26T07:23:10.694 INFO:teuthology.orchestra.run.smithi143.stdout: "name": "prefix", 2023-11-26T07:23:10.694 INFO:teuthology.orchestra.run.smithi143.stdout: "type": "str", 2023-11-26T07:23:10.694 INFO:teuthology.orchestra.run.smithi143.stdout: "level": "advanced", 2023-11-26T07:23:10.694 INFO:teuthology.orchestra.run.smithi143.stdout: "flags": 1, 2023-11-26T07:23:10.695 INFO:teuthology.orchestra.run.smithi143.stdout: "default_value": "", 2023-11-26T07:23:10.695 INFO:teuthology.orchestra.run.smithi143.stdout: "min": "", 2023-11-26T07:23:10.695 INFO:teuthology.orchestra.run.smithi143.stdout: "max": "", 2023-11-26T07:23:10.695 INFO:teuthology.orchestra.run.smithi143.stdout: "enum_allowed": [], 2023-11-26T07:23:10.695 INFO:teuthology.orchestra.run.smithi143.stdout: "desc": "name prefix for any created local pool", 2023-11-26T07:23:10.695 INFO:teuthology.orchestra.run.smithi143.stdout: "long_desc": "", 2023-11-26T07:23:10.695 INFO:teuthology.orchestra.run.smithi143.stdout: "tags": [], 2023-11-26T07:23:10.695 INFO:teuthology.orchestra.run.smithi143.stdout: "see_also": [] 2023-11-26T07:23:10.695 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2023-11-26T07:23:10.695 INFO:teuthology.orchestra.run.smithi143.stdout: "subtree": { 2023-11-26T07:23:10.695 INFO:teuthology.orchestra.run.smithi143.stdout: "name": "subtree", 2023-11-26T07:23:10.695 INFO:teuthology.orchestra.run.smithi143.stdout: "type": "str", 2023-11-26T07:23:10.695 INFO:teuthology.orchestra.run.smithi143.stdout: "level": "advanced", 2023-11-26T07:23:10.696 INFO:teuthology.orchestra.run.smithi143.stdout: "flags": 1, 2023-11-26T07:23:10.696 INFO:teuthology.orchestra.run.smithi143.stdout: "default_value": "rack", 2023-11-26T07:23:10.696 INFO:teuthology.orchestra.run.smithi143.stdout: "min": "", 2023-11-26T07:23:10.696 INFO:teuthology.orchestra.run.smithi143.stdout: "max": "", 2023-11-26T07:23:10.696 INFO:teuthology.orchestra.run.smithi143.stdout: "enum_allowed": [], 2023-11-26T07:23:10.696 INFO:teuthology.orchestra.run.smithi143.stdout: "desc": "CRUSH level for which to create a local pool", 2023-11-26T07:23:10.696 INFO:teuthology.orchestra.run.smithi143.stdout: "long_desc": "which CRUSH subtree type the module should create a pool for.", 2023-11-26T07:23:10.696 INFO:teuthology.orchestra.run.smithi143.stdout: "tags": [], 2023-11-26T07:23:10.696 INFO:teuthology.orchestra.run.smithi143.stdout: "see_also": [] 2023-11-26T07:23:10.696 INFO:teuthology.orchestra.run.smithi143.stdout: } 2023-11-26T07:23:10.696 INFO:teuthology.orchestra.run.smithi143.stdout: } 2023-11-26T07:23:10.696 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2023-11-26T07:23:10.697 INFO:teuthology.orchestra.run.smithi143.stdout: { 2023-11-26T07:23:10.697 INFO:teuthology.orchestra.run.smithi143.stdout: "name": "mirroring", 2023-11-26T07:23:10.697 INFO:teuthology.orchestra.run.smithi143.stdout: "can_run": true, 2023-11-26T07:23:10.697 INFO:teuthology.orchestra.run.smithi143.stdout: "error_string": "", 2023-11-26T07:23:10.697 INFO:teuthology.orchestra.run.smithi143.stdout: "module_options": { 2023-11-26T07:23:10.697 INFO:teuthology.orchestra.run.smithi143.stdout: "log_level": { 2023-11-26T07:23:10.697 INFO:teuthology.orchestra.run.smithi143.stdout: "name": "log_level", 2023-11-26T07:23:10.697 INFO:teuthology.orchestra.run.smithi143.stdout: "type": "str", 2023-11-26T07:23:10.697 INFO:teuthology.orchestra.run.smithi143.stdout: "level": "advanced", 2023-11-26T07:23:10.697 INFO:teuthology.orchestra.run.smithi143.stdout: "flags": 1, 2023-11-26T07:23:10.697 INFO:teuthology.orchestra.run.smithi143.stdout: "default_value": "", 2023-11-26T07:23:10.697 INFO:teuthology.orchestra.run.smithi143.stdout: "min": "", 2023-11-26T07:23:10.697 INFO:teuthology.orchestra.run.smithi143.stdout: "max": "", 2023-11-26T07:23:10.698 INFO:teuthology.orchestra.run.smithi143.stdout: "enum_allowed": [ 2023-11-26T07:23:10.698 INFO:teuthology.orchestra.run.smithi143.stdout: "", 2023-11-26T07:23:10.698 INFO:teuthology.orchestra.run.smithi143.stdout: "critical", 2023-11-26T07:23:10.698 INFO:teuthology.orchestra.run.smithi143.stdout: "debug", 2023-11-26T07:23:10.698 INFO:teuthology.orchestra.run.smithi143.stdout: "error", 2023-11-26T07:23:10.698 INFO:teuthology.orchestra.run.smithi143.stdout: "info", 2023-11-26T07:23:10.698 INFO:teuthology.orchestra.run.smithi143.stdout: "warning" 2023-11-26T07:23:10.698 INFO:teuthology.orchestra.run.smithi143.stdout: ], 2023-11-26T07:23:10.698 INFO:teuthology.orchestra.run.smithi143.stdout: "desc": "", 2023-11-26T07:23:10.698 INFO:teuthology.orchestra.run.smithi143.stdout: "long_desc": "", 2023-11-26T07:23:10.698 INFO:teuthology.orchestra.run.smithi143.stdout: "tags": [], 2023-11-26T07:23:10.698 INFO:teuthology.orchestra.run.smithi143.stdout: "see_also": [] 2023-11-26T07:23:10.699 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2023-11-26T07:23:10.699 INFO:teuthology.orchestra.run.smithi143.stdout: "log_to_cluster": { 2023-11-26T07:23:10.699 INFO:teuthology.orchestra.run.smithi143.stdout: "name": "log_to_cluster", 2023-11-26T07:23:10.699 INFO:teuthology.orchestra.run.smithi143.stdout: "type": "bool", 2023-11-26T07:23:10.699 INFO:teuthology.orchestra.run.smithi143.stdout: "level": "advanced", 2023-11-26T07:23:10.699 INFO:teuthology.orchestra.run.smithi143.stdout: "flags": 1, 2023-11-26T07:23:10.699 INFO:teuthology.orchestra.run.smithi143.stdout: "default_value": "False", 2023-11-26T07:23:10.699 INFO:teuthology.orchestra.run.smithi143.stdout: "min": "", 2023-11-26T07:23:10.699 INFO:teuthology.orchestra.run.smithi143.stdout: "max": "", 2023-11-26T07:23:10.699 INFO:teuthology.orchestra.run.smithi143.stdout: "enum_allowed": [], 2023-11-26T07:23:10.699 INFO:teuthology.orchestra.run.smithi143.stdout: "desc": "", 2023-11-26T07:23:10.699 INFO:teuthology.orchestra.run.smithi143.stdout: "long_desc": "", 2023-11-26T07:23:10.699 INFO:teuthology.orchestra.run.smithi143.stdout: "tags": [], 2023-11-26T07:23:10.700 INFO:teuthology.orchestra.run.smithi143.stdout: "see_also": [] 2023-11-26T07:23:10.700 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2023-11-26T07:23:10.700 INFO:teuthology.orchestra.run.smithi143.stdout: "log_to_cluster_level": { 2023-11-26T07:23:10.700 INFO:teuthology.orchestra.run.smithi143.stdout: "name": "log_to_cluster_level", 2023-11-26T07:23:10.700 INFO:teuthology.orchestra.run.smithi143.stdout: "type": "str", 2023-11-26T07:23:10.700 INFO:teuthology.orchestra.run.smithi143.stdout: "level": "advanced", 2023-11-26T07:23:10.700 INFO:teuthology.orchestra.run.smithi143.stdout: "flags": 1, 2023-11-26T07:23:10.700 INFO:teuthology.orchestra.run.smithi143.stdout: "default_value": "info", 2023-11-26T07:23:10.700 INFO:teuthology.orchestra.run.smithi143.stdout: "min": "", 2023-11-26T07:23:10.700 INFO:teuthology.orchestra.run.smithi143.stdout: "max": "", 2023-11-26T07:23:10.700 INFO:teuthology.orchestra.run.smithi143.stdout: "enum_allowed": [ 2023-11-26T07:23:10.700 INFO:teuthology.orchestra.run.smithi143.stdout: "", 2023-11-26T07:23:10.700 INFO:teuthology.orchestra.run.smithi143.stdout: "critical", 2023-11-26T07:23:10.701 INFO:teuthology.orchestra.run.smithi143.stdout: "debug", 2023-11-26T07:23:10.701 INFO:teuthology.orchestra.run.smithi143.stdout: "error", 2023-11-26T07:23:10.701 INFO:teuthology.orchestra.run.smithi143.stdout: "info", 2023-11-26T07:23:10.701 INFO:teuthology.orchestra.run.smithi143.stdout: "warning" 2023-11-26T07:23:10.701 INFO:teuthology.orchestra.run.smithi143.stdout: ], 2023-11-26T07:23:10.701 INFO:teuthology.orchestra.run.smithi143.stdout: "desc": "", 2023-11-26T07:23:10.701 INFO:teuthology.orchestra.run.smithi143.stdout: "long_desc": "", 2023-11-26T07:23:10.701 INFO:teuthology.orchestra.run.smithi143.stdout: "tags": [], 2023-11-26T07:23:10.701 INFO:teuthology.orchestra.run.smithi143.stdout: "see_also": [] 2023-11-26T07:23:10.701 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2023-11-26T07:23:10.701 INFO:teuthology.orchestra.run.smithi143.stdout: "log_to_file": { 2023-11-26T07:23:10.701 INFO:teuthology.orchestra.run.smithi143.stdout: "name": "log_to_file", 2023-11-26T07:23:10.702 INFO:teuthology.orchestra.run.smithi143.stdout: "type": "bool", 2023-11-26T07:23:10.702 INFO:teuthology.orchestra.run.smithi143.stdout: "level": "advanced", 2023-11-26T07:23:10.702 INFO:teuthology.orchestra.run.smithi143.stdout: "flags": 1, 2023-11-26T07:23:10.702 INFO:teuthology.orchestra.run.smithi143.stdout: "default_value": "False", 2023-11-26T07:23:10.702 INFO:teuthology.orchestra.run.smithi143.stdout: "min": "", 2023-11-26T07:23:10.702 INFO:teuthology.orchestra.run.smithi143.stdout: "max": "", 2023-11-26T07:23:10.702 INFO:teuthology.orchestra.run.smithi143.stdout: "enum_allowed": [], 2023-11-26T07:23:10.702 INFO:teuthology.orchestra.run.smithi143.stdout: "desc": "", 2023-11-26T07:23:10.702 INFO:teuthology.orchestra.run.smithi143.stdout: "long_desc": "", 2023-11-26T07:23:10.702 INFO:teuthology.orchestra.run.smithi143.stdout: "tags": [], 2023-11-26T07:23:10.702 INFO:teuthology.orchestra.run.smithi143.stdout: "see_also": [] 2023-11-26T07:23:10.702 INFO:teuthology.orchestra.run.smithi143.stdout: } 2023-11-26T07:23:10.703 INFO:teuthology.orchestra.run.smithi143.stdout: } 2023-11-26T07:23:10.703 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2023-11-26T07:23:10.703 INFO:teuthology.orchestra.run.smithi143.stdout: { 2023-11-26T07:23:10.703 INFO:teuthology.orchestra.run.smithi143.stdout: "name": "nfs", 2023-11-26T07:23:10.703 INFO:teuthology.orchestra.run.smithi143.stdout: "can_run": true, 2023-11-26T07:23:10.703 INFO:teuthology.orchestra.run.smithi143.stdout: "error_string": "", 2023-11-26T07:23:10.703 INFO:teuthology.orchestra.run.smithi143.stdout: "module_options": { 2023-11-26T07:23:10.703 INFO:teuthology.orchestra.run.smithi143.stdout: "log_level": { 2023-11-26T07:23:10.703 INFO:teuthology.orchestra.run.smithi143.stdout: "name": "log_level", 2023-11-26T07:23:10.703 INFO:teuthology.orchestra.run.smithi143.stdout: "type": "str", 2023-11-26T07:23:10.703 INFO:teuthology.orchestra.run.smithi143.stdout: "level": "advanced", 2023-11-26T07:23:10.703 INFO:teuthology.orchestra.run.smithi143.stdout: "flags": 1, 2023-11-26T07:23:10.703 INFO:teuthology.orchestra.run.smithi143.stdout: "default_value": "", 2023-11-26T07:23:10.704 INFO:teuthology.orchestra.run.smithi143.stdout: "min": "", 2023-11-26T07:23:10.704 INFO:teuthology.orchestra.run.smithi143.stdout: "max": "", 2023-11-26T07:23:10.704 INFO:teuthology.orchestra.run.smithi143.stdout: "enum_allowed": [ 2023-11-26T07:23:10.704 INFO:teuthology.orchestra.run.smithi143.stdout: "", 2023-11-26T07:23:10.704 INFO:teuthology.orchestra.run.smithi143.stdout: "critical", 2023-11-26T07:23:10.704 INFO:teuthology.orchestra.run.smithi143.stdout: "debug", 2023-11-26T07:23:10.704 INFO:teuthology.orchestra.run.smithi143.stdout: "error", 2023-11-26T07:23:10.704 INFO:teuthology.orchestra.run.smithi143.stdout: "info", 2023-11-26T07:23:10.704 INFO:teuthology.orchestra.run.smithi143.stdout: "warning" 2023-11-26T07:23:10.704 INFO:teuthology.orchestra.run.smithi143.stdout: ], 2023-11-26T07:23:10.704 INFO:teuthology.orchestra.run.smithi143.stdout: "desc": "", 2023-11-26T07:23:10.704 INFO:teuthology.orchestra.run.smithi143.stdout: "long_desc": "", 2023-11-26T07:23:10.705 INFO:teuthology.orchestra.run.smithi143.stdout: "tags": [], 2023-11-26T07:23:10.705 INFO:teuthology.orchestra.run.smithi143.stdout: "see_also": [] 2023-11-26T07:23:10.705 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2023-11-26T07:23:10.705 INFO:teuthology.orchestra.run.smithi143.stdout: "log_to_cluster": { 2023-11-26T07:23:10.705 INFO:teuthology.orchestra.run.smithi143.stdout: "name": "log_to_cluster", 2023-11-26T07:23:10.705 INFO:teuthology.orchestra.run.smithi143.stdout: "type": "bool", 2023-11-26T07:23:10.705 INFO:teuthology.orchestra.run.smithi143.stdout: "level": "advanced", 2023-11-26T07:23:10.705 INFO:teuthology.orchestra.run.smithi143.stdout: "flags": 1, 2023-11-26T07:23:10.705 INFO:teuthology.orchestra.run.smithi143.stdout: "default_value": "False", 2023-11-26T07:23:10.705 INFO:teuthology.orchestra.run.smithi143.stdout: "min": "", 2023-11-26T07:23:10.705 INFO:teuthology.orchestra.run.smithi143.stdout: "max": "", 2023-11-26T07:23:10.705 INFO:teuthology.orchestra.run.smithi143.stdout: "enum_allowed": [], 2023-11-26T07:23:10.705 INFO:teuthology.orchestra.run.smithi143.stdout: "desc": "", 2023-11-26T07:23:10.706 INFO:teuthology.orchestra.run.smithi143.stdout: "long_desc": "", 2023-11-26T07:23:10.706 INFO:teuthology.orchestra.run.smithi143.stdout: "tags": [], 2023-11-26T07:23:10.706 INFO:teuthology.orchestra.run.smithi143.stdout: "see_also": [] 2023-11-26T07:23:10.706 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2023-11-26T07:23:10.706 INFO:teuthology.orchestra.run.smithi143.stdout: "log_to_cluster_level": { 2023-11-26T07:23:10.706 INFO:teuthology.orchestra.run.smithi143.stdout: "name": "log_to_cluster_level", 2023-11-26T07:23:10.706 INFO:teuthology.orchestra.run.smithi143.stdout: "type": "str", 2023-11-26T07:23:10.706 INFO:teuthology.orchestra.run.smithi143.stdout: "level": "advanced", 2023-11-26T07:23:10.706 INFO:teuthology.orchestra.run.smithi143.stdout: "flags": 1, 2023-11-26T07:23:10.706 INFO:teuthology.orchestra.run.smithi143.stdout: "default_value": "info", 2023-11-26T07:23:10.706 INFO:teuthology.orchestra.run.smithi143.stdout: "min": "", 2023-11-26T07:23:10.707 INFO:teuthology.orchestra.run.smithi143.stdout: "max": "", 2023-11-26T07:23:10.707 INFO:teuthology.orchestra.run.smithi143.stdout: "enum_allowed": [ 2023-11-26T07:23:10.707 INFO:teuthology.orchestra.run.smithi143.stdout: "", 2023-11-26T07:23:10.707 INFO:teuthology.orchestra.run.smithi143.stdout: "critical", 2023-11-26T07:23:10.707 INFO:teuthology.orchestra.run.smithi143.stdout: "debug", 2023-11-26T07:23:10.707 INFO:teuthology.orchestra.run.smithi143.stdout: "error", 2023-11-26T07:23:10.707 INFO:teuthology.orchestra.run.smithi143.stdout: "info", 2023-11-26T07:23:10.707 INFO:teuthology.orchestra.run.smithi143.stdout: "warning" 2023-11-26T07:23:10.707 INFO:teuthology.orchestra.run.smithi143.stdout: ], 2023-11-26T07:23:10.707 INFO:teuthology.orchestra.run.smithi143.stdout: "desc": "", 2023-11-26T07:23:10.707 INFO:teuthology.orchestra.run.smithi143.stdout: "long_desc": "", 2023-11-26T07:23:10.707 INFO:teuthology.orchestra.run.smithi143.stdout: "tags": [], 2023-11-26T07:23:10.707 INFO:teuthology.orchestra.run.smithi143.stdout: "see_also": [] 2023-11-26T07:23:10.708 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2023-11-26T07:23:10.708 INFO:teuthology.orchestra.run.smithi143.stdout: "log_to_file": { 2023-11-26T07:23:10.708 INFO:teuthology.orchestra.run.smithi143.stdout: "name": "log_to_file", 2023-11-26T07:23:10.708 INFO:teuthology.orchestra.run.smithi143.stdout: "type": "bool", 2023-11-26T07:23:10.708 INFO:teuthology.orchestra.run.smithi143.stdout: "level": "advanced", 2023-11-26T07:23:10.708 INFO:teuthology.orchestra.run.smithi143.stdout: "flags": 1, 2023-11-26T07:23:10.708 INFO:teuthology.orchestra.run.smithi143.stdout: "default_value": "False", 2023-11-26T07:23:10.708 INFO:teuthology.orchestra.run.smithi143.stdout: "min": "", 2023-11-26T07:23:10.708 INFO:teuthology.orchestra.run.smithi143.stdout: "max": "", 2023-11-26T07:23:10.708 INFO:teuthology.orchestra.run.smithi143.stdout: "enum_allowed": [], 2023-11-26T07:23:10.708 INFO:teuthology.orchestra.run.smithi143.stdout: "desc": "", 2023-11-26T07:23:10.708 INFO:teuthology.orchestra.run.smithi143.stdout: "long_desc": "", 2023-11-26T07:23:10.708 INFO:teuthology.orchestra.run.smithi143.stdout: "tags": [], 2023-11-26T07:23:10.709 INFO:teuthology.orchestra.run.smithi143.stdout: "see_also": [] 2023-11-26T07:23:10.709 INFO:teuthology.orchestra.run.smithi143.stdout: } 2023-11-26T07:23:10.709 INFO:teuthology.orchestra.run.smithi143.stdout: } 2023-11-26T07:23:10.709 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2023-11-26T07:23:10.709 INFO:teuthology.orchestra.run.smithi143.stdout: { 2023-11-26T07:23:10.709 INFO:teuthology.orchestra.run.smithi143.stdout: "name": "orchestrator", 2023-11-26T07:23:10.709 INFO:teuthology.orchestra.run.smithi143.stdout: "can_run": true, 2023-11-26T07:23:10.709 INFO:teuthology.orchestra.run.smithi143.stdout: "error_string": "", 2023-11-26T07:23:10.709 INFO:teuthology.orchestra.run.smithi143.stdout: "module_options": { 2023-11-26T07:23:10.709 INFO:teuthology.orchestra.run.smithi143.stdout: "fail_fs": { 2023-11-26T07:23:10.709 INFO:teuthology.orchestra.run.smithi143.stdout: "name": "fail_fs", 2023-11-26T07:23:10.709 INFO:teuthology.orchestra.run.smithi143.stdout: "type": "bool", 2023-11-26T07:23:10.709 INFO:teuthology.orchestra.run.smithi143.stdout: "level": "advanced", 2023-11-26T07:23:10.710 INFO:teuthology.orchestra.run.smithi143.stdout: "flags": 0, 2023-11-26T07:23:10.710 INFO:teuthology.orchestra.run.smithi143.stdout: "default_value": "False", 2023-11-26T07:23:10.710 INFO:teuthology.orchestra.run.smithi143.stdout: "min": "", 2023-11-26T07:23:10.710 INFO:teuthology.orchestra.run.smithi143.stdout: "max": "", 2023-11-26T07:23:10.710 INFO:teuthology.orchestra.run.smithi143.stdout: "enum_allowed": [], 2023-11-26T07:23:10.710 INFO:teuthology.orchestra.run.smithi143.stdout: "desc": "Fail filesystem for rapid multi-rank mds upgrade", 2023-11-26T07:23:10.710 INFO:teuthology.orchestra.run.smithi143.stdout: "long_desc": "", 2023-11-26T07:23:10.710 INFO:teuthology.orchestra.run.smithi143.stdout: "tags": [], 2023-11-26T07:23:10.710 INFO:teuthology.orchestra.run.smithi143.stdout: "see_also": [] 2023-11-26T07:23:10.710 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2023-11-26T07:23:10.710 INFO:teuthology.orchestra.run.smithi143.stdout: "log_level": { 2023-11-26T07:23:10.711 INFO:teuthology.orchestra.run.smithi143.stdout: "name": "log_level", 2023-11-26T07:23:10.711 INFO:teuthology.orchestra.run.smithi143.stdout: "type": "str", 2023-11-26T07:23:10.711 INFO:teuthology.orchestra.run.smithi143.stdout: "level": "advanced", 2023-11-26T07:23:10.711 INFO:teuthology.orchestra.run.smithi143.stdout: "flags": 1, 2023-11-26T07:23:10.711 INFO:teuthology.orchestra.run.smithi143.stdout: "default_value": "", 2023-11-26T07:23:10.711 INFO:teuthology.orchestra.run.smithi143.stdout: "min": "", 2023-11-26T07:23:10.711 INFO:teuthology.orchestra.run.smithi143.stdout: "max": "", 2023-11-26T07:23:10.711 INFO:teuthology.orchestra.run.smithi143.stdout: "enum_allowed": [ 2023-11-26T07:23:10.711 INFO:teuthology.orchestra.run.smithi143.stdout: "", 2023-11-26T07:23:10.711 INFO:teuthology.orchestra.run.smithi143.stdout: "critical", 2023-11-26T07:23:10.711 INFO:teuthology.orchestra.run.smithi143.stdout: "debug", 2023-11-26T07:23:10.711 INFO:teuthology.orchestra.run.smithi143.stdout: "error", 2023-11-26T07:23:10.712 INFO:teuthology.orchestra.run.smithi143.stdout: "info", 2023-11-26T07:23:10.712 INFO:teuthology.orchestra.run.smithi143.stdout: "warning" 2023-11-26T07:23:10.712 INFO:teuthology.orchestra.run.smithi143.stdout: ], 2023-11-26T07:23:10.712 INFO:teuthology.orchestra.run.smithi143.stdout: "desc": "", 2023-11-26T07:23:10.712 INFO:teuthology.orchestra.run.smithi143.stdout: "long_desc": "", 2023-11-26T07:23:10.712 INFO:teuthology.orchestra.run.smithi143.stdout: "tags": [], 2023-11-26T07:23:10.712 INFO:teuthology.orchestra.run.smithi143.stdout: "see_also": [] 2023-11-26T07:23:10.712 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2023-11-26T07:23:10.712 INFO:teuthology.orchestra.run.smithi143.stdout: "log_to_cluster": { 2023-11-26T07:23:10.712 INFO:teuthology.orchestra.run.smithi143.stdout: "name": "log_to_cluster", 2023-11-26T07:23:10.712 INFO:teuthology.orchestra.run.smithi143.stdout: "type": "bool", 2023-11-26T07:23:10.712 INFO:teuthology.orchestra.run.smithi143.stdout: "level": "advanced", 2023-11-26T07:23:10.712 INFO:teuthology.orchestra.run.smithi143.stdout: "flags": 1, 2023-11-26T07:23:10.713 INFO:teuthology.orchestra.run.smithi143.stdout: "default_value": "False", 2023-11-26T07:23:10.713 INFO:teuthology.orchestra.run.smithi143.stdout: "min": "", 2023-11-26T07:23:10.713 INFO:teuthology.orchestra.run.smithi143.stdout: "max": "", 2023-11-26T07:23:10.713 INFO:teuthology.orchestra.run.smithi143.stdout: "enum_allowed": [], 2023-11-26T07:23:10.713 INFO:teuthology.orchestra.run.smithi143.stdout: "desc": "", 2023-11-26T07:23:10.713 INFO:teuthology.orchestra.run.smithi143.stdout: "long_desc": "", 2023-11-26T07:23:10.713 INFO:teuthology.orchestra.run.smithi143.stdout: "tags": [], 2023-11-26T07:23:10.713 INFO:teuthology.orchestra.run.smithi143.stdout: "see_also": [] 2023-11-26T07:23:10.713 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2023-11-26T07:23:10.713 INFO:teuthology.orchestra.run.smithi143.stdout: "log_to_cluster_level": { 2023-11-26T07:23:10.713 INFO:teuthology.orchestra.run.smithi143.stdout: "name": "log_to_cluster_level", 2023-11-26T07:23:10.713 INFO:teuthology.orchestra.run.smithi143.stdout: "type": "str", 2023-11-26T07:23:10.714 INFO:teuthology.orchestra.run.smithi143.stdout: "level": "advanced", 2023-11-26T07:23:10.714 INFO:teuthology.orchestra.run.smithi143.stdout: "flags": 1, 2023-11-26T07:23:10.714 INFO:teuthology.orchestra.run.smithi143.stdout: "default_value": "info", 2023-11-26T07:23:10.714 INFO:teuthology.orchestra.run.smithi143.stdout: "min": "", 2023-11-26T07:23:10.714 INFO:teuthology.orchestra.run.smithi143.stdout: "max": "", 2023-11-26T07:23:10.714 INFO:teuthology.orchestra.run.smithi143.stdout: "enum_allowed": [ 2023-11-26T07:23:10.714 INFO:teuthology.orchestra.run.smithi143.stdout: "", 2023-11-26T07:23:10.714 INFO:teuthology.orchestra.run.smithi143.stdout: "critical", 2023-11-26T07:23:10.714 INFO:teuthology.orchestra.run.smithi143.stdout: "debug", 2023-11-26T07:23:10.714 INFO:teuthology.orchestra.run.smithi143.stdout: "error", 2023-11-26T07:23:10.714 INFO:teuthology.orchestra.run.smithi143.stdout: "info", 2023-11-26T07:23:10.714 INFO:teuthology.orchestra.run.smithi143.stdout: "warning" 2023-11-26T07:23:10.715 INFO:teuthology.orchestra.run.smithi143.stdout: ], 2023-11-26T07:23:10.715 INFO:teuthology.orchestra.run.smithi143.stdout: "desc": "", 2023-11-26T07:23:10.715 INFO:teuthology.orchestra.run.smithi143.stdout: "long_desc": "", 2023-11-26T07:23:10.715 INFO:teuthology.orchestra.run.smithi143.stdout: "tags": [], 2023-11-26T07:23:10.715 INFO:teuthology.orchestra.run.smithi143.stdout: "see_also": [] 2023-11-26T07:23:10.715 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2023-11-26T07:23:10.715 INFO:teuthology.orchestra.run.smithi143.stdout: "log_to_file": { 2023-11-26T07:23:10.715 INFO:teuthology.orchestra.run.smithi143.stdout: "name": "log_to_file", 2023-11-26T07:23:10.715 INFO:teuthology.orchestra.run.smithi143.stdout: "type": "bool", 2023-11-26T07:23:10.715 INFO:teuthology.orchestra.run.smithi143.stdout: "level": "advanced", 2023-11-26T07:23:10.715 INFO:teuthology.orchestra.run.smithi143.stdout: "flags": 1, 2023-11-26T07:23:10.715 INFO:teuthology.orchestra.run.smithi143.stdout: "default_value": "False", 2023-11-26T07:23:10.716 INFO:teuthology.orchestra.run.smithi143.stdout: "min": "", 2023-11-26T07:23:10.716 INFO:teuthology.orchestra.run.smithi143.stdout: "max": "", 2023-11-26T07:23:10.716 INFO:teuthology.orchestra.run.smithi143.stdout: "enum_allowed": [], 2023-11-26T07:23:10.716 INFO:teuthology.orchestra.run.smithi143.stdout: "desc": "", 2023-11-26T07:23:10.716 INFO:teuthology.orchestra.run.smithi143.stdout: "long_desc": "", 2023-11-26T07:23:10.716 INFO:teuthology.orchestra.run.smithi143.stdout: "tags": [], 2023-11-26T07:23:10.716 INFO:teuthology.orchestra.run.smithi143.stdout: "see_also": [] 2023-11-26T07:23:10.716 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2023-11-26T07:23:10.716 INFO:teuthology.orchestra.run.smithi143.stdout: "orchestrator": { 2023-11-26T07:23:10.716 INFO:teuthology.orchestra.run.smithi143.stdout: "name": "orchestrator", 2023-11-26T07:23:10.716 INFO:teuthology.orchestra.run.smithi143.stdout: "type": "str", 2023-11-26T07:23:10.716 INFO:teuthology.orchestra.run.smithi143.stdout: "level": "advanced", 2023-11-26T07:23:10.716 INFO:teuthology.orchestra.run.smithi143.stdout: "flags": 1, 2023-11-26T07:23:10.717 INFO:teuthology.orchestra.run.smithi143.stdout: "default_value": "", 2023-11-26T07:23:10.717 INFO:teuthology.orchestra.run.smithi143.stdout: "min": "", 2023-11-26T07:23:10.717 INFO:teuthology.orchestra.run.smithi143.stdout: "max": "", 2023-11-26T07:23:10.717 INFO:teuthology.orchestra.run.smithi143.stdout: "enum_allowed": [ 2023-11-26T07:23:10.717 INFO:teuthology.orchestra.run.smithi143.stdout: "cephadm", 2023-11-26T07:23:10.717 INFO:teuthology.orchestra.run.smithi143.stdout: "rook", 2023-11-26T07:23:10.717 INFO:teuthology.orchestra.run.smithi143.stdout: "test_orchestrator" 2023-11-26T07:23:10.717 INFO:teuthology.orchestra.run.smithi143.stdout: ], 2023-11-26T07:23:10.717 INFO:teuthology.orchestra.run.smithi143.stdout: "desc": "Orchestrator backend", 2023-11-26T07:23:10.717 INFO:teuthology.orchestra.run.smithi143.stdout: "long_desc": "", 2023-11-26T07:23:10.717 INFO:teuthology.orchestra.run.smithi143.stdout: "tags": [], 2023-11-26T07:23:10.717 INFO:teuthology.orchestra.run.smithi143.stdout: "see_also": [] 2023-11-26T07:23:10.718 INFO:teuthology.orchestra.run.smithi143.stdout: } 2023-11-26T07:23:10.718 INFO:teuthology.orchestra.run.smithi143.stdout: } 2023-11-26T07:23:10.718 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2023-11-26T07:23:10.718 INFO:teuthology.orchestra.run.smithi143.stdout: { 2023-11-26T07:23:10.718 INFO:teuthology.orchestra.run.smithi143.stdout: "name": "osd_perf_query", 2023-11-26T07:23:10.718 INFO:teuthology.orchestra.run.smithi143.stdout: "can_run": true, 2023-11-26T07:23:10.718 INFO:teuthology.orchestra.run.smithi143.stdout: "error_string": "", 2023-11-26T07:23:10.718 INFO:teuthology.orchestra.run.smithi143.stdout: "module_options": { 2023-11-26T07:23:10.718 INFO:teuthology.orchestra.run.smithi143.stdout: "log_level": { 2023-11-26T07:23:10.718 INFO:teuthology.orchestra.run.smithi143.stdout: "name": "log_level", 2023-11-26T07:23:10.718 INFO:teuthology.orchestra.run.smithi143.stdout: "type": "str", 2023-11-26T07:23:10.719 INFO:teuthology.orchestra.run.smithi143.stdout: "level": "advanced", 2023-11-26T07:23:10.719 INFO:teuthology.orchestra.run.smithi143.stdout: "flags": 1, 2023-11-26T07:23:10.719 INFO:teuthology.orchestra.run.smithi143.stdout: "default_value": "", 2023-11-26T07:23:10.719 INFO:teuthology.orchestra.run.smithi143.stdout: "min": "", 2023-11-26T07:23:10.719 INFO:teuthology.orchestra.run.smithi143.stdout: "max": "", 2023-11-26T07:23:10.719 INFO:teuthology.orchestra.run.smithi143.stdout: "enum_allowed": [ 2023-11-26T07:23:10.719 INFO:teuthology.orchestra.run.smithi143.stdout: "", 2023-11-26T07:23:10.719 INFO:teuthology.orchestra.run.smithi143.stdout: "critical", 2023-11-26T07:23:10.719 INFO:teuthology.orchestra.run.smithi143.stdout: "debug", 2023-11-26T07:23:10.719 INFO:teuthology.orchestra.run.smithi143.stdout: "error", 2023-11-26T07:23:10.719 INFO:teuthology.orchestra.run.smithi143.stdout: "info", 2023-11-26T07:23:10.719 INFO:teuthology.orchestra.run.smithi143.stdout: "warning" 2023-11-26T07:23:10.719 INFO:teuthology.orchestra.run.smithi143.stdout: ], 2023-11-26T07:23:10.720 INFO:teuthology.orchestra.run.smithi143.stdout: "desc": "", 2023-11-26T07:23:10.720 INFO:teuthology.orchestra.run.smithi143.stdout: "long_desc": "", 2023-11-26T07:23:10.720 INFO:teuthology.orchestra.run.smithi143.stdout: "tags": [], 2023-11-26T07:23:10.720 INFO:teuthology.orchestra.run.smithi143.stdout: "see_also": [] 2023-11-26T07:23:10.720 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2023-11-26T07:23:10.720 INFO:teuthology.orchestra.run.smithi143.stdout: "log_to_cluster": { 2023-11-26T07:23:10.720 INFO:teuthology.orchestra.run.smithi143.stdout: "name": "log_to_cluster", 2023-11-26T07:23:10.720 INFO:teuthology.orchestra.run.smithi143.stdout: "type": "bool", 2023-11-26T07:23:10.720 INFO:teuthology.orchestra.run.smithi143.stdout: "level": "advanced", 2023-11-26T07:23:10.720 INFO:teuthology.orchestra.run.smithi143.stdout: "flags": 1, 2023-11-26T07:23:10.720 INFO:teuthology.orchestra.run.smithi143.stdout: "default_value": "False", 2023-11-26T07:23:10.720 INFO:teuthology.orchestra.run.smithi143.stdout: "min": "", 2023-11-26T07:23:10.721 INFO:teuthology.orchestra.run.smithi143.stdout: "max": "", 2023-11-26T07:23:10.721 INFO:teuthology.orchestra.run.smithi143.stdout: "enum_allowed": [], 2023-11-26T07:23:10.721 INFO:teuthology.orchestra.run.smithi143.stdout: "desc": "", 2023-11-26T07:23:10.721 INFO:teuthology.orchestra.run.smithi143.stdout: "long_desc": "", 2023-11-26T07:23:10.721 INFO:teuthology.orchestra.run.smithi143.stdout: "tags": [], 2023-11-26T07:23:10.721 INFO:teuthology.orchestra.run.smithi143.stdout: "see_also": [] 2023-11-26T07:23:10.721 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2023-11-26T07:23:10.721 INFO:teuthology.orchestra.run.smithi143.stdout: "log_to_cluster_level": { 2023-11-26T07:23:10.721 INFO:teuthology.orchestra.run.smithi143.stdout: "name": "log_to_cluster_level", 2023-11-26T07:23:10.721 INFO:teuthology.orchestra.run.smithi143.stdout: "type": "str", 2023-11-26T07:23:10.721 INFO:teuthology.orchestra.run.smithi143.stdout: "level": "advanced", 2023-11-26T07:23:10.721 INFO:teuthology.orchestra.run.smithi143.stdout: "flags": 1, 2023-11-26T07:23:10.722 INFO:teuthology.orchestra.run.smithi143.stdout: "default_value": "info", 2023-11-26T07:23:10.722 INFO:teuthology.orchestra.run.smithi143.stdout: "min": "", 2023-11-26T07:23:10.722 INFO:teuthology.orchestra.run.smithi143.stdout: "max": "", 2023-11-26T07:23:10.722 INFO:teuthology.orchestra.run.smithi143.stdout: "enum_allowed": [ 2023-11-26T07:23:10.722 INFO:teuthology.orchestra.run.smithi143.stdout: "", 2023-11-26T07:23:10.722 INFO:teuthology.orchestra.run.smithi143.stdout: "critical", 2023-11-26T07:23:10.722 INFO:teuthology.orchestra.run.smithi143.stdout: "debug", 2023-11-26T07:23:10.722 INFO:teuthology.orchestra.run.smithi143.stdout: "error", 2023-11-26T07:23:10.722 INFO:teuthology.orchestra.run.smithi143.stdout: "info", 2023-11-26T07:23:10.722 INFO:teuthology.orchestra.run.smithi143.stdout: "warning" 2023-11-26T07:23:10.722 INFO:teuthology.orchestra.run.smithi143.stdout: ], 2023-11-26T07:23:10.722 INFO:teuthology.orchestra.run.smithi143.stdout: "desc": "", 2023-11-26T07:23:10.722 INFO:teuthology.orchestra.run.smithi143.stdout: "long_desc": "", 2023-11-26T07:23:10.723 INFO:teuthology.orchestra.run.smithi143.stdout: "tags": [], 2023-11-26T07:23:10.723 INFO:teuthology.orchestra.run.smithi143.stdout: "see_also": [] 2023-11-26T07:23:10.723 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2023-11-26T07:23:10.723 INFO:teuthology.orchestra.run.smithi143.stdout: "log_to_file": { 2023-11-26T07:23:10.723 INFO:teuthology.orchestra.run.smithi143.stdout: "name": "log_to_file", 2023-11-26T07:23:10.723 INFO:teuthology.orchestra.run.smithi143.stdout: "type": "bool", 2023-11-26T07:23:10.723 INFO:teuthology.orchestra.run.smithi143.stdout: "level": "advanced", 2023-11-26T07:23:10.723 INFO:teuthology.orchestra.run.smithi143.stdout: "flags": 1, 2023-11-26T07:23:10.723 INFO:teuthology.orchestra.run.smithi143.stdout: "default_value": "False", 2023-11-26T07:23:10.723 INFO:teuthology.orchestra.run.smithi143.stdout: "min": "", 2023-11-26T07:23:10.723 INFO:teuthology.orchestra.run.smithi143.stdout: "max": "", 2023-11-26T07:23:10.723 INFO:teuthology.orchestra.run.smithi143.stdout: "enum_allowed": [], 2023-11-26T07:23:10.723 INFO:teuthology.orchestra.run.smithi143.stdout: "desc": "", 2023-11-26T07:23:10.724 INFO:teuthology.orchestra.run.smithi143.stdout: "long_desc": "", 2023-11-26T07:23:10.724 INFO:teuthology.orchestra.run.smithi143.stdout: "tags": [], 2023-11-26T07:23:10.724 INFO:teuthology.orchestra.run.smithi143.stdout: "see_also": [] 2023-11-26T07:23:10.724 INFO:teuthology.orchestra.run.smithi143.stdout: } 2023-11-26T07:23:10.724 INFO:teuthology.orchestra.run.smithi143.stdout: } 2023-11-26T07:23:10.724 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2023-11-26T07:23:10.724 INFO:teuthology.orchestra.run.smithi143.stdout: { 2023-11-26T07:23:10.724 INFO:teuthology.orchestra.run.smithi143.stdout: "name": "osd_support", 2023-11-26T07:23:10.724 INFO:teuthology.orchestra.run.smithi143.stdout: "can_run": true, 2023-11-26T07:23:10.724 INFO:teuthology.orchestra.run.smithi143.stdout: "error_string": "", 2023-11-26T07:23:10.724 INFO:teuthology.orchestra.run.smithi143.stdout: "module_options": { 2023-11-26T07:23:10.724 INFO:teuthology.orchestra.run.smithi143.stdout: "log_level": { 2023-11-26T07:23:10.724 INFO:teuthology.orchestra.run.smithi143.stdout: "name": "log_level", 2023-11-26T07:23:10.725 INFO:teuthology.orchestra.run.smithi143.stdout: "type": "str", 2023-11-26T07:23:10.725 INFO:teuthology.orchestra.run.smithi143.stdout: "level": "advanced", 2023-11-26T07:23:10.725 INFO:teuthology.orchestra.run.smithi143.stdout: "flags": 1, 2023-11-26T07:23:10.725 INFO:teuthology.orchestra.run.smithi143.stdout: "default_value": "", 2023-11-26T07:23:10.725 INFO:teuthology.orchestra.run.smithi143.stdout: "min": "", 2023-11-26T07:23:10.725 INFO:teuthology.orchestra.run.smithi143.stdout: "max": "", 2023-11-26T07:23:10.725 INFO:teuthology.orchestra.run.smithi143.stdout: "enum_allowed": [ 2023-11-26T07:23:10.725 INFO:teuthology.orchestra.run.smithi143.stdout: "", 2023-11-26T07:23:10.725 INFO:teuthology.orchestra.run.smithi143.stdout: "critical", 2023-11-26T07:23:10.725 INFO:teuthology.orchestra.run.smithi143.stdout: "debug", 2023-11-26T07:23:10.725 INFO:teuthology.orchestra.run.smithi143.stdout: "error", 2023-11-26T07:23:10.725 INFO:teuthology.orchestra.run.smithi143.stdout: "info", 2023-11-26T07:23:10.725 INFO:teuthology.orchestra.run.smithi143.stdout: "warning" 2023-11-26T07:23:10.726 INFO:teuthology.orchestra.run.smithi143.stdout: ], 2023-11-26T07:23:10.726 INFO:teuthology.orchestra.run.smithi143.stdout: "desc": "", 2023-11-26T07:23:10.726 INFO:teuthology.orchestra.run.smithi143.stdout: "long_desc": "", 2023-11-26T07:23:10.726 INFO:teuthology.orchestra.run.smithi143.stdout: "tags": [], 2023-11-26T07:23:10.726 INFO:teuthology.orchestra.run.smithi143.stdout: "see_also": [] 2023-11-26T07:23:10.726 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2023-11-26T07:23:10.726 INFO:teuthology.orchestra.run.smithi143.stdout: "log_to_cluster": { 2023-11-26T07:23:10.726 INFO:teuthology.orchestra.run.smithi143.stdout: "name": "log_to_cluster", 2023-11-26T07:23:10.726 INFO:teuthology.orchestra.run.smithi143.stdout: "type": "bool", 2023-11-26T07:23:10.726 INFO:teuthology.orchestra.run.smithi143.stdout: "level": "advanced", 2023-11-26T07:23:10.726 INFO:teuthology.orchestra.run.smithi143.stdout: "flags": 1, 2023-11-26T07:23:10.726 INFO:teuthology.orchestra.run.smithi143.stdout: "default_value": "False", 2023-11-26T07:23:10.727 INFO:teuthology.orchestra.run.smithi143.stdout: "min": "", 2023-11-26T07:23:10.727 INFO:teuthology.orchestra.run.smithi143.stdout: "max": "", 2023-11-26T07:23:10.727 INFO:teuthology.orchestra.run.smithi143.stdout: "enum_allowed": [], 2023-11-26T07:23:10.727 INFO:teuthology.orchestra.run.smithi143.stdout: "desc": "", 2023-11-26T07:23:10.727 INFO:teuthology.orchestra.run.smithi143.stdout: "long_desc": "", 2023-11-26T07:23:10.727 INFO:teuthology.orchestra.run.smithi143.stdout: "tags": [], 2023-11-26T07:23:10.727 INFO:teuthology.orchestra.run.smithi143.stdout: "see_also": [] 2023-11-26T07:23:10.727 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2023-11-26T07:23:10.727 INFO:teuthology.orchestra.run.smithi143.stdout: "log_to_cluster_level": { 2023-11-26T07:23:10.727 INFO:teuthology.orchestra.run.smithi143.stdout: "name": "log_to_cluster_level", 2023-11-26T07:23:10.727 INFO:teuthology.orchestra.run.smithi143.stdout: "type": "str", 2023-11-26T07:23:10.727 INFO:teuthology.orchestra.run.smithi143.stdout: "level": "advanced", 2023-11-26T07:23:10.727 INFO:teuthology.orchestra.run.smithi143.stdout: "flags": 1, 2023-11-26T07:23:10.728 INFO:teuthology.orchestra.run.smithi143.stdout: "default_value": "info", 2023-11-26T07:23:10.728 INFO:teuthology.orchestra.run.smithi143.stdout: "min": "", 2023-11-26T07:23:10.728 INFO:teuthology.orchestra.run.smithi143.stdout: "max": "", 2023-11-26T07:23:10.728 INFO:teuthology.orchestra.run.smithi143.stdout: "enum_allowed": [ 2023-11-26T07:23:10.728 INFO:teuthology.orchestra.run.smithi143.stdout: "", 2023-11-26T07:23:10.728 INFO:teuthology.orchestra.run.smithi143.stdout: "critical", 2023-11-26T07:23:10.728 INFO:teuthology.orchestra.run.smithi143.stdout: "debug", 2023-11-26T07:23:10.728 INFO:teuthology.orchestra.run.smithi143.stdout: "error", 2023-11-26T07:23:10.728 INFO:teuthology.orchestra.run.smithi143.stdout: "info", 2023-11-26T07:23:10.728 INFO:teuthology.orchestra.run.smithi143.stdout: "warning" 2023-11-26T07:23:10.728 INFO:teuthology.orchestra.run.smithi143.stdout: ], 2023-11-26T07:23:10.728 INFO:teuthology.orchestra.run.smithi143.stdout: "desc": "", 2023-11-26T07:23:10.728 INFO:teuthology.orchestra.run.smithi143.stdout: "long_desc": "", 2023-11-26T07:23:10.729 INFO:teuthology.orchestra.run.smithi143.stdout: "tags": [], 2023-11-26T07:23:10.729 INFO:teuthology.orchestra.run.smithi143.stdout: "see_also": [] 2023-11-26T07:23:10.729 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2023-11-26T07:23:10.729 INFO:teuthology.orchestra.run.smithi143.stdout: "log_to_file": { 2023-11-26T07:23:10.729 INFO:teuthology.orchestra.run.smithi143.stdout: "name": "log_to_file", 2023-11-26T07:23:10.729 INFO:teuthology.orchestra.run.smithi143.stdout: "type": "bool", 2023-11-26T07:23:10.729 INFO:teuthology.orchestra.run.smithi143.stdout: "level": "advanced", 2023-11-26T07:23:10.729 INFO:teuthology.orchestra.run.smithi143.stdout: "flags": 1, 2023-11-26T07:23:10.729 INFO:teuthology.orchestra.run.smithi143.stdout: "default_value": "False", 2023-11-26T07:23:10.729 INFO:teuthology.orchestra.run.smithi143.stdout: "min": "", 2023-11-26T07:23:10.729 INFO:teuthology.orchestra.run.smithi143.stdout: "max": "", 2023-11-26T07:23:10.729 INFO:teuthology.orchestra.run.smithi143.stdout: "enum_allowed": [], 2023-11-26T07:23:10.729 INFO:teuthology.orchestra.run.smithi143.stdout: "desc": "", 2023-11-26T07:23:10.730 INFO:teuthology.orchestra.run.smithi143.stdout: "long_desc": "", 2023-11-26T07:23:10.730 INFO:teuthology.orchestra.run.smithi143.stdout: "tags": [], 2023-11-26T07:23:10.730 INFO:teuthology.orchestra.run.smithi143.stdout: "see_also": [] 2023-11-26T07:23:10.730 INFO:teuthology.orchestra.run.smithi143.stdout: } 2023-11-26T07:23:10.730 INFO:teuthology.orchestra.run.smithi143.stdout: } 2023-11-26T07:23:10.730 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2023-11-26T07:23:10.730 INFO:teuthology.orchestra.run.smithi143.stdout: { 2023-11-26T07:23:10.730 INFO:teuthology.orchestra.run.smithi143.stdout: "name": "pg_autoscaler", 2023-11-26T07:23:10.730 INFO:teuthology.orchestra.run.smithi143.stdout: "can_run": true, 2023-11-26T07:23:10.730 INFO:teuthology.orchestra.run.smithi143.stdout: "error_string": "", 2023-11-26T07:23:10.730 INFO:teuthology.orchestra.run.smithi143.stdout: "module_options": { 2023-11-26T07:23:10.730 INFO:teuthology.orchestra.run.smithi143.stdout: "log_level": { 2023-11-26T07:23:10.731 INFO:teuthology.orchestra.run.smithi143.stdout: "name": "log_level", 2023-11-26T07:23:10.731 INFO:teuthology.orchestra.run.smithi143.stdout: "type": "str", 2023-11-26T07:23:10.731 INFO:teuthology.orchestra.run.smithi143.stdout: "level": "advanced", 2023-11-26T07:23:10.731 INFO:teuthology.orchestra.run.smithi143.stdout: "flags": 1, 2023-11-26T07:23:10.731 INFO:teuthology.orchestra.run.smithi143.stdout: "default_value": "", 2023-11-26T07:23:10.731 INFO:teuthology.orchestra.run.smithi143.stdout: "min": "", 2023-11-26T07:23:10.731 INFO:teuthology.orchestra.run.smithi143.stdout: "max": "", 2023-11-26T07:23:10.731 INFO:teuthology.orchestra.run.smithi143.stdout: "enum_allowed": [ 2023-11-26T07:23:10.731 INFO:teuthology.orchestra.run.smithi143.stdout: "", 2023-11-26T07:23:10.731 INFO:teuthology.orchestra.run.smithi143.stdout: "critical", 2023-11-26T07:23:10.731 INFO:teuthology.orchestra.run.smithi143.stdout: "debug", 2023-11-26T07:23:10.731 INFO:teuthology.orchestra.run.smithi143.stdout: "error", 2023-11-26T07:23:10.731 INFO:teuthology.orchestra.run.smithi143.stdout: "info", 2023-11-26T07:23:10.732 INFO:teuthology.orchestra.run.smithi143.stdout: "warning" 2023-11-26T07:23:10.732 INFO:teuthology.orchestra.run.smithi143.stdout: ], 2023-11-26T07:23:10.732 INFO:teuthology.orchestra.run.smithi143.stdout: "desc": "", 2023-11-26T07:23:10.732 INFO:teuthology.orchestra.run.smithi143.stdout: "long_desc": "", 2023-11-26T07:23:10.732 INFO:teuthology.orchestra.run.smithi143.stdout: "tags": [], 2023-11-26T07:23:10.732 INFO:teuthology.orchestra.run.smithi143.stdout: "see_also": [] 2023-11-26T07:23:10.732 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2023-11-26T07:23:10.732 INFO:teuthology.orchestra.run.smithi143.stdout: "log_to_cluster": { 2023-11-26T07:23:10.732 INFO:teuthology.orchestra.run.smithi143.stdout: "name": "log_to_cluster", 2023-11-26T07:23:10.732 INFO:teuthology.orchestra.run.smithi143.stdout: "type": "bool", 2023-11-26T07:23:10.732 INFO:teuthology.orchestra.run.smithi143.stdout: "level": "advanced", 2023-11-26T07:23:10.732 INFO:teuthology.orchestra.run.smithi143.stdout: "flags": 1, 2023-11-26T07:23:10.732 INFO:teuthology.orchestra.run.smithi143.stdout: "default_value": "False", 2023-11-26T07:23:10.733 INFO:teuthology.orchestra.run.smithi143.stdout: "min": "", 2023-11-26T07:23:10.733 INFO:teuthology.orchestra.run.smithi143.stdout: "max": "", 2023-11-26T07:23:10.733 INFO:teuthology.orchestra.run.smithi143.stdout: "enum_allowed": [], 2023-11-26T07:23:10.733 INFO:teuthology.orchestra.run.smithi143.stdout: "desc": "", 2023-11-26T07:23:10.733 INFO:teuthology.orchestra.run.smithi143.stdout: "long_desc": "", 2023-11-26T07:23:10.733 INFO:teuthology.orchestra.run.smithi143.stdout: "tags": [], 2023-11-26T07:23:10.733 INFO:teuthology.orchestra.run.smithi143.stdout: "see_also": [] 2023-11-26T07:23:10.733 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2023-11-26T07:23:10.733 INFO:teuthology.orchestra.run.smithi143.stdout: "log_to_cluster_level": { 2023-11-26T07:23:10.733 INFO:teuthology.orchestra.run.smithi143.stdout: "name": "log_to_cluster_level", 2023-11-26T07:23:10.733 INFO:teuthology.orchestra.run.smithi143.stdout: "type": "str", 2023-11-26T07:23:10.733 INFO:teuthology.orchestra.run.smithi143.stdout: "level": "advanced", 2023-11-26T07:23:10.734 INFO:teuthology.orchestra.run.smithi143.stdout: "flags": 1, 2023-11-26T07:23:10.734 INFO:teuthology.orchestra.run.smithi143.stdout: "default_value": "info", 2023-11-26T07:23:10.734 INFO:teuthology.orchestra.run.smithi143.stdout: "min": "", 2023-11-26T07:23:10.734 INFO:teuthology.orchestra.run.smithi143.stdout: "max": "", 2023-11-26T07:23:10.734 INFO:teuthology.orchestra.run.smithi143.stdout: "enum_allowed": [ 2023-11-26T07:23:10.734 INFO:teuthology.orchestra.run.smithi143.stdout: "", 2023-11-26T07:23:10.734 INFO:teuthology.orchestra.run.smithi143.stdout: "critical", 2023-11-26T07:23:10.734 INFO:teuthology.orchestra.run.smithi143.stdout: "debug", 2023-11-26T07:23:10.734 INFO:teuthology.orchestra.run.smithi143.stdout: "error", 2023-11-26T07:23:10.734 INFO:teuthology.orchestra.run.smithi143.stdout: "info", 2023-11-26T07:23:10.734 INFO:teuthology.orchestra.run.smithi143.stdout: "warning" 2023-11-26T07:23:10.734 INFO:teuthology.orchestra.run.smithi143.stdout: ], 2023-11-26T07:23:10.735 INFO:teuthology.orchestra.run.smithi143.stdout: "desc": "", 2023-11-26T07:23:10.735 INFO:teuthology.orchestra.run.smithi143.stdout: "long_desc": "", 2023-11-26T07:23:10.735 INFO:teuthology.orchestra.run.smithi143.stdout: "tags": [], 2023-11-26T07:23:10.735 INFO:teuthology.orchestra.run.smithi143.stdout: "see_also": [] 2023-11-26T07:23:10.735 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2023-11-26T07:23:10.735 INFO:teuthology.orchestra.run.smithi143.stdout: "log_to_file": { 2023-11-26T07:23:10.735 INFO:teuthology.orchestra.run.smithi143.stdout: "name": "log_to_file", 2023-11-26T07:23:10.735 INFO:teuthology.orchestra.run.smithi143.stdout: "type": "bool", 2023-11-26T07:23:10.735 INFO:teuthology.orchestra.run.smithi143.stdout: "level": "advanced", 2023-11-26T07:23:10.735 INFO:teuthology.orchestra.run.smithi143.stdout: "flags": 1, 2023-11-26T07:23:10.735 INFO:teuthology.orchestra.run.smithi143.stdout: "default_value": "False", 2023-11-26T07:23:10.735 INFO:teuthology.orchestra.run.smithi143.stdout: "min": "", 2023-11-26T07:23:10.735 INFO:teuthology.orchestra.run.smithi143.stdout: "max": "", 2023-11-26T07:23:10.736 INFO:teuthology.orchestra.run.smithi143.stdout: "enum_allowed": [], 2023-11-26T07:23:10.736 INFO:teuthology.orchestra.run.smithi143.stdout: "desc": "", 2023-11-26T07:23:10.736 INFO:teuthology.orchestra.run.smithi143.stdout: "long_desc": "", 2023-11-26T07:23:10.736 INFO:teuthology.orchestra.run.smithi143.stdout: "tags": [], 2023-11-26T07:23:10.736 INFO:teuthology.orchestra.run.smithi143.stdout: "see_also": [] 2023-11-26T07:23:10.736 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2023-11-26T07:23:10.736 INFO:teuthology.orchestra.run.smithi143.stdout: "sleep_interval": { 2023-11-26T07:23:10.736 INFO:teuthology.orchestra.run.smithi143.stdout: "name": "sleep_interval", 2023-11-26T07:23:10.736 INFO:teuthology.orchestra.run.smithi143.stdout: "type": "secs", 2023-11-26T07:23:10.736 INFO:teuthology.orchestra.run.smithi143.stdout: "level": "advanced", 2023-11-26T07:23:10.736 INFO:teuthology.orchestra.run.smithi143.stdout: "flags": 0, 2023-11-26T07:23:10.736 INFO:teuthology.orchestra.run.smithi143.stdout: "default_value": "60", 2023-11-26T07:23:10.737 INFO:teuthology.orchestra.run.smithi143.stdout: "min": "", 2023-11-26T07:23:10.737 INFO:teuthology.orchestra.run.smithi143.stdout: "max": "", 2023-11-26T07:23:10.737 INFO:teuthology.orchestra.run.smithi143.stdout: "enum_allowed": [], 2023-11-26T07:23:10.737 INFO:teuthology.orchestra.run.smithi143.stdout: "desc": "", 2023-11-26T07:23:10.737 INFO:teuthology.orchestra.run.smithi143.stdout: "long_desc": "", 2023-11-26T07:23:10.737 INFO:teuthology.orchestra.run.smithi143.stdout: "tags": [], 2023-11-26T07:23:10.737 INFO:teuthology.orchestra.run.smithi143.stdout: "see_also": [] 2023-11-26T07:23:10.737 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2023-11-26T07:23:10.737 INFO:teuthology.orchestra.run.smithi143.stdout: "threshold": { 2023-11-26T07:23:10.737 INFO:teuthology.orchestra.run.smithi143.stdout: "name": "threshold", 2023-11-26T07:23:10.737 INFO:teuthology.orchestra.run.smithi143.stdout: "type": "float", 2023-11-26T07:23:10.737 INFO:teuthology.orchestra.run.smithi143.stdout: "level": "advanced", 2023-11-26T07:23:10.737 INFO:teuthology.orchestra.run.smithi143.stdout: "flags": 0, 2023-11-26T07:23:10.738 INFO:teuthology.orchestra.run.smithi143.stdout: "default_value": "3.0", 2023-11-26T07:23:10.738 INFO:teuthology.orchestra.run.smithi143.stdout: "min": "1.0", 2023-11-26T07:23:10.738 INFO:teuthology.orchestra.run.smithi143.stdout: "max": "", 2023-11-26T07:23:10.738 INFO:teuthology.orchestra.run.smithi143.stdout: "enum_allowed": [], 2023-11-26T07:23:10.738 INFO:teuthology.orchestra.run.smithi143.stdout: "desc": "scaling threshold", 2023-11-26T07:23:10.738 INFO:teuthology.orchestra.run.smithi143.stdout: "long_desc": "The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0", 2023-11-26T07:23:10.738 INFO:teuthology.orchestra.run.smithi143.stdout: "tags": [], 2023-11-26T07:23:10.738 INFO:teuthology.orchestra.run.smithi143.stdout: "see_also": [] 2023-11-26T07:23:10.738 INFO:teuthology.orchestra.run.smithi143.stdout: } 2023-11-26T07:23:10.738 INFO:teuthology.orchestra.run.smithi143.stdout: } 2023-11-26T07:23:10.738 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2023-11-26T07:23:10.738 INFO:teuthology.orchestra.run.smithi143.stdout: { 2023-11-26T07:23:10.739 INFO:teuthology.orchestra.run.smithi143.stdout: "name": "progress", 2023-11-26T07:23:10.739 INFO:teuthology.orchestra.run.smithi143.stdout: "can_run": true, 2023-11-26T07:23:10.739 INFO:teuthology.orchestra.run.smithi143.stdout: "error_string": "", 2023-11-26T07:23:10.739 INFO:teuthology.orchestra.run.smithi143.stdout: "module_options": { 2023-11-26T07:23:10.739 INFO:teuthology.orchestra.run.smithi143.stdout: "allow_pg_recovery_event": { 2023-11-26T07:23:10.739 INFO:teuthology.orchestra.run.smithi143.stdout: "name": "allow_pg_recovery_event", 2023-11-26T07:23:10.739 INFO:teuthology.orchestra.run.smithi143.stdout: "type": "bool", 2023-11-26T07:23:10.739 INFO:teuthology.orchestra.run.smithi143.stdout: "level": "advanced", 2023-11-26T07:23:10.739 INFO:teuthology.orchestra.run.smithi143.stdout: "flags": 1, 2023-11-26T07:23:10.739 INFO:teuthology.orchestra.run.smithi143.stdout: "default_value": "False", 2023-11-26T07:23:10.739 INFO:teuthology.orchestra.run.smithi143.stdout: "min": "", 2023-11-26T07:23:10.739 INFO:teuthology.orchestra.run.smithi143.stdout: "max": "", 2023-11-26T07:23:10.739 INFO:teuthology.orchestra.run.smithi143.stdout: "enum_allowed": [], 2023-11-26T07:23:10.740 INFO:teuthology.orchestra.run.smithi143.stdout: "desc": "allow the module to show pg recovery progress", 2023-11-26T07:23:10.740 INFO:teuthology.orchestra.run.smithi143.stdout: "long_desc": "", 2023-11-26T07:23:10.740 INFO:teuthology.orchestra.run.smithi143.stdout: "tags": [], 2023-11-26T07:23:10.740 INFO:teuthology.orchestra.run.smithi143.stdout: "see_also": [] 2023-11-26T07:23:10.740 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2023-11-26T07:23:10.740 INFO:teuthology.orchestra.run.smithi143.stdout: "enabled": { 2023-11-26T07:23:10.740 INFO:teuthology.orchestra.run.smithi143.stdout: "name": "enabled", 2023-11-26T07:23:10.740 INFO:teuthology.orchestra.run.smithi143.stdout: "type": "bool", 2023-11-26T07:23:10.740 INFO:teuthology.orchestra.run.smithi143.stdout: "level": "advanced", 2023-11-26T07:23:10.740 INFO:teuthology.orchestra.run.smithi143.stdout: "flags": 0, 2023-11-26T07:23:10.740 INFO:teuthology.orchestra.run.smithi143.stdout: "default_value": "True", 2023-11-26T07:23:10.740 INFO:teuthology.orchestra.run.smithi143.stdout: "min": "", 2023-11-26T07:23:10.741 INFO:teuthology.orchestra.run.smithi143.stdout: "max": "", 2023-11-26T07:23:10.741 INFO:teuthology.orchestra.run.smithi143.stdout: "enum_allowed": [], 2023-11-26T07:23:10.741 INFO:teuthology.orchestra.run.smithi143.stdout: "desc": "", 2023-11-26T07:23:10.741 INFO:teuthology.orchestra.run.smithi143.stdout: "long_desc": "", 2023-11-26T07:23:10.741 INFO:teuthology.orchestra.run.smithi143.stdout: "tags": [], 2023-11-26T07:23:10.741 INFO:teuthology.orchestra.run.smithi143.stdout: "see_also": [] 2023-11-26T07:23:10.741 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2023-11-26T07:23:10.741 INFO:teuthology.orchestra.run.smithi143.stdout: "log_level": { 2023-11-26T07:23:10.741 INFO:teuthology.orchestra.run.smithi143.stdout: "name": "log_level", 2023-11-26T07:23:10.741 INFO:teuthology.orchestra.run.smithi143.stdout: "type": "str", 2023-11-26T07:23:10.741 INFO:teuthology.orchestra.run.smithi143.stdout: "level": "advanced", 2023-11-26T07:23:10.741 INFO:teuthology.orchestra.run.smithi143.stdout: "flags": 1, 2023-11-26T07:23:10.741 INFO:teuthology.orchestra.run.smithi143.stdout: "default_value": "", 2023-11-26T07:23:10.742 INFO:teuthology.orchestra.run.smithi143.stdout: "min": "", 2023-11-26T07:23:10.742 INFO:teuthology.orchestra.run.smithi143.stdout: "max": "", 2023-11-26T07:23:10.742 INFO:teuthology.orchestra.run.smithi143.stdout: "enum_allowed": [ 2023-11-26T07:23:10.742 INFO:teuthology.orchestra.run.smithi143.stdout: "", 2023-11-26T07:23:10.742 INFO:teuthology.orchestra.run.smithi143.stdout: "critical", 2023-11-26T07:23:10.742 INFO:teuthology.orchestra.run.smithi143.stdout: "debug", 2023-11-26T07:23:10.742 INFO:teuthology.orchestra.run.smithi143.stdout: "error", 2023-11-26T07:23:10.742 INFO:teuthology.orchestra.run.smithi143.stdout: "info", 2023-11-26T07:23:10.742 INFO:teuthology.orchestra.run.smithi143.stdout: "warning" 2023-11-26T07:23:10.742 INFO:teuthology.orchestra.run.smithi143.stdout: ], 2023-11-26T07:23:10.742 INFO:teuthology.orchestra.run.smithi143.stdout: "desc": "", 2023-11-26T07:23:10.742 INFO:teuthology.orchestra.run.smithi143.stdout: "long_desc": "", 2023-11-26T07:23:10.743 INFO:teuthology.orchestra.run.smithi143.stdout: "tags": [], 2023-11-26T07:23:10.743 INFO:teuthology.orchestra.run.smithi143.stdout: "see_also": [] 2023-11-26T07:23:10.743 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2023-11-26T07:23:10.743 INFO:teuthology.orchestra.run.smithi143.stdout: "log_to_cluster": { 2023-11-26T07:23:10.743 INFO:teuthology.orchestra.run.smithi143.stdout: "name": "log_to_cluster", 2023-11-26T07:23:10.743 INFO:teuthology.orchestra.run.smithi143.stdout: "type": "bool", 2023-11-26T07:23:10.743 INFO:teuthology.orchestra.run.smithi143.stdout: "level": "advanced", 2023-11-26T07:23:10.743 INFO:teuthology.orchestra.run.smithi143.stdout: "flags": 1, 2023-11-26T07:23:10.743 INFO:teuthology.orchestra.run.smithi143.stdout: "default_value": "False", 2023-11-26T07:23:10.743 INFO:teuthology.orchestra.run.smithi143.stdout: "min": "", 2023-11-26T07:23:10.743 INFO:teuthology.orchestra.run.smithi143.stdout: "max": "", 2023-11-26T07:23:10.743 INFO:teuthology.orchestra.run.smithi143.stdout: "enum_allowed": [], 2023-11-26T07:23:10.744 INFO:teuthology.orchestra.run.smithi143.stdout: "desc": "", 2023-11-26T07:23:10.744 INFO:teuthology.orchestra.run.smithi143.stdout: "long_desc": "", 2023-11-26T07:23:10.744 INFO:teuthology.orchestra.run.smithi143.stdout: "tags": [], 2023-11-26T07:23:10.744 INFO:teuthology.orchestra.run.smithi143.stdout: "see_also": [] 2023-11-26T07:23:10.744 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2023-11-26T07:23:10.744 INFO:teuthology.orchestra.run.smithi143.stdout: "log_to_cluster_level": { 2023-11-26T07:23:10.744 INFO:teuthology.orchestra.run.smithi143.stdout: "name": "log_to_cluster_level", 2023-11-26T07:23:10.744 INFO:teuthology.orchestra.run.smithi143.stdout: "type": "str", 2023-11-26T07:23:10.744 INFO:teuthology.orchestra.run.smithi143.stdout: "level": "advanced", 2023-11-26T07:23:10.744 INFO:teuthology.orchestra.run.smithi143.stdout: "flags": 1, 2023-11-26T07:23:10.744 INFO:teuthology.orchestra.run.smithi143.stdout: "default_value": "info", 2023-11-26T07:23:10.744 INFO:teuthology.orchestra.run.smithi143.stdout: "min": "", 2023-11-26T07:23:10.744 INFO:teuthology.orchestra.run.smithi143.stdout: "max": "", 2023-11-26T07:23:10.745 INFO:teuthology.orchestra.run.smithi143.stdout: "enum_allowed": [ 2023-11-26T07:23:10.745 INFO:teuthology.orchestra.run.smithi143.stdout: "", 2023-11-26T07:23:10.745 INFO:teuthology.orchestra.run.smithi143.stdout: "critical", 2023-11-26T07:23:10.745 INFO:teuthology.orchestra.run.smithi143.stdout: "debug", 2023-11-26T07:23:10.745 INFO:teuthology.orchestra.run.smithi143.stdout: "error", 2023-11-26T07:23:10.745 INFO:teuthology.orchestra.run.smithi143.stdout: "info", 2023-11-26T07:23:10.745 INFO:teuthology.orchestra.run.smithi143.stdout: "warning" 2023-11-26T07:23:10.745 INFO:teuthology.orchestra.run.smithi143.stdout: ], 2023-11-26T07:23:10.745 INFO:teuthology.orchestra.run.smithi143.stdout: "desc": "", 2023-11-26T07:23:10.745 INFO:teuthology.orchestra.run.smithi143.stdout: "long_desc": "", 2023-11-26T07:23:10.745 INFO:teuthology.orchestra.run.smithi143.stdout: "tags": [], 2023-11-26T07:23:10.745 INFO:teuthology.orchestra.run.smithi143.stdout: "see_also": [] 2023-11-26T07:23:10.745 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2023-11-26T07:23:10.746 INFO:teuthology.orchestra.run.smithi143.stdout: "log_to_file": { 2023-11-26T07:23:10.746 INFO:teuthology.orchestra.run.smithi143.stdout: "name": "log_to_file", 2023-11-26T07:23:10.746 INFO:teuthology.orchestra.run.smithi143.stdout: "type": "bool", 2023-11-26T07:23:10.746 INFO:teuthology.orchestra.run.smithi143.stdout: "level": "advanced", 2023-11-26T07:23:10.746 INFO:teuthology.orchestra.run.smithi143.stdout: "flags": 1, 2023-11-26T07:23:10.746 INFO:teuthology.orchestra.run.smithi143.stdout: "default_value": "False", 2023-11-26T07:23:10.746 INFO:teuthology.orchestra.run.smithi143.stdout: "min": "", 2023-11-26T07:23:10.746 INFO:teuthology.orchestra.run.smithi143.stdout: "max": "", 2023-11-26T07:23:10.746 INFO:teuthology.orchestra.run.smithi143.stdout: "enum_allowed": [], 2023-11-26T07:23:10.746 INFO:teuthology.orchestra.run.smithi143.stdout: "desc": "", 2023-11-26T07:23:10.746 INFO:teuthology.orchestra.run.smithi143.stdout: "long_desc": "", 2023-11-26T07:23:10.746 INFO:teuthology.orchestra.run.smithi143.stdout: "tags": [], 2023-11-26T07:23:10.747 INFO:teuthology.orchestra.run.smithi143.stdout: "see_also": [] 2023-11-26T07:23:10.747 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2023-11-26T07:23:10.747 INFO:teuthology.orchestra.run.smithi143.stdout: "max_completed_events": { 2023-11-26T07:23:10.747 INFO:teuthology.orchestra.run.smithi143.stdout: "name": "max_completed_events", 2023-11-26T07:23:10.747 INFO:teuthology.orchestra.run.smithi143.stdout: "type": "int", 2023-11-26T07:23:10.747 INFO:teuthology.orchestra.run.smithi143.stdout: "level": "advanced", 2023-11-26T07:23:10.747 INFO:teuthology.orchestra.run.smithi143.stdout: "flags": 1, 2023-11-26T07:23:10.747 INFO:teuthology.orchestra.run.smithi143.stdout: "default_value": "50", 2023-11-26T07:23:10.747 INFO:teuthology.orchestra.run.smithi143.stdout: "min": "", 2023-11-26T07:23:10.747 INFO:teuthology.orchestra.run.smithi143.stdout: "max": "", 2023-11-26T07:23:10.747 INFO:teuthology.orchestra.run.smithi143.stdout: "enum_allowed": [], 2023-11-26T07:23:10.747 INFO:teuthology.orchestra.run.smithi143.stdout: "desc": "number of past completed events to remember", 2023-11-26T07:23:10.748 INFO:teuthology.orchestra.run.smithi143.stdout: "long_desc": "", 2023-11-26T07:23:10.748 INFO:teuthology.orchestra.run.smithi143.stdout: "tags": [], 2023-11-26T07:23:10.748 INFO:teuthology.orchestra.run.smithi143.stdout: "see_also": [] 2023-11-26T07:23:10.748 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2023-11-26T07:23:10.748 INFO:teuthology.orchestra.run.smithi143.stdout: "sleep_interval": { 2023-11-26T07:23:10.748 INFO:teuthology.orchestra.run.smithi143.stdout: "name": "sleep_interval", 2023-11-26T07:23:10.748 INFO:teuthology.orchestra.run.smithi143.stdout: "type": "secs", 2023-11-26T07:23:10.748 INFO:teuthology.orchestra.run.smithi143.stdout: "level": "advanced", 2023-11-26T07:23:10.748 INFO:teuthology.orchestra.run.smithi143.stdout: "flags": 1, 2023-11-26T07:23:10.748 INFO:teuthology.orchestra.run.smithi143.stdout: "default_value": "5", 2023-11-26T07:23:10.748 INFO:teuthology.orchestra.run.smithi143.stdout: "min": "", 2023-11-26T07:23:10.748 INFO:teuthology.orchestra.run.smithi143.stdout: "max": "", 2023-11-26T07:23:10.749 INFO:teuthology.orchestra.run.smithi143.stdout: "enum_allowed": [], 2023-11-26T07:23:10.749 INFO:teuthology.orchestra.run.smithi143.stdout: "desc": "how long the module is going to sleep", 2023-11-26T07:23:10.749 INFO:teuthology.orchestra.run.smithi143.stdout: "long_desc": "", 2023-11-26T07:23:10.749 INFO:teuthology.orchestra.run.smithi143.stdout: "tags": [], 2023-11-26T07:23:10.749 INFO:teuthology.orchestra.run.smithi143.stdout: "see_also": [] 2023-11-26T07:23:10.749 INFO:teuthology.orchestra.run.smithi143.stdout: } 2023-11-26T07:23:10.749 INFO:teuthology.orchestra.run.smithi143.stdout: } 2023-11-26T07:23:10.749 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2023-11-26T07:23:10.749 INFO:teuthology.orchestra.run.smithi143.stdout: { 2023-11-26T07:23:10.749 INFO:teuthology.orchestra.run.smithi143.stdout: "name": "prometheus", 2023-11-26T07:23:10.749 INFO:teuthology.orchestra.run.smithi143.stdout: "can_run": true, 2023-11-26T07:23:10.749 INFO:teuthology.orchestra.run.smithi143.stdout: "error_string": "", 2023-11-26T07:23:10.750 INFO:teuthology.orchestra.run.smithi143.stdout: "module_options": { 2023-11-26T07:23:10.750 INFO:teuthology.orchestra.run.smithi143.stdout: "cache": { 2023-11-26T07:23:10.750 INFO:teuthology.orchestra.run.smithi143.stdout: "name": "cache", 2023-11-26T07:23:10.750 INFO:teuthology.orchestra.run.smithi143.stdout: "type": "bool", 2023-11-26T07:23:10.750 INFO:teuthology.orchestra.run.smithi143.stdout: "level": "advanced", 2023-11-26T07:23:10.750 INFO:teuthology.orchestra.run.smithi143.stdout: "flags": 0, 2023-11-26T07:23:10.750 INFO:teuthology.orchestra.run.smithi143.stdout: "default_value": "True", 2023-11-26T07:23:10.750 INFO:teuthology.orchestra.run.smithi143.stdout: "min": "", 2023-11-26T07:23:10.750 INFO:teuthology.orchestra.run.smithi143.stdout: "max": "", 2023-11-26T07:23:10.750 INFO:teuthology.orchestra.run.smithi143.stdout: "enum_allowed": [], 2023-11-26T07:23:10.750 INFO:teuthology.orchestra.run.smithi143.stdout: "desc": "", 2023-11-26T07:23:10.751 INFO:teuthology.orchestra.run.smithi143.stdout: "long_desc": "", 2023-11-26T07:23:10.751 INFO:teuthology.orchestra.run.smithi143.stdout: "tags": [], 2023-11-26T07:23:10.751 INFO:teuthology.orchestra.run.smithi143.stdout: "see_also": [] 2023-11-26T07:23:10.751 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2023-11-26T07:23:10.751 INFO:teuthology.orchestra.run.smithi143.stdout: "exclude_perf_counters": { 2023-11-26T07:23:10.751 INFO:teuthology.orchestra.run.smithi143.stdout: "name": "exclude_perf_counters", 2023-11-26T07:23:10.751 INFO:teuthology.orchestra.run.smithi143.stdout: "type": "bool", 2023-11-26T07:23:10.751 INFO:teuthology.orchestra.run.smithi143.stdout: "level": "advanced", 2023-11-26T07:23:10.751 INFO:teuthology.orchestra.run.smithi143.stdout: "flags": 1, 2023-11-26T07:23:10.751 INFO:teuthology.orchestra.run.smithi143.stdout: "default_value": "True", 2023-11-26T07:23:10.751 INFO:teuthology.orchestra.run.smithi143.stdout: "min": "", 2023-11-26T07:23:10.751 INFO:teuthology.orchestra.run.smithi143.stdout: "max": "", 2023-11-26T07:23:10.752 INFO:teuthology.orchestra.run.smithi143.stdout: "enum_allowed": [], 2023-11-26T07:23:10.752 INFO:teuthology.orchestra.run.smithi143.stdout: "desc": "Do not include perf-counters in the metrics output", 2023-11-26T07:23:10.752 INFO:teuthology.orchestra.run.smithi143.stdout: "long_desc": "Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.", 2023-11-26T07:23:10.752 INFO:teuthology.orchestra.run.smithi143.stdout: "tags": [], 2023-11-26T07:23:10.752 INFO:teuthology.orchestra.run.smithi143.stdout: "see_also": [] 2023-11-26T07:23:10.752 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2023-11-26T07:23:10.752 INFO:teuthology.orchestra.run.smithi143.stdout: "log_level": { 2023-11-26T07:23:10.752 INFO:teuthology.orchestra.run.smithi143.stdout: "name": "log_level", 2023-11-26T07:23:10.752 INFO:teuthology.orchestra.run.smithi143.stdout: "type": "str", 2023-11-26T07:23:10.752 INFO:teuthology.orchestra.run.smithi143.stdout: "level": "advanced", 2023-11-26T07:23:10.752 INFO:teuthology.orchestra.run.smithi143.stdout: "flags": 1, 2023-11-26T07:23:10.752 INFO:teuthology.orchestra.run.smithi143.stdout: "default_value": "", 2023-11-26T07:23:10.753 INFO:teuthology.orchestra.run.smithi143.stdout: "min": "", 2023-11-26T07:23:10.753 INFO:teuthology.orchestra.run.smithi143.stdout: "max": "", 2023-11-26T07:23:10.753 INFO:teuthology.orchestra.run.smithi143.stdout: "enum_allowed": [ 2023-11-26T07:23:10.753 INFO:teuthology.orchestra.run.smithi143.stdout: "", 2023-11-26T07:23:10.753 INFO:teuthology.orchestra.run.smithi143.stdout: "critical", 2023-11-26T07:23:10.753 INFO:teuthology.orchestra.run.smithi143.stdout: "debug", 2023-11-26T07:23:10.753 INFO:teuthology.orchestra.run.smithi143.stdout: "error", 2023-11-26T07:23:10.753 INFO:teuthology.orchestra.run.smithi143.stdout: "info", 2023-11-26T07:23:10.753 INFO:teuthology.orchestra.run.smithi143.stdout: "warning" 2023-11-26T07:23:10.753 INFO:teuthology.orchestra.run.smithi143.stdout: ], 2023-11-26T07:23:10.753 INFO:teuthology.orchestra.run.smithi143.stdout: "desc": "", 2023-11-26T07:23:10.753 INFO:teuthology.orchestra.run.smithi143.stdout: "long_desc": "", 2023-11-26T07:23:10.753 INFO:teuthology.orchestra.run.smithi143.stdout: "tags": [], 2023-11-26T07:23:10.754 INFO:teuthology.orchestra.run.smithi143.stdout: "see_also": [] 2023-11-26T07:23:10.754 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2023-11-26T07:23:10.754 INFO:teuthology.orchestra.run.smithi143.stdout: "log_to_cluster": { 2023-11-26T07:23:10.754 INFO:teuthology.orchestra.run.smithi143.stdout: "name": "log_to_cluster", 2023-11-26T07:23:10.754 INFO:teuthology.orchestra.run.smithi143.stdout: "type": "bool", 2023-11-26T07:23:10.754 INFO:teuthology.orchestra.run.smithi143.stdout: "level": "advanced", 2023-11-26T07:23:10.754 INFO:teuthology.orchestra.run.smithi143.stdout: "flags": 1, 2023-11-26T07:23:10.754 INFO:teuthology.orchestra.run.smithi143.stdout: "default_value": "False", 2023-11-26T07:23:10.754 INFO:teuthology.orchestra.run.smithi143.stdout: "min": "", 2023-11-26T07:23:10.754 INFO:teuthology.orchestra.run.smithi143.stdout: "max": "", 2023-11-26T07:23:10.754 INFO:teuthology.orchestra.run.smithi143.stdout: "enum_allowed": [], 2023-11-26T07:23:10.754 INFO:teuthology.orchestra.run.smithi143.stdout: "desc": "", 2023-11-26T07:23:10.754 INFO:teuthology.orchestra.run.smithi143.stdout: "long_desc": "", 2023-11-26T07:23:10.755 INFO:teuthology.orchestra.run.smithi143.stdout: "tags": [], 2023-11-26T07:23:10.755 INFO:teuthology.orchestra.run.smithi143.stdout: "see_also": [] 2023-11-26T07:23:10.755 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2023-11-26T07:23:10.755 INFO:teuthology.orchestra.run.smithi143.stdout: "log_to_cluster_level": { 2023-11-26T07:23:10.755 INFO:teuthology.orchestra.run.smithi143.stdout: "name": "log_to_cluster_level", 2023-11-26T07:23:10.755 INFO:teuthology.orchestra.run.smithi143.stdout: "type": "str", 2023-11-26T07:23:10.755 INFO:teuthology.orchestra.run.smithi143.stdout: "level": "advanced", 2023-11-26T07:23:10.755 INFO:teuthology.orchestra.run.smithi143.stdout: "flags": 1, 2023-11-26T07:23:10.755 INFO:teuthology.orchestra.run.smithi143.stdout: "default_value": "info", 2023-11-26T07:23:10.755 INFO:teuthology.orchestra.run.smithi143.stdout: "min": "", 2023-11-26T07:23:10.755 INFO:teuthology.orchestra.run.smithi143.stdout: "max": "", 2023-11-26T07:23:10.755 INFO:teuthology.orchestra.run.smithi143.stdout: "enum_allowed": [ 2023-11-26T07:23:10.755 INFO:teuthology.orchestra.run.smithi143.stdout: "", 2023-11-26T07:23:10.756 INFO:teuthology.orchestra.run.smithi143.stdout: "critical", 2023-11-26T07:23:10.756 INFO:teuthology.orchestra.run.smithi143.stdout: "debug", 2023-11-26T07:23:10.756 INFO:teuthology.orchestra.run.smithi143.stdout: "error", 2023-11-26T07:23:10.756 INFO:teuthology.orchestra.run.smithi143.stdout: "info", 2023-11-26T07:23:10.756 INFO:teuthology.orchestra.run.smithi143.stdout: "warning" 2023-11-26T07:23:10.756 INFO:teuthology.orchestra.run.smithi143.stdout: ], 2023-11-26T07:23:10.756 INFO:teuthology.orchestra.run.smithi143.stdout: "desc": "", 2023-11-26T07:23:10.756 INFO:teuthology.orchestra.run.smithi143.stdout: "long_desc": "", 2023-11-26T07:23:10.756 INFO:teuthology.orchestra.run.smithi143.stdout: "tags": [], 2023-11-26T07:23:10.756 INFO:teuthology.orchestra.run.smithi143.stdout: "see_also": [] 2023-11-26T07:23:10.756 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2023-11-26T07:23:10.756 INFO:teuthology.orchestra.run.smithi143.stdout: "log_to_file": { 2023-11-26T07:23:10.757 INFO:teuthology.orchestra.run.smithi143.stdout: "name": "log_to_file", 2023-11-26T07:23:10.757 INFO:teuthology.orchestra.run.smithi143.stdout: "type": "bool", 2023-11-26T07:23:10.757 INFO:teuthology.orchestra.run.smithi143.stdout: "level": "advanced", 2023-11-26T07:23:10.757 INFO:teuthology.orchestra.run.smithi143.stdout: "flags": 1, 2023-11-26T07:23:10.757 INFO:teuthology.orchestra.run.smithi143.stdout: "default_value": "False", 2023-11-26T07:23:10.757 INFO:teuthology.orchestra.run.smithi143.stdout: "min": "", 2023-11-26T07:23:10.757 INFO:teuthology.orchestra.run.smithi143.stdout: "max": "", 2023-11-26T07:23:10.757 INFO:teuthology.orchestra.run.smithi143.stdout: "enum_allowed": [], 2023-11-26T07:23:10.757 INFO:teuthology.orchestra.run.smithi143.stdout: "desc": "", 2023-11-26T07:23:10.757 INFO:teuthology.orchestra.run.smithi143.stdout: "long_desc": "", 2023-11-26T07:23:10.757 INFO:teuthology.orchestra.run.smithi143.stdout: "tags": [], 2023-11-26T07:23:10.757 INFO:teuthology.orchestra.run.smithi143.stdout: "see_also": [] 2023-11-26T07:23:10.757 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2023-11-26T07:23:10.758 INFO:teuthology.orchestra.run.smithi143.stdout: "rbd_stats_pools": { 2023-11-26T07:23:10.758 INFO:teuthology.orchestra.run.smithi143.stdout: "name": "rbd_stats_pools", 2023-11-26T07:23:10.758 INFO:teuthology.orchestra.run.smithi143.stdout: "type": "str", 2023-11-26T07:23:10.758 INFO:teuthology.orchestra.run.smithi143.stdout: "level": "advanced", 2023-11-26T07:23:10.758 INFO:teuthology.orchestra.run.smithi143.stdout: "flags": 0, 2023-11-26T07:23:10.758 INFO:teuthology.orchestra.run.smithi143.stdout: "default_value": "", 2023-11-26T07:23:10.758 INFO:teuthology.orchestra.run.smithi143.stdout: "min": "", 2023-11-26T07:23:10.758 INFO:teuthology.orchestra.run.smithi143.stdout: "max": "", 2023-11-26T07:23:10.758 INFO:teuthology.orchestra.run.smithi143.stdout: "enum_allowed": [], 2023-11-26T07:23:10.758 INFO:teuthology.orchestra.run.smithi143.stdout: "desc": "", 2023-11-26T07:23:10.758 INFO:teuthology.orchestra.run.smithi143.stdout: "long_desc": "", 2023-11-26T07:23:10.758 INFO:teuthology.orchestra.run.smithi143.stdout: "tags": [], 2023-11-26T07:23:10.759 INFO:teuthology.orchestra.run.smithi143.stdout: "see_also": [] 2023-11-26T07:23:10.759 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2023-11-26T07:23:10.759 INFO:teuthology.orchestra.run.smithi143.stdout: "rbd_stats_pools_refresh_interval": { 2023-11-26T07:23:10.759 INFO:teuthology.orchestra.run.smithi143.stdout: "name": "rbd_stats_pools_refresh_interval", 2023-11-26T07:23:10.759 INFO:teuthology.orchestra.run.smithi143.stdout: "type": "int", 2023-11-26T07:23:10.759 INFO:teuthology.orchestra.run.smithi143.stdout: "level": "advanced", 2023-11-26T07:23:10.759 INFO:teuthology.orchestra.run.smithi143.stdout: "flags": 0, 2023-11-26T07:23:10.759 INFO:teuthology.orchestra.run.smithi143.stdout: "default_value": "300", 2023-11-26T07:23:10.759 INFO:teuthology.orchestra.run.smithi143.stdout: "min": "", 2023-11-26T07:23:10.759 INFO:teuthology.orchestra.run.smithi143.stdout: "max": "", 2023-11-26T07:23:10.759 INFO:teuthology.orchestra.run.smithi143.stdout: "enum_allowed": [], 2023-11-26T07:23:10.759 INFO:teuthology.orchestra.run.smithi143.stdout: "desc": "", 2023-11-26T07:23:10.759 INFO:teuthology.orchestra.run.smithi143.stdout: "long_desc": "", 2023-11-26T07:23:10.760 INFO:teuthology.orchestra.run.smithi143.stdout: "tags": [], 2023-11-26T07:23:10.760 INFO:teuthology.orchestra.run.smithi143.stdout: "see_also": [] 2023-11-26T07:23:10.760 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2023-11-26T07:23:10.760 INFO:teuthology.orchestra.run.smithi143.stdout: "scrape_interval": { 2023-11-26T07:23:10.760 INFO:teuthology.orchestra.run.smithi143.stdout: "name": "scrape_interval", 2023-11-26T07:23:10.760 INFO:teuthology.orchestra.run.smithi143.stdout: "type": "float", 2023-11-26T07:23:10.760 INFO:teuthology.orchestra.run.smithi143.stdout: "level": "advanced", 2023-11-26T07:23:10.760 INFO:teuthology.orchestra.run.smithi143.stdout: "flags": 0, 2023-11-26T07:23:10.760 INFO:teuthology.orchestra.run.smithi143.stdout: "default_value": "15.0", 2023-11-26T07:23:10.760 INFO:teuthology.orchestra.run.smithi143.stdout: "min": "", 2023-11-26T07:23:10.760 INFO:teuthology.orchestra.run.smithi143.stdout: "max": "", 2023-11-26T07:23:10.760 INFO:teuthology.orchestra.run.smithi143.stdout: "enum_allowed": [], 2023-11-26T07:23:10.760 INFO:teuthology.orchestra.run.smithi143.stdout: "desc": "", 2023-11-26T07:23:10.761 INFO:teuthology.orchestra.run.smithi143.stdout: "long_desc": "", 2023-11-26T07:23:10.761 INFO:teuthology.orchestra.run.smithi143.stdout: "tags": [], 2023-11-26T07:23:10.761 INFO:teuthology.orchestra.run.smithi143.stdout: "see_also": [] 2023-11-26T07:23:10.761 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2023-11-26T07:23:10.761 INFO:teuthology.orchestra.run.smithi143.stdout: "server_addr": { 2023-11-26T07:23:10.761 INFO:teuthology.orchestra.run.smithi143.stdout: "name": "server_addr", 2023-11-26T07:23:10.761 INFO:teuthology.orchestra.run.smithi143.stdout: "type": "str", 2023-11-26T07:23:10.761 INFO:teuthology.orchestra.run.smithi143.stdout: "level": "advanced", 2023-11-26T07:23:10.761 INFO:teuthology.orchestra.run.smithi143.stdout: "flags": 0, 2023-11-26T07:23:10.761 INFO:teuthology.orchestra.run.smithi143.stdout: "default_value": "::", 2023-11-26T07:23:10.761 INFO:teuthology.orchestra.run.smithi143.stdout: "min": "", 2023-11-26T07:23:10.761 INFO:teuthology.orchestra.run.smithi143.stdout: "max": "", 2023-11-26T07:23:10.762 INFO:teuthology.orchestra.run.smithi143.stdout: "enum_allowed": [], 2023-11-26T07:23:10.762 INFO:teuthology.orchestra.run.smithi143.stdout: "desc": "the IPv4 or IPv6 address on which the module listens for HTTP requests", 2023-11-26T07:23:10.762 INFO:teuthology.orchestra.run.smithi143.stdout: "long_desc": "", 2023-11-26T07:23:10.762 INFO:teuthology.orchestra.run.smithi143.stdout: "tags": [], 2023-11-26T07:23:10.762 INFO:teuthology.orchestra.run.smithi143.stdout: "see_also": [] 2023-11-26T07:23:10.762 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2023-11-26T07:23:10.762 INFO:teuthology.orchestra.run.smithi143.stdout: "server_port": { 2023-11-26T07:23:10.762 INFO:teuthology.orchestra.run.smithi143.stdout: "name": "server_port", 2023-11-26T07:23:10.762 INFO:teuthology.orchestra.run.smithi143.stdout: "type": "int", 2023-11-26T07:23:10.762 INFO:teuthology.orchestra.run.smithi143.stdout: "level": "advanced", 2023-11-26T07:23:10.762 INFO:teuthology.orchestra.run.smithi143.stdout: "flags": 1, 2023-11-26T07:23:10.762 INFO:teuthology.orchestra.run.smithi143.stdout: "default_value": "9283", 2023-11-26T07:23:10.763 INFO:teuthology.orchestra.run.smithi143.stdout: "min": "", 2023-11-26T07:23:10.763 INFO:teuthology.orchestra.run.smithi143.stdout: "max": "", 2023-11-26T07:23:10.763 INFO:teuthology.orchestra.run.smithi143.stdout: "enum_allowed": [], 2023-11-26T07:23:10.763 INFO:teuthology.orchestra.run.smithi143.stdout: "desc": "the port on which the module listens for HTTP requests", 2023-11-26T07:23:10.763 INFO:teuthology.orchestra.run.smithi143.stdout: "long_desc": "", 2023-11-26T07:23:10.763 INFO:teuthology.orchestra.run.smithi143.stdout: "tags": [], 2023-11-26T07:23:10.763 INFO:teuthology.orchestra.run.smithi143.stdout: "see_also": [] 2023-11-26T07:23:10.763 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2023-11-26T07:23:10.763 INFO:teuthology.orchestra.run.smithi143.stdout: "stale_cache_strategy": { 2023-11-26T07:23:10.763 INFO:teuthology.orchestra.run.smithi143.stdout: "name": "stale_cache_strategy", 2023-11-26T07:23:10.763 INFO:teuthology.orchestra.run.smithi143.stdout: "type": "str", 2023-11-26T07:23:10.763 INFO:teuthology.orchestra.run.smithi143.stdout: "level": "advanced", 2023-11-26T07:23:10.763 INFO:teuthology.orchestra.run.smithi143.stdout: "flags": 0, 2023-11-26T07:23:10.764 INFO:teuthology.orchestra.run.smithi143.stdout: "default_value": "log", 2023-11-26T07:23:10.764 INFO:teuthology.orchestra.run.smithi143.stdout: "min": "", 2023-11-26T07:23:10.764 INFO:teuthology.orchestra.run.smithi143.stdout: "max": "", 2023-11-26T07:23:10.764 INFO:teuthology.orchestra.run.smithi143.stdout: "enum_allowed": [], 2023-11-26T07:23:10.764 INFO:teuthology.orchestra.run.smithi143.stdout: "desc": "", 2023-11-26T07:23:10.764 INFO:teuthology.orchestra.run.smithi143.stdout: "long_desc": "", 2023-11-26T07:23:10.764 INFO:teuthology.orchestra.run.smithi143.stdout: "tags": [], 2023-11-26T07:23:10.764 INFO:teuthology.orchestra.run.smithi143.stdout: "see_also": [] 2023-11-26T07:23:10.764 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2023-11-26T07:23:10.764 INFO:teuthology.orchestra.run.smithi143.stdout: "standby_behaviour": { 2023-11-26T07:23:10.764 INFO:teuthology.orchestra.run.smithi143.stdout: "name": "standby_behaviour", 2023-11-26T07:23:10.764 INFO:teuthology.orchestra.run.smithi143.stdout: "type": "str", 2023-11-26T07:23:10.765 INFO:teuthology.orchestra.run.smithi143.stdout: "level": "advanced", 2023-11-26T07:23:10.765 INFO:teuthology.orchestra.run.smithi143.stdout: "flags": 1, 2023-11-26T07:23:10.765 INFO:teuthology.orchestra.run.smithi143.stdout: "default_value": "default", 2023-11-26T07:23:10.765 INFO:teuthology.orchestra.run.smithi143.stdout: "min": "", 2023-11-26T07:23:10.765 INFO:teuthology.orchestra.run.smithi143.stdout: "max": "", 2023-11-26T07:23:10.765 INFO:teuthology.orchestra.run.smithi143.stdout: "enum_allowed": [ 2023-11-26T07:23:10.765 INFO:teuthology.orchestra.run.smithi143.stdout: "default", 2023-11-26T07:23:10.765 INFO:teuthology.orchestra.run.smithi143.stdout: "error" 2023-11-26T07:23:10.765 INFO:teuthology.orchestra.run.smithi143.stdout: ], 2023-11-26T07:23:10.765 INFO:teuthology.orchestra.run.smithi143.stdout: "desc": "", 2023-11-26T07:23:10.765 INFO:teuthology.orchestra.run.smithi143.stdout: "long_desc": "", 2023-11-26T07:23:10.765 INFO:teuthology.orchestra.run.smithi143.stdout: "tags": [], 2023-11-26T07:23:10.765 INFO:teuthology.orchestra.run.smithi143.stdout: "see_also": [] 2023-11-26T07:23:10.766 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2023-11-26T07:23:10.766 INFO:teuthology.orchestra.run.smithi143.stdout: "standby_error_status_code": { 2023-11-26T07:23:10.766 INFO:teuthology.orchestra.run.smithi143.stdout: "name": "standby_error_status_code", 2023-11-26T07:23:10.766 INFO:teuthology.orchestra.run.smithi143.stdout: "type": "int", 2023-11-26T07:23:10.766 INFO:teuthology.orchestra.run.smithi143.stdout: "level": "advanced", 2023-11-26T07:23:10.766 INFO:teuthology.orchestra.run.smithi143.stdout: "flags": 1, 2023-11-26T07:23:10.766 INFO:teuthology.orchestra.run.smithi143.stdout: "default_value": "500", 2023-11-26T07:23:10.766 INFO:teuthology.orchestra.run.smithi143.stdout: "min": "400", 2023-11-26T07:23:10.766 INFO:teuthology.orchestra.run.smithi143.stdout: "max": "599", 2023-11-26T07:23:10.766 INFO:teuthology.orchestra.run.smithi143.stdout: "enum_allowed": [], 2023-11-26T07:23:10.766 INFO:teuthology.orchestra.run.smithi143.stdout: "desc": "", 2023-11-26T07:23:10.766 INFO:teuthology.orchestra.run.smithi143.stdout: "long_desc": "", 2023-11-26T07:23:10.767 INFO:teuthology.orchestra.run.smithi143.stdout: "tags": [], 2023-11-26T07:23:10.767 INFO:teuthology.orchestra.run.smithi143.stdout: "see_also": [] 2023-11-26T07:23:10.767 INFO:teuthology.orchestra.run.smithi143.stdout: } 2023-11-26T07:23:10.767 INFO:teuthology.orchestra.run.smithi143.stdout: } 2023-11-26T07:23:10.767 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2023-11-26T07:23:10.767 INFO:teuthology.orchestra.run.smithi143.stdout: { 2023-11-26T07:23:10.767 INFO:teuthology.orchestra.run.smithi143.stdout: "name": "rbd_support", 2023-11-26T07:23:10.767 INFO:teuthology.orchestra.run.smithi143.stdout: "can_run": true, 2023-11-26T07:23:10.767 INFO:teuthology.orchestra.run.smithi143.stdout: "error_string": "", 2023-11-26T07:23:10.767 INFO:teuthology.orchestra.run.smithi143.stdout: "module_options": { 2023-11-26T07:23:10.767 INFO:teuthology.orchestra.run.smithi143.stdout: "log_level": { 2023-11-26T07:23:10.767 INFO:teuthology.orchestra.run.smithi143.stdout: "name": "log_level", 2023-11-26T07:23:10.768 INFO:teuthology.orchestra.run.smithi143.stdout: "type": "str", 2023-11-26T07:23:10.768 INFO:teuthology.orchestra.run.smithi143.stdout: "level": "advanced", 2023-11-26T07:23:10.768 INFO:teuthology.orchestra.run.smithi143.stdout: "flags": 1, 2023-11-26T07:23:10.768 INFO:teuthology.orchestra.run.smithi143.stdout: "default_value": "", 2023-11-26T07:23:10.768 INFO:teuthology.orchestra.run.smithi143.stdout: "min": "", 2023-11-26T07:23:10.768 INFO:teuthology.orchestra.run.smithi143.stdout: "max": "", 2023-11-26T07:23:10.768 INFO:teuthology.orchestra.run.smithi143.stdout: "enum_allowed": [ 2023-11-26T07:23:10.768 INFO:teuthology.orchestra.run.smithi143.stdout: "", 2023-11-26T07:23:10.768 INFO:teuthology.orchestra.run.smithi143.stdout: "critical", 2023-11-26T07:23:10.768 INFO:teuthology.orchestra.run.smithi143.stdout: "debug", 2023-11-26T07:23:10.768 INFO:teuthology.orchestra.run.smithi143.stdout: "error", 2023-11-26T07:23:10.768 INFO:teuthology.orchestra.run.smithi143.stdout: "info", 2023-11-26T07:23:10.768 INFO:teuthology.orchestra.run.smithi143.stdout: "warning" 2023-11-26T07:23:10.769 INFO:teuthology.orchestra.run.smithi143.stdout: ], 2023-11-26T07:23:10.769 INFO:teuthology.orchestra.run.smithi143.stdout: "desc": "", 2023-11-26T07:23:10.769 INFO:teuthology.orchestra.run.smithi143.stdout: "long_desc": "", 2023-11-26T07:23:10.769 INFO:teuthology.orchestra.run.smithi143.stdout: "tags": [], 2023-11-26T07:23:10.769 INFO:teuthology.orchestra.run.smithi143.stdout: "see_also": [] 2023-11-26T07:23:10.769 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2023-11-26T07:23:10.769 INFO:teuthology.orchestra.run.smithi143.stdout: "log_to_cluster": { 2023-11-26T07:23:10.769 INFO:teuthology.orchestra.run.smithi143.stdout: "name": "log_to_cluster", 2023-11-26T07:23:10.769 INFO:teuthology.orchestra.run.smithi143.stdout: "type": "bool", 2023-11-26T07:23:10.769 INFO:teuthology.orchestra.run.smithi143.stdout: "level": "advanced", 2023-11-26T07:23:10.769 INFO:teuthology.orchestra.run.smithi143.stdout: "flags": 1, 2023-11-26T07:23:10.769 INFO:teuthology.orchestra.run.smithi143.stdout: "default_value": "False", 2023-11-26T07:23:10.770 INFO:teuthology.orchestra.run.smithi143.stdout: "min": "", 2023-11-26T07:23:10.770 INFO:teuthology.orchestra.run.smithi143.stdout: "max": "", 2023-11-26T07:23:10.770 INFO:teuthology.orchestra.run.smithi143.stdout: "enum_allowed": [], 2023-11-26T07:23:10.770 INFO:teuthology.orchestra.run.smithi143.stdout: "desc": "", 2023-11-26T07:23:10.770 INFO:teuthology.orchestra.run.smithi143.stdout: "long_desc": "", 2023-11-26T07:23:10.770 INFO:teuthology.orchestra.run.smithi143.stdout: "tags": [], 2023-11-26T07:23:10.770 INFO:teuthology.orchestra.run.smithi143.stdout: "see_also": [] 2023-11-26T07:23:10.770 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2023-11-26T07:23:10.770 INFO:teuthology.orchestra.run.smithi143.stdout: "log_to_cluster_level": { 2023-11-26T07:23:10.770 INFO:teuthology.orchestra.run.smithi143.stdout: "name": "log_to_cluster_level", 2023-11-26T07:23:10.770 INFO:teuthology.orchestra.run.smithi143.stdout: "type": "str", 2023-11-26T07:23:10.770 INFO:teuthology.orchestra.run.smithi143.stdout: "level": "advanced", 2023-11-26T07:23:10.770 INFO:teuthology.orchestra.run.smithi143.stdout: "flags": 1, 2023-11-26T07:23:10.771 INFO:teuthology.orchestra.run.smithi143.stdout: "default_value": "info", 2023-11-26T07:23:10.771 INFO:teuthology.orchestra.run.smithi143.stdout: "min": "", 2023-11-26T07:23:10.771 INFO:teuthology.orchestra.run.smithi143.stdout: "max": "", 2023-11-26T07:23:10.771 INFO:teuthology.orchestra.run.smithi143.stdout: "enum_allowed": [ 2023-11-26T07:23:10.771 INFO:teuthology.orchestra.run.smithi143.stdout: "", 2023-11-26T07:23:10.771 INFO:teuthology.orchestra.run.smithi143.stdout: "critical", 2023-11-26T07:23:10.771 INFO:teuthology.orchestra.run.smithi143.stdout: "debug", 2023-11-26T07:23:10.771 INFO:teuthology.orchestra.run.smithi143.stdout: "error", 2023-11-26T07:23:10.771 INFO:teuthology.orchestra.run.smithi143.stdout: "info", 2023-11-26T07:23:10.771 INFO:teuthology.orchestra.run.smithi143.stdout: "warning" 2023-11-26T07:23:10.771 INFO:teuthology.orchestra.run.smithi143.stdout: ], 2023-11-26T07:23:10.771 INFO:teuthology.orchestra.run.smithi143.stdout: "desc": "", 2023-11-26T07:23:10.772 INFO:teuthology.orchestra.run.smithi143.stdout: "long_desc": "", 2023-11-26T07:23:10.772 INFO:teuthology.orchestra.run.smithi143.stdout: "tags": [], 2023-11-26T07:23:10.772 INFO:teuthology.orchestra.run.smithi143.stdout: "see_also": [] 2023-11-26T07:23:10.772 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2023-11-26T07:23:10.772 INFO:teuthology.orchestra.run.smithi143.stdout: "log_to_file": { 2023-11-26T07:23:10.772 INFO:teuthology.orchestra.run.smithi143.stdout: "name": "log_to_file", 2023-11-26T07:23:10.772 INFO:teuthology.orchestra.run.smithi143.stdout: "type": "bool", 2023-11-26T07:23:10.772 INFO:teuthology.orchestra.run.smithi143.stdout: "level": "advanced", 2023-11-26T07:23:10.772 INFO:teuthology.orchestra.run.smithi143.stdout: "flags": 1, 2023-11-26T07:23:10.772 INFO:teuthology.orchestra.run.smithi143.stdout: "default_value": "False", 2023-11-26T07:23:10.772 INFO:teuthology.orchestra.run.smithi143.stdout: "min": "", 2023-11-26T07:23:10.772 INFO:teuthology.orchestra.run.smithi143.stdout: "max": "", 2023-11-26T07:23:10.772 INFO:teuthology.orchestra.run.smithi143.stdout: "enum_allowed": [], 2023-11-26T07:23:10.773 INFO:teuthology.orchestra.run.smithi143.stdout: "desc": "", 2023-11-26T07:23:10.773 INFO:teuthology.orchestra.run.smithi143.stdout: "long_desc": "", 2023-11-26T07:23:10.773 INFO:teuthology.orchestra.run.smithi143.stdout: "tags": [], 2023-11-26T07:23:10.773 INFO:teuthology.orchestra.run.smithi143.stdout: "see_also": [] 2023-11-26T07:23:10.773 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2023-11-26T07:23:10.773 INFO:teuthology.orchestra.run.smithi143.stdout: "max_concurrent_snap_create": { 2023-11-26T07:23:10.773 INFO:teuthology.orchestra.run.smithi143.stdout: "name": "max_concurrent_snap_create", 2023-11-26T07:23:10.773 INFO:teuthology.orchestra.run.smithi143.stdout: "type": "int", 2023-11-26T07:23:10.773 INFO:teuthology.orchestra.run.smithi143.stdout: "level": "advanced", 2023-11-26T07:23:10.773 INFO:teuthology.orchestra.run.smithi143.stdout: "flags": 0, 2023-11-26T07:23:10.773 INFO:teuthology.orchestra.run.smithi143.stdout: "default_value": "10", 2023-11-26T07:23:10.773 INFO:teuthology.orchestra.run.smithi143.stdout: "min": "", 2023-11-26T07:23:10.774 INFO:teuthology.orchestra.run.smithi143.stdout: "max": "", 2023-11-26T07:23:10.774 INFO:teuthology.orchestra.run.smithi143.stdout: "enum_allowed": [], 2023-11-26T07:23:10.774 INFO:teuthology.orchestra.run.smithi143.stdout: "desc": "", 2023-11-26T07:23:10.774 INFO:teuthology.orchestra.run.smithi143.stdout: "long_desc": "", 2023-11-26T07:23:10.774 INFO:teuthology.orchestra.run.smithi143.stdout: "tags": [], 2023-11-26T07:23:10.774 INFO:teuthology.orchestra.run.smithi143.stdout: "see_also": [] 2023-11-26T07:23:10.774 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2023-11-26T07:23:10.774 INFO:teuthology.orchestra.run.smithi143.stdout: "mirror_snapshot_schedule": { 2023-11-26T07:23:10.774 INFO:teuthology.orchestra.run.smithi143.stdout: "name": "mirror_snapshot_schedule", 2023-11-26T07:23:10.774 INFO:teuthology.orchestra.run.smithi143.stdout: "type": "str", 2023-11-26T07:23:10.774 INFO:teuthology.orchestra.run.smithi143.stdout: "level": "advanced", 2023-11-26T07:23:10.774 INFO:teuthology.orchestra.run.smithi143.stdout: "flags": 0, 2023-11-26T07:23:10.775 INFO:teuthology.orchestra.run.smithi143.stdout: "default_value": "", 2023-11-26T07:23:10.775 INFO:teuthology.orchestra.run.smithi143.stdout: "min": "", 2023-11-26T07:23:10.775 INFO:teuthology.orchestra.run.smithi143.stdout: "max": "", 2023-11-26T07:23:10.775 INFO:teuthology.orchestra.run.smithi143.stdout: "enum_allowed": [], 2023-11-26T07:23:10.775 INFO:teuthology.orchestra.run.smithi143.stdout: "desc": "", 2023-11-26T07:23:10.775 INFO:teuthology.orchestra.run.smithi143.stdout: "long_desc": "", 2023-11-26T07:23:10.775 INFO:teuthology.orchestra.run.smithi143.stdout: "tags": [], 2023-11-26T07:23:10.775 INFO:teuthology.orchestra.run.smithi143.stdout: "see_also": [] 2023-11-26T07:23:10.775 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2023-11-26T07:23:10.775 INFO:teuthology.orchestra.run.smithi143.stdout: "trash_purge_schedule": { 2023-11-26T07:23:10.775 INFO:teuthology.orchestra.run.smithi143.stdout: "name": "trash_purge_schedule", 2023-11-26T07:23:10.775 INFO:teuthology.orchestra.run.smithi143.stdout: "type": "str", 2023-11-26T07:23:10.776 INFO:teuthology.orchestra.run.smithi143.stdout: "level": "advanced", 2023-11-26T07:23:10.776 INFO:teuthology.orchestra.run.smithi143.stdout: "flags": 0, 2023-11-26T07:23:10.776 INFO:teuthology.orchestra.run.smithi143.stdout: "default_value": "", 2023-11-26T07:23:10.776 INFO:teuthology.orchestra.run.smithi143.stdout: "min": "", 2023-11-26T07:23:10.776 INFO:teuthology.orchestra.run.smithi143.stdout: "max": "", 2023-11-26T07:23:10.776 INFO:teuthology.orchestra.run.smithi143.stdout: "enum_allowed": [], 2023-11-26T07:23:10.776 INFO:teuthology.orchestra.run.smithi143.stdout: "desc": "", 2023-11-26T07:23:10.776 INFO:teuthology.orchestra.run.smithi143.stdout: "long_desc": "", 2023-11-26T07:23:10.776 INFO:teuthology.orchestra.run.smithi143.stdout: "tags": [], 2023-11-26T07:23:10.776 INFO:teuthology.orchestra.run.smithi143.stdout: "see_also": [] 2023-11-26T07:23:10.776 INFO:teuthology.orchestra.run.smithi143.stdout: } 2023-11-26T07:23:10.776 INFO:teuthology.orchestra.run.smithi143.stdout: } 2023-11-26T07:23:10.776 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2023-11-26T07:23:10.777 INFO:teuthology.orchestra.run.smithi143.stdout: { 2023-11-26T07:23:10.777 INFO:teuthology.orchestra.run.smithi143.stdout: "name": "restful", 2023-11-26T07:23:10.777 INFO:teuthology.orchestra.run.smithi143.stdout: "can_run": true, 2023-11-26T07:23:10.777 INFO:teuthology.orchestra.run.smithi143.stdout: "error_string": "", 2023-11-26T07:23:10.777 INFO:teuthology.orchestra.run.smithi143.stdout: "module_options": { 2023-11-26T07:23:10.777 INFO:teuthology.orchestra.run.smithi143.stdout: "enable_auth": { 2023-11-26T07:23:10.777 INFO:teuthology.orchestra.run.smithi143.stdout: "name": "enable_auth", 2023-11-26T07:23:10.777 INFO:teuthology.orchestra.run.smithi143.stdout: "type": "bool", 2023-11-26T07:23:10.777 INFO:teuthology.orchestra.run.smithi143.stdout: "level": "advanced", 2023-11-26T07:23:10.777 INFO:teuthology.orchestra.run.smithi143.stdout: "flags": 0, 2023-11-26T07:23:10.777 INFO:teuthology.orchestra.run.smithi143.stdout: "default_value": "True", 2023-11-26T07:23:10.777 INFO:teuthology.orchestra.run.smithi143.stdout: "min": "", 2023-11-26T07:23:10.777 INFO:teuthology.orchestra.run.smithi143.stdout: "max": "", 2023-11-26T07:23:10.778 INFO:teuthology.orchestra.run.smithi143.stdout: "enum_allowed": [], 2023-11-26T07:23:10.778 INFO:teuthology.orchestra.run.smithi143.stdout: "desc": "", 2023-11-26T07:23:10.778 INFO:teuthology.orchestra.run.smithi143.stdout: "long_desc": "", 2023-11-26T07:23:10.778 INFO:teuthology.orchestra.run.smithi143.stdout: "tags": [], 2023-11-26T07:23:10.778 INFO:teuthology.orchestra.run.smithi143.stdout: "see_also": [] 2023-11-26T07:23:10.778 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2023-11-26T07:23:10.778 INFO:teuthology.orchestra.run.smithi143.stdout: "key_file": { 2023-11-26T07:23:10.778 INFO:teuthology.orchestra.run.smithi143.stdout: "name": "key_file", 2023-11-26T07:23:10.778 INFO:teuthology.orchestra.run.smithi143.stdout: "type": "str", 2023-11-26T07:23:10.778 INFO:teuthology.orchestra.run.smithi143.stdout: "level": "advanced", 2023-11-26T07:23:10.778 INFO:teuthology.orchestra.run.smithi143.stdout: "flags": 0, 2023-11-26T07:23:10.778 INFO:teuthology.orchestra.run.smithi143.stdout: "default_value": "", 2023-11-26T07:23:10.779 INFO:teuthology.orchestra.run.smithi143.stdout: "min": "", 2023-11-26T07:23:10.779 INFO:teuthology.orchestra.run.smithi143.stdout: "max": "", 2023-11-26T07:23:10.779 INFO:teuthology.orchestra.run.smithi143.stdout: "enum_allowed": [], 2023-11-26T07:23:10.779 INFO:teuthology.orchestra.run.smithi143.stdout: "desc": "", 2023-11-26T07:23:10.779 INFO:teuthology.orchestra.run.smithi143.stdout: "long_desc": "", 2023-11-26T07:23:10.779 INFO:teuthology.orchestra.run.smithi143.stdout: "tags": [], 2023-11-26T07:23:10.779 INFO:teuthology.orchestra.run.smithi143.stdout: "see_also": [] 2023-11-26T07:23:10.779 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2023-11-26T07:23:10.779 INFO:teuthology.orchestra.run.smithi143.stdout: "log_level": { 2023-11-26T07:23:10.779 INFO:teuthology.orchestra.run.smithi143.stdout: "name": "log_level", 2023-11-26T07:23:10.779 INFO:teuthology.orchestra.run.smithi143.stdout: "type": "str", 2023-11-26T07:23:10.779 INFO:teuthology.orchestra.run.smithi143.stdout: "level": "advanced", 2023-11-26T07:23:10.779 INFO:teuthology.orchestra.run.smithi143.stdout: "flags": 1, 2023-11-26T07:23:10.780 INFO:teuthology.orchestra.run.smithi143.stdout: "default_value": "", 2023-11-26T07:23:10.780 INFO:teuthology.orchestra.run.smithi143.stdout: "min": "", 2023-11-26T07:23:10.780 INFO:teuthology.orchestra.run.smithi143.stdout: "max": "", 2023-11-26T07:23:10.780 INFO:teuthology.orchestra.run.smithi143.stdout: "enum_allowed": [ 2023-11-26T07:23:10.780 INFO:teuthology.orchestra.run.smithi143.stdout: "", 2023-11-26T07:23:10.780 INFO:teuthology.orchestra.run.smithi143.stdout: "critical", 2023-11-26T07:23:10.780 INFO:teuthology.orchestra.run.smithi143.stdout: "debug", 2023-11-26T07:23:10.780 INFO:teuthology.orchestra.run.smithi143.stdout: "error", 2023-11-26T07:23:10.780 INFO:teuthology.orchestra.run.smithi143.stdout: "info", 2023-11-26T07:23:10.780 INFO:teuthology.orchestra.run.smithi143.stdout: "warning" 2023-11-26T07:23:10.780 INFO:teuthology.orchestra.run.smithi143.stdout: ], 2023-11-26T07:23:10.780 INFO:teuthology.orchestra.run.smithi143.stdout: "desc": "", 2023-11-26T07:23:10.781 INFO:teuthology.orchestra.run.smithi143.stdout: "long_desc": "", 2023-11-26T07:23:10.781 INFO:teuthology.orchestra.run.smithi143.stdout: "tags": [], 2023-11-26T07:23:10.781 INFO:teuthology.orchestra.run.smithi143.stdout: "see_also": [] 2023-11-26T07:23:10.781 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2023-11-26T07:23:10.781 INFO:teuthology.orchestra.run.smithi143.stdout: "log_to_cluster": { 2023-11-26T07:23:10.781 INFO:teuthology.orchestra.run.smithi143.stdout: "name": "log_to_cluster", 2023-11-26T07:23:10.781 INFO:teuthology.orchestra.run.smithi143.stdout: "type": "bool", 2023-11-26T07:23:10.781 INFO:teuthology.orchestra.run.smithi143.stdout: "level": "advanced", 2023-11-26T07:23:10.781 INFO:teuthology.orchestra.run.smithi143.stdout: "flags": 1, 2023-11-26T07:23:10.781 INFO:teuthology.orchestra.run.smithi143.stdout: "default_value": "False", 2023-11-26T07:23:10.781 INFO:teuthology.orchestra.run.smithi143.stdout: "min": "", 2023-11-26T07:23:10.781 INFO:teuthology.orchestra.run.smithi143.stdout: "max": "", 2023-11-26T07:23:10.782 INFO:teuthology.orchestra.run.smithi143.stdout: "enum_allowed": [], 2023-11-26T07:23:10.782 INFO:teuthology.orchestra.run.smithi143.stdout: "desc": "", 2023-11-26T07:23:10.782 INFO:teuthology.orchestra.run.smithi143.stdout: "long_desc": "", 2023-11-26T07:23:10.782 INFO:teuthology.orchestra.run.smithi143.stdout: "tags": [], 2023-11-26T07:23:10.782 INFO:teuthology.orchestra.run.smithi143.stdout: "see_also": [] 2023-11-26T07:23:10.782 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2023-11-26T07:23:10.782 INFO:teuthology.orchestra.run.smithi143.stdout: "log_to_cluster_level": { 2023-11-26T07:23:10.782 INFO:teuthology.orchestra.run.smithi143.stdout: "name": "log_to_cluster_level", 2023-11-26T07:23:10.782 INFO:teuthology.orchestra.run.smithi143.stdout: "type": "str", 2023-11-26T07:23:10.782 INFO:teuthology.orchestra.run.smithi143.stdout: "level": "advanced", 2023-11-26T07:23:10.782 INFO:teuthology.orchestra.run.smithi143.stdout: "flags": 1, 2023-11-26T07:23:10.783 INFO:teuthology.orchestra.run.smithi143.stdout: "default_value": "info", 2023-11-26T07:23:10.783 INFO:teuthology.orchestra.run.smithi143.stdout: "min": "", 2023-11-26T07:23:10.783 INFO:teuthology.orchestra.run.smithi143.stdout: "max": "", 2023-11-26T07:23:10.783 INFO:teuthology.orchestra.run.smithi143.stdout: "enum_allowed": [ 2023-11-26T07:23:10.783 INFO:teuthology.orchestra.run.smithi143.stdout: "", 2023-11-26T07:23:10.783 INFO:teuthology.orchestra.run.smithi143.stdout: "critical", 2023-11-26T07:23:10.783 INFO:teuthology.orchestra.run.smithi143.stdout: "debug", 2023-11-26T07:23:10.783 INFO:teuthology.orchestra.run.smithi143.stdout: "error", 2023-11-26T07:23:10.783 INFO:teuthology.orchestra.run.smithi143.stdout: "info", 2023-11-26T07:23:10.783 INFO:teuthology.orchestra.run.smithi143.stdout: "warning" 2023-11-26T07:23:10.783 INFO:teuthology.orchestra.run.smithi143.stdout: ], 2023-11-26T07:23:10.783 INFO:teuthology.orchestra.run.smithi143.stdout: "desc": "", 2023-11-26T07:23:10.784 INFO:teuthology.orchestra.run.smithi143.stdout: "long_desc": "", 2023-11-26T07:23:10.784 INFO:teuthology.orchestra.run.smithi143.stdout: "tags": [], 2023-11-26T07:23:10.784 INFO:teuthology.orchestra.run.smithi143.stdout: "see_also": [] 2023-11-26T07:23:10.784 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2023-11-26T07:23:10.784 INFO:teuthology.orchestra.run.smithi143.stdout: "log_to_file": { 2023-11-26T07:23:10.784 INFO:teuthology.orchestra.run.smithi143.stdout: "name": "log_to_file", 2023-11-26T07:23:10.784 INFO:teuthology.orchestra.run.smithi143.stdout: "type": "bool", 2023-11-26T07:23:10.784 INFO:teuthology.orchestra.run.smithi143.stdout: "level": "advanced", 2023-11-26T07:23:10.784 INFO:teuthology.orchestra.run.smithi143.stdout: "flags": 1, 2023-11-26T07:23:10.784 INFO:teuthology.orchestra.run.smithi143.stdout: "default_value": "False", 2023-11-26T07:23:10.784 INFO:teuthology.orchestra.run.smithi143.stdout: "min": "", 2023-11-26T07:23:10.784 INFO:teuthology.orchestra.run.smithi143.stdout: "max": "", 2023-11-26T07:23:10.785 INFO:teuthology.orchestra.run.smithi143.stdout: "enum_allowed": [], 2023-11-26T07:23:10.785 INFO:teuthology.orchestra.run.smithi143.stdout: "desc": "", 2023-11-26T07:23:10.785 INFO:teuthology.orchestra.run.smithi143.stdout: "long_desc": "", 2023-11-26T07:23:10.785 INFO:teuthology.orchestra.run.smithi143.stdout: "tags": [], 2023-11-26T07:23:10.785 INFO:teuthology.orchestra.run.smithi143.stdout: "see_also": [] 2023-11-26T07:23:10.785 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2023-11-26T07:23:10.785 INFO:teuthology.orchestra.run.smithi143.stdout: "server_addr": { 2023-11-26T07:23:10.785 INFO:teuthology.orchestra.run.smithi143.stdout: "name": "server_addr", 2023-11-26T07:23:10.785 INFO:teuthology.orchestra.run.smithi143.stdout: "type": "str", 2023-11-26T07:23:10.785 INFO:teuthology.orchestra.run.smithi143.stdout: "level": "advanced", 2023-11-26T07:23:10.785 INFO:teuthology.orchestra.run.smithi143.stdout: "flags": 0, 2023-11-26T07:23:10.785 INFO:teuthology.orchestra.run.smithi143.stdout: "default_value": "", 2023-11-26T07:23:10.786 INFO:teuthology.orchestra.run.smithi143.stdout: "min": "", 2023-11-26T07:23:10.786 INFO:teuthology.orchestra.run.smithi143.stdout: "max": "", 2023-11-26T07:23:10.786 INFO:teuthology.orchestra.run.smithi143.stdout: "enum_allowed": [], 2023-11-26T07:23:10.786 INFO:teuthology.orchestra.run.smithi143.stdout: "desc": "", 2023-11-26T07:23:10.786 INFO:teuthology.orchestra.run.smithi143.stdout: "long_desc": "", 2023-11-26T07:23:10.786 INFO:teuthology.orchestra.run.smithi143.stdout: "tags": [], 2023-11-26T07:23:10.786 INFO:teuthology.orchestra.run.smithi143.stdout: "see_also": [] 2023-11-26T07:23:10.786 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2023-11-26T07:23:10.786 INFO:teuthology.orchestra.run.smithi143.stdout: "server_port": { 2023-11-26T07:23:10.786 INFO:teuthology.orchestra.run.smithi143.stdout: "name": "server_port", 2023-11-26T07:23:10.786 INFO:teuthology.orchestra.run.smithi143.stdout: "type": "str", 2023-11-26T07:23:10.786 INFO:teuthology.orchestra.run.smithi143.stdout: "level": "advanced", 2023-11-26T07:23:10.786 INFO:teuthology.orchestra.run.smithi143.stdout: "flags": 0, 2023-11-26T07:23:10.787 INFO:teuthology.orchestra.run.smithi143.stdout: "default_value": "", 2023-11-26T07:23:10.787 INFO:teuthology.orchestra.run.smithi143.stdout: "min": "", 2023-11-26T07:23:10.787 INFO:teuthology.orchestra.run.smithi143.stdout: "max": "", 2023-11-26T07:23:10.787 INFO:teuthology.orchestra.run.smithi143.stdout: "enum_allowed": [], 2023-11-26T07:23:10.787 INFO:teuthology.orchestra.run.smithi143.stdout: "desc": "", 2023-11-26T07:23:10.787 INFO:teuthology.orchestra.run.smithi143.stdout: "long_desc": "", 2023-11-26T07:23:10.787 INFO:teuthology.orchestra.run.smithi143.stdout: "tags": [], 2023-11-26T07:23:10.787 INFO:teuthology.orchestra.run.smithi143.stdout: "see_also": [] 2023-11-26T07:23:10.787 INFO:teuthology.orchestra.run.smithi143.stdout: } 2023-11-26T07:23:10.787 INFO:teuthology.orchestra.run.smithi143.stdout: } 2023-11-26T07:23:10.787 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2023-11-26T07:23:10.787 INFO:teuthology.orchestra.run.smithi143.stdout: { 2023-11-26T07:23:10.788 INFO:teuthology.orchestra.run.smithi143.stdout: "name": "selftest", 2023-11-26T07:23:10.788 INFO:teuthology.orchestra.run.smithi143.stdout: "can_run": true, 2023-11-26T07:23:10.788 INFO:teuthology.orchestra.run.smithi143.stdout: "error_string": "", 2023-11-26T07:23:10.788 INFO:teuthology.orchestra.run.smithi143.stdout: "module_options": { 2023-11-26T07:23:10.788 INFO:teuthology.orchestra.run.smithi143.stdout: "log_level": { 2023-11-26T07:23:10.788 INFO:teuthology.orchestra.run.smithi143.stdout: "name": "log_level", 2023-11-26T07:23:10.788 INFO:teuthology.orchestra.run.smithi143.stdout: "type": "str", 2023-11-26T07:23:10.788 INFO:teuthology.orchestra.run.smithi143.stdout: "level": "advanced", 2023-11-26T07:23:10.788 INFO:teuthology.orchestra.run.smithi143.stdout: "flags": 1, 2023-11-26T07:23:10.788 INFO:teuthology.orchestra.run.smithi143.stdout: "default_value": "", 2023-11-26T07:23:10.788 INFO:teuthology.orchestra.run.smithi143.stdout: "min": "", 2023-11-26T07:23:10.788 INFO:teuthology.orchestra.run.smithi143.stdout: "max": "", 2023-11-26T07:23:10.789 INFO:teuthology.orchestra.run.smithi143.stdout: "enum_allowed": [ 2023-11-26T07:23:10.789 INFO:teuthology.orchestra.run.smithi143.stdout: "", 2023-11-26T07:23:10.789 INFO:teuthology.orchestra.run.smithi143.stdout: "critical", 2023-11-26T07:23:10.789 INFO:teuthology.orchestra.run.smithi143.stdout: "debug", 2023-11-26T07:23:10.789 INFO:teuthology.orchestra.run.smithi143.stdout: "error", 2023-11-26T07:23:10.789 INFO:teuthology.orchestra.run.smithi143.stdout: "info", 2023-11-26T07:23:10.789 INFO:teuthology.orchestra.run.smithi143.stdout: "warning" 2023-11-26T07:23:10.789 INFO:teuthology.orchestra.run.smithi143.stdout: ], 2023-11-26T07:23:10.789 INFO:teuthology.orchestra.run.smithi143.stdout: "desc": "", 2023-11-26T07:23:10.789 INFO:teuthology.orchestra.run.smithi143.stdout: "long_desc": "", 2023-11-26T07:23:10.789 INFO:teuthology.orchestra.run.smithi143.stdout: "tags": [], 2023-11-26T07:23:10.789 INFO:teuthology.orchestra.run.smithi143.stdout: "see_also": [] 2023-11-26T07:23:10.790 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2023-11-26T07:23:10.790 INFO:teuthology.orchestra.run.smithi143.stdout: "log_to_cluster": { 2023-11-26T07:23:10.790 INFO:teuthology.orchestra.run.smithi143.stdout: "name": "log_to_cluster", 2023-11-26T07:23:10.790 INFO:teuthology.orchestra.run.smithi143.stdout: "type": "bool", 2023-11-26T07:23:10.790 INFO:teuthology.orchestra.run.smithi143.stdout: "level": "advanced", 2023-11-26T07:23:10.790 INFO:teuthology.orchestra.run.smithi143.stdout: "flags": 1, 2023-11-26T07:23:10.790 INFO:teuthology.orchestra.run.smithi143.stdout: "default_value": "False", 2023-11-26T07:23:10.790 INFO:teuthology.orchestra.run.smithi143.stdout: "min": "", 2023-11-26T07:23:10.790 INFO:teuthology.orchestra.run.smithi143.stdout: "max": "", 2023-11-26T07:23:10.790 INFO:teuthology.orchestra.run.smithi143.stdout: "enum_allowed": [], 2023-11-26T07:23:10.790 INFO:teuthology.orchestra.run.smithi143.stdout: "desc": "", 2023-11-26T07:23:10.790 INFO:teuthology.orchestra.run.smithi143.stdout: "long_desc": "", 2023-11-26T07:23:10.790 INFO:teuthology.orchestra.run.smithi143.stdout: "tags": [], 2023-11-26T07:23:10.791 INFO:teuthology.orchestra.run.smithi143.stdout: "see_also": [] 2023-11-26T07:23:10.791 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2023-11-26T07:23:10.791 INFO:teuthology.orchestra.run.smithi143.stdout: "log_to_cluster_level": { 2023-11-26T07:23:10.791 INFO:teuthology.orchestra.run.smithi143.stdout: "name": "log_to_cluster_level", 2023-11-26T07:23:10.791 INFO:teuthology.orchestra.run.smithi143.stdout: "type": "str", 2023-11-26T07:23:10.791 INFO:teuthology.orchestra.run.smithi143.stdout: "level": "advanced", 2023-11-26T07:23:10.791 INFO:teuthology.orchestra.run.smithi143.stdout: "flags": 1, 2023-11-26T07:23:10.791 INFO:teuthology.orchestra.run.smithi143.stdout: "default_value": "info", 2023-11-26T07:23:10.791 INFO:teuthology.orchestra.run.smithi143.stdout: "min": "", 2023-11-26T07:23:10.791 INFO:teuthology.orchestra.run.smithi143.stdout: "max": "", 2023-11-26T07:23:10.791 INFO:teuthology.orchestra.run.smithi143.stdout: "enum_allowed": [ 2023-11-26T07:23:10.791 INFO:teuthology.orchestra.run.smithi143.stdout: "", 2023-11-26T07:23:10.792 INFO:teuthology.orchestra.run.smithi143.stdout: "critical", 2023-11-26T07:23:10.792 INFO:teuthology.orchestra.run.smithi143.stdout: "debug", 2023-11-26T07:23:10.792 INFO:teuthology.orchestra.run.smithi143.stdout: "error", 2023-11-26T07:23:10.792 INFO:teuthology.orchestra.run.smithi143.stdout: "info", 2023-11-26T07:23:10.792 INFO:teuthology.orchestra.run.smithi143.stdout: "warning" 2023-11-26T07:23:10.792 INFO:teuthology.orchestra.run.smithi143.stdout: ], 2023-11-26T07:23:10.792 INFO:teuthology.orchestra.run.smithi143.stdout: "desc": "", 2023-11-26T07:23:10.792 INFO:teuthology.orchestra.run.smithi143.stdout: "long_desc": "", 2023-11-26T07:23:10.792 INFO:teuthology.orchestra.run.smithi143.stdout: "tags": [], 2023-11-26T07:23:10.792 INFO:teuthology.orchestra.run.smithi143.stdout: "see_also": [] 2023-11-26T07:23:10.792 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2023-11-26T07:23:10.792 INFO:teuthology.orchestra.run.smithi143.stdout: "log_to_file": { 2023-11-26T07:23:10.792 INFO:teuthology.orchestra.run.smithi143.stdout: "name": "log_to_file", 2023-11-26T07:23:10.793 INFO:teuthology.orchestra.run.smithi143.stdout: "type": "bool", 2023-11-26T07:23:10.793 INFO:teuthology.orchestra.run.smithi143.stdout: "level": "advanced", 2023-11-26T07:23:10.793 INFO:teuthology.orchestra.run.smithi143.stdout: "flags": 1, 2023-11-26T07:23:10.793 INFO:teuthology.orchestra.run.smithi143.stdout: "default_value": "False", 2023-11-26T07:23:10.793 INFO:teuthology.orchestra.run.smithi143.stdout: "min": "", 2023-11-26T07:23:10.793 INFO:teuthology.orchestra.run.smithi143.stdout: "max": "", 2023-11-26T07:23:10.793 INFO:teuthology.orchestra.run.smithi143.stdout: "enum_allowed": [], 2023-11-26T07:23:10.793 INFO:teuthology.orchestra.run.smithi143.stdout: "desc": "", 2023-11-26T07:23:10.793 INFO:teuthology.orchestra.run.smithi143.stdout: "long_desc": "", 2023-11-26T07:23:10.793 INFO:teuthology.orchestra.run.smithi143.stdout: "tags": [], 2023-11-26T07:23:10.793 INFO:teuthology.orchestra.run.smithi143.stdout: "see_also": [] 2023-11-26T07:23:10.793 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2023-11-26T07:23:10.794 INFO:teuthology.orchestra.run.smithi143.stdout: "roption1": { 2023-11-26T07:23:10.794 INFO:teuthology.orchestra.run.smithi143.stdout: "name": "roption1", 2023-11-26T07:23:10.794 INFO:teuthology.orchestra.run.smithi143.stdout: "type": "str", 2023-11-26T07:23:10.794 INFO:teuthology.orchestra.run.smithi143.stdout: "level": "advanced", 2023-11-26T07:23:10.794 INFO:teuthology.orchestra.run.smithi143.stdout: "flags": 0, 2023-11-26T07:23:10.794 INFO:teuthology.orchestra.run.smithi143.stdout: "default_value": "", 2023-11-26T07:23:10.794 INFO:teuthology.orchestra.run.smithi143.stdout: "min": "", 2023-11-26T07:23:10.794 INFO:teuthology.orchestra.run.smithi143.stdout: "max": "", 2023-11-26T07:23:10.794 INFO:teuthology.orchestra.run.smithi143.stdout: "enum_allowed": [], 2023-11-26T07:23:10.794 INFO:teuthology.orchestra.run.smithi143.stdout: "desc": "", 2023-11-26T07:23:10.794 INFO:teuthology.orchestra.run.smithi143.stdout: "long_desc": "", 2023-11-26T07:23:10.794 INFO:teuthology.orchestra.run.smithi143.stdout: "tags": [], 2023-11-26T07:23:10.795 INFO:teuthology.orchestra.run.smithi143.stdout: "see_also": [] 2023-11-26T07:23:10.795 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2023-11-26T07:23:10.795 INFO:teuthology.orchestra.run.smithi143.stdout: "roption2": { 2023-11-26T07:23:10.795 INFO:teuthology.orchestra.run.smithi143.stdout: "name": "roption2", 2023-11-26T07:23:10.795 INFO:teuthology.orchestra.run.smithi143.stdout: "type": "str", 2023-11-26T07:23:10.795 INFO:teuthology.orchestra.run.smithi143.stdout: "level": "advanced", 2023-11-26T07:23:10.795 INFO:teuthology.orchestra.run.smithi143.stdout: "flags": 0, 2023-11-26T07:23:10.795 INFO:teuthology.orchestra.run.smithi143.stdout: "default_value": "xyz", 2023-11-26T07:23:10.795 INFO:teuthology.orchestra.run.smithi143.stdout: "min": "", 2023-11-26T07:23:10.795 INFO:teuthology.orchestra.run.smithi143.stdout: "max": "", 2023-11-26T07:23:10.795 INFO:teuthology.orchestra.run.smithi143.stdout: "enum_allowed": [], 2023-11-26T07:23:10.795 INFO:teuthology.orchestra.run.smithi143.stdout: "desc": "", 2023-11-26T07:23:10.795 INFO:teuthology.orchestra.run.smithi143.stdout: "long_desc": "", 2023-11-26T07:23:10.796 INFO:teuthology.orchestra.run.smithi143.stdout: "tags": [], 2023-11-26T07:23:10.796 INFO:teuthology.orchestra.run.smithi143.stdout: "see_also": [] 2023-11-26T07:23:10.796 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2023-11-26T07:23:10.796 INFO:teuthology.orchestra.run.smithi143.stdout: "rwoption1": { 2023-11-26T07:23:10.796 INFO:teuthology.orchestra.run.smithi143.stdout: "name": "rwoption1", 2023-11-26T07:23:10.796 INFO:teuthology.orchestra.run.smithi143.stdout: "type": "str", 2023-11-26T07:23:10.796 INFO:teuthology.orchestra.run.smithi143.stdout: "level": "advanced", 2023-11-26T07:23:10.796 INFO:teuthology.orchestra.run.smithi143.stdout: "flags": 0, 2023-11-26T07:23:10.796 INFO:teuthology.orchestra.run.smithi143.stdout: "default_value": "", 2023-11-26T07:23:10.796 INFO:teuthology.orchestra.run.smithi143.stdout: "min": "", 2023-11-26T07:23:10.796 INFO:teuthology.orchestra.run.smithi143.stdout: "max": "", 2023-11-26T07:23:10.796 INFO:teuthology.orchestra.run.smithi143.stdout: "enum_allowed": [], 2023-11-26T07:23:10.796 INFO:teuthology.orchestra.run.smithi143.stdout: "desc": "", 2023-11-26T07:23:10.797 INFO:teuthology.orchestra.run.smithi143.stdout: "long_desc": "", 2023-11-26T07:23:10.797 INFO:teuthology.orchestra.run.smithi143.stdout: "tags": [], 2023-11-26T07:23:10.797 INFO:teuthology.orchestra.run.smithi143.stdout: "see_also": [] 2023-11-26T07:23:10.797 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2023-11-26T07:23:10.797 INFO:teuthology.orchestra.run.smithi143.stdout: "rwoption2": { 2023-11-26T07:23:10.797 INFO:teuthology.orchestra.run.smithi143.stdout: "name": "rwoption2", 2023-11-26T07:23:10.797 INFO:teuthology.orchestra.run.smithi143.stdout: "type": "int", 2023-11-26T07:23:10.797 INFO:teuthology.orchestra.run.smithi143.stdout: "level": "advanced", 2023-11-26T07:23:10.797 INFO:teuthology.orchestra.run.smithi143.stdout: "flags": 0, 2023-11-26T07:23:10.797 INFO:teuthology.orchestra.run.smithi143.stdout: "default_value": "", 2023-11-26T07:23:10.797 INFO:teuthology.orchestra.run.smithi143.stdout: "min": "", 2023-11-26T07:23:10.797 INFO:teuthology.orchestra.run.smithi143.stdout: "max": "", 2023-11-26T07:23:10.798 INFO:teuthology.orchestra.run.smithi143.stdout: "enum_allowed": [], 2023-11-26T07:23:10.798 INFO:teuthology.orchestra.run.smithi143.stdout: "desc": "", 2023-11-26T07:23:10.798 INFO:teuthology.orchestra.run.smithi143.stdout: "long_desc": "", 2023-11-26T07:23:10.798 INFO:teuthology.orchestra.run.smithi143.stdout: "tags": [], 2023-11-26T07:23:10.798 INFO:teuthology.orchestra.run.smithi143.stdout: "see_also": [] 2023-11-26T07:23:10.798 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2023-11-26T07:23:10.798 INFO:teuthology.orchestra.run.smithi143.stdout: "rwoption3": { 2023-11-26T07:23:10.798 INFO:teuthology.orchestra.run.smithi143.stdout: "name": "rwoption3", 2023-11-26T07:23:10.798 INFO:teuthology.orchestra.run.smithi143.stdout: "type": "float", 2023-11-26T07:23:10.798 INFO:teuthology.orchestra.run.smithi143.stdout: "level": "advanced", 2023-11-26T07:23:10.798 INFO:teuthology.orchestra.run.smithi143.stdout: "flags": 0, 2023-11-26T07:23:10.798 INFO:teuthology.orchestra.run.smithi143.stdout: "default_value": "", 2023-11-26T07:23:10.798 INFO:teuthology.orchestra.run.smithi143.stdout: "min": "", 2023-11-26T07:23:10.799 INFO:teuthology.orchestra.run.smithi143.stdout: "max": "", 2023-11-26T07:23:10.799 INFO:teuthology.orchestra.run.smithi143.stdout: "enum_allowed": [], 2023-11-26T07:23:10.799 INFO:teuthology.orchestra.run.smithi143.stdout: "desc": "", 2023-11-26T07:23:10.799 INFO:teuthology.orchestra.run.smithi143.stdout: "long_desc": "", 2023-11-26T07:23:10.799 INFO:teuthology.orchestra.run.smithi143.stdout: "tags": [], 2023-11-26T07:23:10.799 INFO:teuthology.orchestra.run.smithi143.stdout: "see_also": [] 2023-11-26T07:23:10.799 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2023-11-26T07:23:10.799 INFO:teuthology.orchestra.run.smithi143.stdout: "rwoption4": { 2023-11-26T07:23:10.799 INFO:teuthology.orchestra.run.smithi143.stdout: "name": "rwoption4", 2023-11-26T07:23:10.799 INFO:teuthology.orchestra.run.smithi143.stdout: "type": "str", 2023-11-26T07:23:10.799 INFO:teuthology.orchestra.run.smithi143.stdout: "level": "advanced", 2023-11-26T07:23:10.799 INFO:teuthology.orchestra.run.smithi143.stdout: "flags": 0, 2023-11-26T07:23:10.800 INFO:teuthology.orchestra.run.smithi143.stdout: "default_value": "", 2023-11-26T07:23:10.800 INFO:teuthology.orchestra.run.smithi143.stdout: "min": "", 2023-11-26T07:23:10.800 INFO:teuthology.orchestra.run.smithi143.stdout: "max": "", 2023-11-26T07:23:10.800 INFO:teuthology.orchestra.run.smithi143.stdout: "enum_allowed": [], 2023-11-26T07:23:10.800 INFO:teuthology.orchestra.run.smithi143.stdout: "desc": "", 2023-11-26T07:23:10.800 INFO:teuthology.orchestra.run.smithi143.stdout: "long_desc": "", 2023-11-26T07:23:10.800 INFO:teuthology.orchestra.run.smithi143.stdout: "tags": [], 2023-11-26T07:23:10.800 INFO:teuthology.orchestra.run.smithi143.stdout: "see_also": [] 2023-11-26T07:23:10.800 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2023-11-26T07:23:10.800 INFO:teuthology.orchestra.run.smithi143.stdout: "rwoption5": { 2023-11-26T07:23:10.800 INFO:teuthology.orchestra.run.smithi143.stdout: "name": "rwoption5", 2023-11-26T07:23:10.800 INFO:teuthology.orchestra.run.smithi143.stdout: "type": "bool", 2023-11-26T07:23:10.800 INFO:teuthology.orchestra.run.smithi143.stdout: "level": "advanced", 2023-11-26T07:23:10.800 INFO:teuthology.orchestra.run.smithi143.stdout: "flags": 0, 2023-11-26T07:23:10.801 INFO:teuthology.orchestra.run.smithi143.stdout: "default_value": "", 2023-11-26T07:23:10.801 INFO:teuthology.orchestra.run.smithi143.stdout: "min": "", 2023-11-26T07:23:10.801 INFO:teuthology.orchestra.run.smithi143.stdout: "max": "", 2023-11-26T07:23:10.801 INFO:teuthology.orchestra.run.smithi143.stdout: "enum_allowed": [], 2023-11-26T07:23:10.801 INFO:teuthology.orchestra.run.smithi143.stdout: "desc": "", 2023-11-26T07:23:10.801 INFO:teuthology.orchestra.run.smithi143.stdout: "long_desc": "", 2023-11-26T07:23:10.801 INFO:teuthology.orchestra.run.smithi143.stdout: "tags": [], 2023-11-26T07:23:10.801 INFO:teuthology.orchestra.run.smithi143.stdout: "see_also": [] 2023-11-26T07:23:10.801 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2023-11-26T07:23:10.801 INFO:teuthology.orchestra.run.smithi143.stdout: "rwoption6": { 2023-11-26T07:23:10.801 INFO:teuthology.orchestra.run.smithi143.stdout: "name": "rwoption6", 2023-11-26T07:23:10.801 INFO:teuthology.orchestra.run.smithi143.stdout: "type": "bool", 2023-11-26T07:23:10.801 INFO:teuthology.orchestra.run.smithi143.stdout: "level": "advanced", 2023-11-26T07:23:10.802 INFO:teuthology.orchestra.run.smithi143.stdout: "flags": 0, 2023-11-26T07:23:10.802 INFO:teuthology.orchestra.run.smithi143.stdout: "default_value": "True", 2023-11-26T07:23:10.802 INFO:teuthology.orchestra.run.smithi143.stdout: "min": "", 2023-11-26T07:23:10.802 INFO:teuthology.orchestra.run.smithi143.stdout: "max": "", 2023-11-26T07:23:10.802 INFO:teuthology.orchestra.run.smithi143.stdout: "enum_allowed": [], 2023-11-26T07:23:10.802 INFO:teuthology.orchestra.run.smithi143.stdout: "desc": "", 2023-11-26T07:23:10.802 INFO:teuthology.orchestra.run.smithi143.stdout: "long_desc": "", 2023-11-26T07:23:10.802 INFO:teuthology.orchestra.run.smithi143.stdout: "tags": [], 2023-11-26T07:23:10.802 INFO:teuthology.orchestra.run.smithi143.stdout: "see_also": [] 2023-11-26T07:23:10.802 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2023-11-26T07:23:10.802 INFO:teuthology.orchestra.run.smithi143.stdout: "rwoption7": { 2023-11-26T07:23:10.802 INFO:teuthology.orchestra.run.smithi143.stdout: "name": "rwoption7", 2023-11-26T07:23:10.803 INFO:teuthology.orchestra.run.smithi143.stdout: "type": "int", 2023-11-26T07:23:10.803 INFO:teuthology.orchestra.run.smithi143.stdout: "level": "advanced", 2023-11-26T07:23:10.803 INFO:teuthology.orchestra.run.smithi143.stdout: "flags": 0, 2023-11-26T07:23:10.803 INFO:teuthology.orchestra.run.smithi143.stdout: "default_value": "", 2023-11-26T07:23:10.803 INFO:teuthology.orchestra.run.smithi143.stdout: "min": "1", 2023-11-26T07:23:10.803 INFO:teuthology.orchestra.run.smithi143.stdout: "max": "42", 2023-11-26T07:23:10.803 INFO:teuthology.orchestra.run.smithi143.stdout: "enum_allowed": [], 2023-11-26T07:23:10.803 INFO:teuthology.orchestra.run.smithi143.stdout: "desc": "", 2023-11-26T07:23:10.803 INFO:teuthology.orchestra.run.smithi143.stdout: "long_desc": "", 2023-11-26T07:23:10.803 INFO:teuthology.orchestra.run.smithi143.stdout: "tags": [], 2023-11-26T07:23:10.803 INFO:teuthology.orchestra.run.smithi143.stdout: "see_also": [] 2023-11-26T07:23:10.803 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2023-11-26T07:23:10.803 INFO:teuthology.orchestra.run.smithi143.stdout: "testkey": { 2023-11-26T07:23:10.804 INFO:teuthology.orchestra.run.smithi143.stdout: "name": "testkey", 2023-11-26T07:23:10.804 INFO:teuthology.orchestra.run.smithi143.stdout: "type": "str", 2023-11-26T07:23:10.804 INFO:teuthology.orchestra.run.smithi143.stdout: "level": "advanced", 2023-11-26T07:23:10.804 INFO:teuthology.orchestra.run.smithi143.stdout: "flags": 0, 2023-11-26T07:23:10.804 INFO:teuthology.orchestra.run.smithi143.stdout: "default_value": "", 2023-11-26T07:23:10.804 INFO:teuthology.orchestra.run.smithi143.stdout: "min": "", 2023-11-26T07:23:10.804 INFO:teuthology.orchestra.run.smithi143.stdout: "max": "", 2023-11-26T07:23:10.804 INFO:teuthology.orchestra.run.smithi143.stdout: "enum_allowed": [], 2023-11-26T07:23:10.804 INFO:teuthology.orchestra.run.smithi143.stdout: "desc": "", 2023-11-26T07:23:10.804 INFO:teuthology.orchestra.run.smithi143.stdout: "long_desc": "", 2023-11-26T07:23:10.804 INFO:teuthology.orchestra.run.smithi143.stdout: "tags": [], 2023-11-26T07:23:10.804 INFO:teuthology.orchestra.run.smithi143.stdout: "see_also": [] 2023-11-26T07:23:10.804 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2023-11-26T07:23:10.805 INFO:teuthology.orchestra.run.smithi143.stdout: "testlkey": { 2023-11-26T07:23:10.805 INFO:teuthology.orchestra.run.smithi143.stdout: "name": "testlkey", 2023-11-26T07:23:10.805 INFO:teuthology.orchestra.run.smithi143.stdout: "type": "str", 2023-11-26T07:23:10.805 INFO:teuthology.orchestra.run.smithi143.stdout: "level": "advanced", 2023-11-26T07:23:10.805 INFO:teuthology.orchestra.run.smithi143.stdout: "flags": 0, 2023-11-26T07:23:10.805 INFO:teuthology.orchestra.run.smithi143.stdout: "default_value": "", 2023-11-26T07:23:10.805 INFO:teuthology.orchestra.run.smithi143.stdout: "min": "", 2023-11-26T07:23:10.805 INFO:teuthology.orchestra.run.smithi143.stdout: "max": "", 2023-11-26T07:23:10.805 INFO:teuthology.orchestra.run.smithi143.stdout: "enum_allowed": [], 2023-11-26T07:23:10.805 INFO:teuthology.orchestra.run.smithi143.stdout: "desc": "", 2023-11-26T07:23:10.805 INFO:teuthology.orchestra.run.smithi143.stdout: "long_desc": "", 2023-11-26T07:23:10.805 INFO:teuthology.orchestra.run.smithi143.stdout: "tags": [], 2023-11-26T07:23:10.806 INFO:teuthology.orchestra.run.smithi143.stdout: "see_also": [] 2023-11-26T07:23:10.806 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2023-11-26T07:23:10.806 INFO:teuthology.orchestra.run.smithi143.stdout: "testnewline": { 2023-11-26T07:23:10.806 INFO:teuthology.orchestra.run.smithi143.stdout: "name": "testnewline", 2023-11-26T07:23:10.806 INFO:teuthology.orchestra.run.smithi143.stdout: "type": "str", 2023-11-26T07:23:10.806 INFO:teuthology.orchestra.run.smithi143.stdout: "level": "advanced", 2023-11-26T07:23:10.806 INFO:teuthology.orchestra.run.smithi143.stdout: "flags": 0, 2023-11-26T07:23:10.806 INFO:teuthology.orchestra.run.smithi143.stdout: "default_value": "", 2023-11-26T07:23:10.806 INFO:teuthology.orchestra.run.smithi143.stdout: "min": "", 2023-11-26T07:23:10.806 INFO:teuthology.orchestra.run.smithi143.stdout: "max": "", 2023-11-26T07:23:10.806 INFO:teuthology.orchestra.run.smithi143.stdout: "enum_allowed": [], 2023-11-26T07:23:10.806 INFO:teuthology.orchestra.run.smithi143.stdout: "desc": "", 2023-11-26T07:23:10.807 INFO:teuthology.orchestra.run.smithi143.stdout: "long_desc": "", 2023-11-26T07:23:10.807 INFO:teuthology.orchestra.run.smithi143.stdout: "tags": [], 2023-11-26T07:23:10.807 INFO:teuthology.orchestra.run.smithi143.stdout: "see_also": [] 2023-11-26T07:23:10.807 INFO:teuthology.orchestra.run.smithi143.stdout: } 2023-11-26T07:23:10.807 INFO:teuthology.orchestra.run.smithi143.stdout: } 2023-11-26T07:23:10.807 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2023-11-26T07:23:10.807 INFO:teuthology.orchestra.run.smithi143.stdout: { 2023-11-26T07:23:10.807 INFO:teuthology.orchestra.run.smithi143.stdout: "name": "snap_schedule", 2023-11-26T07:23:10.807 INFO:teuthology.orchestra.run.smithi143.stdout: "can_run": true, 2023-11-26T07:23:10.807 INFO:teuthology.orchestra.run.smithi143.stdout: "error_string": "", 2023-11-26T07:23:10.807 INFO:teuthology.orchestra.run.smithi143.stdout: "module_options": { 2023-11-26T07:23:10.807 INFO:teuthology.orchestra.run.smithi143.stdout: "allow_m_granularity": { 2023-11-26T07:23:10.807 INFO:teuthology.orchestra.run.smithi143.stdout: "name": "allow_m_granularity", 2023-11-26T07:23:10.808 INFO:teuthology.orchestra.run.smithi143.stdout: "type": "bool", 2023-11-26T07:23:10.808 INFO:teuthology.orchestra.run.smithi143.stdout: "level": "advanced", 2023-11-26T07:23:10.808 INFO:teuthology.orchestra.run.smithi143.stdout: "flags": 1, 2023-11-26T07:23:10.808 INFO:teuthology.orchestra.run.smithi143.stdout: "default_value": "False", 2023-11-26T07:23:10.808 INFO:teuthology.orchestra.run.smithi143.stdout: "min": "", 2023-11-26T07:23:10.808 INFO:teuthology.orchestra.run.smithi143.stdout: "max": "", 2023-11-26T07:23:10.808 INFO:teuthology.orchestra.run.smithi143.stdout: "enum_allowed": [], 2023-11-26T07:23:10.808 INFO:teuthology.orchestra.run.smithi143.stdout: "desc": "allow minute scheduled snapshots", 2023-11-26T07:23:10.808 INFO:teuthology.orchestra.run.smithi143.stdout: "long_desc": "", 2023-11-26T07:23:10.808 INFO:teuthology.orchestra.run.smithi143.stdout: "tags": [], 2023-11-26T07:23:10.809 INFO:teuthology.orchestra.run.smithi143.stdout: "see_also": [] 2023-11-26T07:23:10.809 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2023-11-26T07:23:10.809 INFO:teuthology.orchestra.run.smithi143.stdout: "dump_on_update": { 2023-11-26T07:23:10.809 INFO:teuthology.orchestra.run.smithi143.stdout: "name": "dump_on_update", 2023-11-26T07:23:10.809 INFO:teuthology.orchestra.run.smithi143.stdout: "type": "bool", 2023-11-26T07:23:10.809 INFO:teuthology.orchestra.run.smithi143.stdout: "level": "advanced", 2023-11-26T07:23:10.809 INFO:teuthology.orchestra.run.smithi143.stdout: "flags": 1, 2023-11-26T07:23:10.809 INFO:teuthology.orchestra.run.smithi143.stdout: "default_value": "False", 2023-11-26T07:23:10.809 INFO:teuthology.orchestra.run.smithi143.stdout: "min": "", 2023-11-26T07:23:10.809 INFO:teuthology.orchestra.run.smithi143.stdout: "max": "", 2023-11-26T07:23:10.809 INFO:teuthology.orchestra.run.smithi143.stdout: "enum_allowed": [], 2023-11-26T07:23:10.809 INFO:teuthology.orchestra.run.smithi143.stdout: "desc": "dump database to debug log on update", 2023-11-26T07:23:10.810 INFO:teuthology.orchestra.run.smithi143.stdout: "long_desc": "", 2023-11-26T07:23:10.810 INFO:teuthology.orchestra.run.smithi143.stdout: "tags": [], 2023-11-26T07:23:10.810 INFO:teuthology.orchestra.run.smithi143.stdout: "see_also": [] 2023-11-26T07:23:10.810 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2023-11-26T07:23:10.810 INFO:teuthology.orchestra.run.smithi143.stdout: "log_level": { 2023-11-26T07:23:10.810 INFO:teuthology.orchestra.run.smithi143.stdout: "name": "log_level", 2023-11-26T07:23:10.810 INFO:teuthology.orchestra.run.smithi143.stdout: "type": "str", 2023-11-26T07:23:10.810 INFO:teuthology.orchestra.run.smithi143.stdout: "level": "advanced", 2023-11-26T07:23:10.810 INFO:teuthology.orchestra.run.smithi143.stdout: "flags": 1, 2023-11-26T07:23:10.810 INFO:teuthology.orchestra.run.smithi143.stdout: "default_value": "", 2023-11-26T07:23:10.810 INFO:teuthology.orchestra.run.smithi143.stdout: "min": "", 2023-11-26T07:23:10.811 INFO:teuthology.orchestra.run.smithi143.stdout: "max": "", 2023-11-26T07:23:10.811 INFO:teuthology.orchestra.run.smithi143.stdout: "enum_allowed": [ 2023-11-26T07:23:10.811 INFO:teuthology.orchestra.run.smithi143.stdout: "", 2023-11-26T07:23:10.811 INFO:teuthology.orchestra.run.smithi143.stdout: "critical", 2023-11-26T07:23:10.811 INFO:teuthology.orchestra.run.smithi143.stdout: "debug", 2023-11-26T07:23:10.811 INFO:teuthology.orchestra.run.smithi143.stdout: "error", 2023-11-26T07:23:10.811 INFO:teuthology.orchestra.run.smithi143.stdout: "info", 2023-11-26T07:23:10.811 INFO:teuthology.orchestra.run.smithi143.stdout: "warning" 2023-11-26T07:23:10.811 INFO:teuthology.orchestra.run.smithi143.stdout: ], 2023-11-26T07:23:10.811 INFO:teuthology.orchestra.run.smithi143.stdout: "desc": "", 2023-11-26T07:23:10.811 INFO:teuthology.orchestra.run.smithi143.stdout: "long_desc": "", 2023-11-26T07:23:10.811 INFO:teuthology.orchestra.run.smithi143.stdout: "tags": [], 2023-11-26T07:23:10.812 INFO:teuthology.orchestra.run.smithi143.stdout: "see_also": [] 2023-11-26T07:23:10.812 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2023-11-26T07:23:10.812 INFO:teuthology.orchestra.run.smithi143.stdout: "log_to_cluster": { 2023-11-26T07:23:10.812 INFO:teuthology.orchestra.run.smithi143.stdout: "name": "log_to_cluster", 2023-11-26T07:23:10.812 INFO:teuthology.orchestra.run.smithi143.stdout: "type": "bool", 2023-11-26T07:23:10.812 INFO:teuthology.orchestra.run.smithi143.stdout: "level": "advanced", 2023-11-26T07:23:10.812 INFO:teuthology.orchestra.run.smithi143.stdout: "flags": 1, 2023-11-26T07:23:10.812 INFO:teuthology.orchestra.run.smithi143.stdout: "default_value": "False", 2023-11-26T07:23:10.812 INFO:teuthology.orchestra.run.smithi143.stdout: "min": "", 2023-11-26T07:23:10.812 INFO:teuthology.orchestra.run.smithi143.stdout: "max": "", 2023-11-26T07:23:10.812 INFO:teuthology.orchestra.run.smithi143.stdout: "enum_allowed": [], 2023-11-26T07:23:10.812 INFO:teuthology.orchestra.run.smithi143.stdout: "desc": "", 2023-11-26T07:23:10.812 INFO:teuthology.orchestra.run.smithi143.stdout: "long_desc": "", 2023-11-26T07:23:10.813 INFO:teuthology.orchestra.run.smithi143.stdout: "tags": [], 2023-11-26T07:23:10.813 INFO:teuthology.orchestra.run.smithi143.stdout: "see_also": [] 2023-11-26T07:23:10.813 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2023-11-26T07:23:10.813 INFO:teuthology.orchestra.run.smithi143.stdout: "log_to_cluster_level": { 2023-11-26T07:23:10.813 INFO:teuthology.orchestra.run.smithi143.stdout: "name": "log_to_cluster_level", 2023-11-26T07:23:10.813 INFO:teuthology.orchestra.run.smithi143.stdout: "type": "str", 2023-11-26T07:23:10.813 INFO:teuthology.orchestra.run.smithi143.stdout: "level": "advanced", 2023-11-26T07:23:10.813 INFO:teuthology.orchestra.run.smithi143.stdout: "flags": 1, 2023-11-26T07:23:10.813 INFO:teuthology.orchestra.run.smithi143.stdout: "default_value": "info", 2023-11-26T07:23:10.813 INFO:teuthology.orchestra.run.smithi143.stdout: "min": "", 2023-11-26T07:23:10.813 INFO:teuthology.orchestra.run.smithi143.stdout: "max": "", 2023-11-26T07:23:10.813 INFO:teuthology.orchestra.run.smithi143.stdout: "enum_allowed": [ 2023-11-26T07:23:10.813 INFO:teuthology.orchestra.run.smithi143.stdout: "", 2023-11-26T07:23:10.814 INFO:teuthology.orchestra.run.smithi143.stdout: "critical", 2023-11-26T07:23:10.814 INFO:teuthology.orchestra.run.smithi143.stdout: "debug", 2023-11-26T07:23:10.814 INFO:teuthology.orchestra.run.smithi143.stdout: "error", 2023-11-26T07:23:10.814 INFO:teuthology.orchestra.run.smithi143.stdout: "info", 2023-11-26T07:23:10.814 INFO:teuthology.orchestra.run.smithi143.stdout: "warning" 2023-11-26T07:23:10.814 INFO:teuthology.orchestra.run.smithi143.stdout: ], 2023-11-26T07:23:10.814 INFO:teuthology.orchestra.run.smithi143.stdout: "desc": "", 2023-11-26T07:23:10.814 INFO:teuthology.orchestra.run.smithi143.stdout: "long_desc": "", 2023-11-26T07:23:10.814 INFO:teuthology.orchestra.run.smithi143.stdout: "tags": [], 2023-11-26T07:23:10.814 INFO:teuthology.orchestra.run.smithi143.stdout: "see_also": [] 2023-11-26T07:23:10.814 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2023-11-26T07:23:10.815 INFO:teuthology.orchestra.run.smithi143.stdout: "log_to_file": { 2023-11-26T07:23:10.815 INFO:teuthology.orchestra.run.smithi143.stdout: "name": "log_to_file", 2023-11-26T07:23:10.815 INFO:teuthology.orchestra.run.smithi143.stdout: "type": "bool", 2023-11-26T07:23:10.815 INFO:teuthology.orchestra.run.smithi143.stdout: "level": "advanced", 2023-11-26T07:23:10.815 INFO:teuthology.orchestra.run.smithi143.stdout: "flags": 1, 2023-11-26T07:23:10.815 INFO:teuthology.orchestra.run.smithi143.stdout: "default_value": "False", 2023-11-26T07:23:10.815 INFO:teuthology.orchestra.run.smithi143.stdout: "min": "", 2023-11-26T07:23:10.815 INFO:teuthology.orchestra.run.smithi143.stdout: "max": "", 2023-11-26T07:23:10.815 INFO:teuthology.orchestra.run.smithi143.stdout: "enum_allowed": [], 2023-11-26T07:23:10.815 INFO:teuthology.orchestra.run.smithi143.stdout: "desc": "", 2023-11-26T07:23:10.815 INFO:teuthology.orchestra.run.smithi143.stdout: "long_desc": "", 2023-11-26T07:23:10.815 INFO:teuthology.orchestra.run.smithi143.stdout: "tags": [], 2023-11-26T07:23:10.815 INFO:teuthology.orchestra.run.smithi143.stdout: "see_also": [] 2023-11-26T07:23:10.816 INFO:teuthology.orchestra.run.smithi143.stdout: } 2023-11-26T07:23:10.816 INFO:teuthology.orchestra.run.smithi143.stdout: } 2023-11-26T07:23:10.816 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2023-11-26T07:23:10.816 INFO:teuthology.orchestra.run.smithi143.stdout: { 2023-11-26T07:23:10.816 INFO:teuthology.orchestra.run.smithi143.stdout: "name": "stats", 2023-11-26T07:23:10.816 INFO:teuthology.orchestra.run.smithi143.stdout: "can_run": true, 2023-11-26T07:23:10.816 INFO:teuthology.orchestra.run.smithi143.stdout: "error_string": "", 2023-11-26T07:23:10.816 INFO:teuthology.orchestra.run.smithi143.stdout: "module_options": { 2023-11-26T07:23:10.816 INFO:teuthology.orchestra.run.smithi143.stdout: "log_level": { 2023-11-26T07:23:10.816 INFO:teuthology.orchestra.run.smithi143.stdout: "name": "log_level", 2023-11-26T07:23:10.816 INFO:teuthology.orchestra.run.smithi143.stdout: "type": "str", 2023-11-26T07:23:10.817 INFO:teuthology.orchestra.run.smithi143.stdout: "level": "advanced", 2023-11-26T07:23:10.817 INFO:teuthology.orchestra.run.smithi143.stdout: "flags": 1, 2023-11-26T07:23:10.817 INFO:teuthology.orchestra.run.smithi143.stdout: "default_value": "", 2023-11-26T07:23:10.817 INFO:teuthology.orchestra.run.smithi143.stdout: "min": "", 2023-11-26T07:23:10.817 INFO:teuthology.orchestra.run.smithi143.stdout: "max": "", 2023-11-26T07:23:10.817 INFO:teuthology.orchestra.run.smithi143.stdout: "enum_allowed": [ 2023-11-26T07:23:10.817 INFO:teuthology.orchestra.run.smithi143.stdout: "", 2023-11-26T07:23:10.817 INFO:teuthology.orchestra.run.smithi143.stdout: "critical", 2023-11-26T07:23:10.817 INFO:teuthology.orchestra.run.smithi143.stdout: "debug", 2023-11-26T07:23:10.817 INFO:teuthology.orchestra.run.smithi143.stdout: "error", 2023-11-26T07:23:10.817 INFO:teuthology.orchestra.run.smithi143.stdout: "info", 2023-11-26T07:23:10.817 INFO:teuthology.orchestra.run.smithi143.stdout: "warning" 2023-11-26T07:23:10.818 INFO:teuthology.orchestra.run.smithi143.stdout: ], 2023-11-26T07:23:10.818 INFO:teuthology.orchestra.run.smithi143.stdout: "desc": "", 2023-11-26T07:23:10.818 INFO:teuthology.orchestra.run.smithi143.stdout: "long_desc": "", 2023-11-26T07:23:10.818 INFO:teuthology.orchestra.run.smithi143.stdout: "tags": [], 2023-11-26T07:23:10.818 INFO:teuthology.orchestra.run.smithi143.stdout: "see_also": [] 2023-11-26T07:23:10.818 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2023-11-26T07:23:10.818 INFO:teuthology.orchestra.run.smithi143.stdout: "log_to_cluster": { 2023-11-26T07:23:10.818 INFO:teuthology.orchestra.run.smithi143.stdout: "name": "log_to_cluster", 2023-11-26T07:23:10.818 INFO:teuthology.orchestra.run.smithi143.stdout: "type": "bool", 2023-11-26T07:23:10.818 INFO:teuthology.orchestra.run.smithi143.stdout: "level": "advanced", 2023-11-26T07:23:10.818 INFO:teuthology.orchestra.run.smithi143.stdout: "flags": 1, 2023-11-26T07:23:10.819 INFO:teuthology.orchestra.run.smithi143.stdout: "default_value": "False", 2023-11-26T07:23:10.819 INFO:teuthology.orchestra.run.smithi143.stdout: "min": "", 2023-11-26T07:23:10.819 INFO:teuthology.orchestra.run.smithi143.stdout: "max": "", 2023-11-26T07:23:10.819 INFO:teuthology.orchestra.run.smithi143.stdout: "enum_allowed": [], 2023-11-26T07:23:10.819 INFO:teuthology.orchestra.run.smithi143.stdout: "desc": "", 2023-11-26T07:23:10.819 INFO:teuthology.orchestra.run.smithi143.stdout: "long_desc": "", 2023-11-26T07:23:10.819 INFO:teuthology.orchestra.run.smithi143.stdout: "tags": [], 2023-11-26T07:23:10.819 INFO:teuthology.orchestra.run.smithi143.stdout: "see_also": [] 2023-11-26T07:23:10.819 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2023-11-26T07:23:10.819 INFO:teuthology.orchestra.run.smithi143.stdout: "log_to_cluster_level": { 2023-11-26T07:23:10.819 INFO:teuthology.orchestra.run.smithi143.stdout: "name": "log_to_cluster_level", 2023-11-26T07:23:10.819 INFO:teuthology.orchestra.run.smithi143.stdout: "type": "str", 2023-11-26T07:23:10.819 INFO:teuthology.orchestra.run.smithi143.stdout: "level": "advanced", 2023-11-26T07:23:10.821 INFO:teuthology.orchestra.run.smithi143.stdout: "flags": 1, 2023-11-26T07:23:10.821 INFO:teuthology.orchestra.run.smithi143.stdout: "default_value": "info", 2023-11-26T07:23:10.821 INFO:teuthology.orchestra.run.smithi143.stdout: "min": "", 2023-11-26T07:23:10.821 INFO:teuthology.orchestra.run.smithi143.stdout: "max": "", 2023-11-26T07:23:10.821 INFO:teuthology.orchestra.run.smithi143.stdout: "enum_allowed": [ 2023-11-26T07:23:10.821 INFO:teuthology.orchestra.run.smithi143.stdout: "", 2023-11-26T07:23:10.821 INFO:teuthology.orchestra.run.smithi143.stdout: "critical", 2023-11-26T07:23:10.821 INFO:teuthology.orchestra.run.smithi143.stdout: "debug", 2023-11-26T07:23:10.822 INFO:teuthology.orchestra.run.smithi143.stdout: "error", 2023-11-26T07:23:10.822 INFO:teuthology.orchestra.run.smithi143.stdout: "info", 2023-11-26T07:23:10.822 INFO:teuthology.orchestra.run.smithi143.stdout: "warning" 2023-11-26T07:23:10.822 INFO:teuthology.orchestra.run.smithi143.stdout: ], 2023-11-26T07:23:10.822 INFO:teuthology.orchestra.run.smithi143.stdout: "desc": "", 2023-11-26T07:23:10.822 INFO:teuthology.orchestra.run.smithi143.stdout: "long_desc": "", 2023-11-26T07:23:10.822 INFO:teuthology.orchestra.run.smithi143.stdout: "tags": [], 2023-11-26T07:23:10.822 INFO:teuthology.orchestra.run.smithi143.stdout: "see_also": [] 2023-11-26T07:23:10.822 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2023-11-26T07:23:10.822 INFO:teuthology.orchestra.run.smithi143.stdout: "log_to_file": { 2023-11-26T07:23:10.822 INFO:teuthology.orchestra.run.smithi143.stdout: "name": "log_to_file", 2023-11-26T07:23:10.822 INFO:teuthology.orchestra.run.smithi143.stdout: "type": "bool", 2023-11-26T07:23:10.823 INFO:teuthology.orchestra.run.smithi143.stdout: "level": "advanced", 2023-11-26T07:23:10.823 INFO:teuthology.orchestra.run.smithi143.stdout: "flags": 1, 2023-11-26T07:23:10.823 INFO:teuthology.orchestra.run.smithi143.stdout: "default_value": "False", 2023-11-26T07:23:10.823 INFO:teuthology.orchestra.run.smithi143.stdout: "min": "", 2023-11-26T07:23:10.823 INFO:teuthology.orchestra.run.smithi143.stdout: "max": "", 2023-11-26T07:23:10.823 INFO:teuthology.orchestra.run.smithi143.stdout: "enum_allowed": [], 2023-11-26T07:23:10.823 INFO:teuthology.orchestra.run.smithi143.stdout: "desc": "", 2023-11-26T07:23:10.823 INFO:teuthology.orchestra.run.smithi143.stdout: "long_desc": "", 2023-11-26T07:23:10.823 INFO:teuthology.orchestra.run.smithi143.stdout: "tags": [], 2023-11-26T07:23:10.823 INFO:teuthology.orchestra.run.smithi143.stdout: "see_also": [] 2023-11-26T07:23:10.823 INFO:teuthology.orchestra.run.smithi143.stdout: } 2023-11-26T07:23:10.823 INFO:teuthology.orchestra.run.smithi143.stdout: } 2023-11-26T07:23:10.823 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2023-11-26T07:23:10.824 INFO:teuthology.orchestra.run.smithi143.stdout: { 2023-11-26T07:23:10.824 INFO:teuthology.orchestra.run.smithi143.stdout: "name": "status", 2023-11-26T07:23:10.824 INFO:teuthology.orchestra.run.smithi143.stdout: "can_run": true, 2023-11-26T07:23:10.824 INFO:teuthology.orchestra.run.smithi143.stdout: "error_string": "", 2023-11-26T07:23:10.824 INFO:teuthology.orchestra.run.smithi143.stdout: "module_options": { 2023-11-26T07:23:10.824 INFO:teuthology.orchestra.run.smithi143.stdout: "log_level": { 2023-11-26T07:23:10.824 INFO:teuthology.orchestra.run.smithi143.stdout: "name": "log_level", 2023-11-26T07:23:10.824 INFO:teuthology.orchestra.run.smithi143.stdout: "type": "str", 2023-11-26T07:23:10.824 INFO:teuthology.orchestra.run.smithi143.stdout: "level": "advanced", 2023-11-26T07:23:10.824 INFO:teuthology.orchestra.run.smithi143.stdout: "flags": 1, 2023-11-26T07:23:10.824 INFO:teuthology.orchestra.run.smithi143.stdout: "default_value": "", 2023-11-26T07:23:10.824 INFO:teuthology.orchestra.run.smithi143.stdout: "min": "", 2023-11-26T07:23:10.824 INFO:teuthology.orchestra.run.smithi143.stdout: "max": "", 2023-11-26T07:23:10.825 INFO:teuthology.orchestra.run.smithi143.stdout: "enum_allowed": [ 2023-11-26T07:23:10.825 INFO:teuthology.orchestra.run.smithi143.stdout: "", 2023-11-26T07:23:10.825 INFO:teuthology.orchestra.run.smithi143.stdout: "critical", 2023-11-26T07:23:10.825 INFO:teuthology.orchestra.run.smithi143.stdout: "debug", 2023-11-26T07:23:10.825 INFO:teuthology.orchestra.run.smithi143.stdout: "error", 2023-11-26T07:23:10.825 INFO:teuthology.orchestra.run.smithi143.stdout: "info", 2023-11-26T07:23:10.825 INFO:teuthology.orchestra.run.smithi143.stdout: "warning" 2023-11-26T07:23:10.825 INFO:teuthology.orchestra.run.smithi143.stdout: ], 2023-11-26T07:23:10.825 INFO:teuthology.orchestra.run.smithi143.stdout: "desc": "", 2023-11-26T07:23:10.825 INFO:teuthology.orchestra.run.smithi143.stdout: "long_desc": "", 2023-11-26T07:23:10.825 INFO:teuthology.orchestra.run.smithi143.stdout: "tags": [], 2023-11-26T07:23:10.825 INFO:teuthology.orchestra.run.smithi143.stdout: "see_also": [] 2023-11-26T07:23:10.825 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2023-11-26T07:23:10.826 INFO:teuthology.orchestra.run.smithi143.stdout: "log_to_cluster": { 2023-11-26T07:23:10.826 INFO:teuthology.orchestra.run.smithi143.stdout: "name": "log_to_cluster", 2023-11-26T07:23:10.826 INFO:teuthology.orchestra.run.smithi143.stdout: "type": "bool", 2023-11-26T07:23:10.826 INFO:teuthology.orchestra.run.smithi143.stdout: "level": "advanced", 2023-11-26T07:23:10.826 INFO:teuthology.orchestra.run.smithi143.stdout: "flags": 1, 2023-11-26T07:23:10.826 INFO:teuthology.orchestra.run.smithi143.stdout: "default_value": "False", 2023-11-26T07:23:10.826 INFO:teuthology.orchestra.run.smithi143.stdout: "min": "", 2023-11-26T07:23:10.826 INFO:teuthology.orchestra.run.smithi143.stdout: "max": "", 2023-11-26T07:23:10.826 INFO:teuthology.orchestra.run.smithi143.stdout: "enum_allowed": [], 2023-11-26T07:23:10.826 INFO:teuthology.orchestra.run.smithi143.stdout: "desc": "", 2023-11-26T07:23:10.826 INFO:teuthology.orchestra.run.smithi143.stdout: "long_desc": "", 2023-11-26T07:23:10.826 INFO:teuthology.orchestra.run.smithi143.stdout: "tags": [], 2023-11-26T07:23:10.827 INFO:teuthology.orchestra.run.smithi143.stdout: "see_also": [] 2023-11-26T07:23:10.827 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2023-11-26T07:23:10.827 INFO:teuthology.orchestra.run.smithi143.stdout: "log_to_cluster_level": { 2023-11-26T07:23:10.827 INFO:teuthology.orchestra.run.smithi143.stdout: "name": "log_to_cluster_level", 2023-11-26T07:23:10.827 INFO:teuthology.orchestra.run.smithi143.stdout: "type": "str", 2023-11-26T07:23:10.827 INFO:teuthology.orchestra.run.smithi143.stdout: "level": "advanced", 2023-11-26T07:23:10.827 INFO:teuthology.orchestra.run.smithi143.stdout: "flags": 1, 2023-11-26T07:23:10.827 INFO:teuthology.orchestra.run.smithi143.stdout: "default_value": "info", 2023-11-26T07:23:10.827 INFO:teuthology.orchestra.run.smithi143.stdout: "min": "", 2023-11-26T07:23:10.827 INFO:teuthology.orchestra.run.smithi143.stdout: "max": "", 2023-11-26T07:23:10.827 INFO:teuthology.orchestra.run.smithi143.stdout: "enum_allowed": [ 2023-11-26T07:23:10.827 INFO:teuthology.orchestra.run.smithi143.stdout: "", 2023-11-26T07:23:10.827 INFO:teuthology.orchestra.run.smithi143.stdout: "critical", 2023-11-26T07:23:10.828 INFO:teuthology.orchestra.run.smithi143.stdout: "debug", 2023-11-26T07:23:10.828 INFO:teuthology.orchestra.run.smithi143.stdout: "error", 2023-11-26T07:23:10.828 INFO:teuthology.orchestra.run.smithi143.stdout: "info", 2023-11-26T07:23:10.828 INFO:teuthology.orchestra.run.smithi143.stdout: "warning" 2023-11-26T07:23:10.828 INFO:teuthology.orchestra.run.smithi143.stdout: ], 2023-11-26T07:23:10.828 INFO:teuthology.orchestra.run.smithi143.stdout: "desc": "", 2023-11-26T07:23:10.828 INFO:teuthology.orchestra.run.smithi143.stdout: "long_desc": "", 2023-11-26T07:23:10.828 INFO:teuthology.orchestra.run.smithi143.stdout: "tags": [], 2023-11-26T07:23:10.828 INFO:teuthology.orchestra.run.smithi143.stdout: "see_also": [] 2023-11-26T07:23:10.828 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2023-11-26T07:23:10.828 INFO:teuthology.orchestra.run.smithi143.stdout: "log_to_file": { 2023-11-26T07:23:10.828 INFO:teuthology.orchestra.run.smithi143.stdout: "name": "log_to_file", 2023-11-26T07:23:10.829 INFO:teuthology.orchestra.run.smithi143.stdout: "type": "bool", 2023-11-26T07:23:10.829 INFO:teuthology.orchestra.run.smithi143.stdout: "level": "advanced", 2023-11-26T07:23:10.829 INFO:teuthology.orchestra.run.smithi143.stdout: "flags": 1, 2023-11-26T07:23:10.829 INFO:teuthology.orchestra.run.smithi143.stdout: "default_value": "False", 2023-11-26T07:23:10.829 INFO:teuthology.orchestra.run.smithi143.stdout: "min": "", 2023-11-26T07:23:10.829 INFO:teuthology.orchestra.run.smithi143.stdout: "max": "", 2023-11-26T07:23:10.829 INFO:teuthology.orchestra.run.smithi143.stdout: "enum_allowed": [], 2023-11-26T07:23:10.829 INFO:teuthology.orchestra.run.smithi143.stdout: "desc": "", 2023-11-26T07:23:10.829 INFO:teuthology.orchestra.run.smithi143.stdout: "long_desc": "", 2023-11-26T07:23:10.829 INFO:teuthology.orchestra.run.smithi143.stdout: "tags": [], 2023-11-26T07:23:10.829 INFO:teuthology.orchestra.run.smithi143.stdout: "see_also": [] 2023-11-26T07:23:10.829 INFO:teuthology.orchestra.run.smithi143.stdout: } 2023-11-26T07:23:10.829 INFO:teuthology.orchestra.run.smithi143.stdout: } 2023-11-26T07:23:10.830 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2023-11-26T07:23:10.830 INFO:teuthology.orchestra.run.smithi143.stdout: { 2023-11-26T07:23:10.830 INFO:teuthology.orchestra.run.smithi143.stdout: "name": "telegraf", 2023-11-26T07:23:10.830 INFO:teuthology.orchestra.run.smithi143.stdout: "can_run": true, 2023-11-26T07:23:10.830 INFO:teuthology.orchestra.run.smithi143.stdout: "error_string": "", 2023-11-26T07:23:10.830 INFO:teuthology.orchestra.run.smithi143.stdout: "module_options": { 2023-11-26T07:23:10.830 INFO:teuthology.orchestra.run.smithi143.stdout: "address": { 2023-11-26T07:23:10.830 INFO:teuthology.orchestra.run.smithi143.stdout: "name": "address", 2023-11-26T07:23:10.830 INFO:teuthology.orchestra.run.smithi143.stdout: "type": "str", 2023-11-26T07:23:10.830 INFO:teuthology.orchestra.run.smithi143.stdout: "level": "advanced", 2023-11-26T07:23:10.830 INFO:teuthology.orchestra.run.smithi143.stdout: "flags": 0, 2023-11-26T07:23:10.831 INFO:teuthology.orchestra.run.smithi143.stdout: "default_value": "unixgram:///tmp/telegraf.sock", 2023-11-26T07:23:10.831 INFO:teuthology.orchestra.run.smithi143.stdout: "min": "", 2023-11-26T07:23:10.831 INFO:teuthology.orchestra.run.smithi143.stdout: "max": "", 2023-11-26T07:23:10.831 INFO:teuthology.orchestra.run.smithi143.stdout: "enum_allowed": [], 2023-11-26T07:23:10.831 INFO:teuthology.orchestra.run.smithi143.stdout: "desc": "", 2023-11-26T07:23:10.831 INFO:teuthology.orchestra.run.smithi143.stdout: "long_desc": "", 2023-11-26T07:23:10.831 INFO:teuthology.orchestra.run.smithi143.stdout: "tags": [], 2023-11-26T07:23:10.831 INFO:teuthology.orchestra.run.smithi143.stdout: "see_also": [] 2023-11-26T07:23:10.831 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2023-11-26T07:23:10.831 INFO:teuthology.orchestra.run.smithi143.stdout: "interval": { 2023-11-26T07:23:10.831 INFO:teuthology.orchestra.run.smithi143.stdout: "name": "interval", 2023-11-26T07:23:10.831 INFO:teuthology.orchestra.run.smithi143.stdout: "type": "secs", 2023-11-26T07:23:10.831 INFO:teuthology.orchestra.run.smithi143.stdout: "level": "advanced", 2023-11-26T07:23:10.832 INFO:teuthology.orchestra.run.smithi143.stdout: "flags": 0, 2023-11-26T07:23:10.832 INFO:teuthology.orchestra.run.smithi143.stdout: "default_value": "15", 2023-11-26T07:23:10.832 INFO:teuthology.orchestra.run.smithi143.stdout: "min": "", 2023-11-26T07:23:10.832 INFO:teuthology.orchestra.run.smithi143.stdout: "max": "", 2023-11-26T07:23:10.832 INFO:teuthology.orchestra.run.smithi143.stdout: "enum_allowed": [], 2023-11-26T07:23:10.832 INFO:teuthology.orchestra.run.smithi143.stdout: "desc": "", 2023-11-26T07:23:10.832 INFO:teuthology.orchestra.run.smithi143.stdout: "long_desc": "", 2023-11-26T07:23:10.832 INFO:teuthology.orchestra.run.smithi143.stdout: "tags": [], 2023-11-26T07:23:10.832 INFO:teuthology.orchestra.run.smithi143.stdout: "see_also": [] 2023-11-26T07:23:10.832 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2023-11-26T07:23:10.832 INFO:teuthology.orchestra.run.smithi143.stdout: "log_level": { 2023-11-26T07:23:10.832 INFO:teuthology.orchestra.run.smithi143.stdout: "name": "log_level", 2023-11-26T07:23:10.833 INFO:teuthology.orchestra.run.smithi143.stdout: "type": "str", 2023-11-26T07:23:10.833 INFO:teuthology.orchestra.run.smithi143.stdout: "level": "advanced", 2023-11-26T07:23:10.833 INFO:teuthology.orchestra.run.smithi143.stdout: "flags": 1, 2023-11-26T07:23:10.833 INFO:teuthology.orchestra.run.smithi143.stdout: "default_value": "", 2023-11-26T07:23:10.833 INFO:teuthology.orchestra.run.smithi143.stdout: "min": "", 2023-11-26T07:23:10.833 INFO:teuthology.orchestra.run.smithi143.stdout: "max": "", 2023-11-26T07:23:10.833 INFO:teuthology.orchestra.run.smithi143.stdout: "enum_allowed": [ 2023-11-26T07:23:10.833 INFO:teuthology.orchestra.run.smithi143.stdout: "", 2023-11-26T07:23:10.833 INFO:teuthology.orchestra.run.smithi143.stdout: "critical", 2023-11-26T07:23:10.833 INFO:teuthology.orchestra.run.smithi143.stdout: "debug", 2023-11-26T07:23:10.833 INFO:teuthology.orchestra.run.smithi143.stdout: "error", 2023-11-26T07:23:10.833 INFO:teuthology.orchestra.run.smithi143.stdout: "info", 2023-11-26T07:23:10.833 INFO:teuthology.orchestra.run.smithi143.stdout: "warning" 2023-11-26T07:23:10.834 INFO:teuthology.orchestra.run.smithi143.stdout: ], 2023-11-26T07:23:10.834 INFO:teuthology.orchestra.run.smithi143.stdout: "desc": "", 2023-11-26T07:23:10.834 INFO:teuthology.orchestra.run.smithi143.stdout: "long_desc": "", 2023-11-26T07:23:10.834 INFO:teuthology.orchestra.run.smithi143.stdout: "tags": [], 2023-11-26T07:23:10.834 INFO:teuthology.orchestra.run.smithi143.stdout: "see_also": [] 2023-11-26T07:23:10.834 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2023-11-26T07:23:10.834 INFO:teuthology.orchestra.run.smithi143.stdout: "log_to_cluster": { 2023-11-26T07:23:10.834 INFO:teuthology.orchestra.run.smithi143.stdout: "name": "log_to_cluster", 2023-11-26T07:23:10.834 INFO:teuthology.orchestra.run.smithi143.stdout: "type": "bool", 2023-11-26T07:23:10.834 INFO:teuthology.orchestra.run.smithi143.stdout: "level": "advanced", 2023-11-26T07:23:10.834 INFO:teuthology.orchestra.run.smithi143.stdout: "flags": 1, 2023-11-26T07:23:10.834 INFO:teuthology.orchestra.run.smithi143.stdout: "default_value": "False", 2023-11-26T07:23:10.834 INFO:teuthology.orchestra.run.smithi143.stdout: "min": "", 2023-11-26T07:23:10.835 INFO:teuthology.orchestra.run.smithi143.stdout: "max": "", 2023-11-26T07:23:10.835 INFO:teuthology.orchestra.run.smithi143.stdout: "enum_allowed": [], 2023-11-26T07:23:10.835 INFO:teuthology.orchestra.run.smithi143.stdout: "desc": "", 2023-11-26T07:23:10.835 INFO:teuthology.orchestra.run.smithi143.stdout: "long_desc": "", 2023-11-26T07:23:10.835 INFO:teuthology.orchestra.run.smithi143.stdout: "tags": [], 2023-11-26T07:23:10.835 INFO:teuthology.orchestra.run.smithi143.stdout: "see_also": [] 2023-11-26T07:23:10.835 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2023-11-26T07:23:10.835 INFO:teuthology.orchestra.run.smithi143.stdout: "log_to_cluster_level": { 2023-11-26T07:23:10.835 INFO:teuthology.orchestra.run.smithi143.stdout: "name": "log_to_cluster_level", 2023-11-26T07:23:10.835 INFO:teuthology.orchestra.run.smithi143.stdout: "type": "str", 2023-11-26T07:23:10.835 INFO:teuthology.orchestra.run.smithi143.stdout: "level": "advanced", 2023-11-26T07:23:10.835 INFO:teuthology.orchestra.run.smithi143.stdout: "flags": 1, 2023-11-26T07:23:10.835 INFO:teuthology.orchestra.run.smithi143.stdout: "default_value": "info", 2023-11-26T07:23:10.836 INFO:teuthology.orchestra.run.smithi143.stdout: "min": "", 2023-11-26T07:23:10.836 INFO:teuthology.orchestra.run.smithi143.stdout: "max": "", 2023-11-26T07:23:10.836 INFO:teuthology.orchestra.run.smithi143.stdout: "enum_allowed": [ 2023-11-26T07:23:10.836 INFO:teuthology.orchestra.run.smithi143.stdout: "", 2023-11-26T07:23:10.836 INFO:teuthology.orchestra.run.smithi143.stdout: "critical", 2023-11-26T07:23:10.836 INFO:teuthology.orchestra.run.smithi143.stdout: "debug", 2023-11-26T07:23:10.836 INFO:teuthology.orchestra.run.smithi143.stdout: "error", 2023-11-26T07:23:10.836 INFO:teuthology.orchestra.run.smithi143.stdout: "info", 2023-11-26T07:23:10.836 INFO:teuthology.orchestra.run.smithi143.stdout: "warning" 2023-11-26T07:23:10.836 INFO:teuthology.orchestra.run.smithi143.stdout: ], 2023-11-26T07:23:10.836 INFO:teuthology.orchestra.run.smithi143.stdout: "desc": "", 2023-11-26T07:23:10.836 INFO:teuthology.orchestra.run.smithi143.stdout: "long_desc": "", 2023-11-26T07:23:10.837 INFO:teuthology.orchestra.run.smithi143.stdout: "tags": [], 2023-11-26T07:23:10.837 INFO:teuthology.orchestra.run.smithi143.stdout: "see_also": [] 2023-11-26T07:23:10.837 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2023-11-26T07:23:10.837 INFO:teuthology.orchestra.run.smithi143.stdout: "log_to_file": { 2023-11-26T07:23:10.837 INFO:teuthology.orchestra.run.smithi143.stdout: "name": "log_to_file", 2023-11-26T07:23:10.837 INFO:teuthology.orchestra.run.smithi143.stdout: "type": "bool", 2023-11-26T07:23:10.837 INFO:teuthology.orchestra.run.smithi143.stdout: "level": "advanced", 2023-11-26T07:23:10.837 INFO:teuthology.orchestra.run.smithi143.stdout: "flags": 1, 2023-11-26T07:23:10.837 INFO:teuthology.orchestra.run.smithi143.stdout: "default_value": "False", 2023-11-26T07:23:10.837 INFO:teuthology.orchestra.run.smithi143.stdout: "min": "", 2023-11-26T07:23:10.837 INFO:teuthology.orchestra.run.smithi143.stdout: "max": "", 2023-11-26T07:23:10.837 INFO:teuthology.orchestra.run.smithi143.stdout: "enum_allowed": [], 2023-11-26T07:23:10.837 INFO:teuthology.orchestra.run.smithi143.stdout: "desc": "", 2023-11-26T07:23:10.838 INFO:teuthology.orchestra.run.smithi143.stdout: "long_desc": "", 2023-11-26T07:23:10.838 INFO:teuthology.orchestra.run.smithi143.stdout: "tags": [], 2023-11-26T07:23:10.838 INFO:teuthology.orchestra.run.smithi143.stdout: "see_also": [] 2023-11-26T07:23:10.838 INFO:teuthology.orchestra.run.smithi143.stdout: } 2023-11-26T07:23:10.838 INFO:teuthology.orchestra.run.smithi143.stdout: } 2023-11-26T07:23:10.838 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2023-11-26T07:23:10.838 INFO:teuthology.orchestra.run.smithi143.stdout: { 2023-11-26T07:23:10.838 INFO:teuthology.orchestra.run.smithi143.stdout: "name": "telemetry", 2023-11-26T07:23:10.838 INFO:teuthology.orchestra.run.smithi143.stdout: "can_run": true, 2023-11-26T07:23:10.838 INFO:teuthology.orchestra.run.smithi143.stdout: "error_string": "", 2023-11-26T07:23:10.838 INFO:teuthology.orchestra.run.smithi143.stdout: "module_options": { 2023-11-26T07:23:10.838 INFO:teuthology.orchestra.run.smithi143.stdout: "channel_basic": { 2023-11-26T07:23:10.839 INFO:teuthology.orchestra.run.smithi143.stdout: "name": "channel_basic", 2023-11-26T07:23:10.839 INFO:teuthology.orchestra.run.smithi143.stdout: "type": "bool", 2023-11-26T07:23:10.839 INFO:teuthology.orchestra.run.smithi143.stdout: "level": "advanced", 2023-11-26T07:23:10.839 INFO:teuthology.orchestra.run.smithi143.stdout: "flags": 0, 2023-11-26T07:23:10.839 INFO:teuthology.orchestra.run.smithi143.stdout: "default_value": "True", 2023-11-26T07:23:10.839 INFO:teuthology.orchestra.run.smithi143.stdout: "min": "", 2023-11-26T07:23:10.839 INFO:teuthology.orchestra.run.smithi143.stdout: "max": "", 2023-11-26T07:23:10.839 INFO:teuthology.orchestra.run.smithi143.stdout: "enum_allowed": [], 2023-11-26T07:23:10.839 INFO:teuthology.orchestra.run.smithi143.stdout: "desc": "Share basic cluster information (size, version)", 2023-11-26T07:23:10.839 INFO:teuthology.orchestra.run.smithi143.stdout: "long_desc": "", 2023-11-26T07:23:10.839 INFO:teuthology.orchestra.run.smithi143.stdout: "tags": [], 2023-11-26T07:23:10.839 INFO:teuthology.orchestra.run.smithi143.stdout: "see_also": [] 2023-11-26T07:23:10.839 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2023-11-26T07:23:10.840 INFO:teuthology.orchestra.run.smithi143.stdout: "channel_crash": { 2023-11-26T07:23:10.840 INFO:teuthology.orchestra.run.smithi143.stdout: "name": "channel_crash", 2023-11-26T07:23:10.840 INFO:teuthology.orchestra.run.smithi143.stdout: "type": "bool", 2023-11-26T07:23:10.840 INFO:teuthology.orchestra.run.smithi143.stdout: "level": "advanced", 2023-11-26T07:23:10.840 INFO:teuthology.orchestra.run.smithi143.stdout: "flags": 0, 2023-11-26T07:23:10.840 INFO:teuthology.orchestra.run.smithi143.stdout: "default_value": "True", 2023-11-26T07:23:10.840 INFO:teuthology.orchestra.run.smithi143.stdout: "min": "", 2023-11-26T07:23:10.840 INFO:teuthology.orchestra.run.smithi143.stdout: "max": "", 2023-11-26T07:23:10.840 INFO:teuthology.orchestra.run.smithi143.stdout: "enum_allowed": [], 2023-11-26T07:23:10.840 INFO:teuthology.orchestra.run.smithi143.stdout: "desc": "Share metadata about Ceph daemon crashes (version, stack straces, etc)", 2023-11-26T07:23:10.840 INFO:teuthology.orchestra.run.smithi143.stdout: "long_desc": "", 2023-11-26T07:23:10.840 INFO:teuthology.orchestra.run.smithi143.stdout: "tags": [], 2023-11-26T07:23:10.841 INFO:teuthology.orchestra.run.smithi143.stdout: "see_also": [] 2023-11-26T07:23:10.841 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2023-11-26T07:23:10.841 INFO:teuthology.orchestra.run.smithi143.stdout: "channel_device": { 2023-11-26T07:23:10.841 INFO:teuthology.orchestra.run.smithi143.stdout: "name": "channel_device", 2023-11-26T07:23:10.841 INFO:teuthology.orchestra.run.smithi143.stdout: "type": "bool", 2023-11-26T07:23:10.841 INFO:teuthology.orchestra.run.smithi143.stdout: "level": "advanced", 2023-11-26T07:23:10.841 INFO:teuthology.orchestra.run.smithi143.stdout: "flags": 0, 2023-11-26T07:23:10.841 INFO:teuthology.orchestra.run.smithi143.stdout: "default_value": "True", 2023-11-26T07:23:10.841 INFO:teuthology.orchestra.run.smithi143.stdout: "min": "", 2023-11-26T07:23:10.841 INFO:teuthology.orchestra.run.smithi143.stdout: "max": "", 2023-11-26T07:23:10.841 INFO:teuthology.orchestra.run.smithi143.stdout: "enum_allowed": [], 2023-11-26T07:23:10.841 INFO:teuthology.orchestra.run.smithi143.stdout: "desc": "Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)", 2023-11-26T07:23:10.841 INFO:teuthology.orchestra.run.smithi143.stdout: "long_desc": "", 2023-11-26T07:23:10.842 INFO:teuthology.orchestra.run.smithi143.stdout: "tags": [], 2023-11-26T07:23:10.842 INFO:teuthology.orchestra.run.smithi143.stdout: "see_also": [] 2023-11-26T07:23:10.842 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2023-11-26T07:23:10.842 INFO:teuthology.orchestra.run.smithi143.stdout: "channel_ident": { 2023-11-26T07:23:10.842 INFO:teuthology.orchestra.run.smithi143.stdout: "name": "channel_ident", 2023-11-26T07:23:10.842 INFO:teuthology.orchestra.run.smithi143.stdout: "type": "bool", 2023-11-26T07:23:10.842 INFO:teuthology.orchestra.run.smithi143.stdout: "level": "advanced", 2023-11-26T07:23:10.842 INFO:teuthology.orchestra.run.smithi143.stdout: "flags": 0, 2023-11-26T07:23:10.842 INFO:teuthology.orchestra.run.smithi143.stdout: "default_value": "False", 2023-11-26T07:23:10.842 INFO:teuthology.orchestra.run.smithi143.stdout: "min": "", 2023-11-26T07:23:10.842 INFO:teuthology.orchestra.run.smithi143.stdout: "max": "", 2023-11-26T07:23:10.842 INFO:teuthology.orchestra.run.smithi143.stdout: "enum_allowed": [], 2023-11-26T07:23:10.843 INFO:teuthology.orchestra.run.smithi143.stdout: "desc": "Share a user-provided description and/or contact email for the cluster", 2023-11-26T07:23:10.843 INFO:teuthology.orchestra.run.smithi143.stdout: "long_desc": "", 2023-11-26T07:23:10.843 INFO:teuthology.orchestra.run.smithi143.stdout: "tags": [], 2023-11-26T07:23:10.843 INFO:teuthology.orchestra.run.smithi143.stdout: "see_also": [] 2023-11-26T07:23:10.843 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2023-11-26T07:23:10.843 INFO:teuthology.orchestra.run.smithi143.stdout: "channel_perf": { 2023-11-26T07:23:10.843 INFO:teuthology.orchestra.run.smithi143.stdout: "name": "channel_perf", 2023-11-26T07:23:10.843 INFO:teuthology.orchestra.run.smithi143.stdout: "type": "bool", 2023-11-26T07:23:10.843 INFO:teuthology.orchestra.run.smithi143.stdout: "level": "advanced", 2023-11-26T07:23:10.843 INFO:teuthology.orchestra.run.smithi143.stdout: "flags": 0, 2023-11-26T07:23:10.843 INFO:teuthology.orchestra.run.smithi143.stdout: "default_value": "False", 2023-11-26T07:23:10.843 INFO:teuthology.orchestra.run.smithi143.stdout: "min": "", 2023-11-26T07:23:10.844 INFO:teuthology.orchestra.run.smithi143.stdout: "max": "", 2023-11-26T07:23:10.844 INFO:teuthology.orchestra.run.smithi143.stdout: "enum_allowed": [], 2023-11-26T07:23:10.844 INFO:teuthology.orchestra.run.smithi143.stdout: "desc": "Share various performance metrics of a cluster", 2023-11-26T07:23:10.844 INFO:teuthology.orchestra.run.smithi143.stdout: "long_desc": "", 2023-11-26T07:23:10.844 INFO:teuthology.orchestra.run.smithi143.stdout: "tags": [], 2023-11-26T07:23:10.844 INFO:teuthology.orchestra.run.smithi143.stdout: "see_also": [] 2023-11-26T07:23:10.844 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2023-11-26T07:23:10.844 INFO:teuthology.orchestra.run.smithi143.stdout: "contact": { 2023-11-26T07:23:10.844 INFO:teuthology.orchestra.run.smithi143.stdout: "name": "contact", 2023-11-26T07:23:10.844 INFO:teuthology.orchestra.run.smithi143.stdout: "type": "str", 2023-11-26T07:23:10.844 INFO:teuthology.orchestra.run.smithi143.stdout: "level": "advanced", 2023-11-26T07:23:10.844 INFO:teuthology.orchestra.run.smithi143.stdout: "flags": 0, 2023-11-26T07:23:10.844 INFO:teuthology.orchestra.run.smithi143.stdout: "default_value": "", 2023-11-26T07:23:10.845 INFO:teuthology.orchestra.run.smithi143.stdout: "min": "", 2023-11-26T07:23:10.845 INFO:teuthology.orchestra.run.smithi143.stdout: "max": "", 2023-11-26T07:23:10.845 INFO:teuthology.orchestra.run.smithi143.stdout: "enum_allowed": [], 2023-11-26T07:23:10.845 INFO:teuthology.orchestra.run.smithi143.stdout: "desc": "", 2023-11-26T07:23:10.845 INFO:teuthology.orchestra.run.smithi143.stdout: "long_desc": "", 2023-11-26T07:23:10.845 INFO:teuthology.orchestra.run.smithi143.stdout: "tags": [], 2023-11-26T07:23:10.845 INFO:teuthology.orchestra.run.smithi143.stdout: "see_also": [] 2023-11-26T07:23:10.845 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2023-11-26T07:23:10.845 INFO:teuthology.orchestra.run.smithi143.stdout: "description": { 2023-11-26T07:23:10.845 INFO:teuthology.orchestra.run.smithi143.stdout: "name": "description", 2023-11-26T07:23:10.845 INFO:teuthology.orchestra.run.smithi143.stdout: "type": "str", 2023-11-26T07:23:10.845 INFO:teuthology.orchestra.run.smithi143.stdout: "level": "advanced", 2023-11-26T07:23:10.845 INFO:teuthology.orchestra.run.smithi143.stdout: "flags": 0, 2023-11-26T07:23:10.846 INFO:teuthology.orchestra.run.smithi143.stdout: "default_value": "", 2023-11-26T07:23:10.846 INFO:teuthology.orchestra.run.smithi143.stdout: "min": "", 2023-11-26T07:23:10.846 INFO:teuthology.orchestra.run.smithi143.stdout: "max": "", 2023-11-26T07:23:10.846 INFO:teuthology.orchestra.run.smithi143.stdout: "enum_allowed": [], 2023-11-26T07:23:10.846 INFO:teuthology.orchestra.run.smithi143.stdout: "desc": "", 2023-11-26T07:23:10.846 INFO:teuthology.orchestra.run.smithi143.stdout: "long_desc": "", 2023-11-26T07:23:10.846 INFO:teuthology.orchestra.run.smithi143.stdout: "tags": [], 2023-11-26T07:23:10.846 INFO:teuthology.orchestra.run.smithi143.stdout: "see_also": [] 2023-11-26T07:23:10.846 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2023-11-26T07:23:10.846 INFO:teuthology.orchestra.run.smithi143.stdout: "device_url": { 2023-11-26T07:23:10.846 INFO:teuthology.orchestra.run.smithi143.stdout: "name": "device_url", 2023-11-26T07:23:10.846 INFO:teuthology.orchestra.run.smithi143.stdout: "type": "str", 2023-11-26T07:23:10.847 INFO:teuthology.orchestra.run.smithi143.stdout: "level": "advanced", 2023-11-26T07:23:10.847 INFO:teuthology.orchestra.run.smithi143.stdout: "flags": 0, 2023-11-26T07:23:10.847 INFO:teuthology.orchestra.run.smithi143.stdout: "default_value": "https://telemetry.ceph.com/device", 2023-11-26T07:23:10.847 INFO:teuthology.orchestra.run.smithi143.stdout: "min": "", 2023-11-26T07:23:10.847 INFO:teuthology.orchestra.run.smithi143.stdout: "max": "", 2023-11-26T07:23:10.847 INFO:teuthology.orchestra.run.smithi143.stdout: "enum_allowed": [], 2023-11-26T07:23:10.847 INFO:teuthology.orchestra.run.smithi143.stdout: "desc": "", 2023-11-26T07:23:10.847 INFO:teuthology.orchestra.run.smithi143.stdout: "long_desc": "", 2023-11-26T07:23:10.847 INFO:teuthology.orchestra.run.smithi143.stdout: "tags": [], 2023-11-26T07:23:10.847 INFO:teuthology.orchestra.run.smithi143.stdout: "see_also": [] 2023-11-26T07:23:10.847 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2023-11-26T07:23:10.847 INFO:teuthology.orchestra.run.smithi143.stdout: "enabled": { 2023-11-26T07:23:10.847 INFO:teuthology.orchestra.run.smithi143.stdout: "name": "enabled", 2023-11-26T07:23:10.848 INFO:teuthology.orchestra.run.smithi143.stdout: "type": "bool", 2023-11-26T07:23:10.848 INFO:teuthology.orchestra.run.smithi143.stdout: "level": "advanced", 2023-11-26T07:23:10.848 INFO:teuthology.orchestra.run.smithi143.stdout: "flags": 0, 2023-11-26T07:23:10.848 INFO:teuthology.orchestra.run.smithi143.stdout: "default_value": "False", 2023-11-26T07:23:10.848 INFO:teuthology.orchestra.run.smithi143.stdout: "min": "", 2023-11-26T07:23:10.848 INFO:teuthology.orchestra.run.smithi143.stdout: "max": "", 2023-11-26T07:23:10.848 INFO:teuthology.orchestra.run.smithi143.stdout: "enum_allowed": [], 2023-11-26T07:23:10.848 INFO:teuthology.orchestra.run.smithi143.stdout: "desc": "", 2023-11-26T07:23:10.848 INFO:teuthology.orchestra.run.smithi143.stdout: "long_desc": "", 2023-11-26T07:23:10.848 INFO:teuthology.orchestra.run.smithi143.stdout: "tags": [], 2023-11-26T07:23:10.848 INFO:teuthology.orchestra.run.smithi143.stdout: "see_also": [] 2023-11-26T07:23:10.848 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2023-11-26T07:23:10.849 INFO:teuthology.orchestra.run.smithi143.stdout: "interval": { 2023-11-26T07:23:10.849 INFO:teuthology.orchestra.run.smithi143.stdout: "name": "interval", 2023-11-26T07:23:10.849 INFO:teuthology.orchestra.run.smithi143.stdout: "type": "int", 2023-11-26T07:23:10.849 INFO:teuthology.orchestra.run.smithi143.stdout: "level": "advanced", 2023-11-26T07:23:10.849 INFO:teuthology.orchestra.run.smithi143.stdout: "flags": 0, 2023-11-26T07:23:10.849 INFO:teuthology.orchestra.run.smithi143.stdout: "default_value": "24", 2023-11-26T07:23:10.849 INFO:teuthology.orchestra.run.smithi143.stdout: "min": "8", 2023-11-26T07:23:10.849 INFO:teuthology.orchestra.run.smithi143.stdout: "max": "", 2023-11-26T07:23:10.849 INFO:teuthology.orchestra.run.smithi143.stdout: "enum_allowed": [], 2023-11-26T07:23:10.849 INFO:teuthology.orchestra.run.smithi143.stdout: "desc": "", 2023-11-26T07:23:10.849 INFO:teuthology.orchestra.run.smithi143.stdout: "long_desc": "", 2023-11-26T07:23:10.849 INFO:teuthology.orchestra.run.smithi143.stdout: "tags": [], 2023-11-26T07:23:10.849 INFO:teuthology.orchestra.run.smithi143.stdout: "see_also": [] 2023-11-26T07:23:10.850 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2023-11-26T07:23:10.850 INFO:teuthology.orchestra.run.smithi143.stdout: "last_opt_revision": { 2023-11-26T07:23:10.850 INFO:teuthology.orchestra.run.smithi143.stdout: "name": "last_opt_revision", 2023-11-26T07:23:10.850 INFO:teuthology.orchestra.run.smithi143.stdout: "type": "int", 2023-11-26T07:23:10.850 INFO:teuthology.orchestra.run.smithi143.stdout: "level": "advanced", 2023-11-26T07:23:10.850 INFO:teuthology.orchestra.run.smithi143.stdout: "flags": 0, 2023-11-26T07:23:10.850 INFO:teuthology.orchestra.run.smithi143.stdout: "default_value": "1", 2023-11-26T07:23:10.850 INFO:teuthology.orchestra.run.smithi143.stdout: "min": "", 2023-11-26T07:23:10.850 INFO:teuthology.orchestra.run.smithi143.stdout: "max": "", 2023-11-26T07:23:10.850 INFO:teuthology.orchestra.run.smithi143.stdout: "enum_allowed": [], 2023-11-26T07:23:10.850 INFO:teuthology.orchestra.run.smithi143.stdout: "desc": "", 2023-11-26T07:23:10.850 INFO:teuthology.orchestra.run.smithi143.stdout: "long_desc": "", 2023-11-26T07:23:10.851 INFO:teuthology.orchestra.run.smithi143.stdout: "tags": [], 2023-11-26T07:23:10.851 INFO:teuthology.orchestra.run.smithi143.stdout: "see_also": [] 2023-11-26T07:23:10.851 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2023-11-26T07:23:10.851 INFO:teuthology.orchestra.run.smithi143.stdout: "leaderboard": { 2023-11-26T07:23:10.851 INFO:teuthology.orchestra.run.smithi143.stdout: "name": "leaderboard", 2023-11-26T07:23:10.851 INFO:teuthology.orchestra.run.smithi143.stdout: "type": "bool", 2023-11-26T07:23:10.851 INFO:teuthology.orchestra.run.smithi143.stdout: "level": "advanced", 2023-11-26T07:23:10.851 INFO:teuthology.orchestra.run.smithi143.stdout: "flags": 0, 2023-11-26T07:23:10.851 INFO:teuthology.orchestra.run.smithi143.stdout: "default_value": "False", 2023-11-26T07:23:10.851 INFO:teuthology.orchestra.run.smithi143.stdout: "min": "", 2023-11-26T07:23:10.851 INFO:teuthology.orchestra.run.smithi143.stdout: "max": "", 2023-11-26T07:23:10.851 INFO:teuthology.orchestra.run.smithi143.stdout: "enum_allowed": [], 2023-11-26T07:23:10.852 INFO:teuthology.orchestra.run.smithi143.stdout: "desc": "", 2023-11-26T07:23:10.852 INFO:teuthology.orchestra.run.smithi143.stdout: "long_desc": "", 2023-11-26T07:23:10.852 INFO:teuthology.orchestra.run.smithi143.stdout: "tags": [], 2023-11-26T07:23:10.852 INFO:teuthology.orchestra.run.smithi143.stdout: "see_also": [] 2023-11-26T07:23:10.852 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2023-11-26T07:23:10.852 INFO:teuthology.orchestra.run.smithi143.stdout: "leaderboard_description": { 2023-11-26T07:23:10.852 INFO:teuthology.orchestra.run.smithi143.stdout: "name": "leaderboard_description", 2023-11-26T07:23:10.852 INFO:teuthology.orchestra.run.smithi143.stdout: "type": "str", 2023-11-26T07:23:10.852 INFO:teuthology.orchestra.run.smithi143.stdout: "level": "advanced", 2023-11-26T07:23:10.852 INFO:teuthology.orchestra.run.smithi143.stdout: "flags": 0, 2023-11-26T07:23:10.852 INFO:teuthology.orchestra.run.smithi143.stdout: "default_value": "", 2023-11-26T07:23:10.852 INFO:teuthology.orchestra.run.smithi143.stdout: "min": "", 2023-11-26T07:23:10.853 INFO:teuthology.orchestra.run.smithi143.stdout: "max": "", 2023-11-26T07:23:10.853 INFO:teuthology.orchestra.run.smithi143.stdout: "enum_allowed": [], 2023-11-26T07:23:10.853 INFO:teuthology.orchestra.run.smithi143.stdout: "desc": "", 2023-11-26T07:23:10.853 INFO:teuthology.orchestra.run.smithi143.stdout: "long_desc": "", 2023-11-26T07:23:10.853 INFO:teuthology.orchestra.run.smithi143.stdout: "tags": [], 2023-11-26T07:23:10.853 INFO:teuthology.orchestra.run.smithi143.stdout: "see_also": [] 2023-11-26T07:23:10.853 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2023-11-26T07:23:10.853 INFO:teuthology.orchestra.run.smithi143.stdout: "log_level": { 2023-11-26T07:23:10.853 INFO:teuthology.orchestra.run.smithi143.stdout: "name": "log_level", 2023-11-26T07:23:10.853 INFO:teuthology.orchestra.run.smithi143.stdout: "type": "str", 2023-11-26T07:23:10.853 INFO:teuthology.orchestra.run.smithi143.stdout: "level": "advanced", 2023-11-26T07:23:10.853 INFO:teuthology.orchestra.run.smithi143.stdout: "flags": 1, 2023-11-26T07:23:10.854 INFO:teuthology.orchestra.run.smithi143.stdout: "default_value": "", 2023-11-26T07:23:10.854 INFO:teuthology.orchestra.run.smithi143.stdout: "min": "", 2023-11-26T07:23:10.854 INFO:teuthology.orchestra.run.smithi143.stdout: "max": "", 2023-11-26T07:23:10.854 INFO:teuthology.orchestra.run.smithi143.stdout: "enum_allowed": [ 2023-11-26T07:23:10.854 INFO:teuthology.orchestra.run.smithi143.stdout: "", 2023-11-26T07:23:10.854 INFO:teuthology.orchestra.run.smithi143.stdout: "critical", 2023-11-26T07:23:10.854 INFO:teuthology.orchestra.run.smithi143.stdout: "debug", 2023-11-26T07:23:10.854 INFO:teuthology.orchestra.run.smithi143.stdout: "error", 2023-11-26T07:23:10.854 INFO:teuthology.orchestra.run.smithi143.stdout: "info", 2023-11-26T07:23:10.854 INFO:teuthology.orchestra.run.smithi143.stdout: "warning" 2023-11-26T07:23:10.854 INFO:teuthology.orchestra.run.smithi143.stdout: ], 2023-11-26T07:23:10.854 INFO:teuthology.orchestra.run.smithi143.stdout: "desc": "", 2023-11-26T07:23:10.855 INFO:teuthology.orchestra.run.smithi143.stdout: "long_desc": "", 2023-11-26T07:23:10.855 INFO:teuthology.orchestra.run.smithi143.stdout: "tags": [], 2023-11-26T07:23:10.855 INFO:teuthology.orchestra.run.smithi143.stdout: "see_also": [] 2023-11-26T07:23:10.855 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2023-11-26T07:23:10.855 INFO:teuthology.orchestra.run.smithi143.stdout: "log_to_cluster": { 2023-11-26T07:23:10.855 INFO:teuthology.orchestra.run.smithi143.stdout: "name": "log_to_cluster", 2023-11-26T07:23:10.855 INFO:teuthology.orchestra.run.smithi143.stdout: "type": "bool", 2023-11-26T07:23:10.855 INFO:teuthology.orchestra.run.smithi143.stdout: "level": "advanced", 2023-11-26T07:23:10.855 INFO:teuthology.orchestra.run.smithi143.stdout: "flags": 1, 2023-11-26T07:23:10.855 INFO:teuthology.orchestra.run.smithi143.stdout: "default_value": "False", 2023-11-26T07:23:10.855 INFO:teuthology.orchestra.run.smithi143.stdout: "min": "", 2023-11-26T07:23:10.855 INFO:teuthology.orchestra.run.smithi143.stdout: "max": "", 2023-11-26T07:23:10.855 INFO:teuthology.orchestra.run.smithi143.stdout: "enum_allowed": [], 2023-11-26T07:23:10.856 INFO:teuthology.orchestra.run.smithi143.stdout: "desc": "", 2023-11-26T07:23:10.856 INFO:teuthology.orchestra.run.smithi143.stdout: "long_desc": "", 2023-11-26T07:23:10.856 INFO:teuthology.orchestra.run.smithi143.stdout: "tags": [], 2023-11-26T07:23:10.856 INFO:teuthology.orchestra.run.smithi143.stdout: "see_also": [] 2023-11-26T07:23:10.856 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2023-11-26T07:23:10.856 INFO:teuthology.orchestra.run.smithi143.stdout: "log_to_cluster_level": { 2023-11-26T07:23:10.856 INFO:teuthology.orchestra.run.smithi143.stdout: "name": "log_to_cluster_level", 2023-11-26T07:23:10.856 INFO:teuthology.orchestra.run.smithi143.stdout: "type": "str", 2023-11-26T07:23:10.856 INFO:teuthology.orchestra.run.smithi143.stdout: "level": "advanced", 2023-11-26T07:23:10.856 INFO:teuthology.orchestra.run.smithi143.stdout: "flags": 1, 2023-11-26T07:23:10.856 INFO:teuthology.orchestra.run.smithi143.stdout: "default_value": "info", 2023-11-26T07:23:10.856 INFO:teuthology.orchestra.run.smithi143.stdout: "min": "", 2023-11-26T07:23:10.856 INFO:teuthology.orchestra.run.smithi143.stdout: "max": "", 2023-11-26T07:23:10.857 INFO:teuthology.orchestra.run.smithi143.stdout: "enum_allowed": [ 2023-11-26T07:23:10.857 INFO:teuthology.orchestra.run.smithi143.stdout: "", 2023-11-26T07:23:10.857 INFO:teuthology.orchestra.run.smithi143.stdout: "critical", 2023-11-26T07:23:10.857 INFO:teuthology.orchestra.run.smithi143.stdout: "debug", 2023-11-26T07:23:10.857 INFO:teuthology.orchestra.run.smithi143.stdout: "error", 2023-11-26T07:23:10.857 INFO:teuthology.orchestra.run.smithi143.stdout: "info", 2023-11-26T07:23:10.857 INFO:teuthology.orchestra.run.smithi143.stdout: "warning" 2023-11-26T07:23:10.857 INFO:teuthology.orchestra.run.smithi143.stdout: ], 2023-11-26T07:23:10.857 INFO:teuthology.orchestra.run.smithi143.stdout: "desc": "", 2023-11-26T07:23:10.857 INFO:teuthology.orchestra.run.smithi143.stdout: "long_desc": "", 2023-11-26T07:23:10.857 INFO:teuthology.orchestra.run.smithi143.stdout: "tags": [], 2023-11-26T07:23:10.857 INFO:teuthology.orchestra.run.smithi143.stdout: "see_also": [] 2023-11-26T07:23:10.858 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2023-11-26T07:23:10.858 INFO:teuthology.orchestra.run.smithi143.stdout: "log_to_file": { 2023-11-26T07:23:10.858 INFO:teuthology.orchestra.run.smithi143.stdout: "name": "log_to_file", 2023-11-26T07:23:10.858 INFO:teuthology.orchestra.run.smithi143.stdout: "type": "bool", 2023-11-26T07:23:10.858 INFO:teuthology.orchestra.run.smithi143.stdout: "level": "advanced", 2023-11-26T07:23:10.858 INFO:teuthology.orchestra.run.smithi143.stdout: "flags": 1, 2023-11-26T07:23:10.858 INFO:teuthology.orchestra.run.smithi143.stdout: "default_value": "False", 2023-11-26T07:23:10.858 INFO:teuthology.orchestra.run.smithi143.stdout: "min": "", 2023-11-26T07:23:10.858 INFO:teuthology.orchestra.run.smithi143.stdout: "max": "", 2023-11-26T07:23:10.858 INFO:teuthology.orchestra.run.smithi143.stdout: "enum_allowed": [], 2023-11-26T07:23:10.858 INFO:teuthology.orchestra.run.smithi143.stdout: "desc": "", 2023-11-26T07:23:10.858 INFO:teuthology.orchestra.run.smithi143.stdout: "long_desc": "", 2023-11-26T07:23:10.858 INFO:teuthology.orchestra.run.smithi143.stdout: "tags": [], 2023-11-26T07:23:10.859 INFO:teuthology.orchestra.run.smithi143.stdout: "see_also": [] 2023-11-26T07:23:10.859 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2023-11-26T07:23:10.859 INFO:teuthology.orchestra.run.smithi143.stdout: "organization": { 2023-11-26T07:23:10.859 INFO:teuthology.orchestra.run.smithi143.stdout: "name": "organization", 2023-11-26T07:23:10.859 INFO:teuthology.orchestra.run.smithi143.stdout: "type": "str", 2023-11-26T07:23:10.859 INFO:teuthology.orchestra.run.smithi143.stdout: "level": "advanced", 2023-11-26T07:23:10.859 INFO:teuthology.orchestra.run.smithi143.stdout: "flags": 0, 2023-11-26T07:23:10.859 INFO:teuthology.orchestra.run.smithi143.stdout: "default_value": "", 2023-11-26T07:23:10.859 INFO:teuthology.orchestra.run.smithi143.stdout: "min": "", 2023-11-26T07:23:10.859 INFO:teuthology.orchestra.run.smithi143.stdout: "max": "", 2023-11-26T07:23:10.859 INFO:teuthology.orchestra.run.smithi143.stdout: "enum_allowed": [], 2023-11-26T07:23:10.859 INFO:teuthology.orchestra.run.smithi143.stdout: "desc": "", 2023-11-26T07:23:10.859 INFO:teuthology.orchestra.run.smithi143.stdout: "long_desc": "", 2023-11-26T07:23:10.860 INFO:teuthology.orchestra.run.smithi143.stdout: "tags": [], 2023-11-26T07:23:10.860 INFO:teuthology.orchestra.run.smithi143.stdout: "see_also": [] 2023-11-26T07:23:10.860 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2023-11-26T07:23:10.860 INFO:teuthology.orchestra.run.smithi143.stdout: "proxy": { 2023-11-26T07:23:10.860 INFO:teuthology.orchestra.run.smithi143.stdout: "name": "proxy", 2023-11-26T07:23:10.860 INFO:teuthology.orchestra.run.smithi143.stdout: "type": "str", 2023-11-26T07:23:10.860 INFO:teuthology.orchestra.run.smithi143.stdout: "level": "advanced", 2023-11-26T07:23:10.860 INFO:teuthology.orchestra.run.smithi143.stdout: "flags": 0, 2023-11-26T07:23:10.860 INFO:teuthology.orchestra.run.smithi143.stdout: "default_value": "", 2023-11-26T07:23:10.860 INFO:teuthology.orchestra.run.smithi143.stdout: "min": "", 2023-11-26T07:23:10.860 INFO:teuthology.orchestra.run.smithi143.stdout: "max": "", 2023-11-26T07:23:10.860 INFO:teuthology.orchestra.run.smithi143.stdout: "enum_allowed": [], 2023-11-26T07:23:10.860 INFO:teuthology.orchestra.run.smithi143.stdout: "desc": "", 2023-11-26T07:23:10.861 INFO:teuthology.orchestra.run.smithi143.stdout: "long_desc": "", 2023-11-26T07:23:10.861 INFO:teuthology.orchestra.run.smithi143.stdout: "tags": [], 2023-11-26T07:23:10.861 INFO:teuthology.orchestra.run.smithi143.stdout: "see_also": [] 2023-11-26T07:23:10.861 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2023-11-26T07:23:10.861 INFO:teuthology.orchestra.run.smithi143.stdout: "url": { 2023-11-26T07:23:10.861 INFO:teuthology.orchestra.run.smithi143.stdout: "name": "url", 2023-11-26T07:23:10.861 INFO:teuthology.orchestra.run.smithi143.stdout: "type": "str", 2023-11-26T07:23:10.861 INFO:teuthology.orchestra.run.smithi143.stdout: "level": "advanced", 2023-11-26T07:23:10.861 INFO:teuthology.orchestra.run.smithi143.stdout: "flags": 0, 2023-11-26T07:23:10.861 INFO:teuthology.orchestra.run.smithi143.stdout: "default_value": "https://telemetry.ceph.com/report", 2023-11-26T07:23:10.861 INFO:teuthology.orchestra.run.smithi143.stdout: "min": "", 2023-11-26T07:23:10.861 INFO:teuthology.orchestra.run.smithi143.stdout: "max": "", 2023-11-26T07:23:10.862 INFO:teuthology.orchestra.run.smithi143.stdout: "enum_allowed": [], 2023-11-26T07:23:10.862 INFO:teuthology.orchestra.run.smithi143.stdout: "desc": "", 2023-11-26T07:23:10.862 INFO:teuthology.orchestra.run.smithi143.stdout: "long_desc": "", 2023-11-26T07:23:10.862 INFO:teuthology.orchestra.run.smithi143.stdout: "tags": [], 2023-11-26T07:23:10.862 INFO:teuthology.orchestra.run.smithi143.stdout: "see_also": [] 2023-11-26T07:23:10.862 INFO:teuthology.orchestra.run.smithi143.stdout: } 2023-11-26T07:23:10.862 INFO:teuthology.orchestra.run.smithi143.stdout: } 2023-11-26T07:23:10.862 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2023-11-26T07:23:10.862 INFO:teuthology.orchestra.run.smithi143.stdout: { 2023-11-26T07:23:10.862 INFO:teuthology.orchestra.run.smithi143.stdout: "name": "test_orchestrator", 2023-11-26T07:23:10.862 INFO:teuthology.orchestra.run.smithi143.stdout: "can_run": true, 2023-11-26T07:23:10.862 INFO:teuthology.orchestra.run.smithi143.stdout: "error_string": "", 2023-11-26T07:23:10.863 INFO:teuthology.orchestra.run.smithi143.stdout: "module_options": { 2023-11-26T07:23:10.863 INFO:teuthology.orchestra.run.smithi143.stdout: "log_level": { 2023-11-26T07:23:10.863 INFO:teuthology.orchestra.run.smithi143.stdout: "name": "log_level", 2023-11-26T07:23:10.863 INFO:teuthology.orchestra.run.smithi143.stdout: "type": "str", 2023-11-26T07:23:10.863 INFO:teuthology.orchestra.run.smithi143.stdout: "level": "advanced", 2023-11-26T07:23:10.863 INFO:teuthology.orchestra.run.smithi143.stdout: "flags": 1, 2023-11-26T07:23:10.863 INFO:teuthology.orchestra.run.smithi143.stdout: "default_value": "", 2023-11-26T07:23:10.863 INFO:teuthology.orchestra.run.smithi143.stdout: "min": "", 2023-11-26T07:23:10.863 INFO:teuthology.orchestra.run.smithi143.stdout: "max": "", 2023-11-26T07:23:10.863 INFO:teuthology.orchestra.run.smithi143.stdout: "enum_allowed": [ 2023-11-26T07:23:10.863 INFO:teuthology.orchestra.run.smithi143.stdout: "", 2023-11-26T07:23:10.863 INFO:teuthology.orchestra.run.smithi143.stdout: "critical", 2023-11-26T07:23:10.863 INFO:teuthology.orchestra.run.smithi143.stdout: "debug", 2023-11-26T07:23:10.864 INFO:teuthology.orchestra.run.smithi143.stdout: "error", 2023-11-26T07:23:10.864 INFO:teuthology.orchestra.run.smithi143.stdout: "info", 2023-11-26T07:23:10.864 INFO:teuthology.orchestra.run.smithi143.stdout: "warning" 2023-11-26T07:23:10.864 INFO:teuthology.orchestra.run.smithi143.stdout: ], 2023-11-26T07:23:10.864 INFO:teuthology.orchestra.run.smithi143.stdout: "desc": "", 2023-11-26T07:23:10.864 INFO:teuthology.orchestra.run.smithi143.stdout: "long_desc": "", 2023-11-26T07:23:10.864 INFO:teuthology.orchestra.run.smithi143.stdout: "tags": [], 2023-11-26T07:23:10.864 INFO:teuthology.orchestra.run.smithi143.stdout: "see_also": [] 2023-11-26T07:23:10.864 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2023-11-26T07:23:10.864 INFO:teuthology.orchestra.run.smithi143.stdout: "log_to_cluster": { 2023-11-26T07:23:10.864 INFO:teuthology.orchestra.run.smithi143.stdout: "name": "log_to_cluster", 2023-11-26T07:23:10.864 INFO:teuthology.orchestra.run.smithi143.stdout: "type": "bool", 2023-11-26T07:23:10.865 INFO:teuthology.orchestra.run.smithi143.stdout: "level": "advanced", 2023-11-26T07:23:10.865 INFO:teuthology.orchestra.run.smithi143.stdout: "flags": 1, 2023-11-26T07:23:10.865 INFO:teuthology.orchestra.run.smithi143.stdout: "default_value": "False", 2023-11-26T07:23:10.865 INFO:teuthology.orchestra.run.smithi143.stdout: "min": "", 2023-11-26T07:23:10.865 INFO:teuthology.orchestra.run.smithi143.stdout: "max": "", 2023-11-26T07:23:10.865 INFO:teuthology.orchestra.run.smithi143.stdout: "enum_allowed": [], 2023-11-26T07:23:10.865 INFO:teuthology.orchestra.run.smithi143.stdout: "desc": "", 2023-11-26T07:23:10.865 INFO:teuthology.orchestra.run.smithi143.stdout: "long_desc": "", 2023-11-26T07:23:10.865 INFO:teuthology.orchestra.run.smithi143.stdout: "tags": [], 2023-11-26T07:23:10.865 INFO:teuthology.orchestra.run.smithi143.stdout: "see_also": [] 2023-11-26T07:23:10.865 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2023-11-26T07:23:10.865 INFO:teuthology.orchestra.run.smithi143.stdout: "log_to_cluster_level": { 2023-11-26T07:23:10.865 INFO:teuthology.orchestra.run.smithi143.stdout: "name": "log_to_cluster_level", 2023-11-26T07:23:10.866 INFO:teuthology.orchestra.run.smithi143.stdout: "type": "str", 2023-11-26T07:23:10.866 INFO:teuthology.orchestra.run.smithi143.stdout: "level": "advanced", 2023-11-26T07:23:10.866 INFO:teuthology.orchestra.run.smithi143.stdout: "flags": 1, 2023-11-26T07:23:10.866 INFO:teuthology.orchestra.run.smithi143.stdout: "default_value": "info", 2023-11-26T07:23:10.866 INFO:teuthology.orchestra.run.smithi143.stdout: "min": "", 2023-11-26T07:23:10.866 INFO:teuthology.orchestra.run.smithi143.stdout: "max": "", 2023-11-26T07:23:10.866 INFO:teuthology.orchestra.run.smithi143.stdout: "enum_allowed": [ 2023-11-26T07:23:10.866 INFO:teuthology.orchestra.run.smithi143.stdout: "", 2023-11-26T07:23:10.866 INFO:teuthology.orchestra.run.smithi143.stdout: "critical", 2023-11-26T07:23:10.866 INFO:teuthology.orchestra.run.smithi143.stdout: "debug", 2023-11-26T07:23:10.866 INFO:teuthology.orchestra.run.smithi143.stdout: "error", 2023-11-26T07:23:10.866 INFO:teuthology.orchestra.run.smithi143.stdout: "info", 2023-11-26T07:23:10.867 INFO:teuthology.orchestra.run.smithi143.stdout: "warning" 2023-11-26T07:23:10.867 INFO:teuthology.orchestra.run.smithi143.stdout: ], 2023-11-26T07:23:10.867 INFO:teuthology.orchestra.run.smithi143.stdout: "desc": "", 2023-11-26T07:23:10.867 INFO:teuthology.orchestra.run.smithi143.stdout: "long_desc": "", 2023-11-26T07:23:10.867 INFO:teuthology.orchestra.run.smithi143.stdout: "tags": [], 2023-11-26T07:23:10.867 INFO:teuthology.orchestra.run.smithi143.stdout: "see_also": [] 2023-11-26T07:23:10.867 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2023-11-26T07:23:10.867 INFO:teuthology.orchestra.run.smithi143.stdout: "log_to_file": { 2023-11-26T07:23:10.867 INFO:teuthology.orchestra.run.smithi143.stdout: "name": "log_to_file", 2023-11-26T07:23:10.867 INFO:teuthology.orchestra.run.smithi143.stdout: "type": "bool", 2023-11-26T07:23:10.867 INFO:teuthology.orchestra.run.smithi143.stdout: "level": "advanced", 2023-11-26T07:23:10.867 INFO:teuthology.orchestra.run.smithi143.stdout: "flags": 1, 2023-11-26T07:23:10.868 INFO:teuthology.orchestra.run.smithi143.stdout: "default_value": "False", 2023-11-26T07:23:10.868 INFO:teuthology.orchestra.run.smithi143.stdout: "min": "", 2023-11-26T07:23:10.868 INFO:teuthology.orchestra.run.smithi143.stdout: "max": "", 2023-11-26T07:23:10.868 INFO:teuthology.orchestra.run.smithi143.stdout: "enum_allowed": [], 2023-11-26T07:23:10.868 INFO:teuthology.orchestra.run.smithi143.stdout: "desc": "", 2023-11-26T07:23:10.868 INFO:teuthology.orchestra.run.smithi143.stdout: "long_desc": "", 2023-11-26T07:23:10.868 INFO:teuthology.orchestra.run.smithi143.stdout: "tags": [], 2023-11-26T07:23:10.868 INFO:teuthology.orchestra.run.smithi143.stdout: "see_also": [] 2023-11-26T07:23:10.868 INFO:teuthology.orchestra.run.smithi143.stdout: } 2023-11-26T07:23:10.868 INFO:teuthology.orchestra.run.smithi143.stdout: } 2023-11-26T07:23:10.868 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2023-11-26T07:23:10.868 INFO:teuthology.orchestra.run.smithi143.stdout: { 2023-11-26T07:23:10.869 INFO:teuthology.orchestra.run.smithi143.stdout: "name": "volumes", 2023-11-26T07:23:10.869 INFO:teuthology.orchestra.run.smithi143.stdout: "can_run": true, 2023-11-26T07:23:10.869 INFO:teuthology.orchestra.run.smithi143.stdout: "error_string": "", 2023-11-26T07:23:10.869 INFO:teuthology.orchestra.run.smithi143.stdout: "module_options": { 2023-11-26T07:23:10.869 INFO:teuthology.orchestra.run.smithi143.stdout: "log_level": { 2023-11-26T07:23:10.869 INFO:teuthology.orchestra.run.smithi143.stdout: "name": "log_level", 2023-11-26T07:23:10.869 INFO:teuthology.orchestra.run.smithi143.stdout: "type": "str", 2023-11-26T07:23:10.869 INFO:teuthology.orchestra.run.smithi143.stdout: "level": "advanced", 2023-11-26T07:23:10.869 INFO:teuthology.orchestra.run.smithi143.stdout: "flags": 1, 2023-11-26T07:23:10.869 INFO:teuthology.orchestra.run.smithi143.stdout: "default_value": "", 2023-11-26T07:23:10.869 INFO:teuthology.orchestra.run.smithi143.stdout: "min": "", 2023-11-26T07:23:10.869 INFO:teuthology.orchestra.run.smithi143.stdout: "max": "", 2023-11-26T07:23:10.869 INFO:teuthology.orchestra.run.smithi143.stdout: "enum_allowed": [ 2023-11-26T07:23:10.870 INFO:teuthology.orchestra.run.smithi143.stdout: "", 2023-11-26T07:23:10.870 INFO:teuthology.orchestra.run.smithi143.stdout: "critical", 2023-11-26T07:23:10.870 INFO:teuthology.orchestra.run.smithi143.stdout: "debug", 2023-11-26T07:23:10.870 INFO:teuthology.orchestra.run.smithi143.stdout: "error", 2023-11-26T07:23:10.870 INFO:teuthology.orchestra.run.smithi143.stdout: "info", 2023-11-26T07:23:10.870 INFO:teuthology.orchestra.run.smithi143.stdout: "warning" 2023-11-26T07:23:10.870 INFO:teuthology.orchestra.run.smithi143.stdout: ], 2023-11-26T07:23:10.870 INFO:teuthology.orchestra.run.smithi143.stdout: "desc": "", 2023-11-26T07:23:10.870 INFO:teuthology.orchestra.run.smithi143.stdout: "long_desc": "", 2023-11-26T07:23:10.870 INFO:teuthology.orchestra.run.smithi143.stdout: "tags": [], 2023-11-26T07:23:10.870 INFO:teuthology.orchestra.run.smithi143.stdout: "see_also": [] 2023-11-26T07:23:10.870 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2023-11-26T07:23:10.871 INFO:teuthology.orchestra.run.smithi143.stdout: "log_to_cluster": { 2023-11-26T07:23:10.871 INFO:teuthology.orchestra.run.smithi143.stdout: "name": "log_to_cluster", 2023-11-26T07:23:10.871 INFO:teuthology.orchestra.run.smithi143.stdout: "type": "bool", 2023-11-26T07:23:10.871 INFO:teuthology.orchestra.run.smithi143.stdout: "level": "advanced", 2023-11-26T07:23:10.871 INFO:teuthology.orchestra.run.smithi143.stdout: "flags": 1, 2023-11-26T07:23:10.871 INFO:teuthology.orchestra.run.smithi143.stdout: "default_value": "False", 2023-11-26T07:23:10.871 INFO:teuthology.orchestra.run.smithi143.stdout: "min": "", 2023-11-26T07:23:10.871 INFO:teuthology.orchestra.run.smithi143.stdout: "max": "", 2023-11-26T07:23:10.871 INFO:teuthology.orchestra.run.smithi143.stdout: "enum_allowed": [], 2023-11-26T07:23:10.871 INFO:teuthology.orchestra.run.smithi143.stdout: "desc": "", 2023-11-26T07:23:10.871 INFO:teuthology.orchestra.run.smithi143.stdout: "long_desc": "", 2023-11-26T07:23:10.871 INFO:teuthology.orchestra.run.smithi143.stdout: "tags": [], 2023-11-26T07:23:10.872 INFO:teuthology.orchestra.run.smithi143.stdout: "see_also": [] 2023-11-26T07:23:10.872 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2023-11-26T07:23:10.872 INFO:teuthology.orchestra.run.smithi143.stdout: "log_to_cluster_level": { 2023-11-26T07:23:10.872 INFO:teuthology.orchestra.run.smithi143.stdout: "name": "log_to_cluster_level", 2023-11-26T07:23:10.872 INFO:teuthology.orchestra.run.smithi143.stdout: "type": "str", 2023-11-26T07:23:10.872 INFO:teuthology.orchestra.run.smithi143.stdout: "level": "advanced", 2023-11-26T07:23:10.872 INFO:teuthology.orchestra.run.smithi143.stdout: "flags": 1, 2023-11-26T07:23:10.872 INFO:teuthology.orchestra.run.smithi143.stdout: "default_value": "info", 2023-11-26T07:23:10.872 INFO:teuthology.orchestra.run.smithi143.stdout: "min": "", 2023-11-26T07:23:10.872 INFO:teuthology.orchestra.run.smithi143.stdout: "max": "", 2023-11-26T07:23:10.872 INFO:teuthology.orchestra.run.smithi143.stdout: "enum_allowed": [ 2023-11-26T07:23:10.872 INFO:teuthology.orchestra.run.smithi143.stdout: "", 2023-11-26T07:23:10.873 INFO:teuthology.orchestra.run.smithi143.stdout: "critical", 2023-11-26T07:23:10.873 INFO:teuthology.orchestra.run.smithi143.stdout: "debug", 2023-11-26T07:23:10.873 INFO:teuthology.orchestra.run.smithi143.stdout: "error", 2023-11-26T07:23:10.873 INFO:teuthology.orchestra.run.smithi143.stdout: "info", 2023-11-26T07:23:10.873 INFO:teuthology.orchestra.run.smithi143.stdout: "warning" 2023-11-26T07:23:10.873 INFO:teuthology.orchestra.run.smithi143.stdout: ], 2023-11-26T07:23:10.873 INFO:teuthology.orchestra.run.smithi143.stdout: "desc": "", 2023-11-26T07:23:10.873 INFO:teuthology.orchestra.run.smithi143.stdout: "long_desc": "", 2023-11-26T07:23:10.873 INFO:teuthology.orchestra.run.smithi143.stdout: "tags": [], 2023-11-26T07:23:10.873 INFO:teuthology.orchestra.run.smithi143.stdout: "see_also": [] 2023-11-26T07:23:10.873 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2023-11-26T07:23:10.873 INFO:teuthology.orchestra.run.smithi143.stdout: "log_to_file": { 2023-11-26T07:23:10.873 INFO:teuthology.orchestra.run.smithi143.stdout: "name": "log_to_file", 2023-11-26T07:23:10.874 INFO:teuthology.orchestra.run.smithi143.stdout: "type": "bool", 2023-11-26T07:23:10.874 INFO:teuthology.orchestra.run.smithi143.stdout: "level": "advanced", 2023-11-26T07:23:10.874 INFO:teuthology.orchestra.run.smithi143.stdout: "flags": 1, 2023-11-26T07:23:10.874 INFO:teuthology.orchestra.run.smithi143.stdout: "default_value": "False", 2023-11-26T07:23:10.874 INFO:teuthology.orchestra.run.smithi143.stdout: "min": "", 2023-11-26T07:23:10.874 INFO:teuthology.orchestra.run.smithi143.stdout: "max": "", 2023-11-26T07:23:10.874 INFO:teuthology.orchestra.run.smithi143.stdout: "enum_allowed": [], 2023-11-26T07:23:10.874 INFO:teuthology.orchestra.run.smithi143.stdout: "desc": "", 2023-11-26T07:23:10.874 INFO:teuthology.orchestra.run.smithi143.stdout: "long_desc": "", 2023-11-26T07:23:10.874 INFO:teuthology.orchestra.run.smithi143.stdout: "tags": [], 2023-11-26T07:23:10.874 INFO:teuthology.orchestra.run.smithi143.stdout: "see_also": [] 2023-11-26T07:23:10.874 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2023-11-26T07:23:10.875 INFO:teuthology.orchestra.run.smithi143.stdout: "max_concurrent_clones": { 2023-11-26T07:23:10.875 INFO:teuthology.orchestra.run.smithi143.stdout: "name": "max_concurrent_clones", 2023-11-26T07:23:10.875 INFO:teuthology.orchestra.run.smithi143.stdout: "type": "int", 2023-11-26T07:23:10.875 INFO:teuthology.orchestra.run.smithi143.stdout: "level": "advanced", 2023-11-26T07:23:10.875 INFO:teuthology.orchestra.run.smithi143.stdout: "flags": 0, 2023-11-26T07:23:10.875 INFO:teuthology.orchestra.run.smithi143.stdout: "default_value": "4", 2023-11-26T07:23:10.875 INFO:teuthology.orchestra.run.smithi143.stdout: "min": "", 2023-11-26T07:23:10.875 INFO:teuthology.orchestra.run.smithi143.stdout: "max": "", 2023-11-26T07:23:10.875 INFO:teuthology.orchestra.run.smithi143.stdout: "enum_allowed": [], 2023-11-26T07:23:10.875 INFO:teuthology.orchestra.run.smithi143.stdout: "desc": "Number of asynchronous cloner threads", 2023-11-26T07:23:10.875 INFO:teuthology.orchestra.run.smithi143.stdout: "long_desc": "", 2023-11-26T07:23:10.875 INFO:teuthology.orchestra.run.smithi143.stdout: "tags": [], 2023-11-26T07:23:10.875 INFO:teuthology.orchestra.run.smithi143.stdout: "see_also": [] 2023-11-26T07:23:10.876 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2023-11-26T07:23:10.876 INFO:teuthology.orchestra.run.smithi143.stdout: "periodic_async_work": { 2023-11-26T07:23:10.876 INFO:teuthology.orchestra.run.smithi143.stdout: "name": "periodic_async_work", 2023-11-26T07:23:10.876 INFO:teuthology.orchestra.run.smithi143.stdout: "type": "bool", 2023-11-26T07:23:10.876 INFO:teuthology.orchestra.run.smithi143.stdout: "level": "advanced", 2023-11-26T07:23:10.876 INFO:teuthology.orchestra.run.smithi143.stdout: "flags": 0, 2023-11-26T07:23:10.876 INFO:teuthology.orchestra.run.smithi143.stdout: "default_value": "False", 2023-11-26T07:23:10.876 INFO:teuthology.orchestra.run.smithi143.stdout: "min": "", 2023-11-26T07:23:10.876 INFO:teuthology.orchestra.run.smithi143.stdout: "max": "", 2023-11-26T07:23:10.876 INFO:teuthology.orchestra.run.smithi143.stdout: "enum_allowed": [], 2023-11-26T07:23:10.876 INFO:teuthology.orchestra.run.smithi143.stdout: "desc": "Periodically check for async work", 2023-11-26T07:23:10.876 INFO:teuthology.orchestra.run.smithi143.stdout: "long_desc": "", 2023-11-26T07:23:10.877 INFO:teuthology.orchestra.run.smithi143.stdout: "tags": [], 2023-11-26T07:23:10.877 INFO:teuthology.orchestra.run.smithi143.stdout: "see_also": [] 2023-11-26T07:23:10.877 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2023-11-26T07:23:10.877 INFO:teuthology.orchestra.run.smithi143.stdout: "snapshot_clone_delay": { 2023-11-26T07:23:10.877 INFO:teuthology.orchestra.run.smithi143.stdout: "name": "snapshot_clone_delay", 2023-11-26T07:23:10.877 INFO:teuthology.orchestra.run.smithi143.stdout: "type": "int", 2023-11-26T07:23:10.877 INFO:teuthology.orchestra.run.smithi143.stdout: "level": "advanced", 2023-11-26T07:23:10.877 INFO:teuthology.orchestra.run.smithi143.stdout: "flags": 0, 2023-11-26T07:23:10.877 INFO:teuthology.orchestra.run.smithi143.stdout: "default_value": "0", 2023-11-26T07:23:10.877 INFO:teuthology.orchestra.run.smithi143.stdout: "min": "", 2023-11-26T07:23:10.877 INFO:teuthology.orchestra.run.smithi143.stdout: "max": "", 2023-11-26T07:23:10.877 INFO:teuthology.orchestra.run.smithi143.stdout: "enum_allowed": [], 2023-11-26T07:23:10.878 INFO:teuthology.orchestra.run.smithi143.stdout: "desc": "Delay clone begin operation by snapshot_clone_delay seconds", 2023-11-26T07:23:10.878 INFO:teuthology.orchestra.run.smithi143.stdout: "long_desc": "", 2023-11-26T07:23:10.878 INFO:teuthology.orchestra.run.smithi143.stdout: "tags": [], 2023-11-26T07:23:10.878 INFO:teuthology.orchestra.run.smithi143.stdout: "see_also": [] 2023-11-26T07:23:10.878 INFO:teuthology.orchestra.run.smithi143.stdout: } 2023-11-26T07:23:10.878 INFO:teuthology.orchestra.run.smithi143.stdout: } 2023-11-26T07:23:10.878 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2023-11-26T07:23:10.878 INFO:teuthology.orchestra.run.smithi143.stdout: { 2023-11-26T07:23:10.878 INFO:teuthology.orchestra.run.smithi143.stdout: "name": "zabbix", 2023-11-26T07:23:10.878 INFO:teuthology.orchestra.run.smithi143.stdout: "can_run": true, 2023-11-26T07:23:10.878 INFO:teuthology.orchestra.run.smithi143.stdout: "error_string": "", 2023-11-26T07:23:10.878 INFO:teuthology.orchestra.run.smithi143.stdout: "module_options": { 2023-11-26T07:23:10.879 INFO:teuthology.orchestra.run.smithi143.stdout: "discovery_interval": { 2023-11-26T07:23:10.879 INFO:teuthology.orchestra.run.smithi143.stdout: "name": "discovery_interval", 2023-11-26T07:23:10.879 INFO:teuthology.orchestra.run.smithi143.stdout: "type": "uint", 2023-11-26T07:23:10.879 INFO:teuthology.orchestra.run.smithi143.stdout: "level": "advanced", 2023-11-26T07:23:10.879 INFO:teuthology.orchestra.run.smithi143.stdout: "flags": 0, 2023-11-26T07:23:10.879 INFO:teuthology.orchestra.run.smithi143.stdout: "default_value": "100", 2023-11-26T07:23:10.879 INFO:teuthology.orchestra.run.smithi143.stdout: "min": "", 2023-11-26T07:23:10.879 INFO:teuthology.orchestra.run.smithi143.stdout: "max": "", 2023-11-26T07:23:10.879 INFO:teuthology.orchestra.run.smithi143.stdout: "enum_allowed": [], 2023-11-26T07:23:10.879 INFO:teuthology.orchestra.run.smithi143.stdout: "desc": "", 2023-11-26T07:23:10.879 INFO:teuthology.orchestra.run.smithi143.stdout: "long_desc": "", 2023-11-26T07:23:10.879 INFO:teuthology.orchestra.run.smithi143.stdout: "tags": [], 2023-11-26T07:23:10.879 INFO:teuthology.orchestra.run.smithi143.stdout: "see_also": [] 2023-11-26T07:23:10.880 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2023-11-26T07:23:10.880 INFO:teuthology.orchestra.run.smithi143.stdout: "identifier": { 2023-11-26T07:23:10.880 INFO:teuthology.orchestra.run.smithi143.stdout: "name": "identifier", 2023-11-26T07:23:10.880 INFO:teuthology.orchestra.run.smithi143.stdout: "type": "str", 2023-11-26T07:23:10.880 INFO:teuthology.orchestra.run.smithi143.stdout: "level": "advanced", 2023-11-26T07:23:10.880 INFO:teuthology.orchestra.run.smithi143.stdout: "flags": 0, 2023-11-26T07:23:10.880 INFO:teuthology.orchestra.run.smithi143.stdout: "default_value": "", 2023-11-26T07:23:10.880 INFO:teuthology.orchestra.run.smithi143.stdout: "min": "", 2023-11-26T07:23:10.880 INFO:teuthology.orchestra.run.smithi143.stdout: "max": "", 2023-11-26T07:23:10.880 INFO:teuthology.orchestra.run.smithi143.stdout: "enum_allowed": [], 2023-11-26T07:23:10.880 INFO:teuthology.orchestra.run.smithi143.stdout: "desc": "", 2023-11-26T07:23:10.880 INFO:teuthology.orchestra.run.smithi143.stdout: "long_desc": "", 2023-11-26T07:23:10.881 INFO:teuthology.orchestra.run.smithi143.stdout: "tags": [], 2023-11-26T07:23:10.881 INFO:teuthology.orchestra.run.smithi143.stdout: "see_also": [] 2023-11-26T07:23:10.881 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2023-11-26T07:23:10.881 INFO:teuthology.orchestra.run.smithi143.stdout: "interval": { 2023-11-26T07:23:10.881 INFO:teuthology.orchestra.run.smithi143.stdout: "name": "interval", 2023-11-26T07:23:10.881 INFO:teuthology.orchestra.run.smithi143.stdout: "type": "secs", 2023-11-26T07:23:10.881 INFO:teuthology.orchestra.run.smithi143.stdout: "level": "advanced", 2023-11-26T07:23:10.881 INFO:teuthology.orchestra.run.smithi143.stdout: "flags": 0, 2023-11-26T07:23:10.881 INFO:teuthology.orchestra.run.smithi143.stdout: "default_value": "60", 2023-11-26T07:23:10.881 INFO:teuthology.orchestra.run.smithi143.stdout: "min": "", 2023-11-26T07:23:10.881 INFO:teuthology.orchestra.run.smithi143.stdout: "max": "", 2023-11-26T07:23:10.881 INFO:teuthology.orchestra.run.smithi143.stdout: "enum_allowed": [], 2023-11-26T07:23:10.882 INFO:teuthology.orchestra.run.smithi143.stdout: "desc": "", 2023-11-26T07:23:10.882 INFO:teuthology.orchestra.run.smithi143.stdout: "long_desc": "", 2023-11-26T07:23:10.882 INFO:teuthology.orchestra.run.smithi143.stdout: "tags": [], 2023-11-26T07:23:10.882 INFO:teuthology.orchestra.run.smithi143.stdout: "see_also": [] 2023-11-26T07:23:10.882 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2023-11-26T07:23:10.882 INFO:teuthology.orchestra.run.smithi143.stdout: "log_level": { 2023-11-26T07:23:10.882 INFO:teuthology.orchestra.run.smithi143.stdout: "name": "log_level", 2023-11-26T07:23:10.882 INFO:teuthology.orchestra.run.smithi143.stdout: "type": "str", 2023-11-26T07:23:10.882 INFO:teuthology.orchestra.run.smithi143.stdout: "level": "advanced", 2023-11-26T07:23:10.882 INFO:teuthology.orchestra.run.smithi143.stdout: "flags": 1, 2023-11-26T07:23:10.882 INFO:teuthology.orchestra.run.smithi143.stdout: "default_value": "", 2023-11-26T07:23:10.882 INFO:teuthology.orchestra.run.smithi143.stdout: "min": "", 2023-11-26T07:23:10.883 INFO:teuthology.orchestra.run.smithi143.stdout: "max": "", 2023-11-26T07:23:10.883 INFO:teuthology.orchestra.run.smithi143.stdout: "enum_allowed": [ 2023-11-26T07:23:10.883 INFO:teuthology.orchestra.run.smithi143.stdout: "", 2023-11-26T07:23:10.883 INFO:teuthology.orchestra.run.smithi143.stdout: "critical", 2023-11-26T07:23:10.883 INFO:teuthology.orchestra.run.smithi143.stdout: "debug", 2023-11-26T07:23:10.883 INFO:teuthology.orchestra.run.smithi143.stdout: "error", 2023-11-26T07:23:10.883 INFO:teuthology.orchestra.run.smithi143.stdout: "info", 2023-11-26T07:23:10.883 INFO:teuthology.orchestra.run.smithi143.stdout: "warning" 2023-11-26T07:23:10.883 INFO:teuthology.orchestra.run.smithi143.stdout: ], 2023-11-26T07:23:10.883 INFO:teuthology.orchestra.run.smithi143.stdout: "desc": "", 2023-11-26T07:23:10.883 INFO:teuthology.orchestra.run.smithi143.stdout: "long_desc": "", 2023-11-26T07:23:10.883 INFO:teuthology.orchestra.run.smithi143.stdout: "tags": [], 2023-11-26T07:23:10.883 INFO:teuthology.orchestra.run.smithi143.stdout: "see_also": [] 2023-11-26T07:23:10.884 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2023-11-26T07:23:10.884 INFO:teuthology.orchestra.run.smithi143.stdout: "log_to_cluster": { 2023-11-26T07:23:10.884 INFO:teuthology.orchestra.run.smithi143.stdout: "name": "log_to_cluster", 2023-11-26T07:23:10.884 INFO:teuthology.orchestra.run.smithi143.stdout: "type": "bool", 2023-11-26T07:23:10.884 INFO:teuthology.orchestra.run.smithi143.stdout: "level": "advanced", 2023-11-26T07:23:10.884 INFO:teuthology.orchestra.run.smithi143.stdout: "flags": 1, 2023-11-26T07:23:10.884 INFO:teuthology.orchestra.run.smithi143.stdout: "default_value": "False", 2023-11-26T07:23:10.884 INFO:teuthology.orchestra.run.smithi143.stdout: "min": "", 2023-11-26T07:23:10.884 INFO:teuthology.orchestra.run.smithi143.stdout: "max": "", 2023-11-26T07:23:10.884 INFO:teuthology.orchestra.run.smithi143.stdout: "enum_allowed": [], 2023-11-26T07:23:10.884 INFO:teuthology.orchestra.run.smithi143.stdout: "desc": "", 2023-11-26T07:23:10.884 INFO:teuthology.orchestra.run.smithi143.stdout: "long_desc": "", 2023-11-26T07:23:10.885 INFO:teuthology.orchestra.run.smithi143.stdout: "tags": [], 2023-11-26T07:23:10.885 INFO:teuthology.orchestra.run.smithi143.stdout: "see_also": [] 2023-11-26T07:23:10.885 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2023-11-26T07:23:10.885 INFO:teuthology.orchestra.run.smithi143.stdout: "log_to_cluster_level": { 2023-11-26T07:23:10.885 INFO:teuthology.orchestra.run.smithi143.stdout: "name": "log_to_cluster_level", 2023-11-26T07:23:10.885 INFO:teuthology.orchestra.run.smithi143.stdout: "type": "str", 2023-11-26T07:23:10.885 INFO:teuthology.orchestra.run.smithi143.stdout: "level": "advanced", 2023-11-26T07:23:10.885 INFO:teuthology.orchestra.run.smithi143.stdout: "flags": 1, 2023-11-26T07:23:10.885 INFO:teuthology.orchestra.run.smithi143.stdout: "default_value": "info", 2023-11-26T07:23:10.885 INFO:teuthology.orchestra.run.smithi143.stdout: "min": "", 2023-11-26T07:23:10.885 INFO:teuthology.orchestra.run.smithi143.stdout: "max": "", 2023-11-26T07:23:10.885 INFO:teuthology.orchestra.run.smithi143.stdout: "enum_allowed": [ 2023-11-26T07:23:10.886 INFO:teuthology.orchestra.run.smithi143.stdout: "", 2023-11-26T07:23:10.886 INFO:teuthology.orchestra.run.smithi143.stdout: "critical", 2023-11-26T07:23:10.886 INFO:teuthology.orchestra.run.smithi143.stdout: "debug", 2023-11-26T07:23:10.886 INFO:teuthology.orchestra.run.smithi143.stdout: "error", 2023-11-26T07:23:10.886 INFO:teuthology.orchestra.run.smithi143.stdout: "info", 2023-11-26T07:23:10.886 INFO:teuthology.orchestra.run.smithi143.stdout: "warning" 2023-11-26T07:23:10.886 INFO:teuthology.orchestra.run.smithi143.stdout: ], 2023-11-26T07:23:10.886 INFO:teuthology.orchestra.run.smithi143.stdout: "desc": "", 2023-11-26T07:23:10.886 INFO:teuthology.orchestra.run.smithi143.stdout: "long_desc": "", 2023-11-26T07:23:10.886 INFO:teuthology.orchestra.run.smithi143.stdout: "tags": [], 2023-11-26T07:23:10.886 INFO:teuthology.orchestra.run.smithi143.stdout: "see_also": [] 2023-11-26T07:23:10.887 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2023-11-26T07:23:10.887 INFO:teuthology.orchestra.run.smithi143.stdout: "log_to_file": { 2023-11-26T07:23:10.887 INFO:teuthology.orchestra.run.smithi143.stdout: "name": "log_to_file", 2023-11-26T07:23:10.887 INFO:teuthology.orchestra.run.smithi143.stdout: "type": "bool", 2023-11-26T07:23:10.887 INFO:teuthology.orchestra.run.smithi143.stdout: "level": "advanced", 2023-11-26T07:23:10.887 INFO:teuthology.orchestra.run.smithi143.stdout: "flags": 1, 2023-11-26T07:23:10.887 INFO:teuthology.orchestra.run.smithi143.stdout: "default_value": "False", 2023-11-26T07:23:10.887 INFO:teuthology.orchestra.run.smithi143.stdout: "min": "", 2023-11-26T07:23:10.887 INFO:teuthology.orchestra.run.smithi143.stdout: "max": "", 2023-11-26T07:23:10.887 INFO:teuthology.orchestra.run.smithi143.stdout: "enum_allowed": [], 2023-11-26T07:23:10.887 INFO:teuthology.orchestra.run.smithi143.stdout: "desc": "", 2023-11-26T07:23:10.887 INFO:teuthology.orchestra.run.smithi143.stdout: "long_desc": "", 2023-11-26T07:23:10.887 INFO:teuthology.orchestra.run.smithi143.stdout: "tags": [], 2023-11-26T07:23:10.888 INFO:teuthology.orchestra.run.smithi143.stdout: "see_also": [] 2023-11-26T07:23:10.888 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2023-11-26T07:23:10.888 INFO:teuthology.orchestra.run.smithi143.stdout: "zabbix_host": { 2023-11-26T07:23:10.888 INFO:teuthology.orchestra.run.smithi143.stdout: "name": "zabbix_host", 2023-11-26T07:23:10.888 INFO:teuthology.orchestra.run.smithi143.stdout: "type": "str", 2023-11-26T07:23:10.888 INFO:teuthology.orchestra.run.smithi143.stdout: "level": "advanced", 2023-11-26T07:23:10.888 INFO:teuthology.orchestra.run.smithi143.stdout: "flags": 0, 2023-11-26T07:23:10.888 INFO:teuthology.orchestra.run.smithi143.stdout: "default_value": "", 2023-11-26T07:23:10.888 INFO:teuthology.orchestra.run.smithi143.stdout: "min": "", 2023-11-26T07:23:10.888 INFO:teuthology.orchestra.run.smithi143.stdout: "max": "", 2023-11-26T07:23:10.888 INFO:teuthology.orchestra.run.smithi143.stdout: "enum_allowed": [], 2023-11-26T07:23:10.888 INFO:teuthology.orchestra.run.smithi143.stdout: "desc": "", 2023-11-26T07:23:10.888 INFO:teuthology.orchestra.run.smithi143.stdout: "long_desc": "", 2023-11-26T07:23:10.889 INFO:teuthology.orchestra.run.smithi143.stdout: "tags": [], 2023-11-26T07:23:10.889 INFO:teuthology.orchestra.run.smithi143.stdout: "see_also": [] 2023-11-26T07:23:10.889 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2023-11-26T07:23:10.889 INFO:teuthology.orchestra.run.smithi143.stdout: "zabbix_port": { 2023-11-26T07:23:10.889 INFO:teuthology.orchestra.run.smithi143.stdout: "name": "zabbix_port", 2023-11-26T07:23:10.889 INFO:teuthology.orchestra.run.smithi143.stdout: "type": "int", 2023-11-26T07:23:10.889 INFO:teuthology.orchestra.run.smithi143.stdout: "level": "advanced", 2023-11-26T07:23:10.889 INFO:teuthology.orchestra.run.smithi143.stdout: "flags": 0, 2023-11-26T07:23:10.889 INFO:teuthology.orchestra.run.smithi143.stdout: "default_value": "10051", 2023-11-26T07:23:10.889 INFO:teuthology.orchestra.run.smithi143.stdout: "min": "", 2023-11-26T07:23:10.889 INFO:teuthology.orchestra.run.smithi143.stdout: "max": "", 2023-11-26T07:23:10.889 INFO:teuthology.orchestra.run.smithi143.stdout: "enum_allowed": [], 2023-11-26T07:23:10.889 INFO:teuthology.orchestra.run.smithi143.stdout: "desc": "", 2023-11-26T07:23:10.890 INFO:teuthology.orchestra.run.smithi143.stdout: "long_desc": "", 2023-11-26T07:23:10.890 INFO:teuthology.orchestra.run.smithi143.stdout: "tags": [], 2023-11-26T07:23:10.890 INFO:teuthology.orchestra.run.smithi143.stdout: "see_also": [] 2023-11-26T07:23:10.890 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2023-11-26T07:23:10.890 INFO:teuthology.orchestra.run.smithi143.stdout: "zabbix_sender": { 2023-11-26T07:23:10.890 INFO:teuthology.orchestra.run.smithi143.stdout: "name": "zabbix_sender", 2023-11-26T07:23:10.890 INFO:teuthology.orchestra.run.smithi143.stdout: "type": "str", 2023-11-26T07:23:10.890 INFO:teuthology.orchestra.run.smithi143.stdout: "level": "advanced", 2023-11-26T07:23:10.890 INFO:teuthology.orchestra.run.smithi143.stdout: "flags": 0, 2023-11-26T07:23:10.890 INFO:teuthology.orchestra.run.smithi143.stdout: "default_value": "/usr/bin/zabbix_sender", 2023-11-26T07:23:10.890 INFO:teuthology.orchestra.run.smithi143.stdout: "min": "", 2023-11-26T07:23:10.891 INFO:teuthology.orchestra.run.smithi143.stdout: "max": "", 2023-11-26T07:23:10.891 INFO:teuthology.orchestra.run.smithi143.stdout: "enum_allowed": [], 2023-11-26T07:23:10.891 INFO:teuthology.orchestra.run.smithi143.stdout: "desc": "", 2023-11-26T07:23:10.891 INFO:teuthology.orchestra.run.smithi143.stdout: "long_desc": "", 2023-11-26T07:23:10.891 INFO:teuthology.orchestra.run.smithi143.stdout: "tags": [], 2023-11-26T07:23:10.891 INFO:teuthology.orchestra.run.smithi143.stdout: "see_also": [] 2023-11-26T07:23:10.891 INFO:teuthology.orchestra.run.smithi143.stdout: } 2023-11-26T07:23:10.891 INFO:teuthology.orchestra.run.smithi143.stdout: } 2023-11-26T07:23:10.891 INFO:teuthology.orchestra.run.smithi143.stdout: } 2023-11-26T07:23:10.891 INFO:teuthology.orchestra.run.smithi143.stdout: ], 2023-11-26T07:23:10.891 INFO:teuthology.orchestra.run.smithi143.stdout: "services": {}, 2023-11-26T07:23:10.891 INFO:teuthology.orchestra.run.smithi143.stdout: "always_on_modules": { 2023-11-26T07:23:10.891 INFO:teuthology.orchestra.run.smithi143.stdout: "octopus": [ 2023-11-26T07:23:10.892 INFO:teuthology.orchestra.run.smithi143.stdout: "balancer", 2023-11-26T07:23:10.892 INFO:teuthology.orchestra.run.smithi143.stdout: "crash", 2023-11-26T07:23:10.892 INFO:teuthology.orchestra.run.smithi143.stdout: "devicehealth", 2023-11-26T07:23:10.892 INFO:teuthology.orchestra.run.smithi143.stdout: "orchestrator", 2023-11-26T07:23:10.892 INFO:teuthology.orchestra.run.smithi143.stdout: "pg_autoscaler", 2023-11-26T07:23:10.892 INFO:teuthology.orchestra.run.smithi143.stdout: "progress", 2023-11-26T07:23:10.892 INFO:teuthology.orchestra.run.smithi143.stdout: "rbd_support", 2023-11-26T07:23:10.892 INFO:teuthology.orchestra.run.smithi143.stdout: "status", 2023-11-26T07:23:10.892 INFO:teuthology.orchestra.run.smithi143.stdout: "telemetry", 2023-11-26T07:23:10.892 INFO:teuthology.orchestra.run.smithi143.stdout: "volumes" 2023-11-26T07:23:10.892 INFO:teuthology.orchestra.run.smithi143.stdout: ], 2023-11-26T07:23:10.892 INFO:teuthology.orchestra.run.smithi143.stdout: "pacific": [ 2023-11-26T07:23:10.893 INFO:teuthology.orchestra.run.smithi143.stdout: "balancer", 2023-11-26T07:23:10.893 INFO:teuthology.orchestra.run.smithi143.stdout: "crash", 2023-11-26T07:23:10.893 INFO:teuthology.orchestra.run.smithi143.stdout: "devicehealth", 2023-11-26T07:23:10.893 INFO:teuthology.orchestra.run.smithi143.stdout: "orchestrator", 2023-11-26T07:23:10.893 INFO:teuthology.orchestra.run.smithi143.stdout: "pg_autoscaler", 2023-11-26T07:23:10.893 INFO:teuthology.orchestra.run.smithi143.stdout: "progress", 2023-11-26T07:23:10.893 INFO:teuthology.orchestra.run.smithi143.stdout: "rbd_support", 2023-11-26T07:23:10.893 INFO:teuthology.orchestra.run.smithi143.stdout: "status", 2023-11-26T07:23:10.893 INFO:teuthology.orchestra.run.smithi143.stdout: "telemetry", 2023-11-26T07:23:10.893 INFO:teuthology.orchestra.run.smithi143.stdout: "volumes" 2023-11-26T07:23:10.893 INFO:teuthology.orchestra.run.smithi143.stdout: ], 2023-11-26T07:23:10.893 INFO:teuthology.orchestra.run.smithi143.stdout: "quincy": [ 2023-11-26T07:23:10.894 INFO:teuthology.orchestra.run.smithi143.stdout: "balancer", 2023-11-26T07:23:10.894 INFO:teuthology.orchestra.run.smithi143.stdout: "crash", 2023-11-26T07:23:10.894 INFO:teuthology.orchestra.run.smithi143.stdout: "devicehealth", 2023-11-26T07:23:10.894 INFO:teuthology.orchestra.run.smithi143.stdout: "orchestrator", 2023-11-26T07:23:10.894 INFO:teuthology.orchestra.run.smithi143.stdout: "pg_autoscaler", 2023-11-26T07:23:10.894 INFO:teuthology.orchestra.run.smithi143.stdout: "progress", 2023-11-26T07:23:10.894 INFO:teuthology.orchestra.run.smithi143.stdout: "rbd_support", 2023-11-26T07:23:10.894 INFO:teuthology.orchestra.run.smithi143.stdout: "status", 2023-11-26T07:23:10.894 INFO:teuthology.orchestra.run.smithi143.stdout: "telemetry", 2023-11-26T07:23:10.894 INFO:teuthology.orchestra.run.smithi143.stdout: "volumes" 2023-11-26T07:23:10.894 INFO:teuthology.orchestra.run.smithi143.stdout: ], 2023-11-26T07:23:10.894 INFO:teuthology.orchestra.run.smithi143.stdout: "reef": [ 2023-11-26T07:23:10.895 INFO:teuthology.orchestra.run.smithi143.stdout: "balancer", 2023-11-26T07:23:10.895 INFO:teuthology.orchestra.run.smithi143.stdout: "crash", 2023-11-26T07:23:10.895 INFO:teuthology.orchestra.run.smithi143.stdout: "devicehealth", 2023-11-26T07:23:10.895 INFO:teuthology.orchestra.run.smithi143.stdout: "orchestrator", 2023-11-26T07:23:10.895 INFO:teuthology.orchestra.run.smithi143.stdout: "pg_autoscaler", 2023-11-26T07:23:10.895 INFO:teuthology.orchestra.run.smithi143.stdout: "progress", 2023-11-26T07:23:10.895 INFO:teuthology.orchestra.run.smithi143.stdout: "rbd_support", 2023-11-26T07:23:10.895 INFO:teuthology.orchestra.run.smithi143.stdout: "status", 2023-11-26T07:23:10.895 INFO:teuthology.orchestra.run.smithi143.stdout: "telemetry", 2023-11-26T07:23:10.895 INFO:teuthology.orchestra.run.smithi143.stdout: "volumes" 2023-11-26T07:23:10.895 INFO:teuthology.orchestra.run.smithi143.stdout: ] 2023-11-26T07:23:10.895 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2023-11-26T07:23:10.895 INFO:teuthology.orchestra.run.smithi143.stdout: "last_failure_osd_epoch": 0, 2023-11-26T07:23:10.895 INFO:teuthology.orchestra.run.smithi143.stdout: "active_clients": [ 2023-11-26T07:23:10.896 INFO:teuthology.orchestra.run.smithi143.stdout: { 2023-11-26T07:23:10.896 INFO:teuthology.orchestra.run.smithi143.stdout: "name": "devicehealth", 2023-11-26T07:23:10.896 INFO:teuthology.orchestra.run.smithi143.stdout: "addrvec": [ 2023-11-26T07:23:10.896 INFO:teuthology.orchestra.run.smithi143.stdout: { 2023-11-26T07:23:10.896 INFO:teuthology.orchestra.run.smithi143.stdout: "type": "v2", 2023-11-26T07:23:10.896 INFO:teuthology.orchestra.run.smithi143.stdout: "addr": "172.21.15.143:0", 2023-11-26T07:23:10.896 INFO:teuthology.orchestra.run.smithi143.stdout: "nonce": 3516040872 2023-11-26T07:23:10.896 INFO:teuthology.orchestra.run.smithi143.stdout: } 2023-11-26T07:23:10.896 INFO:teuthology.orchestra.run.smithi143.stdout: ] 2023-11-26T07:23:10.896 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2023-11-26T07:23:10.896 INFO:teuthology.orchestra.run.smithi143.stdout: { 2023-11-26T07:23:10.896 INFO:teuthology.orchestra.run.smithi143.stdout: "name": "libcephsqlite", 2023-11-26T07:23:10.897 INFO:teuthology.orchestra.run.smithi143.stdout: "addrvec": [ 2023-11-26T07:23:10.897 INFO:teuthology.orchestra.run.smithi143.stdout: { 2023-11-26T07:23:10.897 INFO:teuthology.orchestra.run.smithi143.stdout: "type": "v2", 2023-11-26T07:23:10.897 INFO:teuthology.orchestra.run.smithi143.stdout: "addr": "172.21.15.143:0", 2023-11-26T07:23:10.897 INFO:teuthology.orchestra.run.smithi143.stdout: "nonce": 1844383386 2023-11-26T07:23:10.897 INFO:teuthology.orchestra.run.smithi143.stdout: } 2023-11-26T07:23:10.897 INFO:teuthology.orchestra.run.smithi143.stdout: ] 2023-11-26T07:23:10.897 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2023-11-26T07:23:10.897 INFO:teuthology.orchestra.run.smithi143.stdout: { 2023-11-26T07:23:10.897 INFO:teuthology.orchestra.run.smithi143.stdout: "name": "rbd_support", 2023-11-26T07:23:10.897 INFO:teuthology.orchestra.run.smithi143.stdout: "addrvec": [ 2023-11-26T07:23:10.897 INFO:teuthology.orchestra.run.smithi143.stdout: { 2023-11-26T07:23:10.897 INFO:teuthology.orchestra.run.smithi143.stdout: "type": "v2", 2023-11-26T07:23:10.898 INFO:teuthology.orchestra.run.smithi143.stdout: "addr": "172.21.15.143:0", 2023-11-26T07:23:10.898 INFO:teuthology.orchestra.run.smithi143.stdout: "nonce": 3813484262 2023-11-26T07:23:10.898 INFO:teuthology.orchestra.run.smithi143.stdout: } 2023-11-26T07:23:10.898 INFO:teuthology.orchestra.run.smithi143.stdout: ] 2023-11-26T07:23:10.898 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2023-11-26T07:23:10.898 INFO:teuthology.orchestra.run.smithi143.stdout: { 2023-11-26T07:23:10.898 INFO:teuthology.orchestra.run.smithi143.stdout: "name": "volumes", 2023-11-26T07:23:10.898 INFO:teuthology.orchestra.run.smithi143.stdout: "addrvec": [ 2023-11-26T07:23:10.898 INFO:teuthology.orchestra.run.smithi143.stdout: { 2023-11-26T07:23:10.898 INFO:teuthology.orchestra.run.smithi143.stdout: "type": "v2", 2023-11-26T07:23:10.898 INFO:teuthology.orchestra.run.smithi143.stdout: "addr": "172.21.15.143:0", 2023-11-26T07:23:10.898 INFO:teuthology.orchestra.run.smithi143.stdout: "nonce": 2591285564 2023-11-26T07:23:10.899 INFO:teuthology.orchestra.run.smithi143.stdout: } 2023-11-26T07:23:10.899 INFO:teuthology.orchestra.run.smithi143.stdout: ] 2023-11-26T07:23:10.899 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2023-11-26T07:23:10.899 INFO:teuthology.orchestra.run.smithi143.stdout: { 2023-11-26T07:23:10.899 INFO:teuthology.orchestra.run.smithi143.stdout: "name": "volumes", 2023-11-26T07:23:10.899 INFO:teuthology.orchestra.run.smithi143.stdout: "addrvec": [ 2023-11-26T07:23:10.899 INFO:teuthology.orchestra.run.smithi143.stdout: { 2023-11-26T07:23:10.899 INFO:teuthology.orchestra.run.smithi143.stdout: "type": "v2", 2023-11-26T07:23:10.899 INFO:teuthology.orchestra.run.smithi143.stdout: "addr": "172.21.15.143:0", 2023-11-26T07:23:10.899 INFO:teuthology.orchestra.run.smithi143.stdout: "nonce": 963510909 2023-11-26T07:23:10.899 INFO:teuthology.orchestra.run.smithi143.stdout: } 2023-11-26T07:23:10.899 INFO:teuthology.orchestra.run.smithi143.stdout: ] 2023-11-26T07:23:10.899 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2023-11-26T07:23:10.900 INFO:teuthology.orchestra.run.smithi143.stdout: { 2023-11-26T07:23:10.900 INFO:teuthology.orchestra.run.smithi143.stdout: "name": "volumes", 2023-11-26T07:23:10.900 INFO:teuthology.orchestra.run.smithi143.stdout: "addrvec": [ 2023-11-26T07:23:10.900 INFO:teuthology.orchestra.run.smithi143.stdout: { 2023-11-26T07:23:10.900 INFO:teuthology.orchestra.run.smithi143.stdout: "type": "v2", 2023-11-26T07:23:10.900 INFO:teuthology.orchestra.run.smithi143.stdout: "addr": "172.21.15.143:0", 2023-11-26T07:23:10.900 INFO:teuthology.orchestra.run.smithi143.stdout: "nonce": 248180418 2023-11-26T07:23:10.900 INFO:teuthology.orchestra.run.smithi143.stdout: } 2023-11-26T07:23:10.900 INFO:teuthology.orchestra.run.smithi143.stdout: ] 2023-11-26T07:23:10.900 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2023-11-26T07:23:10.900 INFO:teuthology.orchestra.run.smithi143.stdout: { 2023-11-26T07:23:10.900 INFO:teuthology.orchestra.run.smithi143.stdout: "name": "volumes", 2023-11-26T07:23:10.900 INFO:teuthology.orchestra.run.smithi143.stdout: "addrvec": [ 2023-11-26T07:23:10.901 INFO:teuthology.orchestra.run.smithi143.stdout: { 2023-11-26T07:23:10.901 INFO:teuthology.orchestra.run.smithi143.stdout: "type": "v2", 2023-11-26T07:23:10.901 INFO:teuthology.orchestra.run.smithi143.stdout: "addr": "172.21.15.143:0", 2023-11-26T07:23:10.901 INFO:teuthology.orchestra.run.smithi143.stdout: "nonce": 2346893898 2023-11-26T07:23:10.901 INFO:teuthology.orchestra.run.smithi143.stdout: } 2023-11-26T07:23:10.901 INFO:teuthology.orchestra.run.smithi143.stdout: ] 2023-11-26T07:23:10.901 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2023-11-26T07:23:10.901 INFO:teuthology.orchestra.run.smithi143.stdout: { 2023-11-26T07:23:10.901 INFO:teuthology.orchestra.run.smithi143.stdout: "name": "volumes", 2023-11-26T07:23:10.901 INFO:teuthology.orchestra.run.smithi143.stdout: "addrvec": [ 2023-11-26T07:23:10.901 INFO:teuthology.orchestra.run.smithi143.stdout: { 2023-11-26T07:23:10.901 INFO:teuthology.orchestra.run.smithi143.stdout: "type": "v2", 2023-11-26T07:23:10.901 INFO:teuthology.orchestra.run.smithi143.stdout: "addr": "172.21.15.143:0", 2023-11-26T07:23:10.902 INFO:teuthology.orchestra.run.smithi143.stdout: "nonce": 103646748 2023-11-26T07:23:10.902 INFO:teuthology.orchestra.run.smithi143.stdout: } 2023-11-26T07:23:10.902 INFO:teuthology.orchestra.run.smithi143.stdout: ] 2023-11-26T07:23:10.902 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2023-11-26T07:23:10.902 INFO:teuthology.orchestra.run.smithi143.stdout: { 2023-11-26T07:23:10.902 INFO:teuthology.orchestra.run.smithi143.stdout: "name": "volumes", 2023-11-26T07:23:10.902 INFO:teuthology.orchestra.run.smithi143.stdout: "addrvec": [ 2023-11-26T07:23:10.902 INFO:teuthology.orchestra.run.smithi143.stdout: { 2023-11-26T07:23:10.902 INFO:teuthology.orchestra.run.smithi143.stdout: "type": "v2", 2023-11-26T07:23:10.902 INFO:teuthology.orchestra.run.smithi143.stdout: "addr": "172.21.15.143:0", 2023-11-26T07:23:10.902 INFO:teuthology.orchestra.run.smithi143.stdout: "nonce": 253926057 2023-11-26T07:23:10.902 INFO:teuthology.orchestra.run.smithi143.stdout: } 2023-11-26T07:23:10.902 INFO:teuthology.orchestra.run.smithi143.stdout: ] 2023-11-26T07:23:10.903 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2023-11-26T07:23:10.903 INFO:teuthology.orchestra.run.smithi143.stdout: { 2023-11-26T07:23:10.903 INFO:teuthology.orchestra.run.smithi143.stdout: "name": "volumes", 2023-11-26T07:23:10.903 INFO:teuthology.orchestra.run.smithi143.stdout: "addrvec": [ 2023-11-26T07:23:10.903 INFO:teuthology.orchestra.run.smithi143.stdout: { 2023-11-26T07:23:10.903 INFO:teuthology.orchestra.run.smithi143.stdout: "type": "v2", 2023-11-26T07:23:10.903 INFO:teuthology.orchestra.run.smithi143.stdout: "addr": "172.21.15.143:0", 2023-11-26T07:23:10.903 INFO:teuthology.orchestra.run.smithi143.stdout: "nonce": 3751308865 2023-11-26T07:23:10.903 INFO:teuthology.orchestra.run.smithi143.stdout: } 2023-11-26T07:23:10.903 INFO:teuthology.orchestra.run.smithi143.stdout: ] 2023-11-26T07:23:10.903 INFO:teuthology.orchestra.run.smithi143.stdout: } 2023-11-26T07:23:10.903 INFO:teuthology.orchestra.run.smithi143.stdout: ] 2023-11-26T07:23:10.903 INFO:teuthology.orchestra.run.smithi143.stdout:} 2023-11-26T07:23:10.905 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:09.995+0000 7f6154806640 1 -- 172.21.15.143:0/2854946509 >> [v2:172.21.15.143:6832/2150557544,v1:172.21.15.143:6833/2150557544] conn(0x7f6130066800 msgr2=0x7f6130068cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:23:10.905 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:09.995+0000 7f6154806640 1 --2- 172.21.15.143:0/2854946509 >> [v2:172.21.15.143:6832/2150557544,v1:172.21.15.143:6833/2150557544] conn(0x7f6130066800 0x7f6130068cc0 secure :-1 s=READY pgs=485 cs=0 l=1 rev1=1 crypto rx=0x7f6140000f70 tx=0x7f6140007cb0 comp rx=0 tx=0).stop 2023-11-26T07:23:10.905 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:09.995+0000 7f6154806640 1 -- 172.21.15.143:0/2854946509 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f61500d1f30 msgr2=0x7f615014f240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:23:10.905 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:09.995+0000 7f6154806640 1 --2- 172.21.15.143:0/2854946509 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f61500d1f30 0x7f615014f240 secure :-1 s=READY pgs=1052 cs=0 l=1 rev1=1 crypto rx=0x7f613800b550 tx=0x7f613800ba10 comp rx=0 tx=0).stop 2023-11-26T07:23:10.905 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:09.995+0000 7f6154806640 1 -- 172.21.15.143:0/2854946509 shutdown_connections 2023-11-26T07:23:10.905 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:09.995+0000 7f6154806640 1 --2- 172.21.15.143:0/2854946509 >> [v2:172.21.15.143:6832/2150557544,v1:172.21.15.143:6833/2150557544] conn(0x7f6130066800 0x7f6130068cc0 unknown :-1 s=CLOSED pgs=485 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:23:10.905 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:09.995+0000 7f6154806640 1 --2- 172.21.15.143:0/2854946509 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f615015d5b0 0x7f615015a390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:23:10.905 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:09.995+0000 7f6154806640 1 --2- 172.21.15.143:0/2854946509 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f61500d1f30 0x7f615014f240 unknown :-1 s=CLOSED pgs=1052 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:23:10.905 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:09.995+0000 7f6154806640 1 --2- 172.21.15.143:0/2854946509 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f61500b36b0 0x7f615015be00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:23:10.906 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:09.995+0000 7f6154806640 1 -- 172.21.15.143:0/2854946509 >> 172.21.15.143:0/2854946509 conn(0x7f615008cc60 msgr2=0x7f61500c7550 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:23:10.906 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:09.995+0000 7f6154806640 1 -- 172.21.15.143:0/2854946509 shutdown_connections 2023-11-26T07:23:10.906 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:09.995+0000 7f6154806640 1 -- 172.21.15.143:0/2854946509 wait complete. 2023-11-26T07:23:10.910 DEBUG:teuthology.orchestra.run.smithi143:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph mgr fail y 2023-11-26T07:23:11.055 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:11.051+0000 7f6830e79640 1 Processor -- start 2023-11-26T07:23:11.055 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:11.055+0000 7f6830e79640 1 -- start start 2023-11-26T07:23:11.056 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:11.055+0000 7f6830e79640 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7f682c13e290 0x7f682c0cf280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:23:11.057 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:11.055+0000 7f6830e79640 1 -- --> v1:172.21.15.143:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f682c15fb20 con 0x7f682c15f6b0 2023-11-26T07:23:11.057 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:11.055+0000 7f6830e79640 1 -- --> v1:172.21.15.188:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f682c06c520 con 0x7f682c0cf7c0 2023-11-26T07:23:11.057 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:11.055+0000 7f6830e79640 1 -- --> v2:172.21.15.188:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f682c0cfc40 con 0x7f682c13e290 2023-11-26T07:23:11.057 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:11.055+0000 7f682ad9d640 1 --1- >> v1:172.21.15.143:6789/0 conn(0x7f682c15f6b0 0x7f682c0cd7b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.143:6789/0 says I am v1:172.21.15.143:54488/0 (socket says 172.21.15.143:54488) 2023-11-26T07:23:11.057 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:11.055+0000 7f682ad9d640 1 -- 172.21.15.143:0/2453890433 learned_addr learned my addr 172.21.15.143:0/2453890433 (peer_addr_for_me v1:172.21.15.143:0/0) 2023-11-26T07:23:11.057 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:11.055+0000 7f682a59c640 1 --2- 172.21.15.143:0/2453890433 >> v2:172.21.15.188:3300/0 conn(0x7f682c13e290 0x7f682c0cf280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:23:11.058 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:11.055+0000 7f682959a640 1 -- 172.21.15.143:0/2453890433 <== mon.0 v1:172.21.15.143:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2287535583 0 0) 0x7f68240014a0 con 0x7f682c15f6b0 2023-11-26T07:23:11.058 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:11.055+0000 7f682959a640 1 -- 172.21.15.143:0/2453890433 --> v1:172.21.15.143:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f6814003530 con 0x7f682c15f6b0 2023-11-26T07:23:11.058 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:11.055+0000 7f682a59c640 1 -- 172.21.15.143:0/2453890433 >> v1:172.21.15.188:6789/0 conn(0x7f682c0cf7c0 legacy=0x7f682c15efa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:23:11.058 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:11.055+0000 7f682a59c640 1 -- 172.21.15.143:0/2453890433 >> v1:172.21.15.143:6789/0 conn(0x7f682c15f6b0 legacy=0x7f682c0cd7b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:23:11.058 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:11.055+0000 7f682a59c640 1 -- 172.21.15.143:0/2453890433 --> v2:172.21.15.188:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f682c06c800 con 0x7f682c13e290 2023-11-26T07:23:11.060 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:11.059+0000 7f682a59c640 1 --2- 172.21.15.143:0/2453890433 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f682c13e290 0x7f682c0cf280 secure :-1 s=READY pgs=411 cs=0 l=1 rev1=1 crypto rx=0x7f681c0097c0 tx=0x7f681c01c450 comp rx=0 tx=0).ready entity=mon.1 client_cookie=fc4bfa8ecf316f44 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:23:11.061 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:11.059+0000 7f682959a640 1 -- 172.21.15.143:0/2453890433 <== mon.1 v2:172.21.15.188:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f681c0ab650 con 0x7f682c13e290 2023-11-26T07:23:11.061 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:11.059+0000 7f682959a640 1 -- 172.21.15.143:0/2453890433 <== mon.1 v2:172.21.15.188:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f681c0ab7b0 con 0x7f682c13e290 2023-11-26T07:23:11.061 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:11.059+0000 7f682959a640 1 -- 172.21.15.143:0/2453890433 <== mon.1 v2:172.21.15.188:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f681c0abb40 con 0x7f682c13e290 2023-11-26T07:23:11.062 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:11.059+0000 7f6830e79640 1 -- 172.21.15.143:0/2453890433 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f682c13e290 msgr2=0x7f682c0cf280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:23:11.062 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:11.059+0000 7f6830e79640 1 --2- 172.21.15.143:0/2453890433 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f682c13e290 0x7f682c0cf280 secure :-1 s=READY pgs=411 cs=0 l=1 rev1=1 crypto rx=0x7f681c0097c0 tx=0x7f681c01c450 comp rx=0 tx=0).stop 2023-11-26T07:23:11.062 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:11.059+0000 7f6830e79640 1 -- 172.21.15.143:0/2453890433 shutdown_connections 2023-11-26T07:23:11.062 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:11.059+0000 7f6830e79640 1 --2- 172.21.15.143:0/2453890433 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f682c13e290 0x7f682c0cf280 unknown :-1 s=CLOSED pgs=411 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:23:11.062 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:11.059+0000 7f6830e79640 1 -- 172.21.15.143:0/2453890433 >> 172.21.15.143:0/2453890433 conn(0x7f682c06a890 msgr2=0x7f682c06aca0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:23:11.062 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:11.059+0000 7f6830e79640 1 -- 172.21.15.143:0/2453890433 shutdown_connections 2023-11-26T07:23:11.062 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:11.059+0000 7f6830e79640 1 -- 172.21.15.143:0/2453890433 wait complete. 2023-11-26T07:23:11.063 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:11.063+0000 7f6830e79640 1 Processor -- start 2023-11-26T07:23:11.063 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:11.063+0000 7f6830e79640 1 -- start start 2023-11-26T07:23:11.064 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:11.063+0000 7f6830e79640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f682c0cf7c0 0x7f682c13b730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:23:11.064 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:11.063+0000 7f6830e79640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f682c13e290 0x7f682c154200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:23:11.065 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:11.063+0000 7f6830e79640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f682c15f6b0 0x7f682c0b0b60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:23:11.065 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:11.063+0000 7f6830e79640 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f682c154740 con 0x7f682c0cf7c0 2023-11-26T07:23:11.065 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:11.063+0000 7f6830e79640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f682c071cf0 con 0x7f682c13e290 2023-11-26T07:23:11.065 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:11.063+0000 7f6830e79640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f682c071fd0 con 0x7f682c15f6b0 2023-11-26T07:23:11.065 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:11.063+0000 7f682a59c640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f682c0cf7c0 0x7f682c13b730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:23:11.065 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:11.063+0000 7f682a59c640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f682c0cf7c0 0x7f682c13b730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:46564/0 (socket says 172.21.15.143:46564) 2023-11-26T07:23:11.065 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:11.063+0000 7f682a59c640 1 -- 172.21.15.143:0/3655492581 learned_addr learned my addr 172.21.15.143:0/3655492581 (peer_addr_for_me v2:172.21.15.143:0/0) 2023-11-26T07:23:11.066 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:11.063+0000 7f682ad9d640 1 --2- 172.21.15.143:0/3655492581 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f682c15f6b0 0x7f682c0b0b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:23:11.066 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:11.063+0000 7f682a59c640 1 -- 172.21.15.143:0/3655492581 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f682c15f6b0 msgr2=0x7f682c0b0b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:23:11.067 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:11.063+0000 7f6829d9b640 1 --2- 172.21.15.143:0/3655492581 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f682c13e290 0x7f682c154200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:23:11.067 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:11.063+0000 7f682a59c640 1 --2- 172.21.15.143:0/3655492581 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f682c15f6b0 0x7f682c0b0b60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:23:11.067 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:11.063+0000 7f682a59c640 1 -- 172.21.15.143:0/3655492581 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f682c13e290 msgr2=0x7f682c154200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:23:11.067 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:11.063+0000 7f682a59c640 1 --2- 172.21.15.143:0/3655492581 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f682c13e290 0x7f682c154200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:23:11.067 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:11.063+0000 7f682a59c640 1 -- 172.21.15.143:0/3655492581 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f681c009420 con 0x7f682c0cf7c0 2023-11-26T07:23:11.067 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:11.063+0000 7f682ad9d640 1 --2- 172.21.15.143:0/3655492581 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f682c15f6b0 0x7f682c0b0b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-26T07:23:11.067 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:11.063+0000 7f6829d9b640 1 --2- 172.21.15.143:0/3655492581 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f682c13e290 0x7f682c154200 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-26T07:23:11.067 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:11.063+0000 7f682a59c640 1 --2- 172.21.15.143:0/3655492581 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f682c0cf7c0 0x7f682c13b730 secure :-1 s=READY pgs=1054 cs=0 l=1 rev1=1 crypto rx=0x7f681c019060 tx=0x7f681c01ac90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:23:11.067 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:11.063+0000 7f681affd640 1 -- 172.21.15.143:0/3655492581 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f681c0b5810 con 0x7f682c0cf7c0 2023-11-26T07:23:11.067 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:11.063+0000 7f681affd640 1 -- 172.21.15.143:0/3655492581 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f681c0b5990 con 0x7f682c0cf7c0 2023-11-26T07:23:11.067 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:11.063+0000 7f681affd640 1 -- 172.21.15.143:0/3655492581 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f681c0b5d40 con 0x7f682c0cf7c0 2023-11-26T07:23:11.068 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:11.063+0000 7f6830e79640 1 -- 172.21.15.143:0/3655492581 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f682c0af370 con 0x7f682c0cf7c0 2023-11-26T07:23:11.068 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:11.063+0000 7f6830e79640 1 -- 172.21.15.143:0/3655492581 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f682c0722b0 con 0x7f682c0cf7c0 2023-11-26T07:23:11.068 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:11.063+0000 7f6830e79640 1 -- 172.21.15.143:0/3655492581 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f682c0af1d0 con 0x7f682c0cf7c0 2023-11-26T07:23:11.068 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:11.067+0000 7f681affd640 1 -- 172.21.15.143:0/3655492581 <== mon.0 v2:172.21.15.143:3300/0 4 ==== mgrmap(e 12) v1 ==== 86209+0+0 (secure 0 0 0) 0x7f681c0b4390 con 0x7f682c0cf7c0 2023-11-26T07:23:11.069 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:11.067+0000 7f681affd640 1 --2- 172.21.15.143:0/3655492581 >> [v2:172.21.15.143:6832/2150557544,v1:172.21.15.143:6833/2150557544] conn(0x7f680c066860 0x7f680c068d20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:23:11.069 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:11.067+0000 7f681affd640 1 -- 172.21.15.143:0/3655492581 <== mon.0 v2:172.21.15.143:3300/0 5 ==== osd_map(71..71 src has 1..71) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f681c11edd0 con 0x7f682c0cf7c0 2023-11-26T07:23:11.069 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:11.067+0000 7f6829d9b640 1 --2- 172.21.15.143:0/3655492581 >> [v2:172.21.15.143:6832/2150557544,v1:172.21.15.143:6833/2150557544] conn(0x7f680c066860 0x7f680c068d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:23:11.069 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:11.067+0000 7f6829d9b640 1 --2- 172.21.15.143:0/3655492581 >> [v2:172.21.15.143:6832/2150557544,v1:172.21.15.143:6833/2150557544] conn(0x7f680c066860 0x7f680c068d20 secure :-1 s=READY pgs=486 cs=0 l=1 rev1=1 crypto rx=0x7f682c136e20 tx=0x7f6820009040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:23:11.073 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:11.071+0000 7f681affd640 1 -- 172.21.15.143:0/3655492581 <== mon.0 v2:172.21.15.143:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172649 (secure 0 0 0) 0x7f682c0af1d0 con 0x7f682c0cf7c0 2023-11-26T07:23:11.259 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:11.259+0000 7f6830e79640 1 -- 172.21.15.143:0/3655492581 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "mgr fail", "who": "y"} v 0) v1 -- 0x7f682c0af1d0 con 0x7f682c0cf7c0 2023-11-26T07:23:11.263 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:11.263+0000 7f681affd640 1 -- 172.21.15.143:0/3655492581 <== mon.0 v2:172.21.15.143:3300/0 7 ==== mon_command_ack([{"prefix": "mgr fail", "who": "y"}]=0 v13) v1 ==== 68+0+0 (secure 0 0 0) 0x7f682c0af1d0 con 0x7f682c0cf7c0 2023-11-26T07:23:11.266 INFO:tasks.ceph.mgr.y.smithi143.stderr:2023-11-26T07:23:11.263+0000 7f50be245640 -1 mgr handle_mgr_map I was active but no longer am 2023-11-26T07:23:11.267 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:11.267+0000 7f6830e79640 1 -- 172.21.15.143:0/3655492581 >> [v2:172.21.15.143:6832/2150557544,v1:172.21.15.143:6833/2150557544] conn(0x7f680c066860 msgr2=0x7f680c068d20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:23:11.267 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:11.267+0000 7f6830e79640 1 --2- 172.21.15.143:0/3655492581 >> [v2:172.21.15.143:6832/2150557544,v1:172.21.15.143:6833/2150557544] conn(0x7f680c066860 0x7f680c068d20 secure :-1 s=READY pgs=486 cs=0 l=1 rev1=1 crypto rx=0x7f682c136e20 tx=0x7f6820009040 comp rx=0 tx=0).stop 2023-11-26T07:23:11.267 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:11.267+0000 7f6830e79640 1 -- 172.21.15.143:0/3655492581 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f682c0cf7c0 msgr2=0x7f682c13b730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:23:11.267 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:11.267+0000 7f6830e79640 1 --2- 172.21.15.143:0/3655492581 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f682c0cf7c0 0x7f682c13b730 secure :-1 s=READY pgs=1054 cs=0 l=1 rev1=1 crypto rx=0x7f681c019060 tx=0x7f681c01ac90 comp rx=0 tx=0).stop 2023-11-26T07:23:11.268 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:11.267+0000 7f6830e79640 1 -- 172.21.15.143:0/3655492581 shutdown_connections 2023-11-26T07:23:11.268 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:11.267+0000 7f6830e79640 1 --2- 172.21.15.143:0/3655492581 >> [v2:172.21.15.143:6832/2150557544,v1:172.21.15.143:6833/2150557544] conn(0x7f680c066860 0x7f680c068d20 secure :-1 s=CLOSED pgs=486 cs=0 l=1 rev1=1 crypto rx=0x7f682c136e20 tx=0x7f6820009040 comp rx=0 tx=0).stop 2023-11-26T07:23:11.268 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:11.267+0000 7f6830e79640 1 --2- 172.21.15.143:0/3655492581 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f682c15f6b0 0x7f682c0b0b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:23:11.268 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:11.267+0000 7f6830e79640 1 --2- 172.21.15.143:0/3655492581 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f682c13e290 0x7f682c154200 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:23:11.268 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:11.267+0000 7f6830e79640 1 --2- 172.21.15.143:0/3655492581 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f682c0cf7c0 0x7f682c13b730 unknown :-1 s=CLOSED pgs=1054 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:23:11.268 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:11.267+0000 7f6830e79640 1 -- 172.21.15.143:0/3655492581 >> 172.21.15.143:0/3655492581 conn(0x7f682c06a890 msgr2=0x7f682c0b1400 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:23:11.269 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:11.267+0000 7f6830e79640 1 -- 172.21.15.143:0/3655492581 shutdown_connections 2023-11-26T07:23:11.269 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:11.267+0000 7f6830e79640 1 -- 172.21.15.143:0/3655492581 wait complete. 2023-11-26T07:23:11.311 DEBUG:teuthology.orchestra.run.smithi143:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-11-26T07:23:11.373 INFO:tasks.ceph.mgr.x.smithi188.stderr:2023-11-26T07:23:11.369+0000 7f04b338e640 -1 client.0 error registering admin socket command: (17) File exists 2023-11-26T07:23:11.374 INFO:tasks.ceph.mgr.x.smithi188.stderr:2023-11-26T07:23:11.369+0000 7f04b338e640 -1 client.0 error registering admin socket command: (17) File exists 2023-11-26T07:23:11.374 INFO:tasks.ceph.mgr.x.smithi188.stderr:2023-11-26T07:23:11.369+0000 7f04b338e640 -1 client.0 error registering admin socket command: (17) File exists 2023-11-26T07:23:11.374 INFO:tasks.ceph.mgr.x.smithi188.stderr:2023-11-26T07:23:11.369+0000 7f04b338e640 -1 client.0 error registering admin socket command: (17) File exists 2023-11-26T07:23:11.374 INFO:tasks.ceph.mgr.x.smithi188.stderr:2023-11-26T07:23:11.369+0000 7f04b338e640 -1 client.0 error registering admin socket command: (17) File exists 2023-11-26T07:23:11.374 INFO:tasks.ceph.mgr.x.smithi188.stderr:2023-11-26T07:23:11.369+0000 7f04b0388640 -1 client.0 error registering admin socket command: (17) File exists 2023-11-26T07:23:11.374 INFO:tasks.ceph.mgr.x.smithi188.stderr:2023-11-26T07:23:11.369+0000 7f04b0388640 -1 client.0 error registering admin socket command: (17) File exists 2023-11-26T07:23:11.375 INFO:tasks.ceph.mgr.x.smithi188.stderr:2023-11-26T07:23:11.369+0000 7f04b0388640 -1 client.0 error registering admin socket command: (17) File exists 2023-11-26T07:23:11.375 INFO:tasks.ceph.mgr.x.smithi188.stderr:2023-11-26T07:23:11.369+0000 7f04b0388640 -1 client.0 error registering admin socket command: (17) File exists 2023-11-26T07:23:11.375 INFO:tasks.ceph.mgr.x.smithi188.stderr:2023-11-26T07:23:11.369+0000 7f04b0388640 -1 client.0 error registering admin socket command: (17) File exists 2023-11-26T07:23:11.490 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:11.487+0000 7f31c3479640 1 Processor -- start 2023-11-26T07:23:11.490 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:11.487+0000 7f31c3479640 1 -- start start 2023-11-26T07:23:11.492 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:11.491+0000 7f31c3479640 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7f31bc15d6e0 0x7f31bc0d0df0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:23:11.492 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:11.491+0000 7f31c3479640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f31bc0b2160 0x7f31bc0d14b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:23:11.492 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:11.491+0000 7f31c3479640 1 -- --> v1:172.21.15.143:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f31bc0cf0e0 con 0x7f31bc0d19f0 2023-11-26T07:23:11.492 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:11.491+0000 7f31c3479640 1 -- --> v2:172.21.15.143:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f31bc15dac0 con 0x7f31bc15d6e0 2023-11-26T07:23:11.492 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:11.491+0000 7f31c3479640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f31bc0d1e80 con 0x7f31bc0b2160 2023-11-26T07:23:11.493 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:11.491+0000 7f31c1215640 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7f31bc15d6e0 0x7f31bc0d0df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:23:11.493 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:11.491+0000 7f31c1a16640 1 --1- >> v1:172.21.15.143:6789/0 conn(0x7f31bc0d19f0 0x7f31bc0cfbf0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.143:6789/0 says I am v1:172.21.15.143:54494/0 (socket says 172.21.15.143:54494) 2023-11-26T07:23:11.493 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:11.491+0000 7f31c1a16640 1 -- 172.21.15.143:0/3587636931 learned_addr learned my addr 172.21.15.143:0/3587636931 (peer_addr_for_me v1:172.21.15.143:0/0) 2023-11-26T07:23:11.493 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:11.491+0000 7f31c0a14640 1 --2- 172.21.15.143:0/3587636931 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f31bc0b2160 0x7f31bc0d14b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:23:11.493 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:11.491+0000 7f31c1215640 1 -- 172.21.15.143:0/3587636931 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f31bc0b2160 msgr2=0x7f31bc0d14b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-26T07:23:11.493 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:11.491+0000 7f31c1215640 1 --2- 172.21.15.143:0/3587636931 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f31bc0b2160 0x7f31bc0d14b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:23:11.493 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:11.491+0000 7f31c1215640 1 -- 172.21.15.143:0/3587636931 >> v1:172.21.15.143:6789/0 conn(0x7f31bc0d19f0 legacy=0x7f31bc0cfbf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:23:11.493 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:11.491+0000 7f31c1215640 1 -- 172.21.15.143:0/3587636931 --> v2:172.21.15.143:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f31bc0ceb20 con 0x7f31bc15d6e0 2023-11-26T07:23:11.493 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:11.491+0000 7f31b3fff640 1 -- 172.21.15.143:0/3587636931 <== mon.0 v1:172.21.15.143:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3959459960 0 0) 0x7f31bc0cf0e0 con 0x7f31bc0d19f0 2023-11-26T07:23:11.494 INFO:tasks.ceph.mgr.y.smithi143.stderr:2023-11-26T07:23:11.491+0000 7f85d6276180 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2023-11-26T07:23:11.495 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:11.491+0000 7f31c1215640 1 --2- 172.21.15.143:0/3587636931 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f31bc15d6e0 0x7f31bc0d0df0 secure :-1 s=READY pgs=1060 cs=0 l=1 rev1=1 crypto rx=0x7f31b80097c0 tx=0x7f31b801c480 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e0c7485573a22045 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:23:11.495 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:11.495+0000 7f31b3fff640 1 -- 172.21.15.143:0/3587636931 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f31b80ab600 con 0x7f31bc15d6e0 2023-11-26T07:23:11.495 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:11.495+0000 7f31b3fff640 1 -- 172.21.15.143:0/3587636931 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f31b80a7050 con 0x7f31bc15d6e0 2023-11-26T07:23:11.495 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:11.495+0000 7f31b3fff640 1 -- 172.21.15.143:0/3587636931 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f31b80ab980 con 0x7f31bc15d6e0 2023-11-26T07:23:11.496 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:11.495+0000 7f31c3479640 1 -- 172.21.15.143:0/3587636931 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f31bc15d6e0 msgr2=0x7f31bc0d0df0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:23:11.496 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:11.495+0000 7f31c3479640 1 --2- 172.21.15.143:0/3587636931 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f31bc15d6e0 0x7f31bc0d0df0 secure :-1 s=READY pgs=1060 cs=0 l=1 rev1=1 crypto rx=0x7f31b80097c0 tx=0x7f31b801c480 comp rx=0 tx=0).stop 2023-11-26T07:23:11.496 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:11.495+0000 7f31c3479640 1 -- 172.21.15.143:0/3587636931 shutdown_connections 2023-11-26T07:23:11.496 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:11.495+0000 7f31c3479640 1 --2- 172.21.15.143:0/3587636931 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f31bc0b2160 0x7f31bc0d14b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:23:11.496 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:11.495+0000 7f31c3479640 1 --2- 172.21.15.143:0/3587636931 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f31bc15d6e0 0x7f31bc0d0df0 unknown :-1 s=CLOSED pgs=1060 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:23:11.497 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:11.495+0000 7f31c3479640 1 -- 172.21.15.143:0/3587636931 >> 172.21.15.143:0/3587636931 conn(0x7f31bc09c550 msgr2=0x7f31bc0d3220 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:23:11.497 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:11.495+0000 7f31c3479640 1 -- 172.21.15.143:0/3587636931 shutdown_connections 2023-11-26T07:23:11.497 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:11.495+0000 7f31c3479640 1 -- 172.21.15.143:0/3587636931 wait complete. 2023-11-26T07:23:11.497 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:11.495+0000 7f31c3479640 1 Processor -- start 2023-11-26T07:23:11.498 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:11.495+0000 7f31c3479640 1 -- start start 2023-11-26T07:23:11.498 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:11.495+0000 7f31c3479640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f31bc0b2160 0x7f31bc1607c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:23:11.499 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:11.495+0000 7f31c3479640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f31bc0d19f0 0x7f31bc14c0f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:23:11.499 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:11.495+0000 7f31c3479640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f31bc15d6e0 0x7f31bc159d70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:23:11.499 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:11.495+0000 7f31c3479640 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f31bc0ccd40 con 0x7f31bc0d19f0 2023-11-26T07:23:11.499 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:11.495+0000 7f31c3479640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f31bc160330 con 0x7f31bc15d6e0 2023-11-26T07:23:11.499 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:11.495+0000 7f31c3479640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f31bc14c840 con 0x7f31bc0b2160 2023-11-26T07:23:11.500 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:11.499+0000 7f31c0a14640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f31bc0d19f0 0x7f31bc14c0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:23:11.500 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:11.499+0000 7f31c0a14640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f31bc0d19f0 0x7f31bc14c0f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:46608/0 (socket says 172.21.15.143:46608) 2023-11-26T07:23:11.501 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:11.499+0000 7f31c1215640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f31bc0b2160 0x7f31bc1607c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:23:11.501 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:11.499+0000 7f31c0a14640 1 -- 172.21.15.143:0/1933249014 learned_addr learned my addr 172.21.15.143:0/1933249014 (peer_addr_for_me v2:172.21.15.143:0/0) 2023-11-26T07:23:11.501 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:11.499+0000 7f31c1a16640 1 --2- 172.21.15.143:0/1933249014 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f31bc15d6e0 0x7f31bc159d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:23:11.501 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:11.499+0000 7f31c0a14640 1 -- 172.21.15.143:0/1933249014 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f31bc0b2160 msgr2=0x7f31bc1607c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:23:11.501 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:11.499+0000 7f31c0a14640 1 --2- 172.21.15.143:0/1933249014 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f31bc0b2160 0x7f31bc1607c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:23:11.501 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:11.499+0000 7f31c0a14640 1 -- 172.21.15.143:0/1933249014 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f31bc15d6e0 msgr2=0x7f31bc159d70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:23:11.501 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:11.499+0000 7f31c0a14640 1 --2- 172.21.15.143:0/1933249014 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f31bc15d6e0 0x7f31bc159d70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:23:11.501 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:11.499+0000 7f31c0a14640 1 -- 172.21.15.143:0/1933249014 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f31b8009420 con 0x7f31bc0d19f0 2023-11-26T07:23:11.502 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:11.499+0000 7f31c0a14640 1 --2- 172.21.15.143:0/1933249014 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f31bc0d19f0 0x7f31bc14c0f0 secure :-1 s=READY pgs=1061 cs=0 l=1 rev1=1 crypto rx=0x7f31ac00e960 tx=0x7f31ac00ee20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:23:11.502 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:11.499+0000 7f31b1ffb640 1 -- 172.21.15.143:0/1933249014 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f31ac009690 con 0x7f31bc0d19f0 2023-11-26T07:23:11.502 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:11.499+0000 7f31b1ffb640 1 -- 172.21.15.143:0/1933249014 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f31ac00ca00 con 0x7f31bc0d19f0 2023-11-26T07:23:11.502 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:11.499+0000 7f31c3479640 1 -- 172.21.15.143:0/1933249014 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f31bc15fd40 con 0x7f31bc0d19f0 2023-11-26T07:23:11.502 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:11.499+0000 7f31b1ffb640 1 -- 172.21.15.143:0/1933249014 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f31ac00cd70 con 0x7f31bc0d19f0 2023-11-26T07:23:11.502 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:11.499+0000 7f31c3479640 1 -- 172.21.15.143:0/1933249014 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f31bc0cd270 con 0x7f31bc0d19f0 2023-11-26T07:23:11.503 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:11.499+0000 7f31c3479640 1 -- 172.21.15.143:0/1933249014 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f31bc0cd490 con 0x7f31bc0d19f0 2023-11-26T07:23:11.503 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:11.499+0000 7f31b1ffb640 1 -- 172.21.15.143:0/1933249014 <== mon.0 v2:172.21.15.143:3300/0 4 ==== mgrmap(e 13) v1 ==== 42894+0+0 (secure 0 0 0) 0x7f31ac013470 con 0x7f31bc0d19f0 2023-11-26T07:23:11.503 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:11.499+0000 7f31b1ffb640 1 -- 172.21.15.143:0/1933249014 <== mon.0 v2:172.21.15.143:3300/0 5 ==== osd_map(72..72 src has 1..72) v4 ==== 6268+0+0 (secure 0 0 0) 0x7f31ac048190 con 0x7f31bc0d19f0 2023-11-26T07:23:11.507 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:11.503+0000 7f31b1ffb640 1 -- 172.21.15.143:0/1933249014 <== mon.0 v2:172.21.15.143:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172649 (secure 0 0 0) 0x7f31ac01b3b0 con 0x7f31bc0d19f0 2023-11-26T07:23:11.693 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:11.691+0000 7f31c3479640 1 -- 172.21.15.143:0/1933249014 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f31bc067680 con 0x7f31bc0d19f0 2023-11-26T07:23:11.693 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:11.691+0000 7f31b1ffb640 1 -- 172.21.15.143:0/1933249014 <== mon.0 v2:172.21.15.143:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 43 v43) v1 ==== 94+0+4924 (secure 0 0 0) 0x7f31ac017070 con 0x7f31bc0d19f0 2023-11-26T07:23:11.693 INFO:teuthology.orchestra.run.smithi143.stdout: 2023-11-26T07:23:11.694 INFO:teuthology.orchestra.run.smithi143.stdout:{"epoch":43,"default_fscid":5,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5927,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.143:6837/578144260","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6835","nonce":578144260},{"type":"v1","addr":"172.21.15.143:6837","nonce":578144260}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":38},{"gid":5930,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.188:6834/1048736873","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6832","nonce":1048736873},{"type":"v1","addr":"172.21.15.188:6834","nonce":1048736873}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":38},{"gid":5933,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.188:6835/2063768716","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6833","nonce":2063768716},{"type":"v1","addr":"172.21.15.188:6835","nonce":2063768716}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":38}],"filesystems":[{"mdsmap":{"epoch":42,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-11-26T07:22:37.220113+0000","modified":"2023-11-26T07:22:40.243052+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":6553},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6553":{"gid":6553,"name":"a","rank":0,"incarnation":39,"state":"up:active","state_seq":3,"addr":"172.21.15.143:6836/2590663349","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6834","nonce":2590663349},{"type":"v1","addr":"172.21.15.143:6836","nonce":2590663349}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[11],"metadata_pool":10,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":5}]} 2023-11-26T07:23:11.694 INFO:teuthology.orchestra.run.smithi143.stderr:dumped fsmap epoch 43 2023-11-26T07:23:11.695 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:11.695+0000 7f31c3479640 1 -- 172.21.15.143:0/1933249014 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f31bc0d19f0 msgr2=0x7f31bc14c0f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:23:11.696 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:11.695+0000 7f31c3479640 1 --2- 172.21.15.143:0/1933249014 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f31bc0d19f0 0x7f31bc14c0f0 secure :-1 s=READY pgs=1061 cs=0 l=1 rev1=1 crypto rx=0x7f31ac00e960 tx=0x7f31ac00ee20 comp rx=0 tx=0).stop 2023-11-26T07:23:11.696 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:11.695+0000 7f31c3479640 1 -- 172.21.15.143:0/1933249014 shutdown_connections 2023-11-26T07:23:11.696 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:11.695+0000 7f31c3479640 1 --2- 172.21.15.143:0/1933249014 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f31bc15d6e0 0x7f31bc159d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:23:11.696 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:11.695+0000 7f31c3479640 1 --2- 172.21.15.143:0/1933249014 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f31bc0d19f0 0x7f31bc14c0f0 unknown :-1 s=CLOSED pgs=1061 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:23:11.696 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:11.695+0000 7f31c3479640 1 --2- 172.21.15.143:0/1933249014 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f31bc0b2160 0x7f31bc1607c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:23:11.696 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:11.695+0000 7f31c3479640 1 -- 172.21.15.143:0/1933249014 >> 172.21.15.143:0/1933249014 conn(0x7f31bc09c550 msgr2=0x7f31bc1560f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:23:11.696 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:11.695+0000 7f31c3479640 1 -- 172.21.15.143:0/1933249014 shutdown_connections 2023-11-26T07:23:11.696 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:11.695+0000 7f31c3479640 1 -- 172.21.15.143:0/1933249014 wait complete. 2023-11-26T07:23:11.716 DEBUG:teuthology.orchestra.run.smithi143:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.a.asok --format=json session ls 2023-11-26T07:23:11.879 INFO:teuthology.orchestra.run.smithi143.stdout:[{"id":6182,"entity":{"name":{"type":"client","num":6182},"addr":{"type":"any","addr":"172.21.15.188:0","nonce":2220520583}},"state":"open","num_leases":0,"num_caps":2,"request_load_avg":2,"uptime":0.48399567700000001,"requests_in_flight":0,"num_completed_requests":0,"num_completed_flushes":0,"reconnecting":false,"recall_caps":{"value":0,"halflife":60},"release_caps":{"value":0,"halflife":60},"recall_caps_throttle":{"value":0,"halflife":1.5},"recall_caps_throttle2o":{"value":0,"halflife":0.5},"session_cache_liveness":{"value":1.9977831783155486,"halflife":300},"cap_acquisition":{"value":0,"halflife":30},"last_trim_completed_requests_tid":2,"last_trim_completed_flushes_tid":0,"delegated_inos":[],"inst":"client.6182 172.21.15.188:0/2220520583","completed_requests":[],"prealloc_inos":[],"client_metadata":{"client_features":{"feature_bits":"0x00000000003fffff"},"metric_spec":{"metric_flags":{"feature_bits":"0x000000000000ffff"}},"ceph_sha1":"44c2237fe77713214148c1fd622c872f346f1b99","ceph_version":"ceph version 18.0.0-7484-g44c2237f (44c2237fe77713214148c1fd622c872f346f1b99) reef (dev)","entity_id":"x","hostname":"smithi188","pid":"38042","root":"/"}},{"id":6898,"entity":{"name":{"type":"client","num":6898},"addr":{"type":"any","addr":"172.21.15.188:0","nonce":1236472543}},"state":"open","num_leases":0,"num_caps":2,"request_load_avg":2,"uptime":0.48399567700000001,"requests_in_flight":0,"num_completed_requests":0,"num_completed_flushes":0,"reconnecting":false,"recall_caps":{"value":0,"halflife":60},"release_caps":{"value":0,"halflife":60},"recall_caps_throttle":{"value":0,"halflife":1.5},"recall_caps_throttle2o":{"value":0,"halflife":0.5},"session_cache_liveness":{"value":1.9977831783155486,"halflife":300},"cap_acquisition":{"value":0,"halflife":30},"last_trim_completed_requests_tid":2,"last_trim_completed_flushes_tid":0,"delegated_inos":[],"inst":"client.6898 172.21.15.188:0/1236472543","completed_requests":[],"prealloc_inos":[],"client_metadata":{"client_features":{"feature_bits":"0x00000000003fffff"},"metric_spec":{"metric_flags":{"feature_bits":"0x000000000000ffff"}},"ceph_sha1":"44c2237fe77713214148c1fd622c872f346f1b99","ceph_version":"ceph version 18.0.0-7484-g44c2237f (44c2237fe77713214148c1fd622c872f346f1b99) reef (dev)","entity_id":"x","hostname":"smithi188","pid":"38042","root":"/"}}] 2023-11-26T07:23:11.880 DEBUG:tasks.cephfs.filesystem:_json_asok output [ { "cap_acquisition": { "halflife": 30, "value": 0 }, "client_metadata": { "ceph_sha1": "44c2237fe77713214148c1fd622c872f346f1b99", "ceph_version": "ceph version 18.0.0-7484-g44c2237f (44c2237fe77713214148c1fd622c872f346f1b99) reef (dev)", "client_features": { "feature_bits": "0x00000000003fffff" }, "entity_id": "x", "hostname": "smithi188", "metric_spec": { "metric_flags": { "feature_bits": "0x000000000000ffff" } }, "pid": "38042", "root": "/" }, "completed_requests": [], "delegated_inos": [], "entity": { "addr": { "addr": "172.21.15.188:0", "nonce": 2220520583, "type": "any" }, "name": { "num": 6182, "type": "client" } }, "id": 6182, "inst": "client.6182 172.21.15.188:0/2220520583", "last_trim_completed_flushes_tid": 0, "last_trim_completed_requests_tid": 2, "num_caps": 2, "num_completed_flushes": 0, "num_completed_requests": 0, "num_leases": 0, "prealloc_inos": [], "recall_caps": { "halflife": 60, "value": 0 }, "recall_caps_throttle": { "halflife": 1.5, "value": 0 }, "recall_caps_throttle2o": { "halflife": 0.5, "value": 0 }, "reconnecting": false, "release_caps": { "halflife": 60, "value": 0 }, "request_load_avg": 2, "requests_in_flight": 0, "session_cache_liveness": { "halflife": 300, "value": 1.9977831783155486 }, "state": "open", "uptime": 0.483995677 }, { "cap_acquisition": { "halflife": 30, "value": 0 }, "client_metadata": { "ceph_sha1": "44c2237fe77713214148c1fd622c872f346f1b99", "ceph_version": "ceph version 18.0.0-7484-g44c2237f (44c2237fe77713214148c1fd622c872f346f1b99) reef (dev)", "client_features": { "feature_bits": "0x00000000003fffff" }, "entity_id": "x", "hostname": "smithi188", "metric_spec": { "metric_flags": { "feature_bits": "0x000000000000ffff" } }, "pid": "38042", "root": "/" }, "completed_requests": [], "delegated_inos": [], "entity": { "addr": { "addr": "172.21.15.188:0", "nonce": 1236472543, "type": "any" }, "name": { "num": 6898, "type": "client" } }, "id": 6898, "inst": "client.6898 172.21.15.188:0/1236472543", "last_trim_completed_flushes_tid": 0, "last_trim_completed_requests_tid": 2, "num_caps": 2, "num_completed_flushes": 0, "num_completed_requests": 0, "num_leases": 0, "prealloc_inos": [], "recall_caps": { "halflife": 60, "value": 0 }, "recall_caps_throttle": { "halflife": 1.5, "value": 0 }, "recall_caps_throttle2o": { "halflife": 0.5, "value": 0 }, "reconnecting": false, "release_caps": { "halflife": 60, "value": 0 }, "request_load_avg": 2, "requests_in_flight": 0, "session_cache_liveness": { "halflife": 300, "value": 1.9977831783155486 }, "state": "open", "uptime": 0.483995677 } ] 2023-11-26T07:23:11.880 DEBUG:tasks.ceph_test_case:wait_until_true: success in 0s and 0 retries 2023-11-26T07:23:11.880 INFO:teuthology.nuke.actions:Clearing teuthology firewall rules... 2023-11-26T07:23:11.880 DEBUG:teuthology.orchestra.run.smithi143:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2023-11-26T07:23:11.931 INFO:tasks.ceph.mgr.y.smithi143.stderr:2023-11-26T07:23:11.931+0000 7f85d6276180 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2023-11-26T07:23:11.969 DEBUG:teuthology.orchestra.run.smithi188:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2023-11-26T07:23:12.034 INFO:teuthology.nuke.actions:Cleared teuthology firewall rules. 2023-11-26T07:23:12.035 DEBUG:teuthology.orchestra.run.smithi143:> set -ex 2023-11-26T07:23:12.035 DEBUG:teuthology.orchestra.run.smithi143:> dd if=/proc/self/mounts of=/dev/stdout 2023-11-26T07:23:12.045 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,relatime,size=16354672k,nr_inodes=4088668,mode=755 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3275384k,mode=755 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=3455 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysusers.service ramfs ro,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3275380k,nr_inodes=818845,mode=700,uid=1000,gid=1271 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3275384k,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2023-11-26T07:23:12.046 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi143.front.sepia.ceph.com 2023-11-26T07:23:12.046 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-26T07:23:12.046 DEBUG:teuthology.orchestra.run.smithi143:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2023-11-26T07:23:12.082 INFO:tasks.ceph.mgr.y.smithi143.stderr:2023-11-26T07:23:12.079+0000 7f85d6276180 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2023-11-26T07:23:12.091 INFO:teuthology.orchestra.run.smithi143.stderr:rmdir: failed to remove '/home/ubuntu/cephtest/mnt.0': No such file or directory 2023-11-26T07:23:12.092 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-11-26T07:23:12.093 DEBUG:teuthology.orchestra.run.smithi188:> set -ex 2023-11-26T07:23:12.093 DEBUG:teuthology.orchestra.run.smithi188:> dd if=/proc/self/mounts of=/dev/stdout 2023-11-26T07:23:12.101 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,relatime,size=16354628k,nr_inodes=4088657,mode=755 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3275376k,mode=755 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=787 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysusers.service ramfs ro,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3275372k,nr_inodes=818843,mode=700,uid=1000,gid=1271 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3275376k,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2023-11-26T07:23:12.101 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi188.front.sepia.ceph.com 2023-11-26T07:23:12.101 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-26T07:23:12.101 DEBUG:teuthology.orchestra.run.smithi188:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.1) 2023-11-26T07:23:12.155 INFO:teuthology.orchestra.run.smithi188.stderr:rmdir: failed to remove '/home/ubuntu/cephtest/mnt.1': No such file or directory 2023-11-26T07:23:12.156 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-11-26T07:23:12.156 DEBUG:teuthology.orchestra.run.smithi143:> set -ex 2023-11-26T07:23:12.156 DEBUG:teuthology.orchestra.run.smithi143:> dd if=/proc/self/mounts of=/dev/stdout 2023-11-26T07:23:12.167 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,relatime,size=16354672k,nr_inodes=4088668,mode=755 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3275384k,mode=755 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=3455 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysusers.service ramfs ro,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3275380k,nr_inodes=818845,mode=700,uid=1000,gid=1271 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3275384k,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2023-11-26T07:23:12.167 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi143.front.sepia.ceph.com 2023-11-26T07:23:12.167 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-26T07:23:12.167 DEBUG:teuthology.orchestra.run.smithi143:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.2) 2023-11-26T07:23:12.220 INFO:teuthology.orchestra.run.smithi143.stderr:rmdir: failed to remove '/home/ubuntu/cephtest/mnt.2': No such file or directory 2023-11-26T07:23:12.220 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-11-26T07:23:12.221 DEBUG:teuthology.orchestra.run.smithi188:> set -ex 2023-11-26T07:23:12.221 DEBUG:teuthology.orchestra.run.smithi188:> dd if=/proc/self/mounts of=/dev/stdout 2023-11-26T07:23:12.230 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,relatime,size=16354628k,nr_inodes=4088657,mode=755 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3275376k,mode=755 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=787 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysusers.service ramfs ro,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3275372k,nr_inodes=818843,mode=700,uid=1000,gid=1271 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3275376k,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2023-11-26T07:23:12.230 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi188.front.sepia.ceph.com 2023-11-26T07:23:12.230 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-26T07:23:12.230 DEBUG:teuthology.orchestra.run.smithi188:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.3) 2023-11-26T07:23:12.284 INFO:teuthology.orchestra.run.smithi188.stderr:rmdir: failed to remove '/home/ubuntu/cephtest/mnt.3': No such file or directory 2023-11-26T07:23:12.284 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-11-26T07:23:12.285 DEBUG:teuthology.orchestra.run.smithi143:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-11-26T07:23:12.426 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:12.423+0000 7f7dd829a640 1 Processor -- start 2023-11-26T07:23:12.426 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:12.423+0000 7f7dd829a640 1 -- start start 2023-11-26T07:23:12.428 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:12.427+0000 7f7dd829a640 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7f7dd015d6e0 0x7f7dd00d0df0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:23:12.428 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:12.427+0000 7f7dd829a640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f7dd00d1bc0 0x7f7dd00b0750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:23:12.428 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:12.427+0000 7f7dd829a640 1 -- --> v1:172.21.15.188:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7dd00cf0e0 con 0x7f7dd00b2160 2023-11-26T07:23:12.428 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:12.427+0000 7f7dd829a640 1 -- --> v2:172.21.15.188:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7dd015dac0 con 0x7f7dd015d6e0 2023-11-26T07:23:12.428 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:12.427+0000 7f7dd829a640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7dd00ceb20 con 0x7f7dd00d1bc0 2023-11-26T07:23:12.429 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:12.427+0000 7f7dd6036640 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7f7dd015d6e0 0x7f7dd00d0df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:23:12.429 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:12.427+0000 7f7dd6837640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f7dd00d1bc0 0x7f7dd00b0750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:23:12.429 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:12.427+0000 7f7dd6036640 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7f7dd015d6e0 0x7f7dd00d0df0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.188:3300/0 says I am v2:172.21.15.143:45002/0 (socket says 172.21.15.143:45002) 2023-11-26T07:23:12.429 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:12.427+0000 7f7dd6837640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f7dd00d1bc0 0x7f7dd00b0750 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.188:3301/0 says I am v2:172.21.15.143:36886/0 (socket says 172.21.15.143:36886) 2023-11-26T07:23:12.429 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:12.427+0000 7f7dd6036640 1 -- 172.21.15.143:0/3943318263 learned_addr learned my addr 172.21.15.143:0/3943318263 (peer_addr_for_me v2:172.21.15.143:0/0) 2023-11-26T07:23:12.430 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:12.427+0000 7f7dd5835640 1 --1- 172.21.15.143:0/3943318263 >> v1:172.21.15.188:6789/0 conn(0x7f7dd00b2160 0x7f7dd00d14b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.188:6789/0 says I am v1:172.21.15.143:45056/0 (socket says 172.21.15.143:45056) 2023-11-26T07:23:12.430 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:12.427+0000 7f7dd6036640 1 -- 172.21.15.143:0/3943318263 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f7dd00d1bc0 msgr2=0x7f7dd00b0750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-26T07:23:12.430 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:12.427+0000 7f7dd6036640 1 --2- 172.21.15.143:0/3943318263 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f7dd00d1bc0 0x7f7dd00b0750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:23:12.430 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:12.427+0000 7f7dd6036640 1 -- 172.21.15.143:0/3943318263 >> v1:172.21.15.188:6789/0 conn(0x7f7dd00b2160 legacy=0x7f7dd00d14b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:23:12.430 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:12.427+0000 7f7dd6036640 1 -- 172.21.15.143:0/3943318263 --> v2:172.21.15.188:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7dd00cee00 con 0x7f7dd015d6e0 2023-11-26T07:23:12.430 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:12.427+0000 7f7dd5034640 1 -- 172.21.15.143:0/3943318263 <== mon.1 v1:172.21.15.188:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2882156888 0 0) 0x7f7dd00cf0e0 con 0x7f7dd00b2160 2023-11-26T07:23:12.430 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:12.427+0000 7f7dd6837640 1 --2- 172.21.15.143:0/3943318263 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f7dd00d1bc0 0x7f7dd00b0750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-26T07:23:12.431 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:12.431+0000 7f7dd6036640 1 --2- 172.21.15.143:0/3943318263 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f7dd015d6e0 0x7f7dd00d0df0 secure :-1 s=READY pgs=416 cs=0 l=1 rev1=1 crypto rx=0x7f7dcc009870 tx=0x7f7dcc01c540 comp rx=0 tx=0).ready entity=mon.1 client_cookie=26854ef859f2667f server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:23:12.432 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:12.431+0000 7f7dd5034640 1 -- 172.21.15.143:0/3943318263 <== mon.1 v2:172.21.15.188:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7dcc0a7070 con 0x7f7dd015d6e0 2023-11-26T07:23:12.432 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:12.431+0000 7f7dd5034640 1 -- 172.21.15.143:0/3943318263 <== mon.1 v2:172.21.15.188:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7dcc0ab430 con 0x7f7dd015d6e0 2023-11-26T07:23:12.432 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:12.431+0000 7f7dd5034640 1 -- 172.21.15.143:0/3943318263 <== mon.1 v2:172.21.15.188:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7dcc0ab7c0 con 0x7f7dd015d6e0 2023-11-26T07:23:12.433 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:12.431+0000 7f7dd829a640 1 -- 172.21.15.143:0/3943318263 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f7dd015d6e0 msgr2=0x7f7dd00d0df0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:23:12.433 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:12.431+0000 7f7dd829a640 1 --2- 172.21.15.143:0/3943318263 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f7dd015d6e0 0x7f7dd00d0df0 secure :-1 s=READY pgs=416 cs=0 l=1 rev1=1 crypto rx=0x7f7dcc009870 tx=0x7f7dcc01c540 comp rx=0 tx=0).stop 2023-11-26T07:23:12.433 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:12.431+0000 7f7dd829a640 1 -- 172.21.15.143:0/3943318263 shutdown_connections 2023-11-26T07:23:12.433 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:12.431+0000 7f7dd829a640 1 --2- 172.21.15.143:0/3943318263 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f7dd00d1bc0 0x7f7dd00b0750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:23:12.433 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:12.431+0000 7f7dd829a640 1 --2- 172.21.15.143:0/3943318263 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f7dd015d6e0 0x7f7dd00d0df0 unknown :-1 s=CLOSED pgs=416 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:23:12.433 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:12.431+0000 7f7dd829a640 1 -- 172.21.15.143:0/3943318263 >> 172.21.15.143:0/3943318263 conn(0x7f7dd009c550 msgr2=0x7f7dd00d3220 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:23:12.433 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:12.431+0000 7f7dd829a640 1 -- 172.21.15.143:0/3943318263 shutdown_connections 2023-11-26T07:23:12.434 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:12.431+0000 7f7dd829a640 1 -- 172.21.15.143:0/3943318263 wait complete. 2023-11-26T07:23:12.434 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:12.431+0000 7f7dd829a640 1 Processor -- start 2023-11-26T07:23:12.434 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:12.431+0000 7f7dd829a640 1 -- start start 2023-11-26T07:23:12.436 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:12.435+0000 7f7dd829a640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f7dd00b2160 0x7f7dd0153260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:23:12.436 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:12.435+0000 7f7dd829a640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f7dd00d1bc0 0x7f7dd00c9260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:23:12.436 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:12.435+0000 7f7dd829a640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f7dd015d6e0 0x7f7dd00c97a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:23:12.436 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:12.435+0000 7f7dd829a640 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7dd00c88a0 con 0x7f7dd00b2160 2023-11-26T07:23:12.437 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:12.435+0000 7f7dd829a640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7dd00c8b80 con 0x7f7dd00d1bc0 2023-11-26T07:23:12.437 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:12.435+0000 7f7dd829a640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7dd00c8e60 con 0x7f7dd015d6e0 2023-11-26T07:23:12.437 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:12.435+0000 7f7dd6036640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f7dd00b2160 0x7f7dd0153260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:23:12.437 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:12.435+0000 7f7dd6036640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f7dd00b2160 0x7f7dd0153260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:46624/0 (socket says 172.21.15.143:46624) 2023-11-26T07:23:12.437 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:12.435+0000 7f7dd6036640 1 -- 172.21.15.143:0/3359544693 learned_addr learned my addr 172.21.15.143:0/3359544693 (peer_addr_for_me v2:172.21.15.143:0/0) 2023-11-26T07:23:12.437 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:12.435+0000 7f7dd5835640 1 --2- 172.21.15.143:0/3359544693 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f7dd00d1bc0 0x7f7dd00c9260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:23:12.437 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:12.435+0000 7f7dd6837640 1 --2- 172.21.15.143:0/3359544693 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f7dd015d6e0 0x7f7dd00c97a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:23:12.437 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:12.435+0000 7f7dd6036640 1 -- 172.21.15.143:0/3359544693 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f7dd015d6e0 msgr2=0x7f7dd00c97a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:23:12.437 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:12.435+0000 7f7dd6036640 1 --2- 172.21.15.143:0/3359544693 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f7dd015d6e0 0x7f7dd00c97a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:23:12.437 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:12.435+0000 7f7dd6036640 1 -- 172.21.15.143:0/3359544693 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f7dd00d1bc0 msgr2=0x7f7dd00c9260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:23:12.437 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:12.435+0000 7f7dd6036640 1 --2- 172.21.15.143:0/3359544693 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f7dd00d1bc0 0x7f7dd00c9260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:23:12.438 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:12.435+0000 7f7dd6036640 1 -- 172.21.15.143:0/3359544693 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7dc8009a70 con 0x7f7dd00b2160 2023-11-26T07:23:12.439 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:12.435+0000 7f7dd6036640 1 --2- 172.21.15.143:0/3359544693 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f7dd00b2160 0x7f7dd0153260 secure :-1 s=READY pgs=1062 cs=0 l=1 rev1=1 crypto rx=0x7f7dcc0ad040 tx=0x7f7dcc0192e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:23:12.439 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:12.435+0000 7f7dc6ffd640 1 -- 172.21.15.143:0/3359544693 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7dcc0b4620 con 0x7f7dd00b2160 2023-11-26T07:23:12.439 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:12.435+0000 7f7dc6ffd640 1 -- 172.21.15.143:0/3359544693 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7dcc0b47a0 con 0x7f7dd00b2160 2023-11-26T07:23:12.439 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:12.435+0000 7f7dd829a640 1 -- 172.21.15.143:0/3359544693 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7dcc0094d0 con 0x7f7dd00b2160 2023-11-26T07:23:12.439 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:12.435+0000 7f7dc6ffd640 1 -- 172.21.15.143:0/3359544693 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7dcc0b4b50 con 0x7f7dd00b2160 2023-11-26T07:23:12.439 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:12.435+0000 7f7dd829a640 1 -- 172.21.15.143:0/3359544693 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7dd01537a0 con 0x7f7dd00b2160 2023-11-26T07:23:12.439 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:12.435+0000 7f7dd829a640 1 -- 172.21.15.143:0/3359544693 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7dd0067680 con 0x7f7dd00b2160 2023-11-26T07:23:12.439 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:12.435+0000 7f7dc6ffd640 1 -- 172.21.15.143:0/3359544693 <== mon.0 v2:172.21.15.143:3300/0 4 ==== mgrmap(e 14) v1 ==== 43229+0+0 (secure 0 0 0) 0x7f7dcc0b4cb0 con 0x7f7dd00b2160 2023-11-26T07:23:12.439 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:12.439+0000 7f7dc6ffd640 1 --2- 172.21.15.143:0/3359544693 >> [v2:172.21.15.188:6836/3401128553,v1:172.21.15.188:6837/3401128553] conn(0x7f7db40354e0 0x7f7db40379a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:23:12.439 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:12.439+0000 7f7dc6ffd640 1 -- 172.21.15.143:0/3359544693 <== mon.0 v2:172.21.15.143:3300/0 5 ==== osd_map(72..72 src has 1..72) v4 ==== 6268+0+0 (secure 0 0 0) 0x7f7dcc0e2660 con 0x7f7dd00b2160 2023-11-26T07:23:12.440 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:12.439+0000 7f7dd5835640 1 --2- 172.21.15.143:0/3359544693 >> [v2:172.21.15.188:6836/3401128553,v1:172.21.15.188:6837/3401128553] conn(0x7f7db40354e0 0x7f7db40379a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:23:12.440 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:12.439+0000 7f7dd5835640 1 --2- 172.21.15.143:0/3359544693 >> [v2:172.21.15.188:6836/3401128553,v1:172.21.15.188:6837/3401128553] conn(0x7f7db40354e0 0x7f7db40379a0 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7f7dd015f8a0 tx=0x7f7dc0007d60 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:23:12.443 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:12.443+0000 7f7dc6ffd640 1 -- 172.21.15.143:0/3359544693 <== mon.0 v2:172.21.15.143:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172649 (secure 0 0 0) 0x7f7dcc0aadf0 con 0x7f7dd00b2160 2023-11-26T07:23:12.586 INFO:tasks.ceph.mgr.y.smithi143.stderr:2023-11-26T07:23:12.583+0000 7f85d6276180 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2023-11-26T07:23:12.630 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:12.627+0000 7f7dd829a640 1 -- 172.21.15.143:0/3359544693 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f7dd00c9020 con 0x7f7dd00b2160 2023-11-26T07:23:12.631 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:12.631+0000 7f7dc6ffd640 1 -- 172.21.15.143:0/3359544693 <== mon.0 v2:172.21.15.143:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 43 v43) v1 ==== 94+0+4924 (secure 0 0 0) 0x7f7dcc0ba090 con 0x7f7dd00b2160 2023-11-26T07:23:12.631 INFO:teuthology.orchestra.run.smithi143.stdout: 2023-11-26T07:23:12.631 INFO:teuthology.orchestra.run.smithi143.stdout:{"epoch":43,"default_fscid":5,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5927,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.143:6837/578144260","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6835","nonce":578144260},{"type":"v1","addr":"172.21.15.143:6837","nonce":578144260}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":38},{"gid":5930,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.188:6834/1048736873","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6832","nonce":1048736873},{"type":"v1","addr":"172.21.15.188:6834","nonce":1048736873}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":38},{"gid":5933,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.188:6835/2063768716","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6833","nonce":2063768716},{"type":"v1","addr":"172.21.15.188:6835","nonce":2063768716}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":38}],"filesystems":[{"mdsmap":{"epoch":42,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-11-26T07:22:37.220113+0000","modified":"2023-11-26T07:22:40.243052+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":6553},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6553":{"gid":6553,"name":"a","rank":0,"incarnation":39,"state":"up:active","state_seq":3,"addr":"172.21.15.143:6836/2590663349","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6834","nonce":2590663349},{"type":"v1","addr":"172.21.15.143:6836","nonce":2590663349}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[11],"metadata_pool":10,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":5}]} 2023-11-26T07:23:12.631 INFO:teuthology.orchestra.run.smithi143.stderr:dumped fsmap epoch 43 2023-11-26T07:23:12.633 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:12.631+0000 7f7dd829a640 1 -- 172.21.15.143:0/3359544693 >> [v2:172.21.15.188:6836/3401128553,v1:172.21.15.188:6837/3401128553] conn(0x7f7db40354e0 msgr2=0x7f7db40379a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:23:12.633 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:12.631+0000 7f7dd829a640 1 --2- 172.21.15.143:0/3359544693 >> [v2:172.21.15.188:6836/3401128553,v1:172.21.15.188:6837/3401128553] conn(0x7f7db40354e0 0x7f7db40379a0 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7f7dd015f8a0 tx=0x7f7dc0007d60 comp rx=0 tx=0).stop 2023-11-26T07:23:12.633 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:12.631+0000 7f7dd829a640 1 -- 172.21.15.143:0/3359544693 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f7dd00b2160 msgr2=0x7f7dd0153260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:23:12.633 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:12.631+0000 7f7dd829a640 1 --2- 172.21.15.143:0/3359544693 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f7dd00b2160 0x7f7dd0153260 secure :-1 s=READY pgs=1062 cs=0 l=1 rev1=1 crypto rx=0x7f7dcc0ad040 tx=0x7f7dcc0192e0 comp rx=0 tx=0).stop 2023-11-26T07:23:12.634 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:12.631+0000 7f7dd829a640 1 -- 172.21.15.143:0/3359544693 shutdown_connections 2023-11-26T07:23:12.634 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:12.631+0000 7f7dd829a640 1 --2- 172.21.15.143:0/3359544693 >> [v2:172.21.15.188:6836/3401128553,v1:172.21.15.188:6837/3401128553] conn(0x7f7db40354e0 0x7f7db40379a0 unknown :-1 s=CLOSED pgs=20 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:23:12.634 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:12.631+0000 7f7dd829a640 1 --2- 172.21.15.143:0/3359544693 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f7dd015d6e0 0x7f7dd00c97a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:23:12.634 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:12.631+0000 7f7dd829a640 1 --2- 172.21.15.143:0/3359544693 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f7dd00d1bc0 0x7f7dd00c9260 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:23:12.634 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:12.631+0000 7f7dd829a640 1 --2- 172.21.15.143:0/3359544693 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f7dd00b2160 0x7f7dd0153260 unknown :-1 s=CLOSED pgs=1062 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:23:12.634 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:12.631+0000 7f7dd829a640 1 -- 172.21.15.143:0/3359544693 >> 172.21.15.143:0/3359544693 conn(0x7f7dd009c550 msgr2=0x7f7dd00734a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:23:12.634 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:12.631+0000 7f7dd829a640 1 -- 172.21.15.143:0/3359544693 shutdown_connections 2023-11-26T07:23:12.634 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:12.631+0000 7f7dd829a640 1 -- 172.21.15.143:0/3359544693 wait complete. 2023-11-26T07:23:12.646 INFO:tasks.ceph.mgr.y.smithi143.stderr:2023-11-26T07:23:12.643+0000 7f85d6276180 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2023-11-26T07:23:12.653 DEBUG:teuthology.orchestra.run.smithi143:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-11-26T07:23:12.705 INFO:tasks.ceph.mgr.y.smithi143.stderr:2023-11-26T07:23:12.703+0000 7f85d6276180 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2023-11-26T07:23:12.793 INFO:tasks.ceph.mgr.y.smithi143.stderr:2023-11-26T07:23:12.791+0000 7f85d6276180 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2023-11-26T07:23:12.794 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:12.791+0000 7ff609406640 1 Processor -- start 2023-11-26T07:23:12.794 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:12.791+0000 7ff609406640 1 -- start start 2023-11-26T07:23:12.797 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:12.795+0000 7ff609406640 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7ff60415d680 0x7ff6040d00e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:23:12.797 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:12.795+0000 7ff609406640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7ff6040d1d90 0x7ff6040cec30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:23:12.797 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:12.795+0000 7ff609406640 1 -- --> v1:172.21.15.188:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff6040cdc40 con 0x7ff60415d2a0 2023-11-26T07:23:12.797 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:12.795+0000 7ff609406640 1 -- --> v2:172.21.15.143:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff6040ce7a0 con 0x7ff60415d680 2023-11-26T07:23:12.798 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:12.795+0000 7ff609406640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff6040d1440 con 0x7ff6040d1d90 2023-11-26T07:23:12.798 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:12.795+0000 7ff6037fe640 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7ff60415d680 0x7ff6040d00e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:23:12.798 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:12.795+0000 7ff608c05640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7ff6040d1d90 0x7ff6040cec30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:23:12.798 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:12.795+0000 7ff6037fe640 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7ff60415d680 0x7ff6040d00e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:46626/0 (socket says 172.21.15.143:46626) 2023-11-26T07:23:12.798 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:12.795+0000 7ff6037fe640 1 -- 172.21.15.143:0/454943726 learned_addr learned my addr 172.21.15.143:0/454943726 (peer_addr_for_me v2:172.21.15.143:0/0) 2023-11-26T07:23:12.798 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:12.795+0000 7ff603fff640 1 --1- >> v1:172.21.15.188:6789/0 conn(0x7ff60415d2a0 0x7ff6040d1680 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.188:6789/0 says I am v1:172.21.15.143:45068/0 (socket says 172.21.15.143:45068) 2023-11-26T07:23:12.799 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:12.795+0000 7ff608c05640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7ff6040d1d90 0x7ff6040cec30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.188:3301/0 says I am v2:172.21.15.143:36906/0 (socket says 172.21.15.143:36906) 2023-11-26T07:23:12.799 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:12.795+0000 7ff6037fe640 1 -- 172.21.15.143:0/454943726 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7ff6040d1d90 msgr2=0x7ff6040cec30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-26T07:23:12.799 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:12.795+0000 7ff6037fe640 1 --2- 172.21.15.143:0/454943726 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7ff6040d1d90 0x7ff6040cec30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:23:12.799 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:12.795+0000 7ff6037fe640 1 -- 172.21.15.143:0/454943726 >> v1:172.21.15.188:6789/0 conn(0x7ff60415d2a0 legacy=0x7ff6040d1680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:23:12.799 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:12.795+0000 7ff602ffd640 1 -- 172.21.15.143:0/454943726 <== mon.1 v1:172.21.15.188:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 933528772 0 0) 0x7ff6040cdc40 con 0x7ff60415d2a0 2023-11-26T07:23:12.799 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:12.795+0000 7ff6037fe640 1 -- 172.21.15.143:0/454943726 --> v2:172.21.15.143:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff6040cde70 con 0x7ff60415d680 2023-11-26T07:23:12.800 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:12.799+0000 7ff6037fe640 1 --2- 172.21.15.143:0/454943726 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7ff60415d680 0x7ff6040d00e0 secure :-1 s=READY pgs=1063 cs=0 l=1 rev1=1 crypto rx=0x7ff5f8009870 tx=0x7ff5f801c510 comp rx=0 tx=0).ready entity=mon.0 client_cookie=31a7e9d0472ffe01 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:23:12.800 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:12.799+0000 7ff602ffd640 1 -- 172.21.15.143:0/454943726 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff5f80ab620 con 0x7ff60415d680 2023-11-26T07:23:12.801 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:12.799+0000 7ff602ffd640 1 -- 172.21.15.143:0/454943726 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff5f80ab780 con 0x7ff60415d680 2023-11-26T07:23:12.801 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:12.799+0000 7ff602ffd640 1 -- 172.21.15.143:0/454943726 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff5f80abb10 con 0x7ff60415d680 2023-11-26T07:23:12.801 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:12.799+0000 7ff609406640 1 -- 172.21.15.143:0/454943726 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7ff60415d680 msgr2=0x7ff6040d00e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:23:12.801 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:12.799+0000 7ff609406640 1 --2- 172.21.15.143:0/454943726 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7ff60415d680 0x7ff6040d00e0 secure :-1 s=READY pgs=1063 cs=0 l=1 rev1=1 crypto rx=0x7ff5f8009870 tx=0x7ff5f801c510 comp rx=0 tx=0).stop 2023-11-26T07:23:12.801 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:12.799+0000 7ff609406640 1 -- 172.21.15.143:0/454943726 shutdown_connections 2023-11-26T07:23:12.802 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:12.799+0000 7ff609406640 1 --2- 172.21.15.143:0/454943726 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7ff60415d680 0x7ff6040d00e0 unknown :-1 s=CLOSED pgs=1063 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:23:12.802 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:12.799+0000 7ff609406640 1 --2- 172.21.15.143:0/454943726 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7ff6040d1d90 0x7ff6040cec30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:23:12.802 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:12.799+0000 7ff609406640 1 -- 172.21.15.143:0/454943726 >> 172.21.15.143:0/454943726 conn(0x7ff6040d30a0 msgr2=0x7ff6040d3480 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:23:12.802 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:12.799+0000 7ff609406640 1 -- 172.21.15.143:0/454943726 shutdown_connections 2023-11-26T07:23:12.802 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:12.799+0000 7ff609406640 1 -- 172.21.15.143:0/454943726 wait complete. 2023-11-26T07:23:12.804 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:12.799+0000 7ff609406640 1 Processor -- start 2023-11-26T07:23:12.805 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:12.799+0000 7ff609406640 1 -- start start 2023-11-26T07:23:12.805 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:12.803+0000 7ff609406640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7ff6040d1d90 0x7ff604154960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:23:12.805 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:12.803+0000 7ff609406640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7ff60415d2a0 0x7ff6040ca170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:23:12.805 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:12.803+0000 7ff609406640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7ff60415d680 0x7ff604155790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:23:12.805 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:12.803+0000 7ff609406640 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff604154ea0 con 0x7ff60415d680 2023-11-26T07:23:12.805 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:12.803+0000 7ff609406640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff60415f0d0 con 0x7ff6040d1d90 2023-11-26T07:23:12.805 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:12.803+0000 7ff609406640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff60415f370 con 0x7ff60415d2a0 2023-11-26T07:23:12.806 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:12.803+0000 7ff608c05640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7ff60415d680 0x7ff604155790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:23:12.806 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:12.803+0000 7ff6037fe640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7ff60415d2a0 0x7ff6040ca170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:23:12.806 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:12.803+0000 7ff603fff640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7ff6040d1d90 0x7ff604154960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:23:12.806 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:12.803+0000 7ff6037fe640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7ff60415d2a0 0x7ff6040ca170 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.188:3301/0 says I am v2:172.21.15.143:36914/0 (socket says 172.21.15.143:36914) 2023-11-26T07:23:12.806 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:12.803+0000 7ff6037fe640 1 -- 172.21.15.143:0/2429248506 learned_addr learned my addr 172.21.15.143:0/2429248506 (peer_addr_for_me v2:172.21.15.143:0/0) 2023-11-26T07:23:12.806 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:12.803+0000 7ff608c05640 1 -- 172.21.15.143:0/2429248506 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7ff60415d2a0 msgr2=0x7ff6040ca170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:23:12.806 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:12.803+0000 7ff608c05640 1 --2- 172.21.15.143:0/2429248506 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7ff60415d2a0 0x7ff6040ca170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:23:12.807 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:12.803+0000 7ff608c05640 1 -- 172.21.15.143:0/2429248506 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7ff6040d1d90 msgr2=0x7ff604154960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:23:12.807 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:12.803+0000 7ff608c05640 1 --2- 172.21.15.143:0/2429248506 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7ff6040d1d90 0x7ff604154960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:23:12.807 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:12.803+0000 7ff608c05640 1 -- 172.21.15.143:0/2429248506 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff5f80094d0 con 0x7ff60415d680 2023-11-26T07:23:12.807 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:12.803+0000 7ff608c05640 1 --2- 172.21.15.143:0/2429248506 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7ff60415d680 0x7ff604155790 secure :-1 s=READY pgs=1064 cs=0 l=1 rev1=1 crypto rx=0x7ff5fc00ef90 tx=0x7ff5fc00c490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:23:12.807 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:12.803+0000 7ff600ff9640 1 -- 172.21.15.143:0/2429248506 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff5fc009c00 con 0x7ff60415d680 2023-11-26T07:23:12.807 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:12.803+0000 7ff600ff9640 1 -- 172.21.15.143:0/2429248506 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff5fc00f040 con 0x7ff60415d680 2023-11-26T07:23:12.807 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:12.803+0000 7ff600ff9640 1 -- 172.21.15.143:0/2429248506 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff5fc0146c0 con 0x7ff60415d680 2023-11-26T07:23:12.808 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:12.803+0000 7ff609406640 1 -- 172.21.15.143:0/2429248506 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff60415f650 con 0x7ff60415d680 2023-11-26T07:23:12.808 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:12.803+0000 7ff609406640 1 -- 172.21.15.143:0/2429248506 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff60415fc10 con 0x7ff60415d680 2023-11-26T07:23:12.809 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:12.803+0000 7ff600ff9640 1 -- 172.21.15.143:0/2429248506 <== mon.0 v2:172.21.15.143:3300/0 4 ==== mgrmap(e 14) v1 ==== 43229+0+0 (secure 0 0 0) 0x7ff5fc014820 con 0x7ff60415d680 2023-11-26T07:23:12.809 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:12.803+0000 7ff609406640 1 -- 172.21.15.143:0/2429248506 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff60415fe00 con 0x7ff60415d680 2023-11-26T07:23:12.809 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:12.803+0000 7ff600ff9640 1 --2- 172.21.15.143:0/2429248506 >> [v2:172.21.15.188:6836/3401128553,v1:172.21.15.188:6837/3401128553] conn(0x7ff5f00354e0 0x7ff5f00379a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:23:12.809 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:12.803+0000 7ff600ff9640 1 -- 172.21.15.143:0/2429248506 <== mon.0 v2:172.21.15.143:3300/0 5 ==== osd_map(72..72 src has 1..72) v4 ==== 6268+0+0 (secure 0 0 0) 0x7ff5fc049090 con 0x7ff60415d680 2023-11-26T07:23:12.809 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:12.803+0000 7ff603fff640 1 --2- 172.21.15.143:0/2429248506 >> [v2:172.21.15.188:6836/3401128553,v1:172.21.15.188:6837/3401128553] conn(0x7ff5f00354e0 0x7ff5f00379a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:23:12.809 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:12.807+0000 7ff603fff640 1 --2- 172.21.15.143:0/2429248506 >> [v2:172.21.15.188:6836/3401128553,v1:172.21.15.188:6837/3401128553] conn(0x7ff5f00354e0 0x7ff5f00379a0 secure :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0x7ff5f4007fd0 tx=0x7ff5f4007d60 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:23:12.811 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:12.811+0000 7ff600ff9640 1 -- 172.21.15.143:0/2429248506 <== mon.0 v2:172.21.15.143:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172649 (secure 0 0 0) 0x7ff5fc01d400 con 0x7ff60415d680 2023-11-26T07:23:12.866 INFO:tasks.ceph.mgr.y.smithi143.stderr:2023-11-26T07:23:12.863+0000 7f85d6276180 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2023-11-26T07:23:12.953 INFO:tasks.ceph.mgr.y.smithi143.stderr:2023-11-26T07:23:12.951+0000 7f85d6276180 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2023-11-26T07:23:12.981 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:12.979+0000 7ff609406640 1 -- 172.21.15.143:0/2429248506 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7ff604067500 con 0x7ff60415d680 2023-11-26T07:23:12.982 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:12.979+0000 7ff600ff9640 1 -- 172.21.15.143:0/2429248506 <== mon.0 v2:172.21.15.143:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v72) v1 ==== 74+0+17423 (secure 0 0 0) 0x7ff5fc018030 con 0x7ff60415d680 2023-11-26T07:23:12.983 INFO:teuthology.orchestra.run.smithi143.stdout: 2023-11-26T07:23:12.983 INFO:teuthology.orchestra.run.smithi143.stdout:{"epoch":72,"fsid":"f5033060-2469-4a07-95fe-2b76f5404bc1","created":"2023-11-26T07:17:28.613663+0000","modified":"2023-11-26T07:23:11.261981+0000","last_up_change":"2023-11-26T07:17:41.157813+0000","last_in_change":"2023-11-26T07:17:31.181658+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":11,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-11-26T07:17:41.758597+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":10,"pool_name":"cephfs_metadata","create_time":"2023-11-26T07:22:35.314819+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"70","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":2.5,"score_stable":2.5,"optimal_score":1,"raw_score_acting":2.5,"raw_score_stable":2.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":11,"pool_name":"cephfs_data","create_time":"2023-11-26T07:22:36.219536+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"71","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"ebbee2c6-023e-44b9-9f1d-6f7bfd309730","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":69,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6817","nonce":639807369},{"type":"v1","addr":"172.21.15.143:6819","nonce":639807369}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6821","nonce":639807369},{"type":"v1","addr":"172.21.15.143:6823","nonce":639807369}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6830","nonce":639807369},{"type":"v1","addr":"172.21.15.143:6831","nonce":639807369}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6825","nonce":639807369},{"type":"v1","addr":"172.21.15.143:6828","nonce":639807369}]},"public_addr":"172.21.15.143:6819/639807369","cluster_addr":"172.21.15.143:6823/639807369","heartbeat_back_addr":"172.21.15.143:6831/639807369","heartbeat_front_addr":"172.21.15.143:6828/639807369","state":["exists","up"]},{"osd":1,"uuid":"fec4b8be-7122-49e9-9a19-5850a0700001","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":69,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6816","nonce":1144942530},{"type":"v1","addr":"172.21.15.143:6818","nonce":1144942530}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6820","nonce":1144942530},{"type":"v1","addr":"172.21.15.143:6822","nonce":1144942530}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6827","nonce":1144942530},{"type":"v1","addr":"172.21.15.143:6829","nonce":1144942530}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6824","nonce":1144942530},{"type":"v1","addr":"172.21.15.143:6826","nonce":1144942530}]},"public_addr":"172.21.15.143:6818/1144942530","cluster_addr":"172.21.15.143:6822/1144942530","heartbeat_back_addr":"172.21.15.143:6829/1144942530","heartbeat_front_addr":"172.21.15.143:6826/1144942530","state":["exists","up"]},{"osd":2,"uuid":"2888af5b-f933-46ca-8f95-6dd494ff43f6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":69,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6808","nonce":3909095186},{"type":"v1","addr":"172.21.15.143:6809","nonce":3909095186}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6810","nonce":3909095186},{"type":"v1","addr":"172.21.15.143:6811","nonce":3909095186}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6814","nonce":3909095186},{"type":"v1","addr":"172.21.15.143:6815","nonce":3909095186}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6812","nonce":3909095186},{"type":"v1","addr":"172.21.15.143:6813","nonce":3909095186}]},"public_addr":"172.21.15.143:6809/3909095186","cluster_addr":"172.21.15.143:6811/3909095186","heartbeat_back_addr":"172.21.15.143:6815/3909095186","heartbeat_front_addr":"172.21.15.143:6813/3909095186","state":["exists","up"]},{"osd":3,"uuid":"d1f52435-7bc1-4081-9550-9cd1ba55f549","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":69,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6800","nonce":3854210991},{"type":"v1","addr":"172.21.15.143:6801","nonce":3854210991}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6802","nonce":3854210991},{"type":"v1","addr":"172.21.15.143:6803","nonce":3854210991}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6806","nonce":3854210991},{"type":"v1","addr":"172.21.15.143:6807","nonce":3854210991}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6804","nonce":3854210991},{"type":"v1","addr":"172.21.15.143:6805","nonce":3854210991}]},"public_addr":"172.21.15.143:6801/3854210991","cluster_addr":"172.21.15.143:6803/3854210991","heartbeat_back_addr":"172.21.15.143:6807/3854210991","heartbeat_front_addr":"172.21.15.143:6805/3854210991","state":["exists","up"]},{"osd":4,"uuid":"e2ae8b6e-6e25-49b2-bcbf-aacce68f06a3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":69,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6800","nonce":3924606902},{"type":"v1","addr":"172.21.15.188:6801","nonce":3924606902}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6802","nonce":3924606902},{"type":"v1","addr":"172.21.15.188:6803","nonce":3924606902}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6806","nonce":3924606902},{"type":"v1","addr":"172.21.15.188:6807","nonce":3924606902}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6804","nonce":3924606902},{"type":"v1","addr":"172.21.15.188:6805","nonce":3924606902}]},"public_addr":"172.21.15.188:6801/3924606902","cluster_addr":"172.21.15.188:6803/3924606902","heartbeat_back_addr":"172.21.15.188:6807/3924606902","heartbeat_front_addr":"172.21.15.188:6805/3924606902","state":["exists","up"]},{"osd":5,"uuid":"3cec8a32-7698-4c36-8e89-e4ba11934f33","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":69,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6813","nonce":186841544},{"type":"v1","addr":"172.21.15.188:6815","nonce":186841544}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6817","nonce":186841544},{"type":"v1","addr":"172.21.15.188:6818","nonce":186841544}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6822","nonce":186841544},{"type":"v1","addr":"172.21.15.188:6823","nonce":186841544}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6820","nonce":186841544},{"type":"v1","addr":"172.21.15.188:6821","nonce":186841544}]},"public_addr":"172.21.15.188:6815/186841544","cluster_addr":"172.21.15.188:6818/186841544","heartbeat_back_addr":"172.21.15.188:6823/186841544","heartbeat_front_addr":"172.21.15.188:6821/186841544","state":["exists","up"]},{"osd":6,"uuid":"58a65d5d-e090-473a-81b4-ad38e85029f9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":69,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6808","nonce":2153499405},{"type":"v1","addr":"172.21.15.188:6809","nonce":2153499405}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6810","nonce":2153499405},{"type":"v1","addr":"172.21.15.188:6811","nonce":2153499405}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6816","nonce":2153499405},{"type":"v1","addr":"172.21.15.188:6819","nonce":2153499405}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6812","nonce":2153499405},{"type":"v1","addr":"172.21.15.188:6814","nonce":2153499405}]},"public_addr":"172.21.15.188:6809/2153499405","cluster_addr":"172.21.15.188:6811/2153499405","heartbeat_back_addr":"172.21.15.188:6819/2153499405","heartbeat_front_addr":"172.21.15.188:6814/2153499405","state":["exists","up"]},{"osd":7,"uuid":"ea5b7f70-3554-4cf8-ac30-f7c59b80678d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":69,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6824","nonce":2722096729},{"type":"v1","addr":"172.21.15.188:6825","nonce":2722096729}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6826","nonce":2722096729},{"type":"v1","addr":"172.21.15.188:6827","nonce":2722096729}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6830","nonce":2722096729},{"type":"v1","addr":"172.21.15.188:6831","nonce":2722096729}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6828","nonce":2722096729},{"type":"v1","addr":"172.21.15.188:6829","nonce":2722096729}]},"public_addr":"172.21.15.188:6825/2722096729","cluster_addr":"172.21.15.188:6827/2722096729","heartbeat_back_addr":"172.21.15.188:6831/2722096729","heartbeat_front_addr":"172.21.15.188:6829/2722096729","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-26T07:17:36.995032+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-26T07:17:36.932266+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-26T07:17:36.829304+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-26T07:17:36.804297+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-26T07:17:36.667892+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-26T07:17:36.964868+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-26T07:17:36.827332+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-26T07:17:36.893179+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.143:0/248180418":"2023-11-27T07:23:11.261937+0000","172.21.15.143:0/253926057":"2023-11-27T07:23:11.261937+0000","172.21.15.143:0/3516040872":"2023-11-27T07:23:11.261937+0000","172.21.15.143:0/963510909":"2023-11-27T07:23:11.261937+0000","172.21.15.143:6833/2150557544":"2023-11-27T07:23:11.261937+0000","172.21.15.143:6832/2150557544":"2023-11-27T07:23:11.261937+0000","172.21.15.143:0/2346893898":"2023-11-27T07:23:11.261937+0000","172.21.15.143:0/3751308865":"2023-11-27T07:23:11.261937+0000","172.21.15.143:0/3813484262":"2023-11-27T07:23:11.261937+0000","172.21.15.143:0/1844383386":"2023-11-27T07:23:11.261937+0000","172.21.15.143:0/2591285564":"2023-11-27T07:23:11.261937+0000","172.21.15.143:0/631097144":"2023-11-27T07:23:11.261937+0000","172.21.15.143:0/103646748":"2023-11-27T07:23:11.261937+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-11-26T07:23:12.985 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:12.983+0000 7ff609406640 1 -- 172.21.15.143:0/2429248506 >> [v2:172.21.15.188:6836/3401128553,v1:172.21.15.188:6837/3401128553] conn(0x7ff5f00354e0 msgr2=0x7ff5f00379a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:23:12.985 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:12.983+0000 7ff609406640 1 --2- 172.21.15.143:0/2429248506 >> [v2:172.21.15.188:6836/3401128553,v1:172.21.15.188:6837/3401128553] conn(0x7ff5f00354e0 0x7ff5f00379a0 secure :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0x7ff5f4007fd0 tx=0x7ff5f4007d60 comp rx=0 tx=0).stop 2023-11-26T07:23:12.985 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:12.983+0000 7ff609406640 1 -- 172.21.15.143:0/2429248506 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7ff60415d680 msgr2=0x7ff604155790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:23:12.985 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:12.983+0000 7ff609406640 1 --2- 172.21.15.143:0/2429248506 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7ff60415d680 0x7ff604155790 secure :-1 s=READY pgs=1064 cs=0 l=1 rev1=1 crypto rx=0x7ff5fc00ef90 tx=0x7ff5fc00c490 comp rx=0 tx=0).stop 2023-11-26T07:23:12.986 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:12.983+0000 7ff609406640 1 -- 172.21.15.143:0/2429248506 shutdown_connections 2023-11-26T07:23:12.986 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:12.983+0000 7ff609406640 1 --2- 172.21.15.143:0/2429248506 >> [v2:172.21.15.188:6836/3401128553,v1:172.21.15.188:6837/3401128553] conn(0x7ff5f00354e0 0x7ff5f00379a0 unknown :-1 s=CLOSED pgs=21 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:23:12.986 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:12.983+0000 7ff609406640 1 --2- 172.21.15.143:0/2429248506 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7ff60415d680 0x7ff604155790 unknown :-1 s=CLOSED pgs=1064 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:23:12.986 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:12.983+0000 7ff609406640 1 --2- 172.21.15.143:0/2429248506 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7ff60415d2a0 0x7ff6040ca170 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:23:12.986 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:12.983+0000 7ff609406640 1 --2- 172.21.15.143:0/2429248506 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7ff6040d1d90 0x7ff604154960 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:23:12.986 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:12.983+0000 7ff609406640 1 -- 172.21.15.143:0/2429248506 >> 172.21.15.143:0/2429248506 conn(0x7ff6040d30a0 msgr2=0x7ff6040b1c50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:23:12.986 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:12.983+0000 7ff609406640 1 -- 172.21.15.143:0/2429248506 shutdown_connections 2023-11-26T07:23:12.987 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:12.983+0000 7ff609406640 1 -- 172.21.15.143:0/2429248506 wait complete. 2023-11-26T07:23:13.005 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-11-26T07:17:41.758597+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 10, 'pool_name': 'cephfs_metadata', 'create_time': '2023-11-26T07:22:35.314819+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '70', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 2.5, 'score_stable': 2.5, 'optimal_score': 1, 'raw_score_acting': 2.5, 'raw_score_stable': 2.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 11, 'pool_name': 'cephfs_data', 'create_time': '2023-11-26T07:22:36.219536+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '71', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.3799999952316284, 'score_stable': 1.3799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.3799999952316284, 'raw_score_stable': 1.3799999952316284, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2023-11-26T07:23:13.005 DEBUG:teuthology.orchestra.run.smithi143:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-11-26T07:23:13.014 INFO:tasks.ceph.mgr.y.smithi143.stderr:2023-11-26T07:23:13.011+0000 7f85d6276180 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2023-11-26T07:23:13.149 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:13.147+0000 7f01f2016640 1 Processor -- start 2023-11-26T07:23:13.149 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:13.147+0000 7f01f2016640 1 -- start start 2023-11-26T07:23:13.150 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:13.147+0000 7f01f2016640 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7f01ec0b2160 0x7f01ec0d0df0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:23:13.150 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:13.147+0000 7f01f2016640 1 -- --> v1:172.21.15.143:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f01ec0cf0e0 con 0x7f01ec15d6e0 2023-11-26T07:23:13.151 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:13.147+0000 7f01f2016640 1 -- --> v1:172.21.15.188:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f01ec0ceb20 con 0x7f01ec0d1bc0 2023-11-26T07:23:13.151 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:13.147+0000 7f01f2016640 1 -- --> v2:172.21.15.143:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f01ec15dac0 con 0x7f01ec0b2160 2023-11-26T07:23:13.151 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:13.147+0000 7f01ebfff640 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7f01ec0b2160 0x7f01ec0d0df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:23:13.151 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:13.147+0000 7f01ebfff640 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7f01ec0b2160 0x7f01ec0d0df0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:46648/0 (socket says 172.21.15.143:46648) 2023-11-26T07:23:13.151 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:13.147+0000 7f01ebfff640 1 -- 172.21.15.143:0/2988558451 learned_addr learned my addr 172.21.15.143:0/2988558451 (peer_addr_for_me v2:172.21.15.143:0/0) 2023-11-26T07:23:13.151 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:13.147+0000 7f01eb7fe640 1 --1- 172.21.15.143:0/2988558451 >> v1:172.21.15.143:6789/0 conn(0x7f01ec15d6e0 0x7f01ec0d14b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.143:6789/0 says I am v1:172.21.15.143:54504/0 (socket says 172.21.15.143:54504) 2023-11-26T07:23:13.152 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:13.151+0000 7f01ebfff640 1 -- 172.21.15.143:0/2988558451 >> v1:172.21.15.188:6789/0 conn(0x7f01ec0d1bc0 legacy=0x7f01ec0b0750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:23:13.152 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:13.151+0000 7f01ebfff640 1 -- 172.21.15.143:0/2988558451 >> v1:172.21.15.143:6789/0 conn(0x7f01ec15d6e0 legacy=0x7f01ec0d14b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:23:13.152 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:13.151+0000 7f01ebfff640 1 -- 172.21.15.143:0/2988558451 --> v2:172.21.15.143:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f01ec0cee00 con 0x7f01ec0b2160 2023-11-26T07:23:13.152 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:13.151+0000 7f01ea7fc640 1 -- 172.21.15.143:0/2988558451 <== mon.1 v1:172.21.15.188:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1502051379 0 0) 0x7f01ec0ceb20 con 0x7f01ec0d1bc0 2023-11-26T07:23:13.153 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:13.151+0000 7f01ebfff640 1 --2- 172.21.15.143:0/2988558451 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f01ec0b2160 0x7f01ec0d0df0 secure :-1 s=READY pgs=1066 cs=0 l=1 rev1=1 crypto rx=0x7f01e4009e50 tx=0x7f01e401ca10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5a6746397acb1c73 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:23:13.154 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:13.151+0000 7f01ea7fc640 1 -- 172.21.15.143:0/2988558451 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f01e40a57d0 con 0x7f01ec0b2160 2023-11-26T07:23:13.154 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:13.151+0000 7f01ea7fc640 1 -- 172.21.15.143:0/2988558451 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f01e40a5930 con 0x7f01ec0b2160 2023-11-26T07:23:13.154 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:13.151+0000 7f01ea7fc640 1 -- 172.21.15.143:0/2988558451 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f01e40a5cc0 con 0x7f01ec0b2160 2023-11-26T07:23:13.155 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:13.155+0000 7f01f2016640 1 -- 172.21.15.143:0/2988558451 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f01ec0b2160 msgr2=0x7f01ec0d0df0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:23:13.155 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:13.155+0000 7f01f2016640 1 --2- 172.21.15.143:0/2988558451 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f01ec0b2160 0x7f01ec0d0df0 secure :-1 s=READY pgs=1066 cs=0 l=1 rev1=1 crypto rx=0x7f01e4009e50 tx=0x7f01e401ca10 comp rx=0 tx=0).stop 2023-11-26T07:23:13.156 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:13.155+0000 7f01f2016640 1 -- 172.21.15.143:0/2988558451 shutdown_connections 2023-11-26T07:23:13.156 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:13.155+0000 7f01f2016640 1 --2- 172.21.15.143:0/2988558451 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f01ec0b2160 0x7f01ec0d0df0 unknown :-1 s=CLOSED pgs=1066 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:23:13.156 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:13.155+0000 7f01f2016640 1 -- 172.21.15.143:0/2988558451 >> 172.21.15.143:0/2988558451 conn(0x7f01ec09c550 msgr2=0x7f01ec0d3220 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:23:13.156 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:13.155+0000 7f01f2016640 1 -- 172.21.15.143:0/2988558451 shutdown_connections 2023-11-26T07:23:13.156 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:13.155+0000 7f01f2016640 1 -- 172.21.15.143:0/2988558451 wait complete. 2023-11-26T07:23:13.157 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:13.155+0000 7f01f2016640 1 Processor -- start 2023-11-26T07:23:13.158 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:13.155+0000 7f01f2016640 1 -- start start 2023-11-26T07:23:13.160 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:13.159+0000 7f01f2016640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f01ec0b2160 0x7f01ec157c50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:23:13.160 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:13.159+0000 7f01f2016640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f01ec0d1bc0 0x7f01ec0b74c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:23:13.160 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:13.159+0000 7f01f2016640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f01ec15d6e0 0x7f01ec155bb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:23:13.160 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:13.159+0000 7f01f2016640 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f01ec0c6d50 con 0x7f01ec0d1bc0 2023-11-26T07:23:13.161 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:13.159+0000 7f01f2016640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f01ec158190 con 0x7f01ec15d6e0 2023-11-26T07:23:13.161 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:13.159+0000 7f01f2016640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f01ec0b7f30 con 0x7f01ec0b2160 2023-11-26T07:23:13.161 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:13.159+0000 7f01eaffd640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f01ec0d1bc0 0x7f01ec0b74c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:23:13.161 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:13.159+0000 7f01eaffd640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f01ec0d1bc0 0x7f01ec0b74c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:46654/0 (socket says 172.21.15.143:46654) 2023-11-26T07:23:13.161 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:13.159+0000 7f01eaffd640 1 -- 172.21.15.143:0/2146900510 learned_addr learned my addr 172.21.15.143:0/2146900510 (peer_addr_for_me v2:172.21.15.143:0/0) 2023-11-26T07:23:13.161 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:13.159+0000 7f01ebfff640 1 --2- 172.21.15.143:0/2146900510 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f01ec15d6e0 0x7f01ec155bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:23:13.161 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:13.159+0000 7f01eb7fe640 1 --2- 172.21.15.143:0/2146900510 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f01ec0b2160 0x7f01ec157c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:23:13.161 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:13.159+0000 7f01eaffd640 1 -- 172.21.15.143:0/2146900510 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f01ec0b2160 msgr2=0x7f01ec157c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:23:13.161 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:13.159+0000 7f01eaffd640 1 --2- 172.21.15.143:0/2146900510 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f01ec0b2160 0x7f01ec157c50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:23:13.161 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:13.159+0000 7f01eaffd640 1 -- 172.21.15.143:0/2146900510 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f01ec15d6e0 msgr2=0x7f01ec155bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:23:13.161 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:13.159+0000 7f01eaffd640 1 --2- 172.21.15.143:0/2146900510 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f01ec15d6e0 0x7f01ec155bb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:23:13.161 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:13.159+0000 7f01eaffd640 1 -- 172.21.15.143:0/2146900510 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f01e4009b30 con 0x7f01ec0d1bc0 2023-11-26T07:23:13.162 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:13.159+0000 7f01ebfff640 1 --2- 172.21.15.143:0/2146900510 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f01ec15d6e0 0x7f01ec155bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-26T07:23:13.162 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:13.159+0000 7f01eaffd640 1 --2- 172.21.15.143:0/2146900510 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f01ec0d1bc0 0x7f01ec0b74c0 secure :-1 s=READY pgs=1067 cs=0 l=1 rev1=1 crypto rx=0x7f01e000c480 tx=0x7f01e000c940 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:23:13.163 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:13.159+0000 7f01d3fff640 1 -- 172.21.15.143:0/2146900510 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f01e0005480 con 0x7f01ec0d1bc0 2023-11-26T07:23:13.163 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:13.159+0000 7f01d3fff640 1 -- 172.21.15.143:0/2146900510 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f01e0003600 con 0x7f01ec0d1bc0 2023-11-26T07:23:13.163 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:13.159+0000 7f01d3fff640 1 -- 172.21.15.143:0/2146900510 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f01e00039b0 con 0x7f01ec0d1bc0 2023-11-26T07:23:13.163 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:13.159+0000 7f01f2016640 1 -- 172.21.15.143:0/2146900510 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f01ec0b8210 con 0x7f01ec0d1bc0 2023-11-26T07:23:13.163 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:13.159+0000 7f01f2016640 1 -- 172.21.15.143:0/2146900510 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f01ec0b87d0 con 0x7f01ec0d1bc0 2023-11-26T07:23:13.163 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:13.159+0000 7f01d3fff640 1 -- 172.21.15.143:0/2146900510 <== mon.0 v2:172.21.15.143:3300/0 4 ==== mgrmap(e 14) v1 ==== 43229+0+0 (secure 0 0 0) 0x7f01e0003b10 con 0x7f01ec0d1bc0 2023-11-26T07:23:13.163 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:13.159+0000 7f01f2016640 1 -- 172.21.15.143:0/2146900510 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f01ec0c6f00 con 0x7f01ec0d1bc0 2023-11-26T07:23:13.163 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:13.159+0000 7f01d3fff640 1 --2- 172.21.15.143:0/2146900510 >> [v2:172.21.15.188:6836/3401128553,v1:172.21.15.188:6837/3401128553] conn(0x7f01d8035510 0x7f01d80379d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:23:13.164 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:13.159+0000 7f01d3fff640 1 -- 172.21.15.143:0/2146900510 <== mon.0 v2:172.21.15.143:3300/0 5 ==== osd_map(72..72 src has 1..72) v4 ==== 6268+0+0 (secure 0 0 0) 0x7f01e0047d90 con 0x7f01ec0d1bc0 2023-11-26T07:23:13.164 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:13.163+0000 7f01eb7fe640 1 --2- 172.21.15.143:0/2146900510 >> [v2:172.21.15.188:6836/3401128553,v1:172.21.15.188:6837/3401128553] conn(0x7f01d8035510 0x7f01d80379d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:23:13.164 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:13.163+0000 7f01eb7fe640 1 --2- 172.21.15.143:0/2146900510 >> [v2:172.21.15.188:6836/3401128553,v1:172.21.15.188:6837/3401128553] conn(0x7f01d8035510 0x7f01d80379d0 secure :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0x7f01dc000f70 tx=0x7f01dc007c90 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:23:13.165 INFO:tasks.ceph.mgr.y.smithi143.stderr:2023-11-26T07:23:13.163+0000 7f85d6276180 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2023-11-26T07:23:13.167 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:13.167+0000 7f01d3fff640 1 -- 172.21.15.143:0/2146900510 <== mon.0 v2:172.21.15.143:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172649 (secure 0 0 0) 0x7f01ec0c6f00 con 0x7f01ec0d1bc0 2023-11-26T07:23:13.226 INFO:tasks.ceph.mgr.y.smithi143.stderr:2023-11-26T07:23:13.223+0000 7f85d6276180 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2023-11-26T07:23:13.323 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:13.323+0000 7f01f2016640 1 -- 172.21.15.143:0/2146900510 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f01ec0c6f00 con 0x7f01ec0d1bc0 2023-11-26T07:23:13.323 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:13.323+0000 7f01d3fff640 1 -- 172.21.15.143:0/2146900510 <== mon.0 v2:172.21.15.143:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v72) v1 ==== 93+0+10 (secure 0 0 0) 0x7f01ec0c6f00 con 0x7f01ec0d1bc0 2023-11-26T07:23:13.323 INFO:teuthology.orchestra.run.smithi143.stdout:pg_num: 1 2023-11-26T07:23:13.325 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:13.323+0000 7f01f2016640 1 -- 172.21.15.143:0/2146900510 >> [v2:172.21.15.188:6836/3401128553,v1:172.21.15.188:6837/3401128553] conn(0x7f01d8035510 msgr2=0x7f01d80379d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:23:13.326 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:13.323+0000 7f01f2016640 1 --2- 172.21.15.143:0/2146900510 >> [v2:172.21.15.188:6836/3401128553,v1:172.21.15.188:6837/3401128553] conn(0x7f01d8035510 0x7f01d80379d0 secure :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0x7f01dc000f70 tx=0x7f01dc007c90 comp rx=0 tx=0).stop 2023-11-26T07:23:13.326 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:13.323+0000 7f01f2016640 1 -- 172.21.15.143:0/2146900510 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f01ec0d1bc0 msgr2=0x7f01ec0b74c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:23:13.326 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:13.323+0000 7f01f2016640 1 --2- 172.21.15.143:0/2146900510 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f01ec0d1bc0 0x7f01ec0b74c0 secure :-1 s=READY pgs=1067 cs=0 l=1 rev1=1 crypto rx=0x7f01e000c480 tx=0x7f01e000c940 comp rx=0 tx=0).stop 2023-11-26T07:23:13.326 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:13.323+0000 7f01f2016640 1 -- 172.21.15.143:0/2146900510 shutdown_connections 2023-11-26T07:23:13.326 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:13.323+0000 7f01f2016640 1 --2- 172.21.15.143:0/2146900510 >> [v2:172.21.15.188:6836/3401128553,v1:172.21.15.188:6837/3401128553] conn(0x7f01d8035510 0x7f01d80379d0 unknown :-1 s=CLOSED pgs=22 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:23:13.326 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:13.323+0000 7f01f2016640 1 --2- 172.21.15.143:0/2146900510 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f01ec15d6e0 0x7f01ec155bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:23:13.326 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:13.323+0000 7f01f2016640 1 --2- 172.21.15.143:0/2146900510 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f01ec0d1bc0 0x7f01ec0b74c0 unknown :-1 s=CLOSED pgs=1067 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:23:13.327 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:13.323+0000 7f01f2016640 1 --2- 172.21.15.143:0/2146900510 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f01ec0b2160 0x7f01ec157c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:23:13.327 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:13.323+0000 7f01f2016640 1 -- 172.21.15.143:0/2146900510 >> 172.21.15.143:0/2146900510 conn(0x7f01ec09c550 msgr2=0x7f01ec0cf490 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:23:13.327 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:13.323+0000 7f01f2016640 1 -- 172.21.15.143:0/2146900510 shutdown_connections 2023-11-26T07:23:13.327 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:13.323+0000 7f01f2016640 1 -- 172.21.15.143:0/2146900510 wait complete. 2023-11-26T07:23:13.347 DEBUG:teuthology.orchestra.run.smithi143:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-11-26T07:23:13.477 INFO:tasks.ceph.mgr.y.smithi143.stderr:/usr/lib/python3/dist-packages/scipy/__init__.py:67: UserWarning: NumPy was imported from a Python sub-interpreter but NumPy does not properly support sub-interpreters. This will likely work for most users but might cause hard to track down issues or subtle bugs. A common user of the rare sub-interpreter feature is wsgi which also allows single-interpreter mode. 2023-11-26T07:23:13.477 INFO:tasks.ceph.mgr.y.smithi143.stderr:Improvements in the case of bugs are welcome, but is not on the NumPy roadmap, and full support may require significant effort to achieve. 2023-11-26T07:23:13.477 INFO:tasks.ceph.mgr.y.smithi143.stderr: from numpy import show_config as show_numpy_config 2023-11-26T07:23:13.485 INFO:tasks.ceph.mgr.y.smithi143.stderr:2023-11-26T07:23:13.483+0000 7f85d6276180 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2023-11-26T07:23:13.575 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:13.575+0000 7f5264901640 1 Processor -- start 2023-11-26T07:23:13.575 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:13.575+0000 7f5264901640 1 -- start start 2023-11-26T07:23:13.577 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:13.575+0000 7f5264901640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f526015d6e0 0x7f52600d0df0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:23:13.577 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:13.575+0000 7f5264901640 1 -- --> v1:172.21.15.143:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f52600cf0e0 con 0x7f52600b2160 2023-11-26T07:23:13.577 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:13.575+0000 7f5264901640 1 -- --> v1:172.21.15.188:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f52600ceb20 con 0x7f52600d1bc0 2023-11-26T07:23:13.577 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:13.575+0000 7f5264901640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f526015dac0 con 0x7f526015d6e0 2023-11-26T07:23:13.577 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:13.575+0000 7f525dd9b640 1 --1- >> v1:172.21.15.143:6789/0 conn(0x7f52600b2160 0x7f52600d14b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.143:6789/0 says I am v1:172.21.15.143:54514/0 (socket says 172.21.15.143:54514) 2023-11-26T07:23:13.577 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:13.575+0000 7f525dd9b640 1 -- 172.21.15.143:0/299499731 learned_addr learned my addr 172.21.15.143:0/299499731 (peer_addr_for_me v1:172.21.15.143:0/0) 2023-11-26T07:23:13.577 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:13.575+0000 7f525e59c640 1 --2- 172.21.15.143:0/299499731 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f526015d6e0 0x7f52600d0df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:23:13.578 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:13.575+0000 7f525d59a640 1 -- 172.21.15.143:0/299499731 <== mon.0 v1:172.21.15.143:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1458440308 0 0) 0x7f52600cf0e0 con 0x7f52600b2160 2023-11-26T07:23:13.578 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:13.575+0000 7f525d59a640 1 -- 172.21.15.143:0/299499731 --> v1:172.21.15.143:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f5248003530 con 0x7f52600b2160 2023-11-26T07:23:13.578 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:13.575+0000 7f525d59a640 1 -- 172.21.15.143:0/299499731 <== mon.1 v1:172.21.15.188:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3367651864 0 0) 0x7f52600ceb20 con 0x7f52600d1bc0 2023-11-26T07:23:13.578 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:13.575+0000 7f525d59a640 1 -- 172.21.15.143:0/299499731 --> v1:172.21.15.188:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f52600cf0e0 con 0x7f52600d1bc0 2023-11-26T07:23:13.578 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:13.575+0000 7f525e59c640 1 -- 172.21.15.143:0/299499731 >> v1:172.21.15.188:6789/0 conn(0x7f52600d1bc0 legacy=0x7f52600b0750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:23:13.578 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:13.575+0000 7f525e59c640 1 -- 172.21.15.143:0/299499731 >> v1:172.21.15.143:6789/0 conn(0x7f52600b2160 legacy=0x7f52600d14b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:23:13.578 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:13.575+0000 7f525e59c640 1 -- 172.21.15.143:0/299499731 --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f52600cee00 con 0x7f526015d6e0 2023-11-26T07:23:13.580 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:13.579+0000 7f525e59c640 1 --2- 172.21.15.143:0/299499731 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f526015d6e0 0x7f52600d0df0 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7f52500097f0 tx=0x7f525001c4c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=48f149050e1dfbdd server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:23:13.580 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:13.579+0000 7f525d59a640 1 -- 172.21.15.143:0/299499731 <== mon.2 v2:172.21.15.188:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f52500ac410 con 0x7f526015d6e0 2023-11-26T07:23:13.581 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:13.579+0000 7f525d59a640 1 -- 172.21.15.143:0/299499731 <== mon.2 v2:172.21.15.188:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f52500ac570 con 0x7f526015d6e0 2023-11-26T07:23:13.581 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:13.579+0000 7f525d59a640 1 -- 172.21.15.143:0/299499731 <== mon.2 v2:172.21.15.188:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f52500ac900 con 0x7f526015d6e0 2023-11-26T07:23:13.581 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:13.579+0000 7f5264901640 1 -- 172.21.15.143:0/299499731 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f526015d6e0 msgr2=0x7f52600d0df0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:23:13.581 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:13.579+0000 7f5264901640 1 --2- 172.21.15.143:0/299499731 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f526015d6e0 0x7f52600d0df0 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7f52500097f0 tx=0x7f525001c4c0 comp rx=0 tx=0).stop 2023-11-26T07:23:13.582 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:13.579+0000 7f5264901640 1 -- 172.21.15.143:0/299499731 shutdown_connections 2023-11-26T07:23:13.582 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:13.579+0000 7f5264901640 1 --2- 172.21.15.143:0/299499731 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f526015d6e0 0x7f52600d0df0 unknown :-1 s=CLOSED pgs=137 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:23:13.582 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:13.579+0000 7f5264901640 1 -- 172.21.15.143:0/299499731 >> 172.21.15.143:0/299499731 conn(0x7f526009c550 msgr2=0x7f52600d3220 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:23:13.582 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:13.579+0000 7f5264901640 1 -- 172.21.15.143:0/299499731 shutdown_connections 2023-11-26T07:23:13.582 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:13.579+0000 7f5264901640 1 -- 172.21.15.143:0/299499731 wait complete. 2023-11-26T07:23:13.583 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:13.583+0000 7f5264901640 1 Processor -- start 2023-11-26T07:23:13.584 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:13.583+0000 7f5264901640 1 -- start start 2023-11-26T07:23:13.585 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:13.583+0000 7f5264901640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f52600b2160 0x7f526015c870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:23:13.585 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:13.583+0000 7f5264901640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f52600d1bc0 0x7f526006bf10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:23:13.585 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:13.583+0000 7f5264901640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f526015d6e0 0x7f52600c8f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:23:13.585 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:13.583+0000 7f5264901640 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5260156110 con 0x7f52600d1bc0 2023-11-26T07:23:13.585 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:13.583+0000 7f5264901640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f526015dca0 con 0x7f526015d6e0 2023-11-26T07:23:13.586 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:13.583+0000 7f5264901640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f526015df80 con 0x7f52600b2160 2023-11-26T07:23:13.586 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:13.583+0000 7f525dd9b640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f52600d1bc0 0x7f526006bf10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:23:13.586 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:13.583+0000 7f525dd9b640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f52600d1bc0 0x7f526006bf10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:46658/0 (socket says 172.21.15.143:46658) 2023-11-26T07:23:13.586 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:13.583+0000 7f525dd9b640 1 -- 172.21.15.143:0/1786312319 learned_addr learned my addr 172.21.15.143:0/1786312319 (peer_addr_for_me v2:172.21.15.143:0/0) 2023-11-26T07:23:13.586 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:13.583+0000 7f525e59c640 1 --2- 172.21.15.143:0/1786312319 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f52600b2160 0x7f526015c870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:23:13.586 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:13.583+0000 7f525ed9d640 1 --2- 172.21.15.143:0/1786312319 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f526015d6e0 0x7f52600c8f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:23:13.586 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:13.583+0000 7f525dd9b640 1 -- 172.21.15.143:0/1786312319 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f52600b2160 msgr2=0x7f526015c870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:23:13.586 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:13.583+0000 7f525dd9b640 1 --2- 172.21.15.143:0/1786312319 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f52600b2160 0x7f526015c870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:23:13.586 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:13.583+0000 7f525dd9b640 1 -- 172.21.15.143:0/1786312319 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f526015d6e0 msgr2=0x7f52600c8f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:23:13.586 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:13.583+0000 7f525dd9b640 1 --2- 172.21.15.143:0/1786312319 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f526015d6e0 0x7f52600c8f30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:23:13.586 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:13.583+0000 7f525dd9b640 1 -- 172.21.15.143:0/1786312319 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5250009420 con 0x7f52600d1bc0 2023-11-26T07:23:13.586 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:13.583+0000 7f525e59c640 1 --2- 172.21.15.143:0/1786312319 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f52600b2160 0x7f526015c870 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-26T07:23:13.586 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:13.583+0000 7f525dd9b640 1 --2- 172.21.15.143:0/1786312319 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f52600d1bc0 0x7f526006bf10 secure :-1 s=READY pgs=1069 cs=0 l=1 rev1=1 crypto rx=0x7f525400c4f0 tx=0x7f525400c9b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:23:13.587 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:13.583+0000 7f524effd640 1 -- 172.21.15.143:0/1786312319 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5254005cf0 con 0x7f52600d1bc0 2023-11-26T07:23:13.588 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:13.583+0000 7f524effd640 1 -- 172.21.15.143:0/1786312319 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5254005e50 con 0x7f52600d1bc0 2023-11-26T07:23:13.588 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:13.583+0000 7f524effd640 1 -- 172.21.15.143:0/1786312319 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5254007270 con 0x7f52600d1bc0 2023-11-26T07:23:13.588 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:13.583+0000 7f5264901640 1 -- 172.21.15.143:0/1786312319 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f526015e260 con 0x7f52600d1bc0 2023-11-26T07:23:13.588 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:13.583+0000 7f5264901640 1 -- 172.21.15.143:0/1786312319 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f526015cdb0 con 0x7f52600d1bc0 2023-11-26T07:23:13.588 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:13.587+0000 7f524effd640 1 -- 172.21.15.143:0/1786312319 <== mon.0 v2:172.21.15.143:3300/0 4 ==== mgrmap(e 14) v1 ==== 43229+0+0 (secure 0 0 0) 0x7f52540073d0 con 0x7f52600d1bc0 2023-11-26T07:23:13.588 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:13.587+0000 7f524effd640 1 --2- 172.21.15.143:0/1786312319 >> [v2:172.21.15.188:6836/3401128553,v1:172.21.15.188:6837/3401128553] conn(0x7f52400354e0 0x7f52400379a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:23:13.588 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:13.587+0000 7f5264901640 1 -- 172.21.15.143:0/1786312319 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5260067680 con 0x7f52600d1bc0 2023-11-26T07:23:13.588 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:13.587+0000 7f524effd640 1 -- 172.21.15.143:0/1786312319 <== mon.0 v2:172.21.15.143:3300/0 5 ==== osd_map(72..72 src has 1..72) v4 ==== 6268+0+0 (secure 0 0 0) 0x7f5254047760 con 0x7f52600d1bc0 2023-11-26T07:23:13.588 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:13.587+0000 7f525e59c640 1 --2- 172.21.15.143:0/1786312319 >> [v2:172.21.15.188:6836/3401128553,v1:172.21.15.188:6837/3401128553] conn(0x7f52400354e0 0x7f52400379a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:23:13.589 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:13.587+0000 7f525e59c640 1 --2- 172.21.15.143:0/1786312319 >> [v2:172.21.15.188:6836/3401128553,v1:172.21.15.188:6837/3401128553] conn(0x7f52400354e0 0x7f52400379a0 secure :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0x7f52500ae000 tx=0x7f5250008040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:23:13.592 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:13.591+0000 7f524effd640 1 -- 172.21.15.143:0/1786312319 <== mon.0 v2:172.21.15.143:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172649 (secure 0 0 0) 0x7f52540198b0 con 0x7f52600d1bc0 2023-11-26T07:23:13.630 INFO:tasks.ceph.mgr.y.smithi143.stderr:2023-11-26T07:23:13.627+0000 7f85d6276180 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2023-11-26T07:23:13.744 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:13.743+0000 7f5264901640 1 -- 172.21.15.143:0/1786312319 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f526015e710 con 0x7f52600d1bc0 2023-11-26T07:23:13.745 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:13.743+0000 7f524effd640 1 -- 172.21.15.143:0/1786312319 <== mon.0 v2:172.21.15.143:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v72) v1 ==== 104+0+11 (secure 0 0 0) 0x7f5254017020 con 0x7f52600d1bc0 2023-11-26T07:23:13.745 INFO:teuthology.orchestra.run.smithi143.stdout:pg_num: 32 2023-11-26T07:23:13.747 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:13.747+0000 7f5264901640 1 -- 172.21.15.143:0/1786312319 >> [v2:172.21.15.188:6836/3401128553,v1:172.21.15.188:6837/3401128553] conn(0x7f52400354e0 msgr2=0x7f52400379a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:23:13.747 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:13.747+0000 7f5264901640 1 --2- 172.21.15.143:0/1786312319 >> [v2:172.21.15.188:6836/3401128553,v1:172.21.15.188:6837/3401128553] conn(0x7f52400354e0 0x7f52400379a0 secure :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0x7f52500ae000 tx=0x7f5250008040 comp rx=0 tx=0).stop 2023-11-26T07:23:13.747 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:13.747+0000 7f5264901640 1 -- 172.21.15.143:0/1786312319 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f52600d1bc0 msgr2=0x7f526006bf10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:23:13.748 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:13.747+0000 7f5264901640 1 --2- 172.21.15.143:0/1786312319 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f52600d1bc0 0x7f526006bf10 secure :-1 s=READY pgs=1069 cs=0 l=1 rev1=1 crypto rx=0x7f525400c4f0 tx=0x7f525400c9b0 comp rx=0 tx=0).stop 2023-11-26T07:23:13.748 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:13.747+0000 7f5264901640 1 -- 172.21.15.143:0/1786312319 shutdown_connections 2023-11-26T07:23:13.748 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:13.747+0000 7f5264901640 1 --2- 172.21.15.143:0/1786312319 >> [v2:172.21.15.188:6836/3401128553,v1:172.21.15.188:6837/3401128553] conn(0x7f52400354e0 0x7f52400379a0 unknown :-1 s=CLOSED pgs=24 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:23:13.748 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:13.747+0000 7f5264901640 1 --2- 172.21.15.143:0/1786312319 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f526015d6e0 0x7f52600c8f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:23:13.748 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:13.747+0000 7f5264901640 1 --2- 172.21.15.143:0/1786312319 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f52600d1bc0 0x7f526006bf10 unknown :-1 s=CLOSED pgs=1069 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:23:13.748 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:13.747+0000 7f5264901640 1 --2- 172.21.15.143:0/1786312319 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f52600b2160 0x7f526015c870 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:23:13.748 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:13.747+0000 7f5264901640 1 -- 172.21.15.143:0/1786312319 >> 172.21.15.143:0/1786312319 conn(0x7f526009c550 msgr2=0x7f52600d3220 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:23:13.748 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:13.747+0000 7f5264901640 1 -- 172.21.15.143:0/1786312319 shutdown_connections 2023-11-26T07:23:13.748 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:13.747+0000 7f5264901640 1 -- 172.21.15.143:0/1786312319 wait complete. 2023-11-26T07:23:13.758 INFO:tasks.ceph.mgr.y.smithi143.stderr:2023-11-26T07:23:13.755+0000 7f85d6276180 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2023-11-26T07:23:13.767 DEBUG:teuthology.orchestra.run.smithi143:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-11-26T07:23:13.833 INFO:tasks.ceph.mgr.y.smithi143.stderr:2023-11-26T07:23:13.831+0000 7f85d6276180 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2023-11-26T07:23:13.911 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:13.911+0000 7fd334e32640 1 Processor -- start 2023-11-26T07:23:13.912 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:13.911+0000 7fd334e32640 1 -- start start 2023-11-26T07:23:13.913 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:13.911+0000 7fd334e32640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fd33015d6e0 0x7fd3300d0df0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:23:13.913 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:13.911+0000 7fd334e32640 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7fd3300d1bc0 0x7fd3300b0750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:23:13.913 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:13.911+0000 7fd334e32640 1 -- --> v1:172.21.15.188:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd3300cf0e0 con 0x7fd3300b2160 2023-11-26T07:23:13.913 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:13.911+0000 7fd334e32640 1 -- --> v2:172.21.15.143:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd33015dac0 con 0x7fd3300d1bc0 2023-11-26T07:23:13.913 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:13.911+0000 7fd334e32640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd3300ceb20 con 0x7fd33015d6e0 2023-11-26T07:23:13.914 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:13.911+0000 7fd32ed9d640 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7fd3300d1bc0 0x7fd3300b0750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:23:13.914 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:13.911+0000 7fd32e59c640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fd33015d6e0 0x7fd3300d0df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:23:13.914 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:13.911+0000 7fd32ed9d640 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7fd3300d1bc0 0x7fd3300b0750 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:46666/0 (socket says 172.21.15.143:46666) 2023-11-26T07:23:13.914 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:13.911+0000 7fd32ed9d640 1 -- 172.21.15.143:0/3390464036 learned_addr learned my addr 172.21.15.143:0/3390464036 (peer_addr_for_me v2:172.21.15.143:0/0) 2023-11-26T07:23:13.914 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:13.911+0000 7fd32dd9b640 1 --1- >> v1:172.21.15.188:6789/0 conn(0x7fd3300b2160 0x7fd3300d14b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.188:6789/0 says I am v1:172.21.15.143:45112/0 (socket says 172.21.15.143:45112) 2023-11-26T07:23:13.914 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:13.911+0000 7fd32ed9d640 1 -- 172.21.15.143:0/3390464036 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fd33015d6e0 msgr2=0x7fd3300d0df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-26T07:23:13.914 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:13.911+0000 7fd32ed9d640 1 --2- 172.21.15.143:0/3390464036 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fd33015d6e0 0x7fd3300d0df0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:23:13.915 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:13.911+0000 7fd32d59a640 1 -- 172.21.15.143:0/3390464036 <== mon.1 v1:172.21.15.188:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1144454827 0 0) 0x7fd3300cf0e0 con 0x7fd3300b2160 2023-11-26T07:23:13.915 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:13.911+0000 7fd32ed9d640 1 -- 172.21.15.143:0/3390464036 >> v1:172.21.15.188:6789/0 conn(0x7fd3300b2160 legacy=0x7fd3300d14b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:23:13.915 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:13.911+0000 7fd32ed9d640 1 -- 172.21.15.143:0/3390464036 --> v2:172.21.15.143:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd3300cee00 con 0x7fd3300d1bc0 2023-11-26T07:23:13.915 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:13.911+0000 7fd32e59c640 1 --2- 172.21.15.143:0/3390464036 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fd33015d6e0 0x7fd3300d0df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-26T07:23:13.916 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:13.915+0000 7fd32ed9d640 1 --2- 172.21.15.143:0/3390464036 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fd3300d1bc0 0x7fd3300b0750 secure :-1 s=READY pgs=1070 cs=0 l=1 rev1=1 crypto rx=0x7fd328009e50 tx=0x7fd32801ca10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=404211a1e22005b1 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:23:13.917 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:13.915+0000 7fd32d59a640 1 -- 172.21.15.143:0/3390464036 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd3280a57d0 con 0x7fd3300d1bc0 2023-11-26T07:23:13.917 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:13.915+0000 7fd32d59a640 1 -- 172.21.15.143:0/3390464036 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd3280a5930 con 0x7fd3300d1bc0 2023-11-26T07:23:13.917 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:13.915+0000 7fd32d59a640 1 -- 172.21.15.143:0/3390464036 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd3280a5cc0 con 0x7fd3300d1bc0 2023-11-26T07:23:13.918 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:13.915+0000 7fd334e32640 1 -- 172.21.15.143:0/3390464036 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fd3300d1bc0 msgr2=0x7fd3300b0750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:23:13.918 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:13.915+0000 7fd334e32640 1 --2- 172.21.15.143:0/3390464036 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fd3300d1bc0 0x7fd3300b0750 secure :-1 s=READY pgs=1070 cs=0 l=1 rev1=1 crypto rx=0x7fd328009e50 tx=0x7fd32801ca10 comp rx=0 tx=0).stop 2023-11-26T07:23:13.918 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:13.915+0000 7fd334e32640 1 -- 172.21.15.143:0/3390464036 shutdown_connections 2023-11-26T07:23:13.918 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:13.915+0000 7fd334e32640 1 --2- 172.21.15.143:0/3390464036 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fd3300d1bc0 0x7fd3300b0750 unknown :-1 s=CLOSED pgs=1070 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:23:13.918 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:13.915+0000 7fd334e32640 1 --2- 172.21.15.143:0/3390464036 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fd33015d6e0 0x7fd3300d0df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:23:13.918 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:13.915+0000 7fd334e32640 1 -- 172.21.15.143:0/3390464036 >> 172.21.15.143:0/3390464036 conn(0x7fd33009c550 msgr2=0x7fd3300d3220 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:23:13.919 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:13.915+0000 7fd334e32640 1 -- 172.21.15.143:0/3390464036 shutdown_connections 2023-11-26T07:23:13.919 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:13.915+0000 7fd334e32640 1 -- 172.21.15.143:0/3390464036 wait complete. 2023-11-26T07:23:13.920 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:13.919+0000 7fd334e32640 1 Processor -- start 2023-11-26T07:23:13.920 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:13.919+0000 7fd334e32640 1 -- start start 2023-11-26T07:23:13.922 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:13.919+0000 7fd334e32640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fd3300b2160 0x7fd3300c6d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:23:13.922 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:13.919+0000 7fd334e32640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fd3300d1bc0 0x7fd33015e610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:23:13.922 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:13.919+0000 7fd334e32640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fd33015d6e0 0x7fd33015fed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:23:13.922 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:13.919+0000 7fd334e32640 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd3300c9bd0 con 0x7fd3300d1bc0 2023-11-26T07:23:13.922 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:13.919+0000 7fd334e32640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd330155b40 con 0x7fd33015d6e0 2023-11-26T07:23:13.922 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:13.919+0000 7fd334e32640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd330154f30 con 0x7fd3300b2160 2023-11-26T07:23:13.922 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:13.919+0000 7fd32dd9b640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fd3300d1bc0 0x7fd33015e610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:23:13.922 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:13.919+0000 7fd32dd9b640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fd3300d1bc0 0x7fd33015e610 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:46682/0 (socket says 172.21.15.143:46682) 2023-11-26T07:23:13.922 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:13.919+0000 7fd32dd9b640 1 -- 172.21.15.143:0/2149189164 learned_addr learned my addr 172.21.15.143:0/2149189164 (peer_addr_for_me v2:172.21.15.143:0/0) 2023-11-26T07:23:13.923 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:13.919+0000 7fd32e59c640 1 --2- 172.21.15.143:0/2149189164 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fd3300b2160 0x7fd3300c6d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:23:13.923 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:13.919+0000 7fd32ed9d640 1 --2- 172.21.15.143:0/2149189164 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fd33015d6e0 0x7fd33015fed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:23:13.923 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:13.919+0000 7fd32dd9b640 1 -- 172.21.15.143:0/2149189164 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fd3300b2160 msgr2=0x7fd3300c6d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:23:13.923 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:13.919+0000 7fd32dd9b640 1 --2- 172.21.15.143:0/2149189164 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fd3300b2160 0x7fd3300c6d50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:23:13.923 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:13.919+0000 7fd32dd9b640 1 -- 172.21.15.143:0/2149189164 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fd33015d6e0 msgr2=0x7fd33015fed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:23:13.923 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:13.919+0000 7fd32dd9b640 1 --2- 172.21.15.143:0/2149189164 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fd33015d6e0 0x7fd33015fed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:23:13.923 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:13.919+0000 7fd32dd9b640 1 -- 172.21.15.143:0/2149189164 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd320009420 con 0x7fd3300d1bc0 2023-11-26T07:23:13.923 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:13.919+0000 7fd32dd9b640 1 --2- 172.21.15.143:0/2149189164 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fd3300d1bc0 0x7fd33015e610 secure :-1 s=READY pgs=1071 cs=0 l=1 rev1=1 crypto rx=0x7fd32400c480 tx=0x7fd32400c940 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:23:13.924 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:13.919+0000 7fd31effd640 1 -- 172.21.15.143:0/2149189164 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd3240074e0 con 0x7fd3300d1bc0 2023-11-26T07:23:13.924 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:13.919+0000 7fd31effd640 1 -- 172.21.15.143:0/2149189164 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd324007660 con 0x7fd3300d1bc0 2023-11-26T07:23:13.924 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:13.919+0000 7fd31effd640 1 -- 172.21.15.143:0/2149189164 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd324004a30 con 0x7fd3300d1bc0 2023-11-26T07:23:13.924 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:13.919+0000 7fd334e32640 1 -- 172.21.15.143:0/2149189164 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd328009b30 con 0x7fd3300d1bc0 2023-11-26T07:23:13.924 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:13.919+0000 7fd334e32640 1 -- 172.21.15.143:0/2149189164 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd3301554f0 con 0x7fd3300d1bc0 2023-11-26T07:23:13.925 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:13.923+0000 7fd334e32640 1 -- 172.21.15.143:0/2149189164 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd330155cb0 con 0x7fd3300d1bc0 2023-11-26T07:23:13.925 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:13.923+0000 7fd31effd640 1 -- 172.21.15.143:0/2149189164 <== mon.0 v2:172.21.15.143:3300/0 4 ==== mgrmap(e 14) v1 ==== 43229+0+0 (secure 0 0 0) 0x7fd324004b90 con 0x7fd3300d1bc0 2023-11-26T07:23:13.925 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:13.923+0000 7fd31effd640 1 --2- 172.21.15.143:0/2149189164 >> [v2:172.21.15.188:6836/3401128553,v1:172.21.15.188:6837/3401128553] conn(0x7fd310035510 0x7fd3100379d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:23:13.925 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:13.923+0000 7fd31effd640 1 -- 172.21.15.143:0/2149189164 <== mon.0 v2:172.21.15.143:3300/0 5 ==== osd_map(72..72 src has 1..72) v4 ==== 6268+0+0 (secure 0 0 0) 0x7fd324047240 con 0x7fd3300d1bc0 2023-11-26T07:23:13.925 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:13.923+0000 7fd32e59c640 1 --2- 172.21.15.143:0/2149189164 >> [v2:172.21.15.188:6836/3401128553,v1:172.21.15.188:6837/3401128553] conn(0x7fd310035510 0x7fd3100379d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:23:13.925 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:13.923+0000 7fd32e59c640 1 --2- 172.21.15.143:0/2149189164 >> [v2:172.21.15.188:6836/3401128553,v1:172.21.15.188:6837/3401128553] conn(0x7fd310035510 0x7fd3100379d0 secure :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0x7fd320009cf0 tx=0x7fd320009190 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:23:13.929 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:13.927+0000 7fd31effd640 1 -- 172.21.15.143:0/2149189164 <== mon.0 v2:172.21.15.143:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172649 (secure 0 0 0) 0x7fd324004e40 con 0x7fd3300d1bc0 2023-11-26T07:23:13.977 INFO:tasks.ceph.mgr.y.smithi143.stderr:2023-11-26T07:23:13.975+0000 7f85d6276180 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2023-11-26T07:23:14.080 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:14.079+0000 7fd334e32640 1 -- 172.21.15.143:0/2149189164 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7fd330067680 con 0x7fd3300d1bc0 2023-11-26T07:23:14.081 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:14.079+0000 7fd31effd640 1 -- 172.21.15.143:0/2149189164 <== mon.0 v2:172.21.15.143:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v72) v1 ==== 100+0+11 (secure 0 0 0) 0x7fd324018020 con 0x7fd3300d1bc0 2023-11-26T07:23:14.081 INFO:teuthology.orchestra.run.smithi143.stdout:pg_num: 64 2023-11-26T07:23:14.083 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:14.083+0000 7fd334e32640 1 -- 172.21.15.143:0/2149189164 >> [v2:172.21.15.188:6836/3401128553,v1:172.21.15.188:6837/3401128553] conn(0x7fd310035510 msgr2=0x7fd3100379d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:23:14.083 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:14.083+0000 7fd334e32640 1 --2- 172.21.15.143:0/2149189164 >> [v2:172.21.15.188:6836/3401128553,v1:172.21.15.188:6837/3401128553] conn(0x7fd310035510 0x7fd3100379d0 secure :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0x7fd320009cf0 tx=0x7fd320009190 comp rx=0 tx=0).stop 2023-11-26T07:23:14.083 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:14.083+0000 7fd334e32640 1 -- 172.21.15.143:0/2149189164 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fd3300d1bc0 msgr2=0x7fd33015e610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:23:14.083 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:14.083+0000 7fd334e32640 1 --2- 172.21.15.143:0/2149189164 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fd3300d1bc0 0x7fd33015e610 secure :-1 s=READY pgs=1071 cs=0 l=1 rev1=1 crypto rx=0x7fd32400c480 tx=0x7fd32400c940 comp rx=0 tx=0).stop 2023-11-26T07:23:14.083 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:14.083+0000 7fd334e32640 1 -- 172.21.15.143:0/2149189164 shutdown_connections 2023-11-26T07:23:14.084 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:14.083+0000 7fd334e32640 1 --2- 172.21.15.143:0/2149189164 >> [v2:172.21.15.188:6836/3401128553,v1:172.21.15.188:6837/3401128553] conn(0x7fd310035510 0x7fd3100379d0 unknown :-1 s=CLOSED pgs=25 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:23:14.084 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:14.083+0000 7fd334e32640 1 --2- 172.21.15.143:0/2149189164 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fd33015d6e0 0x7fd33015fed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:23:14.084 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:14.083+0000 7fd334e32640 1 --2- 172.21.15.143:0/2149189164 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fd3300d1bc0 0x7fd33015e610 unknown :-1 s=CLOSED pgs=1071 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:23:14.084 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:14.083+0000 7fd334e32640 1 --2- 172.21.15.143:0/2149189164 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fd3300b2160 0x7fd3300c6d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:23:14.084 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:14.083+0000 7fd334e32640 1 -- 172.21.15.143:0/2149189164 >> 172.21.15.143:0/2149189164 conn(0x7fd33009c550 msgr2=0x7fd33006c2d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:23:14.084 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:14.083+0000 7fd334e32640 1 -- 172.21.15.143:0/2149189164 shutdown_connections 2023-11-26T07:23:14.084 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:14.083+0000 7fd334e32640 1 -- 172.21.15.143:0/2149189164 wait complete. 2023-11-26T07:23:14.103 DEBUG:teuthology.orchestra.run.smithi143:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-11-26T07:23:14.242 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:14.239+0000 7f1ac725a640 1 Processor -- start 2023-11-26T07:23:14.243 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:14.239+0000 7f1ac725a640 1 -- start start 2023-11-26T07:23:14.244 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:14.243+0000 7f1ac725a640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f1ac00cf990 0x7f1ac015efa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:23:14.244 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:14.243+0000 7f1ac725a640 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7f1ac015f4e0 0x7f1ac00cd7b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:23:14.244 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:14.243+0000 7f1ac725a640 1 -- --> v1:172.21.15.188:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1ac006c520 con 0x7f1ac013e290 2023-11-26T07:23:14.244 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:14.243+0000 7f1ac725a640 1 -- --> v2:172.21.15.188:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1ac006c800 con 0x7f1ac015f4e0 2023-11-26T07:23:14.244 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:14.243+0000 7f1ac725a640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1ac006cae0 con 0x7f1ac00cf990 2023-11-26T07:23:14.245 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:14.243+0000 7f1abffff640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f1ac00cf990 0x7f1ac015efa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:23:14.245 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:14.243+0000 7f1ac57f7640 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7f1ac015f4e0 0x7f1ac00cd7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:23:14.245 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:14.243+0000 7f1ac57f7640 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7f1ac015f4e0 0x7f1ac00cd7b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.188:3300/0 says I am v2:172.21.15.143:45044/0 (socket says 172.21.15.143:45044) 2023-11-26T07:23:14.245 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:14.243+0000 7f1ac57f7640 1 -- 172.21.15.143:0/3431895782 learned_addr learned my addr 172.21.15.143:0/3431895782 (peer_addr_for_me v2:172.21.15.143:0/0) 2023-11-26T07:23:14.245 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:14.243+0000 7f1ac4ff6640 1 --1- 172.21.15.143:0/3431895782 >> v1:172.21.15.188:6789/0 conn(0x7f1ac013e290 0x7f1ac00cf280 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.188:6789/0 says I am v1:172.21.15.143:45128/0 (socket says 172.21.15.143:45128) 2023-11-26T07:23:14.245 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:14.243+0000 7f1abffff640 1 -- 172.21.15.143:0/3431895782 >> v2:172.21.15.188:3300/0 conn(0x7f1ac015f4e0 msgr2=0x7f1ac00cd7b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-26T07:23:14.245 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:14.243+0000 7f1abffff640 1 --2- 172.21.15.143:0/3431895782 >> v2:172.21.15.188:3300/0 conn(0x7f1ac015f4e0 0x7f1ac00cd7b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:23:14.246 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:14.243+0000 7f1abffff640 1 -- 172.21.15.143:0/3431895782 >> v1:172.21.15.188:6789/0 conn(0x7f1ac013e290 legacy=0x7f1ac00cf280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:23:14.246 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:14.243+0000 7f1abf7fe640 1 -- 172.21.15.143:0/3431895782 <== mon.1 v1:172.21.15.188:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3445113493 0 0) 0x7f1ac006c520 con 0x7f1ac013e290 2023-11-26T07:23:14.246 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:14.243+0000 7f1abffff640 1 -- 172.21.15.143:0/3431895782 --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1ac006cdc0 con 0x7f1ac00cf990 2023-11-26T07:23:14.248 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:14.247+0000 7f1abffff640 1 --2- 172.21.15.143:0/3431895782 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f1ac00cf990 0x7f1ac015efa0 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7f1aac0097c0 tx=0x7f1aac01c450 comp rx=0 tx=0).ready entity=mon.2 client_cookie=b0b73ff9dab82eb9 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:23:14.248 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:14.247+0000 7f1abf7fe640 1 -- 172.21.15.143:0/3431895782 <== mon.2 v2:172.21.15.188:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1aac0ab670 con 0x7f1ac00cf990 2023-11-26T07:23:14.248 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:14.247+0000 7f1abf7fe640 1 -- 172.21.15.143:0/3431895782 <== mon.2 v2:172.21.15.188:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1aac0ab7d0 con 0x7f1ac00cf990 2023-11-26T07:23:14.248 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:14.247+0000 7f1abf7fe640 1 -- 172.21.15.143:0/3431895782 <== mon.2 v2:172.21.15.188:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1aac0abb60 con 0x7f1ac00cf990 2023-11-26T07:23:14.249 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:14.247+0000 7f1ac725a640 1 -- 172.21.15.143:0/3431895782 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f1ac00cf990 msgr2=0x7f1ac015efa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:23:14.249 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:14.247+0000 7f1ac725a640 1 --2- 172.21.15.143:0/3431895782 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f1ac00cf990 0x7f1ac015efa0 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7f1aac0097c0 tx=0x7f1aac01c450 comp rx=0 tx=0).stop 2023-11-26T07:23:14.249 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:14.247+0000 7f1ac725a640 1 -- 172.21.15.143:0/3431895782 shutdown_connections 2023-11-26T07:23:14.249 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:14.247+0000 7f1ac725a640 1 --2- 172.21.15.143:0/3431895782 >> v2:172.21.15.188:3300/0 conn(0x7f1ac015f4e0 0x7f1ac00cd7b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:23:14.249 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:14.247+0000 7f1ac725a640 1 --2- 172.21.15.143:0/3431895782 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f1ac00cf990 0x7f1ac015efa0 unknown :-1 s=CLOSED pgs=138 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:23:14.249 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:14.247+0000 7f1ac725a640 1 -- 172.21.15.143:0/3431895782 >> 172.21.15.143:0/3431895782 conn(0x7f1ac006a890 msgr2=0x7f1ac006aca0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:23:14.250 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:14.247+0000 7f1ac725a640 1 -- 172.21.15.143:0/3431895782 shutdown_connections 2023-11-26T07:23:14.250 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:14.247+0000 7f1ac725a640 1 -- 172.21.15.143:0/3431895782 wait complete. 2023-11-26T07:23:14.250 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:14.247+0000 7f1ac725a640 1 Processor -- start 2023-11-26T07:23:14.251 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:14.251+0000 7f1ac725a640 1 -- start start 2023-11-26T07:23:14.252 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:14.251+0000 7f1ac725a640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f1ac00cf990 0x7f1ac01375f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:23:14.252 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:14.251+0000 7f1ac725a640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f1ac013e290 0x7f1ac0150db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:23:14.252 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:14.251+0000 7f1ac725a640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f1ac015f4e0 0x7f1ac0139490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:23:14.252 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:14.251+0000 7f1ac725a640 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1ac00b1340 con 0x7f1ac015f4e0 2023-11-26T07:23:14.252 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:14.251+0000 7f1ac725a640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1ac0151730 con 0x7f1ac00cf990 2023-11-26T07:23:14.253 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:14.251+0000 7f1ac725a640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1ac0154590 con 0x7f1ac013e290 2023-11-26T07:23:14.253 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:14.251+0000 7f1ac4ff6640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f1ac00cf990 0x7f1ac01375f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:23:14.253 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:14.251+0000 7f1ac57f7640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f1ac015f4e0 0x7f1ac0139490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:23:14.253 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:14.251+0000 7f1ac4ff6640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f1ac00cf990 0x7f1ac01375f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.188:3300/0 says I am v2:172.21.15.143:45048/0 (socket says 172.21.15.143:45048) 2023-11-26T07:23:14.253 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:14.251+0000 7f1ac4ff6640 1 -- 172.21.15.143:0/113908839 learned_addr learned my addr 172.21.15.143:0/113908839 (peer_addr_for_me v2:172.21.15.143:0/0) 2023-11-26T07:23:14.253 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:14.251+0000 7f1abffff640 1 --2- 172.21.15.143:0/113908839 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f1ac013e290 0x7f1ac0150db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:23:14.254 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:14.251+0000 7f1ac4ff6640 1 -- 172.21.15.143:0/113908839 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f1ac013e290 msgr2=0x7f1ac0150db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:23:14.254 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:14.251+0000 7f1ac4ff6640 1 --2- 172.21.15.143:0/113908839 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f1ac013e290 0x7f1ac0150db0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:23:14.254 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:14.251+0000 7f1ac4ff6640 1 -- 172.21.15.143:0/113908839 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f1ac015f4e0 msgr2=0x7f1ac0139490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:23:14.254 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:14.251+0000 7f1ac4ff6640 1 --2- 172.21.15.143:0/113908839 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f1ac015f4e0 0x7f1ac0139490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:23:14.254 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:14.251+0000 7f1ac4ff6640 1 -- 172.21.15.143:0/113908839 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1aac009420 con 0x7f1ac00cf990 2023-11-26T07:23:14.254 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:14.251+0000 7f1ac4ff6640 1 --2- 172.21.15.143:0/113908839 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f1ac00cf990 0x7f1ac01375f0 secure :-1 s=READY pgs=422 cs=0 l=1 rev1=1 crypto rx=0x7f1ab40011f0 tx=0x7f1ab400c9b0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:23:14.254 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:14.251+0000 7f1abd7fa640 1 -- 172.21.15.143:0/113908839 <== mon.1 v2:172.21.15.188:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1ab40074e0 con 0x7f1ac00cf990 2023-11-26T07:23:14.254 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:14.251+0000 7f1abd7fa640 1 -- 172.21.15.143:0/113908839 <== mon.1 v2:172.21.15.188:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1ab4007640 con 0x7f1ac00cf990 2023-11-26T07:23:14.255 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:14.251+0000 7f1abd7fa640 1 -- 172.21.15.143:0/113908839 <== mon.1 v2:172.21.15.188:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1ab4004a10 con 0x7f1ac00cf990 2023-11-26T07:23:14.255 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:14.251+0000 7f1ac725a640 1 -- 172.21.15.143:0/113908839 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1ac00b1590 con 0x7f1ac00cf990 2023-11-26T07:23:14.255 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:14.251+0000 7f1ac725a640 1 -- 172.21.15.143:0/113908839 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1ac01512f0 con 0x7f1ac00cf990 2023-11-26T07:23:14.255 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:14.251+0000 7f1abd7fa640 1 -- 172.21.15.143:0/113908839 <== mon.1 v2:172.21.15.188:3300/0 4 ==== mgrmap(e 14) v1 ==== 43229+0+0 (secure 0 0 0) 0x7f1ab4004b70 con 0x7f1ac00cf990 2023-11-26T07:23:14.256 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:14.255+0000 7f1ac725a640 1 -- 172.21.15.143:0/113908839 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1ac00cc700 con 0x7f1ac00cf990 2023-11-26T07:23:14.256 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:14.255+0000 7f1abd7fa640 1 --2- 172.21.15.143:0/113908839 >> [v2:172.21.15.188:6836/3401128553,v1:172.21.15.188:6837/3401128553] conn(0x7f1aa40354e0 0x7f1aa40379a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:23:14.256 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:14.255+0000 7f1abd7fa640 1 -- 172.21.15.143:0/113908839 <== mon.1 v2:172.21.15.188:3300/0 5 ==== osd_map(72..72 src has 1..72) v4 ==== 6268+0+0 (secure 0 0 0) 0x7f1ab40474b0 con 0x7f1ac00cf990 2023-11-26T07:23:14.256 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:14.255+0000 7f1abffff640 1 --2- 172.21.15.143:0/113908839 >> [v2:172.21.15.188:6836/3401128553,v1:172.21.15.188:6837/3401128553] conn(0x7f1aa40354e0 0x7f1aa40379a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:23:14.256 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:14.255+0000 7f1abffff640 1 --2- 172.21.15.143:0/113908839 >> [v2:172.21.15.188:6836/3401128553,v1:172.21.15.188:6837/3401128553] conn(0x7f1aa40354e0 0x7f1aa40379a0 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7f1aac0af000 tx=0x7f1aac0a7000 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:23:14.260 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:14.259+0000 7f1abd7fa640 1 -- 172.21.15.143:0/113908839 <== mon.1 v2:172.21.15.188:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172649 (secure 0 0 0) 0x7f1ab400ece0 con 0x7f1ac00cf990 2023-11-26T07:23:14.449 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:14.447+0000 7f1ac725a640 1 -- 172.21.15.143:0/113908839 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f1ac0067680 con 0x7f1ac00cf990 2023-11-26T07:23:14.450 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:14.447+0000 7f1abd7fa640 1 -- 172.21.15.143:0/113908839 <== mon.1 v2:172.21.15.188:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 43 v43) v1 ==== 94+0+4924 (secure 0 0 0) 0x7f1ab4018020 con 0x7f1ac00cf990 2023-11-26T07:23:14.450 INFO:teuthology.orchestra.run.smithi143.stdout: 2023-11-26T07:23:14.450 INFO:teuthology.orchestra.run.smithi143.stdout:{"epoch":43,"default_fscid":5,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5927,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.143:6837/578144260","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6835","nonce":578144260},{"type":"v1","addr":"172.21.15.143:6837","nonce":578144260}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":38},{"gid":5930,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.188:6834/1048736873","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6832","nonce":1048736873},{"type":"v1","addr":"172.21.15.188:6834","nonce":1048736873}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":38},{"gid":5933,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.188:6835/2063768716","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6833","nonce":2063768716},{"type":"v1","addr":"172.21.15.188:6835","nonce":2063768716}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":38}],"filesystems":[{"mdsmap":{"epoch":42,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-11-26T07:22:37.220113+0000","modified":"2023-11-26T07:22:40.243052+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":6553},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6553":{"gid":6553,"name":"a","rank":0,"incarnation":39,"state":"up:active","state_seq":3,"addr":"172.21.15.143:6836/2590663349","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6834","nonce":2590663349},{"type":"v1","addr":"172.21.15.143:6836","nonce":2590663349}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[11],"metadata_pool":10,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":5}]} 2023-11-26T07:23:14.451 INFO:teuthology.orchestra.run.smithi143.stderr:dumped fsmap epoch 43 2023-11-26T07:23:14.453 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:14.451+0000 7f1ac725a640 1 -- 172.21.15.143:0/113908839 >> [v2:172.21.15.188:6836/3401128553,v1:172.21.15.188:6837/3401128553] conn(0x7f1aa40354e0 msgr2=0x7f1aa40379a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:23:14.453 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:14.451+0000 7f1ac725a640 1 --2- 172.21.15.143:0/113908839 >> [v2:172.21.15.188:6836/3401128553,v1:172.21.15.188:6837/3401128553] conn(0x7f1aa40354e0 0x7f1aa40379a0 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7f1aac0af000 tx=0x7f1aac0a7000 comp rx=0 tx=0).stop 2023-11-26T07:23:14.453 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:14.451+0000 7f1ac725a640 1 -- 172.21.15.143:0/113908839 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f1ac00cf990 msgr2=0x7f1ac01375f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:23:14.453 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:14.451+0000 7f1ac725a640 1 --2- 172.21.15.143:0/113908839 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f1ac00cf990 0x7f1ac01375f0 secure :-1 s=READY pgs=422 cs=0 l=1 rev1=1 crypto rx=0x7f1ab40011f0 tx=0x7f1ab400c9b0 comp rx=0 tx=0).stop 2023-11-26T07:23:14.453 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:14.451+0000 7f1ac725a640 1 -- 172.21.15.143:0/113908839 shutdown_connections 2023-11-26T07:23:14.453 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:14.451+0000 7f1ac725a640 1 --2- 172.21.15.143:0/113908839 >> [v2:172.21.15.188:6836/3401128553,v1:172.21.15.188:6837/3401128553] conn(0x7f1aa40354e0 0x7f1aa40379a0 unknown :-1 s=CLOSED pgs=26 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:23:14.453 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:14.451+0000 7f1ac725a640 1 --2- 172.21.15.143:0/113908839 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f1ac015f4e0 0x7f1ac0139490 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:23:14.453 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:14.451+0000 7f1ac725a640 1 --2- 172.21.15.143:0/113908839 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f1ac013e290 0x7f1ac0150db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:23:14.453 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:14.451+0000 7f1ac725a640 1 --2- 172.21.15.143:0/113908839 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f1ac00cf990 0x7f1ac01375f0 unknown :-1 s=CLOSED pgs=422 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:23:14.453 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:14.451+0000 7f1ac725a640 1 -- 172.21.15.143:0/113908839 >> 172.21.15.143:0/113908839 conn(0x7f1ac006a890 msgr2=0x7f1ac006ac70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:23:14.453 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:14.451+0000 7f1ac725a640 1 -- 172.21.15.143:0/113908839 shutdown_connections 2023-11-26T07:23:14.454 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:14.451+0000 7f1ac725a640 1 -- 172.21.15.143:0/113908839 wait complete. 2023-11-26T07:23:14.472 DEBUG:teuthology.orchestra.run.smithi143:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-11-26T07:23:14.532 INFO:tasks.ceph.mgr.y.smithi143.stderr:2023-11-26T07:23:14.531+0000 7f85d6276180 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2023-11-26T07:23:14.615 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:14.611+0000 7f875309a640 1 Processor -- start 2023-11-26T07:23:14.615 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:14.615+0000 7f875309a640 1 -- start start 2023-11-26T07:23:14.617 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:14.615+0000 7f875309a640 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7f874c0b2240 0x7f874c0d0590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:23:14.618 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:14.615+0000 7f875309a640 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7f874c0d14e0 0x7f874c0d1960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:23:14.618 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:14.615+0000 7f875309a640 1 -- --> v1:172.21.15.143:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f874c0d4140 con 0x7f874c160aa0 2023-11-26T07:23:14.618 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:14.615+0000 7f875309a640 1 -- --> v2:172.21.15.143:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f874c0ce440 con 0x7f874c0b2240 2023-11-26T07:23:14.618 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:14.615+0000 7f875309a640 1 -- --> v2:172.21.15.188:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f874c0d1ea0 con 0x7f874c0d14e0 2023-11-26T07:23:14.618 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:14.615+0000 7f874bfff640 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7f874c0b2240 0x7f874c0d0590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:23:14.618 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:14.615+0000 7f874bfff640 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7f874c0b2240 0x7f874c0d0590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:40764/0 (socket says 172.21.15.143:40764) 2023-11-26T07:23:14.618 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:14.615+0000 7f874bfff640 1 -- 172.21.15.143:0/2277135135 learned_addr learned my addr 172.21.15.143:0/2277135135 (peer_addr_for_me v2:172.21.15.143:0/0) 2023-11-26T07:23:14.618 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:14.615+0000 7f8751637640 1 --2- 172.21.15.143:0/2277135135 >> v2:172.21.15.188:3300/0 conn(0x7f874c0d14e0 0x7f874c0d1960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:23:14.618 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:14.615+0000 7f874bfff640 1 -- 172.21.15.143:0/2277135135 >> v2:172.21.15.188:3300/0 conn(0x7f874c0d14e0 msgr2=0x7f874c0d1960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-26T07:23:14.619 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:14.615+0000 7f874bfff640 1 --2- 172.21.15.143:0/2277135135 >> v2:172.21.15.188:3300/0 conn(0x7f874c0d14e0 0x7f874c0d1960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:23:14.619 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:14.615+0000 7f874bfff640 1 -- 172.21.15.143:0/2277135135 >> v1:172.21.15.143:6789/0 conn(0x7f874c160aa0 legacy=0x7f874c0b2b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:23:14.619 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:14.615+0000 7f874bfff640 1 -- 172.21.15.143:0/2277135135 --> v2:172.21.15.143:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f874c0d3930 con 0x7f874c0b2240 2023-11-26T07:23:14.620 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:14.619+0000 7f874bfff640 1 --2- 172.21.15.143:0/2277135135 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f874c0b2240 0x7f874c0d0590 secure :-1 s=READY pgs=1073 cs=0 l=1 rev1=1 crypto rx=0x7f874c0b4cb0 tx=0x7f873801c470 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f0b1da0095fb0eac server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:23:14.621 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:14.619+0000 7f874b7fe640 1 -- 172.21.15.143:0/2277135135 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f87380ad660 con 0x7f874c0b2240 2023-11-26T07:23:14.621 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:14.619+0000 7f874b7fe640 1 -- 172.21.15.143:0/2277135135 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f87380ad7c0 con 0x7f874c0b2240 2023-11-26T07:23:14.621 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:14.619+0000 7f874b7fe640 1 -- 172.21.15.143:0/2277135135 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f87380adb50 con 0x7f874c0b2240 2023-11-26T07:23:14.622 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:14.619+0000 7f875309a640 1 -- 172.21.15.143:0/2277135135 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f874c0b2240 msgr2=0x7f874c0d0590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:23:14.622 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:14.619+0000 7f875309a640 1 --2- 172.21.15.143:0/2277135135 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f874c0b2240 0x7f874c0d0590 secure :-1 s=READY pgs=1073 cs=0 l=1 rev1=1 crypto rx=0x7f874c0b4cb0 tx=0x7f873801c470 comp rx=0 tx=0).stop 2023-11-26T07:23:14.622 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:14.619+0000 7f875309a640 1 -- 172.21.15.143:0/2277135135 shutdown_connections 2023-11-26T07:23:14.622 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:14.619+0000 7f875309a640 1 --2- 172.21.15.143:0/2277135135 >> v2:172.21.15.188:3300/0 conn(0x7f874c0d14e0 0x7f874c0d1960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:23:14.622 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:14.619+0000 7f875309a640 1 --2- 172.21.15.143:0/2277135135 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f874c0b2240 0x7f874c0d0590 unknown :-1 s=CLOSED pgs=1073 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:23:14.622 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:14.619+0000 7f875309a640 1 -- 172.21.15.143:0/2277135135 >> 172.21.15.143:0/2277135135 conn(0x7f874c09c550 msgr2=0x7f874c0d3220 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:23:14.622 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:14.619+0000 7f875309a640 1 -- 172.21.15.143:0/2277135135 shutdown_connections 2023-11-26T07:23:14.622 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:14.619+0000 7f875309a640 1 -- 172.21.15.143:0/2277135135 wait complete. 2023-11-26T07:23:14.623 INFO:tasks.ceph.mgr.y.smithi143.stderr:2023-11-26T07:23:14.619+0000 7f85d6276180 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2023-11-26T07:23:14.623 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:14.623+0000 7f875309a640 1 Processor -- start 2023-11-26T07:23:14.623 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:14.623+0000 7f875309a640 1 -- start start 2023-11-26T07:23:14.625 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:14.623+0000 7f875309a640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f874c0b2240 0x7f874c154950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:23:14.625 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:14.623+0000 7f875309a640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f874c0d14e0 0x7f874c0c7f90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:23:14.625 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:14.623+0000 7f875309a640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f874c160aa0 0x7f874c0c84d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:23:14.625 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:14.623+0000 7f875309a640 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f874c0ca2f0 con 0x7f874c0b2240 2023-11-26T07:23:14.626 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:14.623+0000 7f875309a640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f874c155f90 con 0x7f874c160aa0 2023-11-26T07:23:14.626 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:14.623+0000 7f875309a640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f874c156270 con 0x7f874c0d14e0 2023-11-26T07:23:14.626 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:14.623+0000 7f8750e36640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f874c0b2240 0x7f874c154950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:23:14.626 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:14.623+0000 7f8750e36640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f874c0b2240 0x7f874c154950 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:40778/0 (socket says 172.21.15.143:40778) 2023-11-26T07:23:14.626 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:14.623+0000 7f8750e36640 1 -- 172.21.15.143:0/2354693322 learned_addr learned my addr 172.21.15.143:0/2354693322 (peer_addr_for_me v2:172.21.15.143:0/0) 2023-11-26T07:23:14.626 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:14.623+0000 7f8751637640 1 --2- 172.21.15.143:0/2354693322 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f874c160aa0 0x7f874c0c84d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:23:14.626 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:14.623+0000 7f874bfff640 1 --2- 172.21.15.143:0/2354693322 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f874c0d14e0 0x7f874c0c7f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:23:14.626 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:14.623+0000 7f8750e36640 1 -- 172.21.15.143:0/2354693322 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f874c0d14e0 msgr2=0x7f874c0c7f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:23:14.626 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:14.623+0000 7f8750e36640 1 --2- 172.21.15.143:0/2354693322 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f874c0d14e0 0x7f874c0c7f90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:23:14.626 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:14.623+0000 7f8750e36640 1 -- 172.21.15.143:0/2354693322 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f874c160aa0 msgr2=0x7f874c0c84d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:23:14.626 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:14.623+0000 7f8750e36640 1 --2- 172.21.15.143:0/2354693322 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f874c160aa0 0x7f874c0c84d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:23:14.627 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:14.623+0000 7f8750e36640 1 -- 172.21.15.143:0/2354693322 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f87380094d0 con 0x7f874c0b2240 2023-11-26T07:23:14.628 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:14.623+0000 7f8750e36640 1 --2- 172.21.15.143:0/2354693322 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f874c0b2240 0x7f874c154950 secure :-1 s=READY pgs=1074 cs=0 l=1 rev1=1 crypto rx=0x7f874000c3c0 tx=0x7f874000c880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:23:14.628 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:14.623+0000 7f87497fa640 1 -- 172.21.15.143:0/2354693322 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f87400052d0 con 0x7f874c0b2240 2023-11-26T07:23:14.628 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:14.623+0000 7f87497fa640 1 -- 172.21.15.143:0/2354693322 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8740005430 con 0x7f874c0b2240 2023-11-26T07:23:14.628 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:14.623+0000 7f87497fa640 1 -- 172.21.15.143:0/2354693322 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8740003840 con 0x7f874c0b2240 2023-11-26T07:23:14.628 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:14.623+0000 7f875309a640 1 -- 172.21.15.143:0/2354693322 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f874c0ca540 con 0x7f874c0b2240 2023-11-26T07:23:14.628 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:14.623+0000 7f875309a640 1 -- 172.21.15.143:0/2354693322 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f874c154e90 con 0x7f874c0b2240 2023-11-26T07:23:14.628 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:14.627+0000 7f87497fa640 1 -- 172.21.15.143:0/2354693322 <== mon.0 v2:172.21.15.143:3300/0 4 ==== mgrmap(e 15) v1 ==== 43285+0+0 (secure 0 0 0) 0x7f87400039a0 con 0x7f874c0b2240 2023-11-26T07:23:14.628 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:14.627+0000 7f875309a640 1 -- 172.21.15.143:0/2354693322 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f874c0caa60 con 0x7f874c0b2240 2023-11-26T07:23:14.629 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:14.627+0000 7f87497fa640 1 --2- 172.21.15.143:0/2354693322 >> [v2:172.21.15.188:6836/3401128553,v1:172.21.15.188:6837/3401128553] conn(0x7f8730035570 0x7f8730037a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:23:14.629 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:14.627+0000 7f87497fa640 1 -- 172.21.15.143:0/2354693322 <== mon.0 v2:172.21.15.143:3300/0 5 ==== osd_map(72..72 src has 1..72) v4 ==== 6268+0+0 (secure 0 0 0) 0x7f87400476f0 con 0x7f874c0b2240 2023-11-26T07:23:14.629 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:14.627+0000 7f874bfff640 1 --2- 172.21.15.143:0/2354693322 >> [v2:172.21.15.188:6836/3401128553,v1:172.21.15.188:6837/3401128553] conn(0x7f8730035570 0x7f8730037a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:23:14.629 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:14.627+0000 7f874bfff640 1 --2- 172.21.15.143:0/2354693322 >> [v2:172.21.15.188:6836/3401128553,v1:172.21.15.188:6837/3401128553] conn(0x7f8730035570 0x7f8730037a30 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7f87380af000 tx=0x7f8738008040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:23:14.632 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:14.631+0000 7f87497fa640 1 -- 172.21.15.143:0/2354693322 <== mon.0 v2:172.21.15.143:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172649 (secure 0 0 0) 0x7f874c0caa60 con 0x7f874c0b2240 2023-11-26T07:23:14.775 INFO:tasks.ceph.mgr.y.smithi143.stderr:2023-11-26T07:23:14.775+0000 7f85d6276180 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2023-11-26T07:23:14.787 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:14.787+0000 7f875309a640 1 -- 172.21.15.143:0/2354693322 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f874c0caa60 con 0x7f874c0b2240 2023-11-26T07:23:14.789 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:14.787+0000 7f87497fa640 1 -- 172.21.15.143:0/2354693322 <== mon.0 v2:172.21.15.143:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v72) v1 ==== 74+0+17423 (secure 0 0 0) 0x7f874c0caa60 con 0x7f874c0b2240 2023-11-26T07:23:14.789 INFO:teuthology.orchestra.run.smithi143.stdout: 2023-11-26T07:23:14.789 INFO:teuthology.orchestra.run.smithi143.stdout:{"epoch":72,"fsid":"f5033060-2469-4a07-95fe-2b76f5404bc1","created":"2023-11-26T07:17:28.613663+0000","modified":"2023-11-26T07:23:11.261981+0000","last_up_change":"2023-11-26T07:17:41.157813+0000","last_in_change":"2023-11-26T07:17:31.181658+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":11,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-11-26T07:17:41.758597+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":10,"pool_name":"cephfs_metadata","create_time":"2023-11-26T07:22:35.314819+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"70","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":2.5,"score_stable":2.5,"optimal_score":1,"raw_score_acting":2.5,"raw_score_stable":2.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":11,"pool_name":"cephfs_data","create_time":"2023-11-26T07:22:36.219536+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"71","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"ebbee2c6-023e-44b9-9f1d-6f7bfd309730","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":69,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6817","nonce":639807369},{"type":"v1","addr":"172.21.15.143:6819","nonce":639807369}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6821","nonce":639807369},{"type":"v1","addr":"172.21.15.143:6823","nonce":639807369}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6830","nonce":639807369},{"type":"v1","addr":"172.21.15.143:6831","nonce":639807369}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6825","nonce":639807369},{"type":"v1","addr":"172.21.15.143:6828","nonce":639807369}]},"public_addr":"172.21.15.143:6819/639807369","cluster_addr":"172.21.15.143:6823/639807369","heartbeat_back_addr":"172.21.15.143:6831/639807369","heartbeat_front_addr":"172.21.15.143:6828/639807369","state":["exists","up"]},{"osd":1,"uuid":"fec4b8be-7122-49e9-9a19-5850a0700001","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":69,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6816","nonce":1144942530},{"type":"v1","addr":"172.21.15.143:6818","nonce":1144942530}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6820","nonce":1144942530},{"type":"v1","addr":"172.21.15.143:6822","nonce":1144942530}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6827","nonce":1144942530},{"type":"v1","addr":"172.21.15.143:6829","nonce":1144942530}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6824","nonce":1144942530},{"type":"v1","addr":"172.21.15.143:6826","nonce":1144942530}]},"public_addr":"172.21.15.143:6818/1144942530","cluster_addr":"172.21.15.143:6822/1144942530","heartbeat_back_addr":"172.21.15.143:6829/1144942530","heartbeat_front_addr":"172.21.15.143:6826/1144942530","state":["exists","up"]},{"osd":2,"uuid":"2888af5b-f933-46ca-8f95-6dd494ff43f6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":69,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6808","nonce":3909095186},{"type":"v1","addr":"172.21.15.143:6809","nonce":3909095186}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6810","nonce":3909095186},{"type":"v1","addr":"172.21.15.143:6811","nonce":3909095186}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6814","nonce":3909095186},{"type":"v1","addr":"172.21.15.143:6815","nonce":3909095186}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6812","nonce":3909095186},{"type":"v1","addr":"172.21.15.143:6813","nonce":3909095186}]},"public_addr":"172.21.15.143:6809/3909095186","cluster_addr":"172.21.15.143:6811/3909095186","heartbeat_back_addr":"172.21.15.143:6815/3909095186","heartbeat_front_addr":"172.21.15.143:6813/3909095186","state":["exists","up"]},{"osd":3,"uuid":"d1f52435-7bc1-4081-9550-9cd1ba55f549","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":69,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6800","nonce":3854210991},{"type":"v1","addr":"172.21.15.143:6801","nonce":3854210991}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6802","nonce":3854210991},{"type":"v1","addr":"172.21.15.143:6803","nonce":3854210991}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6806","nonce":3854210991},{"type":"v1","addr":"172.21.15.143:6807","nonce":3854210991}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6804","nonce":3854210991},{"type":"v1","addr":"172.21.15.143:6805","nonce":3854210991}]},"public_addr":"172.21.15.143:6801/3854210991","cluster_addr":"172.21.15.143:6803/3854210991","heartbeat_back_addr":"172.21.15.143:6807/3854210991","heartbeat_front_addr":"172.21.15.143:6805/3854210991","state":["exists","up"]},{"osd":4,"uuid":"e2ae8b6e-6e25-49b2-bcbf-aacce68f06a3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":69,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6800","nonce":3924606902},{"type":"v1","addr":"172.21.15.188:6801","nonce":3924606902}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6802","nonce":3924606902},{"type":"v1","addr":"172.21.15.188:6803","nonce":3924606902}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6806","nonce":3924606902},{"type":"v1","addr":"172.21.15.188:6807","nonce":3924606902}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6804","nonce":3924606902},{"type":"v1","addr":"172.21.15.188:6805","nonce":3924606902}]},"public_addr":"172.21.15.188:6801/3924606902","cluster_addr":"172.21.15.188:6803/3924606902","heartbeat_back_addr":"172.21.15.188:6807/3924606902","heartbeat_front_addr":"172.21.15.188:6805/3924606902","state":["exists","up"]},{"osd":5,"uuid":"3cec8a32-7698-4c36-8e89-e4ba11934f33","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":69,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6813","nonce":186841544},{"type":"v1","addr":"172.21.15.188:6815","nonce":186841544}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6817","nonce":186841544},{"type":"v1","addr":"172.21.15.188:6818","nonce":186841544}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6822","nonce":186841544},{"type":"v1","addr":"172.21.15.188:6823","nonce":186841544}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6820","nonce":186841544},{"type":"v1","addr":"172.21.15.188:6821","nonce":186841544}]},"public_addr":"172.21.15.188:6815/186841544","cluster_addr":"172.21.15.188:6818/186841544","heartbeat_back_addr":"172.21.15.188:6823/186841544","heartbeat_front_addr":"172.21.15.188:6821/186841544","state":["exists","up"]},{"osd":6,"uuid":"58a65d5d-e090-473a-81b4-ad38e85029f9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":69,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6808","nonce":2153499405},{"type":"v1","addr":"172.21.15.188:6809","nonce":2153499405}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6810","nonce":2153499405},{"type":"v1","addr":"172.21.15.188:6811","nonce":2153499405}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6816","nonce":2153499405},{"type":"v1","addr":"172.21.15.188:6819","nonce":2153499405}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6812","nonce":2153499405},{"type":"v1","addr":"172.21.15.188:6814","nonce":2153499405}]},"public_addr":"172.21.15.188:6809/2153499405","cluster_addr":"172.21.15.188:6811/2153499405","heartbeat_back_addr":"172.21.15.188:6819/2153499405","heartbeat_front_addr":"172.21.15.188:6814/2153499405","state":["exists","up"]},{"osd":7,"uuid":"ea5b7f70-3554-4cf8-ac30-f7c59b80678d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":69,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6824","nonce":2722096729},{"type":"v1","addr":"172.21.15.188:6825","nonce":2722096729}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6826","nonce":2722096729},{"type":"v1","addr":"172.21.15.188:6827","nonce":2722096729}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6830","nonce":2722096729},{"type":"v1","addr":"172.21.15.188:6831","nonce":2722096729}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6828","nonce":2722096729},{"type":"v1","addr":"172.21.15.188:6829","nonce":2722096729}]},"public_addr":"172.21.15.188:6825/2722096729","cluster_addr":"172.21.15.188:6827/2722096729","heartbeat_back_addr":"172.21.15.188:6831/2722096729","heartbeat_front_addr":"172.21.15.188:6829/2722096729","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-26T07:17:36.995032+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-26T07:17:36.932266+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-26T07:17:36.829304+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-26T07:17:36.804297+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-26T07:17:36.667892+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-26T07:17:36.964868+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-26T07:17:36.827332+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-26T07:17:36.893179+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.143:0/248180418":"2023-11-27T07:23:11.261937+0000","172.21.15.143:0/253926057":"2023-11-27T07:23:11.261937+0000","172.21.15.143:0/3516040872":"2023-11-27T07:23:11.261937+0000","172.21.15.143:0/963510909":"2023-11-27T07:23:11.261937+0000","172.21.15.143:6833/2150557544":"2023-11-27T07:23:11.261937+0000","172.21.15.143:6832/2150557544":"2023-11-27T07:23:11.261937+0000","172.21.15.143:0/2346893898":"2023-11-27T07:23:11.261937+0000","172.21.15.143:0/3751308865":"2023-11-27T07:23:11.261937+0000","172.21.15.143:0/3813484262":"2023-11-27T07:23:11.261937+0000","172.21.15.143:0/1844383386":"2023-11-27T07:23:11.261937+0000","172.21.15.143:0/2591285564":"2023-11-27T07:23:11.261937+0000","172.21.15.143:0/631097144":"2023-11-27T07:23:11.261937+0000","172.21.15.143:0/103646748":"2023-11-27T07:23:11.261937+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-11-26T07:23:14.791 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:14.791+0000 7f875309a640 1 -- 172.21.15.143:0/2354693322 >> [v2:172.21.15.188:6836/3401128553,v1:172.21.15.188:6837/3401128553] conn(0x7f8730035570 msgr2=0x7f8730037a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:23:14.791 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:14.791+0000 7f875309a640 1 --2- 172.21.15.143:0/2354693322 >> [v2:172.21.15.188:6836/3401128553,v1:172.21.15.188:6837/3401128553] conn(0x7f8730035570 0x7f8730037a30 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7f87380af000 tx=0x7f8738008040 comp rx=0 tx=0).stop 2023-11-26T07:23:14.791 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:14.791+0000 7f875309a640 1 -- 172.21.15.143:0/2354693322 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f874c0b2240 msgr2=0x7f874c154950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:23:14.792 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:14.791+0000 7f875309a640 1 --2- 172.21.15.143:0/2354693322 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f874c0b2240 0x7f874c154950 secure :-1 s=READY pgs=1074 cs=0 l=1 rev1=1 crypto rx=0x7f874000c3c0 tx=0x7f874000c880 comp rx=0 tx=0).stop 2023-11-26T07:23:14.792 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:14.791+0000 7f875309a640 1 -- 172.21.15.143:0/2354693322 shutdown_connections 2023-11-26T07:23:14.792 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:14.791+0000 7f875309a640 1 --2- 172.21.15.143:0/2354693322 >> [v2:172.21.15.188:6836/3401128553,v1:172.21.15.188:6837/3401128553] conn(0x7f8730035570 0x7f8730037a30 unknown :-1 s=CLOSED pgs=27 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:23:14.792 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:14.791+0000 7f875309a640 1 --2- 172.21.15.143:0/2354693322 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f874c160aa0 0x7f874c0c84d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:23:14.792 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:14.791+0000 7f875309a640 1 --2- 172.21.15.143:0/2354693322 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f874c0d14e0 0x7f874c0c7f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:23:14.792 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:14.791+0000 7f875309a640 1 --2- 172.21.15.143:0/2354693322 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f874c0b2240 0x7f874c154950 unknown :-1 s=CLOSED pgs=1074 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:23:14.792 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:14.791+0000 7f875309a640 1 -- 172.21.15.143:0/2354693322 >> 172.21.15.143:0/2354693322 conn(0x7f874c09c550 msgr2=0x7f874c0b3300 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:23:14.792 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:14.791+0000 7f875309a640 1 -- 172.21.15.143:0/2354693322 shutdown_connections 2023-11-26T07:23:14.792 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:14.791+0000 7f875309a640 1 -- 172.21.15.143:0/2354693322 wait complete. 2023-11-26T07:23:14.811 INFO:tasks.cephfs.filesystem:Destroying file system cephfs and related pools 2023-11-26T07:23:14.811 DEBUG:teuthology.orchestra.run.smithi143:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-11-26T07:23:14.863 INFO:tasks.ceph.mgr.y.smithi143.stderr:2023-11-26T07:23:14.863+0000 7f85d6276180 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2023-11-26T07:23:14.951 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:14.951+0000 7fa67dddf640 1 Processor -- start 2023-11-26T07:23:14.951 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:14.951+0000 7fa67dddf640 1 -- start start 2023-11-26T07:23:14.953 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:14.951+0000 7fa67dddf640 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7fa6781596d0 0x7fa678071850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:23:14.953 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:14.951+0000 7fa67dddf640 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7fa6780ce680 0x7fa6780b2870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:23:14.953 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:14.951+0000 7fa67dddf640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fa6780b2db0 0x7fa67815d2c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:23:14.953 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:14.951+0000 7fa67dddf640 1 -- --> v2:172.21.15.143:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa6780b3240 con 0x7fa6781596d0 2023-11-26T07:23:14.954 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:14.951+0000 7fa67dddf640 1 -- --> v2:172.21.15.188:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa6780cbe00 con 0x7fa6780ce680 2023-11-26T07:23:14.954 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:14.951+0000 7fa67dddf640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa67806c750 con 0x7fa6780b2db0 2023-11-26T07:23:14.954 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:14.951+0000 7fa6777fe640 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7fa6781596d0 0x7fa678071850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:23:14.954 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:14.951+0000 7fa6777fe640 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7fa6781596d0 0x7fa678071850 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:40790/0 (socket says 172.21.15.143:40790) 2023-11-26T07:23:14.954 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:14.951+0000 7fa6777fe640 1 -- 172.21.15.143:0/362191033 learned_addr learned my addr 172.21.15.143:0/362191033 (peer_addr_for_me v2:172.21.15.143:0/0) 2023-11-26T07:23:14.954 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:14.951+0000 7fa677fff640 1 --2- 172.21.15.143:0/362191033 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fa6780b2db0 0x7fa67815d2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:23:14.954 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:14.951+0000 7fa676ffd640 1 --2- 172.21.15.143:0/362191033 >> v2:172.21.15.188:3300/0 conn(0x7fa6780ce680 0x7fa6780b2870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:23:14.954 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:14.951+0000 7fa6777fe640 1 -- 172.21.15.143:0/362191033 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fa6780b2db0 msgr2=0x7fa67815d2c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-26T07:23:14.954 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:14.951+0000 7fa6777fe640 1 --2- 172.21.15.143:0/362191033 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fa6780b2db0 0x7fa67815d2c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:23:14.954 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:14.951+0000 7fa6777fe640 1 -- 172.21.15.143:0/362191033 >> v2:172.21.15.188:3300/0 conn(0x7fa6780ce680 msgr2=0x7fa6780b2870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-26T07:23:14.954 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:14.951+0000 7fa6777fe640 1 --2- 172.21.15.143:0/362191033 >> v2:172.21.15.188:3300/0 conn(0x7fa6780ce680 0x7fa6780b2870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:23:14.954 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:14.951+0000 7fa6777fe640 1 -- 172.21.15.143:0/362191033 --> v2:172.21.15.143:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa6780b3550 con 0x7fa6781596d0 2023-11-26T07:23:14.954 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:14.951+0000 7fa676ffd640 1 --2- 172.21.15.143:0/362191033 >> v2:172.21.15.188:3300/0 conn(0x7fa6780ce680 0x7fa6780b2870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-26T07:23:14.956 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:14.955+0000 7fa6777fe640 1 --2- 172.21.15.143:0/362191033 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fa6781596d0 0x7fa678071850 secure :-1 s=READY pgs=1075 cs=0 l=1 rev1=1 crypto rx=0x7fa6780ce440 tx=0x7fa66801c500 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a09d23d056c27c10 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:23:14.956 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:14.955+0000 7fa6767fc640 1 -- 172.21.15.143:0/362191033 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa6680ab600 con 0x7fa6781596d0 2023-11-26T07:23:14.957 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:14.955+0000 7fa6767fc640 1 -- 172.21.15.143:0/362191033 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa6680a7050 con 0x7fa6781596d0 2023-11-26T07:23:14.957 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:14.955+0000 7fa6767fc640 1 -- 172.21.15.143:0/362191033 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa6680ab980 con 0x7fa6781596d0 2023-11-26T07:23:14.958 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:14.955+0000 7fa67dddf640 1 -- 172.21.15.143:0/362191033 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fa6781596d0 msgr2=0x7fa678071850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:23:14.958 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:14.955+0000 7fa67dddf640 1 --2- 172.21.15.143:0/362191033 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fa6781596d0 0x7fa678071850 secure :-1 s=READY pgs=1075 cs=0 l=1 rev1=1 crypto rx=0x7fa6780ce440 tx=0x7fa66801c500 comp rx=0 tx=0).stop 2023-11-26T07:23:14.958 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:14.955+0000 7fa67dddf640 1 -- 172.21.15.143:0/362191033 shutdown_connections 2023-11-26T07:23:14.958 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:14.955+0000 7fa67dddf640 1 --2- 172.21.15.143:0/362191033 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fa6780b2db0 0x7fa67815d2c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:23:14.958 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:14.955+0000 7fa67dddf640 1 --2- 172.21.15.143:0/362191033 >> v2:172.21.15.188:3300/0 conn(0x7fa6780ce680 0x7fa6780b2870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:23:14.958 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:14.955+0000 7fa67dddf640 1 --2- 172.21.15.143:0/362191033 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fa6781596d0 0x7fa678071850 unknown :-1 s=CLOSED pgs=1075 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:23:14.958 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:14.955+0000 7fa67dddf640 1 -- 172.21.15.143:0/362191033 >> 172.21.15.143:0/362191033 conn(0x7fa67806a890 msgr2=0x7fa67806aca0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:23:14.958 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:14.955+0000 7fa67dddf640 1 -- 172.21.15.143:0/362191033 shutdown_connections 2023-11-26T07:23:14.958 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:14.955+0000 7fa67dddf640 1 -- 172.21.15.143:0/362191033 wait complete. 2023-11-26T07:23:14.959 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:14.959+0000 7fa67dddf640 1 Processor -- start 2023-11-26T07:23:14.959 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:14.959+0000 7fa67dddf640 1 -- start start 2023-11-26T07:23:14.961 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:14.959+0000 7fa67dddf640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fa6780b2db0 0x7fa6781f8a90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:23:14.961 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:14.959+0000 7fa67dddf640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fa6780ce680 0x7fa67815a640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:23:14.961 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:14.959+0000 7fa67dddf640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fa6781596d0 0x7fa67814e4c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:23:14.961 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:14.959+0000 7fa67dddf640 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa6781f8fd0 con 0x7fa6780ce680 2023-11-26T07:23:14.961 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:14.959+0000 7fa67dddf640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa67815b100 con 0x7fa6780b2db0 2023-11-26T07:23:14.961 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:14.959+0000 7fa67dddf640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa67815b3e0 con 0x7fa6781596d0 2023-11-26T07:23:14.961 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:14.959+0000 7fa676ffd640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fa6780ce680 0x7fa67815a640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:23:14.962 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:14.959+0000 7fa676ffd640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fa6780ce680 0x7fa67815a640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:40792/0 (socket says 172.21.15.143:40792) 2023-11-26T07:23:14.962 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:14.959+0000 7fa676ffd640 1 -- 172.21.15.143:0/876367259 learned_addr learned my addr 172.21.15.143:0/876367259 (peer_addr_for_me v2:172.21.15.143:0/0) 2023-11-26T07:23:14.962 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:14.959+0000 7fa6777fe640 1 --2- 172.21.15.143:0/876367259 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fa6780b2db0 0x7fa6781f8a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:23:14.962 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:14.959+0000 7fa677fff640 1 --2- 172.21.15.143:0/876367259 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fa6781596d0 0x7fa67814e4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:23:14.962 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:14.959+0000 7fa676ffd640 1 -- 172.21.15.143:0/876367259 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fa6781596d0 msgr2=0x7fa67814e4c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:23:14.962 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:14.959+0000 7fa676ffd640 1 --2- 172.21.15.143:0/876367259 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fa6781596d0 0x7fa67814e4c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:23:14.962 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:14.959+0000 7fa676ffd640 1 -- 172.21.15.143:0/876367259 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fa6780b2db0 msgr2=0x7fa6781f8a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:23:14.962 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:14.959+0000 7fa676ffd640 1 --2- 172.21.15.143:0/876367259 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fa6780b2db0 0x7fa6781f8a90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:23:14.962 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:14.959+0000 7fa676ffd640 1 -- 172.21.15.143:0/876367259 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa6680094d0 con 0x7fa6780ce680 2023-11-26T07:23:14.962 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:14.959+0000 7fa676ffd640 1 --2- 172.21.15.143:0/876367259 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fa6780ce680 0x7fa67815a640 secure :-1 s=READY pgs=1076 cs=0 l=1 rev1=1 crypto rx=0x7fa66c00e810 tx=0x7fa66c00ecd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:23:14.963 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:14.959+0000 7fa65ffff640 1 -- 172.21.15.143:0/876367259 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa66c00c8d0 con 0x7fa6780ce680 2023-11-26T07:23:14.964 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:14.959+0000 7fa65ffff640 1 -- 172.21.15.143:0/876367259 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa66c00ca30 con 0x7fa6780ce680 2023-11-26T07:23:14.964 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:14.959+0000 7fa65ffff640 1 -- 172.21.15.143:0/876367259 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa66c00cde0 con 0x7fa6780ce680 2023-11-26T07:23:14.964 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:14.959+0000 7fa67dddf640 1 -- 172.21.15.143:0/876367259 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa67815b6c0 con 0x7fa6780ce680 2023-11-26T07:23:14.964 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:14.959+0000 7fa67dddf640 1 -- 172.21.15.143:0/876367259 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa67815bc80 con 0x7fa6780ce680 2023-11-26T07:23:14.964 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:14.963+0000 7fa65ffff640 1 -- 172.21.15.143:0/876367259 <== mon.0 v2:172.21.15.143:3300/0 4 ==== mgrmap(e 15) v1 ==== 43285+0+0 (secure 0 0 0) 0x7fa66c013410 con 0x7fa6780ce680 2023-11-26T07:23:14.964 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:14.963+0000 7fa67dddf640 1 -- 172.21.15.143:0/876367259 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa678067680 con 0x7fa6780ce680 2023-11-26T07:23:14.964 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:14.963+0000 7fa65ffff640 1 --2- 172.21.15.143:0/876367259 >> [v2:172.21.15.188:6836/3401128553,v1:172.21.15.188:6837/3401128553] conn(0x7fa664035570 0x7fa664037a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:23:14.964 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:14.963+0000 7fa65ffff640 1 -- 172.21.15.143:0/876367259 <== mon.0 v2:172.21.15.143:3300/0 5 ==== osd_map(72..72 src has 1..72) v4 ==== 6268+0+0 (secure 0 0 0) 0x7fa66c048120 con 0x7fa6780ce680 2023-11-26T07:23:14.964 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:14.963+0000 7fa6777fe640 1 --2- 172.21.15.143:0/876367259 >> [v2:172.21.15.188:6836/3401128553,v1:172.21.15.188:6837/3401128553] conn(0x7fa664035570 0x7fa664037a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:23:14.965 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:14.963+0000 7fa6777fe640 1 --2- 172.21.15.143:0/876367259 >> [v2:172.21.15.188:6836/3401128553,v1:172.21.15.188:6837/3401128553] conn(0x7fa664035570 0x7fa664037a30 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7fa6680ad000 tx=0x7fa668008040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:23:14.971 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:14.971+0000 7fa65ffff640 1 -- 172.21.15.143:0/876367259 <== mon.0 v2:172.21.15.143:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172649 (secure 0 0 0) 0x7fa66c0136c0 con 0x7fa6780ce680 2023-11-26T07:23:15.153 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:15.151+0000 7fa67dddf640 1 -- 172.21.15.143:0/876367259 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fa6780b19e0 con 0x7fa6780ce680 2023-11-26T07:23:15.154 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:15.151+0000 7fa65ffff640 1 -- 172.21.15.143:0/876367259 <== mon.0 v2:172.21.15.143:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 43 v43) v1 ==== 94+0+4924 (secure 0 0 0) 0x7fa66c01b3b0 con 0x7fa6780ce680 2023-11-26T07:23:15.154 INFO:teuthology.orchestra.run.smithi143.stdout: 2023-11-26T07:23:15.154 INFO:teuthology.orchestra.run.smithi143.stdout:{"epoch":43,"default_fscid":5,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5927,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.143:6837/578144260","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6835","nonce":578144260},{"type":"v1","addr":"172.21.15.143:6837","nonce":578144260}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":38},{"gid":5930,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.188:6834/1048736873","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6832","nonce":1048736873},{"type":"v1","addr":"172.21.15.188:6834","nonce":1048736873}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":38},{"gid":5933,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.188:6835/2063768716","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6833","nonce":2063768716},{"type":"v1","addr":"172.21.15.188:6835","nonce":2063768716}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":38}],"filesystems":[{"mdsmap":{"epoch":42,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-11-26T07:22:37.220113+0000","modified":"2023-11-26T07:22:40.243052+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":6553},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6553":{"gid":6553,"name":"a","rank":0,"incarnation":39,"state":"up:active","state_seq":3,"addr":"172.21.15.143:6836/2590663349","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6834","nonce":2590663349},{"type":"v1","addr":"172.21.15.143:6836","nonce":2590663349}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[11],"metadata_pool":10,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":5}]} 2023-11-26T07:23:15.155 INFO:teuthology.orchestra.run.smithi143.stderr:dumped fsmap epoch 43 2023-11-26T07:23:15.156 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:15.155+0000 7fa67dddf640 1 -- 172.21.15.143:0/876367259 >> [v2:172.21.15.188:6836/3401128553,v1:172.21.15.188:6837/3401128553] conn(0x7fa664035570 msgr2=0x7fa664037a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:23:15.157 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:15.155+0000 7fa67dddf640 1 --2- 172.21.15.143:0/876367259 >> [v2:172.21.15.188:6836/3401128553,v1:172.21.15.188:6837/3401128553] conn(0x7fa664035570 0x7fa664037a30 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7fa6680ad000 tx=0x7fa668008040 comp rx=0 tx=0).stop 2023-11-26T07:23:15.157 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:15.155+0000 7fa67dddf640 1 -- 172.21.15.143:0/876367259 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fa6780ce680 msgr2=0x7fa67815a640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:23:15.157 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:15.155+0000 7fa67dddf640 1 --2- 172.21.15.143:0/876367259 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fa6780ce680 0x7fa67815a640 secure :-1 s=READY pgs=1076 cs=0 l=1 rev1=1 crypto rx=0x7fa66c00e810 tx=0x7fa66c00ecd0 comp rx=0 tx=0).stop 2023-11-26T07:23:15.157 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:15.155+0000 7fa67dddf640 1 -- 172.21.15.143:0/876367259 shutdown_connections 2023-11-26T07:23:15.157 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:15.155+0000 7fa67dddf640 1 --2- 172.21.15.143:0/876367259 >> [v2:172.21.15.188:6836/3401128553,v1:172.21.15.188:6837/3401128553] conn(0x7fa664035570 0x7fa664037a30 unknown :-1 s=CLOSED pgs=28 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:23:15.157 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:15.155+0000 7fa67dddf640 1 --2- 172.21.15.143:0/876367259 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fa6781596d0 0x7fa67814e4c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:23:15.157 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:15.155+0000 7fa67dddf640 1 --2- 172.21.15.143:0/876367259 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fa6780ce680 0x7fa67815a640 unknown :-1 s=CLOSED pgs=1076 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:23:15.157 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:15.155+0000 7fa67dddf640 1 --2- 172.21.15.143:0/876367259 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fa6780b2db0 0x7fa6781f8a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:23:15.157 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:15.155+0000 7fa67dddf640 1 -- 172.21.15.143:0/876367259 >> 172.21.15.143:0/876367259 conn(0x7fa67806a890 msgr2=0x7fa6780ae840 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:23:15.157 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:15.155+0000 7fa67dddf640 1 -- 172.21.15.143:0/876367259 shutdown_connections 2023-11-26T07:23:15.157 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:15.155+0000 7fa67dddf640 1 -- 172.21.15.143:0/876367259 wait complete. 2023-11-26T07:23:15.176 DEBUG:teuthology.orchestra.run.smithi143:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-11-26T07:23:15.192 INFO:tasks.ceph.mgr.y.smithi143.stderr:2023-11-26T07:23:15.191+0000 7f85d6276180 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2023-11-26T07:23:15.254 INFO:tasks.ceph.mgr.y.smithi143.stderr:2023-11-26T07:23:15.251+0000 7f85d6276180 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2023-11-26T07:23:15.317 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:15.315+0000 7f99cdae3640 1 Processor -- start 2023-11-26T07:23:15.317 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:15.315+0000 7f99cdae3640 1 -- start start 2023-11-26T07:23:15.319 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:15.315+0000 7f99cdae3640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f99c8143ac0 0x7f99c8070370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:23:15.319 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:15.315+0000 7f99cdae3640 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7f99c80708b0 0x7f99c80cf280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:23:15.319 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:15.315+0000 7f99cdae3640 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7f99c80cf7c0 0x7f99c80b1ef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:23:15.319 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:15.315+0000 7f99cdae3640 1 -- --> v2:172.21.15.143:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f99c8143ea0 con 0x7f99c80708b0 2023-11-26T07:23:15.320 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:15.315+0000 7f99cdae3640 1 -- --> v2:172.21.15.188:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f99c80cfbf0 con 0x7f99c80cf7c0 2023-11-26T07:23:15.320 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:15.315+0000 7f99cdae3640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f99c80cc730 con 0x7f99c8143ac0 2023-11-26T07:23:15.320 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:15.315+0000 7f99bffff640 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7f99c80708b0 0x7f99c80cf280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:23:15.320 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:15.315+0000 7f99bffff640 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7f99c80708b0 0x7f99c80cf280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:40806/0 (socket says 172.21.15.143:40806) 2023-11-26T07:23:15.320 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:15.315+0000 7f99bffff640 1 -- 172.21.15.143:0/2537265623 learned_addr learned my addr 172.21.15.143:0/2537265623 (peer_addr_for_me v2:172.21.15.143:0/0) 2023-11-26T07:23:15.320 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:15.319+0000 7f99c77fe640 1 --2- 172.21.15.143:0/2537265623 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f99c8143ac0 0x7f99c8070370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:23:15.320 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:15.319+0000 7f99c7fff640 1 --2- 172.21.15.143:0/2537265623 >> v2:172.21.15.188:3300/0 conn(0x7f99c80cf7c0 0x7f99c80b1ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:23:15.320 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:15.319+0000 7f99bffff640 1 -- 172.21.15.143:0/2537265623 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f99c8143ac0 msgr2=0x7f99c8070370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-26T07:23:15.320 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:15.319+0000 7f99bffff640 1 --2- 172.21.15.143:0/2537265623 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f99c8143ac0 0x7f99c8070370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:23:15.320 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:15.319+0000 7f99bffff640 1 -- 172.21.15.143:0/2537265623 >> v2:172.21.15.188:3300/0 conn(0x7f99c80cf7c0 msgr2=0x7f99c80b1ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-26T07:23:15.320 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:15.319+0000 7f99bffff640 1 --2- 172.21.15.143:0/2537265623 >> v2:172.21.15.188:3300/0 conn(0x7f99c80cf7c0 0x7f99c80b1ef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:23:15.320 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:15.319+0000 7f99bffff640 1 -- 172.21.15.143:0/2537265623 --> v2:172.21.15.143:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f99c8070ee0 con 0x7f99c80708b0 2023-11-26T07:23:15.322 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:15.319+0000 7f99bffff640 1 --2- 172.21.15.143:0/2537265623 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f99c80708b0 0x7f99c80cf280 secure :-1 s=READY pgs=1077 cs=0 l=1 rev1=1 crypto rx=0x7f99c80718d0 tx=0x7f99b001e910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=589de037e760d9af server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:23:15.322 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:15.319+0000 7f99c6ffd640 1 -- 172.21.15.143:0/2537265623 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f99b00a78b0 con 0x7f99c80708b0 2023-11-26T07:23:15.322 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:15.319+0000 7f99c6ffd640 1 -- 172.21.15.143:0/2537265623 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f99b00a7a10 con 0x7f99c80708b0 2023-11-26T07:23:15.322 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:15.319+0000 7f99c6ffd640 1 -- 172.21.15.143:0/2537265623 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f99b00a7da0 con 0x7f99c80708b0 2023-11-26T07:23:15.323 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:15.323+0000 7f99cdae3640 1 -- 172.21.15.143:0/2537265623 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f99c80708b0 msgr2=0x7f99c80cf280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:23:15.323 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:15.323+0000 7f99cdae3640 1 --2- 172.21.15.143:0/2537265623 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f99c80708b0 0x7f99c80cf280 secure :-1 s=READY pgs=1077 cs=0 l=1 rev1=1 crypto rx=0x7f99c80718d0 tx=0x7f99b001e910 comp rx=0 tx=0).stop 2023-11-26T07:23:15.323 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:15.323+0000 7f99cdae3640 1 -- 172.21.15.143:0/2537265623 shutdown_connections 2023-11-26T07:23:15.323 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:15.323+0000 7f99cdae3640 1 --2- 172.21.15.143:0/2537265623 >> v2:172.21.15.188:3300/0 conn(0x7f99c80cf7c0 0x7f99c80b1ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:23:15.324 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:15.323+0000 7f99cdae3640 1 --2- 172.21.15.143:0/2537265623 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f99c80708b0 0x7f99c80cf280 unknown :-1 s=CLOSED pgs=1077 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:23:15.324 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:15.323+0000 7f99cdae3640 1 --2- 172.21.15.143:0/2537265623 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f99c8143ac0 0x7f99c8070370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:23:15.324 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:15.323+0000 7f99cdae3640 1 -- 172.21.15.143:0/2537265623 >> 172.21.15.143:0/2537265623 conn(0x7f99c806a890 msgr2=0x7f99c806aca0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:23:15.324 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:15.323+0000 7f99cdae3640 1 -- 172.21.15.143:0/2537265623 shutdown_connections 2023-11-26T07:23:15.324 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:15.323+0000 7f99cdae3640 1 -- 172.21.15.143:0/2537265623 wait complete. 2023-11-26T07:23:15.324 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:15.323+0000 7f99cdae3640 1 Processor -- start 2023-11-26T07:23:15.324 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:15.323+0000 7f99cdae3640 1 -- start start 2023-11-26T07:23:15.327 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:15.323+0000 7f99cdae3640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f99c80708b0 0x7f99c80b15c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:23:15.327 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:15.323+0000 7f99cdae3640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f99c80cf7c0 0x7f99c8159880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:23:15.327 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:15.323+0000 7f99cdae3640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f99c8143ac0 0x7f99c80ada60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:23:15.327 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:15.323+0000 7f99cdae3640 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f99c80ad2f0 con 0x7f99c80cf7c0 2023-11-26T07:23:15.327 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:15.323+0000 7f99cdae3640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f99c8139680 con 0x7f99c8143ac0 2023-11-26T07:23:15.327 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:15.323+0000 7f99cdae3640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f99c8139960 con 0x7f99c80708b0 2023-11-26T07:23:15.328 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:15.323+0000 7f99bffff640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f99c80cf7c0 0x7f99c8159880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:23:15.328 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:15.323+0000 7f99bffff640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f99c80cf7c0 0x7f99c8159880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:40814/0 (socket says 172.21.15.143:40814) 2023-11-26T07:23:15.328 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:15.323+0000 7f99bffff640 1 -- 172.21.15.143:0/2110594605 learned_addr learned my addr 172.21.15.143:0/2110594605 (peer_addr_for_me v2:172.21.15.143:0/0) 2023-11-26T07:23:15.328 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:15.323+0000 7f99bffff640 1 -- 172.21.15.143:0/2110594605 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f99c80708b0 msgr2=0x7f99c80b15c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:23:15.328 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:15.323+0000 7f99bffff640 1 --2- 172.21.15.143:0/2110594605 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f99c80708b0 0x7f99c80b15c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:23:15.328 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:15.323+0000 7f99c7fff640 1 --2- 172.21.15.143:0/2110594605 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f99c8143ac0 0x7f99c80ada60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:23:15.328 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:15.323+0000 7f99bffff640 1 -- 172.21.15.143:0/2110594605 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f99c8143ac0 msgr2=0x7f99c80ada60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:23:15.328 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:15.323+0000 7f99bffff640 1 --2- 172.21.15.143:0/2110594605 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f99c8143ac0 0x7f99c80ada60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:23:15.328 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:15.323+0000 7f99bffff640 1 -- 172.21.15.143:0/2110594605 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f99b000b770 con 0x7f99c80cf7c0 2023-11-26T07:23:15.328 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:15.323+0000 7f99c7fff640 1 --2- 172.21.15.143:0/2110594605 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f99c8143ac0 0x7f99c80ada60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-26T07:23:15.328 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:15.323+0000 7f99bffff640 1 --2- 172.21.15.143:0/2110594605 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f99c80cf7c0 0x7f99c8159880 secure :-1 s=READY pgs=1078 cs=0 l=1 rev1=1 crypto rx=0x7f99b001cfc0 tx=0x7f99b001b6b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:23:15.328 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:15.327+0000 7f99c4ff9640 1 -- 172.21.15.143:0/2110594605 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f99b00af8d0 con 0x7f99c80cf7c0 2023-11-26T07:23:15.329 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:15.327+0000 7f99c4ff9640 1 -- 172.21.15.143:0/2110594605 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f99b00afa30 con 0x7f99c80cf7c0 2023-11-26T07:23:15.329 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:15.327+0000 7f99c4ff9640 1 -- 172.21.15.143:0/2110594605 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f99b00afdc0 con 0x7f99c80cf7c0 2023-11-26T07:23:15.329 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:15.327+0000 7f99cdae3640 1 -- 172.21.15.143:0/2110594605 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f99c80ad540 con 0x7f99c80cf7c0 2023-11-26T07:23:15.329 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:15.327+0000 7f99cdae3640 1 -- 172.21.15.143:0/2110594605 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f99c80ae170 con 0x7f99c80cf7c0 2023-11-26T07:23:15.330 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:15.327+0000 7f99cdae3640 1 -- 172.21.15.143:0/2110594605 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f99c8139b20 con 0x7f99c80cf7c0 2023-11-26T07:23:15.330 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:15.327+0000 7f99c4ff9640 1 -- 172.21.15.143:0/2110594605 <== mon.0 v2:172.21.15.143:3300/0 4 ==== mgrmap(e 15) v1 ==== 43285+0+0 (secure 0 0 0) 0x7f99b00c3450 con 0x7f99c80cf7c0 2023-11-26T07:23:15.330 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:15.327+0000 7f99c4ff9640 1 --2- 172.21.15.143:0/2110594605 >> [v2:172.21.15.188:6836/3401128553,v1:172.21.15.188:6837/3401128553] conn(0x7f99a8035570 0x7f99a8037a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:23:15.330 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:15.327+0000 7f99c4ff9640 1 -- 172.21.15.143:0/2110594605 <== mon.0 v2:172.21.15.143:3300/0 5 ==== osd_map(72..72 src has 1..72) v4 ==== 6268+0+0 (secure 0 0 0) 0x7f99b00e2830 con 0x7f99c80cf7c0 2023-11-26T07:23:15.330 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:15.327+0000 7f99c77fe640 1 --2- 172.21.15.143:0/2110594605 >> [v2:172.21.15.188:6836/3401128553,v1:172.21.15.188:6837/3401128553] conn(0x7f99a8035570 0x7f99a8037a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:23:15.330 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:15.327+0000 7f99c77fe640 1 --2- 172.21.15.143:0/2110594605 >> [v2:172.21.15.188:6836/3401128553,v1:172.21.15.188:6837/3401128553] conn(0x7f99a8035570 0x7f99a8037a30 secure :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0x7f99b8006aa0 tx=0x7f99b8006a30 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:23:15.333 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:15.331+0000 7f99c4ff9640 1 -- 172.21.15.143:0/2110594605 <== mon.0 v2:172.21.15.143:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172649 (secure 0 0 0) 0x7f99b00c3e60 con 0x7f99c80cf7c0 2023-11-26T07:23:15.511 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:15.511+0000 7f99cdae3640 1 -- 172.21.15.143:0/2110594605 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f99c8067680 con 0x7f99c80cf7c0 2023-11-26T07:23:15.512 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:15.511+0000 7f99c4ff9640 1 -- 172.21.15.143:0/2110594605 <== mon.0 v2:172.21.15.143:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 43 v43) v1 ==== 94+0+4924 (secure 0 0 0) 0x7f99b00c23b0 con 0x7f99c80cf7c0 2023-11-26T07:23:15.512 INFO:teuthology.orchestra.run.smithi143.stdout: 2023-11-26T07:23:15.513 INFO:teuthology.orchestra.run.smithi143.stdout:{"epoch":43,"default_fscid":5,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5927,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.143:6837/578144260","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6835","nonce":578144260},{"type":"v1","addr":"172.21.15.143:6837","nonce":578144260}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":38},{"gid":5930,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.188:6834/1048736873","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6832","nonce":1048736873},{"type":"v1","addr":"172.21.15.188:6834","nonce":1048736873}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":38},{"gid":5933,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.188:6835/2063768716","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6833","nonce":2063768716},{"type":"v1","addr":"172.21.15.188:6835","nonce":2063768716}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":38}],"filesystems":[{"mdsmap":{"epoch":42,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-11-26T07:22:37.220113+0000","modified":"2023-11-26T07:22:40.243052+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":6553},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6553":{"gid":6553,"name":"a","rank":0,"incarnation":39,"state":"up:active","state_seq":3,"addr":"172.21.15.143:6836/2590663349","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6834","nonce":2590663349},{"type":"v1","addr":"172.21.15.143:6836","nonce":2590663349}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[11],"metadata_pool":10,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":5}]} 2023-11-26T07:23:15.513 INFO:teuthology.orchestra.run.smithi143.stderr:dumped fsmap epoch 43 2023-11-26T07:23:15.514 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:15.511+0000 7f99cdae3640 1 -- 172.21.15.143:0/2110594605 >> [v2:172.21.15.188:6836/3401128553,v1:172.21.15.188:6837/3401128553] conn(0x7f99a8035570 msgr2=0x7f99a8037a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:23:15.514 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:15.511+0000 7f99cdae3640 1 --2- 172.21.15.143:0/2110594605 >> [v2:172.21.15.188:6836/3401128553,v1:172.21.15.188:6837/3401128553] conn(0x7f99a8035570 0x7f99a8037a30 secure :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0x7f99b8006aa0 tx=0x7f99b8006a30 comp rx=0 tx=0).stop 2023-11-26T07:23:15.515 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:15.511+0000 7f99cdae3640 1 -- 172.21.15.143:0/2110594605 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f99c80cf7c0 msgr2=0x7f99c8159880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:23:15.515 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:15.511+0000 7f99cdae3640 1 --2- 172.21.15.143:0/2110594605 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f99c80cf7c0 0x7f99c8159880 secure :-1 s=READY pgs=1078 cs=0 l=1 rev1=1 crypto rx=0x7f99b001cfc0 tx=0x7f99b001b6b0 comp rx=0 tx=0).stop 2023-11-26T07:23:15.515 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:15.511+0000 7f99cdae3640 1 -- 172.21.15.143:0/2110594605 shutdown_connections 2023-11-26T07:23:15.515 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:15.511+0000 7f99cdae3640 1 --2- 172.21.15.143:0/2110594605 >> [v2:172.21.15.188:6836/3401128553,v1:172.21.15.188:6837/3401128553] conn(0x7f99a8035570 0x7f99a8037a30 unknown :-1 s=CLOSED pgs=30 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:23:15.515 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:15.511+0000 7f99cdae3640 1 --2- 172.21.15.143:0/2110594605 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f99c8143ac0 0x7f99c80ada60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:23:15.515 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:15.511+0000 7f99cdae3640 1 --2- 172.21.15.143:0/2110594605 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f99c80cf7c0 0x7f99c8159880 unknown :-1 s=CLOSED pgs=1078 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:23:15.515 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:15.511+0000 7f99cdae3640 1 --2- 172.21.15.143:0/2110594605 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f99c80708b0 0x7f99c80b15c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:23:15.515 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:15.511+0000 7f99cdae3640 1 -- 172.21.15.143:0/2110594605 >> 172.21.15.143:0/2110594605 conn(0x7f99c806a890 msgr2=0x7f99c80cb0e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:23:15.515 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:15.511+0000 7f99cdae3640 1 -- 172.21.15.143:0/2110594605 shutdown_connections 2023-11-26T07:23:15.515 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:15.511+0000 7f99cdae3640 1 -- 172.21.15.143:0/2110594605 wait complete. 2023-11-26T07:23:15.533 DEBUG:teuthology.orchestra.run.smithi143:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-11-26T07:23:15.712 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:15.711+0000 7f0f6ed65640 1 Processor -- start 2023-11-26T07:23:15.712 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:15.711+0000 7f0f6ed65640 1 -- start start 2023-11-26T07:23:15.713 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:15.711+0000 7f0f6ed65640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f0f6815d6e0 0x7f0f680d0df0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:23:15.713 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:15.711+0000 7f0f6ed65640 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7f0f680b2160 0x7f0f680d14b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:23:15.713 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:15.711+0000 7f0f6ed65640 1 -- --> v1:172.21.15.143:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0f680cf0e0 con 0x7f0f680d19f0 2023-11-26T07:23:15.714 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:15.711+0000 7f0f6ed65640 1 -- --> v2:172.21.15.188:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0f6815dac0 con 0x7f0f680b2160 2023-11-26T07:23:15.714 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:15.711+0000 7f0f6ed65640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0f680d1e80 con 0x7f0f6815d6e0 2023-11-26T07:23:15.715 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:15.711+0000 7f0f6d302640 1 --1- >> v1:172.21.15.143:6789/0 conn(0x7f0f680d19f0 0x7f0f680cfbf0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.143:6789/0 says I am v1:172.21.15.143:36708/0 (socket says 172.21.15.143:36708) 2023-11-26T07:23:15.715 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:15.711+0000 7f0f6d302640 1 -- 172.21.15.143:0/2744090814 learned_addr learned my addr 172.21.15.143:0/2744090814 (peer_addr_for_me v1:172.21.15.143:0/0) 2023-11-26T07:23:15.715 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:15.711+0000 7f0f5ffff640 1 --2- 172.21.15.143:0/2744090814 >> v2:172.21.15.188:3300/0 conn(0x7f0f680b2160 0x7f0f680d14b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:23:15.715 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:15.711+0000 7f0f6cb01640 1 --2- 172.21.15.143:0/2744090814 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f0f6815d6e0 0x7f0f680d0df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:23:15.715 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:15.711+0000 7f0f5f7fe640 1 -- 172.21.15.143:0/2744090814 <== mon.0 v1:172.21.15.143:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1423281628 0 0) 0x7f0f680cf0e0 con 0x7f0f680d19f0 2023-11-26T07:23:15.715 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:15.711+0000 7f0f5f7fe640 1 -- 172.21.15.143:0/2744090814 --> v1:172.21.15.143:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f0f50003530 con 0x7f0f680d19f0 2023-11-26T07:23:15.715 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:15.711+0000 7f0f5ffff640 1 -- 172.21.15.143:0/2744090814 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f0f6815d6e0 msgr2=0x7f0f680d0df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-26T07:23:15.715 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:15.711+0000 7f0f5ffff640 1 --2- 172.21.15.143:0/2744090814 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f0f6815d6e0 0x7f0f680d0df0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:23:15.715 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:15.711+0000 7f0f5ffff640 1 -- 172.21.15.143:0/2744090814 >> v1:172.21.15.143:6789/0 conn(0x7f0f680d19f0 legacy=0x7f0f680cfbf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:23:15.715 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:15.711+0000 7f0f5ffff640 1 -- 172.21.15.143:0/2744090814 --> v2:172.21.15.188:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0f680ceb20 con 0x7f0f680b2160 2023-11-26T07:23:15.716 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:15.711+0000 7f0f6cb01640 1 --2- 172.21.15.143:0/2744090814 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f0f6815d6e0 0x7f0f680d0df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-26T07:23:15.717 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:15.715+0000 7f0f5ffff640 1 --2- 172.21.15.143:0/2744090814 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f0f680b2160 0x7f0f680d14b0 secure :-1 s=READY pgs=423 cs=0 l=1 rev1=1 crypto rx=0x7f0f580097c0 tx=0x7f0f5801c480 comp rx=0 tx=0).ready entity=mon.1 client_cookie=e751d9303fd01749 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:23:15.718 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:15.715+0000 7f0f5f7fe640 1 -- 172.21.15.143:0/2744090814 <== mon.1 v2:172.21.15.188:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0f580ab600 con 0x7f0f680b2160 2023-11-26T07:23:15.718 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:15.715+0000 7f0f5f7fe640 1 -- 172.21.15.143:0/2744090814 <== mon.1 v2:172.21.15.188:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0f580a7050 con 0x7f0f680b2160 2023-11-26T07:23:15.718 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:15.715+0000 7f0f5f7fe640 1 -- 172.21.15.143:0/2744090814 <== mon.1 v2:172.21.15.188:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0f580ab960 con 0x7f0f680b2160 2023-11-26T07:23:15.719 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:15.715+0000 7f0f6ed65640 1 -- 172.21.15.143:0/2744090814 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f0f680b2160 msgr2=0x7f0f680d14b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:23:15.719 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:15.715+0000 7f0f6ed65640 1 --2- 172.21.15.143:0/2744090814 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f0f680b2160 0x7f0f680d14b0 secure :-1 s=READY pgs=423 cs=0 l=1 rev1=1 crypto rx=0x7f0f580097c0 tx=0x7f0f5801c480 comp rx=0 tx=0).stop 2023-11-26T07:23:15.719 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:15.715+0000 7f0f6ed65640 1 -- 172.21.15.143:0/2744090814 shutdown_connections 2023-11-26T07:23:15.719 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:15.715+0000 7f0f6ed65640 1 --2- 172.21.15.143:0/2744090814 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f0f680b2160 0x7f0f680d14b0 unknown :-1 s=CLOSED pgs=423 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:23:15.719 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:15.715+0000 7f0f6ed65640 1 --2- 172.21.15.143:0/2744090814 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f0f6815d6e0 0x7f0f680d0df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:23:15.719 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:15.715+0000 7f0f6ed65640 1 -- 172.21.15.143:0/2744090814 >> 172.21.15.143:0/2744090814 conn(0x7f0f6809c550 msgr2=0x7f0f680d3220 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:23:15.719 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:15.715+0000 7f0f6ed65640 1 -- 172.21.15.143:0/2744090814 shutdown_connections 2023-11-26T07:23:15.719 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:15.715+0000 7f0f6ed65640 1 -- 172.21.15.143:0/2744090814 wait complete. 2023-11-26T07:23:15.720 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:15.719+0000 7f0f6ed65640 1 Processor -- start 2023-11-26T07:23:15.720 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:15.719+0000 7f0f6ed65640 1 -- start start 2023-11-26T07:23:15.723 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:15.719+0000 7f0f6ed65640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f0f680b2160 0x7f0f68156a40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:23:15.723 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:15.719+0000 7f0f6ed65640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f0f680d19f0 0x7f0f68154460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:23:15.723 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:15.719+0000 7f0f6ed65640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f0f6815d6e0 0x7f0f681549a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:23:15.723 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:15.719+0000 7f0f6ed65640 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0f68157020 con 0x7f0f680b2160 2023-11-26T07:23:15.723 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:15.719+0000 7f0f6ed65640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0f680b79f0 con 0x7f0f680d19f0 2023-11-26T07:23:15.723 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:15.719+0000 7f0f6ed65640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0f680b7d60 con 0x7f0f6815d6e0 2023-11-26T07:23:15.723 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:15.719+0000 7f0f6cb01640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f0f680b2160 0x7f0f68156a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:23:15.723 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:15.719+0000 7f0f6cb01640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f0f680b2160 0x7f0f68156a40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:40830/0 (socket says 172.21.15.143:40830) 2023-11-26T07:23:15.723 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:15.719+0000 7f0f6cb01640 1 -- 172.21.15.143:0/667483377 learned_addr learned my addr 172.21.15.143:0/667483377 (peer_addr_for_me v2:172.21.15.143:0/0) 2023-11-26T07:23:15.723 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:15.719+0000 7f0f6d302640 1 --2- 172.21.15.143:0/667483377 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f0f6815d6e0 0x7f0f681549a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:23:15.723 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:15.719+0000 7f0f5ffff640 1 --2- 172.21.15.143:0/667483377 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f0f680d19f0 0x7f0f68154460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:23:15.724 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:15.719+0000 7f0f6cb01640 1 -- 172.21.15.143:0/667483377 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f0f6815d6e0 msgr2=0x7f0f681549a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:23:15.724 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:15.719+0000 7f0f6cb01640 1 --2- 172.21.15.143:0/667483377 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f0f6815d6e0 0x7f0f681549a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:23:15.724 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:15.719+0000 7f0f6cb01640 1 -- 172.21.15.143:0/667483377 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f0f680d19f0 msgr2=0x7f0f68154460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:23:15.724 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:15.719+0000 7f0f6cb01640 1 --2- 172.21.15.143:0/667483377 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f0f680d19f0 0x7f0f68154460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:23:15.724 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:15.719+0000 7f0f6cb01640 1 -- 172.21.15.143:0/667483377 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0f64009420 con 0x7f0f680b2160 2023-11-26T07:23:15.724 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:15.719+0000 7f0f6cb01640 1 --2- 172.21.15.143:0/667483377 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f0f680b2160 0x7f0f68156a40 secure :-1 s=READY pgs=1080 cs=0 l=1 rev1=1 crypto rx=0x7f0f6400e980 tx=0x7f0f6400ee40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:23:15.724 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:15.723+0000 7f0f5d7fa640 1 -- 172.21.15.143:0/667483377 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0f6400ca00 con 0x7f0f680b2160 2023-11-26T07:23:15.724 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:15.723+0000 7f0f5d7fa640 1 -- 172.21.15.143:0/667483377 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0f6400cb60 con 0x7f0f680b2160 2023-11-26T07:23:15.724 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:15.723+0000 7f0f5d7fa640 1 -- 172.21.15.143:0/667483377 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0f64013470 con 0x7f0f680b2160 2023-11-26T07:23:15.724 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:15.723+0000 7f0f6ed65640 1 -- 172.21.15.143:0/667483377 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0f58009420 con 0x7f0f680b2160 2023-11-26T07:23:15.725 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:15.723+0000 7f0f6ed65640 1 -- 172.21.15.143:0/667483377 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0f680b8320 con 0x7f0f680b2160 2023-11-26T07:23:15.725 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:15.723+0000 7f0f6ed65640 1 -- 172.21.15.143:0/667483377 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0f680cdd80 con 0x7f0f680b2160 2023-11-26T07:23:15.725 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:15.723+0000 7f0f5d7fa640 1 -- 172.21.15.143:0/667483377 <== mon.0 v2:172.21.15.143:3300/0 4 ==== mgrmap(e 16) v1 ==== 85954+0+0 (secure 0 0 0) 0x7f0f640135d0 con 0x7f0f680b2160 2023-11-26T07:23:15.726 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:15.723+0000 7f0f5d7fa640 1 --2- 172.21.15.143:0/667483377 >> [v2:172.21.15.188:6836/3401128553,v1:172.21.15.188:6837/3401128553] conn(0x7f0f54066590 0x7f0f54068a50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:23:15.727 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:15.723+0000 7f0f5d7fa640 1 -- 172.21.15.143:0/667483377 <== mon.0 v2:172.21.15.143:3300/0 5 ==== osd_map(72..72 src has 1..72) v4 ==== 6268+0+0 (secure 0 0 0) 0x7f0f640844b0 con 0x7f0f680b2160 2023-11-26T07:23:15.727 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:15.727+0000 7f0f5ffff640 1 --2- 172.21.15.143:0/667483377 >> [v2:172.21.15.188:6836/3401128553,v1:172.21.15.188:6837/3401128553] conn(0x7f0f54066590 0x7f0f54068a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:23:15.727 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:15.727+0000 7f0f5ffff640 1 --2- 172.21.15.143:0/667483377 >> [v2:172.21.15.188:6836/3401128553,v1:172.21.15.188:6837/3401128553] conn(0x7f0f54066590 0x7f0f54068a50 secure :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0x7f0f580ad000 tx=0x7f0f580a6000 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:23:15.731 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:15.731+0000 7f0f5d7fa640 1 -- 172.21.15.143:0/667483377 <== mon.0 v2:172.21.15.143:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172649 (secure 0 0 0) 0x7f0f640515e0 con 0x7f0f680b2160 2023-11-26T07:23:15.879 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:15.879+0000 7f0f6ed65640 1 -- 172.21.15.143:0/667483377 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f0f68067680 con 0x7f0f680b2160 2023-11-26T07:23:15.881 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:15.879+0000 7f0f5d7fa640 1 -- 172.21.15.143:0/667483377 <== mon.0 v2:172.21.15.143:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v72) v1 ==== 74+0+17423 (secure 0 0 0) 0x7f0f64054c00 con 0x7f0f680b2160 2023-11-26T07:23:15.881 INFO:teuthology.orchestra.run.smithi143.stdout: 2023-11-26T07:23:15.881 INFO:teuthology.orchestra.run.smithi143.stdout:{"epoch":72,"fsid":"f5033060-2469-4a07-95fe-2b76f5404bc1","created":"2023-11-26T07:17:28.613663+0000","modified":"2023-11-26T07:23:11.261981+0000","last_up_change":"2023-11-26T07:17:41.157813+0000","last_in_change":"2023-11-26T07:17:31.181658+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":11,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-11-26T07:17:41.758597+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":10,"pool_name":"cephfs_metadata","create_time":"2023-11-26T07:22:35.314819+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"70","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":2.5,"score_stable":2.5,"optimal_score":1,"raw_score_acting":2.5,"raw_score_stable":2.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":11,"pool_name":"cephfs_data","create_time":"2023-11-26T07:22:36.219536+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"71","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"ebbee2c6-023e-44b9-9f1d-6f7bfd309730","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":69,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6817","nonce":639807369},{"type":"v1","addr":"172.21.15.143:6819","nonce":639807369}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6821","nonce":639807369},{"type":"v1","addr":"172.21.15.143:6823","nonce":639807369}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6830","nonce":639807369},{"type":"v1","addr":"172.21.15.143:6831","nonce":639807369}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6825","nonce":639807369},{"type":"v1","addr":"172.21.15.143:6828","nonce":639807369}]},"public_addr":"172.21.15.143:6819/639807369","cluster_addr":"172.21.15.143:6823/639807369","heartbeat_back_addr":"172.21.15.143:6831/639807369","heartbeat_front_addr":"172.21.15.143:6828/639807369","state":["exists","up"]},{"osd":1,"uuid":"fec4b8be-7122-49e9-9a19-5850a0700001","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":69,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6816","nonce":1144942530},{"type":"v1","addr":"172.21.15.143:6818","nonce":1144942530}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6820","nonce":1144942530},{"type":"v1","addr":"172.21.15.143:6822","nonce":1144942530}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6827","nonce":1144942530},{"type":"v1","addr":"172.21.15.143:6829","nonce":1144942530}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6824","nonce":1144942530},{"type":"v1","addr":"172.21.15.143:6826","nonce":1144942530}]},"public_addr":"172.21.15.143:6818/1144942530","cluster_addr":"172.21.15.143:6822/1144942530","heartbeat_back_addr":"172.21.15.143:6829/1144942530","heartbeat_front_addr":"172.21.15.143:6826/1144942530","state":["exists","up"]},{"osd":2,"uuid":"2888af5b-f933-46ca-8f95-6dd494ff43f6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":69,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6808","nonce":3909095186},{"type":"v1","addr":"172.21.15.143:6809","nonce":3909095186}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6810","nonce":3909095186},{"type":"v1","addr":"172.21.15.143:6811","nonce":3909095186}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6814","nonce":3909095186},{"type":"v1","addr":"172.21.15.143:6815","nonce":3909095186}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6812","nonce":3909095186},{"type":"v1","addr":"172.21.15.143:6813","nonce":3909095186}]},"public_addr":"172.21.15.143:6809/3909095186","cluster_addr":"172.21.15.143:6811/3909095186","heartbeat_back_addr":"172.21.15.143:6815/3909095186","heartbeat_front_addr":"172.21.15.143:6813/3909095186","state":["exists","up"]},{"osd":3,"uuid":"d1f52435-7bc1-4081-9550-9cd1ba55f549","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":69,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6800","nonce":3854210991},{"type":"v1","addr":"172.21.15.143:6801","nonce":3854210991}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6802","nonce":3854210991},{"type":"v1","addr":"172.21.15.143:6803","nonce":3854210991}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6806","nonce":3854210991},{"type":"v1","addr":"172.21.15.143:6807","nonce":3854210991}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6804","nonce":3854210991},{"type":"v1","addr":"172.21.15.143:6805","nonce":3854210991}]},"public_addr":"172.21.15.143:6801/3854210991","cluster_addr":"172.21.15.143:6803/3854210991","heartbeat_back_addr":"172.21.15.143:6807/3854210991","heartbeat_front_addr":"172.21.15.143:6805/3854210991","state":["exists","up"]},{"osd":4,"uuid":"e2ae8b6e-6e25-49b2-bcbf-aacce68f06a3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":69,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6800","nonce":3924606902},{"type":"v1","addr":"172.21.15.188:6801","nonce":3924606902}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6802","nonce":3924606902},{"type":"v1","addr":"172.21.15.188:6803","nonce":3924606902}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6806","nonce":3924606902},{"type":"v1","addr":"172.21.15.188:6807","nonce":3924606902}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6804","nonce":3924606902},{"type":"v1","addr":"172.21.15.188:6805","nonce":3924606902}]},"public_addr":"172.21.15.188:6801/3924606902","cluster_addr":"172.21.15.188:6803/3924606902","heartbeat_back_addr":"172.21.15.188:6807/3924606902","heartbeat_front_addr":"172.21.15.188:6805/3924606902","state":["exists","up"]},{"osd":5,"uuid":"3cec8a32-7698-4c36-8e89-e4ba11934f33","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":69,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6813","nonce":186841544},{"type":"v1","addr":"172.21.15.188:6815","nonce":186841544}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6817","nonce":186841544},{"type":"v1","addr":"172.21.15.188:6818","nonce":186841544}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6822","nonce":186841544},{"type":"v1","addr":"172.21.15.188:6823","nonce":186841544}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6820","nonce":186841544},{"type":"v1","addr":"172.21.15.188:6821","nonce":186841544}]},"public_addr":"172.21.15.188:6815/186841544","cluster_addr":"172.21.15.188:6818/186841544","heartbeat_back_addr":"172.21.15.188:6823/186841544","heartbeat_front_addr":"172.21.15.188:6821/186841544","state":["exists","up"]},{"osd":6,"uuid":"58a65d5d-e090-473a-81b4-ad38e85029f9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":69,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6808","nonce":2153499405},{"type":"v1","addr":"172.21.15.188:6809","nonce":2153499405}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6810","nonce":2153499405},{"type":"v1","addr":"172.21.15.188:6811","nonce":2153499405}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6816","nonce":2153499405},{"type":"v1","addr":"172.21.15.188:6819","nonce":2153499405}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6812","nonce":2153499405},{"type":"v1","addr":"172.21.15.188:6814","nonce":2153499405}]},"public_addr":"172.21.15.188:6809/2153499405","cluster_addr":"172.21.15.188:6811/2153499405","heartbeat_back_addr":"172.21.15.188:6819/2153499405","heartbeat_front_addr":"172.21.15.188:6814/2153499405","state":["exists","up"]},{"osd":7,"uuid":"ea5b7f70-3554-4cf8-ac30-f7c59b80678d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":69,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6824","nonce":2722096729},{"type":"v1","addr":"172.21.15.188:6825","nonce":2722096729}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6826","nonce":2722096729},{"type":"v1","addr":"172.21.15.188:6827","nonce":2722096729}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6830","nonce":2722096729},{"type":"v1","addr":"172.21.15.188:6831","nonce":2722096729}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6828","nonce":2722096729},{"type":"v1","addr":"172.21.15.188:6829","nonce":2722096729}]},"public_addr":"172.21.15.188:6825/2722096729","cluster_addr":"172.21.15.188:6827/2722096729","heartbeat_back_addr":"172.21.15.188:6831/2722096729","heartbeat_front_addr":"172.21.15.188:6829/2722096729","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-26T07:17:36.995032+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-26T07:17:36.932266+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-26T07:17:36.829304+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-26T07:17:36.804297+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-26T07:17:36.667892+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-26T07:17:36.964868+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-26T07:17:36.827332+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-26T07:17:36.893179+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.143:0/248180418":"2023-11-27T07:23:11.261937+0000","172.21.15.143:0/253926057":"2023-11-27T07:23:11.261937+0000","172.21.15.143:0/3516040872":"2023-11-27T07:23:11.261937+0000","172.21.15.143:0/963510909":"2023-11-27T07:23:11.261937+0000","172.21.15.143:6833/2150557544":"2023-11-27T07:23:11.261937+0000","172.21.15.143:6832/2150557544":"2023-11-27T07:23:11.261937+0000","172.21.15.143:0/2346893898":"2023-11-27T07:23:11.261937+0000","172.21.15.143:0/3751308865":"2023-11-27T07:23:11.261937+0000","172.21.15.143:0/3813484262":"2023-11-27T07:23:11.261937+0000","172.21.15.143:0/1844383386":"2023-11-27T07:23:11.261937+0000","172.21.15.143:0/2591285564":"2023-11-27T07:23:11.261937+0000","172.21.15.143:0/631097144":"2023-11-27T07:23:11.261937+0000","172.21.15.143:0/103646748":"2023-11-27T07:23:11.261937+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-11-26T07:23:15.883 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:15.883+0000 7f0f6ed65640 1 -- 172.21.15.143:0/667483377 >> [v2:172.21.15.188:6836/3401128553,v1:172.21.15.188:6837/3401128553] conn(0x7f0f54066590 msgr2=0x7f0f54068a50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:23:15.883 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:15.883+0000 7f0f6ed65640 1 --2- 172.21.15.143:0/667483377 >> [v2:172.21.15.188:6836/3401128553,v1:172.21.15.188:6837/3401128553] conn(0x7f0f54066590 0x7f0f54068a50 secure :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0x7f0f580ad000 tx=0x7f0f580a6000 comp rx=0 tx=0).stop 2023-11-26T07:23:15.883 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:15.883+0000 7f0f6ed65640 1 -- 172.21.15.143:0/667483377 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f0f680b2160 msgr2=0x7f0f68156a40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:23:15.884 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:15.883+0000 7f0f6ed65640 1 --2- 172.21.15.143:0/667483377 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f0f680b2160 0x7f0f68156a40 secure :-1 s=READY pgs=1080 cs=0 l=1 rev1=1 crypto rx=0x7f0f6400e980 tx=0x7f0f6400ee40 comp rx=0 tx=0).stop 2023-11-26T07:23:15.884 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:15.883+0000 7f0f6ed65640 1 -- 172.21.15.143:0/667483377 shutdown_connections 2023-11-26T07:23:15.884 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:15.883+0000 7f0f6ed65640 1 --2- 172.21.15.143:0/667483377 >> [v2:172.21.15.188:6836/3401128553,v1:172.21.15.188:6837/3401128553] conn(0x7f0f54066590 0x7f0f54068a50 unknown :-1 s=CLOSED pgs=31 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:23:15.884 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:15.883+0000 7f0f6ed65640 1 --2- 172.21.15.143:0/667483377 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f0f6815d6e0 0x7f0f681549a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:23:15.884 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:15.883+0000 7f0f6ed65640 1 --2- 172.21.15.143:0/667483377 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f0f680d19f0 0x7f0f68154460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:23:15.884 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:15.883+0000 7f0f6ed65640 1 --2- 172.21.15.143:0/667483377 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f0f680b2160 0x7f0f68156a40 unknown :-1 s=CLOSED pgs=1080 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:23:15.884 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:15.883+0000 7f0f6ed65640 1 -- 172.21.15.143:0/667483377 >> 172.21.15.143:0/667483377 conn(0x7f0f6809c550 msgr2=0x7f0f680b8a80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:23:15.884 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:15.883+0000 7f0f6ed65640 1 -- 172.21.15.143:0/667483377 shutdown_connections 2023-11-26T07:23:15.884 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:15.883+0000 7f0f6ed65640 1 -- 172.21.15.143:0/667483377 wait complete. 2023-11-26T07:23:15.902 DEBUG:teuthology.orchestra.run.smithi143:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs fail cephfs 2023-11-26T07:23:16.041 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:16.039+0000 7fbd923aa640 1 Processor -- start 2023-11-26T07:23:16.041 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:16.039+0000 7fbd923aa640 1 -- start start 2023-11-26T07:23:16.043 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:16.039+0000 7fbd923aa640 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7fbd8c0ce7c0 0x7fbd8c15f6b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:23:16.043 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:16.039+0000 7fbd923aa640 1 -- --> v1:172.21.15.143:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbd8c06c520 con 0x7fbd8c13e290 2023-11-26T07:23:16.043 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:16.039+0000 7fbd923aa640 1 -- --> v1:172.21.15.188:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbd8c06c800 con 0x7fbd8c0cf990 2023-11-26T07:23:16.043 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:16.039+0000 7fbd923aa640 1 -- --> v2:172.21.15.188:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fbd8c06cae0 con 0x7fbd8c0ce7c0 2023-11-26T07:23:16.043 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:16.043+0000 7fbd8bfff640 1 --1- >> v1:172.21.15.143:6789/0 conn(0x7fbd8c13e290 0x7fbd8c0cf280 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.143:6789/0 says I am v1:172.21.15.143:36720/0 (socket says 172.21.15.143:36720) 2023-11-26T07:23:16.043 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:16.043+0000 7fbd8bfff640 1 -- 172.21.15.143:0/406552247 learned_addr learned my addr 172.21.15.143:0/406552247 (peer_addr_for_me v1:172.21.15.143:0/0) 2023-11-26T07:23:16.044 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:16.043+0000 7fbd90947640 1 --2- 172.21.15.143:0/406552247 >> v2:172.21.15.188:3300/0 conn(0x7fbd8c0ce7c0 0x7fbd8c15f6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:23:16.044 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:16.043+0000 7fbd8affd640 1 -- 172.21.15.143:0/406552247 <== mon.0 v1:172.21.15.143:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4063366360 0 0) 0x7fbd8c06c520 con 0x7fbd8c13e290 2023-11-26T07:23:16.044 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:16.043+0000 7fbd8affd640 1 -- 172.21.15.143:0/406552247 --> v1:172.21.15.143:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fbd74003530 con 0x7fbd8c13e290 2023-11-26T07:23:16.044 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:16.043+0000 7fbd8affd640 1 -- 172.21.15.143:0/406552247 <== mon.0 v1:172.21.15.143:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1272119003 0 0) 0x7fbd74003530 con 0x7fbd8c13e290 2023-11-26T07:23:16.044 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:16.043+0000 7fbd90947640 1 -- 172.21.15.143:0/406552247 >> v1:172.21.15.188:6789/0 conn(0x7fbd8c0cf990 legacy=0x7fbd8c15efa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:23:16.044 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:16.043+0000 7fbd90947640 1 -- 172.21.15.143:0/406552247 >> v1:172.21.15.143:6789/0 conn(0x7fbd8c13e290 legacy=0x7fbd8c0cf280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:23:16.044 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:16.043+0000 7fbd90947640 1 -- 172.21.15.143:0/406552247 --> v2:172.21.15.188:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbd8c06cdc0 con 0x7fbd8c0ce7c0 2023-11-26T07:23:16.046 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:16.043+0000 7fbd90947640 1 --2- 172.21.15.143:0/406552247 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fbd8c0ce7c0 0x7fbd8c15f6b0 secure :-1 s=READY pgs=425 cs=0 l=1 rev1=1 crypto rx=0x7fbd84009d90 tx=0x7fbd8401c7b0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=92de630bc0fa27aa server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:23:16.047 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:16.047+0000 7fbd8affd640 1 -- 172.21.15.143:0/406552247 <== mon.1 v2:172.21.15.188:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbd7c001680 con 0x7fbd8c0ce7c0 2023-11-26T07:23:16.047 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:16.047+0000 7fbd8affd640 1 -- 172.21.15.143:0/406552247 <== mon.1 v2:172.21.15.188:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbd840a5610 con 0x7fbd8c0ce7c0 2023-11-26T07:23:16.047 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:16.047+0000 7fbd8affd640 1 -- 172.21.15.143:0/406552247 <== mon.1 v2:172.21.15.188:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbd840a5980 con 0x7fbd8c0ce7c0 2023-11-26T07:23:16.048 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:16.047+0000 7fbd923aa640 1 -- 172.21.15.143:0/406552247 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fbd8c0ce7c0 msgr2=0x7fbd8c15f6b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:23:16.048 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:16.047+0000 7fbd923aa640 1 --2- 172.21.15.143:0/406552247 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fbd8c0ce7c0 0x7fbd8c15f6b0 secure :-1 s=READY pgs=425 cs=0 l=1 rev1=1 crypto rx=0x7fbd84009d90 tx=0x7fbd8401c7b0 comp rx=0 tx=0).stop 2023-11-26T07:23:16.048 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:16.047+0000 7fbd923aa640 1 -- 172.21.15.143:0/406552247 shutdown_connections 2023-11-26T07:23:16.048 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:16.047+0000 7fbd923aa640 1 --2- 172.21.15.143:0/406552247 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fbd8c0ce7c0 0x7fbd8c15f6b0 unknown :-1 s=CLOSED pgs=425 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:23:16.048 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:16.047+0000 7fbd923aa640 1 -- 172.21.15.143:0/406552247 >> 172.21.15.143:0/406552247 conn(0x7fbd8c06a890 msgr2=0x7fbd8c06aca0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:23:16.048 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:16.047+0000 7fbd923aa640 1 -- 172.21.15.143:0/406552247 shutdown_connections 2023-11-26T07:23:16.048 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:16.047+0000 7fbd923aa640 1 -- 172.21.15.143:0/406552247 wait complete. 2023-11-26T07:23:16.049 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:16.047+0000 7fbd923aa640 1 Processor -- start 2023-11-26T07:23:16.049 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:16.047+0000 7fbd923aa640 1 -- start start 2023-11-26T07:23:16.051 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:16.047+0000 7fbd923aa640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fbd8c0ce7c0 0x7fbd8c1511f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:23:16.051 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:16.047+0000 7fbd923aa640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fbd8c0cf990 0x7fbd8c1f31e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:23:16.051 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:16.047+0000 7fbd923aa640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fbd8c13e290 0x7fbd8c0b0ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:23:16.051 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:16.047+0000 7fbd923aa640 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbd8c1f6580 con 0x7fbd8c13e290 2023-11-26T07:23:16.051 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:16.047+0000 7fbd923aa640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbd8c151730 con 0x7fbd8c0cf990 2023-11-26T07:23:16.051 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:16.047+0000 7fbd923aa640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbd8c0b1e50 con 0x7fbd8c0ce7c0 2023-11-26T07:23:16.051 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:16.047+0000 7fbd90947640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fbd8c13e290 0x7fbd8c0b0ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:23:16.052 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:16.047+0000 7fbd90947640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fbd8c13e290 0x7fbd8c0b0ed0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:40840/0 (socket says 172.21.15.143:40840) 2023-11-26T07:23:16.052 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:16.047+0000 7fbd90947640 1 -- 172.21.15.143:0/107197664 learned_addr learned my addr 172.21.15.143:0/107197664 (peer_addr_for_me v2:172.21.15.143:0/0) 2023-11-26T07:23:16.052 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:16.047+0000 7fbd8b7fe640 1 --2- 172.21.15.143:0/107197664 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fbd8c0cf990 0x7fbd8c1f31e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:23:16.052 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:16.047+0000 7fbd8bfff640 1 --2- 172.21.15.143:0/107197664 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fbd8c0ce7c0 0x7fbd8c1511f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:23:16.052 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:16.051+0000 7fbd90947640 1 -- 172.21.15.143:0/107197664 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fbd8c0ce7c0 msgr2=0x7fbd8c1511f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:23:16.052 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:16.051+0000 7fbd90947640 1 --2- 172.21.15.143:0/107197664 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fbd8c0ce7c0 0x7fbd8c1511f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:23:16.052 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:16.051+0000 7fbd90947640 1 -- 172.21.15.143:0/107197664 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fbd8c0cf990 msgr2=0x7fbd8c1f31e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:23:16.052 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:16.051+0000 7fbd90947640 1 --2- 172.21.15.143:0/107197664 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fbd8c0cf990 0x7fbd8c1f31e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:23:16.052 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:16.051+0000 7fbd90947640 1 -- 172.21.15.143:0/107197664 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbd84009a70 con 0x7fbd8c13e290 2023-11-26T07:23:16.052 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:16.051+0000 7fbd8bfff640 1 --2- 172.21.15.143:0/107197664 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fbd8c0ce7c0 0x7fbd8c1511f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-26T07:23:16.052 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:16.051+0000 7fbd90947640 1 --2- 172.21.15.143:0/107197664 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fbd8c13e290 0x7fbd8c0b0ed0 secure :-1 s=READY pgs=1082 cs=0 l=1 rev1=1 crypto rx=0x7fbd84009d90 tx=0x7fbd84019550 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:23:16.053 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:16.051+0000 7fbd88ff9640 1 -- 172.21.15.143:0/107197664 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbd840ad650 con 0x7fbd8c13e290 2023-11-26T07:23:16.053 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:16.051+0000 7fbd88ff9640 1 -- 172.21.15.143:0/107197664 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbd840ad7b0 con 0x7fbd8c13e290 2023-11-26T07:23:16.053 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:16.051+0000 7fbd88ff9640 1 -- 172.21.15.143:0/107197664 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbd840adb20 con 0x7fbd8c13e290 2023-11-26T07:23:16.053 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:16.051+0000 7fbd923aa640 1 -- 172.21.15.143:0/107197664 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbd8c0b2130 con 0x7fbd8c13e290 2023-11-26T07:23:16.054 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:16.051+0000 7fbd923aa640 1 -- 172.21.15.143:0/107197664 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbd8c0b26f0 con 0x7fbd8c13e290 2023-11-26T07:23:16.054 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:16.051+0000 7fbd923aa640 1 -- 172.21.15.143:0/107197664 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbd8c067680 con 0x7fbd8c13e290 2023-11-26T07:23:16.054 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:16.051+0000 7fbd88ff9640 1 -- 172.21.15.143:0/107197664 <== mon.0 v2:172.21.15.143:3300/0 4 ==== mgrmap(e 16) v1 ==== 85954+0+0 (secure 0 0 0) 0x7fbd840adc80 con 0x7fbd8c13e290 2023-11-26T07:23:16.054 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:16.051+0000 7fbd88ff9640 1 --2- 172.21.15.143:0/107197664 >> [v2:172.21.15.188:6836/3401128553,v1:172.21.15.188:6837/3401128553] conn(0x7fbd780665e0 0x7fbd78068aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:23:16.054 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:16.051+0000 7fbd88ff9640 1 -- 172.21.15.143:0/107197664 <== mon.0 v2:172.21.15.143:3300/0 5 ==== osd_map(72..72 src has 1..72) v4 ==== 6268+0+0 (secure 0 0 0) 0x7fbd8411f390 con 0x7fbd8c13e290 2023-11-26T07:23:16.055 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:16.055+0000 7fbd8bfff640 1 --2- 172.21.15.143:0/107197664 >> [v2:172.21.15.188:6836/3401128553,v1:172.21.15.188:6837/3401128553] conn(0x7fbd780665e0 0x7fbd78068aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:23:16.055 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:16.055+0000 7fbd8bfff640 1 --2- 172.21.15.143:0/107197664 >> [v2:172.21.15.188:6836/3401128553,v1:172.21.15.188:6837/3401128553] conn(0x7fbd780665e0 0x7fbd78068aa0 secure :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0x7fbd7c000fc0 tx=0x7fbd7c009040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:23:16.058 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:16.055+0000 7fbd88ff9640 1 -- 172.21.15.143:0/107197664 <== mon.0 v2:172.21.15.143:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172649 (secure 0 0 0) 0x7fbd840ec710 con 0x7fbd8c13e290 2023-11-26T07:23:16.237 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:16.235+0000 7fbd923aa640 1 -- 172.21.15.143:0/107197664 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs"} v 0) v1 -- 0x7fbd8c06df50 con 0x7fbd8c13e290 2023-11-26T07:23:16.238 INFO:tasks.ceph.mon.a.smithi143.stderr:2023-11-26T07:23:16.235+0000 7f6946f73640 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2023-11-26T07:23:16.240 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:16.239+0000 7fbd88ff9640 1 -- 172.21.15.143:0/107197664 <== mon.0 v2:172.21.15.143:3300/0 7 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs"}]=0 cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v44) v1 ==== 161+0+0 (secure 0 0 0) 0x7fbd840efd30 con 0x7fbd8c13e290 2023-11-26T07:23:16.240 INFO:teuthology.orchestra.run.smithi143.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2023-11-26T07:23:16.242 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:16.239+0000 7fbd923aa640 1 -- 172.21.15.143:0/107197664 >> [v2:172.21.15.188:6836/3401128553,v1:172.21.15.188:6837/3401128553] conn(0x7fbd780665e0 msgr2=0x7fbd78068aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:23:16.243 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:16.239+0000 7fbd923aa640 1 --2- 172.21.15.143:0/107197664 >> [v2:172.21.15.188:6836/3401128553,v1:172.21.15.188:6837/3401128553] conn(0x7fbd780665e0 0x7fbd78068aa0 secure :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0x7fbd7c000fc0 tx=0x7fbd7c009040 comp rx=0 tx=0).stop 2023-11-26T07:23:16.243 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:16.239+0000 7fbd923aa640 1 -- 172.21.15.143:0/107197664 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fbd8c13e290 msgr2=0x7fbd8c0b0ed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:23:16.243 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:16.239+0000 7fbd923aa640 1 --2- 172.21.15.143:0/107197664 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fbd8c13e290 0x7fbd8c0b0ed0 secure :-1 s=READY pgs=1082 cs=0 l=1 rev1=1 crypto rx=0x7fbd84009d90 tx=0x7fbd84019550 comp rx=0 tx=0).stop 2023-11-26T07:23:16.243 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:16.243+0000 7fbd923aa640 1 -- 172.21.15.143:0/107197664 shutdown_connections 2023-11-26T07:23:16.243 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:16.243+0000 7fbd923aa640 1 --2- 172.21.15.143:0/107197664 >> [v2:172.21.15.188:6836/3401128553,v1:172.21.15.188:6837/3401128553] conn(0x7fbd780665e0 0x7fbd78068aa0 unknown :-1 s=CLOSED pgs=32 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:23:16.244 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:16.243+0000 7fbd923aa640 1 --2- 172.21.15.143:0/107197664 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fbd8c13e290 0x7fbd8c0b0ed0 unknown :-1 s=CLOSED pgs=1082 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:23:16.244 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:16.243+0000 7fbd923aa640 1 --2- 172.21.15.143:0/107197664 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fbd8c0cf990 0x7fbd8c1f31e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:23:16.244 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:16.243+0000 7fbd923aa640 1 --2- 172.21.15.143:0/107197664 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fbd8c0ce7c0 0x7fbd8c1511f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:23:16.244 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:16.243+0000 7fbd923aa640 1 -- 172.21.15.143:0/107197664 >> 172.21.15.143:0/107197664 conn(0x7fbd8c06a890 msgr2=0x7fbd8c06ac70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:23:16.244 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:16.243+0000 7fbd923aa640 1 -- 172.21.15.143:0/107197664 shutdown_connections 2023-11-26T07:23:16.244 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:16.243+0000 7fbd923aa640 1 -- 172.21.15.143:0/107197664 wait complete. 2023-11-26T07:23:16.255 INFO:tasks.ceph.mds.a.smithi143.stderr: -6517> 2023-11-26T07:22:42.832+0000 7fa827c91640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2023-11-26T07:23:16.263 DEBUG:teuthology.orchestra.run.smithi143:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs rm cephfs --yes-i-really-mean-it 2023-11-26T07:23:16.305 INFO:tasks.ceph.mds.a.smithi143.stdout:starting mds.a at 2023-11-26T07:23:16.405 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:16.403+0000 7ffacc9c4640 1 Processor -- start 2023-11-26T07:23:16.406 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:16.403+0000 7ffacc9c4640 1 -- start start 2023-11-26T07:23:16.407 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:16.407+0000 7ffacc9c4640 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7ffac815d6e0 0x7ffac80d0df0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:23:16.407 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:16.407+0000 7ffacc9c4640 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7ffac80b2160 0x7ffac80d14b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:23:16.407 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:16.407+0000 7ffacc9c4640 1 -- --> v1:172.21.15.188:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ffac80cf0e0 con 0x7ffac80d19f0 2023-11-26T07:23:16.407 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:16.407+0000 7ffacc9c4640 1 -- --> v2:172.21.15.143:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ffac815dac0 con 0x7ffac815d6e0 2023-11-26T07:23:16.407 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:16.407+0000 7ffacc9c4640 1 -- --> v2:172.21.15.188:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ffac80d1e80 con 0x7ffac80b2160 2023-11-26T07:23:16.407 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:16.407+0000 7ffac659c640 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7ffac815d6e0 0x7ffac80d0df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:23:16.408 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:16.407+0000 7ffac659c640 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7ffac815d6e0 0x7ffac80d0df0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:40860/0 (socket says 172.21.15.143:40860) 2023-11-26T07:23:16.408 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:16.407+0000 7ffac659c640 1 -- 172.21.15.143:0/260913759 learned_addr learned my addr 172.21.15.143:0/260913759 (peer_addr_for_me v2:172.21.15.143:0/0) 2023-11-26T07:23:16.408 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:16.407+0000 7ffabffff640 1 --2- 172.21.15.143:0/260913759 >> v2:172.21.15.188:3300/0 conn(0x7ffac80b2160 0x7ffac80d14b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:23:16.408 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:16.407+0000 7ffac659c640 1 -- 172.21.15.143:0/260913759 >> v2:172.21.15.188:3300/0 conn(0x7ffac80b2160 msgr2=0x7ffac80d14b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-26T07:23:16.408 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:16.407+0000 7ffac659c640 1 --2- 172.21.15.143:0/260913759 >> v2:172.21.15.188:3300/0 conn(0x7ffac80b2160 0x7ffac80d14b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:23:16.408 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:16.407+0000 7ffac659c640 1 -- 172.21.15.143:0/260913759 >> v1:172.21.15.188:6789/0 conn(0x7ffac80d19f0 legacy=0x7ffac80cfbf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:23:16.409 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:16.407+0000 7ffac659c640 1 -- 172.21.15.143:0/260913759 --> v2:172.21.15.143:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ffac80ceb20 con 0x7ffac815d6e0 2023-11-26T07:23:16.411 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:16.407+0000 7ffac659c640 1 --2- 172.21.15.143:0/260913759 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7ffac815d6e0 0x7ffac80d0df0 secure :-1 s=READY pgs=1086 cs=0 l=1 rev1=1 crypto rx=0x7ffac80ca2a0 tx=0x7ffab801c500 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c8ffdbca0771450e server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:23:16.411 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:16.411+0000 7ffac5d9b640 1 -- 172.21.15.143:0/260913759 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffab80ab5f0 con 0x7ffac815d6e0 2023-11-26T07:23:16.411 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:16.411+0000 7ffac5d9b640 1 -- 172.21.15.143:0/260913759 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ffab80ab750 con 0x7ffac815d6e0 2023-11-26T07:23:16.411 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:16.411+0000 7ffac5d9b640 1 -- 172.21.15.143:0/260913759 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffab80abae0 con 0x7ffac815d6e0 2023-11-26T07:23:16.412 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:16.411+0000 7ffacc9c4640 1 -- 172.21.15.143:0/260913759 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7ffac815d6e0 msgr2=0x7ffac80d0df0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:23:16.412 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:16.411+0000 7ffacc9c4640 1 --2- 172.21.15.143:0/260913759 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7ffac815d6e0 0x7ffac80d0df0 secure :-1 s=READY pgs=1086 cs=0 l=1 rev1=1 crypto rx=0x7ffac80ca2a0 tx=0x7ffab801c500 comp rx=0 tx=0).stop 2023-11-26T07:23:16.412 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:16.411+0000 7ffacc9c4640 1 -- 172.21.15.143:0/260913759 shutdown_connections 2023-11-26T07:23:16.412 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:16.411+0000 7ffacc9c4640 1 --2- 172.21.15.143:0/260913759 >> v2:172.21.15.188:3300/0 conn(0x7ffac80b2160 0x7ffac80d14b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:23:16.412 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:16.411+0000 7ffacc9c4640 1 --2- 172.21.15.143:0/260913759 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7ffac815d6e0 0x7ffac80d0df0 unknown :-1 s=CLOSED pgs=1086 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:23:16.412 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:16.411+0000 7ffacc9c4640 1 -- 172.21.15.143:0/260913759 >> 172.21.15.143:0/260913759 conn(0x7ffac809c550 msgr2=0x7ffac80d3220 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:23:16.412 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:16.411+0000 7ffacc9c4640 1 -- 172.21.15.143:0/260913759 shutdown_connections 2023-11-26T07:23:16.412 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:16.411+0000 7ffacc9c4640 1 -- 172.21.15.143:0/260913759 wait complete. 2023-11-26T07:23:16.413 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:16.411+0000 7ffacc9c4640 1 Processor -- start 2023-11-26T07:23:16.413 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:16.411+0000 7ffacc9c4640 1 -- start start 2023-11-26T07:23:16.415 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:16.411+0000 7ffacc9c4640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7ffac80b2160 0x7ffac80c95b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:23:16.415 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:16.411+0000 7ffacc9c4640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7ffac80d19f0 0x7ffac814c390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:23:16.415 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:16.411+0000 7ffacc9c4640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7ffac815d6e0 0x7ffac815bbd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:23:16.415 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:16.411+0000 7ffacc9c4640 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffac8160400 con 0x7ffac815d6e0 2023-11-26T07:23:16.415 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:16.411+0000 7ffacc9c4640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffac814bbb0 con 0x7ffac80d19f0 2023-11-26T07:23:16.416 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:16.411+0000 7ffacc9c4640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ffac81606a0 con 0x7ffac80b2160 2023-11-26T07:23:16.416 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:16.411+0000 7ffac6d9d640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7ffac815d6e0 0x7ffac815bbd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:23:16.416 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:16.411+0000 7ffac659c640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7ffac80b2160 0x7ffac80c95b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:23:16.416 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:16.411+0000 7ffac6d9d640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7ffac815d6e0 0x7ffac815bbd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:40868/0 (socket says 172.21.15.143:40868) 2023-11-26T07:23:16.416 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:16.415+0000 7ffac6d9d640 1 -- 172.21.15.143:0/3056087430 learned_addr learned my addr 172.21.15.143:0/3056087430 (peer_addr_for_me v2:172.21.15.143:0/0) 2023-11-26T07:23:16.416 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:16.415+0000 7ffabffff640 1 --2- 172.21.15.143:0/3056087430 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7ffac80d19f0 0x7ffac814c390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:23:16.416 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:16.415+0000 7ffac6d9d640 1 -- 172.21.15.143:0/3056087430 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7ffac80b2160 msgr2=0x7ffac80c95b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:23:16.416 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:16.415+0000 7ffac6d9d640 1 --2- 172.21.15.143:0/3056087430 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7ffac80b2160 0x7ffac80c95b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:23:16.416 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:16.415+0000 7ffac6d9d640 1 -- 172.21.15.143:0/3056087430 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7ffac80d19f0 msgr2=0x7ffac814c390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:23:16.416 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:16.415+0000 7ffac6d9d640 1 --2- 172.21.15.143:0/3056087430 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7ffac80d19f0 0x7ffac814c390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:23:16.416 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:16.415+0000 7ffac6d9d640 1 -- 172.21.15.143:0/3056087430 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ffab80094d0 con 0x7ffac815d6e0 2023-11-26T07:23:16.416 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:16.415+0000 7ffac6d9d640 1 --2- 172.21.15.143:0/3056087430 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7ffac815d6e0 0x7ffac815bbd0 secure :-1 s=READY pgs=1087 cs=0 l=1 rev1=1 crypto rx=0x7ffac000c770 tx=0x7ffac000cc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:23:16.417 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:16.415+0000 7ffabeffd640 1 -- 172.21.15.143:0/3056087430 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffac0003b80 con 0x7ffac815d6e0 2023-11-26T07:23:16.417 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:16.415+0000 7ffabeffd640 1 -- 172.21.15.143:0/3056087430 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ffac0003ce0 con 0x7ffac815d6e0 2023-11-26T07:23:16.418 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:16.415+0000 7ffabeffd640 1 -- 172.21.15.143:0/3056087430 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffac000aa90 con 0x7ffac815d6e0 2023-11-26T07:23:16.418 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:16.415+0000 7ffacc9c4640 1 -- 172.21.15.143:0/3056087430 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ffac8160980 con 0x7ffac815d6e0 2023-11-26T07:23:16.418 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:16.415+0000 7ffacc9c4640 1 -- 172.21.15.143:0/3056087430 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ffac80c9af0 con 0x7ffac815d6e0 2023-11-26T07:23:16.418 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:16.415+0000 7ffacc9c4640 1 -- 172.21.15.143:0/3056087430 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ffac8067680 con 0x7ffac815d6e0 2023-11-26T07:23:16.418 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:16.415+0000 7ffabeffd640 1 -- 172.21.15.143:0/3056087430 <== mon.0 v2:172.21.15.143:3300/0 4 ==== mgrmap(e 16) v1 ==== 85954+0+0 (secure 0 0 0) 0x7ffac000abf0 con 0x7ffac815d6e0 2023-11-26T07:23:16.418 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:16.415+0000 7ffabeffd640 1 --2- 172.21.15.143:0/3056087430 >> [v2:172.21.15.188:6836/3401128553,v1:172.21.15.188:6837/3401128553] conn(0x7ffaa8066590 0x7ffaa8068a50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:23:16.419 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:16.415+0000 7ffabeffd640 1 -- 172.21.15.143:0/3056087430 <== mon.0 v2:172.21.15.143:3300/0 5 ==== osd_map(73..73 src has 1..73) v4 ==== 6354+0+0 (secure 0 0 0) 0x7ffac0083f80 con 0x7ffac815d6e0 2023-11-26T07:23:16.419 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:16.419+0000 7ffac659c640 1 --2- 172.21.15.143:0/3056087430 >> [v2:172.21.15.188:6836/3401128553,v1:172.21.15.188:6837/3401128553] conn(0x7ffaa8066590 0x7ffaa8068a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:23:16.419 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:16.419+0000 7ffac659c640 1 --2- 172.21.15.143:0/3056087430 >> [v2:172.21.15.188:6836/3401128553,v1:172.21.15.188:6837/3401128553] conn(0x7ffaa8066590 0x7ffaa8068a50 secure :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0x7ffab80af000 tx=0x7ffab80b6040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:23:16.422 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:16.419+0000 7ffabeffd640 1 -- 172.21.15.143:0/3056087430 <== mon.0 v2:172.21.15.143:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172649 (secure 0 0 0) 0x7ffac00b5280 con 0x7ffac815d6e0 2023-11-26T07:23:16.604 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:16.603+0000 7ffacc9c4640 1 -- 172.21.15.143:0/3056087430 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true} v 0) v1 -- 0x7ffac814c040 con 0x7ffac815d6e0 2023-11-26T07:23:17.242 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:17.239+0000 7ffabeffd640 1 -- 172.21.15.143:0/3056087430 <== mon.0 v2:172.21.15.143:3300/0 7 ==== mon_command_ack([{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]=0 v45) v1 ==== 104+0+0 (secure 0 0 0) 0x7ffac0051110 con 0x7ffac815d6e0 2023-11-26T07:23:17.245 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:17.243+0000 7ffacc9c4640 1 -- 172.21.15.143:0/3056087430 >> [v2:172.21.15.188:6836/3401128553,v1:172.21.15.188:6837/3401128553] conn(0x7ffaa8066590 msgr2=0x7ffaa8068a50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:23:17.245 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:17.243+0000 7ffacc9c4640 1 --2- 172.21.15.143:0/3056087430 >> [v2:172.21.15.188:6836/3401128553,v1:172.21.15.188:6837/3401128553] conn(0x7ffaa8066590 0x7ffaa8068a50 secure :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0x7ffab80af000 tx=0x7ffab80b6040 comp rx=0 tx=0).stop 2023-11-26T07:23:17.245 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:17.243+0000 7ffacc9c4640 1 -- 172.21.15.143:0/3056087430 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7ffac815d6e0 msgr2=0x7ffac815bbd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:23:17.245 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:17.243+0000 7ffacc9c4640 1 --2- 172.21.15.143:0/3056087430 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7ffac815d6e0 0x7ffac815bbd0 secure :-1 s=READY pgs=1087 cs=0 l=1 rev1=1 crypto rx=0x7ffac000c770 tx=0x7ffac000cc30 comp rx=0 tx=0).stop 2023-11-26T07:23:17.245 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:17.243+0000 7ffacc9c4640 1 -- 172.21.15.143:0/3056087430 shutdown_connections 2023-11-26T07:23:17.245 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:17.243+0000 7ffacc9c4640 1 --2- 172.21.15.143:0/3056087430 >> [v2:172.21.15.188:6836/3401128553,v1:172.21.15.188:6837/3401128553] conn(0x7ffaa8066590 0x7ffaa8068a50 unknown :-1 s=CLOSED pgs=34 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:23:17.245 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:17.243+0000 7ffacc9c4640 1 --2- 172.21.15.143:0/3056087430 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7ffac815d6e0 0x7ffac815bbd0 unknown :-1 s=CLOSED pgs=1087 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:23:17.245 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:17.243+0000 7ffacc9c4640 1 --2- 172.21.15.143:0/3056087430 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7ffac80d19f0 0x7ffac814c390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:23:17.245 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:17.243+0000 7ffacc9c4640 1 --2- 172.21.15.143:0/3056087430 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7ffac80b2160 0x7ffac80c95b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:23:17.246 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:17.243+0000 7ffacc9c4640 1 -- 172.21.15.143:0/3056087430 >> 172.21.15.143:0/3056087430 conn(0x7ffac809c550 msgr2=0x7ffac81560a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:23:17.246 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:17.243+0000 7ffacc9c4640 1 -- 172.21.15.143:0/3056087430 shutdown_connections 2023-11-26T07:23:17.246 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:17.243+0000 7ffacc9c4640 1 -- 172.21.15.143:0/3056087430 wait complete. 2023-11-26T07:23:17.268 DEBUG:teuthology.orchestra.run.smithi143:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_metadata cephfs_metadata --yes-i-really-really-mean-it 2023-11-26T07:23:17.453 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:17.451+0000 7fbecc832640 1 Processor -- start 2023-11-26T07:23:17.453 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:17.451+0000 7fbecc832640 1 -- start start 2023-11-26T07:23:17.455 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:17.451+0000 7fbecc832640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fbec8070ba0 0x7fbec806bae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:23:17.455 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:17.451+0000 7fbecc832640 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7fbec80cdd10 0x7fbec806d950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:23:17.455 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:17.451+0000 7fbecc832640 1 -- --> v1:172.21.15.143:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbec80cd610 con 0x7fbec8161cf0 2023-11-26T07:23:17.455 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:17.451+0000 7fbecc832640 1 -- --> v2:172.21.15.143:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fbec80cd8f0 con 0x7fbec80cdd10 2023-11-26T07:23:17.455 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:17.451+0000 7fbecc832640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbec8161210 con 0x7fbec8070ba0 2023-11-26T07:23:17.455 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:17.451+0000 7fbec6d9d640 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7fbec80cdd10 0x7fbec806d950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:23:17.455 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:17.451+0000 7fbec6d9d640 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7fbec80cdd10 0x7fbec806d950 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:40874/0 (socket says 172.21.15.143:40874) 2023-11-26T07:23:17.455 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:17.451+0000 7fbec6d9d640 1 -- 172.21.15.143:0/3541325386 learned_addr learned my addr 172.21.15.143:0/3541325386 (peer_addr_for_me v2:172.21.15.143:0/0) 2023-11-26T07:23:17.456 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:17.451+0000 7fbebdd9b640 1 --1- 172.21.15.143:0/3541325386 >> v1:172.21.15.143:6789/0 conn(0x7fbec8161cf0 0x7fbec8070f80 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.143:6789/0 says I am v1:172.21.15.143:36738/0 (socket says 172.21.15.143:36738) 2023-11-26T07:23:17.456 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:17.451+0000 7fbec659c640 1 --2- 172.21.15.143:0/3541325386 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fbec8070ba0 0x7fbec806bae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:23:17.456 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:17.455+0000 7fbec6d9d640 1 -- 172.21.15.143:0/3541325386 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fbec8070ba0 msgr2=0x7fbec806bae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-26T07:23:17.456 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:17.455+0000 7fbec6d9d640 1 --2- 172.21.15.143:0/3541325386 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fbec8070ba0 0x7fbec806bae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:23:17.456 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:17.455+0000 7fbec6d9d640 1 -- 172.21.15.143:0/3541325386 >> v1:172.21.15.143:6789/0 conn(0x7fbec8161cf0 legacy=0x7fbec8070f80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:23:17.456 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:17.455+0000 7fbec6d9d640 1 -- 172.21.15.143:0/3541325386 --> v2:172.21.15.143:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbec81614f0 con 0x7fbec80cdd10 2023-11-26T07:23:17.456 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:17.455+0000 7fbec659c640 1 --2- 172.21.15.143:0/3541325386 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fbec8070ba0 0x7fbec806bae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-26T07:23:17.458 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:17.455+0000 7fbec6d9d640 1 --2- 172.21.15.143:0/3541325386 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fbec80cdd10 0x7fbec806d950 secure :-1 s=READY pgs=1089 cs=0 l=1 rev1=1 crypto rx=0x7fbec00098a0 tx=0x7fbec001c520 comp rx=0 tx=0).ready entity=mon.0 client_cookie=279674386ef72dbd server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:23:17.459 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:17.455+0000 7fbec5d9b640 1 -- 172.21.15.143:0/3541325386 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbec00a7070 con 0x7fbec80cdd10 2023-11-26T07:23:17.459 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:17.455+0000 7fbec5d9b640 1 -- 172.21.15.143:0/3541325386 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbec00ab430 con 0x7fbec80cdd10 2023-11-26T07:23:17.459 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:17.455+0000 7fbec5d9b640 1 -- 172.21.15.143:0/3541325386 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbec00ab7c0 con 0x7fbec80cdd10 2023-11-26T07:23:17.459 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:17.455+0000 7fbecc832640 1 -- 172.21.15.143:0/3541325386 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fbec80cdd10 msgr2=0x7fbec806d950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:23:17.459 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:17.455+0000 7fbecc832640 1 --2- 172.21.15.143:0/3541325386 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fbec80cdd10 0x7fbec806d950 secure :-1 s=READY pgs=1089 cs=0 l=1 rev1=1 crypto rx=0x7fbec00098a0 tx=0x7fbec001c520 comp rx=0 tx=0).stop 2023-11-26T07:23:17.459 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:17.459+0000 7fbecc832640 1 -- 172.21.15.143:0/3541325386 shutdown_connections 2023-11-26T07:23:17.459 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:17.459+0000 7fbecc832640 1 --2- 172.21.15.143:0/3541325386 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fbec80cdd10 0x7fbec806d950 unknown :-1 s=CLOSED pgs=1089 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:23:17.459 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:17.459+0000 7fbecc832640 1 --2- 172.21.15.143:0/3541325386 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fbec8070ba0 0x7fbec806bae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:23:17.459 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:17.459+0000 7fbecc832640 1 -- 172.21.15.143:0/3541325386 >> 172.21.15.143:0/3541325386 conn(0x7fbec80cf710 msgr2=0x7fbec80cfb20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:23:17.460 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:17.459+0000 7fbecc832640 1 -- 172.21.15.143:0/3541325386 shutdown_connections 2023-11-26T07:23:17.460 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:17.459+0000 7fbecc832640 1 -- 172.21.15.143:0/3541325386 wait complete. 2023-11-26T07:23:17.461 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:17.459+0000 7fbecc832640 1 Processor -- start 2023-11-26T07:23:17.461 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:17.459+0000 7fbecc832640 1 -- start start 2023-11-26T07:23:17.463 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:17.459+0000 7fbecc832640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fbec8070ba0 0x7fbec813f220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:23:17.463 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:17.459+0000 7fbecc832640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fbec80cdd10 0x7fbec80ae960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:23:17.463 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:17.459+0000 7fbecc832640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fbec8161cf0 0x7fbec815c430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:23:17.463 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:17.459+0000 7fbecc832640 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbec80adc70 con 0x7fbec8070ba0 2023-11-26T07:23:17.463 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:17.459+0000 7fbecc832640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbec813f760 con 0x7fbec8161cf0 2023-11-26T07:23:17.463 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:17.459+0000 7fbecc832640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbec815d140 con 0x7fbec80cdd10 2023-11-26T07:23:17.463 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:17.459+0000 7fbec659c640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fbec8070ba0 0x7fbec813f220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:23:17.463 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:17.459+0000 7fbec659c640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fbec8070ba0 0x7fbec813f220 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:40886/0 (socket says 172.21.15.143:40886) 2023-11-26T07:23:17.464 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:17.459+0000 7fbec659c640 1 -- 172.21.15.143:0/3619143983 learned_addr learned my addr 172.21.15.143:0/3619143983 (peer_addr_for_me v2:172.21.15.143:0/0) 2023-11-26T07:23:17.464 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:17.459+0000 7fbebdd9b640 1 --2- 172.21.15.143:0/3619143983 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fbec80cdd10 0x7fbec80ae960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:23:17.464 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:17.459+0000 7fbec6d9d640 1 --2- 172.21.15.143:0/3619143983 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fbec8161cf0 0x7fbec815c430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:23:17.464 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:17.459+0000 7fbec659c640 1 -- 172.21.15.143:0/3619143983 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fbec80cdd10 msgr2=0x7fbec80ae960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:23:17.464 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:17.459+0000 7fbec659c640 1 --2- 172.21.15.143:0/3619143983 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fbec80cdd10 0x7fbec80ae960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:23:17.464 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:17.459+0000 7fbec659c640 1 -- 172.21.15.143:0/3619143983 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fbec8161cf0 msgr2=0x7fbec815c430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:23:17.464 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:17.459+0000 7fbec659c640 1 --2- 172.21.15.143:0/3619143983 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fbec8161cf0 0x7fbec815c430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:23:17.464 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:17.459+0000 7fbec659c640 1 -- 172.21.15.143:0/3619143983 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbec00094d0 con 0x7fbec8070ba0 2023-11-26T07:23:17.464 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:17.463+0000 7fbec659c640 1 --2- 172.21.15.143:0/3619143983 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fbec8070ba0 0x7fbec813f220 secure :-1 s=READY pgs=1090 cs=0 l=1 rev1=1 crypto rx=0x7fbeb800d100 tx=0x7fbeb800d5c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:23:17.464 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:17.463+0000 7fbebf7fe640 1 -- 172.21.15.143:0/3619143983 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbeb8003ba0 con 0x7fbec8070ba0 2023-11-26T07:23:17.465 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:17.463+0000 7fbecc832640 1 -- 172.21.15.143:0/3619143983 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbec815ce60 con 0x7fbec8070ba0 2023-11-26T07:23:17.465 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:17.463+0000 7fbebf7fe640 1 -- 172.21.15.143:0/3619143983 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbeb800f040 con 0x7fbec8070ba0 2023-11-26T07:23:17.465 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:17.463+0000 7fbebf7fe640 1 -- 172.21.15.143:0/3619143983 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbeb8004030 con 0x7fbec8070ba0 2023-11-26T07:23:17.465 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:17.463+0000 7fbecc832640 1 -- 172.21.15.143:0/3619143983 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbec80ae160 con 0x7fbec8070ba0 2023-11-26T07:23:17.465 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:17.463+0000 7fbecc832640 1 -- 172.21.15.143:0/3619143983 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbec8067680 con 0x7fbec8070ba0 2023-11-26T07:23:17.465 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:17.463+0000 7fbebf7fe640 1 -- 172.21.15.143:0/3619143983 <== mon.0 v2:172.21.15.143:3300/0 4 ==== mgrmap(e 16) v1 ==== 85954+0+0 (secure 0 0 0) 0x7fbeb8004190 con 0x7fbec8070ba0 2023-11-26T07:23:17.466 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:17.463+0000 7fbebf7fe640 1 --2- 172.21.15.143:0/3619143983 >> [v2:172.21.15.188:6836/3401128553,v1:172.21.15.188:6837/3401128553] conn(0x7fbea8066590 0x7fbea8068a50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:23:17.466 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:17.463+0000 7fbebf7fe640 1 -- 172.21.15.143:0/3619143983 <== mon.0 v2:172.21.15.143:3300/0 5 ==== osd_map(73..73 src has 1..73) v4 ==== 6354+0+0 (secure 0 0 0) 0x7fbeb8084750 con 0x7fbec8070ba0 2023-11-26T07:23:17.466 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:17.463+0000 7fbebdd9b640 1 --2- 172.21.15.143:0/3619143983 >> [v2:172.21.15.188:6836/3401128553,v1:172.21.15.188:6837/3401128553] conn(0x7fbea8066590 0x7fbea8068a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:23:17.467 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:17.467+0000 7fbebdd9b640 1 --2- 172.21.15.143:0/3619143983 >> [v2:172.21.15.188:6836/3401128553,v1:172.21.15.188:6837/3401128553] conn(0x7fbea8066590 0x7fbea8068a50 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7fbeb0000f70 tx=0x7fbeb0007c90 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:23:17.470 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:17.467+0000 7fbebf7fe640 1 -- 172.21.15.143:0/3619143983 <== mon.0 v2:172.21.15.143:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172649 (secure 0 0 0) 0x7fbeb80516b0 con 0x7fbec8070ba0 2023-11-26T07:23:17.623 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:17.623+0000 7fbecc832640 1 -- 172.21.15.143:0/3619143983 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7fbec81536a0 con 0x7fbec8070ba0 2023-11-26T07:23:18.574 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:18.571+0000 7fbebf7fe640 1 -- 172.21.15.143:0/3619143983 <== mon.0 v2:172.21.15.143:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_metadata' removed v74) v1 ==== 181+0+0 (secure 0 0 0) 0x7fbeb8054cd0 con 0x7fbec8070ba0 2023-11-26T07:23:18.574 INFO:teuthology.orchestra.run.smithi143.stderr:pool 'cephfs_metadata' removed 2023-11-26T07:23:18.578 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:18.575+0000 7fbecc832640 1 -- 172.21.15.143:0/3619143983 >> [v2:172.21.15.188:6836/3401128553,v1:172.21.15.188:6837/3401128553] conn(0x7fbea8066590 msgr2=0x7fbea8068a50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:23:18.578 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:18.575+0000 7fbecc832640 1 --2- 172.21.15.143:0/3619143983 >> [v2:172.21.15.188:6836/3401128553,v1:172.21.15.188:6837/3401128553] conn(0x7fbea8066590 0x7fbea8068a50 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7fbeb0000f70 tx=0x7fbeb0007c90 comp rx=0 tx=0).stop 2023-11-26T07:23:18.578 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:18.575+0000 7fbecc832640 1 -- 172.21.15.143:0/3619143983 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fbec8070ba0 msgr2=0x7fbec813f220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:23:18.578 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:18.575+0000 7fbecc832640 1 --2- 172.21.15.143:0/3619143983 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fbec8070ba0 0x7fbec813f220 secure :-1 s=READY pgs=1090 cs=0 l=1 rev1=1 crypto rx=0x7fbeb800d100 tx=0x7fbeb800d5c0 comp rx=0 tx=0).stop 2023-11-26T07:23:18.578 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:18.575+0000 7fbecc832640 1 -- 172.21.15.143:0/3619143983 shutdown_connections 2023-11-26T07:23:18.578 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:18.575+0000 7fbecc832640 1 --2- 172.21.15.143:0/3619143983 >> [v2:172.21.15.188:6836/3401128553,v1:172.21.15.188:6837/3401128553] conn(0x7fbea8066590 0x7fbea8068a50 unknown :-1 s=CLOSED pgs=36 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:23:18.578 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:18.575+0000 7fbecc832640 1 --2- 172.21.15.143:0/3619143983 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fbec8161cf0 0x7fbec815c430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:23:18.578 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:18.575+0000 7fbecc832640 1 --2- 172.21.15.143:0/3619143983 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fbec80cdd10 0x7fbec80ae960 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:23:18.579 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:18.575+0000 7fbecc832640 1 --2- 172.21.15.143:0/3619143983 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fbec8070ba0 0x7fbec813f220 unknown :-1 s=CLOSED pgs=1090 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:23:18.579 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:18.575+0000 7fbecc832640 1 -- 172.21.15.143:0/3619143983 >> 172.21.15.143:0/3619143983 conn(0x7fbec80cf710 msgr2=0x7fbec8152af0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:23:18.579 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:18.575+0000 7fbecc832640 1 -- 172.21.15.143:0/3619143983 shutdown_connections 2023-11-26T07:23:18.579 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:18.575+0000 7fbecc832640 1 -- 172.21.15.143:0/3619143983 wait complete. 2023-11-26T07:23:18.603 DEBUG:teuthology.orchestra.run.smithi143:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_data cephfs_data --yes-i-really-really-mean-it 2023-11-26T07:23:18.748 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:18.747+0000 7fe2ea888640 1 Processor -- start 2023-11-26T07:23:18.749 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:18.747+0000 7fe2ea888640 1 -- start start 2023-11-26T07:23:18.751 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:18.747+0000 7fe2ea888640 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7fe2e415aa50 0x7fe2e40add50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:23:18.751 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:18.747+0000 7fe2ea888640 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7fe2e415ae30 0x7fe2e40ae290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:23:18.751 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:18.747+0000 7fe2ea888640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fe2e4072540 0x7fe2e415d9a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:23:18.751 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:18.747+0000 7fe2ea888640 1 -- --> v2:172.21.15.143:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe2e4071780 con 0x7fe2e415aa50 2023-11-26T07:23:18.751 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:18.747+0000 7fe2ea888640 1 -- --> v2:172.21.15.188:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe2e4071a60 con 0x7fe2e415ae30 2023-11-26T07:23:18.751 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:18.747+0000 7fe2ea888640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe2e4071d40 con 0x7fe2e4072540 2023-11-26T07:23:18.751 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:18.747+0000 7fe2e9886640 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7fe2e415aa50 0x7fe2e40add50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:23:18.751 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:18.747+0000 7fe2e9886640 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7fe2e415aa50 0x7fe2e40add50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:40898/0 (socket says 172.21.15.143:40898) 2023-11-26T07:23:18.751 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:18.747+0000 7fe2e9886640 1 -- 172.21.15.143:0/4069123645 learned_addr learned my addr 172.21.15.143:0/4069123645 (peer_addr_for_me v2:172.21.15.143:0/0) 2023-11-26T07:23:18.751 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:18.747+0000 7fe2e9085640 1 --2- 172.21.15.143:0/4069123645 >> v2:172.21.15.188:3300/0 conn(0x7fe2e415ae30 0x7fe2e40ae290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:23:18.752 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:18.747+0000 7fe2ea087640 1 --2- 172.21.15.143:0/4069123645 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fe2e4072540 0x7fe2e415d9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:23:18.752 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:18.751+0000 7fe2e9886640 1 -- 172.21.15.143:0/4069123645 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fe2e4072540 msgr2=0x7fe2e415d9a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-26T07:23:18.752 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:18.751+0000 7fe2e9886640 1 --2- 172.21.15.143:0/4069123645 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fe2e4072540 0x7fe2e415d9a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:23:18.752 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:18.751+0000 7fe2e9886640 1 -- 172.21.15.143:0/4069123645 >> v2:172.21.15.188:3300/0 conn(0x7fe2e415ae30 msgr2=0x7fe2e40ae290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-26T07:23:18.752 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:18.751+0000 7fe2e9886640 1 --2- 172.21.15.143:0/4069123645 >> v2:172.21.15.188:3300/0 conn(0x7fe2e415ae30 0x7fe2e40ae290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:23:18.752 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:18.751+0000 7fe2e9886640 1 -- 172.21.15.143:0/4069123645 --> v2:172.21.15.143:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe2e4072020 con 0x7fe2e415aa50 2023-11-26T07:23:18.754 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:18.751+0000 7fe2e9886640 1 --2- 172.21.15.143:0/4069123645 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fe2e415aa50 0x7fe2e40add50 secure :-1 s=READY pgs=1091 cs=0 l=1 rev1=1 crypto rx=0x7fe2e415e750 tx=0x7fe2e001c500 comp rx=0 tx=0).ready entity=mon.0 client_cookie=da23e2f0eb845ce9 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:23:18.754 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:18.751+0000 7fe2e8884640 1 -- 172.21.15.143:0/4069123645 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe2e00aa640 con 0x7fe2e415aa50 2023-11-26T07:23:18.754 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:18.751+0000 7fe2e8884640 1 -- 172.21.15.143:0/4069123645 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe2e00aa7a0 con 0x7fe2e415aa50 2023-11-26T07:23:18.754 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:18.751+0000 7fe2e8884640 1 -- 172.21.15.143:0/4069123645 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe2e00aab30 con 0x7fe2e415aa50 2023-11-26T07:23:18.755 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:18.751+0000 7fe2ea888640 1 -- 172.21.15.143:0/4069123645 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fe2e415aa50 msgr2=0x7fe2e40add50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:23:18.755 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:18.751+0000 7fe2ea888640 1 --2- 172.21.15.143:0/4069123645 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fe2e415aa50 0x7fe2e40add50 secure :-1 s=READY pgs=1091 cs=0 l=1 rev1=1 crypto rx=0x7fe2e415e750 tx=0x7fe2e001c500 comp rx=0 tx=0).stop 2023-11-26T07:23:18.755 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:18.751+0000 7fe2ea888640 1 -- 172.21.15.143:0/4069123645 shutdown_connections 2023-11-26T07:23:18.755 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:18.751+0000 7fe2ea888640 1 --2- 172.21.15.143:0/4069123645 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fe2e4072540 0x7fe2e415d9a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:23:18.755 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:18.751+0000 7fe2ea888640 1 --2- 172.21.15.143:0/4069123645 >> v2:172.21.15.188:3300/0 conn(0x7fe2e415ae30 0x7fe2e40ae290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:23:18.755 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:18.751+0000 7fe2ea888640 1 --2- 172.21.15.143:0/4069123645 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fe2e415aa50 0x7fe2e40add50 unknown :-1 s=CLOSED pgs=1091 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:23:18.755 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:18.751+0000 7fe2ea888640 1 -- 172.21.15.143:0/4069123645 >> 172.21.15.143:0/4069123645 conn(0x7fe2e406a890 msgr2=0x7fe2e406aca0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:23:18.755 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:18.755+0000 7fe2ea888640 1 -- 172.21.15.143:0/4069123645 shutdown_connections 2023-11-26T07:23:18.755 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:18.755+0000 7fe2ea888640 1 -- 172.21.15.143:0/4069123645 wait complete. 2023-11-26T07:23:18.756 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:18.755+0000 7fe2ea888640 1 Processor -- start 2023-11-26T07:23:18.756 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:18.755+0000 7fe2ea888640 1 -- start start 2023-11-26T07:23:18.759 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:18.755+0000 7fe2ea888640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fe2e415aa50 0x7fe2e41420b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:23:18.759 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:18.755+0000 7fe2ea888640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fe2e415ae30 0x7fe2e4166830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:23:18.759 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:18.755+0000 7fe2ea888640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fe2e40cc7c0 0x7fe2e415bad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:23:18.759 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:18.755+0000 7fe2ea888640 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe2e40cc3a0 con 0x7fe2e415aa50 2023-11-26T07:23:18.759 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:18.755+0000 7fe2ea888640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe2e40cb7b0 con 0x7fe2e415ae30 2023-11-26T07:23:18.759 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:18.755+0000 7fe2ea888640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe2e4166d70 con 0x7fe2e40cc7c0 2023-11-26T07:23:18.759 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:18.755+0000 7fe2e9886640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fe2e415aa50 0x7fe2e41420b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:23:18.759 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:18.755+0000 7fe2e9886640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fe2e415aa50 0x7fe2e41420b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:40914/0 (socket says 172.21.15.143:40914) 2023-11-26T07:23:18.759 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:18.755+0000 7fe2e9886640 1 -- 172.21.15.143:0/1641513188 learned_addr learned my addr 172.21.15.143:0/1641513188 (peer_addr_for_me v2:172.21.15.143:0/0) 2023-11-26T07:23:18.760 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:18.755+0000 7fe2e9085640 1 --2- 172.21.15.143:0/1641513188 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fe2e415ae30 0x7fe2e4166830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:23:18.760 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:18.755+0000 7fe2ea087640 1 --2- 172.21.15.143:0/1641513188 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fe2e40cc7c0 0x7fe2e415bad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:23:18.760 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:18.755+0000 7fe2e9886640 1 -- 172.21.15.143:0/1641513188 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fe2e40cc7c0 msgr2=0x7fe2e415bad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:23:18.760 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:18.755+0000 7fe2e9886640 1 --2- 172.21.15.143:0/1641513188 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fe2e40cc7c0 0x7fe2e415bad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:23:18.760 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:18.755+0000 7fe2e9886640 1 -- 172.21.15.143:0/1641513188 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fe2e415ae30 msgr2=0x7fe2e4166830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:23:18.760 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:18.755+0000 7fe2e9886640 1 --2- 172.21.15.143:0/1641513188 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fe2e415ae30 0x7fe2e4166830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:23:18.760 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:18.755+0000 7fe2e9886640 1 -- 172.21.15.143:0/1641513188 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe2e00094d0 con 0x7fe2e415aa50 2023-11-26T07:23:18.760 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:18.755+0000 7fe2e9886640 1 --2- 172.21.15.143:0/1641513188 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fe2e415aa50 0x7fe2e41420b0 secure :-1 s=READY pgs=1092 cs=0 l=1 rev1=1 crypto rx=0x7fe2e00ac040 tx=0x7fe2e001ad40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:23:18.760 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:18.759+0000 7fe2da7fc640 1 -- 172.21.15.143:0/1641513188 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe2e00bd3f0 con 0x7fe2e415aa50 2023-11-26T07:23:18.760 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:18.759+0000 7fe2da7fc640 1 -- 172.21.15.143:0/1641513188 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe2e00bd550 con 0x7fe2e415aa50 2023-11-26T07:23:18.760 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:18.759+0000 7fe2ea888640 1 -- 172.21.15.143:0/1641513188 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe2e40cba00 con 0x7fe2e415aa50 2023-11-26T07:23:18.760 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:18.759+0000 7fe2da7fc640 1 -- 172.21.15.143:0/1641513188 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe2e00bd900 con 0x7fe2e415aa50 2023-11-26T07:23:18.760 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:18.759+0000 7fe2ea888640 1 -- 172.21.15.143:0/1641513188 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe2e40cbfc0 con 0x7fe2e415aa50 2023-11-26T07:23:18.761 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:18.759+0000 7fe2ea888640 1 -- 172.21.15.143:0/1641513188 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe2e40cb240 con 0x7fe2e415aa50 2023-11-26T07:23:18.761 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:18.759+0000 7fe2da7fc640 1 -- 172.21.15.143:0/1641513188 <== mon.0 v2:172.21.15.143:3300/0 4 ==== mgrmap(e 16) v1 ==== 85954+0+0 (secure 0 0 0) 0x7fe2e00a8050 con 0x7fe2e415aa50 2023-11-26T07:23:18.762 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:18.759+0000 7fe2da7fc640 1 --2- 172.21.15.143:0/1641513188 >> [v2:172.21.15.188:6836/3401128553,v1:172.21.15.188:6837/3401128553] conn(0x7fe2c8066590 0x7fe2c8068a50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:23:18.762 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:18.759+0000 7fe2da7fc640 1 -- 172.21.15.143:0/1641513188 <== mon.0 v2:172.21.15.143:3300/0 5 ==== osd_map(74..74 src has 1..74) v4 ==== 5916+0+0 (secure 0 0 0) 0x7fe2e011e7f0 con 0x7fe2e415aa50 2023-11-26T07:23:18.762 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:18.759+0000 7fe2e9085640 1 --2- 172.21.15.143:0/1641513188 >> [v2:172.21.15.188:6836/3401128553,v1:172.21.15.188:6837/3401128553] conn(0x7fe2c8066590 0x7fe2c8068a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:23:18.762 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:18.763+0000 7fe2e9085640 1 --2- 172.21.15.143:0/1641513188 >> [v2:172.21.15.188:6836/3401128553,v1:172.21.15.188:6837/3401128553] conn(0x7fe2c8066590 0x7fe2c8068a50 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7fe2e40b4180 tx=0x7fe2d4008040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:23:18.766 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:18.763+0000 7fe2da7fc640 1 -- 172.21.15.143:0/1641513188 <== mon.0 v2:172.21.15.143:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172649 (secure 0 0 0) 0x7fe2e00eba10 con 0x7fe2e415aa50 2023-11-26T07:23:18.917 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:18.915+0000 7fe2ea888640 1 -- 172.21.15.143:0/1641513188 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7fe2e4067500 con 0x7fe2e415aa50 2023-11-26T07:23:19.582 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:19.579+0000 7fe2da7fc640 1 -- 172.21.15.143:0/1641513188 <== mon.0 v2:172.21.15.143:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_data' removed v75) v1 ==== 169+0+0 (secure 0 0 0) 0x7fe2e00ef030 con 0x7fe2e415aa50 2023-11-26T07:23:19.582 INFO:teuthology.orchestra.run.smithi143.stderr:pool 'cephfs_data' removed 2023-11-26T07:23:19.585 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:19.583+0000 7fe2ea888640 1 -- 172.21.15.143:0/1641513188 >> [v2:172.21.15.188:6836/3401128553,v1:172.21.15.188:6837/3401128553] conn(0x7fe2c8066590 msgr2=0x7fe2c8068a50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:23:19.585 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:19.583+0000 7fe2ea888640 1 --2- 172.21.15.143:0/1641513188 >> [v2:172.21.15.188:6836/3401128553,v1:172.21.15.188:6837/3401128553] conn(0x7fe2c8066590 0x7fe2c8068a50 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7fe2e40b4180 tx=0x7fe2d4008040 comp rx=0 tx=0).stop 2023-11-26T07:23:19.585 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:19.583+0000 7fe2ea888640 1 -- 172.21.15.143:0/1641513188 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fe2e415aa50 msgr2=0x7fe2e41420b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:23:19.585 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:19.583+0000 7fe2ea888640 1 --2- 172.21.15.143:0/1641513188 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fe2e415aa50 0x7fe2e41420b0 secure :-1 s=READY pgs=1092 cs=0 l=1 rev1=1 crypto rx=0x7fe2e00ac040 tx=0x7fe2e001ad40 comp rx=0 tx=0).stop 2023-11-26T07:23:19.585 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:19.583+0000 7fe2ea888640 1 -- 172.21.15.143:0/1641513188 shutdown_connections 2023-11-26T07:23:19.585 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:19.583+0000 7fe2ea888640 1 --2- 172.21.15.143:0/1641513188 >> [v2:172.21.15.188:6836/3401128553,v1:172.21.15.188:6837/3401128553] conn(0x7fe2c8066590 0x7fe2c8068a50 unknown :-1 s=CLOSED pgs=37 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:23:19.585 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:19.583+0000 7fe2ea888640 1 --2- 172.21.15.143:0/1641513188 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fe2e40cc7c0 0x7fe2e415bad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:23:19.586 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:19.583+0000 7fe2ea888640 1 --2- 172.21.15.143:0/1641513188 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fe2e415ae30 0x7fe2e4166830 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:23:19.586 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:19.583+0000 7fe2ea888640 1 --2- 172.21.15.143:0/1641513188 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fe2e415aa50 0x7fe2e41420b0 unknown :-1 s=CLOSED pgs=1092 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:23:19.586 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:19.583+0000 7fe2ea888640 1 -- 172.21.15.143:0/1641513188 >> 172.21.15.143:0/1641513188 conn(0x7fe2e406a890 msgr2=0x7fe2e415fcc0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:23:19.586 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:19.583+0000 7fe2ea888640 1 -- 172.21.15.143:0/1641513188 shutdown_connections 2023-11-26T07:23:19.586 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:19.583+0000 7fe2ea888640 1 -- 172.21.15.143:0/1641513188 wait complete. 2023-11-26T07:23:19.612 DEBUG:teuthology.orchestra.run.smithi143:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config rm mon mon_allow_pool_delete 2023-11-26T07:23:19.770 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:19.767+0000 7f4f23180640 1 Processor -- start 2023-11-26T07:23:19.770 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:19.767+0000 7f4f23180640 1 -- start start 2023-11-26T07:23:19.771 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:19.771+0000 7f4f23180640 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7f4f1c0ce7c0 0x7f4f1c15f6b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:23:19.772 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:19.771+0000 7f4f23180640 1 -- --> v1:172.21.15.143:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4f1c06c520 con 0x7f4f1c0cf990 2023-11-26T07:23:19.772 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:19.771+0000 7f4f23180640 1 -- --> v1:172.21.15.188:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4f1c06c800 con 0x7f4f1c13e290 2023-11-26T07:23:19.772 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:19.771+0000 7f4f23180640 1 -- --> v2:172.21.15.188:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4f1c06cae0 con 0x7f4f1c0ce7c0 2023-11-26T07:23:19.772 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:19.771+0000 7f4f13fff640 1 --1- >> v1:172.21.15.143:6789/0 conn(0x7f4f1c0cf990 0x7f4f1c15efa0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.143:6789/0 says I am v1:172.21.15.143:36742/0 (socket says 172.21.15.143:36742) 2023-11-26T07:23:19.772 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:19.771+0000 7f4f13fff640 1 -- 172.21.15.143:0/1215639406 learned_addr learned my addr 172.21.15.143:0/1215639406 (peer_addr_for_me v1:172.21.15.143:0/0) 2023-11-26T07:23:19.774 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:19.771+0000 7f4f2171d640 1 --2- 172.21.15.143:0/1215639406 >> v2:172.21.15.188:3300/0 conn(0x7f4f1c0ce7c0 0x7f4f1c15f6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:23:19.774 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:19.771+0000 7f4f137fe640 1 -- 172.21.15.143:0/1215639406 <== mon.0 v1:172.21.15.143:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3884178716 0 0) 0x7f4f1c06c520 con 0x7f4f1c0cf990 2023-11-26T07:23:19.774 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:19.771+0000 7f4f137fe640 1 -- 172.21.15.143:0/1215639406 --> v1:172.21.15.143:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f4f04003530 con 0x7f4f1c0cf990 2023-11-26T07:23:19.774 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:19.771+0000 7f4f137fe640 1 -- 172.21.15.143:0/1215639406 <== mon.0 v1:172.21.15.143:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 872504920 0 0) 0x7f4f04003530 con 0x7f4f1c0cf990 2023-11-26T07:23:19.774 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:19.771+0000 7f4f137fe640 1 -- 172.21.15.143:0/1215639406 >> v2:172.21.15.188:3300/0 conn(0x7f4f1c0ce7c0 msgr2=0x7f4f1c15f6b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-26T07:23:19.774 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:19.771+0000 7f4f137fe640 1 --2- 172.21.15.143:0/1215639406 >> v2:172.21.15.188:3300/0 conn(0x7f4f1c0ce7c0 0x7f4f1c15f6b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:23:19.775 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:19.771+0000 7f4f137fe640 1 -- 172.21.15.143:0/1215639406 >> v1:172.21.15.188:6789/0 conn(0x7f4f1c13e290 legacy=0x7f4f1c0cf280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:23:19.775 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:19.771+0000 7f4f137fe640 1 -- 172.21.15.143:0/1215639406 --> v1:172.21.15.143:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4f1c06cdc0 con 0x7f4f1c0cf990 2023-11-26T07:23:19.775 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:19.771+0000 7f4f137fe640 1 -- 172.21.15.143:0/1215639406 <== mon.0 v1:172.21.15.143:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2137882523 0 0) 0x7f4f0c001680 con 0x7f4f1c0cf990 2023-11-26T07:23:19.775 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:19.771+0000 7f4f2171d640 1 --2- 172.21.15.143:0/1215639406 >> v2:172.21.15.188:3300/0 conn(0x7f4f1c0ce7c0 0x7f4f1c15f6b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-26T07:23:19.775 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:19.771+0000 7f4f137fe640 1 -- 172.21.15.143:0/1215639406 >> v1:172.21.15.143:6789/0 conn(0x7f4f1c0cf990 legacy=0x7f4f1c15efa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:23:19.775 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:19.771+0000 7f4f137fe640 1 --2- 172.21.15.143:0/1215639406 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f4f04003980 0x7f4f04003da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:23:19.775 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:19.771+0000 7f4f137fe640 1 --2- 172.21.15.143:0/1215639406 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f4f040047d0 0x7f4f04004c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:23:19.775 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:19.771+0000 7f4f137fe640 1 --2- 172.21.15.143:0/1215639406 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f4f04005190 0x7f4f0400b990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:23:19.775 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:19.771+0000 7f4f137fe640 1 -- 172.21.15.143:0/1215639406 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4f1c06cae0 con 0x7f4f04003980 2023-11-26T07:23:19.775 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:19.771+0000 7f4f137fe640 1 -- 172.21.15.143:0/1215639406 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4f0400c130 con 0x7f4f040047d0 2023-11-26T07:23:19.775 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:19.771+0000 7f4f137fe640 1 -- 172.21.15.143:0/1215639406 --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4f0400c2f0 con 0x7f4f04005190 2023-11-26T07:23:19.776 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:19.771+0000 7f4f20f1c640 1 --2- 172.21.15.143:0/1215639406 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f4f04003980 0x7f4f04003da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:23:19.776 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:19.771+0000 7f4f13fff640 1 --2- 172.21.15.143:0/1215639406 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f4f040047d0 0x7f4f04004c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:23:19.776 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:19.771+0000 7f4f2171d640 1 --2- 172.21.15.143:0/1215639406 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f4f04005190 0x7f4f0400b990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:23:19.776 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:19.771+0000 7f4f20f1c640 1 -- 172.21.15.143:0/1215639406 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f4f04005190 msgr2=0x7f4f0400b990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-26T07:23:19.776 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:19.771+0000 7f4f20f1c640 1 --2- 172.21.15.143:0/1215639406 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f4f04005190 0x7f4f0400b990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:23:19.776 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:19.771+0000 7f4f20f1c640 1 -- 172.21.15.143:0/1215639406 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f4f040047d0 msgr2=0x7f4f04004c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-26T07:23:19.776 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:19.771+0000 7f4f20f1c640 1 --2- 172.21.15.143:0/1215639406 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f4f040047d0 0x7f4f04004c50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:23:19.776 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:19.771+0000 7f4f20f1c640 1 -- 172.21.15.143:0/1215639406 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4f04003530 con 0x7f4f04003980 2023-11-26T07:23:19.776 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:19.771+0000 7f4f2171d640 1 -- 172.21.15.143:0/1215639406 reap_dead start 2023-11-26T07:23:19.777 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:19.775+0000 7f4f20f1c640 1 --2- 172.21.15.143:0/1215639406 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f4f04003980 0x7f4f04003da0 secure :-1 s=READY pgs=1094 cs=0 l=1 rev1=1 crypto rx=0x7f4f18001780 tx=0x7f4f1801c3d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9ecac6de2a63c63e server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:23:19.777 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:19.775+0000 7f4f137fe640 1 -- 172.21.15.143:0/1215639406 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4f180052d0 con 0x7f4f04003980 2023-11-26T07:23:19.777 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:19.775+0000 7f4f137fe640 1 -- 172.21.15.143:0/1215639406 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4f180a8050 con 0x7f4f04003980 2023-11-26T07:23:19.777 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:19.775+0000 7f4f137fe640 1 -- 172.21.15.143:0/1215639406 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4f18003600 con 0x7f4f04003980 2023-11-26T07:23:19.777 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:19.775+0000 7f4f23180640 1 -- 172.21.15.143:0/1215639406 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f4f04003980 msgr2=0x7f4f04003da0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:23:19.778 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:19.775+0000 7f4f23180640 1 --2- 172.21.15.143:0/1215639406 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f4f04003980 0x7f4f04003da0 secure :-1 s=READY pgs=1094 cs=0 l=1 rev1=1 crypto rx=0x7f4f18001780 tx=0x7f4f1801c3d0 comp rx=0 tx=0).stop 2023-11-26T07:23:19.778 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:19.775+0000 7f4f23180640 1 -- 172.21.15.143:0/1215639406 shutdown_connections 2023-11-26T07:23:19.778 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:19.775+0000 7f4f23180640 1 --2- 172.21.15.143:0/1215639406 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f4f04003980 0x7f4f04003da0 unknown :-1 s=CLOSED pgs=1094 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:23:19.778 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:19.775+0000 7f4f23180640 1 -- 172.21.15.143:0/1215639406 >> 172.21.15.143:0/1215639406 conn(0x7f4f1c06a890 msgr2=0x7f4f1c06aca0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:23:19.778 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:19.775+0000 7f4f23180640 1 -- 172.21.15.143:0/1215639406 shutdown_connections 2023-11-26T07:23:19.778 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:19.775+0000 7f4f23180640 1 -- 172.21.15.143:0/1215639406 wait complete. 2023-11-26T07:23:19.779 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:19.779+0000 7f4f23180640 1 Processor -- start 2023-11-26T07:23:19.779 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:19.779+0000 7f4f23180640 1 -- start start 2023-11-26T07:23:19.780 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:19.779+0000 7f4f23180640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f4f04003980 0x7f4f1c1546e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:23:19.780 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:19.779+0000 7f4f23180640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f4f1c138f90 0x7f4f1c1393f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:23:19.780 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:19.779+0000 7f4f23180640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f4f1c15cb80 0x7f4f1c15d030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:23:19.780 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:19.779+0000 7f4f23180640 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4f1c139930 con 0x7f4f04003980 2023-11-26T07:23:19.780 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:19.779+0000 7f4f23180640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4f1c15d8b0 con 0x7f4f1c138f90 2023-11-26T07:23:19.781 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:19.779+0000 7f4f23180640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4f1c15db90 con 0x7f4f1c15cb80 2023-11-26T07:23:19.781 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:19.779+0000 7f4f20f1c640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f4f04003980 0x7f4f1c1546e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:23:19.781 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:19.779+0000 7f4f20f1c640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f4f04003980 0x7f4f1c1546e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:40934/0 (socket says 172.21.15.143:40934) 2023-11-26T07:23:19.781 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:19.779+0000 7f4f20f1c640 1 -- 172.21.15.143:0/1523701885 learned_addr learned my addr 172.21.15.143:0/1523701885 (peer_addr_for_me v2:172.21.15.143:0/0) 2023-11-26T07:23:19.782 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:19.779+0000 7f4f2171d640 1 --2- 172.21.15.143:0/1523701885 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f4f1c15cb80 0x7f4f1c15d030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:23:19.782 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:19.779+0000 7f4f20f1c640 1 -- 172.21.15.143:0/1523701885 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f4f1c15cb80 msgr2=0x7f4f1c15d030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:23:19.782 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:19.779+0000 7f4f20f1c640 1 --2- 172.21.15.143:0/1523701885 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f4f1c15cb80 0x7f4f1c15d030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:23:19.782 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:19.779+0000 7f4f20f1c640 1 -- 172.21.15.143:0/1523701885 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f4f1c138f90 msgr2=0x7f4f1c1393f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:23:19.783 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:19.779+0000 7f4f13fff640 1 --2- 172.21.15.143:0/1523701885 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f4f1c138f90 0x7f4f1c1393f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:23:19.783 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:19.779+0000 7f4f20f1c640 1 --2- 172.21.15.143:0/1523701885 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f4f1c138f90 0x7f4f1c1393f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:23:19.783 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:19.779+0000 7f4f20f1c640 1 -- 172.21.15.143:0/1523701885 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4f1c15de70 con 0x7f4f04003980 2023-11-26T07:23:19.783 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:19.779+0000 7f4f2171d640 1 --2- 172.21.15.143:0/1523701885 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f4f1c15cb80 0x7f4f1c15d030 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-26T07:23:19.783 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:19.779+0000 7f4f13fff640 1 --2- 172.21.15.143:0/1523701885 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f4f1c138f90 0x7f4f1c1393f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-26T07:23:19.783 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:19.779+0000 7f4f20f1c640 1 --2- 172.21.15.143:0/1523701885 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f4f04003980 0x7f4f1c1546e0 secure :-1 s=READY pgs=1095 cs=0 l=1 rev1=1 crypto rx=0x7f4f18005040 tx=0x7f4f04005800 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:23:19.783 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:19.779+0000 7f4f117fa640 1 -- 172.21.15.143:0/1523701885 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4f18007590 con 0x7f4f04003980 2023-11-26T07:23:19.783 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:19.779+0000 7f4f117fa640 1 -- 172.21.15.143:0/1523701885 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4f180067e0 con 0x7f4f04003980 2023-11-26T07:23:19.783 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:19.779+0000 7f4f117fa640 1 -- 172.21.15.143:0/1523701885 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4f180076f0 con 0x7f4f04003980 2023-11-26T07:23:19.783 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:19.779+0000 7f4f23180640 1 -- 172.21.15.143:0/1523701885 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4f1c15e150 con 0x7f4f04003980 2023-11-26T07:23:19.783 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:19.779+0000 7f4f23180640 1 -- 172.21.15.143:0/1523701885 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4f1c15d570 con 0x7f4f04003980 2023-11-26T07:23:19.784 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:19.779+0000 7f4f23180640 1 -- 172.21.15.143:0/1523701885 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4f1c067680 con 0x7f4f04003980 2023-11-26T07:23:19.784 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:19.783+0000 7f4f117fa640 1 -- 172.21.15.143:0/1523701885 <== mon.0 v2:172.21.15.143:3300/0 4 ==== mgrmap(e 16) v1 ==== 85954+0+0 (secure 0 0 0) 0x7f4f18006b90 con 0x7f4f04003980 2023-11-26T07:23:19.784 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:19.783+0000 7f4f117fa640 1 --2- 172.21.15.143:0/1523701885 >> [v2:172.21.15.188:6836/3401128553,v1:172.21.15.188:6837/3401128553] conn(0x7f4f080665c0 0x7f4f08068a80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:23:19.784 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:19.783+0000 7f4f117fa640 1 -- 172.21.15.143:0/1523701885 <== mon.0 v2:172.21.15.143:3300/0 5 ==== osd_map(75..75 src has 1..75) v4 ==== 5502+0+0 (secure 0 0 0) 0x7f4f1811e590 con 0x7f4f04003980 2023-11-26T07:23:19.785 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:19.783+0000 7f4f13fff640 1 --2- 172.21.15.143:0/1523701885 >> [v2:172.21.15.188:6836/3401128553,v1:172.21.15.188:6837/3401128553] conn(0x7f4f080665c0 0x7f4f08068a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:23:19.785 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:19.783+0000 7f4f13fff640 1 --2- 172.21.15.143:0/1523701885 >> [v2:172.21.15.188:6836/3401128553,v1:172.21.15.188:6837/3401128553] conn(0x7f4f080665c0 0x7f4f08068a80 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7f4f1c139f30 tx=0x7f4f0c009040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:23:19.788 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:19.787+0000 7f4f117fa640 1 -- 172.21.15.143:0/1523701885 <== mon.0 v2:172.21.15.143:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172649 (secure 0 0 0) 0x7f4f180eb820 con 0x7f4f04003980 2023-11-26T07:23:19.946 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:19.943+0000 7f4f23180640 1 -- 172.21.15.143:0/1523701885 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "config rm", "who": "mon", "name": "mon_allow_pool_delete"} v 0) v1 -- 0x7f4f1c06df50 con 0x7f4f04003980 2023-11-26T07:23:19.949 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:19.947+0000 7f4f117fa640 1 -- 172.21.15.143:0/1523701885 <== mon.0 v2:172.21.15.143:3300/0 7 ==== mon_command_ack([{"prefix": "config rm", "who": "mon", "name": "mon_allow_pool_delete"}]=0 v17) v1 ==== 104+0+0 (secure 0 0 0) 0x7f4f180eee40 con 0x7f4f04003980 2023-11-26T07:23:19.952 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:19.951+0000 7f4f23180640 1 -- 172.21.15.143:0/1523701885 >> [v2:172.21.15.188:6836/3401128553,v1:172.21.15.188:6837/3401128553] conn(0x7f4f080665c0 msgr2=0x7f4f08068a80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:23:19.952 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:19.951+0000 7f4f23180640 1 --2- 172.21.15.143:0/1523701885 >> [v2:172.21.15.188:6836/3401128553,v1:172.21.15.188:6837/3401128553] conn(0x7f4f080665c0 0x7f4f08068a80 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7f4f1c139f30 tx=0x7f4f0c009040 comp rx=0 tx=0).stop 2023-11-26T07:23:19.952 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:19.951+0000 7f4f23180640 1 -- 172.21.15.143:0/1523701885 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f4f04003980 msgr2=0x7f4f1c1546e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:23:19.952 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:19.951+0000 7f4f23180640 1 --2- 172.21.15.143:0/1523701885 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f4f04003980 0x7f4f1c1546e0 secure :-1 s=READY pgs=1095 cs=0 l=1 rev1=1 crypto rx=0x7f4f18005040 tx=0x7f4f04005800 comp rx=0 tx=0).stop 2023-11-26T07:23:19.953 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:19.951+0000 7f4f23180640 1 -- 172.21.15.143:0/1523701885 shutdown_connections 2023-11-26T07:23:19.953 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:19.951+0000 7f4f23180640 1 --2- 172.21.15.143:0/1523701885 >> [v2:172.21.15.188:6836/3401128553,v1:172.21.15.188:6837/3401128553] conn(0x7f4f080665c0 0x7f4f08068a80 unknown :-1 s=CLOSED pgs=38 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:23:19.953 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:19.951+0000 7f4f23180640 1 --2- 172.21.15.143:0/1523701885 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f4f1c15cb80 0x7f4f1c15d030 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:23:19.953 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:19.951+0000 7f4f23180640 1 --2- 172.21.15.143:0/1523701885 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f4f1c138f90 0x7f4f1c1393f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:23:19.953 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:19.951+0000 7f4f23180640 1 --2- 172.21.15.143:0/1523701885 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f4f04003980 0x7f4f1c1546e0 unknown :-1 s=CLOSED pgs=1095 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:23:19.953 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:19.951+0000 7f4f23180640 1 -- 172.21.15.143:0/1523701885 >> 172.21.15.143:0/1523701885 conn(0x7f4f1c06a890 msgr2=0x7f4f1c0b20d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:23:19.953 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:19.951+0000 7f4f23180640 1 -- 172.21.15.143:0/1523701885 shutdown_connections 2023-11-26T07:23:19.954 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:19.951+0000 7f4f23180640 1 -- 172.21.15.143:0/1523701885 wait complete. 2023-11-26T07:23:19.972 DEBUG:teuthology.orchestra.run.smithi143:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Ended test tasks.cephfs.test_volumes.TestMisc.test_mgr_eviction' 2023-11-26T07:23:20.109 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:20.107+0000 7fa2d1565640 1 Processor -- start 2023-11-26T07:23:20.109 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:20.107+0000 7fa2d1565640 1 -- start start 2023-11-26T07:23:20.110 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:20.107+0000 7fa2d1565640 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7fa2cc143ac0 0x7fa2cc070370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:23:20.111 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:20.107+0000 7fa2d1565640 1 -- --> v1:172.21.15.143:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa2cc0cc730 con 0x7fa2cc0cf990 2023-11-26T07:23:20.111 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:20.107+0000 7fa2d1565640 1 -- --> v1:172.21.15.188:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa2cc070ee0 con 0x7fa2cc0708b0 2023-11-26T07:23:20.111 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:20.107+0000 7fa2d1565640 1 -- --> v2:172.21.15.143:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa2cc143ea0 con 0x7fa2cc143ac0 2023-11-26T07:23:20.111 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:20.107+0000 7fa2caffd640 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7fa2cc143ac0 0x7fa2cc070370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:23:20.111 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:20.107+0000 7fa2cb7fe640 1 --1- >> v1:172.21.15.143:6789/0 conn(0x7fa2cc0cf990 0x7fa2cc0cd1b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.143:6789/0 says I am v1:172.21.15.143:36746/0 (socket says 172.21.15.143:36746) 2023-11-26T07:23:20.111 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:20.111+0000 7fa2cb7fe640 1 -- 172.21.15.143:0/360919337 learned_addr learned my addr 172.21.15.143:0/360919337 (peer_addr_for_me v1:172.21.15.143:0/0) 2023-11-26T07:23:20.112 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:20.111+0000 7fa2caffd640 1 -- 172.21.15.143:0/360919337 >> v1:172.21.15.188:6789/0 conn(0x7fa2cc0708b0 legacy=0x7fa2cc0cf280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-26T07:23:20.112 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:20.111+0000 7fa2caffd640 1 -- 172.21.15.143:0/360919337 >> v1:172.21.15.143:6789/0 conn(0x7fa2cc0cf990 legacy=0x7fa2cc0cd1b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:23:20.112 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:20.111+0000 7fa2caffd640 1 -- 172.21.15.143:0/360919337 --> v2:172.21.15.143:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa2cc0711c0 con 0x7fa2cc143ac0 2023-11-26T07:23:20.112 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:20.111+0000 7fa2ca7fc640 1 -- 172.21.15.143:0/360919337 <== mon.0 v1:172.21.15.143:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1563177795 0 0) 0x7fa2cc0cc730 con 0x7fa2cc0cf990 2023-11-26T07:23:20.114 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:20.111+0000 7fa2caffd640 1 --2- 172.21.15.143:0/360919337 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fa2cc143ac0 0x7fa2cc070370 secure :-1 s=READY pgs=1097 cs=0 l=1 rev1=1 crypto rx=0x7fa2b40097c0 tx=0x7fa2b401c440 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ee387535c0500874 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:23:20.114 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:20.111+0000 7fa2ca7fc640 1 -- 172.21.15.143:0/360919337 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa2b40ab640 con 0x7fa2cc143ac0 2023-11-26T07:23:20.114 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:20.111+0000 7fa2ca7fc640 1 -- 172.21.15.143:0/360919337 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa2b40a7070 con 0x7fa2cc143ac0 2023-11-26T07:23:20.115 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:20.111+0000 7fa2ca7fc640 1 -- 172.21.15.143:0/360919337 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa2b40ab9c0 con 0x7fa2cc143ac0 2023-11-26T07:23:20.115 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:20.115+0000 7fa2d1565640 1 -- 172.21.15.143:0/360919337 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fa2cc143ac0 msgr2=0x7fa2cc070370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:23:20.115 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:20.115+0000 7fa2d1565640 1 --2- 172.21.15.143:0/360919337 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fa2cc143ac0 0x7fa2cc070370 secure :-1 s=READY pgs=1097 cs=0 l=1 rev1=1 crypto rx=0x7fa2b40097c0 tx=0x7fa2b401c440 comp rx=0 tx=0).stop 2023-11-26T07:23:20.116 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:20.115+0000 7fa2d1565640 1 -- 172.21.15.143:0/360919337 shutdown_connections 2023-11-26T07:23:20.116 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:20.115+0000 7fa2d1565640 1 --2- 172.21.15.143:0/360919337 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fa2cc143ac0 0x7fa2cc070370 unknown :-1 s=CLOSED pgs=1097 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:23:20.116 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:20.115+0000 7fa2d1565640 1 -- 172.21.15.143:0/360919337 >> 172.21.15.143:0/360919337 conn(0x7fa2cc06a890 msgr2=0x7fa2cc06aca0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:23:20.116 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:20.115+0000 7fa2d1565640 1 -- 172.21.15.143:0/360919337 shutdown_connections 2023-11-26T07:23:20.116 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:20.115+0000 7fa2d1565640 1 -- 172.21.15.143:0/360919337 wait complete. 2023-11-26T07:23:20.117 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:20.115+0000 7fa2d1565640 1 Processor -- start 2023-11-26T07:23:20.117 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:20.115+0000 7fa2d1565640 1 -- start start 2023-11-26T07:23:20.118 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:20.115+0000 7fa2d1565640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fa2cc0708b0 0x7fa2cc0c40c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:23:20.118 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:20.115+0000 7fa2d1565640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fa2cc0cf990 0x7fa2cc15cca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:23:20.119 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:20.115+0000 7fa2d1565640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fa2cc143ac0 0x7fa2cc15d1e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:23:20.119 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:20.115+0000 7fa2d1565640 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa2cc15c880 con 0x7fa2cc143ac0 2023-11-26T07:23:20.119 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:20.115+0000 7fa2d1565640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa2cc15bc00 con 0x7fa2cc0708b0 2023-11-26T07:23:20.119 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:20.115+0000 7fa2d1565640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa2cc15bee0 con 0x7fa2cc0cf990 2023-11-26T07:23:20.119 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:20.115+0000 7fa2cb7fe640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fa2cc143ac0 0x7fa2cc15d1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:23:20.119 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:20.115+0000 7fa2cb7fe640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fa2cc143ac0 0x7fa2cc15d1e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:40964/0 (socket says 172.21.15.143:40964) 2023-11-26T07:23:20.120 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:20.115+0000 7fa2cb7fe640 1 -- 172.21.15.143:0/734070627 learned_addr learned my addr 172.21.15.143:0/734070627 (peer_addr_for_me v2:172.21.15.143:0/0) 2023-11-26T07:23:20.120 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:20.115+0000 7fa2caffd640 1 --2- 172.21.15.143:0/734070627 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fa2cc0708b0 0x7fa2cc0c40c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:23:20.120 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:20.115+0000 7fa2c27fc640 1 --2- 172.21.15.143:0/734070627 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fa2cc0cf990 0x7fa2cc15cca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:23:20.120 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:20.115+0000 7fa2cb7fe640 1 -- 172.21.15.143:0/734070627 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fa2cc0cf990 msgr2=0x7fa2cc15cca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:23:20.120 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:20.115+0000 7fa2cb7fe640 1 --2- 172.21.15.143:0/734070627 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fa2cc0cf990 0x7fa2cc15cca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:23:20.120 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:20.115+0000 7fa2cb7fe640 1 -- 172.21.15.143:0/734070627 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fa2cc0708b0 msgr2=0x7fa2cc0c40c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:23:20.120 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:20.115+0000 7fa2cb7fe640 1 --2- 172.21.15.143:0/734070627 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fa2cc0708b0 0x7fa2cc0c40c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:23:20.121 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:20.115+0000 7fa2cb7fe640 1 -- 172.21.15.143:0/734070627 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa2b4009420 con 0x7fa2cc143ac0 2023-11-26T07:23:20.121 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:20.115+0000 7fa2c27fc640 1 --2- 172.21.15.143:0/734070627 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fa2cc0cf990 0x7fa2cc15cca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-26T07:23:20.121 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:20.115+0000 7fa2cb7fe640 1 --2- 172.21.15.143:0/734070627 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fa2cc143ac0 0x7fa2cc15d1e0 secure :-1 s=READY pgs=1098 cs=0 l=1 rev1=1 crypto rx=0x7fa2c400c780 tx=0x7fa2c400cc40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:23:20.122 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:20.119+0000 7fa2c3fff640 1 -- 172.21.15.143:0/734070627 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa2c400ce90 con 0x7fa2cc143ac0 2023-11-26T07:23:20.122 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:20.119+0000 7fa2c3fff640 1 -- 172.21.15.143:0/734070627 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa2c400a640 con 0x7fa2cc143ac0 2023-11-26T07:23:20.123 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:20.119+0000 7fa2c3fff640 1 -- 172.21.15.143:0/734070627 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa2c400a9d0 con 0x7fa2cc143ac0 2023-11-26T07:23:20.123 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:20.119+0000 7fa2d1565640 1 -- 172.21.15.143:0/734070627 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa2cc15c1c0 con 0x7fa2cc143ac0 2023-11-26T07:23:20.123 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:20.119+0000 7fa2d1565640 1 -- 172.21.15.143:0/734070627 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa2cc0c4600 con 0x7fa2cc143ac0 2023-11-26T07:23:20.123 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:20.119+0000 7fa2d1565640 1 -- 172.21.15.143:0/734070627 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa2cc15c9c0 con 0x7fa2cc143ac0 2023-11-26T07:23:20.123 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:20.119+0000 7fa2c3fff640 1 -- 172.21.15.143:0/734070627 <== mon.0 v2:172.21.15.143:3300/0 4 ==== mgrmap(e 16) v1 ==== 85954+0+0 (secure 0 0 0) 0x7fa2c400ab30 con 0x7fa2cc143ac0 2023-11-26T07:23:20.123 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:20.119+0000 7fa2c3fff640 1 --2- 172.21.15.143:0/734070627 >> [v2:172.21.15.188:6836/3401128553,v1:172.21.15.188:6837/3401128553] conn(0x7fa2ac066590 0x7fa2ac068a50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:23:20.123 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:20.119+0000 7fa2c3fff640 1 -- 172.21.15.143:0/734070627 <== mon.0 v2:172.21.15.143:3300/0 5 ==== osd_map(75..75 src has 1..75) v4 ==== 5502+0+0 (secure 0 0 0) 0x7fa2c4083360 con 0x7fa2cc143ac0 2023-11-26T07:23:20.124 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:20.119+0000 7fa2caffd640 1 --2- 172.21.15.143:0/734070627 >> [v2:172.21.15.188:6836/3401128553,v1:172.21.15.188:6837/3401128553] conn(0x7fa2ac066590 0x7fa2ac068a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:23:20.124 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:20.123+0000 7fa2caffd640 1 --2- 172.21.15.143:0/734070627 >> [v2:172.21.15.188:6836/3401128553,v1:172.21.15.188:6837/3401128553] conn(0x7fa2ac066590 0x7fa2ac068a50 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7fa2b40af000 tx=0x7fa2b40a7000 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:23:20.126 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:20.123+0000 7fa2c3fff640 1 -- 172.21.15.143:0/734070627 <== mon.0 v2:172.21.15.143:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172649 (secure 0 0 0) 0x7fa2cc15c9c0 con 0x7fa2cc143ac0 2023-11-26T07:23:20.327 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:20.327+0000 7fa2d1565640 1 -- 172.21.15.143:0/734070627 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_volumes.TestMisc.test_mgr_eviction"]} v 0) v1 -- 0x7fa2cc15c9c0 con 0x7fa2cc143ac0 2023-11-26T07:23:20.584 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:20.583+0000 7fa2c3fff640 1 -- 172.21.15.143:0/734070627 <== mon.0 v2:172.21.15.143:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_volumes.TestMisc.test_mgr_eviction"]}]=0 v287) v1 ==== 131+0+0 (secure 0 0 0) 0x7fa2cc15c9c0 con 0x7fa2cc143ac0 2023-11-26T07:23:20.833 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:20.587+0000 7fa2d1565640 1 -- 172.21.15.143:0/734070627 >> [v2:172.21.15.188:6836/3401128553,v1:172.21.15.188:6837/3401128553] conn(0x7fa2ac066590 msgr2=0x7fa2ac068a50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:23:20.833 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:20.587+0000 7fa2d1565640 1 --2- 172.21.15.143:0/734070627 >> [v2:172.21.15.188:6836/3401128553,v1:172.21.15.188:6837/3401128553] conn(0x7fa2ac066590 0x7fa2ac068a50 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7fa2b40af000 tx=0x7fa2b40a7000 comp rx=0 tx=0).stop 2023-11-26T07:23:20.834 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:20.587+0000 7fa2d1565640 1 -- 172.21.15.143:0/734070627 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fa2cc143ac0 msgr2=0x7fa2cc15d1e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:23:20.834 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:20.587+0000 7fa2d1565640 1 --2- 172.21.15.143:0/734070627 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fa2cc143ac0 0x7fa2cc15d1e0 secure :-1 s=READY pgs=1098 cs=0 l=1 rev1=1 crypto rx=0x7fa2c400c780 tx=0x7fa2c400cc40 comp rx=0 tx=0).stop 2023-11-26T07:23:20.834 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:20.587+0000 7fa2d1565640 1 -- 172.21.15.143:0/734070627 shutdown_connections 2023-11-26T07:23:20.834 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:20.587+0000 7fa2d1565640 1 --2- 172.21.15.143:0/734070627 >> [v2:172.21.15.188:6836/3401128553,v1:172.21.15.188:6837/3401128553] conn(0x7fa2ac066590 0x7fa2ac068a50 unknown :-1 s=CLOSED pgs=39 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:23:20.834 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:20.587+0000 7fa2d1565640 1 --2- 172.21.15.143:0/734070627 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fa2cc143ac0 0x7fa2cc15d1e0 unknown :-1 s=CLOSED pgs=1098 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:23:20.834 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:20.587+0000 7fa2d1565640 1 --2- 172.21.15.143:0/734070627 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fa2cc0cf990 0x7fa2cc15cca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:23:20.834 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:20.587+0000 7fa2d1565640 1 --2- 172.21.15.143:0/734070627 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fa2cc0708b0 0x7fa2cc0c40c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:23:20.834 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:20.587+0000 7fa2d1565640 1 -- 172.21.15.143:0/734070627 >> 172.21.15.143:0/734070627 conn(0x7fa2cc06a890 msgr2=0x7fa2cc0b2490 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:23:20.835 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:20.587+0000 7fa2d1565640 1 -- 172.21.15.143:0/734070627 shutdown_connections 2023-11-26T07:23:20.835 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:20.587+0000 7fa2d1565640 1 -- 172.21.15.143:0/734070627 wait complete. 2023-11-26T07:23:20.835 INFO:tasks.cephfs_test_runner:test_mgr_eviction (tasks.cephfs.test_volumes.TestMisc) ... ok 2023-11-26T07:23:20.835 INFO:tasks.cephfs_test_runner:Starting test: test_names_can_only_be_goodchars (tasks.cephfs.test_volumes.TestMisc) Test the creating vols, subvols subvolgroups fails when their names uses 2023-11-26T07:23:20.835 INFO:tasks.cephfs_test_runner:test_names_can_only_be_goodchars (tasks.cephfs.test_volumes.TestMisc) 2023-11-26T07:23:20.836 DEBUG:teuthology.orchestra.run.smithi143:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-11-26T07:23:20.979 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:20.975+0000 7f4892fe7640 1 Processor -- start 2023-11-26T07:23:20.979 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:20.975+0000 7f4892fe7640 1 -- start start 2023-11-26T07:23:20.979 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:20.975+0000 7f4892fe7640 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7f488c15d6e0 0x7f488c0d0df0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:23:20.980 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:20.975+0000 7f4892fe7640 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7f488c0b2160 0x7f488c0d14b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:23:20.980 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:20.975+0000 7f4892fe7640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f488c0d19f0 0x7f488c0cfbf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:23:20.980 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:20.975+0000 7f4892fe7640 1 -- --> v2:172.21.15.143:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f488c15dac0 con 0x7f488c15d6e0 2023-11-26T07:23:20.980 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:20.975+0000 7f4892fe7640 1 -- --> v2:172.21.15.188:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f488c0d1e80 con 0x7f488c0b2160 2023-11-26T07:23:20.980 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:20.975+0000 7f4892fe7640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f488c0cf0e0 con 0x7f488c0d19f0 2023-11-26T07:23:20.981 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:20.975+0000 7f4890d83640 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7f488c15d6e0 0x7f488c0d0df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:23:20.981 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:20.975+0000 7f4890d83640 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7f488c15d6e0 0x7f488c0d0df0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:40974/0 (socket says 172.21.15.143:40974) 2023-11-26T07:23:20.981 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:20.975+0000 7f4890d83640 1 -- 172.21.15.143:0/789404238 learned_addr learned my addr 172.21.15.143:0/789404238 (peer_addr_for_me v2:172.21.15.143:0/0) 2023-11-26T07:23:20.981 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:20.975+0000 7f4890d83640 1 -- 172.21.15.143:0/789404238 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f488c0d19f0 msgr2=0x7f488c0cfbf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-26T07:23:20.981 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:20.975+0000 7f4890d83640 1 --2- 172.21.15.143:0/789404238 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f488c0d19f0 0x7f488c0cfbf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:23:20.981 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:20.975+0000 7f4890d83640 1 -- 172.21.15.143:0/789404238 >> v2:172.21.15.188:3300/0 conn(0x7f488c0b2160 msgr2=0x7f488c0d14b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-26T07:23:20.981 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:20.975+0000 7f4883fff640 1 --2- 172.21.15.143:0/789404238 >> v2:172.21.15.188:3300/0 conn(0x7f488c0b2160 0x7f488c0d14b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:23:20.982 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:20.975+0000 7f4890d83640 1 --2- 172.21.15.143:0/789404238 >> v2:172.21.15.188:3300/0 conn(0x7f488c0b2160 0x7f488c0d14b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:23:20.982 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:20.975+0000 7f4890d83640 1 -- 172.21.15.143:0/789404238 --> v2:172.21.15.143:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f488c0ceb20 con 0x7f488c15d6e0 2023-11-26T07:23:20.983 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:20.979+0000 7f4890d83640 1 --2- 172.21.15.143:0/789404238 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f488c15d6e0 0x7f488c0d0df0 secure :-1 s=READY pgs=1099 cs=0 l=1 rev1=1 crypto rx=0x7f488c0ca2a0 tx=0x7f488801c240 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b43e4a41fa040327 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:23:20.983 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:20.979+0000 7f48837fe640 1 -- 172.21.15.143:0/789404238 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f48880ab600 con 0x7f488c15d6e0 2023-11-26T07:23:20.983 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:20.979+0000 7f48837fe640 1 -- 172.21.15.143:0/789404238 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f48880a7050 con 0x7f488c15d6e0 2023-11-26T07:23:20.984 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:20.979+0000 7f48837fe640 1 -- 172.21.15.143:0/789404238 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f48880ab980 con 0x7f488c15d6e0 2023-11-26T07:23:20.984 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:20.979+0000 7f4892fe7640 1 -- 172.21.15.143:0/789404238 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f488c15d6e0 msgr2=0x7f488c0d0df0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:23:20.984 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:20.979+0000 7f4892fe7640 1 --2- 172.21.15.143:0/789404238 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f488c15d6e0 0x7f488c0d0df0 secure :-1 s=READY pgs=1099 cs=0 l=1 rev1=1 crypto rx=0x7f488c0ca2a0 tx=0x7f488801c240 comp rx=0 tx=0).stop 2023-11-26T07:23:20.984 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:20.979+0000 7f4892fe7640 1 -- 172.21.15.143:0/789404238 shutdown_connections 2023-11-26T07:23:20.984 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:20.979+0000 7f4892fe7640 1 --2- 172.21.15.143:0/789404238 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f488c0d19f0 0x7f488c0cfbf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:23:20.984 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:20.979+0000 7f4892fe7640 1 --2- 172.21.15.143:0/789404238 >> v2:172.21.15.188:3300/0 conn(0x7f488c0b2160 0x7f488c0d14b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:23:20.985 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:20.979+0000 7f4892fe7640 1 --2- 172.21.15.143:0/789404238 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f488c15d6e0 0x7f488c0d0df0 unknown :-1 s=CLOSED pgs=1099 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:23:20.985 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:20.979+0000 7f4892fe7640 1 -- 172.21.15.143:0/789404238 >> 172.21.15.143:0/789404238 conn(0x7f488c09c550 msgr2=0x7f488c0d3220 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:23:20.985 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:20.979+0000 7f4892fe7640 1 -- 172.21.15.143:0/789404238 shutdown_connections 2023-11-26T07:23:20.985 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:20.979+0000 7f4892fe7640 1 -- 172.21.15.143:0/789404238 wait complete. 2023-11-26T07:23:20.986 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:20.983+0000 7f4892fe7640 1 Processor -- start 2023-11-26T07:23:20.986 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:20.983+0000 7f4892fe7640 1 -- start start 2023-11-26T07:23:20.989 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:20.983+0000 7f4892fe7640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f488c0b2160 0x7f488c0b7f40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:23:20.989 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:20.983+0000 7f4892fe7640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f488c0d19f0 0x7f488c159420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:23:20.990 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:20.983+0000 7f4892fe7640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f488c15d6e0 0x7f488c154a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:23:20.990 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:20.983+0000 7f4892fe7640 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f488c159c50 con 0x7f488c15d6e0 2023-11-26T07:23:20.990 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:20.983+0000 7f4892fe7640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f488c159f30 con 0x7f488c0b2160 2023-11-26T07:23:20.990 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:20.983+0000 7f4892fe7640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f488c15a210 con 0x7f488c0d19f0 2023-11-26T07:23:20.990 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:20.983+0000 7f4891584640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f488c15d6e0 0x7f488c154a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:23:20.990 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:20.983+0000 7f4891584640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f488c15d6e0 0x7f488c154a70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:40990/0 (socket says 172.21.15.143:40990) 2023-11-26T07:23:20.990 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:20.983+0000 7f4891584640 1 -- 172.21.15.143:0/86031361 learned_addr learned my addr 172.21.15.143:0/86031361 (peer_addr_for_me v2:172.21.15.143:0/0) 2023-11-26T07:23:20.991 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:20.983+0000 7f4883fff640 1 --2- 172.21.15.143:0/86031361 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f488c0d19f0 0x7f488c159420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:23:20.991 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:20.983+0000 7f4890d83640 1 --2- 172.21.15.143:0/86031361 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f488c0b2160 0x7f488c0b7f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:23:20.991 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:20.983+0000 7f4891584640 1 -- 172.21.15.143:0/86031361 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f488c0d19f0 msgr2=0x7f488c159420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:23:20.991 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:20.983+0000 7f4891584640 1 --2- 172.21.15.143:0/86031361 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f488c0d19f0 0x7f488c159420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:23:20.991 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:20.983+0000 7f4891584640 1 -- 172.21.15.143:0/86031361 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f488c0b2160 msgr2=0x7f488c0b7f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:23:20.991 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:20.983+0000 7f4891584640 1 --2- 172.21.15.143:0/86031361 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f488c0b2160 0x7f488c0b7f40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:23:20.991 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:20.983+0000 7f4891584640 1 -- 172.21.15.143:0/86031361 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f48880094d0 con 0x7f488c15d6e0 2023-11-26T07:23:20.992 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:20.983+0000 7f4891584640 1 --2- 172.21.15.143:0/86031361 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f488c15d6e0 0x7f488c154a70 secure :-1 s=READY pgs=1100 cs=0 l=1 rev1=1 crypto rx=0x7f488400b970 tx=0x7f488400be30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:23:20.992 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:20.983+0000 7f48817fa640 1 -- 172.21.15.143:0/86031361 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f488400cb40 con 0x7f488c15d6e0 2023-11-26T07:23:20.992 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:20.983+0000 7f48817fa640 1 -- 172.21.15.143:0/86031361 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f488400ccc0 con 0x7f488c15d6e0 2023-11-26T07:23:20.992 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:20.983+0000 7f4892fe7640 1 -- 172.21.15.143:0/86031361 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f488c15a4f0 con 0x7f488c15d6e0 2023-11-26T07:23:20.992 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:20.983+0000 7f48817fa640 1 -- 172.21.15.143:0/86031361 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4884014680 con 0x7f488c15d6e0 2023-11-26T07:23:20.992 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:20.983+0000 7f4892fe7640 1 -- 172.21.15.143:0/86031361 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f488c159960 con 0x7f488c15d6e0 2023-11-26T07:23:20.992 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:20.987+0000 7f4892fe7640 1 -- 172.21.15.143:0/86031361 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f488c15aa10 con 0x7f488c15d6e0 2023-11-26T07:23:20.993 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:20.987+0000 7f48817fa640 1 -- 172.21.15.143:0/86031361 <== mon.0 v2:172.21.15.143:3300/0 4 ==== mgrmap(e 16) v1 ==== 85954+0+0 (secure 0 0 0) 0x7f488401d3b0 con 0x7f488c15d6e0 2023-11-26T07:23:20.993 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:20.987+0000 7f48817fa640 1 --2- 172.21.15.143:0/86031361 >> [v2:172.21.15.188:6836/3401128553,v1:172.21.15.188:6837/3401128553] conn(0x7f4878066590 0x7f4878068a50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:23:20.993 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:20.987+0000 7f48817fa640 1 -- 172.21.15.143:0/86031361 <== mon.0 v2:172.21.15.143:3300/0 5 ==== osd_map(75..75 src has 1..75) v4 ==== 5502+0+0 (secure 0 0 0) 0x7f4884083f10 con 0x7f488c15d6e0 2023-11-26T07:23:20.993 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:20.987+0000 7f4890d83640 1 --2- 172.21.15.143:0/86031361 >> [v2:172.21.15.188:6836/3401128553,v1:172.21.15.188:6837/3401128553] conn(0x7f4878066590 0x7f4878068a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:23:20.993 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:20.987+0000 7f4890d83640 1 --2- 172.21.15.143:0/86031361 >> [v2:172.21.15.188:6836/3401128553,v1:172.21.15.188:6837/3401128553] conn(0x7f4878066590 0x7f4878068a50 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7f48880ad000 tx=0x7f4888008040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:23:20.993 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:20.991+0000 7f48817fa640 1 -- 172.21.15.143:0/86031361 <== mon.0 v2:172.21.15.143:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172649 (secure 0 0 0) 0x7f488c15aa10 con 0x7f488c15d6e0 2023-11-26T07:23:21.142 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:21.139+0000 7f4892fe7640 1 -- 172.21.15.143:0/86031361 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f488c15aa10 con 0x7f488c15d6e0 2023-11-26T07:23:21.143 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:21.143+0000 7f48817fa640 1 -- 172.21.15.143:0/86031361 <== mon.0 v2:172.21.15.143:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v75) v1 ==== 74+0+13845 (secure 0 0 0) 0x7f488c15aa10 con 0x7f488c15d6e0 2023-11-26T07:23:21.143 INFO:teuthology.orchestra.run.smithi143.stdout: 2023-11-26T07:23:21.143 INFO:teuthology.orchestra.run.smithi143.stdout:{"epoch":75,"fsid":"f5033060-2469-4a07-95fe-2b76f5404bc1","created":"2023-11-26T07:17:28.613663+0000","modified":"2023-11-26T07:23:19.576553+0000","last_up_change":"2023-11-26T07:17:41.157813+0000","last_in_change":"2023-11-26T07:17:31.181658+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":11,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-11-26T07:17:41.758597+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"ebbee2c6-023e-44b9-9f1d-6f7bfd309730","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":69,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6817","nonce":639807369},{"type":"v1","addr":"172.21.15.143:6819","nonce":639807369}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6821","nonce":639807369},{"type":"v1","addr":"172.21.15.143:6823","nonce":639807369}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6830","nonce":639807369},{"type":"v1","addr":"172.21.15.143:6831","nonce":639807369}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6825","nonce":639807369},{"type":"v1","addr":"172.21.15.143:6828","nonce":639807369}]},"public_addr":"172.21.15.143:6819/639807369","cluster_addr":"172.21.15.143:6823/639807369","heartbeat_back_addr":"172.21.15.143:6831/639807369","heartbeat_front_addr":"172.21.15.143:6828/639807369","state":["exists","up"]},{"osd":1,"uuid":"fec4b8be-7122-49e9-9a19-5850a0700001","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":69,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6816","nonce":1144942530},{"type":"v1","addr":"172.21.15.143:6818","nonce":1144942530}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6820","nonce":1144942530},{"type":"v1","addr":"172.21.15.143:6822","nonce":1144942530}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6827","nonce":1144942530},{"type":"v1","addr":"172.21.15.143:6829","nonce":1144942530}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6824","nonce":1144942530},{"type":"v1","addr":"172.21.15.143:6826","nonce":1144942530}]},"public_addr":"172.21.15.143:6818/1144942530","cluster_addr":"172.21.15.143:6822/1144942530","heartbeat_back_addr":"172.21.15.143:6829/1144942530","heartbeat_front_addr":"172.21.15.143:6826/1144942530","state":["exists","up"]},{"osd":2,"uuid":"2888af5b-f933-46ca-8f95-6dd494ff43f6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":69,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6808","nonce":3909095186},{"type":"v1","addr":"172.21.15.143:6809","nonce":3909095186}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6810","nonce":3909095186},{"type":"v1","addr":"172.21.15.143:6811","nonce":3909095186}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6814","nonce":3909095186},{"type":"v1","addr":"172.21.15.143:6815","nonce":3909095186}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6812","nonce":3909095186},{"type":"v1","addr":"172.21.15.143:6813","nonce":3909095186}]},"public_addr":"172.21.15.143:6809/3909095186","cluster_addr":"172.21.15.143:6811/3909095186","heartbeat_back_addr":"172.21.15.143:6815/3909095186","heartbeat_front_addr":"172.21.15.143:6813/3909095186","state":["exists","up"]},{"osd":3,"uuid":"d1f52435-7bc1-4081-9550-9cd1ba55f549","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":69,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6800","nonce":3854210991},{"type":"v1","addr":"172.21.15.143:6801","nonce":3854210991}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6802","nonce":3854210991},{"type":"v1","addr":"172.21.15.143:6803","nonce":3854210991}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6806","nonce":3854210991},{"type":"v1","addr":"172.21.15.143:6807","nonce":3854210991}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6804","nonce":3854210991},{"type":"v1","addr":"172.21.15.143:6805","nonce":3854210991}]},"public_addr":"172.21.15.143:6801/3854210991","cluster_addr":"172.21.15.143:6803/3854210991","heartbeat_back_addr":"172.21.15.143:6807/3854210991","heartbeat_front_addr":"172.21.15.143:6805/3854210991","state":["exists","up"]},{"osd":4,"uuid":"e2ae8b6e-6e25-49b2-bcbf-aacce68f06a3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":69,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6800","nonce":3924606902},{"type":"v1","addr":"172.21.15.188:6801","nonce":3924606902}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6802","nonce":3924606902},{"type":"v1","addr":"172.21.15.188:6803","nonce":3924606902}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6806","nonce":3924606902},{"type":"v1","addr":"172.21.15.188:6807","nonce":3924606902}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6804","nonce":3924606902},{"type":"v1","addr":"172.21.15.188:6805","nonce":3924606902}]},"public_addr":"172.21.15.188:6801/3924606902","cluster_addr":"172.21.15.188:6803/3924606902","heartbeat_back_addr":"172.21.15.188:6807/3924606902","heartbeat_front_addr":"172.21.15.188:6805/3924606902","state":["exists","up"]},{"osd":5,"uuid":"3cec8a32-7698-4c36-8e89-e4ba11934f33","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":69,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6813","nonce":186841544},{"type":"v1","addr":"172.21.15.188:6815","nonce":186841544}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6817","nonce":186841544},{"type":"v1","addr":"172.21.15.188:6818","nonce":186841544}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6822","nonce":186841544},{"type":"v1","addr":"172.21.15.188:6823","nonce":186841544}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6820","nonce":186841544},{"type":"v1","addr":"172.21.15.188:6821","nonce":186841544}]},"public_addr":"172.21.15.188:6815/186841544","cluster_addr":"172.21.15.188:6818/186841544","heartbeat_back_addr":"172.21.15.188:6823/186841544","heartbeat_front_addr":"172.21.15.188:6821/186841544","state":["exists","up"]},{"osd":6,"uuid":"58a65d5d-e090-473a-81b4-ad38e85029f9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":69,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6808","nonce":2153499405},{"type":"v1","addr":"172.21.15.188:6809","nonce":2153499405}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6810","nonce":2153499405},{"type":"v1","addr":"172.21.15.188:6811","nonce":2153499405}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6816","nonce":2153499405},{"type":"v1","addr":"172.21.15.188:6819","nonce":2153499405}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6812","nonce":2153499405},{"type":"v1","addr":"172.21.15.188:6814","nonce":2153499405}]},"public_addr":"172.21.15.188:6809/2153499405","cluster_addr":"172.21.15.188:6811/2153499405","heartbeat_back_addr":"172.21.15.188:6819/2153499405","heartbeat_front_addr":"172.21.15.188:6814/2153499405","state":["exists","up"]},{"osd":7,"uuid":"ea5b7f70-3554-4cf8-ac30-f7c59b80678d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":69,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6824","nonce":2722096729},{"type":"v1","addr":"172.21.15.188:6825","nonce":2722096729}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6826","nonce":2722096729},{"type":"v1","addr":"172.21.15.188:6827","nonce":2722096729}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6830","nonce":2722096729},{"type":"v1","addr":"172.21.15.188:6831","nonce":2722096729}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6828","nonce":2722096729},{"type":"v1","addr":"172.21.15.188:6829","nonce":2722096729}]},"public_addr":"172.21.15.188:6825/2722096729","cluster_addr":"172.21.15.188:6827/2722096729","heartbeat_back_addr":"172.21.15.188:6831/2722096729","heartbeat_front_addr":"172.21.15.188:6829/2722096729","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-26T07:17:36.995032+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-26T07:17:36.932266+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-26T07:17:36.829304+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-26T07:17:36.804297+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-26T07:17:36.667892+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-26T07:17:36.964868+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-26T07:17:36.827332+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-26T07:17:36.893179+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.143:6836/2590663349":"2023-11-27T07:23:16.239806+0000","172.21.15.143:6834/2590663349":"2023-11-27T07:23:16.239806+0000","172.21.15.143:0/103646748":"2023-11-27T07:23:11.261937+0000","172.21.15.143:0/631097144":"2023-11-27T07:23:11.261937+0000","172.21.15.143:0/2591285564":"2023-11-27T07:23:11.261937+0000","172.21.15.143:0/1844383386":"2023-11-27T07:23:11.261937+0000","172.21.15.143:0/3813484262":"2023-11-27T07:23:11.261937+0000","172.21.15.143:0/3751308865":"2023-11-27T07:23:11.261937+0000","172.21.15.143:0/2346893898":"2023-11-27T07:23:11.261937+0000","172.21.15.143:6832/2150557544":"2023-11-27T07:23:11.261937+0000","172.21.15.143:6833/2150557544":"2023-11-27T07:23:11.261937+0000","172.21.15.143:0/963510909":"2023-11-27T07:23:11.261937+0000","172.21.15.143:0/3516040872":"2023-11-27T07:23:11.261937+0000","172.21.15.143:0/253926057":"2023-11-27T07:23:11.261937+0000","172.21.15.143:0/248180418":"2023-11-27T07:23:11.261937+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-11-26T07:23:21.145 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:21.143+0000 7f4892fe7640 1 -- 172.21.15.143:0/86031361 >> [v2:172.21.15.188:6836/3401128553,v1:172.21.15.188:6837/3401128553] conn(0x7f4878066590 msgr2=0x7f4878068a50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:23:21.145 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:21.143+0000 7f4892fe7640 1 --2- 172.21.15.143:0/86031361 >> [v2:172.21.15.188:6836/3401128553,v1:172.21.15.188:6837/3401128553] conn(0x7f4878066590 0x7f4878068a50 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7f48880ad000 tx=0x7f4888008040 comp rx=0 tx=0).stop 2023-11-26T07:23:21.145 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:21.143+0000 7f4892fe7640 1 -- 172.21.15.143:0/86031361 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f488c15d6e0 msgr2=0x7f488c154a70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:23:21.145 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:21.143+0000 7f4892fe7640 1 --2- 172.21.15.143:0/86031361 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f488c15d6e0 0x7f488c154a70 secure :-1 s=READY pgs=1100 cs=0 l=1 rev1=1 crypto rx=0x7f488400b970 tx=0x7f488400be30 comp rx=0 tx=0).stop 2023-11-26T07:23:21.145 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:21.143+0000 7f4892fe7640 1 -- 172.21.15.143:0/86031361 shutdown_connections 2023-11-26T07:23:21.145 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:21.143+0000 7f4892fe7640 1 --2- 172.21.15.143:0/86031361 >> [v2:172.21.15.188:6836/3401128553,v1:172.21.15.188:6837/3401128553] conn(0x7f4878066590 0x7f4878068a50 unknown :-1 s=CLOSED pgs=40 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:23:21.146 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:21.143+0000 7f4892fe7640 1 --2- 172.21.15.143:0/86031361 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f488c15d6e0 0x7f488c154a70 unknown :-1 s=CLOSED pgs=1100 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:23:21.146 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:21.143+0000 7f4892fe7640 1 --2- 172.21.15.143:0/86031361 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f488c0d19f0 0x7f488c159420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:23:21.146 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:21.143+0000 7f4892fe7640 1 --2- 172.21.15.143:0/86031361 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f488c0b2160 0x7f488c0b7f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:23:21.146 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:21.143+0000 7f4892fe7640 1 -- 172.21.15.143:0/86031361 >> 172.21.15.143:0/86031361 conn(0x7f488c09c550 msgr2=0x7f488c06ba60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:23:21.146 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:21.143+0000 7f4892fe7640 1 -- 172.21.15.143:0/86031361 shutdown_connections 2023-11-26T07:23:21.146 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:21.143+0000 7f4892fe7640 1 -- 172.21.15.143:0/86031361 wait complete. 2023-11-26T07:23:21.164 INFO:tasks.ceph_test_case.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-11-26T07:17:41.758597+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2023-11-26T07:23:21.164 DEBUG:teuthology.orchestra.run.smithi143:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-11-26T07:23:21.305 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:21.303+0000 7f46ab832640 1 Processor -- start 2023-11-26T07:23:21.305 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:21.303+0000 7f46ab832640 1 -- start start 2023-11-26T07:23:21.306 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:21.303+0000 7f46ab832640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f46a414b1e0 0x7f46a40cf280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:23:21.307 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:21.303+0000 7f46ab832640 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7f46a4070fd0 0x7f46a4162510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:23:21.307 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:21.303+0000 7f46ab832640 1 -- --> v1:172.21.15.143:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f46a40cfc20 con 0x7f46a40cf7c0 2023-11-26T07:23:21.307 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:21.303+0000 7f46ab832640 1 -- --> v2:172.21.15.143:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f46a4162080 con 0x7f46a4070fd0 2023-11-26T07:23:21.307 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:21.303+0000 7f46ab832640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f46a4161d70 con 0x7f46a414b1e0 2023-11-26T07:23:21.307 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:21.303+0000 7f46a9dcf640 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7f46a4070fd0 0x7f46a4162510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:23:21.307 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:21.303+0000 7f46a8dcd640 1 --1- >> v1:172.21.15.143:6789/0 conn(0x7f46a40cf7c0 0x7f46a40708c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.143:6789/0 says I am v1:172.21.15.143:36748/0 (socket says 172.21.15.143:36748) 2023-11-26T07:23:21.308 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:21.303+0000 7f46a8dcd640 1 -- 172.21.15.143:0/2569419999 learned_addr learned my addr 172.21.15.143:0/2569419999 (peer_addr_for_me v1:172.21.15.143:0/0) 2023-11-26T07:23:21.308 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:21.307+0000 7f46a95ce640 1 --2- 172.21.15.143:0/2569419999 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f46a414b1e0 0x7f46a40cf280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:23:21.308 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:21.307+0000 7f46a9dcf640 1 -- 172.21.15.143:0/2569419999 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f46a414b1e0 msgr2=0x7f46a40cf280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-26T07:23:21.308 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:21.307+0000 7f46a9dcf640 1 --2- 172.21.15.143:0/2569419999 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f46a414b1e0 0x7f46a40cf280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:23:21.309 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:21.307+0000 7f46a9dcf640 1 -- 172.21.15.143:0/2569419999 >> v1:172.21.15.143:6789/0 conn(0x7f46a40cf7c0 legacy=0x7f46a40708c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:23:21.309 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:21.307+0000 7f469bfff640 1 -- 172.21.15.143:0/2569419999 <== mon.0 v1:172.21.15.143:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2570843940 0 0) 0x7f46a40cfc20 con 0x7f46a40cf7c0 2023-11-26T07:23:21.309 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:21.307+0000 7f46a9dcf640 1 -- 172.21.15.143:0/2569419999 --> v2:172.21.15.143:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f46a414b5c0 con 0x7f46a4070fd0 2023-11-26T07:23:21.309 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:21.307+0000 7f46a95ce640 1 --2- 172.21.15.143:0/2569419999 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f46a414b1e0 0x7f46a40cf280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-26T07:23:21.310 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:21.307+0000 7f46a9dcf640 1 --2- 172.21.15.143:0/2569419999 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f46a4070fd0 0x7f46a4162510 secure :-1 s=READY pgs=1102 cs=0 l=1 rev1=1 crypto rx=0x7f469c009d90 tx=0x7f469c01c920 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2ead02580639ffc server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:23:21.310 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:21.307+0000 7f469bfff640 1 -- 172.21.15.143:0/2569419999 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f469c0a59e0 con 0x7f46a4070fd0 2023-11-26T07:23:21.310 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:21.307+0000 7f469bfff640 1 -- 172.21.15.143:0/2569419999 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f469c0a5b40 con 0x7f46a4070fd0 2023-11-26T07:23:21.310 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:21.307+0000 7f469bfff640 1 -- 172.21.15.143:0/2569419999 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f469c0ad440 con 0x7f46a4070fd0 2023-11-26T07:23:21.311 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:21.311+0000 7f46ab832640 1 -- 172.21.15.143:0/2569419999 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f46a4070fd0 msgr2=0x7f46a4162510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:23:21.311 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:21.311+0000 7f46ab832640 1 --2- 172.21.15.143:0/2569419999 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f46a4070fd0 0x7f46a4162510 secure :-1 s=READY pgs=1102 cs=0 l=1 rev1=1 crypto rx=0x7f469c009d90 tx=0x7f469c01c920 comp rx=0 tx=0).stop 2023-11-26T07:23:21.312 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:21.311+0000 7f46ab832640 1 -- 172.21.15.143:0/2569419999 shutdown_connections 2023-11-26T07:23:21.312 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:21.311+0000 7f46ab832640 1 --2- 172.21.15.143:0/2569419999 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f46a4070fd0 0x7f46a4162510 unknown :-1 s=CLOSED pgs=1102 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:23:21.312 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:21.311+0000 7f46ab832640 1 --2- 172.21.15.143:0/2569419999 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f46a414b1e0 0x7f46a40cf280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:23:21.312 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:21.311+0000 7f46ab832640 1 -- 172.21.15.143:0/2569419999 >> 172.21.15.143:0/2569419999 conn(0x7f46a406a890 msgr2=0x7f46a406aca0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:23:21.312 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:21.311+0000 7f46ab832640 1 -- 172.21.15.143:0/2569419999 shutdown_connections 2023-11-26T07:23:21.312 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:21.311+0000 7f46ab832640 1 -- 172.21.15.143:0/2569419999 wait complete. 2023-11-26T07:23:21.313 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:21.311+0000 7f46ab832640 1 Processor -- start 2023-11-26T07:23:21.313 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:21.311+0000 7f46ab832640 1 -- start start 2023-11-26T07:23:21.314 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:21.311+0000 7f46ab832640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f46a4070fd0 0x7f46a414e5f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:23:21.315 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:21.311+0000 7f46ab832640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f46a40cf7c0 0x7f46a415faf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:23:21.315 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:21.311+0000 7f46ab832640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f46a414b1e0 0x7f46a4149870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:23:21.315 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:21.311+0000 7f46ab832640 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f46a4160030 con 0x7f46a4070fd0 2023-11-26T07:23:21.315 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:21.311+0000 7f46ab832640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f46a41602d0 con 0x7f46a414b1e0 2023-11-26T07:23:21.315 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:21.311+0000 7f46ab832640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f46a41605b0 con 0x7f46a40cf7c0 2023-11-26T07:23:21.315 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:21.311+0000 7f46a95ce640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f46a4070fd0 0x7f46a414e5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:23:21.316 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:21.311+0000 7f46a95ce640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f46a4070fd0 0x7f46a414e5f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:41000/0 (socket says 172.21.15.143:41000) 2023-11-26T07:23:21.316 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:21.311+0000 7f46a95ce640 1 -- 172.21.15.143:0/3448099568 learned_addr learned my addr 172.21.15.143:0/3448099568 (peer_addr_for_me v2:172.21.15.143:0/0) 2023-11-26T07:23:21.316 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:21.311+0000 7f46a8dcd640 1 --2- 172.21.15.143:0/3448099568 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f46a40cf7c0 0x7f46a415faf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:23:21.316 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:21.311+0000 7f46a9dcf640 1 --2- 172.21.15.143:0/3448099568 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f46a414b1e0 0x7f46a4149870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:23:21.316 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:21.311+0000 7f46a95ce640 1 -- 172.21.15.143:0/3448099568 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f46a40cf7c0 msgr2=0x7f46a415faf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:23:21.316 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:21.311+0000 7f46a95ce640 1 --2- 172.21.15.143:0/3448099568 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f46a40cf7c0 0x7f46a415faf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:23:21.316 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:21.311+0000 7f46a95ce640 1 -- 172.21.15.143:0/3448099568 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f46a414b1e0 msgr2=0x7f46a4149870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:23:21.317 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:21.311+0000 7f46a95ce640 1 --2- 172.21.15.143:0/3448099568 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f46a414b1e0 0x7f46a4149870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:23:21.317 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:21.311+0000 7f46a95ce640 1 -- 172.21.15.143:0/3448099568 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f469c009a70 con 0x7f46a4070fd0 2023-11-26T07:23:21.317 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:21.315+0000 7f46a95ce640 1 --2- 172.21.15.143:0/3448099568 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f46a4070fd0 0x7f46a414e5f0 secure :-1 s=READY pgs=1103 cs=0 l=1 rev1=1 crypto rx=0x7f46a000c7c0 tx=0x7f46a000cc80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:23:21.318 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:21.315+0000 7f4699ffb640 1 -- 172.21.15.143:0/3448099568 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f46a0007a30 con 0x7f46a4070fd0 2023-11-26T07:23:21.318 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:21.315+0000 7f4699ffb640 1 -- 172.21.15.143:0/3448099568 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f46a0007b90 con 0x7f46a4070fd0 2023-11-26T07:23:21.318 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:21.315+0000 7f4699ffb640 1 -- 172.21.15.143:0/3448099568 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f46a000ce70 con 0x7f46a4070fd0 2023-11-26T07:23:21.319 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:21.315+0000 7f46ab832640 1 -- 172.21.15.143:0/3448099568 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f46a4160890 con 0x7f46a4070fd0 2023-11-26T07:23:21.319 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:21.315+0000 7f46ab832640 1 -- 172.21.15.143:0/3448099568 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f46a4149db0 con 0x7f46a4070fd0 2023-11-26T07:23:21.319 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:21.315+0000 7f46ab832640 1 -- 172.21.15.143:0/3448099568 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f46a414b840 con 0x7f46a4070fd0 2023-11-26T07:23:21.319 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:21.315+0000 7f4699ffb640 1 -- 172.21.15.143:0/3448099568 <== mon.0 v2:172.21.15.143:3300/0 4 ==== mgrmap(e 16) v1 ==== 85954+0+0 (secure 0 0 0) 0x7f46a00113f0 con 0x7f46a4070fd0 2023-11-26T07:23:21.319 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:21.315+0000 7f4699ffb640 1 --2- 172.21.15.143:0/3448099568 >> [v2:172.21.15.188:6836/3401128553,v1:172.21.15.188:6837/3401128553] conn(0x7f4688066590 0x7f4688068a50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:23:21.320 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:21.315+0000 7f4699ffb640 1 -- 172.21.15.143:0/3448099568 <== mon.0 v2:172.21.15.143:3300/0 5 ==== osd_map(75..75 src has 1..75) v4 ==== 5502+0+0 (secure 0 0 0) 0x7f46a0084380 con 0x7f46a4070fd0 2023-11-26T07:23:21.320 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:21.315+0000 7f46a8dcd640 1 --2- 172.21.15.143:0/3448099568 >> [v2:172.21.15.188:6836/3401128553,v1:172.21.15.188:6837/3401128553] conn(0x7f4688066590 0x7f4688068a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:23:21.320 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:21.319+0000 7f46a8dcd640 1 --2- 172.21.15.143:0/3448099568 >> [v2:172.21.15.188:6836/3401128553,v1:172.21.15.188:6837/3401128553] conn(0x7f4688066590 0x7f4688068a50 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7f46a40ae600 tx=0x7f4694007d60 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:23:21.322 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:21.319+0000 7f4699ffb640 1 -- 172.21.15.143:0/3448099568 <== mon.0 v2:172.21.15.143:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172649 (secure 0 0 0) 0x7f46a00517f0 con 0x7f46a4070fd0 2023-11-26T07:23:21.473 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:21.471+0000 7f46ab832640 1 -- 172.21.15.143:0/3448099568 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f46a4067680 con 0x7f46a4070fd0 2023-11-26T07:23:21.474 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:21.471+0000 7f4699ffb640 1 -- 172.21.15.143:0/3448099568 <== mon.0 v2:172.21.15.143:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v75) v1 ==== 93+0+10 (secure 0 0 0) 0x7f46a0054e10 con 0x7f46a4070fd0 2023-11-26T07:23:21.474 INFO:teuthology.orchestra.run.smithi143.stdout:pg_num: 1 2023-11-26T07:23:21.476 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:21.475+0000 7f46ab832640 1 -- 172.21.15.143:0/3448099568 >> [v2:172.21.15.188:6836/3401128553,v1:172.21.15.188:6837/3401128553] conn(0x7f4688066590 msgr2=0x7f4688068a50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:23:21.476 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:21.475+0000 7f46ab832640 1 --2- 172.21.15.143:0/3448099568 >> [v2:172.21.15.188:6836/3401128553,v1:172.21.15.188:6837/3401128553] conn(0x7f4688066590 0x7f4688068a50 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7f46a40ae600 tx=0x7f4694007d60 comp rx=0 tx=0).stop 2023-11-26T07:23:21.476 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:21.475+0000 7f46ab832640 1 -- 172.21.15.143:0/3448099568 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f46a4070fd0 msgr2=0x7f46a414e5f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:23:21.476 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:21.475+0000 7f46ab832640 1 --2- 172.21.15.143:0/3448099568 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f46a4070fd0 0x7f46a414e5f0 secure :-1 s=READY pgs=1103 cs=0 l=1 rev1=1 crypto rx=0x7f46a000c7c0 tx=0x7f46a000cc80 comp rx=0 tx=0).stop 2023-11-26T07:23:21.476 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:21.475+0000 7f46ab832640 1 -- 172.21.15.143:0/3448099568 shutdown_connections 2023-11-26T07:23:21.477 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:21.475+0000 7f46ab832640 1 --2- 172.21.15.143:0/3448099568 >> [v2:172.21.15.188:6836/3401128553,v1:172.21.15.188:6837/3401128553] conn(0x7f4688066590 0x7f4688068a50 unknown :-1 s=CLOSED pgs=41 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:23:21.477 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:21.475+0000 7f46ab832640 1 --2- 172.21.15.143:0/3448099568 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f46a414b1e0 0x7f46a4149870 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:23:21.477 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:21.475+0000 7f46ab832640 1 --2- 172.21.15.143:0/3448099568 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f46a40cf7c0 0x7f46a415faf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:23:21.477 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:21.475+0000 7f46ab832640 1 --2- 172.21.15.143:0/3448099568 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f46a4070fd0 0x7f46a414e5f0 unknown :-1 s=CLOSED pgs=1103 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:23:21.477 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:21.475+0000 7f46ab832640 1 -- 172.21.15.143:0/3448099568 >> 172.21.15.143:0/3448099568 conn(0x7f46a406a890 msgr2=0x7f46a40b0f80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:23:21.477 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:21.475+0000 7f46ab832640 1 -- 172.21.15.143:0/3448099568 shutdown_connections 2023-11-26T07:23:21.477 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:21.475+0000 7f46ab832640 1 -- 172.21.15.143:0/3448099568 wait complete. 2023-11-26T07:23:21.495 DEBUG:teuthology.orchestra.run.smithi143:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Starting test tasks.cephfs.test_volumes.TestMisc.test_names_can_only_be_goodchars' 2023-11-26T07:23:21.634 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:21.631+0000 7f8b6ae91640 1 Processor -- start 2023-11-26T07:23:21.634 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:21.631+0000 7f8b6ae91640 1 -- start start 2023-11-26T07:23:21.637 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:21.635+0000 7f8b6ae91640 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7f8b64143ac0 0x7f8b64070370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:23:21.637 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:21.635+0000 7f8b6ae91640 1 -- --> v1:172.21.15.143:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8b640cc730 con 0x7f8b640cf990 2023-11-26T07:23:21.637 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:21.635+0000 7f8b6ae91640 1 -- --> v1:172.21.15.188:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8b64070ee0 con 0x7f8b640708b0 2023-11-26T07:23:21.637 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:21.635+0000 7f8b6ae91640 1 -- --> v2:172.21.15.188:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8b64143ea0 con 0x7f8b64143ac0 2023-11-26T07:23:21.638 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:21.635+0000 7f8b6942e640 1 --1- >> v1:172.21.15.143:6789/0 conn(0x7f8b640cf990 0x7f8b640cd1b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.143:6789/0 says I am v1:172.21.15.143:36762/0 (socket says 172.21.15.143:36762) 2023-11-26T07:23:21.638 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:21.635+0000 7f8b6942e640 1 -- 172.21.15.143:0/438056105 learned_addr learned my addr 172.21.15.143:0/438056105 (peer_addr_for_me v1:172.21.15.143:0/0) 2023-11-26T07:23:21.638 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:21.635+0000 7f8b68c2d640 1 --2- 172.21.15.143:0/438056105 >> v2:172.21.15.188:3300/0 conn(0x7f8b64143ac0 0x7f8b64070370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:23:21.638 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:21.635+0000 7f8b637fe640 1 -- 172.21.15.143:0/438056105 <== mon.0 v1:172.21.15.143:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4232876429 0 0) 0x7f8b640cc730 con 0x7f8b640cf990 2023-11-26T07:23:21.638 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:21.635+0000 7f8b637fe640 1 -- 172.21.15.143:0/438056105 --> v1:172.21.15.143:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f8b54003530 con 0x7f8b640cf990 2023-11-26T07:23:21.638 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:21.635+0000 7f8b637fe640 1 -- 172.21.15.143:0/438056105 <== mon.0 v1:172.21.15.143:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2829983546 0 0) 0x7f8b54003530 con 0x7f8b640cf990 2023-11-26T07:23:21.639 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:21.635+0000 7f8b637fe640 1 -- 172.21.15.143:0/438056105 >> v2:172.21.15.188:3300/0 conn(0x7f8b64143ac0 msgr2=0x7f8b64070370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-26T07:23:21.639 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:21.635+0000 7f8b637fe640 1 --2- 172.21.15.143:0/438056105 >> v2:172.21.15.188:3300/0 conn(0x7f8b64143ac0 0x7f8b64070370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:23:21.639 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:21.635+0000 7f8b637fe640 1 -- 172.21.15.143:0/438056105 >> v1:172.21.15.188:6789/0 conn(0x7f8b640708b0 legacy=0x7f8b640cf280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:23:21.639 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:21.635+0000 7f8b637fe640 1 -- 172.21.15.143:0/438056105 --> v1:172.21.15.143:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8b640711c0 con 0x7f8b640cf990 2023-11-26T07:23:21.639 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:21.635+0000 7f8b637fe640 1 -- 172.21.15.143:0/438056105 <== mon.0 v1:172.21.15.143:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2137882523 0 0) 0x7f8b5c001a00 con 0x7f8b640cf990 2023-11-26T07:23:21.639 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:21.635+0000 7f8b637fe640 1 -- 172.21.15.143:0/438056105 >> v1:172.21.15.143:6789/0 conn(0x7f8b640cf990 legacy=0x7f8b640cd1b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:23:21.639 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:21.635+0000 7f8b637fe640 1 --2- 172.21.15.143:0/438056105 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f8b54003980 0x7f8b54003da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:23:21.640 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:21.635+0000 7f8b637fe640 1 --2- 172.21.15.143:0/438056105 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f8b540047d0 0x7f8b54004c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:23:21.640 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:21.635+0000 7f8b637fe640 1 --2- 172.21.15.143:0/438056105 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f8b54005190 0x7f8b5400b990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:23:21.640 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:21.635+0000 7f8b637fe640 1 -- 172.21.15.143:0/438056105 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8b5400c000 con 0x7f8b54003980 2023-11-26T07:23:21.640 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:21.635+0000 7f8b637fe640 1 -- 172.21.15.143:0/438056105 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8b5400c170 con 0x7f8b540047d0 2023-11-26T07:23:21.640 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:21.635+0000 7f8b637fe640 1 -- 172.21.15.143:0/438056105 --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8b5400c330 con 0x7f8b54005190 2023-11-26T07:23:21.640 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:21.635+0000 7f8b68c2d640 1 --2- 172.21.15.143:0/438056105 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f8b54003980 0x7f8b54003da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:23:21.641 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:21.635+0000 7f8b63fff640 1 --2- 172.21.15.143:0/438056105 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f8b540047d0 0x7f8b54004c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:23:21.641 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:21.635+0000 7f8b6942e640 1 --2- 172.21.15.143:0/438056105 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f8b54005190 0x7f8b5400b990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:23:21.641 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:21.635+0000 7f8b68c2d640 1 -- 172.21.15.143:0/438056105 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f8b54005190 msgr2=0x7f8b5400b990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-26T07:23:21.641 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:21.635+0000 7f8b68c2d640 1 --2- 172.21.15.143:0/438056105 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f8b54005190 0x7f8b5400b990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:23:21.641 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:21.635+0000 7f8b68c2d640 1 -- 172.21.15.143:0/438056105 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f8b540047d0 msgr2=0x7f8b54004c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-26T07:23:21.641 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:21.635+0000 7f8b68c2d640 1 --2- 172.21.15.143:0/438056105 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f8b540047d0 0x7f8b54004c50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:23:21.641 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:21.635+0000 7f8b68c2d640 1 -- 172.21.15.143:0/438056105 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8b54003530 con 0x7f8b54003980 2023-11-26T07:23:21.642 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:21.635+0000 7f8b6942e640 1 --2- 172.21.15.143:0/438056105 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f8b54005190 0x7f8b5400b990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-26T07:23:21.642 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:21.635+0000 7f8b6942e640 1 -- 172.21.15.143:0/438056105 reap_dead start 2023-11-26T07:23:21.643 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:21.639+0000 7f8b68c2d640 1 --2- 172.21.15.143:0/438056105 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f8b54003980 0x7f8b54003da0 secure :-1 s=READY pgs=1105 cs=0 l=1 rev1=1 crypto rx=0x7f8b58009ae0 tx=0x7f8b5801ca50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8e6fa636ce568e8c server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:23:21.643 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:21.639+0000 7f8b637fe640 1 -- 172.21.15.143:0/438056105 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8b580a5530 con 0x7f8b54003980 2023-11-26T07:23:21.644 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:21.639+0000 7f8b637fe640 1 -- 172.21.15.143:0/438056105 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8b580a5690 con 0x7f8b54003980 2023-11-26T07:23:21.644 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:21.639+0000 7f8b637fe640 1 -- 172.21.15.143:0/438056105 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8b580a5a40 con 0x7f8b54003980 2023-11-26T07:23:21.644 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:21.639+0000 7f8b6ae91640 1 -- 172.21.15.143:0/438056105 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f8b54003980 msgr2=0x7f8b54003da0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:23:21.644 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:21.639+0000 7f8b6ae91640 1 --2- 172.21.15.143:0/438056105 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f8b54003980 0x7f8b54003da0 secure :-1 s=READY pgs=1105 cs=0 l=1 rev1=1 crypto rx=0x7f8b58009ae0 tx=0x7f8b5801ca50 comp rx=0 tx=0).stop 2023-11-26T07:23:21.644 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:21.639+0000 7f8b6ae91640 1 -- 172.21.15.143:0/438056105 shutdown_connections 2023-11-26T07:23:21.644 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:21.639+0000 7f8b6ae91640 1 --2- 172.21.15.143:0/438056105 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f8b54003980 0x7f8b54003da0 unknown :-1 s=CLOSED pgs=1105 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:23:21.644 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:21.639+0000 7f8b6ae91640 1 -- 172.21.15.143:0/438056105 >> 172.21.15.143:0/438056105 conn(0x7f8b6406a890 msgr2=0x7f8b6406aca0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:23:21.645 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:21.639+0000 7f8b6ae91640 1 -- 172.21.15.143:0/438056105 shutdown_connections 2023-11-26T07:23:21.645 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:21.639+0000 7f8b6ae91640 1 -- 172.21.15.143:0/438056105 wait complete. 2023-11-26T07:23:21.645 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:21.643+0000 7f8b6ae91640 1 Processor -- start 2023-11-26T07:23:21.645 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:21.643+0000 7f8b6ae91640 1 -- start start 2023-11-26T07:23:21.647 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:21.643+0000 7f8b6ae91640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f8b54003980 0x7f8b641f6200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:23:21.647 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:21.643+0000 7f8b6ae91640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f8b6415a780 0x7f8b6415c0b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:23:21.647 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:21.643+0000 7f8b6ae91640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f8b6415c5f0 0x7f8b640b0190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:23:21.648 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:21.643+0000 7f8b6ae91640 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8b640afd00 con 0x7f8b6415c5f0 2023-11-26T07:23:21.648 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:21.643+0000 7f8b6ae91640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8b64140fc0 con 0x7f8b6415a780 2023-11-26T07:23:21.648 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:21.643+0000 7f8b6ae91640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8b6415ac00 con 0x7f8b54003980 2023-11-26T07:23:21.648 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:21.643+0000 7f8b6942e640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f8b6415c5f0 0x7f8b640b0190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:23:21.648 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:21.643+0000 7f8b6942e640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f8b6415c5f0 0x7f8b640b0190 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:41024/0 (socket says 172.21.15.143:41024) 2023-11-26T07:23:21.648 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:21.643+0000 7f8b6942e640 1 -- 172.21.15.143:0/2198908316 learned_addr learned my addr 172.21.15.143:0/2198908316 (peer_addr_for_me v2:172.21.15.143:0/0) 2023-11-26T07:23:21.648 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:21.643+0000 7f8b63fff640 1 --2- 172.21.15.143:0/2198908316 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f8b6415a780 0x7f8b6415c0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:23:21.649 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:21.643+0000 7f8b68c2d640 1 --2- 172.21.15.143:0/2198908316 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f8b54003980 0x7f8b641f6200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:23:21.649 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:21.643+0000 7f8b6942e640 1 -- 172.21.15.143:0/2198908316 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f8b54003980 msgr2=0x7f8b641f6200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:23:21.649 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:21.643+0000 7f8b6942e640 1 --2- 172.21.15.143:0/2198908316 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f8b54003980 0x7f8b641f6200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:23:21.649 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:21.643+0000 7f8b6942e640 1 -- 172.21.15.143:0/2198908316 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f8b6415a780 msgr2=0x7f8b6415c0b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:23:21.649 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:21.643+0000 7f8b6942e640 1 --2- 172.21.15.143:0/2198908316 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f8b6415a780 0x7f8b6415c0b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:23:21.649 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:21.643+0000 7f8b6942e640 1 -- 172.21.15.143:0/2198908316 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8b64140ce0 con 0x7f8b6415c5f0 2023-11-26T07:23:21.650 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:21.643+0000 7f8b63fff640 1 --2- 172.21.15.143:0/2198908316 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f8b6415a780 0x7f8b6415c0b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-26T07:23:21.650 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:21.643+0000 7f8b6942e640 1 --2- 172.21.15.143:0/2198908316 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f8b6415c5f0 0x7f8b640b0190 secure :-1 s=READY pgs=1106 cs=0 l=1 rev1=1 crypto rx=0x7f8b6406e880 tx=0x7f8b5c00e530 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:23:21.650 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:21.643+0000 7f8b617fa640 1 -- 172.21.15.143:0/2198908316 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8b5c00eb90 con 0x7f8b6415c5f0 2023-11-26T07:23:21.650 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:21.643+0000 7f8b617fa640 1 -- 172.21.15.143:0/2198908316 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8b5c00ed10 con 0x7f8b6415c5f0 2023-11-26T07:23:21.650 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:21.643+0000 7f8b6ae91640 1 -- 172.21.15.143:0/2198908316 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8b640af6b0 con 0x7f8b6415c5f0 2023-11-26T07:23:21.650 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:21.643+0000 7f8b617fa640 1 -- 172.21.15.143:0/2198908316 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8b5c013660 con 0x7f8b6415c5f0 2023-11-26T07:23:21.650 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:21.643+0000 7f8b6ae91640 1 -- 172.21.15.143:0/2198908316 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8b6415ca70 con 0x7f8b6415c5f0 2023-11-26T07:23:21.651 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:21.643+0000 7f8b6ae91640 1 -- 172.21.15.143:0/2198908316 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8b64067680 con 0x7f8b6415c5f0 2023-11-26T07:23:21.651 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:21.647+0000 7f8b617fa640 1 -- 172.21.15.143:0/2198908316 <== mon.0 v2:172.21.15.143:3300/0 4 ==== mgrmap(e 16) v1 ==== 85954+0+0 (secure 0 0 0) 0x7f8b5c0137c0 con 0x7f8b6415c5f0 2023-11-26T07:23:21.651 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:21.647+0000 7f8b617fa640 1 --2- 172.21.15.143:0/2198908316 >> [v2:172.21.15.188:6836/3401128553,v1:172.21.15.188:6837/3401128553] conn(0x7f8b48066590 0x7f8b48068a50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:23:21.652 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:21.647+0000 7f8b617fa640 1 -- 172.21.15.143:0/2198908316 <== mon.0 v2:172.21.15.143:3300/0 5 ==== osd_map(75..75 src has 1..75) v4 ==== 5502+0+0 (secure 0 0 0) 0x7f8b5c084270 con 0x7f8b6415c5f0 2023-11-26T07:23:21.652 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:21.647+0000 7f8b68c2d640 1 --2- 172.21.15.143:0/2198908316 >> [v2:172.21.15.188:6836/3401128553,v1:172.21.15.188:6837/3401128553] conn(0x7f8b48066590 0x7f8b48068a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:23:21.652 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:21.647+0000 7f8b68c2d640 1 --2- 172.21.15.143:0/2198908316 >> [v2:172.21.15.188:6836/3401128553,v1:172.21.15.188:6837/3401128553] conn(0x7f8b48066590 0x7f8b48068a50 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7f8b580af000 tx=0x7f8b580046d0 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:23:21.652 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:21.651+0000 7f8b617fa640 1 -- 172.21.15.143:0/2198908316 <== mon.0 v2:172.21.15.143:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172649 (secure 0 0 0) 0x7f8b5c0516c0 con 0x7f8b6415c5f0 2023-11-26T07:23:21.854 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:21.851+0000 7f8b6ae91640 1 -- 172.21.15.143:0/2198908316 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_volumes.TestMisc.test_names_can_only_be_goodchars"]} v 0) v1 -- 0x7f8b640cb180 con 0x7f8b6415c5f0 2023-11-26T07:23:22.465 DEBUG:teuthology.orchestra.run.smithi143:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-26T07:23:22.469 DEBUG:teuthology.orchestra.run.smithi188:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-26T07:23:22.594 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:22.591+0000 7f8b617fa640 1 -- 172.21.15.143:0/2198908316 <== mon.0 v2:172.21.15.143:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_volumes.TestMisc.test_names_can_only_be_goodchars"]}]=0 v289) v1 ==== 149+0+0 (secure 0 0 0) 0x7f8b5c054ce0 con 0x7f8b6415c5f0 2023-11-26T07:23:22.597 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:22.595+0000 7f8b6ae91640 1 -- 172.21.15.143:0/2198908316 >> [v2:172.21.15.188:6836/3401128553,v1:172.21.15.188:6837/3401128553] conn(0x7f8b48066590 msgr2=0x7f8b48068a50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:23:22.597 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:22.595+0000 7f8b6ae91640 1 --2- 172.21.15.143:0/2198908316 >> [v2:172.21.15.188:6836/3401128553,v1:172.21.15.188:6837/3401128553] conn(0x7f8b48066590 0x7f8b48068a50 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7f8b580af000 tx=0x7f8b580046d0 comp rx=0 tx=0).stop 2023-11-26T07:23:22.597 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:22.595+0000 7f8b6ae91640 1 -- 172.21.15.143:0/2198908316 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f8b6415c5f0 msgr2=0x7f8b640b0190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:23:22.597 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:22.595+0000 7f8b6ae91640 1 --2- 172.21.15.143:0/2198908316 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f8b6415c5f0 0x7f8b640b0190 secure :-1 s=READY pgs=1106 cs=0 l=1 rev1=1 crypto rx=0x7f8b6406e880 tx=0x7f8b5c00e530 comp rx=0 tx=0).stop 2023-11-26T07:23:22.597 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:22.595+0000 7f8b6ae91640 1 -- 172.21.15.143:0/2198908316 shutdown_connections 2023-11-26T07:23:22.597 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:22.595+0000 7f8b6ae91640 1 --2- 172.21.15.143:0/2198908316 >> [v2:172.21.15.188:6836/3401128553,v1:172.21.15.188:6837/3401128553] conn(0x7f8b48066590 0x7f8b48068a50 unknown :-1 s=CLOSED pgs=42 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:23:22.597 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:22.595+0000 7f8b6ae91640 1 --2- 172.21.15.143:0/2198908316 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f8b6415c5f0 0x7f8b640b0190 unknown :-1 s=CLOSED pgs=1106 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:23:22.597 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:22.595+0000 7f8b6ae91640 1 --2- 172.21.15.143:0/2198908316 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f8b6415a780 0x7f8b6415c0b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:23:22.598 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:22.595+0000 7f8b6ae91640 1 --2- 172.21.15.143:0/2198908316 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f8b54003980 0x7f8b641f6200 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:23:22.598 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:22.595+0000 7f8b6ae91640 1 -- 172.21.15.143:0/2198908316 >> 172.21.15.143:0/2198908316 conn(0x7f8b6406a890 msgr2=0x7f8b640cd120 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:23:22.598 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:22.595+0000 7f8b6ae91640 1 -- 172.21.15.143:0/2198908316 shutdown_connections 2023-11-26T07:23:22.598 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:22.595+0000 7f8b6ae91640 1 -- 172.21.15.143:0/2198908316 wait complete. 2023-11-26T07:23:22.618 DEBUG:teuthology.orchestra.run.smithi143:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mon mon_allow_pool_delete True 2023-11-26T07:23:22.762 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:22.759+0000 7f8142d65640 1 Processor -- start 2023-11-26T07:23:22.762 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:22.759+0000 7f8142d65640 1 -- start start 2023-11-26T07:23:22.764 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:22.763+0000 7f8142d65640 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7f813c143ac0 0x7f813c070370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:23:22.764 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:22.763+0000 7f8142d65640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f813c0cf990 0x7f813c0cd1b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:23:22.764 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:22.763+0000 7f8142d65640 1 -- --> v1:172.21.15.143:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f813c0cc730 con 0x7f813c0708b0 2023-11-26T07:23:22.764 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:22.763+0000 7f8142d65640 1 -- --> v2:172.21.15.143:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f813c143ea0 con 0x7f813c143ac0 2023-11-26T07:23:22.764 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:22.763+0000 7f8142d65640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f813c070ee0 con 0x7f813c0cf990 2023-11-26T07:23:22.764 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:22.763+0000 7f8140b01640 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7f813c143ac0 0x7f813c070370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:23:22.764 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:22.763+0000 7f8133fff640 1 --1- >> v1:172.21.15.143:6789/0 conn(0x7f813c0708b0 0x7f813c0cf280 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.143:6789/0 says I am v1:172.21.15.143:36766/0 (socket says 172.21.15.143:36766) 2023-11-26T07:23:22.764 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:22.763+0000 7f8133fff640 1 -- 172.21.15.143:0/2082438167 learned_addr learned my addr 172.21.15.143:0/2082438167 (peer_addr_for_me v1:172.21.15.143:0/0) 2023-11-26T07:23:22.764 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:22.763+0000 7f8141302640 1 --2- 172.21.15.143:0/2082438167 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f813c0cf990 0x7f813c0cd1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:23:22.765 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:22.763+0000 7f8140b01640 1 -- 172.21.15.143:0/2082438167 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f813c0cf990 msgr2=0x7f813c0cd1b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-26T07:23:22.765 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:22.763+0000 7f8140b01640 1 --2- 172.21.15.143:0/2082438167 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f813c0cf990 0x7f813c0cd1b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:23:22.765 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:22.763+0000 7f8140b01640 1 -- 172.21.15.143:0/2082438167 >> v1:172.21.15.143:6789/0 conn(0x7f813c0708b0 legacy=0x7f813c0cf280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:23:22.765 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:22.763+0000 7f8140b01640 1 -- 172.21.15.143:0/2082438167 --> v2:172.21.15.143:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f813c0711c0 con 0x7f813c143ac0 2023-11-26T07:23:22.765 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:22.763+0000 7f81337fe640 1 -- 172.21.15.143:0/2082438167 <== mon.0 v1:172.21.15.143:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3286240220 0 0) 0x7f813c0cc730 con 0x7f813c0708b0 2023-11-26T07:23:22.767 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:22.767+0000 7f8140b01640 1 --2- 172.21.15.143:0/2082438167 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f813c143ac0 0x7f813c070370 secure :-1 s=READY pgs=1108 cs=0 l=1 rev1=1 crypto rx=0x7f81380097c0 tx=0x7f813801c480 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2ffb76520c3cde38 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:23:22.767 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:22.767+0000 7f81337fe640 1 -- 172.21.15.143:0/2082438167 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f81380ad640 con 0x7f813c143ac0 2023-11-26T07:23:22.767 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:22.767+0000 7f81337fe640 1 -- 172.21.15.143:0/2082438167 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f81380ad7a0 con 0x7f813c143ac0 2023-11-26T07:23:22.767 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:22.767+0000 7f81337fe640 1 -- 172.21.15.143:0/2082438167 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f81380adb30 con 0x7f813c143ac0 2023-11-26T07:23:22.768 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:22.767+0000 7f8142d65640 1 -- 172.21.15.143:0/2082438167 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f813c143ac0 msgr2=0x7f813c070370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:23:22.768 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:22.767+0000 7f8142d65640 1 --2- 172.21.15.143:0/2082438167 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f813c143ac0 0x7f813c070370 secure :-1 s=READY pgs=1108 cs=0 l=1 rev1=1 crypto rx=0x7f81380097c0 tx=0x7f813801c480 comp rx=0 tx=0).stop 2023-11-26T07:23:22.768 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:22.767+0000 7f8142d65640 1 -- 172.21.15.143:0/2082438167 shutdown_connections 2023-11-26T07:23:22.768 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:22.767+0000 7f8142d65640 1 --2- 172.21.15.143:0/2082438167 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f813c0cf990 0x7f813c0cd1b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:23:22.768 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:22.767+0000 7f8142d65640 1 --2- 172.21.15.143:0/2082438167 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f813c143ac0 0x7f813c070370 unknown :-1 s=CLOSED pgs=1108 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:23:22.768 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:22.767+0000 7f8142d65640 1 -- 172.21.15.143:0/2082438167 >> 172.21.15.143:0/2082438167 conn(0x7f813c06a890 msgr2=0x7f813c06aca0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:23:22.768 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:22.767+0000 7f8142d65640 1 -- 172.21.15.143:0/2082438167 shutdown_connections 2023-11-26T07:23:22.768 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:22.767+0000 7f8142d65640 1 -- 172.21.15.143:0/2082438167 wait complete. 2023-11-26T07:23:22.769 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:22.767+0000 7f8142d65640 1 Processor -- start 2023-11-26T07:23:22.769 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:22.767+0000 7f8142d65640 1 -- start start 2023-11-26T07:23:22.772 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:22.771+0000 7f8142d65640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f813c0708b0 0x7f813c13b1c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:23:22.772 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:22.771+0000 7f8142d65640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f813c0cf990 0x7f813c13b760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:23:22.772 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:22.771+0000 7f8142d65640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f813c143ac0 0x7f813c0acdc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:23:22.772 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:22.771+0000 7f8142d65640 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f813c157ab0 con 0x7f813c0708b0 2023-11-26T07:23:22.772 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:22.771+0000 7f8142d65640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f813c15aa80 con 0x7f813c0cf990 2023-11-26T07:23:22.772 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:22.771+0000 7f8142d65640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f813c15cf20 con 0x7f813c143ac0 2023-11-26T07:23:22.773 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:22.771+0000 7f8140b01640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f813c0708b0 0x7f813c13b1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:23:22.773 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:22.771+0000 7f8140b01640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f813c0708b0 0x7f813c13b1c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:41042/0 (socket says 172.21.15.143:41042) 2023-11-26T07:23:22.773 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:22.771+0000 7f8140b01640 1 -- 172.21.15.143:0/3422310425 learned_addr learned my addr 172.21.15.143:0/3422310425 (peer_addr_for_me v2:172.21.15.143:0/0) 2023-11-26T07:23:22.773 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:22.771+0000 7f8141302640 1 --2- 172.21.15.143:0/3422310425 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f813c143ac0 0x7f813c0acdc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:23:22.773 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:22.771+0000 7f8133fff640 1 --2- 172.21.15.143:0/3422310425 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f813c0cf990 0x7f813c13b760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:23:22.773 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:22.771+0000 7f8140b01640 1 -- 172.21.15.143:0/3422310425 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f813c143ac0 msgr2=0x7f813c0acdc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:23:22.773 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:22.771+0000 7f8140b01640 1 --2- 172.21.15.143:0/3422310425 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f813c143ac0 0x7f813c0acdc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:23:22.773 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:22.771+0000 7f8140b01640 1 -- 172.21.15.143:0/3422310425 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f813c0cf990 msgr2=0x7f813c13b760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:23:22.773 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:22.771+0000 7f8140b01640 1 --2- 172.21.15.143:0/3422310425 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f813c0cf990 0x7f813c13b760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:23:22.773 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:22.771+0000 7f8140b01640 1 -- 172.21.15.143:0/3422310425 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8138009420 con 0x7f813c0708b0 2023-11-26T07:23:22.773 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:22.771+0000 7f8140b01640 1 --2- 172.21.15.143:0/3422310425 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f813c0708b0 0x7f813c13b1c0 secure :-1 s=READY pgs=1109 cs=0 l=1 rev1=1 crypto rx=0x7f813801a0b0 tx=0x7f8138019220 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:23:22.773 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:22.771+0000 7f81317fa640 1 -- 172.21.15.143:0/3422310425 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f81380b6610 con 0x7f813c0708b0 2023-11-26T07:23:22.773 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:22.771+0000 7f81317fa640 1 -- 172.21.15.143:0/3422310425 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f81380a6040 con 0x7f813c0708b0 2023-11-26T07:23:22.774 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:22.771+0000 7f81317fa640 1 -- 172.21.15.143:0/3422310425 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f81380b69b0 con 0x7f813c0708b0 2023-11-26T07:23:22.774 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:22.771+0000 7f8142d65640 1 -- 172.21.15.143:0/3422310425 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f813c15d200 con 0x7f813c0708b0 2023-11-26T07:23:22.774 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:22.771+0000 7f8142d65640 1 -- 172.21.15.143:0/3422310425 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f813c13e280 con 0x7f813c0708b0 2023-11-26T07:23:22.774 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:22.771+0000 7f8142d65640 1 -- 172.21.15.143:0/3422310425 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f813c15d3f0 con 0x7f813c0708b0 2023-11-26T07:23:22.774 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:22.771+0000 7f81317fa640 1 -- 172.21.15.143:0/3422310425 <== mon.0 v2:172.21.15.143:3300/0 4 ==== mgrmap(e 16) v1 ==== 85954+0+0 (secure 0 0 0) 0x7f81380b4040 con 0x7f813c0708b0 2023-11-26T07:23:22.775 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:22.775+0000 7f81317fa640 1 --2- 172.21.15.143:0/3422310425 >> [v2:172.21.15.188:6836/3401128553,v1:172.21.15.188:6837/3401128553] conn(0x7f81280665c0 0x7f8128068a80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:23:22.775 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:22.775+0000 7f81317fa640 1 -- 172.21.15.143:0/3422310425 <== mon.0 v2:172.21.15.143:3300/0 5 ==== osd_map(75..75 src has 1..75) v4 ==== 5502+0+0 (secure 0 0 0) 0x7f813811dee0 con 0x7f813c0708b0 2023-11-26T07:23:22.775 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:22.775+0000 7f8133fff640 1 --2- 172.21.15.143:0/3422310425 >> [v2:172.21.15.188:6836/3401128553,v1:172.21.15.188:6837/3401128553] conn(0x7f81280665c0 0x7f8128068a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:23:22.775 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:22.775+0000 7f8133fff640 1 --2- 172.21.15.143:0/3422310425 >> [v2:172.21.15.188:6836/3401128553,v1:172.21.15.188:6837/3401128553] conn(0x7f81280665c0 0x7f8128068a80 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7f813c0b1160 tx=0x7f812c009040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:23:22.778 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:22.779+0000 7f81317fa640 1 -- 172.21.15.143:0/3422310425 <== mon.0 v2:172.21.15.143:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172649 (secure 0 0 0) 0x7f81380eb330 con 0x7f813c0708b0 2023-11-26T07:23:22.936 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:22.935+0000 7f8142d65640 1 -- 172.21.15.143:0/3422310425 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command([{prefix=config set, name=mon_allow_pool_delete}] v 0) v1 -- 0x7f813c067680 con 0x7f813c0708b0 2023-11-26T07:23:22.940 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:22.939+0000 7f81317fa640 1 -- 172.21.15.143:0/3422310425 <== mon.0 v2:172.21.15.143:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mon_allow_pool_delete}]=0 v18)=0 v18) v1 ==== 122+0+0 (secure 0 0 0) 0x7f81380b2020 con 0x7f813c0708b0 2023-11-26T07:23:22.943 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:22.939+0000 7f8142d65640 1 -- 172.21.15.143:0/3422310425 >> [v2:172.21.15.188:6836/3401128553,v1:172.21.15.188:6837/3401128553] conn(0x7f81280665c0 msgr2=0x7f8128068a80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:23:22.943 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:22.939+0000 7f8142d65640 1 --2- 172.21.15.143:0/3422310425 >> [v2:172.21.15.188:6836/3401128553,v1:172.21.15.188:6837/3401128553] conn(0x7f81280665c0 0x7f8128068a80 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7f813c0b1160 tx=0x7f812c009040 comp rx=0 tx=0).stop 2023-11-26T07:23:22.943 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:22.939+0000 7f8142d65640 1 -- 172.21.15.143:0/3422310425 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f813c0708b0 msgr2=0x7f813c13b1c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:23:22.943 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:22.939+0000 7f8142d65640 1 --2- 172.21.15.143:0/3422310425 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f813c0708b0 0x7f813c13b1c0 secure :-1 s=READY pgs=1109 cs=0 l=1 rev1=1 crypto rx=0x7f813801a0b0 tx=0x7f8138019220 comp rx=0 tx=0).stop 2023-11-26T07:23:22.943 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:22.939+0000 7f8142d65640 1 -- 172.21.15.143:0/3422310425 shutdown_connections 2023-11-26T07:23:22.943 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:22.939+0000 7f8142d65640 1 --2- 172.21.15.143:0/3422310425 >> [v2:172.21.15.188:6836/3401128553,v1:172.21.15.188:6837/3401128553] conn(0x7f81280665c0 0x7f8128068a80 unknown :-1 s=CLOSED pgs=43 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:23:22.943 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:22.939+0000 7f8142d65640 1 --2- 172.21.15.143:0/3422310425 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f813c143ac0 0x7f813c0acdc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:23:22.943 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:22.939+0000 7f8142d65640 1 --2- 172.21.15.143:0/3422310425 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f813c0cf990 0x7f813c13b760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:23:22.943 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:22.939+0000 7f8142d65640 1 --2- 172.21.15.143:0/3422310425 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f813c0708b0 0x7f813c13b1c0 unknown :-1 s=CLOSED pgs=1109 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:23:22.943 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:22.939+0000 7f8142d65640 1 -- 172.21.15.143:0/3422310425 >> 172.21.15.143:0/3422310425 conn(0x7f813c06a890 msgr2=0x7f813c06ac70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:23:22.943 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:22.939+0000 7f8142d65640 1 -- 172.21.15.143:0/3422310425 shutdown_connections 2023-11-26T07:23:22.943 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:22.943+0000 7f8142d65640 1 -- 172.21.15.143:0/3422310425 wait complete. 2023-11-26T07:23:22.962 INFO:teuthology.nuke.actions:Clearing teuthology firewall rules... 2023-11-26T07:23:22.962 DEBUG:teuthology.orchestra.run.smithi143:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2023-11-26T07:23:23.053 DEBUG:teuthology.orchestra.run.smithi188:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2023-11-26T07:23:23.126 INFO:teuthology.nuke.actions:Cleared teuthology firewall rules. 2023-11-26T07:23:23.126 DEBUG:teuthology.orchestra.run.smithi143:> set -ex 2023-11-26T07:23:23.126 DEBUG:teuthology.orchestra.run.smithi143:> dd if=/proc/self/mounts of=/dev/stdout 2023-11-26T07:23:23.135 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,relatime,size=16354672k,nr_inodes=4088668,mode=755 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3275384k,mode=755 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=3455 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysusers.service ramfs ro,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3275380k,nr_inodes=818845,mode=700,uid=1000,gid=1271 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3275384k,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2023-11-26T07:23:23.136 DEBUG:teuthology.orchestra.run.smithi188:> set -ex 2023-11-26T07:23:23.136 DEBUG:teuthology.orchestra.run.smithi188:> dd if=/proc/self/mounts of=/dev/stdout 2023-11-26T07:23:23.145 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,relatime,size=16354628k,nr_inodes=4088657,mode=755 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3275376k,mode=755 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=787 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysusers.service ramfs ro,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3275372k,nr_inodes=818843,mode=700,uid=1000,gid=1271 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3275376k,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2023-11-26T07:23:23.145 DEBUG:teuthology.orchestra.run.smithi143:> set -ex 2023-11-26T07:23:23.145 DEBUG:teuthology.orchestra.run.smithi143:> dd if=/proc/self/mounts of=/dev/stdout 2023-11-26T07:23:23.187 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,relatime,size=16354672k,nr_inodes=4088668,mode=755 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3275384k,mode=755 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=3455 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysusers.service ramfs ro,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3275380k,nr_inodes=818845,mode=700,uid=1000,gid=1271 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3275384k,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2023-11-26T07:23:23.187 DEBUG:teuthology.orchestra.run.smithi188:> set -ex 2023-11-26T07:23:23.187 DEBUG:teuthology.orchestra.run.smithi188:> dd if=/proc/self/mounts of=/dev/stdout 2023-11-26T07:23:23.200 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,relatime,size=16354628k,nr_inodes=4088657,mode=755 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3275376k,mode=755 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=787 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysusers.service ramfs ro,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3275372k,nr_inodes=818843,mode=700,uid=1000,gid=1271 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3275376k,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2023-11-26T07:23:23.200 INFO:tasks.cephfs.cephfs_test_case:[, , , ] 2023-11-26T07:23:23.200 DEBUG:teuthology.orchestra.run.smithi143:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-11-26T07:23:23.367 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:23.363+0000 7fc906c9c640 1 Processor -- start 2023-11-26T07:23:23.367 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:23.367+0000 7fc906c9c640 1 -- start start 2023-11-26T07:23:23.368 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:23.367+0000 7fc906c9c640 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7fc90015d6e0 0x7fc9000d0df0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:23:23.368 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:23.367+0000 7fc906c9c640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fc9000d1bc0 0x7fc9000b0750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:23:23.368 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:23.367+0000 7fc906c9c640 1 -- --> v1:172.21.15.188:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc9000cf0e0 con 0x7fc9000b2160 2023-11-26T07:23:23.368 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:23.367+0000 7fc906c9c640 1 -- --> v2:172.21.15.188:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc90015dac0 con 0x7fc90015d6e0 2023-11-26T07:23:23.368 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:23.367+0000 7fc906c9c640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc9000ceb20 con 0x7fc9000d1bc0 2023-11-26T07:23:23.369 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:23.367+0000 7fc905239640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fc9000d1bc0 0x7fc9000b0750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:23:23.369 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:23.367+0000 7fc904a38640 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7fc90015d6e0 0x7fc9000d0df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:23:23.369 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:23.367+0000 7fc905239640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fc9000d1bc0 0x7fc9000b0750 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.188:3301/0 says I am v2:172.21.15.143:51338/0 (socket says 172.21.15.143:51338) 2023-11-26T07:23:23.369 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:23.367+0000 7fc904a38640 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7fc90015d6e0 0x7fc9000d0df0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.188:3300/0 says I am v2:172.21.15.143:42686/0 (socket says 172.21.15.143:42686) 2023-11-26T07:23:23.369 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:23.367+0000 7fc905239640 1 -- 172.21.15.143:0/33229702 learned_addr learned my addr 172.21.15.143:0/33229702 (peer_addr_for_me v2:172.21.15.143:0/0) 2023-11-26T07:23:23.369 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:23.367+0000 7fc905239640 1 -- 172.21.15.143:0/33229702 >> v2:172.21.15.188:3300/0 conn(0x7fc90015d6e0 msgr2=0x7fc9000d0df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-26T07:23:23.369 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:23.367+0000 7fc905239640 1 --2- 172.21.15.143:0/33229702 >> v2:172.21.15.188:3300/0 conn(0x7fc90015d6e0 0x7fc9000d0df0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:23:23.369 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:23.367+0000 7fc905239640 1 -- 172.21.15.143:0/33229702 >> v1:172.21.15.188:6789/0 conn(0x7fc9000b2160 legacy=0x7fc9000d14b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:23:23.369 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:23.367+0000 7fc905239640 1 -- 172.21.15.143:0/33229702 --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc9000cee00 con 0x7fc9000d1bc0 2023-11-26T07:23:23.369 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:23.367+0000 7fc904a38640 1 --2- 172.21.15.143:0/33229702 >> v2:172.21.15.188:3300/0 conn(0x7fc90015d6e0 0x7fc9000d0df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-26T07:23:23.372 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:23.371+0000 7fc905239640 1 --2- 172.21.15.143:0/33229702 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fc9000d1bc0 0x7fc9000b0750 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7fc8f8009e50 tx=0x7fc8f801ca10 comp rx=0 tx=0).ready entity=mon.2 client_cookie=add442675d58e7ea server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:23:23.372 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:23.371+0000 7fc8f77fe640 1 -- 172.21.15.143:0/33229702 <== mon.2 v2:172.21.15.188:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc8f80a57d0 con 0x7fc9000d1bc0 2023-11-26T07:23:23.372 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:23.371+0000 7fc8f77fe640 1 -- 172.21.15.143:0/33229702 <== mon.2 v2:172.21.15.188:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc8f80a5930 con 0x7fc9000d1bc0 2023-11-26T07:23:23.372 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:23.371+0000 7fc8f77fe640 1 -- 172.21.15.143:0/33229702 <== mon.2 v2:172.21.15.188:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc8f80a5cc0 con 0x7fc9000d1bc0 2023-11-26T07:23:23.373 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:23.371+0000 7fc906c9c640 1 -- 172.21.15.143:0/33229702 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fc9000d1bc0 msgr2=0x7fc9000b0750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:23:23.373 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:23.371+0000 7fc906c9c640 1 --2- 172.21.15.143:0/33229702 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fc9000d1bc0 0x7fc9000b0750 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7fc8f8009e50 tx=0x7fc8f801ca10 comp rx=0 tx=0).stop 2023-11-26T07:23:23.373 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:23.371+0000 7fc906c9c640 1 -- 172.21.15.143:0/33229702 shutdown_connections 2023-11-26T07:23:23.373 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:23.371+0000 7fc906c9c640 1 --2- 172.21.15.143:0/33229702 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fc9000d1bc0 0x7fc9000b0750 unknown :-1 s=CLOSED pgs=139 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:23:23.373 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:23.371+0000 7fc906c9c640 1 --2- 172.21.15.143:0/33229702 >> v2:172.21.15.188:3300/0 conn(0x7fc90015d6e0 0x7fc9000d0df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:23:23.373 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:23.371+0000 7fc906c9c640 1 -- 172.21.15.143:0/33229702 >> 172.21.15.143:0/33229702 conn(0x7fc90009c550 msgr2=0x7fc9000d3220 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:23:23.373 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:23.371+0000 7fc906c9c640 1 -- 172.21.15.143:0/33229702 shutdown_connections 2023-11-26T07:23:23.373 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:23.371+0000 7fc906c9c640 1 -- 172.21.15.143:0/33229702 wait complete. 2023-11-26T07:23:23.374 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:23.371+0000 7fc906c9c640 1 Processor -- start 2023-11-26T07:23:23.374 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:23.375+0000 7fc906c9c640 1 -- start start 2023-11-26T07:23:23.376 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:23.375+0000 7fc906c9c640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fc9000b2160 0x7fc90015dc00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:23:23.376 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:23.375+0000 7fc906c9c640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fc9000d1bc0 0x7fc90015f610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:23:23.376 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:23.375+0000 7fc906c9c640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fc90015d6e0 0x7fc90015fb50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:23:23.376 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:23.375+0000 7fc906c9c640 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc9000cc8f0 con 0x7fc9000d1bc0 2023-11-26T07:23:23.376 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:23.375+0000 7fc906c9c640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc900158730 con 0x7fc90015d6e0 2023-11-26T07:23:23.377 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:23.375+0000 7fc906c9c640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc9000ccb00 con 0x7fc9000b2160 2023-11-26T07:23:23.377 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:23.375+0000 7fc8f7fff640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fc9000d1bc0 0x7fc90015f610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:23:23.377 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:23.375+0000 7fc8f7fff640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fc9000d1bc0 0x7fc90015f610 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:41054/0 (socket says 172.21.15.143:41054) 2023-11-26T07:23:23.377 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:23.375+0000 7fc8f7fff640 1 -- 172.21.15.143:0/231241974 learned_addr learned my addr 172.21.15.143:0/231241974 (peer_addr_for_me v2:172.21.15.143:0/0) 2023-11-26T07:23:23.377 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:23.375+0000 7fc904a38640 1 --2- 172.21.15.143:0/231241974 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fc9000b2160 0x7fc90015dc00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:23:23.377 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:23.375+0000 7fc905239640 1 --2- 172.21.15.143:0/231241974 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fc90015d6e0 0x7fc90015fb50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:23:23.377 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:23.375+0000 7fc8f7fff640 1 -- 172.21.15.143:0/231241974 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fc9000b2160 msgr2=0x7fc90015dc00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:23:23.377 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:23.375+0000 7fc8f7fff640 1 --2- 172.21.15.143:0/231241974 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fc9000b2160 0x7fc90015dc00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:23:23.377 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:23.375+0000 7fc8f7fff640 1 -- 172.21.15.143:0/231241974 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fc90015d6e0 msgr2=0x7fc90015fb50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:23:23.377 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:23.375+0000 7fc8f7fff640 1 --2- 172.21.15.143:0/231241974 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fc90015d6e0 0x7fc90015fb50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:23:23.377 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:23.375+0000 7fc8f7fff640 1 -- 172.21.15.143:0/231241974 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc8fc009420 con 0x7fc9000d1bc0 2023-11-26T07:23:23.378 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:23.375+0000 7fc8f7fff640 1 --2- 172.21.15.143:0/231241974 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fc9000d1bc0 0x7fc90015f610 secure :-1 s=READY pgs=1110 cs=0 l=1 rev1=1 crypto rx=0x7fc8f000c3c0 tx=0x7fc8f000c880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:23:23.378 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:23.375+0000 7fc8f57fa640 1 -- 172.21.15.143:0/231241974 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc8f00052d0 con 0x7fc9000d1bc0 2023-11-26T07:23:23.378 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:23.375+0000 7fc8f57fa640 1 -- 172.21.15.143:0/231241974 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc8f0005430 con 0x7fc9000d1bc0 2023-11-26T07:23:23.378 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:23.375+0000 7fc906c9c640 1 -- 172.21.15.143:0/231241974 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc8f8009b30 con 0x7fc9000d1bc0 2023-11-26T07:23:23.378 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:23.375+0000 7fc8f57fa640 1 -- 172.21.15.143:0/231241974 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc8f0003840 con 0x7fc9000d1bc0 2023-11-26T07:23:23.379 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:23.375+0000 7fc906c9c640 1 -- 172.21.15.143:0/231241974 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc9001583c0 con 0x7fc9000d1bc0 2023-11-26T07:23:23.379 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:23.375+0000 7fc906c9c640 1 -- 172.21.15.143:0/231241974 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc9000d3610 con 0x7fc9000d1bc0 2023-11-26T07:23:23.379 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:23.379+0000 7fc8f57fa640 1 -- 172.21.15.143:0/231241974 <== mon.0 v2:172.21.15.143:3300/0 4 ==== mgrmap(e 16) v1 ==== 85954+0+0 (secure 0 0 0) 0x7fc8f00039a0 con 0x7fc9000d1bc0 2023-11-26T07:23:23.380 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:23.379+0000 7fc8f57fa640 1 --2- 172.21.15.143:0/231241974 >> [v2:172.21.15.188:6836/3401128553,v1:172.21.15.188:6837/3401128553] conn(0x7fc8ec066590 0x7fc8ec068a50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:23:23.380 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:23.379+0000 7fc8f57fa640 1 -- 172.21.15.143:0/231241974 <== mon.0 v2:172.21.15.143:3300/0 5 ==== osd_map(75..75 src has 1..75) v4 ==== 5502+0+0 (secure 0 0 0) 0x7fc8f00833e0 con 0x7fc9000d1bc0 2023-11-26T07:23:23.380 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:23.379+0000 7fc904a38640 1 --2- 172.21.15.143:0/231241974 >> [v2:172.21.15.188:6836/3401128553,v1:172.21.15.188:6837/3401128553] conn(0x7fc8ec066590 0x7fc8ec068a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:23:23.381 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:23.379+0000 7fc904a38640 1 --2- 172.21.15.143:0/231241974 >> [v2:172.21.15.188:6836/3401128553,v1:172.21.15.188:6837/3401128553] conn(0x7fc8ec066590 0x7fc8ec068a50 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7fc8fc009cf0 tx=0x7fc8fc009190 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:23:23.384 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:23.383+0000 7fc8f57fa640 1 -- 172.21.15.143:0/231241974 <== mon.0 v2:172.21.15.143:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172649 (secure 0 0 0) 0x7fc8f00508c0 con 0x7fc9000d1bc0 2023-11-26T07:23:23.563 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:23.563+0000 7fc906c9c640 1 -- 172.21.15.143:0/231241974 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fc900067680 con 0x7fc9000d1bc0 2023-11-26T07:23:23.564 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:23.563+0000 7fc8f57fa640 1 -- 172.21.15.143:0/231241974 <== mon.0 v2:172.21.15.143:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 45 v45) v1 ==== 94+0+3708 (secure 0 0 0) 0x7fc8f0053ee0 con 0x7fc9000d1bc0 2023-11-26T07:23:23.564 INFO:teuthology.orchestra.run.smithi143.stdout: 2023-11-26T07:23:23.564 INFO:teuthology.orchestra.run.smithi143.stdout:{"epoch":45,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5927,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.143:6837/578144260","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6835","nonce":578144260},{"type":"v1","addr":"172.21.15.143:6837","nonce":578144260}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":38},{"gid":5930,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.188:6834/1048736873","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6832","nonce":1048736873},{"type":"v1","addr":"172.21.15.188:6834","nonce":1048736873}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":38},{"gid":5933,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.188:6835/2063768716","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6833","nonce":2063768716},{"type":"v1","addr":"172.21.15.188:6835","nonce":2063768716}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":38},{"gid":7377,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.143:6833/1918262770","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6832","nonce":1918262770},{"type":"v1","addr":"172.21.15.143:6833","nonce":1918262770}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":45}],"filesystems":[]} 2023-11-26T07:23:23.564 INFO:teuthology.orchestra.run.smithi143.stderr:dumped fsmap epoch 45 2023-11-26T07:23:23.566 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:23.563+0000 7fc906c9c640 1 -- 172.21.15.143:0/231241974 >> [v2:172.21.15.188:6836/3401128553,v1:172.21.15.188:6837/3401128553] conn(0x7fc8ec066590 msgr2=0x7fc8ec068a50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:23:23.566 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:23.563+0000 7fc906c9c640 1 --2- 172.21.15.143:0/231241974 >> [v2:172.21.15.188:6836/3401128553,v1:172.21.15.188:6837/3401128553] conn(0x7fc8ec066590 0x7fc8ec068a50 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7fc8fc009cf0 tx=0x7fc8fc009190 comp rx=0 tx=0).stop 2023-11-26T07:23:23.566 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:23.563+0000 7fc906c9c640 1 -- 172.21.15.143:0/231241974 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fc9000d1bc0 msgr2=0x7fc90015f610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:23:23.566 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:23.563+0000 7fc906c9c640 1 --2- 172.21.15.143:0/231241974 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fc9000d1bc0 0x7fc90015f610 secure :-1 s=READY pgs=1110 cs=0 l=1 rev1=1 crypto rx=0x7fc8f000c3c0 tx=0x7fc8f000c880 comp rx=0 tx=0).stop 2023-11-26T07:23:23.566 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:23.563+0000 7fc906c9c640 1 -- 172.21.15.143:0/231241974 shutdown_connections 2023-11-26T07:23:23.566 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:23.563+0000 7fc906c9c640 1 --2- 172.21.15.143:0/231241974 >> [v2:172.21.15.188:6836/3401128553,v1:172.21.15.188:6837/3401128553] conn(0x7fc8ec066590 0x7fc8ec068a50 unknown :-1 s=CLOSED pgs=44 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:23:23.566 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:23.563+0000 7fc906c9c640 1 --2- 172.21.15.143:0/231241974 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fc90015d6e0 0x7fc90015fb50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:23:23.566 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:23.563+0000 7fc906c9c640 1 --2- 172.21.15.143:0/231241974 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fc9000d1bc0 0x7fc90015f610 unknown :-1 s=CLOSED pgs=1110 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:23:23.567 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:23.563+0000 7fc906c9c640 1 --2- 172.21.15.143:0/231241974 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fc9000b2160 0x7fc90015dc00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:23:23.567 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:23.563+0000 7fc906c9c640 1 -- 172.21.15.143:0/231241974 >> 172.21.15.143:0/231241974 conn(0x7fc90009c550 msgr2=0x7fc90015c7f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:23:23.567 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:23.563+0000 7fc906c9c640 1 -- 172.21.15.143:0/231241974 shutdown_connections 2023-11-26T07:23:23.567 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:23.563+0000 7fc906c9c640 1 -- 172.21.15.143:0/231241974 wait complete. 2023-11-26T07:23:23.585 INFO:tasks.ceph.mds.a:Restarting daemon 2023-11-26T07:23:23.585 INFO:tasks.ceph.mds.a:Stopping old one... 2023-11-26T07:23:23.585 DEBUG:tasks.ceph.mds.a:waiting for process to exit 2023-11-26T07:23:23.585 INFO:teuthology.orchestra.run:waiting for 300 2023-11-26T07:23:23.586 INFO:tasks.ceph.mds.c:Restarting daemon 2023-11-26T07:23:23.586 INFO:tasks.ceph.mds.c:Stopping old one... 2023-11-26T07:23:23.586 DEBUG:tasks.ceph.mds.c:waiting for process to exit 2023-11-26T07:23:23.586 INFO:teuthology.orchestra.run:waiting for 300 2023-11-26T07:23:23.586 INFO:tasks.ceph.mds.b:Restarting daemon 2023-11-26T07:23:23.586 INFO:tasks.ceph.mds.b:Stopping old one... 2023-11-26T07:23:23.586 DEBUG:tasks.ceph.mds.b:waiting for process to exit 2023-11-26T07:23:23.587 INFO:teuthology.orchestra.run:waiting for 300 2023-11-26T07:23:23.587 INFO:tasks.ceph.mds.d:Restarting daemon 2023-11-26T07:23:23.587 INFO:tasks.ceph.mds.d:Stopping old one... 2023-11-26T07:23:23.587 DEBUG:tasks.ceph.mds.d:waiting for process to exit 2023-11-26T07:23:23.587 INFO:teuthology.orchestra.run:waiting for 300 2023-11-26T07:23:23.598 INFO:tasks.ceph.mds.a:Stopped 2023-11-26T07:23:23.598 DEBUG:teuthology.orchestra.run.smithi143:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i a 2023-11-26T07:23:23.599 INFO:tasks.ceph.mds.c:Stopped 2023-11-26T07:23:23.599 DEBUG:teuthology.orchestra.run.smithi143:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i c 2023-11-26T07:23:23.605 INFO:tasks.ceph.mds.b:Stopped 2023-11-26T07:23:23.605 DEBUG:teuthology.orchestra.run.smithi188:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i b 2023-11-26T07:23:23.606 INFO:tasks.ceph.mds.d:Stopped 2023-11-26T07:23:23.606 DEBUG:teuthology.orchestra.run.smithi188:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i d 2023-11-26T07:23:23.649 INFO:tasks.ceph.mds.a:Started 2023-11-26T07:23:23.649 DEBUG:teuthology.parallel:result is None 2023-11-26T07:23:23.651 INFO:tasks.ceph.mds.c:Started 2023-11-26T07:23:23.651 DEBUG:teuthology.parallel:result is None 2023-11-26T07:23:23.654 INFO:tasks.ceph.mds.b:Started 2023-11-26T07:23:23.655 DEBUG:teuthology.parallel:result is None 2023-11-26T07:23:23.656 INFO:tasks.ceph.mds.d:Started 2023-11-26T07:23:23.657 DEBUG:teuthology.parallel:result is None 2023-11-26T07:23:23.657 DEBUG:teuthology.orchestra.run.smithi143:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist clear 2023-11-26T07:23:23.740 INFO:tasks.ceph.mds.c.smithi143.stdout:starting mds.c at 2023-11-26T07:23:23.740 INFO:tasks.ceph.mds.a.smithi143.stdout:starting mds.a at 2023-11-26T07:23:23.741 INFO:tasks.ceph.mds.a.smithi143.stderr:2023-11-26T07:23:23.739+0000 7f70d4121640 -1 PosixStack listen unable to listen on v2:0.0.0.0:6832/0: (98) Address already in use 2023-11-26T07:23:23.748 INFO:tasks.ceph.mds.b.smithi188.stdout:starting mds.b at 2023-11-26T07:23:23.748 INFO:tasks.ceph.mds.d.smithi188.stdout:starting mds.d at 2023-11-26T07:23:23.803 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:23.803+0000 7fb8db59e640 1 Processor -- start 2023-11-26T07:23:23.804 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:23.803+0000 7fb8db59e640 1 -- start start 2023-11-26T07:23:23.805 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:23.803+0000 7fb8db59e640 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7fb8dc144720 0x7fb8dc06d300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:23:23.806 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:23.803+0000 7fb8db59e640 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7fb8dc06d840 0x7fb8dc0714e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:23:23.806 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:23.803+0000 7fb8db59e640 1 -- --> v1:172.21.15.188:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb8dc071e50 con 0x7fb8dc071a20 2023-11-26T07:23:23.806 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:23.803+0000 7fb8db59e640 1 -- --> v2:172.21.15.143:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb8dc06e3f0 con 0x7fb8dc06d840 2023-11-26T07:23:23.806 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:23.803+0000 7fb8db59e640 1 -- --> v2:172.21.15.188:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb8dc0722f0 con 0x7fb8dc144720 2023-11-26T07:23:23.806 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:23.803+0000 7fb8d9d9b640 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7fb8dc06d840 0x7fb8dc0714e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:23:23.806 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:23.803+0000 7fb8d9d9b640 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7fb8dc06d840 0x7fb8dc0714e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:41090/0 (socket says 172.21.15.143:41090) 2023-11-26T07:23:23.806 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:23.803+0000 7fb8d9d9b640 1 -- 172.21.15.143:0/2444085565 learned_addr learned my addr 172.21.15.143:0/2444085565 (peer_addr_for_me v2:172.21.15.143:0/0) 2023-11-26T07:23:23.806 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:23.803+0000 7fb8da59c640 1 --2- 172.21.15.143:0/2444085565 >> v2:172.21.15.188:3300/0 conn(0x7fb8dc144720 0x7fb8dc06d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:23:23.806 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:23.803+0000 7fb8d9d9b640 1 -- 172.21.15.143:0/2444085565 >> v2:172.21.15.188:3300/0 conn(0x7fb8dc144720 msgr2=0x7fb8dc06d300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-26T07:23:23.806 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:23.803+0000 7fb8d9d9b640 1 --2- 172.21.15.143:0/2444085565 >> v2:172.21.15.188:3300/0 conn(0x7fb8dc144720 0x7fb8dc06d300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:23:23.806 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:23.803+0000 7fb8d9d9b640 1 -- 172.21.15.143:0/2444085565 >> v1:172.21.15.188:6789/0 conn(0x7fb8dc071a20 legacy=0x7fb8dc0cc170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-26T07:23:23.806 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:23.803+0000 7fb8d9d9b640 1 -- 172.21.15.143:0/2444085565 --> v2:172.21.15.143:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb8dc06de30 con 0x7fb8dc06d840 2023-11-26T07:23:23.807 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:23.803+0000 7fb8da59c640 1 --2- 172.21.15.143:0/2444085565 >> v2:172.21.15.188:3300/0 conn(0x7fb8dc144720 0x7fb8dc06d300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-26T07:23:23.809 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:23.807+0000 7fb8d9d9b640 1 --2- 172.21.15.143:0/2444085565 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fb8dc06d840 0x7fb8dc0714e0 secure :-1 s=READY pgs=1119 cs=0 l=1 rev1=1 crypto rx=0x7fb8dc070ae0 tx=0x7fb8d001c470 comp rx=0 tx=0).ready entity=mon.0 client_cookie=afe838362de6a33c server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:23:23.809 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:23.807+0000 7fb8d959a640 1 -- 172.21.15.143:0/2444085565 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb8d00ab670 con 0x7fb8dc06d840 2023-11-26T07:23:23.809 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:23.807+0000 7fb8d959a640 1 -- 172.21.15.143:0/2444085565 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb8d00ab7d0 con 0x7fb8dc06d840 2023-11-26T07:23:23.809 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:23.807+0000 7fb8d959a640 1 -- 172.21.15.143:0/2444085565 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb8d00abb60 con 0x7fb8dc06d840 2023-11-26T07:23:23.810 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:23.807+0000 7fb8db59e640 1 -- 172.21.15.143:0/2444085565 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fb8dc06d840 msgr2=0x7fb8dc0714e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:23:23.810 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:23.807+0000 7fb8db59e640 1 --2- 172.21.15.143:0/2444085565 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fb8dc06d840 0x7fb8dc0714e0 secure :-1 s=READY pgs=1119 cs=0 l=1 rev1=1 crypto rx=0x7fb8dc070ae0 tx=0x7fb8d001c470 comp rx=0 tx=0).stop 2023-11-26T07:23:23.810 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:23.807+0000 7fb8db59e640 1 -- 172.21.15.143:0/2444085565 shutdown_connections 2023-11-26T07:23:23.810 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:23.807+0000 7fb8db59e640 1 --2- 172.21.15.143:0/2444085565 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fb8dc06d840 0x7fb8dc0714e0 unknown :-1 s=CLOSED pgs=1119 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:23:23.810 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:23.807+0000 7fb8db59e640 1 --2- 172.21.15.143:0/2444085565 >> v2:172.21.15.188:3300/0 conn(0x7fb8dc144720 0x7fb8dc06d300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:23:23.810 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:23.807+0000 7fb8db59e640 1 -- 172.21.15.143:0/2444085565 >> 172.21.15.143:0/2444085565 conn(0x7fb8dc06a890 msgr2=0x7fb8dc06aca0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:23:23.810 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:23.807+0000 7fb8db59e640 1 -- 172.21.15.143:0/2444085565 shutdown_connections 2023-11-26T07:23:23.810 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:23.807+0000 7fb8db59e640 1 -- 172.21.15.143:0/2444085565 wait complete. 2023-11-26T07:23:23.811 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:23.811+0000 7fb8db59e640 1 Processor -- start 2023-11-26T07:23:23.811 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:23.811+0000 7fb8db59e640 1 -- start start 2023-11-26T07:23:23.813 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:23.811+0000 7fb8db59e640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fb8dc06d840 0x7fb8dc146620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:23:23.813 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:23.811+0000 7fb8db59e640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fb8dc071a20 0x7fb8dc13c610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:23:23.813 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:23.811+0000 7fb8db59e640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fb8dc144720 0x7fb8dc144c80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:23:23.813 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:23.811+0000 7fb8db59e640 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb8dc145580 con 0x7fb8dc06d840 2023-11-26T07:23:23.814 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:23.811+0000 7fb8db59e640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb8dc145860 con 0x7fb8dc071a20 2023-11-26T07:23:23.814 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:23.811+0000 7fb8db59e640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb8dc145b40 con 0x7fb8dc144720 2023-11-26T07:23:23.814 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:23.811+0000 7fb8da59c640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fb8dc06d840 0x7fb8dc146620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:23:23.814 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:23.811+0000 7fb8da59c640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fb8dc06d840 0x7fb8dc146620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:41100/0 (socket says 172.21.15.143:41100) 2023-11-26T07:23:23.814 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:23.811+0000 7fb8da59c640 1 -- 172.21.15.143:0/1296159150 learned_addr learned my addr 172.21.15.143:0/1296159150 (peer_addr_for_me v2:172.21.15.143:0/0) 2023-11-26T07:23:23.814 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:23.811+0000 7fb8dad9d640 1 --2- 172.21.15.143:0/1296159150 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fb8dc144720 0x7fb8dc144c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:23:23.814 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:23.811+0000 7fb8d9d9b640 1 --2- 172.21.15.143:0/1296159150 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fb8dc071a20 0x7fb8dc13c610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:23:23.814 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:23.811+0000 7fb8da59c640 1 -- 172.21.15.143:0/1296159150 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fb8dc144720 msgr2=0x7fb8dc144c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:23:23.814 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:23.811+0000 7fb8da59c640 1 --2- 172.21.15.143:0/1296159150 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fb8dc144720 0x7fb8dc144c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:23:23.814 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:23.811+0000 7fb8da59c640 1 -- 172.21.15.143:0/1296159150 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fb8dc071a20 msgr2=0x7fb8dc13c610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:23:23.814 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:23.811+0000 7fb8da59c640 1 --2- 172.21.15.143:0/1296159150 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fb8dc071a20 0x7fb8dc13c610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:23:23.814 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:23.811+0000 7fb8da59c640 1 -- 172.21.15.143:0/1296159150 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb8d00094d0 con 0x7fb8dc06d840 2023-11-26T07:23:23.815 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:23.811+0000 7fb8da59c640 1 --2- 172.21.15.143:0/1296159150 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fb8dc06d840 0x7fb8dc146620 secure :-1 s=READY pgs=1120 cs=0 l=1 rev1=1 crypto rx=0x7fb8cc00d800 tx=0x7fb8cc00dcc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:23:23.815 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:23.811+0000 7fb8caffd640 1 -- 172.21.15.143:0/1296159150 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb8cc00b6c0 con 0x7fb8dc06d840 2023-11-26T07:23:23.815 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:23.811+0000 7fb8caffd640 1 -- 172.21.15.143:0/1296159150 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb8cc00b820 con 0x7fb8dc06d840 2023-11-26T07:23:23.815 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:23.811+0000 7fb8db59e640 1 -- 172.21.15.143:0/1296159150 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb8dc145e20 con 0x7fb8dc06d840 2023-11-26T07:23:23.816 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:23.811+0000 7fb8caffd640 1 -- 172.21.15.143:0/1296159150 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb8cc00bbd0 con 0x7fb8dc06d840 2023-11-26T07:23:23.816 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:23.811+0000 7fb8db59e640 1 -- 172.21.15.143:0/1296159150 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb8dc13cb50 con 0x7fb8dc06d840 2023-11-26T07:23:23.816 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:23.815+0000 7fb8db59e640 1 -- 172.21.15.143:0/1296159150 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb8dc067500 con 0x7fb8dc06d840 2023-11-26T07:23:23.816 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:23.815+0000 7fb8caffd640 1 -- 172.21.15.143:0/1296159150 <== mon.0 v2:172.21.15.143:3300/0 4 ==== mgrmap(e 16) v1 ==== 85954+0+0 (secure 0 0 0) 0x7fb8cc00bd30 con 0x7fb8dc06d840 2023-11-26T07:23:23.816 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:23.815+0000 7fb8caffd640 1 --2- 172.21.15.143:0/1296159150 >> [v2:172.21.15.188:6836/3401128553,v1:172.21.15.188:6837/3401128553] conn(0x7fb8bc066590 0x7fb8bc068a50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:23:23.817 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:23.815+0000 7fb8caffd640 1 -- 172.21.15.143:0/1296159150 <== mon.0 v2:172.21.15.143:3300/0 5 ==== osd_map(75..75 src has 1..75) v4 ==== 5502+0+0 (secure 0 0 0) 0x7fb8cc0844d0 con 0x7fb8dc06d840 2023-11-26T07:23:23.817 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:23.815+0000 7fb8d9d9b640 1 --2- 172.21.15.143:0/1296159150 >> [v2:172.21.15.188:6836/3401128553,v1:172.21.15.188:6837/3401128553] conn(0x7fb8bc066590 0x7fb8bc068a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:23:23.818 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:23.815+0000 7fb8d9d9b640 1 --2- 172.21.15.143:0/1296159150 >> [v2:172.21.15.188:6836/3401128553,v1:172.21.15.188:6837/3401128553] conn(0x7fb8bc066590 0x7fb8bc068a50 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7fb8d00af000 tx=0x7fb8d00a7000 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:23:23.821 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:23.819+0000 7fb8caffd640 1 -- 172.21.15.143:0/1296159150 <== mon.0 v2:172.21.15.143:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172649 (secure 0 0 0) 0x7fb8cc051870 con 0x7fb8dc06d840 2023-11-26T07:23:23.985 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:23.983+0000 7fb8db59e640 1 -- 172.21.15.143:0/1296159150 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "osd blocklist clear"} v 0) v1 -- 0x7fb8dc06b4a0 con 0x7fb8dc06d840 2023-11-26T07:23:24.967 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:24.967+0000 7fb8caffd640 1 -- 172.21.15.143:0/1296159150 <== mon.0 v2:172.21.15.143:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist clear"}]=0 removed all blocklist entries v80) v1 ==== 97+0+0 (secure 0 0 0) 0x7fb8cc054e90 con 0x7fb8dc06d840 2023-11-26T07:23:24.967 INFO:teuthology.orchestra.run.smithi143.stderr: removed all blocklist entries 2023-11-26T07:23:24.970 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:24.967+0000 7fb8db59e640 1 -- 172.21.15.143:0/1296159150 >> [v2:172.21.15.188:6836/3401128553,v1:172.21.15.188:6837/3401128553] conn(0x7fb8bc066590 msgr2=0x7fb8bc068a50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:23:24.970 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:24.967+0000 7fb8db59e640 1 --2- 172.21.15.143:0/1296159150 >> [v2:172.21.15.188:6836/3401128553,v1:172.21.15.188:6837/3401128553] conn(0x7fb8bc066590 0x7fb8bc068a50 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7fb8d00af000 tx=0x7fb8d00a7000 comp rx=0 tx=0).stop 2023-11-26T07:23:24.970 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:24.967+0000 7fb8db59e640 1 -- 172.21.15.143:0/1296159150 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fb8dc06d840 msgr2=0x7fb8dc146620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:23:24.970 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:24.967+0000 7fb8db59e640 1 --2- 172.21.15.143:0/1296159150 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fb8dc06d840 0x7fb8dc146620 secure :-1 s=READY pgs=1120 cs=0 l=1 rev1=1 crypto rx=0x7fb8cc00d800 tx=0x7fb8cc00dcc0 comp rx=0 tx=0).stop 2023-11-26T07:23:24.970 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:24.967+0000 7fb8db59e640 1 -- 172.21.15.143:0/1296159150 shutdown_connections 2023-11-26T07:23:24.971 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:24.967+0000 7fb8db59e640 1 --2- 172.21.15.143:0/1296159150 >> [v2:172.21.15.188:6836/3401128553,v1:172.21.15.188:6837/3401128553] conn(0x7fb8bc066590 0x7fb8bc068a50 unknown :-1 s=CLOSED pgs=45 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:23:24.971 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:24.967+0000 7fb8db59e640 1 --2- 172.21.15.143:0/1296159150 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fb8dc144720 0x7fb8dc144c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:23:24.971 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:24.967+0000 7fb8db59e640 1 --2- 172.21.15.143:0/1296159150 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fb8dc071a20 0x7fb8dc13c610 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:23:24.971 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:24.967+0000 7fb8db59e640 1 --2- 172.21.15.143:0/1296159150 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fb8dc06d840 0x7fb8dc146620 unknown :-1 s=CLOSED pgs=1120 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:23:24.971 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:24.967+0000 7fb8db59e640 1 -- 172.21.15.143:0/1296159150 >> 172.21.15.143:0/1296159150 conn(0x7fb8dc06a890 msgr2=0x7fb8dc06d060 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:23:24.971 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:24.967+0000 7fb8db59e640 1 -- 172.21.15.143:0/1296159150 shutdown_connections 2023-11-26T07:23:24.971 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:24.967+0000 7fb8db59e640 1 -- 172.21.15.143:0/1296159150 wait complete. 2023-11-26T07:23:24.993 DEBUG:teuthology.orchestra.run.smithi143:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth ls --format=json-pretty 2023-11-26T07:23:25.134 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:25.131+0000 7fb12ecd6640 1 Processor -- start 2023-11-26T07:23:25.134 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:25.131+0000 7fb12ecd6640 1 -- start start 2023-11-26T07:23:25.136 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:25.135+0000 7fb12ecd6640 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7fb128143ac0 0x7fb128070370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:23:25.136 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:25.135+0000 7fb12ecd6640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fb1280708b0 0x7fb1280cf280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:23:25.136 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:25.135+0000 7fb12ecd6640 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7fb1280cf7c0 0x7fb1280b1ef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:23:25.137 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:25.135+0000 7fb12ecd6640 1 -- --> v2:172.21.15.143:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb128143ea0 con 0x7fb128143ac0 2023-11-26T07:23:25.137 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:25.135+0000 7fb12ecd6640 1 -- --> v2:172.21.15.188:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb1280cfbf0 con 0x7fb1280cf7c0 2023-11-26T07:23:25.137 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:25.135+0000 7fb12ecd6640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb1280cc730 con 0x7fb1280708b0 2023-11-26T07:23:25.137 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:25.135+0000 7fb12ca72640 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7fb128143ac0 0x7fb128070370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:23:25.137 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:25.135+0000 7fb12ca72640 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7fb128143ac0 0x7fb128070370 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:46178/0 (socket says 172.21.15.143:46178) 2023-11-26T07:23:25.137 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:25.135+0000 7fb12ca72640 1 -- 172.21.15.143:0/3236790866 learned_addr learned my addr 172.21.15.143:0/3236790866 (peer_addr_for_me v2:172.21.15.143:0/0) 2023-11-26T07:23:25.137 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:25.135+0000 7fb11ffff640 1 --2- 172.21.15.143:0/3236790866 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fb1280708b0 0x7fb1280cf280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:23:25.137 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:25.135+0000 7fb12d273640 1 --2- 172.21.15.143:0/3236790866 >> v2:172.21.15.188:3300/0 conn(0x7fb1280cf7c0 0x7fb1280b1ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:23:25.137 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:25.135+0000 7fb12ca72640 1 -- 172.21.15.143:0/3236790866 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fb1280708b0 msgr2=0x7fb1280cf280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-26T07:23:25.137 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:25.135+0000 7fb12ca72640 1 --2- 172.21.15.143:0/3236790866 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fb1280708b0 0x7fb1280cf280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:23:25.137 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:25.135+0000 7fb12ca72640 1 -- 172.21.15.143:0/3236790866 >> v2:172.21.15.188:3300/0 conn(0x7fb1280cf7c0 msgr2=0x7fb1280b1ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-26T07:23:25.137 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:25.135+0000 7fb12ca72640 1 --2- 172.21.15.143:0/3236790866 >> v2:172.21.15.188:3300/0 conn(0x7fb1280cf7c0 0x7fb1280b1ef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:23:25.138 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:25.135+0000 7fb12ca72640 1 -- 172.21.15.143:0/3236790866 --> v2:172.21.15.143:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb128070ee0 con 0x7fb128143ac0 2023-11-26T07:23:25.139 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:25.139+0000 7fb12ca72640 1 --2- 172.21.15.143:0/3236790866 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fb128143ac0 0x7fb128070370 secure :-1 s=READY pgs=1121 cs=0 l=1 rev1=1 crypto rx=0x7fb1280718d0 tx=0x7fb12401c240 comp rx=0 tx=0).ready entity=mon.0 client_cookie=15cddba624e7c800 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:23:25.139 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:25.139+0000 7fb11f7fe640 1 -- 172.21.15.143:0/3236790866 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb1240ab600 con 0x7fb128143ac0 2023-11-26T07:23:25.140 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:25.139+0000 7fb11f7fe640 1 -- 172.21.15.143:0/3236790866 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb1240a7050 con 0x7fb128143ac0 2023-11-26T07:23:25.140 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:25.139+0000 7fb11f7fe640 1 -- 172.21.15.143:0/3236790866 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb1240ab980 con 0x7fb128143ac0 2023-11-26T07:23:25.140 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:25.139+0000 7fb12ecd6640 1 -- 172.21.15.143:0/3236790866 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fb128143ac0 msgr2=0x7fb128070370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:23:25.140 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:25.139+0000 7fb12ecd6640 1 --2- 172.21.15.143:0/3236790866 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fb128143ac0 0x7fb128070370 secure :-1 s=READY pgs=1121 cs=0 l=1 rev1=1 crypto rx=0x7fb1280718d0 tx=0x7fb12401c240 comp rx=0 tx=0).stop 2023-11-26T07:23:25.141 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:25.139+0000 7fb12ecd6640 1 -- 172.21.15.143:0/3236790866 shutdown_connections 2023-11-26T07:23:25.141 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:25.139+0000 7fb12ecd6640 1 --2- 172.21.15.143:0/3236790866 >> v2:172.21.15.188:3300/0 conn(0x7fb1280cf7c0 0x7fb1280b1ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:23:25.141 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:25.139+0000 7fb12ecd6640 1 --2- 172.21.15.143:0/3236790866 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fb1280708b0 0x7fb1280cf280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:23:25.141 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:25.139+0000 7fb12ecd6640 1 --2- 172.21.15.143:0/3236790866 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fb128143ac0 0x7fb128070370 unknown :-1 s=CLOSED pgs=1121 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:23:25.141 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:25.139+0000 7fb12ecd6640 1 -- 172.21.15.143:0/3236790866 >> 172.21.15.143:0/3236790866 conn(0x7fb12806a890 msgr2=0x7fb12806aca0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:23:25.141 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:25.139+0000 7fb12ecd6640 1 -- 172.21.15.143:0/3236790866 shutdown_connections 2023-11-26T07:23:25.141 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:25.139+0000 7fb12ecd6640 1 -- 172.21.15.143:0/3236790866 wait complete. 2023-11-26T07:23:25.142 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:25.139+0000 7fb12ecd6640 1 Processor -- start 2023-11-26T07:23:25.142 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:25.139+0000 7fb12ecd6640 1 -- start start 2023-11-26T07:23:25.144 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:25.143+0000 7fb12ecd6640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fb1280708b0 0x7fb128139770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:23:25.144 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:25.143+0000 7fb12ecd6640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fb1280cf7c0 0x7fb1280b1440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:23:25.144 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:25.143+0000 7fb12ecd6640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fb128143ac0 0x7fb128140700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:23:25.144 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:25.143+0000 7fb12ecd6640 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb128139cb0 con 0x7fb1280708b0 2023-11-26T07:23:25.145 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:25.143+0000 7fb12ecd6640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb128157880 con 0x7fb128143ac0 2023-11-26T07:23:25.145 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:25.143+0000 7fb12ecd6640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb128157a20 con 0x7fb1280cf7c0 2023-11-26T07:23:25.145 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:25.143+0000 7fb12ca72640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fb1280708b0 0x7fb128139770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:23:25.145 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:25.143+0000 7fb12ca72640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fb1280708b0 0x7fb128139770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:46190/0 (socket says 172.21.15.143:46190) 2023-11-26T07:23:25.145 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:25.143+0000 7fb12ca72640 1 -- 172.21.15.143:0/1868766217 learned_addr learned my addr 172.21.15.143:0/1868766217 (peer_addr_for_me v2:172.21.15.143:0/0) 2023-11-26T07:23:25.145 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:25.143+0000 7fb12d273640 1 --2- 172.21.15.143:0/1868766217 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fb128143ac0 0x7fb128140700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:23:25.145 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:25.143+0000 7fb11ffff640 1 --2- 172.21.15.143:0/1868766217 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fb1280cf7c0 0x7fb1280b1440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:23:25.145 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:25.143+0000 7fb12ca72640 1 -- 172.21.15.143:0/1868766217 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fb1280cf7c0 msgr2=0x7fb1280b1440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:23:25.145 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:25.143+0000 7fb12ca72640 1 --2- 172.21.15.143:0/1868766217 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fb1280cf7c0 0x7fb1280b1440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:23:25.145 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:25.143+0000 7fb12ca72640 1 -- 172.21.15.143:0/1868766217 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fb128143ac0 msgr2=0x7fb128140700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:23:25.145 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:25.143+0000 7fb12ca72640 1 --2- 172.21.15.143:0/1868766217 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fb128143ac0 0x7fb128140700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:23:25.145 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:25.143+0000 7fb12ca72640 1 -- 172.21.15.143:0/1868766217 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb1240094d0 con 0x7fb1280708b0 2023-11-26T07:23:25.145 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:25.143+0000 7fb12ca72640 1 --2- 172.21.15.143:0/1868766217 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fb1280708b0 0x7fb128139770 secure :-1 s=READY pgs=1122 cs=0 l=1 rev1=1 crypto rx=0x7fb12401cae0 tx=0x7fb124018fe0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:23:25.145 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:25.143+0000 7fb11d7fa640 1 -- 172.21.15.143:0/1868766217 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb1240b4420 con 0x7fb1280708b0 2023-11-26T07:23:25.146 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:25.143+0000 7fb11d7fa640 1 -- 172.21.15.143:0/1868766217 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb1240b45a0 con 0x7fb1280708b0 2023-11-26T07:23:25.146 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:25.143+0000 7fb11d7fa640 1 -- 172.21.15.143:0/1868766217 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb1240b4950 con 0x7fb1280708b0 2023-11-26T07:23:25.146 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:25.143+0000 7fb12ecd6640 1 -- 172.21.15.143:0/1868766217 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb1280cde90 con 0x7fb1280708b0 2023-11-26T07:23:25.146 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:25.143+0000 7fb12ecd6640 1 -- 172.21.15.143:0/1868766217 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb1281418f0 con 0x7fb1280708b0 2023-11-26T07:23:25.147 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:25.143+0000 7fb12ecd6640 1 -- 172.21.15.143:0/1868766217 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb12813b660 con 0x7fb1280708b0 2023-11-26T07:23:25.147 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:25.147+0000 7fb11d7fa640 1 -- 172.21.15.143:0/1868766217 <== mon.0 v2:172.21.15.143:3300/0 4 ==== mgrmap(e 16) v1 ==== 85954+0+0 (secure 0 0 0) 0x7fb1240b2070 con 0x7fb1280708b0 2023-11-26T07:23:25.147 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:25.147+0000 7fb11d7fa640 1 --2- 172.21.15.143:0/1868766217 >> [v2:172.21.15.188:6836/3401128553,v1:172.21.15.188:6837/3401128553] conn(0x7fb114066590 0x7fb114068a50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:23:25.148 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:25.147+0000 7fb11d7fa640 1 -- 172.21.15.143:0/1868766217 <== mon.0 v2:172.21.15.143:3300/0 5 ==== osd_map(80..80 src has 1..80) v4 ==== 4857+0+0 (secure 0 0 0) 0x7fb12411d6c0 con 0x7fb1280708b0 2023-11-26T07:23:25.148 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:25.147+0000 7fb11ffff640 1 --2- 172.21.15.143:0/1868766217 >> [v2:172.21.15.188:6836/3401128553,v1:172.21.15.188:6837/3401128553] conn(0x7fb114066590 0x7fb114068a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:23:25.148 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:25.147+0000 7fb11ffff640 1 --2- 172.21.15.143:0/1868766217 >> [v2:172.21.15.188:6836/3401128553,v1:172.21.15.188:6837/3401128553] conn(0x7fb114066590 0x7fb114068a50 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7fb12813ed20 tx=0x7fb118009190 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:23:25.151 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:25.151+0000 7fb11d7fa640 1 -- 172.21.15.143:0/1868766217 <== mon.0 v2:172.21.15.143:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172649 (secure 0 0 0) 0x7fb1240eae30 con 0x7fb1280708b0 2023-11-26T07:23:25.351 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:25.351+0000 7fb12ecd6640 1 -- 172.21.15.143:0/1868766217 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "auth ls", "format": "json-pretty"} v 0) v1 -- 0x7fb128067680 con 0x7fb1280708b0 2023-11-26T07:23:25.352 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:25.351+0000 7fb11d7fa640 1 -- 172.21.15.143:0/1868766217 <== mon.0 v2:172.21.15.143:3300/0 7 ==== mon_command_ack([{"prefix": "auth ls", "format": "json-pretty"}]=0 v18) v1 ==== 80+0+5304 (secure 0 0 0) 0x7fb1240ee450 con 0x7fb1280708b0 2023-11-26T07:23:25.352 INFO:teuthology.orchestra.run.smithi143.stdout: 2023-11-26T07:23:25.352 INFO:teuthology.orchestra.run.smithi143.stdout:{ 2023-11-26T07:23:25.352 INFO:teuthology.orchestra.run.smithi143.stdout: "auth_dump": [ 2023-11-26T07:23:25.353 INFO:teuthology.orchestra.run.smithi143.stdout: { 2023-11-26T07:23:25.353 INFO:teuthology.orchestra.run.smithi143.stdout: "entity": "mds.a", 2023-11-26T07:23:25.353 INFO:teuthology.orchestra.run.smithi143.stdout: "key": "AQDl8GJldeBsOBAAhk75XZnQRURKcr0MrW17Lg==", 2023-11-26T07:23:25.353 INFO:teuthology.orchestra.run.smithi143.stdout: "caps": { 2023-11-26T07:23:25.353 INFO:teuthology.orchestra.run.smithi143.stdout: "mds": "allow", 2023-11-26T07:23:25.353 INFO:teuthology.orchestra.run.smithi143.stdout: "mgr": "allow *", 2023-11-26T07:23:25.353 INFO:teuthology.orchestra.run.smithi143.stdout: "mon": "allow *", 2023-11-26T07:23:25.353 INFO:teuthology.orchestra.run.smithi143.stdout: "osd": "allow *" 2023-11-26T07:23:25.353 INFO:teuthology.orchestra.run.smithi143.stdout: } 2023-11-26T07:23:25.353 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2023-11-26T07:23:25.354 INFO:teuthology.orchestra.run.smithi143.stdout: { 2023-11-26T07:23:25.354 INFO:teuthology.orchestra.run.smithi143.stdout: "entity": "mds.b", 2023-11-26T07:23:25.354 INFO:teuthology.orchestra.run.smithi143.stdout: "key": "AQDm8GJl0lvhCBAAXuvHKoB8TAhp0bJ4DcV94g==", 2023-11-26T07:23:25.354 INFO:teuthology.orchestra.run.smithi143.stdout: "caps": { 2023-11-26T07:23:25.354 INFO:teuthology.orchestra.run.smithi143.stdout: "mds": "allow", 2023-11-26T07:23:25.354 INFO:teuthology.orchestra.run.smithi143.stdout: "mgr": "allow *", 2023-11-26T07:23:25.354 INFO:teuthology.orchestra.run.smithi143.stdout: "mon": "allow *", 2023-11-26T07:23:25.354 INFO:teuthology.orchestra.run.smithi143.stdout: "osd": "allow *" 2023-11-26T07:23:25.354 INFO:teuthology.orchestra.run.smithi143.stdout: } 2023-11-26T07:23:25.354 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2023-11-26T07:23:25.354 INFO:teuthology.orchestra.run.smithi143.stdout: { 2023-11-26T07:23:25.355 INFO:teuthology.orchestra.run.smithi143.stdout: "entity": "mds.c", 2023-11-26T07:23:25.355 INFO:teuthology.orchestra.run.smithi143.stdout: "key": "AQDm8GJluQ0PBBAA23EmVdD3bVk52h6izUXDRQ==", 2023-11-26T07:23:25.355 INFO:teuthology.orchestra.run.smithi143.stdout: "caps": { 2023-11-26T07:23:25.355 INFO:teuthology.orchestra.run.smithi143.stdout: "mds": "allow", 2023-11-26T07:23:25.355 INFO:teuthology.orchestra.run.smithi143.stdout: "mgr": "allow *", 2023-11-26T07:23:25.355 INFO:teuthology.orchestra.run.smithi143.stdout: "mon": "allow *", 2023-11-26T07:23:25.355 INFO:teuthology.orchestra.run.smithi143.stdout: "osd": "allow *" 2023-11-26T07:23:25.355 INFO:teuthology.orchestra.run.smithi143.stdout: } 2023-11-26T07:23:25.355 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2023-11-26T07:23:25.356 INFO:teuthology.orchestra.run.smithi143.stdout: { 2023-11-26T07:23:25.356 INFO:teuthology.orchestra.run.smithi143.stdout: "entity": "mds.d", 2023-11-26T07:23:25.356 INFO:teuthology.orchestra.run.smithi143.stdout: "key": "AQDm8GJlkt9nEBAAg+LYtKVNHAN/fwYjEcNJHQ==", 2023-11-26T07:23:25.356 INFO:teuthology.orchestra.run.smithi143.stdout: "caps": { 2023-11-26T07:23:25.356 INFO:teuthology.orchestra.run.smithi143.stdout: "mds": "allow", 2023-11-26T07:23:25.356 INFO:teuthology.orchestra.run.smithi143.stdout: "mgr": "allow *", 2023-11-26T07:23:25.356 INFO:teuthology.orchestra.run.smithi143.stdout: "mon": "allow *", 2023-11-26T07:23:25.356 INFO:teuthology.orchestra.run.smithi143.stdout: "osd": "allow *" 2023-11-26T07:23:25.356 INFO:teuthology.orchestra.run.smithi143.stdout: } 2023-11-26T07:23:25.356 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2023-11-26T07:23:25.356 INFO:teuthology.orchestra.run.smithi143.stdout: { 2023-11-26T07:23:25.357 INFO:teuthology.orchestra.run.smithi143.stdout: "entity": "osd.0", 2023-11-26T07:23:25.357 INFO:teuthology.orchestra.run.smithi143.stdout: "key": "AQDp8GJlNscZLhAA31zK4vTcnxwiRIRgxzewpA==", 2023-11-26T07:23:25.357 INFO:teuthology.orchestra.run.smithi143.stdout: "caps": { 2023-11-26T07:23:25.357 INFO:teuthology.orchestra.run.smithi143.stdout: "mgr": "allow profile osd", 2023-11-26T07:23:25.357 INFO:teuthology.orchestra.run.smithi143.stdout: "mon": "allow profile osd", 2023-11-26T07:23:25.357 INFO:teuthology.orchestra.run.smithi143.stdout: "osd": "allow *" 2023-11-26T07:23:25.357 INFO:teuthology.orchestra.run.smithi143.stdout: } 2023-11-26T07:23:25.357 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2023-11-26T07:23:25.357 INFO:teuthology.orchestra.run.smithi143.stdout: { 2023-11-26T07:23:25.357 INFO:teuthology.orchestra.run.smithi143.stdout: "entity": "osd.1", 2023-11-26T07:23:25.357 INFO:teuthology.orchestra.run.smithi143.stdout: "key": "AQDs8GJlsyKUNBAA+dQ1ElGPrdP1GV7lk4JQjQ==", 2023-11-26T07:23:25.357 INFO:teuthology.orchestra.run.smithi143.stdout: "caps": { 2023-11-26T07:23:25.358 INFO:teuthology.orchestra.run.smithi143.stdout: "mgr": "allow profile osd", 2023-11-26T07:23:25.358 INFO:teuthology.orchestra.run.smithi143.stdout: "mon": "allow profile osd", 2023-11-26T07:23:25.358 INFO:teuthology.orchestra.run.smithi143.stdout: "osd": "allow *" 2023-11-26T07:23:25.358 INFO:teuthology.orchestra.run.smithi143.stdout: } 2023-11-26T07:23:25.358 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2023-11-26T07:23:25.358 INFO:teuthology.orchestra.run.smithi143.stdout: { 2023-11-26T07:23:25.358 INFO:teuthology.orchestra.run.smithi143.stdout: "entity": "osd.2", 2023-11-26T07:23:25.358 INFO:teuthology.orchestra.run.smithi143.stdout: "key": "AQDv8GJlXsy7OhAAJJdldsodgGZVtyV2HuyC4A==", 2023-11-26T07:23:25.358 INFO:teuthology.orchestra.run.smithi143.stdout: "caps": { 2023-11-26T07:23:25.358 INFO:teuthology.orchestra.run.smithi143.stdout: "mgr": "allow profile osd", 2023-11-26T07:23:25.358 INFO:teuthology.orchestra.run.smithi143.stdout: "mon": "allow profile osd", 2023-11-26T07:23:25.358 INFO:teuthology.orchestra.run.smithi143.stdout: "osd": "allow *" 2023-11-26T07:23:25.359 INFO:teuthology.orchestra.run.smithi143.stdout: } 2023-11-26T07:23:25.359 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2023-11-26T07:23:25.359 INFO:teuthology.orchestra.run.smithi143.stdout: { 2023-11-26T07:23:25.359 INFO:teuthology.orchestra.run.smithi143.stdout: "entity": "osd.3", 2023-11-26T07:23:25.359 INFO:teuthology.orchestra.run.smithi143.stdout: "key": "AQDz8GJloLhMBxAAOWD9ec2y4c8nkdJmQnne8Q==", 2023-11-26T07:23:25.359 INFO:teuthology.orchestra.run.smithi143.stdout: "caps": { 2023-11-26T07:23:25.359 INFO:teuthology.orchestra.run.smithi143.stdout: "mgr": "allow profile osd", 2023-11-26T07:23:25.359 INFO:teuthology.orchestra.run.smithi143.stdout: "mon": "allow profile osd", 2023-11-26T07:23:25.359 INFO:teuthology.orchestra.run.smithi143.stdout: "osd": "allow *" 2023-11-26T07:23:25.359 INFO:teuthology.orchestra.run.smithi143.stdout: } 2023-11-26T07:23:25.359 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2023-11-26T07:23:25.360 INFO:teuthology.orchestra.run.smithi143.stdout: { 2023-11-26T07:23:25.360 INFO:teuthology.orchestra.run.smithi143.stdout: "entity": "osd.4", 2023-11-26T07:23:25.360 INFO:teuthology.orchestra.run.smithi143.stdout: "key": "AQD58GJlDlZsDBAAZNauAPPPfAZ+KIqSmgSQhg==", 2023-11-26T07:23:25.360 INFO:teuthology.orchestra.run.smithi143.stdout: "caps": { 2023-11-26T07:23:25.360 INFO:teuthology.orchestra.run.smithi143.stdout: "mgr": "allow profile osd", 2023-11-26T07:23:25.360 INFO:teuthology.orchestra.run.smithi143.stdout: "mon": "allow profile osd", 2023-11-26T07:23:25.360 INFO:teuthology.orchestra.run.smithi143.stdout: "osd": "allow *" 2023-11-26T07:23:25.360 INFO:teuthology.orchestra.run.smithi143.stdout: } 2023-11-26T07:23:25.360 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2023-11-26T07:23:25.360 INFO:teuthology.orchestra.run.smithi143.stdout: { 2023-11-26T07:23:25.360 INFO:teuthology.orchestra.run.smithi143.stdout: "entity": "osd.5", 2023-11-26T07:23:25.361 INFO:teuthology.orchestra.run.smithi143.stdout: "key": "AQD88GJlNE4mExAA5jYkCa0tE3oR6rgK1Dt5cA==", 2023-11-26T07:23:25.361 INFO:teuthology.orchestra.run.smithi143.stdout: "caps": { 2023-11-26T07:23:25.361 INFO:teuthology.orchestra.run.smithi143.stdout: "mgr": "allow profile osd", 2023-11-26T07:23:25.361 INFO:teuthology.orchestra.run.smithi143.stdout: "mon": "allow profile osd", 2023-11-26T07:23:25.361 INFO:teuthology.orchestra.run.smithi143.stdout: "osd": "allow *" 2023-11-26T07:23:25.361 INFO:teuthology.orchestra.run.smithi143.stdout: } 2023-11-26T07:23:25.361 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2023-11-26T07:23:25.361 INFO:teuthology.orchestra.run.smithi143.stdout: { 2023-11-26T07:23:25.361 INFO:teuthology.orchestra.run.smithi143.stdout: "entity": "osd.6", 2023-11-26T07:23:25.361 INFO:teuthology.orchestra.run.smithi143.stdout: "key": "AQD/8GJlh0lYGRAArHalCyZ36UvBOVMZor+N/g==", 2023-11-26T07:23:25.361 INFO:teuthology.orchestra.run.smithi143.stdout: "caps": { 2023-11-26T07:23:25.362 INFO:teuthology.orchestra.run.smithi143.stdout: "mgr": "allow profile osd", 2023-11-26T07:23:25.362 INFO:teuthology.orchestra.run.smithi143.stdout: "mon": "allow profile osd", 2023-11-26T07:23:25.362 INFO:teuthology.orchestra.run.smithi143.stdout: "osd": "allow *" 2023-11-26T07:23:25.362 INFO:teuthology.orchestra.run.smithi143.stdout: } 2023-11-26T07:23:25.362 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2023-11-26T07:23:25.362 INFO:teuthology.orchestra.run.smithi143.stdout: { 2023-11-26T07:23:25.362 INFO:teuthology.orchestra.run.smithi143.stdout: "entity": "osd.7", 2023-11-26T07:23:25.362 INFO:teuthology.orchestra.run.smithi143.stdout: "key": "AQAC8WJlJavtIBAAKMvnR4AnnrDVOXlzN7j05w==", 2023-11-26T07:23:25.362 INFO:teuthology.orchestra.run.smithi143.stdout: "caps": { 2023-11-26T07:23:25.362 INFO:teuthology.orchestra.run.smithi143.stdout: "mgr": "allow profile osd", 2023-11-26T07:23:25.362 INFO:teuthology.orchestra.run.smithi143.stdout: "mon": "allow profile osd", 2023-11-26T07:23:25.363 INFO:teuthology.orchestra.run.smithi143.stdout: "osd": "allow *" 2023-11-26T07:23:25.363 INFO:teuthology.orchestra.run.smithi143.stdout: } 2023-11-26T07:23:25.363 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2023-11-26T07:23:25.363 INFO:teuthology.orchestra.run.smithi143.stdout: { 2023-11-26T07:23:25.363 INFO:teuthology.orchestra.run.smithi143.stdout: "entity": "client.0", 2023-11-26T07:23:25.363 INFO:teuthology.orchestra.run.smithi143.stdout: "key": "AQDm8GJlWMh9FBAAslVzi6fm4/wt7vr/rHCHHQ==", 2023-11-26T07:23:25.363 INFO:teuthology.orchestra.run.smithi143.stdout: "caps": { 2023-11-26T07:23:25.363 INFO:teuthology.orchestra.run.smithi143.stdout: "mds": "allow", 2023-11-26T07:23:25.363 INFO:teuthology.orchestra.run.smithi143.stdout: "mon": "allow r", 2023-11-26T07:23:25.363 INFO:teuthology.orchestra.run.smithi143.stdout: "osd": "allow rw tag cephfs data=cephfs" 2023-11-26T07:23:25.363 INFO:teuthology.orchestra.run.smithi143.stdout: } 2023-11-26T07:23:25.364 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2023-11-26T07:23:25.364 INFO:teuthology.orchestra.run.smithi143.stdout: { 2023-11-26T07:23:25.364 INFO:teuthology.orchestra.run.smithi143.stdout: "entity": "client.1", 2023-11-26T07:23:25.364 INFO:teuthology.orchestra.run.smithi143.stdout: "key": "AQDm8GJl5vkYHhAAgP+gKH2+lpl3aTlgqZNIXg==", 2023-11-26T07:23:25.364 INFO:teuthology.orchestra.run.smithi143.stdout: "caps": { 2023-11-26T07:23:25.364 INFO:teuthology.orchestra.run.smithi143.stdout: "mds": "allow", 2023-11-26T07:23:25.364 INFO:teuthology.orchestra.run.smithi143.stdout: "mgr": "allow r", 2023-11-26T07:23:25.364 INFO:teuthology.orchestra.run.smithi143.stdout: "mon": "allow rw", 2023-11-26T07:23:25.364 INFO:teuthology.orchestra.run.smithi143.stdout: "osd": "allow rwx" 2023-11-26T07:23:25.364 INFO:teuthology.orchestra.run.smithi143.stdout: } 2023-11-26T07:23:25.364 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2023-11-26T07:23:25.365 INFO:teuthology.orchestra.run.smithi143.stdout: { 2023-11-26T07:23:25.365 INFO:teuthology.orchestra.run.smithi143.stdout: "entity": "client.2", 2023-11-26T07:23:25.365 INFO:teuthology.orchestra.run.smithi143.stdout: "key": "AQDm8GJlW4KlGhAANFKsyVJems0ZoE6bo3WmfQ==", 2023-11-26T07:23:25.365 INFO:teuthology.orchestra.run.smithi143.stdout: "caps": { 2023-11-26T07:23:25.365 INFO:teuthology.orchestra.run.smithi143.stdout: "mds": "allow", 2023-11-26T07:23:25.365 INFO:teuthology.orchestra.run.smithi143.stdout: "mgr": "allow r", 2023-11-26T07:23:25.365 INFO:teuthology.orchestra.run.smithi143.stdout: "mon": "allow rw", 2023-11-26T07:23:25.365 INFO:teuthology.orchestra.run.smithi143.stdout: "osd": "allow rwx" 2023-11-26T07:23:25.365 INFO:teuthology.orchestra.run.smithi143.stdout: } 2023-11-26T07:23:25.365 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2023-11-26T07:23:25.365 INFO:teuthology.orchestra.run.smithi143.stdout: { 2023-11-26T07:23:25.366 INFO:teuthology.orchestra.run.smithi143.stdout: "entity": "client.3", 2023-11-26T07:23:25.366 INFO:teuthology.orchestra.run.smithi143.stdout: "key": "AQDm8GJlAJZSJBAAUSUNcwRBY01hOuilBJjlFQ==", 2023-11-26T07:23:25.366 INFO:teuthology.orchestra.run.smithi143.stdout: "caps": { 2023-11-26T07:23:25.366 INFO:teuthology.orchestra.run.smithi143.stdout: "mds": "allow", 2023-11-26T07:23:25.366 INFO:teuthology.orchestra.run.smithi143.stdout: "mgr": "allow r", 2023-11-26T07:23:25.366 INFO:teuthology.orchestra.run.smithi143.stdout: "mon": "allow rw", 2023-11-26T07:23:25.366 INFO:teuthology.orchestra.run.smithi143.stdout: "osd": "allow rwx" 2023-11-26T07:23:25.366 INFO:teuthology.orchestra.run.smithi143.stdout: } 2023-11-26T07:23:25.366 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2023-11-26T07:23:25.366 INFO:teuthology.orchestra.run.smithi143.stdout: { 2023-11-26T07:23:25.367 INFO:teuthology.orchestra.run.smithi143.stdout: "entity": "client.admin", 2023-11-26T07:23:25.367 INFO:teuthology.orchestra.run.smithi143.stdout: "key": "AQDl8GJlbQv7GxAA0Midh8tAO2KO+dH10OV8Xg==", 2023-11-26T07:23:25.367 INFO:teuthology.orchestra.run.smithi143.stdout: "caps": { 2023-11-26T07:23:25.367 INFO:teuthology.orchestra.run.smithi143.stdout: "mds": "allow *", 2023-11-26T07:23:25.367 INFO:teuthology.orchestra.run.smithi143.stdout: "mgr": "allow *", 2023-11-26T07:23:25.367 INFO:teuthology.orchestra.run.smithi143.stdout: "mon": "allow *", 2023-11-26T07:23:25.367 INFO:teuthology.orchestra.run.smithi143.stdout: "osd": "allow *" 2023-11-26T07:23:25.367 INFO:teuthology.orchestra.run.smithi143.stdout: } 2023-11-26T07:23:25.367 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2023-11-26T07:23:25.367 INFO:teuthology.orchestra.run.smithi143.stdout: { 2023-11-26T07:23:25.367 INFO:teuthology.orchestra.run.smithi143.stdout: "entity": "mgr.x", 2023-11-26T07:23:25.368 INFO:teuthology.orchestra.run.smithi143.stdout: "key": "AQDl8GJlXy8cNBAA4f5RDC1MASCqZPcJQvnUDA==", 2023-11-26T07:23:25.368 INFO:teuthology.orchestra.run.smithi143.stdout: "caps": { 2023-11-26T07:23:25.368 INFO:teuthology.orchestra.run.smithi143.stdout: "mds": "allow *", 2023-11-26T07:23:25.368 INFO:teuthology.orchestra.run.smithi143.stdout: "mon": "allow profile mgr", 2023-11-26T07:23:25.368 INFO:teuthology.orchestra.run.smithi143.stdout: "osd": "allow *" 2023-11-26T07:23:25.368 INFO:teuthology.orchestra.run.smithi143.stdout: } 2023-11-26T07:23:25.368 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2023-11-26T07:23:25.368 INFO:teuthology.orchestra.run.smithi143.stdout: { 2023-11-26T07:23:25.368 INFO:teuthology.orchestra.run.smithi143.stdout: "entity": "mgr.y", 2023-11-26T07:23:25.368 INFO:teuthology.orchestra.run.smithi143.stdout: "key": "AQDl8GJlfuFPMBAA3TYEFvdfk4DyGsBkbpv0hg==", 2023-11-26T07:23:25.369 INFO:teuthology.orchestra.run.smithi143.stdout: "caps": { 2023-11-26T07:23:25.369 INFO:teuthology.orchestra.run.smithi143.stdout: "mds": "allow *", 2023-11-26T07:23:25.369 INFO:teuthology.orchestra.run.smithi143.stdout: "mon": "allow profile mgr", 2023-11-26T07:23:25.369 INFO:teuthology.orchestra.run.smithi143.stdout: "osd": "allow *" 2023-11-26T07:23:25.369 INFO:teuthology.orchestra.run.smithi143.stdout: } 2023-11-26T07:23:25.369 INFO:teuthology.orchestra.run.smithi143.stdout: } 2023-11-26T07:23:25.369 INFO:teuthology.orchestra.run.smithi143.stdout: ] 2023-11-26T07:23:25.369 INFO:teuthology.orchestra.run.smithi143.stdout:} 2023-11-26T07:23:25.370 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:25.351+0000 7fb12ecd6640 1 -- 172.21.15.143:0/1868766217 >> [v2:172.21.15.188:6836/3401128553,v1:172.21.15.188:6837/3401128553] conn(0x7fb114066590 msgr2=0x7fb114068a50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:23:25.370 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:25.351+0000 7fb12ecd6640 1 --2- 172.21.15.143:0/1868766217 >> [v2:172.21.15.188:6836/3401128553,v1:172.21.15.188:6837/3401128553] conn(0x7fb114066590 0x7fb114068a50 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7fb12813ed20 tx=0x7fb118009190 comp rx=0 tx=0).stop 2023-11-26T07:23:25.371 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:25.351+0000 7fb12ecd6640 1 -- 172.21.15.143:0/1868766217 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fb1280708b0 msgr2=0x7fb128139770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:23:25.371 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:25.351+0000 7fb12ecd6640 1 --2- 172.21.15.143:0/1868766217 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fb1280708b0 0x7fb128139770 secure :-1 s=READY pgs=1122 cs=0 l=1 rev1=1 crypto rx=0x7fb12401cae0 tx=0x7fb124018fe0 comp rx=0 tx=0).stop 2023-11-26T07:23:25.371 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:25.351+0000 7fb12ecd6640 1 -- 172.21.15.143:0/1868766217 shutdown_connections 2023-11-26T07:23:25.371 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:25.351+0000 7fb12ecd6640 1 --2- 172.21.15.143:0/1868766217 >> [v2:172.21.15.188:6836/3401128553,v1:172.21.15.188:6837/3401128553] conn(0x7fb114066590 0x7fb114068a50 unknown :-1 s=CLOSED pgs=50 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:23:25.371 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:25.351+0000 7fb12ecd6640 1 --2- 172.21.15.143:0/1868766217 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fb128143ac0 0x7fb128140700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:23:25.371 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:25.351+0000 7fb12ecd6640 1 --2- 172.21.15.143:0/1868766217 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fb1280cf7c0 0x7fb1280b1440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:23:25.371 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:25.351+0000 7fb12ecd6640 1 --2- 172.21.15.143:0/1868766217 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fb1280708b0 0x7fb128139770 unknown :-1 s=CLOSED pgs=1122 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:23:25.371 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:25.351+0000 7fb12ecd6640 1 -- 172.21.15.143:0/1868766217 >> 172.21.15.143:0/1868766217 conn(0x7fb12806a890 msgr2=0x7fb1280caf60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:23:25.371 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:25.351+0000 7fb12ecd6640 1 -- 172.21.15.143:0/1868766217 shutdown_connections 2023-11-26T07:23:25.371 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:25.351+0000 7fb12ecd6640 1 -- 172.21.15.143:0/1868766217 wait complete. 2023-11-26T07:23:25.374 DEBUG:teuthology.orchestra.run.smithi143:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-11-26T07:23:25.514 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:25.511+0000 7f4ea9fa1640 1 Processor -- start 2023-11-26T07:23:25.514 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:25.511+0000 7f4ea9fa1640 1 -- start start 2023-11-26T07:23:25.514 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:25.511+0000 7f4ea9fa1640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f4ea40cf990 0x7f4ea415efa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:23:25.514 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:25.511+0000 7f4ea9fa1640 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7f4ea415f4e0 0x7f4ea40cd7b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:23:25.515 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:25.511+0000 7f4ea9fa1640 1 -- --> v1:172.21.15.188:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4ea406c520 con 0x7f4ea413e290 2023-11-26T07:23:25.515 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:25.511+0000 7f4ea9fa1640 1 -- --> v2:172.21.15.143:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4ea406c800 con 0x7f4ea415f4e0 2023-11-26T07:23:25.515 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:25.511+0000 7f4ea9fa1640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4ea406cae0 con 0x7f4ea40cf990 2023-11-26T07:23:25.515 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:25.511+0000 7f4ea3fff640 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7f4ea415f4e0 0x7f4ea40cd7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:23:25.515 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:25.511+0000 7f4ea3fff640 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7f4ea415f4e0 0x7f4ea40cd7b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:46198/0 (socket says 172.21.15.143:46198) 2023-11-26T07:23:25.515 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:25.511+0000 7f4ea3fff640 1 -- 172.21.15.143:0/3405504802 learned_addr learned my addr 172.21.15.143:0/3405504802 (peer_addr_for_me v2:172.21.15.143:0/0) 2023-11-26T07:23:25.515 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:25.511+0000 7f4ea2ffd640 1 --2- 172.21.15.143:0/3405504802 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f4ea40cf990 0x7f4ea415efa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:23:25.515 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:25.511+0000 7f4ea3fff640 1 -- 172.21.15.143:0/3405504802 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f4ea40cf990 msgr2=0x7f4ea415efa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-26T07:23:25.515 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:25.511+0000 7f4ea3fff640 1 --2- 172.21.15.143:0/3405504802 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f4ea40cf990 0x7f4ea415efa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:23:25.515 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:25.511+0000 7f4ea3fff640 1 -- 172.21.15.143:0/3405504802 >> v1:172.21.15.188:6789/0 conn(0x7f4ea413e290 legacy=0x7f4ea40cf280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:23:25.515 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:25.511+0000 7f4ea3fff640 1 -- 172.21.15.143:0/3405504802 --> v2:172.21.15.143:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4ea406cdc0 con 0x7f4ea415f4e0 2023-11-26T07:23:25.516 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:25.515+0000 7f4ea3fff640 1 --2- 172.21.15.143:0/3405504802 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f4ea415f4e0 0x7f4ea40cd7b0 secure :-1 s=READY pgs=1123 cs=0 l=1 rev1=1 crypto rx=0x7f4e9c009e50 tx=0x7f4e9c01c9e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=35b38379622be801 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:23:25.516 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:25.515+0000 7f4ea27fc640 1 -- 172.21.15.143:0/3405504802 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4e9c0a58d0 con 0x7f4ea415f4e0 2023-11-26T07:23:25.516 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:25.515+0000 7f4ea27fc640 1 -- 172.21.15.143:0/3405504802 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4e9c0a5a30 con 0x7f4ea415f4e0 2023-11-26T07:23:25.516 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:25.515+0000 7f4ea27fc640 1 -- 172.21.15.143:0/3405504802 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4e9c0a5dc0 con 0x7f4ea415f4e0 2023-11-26T07:23:25.517 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:25.515+0000 7f4ea9fa1640 1 -- 172.21.15.143:0/3405504802 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f4ea415f4e0 msgr2=0x7f4ea40cd7b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:23:25.517 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:25.515+0000 7f4ea9fa1640 1 --2- 172.21.15.143:0/3405504802 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f4ea415f4e0 0x7f4ea40cd7b0 secure :-1 s=READY pgs=1123 cs=0 l=1 rev1=1 crypto rx=0x7f4e9c009e50 tx=0x7f4e9c01c9e0 comp rx=0 tx=0).stop 2023-11-26T07:23:25.517 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:25.515+0000 7f4ea9fa1640 1 -- 172.21.15.143:0/3405504802 shutdown_connections 2023-11-26T07:23:25.517 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:25.515+0000 7f4ea9fa1640 1 --2- 172.21.15.143:0/3405504802 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f4ea415f4e0 0x7f4ea40cd7b0 unknown :-1 s=CLOSED pgs=1123 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:23:25.517 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:25.515+0000 7f4ea9fa1640 1 --2- 172.21.15.143:0/3405504802 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f4ea40cf990 0x7f4ea415efa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:23:25.518 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:25.515+0000 7f4ea9fa1640 1 -- 172.21.15.143:0/3405504802 >> 172.21.15.143:0/3405504802 conn(0x7f4ea406a890 msgr2=0x7f4ea406aca0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:23:25.518 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:25.515+0000 7f4ea9fa1640 1 -- 172.21.15.143:0/3405504802 shutdown_connections 2023-11-26T07:23:25.518 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:25.515+0000 7f4ea9fa1640 1 -- 172.21.15.143:0/3405504802 wait complete. 2023-11-26T07:23:25.518 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:25.519+0000 7f4ea9fa1640 1 Processor -- start 2023-11-26T07:23:25.519 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:25.519+0000 7f4ea9fa1640 1 -- start start 2023-11-26T07:23:25.520 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:25.519+0000 7f4ea9fa1640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f4ea40cf990 0x7f4ea41f5560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:23:25.520 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:25.519+0000 7f4ea9fa1640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f4ea413e290 0x7f4ea41f65b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:23:25.520 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:25.519+0000 7f4ea9fa1640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f4ea415f4e0 0x7f4ea4139900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:23:25.521 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:25.519+0000 7f4ea9fa1640 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4ea4154f70 con 0x7f4ea415f4e0 2023-11-26T07:23:25.521 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:25.519+0000 7f4ea9fa1640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4ea415bc80 con 0x7f4ea40cf990 2023-11-26T07:23:25.521 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:25.519+0000 7f4ea9fa1640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4ea415b9a0 con 0x7f4ea413e290 2023-11-26T07:23:25.521 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:25.519+0000 7f4ea3fff640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f4ea415f4e0 0x7f4ea4139900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:23:25.521 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:25.519+0000 7f4ea3fff640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f4ea415f4e0 0x7f4ea4139900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:46214/0 (socket says 172.21.15.143:46214) 2023-11-26T07:23:25.521 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:25.519+0000 7f4ea3fff640 1 -- 172.21.15.143:0/181846715 learned_addr learned my addr 172.21.15.143:0/181846715 (peer_addr_for_me v2:172.21.15.143:0/0) 2023-11-26T07:23:25.521 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:25.519+0000 7f4ea2ffd640 1 --2- 172.21.15.143:0/181846715 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f4ea413e290 0x7f4ea41f65b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:23:25.521 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:25.519+0000 7f4ea37fe640 1 --2- 172.21.15.143:0/181846715 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f4ea40cf990 0x7f4ea41f5560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:23:25.521 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:25.519+0000 7f4ea3fff640 1 -- 172.21.15.143:0/181846715 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f4ea413e290 msgr2=0x7f4ea41f65b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:23:25.521 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:25.519+0000 7f4ea3fff640 1 --2- 172.21.15.143:0/181846715 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f4ea413e290 0x7f4ea41f65b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:23:25.521 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:25.519+0000 7f4ea3fff640 1 -- 172.21.15.143:0/181846715 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f4ea40cf990 msgr2=0x7f4ea41f5560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:23:25.521 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:25.519+0000 7f4ea3fff640 1 --2- 172.21.15.143:0/181846715 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f4ea40cf990 0x7f4ea41f5560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:23:25.522 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:25.519+0000 7f4ea3fff640 1 -- 172.21.15.143:0/181846715 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4e9c009b30 con 0x7f4ea415f4e0 2023-11-26T07:23:25.522 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:25.519+0000 7f4ea3fff640 1 --2- 172.21.15.143:0/181846715 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f4ea415f4e0 0x7f4ea4139900 secure :-1 s=READY pgs=1124 cs=0 l=1 rev1=1 crypto rx=0x7f4e9c0af040 tx=0x7f4e9c019780 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:23:25.523 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:25.519+0000 7f4e8bfff640 1 -- 172.21.15.143:0/181846715 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4e9c0b8670 con 0x7f4ea415f4e0 2023-11-26T07:23:25.523 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:25.519+0000 7f4e8bfff640 1 -- 172.21.15.143:0/181846715 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4e9c0b87f0 con 0x7f4ea415f4e0 2023-11-26T07:23:25.523 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:25.519+0000 7f4ea9fa1640 1 -- 172.21.15.143:0/181846715 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4ea4154920 con 0x7f4ea415f4e0 2023-11-26T07:23:25.523 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:25.519+0000 7f4e8bfff640 1 -- 172.21.15.143:0/181846715 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4e9c0b8ba0 con 0x7f4ea415f4e0 2023-11-26T07:23:25.523 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:25.519+0000 7f4ea9fa1640 1 -- 172.21.15.143:0/181846715 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4ea414fa40 con 0x7f4ea415f4e0 2023-11-26T07:23:25.523 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:25.519+0000 7f4ea9fa1640 1 -- 172.21.15.143:0/181846715 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4ea4067680 con 0x7f4ea415f4e0 2023-11-26T07:23:25.523 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:25.523+0000 7f4e8bfff640 1 -- 172.21.15.143:0/181846715 <== mon.0 v2:172.21.15.143:3300/0 4 ==== mgrmap(e 16) v1 ==== 85954+0+0 (secure 0 0 0) 0x7f4e9c0b8d00 con 0x7f4ea415f4e0 2023-11-26T07:23:25.524 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:25.523+0000 7f4e8bfff640 1 --2- 172.21.15.143:0/181846715 >> [v2:172.21.15.188:6836/3401128553,v1:172.21.15.188:6837/3401128553] conn(0x7f4e900665c0 0x7f4e90068a80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:23:25.524 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:25.523+0000 7f4e8bfff640 1 -- 172.21.15.143:0/181846715 <== mon.0 v2:172.21.15.143:3300/0 5 ==== osd_map(80..80 src has 1..80) v4 ==== 4857+0+0 (secure 0 0 0) 0x7f4e9c11ec60 con 0x7f4ea415f4e0 2023-11-26T07:23:25.524 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:25.523+0000 7f4ea37fe640 1 --2- 172.21.15.143:0/181846715 >> [v2:172.21.15.188:6836/3401128553,v1:172.21.15.188:6837/3401128553] conn(0x7f4e900665c0 0x7f4e90068a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:23:25.525 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:25.523+0000 7f4ea37fe640 1 --2- 172.21.15.143:0/181846715 >> [v2:172.21.15.188:6836/3401128553,v1:172.21.15.188:6837/3401128553] conn(0x7f4e900665c0 0x7f4e90068a80 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7f4e94007fd0 tx=0x7f4e94009040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:23:25.527 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:25.527+0000 7f4e8bfff640 1 -- 172.21.15.143:0/181846715 <== mon.0 v2:172.21.15.143:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172649 (secure 0 0 0) 0x7f4e9c0ec2a0 con 0x7f4ea415f4e0 2023-11-26T07:23:25.681 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:25.679+0000 7f4ea9fa1640 1 -- 172.21.15.143:0/181846715 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f4ea41550b0 con 0x7f4ea415f4e0 2023-11-26T07:23:25.682 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:25.679+0000 7f4e8bfff640 1 -- 172.21.15.143:0/181846715 <== mon.0 v2:172.21.15.143:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v80) v1 ==== 74+0+12894 (secure 0 0 0) 0x7f4ea41550b0 con 0x7f4ea415f4e0 2023-11-26T07:23:25.682 INFO:teuthology.orchestra.run.smithi143.stdout: 2023-11-26T07:23:25.682 INFO:teuthology.orchestra.run.smithi143.stdout:{"epoch":80,"fsid":"f5033060-2469-4a07-95fe-2b76f5404bc1","created":"2023-11-26T07:17:28.613663+0000","modified":"2023-11-26T07:23:24.965501+0000","last_up_change":"2023-11-26T07:17:41.157813+0000","last_in_change":"2023-11-26T07:17:31.181658+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":11,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-11-26T07:17:41.758597+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"ebbee2c6-023e-44b9-9f1d-6f7bfd309730","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":69,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6817","nonce":639807369},{"type":"v1","addr":"172.21.15.143:6819","nonce":639807369}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6821","nonce":639807369},{"type":"v1","addr":"172.21.15.143:6823","nonce":639807369}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6830","nonce":639807369},{"type":"v1","addr":"172.21.15.143:6831","nonce":639807369}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6825","nonce":639807369},{"type":"v1","addr":"172.21.15.143:6828","nonce":639807369}]},"public_addr":"172.21.15.143:6819/639807369","cluster_addr":"172.21.15.143:6823/639807369","heartbeat_back_addr":"172.21.15.143:6831/639807369","heartbeat_front_addr":"172.21.15.143:6828/639807369","state":["exists","up"]},{"osd":1,"uuid":"fec4b8be-7122-49e9-9a19-5850a0700001","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":69,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6816","nonce":1144942530},{"type":"v1","addr":"172.21.15.143:6818","nonce":1144942530}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6820","nonce":1144942530},{"type":"v1","addr":"172.21.15.143:6822","nonce":1144942530}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6827","nonce":1144942530},{"type":"v1","addr":"172.21.15.143:6829","nonce":1144942530}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6824","nonce":1144942530},{"type":"v1","addr":"172.21.15.143:6826","nonce":1144942530}]},"public_addr":"172.21.15.143:6818/1144942530","cluster_addr":"172.21.15.143:6822/1144942530","heartbeat_back_addr":"172.21.15.143:6829/1144942530","heartbeat_front_addr":"172.21.15.143:6826/1144942530","state":["exists","up"]},{"osd":2,"uuid":"2888af5b-f933-46ca-8f95-6dd494ff43f6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":69,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6808","nonce":3909095186},{"type":"v1","addr":"172.21.15.143:6809","nonce":3909095186}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6810","nonce":3909095186},{"type":"v1","addr":"172.21.15.143:6811","nonce":3909095186}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6814","nonce":3909095186},{"type":"v1","addr":"172.21.15.143:6815","nonce":3909095186}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6812","nonce":3909095186},{"type":"v1","addr":"172.21.15.143:6813","nonce":3909095186}]},"public_addr":"172.21.15.143:6809/3909095186","cluster_addr":"172.21.15.143:6811/3909095186","heartbeat_back_addr":"172.21.15.143:6815/3909095186","heartbeat_front_addr":"172.21.15.143:6813/3909095186","state":["exists","up"]},{"osd":3,"uuid":"d1f52435-7bc1-4081-9550-9cd1ba55f549","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":69,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6800","nonce":3854210991},{"type":"v1","addr":"172.21.15.143:6801","nonce":3854210991}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6802","nonce":3854210991},{"type":"v1","addr":"172.21.15.143:6803","nonce":3854210991}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6806","nonce":3854210991},{"type":"v1","addr":"172.21.15.143:6807","nonce":3854210991}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6804","nonce":3854210991},{"type":"v1","addr":"172.21.15.143:6805","nonce":3854210991}]},"public_addr":"172.21.15.143:6801/3854210991","cluster_addr":"172.21.15.143:6803/3854210991","heartbeat_back_addr":"172.21.15.143:6807/3854210991","heartbeat_front_addr":"172.21.15.143:6805/3854210991","state":["exists","up"]},{"osd":4,"uuid":"e2ae8b6e-6e25-49b2-bcbf-aacce68f06a3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":69,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6800","nonce":3924606902},{"type":"v1","addr":"172.21.15.188:6801","nonce":3924606902}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6802","nonce":3924606902},{"type":"v1","addr":"172.21.15.188:6803","nonce":3924606902}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6806","nonce":3924606902},{"type":"v1","addr":"172.21.15.188:6807","nonce":3924606902}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6804","nonce":3924606902},{"type":"v1","addr":"172.21.15.188:6805","nonce":3924606902}]},"public_addr":"172.21.15.188:6801/3924606902","cluster_addr":"172.21.15.188:6803/3924606902","heartbeat_back_addr":"172.21.15.188:6807/3924606902","heartbeat_front_addr":"172.21.15.188:6805/3924606902","state":["exists","up"]},{"osd":5,"uuid":"3cec8a32-7698-4c36-8e89-e4ba11934f33","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":69,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6813","nonce":186841544},{"type":"v1","addr":"172.21.15.188:6815","nonce":186841544}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6817","nonce":186841544},{"type":"v1","addr":"172.21.15.188:6818","nonce":186841544}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6822","nonce":186841544},{"type":"v1","addr":"172.21.15.188:6823","nonce":186841544}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6820","nonce":186841544},{"type":"v1","addr":"172.21.15.188:6821","nonce":186841544}]},"public_addr":"172.21.15.188:6815/186841544","cluster_addr":"172.21.15.188:6818/186841544","heartbeat_back_addr":"172.21.15.188:6823/186841544","heartbeat_front_addr":"172.21.15.188:6821/186841544","state":["exists","up"]},{"osd":6,"uuid":"58a65d5d-e090-473a-81b4-ad38e85029f9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":69,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6808","nonce":2153499405},{"type":"v1","addr":"172.21.15.188:6809","nonce":2153499405}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6810","nonce":2153499405},{"type":"v1","addr":"172.21.15.188:6811","nonce":2153499405}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6816","nonce":2153499405},{"type":"v1","addr":"172.21.15.188:6819","nonce":2153499405}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6812","nonce":2153499405},{"type":"v1","addr":"172.21.15.188:6814","nonce":2153499405}]},"public_addr":"172.21.15.188:6809/2153499405","cluster_addr":"172.21.15.188:6811/2153499405","heartbeat_back_addr":"172.21.15.188:6819/2153499405","heartbeat_front_addr":"172.21.15.188:6814/2153499405","state":["exists","up"]},{"osd":7,"uuid":"ea5b7f70-3554-4cf8-ac30-f7c59b80678d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":69,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6824","nonce":2722096729},{"type":"v1","addr":"172.21.15.188:6825","nonce":2722096729}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6826","nonce":2722096729},{"type":"v1","addr":"172.21.15.188:6827","nonce":2722096729}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6830","nonce":2722096729},{"type":"v1","addr":"172.21.15.188:6831","nonce":2722096729}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6828","nonce":2722096729},{"type":"v1","addr":"172.21.15.188:6829","nonce":2722096729}]},"public_addr":"172.21.15.188:6825/2722096729","cluster_addr":"172.21.15.188:6827/2722096729","heartbeat_back_addr":"172.21.15.188:6831/2722096729","heartbeat_front_addr":"172.21.15.188:6829/2722096729","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-26T07:17:36.995032+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-26T07:17:36.932266+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-26T07:17:36.829304+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-26T07:17:36.804297+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-26T07:17:36.667892+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-26T07:17:36.964868+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-26T07:17:36.827332+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-26T07:17:36.893179+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-11-26T07:23:25.684 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:25.683+0000 7f4ea9fa1640 1 -- 172.21.15.143:0/181846715 >> [v2:172.21.15.188:6836/3401128553,v1:172.21.15.188:6837/3401128553] conn(0x7f4e900665c0 msgr2=0x7f4e90068a80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:23:25.684 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:25.683+0000 7f4ea9fa1640 1 --2- 172.21.15.143:0/181846715 >> [v2:172.21.15.188:6836/3401128553,v1:172.21.15.188:6837/3401128553] conn(0x7f4e900665c0 0x7f4e90068a80 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7f4e94007fd0 tx=0x7f4e94009040 comp rx=0 tx=0).stop 2023-11-26T07:23:25.684 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:25.683+0000 7f4ea9fa1640 1 -- 172.21.15.143:0/181846715 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f4ea415f4e0 msgr2=0x7f4ea4139900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:23:25.685 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:25.683+0000 7f4ea9fa1640 1 --2- 172.21.15.143:0/181846715 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f4ea415f4e0 0x7f4ea4139900 secure :-1 s=READY pgs=1124 cs=0 l=1 rev1=1 crypto rx=0x7f4e9c0af040 tx=0x7f4e9c019780 comp rx=0 tx=0).stop 2023-11-26T07:23:25.685 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:25.683+0000 7f4ea9fa1640 1 -- 172.21.15.143:0/181846715 shutdown_connections 2023-11-26T07:23:25.685 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:25.683+0000 7f4ea9fa1640 1 --2- 172.21.15.143:0/181846715 >> [v2:172.21.15.188:6836/3401128553,v1:172.21.15.188:6837/3401128553] conn(0x7f4e900665c0 0x7f4e90068a80 unknown :-1 s=CLOSED pgs=51 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:23:25.685 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:25.683+0000 7f4ea9fa1640 1 --2- 172.21.15.143:0/181846715 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f4ea415f4e0 0x7f4ea4139900 unknown :-1 s=CLOSED pgs=1124 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:23:25.685 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:25.683+0000 7f4ea9fa1640 1 --2- 172.21.15.143:0/181846715 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f4ea413e290 0x7f4ea41f65b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:23:25.685 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:25.683+0000 7f4ea9fa1640 1 --2- 172.21.15.143:0/181846715 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f4ea40cf990 0x7f4ea41f5560 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:23:25.685 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:25.683+0000 7f4ea9fa1640 1 -- 172.21.15.143:0/181846715 >> 172.21.15.143:0/181846715 conn(0x7f4ea406a890 msgr2=0x7f4ea40cf250 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:23:25.685 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:25.683+0000 7f4ea9fa1640 1 -- 172.21.15.143:0/181846715 shutdown_connections 2023-11-26T07:23:25.685 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:25.683+0000 7f4ea9fa1640 1 -- 172.21.15.143:0/181846715 wait complete. 2023-11-26T07:23:25.704 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-11-26T07:17:41.758597+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2023-11-26T07:23:25.704 DEBUG:teuthology.orchestra.run.smithi143:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-11-26T07:23:25.845 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:25.843+0000 7f143cae3640 1 Processor -- start 2023-11-26T07:23:25.845 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:25.843+0000 7f143cae3640 1 -- start start 2023-11-26T07:23:25.846 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:25.843+0000 7f143cae3640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f14381596d0 0x7f1438071850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:23:25.846 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:25.843+0000 7f143cae3640 1 -- --> v1:172.21.15.143:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f143806c750 con 0x7f14380b2f80 2023-11-26T07:23:25.846 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:25.843+0000 7f143cae3640 1 -- --> v1:172.21.15.188:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f14380b3550 con 0x7f14380ce680 2023-11-26T07:23:25.847 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:25.843+0000 7f143cae3640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f14380cbe00 con 0x7f14381596d0 2023-11-26T07:23:25.847 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:25.847+0000 7f1436d9d640 1 --1- >> v1:172.21.15.143:6789/0 conn(0x7f14380b2f80 0x7f143806cbe0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.143:6789/0 says I am v1:172.21.15.143:43122/0 (socket says 172.21.15.143:43122) 2023-11-26T07:23:25.847 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:25.847+0000 7f1436d9d640 1 -- 172.21.15.143:0/2427969383 learned_addr learned my addr 172.21.15.143:0/2427969383 (peer_addr_for_me v1:172.21.15.143:0/0) 2023-11-26T07:23:25.848 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:25.847+0000 7f143659c640 1 --2- 172.21.15.143:0/2427969383 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f14381596d0 0x7f1438071850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:23:25.848 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:25.847+0000 7f1435d9b640 1 -- 172.21.15.143:0/2427969383 <== mon.0 v1:172.21.15.143:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2977303042 0 0) 0x7f143806c750 con 0x7f14380b2f80 2023-11-26T07:23:25.848 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:25.847+0000 7f1435d9b640 1 -- 172.21.15.143:0/2427969383 --> v1:172.21.15.143:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f1424004640 con 0x7f14380b2f80 2023-11-26T07:23:25.848 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:25.847+0000 7f1435d9b640 1 -- 172.21.15.143:0/2427969383 <== mon.1 v1:172.21.15.188:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3732514311 0 0) 0x7f14380b3550 con 0x7f14380ce680 2023-11-26T07:23:25.848 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:25.847+0000 7f1435d9b640 1 -- 172.21.15.143:0/2427969383 --> v1:172.21.15.188:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f143806c750 con 0x7f14380ce680 2023-11-26T07:23:25.849 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:25.847+0000 7f143659c640 1 -- 172.21.15.143:0/2427969383 >> v1:172.21.15.188:6789/0 conn(0x7f14380ce680 legacy=0x7f14380b2870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:23:25.849 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:25.847+0000 7f143659c640 1 -- 172.21.15.143:0/2427969383 >> v1:172.21.15.143:6789/0 conn(0x7f14380b2f80 legacy=0x7f143806cbe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:23:25.849 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:25.847+0000 7f143659c640 1 -- 172.21.15.143:0/2427969383 --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1438071e80 con 0x7f14381596d0 2023-11-26T07:23:25.850 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:25.847+0000 7f143659c640 1 --2- 172.21.15.143:0/2427969383 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f14381596d0 0x7f1438071850 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7f14200097f0 tx=0x7f142001c4c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=669620cffa319d99 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:23:25.851 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:25.851+0000 7f1435d9b640 1 -- 172.21.15.143:0/2427969383 <== mon.2 v2:172.21.15.188:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f14200ab600 con 0x7f14381596d0 2023-11-26T07:23:25.851 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:25.851+0000 7f1435d9b640 1 -- 172.21.15.143:0/2427969383 <== mon.2 v2:172.21.15.188:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f14200a7050 con 0x7f14381596d0 2023-11-26T07:23:25.851 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:25.851+0000 7f1435d9b640 1 -- 172.21.15.143:0/2427969383 <== mon.2 v2:172.21.15.188:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f14200ab980 con 0x7f14381596d0 2023-11-26T07:23:25.852 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:25.851+0000 7f143cae3640 1 -- 172.21.15.143:0/2427969383 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f14381596d0 msgr2=0x7f1438071850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:23:25.852 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:25.851+0000 7f143cae3640 1 --2- 172.21.15.143:0/2427969383 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f14381596d0 0x7f1438071850 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7f14200097f0 tx=0x7f142001c4c0 comp rx=0 tx=0).stop 2023-11-26T07:23:25.852 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:25.851+0000 7f143cae3640 1 -- 172.21.15.143:0/2427969383 shutdown_connections 2023-11-26T07:23:25.852 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:25.851+0000 7f143cae3640 1 --2- 172.21.15.143:0/2427969383 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f14381596d0 0x7f1438071850 unknown :-1 s=CLOSED pgs=142 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:23:25.852 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:25.851+0000 7f143cae3640 1 -- 172.21.15.143:0/2427969383 >> 172.21.15.143:0/2427969383 conn(0x7f143806a890 msgr2=0x7f143806aca0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:23:25.853 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:25.851+0000 7f143cae3640 1 -- 172.21.15.143:0/2427969383 shutdown_connections 2023-11-26T07:23:25.853 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:25.851+0000 7f143cae3640 1 -- 172.21.15.143:0/2427969383 wait complete. 2023-11-26T07:23:25.853 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:25.851+0000 7f143cae3640 1 Processor -- start 2023-11-26T07:23:25.853 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:25.851+0000 7f143cae3640 1 -- start start 2023-11-26T07:23:25.857 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:25.851+0000 7f143cae3640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f14380b2f80 0x7f143815bc60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:23:25.857 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:25.851+0000 7f143cae3640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f14380ce680 0x7f143814a8c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:23:25.857 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:25.851+0000 7f143cae3640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f14381596d0 0x7f143814d620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:23:25.857 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:25.851+0000 7f143cae3640 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f14381514f0 con 0x7f14381596d0 2023-11-26T07:23:25.857 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:25.851+0000 7f143cae3640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f14381517d0 con 0x7f14380ce680 2023-11-26T07:23:25.858 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:25.851+0000 7f143cae3640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f14381f3ce0 con 0x7f14380b2f80 2023-11-26T07:23:25.858 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:25.855+0000 7f1436d9d640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f14381596d0 0x7f143814d620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:23:25.858 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:25.855+0000 7f1436d9d640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f14381596d0 0x7f143814d620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:46226/0 (socket says 172.21.15.143:46226) 2023-11-26T07:23:25.858 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:25.855+0000 7f1436d9d640 1 -- 172.21.15.143:0/2883891271 learned_addr learned my addr 172.21.15.143:0/2883891271 (peer_addr_for_me v2:172.21.15.143:0/0) 2023-11-26T07:23:25.858 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:25.855+0000 7f143659c640 1 --2- 172.21.15.143:0/2883891271 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f14380b2f80 0x7f143815bc60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:23:25.859 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:25.855+0000 7f142dd9b640 1 --2- 172.21.15.143:0/2883891271 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f14380ce680 0x7f143814a8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:23:25.859 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:25.855+0000 7f1436d9d640 1 -- 172.21.15.143:0/2883891271 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f14380b2f80 msgr2=0x7f143815bc60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:23:25.859 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:25.855+0000 7f1436d9d640 1 --2- 172.21.15.143:0/2883891271 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f14380b2f80 0x7f143815bc60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:23:25.859 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:25.855+0000 7f1436d9d640 1 -- 172.21.15.143:0/2883891271 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f14380ce680 msgr2=0x7f143814a8c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:23:25.859 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:25.855+0000 7f1436d9d640 1 --2- 172.21.15.143:0/2883891271 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f14380ce680 0x7f143814a8c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:23:25.859 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:25.855+0000 7f1436d9d640 1 -- 172.21.15.143:0/2883891271 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1420009420 con 0x7f14381596d0 2023-11-26T07:23:25.859 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:25.855+0000 7f142dd9b640 1 --2- 172.21.15.143:0/2883891271 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f14380ce680 0x7f143814a8c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-26T07:23:25.860 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:25.855+0000 7f1436d9d640 1 --2- 172.21.15.143:0/2883891271 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f14381596d0 0x7f143814d620 secure :-1 s=READY pgs=1126 cs=0 l=1 rev1=1 crypto rx=0x7f143000cb70 tx=0x7f143000a430 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:23:25.860 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:25.855+0000 7f142f7fe640 1 -- 172.21.15.143:0/2883891271 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f143000ace0 con 0x7f14381596d0 2023-11-26T07:23:25.860 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:25.855+0000 7f142f7fe640 1 -- 172.21.15.143:0/2883891271 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f143000ae40 con 0x7f14381596d0 2023-11-26T07:23:25.860 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:25.855+0000 7f143cae3640 1 -- 172.21.15.143:0/2883891271 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f14381f3fc0 con 0x7f14381596d0 2023-11-26T07:23:25.861 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:25.855+0000 7f142f7fe640 1 -- 172.21.15.143:0/2883891271 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1430012620 con 0x7f14381596d0 2023-11-26T07:23:25.861 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:25.855+0000 7f143cae3640 1 -- 172.21.15.143:0/2883891271 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1438151ab0 con 0x7f14381596d0 2023-11-26T07:23:25.861 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:25.855+0000 7f143cae3640 1 -- 172.21.15.143:0/2883891271 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f14381f4200 con 0x7f14381596d0 2023-11-26T07:23:25.862 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:25.855+0000 7f142f7fe640 1 -- 172.21.15.143:0/2883891271 <== mon.0 v2:172.21.15.143:3300/0 4 ==== mgrmap(e 16) v1 ==== 85954+0+0 (secure 0 0 0) 0x7f1430007070 con 0x7f14381596d0 2023-11-26T07:23:25.862 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:25.859+0000 7f142f7fe640 1 --2- 172.21.15.143:0/2883891271 >> [v2:172.21.15.188:6836/3401128553,v1:172.21.15.188:6837/3401128553] conn(0x7f1418066590 0x7f1418068a50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:23:25.862 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:25.859+0000 7f142f7fe640 1 -- 172.21.15.143:0/2883891271 <== mon.0 v2:172.21.15.143:3300/0 5 ==== osd_map(80..80 src has 1..80) v4 ==== 4857+0+0 (secure 0 0 0) 0x7f1430083bf0 con 0x7f14381596d0 2023-11-26T07:23:25.863 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:25.859+0000 7f143659c640 1 --2- 172.21.15.143:0/2883891271 >> [v2:172.21.15.188:6836/3401128553,v1:172.21.15.188:6837/3401128553] conn(0x7f1418066590 0x7f1418068a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:23:25.863 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:25.859+0000 7f143659c640 1 --2- 172.21.15.143:0/2883891271 >> [v2:172.21.15.188:6836/3401128553,v1:172.21.15.188:6837/3401128553] conn(0x7f1418066590 0x7f1418068a50 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7f14200ad000 tx=0x7f14200a6000 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:23:25.863 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:25.863+0000 7f142f7fe640 1 -- 172.21.15.143:0/2883891271 <== mon.0 v2:172.21.15.143:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172649 (secure 0 0 0) 0x7f14381f4200 con 0x7f14381596d0 2023-11-26T07:23:26.015 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:26.015+0000 7f143cae3640 1 -- 172.21.15.143:0/2883891271 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f14381f4200 con 0x7f14381596d0 2023-11-26T07:23:26.015 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:26.015+0000 7f142f7fe640 1 -- 172.21.15.143:0/2883891271 <== mon.0 v2:172.21.15.143:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v80) v1 ==== 93+0+10 (secure 0 0 0) 0x7f14381f4200 con 0x7f14381596d0 2023-11-26T07:23:26.016 INFO:teuthology.orchestra.run.smithi143.stdout:pg_num: 1 2023-11-26T07:23:26.017 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:26.015+0000 7f143cae3640 1 -- 172.21.15.143:0/2883891271 >> [v2:172.21.15.188:6836/3401128553,v1:172.21.15.188:6837/3401128553] conn(0x7f1418066590 msgr2=0x7f1418068a50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:23:26.018 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:26.015+0000 7f143cae3640 1 --2- 172.21.15.143:0/2883891271 >> [v2:172.21.15.188:6836/3401128553,v1:172.21.15.188:6837/3401128553] conn(0x7f1418066590 0x7f1418068a50 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7f14200ad000 tx=0x7f14200a6000 comp rx=0 tx=0).stop 2023-11-26T07:23:26.018 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:26.015+0000 7f143cae3640 1 -- 172.21.15.143:0/2883891271 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f14381596d0 msgr2=0x7f143814d620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:23:26.018 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:26.015+0000 7f143cae3640 1 --2- 172.21.15.143:0/2883891271 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f14381596d0 0x7f143814d620 secure :-1 s=READY pgs=1126 cs=0 l=1 rev1=1 crypto rx=0x7f143000cb70 tx=0x7f143000a430 comp rx=0 tx=0).stop 2023-11-26T07:23:26.018 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:26.015+0000 7f143cae3640 1 -- 172.21.15.143:0/2883891271 shutdown_connections 2023-11-26T07:23:26.018 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:26.015+0000 7f143cae3640 1 --2- 172.21.15.143:0/2883891271 >> [v2:172.21.15.188:6836/3401128553,v1:172.21.15.188:6837/3401128553] conn(0x7f1418066590 0x7f1418068a50 unknown :-1 s=CLOSED pgs=52 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:23:26.018 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:26.015+0000 7f143cae3640 1 --2- 172.21.15.143:0/2883891271 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f14381596d0 0x7f143814d620 unknown :-1 s=CLOSED pgs=1126 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:23:26.018 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:26.015+0000 7f143cae3640 1 --2- 172.21.15.143:0/2883891271 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f14380ce680 0x7f143814a8c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:23:26.018 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:26.015+0000 7f143cae3640 1 --2- 172.21.15.143:0/2883891271 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f14380b2f80 0x7f143815bc60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:23:26.018 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:26.015+0000 7f143cae3640 1 -- 172.21.15.143:0/2883891271 >> 172.21.15.143:0/2883891271 conn(0x7f143806a890 msgr2=0x7f14380cddf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:23:26.018 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:26.015+0000 7f143cae3640 1 -- 172.21.15.143:0/2883891271 shutdown_connections 2023-11-26T07:23:26.019 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:26.015+0000 7f143cae3640 1 -- 172.21.15.143:0/2883891271 wait complete. 2023-11-26T07:23:26.037 DEBUG:teuthology.orchestra.run.smithi143:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --format=json-pretty osd lspools 2023-11-26T07:23:26.174 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:26.171+0000 7ff892cfe640 1 Processor -- start 2023-11-26T07:23:26.175 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:26.171+0000 7ff892cfe640 1 -- start start 2023-11-26T07:23:26.176 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:26.175+0000 7ff892cfe640 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7ff88c13e290 0x7ff88c0cf280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:23:26.176 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:26.175+0000 7ff892cfe640 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7ff88c15f6b0 0x7ff88c0cd7b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:23:26.176 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:26.175+0000 7ff892cfe640 1 -- --> v1:172.21.15.143:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff88c15fb20 con 0x7ff88c0cf7c0 2023-11-26T07:23:26.176 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:26.175+0000 7ff892cfe640 1 -- --> v2:172.21.15.143:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff88c0cfc40 con 0x7ff88c15f6b0 2023-11-26T07:23:26.176 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:26.175+0000 7ff892cfe640 1 -- --> v2:172.21.15.188:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff88c06c520 con 0x7ff88c13e290 2023-11-26T07:23:26.176 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:26.175+0000 7ff89129b640 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7ff88c15f6b0 0x7ff88c0cd7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:23:26.176 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:26.175+0000 7ff883fff640 1 --1- >> v1:172.21.15.143:6789/0 conn(0x7ff88c0cf7c0 0x7ff88c15efa0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.143:6789/0 says I am v1:172.21.15.143:43128/0 (socket says 172.21.15.143:43128) 2023-11-26T07:23:26.176 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:26.175+0000 7ff883fff640 1 -- 172.21.15.143:0/3524290835 learned_addr learned my addr 172.21.15.143:0/3524290835 (peer_addr_for_me v1:172.21.15.143:0/0) 2023-11-26T07:23:26.177 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:26.175+0000 7ff890a9a640 1 --2- 172.21.15.143:0/3524290835 >> v2:172.21.15.188:3300/0 conn(0x7ff88c13e290 0x7ff88c0cf280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:23:26.177 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:26.175+0000 7ff89129b640 1 -- 172.21.15.143:0/3524290835 >> v2:172.21.15.188:3300/0 conn(0x7ff88c13e290 msgr2=0x7ff88c0cf280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-26T07:23:26.177 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:26.175+0000 7ff89129b640 1 --2- 172.21.15.143:0/3524290835 >> v2:172.21.15.188:3300/0 conn(0x7ff88c13e290 0x7ff88c0cf280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:23:26.177 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:26.175+0000 7ff89129b640 1 -- 172.21.15.143:0/3524290835 >> v1:172.21.15.143:6789/0 conn(0x7ff88c0cf7c0 legacy=0x7ff88c15efa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:23:26.177 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:26.175+0000 7ff89129b640 1 -- 172.21.15.143:0/3524290835 --> v2:172.21.15.143:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff88c06c800 con 0x7ff88c15f6b0 2023-11-26T07:23:26.178 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:26.179+0000 7ff89129b640 1 --2- 172.21.15.143:0/3524290835 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7ff88c15f6b0 0x7ff88c0cd7b0 secure :-1 s=READY pgs=1128 cs=0 l=1 rev1=1 crypto rx=0x7ff88c0725a0 tx=0x7ff88401c8e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ecae7eb6b54bc92a server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:23:26.179 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:26.179+0000 7ff8837fe640 1 -- 172.21.15.143:0/3524290835 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff8840a59a0 con 0x7ff88c15f6b0 2023-11-26T07:23:26.180 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:26.179+0000 7ff8837fe640 1 -- 172.21.15.143:0/3524290835 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff8840a5b00 con 0x7ff88c15f6b0 2023-11-26T07:23:26.180 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:26.179+0000 7ff8837fe640 1 -- 172.21.15.143:0/3524290835 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff8840a5e90 con 0x7ff88c15f6b0 2023-11-26T07:23:26.180 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:26.179+0000 7ff892cfe640 1 -- 172.21.15.143:0/3524290835 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7ff88c15f6b0 msgr2=0x7ff88c0cd7b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:23:26.180 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:26.179+0000 7ff892cfe640 1 --2- 172.21.15.143:0/3524290835 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7ff88c15f6b0 0x7ff88c0cd7b0 secure :-1 s=READY pgs=1128 cs=0 l=1 rev1=1 crypto rx=0x7ff88c0725a0 tx=0x7ff88401c8e0 comp rx=0 tx=0).stop 2023-11-26T07:23:26.180 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:26.179+0000 7ff892cfe640 1 -- 172.21.15.143:0/3524290835 shutdown_connections 2023-11-26T07:23:26.180 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:26.179+0000 7ff892cfe640 1 --2- 172.21.15.143:0/3524290835 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7ff88c15f6b0 0x7ff88c0cd7b0 unknown :-1 s=CLOSED pgs=1128 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:23:26.180 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:26.179+0000 7ff892cfe640 1 --2- 172.21.15.143:0/3524290835 >> v2:172.21.15.188:3300/0 conn(0x7ff88c13e290 0x7ff88c0cf280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:23:26.180 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:26.179+0000 7ff892cfe640 1 -- 172.21.15.143:0/3524290835 >> 172.21.15.143:0/3524290835 conn(0x7ff88c06a890 msgr2=0x7ff88c06aca0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:23:26.180 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:26.179+0000 7ff892cfe640 1 -- 172.21.15.143:0/3524290835 shutdown_connections 2023-11-26T07:23:26.180 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:26.179+0000 7ff892cfe640 1 -- 172.21.15.143:0/3524290835 wait complete. 2023-11-26T07:23:26.181 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:26.179+0000 7ff892cfe640 1 Processor -- start 2023-11-26T07:23:26.181 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:26.179+0000 7ff892cfe640 1 -- start start 2023-11-26T07:23:26.184 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:26.179+0000 7ff892cfe640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7ff88c0cf7c0 0x7ff88c0b3e40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:23:26.184 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:26.179+0000 7ff892cfe640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7ff88c13e290 0x7ff88c154a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:23:26.184 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:26.179+0000 7ff892cfe640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7ff88c15f6b0 0x7ff88c0aebf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:23:26.184 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:26.179+0000 7ff892cfe640 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff88c0b4380 con 0x7ff88c15f6b0 2023-11-26T07:23:26.184 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:26.179+0000 7ff892cfe640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff88c15dc20 con 0x7ff88c13e290 2023-11-26T07:23:26.184 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:26.179+0000 7ff892cfe640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff88c15d010 con 0x7ff88c0cf7c0 2023-11-26T07:23:26.185 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:26.183+0000 7ff89129b640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7ff88c15f6b0 0x7ff88c0aebf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:23:26.185 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:26.183+0000 7ff89129b640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7ff88c15f6b0 0x7ff88c0aebf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:46244/0 (socket says 172.21.15.143:46244) 2023-11-26T07:23:26.185 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:26.183+0000 7ff89129b640 1 -- 172.21.15.143:0/3339547210 learned_addr learned my addr 172.21.15.143:0/3339547210 (peer_addr_for_me v2:172.21.15.143:0/0) 2023-11-26T07:23:26.185 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:26.183+0000 7ff883fff640 1 --2- 172.21.15.143:0/3339547210 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7ff88c13e290 0x7ff88c154a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:23:26.185 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:26.183+0000 7ff890a9a640 1 --2- 172.21.15.143:0/3339547210 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7ff88c0cf7c0 0x7ff88c0b3e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:23:26.185 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:26.183+0000 7ff89129b640 1 -- 172.21.15.143:0/3339547210 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7ff88c0cf7c0 msgr2=0x7ff88c0b3e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:23:26.185 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:26.183+0000 7ff89129b640 1 --2- 172.21.15.143:0/3339547210 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7ff88c0cf7c0 0x7ff88c0b3e40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:23:26.186 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:26.183+0000 7ff89129b640 1 -- 172.21.15.143:0/3339547210 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7ff88c13e290 msgr2=0x7ff88c154a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:23:26.186 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:26.183+0000 7ff89129b640 1 --2- 172.21.15.143:0/3339547210 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7ff88c13e290 0x7ff88c154a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:23:26.186 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:26.183+0000 7ff89129b640 1 -- 172.21.15.143:0/3339547210 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff884009a70 con 0x7ff88c15f6b0 2023-11-26T07:23:26.186 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:26.183+0000 7ff890a9a640 1 --2- 172.21.15.143:0/3339547210 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7ff88c0cf7c0 0x7ff88c0b3e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-26T07:23:26.186 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:26.183+0000 7ff89129b640 1 --2- 172.21.15.143:0/3339547210 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7ff88c15f6b0 0x7ff88c0aebf0 secure :-1 s=READY pgs=1129 cs=0 l=1 rev1=1 crypto rx=0x7ff8840194f0 tx=0x7ff884019680 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:23:26.186 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:26.183+0000 7ff8817fa640 1 -- 172.21.15.143:0/3339547210 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff8840ade00 con 0x7ff88c15f6b0 2023-11-26T07:23:26.186 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:26.183+0000 7ff8817fa640 1 -- 172.21.15.143:0/3339547210 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff884009450 con 0x7ff88c15f6b0 2023-11-26T07:23:26.186 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:26.183+0000 7ff8817fa640 1 -- 172.21.15.143:0/3339547210 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff8840b9690 con 0x7ff88c15f6b0 2023-11-26T07:23:26.187 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:26.183+0000 7ff892cfe640 1 -- 172.21.15.143:0/3339547210 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff88c15d2f0 con 0x7ff88c15f6b0 2023-11-26T07:23:26.187 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:26.183+0000 7ff892cfe640 1 -- 172.21.15.143:0/3339547210 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff88c15d8b0 con 0x7ff88c15f6b0 2023-11-26T07:23:26.188 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:26.183+0000 7ff892cfe640 1 -- 172.21.15.143:0/3339547210 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff88c15dd90 con 0x7ff88c15f6b0 2023-11-26T07:23:26.188 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:26.183+0000 7ff8817fa640 1 -- 172.21.15.143:0/3339547210 <== mon.0 v2:172.21.15.143:3300/0 4 ==== mgrmap(e 16) v1 ==== 85954+0+0 (secure 0 0 0) 0x7ff8840b97f0 con 0x7ff88c15f6b0 2023-11-26T07:23:26.188 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:26.183+0000 7ff8817fa640 1 --2- 172.21.15.143:0/3339547210 >> [v2:172.21.15.188:6836/3401128553,v1:172.21.15.188:6837/3401128553] conn(0x7ff878066590 0x7ff878068a50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:23:26.188 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:26.183+0000 7ff8817fa640 1 -- 172.21.15.143:0/3339547210 <== mon.0 v2:172.21.15.143:3300/0 5 ==== osd_map(80..80 src has 1..80) v4 ==== 4857+0+0 (secure 0 0 0) 0x7ff88411f410 con 0x7ff88c15f6b0 2023-11-26T07:23:26.188 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:26.187+0000 7ff890a9a640 1 --2- 172.21.15.143:0/3339547210 >> [v2:172.21.15.188:6836/3401128553,v1:172.21.15.188:6837/3401128553] conn(0x7ff878066590 0x7ff878068a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:23:26.188 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:26.187+0000 7ff890a9a640 1 --2- 172.21.15.143:0/3339547210 >> [v2:172.21.15.188:6836/3401128553,v1:172.21.15.188:6837/3401128553] conn(0x7ff878066590 0x7ff878068a50 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7ff888007a70 tx=0x7ff888008040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:23:26.190 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:26.187+0000 7ff8817fa640 1 -- 172.21.15.143:0/3339547210 <== mon.0 v2:172.21.15.143:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172649 (secure 0 0 0) 0x7ff8840ecac0 con 0x7ff88c15f6b0 2023-11-26T07:23:26.343 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:26.339+0000 7ff892cfe640 1 -- 172.21.15.143:0/3339547210 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "osd lspools", "format": "json-pretty"} v 0) v1 -- 0x7ff88c067680 con 0x7ff88c15f6b0 2023-11-26T07:23:26.343 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:26.343+0000 7ff8817fa640 1 -- 172.21.15.143:0/3339547210 <== mon.0 v2:172.21.15.143:3300/0 7 ==== mon_command_ack([{"prefix": "osd lspools", "format": "json-pretty"}]=0 v80) v1 ==== 84+0+65 (secure 0 0 0) 0x7ff8840f00e0 con 0x7ff88c15f6b0 2023-11-26T07:23:26.343 INFO:teuthology.orchestra.run.smithi143.stdout: 2023-11-26T07:23:26.343 INFO:teuthology.orchestra.run.smithi143.stdout:[ 2023-11-26T07:23:26.343 INFO:teuthology.orchestra.run.smithi143.stdout: { 2023-11-26T07:23:26.344 INFO:teuthology.orchestra.run.smithi143.stdout: "poolnum": 1, 2023-11-26T07:23:26.344 INFO:teuthology.orchestra.run.smithi143.stdout: "poolname": ".mgr" 2023-11-26T07:23:26.344 INFO:teuthology.orchestra.run.smithi143.stdout: } 2023-11-26T07:23:26.344 INFO:teuthology.orchestra.run.smithi143.stdout:] 2023-11-26T07:23:26.345 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:26.343+0000 7ff892cfe640 1 -- 172.21.15.143:0/3339547210 >> [v2:172.21.15.188:6836/3401128553,v1:172.21.15.188:6837/3401128553] conn(0x7ff878066590 msgr2=0x7ff878068a50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:23:26.345 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:26.343+0000 7ff892cfe640 1 --2- 172.21.15.143:0/3339547210 >> [v2:172.21.15.188:6836/3401128553,v1:172.21.15.188:6837/3401128553] conn(0x7ff878066590 0x7ff878068a50 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7ff888007a70 tx=0x7ff888008040 comp rx=0 tx=0).stop 2023-11-26T07:23:26.345 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:26.343+0000 7ff892cfe640 1 -- 172.21.15.143:0/3339547210 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7ff88c15f6b0 msgr2=0x7ff88c0aebf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:23:26.345 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:26.343+0000 7ff892cfe640 1 --2- 172.21.15.143:0/3339547210 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7ff88c15f6b0 0x7ff88c0aebf0 secure :-1 s=READY pgs=1129 cs=0 l=1 rev1=1 crypto rx=0x7ff8840194f0 tx=0x7ff884019680 comp rx=0 tx=0).stop 2023-11-26T07:23:26.346 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:26.343+0000 7ff892cfe640 1 -- 172.21.15.143:0/3339547210 shutdown_connections 2023-11-26T07:23:26.346 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:26.343+0000 7ff892cfe640 1 --2- 172.21.15.143:0/3339547210 >> [v2:172.21.15.188:6836/3401128553,v1:172.21.15.188:6837/3401128553] conn(0x7ff878066590 0x7ff878068a50 unknown :-1 s=CLOSED pgs=53 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:23:26.346 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:26.343+0000 7ff892cfe640 1 --2- 172.21.15.143:0/3339547210 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7ff88c15f6b0 0x7ff88c0aebf0 unknown :-1 s=CLOSED pgs=1129 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:23:26.346 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:26.343+0000 7ff892cfe640 1 --2- 172.21.15.143:0/3339547210 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7ff88c13e290 0x7ff88c154a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:23:26.346 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:26.343+0000 7ff892cfe640 1 --2- 172.21.15.143:0/3339547210 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7ff88c0cf7c0 0x7ff88c0b3e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:23:26.346 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:26.343+0000 7ff892cfe640 1 -- 172.21.15.143:0/3339547210 >> 172.21.15.143:0/3339547210 conn(0x7ff88c06a890 msgr2=0x7ff88c13c720 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:23:26.346 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:26.343+0000 7ff892cfe640 1 -- 172.21.15.143:0/3339547210 shutdown_connections 2023-11-26T07:23:26.346 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:26.343+0000 7ff892cfe640 1 -- 172.21.15.143:0/3339547210 wait complete. 2023-11-26T07:23:26.364 DEBUG:tasks.cephfs.filesystem:Creating filesystem 'cephfs' 2023-11-26T07:23:26.364 DEBUG:teuthology.orchestra.run.smithi143:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_metadata --pg_num_min 64 2023-11-26T07:23:26.507 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:26.507+0000 7f1eb529c640 1 Processor -- start 2023-11-26T07:23:26.507 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:26.507+0000 7f1eb529c640 1 -- start start 2023-11-26T07:23:26.508 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:26.507+0000 7f1eb529c640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f1eb0071bb0 0x7f1eb0072030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:23:26.508 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:26.507+0000 7f1eb529c640 1 -- --> v1:172.21.15.143:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1eb00ceb50 con 0x7f1eb0143780 2023-11-26T07:23:26.508 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:26.507+0000 7f1eb529c640 1 -- --> v1:172.21.15.188:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1eb015c110 con 0x7f1eb015c5a0 2023-11-26T07:23:26.508 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:26.507+0000 7f1eb529c640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1eb0072570 con 0x7f1eb0071bb0 2023-11-26T07:23:26.509 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:26.507+0000 7f1eaeffd640 1 --1- >> v1:172.21.15.143:6789/0 conn(0x7f1eb0143780 0x7f1eb00cdfe0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.143:6789/0 says I am v1:172.21.15.143:43136/0 (socket says 172.21.15.143:43136) 2023-11-26T07:23:26.509 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:26.507+0000 7f1eaeffd640 1 -- 172.21.15.143:0/504488874 learned_addr learned my addr 172.21.15.143:0/504488874 (peer_addr_for_me v1:172.21.15.143:0/0) 2023-11-26T07:23:26.509 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:26.507+0000 7f1ea67fc640 1 --2- 172.21.15.143:0/504488874 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f1eb0071bb0 0x7f1eb0072030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:23:26.509 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:26.507+0000 7f1eae7fc640 1 -- 172.21.15.143:0/504488874 <== mon.0 v1:172.21.15.143:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2205799354 0 0) 0x7f1ea00014a0 con 0x7f1eb0143780 2023-11-26T07:23:26.510 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:26.507+0000 7f1eae7fc640 1 -- 172.21.15.143:0/504488874 --> v1:172.21.15.143:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f1e9c005740 con 0x7f1eb0143780 2023-11-26T07:23:26.510 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:26.507+0000 7f1ea67fc640 1 -- 172.21.15.143:0/504488874 >> v1:172.21.15.188:6789/0 conn(0x7f1eb015c5a0 legacy=0x7f1eb015ca30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:23:26.510 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:26.507+0000 7f1ea67fc640 1 -- 172.21.15.143:0/504488874 >> v1:172.21.15.143:6789/0 conn(0x7f1eb0143780 legacy=0x7f1eb00cdfe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:23:26.510 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:26.507+0000 7f1ea67fc640 1 -- 172.21.15.143:0/504488874 --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1eb015b7e0 con 0x7f1eb0071bb0 2023-11-26T07:23:26.512 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:26.511+0000 7f1ea67fc640 1 --2- 172.21.15.143:0/504488874 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f1eb0071bb0 0x7f1eb0072030 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7f1e980097f0 tx=0x7f1e9801c4c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=742e86de6478973c server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:23:26.512 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:26.511+0000 7f1eae7fc640 1 -- 172.21.15.143:0/504488874 <== mon.2 v2:172.21.15.188:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1e980aa620 con 0x7f1eb0071bb0 2023-11-26T07:23:26.512 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:26.511+0000 7f1eae7fc640 1 -- 172.21.15.143:0/504488874 <== mon.2 v2:172.21.15.188:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1e980aa780 con 0x7f1eb0071bb0 2023-11-26T07:23:26.512 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:26.511+0000 7f1eae7fc640 1 -- 172.21.15.143:0/504488874 <== mon.2 v2:172.21.15.188:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1e980aab10 con 0x7f1eb0071bb0 2023-11-26T07:23:26.513 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:26.511+0000 7f1eb529c640 1 -- 172.21.15.143:0/504488874 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f1eb0071bb0 msgr2=0x7f1eb0072030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:23:26.513 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:26.511+0000 7f1eb529c640 1 --2- 172.21.15.143:0/504488874 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f1eb0071bb0 0x7f1eb0072030 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7f1e980097f0 tx=0x7f1e9801c4c0 comp rx=0 tx=0).stop 2023-11-26T07:23:26.513 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:26.511+0000 7f1eb529c640 1 -- 172.21.15.143:0/504488874 shutdown_connections 2023-11-26T07:23:26.513 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:26.511+0000 7f1eb529c640 1 --2- 172.21.15.143:0/504488874 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f1eb0071bb0 0x7f1eb0072030 unknown :-1 s=CLOSED pgs=143 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:23:26.513 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:26.511+0000 7f1eb529c640 1 -- 172.21.15.143:0/504488874 >> 172.21.15.143:0/504488874 conn(0x7f1eb006c3f0 msgr2=0x7f1eb006c800 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:23:26.514 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:26.511+0000 7f1eb529c640 1 -- 172.21.15.143:0/504488874 shutdown_connections 2023-11-26T07:23:26.514 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:26.511+0000 7f1eb529c640 1 -- 172.21.15.143:0/504488874 wait complete. 2023-11-26T07:23:26.514 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:26.511+0000 7f1eb529c640 1 Processor -- start 2023-11-26T07:23:26.515 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:26.515+0000 7f1eb529c640 1 -- start start 2023-11-26T07:23:26.518 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:26.515+0000 7f1eb529c640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f1eb0071bb0 0x7f1eb00af040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:23:26.518 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:26.515+0000 7f1eb529c640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f1eb0143780 0x7f1eb01594c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:23:26.518 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:26.515+0000 7f1eb529c640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f1eb015c5a0 0x7f1eb0159a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:23:26.518 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:26.515+0000 7f1eb529c640 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1eb0136470 con 0x7f1eb015c5a0 2023-11-26T07:23:26.518 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:26.515+0000 7f1eb529c640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1eb0136750 con 0x7f1eb0071bb0 2023-11-26T07:23:26.518 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:26.515+0000 7f1eb529c640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1eb0130160 con 0x7f1eb0143780 2023-11-26T07:23:26.518 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:26.515+0000 7f1eaf7fe640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f1eb015c5a0 0x7f1eb0159a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:23:26.518 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:26.515+0000 7f1eaeffd640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f1eb0071bb0 0x7f1eb00af040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:23:26.519 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:26.515+0000 7f1eaf7fe640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f1eb015c5a0 0x7f1eb0159a00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:46250/0 (socket says 172.21.15.143:46250) 2023-11-26T07:23:26.519 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:26.515+0000 7f1eaf7fe640 1 -- 172.21.15.143:0/4017772214 learned_addr learned my addr 172.21.15.143:0/4017772214 (peer_addr_for_me v2:172.21.15.143:0/0) 2023-11-26T07:23:26.519 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:26.515+0000 7f1ea67fc640 1 --2- 172.21.15.143:0/4017772214 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f1eb0143780 0x7f1eb01594c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:23:26.519 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:26.515+0000 7f1eaeffd640 1 -- 172.21.15.143:0/4017772214 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f1eb0143780 msgr2=0x7f1eb01594c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:23:26.519 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:26.515+0000 7f1eaeffd640 1 --2- 172.21.15.143:0/4017772214 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f1eb0143780 0x7f1eb01594c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:23:26.519 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:26.515+0000 7f1eaeffd640 1 -- 172.21.15.143:0/4017772214 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f1eb015c5a0 msgr2=0x7f1eb0159a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:23:26.519 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:26.515+0000 7f1eaeffd640 1 --2- 172.21.15.143:0/4017772214 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f1eb015c5a0 0x7f1eb0159a00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:23:26.519 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:26.515+0000 7f1eaeffd640 1 -- 172.21.15.143:0/4017772214 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1e98009420 con 0x7f1eb0071bb0 2023-11-26T07:23:26.519 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:26.515+0000 7f1eaeffd640 1 --2- 172.21.15.143:0/4017772214 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f1eb0071bb0 0x7f1eb00af040 secure :-1 s=READY pgs=439 cs=0 l=1 rev1=1 crypto rx=0x7f1ea000cb10 tx=0x7f1ea000a060 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:23:26.520 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:26.515+0000 7f1ea7fff640 1 -- 172.21.15.143:0/4017772214 <== mon.1 v2:172.21.15.188:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1ea000ac70 con 0x7f1eb0071bb0 2023-11-26T07:23:26.520 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:26.515+0000 7f1ea7fff640 1 -- 172.21.15.143:0/4017772214 <== mon.1 v2:172.21.15.188:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1ea000add0 con 0x7f1eb0071bb0 2023-11-26T07:23:26.520 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:26.515+0000 7f1eb529c640 1 -- 172.21.15.143:0/4017772214 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1eb0130440 con 0x7f1eb0071bb0 2023-11-26T07:23:26.520 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:26.515+0000 7f1ea7fff640 1 -- 172.21.15.143:0/4017772214 <== mon.1 v2:172.21.15.188:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1ea0012620 con 0x7f1eb0071bb0 2023-11-26T07:23:26.520 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:26.515+0000 7f1eb529c640 1 -- 172.21.15.143:0/4017772214 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1eb00af580 con 0x7f1eb0071bb0 2023-11-26T07:23:26.521 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:26.515+0000 7f1eb529c640 1 -- 172.21.15.143:0/4017772214 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1eb0130680 con 0x7f1eb0071bb0 2023-11-26T07:23:26.521 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:26.519+0000 7f1ea7fff640 1 -- 172.21.15.143:0/4017772214 <== mon.1 v2:172.21.15.188:3300/0 4 ==== mgrmap(e 16) v1 ==== 85954+0+0 (secure 0 0 0) 0x7f1ea0007070 con 0x7f1eb0071bb0 2023-11-26T07:23:26.521 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:26.519+0000 7f1ea7fff640 1 --2- 172.21.15.143:0/4017772214 >> [v2:172.21.15.188:6836/3401128553,v1:172.21.15.188:6837/3401128553] conn(0x7f1e90066590 0x7f1e90068a50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:23:26.522 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:26.519+0000 7f1ea7fff640 1 -- 172.21.15.143:0/4017772214 <== mon.1 v2:172.21.15.188:3300/0 5 ==== osd_map(80..80 src has 1..80) v4 ==== 4857+0+0 (secure 0 0 0) 0x7f1ea0083a20 con 0x7f1eb0071bb0 2023-11-26T07:23:26.522 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:26.519+0000 7f1ea67fc640 1 --2- 172.21.15.143:0/4017772214 >> [v2:172.21.15.188:6836/3401128553,v1:172.21.15.188:6837/3401128553] conn(0x7f1e90066590 0x7f1e90068a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:23:26.522 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:26.519+0000 7f1ea67fc640 1 --2- 172.21.15.143:0/4017772214 >> [v2:172.21.15.188:6836/3401128553,v1:172.21.15.188:6837/3401128553] conn(0x7f1e90066590 0x7f1e90068a50 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7f1e980ae000 tx=0x7f1e980a7000 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:23:26.524 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:26.523+0000 7f1ea7fff640 1 -- 172.21.15.143:0/4017772214 <== mon.1 v2:172.21.15.188:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172649 (secure 0 0 0) 0x7f1ea00510f0 con 0x7f1eb0071bb0 2023-11-26T07:23:26.677 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:26.675+0000 7f1eb529c640 1 -- 172.21.15.143:0/4017772214 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64} v 0) v1 -- 0x7f1eb00ceb50 con 0x7f1eb0071bb0 2023-11-26T07:23:27.621 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:27.619+0000 7f1ea7fff640 1 -- 172.21.15.143:0/4017772214 <== mon.1 v2:172.21.15.188:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]=0 pool 'cephfs_metadata' created v81) v1 ==== 138+0+0 (secure 0 0 0) 0x7f1eb00ceb50 con 0x7f1eb0071bb0 2023-11-26T07:23:27.621 INFO:teuthology.orchestra.run.smithi143.stderr:pool 'cephfs_metadata' created 2023-11-26T07:23:27.626 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:27.623+0000 7f1eb529c640 1 -- 172.21.15.143:0/4017772214 >> [v2:172.21.15.188:6836/3401128553,v1:172.21.15.188:6837/3401128553] conn(0x7f1e90066590 msgr2=0x7f1e90068a50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:23:27.626 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:27.623+0000 7f1eb529c640 1 --2- 172.21.15.143:0/4017772214 >> [v2:172.21.15.188:6836/3401128553,v1:172.21.15.188:6837/3401128553] conn(0x7f1e90066590 0x7f1e90068a50 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7f1e980ae000 tx=0x7f1e980a7000 comp rx=0 tx=0).stop 2023-11-26T07:23:27.626 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:27.623+0000 7f1eb529c640 1 -- 172.21.15.143:0/4017772214 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f1eb0071bb0 msgr2=0x7f1eb00af040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:23:27.626 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:27.623+0000 7f1eb529c640 1 --2- 172.21.15.143:0/4017772214 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f1eb0071bb0 0x7f1eb00af040 secure :-1 s=READY pgs=439 cs=0 l=1 rev1=1 crypto rx=0x7f1ea000cb10 tx=0x7f1ea000a060 comp rx=0 tx=0).stop 2023-11-26T07:23:27.626 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:27.623+0000 7f1eb529c640 1 -- 172.21.15.143:0/4017772214 shutdown_connections 2023-11-26T07:23:27.627 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:27.623+0000 7f1eb529c640 1 --2- 172.21.15.143:0/4017772214 >> [v2:172.21.15.188:6836/3401128553,v1:172.21.15.188:6837/3401128553] conn(0x7f1e90066590 0x7f1e90068a50 unknown :-1 s=CLOSED pgs=54 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:23:27.627 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:27.623+0000 7f1eb529c640 1 --2- 172.21.15.143:0/4017772214 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f1eb015c5a0 0x7f1eb0159a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:23:27.627 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:27.623+0000 7f1eb529c640 1 --2- 172.21.15.143:0/4017772214 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f1eb0143780 0x7f1eb01594c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:23:27.627 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:27.623+0000 7f1eb529c640 1 --2- 172.21.15.143:0/4017772214 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f1eb0071bb0 0x7f1eb00af040 unknown :-1 s=CLOSED pgs=439 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:23:27.627 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:27.623+0000 7f1eb529c640 1 -- 172.21.15.143:0/4017772214 >> 172.21.15.143:0/4017772214 conn(0x7f1eb006c3f0 msgr2=0x7f1eb0138500 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:23:27.627 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:27.627+0000 7f1eb529c640 1 -- 172.21.15.143:0/4017772214 shutdown_connections 2023-11-26T07:23:27.628 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:27.627+0000 7f1eb529c640 1 -- 172.21.15.143:0/4017772214 wait complete. 2023-11-26T07:23:27.652 DEBUG:teuthology.orchestra.run.smithi143:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_data 64 --pg_num_min 64 --target_size_ratio 0.9 2023-11-26T07:23:27.803 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:27.803+0000 7f948acd6640 1 Processor -- start 2023-11-26T07:23:27.803 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:27.803+0000 7f948acd6640 1 -- start start 2023-11-26T07:23:27.805 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:27.803+0000 7f948acd6640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f948406dc70 0x7f948406c540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:23:27.805 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:27.803+0000 7f948acd6640 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7f94840cb8d0 0x7f94840cbd60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:23:27.805 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:27.803+0000 7f948acd6640 1 -- --> v1:172.21.15.143:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f948406d7e0 con 0x7f94841449f0 2023-11-26T07:23:27.805 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:27.803+0000 7f948acd6640 1 -- --> v2:172.21.15.143:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f94840cc2a0 con 0x7f94840cb8d0 2023-11-26T07:23:27.805 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:27.803+0000 7f948acd6640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f948406cbd0 con 0x7f948406dc70 2023-11-26T07:23:27.805 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:27.803+0000 7f9489273640 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7f94840cb8d0 0x7f94840cbd60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:23:27.806 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:27.803+0000 7f9488a72640 1 --1- >> v1:172.21.15.143:6789/0 conn(0x7f94841449f0 0x7f94840ce6b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.143:6789/0 says I am v1:172.21.15.143:43140/0 (socket says 172.21.15.143:43140) 2023-11-26T07:23:27.806 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:27.803+0000 7f9488a72640 1 -- 172.21.15.143:0/1259636632 learned_addr learned my addr 172.21.15.143:0/1259636632 (peer_addr_for_me v1:172.21.15.143:0/0) 2023-11-26T07:23:27.806 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:27.803+0000 7f947bfff640 1 --2- 172.21.15.143:0/1259636632 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f948406dc70 0x7f948406c540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:23:27.806 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:27.803+0000 7f947b7fe640 1 -- 172.21.15.143:0/1259636632 <== mon.0 v1:172.21.15.143:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2615238540 0 0) 0x7f948406d7e0 con 0x7f94841449f0 2023-11-26T07:23:27.806 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:27.803+0000 7f947b7fe640 1 -- 172.21.15.143:0/1259636632 --> v1:172.21.15.143:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f946c003530 con 0x7f94841449f0 2023-11-26T07:23:27.806 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:27.803+0000 7f9489273640 1 -- 172.21.15.143:0/1259636632 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f948406dc70 msgr2=0x7f948406c540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-26T07:23:27.806 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:27.803+0000 7f9489273640 1 --2- 172.21.15.143:0/1259636632 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f948406dc70 0x7f948406c540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:23:27.806 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:27.803+0000 7f9489273640 1 -- 172.21.15.143:0/1259636632 >> v1:172.21.15.143:6789/0 conn(0x7f94841449f0 legacy=0x7f94840ce6b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:23:27.806 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:27.803+0000 7f9489273640 1 -- 172.21.15.143:0/1259636632 --> v2:172.21.15.143:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f948406ceb0 con 0x7f94840cb8d0 2023-11-26T07:23:27.808 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:27.807+0000 7f9489273640 1 --2- 172.21.15.143:0/1259636632 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f94840cb8d0 0x7f94840cbd60 secure :-1 s=READY pgs=1132 cs=0 l=1 rev1=1 crypto rx=0x7f947c0097c0 tx=0x7f947c01c3c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=142acc483ae87ccc server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:23:27.808 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:27.807+0000 7f947b7fe640 1 -- 172.21.15.143:0/1259636632 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f947c0ac620 con 0x7f94840cb8d0 2023-11-26T07:23:27.808 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:27.807+0000 7f947b7fe640 1 -- 172.21.15.143:0/1259636632 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f947c0ac780 con 0x7f94840cb8d0 2023-11-26T07:23:27.808 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:27.807+0000 7f947b7fe640 1 -- 172.21.15.143:0/1259636632 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f947c0acb10 con 0x7f94840cb8d0 2023-11-26T07:23:27.809 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:27.807+0000 7f948acd6640 1 -- 172.21.15.143:0/1259636632 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f94840cb8d0 msgr2=0x7f94840cbd60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:23:27.809 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:27.807+0000 7f948acd6640 1 --2- 172.21.15.143:0/1259636632 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f94840cb8d0 0x7f94840cbd60 secure :-1 s=READY pgs=1132 cs=0 l=1 rev1=1 crypto rx=0x7f947c0097c0 tx=0x7f947c01c3c0 comp rx=0 tx=0).stop 2023-11-26T07:23:27.809 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:27.807+0000 7f948acd6640 1 -- 172.21.15.143:0/1259636632 shutdown_connections 2023-11-26T07:23:27.809 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:27.807+0000 7f948acd6640 1 --2- 172.21.15.143:0/1259636632 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f94840cb8d0 0x7f94840cbd60 unknown :-1 s=CLOSED pgs=1132 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:23:27.809 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:27.807+0000 7f948acd6640 1 --2- 172.21.15.143:0/1259636632 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f948406dc70 0x7f948406c540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:23:27.809 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:27.807+0000 7f948acd6640 1 -- 172.21.15.143:0/1259636632 >> 172.21.15.143:0/1259636632 conn(0x7f94840b0bf0 msgr2=0x7f94840b1000 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:23:27.810 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:27.807+0000 7f948acd6640 1 -- 172.21.15.143:0/1259636632 shutdown_connections 2023-11-26T07:23:27.810 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:27.807+0000 7f948acd6640 1 -- 172.21.15.143:0/1259636632 wait complete. 2023-11-26T07:23:27.810 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:27.811+0000 7f948acd6640 1 Processor -- start 2023-11-26T07:23:27.810 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:27.811+0000 7f948acd6640 1 -- start start 2023-11-26T07:23:27.812 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:27.811+0000 7f948acd6640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f948406dc70 0x7f948415b3a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:23:27.812 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:27.811+0000 7f948acd6640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f94840cb8d0 0x7f94840ad360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:23:27.813 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:27.811+0000 7f948acd6640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f94841449f0 0x7f94840aa850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:23:27.813 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:27.811+0000 7f948acd6640 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f94840adc90 con 0x7f94840cb8d0 2023-11-26T07:23:27.813 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:27.811+0000 7f948acd6640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f94840adf70 con 0x7f948406dc70 2023-11-26T07:23:27.813 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:27.811+0000 7f948acd6640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f94840ae250 con 0x7f94841449f0 2023-11-26T07:23:27.813 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:27.811+0000 7f947bfff640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f94840cb8d0 0x7f94840ad360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:23:27.813 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:27.811+0000 7f947bfff640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f94840cb8d0 0x7f94840ad360 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:46266/0 (socket says 172.21.15.143:46266) 2023-11-26T07:23:27.813 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:27.811+0000 7f947bfff640 1 -- 172.21.15.143:0/1320645075 learned_addr learned my addr 172.21.15.143:0/1320645075 (peer_addr_for_me v2:172.21.15.143:0/0) 2023-11-26T07:23:27.813 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:27.811+0000 7f9489273640 1 --2- 172.21.15.143:0/1320645075 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f94841449f0 0x7f94840aa850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:23:27.813 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:27.811+0000 7f9488a72640 1 --2- 172.21.15.143:0/1320645075 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f948406dc70 0x7f948415b3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:23:27.813 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:27.811+0000 7f947bfff640 1 -- 172.21.15.143:0/1320645075 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f94841449f0 msgr2=0x7f94840aa850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:23:27.813 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:27.811+0000 7f947bfff640 1 --2- 172.21.15.143:0/1320645075 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f94841449f0 0x7f94840aa850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:23:27.813 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:27.811+0000 7f947bfff640 1 -- 172.21.15.143:0/1320645075 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f948406dc70 msgr2=0x7f948415b3a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:23:27.814 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:27.811+0000 7f947bfff640 1 --2- 172.21.15.143:0/1320645075 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f948406dc70 0x7f948415b3a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:23:27.814 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:27.811+0000 7f947bfff640 1 -- 172.21.15.143:0/1320645075 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f947c009420 con 0x7f94840cb8d0 2023-11-26T07:23:27.814 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:27.811+0000 7f9489273640 1 --2- 172.21.15.143:0/1320645075 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f94841449f0 0x7f94840aa850 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-26T07:23:27.814 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:27.811+0000 7f947bfff640 1 --2- 172.21.15.143:0/1320645075 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f94840cb8d0 0x7f94840ad360 secure :-1 s=READY pgs=1133 cs=0 l=1 rev1=1 crypto rx=0x7f947400e930 tx=0x7f947400edf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:23:27.815 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:27.811+0000 7f94797fa640 1 -- 172.21.15.143:0/1320645075 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9474009690 con 0x7f94840cb8d0 2023-11-26T07:23:27.815 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:27.811+0000 7f94797fa640 1 -- 172.21.15.143:0/1320645075 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f947400ca00 con 0x7f94840cb8d0 2023-11-26T07:23:27.815 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:27.811+0000 7f94797fa640 1 -- 172.21.15.143:0/1320645075 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f947400cd70 con 0x7f94840cb8d0 2023-11-26T07:23:27.815 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:27.811+0000 7f948acd6640 1 -- 172.21.15.143:0/1320645075 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f94840a8950 con 0x7f94840cb8d0 2023-11-26T07:23:27.815 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:27.811+0000 7f948acd6640 1 -- 172.21.15.143:0/1320645075 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f94840ad8a0 con 0x7f94840cb8d0 2023-11-26T07:23:27.815 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:27.811+0000 7f948acd6640 1 -- 172.21.15.143:0/1320645075 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9484067720 con 0x7f94840cb8d0 2023-11-26T07:23:27.815 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:27.815+0000 7f94797fa640 1 -- 172.21.15.143:0/1320645075 <== mon.0 v2:172.21.15.143:3300/0 4 ==== mgrmap(e 16) v1 ==== 85954+0+0 (secure 0 0 0) 0x7f9474013470 con 0x7f94840cb8d0 2023-11-26T07:23:27.816 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:27.815+0000 7f94797fa640 1 --2- 172.21.15.143:0/1320645075 >> [v2:172.21.15.188:6836/3401128553,v1:172.21.15.188:6837/3401128553] conn(0x7f9470066590 0x7f9470068a50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:23:27.816 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:27.815+0000 7f94797fa640 1 -- 172.21.15.143:0/1320645075 <== mon.0 v2:172.21.15.143:3300/0 5 ==== osd_map(81..81 src has 1..81) v4 ==== 5227+0+0 (secure 0 0 0) 0x7f9474084190 con 0x7f94840cb8d0 2023-11-26T07:23:27.817 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:27.815+0000 7f9488a72640 1 --2- 172.21.15.143:0/1320645075 >> [v2:172.21.15.188:6836/3401128553,v1:172.21.15.188:6837/3401128553] conn(0x7f9470066590 0x7f9470068a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:23:27.817 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:27.815+0000 7f9488a72640 1 --2- 172.21.15.143:0/1320645075 >> [v2:172.21.15.188:6836/3401128553,v1:172.21.15.188:6837/3401128553] conn(0x7f9470066590 0x7f9470068a50 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7f9480001570 tx=0x7f9480009040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:23:27.821 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:27.819+0000 7f94797fa640 1 -- 172.21.15.143:0/1320645075 <== mon.0 v2:172.21.15.143:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172649 (secure 0 0 0) 0x7f9474051700 con 0x7f94840cb8d0 2023-11-26T07:23:27.973 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:27.971+0000 7f948acd6640 1 -- 172.21.15.143:0/1320645075 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9} v 0) v1 -- 0x7f9484070400 con 0x7f94840cb8d0 2023-11-26T07:23:28.621 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:28.619+0000 7f94797fa640 1 -- 172.21.15.143:0/1320645075 <== mon.0 v2:172.21.15.143:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]=0 pool 'cephfs_data' created v82) v1 ==== 170+0+0 (secure 0 0 0) 0x7f9484070400 con 0x7f94840cb8d0 2023-11-26T07:23:28.622 INFO:teuthology.orchestra.run.smithi143.stderr:pool 'cephfs_data' created 2023-11-26T07:23:28.623 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:28.623+0000 7f948acd6640 1 -- 172.21.15.143:0/1320645075 >> [v2:172.21.15.188:6836/3401128553,v1:172.21.15.188:6837/3401128553] conn(0x7f9470066590 msgr2=0x7f9470068a50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:23:28.623 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:28.623+0000 7f948acd6640 1 --2- 172.21.15.143:0/1320645075 >> [v2:172.21.15.188:6836/3401128553,v1:172.21.15.188:6837/3401128553] conn(0x7f9470066590 0x7f9470068a50 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7f9480001570 tx=0x7f9480009040 comp rx=0 tx=0).stop 2023-11-26T07:23:28.624 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:28.623+0000 7f948acd6640 1 -- 172.21.15.143:0/1320645075 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f94840cb8d0 msgr2=0x7f94840ad360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:23:28.624 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:28.623+0000 7f948acd6640 1 --2- 172.21.15.143:0/1320645075 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f94840cb8d0 0x7f94840ad360 secure :-1 s=READY pgs=1133 cs=0 l=1 rev1=1 crypto rx=0x7f947400e930 tx=0x7f947400edf0 comp rx=0 tx=0).stop 2023-11-26T07:23:28.624 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:28.623+0000 7f948acd6640 1 -- 172.21.15.143:0/1320645075 shutdown_connections 2023-11-26T07:23:28.624 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:28.623+0000 7f948acd6640 1 --2- 172.21.15.143:0/1320645075 >> [v2:172.21.15.188:6836/3401128553,v1:172.21.15.188:6837/3401128553] conn(0x7f9470066590 0x7f9470068a50 unknown :-1 s=CLOSED pgs=55 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:23:28.624 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:28.623+0000 7f948acd6640 1 --2- 172.21.15.143:0/1320645075 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f94841449f0 0x7f94840aa850 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:23:28.625 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:28.623+0000 7f948acd6640 1 --2- 172.21.15.143:0/1320645075 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f94840cb8d0 0x7f94840ad360 unknown :-1 s=CLOSED pgs=1133 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:23:28.625 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:28.623+0000 7f948acd6640 1 --2- 172.21.15.143:0/1320645075 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f948406dc70 0x7f948415b3a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:23:28.625 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:28.623+0000 7f948acd6640 1 -- 172.21.15.143:0/1320645075 >> 172.21.15.143:0/1320645075 conn(0x7f94840b0bf0 msgr2=0x7f94840b0fd0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:23:28.625 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:28.623+0000 7f948acd6640 1 -- 172.21.15.143:0/1320645075 shutdown_connections 2023-11-26T07:23:28.625 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:28.623+0000 7f948acd6640 1 -- 172.21.15.143:0/1320645075 wait complete. 2023-11-26T07:23:28.663 DEBUG:teuthology.orchestra.run.smithi143:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs new cephfs cephfs_metadata cephfs_data 2023-11-26T07:23:28.805 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:28.803+0000 7f3229c5a640 1 Processor -- start 2023-11-26T07:23:28.805 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:28.803+0000 7f3229c5a640 1 -- start start 2023-11-26T07:23:28.806 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:28.803+0000 7f3229c5a640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f32240cf990 0x7f322415efa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:23:28.806 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:28.803+0000 7f3229c5a640 1 -- --> v1:172.21.15.143:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f322406c520 con 0x7f322413e290 2023-11-26T07:23:28.807 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:28.803+0000 7f3229c5a640 1 -- --> v1:172.21.15.188:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f322406c800 con 0x7f322415f4e0 2023-11-26T07:23:28.807 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:28.803+0000 7f3229c5a640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f322406cae0 con 0x7f32240cf990 2023-11-26T07:23:28.809 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:28.807+0000 7f32237fe640 1 --1- >> v1:172.21.15.143:6789/0 conn(0x7f322413e290 0x7f32240cf280 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.143:6789/0 says I am v1:172.21.15.143:43156/0 (socket says 172.21.15.143:43156) 2023-11-26T07:23:28.809 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:28.807+0000 7f32237fe640 1 -- 172.21.15.143:0/1365526230 learned_addr learned my addr 172.21.15.143:0/1365526230 (peer_addr_for_me v1:172.21.15.143:0/0) 2023-11-26T07:23:28.809 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:28.807+0000 7f3222ffd640 1 --2- 172.21.15.143:0/1365526230 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f32240cf990 0x7f322415efa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:23:28.809 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:28.807+0000 7f32227fc640 1 -- 172.21.15.143:0/1365526230 <== mon.0 v1:172.21.15.143:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1402460222 0 0) 0x7f322406c520 con 0x7f322413e290 2023-11-26T07:23:28.809 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:28.807+0000 7f32227fc640 1 -- 172.21.15.143:0/1365526230 --> v1:172.21.15.143:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f320c003530 con 0x7f322413e290 2023-11-26T07:23:28.809 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:28.807+0000 7f32227fc640 1 -- 172.21.15.143:0/1365526230 <== mon.1 v1:172.21.15.188:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3801813026 0 0) 0x7f322406c800 con 0x7f322415f4e0 2023-11-26T07:23:28.809 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:28.807+0000 7f32227fc640 1 -- 172.21.15.143:0/1365526230 --> v1:172.21.15.188:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f322406c520 con 0x7f322415f4e0 2023-11-26T07:23:28.809 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:28.807+0000 7f32227fc640 1 -- 172.21.15.143:0/1365526230 <== mon.0 v1:172.21.15.143:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1114175727 0 0) 0x7f320c003530 con 0x7f322413e290 2023-11-26T07:23:28.809 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:28.807+0000 7f32227fc640 1 -- 172.21.15.143:0/1365526230 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f32240cf990 msgr2=0x7f322415efa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-26T07:23:28.809 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:28.807+0000 7f32227fc640 1 --2- 172.21.15.143:0/1365526230 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f32240cf990 0x7f322415efa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:23:28.809 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:28.807+0000 7f32227fc640 1 -- 172.21.15.143:0/1365526230 >> v1:172.21.15.188:6789/0 conn(0x7f322415f4e0 legacy=0x7f32240cd7b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:23:28.810 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:28.807+0000 7f32227fc640 1 -- 172.21.15.143:0/1365526230 --> v1:172.21.15.143:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f322406cdc0 con 0x7f322413e290 2023-11-26T07:23:28.810 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:28.807+0000 7f32227fc640 1 -- 172.21.15.143:0/1365526230 <== mon.0 v1:172.21.15.143:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2137882523 0 0) 0x7f3214001680 con 0x7f322413e290 2023-11-26T07:23:28.810 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:28.807+0000 7f32227fc640 1 -- 172.21.15.143:0/1365526230 >> v1:172.21.15.143:6789/0 conn(0x7f322413e290 legacy=0x7f32240cf280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:23:28.810 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:28.807+0000 7f32227fc640 1 --2- 172.21.15.143:0/1365526230 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f320c003b40 0x7f320c003f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:23:28.810 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:28.807+0000 7f32227fc640 1 --2- 172.21.15.143:0/1365526230 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f320c0047d0 0x7f320c004c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:23:28.810 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:28.807+0000 7f32227fc640 1 --2- 172.21.15.143:0/1365526230 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f32240cf990 0x7f320c005190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:23:28.810 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:28.807+0000 7f32227fc640 1 -- 172.21.15.143:0/1365526230 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f322406cae0 con 0x7f320c003b40 2023-11-26T07:23:28.810 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:28.807+0000 7f32227fc640 1 -- 172.21.15.143:0/1365526230 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f320c00ba40 con 0x7f320c0047d0 2023-11-26T07:23:28.810 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:28.807+0000 7f32227fc640 1 -- 172.21.15.143:0/1365526230 --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f320c00bbd0 con 0x7f32240cf990 2023-11-26T07:23:28.810 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:28.807+0000 7f32237fe640 1 --2- 172.21.15.143:0/1365526230 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f320c003b40 0x7f320c003f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:23:28.810 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:28.807+0000 7f3222ffd640 1 --2- 172.21.15.143:0/1365526230 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f320c0047d0 0x7f320c004c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:23:28.811 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:28.807+0000 7f3223fff640 1 --2- 172.21.15.143:0/1365526230 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f32240cf990 0x7f320c005190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:23:28.811 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:28.807+0000 7f32237fe640 1 -- 172.21.15.143:0/1365526230 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f32240cf990 msgr2=0x7f320c005190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-26T07:23:28.811 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:28.807+0000 7f32237fe640 1 --2- 172.21.15.143:0/1365526230 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f32240cf990 0x7f320c005190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:23:28.811 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:28.807+0000 7f32237fe640 1 -- 172.21.15.143:0/1365526230 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f320c0047d0 msgr2=0x7f320c004c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-26T07:23:28.811 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:28.807+0000 7f32237fe640 1 --2- 172.21.15.143:0/1365526230 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f320c0047d0 0x7f320c004c50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:23:28.811 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:28.807+0000 7f32237fe640 1 -- 172.21.15.143:0/1365526230 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f320c003530 con 0x7f320c003b40 2023-11-26T07:23:28.811 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:28.811+0000 7f32237fe640 1 --2- 172.21.15.143:0/1365526230 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f320c003b40 0x7f320c003f40 secure :-1 s=READY pgs=1135 cs=0 l=1 rev1=1 crypto rx=0x7f321400c160 tx=0x7f321401cb30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2352ea53a3e11e2b server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:23:28.812 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:28.811+0000 7f32227fc640 1 -- 172.21.15.143:0/1365526230 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f32140058e0 con 0x7f320c003b40 2023-11-26T07:23:28.812 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:28.811+0000 7f32227fc640 1 -- 172.21.15.143:0/1365526230 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3214005a60 con 0x7f320c003b40 2023-11-26T07:23:28.812 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:28.811+0000 7f32227fc640 1 -- 172.21.15.143:0/1365526230 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3214005e10 con 0x7f320c003b40 2023-11-26T07:23:28.812 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:28.811+0000 7f3229c5a640 1 -- 172.21.15.143:0/1365526230 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f320c003b40 msgr2=0x7f320c003f40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:23:28.813 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:28.811+0000 7f3229c5a640 1 --2- 172.21.15.143:0/1365526230 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f320c003b40 0x7f320c003f40 secure :-1 s=READY pgs=1135 cs=0 l=1 rev1=1 crypto rx=0x7f321400c160 tx=0x7f321401cb30 comp rx=0 tx=0).stop 2023-11-26T07:23:28.813 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:28.811+0000 7f3223fff640 1 -- 172.21.15.143:0/1365526230 reap_dead start 2023-11-26T07:23:28.813 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:28.811+0000 7f3229c5a640 1 -- 172.21.15.143:0/1365526230 shutdown_connections 2023-11-26T07:23:28.813 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:28.811+0000 7f3229c5a640 1 -- 172.21.15.143:0/1365526230 >> 172.21.15.143:0/1365526230 conn(0x7f322406a890 msgr2=0x7f322406aca0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:23:28.813 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:28.811+0000 7f3229c5a640 1 -- 172.21.15.143:0/1365526230 shutdown_connections 2023-11-26T07:23:28.813 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:28.811+0000 7f3229c5a640 1 -- 172.21.15.143:0/1365526230 wait complete. 2023-11-26T07:23:28.814 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:28.811+0000 7f3229c5a640 1 Processor -- start 2023-11-26T07:23:28.814 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:28.811+0000 7f3229c5a640 1 -- start start 2023-11-26T07:23:28.816 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:28.815+0000 7f3229c5a640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f320c003b40 0x7f3224154cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:23:28.816 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:28.815+0000 7f3229c5a640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f322413ac60 0x7f322413c710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:23:28.817 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:28.815+0000 7f3229c5a640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f322415d520 0x7f322413cc50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:23:28.817 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:28.815+0000 7f3229c5a640 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3224155200 con 0x7f322415d520 2023-11-26T07:23:28.817 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:28.815+0000 7f3229c5a640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f322413a890 con 0x7f320c003b40 2023-11-26T07:23:28.817 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:28.815+0000 7f3229c5a640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f32240b1770 con 0x7f322413ac60 2023-11-26T07:23:28.817 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:28.815+0000 7f3223fff640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f322415d520 0x7f322413cc50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:23:28.817 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:28.815+0000 7f3223fff640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f322415d520 0x7f322413cc50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:46288/0 (socket says 172.21.15.143:46288) 2023-11-26T07:23:28.817 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:28.815+0000 7f3223fff640 1 -- 172.21.15.143:0/347999423 learned_addr learned my addr 172.21.15.143:0/347999423 (peer_addr_for_me v2:172.21.15.143:0/0) 2023-11-26T07:23:28.817 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:28.815+0000 7f32237fe640 1 --2- 172.21.15.143:0/347999423 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f320c003b40 0x7f3224154cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:23:28.817 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:28.815+0000 7f3222ffd640 1 --2- 172.21.15.143:0/347999423 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f322413ac60 0x7f322413c710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:23:28.817 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:28.815+0000 7f3223fff640 1 -- 172.21.15.143:0/347999423 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f322413ac60 msgr2=0x7f322413c710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:23:28.817 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:28.815+0000 7f3223fff640 1 --2- 172.21.15.143:0/347999423 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f322413ac60 0x7f322413c710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:23:28.817 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:28.815+0000 7f3223fff640 1 -- 172.21.15.143:0/347999423 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f320c003b40 msgr2=0x7f3224154cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:23:28.818 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:28.815+0000 7f3223fff640 1 --2- 172.21.15.143:0/347999423 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f320c003b40 0x7f3224154cc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:23:28.818 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:28.815+0000 7f3223fff640 1 -- 172.21.15.143:0/347999423 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f322413a2d0 con 0x7f322415d520 2023-11-26T07:23:28.818 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:28.815+0000 7f3222ffd640 1 --2- 172.21.15.143:0/347999423 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f322413ac60 0x7f322413c710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-26T07:23:28.818 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:28.815+0000 7f3223fff640 1 --2- 172.21.15.143:0/347999423 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f322415d520 0x7f322413cc50 secure :-1 s=READY pgs=1136 cs=0 l=1 rev1=1 crypto rx=0x7f3224072660 tx=0x7f321c00eb20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:23:28.818 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:28.815+0000 7f320bfff640 1 -- 172.21.15.143:0/347999423 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f321c003980 con 0x7f322415d520 2023-11-26T07:23:28.818 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:28.815+0000 7f320bfff640 1 -- 172.21.15.143:0/347999423 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f321c003ae0 con 0x7f322415d520 2023-11-26T07:23:28.818 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:28.815+0000 7f320bfff640 1 -- 172.21.15.143:0/347999423 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f321c003e90 con 0x7f322415d520 2023-11-26T07:23:28.818 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:28.815+0000 7f3229c5a640 1 -- 172.21.15.143:0/347999423 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f322413a5b0 con 0x7f322415d520 2023-11-26T07:23:28.819 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:28.815+0000 7f3229c5a640 1 -- 172.21.15.143:0/347999423 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f322413d190 con 0x7f322415d520 2023-11-26T07:23:28.819 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:28.815+0000 7f3229c5a640 1 -- 172.21.15.143:0/347999423 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3224067680 con 0x7f322415d520 2023-11-26T07:23:28.819 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:28.819+0000 7f320bfff640 1 -- 172.21.15.143:0/347999423 <== mon.0 v2:172.21.15.143:3300/0 4 ==== mgrmap(e 16) v1 ==== 85954+0+0 (secure 0 0 0) 0x7f321c0050d0 con 0x7f322415d520 2023-11-26T07:23:28.820 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:28.819+0000 7f320bfff640 1 --2- 172.21.15.143:0/347999423 >> [v2:172.21.15.188:6836/3401128553,v1:172.21.15.188:6837/3401128553] conn(0x7f3210066590 0x7f3210068a50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:23:28.820 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:28.819+0000 7f320bfff640 1 -- 172.21.15.143:0/347999423 <== mon.0 v2:172.21.15.143:3300/0 5 ==== osd_map(82..82 src has 1..82) v4 ==== 5609+0+0 (secure 0 0 0) 0x7f321c083cb0 con 0x7f322415d520 2023-11-26T07:23:28.820 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:28.819+0000 7f32237fe640 1 --2- 172.21.15.143:0/347999423 >> [v2:172.21.15.188:6836/3401128553,v1:172.21.15.188:6837/3401128553] conn(0x7f3210066590 0x7f3210068a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:23:28.820 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:28.819+0000 7f32237fe640 1 --2- 172.21.15.143:0/347999423 >> [v2:172.21.15.188:6836/3401128553,v1:172.21.15.188:6837/3401128553] conn(0x7f3210066590 0x7f3210068a50 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7f32140aa000 tx=0x7f3214001a00 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:23:28.824 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:28.823+0000 7f320bfff640 1 -- 172.21.15.143:0/347999423 <== mon.0 v2:172.21.15.143:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172649 (secure 0 0 0) 0x7f321c051070 con 0x7f322415d520 2023-11-26T07:23:29.005 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:29.003+0000 7f3229c5a640 1 -- 172.21.15.143:0/347999423 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"} v 0) v1 -- 0x7f3224067680 con 0x7f322415d520 2023-11-26T07:23:29.651 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:29.647+0000 7f320bfff640 1 -- 172.21.15.143:0/347999423 <== mon.0 v2:172.21.15.143:3300/0 7 ==== mon_command_ack([{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]=0 new fs with metadata pool 12 and data pool 13 v47) v1 ==== 174+0+0 (secure 0 0 0) 0x7f321c054690 con 0x7f322415d520 2023-11-26T07:23:29.654 INFO:teuthology.orchestra.run.smithi143.stderr:new fs with metadata pool 12 and data pool 13 2023-11-26T07:23:29.658 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:29.655+0000 7f3229c5a640 1 -- 172.21.15.143:0/347999423 >> [v2:172.21.15.188:6836/3401128553,v1:172.21.15.188:6837/3401128553] conn(0x7f3210066590 msgr2=0x7f3210068a50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:23:29.658 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:29.655+0000 7f3229c5a640 1 --2- 172.21.15.143:0/347999423 >> [v2:172.21.15.188:6836/3401128553,v1:172.21.15.188:6837/3401128553] conn(0x7f3210066590 0x7f3210068a50 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7f32140aa000 tx=0x7f3214001a00 comp rx=0 tx=0).stop 2023-11-26T07:23:29.658 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:29.655+0000 7f3229c5a640 1 -- 172.21.15.143:0/347999423 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f322415d520 msgr2=0x7f322413cc50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:23:29.658 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:29.655+0000 7f3229c5a640 1 --2- 172.21.15.143:0/347999423 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f322415d520 0x7f322413cc50 secure :-1 s=READY pgs=1136 cs=0 l=1 rev1=1 crypto rx=0x7f3224072660 tx=0x7f321c00eb20 comp rx=0 tx=0).stop 2023-11-26T07:23:29.660 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:29.659+0000 7f3229c5a640 1 -- 172.21.15.143:0/347999423 shutdown_connections 2023-11-26T07:23:29.660 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:29.659+0000 7f3229c5a640 1 --2- 172.21.15.143:0/347999423 >> [v2:172.21.15.188:6836/3401128553,v1:172.21.15.188:6837/3401128553] conn(0x7f3210066590 0x7f3210068a50 unknown :-1 s=CLOSED pgs=56 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:23:29.660 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:29.659+0000 7f3229c5a640 1 --2- 172.21.15.143:0/347999423 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f322415d520 0x7f322413cc50 unknown :-1 s=CLOSED pgs=1136 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:23:29.660 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:29.659+0000 7f3229c5a640 1 --2- 172.21.15.143:0/347999423 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f322413ac60 0x7f322413c710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:23:29.660 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:29.659+0000 7f3229c5a640 1 --2- 172.21.15.143:0/347999423 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f320c003b40 0x7f3224154cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:23:29.660 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:29.659+0000 7f3229c5a640 1 -- 172.21.15.143:0/347999423 >> 172.21.15.143:0/347999423 conn(0x7f322406a890 msgr2=0x7f3224160c50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:23:29.660 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:29.659+0000 7f3229c5a640 1 -- 172.21.15.143:0/347999423 shutdown_connections 2023-11-26T07:23:29.660 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:29.659+0000 7f3229c5a640 1 -- 172.21.15.143:0/347999423 wait complete. 2023-11-26T07:23:29.693 DEBUG:teuthology.orchestra.run.smithi143:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-11-26T07:23:29.832 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:29.831+0000 7f48a6535640 1 Processor -- start 2023-11-26T07:23:29.832 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:29.831+0000 7f48a6535640 1 -- start start 2023-11-26T07:23:29.833 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:29.831+0000 7f48a6535640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f48a00d17f0 0x7f48a00d1c30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:23:29.833 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:29.831+0000 7f48a6535640 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7f48a00b0490 0x7f48a00b0920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:23:29.833 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:29.831+0000 7f48a6535640 1 -- --> v1:172.21.15.143:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f48a00d0bc0 con 0x7f48a015d580 2023-11-26T07:23:29.833 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:29.831+0000 7f48a6535640 1 -- --> v2:172.21.15.188:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f48a00d0320 con 0x7f48a00b0490 2023-11-26T07:23:29.834 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:29.831+0000 7f48a6535640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f48a00d0600 con 0x7f48a00d17f0 2023-11-26T07:23:29.834 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:29.831+0000 7f489ffff640 1 --1- >> v1:172.21.15.143:6789/0 conn(0x7f48a015d580 0x7f48a00d3920 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.143:6789/0 says I am v1:172.21.15.143:43172/0 (socket says 172.21.15.143:43172) 2023-11-26T07:23:29.834 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:29.831+0000 7f489ffff640 1 -- 172.21.15.143:0/1506494745 learned_addr learned my addr 172.21.15.143:0/1506494745 (peer_addr_for_me v1:172.21.15.143:0/0) 2023-11-26T07:23:29.836 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:29.831+0000 7f489effd640 1 -- 172.21.15.143:0/1506494745 <== mon.0 v1:172.21.15.143:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2995730624 0 0) 0x7f48a00d0bc0 con 0x7f48a015d580 2023-11-26T07:23:29.836 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:29.831+0000 7f48a4ad2640 1 --2- 172.21.15.143:0/1506494745 >> v2:172.21.15.188:3300/0 conn(0x7f48a00b0490 0x7f48a00b0920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:23:29.836 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:29.831+0000 7f489effd640 1 -- 172.21.15.143:0/1506494745 --> v1:172.21.15.143:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f4888003530 con 0x7f48a015d580 2023-11-26T07:23:29.836 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:29.831+0000 7f489f7fe640 1 --2- 172.21.15.143:0/1506494745 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f48a00d17f0 0x7f48a00d1c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:23:29.836 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:29.831+0000 7f489effd640 1 -- 172.21.15.143:0/1506494745 <== mon.0 v1:172.21.15.143:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3965889241 0 0) 0x7f4888003530 con 0x7f48a015d580 2023-11-26T07:23:29.836 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:29.831+0000 7f489effd640 1 -- 172.21.15.143:0/1506494745 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f48a00d17f0 msgr2=0x7f48a00d1c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-26T07:23:29.836 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:29.831+0000 7f489effd640 1 --2- 172.21.15.143:0/1506494745 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f48a00d17f0 0x7f48a00d1c30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:23:29.836 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:29.831+0000 7f489effd640 1 -- 172.21.15.143:0/1506494745 >> v2:172.21.15.188:3300/0 conn(0x7f48a00b0490 msgr2=0x7f48a00b0920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-26T07:23:29.836 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:29.831+0000 7f489effd640 1 --2- 172.21.15.143:0/1506494745 >> v2:172.21.15.188:3300/0 conn(0x7f48a00b0490 0x7f48a00b0920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:23:29.836 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:29.831+0000 7f489effd640 1 -- 172.21.15.143:0/1506494745 --> v1:172.21.15.143:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f48a00d08e0 con 0x7f48a015d580 2023-11-26T07:23:29.836 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:29.831+0000 7f489effd640 1 -- 172.21.15.143:0/1506494745 <== mon.0 v1:172.21.15.143:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2137882523 0 0) 0x7f4890001680 con 0x7f48a015d580 2023-11-26T07:23:29.837 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:29.831+0000 7f489effd640 1 -- 172.21.15.143:0/1506494745 >> v1:172.21.15.143:6789/0 conn(0x7f48a015d580 legacy=0x7f48a00d3920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:23:29.837 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:29.831+0000 7f489effd640 1 --2- 172.21.15.143:0/1506494745 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f4888003980 0x7f4888003da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:23:29.837 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:29.831+0000 7f489effd640 1 --2- 172.21.15.143:0/1506494745 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f48880047d0 0x7f4888004c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:23:29.837 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:29.831+0000 7f489effd640 1 --2- 172.21.15.143:0/1506494745 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f4888005190 0x7f488800b990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:23:29.837 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:29.831+0000 7f489effd640 1 -- 172.21.15.143:0/1506494745 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f48a00d0320 con 0x7f4888003980 2023-11-26T07:23:29.837 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:29.831+0000 7f489effd640 1 -- 172.21.15.143:0/1506494745 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f48a00d0600 con 0x7f48880047d0 2023-11-26T07:23:29.837 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:29.835+0000 7f489effd640 1 -- 172.21.15.143:0/1506494745 --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f488800bf00 con 0x7f4888005190 2023-11-26T07:23:29.837 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:29.835+0000 7f489f7fe640 1 --2- 172.21.15.143:0/1506494745 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f48a00d17f0 0x7f48a00d1c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-26T07:23:29.837 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:29.835+0000 7f489ffff640 1 --2- 172.21.15.143:0/1506494745 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f4888003980 0x7f4888003da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:23:29.837 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:29.835+0000 7f48a4ad2640 1 --2- 172.21.15.143:0/1506494745 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f4888005190 0x7f488800b990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:23:29.837 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:29.835+0000 7f489f7fe640 1 --2- 172.21.15.143:0/1506494745 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f48880047d0 0x7f4888004c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:23:29.837 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:29.835+0000 7f489ffff640 1 -- 172.21.15.143:0/1506494745 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f4888005190 msgr2=0x7f488800b990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-26T07:23:29.838 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:29.835+0000 7f489ffff640 1 --2- 172.21.15.143:0/1506494745 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f4888005190 0x7f488800b990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:23:29.838 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:29.835+0000 7f489ffff640 1 -- 172.21.15.143:0/1506494745 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f48880047d0 msgr2=0x7f4888004c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-26T07:23:29.838 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:29.835+0000 7f489ffff640 1 --2- 172.21.15.143:0/1506494745 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f48880047d0 0x7f4888004c50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:23:29.838 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:29.835+0000 7f489ffff640 1 -- 172.21.15.143:0/1506494745 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4888003530 con 0x7f4888003980 2023-11-26T07:23:29.839 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:29.835+0000 7f489ffff640 1 --2- 172.21.15.143:0/1506494745 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f4888003980 0x7f4888003da0 secure :-1 s=READY pgs=1138 cs=0 l=1 rev1=1 crypto rx=0x7f489000c240 tx=0x7f489001cbd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ffb13918997624a4 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:23:29.839 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:29.835+0000 7f489effd640 1 -- 172.21.15.143:0/1506494745 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f48900058e0 con 0x7f4888003980 2023-11-26T07:23:29.839 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:29.835+0000 7f489effd640 1 -- 172.21.15.143:0/1506494745 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4890005a40 con 0x7f4888003980 2023-11-26T07:23:29.839 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:29.835+0000 7f489effd640 1 -- 172.21.15.143:0/1506494745 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4890005e10 con 0x7f4888003980 2023-11-26T07:23:29.839 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:29.839+0000 7f48a6535640 1 -- 172.21.15.143:0/1506494745 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f4888003980 msgr2=0x7f4888003da0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:23:29.839 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:29.839+0000 7f48a6535640 1 --2- 172.21.15.143:0/1506494745 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f4888003980 0x7f4888003da0 secure :-1 s=READY pgs=1138 cs=0 l=1 rev1=1 crypto rx=0x7f489000c240 tx=0x7f489001cbd0 comp rx=0 tx=0).stop 2023-11-26T07:23:29.839 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:29.839+0000 7f48a4ad2640 1 -- 172.21.15.143:0/1506494745 reap_dead start 2023-11-26T07:23:29.839 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:29.839+0000 7f48a6535640 1 -- 172.21.15.143:0/1506494745 shutdown_connections 2023-11-26T07:23:29.839 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:29.839+0000 7f48a6535640 1 -- 172.21.15.143:0/1506494745 >> 172.21.15.143:0/1506494745 conn(0x7f48a009c550 msgr2=0x7f48a00d3210 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:23:29.840 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:29.839+0000 7f48a6535640 1 -- 172.21.15.143:0/1506494745 shutdown_connections 2023-11-26T07:23:29.840 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:29.839+0000 7f48a6535640 1 -- 172.21.15.143:0/1506494745 wait complete. 2023-11-26T07:23:29.840 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:29.839+0000 7f48a6535640 1 Processor -- start 2023-11-26T07:23:29.841 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:29.839+0000 7f48a6535640 1 -- start start 2023-11-26T07:23:29.843 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:29.839+0000 7f48a6535640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f4888003980 0x7f48a015a6d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:23:29.843 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:29.839+0000 7f48a6535640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f48a015b6a0 0x7f48a01412a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:23:29.843 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:29.839+0000 7f48a6535640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f48a015bb00 0x7f48a01417e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:23:29.843 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:29.839+0000 7f48a6535640 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f48a015b280 con 0x7f4888003980 2023-11-26T07:23:29.843 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:29.839+0000 7f48a6535640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f48a0152c00 con 0x7f48a015b6a0 2023-11-26T07:23:29.843 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:29.839+0000 7f48a6535640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f48a015ac10 con 0x7f48a015bb00 2023-11-26T07:23:29.843 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:29.839+0000 7f489ffff640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f4888003980 0x7f48a015a6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:23:29.843 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:29.839+0000 7f489ffff640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f4888003980 0x7f48a015a6d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:46312/0 (socket says 172.21.15.143:46312) 2023-11-26T07:23:29.843 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:29.839+0000 7f489ffff640 1 -- 172.21.15.143:0/1167428471 learned_addr learned my addr 172.21.15.143:0/1167428471 (peer_addr_for_me v2:172.21.15.143:0/0) 2023-11-26T07:23:29.843 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:29.839+0000 7f489f7fe640 1 --2- 172.21.15.143:0/1167428471 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f48a015b6a0 0x7f48a01412a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:23:29.843 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:29.839+0000 7f48a4ad2640 1 --2- 172.21.15.143:0/1167428471 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f48a015bb00 0x7f48a01417e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:23:29.844 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:29.843+0000 7f489ffff640 1 -- 172.21.15.143:0/1167428471 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f48a015bb00 msgr2=0x7f48a01417e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:23:29.844 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:29.843+0000 7f489ffff640 1 --2- 172.21.15.143:0/1167428471 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f48a015bb00 0x7f48a01417e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:23:29.844 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:29.843+0000 7f489ffff640 1 -- 172.21.15.143:0/1167428471 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f48a015b6a0 msgr2=0x7f48a01412a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:23:29.844 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:29.843+0000 7f489ffff640 1 --2- 172.21.15.143:0/1167428471 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f48a015b6a0 0x7f48a01412a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:23:29.844 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:29.843+0000 7f489ffff640 1 -- 172.21.15.143:0/1167428471 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f48a015dd10 con 0x7f4888003980 2023-11-26T07:23:29.844 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:29.843+0000 7f489ffff640 1 --2- 172.21.15.143:0/1167428471 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f4888003980 0x7f48a015a6d0 secure :-1 s=READY pgs=1139 cs=0 l=1 rev1=1 crypto rx=0x7f489000c240 tx=0x7f4888005800 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:23:29.845 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:29.843+0000 7f489cff9640 1 -- 172.21.15.143:0/1167428471 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f48900a58b0 con 0x7f4888003980 2023-11-26T07:23:29.845 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:29.843+0000 7f489cff9640 1 -- 172.21.15.143:0/1167428471 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f48900a5a10 con 0x7f4888003980 2023-11-26T07:23:29.845 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:29.843+0000 7f489cff9640 1 -- 172.21.15.143:0/1167428471 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f48900a5da0 con 0x7f4888003980 2023-11-26T07:23:29.845 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:29.843+0000 7f48a6535640 1 -- 172.21.15.143:0/1167428471 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f48a015dff0 con 0x7f4888003980 2023-11-26T07:23:29.845 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:29.843+0000 7f48a6535640 1 -- 172.21.15.143:0/1167428471 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f48a01432d0 con 0x7f4888003980 2023-11-26T07:23:29.845 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:29.843+0000 7f48a6535640 1 -- 172.21.15.143:0/1167428471 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f48a0067680 con 0x7f4888003980 2023-11-26T07:23:29.845 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:29.843+0000 7f489cff9640 1 -- 172.21.15.143:0/1167428471 <== mon.0 v2:172.21.15.143:3300/0 4 ==== mgrmap(e 16) v1 ==== 85954+0+0 (secure 0 0 0) 0x7f48900047f0 con 0x7f4888003980 2023-11-26T07:23:29.846 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:29.843+0000 7f489cff9640 1 --2- 172.21.15.143:0/1167428471 >> [v2:172.21.15.188:6836/3401128553,v1:172.21.15.188:6837/3401128553] conn(0x7f488c0665c0 0x7f488c068a80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:23:29.846 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:29.843+0000 7f489cff9640 1 -- 172.21.15.143:0/1167428471 <== mon.0 v2:172.21.15.143:3300/0 5 ==== osd_map(83..83 src has 1..83) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f489011db80 con 0x7f4888003980 2023-11-26T07:23:29.846 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:29.847+0000 7f489f7fe640 1 --2- 172.21.15.143:0/1167428471 >> [v2:172.21.15.188:6836/3401128553,v1:172.21.15.188:6837/3401128553] conn(0x7f488c0665c0 0x7f488c068a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:23:29.847 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:29.847+0000 7f489f7fe640 1 --2- 172.21.15.143:0/1167428471 >> [v2:172.21.15.188:6836/3401128553,v1:172.21.15.188:6837/3401128553] conn(0x7f488c0665c0 0x7f488c068a80 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7f48a015f460 tx=0x7f4894008040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:23:29.850 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:29.847+0000 7f489cff9640 1 -- 172.21.15.143:0/1167428471 <== mon.0 v2:172.21.15.143:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172649 (secure 0 0 0) 0x7f48900eaee0 con 0x7f4888003980 2023-11-26T07:23:30.003 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:30.003+0000 7f48a6535640 1 -- 172.21.15.143:0/1167428471 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f48a00ce6e0 con 0x7f4888003980 2023-11-26T07:23:30.004 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:30.003+0000 7f489cff9640 1 -- 172.21.15.143:0/1167428471 <== mon.0 v2:172.21.15.143:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v83) v1 ==== 74+0+16609 (secure 0 0 0) 0x7f48900ee500 con 0x7f4888003980 2023-11-26T07:23:30.005 INFO:teuthology.orchestra.run.smithi143.stdout: 2023-11-26T07:23:30.005 INFO:teuthology.orchestra.run.smithi143.stdout:{"epoch":83,"fsid":"f5033060-2469-4a07-95fe-2b76f5404bc1","created":"2023-11-26T07:17:28.613663+0000","modified":"2023-11-26T07:23:29.008137+0000","last_up_change":"2023-11-26T07:17:41.157813+0000","last_in_change":"2023-11-26T07:17:31.181658+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":13,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-11-26T07:17:41.758597+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":12,"pool_name":"cephfs_metadata","create_time":"2023-11-26T07:23:26.724016+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"83","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":13,"pool_name":"cephfs_data","create_time":"2023-11-26T07:23:28.019663+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"83","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"ebbee2c6-023e-44b9-9f1d-6f7bfd309730","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6817","nonce":639807369},{"type":"v1","addr":"172.21.15.143:6819","nonce":639807369}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6821","nonce":639807369},{"type":"v1","addr":"172.21.15.143:6823","nonce":639807369}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6830","nonce":639807369},{"type":"v1","addr":"172.21.15.143:6831","nonce":639807369}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6825","nonce":639807369},{"type":"v1","addr":"172.21.15.143:6828","nonce":639807369}]},"public_addr":"172.21.15.143:6819/639807369","cluster_addr":"172.21.15.143:6823/639807369","heartbeat_back_addr":"172.21.15.143:6831/639807369","heartbeat_front_addr":"172.21.15.143:6828/639807369","state":["exists","up"]},{"osd":1,"uuid":"fec4b8be-7122-49e9-9a19-5850a0700001","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6816","nonce":1144942530},{"type":"v1","addr":"172.21.15.143:6818","nonce":1144942530}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6820","nonce":1144942530},{"type":"v1","addr":"172.21.15.143:6822","nonce":1144942530}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6827","nonce":1144942530},{"type":"v1","addr":"172.21.15.143:6829","nonce":1144942530}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6824","nonce":1144942530},{"type":"v1","addr":"172.21.15.143:6826","nonce":1144942530}]},"public_addr":"172.21.15.143:6818/1144942530","cluster_addr":"172.21.15.143:6822/1144942530","heartbeat_back_addr":"172.21.15.143:6829/1144942530","heartbeat_front_addr":"172.21.15.143:6826/1144942530","state":["exists","up"]},{"osd":2,"uuid":"2888af5b-f933-46ca-8f95-6dd494ff43f6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6808","nonce":3909095186},{"type":"v1","addr":"172.21.15.143:6809","nonce":3909095186}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6810","nonce":3909095186},{"type":"v1","addr":"172.21.15.143:6811","nonce":3909095186}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6814","nonce":3909095186},{"type":"v1","addr":"172.21.15.143:6815","nonce":3909095186}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6812","nonce":3909095186},{"type":"v1","addr":"172.21.15.143:6813","nonce":3909095186}]},"public_addr":"172.21.15.143:6809/3909095186","cluster_addr":"172.21.15.143:6811/3909095186","heartbeat_back_addr":"172.21.15.143:6815/3909095186","heartbeat_front_addr":"172.21.15.143:6813/3909095186","state":["exists","up"]},{"osd":3,"uuid":"d1f52435-7bc1-4081-9550-9cd1ba55f549","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6800","nonce":3854210991},{"type":"v1","addr":"172.21.15.143:6801","nonce":3854210991}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6802","nonce":3854210991},{"type":"v1","addr":"172.21.15.143:6803","nonce":3854210991}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6806","nonce":3854210991},{"type":"v1","addr":"172.21.15.143:6807","nonce":3854210991}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6804","nonce":3854210991},{"type":"v1","addr":"172.21.15.143:6805","nonce":3854210991}]},"public_addr":"172.21.15.143:6801/3854210991","cluster_addr":"172.21.15.143:6803/3854210991","heartbeat_back_addr":"172.21.15.143:6807/3854210991","heartbeat_front_addr":"172.21.15.143:6805/3854210991","state":["exists","up"]},{"osd":4,"uuid":"e2ae8b6e-6e25-49b2-bcbf-aacce68f06a3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6800","nonce":3924606902},{"type":"v1","addr":"172.21.15.188:6801","nonce":3924606902}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6802","nonce":3924606902},{"type":"v1","addr":"172.21.15.188:6803","nonce":3924606902}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6806","nonce":3924606902},{"type":"v1","addr":"172.21.15.188:6807","nonce":3924606902}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6804","nonce":3924606902},{"type":"v1","addr":"172.21.15.188:6805","nonce":3924606902}]},"public_addr":"172.21.15.188:6801/3924606902","cluster_addr":"172.21.15.188:6803/3924606902","heartbeat_back_addr":"172.21.15.188:6807/3924606902","heartbeat_front_addr":"172.21.15.188:6805/3924606902","state":["exists","up"]},{"osd":5,"uuid":"3cec8a32-7698-4c36-8e89-e4ba11934f33","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6813","nonce":186841544},{"type":"v1","addr":"172.21.15.188:6815","nonce":186841544}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6817","nonce":186841544},{"type":"v1","addr":"172.21.15.188:6818","nonce":186841544}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6822","nonce":186841544},{"type":"v1","addr":"172.21.15.188:6823","nonce":186841544}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6820","nonce":186841544},{"type":"v1","addr":"172.21.15.188:6821","nonce":186841544}]},"public_addr":"172.21.15.188:6815/186841544","cluster_addr":"172.21.15.188:6818/186841544","heartbeat_back_addr":"172.21.15.188:6823/186841544","heartbeat_front_addr":"172.21.15.188:6821/186841544","state":["exists","up"]},{"osd":6,"uuid":"58a65d5d-e090-473a-81b4-ad38e85029f9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6808","nonce":2153499405},{"type":"v1","addr":"172.21.15.188:6809","nonce":2153499405}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6810","nonce":2153499405},{"type":"v1","addr":"172.21.15.188:6811","nonce":2153499405}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6816","nonce":2153499405},{"type":"v1","addr":"172.21.15.188:6819","nonce":2153499405}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6812","nonce":2153499405},{"type":"v1","addr":"172.21.15.188:6814","nonce":2153499405}]},"public_addr":"172.21.15.188:6809/2153499405","cluster_addr":"172.21.15.188:6811/2153499405","heartbeat_back_addr":"172.21.15.188:6819/2153499405","heartbeat_front_addr":"172.21.15.188:6814/2153499405","state":["exists","up"]},{"osd":7,"uuid":"ea5b7f70-3554-4cf8-ac30-f7c59b80678d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6824","nonce":2722096729},{"type":"v1","addr":"172.21.15.188:6825","nonce":2722096729}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6826","nonce":2722096729},{"type":"v1","addr":"172.21.15.188:6827","nonce":2722096729}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6830","nonce":2722096729},{"type":"v1","addr":"172.21.15.188:6831","nonce":2722096729}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6828","nonce":2722096729},{"type":"v1","addr":"172.21.15.188:6829","nonce":2722096729}]},"public_addr":"172.21.15.188:6825/2722096729","cluster_addr":"172.21.15.188:6827/2722096729","heartbeat_back_addr":"172.21.15.188:6831/2722096729","heartbeat_front_addr":"172.21.15.188:6829/2722096729","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-26T07:17:36.995032+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-26T07:17:36.932266+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-26T07:17:36.829304+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-26T07:17:36.804297+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-26T07:17:36.667892+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-26T07:17:36.964868+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-26T07:17:36.827332+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-26T07:17:36.893179+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-11-26T07:23:30.006 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:30.003+0000 7f48a6535640 1 -- 172.21.15.143:0/1167428471 >> [v2:172.21.15.188:6836/3401128553,v1:172.21.15.188:6837/3401128553] conn(0x7f488c0665c0 msgr2=0x7f488c068a80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:23:30.007 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:30.003+0000 7f48a6535640 1 --2- 172.21.15.143:0/1167428471 >> [v2:172.21.15.188:6836/3401128553,v1:172.21.15.188:6837/3401128553] conn(0x7f488c0665c0 0x7f488c068a80 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7f48a015f460 tx=0x7f4894008040 comp rx=0 tx=0).stop 2023-11-26T07:23:30.007 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:30.003+0000 7f48a6535640 1 -- 172.21.15.143:0/1167428471 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f4888003980 msgr2=0x7f48a015a6d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:23:30.007 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:30.003+0000 7f48a6535640 1 --2- 172.21.15.143:0/1167428471 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f4888003980 0x7f48a015a6d0 secure :-1 s=READY pgs=1139 cs=0 l=1 rev1=1 crypto rx=0x7f489000c240 tx=0x7f4888005800 comp rx=0 tx=0).stop 2023-11-26T07:23:30.007 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:30.007+0000 7f48a6535640 1 -- 172.21.15.143:0/1167428471 shutdown_connections 2023-11-26T07:23:30.007 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:30.007+0000 7f48a6535640 1 --2- 172.21.15.143:0/1167428471 >> [v2:172.21.15.188:6836/3401128553,v1:172.21.15.188:6837/3401128553] conn(0x7f488c0665c0 0x7f488c068a80 unknown :-1 s=CLOSED pgs=57 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:23:30.007 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:30.007+0000 7f48a6535640 1 --2- 172.21.15.143:0/1167428471 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f48a015bb00 0x7f48a01417e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:23:30.007 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:30.007+0000 7f48a6535640 1 --2- 172.21.15.143:0/1167428471 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f48a015b6a0 0x7f48a01412a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:23:30.007 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:30.007+0000 7f48a6535640 1 --2- 172.21.15.143:0/1167428471 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f4888003980 0x7f48a015a6d0 unknown :-1 s=CLOSED pgs=1139 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:23:30.007 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:30.007+0000 7f48a6535640 1 -- 172.21.15.143:0/1167428471 >> 172.21.15.143:0/1167428471 conn(0x7f48a009c550 msgr2=0x7f48a00d3180 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:23:30.007 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:30.007+0000 7f48a6535640 1 -- 172.21.15.143:0/1167428471 shutdown_connections 2023-11-26T07:23:30.007 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:30.007+0000 7f48a6535640 1 -- 172.21.15.143:0/1167428471 wait complete. 2023-11-26T07:23:30.027 DEBUG:teuthology.orchestra.run.smithi143:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-11-26T07:23:30.164 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:30.163+0000 7f6d4c11d640 1 Processor -- start 2023-11-26T07:23:30.165 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:30.163+0000 7f6d4c11d640 1 -- start start 2023-11-26T07:23:30.166 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:30.163+0000 7f6d4c11d640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f6d4414b1e0 0x7f6d440cf280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:23:30.166 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:30.163+0000 7f6d4c11d640 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7f6d44070fd0 0x7f6d44162510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:23:30.166 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:30.163+0000 7f6d4c11d640 1 -- --> v1:172.21.15.143:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6d440cfc20 con 0x7f6d440cf7c0 2023-11-26T07:23:30.166 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:30.163+0000 7f6d4c11d640 1 -- --> v2:172.21.15.143:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6d44162080 con 0x7f6d44070fd0 2023-11-26T07:23:30.166 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:30.163+0000 7f6d4c11d640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6d44161d70 con 0x7f6d4414b1e0 2023-11-26T07:23:30.167 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:30.163+0000 7f6d496b8640 1 --1- >> v1:172.21.15.143:6789/0 conn(0x7f6d440cf7c0 0x7f6d440708c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.143:6789/0 says I am v1:172.21.15.143:43182/0 (socket says 172.21.15.143:43182) 2023-11-26T07:23:30.167 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:30.163+0000 7f6d4a6ba640 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7f6d44070fd0 0x7f6d44162510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:23:30.167 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:30.167+0000 7f6d496b8640 1 -- 172.21.15.143:0/707736701 learned_addr learned my addr 172.21.15.143:0/707736701 (peer_addr_for_me v1:172.21.15.143:0/0) 2023-11-26T07:23:30.167 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:30.167+0000 7f6d49eb9640 1 --2- 172.21.15.143:0/707736701 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f6d4414b1e0 0x7f6d440cf280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:23:30.167 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:30.167+0000 7f6d4a6ba640 1 -- 172.21.15.143:0/707736701 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f6d4414b1e0 msgr2=0x7f6d440cf280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-26T07:23:30.168 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:30.167+0000 7f6d4a6ba640 1 --2- 172.21.15.143:0/707736701 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f6d4414b1e0 0x7f6d440cf280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:23:30.168 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:30.167+0000 7f6d4a6ba640 1 -- 172.21.15.143:0/707736701 >> v1:172.21.15.143:6789/0 conn(0x7f6d440cf7c0 legacy=0x7f6d440708c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:23:30.168 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:30.167+0000 7f6d4a6ba640 1 -- 172.21.15.143:0/707736701 --> v2:172.21.15.143:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6d4414b5c0 con 0x7f6d44070fd0 2023-11-26T07:23:30.168 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:30.167+0000 7f6d48eb7640 1 -- 172.21.15.143:0/707736701 <== mon.0 v1:172.21.15.143:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 478509739 0 0) 0x7f6d440cfc20 con 0x7f6d440cf7c0 2023-11-26T07:23:30.170 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:30.167+0000 7f6d4a6ba640 1 --2- 172.21.15.143:0/707736701 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f6d44070fd0 0x7f6d44162510 secure :-1 s=READY pgs=1141 cs=0 l=1 rev1=1 crypto rx=0x7f6d3c009d90 tx=0x7f6d3c01c920 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dc8633036ae8fc34 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:23:30.170 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:30.167+0000 7f6d48eb7640 1 -- 172.21.15.143:0/707736701 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6d3c0a59e0 con 0x7f6d44070fd0 2023-11-26T07:23:30.170 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:30.167+0000 7f6d48eb7640 1 -- 172.21.15.143:0/707736701 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6d3c0a5b40 con 0x7f6d44070fd0 2023-11-26T07:23:30.170 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:30.167+0000 7f6d48eb7640 1 -- 172.21.15.143:0/707736701 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6d3c0ad440 con 0x7f6d44070fd0 2023-11-26T07:23:30.171 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:30.171+0000 7f6d4c11d640 1 -- 172.21.15.143:0/707736701 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f6d44070fd0 msgr2=0x7f6d44162510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:23:30.171 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:30.171+0000 7f6d4c11d640 1 --2- 172.21.15.143:0/707736701 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f6d44070fd0 0x7f6d44162510 secure :-1 s=READY pgs=1141 cs=0 l=1 rev1=1 crypto rx=0x7f6d3c009d90 tx=0x7f6d3c01c920 comp rx=0 tx=0).stop 2023-11-26T07:23:30.171 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:30.171+0000 7f6d4c11d640 1 -- 172.21.15.143:0/707736701 shutdown_connections 2023-11-26T07:23:30.171 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:30.171+0000 7f6d4c11d640 1 --2- 172.21.15.143:0/707736701 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f6d44070fd0 0x7f6d44162510 unknown :-1 s=CLOSED pgs=1141 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:23:30.171 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:30.171+0000 7f6d4c11d640 1 --2- 172.21.15.143:0/707736701 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f6d4414b1e0 0x7f6d440cf280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:23:30.171 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:30.171+0000 7f6d4c11d640 1 -- 172.21.15.143:0/707736701 >> 172.21.15.143:0/707736701 conn(0x7f6d4406a890 msgr2=0x7f6d4406aca0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:23:30.171 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:30.171+0000 7f6d4c11d640 1 -- 172.21.15.143:0/707736701 shutdown_connections 2023-11-26T07:23:30.171 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:30.171+0000 7f6d4c11d640 1 -- 172.21.15.143:0/707736701 wait complete. 2023-11-26T07:23:30.172 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:30.171+0000 7f6d4c11d640 1 Processor -- start 2023-11-26T07:23:30.172 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:30.171+0000 7f6d4c11d640 1 -- start start 2023-11-26T07:23:30.175 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:30.171+0000 7f6d4c11d640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f6d44070fd0 0x7f6d4414ed40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:23:30.175 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:30.171+0000 7f6d4c11d640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f6d440cf7c0 0x7f6d440aa390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:23:30.175 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:30.171+0000 7f6d4c11d640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f6d4414b1e0 0x7f6d44146640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:23:30.175 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:30.171+0000 7f6d4c11d640 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6d440abba0 con 0x7f6d440cf7c0 2023-11-26T07:23:30.175 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:30.171+0000 7f6d4c11d640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6d440abe80 con 0x7f6d4414b1e0 2023-11-26T07:23:30.175 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:30.171+0000 7f6d4c11d640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6d440ac160 con 0x7f6d44070fd0 2023-11-26T07:23:30.175 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:30.171+0000 7f6d496b8640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f6d440cf7c0 0x7f6d440aa390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:23:30.175 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:30.171+0000 7f6d496b8640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f6d440cf7c0 0x7f6d440aa390 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:46324/0 (socket says 172.21.15.143:46324) 2023-11-26T07:23:30.175 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:30.171+0000 7f6d496b8640 1 -- 172.21.15.143:0/2599160200 learned_addr learned my addr 172.21.15.143:0/2599160200 (peer_addr_for_me v2:172.21.15.143:0/0) 2023-11-26T07:23:30.176 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:30.171+0000 7f6d496b8640 1 -- 172.21.15.143:0/2599160200 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f6d44070fd0 msgr2=0x7f6d4414ed40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:23:30.176 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:30.171+0000 7f6d49eb9640 1 --2- 172.21.15.143:0/2599160200 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f6d44070fd0 0x7f6d4414ed40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:23:30.176 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:30.171+0000 7f6d4a6ba640 1 --2- 172.21.15.143:0/2599160200 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f6d4414b1e0 0x7f6d44146640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:23:30.176 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:30.171+0000 7f6d496b8640 1 --2- 172.21.15.143:0/2599160200 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f6d44070fd0 0x7f6d4414ed40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:23:30.176 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:30.171+0000 7f6d496b8640 1 -- 172.21.15.143:0/2599160200 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f6d4414b1e0 msgr2=0x7f6d44146640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:23:30.176 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:30.171+0000 7f6d496b8640 1 --2- 172.21.15.143:0/2599160200 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f6d4414b1e0 0x7f6d44146640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:23:30.176 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:30.171+0000 7f6d496b8640 1 -- 172.21.15.143:0/2599160200 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6d3c009a70 con 0x7f6d440cf7c0 2023-11-26T07:23:30.176 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:30.171+0000 7f6d49eb9640 1 --2- 172.21.15.143:0/2599160200 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f6d44070fd0 0x7f6d4414ed40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-26T07:23:30.176 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:30.171+0000 7f6d4a6ba640 1 --2- 172.21.15.143:0/2599160200 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f6d4414b1e0 0x7f6d44146640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-26T07:23:30.176 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:30.175+0000 7f6d496b8640 1 --2- 172.21.15.143:0/2599160200 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f6d440cf7c0 0x7f6d440aa390 secure :-1 s=READY pgs=1142 cs=0 l=1 rev1=1 crypto rx=0x7f6d340011f0 tx=0x7f6d3400c9b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:23:30.176 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:30.175+0000 7f6d3a7fc640 1 -- 172.21.15.143:0/2599160200 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6d340074e0 con 0x7f6d440cf7c0 2023-11-26T07:23:30.176 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:30.175+0000 7f6d3a7fc640 1 -- 172.21.15.143:0/2599160200 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6d34007640 con 0x7f6d440cf7c0 2023-11-26T07:23:30.177 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:30.175+0000 7f6d4c11d640 1 -- 172.21.15.143:0/2599160200 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6d440ac440 con 0x7f6d440cf7c0 2023-11-26T07:23:30.177 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:30.175+0000 7f6d3a7fc640 1 -- 172.21.15.143:0/2599160200 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6d34004a10 con 0x7f6d440cf7c0 2023-11-26T07:23:30.177 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:30.175+0000 7f6d4c11d640 1 -- 172.21.15.143:0/2599160200 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6d440aa8d0 con 0x7f6d440cf7c0 2023-11-26T07:23:30.177 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:30.175+0000 7f6d4c11d640 1 -- 172.21.15.143:0/2599160200 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6d44067680 con 0x7f6d440cf7c0 2023-11-26T07:23:30.178 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:30.175+0000 7f6d3a7fc640 1 -- 172.21.15.143:0/2599160200 <== mon.0 v2:172.21.15.143:3300/0 4 ==== mgrmap(e 16) v1 ==== 85954+0+0 (secure 0 0 0) 0x7f6d34004b70 con 0x7f6d440cf7c0 2023-11-26T07:23:30.178 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:30.175+0000 7f6d3a7fc640 1 --2- 172.21.15.143:0/2599160200 >> [v2:172.21.15.188:6836/3401128553,v1:172.21.15.188:6837/3401128553] conn(0x7f6d28066590 0x7f6d28068a50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:23:30.178 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:30.175+0000 7f6d3a7fc640 1 -- 172.21.15.143:0/2599160200 <== mon.0 v2:172.21.15.143:3300/0 5 ==== osd_map(83..83 src has 1..83) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f6d340835e0 con 0x7f6d440cf7c0 2023-11-26T07:23:30.178 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:30.175+0000 7f6d49eb9640 1 --2- 172.21.15.143:0/2599160200 >> [v2:172.21.15.188:6836/3401128553,v1:172.21.15.188:6837/3401128553] conn(0x7f6d28066590 0x7f6d28068a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:23:30.178 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:30.179+0000 7f6d49eb9640 1 --2- 172.21.15.143:0/2599160200 >> [v2:172.21.15.188:6836/3401128553,v1:172.21.15.188:6837/3401128553] conn(0x7f6d28066590 0x7f6d28068a50 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7f6d40009840 tx=0x7f6d40008040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:23:30.181 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:30.179+0000 7f6d3a7fc640 1 -- 172.21.15.143:0/2599160200 <== mon.0 v2:172.21.15.143:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172649 (secure 0 0 0) 0x7f6d34050950 con 0x7f6d440cf7c0 2023-11-26T07:23:30.330 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:30.331+0000 7f6d4c11d640 1 -- 172.21.15.143:0/2599160200 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f6d440ac8f0 con 0x7f6d440cf7c0 2023-11-26T07:23:30.332 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:30.331+0000 7f6d3a7fc640 1 -- 172.21.15.143:0/2599160200 <== mon.0 v2:172.21.15.143:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v83) v1 ==== 74+0+16609 (secure 0 0 0) 0x7f6d34053f70 con 0x7f6d440cf7c0 2023-11-26T07:23:30.332 INFO:teuthology.orchestra.run.smithi143.stdout: 2023-11-26T07:23:30.332 INFO:teuthology.orchestra.run.smithi143.stdout:{"epoch":83,"fsid":"f5033060-2469-4a07-95fe-2b76f5404bc1","created":"2023-11-26T07:17:28.613663+0000","modified":"2023-11-26T07:23:29.008137+0000","last_up_change":"2023-11-26T07:17:41.157813+0000","last_in_change":"2023-11-26T07:17:31.181658+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":13,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-11-26T07:17:41.758597+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":12,"pool_name":"cephfs_metadata","create_time":"2023-11-26T07:23:26.724016+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"83","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":13,"pool_name":"cephfs_data","create_time":"2023-11-26T07:23:28.019663+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"83","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"ebbee2c6-023e-44b9-9f1d-6f7bfd309730","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6817","nonce":639807369},{"type":"v1","addr":"172.21.15.143:6819","nonce":639807369}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6821","nonce":639807369},{"type":"v1","addr":"172.21.15.143:6823","nonce":639807369}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6830","nonce":639807369},{"type":"v1","addr":"172.21.15.143:6831","nonce":639807369}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6825","nonce":639807369},{"type":"v1","addr":"172.21.15.143:6828","nonce":639807369}]},"public_addr":"172.21.15.143:6819/639807369","cluster_addr":"172.21.15.143:6823/639807369","heartbeat_back_addr":"172.21.15.143:6831/639807369","heartbeat_front_addr":"172.21.15.143:6828/639807369","state":["exists","up"]},{"osd":1,"uuid":"fec4b8be-7122-49e9-9a19-5850a0700001","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6816","nonce":1144942530},{"type":"v1","addr":"172.21.15.143:6818","nonce":1144942530}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6820","nonce":1144942530},{"type":"v1","addr":"172.21.15.143:6822","nonce":1144942530}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6827","nonce":1144942530},{"type":"v1","addr":"172.21.15.143:6829","nonce":1144942530}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6824","nonce":1144942530},{"type":"v1","addr":"172.21.15.143:6826","nonce":1144942530}]},"public_addr":"172.21.15.143:6818/1144942530","cluster_addr":"172.21.15.143:6822/1144942530","heartbeat_back_addr":"172.21.15.143:6829/1144942530","heartbeat_front_addr":"172.21.15.143:6826/1144942530","state":["exists","up"]},{"osd":2,"uuid":"2888af5b-f933-46ca-8f95-6dd494ff43f6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6808","nonce":3909095186},{"type":"v1","addr":"172.21.15.143:6809","nonce":3909095186}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6810","nonce":3909095186},{"type":"v1","addr":"172.21.15.143:6811","nonce":3909095186}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6814","nonce":3909095186},{"type":"v1","addr":"172.21.15.143:6815","nonce":3909095186}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6812","nonce":3909095186},{"type":"v1","addr":"172.21.15.143:6813","nonce":3909095186}]},"public_addr":"172.21.15.143:6809/3909095186","cluster_addr":"172.21.15.143:6811/3909095186","heartbeat_back_addr":"172.21.15.143:6815/3909095186","heartbeat_front_addr":"172.21.15.143:6813/3909095186","state":["exists","up"]},{"osd":3,"uuid":"d1f52435-7bc1-4081-9550-9cd1ba55f549","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6800","nonce":3854210991},{"type":"v1","addr":"172.21.15.143:6801","nonce":3854210991}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6802","nonce":3854210991},{"type":"v1","addr":"172.21.15.143:6803","nonce":3854210991}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6806","nonce":3854210991},{"type":"v1","addr":"172.21.15.143:6807","nonce":3854210991}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6804","nonce":3854210991},{"type":"v1","addr":"172.21.15.143:6805","nonce":3854210991}]},"public_addr":"172.21.15.143:6801/3854210991","cluster_addr":"172.21.15.143:6803/3854210991","heartbeat_back_addr":"172.21.15.143:6807/3854210991","heartbeat_front_addr":"172.21.15.143:6805/3854210991","state":["exists","up"]},{"osd":4,"uuid":"e2ae8b6e-6e25-49b2-bcbf-aacce68f06a3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6800","nonce":3924606902},{"type":"v1","addr":"172.21.15.188:6801","nonce":3924606902}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6802","nonce":3924606902},{"type":"v1","addr":"172.21.15.188:6803","nonce":3924606902}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6806","nonce":3924606902},{"type":"v1","addr":"172.21.15.188:6807","nonce":3924606902}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6804","nonce":3924606902},{"type":"v1","addr":"172.21.15.188:6805","nonce":3924606902}]},"public_addr":"172.21.15.188:6801/3924606902","cluster_addr":"172.21.15.188:6803/3924606902","heartbeat_back_addr":"172.21.15.188:6807/3924606902","heartbeat_front_addr":"172.21.15.188:6805/3924606902","state":["exists","up"]},{"osd":5,"uuid":"3cec8a32-7698-4c36-8e89-e4ba11934f33","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6813","nonce":186841544},{"type":"v1","addr":"172.21.15.188:6815","nonce":186841544}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6817","nonce":186841544},{"type":"v1","addr":"172.21.15.188:6818","nonce":186841544}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6822","nonce":186841544},{"type":"v1","addr":"172.21.15.188:6823","nonce":186841544}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6820","nonce":186841544},{"type":"v1","addr":"172.21.15.188:6821","nonce":186841544}]},"public_addr":"172.21.15.188:6815/186841544","cluster_addr":"172.21.15.188:6818/186841544","heartbeat_back_addr":"172.21.15.188:6823/186841544","heartbeat_front_addr":"172.21.15.188:6821/186841544","state":["exists","up"]},{"osd":6,"uuid":"58a65d5d-e090-473a-81b4-ad38e85029f9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6808","nonce":2153499405},{"type":"v1","addr":"172.21.15.188:6809","nonce":2153499405}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6810","nonce":2153499405},{"type":"v1","addr":"172.21.15.188:6811","nonce":2153499405}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6816","nonce":2153499405},{"type":"v1","addr":"172.21.15.188:6819","nonce":2153499405}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6812","nonce":2153499405},{"type":"v1","addr":"172.21.15.188:6814","nonce":2153499405}]},"public_addr":"172.21.15.188:6809/2153499405","cluster_addr":"172.21.15.188:6811/2153499405","heartbeat_back_addr":"172.21.15.188:6819/2153499405","heartbeat_front_addr":"172.21.15.188:6814/2153499405","state":["exists","up"]},{"osd":7,"uuid":"ea5b7f70-3554-4cf8-ac30-f7c59b80678d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6824","nonce":2722096729},{"type":"v1","addr":"172.21.15.188:6825","nonce":2722096729}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6826","nonce":2722096729},{"type":"v1","addr":"172.21.15.188:6827","nonce":2722096729}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6830","nonce":2722096729},{"type":"v1","addr":"172.21.15.188:6831","nonce":2722096729}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6828","nonce":2722096729},{"type":"v1","addr":"172.21.15.188:6829","nonce":2722096729}]},"public_addr":"172.21.15.188:6825/2722096729","cluster_addr":"172.21.15.188:6827/2722096729","heartbeat_back_addr":"172.21.15.188:6831/2722096729","heartbeat_front_addr":"172.21.15.188:6829/2722096729","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-26T07:17:36.995032+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-26T07:17:36.932266+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-26T07:17:36.829304+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-26T07:17:36.804297+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-26T07:17:36.667892+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-26T07:17:36.964868+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-26T07:17:36.827332+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-26T07:17:36.893179+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-11-26T07:23:30.334 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:30.331+0000 7f6d4c11d640 1 -- 172.21.15.143:0/2599160200 >> [v2:172.21.15.188:6836/3401128553,v1:172.21.15.188:6837/3401128553] conn(0x7f6d28066590 msgr2=0x7f6d28068a50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:23:30.334 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:30.331+0000 7f6d4c11d640 1 --2- 172.21.15.143:0/2599160200 >> [v2:172.21.15.188:6836/3401128553,v1:172.21.15.188:6837/3401128553] conn(0x7f6d28066590 0x7f6d28068a50 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7f6d40009840 tx=0x7f6d40008040 comp rx=0 tx=0).stop 2023-11-26T07:23:30.334 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:30.331+0000 7f6d4c11d640 1 -- 172.21.15.143:0/2599160200 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f6d440cf7c0 msgr2=0x7f6d440aa390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:23:30.334 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:30.331+0000 7f6d4c11d640 1 --2- 172.21.15.143:0/2599160200 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f6d440cf7c0 0x7f6d440aa390 secure :-1 s=READY pgs=1142 cs=0 l=1 rev1=1 crypto rx=0x7f6d340011f0 tx=0x7f6d3400c9b0 comp rx=0 tx=0).stop 2023-11-26T07:23:30.334 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:30.331+0000 7f6d4c11d640 1 -- 172.21.15.143:0/2599160200 shutdown_connections 2023-11-26T07:23:30.334 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:30.331+0000 7f6d4c11d640 1 --2- 172.21.15.143:0/2599160200 >> [v2:172.21.15.188:6836/3401128553,v1:172.21.15.188:6837/3401128553] conn(0x7f6d28066590 0x7f6d28068a50 unknown :-1 s=CLOSED pgs=58 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:23:30.334 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:30.331+0000 7f6d4c11d640 1 --2- 172.21.15.143:0/2599160200 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f6d4414b1e0 0x7f6d44146640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:23:30.335 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:30.331+0000 7f6d4c11d640 1 --2- 172.21.15.143:0/2599160200 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f6d440cf7c0 0x7f6d440aa390 unknown :-1 s=CLOSED pgs=1142 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:23:30.335 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:30.331+0000 7f6d4c11d640 1 --2- 172.21.15.143:0/2599160200 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f6d44070fd0 0x7f6d4414ed40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:23:30.335 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:30.331+0000 7f6d4c11d640 1 -- 172.21.15.143:0/2599160200 >> 172.21.15.143:0/2599160200 conn(0x7f6d4406a890 msgr2=0x7f6d44144aa0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:23:30.335 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:30.331+0000 7f6d4c11d640 1 -- 172.21.15.143:0/2599160200 shutdown_connections 2023-11-26T07:23:30.335 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:30.331+0000 7f6d4c11d640 1 -- 172.21.15.143:0/2599160200 wait complete. 2023-11-26T07:23:30.353 DEBUG:teuthology.orchestra.run.smithi143:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs standby_count_wanted 0 2023-11-26T07:23:30.494 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:30.491+0000 7fbcf493d640 1 Processor -- start 2023-11-26T07:23:30.494 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:30.491+0000 7fbcf493d640 1 -- start start 2023-11-26T07:23:30.495 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:30.495+0000 7fbcf493d640 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7fbcf00d3d30 0x7fbcf00cda90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:23:30.495 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:30.495+0000 7fbcf493d640 1 -- --> v1:172.21.15.143:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbcf00d4170 con 0x7fbcf0160850 2023-11-26T07:23:30.495 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:30.495+0000 7fbcf493d640 1 -- --> v1:172.21.15.188:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbcf00cdfd0 con 0x7fbcf0160f60 2023-11-26T07:23:30.495 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:30.495+0000 7fbcf493d640 1 -- --> v2:172.21.15.188:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fbcf00ce290 con 0x7fbcf00d3d30 2023-11-26T07:23:30.495 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:30.495+0000 7fbceed9d640 1 --1- >> v1:172.21.15.143:6789/0 conn(0x7fbcf0160850 0x7fbcf00cf330 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.143:6789/0 says I am v1:172.21.15.143:43194/0 (socket says 172.21.15.143:43194) 2023-11-26T07:23:30.496 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:30.495+0000 7fbceed9d640 1 -- 172.21.15.143:0/2764642011 learned_addr learned my addr 172.21.15.143:0/2764642011 (peer_addr_for_me v1:172.21.15.143:0/0) 2023-11-26T07:23:30.496 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:30.495+0000 7fbce7fff640 1 --2- 172.21.15.143:0/2764642011 >> v2:172.21.15.188:3300/0 conn(0x7fbcf00d3d30 0x7fbcf00cda90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:23:30.496 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:30.495+0000 7fbcee59c640 1 -- 172.21.15.143:0/2764642011 <== mon.0 v1:172.21.15.143:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 19703419 0 0) 0x7fbcf00d4170 con 0x7fbcf0160850 2023-11-26T07:23:30.496 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:30.495+0000 7fbcee59c640 1 -- 172.21.15.143:0/2764642011 --> v1:172.21.15.143:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fbcdc003530 con 0x7fbcf0160850 2023-11-26T07:23:30.497 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:30.495+0000 7fbcee59c640 1 -- 172.21.15.143:0/2764642011 <== mon.0 v1:172.21.15.143:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2641461191 0 0) 0x7fbcdc003530 con 0x7fbcf0160850 2023-11-26T07:23:30.497 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:30.495+0000 7fbce7fff640 1 -- 172.21.15.143:0/2764642011 >> v1:172.21.15.188:6789/0 conn(0x7fbcf0160f60 legacy=0x7fbcf00b27e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:23:30.497 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:30.495+0000 7fbce7fff640 1 -- 172.21.15.143:0/2764642011 >> v1:172.21.15.143:6789/0 conn(0x7fbcf0160850 legacy=0x7fbcf00cf330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:23:30.497 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:30.495+0000 7fbce7fff640 1 -- 172.21.15.143:0/2764642011 --> v2:172.21.15.188:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbcf00ce570 con 0x7fbcf00d3d30 2023-11-26T07:23:30.499 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:30.499+0000 7fbce7fff640 1 --2- 172.21.15.143:0/2764642011 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fbcf00d3d30 0x7fbcf00cda90 secure :-1 s=READY pgs=442 cs=0 l=1 rev1=1 crypto rx=0x7fbcd80097c0 tx=0x7fbcd801c370 comp rx=0 tx=0).ready entity=mon.1 client_cookie=8eef926b692fb3bc server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:23:30.499 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:30.499+0000 7fbcee59c640 1 -- 172.21.15.143:0/2764642011 <== mon.1 v2:172.21.15.188:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbce0001680 con 0x7fbcf00d3d30 2023-11-26T07:23:30.499 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:30.499+0000 7fbcee59c640 1 -- 172.21.15.143:0/2764642011 <== mon.1 v2:172.21.15.188:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbcd80ac440 con 0x7fbcf00d3d30 2023-11-26T07:23:30.499 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:30.499+0000 7fbcee59c640 1 -- 172.21.15.143:0/2764642011 <== mon.1 v2:172.21.15.188:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbcd80ac7d0 con 0x7fbcf00d3d30 2023-11-26T07:23:30.500 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:30.499+0000 7fbcf493d640 1 -- 172.21.15.143:0/2764642011 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fbcf00d3d30 msgr2=0x7fbcf00cda90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:23:30.500 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:30.499+0000 7fbcf493d640 1 --2- 172.21.15.143:0/2764642011 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fbcf00d3d30 0x7fbcf00cda90 secure :-1 s=READY pgs=442 cs=0 l=1 rev1=1 crypto rx=0x7fbcd80097c0 tx=0x7fbcd801c370 comp rx=0 tx=0).stop 2023-11-26T07:23:30.500 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:30.499+0000 7fbcf493d640 1 -- 172.21.15.143:0/2764642011 shutdown_connections 2023-11-26T07:23:30.500 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:30.499+0000 7fbcf493d640 1 --2- 172.21.15.143:0/2764642011 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fbcf00d3d30 0x7fbcf00cda90 unknown :-1 s=CLOSED pgs=442 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:23:30.500 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:30.499+0000 7fbcf493d640 1 -- 172.21.15.143:0/2764642011 >> 172.21.15.143:0/2764642011 conn(0x7fbcf009c5a0 msgr2=0x7fbcf00d3260 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:23:30.501 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:30.499+0000 7fbcf493d640 1 -- 172.21.15.143:0/2764642011 shutdown_connections 2023-11-26T07:23:30.501 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:30.499+0000 7fbcf493d640 1 -- 172.21.15.143:0/2764642011 wait complete. 2023-11-26T07:23:30.502 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:30.499+0000 7fbcf493d640 1 Processor -- start 2023-11-26T07:23:30.502 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:30.499+0000 7fbcf493d640 1 -- start start 2023-11-26T07:23:30.504 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:30.503+0000 7fbcf493d640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fbcf00d3d30 0x7fbcf00ccf80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:23:30.504 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:30.503+0000 7fbcf493d640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fbcf0160850 0x7fbcf00b4c20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:23:30.504 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:30.503+0000 7fbcf493d640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fbcf0160f60 0x7fbcf00b5160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:23:30.504 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:30.503+0000 7fbcf493d640 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbcf015c510 con 0x7fbcf0160f60 2023-11-26T07:23:30.504 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:30.503+0000 7fbcf493d640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbcf015c7f0 con 0x7fbcf00d3d30 2023-11-26T07:23:30.504 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:30.503+0000 7fbcf493d640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbcf00b7f40 con 0x7fbcf0160850 2023-11-26T07:23:30.504 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:30.503+0000 7fbceed9d640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fbcf00d3d30 0x7fbcf00ccf80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:23:30.504 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:30.503+0000 7fbcef59e640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fbcf0160f60 0x7fbcf00b5160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:23:30.504 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:30.503+0000 7fbceed9d640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fbcf00d3d30 0x7fbcf00ccf80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.188:3300/0 says I am v2:172.21.15.143:60614/0 (socket says 172.21.15.143:60614) 2023-11-26T07:23:30.504 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:30.503+0000 7fbceed9d640 1 -- 172.21.15.143:0/2547650777 learned_addr learned my addr 172.21.15.143:0/2547650777 (peer_addr_for_me v2:172.21.15.143:0/0) 2023-11-26T07:23:30.504 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:30.503+0000 7fbce7fff640 1 --2- 172.21.15.143:0/2547650777 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fbcf0160850 0x7fbcf00b4c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:23:30.505 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:30.503+0000 7fbcef59e640 1 -- 172.21.15.143:0/2547650777 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fbcf0160850 msgr2=0x7fbcf00b4c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:23:30.505 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:30.503+0000 7fbcef59e640 1 --2- 172.21.15.143:0/2547650777 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fbcf0160850 0x7fbcf00b4c20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:23:30.505 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:30.503+0000 7fbcef59e640 1 -- 172.21.15.143:0/2547650777 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fbcf00d3d30 msgr2=0x7fbcf00ccf80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:23:30.505 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:30.503+0000 7fbcef59e640 1 --2- 172.21.15.143:0/2547650777 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fbcf00d3d30 0x7fbcf00ccf80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:23:30.505 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:30.503+0000 7fbcef59e640 1 -- 172.21.15.143:0/2547650777 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbcd8009420 con 0x7fbcf0160f60 2023-11-26T07:23:30.505 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:30.503+0000 7fbcef59e640 1 --2- 172.21.15.143:0/2547650777 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fbcf0160f60 0x7fbcf00b5160 secure :-1 s=READY pgs=1144 cs=0 l=1 rev1=1 crypto rx=0x7fbce800c700 tx=0x7fbce800cbc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:23:30.506 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:30.503+0000 7fbce77fe640 1 -- 172.21.15.143:0/2547650777 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbce8003b80 con 0x7fbcf0160f60 2023-11-26T07:23:30.506 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:30.503+0000 7fbce77fe640 1 -- 172.21.15.143:0/2547650777 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbce8003ce0 con 0x7fbcf0160f60 2023-11-26T07:23:30.506 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:30.503+0000 7fbce77fe640 1 -- 172.21.15.143:0/2547650777 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbce8005310 con 0x7fbcf0160f60 2023-11-26T07:23:30.506 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:30.503+0000 7fbcf493d640 1 -- 172.21.15.143:0/2547650777 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbcf00b8220 con 0x7fbcf0160f60 2023-11-26T07:23:30.506 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:30.503+0000 7fbcf493d640 1 -- 172.21.15.143:0/2547650777 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbcf00cd4c0 con 0x7fbcf0160f60 2023-11-26T07:23:30.506 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:30.503+0000 7fbcf493d640 1 -- 172.21.15.143:0/2547650777 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbcf015c960 con 0x7fbcf0160f60 2023-11-26T07:23:30.506 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:30.503+0000 7fbce77fe640 1 -- 172.21.15.143:0/2547650777 <== mon.0 v2:172.21.15.143:3300/0 4 ==== mgrmap(e 16) v1 ==== 85954+0+0 (secure 0 0 0) 0x7fbce8005470 con 0x7fbcf0160f60 2023-11-26T07:23:30.507 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:30.507+0000 7fbce77fe640 1 --2- 172.21.15.143:0/2547650777 >> [v2:172.21.15.188:6836/3401128553,v1:172.21.15.188:6837/3401128553] conn(0x7fbcd0066590 0x7fbcd0068a50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:23:30.507 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:30.507+0000 7fbce77fe640 1 -- 172.21.15.143:0/2547650777 <== mon.0 v2:172.21.15.143:3300/0 5 ==== osd_map(83..83 src has 1..83) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fbce8083b20 con 0x7fbcf0160f60 2023-11-26T07:23:30.507 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:30.507+0000 7fbceed9d640 1 --2- 172.21.15.143:0/2547650777 >> [v2:172.21.15.188:6836/3401128553,v1:172.21.15.188:6837/3401128553] conn(0x7fbcd0066590 0x7fbcd0068a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:23:30.508 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:30.507+0000 7fbceed9d640 1 --2- 172.21.15.143:0/2547650777 >> [v2:172.21.15.188:6836/3401128553,v1:172.21.15.188:6837/3401128553] conn(0x7fbcd0066590 0x7fbcd0068a50 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7fbce0001510 tx=0x7fbce00014a0 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:23:30.511 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:30.511+0000 7fbce77fe640 1 -- 172.21.15.143:0/2547650777 <== mon.0 v2:172.21.15.143:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172649 (secure 0 0 0) 0x7fbce8050fd0 con 0x7fbcf0160f60 2023-11-26T07:23:30.694 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:30.691+0000 7fbcf493d640 1 -- 172.21.15.143:0/2547650777 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"} v 0) v1 -- 0x7fbcf0067500 con 0x7fbcf0160f60 2023-11-26T07:23:31.662 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:31.659+0000 7fbce77fe640 1 -- 172.21.15.143:0/2547650777 <== mon.0 v2:172.21.15.143:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]=0 v50) v1 ==== 118+0+0 (secure 0 0 0) 0x7fbce80545f0 con 0x7fbcf0160f60 2023-11-26T07:23:31.664 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:31.663+0000 7fbcf493d640 1 -- 172.21.15.143:0/2547650777 >> [v2:172.21.15.188:6836/3401128553,v1:172.21.15.188:6837/3401128553] conn(0x7fbcd0066590 msgr2=0x7fbcd0068a50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:23:31.664 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:31.663+0000 7fbcf493d640 1 --2- 172.21.15.143:0/2547650777 >> [v2:172.21.15.188:6836/3401128553,v1:172.21.15.188:6837/3401128553] conn(0x7fbcd0066590 0x7fbcd0068a50 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7fbce0001510 tx=0x7fbce00014a0 comp rx=0 tx=0).stop 2023-11-26T07:23:31.665 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:31.663+0000 7fbcf493d640 1 -- 172.21.15.143:0/2547650777 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fbcf0160f60 msgr2=0x7fbcf00b5160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:23:31.665 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:31.663+0000 7fbcf493d640 1 --2- 172.21.15.143:0/2547650777 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fbcf0160f60 0x7fbcf00b5160 secure :-1 s=READY pgs=1144 cs=0 l=1 rev1=1 crypto rx=0x7fbce800c700 tx=0x7fbce800cbc0 comp rx=0 tx=0).stop 2023-11-26T07:23:31.665 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:31.663+0000 7fbcf493d640 1 -- 172.21.15.143:0/2547650777 shutdown_connections 2023-11-26T07:23:31.665 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:31.663+0000 7fbcf493d640 1 --2- 172.21.15.143:0/2547650777 >> [v2:172.21.15.188:6836/3401128553,v1:172.21.15.188:6837/3401128553] conn(0x7fbcd0066590 0x7fbcd0068a50 unknown :-1 s=CLOSED pgs=59 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:23:31.665 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:31.663+0000 7fbcf493d640 1 --2- 172.21.15.143:0/2547650777 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fbcf0160f60 0x7fbcf00b5160 unknown :-1 s=CLOSED pgs=1144 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:23:31.665 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:31.663+0000 7fbcf493d640 1 --2- 172.21.15.143:0/2547650777 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fbcf0160850 0x7fbcf00b4c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:23:31.665 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:31.663+0000 7fbcf493d640 1 --2- 172.21.15.143:0/2547650777 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fbcf00d3d30 0x7fbcf00ccf80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:23:31.665 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:31.663+0000 7fbcf493d640 1 -- 172.21.15.143:0/2547650777 >> 172.21.15.143:0/2547650777 conn(0x7fbcf009c5a0 msgr2=0x7fbcf00b1a50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:23:31.665 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:31.663+0000 7fbcf493d640 1 -- 172.21.15.143:0/2547650777 shutdown_connections 2023-11-26T07:23:31.665 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:31.663+0000 7fbcf493d640 1 -- 172.21.15.143:0/2547650777 wait complete. 2023-11-26T07:23:31.686 DEBUG:tasks.cephfs.filesystem:fs_config: {} 2023-11-26T07:23:31.687 DEBUG:teuthology.orchestra.run.smithi143:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs allow_standby_replay false 2023-11-26T07:23:31.830 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:31.827+0000 7f513985a640 1 Processor -- start 2023-11-26T07:23:31.831 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:31.831+0000 7f513985a640 1 -- start start 2023-11-26T07:23:31.831 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:31.831+0000 7f513985a640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f513415d580 0x7f51340d3920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:23:31.832 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:31.831+0000 7f513985a640 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7f51340d1f00 0x7f51340b0490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:23:31.832 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:31.831+0000 7f513985a640 1 -- --> v1:172.21.15.143:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f51340d0bc0 con 0x7f51340d3e60 2023-11-26T07:23:31.832 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:31.831+0000 7f513985a640 1 -- --> v2:172.21.15.188:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f51340d0320 con 0x7f51340d1f00 2023-11-26T07:23:31.832 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:31.831+0000 7f513985a640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f51340d0600 con 0x7f513415d580 2023-11-26T07:23:31.833 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:31.831+0000 7f512a7fc640 1 --1- >> v1:172.21.15.143:6789/0 conn(0x7f51340d3e60 0x7f51340d17f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.143:6789/0 says I am v1:172.21.15.143:43204/0 (socket says 172.21.15.143:43204) 2023-11-26T07:23:31.833 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:31.831+0000 7f512a7fc640 1 -- 172.21.15.143:0/2030386204 learned_addr learned my addr 172.21.15.143:0/2030386204 (peer_addr_for_me v1:172.21.15.143:0/0) 2023-11-26T07:23:31.833 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:31.831+0000 7f5132ffd640 1 --2- 172.21.15.143:0/2030386204 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f513415d580 0x7f51340d3920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:23:31.833 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:31.831+0000 7f51337fe640 1 --2- 172.21.15.143:0/2030386204 >> v2:172.21.15.188:3300/0 conn(0x7f51340d1f00 0x7f51340b0490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:23:31.833 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:31.831+0000 7f51327fc640 1 -- 172.21.15.143:0/2030386204 <== mon.0 v1:172.21.15.143:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2696533786 0 0) 0x7f51340d0bc0 con 0x7f51340d3e60 2023-11-26T07:23:31.833 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:31.831+0000 7f51327fc640 1 -- 172.21.15.143:0/2030386204 --> v1:172.21.15.143:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f5120003530 con 0x7f51340d3e60 2023-11-26T07:23:31.833 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:31.831+0000 7f51337fe640 1 -- 172.21.15.143:0/2030386204 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f513415d580 msgr2=0x7f51340d3920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-26T07:23:31.833 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:31.831+0000 7f51337fe640 1 --2- 172.21.15.143:0/2030386204 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f513415d580 0x7f51340d3920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:23:31.833 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:31.831+0000 7f51337fe640 1 -- 172.21.15.143:0/2030386204 >> v1:172.21.15.143:6789/0 conn(0x7f51340d3e60 legacy=0x7f51340d17f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:23:31.833 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:31.831+0000 7f51337fe640 1 -- 172.21.15.143:0/2030386204 --> v2:172.21.15.188:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f51340d08e0 con 0x7f51340d1f00 2023-11-26T07:23:31.833 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:31.831+0000 7f5132ffd640 1 --2- 172.21.15.143:0/2030386204 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f513415d580 0x7f51340d3920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-26T07:23:31.835 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:31.835+0000 7f51337fe640 1 --2- 172.21.15.143:0/2030386204 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f51340d1f00 0x7f51340b0490 secure :-1 s=READY pgs=443 cs=0 l=1 rev1=1 crypto rx=0x7f512c009d90 tx=0x7f512c01c920 comp rx=0 tx=0).ready entity=mon.1 client_cookie=6269fe1dafae3a38 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:23:31.836 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:31.835+0000 7f51327fc640 1 -- 172.21.15.143:0/2030386204 <== mon.1 v2:172.21.15.188:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f512c0a5910 con 0x7f51340d1f00 2023-11-26T07:23:31.836 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:31.835+0000 7f51327fc640 1 -- 172.21.15.143:0/2030386204 <== mon.1 v2:172.21.15.188:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f512c0a5a70 con 0x7f51340d1f00 2023-11-26T07:23:31.836 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:31.835+0000 7f51327fc640 1 -- 172.21.15.143:0/2030386204 <== mon.1 v2:172.21.15.188:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f512c0a5e00 con 0x7f51340d1f00 2023-11-26T07:23:31.837 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:31.835+0000 7f513985a640 1 -- 172.21.15.143:0/2030386204 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f51340d1f00 msgr2=0x7f51340b0490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:23:31.837 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:31.835+0000 7f513985a640 1 --2- 172.21.15.143:0/2030386204 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f51340d1f00 0x7f51340b0490 secure :-1 s=READY pgs=443 cs=0 l=1 rev1=1 crypto rx=0x7f512c009d90 tx=0x7f512c01c920 comp rx=0 tx=0).stop 2023-11-26T07:23:31.837 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:31.835+0000 7f513985a640 1 -- 172.21.15.143:0/2030386204 shutdown_connections 2023-11-26T07:23:31.838 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:31.835+0000 7f513985a640 1 --2- 172.21.15.143:0/2030386204 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f51340d1f00 0x7f51340b0490 unknown :-1 s=CLOSED pgs=443 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:23:31.838 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:31.835+0000 7f513985a640 1 --2- 172.21.15.143:0/2030386204 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f513415d580 0x7f51340d3920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:23:31.838 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:31.835+0000 7f513985a640 1 -- 172.21.15.143:0/2030386204 >> 172.21.15.143:0/2030386204 conn(0x7f513409c550 msgr2=0x7f51340d3210 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:23:31.838 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:31.835+0000 7f513985a640 1 -- 172.21.15.143:0/2030386204 shutdown_connections 2023-11-26T07:23:31.838 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:31.835+0000 7f513985a640 1 -- 172.21.15.143:0/2030386204 wait complete. 2023-11-26T07:23:31.838 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:31.839+0000 7f513985a640 1 Processor -- start 2023-11-26T07:23:31.838 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:31.839+0000 7f513985a640 1 -- start start 2023-11-26T07:23:31.841 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:31.839+0000 7f513985a640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f51340d1f00 0x7f51341578e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:23:31.841 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:31.839+0000 7f513985a640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f51340d3e60 0x7f51340c8d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:23:31.841 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:31.839+0000 7f513985a640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f513415d580 0x7f51340c9280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:23:31.842 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:31.839+0000 7f513985a640 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f51340ca2e0 con 0x7f513415d580 2023-11-26T07:23:31.842 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:31.839+0000 7f513985a640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f51340b8870 con 0x7f51340d3e60 2023-11-26T07:23:31.842 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:31.839+0000 7f513985a640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f51340b8b50 con 0x7f51340d1f00 2023-11-26T07:23:31.842 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:31.839+0000 7f51337fe640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f513415d580 0x7f51340c9280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:23:31.842 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:31.839+0000 7f51337fe640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f513415d580 0x7f51340c9280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:46342/0 (socket says 172.21.15.143:46342) 2023-11-26T07:23:31.842 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:31.839+0000 7f51337fe640 1 -- 172.21.15.143:0/1116027134 learned_addr learned my addr 172.21.15.143:0/1116027134 (peer_addr_for_me v2:172.21.15.143:0/0) 2023-11-26T07:23:31.842 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:31.839+0000 7f512a7fc640 1 --2- 172.21.15.143:0/1116027134 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f51340d3e60 0x7f51340c8d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:23:31.842 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:31.839+0000 7f5132ffd640 1 --2- 172.21.15.143:0/1116027134 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f51340d1f00 0x7f51341578e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:23:31.842 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:31.839+0000 7f51337fe640 1 -- 172.21.15.143:0/1116027134 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f51340d1f00 msgr2=0x7f51341578e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:23:31.842 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:31.839+0000 7f51337fe640 1 --2- 172.21.15.143:0/1116027134 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f51340d1f00 0x7f51341578e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:23:31.842 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:31.839+0000 7f51337fe640 1 -- 172.21.15.143:0/1116027134 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f51340d3e60 msgr2=0x7f51340c8d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:23:31.842 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:31.839+0000 7f51337fe640 1 --2- 172.21.15.143:0/1116027134 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f51340d3e60 0x7f51340c8d40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:23:31.842 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:31.839+0000 7f51337fe640 1 -- 172.21.15.143:0/1116027134 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5124009420 con 0x7f513415d580 2023-11-26T07:23:31.843 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:31.839+0000 7f51337fe640 1 --2- 172.21.15.143:0/1116027134 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f513415d580 0x7f51340c9280 secure :-1 s=READY pgs=1146 cs=0 l=1 rev1=1 crypto rx=0x7f512c01a550 tx=0x7f512c0196c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:23:31.843 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:31.839+0000 7f5130ff9640 1 -- 172.21.15.143:0/1116027134 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f512c0ad640 con 0x7f513415d580 2023-11-26T07:23:31.843 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:31.839+0000 7f5130ff9640 1 -- 172.21.15.143:0/1116027134 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f512c0ad7a0 con 0x7f513415d580 2023-11-26T07:23:31.843 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:31.839+0000 7f5130ff9640 1 -- 172.21.15.143:0/1116027134 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f512c0adb50 con 0x7f513415d580 2023-11-26T07:23:31.843 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:31.839+0000 7f513985a640 1 -- 172.21.15.143:0/1116027134 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f512c009a70 con 0x7f513415d580 2023-11-26T07:23:31.843 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:31.839+0000 7f513985a640 1 -- 172.21.15.143:0/1116027134 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f51340ca530 con 0x7f513415d580 2023-11-26T07:23:31.843 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:31.839+0000 7f513985a640 1 -- 172.21.15.143:0/1116027134 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f51340d2660 con 0x7f513415d580 2023-11-26T07:23:31.843 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:31.843+0000 7f5130ff9640 1 -- 172.21.15.143:0/1116027134 <== mon.0 v2:172.21.15.143:3300/0 4 ==== mgrmap(e 16) v1 ==== 85954+0+0 (secure 0 0 0) 0x7f512c0b4070 con 0x7f513415d580 2023-11-26T07:23:31.844 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:31.843+0000 7f5130ff9640 1 --2- 172.21.15.143:0/1116027134 >> [v2:172.21.15.188:6836/3401128553,v1:172.21.15.188:6837/3401128553] conn(0x7f5114066590 0x7f5114068a50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:23:31.844 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:31.843+0000 7f5130ff9640 1 -- 172.21.15.143:0/1116027134 <== mon.0 v2:172.21.15.143:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f512c11ea10 con 0x7f513415d580 2023-11-26T07:23:31.845 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:31.843+0000 7f5132ffd640 1 --2- 172.21.15.143:0/1116027134 >> [v2:172.21.15.188:6836/3401128553,v1:172.21.15.188:6837/3401128553] conn(0x7f5114066590 0x7f5114068a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:23:31.845 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:31.843+0000 7f5132ffd640 1 --2- 172.21.15.143:0/1116027134 >> [v2:172.21.15.188:6836/3401128553,v1:172.21.15.188:6837/3401128553] conn(0x7f5114066590 0x7f5114068a50 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7f5124009c50 tx=0x7f5124009190 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:23:31.848 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:31.847+0000 7f5130ff9640 1 -- 172.21.15.143:0/1116027134 <== mon.0 v2:172.21.15.143:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172649 (secure 0 0 0) 0x7f512c0ebd90 con 0x7f513415d580 2023-11-26T07:23:32.030 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:32.027+0000 7f513985a640 1 -- 172.21.15.143:0/1116027134 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"} v 0) v1 -- 0x7f51340ca720 con 0x7f513415d580 2023-11-26T07:23:32.670 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:32.667+0000 7f5130ff9640 1 -- 172.21.15.143:0/1116027134 <== mon.0 v2:172.21.15.143:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"}]=0 v51) v1 ==== 122+0+0 (secure 0 0 0) 0x7f512c0b2070 con 0x7f513415d580 2023-11-26T07:23:32.672 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:32.671+0000 7f513985a640 1 -- 172.21.15.143:0/1116027134 >> [v2:172.21.15.188:6836/3401128553,v1:172.21.15.188:6837/3401128553] conn(0x7f5114066590 msgr2=0x7f5114068a50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:23:32.672 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:32.671+0000 7f513985a640 1 --2- 172.21.15.143:0/1116027134 >> [v2:172.21.15.188:6836/3401128553,v1:172.21.15.188:6837/3401128553] conn(0x7f5114066590 0x7f5114068a50 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7f5124009c50 tx=0x7f5124009190 comp rx=0 tx=0).stop 2023-11-26T07:23:32.672 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:32.671+0000 7f513985a640 1 -- 172.21.15.143:0/1116027134 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f513415d580 msgr2=0x7f51340c9280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:23:32.672 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:32.671+0000 7f513985a640 1 --2- 172.21.15.143:0/1116027134 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f513415d580 0x7f51340c9280 secure :-1 s=READY pgs=1146 cs=0 l=1 rev1=1 crypto rx=0x7f512c01a550 tx=0x7f512c0196c0 comp rx=0 tx=0).stop 2023-11-26T07:23:32.672 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:32.671+0000 7f513985a640 1 -- 172.21.15.143:0/1116027134 shutdown_connections 2023-11-26T07:23:32.672 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:32.671+0000 7f513985a640 1 --2- 172.21.15.143:0/1116027134 >> [v2:172.21.15.188:6836/3401128553,v1:172.21.15.188:6837/3401128553] conn(0x7f5114066590 0x7f5114068a50 unknown :-1 s=CLOSED pgs=60 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:23:32.672 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:32.671+0000 7f513985a640 1 --2- 172.21.15.143:0/1116027134 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f513415d580 0x7f51340c9280 unknown :-1 s=CLOSED pgs=1146 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:23:32.672 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:32.671+0000 7f513985a640 1 --2- 172.21.15.143:0/1116027134 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f51340d3e60 0x7f51340c8d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:23:32.673 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:32.671+0000 7f513985a640 1 --2- 172.21.15.143:0/1116027134 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f51340d1f00 0x7f51341578e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:23:32.673 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:32.671+0000 7f513985a640 1 -- 172.21.15.143:0/1116027134 >> 172.21.15.143:0/1116027134 conn(0x7f513409c550 msgr2=0x7f51340d0b20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:23:32.673 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:32.671+0000 7f513985a640 1 -- 172.21.15.143:0/1116027134 shutdown_connections 2023-11-26T07:23:32.673 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:32.671+0000 7f513985a640 1 -- 172.21.15.143:0/1116027134 wait complete. 2023-11-26T07:23:32.694 DEBUG:tasks.cephfs.filesystem:Not Creating any subvols for filesystem 'cephfs' 2023-11-26T07:23:32.694 DEBUG:teuthology.orchestra.run.smithi143:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-11-26T07:23:32.834 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:32.831+0000 7f6de8baa640 1 Processor -- start 2023-11-26T07:23:32.834 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:32.831+0000 7f6de8baa640 1 -- start start 2023-11-26T07:23:32.836 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:32.835+0000 7f6de8baa640 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7f6de40cdb60 0x7f6de40aead0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:23:32.836 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:32.835+0000 7f6de8baa640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f6de406a490 0x7f6de406a920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:23:32.836 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:32.835+0000 7f6de8baa640 1 -- --> v1:172.21.15.143:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6de40b3690 con 0x7f6de4070ba0 2023-11-26T07:23:32.836 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:32.835+0000 7f6de8baa640 1 -- --> v2:172.21.15.143:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6de40b3ca0 con 0x7f6de40cdb60 2023-11-26T07:23:32.836 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:32.835+0000 7f6de8baa640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6de40cf080 con 0x7f6de406a490 2023-11-26T07:23:32.836 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:32.835+0000 7f6de259c640 1 --1- >> v1:172.21.15.143:6789/0 conn(0x7f6de4070ba0 0x7f6de4070f80 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.143:6789/0 says I am v1:172.21.15.143:43212/0 (socket says 172.21.15.143:43212) 2023-11-26T07:23:32.836 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:32.835+0000 7f6de1d9b640 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7f6de40cdb60 0x7f6de40aead0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:23:32.836 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:32.835+0000 7f6de259c640 1 -- 172.21.15.143:0/889285885 learned_addr learned my addr 172.21.15.143:0/889285885 (peer_addr_for_me v1:172.21.15.143:0/0) 2023-11-26T07:23:32.836 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:32.835+0000 7f6de2d9d640 1 --2- 172.21.15.143:0/889285885 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f6de406a490 0x7f6de406a920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:23:32.837 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:32.835+0000 7f6de1d9b640 1 -- 172.21.15.143:0/889285885 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f6de406a490 msgr2=0x7f6de406a920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-26T07:23:32.837 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:32.835+0000 7f6de1d9b640 1 --2- 172.21.15.143:0/889285885 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f6de406a490 0x7f6de406a920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:23:32.837 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:32.835+0000 7f6de1d9b640 1 -- 172.21.15.143:0/889285885 >> v1:172.21.15.143:6789/0 conn(0x7f6de4070ba0 legacy=0x7f6de4070f80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:23:32.837 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:32.835+0000 7f6de1d9b640 1 -- 172.21.15.143:0/889285885 --> v2:172.21.15.143:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6de40cd8c0 con 0x7f6de40cdb60 2023-11-26T07:23:32.837 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:32.835+0000 7f6de159a640 1 -- 172.21.15.143:0/889285885 <== mon.0 v1:172.21.15.143:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1280807346 0 0) 0x7f6de40b3690 con 0x7f6de4070ba0 2023-11-26T07:23:32.837 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:32.835+0000 7f6de2d9d640 1 --2- 172.21.15.143:0/889285885 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f6de406a490 0x7f6de406a920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-26T07:23:32.839 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:32.839+0000 7f6de1d9b640 1 --2- 172.21.15.143:0/889285885 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f6de40cdb60 0x7f6de40aead0 secure :-1 s=READY pgs=1148 cs=0 l=1 rev1=1 crypto rx=0x7f6dd80097c0 tx=0x7f6dd801c3f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a4541a4589c3de57 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:23:32.839 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:32.839+0000 7f6de159a640 1 -- 172.21.15.143:0/889285885 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6dd80ab5f0 con 0x7f6de40cdb60 2023-11-26T07:23:32.839 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:32.839+0000 7f6de159a640 1 -- 172.21.15.143:0/889285885 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6dd80ab750 con 0x7f6de40cdb60 2023-11-26T07:23:32.840 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:32.839+0000 7f6de159a640 1 -- 172.21.15.143:0/889285885 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6dd80abae0 con 0x7f6de40cdb60 2023-11-26T07:23:32.840 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:32.839+0000 7f6de8baa640 1 -- 172.21.15.143:0/889285885 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f6de40cdb60 msgr2=0x7f6de40aead0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:23:32.840 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:32.839+0000 7f6de8baa640 1 --2- 172.21.15.143:0/889285885 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f6de40cdb60 0x7f6de40aead0 secure :-1 s=READY pgs=1148 cs=0 l=1 rev1=1 crypto rx=0x7f6dd80097c0 tx=0x7f6dd801c3f0 comp rx=0 tx=0).stop 2023-11-26T07:23:32.840 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:32.839+0000 7f6de8baa640 1 -- 172.21.15.143:0/889285885 shutdown_connections 2023-11-26T07:23:32.840 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:32.839+0000 7f6de8baa640 1 --2- 172.21.15.143:0/889285885 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f6de406a490 0x7f6de406a920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:23:32.841 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:32.839+0000 7f6de8baa640 1 --2- 172.21.15.143:0/889285885 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f6de40cdb60 0x7f6de40aead0 unknown :-1 s=CLOSED pgs=1148 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:23:32.841 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:32.839+0000 7f6de8baa640 1 -- 172.21.15.143:0/889285885 >> 172.21.15.143:0/889285885 conn(0x7f6de40cf710 msgr2=0x7f6de40cfb20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:23:32.841 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:32.839+0000 7f6de8baa640 1 -- 172.21.15.143:0/889285885 shutdown_connections 2023-11-26T07:23:32.841 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:32.839+0000 7f6de8baa640 1 -- 172.21.15.143:0/889285885 wait complete. 2023-11-26T07:23:32.842 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:32.839+0000 7f6de8baa640 1 Processor -- start 2023-11-26T07:23:32.842 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:32.839+0000 7f6de8baa640 1 -- start start 2023-11-26T07:23:32.844 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:32.843+0000 7f6de8baa640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f6de406a490 0x7f6de4140230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:23:32.844 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:32.843+0000 7f6de8baa640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f6de4070ba0 0x7f6de415c760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:23:32.844 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:32.843+0000 7f6de8baa640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f6de40cdb60 0x7f6de415d570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:23:32.844 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:32.843+0000 7f6de8baa640 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6de415c2d0 con 0x7f6de40cdb60 2023-11-26T07:23:32.844 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:32.843+0000 7f6de8baa640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6de4140770 con 0x7f6de406a490 2023-11-26T07:23:32.844 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:32.843+0000 7f6de8baa640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6de415b9a0 con 0x7f6de4070ba0 2023-11-26T07:23:32.844 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:32.843+0000 7f6de2d9d640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f6de40cdb60 0x7f6de415d570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:23:32.844 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:32.843+0000 7f6de2d9d640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f6de40cdb60 0x7f6de415d570 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:46362/0 (socket says 172.21.15.143:46362) 2023-11-26T07:23:32.844 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:32.843+0000 7f6de2d9d640 1 -- 172.21.15.143:0/2416538307 learned_addr learned my addr 172.21.15.143:0/2416538307 (peer_addr_for_me v2:172.21.15.143:0/0) 2023-11-26T07:23:32.844 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:32.843+0000 7f6de1d9b640 1 --2- 172.21.15.143:0/2416538307 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f6de4070ba0 0x7f6de415c760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:23:32.845 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:32.843+0000 7f6de259c640 1 --2- 172.21.15.143:0/2416538307 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f6de406a490 0x7f6de4140230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:23:32.845 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:32.843+0000 7f6de2d9d640 1 -- 172.21.15.143:0/2416538307 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f6de4070ba0 msgr2=0x7f6de415c760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:23:32.845 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:32.843+0000 7f6de2d9d640 1 --2- 172.21.15.143:0/2416538307 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f6de4070ba0 0x7f6de415c760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:23:32.845 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:32.843+0000 7f6de2d9d640 1 -- 172.21.15.143:0/2416538307 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f6de406a490 msgr2=0x7f6de4140230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:23:32.845 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:32.843+0000 7f6de2d9d640 1 --2- 172.21.15.143:0/2416538307 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f6de406a490 0x7f6de4140230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:23:32.845 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:32.843+0000 7f6de2d9d640 1 -- 172.21.15.143:0/2416538307 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6dd8009420 con 0x7f6de40cdb60 2023-11-26T07:23:32.845 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:32.843+0000 7f6de1d9b640 1 --2- 172.21.15.143:0/2416538307 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f6de4070ba0 0x7f6de415c760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-26T07:23:32.845 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:32.843+0000 7f6de2d9d640 1 --2- 172.21.15.143:0/2416538307 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f6de40cdb60 0x7f6de415d570 secure :-1 s=READY pgs=1149 cs=0 l=1 rev1=1 crypto rx=0x7f6ddc007ef0 tx=0x7f6ddc00d4a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:23:32.846 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:32.843+0000 7f6dd2ffd640 1 -- 172.21.15.143:0/2416538307 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6ddc003ba0 con 0x7f6de40cdb60 2023-11-26T07:23:32.846 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:32.843+0000 7f6dd2ffd640 1 -- 172.21.15.143:0/2416538307 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6ddc00f040 con 0x7f6de40cdb60 2023-11-26T07:23:32.846 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:32.843+0000 7f6dd2ffd640 1 -- 172.21.15.143:0/2416538307 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6ddc00de20 con 0x7f6de40cdb60 2023-11-26T07:23:32.846 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:32.843+0000 7f6de8baa640 1 -- 172.21.15.143:0/2416538307 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6de415bc80 con 0x7f6de40cdb60 2023-11-26T07:23:32.846 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:32.843+0000 7f6de8baa640 1 -- 172.21.15.143:0/2416538307 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6de414db50 con 0x7f6de40cdb60 2023-11-26T07:23:32.846 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:32.843+0000 7f6de8baa640 1 -- 172.21.15.143:0/2416538307 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6de4067680 con 0x7f6de40cdb60 2023-11-26T07:23:32.847 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:32.847+0000 7f6dd2ffd640 1 -- 172.21.15.143:0/2416538307 <== mon.0 v2:172.21.15.143:3300/0 4 ==== mgrmap(e 16) v1 ==== 85954+0+0 (secure 0 0 0) 0x7f6ddc004030 con 0x7f6de40cdb60 2023-11-26T07:23:32.847 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:32.847+0000 7f6dd2ffd640 1 --2- 172.21.15.143:0/2416538307 >> [v2:172.21.15.188:6836/3401128553,v1:172.21.15.188:6837/3401128553] conn(0x7f6dc4066590 0x7f6dc4068a50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:23:32.847 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:32.847+0000 7f6dd2ffd640 1 -- 172.21.15.143:0/2416538307 <== mon.0 v2:172.21.15.143:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f6ddc084380 con 0x7f6de40cdb60 2023-11-26T07:23:32.848 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:32.847+0000 7f6de259c640 1 --2- 172.21.15.143:0/2416538307 >> [v2:172.21.15.188:6836/3401128553,v1:172.21.15.188:6837/3401128553] conn(0x7f6dc4066590 0x7f6dc4068a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:23:32.848 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:32.847+0000 7f6de259c640 1 --2- 172.21.15.143:0/2416538307 >> [v2:172.21.15.188:6836/3401128553,v1:172.21.15.188:6837/3401128553] conn(0x7f6dc4066590 0x7f6dc4068a50 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7f6dd4000f70 tx=0x7f6dd4007b60 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:23:32.851 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:32.851+0000 7f6dd2ffd640 1 -- 172.21.15.143:0/2416538307 <== mon.0 v2:172.21.15.143:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172649 (secure 0 0 0) 0x7f6ddc0515f0 con 0x7f6de40cdb60 2023-11-26T07:23:33.032 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:33.031+0000 7f6de8baa640 1 -- 172.21.15.143:0/2416538307 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f6de415c410 con 0x7f6de40cdb60 2023-11-26T07:23:33.033 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:33.031+0000 7f6dd2ffd640 1 -- 172.21.15.143:0/2416538307 <== mon.0 v2:172.21.15.143:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 51 v51) v1 ==== 94+0+4924 (secure 0 0 0) 0x7f6ddc054c10 con 0x7f6de40cdb60 2023-11-26T07:23:33.033 INFO:teuthology.orchestra.run.smithi143.stdout: 2023-11-26T07:23:33.033 INFO:teuthology.orchestra.run.smithi143.stdout:{"epoch":51,"default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6305,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.188:6834/3670325785","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6832","nonce":3670325785},{"type":"v1","addr":"172.21.15.188:6834","nonce":3670325785}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":46},{"gid":7467,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.143:6834/4060405000","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6832","nonce":4060405000},{"type":"v1","addr":"172.21.15.143:6834","nonce":4060405000}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":46},{"gid":7470,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.143:6835/4031724325","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6833","nonce":4031724325},{"type":"v1","addr":"172.21.15.143:6835","nonce":4031724325}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":46}],"filesystems":[{"mdsmap":{"epoch":51,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-11-26T07:23:29.008560+0000","modified":"2023-11-26T07:23:32.033055+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":7476},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7476":{"gid":7476,"name":"b","rank":0,"incarnation":47,"state":"up:active","state_seq":3,"addr":"172.21.15.188:6835/278875865","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6833","nonce":278875865},{"type":"v1","addr":"172.21.15.188:6835","nonce":278875865}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[13],"metadata_pool":12,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":6}]} 2023-11-26T07:23:33.034 INFO:teuthology.orchestra.run.smithi143.stderr:dumped fsmap epoch 51 2023-11-26T07:23:33.035 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:33.035+0000 7f6de8baa640 1 -- 172.21.15.143:0/2416538307 >> [v2:172.21.15.188:6836/3401128553,v1:172.21.15.188:6837/3401128553] conn(0x7f6dc4066590 msgr2=0x7f6dc4068a50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:23:33.035 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:33.035+0000 7f6de8baa640 1 --2- 172.21.15.143:0/2416538307 >> [v2:172.21.15.188:6836/3401128553,v1:172.21.15.188:6837/3401128553] conn(0x7f6dc4066590 0x7f6dc4068a50 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7f6dd4000f70 tx=0x7f6dd4007b60 comp rx=0 tx=0).stop 2023-11-26T07:23:33.035 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:33.035+0000 7f6de8baa640 1 -- 172.21.15.143:0/2416538307 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f6de40cdb60 msgr2=0x7f6de415d570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:23:33.035 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:33.035+0000 7f6de8baa640 1 --2- 172.21.15.143:0/2416538307 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f6de40cdb60 0x7f6de415d570 secure :-1 s=READY pgs=1149 cs=0 l=1 rev1=1 crypto rx=0x7f6ddc007ef0 tx=0x7f6ddc00d4a0 comp rx=0 tx=0).stop 2023-11-26T07:23:33.036 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:33.035+0000 7f6de8baa640 1 -- 172.21.15.143:0/2416538307 shutdown_connections 2023-11-26T07:23:33.036 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:33.035+0000 7f6de8baa640 1 --2- 172.21.15.143:0/2416538307 >> [v2:172.21.15.188:6836/3401128553,v1:172.21.15.188:6837/3401128553] conn(0x7f6dc4066590 0x7f6dc4068a50 unknown :-1 s=CLOSED pgs=61 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:23:33.036 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:33.035+0000 7f6de8baa640 1 --2- 172.21.15.143:0/2416538307 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f6de40cdb60 0x7f6de415d570 unknown :-1 s=CLOSED pgs=1149 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:23:33.036 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:33.035+0000 7f6de8baa640 1 --2- 172.21.15.143:0/2416538307 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f6de4070ba0 0x7f6de415c760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:23:33.036 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:33.035+0000 7f6de8baa640 1 --2- 172.21.15.143:0/2416538307 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f6de406a490 0x7f6de4140230 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:23:33.036 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:33.035+0000 7f6de8baa640 1 -- 172.21.15.143:0/2416538307 >> 172.21.15.143:0/2416538307 conn(0x7f6de40cf710 msgr2=0x7f6de40cca10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:23:33.036 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:33.035+0000 7f6de8baa640 1 -- 172.21.15.143:0/2416538307 shutdown_connections 2023-11-26T07:23:33.036 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:33.035+0000 7f6de8baa640 1 -- 172.21.15.143:0/2416538307 wait complete. 2023-11-26T07:23:33.055 DEBUG:teuthology.orchestra.run.smithi143:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-11-26T07:23:33.192 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:33.191+0000 7f3604fc4640 1 Processor -- start 2023-11-26T07:23:33.192 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:33.191+0000 7f3604fc4640 1 -- start start 2023-11-26T07:23:33.194 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:33.191+0000 7f3604fc4640 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7f36000d1bc0 0x7f36000d0df0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:23:33.194 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:33.191+0000 7f3604fc4640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f36000b2160 0x7f36000b0750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:23:33.194 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:33.191+0000 7f3604fc4640 1 -- --> v1:172.21.15.143:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f36000cf0e0 con 0x7f360015d6e0 2023-11-26T07:23:33.194 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:33.191+0000 7f3604fc4640 1 -- --> v2:172.21.15.188:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f360015dac0 con 0x7f36000d1bc0 2023-11-26T07:23:33.194 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:33.191+0000 7f3604fc4640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f36000ceb20 con 0x7f36000b2160 2023-11-26T07:23:33.194 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:33.191+0000 7f35fe59c640 1 --1- >> v1:172.21.15.143:6789/0 conn(0x7f360015d6e0 0x7f36000d14b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.143:6789/0 says I am v1:172.21.15.143:43214/0 (socket says 172.21.15.143:43214) 2023-11-26T07:23:33.194 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:33.191+0000 7f35fe59c640 1 -- 172.21.15.143:0/1806004381 learned_addr learned my addr 172.21.15.143:0/1806004381 (peer_addr_for_me v1:172.21.15.143:0/0) 2023-11-26T07:23:33.194 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:33.191+0000 7f35fdd9b640 1 --2- 172.21.15.143:0/1806004381 >> v2:172.21.15.188:3300/0 conn(0x7f36000d1bc0 0x7f36000d0df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:23:33.194 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:33.191+0000 7f35fed9d640 1 --2- 172.21.15.143:0/1806004381 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f36000b2160 0x7f36000b0750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:23:33.194 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:33.191+0000 7f35fd59a640 1 -- 172.21.15.143:0/1806004381 <== mon.0 v1:172.21.15.143:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4163952859 0 0) 0x7f36000cf0e0 con 0x7f360015d6e0 2023-11-26T07:23:33.194 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:33.191+0000 7f35fd59a640 1 -- 172.21.15.143:0/1806004381 --> v1:172.21.15.143:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f35e8003530 con 0x7f360015d6e0 2023-11-26T07:23:33.196 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:33.191+0000 7f35fd59a640 1 -- 172.21.15.143:0/1806004381 <== mon.0 v1:172.21.15.143:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 946674401 0 0) 0x7f35e8003530 con 0x7f360015d6e0 2023-11-26T07:23:33.196 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:33.191+0000 7f35fd59a640 1 -- 172.21.15.143:0/1806004381 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f36000b2160 msgr2=0x7f36000b0750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-26T07:23:33.196 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:33.191+0000 7f35fd59a640 1 --2- 172.21.15.143:0/1806004381 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f36000b2160 0x7f36000b0750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:23:33.196 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:33.191+0000 7f35fd59a640 1 -- 172.21.15.143:0/1806004381 >> v2:172.21.15.188:3300/0 conn(0x7f36000d1bc0 msgr2=0x7f36000d0df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-26T07:23:33.196 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:33.191+0000 7f35fd59a640 1 --2- 172.21.15.143:0/1806004381 >> v2:172.21.15.188:3300/0 conn(0x7f36000d1bc0 0x7f36000d0df0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:23:33.196 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:33.191+0000 7f35fd59a640 1 -- 172.21.15.143:0/1806004381 --> v1:172.21.15.143:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f36000cee00 con 0x7f360015d6e0 2023-11-26T07:23:33.197 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:33.191+0000 7f35fd59a640 1 -- 172.21.15.143:0/1806004381 <== mon.0 v1:172.21.15.143:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2137882523 0 0) 0x7f35f0001680 con 0x7f360015d6e0 2023-11-26T07:23:33.197 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:33.195+0000 7f35fd59a640 1 -- 172.21.15.143:0/1806004381 >> v1:172.21.15.143:6789/0 conn(0x7f360015d6e0 legacy=0x7f36000d14b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:23:33.197 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:33.195+0000 7f35fd59a640 1 --2- 172.21.15.143:0/1806004381 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f35e8003980 0x7f35e8003da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:23:33.197 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:33.195+0000 7f35fd59a640 1 --2- 172.21.15.143:0/1806004381 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f36000b2160 0x7f35e80047d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:23:33.197 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:33.195+0000 7f35fd59a640 1 --2- 172.21.15.143:0/1806004381 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f35e8004d10 0x7f35e8005190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:23:33.197 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:33.195+0000 7f35fd59a640 1 -- 172.21.15.143:0/1806004381 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f360015dac0 con 0x7f35e8004d10 2023-11-26T07:23:33.197 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:33.195+0000 7f35fd59a640 1 -- 172.21.15.143:0/1806004381 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f36000ceb20 con 0x7f35e8003980 2023-11-26T07:23:33.197 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:33.195+0000 7f35fd59a640 1 -- 172.21.15.143:0/1806004381 --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f35e800b990 con 0x7f36000b2160 2023-11-26T07:23:33.197 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:33.195+0000 7f35fed9d640 1 --2- 172.21.15.143:0/1806004381 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f35e8004d10 0x7f35e8005190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:23:33.197 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:33.195+0000 7f35fe59c640 1 --2- 172.21.15.143:0/1806004381 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f35e8003980 0x7f35e8003da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:23:33.197 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:33.195+0000 7f35fdd9b640 1 --2- 172.21.15.143:0/1806004381 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f36000b2160 0x7f35e80047d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:23:33.197 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:33.195+0000 7f35fed9d640 1 -- 172.21.15.143:0/1806004381 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f36000b2160 msgr2=0x7f35e80047d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-26T07:23:33.198 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:33.195+0000 7f35fed9d640 1 --2- 172.21.15.143:0/1806004381 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f36000b2160 0x7f35e80047d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:23:33.198 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:33.195+0000 7f35fed9d640 1 -- 172.21.15.143:0/1806004381 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f35e8003980 msgr2=0x7f35e8003da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-26T07:23:33.198 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:33.195+0000 7f35fed9d640 1 --2- 172.21.15.143:0/1806004381 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f35e8003980 0x7f35e8003da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:23:33.198 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:33.195+0000 7f35fed9d640 1 -- 172.21.15.143:0/1806004381 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f35e8003530 con 0x7f35e8004d10 2023-11-26T07:23:33.198 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:33.195+0000 7f35fe59c640 1 --2- 172.21.15.143:0/1806004381 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f35e8003980 0x7f35e8003da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-26T07:23:33.199 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:33.195+0000 7f35fed9d640 1 --2- 172.21.15.143:0/1806004381 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f35e8004d10 0x7f35e8005190 secure :-1 s=READY pgs=1151 cs=0 l=1 rev1=1 crypto rx=0x7f35f8009f90 tx=0x7f35f801cc20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=816970c75d32becf server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:23:33.199 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:33.195+0000 7f35fd59a640 1 -- 172.21.15.143:0/1806004381 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f35f80a5c30 con 0x7f35e8004d10 2023-11-26T07:23:33.199 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:33.195+0000 7f35fd59a640 1 -- 172.21.15.143:0/1806004381 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f35f800f040 con 0x7f35e8004d10 2023-11-26T07:23:33.199 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:33.195+0000 7f35fd59a640 1 -- 172.21.15.143:0/1806004381 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f35f80ad440 con 0x7f35e8004d10 2023-11-26T07:23:33.199 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:33.199+0000 7f3604fc4640 1 -- 172.21.15.143:0/1806004381 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f35e8004d10 msgr2=0x7f35e8005190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:23:33.199 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:33.199+0000 7f3604fc4640 1 --2- 172.21.15.143:0/1806004381 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f35e8004d10 0x7f35e8005190 secure :-1 s=READY pgs=1151 cs=0 l=1 rev1=1 crypto rx=0x7f35f8009f90 tx=0x7f35f801cc20 comp rx=0 tx=0).stop 2023-11-26T07:23:33.200 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:33.199+0000 7f35fed9d640 1 -- 172.21.15.143:0/1806004381 reap_dead start 2023-11-26T07:23:33.200 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:33.199+0000 7f3604fc4640 1 -- 172.21.15.143:0/1806004381 shutdown_connections 2023-11-26T07:23:33.200 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:33.199+0000 7f3604fc4640 1 -- 172.21.15.143:0/1806004381 >> 172.21.15.143:0/1806004381 conn(0x7f360009c550 msgr2=0x7f36000d3220 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:23:33.200 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:33.199+0000 7f3604fc4640 1 -- 172.21.15.143:0/1806004381 shutdown_connections 2023-11-26T07:23:33.200 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:33.199+0000 7f3604fc4640 1 -- 172.21.15.143:0/1806004381 wait complete. 2023-11-26T07:23:33.201 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:33.199+0000 7f3604fc4640 1 Processor -- start 2023-11-26T07:23:33.201 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:33.199+0000 7f3604fc4640 1 -- start start 2023-11-26T07:23:33.203 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:33.199+0000 7f3604fc4640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f35e8004d10 0x7f36001603b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:23:33.203 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:33.199+0000 7f3604fc4640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f36000b3330 0x7f36000cffa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:23:33.203 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:33.199+0000 7f3604fc4640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f36000d04e0 0x7f36001f32c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:23:33.203 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:33.199+0000 7f3604fc4640 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f36000d0990 con 0x7f36000b3330 2023-11-26T07:23:33.203 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:33.199+0000 7f3604fc4640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f36000b2600 con 0x7f36000d04e0 2023-11-26T07:23:33.203 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:33.199+0000 7f3604fc4640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3600160990 con 0x7f35e8004d10 2023-11-26T07:23:33.203 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:33.199+0000 7f35fdd9b640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f36000b3330 0x7f36000cffa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:23:33.203 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:33.199+0000 7f35fdd9b640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f36000b3330 0x7f36000cffa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:46384/0 (socket says 172.21.15.143:46384) 2023-11-26T07:23:33.204 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:33.199+0000 7f35fdd9b640 1 -- 172.21.15.143:0/3607634881 learned_addr learned my addr 172.21.15.143:0/3607634881 (peer_addr_for_me v2:172.21.15.143:0/0) 2023-11-26T07:23:33.204 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:33.199+0000 7f35fed9d640 1 --2- 172.21.15.143:0/3607634881 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f36000d04e0 0x7f36001f32c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:23:33.204 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:33.199+0000 7f35fe59c640 1 --2- 172.21.15.143:0/3607634881 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f35e8004d10 0x7f36001603b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:23:33.204 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:33.203+0000 7f35fdd9b640 1 -- 172.21.15.143:0/3607634881 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f35e8004d10 msgr2=0x7f36001603b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:23:33.204 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:33.203+0000 7f35fdd9b640 1 --2- 172.21.15.143:0/3607634881 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f35e8004d10 0x7f36001603b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:23:33.204 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:33.203+0000 7f35fdd9b640 1 -- 172.21.15.143:0/3607634881 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f36000d04e0 msgr2=0x7f36001f32c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:23:33.204 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:33.203+0000 7f35fdd9b640 1 --2- 172.21.15.143:0/3607634881 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f36000d04e0 0x7f36001f32c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:23:33.204 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:33.203+0000 7f35fdd9b640 1 -- 172.21.15.143:0/3607634881 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f36000b2850 con 0x7f36000b3330 2023-11-26T07:23:33.204 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:33.203+0000 7f35fdd9b640 1 --2- 172.21.15.143:0/3607634881 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f36000b3330 0x7f36000cffa0 secure :-1 s=READY pgs=1152 cs=0 l=1 rev1=1 crypto rx=0x7f35f4011660 tx=0x7f35f4011b20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:23:33.204 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:33.203+0000 7f35eeffd640 1 -- 172.21.15.143:0/3607634881 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f35f400f9c0 con 0x7f36000b3330 2023-11-26T07:23:33.204 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:33.203+0000 7f35eeffd640 1 -- 172.21.15.143:0/3607634881 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f35f400fb20 con 0x7f36000b3330 2023-11-26T07:23:33.204 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:33.203+0000 7f3604fc4640 1 -- 172.21.15.143:0/3607634881 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f36000b2b30 con 0x7f36000b3330 2023-11-26T07:23:33.205 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:33.203+0000 7f35eeffd640 1 -- 172.21.15.143:0/3607634881 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f35f400fe90 con 0x7f36000b3330 2023-11-26T07:23:33.205 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:33.203+0000 7f3604fc4640 1 -- 172.21.15.143:0/3607634881 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f36001f3800 con 0x7f36000b3330 2023-11-26T07:23:33.205 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:33.203+0000 7f3604fc4640 1 -- 172.21.15.143:0/3607634881 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3600067680 con 0x7f36000b3330 2023-11-26T07:23:33.205 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:33.203+0000 7f35eeffd640 1 -- 172.21.15.143:0/3607634881 <== mon.0 v2:172.21.15.143:3300/0 4 ==== mgrmap(e 16) v1 ==== 85954+0+0 (secure 0 0 0) 0x7f35f4015460 con 0x7f36000b3330 2023-11-26T07:23:33.206 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:33.203+0000 7f35eeffd640 1 --2- 172.21.15.143:0/3607634881 >> [v2:172.21.15.188:6836/3401128553,v1:172.21.15.188:6837/3401128553] conn(0x7f35e0066590 0x7f35e0068a50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:23:33.206 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:33.203+0000 7f35eeffd640 1 -- 172.21.15.143:0/3607634881 <== mon.0 v2:172.21.15.143:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f35f40849a0 con 0x7f36000b3330 2023-11-26T07:23:33.206 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:33.203+0000 7f35fe59c640 1 --2- 172.21.15.143:0/3607634881 >> [v2:172.21.15.188:6836/3401128553,v1:172.21.15.188:6837/3401128553] conn(0x7f35e0066590 0x7f35e0068a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:23:33.207 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:33.207+0000 7f35fe59c640 1 --2- 172.21.15.143:0/3607634881 >> [v2:172.21.15.188:6836/3401128553,v1:172.21.15.188:6837/3401128553] conn(0x7f35e0066590 0x7f35e0068a50 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7f35f0000c10 tx=0x7f35f0001e40 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:23:33.209 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:33.207+0000 7f35eeffd640 1 -- 172.21.15.143:0/3607634881 <== mon.0 v2:172.21.15.143:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172649 (secure 0 0 0) 0x7f35f4051d20 con 0x7f36000b3330 2023-11-26T07:23:33.360 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:33.359+0000 7f3604fc4640 1 -- 172.21.15.143:0/3607634881 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f360006b660 con 0x7f36000b3330 2023-11-26T07:23:33.361 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:33.359+0000 7f35eeffd640 1 -- 172.21.15.143:0/3607634881 <== mon.0 v2:172.21.15.143:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v84) v1 ==== 74+0+16596 (secure 0 0 0) 0x7f35f4055340 con 0x7f36000b3330 2023-11-26T07:23:33.362 INFO:teuthology.orchestra.run.smithi143.stdout: 2023-11-26T07:23:33.362 INFO:teuthology.orchestra.run.smithi143.stdout:{"epoch":84,"fsid":"f5033060-2469-4a07-95fe-2b76f5404bc1","created":"2023-11-26T07:17:28.613663+0000","modified":"2023-11-26T07:23:30.637727+0000","last_up_change":"2023-11-26T07:17:41.157813+0000","last_in_change":"2023-11-26T07:17:31.181658+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":13,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-11-26T07:17:41.758597+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":12,"pool_name":"cephfs_metadata","create_time":"2023-11-26T07:23:26.724016+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"83","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":13,"pool_name":"cephfs_data","create_time":"2023-11-26T07:23:28.019663+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"84","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"ebbee2c6-023e-44b9-9f1d-6f7bfd309730","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6817","nonce":639807369},{"type":"v1","addr":"172.21.15.143:6819","nonce":639807369}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6821","nonce":639807369},{"type":"v1","addr":"172.21.15.143:6823","nonce":639807369}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6830","nonce":639807369},{"type":"v1","addr":"172.21.15.143:6831","nonce":639807369}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6825","nonce":639807369},{"type":"v1","addr":"172.21.15.143:6828","nonce":639807369}]},"public_addr":"172.21.15.143:6819/639807369","cluster_addr":"172.21.15.143:6823/639807369","heartbeat_back_addr":"172.21.15.143:6831/639807369","heartbeat_front_addr":"172.21.15.143:6828/639807369","state":["exists","up"]},{"osd":1,"uuid":"fec4b8be-7122-49e9-9a19-5850a0700001","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6816","nonce":1144942530},{"type":"v1","addr":"172.21.15.143:6818","nonce":1144942530}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6820","nonce":1144942530},{"type":"v1","addr":"172.21.15.143:6822","nonce":1144942530}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6827","nonce":1144942530},{"type":"v1","addr":"172.21.15.143:6829","nonce":1144942530}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6824","nonce":1144942530},{"type":"v1","addr":"172.21.15.143:6826","nonce":1144942530}]},"public_addr":"172.21.15.143:6818/1144942530","cluster_addr":"172.21.15.143:6822/1144942530","heartbeat_back_addr":"172.21.15.143:6829/1144942530","heartbeat_front_addr":"172.21.15.143:6826/1144942530","state":["exists","up"]},{"osd":2,"uuid":"2888af5b-f933-46ca-8f95-6dd494ff43f6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6808","nonce":3909095186},{"type":"v1","addr":"172.21.15.143:6809","nonce":3909095186}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6810","nonce":3909095186},{"type":"v1","addr":"172.21.15.143:6811","nonce":3909095186}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6814","nonce":3909095186},{"type":"v1","addr":"172.21.15.143:6815","nonce":3909095186}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6812","nonce":3909095186},{"type":"v1","addr":"172.21.15.143:6813","nonce":3909095186}]},"public_addr":"172.21.15.143:6809/3909095186","cluster_addr":"172.21.15.143:6811/3909095186","heartbeat_back_addr":"172.21.15.143:6815/3909095186","heartbeat_front_addr":"172.21.15.143:6813/3909095186","state":["exists","up"]},{"osd":3,"uuid":"d1f52435-7bc1-4081-9550-9cd1ba55f549","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6800","nonce":3854210991},{"type":"v1","addr":"172.21.15.143:6801","nonce":3854210991}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6802","nonce":3854210991},{"type":"v1","addr":"172.21.15.143:6803","nonce":3854210991}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6806","nonce":3854210991},{"type":"v1","addr":"172.21.15.143:6807","nonce":3854210991}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6804","nonce":3854210991},{"type":"v1","addr":"172.21.15.143:6805","nonce":3854210991}]},"public_addr":"172.21.15.143:6801/3854210991","cluster_addr":"172.21.15.143:6803/3854210991","heartbeat_back_addr":"172.21.15.143:6807/3854210991","heartbeat_front_addr":"172.21.15.143:6805/3854210991","state":["exists","up"]},{"osd":4,"uuid":"e2ae8b6e-6e25-49b2-bcbf-aacce68f06a3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6800","nonce":3924606902},{"type":"v1","addr":"172.21.15.188:6801","nonce":3924606902}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6802","nonce":3924606902},{"type":"v1","addr":"172.21.15.188:6803","nonce":3924606902}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6806","nonce":3924606902},{"type":"v1","addr":"172.21.15.188:6807","nonce":3924606902}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6804","nonce":3924606902},{"type":"v1","addr":"172.21.15.188:6805","nonce":3924606902}]},"public_addr":"172.21.15.188:6801/3924606902","cluster_addr":"172.21.15.188:6803/3924606902","heartbeat_back_addr":"172.21.15.188:6807/3924606902","heartbeat_front_addr":"172.21.15.188:6805/3924606902","state":["exists","up"]},{"osd":5,"uuid":"3cec8a32-7698-4c36-8e89-e4ba11934f33","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6813","nonce":186841544},{"type":"v1","addr":"172.21.15.188:6815","nonce":186841544}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6817","nonce":186841544},{"type":"v1","addr":"172.21.15.188:6818","nonce":186841544}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6822","nonce":186841544},{"type":"v1","addr":"172.21.15.188:6823","nonce":186841544}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6820","nonce":186841544},{"type":"v1","addr":"172.21.15.188:6821","nonce":186841544}]},"public_addr":"172.21.15.188:6815/186841544","cluster_addr":"172.21.15.188:6818/186841544","heartbeat_back_addr":"172.21.15.188:6823/186841544","heartbeat_front_addr":"172.21.15.188:6821/186841544","state":["exists","up"]},{"osd":6,"uuid":"58a65d5d-e090-473a-81b4-ad38e85029f9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6808","nonce":2153499405},{"type":"v1","addr":"172.21.15.188:6809","nonce":2153499405}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6810","nonce":2153499405},{"type":"v1","addr":"172.21.15.188:6811","nonce":2153499405}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6816","nonce":2153499405},{"type":"v1","addr":"172.21.15.188:6819","nonce":2153499405}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6812","nonce":2153499405},{"type":"v1","addr":"172.21.15.188:6814","nonce":2153499405}]},"public_addr":"172.21.15.188:6809/2153499405","cluster_addr":"172.21.15.188:6811/2153499405","heartbeat_back_addr":"172.21.15.188:6819/2153499405","heartbeat_front_addr":"172.21.15.188:6814/2153499405","state":["exists","up"]},{"osd":7,"uuid":"ea5b7f70-3554-4cf8-ac30-f7c59b80678d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6824","nonce":2722096729},{"type":"v1","addr":"172.21.15.188:6825","nonce":2722096729}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6826","nonce":2722096729},{"type":"v1","addr":"172.21.15.188:6827","nonce":2722096729}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6830","nonce":2722096729},{"type":"v1","addr":"172.21.15.188:6831","nonce":2722096729}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6828","nonce":2722096729},{"type":"v1","addr":"172.21.15.188:6829","nonce":2722096729}]},"public_addr":"172.21.15.188:6825/2722096729","cluster_addr":"172.21.15.188:6827/2722096729","heartbeat_back_addr":"172.21.15.188:6831/2722096729","heartbeat_front_addr":"172.21.15.188:6829/2722096729","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-26T07:17:36.995032+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-26T07:17:36.932266+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-26T07:17:36.829304+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-26T07:17:36.804297+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-26T07:17:36.667892+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-26T07:17:36.964868+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-26T07:17:36.827332+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-26T07:17:36.893179+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-11-26T07:23:33.364 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:33.363+0000 7f3604fc4640 1 -- 172.21.15.143:0/3607634881 >> [v2:172.21.15.188:6836/3401128553,v1:172.21.15.188:6837/3401128553] conn(0x7f35e0066590 msgr2=0x7f35e0068a50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:23:33.364 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:33.363+0000 7f3604fc4640 1 --2- 172.21.15.143:0/3607634881 >> [v2:172.21.15.188:6836/3401128553,v1:172.21.15.188:6837/3401128553] conn(0x7f35e0066590 0x7f35e0068a50 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7f35f0000c10 tx=0x7f35f0001e40 comp rx=0 tx=0).stop 2023-11-26T07:23:33.364 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:33.363+0000 7f3604fc4640 1 -- 172.21.15.143:0/3607634881 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f36000b3330 msgr2=0x7f36000cffa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:23:33.364 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:33.363+0000 7f3604fc4640 1 --2- 172.21.15.143:0/3607634881 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f36000b3330 0x7f36000cffa0 secure :-1 s=READY pgs=1152 cs=0 l=1 rev1=1 crypto rx=0x7f35f4011660 tx=0x7f35f4011b20 comp rx=0 tx=0).stop 2023-11-26T07:23:33.364 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:33.363+0000 7f3604fc4640 1 -- 172.21.15.143:0/3607634881 shutdown_connections 2023-11-26T07:23:33.364 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:33.363+0000 7f3604fc4640 1 --2- 172.21.15.143:0/3607634881 >> [v2:172.21.15.188:6836/3401128553,v1:172.21.15.188:6837/3401128553] conn(0x7f35e0066590 0x7f35e0068a50 unknown :-1 s=CLOSED pgs=62 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:23:33.365 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:33.363+0000 7f3604fc4640 1 --2- 172.21.15.143:0/3607634881 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f36000d04e0 0x7f36001f32c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:23:33.365 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:33.363+0000 7f3604fc4640 1 --2- 172.21.15.143:0/3607634881 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f36000b3330 0x7f36000cffa0 unknown :-1 s=CLOSED pgs=1152 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:23:33.365 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:33.363+0000 7f3604fc4640 1 --2- 172.21.15.143:0/3607634881 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f35e8004d10 0x7f36001603b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:23:33.365 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:33.363+0000 7f3604fc4640 1 -- 172.21.15.143:0/3607634881 >> 172.21.15.143:0/3607634881 conn(0x7f360009c550 msgr2=0x7f36000cf490 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:23:33.365 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:33.363+0000 7f3604fc4640 1 -- 172.21.15.143:0/3607634881 shutdown_connections 2023-11-26T07:23:33.365 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:33.363+0000 7f3604fc4640 1 -- 172.21.15.143:0/3607634881 wait complete. 2023-11-26T07:23:33.383 INFO:tasks.cephfs.filesystem.ceph_manager:waiting for clean 2023-11-26T07:23:33.383 DEBUG:teuthology.orchestra.run.smithi143:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-11-26T07:23:33.521 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:33.519+0000 7efda33df640 1 Processor -- start 2023-11-26T07:23:33.522 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:33.519+0000 7efda33df640 1 -- start start 2023-11-26T07:23:33.524 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:33.523+0000 7efda33df640 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7efd9c070d10 0x7efd9c15d900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:23:33.524 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:33.523+0000 7efda33df640 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7efd9c06cd50 0x7efd9c0b2180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:23:33.524 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:33.523+0000 7efda33df640 1 -- --> v1:172.21.15.143:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7efd9c15b2f0 con 0x7efd9c0cf280 2023-11-26T07:23:33.524 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:33.523+0000 7efda33df640 1 -- --> v2:172.21.15.143:3300/0 -- mon_getmap magic: 0 v1 -- 0x7efd9c06c5c0 con 0x7efd9c06cd50 2023-11-26T07:23:33.524 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:33.523+0000 7efda33df640 1 -- --> v2:172.21.15.188:3300/0 -- mon_getmap magic: 0 v1 -- 0x7efd9c15e1a0 con 0x7efd9c070d10 2023-11-26T07:23:33.524 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:33.523+0000 7efda197c640 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7efd9c06cd50 0x7efd9c0b2180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:23:33.524 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:33.523+0000 7efda117b640 1 --1- >> v1:172.21.15.143:6789/0 conn(0x7efd9c0cf280 0x7efd9c0cf660 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.143:6789/0 says I am v1:172.21.15.143:43218/0 (socket says 172.21.15.143:43218) 2023-11-26T07:23:33.524 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:33.523+0000 7efda117b640 1 -- 172.21.15.143:0/2978026727 learned_addr learned my addr 172.21.15.143:0/2978026727 (peer_addr_for_me v1:172.21.15.143:0/0) 2023-11-26T07:23:33.524 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:33.523+0000 7efda097a640 1 --2- 172.21.15.143:0/2978026727 >> v2:172.21.15.188:3300/0 conn(0x7efd9c070d10 0x7efd9c15d900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:23:33.524 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:33.523+0000 7efd93fff640 1 -- 172.21.15.143:0/2978026727 <== mon.0 v1:172.21.15.143:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 517258637 0 0) 0x7efd98001120 con 0x7efd9c0cf280 2023-11-26T07:23:33.524 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:33.523+0000 7efd93fff640 1 -- 172.21.15.143:0/2978026727 --> v1:172.21.15.143:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7efd88003530 con 0x7efd9c0cf280 2023-11-26T07:23:33.525 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:33.523+0000 7efda197c640 1 -- 172.21.15.143:0/2978026727 >> v2:172.21.15.188:3300/0 conn(0x7efd9c070d10 msgr2=0x7efd9c15d900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-26T07:23:33.525 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:33.523+0000 7efda197c640 1 --2- 172.21.15.143:0/2978026727 >> v2:172.21.15.188:3300/0 conn(0x7efd9c070d10 0x7efd9c15d900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:23:33.525 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:33.523+0000 7efda197c640 1 -- 172.21.15.143:0/2978026727 >> v1:172.21.15.143:6789/0 conn(0x7efd9c0cf280 legacy=0x7efd9c0cf660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:23:33.525 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:33.523+0000 7efda197c640 1 -- 172.21.15.143:0/2978026727 --> v2:172.21.15.143:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efd9c15e480 con 0x7efd9c06cd50 2023-11-26T07:23:33.526 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:33.527+0000 7efda197c640 1 --2- 172.21.15.143:0/2978026727 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7efd9c06cd50 0x7efd9c0b2180 secure :-1 s=READY pgs=1154 cs=0 l=1 rev1=1 crypto rx=0x7efd9c06e480 tx=0x7efd9401f0f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fe7c16098f295411 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:23:33.527 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:33.527+0000 7efd93fff640 1 -- 172.21.15.143:0/2978026727 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efd940b0440 con 0x7efd9c06cd50 2023-11-26T07:23:33.527 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:33.527+0000 7efd93fff640 1 -- 172.21.15.143:0/2978026727 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7efd940b05a0 con 0x7efd9c06cd50 2023-11-26T07:23:33.527 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:33.527+0000 7efd93fff640 1 -- 172.21.15.143:0/2978026727 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efd940b0930 con 0x7efd9c06cd50 2023-11-26T07:23:33.528 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:33.527+0000 7efda33df640 1 -- 172.21.15.143:0/2978026727 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7efd9c06cd50 msgr2=0x7efd9c0b2180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:23:33.528 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:33.527+0000 7efda33df640 1 --2- 172.21.15.143:0/2978026727 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7efd9c06cd50 0x7efd9c0b2180 secure :-1 s=READY pgs=1154 cs=0 l=1 rev1=1 crypto rx=0x7efd9c06e480 tx=0x7efd9401f0f0 comp rx=0 tx=0).stop 2023-11-26T07:23:33.528 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:33.527+0000 7efda33df640 1 -- 172.21.15.143:0/2978026727 shutdown_connections 2023-11-26T07:23:33.528 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:33.527+0000 7efda33df640 1 --2- 172.21.15.143:0/2978026727 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7efd9c06cd50 0x7efd9c0b2180 unknown :-1 s=CLOSED pgs=1154 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:23:33.528 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:33.527+0000 7efda33df640 1 --2- 172.21.15.143:0/2978026727 >> v2:172.21.15.188:3300/0 conn(0x7efd9c070d10 0x7efd9c15d900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:23:33.528 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:33.527+0000 7efda33df640 1 -- 172.21.15.143:0/2978026727 >> 172.21.15.143:0/2978026727 conn(0x7efd9c06a890 msgr2=0x7efd9c06aca0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:23:33.528 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:33.527+0000 7efda33df640 1 -- 172.21.15.143:0/2978026727 shutdown_connections 2023-11-26T07:23:33.528 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:33.527+0000 7efda33df640 1 -- 172.21.15.143:0/2978026727 wait complete. 2023-11-26T07:23:33.529 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:33.527+0000 7efda33df640 1 Processor -- start 2023-11-26T07:23:33.529 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:33.527+0000 7efda33df640 1 -- start start 2023-11-26T07:23:33.532 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:33.531+0000 7efda33df640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7efd9c06cd50 0x7efd9c1e4f90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:23:33.532 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:33.531+0000 7efda33df640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7efd9c070d10 0x7efd9c146730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:23:33.532 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:33.531+0000 7efda33df640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7efd9c0cf280 0x7efd9c1e35f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:23:33.532 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:33.531+0000 7efda33df640 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efd9c1e54d0 con 0x7efd9c06cd50 2023-11-26T07:23:33.532 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:33.531+0000 7efda33df640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efd9c1e3ef0 con 0x7efd9c070d10 2023-11-26T07:23:33.532 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:33.531+0000 7efda33df640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7efd9c1e41d0 con 0x7efd9c0cf280 2023-11-26T07:23:33.532 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:33.531+0000 7efda117b640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7efd9c06cd50 0x7efd9c1e4f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:23:33.532 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:33.531+0000 7efda117b640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7efd9c06cd50 0x7efd9c1e4f90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:46404/0 (socket says 172.21.15.143:46404) 2023-11-26T07:23:33.532 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:33.531+0000 7efda117b640 1 -- 172.21.15.143:0/2729490386 learned_addr learned my addr 172.21.15.143:0/2729490386 (peer_addr_for_me v2:172.21.15.143:0/0) 2023-11-26T07:23:33.532 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:33.531+0000 7efda097a640 1 --2- 172.21.15.143:0/2729490386 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7efd9c070d10 0x7efd9c146730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:23:33.532 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:33.531+0000 7efda197c640 1 --2- 172.21.15.143:0/2729490386 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7efd9c0cf280 0x7efd9c1e35f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:23:33.533 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:33.531+0000 7efda117b640 1 -- 172.21.15.143:0/2729490386 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7efd9c0cf280 msgr2=0x7efd9c1e35f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:23:33.533 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:33.531+0000 7efda117b640 1 --2- 172.21.15.143:0/2729490386 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7efd9c0cf280 0x7efd9c1e35f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:23:33.533 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:33.531+0000 7efda117b640 1 -- 172.21.15.143:0/2729490386 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7efd9c070d10 msgr2=0x7efd9c146730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:23:33.533 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:33.531+0000 7efda117b640 1 --2- 172.21.15.143:0/2729490386 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7efd9c070d10 0x7efd9c146730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:23:33.533 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:33.531+0000 7efda117b640 1 -- 172.21.15.143:0/2729490386 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efd94009a70 con 0x7efd9c06cd50 2023-11-26T07:23:33.533 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:33.531+0000 7efda197c640 1 --2- 172.21.15.143:0/2729490386 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7efd9c0cf280 0x7efd9c1e35f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-26T07:23:33.533 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:33.531+0000 7efda117b640 1 --2- 172.21.15.143:0/2729490386 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7efd9c06cd50 0x7efd9c1e4f90 secure :-1 s=READY pgs=1155 cs=0 l=1 rev1=1 crypto rx=0x7efd9800c700 tx=0x7efd9800cbc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:23:33.533 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:33.531+0000 7efd91ffb640 1 -- 172.21.15.143:0/2729490386 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efd98005ac0 con 0x7efd9c06cd50 2023-11-26T07:23:33.533 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:33.531+0000 7efd91ffb640 1 -- 172.21.15.143:0/2729490386 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7efd98005c60 con 0x7efd9c06cd50 2023-11-26T07:23:33.533 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:33.531+0000 7efda33df640 1 -- 172.21.15.143:0/2729490386 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7efd9c1e44b0 con 0x7efd9c06cd50 2023-11-26T07:23:33.533 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:33.531+0000 7efd91ffb640 1 -- 172.21.15.143:0/2729490386 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efd98001d60 con 0x7efd9c06cd50 2023-11-26T07:23:33.533 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:33.531+0000 7efda33df640 1 -- 172.21.15.143:0/2729490386 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7efd9c1e4a70 con 0x7efd9c06cd50 2023-11-26T07:23:33.534 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:33.531+0000 7efda33df640 1 -- 172.21.15.143:0/2729490386 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7efd9c067680 con 0x7efd9c06cd50 2023-11-26T07:23:33.534 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:33.531+0000 7efd91ffb640 1 -- 172.21.15.143:0/2729490386 <== mon.0 v2:172.21.15.143:3300/0 4 ==== mgrmap(e 16) v1 ==== 85954+0+0 (secure 0 0 0) 0x7efd98007070 con 0x7efd9c06cd50 2023-11-26T07:23:33.534 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:33.535+0000 7efd91ffb640 1 --2- 172.21.15.143:0/2729490386 >> [v2:172.21.15.188:6836/3401128553,v1:172.21.15.188:6837/3401128553] conn(0x7efd80066590 0x7efd80068a50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:23:33.535 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:33.535+0000 7efd91ffb640 1 -- 172.21.15.143:0/2729490386 <== mon.0 v2:172.21.15.143:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 5709+0+0 (secure 0 0 0) 0x7efd98083310 con 0x7efd9c06cd50 2023-11-26T07:23:33.535 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:33.535+0000 7efda097a640 1 --2- 172.21.15.143:0/2729490386 >> [v2:172.21.15.188:6836/3401128553,v1:172.21.15.188:6837/3401128553] conn(0x7efd80066590 0x7efd80068a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:23:33.535 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:33.535+0000 7efda097a640 1 --2- 172.21.15.143:0/2729490386 >> [v2:172.21.15.188:6836/3401128553,v1:172.21.15.188:6837/3401128553] conn(0x7efd80066590 0x7efd80068a50 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7efd9c146ff0 tx=0x7efd8c008040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:23:33.538 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:33.539+0000 7efd91ffb640 1 -- 172.21.15.143:0/2729490386 <== mon.0 v2:172.21.15.143:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172649 (secure 0 0 0) 0x7efd98050720 con 0x7efd9c06cd50 2023-11-26T07:23:33.688 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:33.687+0000 7efda33df640 1 -- 172.21.15.143:0/2729490386 --> [v2:172.21.15.188:6836/3401128553,v1:172.21.15.188:6837/3401128553] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7efd9c0b3fa0 con 0x7efd80066590 2023-11-26T07:23:33.698 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:33.699+0000 7efd91ffb640 1 -- 172.21.15.143:0/2729490386 <== mgr.4108 v2:172.21.15.188:6836/3401128553 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+289498 (secure 0 0 0) 0x7efd9c0b3fa0 con 0x7efd80066590 2023-11-26T07:23:33.701 INFO:teuthology.orchestra.run.smithi143.stdout: 2023-11-26T07:23:33.703 INFO:teuthology.orchestra.run.smithi143.stderr:dumped all 2023-11-26T07:23:33.704 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:33.699+0000 7efda33df640 1 -- 172.21.15.143:0/2729490386 >> [v2:172.21.15.188:6836/3401128553,v1:172.21.15.188:6837/3401128553] conn(0x7efd80066590 msgr2=0x7efd80068a50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:23:33.704 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:33.703+0000 7efda33df640 1 --2- 172.21.15.143:0/2729490386 >> [v2:172.21.15.188:6836/3401128553,v1:172.21.15.188:6837/3401128553] conn(0x7efd80066590 0x7efd80068a50 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7efd9c146ff0 tx=0x7efd8c008040 comp rx=0 tx=0).stop 2023-11-26T07:23:33.704 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:33.703+0000 7efda33df640 1 -- 172.21.15.143:0/2729490386 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7efd9c06cd50 msgr2=0x7efd9c1e4f90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:23:33.704 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:33.703+0000 7efda33df640 1 --2- 172.21.15.143:0/2729490386 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7efd9c06cd50 0x7efd9c1e4f90 secure :-1 s=READY pgs=1155 cs=0 l=1 rev1=1 crypto rx=0x7efd9800c700 tx=0x7efd9800cbc0 comp rx=0 tx=0).stop 2023-11-26T07:23:33.704 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:33.703+0000 7efda33df640 1 -- 172.21.15.143:0/2729490386 shutdown_connections 2023-11-26T07:23:33.704 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:33.703+0000 7efda33df640 1 --2- 172.21.15.143:0/2729490386 >> [v2:172.21.15.188:6836/3401128553,v1:172.21.15.188:6837/3401128553] conn(0x7efd80066590 0x7efd80068a50 unknown :-1 s=CLOSED pgs=63 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:23:33.704 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:33.703+0000 7efda33df640 1 --2- 172.21.15.143:0/2729490386 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7efd9c0cf280 0x7efd9c1e35f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:23:33.704 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:33.703+0000 7efda33df640 1 --2- 172.21.15.143:0/2729490386 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7efd9c070d10 0x7efd9c146730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:23:33.704 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:33.703+0000 7efda33df640 1 --2- 172.21.15.143:0/2729490386 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7efd9c06cd50 0x7efd9c1e4f90 unknown :-1 s=CLOSED pgs=1155 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:23:33.705 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:33.703+0000 7efda33df640 1 -- 172.21.15.143:0/2729490386 >> 172.21.15.143:0/2729490386 conn(0x7efd9c06a890 msgr2=0x7efd9c15cb70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:23:33.705 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:33.703+0000 7efda33df640 1 -- 172.21.15.143:0/2729490386 shutdown_connections 2023-11-26T07:23:33.705 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:33.703+0000 7efda33df640 1 -- 172.21.15.143:0/2729490386 wait complete. 2023-11-26T07:23:33.721 INFO:teuthology.orchestra.run.smithi143.stdout:{"pg_ready":true,"pg_map":{"version":27,"stamp":"2023-11-26T07:23:33.288565+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592778,"num_objects":23,"num_object_clones":0,"num_object_copies":46,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":23,"num_whiteouts":0,"num_read":212,"num_read_kb":306,"num_write":223,"num_write_kb":2430,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":166,"ondisk_log_size":166,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":189,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":254596,"kb_used_data":9288,"kb_used_omap":12,"kb_used_meta":245171,"kb_avail":754720124,"statfs":{"total":773094113280,"available":772833406976,"internally_reserved":0,"allocated":9510912,"data_stored":4739068,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12749,"internal_metadata":251055667},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":42,"apply_latency_ms":42,"commit_latency_ns":42000000,"apply_latency_ns":42000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2410,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"5.670150"},"pg_stats":[{"pgid":"13.3f","version":"0'0","reported_seq":11,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-11-26T07:23:30.644611+0000","last_change":"2023-11-26T07:23:29.664581+0000","last_active":"2023-11-26T07:23:30.644611+0000","last_peered":"2023-11-26T07:23:30.644611+0000","last_clean":"2023-11-26T07:23:30.644611+0000","last_became_active":"2023-11-26T07:23:29.661507+0000","last_became_peered":"2023-11-26T07:23:29.661507+0000","last_unstale":"2023-11-26T07:23:30.644611+0000","last_undegraded":"2023-11-26T07:23:30.644611+0000","last_fullsized":"2023-11-26T07:23:30.644611+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:23:28.619155+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:23:28.619155+0000","last_clean_scrub_stamp":"2023-11-26T07:23:28.619155+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T11:28:13.301129+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.3e","version":"0'0","reported_seq":11,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-11-26T07:23:31.156930+0000","last_change":"2023-11-26T07:23:29.678054+0000","last_active":"2023-11-26T07:23:31.156930+0000","last_peered":"2023-11-26T07:23:31.156930+0000","last_clean":"2023-11-26T07:23:31.156930+0000","last_became_active":"2023-11-26T07:23:29.677694+0000","last_became_peered":"2023-11-26T07:23:29.677694+0000","last_unstale":"2023-11-26T07:23:31.156930+0000","last_undegraded":"2023-11-26T07:23:31.156930+0000","last_fullsized":"2023-11-26T07:23:31.156930+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:23:28.619155+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:23:28.619155+0000","last_clean_scrub_stamp":"2023-11-26T07:23:28.619155+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T17:13:41.724273+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.3d","version":"0'0","reported_seq":11,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-11-26T07:23:31.154692+0000","last_change":"2023-11-26T07:23:29.654061+0000","last_active":"2023-11-26T07:23:31.154692+0000","last_peered":"2023-11-26T07:23:31.154692+0000","last_clean":"2023-11-26T07:23:31.154692+0000","last_became_active":"2023-11-26T07:23:29.653070+0000","last_became_peered":"2023-11-26T07:23:29.653070+0000","last_unstale":"2023-11-26T07:23:31.154692+0000","last_undegraded":"2023-11-26T07:23:31.154692+0000","last_fullsized":"2023-11-26T07:23:31.154692+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:23:28.619155+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:23:28.619155+0000","last_clean_scrub_stamp":"2023-11-26T07:23:28.619155+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T09:04:09.764282+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"13.3c","version":"0'0","reported_seq":11,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-11-26T07:23:30.643254+0000","last_change":"2023-11-26T07:23:29.664735+0000","last_active":"2023-11-26T07:23:30.643254+0000","last_peered":"2023-11-26T07:23:30.643254+0000","last_clean":"2023-11-26T07:23:30.643254+0000","last_became_active":"2023-11-26T07:23:29.664220+0000","last_became_peered":"2023-11-26T07:23:29.664220+0000","last_unstale":"2023-11-26T07:23:30.643254+0000","last_undegraded":"2023-11-26T07:23:30.643254+0000","last_fullsized":"2023-11-26T07:23:30.643254+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:23:28.619155+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:23:28.619155+0000","last_clean_scrub_stamp":"2023-11-26T07:23:28.619155+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T16:05:51.826873+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"13.3b","version":"0'0","reported_seq":11,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-11-26T07:23:31.153983+0000","last_change":"2023-11-26T07:23:29.673766+0000","last_active":"2023-11-26T07:23:31.153983+0000","last_peered":"2023-11-26T07:23:31.153983+0000","last_clean":"2023-11-26T07:23:31.153983+0000","last_became_active":"2023-11-26T07:23:29.670876+0000","last_became_peered":"2023-11-26T07:23:29.670876+0000","last_unstale":"2023-11-26T07:23:31.153983+0000","last_undegraded":"2023-11-26T07:23:31.153983+0000","last_fullsized":"2023-11-26T07:23:31.153983+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:23:28.619155+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:23:28.619155+0000","last_clean_scrub_stamp":"2023-11-26T07:23:28.619155+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T09:05:12.255253+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"13.3a","version":"0'0","reported_seq":11,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-11-26T07:23:30.688618+0000","last_change":"2023-11-26T07:23:29.666579+0000","last_active":"2023-11-26T07:23:30.688618+0000","last_peered":"2023-11-26T07:23:30.688618+0000","last_clean":"2023-11-26T07:23:30.688618+0000","last_became_active":"2023-11-26T07:23:29.666318+0000","last_became_peered":"2023-11-26T07:23:29.666318+0000","last_unstale":"2023-11-26T07:23:30.688618+0000","last_undegraded":"2023-11-26T07:23:30.688618+0000","last_fullsized":"2023-11-26T07:23:30.688618+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:23:28.619155+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:23:28.619155+0000","last_clean_scrub_stamp":"2023-11-26T07:23:28.619155+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T10:07:49.922280+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"13.39","version":"0'0","reported_seq":11,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-11-26T07:23:30.643204+0000","last_change":"2023-11-26T07:23:29.662660+0000","last_active":"2023-11-26T07:23:30.643204+0000","last_peered":"2023-11-26T07:23:30.643204+0000","last_clean":"2023-11-26T07:23:30.643204+0000","last_became_active":"2023-11-26T07:23:29.662287+0000","last_became_peered":"2023-11-26T07:23:29.662287+0000","last_unstale":"2023-11-26T07:23:30.643204+0000","last_undegraded":"2023-11-26T07:23:30.643204+0000","last_fullsized":"2023-11-26T07:23:30.643204+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:23:28.619155+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:23:28.619155+0000","last_clean_scrub_stamp":"2023-11-26T07:23:28.619155+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T07:56:03.491313+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"13.38","version":"0'0","reported_seq":11,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-11-26T07:23:30.688623+0000","last_change":"2023-11-26T07:23:29.660746+0000","last_active":"2023-11-26T07:23:30.688623+0000","last_peered":"2023-11-26T07:23:30.688623+0000","last_clean":"2023-11-26T07:23:30.688623+0000","last_became_active":"2023-11-26T07:23:29.660479+0000","last_became_peered":"2023-11-26T07:23:29.660479+0000","last_unstale":"2023-11-26T07:23:30.688623+0000","last_undegraded":"2023-11-26T07:23:30.688623+0000","last_fullsized":"2023-11-26T07:23:30.688623+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:23:28.619155+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:23:28.619155+0000","last_clean_scrub_stamp":"2023-11-26T07:23:28.619155+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T09:25:04.676044+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"13.37","version":"0'0","reported_seq":11,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-11-26T07:23:30.688912+0000","last_change":"2023-11-26T07:23:29.660637+0000","last_active":"2023-11-26T07:23:30.688912+0000","last_peered":"2023-11-26T07:23:30.688912+0000","last_clean":"2023-11-26T07:23:30.688912+0000","last_became_active":"2023-11-26T07:23:29.660358+0000","last_became_peered":"2023-11-26T07:23:29.660358+0000","last_unstale":"2023-11-26T07:23:30.688912+0000","last_undegraded":"2023-11-26T07:23:30.688912+0000","last_fullsized":"2023-11-26T07:23:30.688912+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:23:28.619155+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:23:28.619155+0000","last_clean_scrub_stamp":"2023-11-26T07:23:28.619155+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T16:10:09.119282+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"13.36","version":"0'0","reported_seq":11,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-11-26T07:23:30.644543+0000","last_change":"2023-11-26T07:23:29.660073+0000","last_active":"2023-11-26T07:23:30.644543+0000","last_peered":"2023-11-26T07:23:30.644543+0000","last_clean":"2023-11-26T07:23:30.644543+0000","last_became_active":"2023-11-26T07:23:29.659776+0000","last_became_peered":"2023-11-26T07:23:29.659776+0000","last_unstale":"2023-11-26T07:23:30.644543+0000","last_undegraded":"2023-11-26T07:23:30.644543+0000","last_fullsized":"2023-11-26T07:23:30.644543+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:23:28.619155+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:23:28.619155+0000","last_clean_scrub_stamp":"2023-11-26T07:23:28.619155+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T13:34:20.128647+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.35","version":"0'0","reported_seq":11,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-11-26T07:23:31.156914+0000","last_change":"2023-11-26T07:23:29.664297+0000","last_active":"2023-11-26T07:23:31.156914+0000","last_peered":"2023-11-26T07:23:31.156914+0000","last_clean":"2023-11-26T07:23:31.156914+0000","last_became_active":"2023-11-26T07:23:29.663421+0000","last_became_peered":"2023-11-26T07:23:29.663421+0000","last_unstale":"2023-11-26T07:23:31.156914+0000","last_undegraded":"2023-11-26T07:23:31.156914+0000","last_fullsized":"2023-11-26T07:23:31.156914+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:23:28.619155+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:23:28.619155+0000","last_clean_scrub_stamp":"2023-11-26T07:23:28.619155+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T09:12:34.800308+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.34","version":"0'0","reported_seq":11,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-11-26T07:23:30.644600+0000","last_change":"2023-11-26T07:23:29.659988+0000","last_active":"2023-11-26T07:23:30.644600+0000","last_peered":"2023-11-26T07:23:30.644600+0000","last_clean":"2023-11-26T07:23:30.644600+0000","last_became_active":"2023-11-26T07:23:29.659673+0000","last_became_peered":"2023-11-26T07:23:29.659673+0000","last_unstale":"2023-11-26T07:23:30.644600+0000","last_undegraded":"2023-11-26T07:23:30.644600+0000","last_fullsized":"2023-11-26T07:23:30.644600+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:23:28.619155+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:23:28.619155+0000","last_clean_scrub_stamp":"2023-11-26T07:23:28.619155+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T15:32:15.823182+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.33","version":"0'0","reported_seq":11,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-11-26T07:23:31.154242+0000","last_change":"2023-11-26T07:23:29.662429+0000","last_active":"2023-11-26T07:23:31.154242+0000","last_peered":"2023-11-26T07:23:31.154242+0000","last_clean":"2023-11-26T07:23:31.154242+0000","last_became_active":"2023-11-26T07:23:29.662131+0000","last_became_peered":"2023-11-26T07:23:29.662131+0000","last_unstale":"2023-11-26T07:23:31.154242+0000","last_undegraded":"2023-11-26T07:23:31.154242+0000","last_fullsized":"2023-11-26T07:23:31.154242+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:23:28.619155+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:23:28.619155+0000","last_clean_scrub_stamp":"2023-11-26T07:23:28.619155+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T17:06:44.573286+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"13.32","version":"0'0","reported_seq":11,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-11-26T07:23:30.642938+0000","last_change":"2023-11-26T07:23:29.666985+0000","last_active":"2023-11-26T07:23:30.642938+0000","last_peered":"2023-11-26T07:23:30.642938+0000","last_clean":"2023-11-26T07:23:30.642938+0000","last_became_active":"2023-11-26T07:23:29.666656+0000","last_became_peered":"2023-11-26T07:23:29.666656+0000","last_unstale":"2023-11-26T07:23:30.642938+0000","last_undegraded":"2023-11-26T07:23:30.642938+0000","last_fullsized":"2023-11-26T07:23:30.642938+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:23:28.619155+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:23:28.619155+0000","last_clean_scrub_stamp":"2023-11-26T07:23:28.619155+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T11:10:41.460364+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"13.31","version":"0'0","reported_seq":11,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-11-26T07:23:31.155303+0000","last_change":"2023-11-26T07:23:29.646803+0000","last_active":"2023-11-26T07:23:31.155303+0000","last_peered":"2023-11-26T07:23:31.155303+0000","last_clean":"2023-11-26T07:23:31.155303+0000","last_became_active":"2023-11-26T07:23:29.646514+0000","last_became_peered":"2023-11-26T07:23:29.646514+0000","last_unstale":"2023-11-26T07:23:31.155303+0000","last_undegraded":"2023-11-26T07:23:31.155303+0000","last_fullsized":"2023-11-26T07:23:31.155303+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:23:28.619155+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:23:28.619155+0000","last_clean_scrub_stamp":"2023-11-26T07:23:28.619155+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T17:06:44.573286+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"13.30","version":"0'0","reported_seq":11,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-11-26T07:23:31.153951+0000","last_change":"2023-11-26T07:23:29.672255+0000","last_active":"2023-11-26T07:23:31.153951+0000","last_peered":"2023-11-26T07:23:31.153951+0000","last_clean":"2023-11-26T07:23:31.153951+0000","last_became_active":"2023-11-26T07:23:29.671965+0000","last_became_peered":"2023-11-26T07:23:29.671965+0000","last_unstale":"2023-11-26T07:23:31.153951+0000","last_undegraded":"2023-11-26T07:23:31.153951+0000","last_fullsized":"2023-11-26T07:23:31.153951+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:23:28.619155+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:23:28.619155+0000","last_clean_scrub_stamp":"2023-11-26T07:23:28.619155+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T13:22:18.189493+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"13.2f","version":"0'0","reported_seq":11,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-11-26T07:23:31.154225+0000","last_change":"2023-11-26T07:23:29.667334+0000","last_active":"2023-11-26T07:23:31.154225+0000","last_peered":"2023-11-26T07:23:31.154225+0000","last_clean":"2023-11-26T07:23:31.154225+0000","last_became_active":"2023-11-26T07:23:29.666606+0000","last_became_peered":"2023-11-26T07:23:29.666606+0000","last_unstale":"2023-11-26T07:23:31.154225+0000","last_undegraded":"2023-11-26T07:23:31.154225+0000","last_fullsized":"2023-11-26T07:23:31.154225+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:23:28.619155+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:23:28.619155+0000","last_clean_scrub_stamp":"2023-11-26T07:23:28.619155+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T16:10:09.119282+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"13.2e","version":"0'0","reported_seq":11,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-11-26T07:23:30.642909+0000","last_change":"2023-11-26T07:23:29.665013+0000","last_active":"2023-11-26T07:23:30.642909+0000","last_peered":"2023-11-26T07:23:30.642909+0000","last_clean":"2023-11-26T07:23:30.642909+0000","last_became_active":"2023-11-26T07:23:29.663771+0000","last_became_peered":"2023-11-26T07:23:29.663771+0000","last_unstale":"2023-11-26T07:23:30.642909+0000","last_undegraded":"2023-11-26T07:23:30.642909+0000","last_fullsized":"2023-11-26T07:23:30.642909+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:23:28.619155+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:23:28.619155+0000","last_clean_scrub_stamp":"2023-11-26T07:23:28.619155+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T08:06:41.980749+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"13.2d","version":"0'0","reported_seq":11,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-11-26T07:23:31.157149+0000","last_change":"2023-11-26T07:23:29.652309+0000","last_active":"2023-11-26T07:23:31.157149+0000","last_peered":"2023-11-26T07:23:31.157149+0000","last_clean":"2023-11-26T07:23:31.157149+0000","last_became_active":"2023-11-26T07:23:29.651997+0000","last_became_peered":"2023-11-26T07:23:29.651997+0000","last_unstale":"2023-11-26T07:23:31.157149+0000","last_undegraded":"2023-11-26T07:23:31.157149+0000","last_fullsized":"2023-11-26T07:23:31.157149+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:23:28.619155+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:23:28.619155+0000","last_clean_scrub_stamp":"2023-11-26T07:23:28.619155+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T11:31:08.516775+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.2c","version":"0'0","reported_seq":11,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-11-26T07:23:30.643382+0000","last_change":"2023-11-26T07:23:29.664507+0000","last_active":"2023-11-26T07:23:30.643382+0000","last_peered":"2023-11-26T07:23:30.643382+0000","last_clean":"2023-11-26T07:23:30.643382+0000","last_became_active":"2023-11-26T07:23:29.663927+0000","last_became_peered":"2023-11-26T07:23:29.663927+0000","last_unstale":"2023-11-26T07:23:30.643382+0000","last_undegraded":"2023-11-26T07:23:30.643382+0000","last_fullsized":"2023-11-26T07:23:30.643382+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:23:28.619155+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:23:28.619155+0000","last_clean_scrub_stamp":"2023-11-26T07:23:28.619155+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T12:03:35.864250+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"13.2b","version":"0'0","reported_seq":11,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-11-26T07:23:30.642837+0000","last_change":"2023-11-26T07:23:29.665728+0000","last_active":"2023-11-26T07:23:30.642837+0000","last_peered":"2023-11-26T07:23:30.642837+0000","last_clean":"2023-11-26T07:23:30.642837+0000","last_became_active":"2023-11-26T07:23:29.665433+0000","last_became_peered":"2023-11-26T07:23:29.665433+0000","last_unstale":"2023-11-26T07:23:30.642837+0000","last_undegraded":"2023-11-26T07:23:30.642837+0000","last_fullsized":"2023-11-26T07:23:30.642837+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:23:28.619155+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:23:28.619155+0000","last_clean_scrub_stamp":"2023-11-26T07:23:28.619155+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T08:18:30.568517+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"13.2a","version":"0'0","reported_seq":11,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-11-26T07:23:31.156298+0000","last_change":"2023-11-26T07:23:29.672791+0000","last_active":"2023-11-26T07:23:31.156298+0000","last_peered":"2023-11-26T07:23:31.156298+0000","last_clean":"2023-11-26T07:23:31.156298+0000","last_became_active":"2023-11-26T07:23:29.672520+0000","last_became_peered":"2023-11-26T07:23:29.672520+0000","last_unstale":"2023-11-26T07:23:31.156298+0000","last_undegraded":"2023-11-26T07:23:31.156298+0000","last_fullsized":"2023-11-26T07:23:31.156298+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:23:28.619155+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:23:28.619155+0000","last_clean_scrub_stamp":"2023-11-26T07:23:28.619155+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T14:22:16.614334+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.29","version":"0'0","reported_seq":11,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-11-26T07:23:30.643988+0000","last_change":"2023-11-26T07:23:29.658265+0000","last_active":"2023-11-26T07:23:30.643988+0000","last_peered":"2023-11-26T07:23:30.643988+0000","last_clean":"2023-11-26T07:23:30.643988+0000","last_became_active":"2023-11-26T07:23:29.657987+0000","last_became_peered":"2023-11-26T07:23:29.657987+0000","last_unstale":"2023-11-26T07:23:30.643988+0000","last_undegraded":"2023-11-26T07:23:30.643988+0000","last_fullsized":"2023-11-26T07:23:30.643988+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:23:28.619155+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:23:28.619155+0000","last_clean_scrub_stamp":"2023-11-26T07:23:28.619155+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T08:36:27.045499+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.28","version":"0'0","reported_seq":11,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-11-26T07:23:30.642665+0000","last_change":"2023-11-26T07:23:29.662145+0000","last_active":"2023-11-26T07:23:30.642665+0000","last_peered":"2023-11-26T07:23:30.642665+0000","last_clean":"2023-11-26T07:23:30.642665+0000","last_became_active":"2023-11-26T07:23:29.661765+0000","last_became_peered":"2023-11-26T07:23:29.661765+0000","last_unstale":"2023-11-26T07:23:30.642665+0000","last_undegraded":"2023-11-26T07:23:30.642665+0000","last_fullsized":"2023-11-26T07:23:30.642665+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:23:28.619155+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:23:28.619155+0000","last_clean_scrub_stamp":"2023-11-26T07:23:28.619155+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T10:26:33.963940+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"13.27","version":"0'0","reported_seq":11,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-11-26T07:23:31.156627+0000","last_change":"2023-11-26T07:23:29.670931+0000","last_active":"2023-11-26T07:23:31.156627+0000","last_peered":"2023-11-26T07:23:31.156627+0000","last_clean":"2023-11-26T07:23:31.156627+0000","last_became_active":"2023-11-26T07:23:29.669176+0000","last_became_peered":"2023-11-26T07:23:29.669176+0000","last_unstale":"2023-11-26T07:23:31.156627+0000","last_undegraded":"2023-11-26T07:23:31.156627+0000","last_fullsized":"2023-11-26T07:23:31.156627+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:23:28.619155+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:23:28.619155+0000","last_clean_scrub_stamp":"2023-11-26T07:23:28.619155+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T15:11:46.786215+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.26","version":"0'0","reported_seq":11,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-11-26T07:23:31.154789+0000","last_change":"2023-11-26T07:23:29.659262+0000","last_active":"2023-11-26T07:23:31.154789+0000","last_peered":"2023-11-26T07:23:31.154789+0000","last_clean":"2023-11-26T07:23:31.154789+0000","last_became_active":"2023-11-26T07:23:29.658907+0000","last_became_peered":"2023-11-26T07:23:29.658907+0000","last_unstale":"2023-11-26T07:23:31.154789+0000","last_undegraded":"2023-11-26T07:23:31.154789+0000","last_fullsized":"2023-11-26T07:23:31.154789+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:23:28.619155+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:23:28.619155+0000","last_clean_scrub_stamp":"2023-11-26T07:23:28.619155+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T07:37:04.923479+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"13.25","version":"0'0","reported_seq":11,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-11-26T07:23:31.154276+0000","last_change":"2023-11-26T07:23:29.662490+0000","last_active":"2023-11-26T07:23:31.154276+0000","last_peered":"2023-11-26T07:23:31.154276+0000","last_clean":"2023-11-26T07:23:31.154276+0000","last_became_active":"2023-11-26T07:23:29.660537+0000","last_became_peered":"2023-11-26T07:23:29.660537+0000","last_unstale":"2023-11-26T07:23:31.154276+0000","last_undegraded":"2023-11-26T07:23:31.154276+0000","last_fullsized":"2023-11-26T07:23:31.154276+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:23:28.619155+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:23:28.619155+0000","last_clean_scrub_stamp":"2023-11-26T07:23:28.619155+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T13:24:22.782822+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"13.24","version":"0'0","reported_seq":11,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-11-26T07:23:30.644373+0000","last_change":"2023-11-26T07:23:29.664641+0000","last_active":"2023-11-26T07:23:30.644373+0000","last_peered":"2023-11-26T07:23:30.644373+0000","last_clean":"2023-11-26T07:23:30.644373+0000","last_became_active":"2023-11-26T07:23:29.664308+0000","last_became_peered":"2023-11-26T07:23:29.664308+0000","last_unstale":"2023-11-26T07:23:30.644373+0000","last_undegraded":"2023-11-26T07:23:30.644373+0000","last_fullsized":"2023-11-26T07:23:30.644373+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:23:28.619155+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:23:28.619155+0000","last_clean_scrub_stamp":"2023-11-26T07:23:28.619155+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T15:42:52.859136+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.23","version":"0'0","reported_seq":11,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-11-26T07:23:31.156370+0000","last_change":"2023-11-26T07:23:29.661733+0000","last_active":"2023-11-26T07:23:31.156370+0000","last_peered":"2023-11-26T07:23:31.156370+0000","last_clean":"2023-11-26T07:23:31.156370+0000","last_became_active":"2023-11-26T07:23:29.660774+0000","last_became_peered":"2023-11-26T07:23:29.660774+0000","last_unstale":"2023-11-26T07:23:31.156370+0000","last_undegraded":"2023-11-26T07:23:31.156370+0000","last_fullsized":"2023-11-26T07:23:31.156370+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:23:28.619155+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:23:28.619155+0000","last_clean_scrub_stamp":"2023-11-26T07:23:28.619155+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T12:00:16.132422+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.22","version":"0'0","reported_seq":11,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-11-26T07:23:31.156349+0000","last_change":"2023-11-26T07:23:29.670488+0000","last_active":"2023-11-26T07:23:31.156349+0000","last_peered":"2023-11-26T07:23:31.156349+0000","last_clean":"2023-11-26T07:23:31.156349+0000","last_became_active":"2023-11-26T07:23:29.670007+0000","last_became_peered":"2023-11-26T07:23:29.670007+0000","last_unstale":"2023-11-26T07:23:31.156349+0000","last_undegraded":"2023-11-26T07:23:31.156349+0000","last_fullsized":"2023-11-26T07:23:31.156349+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:23:28.619155+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:23:28.619155+0000","last_clean_scrub_stamp":"2023-11-26T07:23:28.619155+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T14:16:44.836853+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.21","version":"0'0","reported_seq":11,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-11-26T07:23:31.154532+0000","last_change":"2023-11-26T07:23:29.658642+0000","last_active":"2023-11-26T07:23:31.154532+0000","last_peered":"2023-11-26T07:23:31.154532+0000","last_clean":"2023-11-26T07:23:31.154532+0000","last_became_active":"2023-11-26T07:23:29.658320+0000","last_became_peered":"2023-11-26T07:23:29.658320+0000","last_unstale":"2023-11-26T07:23:31.154532+0000","last_undegraded":"2023-11-26T07:23:31.154532+0000","last_fullsized":"2023-11-26T07:23:31.154532+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:23:28.619155+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:23:28.619155+0000","last_clean_scrub_stamp":"2023-11-26T07:23:28.619155+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T08:36:27.045499+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"13.20","version":"0'0","reported_seq":11,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-11-26T07:23:30.643958+0000","last_change":"2023-11-26T07:23:29.660914+0000","last_active":"2023-11-26T07:23:30.643958+0000","last_peered":"2023-11-26T07:23:30.643958+0000","last_clean":"2023-11-26T07:23:30.643958+0000","last_became_active":"2023-11-26T07:23:29.660636+0000","last_became_peered":"2023-11-26T07:23:29.660636+0000","last_unstale":"2023-11-26T07:23:30.643958+0000","last_undegraded":"2023-11-26T07:23:30.643958+0000","last_fullsized":"2023-11-26T07:23:30.643958+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:23:28.619155+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:23:28.619155+0000","last_clean_scrub_stamp":"2023-11-26T07:23:28.619155+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T17:48:40.805546+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.1e","version":"0'0","reported_seq":11,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-11-26T07:23:31.154262+0000","last_change":"2023-11-26T07:23:29.669100+0000","last_active":"2023-11-26T07:23:31.154262+0000","last_peered":"2023-11-26T07:23:31.154262+0000","last_clean":"2023-11-26T07:23:31.154262+0000","last_became_active":"2023-11-26T07:23:29.668786+0000","last_became_peered":"2023-11-26T07:23:29.668786+0000","last_unstale":"2023-11-26T07:23:31.154262+0000","last_undegraded":"2023-11-26T07:23:31.154262+0000","last_fullsized":"2023-11-26T07:23:31.154262+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:23:28.619155+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:23:28.619155+0000","last_clean_scrub_stamp":"2023-11-26T07:23:28.619155+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T08:26:22.625844+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"12.1f","version":"83'4","reported_seq":16,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-11-26T07:23:30.644694+0000","last_change":"2023-11-26T07:23:28.646168+0000","last_active":"2023-11-26T07:23:30.644694+0000","last_peered":"2023-11-26T07:23:30.644694+0000","last_clean":"2023-11-26T07:23:30.644694+0000","last_became_active":"2023-11-26T07:23:28.645841+0000","last_became_peered":"2023-11-26T07:23:28.645841+0000","last_unstale":"2023-11-26T07:23:30.644694+0000","last_undegraded":"2023-11-26T07:23:30.644694+0000","last_fullsized":"2023-11-26T07:23:30.644694+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:23:27.611790+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:23:27.611790+0000","last_clean_scrub_stamp":"2023-11-26T07:23:27.611790+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T14:42:49.850606+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.1f","version":"0'0","reported_seq":11,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-11-26T07:23:30.644581+0000","last_change":"2023-11-26T07:23:29.661788+0000","last_active":"2023-11-26T07:23:30.644581+0000","last_peered":"2023-11-26T07:23:30.644581+0000","last_clean":"2023-11-26T07:23:30.644581+0000","last_became_active":"2023-11-26T07:23:29.661123+0000","last_became_peered":"2023-11-26T07:23:29.661123+0000","last_unstale":"2023-11-26T07:23:30.644581+0000","last_undegraded":"2023-11-26T07:23:30.644581+0000","last_fullsized":"2023-11-26T07:23:30.644581+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:23:28.619155+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:23:28.619155+0000","last_clean_scrub_stamp":"2023-11-26T07:23:28.619155+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T10:41:40.854294+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"12.1e","version":"83'1","reported_seq":13,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-11-26T07:23:31.154415+0000","last_change":"2023-11-26T07:23:28.644249+0000","last_active":"2023-11-26T07:23:31.154415+0000","last_peered":"2023-11-26T07:23:31.154415+0000","last_clean":"2023-11-26T07:23:31.154415+0000","last_became_active":"2023-11-26T07:23:28.643393+0000","last_became_peered":"2023-11-26T07:23:28.643393+0000","last_unstale":"2023-11-26T07:23:31.154415+0000","last_undegraded":"2023-11-26T07:23:31.154415+0000","last_fullsized":"2023-11-26T07:23:31.154415+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:23:27.611790+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:23:27.611790+0000","last_clean_scrub_stamp":"2023-11-26T07:23:27.611790+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T12:43:28.486650+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"13.1c","version":"0'0","reported_seq":11,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-11-26T07:23:30.644530+0000","last_change":"2023-11-26T07:23:29.661340+0000","last_active":"2023-11-26T07:23:30.644530+0000","last_peered":"2023-11-26T07:23:30.644530+0000","last_clean":"2023-11-26T07:23:30.644530+0000","last_became_active":"2023-11-26T07:23:29.660953+0000","last_became_peered":"2023-11-26T07:23:29.660953+0000","last_unstale":"2023-11-26T07:23:30.644530+0000","last_undegraded":"2023-11-26T07:23:30.644530+0000","last_fullsized":"2023-11-26T07:23:30.644530+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:23:28.619155+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:23:28.619155+0000","last_clean_scrub_stamp":"2023-11-26T07:23:28.619155+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T16:34:57.083451+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"12.1d","version":"83'2","reported_seq":14,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-11-26T07:23:31.157239+0000","last_change":"2023-11-26T07:23:28.646975+0000","last_active":"2023-11-26T07:23:31.157239+0000","last_peered":"2023-11-26T07:23:31.157239+0000","last_clean":"2023-11-26T07:23:31.157239+0000","last_became_active":"2023-11-26T07:23:28.646704+0000","last_became_peered":"2023-11-26T07:23:28.646704+0000","last_unstale":"2023-11-26T07:23:31.157239+0000","last_undegraded":"2023-11-26T07:23:31.157239+0000","last_fullsized":"2023-11-26T07:23:31.157239+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:23:27.611790+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:23:27.611790+0000","last_clean_scrub_stamp":"2023-11-26T07:23:27.611790+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T16:34:56.076085+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.1d","version":"0'0","reported_seq":11,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-11-26T07:23:31.154595+0000","last_change":"2023-11-26T07:23:29.663595+0000","last_active":"2023-11-26T07:23:31.154595+0000","last_peered":"2023-11-26T07:23:31.154595+0000","last_clean":"2023-11-26T07:23:31.154595+0000","last_became_active":"2023-11-26T07:23:29.663320+0000","last_became_peered":"2023-11-26T07:23:29.663320+0000","last_unstale":"2023-11-26T07:23:31.154595+0000","last_undegraded":"2023-11-26T07:23:31.154595+0000","last_fullsized":"2023-11-26T07:23:31.154595+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:23:28.619155+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:23:28.619155+0000","last_clean_scrub_stamp":"2023-11-26T07:23:28.619155+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T17:23:44.263813+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.1c","version":"83'2","reported_seq":14,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-11-26T07:23:31.156901+0000","last_change":"2023-11-26T07:23:28.646394+0000","last_active":"2023-11-26T07:23:31.156901+0000","last_peered":"2023-11-26T07:23:31.156901+0000","last_clean":"2023-11-26T07:23:31.156901+0000","last_became_active":"2023-11-26T07:23:28.646082+0000","last_became_peered":"2023-11-26T07:23:28.646082+0000","last_unstale":"2023-11-26T07:23:31.156901+0000","last_undegraded":"2023-11-26T07:23:31.156901+0000","last_fullsized":"2023-11-26T07:23:31.156901+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:23:27.611790+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:23:27.611790+0000","last_clean_scrub_stamp":"2023-11-26T07:23:27.611790+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T10:41:39.846929+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.1a","version":"0'0","reported_seq":11,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-11-26T07:23:31.156433+0000","last_change":"2023-11-26T07:23:29.670275+0000","last_active":"2023-11-26T07:23:31.156433+0000","last_peered":"2023-11-26T07:23:31.156433+0000","last_clean":"2023-11-26T07:23:31.156433+0000","last_became_active":"2023-11-26T07:23:29.669689+0000","last_became_peered":"2023-11-26T07:23:29.669689+0000","last_unstale":"2023-11-26T07:23:31.156433+0000","last_undegraded":"2023-11-26T07:23:31.156433+0000","last_fullsized":"2023-11-26T07:23:31.156433+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:23:28.619155+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:23:28.619155+0000","last_clean_scrub_stamp":"2023-11-26T07:23:28.619155+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T17:25:21.959119+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"12.1b","version":"83'1","reported_seq":13,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-11-26T07:23:31.154302+0000","last_change":"2023-11-26T07:23:28.652973+0000","last_active":"2023-11-26T07:23:31.154302+0000","last_peered":"2023-11-26T07:23:31.154302+0000","last_clean":"2023-11-26T07:23:31.154302+0000","last_became_active":"2023-11-26T07:23:28.652601+0000","last_became_peered":"2023-11-26T07:23:28.652601+0000","last_unstale":"2023-11-26T07:23:31.154302+0000","last_undegraded":"2023-11-26T07:23:31.154302+0000","last_fullsized":"2023-11-26T07:23:31.154302+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:23:27.611790+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:23:27.611790+0000","last_clean_scrub_stamp":"2023-11-26T07:23:27.611790+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T10:07:48.914914+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"13.1b","version":"0'0","reported_seq":11,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-11-26T07:23:31.154192+0000","last_change":"2023-11-26T07:23:29.668155+0000","last_active":"2023-11-26T07:23:31.154192+0000","last_peered":"2023-11-26T07:23:31.154192+0000","last_clean":"2023-11-26T07:23:31.154192+0000","last_became_active":"2023-11-26T07:23:29.667819+0000","last_became_peered":"2023-11-26T07:23:29.667819+0000","last_unstale":"2023-11-26T07:23:31.154192+0000","last_undegraded":"2023-11-26T07:23:31.154192+0000","last_fullsized":"2023-11-26T07:23:31.154192+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:23:28.619155+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:23:28.619155+0000","last_clean_scrub_stamp":"2023-11-26T07:23:28.619155+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T10:15:59.819723+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"12.1a","version":"83'1","reported_seq":13,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-11-26T07:23:31.154016+0000","last_change":"2023-11-26T07:23:28.644425+0000","last_active":"2023-11-26T07:23:31.154016+0000","last_peered":"2023-11-26T07:23:31.154016+0000","last_clean":"2023-11-26T07:23:31.154016+0000","last_became_active":"2023-11-26T07:23:28.643691+0000","last_became_peered":"2023-11-26T07:23:28.643691+0000","last_unstale":"2023-11-26T07:23:31.154016+0000","last_undegraded":"2023-11-26T07:23:31.154016+0000","last_fullsized":"2023-11-26T07:23:31.154016+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:23:27.611790+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:23:27.611790+0000","last_clean_scrub_stamp":"2023-11-26T07:23:27.611790+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T09:53:47.961982+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"13.18","version":"0'0","reported_seq":11,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-11-26T07:23:31.153623+0000","last_change":"2023-11-26T07:23:29.649769+0000","last_active":"2023-11-26T07:23:31.153623+0000","last_peered":"2023-11-26T07:23:31.153623+0000","last_clean":"2023-11-26T07:23:31.153623+0000","last_became_active":"2023-11-26T07:23:29.649522+0000","last_became_peered":"2023-11-26T07:23:29.649522+0000","last_unstale":"2023-11-26T07:23:31.153623+0000","last_undegraded":"2023-11-26T07:23:31.153623+0000","last_fullsized":"2023-11-26T07:23:31.153623+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:23:28.619155+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:23:28.619155+0000","last_clean_scrub_stamp":"2023-11-26T07:23:28.619155+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T19:16:31.468189+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.19","version":"0'0","reported_seq":12,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-11-26T07:23:31.156313+0000","last_change":"2023-11-26T07:23:28.645338+0000","last_active":"2023-11-26T07:23:31.156313+0000","last_peered":"2023-11-26T07:23:31.156313+0000","last_clean":"2023-11-26T07:23:31.156313+0000","last_became_active":"2023-11-26T07:23:28.644860+0000","last_became_peered":"2023-11-26T07:23:28.644860+0000","last_unstale":"2023-11-26T07:23:31.156313+0000","last_undegraded":"2023-11-26T07:23:31.156313+0000","last_fullsized":"2023-11-26T07:23:31.156313+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:23:27.611790+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:23:27.611790+0000","last_clean_scrub_stamp":"2023-11-26T07:23:27.611790+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T15:32:14.815817+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.19","version":"0'0","reported_seq":11,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-11-26T07:23:31.155455+0000","last_change":"2023-11-26T07:23:29.646796+0000","last_active":"2023-11-26T07:23:31.155455+0000","last_peered":"2023-11-26T07:23:31.155455+0000","last_clean":"2023-11-26T07:23:31.155455+0000","last_became_active":"2023-11-26T07:23:29.646505+0000","last_became_peered":"2023-11-26T07:23:29.646505+0000","last_unstale":"2023-11-26T07:23:31.155455+0000","last_undegraded":"2023-11-26T07:23:31.155455+0000","last_fullsized":"2023-11-26T07:23:31.155455+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:23:28.619155+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:23:28.619155+0000","last_clean_scrub_stamp":"2023-11-26T07:23:28.619155+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T17:23:44.263813+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"12.18","version":"0'0","reported_seq":12,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-11-26T07:23:31.156022+0000","last_change":"2023-11-26T07:23:28.648143+0000","last_active":"2023-11-26T07:23:31.156022+0000","last_peered":"2023-11-26T07:23:31.156022+0000","last_clean":"2023-11-26T07:23:31.156022+0000","last_became_active":"2023-11-26T07:23:28.647829+0000","last_became_peered":"2023-11-26T07:23:28.647829+0000","last_unstale":"2023-11-26T07:23:31.156022+0000","last_undegraded":"2023-11-26T07:23:31.156022+0000","last_fullsized":"2023-11-26T07:23:31.156022+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:23:27.611790+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:23:27.611790+0000","last_clean_scrub_stamp":"2023-11-26T07:23:27.611790+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T16:37:41.275330+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.16","version":"0'0","reported_seq":11,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-11-26T07:23:31.155587+0000","last_change":"2023-11-26T07:23:29.660260+0000","last_active":"2023-11-26T07:23:31.155587+0000","last_peered":"2023-11-26T07:23:31.155587+0000","last_clean":"2023-11-26T07:23:31.155587+0000","last_became_active":"2023-11-26T07:23:29.659968+0000","last_became_peered":"2023-11-26T07:23:29.659968+0000","last_unstale":"2023-11-26T07:23:31.155587+0000","last_undegraded":"2023-11-26T07:23:31.155587+0000","last_fullsized":"2023-11-26T07:23:31.155587+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:23:28.619155+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:23:28.619155+0000","last_clean_scrub_stamp":"2023-11-26T07:23:28.619155+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T09:04:09.764282+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"12.17","version":"0'0","reported_seq":12,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-11-26T07:23:30.644759+0000","last_change":"2023-11-26T07:23:28.644791+0000","last_active":"2023-11-26T07:23:30.644759+0000","last_peered":"2023-11-26T07:23:30.644759+0000","last_clean":"2023-11-26T07:23:30.644759+0000","last_became_active":"2023-11-26T07:23:28.644512+0000","last_became_peered":"2023-11-26T07:23:28.644512+0000","last_unstale":"2023-11-26T07:23:30.644759+0000","last_undegraded":"2023-11-26T07:23:30.644759+0000","last_fullsized":"2023-11-26T07:23:30.644759+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:23:27.611790+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:23:27.611790+0000","last_clean_scrub_stamp":"2023-11-26T07:23:27.611790+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T13:24:21.775457+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.17","version":"0'0","reported_seq":11,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-11-26T07:23:30.688939+0000","last_change":"2023-11-26T07:23:29.665721+0000","last_active":"2023-11-26T07:23:30.688939+0000","last_peered":"2023-11-26T07:23:30.688939+0000","last_clean":"2023-11-26T07:23:30.688939+0000","last_became_active":"2023-11-26T07:23:29.665429+0000","last_became_peered":"2023-11-26T07:23:29.665429+0000","last_unstale":"2023-11-26T07:23:30.688939+0000","last_undegraded":"2023-11-26T07:23:30.688939+0000","last_fullsized":"2023-11-26T07:23:30.688939+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:23:28.619155+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:23:28.619155+0000","last_clean_scrub_stamp":"2023-11-26T07:23:28.619155+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T08:26:22.625844+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"12.16","version":"0'0","reported_seq":12,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-11-26T07:23:30.644562+0000","last_change":"2023-11-26T07:23:28.638932+0000","last_active":"2023-11-26T07:23:30.644562+0000","last_peered":"2023-11-26T07:23:30.644562+0000","last_clean":"2023-11-26T07:23:30.644562+0000","last_became_active":"2023-11-26T07:23:28.638443+0000","last_became_peered":"2023-11-26T07:23:28.638443+0000","last_unstale":"2023-11-26T07:23:30.644562+0000","last_undegraded":"2023-11-26T07:23:30.644562+0000","last_fullsized":"2023-11-26T07:23:30.644562+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:23:27.611790+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:23:27.611790+0000","last_clean_scrub_stamp":"2023-11-26T07:23:27.611790+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T09:05:11.247887+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.14","version":"0'0","reported_seq":11,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-11-26T07:23:31.156838+0000","last_change":"2023-11-26T07:23:29.654472+0000","last_active":"2023-11-26T07:23:31.156838+0000","last_peered":"2023-11-26T07:23:31.156838+0000","last_clean":"2023-11-26T07:23:31.156838+0000","last_became_active":"2023-11-26T07:23:29.654202+0000","last_became_peered":"2023-11-26T07:23:29.654202+0000","last_unstale":"2023-11-26T07:23:31.156838+0000","last_undegraded":"2023-11-26T07:23:31.156838+0000","last_fullsized":"2023-11-26T07:23:31.156838+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:23:28.619155+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:23:28.619155+0000","last_clean_scrub_stamp":"2023-11-26T07:23:28.619155+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T19:16:31.468189+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"12.15","version":"83'3","reported_seq":15,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-11-26T07:23:30.644309+0000","last_change":"2023-11-26T07:23:28.653374+0000","last_active":"2023-11-26T07:23:30.644309+0000","last_peered":"2023-11-26T07:23:30.644309+0000","last_clean":"2023-11-26T07:23:30.644309+0000","last_became_active":"2023-11-26T07:23:28.652820+0000","last_became_peered":"2023-11-26T07:23:28.652820+0000","last_unstale":"2023-11-26T07:23:30.644309+0000","last_undegraded":"2023-11-26T07:23:30.644309+0000","last_fullsized":"2023-11-26T07:23:30.644309+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:23:27.611790+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:23:27.611790+0000","last_clean_scrub_stamp":"2023-11-26T07:23:27.611790+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T17:54:57.000661+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.15","version":"0'0","reported_seq":11,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-11-26T07:23:31.154082+0000","last_change":"2023-11-26T07:23:29.658456+0000","last_active":"2023-11-26T07:23:31.154082+0000","last_peered":"2023-11-26T07:23:31.154082+0000","last_clean":"2023-11-26T07:23:31.154082+0000","last_became_active":"2023-11-26T07:23:29.657988+0000","last_became_peered":"2023-11-26T07:23:29.657988+0000","last_unstale":"2023-11-26T07:23:31.154082+0000","last_undegraded":"2023-11-26T07:23:31.154082+0000","last_fullsized":"2023-11-26T07:23:31.154082+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:23:28.619155+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:23:28.619155+0000","last_clean_scrub_stamp":"2023-11-26T07:23:28.619155+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T09:25:04.676044+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"12.14","version":"83'6","reported_seq":18,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-11-26T07:23:30.688866+0000","last_change":"2023-11-26T07:23:28.649584+0000","last_active":"2023-11-26T07:23:30.688866+0000","last_peered":"2023-11-26T07:23:30.688866+0000","last_clean":"2023-11-26T07:23:30.688866+0000","last_became_active":"2023-11-26T07:23:28.649029+0000","last_became_peered":"2023-11-26T07:23:28.649029+0000","last_unstale":"2023-11-26T07:23:30.688866+0000","last_undegraded":"2023-11-26T07:23:30.688866+0000","last_fullsized":"2023-11-26T07:23:30.688866+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:23:27.611790+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:23:27.611790+0000","last_clean_scrub_stamp":"2023-11-26T07:23:27.611790+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T14:37:41.627112+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"13.12","version":"0'0","reported_seq":11,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-11-26T07:23:31.156580+0000","last_change":"2023-11-26T07:23:29.661802+0000","last_active":"2023-11-26T07:23:31.156580+0000","last_peered":"2023-11-26T07:23:31.156580+0000","last_clean":"2023-11-26T07:23:31.156580+0000","last_became_active":"2023-11-26T07:23:29.661431+0000","last_became_peered":"2023-11-26T07:23:29.661431+0000","last_unstale":"2023-11-26T07:23:31.156580+0000","last_undegraded":"2023-11-26T07:23:31.156580+0000","last_fullsized":"2023-11-26T07:23:31.156580+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:23:28.619155+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:23:28.619155+0000","last_clean_scrub_stamp":"2023-11-26T07:23:28.619155+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T08:43:34.194258+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"12.13","version":"83'2","reported_seq":14,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-11-26T07:23:31.156472+0000","last_change":"2023-11-26T07:23:28.653044+0000","last_active":"2023-11-26T07:23:31.156472+0000","last_peered":"2023-11-26T07:23:31.156472+0000","last_clean":"2023-11-26T07:23:31.156472+0000","last_became_active":"2023-11-26T07:23:28.652432+0000","last_became_peered":"2023-11-26T07:23:28.652432+0000","last_unstale":"2023-11-26T07:23:31.156472+0000","last_undegraded":"2023-11-26T07:23:31.156472+0000","last_fullsized":"2023-11-26T07:23:31.156472+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:23:27.611790+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:23:27.611790+0000","last_clean_scrub_stamp":"2023-11-26T07:23:27.611790+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T14:28:24.475819+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.13","version":"0'0","reported_seq":11,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-11-26T07:23:31.153703+0000","last_change":"2023-11-26T07:23:29.649101+0000","last_active":"2023-11-26T07:23:31.153703+0000","last_peered":"2023-11-26T07:23:31.153703+0000","last_clean":"2023-11-26T07:23:31.153703+0000","last_became_active":"2023-11-26T07:23:29.648737+0000","last_became_peered":"2023-11-26T07:23:29.648737+0000","last_unstale":"2023-11-26T07:23:31.153703+0000","last_undegraded":"2023-11-26T07:23:31.153703+0000","last_fullsized":"2023-11-26T07:23:31.153703+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:23:28.619155+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:23:28.619155+0000","last_clean_scrub_stamp":"2023-11-26T07:23:28.619155+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T11:31:08.516775+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.12","version":"0'0","reported_seq":12,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-11-26T07:23:31.155321+0000","last_change":"2023-11-26T07:23:28.654881+0000","last_active":"2023-11-26T07:23:31.155321+0000","last_peered":"2023-11-26T07:23:31.155321+0000","last_clean":"2023-11-26T07:23:31.155321+0000","last_became_active":"2023-11-26T07:23:28.654562+0000","last_became_peered":"2023-11-26T07:23:28.654562+0000","last_unstale":"2023-11-26T07:23:31.155321+0000","last_undegraded":"2023-11-26T07:23:31.155321+0000","last_fullsized":"2023-11-26T07:23:31.155321+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:23:27.611790+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:23:27.611790+0000","last_clean_scrub_stamp":"2023-11-26T07:23:27.611790+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T19:16:30.460823+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"13.10","version":"0'0","reported_seq":11,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-11-26T07:23:31.156029+0000","last_change":"2023-11-26T07:23:29.669153+0000","last_active":"2023-11-26T07:23:31.156029+0000","last_peered":"2023-11-26T07:23:31.156029+0000","last_clean":"2023-11-26T07:23:31.156029+0000","last_became_active":"2023-11-26T07:23:29.668883+0000","last_became_peered":"2023-11-26T07:23:29.668883+0000","last_unstale":"2023-11-26T07:23:31.156029+0000","last_undegraded":"2023-11-26T07:23:31.156029+0000","last_fullsized":"2023-11-26T07:23:31.156029+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:23:28.619155+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:23:28.619155+0000","last_clean_scrub_stamp":"2023-11-26T07:23:28.619155+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T17:25:32.924640+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"12.11","version":"0'0","reported_seq":12,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-11-26T07:23:30.688611+0000","last_change":"2023-11-26T07:23:28.644925+0000","last_active":"2023-11-26T07:23:30.688611+0000","last_peered":"2023-11-26T07:23:30.688611+0000","last_clean":"2023-11-26T07:23:30.688611+0000","last_became_active":"2023-11-26T07:23:28.644653+0000","last_became_peered":"2023-11-26T07:23:28.644653+0000","last_unstale":"2023-11-26T07:23:30.688611+0000","last_undegraded":"2023-11-26T07:23:30.688611+0000","last_fullsized":"2023-11-26T07:23:30.688611+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:23:27.611790+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:23:27.611790+0000","last_clean_scrub_stamp":"2023-11-26T07:23:27.611790+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T09:53:47.961982+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"13.11","version":"0'0","reported_seq":11,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-11-26T07:23:31.155992+0000","last_change":"2023-11-26T07:23:29.670341+0000","last_active":"2023-11-26T07:23:31.155992+0000","last_peered":"2023-11-26T07:23:31.155992+0000","last_clean":"2023-11-26T07:23:31.155992+0000","last_became_active":"2023-11-26T07:23:29.670002+0000","last_became_peered":"2023-11-26T07:23:29.670002+0000","last_unstale":"2023-11-26T07:23:31.155992+0000","last_undegraded":"2023-11-26T07:23:31.155992+0000","last_fullsized":"2023-11-26T07:23:31.155992+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:23:28.619155+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:23:28.619155+0000","last_clean_scrub_stamp":"2023-11-26T07:23:28.619155+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T15:35:16.874242+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"12.10","version":"83'8","reported_seq":20,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-11-26T07:23:31.156057+0000","last_change":"2023-11-26T07:23:28.654251+0000","last_active":"2023-11-26T07:23:31.156057+0000","last_peered":"2023-11-26T07:23:31.156057+0000","last_clean":"2023-11-26T07:23:31.156057+0000","last_became_active":"2023-11-26T07:23:28.651696+0000","last_became_peered":"2023-11-26T07:23:28.651696+0000","last_unstale":"2023-11-26T07:23:31.156057+0000","last_undegraded":"2023-11-26T07:23:31.156057+0000","last_fullsized":"2023-11-26T07:23:31.156057+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:23:27.611790+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:23:27.611790+0000","last_clean_scrub_stamp":"2023-11-26T07:23:27.611790+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T15:42:51.851770+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.e","version":"0'0","reported_seq":11,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-11-26T07:23:31.154797+0000","last_change":"2023-11-26T07:23:29.654149+0000","last_active":"2023-11-26T07:23:31.154797+0000","last_peered":"2023-11-26T07:23:31.154797+0000","last_clean":"2023-11-26T07:23:31.154797+0000","last_became_active":"2023-11-26T07:23:29.652774+0000","last_became_peered":"2023-11-26T07:23:29.652774+0000","last_unstale":"2023-11-26T07:23:31.154797+0000","last_undegraded":"2023-11-26T07:23:31.154797+0000","last_fullsized":"2023-11-26T07:23:31.154797+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:23:28.619155+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:23:28.619155+0000","last_clean_scrub_stamp":"2023-11-26T07:23:28.619155+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T12:37:58.279115+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.f","version":"83'1","reported_seq":13,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-11-26T07:23:30.644808+0000","last_change":"2023-11-26T07:23:28.647997+0000","last_active":"2023-11-26T07:23:30.644808+0000","last_peered":"2023-11-26T07:23:30.644808+0000","last_clean":"2023-11-26T07:23:30.644808+0000","last_became_active":"2023-11-26T07:23:28.647641+0000","last_became_peered":"2023-11-26T07:23:28.647641+0000","last_unstale":"2023-11-26T07:23:30.644808+0000","last_undegraded":"2023-11-26T07:23:30.644808+0000","last_fullsized":"2023-11-26T07:23:30.644808+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:23:27.611790+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:23:27.611790+0000","last_clean_scrub_stamp":"2023-11-26T07:23:27.611790+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T09:45:54.208868+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.f","version":"0'0","reported_seq":11,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-11-26T07:23:30.644734+0000","last_change":"2023-11-26T07:23:29.662942+0000","last_active":"2023-11-26T07:23:30.644734+0000","last_peered":"2023-11-26T07:23:30.644734+0000","last_clean":"2023-11-26T07:23:30.644734+0000","last_became_active":"2023-11-26T07:23:29.662591+0000","last_became_peered":"2023-11-26T07:23:29.662591+0000","last_unstale":"2023-11-26T07:23:30.644734+0000","last_undegraded":"2023-11-26T07:23:30.644734+0000","last_fullsized":"2023-11-26T07:23:30.644734+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:23:28.619155+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:23:28.619155+0000","last_clean_scrub_stamp":"2023-11-26T07:23:28.619155+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T08:43:34.194258+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"12.e","version":"83'2","reported_seq":14,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-11-26T07:23:31.156940+0000","last_change":"2023-11-26T07:23:28.641634+0000","last_active":"2023-11-26T07:23:31.156940+0000","last_peered":"2023-11-26T07:23:31.156940+0000","last_clean":"2023-11-26T07:23:31.156940+0000","last_became_active":"2023-11-26T07:23:28.641290+0000","last_became_peered":"2023-11-26T07:23:28.641290+0000","last_unstale":"2023-11-26T07:23:31.156940+0000","last_undegraded":"2023-11-26T07:23:31.156940+0000","last_fullsized":"2023-11-26T07:23:31.156940+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:23:27.611790+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:23:27.611790+0000","last_clean_scrub_stamp":"2023-11-26T07:23:27.611790+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T18:11:06.634569+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.d","version":"0'0","reported_seq":11,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-11-26T07:23:31.154275+0000","last_change":"2023-11-26T07:23:29.661133+0000","last_active":"2023-11-26T07:23:31.154275+0000","last_peered":"2023-11-26T07:23:31.154275+0000","last_clean":"2023-11-26T07:23:31.154275+0000","last_became_active":"2023-11-26T07:23:29.660870+0000","last_became_peered":"2023-11-26T07:23:29.660870+0000","last_unstale":"2023-11-26T07:23:31.154275+0000","last_undegraded":"2023-11-26T07:23:31.154275+0000","last_fullsized":"2023-11-26T07:23:31.154275+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:23:28.619155+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:23:28.619155+0000","last_clean_scrub_stamp":"2023-11-26T07:23:28.619155+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T14:42:50.857972+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"12.c","version":"0'0","reported_seq":12,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-11-26T07:23:31.156938+0000","last_change":"2023-11-26T07:23:28.649200+0000","last_active":"2023-11-26T07:23:31.156938+0000","last_peered":"2023-11-26T07:23:31.156938+0000","last_clean":"2023-11-26T07:23:31.156938+0000","last_became_active":"2023-11-26T07:23:28.648928+0000","last_became_peered":"2023-11-26T07:23:28.648928+0000","last_unstale":"2023-11-26T07:23:31.156938+0000","last_undegraded":"2023-11-26T07:23:31.156938+0000","last_fullsized":"2023-11-26T07:23:31.156938+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:23:27.611790+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:23:27.611790+0000","last_clean_scrub_stamp":"2023-11-26T07:23:27.611790+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T11:28:12.293763+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.a","version":"0'0","reported_seq":11,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-11-26T07:23:30.643067+0000","last_change":"2023-11-26T07:23:29.667394+0000","last_active":"2023-11-26T07:23:30.643067+0000","last_peered":"2023-11-26T07:23:30.643067+0000","last_clean":"2023-11-26T07:23:30.643067+0000","last_became_active":"2023-11-26T07:23:29.667077+0000","last_became_peered":"2023-11-26T07:23:29.667077+0000","last_unstale":"2023-11-26T07:23:30.643067+0000","last_undegraded":"2023-11-26T07:23:30.643067+0000","last_fullsized":"2023-11-26T07:23:30.643067+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:23:28.619155+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:23:28.619155+0000","last_clean_scrub_stamp":"2023-11-26T07:23:28.619155+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T09:25:23.557193+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"12.b","version":"83'2","reported_seq":14,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-11-26T07:23:31.155580+0000","last_change":"2023-11-26T07:23:28.653466+0000","last_active":"2023-11-26T07:23:31.155580+0000","last_peered":"2023-11-26T07:23:31.155580+0000","last_clean":"2023-11-26T07:23:31.155580+0000","last_became_active":"2023-11-26T07:23:28.652010+0000","last_became_peered":"2023-11-26T07:23:28.652010+0000","last_unstale":"2023-11-26T07:23:31.155580+0000","last_undegraded":"2023-11-26T07:23:31.155580+0000","last_fullsized":"2023-11-26T07:23:31.155580+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:23:27.611790+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:23:27.611790+0000","last_clean_scrub_stamp":"2023-11-26T07:23:27.611790+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T11:28:12.293763+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"13.b","version":"0'0","reported_seq":11,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-11-26T07:23:30.688631+0000","last_change":"2023-11-26T07:23:29.657898+0000","last_active":"2023-11-26T07:23:30.688631+0000","last_peered":"2023-11-26T07:23:30.688631+0000","last_clean":"2023-11-26T07:23:30.688631+0000","last_became_active":"2023-11-26T07:23:29.657623+0000","last_became_peered":"2023-11-26T07:23:29.657623+0000","last_unstale":"2023-11-26T07:23:30.688631+0000","last_undegraded":"2023-11-26T07:23:30.688631+0000","last_fullsized":"2023-11-26T07:23:30.688631+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:23:28.619155+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:23:28.619155+0000","last_clean_scrub_stamp":"2023-11-26T07:23:28.619155+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T13:22:18.189493+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"12.a","version":"0'0","reported_seq":12,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-11-26T07:23:30.643144+0000","last_change":"2023-11-26T07:23:28.646461+0000","last_active":"2023-11-26T07:23:30.643144+0000","last_peered":"2023-11-26T07:23:30.643144+0000","last_clean":"2023-11-26T07:23:30.643144+0000","last_became_active":"2023-11-26T07:23:28.645546+0000","last_became_peered":"2023-11-26T07:23:28.645546+0000","last_unstale":"2023-11-26T07:23:30.643144+0000","last_undegraded":"2023-11-26T07:23:30.643144+0000","last_fullsized":"2023-11-26T07:23:30.643144+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:23:27.611790+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:23:27.611790+0000","last_clean_scrub_stamp":"2023-11-26T07:23:27.611790+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T19:15:49.793351+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"13.8","version":"0'0","reported_seq":11,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-11-26T07:23:30.642665+0000","last_change":"2023-11-26T07:23:29.664106+0000","last_active":"2023-11-26T07:23:30.642665+0000","last_peered":"2023-11-26T07:23:30.642665+0000","last_clean":"2023-11-26T07:23:30.642665+0000","last_became_active":"2023-11-26T07:23:29.663802+0000","last_became_peered":"2023-11-26T07:23:29.663802+0000","last_unstale":"2023-11-26T07:23:30.642665+0000","last_undegraded":"2023-11-26T07:23:30.642665+0000","last_fullsized":"2023-11-26T07:23:30.642665+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:23:28.619155+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:23:28.619155+0000","last_clean_scrub_stamp":"2023-11-26T07:23:28.619155+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T13:58:11.826957+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"12.9","version":"0'0","reported_seq":12,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-11-26T07:23:30.642648+0000","last_change":"2023-11-26T07:23:28.643600+0000","last_active":"2023-11-26T07:23:30.642648+0000","last_peered":"2023-11-26T07:23:30.642648+0000","last_clean":"2023-11-26T07:23:30.642648+0000","last_became_active":"2023-11-26T07:23:28.643304+0000","last_became_peered":"2023-11-26T07:23:28.643304+0000","last_unstale":"2023-11-26T07:23:30.642648+0000","last_undegraded":"2023-11-26T07:23:30.642648+0000","last_fullsized":"2023-11-26T07:23:30.642648+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:23:27.611790+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:23:27.611790+0000","last_clean_scrub_stamp":"2023-11-26T07:23:27.611790+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T17:21:18.210966+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"13.9","version":"0'0","reported_seq":11,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-11-26T07:23:30.688648+0000","last_change":"2023-11-26T07:23:29.656753+0000","last_active":"2023-11-26T07:23:30.688648+0000","last_peered":"2023-11-26T07:23:30.688648+0000","last_clean":"2023-11-26T07:23:30.688648+0000","last_became_active":"2023-11-26T07:23:29.656231+0000","last_became_peered":"2023-11-26T07:23:29.656231+0000","last_unstale":"2023-11-26T07:23:30.688648+0000","last_undegraded":"2023-11-26T07:23:30.688648+0000","last_fullsized":"2023-11-26T07:23:30.688648+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:23:28.619155+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:23:28.619155+0000","last_clean_scrub_stamp":"2023-11-26T07:23:28.619155+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T10:15:59.819723+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"12.8","version":"0'0","reported_seq":12,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-11-26T07:23:31.155222+0000","last_change":"2023-11-26T07:23:28.652672+0000","last_active":"2023-11-26T07:23:31.155222+0000","last_peered":"2023-11-26T07:23:31.155222+0000","last_clean":"2023-11-26T07:23:31.155222+0000","last_became_active":"2023-11-26T07:23:28.652413+0000","last_became_peered":"2023-11-26T07:23:28.652413+0000","last_unstale":"2023-11-26T07:23:31.155222+0000","last_undegraded":"2023-11-26T07:23:31.155222+0000","last_fullsized":"2023-11-26T07:23:31.155222+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:23:27.611790+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:23:27.611790+0000","last_clean_scrub_stamp":"2023-11-26T07:23:27.611790+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T10:41:39.846929+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"13.6","version":"0'0","reported_seq":11,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-11-26T07:23:31.154398+0000","last_change":"2023-11-26T07:23:29.672449+0000","last_active":"2023-11-26T07:23:31.154398+0000","last_peered":"2023-11-26T07:23:31.154398+0000","last_clean":"2023-11-26T07:23:31.154398+0000","last_became_active":"2023-11-26T07:23:29.671495+0000","last_became_peered":"2023-11-26T07:23:29.671495+0000","last_unstale":"2023-11-26T07:23:31.154398+0000","last_undegraded":"2023-11-26T07:23:31.154398+0000","last_fullsized":"2023-11-26T07:23:31.154398+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:23:28.619155+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:23:28.619155+0000","last_clean_scrub_stamp":"2023-11-26T07:23:28.619155+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T09:45:55.216234+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"12.7","version":"83'2","reported_seq":14,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-11-26T07:23:31.156653+0000","last_change":"2023-11-26T07:23:28.653628+0000","last_active":"2023-11-26T07:23:31.156653+0000","last_peered":"2023-11-26T07:23:31.156653+0000","last_clean":"2023-11-26T07:23:31.156653+0000","last_became_active":"2023-11-26T07:23:28.653346+0000","last_became_peered":"2023-11-26T07:23:28.653346+0000","last_unstale":"2023-11-26T07:23:31.156653+0000","last_undegraded":"2023-11-26T07:23:31.156653+0000","last_fullsized":"2023-11-26T07:23:31.156653+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:23:27.611790+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:23:27.611790+0000","last_clean_scrub_stamp":"2023-11-26T07:23:27.611790+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T17:07:02.629426+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.7","version":"0'0","reported_seq":11,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-11-26T07:23:31.154912+0000","last_change":"2023-11-26T07:23:29.652585+0000","last_active":"2023-11-26T07:23:31.154912+0000","last_peered":"2023-11-26T07:23:31.154912+0000","last_clean":"2023-11-26T07:23:31.154912+0000","last_became_active":"2023-11-26T07:23:29.651796+0000","last_became_peered":"2023-11-26T07:23:29.651796+0000","last_unstale":"2023-11-26T07:23:31.154912+0000","last_undegraded":"2023-11-26T07:23:31.154912+0000","last_fullsized":"2023-11-26T07:23:31.154912+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:23:28.619155+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:23:28.619155+0000","last_clean_scrub_stamp":"2023-11-26T07:23:28.619155+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T08:50:54.551576+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.6","version":"0'0","reported_seq":12,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-11-26T07:23:31.156802+0000","last_change":"2023-11-26T07:23:28.651746+0000","last_active":"2023-11-26T07:23:31.156802+0000","last_peered":"2023-11-26T07:23:31.156802+0000","last_clean":"2023-11-26T07:23:31.156802+0000","last_became_active":"2023-11-26T07:23:28.651476+0000","last_became_peered":"2023-11-26T07:23:28.651476+0000","last_unstale":"2023-11-26T07:23:31.156802+0000","last_undegraded":"2023-11-26T07:23:31.156802+0000","last_fullsized":"2023-11-26T07:23:31.156802+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:23:27.611790+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:23:27.611790+0000","last_clean_scrub_stamp":"2023-11-26T07:23:27.611790+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T14:29:53.425427+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.4","version":"0'0","reported_seq":11,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-11-26T07:23:30.644682+0000","last_change":"2023-11-26T07:23:29.665782+0000","last_active":"2023-11-26T07:23:30.644682+0000","last_peered":"2023-11-26T07:23:30.644682+0000","last_clean":"2023-11-26T07:23:30.644682+0000","last_became_active":"2023-11-26T07:23:29.665491+0000","last_became_peered":"2023-11-26T07:23:29.665491+0000","last_unstale":"2023-11-26T07:23:30.644682+0000","last_undegraded":"2023-11-26T07:23:30.644682+0000","last_fullsized":"2023-11-26T07:23:30.644682+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:23:28.619155+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:23:28.619155+0000","last_clean_scrub_stamp":"2023-11-26T07:23:28.619155+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T18:11:07.641934+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"12.5","version":"0'0","reported_seq":12,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-11-26T07:23:30.644246+0000","last_change":"2023-11-26T07:23:28.646051+0000","last_active":"2023-11-26T07:23:30.644246+0000","last_peered":"2023-11-26T07:23:30.644246+0000","last_clean":"2023-11-26T07:23:30.644246+0000","last_became_active":"2023-11-26T07:23:28.645736+0000","last_became_peered":"2023-11-26T07:23:28.645736+0000","last_unstale":"2023-11-26T07:23:30.644246+0000","last_undegraded":"2023-11-26T07:23:30.644246+0000","last_fullsized":"2023-11-26T07:23:30.644246+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:23:27.611790+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:23:27.611790+0000","last_clean_scrub_stamp":"2023-11-26T07:23:27.611790+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T17:13:40.716907+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.5","version":"0'0","reported_seq":11,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-11-26T07:23:31.154442+0000","last_change":"2023-11-26T07:23:29.658254+0000","last_active":"2023-11-26T07:23:31.154442+0000","last_peered":"2023-11-26T07:23:31.154442+0000","last_clean":"2023-11-26T07:23:31.154442+0000","last_became_active":"2023-11-26T07:23:29.657633+0000","last_became_peered":"2023-11-26T07:23:29.657633+0000","last_unstale":"2023-11-26T07:23:31.154442+0000","last_undegraded":"2023-11-26T07:23:31.154442+0000","last_fullsized":"2023-11-26T07:23:31.154442+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:23:28.619155+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:23:28.619155+0000","last_clean_scrub_stamp":"2023-11-26T07:23:28.619155+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T17:48:40.805546+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"12.4","version":"0'0","reported_seq":12,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-11-26T07:23:30.644701+0000","last_change":"2023-11-26T07:23:28.646843+0000","last_active":"2023-11-26T07:23:30.644701+0000","last_peered":"2023-11-26T07:23:30.644701+0000","last_clean":"2023-11-26T07:23:30.644701+0000","last_became_active":"2023-11-26T07:23:28.646559+0000","last_became_peered":"2023-11-26T07:23:28.646559+0000","last_unstale":"2023-11-26T07:23:30.644701+0000","last_undegraded":"2023-11-26T07:23:30.644701+0000","last_fullsized":"2023-11-26T07:23:30.644701+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:23:27.611790+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:23:27.611790+0000","last_clean_scrub_stamp":"2023-11-26T07:23:27.611790+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T08:26:21.618478+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.2","version":"0'0","reported_seq":11,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-11-26T07:23:30.688621+0000","last_change":"2023-11-26T07:23:29.661239+0000","last_active":"2023-11-26T07:23:30.688621+0000","last_peered":"2023-11-26T07:23:30.688621+0000","last_clean":"2023-11-26T07:23:30.688621+0000","last_became_active":"2023-11-26T07:23:29.660971+0000","last_became_peered":"2023-11-26T07:23:29.660971+0000","last_unstale":"2023-11-26T07:23:30.688621+0000","last_undegraded":"2023-11-26T07:23:30.688621+0000","last_fullsized":"2023-11-26T07:23:30.688621+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:23:28.619155+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:23:28.619155+0000","last_clean_scrub_stamp":"2023-11-26T07:23:28.619155+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T09:05:12.255253+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"12.3","version":"0'0","reported_seq":12,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-11-26T07:23:31.157235+0000","last_change":"2023-11-26T07:23:28.642964+0000","last_active":"2023-11-26T07:23:31.157235+0000","last_peered":"2023-11-26T07:23:31.157235+0000","last_clean":"2023-11-26T07:23:31.157235+0000","last_became_active":"2023-11-26T07:23:28.642675+0000","last_became_peered":"2023-11-26T07:23:28.642675+0000","last_unstale":"2023-11-26T07:23:31.157235+0000","last_undegraded":"2023-11-26T07:23:31.157235+0000","last_fullsized":"2023-11-26T07:23:31.157235+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:23:27.611790+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:23:27.611790+0000","last_clean_scrub_stamp":"2023-11-26T07:23:27.611790+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T13:34:19.121282+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.3","version":"0'0","reported_seq":11,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-11-26T07:23:31.157192+0000","last_change":"2023-11-26T07:23:29.663381+0000","last_active":"2023-11-26T07:23:31.157192+0000","last_peered":"2023-11-26T07:23:31.157192+0000","last_clean":"2023-11-26T07:23:31.157192+0000","last_became_active":"2023-11-26T07:23:29.663006+0000","last_became_peered":"2023-11-26T07:23:29.663006+0000","last_unstale":"2023-11-26T07:23:31.157192+0000","last_undegraded":"2023-11-26T07:23:31.157192+0000","last_fullsized":"2023-11-26T07:23:31.157192+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:23:28.619155+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:23:28.619155+0000","last_clean_scrub_stamp":"2023-11-26T07:23:28.619155+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T10:32:40.375411+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"12.2","version":"0'0","reported_seq":12,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-11-26T07:23:31.154176+0000","last_change":"2023-11-26T07:23:28.643979+0000","last_active":"2023-11-26T07:23:31.154176+0000","last_peered":"2023-11-26T07:23:31.154176+0000","last_clean":"2023-11-26T07:23:31.154176+0000","last_became_active":"2023-11-26T07:23:28.642738+0000","last_became_peered":"2023-11-26T07:23:28.642738+0000","last_unstale":"2023-11-26T07:23:31.154176+0000","last_undegraded":"2023-11-26T07:23:31.154176+0000","last_fullsized":"2023-11-26T07:23:31.154176+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:23:27.611790+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:23:27.611790+0000","last_clean_scrub_stamp":"2023-11-26T07:23:27.611790+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T14:37:41.627112+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"13.0","version":"0'0","reported_seq":11,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-11-26T07:23:31.153856+0000","last_change":"2023-11-26T07:23:29.658136+0000","last_active":"2023-11-26T07:23:31.153856+0000","last_peered":"2023-11-26T07:23:31.153856+0000","last_clean":"2023-11-26T07:23:31.153856+0000","last_became_active":"2023-11-26T07:23:29.657860+0000","last_became_peered":"2023-11-26T07:23:29.657860+0000","last_unstale":"2023-11-26T07:23:31.153856+0000","last_undegraded":"2023-11-26T07:23:31.153856+0000","last_fullsized":"2023-11-26T07:23:31.153856+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:23:28.619155+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:23:28.619155+0000","last_clean_scrub_stamp":"2023-11-26T07:23:28.619155+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T10:32:40.375411+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.1","version":"0'0","reported_seq":12,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-11-26T07:23:30.644179+0000","last_change":"2023-11-26T07:23:28.644096+0000","last_active":"2023-11-26T07:23:30.644179+0000","last_peered":"2023-11-26T07:23:30.644179+0000","last_clean":"2023-11-26T07:23:30.644179+0000","last_became_active":"2023-11-26T07:23:28.643353+0000","last_became_peered":"2023-11-26T07:23:28.643353+0000","last_unstale":"2023-11-26T07:23:30.644179+0000","last_undegraded":"2023-11-26T07:23:30.644179+0000","last_fullsized":"2023-11-26T07:23:30.644179+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:23:27.611790+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:23:27.611790+0000","last_clean_scrub_stamp":"2023-11-26T07:23:27.611790+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T09:53:47.961982+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.1","version":"0'0","reported_seq":11,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-11-26T07:23:30.644066+0000","last_change":"2023-11-26T07:23:29.667904+0000","last_active":"2023-11-26T07:23:30.644066+0000","last_peered":"2023-11-26T07:23:30.644066+0000","last_clean":"2023-11-26T07:23:30.644066+0000","last_became_active":"2023-11-26T07:23:29.667512+0000","last_became_peered":"2023-11-26T07:23:29.667512+0000","last_unstale":"2023-11-26T07:23:30.644066+0000","last_undegraded":"2023-11-26T07:23:30.644066+0000","last_fullsized":"2023-11-26T07:23:30.644066+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:23:28.619155+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:23:28.619155+0000","last_clean_scrub_stamp":"2023-11-26T07:23:28.619155+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T08:23:08.828343+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"12.0","version":"0'0","reported_seq":12,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-11-26T07:23:31.153778+0000","last_change":"2023-11-26T07:23:28.640179+0000","last_active":"2023-11-26T07:23:31.153778+0000","last_peered":"2023-11-26T07:23:31.153778+0000","last_clean":"2023-11-26T07:23:31.153778+0000","last_became_active":"2023-11-26T07:23:28.639867+0000","last_became_peered":"2023-11-26T07:23:28.639867+0000","last_unstale":"2023-11-26T07:23:31.153778+0000","last_undegraded":"2023-11-26T07:23:31.153778+0000","last_fullsized":"2023-11-26T07:23:31.153778+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:23:27.611790+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:23:27.611790+0000","last_clean_scrub_stamp":"2023-11-26T07:23:27.611790+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T13:16:03.358808+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"13.c","version":"0'0","reported_seq":11,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-11-26T07:23:31.156681+0000","last_change":"2023-11-26T07:23:29.671830+0000","last_active":"2023-11-26T07:23:31.156681+0000","last_peered":"2023-11-26T07:23:31.156681+0000","last_clean":"2023-11-26T07:23:31.156681+0000","last_became_active":"2023-11-26T07:23:29.671581+0000","last_became_peered":"2023-11-26T07:23:29.671581+0000","last_unstale":"2023-11-26T07:23:31.156681+0000","last_undegraded":"2023-11-26T07:23:31.156681+0000","last_fullsized":"2023-11-26T07:23:31.156681+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:23:28.619155+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:23:28.619155+0000","last_clean_scrub_stamp":"2023-11-26T07:23:28.619155+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T12:43:29.494016+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"12.d","version":"83'1","reported_seq":13,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-11-26T07:23:31.154710+0000","last_change":"2023-11-26T07:23:28.652819+0000","last_active":"2023-11-26T07:23:31.154710+0000","last_peered":"2023-11-26T07:23:31.154710+0000","last_clean":"2023-11-26T07:23:31.154710+0000","last_became_active":"2023-11-26T07:23:28.652526+0000","last_became_peered":"2023-11-26T07:23:28.652526+0000","last_unstale":"2023-11-26T07:23:31.154710+0000","last_undegraded":"2023-11-26T07:23:31.154710+0000","last_fullsized":"2023-11-26T07:23:31.154710+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:23:27.611790+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:23:27.611790+0000","last_clean_scrub_stamp":"2023-11-26T07:23:27.611790+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T11:28:12.293763+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.0","version":"72'128","reported_seq":255,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-11-26T07:23:29.641547+0000","last_change":"2023-11-26T07:17:43.827802+0000","last_active":"2023-11-26T07:23:29.641547+0000","last_peered":"2023-11-26T07:23:29.641547+0000","last_clean":"2023-11-26T07:23:29.641547+0000","last_became_active":"2023-11-26T07:17:43.827308+0000","last_became_peered":"2023-11-26T07:17:43.827308+0000","last_unstale":"2023-11-26T07:23:29.641547+0000","last_undegraded":"2023-11-26T07:23:29.641547+0000","last_fullsized":"2023-11-26T07:23:29.641547+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:17:42.160945+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:17:42.160945+0000","last_clean_scrub_stamp":"2023-11-26T07:17:42.160945+0000","objects_scrubbed":0,"log_size":128,"log_dups_size":0,"ondisk_log_size":128,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T08:17:54.167366+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":212,"num_read_kb":306,"num_write":179,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":13,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":12,"num_pg":32,"stat_sum":{"num_bytes":2410,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":38,"ondisk_log_size":38,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":212,"num_read_kb":306,"num_write":179,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":128,"ondisk_log_size":128,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574920,"num_pgs":19,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32772,"kb_used_data":1592,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339068,"statfs":{"total":96636764160,"available":96603205632,"internally_reserved":0,"allocated":1630208,"data_stored":1034627,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":31914442},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Nov 26 07:22:55 2023","interfaces":[{"interface":"back","average":{"1min":0.66900000000000004,"5min":0.70999999999999996,"15min":0.71299999999999997},"min":{"1min":0.437,"5min":0.42999999999999999,"15min":0.42999999999999999},"max":{"1min":0.873,"5min":0.98199999999999998,"15min":0.98199999999999998},"last":0.77300000000000002},{"interface":"front","average":{"1min":0.70699999999999996,"5min":0.72799999999999998,"15min":0.71499999999999997},"min":{"1min":0.52900000000000003,"5min":0.52900000000000003,"15min":0.52900000000000003},"max":{"1min":0.92800000000000005,"5min":1.0980000000000001,"15min":1.0980000000000001},"last":0.82899999999999996}]},{"osd":1,"last update":"Sun Nov 26 07:22:55 2023","interfaces":[{"interface":"back","average":{"1min":0.67800000000000005,"5min":0.68700000000000006,"15min":0.68300000000000005},"min":{"1min":0.435,"5min":0.435,"15min":0.435},"max":{"1min":0.90800000000000003,"5min":1.0669999999999999,"15min":1.0669999999999999},"last":0.66100000000000003},{"interface":"front","average":{"1min":0.67300000000000004,"5min":0.71099999999999997,"15min":0.79600000000000004},"min":{"1min":0.49199999999999999,"5min":0.47599999999999998,"15min":0.47599999999999998},"max":{"1min":0.88700000000000001,"5min":4.5309999999999997,"15min":4.5309999999999997},"last":0.68999999999999995}]},{"osd":2,"last update":"Sun Nov 26 07:22:55 2023","interfaces":[{"interface":"back","average":{"1min":0.68899999999999995,"5min":0.69499999999999995,"15min":0.68200000000000005},"min":{"1min":0.503,"5min":0.39700000000000002,"15min":0.39700000000000002},"max":{"1min":0.88,"5min":1.0800000000000001,"15min":1.0800000000000001},"last":0.91600000000000004},{"interface":"front","average":{"1min":0.69899999999999995,"5min":0.70999999999999996,"15min":0.71199999999999997},"min":{"1min":0.40999999999999998,"5min":0.40999999999999998,"15min":0.40999999999999998},"max":{"1min":0.97699999999999998,"5min":1.1339999999999999,"15min":1.1339999999999999},"last":0.72099999999999997}]},{"osd":3,"last update":"Sun Nov 26 07:22:55 2023","interfaces":[{"interface":"back","average":{"1min":0.70799999999999996,"5min":0.72099999999999997,"15min":0.70499999999999996},"min":{"1min":0.52700000000000002,"5min":0.47799999999999998,"15min":0.47799999999999998},"max":{"1min":0.98399999999999999,"5min":1.1240000000000001,"15min":1.1240000000000001},"last":0.73399999999999999},{"interface":"front","average":{"1min":0.71299999999999997,"5min":0.74399999999999999,"15min":0.73599999999999999},"min":{"1min":0.51700000000000002,"5min":0.433,"15min":0.433},"max":{"1min":1.0029999999999999,"5min":1.4490000000000001,"15min":1.4490000000000001},"last":0.88800000000000001}]},{"osd":4,"last update":"Sun Nov 26 07:22:55 2023","interfaces":[{"interface":"back","average":{"1min":0.60599999999999998,"5min":0.61299999999999999,"15min":0.60399999999999998},"min":{"1min":0.39400000000000002,"5min":0.39400000000000002,"15min":0.39400000000000002},"max":{"1min":0.88,"5min":0.96299999999999997,"15min":0.96299999999999997},"last":0.76000000000000001},{"interface":"front","average":{"1min":0.60899999999999999,"5min":0.64800000000000002,"15min":0.66400000000000003},"min":{"1min":0.35999999999999999,"5min":0.35999999999999999,"15min":0.35999999999999999},"max":{"1min":0.92600000000000005,"5min":1.5509999999999999,"15min":1.5509999999999999},"last":0.78900000000000003}]},{"osd":5,"last update":"Sun Nov 26 07:22:55 2023","interfaces":[{"interface":"back","average":{"1min":0.63100000000000001,"5min":0.67500000000000004,"15min":0.66900000000000004},"min":{"1min":0.45000000000000001,"5min":0.443,"15min":0.443},"max":{"1min":0.95899999999999996,"5min":1.415,"15min":1.415},"last":0.81299999999999994},{"interface":"front","average":{"1min":0.68700000000000006,"5min":0.68799999999999994,"15min":0.70499999999999996},"min":{"1min":0.47499999999999998,"5min":0.47499999999999998,"15min":0.47499999999999998},"max":{"1min":0.93200000000000005,"5min":1.3859999999999999,"15min":1.3859999999999999},"last":0.873}]},{"osd":6,"last update":"Sun Nov 26 07:22:55 2023","interfaces":[{"interface":"back","average":{"1min":0.77800000000000002,"5min":0.78000000000000003,"15min":0.77400000000000002},"min":{"1min":0.57099999999999995,"5min":0.51700000000000002,"15min":0.51700000000000002},"max":{"1min":0.97599999999999998,"5min":1.3069999999999999,"15min":1.3069999999999999},"last":0.90200000000000002},{"interface":"front","average":{"1min":0.74099999999999999,"5min":0.75600000000000001,"15min":0.76600000000000001},"min":{"1min":0.49399999999999999,"5min":0.46899999999999997,"15min":0.46899999999999997},"max":{"1min":1.032,"5min":1.6759999999999999,"15min":1.6759999999999999},"last":0.85499999999999998}]}]},{"osd":6,"up_from":13,"seq":55834574922,"num_pgs":30,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32208,"kb_used_data":1020,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339632,"statfs":{"total":96636764160,"available":96603783168,"internally_reserved":0,"allocated":1044480,"data_stored":444883,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Nov 26 07:22:55 2023","interfaces":[{"interface":"back","average":{"1min":0.61199999999999999,"5min":0.68400000000000005,"15min":0.68400000000000005},"min":{"1min":0.32200000000000001,"5min":0.32200000000000001,"15min":0.32200000000000001},"max":{"1min":0.81999999999999995,"5min":1.7549999999999999,"15min":1.7549999999999999},"last":0.82399999999999995},{"interface":"front","average":{"1min":0.65300000000000002,"5min":0.75700000000000001,"15min":0.89100000000000001},"min":{"1min":0.42799999999999999,"5min":0.38300000000000001,"15min":0.38300000000000001},"max":{"1min":0.89400000000000002,"5min":5.7489999999999997,"15min":5.7489999999999997},"last":0.67900000000000005}]},{"osd":1,"last update":"Sun Nov 26 07:22:55 2023","interfaces":[{"interface":"back","average":{"1min":0.70099999999999996,"5min":0.76100000000000001,"15min":0.86299999999999999},"min":{"1min":0.50800000000000001,"5min":0.41199999999999998,"15min":0.41199999999999998},"max":{"1min":0.97699999999999998,"5min":5.4240000000000004,"15min":5.4240000000000004},"last":0.75800000000000001},{"interface":"front","average":{"1min":0.68000000000000005,"5min":0.751,"15min":0.84799999999999998},"min":{"1min":0.495,"5min":0.495,"15min":0.495},"max":{"1min":0.83599999999999997,"5min":5.6020000000000003,"15min":5.6020000000000003},"last":0.72999999999999998}]},{"osd":2,"last update":"Sun Nov 26 07:22:55 2023","interfaces":[{"interface":"back","average":{"1min":0.65000000000000002,"5min":0.68100000000000005,"15min":0.69099999999999995},"min":{"1min":0.41699999999999998,"5min":0.41699999999999998,"15min":0.41699999999999998},"max":{"1min":0.86799999999999999,"5min":1.8520000000000001,"15min":1.8520000000000001},"last":0.74099999999999999},{"interface":"front","average":{"1min":0.69699999999999995,"5min":0.72599999999999998,"15min":0.73099999999999998},"min":{"1min":0.54100000000000004,"5min":0.437,"15min":0.437},"max":{"1min":0.91800000000000004,"5min":1.9159999999999999,"15min":1.9159999999999999},"last":0.79500000000000004}]},{"osd":3,"last update":"Sun Nov 26 07:22:55 2023","interfaces":[{"interface":"back","average":{"1min":0.67700000000000005,"5min":0.755,"15min":0.85299999999999998},"min":{"1min":0.57099999999999995,"5min":0.46899999999999997,"15min":0.46899999999999997},"max":{"1min":0.86699999999999999,"5min":5.4390000000000001,"15min":5.4390000000000001},"last":0.78000000000000003},{"interface":"front","average":{"1min":0.68400000000000005,"5min":0.76300000000000001,"15min":0.88400000000000001},"min":{"1min":0.54200000000000004,"5min":0.48399999999999999,"15min":0.48399999999999999},"max":{"1min":0.85699999999999998,"5min":5.5490000000000004,"15min":5.5490000000000004},"last":0.69999999999999996}]},{"osd":4,"last update":"Sun Nov 26 07:22:55 2023","interfaces":[{"interface":"back","average":{"1min":0.59499999999999997,"5min":0.61699999999999999,"15min":0.63300000000000001},"min":{"1min":0.42199999999999999,"5min":0.371,"15min":0.371},"max":{"1min":0.85199999999999998,"5min":1.0129999999999999,"15min":1.0129999999999999},"last":0.505},{"interface":"front","average":{"1min":0.63300000000000001,"5min":0.64500000000000002,"15min":0.66500000000000004},"min":{"1min":0.40000000000000002,"5min":0.38600000000000001,"15min":0.38600000000000001},"max":{"1min":0.81999999999999995,"5min":1.0349999999999999,"15min":1.0349999999999999},"last":0.52500000000000002}]},{"osd":5,"last update":"Sun Nov 26 07:22:55 2023","interfaces":[{"interface":"back","average":{"1min":0.63900000000000001,"5min":0.63400000000000001,"15min":0.65000000000000002},"min":{"1min":0.42899999999999999,"5min":0.38200000000000001,"15min":0.38200000000000001},"max":{"1min":0.91000000000000003,"5min":0.91000000000000003,"15min":0.91000000000000003},"last":0.55600000000000005},{"interface":"front","average":{"1min":0.70499999999999996,"5min":0.69699999999999995,"15min":0.70699999999999996},"min":{"1min":0.46600000000000003,"5min":0.436,"15min":0.436},"max":{"1min":1.014,"5min":1.0960000000000001,"15min":1.0960000000000001},"last":0.65400000000000003}]},{"osd":7,"last update":"Sun Nov 26 07:22:55 2023","interfaces":[{"interface":"back","average":{"1min":0.72299999999999998,"5min":0.71599999999999997,"15min":0.73299999999999998},"min":{"1min":0.45100000000000001,"5min":0.42599999999999999,"15min":0.42599999999999999},"max":{"1min":0.95099999999999996,"5min":0.998,"15min":0.998},"last":0.57099999999999995},{"interface":"front","average":{"1min":0.70699999999999996,"5min":0.69599999999999995,"15min":0.70899999999999996},"min":{"1min":0.51600000000000001,"5min":0.46400000000000002,"15min":0.46400000000000002},"max":{"1min":0.90000000000000002,"5min":0.98799999999999999,"15min":0.98799999999999999},"last":0.63100000000000001}]}]},{"osd":5,"up_from":13,"seq":55834574920,"num_pgs":16,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":28052,"kb_used_data":1024,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343788,"statfs":{"total":96636764160,"available":96608038912,"internally_reserved":0,"allocated":1048576,"data_stored":445823,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Nov 26 07:22:49 2023","interfaces":[{"interface":"back","average":{"1min":0.80100000000000005,"5min":0.71499999999999997,"15min":0.68200000000000005},"min":{"1min":0.36499999999999999,"5min":0.36499999999999999,"15min":0.36499999999999999},"max":{"1min":2.6949999999999998,"5min":2.6949999999999998,"15min":2.6949999999999998},"last":0.72499999999999998},{"interface":"front","average":{"1min":0.76900000000000002,"5min":0.71699999999999997,"15min":0.67900000000000005},"min":{"1min":0.38200000000000001,"5min":0.38200000000000001,"15min":0.38200000000000001},"max":{"1min":2.0150000000000001,"5min":2.0150000000000001,"15min":2.0150000000000001},"last":0.66600000000000004}]},{"osd":1,"last update":"Sun Nov 26 07:22:49 2023","interfaces":[{"interface":"back","average":{"1min":0.71199999999999997,"5min":0.70399999999999996,"15min":0.68000000000000005},"min":{"1min":0.41799999999999998,"5min":0.41599999999999998,"15min":0.41599999999999998},"max":{"1min":1.532,"5min":1.532,"15min":1.532},"last":0.621},{"interface":"front","average":{"1min":0.79800000000000004,"5min":0.73099999999999998,"15min":0.70099999999999996},"min":{"1min":0.497,"5min":0.443,"15min":0.443},"max":{"1min":2.6400000000000001,"5min":2.6400000000000001,"15min":2.6400000000000001},"last":0.85499999999999998}]},{"osd":2,"last update":"Sun Nov 26 07:22:49 2023","interfaces":[{"interface":"back","average":{"1min":0.72699999999999998,"5min":0.70899999999999996,"15min":0.68600000000000005},"min":{"1min":0.47599999999999998,"5min":0.45900000000000002,"15min":0.45900000000000002},"max":{"1min":1.665,"5min":1.665,"15min":1.665},"last":0.86799999999999999},{"interface":"front","average":{"1min":0.77000000000000002,"5min":0.75,"15min":0.69599999999999995},"min":{"1min":0.46200000000000002,"5min":0.46200000000000002,"15min":0.46200000000000002},"max":{"1min":2.125,"5min":2.125,"15min":2.125},"last":0.78000000000000003}]},{"osd":3,"last update":"Sun Nov 26 07:22:49 2023","interfaces":[{"interface":"back","average":{"1min":0.80800000000000005,"5min":0.751,"15min":0.72999999999999998},"min":{"1min":0.57199999999999995,"5min":0.43099999999999999,"15min":0.43099999999999999},"max":{"1min":2.0529999999999999,"5min":2.0529999999999999,"15min":2.0529999999999999},"last":0.70999999999999996},{"interface":"front","average":{"1min":0.76200000000000001,"5min":0.73999999999999999,"15min":0.69599999999999995},"min":{"1min":0.51900000000000002,"5min":0.44600000000000001,"15min":0.44600000000000001},"max":{"1min":1.629,"5min":1.629,"15min":1.629},"last":0.755}]},{"osd":4,"last update":"Sun Nov 26 07:22:49 2023","interfaces":[{"interface":"back","average":{"1min":0.69499999999999995,"5min":0.61199999999999999,"15min":0.627},"min":{"1min":0.41799999999999998,"5min":0.35299999999999998,"15min":0.35299999999999998},"max":{"1min":2.7429999999999999,"5min":2.7429999999999999,"15min":2.7429999999999999},"last":0.81499999999999995},{"interface":"front","average":{"1min":0.77000000000000002,"5min":0.68200000000000005,"15min":0.69999999999999996},"min":{"1min":0.48599999999999999,"5min":0.41299999999999998,"15min":0.41299999999999998},"max":{"1min":2.4630000000000001,"5min":2.4630000000000001,"15min":2.4630000000000001},"last":0.80300000000000005}]},{"osd":6,"last update":"Sun Nov 26 07:22:49 2023","interfaces":[{"interface":"back","average":{"1min":0.73199999999999998,"5min":0.67400000000000004,"15min":0.67200000000000004},"min":{"1min":0.41999999999999998,"5min":0.41999999999999998,"15min":0.41999999999999998},"max":{"1min":2.1619999999999999,"5min":2.1619999999999999,"15min":2.1619999999999999},"last":0.89900000000000002},{"interface":"front","average":{"1min":0.77000000000000002,"5min":0.71199999999999997,"15min":0.72199999999999998},"min":{"1min":0.47599999999999998,"5min":0.436,"15min":0.436},"max":{"1min":2.7869999999999999,"5min":2.7869999999999999,"15min":2.7869999999999999},"last":0.88600000000000001}]},{"osd":7,"last update":"Sun Nov 26 07:22:49 2023","interfaces":[{"interface":"back","average":{"1min":0.80600000000000005,"5min":0.75900000000000001,"15min":0.78900000000000003},"min":{"1min":0.45700000000000002,"5min":0.45700000000000002,"15min":0.45700000000000002},"max":{"1min":2.1960000000000002,"5min":2.1960000000000002,"15min":2.1960000000000002},"last":0.83799999999999997},{"interface":"front","average":{"1min":0.83499999999999996,"5min":0.76300000000000001,"15min":0.77100000000000002},"min":{"1min":0.59599999999999997,"5min":0.45000000000000001,"15min":0.45000000000000001},"max":{"1min":1.9510000000000001,"5min":1.9510000000000001,"15min":1.9510000000000001},"last":0.69099999999999995}]}]},{"osd":4,"up_from":13,"seq":55834574922,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32188,"kb_used_data":1008,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339652,"statfs":{"total":96636764160,"available":96603803648,"internally_reserved":0,"allocated":1032192,"data_stored":444189,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Nov 26 07:22:53 2023","interfaces":[{"interface":"back","average":{"1min":0.68600000000000005,"5min":0.68700000000000006,"15min":0.75800000000000001},"min":{"1min":0.51800000000000002,"5min":0.32100000000000001,"15min":0.32100000000000001},"max":{"1min":0.86799999999999999,"5min":4.1689999999999996,"15min":4.1689999999999996},"last":0.55600000000000005},{"interface":"front","average":{"1min":0.66900000000000004,"5min":0.91700000000000004,"15min":0.97299999999999998},"min":{"1min":0.53800000000000003,"5min":0.46200000000000002,"15min":0.46200000000000002},"max":{"1min":0.85699999999999998,"5min":18.021000000000001,"15min":18.021000000000001},"last":0.79000000000000004}]},{"osd":1,"last update":"Sun Nov 26 07:22:53 2023","interfaces":[{"interface":"back","average":{"1min":0.68500000000000005,"5min":0.71899999999999997,"15min":0.67100000000000004},"min":{"1min":0.52200000000000002,"5min":0.43099999999999999,"15min":0.43099999999999999},"max":{"1min":0.84099999999999997,"5min":4.3159999999999998,"15min":4.3159999999999998},"last":0.57699999999999996},{"interface":"front","average":{"1min":0.65800000000000003,"5min":0.79500000000000004,"15min":0.81100000000000005},"min":{"1min":0.48699999999999999,"5min":0.39800000000000002,"15min":0.39800000000000002},"max":{"1min":0.876,"5min":11.206,"15min":11.206},"last":0.748}]},{"osd":2,"last update":"Sun Nov 26 07:22:53 2023","interfaces":[{"interface":"back","average":{"1min":0.67200000000000004,"5min":0.871,"15min":0.82799999999999996},"min":{"1min":0.46600000000000003,"5min":0.433,"15min":0.433},"max":{"1min":0.84399999999999997,"5min":15.441000000000001,"15min":15.441000000000001},"last":0.71299999999999997},{"interface":"front","average":{"1min":0.76900000000000002,"5min":0.91500000000000004,"15min":1.0149999999999999},"min":{"1min":0.64200000000000002,"5min":0.53400000000000003,"15min":0.53400000000000003},"max":{"1min":0.93300000000000005,"5min":9.4800000000000004,"15min":9.4800000000000004},"last":0.72599999999999998}]},{"osd":3,"last update":"Sun Nov 26 07:22:53 2023","interfaces":[{"interface":"back","average":{"1min":0.71299999999999997,"5min":0.83199999999999996,"15min":0.76400000000000001},"min":{"1min":0.498,"5min":0.441,"15min":0.441},"max":{"1min":0.92400000000000004,"5min":12.715,"15min":12.715},"last":0.60299999999999998},{"interface":"front","average":{"1min":0.72099999999999997,"5min":0.83799999999999997,"15min":0.86499999999999999},"min":{"1min":0.51900000000000002,"5min":0.50700000000000001,"15min":0.50700000000000001},"max":{"1min":0.89800000000000002,"5min":8.3239999999999998,"15min":8.3239999999999998},"last":0.65200000000000002}]},{"osd":5,"last update":"Sun Nov 26 07:22:53 2023","interfaces":[{"interface":"back","average":{"1min":0.625,"5min":0.61599999999999999,"15min":0.61199999999999999},"min":{"1min":0.46300000000000002,"5min":0.38200000000000001,"15min":0.38200000000000001},"max":{"1min":0.86299999999999999,"5min":0.96199999999999997,"15min":0.96199999999999997},"last":0.63400000000000001},{"interface":"front","average":{"1min":0.60299999999999998,"5min":0.71999999999999997,"15min":0.66700000000000004},"min":{"1min":0.41299999999999998,"5min":0.35599999999999998,"15min":0.35599999999999998},"max":{"1min":0.80800000000000005,"5min":11.148999999999999,"15min":11.148999999999999},"last":0.69399999999999995}]},{"osd":6,"last update":"Sun Nov 26 07:22:53 2023","interfaces":[{"interface":"back","average":{"1min":0.61299999999999999,"5min":0.63900000000000001,"15min":0.74199999999999999},"min":{"1min":0.41099999999999998,"5min":0.372,"15min":0.372},"max":{"1min":0.89200000000000002,"5min":2.8330000000000002,"15min":2.8330000000000002},"last":0.76700000000000002},{"interface":"front","average":{"1min":0.68400000000000005,"5min":0.76800000000000002,"15min":0.92800000000000005},"min":{"1min":0.42699999999999999,"5min":0.41299999999999998,"15min":0.41299999999999998},"max":{"1min":0.93999999999999995,"5min":7.6050000000000004,"15min":7.6050000000000004},"last":0.67800000000000005}]},{"osd":7,"last update":"Sun Nov 26 07:22:53 2023","interfaces":[{"interface":"back","average":{"1min":0.68899999999999995,"5min":0.84699999999999998,"15min":0.85599999999999998},"min":{"1min":0.53900000000000003,"5min":0.44700000000000001,"15min":0.44700000000000001},"max":{"1min":0.871,"5min":12.653,"15min":12.653},"last":0.78200000000000003},{"interface":"front","average":{"1min":0.75900000000000001,"5min":0.79500000000000004,"15min":0.85899999999999999},"min":{"1min":0.56699999999999995,"5min":0.33300000000000002,"15min":0.33300000000000002},"max":{"1min":0.95499999999999996,"5min":3.8319999999999999,"15min":3.8319999999999999},"last":0.81000000000000005}]}]},{"osd":3,"up_from":13,"seq":55834574923,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32200,"kb_used_data":1020,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339640,"statfs":{"total":96636764160,"available":96603791360,"internally_reserved":0,"allocated":1044480,"data_stored":445801,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1592,"internal_metadata":31914440},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Nov 26 07:22:49 2023","interfaces":[{"interface":"back","average":{"1min":0.39000000000000001,"5min":0.41499999999999998,"15min":0.39100000000000001},"min":{"1min":0.218,"5min":0.218,"15min":0.218},"max":{"1min":0.56499999999999995,"5min":0.72799999999999998,"15min":0.72799999999999998},"last":0.44700000000000001},{"interface":"front","average":{"1min":0.439,"5min":0.46000000000000002,"15min":0.442},"min":{"1min":0.254,"5min":0.254,"15min":0.254},"max":{"1min":0.65200000000000002,"5min":0.77300000000000002,"15min":0.77300000000000002},"last":0.46999999999999997}]},{"osd":1,"last update":"Sun Nov 26 07:22:49 2023","interfaces":[{"interface":"back","average":{"1min":0.40999999999999998,"5min":0.436,"15min":0.42599999999999999},"min":{"1min":0.32100000000000001,"5min":0.28799999999999998,"15min":0.28799999999999998},"max":{"1min":0.57199999999999995,"5min":0.752,"15min":0.752},"last":0.59999999999999998},{"interface":"front","average":{"1min":0.439,"5min":0.48899999999999999,"15min":0.48499999999999999},"min":{"1min":0.29899999999999999,"5min":0.29899999999999999,"15min":0.29899999999999999},"max":{"1min":0.69799999999999995,"5min":0.872,"15min":0.872},"last":0.42399999999999999}]},{"osd":2,"last update":"Sun Nov 26 07:22:49 2023","interfaces":[{"interface":"back","average":{"1min":0.52000000000000002,"5min":0.56100000000000005,"15min":0.54200000000000004},"min":{"1min":0.42399999999999999,"5min":0.376,"15min":0.376},"max":{"1min":0.76100000000000001,"5min":0.97199999999999998,"15min":0.97199999999999998},"last":0.67400000000000004},{"interface":"front","average":{"1min":0.48899999999999999,"5min":0.53300000000000003,"15min":0.51400000000000001},"min":{"1min":0.32100000000000001,"5min":0.32100000000000001,"15min":0.32100000000000001},"max":{"1min":0.64200000000000002,"5min":0.89700000000000002,"15min":0.89700000000000002},"last":0.53300000000000003}]},{"osd":4,"last update":"Sun Nov 26 07:22:49 2023","interfaces":[{"interface":"back","average":{"1min":0.65000000000000002,"5min":0.70699999999999996,"15min":0.70599999999999996},"min":{"1min":0.39600000000000002,"5min":0.39600000000000002,"15min":0.39600000000000002},"max":{"1min":0.879,"5min":1.091,"15min":1.091},"last":0.51400000000000001},{"interface":"front","average":{"1min":0.65200000000000002,"5min":0.72299999999999998,"15min":0.72599999999999998},"min":{"1min":0.42499999999999999,"5min":0.39900000000000002,"15min":0.39900000000000002},"max":{"1min":0.85799999999999998,"5min":1.242,"15min":1.242},"last":0.65400000000000003}]},{"osd":5,"last update":"Sun Nov 26 07:22:49 2023","interfaces":[{"interface":"back","average":{"1min":0.60599999999999998,"5min":0.70699999999999996,"15min":0.72799999999999998},"min":{"1min":0.45900000000000002,"5min":0.45900000000000002,"15min":0.45900000000000002},"max":{"1min":0.73999999999999999,"5min":1.105,"15min":1.105},"last":0.48699999999999999},{"interface":"front","average":{"1min":0.66600000000000004,"5min":0.73599999999999999,"15min":0.73699999999999999},"min":{"1min":0.47299999999999998,"5min":0.47299999999999998,"15min":0.47299999999999998},"max":{"1min":0.876,"5min":1.1319999999999999,"15min":1.1319999999999999},"last":0.54800000000000004}]},{"osd":6,"last update":"Sun Nov 26 07:22:49 2023","interfaces":[{"interface":"back","average":{"1min":0.68300000000000005,"5min":0.76800000000000002,"15min":0.79800000000000004},"min":{"1min":0.5,"5min":0.5,"15min":0.5},"max":{"1min":0.90600000000000003,"5min":1.2729999999999999,"15min":1.2729999999999999},"last":0.63100000000000001},{"interface":"front","average":{"1min":0.69299999999999995,"5min":0.75900000000000001,"15min":0.78100000000000003},"min":{"1min":0.48799999999999999,"5min":0.47299999999999998,"15min":0.47299999999999998},"max":{"1min":0.88,"5min":1.204,"15min":1.204},"last":0.61799999999999999}]},{"osd":7,"last update":"Sun Nov 26 07:22:49 2023","interfaces":[{"interface":"back","average":{"1min":0.70999999999999996,"5min":0.75800000000000001,"15min":0.78600000000000003},"min":{"1min":0.437,"5min":0.437,"15min":0.437},"max":{"1min":0.94699999999999995,"5min":1.3600000000000001,"15min":1.3600000000000001},"last":0.58699999999999997},{"interface":"front","average":{"1min":0.72699999999999998,"5min":0.79800000000000004,"15min":0.81599999999999995},"min":{"1min":0.54500000000000004,"5min":0.47299999999999998,"15min":0.47299999999999998},"max":{"1min":0.98999999999999999,"5min":1.617,"15min":1.617},"last":0.69699999999999995}]}]},{"osd":2,"up_from":13,"seq":55834574923,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32200,"kb_used_data":1012,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339640,"statfs":{"total":96636764160,"available":96603791360,"internally_reserved":0,"allocated":1036288,"data_stored":444771,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":7,"apply_latency_ms":7,"commit_latency_ns":7000000,"apply_latency_ns":7000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Nov 26 07:22:50 2023","interfaces":[{"interface":"back","average":{"1min":0.39500000000000002,"5min":0.48799999999999999,"15min":0.63800000000000001},"min":{"1min":0.27100000000000002,"5min":0.27100000000000002,"15min":0.27100000000000002},"max":{"1min":0.624,"5min":4.7030000000000003,"15min":4.7030000000000003},"last":0.41599999999999998},{"interface":"front","average":{"1min":0.41299999999999998,"5min":0.48299999999999998,"15min":0.53500000000000003},"min":{"1min":0.313,"5min":0.27300000000000002,"15min":0.27300000000000002},"max":{"1min":0.64300000000000002,"5min":2.4340000000000002,"15min":2.4340000000000002},"last":0.44400000000000001}]},{"osd":1,"last update":"Sun Nov 26 07:22:50 2023","interfaces":[{"interface":"back","average":{"1min":0.46000000000000002,"5min":0.56799999999999995,"15min":0.76500000000000001},"min":{"1min":0.318,"5min":0.318,"15min":0.318},"max":{"1min":0.81000000000000005,"5min":4.2690000000000001,"15min":4.2690000000000001},"last":0.47799999999999998},{"interface":"front","average":{"1min":0.44500000000000001,"5min":0.54900000000000004,"15min":0.67700000000000005},"min":{"1min":0.35899999999999999,"5min":0.31900000000000001,"15min":0.31900000000000001},"max":{"1min":0.58599999999999997,"5min":3.9940000000000002,"15min":3.9940000000000002},"last":0.52200000000000002}]},{"osd":3,"last update":"Sun Nov 26 07:22:50 2023","interfaces":[{"interface":"back","average":{"1min":0.42099999999999999,"5min":0.50700000000000001,"15min":0.59699999999999998},"min":{"1min":0.29699999999999999,"5min":0.29699999999999999,"15min":0.29699999999999999},"max":{"1min":0.54000000000000004,"5min":4.0300000000000002,"15min":4.0300000000000002},"last":0.55300000000000005},{"interface":"front","average":{"1min":0.50700000000000001,"5min":0.622,"15min":0.81599999999999995},"min":{"1min":0.38600000000000001,"5min":0.36099999999999999,"15min":0.36099999999999999},"max":{"1min":0.77000000000000002,"5min":4.5810000000000004,"15min":4.5810000000000004},"last":0.51100000000000001}]},{"osd":4,"last update":"Sun Nov 26 07:22:50 2023","interfaces":[{"interface":"back","average":{"1min":0.78900000000000003,"5min":0.81699999999999995,"15min":0.92900000000000005},"min":{"1min":0.52600000000000002,"5min":0.51400000000000001,"15min":0.51400000000000001},"max":{"1min":0.91300000000000003,"5min":4.4269999999999996,"15min":4.4269999999999996},"last":0.70999999999999996},{"interface":"front","average":{"1min":0.71499999999999997,"5min":0.82099999999999995,"15min":0.94999999999999996},"min":{"1min":0.54000000000000004,"5min":0.54000000000000004,"15min":0.54000000000000004},"max":{"1min":0.91900000000000004,"5min":4.085,"15min":4.085},"last":0.624}]},{"osd":5,"last update":"Sun Nov 26 07:22:50 2023","interfaces":[{"interface":"back","average":{"1min":0.753,"5min":0.78600000000000003,"15min":0.84399999999999997},"min":{"1min":0.51800000000000002,"5min":0.49199999999999999,"15min":0.49199999999999999},"max":{"1min":0.95399999999999996,"5min":3.9590000000000001,"15min":3.9590000000000001},"last":0.71999999999999997},{"interface":"front","average":{"1min":0.79400000000000004,"5min":0.83499999999999996,"15min":0.93700000000000006},"min":{"1min":0.53700000000000003,"5min":0.499,"15min":0.499},"max":{"1min":1.012,"5min":4.3849999999999998,"15min":4.3849999999999998},"last":0.58399999999999996}]},{"osd":6,"last update":"Sun Nov 26 07:22:50 2023","interfaces":[{"interface":"back","average":{"1min":0.78600000000000003,"5min":0.81999999999999995,"15min":0.93700000000000006},"min":{"1min":0.58099999999999996,"5min":0.51800000000000002,"15min":0.51800000000000002},"max":{"1min":1.083,"5min":4.0449999999999999,"15min":4.0449999999999999},"last":0.65800000000000003},{"interface":"front","average":{"1min":0.749,"5min":0.80100000000000005,"15min":0.879},"min":{"1min":0.49099999999999999,"5min":0.48099999999999998,"15min":0.48099999999999998},"max":{"1min":1.0349999999999999,"5min":3.992,"15min":3.992},"last":0.60499999999999998}]},{"osd":7,"last update":"Sun Nov 26 07:22:50 2023","interfaces":[{"interface":"back","average":{"1min":0.78700000000000003,"5min":0.83599999999999997,"15min":0.96799999999999997},"min":{"1min":0.59599999999999997,"5min":0.52200000000000002,"15min":0.52200000000000002},"max":{"1min":1.0209999999999999,"5min":4.1219999999999999,"15min":4.1219999999999999},"last":0.68700000000000006},{"interface":"front","average":{"1min":0.75,"5min":0.81999999999999995,"15min":0.89800000000000002},"min":{"1min":0.48199999999999998,"5min":0.48199999999999998,"15min":0.48199999999999998},"max":{"1min":0.98799999999999999,"5min":3.9159999999999999,"15min":3.9159999999999999},"last":0.64200000000000002}]}]},{"osd":1,"up_from":13,"seq":55834574922,"num_pgs":29,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32200,"kb_used_data":1020,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339640,"statfs":{"total":96636764160,"available":96603791360,"internally_reserved":0,"allocated":1044480,"data_stored":444383,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":31914439},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Nov 26 07:22:47 2023","interfaces":[{"interface":"back","average":{"1min":0.40600000000000003,"5min":0.40799999999999997,"15min":0.41399999999999998},"min":{"1min":0.28899999999999998,"5min":0.23599999999999999,"15min":0.23599999999999999},"max":{"1min":0.55000000000000004,"5min":0.61199999999999999,"15min":0.61199999999999999},"last":0.27300000000000002},{"interface":"front","average":{"1min":0.45800000000000002,"5min":0.45700000000000002,"15min":0.47599999999999998},"min":{"1min":0.28399999999999997,"5min":0.26700000000000002,"15min":0.26700000000000002},"max":{"1min":0.749,"5min":0.78700000000000003,"15min":0.78700000000000003},"last":0.45300000000000001}]},{"osd":2,"last update":"Sun Nov 26 07:22:47 2023","interfaces":[{"interface":"back","average":{"1min":0.45000000000000001,"5min":0.442,"15min":0.45000000000000001},"min":{"1min":0.34499999999999997,"5min":0.27800000000000002,"15min":0.27800000000000002},"max":{"1min":0.71399999999999997,"5min":0.71399999999999997,"15min":0.71399999999999997},"last":0.47199999999999998},{"interface":"front","average":{"1min":0.47199999999999998,"5min":0.49099999999999999,"15min":0.502},"min":{"1min":0.36599999999999999,"5min":0.27700000000000002,"15min":0.27700000000000002},"max":{"1min":0.63200000000000001,"5min":0.80600000000000005,"15min":0.80600000000000005},"last":0.318}]},{"osd":3,"last update":"Sun Nov 26 07:22:47 2023","interfaces":[{"interface":"back","average":{"1min":0.53200000000000003,"5min":0.52900000000000003,"15min":0.52900000000000003},"min":{"1min":0.441,"5min":0.38400000000000001,"15min":0.38400000000000001},"max":{"1min":0.65000000000000002,"5min":0.748,"15min":0.748},"last":0.36099999999999999},{"interface":"front","average":{"1min":0.496,"5min":0.499,"15min":0.497},"min":{"1min":0.35799999999999998,"5min":0.31,"15min":0.31},"max":{"1min":0.66700000000000004,"5min":0.80700000000000005,"15min":0.80700000000000005},"last":0.39500000000000002}]},{"osd":4,"last update":"Sun Nov 26 07:22:47 2023","interfaces":[{"interface":"back","average":{"1min":0.74199999999999999,"5min":0.72899999999999998,"15min":0.70499999999999996},"min":{"1min":0.52400000000000002,"5min":0.41299999999999998,"15min":0.41299999999999998},"max":{"1min":1.0640000000000001,"5min":1.1000000000000001,"15min":1.1000000000000001},"last":0.84199999999999997},{"interface":"front","average":{"1min":0.66200000000000003,"5min":0.71099999999999997,"15min":0.68999999999999995},"min":{"1min":0.44900000000000001,"5min":0.44500000000000001,"15min":0.44500000000000001},"max":{"1min":1.0009999999999999,"5min":1.131,"15min":1.131},"last":0.69999999999999996}]},{"osd":5,"last update":"Sun Nov 26 07:22:47 2023","interfaces":[{"interface":"back","average":{"1min":0.74399999999999999,"5min":0.73599999999999999,"15min":0.72399999999999998},"min":{"1min":0.54700000000000004,"5min":0.49299999999999999,"15min":0.49299999999999999},"max":{"1min":1.0409999999999999,"5min":1.1919999999999999,"15min":1.1919999999999999},"last":0.67600000000000005},{"interface":"front","average":{"1min":0.76500000000000001,"5min":0.73699999999999999,"15min":0.72299999999999998},"min":{"1min":0.47099999999999997,"5min":0.47099999999999997,"15min":0.47099999999999997},"max":{"1min":1.0389999999999999,"5min":1.0609999999999999,"15min":1.0609999999999999},"last":0.78500000000000003}]},{"osd":6,"last update":"Sun Nov 26 07:22:47 2023","interfaces":[{"interface":"back","average":{"1min":0.755,"5min":0.74199999999999999,"15min":0.71599999999999997},"min":{"1min":0.53200000000000003,"5min":0.50600000000000001,"15min":0.50600000000000001},"max":{"1min":0.98699999999999999,"5min":1.0369999999999999,"15min":1.0369999999999999},"last":0.93100000000000005},{"interface":"front","average":{"1min":0.72799999999999998,"5min":0.73899999999999999,"15min":0.71299999999999997},"min":{"1min":0.46999999999999997,"5min":0.46999999999999997,"15min":0.46999999999999997},"max":{"1min":1.079,"5min":1.1339999999999999,"15min":1.1339999999999999},"last":0.94499999999999995}]},{"osd":7,"last update":"Sun Nov 26 07:22:47 2023","interfaces":[{"interface":"back","average":{"1min":0.73999999999999999,"5min":0.746,"15min":0.72499999999999998},"min":{"1min":0.55400000000000005,"5min":0.50600000000000001,"15min":0.50600000000000001},"max":{"1min":1.0149999999999999,"5min":1.0149999999999999,"15min":1.0149999999999999},"last":0.76200000000000001},{"interface":"front","average":{"1min":0.73199999999999998,"5min":0.74199999999999999,"15min":0.749},"min":{"1min":0.53700000000000003,"5min":0.48499999999999999,"15min":0.48499999999999999},"max":{"1min":0.95299999999999996,"5min":1.024,"15min":1.024},"last":0.86499999999999999}]}]},{"osd":0,"up_from":13,"seq":55834574922,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32776,"kb_used_data":1592,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339064,"statfs":{"total":96636764160,"available":96603201536,"internally_reserved":0,"allocated":1630208,"data_stored":1034591,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Sun Nov 26 07:22:46 2023","interfaces":[{"interface":"back","average":{"1min":0.46600000000000003,"5min":0.433,"15min":0.43099999999999999},"min":{"1min":0.29199999999999998,"5min":0.27000000000000002,"15min":0.27000000000000002},"max":{"1min":1.26,"5min":1.26,"15min":1.26},"last":0.47099999999999997},{"interface":"front","average":{"1min":0.499,"5min":0.47199999999999998,"15min":0.47899999999999998},"min":{"1min":0.30199999999999999,"5min":0.29899999999999999,"15min":0.29899999999999999},"max":{"1min":1.212,"5min":1.212,"15min":1.212},"last":0.33300000000000002}]},{"osd":2,"last update":"Sun Nov 26 07:22:46 2023","interfaces":[{"interface":"back","average":{"1min":0.51400000000000001,"5min":0.51300000000000001,"15min":0.52900000000000003},"min":{"1min":0.38500000000000001,"5min":0.34699999999999998,"15min":0.34699999999999998},"max":{"1min":0.75600000000000001,"5min":0.84199999999999997,"15min":0.84199999999999997},"last":0.55600000000000005},{"interface":"front","average":{"1min":0.54700000000000004,"5min":0.55300000000000005,"15min":0.56000000000000005},"min":{"1min":0.40100000000000002,"5min":0.36399999999999999,"15min":0.36399999999999999},"max":{"1min":1.0800000000000001,"5min":1.0800000000000001,"15min":1.0800000000000001},"last":0.59199999999999997}]},{"osd":3,"last update":"Sun Nov 26 07:22:46 2023","interfaces":[{"interface":"back","average":{"1min":0.51700000000000002,"5min":0.51900000000000002,"15min":0.52700000000000002},"min":{"1min":0.38700000000000001,"5min":0.34999999999999998,"15min":0.34999999999999998},"max":{"1min":0.873,"5min":1.3149999999999999,"15min":1.3149999999999999},"last":0.41799999999999998},{"interface":"front","average":{"1min":0.50800000000000001,"5min":0.53300000000000003,"15min":0.55200000000000005},"min":{"1min":0.39700000000000002,"5min":0.371,"15min":0.371},"max":{"1min":0.63,"5min":1.2509999999999999,"15min":1.2509999999999999},"last":0.45600000000000002}]},{"osd":4,"last update":"Sun Nov 26 07:22:46 2023","interfaces":[{"interface":"back","average":{"1min":0.747,"5min":0.747,"15min":0.755},"min":{"1min":0.59099999999999997,"5min":0.44700000000000001,"15min":0.44700000000000001},"max":{"1min":0.98799999999999999,"5min":1.2010000000000001,"15min":1.2010000000000001},"last":0.71299999999999997},{"interface":"front","average":{"1min":0.78100000000000003,"5min":0.78700000000000003,"15min":0.78800000000000003},"min":{"1min":0.63800000000000001,"5min":0.51100000000000001,"15min":0.51100000000000001},"max":{"1min":1.0529999999999999,"5min":1.381,"15min":1.381},"last":0.73199999999999998}]},{"osd":5,"last update":"Sun Nov 26 07:22:46 2023","interfaces":[{"interface":"back","average":{"1min":0.73499999999999999,"5min":0.75600000000000001,"15min":0.75800000000000001},"min":{"1min":0.49399999999999999,"5min":0.42199999999999999,"15min":0.42199999999999999},"max":{"1min":1.1819999999999999,"5min":1.1819999999999999,"15min":1.1819999999999999},"last":0.76100000000000001},{"interface":"front","average":{"1min":0.79200000000000004,"5min":0.80500000000000005,"15min":0.79900000000000004},"min":{"1min":0.60099999999999998,"5min":0.53600000000000003,"15min":0.53600000000000003},"max":{"1min":1.143,"5min":1.399,"15min":1.399},"last":0.77400000000000002}]},{"osd":6,"last update":"Sun Nov 26 07:22:46 2023","interfaces":[{"interface":"back","average":{"1min":0.67300000000000004,"5min":0.70399999999999996,"15min":0.74399999999999999},"min":{"1min":0.54900000000000004,"5min":0.50800000000000001,"15min":0.50800000000000001},"max":{"1min":1.0069999999999999,"5min":1.3029999999999999,"15min":1.3029999999999999},"last":0.67100000000000004},{"interface":"front","average":{"1min":0.70999999999999996,"5min":0.745,"15min":0.76100000000000001},"min":{"1min":0.51200000000000001,"5min":0.47199999999999998,"15min":0.47199999999999998},"max":{"1min":0.997,"5min":1.3600000000000001,"15min":1.3600000000000001},"last":0.81499999999999995}]},{"osd":7,"last update":"Sun Nov 26 07:22:46 2023","interfaces":[{"interface":"back","average":{"1min":0.75,"5min":0.77500000000000002,"15min":0.79900000000000004},"min":{"1min":0.62,"5min":0.56899999999999995,"15min":0.56899999999999995},"max":{"1min":0.88500000000000001,"5min":1.177,"15min":1.177},"last":0.79000000000000004},{"interface":"front","average":{"1min":0.78800000000000003,"5min":0.76500000000000001,"15min":0.78700000000000003},"min":{"1min":0.47899999999999998,"5min":0.47899999999999998,"15min":0.47899999999999998},"max":{"1min":1.238,"5min":1.2390000000000001,"15min":1.2390000000000001},"last":0.65000000000000002}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":34,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":194,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1634,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":694,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":70,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-11-26T07:23:33.726 DEBUG:teuthology.orchestra.run.smithi143:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-11-26T07:23:33.899 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:33.899+0000 7fa079972640 1 Processor -- start 2023-11-26T07:23:33.899 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:33.899+0000 7fa079972640 1 -- start start 2023-11-26T07:23:33.901 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:33.899+0000 7fa079972640 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7fa0740b3610 0x7fa0740cfa40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:23:33.901 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:33.899+0000 7fa079972640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fa0740b45d0 0x7fa0740cff80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:23:33.901 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:33.899+0000 7fa079972640 1 -- --> v1:172.21.15.143:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa0740cefa0 con 0x7fa0740b4080 2023-11-26T07:23:33.901 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:33.899+0000 7fa079972640 1 -- --> v2:172.21.15.143:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa0740c9fa0 con 0x7fa0740b3610 2023-11-26T07:23:33.901 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:33.899+0000 7fa079972640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa0740b3ef0 con 0x7fa0740b45d0 2023-11-26T07:23:33.901 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:33.899+0000 7fa072ffd640 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7fa0740b3610 0x7fa0740cfa40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:23:33.901 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:33.899+0000 7fa0737fe640 1 --1- >> v1:172.21.15.143:6789/0 conn(0x7fa0740b4080 0x7fa0740cddd0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.143:6789/0 says I am v1:172.21.15.143:43224/0 (socket says 172.21.15.143:43224) 2023-11-26T07:23:33.901 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:33.899+0000 7fa0737fe640 1 -- 172.21.15.143:0/1485053629 learned_addr learned my addr 172.21.15.143:0/1485053629 (peer_addr_for_me v1:172.21.15.143:0/0) 2023-11-26T07:23:33.901 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:33.899+0000 7fa0727fc640 1 --2- 172.21.15.143:0/1485053629 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fa0740b45d0 0x7fa0740cff80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:23:33.902 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:33.899+0000 7fa072ffd640 1 -- 172.21.15.143:0/1485053629 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fa0740b45d0 msgr2=0x7fa0740cff80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-26T07:23:33.902 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:33.899+0000 7fa072ffd640 1 --2- 172.21.15.143:0/1485053629 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fa0740b45d0 0x7fa0740cff80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:23:33.902 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:33.899+0000 7fa072ffd640 1 -- 172.21.15.143:0/1485053629 >> v1:172.21.15.143:6789/0 conn(0x7fa0740b4080 legacy=0x7fa0740cddd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:23:33.902 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:33.899+0000 7fa072ffd640 1 -- 172.21.15.143:0/1485053629 --> v2:172.21.15.143:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa0740d0f90 con 0x7fa0740b3610 2023-11-26T07:23:33.902 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:33.899+0000 7fa0727fc640 1 --2- 172.21.15.143:0/1485053629 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fa0740b45d0 0x7fa0740cff80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-26T07:23:33.903 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:33.899+0000 7fa071ffb640 1 -- 172.21.15.143:0/1485053629 <== mon.0 v1:172.21.15.143:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3588255979 0 0) 0x7fa0740cefa0 con 0x7fa0740b4080 2023-11-26T07:23:33.904 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:33.903+0000 7fa072ffd640 1 --2- 172.21.15.143:0/1485053629 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fa0740b3610 0x7fa0740cfa40 secure :-1 s=READY pgs=1157 cs=0 l=1 rev1=1 crypto rx=0x7fa0640097c0 tx=0x7fa06401ec70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=222c644d18290b9c server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:23:33.904 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:33.903+0000 7fa071ffb640 1 -- 172.21.15.143:0/1485053629 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa0640a8970 con 0x7fa0740b3610 2023-11-26T07:23:33.905 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:33.903+0000 7fa071ffb640 1 -- 172.21.15.143:0/1485053629 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa0640a8ad0 con 0x7fa0740b3610 2023-11-26T07:23:33.905 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:33.903+0000 7fa071ffb640 1 -- 172.21.15.143:0/1485053629 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa0640a8e60 con 0x7fa0740b3610 2023-11-26T07:23:33.905 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:33.903+0000 7fa079972640 1 -- 172.21.15.143:0/1485053629 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fa0740b3610 msgr2=0x7fa0740cfa40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:23:33.905 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:33.903+0000 7fa079972640 1 --2- 172.21.15.143:0/1485053629 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fa0740b3610 0x7fa0740cfa40 secure :-1 s=READY pgs=1157 cs=0 l=1 rev1=1 crypto rx=0x7fa0640097c0 tx=0x7fa06401ec70 comp rx=0 tx=0).stop 2023-11-26T07:23:33.906 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:33.903+0000 7fa079972640 1 -- 172.21.15.143:0/1485053629 shutdown_connections 2023-11-26T07:23:33.906 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:33.903+0000 7fa079972640 1 --2- 172.21.15.143:0/1485053629 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fa0740b45d0 0x7fa0740cff80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:23:33.906 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:33.903+0000 7fa079972640 1 --2- 172.21.15.143:0/1485053629 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fa0740b3610 0x7fa0740cfa40 unknown :-1 s=CLOSED pgs=1157 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:23:33.906 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:33.903+0000 7fa079972640 1 -- 172.21.15.143:0/1485053629 >> 172.21.15.143:0/1485053629 conn(0x7fa0740d3030 msgr2=0x7fa0740d3440 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:23:33.906 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:33.903+0000 7fa079972640 1 -- 172.21.15.143:0/1485053629 shutdown_connections 2023-11-26T07:23:33.906 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:33.903+0000 7fa079972640 1 -- 172.21.15.143:0/1485053629 wait complete. 2023-11-26T07:23:33.906 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:33.907+0000 7fa079972640 1 Processor -- start 2023-11-26T07:23:33.907 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:33.907+0000 7fa079972640 1 -- start start 2023-11-26T07:23:33.909 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:33.907+0000 7fa079972640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fa0740b3610 0x7fa074156a40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:23:33.909 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:33.907+0000 7fa079972640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fa0740b4080 0x7fa0741570e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:23:33.909 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:33.907+0000 7fa079972640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fa0740b45d0 0x7fa074157620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:23:33.909 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:33.907+0000 7fa079972640 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa074156f80 con 0x7fa0740b45d0 2023-11-26T07:23:33.909 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:33.907+0000 7fa079972640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa0740b72e0 con 0x7fa0740b3610 2023-11-26T07:23:33.909 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:33.907+0000 7fa079972640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa0740cc6d0 con 0x7fa0740b4080 2023-11-26T07:23:33.909 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:33.907+0000 7fa0737fe640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fa0740b45d0 0x7fa074157620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:23:33.909 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:33.907+0000 7fa0737fe640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fa0740b45d0 0x7fa074157620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:46430/0 (socket says 172.21.15.143:46430) 2023-11-26T07:23:33.909 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:33.907+0000 7fa0737fe640 1 -- 172.21.15.143:0/513192395 learned_addr learned my addr 172.21.15.143:0/513192395 (peer_addr_for_me v2:172.21.15.143:0/0) 2023-11-26T07:23:33.909 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:33.907+0000 7fa072ffd640 1 --2- 172.21.15.143:0/513192395 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fa0740b3610 0x7fa074156a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:23:33.910 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:33.907+0000 7fa0727fc640 1 --2- 172.21.15.143:0/513192395 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fa0740b4080 0x7fa0741570e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:23:33.910 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:33.907+0000 7fa0737fe640 1 -- 172.21.15.143:0/513192395 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fa0740b4080 msgr2=0x7fa0741570e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:23:33.910 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:33.907+0000 7fa0737fe640 1 --2- 172.21.15.143:0/513192395 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fa0740b4080 0x7fa0741570e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:23:33.910 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:33.907+0000 7fa0737fe640 1 -- 172.21.15.143:0/513192395 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fa0740b3610 msgr2=0x7fa074156a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:23:33.910 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:33.907+0000 7fa0737fe640 1 --2- 172.21.15.143:0/513192395 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fa0740b3610 0x7fa074156a40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:23:33.910 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:33.907+0000 7fa0737fe640 1 -- 172.21.15.143:0/513192395 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa064009420 con 0x7fa0740b45d0 2023-11-26T07:23:33.910 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:33.907+0000 7fa072ffd640 1 --2- 172.21.15.143:0/513192395 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fa0740b3610 0x7fa074156a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-26T07:23:33.910 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:33.907+0000 7fa0737fe640 1 --2- 172.21.15.143:0/513192395 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fa0740b45d0 0x7fa074157620 secure :-1 s=READY pgs=1158 cs=0 l=1 rev1=1 crypto rx=0x7fa06c001570 tx=0x7fa06c00cca0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:23:33.911 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:33.907+0000 7fa05bfff640 1 -- 172.21.15.143:0/513192395 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa06c00a640 con 0x7fa0740b45d0 2023-11-26T07:23:33.911 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:33.907+0000 7fa05bfff640 1 -- 172.21.15.143:0/513192395 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa06c00a7c0 con 0x7fa0740b45d0 2023-11-26T07:23:33.911 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:33.907+0000 7fa05bfff640 1 -- 172.21.15.143:0/513192395 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa06c00ab70 con 0x7fa0740b45d0 2023-11-26T07:23:33.911 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:33.907+0000 7fa079972640 1 -- 172.21.15.143:0/513192395 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa0740cbe30 con 0x7fa0740b45d0 2023-11-26T07:23:33.911 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:33.907+0000 7fa079972640 1 -- 172.21.15.143:0/513192395 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa0740cc3f0 con 0x7fa0740b45d0 2023-11-26T07:23:33.911 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:33.907+0000 7fa079972640 1 -- 172.21.15.143:0/513192395 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa0740d3ed0 con 0x7fa0740b45d0 2023-11-26T07:23:33.912 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:33.911+0000 7fa05bfff640 1 -- 172.21.15.143:0/513192395 <== mon.0 v2:172.21.15.143:3300/0 4 ==== mgrmap(e 16) v1 ==== 85954+0+0 (secure 0 0 0) 0x7fa06c00acd0 con 0x7fa0740b45d0 2023-11-26T07:23:33.912 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:33.911+0000 7fa05bfff640 1 --2- 172.21.15.143:0/513192395 >> [v2:172.21.15.188:6836/3401128553,v1:172.21.15.188:6837/3401128553] conn(0x7fa0600665c0 0x7fa060068a80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:23:33.912 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:33.911+0000 7fa05bfff640 1 -- 172.21.15.143:0/513192395 <== mon.0 v2:172.21.15.143:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fa06c0834b0 con 0x7fa0740b45d0 2023-11-26T07:23:33.913 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:33.911+0000 7fa072ffd640 1 --2- 172.21.15.143:0/513192395 >> [v2:172.21.15.188:6836/3401128553,v1:172.21.15.188:6837/3401128553] conn(0x7fa0600665c0 0x7fa060068a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:23:33.913 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:33.911+0000 7fa072ffd640 1 --2- 172.21.15.143:0/513192395 >> [v2:172.21.15.188:6836/3401128553,v1:172.21.15.188:6837/3401128553] conn(0x7fa0600665c0 0x7fa060068a80 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7fa0640b2000 tx=0x7fa064008040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:23:33.916 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:33.915+0000 7fa05bfff640 1 -- 172.21.15.143:0/513192395 <== mon.0 v2:172.21.15.143:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172649 (secure 0 0 0) 0x7fa0740d3ed0 con 0x7fa0740b45d0 2023-11-26T07:23:34.063 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:34.063+0000 7fa079972640 1 -- 172.21.15.143:0/513192395 --> [v2:172.21.15.188:6836/3401128553,v1:172.21.15.188:6837/3401128553] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7fa0740d3ed0 con 0x7fa0600665c0 2023-11-26T07:23:34.071 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:34.071+0000 7fa05bfff640 1 -- 172.21.15.143:0/513192395 <== mgr.4108 v2:172.21.15.188:6836/3401128553 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+289498 (secure 0 0 0) 0x7fa0740d3ed0 con 0x7fa0600665c0 2023-11-26T07:23:34.074 INFO:teuthology.orchestra.run.smithi143.stdout: 2023-11-26T07:23:34.076 INFO:teuthology.orchestra.run.smithi143.stderr:dumped all 2023-11-26T07:23:34.077 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:34.075+0000 7fa079972640 1 -- 172.21.15.143:0/513192395 >> [v2:172.21.15.188:6836/3401128553,v1:172.21.15.188:6837/3401128553] conn(0x7fa0600665c0 msgr2=0x7fa060068a80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:23:34.077 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:34.075+0000 7fa079972640 1 --2- 172.21.15.143:0/513192395 >> [v2:172.21.15.188:6836/3401128553,v1:172.21.15.188:6837/3401128553] conn(0x7fa0600665c0 0x7fa060068a80 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7fa0640b2000 tx=0x7fa064008040 comp rx=0 tx=0).stop 2023-11-26T07:23:34.077 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:34.075+0000 7fa079972640 1 -- 172.21.15.143:0/513192395 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fa0740b45d0 msgr2=0x7fa074157620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:23:34.077 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:34.075+0000 7fa079972640 1 --2- 172.21.15.143:0/513192395 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fa0740b45d0 0x7fa074157620 secure :-1 s=READY pgs=1158 cs=0 l=1 rev1=1 crypto rx=0x7fa06c001570 tx=0x7fa06c00cca0 comp rx=0 tx=0).stop 2023-11-26T07:23:34.077 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:34.075+0000 7fa079972640 1 -- 172.21.15.143:0/513192395 shutdown_connections 2023-11-26T07:23:34.077 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:34.075+0000 7fa079972640 1 --2- 172.21.15.143:0/513192395 >> [v2:172.21.15.188:6836/3401128553,v1:172.21.15.188:6837/3401128553] conn(0x7fa0600665c0 0x7fa060068a80 unknown :-1 s=CLOSED pgs=64 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:23:34.078 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:34.075+0000 7fa079972640 1 --2- 172.21.15.143:0/513192395 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fa0740b45d0 0x7fa074157620 unknown :-1 s=CLOSED pgs=1158 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:23:34.078 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:34.075+0000 7fa079972640 1 --2- 172.21.15.143:0/513192395 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fa0740b4080 0x7fa0741570e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:23:34.078 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:34.075+0000 7fa079972640 1 --2- 172.21.15.143:0/513192395 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fa0740b3610 0x7fa074156a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:23:34.078 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:34.075+0000 7fa079972640 1 -- 172.21.15.143:0/513192395 >> 172.21.15.143:0/513192395 conn(0x7fa0740d3030 msgr2=0x7fa0740cdc40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:23:34.078 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:34.075+0000 7fa079972640 1 -- 172.21.15.143:0/513192395 shutdown_connections 2023-11-26T07:23:34.078 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:34.075+0000 7fa079972640 1 -- 172.21.15.143:0/513192395 wait complete. 2023-11-26T07:23:34.094 INFO:teuthology.orchestra.run.smithi143.stdout:{"pg_ready":true,"pg_map":{"version":27,"stamp":"2023-11-26T07:23:33.288565+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592778,"num_objects":23,"num_object_clones":0,"num_object_copies":46,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":23,"num_whiteouts":0,"num_read":212,"num_read_kb":306,"num_write":223,"num_write_kb":2430,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":166,"ondisk_log_size":166,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":189,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":254596,"kb_used_data":9288,"kb_used_omap":12,"kb_used_meta":245171,"kb_avail":754720124,"statfs":{"total":773094113280,"available":772833406976,"internally_reserved":0,"allocated":9510912,"data_stored":4739068,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12749,"internal_metadata":251055667},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":42,"apply_latency_ms":42,"commit_latency_ns":42000000,"apply_latency_ns":42000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2410,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"5.670150"},"pg_stats":[{"pgid":"13.3f","version":"0'0","reported_seq":11,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-11-26T07:23:30.644611+0000","last_change":"2023-11-26T07:23:29.664581+0000","last_active":"2023-11-26T07:23:30.644611+0000","last_peered":"2023-11-26T07:23:30.644611+0000","last_clean":"2023-11-26T07:23:30.644611+0000","last_became_active":"2023-11-26T07:23:29.661507+0000","last_became_peered":"2023-11-26T07:23:29.661507+0000","last_unstale":"2023-11-26T07:23:30.644611+0000","last_undegraded":"2023-11-26T07:23:30.644611+0000","last_fullsized":"2023-11-26T07:23:30.644611+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:23:28.619155+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:23:28.619155+0000","last_clean_scrub_stamp":"2023-11-26T07:23:28.619155+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T11:28:13.301129+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.3e","version":"0'0","reported_seq":11,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-11-26T07:23:31.156930+0000","last_change":"2023-11-26T07:23:29.678054+0000","last_active":"2023-11-26T07:23:31.156930+0000","last_peered":"2023-11-26T07:23:31.156930+0000","last_clean":"2023-11-26T07:23:31.156930+0000","last_became_active":"2023-11-26T07:23:29.677694+0000","last_became_peered":"2023-11-26T07:23:29.677694+0000","last_unstale":"2023-11-26T07:23:31.156930+0000","last_undegraded":"2023-11-26T07:23:31.156930+0000","last_fullsized":"2023-11-26T07:23:31.156930+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:23:28.619155+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:23:28.619155+0000","last_clean_scrub_stamp":"2023-11-26T07:23:28.619155+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T17:13:41.724273+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.3d","version":"0'0","reported_seq":11,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-11-26T07:23:31.154692+0000","last_change":"2023-11-26T07:23:29.654061+0000","last_active":"2023-11-26T07:23:31.154692+0000","last_peered":"2023-11-26T07:23:31.154692+0000","last_clean":"2023-11-26T07:23:31.154692+0000","last_became_active":"2023-11-26T07:23:29.653070+0000","last_became_peered":"2023-11-26T07:23:29.653070+0000","last_unstale":"2023-11-26T07:23:31.154692+0000","last_undegraded":"2023-11-26T07:23:31.154692+0000","last_fullsized":"2023-11-26T07:23:31.154692+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:23:28.619155+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:23:28.619155+0000","last_clean_scrub_stamp":"2023-11-26T07:23:28.619155+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T09:04:09.764282+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"13.3c","version":"0'0","reported_seq":11,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-11-26T07:23:30.643254+0000","last_change":"2023-11-26T07:23:29.664735+0000","last_active":"2023-11-26T07:23:30.643254+0000","last_peered":"2023-11-26T07:23:30.643254+0000","last_clean":"2023-11-26T07:23:30.643254+0000","last_became_active":"2023-11-26T07:23:29.664220+0000","last_became_peered":"2023-11-26T07:23:29.664220+0000","last_unstale":"2023-11-26T07:23:30.643254+0000","last_undegraded":"2023-11-26T07:23:30.643254+0000","last_fullsized":"2023-11-26T07:23:30.643254+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:23:28.619155+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:23:28.619155+0000","last_clean_scrub_stamp":"2023-11-26T07:23:28.619155+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T16:05:51.826873+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"13.3b","version":"0'0","reported_seq":11,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-11-26T07:23:31.153983+0000","last_change":"2023-11-26T07:23:29.673766+0000","last_active":"2023-11-26T07:23:31.153983+0000","last_peered":"2023-11-26T07:23:31.153983+0000","last_clean":"2023-11-26T07:23:31.153983+0000","last_became_active":"2023-11-26T07:23:29.670876+0000","last_became_peered":"2023-11-26T07:23:29.670876+0000","last_unstale":"2023-11-26T07:23:31.153983+0000","last_undegraded":"2023-11-26T07:23:31.153983+0000","last_fullsized":"2023-11-26T07:23:31.153983+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:23:28.619155+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:23:28.619155+0000","last_clean_scrub_stamp":"2023-11-26T07:23:28.619155+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T09:05:12.255253+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"13.3a","version":"0'0","reported_seq":11,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-11-26T07:23:30.688618+0000","last_change":"2023-11-26T07:23:29.666579+0000","last_active":"2023-11-26T07:23:30.688618+0000","last_peered":"2023-11-26T07:23:30.688618+0000","last_clean":"2023-11-26T07:23:30.688618+0000","last_became_active":"2023-11-26T07:23:29.666318+0000","last_became_peered":"2023-11-26T07:23:29.666318+0000","last_unstale":"2023-11-26T07:23:30.688618+0000","last_undegraded":"2023-11-26T07:23:30.688618+0000","last_fullsized":"2023-11-26T07:23:30.688618+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:23:28.619155+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:23:28.619155+0000","last_clean_scrub_stamp":"2023-11-26T07:23:28.619155+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T10:07:49.922280+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"13.39","version":"0'0","reported_seq":11,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-11-26T07:23:30.643204+0000","last_change":"2023-11-26T07:23:29.662660+0000","last_active":"2023-11-26T07:23:30.643204+0000","last_peered":"2023-11-26T07:23:30.643204+0000","last_clean":"2023-11-26T07:23:30.643204+0000","last_became_active":"2023-11-26T07:23:29.662287+0000","last_became_peered":"2023-11-26T07:23:29.662287+0000","last_unstale":"2023-11-26T07:23:30.643204+0000","last_undegraded":"2023-11-26T07:23:30.643204+0000","last_fullsized":"2023-11-26T07:23:30.643204+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:23:28.619155+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:23:28.619155+0000","last_clean_scrub_stamp":"2023-11-26T07:23:28.619155+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T07:56:03.491313+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"13.38","version":"0'0","reported_seq":11,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-11-26T07:23:30.688623+0000","last_change":"2023-11-26T07:23:29.660746+0000","last_active":"2023-11-26T07:23:30.688623+0000","last_peered":"2023-11-26T07:23:30.688623+0000","last_clean":"2023-11-26T07:23:30.688623+0000","last_became_active":"2023-11-26T07:23:29.660479+0000","last_became_peered":"2023-11-26T07:23:29.660479+0000","last_unstale":"2023-11-26T07:23:30.688623+0000","last_undegraded":"2023-11-26T07:23:30.688623+0000","last_fullsized":"2023-11-26T07:23:30.688623+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:23:28.619155+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:23:28.619155+0000","last_clean_scrub_stamp":"2023-11-26T07:23:28.619155+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T09:25:04.676044+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"13.37","version":"0'0","reported_seq":11,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-11-26T07:23:30.688912+0000","last_change":"2023-11-26T07:23:29.660637+0000","last_active":"2023-11-26T07:23:30.688912+0000","last_peered":"2023-11-26T07:23:30.688912+0000","last_clean":"2023-11-26T07:23:30.688912+0000","last_became_active":"2023-11-26T07:23:29.660358+0000","last_became_peered":"2023-11-26T07:23:29.660358+0000","last_unstale":"2023-11-26T07:23:30.688912+0000","last_undegraded":"2023-11-26T07:23:30.688912+0000","last_fullsized":"2023-11-26T07:23:30.688912+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:23:28.619155+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:23:28.619155+0000","last_clean_scrub_stamp":"2023-11-26T07:23:28.619155+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T16:10:09.119282+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"13.36","version":"0'0","reported_seq":11,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-11-26T07:23:30.644543+0000","last_change":"2023-11-26T07:23:29.660073+0000","last_active":"2023-11-26T07:23:30.644543+0000","last_peered":"2023-11-26T07:23:30.644543+0000","last_clean":"2023-11-26T07:23:30.644543+0000","last_became_active":"2023-11-26T07:23:29.659776+0000","last_became_peered":"2023-11-26T07:23:29.659776+0000","last_unstale":"2023-11-26T07:23:30.644543+0000","last_undegraded":"2023-11-26T07:23:30.644543+0000","last_fullsized":"2023-11-26T07:23:30.644543+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:23:28.619155+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:23:28.619155+0000","last_clean_scrub_stamp":"2023-11-26T07:23:28.619155+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T13:34:20.128647+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.35","version":"0'0","reported_seq":11,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-11-26T07:23:31.156914+0000","last_change":"2023-11-26T07:23:29.664297+0000","last_active":"2023-11-26T07:23:31.156914+0000","last_peered":"2023-11-26T07:23:31.156914+0000","last_clean":"2023-11-26T07:23:31.156914+0000","last_became_active":"2023-11-26T07:23:29.663421+0000","last_became_peered":"2023-11-26T07:23:29.663421+0000","last_unstale":"2023-11-26T07:23:31.156914+0000","last_undegraded":"2023-11-26T07:23:31.156914+0000","last_fullsized":"2023-11-26T07:23:31.156914+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:23:28.619155+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:23:28.619155+0000","last_clean_scrub_stamp":"2023-11-26T07:23:28.619155+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T09:12:34.800308+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.34","version":"0'0","reported_seq":11,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-11-26T07:23:30.644600+0000","last_change":"2023-11-26T07:23:29.659988+0000","last_active":"2023-11-26T07:23:30.644600+0000","last_peered":"2023-11-26T07:23:30.644600+0000","last_clean":"2023-11-26T07:23:30.644600+0000","last_became_active":"2023-11-26T07:23:29.659673+0000","last_became_peered":"2023-11-26T07:23:29.659673+0000","last_unstale":"2023-11-26T07:23:30.644600+0000","last_undegraded":"2023-11-26T07:23:30.644600+0000","last_fullsized":"2023-11-26T07:23:30.644600+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:23:28.619155+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:23:28.619155+0000","last_clean_scrub_stamp":"2023-11-26T07:23:28.619155+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T15:32:15.823182+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.33","version":"0'0","reported_seq":11,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-11-26T07:23:31.154242+0000","last_change":"2023-11-26T07:23:29.662429+0000","last_active":"2023-11-26T07:23:31.154242+0000","last_peered":"2023-11-26T07:23:31.154242+0000","last_clean":"2023-11-26T07:23:31.154242+0000","last_became_active":"2023-11-26T07:23:29.662131+0000","last_became_peered":"2023-11-26T07:23:29.662131+0000","last_unstale":"2023-11-26T07:23:31.154242+0000","last_undegraded":"2023-11-26T07:23:31.154242+0000","last_fullsized":"2023-11-26T07:23:31.154242+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:23:28.619155+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:23:28.619155+0000","last_clean_scrub_stamp":"2023-11-26T07:23:28.619155+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T17:06:44.573286+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"13.32","version":"0'0","reported_seq":11,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-11-26T07:23:30.642938+0000","last_change":"2023-11-26T07:23:29.666985+0000","last_active":"2023-11-26T07:23:30.642938+0000","last_peered":"2023-11-26T07:23:30.642938+0000","last_clean":"2023-11-26T07:23:30.642938+0000","last_became_active":"2023-11-26T07:23:29.666656+0000","last_became_peered":"2023-11-26T07:23:29.666656+0000","last_unstale":"2023-11-26T07:23:30.642938+0000","last_undegraded":"2023-11-26T07:23:30.642938+0000","last_fullsized":"2023-11-26T07:23:30.642938+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:23:28.619155+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:23:28.619155+0000","last_clean_scrub_stamp":"2023-11-26T07:23:28.619155+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T11:10:41.460364+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"13.31","version":"0'0","reported_seq":11,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-11-26T07:23:31.155303+0000","last_change":"2023-11-26T07:23:29.646803+0000","last_active":"2023-11-26T07:23:31.155303+0000","last_peered":"2023-11-26T07:23:31.155303+0000","last_clean":"2023-11-26T07:23:31.155303+0000","last_became_active":"2023-11-26T07:23:29.646514+0000","last_became_peered":"2023-11-26T07:23:29.646514+0000","last_unstale":"2023-11-26T07:23:31.155303+0000","last_undegraded":"2023-11-26T07:23:31.155303+0000","last_fullsized":"2023-11-26T07:23:31.155303+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:23:28.619155+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:23:28.619155+0000","last_clean_scrub_stamp":"2023-11-26T07:23:28.619155+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T17:06:44.573286+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"13.30","version":"0'0","reported_seq":11,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-11-26T07:23:31.153951+0000","last_change":"2023-11-26T07:23:29.672255+0000","last_active":"2023-11-26T07:23:31.153951+0000","last_peered":"2023-11-26T07:23:31.153951+0000","last_clean":"2023-11-26T07:23:31.153951+0000","last_became_active":"2023-11-26T07:23:29.671965+0000","last_became_peered":"2023-11-26T07:23:29.671965+0000","last_unstale":"2023-11-26T07:23:31.153951+0000","last_undegraded":"2023-11-26T07:23:31.153951+0000","last_fullsized":"2023-11-26T07:23:31.153951+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:23:28.619155+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:23:28.619155+0000","last_clean_scrub_stamp":"2023-11-26T07:23:28.619155+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T13:22:18.189493+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"13.2f","version":"0'0","reported_seq":11,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-11-26T07:23:31.154225+0000","last_change":"2023-11-26T07:23:29.667334+0000","last_active":"2023-11-26T07:23:31.154225+0000","last_peered":"2023-11-26T07:23:31.154225+0000","last_clean":"2023-11-26T07:23:31.154225+0000","last_became_active":"2023-11-26T07:23:29.666606+0000","last_became_peered":"2023-11-26T07:23:29.666606+0000","last_unstale":"2023-11-26T07:23:31.154225+0000","last_undegraded":"2023-11-26T07:23:31.154225+0000","last_fullsized":"2023-11-26T07:23:31.154225+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:23:28.619155+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:23:28.619155+0000","last_clean_scrub_stamp":"2023-11-26T07:23:28.619155+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T16:10:09.119282+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"13.2e","version":"0'0","reported_seq":11,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-11-26T07:23:30.642909+0000","last_change":"2023-11-26T07:23:29.665013+0000","last_active":"2023-11-26T07:23:30.642909+0000","last_peered":"2023-11-26T07:23:30.642909+0000","last_clean":"2023-11-26T07:23:30.642909+0000","last_became_active":"2023-11-26T07:23:29.663771+0000","last_became_peered":"2023-11-26T07:23:29.663771+0000","last_unstale":"2023-11-26T07:23:30.642909+0000","last_undegraded":"2023-11-26T07:23:30.642909+0000","last_fullsized":"2023-11-26T07:23:30.642909+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:23:28.619155+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:23:28.619155+0000","last_clean_scrub_stamp":"2023-11-26T07:23:28.619155+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T08:06:41.980749+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"13.2d","version":"0'0","reported_seq":11,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-11-26T07:23:31.157149+0000","last_change":"2023-11-26T07:23:29.652309+0000","last_active":"2023-11-26T07:23:31.157149+0000","last_peered":"2023-11-26T07:23:31.157149+0000","last_clean":"2023-11-26T07:23:31.157149+0000","last_became_active":"2023-11-26T07:23:29.651997+0000","last_became_peered":"2023-11-26T07:23:29.651997+0000","last_unstale":"2023-11-26T07:23:31.157149+0000","last_undegraded":"2023-11-26T07:23:31.157149+0000","last_fullsized":"2023-11-26T07:23:31.157149+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:23:28.619155+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:23:28.619155+0000","last_clean_scrub_stamp":"2023-11-26T07:23:28.619155+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T11:31:08.516775+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.2c","version":"0'0","reported_seq":11,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-11-26T07:23:30.643382+0000","last_change":"2023-11-26T07:23:29.664507+0000","last_active":"2023-11-26T07:23:30.643382+0000","last_peered":"2023-11-26T07:23:30.643382+0000","last_clean":"2023-11-26T07:23:30.643382+0000","last_became_active":"2023-11-26T07:23:29.663927+0000","last_became_peered":"2023-11-26T07:23:29.663927+0000","last_unstale":"2023-11-26T07:23:30.643382+0000","last_undegraded":"2023-11-26T07:23:30.643382+0000","last_fullsized":"2023-11-26T07:23:30.643382+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:23:28.619155+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:23:28.619155+0000","last_clean_scrub_stamp":"2023-11-26T07:23:28.619155+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T12:03:35.864250+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"13.2b","version":"0'0","reported_seq":11,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-11-26T07:23:30.642837+0000","last_change":"2023-11-26T07:23:29.665728+0000","last_active":"2023-11-26T07:23:30.642837+0000","last_peered":"2023-11-26T07:23:30.642837+0000","last_clean":"2023-11-26T07:23:30.642837+0000","last_became_active":"2023-11-26T07:23:29.665433+0000","last_became_peered":"2023-11-26T07:23:29.665433+0000","last_unstale":"2023-11-26T07:23:30.642837+0000","last_undegraded":"2023-11-26T07:23:30.642837+0000","last_fullsized":"2023-11-26T07:23:30.642837+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:23:28.619155+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:23:28.619155+0000","last_clean_scrub_stamp":"2023-11-26T07:23:28.619155+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T08:18:30.568517+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"13.2a","version":"0'0","reported_seq":11,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-11-26T07:23:31.156298+0000","last_change":"2023-11-26T07:23:29.672791+0000","last_active":"2023-11-26T07:23:31.156298+0000","last_peered":"2023-11-26T07:23:31.156298+0000","last_clean":"2023-11-26T07:23:31.156298+0000","last_became_active":"2023-11-26T07:23:29.672520+0000","last_became_peered":"2023-11-26T07:23:29.672520+0000","last_unstale":"2023-11-26T07:23:31.156298+0000","last_undegraded":"2023-11-26T07:23:31.156298+0000","last_fullsized":"2023-11-26T07:23:31.156298+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:23:28.619155+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:23:28.619155+0000","last_clean_scrub_stamp":"2023-11-26T07:23:28.619155+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T14:22:16.614334+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.29","version":"0'0","reported_seq":11,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-11-26T07:23:30.643988+0000","last_change":"2023-11-26T07:23:29.658265+0000","last_active":"2023-11-26T07:23:30.643988+0000","last_peered":"2023-11-26T07:23:30.643988+0000","last_clean":"2023-11-26T07:23:30.643988+0000","last_became_active":"2023-11-26T07:23:29.657987+0000","last_became_peered":"2023-11-26T07:23:29.657987+0000","last_unstale":"2023-11-26T07:23:30.643988+0000","last_undegraded":"2023-11-26T07:23:30.643988+0000","last_fullsized":"2023-11-26T07:23:30.643988+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:23:28.619155+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:23:28.619155+0000","last_clean_scrub_stamp":"2023-11-26T07:23:28.619155+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T08:36:27.045499+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.28","version":"0'0","reported_seq":11,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-11-26T07:23:30.642665+0000","last_change":"2023-11-26T07:23:29.662145+0000","last_active":"2023-11-26T07:23:30.642665+0000","last_peered":"2023-11-26T07:23:30.642665+0000","last_clean":"2023-11-26T07:23:30.642665+0000","last_became_active":"2023-11-26T07:23:29.661765+0000","last_became_peered":"2023-11-26T07:23:29.661765+0000","last_unstale":"2023-11-26T07:23:30.642665+0000","last_undegraded":"2023-11-26T07:23:30.642665+0000","last_fullsized":"2023-11-26T07:23:30.642665+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:23:28.619155+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:23:28.619155+0000","last_clean_scrub_stamp":"2023-11-26T07:23:28.619155+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T10:26:33.963940+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"13.27","version":"0'0","reported_seq":11,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-11-26T07:23:31.156627+0000","last_change":"2023-11-26T07:23:29.670931+0000","last_active":"2023-11-26T07:23:31.156627+0000","last_peered":"2023-11-26T07:23:31.156627+0000","last_clean":"2023-11-26T07:23:31.156627+0000","last_became_active":"2023-11-26T07:23:29.669176+0000","last_became_peered":"2023-11-26T07:23:29.669176+0000","last_unstale":"2023-11-26T07:23:31.156627+0000","last_undegraded":"2023-11-26T07:23:31.156627+0000","last_fullsized":"2023-11-26T07:23:31.156627+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:23:28.619155+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:23:28.619155+0000","last_clean_scrub_stamp":"2023-11-26T07:23:28.619155+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T15:11:46.786215+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.26","version":"0'0","reported_seq":11,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-11-26T07:23:31.154789+0000","last_change":"2023-11-26T07:23:29.659262+0000","last_active":"2023-11-26T07:23:31.154789+0000","last_peered":"2023-11-26T07:23:31.154789+0000","last_clean":"2023-11-26T07:23:31.154789+0000","last_became_active":"2023-11-26T07:23:29.658907+0000","last_became_peered":"2023-11-26T07:23:29.658907+0000","last_unstale":"2023-11-26T07:23:31.154789+0000","last_undegraded":"2023-11-26T07:23:31.154789+0000","last_fullsized":"2023-11-26T07:23:31.154789+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:23:28.619155+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:23:28.619155+0000","last_clean_scrub_stamp":"2023-11-26T07:23:28.619155+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T07:37:04.923479+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"13.25","version":"0'0","reported_seq":11,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-11-26T07:23:31.154276+0000","last_change":"2023-11-26T07:23:29.662490+0000","last_active":"2023-11-26T07:23:31.154276+0000","last_peered":"2023-11-26T07:23:31.154276+0000","last_clean":"2023-11-26T07:23:31.154276+0000","last_became_active":"2023-11-26T07:23:29.660537+0000","last_became_peered":"2023-11-26T07:23:29.660537+0000","last_unstale":"2023-11-26T07:23:31.154276+0000","last_undegraded":"2023-11-26T07:23:31.154276+0000","last_fullsized":"2023-11-26T07:23:31.154276+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:23:28.619155+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:23:28.619155+0000","last_clean_scrub_stamp":"2023-11-26T07:23:28.619155+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T13:24:22.782822+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"13.24","version":"0'0","reported_seq":11,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-11-26T07:23:30.644373+0000","last_change":"2023-11-26T07:23:29.664641+0000","last_active":"2023-11-26T07:23:30.644373+0000","last_peered":"2023-11-26T07:23:30.644373+0000","last_clean":"2023-11-26T07:23:30.644373+0000","last_became_active":"2023-11-26T07:23:29.664308+0000","last_became_peered":"2023-11-26T07:23:29.664308+0000","last_unstale":"2023-11-26T07:23:30.644373+0000","last_undegraded":"2023-11-26T07:23:30.644373+0000","last_fullsized":"2023-11-26T07:23:30.644373+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:23:28.619155+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:23:28.619155+0000","last_clean_scrub_stamp":"2023-11-26T07:23:28.619155+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T15:42:52.859136+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.23","version":"0'0","reported_seq":11,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-11-26T07:23:31.156370+0000","last_change":"2023-11-26T07:23:29.661733+0000","last_active":"2023-11-26T07:23:31.156370+0000","last_peered":"2023-11-26T07:23:31.156370+0000","last_clean":"2023-11-26T07:23:31.156370+0000","last_became_active":"2023-11-26T07:23:29.660774+0000","last_became_peered":"2023-11-26T07:23:29.660774+0000","last_unstale":"2023-11-26T07:23:31.156370+0000","last_undegraded":"2023-11-26T07:23:31.156370+0000","last_fullsized":"2023-11-26T07:23:31.156370+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:23:28.619155+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:23:28.619155+0000","last_clean_scrub_stamp":"2023-11-26T07:23:28.619155+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T12:00:16.132422+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.22","version":"0'0","reported_seq":11,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-11-26T07:23:31.156349+0000","last_change":"2023-11-26T07:23:29.670488+0000","last_active":"2023-11-26T07:23:31.156349+0000","last_peered":"2023-11-26T07:23:31.156349+0000","last_clean":"2023-11-26T07:23:31.156349+0000","last_became_active":"2023-11-26T07:23:29.670007+0000","last_became_peered":"2023-11-26T07:23:29.670007+0000","last_unstale":"2023-11-26T07:23:31.156349+0000","last_undegraded":"2023-11-26T07:23:31.156349+0000","last_fullsized":"2023-11-26T07:23:31.156349+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:23:28.619155+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:23:28.619155+0000","last_clean_scrub_stamp":"2023-11-26T07:23:28.619155+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T14:16:44.836853+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.21","version":"0'0","reported_seq":11,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-11-26T07:23:31.154532+0000","last_change":"2023-11-26T07:23:29.658642+0000","last_active":"2023-11-26T07:23:31.154532+0000","last_peered":"2023-11-26T07:23:31.154532+0000","last_clean":"2023-11-26T07:23:31.154532+0000","last_became_active":"2023-11-26T07:23:29.658320+0000","last_became_peered":"2023-11-26T07:23:29.658320+0000","last_unstale":"2023-11-26T07:23:31.154532+0000","last_undegraded":"2023-11-26T07:23:31.154532+0000","last_fullsized":"2023-11-26T07:23:31.154532+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:23:28.619155+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:23:28.619155+0000","last_clean_scrub_stamp":"2023-11-26T07:23:28.619155+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T08:36:27.045499+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"13.20","version":"0'0","reported_seq":11,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-11-26T07:23:30.643958+0000","last_change":"2023-11-26T07:23:29.660914+0000","last_active":"2023-11-26T07:23:30.643958+0000","last_peered":"2023-11-26T07:23:30.643958+0000","last_clean":"2023-11-26T07:23:30.643958+0000","last_became_active":"2023-11-26T07:23:29.660636+0000","last_became_peered":"2023-11-26T07:23:29.660636+0000","last_unstale":"2023-11-26T07:23:30.643958+0000","last_undegraded":"2023-11-26T07:23:30.643958+0000","last_fullsized":"2023-11-26T07:23:30.643958+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:23:28.619155+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:23:28.619155+0000","last_clean_scrub_stamp":"2023-11-26T07:23:28.619155+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T17:48:40.805546+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.1e","version":"0'0","reported_seq":11,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-11-26T07:23:31.154262+0000","last_change":"2023-11-26T07:23:29.669100+0000","last_active":"2023-11-26T07:23:31.154262+0000","last_peered":"2023-11-26T07:23:31.154262+0000","last_clean":"2023-11-26T07:23:31.154262+0000","last_became_active":"2023-11-26T07:23:29.668786+0000","last_became_peered":"2023-11-26T07:23:29.668786+0000","last_unstale":"2023-11-26T07:23:31.154262+0000","last_undegraded":"2023-11-26T07:23:31.154262+0000","last_fullsized":"2023-11-26T07:23:31.154262+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:23:28.619155+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:23:28.619155+0000","last_clean_scrub_stamp":"2023-11-26T07:23:28.619155+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T08:26:22.625844+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"12.1f","version":"83'4","reported_seq":16,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-11-26T07:23:30.644694+0000","last_change":"2023-11-26T07:23:28.646168+0000","last_active":"2023-11-26T07:23:30.644694+0000","last_peered":"2023-11-26T07:23:30.644694+0000","last_clean":"2023-11-26T07:23:30.644694+0000","last_became_active":"2023-11-26T07:23:28.645841+0000","last_became_peered":"2023-11-26T07:23:28.645841+0000","last_unstale":"2023-11-26T07:23:30.644694+0000","last_undegraded":"2023-11-26T07:23:30.644694+0000","last_fullsized":"2023-11-26T07:23:30.644694+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:23:27.611790+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:23:27.611790+0000","last_clean_scrub_stamp":"2023-11-26T07:23:27.611790+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T14:42:49.850606+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.1f","version":"0'0","reported_seq":11,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-11-26T07:23:30.644581+0000","last_change":"2023-11-26T07:23:29.661788+0000","last_active":"2023-11-26T07:23:30.644581+0000","last_peered":"2023-11-26T07:23:30.644581+0000","last_clean":"2023-11-26T07:23:30.644581+0000","last_became_active":"2023-11-26T07:23:29.661123+0000","last_became_peered":"2023-11-26T07:23:29.661123+0000","last_unstale":"2023-11-26T07:23:30.644581+0000","last_undegraded":"2023-11-26T07:23:30.644581+0000","last_fullsized":"2023-11-26T07:23:30.644581+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:23:28.619155+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:23:28.619155+0000","last_clean_scrub_stamp":"2023-11-26T07:23:28.619155+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T10:41:40.854294+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"12.1e","version":"83'1","reported_seq":13,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-11-26T07:23:31.154415+0000","last_change":"2023-11-26T07:23:28.644249+0000","last_active":"2023-11-26T07:23:31.154415+0000","last_peered":"2023-11-26T07:23:31.154415+0000","last_clean":"2023-11-26T07:23:31.154415+0000","last_became_active":"2023-11-26T07:23:28.643393+0000","last_became_peered":"2023-11-26T07:23:28.643393+0000","last_unstale":"2023-11-26T07:23:31.154415+0000","last_undegraded":"2023-11-26T07:23:31.154415+0000","last_fullsized":"2023-11-26T07:23:31.154415+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:23:27.611790+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:23:27.611790+0000","last_clean_scrub_stamp":"2023-11-26T07:23:27.611790+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T12:43:28.486650+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"13.1c","version":"0'0","reported_seq":11,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-11-26T07:23:30.644530+0000","last_change":"2023-11-26T07:23:29.661340+0000","last_active":"2023-11-26T07:23:30.644530+0000","last_peered":"2023-11-26T07:23:30.644530+0000","last_clean":"2023-11-26T07:23:30.644530+0000","last_became_active":"2023-11-26T07:23:29.660953+0000","last_became_peered":"2023-11-26T07:23:29.660953+0000","last_unstale":"2023-11-26T07:23:30.644530+0000","last_undegraded":"2023-11-26T07:23:30.644530+0000","last_fullsized":"2023-11-26T07:23:30.644530+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:23:28.619155+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:23:28.619155+0000","last_clean_scrub_stamp":"2023-11-26T07:23:28.619155+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T16:34:57.083451+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"12.1d","version":"83'2","reported_seq":14,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-11-26T07:23:31.157239+0000","last_change":"2023-11-26T07:23:28.646975+0000","last_active":"2023-11-26T07:23:31.157239+0000","last_peered":"2023-11-26T07:23:31.157239+0000","last_clean":"2023-11-26T07:23:31.157239+0000","last_became_active":"2023-11-26T07:23:28.646704+0000","last_became_peered":"2023-11-26T07:23:28.646704+0000","last_unstale":"2023-11-26T07:23:31.157239+0000","last_undegraded":"2023-11-26T07:23:31.157239+0000","last_fullsized":"2023-11-26T07:23:31.157239+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:23:27.611790+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:23:27.611790+0000","last_clean_scrub_stamp":"2023-11-26T07:23:27.611790+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T16:34:56.076085+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.1d","version":"0'0","reported_seq":11,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-11-26T07:23:31.154595+0000","last_change":"2023-11-26T07:23:29.663595+0000","last_active":"2023-11-26T07:23:31.154595+0000","last_peered":"2023-11-26T07:23:31.154595+0000","last_clean":"2023-11-26T07:23:31.154595+0000","last_became_active":"2023-11-26T07:23:29.663320+0000","last_became_peered":"2023-11-26T07:23:29.663320+0000","last_unstale":"2023-11-26T07:23:31.154595+0000","last_undegraded":"2023-11-26T07:23:31.154595+0000","last_fullsized":"2023-11-26T07:23:31.154595+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:23:28.619155+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:23:28.619155+0000","last_clean_scrub_stamp":"2023-11-26T07:23:28.619155+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T17:23:44.263813+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.1c","version":"83'2","reported_seq":14,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-11-26T07:23:31.156901+0000","last_change":"2023-11-26T07:23:28.646394+0000","last_active":"2023-11-26T07:23:31.156901+0000","last_peered":"2023-11-26T07:23:31.156901+0000","last_clean":"2023-11-26T07:23:31.156901+0000","last_became_active":"2023-11-26T07:23:28.646082+0000","last_became_peered":"2023-11-26T07:23:28.646082+0000","last_unstale":"2023-11-26T07:23:31.156901+0000","last_undegraded":"2023-11-26T07:23:31.156901+0000","last_fullsized":"2023-11-26T07:23:31.156901+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:23:27.611790+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:23:27.611790+0000","last_clean_scrub_stamp":"2023-11-26T07:23:27.611790+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T10:41:39.846929+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.1a","version":"0'0","reported_seq":11,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-11-26T07:23:31.156433+0000","last_change":"2023-11-26T07:23:29.670275+0000","last_active":"2023-11-26T07:23:31.156433+0000","last_peered":"2023-11-26T07:23:31.156433+0000","last_clean":"2023-11-26T07:23:31.156433+0000","last_became_active":"2023-11-26T07:23:29.669689+0000","last_became_peered":"2023-11-26T07:23:29.669689+0000","last_unstale":"2023-11-26T07:23:31.156433+0000","last_undegraded":"2023-11-26T07:23:31.156433+0000","last_fullsized":"2023-11-26T07:23:31.156433+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:23:28.619155+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:23:28.619155+0000","last_clean_scrub_stamp":"2023-11-26T07:23:28.619155+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T17:25:21.959119+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"12.1b","version":"83'1","reported_seq":13,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-11-26T07:23:31.154302+0000","last_change":"2023-11-26T07:23:28.652973+0000","last_active":"2023-11-26T07:23:31.154302+0000","last_peered":"2023-11-26T07:23:31.154302+0000","last_clean":"2023-11-26T07:23:31.154302+0000","last_became_active":"2023-11-26T07:23:28.652601+0000","last_became_peered":"2023-11-26T07:23:28.652601+0000","last_unstale":"2023-11-26T07:23:31.154302+0000","last_undegraded":"2023-11-26T07:23:31.154302+0000","last_fullsized":"2023-11-26T07:23:31.154302+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:23:27.611790+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:23:27.611790+0000","last_clean_scrub_stamp":"2023-11-26T07:23:27.611790+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T10:07:48.914914+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"13.1b","version":"0'0","reported_seq":11,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-11-26T07:23:31.154192+0000","last_change":"2023-11-26T07:23:29.668155+0000","last_active":"2023-11-26T07:23:31.154192+0000","last_peered":"2023-11-26T07:23:31.154192+0000","last_clean":"2023-11-26T07:23:31.154192+0000","last_became_active":"2023-11-26T07:23:29.667819+0000","last_became_peered":"2023-11-26T07:23:29.667819+0000","last_unstale":"2023-11-26T07:23:31.154192+0000","last_undegraded":"2023-11-26T07:23:31.154192+0000","last_fullsized":"2023-11-26T07:23:31.154192+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:23:28.619155+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:23:28.619155+0000","last_clean_scrub_stamp":"2023-11-26T07:23:28.619155+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T10:15:59.819723+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"12.1a","version":"83'1","reported_seq":13,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-11-26T07:23:31.154016+0000","last_change":"2023-11-26T07:23:28.644425+0000","last_active":"2023-11-26T07:23:31.154016+0000","last_peered":"2023-11-26T07:23:31.154016+0000","last_clean":"2023-11-26T07:23:31.154016+0000","last_became_active":"2023-11-26T07:23:28.643691+0000","last_became_peered":"2023-11-26T07:23:28.643691+0000","last_unstale":"2023-11-26T07:23:31.154016+0000","last_undegraded":"2023-11-26T07:23:31.154016+0000","last_fullsized":"2023-11-26T07:23:31.154016+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:23:27.611790+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:23:27.611790+0000","last_clean_scrub_stamp":"2023-11-26T07:23:27.611790+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T09:53:47.961982+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"13.18","version":"0'0","reported_seq":11,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-11-26T07:23:31.153623+0000","last_change":"2023-11-26T07:23:29.649769+0000","last_active":"2023-11-26T07:23:31.153623+0000","last_peered":"2023-11-26T07:23:31.153623+0000","last_clean":"2023-11-26T07:23:31.153623+0000","last_became_active":"2023-11-26T07:23:29.649522+0000","last_became_peered":"2023-11-26T07:23:29.649522+0000","last_unstale":"2023-11-26T07:23:31.153623+0000","last_undegraded":"2023-11-26T07:23:31.153623+0000","last_fullsized":"2023-11-26T07:23:31.153623+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:23:28.619155+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:23:28.619155+0000","last_clean_scrub_stamp":"2023-11-26T07:23:28.619155+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T19:16:31.468189+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.19","version":"0'0","reported_seq":12,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-11-26T07:23:31.156313+0000","last_change":"2023-11-26T07:23:28.645338+0000","last_active":"2023-11-26T07:23:31.156313+0000","last_peered":"2023-11-26T07:23:31.156313+0000","last_clean":"2023-11-26T07:23:31.156313+0000","last_became_active":"2023-11-26T07:23:28.644860+0000","last_became_peered":"2023-11-26T07:23:28.644860+0000","last_unstale":"2023-11-26T07:23:31.156313+0000","last_undegraded":"2023-11-26T07:23:31.156313+0000","last_fullsized":"2023-11-26T07:23:31.156313+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:23:27.611790+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:23:27.611790+0000","last_clean_scrub_stamp":"2023-11-26T07:23:27.611790+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T15:32:14.815817+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.19","version":"0'0","reported_seq":11,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-11-26T07:23:31.155455+0000","last_change":"2023-11-26T07:23:29.646796+0000","last_active":"2023-11-26T07:23:31.155455+0000","last_peered":"2023-11-26T07:23:31.155455+0000","last_clean":"2023-11-26T07:23:31.155455+0000","last_became_active":"2023-11-26T07:23:29.646505+0000","last_became_peered":"2023-11-26T07:23:29.646505+0000","last_unstale":"2023-11-26T07:23:31.155455+0000","last_undegraded":"2023-11-26T07:23:31.155455+0000","last_fullsized":"2023-11-26T07:23:31.155455+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:23:28.619155+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:23:28.619155+0000","last_clean_scrub_stamp":"2023-11-26T07:23:28.619155+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T17:23:44.263813+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"12.18","version":"0'0","reported_seq":12,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-11-26T07:23:31.156022+0000","last_change":"2023-11-26T07:23:28.648143+0000","last_active":"2023-11-26T07:23:31.156022+0000","last_peered":"2023-11-26T07:23:31.156022+0000","last_clean":"2023-11-26T07:23:31.156022+0000","last_became_active":"2023-11-26T07:23:28.647829+0000","last_became_peered":"2023-11-26T07:23:28.647829+0000","last_unstale":"2023-11-26T07:23:31.156022+0000","last_undegraded":"2023-11-26T07:23:31.156022+0000","last_fullsized":"2023-11-26T07:23:31.156022+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:23:27.611790+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:23:27.611790+0000","last_clean_scrub_stamp":"2023-11-26T07:23:27.611790+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T16:37:41.275330+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.16","version":"0'0","reported_seq":11,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-11-26T07:23:31.155587+0000","last_change":"2023-11-26T07:23:29.660260+0000","last_active":"2023-11-26T07:23:31.155587+0000","last_peered":"2023-11-26T07:23:31.155587+0000","last_clean":"2023-11-26T07:23:31.155587+0000","last_became_active":"2023-11-26T07:23:29.659968+0000","last_became_peered":"2023-11-26T07:23:29.659968+0000","last_unstale":"2023-11-26T07:23:31.155587+0000","last_undegraded":"2023-11-26T07:23:31.155587+0000","last_fullsized":"2023-11-26T07:23:31.155587+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:23:28.619155+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:23:28.619155+0000","last_clean_scrub_stamp":"2023-11-26T07:23:28.619155+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T09:04:09.764282+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"12.17","version":"0'0","reported_seq":12,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-11-26T07:23:30.644759+0000","last_change":"2023-11-26T07:23:28.644791+0000","last_active":"2023-11-26T07:23:30.644759+0000","last_peered":"2023-11-26T07:23:30.644759+0000","last_clean":"2023-11-26T07:23:30.644759+0000","last_became_active":"2023-11-26T07:23:28.644512+0000","last_became_peered":"2023-11-26T07:23:28.644512+0000","last_unstale":"2023-11-26T07:23:30.644759+0000","last_undegraded":"2023-11-26T07:23:30.644759+0000","last_fullsized":"2023-11-26T07:23:30.644759+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:23:27.611790+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:23:27.611790+0000","last_clean_scrub_stamp":"2023-11-26T07:23:27.611790+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T13:24:21.775457+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.17","version":"0'0","reported_seq":11,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-11-26T07:23:30.688939+0000","last_change":"2023-11-26T07:23:29.665721+0000","last_active":"2023-11-26T07:23:30.688939+0000","last_peered":"2023-11-26T07:23:30.688939+0000","last_clean":"2023-11-26T07:23:30.688939+0000","last_became_active":"2023-11-26T07:23:29.665429+0000","last_became_peered":"2023-11-26T07:23:29.665429+0000","last_unstale":"2023-11-26T07:23:30.688939+0000","last_undegraded":"2023-11-26T07:23:30.688939+0000","last_fullsized":"2023-11-26T07:23:30.688939+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:23:28.619155+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:23:28.619155+0000","last_clean_scrub_stamp":"2023-11-26T07:23:28.619155+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T08:26:22.625844+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"12.16","version":"0'0","reported_seq":12,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-11-26T07:23:30.644562+0000","last_change":"2023-11-26T07:23:28.638932+0000","last_active":"2023-11-26T07:23:30.644562+0000","last_peered":"2023-11-26T07:23:30.644562+0000","last_clean":"2023-11-26T07:23:30.644562+0000","last_became_active":"2023-11-26T07:23:28.638443+0000","last_became_peered":"2023-11-26T07:23:28.638443+0000","last_unstale":"2023-11-26T07:23:30.644562+0000","last_undegraded":"2023-11-26T07:23:30.644562+0000","last_fullsized":"2023-11-26T07:23:30.644562+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:23:27.611790+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:23:27.611790+0000","last_clean_scrub_stamp":"2023-11-26T07:23:27.611790+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T09:05:11.247887+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.14","version":"0'0","reported_seq":11,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-11-26T07:23:31.156838+0000","last_change":"2023-11-26T07:23:29.654472+0000","last_active":"2023-11-26T07:23:31.156838+0000","last_peered":"2023-11-26T07:23:31.156838+0000","last_clean":"2023-11-26T07:23:31.156838+0000","last_became_active":"2023-11-26T07:23:29.654202+0000","last_became_peered":"2023-11-26T07:23:29.654202+0000","last_unstale":"2023-11-26T07:23:31.156838+0000","last_undegraded":"2023-11-26T07:23:31.156838+0000","last_fullsized":"2023-11-26T07:23:31.156838+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:23:28.619155+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:23:28.619155+0000","last_clean_scrub_stamp":"2023-11-26T07:23:28.619155+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T19:16:31.468189+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"12.15","version":"83'3","reported_seq":15,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-11-26T07:23:30.644309+0000","last_change":"2023-11-26T07:23:28.653374+0000","last_active":"2023-11-26T07:23:30.644309+0000","last_peered":"2023-11-26T07:23:30.644309+0000","last_clean":"2023-11-26T07:23:30.644309+0000","last_became_active":"2023-11-26T07:23:28.652820+0000","last_became_peered":"2023-11-26T07:23:28.652820+0000","last_unstale":"2023-11-26T07:23:30.644309+0000","last_undegraded":"2023-11-26T07:23:30.644309+0000","last_fullsized":"2023-11-26T07:23:30.644309+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:23:27.611790+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:23:27.611790+0000","last_clean_scrub_stamp":"2023-11-26T07:23:27.611790+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T17:54:57.000661+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.15","version":"0'0","reported_seq":11,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-11-26T07:23:31.154082+0000","last_change":"2023-11-26T07:23:29.658456+0000","last_active":"2023-11-26T07:23:31.154082+0000","last_peered":"2023-11-26T07:23:31.154082+0000","last_clean":"2023-11-26T07:23:31.154082+0000","last_became_active":"2023-11-26T07:23:29.657988+0000","last_became_peered":"2023-11-26T07:23:29.657988+0000","last_unstale":"2023-11-26T07:23:31.154082+0000","last_undegraded":"2023-11-26T07:23:31.154082+0000","last_fullsized":"2023-11-26T07:23:31.154082+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:23:28.619155+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:23:28.619155+0000","last_clean_scrub_stamp":"2023-11-26T07:23:28.619155+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T09:25:04.676044+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"12.14","version":"83'6","reported_seq":18,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-11-26T07:23:30.688866+0000","last_change":"2023-11-26T07:23:28.649584+0000","last_active":"2023-11-26T07:23:30.688866+0000","last_peered":"2023-11-26T07:23:30.688866+0000","last_clean":"2023-11-26T07:23:30.688866+0000","last_became_active":"2023-11-26T07:23:28.649029+0000","last_became_peered":"2023-11-26T07:23:28.649029+0000","last_unstale":"2023-11-26T07:23:30.688866+0000","last_undegraded":"2023-11-26T07:23:30.688866+0000","last_fullsized":"2023-11-26T07:23:30.688866+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:23:27.611790+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:23:27.611790+0000","last_clean_scrub_stamp":"2023-11-26T07:23:27.611790+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T14:37:41.627112+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"13.12","version":"0'0","reported_seq":11,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-11-26T07:23:31.156580+0000","last_change":"2023-11-26T07:23:29.661802+0000","last_active":"2023-11-26T07:23:31.156580+0000","last_peered":"2023-11-26T07:23:31.156580+0000","last_clean":"2023-11-26T07:23:31.156580+0000","last_became_active":"2023-11-26T07:23:29.661431+0000","last_became_peered":"2023-11-26T07:23:29.661431+0000","last_unstale":"2023-11-26T07:23:31.156580+0000","last_undegraded":"2023-11-26T07:23:31.156580+0000","last_fullsized":"2023-11-26T07:23:31.156580+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:23:28.619155+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:23:28.619155+0000","last_clean_scrub_stamp":"2023-11-26T07:23:28.619155+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T08:43:34.194258+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"12.13","version":"83'2","reported_seq":14,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-11-26T07:23:31.156472+0000","last_change":"2023-11-26T07:23:28.653044+0000","last_active":"2023-11-26T07:23:31.156472+0000","last_peered":"2023-11-26T07:23:31.156472+0000","last_clean":"2023-11-26T07:23:31.156472+0000","last_became_active":"2023-11-26T07:23:28.652432+0000","last_became_peered":"2023-11-26T07:23:28.652432+0000","last_unstale":"2023-11-26T07:23:31.156472+0000","last_undegraded":"2023-11-26T07:23:31.156472+0000","last_fullsized":"2023-11-26T07:23:31.156472+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:23:27.611790+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:23:27.611790+0000","last_clean_scrub_stamp":"2023-11-26T07:23:27.611790+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T14:28:24.475819+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.13","version":"0'0","reported_seq":11,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-11-26T07:23:31.153703+0000","last_change":"2023-11-26T07:23:29.649101+0000","last_active":"2023-11-26T07:23:31.153703+0000","last_peered":"2023-11-26T07:23:31.153703+0000","last_clean":"2023-11-26T07:23:31.153703+0000","last_became_active":"2023-11-26T07:23:29.648737+0000","last_became_peered":"2023-11-26T07:23:29.648737+0000","last_unstale":"2023-11-26T07:23:31.153703+0000","last_undegraded":"2023-11-26T07:23:31.153703+0000","last_fullsized":"2023-11-26T07:23:31.153703+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:23:28.619155+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:23:28.619155+0000","last_clean_scrub_stamp":"2023-11-26T07:23:28.619155+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T11:31:08.516775+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.12","version":"0'0","reported_seq":12,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-11-26T07:23:31.155321+0000","last_change":"2023-11-26T07:23:28.654881+0000","last_active":"2023-11-26T07:23:31.155321+0000","last_peered":"2023-11-26T07:23:31.155321+0000","last_clean":"2023-11-26T07:23:31.155321+0000","last_became_active":"2023-11-26T07:23:28.654562+0000","last_became_peered":"2023-11-26T07:23:28.654562+0000","last_unstale":"2023-11-26T07:23:31.155321+0000","last_undegraded":"2023-11-26T07:23:31.155321+0000","last_fullsized":"2023-11-26T07:23:31.155321+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:23:27.611790+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:23:27.611790+0000","last_clean_scrub_stamp":"2023-11-26T07:23:27.611790+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T19:16:30.460823+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"13.10","version":"0'0","reported_seq":11,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-11-26T07:23:31.156029+0000","last_change":"2023-11-26T07:23:29.669153+0000","last_active":"2023-11-26T07:23:31.156029+0000","last_peered":"2023-11-26T07:23:31.156029+0000","last_clean":"2023-11-26T07:23:31.156029+0000","last_became_active":"2023-11-26T07:23:29.668883+0000","last_became_peered":"2023-11-26T07:23:29.668883+0000","last_unstale":"2023-11-26T07:23:31.156029+0000","last_undegraded":"2023-11-26T07:23:31.156029+0000","last_fullsized":"2023-11-26T07:23:31.156029+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:23:28.619155+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:23:28.619155+0000","last_clean_scrub_stamp":"2023-11-26T07:23:28.619155+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T17:25:32.924640+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"12.11","version":"0'0","reported_seq":12,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-11-26T07:23:30.688611+0000","last_change":"2023-11-26T07:23:28.644925+0000","last_active":"2023-11-26T07:23:30.688611+0000","last_peered":"2023-11-26T07:23:30.688611+0000","last_clean":"2023-11-26T07:23:30.688611+0000","last_became_active":"2023-11-26T07:23:28.644653+0000","last_became_peered":"2023-11-26T07:23:28.644653+0000","last_unstale":"2023-11-26T07:23:30.688611+0000","last_undegraded":"2023-11-26T07:23:30.688611+0000","last_fullsized":"2023-11-26T07:23:30.688611+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:23:27.611790+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:23:27.611790+0000","last_clean_scrub_stamp":"2023-11-26T07:23:27.611790+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T09:53:47.961982+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"13.11","version":"0'0","reported_seq":11,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-11-26T07:23:31.155992+0000","last_change":"2023-11-26T07:23:29.670341+0000","last_active":"2023-11-26T07:23:31.155992+0000","last_peered":"2023-11-26T07:23:31.155992+0000","last_clean":"2023-11-26T07:23:31.155992+0000","last_became_active":"2023-11-26T07:23:29.670002+0000","last_became_peered":"2023-11-26T07:23:29.670002+0000","last_unstale":"2023-11-26T07:23:31.155992+0000","last_undegraded":"2023-11-26T07:23:31.155992+0000","last_fullsized":"2023-11-26T07:23:31.155992+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:23:28.619155+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:23:28.619155+0000","last_clean_scrub_stamp":"2023-11-26T07:23:28.619155+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T15:35:16.874242+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"12.10","version":"83'8","reported_seq":20,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-11-26T07:23:31.156057+0000","last_change":"2023-11-26T07:23:28.654251+0000","last_active":"2023-11-26T07:23:31.156057+0000","last_peered":"2023-11-26T07:23:31.156057+0000","last_clean":"2023-11-26T07:23:31.156057+0000","last_became_active":"2023-11-26T07:23:28.651696+0000","last_became_peered":"2023-11-26T07:23:28.651696+0000","last_unstale":"2023-11-26T07:23:31.156057+0000","last_undegraded":"2023-11-26T07:23:31.156057+0000","last_fullsized":"2023-11-26T07:23:31.156057+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:23:27.611790+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:23:27.611790+0000","last_clean_scrub_stamp":"2023-11-26T07:23:27.611790+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T15:42:51.851770+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.e","version":"0'0","reported_seq":11,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-11-26T07:23:31.154797+0000","last_change":"2023-11-26T07:23:29.654149+0000","last_active":"2023-11-26T07:23:31.154797+0000","last_peered":"2023-11-26T07:23:31.154797+0000","last_clean":"2023-11-26T07:23:31.154797+0000","last_became_active":"2023-11-26T07:23:29.652774+0000","last_became_peered":"2023-11-26T07:23:29.652774+0000","last_unstale":"2023-11-26T07:23:31.154797+0000","last_undegraded":"2023-11-26T07:23:31.154797+0000","last_fullsized":"2023-11-26T07:23:31.154797+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:23:28.619155+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:23:28.619155+0000","last_clean_scrub_stamp":"2023-11-26T07:23:28.619155+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T12:37:58.279115+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.f","version":"83'1","reported_seq":13,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-11-26T07:23:30.644808+0000","last_change":"2023-11-26T07:23:28.647997+0000","last_active":"2023-11-26T07:23:30.644808+0000","last_peered":"2023-11-26T07:23:30.644808+0000","last_clean":"2023-11-26T07:23:30.644808+0000","last_became_active":"2023-11-26T07:23:28.647641+0000","last_became_peered":"2023-11-26T07:23:28.647641+0000","last_unstale":"2023-11-26T07:23:30.644808+0000","last_undegraded":"2023-11-26T07:23:30.644808+0000","last_fullsized":"2023-11-26T07:23:30.644808+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:23:27.611790+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:23:27.611790+0000","last_clean_scrub_stamp":"2023-11-26T07:23:27.611790+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T09:45:54.208868+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.f","version":"0'0","reported_seq":11,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-11-26T07:23:30.644734+0000","last_change":"2023-11-26T07:23:29.662942+0000","last_active":"2023-11-26T07:23:30.644734+0000","last_peered":"2023-11-26T07:23:30.644734+0000","last_clean":"2023-11-26T07:23:30.644734+0000","last_became_active":"2023-11-26T07:23:29.662591+0000","last_became_peered":"2023-11-26T07:23:29.662591+0000","last_unstale":"2023-11-26T07:23:30.644734+0000","last_undegraded":"2023-11-26T07:23:30.644734+0000","last_fullsized":"2023-11-26T07:23:30.644734+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:23:28.619155+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:23:28.619155+0000","last_clean_scrub_stamp":"2023-11-26T07:23:28.619155+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T08:43:34.194258+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"12.e","version":"83'2","reported_seq":14,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-11-26T07:23:31.156940+0000","last_change":"2023-11-26T07:23:28.641634+0000","last_active":"2023-11-26T07:23:31.156940+0000","last_peered":"2023-11-26T07:23:31.156940+0000","last_clean":"2023-11-26T07:23:31.156940+0000","last_became_active":"2023-11-26T07:23:28.641290+0000","last_became_peered":"2023-11-26T07:23:28.641290+0000","last_unstale":"2023-11-26T07:23:31.156940+0000","last_undegraded":"2023-11-26T07:23:31.156940+0000","last_fullsized":"2023-11-26T07:23:31.156940+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:23:27.611790+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:23:27.611790+0000","last_clean_scrub_stamp":"2023-11-26T07:23:27.611790+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T18:11:06.634569+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.d","version":"0'0","reported_seq":11,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-11-26T07:23:31.154275+0000","last_change":"2023-11-26T07:23:29.661133+0000","last_active":"2023-11-26T07:23:31.154275+0000","last_peered":"2023-11-26T07:23:31.154275+0000","last_clean":"2023-11-26T07:23:31.154275+0000","last_became_active":"2023-11-26T07:23:29.660870+0000","last_became_peered":"2023-11-26T07:23:29.660870+0000","last_unstale":"2023-11-26T07:23:31.154275+0000","last_undegraded":"2023-11-26T07:23:31.154275+0000","last_fullsized":"2023-11-26T07:23:31.154275+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:23:28.619155+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:23:28.619155+0000","last_clean_scrub_stamp":"2023-11-26T07:23:28.619155+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T14:42:50.857972+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"12.c","version":"0'0","reported_seq":12,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-11-26T07:23:31.156938+0000","last_change":"2023-11-26T07:23:28.649200+0000","last_active":"2023-11-26T07:23:31.156938+0000","last_peered":"2023-11-26T07:23:31.156938+0000","last_clean":"2023-11-26T07:23:31.156938+0000","last_became_active":"2023-11-26T07:23:28.648928+0000","last_became_peered":"2023-11-26T07:23:28.648928+0000","last_unstale":"2023-11-26T07:23:31.156938+0000","last_undegraded":"2023-11-26T07:23:31.156938+0000","last_fullsized":"2023-11-26T07:23:31.156938+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:23:27.611790+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:23:27.611790+0000","last_clean_scrub_stamp":"2023-11-26T07:23:27.611790+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T11:28:12.293763+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.a","version":"0'0","reported_seq":11,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-11-26T07:23:30.643067+0000","last_change":"2023-11-26T07:23:29.667394+0000","last_active":"2023-11-26T07:23:30.643067+0000","last_peered":"2023-11-26T07:23:30.643067+0000","last_clean":"2023-11-26T07:23:30.643067+0000","last_became_active":"2023-11-26T07:23:29.667077+0000","last_became_peered":"2023-11-26T07:23:29.667077+0000","last_unstale":"2023-11-26T07:23:30.643067+0000","last_undegraded":"2023-11-26T07:23:30.643067+0000","last_fullsized":"2023-11-26T07:23:30.643067+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:23:28.619155+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:23:28.619155+0000","last_clean_scrub_stamp":"2023-11-26T07:23:28.619155+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T09:25:23.557193+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"12.b","version":"83'2","reported_seq":14,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-11-26T07:23:31.155580+0000","last_change":"2023-11-26T07:23:28.653466+0000","last_active":"2023-11-26T07:23:31.155580+0000","last_peered":"2023-11-26T07:23:31.155580+0000","last_clean":"2023-11-26T07:23:31.155580+0000","last_became_active":"2023-11-26T07:23:28.652010+0000","last_became_peered":"2023-11-26T07:23:28.652010+0000","last_unstale":"2023-11-26T07:23:31.155580+0000","last_undegraded":"2023-11-26T07:23:31.155580+0000","last_fullsized":"2023-11-26T07:23:31.155580+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:23:27.611790+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:23:27.611790+0000","last_clean_scrub_stamp":"2023-11-26T07:23:27.611790+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T11:28:12.293763+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"13.b","version":"0'0","reported_seq":11,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-11-26T07:23:30.688631+0000","last_change":"2023-11-26T07:23:29.657898+0000","last_active":"2023-11-26T07:23:30.688631+0000","last_peered":"2023-11-26T07:23:30.688631+0000","last_clean":"2023-11-26T07:23:30.688631+0000","last_became_active":"2023-11-26T07:23:29.657623+0000","last_became_peered":"2023-11-26T07:23:29.657623+0000","last_unstale":"2023-11-26T07:23:30.688631+0000","last_undegraded":"2023-11-26T07:23:30.688631+0000","last_fullsized":"2023-11-26T07:23:30.688631+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:23:28.619155+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:23:28.619155+0000","last_clean_scrub_stamp":"2023-11-26T07:23:28.619155+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T13:22:18.189493+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"12.a","version":"0'0","reported_seq":12,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-11-26T07:23:30.643144+0000","last_change":"2023-11-26T07:23:28.646461+0000","last_active":"2023-11-26T07:23:30.643144+0000","last_peered":"2023-11-26T07:23:30.643144+0000","last_clean":"2023-11-26T07:23:30.643144+0000","last_became_active":"2023-11-26T07:23:28.645546+0000","last_became_peered":"2023-11-26T07:23:28.645546+0000","last_unstale":"2023-11-26T07:23:30.643144+0000","last_undegraded":"2023-11-26T07:23:30.643144+0000","last_fullsized":"2023-11-26T07:23:30.643144+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:23:27.611790+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:23:27.611790+0000","last_clean_scrub_stamp":"2023-11-26T07:23:27.611790+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T19:15:49.793351+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"13.8","version":"0'0","reported_seq":11,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-11-26T07:23:30.642665+0000","last_change":"2023-11-26T07:23:29.664106+0000","last_active":"2023-11-26T07:23:30.642665+0000","last_peered":"2023-11-26T07:23:30.642665+0000","last_clean":"2023-11-26T07:23:30.642665+0000","last_became_active":"2023-11-26T07:23:29.663802+0000","last_became_peered":"2023-11-26T07:23:29.663802+0000","last_unstale":"2023-11-26T07:23:30.642665+0000","last_undegraded":"2023-11-26T07:23:30.642665+0000","last_fullsized":"2023-11-26T07:23:30.642665+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:23:28.619155+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:23:28.619155+0000","last_clean_scrub_stamp":"2023-11-26T07:23:28.619155+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T13:58:11.826957+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"12.9","version":"0'0","reported_seq":12,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-11-26T07:23:30.642648+0000","last_change":"2023-11-26T07:23:28.643600+0000","last_active":"2023-11-26T07:23:30.642648+0000","last_peered":"2023-11-26T07:23:30.642648+0000","last_clean":"2023-11-26T07:23:30.642648+0000","last_became_active":"2023-11-26T07:23:28.643304+0000","last_became_peered":"2023-11-26T07:23:28.643304+0000","last_unstale":"2023-11-26T07:23:30.642648+0000","last_undegraded":"2023-11-26T07:23:30.642648+0000","last_fullsized":"2023-11-26T07:23:30.642648+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:23:27.611790+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:23:27.611790+0000","last_clean_scrub_stamp":"2023-11-26T07:23:27.611790+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T17:21:18.210966+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"13.9","version":"0'0","reported_seq":11,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-11-26T07:23:30.688648+0000","last_change":"2023-11-26T07:23:29.656753+0000","last_active":"2023-11-26T07:23:30.688648+0000","last_peered":"2023-11-26T07:23:30.688648+0000","last_clean":"2023-11-26T07:23:30.688648+0000","last_became_active":"2023-11-26T07:23:29.656231+0000","last_became_peered":"2023-11-26T07:23:29.656231+0000","last_unstale":"2023-11-26T07:23:30.688648+0000","last_undegraded":"2023-11-26T07:23:30.688648+0000","last_fullsized":"2023-11-26T07:23:30.688648+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:23:28.619155+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:23:28.619155+0000","last_clean_scrub_stamp":"2023-11-26T07:23:28.619155+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T10:15:59.819723+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"12.8","version":"0'0","reported_seq":12,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-11-26T07:23:31.155222+0000","last_change":"2023-11-26T07:23:28.652672+0000","last_active":"2023-11-26T07:23:31.155222+0000","last_peered":"2023-11-26T07:23:31.155222+0000","last_clean":"2023-11-26T07:23:31.155222+0000","last_became_active":"2023-11-26T07:23:28.652413+0000","last_became_peered":"2023-11-26T07:23:28.652413+0000","last_unstale":"2023-11-26T07:23:31.155222+0000","last_undegraded":"2023-11-26T07:23:31.155222+0000","last_fullsized":"2023-11-26T07:23:31.155222+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:23:27.611790+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:23:27.611790+0000","last_clean_scrub_stamp":"2023-11-26T07:23:27.611790+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T10:41:39.846929+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"13.6","version":"0'0","reported_seq":11,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-11-26T07:23:31.154398+0000","last_change":"2023-11-26T07:23:29.672449+0000","last_active":"2023-11-26T07:23:31.154398+0000","last_peered":"2023-11-26T07:23:31.154398+0000","last_clean":"2023-11-26T07:23:31.154398+0000","last_became_active":"2023-11-26T07:23:29.671495+0000","last_became_peered":"2023-11-26T07:23:29.671495+0000","last_unstale":"2023-11-26T07:23:31.154398+0000","last_undegraded":"2023-11-26T07:23:31.154398+0000","last_fullsized":"2023-11-26T07:23:31.154398+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:23:28.619155+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:23:28.619155+0000","last_clean_scrub_stamp":"2023-11-26T07:23:28.619155+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T09:45:55.216234+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"12.7","version":"83'2","reported_seq":14,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-11-26T07:23:31.156653+0000","last_change":"2023-11-26T07:23:28.653628+0000","last_active":"2023-11-26T07:23:31.156653+0000","last_peered":"2023-11-26T07:23:31.156653+0000","last_clean":"2023-11-26T07:23:31.156653+0000","last_became_active":"2023-11-26T07:23:28.653346+0000","last_became_peered":"2023-11-26T07:23:28.653346+0000","last_unstale":"2023-11-26T07:23:31.156653+0000","last_undegraded":"2023-11-26T07:23:31.156653+0000","last_fullsized":"2023-11-26T07:23:31.156653+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:23:27.611790+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:23:27.611790+0000","last_clean_scrub_stamp":"2023-11-26T07:23:27.611790+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T17:07:02.629426+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.7","version":"0'0","reported_seq":11,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-11-26T07:23:31.154912+0000","last_change":"2023-11-26T07:23:29.652585+0000","last_active":"2023-11-26T07:23:31.154912+0000","last_peered":"2023-11-26T07:23:31.154912+0000","last_clean":"2023-11-26T07:23:31.154912+0000","last_became_active":"2023-11-26T07:23:29.651796+0000","last_became_peered":"2023-11-26T07:23:29.651796+0000","last_unstale":"2023-11-26T07:23:31.154912+0000","last_undegraded":"2023-11-26T07:23:31.154912+0000","last_fullsized":"2023-11-26T07:23:31.154912+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:23:28.619155+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:23:28.619155+0000","last_clean_scrub_stamp":"2023-11-26T07:23:28.619155+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T08:50:54.551576+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.6","version":"0'0","reported_seq":12,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-11-26T07:23:31.156802+0000","last_change":"2023-11-26T07:23:28.651746+0000","last_active":"2023-11-26T07:23:31.156802+0000","last_peered":"2023-11-26T07:23:31.156802+0000","last_clean":"2023-11-26T07:23:31.156802+0000","last_became_active":"2023-11-26T07:23:28.651476+0000","last_became_peered":"2023-11-26T07:23:28.651476+0000","last_unstale":"2023-11-26T07:23:31.156802+0000","last_undegraded":"2023-11-26T07:23:31.156802+0000","last_fullsized":"2023-11-26T07:23:31.156802+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:23:27.611790+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:23:27.611790+0000","last_clean_scrub_stamp":"2023-11-26T07:23:27.611790+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T14:29:53.425427+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.4","version":"0'0","reported_seq":11,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-11-26T07:23:30.644682+0000","last_change":"2023-11-26T07:23:29.665782+0000","last_active":"2023-11-26T07:23:30.644682+0000","last_peered":"2023-11-26T07:23:30.644682+0000","last_clean":"2023-11-26T07:23:30.644682+0000","last_became_active":"2023-11-26T07:23:29.665491+0000","last_became_peered":"2023-11-26T07:23:29.665491+0000","last_unstale":"2023-11-26T07:23:30.644682+0000","last_undegraded":"2023-11-26T07:23:30.644682+0000","last_fullsized":"2023-11-26T07:23:30.644682+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:23:28.619155+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:23:28.619155+0000","last_clean_scrub_stamp":"2023-11-26T07:23:28.619155+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T18:11:07.641934+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"12.5","version":"0'0","reported_seq":12,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-11-26T07:23:30.644246+0000","last_change":"2023-11-26T07:23:28.646051+0000","last_active":"2023-11-26T07:23:30.644246+0000","last_peered":"2023-11-26T07:23:30.644246+0000","last_clean":"2023-11-26T07:23:30.644246+0000","last_became_active":"2023-11-26T07:23:28.645736+0000","last_became_peered":"2023-11-26T07:23:28.645736+0000","last_unstale":"2023-11-26T07:23:30.644246+0000","last_undegraded":"2023-11-26T07:23:30.644246+0000","last_fullsized":"2023-11-26T07:23:30.644246+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:23:27.611790+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:23:27.611790+0000","last_clean_scrub_stamp":"2023-11-26T07:23:27.611790+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T17:13:40.716907+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.5","version":"0'0","reported_seq":11,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-11-26T07:23:31.154442+0000","last_change":"2023-11-26T07:23:29.658254+0000","last_active":"2023-11-26T07:23:31.154442+0000","last_peered":"2023-11-26T07:23:31.154442+0000","last_clean":"2023-11-26T07:23:31.154442+0000","last_became_active":"2023-11-26T07:23:29.657633+0000","last_became_peered":"2023-11-26T07:23:29.657633+0000","last_unstale":"2023-11-26T07:23:31.154442+0000","last_undegraded":"2023-11-26T07:23:31.154442+0000","last_fullsized":"2023-11-26T07:23:31.154442+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:23:28.619155+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:23:28.619155+0000","last_clean_scrub_stamp":"2023-11-26T07:23:28.619155+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T17:48:40.805546+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"12.4","version":"0'0","reported_seq":12,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-11-26T07:23:30.644701+0000","last_change":"2023-11-26T07:23:28.646843+0000","last_active":"2023-11-26T07:23:30.644701+0000","last_peered":"2023-11-26T07:23:30.644701+0000","last_clean":"2023-11-26T07:23:30.644701+0000","last_became_active":"2023-11-26T07:23:28.646559+0000","last_became_peered":"2023-11-26T07:23:28.646559+0000","last_unstale":"2023-11-26T07:23:30.644701+0000","last_undegraded":"2023-11-26T07:23:30.644701+0000","last_fullsized":"2023-11-26T07:23:30.644701+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:23:27.611790+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:23:27.611790+0000","last_clean_scrub_stamp":"2023-11-26T07:23:27.611790+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T08:26:21.618478+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.2","version":"0'0","reported_seq":11,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-11-26T07:23:30.688621+0000","last_change":"2023-11-26T07:23:29.661239+0000","last_active":"2023-11-26T07:23:30.688621+0000","last_peered":"2023-11-26T07:23:30.688621+0000","last_clean":"2023-11-26T07:23:30.688621+0000","last_became_active":"2023-11-26T07:23:29.660971+0000","last_became_peered":"2023-11-26T07:23:29.660971+0000","last_unstale":"2023-11-26T07:23:30.688621+0000","last_undegraded":"2023-11-26T07:23:30.688621+0000","last_fullsized":"2023-11-26T07:23:30.688621+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:23:28.619155+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:23:28.619155+0000","last_clean_scrub_stamp":"2023-11-26T07:23:28.619155+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T09:05:12.255253+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"12.3","version":"0'0","reported_seq":12,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-11-26T07:23:31.157235+0000","last_change":"2023-11-26T07:23:28.642964+0000","last_active":"2023-11-26T07:23:31.157235+0000","last_peered":"2023-11-26T07:23:31.157235+0000","last_clean":"2023-11-26T07:23:31.157235+0000","last_became_active":"2023-11-26T07:23:28.642675+0000","last_became_peered":"2023-11-26T07:23:28.642675+0000","last_unstale":"2023-11-26T07:23:31.157235+0000","last_undegraded":"2023-11-26T07:23:31.157235+0000","last_fullsized":"2023-11-26T07:23:31.157235+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:23:27.611790+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:23:27.611790+0000","last_clean_scrub_stamp":"2023-11-26T07:23:27.611790+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T13:34:19.121282+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.3","version":"0'0","reported_seq":11,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-11-26T07:23:31.157192+0000","last_change":"2023-11-26T07:23:29.663381+0000","last_active":"2023-11-26T07:23:31.157192+0000","last_peered":"2023-11-26T07:23:31.157192+0000","last_clean":"2023-11-26T07:23:31.157192+0000","last_became_active":"2023-11-26T07:23:29.663006+0000","last_became_peered":"2023-11-26T07:23:29.663006+0000","last_unstale":"2023-11-26T07:23:31.157192+0000","last_undegraded":"2023-11-26T07:23:31.157192+0000","last_fullsized":"2023-11-26T07:23:31.157192+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:23:28.619155+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:23:28.619155+0000","last_clean_scrub_stamp":"2023-11-26T07:23:28.619155+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T10:32:40.375411+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"12.2","version":"0'0","reported_seq":12,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-11-26T07:23:31.154176+0000","last_change":"2023-11-26T07:23:28.643979+0000","last_active":"2023-11-26T07:23:31.154176+0000","last_peered":"2023-11-26T07:23:31.154176+0000","last_clean":"2023-11-26T07:23:31.154176+0000","last_became_active":"2023-11-26T07:23:28.642738+0000","last_became_peered":"2023-11-26T07:23:28.642738+0000","last_unstale":"2023-11-26T07:23:31.154176+0000","last_undegraded":"2023-11-26T07:23:31.154176+0000","last_fullsized":"2023-11-26T07:23:31.154176+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:23:27.611790+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:23:27.611790+0000","last_clean_scrub_stamp":"2023-11-26T07:23:27.611790+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T14:37:41.627112+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"13.0","version":"0'0","reported_seq":11,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-11-26T07:23:31.153856+0000","last_change":"2023-11-26T07:23:29.658136+0000","last_active":"2023-11-26T07:23:31.153856+0000","last_peered":"2023-11-26T07:23:31.153856+0000","last_clean":"2023-11-26T07:23:31.153856+0000","last_became_active":"2023-11-26T07:23:29.657860+0000","last_became_peered":"2023-11-26T07:23:29.657860+0000","last_unstale":"2023-11-26T07:23:31.153856+0000","last_undegraded":"2023-11-26T07:23:31.153856+0000","last_fullsized":"2023-11-26T07:23:31.153856+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:23:28.619155+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:23:28.619155+0000","last_clean_scrub_stamp":"2023-11-26T07:23:28.619155+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T10:32:40.375411+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.1","version":"0'0","reported_seq":12,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-11-26T07:23:30.644179+0000","last_change":"2023-11-26T07:23:28.644096+0000","last_active":"2023-11-26T07:23:30.644179+0000","last_peered":"2023-11-26T07:23:30.644179+0000","last_clean":"2023-11-26T07:23:30.644179+0000","last_became_active":"2023-11-26T07:23:28.643353+0000","last_became_peered":"2023-11-26T07:23:28.643353+0000","last_unstale":"2023-11-26T07:23:30.644179+0000","last_undegraded":"2023-11-26T07:23:30.644179+0000","last_fullsized":"2023-11-26T07:23:30.644179+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:23:27.611790+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:23:27.611790+0000","last_clean_scrub_stamp":"2023-11-26T07:23:27.611790+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T09:53:47.961982+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.1","version":"0'0","reported_seq":11,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-11-26T07:23:30.644066+0000","last_change":"2023-11-26T07:23:29.667904+0000","last_active":"2023-11-26T07:23:30.644066+0000","last_peered":"2023-11-26T07:23:30.644066+0000","last_clean":"2023-11-26T07:23:30.644066+0000","last_became_active":"2023-11-26T07:23:29.667512+0000","last_became_peered":"2023-11-26T07:23:29.667512+0000","last_unstale":"2023-11-26T07:23:30.644066+0000","last_undegraded":"2023-11-26T07:23:30.644066+0000","last_fullsized":"2023-11-26T07:23:30.644066+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:23:28.619155+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:23:28.619155+0000","last_clean_scrub_stamp":"2023-11-26T07:23:28.619155+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T08:23:08.828343+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"12.0","version":"0'0","reported_seq":12,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-11-26T07:23:31.153778+0000","last_change":"2023-11-26T07:23:28.640179+0000","last_active":"2023-11-26T07:23:31.153778+0000","last_peered":"2023-11-26T07:23:31.153778+0000","last_clean":"2023-11-26T07:23:31.153778+0000","last_became_active":"2023-11-26T07:23:28.639867+0000","last_became_peered":"2023-11-26T07:23:28.639867+0000","last_unstale":"2023-11-26T07:23:31.153778+0000","last_undegraded":"2023-11-26T07:23:31.153778+0000","last_fullsized":"2023-11-26T07:23:31.153778+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:23:27.611790+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:23:27.611790+0000","last_clean_scrub_stamp":"2023-11-26T07:23:27.611790+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T13:16:03.358808+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"13.c","version":"0'0","reported_seq":11,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-11-26T07:23:31.156681+0000","last_change":"2023-11-26T07:23:29.671830+0000","last_active":"2023-11-26T07:23:31.156681+0000","last_peered":"2023-11-26T07:23:31.156681+0000","last_clean":"2023-11-26T07:23:31.156681+0000","last_became_active":"2023-11-26T07:23:29.671581+0000","last_became_peered":"2023-11-26T07:23:29.671581+0000","last_unstale":"2023-11-26T07:23:31.156681+0000","last_undegraded":"2023-11-26T07:23:31.156681+0000","last_fullsized":"2023-11-26T07:23:31.156681+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:23:28.619155+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:23:28.619155+0000","last_clean_scrub_stamp":"2023-11-26T07:23:28.619155+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T12:43:29.494016+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"12.d","version":"83'1","reported_seq":13,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-11-26T07:23:31.154710+0000","last_change":"2023-11-26T07:23:28.652819+0000","last_active":"2023-11-26T07:23:31.154710+0000","last_peered":"2023-11-26T07:23:31.154710+0000","last_clean":"2023-11-26T07:23:31.154710+0000","last_became_active":"2023-11-26T07:23:28.652526+0000","last_became_peered":"2023-11-26T07:23:28.652526+0000","last_unstale":"2023-11-26T07:23:31.154710+0000","last_undegraded":"2023-11-26T07:23:31.154710+0000","last_fullsized":"2023-11-26T07:23:31.154710+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:23:27.611790+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:23:27.611790+0000","last_clean_scrub_stamp":"2023-11-26T07:23:27.611790+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T11:28:12.293763+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.0","version":"72'128","reported_seq":255,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-11-26T07:23:29.641547+0000","last_change":"2023-11-26T07:17:43.827802+0000","last_active":"2023-11-26T07:23:29.641547+0000","last_peered":"2023-11-26T07:23:29.641547+0000","last_clean":"2023-11-26T07:23:29.641547+0000","last_became_active":"2023-11-26T07:17:43.827308+0000","last_became_peered":"2023-11-26T07:17:43.827308+0000","last_unstale":"2023-11-26T07:23:29.641547+0000","last_undegraded":"2023-11-26T07:23:29.641547+0000","last_fullsized":"2023-11-26T07:23:29.641547+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:17:42.160945+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:17:42.160945+0000","last_clean_scrub_stamp":"2023-11-26T07:17:42.160945+0000","objects_scrubbed":0,"log_size":128,"log_dups_size":0,"ondisk_log_size":128,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T08:17:54.167366+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":212,"num_read_kb":306,"num_write":179,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":13,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":12,"num_pg":32,"stat_sum":{"num_bytes":2410,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":38,"ondisk_log_size":38,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":212,"num_read_kb":306,"num_write":179,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":128,"ondisk_log_size":128,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574920,"num_pgs":19,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32772,"kb_used_data":1592,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339068,"statfs":{"total":96636764160,"available":96603205632,"internally_reserved":0,"allocated":1630208,"data_stored":1034627,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":31914442},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Nov 26 07:22:55 2023","interfaces":[{"interface":"back","average":{"1min":0.66900000000000004,"5min":0.70999999999999996,"15min":0.71299999999999997},"min":{"1min":0.437,"5min":0.42999999999999999,"15min":0.42999999999999999},"max":{"1min":0.873,"5min":0.98199999999999998,"15min":0.98199999999999998},"last":0.77300000000000002},{"interface":"front","average":{"1min":0.70699999999999996,"5min":0.72799999999999998,"15min":0.71499999999999997},"min":{"1min":0.52900000000000003,"5min":0.52900000000000003,"15min":0.52900000000000003},"max":{"1min":0.92800000000000005,"5min":1.0980000000000001,"15min":1.0980000000000001},"last":0.82899999999999996}]},{"osd":1,"last update":"Sun Nov 26 07:22:55 2023","interfaces":[{"interface":"back","average":{"1min":0.67800000000000005,"5min":0.68700000000000006,"15min":0.68300000000000005},"min":{"1min":0.435,"5min":0.435,"15min":0.435},"max":{"1min":0.90800000000000003,"5min":1.0669999999999999,"15min":1.0669999999999999},"last":0.66100000000000003},{"interface":"front","average":{"1min":0.67300000000000004,"5min":0.71099999999999997,"15min":0.79600000000000004},"min":{"1min":0.49199999999999999,"5min":0.47599999999999998,"15min":0.47599999999999998},"max":{"1min":0.88700000000000001,"5min":4.5309999999999997,"15min":4.5309999999999997},"last":0.68999999999999995}]},{"osd":2,"last update":"Sun Nov 26 07:22:55 2023","interfaces":[{"interface":"back","average":{"1min":0.68899999999999995,"5min":0.69499999999999995,"15min":0.68200000000000005},"min":{"1min":0.503,"5min":0.39700000000000002,"15min":0.39700000000000002},"max":{"1min":0.88,"5min":1.0800000000000001,"15min":1.0800000000000001},"last":0.91600000000000004},{"interface":"front","average":{"1min":0.69899999999999995,"5min":0.70999999999999996,"15min":0.71199999999999997},"min":{"1min":0.40999999999999998,"5min":0.40999999999999998,"15min":0.40999999999999998},"max":{"1min":0.97699999999999998,"5min":1.1339999999999999,"15min":1.1339999999999999},"last":0.72099999999999997}]},{"osd":3,"last update":"Sun Nov 26 07:22:55 2023","interfaces":[{"interface":"back","average":{"1min":0.70799999999999996,"5min":0.72099999999999997,"15min":0.70499999999999996},"min":{"1min":0.52700000000000002,"5min":0.47799999999999998,"15min":0.47799999999999998},"max":{"1min":0.98399999999999999,"5min":1.1240000000000001,"15min":1.1240000000000001},"last":0.73399999999999999},{"interface":"front","average":{"1min":0.71299999999999997,"5min":0.74399999999999999,"15min":0.73599999999999999},"min":{"1min":0.51700000000000002,"5min":0.433,"15min":0.433},"max":{"1min":1.0029999999999999,"5min":1.4490000000000001,"15min":1.4490000000000001},"last":0.88800000000000001}]},{"osd":4,"last update":"Sun Nov 26 07:22:55 2023","interfaces":[{"interface":"back","average":{"1min":0.60599999999999998,"5min":0.61299999999999999,"15min":0.60399999999999998},"min":{"1min":0.39400000000000002,"5min":0.39400000000000002,"15min":0.39400000000000002},"max":{"1min":0.88,"5min":0.96299999999999997,"15min":0.96299999999999997},"last":0.76000000000000001},{"interface":"front","average":{"1min":0.60899999999999999,"5min":0.64800000000000002,"15min":0.66400000000000003},"min":{"1min":0.35999999999999999,"5min":0.35999999999999999,"15min":0.35999999999999999},"max":{"1min":0.92600000000000005,"5min":1.5509999999999999,"15min":1.5509999999999999},"last":0.78900000000000003}]},{"osd":5,"last update":"Sun Nov 26 07:22:55 2023","interfaces":[{"interface":"back","average":{"1min":0.63100000000000001,"5min":0.67500000000000004,"15min":0.66900000000000004},"min":{"1min":0.45000000000000001,"5min":0.443,"15min":0.443},"max":{"1min":0.95899999999999996,"5min":1.415,"15min":1.415},"last":0.81299999999999994},{"interface":"front","average":{"1min":0.68700000000000006,"5min":0.68799999999999994,"15min":0.70499999999999996},"min":{"1min":0.47499999999999998,"5min":0.47499999999999998,"15min":0.47499999999999998},"max":{"1min":0.93200000000000005,"5min":1.3859999999999999,"15min":1.3859999999999999},"last":0.873}]},{"osd":6,"last update":"Sun Nov 26 07:22:55 2023","interfaces":[{"interface":"back","average":{"1min":0.77800000000000002,"5min":0.78000000000000003,"15min":0.77400000000000002},"min":{"1min":0.57099999999999995,"5min":0.51700000000000002,"15min":0.51700000000000002},"max":{"1min":0.97599999999999998,"5min":1.3069999999999999,"15min":1.3069999999999999},"last":0.90200000000000002},{"interface":"front","average":{"1min":0.74099999999999999,"5min":0.75600000000000001,"15min":0.76600000000000001},"min":{"1min":0.49399999999999999,"5min":0.46899999999999997,"15min":0.46899999999999997},"max":{"1min":1.032,"5min":1.6759999999999999,"15min":1.6759999999999999},"last":0.85499999999999998}]}]},{"osd":6,"up_from":13,"seq":55834574922,"num_pgs":30,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32208,"kb_used_data":1020,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339632,"statfs":{"total":96636764160,"available":96603783168,"internally_reserved":0,"allocated":1044480,"data_stored":444883,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Nov 26 07:22:55 2023","interfaces":[{"interface":"back","average":{"1min":0.61199999999999999,"5min":0.68400000000000005,"15min":0.68400000000000005},"min":{"1min":0.32200000000000001,"5min":0.32200000000000001,"15min":0.32200000000000001},"max":{"1min":0.81999999999999995,"5min":1.7549999999999999,"15min":1.7549999999999999},"last":0.82399999999999995},{"interface":"front","average":{"1min":0.65300000000000002,"5min":0.75700000000000001,"15min":0.89100000000000001},"min":{"1min":0.42799999999999999,"5min":0.38300000000000001,"15min":0.38300000000000001},"max":{"1min":0.89400000000000002,"5min":5.7489999999999997,"15min":5.7489999999999997},"last":0.67900000000000005}]},{"osd":1,"last update":"Sun Nov 26 07:22:55 2023","interfaces":[{"interface":"back","average":{"1min":0.70099999999999996,"5min":0.76100000000000001,"15min":0.86299999999999999},"min":{"1min":0.50800000000000001,"5min":0.41199999999999998,"15min":0.41199999999999998},"max":{"1min":0.97699999999999998,"5min":5.4240000000000004,"15min":5.4240000000000004},"last":0.75800000000000001},{"interface":"front","average":{"1min":0.68000000000000005,"5min":0.751,"15min":0.84799999999999998},"min":{"1min":0.495,"5min":0.495,"15min":0.495},"max":{"1min":0.83599999999999997,"5min":5.6020000000000003,"15min":5.6020000000000003},"last":0.72999999999999998}]},{"osd":2,"last update":"Sun Nov 26 07:22:55 2023","interfaces":[{"interface":"back","average":{"1min":0.65000000000000002,"5min":0.68100000000000005,"15min":0.69099999999999995},"min":{"1min":0.41699999999999998,"5min":0.41699999999999998,"15min":0.41699999999999998},"max":{"1min":0.86799999999999999,"5min":1.8520000000000001,"15min":1.8520000000000001},"last":0.74099999999999999},{"interface":"front","average":{"1min":0.69699999999999995,"5min":0.72599999999999998,"15min":0.73099999999999998},"min":{"1min":0.54100000000000004,"5min":0.437,"15min":0.437},"max":{"1min":0.91800000000000004,"5min":1.9159999999999999,"15min":1.9159999999999999},"last":0.79500000000000004}]},{"osd":3,"last update":"Sun Nov 26 07:22:55 2023","interfaces":[{"interface":"back","average":{"1min":0.67700000000000005,"5min":0.755,"15min":0.85299999999999998},"min":{"1min":0.57099999999999995,"5min":0.46899999999999997,"15min":0.46899999999999997},"max":{"1min":0.86699999999999999,"5min":5.4390000000000001,"15min":5.4390000000000001},"last":0.78000000000000003},{"interface":"front","average":{"1min":0.68400000000000005,"5min":0.76300000000000001,"15min":0.88400000000000001},"min":{"1min":0.54200000000000004,"5min":0.48399999999999999,"15min":0.48399999999999999},"max":{"1min":0.85699999999999998,"5min":5.5490000000000004,"15min":5.5490000000000004},"last":0.69999999999999996}]},{"osd":4,"last update":"Sun Nov 26 07:22:55 2023","interfaces":[{"interface":"back","average":{"1min":0.59499999999999997,"5min":0.61699999999999999,"15min":0.63300000000000001},"min":{"1min":0.42199999999999999,"5min":0.371,"15min":0.371},"max":{"1min":0.85199999999999998,"5min":1.0129999999999999,"15min":1.0129999999999999},"last":0.505},{"interface":"front","average":{"1min":0.63300000000000001,"5min":0.64500000000000002,"15min":0.66500000000000004},"min":{"1min":0.40000000000000002,"5min":0.38600000000000001,"15min":0.38600000000000001},"max":{"1min":0.81999999999999995,"5min":1.0349999999999999,"15min":1.0349999999999999},"last":0.52500000000000002}]},{"osd":5,"last update":"Sun Nov 26 07:22:55 2023","interfaces":[{"interface":"back","average":{"1min":0.63900000000000001,"5min":0.63400000000000001,"15min":0.65000000000000002},"min":{"1min":0.42899999999999999,"5min":0.38200000000000001,"15min":0.38200000000000001},"max":{"1min":0.91000000000000003,"5min":0.91000000000000003,"15min":0.91000000000000003},"last":0.55600000000000005},{"interface":"front","average":{"1min":0.70499999999999996,"5min":0.69699999999999995,"15min":0.70699999999999996},"min":{"1min":0.46600000000000003,"5min":0.436,"15min":0.436},"max":{"1min":1.014,"5min":1.0960000000000001,"15min":1.0960000000000001},"last":0.65400000000000003}]},{"osd":7,"last update":"Sun Nov 26 07:22:55 2023","interfaces":[{"interface":"back","average":{"1min":0.72299999999999998,"5min":0.71599999999999997,"15min":0.73299999999999998},"min":{"1min":0.45100000000000001,"5min":0.42599999999999999,"15min":0.42599999999999999},"max":{"1min":0.95099999999999996,"5min":0.998,"15min":0.998},"last":0.57099999999999995},{"interface":"front","average":{"1min":0.70699999999999996,"5min":0.69599999999999995,"15min":0.70899999999999996},"min":{"1min":0.51600000000000001,"5min":0.46400000000000002,"15min":0.46400000000000002},"max":{"1min":0.90000000000000002,"5min":0.98799999999999999,"15min":0.98799999999999999},"last":0.63100000000000001}]}]},{"osd":5,"up_from":13,"seq":55834574920,"num_pgs":16,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":28052,"kb_used_data":1024,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343788,"statfs":{"total":96636764160,"available":96608038912,"internally_reserved":0,"allocated":1048576,"data_stored":445823,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Nov 26 07:22:49 2023","interfaces":[{"interface":"back","average":{"1min":0.80100000000000005,"5min":0.71499999999999997,"15min":0.68200000000000005},"min":{"1min":0.36499999999999999,"5min":0.36499999999999999,"15min":0.36499999999999999},"max":{"1min":2.6949999999999998,"5min":2.6949999999999998,"15min":2.6949999999999998},"last":0.72499999999999998},{"interface":"front","average":{"1min":0.76900000000000002,"5min":0.71699999999999997,"15min":0.67900000000000005},"min":{"1min":0.38200000000000001,"5min":0.38200000000000001,"15min":0.38200000000000001},"max":{"1min":2.0150000000000001,"5min":2.0150000000000001,"15min":2.0150000000000001},"last":0.66600000000000004}]},{"osd":1,"last update":"Sun Nov 26 07:22:49 2023","interfaces":[{"interface":"back","average":{"1min":0.71199999999999997,"5min":0.70399999999999996,"15min":0.68000000000000005},"min":{"1min":0.41799999999999998,"5min":0.41599999999999998,"15min":0.41599999999999998},"max":{"1min":1.532,"5min":1.532,"15min":1.532},"last":0.621},{"interface":"front","average":{"1min":0.79800000000000004,"5min":0.73099999999999998,"15min":0.70099999999999996},"min":{"1min":0.497,"5min":0.443,"15min":0.443},"max":{"1min":2.6400000000000001,"5min":2.6400000000000001,"15min":2.6400000000000001},"last":0.85499999999999998}]},{"osd":2,"last update":"Sun Nov 26 07:22:49 2023","interfaces":[{"interface":"back","average":{"1min":0.72699999999999998,"5min":0.70899999999999996,"15min":0.68600000000000005},"min":{"1min":0.47599999999999998,"5min":0.45900000000000002,"15min":0.45900000000000002},"max":{"1min":1.665,"5min":1.665,"15min":1.665},"last":0.86799999999999999},{"interface":"front","average":{"1min":0.77000000000000002,"5min":0.75,"15min":0.69599999999999995},"min":{"1min":0.46200000000000002,"5min":0.46200000000000002,"15min":0.46200000000000002},"max":{"1min":2.125,"5min":2.125,"15min":2.125},"last":0.78000000000000003}]},{"osd":3,"last update":"Sun Nov 26 07:22:49 2023","interfaces":[{"interface":"back","average":{"1min":0.80800000000000005,"5min":0.751,"15min":0.72999999999999998},"min":{"1min":0.57199999999999995,"5min":0.43099999999999999,"15min":0.43099999999999999},"max":{"1min":2.0529999999999999,"5min":2.0529999999999999,"15min":2.0529999999999999},"last":0.70999999999999996},{"interface":"front","average":{"1min":0.76200000000000001,"5min":0.73999999999999999,"15min":0.69599999999999995},"min":{"1min":0.51900000000000002,"5min":0.44600000000000001,"15min":0.44600000000000001},"max":{"1min":1.629,"5min":1.629,"15min":1.629},"last":0.755}]},{"osd":4,"last update":"Sun Nov 26 07:22:49 2023","interfaces":[{"interface":"back","average":{"1min":0.69499999999999995,"5min":0.61199999999999999,"15min":0.627},"min":{"1min":0.41799999999999998,"5min":0.35299999999999998,"15min":0.35299999999999998},"max":{"1min":2.7429999999999999,"5min":2.7429999999999999,"15min":2.7429999999999999},"last":0.81499999999999995},{"interface":"front","average":{"1min":0.77000000000000002,"5min":0.68200000000000005,"15min":0.69999999999999996},"min":{"1min":0.48599999999999999,"5min":0.41299999999999998,"15min":0.41299999999999998},"max":{"1min":2.4630000000000001,"5min":2.4630000000000001,"15min":2.4630000000000001},"last":0.80300000000000005}]},{"osd":6,"last update":"Sun Nov 26 07:22:49 2023","interfaces":[{"interface":"back","average":{"1min":0.73199999999999998,"5min":0.67400000000000004,"15min":0.67200000000000004},"min":{"1min":0.41999999999999998,"5min":0.41999999999999998,"15min":0.41999999999999998},"max":{"1min":2.1619999999999999,"5min":2.1619999999999999,"15min":2.1619999999999999},"last":0.89900000000000002},{"interface":"front","average":{"1min":0.77000000000000002,"5min":0.71199999999999997,"15min":0.72199999999999998},"min":{"1min":0.47599999999999998,"5min":0.436,"15min":0.436},"max":{"1min":2.7869999999999999,"5min":2.7869999999999999,"15min":2.7869999999999999},"last":0.88600000000000001}]},{"osd":7,"last update":"Sun Nov 26 07:22:49 2023","interfaces":[{"interface":"back","average":{"1min":0.80600000000000005,"5min":0.75900000000000001,"15min":0.78900000000000003},"min":{"1min":0.45700000000000002,"5min":0.45700000000000002,"15min":0.45700000000000002},"max":{"1min":2.1960000000000002,"5min":2.1960000000000002,"15min":2.1960000000000002},"last":0.83799999999999997},{"interface":"front","average":{"1min":0.83499999999999996,"5min":0.76300000000000001,"15min":0.77100000000000002},"min":{"1min":0.59599999999999997,"5min":0.45000000000000001,"15min":0.45000000000000001},"max":{"1min":1.9510000000000001,"5min":1.9510000000000001,"15min":1.9510000000000001},"last":0.69099999999999995}]}]},{"osd":4,"up_from":13,"seq":55834574922,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32188,"kb_used_data":1008,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339652,"statfs":{"total":96636764160,"available":96603803648,"internally_reserved":0,"allocated":1032192,"data_stored":444189,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Nov 26 07:22:53 2023","interfaces":[{"interface":"back","average":{"1min":0.68600000000000005,"5min":0.68700000000000006,"15min":0.75800000000000001},"min":{"1min":0.51800000000000002,"5min":0.32100000000000001,"15min":0.32100000000000001},"max":{"1min":0.86799999999999999,"5min":4.1689999999999996,"15min":4.1689999999999996},"last":0.55600000000000005},{"interface":"front","average":{"1min":0.66900000000000004,"5min":0.91700000000000004,"15min":0.97299999999999998},"min":{"1min":0.53800000000000003,"5min":0.46200000000000002,"15min":0.46200000000000002},"max":{"1min":0.85699999999999998,"5min":18.021000000000001,"15min":18.021000000000001},"last":0.79000000000000004}]},{"osd":1,"last update":"Sun Nov 26 07:22:53 2023","interfaces":[{"interface":"back","average":{"1min":0.68500000000000005,"5min":0.71899999999999997,"15min":0.67100000000000004},"min":{"1min":0.52200000000000002,"5min":0.43099999999999999,"15min":0.43099999999999999},"max":{"1min":0.84099999999999997,"5min":4.3159999999999998,"15min":4.3159999999999998},"last":0.57699999999999996},{"interface":"front","average":{"1min":0.65800000000000003,"5min":0.79500000000000004,"15min":0.81100000000000005},"min":{"1min":0.48699999999999999,"5min":0.39800000000000002,"15min":0.39800000000000002},"max":{"1min":0.876,"5min":11.206,"15min":11.206},"last":0.748}]},{"osd":2,"last update":"Sun Nov 26 07:22:53 2023","interfaces":[{"interface":"back","average":{"1min":0.67200000000000004,"5min":0.871,"15min":0.82799999999999996},"min":{"1min":0.46600000000000003,"5min":0.433,"15min":0.433},"max":{"1min":0.84399999999999997,"5min":15.441000000000001,"15min":15.441000000000001},"last":0.71299999999999997},{"interface":"front","average":{"1min":0.76900000000000002,"5min":0.91500000000000004,"15min":1.0149999999999999},"min":{"1min":0.64200000000000002,"5min":0.53400000000000003,"15min":0.53400000000000003},"max":{"1min":0.93300000000000005,"5min":9.4800000000000004,"15min":9.4800000000000004},"last":0.72599999999999998}]},{"osd":3,"last update":"Sun Nov 26 07:22:53 2023","interfaces":[{"interface":"back","average":{"1min":0.71299999999999997,"5min":0.83199999999999996,"15min":0.76400000000000001},"min":{"1min":0.498,"5min":0.441,"15min":0.441},"max":{"1min":0.92400000000000004,"5min":12.715,"15min":12.715},"last":0.60299999999999998},{"interface":"front","average":{"1min":0.72099999999999997,"5min":0.83799999999999997,"15min":0.86499999999999999},"min":{"1min":0.51900000000000002,"5min":0.50700000000000001,"15min":0.50700000000000001},"max":{"1min":0.89800000000000002,"5min":8.3239999999999998,"15min":8.3239999999999998},"last":0.65200000000000002}]},{"osd":5,"last update":"Sun Nov 26 07:22:53 2023","interfaces":[{"interface":"back","average":{"1min":0.625,"5min":0.61599999999999999,"15min":0.61199999999999999},"min":{"1min":0.46300000000000002,"5min":0.38200000000000001,"15min":0.38200000000000001},"max":{"1min":0.86299999999999999,"5min":0.96199999999999997,"15min":0.96199999999999997},"last":0.63400000000000001},{"interface":"front","average":{"1min":0.60299999999999998,"5min":0.71999999999999997,"15min":0.66700000000000004},"min":{"1min":0.41299999999999998,"5min":0.35599999999999998,"15min":0.35599999999999998},"max":{"1min":0.80800000000000005,"5min":11.148999999999999,"15min":11.148999999999999},"last":0.69399999999999995}]},{"osd":6,"last update":"Sun Nov 26 07:22:53 2023","interfaces":[{"interface":"back","average":{"1min":0.61299999999999999,"5min":0.63900000000000001,"15min":0.74199999999999999},"min":{"1min":0.41099999999999998,"5min":0.372,"15min":0.372},"max":{"1min":0.89200000000000002,"5min":2.8330000000000002,"15min":2.8330000000000002},"last":0.76700000000000002},{"interface":"front","average":{"1min":0.68400000000000005,"5min":0.76800000000000002,"15min":0.92800000000000005},"min":{"1min":0.42699999999999999,"5min":0.41299999999999998,"15min":0.41299999999999998},"max":{"1min":0.93999999999999995,"5min":7.6050000000000004,"15min":7.6050000000000004},"last":0.67800000000000005}]},{"osd":7,"last update":"Sun Nov 26 07:22:53 2023","interfaces":[{"interface":"back","average":{"1min":0.68899999999999995,"5min":0.84699999999999998,"15min":0.85599999999999998},"min":{"1min":0.53900000000000003,"5min":0.44700000000000001,"15min":0.44700000000000001},"max":{"1min":0.871,"5min":12.653,"15min":12.653},"last":0.78200000000000003},{"interface":"front","average":{"1min":0.75900000000000001,"5min":0.79500000000000004,"15min":0.85899999999999999},"min":{"1min":0.56699999999999995,"5min":0.33300000000000002,"15min":0.33300000000000002},"max":{"1min":0.95499999999999996,"5min":3.8319999999999999,"15min":3.8319999999999999},"last":0.81000000000000005}]}]},{"osd":3,"up_from":13,"seq":55834574923,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32200,"kb_used_data":1020,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339640,"statfs":{"total":96636764160,"available":96603791360,"internally_reserved":0,"allocated":1044480,"data_stored":445801,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1592,"internal_metadata":31914440},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Nov 26 07:22:49 2023","interfaces":[{"interface":"back","average":{"1min":0.39000000000000001,"5min":0.41499999999999998,"15min":0.39100000000000001},"min":{"1min":0.218,"5min":0.218,"15min":0.218},"max":{"1min":0.56499999999999995,"5min":0.72799999999999998,"15min":0.72799999999999998},"last":0.44700000000000001},{"interface":"front","average":{"1min":0.439,"5min":0.46000000000000002,"15min":0.442},"min":{"1min":0.254,"5min":0.254,"15min":0.254},"max":{"1min":0.65200000000000002,"5min":0.77300000000000002,"15min":0.77300000000000002},"last":0.46999999999999997}]},{"osd":1,"last update":"Sun Nov 26 07:22:49 2023","interfaces":[{"interface":"back","average":{"1min":0.40999999999999998,"5min":0.436,"15min":0.42599999999999999},"min":{"1min":0.32100000000000001,"5min":0.28799999999999998,"15min":0.28799999999999998},"max":{"1min":0.57199999999999995,"5min":0.752,"15min":0.752},"last":0.59999999999999998},{"interface":"front","average":{"1min":0.439,"5min":0.48899999999999999,"15min":0.48499999999999999},"min":{"1min":0.29899999999999999,"5min":0.29899999999999999,"15min":0.29899999999999999},"max":{"1min":0.69799999999999995,"5min":0.872,"15min":0.872},"last":0.42399999999999999}]},{"osd":2,"last update":"Sun Nov 26 07:22:49 2023","interfaces":[{"interface":"back","average":{"1min":0.52000000000000002,"5min":0.56100000000000005,"15min":0.54200000000000004},"min":{"1min":0.42399999999999999,"5min":0.376,"15min":0.376},"max":{"1min":0.76100000000000001,"5min":0.97199999999999998,"15min":0.97199999999999998},"last":0.67400000000000004},{"interface":"front","average":{"1min":0.48899999999999999,"5min":0.53300000000000003,"15min":0.51400000000000001},"min":{"1min":0.32100000000000001,"5min":0.32100000000000001,"15min":0.32100000000000001},"max":{"1min":0.64200000000000002,"5min":0.89700000000000002,"15min":0.89700000000000002},"last":0.53300000000000003}]},{"osd":4,"last update":"Sun Nov 26 07:22:49 2023","interfaces":[{"interface":"back","average":{"1min":0.65000000000000002,"5min":0.70699999999999996,"15min":0.70599999999999996},"min":{"1min":0.39600000000000002,"5min":0.39600000000000002,"15min":0.39600000000000002},"max":{"1min":0.879,"5min":1.091,"15min":1.091},"last":0.51400000000000001},{"interface":"front","average":{"1min":0.65200000000000002,"5min":0.72299999999999998,"15min":0.72599999999999998},"min":{"1min":0.42499999999999999,"5min":0.39900000000000002,"15min":0.39900000000000002},"max":{"1min":0.85799999999999998,"5min":1.242,"15min":1.242},"last":0.65400000000000003}]},{"osd":5,"last update":"Sun Nov 26 07:22:49 2023","interfaces":[{"interface":"back","average":{"1min":0.60599999999999998,"5min":0.70699999999999996,"15min":0.72799999999999998},"min":{"1min":0.45900000000000002,"5min":0.45900000000000002,"15min":0.45900000000000002},"max":{"1min":0.73999999999999999,"5min":1.105,"15min":1.105},"last":0.48699999999999999},{"interface":"front","average":{"1min":0.66600000000000004,"5min":0.73599999999999999,"15min":0.73699999999999999},"min":{"1min":0.47299999999999998,"5min":0.47299999999999998,"15min":0.47299999999999998},"max":{"1min":0.876,"5min":1.1319999999999999,"15min":1.1319999999999999},"last":0.54800000000000004}]},{"osd":6,"last update":"Sun Nov 26 07:22:49 2023","interfaces":[{"interface":"back","average":{"1min":0.68300000000000005,"5min":0.76800000000000002,"15min":0.79800000000000004},"min":{"1min":0.5,"5min":0.5,"15min":0.5},"max":{"1min":0.90600000000000003,"5min":1.2729999999999999,"15min":1.2729999999999999},"last":0.63100000000000001},{"interface":"front","average":{"1min":0.69299999999999995,"5min":0.75900000000000001,"15min":0.78100000000000003},"min":{"1min":0.48799999999999999,"5min":0.47299999999999998,"15min":0.47299999999999998},"max":{"1min":0.88,"5min":1.204,"15min":1.204},"last":0.61799999999999999}]},{"osd":7,"last update":"Sun Nov 26 07:22:49 2023","interfaces":[{"interface":"back","average":{"1min":0.70999999999999996,"5min":0.75800000000000001,"15min":0.78600000000000003},"min":{"1min":0.437,"5min":0.437,"15min":0.437},"max":{"1min":0.94699999999999995,"5min":1.3600000000000001,"15min":1.3600000000000001},"last":0.58699999999999997},{"interface":"front","average":{"1min":0.72699999999999998,"5min":0.79800000000000004,"15min":0.81599999999999995},"min":{"1min":0.54500000000000004,"5min":0.47299999999999998,"15min":0.47299999999999998},"max":{"1min":0.98999999999999999,"5min":1.617,"15min":1.617},"last":0.69699999999999995}]}]},{"osd":2,"up_from":13,"seq":55834574923,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32200,"kb_used_data":1012,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339640,"statfs":{"total":96636764160,"available":96603791360,"internally_reserved":0,"allocated":1036288,"data_stored":444771,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":7,"apply_latency_ms":7,"commit_latency_ns":7000000,"apply_latency_ns":7000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Nov 26 07:22:50 2023","interfaces":[{"interface":"back","average":{"1min":0.39500000000000002,"5min":0.48799999999999999,"15min":0.63800000000000001},"min":{"1min":0.27100000000000002,"5min":0.27100000000000002,"15min":0.27100000000000002},"max":{"1min":0.624,"5min":4.7030000000000003,"15min":4.7030000000000003},"last":0.41599999999999998},{"interface":"front","average":{"1min":0.41299999999999998,"5min":0.48299999999999998,"15min":0.53500000000000003},"min":{"1min":0.313,"5min":0.27300000000000002,"15min":0.27300000000000002},"max":{"1min":0.64300000000000002,"5min":2.4340000000000002,"15min":2.4340000000000002},"last":0.44400000000000001}]},{"osd":1,"last update":"Sun Nov 26 07:22:50 2023","interfaces":[{"interface":"back","average":{"1min":0.46000000000000002,"5min":0.56799999999999995,"15min":0.76500000000000001},"min":{"1min":0.318,"5min":0.318,"15min":0.318},"max":{"1min":0.81000000000000005,"5min":4.2690000000000001,"15min":4.2690000000000001},"last":0.47799999999999998},{"interface":"front","average":{"1min":0.44500000000000001,"5min":0.54900000000000004,"15min":0.67700000000000005},"min":{"1min":0.35899999999999999,"5min":0.31900000000000001,"15min":0.31900000000000001},"max":{"1min":0.58599999999999997,"5min":3.9940000000000002,"15min":3.9940000000000002},"last":0.52200000000000002}]},{"osd":3,"last update":"Sun Nov 26 07:22:50 2023","interfaces":[{"interface":"back","average":{"1min":0.42099999999999999,"5min":0.50700000000000001,"15min":0.59699999999999998},"min":{"1min":0.29699999999999999,"5min":0.29699999999999999,"15min":0.29699999999999999},"max":{"1min":0.54000000000000004,"5min":4.0300000000000002,"15min":4.0300000000000002},"last":0.55300000000000005},{"interface":"front","average":{"1min":0.50700000000000001,"5min":0.622,"15min":0.81599999999999995},"min":{"1min":0.38600000000000001,"5min":0.36099999999999999,"15min":0.36099999999999999},"max":{"1min":0.77000000000000002,"5min":4.5810000000000004,"15min":4.5810000000000004},"last":0.51100000000000001}]},{"osd":4,"last update":"Sun Nov 26 07:22:50 2023","interfaces":[{"interface":"back","average":{"1min":0.78900000000000003,"5min":0.81699999999999995,"15min":0.92900000000000005},"min":{"1min":0.52600000000000002,"5min":0.51400000000000001,"15min":0.51400000000000001},"max":{"1min":0.91300000000000003,"5min":4.4269999999999996,"15min":4.4269999999999996},"last":0.70999999999999996},{"interface":"front","average":{"1min":0.71499999999999997,"5min":0.82099999999999995,"15min":0.94999999999999996},"min":{"1min":0.54000000000000004,"5min":0.54000000000000004,"15min":0.54000000000000004},"max":{"1min":0.91900000000000004,"5min":4.085,"15min":4.085},"last":0.624}]},{"osd":5,"last update":"Sun Nov 26 07:22:50 2023","interfaces":[{"interface":"back","average":{"1min":0.753,"5min":0.78600000000000003,"15min":0.84399999999999997},"min":{"1min":0.51800000000000002,"5min":0.49199999999999999,"15min":0.49199999999999999},"max":{"1min":0.95399999999999996,"5min":3.9590000000000001,"15min":3.9590000000000001},"last":0.71999999999999997},{"interface":"front","average":{"1min":0.79400000000000004,"5min":0.83499999999999996,"15min":0.93700000000000006},"min":{"1min":0.53700000000000003,"5min":0.499,"15min":0.499},"max":{"1min":1.012,"5min":4.3849999999999998,"15min":4.3849999999999998},"last":0.58399999999999996}]},{"osd":6,"last update":"Sun Nov 26 07:22:50 2023","interfaces":[{"interface":"back","average":{"1min":0.78600000000000003,"5min":0.81999999999999995,"15min":0.93700000000000006},"min":{"1min":0.58099999999999996,"5min":0.51800000000000002,"15min":0.51800000000000002},"max":{"1min":1.083,"5min":4.0449999999999999,"15min":4.0449999999999999},"last":0.65800000000000003},{"interface":"front","average":{"1min":0.749,"5min":0.80100000000000005,"15min":0.879},"min":{"1min":0.49099999999999999,"5min":0.48099999999999998,"15min":0.48099999999999998},"max":{"1min":1.0349999999999999,"5min":3.992,"15min":3.992},"last":0.60499999999999998}]},{"osd":7,"last update":"Sun Nov 26 07:22:50 2023","interfaces":[{"interface":"back","average":{"1min":0.78700000000000003,"5min":0.83599999999999997,"15min":0.96799999999999997},"min":{"1min":0.59599999999999997,"5min":0.52200000000000002,"15min":0.52200000000000002},"max":{"1min":1.0209999999999999,"5min":4.1219999999999999,"15min":4.1219999999999999},"last":0.68700000000000006},{"interface":"front","average":{"1min":0.75,"5min":0.81999999999999995,"15min":0.89800000000000002},"min":{"1min":0.48199999999999998,"5min":0.48199999999999998,"15min":0.48199999999999998},"max":{"1min":0.98799999999999999,"5min":3.9159999999999999,"15min":3.9159999999999999},"last":0.64200000000000002}]}]},{"osd":1,"up_from":13,"seq":55834574922,"num_pgs":29,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32200,"kb_used_data":1020,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339640,"statfs":{"total":96636764160,"available":96603791360,"internally_reserved":0,"allocated":1044480,"data_stored":444383,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":31914439},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Nov 26 07:22:47 2023","interfaces":[{"interface":"back","average":{"1min":0.40600000000000003,"5min":0.40799999999999997,"15min":0.41399999999999998},"min":{"1min":0.28899999999999998,"5min":0.23599999999999999,"15min":0.23599999999999999},"max":{"1min":0.55000000000000004,"5min":0.61199999999999999,"15min":0.61199999999999999},"last":0.27300000000000002},{"interface":"front","average":{"1min":0.45800000000000002,"5min":0.45700000000000002,"15min":0.47599999999999998},"min":{"1min":0.28399999999999997,"5min":0.26700000000000002,"15min":0.26700000000000002},"max":{"1min":0.749,"5min":0.78700000000000003,"15min":0.78700000000000003},"last":0.45300000000000001}]},{"osd":2,"last update":"Sun Nov 26 07:22:47 2023","interfaces":[{"interface":"back","average":{"1min":0.45000000000000001,"5min":0.442,"15min":0.45000000000000001},"min":{"1min":0.34499999999999997,"5min":0.27800000000000002,"15min":0.27800000000000002},"max":{"1min":0.71399999999999997,"5min":0.71399999999999997,"15min":0.71399999999999997},"last":0.47199999999999998},{"interface":"front","average":{"1min":0.47199999999999998,"5min":0.49099999999999999,"15min":0.502},"min":{"1min":0.36599999999999999,"5min":0.27700000000000002,"15min":0.27700000000000002},"max":{"1min":0.63200000000000001,"5min":0.80600000000000005,"15min":0.80600000000000005},"last":0.318}]},{"osd":3,"last update":"Sun Nov 26 07:22:47 2023","interfaces":[{"interface":"back","average":{"1min":0.53200000000000003,"5min":0.52900000000000003,"15min":0.52900000000000003},"min":{"1min":0.441,"5min":0.38400000000000001,"15min":0.38400000000000001},"max":{"1min":0.65000000000000002,"5min":0.748,"15min":0.748},"last":0.36099999999999999},{"interface":"front","average":{"1min":0.496,"5min":0.499,"15min":0.497},"min":{"1min":0.35799999999999998,"5min":0.31,"15min":0.31},"max":{"1min":0.66700000000000004,"5min":0.80700000000000005,"15min":0.80700000000000005},"last":0.39500000000000002}]},{"osd":4,"last update":"Sun Nov 26 07:22:47 2023","interfaces":[{"interface":"back","average":{"1min":0.74199999999999999,"5min":0.72899999999999998,"15min":0.70499999999999996},"min":{"1min":0.52400000000000002,"5min":0.41299999999999998,"15min":0.41299999999999998},"max":{"1min":1.0640000000000001,"5min":1.1000000000000001,"15min":1.1000000000000001},"last":0.84199999999999997},{"interface":"front","average":{"1min":0.66200000000000003,"5min":0.71099999999999997,"15min":0.68999999999999995},"min":{"1min":0.44900000000000001,"5min":0.44500000000000001,"15min":0.44500000000000001},"max":{"1min":1.0009999999999999,"5min":1.131,"15min":1.131},"last":0.69999999999999996}]},{"osd":5,"last update":"Sun Nov 26 07:22:47 2023","interfaces":[{"interface":"back","average":{"1min":0.74399999999999999,"5min":0.73599999999999999,"15min":0.72399999999999998},"min":{"1min":0.54700000000000004,"5min":0.49299999999999999,"15min":0.49299999999999999},"max":{"1min":1.0409999999999999,"5min":1.1919999999999999,"15min":1.1919999999999999},"last":0.67600000000000005},{"interface":"front","average":{"1min":0.76500000000000001,"5min":0.73699999999999999,"15min":0.72299999999999998},"min":{"1min":0.47099999999999997,"5min":0.47099999999999997,"15min":0.47099999999999997},"max":{"1min":1.0389999999999999,"5min":1.0609999999999999,"15min":1.0609999999999999},"last":0.78500000000000003}]},{"osd":6,"last update":"Sun Nov 26 07:22:47 2023","interfaces":[{"interface":"back","average":{"1min":0.755,"5min":0.74199999999999999,"15min":0.71599999999999997},"min":{"1min":0.53200000000000003,"5min":0.50600000000000001,"15min":0.50600000000000001},"max":{"1min":0.98699999999999999,"5min":1.0369999999999999,"15min":1.0369999999999999},"last":0.93100000000000005},{"interface":"front","average":{"1min":0.72799999999999998,"5min":0.73899999999999999,"15min":0.71299999999999997},"min":{"1min":0.46999999999999997,"5min":0.46999999999999997,"15min":0.46999999999999997},"max":{"1min":1.079,"5min":1.1339999999999999,"15min":1.1339999999999999},"last":0.94499999999999995}]},{"osd":7,"last update":"Sun Nov 26 07:22:47 2023","interfaces":[{"interface":"back","average":{"1min":0.73999999999999999,"5min":0.746,"15min":0.72499999999999998},"min":{"1min":0.55400000000000005,"5min":0.50600000000000001,"15min":0.50600000000000001},"max":{"1min":1.0149999999999999,"5min":1.0149999999999999,"15min":1.0149999999999999},"last":0.76200000000000001},{"interface":"front","average":{"1min":0.73199999999999998,"5min":0.74199999999999999,"15min":0.749},"min":{"1min":0.53700000000000003,"5min":0.48499999999999999,"15min":0.48499999999999999},"max":{"1min":0.95299999999999996,"5min":1.024,"15min":1.024},"last":0.86499999999999999}]}]},{"osd":0,"up_from":13,"seq":55834574922,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32776,"kb_used_data":1592,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339064,"statfs":{"total":96636764160,"available":96603201536,"internally_reserved":0,"allocated":1630208,"data_stored":1034591,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Sun Nov 26 07:22:46 2023","interfaces":[{"interface":"back","average":{"1min":0.46600000000000003,"5min":0.433,"15min":0.43099999999999999},"min":{"1min":0.29199999999999998,"5min":0.27000000000000002,"15min":0.27000000000000002},"max":{"1min":1.26,"5min":1.26,"15min":1.26},"last":0.47099999999999997},{"interface":"front","average":{"1min":0.499,"5min":0.47199999999999998,"15min":0.47899999999999998},"min":{"1min":0.30199999999999999,"5min":0.29899999999999999,"15min":0.29899999999999999},"max":{"1min":1.212,"5min":1.212,"15min":1.212},"last":0.33300000000000002}]},{"osd":2,"last update":"Sun Nov 26 07:22:46 2023","interfaces":[{"interface":"back","average":{"1min":0.51400000000000001,"5min":0.51300000000000001,"15min":0.52900000000000003},"min":{"1min":0.38500000000000001,"5min":0.34699999999999998,"15min":0.34699999999999998},"max":{"1min":0.75600000000000001,"5min":0.84199999999999997,"15min":0.84199999999999997},"last":0.55600000000000005},{"interface":"front","average":{"1min":0.54700000000000004,"5min":0.55300000000000005,"15min":0.56000000000000005},"min":{"1min":0.40100000000000002,"5min":0.36399999999999999,"15min":0.36399999999999999},"max":{"1min":1.0800000000000001,"5min":1.0800000000000001,"15min":1.0800000000000001},"last":0.59199999999999997}]},{"osd":3,"last update":"Sun Nov 26 07:22:46 2023","interfaces":[{"interface":"back","average":{"1min":0.51700000000000002,"5min":0.51900000000000002,"15min":0.52700000000000002},"min":{"1min":0.38700000000000001,"5min":0.34999999999999998,"15min":0.34999999999999998},"max":{"1min":0.873,"5min":1.3149999999999999,"15min":1.3149999999999999},"last":0.41799999999999998},{"interface":"front","average":{"1min":0.50800000000000001,"5min":0.53300000000000003,"15min":0.55200000000000005},"min":{"1min":0.39700000000000002,"5min":0.371,"15min":0.371},"max":{"1min":0.63,"5min":1.2509999999999999,"15min":1.2509999999999999},"last":0.45600000000000002}]},{"osd":4,"last update":"Sun Nov 26 07:22:46 2023","interfaces":[{"interface":"back","average":{"1min":0.747,"5min":0.747,"15min":0.755},"min":{"1min":0.59099999999999997,"5min":0.44700000000000001,"15min":0.44700000000000001},"max":{"1min":0.98799999999999999,"5min":1.2010000000000001,"15min":1.2010000000000001},"last":0.71299999999999997},{"interface":"front","average":{"1min":0.78100000000000003,"5min":0.78700000000000003,"15min":0.78800000000000003},"min":{"1min":0.63800000000000001,"5min":0.51100000000000001,"15min":0.51100000000000001},"max":{"1min":1.0529999999999999,"5min":1.381,"15min":1.381},"last":0.73199999999999998}]},{"osd":5,"last update":"Sun Nov 26 07:22:46 2023","interfaces":[{"interface":"back","average":{"1min":0.73499999999999999,"5min":0.75600000000000001,"15min":0.75800000000000001},"min":{"1min":0.49399999999999999,"5min":0.42199999999999999,"15min":0.42199999999999999},"max":{"1min":1.1819999999999999,"5min":1.1819999999999999,"15min":1.1819999999999999},"last":0.76100000000000001},{"interface":"front","average":{"1min":0.79200000000000004,"5min":0.80500000000000005,"15min":0.79900000000000004},"min":{"1min":0.60099999999999998,"5min":0.53600000000000003,"15min":0.53600000000000003},"max":{"1min":1.143,"5min":1.399,"15min":1.399},"last":0.77400000000000002}]},{"osd":6,"last update":"Sun Nov 26 07:22:46 2023","interfaces":[{"interface":"back","average":{"1min":0.67300000000000004,"5min":0.70399999999999996,"15min":0.74399999999999999},"min":{"1min":0.54900000000000004,"5min":0.50800000000000001,"15min":0.50800000000000001},"max":{"1min":1.0069999999999999,"5min":1.3029999999999999,"15min":1.3029999999999999},"last":0.67100000000000004},{"interface":"front","average":{"1min":0.70999999999999996,"5min":0.745,"15min":0.76100000000000001},"min":{"1min":0.51200000000000001,"5min":0.47199999999999998,"15min":0.47199999999999998},"max":{"1min":0.997,"5min":1.3600000000000001,"15min":1.3600000000000001},"last":0.81499999999999995}]},{"osd":7,"last update":"Sun Nov 26 07:22:46 2023","interfaces":[{"interface":"back","average":{"1min":0.75,"5min":0.77500000000000002,"15min":0.79900000000000004},"min":{"1min":0.62,"5min":0.56899999999999995,"15min":0.56899999999999995},"max":{"1min":0.88500000000000001,"5min":1.177,"15min":1.177},"last":0.79000000000000004},{"interface":"front","average":{"1min":0.78800000000000003,"5min":0.76500000000000001,"15min":0.78700000000000003},"min":{"1min":0.47899999999999998,"5min":0.47899999999999998,"15min":0.47899999999999998},"max":{"1min":1.238,"5min":1.2390000000000001,"15min":1.2390000000000001},"last":0.65000000000000002}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":34,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":194,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1634,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":694,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":70,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-11-26T07:23:34.099 INFO:tasks.cephfs.filesystem.ceph_manager:clean! 2023-11-26T07:23:34.100 DEBUG:teuthology.orchestra.run.smithi143:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth caps client.0 mon 'allow r' osd 'allow rw tag cephfs data=cephfs' mds allow 2023-11-26T07:23:34.291 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:34.287+0000 7efe07217640 1 Processor -- start 2023-11-26T07:23:34.291 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:34.287+0000 7efe07217640 1 -- start start 2023-11-26T07:23:34.291 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:34.287+0000 7efe07217640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7efe000d3d30 0x7efe000cda90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:23:34.292 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:34.287+0000 7efe07217640 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7efe00160f60 0x7efe001613f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:23:34.292 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:34.287+0000 7efe07217640 1 -- --> v1:172.21.15.143:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7efe000d4170 con 0x7efe00160850 2023-11-26T07:23:34.292 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:34.287+0000 7efe07217640 1 -- --> v2:172.21.15.188:3300/0 -- mon_getmap magic: 0 v1 -- 0x7efe00161930 con 0x7efe00160f60 2023-11-26T07:23:34.292 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:34.287+0000 7efe07217640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7efe000cdfd0 con 0x7efe000d3d30 2023-11-26T07:23:34.292 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:34.287+0000 7efe06215640 1 --1- >> v1:172.21.15.143:6789/0 conn(0x7efe00160850 0x7efe000cf330 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.143:6789/0 says I am v1:172.21.15.143:43240/0 (socket says 172.21.15.143:43240) 2023-11-26T07:23:34.292 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:34.287+0000 7efe06215640 1 -- 172.21.15.143:0/196895035 learned_addr learned my addr 172.21.15.143:0/196895035 (peer_addr_for_me v1:172.21.15.143:0/0) 2023-11-26T07:23:34.292 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:34.287+0000 7efe06a16640 1 --2- 172.21.15.143:0/196895035 >> v2:172.21.15.188:3300/0 conn(0x7efe00160f60 0x7efe001613f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:23:34.292 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:34.287+0000 7efe05213640 1 -- 172.21.15.143:0/196895035 <== mon.0 v1:172.21.15.143:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1169462465 0 0) 0x7efe000d4170 con 0x7efe00160850 2023-11-26T07:23:34.292 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:34.287+0000 7efe05a14640 1 --2- 172.21.15.143:0/196895035 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7efe000d3d30 0x7efe000cda90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:23:34.292 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:34.287+0000 7efe05213640 1 -- 172.21.15.143:0/196895035 --> v1:172.21.15.143:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7efdec003530 con 0x7efe00160850 2023-11-26T07:23:34.292 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:34.287+0000 7efe05213640 1 -- 172.21.15.143:0/196895035 <== mon.0 v1:172.21.15.143:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2678037030 0 0) 0x7efdec003530 con 0x7efe00160850 2023-11-26T07:23:34.292 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:34.287+0000 7efe05213640 1 -- 172.21.15.143:0/196895035 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7efe000d3d30 msgr2=0x7efe000cda90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-26T07:23:34.292 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:34.287+0000 7efe05213640 1 --2- 172.21.15.143:0/196895035 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7efe000d3d30 0x7efe000cda90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:23:34.293 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:34.287+0000 7efe05213640 1 -- 172.21.15.143:0/196895035 >> v2:172.21.15.188:3300/0 conn(0x7efe00160f60 msgr2=0x7efe001613f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-26T07:23:34.293 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:34.287+0000 7efe05213640 1 --2- 172.21.15.143:0/196895035 >> v2:172.21.15.188:3300/0 conn(0x7efe00160f60 0x7efe001613f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:23:34.293 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:34.287+0000 7efe05213640 1 -- 172.21.15.143:0/196895035 --> v1:172.21.15.143:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efe000ce290 con 0x7efe00160850 2023-11-26T07:23:34.293 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:34.287+0000 7efe05213640 1 -- 172.21.15.143:0/196895035 <== mon.0 v1:172.21.15.143:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2137882523 0 0) 0x7efdfc001680 con 0x7efe00160850 2023-11-26T07:23:34.293 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:34.287+0000 7efe05213640 1 -- 172.21.15.143:0/196895035 >> v1:172.21.15.143:6789/0 conn(0x7efe00160850 legacy=0x7efe000cf330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:23:34.293 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:34.287+0000 7efe05213640 1 --2- 172.21.15.143:0/196895035 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7efdec003980 0x7efdec003da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:23:34.293 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:34.287+0000 7efe05213640 1 --2- 172.21.15.143:0/196895035 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7efdec0047d0 0x7efdec004c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:23:34.293 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:34.287+0000 7efe05213640 1 --2- 172.21.15.143:0/196895035 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7efe000d3d30 0x7efdec005190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:23:34.293 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:34.287+0000 7efe05213640 1 -- 172.21.15.143:0/196895035 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efe00161930 con 0x7efdec0047d0 2023-11-26T07:23:34.293 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:34.287+0000 7efe05213640 1 -- 172.21.15.143:0/196895035 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efe000cdfd0 con 0x7efdec003980 2023-11-26T07:23:34.293 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:34.287+0000 7efe05213640 1 -- 172.21.15.143:0/196895035 --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7efdec00b990 con 0x7efe000d3d30 2023-11-26T07:23:34.293 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:34.287+0000 7efe05a14640 1 --2- 172.21.15.143:0/196895035 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7efdec0047d0 0x7efdec004c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:23:34.294 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:34.287+0000 7efe06a16640 1 --2- 172.21.15.143:0/196895035 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7efe000d3d30 0x7efdec005190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:23:34.294 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:34.287+0000 7efe06215640 1 --2- 172.21.15.143:0/196895035 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7efdec003980 0x7efdec003da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:23:34.294 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:34.291+0000 7efe05a14640 1 -- 172.21.15.143:0/196895035 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7efe000d3d30 msgr2=0x7efdec005190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-26T07:23:34.294 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:34.291+0000 7efe05a14640 1 --2- 172.21.15.143:0/196895035 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7efe000d3d30 0x7efdec005190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:23:34.294 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:34.291+0000 7efe05a14640 1 -- 172.21.15.143:0/196895035 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7efdec003980 msgr2=0x7efdec003da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-26T07:23:34.294 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:34.291+0000 7efe05a14640 1 --2- 172.21.15.143:0/196895035 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7efdec003980 0x7efdec003da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:23:34.294 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:34.291+0000 7efe05a14640 1 -- 172.21.15.143:0/196895035 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efdec003530 con 0x7efdec0047d0 2023-11-26T07:23:34.295 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:34.291+0000 7efe05a14640 1 --2- 172.21.15.143:0/196895035 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7efdec0047d0 0x7efdec004c50 secure :-1 s=READY pgs=1160 cs=0 l=1 rev1=1 crypto rx=0x7efdf0009ae0 tx=0x7efdf001c9f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c6200e9aacc0a3e0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:23:34.295 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:34.291+0000 7efe05213640 1 -- 172.21.15.143:0/196895035 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efdf00a6460 con 0x7efdec0047d0 2023-11-26T07:23:34.295 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:34.291+0000 7efe05213640 1 -- 172.21.15.143:0/196895035 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7efdf00a65c0 con 0x7efdec0047d0 2023-11-26T07:23:34.295 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:34.291+0000 7efe05213640 1 -- 172.21.15.143:0/196895035 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efdf00a6970 con 0x7efdec0047d0 2023-11-26T07:23:34.296 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:34.291+0000 7efe07217640 1 -- 172.21.15.143:0/196895035 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7efdec0047d0 msgr2=0x7efdec004c50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:23:34.296 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:34.291+0000 7efe07217640 1 --2- 172.21.15.143:0/196895035 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7efdec0047d0 0x7efdec004c50 secure :-1 s=READY pgs=1160 cs=0 l=1 rev1=1 crypto rx=0x7efdf0009ae0 tx=0x7efdf001c9f0 comp rx=0 tx=0).stop 2023-11-26T07:23:34.296 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:34.291+0000 7efe06a16640 1 -- 172.21.15.143:0/196895035 reap_dead start 2023-11-26T07:23:34.296 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:34.295+0000 7efe07217640 1 -- 172.21.15.143:0/196895035 shutdown_connections 2023-11-26T07:23:34.296 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:34.295+0000 7efe07217640 1 -- 172.21.15.143:0/196895035 >> 172.21.15.143:0/196895035 conn(0x7efe0009c5a0 msgr2=0x7efe000d3260 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:23:34.296 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:34.295+0000 7efe07217640 1 -- 172.21.15.143:0/196895035 shutdown_connections 2023-11-26T07:23:34.296 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:34.295+0000 7efe07217640 1 -- 172.21.15.143:0/196895035 wait complete. 2023-11-26T07:23:34.296 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:34.295+0000 7efe07217640 1 Processor -- start 2023-11-26T07:23:34.297 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:34.295+0000 7efe07217640 1 -- start start 2023-11-26T07:23:34.298 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:34.295+0000 7efe07217640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7efdec0047d0 0x7efe000b0320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:23:34.298 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:34.295+0000 7efe07217640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7efe000ca330 0x7efe001474f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:23:34.299 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:34.295+0000 7efe07217640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7efe00147a30 0x7efe000b0e60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:23:34.299 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:34.295+0000 7efe07217640 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efe000b8ac0 con 0x7efe000ca330 2023-11-26T07:23:34.299 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:34.295+0000 7efe07217640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efe000b09d0 con 0x7efdec0047d0 2023-11-26T07:23:34.299 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:34.295+0000 7efe07217640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7efe000ccf90 con 0x7efe00147a30 2023-11-26T07:23:34.299 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:34.295+0000 7efe05a14640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7efe000ca330 0x7efe001474f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:23:34.299 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:34.295+0000 7efe05a14640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7efe000ca330 0x7efe001474f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:46448/0 (socket says 172.21.15.143:46448) 2023-11-26T07:23:34.299 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:34.295+0000 7efe05a14640 1 -- 172.21.15.143:0/3396783847 learned_addr learned my addr 172.21.15.143:0/3396783847 (peer_addr_for_me v2:172.21.15.143:0/0) 2023-11-26T07:23:34.299 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:34.295+0000 7efe06215640 1 --2- 172.21.15.143:0/3396783847 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7efdec0047d0 0x7efe000b0320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:23:34.299 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:34.295+0000 7efe06a16640 1 --2- 172.21.15.143:0/3396783847 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7efe00147a30 0x7efe000b0e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:23:34.299 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:34.295+0000 7efe05a14640 1 -- 172.21.15.143:0/3396783847 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7efe00147a30 msgr2=0x7efe000b0e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:23:34.299 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:34.295+0000 7efe05a14640 1 --2- 172.21.15.143:0/3396783847 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7efe00147a30 0x7efe000b0e60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:23:34.299 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:34.295+0000 7efe05a14640 1 -- 172.21.15.143:0/3396783847 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7efdec0047d0 msgr2=0x7efe000b0320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:23:34.300 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:34.295+0000 7efe05a14640 1 --2- 172.21.15.143:0/3396783847 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7efdec0047d0 0x7efe000b0320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:23:34.300 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:34.295+0000 7efe05a14640 1 -- 172.21.15.143:0/3396783847 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efe000cd270 con 0x7efe000ca330 2023-11-26T07:23:34.300 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:34.295+0000 7efe06215640 1 --2- 172.21.15.143:0/3396783847 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7efdec0047d0 0x7efe000b0320 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-26T07:23:34.300 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:34.295+0000 7efe05a14640 1 --2- 172.21.15.143:0/3396783847 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7efe000ca330 0x7efe001474f0 secure :-1 s=READY pgs=1161 cs=0 l=1 rev1=1 crypto rx=0x7efdf0009ae0 tx=0x7efdec005800 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:23:34.300 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:34.299+0000 7efdf6ffd640 1 -- 172.21.15.143:0/3396783847 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efdf00b5070 con 0x7efe000ca330 2023-11-26T07:23:34.300 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:34.299+0000 7efdf6ffd640 1 -- 172.21.15.143:0/3396783847 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7efdf000f000 con 0x7efe000ca330 2023-11-26T07:23:34.300 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:34.299+0000 7efdf6ffd640 1 -- 172.21.15.143:0/3396783847 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efdf00a7440 con 0x7efe000ca330 2023-11-26T07:23:34.300 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:34.299+0000 7efe07217640 1 -- 172.21.15.143:0/3396783847 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7efe000cd550 con 0x7efe000ca330 2023-11-26T07:23:34.300 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:34.299+0000 7efe07217640 1 -- 172.21.15.143:0/3396783847 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7efe000ca8a0 con 0x7efe000ca330 2023-11-26T07:23:34.301 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:34.299+0000 7efe07217640 1 -- 172.21.15.143:0/3396783847 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7efe00067500 con 0x7efe000ca330 2023-11-26T07:23:34.301 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:34.299+0000 7efdf6ffd640 1 -- 172.21.15.143:0/3396783847 <== mon.0 v2:172.21.15.143:3300/0 4 ==== mgrmap(e 16) v1 ==== 85954+0+0 (secure 0 0 0) 0x7efdf00b9070 con 0x7efe000ca330 2023-11-26T07:23:34.301 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:34.299+0000 7efdf6ffd640 1 --2- 172.21.15.143:0/3396783847 >> [v2:172.21.15.188:6836/3401128553,v1:172.21.15.188:6837/3401128553] conn(0x7efde4066590 0x7efde4068a50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:23:34.301 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:34.299+0000 7efdf6ffd640 1 -- 172.21.15.143:0/3396783847 <== mon.0 v2:172.21.15.143:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 5709+0+0 (secure 0 0 0) 0x7efdf011dcb0 con 0x7efe000ca330 2023-11-26T07:23:34.302 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:34.299+0000 7efe06215640 1 --2- 172.21.15.143:0/3396783847 >> [v2:172.21.15.188:6836/3401128553,v1:172.21.15.188:6837/3401128553] conn(0x7efde4066590 0x7efde4068a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:23:34.302 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:34.303+0000 7efe06215640 1 --2- 172.21.15.143:0/3396783847 >> [v2:172.21.15.188:6836/3401128553,v1:172.21.15.188:6837/3401128553] conn(0x7efde4066590 0x7efde4068a50 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7efdfc000c10 tx=0x7efdfc009040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:23:34.305 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:34.303+0000 7efdf6ffd640 1 -- 172.21.15.143:0/3396783847 <== mon.0 v2:172.21.15.143:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172649 (secure 0 0 0) 0x7efdf00eaf90 con 0x7efe000ca330 2023-11-26T07:23:34.506 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:34.503+0000 7efe07217640 1 -- 172.21.15.143:0/3396783847 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]} v 0) v1 -- 0x7efe0009ca80 con 0x7efe000ca330 2023-11-26T07:23:34.509 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:34.507+0000 7efdf6ffd640 1 -- 172.21.15.143:0/3396783847 <== mon.0 v2:172.21.15.143:3300/0 7 ==== mon_command_ack([{"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]}]=0 updated caps for client.0 v19) v1 ==== 190+0+147 (secure 0 0 0) 0x7efdf00c0070 con 0x7efe000ca330 2023-11-26T07:23:34.509 INFO:teuthology.orchestra.run.smithi143.stdout:[client.0] 2023-11-26T07:23:34.509 INFO:teuthology.orchestra.run.smithi143.stdout: key = AQDm8GJlWMh9FBAAslVzi6fm4/wt7vr/rHCHHQ== 2023-11-26T07:23:34.509 INFO:teuthology.orchestra.run.smithi143.stdout: caps mds = "allow" 2023-11-26T07:23:34.509 INFO:teuthology.orchestra.run.smithi143.stdout: caps mon = "allow r" 2023-11-26T07:23:34.509 INFO:teuthology.orchestra.run.smithi143.stdout: caps osd = "allow rw tag cephfs data=cephfs" 2023-11-26T07:23:34.510 INFO:teuthology.orchestra.run.smithi143.stderr:updated caps for client.0 2023-11-26T07:23:34.511 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:34.511+0000 7efe07217640 1 -- 172.21.15.143:0/3396783847 >> [v2:172.21.15.188:6836/3401128553,v1:172.21.15.188:6837/3401128553] conn(0x7efde4066590 msgr2=0x7efde4068a50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:23:34.511 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:34.511+0000 7efe07217640 1 --2- 172.21.15.143:0/3396783847 >> [v2:172.21.15.188:6836/3401128553,v1:172.21.15.188:6837/3401128553] conn(0x7efde4066590 0x7efde4068a50 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7efdfc000c10 tx=0x7efdfc009040 comp rx=0 tx=0).stop 2023-11-26T07:23:34.511 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:34.511+0000 7efe07217640 1 -- 172.21.15.143:0/3396783847 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7efe000ca330 msgr2=0x7efe001474f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:23:34.511 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:34.511+0000 7efe07217640 1 --2- 172.21.15.143:0/3396783847 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7efe000ca330 0x7efe001474f0 secure :-1 s=READY pgs=1161 cs=0 l=1 rev1=1 crypto rx=0x7efdf0009ae0 tx=0x7efdec005800 comp rx=0 tx=0).stop 2023-11-26T07:23:34.512 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:34.511+0000 7efe07217640 1 -- 172.21.15.143:0/3396783847 shutdown_connections 2023-11-26T07:23:34.512 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:34.511+0000 7efe07217640 1 --2- 172.21.15.143:0/3396783847 >> [v2:172.21.15.188:6836/3401128553,v1:172.21.15.188:6837/3401128553] conn(0x7efde4066590 0x7efde4068a50 unknown :-1 s=CLOSED pgs=65 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:23:34.512 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:34.511+0000 7efe07217640 1 --2- 172.21.15.143:0/3396783847 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7efe00147a30 0x7efe000b0e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:23:34.512 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:34.511+0000 7efe07217640 1 --2- 172.21.15.143:0/3396783847 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7efe000ca330 0x7efe001474f0 unknown :-1 s=CLOSED pgs=1161 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:23:34.512 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:34.511+0000 7efe07217640 1 --2- 172.21.15.143:0/3396783847 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7efdec0047d0 0x7efe000b0320 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:23:34.512 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:34.511+0000 7efe07217640 1 -- 172.21.15.143:0/3396783847 >> 172.21.15.143:0/3396783847 conn(0x7efe0009c5a0 msgr2=0x7efe0006b1e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:23:34.512 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:34.511+0000 7efe07217640 1 -- 172.21.15.143:0/3396783847 shutdown_connections 2023-11-26T07:23:34.512 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:34.511+0000 7efe07217640 1 -- 172.21.15.143:0/3396783847 wait complete. 2023-11-26T07:23:34.530 DEBUG:teuthology.orchestra.run.smithi143:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-11-26T07:23:34.610 INFO:tasks.ceph.mds.b.smithi188.stderr:2023-11-26T07:23:34.605+0000 7ff8cbe33640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2023-11-26T07:23:34.668 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:34.667+0000 7ff2dc679640 1 Processor -- start 2023-11-26T07:23:34.668 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:34.667+0000 7ff2dc679640 1 -- start start 2023-11-26T07:23:34.670 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:34.667+0000 7ff2dc679640 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7ff2d413e290 0x7ff2d40cf280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:23:34.670 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:34.667+0000 7ff2dc679640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7ff2d40cf7c0 0x7ff2d415efa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:23:34.670 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:34.667+0000 7ff2dc679640 1 -- --> v1:172.21.15.143:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff2d406c520 con 0x7ff2d415f4e0 2023-11-26T07:23:34.670 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:34.667+0000 7ff2dc679640 1 -- --> v2:172.21.15.143:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff2d40cfc40 con 0x7ff2d413e290 2023-11-26T07:23:34.670 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:34.667+0000 7ff2dc679640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff2d406c800 con 0x7ff2d40cf7c0 2023-11-26T07:23:34.671 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:34.667+0000 7ff2da415640 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7ff2d413e290 0x7ff2d40cf280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:23:34.671 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:34.667+0000 7ff2dac16640 1 --1- >> v1:172.21.15.143:6789/0 conn(0x7ff2d415f4e0 0x7ff2d41408e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.143:6789/0 says I am v1:172.21.15.143:47972/0 (socket says 172.21.15.143:47972) 2023-11-26T07:23:34.671 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:34.667+0000 7ff2dac16640 1 -- 172.21.15.143:0/3154888038 learned_addr learned my addr 172.21.15.143:0/3154888038 (peer_addr_for_me v1:172.21.15.143:0/0) 2023-11-26T07:23:34.671 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:34.667+0000 7ff2d9c14640 1 --2- 172.21.15.143:0/3154888038 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7ff2d40cf7c0 0x7ff2d415efa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:23:34.671 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:34.667+0000 7ff2da415640 1 -- 172.21.15.143:0/3154888038 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7ff2d40cf7c0 msgr2=0x7ff2d415efa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-26T07:23:34.671 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:34.667+0000 7ff2da415640 1 --2- 172.21.15.143:0/3154888038 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7ff2d40cf7c0 0x7ff2d415efa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:23:34.671 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:34.667+0000 7ff2da415640 1 -- 172.21.15.143:0/3154888038 >> v1:172.21.15.143:6789/0 conn(0x7ff2d415f4e0 legacy=0x7ff2d41408e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:23:34.671 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:34.667+0000 7ff2da415640 1 -- 172.21.15.143:0/3154888038 --> v2:172.21.15.143:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff2d406cae0 con 0x7ff2d413e290 2023-11-26T07:23:34.673 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:34.671+0000 7ff2da415640 1 --2- 172.21.15.143:0/3154888038 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7ff2d413e290 0x7ff2d40cf280 secure :-1 s=READY pgs=1163 cs=0 l=1 rev1=1 crypto rx=0x7ff2d00097c0 tx=0x7ff2d001c450 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1097efbf0b9a5ce2 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:23:34.673 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:34.671+0000 7ff2d9413640 1 -- 172.21.15.143:0/3154888038 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff2d00ab650 con 0x7ff2d413e290 2023-11-26T07:23:34.673 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:34.671+0000 7ff2d9413640 1 -- 172.21.15.143:0/3154888038 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff2d00ab7b0 con 0x7ff2d413e290 2023-11-26T07:23:34.673 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:34.671+0000 7ff2d9413640 1 -- 172.21.15.143:0/3154888038 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff2d00abb40 con 0x7ff2d413e290 2023-11-26T07:23:34.674 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:34.671+0000 7ff2dc679640 1 -- 172.21.15.143:0/3154888038 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7ff2d413e290 msgr2=0x7ff2d40cf280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:23:34.674 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:34.671+0000 7ff2dc679640 1 --2- 172.21.15.143:0/3154888038 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7ff2d413e290 0x7ff2d40cf280 secure :-1 s=READY pgs=1163 cs=0 l=1 rev1=1 crypto rx=0x7ff2d00097c0 tx=0x7ff2d001c450 comp rx=0 tx=0).stop 2023-11-26T07:23:34.674 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:34.671+0000 7ff2dc679640 1 -- 172.21.15.143:0/3154888038 shutdown_connections 2023-11-26T07:23:34.675 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:34.671+0000 7ff2dc679640 1 --2- 172.21.15.143:0/3154888038 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7ff2d40cf7c0 0x7ff2d415efa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:23:34.675 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:34.671+0000 7ff2dc679640 1 --2- 172.21.15.143:0/3154888038 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7ff2d413e290 0x7ff2d40cf280 unknown :-1 s=CLOSED pgs=1163 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:23:34.675 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:34.671+0000 7ff2dc679640 1 -- 172.21.15.143:0/3154888038 >> 172.21.15.143:0/3154888038 conn(0x7ff2d406a890 msgr2=0x7ff2d406aca0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:23:34.675 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:34.671+0000 7ff2dc679640 1 -- 172.21.15.143:0/3154888038 shutdown_connections 2023-11-26T07:23:34.675 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:34.671+0000 7ff2dc679640 1 -- 172.21.15.143:0/3154888038 wait complete. 2023-11-26T07:23:34.676 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:34.675+0000 7ff2dc679640 1 Processor -- start 2023-11-26T07:23:34.676 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:34.675+0000 7ff2dc679640 1 -- start start 2023-11-26T07:23:34.677 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:34.675+0000 7ff2dc679640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7ff2d40cf7c0 0x7ff2d41418f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:23:34.677 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:34.675+0000 7ff2dc679640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7ff2d413e290 0x7ff2d4137890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:23:34.677 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:34.675+0000 7ff2dc679640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7ff2d415f4e0 0x7ff2d415a150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:23:34.678 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:34.675+0000 7ff2dc679640 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff2d4137dd0 con 0x7ff2d415f4e0 2023-11-26T07:23:34.678 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:34.675+0000 7ff2dc679640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff2d4143770 con 0x7ff2d413e290 2023-11-26T07:23:34.678 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:34.675+0000 7ff2dc679640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff2d4143a50 con 0x7ff2d40cf7c0 2023-11-26T07:23:34.678 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:34.675+0000 7ff2dac16640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7ff2d415f4e0 0x7ff2d415a150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:23:34.678 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:34.675+0000 7ff2dac16640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7ff2d415f4e0 0x7ff2d415a150 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:55662/0 (socket says 172.21.15.143:55662) 2023-11-26T07:23:34.678 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:34.675+0000 7ff2dac16640 1 -- 172.21.15.143:0/2952752158 learned_addr learned my addr 172.21.15.143:0/2952752158 (peer_addr_for_me v2:172.21.15.143:0/0) 2023-11-26T07:23:34.678 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:34.675+0000 7ff2da415640 1 --2- 172.21.15.143:0/2952752158 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7ff2d40cf7c0 0x7ff2d41418f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:23:34.678 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:34.675+0000 7ff2d9c14640 1 --2- 172.21.15.143:0/2952752158 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7ff2d413e290 0x7ff2d4137890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:23:34.678 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:34.675+0000 7ff2dac16640 1 -- 172.21.15.143:0/2952752158 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7ff2d40cf7c0 msgr2=0x7ff2d41418f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:23:34.678 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:34.675+0000 7ff2dac16640 1 --2- 172.21.15.143:0/2952752158 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7ff2d40cf7c0 0x7ff2d41418f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:23:34.678 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:34.675+0000 7ff2dac16640 1 -- 172.21.15.143:0/2952752158 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7ff2d413e290 msgr2=0x7ff2d4137890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:23:34.679 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:34.675+0000 7ff2dac16640 1 --2- 172.21.15.143:0/2952752158 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7ff2d413e290 0x7ff2d4137890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:23:34.679 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:34.675+0000 7ff2dac16640 1 -- 172.21.15.143:0/2952752158 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff2d0009420 con 0x7ff2d415f4e0 2023-11-26T07:23:34.679 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:34.675+0000 7ff2dac16640 1 --2- 172.21.15.143:0/2952752158 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7ff2d415f4e0 0x7ff2d415a150 secure :-1 s=READY pgs=1164 cs=0 l=1 rev1=1 crypto rx=0x7ff2cc00ca80 tx=0x7ff2cc00cf40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:23:34.680 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:34.675+0000 7ff2caffd640 1 -- 172.21.15.143:0/2952752158 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff2cc00ac30 con 0x7ff2d415f4e0 2023-11-26T07:23:34.680 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:34.675+0000 7ff2caffd640 1 -- 172.21.15.143:0/2952752158 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff2cc00ad90 con 0x7ff2d415f4e0 2023-11-26T07:23:34.680 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:34.675+0000 7ff2caffd640 1 -- 172.21.15.143:0/2952752158 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff2cc012600 con 0x7ff2d415f4e0 2023-11-26T07:23:34.680 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:34.675+0000 7ff2dc679640 1 -- 172.21.15.143:0/2952752158 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff2d4143d30 con 0x7ff2d415f4e0 2023-11-26T07:23:34.680 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:34.675+0000 7ff2dc679640 1 -- 172.21.15.143:0/2952752158 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff2d41442f0 con 0x7ff2d415f4e0 2023-11-26T07:23:34.680 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:34.679+0000 7ff2dc679640 1 -- 172.21.15.143:0/2952752158 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff2d41444e0 con 0x7ff2d415f4e0 2023-11-26T07:23:34.683 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:34.683+0000 7ff2caffd640 1 -- 172.21.15.143:0/2952752158 <== mon.0 v2:172.21.15.143:3300/0 4 ==== mgrmap(e 16) v1 ==== 85954+0+0 (secure 0 0 0) 0x7ff2cc007070 con 0x7ff2d415f4e0 2023-11-26T07:23:34.684 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:34.683+0000 7ff2caffd640 1 --2- 172.21.15.143:0/2952752158 >> [v2:172.21.15.188:6836/3401128553,v1:172.21.15.188:6837/3401128553] conn(0x7ff2b8066590 0x7ff2b8068a50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:23:34.684 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:34.683+0000 7ff2caffd640 1 -- 172.21.15.143:0/2952752158 <== mon.0 v2:172.21.15.143:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 5709+0+0 (secure 0 0 0) 0x7ff2cc084110 con 0x7ff2d415f4e0 2023-11-26T07:23:34.684 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:34.683+0000 7ff2da415640 1 --2- 172.21.15.143:0/2952752158 >> [v2:172.21.15.188:6836/3401128553,v1:172.21.15.188:6837/3401128553] conn(0x7ff2b8066590 0x7ff2b8068a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:23:34.684 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:34.683+0000 7ff2da415640 1 --2- 172.21.15.143:0/2952752158 >> [v2:172.21.15.188:6836/3401128553,v1:172.21.15.188:6837/3401128553] conn(0x7ff2b8066590 0x7ff2b8068a50 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7ff2d00af000 tx=0x7ff2d00a7000 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:23:34.686 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:34.683+0000 7ff2caffd640 1 -- 172.21.15.143:0/2952752158 <== mon.0 v2:172.21.15.143:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172649 (secure 0 0 0) 0x7ff2cc0516a0 con 0x7ff2d415f4e0 2023-11-26T07:23:34.866 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:34.863+0000 7ff2dc679640 1 -- 172.21.15.143:0/2952752158 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7ff2d406df50 con 0x7ff2d415f4e0 2023-11-26T07:23:34.867 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:34.867+0000 7ff2caffd640 1 -- 172.21.15.143:0/2952752158 <== mon.0 v2:172.21.15.143:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 51 v51) v1 ==== 94+0+4924 (secure 0 0 0) 0x7ff2cc054cc0 con 0x7ff2d415f4e0 2023-11-26T07:23:34.867 INFO:teuthology.orchestra.run.smithi143.stdout: 2023-11-26T07:23:34.867 INFO:teuthology.orchestra.run.smithi143.stdout:{"epoch":51,"default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6305,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.188:6834/3670325785","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6832","nonce":3670325785},{"type":"v1","addr":"172.21.15.188:6834","nonce":3670325785}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":46},{"gid":7467,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.143:6834/4060405000","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6832","nonce":4060405000},{"type":"v1","addr":"172.21.15.143:6834","nonce":4060405000}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":46},{"gid":7470,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.143:6835/4031724325","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6833","nonce":4031724325},{"type":"v1","addr":"172.21.15.143:6835","nonce":4031724325}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":46}],"filesystems":[{"mdsmap":{"epoch":51,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-11-26T07:23:29.008560+0000","modified":"2023-11-26T07:23:32.033055+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":7476},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7476":{"gid":7476,"name":"b","rank":0,"incarnation":47,"state":"up:active","state_seq":3,"addr":"172.21.15.188:6835/278875865","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6833","nonce":278875865},{"type":"v1","addr":"172.21.15.188:6835","nonce":278875865}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[13],"metadata_pool":12,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":6}]} 2023-11-26T07:23:34.867 INFO:teuthology.orchestra.run.smithi143.stderr:dumped fsmap epoch 51 2023-11-26T07:23:34.869 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:34.867+0000 7ff2dc679640 1 -- 172.21.15.143:0/2952752158 >> [v2:172.21.15.188:6836/3401128553,v1:172.21.15.188:6837/3401128553] conn(0x7ff2b8066590 msgr2=0x7ff2b8068a50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:23:34.869 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:34.867+0000 7ff2dc679640 1 --2- 172.21.15.143:0/2952752158 >> [v2:172.21.15.188:6836/3401128553,v1:172.21.15.188:6837/3401128553] conn(0x7ff2b8066590 0x7ff2b8068a50 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7ff2d00af000 tx=0x7ff2d00a7000 comp rx=0 tx=0).stop 2023-11-26T07:23:34.869 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:34.867+0000 7ff2dc679640 1 -- 172.21.15.143:0/2952752158 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7ff2d415f4e0 msgr2=0x7ff2d415a150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:23:34.869 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:34.867+0000 7ff2dc679640 1 --2- 172.21.15.143:0/2952752158 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7ff2d415f4e0 0x7ff2d415a150 secure :-1 s=READY pgs=1164 cs=0 l=1 rev1=1 crypto rx=0x7ff2cc00ca80 tx=0x7ff2cc00cf40 comp rx=0 tx=0).stop 2023-11-26T07:23:34.869 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:34.867+0000 7ff2dc679640 1 -- 172.21.15.143:0/2952752158 shutdown_connections 2023-11-26T07:23:34.869 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:34.867+0000 7ff2dc679640 1 --2- 172.21.15.143:0/2952752158 >> [v2:172.21.15.188:6836/3401128553,v1:172.21.15.188:6837/3401128553] conn(0x7ff2b8066590 0x7ff2b8068a50 unknown :-1 s=CLOSED pgs=66 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:23:34.870 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:34.867+0000 7ff2dc679640 1 --2- 172.21.15.143:0/2952752158 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7ff2d415f4e0 0x7ff2d415a150 unknown :-1 s=CLOSED pgs=1164 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:23:34.870 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:34.867+0000 7ff2dc679640 1 --2- 172.21.15.143:0/2952752158 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7ff2d413e290 0x7ff2d4137890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:23:34.870 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:34.867+0000 7ff2dc679640 1 --2- 172.21.15.143:0/2952752158 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7ff2d40cf7c0 0x7ff2d41418f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:23:34.870 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:34.867+0000 7ff2dc679640 1 -- 172.21.15.143:0/2952752158 >> 172.21.15.143:0/2952752158 conn(0x7ff2d406a890 msgr2=0x7ff2d413b520 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:23:34.870 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:34.867+0000 7ff2dc679640 1 -- 172.21.15.143:0/2952752158 shutdown_connections 2023-11-26T07:23:34.870 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:34.867+0000 7ff2dc679640 1 -- 172.21.15.143:0/2952752158 wait complete. 2023-11-26T07:23:35.290 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 51, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2023-11-26T07:23:29.008560+0000', 'modified': '2023-11-26T07:23:32.033055+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 7476}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_7476': {'gid': 7476, 'name': 'b', 'rank': 0, 'incarnation': 47, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.188:6835/278875865', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.188:6833', 'nonce': 278875865}, {'type': 'v1', 'addr': '172.21.15.188:6835', 'nonce': 278875865}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138322906710015, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}}}, 'data_pools': [13], 'metadata_pool': 12, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0} 2023-11-26T07:23:35.290 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2023-11-26T07:23:35.291 DEBUG:teuthology.orchestra.run.smithi143:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.b status 2023-11-26T07:23:35.431 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:35.431+0000 7f49ec29c640 1 Processor -- start 2023-11-26T07:23:35.431 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:35.431+0000 7f49ec29c640 1 -- start start 2023-11-26T07:23:35.432 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:35.431+0000 7f49ec29c640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f49e41596d0 0x7f49e4071850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:23:35.432 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:35.431+0000 7f49ec29c640 1 -- --> v1:172.21.15.143:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f49e406c750 con 0x7f49e40ce680 2023-11-26T07:23:35.432 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:35.431+0000 7f49ec29c640 1 -- --> v1:172.21.15.188:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f49e40b3550 con 0x7f49e40b2f80 2023-11-26T07:23:35.433 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:35.431+0000 7f49ec29c640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f49e40cbe00 con 0x7f49e41596d0 2023-11-26T07:23:35.433 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:35.431+0000 7f49e9837640 1 --1- >> v1:172.21.15.143:6789/0 conn(0x7f49e40ce680 0x7f49e40b2870 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.143:6789/0 says I am v1:172.21.15.143:47986/0 (socket says 172.21.15.143:47986) 2023-11-26T07:23:35.433 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:35.431+0000 7f49e9837640 1 -- 172.21.15.143:0/896432529 learned_addr learned my addr 172.21.15.143:0/896432529 (peer_addr_for_me v1:172.21.15.143:0/0) 2023-11-26T07:23:35.433 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:35.431+0000 7f49ea038640 1 --2- 172.21.15.143:0/896432529 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f49e41596d0 0x7f49e4071850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:23:35.433 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:35.431+0000 7f49e9036640 1 -- 172.21.15.143:0/896432529 <== mon.0 v1:172.21.15.143:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 318484120 0 0) 0x7f49e406c750 con 0x7f49e40ce680 2023-11-26T07:23:35.434 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:35.431+0000 7f49e9036640 1 -- 172.21.15.143:0/896432529 --> v1:172.21.15.143:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f49d0003530 con 0x7f49e40ce680 2023-11-26T07:23:35.434 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:35.431+0000 7f49ea038640 1 -- 172.21.15.143:0/896432529 >> v1:172.21.15.188:6789/0 conn(0x7f49e40b2f80 legacy=0x7f49e406cbe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:23:35.434 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:35.431+0000 7f49e9036640 1 -- 172.21.15.143:0/896432529 <== mon.1 v1:172.21.15.188:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2680900728 0 0) 0x7f49e40b3550 con 0x7f49e40b2f80 2023-11-26T07:23:35.434 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:35.431+0000 7f49ea038640 1 -- 172.21.15.143:0/896432529 >> v1:172.21.15.143:6789/0 conn(0x7f49e40ce680 legacy=0x7f49e40b2870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:23:35.434 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:35.431+0000 7f49ea038640 1 -- 172.21.15.143:0/896432529 --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f49e4071e80 con 0x7f49e41596d0 2023-11-26T07:23:35.436 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:35.435+0000 7f49ea038640 1 --2- 172.21.15.143:0/896432529 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f49e41596d0 0x7f49e4071850 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7f49e00097f0 tx=0x7f49e001c4c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=dc2392ff5684aea7 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:23:35.436 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:35.435+0000 7f49e9036640 1 -- 172.21.15.143:0/896432529 <== mon.2 v2:172.21.15.188:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f49e00ab600 con 0x7f49e41596d0 2023-11-26T07:23:35.437 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:35.435+0000 7f49e9036640 1 -- 172.21.15.143:0/896432529 <== mon.2 v2:172.21.15.188:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f49e00a7050 con 0x7f49e41596d0 2023-11-26T07:23:35.437 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:35.435+0000 7f49e9036640 1 -- 172.21.15.143:0/896432529 <== mon.2 v2:172.21.15.188:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f49e00ab960 con 0x7f49e41596d0 2023-11-26T07:23:35.437 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:35.435+0000 7f49ec29c640 1 -- 172.21.15.143:0/896432529 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f49e41596d0 msgr2=0x7f49e4071850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:23:35.437 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:35.435+0000 7f49ec29c640 1 --2- 172.21.15.143:0/896432529 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f49e41596d0 0x7f49e4071850 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7f49e00097f0 tx=0x7f49e001c4c0 comp rx=0 tx=0).stop 2023-11-26T07:23:35.438 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:35.435+0000 7f49ec29c640 1 -- 172.21.15.143:0/896432529 shutdown_connections 2023-11-26T07:23:35.438 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:35.435+0000 7f49ec29c640 1 --2- 172.21.15.143:0/896432529 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f49e41596d0 0x7f49e4071850 unknown :-1 s=CLOSED pgs=144 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:23:35.438 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:35.435+0000 7f49ec29c640 1 -- 172.21.15.143:0/896432529 >> 172.21.15.143:0/896432529 conn(0x7f49e406a890 msgr2=0x7f49e406aca0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:23:35.438 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:35.435+0000 7f49ec29c640 1 -- 172.21.15.143:0/896432529 shutdown_connections 2023-11-26T07:23:35.438 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:35.435+0000 7f49ec29c640 1 -- 172.21.15.143:0/896432529 wait complete. 2023-11-26T07:23:35.439 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:35.439+0000 7f49ec29c640 1 Processor -- start 2023-11-26T07:23:35.439 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:35.439+0000 7f49ec29c640 1 -- start start 2023-11-26T07:23:35.441 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:35.439+0000 7f49ec29c640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f49e40b2f80 0x7f49e415ab60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:23:35.441 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:35.439+0000 7f49ec29c640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f49e40ce680 0x7f49e414f7e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:23:35.441 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:35.439+0000 7f49ec29c640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f49e41596d0 0x7f49e415c090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:23:35.441 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:35.439+0000 7f49ec29c640 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f49e415b0a0 con 0x7f49e40ce680 2023-11-26T07:23:35.442 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:35.439+0000 7f49ec29c640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f49e4152a40 con 0x7f49e40b2f80 2023-11-26T07:23:35.442 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:35.439+0000 7f49ec29c640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f49e415bac0 con 0x7f49e41596d0 2023-11-26T07:23:35.442 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:35.439+0000 7f49e9837640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f49e40ce680 0x7f49e414f7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:23:35.442 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:35.439+0000 7f49e9837640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f49e40ce680 0x7f49e414f7e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:55670/0 (socket says 172.21.15.143:55670) 2023-11-26T07:23:35.442 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:35.439+0000 7f49e9837640 1 -- 172.21.15.143:0/1858828909 learned_addr learned my addr 172.21.15.143:0/1858828909 (peer_addr_for_me v2:172.21.15.143:0/0) 2023-11-26T07:23:35.442 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:35.439+0000 7f49ea038640 1 --2- 172.21.15.143:0/1858828909 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f49e40b2f80 0x7f49e415ab60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:23:35.442 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:35.439+0000 7f49ea839640 1 --2- 172.21.15.143:0/1858828909 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f49e41596d0 0x7f49e415c090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:23:35.442 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:35.439+0000 7f49e9837640 1 -- 172.21.15.143:0/1858828909 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f49e41596d0 msgr2=0x7f49e415c090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:23:35.442 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:35.439+0000 7f49e9837640 1 --2- 172.21.15.143:0/1858828909 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f49e41596d0 0x7f49e415c090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:23:35.442 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:35.439+0000 7f49e9837640 1 -- 172.21.15.143:0/1858828909 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f49e40b2f80 msgr2=0x7f49e415ab60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:23:35.442 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:35.439+0000 7f49e9837640 1 --2- 172.21.15.143:0/1858828909 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f49e40b2f80 0x7f49e415ab60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:23:35.443 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:35.439+0000 7f49e9837640 1 -- 172.21.15.143:0/1858828909 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f49e0009420 con 0x7f49e40ce680 2023-11-26T07:23:35.443 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:35.439+0000 7f49e9837640 1 --2- 172.21.15.143:0/1858828909 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f49e40ce680 0x7f49e414f7e0 secure :-1 s=READY pgs=1166 cs=0 l=1 rev1=1 crypto rx=0x7f49d400c450 tx=0x7f49d400c910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:23:35.443 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:35.439+0000 7f49daffd640 1 -- 172.21.15.143:0/1858828909 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f49d4005cf0 con 0x7f49e40ce680 2023-11-26T07:23:35.443 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:35.439+0000 7f49daffd640 1 -- 172.21.15.143:0/1858828909 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f49d4005e50 con 0x7f49e40ce680 2023-11-26T07:23:35.443 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:35.439+0000 7f49ec29c640 1 -- 172.21.15.143:0/1858828909 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f49e415bda0 con 0x7f49e40ce680 2023-11-26T07:23:35.443 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:35.439+0000 7f49daffd640 1 -- 172.21.15.143:0/1858828909 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f49d4007290 con 0x7f49e40ce680 2023-11-26T07:23:35.443 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:35.439+0000 7f49ec29c640 1 -- 172.21.15.143:0/1858828909 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f49e41526d0 con 0x7f49e40ce680 2023-11-26T07:23:35.443 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:35.443+0000 7f49daffd640 1 -- 172.21.15.143:0/1858828909 <== mon.0 v2:172.21.15.143:3300/0 4 ==== mgrmap(e 16) v1 ==== 85954+0+0 (secure 0 0 0) 0x7f49d40073f0 con 0x7f49e40ce680 2023-11-26T07:23:35.445 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:35.443+0000 7f49daffd640 1 --2- 172.21.15.143:0/1858828909 >> [v2:172.21.15.188:6836/3401128553,v1:172.21.15.188:6837/3401128553] conn(0x7f49c8066610 0x7f49c8068ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:23:35.445 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:35.443+0000 7f49daffd640 1 -- 172.21.15.143:0/1858828909 <== mon.0 v2:172.21.15.143:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f49d4083530 con 0x7f49e40ce680 2023-11-26T07:23:35.445 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:35.443+0000 7f49ea038640 1 --2- 172.21.15.143:0/1858828909 >> [v2:172.21.15.188:6836/3401128553,v1:172.21.15.188:6837/3401128553] conn(0x7f49c8066610 0x7f49c8068ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:23:35.446 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:35.443+0000 7f49ef615000 1 Processor -- start 2023-11-26T07:23:35.446 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:35.443+0000 7f49ef615000 1 -- start start 2023-11-26T07:23:35.447 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:35.443+0000 7f49ea038640 1 --2- 172.21.15.143:0/1858828909 >> [v2:172.21.15.188:6836/3401128553,v1:172.21.15.188:6837/3401128553] conn(0x7f49c8066610 0x7f49c8068ad0 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7f49e00ad000 tx=0x7f49e00a6000 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:23:35.447 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:35.447+0000 7f49ef615000 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x562ae0c94c90 0x562ae0c95070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:23:35.447 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:35.447+0000 7f49ef615000 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x562ae0c955b0 0x562ae0c4ade0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:23:35.448 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:35.447+0000 7f49ef615000 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x562ae0c4b320 0x562ae0c4b700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:23:35.448 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:35.447+0000 7f49ef615000 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x562ae0ab5130 con 0x562ae0c955b0 2023-11-26T07:23:35.448 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:35.447+0000 7f49ef615000 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x562ae0aa29a0 con 0x562ae0c94c90 2023-11-26T07:23:35.448 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:35.447+0000 7f49ef615000 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x562ae0a9fed0 con 0x562ae0c4b320 2023-11-26T07:23:35.448 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:35.447+0000 7f49e9837640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x562ae0c955b0 0x562ae0c4ade0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:23:35.448 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:35.447+0000 7f49e9837640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x562ae0c955b0 0x562ae0c4ade0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:55674/0 (socket says 172.21.15.143:55674) 2023-11-26T07:23:35.448 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:35.447+0000 7f49e9837640 1 -- 172.21.15.143:0/2682330280 learned_addr learned my addr 172.21.15.143:0/2682330280 (peer_addr_for_me v2:172.21.15.143:0/0) 2023-11-26T07:23:35.448 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:35.447+0000 7f49ea839640 1 --2- 172.21.15.143:0/2682330280 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x562ae0c4b320 0x562ae0c4b700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:23:35.448 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:35.447+0000 7f49ea038640 1 --2- 172.21.15.143:0/2682330280 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x562ae0c94c90 0x562ae0c95070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:23:35.448 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:35.447+0000 7f49e9837640 1 -- 172.21.15.143:0/2682330280 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x562ae0c4b320 msgr2=0x562ae0c4b700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-26T07:23:35.448 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:35.447+0000 7f49e9837640 1 --2- 172.21.15.143:0/2682330280 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x562ae0c4b320 0x562ae0c4b700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:23:35.449 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:35.447+0000 7f49e9837640 1 -- 172.21.15.143:0/2682330280 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x562ae0c94c90 msgr2=0x562ae0c95070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-26T07:23:35.449 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:35.447+0000 7f49e9837640 1 --2- 172.21.15.143:0/2682330280 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x562ae0c94c90 0x562ae0c95070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:23:35.449 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:35.447+0000 7f49e9837640 1 -- 172.21.15.143:0/2682330280 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x562ae0793890 con 0x562ae0c955b0 2023-11-26T07:23:35.449 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:35.447+0000 7f49e9837640 1 --2- 172.21.15.143:0/2682330280 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x562ae0c955b0 0x562ae0c4ade0 secure :-1 s=READY pgs=1167 cs=0 l=1 rev1=1 crypto rx=0x7f49d4050af0 tx=0x7f49d4058690 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2820de1aeb28b144 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:23:35.449 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:35.447+0000 7f49c7fff640 1 -- 172.21.15.143:0/2682330280 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f49d40527f0 con 0x562ae0c955b0 2023-11-26T07:23:35.449 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:35.447+0000 7f49c7fff640 1 -- 172.21.15.143:0/2682330280 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f49d4081a30 con 0x562ae0c955b0 2023-11-26T07:23:35.449 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:35.447+0000 7f49c7fff640 1 -- 172.21.15.143:0/2682330280 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f49d4081ba0 con 0x562ae0c955b0 2023-11-26T07:23:35.450 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:35.447+0000 7f49ef615000 1 -- 172.21.15.143:0/2682330280 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x562ae0c955b0 msgr2=0x562ae0c4ade0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:23:35.450 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:35.447+0000 7f49ef615000 1 --2- 172.21.15.143:0/2682330280 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x562ae0c955b0 0x562ae0c4ade0 secure :-1 s=READY pgs=1167 cs=0 l=1 rev1=1 crypto rx=0x7f49d4050af0 tx=0x7f49d4058690 comp rx=0 tx=0).stop 2023-11-26T07:23:35.450 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:35.447+0000 7f49ef615000 1 -- 172.21.15.143:0/2682330280 shutdown_connections 2023-11-26T07:23:35.450 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:35.447+0000 7f49ef615000 1 --2- 172.21.15.143:0/2682330280 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x562ae0c4b320 0x562ae0c4b700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:23:35.450 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:35.447+0000 7f49ef615000 1 --2- 172.21.15.143:0/2682330280 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x562ae0c955b0 0x562ae0c4ade0 unknown :-1 s=CLOSED pgs=1167 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:23:35.451 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:35.447+0000 7f49ef615000 1 --2- 172.21.15.143:0/2682330280 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x562ae0c94c90 0x562ae0c95070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:23:35.451 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:35.447+0000 7f49ef615000 1 -- 172.21.15.143:0/2682330280 >> 172.21.15.143:0/2682330280 conn(0x562ae0c472d0 msgr2=0x562ae0c496c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:23:35.451 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:35.447+0000 7f49ef615000 1 -- 172.21.15.143:0/2682330280 shutdown_connections 2023-11-26T07:23:35.451 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:35.447+0000 7f49ef615000 1 -- 172.21.15.143:0/2682330280 wait complete. 2023-11-26T07:23:35.451 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:35.447+0000 7f49ef615000 1 -- start start 2023-11-26T07:23:35.451 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:35.447+0000 7f49ef615000 1 Processor -- start 2023-11-26T07:23:35.453 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:35.451+0000 7f49ef615000 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x562ae0c4b320 0x562ae0c4fd40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:23:35.454 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:35.451+0000 7f49ef615000 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x562ae0c94c90 0x562ae0c50280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:23:35.454 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:35.451+0000 7f49ef615000 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x562ae0c955b0 0x562ae0c63910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:23:35.454 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:35.451+0000 7f49ef615000 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x562ae0a9b7c0 con 0x562ae0c955b0 2023-11-26T07:23:35.454 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:35.451+0000 7f49ef615000 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x562ae0a53bf0 con 0x562ae0c4b320 2023-11-26T07:23:35.454 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:35.451+0000 7f49ef615000 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x562ae0a59a90 con 0x562ae0c94c90 2023-11-26T07:23:35.454 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:35.451+0000 7f49ef615000 10 client.0 _ll_register_callbacks cb 0x562ae0c363c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-11-26T07:23:35.454 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:35.451+0000 7f49ea839640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x562ae0c955b0 0x562ae0c63910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:23:35.454 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:35.451+0000 7f49ea839640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x562ae0c955b0 0x562ae0c63910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:55686/0 (socket says 172.21.15.143:55686) 2023-11-26T07:23:35.454 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:35.451+0000 7f49ea839640 1 -- 172.21.15.143:0/303300181 learned_addr learned my addr 172.21.15.143:0/303300181 (peer_addr_for_me v2:172.21.15.143:0/0) 2023-11-26T07:23:35.454 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:35.451+0000 7f49e9837640 1 --2- 172.21.15.143:0/303300181 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x562ae0c94c90 0x562ae0c50280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:23:35.455 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:35.451+0000 7f49ea038640 1 --2- 172.21.15.143:0/303300181 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x562ae0c4b320 0x562ae0c4fd40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:23:35.455 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:35.451+0000 7f49ea839640 1 -- 172.21.15.143:0/303300181 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x562ae0c94c90 msgr2=0x562ae0c50280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-26T07:23:35.455 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:35.451+0000 7f49ea839640 1 --2- 172.21.15.143:0/303300181 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x562ae0c94c90 0x562ae0c50280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:23:35.455 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:35.451+0000 7f49ea839640 1 -- 172.21.15.143:0/303300181 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x562ae0c4b320 msgr2=0x562ae0c4fd40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-26T07:23:35.455 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:35.451+0000 7f49ea839640 1 --2- 172.21.15.143:0/303300181 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x562ae0c4b320 0x562ae0c4fd40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:23:35.455 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:35.451+0000 7f49ea839640 1 -- 172.21.15.143:0/303300181 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f49e41526d0 con 0x562ae0c955b0 2023-11-26T07:23:35.455 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:35.451+0000 7f49ea839640 1 --2- 172.21.15.143:0/303300181 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x562ae0c955b0 0x562ae0c63910 secure :-1 s=READY pgs=1168 cs=0 l=1 rev1=1 crypto rx=0x7f49dc0056b0 tx=0x7f49dc001cd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=265a4e8a4c4bfc87 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:23:35.455 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:35.451+0000 7f49c57fa640 10 client.0 ms_handle_connect on v2:172.21.15.143:3300/0 2023-11-26T07:23:35.455 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:35.451+0000 7f49c57fa640 1 -- 172.21.15.143:0/303300181 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f49dc006d50 con 0x562ae0c955b0 2023-11-26T07:23:35.455 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:35.451+0000 7f49c57fa640 1 -- 172.21.15.143:0/303300181 <== mon.0 v2:172.21.15.143:3300/0 2 ==== osd_map(84..84 src has 1..84) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f49dc016ae0 con 0x562ae0c955b0 2023-11-26T07:23:35.455 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:35.451+0000 7f49ef615000 1 -- 172.21.15.143:0/303300181 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x562ae0a5d470 con 0x562ae0c955b0 2023-11-26T07:23:35.455 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:35.451+0000 7f49c57fa640 1 -- 172.21.15.143:0/303300181 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_get_version_reply(handle=1 version=51) v2 ==== 24+0+0 (secure 0 0 0) 0x7f49dc016ca0 con 0x562ae0c955b0 2023-11-26T07:23:35.455 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:35.451+0000 7f49ef615000 10 client.7617 fetch_fsmap learned FSMap version 51 2023-11-26T07:23:35.456 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:35.451+0000 7f49ef615000 1 -- 172.21.15.143:0/303300181 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,fsmap=51,monmap=2+}) v3 -- 0x562ae0a5c190 con 0x562ae0c955b0 2023-11-26T07:23:35.456 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:35.451+0000 7f49c57fa640 1 -- 172.21.15.143:0/303300181 <== mon.0 v2:172.21.15.143:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x562ae0a5d470 con 0x562ae0c955b0 2023-11-26T07:23:35.456 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:35.451+0000 7f49c57fa640 1 -- 172.21.15.143:0/303300181 <== mon.0 v2:172.21.15.143:3300/0 5 ==== fsmap(e 51) v1 ==== 3294+0+0 (secure 0 0 0) 0x7f49dc007030 con 0x562ae0c955b0 2023-11-26T07:23:35.456 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:35.451+0000 7f49ef615000 10 client.7617 fetch_fsmap finished waiting for FSMap version 51 2023-11-26T07:23:35.456 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:35.451+0000 7f49ef615000 10 client.7617 resolve_mds: resolved name 'b' to daemon mds.b 2023-11-26T07:23:35.456 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:35.451+0000 7f49ef615000 20 client.7617 populate_metadata read hostname 'smithi143' 2023-11-26T07:23:35.456 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:35.451+0000 7f49ef615000 1 --2- 172.21.15.143:0/303300181 >> [v2:172.21.15.188:6833/278875865,v1:172.21.15.188:6835/278875865] conn(0x562ae0c64320 0x562ae0ca6f50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:23:35.456 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:35.451+0000 7f49ef615000 4 client.7617 mds_command: new command op to 7476 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2023-11-26T07:23:35.456 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:35.451+0000 7f49ef615000 1 -- 172.21.15.143:0/303300181 --> [v2:172.21.15.188:6833/278875865,v1:172.21.15.188:6835/278875865] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x562ae0a60b50 con 0x562ae0c64320 2023-11-26T07:23:35.456 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:35.451+0000 7f49ea038640 1 --2- 172.21.15.143:0/303300181 >> [v2:172.21.15.188:6833/278875865,v1:172.21.15.188:6835/278875865] conn(0x562ae0c64320 0x562ae0ca6f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:23:35.456 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:35.451+0000 7f49ea038640 1 --2- 172.21.15.143:0/303300181 >> [v2:172.21.15.188:6833/278875865,v1:172.21.15.188:6835/278875865] conn(0x562ae0c64320 0x562ae0ca6f50 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b1cf14685d51eee server_cookie=52addf3c5f7fd865 in_seq=0 out_seq=0 2023-11-26T07:23:35.456 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:35.451+0000 7f49c57fa640 10 client.7617 ms_handle_connect on v2:172.21.15.188:6833/278875865 2023-11-26T07:23:35.457 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:35.455+0000 7f49c57fa640 1 -- 172.21.15.143:0/303300181 <== mds.0 v2:172.21.15.188:6833/278875865 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x562ae0a60b50 con 0x562ae0c64320 2023-11-26T07:23:35.458 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:35.455+0000 7f49c57fa640 10 client.7617 handle_command_reply: tid=0 2023-11-26T07:23:35.458 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:35.455+0000 7f49ef615000 1 client.7617 shutdown 2023-11-26T07:23:35.458 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:35.455+0000 7f49ef615000 1 -- 172.21.15.143:0/303300181 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x562ae0c955b0 msgr2=0x562ae0c63910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:23:35.458 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:35.455+0000 7f49ef615000 1 --2- 172.21.15.143:0/303300181 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x562ae0c955b0 0x562ae0c63910 secure :-1 s=READY pgs=1168 cs=0 l=1 rev1=1 crypto rx=0x7f49dc0056b0 tx=0x7f49dc001cd0 comp rx=0 tx=0).stop 2023-11-26T07:23:35.458 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:35.455+0000 7f49ef615000 1 -- 172.21.15.143:0/303300181 shutdown_connections 2023-11-26T07:23:35.458 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:35.455+0000 7f49ef615000 1 --2- 172.21.15.143:0/303300181 >> [v2:172.21.15.188:6833/278875865,v1:172.21.15.188:6835/278875865] conn(0x562ae0c64320 0x562ae0ca6f50 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:23:35.458 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:35.455+0000 7f49ef615000 1 --2- 172.21.15.143:0/303300181 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x562ae0c955b0 0x562ae0c63910 unknown :-1 s=CLOSED pgs=1168 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:23:35.458 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:35.455+0000 7f49ef615000 1 --2- 172.21.15.143:0/303300181 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x562ae0c94c90 0x562ae0c50280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:23:35.458 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:35.455+0000 7f49c57fa640 0 client.7617 ms_handle_reset on v2:172.21.15.188:6833/278875865 2023-11-26T07:23:35.458 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:35.455+0000 7f49ef615000 1 --2- 172.21.15.143:0/303300181 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x562ae0c4b320 0x562ae0c4fd40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:23:35.459 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:35.455+0000 7f49ef615000 1 -- 172.21.15.143:0/303300181 >> 172.21.15.143:0/303300181 conn(0x562ae0c472d0 msgr2=0x562ae0c496c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:23:35.459 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:35.455+0000 7f49ef615000 1 -- 172.21.15.143:0/303300181 shutdown_connections 2023-11-26T07:23:35.459 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:35.455+0000 7f49ef615000 1 -- 172.21.15.143:0/303300181 wait complete. 2023-11-26T07:23:35.459 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:35.455+0000 7f49ef615000 20 client.7617 trim_cache size 0 max 16384 2023-11-26T07:23:35.470 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:35.467+0000 7f49ef615000 1 Processor -- start 2023-11-26T07:23:35.470 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:35.467+0000 7f49ef615000 1 -- start start 2023-11-26T07:23:35.472 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:35.471+0000 7f49ef615000 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x562ae0c4b320 0x562ae0c92b30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:23:35.472 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:35.471+0000 7f49ef615000 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x562ae0c94c90 0x562ae0c93070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:23:35.472 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:35.471+0000 7f49ef615000 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x562ae0c955b0 0x562ae0c4b700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:23:35.472 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:35.471+0000 7f49ef615000 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x562ae0754940 con 0x562ae0c955b0 2023-11-26T07:23:35.472 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:35.471+0000 7f49ef615000 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x562ae0c50820 con 0x562ae0c4b320 2023-11-26T07:23:35.472 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:35.471+0000 7f49ef615000 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x562ae0c50fb0 con 0x562ae0c94c90 2023-11-26T07:23:35.472 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:35.471+0000 7f49ea839640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x562ae0c955b0 0x562ae0c4b700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:23:35.472 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:35.471+0000 7f49ea839640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x562ae0c955b0 0x562ae0c4b700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:55694/0 (socket says 172.21.15.143:55694) 2023-11-26T07:23:35.473 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:35.471+0000 7f49ea839640 1 -- 172.21.15.143:0/351001284 learned_addr learned my addr 172.21.15.143:0/351001284 (peer_addr_for_me v2:172.21.15.143:0/0) 2023-11-26T07:23:35.473 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:35.471+0000 7f49ea038640 1 --2- 172.21.15.143:0/351001284 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x562ae0c4b320 0x562ae0c92b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:23:35.473 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:35.471+0000 7f49e9837640 1 --2- 172.21.15.143:0/351001284 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x562ae0c94c90 0x562ae0c93070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:23:35.473 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:35.471+0000 7f49ea839640 1 -- 172.21.15.143:0/351001284 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x562ae0c94c90 msgr2=0x562ae0c93070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-26T07:23:35.473 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:35.471+0000 7f49ea839640 1 --2- 172.21.15.143:0/351001284 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x562ae0c94c90 0x562ae0c93070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:23:35.473 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:35.471+0000 7f49ea839640 1 -- 172.21.15.143:0/351001284 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x562ae0c4b320 msgr2=0x562ae0c92b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-26T07:23:35.473 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:35.471+0000 7f49ea839640 1 --2- 172.21.15.143:0/351001284 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x562ae0c4b320 0x562ae0c92b30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:23:35.473 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:35.471+0000 7f49ea839640 1 -- 172.21.15.143:0/351001284 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x562ae0a42490 con 0x562ae0c955b0 2023-11-26T07:23:35.473 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:35.471+0000 7f49ea038640 1 --2- 172.21.15.143:0/351001284 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x562ae0c4b320 0x562ae0c92b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-26T07:23:35.473 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:35.471+0000 7f49ea839640 1 --2- 172.21.15.143:0/351001284 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x562ae0c955b0 0x562ae0c4b700 secure :-1 s=READY pgs=1169 cs=0 l=1 rev1=1 crypto rx=0x7f49dc001d10 tx=0x7f49dc002200 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7117898c3c32645d server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:23:35.473 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:35.471+0000 7f49d8ff9640 1 -- 172.21.15.143:0/351001284 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f49dc0012d0 con 0x562ae0c955b0 2023-11-26T07:23:35.473 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:35.471+0000 7f49d8ff9640 1 -- 172.21.15.143:0/351001284 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f49dc0078c0 con 0x562ae0c955b0 2023-11-26T07:23:35.474 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:35.471+0000 7f49d8ff9640 1 -- 172.21.15.143:0/351001284 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f49dc007a30 con 0x562ae0c955b0 2023-11-26T07:23:35.476 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:35.471+0000 7f49ef615000 1 -- 172.21.15.143:0/351001284 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x562ae0c955b0 msgr2=0x562ae0c4b700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:23:35.477 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:35.471+0000 7f49ef615000 1 --2- 172.21.15.143:0/351001284 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x562ae0c955b0 0x562ae0c4b700 secure :-1 s=READY pgs=1169 cs=0 l=1 rev1=1 crypto rx=0x7f49dc001d10 tx=0x7f49dc002200 comp rx=0 tx=0).stop 2023-11-26T07:23:35.477 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:35.471+0000 7f49ef615000 1 -- 172.21.15.143:0/351001284 shutdown_connections 2023-11-26T07:23:35.477 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:35.471+0000 7f49ef615000 1 --2- 172.21.15.143:0/351001284 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x562ae0c955b0 0x562ae0c4b700 unknown :-1 s=CLOSED pgs=1169 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:23:35.477 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:35.471+0000 7f49ef615000 1 --2- 172.21.15.143:0/351001284 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x562ae0c94c90 0x562ae0c93070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:23:35.477 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:35.471+0000 7f49ef615000 1 --2- 172.21.15.143:0/351001284 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x562ae0c4b320 0x562ae0c92b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:23:35.477 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:35.471+0000 7f49ef615000 1 -- 172.21.15.143:0/351001284 >> 172.21.15.143:0/351001284 conn(0x562ae0c472d0 msgr2=0x562ae0c496c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:23:35.477 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:35.471+0000 7f49ef615000 1 -- 172.21.15.143:0/351001284 shutdown_connections 2023-11-26T07:23:35.477 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:35.471+0000 7f49ef615000 1 -- 172.21.15.143:0/351001284 wait complete. 2023-11-26T07:23:35.477 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:35.471+0000 7f49ef615000 1 -- start start 2023-11-26T07:23:35.477 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:35.471+0000 7f49ef615000 1 Processor -- start 2023-11-26T07:23:35.478 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:35.475+0000 7f49ef615000 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x562ae0c4b320 0x562ae0ca7fa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:23:35.478 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:35.475+0000 7f49ef615000 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x562ae0c94c90 0x562ae0ca84e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:23:35.478 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:35.475+0000 7f49ef615000 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x562ae0c955b0 0x562ae0cbb1f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:23:35.478 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:35.475+0000 7f49ef615000 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x562ae0a98bc0 con 0x562ae0c94c90 2023-11-26T07:23:35.478 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:35.475+0000 7f49ef615000 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x562ae0c46c70 con 0x562ae0c4b320 2023-11-26T07:23:35.478 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:35.475+0000 7f49ef615000 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x562ae0a5d470 con 0x562ae0c955b0 2023-11-26T07:23:35.478 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:35.475+0000 7f49ef615000 10 client.0 _ll_register_callbacks cb 0x562ae0ae9c00 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-11-26T07:23:35.478 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:35.475+0000 7f49e9837640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x562ae0c94c90 0x562ae0ca84e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:23:35.478 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:35.475+0000 7f49e9837640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x562ae0c94c90 0x562ae0ca84e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:55706/0 (socket says 172.21.15.143:55706) 2023-11-26T07:23:35.478 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:35.475+0000 7f49e9837640 1 -- 172.21.15.143:0/4291645368 learned_addr learned my addr 172.21.15.143:0/4291645368 (peer_addr_for_me v2:172.21.15.143:0/0) 2023-11-26T07:23:35.478 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:35.475+0000 7f49ea839640 1 --2- 172.21.15.143:0/4291645368 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x562ae0c955b0 0x562ae0cbb1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:23:35.479 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:35.475+0000 7f49ea038640 1 --2- 172.21.15.143:0/4291645368 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x562ae0c4b320 0x562ae0ca7fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:23:35.479 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:35.475+0000 7f49e9837640 1 -- 172.21.15.143:0/4291645368 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x562ae0c955b0 msgr2=0x562ae0cbb1f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-26T07:23:35.479 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:35.475+0000 7f49e9837640 1 --2- 172.21.15.143:0/4291645368 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x562ae0c955b0 0x562ae0cbb1f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:23:35.479 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:35.475+0000 7f49e9837640 1 -- 172.21.15.143:0/4291645368 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x562ae0c4b320 msgr2=0x562ae0ca7fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-26T07:23:35.479 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:35.475+0000 7f49e9837640 1 --2- 172.21.15.143:0/4291645368 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x562ae0c4b320 0x562ae0ca7fa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:23:35.479 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:35.475+0000 7f49e9837640 1 -- 172.21.15.143:0/4291645368 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x562ae0a5c190 con 0x562ae0c94c90 2023-11-26T07:23:35.479 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:35.475+0000 7f49e9837640 1 --2- 172.21.15.143:0/4291645368 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x562ae0c94c90 0x562ae0ca84e0 secure :-1 s=READY pgs=1170 cs=0 l=1 rev1=1 crypto rx=0x7f49d400ce10 tx=0x7f49d4000ee0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e13797be8cbc11b8 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:23:35.479 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:35.475+0000 7f49c57fa640 10 client.0 ms_handle_connect on v2:172.21.15.143:3300/0 2023-11-26T07:23:35.479 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:35.475+0000 7f49c57fa640 1 -- 172.21.15.143:0/4291645368 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f49d4081ba0 con 0x562ae0c94c90 2023-11-26T07:23:35.479 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:35.475+0000 7f49c57fa640 1 -- 172.21.15.143:0/4291645368 <== mon.0 v2:172.21.15.143:3300/0 2 ==== osd_map(84..84 src has 1..84) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f49d4053ae0 con 0x562ae0c94c90 2023-11-26T07:23:35.479 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:35.475+0000 7f49ef615000 1 -- 172.21.15.143:0/4291645368 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x562ae0a9a110 con 0x562ae0c94c90 2023-11-26T07:23:35.479 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:35.475+0000 7f49c57fa640 1 -- 172.21.15.143:0/4291645368 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_get_version_reply(handle=1 version=51) v2 ==== 24+0+0 (secure 0 0 0) 0x7f49d4053ca0 con 0x562ae0c94c90 2023-11-26T07:23:35.480 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:35.475+0000 7f49ef615000 10 client.7623 fetch_fsmap learned FSMap version 51 2023-11-26T07:23:35.480 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:35.475+0000 7f49ef615000 1 -- 172.21.15.143:0/4291645368 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,fsmap=51,monmap=2+}) v3 -- 0x562ae0793cc0 con 0x562ae0c94c90 2023-11-26T07:23:35.480 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:35.475+0000 7f49c57fa640 1 -- 172.21.15.143:0/4291645368 <== mon.0 v2:172.21.15.143:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f49d4076070 con 0x562ae0c94c90 2023-11-26T07:23:35.480 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:35.475+0000 7f49c57fa640 1 -- 172.21.15.143:0/4291645368 <== mon.0 v2:172.21.15.143:3300/0 5 ==== fsmap(e 51) v1 ==== 3294+0+0 (secure 0 0 0) 0x7f49d4072260 con 0x562ae0c94c90 2023-11-26T07:23:35.480 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:35.475+0000 7f49ef615000 10 client.7623 fetch_fsmap finished waiting for FSMap version 51 2023-11-26T07:23:35.480 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:35.475+0000 7f49ef615000 10 client.7623 resolve_mds: resolved name 'b' to daemon mds.b 2023-11-26T07:23:35.480 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:35.475+0000 7f49ef615000 20 client.7623 populate_metadata read hostname 'smithi143' 2023-11-26T07:23:35.480 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:35.475+0000 7f49ef615000 1 --2- 172.21.15.143:0/4291645368 >> [v2:172.21.15.188:6833/278875865,v1:172.21.15.188:6835/278875865] conn(0x562ae0cbbb80 0x562ae0d06660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:23:35.480 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:35.475+0000 7f49ef615000 4 client.7623 mds_command: new command op to 7476 tid=0 multi_id=0 [{"prefix": "status"}] 2023-11-26T07:23:35.480 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:35.475+0000 7f49ef615000 1 -- 172.21.15.143:0/4291645368 --> [v2:172.21.15.188:6833/278875865,v1:172.21.15.188:6835/278875865] -- command(tid 0: {"prefix": "status"}) v1 -- 0x562ae0ac99a0 con 0x562ae0cbbb80 2023-11-26T07:23:35.480 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:35.475+0000 7f49ea038640 1 --2- 172.21.15.143:0/4291645368 >> [v2:172.21.15.188:6833/278875865,v1:172.21.15.188:6835/278875865] conn(0x562ae0cbbb80 0x562ae0d06660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:23:35.481 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:35.475+0000 7f49ea038640 1 --2- 172.21.15.143:0/4291645368 >> [v2:172.21.15.188:6833/278875865,v1:172.21.15.188:6835/278875865] conn(0x562ae0cbbb80 0x562ae0d06660 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=822be29515a40f23 server_cookie=2b86c5a38c187bb5 in_seq=0 out_seq=0 2023-11-26T07:23:35.482 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:35.475+0000 7f49c57fa640 10 client.7623 ms_handle_connect on v2:172.21.15.188:6833/278875865 2023-11-26T07:23:35.482 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:35.475+0000 7f49c57fa640 1 -- 172.21.15.143:0/4291645368 <== mds.0 v2:172.21.15.188:6833/278875865 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+331 (crc 0 0 0) 0x562ae0ac99a0 con 0x562ae0cbbb80 2023-11-26T07:23:35.482 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:35.475+0000 7f49c57fa640 10 client.7623 handle_command_reply: tid=0 2023-11-26T07:23:35.482 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:35.475+0000 7f49ef615000 1 client.7623 shutdown 2023-11-26T07:23:35.482 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:35.475+0000 7f49ef615000 1 -- 172.21.15.143:0/4291645368 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x562ae0c94c90 msgr2=0x562ae0ca84e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:23:35.482 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:35.475+0000 7f49ef615000 1 --2- 172.21.15.143:0/4291645368 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x562ae0c94c90 0x562ae0ca84e0 secure :-1 s=READY pgs=1170 cs=0 l=1 rev1=1 crypto rx=0x7f49d400ce10 tx=0x7f49d4000ee0 comp rx=0 tx=0).stop 2023-11-26T07:23:35.482 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:35.475+0000 7f49ef615000 1 -- 172.21.15.143:0/4291645368 shutdown_connections 2023-11-26T07:23:35.482 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:35.475+0000 7f49ef615000 1 --2- 172.21.15.143:0/4291645368 >> [v2:172.21.15.188:6833/278875865,v1:172.21.15.188:6835/278875865] conn(0x562ae0cbbb80 0x562ae0d06660 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:23:35.482 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:35.475+0000 7f49ef615000 1 --2- 172.21.15.143:0/4291645368 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x562ae0c955b0 0x562ae0cbb1f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:23:35.482 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:35.475+0000 7f49ef615000 1 --2- 172.21.15.143:0/4291645368 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x562ae0c94c90 0x562ae0ca84e0 unknown :-1 s=CLOSED pgs=1170 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:23:35.482 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:35.475+0000 7f49c57fa640 0 client.7623 ms_handle_reset on v2:172.21.15.188:6833/278875865 2023-11-26T07:23:35.483 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:35.475+0000 7f49ef615000 1 --2- 172.21.15.143:0/4291645368 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x562ae0c4b320 0x562ae0ca7fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:23:35.483 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:35.475+0000 7f49ef615000 1 -- 172.21.15.143:0/4291645368 >> 172.21.15.143:0/4291645368 conn(0x562ae0c472d0 msgr2=0x562ae0c49f50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:23:35.483 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:35.475+0000 7f49ef615000 1 -- 172.21.15.143:0/4291645368 shutdown_connections 2023-11-26T07:23:35.483 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:35.475+0000 7f49ef615000 1 -- 172.21.15.143:0/4291645368 wait complete. 2023-11-26T07:23:35.483 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:35.475+0000 7f49ef615000 20 client.7623 trim_cache size 0 max 16384 2023-11-26T07:23:35.483 INFO:teuthology.orchestra.run.smithi143.stdout:{ 2023-11-26T07:23:35.483 INFO:teuthology.orchestra.run.smithi143.stdout: "cluster_fsid": "f5033060-2469-4a07-95fe-2b76f5404bc1", 2023-11-26T07:23:35.483 INFO:teuthology.orchestra.run.smithi143.stdout: "whoami": 0, 2023-11-26T07:23:35.484 INFO:teuthology.orchestra.run.smithi143.stdout: "id": 7476, 2023-11-26T07:23:35.484 INFO:teuthology.orchestra.run.smithi143.stdout: "want_state": "up:active", 2023-11-26T07:23:35.484 INFO:teuthology.orchestra.run.smithi143.stdout: "state": "up:active", 2023-11-26T07:23:35.484 INFO:teuthology.orchestra.run.smithi143.stdout: "fs_name": "cephfs", 2023-11-26T07:23:35.484 INFO:teuthology.orchestra.run.smithi143.stdout: "rank_uptime": 5.8271731300000003, 2023-11-26T07:23:35.484 INFO:teuthology.orchestra.run.smithi143.stdout: "mdsmap_epoch": 51, 2023-11-26T07:23:35.484 INFO:teuthology.orchestra.run.smithi143.stdout: "osdmap_epoch": 84, 2023-11-26T07:23:35.484 INFO:teuthology.orchestra.run.smithi143.stdout: "osdmap_epoch_barrier": 84, 2023-11-26T07:23:35.484 INFO:teuthology.orchestra.run.smithi143.stdout: "uptime": 11.726179653999999 2023-11-26T07:23:35.484 INFO:teuthology.orchestra.run.smithi143.stdout:} 2023-11-26T07:23:35.485 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:35.479+0000 7f49ec29c640 1 -- 172.21.15.143:0/1858828909 >> [v2:172.21.15.188:6836/3401128553,v1:172.21.15.188:6837/3401128553] conn(0x7f49c8066610 msgr2=0x7f49c8068ad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:23:35.485 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:35.479+0000 7f49ec29c640 1 --2- 172.21.15.143:0/1858828909 >> [v2:172.21.15.188:6836/3401128553,v1:172.21.15.188:6837/3401128553] conn(0x7f49c8066610 0x7f49c8068ad0 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7f49e00ad000 tx=0x7f49e00a6000 comp rx=0 tx=0).stop 2023-11-26T07:23:35.485 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:35.479+0000 7f49ec29c640 1 -- 172.21.15.143:0/1858828909 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f49e40ce680 msgr2=0x7f49e414f7e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:23:35.485 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:35.479+0000 7f49ec29c640 1 --2- 172.21.15.143:0/1858828909 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f49e40ce680 0x7f49e414f7e0 secure :-1 s=READY pgs=1166 cs=0 l=1 rev1=1 crypto rx=0x7f49d400c450 tx=0x7f49d400c910 comp rx=0 tx=0).stop 2023-11-26T07:23:35.485 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:35.479+0000 7f49ec29c640 1 -- 172.21.15.143:0/1858828909 shutdown_connections 2023-11-26T07:23:35.485 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:35.479+0000 7f49ec29c640 1 --2- 172.21.15.143:0/1858828909 >> [v2:172.21.15.188:6836/3401128553,v1:172.21.15.188:6837/3401128553] conn(0x7f49c8066610 0x7f49c8068ad0 unknown :-1 s=CLOSED pgs=67 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:23:35.485 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:35.479+0000 7f49ec29c640 1 --2- 172.21.15.143:0/1858828909 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f49e41596d0 0x7f49e415c090 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:23:35.486 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:35.479+0000 7f49ec29c640 1 --2- 172.21.15.143:0/1858828909 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f49e40ce680 0x7f49e414f7e0 unknown :-1 s=CLOSED pgs=1166 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:23:35.486 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:35.479+0000 7f49ec29c640 1 --2- 172.21.15.143:0/1858828909 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f49e40b2f80 0x7f49e415ab60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:23:35.486 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:35.479+0000 7f49ec29c640 1 -- 172.21.15.143:0/1858828909 >> 172.21.15.143:0/1858828909 conn(0x7f49e406a890 msgr2=0x7f49e40ae890 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:23:35.486 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:35.479+0000 7f49ec29c640 1 -- 172.21.15.143:0/1858828909 shutdown_connections 2023-11-26T07:23:35.486 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:35.479+0000 7f49ec29c640 1 -- 172.21.15.143:0/1858828909 wait complete. 2023-11-26T07:23:35.498 DEBUG:teuthology.orchestra.run.smithi143:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2023-11-26T07:23:35.636 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:35.635+0000 7f83577f8640 1 Processor -- start 2023-11-26T07:23:35.636 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:35.635+0000 7f83577f8640 1 -- start start 2023-11-26T07:23:35.638 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:35.635+0000 7f83577f8640 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7f83500cf280 0x7f83500cf710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:23:35.638 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:35.635+0000 7f83577f8640 1 -- --> v1:172.21.15.143:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f83500cc730 con 0x7f83500cd1b0 2023-11-26T07:23:35.638 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:35.635+0000 7f83577f8640 1 -- --> v1:172.21.15.188:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8350070ee0 con 0x7f8350143ac0 2023-11-26T07:23:35.638 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:35.635+0000 7f83577f8640 1 -- --> v2:172.21.15.188:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8350143ea0 con 0x7f83500cf280 2023-11-26T07:23:35.638 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:35.635+0000 7f8355d95640 1 --1- >> v1:172.21.15.143:6789/0 conn(0x7f83500cd1b0 0x7f83500cd5c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.143:6789/0 says I am v1:172.21.15.143:47988/0 (socket says 172.21.15.143:47988) 2023-11-26T07:23:35.639 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:35.635+0000 7f8355d95640 1 -- 172.21.15.143:0/2343021581 learned_addr learned my addr 172.21.15.143:0/2343021581 (peer_addr_for_me v1:172.21.15.143:0/0) 2023-11-26T07:23:35.640 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:35.635+0000 7f8354d93640 1 --2- 172.21.15.143:0/2343021581 >> v2:172.21.15.188:3300/0 conn(0x7f83500cf280 0x7f83500cf710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:23:35.640 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:35.639+0000 7f8347fff640 1 -- 172.21.15.143:0/2343021581 <== mon.0 v1:172.21.15.143:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2196885465 0 0) 0x7f83500cc730 con 0x7f83500cd1b0 2023-11-26T07:23:35.640 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:35.639+0000 7f8347fff640 1 -- 172.21.15.143:0/2343021581 --> v1:172.21.15.143:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f833c003530 con 0x7f83500cd1b0 2023-11-26T07:23:35.641 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:35.639+0000 7f8347fff640 1 -- 172.21.15.143:0/2343021581 <== mon.0 v1:172.21.15.143:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1660870955 0 0) 0x7f833c003530 con 0x7f83500cd1b0 2023-11-26T07:23:35.641 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:35.639+0000 7f8347fff640 1 -- 172.21.15.143:0/2343021581 >> v2:172.21.15.188:3300/0 conn(0x7f83500cf280 msgr2=0x7f83500cf710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-26T07:23:35.641 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:35.639+0000 7f8347fff640 1 --2- 172.21.15.143:0/2343021581 >> v2:172.21.15.188:3300/0 conn(0x7f83500cf280 0x7f83500cf710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:23:35.641 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:35.639+0000 7f8347fff640 1 -- 172.21.15.143:0/2343021581 >> v1:172.21.15.188:6789/0 conn(0x7f8350143ac0 legacy=0x7f8350070370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:23:35.641 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:35.639+0000 7f8347fff640 1 -- 172.21.15.143:0/2343021581 --> v1:172.21.15.143:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f83500711c0 con 0x7f83500cd1b0 2023-11-26T07:23:35.641 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:35.639+0000 7f8347fff640 1 -- 172.21.15.143:0/2343021581 <== mon.0 v1:172.21.15.143:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2137882523 0 0) 0x7f8348001a00 con 0x7f83500cd1b0 2023-11-26T07:23:35.642 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:35.639+0000 7f8347fff640 1 -- 172.21.15.143:0/2343021581 >> v1:172.21.15.143:6789/0 conn(0x7f83500cd1b0 legacy=0x7f83500cd5c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:23:35.642 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:35.639+0000 7f8347fff640 1 --2- 172.21.15.143:0/2343021581 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f833c003980 0x7f833c003da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:23:35.642 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:35.639+0000 7f8347fff640 1 --2- 172.21.15.143:0/2343021581 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f833c0047d0 0x7f833c004c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:23:35.642 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:35.639+0000 7f8347fff640 1 --2- 172.21.15.143:0/2343021581 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f833c005190 0x7f833c00b990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:23:35.642 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:35.639+0000 7f8347fff640 1 -- 172.21.15.143:0/2343021581 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f833c00c000 con 0x7f833c005190 2023-11-26T07:23:35.642 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:35.639+0000 7f8347fff640 1 -- 172.21.15.143:0/2343021581 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f833c00c170 con 0x7f833c003980 2023-11-26T07:23:35.643 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:35.639+0000 7f8347fff640 1 -- 172.21.15.143:0/2343021581 --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f833c00e540 con 0x7f833c0047d0 2023-11-26T07:23:35.643 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:35.639+0000 7f8355d95640 1 --2- 172.21.15.143:0/2343021581 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f833c005190 0x7f833c00b990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:23:35.643 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:35.639+0000 7f8354d93640 1 --2- 172.21.15.143:0/2343021581 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f833c0047d0 0x7f833c004c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:23:35.643 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:35.639+0000 7f8355594640 1 --2- 172.21.15.143:0/2343021581 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f833c003980 0x7f833c003da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:23:35.643 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:35.639+0000 7f8354d93640 1 -- 172.21.15.143:0/2343021581 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f833c003980 msgr2=0x7f833c003da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-26T07:23:35.643 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:35.639+0000 7f8354d93640 1 --2- 172.21.15.143:0/2343021581 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f833c003980 0x7f833c003da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:23:35.643 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:35.639+0000 7f8354d93640 1 -- 172.21.15.143:0/2343021581 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f833c005190 msgr2=0x7f833c00b990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-26T07:23:35.644 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:35.639+0000 7f8354d93640 1 --2- 172.21.15.143:0/2343021581 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f833c005190 0x7f833c00b990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:23:35.644 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:35.639+0000 7f8354d93640 1 -- 172.21.15.143:0/2343021581 --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f833c003530 con 0x7f833c0047d0 2023-11-26T07:23:35.644 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:35.639+0000 7f8355594640 1 --2- 172.21.15.143:0/2343021581 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f833c003980 0x7f833c003da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-26T07:23:35.644 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:35.639+0000 7f8355d95640 1 --2- 172.21.15.143:0/2343021581 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f833c005190 0x7f833c00b990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-26T07:23:35.644 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:35.639+0000 7f8355d95640 1 -- 172.21.15.143:0/2343021581 reap_dead start 2023-11-26T07:23:35.646 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:35.643+0000 7f8354d93640 1 --2- 172.21.15.143:0/2343021581 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f833c0047d0 0x7f833c004c50 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7f8340009ae0 tx=0x7f834001ca50 comp rx=0 tx=0).ready entity=mon.2 client_cookie=79f7a67afbc8d5e8 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:23:35.646 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:35.643+0000 7f8347fff640 1 -- 172.21.15.143:0/2343021581 <== mon.2 v2:172.21.15.188:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f83400a5640 con 0x7f833c0047d0 2023-11-26T07:23:35.646 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:35.643+0000 7f8347fff640 1 -- 172.21.15.143:0/2343021581 <== mon.2 v2:172.21.15.188:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f83400a57a0 con 0x7f833c0047d0 2023-11-26T07:23:35.646 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:35.643+0000 7f8347fff640 1 -- 172.21.15.143:0/2343021581 <== mon.2 v2:172.21.15.188:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f83400a5b50 con 0x7f833c0047d0 2023-11-26T07:23:35.646 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:35.643+0000 7f83577f8640 1 -- 172.21.15.143:0/2343021581 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f833c0047d0 msgr2=0x7f833c004c50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:23:35.646 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:35.643+0000 7f83577f8640 1 --2- 172.21.15.143:0/2343021581 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f833c0047d0 0x7f833c004c50 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7f8340009ae0 tx=0x7f834001ca50 comp rx=0 tx=0).stop 2023-11-26T07:23:35.647 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:35.643+0000 7f83577f8640 1 -- 172.21.15.143:0/2343021581 shutdown_connections 2023-11-26T07:23:35.647 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:35.643+0000 7f83577f8640 1 --2- 172.21.15.143:0/2343021581 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f833c0047d0 0x7f833c004c50 unknown :-1 s=CLOSED pgs=145 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:23:35.647 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:35.643+0000 7f83577f8640 1 -- 172.21.15.143:0/2343021581 >> 172.21.15.143:0/2343021581 conn(0x7f835006a890 msgr2=0x7f835006aca0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:23:35.648 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:35.643+0000 7f83577f8640 1 -- 172.21.15.143:0/2343021581 shutdown_connections 2023-11-26T07:23:35.648 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:35.643+0000 7f83577f8640 1 -- 172.21.15.143:0/2343021581 wait complete. 2023-11-26T07:23:35.648 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:35.643+0000 7f83577f8640 1 Processor -- start 2023-11-26T07:23:35.648 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:35.643+0000 7f83577f8640 1 -- start start 2023-11-26T07:23:35.650 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:35.643+0000 7f83577f8640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f833c0047d0 0x7f835013c020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:23:35.651 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:35.643+0000 7f83577f8640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f835013a310 0x7f835013a770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:23:35.651 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:35.647+0000 7f83577f8640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f83500b0a10 0x7f83500b0e90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:23:35.651 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:35.647+0000 7f83577f8640 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f835015b830 con 0x7f83500b0a10 2023-11-26T07:23:35.651 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:35.647+0000 7f83577f8640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f835013acb0 con 0x7f833c0047d0 2023-11-26T07:23:35.651 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:35.647+0000 7f83577f8640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f835015ac20 con 0x7f835013a310 2023-11-26T07:23:35.651 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:35.647+0000 7f8355d95640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f83500b0a10 0x7f83500b0e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:23:35.652 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:35.647+0000 7f8355d95640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f83500b0a10 0x7f83500b0e90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:55728/0 (socket says 172.21.15.143:55728) 2023-11-26T07:23:35.652 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:35.647+0000 7f8355d95640 1 -- 172.21.15.143:0/761508822 learned_addr learned my addr 172.21.15.143:0/761508822 (peer_addr_for_me v2:172.21.15.143:0/0) 2023-11-26T07:23:35.652 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:35.647+0000 7f8355594640 1 --2- 172.21.15.143:0/761508822 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f833c0047d0 0x7f835013c020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:23:35.653 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:35.647+0000 7f8354d93640 1 --2- 172.21.15.143:0/761508822 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f835013a310 0x7f835013a770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:23:35.653 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:35.647+0000 7f8355d95640 1 -- 172.21.15.143:0/761508822 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f835013a310 msgr2=0x7f835013a770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:23:35.653 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:35.647+0000 7f8355d95640 1 --2- 172.21.15.143:0/761508822 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f835013a310 0x7f835013a770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:23:35.653 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:35.647+0000 7f8355d95640 1 -- 172.21.15.143:0/761508822 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f833c0047d0 msgr2=0x7f835013c020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:23:35.653 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:35.647+0000 7f8355d95640 1 --2- 172.21.15.143:0/761508822 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f833c0047d0 0x7f835013c020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:23:35.653 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:35.647+0000 7f8355d95640 1 -- 172.21.15.143:0/761508822 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f835015af00 con 0x7f83500b0a10 2023-11-26T07:23:35.653 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:35.647+0000 7f8355d95640 1 --2- 172.21.15.143:0/761508822 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f83500b0a10 0x7f83500b0e90 secure :-1 s=READY pgs=1172 cs=0 l=1 rev1=1 crypto rx=0x7f83500718d0 tx=0x7f8348007430 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:23:35.654 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:35.647+0000 7f8345ffb640 1 -- 172.21.15.143:0/761508822 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8348007b00 con 0x7f83500b0a10 2023-11-26T07:23:35.654 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:35.647+0000 7f8345ffb640 1 -- 172.21.15.143:0/761508822 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f834800ae60 con 0x7f83500b0a10 2023-11-26T07:23:35.654 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:35.647+0000 7f8345ffb640 1 -- 172.21.15.143:0/761508822 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f83480135f0 con 0x7f83500b0a10 2023-11-26T07:23:35.654 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:35.647+0000 7f83577f8640 1 -- 172.21.15.143:0/761508822 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f835015b1e0 con 0x7f83500b0a10 2023-11-26T07:23:35.654 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:35.647+0000 7f83577f8640 1 -- 172.21.15.143:0/761508822 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f83500b13d0 con 0x7f83500b0a10 2023-11-26T07:23:35.654 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:35.647+0000 7f83577f8640 1 -- 172.21.15.143:0/761508822 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f835015b9a0 con 0x7f83500b0a10 2023-11-26T07:23:35.654 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:35.647+0000 7f8345ffb640 1 -- 172.21.15.143:0/761508822 <== mon.0 v2:172.21.15.143:3300/0 4 ==== mgrmap(e 16) v1 ==== 85954+0+0 (secure 0 0 0) 0x7f8348013750 con 0x7f83500b0a10 2023-11-26T07:23:35.655 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:35.651+0000 7f8345ffb640 1 --2- 172.21.15.143:0/761508822 >> [v2:172.21.15.188:6836/3401128553,v1:172.21.15.188:6837/3401128553] conn(0x7f83340665c0 0x7f8334068a80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:23:35.655 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:35.651+0000 7f8345ffb640 1 -- 172.21.15.143:0/761508822 <== mon.0 v2:172.21.15.143:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f8348084600 con 0x7f83500b0a10 2023-11-26T07:23:35.655 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:35.651+0000 7f8355594640 1 --2- 172.21.15.143:0/761508822 >> [v2:172.21.15.188:6836/3401128553,v1:172.21.15.188:6837/3401128553] conn(0x7f83340665c0 0x7f8334068a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:23:35.655 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:35.651+0000 7f8355594640 1 --2- 172.21.15.143:0/761508822 >> [v2:172.21.15.188:6836/3401128553,v1:172.21.15.188:6837/3401128553] conn(0x7f83340665c0 0x7f8334068a80 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7f834c001370 tx=0x7f834c009040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:23:35.656 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:35.651+0000 7f8345ffb640 1 -- 172.21.15.143:0/761508822 <== mon.0 v2:172.21.15.143:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172649 (secure 0 0 0) 0x7f83480518f0 con 0x7f83500b0a10 2023-11-26T07:23:35.843 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:35.843+0000 7f83577f8640 1 -- 172.21.15.143:0/761508822 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "fs ls"} v 0) v1 -- 0x7f8350067680 con 0x7f83500b0a10 2023-11-26T07:23:35.844 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:35.843+0000 7f8345ffb640 1 -- 172.21.15.143:0/761508822 <== mon.0 v2:172.21.15.143:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v51) v1 ==== 53+0+73 (secure 0 0 0) 0x7f8348054f10 con 0x7f83500b0a10 2023-11-26T07:23:35.844 INFO:teuthology.orchestra.run.smithi143.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2023-11-26T07:23:35.846 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:35.843+0000 7f83577f8640 1 -- 172.21.15.143:0/761508822 >> [v2:172.21.15.188:6836/3401128553,v1:172.21.15.188:6837/3401128553] conn(0x7f83340665c0 msgr2=0x7f8334068a80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:23:35.846 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:35.843+0000 7f83577f8640 1 --2- 172.21.15.143:0/761508822 >> [v2:172.21.15.188:6836/3401128553,v1:172.21.15.188:6837/3401128553] conn(0x7f83340665c0 0x7f8334068a80 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7f834c001370 tx=0x7f834c009040 comp rx=0 tx=0).stop 2023-11-26T07:23:35.846 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:35.843+0000 7f83577f8640 1 -- 172.21.15.143:0/761508822 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f83500b0a10 msgr2=0x7f83500b0e90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:23:35.846 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:35.843+0000 7f83577f8640 1 --2- 172.21.15.143:0/761508822 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f83500b0a10 0x7f83500b0e90 secure :-1 s=READY pgs=1172 cs=0 l=1 rev1=1 crypto rx=0x7f83500718d0 tx=0x7f8348007430 comp rx=0 tx=0).stop 2023-11-26T07:23:35.846 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:35.843+0000 7f83577f8640 1 -- 172.21.15.143:0/761508822 shutdown_connections 2023-11-26T07:23:35.847 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:35.843+0000 7f83577f8640 1 --2- 172.21.15.143:0/761508822 >> [v2:172.21.15.188:6836/3401128553,v1:172.21.15.188:6837/3401128553] conn(0x7f83340665c0 0x7f8334068a80 unknown :-1 s=CLOSED pgs=68 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:23:35.847 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:35.843+0000 7f83577f8640 1 --2- 172.21.15.143:0/761508822 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f83500b0a10 0x7f83500b0e90 unknown :-1 s=CLOSED pgs=1172 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:23:35.847 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:35.843+0000 7f83577f8640 1 --2- 172.21.15.143:0/761508822 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f835013a310 0x7f835013a770 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:23:35.847 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:35.843+0000 7f83577f8640 1 --2- 172.21.15.143:0/761508822 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f833c0047d0 0x7f835013c020 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:23:35.847 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:35.843+0000 7f83577f8640 1 -- 172.21.15.143:0/761508822 >> 172.21.15.143:0/761508822 conn(0x7f835006a890 msgr2=0x7f835006ac70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:23:35.847 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:35.843+0000 7f83577f8640 1 -- 172.21.15.143:0/761508822 shutdown_connections 2023-11-26T07:23:35.847 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:35.843+0000 7f83577f8640 1 -- 172.21.15.143:0/761508822 wait complete. 2023-11-26T07:23:35.865 INFO:tasks.cephfs.mount:Mounting default Ceph FS; just confirmed its presence on cluster 2023-11-26T07:23:35.865 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2023-11-26T07:23:35.865 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi143.front.sepia.ceph.com 2023-11-26T07:23:35.865 INFO:tasks.cephfs.mount:self.client.name = client.0 2023-11-26T07:23:35.865 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2023-11-26T07:23:35.865 INFO:tasks.cephfs.mount:self.cephfs_name = None 2023-11-26T07:23:35.865 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2023-11-26T07:23:35.865 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2023-11-26T07:23:35.866 INFO:tasks.cephfs.mount:Setting the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2023-11-26T07:23:35.866 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-26T07:23:35.866 DEBUG:teuthology.orchestra.run.smithi143:> ip addr 2023-11-26T07:23:35.876 INFO:teuthology.orchestra.run.smithi143.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2023-11-26T07:23:35.876 INFO:teuthology.orchestra.run.smithi143.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2023-11-26T07:23:35.876 INFO:teuthology.orchestra.run.smithi143.stdout: inet 127.0.0.1/8 scope host lo 2023-11-26T07:23:35.876 INFO:teuthology.orchestra.run.smithi143.stdout: valid_lft forever preferred_lft forever 2023-11-26T07:23:35.876 INFO:teuthology.orchestra.run.smithi143.stdout: inet6 ::1/128 scope host 2023-11-26T07:23:35.876 INFO:teuthology.orchestra.run.smithi143.stdout: valid_lft forever preferred_lft forever 2023-11-26T07:23:35.876 INFO:teuthology.orchestra.run.smithi143.stdout:2: eno1: mtu 1500 qdisc noop state DOWN group default qlen 1000 2023-11-26T07:23:35.876 INFO:teuthology.orchestra.run.smithi143.stdout: link/ether 0c:c4:7a:d9:3b:94 brd ff:ff:ff:ff:ff:ff 2023-11-26T07:23:35.877 INFO:teuthology.orchestra.run.smithi143.stdout: altname enp7s0f0 2023-11-26T07:23:35.877 INFO:teuthology.orchestra.run.smithi143.stdout:3: eno2: mtu 1500 qdisc noop state DOWN group default qlen 1000 2023-11-26T07:23:35.877 INFO:teuthology.orchestra.run.smithi143.stdout: link/ether 0c:c4:7a:d9:3b:95 brd ff:ff:ff:ff:ff:ff 2023-11-26T07:23:35.877 INFO:teuthology.orchestra.run.smithi143.stdout: altname enp7s0f1 2023-11-26T07:23:35.877 INFO:teuthology.orchestra.run.smithi143.stdout:4: enp3s0f0: mtu 1500 qdisc noop state DOWN group default qlen 1000 2023-11-26T07:23:35.877 INFO:teuthology.orchestra.run.smithi143.stdout: link/ether 0c:c4:7a:88:77:82 brd ff:ff:ff:ff:ff:ff 2023-11-26T07:23:35.877 INFO:teuthology.orchestra.run.smithi143.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2023-11-26T07:23:35.877 INFO:teuthology.orchestra.run.smithi143.stdout: link/ether 0c:c4:7a:88:77:83 brd ff:ff:ff:ff:ff:ff 2023-11-26T07:23:35.877 INFO:teuthology.orchestra.run.smithi143.stdout: inet 172.21.15.143/20 metric 100 brd 172.21.15.255 scope global dynamic enp3s0f1 2023-11-26T07:23:35.877 INFO:teuthology.orchestra.run.smithi143.stdout: valid_lft 42058sec preferred_lft 42058sec 2023-11-26T07:23:35.877 INFO:teuthology.orchestra.run.smithi143.stdout: inet6 fe80::ec4:7aff:fe88:7783/64 scope link 2023-11-26T07:23:35.877 INFO:teuthology.orchestra.run.smithi143.stdout: valid_lft forever preferred_lft forever 2023-11-26T07:23:35.878 INFO:teuthology.orchestra.run.smithi143.stdout:6: docker0: mtu 1500 qdisc noqueue state DOWN group default 2023-11-26T07:23:35.878 INFO:teuthology.orchestra.run.smithi143.stdout: link/ether 02:42:c4:b4:2f:eb brd ff:ff:ff:ff:ff:ff 2023-11-26T07:23:35.878 INFO:teuthology.orchestra.run.smithi143.stdout: inet 172.17.0.1/16 brd 172.17.255.255 scope global docker0 2023-11-26T07:23:35.878 INFO:teuthology.orchestra.run.smithi143.stdout: valid_lft forever preferred_lft forever 2023-11-26T07:23:35.878 INFO:teuthology.orchestra.run.smithi143.stdout:7: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2023-11-26T07:23:35.878 INFO:teuthology.orchestra.run.smithi143.stdout: link/ether 5e:1f:84:6d:79:ae brd ff:ff:ff:ff:ff:ff 2023-11-26T07:23:35.878 INFO:teuthology.orchestra.run.smithi143.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2023-11-26T07:23:35.878 INFO:teuthology.orchestra.run.smithi143.stdout: valid_lft forever preferred_lft forever 2023-11-26T07:23:35.878 INFO:teuthology.orchestra.run.smithi143.stdout: inet6 fe80::5c1f:84ff:fe6d:79ae/64 scope link 2023-11-26T07:23:35.878 INFO:teuthology.orchestra.run.smithi143.stdout: valid_lft forever preferred_lft forever 2023-11-26T07:23:35.878 INFO:teuthology.orchestra.run.smithi143.stdout:8: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2023-11-26T07:23:35.879 INFO:teuthology.orchestra.run.smithi143.stdout: link/ether 0e:5c:f6:1f:82:2f brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.0 2023-11-26T07:23:35.879 INFO:teuthology.orchestra.run.smithi143.stdout: inet6 fe80::c5c:f6ff:fe1f:822f/64 scope link 2023-11-26T07:23:35.879 INFO:teuthology.orchestra.run.smithi143.stdout: valid_lft forever preferred_lft forever 2023-11-26T07:23:35.879 INFO:teuthology.orchestra.run.smithi143.stdout:9: brx.1@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2023-11-26T07:23:35.879 INFO:teuthology.orchestra.run.smithi143.stdout: link/ether c6:a6:3d:69:d0:db brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.2 2023-11-26T07:23:35.879 INFO:teuthology.orchestra.run.smithi143.stdout: inet6 fe80::c4a6:3dff:fe69:d0db/64 scope link 2023-11-26T07:23:35.879 INFO:teuthology.orchestra.run.smithi143.stdout: valid_lft forever preferred_lft forever 2023-11-26T07:23:35.879 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-26T07:23:35.879 DEBUG:teuthology.orchestra.run.smithi143:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2023-11-26T07:23:35.936 INFO:teuthology.orchestra.run.smithi143.stdout:1 2023-11-26T07:23:35.938 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-26T07:23:35.939 DEBUG:teuthology.orchestra.run.smithi143:> route 2023-11-26T07:23:35.993 INFO:teuthology.orchestra.run.smithi143.stdout:Kernel IP routing table 2023-11-26T07:23:35.993 INFO:teuthology.orchestra.run.smithi143.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2023-11-26T07:23:35.993 INFO:teuthology.orchestra.run.smithi143.stdout:default _gateway 0.0.0.0 UG 0 0 0 enp3s0f1 2023-11-26T07:23:35.993 INFO:teuthology.orchestra.run.smithi143.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2023-11-26T07:23:35.993 INFO:teuthology.orchestra.run.smithi143.stdout:172.17.0.0 0.0.0.0 255.255.0.0 U 0 0 0 docker0 2023-11-26T07:23:35.993 INFO:teuthology.orchestra.run.smithi143.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 enp3s0f1 2023-11-26T07:23:35.994 INFO:teuthology.orchestra.run.smithi143.stdout:ns1.front.sepia 0.0.0.0 255.255.255.255 UH 100 0 0 enp3s0f1 2023-11-26T07:23:35.994 INFO:teuthology.orchestra.run.smithi143.stdout:172.21.0.2 0.0.0.0 255.255.255.255 UH 100 0 0 enp3s0f1 2023-11-26T07:23:35.994 INFO:teuthology.orchestra.run.smithi143.stdout:_gateway 0.0.0.0 255.255.255.255 UH 100 0 0 enp3s0f1 2023-11-26T07:23:35.994 INFO:teuthology.orchestra.run.smithi143.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2023-11-26T07:23:35.994 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-26T07:23:35.994 DEBUG:teuthology.orchestra.run.smithi143:> (cd / && exec bash -c ' 2023-11-26T07:23:35.994 DEBUG:teuthology.orchestra.run.smithi143:> set -e 2023-11-26T07:23:35.995 DEBUG:teuthology.orchestra.run.smithi143:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2023-11-26T07:23:35.995 DEBUG:teuthology.orchestra.run.smithi143:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2023-11-26T07:23:35.995 DEBUG:teuthology.orchestra.run.smithi143:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2023-11-26T07:23:35.995 DEBUG:teuthology.orchestra.run.smithi143:> ') 2023-11-26T07:23:36.084 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-26T07:23:36.085 DEBUG:teuthology.orchestra.run.smithi143:> ip netns list 2023-11-26T07:23:36.093 INFO:teuthology.orchestra.run.smithi143.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2023-11-26T07:23:36.093 INFO:teuthology.orchestra.run.smithi143.stdout:ceph-ns--home-ubuntu-cephtest-mnt.2 (id: 1) 2023-11-26T07:23:36.094 INFO:tasks.cephfs.mount:Resuming the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2023-11-26T07:23:36.094 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-26T07:23:36.095 DEBUG:teuthology.orchestra.run.smithi143:> sudo ip link set brx.0 up 2023-11-26T07:23:36.158 INFO:tasks.cephfs.fuse_mount:Client client.0 config is {} 2023-11-26T07:23:36.159 INFO:teuthology.orchestra.run:Running command with timeout 60 2023-11-26T07:23:36.159 DEBUG:teuthology.orchestra.run.smithi143:> mkdir -p -v /home/ubuntu/cephtest/mnt.0 2023-11-26T07:23:36.214 INFO:teuthology.orchestra.run.smithi143.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.0' 2023-11-26T07:23:36.215 INFO:teuthology.orchestra.run:Running command with timeout 60 2023-11-26T07:23:36.215 DEBUG:teuthology.orchestra.run.smithi143:> chmod 0000 /home/ubuntu/cephtest/mnt.0 2023-11-26T07:23:36.266 DEBUG:teuthology.orchestra.run.smithi143:> sudo modprobe fuse 2023-11-26T07:23:36.329 DEBUG:teuthology.orchestra.run.smithi143:> cat /proc/self/mounts | awk '{print $2}' 2023-11-26T07:23:36.380 INFO:teuthology.orchestra.run.smithi143.stdout:/sys 2023-11-26T07:23:36.380 INFO:teuthology.orchestra.run.smithi143.stdout:/proc 2023-11-26T07:23:36.380 INFO:teuthology.orchestra.run.smithi143.stdout:/dev 2023-11-26T07:23:36.380 INFO:teuthology.orchestra.run.smithi143.stdout:/dev/pts 2023-11-26T07:23:36.380 INFO:teuthology.orchestra.run.smithi143.stdout:/run 2023-11-26T07:23:36.380 INFO:teuthology.orchestra.run.smithi143.stdout:/ 2023-11-26T07:23:36.381 INFO:teuthology.orchestra.run.smithi143.stdout:/sys/kernel/security 2023-11-26T07:23:36.381 INFO:teuthology.orchestra.run.smithi143.stdout:/dev/shm 2023-11-26T07:23:36.381 INFO:teuthology.orchestra.run.smithi143.stdout:/run/lock 2023-11-26T07:23:36.381 INFO:teuthology.orchestra.run.smithi143.stdout:/sys/fs/cgroup 2023-11-26T07:23:36.381 INFO:teuthology.orchestra.run.smithi143.stdout:/sys/fs/pstore 2023-11-26T07:23:36.381 INFO:teuthology.orchestra.run.smithi143.stdout:/sys/fs/bpf 2023-11-26T07:23:36.381 INFO:teuthology.orchestra.run.smithi143.stdout:/proc/sys/fs/binfmt_misc 2023-11-26T07:23:36.381 INFO:teuthology.orchestra.run.smithi143.stdout:/dev/hugepages 2023-11-26T07:23:36.382 INFO:teuthology.orchestra.run.smithi143.stdout:/dev/mqueue 2023-11-26T07:23:36.382 INFO:teuthology.orchestra.run.smithi143.stdout:/sys/kernel/debug 2023-11-26T07:23:36.382 INFO:teuthology.orchestra.run.smithi143.stdout:/sys/kernel/tracing 2023-11-26T07:23:36.382 INFO:teuthology.orchestra.run.smithi143.stdout:/proc/fs/nfsd 2023-11-26T07:23:36.382 INFO:teuthology.orchestra.run.smithi143.stdout:/sys/fs/fuse/connections 2023-11-26T07:23:36.382 INFO:teuthology.orchestra.run.smithi143.stdout:/sys/kernel/config 2023-11-26T07:23:36.383 INFO:teuthology.orchestra.run.smithi143.stdout:/run/credentials/systemd-sysusers.service 2023-11-26T07:23:36.383 INFO:teuthology.orchestra.run.smithi143.stdout:/run/rpc_pipefs 2023-11-26T07:23:36.383 INFO:teuthology.orchestra.run.smithi143.stdout:/run/user/1000 2023-11-26T07:23:36.383 INFO:teuthology.orchestra.run.smithi143.stdout:/var/lib/ceph 2023-11-26T07:23:36.383 INFO:teuthology.orchestra.run.smithi143.stdout:/var/lib/ceph/osd/ceph-0 2023-11-26T07:23:36.383 INFO:teuthology.orchestra.run.smithi143.stdout:/var/lib/ceph/osd/ceph-1 2023-11-26T07:23:36.383 INFO:teuthology.orchestra.run.smithi143.stdout:/var/lib/ceph/osd/ceph-2 2023-11-26T07:23:36.384 INFO:teuthology.orchestra.run.smithi143.stdout:/var/lib/ceph/osd/ceph-3 2023-11-26T07:23:36.384 INFO:teuthology.orchestra.run.smithi143.stdout:/run/netns 2023-11-26T07:23:36.384 INFO:teuthology.orchestra.run.smithi143.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2023-11-26T07:23:36.384 INFO:teuthology.orchestra.run.smithi143.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2023-11-26T07:23:36.384 INFO:teuthology.orchestra.run.smithi143.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 2023-11-26T07:23:36.384 INFO:teuthology.orchestra.run.smithi143.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 2023-11-26T07:23:36.385 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-26T07:23:36.385 DEBUG:teuthology.orchestra.run.smithi143:> ls /sys/fs/fuse/connections 2023-11-26T07:23:36.434 INFO:tasks.cephfs.fuse_mount:Pre-mount connections: [] 2023-11-26T07:23:36.434 DEBUG:teuthology.orchestra.run.smithi143:> (cd /home/ubuntu/cephtest && exec sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-fuse -f --admin-socket '/var/run/ceph/$cluster-$name.$pid.asok' /home/ubuntu/cephtest/mnt.0 --id 0) 2023-11-26T07:23:36.477 DEBUG:teuthology.orchestra.run.smithi143:> sudo modprobe fuse 2023-11-26T07:23:36.498 DEBUG:teuthology.orchestra.run.smithi143:> cat /proc/self/mounts | awk '{print $2}' 2023-11-26T07:23:36.553 INFO:teuthology.orchestra.run.smithi143.stdout:/sys 2023-11-26T07:23:36.554 INFO:teuthology.orchestra.run.smithi143.stdout:/proc 2023-11-26T07:23:36.554 INFO:teuthology.orchestra.run.smithi143.stdout:/dev 2023-11-26T07:23:36.554 INFO:teuthology.orchestra.run.smithi143.stdout:/dev/pts 2023-11-26T07:23:36.554 INFO:teuthology.orchestra.run.smithi143.stdout:/run 2023-11-26T07:23:36.554 INFO:teuthology.orchestra.run.smithi143.stdout:/ 2023-11-26T07:23:36.554 INFO:teuthology.orchestra.run.smithi143.stdout:/sys/kernel/security 2023-11-26T07:23:36.554 INFO:teuthology.orchestra.run.smithi143.stdout:/dev/shm 2023-11-26T07:23:36.554 INFO:teuthology.orchestra.run.smithi143.stdout:/run/lock 2023-11-26T07:23:36.554 INFO:teuthology.orchestra.run.smithi143.stdout:/sys/fs/cgroup 2023-11-26T07:23:36.554 INFO:teuthology.orchestra.run.smithi143.stdout:/sys/fs/pstore 2023-11-26T07:23:36.554 INFO:teuthology.orchestra.run.smithi143.stdout:/sys/fs/bpf 2023-11-26T07:23:36.555 INFO:teuthology.orchestra.run.smithi143.stdout:/proc/sys/fs/binfmt_misc 2023-11-26T07:23:36.555 INFO:teuthology.orchestra.run.smithi143.stdout:/dev/hugepages 2023-11-26T07:23:36.555 INFO:teuthology.orchestra.run.smithi143.stdout:/dev/mqueue 2023-11-26T07:23:36.555 INFO:teuthology.orchestra.run.smithi143.stdout:/sys/kernel/debug 2023-11-26T07:23:36.555 INFO:teuthology.orchestra.run.smithi143.stdout:/sys/kernel/tracing 2023-11-26T07:23:36.555 INFO:teuthology.orchestra.run.smithi143.stdout:/proc/fs/nfsd 2023-11-26T07:23:36.555 INFO:teuthology.orchestra.run.smithi143.stdout:/sys/fs/fuse/connections 2023-11-26T07:23:36.555 INFO:teuthology.orchestra.run.smithi143.stdout:/sys/kernel/config 2023-11-26T07:23:36.555 INFO:teuthology.orchestra.run.smithi143.stdout:/run/credentials/systemd-sysusers.service 2023-11-26T07:23:36.555 INFO:teuthology.orchestra.run.smithi143.stdout:/run/rpc_pipefs 2023-11-26T07:23:36.555 INFO:teuthology.orchestra.run.smithi143.stdout:/run/user/1000 2023-11-26T07:23:36.555 INFO:teuthology.orchestra.run.smithi143.stdout:/var/lib/ceph 2023-11-26T07:23:36.556 INFO:teuthology.orchestra.run.smithi143.stdout:/var/lib/ceph/osd/ceph-0 2023-11-26T07:23:36.556 INFO:teuthology.orchestra.run.smithi143.stdout:/var/lib/ceph/osd/ceph-1 2023-11-26T07:23:36.556 INFO:teuthology.orchestra.run.smithi143.stdout:/var/lib/ceph/osd/ceph-2 2023-11-26T07:23:36.556 INFO:teuthology.orchestra.run.smithi143.stdout:/var/lib/ceph/osd/ceph-3 2023-11-26T07:23:36.556 INFO:teuthology.orchestra.run.smithi143.stdout:/run/netns 2023-11-26T07:23:36.556 INFO:teuthology.orchestra.run.smithi143.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2023-11-26T07:23:36.556 INFO:teuthology.orchestra.run.smithi143.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2023-11-26T07:23:36.556 INFO:teuthology.orchestra.run.smithi143.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 2023-11-26T07:23:36.556 INFO:teuthology.orchestra.run.smithi143.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 2023-11-26T07:23:36.557 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-26T07:23:36.557 DEBUG:teuthology.orchestra.run.smithi143:> ls /sys/fs/fuse/connections 2023-11-26T07:23:36.575 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi143.stderr:2023-11-26T07:23:36.575+0000 7f745ac2a580 -1 init, newargv = 0x55700d5350a0 newargc=15 2023-11-26T07:23:36.575 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi143.stderr:ceph-fuse[57362]: starting ceph client 2023-11-26T07:23:36.583 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi143.stderr:ceph-fuse[57362]: starting fuse 2023-11-26T07:23:36.584 INFO:teuthology.orchestra.run.smithi143.stdout:43 2023-11-26T07:23:36.584 INFO:tasks.cephfs.fuse_mount:Post-mount connections: [43] 2023-11-26T07:23:36.585 DEBUG:teuthology.orchestra.run.smithi143:> sudo stdin-killer -- python3 -c ' 2023-11-26T07:23:36.585 DEBUG:teuthology.orchestra.run.smithi143:> import glob 2023-11-26T07:23:36.585 DEBUG:teuthology.orchestra.run.smithi143:> import re 2023-11-26T07:23:36.585 DEBUG:teuthology.orchestra.run.smithi143:> import os 2023-11-26T07:23:36.585 DEBUG:teuthology.orchestra.run.smithi143:> import subprocess 2023-11-26T07:23:36.585 DEBUG:teuthology.orchestra.run.smithi143:> 2023-11-26T07:23:36.585 DEBUG:teuthology.orchestra.run.smithi143:> def _find_admin_socket(client_name): 2023-11-26T07:23:36.585 DEBUG:teuthology.orchestra.run.smithi143:> asok_path = "/var/run/ceph/ceph-client.0.*.asok" 2023-11-26T07:23:36.585 DEBUG:teuthology.orchestra.run.smithi143:> files = glob.glob(asok_path) 2023-11-26T07:23:36.585 DEBUG:teuthology.orchestra.run.smithi143:> mountpoint = "/home/ubuntu/cephtest/mnt.0" 2023-11-26T07:23:36.585 DEBUG:teuthology.orchestra.run.smithi143:> 2023-11-26T07:23:36.586 DEBUG:teuthology.orchestra.run.smithi143:> # Given a non-glob path, it better be there 2023-11-26T07:23:36.586 DEBUG:teuthology.orchestra.run.smithi143:> if "*" not in asok_path: 2023-11-26T07:23:36.586 DEBUG:teuthology.orchestra.run.smithi143:> assert(len(files) == 1) 2023-11-26T07:23:36.586 DEBUG:teuthology.orchestra.run.smithi143:> return files[0] 2023-11-26T07:23:36.586 DEBUG:teuthology.orchestra.run.smithi143:> 2023-11-26T07:23:36.586 DEBUG:teuthology.orchestra.run.smithi143:> for f in files: 2023-11-26T07:23:36.586 DEBUG:teuthology.orchestra.run.smithi143:> pid = re.match(".*\.(\d+)\.asok$", f).group(1) 2023-11-26T07:23:36.586 DEBUG:teuthology.orchestra.run.smithi143:> if os.path.exists("/proc/{0}".format(pid)): 2023-11-26T07:23:36.586 DEBUG:teuthology.orchestra.run.smithi143:> with open("/proc/{0}/cmdline".format(pid), '"'"'r'"'"') as proc_f: 2023-11-26T07:23:36.586 DEBUG:teuthology.orchestra.run.smithi143:> contents = proc_f.read() 2023-11-26T07:23:36.586 DEBUG:teuthology.orchestra.run.smithi143:> if mountpoint in contents: 2023-11-26T07:23:36.586 DEBUG:teuthology.orchestra.run.smithi143:> return f 2023-11-26T07:23:36.587 DEBUG:teuthology.orchestra.run.smithi143:> raise RuntimeError("Client socket {0} not found".format(client_name)) 2023-11-26T07:23:36.587 DEBUG:teuthology.orchestra.run.smithi143:> 2023-11-26T07:23:36.587 DEBUG:teuthology.orchestra.run.smithi143:> print(_find_admin_socket("client.0")) 2023-11-26T07:23:36.587 DEBUG:teuthology.orchestra.run.smithi143:> ' 2023-11-26T07:23:36.713 INFO:teuthology.orchestra.run.smithi143.stdout:/var/run/ceph/ceph-client.0.57362.asok 2023-11-26T07:23:36.717 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:36 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2023-11-26T07:23:36.725 INFO:tasks.cephfs.fuse_mount:Found client admin socket at /var/run/ceph/ceph-client.0.57362.asok 2023-11-26T07:23:36.725 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-26T07:23:36.725 DEBUG:teuthology.orchestra.run.smithi143:> sudo ceph --admin-daemon /var/run/ceph/ceph-client.0.57362.asok status 2023-11-26T07:23:36.894 INFO:teuthology.orchestra.run.smithi143.stdout:{ 2023-11-26T07:23:36.894 INFO:teuthology.orchestra.run.smithi143.stdout: "metadata": { 2023-11-26T07:23:36.894 INFO:teuthology.orchestra.run.smithi143.stdout: "ceph_sha1": "44c2237fe77713214148c1fd622c872f346f1b99", 2023-11-26T07:23:36.895 INFO:teuthology.orchestra.run.smithi143.stdout: "ceph_version": "ceph version 18.0.0-7484-g44c2237f (44c2237fe77713214148c1fd622c872f346f1b99) reef (dev)", 2023-11-26T07:23:36.895 INFO:teuthology.orchestra.run.smithi143.stdout: "entity_id": "0", 2023-11-26T07:23:36.895 INFO:teuthology.orchestra.run.smithi143.stdout: "hostname": "smithi143", 2023-11-26T07:23:36.895 INFO:teuthology.orchestra.run.smithi143.stdout: "mount_point": "/home/ubuntu/cephtest/mnt.0", 2023-11-26T07:23:36.895 INFO:teuthology.orchestra.run.smithi143.stdout: "pid": "57362", 2023-11-26T07:23:36.895 INFO:teuthology.orchestra.run.smithi143.stdout: "root": "/" 2023-11-26T07:23:36.895 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2023-11-26T07:23:36.895 INFO:teuthology.orchestra.run.smithi143.stdout: "dentry_count": 0, 2023-11-26T07:23:36.896 INFO:teuthology.orchestra.run.smithi143.stdout: "dentry_pinned_count": 0, 2023-11-26T07:23:36.896 INFO:teuthology.orchestra.run.smithi143.stdout: "id": 6422, 2023-11-26T07:23:36.896 INFO:teuthology.orchestra.run.smithi143.stdout: "inst": { 2023-11-26T07:23:36.896 INFO:teuthology.orchestra.run.smithi143.stdout: "name": { 2023-11-26T07:23:36.896 INFO:teuthology.orchestra.run.smithi143.stdout: "type": "client", 2023-11-26T07:23:36.896 INFO:teuthology.orchestra.run.smithi143.stdout: "num": 6422 2023-11-26T07:23:36.896 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2023-11-26T07:23:36.896 INFO:teuthology.orchestra.run.smithi143.stdout: "addr": { 2023-11-26T07:23:36.896 INFO:teuthology.orchestra.run.smithi143.stdout: "type": "v1", 2023-11-26T07:23:36.896 INFO:teuthology.orchestra.run.smithi143.stdout: "addr": "192.168.0.1:0", 2023-11-26T07:23:36.896 INFO:teuthology.orchestra.run.smithi143.stdout: "nonce": 3673233968 2023-11-26T07:23:36.897 INFO:teuthology.orchestra.run.smithi143.stdout: } 2023-11-26T07:23:36.897 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2023-11-26T07:23:36.897 INFO:teuthology.orchestra.run.smithi143.stdout: "addr": { 2023-11-26T07:23:36.897 INFO:teuthology.orchestra.run.smithi143.stdout: "type": "v1", 2023-11-26T07:23:36.897 INFO:teuthology.orchestra.run.smithi143.stdout: "addr": "192.168.0.1:0", 2023-11-26T07:23:36.897 INFO:teuthology.orchestra.run.smithi143.stdout: "nonce": 3673233968 2023-11-26T07:23:36.897 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2023-11-26T07:23:36.897 INFO:teuthology.orchestra.run.smithi143.stdout: "inst_str": "client.6422 192.168.0.1:0/3673233968", 2023-11-26T07:23:36.897 INFO:teuthology.orchestra.run.smithi143.stdout: "addr_str": "192.168.0.1:0/3673233968", 2023-11-26T07:23:36.898 INFO:teuthology.orchestra.run.smithi143.stdout: "inode_count": 1, 2023-11-26T07:23:36.898 INFO:teuthology.orchestra.run.smithi143.stdout: "mds_epoch": 51, 2023-11-26T07:23:36.898 INFO:teuthology.orchestra.run.smithi143.stdout: "osd_epoch": 84, 2023-11-26T07:23:36.898 INFO:teuthology.orchestra.run.smithi143.stdout: "osd_epoch_barrier": 0, 2023-11-26T07:23:36.898 INFO:teuthology.orchestra.run.smithi143.stdout: "blocklisted": false, 2023-11-26T07:23:36.898 INFO:teuthology.orchestra.run.smithi143.stdout: "fs_name": "cephfs" 2023-11-26T07:23:36.898 INFO:teuthology.orchestra.run.smithi143.stdout:} 2023-11-26T07:23:36.909 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-26T07:23:36.910 DEBUG:teuthology.orchestra.run.smithi143:> stat --file-system '--printf=%T 2023-11-26T07:23:36.910 DEBUG:teuthology.orchestra.run.smithi143:> ' -- /home/ubuntu/cephtest/mnt.0 2023-11-26T07:23:36.920 INFO:teuthology.orchestra.run.smithi143.stdout:fuseblk 2023-11-26T07:23:36.920 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.0 2023-11-26T07:23:36.920 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-26T07:23:36.920 DEBUG:teuthology.orchestra.run.smithi143:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.0 2023-11-26T07:23:36.983 DEBUG:teuthology.orchestra.run.smithi143:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2023-11-26T07:23:37.162 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:37.159+0000 7fb6f492a640 1 Processor -- start 2023-11-26T07:23:37.162 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:37.159+0000 7fb6f492a640 1 -- start start 2023-11-26T07:23:37.163 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:37.163+0000 7fb6f492a640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fb6f0143ac0 0x7fb6f0070370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:23:37.163 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:37.163+0000 7fb6f492a640 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7fb6f00cf990 0x7fb6f00cd1b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:23:37.163 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:37.163+0000 7fb6f492a640 1 -- --> v1:172.21.15.188:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb6f00cc730 con 0x7fb6f00708b0 2023-11-26T07:23:37.164 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:37.163+0000 7fb6f492a640 1 -- --> v2:172.21.15.188:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb6f0143ea0 con 0x7fb6f00cf990 2023-11-26T07:23:37.164 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:37.163+0000 7fb6f492a640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb6f0070ee0 con 0x7fb6f0143ac0 2023-11-26T07:23:37.165 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:37.163+0000 7fb6eed9d640 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7fb6f00cf990 0x7fb6f00cd1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:23:37.165 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:37.163+0000 7fb6eed9d640 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7fb6f00cf990 0x7fb6f00cd1b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.188:3300/0 says I am v2:172.21.15.143:48268/0 (socket says 172.21.15.143:48268) 2023-11-26T07:23:37.165 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:37.163+0000 7fb6eed9d640 1 -- 172.21.15.143:0/1398167087 learned_addr learned my addr 172.21.15.143:0/1398167087 (peer_addr_for_me v2:172.21.15.143:0/0) 2023-11-26T07:23:37.165 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:37.163+0000 7fb6ee59c640 1 --2- 172.21.15.143:0/1398167087 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fb6f0143ac0 0x7fb6f0070370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:23:37.165 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:37.163+0000 7fb6eed9d640 1 -- 172.21.15.143:0/1398167087 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fb6f0143ac0 msgr2=0x7fb6f0070370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-26T07:23:37.165 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:37.163+0000 7fb6eed9d640 1 --2- 172.21.15.143:0/1398167087 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fb6f0143ac0 0x7fb6f0070370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:23:37.165 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:37.163+0000 7fb6eed9d640 1 -- 172.21.15.143:0/1398167087 >> v1:172.21.15.188:6789/0 conn(0x7fb6f00708b0 legacy=0x7fb6f00cf280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:23:37.165 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:37.163+0000 7fb6eed9d640 1 -- 172.21.15.143:0/1398167087 --> v2:172.21.15.188:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb6f00711c0 con 0x7fb6f00cf990 2023-11-26T07:23:37.165 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:37.163+0000 7fb6ee59c640 1 --2- 172.21.15.143:0/1398167087 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fb6f0143ac0 0x7fb6f0070370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-26T07:23:37.167 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:37.167+0000 7fb6eed9d640 1 --2- 172.21.15.143:0/1398167087 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fb6f00cf990 0x7fb6f00cd1b0 secure :-1 s=READY pgs=448 cs=0 l=1 rev1=1 crypto rx=0x7fb6e8009e50 tx=0x7fb6e801ca10 comp rx=0 tx=0).ready entity=mon.1 client_cookie=4e73ce82f02811f0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:23:37.168 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:37.167+0000 7fb6edd9b640 1 -- 172.21.15.143:0/1398167087 <== mon.1 v2:172.21.15.188:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb6e80a57f0 con 0x7fb6f00cf990 2023-11-26T07:23:37.168 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:37.167+0000 7fb6edd9b640 1 -- 172.21.15.143:0/1398167087 <== mon.1 v2:172.21.15.188:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb6e80a5950 con 0x7fb6f00cf990 2023-11-26T07:23:37.168 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:37.167+0000 7fb6edd9b640 1 -- 172.21.15.143:0/1398167087 <== mon.1 v2:172.21.15.188:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb6e80a5ce0 con 0x7fb6f00cf990 2023-11-26T07:23:37.168 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:37.167+0000 7fb6f492a640 1 -- 172.21.15.143:0/1398167087 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fb6f00cf990 msgr2=0x7fb6f00cd1b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:23:37.168 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:37.167+0000 7fb6f492a640 1 --2- 172.21.15.143:0/1398167087 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fb6f00cf990 0x7fb6f00cd1b0 secure :-1 s=READY pgs=448 cs=0 l=1 rev1=1 crypto rx=0x7fb6e8009e50 tx=0x7fb6e801ca10 comp rx=0 tx=0).stop 2023-11-26T07:23:37.169 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:37.167+0000 7fb6f492a640 1 -- 172.21.15.143:0/1398167087 shutdown_connections 2023-11-26T07:23:37.169 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:37.167+0000 7fb6f492a640 1 --2- 172.21.15.143:0/1398167087 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fb6f00cf990 0x7fb6f00cd1b0 unknown :-1 s=CLOSED pgs=448 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:23:37.169 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:37.167+0000 7fb6f492a640 1 --2- 172.21.15.143:0/1398167087 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fb6f0143ac0 0x7fb6f0070370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:23:37.169 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:37.167+0000 7fb6f492a640 1 -- 172.21.15.143:0/1398167087 >> 172.21.15.143:0/1398167087 conn(0x7fb6f006a890 msgr2=0x7fb6f006aca0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:23:37.169 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:37.167+0000 7fb6f492a640 1 -- 172.21.15.143:0/1398167087 shutdown_connections 2023-11-26T07:23:37.169 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:37.167+0000 7fb6f492a640 1 -- 172.21.15.143:0/1398167087 wait complete. 2023-11-26T07:23:37.170 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:37.171+0000 7fb6f492a640 1 Processor -- start 2023-11-26T07:23:37.170 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:37.171+0000 7fb6f492a640 1 -- start start 2023-11-26T07:23:37.172 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:37.171+0000 7fb6f492a640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fb6f00708b0 0x7fb6f0137990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:23:37.172 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:37.171+0000 7fb6f492a640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fb6f00cf990 0x7fb6f00add60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:23:37.172 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:37.171+0000 7fb6f492a640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fb6f0143ac0 0x7fb6f00aeb10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:23:37.172 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:37.171+0000 7fb6f492a640 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb6f00aca70 con 0x7fb6f00cf990 2023-11-26T07:23:37.172 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:37.171+0000 7fb6f492a640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb6f015a010 con 0x7fb6f0143ac0 2023-11-26T07:23:37.172 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:37.171+0000 7fb6f492a640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb6f00accc0 con 0x7fb6f00708b0 2023-11-26T07:23:37.172 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:37.171+0000 7fb6e5d9b640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fb6f00cf990 0x7fb6f00add60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:23:37.172 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:37.171+0000 7fb6e5d9b640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fb6f00cf990 0x7fb6f00add60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:55740/0 (socket says 172.21.15.143:55740) 2023-11-26T07:23:37.172 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:37.171+0000 7fb6e5d9b640 1 -- 172.21.15.143:0/4129329165 learned_addr learned my addr 172.21.15.143:0/4129329165 (peer_addr_for_me v2:172.21.15.143:0/0) 2023-11-26T07:23:37.173 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:37.171+0000 7fb6ee59c640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fb6f00708b0 0x7fb6f0137990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:23:37.173 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:37.171+0000 7fb6eed9d640 1 --2- 172.21.15.143:0/4129329165 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fb6f0143ac0 0x7fb6f00aeb10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:23:37.174 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:37.171+0000 7fb6e5d9b640 1 -- 172.21.15.143:0/4129329165 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fb6f00708b0 msgr2=0x7fb6f0137990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:23:37.174 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:37.171+0000 7fb6e5d9b640 1 --2- 172.21.15.143:0/4129329165 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fb6f00708b0 0x7fb6f0137990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:23:37.174 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:37.171+0000 7fb6e5d9b640 1 -- 172.21.15.143:0/4129329165 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fb6f0143ac0 msgr2=0x7fb6f00aeb10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:23:37.174 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:37.171+0000 7fb6e5d9b640 1 --2- 172.21.15.143:0/4129329165 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fb6f0143ac0 0x7fb6f00aeb10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:23:37.174 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:37.171+0000 7fb6e5d9b640 1 -- 172.21.15.143:0/4129329165 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb6d8009420 con 0x7fb6f00cf990 2023-11-26T07:23:37.174 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:37.171+0000 7fb6e5d9b640 1 --2- 172.21.15.143:0/4129329165 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fb6f00cf990 0x7fb6f00add60 secure :-1 s=READY pgs=1175 cs=0 l=1 rev1=1 crypto rx=0x7fb6e000c450 tx=0x7fb6e000c910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:23:37.174 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:37.171+0000 7fb6e77fe640 1 -- 172.21.15.143:0/4129329165 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb6e00052d0 con 0x7fb6f00cf990 2023-11-26T07:23:37.174 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:37.171+0000 7fb6e77fe640 1 -- 172.21.15.143:0/4129329165 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb6e0005450 con 0x7fb6f00cf990 2023-11-26T07:23:37.174 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:37.171+0000 7fb6e77fe640 1 -- 172.21.15.143:0/4129329165 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb6e0012870 con 0x7fb6f00cf990 2023-11-26T07:23:37.174 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:37.171+0000 7fb6f492a640 1 -- 172.21.15.143:0/4129329165 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb6e8009b30 con 0x7fb6f00cf990 2023-11-26T07:23:37.175 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:37.171+0000 7fb6f492a640 1 -- 172.21.15.143:0/4129329165 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb6f00ad280 con 0x7fb6f00cf990 2023-11-26T07:23:37.175 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:37.171+0000 7fb6f492a640 1 -- 172.21.15.143:0/4129329165 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb6f0067680 con 0x7fb6f00cf990 2023-11-26T07:23:37.175 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:37.175+0000 7fb6e77fe640 1 -- 172.21.15.143:0/4129329165 <== mon.0 v2:172.21.15.143:3300/0 4 ==== mgrmap(e 16) v1 ==== 85954+0+0 (secure 0 0 0) 0x7fb6e00129d0 con 0x7fb6f00cf990 2023-11-26T07:23:37.176 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:37.175+0000 7fb6e77fe640 1 --2- 172.21.15.143:0/4129329165 >> [v2:172.21.15.188:6836/3401128553,v1:172.21.15.188:6837/3401128553] conn(0x7fb6d0066590 0x7fb6d0068a50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:23:37.176 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:37.175+0000 7fb6e77fe640 1 -- 172.21.15.143:0/4129329165 <== mon.0 v2:172.21.15.143:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fb6e0083a60 con 0x7fb6f00cf990 2023-11-26T07:23:37.176 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:37.175+0000 7fb6ee59c640 1 --2- 172.21.15.143:0/4129329165 >> [v2:172.21.15.188:6836/3401128553,v1:172.21.15.188:6837/3401128553] conn(0x7fb6d0066590 0x7fb6d0068a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:23:37.177 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:37.175+0000 7fb6ee59c640 1 --2- 172.21.15.143:0/4129329165 >> [v2:172.21.15.188:6836/3401128553,v1:172.21.15.188:6837/3401128553] conn(0x7fb6d0066590 0x7fb6d0068a50 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7fb6d8009c50 tx=0x7fb6d8009190 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:23:37.179 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:37.179+0000 7fb6e77fe640 1 -- 172.21.15.143:0/4129329165 <== mon.0 v2:172.21.15.143:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172649 (secure 0 0 0) 0x7fb6e0050de0 con 0x7fb6f00cf990 2023-11-26T07:23:37.361 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:37.359+0000 7fb6f492a640 1 -- 172.21.15.143:0/4129329165 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "fs ls"} v 0) v1 -- 0x7fb6f006b060 con 0x7fb6f00cf990 2023-11-26T07:23:37.362 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:37.359+0000 7fb6e77fe640 1 -- 172.21.15.143:0/4129329165 <== mon.0 v2:172.21.15.143:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v51) v1 ==== 53+0+73 (secure 0 0 0) 0x7fb6e0054400 con 0x7fb6f00cf990 2023-11-26T07:23:37.362 INFO:teuthology.orchestra.run.smithi143.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2023-11-26T07:23:37.364 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:37.363+0000 7fb6f492a640 1 -- 172.21.15.143:0/4129329165 >> [v2:172.21.15.188:6836/3401128553,v1:172.21.15.188:6837/3401128553] conn(0x7fb6d0066590 msgr2=0x7fb6d0068a50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:23:37.364 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:37.363+0000 7fb6f492a640 1 --2- 172.21.15.143:0/4129329165 >> [v2:172.21.15.188:6836/3401128553,v1:172.21.15.188:6837/3401128553] conn(0x7fb6d0066590 0x7fb6d0068a50 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7fb6d8009c50 tx=0x7fb6d8009190 comp rx=0 tx=0).stop 2023-11-26T07:23:37.364 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:37.363+0000 7fb6f492a640 1 -- 172.21.15.143:0/4129329165 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fb6f00cf990 msgr2=0x7fb6f00add60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:23:37.364 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:37.363+0000 7fb6f492a640 1 --2- 172.21.15.143:0/4129329165 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fb6f00cf990 0x7fb6f00add60 secure :-1 s=READY pgs=1175 cs=0 l=1 rev1=1 crypto rx=0x7fb6e000c450 tx=0x7fb6e000c910 comp rx=0 tx=0).stop 2023-11-26T07:23:37.364 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:37.363+0000 7fb6f492a640 1 -- 172.21.15.143:0/4129329165 shutdown_connections 2023-11-26T07:23:37.364 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:37.363+0000 7fb6f492a640 1 --2- 172.21.15.143:0/4129329165 >> [v2:172.21.15.188:6836/3401128553,v1:172.21.15.188:6837/3401128553] conn(0x7fb6d0066590 0x7fb6d0068a50 unknown :-1 s=CLOSED pgs=69 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:23:37.364 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:37.363+0000 7fb6f492a640 1 --2- 172.21.15.143:0/4129329165 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fb6f0143ac0 0x7fb6f00aeb10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:23:37.364 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:37.363+0000 7fb6f492a640 1 --2- 172.21.15.143:0/4129329165 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fb6f00cf990 0x7fb6f00add60 unknown :-1 s=CLOSED pgs=1175 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:23:37.364 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:37.363+0000 7fb6f492a640 1 --2- 172.21.15.143:0/4129329165 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fb6f00708b0 0x7fb6f0137990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:23:37.364 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:37.363+0000 7fb6f492a640 1 -- 172.21.15.143:0/4129329165 >> 172.21.15.143:0/4129329165 conn(0x7fb6f006a890 msgr2=0x7fb6f015b090 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:23:37.364 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:37.363+0000 7fb6f492a640 1 -- 172.21.15.143:0/4129329165 shutdown_connections 2023-11-26T07:23:37.364 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:37.363+0000 7fb6f492a640 1 -- 172.21.15.143:0/4129329165 wait complete. 2023-11-26T07:23:37.382 INFO:tasks.cephfs.mount:Mounting default Ceph FS; just confirmed its presence on cluster 2023-11-26T07:23:37.382 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2023-11-26T07:23:37.383 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi188.front.sepia.ceph.com 2023-11-26T07:23:37.383 INFO:tasks.cephfs.mount:self.client.name = client.1 2023-11-26T07:23:37.383 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.1 2023-11-26T07:23:37.383 INFO:tasks.cephfs.mount:self.cephfs_name = None 2023-11-26T07:23:37.383 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2023-11-26T07:23:37.383 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2023-11-26T07:23:37.383 INFO:tasks.cephfs.mount:Setting the 'ceph-ns--home-ubuntu-cephtest-mnt.1' netns for '/home/ubuntu/cephtest/mnt.1' 2023-11-26T07:23:37.383 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-26T07:23:37.384 DEBUG:teuthology.orchestra.run.smithi188:> ip addr 2023-11-26T07:23:37.394 INFO:teuthology.orchestra.run.smithi188.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2023-11-26T07:23:37.394 INFO:teuthology.orchestra.run.smithi188.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2023-11-26T07:23:37.394 INFO:teuthology.orchestra.run.smithi188.stdout: inet 127.0.0.1/8 scope host lo 2023-11-26T07:23:37.394 INFO:teuthology.orchestra.run.smithi188.stdout: valid_lft forever preferred_lft forever 2023-11-26T07:23:37.395 INFO:teuthology.orchestra.run.smithi188.stdout: inet6 ::1/128 scope host 2023-11-26T07:23:37.395 INFO:teuthology.orchestra.run.smithi188.stdout: valid_lft forever preferred_lft forever 2023-11-26T07:23:37.395 INFO:teuthology.orchestra.run.smithi188.stdout:2: eno1: mtu 1500 qdisc noop state DOWN group default qlen 1000 2023-11-26T07:23:37.395 INFO:teuthology.orchestra.run.smithi188.stdout: link/ether 0c:c4:7a:d9:37:48 brd ff:ff:ff:ff:ff:ff 2023-11-26T07:23:37.395 INFO:teuthology.orchestra.run.smithi188.stdout: altname enp7s0f0 2023-11-26T07:23:37.395 INFO:teuthology.orchestra.run.smithi188.stdout:3: eno2: mtu 1500 qdisc noop state DOWN group default qlen 1000 2023-11-26T07:23:37.395 INFO:teuthology.orchestra.run.smithi188.stdout: link/ether 0c:c4:7a:d9:37:49 brd ff:ff:ff:ff:ff:ff 2023-11-26T07:23:37.395 INFO:teuthology.orchestra.run.smithi188.stdout: altname enp7s0f1 2023-11-26T07:23:37.395 INFO:teuthology.orchestra.run.smithi188.stdout:4: enp3s0f0: mtu 1500 qdisc noop state DOWN group default qlen 1000 2023-11-26T07:23:37.395 INFO:teuthology.orchestra.run.smithi188.stdout: link/ether 0c:c4:7a:88:80:38 brd ff:ff:ff:ff:ff:ff 2023-11-26T07:23:37.395 INFO:teuthology.orchestra.run.smithi188.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2023-11-26T07:23:37.395 INFO:teuthology.orchestra.run.smithi188.stdout: link/ether 0c:c4:7a:88:80:39 brd ff:ff:ff:ff:ff:ff 2023-11-26T07:23:37.395 INFO:teuthology.orchestra.run.smithi188.stdout: inet 172.21.15.188/20 metric 100 brd 172.21.15.255 scope global dynamic enp3s0f1 2023-11-26T07:23:37.396 INFO:teuthology.orchestra.run.smithi188.stdout: valid_lft 42051sec preferred_lft 42051sec 2023-11-26T07:23:37.396 INFO:teuthology.orchestra.run.smithi188.stdout: inet6 fe80::ec4:7aff:fe88:8039/64 scope link 2023-11-26T07:23:37.396 INFO:teuthology.orchestra.run.smithi188.stdout: valid_lft forever preferred_lft forever 2023-11-26T07:23:37.396 INFO:teuthology.orchestra.run.smithi188.stdout:6: docker0: mtu 1500 qdisc noqueue state DOWN group default 2023-11-26T07:23:37.396 INFO:teuthology.orchestra.run.smithi188.stdout: link/ether 02:42:d8:0d:a0:fa brd ff:ff:ff:ff:ff:ff 2023-11-26T07:23:37.396 INFO:teuthology.orchestra.run.smithi188.stdout: inet 172.17.0.1/16 brd 172.17.255.255 scope global docker0 2023-11-26T07:23:37.396 INFO:teuthology.orchestra.run.smithi188.stdout: valid_lft forever preferred_lft forever 2023-11-26T07:23:37.396 INFO:teuthology.orchestra.run.smithi188.stdout:7: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2023-11-26T07:23:37.396 INFO:teuthology.orchestra.run.smithi188.stdout: link/ether 5e:1f:84:6d:79:ae brd ff:ff:ff:ff:ff:ff 2023-11-26T07:23:37.396 INFO:teuthology.orchestra.run.smithi188.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2023-11-26T07:23:37.396 INFO:teuthology.orchestra.run.smithi188.stdout: valid_lft forever preferred_lft forever 2023-11-26T07:23:37.396 INFO:teuthology.orchestra.run.smithi188.stdout: inet6 fe80::5c1f:84ff:fe6d:79ae/64 scope link 2023-11-26T07:23:37.396 INFO:teuthology.orchestra.run.smithi188.stdout: valid_lft forever preferred_lft forever 2023-11-26T07:23:37.397 INFO:teuthology.orchestra.run.smithi188.stdout:8: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2023-11-26T07:23:37.397 INFO:teuthology.orchestra.run.smithi188.stdout: link/ether 0e:5c:f6:1f:82:2f brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.1 2023-11-26T07:23:37.397 INFO:teuthology.orchestra.run.smithi188.stdout: inet6 fe80::c5c:f6ff:fe1f:822f/64 scope link 2023-11-26T07:23:37.397 INFO:teuthology.orchestra.run.smithi188.stdout: valid_lft forever preferred_lft forever 2023-11-26T07:23:37.397 INFO:teuthology.orchestra.run.smithi188.stdout:9: brx.1@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2023-11-26T07:23:37.397 INFO:teuthology.orchestra.run.smithi188.stdout: link/ether c6:a6:3d:69:d0:db brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.3 2023-11-26T07:23:37.397 INFO:teuthology.orchestra.run.smithi188.stdout: inet6 fe80::c4a6:3dff:fe69:d0db/64 scope link 2023-11-26T07:23:37.397 INFO:teuthology.orchestra.run.smithi188.stdout: valid_lft forever preferred_lft forever 2023-11-26T07:23:37.398 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-26T07:23:37.398 DEBUG:teuthology.orchestra.run.smithi188:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2023-11-26T07:23:37.460 INFO:teuthology.orchestra.run.smithi188.stdout:1 2023-11-26T07:23:37.462 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-26T07:23:37.462 DEBUG:teuthology.orchestra.run.smithi188:> route 2023-11-26T07:23:37.515 INFO:teuthology.orchestra.run.smithi188.stdout:Kernel IP routing table 2023-11-26T07:23:37.515 INFO:teuthology.orchestra.run.smithi188.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2023-11-26T07:23:37.515 INFO:teuthology.orchestra.run.smithi188.stdout:default _gateway 0.0.0.0 UG 0 0 0 enp3s0f1 2023-11-26T07:23:37.515 INFO:teuthology.orchestra.run.smithi188.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2023-11-26T07:23:37.515 INFO:teuthology.orchestra.run.smithi188.stdout:172.17.0.0 0.0.0.0 255.255.0.0 U 0 0 0 docker0 2023-11-26T07:23:37.515 INFO:teuthology.orchestra.run.smithi188.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 enp3s0f1 2023-11-26T07:23:37.515 INFO:teuthology.orchestra.run.smithi188.stdout:ns1.front.sepia 0.0.0.0 255.255.255.255 UH 100 0 0 enp3s0f1 2023-11-26T07:23:37.515 INFO:teuthology.orchestra.run.smithi188.stdout:172.21.0.2 0.0.0.0 255.255.255.255 UH 100 0 0 enp3s0f1 2023-11-26T07:23:37.516 INFO:teuthology.orchestra.run.smithi188.stdout:_gateway 0.0.0.0 255.255.255.255 UH 100 0 0 enp3s0f1 2023-11-26T07:23:37.516 INFO:teuthology.orchestra.run.smithi188.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2023-11-26T07:23:37.516 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-26T07:23:37.516 DEBUG:teuthology.orchestra.run.smithi188:> (cd / && exec bash -c ' 2023-11-26T07:23:37.516 DEBUG:teuthology.orchestra.run.smithi188:> set -e 2023-11-26T07:23:37.516 DEBUG:teuthology.orchestra.run.smithi188:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2023-11-26T07:23:37.516 DEBUG:teuthology.orchestra.run.smithi188:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2023-11-26T07:23:37.516 DEBUG:teuthology.orchestra.run.smithi188:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2023-11-26T07:23:37.516 DEBUG:teuthology.orchestra.run.smithi188:> ') 2023-11-26T07:23:37.611 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-26T07:23:37.611 DEBUG:teuthology.orchestra.run.smithi188:> ip netns list 2023-11-26T07:23:37.620 INFO:teuthology.orchestra.run.smithi188.stdout:ceph-ns--home-ubuntu-cephtest-mnt.1 (id: 0) 2023-11-26T07:23:37.620 INFO:teuthology.orchestra.run.smithi188.stdout:ceph-ns--home-ubuntu-cephtest-mnt.3 (id: 1) 2023-11-26T07:23:37.621 INFO:tasks.cephfs.mount:Resuming the 'ceph-ns--home-ubuntu-cephtest-mnt.1' netns for '/home/ubuntu/cephtest/mnt.1' 2023-11-26T07:23:37.621 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-26T07:23:37.621 DEBUG:teuthology.orchestra.run.smithi188:> sudo ip link set brx.0 up 2023-11-26T07:23:37.690 INFO:tasks.cephfs.fuse_mount:Client client.1 config is {} 2023-11-26T07:23:37.690 INFO:teuthology.orchestra.run:Running command with timeout 60 2023-11-26T07:23:37.690 DEBUG:teuthology.orchestra.run.smithi188:> mkdir -p -v /home/ubuntu/cephtest/mnt.1 2023-11-26T07:23:37.744 INFO:teuthology.orchestra.run.smithi188.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.1' 2023-11-26T07:23:37.744 INFO:teuthology.orchestra.run:Running command with timeout 60 2023-11-26T07:23:37.744 DEBUG:teuthology.orchestra.run.smithi188:> chmod 0000 /home/ubuntu/cephtest/mnt.1 2023-11-26T07:23:37.795 DEBUG:teuthology.orchestra.run.smithi188:> sudo modprobe fuse 2023-11-26T07:23:37.857 DEBUG:teuthology.orchestra.run.smithi188:> cat /proc/self/mounts | awk '{print $2}' 2023-11-26T07:23:37.909 INFO:teuthology.orchestra.run.smithi188.stdout:/sys 2023-11-26T07:23:37.909 INFO:teuthology.orchestra.run.smithi188.stdout:/proc 2023-11-26T07:23:37.910 INFO:teuthology.orchestra.run.smithi188.stdout:/dev 2023-11-26T07:23:37.910 INFO:teuthology.orchestra.run.smithi188.stdout:/dev/pts 2023-11-26T07:23:37.910 INFO:teuthology.orchestra.run.smithi188.stdout:/run 2023-11-26T07:23:37.910 INFO:teuthology.orchestra.run.smithi188.stdout:/ 2023-11-26T07:23:37.910 INFO:teuthology.orchestra.run.smithi188.stdout:/sys/kernel/security 2023-11-26T07:23:37.910 INFO:teuthology.orchestra.run.smithi188.stdout:/dev/shm 2023-11-26T07:23:37.910 INFO:teuthology.orchestra.run.smithi188.stdout:/run/lock 2023-11-26T07:23:37.910 INFO:teuthology.orchestra.run.smithi188.stdout:/sys/fs/cgroup 2023-11-26T07:23:37.910 INFO:teuthology.orchestra.run.smithi188.stdout:/sys/fs/pstore 2023-11-26T07:23:37.910 INFO:teuthology.orchestra.run.smithi188.stdout:/sys/fs/bpf 2023-11-26T07:23:37.911 INFO:teuthology.orchestra.run.smithi188.stdout:/proc/sys/fs/binfmt_misc 2023-11-26T07:23:37.911 INFO:teuthology.orchestra.run.smithi188.stdout:/dev/hugepages 2023-11-26T07:23:37.911 INFO:teuthology.orchestra.run.smithi188.stdout:/dev/mqueue 2023-11-26T07:23:37.911 INFO:teuthology.orchestra.run.smithi188.stdout:/sys/kernel/debug 2023-11-26T07:23:37.911 INFO:teuthology.orchestra.run.smithi188.stdout:/sys/kernel/tracing 2023-11-26T07:23:37.911 INFO:teuthology.orchestra.run.smithi188.stdout:/proc/fs/nfsd 2023-11-26T07:23:37.911 INFO:teuthology.orchestra.run.smithi188.stdout:/sys/fs/fuse/connections 2023-11-26T07:23:37.911 INFO:teuthology.orchestra.run.smithi188.stdout:/sys/kernel/config 2023-11-26T07:23:37.911 INFO:teuthology.orchestra.run.smithi188.stdout:/run/credentials/systemd-sysusers.service 2023-11-26T07:23:37.911 INFO:teuthology.orchestra.run.smithi188.stdout:/run/rpc_pipefs 2023-11-26T07:23:37.911 INFO:teuthology.orchestra.run.smithi188.stdout:/run/user/1000 2023-11-26T07:23:37.911 INFO:teuthology.orchestra.run.smithi188.stdout:/var/lib/ceph 2023-11-26T07:23:37.912 INFO:teuthology.orchestra.run.smithi188.stdout:/var/lib/ceph/osd/ceph-4 2023-11-26T07:23:37.912 INFO:teuthology.orchestra.run.smithi188.stdout:/var/lib/ceph/osd/ceph-5 2023-11-26T07:23:37.912 INFO:teuthology.orchestra.run.smithi188.stdout:/var/lib/ceph/osd/ceph-6 2023-11-26T07:23:37.912 INFO:teuthology.orchestra.run.smithi188.stdout:/var/lib/ceph/osd/ceph-7 2023-11-26T07:23:37.912 INFO:teuthology.orchestra.run.smithi188.stdout:/run/netns 2023-11-26T07:23:37.912 INFO:teuthology.orchestra.run.smithi188.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 2023-11-26T07:23:37.912 INFO:teuthology.orchestra.run.smithi188.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 2023-11-26T07:23:37.912 INFO:teuthology.orchestra.run.smithi188.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 2023-11-26T07:23:37.912 INFO:teuthology.orchestra.run.smithi188.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 2023-11-26T07:23:37.913 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-26T07:23:37.913 DEBUG:teuthology.orchestra.run.smithi188:> ls /sys/fs/fuse/connections 2023-11-26T07:23:37.963 INFO:tasks.cephfs.fuse_mount:Pre-mount connections: [] 2023-11-26T07:23:37.964 DEBUG:teuthology.orchestra.run.smithi188:> (cd /home/ubuntu/cephtest && exec sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-fuse -f --admin-socket '/var/run/ceph/$cluster-$name.$pid.asok' /home/ubuntu/cephtest/mnt.1 --id 1) 2023-11-26T07:23:38.010 DEBUG:teuthology.orchestra.run.smithi188:> sudo modprobe fuse 2023-11-26T07:23:38.031 DEBUG:teuthology.orchestra.run.smithi188:> cat /proc/self/mounts | awk '{print $2}' 2023-11-26T07:23:38.085 INFO:teuthology.orchestra.run.smithi188.stdout:/sys 2023-11-26T07:23:38.085 INFO:teuthology.orchestra.run.smithi188.stdout:/proc 2023-11-26T07:23:38.086 INFO:teuthology.orchestra.run.smithi188.stdout:/dev 2023-11-26T07:23:38.086 INFO:teuthology.orchestra.run.smithi188.stdout:/dev/pts 2023-11-26T07:23:38.086 INFO:teuthology.orchestra.run.smithi188.stdout:/run 2023-11-26T07:23:38.086 INFO:teuthology.orchestra.run.smithi188.stdout:/ 2023-11-26T07:23:38.086 INFO:teuthology.orchestra.run.smithi188.stdout:/sys/kernel/security 2023-11-26T07:23:38.086 INFO:teuthology.orchestra.run.smithi188.stdout:/dev/shm 2023-11-26T07:23:38.086 INFO:teuthology.orchestra.run.smithi188.stdout:/run/lock 2023-11-26T07:23:38.086 INFO:teuthology.orchestra.run.smithi188.stdout:/sys/fs/cgroup 2023-11-26T07:23:38.086 INFO:teuthology.orchestra.run.smithi188.stdout:/sys/fs/pstore 2023-11-26T07:23:38.086 INFO:teuthology.orchestra.run.smithi188.stdout:/sys/fs/bpf 2023-11-26T07:23:38.086 INFO:teuthology.orchestra.run.smithi188.stdout:/proc/sys/fs/binfmt_misc 2023-11-26T07:23:38.087 INFO:teuthology.orchestra.run.smithi188.stdout:/dev/hugepages 2023-11-26T07:23:38.087 INFO:teuthology.orchestra.run.smithi188.stdout:/dev/mqueue 2023-11-26T07:23:38.087 INFO:teuthology.orchestra.run.smithi188.stdout:/sys/kernel/debug 2023-11-26T07:23:38.087 INFO:teuthology.orchestra.run.smithi188.stdout:/sys/kernel/tracing 2023-11-26T07:23:38.087 INFO:teuthology.orchestra.run.smithi188.stdout:/proc/fs/nfsd 2023-11-26T07:23:38.087 INFO:teuthology.orchestra.run.smithi188.stdout:/sys/fs/fuse/connections 2023-11-26T07:23:38.087 INFO:teuthology.orchestra.run.smithi188.stdout:/sys/kernel/config 2023-11-26T07:23:38.087 INFO:teuthology.orchestra.run.smithi188.stdout:/run/credentials/systemd-sysusers.service 2023-11-26T07:23:38.087 INFO:teuthology.orchestra.run.smithi188.stdout:/run/rpc_pipefs 2023-11-26T07:23:38.087 INFO:teuthology.orchestra.run.smithi188.stdout:/run/user/1000 2023-11-26T07:23:38.087 INFO:teuthology.orchestra.run.smithi188.stdout:/var/lib/ceph 2023-11-26T07:23:38.087 INFO:teuthology.orchestra.run.smithi188.stdout:/var/lib/ceph/osd/ceph-4 2023-11-26T07:23:38.088 INFO:teuthology.orchestra.run.smithi188.stdout:/var/lib/ceph/osd/ceph-5 2023-11-26T07:23:38.088 INFO:teuthology.orchestra.run.smithi188.stdout:/var/lib/ceph/osd/ceph-6 2023-11-26T07:23:38.088 INFO:teuthology.orchestra.run.smithi188.stdout:/var/lib/ceph/osd/ceph-7 2023-11-26T07:23:38.088 INFO:teuthology.orchestra.run.smithi188.stdout:/run/netns 2023-11-26T07:23:38.088 INFO:teuthology.orchestra.run.smithi188.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 2023-11-26T07:23:38.088 INFO:teuthology.orchestra.run.smithi188.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 2023-11-26T07:23:38.088 INFO:teuthology.orchestra.run.smithi188.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 2023-11-26T07:23:38.088 INFO:teuthology.orchestra.run.smithi188.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 2023-11-26T07:23:38.089 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-26T07:23:38.089 DEBUG:teuthology.orchestra.run.smithi188:> ls /sys/fs/fuse/connections 2023-11-26T07:23:38.107 INFO:tasks.cephfs.fuse_mount.ceph-fuse.1.smithi188.stderr:2023-11-26T07:23:38.101+0000 7fece8c48580 -1 init, newargv = 0x562519710270 newargc=15 2023-11-26T07:23:38.107 INFO:tasks.cephfs.fuse_mount.ceph-fuse.1.smithi188.stderr:ceph-fuse[45157]: starting ceph client 2023-11-26T07:23:38.114 INFO:tasks.cephfs.fuse_mount.ceph-fuse.1.smithi188.stderr:ceph-fuse[45157]: starting fuse 2023-11-26T07:23:38.116 INFO:teuthology.orchestra.run.smithi188.stdout:43 2023-11-26T07:23:38.116 INFO:tasks.cephfs.fuse_mount:Post-mount connections: [43] 2023-11-26T07:23:38.116 DEBUG:teuthology.orchestra.run.smithi188:> sudo stdin-killer -- python3 -c ' 2023-11-26T07:23:38.117 DEBUG:teuthology.orchestra.run.smithi188:> import glob 2023-11-26T07:23:38.117 DEBUG:teuthology.orchestra.run.smithi188:> import re 2023-11-26T07:23:38.117 DEBUG:teuthology.orchestra.run.smithi188:> import os 2023-11-26T07:23:38.117 DEBUG:teuthology.orchestra.run.smithi188:> import subprocess 2023-11-26T07:23:38.117 DEBUG:teuthology.orchestra.run.smithi188:> 2023-11-26T07:23:38.117 DEBUG:teuthology.orchestra.run.smithi188:> def _find_admin_socket(client_name): 2023-11-26T07:23:38.117 DEBUG:teuthology.orchestra.run.smithi188:> asok_path = "/var/run/ceph/ceph-client.1.*.asok" 2023-11-26T07:23:38.117 DEBUG:teuthology.orchestra.run.smithi188:> files = glob.glob(asok_path) 2023-11-26T07:23:38.117 DEBUG:teuthology.orchestra.run.smithi188:> mountpoint = "/home/ubuntu/cephtest/mnt.1" 2023-11-26T07:23:38.117 DEBUG:teuthology.orchestra.run.smithi188:> 2023-11-26T07:23:38.118 DEBUG:teuthology.orchestra.run.smithi188:> # Given a non-glob path, it better be there 2023-11-26T07:23:38.118 DEBUG:teuthology.orchestra.run.smithi188:> if "*" not in asok_path: 2023-11-26T07:23:38.118 DEBUG:teuthology.orchestra.run.smithi188:> assert(len(files) == 1) 2023-11-26T07:23:38.118 DEBUG:teuthology.orchestra.run.smithi188:> return files[0] 2023-11-26T07:23:38.118 DEBUG:teuthology.orchestra.run.smithi188:> 2023-11-26T07:23:38.118 DEBUG:teuthology.orchestra.run.smithi188:> for f in files: 2023-11-26T07:23:38.118 DEBUG:teuthology.orchestra.run.smithi188:> pid = re.match(".*\.(\d+)\.asok$", f).group(1) 2023-11-26T07:23:38.118 DEBUG:teuthology.orchestra.run.smithi188:> if os.path.exists("/proc/{0}".format(pid)): 2023-11-26T07:23:38.118 DEBUG:teuthology.orchestra.run.smithi188:> with open("/proc/{0}/cmdline".format(pid), '"'"'r'"'"') as proc_f: 2023-11-26T07:23:38.118 DEBUG:teuthology.orchestra.run.smithi188:> contents = proc_f.read() 2023-11-26T07:23:38.119 DEBUG:teuthology.orchestra.run.smithi188:> if mountpoint in contents: 2023-11-26T07:23:38.119 DEBUG:teuthology.orchestra.run.smithi188:> return f 2023-11-26T07:23:38.119 DEBUG:teuthology.orchestra.run.smithi188:> raise RuntimeError("Client socket {0} not found".format(client_name)) 2023-11-26T07:23:38.119 DEBUG:teuthology.orchestra.run.smithi188:> 2023-11-26T07:23:38.119 DEBUG:teuthology.orchestra.run.smithi188:> print(_find_admin_socket("client.1")) 2023-11-26T07:23:38.119 DEBUG:teuthology.orchestra.run.smithi188:> ' 2023-11-26T07:23:38.240 INFO:teuthology.orchestra.run.smithi188.stdout:/var/run/ceph/ceph-client.1.45157.asok 2023-11-26T07:23:38.243 INFO:teuthology.orchestra.run.smithi188.stderr:2023-11-26T07:23:38 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2023-11-26T07:23:38.251 INFO:tasks.cephfs.fuse_mount:Found client admin socket at /var/run/ceph/ceph-client.1.45157.asok 2023-11-26T07:23:38.252 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-26T07:23:38.252 DEBUG:teuthology.orchestra.run.smithi188:> sudo ceph --admin-daemon /var/run/ceph/ceph-client.1.45157.asok status 2023-11-26T07:23:38.427 INFO:teuthology.orchestra.run.smithi188.stdout:{ 2023-11-26T07:23:38.427 INFO:teuthology.orchestra.run.smithi188.stdout: "metadata": { 2023-11-26T07:23:38.427 INFO:teuthology.orchestra.run.smithi188.stdout: "ceph_sha1": "44c2237fe77713214148c1fd622c872f346f1b99", 2023-11-26T07:23:38.427 INFO:teuthology.orchestra.run.smithi188.stdout: "ceph_version": "ceph version 18.0.0-7484-g44c2237f (44c2237fe77713214148c1fd622c872f346f1b99) reef (dev)", 2023-11-26T07:23:38.427 INFO:teuthology.orchestra.run.smithi188.stdout: "entity_id": "1", 2023-11-26T07:23:38.427 INFO:teuthology.orchestra.run.smithi188.stdout: "hostname": "smithi188", 2023-11-26T07:23:38.427 INFO:teuthology.orchestra.run.smithi188.stdout: "mount_point": "/home/ubuntu/cephtest/mnt.1", 2023-11-26T07:23:38.428 INFO:teuthology.orchestra.run.smithi188.stdout: "pid": "45157", 2023-11-26T07:23:38.428 INFO:teuthology.orchestra.run.smithi188.stdout: "root": "/" 2023-11-26T07:23:38.428 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2023-11-26T07:23:38.428 INFO:teuthology.orchestra.run.smithi188.stdout: "dentry_count": 0, 2023-11-26T07:23:38.428 INFO:teuthology.orchestra.run.smithi188.stdout: "dentry_pinned_count": 0, 2023-11-26T07:23:38.428 INFO:teuthology.orchestra.run.smithi188.stdout: "id": 7213, 2023-11-26T07:23:38.428 INFO:teuthology.orchestra.run.smithi188.stdout: "inst": { 2023-11-26T07:23:38.428 INFO:teuthology.orchestra.run.smithi188.stdout: "name": { 2023-11-26T07:23:38.428 INFO:teuthology.orchestra.run.smithi188.stdout: "type": "client", 2023-11-26T07:23:38.429 INFO:teuthology.orchestra.run.smithi188.stdout: "num": 7213 2023-11-26T07:23:38.429 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2023-11-26T07:23:38.429 INFO:teuthology.orchestra.run.smithi188.stdout: "addr": { 2023-11-26T07:23:38.429 INFO:teuthology.orchestra.run.smithi188.stdout: "type": "v1", 2023-11-26T07:23:38.429 INFO:teuthology.orchestra.run.smithi188.stdout: "addr": "192.168.0.1:0", 2023-11-26T07:23:38.429 INFO:teuthology.orchestra.run.smithi188.stdout: "nonce": 2874715281 2023-11-26T07:23:38.429 INFO:teuthology.orchestra.run.smithi188.stdout: } 2023-11-26T07:23:38.429 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2023-11-26T07:23:38.429 INFO:teuthology.orchestra.run.smithi188.stdout: "addr": { 2023-11-26T07:23:38.429 INFO:teuthology.orchestra.run.smithi188.stdout: "type": "v1", 2023-11-26T07:23:38.429 INFO:teuthology.orchestra.run.smithi188.stdout: "addr": "192.168.0.1:0", 2023-11-26T07:23:38.430 INFO:teuthology.orchestra.run.smithi188.stdout: "nonce": 2874715281 2023-11-26T07:23:38.430 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2023-11-26T07:23:38.430 INFO:teuthology.orchestra.run.smithi188.stdout: "inst_str": "client.7213 192.168.0.1:0/2874715281", 2023-11-26T07:23:38.430 INFO:teuthology.orchestra.run.smithi188.stdout: "addr_str": "192.168.0.1:0/2874715281", 2023-11-26T07:23:38.430 INFO:teuthology.orchestra.run.smithi188.stdout: "inode_count": 1, 2023-11-26T07:23:38.430 INFO:teuthology.orchestra.run.smithi188.stdout: "mds_epoch": 51, 2023-11-26T07:23:38.430 INFO:teuthology.orchestra.run.smithi188.stdout: "osd_epoch": 84, 2023-11-26T07:23:38.430 INFO:teuthology.orchestra.run.smithi188.stdout: "osd_epoch_barrier": 0, 2023-11-26T07:23:38.431 INFO:teuthology.orchestra.run.smithi188.stdout: "blocklisted": false, 2023-11-26T07:23:38.431 INFO:teuthology.orchestra.run.smithi188.stdout: "fs_name": "cephfs" 2023-11-26T07:23:38.431 INFO:teuthology.orchestra.run.smithi188.stdout:} 2023-11-26T07:23:38.442 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-26T07:23:38.442 DEBUG:teuthology.orchestra.run.smithi188:> stat --file-system '--printf=%T 2023-11-26T07:23:38.442 DEBUG:teuthology.orchestra.run.smithi188:> ' -- /home/ubuntu/cephtest/mnt.1 2023-11-26T07:23:38.451 INFO:teuthology.orchestra.run.smithi188.stdout:fuseblk 2023-11-26T07:23:38.452 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.1 2023-11-26T07:23:38.452 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-26T07:23:38.452 DEBUG:teuthology.orchestra.run.smithi188:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.1 2023-11-26T07:23:38.516 DEBUG:teuthology.orchestra.run.smithi143:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs flag set enable_multiple true --yes-i-really-mean-it 2023-11-26T07:23:38.667 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:38.667+0000 7ff591a91640 1 Processor -- start 2023-11-26T07:23:38.668 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:38.667+0000 7ff591a91640 1 -- start start 2023-11-26T07:23:38.669 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:38.667+0000 7ff591a91640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7ff58c1596d0 0x7ff58c071850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:23:38.670 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:38.667+0000 7ff591a91640 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7ff58c0ce680 0x7ff58c0b2870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:23:38.670 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:38.667+0000 7ff591a91640 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7ff58c0b2db0 0x7ff58c15d2c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:23:38.670 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:38.667+0000 7ff591a91640 1 -- --> v2:172.21.15.143:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff58c0b3240 con 0x7ff58c0b2db0 2023-11-26T07:23:38.670 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:38.667+0000 7ff591a91640 1 -- --> v2:172.21.15.188:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff58c0cbe00 con 0x7ff58c0ce680 2023-11-26T07:23:38.670 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:38.667+0000 7ff591a91640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff58c06c750 con 0x7ff58c1596d0 2023-11-26T07:23:38.670 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:38.667+0000 7ff58bfff640 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7ff58c0b2db0 0x7ff58c15d2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:23:38.670 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:38.667+0000 7ff58bfff640 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7ff58c0b2db0 0x7ff58c15d2c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:55752/0 (socket says 172.21.15.143:55752) 2023-11-26T07:23:38.670 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:38.667+0000 7ff58bfff640 1 -- 172.21.15.143:0/1552150908 learned_addr learned my addr 172.21.15.143:0/1552150908 (peer_addr_for_me v2:172.21.15.143:0/0) 2023-11-26T07:23:38.670 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:38.667+0000 7ff58b7fe640 1 --2- 172.21.15.143:0/1552150908 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7ff58c1596d0 0x7ff58c071850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:23:38.670 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:38.667+0000 7ff58affd640 1 --2- 172.21.15.143:0/1552150908 >> v2:172.21.15.188:3300/0 conn(0x7ff58c0ce680 0x7ff58c0b2870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:23:38.671 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:38.667+0000 7ff58bfff640 1 -- 172.21.15.143:0/1552150908 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7ff58c1596d0 msgr2=0x7ff58c071850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-26T07:23:38.671 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:38.667+0000 7ff58bfff640 1 --2- 172.21.15.143:0/1552150908 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7ff58c1596d0 0x7ff58c071850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:23:38.671 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:38.667+0000 7ff58bfff640 1 -- 172.21.15.143:0/1552150908 >> v2:172.21.15.188:3300/0 conn(0x7ff58c0ce680 msgr2=0x7ff58c0b2870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-26T07:23:38.671 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:38.667+0000 7ff58bfff640 1 --2- 172.21.15.143:0/1552150908 >> v2:172.21.15.188:3300/0 conn(0x7ff58c0ce680 0x7ff58c0b2870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:23:38.671 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:38.667+0000 7ff58bfff640 1 -- 172.21.15.143:0/1552150908 --> v2:172.21.15.143:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff58c0b3550 con 0x7ff58c0b2db0 2023-11-26T07:23:38.671 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:38.667+0000 7ff58affd640 1 --2- 172.21.15.143:0/1552150908 >> v2:172.21.15.188:3300/0 conn(0x7ff58c0ce680 0x7ff58c0b2870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-26T07:23:38.671 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:38.667+0000 7ff58b7fe640 1 --2- 172.21.15.143:0/1552150908 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7ff58c1596d0 0x7ff58c071850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-26T07:23:38.672 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:38.671+0000 7ff58bfff640 1 --2- 172.21.15.143:0/1552150908 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7ff58c0b2db0 0x7ff58c15d2c0 secure :-1 s=READY pgs=1176 cs=0 l=1 rev1=1 crypto rx=0x7ff58c06e170 tx=0x7ff58401c9d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=84e289f9d618e278 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:23:38.672 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:38.671+0000 7ff58a7fc640 1 -- 172.21.15.143:0/1552150908 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff5840a56c0 con 0x7ff58c0b2db0 2023-11-26T07:23:38.673 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:38.671+0000 7ff58a7fc640 1 -- 172.21.15.143:0/1552150908 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff5840a5820 con 0x7ff58c0b2db0 2023-11-26T07:23:38.673 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:38.671+0000 7ff58a7fc640 1 -- 172.21.15.143:0/1552150908 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff5840a5bb0 con 0x7ff58c0b2db0 2023-11-26T07:23:38.673 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:38.671+0000 7ff591a91640 1 -- 172.21.15.143:0/1552150908 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7ff58c0b2db0 msgr2=0x7ff58c15d2c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:23:38.673 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:38.671+0000 7ff591a91640 1 --2- 172.21.15.143:0/1552150908 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7ff58c0b2db0 0x7ff58c15d2c0 secure :-1 s=READY pgs=1176 cs=0 l=1 rev1=1 crypto rx=0x7ff58c06e170 tx=0x7ff58401c9d0 comp rx=0 tx=0).stop 2023-11-26T07:23:38.674 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:38.671+0000 7ff591a91640 1 -- 172.21.15.143:0/1552150908 shutdown_connections 2023-11-26T07:23:38.674 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:38.671+0000 7ff591a91640 1 --2- 172.21.15.143:0/1552150908 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7ff58c0b2db0 0x7ff58c15d2c0 unknown :-1 s=CLOSED pgs=1176 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:23:38.674 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:38.671+0000 7ff591a91640 1 --2- 172.21.15.143:0/1552150908 >> v2:172.21.15.188:3300/0 conn(0x7ff58c0ce680 0x7ff58c0b2870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:23:38.674 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:38.671+0000 7ff591a91640 1 --2- 172.21.15.143:0/1552150908 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7ff58c1596d0 0x7ff58c071850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:23:38.674 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:38.671+0000 7ff591a91640 1 -- 172.21.15.143:0/1552150908 >> 172.21.15.143:0/1552150908 conn(0x7ff58c06a890 msgr2=0x7ff58c06aca0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:23:38.674 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:38.671+0000 7ff591a91640 1 -- 172.21.15.143:0/1552150908 shutdown_connections 2023-11-26T07:23:38.674 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:38.671+0000 7ff591a91640 1 -- 172.21.15.143:0/1552150908 wait complete. 2023-11-26T07:23:38.675 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:38.675+0000 7ff591a91640 1 Processor -- start 2023-11-26T07:23:38.675 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:38.675+0000 7ff591a91640 1 -- start start 2023-11-26T07:23:38.677 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:38.675+0000 7ff591a91640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7ff58c0b2db0 0x7ff58c14f790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:23:38.677 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:38.675+0000 7ff591a91640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7ff58c0ce680 0x7ff58c150e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:23:38.677 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:38.675+0000 7ff591a91640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7ff58c1596d0 0x7ff58c0cd610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:23:38.677 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:38.675+0000 7ff591a91640 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff58c151350 con 0x7ff58c0ce680 2023-11-26T07:23:38.677 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:38.675+0000 7ff591a91640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff58c14fd70 con 0x7ff58c0b2db0 2023-11-26T07:23:38.677 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:38.675+0000 7ff591a91640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff58c150050 con 0x7ff58c1596d0 2023-11-26T07:23:38.677 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:38.675+0000 7ff58affd640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7ff58c0ce680 0x7ff58c150e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:23:38.677 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:38.675+0000 7ff58affd640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7ff58c0ce680 0x7ff58c150e10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:55760/0 (socket says 172.21.15.143:55760) 2023-11-26T07:23:38.678 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:38.675+0000 7ff58affd640 1 -- 172.21.15.143:0/550476057 learned_addr learned my addr 172.21.15.143:0/550476057 (peer_addr_for_me v2:172.21.15.143:0/0) 2023-11-26T07:23:38.678 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:38.675+0000 7ff58bfff640 1 --2- 172.21.15.143:0/550476057 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7ff58c1596d0 0x7ff58c0cd610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:23:38.678 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:38.675+0000 7ff58b7fe640 1 --2- 172.21.15.143:0/550476057 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7ff58c0b2db0 0x7ff58c14f790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:23:38.678 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:38.675+0000 7ff58affd640 1 -- 172.21.15.143:0/550476057 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7ff58c1596d0 msgr2=0x7ff58c0cd610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:23:38.678 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:38.675+0000 7ff58affd640 1 --2- 172.21.15.143:0/550476057 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7ff58c1596d0 0x7ff58c0cd610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:23:38.678 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:38.675+0000 7ff58affd640 1 -- 172.21.15.143:0/550476057 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7ff58c0b2db0 msgr2=0x7ff58c14f790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:23:38.678 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:38.675+0000 7ff58affd640 1 --2- 172.21.15.143:0/550476057 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7ff58c0b2db0 0x7ff58c14f790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:23:38.678 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:38.675+0000 7ff58affd640 1 -- 172.21.15.143:0/550476057 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff584009b30 con 0x7ff58c0ce680 2023-11-26T07:23:38.678 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:38.675+0000 7ff58bfff640 1 --2- 172.21.15.143:0/550476057 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7ff58c1596d0 0x7ff58c0cd610 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-26T07:23:38.678 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:38.675+0000 7ff58affd640 1 --2- 172.21.15.143:0/550476057 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7ff58c0ce680 0x7ff58c150e10 secure :-1 s=READY pgs=1177 cs=0 l=1 rev1=1 crypto rx=0x7ff58000c7c0 tx=0x7ff58000cc80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:23:38.679 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:38.675+0000 7ff573fff640 1 -- 172.21.15.143:0/550476057 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff580007a30 con 0x7ff58c0ce680 2023-11-26T07:23:38.679 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:38.675+0000 7ff573fff640 1 -- 172.21.15.143:0/550476057 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff580007b90 con 0x7ff58c0ce680 2023-11-26T07:23:38.679 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:38.675+0000 7ff591a91640 1 -- 172.21.15.143:0/550476057 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff58c150330 con 0x7ff58c0ce680 2023-11-26T07:23:38.679 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:38.675+0000 7ff573fff640 1 -- 172.21.15.143:0/550476057 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff58000ce70 con 0x7ff58c0ce680 2023-11-26T07:23:38.679 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:38.675+0000 7ff591a91640 1 -- 172.21.15.143:0/550476057 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff58c1508f0 con 0x7ff58c0ce680 2023-11-26T07:23:38.679 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:38.679+0000 7ff591a91640 1 -- 172.21.15.143:0/550476057 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff58c067680 con 0x7ff58c0ce680 2023-11-26T07:23:38.680 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:38.679+0000 7ff573fff640 1 -- 172.21.15.143:0/550476057 <== mon.0 v2:172.21.15.143:3300/0 4 ==== mgrmap(e 16) v1 ==== 85954+0+0 (secure 0 0 0) 0x7ff5800113f0 con 0x7ff58c0ce680 2023-11-26T07:23:38.680 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:38.679+0000 7ff573fff640 1 --2- 172.21.15.143:0/550476057 >> [v2:172.21.15.188:6836/3401128553,v1:172.21.15.188:6837/3401128553] conn(0x7ff578066590 0x7ff578068a50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:23:38.680 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:38.679+0000 7ff573fff640 1 -- 172.21.15.143:0/550476057 <== mon.0 v2:172.21.15.143:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 5709+0+0 (secure 0 0 0) 0x7ff580084570 con 0x7ff58c0ce680 2023-11-26T07:23:38.681 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:38.679+0000 7ff58b7fe640 1 --2- 172.21.15.143:0/550476057 >> [v2:172.21.15.188:6836/3401128553,v1:172.21.15.188:6837/3401128553] conn(0x7ff578066590 0x7ff578068a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:23:38.681 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:38.679+0000 7ff58b7fe640 1 --2- 172.21.15.143:0/550476057 >> [v2:172.21.15.188:6836/3401128553,v1:172.21.15.188:6837/3401128553] conn(0x7ff578066590 0x7ff578068a50 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7ff57c006fd0 tx=0x7ff57c008040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:23:38.685 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:38.683+0000 7ff573fff640 1 -- 172.21.15.143:0/550476057 <== mon.0 v2:172.21.15.143:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172649 (secure 0 0 0) 0x7ff580051800 con 0x7ff58c0ce680 2023-11-26T07:23:38.867 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:38.867+0000 7ff591a91640 1 -- 172.21.15.143:0/550476057 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "fs flag set", "flag_name": "enable_multiple", "val": "true", "yes_i_really_mean_it": true} v 0) v1 -- 0x7ff58c15de90 con 0x7ff58c0ce680 2023-11-26T07:23:39.703 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:39.703+0000 7ff573fff640 1 -- 172.21.15.143:0/550476057 <== mon.0 v2:172.21.15.143:3300/0 7 ==== mon_command_ack([{"prefix": "fs flag set", "flag_name": "enable_multiple", "val": "true", "yes_i_really_mean_it": true}]=0 v52) v1 ==== 136+0+0 (secure 0 0 0) 0x7ff580054e20 con 0x7ff58c0ce680 2023-11-26T07:23:39.705 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:39.703+0000 7ff591a91640 1 -- 172.21.15.143:0/550476057 >> [v2:172.21.15.188:6836/3401128553,v1:172.21.15.188:6837/3401128553] conn(0x7ff578066590 msgr2=0x7ff578068a50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:23:39.705 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:39.703+0000 7ff591a91640 1 --2- 172.21.15.143:0/550476057 >> [v2:172.21.15.188:6836/3401128553,v1:172.21.15.188:6837/3401128553] conn(0x7ff578066590 0x7ff578068a50 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7ff57c006fd0 tx=0x7ff57c008040 comp rx=0 tx=0).stop 2023-11-26T07:23:39.705 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:39.703+0000 7ff591a91640 1 -- 172.21.15.143:0/550476057 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7ff58c0ce680 msgr2=0x7ff58c150e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:23:39.705 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:39.703+0000 7ff591a91640 1 --2- 172.21.15.143:0/550476057 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7ff58c0ce680 0x7ff58c150e10 secure :-1 s=READY pgs=1177 cs=0 l=1 rev1=1 crypto rx=0x7ff58000c7c0 tx=0x7ff58000cc80 comp rx=0 tx=0).stop 2023-11-26T07:23:39.706 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:39.703+0000 7ff591a91640 1 -- 172.21.15.143:0/550476057 shutdown_connections 2023-11-26T07:23:39.706 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:39.703+0000 7ff591a91640 1 --2- 172.21.15.143:0/550476057 >> [v2:172.21.15.188:6836/3401128553,v1:172.21.15.188:6837/3401128553] conn(0x7ff578066590 0x7ff578068a50 unknown :-1 s=CLOSED pgs=70 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:23:39.706 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:39.703+0000 7ff591a91640 1 --2- 172.21.15.143:0/550476057 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7ff58c1596d0 0x7ff58c0cd610 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:23:39.706 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:39.703+0000 7ff591a91640 1 --2- 172.21.15.143:0/550476057 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7ff58c0ce680 0x7ff58c150e10 unknown :-1 s=CLOSED pgs=1177 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:23:39.706 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:39.703+0000 7ff591a91640 1 --2- 172.21.15.143:0/550476057 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7ff58c0b2db0 0x7ff58c14f790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:23:39.706 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:39.703+0000 7ff591a91640 1 -- 172.21.15.143:0/550476057 >> 172.21.15.143:0/550476057 conn(0x7ff58c06a890 msgr2=0x7ff58c0b03a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:23:39.706 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:39.703+0000 7ff591a91640 1 -- 172.21.15.143:0/550476057 shutdown_connections 2023-11-26T07:23:39.706 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:39.703+0000 7ff591a91640 1 -- 172.21.15.143:0/550476057 wait complete. 2023-11-26T07:23:39.728 DEBUG:teuthology.orchestra.run.smithi143:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs volume ls 2023-11-26T07:23:39.868 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:39.867+0000 7fa6eee6b640 1 Processor -- start 2023-11-26T07:23:39.868 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:39.867+0000 7fa6eee6b640 1 -- start start 2023-11-26T07:23:39.870 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:39.867+0000 7fa6eee6b640 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7fa6e81614b0 0x7fa6e80cd610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:23:39.870 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:39.867+0000 7fa6eee6b640 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7fa6e8071970 0x7fa6e8161890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:23:39.871 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:39.867+0000 7fa6eee6b640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fa6e8070ba0 0x7fa6e8071010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:23:39.871 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:39.867+0000 7fa6eee6b640 1 -- --> v2:172.21.15.143:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa6e8071550 con 0x7fa6e8071970 2023-11-26T07:23:39.871 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:39.867+0000 7fa6eee6b640 1 -- --> v2:172.21.15.188:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa6e815d970 con 0x7fa6e81614b0 2023-11-26T07:23:39.871 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:39.867+0000 7fa6eee6b640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa6e8161dd0 con 0x7fa6e8070ba0 2023-11-26T07:23:39.871 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:39.867+0000 7fa6dffff640 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7fa6e8071970 0x7fa6e8161890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:23:39.871 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:39.867+0000 7fa6dffff640 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7fa6e8071970 0x7fa6e8161890 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:55770/0 (socket says 172.21.15.143:55770) 2023-11-26T07:23:39.871 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:39.867+0000 7fa6dffff640 1 -- 172.21.15.143:0/978023904 learned_addr learned my addr 172.21.15.143:0/978023904 (peer_addr_for_me v2:172.21.15.143:0/0) 2023-11-26T07:23:39.871 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:39.867+0000 7fa6ed408640 1 --2- 172.21.15.143:0/978023904 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fa6e8070ba0 0x7fa6e8071010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:23:39.871 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:39.867+0000 7fa6ecc07640 1 --2- 172.21.15.143:0/978023904 >> v2:172.21.15.188:3300/0 conn(0x7fa6e81614b0 0x7fa6e80cd610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:23:39.872 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:39.871+0000 7fa6dffff640 1 -- 172.21.15.143:0/978023904 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fa6e8070ba0 msgr2=0x7fa6e8071010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-26T07:23:39.872 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:39.871+0000 7fa6dffff640 1 --2- 172.21.15.143:0/978023904 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fa6e8070ba0 0x7fa6e8071010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:23:39.872 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:39.871+0000 7fa6dffff640 1 -- 172.21.15.143:0/978023904 >> v2:172.21.15.188:3300/0 conn(0x7fa6e81614b0 msgr2=0x7fa6e80cd610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-26T07:23:39.872 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:39.871+0000 7fa6dffff640 1 --2- 172.21.15.143:0/978023904 >> v2:172.21.15.188:3300/0 conn(0x7fa6e81614b0 0x7fa6e80cd610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:23:39.872 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:39.871+0000 7fa6dffff640 1 -- 172.21.15.143:0/978023904 --> v2:172.21.15.143:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa6e815dc30 con 0x7fa6e8071970 2023-11-26T07:23:39.872 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:39.871+0000 7fa6ed408640 1 --2- 172.21.15.143:0/978023904 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fa6e8070ba0 0x7fa6e8071010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-26T07:23:39.873 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:39.871+0000 7fa6dffff640 1 --2- 172.21.15.143:0/978023904 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fa6e8071970 0x7fa6e8161890 secure :-1 s=READY pgs=1178 cs=0 l=1 rev1=1 crypto rx=0x7fa6e8067300 tx=0x7fa6d801c4d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8071d3606013890b server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:23:39.874 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:39.871+0000 7fa6df7fe640 1 -- 172.21.15.143:0/978023904 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa6d80aa5f0 con 0x7fa6e8071970 2023-11-26T07:23:39.874 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:39.871+0000 7fa6df7fe640 1 -- 172.21.15.143:0/978023904 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa6d80aa750 con 0x7fa6e8071970 2023-11-26T07:23:39.874 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:39.871+0000 7fa6df7fe640 1 -- 172.21.15.143:0/978023904 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa6d80aaae0 con 0x7fa6e8071970 2023-11-26T07:23:39.875 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:39.871+0000 7fa6eee6b640 1 -- 172.21.15.143:0/978023904 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fa6e8071970 msgr2=0x7fa6e8161890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:23:39.875 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:39.871+0000 7fa6eee6b640 1 --2- 172.21.15.143:0/978023904 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fa6e8071970 0x7fa6e8161890 secure :-1 s=READY pgs=1178 cs=0 l=1 rev1=1 crypto rx=0x7fa6e8067300 tx=0x7fa6d801c4d0 comp rx=0 tx=0).stop 2023-11-26T07:23:39.875 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:39.871+0000 7fa6eee6b640 1 -- 172.21.15.143:0/978023904 shutdown_connections 2023-11-26T07:23:39.875 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:39.871+0000 7fa6eee6b640 1 --2- 172.21.15.143:0/978023904 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fa6e8070ba0 0x7fa6e8071010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:23:39.875 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:39.871+0000 7fa6eee6b640 1 --2- 172.21.15.143:0/978023904 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fa6e8071970 0x7fa6e8161890 unknown :-1 s=CLOSED pgs=1178 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:23:39.875 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:39.871+0000 7fa6eee6b640 1 --2- 172.21.15.143:0/978023904 >> v2:172.21.15.188:3300/0 conn(0x7fa6e81614b0 0x7fa6e80cd610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:23:39.875 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:39.871+0000 7fa6eee6b640 1 -- 172.21.15.143:0/978023904 >> 172.21.15.143:0/978023904 conn(0x7fa6e80cf710 msgr2=0x7fa6e80cfb20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:23:39.875 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:39.871+0000 7fa6eee6b640 1 -- 172.21.15.143:0/978023904 shutdown_connections 2023-11-26T07:23:39.875 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:39.871+0000 7fa6eee6b640 1 -- 172.21.15.143:0/978023904 wait complete. 2023-11-26T07:23:39.876 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:39.875+0000 7fa6eee6b640 1 Processor -- start 2023-11-26T07:23:39.876 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:39.875+0000 7fa6eee6b640 1 -- start start 2023-11-26T07:23:39.879 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:39.875+0000 7fa6eee6b640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fa6e8070ba0 0x7fa6e8157f60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:23:39.879 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:39.875+0000 7fa6eee6b640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fa6e8071970 0x7fa6e80ae130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:23:39.879 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:39.875+0000 7fa6eee6b640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fa6e81614b0 0x7fa6e8154e60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:23:39.879 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:39.875+0000 7fa6eee6b640 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa6e80b2b20 con 0x7fa6e8070ba0 2023-11-26T07:23:39.879 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:39.875+0000 7fa6eee6b640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa6e8166f60 con 0x7fa6e81614b0 2023-11-26T07:23:39.879 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:39.875+0000 7fa6eee6b640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa6e8166c80 con 0x7fa6e8071970 2023-11-26T07:23:39.879 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:39.875+0000 7fa6ecc07640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fa6e8070ba0 0x7fa6e8157f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:23:39.880 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:39.875+0000 7fa6ecc07640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fa6e8070ba0 0x7fa6e8157f60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:55772/0 (socket says 172.21.15.143:55772) 2023-11-26T07:23:39.880 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:39.875+0000 7fa6ecc07640 1 -- 172.21.15.143:0/3832626314 learned_addr learned my addr 172.21.15.143:0/3832626314 (peer_addr_for_me v2:172.21.15.143:0/0) 2023-11-26T07:23:39.880 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:39.875+0000 7fa6dffff640 1 --2- 172.21.15.143:0/3832626314 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fa6e8071970 0x7fa6e80ae130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:23:39.880 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:39.875+0000 7fa6ed408640 1 --2- 172.21.15.143:0/3832626314 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fa6e81614b0 0x7fa6e8154e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:23:39.880 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:39.875+0000 7fa6ecc07640 1 -- 172.21.15.143:0/3832626314 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fa6e8071970 msgr2=0x7fa6e80ae130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:23:39.880 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:39.875+0000 7fa6ecc07640 1 --2- 172.21.15.143:0/3832626314 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fa6e8071970 0x7fa6e80ae130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:23:39.880 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:39.875+0000 7fa6ecc07640 1 -- 172.21.15.143:0/3832626314 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fa6e81614b0 msgr2=0x7fa6e8154e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:23:39.880 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:39.875+0000 7fa6ecc07640 1 --2- 172.21.15.143:0/3832626314 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fa6e81614b0 0x7fa6e8154e60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:23:39.880 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:39.875+0000 7fa6ecc07640 1 -- 172.21.15.143:0/3832626314 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa6d80094d0 con 0x7fa6e8070ba0 2023-11-26T07:23:39.881 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:39.875+0000 7fa6ecc07640 1 --2- 172.21.15.143:0/3832626314 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fa6e8070ba0 0x7fa6e8157f60 secure :-1 s=READY pgs=1179 cs=0 l=1 rev1=1 crypto rx=0x7fa6e400e960 tx=0x7fa6e400ee20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:23:39.881 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:39.879+0000 7fa6dd7fa640 1 -- 172.21.15.143:0/3832626314 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa6e4009690 con 0x7fa6e8070ba0 2023-11-26T07:23:39.881 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:39.879+0000 7fa6dd7fa640 1 -- 172.21.15.143:0/3832626314 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa6e400ca00 con 0x7fa6e8070ba0 2023-11-26T07:23:39.881 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:39.879+0000 7fa6dd7fa640 1 -- 172.21.15.143:0/3832626314 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa6e400cd70 con 0x7fa6e8070ba0 2023-11-26T07:23:39.881 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:39.879+0000 7fa6eee6b640 1 -- 172.21.15.143:0/3832626314 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa6e80b2460 con 0x7fa6e8070ba0 2023-11-26T07:23:39.881 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:39.879+0000 7fa6eee6b640 1 -- 172.21.15.143:0/3832626314 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa6e8157690 con 0x7fa6e8070ba0 2023-11-26T07:23:39.882 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:39.879+0000 7fa6eee6b640 1 -- 172.21.15.143:0/3832626314 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa6e8067680 con 0x7fa6e8070ba0 2023-11-26T07:23:39.882 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:39.879+0000 7fa6dd7fa640 1 -- 172.21.15.143:0/3832626314 <== mon.0 v2:172.21.15.143:3300/0 4 ==== mgrmap(e 16) v1 ==== 85954+0+0 (secure 0 0 0) 0x7fa6e4013470 con 0x7fa6e8070ba0 2023-11-26T07:23:39.882 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:39.879+0000 7fa6dd7fa640 1 --2- 172.21.15.143:0/3832626314 >> [v2:172.21.15.188:6836/3401128553,v1:172.21.15.188:6837/3401128553] conn(0x7fa6d4066590 0x7fa6d4068a50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:23:39.882 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:39.879+0000 7fa6dd7fa640 1 -- 172.21.15.143:0/3832626314 <== mon.0 v2:172.21.15.143:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fa6e4084190 con 0x7fa6e8070ba0 2023-11-26T07:23:39.882 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:39.879+0000 7fa6dffff640 1 --2- 172.21.15.143:0/3832626314 >> [v2:172.21.15.188:6836/3401128553,v1:172.21.15.188:6837/3401128553] conn(0x7fa6d4066590 0x7fa6d4068a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:23:39.883 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:39.883+0000 7fa6dffff640 1 --2- 172.21.15.143:0/3832626314 >> [v2:172.21.15.188:6836/3401128553,v1:172.21.15.188:6837/3401128553] conn(0x7fa6d4066590 0x7fa6d4068a50 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7fa6d80ae000 tx=0x7fa6d80b5040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:23:39.886 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:39.883+0000 7fa6dd7fa640 1 -- 172.21.15.143:0/3832626314 <== mon.0 v2:172.21.15.143:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172649 (secure 0 0 0) 0x7fa6e40515a0 con 0x7fa6e8070ba0 2023-11-26T07:23:40.056 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:40.055+0000 7fa6eee6b640 1 -- 172.21.15.143:0/3832626314 --> [v2:172.21.15.188:6836/3401128553,v1:172.21.15.188:6837/3401128553] -- mgr_command(tid 0: {"prefix": "fs volume ls", "target": ["mon-mgr", ""]}) v1 -- 0x7fa6e8161890 con 0x7fa6d4066590 2023-11-26T07:23:40.058 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:40.055+0000 7fa6dd7fa640 1 -- 172.21.15.143:0/3832626314 <== mgr.4108 v2:172.21.15.188:6836/3401128553 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+41 (secure 0 0 0) 0x7fa6e8161890 con 0x7fa6d4066590 2023-11-26T07:23:40.058 INFO:teuthology.orchestra.run.smithi143.stdout:[ 2023-11-26T07:23:40.058 INFO:teuthology.orchestra.run.smithi143.stdout: { 2023-11-26T07:23:40.058 INFO:teuthology.orchestra.run.smithi143.stdout: "name": "cephfs" 2023-11-26T07:23:40.058 INFO:teuthology.orchestra.run.smithi143.stdout: } 2023-11-26T07:23:40.058 INFO:teuthology.orchestra.run.smithi143.stdout:] 2023-11-26T07:23:40.060 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:40.059+0000 7fa6eee6b640 1 -- 172.21.15.143:0/3832626314 >> [v2:172.21.15.188:6836/3401128553,v1:172.21.15.188:6837/3401128553] conn(0x7fa6d4066590 msgr2=0x7fa6d4068a50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:23:40.060 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:40.059+0000 7fa6eee6b640 1 --2- 172.21.15.143:0/3832626314 >> [v2:172.21.15.188:6836/3401128553,v1:172.21.15.188:6837/3401128553] conn(0x7fa6d4066590 0x7fa6d4068a50 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7fa6d80ae000 tx=0x7fa6d80b5040 comp rx=0 tx=0).stop 2023-11-26T07:23:40.060 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:40.059+0000 7fa6eee6b640 1 -- 172.21.15.143:0/3832626314 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fa6e8070ba0 msgr2=0x7fa6e8157f60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:23:40.060 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:40.059+0000 7fa6eee6b640 1 --2- 172.21.15.143:0/3832626314 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fa6e8070ba0 0x7fa6e8157f60 secure :-1 s=READY pgs=1179 cs=0 l=1 rev1=1 crypto rx=0x7fa6e400e960 tx=0x7fa6e400ee20 comp rx=0 tx=0).stop 2023-11-26T07:23:40.060 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:40.059+0000 7fa6eee6b640 1 -- 172.21.15.143:0/3832626314 shutdown_connections 2023-11-26T07:23:40.061 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:40.059+0000 7fa6eee6b640 1 --2- 172.21.15.143:0/3832626314 >> [v2:172.21.15.188:6836/3401128553,v1:172.21.15.188:6837/3401128553] conn(0x7fa6d4066590 0x7fa6d4068a50 unknown :-1 s=CLOSED pgs=71 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:23:40.061 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:40.059+0000 7fa6eee6b640 1 --2- 172.21.15.143:0/3832626314 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fa6e81614b0 0x7fa6e8154e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:23:40.061 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:40.059+0000 7fa6eee6b640 1 --2- 172.21.15.143:0/3832626314 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fa6e8071970 0x7fa6e80ae130 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:23:40.061 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:40.059+0000 7fa6eee6b640 1 --2- 172.21.15.143:0/3832626314 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fa6e8070ba0 0x7fa6e8157f60 unknown :-1 s=CLOSED pgs=1179 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:23:40.061 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:40.059+0000 7fa6eee6b640 1 -- 172.21.15.143:0/3832626314 >> 172.21.15.143:0/3832626314 conn(0x7fa6e80cf710 msgr2=0x7fa6e815eaa0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:23:40.061 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:40.059+0000 7fa6eee6b640 1 -- 172.21.15.143:0/3832626314 shutdown_connections 2023-11-26T07:23:40.061 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:40.059+0000 7fa6eee6b640 1 -- 172.21.15.143:0/3832626314 wait complete. 2023-11-26T07:23:40.079 DEBUG:teuthology.orchestra.run.smithi143:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mon mon_allow_pool_delete True 2023-11-26T07:23:40.219 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:40.219+0000 7fd0ccfc4640 1 Processor -- start 2023-11-26T07:23:40.219 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:40.219+0000 7fd0ccfc4640 1 -- start start 2023-11-26T07:23:40.220 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:40.219+0000 7fd0ccfc4640 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7fd0c80cf280 0x7fd0c80cf710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:23:40.220 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:40.219+0000 7fd0ccfc4640 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7fd0c80cd1b0 0x7fd0c80cd5c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:23:40.221 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:40.219+0000 7fd0ccfc4640 1 -- --> v1:172.21.15.143:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd0c80cc730 con 0x7fd0c8143ac0 2023-11-26T07:23:40.221 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:40.219+0000 7fd0ccfc4640 1 -- --> v2:172.21.15.143:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd0c8143ea0 con 0x7fd0c80cf280 2023-11-26T07:23:40.221 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:40.219+0000 7fd0ccfc4640 1 -- --> v2:172.21.15.188:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd0c80cfc50 con 0x7fd0c80cd1b0 2023-11-26T07:23:40.221 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:40.219+0000 7fd0c659c640 1 --1- >> v1:172.21.15.143:6789/0 conn(0x7fd0c8143ac0 0x7fd0c8070370 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.143:6789/0 says I am v1:172.21.15.143:48000/0 (socket says 172.21.15.143:48000) 2023-11-26T07:23:40.221 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:40.219+0000 7fd0c659c640 1 -- 172.21.15.143:0/2995580695 learned_addr learned my addr 172.21.15.143:0/2995580695 (peer_addr_for_me v1:172.21.15.143:0/0) 2023-11-26T07:23:40.221 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:40.219+0000 7fd0c5d9b640 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7fd0c80cf280 0x7fd0c80cf710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:23:40.221 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:40.219+0000 7fd0c6d9d640 1 --2- 172.21.15.143:0/2995580695 >> v2:172.21.15.188:3300/0 conn(0x7fd0c80cd1b0 0x7fd0c80cd5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:23:40.222 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:40.219+0000 7fd0c5d9b640 1 -- 172.21.15.143:0/2995580695 >> v2:172.21.15.188:3300/0 conn(0x7fd0c80cd1b0 msgr2=0x7fd0c80cd5c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-26T07:23:40.222 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:40.219+0000 7fd0c5d9b640 1 --2- 172.21.15.143:0/2995580695 >> v2:172.21.15.188:3300/0 conn(0x7fd0c80cd1b0 0x7fd0c80cd5c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:23:40.222 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:40.219+0000 7fd0c5d9b640 1 -- 172.21.15.143:0/2995580695 >> v1:172.21.15.143:6789/0 conn(0x7fd0c8143ac0 legacy=0x7fd0c8070370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:23:40.223 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:40.219+0000 7fd0c5d9b640 1 -- 172.21.15.143:0/2995580695 --> v2:172.21.15.143:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd0c8070ee0 con 0x7fd0c80cf280 2023-11-26T07:23:40.223 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:40.219+0000 7fd0c559a640 1 -- 172.21.15.143:0/2995580695 <== mon.0 v1:172.21.15.143:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 901762314 0 0) 0x7fd0c80cc730 con 0x7fd0c8143ac0 2023-11-26T07:23:40.223 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:40.223+0000 7fd0c5d9b640 1 --2- 172.21.15.143:0/2995580695 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fd0c80cf280 0x7fd0c80cf710 secure :-1 s=READY pgs=1181 cs=0 l=1 rev1=1 crypto rx=0x7fd0c806e570 tx=0x7fd0bc01c460 comp rx=0 tx=0).ready entity=mon.0 client_cookie=37bbbbf7ad74fa28 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:23:40.224 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:40.223+0000 7fd0c559a640 1 -- 172.21.15.143:0/2995580695 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd0bc0ab660 con 0x7fd0c80cf280 2023-11-26T07:23:40.224 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:40.223+0000 7fd0c559a640 1 -- 172.21.15.143:0/2995580695 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd0bc0ab7c0 con 0x7fd0c80cf280 2023-11-26T07:23:40.224 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:40.223+0000 7fd0c559a640 1 -- 172.21.15.143:0/2995580695 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd0bc0abb50 con 0x7fd0c80cf280 2023-11-26T07:23:40.224 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:40.223+0000 7fd0ccfc4640 1 -- 172.21.15.143:0/2995580695 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fd0c80cf280 msgr2=0x7fd0c80cf710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:23:40.225 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:40.223+0000 7fd0ccfc4640 1 --2- 172.21.15.143:0/2995580695 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fd0c80cf280 0x7fd0c80cf710 secure :-1 s=READY pgs=1181 cs=0 l=1 rev1=1 crypto rx=0x7fd0c806e570 tx=0x7fd0bc01c460 comp rx=0 tx=0).stop 2023-11-26T07:23:40.225 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:40.223+0000 7fd0ccfc4640 1 -- 172.21.15.143:0/2995580695 shutdown_connections 2023-11-26T07:23:40.225 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:40.223+0000 7fd0ccfc4640 1 --2- 172.21.15.143:0/2995580695 >> v2:172.21.15.188:3300/0 conn(0x7fd0c80cd1b0 0x7fd0c80cd5c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:23:40.225 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:40.223+0000 7fd0ccfc4640 1 --2- 172.21.15.143:0/2995580695 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fd0c80cf280 0x7fd0c80cf710 unknown :-1 s=CLOSED pgs=1181 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:23:40.226 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:40.223+0000 7fd0ccfc4640 1 -- 172.21.15.143:0/2995580695 >> 172.21.15.143:0/2995580695 conn(0x7fd0c806a890 msgr2=0x7fd0c806aca0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:23:40.226 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:40.223+0000 7fd0ccfc4640 1 -- 172.21.15.143:0/2995580695 shutdown_connections 2023-11-26T07:23:40.226 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:40.223+0000 7fd0ccfc4640 1 -- 172.21.15.143:0/2995580695 wait complete. 2023-11-26T07:23:40.226 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:40.227+0000 7fd0ccfc4640 1 Processor -- start 2023-11-26T07:23:40.226 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:40.227+0000 7fd0ccfc4640 1 -- start start 2023-11-26T07:23:40.228 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:40.227+0000 7fd0ccfc4640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fd0c80cd1b0 0x7fd0c815c170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:23:40.228 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:40.227+0000 7fd0ccfc4640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fd0c80cf280 0x7fd0c815a880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:23:40.228 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:40.227+0000 7fd0ccfc4640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fd0c8143ac0 0x7fd0c813ce50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:23:40.229 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:40.227+0000 7fd0ccfc4640 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd0c815b0d0 con 0x7fd0c80cf280 2023-11-26T07:23:40.229 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:40.227+0000 7fd0ccfc4640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd0c815b3b0 con 0x7fd0c8143ac0 2023-11-26T07:23:40.229 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:40.227+0000 7fd0ccfc4640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd0c815b690 con 0x7fd0c80cd1b0 2023-11-26T07:23:40.229 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:40.227+0000 7fd0c5d9b640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fd0c80cf280 0x7fd0c815a880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:23:40.229 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:40.227+0000 7fd0c5d9b640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fd0c80cf280 0x7fd0c815a880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:55786/0 (socket says 172.21.15.143:55786) 2023-11-26T07:23:40.229 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:40.227+0000 7fd0c5d9b640 1 -- 172.21.15.143:0/3592878937 learned_addr learned my addr 172.21.15.143:0/3592878937 (peer_addr_for_me v2:172.21.15.143:0/0) 2023-11-26T07:23:40.229 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:40.227+0000 7fd0c659c640 1 --2- 172.21.15.143:0/3592878937 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fd0c80cd1b0 0x7fd0c815c170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:23:40.229 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:40.227+0000 7fd0c6d9d640 1 --2- 172.21.15.143:0/3592878937 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fd0c8143ac0 0x7fd0c813ce50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:23:40.229 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:40.227+0000 7fd0c5d9b640 1 -- 172.21.15.143:0/3592878937 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fd0c80cd1b0 msgr2=0x7fd0c815c170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:23:40.229 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:40.227+0000 7fd0c5d9b640 1 --2- 172.21.15.143:0/3592878937 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fd0c80cd1b0 0x7fd0c815c170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:23:40.229 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:40.227+0000 7fd0c5d9b640 1 -- 172.21.15.143:0/3592878937 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fd0c8143ac0 msgr2=0x7fd0c813ce50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:23:40.229 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:40.227+0000 7fd0c5d9b640 1 --2- 172.21.15.143:0/3592878937 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fd0c8143ac0 0x7fd0c813ce50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:23:40.230 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:40.227+0000 7fd0c5d9b640 1 -- 172.21.15.143:0/3592878937 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd0bc009420 con 0x7fd0c80cf280 2023-11-26T07:23:40.230 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:40.227+0000 7fd0c5d9b640 1 --2- 172.21.15.143:0/3592878937 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fd0c80cf280 0x7fd0c815a880 secure :-1 s=READY pgs=1182 cs=0 l=1 rev1=1 crypto rx=0x7fd0bc019070 tx=0x7fd0bc01aca0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:23:40.230 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:40.227+0000 7fd0b6ffd640 1 -- 172.21.15.143:0/3592878937 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd0bc0b5810 con 0x7fd0c80cf280 2023-11-26T07:23:40.231 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:40.227+0000 7fd0b6ffd640 1 -- 172.21.15.143:0/3592878937 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd0bc0b5990 con 0x7fd0c80cf280 2023-11-26T07:23:40.231 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:40.227+0000 7fd0ccfc4640 1 -- 172.21.15.143:0/3592878937 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd0c815b970 con 0x7fd0c80cf280 2023-11-26T07:23:40.231 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:40.227+0000 7fd0b6ffd640 1 -- 172.21.15.143:0/3592878937 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd0bc0b5d40 con 0x7fd0c80cf280 2023-11-26T07:23:40.231 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:40.227+0000 7fd0ccfc4640 1 -- 172.21.15.143:0/3592878937 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd0c815adc0 con 0x7fd0c80cf280 2023-11-26T07:23:40.231 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:40.227+0000 7fd0ccfc4640 1 -- 172.21.15.143:0/3592878937 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd0c815be90 con 0x7fd0c80cf280 2023-11-26T07:23:40.231 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:40.231+0000 7fd0b6ffd640 1 -- 172.21.15.143:0/3592878937 <== mon.0 v2:172.21.15.143:3300/0 4 ==== mgrmap(e 16) v1 ==== 85954+0+0 (secure 0 0 0) 0x7fd0bc0a9070 con 0x7fd0c80cf280 2023-11-26T07:23:40.232 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:40.231+0000 7fd0b6ffd640 1 --2- 172.21.15.143:0/3592878937 >> [v2:172.21.15.188:6836/3401128553,v1:172.21.15.188:6837/3401128553] conn(0x7fd0a8066590 0x7fd0a8068a50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:23:40.232 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:40.231+0000 7fd0b6ffd640 1 -- 172.21.15.143:0/3592878937 <== mon.0 v2:172.21.15.143:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fd0bc11e860 con 0x7fd0c80cf280 2023-11-26T07:23:40.232 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:40.231+0000 7fd0c659c640 1 --2- 172.21.15.143:0/3592878937 >> [v2:172.21.15.188:6836/3401128553,v1:172.21.15.188:6837/3401128553] conn(0x7fd0a8066590 0x7fd0a8068a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:23:40.233 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:40.231+0000 7fd0c659c640 1 --2- 172.21.15.143:0/3592878937 >> [v2:172.21.15.188:6836/3401128553,v1:172.21.15.188:6837/3401128553] conn(0x7fd0a8066590 0x7fd0a8068a50 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7fd0b8007fd0 tx=0x7fd0b8007d60 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:23:40.235 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:40.235+0000 7fd0b6ffd640 1 -- 172.21.15.143:0/3592878937 <== mon.0 v2:172.21.15.143:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172649 (secure 0 0 0) 0x7fd0c815be90 con 0x7fd0c80cf280 2023-11-26T07:23:40.384 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:40.383+0000 7fd0ccfc4640 1 -- 172.21.15.143:0/3592878937 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command([{prefix=config set, name=mon_allow_pool_delete}] v 0) v1 -- 0x7fd0c815be90 con 0x7fd0c80cf280 2023-11-26T07:23:40.384 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:40.383+0000 7fd0b6ffd640 1 -- 172.21.15.143:0/3592878937 <== mon.0 v2:172.21.15.143:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mon_allow_pool_delete}]=0 v18)=0 v18) v1 ==== 122+0+0 (secure 0 0 0) 0x7fd0c815be90 con 0x7fd0c80cf280 2023-11-26T07:23:40.386 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:40.383+0000 7fd0ccfc4640 1 -- 172.21.15.143:0/3592878937 >> [v2:172.21.15.188:6836/3401128553,v1:172.21.15.188:6837/3401128553] conn(0x7fd0a8066590 msgr2=0x7fd0a8068a50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:23:40.386 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:40.383+0000 7fd0ccfc4640 1 --2- 172.21.15.143:0/3592878937 >> [v2:172.21.15.188:6836/3401128553,v1:172.21.15.188:6837/3401128553] conn(0x7fd0a8066590 0x7fd0a8068a50 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7fd0b8007fd0 tx=0x7fd0b8007d60 comp rx=0 tx=0).stop 2023-11-26T07:23:40.387 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:40.383+0000 7fd0ccfc4640 1 -- 172.21.15.143:0/3592878937 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fd0c80cf280 msgr2=0x7fd0c815a880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:23:40.387 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:40.383+0000 7fd0ccfc4640 1 --2- 172.21.15.143:0/3592878937 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fd0c80cf280 0x7fd0c815a880 secure :-1 s=READY pgs=1182 cs=0 l=1 rev1=1 crypto rx=0x7fd0bc019070 tx=0x7fd0bc01aca0 comp rx=0 tx=0).stop 2023-11-26T07:23:40.387 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:40.383+0000 7fd0ccfc4640 1 -- 172.21.15.143:0/3592878937 shutdown_connections 2023-11-26T07:23:40.387 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:40.387+0000 7fd0ccfc4640 1 --2- 172.21.15.143:0/3592878937 >> [v2:172.21.15.188:6836/3401128553,v1:172.21.15.188:6837/3401128553] conn(0x7fd0a8066590 0x7fd0a8068a50 unknown :-1 s=CLOSED pgs=72 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:23:40.387 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:40.387+0000 7fd0ccfc4640 1 --2- 172.21.15.143:0/3592878937 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fd0c8143ac0 0x7fd0c813ce50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:23:40.387 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:40.387+0000 7fd0ccfc4640 1 --2- 172.21.15.143:0/3592878937 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fd0c80cf280 0x7fd0c815a880 unknown :-1 s=CLOSED pgs=1182 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:23:40.387 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:40.387+0000 7fd0ccfc4640 1 --2- 172.21.15.143:0/3592878937 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fd0c80cd1b0 0x7fd0c815c170 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:23:40.387 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:40.387+0000 7fd0ccfc4640 1 -- 172.21.15.143:0/3592878937 >> 172.21.15.143:0/3592878937 conn(0x7fd0c806a890 msgr2=0x7fd0c8140610 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:23:40.387 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:40.387+0000 7fd0ccfc4640 1 -- 172.21.15.143:0/3592878937 shutdown_connections 2023-11-26T07:23:40.387 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:40.387+0000 7fd0ccfc4640 1 -- 172.21.15.143:0/3592878937 wait complete. 2023-11-26T07:23:40.405 DEBUG:teuthology.orchestra.run.smithi143:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs volume create 'abcd@#' 2023-11-26T07:23:40.544 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:40.543+0000 7fae60b72640 1 Processor -- start 2023-11-26T07:23:40.544 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:40.543+0000 7fae60b72640 1 -- start start 2023-11-26T07:23:40.545 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:40.543+0000 7fae60b72640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fae5c15d6e0 0x7fae5c0d0df0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:23:40.545 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:40.543+0000 7fae60b72640 1 -- --> v1:172.21.15.143:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fae5c0cf0e0 con 0x7fae5c0d1bc0 2023-11-26T07:23:40.545 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:40.543+0000 7fae60b72640 1 -- --> v1:172.21.15.188:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fae5c0ceb20 con 0x7fae5c0b2160 2023-11-26T07:23:40.545 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:40.543+0000 7fae60b72640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fae5c15dac0 con 0x7fae5c15d6e0 2023-11-26T07:23:40.545 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:40.543+0000 7fae5ad9d640 1 --1- >> v1:172.21.15.143:6789/0 conn(0x7fae5c0d1bc0 0x7fae5c0b0750 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.143:6789/0 says I am v1:172.21.15.143:48002/0 (socket says 172.21.15.143:48002) 2023-11-26T07:23:40.546 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:40.543+0000 7fae5ad9d640 1 -- 172.21.15.143:0/1095129109 learned_addr learned my addr 172.21.15.143:0/1095129109 (peer_addr_for_me v1:172.21.15.143:0/0) 2023-11-26T07:23:40.546 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:40.543+0000 7fae5a59c640 1 --2- 172.21.15.143:0/1095129109 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fae5c15d6e0 0x7fae5c0d0df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:23:40.546 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:40.543+0000 7fae59d9b640 1 -- 172.21.15.143:0/1095129109 <== mon.0 v1:172.21.15.143:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2666523207 0 0) 0x7fae5c0cf0e0 con 0x7fae5c0d1bc0 2023-11-26T07:23:40.547 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:40.543+0000 7fae59d9b640 1 -- 172.21.15.143:0/1095129109 --> v1:172.21.15.143:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fae48003530 con 0x7fae5c0d1bc0 2023-11-26T07:23:40.547 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:40.543+0000 7fae5a59c640 1 -- 172.21.15.143:0/1095129109 >> v1:172.21.15.188:6789/0 conn(0x7fae5c0b2160 legacy=0x7fae5c0d14b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:23:40.547 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:40.543+0000 7fae59d9b640 1 -- 172.21.15.143:0/1095129109 <== mon.1 v1:172.21.15.188:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2810334673 0 0) 0x7fae5c0ceb20 con 0x7fae5c0b2160 2023-11-26T07:23:40.547 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:40.543+0000 7fae5a59c640 1 -- 172.21.15.143:0/1095129109 >> v1:172.21.15.143:6789/0 conn(0x7fae5c0d1bc0 legacy=0x7fae5c0b0750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:23:40.547 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:40.543+0000 7fae5a59c640 1 -- 172.21.15.143:0/1095129109 --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fae5c0cee00 con 0x7fae5c15d6e0 2023-11-26T07:23:40.549 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:40.547+0000 7fae5a59c640 1 --2- 172.21.15.143:0/1095129109 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fae5c15d6e0 0x7fae5c0d0df0 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7fae4c0097f0 tx=0x7fae4c01c4c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=2518f73effbd5e90 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:23:40.549 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:40.547+0000 7fae59d9b640 1 -- 172.21.15.143:0/1095129109 <== mon.2 v2:172.21.15.188:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fae4c0ac410 con 0x7fae5c15d6e0 2023-11-26T07:23:40.550 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:40.547+0000 7fae59d9b640 1 -- 172.21.15.143:0/1095129109 <== mon.2 v2:172.21.15.188:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fae4c0ac570 con 0x7fae5c15d6e0 2023-11-26T07:23:40.550 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:40.547+0000 7fae59d9b640 1 -- 172.21.15.143:0/1095129109 <== mon.2 v2:172.21.15.188:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fae4c0ac8e0 con 0x7fae5c15d6e0 2023-11-26T07:23:40.550 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:40.547+0000 7fae60b72640 1 -- 172.21.15.143:0/1095129109 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fae5c15d6e0 msgr2=0x7fae5c0d0df0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:23:40.551 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:40.547+0000 7fae60b72640 1 --2- 172.21.15.143:0/1095129109 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fae5c15d6e0 0x7fae5c0d0df0 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7fae4c0097f0 tx=0x7fae4c01c4c0 comp rx=0 tx=0).stop 2023-11-26T07:23:40.551 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:40.551+0000 7fae60b72640 1 -- 172.21.15.143:0/1095129109 shutdown_connections 2023-11-26T07:23:40.551 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:40.551+0000 7fae60b72640 1 --2- 172.21.15.143:0/1095129109 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fae5c15d6e0 0x7fae5c0d0df0 unknown :-1 s=CLOSED pgs=148 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:23:40.551 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:40.551+0000 7fae60b72640 1 -- 172.21.15.143:0/1095129109 >> 172.21.15.143:0/1095129109 conn(0x7fae5c09c550 msgr2=0x7fae5c0d3220 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:23:40.551 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:40.551+0000 7fae60b72640 1 -- 172.21.15.143:0/1095129109 shutdown_connections 2023-11-26T07:23:40.551 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:40.551+0000 7fae60b72640 1 -- 172.21.15.143:0/1095129109 wait complete. 2023-11-26T07:23:40.552 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:40.551+0000 7fae60b72640 1 Processor -- start 2023-11-26T07:23:40.552 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:40.551+0000 7fae60b72640 1 -- start start 2023-11-26T07:23:40.554 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:40.551+0000 7fae60b72640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fae5c0b2160 0x7fae5c0b7dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:23:40.554 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:40.551+0000 7fae60b72640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fae5c0d1bc0 0x7fae5c152920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:23:40.554 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:40.551+0000 7fae60b72640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fae5c15d6e0 0x7fae5c152e60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:23:40.555 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:40.551+0000 7fae60b72640 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fae5c0b6bc0 con 0x7fae5c0d1bc0 2023-11-26T07:23:40.555 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:40.551+0000 7fae60b72640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fae5c0b5fb0 con 0x7fae5c15d6e0 2023-11-26T07:23:40.555 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:40.551+0000 7fae60b72640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fae5c0b6290 con 0x7fae5c0b2160 2023-11-26T07:23:40.555 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:40.551+0000 7fae53fff640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fae5c0d1bc0 0x7fae5c152920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:23:40.555 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:40.551+0000 7fae53fff640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fae5c0d1bc0 0x7fae5c152920 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:55794/0 (socket says 172.21.15.143:55794) 2023-11-26T07:23:40.555 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:40.551+0000 7fae53fff640 1 -- 172.21.15.143:0/3640095257 learned_addr learned my addr 172.21.15.143:0/3640095257 (peer_addr_for_me v2:172.21.15.143:0/0) 2023-11-26T07:23:40.555 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:40.551+0000 7fae5a59c640 1 --2- 172.21.15.143:0/3640095257 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fae5c0b2160 0x7fae5c0b7dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:23:40.555 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:40.551+0000 7fae5ad9d640 1 --2- 172.21.15.143:0/3640095257 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fae5c15d6e0 0x7fae5c152e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:23:40.555 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:40.551+0000 7fae53fff640 1 -- 172.21.15.143:0/3640095257 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fae5c0b2160 msgr2=0x7fae5c0b7dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:23:40.555 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:40.551+0000 7fae53fff640 1 --2- 172.21.15.143:0/3640095257 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fae5c0b2160 0x7fae5c0b7dc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:23:40.555 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:40.551+0000 7fae53fff640 1 -- 172.21.15.143:0/3640095257 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fae5c15d6e0 msgr2=0x7fae5c152e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:23:40.555 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:40.551+0000 7fae53fff640 1 --2- 172.21.15.143:0/3640095257 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fae5c15d6e0 0x7fae5c152e60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:23:40.556 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:40.551+0000 7fae53fff640 1 -- 172.21.15.143:0/3640095257 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fae4c009420 con 0x7fae5c0d1bc0 2023-11-26T07:23:40.556 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:40.551+0000 7fae53fff640 1 --2- 172.21.15.143:0/3640095257 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fae5c0d1bc0 0x7fae5c152920 secure :-1 s=READY pgs=1184 cs=0 l=1 rev1=1 crypto rx=0x7fae4400c4d0 tx=0x7fae4400c990 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:23:40.556 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:40.555+0000 7fae52ffd640 1 -- 172.21.15.143:0/3640095257 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fae44005480 con 0x7fae5c0d1bc0 2023-11-26T07:23:40.556 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:40.555+0000 7fae52ffd640 1 -- 172.21.15.143:0/3640095257 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fae44003600 con 0x7fae5c0d1bc0 2023-11-26T07:23:40.556 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:40.555+0000 7fae52ffd640 1 -- 172.21.15.143:0/3640095257 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fae440039b0 con 0x7fae5c0d1bc0 2023-11-26T07:23:40.556 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:40.555+0000 7fae60b72640 1 -- 172.21.15.143:0/3640095257 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fae5c0b6570 con 0x7fae5c0d1bc0 2023-11-26T07:23:40.556 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:40.555+0000 7fae60b72640 1 -- 172.21.15.143:0/3640095257 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fae5c0b8300 con 0x7fae5c0d1bc0 2023-11-26T07:23:40.556 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:40.555+0000 7fae60b72640 1 -- 172.21.15.143:0/3640095257 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fae5c0b6d30 con 0x7fae5c0d1bc0 2023-11-26T07:23:40.557 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:40.555+0000 7fae52ffd640 1 -- 172.21.15.143:0/3640095257 <== mon.0 v2:172.21.15.143:3300/0 4 ==== mgrmap(e 16) v1 ==== 85954+0+0 (secure 0 0 0) 0x7fae44003b10 con 0x7fae5c0d1bc0 2023-11-26T07:23:40.557 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:40.555+0000 7fae52ffd640 1 --2- 172.21.15.143:0/3640095257 >> [v2:172.21.15.188:6836/3401128553,v1:172.21.15.188:6837/3401128553] conn(0x7fae3c066590 0x7fae3c068a50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:23:40.558 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:40.555+0000 7fae52ffd640 1 -- 172.21.15.143:0/3640095257 <== mon.0 v2:172.21.15.143:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fae44083560 con 0x7fae5c0d1bc0 2023-11-26T07:23:40.558 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:40.555+0000 7fae5a59c640 1 --2- 172.21.15.143:0/3640095257 >> [v2:172.21.15.188:6836/3401128553,v1:172.21.15.188:6837/3401128553] conn(0x7fae3c066590 0x7fae3c068a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:23:40.558 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:40.559+0000 7fae5a59c640 1 --2- 172.21.15.143:0/3640095257 >> [v2:172.21.15.188:6836/3401128553,v1:172.21.15.188:6837/3401128553] conn(0x7fae3c066590 0x7fae3c068a50 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7fae4c0ae000 tx=0x7fae4c0a6040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:23:40.561 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:40.559+0000 7fae52ffd640 1 -- 172.21.15.143:0/3640095257 <== mon.0 v2:172.21.15.143:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172649 (secure 0 0 0) 0x7fae44050a20 con 0x7fae5c0d1bc0 2023-11-26T07:23:40.732 INFO:teuthology.orchestra.run.smithi143.stderr:Invalid command: invalid chars @# in abcd@# 2023-11-26T07:23:40.733 INFO:teuthology.orchestra.run.smithi143.stderr:fs volume create [] : Create a CephFS volume 2023-11-26T07:23:40.733 INFO:teuthology.orchestra.run.smithi143.stderr:Error EINVAL: invalid command 2023-11-26T07:23:40.739 DEBUG:teuthology.orchestra.run:got remote process result: 22 2023-11-26T07:23:40.740 DEBUG:teuthology.orchestra.run.smithi143:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs volume create testvol 2023-11-26T07:23:40.879 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:40.879+0000 7fce13135640 1 Processor -- start 2023-11-26T07:23:40.879 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:40.879+0000 7fce13135640 1 -- start start 2023-11-26T07:23:40.880 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:40.879+0000 7fce13135640 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7fce0c143ac0 0x7fce0c070370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:23:40.881 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:40.879+0000 7fce13135640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fce0c0708b0 0x7fce0c0cf280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:23:40.881 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:40.879+0000 7fce13135640 1 -- --> v1:172.21.15.143:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fce0c0cfbf0 con 0x7fce0c0cf7c0 2023-11-26T07:23:40.881 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:40.879+0000 7fce13135640 1 -- --> v2:172.21.15.188:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fce0c143ea0 con 0x7fce0c143ac0 2023-11-26T07:23:40.881 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:40.879+0000 7fce13135640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fce0c0cc730 con 0x7fce0c0708b0 2023-11-26T07:23:40.881 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:40.879+0000 7fce116d2640 1 --1- >> v1:172.21.15.143:6789/0 conn(0x7fce0c0cf7c0 0x7fce0c0b1ef0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.143:6789/0 says I am v1:172.21.15.143:48010/0 (socket says 172.21.15.143:48010) 2023-11-26T07:23:40.881 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:40.879+0000 7fce116d2640 1 -- 172.21.15.143:0/2474278771 learned_addr learned my addr 172.21.15.143:0/2474278771 (peer_addr_for_me v1:172.21.15.143:0/0) 2023-11-26T07:23:40.881 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:40.879+0000 7fce0bfff640 1 --2- 172.21.15.143:0/2474278771 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fce0c0708b0 0x7fce0c0cf280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:23:40.881 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:40.879+0000 7fce10ed1640 1 --2- 172.21.15.143:0/2474278771 >> v2:172.21.15.188:3300/0 conn(0x7fce0c143ac0 0x7fce0c070370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:23:40.882 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:40.879+0000 7fce0b7fe640 1 -- 172.21.15.143:0/2474278771 <== mon.0 v1:172.21.15.143:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4289039874 0 0) 0x7fce0c0cfbf0 con 0x7fce0c0cf7c0 2023-11-26T07:23:40.882 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:40.879+0000 7fce0b7fe640 1 -- 172.21.15.143:0/2474278771 --> v1:172.21.15.143:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fcdfc003530 con 0x7fce0c0cf7c0 2023-11-26T07:23:40.882 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:40.879+0000 7fce10ed1640 1 -- 172.21.15.143:0/2474278771 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fce0c0708b0 msgr2=0x7fce0c0cf280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-26T07:23:40.883 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:40.879+0000 7fce10ed1640 1 --2- 172.21.15.143:0/2474278771 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fce0c0708b0 0x7fce0c0cf280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:23:40.883 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:40.879+0000 7fce10ed1640 1 -- 172.21.15.143:0/2474278771 >> v1:172.21.15.143:6789/0 conn(0x7fce0c0cf7c0 legacy=0x7fce0c0b1ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:23:40.883 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:40.879+0000 7fce10ed1640 1 -- 172.21.15.143:0/2474278771 --> v2:172.21.15.188:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fce0c070ee0 con 0x7fce0c143ac0 2023-11-26T07:23:40.883 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:40.879+0000 7fce0bfff640 1 --2- 172.21.15.143:0/2474278771 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fce0c0708b0 0x7fce0c0cf280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-26T07:23:40.884 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:40.883+0000 7fce10ed1640 1 --2- 172.21.15.143:0/2474278771 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fce0c143ac0 0x7fce0c070370 secure :-1 s=READY pgs=451 cs=0 l=1 rev1=1 crypto rx=0x7fce000097c0 tx=0x7fce0001c480 comp rx=0 tx=0).ready entity=mon.1 client_cookie=24463d2ced9c1cde server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:23:40.886 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:40.883+0000 7fce0b7fe640 1 -- 172.21.15.143:0/2474278771 <== mon.1 v2:172.21.15.188:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fce000ad640 con 0x7fce0c143ac0 2023-11-26T07:23:40.886 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:40.883+0000 7fce0b7fe640 1 -- 172.21.15.143:0/2474278771 <== mon.1 v2:172.21.15.188:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fce000ad7a0 con 0x7fce0c143ac0 2023-11-26T07:23:40.886 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:40.883+0000 7fce0b7fe640 1 -- 172.21.15.143:0/2474278771 <== mon.1 v2:172.21.15.188:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fce000adb30 con 0x7fce0c143ac0 2023-11-26T07:23:40.886 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:40.883+0000 7fce13135640 1 -- 172.21.15.143:0/2474278771 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fce0c143ac0 msgr2=0x7fce0c070370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:23:40.886 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:40.883+0000 7fce13135640 1 --2- 172.21.15.143:0/2474278771 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fce0c143ac0 0x7fce0c070370 secure :-1 s=READY pgs=451 cs=0 l=1 rev1=1 crypto rx=0x7fce000097c0 tx=0x7fce0001c480 comp rx=0 tx=0).stop 2023-11-26T07:23:40.886 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:40.883+0000 7fce13135640 1 -- 172.21.15.143:0/2474278771 shutdown_connections 2023-11-26T07:23:40.886 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:40.883+0000 7fce13135640 1 --2- 172.21.15.143:0/2474278771 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fce0c0708b0 0x7fce0c0cf280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:23:40.886 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:40.883+0000 7fce13135640 1 --2- 172.21.15.143:0/2474278771 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fce0c143ac0 0x7fce0c070370 unknown :-1 s=CLOSED pgs=451 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:23:40.887 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:40.883+0000 7fce13135640 1 -- 172.21.15.143:0/2474278771 >> 172.21.15.143:0/2474278771 conn(0x7fce0c06a890 msgr2=0x7fce0c06aca0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:23:40.887 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:40.883+0000 7fce13135640 1 -- 172.21.15.143:0/2474278771 shutdown_connections 2023-11-26T07:23:40.887 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:40.883+0000 7fce13135640 1 -- 172.21.15.143:0/2474278771 wait complete. 2023-11-26T07:23:40.887 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:40.887+0000 7fce13135640 1 Processor -- start 2023-11-26T07:23:40.888 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:40.887+0000 7fce13135640 1 -- start start 2023-11-26T07:23:40.889 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:40.887+0000 7fce13135640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fce0c0708b0 0x7fce0c139940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:23:40.889 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:40.887+0000 7fce13135640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fce0c0cf7c0 0x7fce0c142420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:23:40.889 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:40.887+0000 7fce13135640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fce0c143ac0 0x7fce0c142960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:23:40.890 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:40.887+0000 7fce13135640 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fce0c142ea0 con 0x7fce0c0cf7c0 2023-11-26T07:23:40.890 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:40.887+0000 7fce13135640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fce0c0afef0 con 0x7fce0c143ac0 2023-11-26T07:23:40.890 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:40.887+0000 7fce13135640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fce0c0af5c0 con 0x7fce0c0708b0 2023-11-26T07:23:40.890 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:40.887+0000 7fce0bfff640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fce0c0cf7c0 0x7fce0c142420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:23:40.890 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:40.887+0000 7fce0bfff640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fce0c0cf7c0 0x7fce0c142420 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:55808/0 (socket says 172.21.15.143:55808) 2023-11-26T07:23:40.890 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:40.887+0000 7fce0bfff640 1 -- 172.21.15.143:0/919453518 learned_addr learned my addr 172.21.15.143:0/919453518 (peer_addr_for_me v2:172.21.15.143:0/0) 2023-11-26T07:23:40.890 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:40.887+0000 7fce10ed1640 1 --2- 172.21.15.143:0/919453518 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fce0c0708b0 0x7fce0c139940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:23:40.890 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:40.887+0000 7fce116d2640 1 --2- 172.21.15.143:0/919453518 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fce0c143ac0 0x7fce0c142960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:23:40.890 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:40.887+0000 7fce0bfff640 1 -- 172.21.15.143:0/919453518 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fce0c0708b0 msgr2=0x7fce0c139940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:23:40.890 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:40.887+0000 7fce0bfff640 1 --2- 172.21.15.143:0/919453518 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fce0c0708b0 0x7fce0c139940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:23:40.890 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:40.887+0000 7fce0bfff640 1 -- 172.21.15.143:0/919453518 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fce0c143ac0 msgr2=0x7fce0c142960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:23:40.891 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:40.887+0000 7fce0bfff640 1 --2- 172.21.15.143:0/919453518 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fce0c143ac0 0x7fce0c142960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:23:40.891 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:40.887+0000 7fce0bfff640 1 -- 172.21.15.143:0/919453518 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcdf8009420 con 0x7fce0c0cf7c0 2023-11-26T07:23:40.891 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:40.887+0000 7fce0bfff640 1 --2- 172.21.15.143:0/919453518 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fce0c0cf7c0 0x7fce0c142420 secure :-1 s=READY pgs=1186 cs=0 l=1 rev1=1 crypto rx=0x7fcdf800e980 tx=0x7fcdf800ee40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:23:40.892 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:40.887+0000 7fce097fa640 1 -- 172.21.15.143:0/919453518 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcdf800ca00 con 0x7fce0c0cf7c0 2023-11-26T07:23:40.892 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:40.887+0000 7fce097fa640 1 -- 172.21.15.143:0/919453518 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcdf800cb60 con 0x7fce0c0cf7c0 2023-11-26T07:23:40.892 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:40.887+0000 7fce097fa640 1 -- 172.21.15.143:0/919453518 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcdf8013470 con 0x7fce0c0cf7c0 2023-11-26T07:23:40.892 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:40.887+0000 7fce13135640 1 -- 172.21.15.143:0/919453518 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fce00009420 con 0x7fce0c0cf7c0 2023-11-26T07:23:40.892 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:40.887+0000 7fce13135640 1 -- 172.21.15.143:0/919453518 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fce0c0afb80 con 0x7fce0c0cf7c0 2023-11-26T07:23:40.892 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:40.891+0000 7fce13135640 1 -- 172.21.15.143:0/919453518 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fce0c0b00a0 con 0x7fce0c0cf7c0 2023-11-26T07:23:40.892 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:40.891+0000 7fce097fa640 1 -- 172.21.15.143:0/919453518 <== mon.0 v2:172.21.15.143:3300/0 4 ==== mgrmap(e 16) v1 ==== 85954+0+0 (secure 0 0 0) 0x7fcdf80135d0 con 0x7fce0c0cf7c0 2023-11-26T07:23:40.893 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:40.891+0000 7fce097fa640 1 --2- 172.21.15.143:0/919453518 >> [v2:172.21.15.188:6836/3401128553,v1:172.21.15.188:6837/3401128553] conn(0x7fcdf0066590 0x7fcdf0068a50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:23:40.893 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:40.891+0000 7fce097fa640 1 -- 172.21.15.143:0/919453518 <== mon.0 v2:172.21.15.143:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fcdf80842a0 con 0x7fce0c0cf7c0 2023-11-26T07:23:40.893 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:40.891+0000 7fce10ed1640 1 --2- 172.21.15.143:0/919453518 >> [v2:172.21.15.188:6836/3401128553,v1:172.21.15.188:6837/3401128553] conn(0x7fcdf0066590 0x7fcdf0068a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:23:40.893 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:40.891+0000 7fce10ed1640 1 --2- 172.21.15.143:0/919453518 >> [v2:172.21.15.188:6836/3401128553,v1:172.21.15.188:6837/3401128553] conn(0x7fcdf0066590 0x7fcdf0068a50 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7fce000af000 tx=0x7fce00008040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:23:40.896 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:40.895+0000 7fce097fa640 1 -- 172.21.15.143:0/919453518 <== mon.0 v2:172.21.15.143:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172649 (secure 0 0 0) 0x7fce0c0b00a0 con 0x7fce0c0cf7c0 2023-11-26T07:23:41.068 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:41.067+0000 7fce13135640 1 -- 172.21.15.143:0/919453518 --> [v2:172.21.15.188:6836/3401128553,v1:172.21.15.188:6837/3401128553] -- mgr_command(tid 0: {"prefix": "fs volume create", "name": "testvol", "target": ["mon-mgr", ""]}) v1 -- 0x7fce0c0af730 con 0x7fcdf0066590 2023-11-26T07:23:43.732 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:43.731+0000 7fce097fa640 1 -- 172.21.15.143:0/919453518 <== mgr.4108 v2:172.21.15.188:6836/3401128553 1 ==== mgr_command_reply(tid 0: 0 Volume created successfully (no MDS daemons created)) v1 ==== 60+0+0 (secure 0 0 0) 0x7fce0c0af730 con 0x7fcdf0066590 2023-11-26T07:23:43.732 INFO:teuthology.orchestra.run.smithi143.stderr:Volume created successfully (no MDS daemons created) 2023-11-26T07:23:43.736 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:43.735+0000 7fce13135640 1 -- 172.21.15.143:0/919453518 >> [v2:172.21.15.188:6836/3401128553,v1:172.21.15.188:6837/3401128553] conn(0x7fcdf0066590 msgr2=0x7fcdf0068a50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:23:43.736 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:43.735+0000 7fce13135640 1 --2- 172.21.15.143:0/919453518 >> [v2:172.21.15.188:6836/3401128553,v1:172.21.15.188:6837/3401128553] conn(0x7fcdf0066590 0x7fcdf0068a50 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7fce000af000 tx=0x7fce00008040 comp rx=0 tx=0).stop 2023-11-26T07:23:43.736 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:43.735+0000 7fce13135640 1 -- 172.21.15.143:0/919453518 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fce0c0cf7c0 msgr2=0x7fce0c142420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:23:43.737 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:43.735+0000 7fce13135640 1 --2- 172.21.15.143:0/919453518 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fce0c0cf7c0 0x7fce0c142420 secure :-1 s=READY pgs=1186 cs=0 l=1 rev1=1 crypto rx=0x7fcdf800e980 tx=0x7fcdf800ee40 comp rx=0 tx=0).stop 2023-11-26T07:23:43.737 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:43.735+0000 7fce13135640 1 -- 172.21.15.143:0/919453518 shutdown_connections 2023-11-26T07:23:43.737 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:43.735+0000 7fce13135640 1 --2- 172.21.15.143:0/919453518 >> [v2:172.21.15.188:6836/3401128553,v1:172.21.15.188:6837/3401128553] conn(0x7fcdf0066590 0x7fcdf0068a50 unknown :-1 s=CLOSED pgs=74 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:23:43.738 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:43.735+0000 7fce13135640 1 --2- 172.21.15.143:0/919453518 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fce0c143ac0 0x7fce0c142960 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:23:43.738 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:43.735+0000 7fce13135640 1 --2- 172.21.15.143:0/919453518 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fce0c0cf7c0 0x7fce0c142420 unknown :-1 s=CLOSED pgs=1186 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:23:43.738 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:43.735+0000 7fce13135640 1 --2- 172.21.15.143:0/919453518 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fce0c0708b0 0x7fce0c139940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:23:43.738 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:43.735+0000 7fce13135640 1 -- 172.21.15.143:0/919453518 >> 172.21.15.143:0/919453518 conn(0x7fce0c06a890 msgr2=0x7fce0c0b3930 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:23:43.738 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:43.735+0000 7fce13135640 1 -- 172.21.15.143:0/919453518 shutdown_connections 2023-11-26T07:23:43.738 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:43.735+0000 7fce13135640 1 -- 172.21.15.143:0/919453518 wait complete. 2023-11-26T07:23:43.767 DEBUG:teuthology.orchestra.run.smithi143:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolumegroup create testvol 'abcd@#' 2023-11-26T07:23:43.911 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:43.911+0000 7f85dcfc1640 1 Processor -- start 2023-11-26T07:23:43.911 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:43.911+0000 7f85dcfc1640 1 -- start start 2023-11-26T07:23:43.912 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:43.911+0000 7f85dcfc1640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f85d813e290 0x7f85d80cf280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:23:43.912 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:43.911+0000 7f85dcfc1640 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7f85d80cf7c0 0x7f85d815efa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:23:43.912 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:43.911+0000 7f85dcfc1640 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7f85d815f4e0 0x7f85d81408e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:23:43.912 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:43.911+0000 7f85dcfc1640 1 -- --> v2:172.21.15.143:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f85d80cfc40 con 0x7f85d80cf7c0 2023-11-26T07:23:43.913 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:43.911+0000 7f85dcfc1640 1 -- --> v2:172.21.15.188:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f85d806c520 con 0x7f85d815f4e0 2023-11-26T07:23:43.913 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:43.911+0000 7f85dcfc1640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f85d806c800 con 0x7f85d813e290 2023-11-26T07:23:43.913 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:43.911+0000 7f85d5d9b640 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7f85d80cf7c0 0x7f85d815efa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:23:43.913 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:43.911+0000 7f85d5d9b640 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7f85d80cf7c0 0x7f85d815efa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:55820/0 (socket says 172.21.15.143:55820) 2023-11-26T07:23:43.913 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:43.911+0000 7f85d5d9b640 1 -- 172.21.15.143:0/4128468733 learned_addr learned my addr 172.21.15.143:0/4128468733 (peer_addr_for_me v2:172.21.15.143:0/0) 2023-11-26T07:23:43.914 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:43.911+0000 7f85d6d9d640 1 --2- 172.21.15.143:0/4128468733 >> v2:172.21.15.188:3300/0 conn(0x7f85d815f4e0 0x7f85d81408e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:23:43.914 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:43.911+0000 7f85d659c640 1 --2- 172.21.15.143:0/4128468733 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f85d813e290 0x7f85d80cf280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:23:43.914 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:43.911+0000 7f85d5d9b640 1 -- 172.21.15.143:0/4128468733 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f85d813e290 msgr2=0x7f85d80cf280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-26T07:23:43.914 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:43.911+0000 7f85d5d9b640 1 --2- 172.21.15.143:0/4128468733 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f85d813e290 0x7f85d80cf280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:23:43.914 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:43.911+0000 7f85d5d9b640 1 -- 172.21.15.143:0/4128468733 >> v2:172.21.15.188:3300/0 conn(0x7f85d815f4e0 msgr2=0x7f85d81408e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-26T07:23:43.914 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:43.911+0000 7f85d5d9b640 1 --2- 172.21.15.143:0/4128468733 >> v2:172.21.15.188:3300/0 conn(0x7f85d815f4e0 0x7f85d81408e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:23:43.914 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:43.911+0000 7f85d5d9b640 1 -- 172.21.15.143:0/4128468733 --> v2:172.21.15.143:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f85d806cae0 con 0x7f85d80cf7c0 2023-11-26T07:23:43.915 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:43.915+0000 7f85d5d9b640 1 --2- 172.21.15.143:0/4128468733 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f85d80cf7c0 0x7f85d815efa0 secure :-1 s=READY pgs=1187 cs=0 l=1 rev1=1 crypto rx=0x7f85d8072660 tx=0x7f85cc01c210 comp rx=0 tx=0).ready entity=mon.0 client_cookie=72109b04f8221bb1 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:23:43.916 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:43.915+0000 7f85d559a640 1 -- 172.21.15.143:0/4128468733 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f85cc0ab600 con 0x7f85d80cf7c0 2023-11-26T07:23:43.916 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:43.915+0000 7f85d559a640 1 -- 172.21.15.143:0/4128468733 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f85cc0a7050 con 0x7f85d80cf7c0 2023-11-26T07:23:43.916 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:43.915+0000 7f85d559a640 1 -- 172.21.15.143:0/4128468733 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f85cc0ab980 con 0x7f85d80cf7c0 2023-11-26T07:23:43.917 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:43.915+0000 7f85dcfc1640 1 -- 172.21.15.143:0/4128468733 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f85d80cf7c0 msgr2=0x7f85d815efa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:23:43.917 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:43.915+0000 7f85dcfc1640 1 --2- 172.21.15.143:0/4128468733 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f85d80cf7c0 0x7f85d815efa0 secure :-1 s=READY pgs=1187 cs=0 l=1 rev1=1 crypto rx=0x7f85d8072660 tx=0x7f85cc01c210 comp rx=0 tx=0).stop 2023-11-26T07:23:43.917 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:43.915+0000 7f85dcfc1640 1 -- 172.21.15.143:0/4128468733 shutdown_connections 2023-11-26T07:23:43.917 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:43.915+0000 7f85dcfc1640 1 --2- 172.21.15.143:0/4128468733 >> v2:172.21.15.188:3300/0 conn(0x7f85d815f4e0 0x7f85d81408e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:23:43.917 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:43.915+0000 7f85dcfc1640 1 --2- 172.21.15.143:0/4128468733 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f85d80cf7c0 0x7f85d815efa0 unknown :-1 s=CLOSED pgs=1187 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:23:43.917 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:43.915+0000 7f85dcfc1640 1 --2- 172.21.15.143:0/4128468733 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f85d813e290 0x7f85d80cf280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:23:43.917 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:43.915+0000 7f85dcfc1640 1 -- 172.21.15.143:0/4128468733 >> 172.21.15.143:0/4128468733 conn(0x7f85d806a890 msgr2=0x7f85d806aca0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:23:43.918 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:43.915+0000 7f85dcfc1640 1 -- 172.21.15.143:0/4128468733 shutdown_connections 2023-11-26T07:23:43.918 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:43.915+0000 7f85dcfc1640 1 -- 172.21.15.143:0/4128468733 wait complete. 2023-11-26T07:23:43.919 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:43.919+0000 7f85dcfc1640 1 Processor -- start 2023-11-26T07:23:43.919 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:43.919+0000 7f85dcfc1640 1 -- start start 2023-11-26T07:23:43.921 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:43.919+0000 7f85dcfc1640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f85d80cf7c0 0x7f85d81515d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:23:43.921 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:43.919+0000 7f85dcfc1640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f85d813e290 0x7f85d8139be0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:23:43.921 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:43.919+0000 7f85dcfc1640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f85d815f4e0 0x7f85d813a120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:23:43.921 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:43.919+0000 7f85dcfc1640 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f85d8154c00 con 0x7f85d80cf7c0 2023-11-26T07:23:43.921 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:43.919+0000 7f85dcfc1640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f85d81439d0 con 0x7f85d813e290 2023-11-26T07:23:43.921 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:43.919+0000 7f85dcfc1640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f85d8143cb0 con 0x7f85d815f4e0 2023-11-26T07:23:43.922 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:43.919+0000 7f85d659c640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f85d80cf7c0 0x7f85d81515d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:23:43.922 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:43.919+0000 7f85d659c640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f85d80cf7c0 0x7f85d81515d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:55836/0 (socket says 172.21.15.143:55836) 2023-11-26T07:23:43.922 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:43.919+0000 7f85d659c640 1 -- 172.21.15.143:0/1844531835 learned_addr learned my addr 172.21.15.143:0/1844531835 (peer_addr_for_me v2:172.21.15.143:0/0) 2023-11-26T07:23:43.922 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:43.919+0000 7f85d6d9d640 1 --2- 172.21.15.143:0/1844531835 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f85d815f4e0 0x7f85d813a120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:23:43.922 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:43.919+0000 7f85d5d9b640 1 --2- 172.21.15.143:0/1844531835 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f85d813e290 0x7f85d8139be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:23:43.922 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:43.919+0000 7f85d659c640 1 -- 172.21.15.143:0/1844531835 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f85d815f4e0 msgr2=0x7f85d813a120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:23:43.922 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:43.919+0000 7f85d659c640 1 --2- 172.21.15.143:0/1844531835 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f85d815f4e0 0x7f85d813a120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:23:43.922 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:43.919+0000 7f85d659c640 1 -- 172.21.15.143:0/1844531835 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f85d813e290 msgr2=0x7f85d8139be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:23:43.922 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:43.919+0000 7f85d659c640 1 --2- 172.21.15.143:0/1844531835 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f85d813e290 0x7f85d8139be0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:23:43.922 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:43.919+0000 7f85d659c640 1 -- 172.21.15.143:0/1844531835 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f85cc0094d0 con 0x7f85d80cf7c0 2023-11-26T07:23:43.922 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:43.919+0000 7f85d5d9b640 1 --2- 172.21.15.143:0/1844531835 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f85d813e290 0x7f85d8139be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-26T07:23:43.922 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:43.919+0000 7f85d659c640 1 --2- 172.21.15.143:0/1844531835 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f85d80cf7c0 0x7f85d81515d0 secure :-1 s=READY pgs=1188 cs=0 l=1 rev1=1 crypto rx=0x7f85c800d800 tx=0x7f85c800dcc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:23:43.923 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:43.919+0000 7f85c6ffd640 1 -- 172.21.15.143:0/1844531835 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f85c80096a0 con 0x7f85d80cf7c0 2023-11-26T07:23:43.923 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:43.919+0000 7f85c6ffd640 1 -- 172.21.15.143:0/1844531835 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f85c8009800 con 0x7f85d80cf7c0 2023-11-26T07:23:43.923 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:43.919+0000 7f85c6ffd640 1 -- 172.21.15.143:0/1844531835 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f85c8009b70 con 0x7f85d80cf7c0 2023-11-26T07:23:43.923 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:43.919+0000 7f85dcfc1640 1 -- 172.21.15.143:0/1844531835 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f85d8143f90 con 0x7f85d80cf7c0 2023-11-26T07:23:43.923 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:43.919+0000 7f85dcfc1640 1 -- 172.21.15.143:0/1844531835 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f85d8144790 con 0x7f85d80cf7c0 2023-11-26T07:23:43.923 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:43.919+0000 7f85dcfc1640 1 -- 172.21.15.143:0/1844531835 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f85d8067680 con 0x7f85d80cf7c0 2023-11-26T07:23:43.924 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:43.923+0000 7f85c6ffd640 1 -- 172.21.15.143:0/1844531835 <== mon.0 v2:172.21.15.143:3300/0 4 ==== mgrmap(e 16) v1 ==== 85954+0+0 (secure 0 0 0) 0x7f85c8009cd0 con 0x7f85d80cf7c0 2023-11-26T07:23:43.926 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:43.923+0000 7f85c6ffd640 1 --2- 172.21.15.143:0/1844531835 >> [v2:172.21.15.188:6836/3401128553,v1:172.21.15.188:6837/3401128553] conn(0x7f85b8066590 0x7f85b8068a50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:23:43.926 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:43.923+0000 7f85c6ffd640 1 -- 172.21.15.143:0/1844531835 <== mon.0 v2:172.21.15.143:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 6543+0+0 (secure 0 0 0) 0x7f85c8084eb0 con 0x7f85d80cf7c0 2023-11-26T07:23:43.927 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:43.923+0000 7f85d5d9b640 1 --2- 172.21.15.143:0/1844531835 >> [v2:172.21.15.188:6836/3401128553,v1:172.21.15.188:6837/3401128553] conn(0x7f85b8066590 0x7f85b8068a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:23:43.927 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:43.923+0000 7f85d5d9b640 1 --2- 172.21.15.143:0/1844531835 >> [v2:172.21.15.188:6836/3401128553,v1:172.21.15.188:6837/3401128553] conn(0x7f85b8066590 0x7f85b8068a50 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7f85cc0ad000 tx=0x7f85cc008040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:23:43.928 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:43.927+0000 7f85c6ffd640 1 -- 172.21.15.143:0/1844531835 <== mon.0 v2:172.21.15.143:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172649 (secure 0 0 0) 0x7f85c8051f50 con 0x7f85d80cf7c0 2023-11-26T07:23:44.098 INFO:teuthology.orchestra.run.smithi143.stderr:Invalid command: invalid chars @# in abcd@# 2023-11-26T07:23:44.098 INFO:teuthology.orchestra.run.smithi143.stderr:fs subvolumegroup create [] [] [] [] [] : Create a CephFS subvolume group in a volume, and optionally, with a specific data pool layout, and a specific numeric mode 2023-11-26T07:23:44.099 INFO:teuthology.orchestra.run.smithi143.stderr:Error EINVAL: invalid command 2023-11-26T07:23:44.105 DEBUG:teuthology.orchestra.run:got remote process result: 22 2023-11-26T07:23:44.106 DEBUG:teuthology.orchestra.run.smithi143:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume create testvol 'abcd@#' 2023-11-26T07:23:44.245 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:44.243+0000 7fbf3faea640 1 Processor -- start 2023-11-26T07:23:44.245 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:44.243+0000 7fbf3faea640 1 -- start start 2023-11-26T07:23:44.246 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:44.243+0000 7fbf3faea640 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7fbf380ce7c0 0x7fbf3815f6b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:23:44.246 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:44.243+0000 7fbf3faea640 1 -- --> v1:172.21.15.143:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbf3806c520 con 0x7fbf380cf990 2023-11-26T07:23:44.246 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:44.243+0000 7fbf3faea640 1 -- --> v1:172.21.15.188:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbf3806c800 con 0x7fbf3813e290 2023-11-26T07:23:44.247 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:44.243+0000 7fbf3faea640 1 -- --> v2:172.21.15.143:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fbf3806cae0 con 0x7fbf380ce7c0 2023-11-26T07:23:44.247 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:44.247+0000 7fbf3d085640 1 --1- >> v1:172.21.15.143:6789/0 conn(0x7fbf380cf990 0x7fbf3815efa0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.143:6789/0 says I am v1:172.21.15.143:48020/0 (socket says 172.21.15.143:48020) 2023-11-26T07:23:44.247 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:44.247+0000 7fbf3e087640 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7fbf380ce7c0 0x7fbf3815f6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:23:44.247 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:44.247+0000 7fbf3d085640 1 -- 172.21.15.143:0/231143197 learned_addr learned my addr 172.21.15.143:0/231143197 (peer_addr_for_me v1:172.21.15.143:0/0) 2023-11-26T07:23:44.248 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:44.247+0000 7fbf3e087640 1 -- 172.21.15.143:0/231143197 >> v1:172.21.15.188:6789/0 conn(0x7fbf3813e290 legacy=0x7fbf380cf280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:23:44.248 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:44.247+0000 7fbf3e087640 1 -- 172.21.15.143:0/231143197 >> v1:172.21.15.143:6789/0 conn(0x7fbf380cf990 legacy=0x7fbf3815efa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:23:44.248 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:44.247+0000 7fbf3c884640 1 -- 172.21.15.143:0/231143197 <== mon.0 v1:172.21.15.143:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3011000451 0 0) 0x7fbf3806c520 con 0x7fbf380cf990 2023-11-26T07:23:44.248 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:44.247+0000 7fbf3e087640 1 -- 172.21.15.143:0/231143197 --> v2:172.21.15.143:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbf3806cdc0 con 0x7fbf380ce7c0 2023-11-26T07:23:44.250 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:44.247+0000 7fbf3e087640 1 --2- 172.21.15.143:0/231143197 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fbf380ce7c0 0x7fbf3815f6b0 secure :-1 s=READY pgs=1190 cs=0 l=1 rev1=1 crypto rx=0x7fbf30009d90 tx=0x7fbf3001c890 comp rx=0 tx=0).ready entity=mon.0 client_cookie=47da9b6822b8745d server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:23:44.250 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:44.247+0000 7fbf3c884640 1 -- 172.21.15.143:0/231143197 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbf300a5970 con 0x7fbf380ce7c0 2023-11-26T07:23:44.250 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:44.247+0000 7fbf3c884640 1 -- 172.21.15.143:0/231143197 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbf300a5ad0 con 0x7fbf380ce7c0 2023-11-26T07:23:44.250 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:44.247+0000 7fbf3c884640 1 -- 172.21.15.143:0/231143197 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbf300a5e60 con 0x7fbf380ce7c0 2023-11-26T07:23:44.251 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:44.251+0000 7fbf3faea640 1 -- 172.21.15.143:0/231143197 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fbf380ce7c0 msgr2=0x7fbf3815f6b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:23:44.251 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:44.251+0000 7fbf3faea640 1 --2- 172.21.15.143:0/231143197 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fbf380ce7c0 0x7fbf3815f6b0 secure :-1 s=READY pgs=1190 cs=0 l=1 rev1=1 crypto rx=0x7fbf30009d90 tx=0x7fbf3001c890 comp rx=0 tx=0).stop 2023-11-26T07:23:44.251 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:44.251+0000 7fbf3faea640 1 -- 172.21.15.143:0/231143197 shutdown_connections 2023-11-26T07:23:44.251 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:44.251+0000 7fbf3faea640 1 --2- 172.21.15.143:0/231143197 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fbf380ce7c0 0x7fbf3815f6b0 unknown :-1 s=CLOSED pgs=1190 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:23:44.251 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:44.251+0000 7fbf3faea640 1 -- 172.21.15.143:0/231143197 >> 172.21.15.143:0/231143197 conn(0x7fbf3806a890 msgr2=0x7fbf3806aca0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:23:44.251 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:44.251+0000 7fbf3faea640 1 -- 172.21.15.143:0/231143197 shutdown_connections 2023-11-26T07:23:44.251 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:44.251+0000 7fbf3faea640 1 -- 172.21.15.143:0/231143197 wait complete. 2023-11-26T07:23:44.252 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:44.251+0000 7fbf3faea640 1 Processor -- start 2023-11-26T07:23:44.252 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:44.251+0000 7fbf3faea640 1 -- start start 2023-11-26T07:23:44.254 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:44.251+0000 7fbf3faea640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fbf380ce7c0 0x7fbf380b2f60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:23:44.255 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:44.251+0000 7fbf3faea640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fbf380cf990 0x7fbf3815c6e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:23:44.255 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:44.251+0000 7fbf3faea640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fbf3813e290 0x7fbf38137510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:23:44.255 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:44.251+0000 7fbf3faea640 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbf3815d0c0 con 0x7fbf380cf990 2023-11-26T07:23:44.255 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:44.251+0000 7fbf3faea640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbf3815d3a0 con 0x7fbf380ce7c0 2023-11-26T07:23:44.255 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:44.251+0000 7fbf3faea640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbf3815d680 con 0x7fbf3813e290 2023-11-26T07:23:44.255 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:44.251+0000 7fbf3d085640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fbf380cf990 0x7fbf3815c6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:23:44.255 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:44.251+0000 7fbf3d085640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fbf380cf990 0x7fbf3815c6e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:55844/0 (socket says 172.21.15.143:55844) 2023-11-26T07:23:44.255 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:44.251+0000 7fbf3d085640 1 -- 172.21.15.143:0/138187200 learned_addr learned my addr 172.21.15.143:0/138187200 (peer_addr_for_me v2:172.21.15.143:0/0) 2023-11-26T07:23:44.255 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:44.251+0000 7fbf3d886640 1 --2- 172.21.15.143:0/138187200 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fbf380ce7c0 0x7fbf380b2f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:23:44.255 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:44.251+0000 7fbf3e087640 1 --2- 172.21.15.143:0/138187200 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fbf3813e290 0x7fbf38137510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:23:44.255 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:44.251+0000 7fbf3d085640 1 -- 172.21.15.143:0/138187200 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fbf3813e290 msgr2=0x7fbf38137510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:23:44.256 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:44.251+0000 7fbf3d085640 1 --2- 172.21.15.143:0/138187200 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fbf3813e290 0x7fbf38137510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:23:44.256 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:44.251+0000 7fbf3d085640 1 -- 172.21.15.143:0/138187200 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fbf380ce7c0 msgr2=0x7fbf380b2f60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:23:44.256 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:44.251+0000 7fbf3d085640 1 --2- 172.21.15.143:0/138187200 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fbf380ce7c0 0x7fbf380b2f60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:23:44.256 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:44.251+0000 7fbf3d085640 1 -- 172.21.15.143:0/138187200 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbf30009a70 con 0x7fbf380cf990 2023-11-26T07:23:44.256 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:44.255+0000 7fbf3d085640 1 --2- 172.21.15.143:0/138187200 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fbf380cf990 0x7fbf3815c6e0 secure :-1 s=READY pgs=1191 cs=0 l=1 rev1=1 crypto rx=0x7fbf2800c4a0 tx=0x7fbf2800c960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:23:44.256 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:44.255+0000 7fbf2e7fc640 1 -- 172.21.15.143:0/138187200 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbf280074e0 con 0x7fbf380cf990 2023-11-26T07:23:44.256 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:44.255+0000 7fbf2e7fc640 1 -- 172.21.15.143:0/138187200 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbf28007640 con 0x7fbf380cf990 2023-11-26T07:23:44.256 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:44.255+0000 7fbf3faea640 1 -- 172.21.15.143:0/138187200 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbf3815d960 con 0x7fbf380cf990 2023-11-26T07:23:44.256 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:44.255+0000 7fbf2e7fc640 1 -- 172.21.15.143:0/138187200 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbf28004a30 con 0x7fbf380cf990 2023-11-26T07:23:44.256 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:44.255+0000 7fbf3faea640 1 -- 172.21.15.143:0/138187200 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbf3815e160 con 0x7fbf380cf990 2023-11-26T07:23:44.257 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:44.255+0000 7fbf3faea640 1 -- 172.21.15.143:0/138187200 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbf3815de80 con 0x7fbf380cf990 2023-11-26T07:23:44.257 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:44.255+0000 7fbf2e7fc640 1 -- 172.21.15.143:0/138187200 <== mon.0 v2:172.21.15.143:3300/0 4 ==== mgrmap(e 16) v1 ==== 85954+0+0 (secure 0 0 0) 0x7fbf28004b90 con 0x7fbf380cf990 2023-11-26T07:23:44.257 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:44.255+0000 7fbf2e7fc640 1 --2- 172.21.15.143:0/138187200 >> [v2:172.21.15.188:6836/3401128553,v1:172.21.15.188:6837/3401128553] conn(0x7fbf1c066590 0x7fbf1c068a50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:23:44.257 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:44.255+0000 7fbf2e7fc640 1 -- 172.21.15.143:0/138187200 <== mon.0 v2:172.21.15.143:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 6543+0+0 (secure 0 0 0) 0x7fbf28083870 con 0x7fbf380cf990 2023-11-26T07:23:44.258 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:44.255+0000 7fbf3d886640 1 --2- 172.21.15.143:0/138187200 >> [v2:172.21.15.188:6836/3401128553,v1:172.21.15.188:6837/3401128553] conn(0x7fbf1c066590 0x7fbf1c068a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:23:44.258 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:44.259+0000 7fbf3d886640 1 --2- 172.21.15.143:0/138187200 >> [v2:172.21.15.188:6836/3401128553,v1:172.21.15.188:6837/3401128553] conn(0x7fbf1c066590 0x7fbf1c068a50 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7fbf34000f70 tx=0x7fbf34009040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:23:44.261 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:44.259+0000 7fbf2e7fc640 1 -- 172.21.15.143:0/138187200 <== mon.0 v2:172.21.15.143:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172649 (secure 0 0 0) 0x7fbf3815de80 con 0x7fbf380cf990 2023-11-26T07:23:44.430 INFO:teuthology.orchestra.run.smithi143.stderr:Invalid command: invalid chars #@ in abcd@# 2023-11-26T07:23:44.430 INFO:teuthology.orchestra.run.smithi143.stderr:fs subvolume create [] [] [] [] [] [] [--namespace-isolated] : Create a CephFS subvolume in a volume, and optionally, with a specific size (in bytes), a specific data pool layout, a specific mode, in a specific subvolume group and in separate RADOS namespace 2023-11-26T07:23:44.430 INFO:teuthology.orchestra.run.smithi143.stderr:Error EINVAL: invalid command 2023-11-26T07:23:44.437 DEBUG:teuthology.orchestra.run:got remote process result: 22 2023-11-26T07:23:44.438 DEBUG:teuthology.orchestra.run.smithi143:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs volume rm testvol --yes-i-really-mean-it 2023-11-26T07:23:44.587 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:44.587+0000 7fab4432a640 1 Processor -- start 2023-11-26T07:23:44.587 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:44.587+0000 7fab4432a640 1 -- start start 2023-11-26T07:23:44.589 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:44.587+0000 7fab4432a640 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7fab3c143ac0 0x7fab3c070370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:23:44.590 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:44.587+0000 7fab4432a640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fab3c0cf990 0x7fab3c0cd1b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:23:44.590 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:44.587+0000 7fab4432a640 1 -- --> v1:172.21.15.143:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fab3c0cc730 con 0x7fab3c0708b0 2023-11-26T07:23:44.590 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:44.587+0000 7fab4432a640 1 -- --> v2:172.21.15.143:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fab3c143ea0 con 0x7fab3c143ac0 2023-11-26T07:23:44.590 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:44.587+0000 7fab4432a640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fab3c070ee0 con 0x7fab3c0cf990 2023-11-26T07:23:44.590 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:44.587+0000 7fab420c6640 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7fab3c143ac0 0x7fab3c070370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:23:44.590 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:44.587+0000 7fab418c5640 1 --1- >> v1:172.21.15.143:6789/0 conn(0x7fab3c0708b0 0x7fab3c0cf280 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.143:6789/0 says I am v1:172.21.15.143:46400/0 (socket says 172.21.15.143:46400) 2023-11-26T07:23:44.590 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:44.587+0000 7fab418c5640 1 -- 172.21.15.143:0/2826964185 learned_addr learned my addr 172.21.15.143:0/2826964185 (peer_addr_for_me v1:172.21.15.143:0/0) 2023-11-26T07:23:44.590 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:44.587+0000 7fab428c7640 1 --2- 172.21.15.143:0/2826964185 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fab3c0cf990 0x7fab3c0cd1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:23:44.590 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:44.587+0000 7fab420c6640 1 -- 172.21.15.143:0/2826964185 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fab3c0cf990 msgr2=0x7fab3c0cd1b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-26T07:23:44.591 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:44.587+0000 7fab420c6640 1 --2- 172.21.15.143:0/2826964185 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fab3c0cf990 0x7fab3c0cd1b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:23:44.591 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:44.587+0000 7fab420c6640 1 -- 172.21.15.143:0/2826964185 >> v1:172.21.15.143:6789/0 conn(0x7fab3c0708b0 legacy=0x7fab3c0cf280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:23:44.591 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:44.587+0000 7fab420c6640 1 -- 172.21.15.143:0/2826964185 --> v2:172.21.15.143:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fab3c0711c0 con 0x7fab3c143ac0 2023-11-26T07:23:44.591 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:44.587+0000 7fab410c4640 1 -- 172.21.15.143:0/2826964185 <== mon.0 v1:172.21.15.143:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2098302811 0 0) 0x7fab3c0cc730 con 0x7fab3c0708b0 2023-11-26T07:23:44.593 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:44.591+0000 7fab420c6640 1 --2- 172.21.15.143:0/2826964185 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fab3c143ac0 0x7fab3c070370 secure :-1 s=READY pgs=1193 cs=0 l=1 rev1=1 crypto rx=0x7fab380097c0 tx=0x7fab3801c480 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6a648528b133ec95 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:23:44.593 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:44.591+0000 7fab410c4640 1 -- 172.21.15.143:0/2826964185 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fab380ad640 con 0x7fab3c143ac0 2023-11-26T07:23:44.593 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:44.591+0000 7fab410c4640 1 -- 172.21.15.143:0/2826964185 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fab380ad7a0 con 0x7fab3c143ac0 2023-11-26T07:23:44.593 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:44.591+0000 7fab410c4640 1 -- 172.21.15.143:0/2826964185 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fab380adb30 con 0x7fab3c143ac0 2023-11-26T07:23:44.594 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:44.591+0000 7fab4432a640 1 -- 172.21.15.143:0/2826964185 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fab3c143ac0 msgr2=0x7fab3c070370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:23:44.594 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:44.591+0000 7fab4432a640 1 --2- 172.21.15.143:0/2826964185 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fab3c143ac0 0x7fab3c070370 secure :-1 s=READY pgs=1193 cs=0 l=1 rev1=1 crypto rx=0x7fab380097c0 tx=0x7fab3801c480 comp rx=0 tx=0).stop 2023-11-26T07:23:44.595 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:44.591+0000 7fab4432a640 1 -- 172.21.15.143:0/2826964185 shutdown_connections 2023-11-26T07:23:44.595 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:44.591+0000 7fab4432a640 1 --2- 172.21.15.143:0/2826964185 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fab3c0cf990 0x7fab3c0cd1b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:23:44.595 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:44.591+0000 7fab4432a640 1 --2- 172.21.15.143:0/2826964185 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fab3c143ac0 0x7fab3c070370 unknown :-1 s=CLOSED pgs=1193 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:23:44.595 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:44.591+0000 7fab4432a640 1 -- 172.21.15.143:0/2826964185 >> 172.21.15.143:0/2826964185 conn(0x7fab3c06a890 msgr2=0x7fab3c06aca0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:23:44.595 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:44.595+0000 7fab4432a640 1 -- 172.21.15.143:0/2826964185 shutdown_connections 2023-11-26T07:23:44.595 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:44.595+0000 7fab4432a640 1 -- 172.21.15.143:0/2826964185 wait complete. 2023-11-26T07:23:44.596 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:44.595+0000 7fab4432a640 1 Processor -- start 2023-11-26T07:23:44.596 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:44.595+0000 7fab4432a640 1 -- start start 2023-11-26T07:23:44.597 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:44.595+0000 7fab4432a640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fab3c0708b0 0x7fab3c0b0960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:23:44.597 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:44.595+0000 7fab4432a640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fab3c0cf990 0x7fab3c139ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:23:44.597 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:44.595+0000 7fab4432a640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fab3c143ac0 0x7fab3c15b360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:23:44.597 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:44.595+0000 7fab4432a640 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fab3c15c5b0 con 0x7fab3c143ac0 2023-11-26T07:23:44.597 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:44.595+0000 7fab4432a640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fab3c0b1830 con 0x7fab3c0cf990 2023-11-26T07:23:44.597 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:44.595+0000 7fab4432a640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fab3c0acaf0 con 0x7fab3c0708b0 2023-11-26T07:23:44.599 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:44.595+0000 7fab428c7640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fab3c143ac0 0x7fab3c15b360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:23:44.599 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:44.595+0000 7fab428c7640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fab3c143ac0 0x7fab3c15b360 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:36788/0 (socket says 172.21.15.143:36788) 2023-11-26T07:23:44.599 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:44.595+0000 7fab428c7640 1 -- 172.21.15.143:0/1184352842 learned_addr learned my addr 172.21.15.143:0/1184352842 (peer_addr_for_me v2:172.21.15.143:0/0) 2023-11-26T07:23:44.599 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:44.595+0000 7fab420c6640 1 --2- 172.21.15.143:0/1184352842 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fab3c0708b0 0x7fab3c0b0960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:23:44.599 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:44.595+0000 7fab418c5640 1 --2- 172.21.15.143:0/1184352842 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fab3c0cf990 0x7fab3c139ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:23:44.600 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:44.595+0000 7fab428c7640 1 -- 172.21.15.143:0/1184352842 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fab3c0708b0 msgr2=0x7fab3c0b0960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:23:44.600 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:44.595+0000 7fab428c7640 1 --2- 172.21.15.143:0/1184352842 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fab3c0708b0 0x7fab3c0b0960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:23:44.600 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:44.595+0000 7fab428c7640 1 -- 172.21.15.143:0/1184352842 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fab3c0cf990 msgr2=0x7fab3c139ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:23:44.600 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:44.595+0000 7fab428c7640 1 --2- 172.21.15.143:0/1184352842 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fab3c0cf990 0x7fab3c139ab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:23:44.600 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:44.595+0000 7fab428c7640 1 -- 172.21.15.143:0/1184352842 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fab38009420 con 0x7fab3c143ac0 2023-11-26T07:23:44.600 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:44.595+0000 7fab428c7640 1 --2- 172.21.15.143:0/1184352842 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fab3c143ac0 0x7fab3c15b360 secure :-1 s=READY pgs=1194 cs=0 l=1 rev1=1 crypto rx=0x7fab34007e70 tx=0x7fab3400e4b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:23:44.600 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:44.595+0000 7fab32ffd640 1 -- 172.21.15.143:0/1184352842 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fab34014620 con 0x7fab3c143ac0 2023-11-26T07:23:44.600 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:44.599+0000 7fab32ffd640 1 -- 172.21.15.143:0/1184352842 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fab340147a0 con 0x7fab3c143ac0 2023-11-26T07:23:44.600 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:44.599+0000 7fab32ffd640 1 -- 172.21.15.143:0/1184352842 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fab34014b50 con 0x7fab3c143ac0 2023-11-26T07:23:44.600 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:44.599+0000 7fab4432a640 1 -- 172.21.15.143:0/1184352842 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fab3c0b0f90 con 0x7fab3c143ac0 2023-11-26T07:23:44.600 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:44.599+0000 7fab4432a640 1 -- 172.21.15.143:0/1184352842 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fab3c0b1550 con 0x7fab3c143ac0 2023-11-26T07:23:44.601 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:44.599+0000 7fab4432a640 1 -- 172.21.15.143:0/1184352842 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fab3c0acca0 con 0x7fab3c143ac0 2023-11-26T07:23:44.601 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:44.599+0000 7fab32ffd640 1 -- 172.21.15.143:0/1184352842 <== mon.0 v2:172.21.15.143:3300/0 4 ==== mgrmap(e 16) v1 ==== 85954+0+0 (secure 0 0 0) 0x7fab34014cb0 con 0x7fab3c143ac0 2023-11-26T07:23:44.601 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:44.599+0000 7fab32ffd640 1 --2- 172.21.15.143:0/1184352842 >> [v2:172.21.15.188:6836/3401128553,v1:172.21.15.188:6837/3401128553] conn(0x7fab200665c0 0x7fab20068a80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:23:44.601 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:44.599+0000 7fab32ffd640 1 -- 172.21.15.143:0/1184352842 <== mon.0 v2:172.21.15.143:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 6543+0+0 (secure 0 0 0) 0x7fab34084c60 con 0x7fab3c143ac0 2023-11-26T07:23:44.602 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:44.599+0000 7fab420c6640 1 --2- 172.21.15.143:0/1184352842 >> [v2:172.21.15.188:6836/3401128553,v1:172.21.15.188:6837/3401128553] conn(0x7fab200665c0 0x7fab20068a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:23:44.602 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:44.603+0000 7fab420c6640 1 --2- 172.21.15.143:0/1184352842 >> [v2:172.21.15.188:6836/3401128553,v1:172.21.15.188:6837/3401128553] conn(0x7fab200665c0 0x7fab20068a80 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7fab380af000 tx=0x7fab38008040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:23:44.605 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:44.603+0000 7fab32ffd640 1 -- 172.21.15.143:0/1184352842 <== mon.0 v2:172.21.15.143:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172649 (secure 0 0 0) 0x7fab3c0acca0 con 0x7fab3c143ac0 2023-11-26T07:23:44.787 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:44.787+0000 7fab4432a640 1 -- 172.21.15.143:0/1184352842 --> [v2:172.21.15.188:6836/3401128553,v1:172.21.15.188:6837/3401128553] -- mgr_command(tid 0: {"prefix": "fs volume rm", "vol_name": "testvol", "yes-i-really-mean-it": "--yes-i-really-mean-it", "target": ["mon-mgr", ""]}) v1 -- 0x7fab3c0ac8e0 con 0x7fab200665c0 2023-11-26T07:23:44.792 INFO:tasks.ceph.mon.a.smithi143.stderr:2023-11-26T07:23:44.791+0000 7f6946f73640 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2023-11-26T07:23:44.798 INFO:tasks.ceph.mds.a.smithi143.stderr: -1166> 2023-11-26T07:23:23.739+0000 7f70d4121640 -1 PosixStack listen unable to listen on v2:0.0.0.0:6832/0: (98) Address already in use 2023-11-26T07:23:44.840 INFO:tasks.ceph.mds.a.smithi143.stdout:starting mds.a at 2023-11-26T07:23:47.820 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:47.819+0000 7fab32ffd640 1 -- 172.21.15.143:0/1184352842 <== mgr.4108 v2:172.21.15.188:6836/3401128553 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+78 (secure 0 0 0) 0x7fab3c0ac8e0 con 0x7fab200665c0 2023-11-26T07:23:47.820 INFO:teuthology.orchestra.run.smithi143.stdout:metadata pool: cephfs.testvol.meta data pool: ['cephfs.testvol.data'] removed 2023-11-26T07:23:47.823 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:47.823+0000 7fab4432a640 1 -- 172.21.15.143:0/1184352842 >> [v2:172.21.15.188:6836/3401128553,v1:172.21.15.188:6837/3401128553] conn(0x7fab200665c0 msgr2=0x7fab20068a80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:23:47.823 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:47.823+0000 7fab4432a640 1 --2- 172.21.15.143:0/1184352842 >> [v2:172.21.15.188:6836/3401128553,v1:172.21.15.188:6837/3401128553] conn(0x7fab200665c0 0x7fab20068a80 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7fab380af000 tx=0x7fab38008040 comp rx=0 tx=0).stop 2023-11-26T07:23:47.824 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:47.823+0000 7fab4432a640 1 -- 172.21.15.143:0/1184352842 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fab3c143ac0 msgr2=0x7fab3c15b360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:23:47.824 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:47.823+0000 7fab4432a640 1 --2- 172.21.15.143:0/1184352842 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fab3c143ac0 0x7fab3c15b360 secure :-1 s=READY pgs=1194 cs=0 l=1 rev1=1 crypto rx=0x7fab34007e70 tx=0x7fab3400e4b0 comp rx=0 tx=0).stop 2023-11-26T07:23:47.824 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:47.823+0000 7fab4432a640 1 -- 172.21.15.143:0/1184352842 shutdown_connections 2023-11-26T07:23:47.824 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:47.823+0000 7fab4432a640 1 --2- 172.21.15.143:0/1184352842 >> [v2:172.21.15.188:6836/3401128553,v1:172.21.15.188:6837/3401128553] conn(0x7fab200665c0 0x7fab20068a80 unknown :-1 s=CLOSED pgs=77 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:23:47.824 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:47.823+0000 7fab4432a640 1 --2- 172.21.15.143:0/1184352842 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fab3c143ac0 0x7fab3c15b360 unknown :-1 s=CLOSED pgs=1194 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:23:47.824 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:47.823+0000 7fab4432a640 1 --2- 172.21.15.143:0/1184352842 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fab3c0cf990 0x7fab3c139ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:23:47.824 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:47.823+0000 7fab4432a640 1 --2- 172.21.15.143:0/1184352842 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fab3c0708b0 0x7fab3c0b0960 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:23:47.824 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:47.823+0000 7fab4432a640 1 -- 172.21.15.143:0/1184352842 >> 172.21.15.143:0/1184352842 conn(0x7fab3c06a890 msgr2=0x7fab3c140350 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:23:47.824 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:47.823+0000 7fab4432a640 1 -- 172.21.15.143:0/1184352842 shutdown_connections 2023-11-26T07:23:47.824 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:47.823+0000 7fab4432a640 1 -- 172.21.15.143:0/1184352842 wait complete. 2023-11-26T07:23:47.846 INFO:teuthology.nuke.actions:Clearing teuthology firewall rules... 2023-11-26T07:23:47.846 DEBUG:teuthology.orchestra.run.smithi143:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2023-11-26T07:23:47.936 DEBUG:teuthology.orchestra.run.smithi188:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2023-11-26T07:23:48.026 INFO:teuthology.nuke.actions:Cleared teuthology firewall rules. 2023-11-26T07:23:48.027 DEBUG:teuthology.orchestra.run.smithi143:> set -ex 2023-11-26T07:23:48.027 DEBUG:teuthology.orchestra.run.smithi143:> dd if=/proc/self/mounts of=/dev/stdout 2023-11-26T07:23:48.036 DEBUG:teuthology.orchestra.run.smithi143:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-11-26T07:23:48.225 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:48.223+0000 7faecedaa640 1 Processor -- start 2023-11-26T07:23:48.226 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:48.223+0000 7faecedaa640 1 -- start start 2023-11-26T07:23:48.227 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:48.227+0000 7faecedaa640 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7faec8143ac0 0x7faec8070370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:23:48.227 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:48.227+0000 7faecedaa640 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7faec80cf990 0x7faec80cd1b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:23:48.227 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:48.227+0000 7faecedaa640 1 -- --> v1:172.21.15.188:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7faec80cc730 con 0x7faec80708b0 2023-11-26T07:23:48.227 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:48.227+0000 7faecedaa640 1 -- --> v2:172.21.15.143:3300/0 -- mon_getmap magic: 0 v1 -- 0x7faec8143ea0 con 0x7faec8143ac0 2023-11-26T07:23:48.227 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:48.227+0000 7faecedaa640 1 -- --> v2:172.21.15.188:3300/0 -- mon_getmap magic: 0 v1 -- 0x7faec8070ee0 con 0x7faec80cf990 2023-11-26T07:23:48.227 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:48.227+0000 7faeccb46640 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7faec8143ac0 0x7faec8070370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:23:48.228 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:48.227+0000 7faeccb46640 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7faec8143ac0 0x7faec8070370 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:36796/0 (socket says 172.21.15.143:36796) 2023-11-26T07:23:48.228 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:48.227+0000 7faeccb46640 1 -- 172.21.15.143:0/2629248307 learned_addr learned my addr 172.21.15.143:0/2629248307 (peer_addr_for_me v2:172.21.15.143:0/0) 2023-11-26T07:23:48.228 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:48.227+0000 7faecd347640 1 --2- 172.21.15.143:0/2629248307 >> v2:172.21.15.188:3300/0 conn(0x7faec80cf990 0x7faec80cd1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:23:48.228 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:48.227+0000 7faeccb46640 1 -- 172.21.15.143:0/2629248307 >> v2:172.21.15.188:3300/0 conn(0x7faec80cf990 msgr2=0x7faec80cd1b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-26T07:23:48.229 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:48.227+0000 7faeccb46640 1 --2- 172.21.15.143:0/2629248307 >> v2:172.21.15.188:3300/0 conn(0x7faec80cf990 0x7faec80cd1b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:23:48.229 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:48.227+0000 7faeccb46640 1 -- 172.21.15.143:0/2629248307 >> v1:172.21.15.188:6789/0 conn(0x7faec80708b0 legacy=0x7faec80cf280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-26T07:23:48.229 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:48.227+0000 7faeccb46640 1 -- 172.21.15.143:0/2629248307 --> v2:172.21.15.143:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7faec80711c0 con 0x7faec8143ac0 2023-11-26T07:23:48.230 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:48.231+0000 7faeccb46640 1 --2- 172.21.15.143:0/2629248307 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7faec8143ac0 0x7faec8070370 secure :-1 s=READY pgs=1197 cs=0 l=1 rev1=1 crypto rx=0x7faec806e570 tx=0x7faebc01c4c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=34e46617aa4442d8 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:23:48.231 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:48.231+0000 7faec77fe640 1 -- 172.21.15.143:0/2629248307 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faebc0aa620 con 0x7faec8143ac0 2023-11-26T07:23:48.231 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:48.231+0000 7faec77fe640 1 -- 172.21.15.143:0/2629248307 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7faebc0aa780 con 0x7faec8143ac0 2023-11-26T07:23:48.231 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:48.231+0000 7faec77fe640 1 -- 172.21.15.143:0/2629248307 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faebc0aab10 con 0x7faec8143ac0 2023-11-26T07:23:48.232 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:48.231+0000 7faecedaa640 1 -- 172.21.15.143:0/2629248307 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7faec8143ac0 msgr2=0x7faec8070370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:23:48.232 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:48.231+0000 7faecedaa640 1 --2- 172.21.15.143:0/2629248307 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7faec8143ac0 0x7faec8070370 secure :-1 s=READY pgs=1197 cs=0 l=1 rev1=1 crypto rx=0x7faec806e570 tx=0x7faebc01c4c0 comp rx=0 tx=0).stop 2023-11-26T07:23:48.232 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:48.231+0000 7faecedaa640 1 -- 172.21.15.143:0/2629248307 shutdown_connections 2023-11-26T07:23:48.232 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:48.231+0000 7faecedaa640 1 --2- 172.21.15.143:0/2629248307 >> v2:172.21.15.188:3300/0 conn(0x7faec80cf990 0x7faec80cd1b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:23:48.232 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:48.231+0000 7faecedaa640 1 --2- 172.21.15.143:0/2629248307 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7faec8143ac0 0x7faec8070370 unknown :-1 s=CLOSED pgs=1197 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:23:48.232 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:48.231+0000 7faecedaa640 1 -- 172.21.15.143:0/2629248307 >> 172.21.15.143:0/2629248307 conn(0x7faec806a890 msgr2=0x7faec806aca0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:23:48.232 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:48.231+0000 7faecedaa640 1 -- 172.21.15.143:0/2629248307 shutdown_connections 2023-11-26T07:23:48.232 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:48.231+0000 7faecedaa640 1 -- 172.21.15.143:0/2629248307 wait complete. 2023-11-26T07:23:48.233 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:48.231+0000 7faecedaa640 1 Processor -- start 2023-11-26T07:23:48.233 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:48.231+0000 7faecedaa640 1 -- start start 2023-11-26T07:23:48.235 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:48.235+0000 7faecedaa640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7faec80708b0 0x7faec813e1f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:23:48.235 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:48.235+0000 7faecedaa640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7faec80cf990 0x7faec815e9d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:23:48.235 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:48.235+0000 7faecedaa640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7faec8143ac0 0x7faec8140030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:23:48.235 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:48.235+0000 7faecedaa640 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faec80b0d50 con 0x7faec80708b0 2023-11-26T07:23:48.235 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:48.235+0000 7faecedaa640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faec80b1030 con 0x7faec8143ac0 2023-11-26T07:23:48.235 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:48.235+0000 7faecedaa640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7faec80b1310 con 0x7faec80cf990 2023-11-26T07:23:48.236 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:48.235+0000 7faeccb46640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7faec80708b0 0x7faec813e1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:23:48.236 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:48.235+0000 7faeccb46640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7faec80708b0 0x7faec813e1f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:36806/0 (socket says 172.21.15.143:36806) 2023-11-26T07:23:48.236 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:48.235+0000 7faeccb46640 1 -- 172.21.15.143:0/2146419407 learned_addr learned my addr 172.21.15.143:0/2146419407 (peer_addr_for_me v2:172.21.15.143:0/0) 2023-11-26T07:23:48.236 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:48.235+0000 7faeccb46640 1 -- 172.21.15.143:0/2146419407 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7faec80cf990 msgr2=0x7faec815e9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:23:48.236 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:48.235+0000 7faeccb46640 1 --2- 172.21.15.143:0/2146419407 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7faec80cf990 0x7faec815e9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:23:48.236 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:48.235+0000 7faeccb46640 1 -- 172.21.15.143:0/2146419407 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7faec8143ac0 msgr2=0x7faec8140030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:23:48.236 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:48.235+0000 7faeccb46640 1 --2- 172.21.15.143:0/2146419407 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7faec8143ac0 0x7faec8140030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:23:48.236 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:48.235+0000 7faeccb46640 1 -- 172.21.15.143:0/2146419407 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7faebc0094d0 con 0x7faec80708b0 2023-11-26T07:23:48.237 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:48.235+0000 7faeccb46640 1 --2- 172.21.15.143:0/2146419407 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7faec80708b0 0x7faec813e1f0 secure :-1 s=READY pgs=1198 cs=0 l=1 rev1=1 crypto rx=0x7faebc01a0f0 tx=0x7faebc01ad00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:23:48.237 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:48.235+0000 7faec57fa640 1 -- 172.21.15.143:0/2146419407 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faebc0bd3f0 con 0x7faec80708b0 2023-11-26T07:23:48.237 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:48.235+0000 7faec57fa640 1 -- 172.21.15.143:0/2146419407 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7faebc0bd570 con 0x7faec80708b0 2023-11-26T07:23:48.238 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:48.235+0000 7faecedaa640 1 -- 172.21.15.143:0/2146419407 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7faec80b15f0 con 0x7faec80708b0 2023-11-26T07:23:48.238 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:48.235+0000 7faec57fa640 1 -- 172.21.15.143:0/2146419407 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faebc0bd920 con 0x7faec80708b0 2023-11-26T07:23:48.238 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:48.235+0000 7faecedaa640 1 -- 172.21.15.143:0/2146419407 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7faec813e730 con 0x7faec80708b0 2023-11-26T07:23:48.238 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:48.235+0000 7faecedaa640 1 -- 172.21.15.143:0/2146419407 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7faec8067680 con 0x7faec80708b0 2023-11-26T07:23:48.238 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:48.235+0000 7faec57fa640 1 -- 172.21.15.143:0/2146419407 <== mon.0 v2:172.21.15.143:3300/0 4 ==== mgrmap(e 16) v1 ==== 85954+0+0 (secure 0 0 0) 0x7faebc0b33f0 con 0x7faec80708b0 2023-11-26T07:23:48.238 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:48.239+0000 7faec57fa640 1 --2- 172.21.15.143:0/2146419407 >> [v2:172.21.15.188:6836/3401128553,v1:172.21.15.188:6837/3401128553] conn(0x7faeac066590 0x7faeac068a50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:23:48.239 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:48.239+0000 7faec57fa640 1 -- 172.21.15.143:0/2146419407 <== mon.0 v2:172.21.15.143:3300/0 5 ==== osd_map(91..91 src has 1..91) v4 ==== 5795+0+0 (secure 0 0 0) 0x7faebc11e6d0 con 0x7faec80708b0 2023-11-26T07:23:48.239 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:48.239+0000 7faec7fff640 1 --2- 172.21.15.143:0/2146419407 >> [v2:172.21.15.188:6836/3401128553,v1:172.21.15.188:6837/3401128553] conn(0x7faeac066590 0x7faeac068a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:23:48.239 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:48.239+0000 7faec7fff640 1 --2- 172.21.15.143:0/2146419407 >> [v2:172.21.15.188:6836/3401128553,v1:172.21.15.188:6837/3401128553] conn(0x7faeac066590 0x7faeac068a50 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7faec813eb70 tx=0x7faeb40079c0 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:23:48.242 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:48.239+0000 7faec57fa640 1 -- 172.21.15.143:0/2146419407 <== mon.0 v2:172.21.15.143:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172649 (secure 0 0 0) 0x7faebc0eb950 con 0x7faec80708b0 2023-11-26T07:23:48.391 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:48.391+0000 7faecedaa640 1 -- 172.21.15.143:0/2146419407 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7faec806e1f0 con 0x7faec80708b0 2023-11-26T07:23:48.392 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:48.391+0000 7faec57fa640 1 -- 172.21.15.143:0/2146419407 <== mon.0 v2:172.21.15.143:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v91) v1 ==== 74+0+16727 (secure 0 0 0) 0x7faebc0eef70 con 0x7faec80708b0 2023-11-26T07:23:48.393 INFO:teuthology.orchestra.run.smithi143.stdout: 2023-11-26T07:23:48.393 INFO:teuthology.orchestra.run.smithi143.stdout:{"epoch":91,"fsid":"f5033060-2469-4a07-95fe-2b76f5404bc1","created":"2023-11-26T07:17:28.613663+0000","modified":"2023-11-26T07:23:47.807341+0000","last_up_change":"2023-11-26T07:17:41.157813+0000","last_in_change":"2023-11-26T07:17:31.181658+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":15,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-11-26T07:17:41.758597+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":12,"pool_name":"cephfs_metadata","create_time":"2023-11-26T07:23:26.724016+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"83","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":13,"pool_name":"cephfs_data","create_time":"2023-11-26T07:23:28.019663+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"84","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"ebbee2c6-023e-44b9-9f1d-6f7bfd309730","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6817","nonce":639807369},{"type":"v1","addr":"172.21.15.143:6819","nonce":639807369}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6821","nonce":639807369},{"type":"v1","addr":"172.21.15.143:6823","nonce":639807369}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6830","nonce":639807369},{"type":"v1","addr":"172.21.15.143:6831","nonce":639807369}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6825","nonce":639807369},{"type":"v1","addr":"172.21.15.143:6828","nonce":639807369}]},"public_addr":"172.21.15.143:6819/639807369","cluster_addr":"172.21.15.143:6823/639807369","heartbeat_back_addr":"172.21.15.143:6831/639807369","heartbeat_front_addr":"172.21.15.143:6828/639807369","state":["exists","up"]},{"osd":1,"uuid":"fec4b8be-7122-49e9-9a19-5850a0700001","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6816","nonce":1144942530},{"type":"v1","addr":"172.21.15.143:6818","nonce":1144942530}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6820","nonce":1144942530},{"type":"v1","addr":"172.21.15.143:6822","nonce":1144942530}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6827","nonce":1144942530},{"type":"v1","addr":"172.21.15.143:6829","nonce":1144942530}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6824","nonce":1144942530},{"type":"v1","addr":"172.21.15.143:6826","nonce":1144942530}]},"public_addr":"172.21.15.143:6818/1144942530","cluster_addr":"172.21.15.143:6822/1144942530","heartbeat_back_addr":"172.21.15.143:6829/1144942530","heartbeat_front_addr":"172.21.15.143:6826/1144942530","state":["exists","up"]},{"osd":2,"uuid":"2888af5b-f933-46ca-8f95-6dd494ff43f6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6808","nonce":3909095186},{"type":"v1","addr":"172.21.15.143:6809","nonce":3909095186}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6810","nonce":3909095186},{"type":"v1","addr":"172.21.15.143:6811","nonce":3909095186}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6814","nonce":3909095186},{"type":"v1","addr":"172.21.15.143:6815","nonce":3909095186}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6812","nonce":3909095186},{"type":"v1","addr":"172.21.15.143:6813","nonce":3909095186}]},"public_addr":"172.21.15.143:6809/3909095186","cluster_addr":"172.21.15.143:6811/3909095186","heartbeat_back_addr":"172.21.15.143:6815/3909095186","heartbeat_front_addr":"172.21.15.143:6813/3909095186","state":["exists","up"]},{"osd":3,"uuid":"d1f52435-7bc1-4081-9550-9cd1ba55f549","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6800","nonce":3854210991},{"type":"v1","addr":"172.21.15.143:6801","nonce":3854210991}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6802","nonce":3854210991},{"type":"v1","addr":"172.21.15.143:6803","nonce":3854210991}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6806","nonce":3854210991},{"type":"v1","addr":"172.21.15.143:6807","nonce":3854210991}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6804","nonce":3854210991},{"type":"v1","addr":"172.21.15.143:6805","nonce":3854210991}]},"public_addr":"172.21.15.143:6801/3854210991","cluster_addr":"172.21.15.143:6803/3854210991","heartbeat_back_addr":"172.21.15.143:6807/3854210991","heartbeat_front_addr":"172.21.15.143:6805/3854210991","state":["exists","up"]},{"osd":4,"uuid":"e2ae8b6e-6e25-49b2-bcbf-aacce68f06a3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6800","nonce":3924606902},{"type":"v1","addr":"172.21.15.188:6801","nonce":3924606902}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6802","nonce":3924606902},{"type":"v1","addr":"172.21.15.188:6803","nonce":3924606902}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6806","nonce":3924606902},{"type":"v1","addr":"172.21.15.188:6807","nonce":3924606902}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6804","nonce":3924606902},{"type":"v1","addr":"172.21.15.188:6805","nonce":3924606902}]},"public_addr":"172.21.15.188:6801/3924606902","cluster_addr":"172.21.15.188:6803/3924606902","heartbeat_back_addr":"172.21.15.188:6807/3924606902","heartbeat_front_addr":"172.21.15.188:6805/3924606902","state":["exists","up"]},{"osd":5,"uuid":"3cec8a32-7698-4c36-8e89-e4ba11934f33","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6813","nonce":186841544},{"type":"v1","addr":"172.21.15.188:6815","nonce":186841544}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6817","nonce":186841544},{"type":"v1","addr":"172.21.15.188:6818","nonce":186841544}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6822","nonce":186841544},{"type":"v1","addr":"172.21.15.188:6823","nonce":186841544}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6820","nonce":186841544},{"type":"v1","addr":"172.21.15.188:6821","nonce":186841544}]},"public_addr":"172.21.15.188:6815/186841544","cluster_addr":"172.21.15.188:6818/186841544","heartbeat_back_addr":"172.21.15.188:6823/186841544","heartbeat_front_addr":"172.21.15.188:6821/186841544","state":["exists","up"]},{"osd":6,"uuid":"58a65d5d-e090-473a-81b4-ad38e85029f9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6808","nonce":2153499405},{"type":"v1","addr":"172.21.15.188:6809","nonce":2153499405}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6810","nonce":2153499405},{"type":"v1","addr":"172.21.15.188:6811","nonce":2153499405}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6816","nonce":2153499405},{"type":"v1","addr":"172.21.15.188:6819","nonce":2153499405}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6812","nonce":2153499405},{"type":"v1","addr":"172.21.15.188:6814","nonce":2153499405}]},"public_addr":"172.21.15.188:6809/2153499405","cluster_addr":"172.21.15.188:6811/2153499405","heartbeat_back_addr":"172.21.15.188:6819/2153499405","heartbeat_front_addr":"172.21.15.188:6814/2153499405","state":["exists","up"]},{"osd":7,"uuid":"ea5b7f70-3554-4cf8-ac30-f7c59b80678d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6824","nonce":2722096729},{"type":"v1","addr":"172.21.15.188:6825","nonce":2722096729}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6826","nonce":2722096729},{"type":"v1","addr":"172.21.15.188:6827","nonce":2722096729}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6830","nonce":2722096729},{"type":"v1","addr":"172.21.15.188:6831","nonce":2722096729}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6828","nonce":2722096729},{"type":"v1","addr":"172.21.15.188:6829","nonce":2722096729}]},"public_addr":"172.21.15.188:6825/2722096729","cluster_addr":"172.21.15.188:6827/2722096729","heartbeat_back_addr":"172.21.15.188:6831/2722096729","heartbeat_front_addr":"172.21.15.188:6829/2722096729","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-26T07:17:36.995032+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-26T07:17:36.932266+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-26T07:17:36.829304+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-26T07:17:36.804297+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-26T07:17:36.667892+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-26T07:17:36.964868+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-26T07:17:36.827332+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-26T07:17:36.893179+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.143:6835/4031724325":"2023-11-27T07:23:44.794394+0000","172.21.15.143:6833/4031724325":"2023-11-27T07:23:44.794394+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-11-26T07:23:48.394 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:48.395+0000 7faecedaa640 1 -- 172.21.15.143:0/2146419407 >> [v2:172.21.15.188:6836/3401128553,v1:172.21.15.188:6837/3401128553] conn(0x7faeac066590 msgr2=0x7faeac068a50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:23:48.395 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:48.395+0000 7faecedaa640 1 --2- 172.21.15.143:0/2146419407 >> [v2:172.21.15.188:6836/3401128553,v1:172.21.15.188:6837/3401128553] conn(0x7faeac066590 0x7faeac068a50 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7faec813eb70 tx=0x7faeb40079c0 comp rx=0 tx=0).stop 2023-11-26T07:23:48.395 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:48.395+0000 7faecedaa640 1 -- 172.21.15.143:0/2146419407 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7faec80708b0 msgr2=0x7faec813e1f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:23:48.395 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:48.395+0000 7faecedaa640 1 --2- 172.21.15.143:0/2146419407 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7faec80708b0 0x7faec813e1f0 secure :-1 s=READY pgs=1198 cs=0 l=1 rev1=1 crypto rx=0x7faebc01a0f0 tx=0x7faebc01ad00 comp rx=0 tx=0).stop 2023-11-26T07:23:48.395 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:48.395+0000 7faecedaa640 1 -- 172.21.15.143:0/2146419407 shutdown_connections 2023-11-26T07:23:48.395 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:48.395+0000 7faecedaa640 1 --2- 172.21.15.143:0/2146419407 >> [v2:172.21.15.188:6836/3401128553,v1:172.21.15.188:6837/3401128553] conn(0x7faeac066590 0x7faeac068a50 unknown :-1 s=CLOSED pgs=79 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:23:48.395 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:48.395+0000 7faecedaa640 1 --2- 172.21.15.143:0/2146419407 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7faec8143ac0 0x7faec8140030 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:23:48.395 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:48.395+0000 7faecedaa640 1 --2- 172.21.15.143:0/2146419407 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7faec80cf990 0x7faec815e9d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:23:48.395 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:48.395+0000 7faecedaa640 1 --2- 172.21.15.143:0/2146419407 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7faec80708b0 0x7faec813e1f0 unknown :-1 s=CLOSED pgs=1198 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:23:48.395 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:48.395+0000 7faecedaa640 1 -- 172.21.15.143:0/2146419407 >> 172.21.15.143:0/2146419407 conn(0x7faec806a890 msgr2=0x7faec815a700 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:23:48.396 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:48.395+0000 7faecedaa640 1 -- 172.21.15.143:0/2146419407 shutdown_connections 2023-11-26T07:23:48.396 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:48.395+0000 7faecedaa640 1 -- 172.21.15.143:0/2146419407 wait complete. 2023-11-26T07:23:48.413 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-11-26T07:17:41.758597+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 12, 'pool_name': 'cephfs_metadata', 'create_time': '2023-11-26T07:23:26.724016+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '83', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 2, 'score_stable': 2, 'optimal_score': 1, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 13, 'pool_name': 'cephfs_data', 'create_time': '2023-11-26T07:23:28.019663+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '84', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.3799999952316284, 'score_stable': 1.3799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.3799999952316284, 'raw_score_stable': 1.3799999952316284, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2023-11-26T07:23:48.414 DEBUG:teuthology.orchestra.run.smithi143:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-11-26T07:23:48.555 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:48.555+0000 7f420913e640 1 Processor -- start 2023-11-26T07:23:48.555 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:48.555+0000 7f420913e640 1 -- start start 2023-11-26T07:23:48.556 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:48.555+0000 7f420913e640 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7f4204143ac0 0x7f4204070370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:23:48.556 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:48.555+0000 7f420913e640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f42040cf990 0x7f42040cd1b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:23:48.556 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:48.555+0000 7f420913e640 1 -- --> v1:172.21.15.143:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f42040cc730 con 0x7f42040708b0 2023-11-26T07:23:48.557 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:48.555+0000 7f420913e640 1 -- --> v2:172.21.15.143:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4204143ea0 con 0x7f4204143ac0 2023-11-26T07:23:48.557 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:48.555+0000 7f420913e640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4204070ee0 con 0x7f42040cf990 2023-11-26T07:23:48.557 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:48.555+0000 7f4202d9d640 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7f4204143ac0 0x7f4204070370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:23:48.557 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:48.555+0000 7f420259c640 1 --1- >> v1:172.21.15.143:6789/0 conn(0x7f42040708b0 0x7f42040cf280 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.143:6789/0 says I am v1:172.21.15.143:46418/0 (socket says 172.21.15.143:46418) 2023-11-26T07:23:48.557 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:48.555+0000 7f420259c640 1 -- 172.21.15.143:0/1005603990 learned_addr learned my addr 172.21.15.143:0/1005603990 (peer_addr_for_me v1:172.21.15.143:0/0) 2023-11-26T07:23:48.557 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:48.555+0000 7f4202d9d640 1 -- 172.21.15.143:0/1005603990 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f42040cf990 msgr2=0x7f42040cd1b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-26T07:23:48.558 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:48.555+0000 7f4202d9d640 1 --2- 172.21.15.143:0/1005603990 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f42040cf990 0x7f42040cd1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:23:48.558 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:48.555+0000 7f4202d9d640 1 -- 172.21.15.143:0/1005603990 >> v1:172.21.15.143:6789/0 conn(0x7f42040708b0 legacy=0x7f42040cf280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:23:48.558 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:48.555+0000 7f4202d9d640 1 -- 172.21.15.143:0/1005603990 --> v2:172.21.15.143:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f42040711c0 con 0x7f4204143ac0 2023-11-26T07:23:48.560 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:48.559+0000 7f4202d9d640 1 --2- 172.21.15.143:0/1005603990 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f4204143ac0 0x7f4204070370 secure :-1 s=READY pgs=1200 cs=0 l=1 rev1=1 crypto rx=0x7f41f40097c0 tx=0x7f41f401c310 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c20f0675c8ac283 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:23:48.560 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:48.559+0000 7f4201d9b640 1 -- 172.21.15.143:0/1005603990 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f41f40ab600 con 0x7f4204143ac0 2023-11-26T07:23:48.560 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:48.559+0000 7f4201d9b640 1 -- 172.21.15.143:0/1005603990 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f41f40a7050 con 0x7f4204143ac0 2023-11-26T07:23:48.560 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:48.559+0000 7f4201d9b640 1 -- 172.21.15.143:0/1005603990 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f41f40ab980 con 0x7f4204143ac0 2023-11-26T07:23:48.561 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:48.559+0000 7f420913e640 1 -- 172.21.15.143:0/1005603990 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f4204143ac0 msgr2=0x7f4204070370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:23:48.561 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:48.559+0000 7f420913e640 1 --2- 172.21.15.143:0/1005603990 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f4204143ac0 0x7f4204070370 secure :-1 s=READY pgs=1200 cs=0 l=1 rev1=1 crypto rx=0x7f41f40097c0 tx=0x7f41f401c310 comp rx=0 tx=0).stop 2023-11-26T07:23:48.561 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:48.559+0000 7f420913e640 1 -- 172.21.15.143:0/1005603990 shutdown_connections 2023-11-26T07:23:48.561 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:48.559+0000 7f420913e640 1 --2- 172.21.15.143:0/1005603990 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f42040cf990 0x7f42040cd1b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:23:48.561 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:48.559+0000 7f420913e640 1 --2- 172.21.15.143:0/1005603990 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f4204143ac0 0x7f4204070370 unknown :-1 s=CLOSED pgs=1200 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:23:48.561 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:48.559+0000 7f420913e640 1 -- 172.21.15.143:0/1005603990 >> 172.21.15.143:0/1005603990 conn(0x7f420406a890 msgr2=0x7f420406aca0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:23:48.561 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:48.559+0000 7f420913e640 1 -- 172.21.15.143:0/1005603990 shutdown_connections 2023-11-26T07:23:48.562 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:48.559+0000 7f420913e640 1 -- 172.21.15.143:0/1005603990 wait complete. 2023-11-26T07:23:48.563 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:48.563+0000 7f420913e640 1 Processor -- start 2023-11-26T07:23:48.563 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:48.563+0000 7f420913e640 1 -- start start 2023-11-26T07:23:48.564 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:48.563+0000 7f420913e640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f42040708b0 0x7f42041e2a40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:23:48.564 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:48.563+0000 7f420913e640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f42040cf990 0x7f42041e4330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:23:48.564 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:48.563+0000 7f420913e640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f4204143ac0 0x7f42041e5350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:23:48.564 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:48.563+0000 7f420913e640 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f42041e3290 con 0x7f4204143ac0 2023-11-26T07:23:48.565 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:48.563+0000 7f420913e640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f42041e3570 con 0x7f42040708b0 2023-11-26T07:23:48.565 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:48.563+0000 7f420913e640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f42041e3850 con 0x7f42040cf990 2023-11-26T07:23:48.565 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:48.563+0000 7f420359e640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f4204143ac0 0x7f42041e5350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:23:48.565 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:48.563+0000 7f420359e640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f4204143ac0 0x7f42041e5350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:36830/0 (socket says 172.21.15.143:36830) 2023-11-26T07:23:48.565 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:48.563+0000 7f420359e640 1 -- 172.21.15.143:0/3431131277 learned_addr learned my addr 172.21.15.143:0/3431131277 (peer_addr_for_me v2:172.21.15.143:0/0) 2023-11-26T07:23:48.566 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:48.563+0000 7f4202d9d640 1 --2- 172.21.15.143:0/3431131277 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f42040708b0 0x7f42041e2a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:23:48.566 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:48.563+0000 7f420259c640 1 --2- 172.21.15.143:0/3431131277 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f42040cf990 0x7f42041e4330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:23:48.567 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:48.563+0000 7f420359e640 1 -- 172.21.15.143:0/3431131277 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f42040cf990 msgr2=0x7f42041e4330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:23:48.567 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:48.563+0000 7f420359e640 1 --2- 172.21.15.143:0/3431131277 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f42040cf990 0x7f42041e4330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:23:48.567 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:48.563+0000 7f420359e640 1 -- 172.21.15.143:0/3431131277 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f42040708b0 msgr2=0x7f42041e2a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:23:48.567 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:48.563+0000 7f420359e640 1 --2- 172.21.15.143:0/3431131277 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f42040708b0 0x7f42041e2a40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:23:48.567 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:48.563+0000 7f420359e640 1 -- 172.21.15.143:0/3431131277 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f41f4009420 con 0x7f4204143ac0 2023-11-26T07:23:48.567 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:48.563+0000 7f420359e640 1 --2- 172.21.15.143:0/3431131277 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f4204143ac0 0x7f42041e5350 secure :-1 s=READY pgs=1201 cs=0 l=1 rev1=1 crypto rx=0x7f41fc00b970 tx=0x7f41fc00be30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:23:48.567 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:48.563+0000 7f41eb7fe640 1 -- 172.21.15.143:0/3431131277 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f41fc00cb40 con 0x7f4204143ac0 2023-11-26T07:23:48.567 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:48.563+0000 7f41eb7fe640 1 -- 172.21.15.143:0/3431131277 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f41fc00ccc0 con 0x7f4204143ac0 2023-11-26T07:23:48.567 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:48.563+0000 7f420913e640 1 -- 172.21.15.143:0/3431131277 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f42041e3b30 con 0x7f4204143ac0 2023-11-26T07:23:48.568 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:48.563+0000 7f41eb7fe640 1 -- 172.21.15.143:0/3431131277 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f41fc014680 con 0x7f4204143ac0 2023-11-26T07:23:48.568 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:48.563+0000 7f420913e640 1 -- 172.21.15.143:0/3431131277 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f42041e2f80 con 0x7f4204143ac0 2023-11-26T07:23:48.568 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:48.567+0000 7f420913e640 1 -- 172.21.15.143:0/3431131277 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f42040cb840 con 0x7f4204143ac0 2023-11-26T07:23:48.568 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:48.567+0000 7f41eb7fe640 1 -- 172.21.15.143:0/3431131277 <== mon.0 v2:172.21.15.143:3300/0 4 ==== mgrmap(e 16) v1 ==== 85954+0+0 (secure 0 0 0) 0x7f41fc01d3b0 con 0x7f4204143ac0 2023-11-26T07:23:48.568 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:48.567+0000 7f41eb7fe640 1 --2- 172.21.15.143:0/3431131277 >> [v2:172.21.15.188:6836/3401128553,v1:172.21.15.188:6837/3401128553] conn(0x7f41f0066590 0x7f41f0068a50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:23:48.569 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:48.567+0000 7f41eb7fe640 1 -- 172.21.15.143:0/3431131277 <== mon.0 v2:172.21.15.143:3300/0 5 ==== osd_map(91..91 src has 1..91) v4 ==== 5795+0+0 (secure 0 0 0) 0x7f41fc083f30 con 0x7f4204143ac0 2023-11-26T07:23:48.569 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:48.567+0000 7f4202d9d640 1 --2- 172.21.15.143:0/3431131277 >> [v2:172.21.15.188:6836/3401128553,v1:172.21.15.188:6837/3401128553] conn(0x7f41f0066590 0x7f41f0068a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:23:48.569 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:48.567+0000 7f4202d9d640 1 --2- 172.21.15.143:0/3431131277 >> [v2:172.21.15.188:6836/3401128553,v1:172.21.15.188:6837/3401128553] conn(0x7f41f0066590 0x7f41f0068a50 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7f41f40ad000 tx=0x7f41f40a6000 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:23:48.572 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:48.571+0000 7f41eb7fe640 1 -- 172.21.15.143:0/3431131277 <== mon.0 v2:172.21.15.143:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172649 (secure 0 0 0) 0x7f41fc0512f0 con 0x7f4204143ac0 2023-11-26T07:23:48.723 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:48.723+0000 7f420913e640 1 -- 172.21.15.143:0/3431131277 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f42041e4050 con 0x7f4204143ac0 2023-11-26T07:23:48.723 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:48.723+0000 7f41eb7fe640 1 -- 172.21.15.143:0/3431131277 <== mon.0 v2:172.21.15.143:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v91) v1 ==== 93+0+10 (secure 0 0 0) 0x7f42041e4050 con 0x7f4204143ac0 2023-11-26T07:23:48.724 INFO:teuthology.orchestra.run.smithi143.stdout:pg_num: 1 2023-11-26T07:23:48.726 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:48.723+0000 7f420913e640 1 -- 172.21.15.143:0/3431131277 >> [v2:172.21.15.188:6836/3401128553,v1:172.21.15.188:6837/3401128553] conn(0x7f41f0066590 msgr2=0x7f41f0068a50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:23:48.726 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:48.723+0000 7f420913e640 1 --2- 172.21.15.143:0/3431131277 >> [v2:172.21.15.188:6836/3401128553,v1:172.21.15.188:6837/3401128553] conn(0x7f41f0066590 0x7f41f0068a50 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7f41f40ad000 tx=0x7f41f40a6000 comp rx=0 tx=0).stop 2023-11-26T07:23:48.726 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:48.723+0000 7f420913e640 1 -- 172.21.15.143:0/3431131277 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f4204143ac0 msgr2=0x7f42041e5350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:23:48.726 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:48.723+0000 7f420913e640 1 --2- 172.21.15.143:0/3431131277 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f4204143ac0 0x7f42041e5350 secure :-1 s=READY pgs=1201 cs=0 l=1 rev1=1 crypto rx=0x7f41fc00b970 tx=0x7f41fc00be30 comp rx=0 tx=0).stop 2023-11-26T07:23:48.726 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:48.723+0000 7f420913e640 1 -- 172.21.15.143:0/3431131277 shutdown_connections 2023-11-26T07:23:48.726 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:48.723+0000 7f420913e640 1 --2- 172.21.15.143:0/3431131277 >> [v2:172.21.15.188:6836/3401128553,v1:172.21.15.188:6837/3401128553] conn(0x7f41f0066590 0x7f41f0068a50 unknown :-1 s=CLOSED pgs=80 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:23:48.726 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:48.723+0000 7f420913e640 1 --2- 172.21.15.143:0/3431131277 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f4204143ac0 0x7f42041e5350 unknown :-1 s=CLOSED pgs=1201 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:23:48.726 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:48.723+0000 7f420913e640 1 --2- 172.21.15.143:0/3431131277 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f42040cf990 0x7f42041e4330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:23:48.726 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:48.723+0000 7f420913e640 1 --2- 172.21.15.143:0/3431131277 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f42040708b0 0x7f42041e2a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:23:48.726 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:48.723+0000 7f420913e640 1 -- 172.21.15.143:0/3431131277 >> 172.21.15.143:0/3431131277 conn(0x7f420406a890 msgr2=0x7f420406c520 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:23:48.727 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:48.723+0000 7f420913e640 1 -- 172.21.15.143:0/3431131277 shutdown_connections 2023-11-26T07:23:48.727 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:48.723+0000 7f420913e640 1 -- 172.21.15.143:0/3431131277 wait complete. 2023-11-26T07:23:48.745 DEBUG:teuthology.orchestra.run.smithi143:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-11-26T07:23:48.890 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:48.887+0000 7f7d9a488640 1 Processor -- start 2023-11-26T07:23:48.890 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:48.887+0000 7f7d9a488640 1 -- start start 2023-11-26T07:23:48.891 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:48.891+0000 7f7d9a488640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f7d940b23f0 0x7f7d940ada60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:23:48.891 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:48.891+0000 7f7d9a488640 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7f7d9406b9d0 0x7f7d9406be10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:23:48.892 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:48.891+0000 7f7d9a488640 1 -- --> v1:172.21.15.143:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7d940cc050 con 0x7f7d940cfaf0 2023-11-26T07:23:48.892 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:48.891+0000 7f7d9a488640 1 -- --> v2:172.21.15.143:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7d9406c350 con 0x7f7d9406b9d0 2023-11-26T07:23:48.892 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:48.891+0000 7f7d9a488640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7d940cb440 con 0x7f7d940b23f0 2023-11-26T07:23:48.892 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:48.891+0000 7f7d98c85640 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7f7d9406b9d0 0x7f7d9406be10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:23:48.892 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:48.891+0000 7f7d99c87640 1 --1- >> v1:172.21.15.143:6789/0 conn(0x7f7d940cfaf0 0x7f7d940cff50 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.143:6789/0 says I am v1:172.21.15.143:46426/0 (socket says 172.21.15.143:46426) 2023-11-26T07:23:48.892 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:48.891+0000 7f7d99c87640 1 -- 172.21.15.143:0/2917354981 learned_addr learned my addr 172.21.15.143:0/2917354981 (peer_addr_for_me v1:172.21.15.143:0/0) 2023-11-26T07:23:48.892 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:48.891+0000 7f7d99486640 1 --2- 172.21.15.143:0/2917354981 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f7d940b23f0 0x7f7d940ada60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:23:48.893 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:48.891+0000 7f7d98c85640 1 -- 172.21.15.143:0/2917354981 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f7d940b23f0 msgr2=0x7f7d940ada60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-26T07:23:48.893 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:48.891+0000 7f7d98c85640 1 --2- 172.21.15.143:0/2917354981 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f7d940b23f0 0x7f7d940ada60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:23:48.893 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:48.891+0000 7f7d98c85640 1 -- 172.21.15.143:0/2917354981 >> v1:172.21.15.143:6789/0 conn(0x7f7d940cfaf0 legacy=0x7f7d940cff50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:23:48.893 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:48.891+0000 7f7d98c85640 1 -- 172.21.15.143:0/2917354981 --> v2:172.21.15.143:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7d940cb720 con 0x7f7d9406b9d0 2023-11-26T07:23:48.893 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:48.891+0000 7f7d8bfff640 1 -- 172.21.15.143:0/2917354981 <== mon.0 v1:172.21.15.143:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3454908598 0 0) 0x7f7d940cc050 con 0x7f7d940cfaf0 2023-11-26T07:23:48.894 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:48.895+0000 7f7d98c85640 1 --2- 172.21.15.143:0/2917354981 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f7d9406b9d0 0x7f7d9406be10 secure :-1 s=READY pgs=1203 cs=0 l=1 rev1=1 crypto rx=0x7f7d840097c0 tx=0x7f7d8401c480 comp rx=0 tx=0).ready entity=mon.0 client_cookie=283260d5661deabf server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:23:48.895 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:48.895+0000 7f7d8bfff640 1 -- 172.21.15.143:0/2917354981 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7d840ab600 con 0x7f7d9406b9d0 2023-11-26T07:23:48.895 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:48.895+0000 7f7d8bfff640 1 -- 172.21.15.143:0/2917354981 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7d840a7050 con 0x7f7d9406b9d0 2023-11-26T07:23:48.895 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:48.895+0000 7f7d8bfff640 1 -- 172.21.15.143:0/2917354981 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7d840ab980 con 0x7f7d9406b9d0 2023-11-26T07:23:48.896 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:48.895+0000 7f7d9a488640 1 -- 172.21.15.143:0/2917354981 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f7d9406b9d0 msgr2=0x7f7d9406be10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:23:48.896 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:48.895+0000 7f7d9a488640 1 --2- 172.21.15.143:0/2917354981 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f7d9406b9d0 0x7f7d9406be10 secure :-1 s=READY pgs=1203 cs=0 l=1 rev1=1 crypto rx=0x7f7d840097c0 tx=0x7f7d8401c480 comp rx=0 tx=0).stop 2023-11-26T07:23:48.896 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:48.895+0000 7f7d9a488640 1 -- 172.21.15.143:0/2917354981 shutdown_connections 2023-11-26T07:23:48.896 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:48.895+0000 7f7d9a488640 1 --2- 172.21.15.143:0/2917354981 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f7d9406b9d0 0x7f7d9406be10 unknown :-1 s=CLOSED pgs=1203 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:23:48.896 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:48.895+0000 7f7d9a488640 1 --2- 172.21.15.143:0/2917354981 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f7d940b23f0 0x7f7d940ada60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:23:48.896 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:48.895+0000 7f7d9a488640 1 -- 172.21.15.143:0/2917354981 >> 172.21.15.143:0/2917354981 conn(0x7f7d940d0df0 msgr2=0x7f7d940d1200 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:23:48.896 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:48.895+0000 7f7d9a488640 1 -- 172.21.15.143:0/2917354981 shutdown_connections 2023-11-26T07:23:48.896 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:48.895+0000 7f7d9a488640 1 -- 172.21.15.143:0/2917354981 wait complete. 2023-11-26T07:23:48.897 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:48.895+0000 7f7d9a488640 1 Processor -- start 2023-11-26T07:23:48.897 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:48.895+0000 7f7d9a488640 1 -- start start 2023-11-26T07:23:48.899 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:48.899+0000 7f7d9a488640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f7d9406b9d0 0x7f7d94155650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:23:48.899 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:48.899+0000 7f7d9a488640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f7d940b23f0 0x7f7d940caa50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:23:48.899 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:48.899+0000 7f7d9a488640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f7d940cfaf0 0x7f7d940b0040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:23:48.899 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:48.899+0000 7f7d9a488640 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7d94155dd0 con 0x7f7d940cfaf0 2023-11-26T07:23:48.899 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:48.899+0000 7f7d9a488640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7d94155b90 con 0x7f7d940b23f0 2023-11-26T07:23:48.899 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:48.899+0000 7f7d9a488640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7d94156020 con 0x7f7d9406b9d0 2023-11-26T07:23:48.900 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:48.899+0000 7f7d99c87640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f7d940cfaf0 0x7f7d940b0040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:23:48.900 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:48.899+0000 7f7d99c87640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f7d940cfaf0 0x7f7d940b0040 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:36848/0 (socket says 172.21.15.143:36848) 2023-11-26T07:23:48.900 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:48.899+0000 7f7d99486640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f7d9406b9d0 0x7f7d94155650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:23:48.900 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:48.899+0000 7f7d99c87640 1 -- 172.21.15.143:0/3830981639 learned_addr learned my addr 172.21.15.143:0/3830981639 (peer_addr_for_me v2:172.21.15.143:0/0) 2023-11-26T07:23:48.900 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:48.899+0000 7f7d98c85640 1 --2- 172.21.15.143:0/3830981639 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f7d940b23f0 0x7f7d940caa50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:23:48.900 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:48.899+0000 7f7d99c87640 1 -- 172.21.15.143:0/3830981639 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f7d9406b9d0 msgr2=0x7f7d94155650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:23:48.900 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:48.899+0000 7f7d99c87640 1 --2- 172.21.15.143:0/3830981639 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f7d9406b9d0 0x7f7d94155650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:23:48.900 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:48.899+0000 7f7d99c87640 1 -- 172.21.15.143:0/3830981639 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f7d940b23f0 msgr2=0x7f7d940caa50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:23:48.900 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:48.899+0000 7f7d99c87640 1 --2- 172.21.15.143:0/3830981639 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f7d940b23f0 0x7f7d940caa50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:23:48.900 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:48.899+0000 7f7d99c87640 1 -- 172.21.15.143:0/3830981639 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7d84009420 con 0x7f7d940cfaf0 2023-11-26T07:23:48.900 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:48.899+0000 7f7d99c87640 1 --2- 172.21.15.143:0/3830981639 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f7d940cfaf0 0x7f7d940b0040 secure :-1 s=READY pgs=1204 cs=0 l=1 rev1=1 crypto rx=0x7f7d8c001570 tx=0x7f7d8c00cca0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:23:48.901 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:48.899+0000 7f7d89ffb640 1 -- 172.21.15.143:0/3830981639 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7d8c00a640 con 0x7f7d940cfaf0 2023-11-26T07:23:48.901 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:48.899+0000 7f7d89ffb640 1 -- 172.21.15.143:0/3830981639 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7d8c00a7c0 con 0x7f7d940cfaf0 2023-11-26T07:23:48.902 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:48.899+0000 7f7d89ffb640 1 -- 172.21.15.143:0/3830981639 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7d8c00ab70 con 0x7f7d940cfaf0 2023-11-26T07:23:48.902 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:48.899+0000 7f7d9a488640 1 -- 172.21.15.143:0/3830981639 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7d94156300 con 0x7f7d940cfaf0 2023-11-26T07:23:48.902 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:48.899+0000 7f7d9a488640 1 -- 172.21.15.143:0/3830981639 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7d941568c0 con 0x7f7d940cfaf0 2023-11-26T07:23:48.902 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:48.899+0000 7f7d9a488640 1 -- 172.21.15.143:0/3830981639 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7d9406d4f0 con 0x7f7d940cfaf0 2023-11-26T07:23:48.902 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:48.899+0000 7f7d89ffb640 1 -- 172.21.15.143:0/3830981639 <== mon.0 v2:172.21.15.143:3300/0 4 ==== mgrmap(e 16) v1 ==== 85954+0+0 (secure 0 0 0) 0x7f7d8c00acd0 con 0x7f7d940cfaf0 2023-11-26T07:23:48.902 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:48.903+0000 7f7d89ffb640 1 --2- 172.21.15.143:0/3830981639 >> [v2:172.21.15.188:6836/3401128553,v1:172.21.15.188:6837/3401128553] conn(0x7f7d78066590 0x7f7d78068a50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:23:48.903 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:48.903+0000 7f7d89ffb640 1 -- 172.21.15.143:0/3830981639 <== mon.0 v2:172.21.15.143:3300/0 5 ==== osd_map(91..91 src has 1..91) v4 ==== 5795+0+0 (secure 0 0 0) 0x7f7d8c083640 con 0x7f7d940cfaf0 2023-11-26T07:23:48.903 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:48.903+0000 7f7d99486640 1 --2- 172.21.15.143:0/3830981639 >> [v2:172.21.15.188:6836/3401128553,v1:172.21.15.188:6837/3401128553] conn(0x7f7d78066590 0x7f7d78068a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:23:48.904 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:48.903+0000 7f7d99486640 1 --2- 172.21.15.143:0/3830981639 >> [v2:172.21.15.188:6836/3401128553,v1:172.21.15.188:6837/3401128553] conn(0x7f7d78066590 0x7f7d78068a50 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7f7d940c5a50 tx=0x7f7d90009190 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:23:48.906 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:48.907+0000 7f7d89ffb640 1 -- 172.21.15.143:0/3830981639 <== mon.0 v2:172.21.15.143:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172649 (secure 0 0 0) 0x7f7d8c050960 con 0x7f7d940cfaf0 2023-11-26T07:23:49.058 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:49.055+0000 7f7d9a488640 1 -- 172.21.15.143:0/3830981639 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f7d9406cef0 con 0x7f7d940cfaf0 2023-11-26T07:23:49.058 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:49.059+0000 7f7d89ffb640 1 -- 172.21.15.143:0/3830981639 <== mon.0 v2:172.21.15.143:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v91) v1 ==== 104+0+11 (secure 0 0 0) 0x7f7d9406cef0 con 0x7f7d940cfaf0 2023-11-26T07:23:49.058 INFO:teuthology.orchestra.run.smithi143.stdout:pg_num: 32 2023-11-26T07:23:49.060 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:49.059+0000 7f7d9a488640 1 -- 172.21.15.143:0/3830981639 >> [v2:172.21.15.188:6836/3401128553,v1:172.21.15.188:6837/3401128553] conn(0x7f7d78066590 msgr2=0x7f7d78068a50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:23:49.060 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:49.059+0000 7f7d9a488640 1 --2- 172.21.15.143:0/3830981639 >> [v2:172.21.15.188:6836/3401128553,v1:172.21.15.188:6837/3401128553] conn(0x7f7d78066590 0x7f7d78068a50 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7f7d940c5a50 tx=0x7f7d90009190 comp rx=0 tx=0).stop 2023-11-26T07:23:49.060 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:49.059+0000 7f7d9a488640 1 -- 172.21.15.143:0/3830981639 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f7d940cfaf0 msgr2=0x7f7d940b0040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:23:49.060 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:49.059+0000 7f7d9a488640 1 --2- 172.21.15.143:0/3830981639 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f7d940cfaf0 0x7f7d940b0040 secure :-1 s=READY pgs=1204 cs=0 l=1 rev1=1 crypto rx=0x7f7d8c001570 tx=0x7f7d8c00cca0 comp rx=0 tx=0).stop 2023-11-26T07:23:49.061 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:49.059+0000 7f7d9a488640 1 -- 172.21.15.143:0/3830981639 shutdown_connections 2023-11-26T07:23:49.061 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:49.059+0000 7f7d9a488640 1 --2- 172.21.15.143:0/3830981639 >> [v2:172.21.15.188:6836/3401128553,v1:172.21.15.188:6837/3401128553] conn(0x7f7d78066590 0x7f7d78068a50 unknown :-1 s=CLOSED pgs=81 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:23:49.061 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:49.059+0000 7f7d9a488640 1 --2- 172.21.15.143:0/3830981639 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f7d940cfaf0 0x7f7d940b0040 unknown :-1 s=CLOSED pgs=1204 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:23:49.061 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:49.059+0000 7f7d9a488640 1 --2- 172.21.15.143:0/3830981639 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f7d940b23f0 0x7f7d940caa50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:23:49.061 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:49.059+0000 7f7d9a488640 1 --2- 172.21.15.143:0/3830981639 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f7d9406b9d0 0x7f7d94155650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:23:49.061 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:49.059+0000 7f7d9a488640 1 -- 172.21.15.143:0/3830981639 >> 172.21.15.143:0/3830981639 conn(0x7f7d940d0df0 msgr2=0x7f7d940d11d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:23:49.061 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:49.059+0000 7f7d9a488640 1 -- 172.21.15.143:0/3830981639 shutdown_connections 2023-11-26T07:23:49.061 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:49.059+0000 7f7d9a488640 1 -- 172.21.15.143:0/3830981639 wait complete. 2023-11-26T07:23:49.079 DEBUG:teuthology.orchestra.run.smithi143:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-11-26T07:23:49.219 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:49.219+0000 7f8d32d2b640 1 Processor -- start 2023-11-26T07:23:49.219 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:49.219+0000 7f8d32d2b640 1 -- start start 2023-11-26T07:23:49.221 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:49.219+0000 7f8d32d2b640 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7f8d2c0b1860 0x7f8d2c0d0e40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:23:49.221 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:49.219+0000 7f8d32d2b640 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7f8d2c0cdd20 0x7f8d2c0ce190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:23:49.221 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:49.219+0000 7f8d32d2b640 1 -- --> v1:172.21.15.143:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8d2c0d3d20 con 0x7f8d2c0cfa70 2023-11-26T07:23:49.221 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:49.219+0000 7f8d32d2b640 1 -- --> v2:172.21.15.143:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8d2c0d4000 con 0x7f8d2c0cdd20 2023-11-26T07:23:49.221 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:49.219+0000 7f8d32d2b640 1 -- --> v2:172.21.15.188:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8d2c0afd70 con 0x7f8d2c0b1860 2023-11-26T07:23:49.221 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:49.219+0000 7f8d31d29640 1 --1- >> v1:172.21.15.143:6789/0 conn(0x7f8d2c0cfa70 0x7f8d2c0cfe50 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.143:6789/0 says I am v1:172.21.15.143:46436/0 (socket says 172.21.15.143:46436) 2023-11-26T07:23:49.221 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:49.219+0000 7f8d31d29640 1 -- 172.21.15.143:0/3054297968 learned_addr learned my addr 172.21.15.143:0/3054297968 (peer_addr_for_me v1:172.21.15.143:0/0) 2023-11-26T07:23:49.221 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:49.219+0000 7f8d3252a640 1 --2- 172.21.15.143:0/3054297968 >> v2:172.21.15.143:3300/0 conn(0x7f8d2c0cdd20 0x7f8d2c0ce190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:23:49.221 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:49.219+0000 7f8d31528640 1 --2- 172.21.15.143:0/3054297968 >> v2:172.21.15.188:3300/0 conn(0x7f8d2c0b1860 0x7f8d2c0d0e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:23:49.222 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:49.219+0000 7f8d30d27640 1 -- 172.21.15.143:0/3054297968 <== mon.0 v1:172.21.15.143:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1323679736 0 0) 0x7f8d2c0d3d20 con 0x7f8d2c0cfa70 2023-11-26T07:23:49.222 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:49.219+0000 7f8d30d27640 1 -- 172.21.15.143:0/3054297968 --> v1:172.21.15.143:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f8d18003530 con 0x7f8d2c0cfa70 2023-11-26T07:23:49.222 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:49.219+0000 7f8d3252a640 1 -- 172.21.15.143:0/3054297968 >> v2:172.21.15.188:3300/0 conn(0x7f8d2c0b1860 msgr2=0x7f8d2c0d0e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-26T07:23:49.222 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:49.219+0000 7f8d3252a640 1 --2- 172.21.15.143:0/3054297968 >> v2:172.21.15.188:3300/0 conn(0x7f8d2c0b1860 0x7f8d2c0d0e40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:23:49.222 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:49.219+0000 7f8d3252a640 1 -- 172.21.15.143:0/3054297968 >> v1:172.21.15.143:6789/0 conn(0x7f8d2c0cfa70 legacy=0x7f8d2c0cfe50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:23:49.222 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:49.219+0000 7f8d3252a640 1 -- 172.21.15.143:0/3054297968 --> v2:172.21.15.143:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8d2c0b0050 con 0x7f8d2c0cdd20 2023-11-26T07:23:49.222 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:49.219+0000 7f8d31528640 1 --2- 172.21.15.143:0/3054297968 >> v2:172.21.15.188:3300/0 conn(0x7f8d2c0b1860 0x7f8d2c0d0e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-26T07:23:49.224 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:49.223+0000 7f8d3252a640 1 --2- 172.21.15.143:0/3054297968 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f8d2c0cdd20 0x7f8d2c0ce190 secure :-1 s=READY pgs=1206 cs=0 l=1 rev1=1 crypto rx=0x7f8d2c08b600 tx=0x7f8d2401c8e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a938adafeb45ffa5 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:23:49.224 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:49.223+0000 7f8d30d27640 1 -- 172.21.15.143:0/3054297968 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8d240a59c0 con 0x7f8d2c0cdd20 2023-11-26T07:23:49.224 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:49.223+0000 7f8d30d27640 1 -- 172.21.15.143:0/3054297968 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8d240a5b20 con 0x7f8d2c0cdd20 2023-11-26T07:23:49.224 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:49.223+0000 7f8d30d27640 1 -- 172.21.15.143:0/3054297968 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8d240ad440 con 0x7f8d2c0cdd20 2023-11-26T07:23:49.225 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:49.223+0000 7f8d32d2b640 1 -- 172.21.15.143:0/3054297968 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f8d2c0cdd20 msgr2=0x7f8d2c0ce190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:23:49.225 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:49.223+0000 7f8d32d2b640 1 --2- 172.21.15.143:0/3054297968 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f8d2c0cdd20 0x7f8d2c0ce190 secure :-1 s=READY pgs=1206 cs=0 l=1 rev1=1 crypto rx=0x7f8d2c08b600 tx=0x7f8d2401c8e0 comp rx=0 tx=0).stop 2023-11-26T07:23:49.225 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:49.223+0000 7f8d32d2b640 1 -- 172.21.15.143:0/3054297968 shutdown_connections 2023-11-26T07:23:49.225 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:49.223+0000 7f8d32d2b640 1 --2- 172.21.15.143:0/3054297968 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f8d2c0cdd20 0x7f8d2c0ce190 unknown :-1 s=CLOSED pgs=1206 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:23:49.225 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:49.223+0000 7f8d32d2b640 1 --2- 172.21.15.143:0/3054297968 >> v2:172.21.15.188:3300/0 conn(0x7f8d2c0b1860 0x7f8d2c0d0e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:23:49.225 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:49.223+0000 7f8d32d2b640 1 -- 172.21.15.143:0/3054297968 >> 172.21.15.143:0/3054297968 conn(0x7f8d2c0d3060 msgr2=0x7f8d2c0d3470 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:23:49.225 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:49.223+0000 7f8d32d2b640 1 -- 172.21.15.143:0/3054297968 shutdown_connections 2023-11-26T07:23:49.226 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:49.223+0000 7f8d32d2b640 1 -- 172.21.15.143:0/3054297968 wait complete. 2023-11-26T07:23:49.226 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:49.227+0000 7f8d32d2b640 1 Processor -- start 2023-11-26T07:23:49.226 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:49.227+0000 7f8d32d2b640 1 -- start start 2023-11-26T07:23:49.228 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:49.227+0000 7f8d32d2b640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f8d2c0b1860 0x7f8d2c1511e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:23:49.228 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:49.227+0000 7f8d32d2b640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f8d2c0cdd20 0x7f8d2c0cc240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:23:49.228 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:49.227+0000 7f8d32d2b640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f8d2c0cfa70 0x7f8d2c0cc780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:23:49.229 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:49.227+0000 7f8d32d2b640 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8d2c15e670 con 0x7f8d2c0cdd20 2023-11-26T07:23:49.229 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:49.227+0000 7f8d32d2b640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8d2c151720 con 0x7f8d2c0b1860 2023-11-26T07:23:49.229 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:49.227+0000 7f8d32d2b640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8d2c15dd60 con 0x7f8d2c0cfa70 2023-11-26T07:23:49.229 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:49.227+0000 7f8d31528640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f8d2c0cdd20 0x7f8d2c0cc240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:23:49.229 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:49.227+0000 7f8d31528640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f8d2c0cdd20 0x7f8d2c0cc240 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:36860/0 (socket says 172.21.15.143:36860) 2023-11-26T07:23:49.229 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:49.227+0000 7f8d31528640 1 -- 172.21.15.143:0/2559272340 learned_addr learned my addr 172.21.15.143:0/2559272340 (peer_addr_for_me v2:172.21.15.143:0/0) 2023-11-26T07:23:49.229 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:49.227+0000 7f8d31d29640 1 --2- 172.21.15.143:0/2559272340 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f8d2c0b1860 0x7f8d2c1511e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:23:49.229 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:49.227+0000 7f8d3252a640 1 --2- 172.21.15.143:0/2559272340 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f8d2c0cfa70 0x7f8d2c0cc780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:23:49.229 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:49.227+0000 7f8d31528640 1 -- 172.21.15.143:0/2559272340 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f8d2c0cfa70 msgr2=0x7f8d2c0cc780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:23:49.229 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:49.227+0000 7f8d31528640 1 --2- 172.21.15.143:0/2559272340 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f8d2c0cfa70 0x7f8d2c0cc780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:23:49.229 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:49.227+0000 7f8d31528640 1 -- 172.21.15.143:0/2559272340 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f8d2c0b1860 msgr2=0x7f8d2c1511e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:23:49.230 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:49.227+0000 7f8d31528640 1 --2- 172.21.15.143:0/2559272340 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f8d2c0b1860 0x7f8d2c1511e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:23:49.230 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:49.227+0000 7f8d31528640 1 -- 172.21.15.143:0/2559272340 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8d24009a70 con 0x7f8d2c0cdd20 2023-11-26T07:23:49.230 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:49.227+0000 7f8d31528640 1 --2- 172.21.15.143:0/2559272340 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f8d2c0cdd20 0x7f8d2c0cc240 secure :-1 s=READY pgs=1207 cs=0 l=1 rev1=1 crypto rx=0x7f8d1c00e7e0 tx=0x7f8d1c00eca0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:23:49.231 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:49.227+0000 7f8d227fc640 1 -- 172.21.15.143:0/2559272340 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8d1c00c6d0 con 0x7f8d2c0cdd20 2023-11-26T07:23:49.231 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:49.227+0000 7f8d227fc640 1 -- 172.21.15.143:0/2559272340 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8d1c00c830 con 0x7f8d2c0cdd20 2023-11-26T07:23:49.231 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:49.227+0000 7f8d227fc640 1 -- 172.21.15.143:0/2559272340 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8d1c00cbe0 con 0x7f8d2c0cdd20 2023-11-26T07:23:49.231 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:49.227+0000 7f8d32d2b640 1 -- 172.21.15.143:0/2559272340 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8d2c0cce00 con 0x7f8d2c0cdd20 2023-11-26T07:23:49.231 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:49.227+0000 7f8d32d2b640 1 -- 172.21.15.143:0/2559272340 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8d2c15e290 con 0x7f8d2c0cdd20 2023-11-26T07:23:49.231 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:49.227+0000 7f8d32d2b640 1 -- 172.21.15.143:0/2559272340 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8d2c15e7b0 con 0x7f8d2c0cdd20 2023-11-26T07:23:49.231 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:49.231+0000 7f8d227fc640 1 -- 172.21.15.143:0/2559272340 <== mon.0 v2:172.21.15.143:3300/0 4 ==== mgrmap(e 16) v1 ==== 85954+0+0 (secure 0 0 0) 0x7f8d1c00cd40 con 0x7f8d2c0cdd20 2023-11-26T07:23:49.232 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:49.231+0000 7f8d227fc640 1 --2- 172.21.15.143:0/2559272340 >> [v2:172.21.15.188:6836/3401128553,v1:172.21.15.188:6837/3401128553] conn(0x7f8d10066590 0x7f8d10068a50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:23:49.232 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:49.231+0000 7f8d227fc640 1 -- 172.21.15.143:0/2559272340 <== mon.0 v2:172.21.15.143:3300/0 5 ==== osd_map(91..91 src has 1..91) v4 ==== 5795+0+0 (secure 0 0 0) 0x7f8d1c083f90 con 0x7f8d2c0cdd20 2023-11-26T07:23:49.232 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:49.231+0000 7f8d31d29640 1 --2- 172.21.15.143:0/2559272340 >> [v2:172.21.15.188:6836/3401128553,v1:172.21.15.188:6837/3401128553] conn(0x7f8d10066590 0x7f8d10068a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:23:49.232 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:49.231+0000 7f8d31d29640 1 --2- 172.21.15.143:0/2559272340 >> [v2:172.21.15.188:6836/3401128553,v1:172.21.15.188:6837/3401128553] conn(0x7f8d10066590 0x7f8d10068a50 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7f8d280011f0 tx=0x7f8d28007d40 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:23:49.235 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:49.235+0000 7f8d227fc640 1 -- 172.21.15.143:0/2559272340 <== mon.0 v2:172.21.15.143:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172649 (secure 0 0 0) 0x7f8d2c15e7b0 con 0x7f8d2c0cdd20 2023-11-26T07:23:49.383 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:49.383+0000 7f8d32d2b640 1 -- 172.21.15.143:0/2559272340 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f8d2c15e7b0 con 0x7f8d2c0cdd20 2023-11-26T07:23:49.384 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:49.383+0000 7f8d227fc640 1 -- 172.21.15.143:0/2559272340 <== mon.0 v2:172.21.15.143:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v91) v1 ==== 100+0+11 (secure 0 0 0) 0x7f8d2c15e7b0 con 0x7f8d2c0cdd20 2023-11-26T07:23:49.384 INFO:teuthology.orchestra.run.smithi143.stdout:pg_num: 64 2023-11-26T07:23:49.386 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:49.383+0000 7f8d32d2b640 1 -- 172.21.15.143:0/2559272340 >> [v2:172.21.15.188:6836/3401128553,v1:172.21.15.188:6837/3401128553] conn(0x7f8d10066590 msgr2=0x7f8d10068a50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:23:49.386 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:49.383+0000 7f8d32d2b640 1 --2- 172.21.15.143:0/2559272340 >> [v2:172.21.15.188:6836/3401128553,v1:172.21.15.188:6837/3401128553] conn(0x7f8d10066590 0x7f8d10068a50 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7f8d280011f0 tx=0x7f8d28007d40 comp rx=0 tx=0).stop 2023-11-26T07:23:49.386 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:49.383+0000 7f8d32d2b640 1 -- 172.21.15.143:0/2559272340 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f8d2c0cdd20 msgr2=0x7f8d2c0cc240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:23:49.386 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:49.383+0000 7f8d32d2b640 1 --2- 172.21.15.143:0/2559272340 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f8d2c0cdd20 0x7f8d2c0cc240 secure :-1 s=READY pgs=1207 cs=0 l=1 rev1=1 crypto rx=0x7f8d1c00e7e0 tx=0x7f8d1c00eca0 comp rx=0 tx=0).stop 2023-11-26T07:23:49.386 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:49.383+0000 7f8d32d2b640 1 -- 172.21.15.143:0/2559272340 shutdown_connections 2023-11-26T07:23:49.386 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:49.383+0000 7f8d32d2b640 1 --2- 172.21.15.143:0/2559272340 >> [v2:172.21.15.188:6836/3401128553,v1:172.21.15.188:6837/3401128553] conn(0x7f8d10066590 0x7f8d10068a50 unknown :-1 s=CLOSED pgs=82 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:23:49.386 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:49.383+0000 7f8d32d2b640 1 --2- 172.21.15.143:0/2559272340 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f8d2c0cfa70 0x7f8d2c0cc780 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:23:49.386 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:49.383+0000 7f8d32d2b640 1 --2- 172.21.15.143:0/2559272340 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f8d2c0cdd20 0x7f8d2c0cc240 unknown :-1 s=CLOSED pgs=1207 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:23:49.386 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:49.383+0000 7f8d32d2b640 1 --2- 172.21.15.143:0/2559272340 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f8d2c0b1860 0x7f8d2c1511e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:23:49.386 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:49.383+0000 7f8d32d2b640 1 -- 172.21.15.143:0/2559272340 >> 172.21.15.143:0/2559272340 conn(0x7f8d2c0d3060 msgr2=0x7f8d2c06c480 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:23:49.387 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:49.383+0000 7f8d32d2b640 1 -- 172.21.15.143:0/2559272340 shutdown_connections 2023-11-26T07:23:49.387 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:49.383+0000 7f8d32d2b640 1 -- 172.21.15.143:0/2559272340 wait complete. 2023-11-26T07:23:49.405 DEBUG:teuthology.orchestra.run.smithi143:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2023-11-26T07:23:49.542 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:49.543+0000 7f36147c1640 1 Processor -- start 2023-11-26T07:23:49.543 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:49.543+0000 7f36147c1640 1 -- start start 2023-11-26T07:23:49.544 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:49.543+0000 7f36147c1640 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7f360c145fa0 0x7f360c072420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:23:49.544 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:49.543+0000 7f36147c1640 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7f360c06b190 0x7f360c146530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:23:49.545 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:49.543+0000 7f36147c1640 1 -- --> v1:172.21.15.188:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f360c146380 con 0x7f360c06a490 2023-11-26T07:23:49.545 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:49.543+0000 7f36147c1640 1 -- --> v2:172.21.15.143:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f360c071480 con 0x7f360c06b190 2023-11-26T07:23:49.545 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:49.543+0000 7f36147c1640 1 -- --> v2:172.21.15.188:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f360c072000 con 0x7f360c145fa0 2023-11-26T07:23:49.545 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:49.543+0000 7f3611d5c640 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7f360c06b190 0x7f360c146530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:23:49.545 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:49.543+0000 7f3611d5c640 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7f360c06b190 0x7f360c146530 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:36868/0 (socket says 172.21.15.143:36868) 2023-11-26T07:23:49.545 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:49.543+0000 7f3611d5c640 1 -- 172.21.15.143:0/1723706554 learned_addr learned my addr 172.21.15.143:0/1723706554 (peer_addr_for_me v2:172.21.15.143:0/0) 2023-11-26T07:23:49.545 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:49.543+0000 7f361255d640 1 --2- 172.21.15.143:0/1723706554 >> v2:172.21.15.188:3300/0 conn(0x7f360c145fa0 0x7f360c072420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:23:49.546 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:49.543+0000 7f3611d5c640 1 -- 172.21.15.143:0/1723706554 >> v2:172.21.15.188:3300/0 conn(0x7f360c145fa0 msgr2=0x7f360c072420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-26T07:23:49.546 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:49.543+0000 7f3611d5c640 1 --2- 172.21.15.143:0/1723706554 >> v2:172.21.15.188:3300/0 conn(0x7f360c145fa0 0x7f360c072420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:23:49.546 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:49.543+0000 7f3611d5c640 1 -- 172.21.15.143:0/1723706554 >> v1:172.21.15.188:6789/0 conn(0x7f360c06a490 legacy=0x7f360c06e890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:23:49.546 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:49.543+0000 7f3611d5c640 1 -- 172.21.15.143:0/1723706554 --> v2:172.21.15.143:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f360c072960 con 0x7f360c06b190 2023-11-26T07:23:49.546 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:49.543+0000 7f361155b640 1 -- 172.21.15.143:0/1723706554 <== mon.1 v1:172.21.15.188:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3935533109 0 0) 0x7f360c146380 con 0x7f360c06a490 2023-11-26T07:23:49.548 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:49.547+0000 7f3611d5c640 1 --2- 172.21.15.143:0/1723706554 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f360c06b190 0x7f360c146530 secure :-1 s=READY pgs=1208 cs=0 l=1 rev1=1 crypto rx=0x7f360c072b60 tx=0x7f35fc01c4d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=25737bd76bc1bc82 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:23:49.548 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:49.547+0000 7f361155b640 1 -- 172.21.15.143:0/1723706554 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f35fc0aa430 con 0x7f360c06b190 2023-11-26T07:23:49.548 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:49.547+0000 7f361155b640 1 -- 172.21.15.143:0/1723706554 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f35fc0aa590 con 0x7f360c06b190 2023-11-26T07:23:49.548 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:49.547+0000 7f361155b640 1 -- 172.21.15.143:0/1723706554 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f35fc0aa920 con 0x7f360c06b190 2023-11-26T07:23:49.549 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:49.547+0000 7f36147c1640 1 -- 172.21.15.143:0/1723706554 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f360c06b190 msgr2=0x7f360c146530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:23:49.549 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:49.547+0000 7f36147c1640 1 --2- 172.21.15.143:0/1723706554 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f360c06b190 0x7f360c146530 secure :-1 s=READY pgs=1208 cs=0 l=1 rev1=1 crypto rx=0x7f360c072b60 tx=0x7f35fc01c4d0 comp rx=0 tx=0).stop 2023-11-26T07:23:49.549 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:49.547+0000 7f36147c1640 1 -- 172.21.15.143:0/1723706554 shutdown_connections 2023-11-26T07:23:49.549 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:49.547+0000 7f36147c1640 1 --2- 172.21.15.143:0/1723706554 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f360c06b190 0x7f360c146530 unknown :-1 s=CLOSED pgs=1208 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:23:49.549 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:49.547+0000 7f36147c1640 1 --2- 172.21.15.143:0/1723706554 >> v2:172.21.15.188:3300/0 conn(0x7f360c145fa0 0x7f360c072420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:23:49.550 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:49.547+0000 7f36147c1640 1 -- 172.21.15.143:0/1723706554 >> 172.21.15.143:0/1723706554 conn(0x7f360c0cff00 msgr2=0x7f360c0d0310 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:23:49.550 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:49.547+0000 7f36147c1640 1 -- 172.21.15.143:0/1723706554 shutdown_connections 2023-11-26T07:23:49.550 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:49.547+0000 7f36147c1640 1 -- 172.21.15.143:0/1723706554 wait complete. 2023-11-26T07:23:49.550 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:49.551+0000 7f36147c1640 1 Processor -- start 2023-11-26T07:23:49.550 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:49.551+0000 7f36147c1640 1 -- start start 2023-11-26T07:23:49.552 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:49.551+0000 7f36147c1640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f360c06a490 0x7f360c15aac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:23:49.552 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:49.551+0000 7f36147c1640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f360c06b190 0x7f360c1442a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:23:49.552 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:49.551+0000 7f36147c1640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f360c145fa0 0x7f360c15f820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:23:49.553 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:49.551+0000 7f36147c1640 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f360c15b000 con 0x7f360c145fa0 2023-11-26T07:23:49.553 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:49.551+0000 7f36147c1640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f360c158c80 con 0x7f360c06b190 2023-11-26T07:23:49.553 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:49.551+0000 7f36147c1640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f360c158f60 con 0x7f360c06a490 2023-11-26T07:23:49.553 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:49.551+0000 7f3612d5e640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f360c145fa0 0x7f360c15f820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:23:49.553 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:49.551+0000 7f3612d5e640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f360c145fa0 0x7f360c15f820 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:36880/0 (socket says 172.21.15.143:36880) 2023-11-26T07:23:49.553 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:49.551+0000 7f3612d5e640 1 -- 172.21.15.143:0/1799686811 learned_addr learned my addr 172.21.15.143:0/1799686811 (peer_addr_for_me v2:172.21.15.143:0/0) 2023-11-26T07:23:49.553 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:49.551+0000 7f361255d640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f360c06a490 0x7f360c15aac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:23:49.553 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:49.551+0000 7f3611d5c640 1 --2- 172.21.15.143:0/1799686811 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f360c06b190 0x7f360c1442a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:23:49.553 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:49.551+0000 7f3612d5e640 1 -- 172.21.15.143:0/1799686811 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f360c06a490 msgr2=0x7f360c15aac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:23:49.553 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:49.551+0000 7f3612d5e640 1 --2- 172.21.15.143:0/1799686811 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f360c06a490 0x7f360c15aac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:23:49.553 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:49.551+0000 7f3612d5e640 1 -- 172.21.15.143:0/1799686811 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f360c06b190 msgr2=0x7f360c1442a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:23:49.554 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:49.551+0000 7f3612d5e640 1 --2- 172.21.15.143:0/1799686811 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f360c06b190 0x7f360c1442a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:23:49.554 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:49.551+0000 7f3612d5e640 1 -- 172.21.15.143:0/1799686811 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f35fc0094d0 con 0x7f360c145fa0 2023-11-26T07:23:49.554 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:49.551+0000 7f3612d5e640 1 --2- 172.21.15.143:0/1799686811 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f360c145fa0 0x7f360c15f820 secure :-1 s=READY pgs=1209 cs=0 l=1 rev1=1 crypto rx=0x7f3604001570 tx=0x7f360400cd20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:23:49.555 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:49.551+0000 7f3602ffd640 1 -- 172.21.15.143:0/1799686811 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3604003db0 con 0x7f360c145fa0 2023-11-26T07:23:49.555 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:49.551+0000 7f3602ffd640 1 -- 172.21.15.143:0/1799686811 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3604005070 con 0x7f360c145fa0 2023-11-26T07:23:49.555 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:49.551+0000 7f36147c1640 1 -- 172.21.15.143:0/1799686811 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f360c06ae30 con 0x7f360c145fa0 2023-11-26T07:23:49.555 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:49.551+0000 7f3602ffd640 1 -- 172.21.15.143:0/1799686811 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3604005420 con 0x7f360c145fa0 2023-11-26T07:23:49.555 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:49.551+0000 7f36147c1640 1 -- 172.21.15.143:0/1799686811 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f360c159520 con 0x7f360c145fa0 2023-11-26T07:23:49.555 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:49.551+0000 7f36147c1640 1 -- 172.21.15.143:0/1799686811 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f360c067680 con 0x7f360c145fa0 2023-11-26T07:23:49.555 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:49.555+0000 7f3602ffd640 1 -- 172.21.15.143:0/1799686811 <== mon.0 v2:172.21.15.143:3300/0 4 ==== mgrmap(e 16) v1 ==== 85954+0+0 (secure 0 0 0) 0x7f3604005580 con 0x7f360c145fa0 2023-11-26T07:23:49.556 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:49.555+0000 7f3602ffd640 1 --2- 172.21.15.143:0/1799686811 >> [v2:172.21.15.188:6836/3401128553,v1:172.21.15.188:6837/3401128553] conn(0x7f35f0066590 0x7f35f0068a50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:23:49.556 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:49.555+0000 7f3602ffd640 1 -- 172.21.15.143:0/1799686811 <== mon.0 v2:172.21.15.143:3300/0 5 ==== osd_map(91..91 src has 1..91) v4 ==== 5795+0+0 (secure 0 0 0) 0x7f3604083920 con 0x7f360c145fa0 2023-11-26T07:23:49.557 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:49.555+0000 7f361255d640 1 --2- 172.21.15.143:0/1799686811 >> [v2:172.21.15.188:6836/3401128553,v1:172.21.15.188:6837/3401128553] conn(0x7f35f0066590 0x7f35f0068a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:23:49.557 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:49.555+0000 7f361255d640 1 --2- 172.21.15.143:0/1799686811 >> [v2:172.21.15.188:6836/3401128553,v1:172.21.15.188:6837/3401128553] conn(0x7f35f0066590 0x7f35f0068a50 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7f360c163430 tx=0x7f3608009190 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:23:49.560 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:49.559+0000 7f3602ffd640 1 -- 172.21.15.143:0/1799686811 <== mon.0 v2:172.21.15.143:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172649 (secure 0 0 0) 0x7f3604050c40 con 0x7f360c145fa0 2023-11-26T07:23:49.721 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:49.719+0000 7f36147c1640 1 -- 172.21.15.143:0/1799686811 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) v1 -- 0x7f360c0726e0 con 0x7f360c145fa0 2023-11-26T07:23:49.721 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:49.719+0000 7f3602ffd640 1 -- 172.21.15.143:0/1799686811 <== mon.0 v2:172.21.15.143:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 2 entries v91) v1 ==== 80+0+124 (secure 0 0 0) 0x7f3604017070 con 0x7f360c145fa0 2023-11-26T07:23:49.722 INFO:teuthology.orchestra.run.smithi143.stdout:172.21.15.143:6835/4031724325 2023-11-27T07:23:44.794394+0000 2023-11-26T07:23:49.722 INFO:teuthology.orchestra.run.smithi143.stdout:172.21.15.143:6833/4031724325 2023-11-27T07:23:44.794394+0000 2023-11-26T07:23:49.722 INFO:teuthology.orchestra.run.smithi143.stderr:listed 2 entries 2023-11-26T07:23:49.723 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:49.723+0000 7f36147c1640 1 -- 172.21.15.143:0/1799686811 >> [v2:172.21.15.188:6836/3401128553,v1:172.21.15.188:6837/3401128553] conn(0x7f35f0066590 msgr2=0x7f35f0068a50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:23:49.724 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:49.723+0000 7f36147c1640 1 --2- 172.21.15.143:0/1799686811 >> [v2:172.21.15.188:6836/3401128553,v1:172.21.15.188:6837/3401128553] conn(0x7f35f0066590 0x7f35f0068a50 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7f360c163430 tx=0x7f3608009190 comp rx=0 tx=0).stop 2023-11-26T07:23:49.724 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:49.723+0000 7f36147c1640 1 -- 172.21.15.143:0/1799686811 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f360c145fa0 msgr2=0x7f360c15f820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:23:49.724 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:49.723+0000 7f36147c1640 1 --2- 172.21.15.143:0/1799686811 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f360c145fa0 0x7f360c15f820 secure :-1 s=READY pgs=1209 cs=0 l=1 rev1=1 crypto rx=0x7f3604001570 tx=0x7f360400cd20 comp rx=0 tx=0).stop 2023-11-26T07:23:49.724 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:49.723+0000 7f36147c1640 1 -- 172.21.15.143:0/1799686811 shutdown_connections 2023-11-26T07:23:49.724 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:49.723+0000 7f36147c1640 1 --2- 172.21.15.143:0/1799686811 >> [v2:172.21.15.188:6836/3401128553,v1:172.21.15.188:6837/3401128553] conn(0x7f35f0066590 0x7f35f0068a50 unknown :-1 s=CLOSED pgs=83 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:23:49.724 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:49.723+0000 7f36147c1640 1 --2- 172.21.15.143:0/1799686811 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f360c145fa0 0x7f360c15f820 unknown :-1 s=CLOSED pgs=1209 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:23:49.724 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:49.723+0000 7f36147c1640 1 --2- 172.21.15.143:0/1799686811 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f360c06b190 0x7f360c1442a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:23:49.724 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:49.723+0000 7f36147c1640 1 --2- 172.21.15.143:0/1799686811 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f360c06a490 0x7f360c15aac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:23:49.724 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:49.723+0000 7f36147c1640 1 -- 172.21.15.143:0/1799686811 >> 172.21.15.143:0/1799686811 conn(0x7f360c0cff00 msgr2=0x7f360c0b33f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:23:49.724 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:49.723+0000 7f36147c1640 1 -- 172.21.15.143:0/1799686811 shutdown_connections 2023-11-26T07:23:49.724 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:49.723+0000 7f36147c1640 1 -- 172.21.15.143:0/1799686811 wait complete. 2023-11-26T07:23:49.742 INFO:tasks.cephfs.fuse_mount:Running fusermount -u on ubuntu@smithi143.front.sepia.ceph.com... 2023-11-26T07:23:49.742 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-26T07:23:49.743 DEBUG:teuthology.orchestra.run.smithi143:> sudo fusermount -u /home/ubuntu/cephtest/mnt.0 2023-11-26T07:23:49.761 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi143.stderr:ceph-fuse[57362]: fuse finished with error 0 and tester_r 0 2023-11-26T07:23:49.764 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi143.front.sepia.ceph.com 2023-11-26T07:23:49.764 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-26T07:23:49.764 DEBUG:teuthology.orchestra.run.smithi143:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2023-11-26T07:23:49.932 DEBUG:teuthology.orchestra.run.smithi188:> set -ex 2023-11-26T07:23:49.932 DEBUG:teuthology.orchestra.run.smithi188:> dd if=/proc/self/mounts of=/dev/stdout 2023-11-26T07:23:49.942 DEBUG:teuthology.orchestra.run.smithi143:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-11-26T07:23:50.115 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:50.115+0000 7fb9fe757640 1 Processor -- start 2023-11-26T07:23:50.115 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:50.115+0000 7fb9fe757640 1 -- start start 2023-11-26T07:23:50.116 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:50.115+0000 7fb9fe757640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fb9f815d580 0x7fb9f80d3920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:23:50.116 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:50.115+0000 7fb9fe757640 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7fb9f80d3e60 0x7fb9f80d17f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:23:50.116 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:50.115+0000 7fb9fe757640 1 -- --> v1:172.21.15.143:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb9f80d21e0 con 0x7fb9f80d1d30 2023-11-26T07:23:50.116 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:50.115+0000 7fb9fe757640 1 -- --> v2:172.21.15.143:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb9f80d0bc0 con 0x7fb9f80d3e60 2023-11-26T07:23:50.117 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:50.115+0000 7fb9fe757640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb9f80d0320 con 0x7fb9f815d580 2023-11-26T07:23:50.117 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:50.115+0000 7fb9fccf4640 1 --1- >> v1:172.21.15.143:6789/0 conn(0x7fb9f80d1d30 0x7fb9f80cdce0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.143:6789/0 says I am v1:172.21.15.143:46442/0 (socket says 172.21.15.143:46442) 2023-11-26T07:23:50.117 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:50.115+0000 7fb9fccf4640 1 -- 172.21.15.143:0/28534784 learned_addr learned my addr 172.21.15.143:0/28534784 (peer_addr_for_me v1:172.21.15.143:0/0) 2023-11-26T07:23:50.117 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:50.115+0000 7fb9f77fe640 1 --2- 172.21.15.143:0/28534784 >> v2:172.21.15.143:3300/0 conn(0x7fb9f80d3e60 0x7fb9f80d17f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:23:50.118 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:50.115+0000 7fb9f7fff640 1 --2- 172.21.15.143:0/28534784 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fb9f815d580 0x7fb9f80d3920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:23:50.118 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:50.115+0000 7fb9f77fe640 1 -- 172.21.15.143:0/28534784 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fb9f815d580 msgr2=0x7fb9f80d3920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-26T07:23:50.118 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:50.115+0000 7fb9f77fe640 1 --2- 172.21.15.143:0/28534784 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fb9f815d580 0x7fb9f80d3920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:23:50.118 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:50.115+0000 7fb9f6ffd640 1 -- 172.21.15.143:0/28534784 <== mon.0 v1:172.21.15.143:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 804467768 0 0) 0x7fb9f80d21e0 con 0x7fb9f80d1d30 2023-11-26T07:23:50.118 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:50.115+0000 7fb9f77fe640 1 -- 172.21.15.143:0/28534784 >> v1:172.21.15.143:6789/0 conn(0x7fb9f80d1d30 legacy=0x7fb9f80cdce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:23:50.118 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:50.115+0000 7fb9f77fe640 1 -- 172.21.15.143:0/28534784 --> v2:172.21.15.143:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb9f80d0600 con 0x7fb9f80d3e60 2023-11-26T07:23:50.120 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:50.119+0000 7fb9f77fe640 1 --2- 172.21.15.143:0/28534784 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fb9f80d3e60 0x7fb9f80d17f0 secure :-1 s=READY pgs=1211 cs=0 l=1 rev1=1 crypto rx=0x7fb9ec0097c0 tx=0x7fb9ec01c450 comp rx=0 tx=0).ready entity=mon.0 client_cookie=44af590d1542b1cf server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:23:50.120 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:50.119+0000 7fb9f6ffd640 1 -- 172.21.15.143:0/28534784 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb9ec0ab650 con 0x7fb9f80d3e60 2023-11-26T07:23:50.121 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:50.119+0000 7fb9f6ffd640 1 -- 172.21.15.143:0/28534784 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb9ec0ab7b0 con 0x7fb9f80d3e60 2023-11-26T07:23:50.121 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:50.119+0000 7fb9f6ffd640 1 -- 172.21.15.143:0/28534784 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb9ec0abb40 con 0x7fb9f80d3e60 2023-11-26T07:23:50.122 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:50.119+0000 7fb9fe757640 1 -- 172.21.15.143:0/28534784 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fb9f80d3e60 msgr2=0x7fb9f80d17f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:23:50.122 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:50.119+0000 7fb9fe757640 1 --2- 172.21.15.143:0/28534784 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fb9f80d3e60 0x7fb9f80d17f0 secure :-1 s=READY pgs=1211 cs=0 l=1 rev1=1 crypto rx=0x7fb9ec0097c0 tx=0x7fb9ec01c450 comp rx=0 tx=0).stop 2023-11-26T07:23:50.122 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:50.119+0000 7fb9fe757640 1 -- 172.21.15.143:0/28534784 shutdown_connections 2023-11-26T07:23:50.122 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:50.119+0000 7fb9fe757640 1 --2- 172.21.15.143:0/28534784 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fb9f80d3e60 0x7fb9f80d17f0 unknown :-1 s=CLOSED pgs=1211 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:23:50.122 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:50.119+0000 7fb9fe757640 1 --2- 172.21.15.143:0/28534784 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fb9f815d580 0x7fb9f80d3920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:23:50.122 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:50.119+0000 7fb9fe757640 1 -- 172.21.15.143:0/28534784 >> 172.21.15.143:0/28534784 conn(0x7fb9f809c550 msgr2=0x7fb9f80d3210 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:23:50.123 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:50.119+0000 7fb9fe757640 1 -- 172.21.15.143:0/28534784 shutdown_connections 2023-11-26T07:23:50.123 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:50.119+0000 7fb9fe757640 1 -- 172.21.15.143:0/28534784 wait complete. 2023-11-26T07:23:50.123 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:50.123+0000 7fb9fe757640 1 Processor -- start 2023-11-26T07:23:50.123 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:50.123+0000 7fb9fe757640 1 -- start start 2023-11-26T07:23:50.126 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:50.123+0000 7fb9fe757640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fb9f80d1d30 0x7fb9f8160500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:23:50.126 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:50.123+0000 7fb9fe757640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fb9f80d3e60 0x7fb9f80c9010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:23:50.126 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:50.123+0000 7fb9fe757640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fb9f815d580 0x7fb9f80b79c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:23:50.126 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:50.123+0000 7fb9fe757640 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb9f815e220 con 0x7fb9f80d3e60 2023-11-26T07:23:50.126 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:50.123+0000 7fb9fe757640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb9f8154490 con 0x7fb9f80d1d30 2023-11-26T07:23:50.126 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:50.123+0000 7fb9fe757640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb9f8155aa0 con 0x7fb9f815d580 2023-11-26T07:23:50.127 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:50.123+0000 7fb9f77fe640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fb9f80d3e60 0x7fb9f80c9010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:23:50.127 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:50.123+0000 7fb9f77fe640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fb9f80d3e60 0x7fb9f80c9010 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:36900/0 (socket says 172.21.15.143:36900) 2023-11-26T07:23:50.127 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:50.123+0000 7fb9f77fe640 1 -- 172.21.15.143:0/3989246761 learned_addr learned my addr 172.21.15.143:0/3989246761 (peer_addr_for_me v2:172.21.15.143:0/0) 2023-11-26T07:23:50.127 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:50.123+0000 7fb9fccf4640 1 --2- 172.21.15.143:0/3989246761 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fb9f815d580 0x7fb9f80b79c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:23:50.127 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:50.123+0000 7fb9f7fff640 1 --2- 172.21.15.143:0/3989246761 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fb9f80d1d30 0x7fb9f8160500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:23:50.127 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:50.123+0000 7fb9f77fe640 1 -- 172.21.15.143:0/3989246761 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fb9f815d580 msgr2=0x7fb9f80b79c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:23:50.127 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:50.123+0000 7fb9f77fe640 1 --2- 172.21.15.143:0/3989246761 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fb9f815d580 0x7fb9f80b79c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:23:50.128 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:50.123+0000 7fb9f77fe640 1 -- 172.21.15.143:0/3989246761 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fb9f80d1d30 msgr2=0x7fb9f8160500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:23:50.128 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:50.123+0000 7fb9f77fe640 1 --2- 172.21.15.143:0/3989246761 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fb9f80d1d30 0x7fb9f8160500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:23:50.128 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:50.123+0000 7fb9f77fe640 1 -- 172.21.15.143:0/3989246761 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb9ec009420 con 0x7fb9f80d3e60 2023-11-26T07:23:50.128 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:50.123+0000 7fb9f7fff640 1 --2- 172.21.15.143:0/3989246761 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fb9f80d1d30 0x7fb9f8160500 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-26T07:23:50.128 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:50.123+0000 7fb9f77fe640 1 --2- 172.21.15.143:0/3989246761 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fb9f80d3e60 0x7fb9f80c9010 secure :-1 s=READY pgs=1212 cs=0 l=1 rev1=1 crypto rx=0x7fb9ec0ad040 tx=0x7fb9ec01ac90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:23:50.128 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:50.123+0000 7fb9f4ff9640 1 -- 172.21.15.143:0/3989246761 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb9ec0b5810 con 0x7fb9f80d3e60 2023-11-26T07:23:50.128 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:50.123+0000 7fb9f4ff9640 1 -- 172.21.15.143:0/3989246761 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb9ec0b5970 con 0x7fb9f80d3e60 2023-11-26T07:23:50.129 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:50.123+0000 7fb9f4ff9640 1 -- 172.21.15.143:0/3989246761 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb9ec0b5d20 con 0x7fb9f80d3e60 2023-11-26T07:23:50.129 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:50.123+0000 7fb9fe757640 1 -- 172.21.15.143:0/3989246761 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb9f8160a40 con 0x7fb9f80d3e60 2023-11-26T07:23:50.129 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:50.123+0000 7fb9fe757640 1 -- 172.21.15.143:0/3989246761 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb9f815dca0 con 0x7fb9f80d3e60 2023-11-26T07:23:50.130 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:50.127+0000 7fb9fe757640 1 -- 172.21.15.143:0/3989246761 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb9f81545d0 con 0x7fb9f80d3e60 2023-11-26T07:23:50.130 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:50.127+0000 7fb9f4ff9640 1 -- 172.21.15.143:0/3989246761 <== mon.0 v2:172.21.15.143:3300/0 4 ==== mgrmap(e 16) v1 ==== 85954+0+0 (secure 0 0 0) 0x7fb9ec0bf410 con 0x7fb9f80d3e60 2023-11-26T07:23:50.130 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:50.127+0000 7fb9f4ff9640 1 --2- 172.21.15.143:0/3989246761 >> [v2:172.21.15.188:6836/3401128553,v1:172.21.15.188:6837/3401128553] conn(0x7fb9e4066590 0x7fb9e4068a50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:23:50.130 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:50.127+0000 7fb9f4ff9640 1 -- 172.21.15.143:0/3989246761 <== mon.0 v2:172.21.15.143:3300/0 5 ==== osd_map(91..91 src has 1..91) v4 ==== 5795+0+0 (secure 0 0 0) 0x7fb9ec11f1c0 con 0x7fb9f80d3e60 2023-11-26T07:23:50.130 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:50.127+0000 7fb9f7fff640 1 --2- 172.21.15.143:0/3989246761 >> [v2:172.21.15.188:6836/3401128553,v1:172.21.15.188:6837/3401128553] conn(0x7fb9e4066590 0x7fb9e4068a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:23:50.131 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:50.127+0000 7fb9f7fff640 1 --2- 172.21.15.143:0/3989246761 >> [v2:172.21.15.188:6836/3401128553,v1:172.21.15.188:6837/3401128553] conn(0x7fb9e4066590 0x7fb9e4068a50 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7fb9f80b1fb0 tx=0x7fb9e8008040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:23:50.132 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:50.131+0000 7fb9f4ff9640 1 -- 172.21.15.143:0/3989246761 <== mon.0 v2:172.21.15.143:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172649 (secure 0 0 0) 0x7fb9f81545d0 con 0x7fb9f80d3e60 2023-11-26T07:23:50.283 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:50.283+0000 7fb9fe757640 1 -- 172.21.15.143:0/3989246761 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fb9f81545d0 con 0x7fb9f80d3e60 2023-11-26T07:23:50.284 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:50.283+0000 7fb9f4ff9640 1 -- 172.21.15.143:0/3989246761 <== mon.0 v2:172.21.15.143:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v91) v1 ==== 74+0+16727 (secure 0 0 0) 0x7fb9f81545d0 con 0x7fb9f80d3e60 2023-11-26T07:23:50.284 INFO:teuthology.orchestra.run.smithi143.stdout: 2023-11-26T07:23:50.285 INFO:teuthology.orchestra.run.smithi143.stdout:{"epoch":91,"fsid":"f5033060-2469-4a07-95fe-2b76f5404bc1","created":"2023-11-26T07:17:28.613663+0000","modified":"2023-11-26T07:23:47.807341+0000","last_up_change":"2023-11-26T07:17:41.157813+0000","last_in_change":"2023-11-26T07:17:31.181658+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":15,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-11-26T07:17:41.758597+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":12,"pool_name":"cephfs_metadata","create_time":"2023-11-26T07:23:26.724016+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"83","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":13,"pool_name":"cephfs_data","create_time":"2023-11-26T07:23:28.019663+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"84","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"ebbee2c6-023e-44b9-9f1d-6f7bfd309730","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6817","nonce":639807369},{"type":"v1","addr":"172.21.15.143:6819","nonce":639807369}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6821","nonce":639807369},{"type":"v1","addr":"172.21.15.143:6823","nonce":639807369}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6830","nonce":639807369},{"type":"v1","addr":"172.21.15.143:6831","nonce":639807369}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6825","nonce":639807369},{"type":"v1","addr":"172.21.15.143:6828","nonce":639807369}]},"public_addr":"172.21.15.143:6819/639807369","cluster_addr":"172.21.15.143:6823/639807369","heartbeat_back_addr":"172.21.15.143:6831/639807369","heartbeat_front_addr":"172.21.15.143:6828/639807369","state":["exists","up"]},{"osd":1,"uuid":"fec4b8be-7122-49e9-9a19-5850a0700001","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6816","nonce":1144942530},{"type":"v1","addr":"172.21.15.143:6818","nonce":1144942530}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6820","nonce":1144942530},{"type":"v1","addr":"172.21.15.143:6822","nonce":1144942530}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6827","nonce":1144942530},{"type":"v1","addr":"172.21.15.143:6829","nonce":1144942530}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6824","nonce":1144942530},{"type":"v1","addr":"172.21.15.143:6826","nonce":1144942530}]},"public_addr":"172.21.15.143:6818/1144942530","cluster_addr":"172.21.15.143:6822/1144942530","heartbeat_back_addr":"172.21.15.143:6829/1144942530","heartbeat_front_addr":"172.21.15.143:6826/1144942530","state":["exists","up"]},{"osd":2,"uuid":"2888af5b-f933-46ca-8f95-6dd494ff43f6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6808","nonce":3909095186},{"type":"v1","addr":"172.21.15.143:6809","nonce":3909095186}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6810","nonce":3909095186},{"type":"v1","addr":"172.21.15.143:6811","nonce":3909095186}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6814","nonce":3909095186},{"type":"v1","addr":"172.21.15.143:6815","nonce":3909095186}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6812","nonce":3909095186},{"type":"v1","addr":"172.21.15.143:6813","nonce":3909095186}]},"public_addr":"172.21.15.143:6809/3909095186","cluster_addr":"172.21.15.143:6811/3909095186","heartbeat_back_addr":"172.21.15.143:6815/3909095186","heartbeat_front_addr":"172.21.15.143:6813/3909095186","state":["exists","up"]},{"osd":3,"uuid":"d1f52435-7bc1-4081-9550-9cd1ba55f549","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6800","nonce":3854210991},{"type":"v1","addr":"172.21.15.143:6801","nonce":3854210991}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6802","nonce":3854210991},{"type":"v1","addr":"172.21.15.143:6803","nonce":3854210991}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6806","nonce":3854210991},{"type":"v1","addr":"172.21.15.143:6807","nonce":3854210991}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6804","nonce":3854210991},{"type":"v1","addr":"172.21.15.143:6805","nonce":3854210991}]},"public_addr":"172.21.15.143:6801/3854210991","cluster_addr":"172.21.15.143:6803/3854210991","heartbeat_back_addr":"172.21.15.143:6807/3854210991","heartbeat_front_addr":"172.21.15.143:6805/3854210991","state":["exists","up"]},{"osd":4,"uuid":"e2ae8b6e-6e25-49b2-bcbf-aacce68f06a3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6800","nonce":3924606902},{"type":"v1","addr":"172.21.15.188:6801","nonce":3924606902}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6802","nonce":3924606902},{"type":"v1","addr":"172.21.15.188:6803","nonce":3924606902}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6806","nonce":3924606902},{"type":"v1","addr":"172.21.15.188:6807","nonce":3924606902}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6804","nonce":3924606902},{"type":"v1","addr":"172.21.15.188:6805","nonce":3924606902}]},"public_addr":"172.21.15.188:6801/3924606902","cluster_addr":"172.21.15.188:6803/3924606902","heartbeat_back_addr":"172.21.15.188:6807/3924606902","heartbeat_front_addr":"172.21.15.188:6805/3924606902","state":["exists","up"]},{"osd":5,"uuid":"3cec8a32-7698-4c36-8e89-e4ba11934f33","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6813","nonce":186841544},{"type":"v1","addr":"172.21.15.188:6815","nonce":186841544}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6817","nonce":186841544},{"type":"v1","addr":"172.21.15.188:6818","nonce":186841544}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6822","nonce":186841544},{"type":"v1","addr":"172.21.15.188:6823","nonce":186841544}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6820","nonce":186841544},{"type":"v1","addr":"172.21.15.188:6821","nonce":186841544}]},"public_addr":"172.21.15.188:6815/186841544","cluster_addr":"172.21.15.188:6818/186841544","heartbeat_back_addr":"172.21.15.188:6823/186841544","heartbeat_front_addr":"172.21.15.188:6821/186841544","state":["exists","up"]},{"osd":6,"uuid":"58a65d5d-e090-473a-81b4-ad38e85029f9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6808","nonce":2153499405},{"type":"v1","addr":"172.21.15.188:6809","nonce":2153499405}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6810","nonce":2153499405},{"type":"v1","addr":"172.21.15.188:6811","nonce":2153499405}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6816","nonce":2153499405},{"type":"v1","addr":"172.21.15.188:6819","nonce":2153499405}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6812","nonce":2153499405},{"type":"v1","addr":"172.21.15.188:6814","nonce":2153499405}]},"public_addr":"172.21.15.188:6809/2153499405","cluster_addr":"172.21.15.188:6811/2153499405","heartbeat_back_addr":"172.21.15.188:6819/2153499405","heartbeat_front_addr":"172.21.15.188:6814/2153499405","state":["exists","up"]},{"osd":7,"uuid":"ea5b7f70-3554-4cf8-ac30-f7c59b80678d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6824","nonce":2722096729},{"type":"v1","addr":"172.21.15.188:6825","nonce":2722096729}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6826","nonce":2722096729},{"type":"v1","addr":"172.21.15.188:6827","nonce":2722096729}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6830","nonce":2722096729},{"type":"v1","addr":"172.21.15.188:6831","nonce":2722096729}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6828","nonce":2722096729},{"type":"v1","addr":"172.21.15.188:6829","nonce":2722096729}]},"public_addr":"172.21.15.188:6825/2722096729","cluster_addr":"172.21.15.188:6827/2722096729","heartbeat_back_addr":"172.21.15.188:6831/2722096729","heartbeat_front_addr":"172.21.15.188:6829/2722096729","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-26T07:17:36.995032+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-26T07:17:36.932266+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-26T07:17:36.829304+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-26T07:17:36.804297+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-26T07:17:36.667892+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-26T07:17:36.964868+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-26T07:17:36.827332+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-26T07:17:36.893179+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.143:6835/4031724325":"2023-11-27T07:23:44.794394+0000","172.21.15.143:6833/4031724325":"2023-11-27T07:23:44.794394+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-11-26T07:23:50.287 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:50.287+0000 7fb9fe757640 1 -- 172.21.15.143:0/3989246761 >> [v2:172.21.15.188:6836/3401128553,v1:172.21.15.188:6837/3401128553] conn(0x7fb9e4066590 msgr2=0x7fb9e4068a50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:23:50.287 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:50.287+0000 7fb9fe757640 1 --2- 172.21.15.143:0/3989246761 >> [v2:172.21.15.188:6836/3401128553,v1:172.21.15.188:6837/3401128553] conn(0x7fb9e4066590 0x7fb9e4068a50 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7fb9f80b1fb0 tx=0x7fb9e8008040 comp rx=0 tx=0).stop 2023-11-26T07:23:50.287 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:50.287+0000 7fb9fe757640 1 -- 172.21.15.143:0/3989246761 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fb9f80d3e60 msgr2=0x7fb9f80c9010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:23:50.287 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:50.287+0000 7fb9fe757640 1 --2- 172.21.15.143:0/3989246761 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fb9f80d3e60 0x7fb9f80c9010 secure :-1 s=READY pgs=1212 cs=0 l=1 rev1=1 crypto rx=0x7fb9ec0ad040 tx=0x7fb9ec01ac90 comp rx=0 tx=0).stop 2023-11-26T07:23:50.287 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:50.287+0000 7fb9fe757640 1 -- 172.21.15.143:0/3989246761 shutdown_connections 2023-11-26T07:23:50.287 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:50.287+0000 7fb9fe757640 1 --2- 172.21.15.143:0/3989246761 >> [v2:172.21.15.188:6836/3401128553,v1:172.21.15.188:6837/3401128553] conn(0x7fb9e4066590 0x7fb9e4068a50 unknown :-1 s=CLOSED pgs=84 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:23:50.287 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:50.287+0000 7fb9fe757640 1 --2- 172.21.15.143:0/3989246761 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fb9f815d580 0x7fb9f80b79c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:23:50.287 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:50.287+0000 7fb9fe757640 1 --2- 172.21.15.143:0/3989246761 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fb9f80d3e60 0x7fb9f80c9010 unknown :-1 s=CLOSED pgs=1212 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:23:50.287 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:50.287+0000 7fb9fe757640 1 --2- 172.21.15.143:0/3989246761 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fb9f80d1d30 0x7fb9f8160500 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:23:50.287 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:50.287+0000 7fb9fe757640 1 -- 172.21.15.143:0/3989246761 >> 172.21.15.143:0/3989246761 conn(0x7fb9f809c550 msgr2=0x7fb9f80b2460 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:23:50.288 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:50.287+0000 7fb9fe757640 1 -- 172.21.15.143:0/3989246761 shutdown_connections 2023-11-26T07:23:50.288 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:50.287+0000 7fb9fe757640 1 -- 172.21.15.143:0/3989246761 wait complete. 2023-11-26T07:23:50.306 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-11-26T07:17:41.758597+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 12, 'pool_name': 'cephfs_metadata', 'create_time': '2023-11-26T07:23:26.724016+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '83', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 2, 'score_stable': 2, 'optimal_score': 1, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 13, 'pool_name': 'cephfs_data', 'create_time': '2023-11-26T07:23:28.019663+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '84', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.3799999952316284, 'score_stable': 1.3799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.3799999952316284, 'raw_score_stable': 1.3799999952316284, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2023-11-26T07:23:50.307 DEBUG:teuthology.orchestra.run.smithi143:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-11-26T07:23:50.447 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:50.447+0000 7fbb77c5a640 1 Processor -- start 2023-11-26T07:23:50.447 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:50.447+0000 7fbb77c5a640 1 -- start start 2023-11-26T07:23:50.448 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:50.447+0000 7fbb77c5a640 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7fbb70143ac0 0x7fbb70070370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:23:50.449 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:50.447+0000 7fbb77c5a640 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7fbb700cf990 0x7fbb700cd1b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:23:50.449 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:50.447+0000 7fbb77c5a640 1 -- --> v1:172.21.15.143:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbb700cc730 con 0x7fbb700708b0 2023-11-26T07:23:50.449 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:50.447+0000 7fbb77c5a640 1 -- --> v2:172.21.15.143:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fbb70143ea0 con 0x7fbb700cf990 2023-11-26T07:23:50.449 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:50.447+0000 7fbb77c5a640 1 -- --> v2:172.21.15.188:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fbb70070ee0 con 0x7fbb70143ac0 2023-11-26T07:23:50.449 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:50.447+0000 7fbb761f7640 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7fbb700cf990 0x7fbb700cd1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:23:50.449 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:50.447+0000 7fbb751f5640 1 --1- >> v1:172.21.15.143:6789/0 conn(0x7fbb700708b0 0x7fbb700cf280 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.143:6789/0 says I am v1:172.21.15.143:46444/0 (socket says 172.21.15.143:46444) 2023-11-26T07:23:50.449 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:50.447+0000 7fbb761f7640 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7fbb700cf990 0x7fbb700cd1b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:36908/0 (socket says 172.21.15.143:36908) 2023-11-26T07:23:50.450 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:50.447+0000 7fbb751f5640 1 -- 172.21.15.143:0/1138230512 learned_addr learned my addr 172.21.15.143:0/1138230512 (peer_addr_for_me v1:172.21.15.143:0/0) 2023-11-26T07:23:50.450 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:50.447+0000 7fbb759f6640 1 --2- 172.21.15.143:0/1138230512 >> v2:172.21.15.188:3300/0 conn(0x7fbb70143ac0 0x7fbb70070370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:23:50.450 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:50.447+0000 7fbb761f7640 1 -- 172.21.15.143:0/1138230512 >> v2:172.21.15.188:3300/0 conn(0x7fbb70143ac0 msgr2=0x7fbb70070370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-26T07:23:50.451 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:50.447+0000 7fbb761f7640 1 --2- 172.21.15.143:0/1138230512 >> v2:172.21.15.188:3300/0 conn(0x7fbb70143ac0 0x7fbb70070370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:23:50.451 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:50.447+0000 7fbb761f7640 1 -- 172.21.15.143:0/1138230512 >> v1:172.21.15.143:6789/0 conn(0x7fbb700708b0 legacy=0x7fbb700cf280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:23:50.451 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:50.447+0000 7fbb761f7640 1 -- 172.21.15.143:0/1138230512 --> v2:172.21.15.143:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbb700711c0 con 0x7fbb700cf990 2023-11-26T07:23:50.451 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:50.447+0000 7fbb759f6640 1 --2- 172.21.15.143:0/1138230512 >> v2:172.21.15.188:3300/0 conn(0x7fbb70143ac0 0x7fbb70070370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-26T07:23:50.451 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:50.451+0000 7fbb761f7640 1 --2- 172.21.15.143:0/1138230512 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fbb700cf990 0x7fbb700cd1b0 secure :-1 s=READY pgs=1214 cs=0 l=1 rev1=1 crypto rx=0x7fbb7006dca0 tx=0x7fbb6801c910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c7a8bd3a8f9effb3 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:23:50.452 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:50.451+0000 7fbb749f4640 1 -- 172.21.15.143:0/1138230512 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbb680a58e0 con 0x7fbb700cf990 2023-11-26T07:23:50.452 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:50.451+0000 7fbb749f4640 1 -- 172.21.15.143:0/1138230512 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbb680a5a40 con 0x7fbb700cf990 2023-11-26T07:23:50.452 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:50.451+0000 7fbb749f4640 1 -- 172.21.15.143:0/1138230512 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbb680a5dd0 con 0x7fbb700cf990 2023-11-26T07:23:50.453 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:50.451+0000 7fbb77c5a640 1 -- 172.21.15.143:0/1138230512 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fbb700cf990 msgr2=0x7fbb700cd1b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:23:50.453 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:50.451+0000 7fbb77c5a640 1 --2- 172.21.15.143:0/1138230512 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fbb700cf990 0x7fbb700cd1b0 secure :-1 s=READY pgs=1214 cs=0 l=1 rev1=1 crypto rx=0x7fbb7006dca0 tx=0x7fbb6801c910 comp rx=0 tx=0).stop 2023-11-26T07:23:50.453 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:50.451+0000 7fbb77c5a640 1 -- 172.21.15.143:0/1138230512 shutdown_connections 2023-11-26T07:23:50.454 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:50.451+0000 7fbb77c5a640 1 --2- 172.21.15.143:0/1138230512 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fbb700cf990 0x7fbb700cd1b0 unknown :-1 s=CLOSED pgs=1214 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:23:50.454 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:50.451+0000 7fbb77c5a640 1 --2- 172.21.15.143:0/1138230512 >> v2:172.21.15.188:3300/0 conn(0x7fbb70143ac0 0x7fbb70070370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:23:50.454 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:50.451+0000 7fbb77c5a640 1 -- 172.21.15.143:0/1138230512 >> 172.21.15.143:0/1138230512 conn(0x7fbb7006a890 msgr2=0x7fbb7006aca0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:23:50.454 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:50.451+0000 7fbb77c5a640 1 -- 172.21.15.143:0/1138230512 shutdown_connections 2023-11-26T07:23:50.454 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:50.451+0000 7fbb77c5a640 1 -- 172.21.15.143:0/1138230512 wait complete. 2023-11-26T07:23:50.455 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:50.455+0000 7fbb77c5a640 1 Processor -- start 2023-11-26T07:23:50.455 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:50.455+0000 7fbb77c5a640 1 -- start start 2023-11-26T07:23:50.456 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:50.455+0000 7fbb77c5a640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fbb700708b0 0x7fbb70159c90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:23:50.456 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:50.455+0000 7fbb77c5a640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fbb700cf990 0x7fbb701f5bb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:23:50.456 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:50.455+0000 7fbb77c5a640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fbb70143ac0 0x7fbb70140f60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:23:50.457 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:50.455+0000 7fbb77c5a640 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbb700aebe0 con 0x7fbb700708b0 2023-11-26T07:23:50.457 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:50.455+0000 7fbb77c5a640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbb700aeec0 con 0x7fbb700cf990 2023-11-26T07:23:50.457 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:50.455+0000 7fbb77c5a640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbb700af1a0 con 0x7fbb70143ac0 2023-11-26T07:23:50.457 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:50.455+0000 7fbb759f6640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fbb700708b0 0x7fbb70159c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:23:50.457 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:50.455+0000 7fbb759f6640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fbb700708b0 0x7fbb70159c90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:36914/0 (socket says 172.21.15.143:36914) 2023-11-26T07:23:50.457 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:50.455+0000 7fbb759f6640 1 -- 172.21.15.143:0/2139693253 learned_addr learned my addr 172.21.15.143:0/2139693253 (peer_addr_for_me v2:172.21.15.143:0/0) 2023-11-26T07:23:50.457 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:50.455+0000 7fbb751f5640 1 --2- 172.21.15.143:0/2139693253 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fbb700cf990 0x7fbb701f5bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:23:50.458 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:50.455+0000 7fbb761f7640 1 --2- 172.21.15.143:0/2139693253 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fbb70143ac0 0x7fbb70140f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:23:50.458 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:50.455+0000 7fbb759f6640 1 -- 172.21.15.143:0/2139693253 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fbb70143ac0 msgr2=0x7fbb70140f60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:23:50.458 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:50.455+0000 7fbb759f6640 1 --2- 172.21.15.143:0/2139693253 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fbb70143ac0 0x7fbb70140f60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:23:50.458 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:50.455+0000 7fbb759f6640 1 -- 172.21.15.143:0/2139693253 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fbb700cf990 msgr2=0x7fbb701f5bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:23:50.458 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:50.455+0000 7fbb759f6640 1 --2- 172.21.15.143:0/2139693253 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fbb700cf990 0x7fbb701f5bb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:23:50.458 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:50.455+0000 7fbb759f6640 1 -- 172.21.15.143:0/2139693253 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbb68009a70 con 0x7fbb700708b0 2023-11-26T07:23:50.460 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:50.455+0000 7fbb759f6640 1 --2- 172.21.15.143:0/2139693253 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fbb700708b0 0x7fbb70159c90 secure :-1 s=READY pgs=1215 cs=0 l=1 rev1=1 crypto rx=0x7fbb6c00c7f0 tx=0x7fbb6c00ccb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:23:50.460 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:50.455+0000 7fbb667fc640 1 -- 172.21.15.143:0/2139693253 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbb6c007820 con 0x7fbb700708b0 2023-11-26T07:23:50.460 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:50.455+0000 7fbb667fc640 1 -- 172.21.15.143:0/2139693253 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbb6c007980 con 0x7fbb700708b0 2023-11-26T07:23:50.460 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:50.455+0000 7fbb667fc640 1 -- 172.21.15.143:0/2139693253 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbb6c007d30 con 0x7fbb700708b0 2023-11-26T07:23:50.460 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:50.455+0000 7fbb77c5a640 1 -- 172.21.15.143:0/2139693253 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbb700af480 con 0x7fbb700708b0 2023-11-26T07:23:50.460 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:50.455+0000 7fbb77c5a640 1 -- 172.21.15.143:0/2139693253 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbb7015a1d0 con 0x7fbb700708b0 2023-11-26T07:23:50.461 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:50.455+0000 7fbb77c5a640 1 -- 172.21.15.143:0/2139693253 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbb70067680 con 0x7fbb700708b0 2023-11-26T07:23:50.461 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:50.459+0000 7fbb667fc640 1 -- 172.21.15.143:0/2139693253 <== mon.0 v2:172.21.15.143:3300/0 4 ==== mgrmap(e 16) v1 ==== 85954+0+0 (secure 0 0 0) 0x7fbb6c0113f0 con 0x7fbb700708b0 2023-11-26T07:23:50.461 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:50.459+0000 7fbb667fc640 1 --2- 172.21.15.143:0/2139693253 >> [v2:172.21.15.188:6836/3401128553,v1:172.21.15.188:6837/3401128553] conn(0x7fbb54066590 0x7fbb54068a50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:23:50.461 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:50.459+0000 7fbb667fc640 1 -- 172.21.15.143:0/2139693253 <== mon.0 v2:172.21.15.143:3300/0 5 ==== osd_map(91..91 src has 1..91) v4 ==== 5795+0+0 (secure 0 0 0) 0x7fbb6c084350 con 0x7fbb700708b0 2023-11-26T07:23:50.461 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:50.459+0000 7fbb751f5640 1 --2- 172.21.15.143:0/2139693253 >> [v2:172.21.15.188:6836/3401128553,v1:172.21.15.188:6837/3401128553] conn(0x7fbb54066590 0x7fbb54068a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:23:50.462 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:50.459+0000 7fbb751f5640 1 --2- 172.21.15.143:0/2139693253 >> [v2:172.21.15.188:6836/3401128553,v1:172.21.15.188:6837/3401128553] conn(0x7fbb54066590 0x7fbb54068a50 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7fbb7013c3c0 tx=0x7fbb60007e90 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:23:50.463 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:50.463+0000 7fbb667fc640 1 -- 172.21.15.143:0/2139693253 <== mon.0 v2:172.21.15.143:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172649 (secure 0 0 0) 0x7fbb6c051690 con 0x7fbb700708b0 2023-11-26T07:23:50.613 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:50.611+0000 7fbb77c5a640 1 -- 172.21.15.143:0/2139693253 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7fbb700af930 con 0x7fbb700708b0 2023-11-26T07:23:50.614 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:50.611+0000 7fbb667fc640 1 -- 172.21.15.143:0/2139693253 <== mon.0 v2:172.21.15.143:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v91) v1 ==== 93+0+10 (secure 0 0 0) 0x7fbb6c054cb0 con 0x7fbb700708b0 2023-11-26T07:23:50.614 INFO:teuthology.orchestra.run.smithi143.stdout:pg_num: 1 2023-11-26T07:23:50.616 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:50.615+0000 7fbb77c5a640 1 -- 172.21.15.143:0/2139693253 >> [v2:172.21.15.188:6836/3401128553,v1:172.21.15.188:6837/3401128553] conn(0x7fbb54066590 msgr2=0x7fbb54068a50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:23:50.616 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:50.615+0000 7fbb77c5a640 1 --2- 172.21.15.143:0/2139693253 >> [v2:172.21.15.188:6836/3401128553,v1:172.21.15.188:6837/3401128553] conn(0x7fbb54066590 0x7fbb54068a50 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7fbb7013c3c0 tx=0x7fbb60007e90 comp rx=0 tx=0).stop 2023-11-26T07:23:50.617 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:50.615+0000 7fbb77c5a640 1 -- 172.21.15.143:0/2139693253 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fbb700708b0 msgr2=0x7fbb70159c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:23:50.617 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:50.615+0000 7fbb77c5a640 1 --2- 172.21.15.143:0/2139693253 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fbb700708b0 0x7fbb70159c90 secure :-1 s=READY pgs=1215 cs=0 l=1 rev1=1 crypto rx=0x7fbb6c00c7f0 tx=0x7fbb6c00ccb0 comp rx=0 tx=0).stop 2023-11-26T07:23:50.617 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:50.615+0000 7fbb77c5a640 1 -- 172.21.15.143:0/2139693253 shutdown_connections 2023-11-26T07:23:50.617 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:50.615+0000 7fbb77c5a640 1 --2- 172.21.15.143:0/2139693253 >> [v2:172.21.15.188:6836/3401128553,v1:172.21.15.188:6837/3401128553] conn(0x7fbb54066590 0x7fbb54068a50 unknown :-1 s=CLOSED pgs=85 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:23:50.617 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:50.615+0000 7fbb77c5a640 1 --2- 172.21.15.143:0/2139693253 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fbb70143ac0 0x7fbb70140f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:23:50.617 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:50.615+0000 7fbb77c5a640 1 --2- 172.21.15.143:0/2139693253 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fbb700cf990 0x7fbb701f5bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:23:50.617 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:50.615+0000 7fbb77c5a640 1 --2- 172.21.15.143:0/2139693253 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fbb700708b0 0x7fbb70159c90 unknown :-1 s=CLOSED pgs=1215 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:23:50.618 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:50.615+0000 7fbb77c5a640 1 -- 172.21.15.143:0/2139693253 >> 172.21.15.143:0/2139693253 conn(0x7fbb7006a890 msgr2=0x7fbb700adde0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:23:50.618 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:50.615+0000 7fbb77c5a640 1 -- 172.21.15.143:0/2139693253 shutdown_connections 2023-11-26T07:23:50.618 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:50.615+0000 7fbb77c5a640 1 -- 172.21.15.143:0/2139693253 wait complete. 2023-11-26T07:23:50.635 DEBUG:teuthology.orchestra.run.smithi143:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-11-26T07:23:50.776 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:50.775+0000 7f8a1bcf7640 1 Processor -- start 2023-11-26T07:23:50.776 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:50.775+0000 7f8a1bcf7640 1 -- start start 2023-11-26T07:23:50.778 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:50.775+0000 7f8a1bcf7640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f8a1413e290 0x7f8a140cf280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:23:50.778 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:50.775+0000 7f8a1bcf7640 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7f8a140cf7c0 0x7f8a1415efa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:23:50.778 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:50.775+0000 7f8a1bcf7640 1 -- --> v1:172.21.15.143:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8a1406c520 con 0x7f8a1415f4e0 2023-11-26T07:23:50.779 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:50.775+0000 7f8a1bcf7640 1 -- --> v2:172.21.15.143:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8a140cfc40 con 0x7f8a140cf7c0 2023-11-26T07:23:50.779 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:50.775+0000 7f8a1bcf7640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8a1406c800 con 0x7f8a1413e290 2023-11-26T07:23:50.779 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:50.775+0000 7f8a19292640 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7f8a140cf7c0 0x7f8a1415efa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:23:50.779 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:50.775+0000 7f8a1a294640 1 --1- >> v1:172.21.15.143:6789/0 conn(0x7f8a1415f4e0 0x7f8a141408e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.143:6789/0 says I am v1:172.21.15.143:46456/0 (socket says 172.21.15.143:46456) 2023-11-26T07:23:50.779 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:50.775+0000 7f8a1a294640 1 -- 172.21.15.143:0/3909004304 learned_addr learned my addr 172.21.15.143:0/3909004304 (peer_addr_for_me v1:172.21.15.143:0/0) 2023-11-26T07:23:50.779 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:50.775+0000 7f8a19a93640 1 --2- 172.21.15.143:0/3909004304 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f8a1413e290 0x7f8a140cf280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:23:50.779 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:50.779+0000 7f8a19292640 1 -- 172.21.15.143:0/3909004304 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f8a1413e290 msgr2=0x7f8a140cf280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-26T07:23:50.780 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:50.779+0000 7f8a19292640 1 --2- 172.21.15.143:0/3909004304 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f8a1413e290 0x7f8a140cf280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:23:50.780 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:50.779+0000 7f8a19292640 1 -- 172.21.15.143:0/3909004304 >> v1:172.21.15.143:6789/0 conn(0x7f8a1415f4e0 legacy=0x7f8a141408e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:23:50.780 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:50.779+0000 7f8a19292640 1 -- 172.21.15.143:0/3909004304 --> v2:172.21.15.143:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8a1406cae0 con 0x7f8a140cf7c0 2023-11-26T07:23:50.780 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:50.779+0000 7f8a18a91640 1 -- 172.21.15.143:0/3909004304 <== mon.0 v1:172.21.15.143:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1139199179 0 0) 0x7f8a1406c520 con 0x7f8a1415f4e0 2023-11-26T07:23:50.781 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:50.779+0000 7f8a19292640 1 --2- 172.21.15.143:0/3909004304 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f8a140cf7c0 0x7f8a1415efa0 secure :-1 s=READY pgs=1217 cs=0 l=1 rev1=1 crypto rx=0x7f8a040097c0 tx=0x7f8a0401c450 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f7b9e234ba805290 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:23:50.782 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:50.779+0000 7f8a18a91640 1 -- 172.21.15.143:0/3909004304 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8a040ab650 con 0x7f8a140cf7c0 2023-11-26T07:23:50.782 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:50.779+0000 7f8a18a91640 1 -- 172.21.15.143:0/3909004304 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8a040ab7b0 con 0x7f8a140cf7c0 2023-11-26T07:23:50.782 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:50.779+0000 7f8a18a91640 1 -- 172.21.15.143:0/3909004304 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8a040abb40 con 0x7f8a140cf7c0 2023-11-26T07:23:50.782 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:50.779+0000 7f8a1bcf7640 1 -- 172.21.15.143:0/3909004304 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f8a140cf7c0 msgr2=0x7f8a1415efa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:23:50.783 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:50.779+0000 7f8a1bcf7640 1 --2- 172.21.15.143:0/3909004304 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f8a140cf7c0 0x7f8a1415efa0 secure :-1 s=READY pgs=1217 cs=0 l=1 rev1=1 crypto rx=0x7f8a040097c0 tx=0x7f8a0401c450 comp rx=0 tx=0).stop 2023-11-26T07:23:50.783 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:50.779+0000 7f8a1bcf7640 1 -- 172.21.15.143:0/3909004304 shutdown_connections 2023-11-26T07:23:50.783 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:50.779+0000 7f8a1bcf7640 1 --2- 172.21.15.143:0/3909004304 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f8a140cf7c0 0x7f8a1415efa0 unknown :-1 s=CLOSED pgs=1217 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:23:50.783 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:50.779+0000 7f8a1bcf7640 1 --2- 172.21.15.143:0/3909004304 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f8a1413e290 0x7f8a140cf280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:23:50.783 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:50.779+0000 7f8a1bcf7640 1 -- 172.21.15.143:0/3909004304 >> 172.21.15.143:0/3909004304 conn(0x7f8a1406a890 msgr2=0x7f8a1406aca0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:23:50.783 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:50.783+0000 7f8a1bcf7640 1 -- 172.21.15.143:0/3909004304 shutdown_connections 2023-11-26T07:23:50.783 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:50.783+0000 7f8a1bcf7640 1 -- 172.21.15.143:0/3909004304 wait complete. 2023-11-26T07:23:50.784 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:50.783+0000 7f8a1bcf7640 1 Processor -- start 2023-11-26T07:23:50.784 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:50.783+0000 7f8a1bcf7640 1 -- start start 2023-11-26T07:23:50.786 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:50.783+0000 7f8a1bcf7640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f8a140cf7c0 0x7f8a1415a150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:23:50.786 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:50.783+0000 7f8a1bcf7640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f8a1413e290 0x7f8a1415a690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:23:50.787 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:50.783+0000 7f8a1bcf7640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f8a1415f4e0 0x7f8a1413edf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:23:50.787 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:50.783+0000 7f8a1bcf7640 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8a141f66a0 con 0x7f8a140cf7c0 2023-11-26T07:23:50.787 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:50.783+0000 7f8a1bcf7640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8a141f6980 con 0x7f8a1415f4e0 2023-11-26T07:23:50.787 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:50.783+0000 7f8a1bcf7640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8a141f6c60 con 0x7f8a1413e290 2023-11-26T07:23:50.787 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:50.783+0000 7f8a19a93640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f8a140cf7c0 0x7f8a1415a150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:23:50.787 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:50.783+0000 7f8a19a93640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f8a140cf7c0 0x7f8a1415a150 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:36926/0 (socket says 172.21.15.143:36926) 2023-11-26T07:23:50.787 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:50.783+0000 7f8a19a93640 1 -- 172.21.15.143:0/588481540 learned_addr learned my addr 172.21.15.143:0/588481540 (peer_addr_for_me v2:172.21.15.143:0/0) 2023-11-26T07:23:50.788 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:50.783+0000 7f8a19292640 1 --2- 172.21.15.143:0/588481540 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f8a1413e290 0x7f8a1415a690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:23:50.788 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:50.783+0000 7f8a1a294640 1 --2- 172.21.15.143:0/588481540 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f8a1415f4e0 0x7f8a1413edf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:23:50.788 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:50.783+0000 7f8a19a93640 1 -- 172.21.15.143:0/588481540 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f8a1413e290 msgr2=0x7f8a1415a690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:23:50.788 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:50.783+0000 7f8a19a93640 1 --2- 172.21.15.143:0/588481540 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f8a1413e290 0x7f8a1415a690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:23:50.788 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:50.783+0000 7f8a19a93640 1 -- 172.21.15.143:0/588481540 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f8a1415f4e0 msgr2=0x7f8a1413edf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:23:50.788 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:50.783+0000 7f8a19a93640 1 --2- 172.21.15.143:0/588481540 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f8a1415f4e0 0x7f8a1413edf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:23:50.788 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:50.783+0000 7f8a19a93640 1 -- 172.21.15.143:0/588481540 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8a04009420 con 0x7f8a140cf7c0 2023-11-26T07:23:50.789 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:50.783+0000 7f8a1a294640 1 --2- 172.21.15.143:0/588481540 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f8a1415f4e0 0x7f8a1413edf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-26T07:23:50.789 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:50.783+0000 7f8a19a93640 1 --2- 172.21.15.143:0/588481540 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f8a140cf7c0 0x7f8a1415a150 secure :-1 s=READY pgs=1218 cs=0 l=1 rev1=1 crypto rx=0x7f8a1000e930 tx=0x7f8a1000edf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:23:50.789 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:50.787+0000 7f8a0a7fc640 1 -- 172.21.15.143:0/588481540 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8a10009690 con 0x7f8a140cf7c0 2023-11-26T07:23:50.789 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:50.787+0000 7f8a0a7fc640 1 -- 172.21.15.143:0/588481540 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8a1000ca00 con 0x7f8a140cf7c0 2023-11-26T07:23:50.789 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:50.787+0000 7f8a0a7fc640 1 -- 172.21.15.143:0/588481540 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8a1000cd70 con 0x7f8a140cf7c0 2023-11-26T07:23:50.789 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:50.787+0000 7f8a1bcf7640 1 -- 172.21.15.143:0/588481540 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8a141f6f40 con 0x7f8a140cf7c0 2023-11-26T07:23:50.789 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:50.787+0000 7f8a1bcf7640 1 -- 172.21.15.143:0/588481540 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8a141f62c0 con 0x7f8a140cf7c0 2023-11-26T07:23:50.790 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:50.787+0000 7f8a1bcf7640 1 -- 172.21.15.143:0/588481540 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8a14067680 con 0x7f8a140cf7c0 2023-11-26T07:23:50.791 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:50.787+0000 7f8a0a7fc640 1 -- 172.21.15.143:0/588481540 <== mon.0 v2:172.21.15.143:3300/0 4 ==== mgrmap(e 16) v1 ==== 85954+0+0 (secure 0 0 0) 0x7f8a10013470 con 0x7f8a140cf7c0 2023-11-26T07:23:50.791 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:50.787+0000 7f8a0a7fc640 1 --2- 172.21.15.143:0/588481540 >> [v2:172.21.15.188:6836/3401128553,v1:172.21.15.188:6837/3401128553] conn(0x7f89f8066590 0x7f89f8068a50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:23:50.791 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:50.787+0000 7f8a0a7fc640 1 -- 172.21.15.143:0/588481540 <== mon.0 v2:172.21.15.143:3300/0 5 ==== osd_map(91..91 src has 1..91) v4 ==== 5795+0+0 (secure 0 0 0) 0x7f8a100842c0 con 0x7f8a140cf7c0 2023-11-26T07:23:50.791 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:50.787+0000 7f8a19292640 1 --2- 172.21.15.143:0/588481540 >> [v2:172.21.15.188:6836/3401128553,v1:172.21.15.188:6837/3401128553] conn(0x7f89f8066590 0x7f89f8068a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:23:50.791 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:50.791+0000 7f8a19292640 1 --2- 172.21.15.143:0/588481540 >> [v2:172.21.15.188:6836/3401128553,v1:172.21.15.188:6837/3401128553] conn(0x7f89f8066590 0x7f89f8068a50 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7f8a040af000 tx=0x7f8a040a7000 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:23:50.793 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:50.791+0000 7f8a0a7fc640 1 -- 172.21.15.143:0/588481540 <== mon.0 v2:172.21.15.143:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172649 (secure 0 0 0) 0x7f8a10051680 con 0x7f8a140cf7c0 2023-11-26T07:23:50.943 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:50.943+0000 7f8a1bcf7640 1 -- 172.21.15.143:0/588481540 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f8a14153510 con 0x7f8a140cf7c0 2023-11-26T07:23:50.943 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:50.943+0000 7f8a0a7fc640 1 -- 172.21.15.143:0/588481540 <== mon.0 v2:172.21.15.143:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v91) v1 ==== 104+0+11 (secure 0 0 0) 0x7f8a10054ca0 con 0x7f8a140cf7c0 2023-11-26T07:23:50.944 INFO:teuthology.orchestra.run.smithi143.stdout:pg_num: 32 2023-11-26T07:23:50.945 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:50.943+0000 7f8a1bcf7640 1 -- 172.21.15.143:0/588481540 >> [v2:172.21.15.188:6836/3401128553,v1:172.21.15.188:6837/3401128553] conn(0x7f89f8066590 msgr2=0x7f89f8068a50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:23:50.946 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:50.943+0000 7f8a1bcf7640 1 --2- 172.21.15.143:0/588481540 >> [v2:172.21.15.188:6836/3401128553,v1:172.21.15.188:6837/3401128553] conn(0x7f89f8066590 0x7f89f8068a50 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7f8a040af000 tx=0x7f8a040a7000 comp rx=0 tx=0).stop 2023-11-26T07:23:50.946 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:50.943+0000 7f8a1bcf7640 1 -- 172.21.15.143:0/588481540 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f8a140cf7c0 msgr2=0x7f8a1415a150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:23:50.946 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:50.943+0000 7f8a1bcf7640 1 --2- 172.21.15.143:0/588481540 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f8a140cf7c0 0x7f8a1415a150 secure :-1 s=READY pgs=1218 cs=0 l=1 rev1=1 crypto rx=0x7f8a1000e930 tx=0x7f8a1000edf0 comp rx=0 tx=0).stop 2023-11-26T07:23:50.946 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:50.943+0000 7f8a1bcf7640 1 -- 172.21.15.143:0/588481540 shutdown_connections 2023-11-26T07:23:50.946 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:50.943+0000 7f8a1bcf7640 1 --2- 172.21.15.143:0/588481540 >> [v2:172.21.15.188:6836/3401128553,v1:172.21.15.188:6837/3401128553] conn(0x7f89f8066590 0x7f89f8068a50 unknown :-1 s=CLOSED pgs=86 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:23:50.946 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:50.943+0000 7f8a1bcf7640 1 --2- 172.21.15.143:0/588481540 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f8a1415f4e0 0x7f8a1413edf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:23:50.947 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:50.943+0000 7f8a1bcf7640 1 --2- 172.21.15.143:0/588481540 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f8a1413e290 0x7f8a1415a690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:23:50.947 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:50.943+0000 7f8a1bcf7640 1 --2- 172.21.15.143:0/588481540 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f8a140cf7c0 0x7f8a1415a150 unknown :-1 s=CLOSED pgs=1218 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:23:50.947 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:50.943+0000 7f8a1bcf7640 1 -- 172.21.15.143:0/588481540 >> 172.21.15.143:0/588481540 conn(0x7f8a1406a890 msgr2=0x7f8a14139860 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:23:50.947 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:50.943+0000 7f8a1bcf7640 1 -- 172.21.15.143:0/588481540 shutdown_connections 2023-11-26T07:23:50.947 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:50.943+0000 7f8a1bcf7640 1 -- 172.21.15.143:0/588481540 wait complete. 2023-11-26T07:23:50.965 DEBUG:teuthology.orchestra.run.smithi143:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-11-26T07:23:51.106 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:51.103+0000 7fec4609a640 1 Processor -- start 2023-11-26T07:23:51.106 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:51.107+0000 7fec4609a640 1 -- start start 2023-11-26T07:23:51.107 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:51.107+0000 7fec4609a640 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7fec400cf280 0x7fec400cf710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:23:51.108 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:51.107+0000 7fec4609a640 1 -- --> v1:172.21.15.143:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fec400cc730 con 0x7fec40143ac0 2023-11-26T07:23:51.108 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:51.107+0000 7fec4609a640 1 -- --> v1:172.21.15.188:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fec40070ee0 con 0x7fec400cd1b0 2023-11-26T07:23:51.108 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:51.107+0000 7fec4609a640 1 -- --> v2:172.21.15.143:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fec40143ea0 con 0x7fec400cf280 2023-11-26T07:23:51.108 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:51.107+0000 7fec3f7fe640 1 --1- >> v1:172.21.15.143:6789/0 conn(0x7fec40143ac0 0x7fec40070370 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.143:6789/0 says I am v1:172.21.15.143:46464/0 (socket says 172.21.15.143:46464) 2023-11-26T07:23:51.108 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:51.107+0000 7fec3f7fe640 1 -- 172.21.15.143:0/286344711 learned_addr learned my addr 172.21.15.143:0/286344711 (peer_addr_for_me v1:172.21.15.143:0/0) 2023-11-26T07:23:51.108 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:51.107+0000 7fec3effd640 1 --2- 172.21.15.143:0/286344711 >> v2:172.21.15.143:3300/0 conn(0x7fec400cf280 0x7fec400cf710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:23:51.109 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:51.107+0000 7fec3e7fc640 1 -- 172.21.15.143:0/286344711 <== mon.0 v1:172.21.15.143:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3434767411 0 0) 0x7fec400cc730 con 0x7fec40143ac0 2023-11-26T07:23:51.109 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:51.107+0000 7fec3e7fc640 1 -- 172.21.15.143:0/286344711 --> v1:172.21.15.143:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fec28003530 con 0x7fec40143ac0 2023-11-26T07:23:51.109 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:51.107+0000 7fec3effd640 1 -- 172.21.15.143:0/286344711 >> v1:172.21.15.188:6789/0 conn(0x7fec400cd1b0 legacy=0x7fec400cd5c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:23:51.110 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:51.107+0000 7fec3effd640 1 -- 172.21.15.143:0/286344711 >> v1:172.21.15.143:6789/0 conn(0x7fec40143ac0 legacy=0x7fec40070370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:23:51.110 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:51.107+0000 7fec3effd640 1 -- 172.21.15.143:0/286344711 --> v2:172.21.15.143:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fec400711c0 con 0x7fec400cf280 2023-11-26T07:23:51.111 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:51.111+0000 7fec3effd640 1 --2- 172.21.15.143:0/286344711 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fec400cf280 0x7fec400cf710 secure :-1 s=READY pgs=1220 cs=0 l=1 rev1=1 crypto rx=0x7fec340097c0 tx=0x7fec3401c480 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7059368aa0b45f05 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:23:51.111 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:51.111+0000 7fec3e7fc640 1 -- 172.21.15.143:0/286344711 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fec340a8050 con 0x7fec400cf280 2023-11-26T07:23:51.112 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:51.111+0000 7fec3e7fc640 1 -- 172.21.15.143:0/286344711 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fec340ac5f0 con 0x7fec400cf280 2023-11-26T07:23:51.112 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:51.111+0000 7fec3e7fc640 1 -- 172.21.15.143:0/286344711 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fec340ac980 con 0x7fec400cf280 2023-11-26T07:23:51.112 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:51.111+0000 7fec4609a640 1 -- 172.21.15.143:0/286344711 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fec400cf280 msgr2=0x7fec400cf710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:23:51.112 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:51.111+0000 7fec4609a640 1 --2- 172.21.15.143:0/286344711 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fec400cf280 0x7fec400cf710 secure :-1 s=READY pgs=1220 cs=0 l=1 rev1=1 crypto rx=0x7fec340097c0 tx=0x7fec3401c480 comp rx=0 tx=0).stop 2023-11-26T07:23:51.113 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:51.111+0000 7fec4609a640 1 -- 172.21.15.143:0/286344711 shutdown_connections 2023-11-26T07:23:51.113 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:51.111+0000 7fec4609a640 1 --2- 172.21.15.143:0/286344711 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fec400cf280 0x7fec400cf710 unknown :-1 s=CLOSED pgs=1220 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:23:51.113 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:51.111+0000 7fec4609a640 1 -- 172.21.15.143:0/286344711 >> 172.21.15.143:0/286344711 conn(0x7fec4006a890 msgr2=0x7fec4006aca0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:23:51.113 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:51.111+0000 7fec4609a640 1 -- 172.21.15.143:0/286344711 shutdown_connections 2023-11-26T07:23:51.113 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:51.111+0000 7fec4609a640 1 -- 172.21.15.143:0/286344711 wait complete. 2023-11-26T07:23:51.114 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:51.111+0000 7fec4609a640 1 Processor -- start 2023-11-26T07:23:51.114 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:51.111+0000 7fec4609a640 1 -- start start 2023-11-26T07:23:51.116 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:51.115+0000 7fec4609a640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fec400cd1b0 0x7fec4013d360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:23:51.116 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:51.115+0000 7fec4609a640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fec400cf280 0x7fec400aeb10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:23:51.116 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:51.115+0000 7fec4609a640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fec40143ac0 0x7fec4015b650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:23:51.117 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:51.115+0000 7fec4609a640 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fec400af5b0 con 0x7fec400cd1b0 2023-11-26T07:23:51.117 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:51.115+0000 7fec4609a640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fec400af890 con 0x7fec40143ac0 2023-11-26T07:23:51.117 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:51.115+0000 7fec4609a640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fec4013ec40 con 0x7fec400cf280 2023-11-26T07:23:51.117 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:51.115+0000 7fec3f7fe640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fec400cd1b0 0x7fec4013d360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:23:51.117 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:51.115+0000 7fec3f7fe640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fec400cd1b0 0x7fec4013d360 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:36942/0 (socket says 172.21.15.143:36942) 2023-11-26T07:23:51.117 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:51.115+0000 7fec3f7fe640 1 -- 172.21.15.143:0/172586270 learned_addr learned my addr 172.21.15.143:0/172586270 (peer_addr_for_me v2:172.21.15.143:0/0) 2023-11-26T07:23:51.118 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:51.115+0000 7fec3ffff640 1 --2- 172.21.15.143:0/172586270 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fec40143ac0 0x7fec4015b650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:23:51.118 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:51.115+0000 7fec3effd640 1 --2- 172.21.15.143:0/172586270 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fec400cf280 0x7fec400aeb10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:23:51.118 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:51.115+0000 7fec3f7fe640 1 -- 172.21.15.143:0/172586270 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fec400cf280 msgr2=0x7fec400aeb10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:23:51.118 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:51.115+0000 7fec3f7fe640 1 --2- 172.21.15.143:0/172586270 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fec400cf280 0x7fec400aeb10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:23:51.118 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:51.115+0000 7fec3f7fe640 1 -- 172.21.15.143:0/172586270 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fec40143ac0 msgr2=0x7fec4015b650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:23:51.118 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:51.115+0000 7fec3f7fe640 1 --2- 172.21.15.143:0/172586270 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fec40143ac0 0x7fec4015b650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:23:51.118 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:51.115+0000 7fec3f7fe640 1 -- 172.21.15.143:0/172586270 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fec34009420 con 0x7fec400cd1b0 2023-11-26T07:23:51.119 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:51.115+0000 7fec3f7fe640 1 --2- 172.21.15.143:0/172586270 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fec400cd1b0 0x7fec4013d360 secure :-1 s=READY pgs=1221 cs=0 l=1 rev1=1 crypto rx=0x7fec3000c420 tx=0x7fec3000c8e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:23:51.119 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:51.115+0000 7fec27fff640 1 -- 172.21.15.143:0/172586270 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fec30005cf0 con 0x7fec400cd1b0 2023-11-26T07:23:51.119 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:51.115+0000 7fec27fff640 1 -- 172.21.15.143:0/172586270 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fec30005e50 con 0x7fec400cd1b0 2023-11-26T07:23:51.119 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:51.115+0000 7fec4609a640 1 -- 172.21.15.143:0/172586270 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fec4013ef20 con 0x7fec400cd1b0 2023-11-26T07:23:51.119 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:51.115+0000 7fec27fff640 1 -- 172.21.15.143:0/172586270 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fec300072d0 con 0x7fec400cd1b0 2023-11-26T07:23:51.119 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:51.115+0000 7fec4609a640 1 -- 172.21.15.143:0/172586270 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fec400afb70 con 0x7fec400cd1b0 2023-11-26T07:23:51.120 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:51.115+0000 7fec4609a640 1 -- 172.21.15.143:0/172586270 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fec4006c4a0 con 0x7fec400cd1b0 2023-11-26T07:23:51.120 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:51.115+0000 7fec27fff640 1 -- 172.21.15.143:0/172586270 <== mon.0 v2:172.21.15.143:3300/0 4 ==== mgrmap(e 16) v1 ==== 85954+0+0 (secure 0 0 0) 0x7fec30007430 con 0x7fec400cd1b0 2023-11-26T07:23:51.121 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:51.119+0000 7fec27fff640 1 --2- 172.21.15.143:0/172586270 >> [v2:172.21.15.188:6836/3401128553,v1:172.21.15.188:6837/3401128553] conn(0x7fec2c0665c0 0x7fec2c068a80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:23:51.121 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:51.119+0000 7fec27fff640 1 -- 172.21.15.143:0/172586270 <== mon.0 v2:172.21.15.143:3300/0 5 ==== osd_map(91..91 src has 1..91) v4 ==== 5795+0+0 (secure 0 0 0) 0x7fec30083580 con 0x7fec400cd1b0 2023-11-26T07:23:51.121 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:51.119+0000 7fec3effd640 1 --2- 172.21.15.143:0/172586270 >> [v2:172.21.15.188:6836/3401128553,v1:172.21.15.188:6837/3401128553] conn(0x7fec2c0665c0 0x7fec2c068a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:23:51.121 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:51.119+0000 7fec3effd640 1 --2- 172.21.15.143:0/172586270 >> [v2:172.21.15.188:6836/3401128553,v1:172.21.15.188:6837/3401128553] conn(0x7fec2c0665c0 0x7fec2c068a80 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7fec340ae000 tx=0x7fec34008040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:23:51.123 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:51.123+0000 7fec27fff640 1 -- 172.21.15.143:0/172586270 <== mon.0 v2:172.21.15.143:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172649 (secure 0 0 0) 0x7fec30050930 con 0x7fec400cd1b0 2023-11-26T07:23:51.273 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:51.271+0000 7fec4609a640 1 -- 172.21.15.143:0/172586270 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7fec40067680 con 0x7fec400cd1b0 2023-11-26T07:23:51.273 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:51.271+0000 7fec27fff640 1 -- 172.21.15.143:0/172586270 <== mon.0 v2:172.21.15.143:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v91) v1 ==== 100+0+11 (secure 0 0 0) 0x7fec30053f50 con 0x7fec400cd1b0 2023-11-26T07:23:51.274 INFO:teuthology.orchestra.run.smithi143.stdout:pg_num: 64 2023-11-26T07:23:51.276 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:51.275+0000 7fec4609a640 1 -- 172.21.15.143:0/172586270 >> [v2:172.21.15.188:6836/3401128553,v1:172.21.15.188:6837/3401128553] conn(0x7fec2c0665c0 msgr2=0x7fec2c068a80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:23:51.276 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:51.275+0000 7fec4609a640 1 --2- 172.21.15.143:0/172586270 >> [v2:172.21.15.188:6836/3401128553,v1:172.21.15.188:6837/3401128553] conn(0x7fec2c0665c0 0x7fec2c068a80 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7fec340ae000 tx=0x7fec34008040 comp rx=0 tx=0).stop 2023-11-26T07:23:51.276 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:51.275+0000 7fec4609a640 1 -- 172.21.15.143:0/172586270 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fec400cd1b0 msgr2=0x7fec4013d360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:23:51.276 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:51.275+0000 7fec4609a640 1 --2- 172.21.15.143:0/172586270 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fec400cd1b0 0x7fec4013d360 secure :-1 s=READY pgs=1221 cs=0 l=1 rev1=1 crypto rx=0x7fec3000c420 tx=0x7fec3000c8e0 comp rx=0 tx=0).stop 2023-11-26T07:23:51.276 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:51.275+0000 7fec4609a640 1 -- 172.21.15.143:0/172586270 shutdown_connections 2023-11-26T07:23:51.276 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:51.275+0000 7fec4609a640 1 --2- 172.21.15.143:0/172586270 >> [v2:172.21.15.188:6836/3401128553,v1:172.21.15.188:6837/3401128553] conn(0x7fec2c0665c0 0x7fec2c068a80 unknown :-1 s=CLOSED pgs=87 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:23:51.277 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:51.275+0000 7fec4609a640 1 --2- 172.21.15.143:0/172586270 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fec40143ac0 0x7fec4015b650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:23:51.277 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:51.275+0000 7fec4609a640 1 --2- 172.21.15.143:0/172586270 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fec400cf280 0x7fec400aeb10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:23:51.277 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:51.275+0000 7fec4609a640 1 --2- 172.21.15.143:0/172586270 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fec400cd1b0 0x7fec4013d360 unknown :-1 s=CLOSED pgs=1221 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:23:51.277 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:51.275+0000 7fec4609a640 1 -- 172.21.15.143:0/172586270 >> 172.21.15.143:0/172586270 conn(0x7fec4006a890 msgr2=0x7fec400aca70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:23:51.277 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:51.275+0000 7fec4609a640 1 -- 172.21.15.143:0/172586270 shutdown_connections 2023-11-26T07:23:51.277 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:51.275+0000 7fec4609a640 1 -- 172.21.15.143:0/172586270 wait complete. 2023-11-26T07:23:51.295 DEBUG:teuthology.orchestra.run.smithi143:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2023-11-26T07:23:51.434 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:51.431+0000 7f3e708ad640 1 Processor -- start 2023-11-26T07:23:51.434 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:51.435+0000 7f3e708ad640 1 -- start start 2023-11-26T07:23:51.436 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:51.435+0000 7f3e708ad640 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7f3e6c0cf990 0x7f3e6c15efa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:23:51.436 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:51.435+0000 7f3e708ad640 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7f3e6c15f4e0 0x7f3e6c0cd7b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:23:51.436 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:51.435+0000 7f3e708ad640 1 -- --> v1:172.21.15.188:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3e6c06c520 con 0x7f3e6c13e290 2023-11-26T07:23:51.437 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:51.435+0000 7f3e708ad640 1 -- --> v2:172.21.15.143:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3e6c06c800 con 0x7f3e6c0cf990 2023-11-26T07:23:51.437 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:51.435+0000 7f3e708ad640 1 -- --> v2:172.21.15.188:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3e6c06cae0 con 0x7f3e6c15f4e0 2023-11-26T07:23:51.437 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:51.435+0000 7f3e63fff640 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7f3e6c0cf990 0x7f3e6c15efa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:23:51.437 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:51.435+0000 7f3e63fff640 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7f3e6c0cf990 0x7f3e6c15efa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:36950/0 (socket says 172.21.15.143:36950) 2023-11-26T07:23:51.437 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:51.435+0000 7f3e63fff640 1 -- 172.21.15.143:0/1768409027 learned_addr learned my addr 172.21.15.143:0/1768409027 (peer_addr_for_me v2:172.21.15.143:0/0) 2023-11-26T07:23:51.437 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:51.435+0000 7f3e6ad9d640 1 --2- 172.21.15.143:0/1768409027 >> v2:172.21.15.188:3300/0 conn(0x7f3e6c15f4e0 0x7f3e6c0cd7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:23:51.437 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:51.435+0000 7f3e63fff640 1 -- 172.21.15.143:0/1768409027 >> v2:172.21.15.188:3300/0 conn(0x7f3e6c15f4e0 msgr2=0x7f3e6c0cd7b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-26T07:23:51.438 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:51.435+0000 7f3e63fff640 1 --2- 172.21.15.143:0/1768409027 >> v2:172.21.15.188:3300/0 conn(0x7f3e6c15f4e0 0x7f3e6c0cd7b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:23:51.438 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:51.435+0000 7f3e63fff640 1 -- 172.21.15.143:0/1768409027 >> v1:172.21.15.188:6789/0 conn(0x7f3e6c13e290 legacy=0x7f3e6c0cf280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-26T07:23:51.438 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:51.435+0000 7f3e63fff640 1 -- 172.21.15.143:0/1768409027 --> v2:172.21.15.143:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3e6c06cdc0 con 0x7f3e6c0cf990 2023-11-26T07:23:51.438 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:51.435+0000 7f3e6ad9d640 1 --2- 172.21.15.143:0/1768409027 >> v2:172.21.15.188:3300/0 conn(0x7f3e6c15f4e0 0x7f3e6c0cd7b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-26T07:23:51.439 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:51.439+0000 7f3e63fff640 1 --2- 172.21.15.143:0/1768409027 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f3e6c0cf990 0x7f3e6c15efa0 secure :-1 s=READY pgs=1222 cs=0 l=1 rev1=1 crypto rx=0x7f3e6c155cf0 tx=0x7f3e5401c490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4eb7c71ecc05c34e server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:23:51.439 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:51.439+0000 7f3e69d9b640 1 -- 172.21.15.143:0/1768409027 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3e540aa610 con 0x7f3e6c0cf990 2023-11-26T07:23:51.439 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:51.439+0000 7f3e69d9b640 1 -- 172.21.15.143:0/1768409027 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3e540a6040 con 0x7f3e6c0cf990 2023-11-26T07:23:51.440 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:51.439+0000 7f3e69d9b640 1 -- 172.21.15.143:0/1768409027 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3e540aa990 con 0x7f3e6c0cf990 2023-11-26T07:23:51.440 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:51.439+0000 7f3e708ad640 1 -- 172.21.15.143:0/1768409027 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f3e6c0cf990 msgr2=0x7f3e6c15efa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:23:51.441 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:51.439+0000 7f3e708ad640 1 --2- 172.21.15.143:0/1768409027 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f3e6c0cf990 0x7f3e6c15efa0 secure :-1 s=READY pgs=1222 cs=0 l=1 rev1=1 crypto rx=0x7f3e6c155cf0 tx=0x7f3e5401c490 comp rx=0 tx=0).stop 2023-11-26T07:23:51.441 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:51.439+0000 7f3e708ad640 1 -- 172.21.15.143:0/1768409027 shutdown_connections 2023-11-26T07:23:51.441 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:51.439+0000 7f3e708ad640 1 --2- 172.21.15.143:0/1768409027 >> v2:172.21.15.188:3300/0 conn(0x7f3e6c15f4e0 0x7f3e6c0cd7b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:23:51.441 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:51.439+0000 7f3e708ad640 1 --2- 172.21.15.143:0/1768409027 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f3e6c0cf990 0x7f3e6c15efa0 unknown :-1 s=CLOSED pgs=1222 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:23:51.441 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:51.439+0000 7f3e708ad640 1 -- 172.21.15.143:0/1768409027 >> 172.21.15.143:0/1768409027 conn(0x7f3e6c06a890 msgr2=0x7f3e6c06aca0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:23:51.441 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:51.439+0000 7f3e708ad640 1 -- 172.21.15.143:0/1768409027 shutdown_connections 2023-11-26T07:23:51.441 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:51.439+0000 7f3e708ad640 1 -- 172.21.15.143:0/1768409027 wait complete. 2023-11-26T07:23:51.442 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:51.439+0000 7f3e708ad640 1 Processor -- start 2023-11-26T07:23:51.442 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:51.439+0000 7f3e708ad640 1 -- start start 2023-11-26T07:23:51.443 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:51.443+0000 7f3e708ad640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f3e6c0cf990 0x7f3e6c0ad9b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:23:51.444 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:51.443+0000 7f3e708ad640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f3e6c13e290 0x7f3e6c15ce80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:23:51.444 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:51.443+0000 7f3e708ad640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f3e6c15f4e0 0x7f3e6c070e70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:23:51.444 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:51.443+0000 7f3e708ad640 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3e6c0adef0 con 0x7f3e6c13e290 2023-11-26T07:23:51.444 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:51.443+0000 7f3e708ad640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3e6c15d3c0 con 0x7f3e6c15f4e0 2023-11-26T07:23:51.444 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:51.443+0000 7f3e708ad640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3e6c15be00 con 0x7f3e6c0cf990 2023-11-26T07:23:51.444 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:51.443+0000 7f3e63fff640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f3e6c13e290 0x7f3e6c15ce80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:23:51.444 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:51.443+0000 7f3e63fff640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f3e6c13e290 0x7f3e6c15ce80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:36966/0 (socket says 172.21.15.143:36966) 2023-11-26T07:23:51.444 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:51.443+0000 7f3e63fff640 1 -- 172.21.15.143:0/231347503 learned_addr learned my addr 172.21.15.143:0/231347503 (peer_addr_for_me v2:172.21.15.143:0/0) 2023-11-26T07:23:51.444 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:51.443+0000 7f3e6a59c640 1 --2- 172.21.15.143:0/231347503 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f3e6c0cf990 0x7f3e6c0ad9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:23:51.444 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:51.443+0000 7f3e6ad9d640 1 --2- 172.21.15.143:0/231347503 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f3e6c15f4e0 0x7f3e6c070e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:23:51.444 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:51.443+0000 7f3e63fff640 1 -- 172.21.15.143:0/231347503 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f3e6c0cf990 msgr2=0x7f3e6c0ad9b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:23:51.444 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:51.443+0000 7f3e63fff640 1 --2- 172.21.15.143:0/231347503 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f3e6c0cf990 0x7f3e6c0ad9b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:23:51.444 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:51.443+0000 7f3e63fff640 1 -- 172.21.15.143:0/231347503 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f3e6c15f4e0 msgr2=0x7f3e6c070e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:23:51.445 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:51.443+0000 7f3e63fff640 1 --2- 172.21.15.143:0/231347503 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f3e6c15f4e0 0x7f3e6c070e70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:23:51.445 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:51.443+0000 7f3e63fff640 1 -- 172.21.15.143:0/231347503 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3e540094d0 con 0x7f3e6c13e290 2023-11-26T07:23:51.445 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:51.443+0000 7f3e63fff640 1 --2- 172.21.15.143:0/231347503 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f3e6c13e290 0x7f3e6c15ce80 secure :-1 s=READY pgs=1223 cs=0 l=1 rev1=1 crypto rx=0x7f3e540ac040 tx=0x7f3e5401acd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:23:51.446 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:51.443+0000 7f3e62ffd640 1 -- 172.21.15.143:0/231347503 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3e540b3600 con 0x7f3e6c13e290 2023-11-26T07:23:51.446 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:51.443+0000 7f3e62ffd640 1 -- 172.21.15.143:0/231347503 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3e540b3780 con 0x7f3e6c13e290 2023-11-26T07:23:51.446 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:51.443+0000 7f3e708ad640 1 -- 172.21.15.143:0/231347503 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3e6c15c0c0 con 0x7f3e6c13e290 2023-11-26T07:23:51.446 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:51.443+0000 7f3e62ffd640 1 -- 172.21.15.143:0/231347503 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3e540b3b30 con 0x7f3e6c13e290 2023-11-26T07:23:51.446 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:51.443+0000 7f3e708ad640 1 -- 172.21.15.143:0/231347503 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3e6c15c680 con 0x7f3e6c13e290 2023-11-26T07:23:51.446 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:51.443+0000 7f3e708ad640 1 -- 172.21.15.143:0/231347503 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3e6c067680 con 0x7f3e6c13e290 2023-11-26T07:23:51.446 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:51.447+0000 7f3e62ffd640 1 -- 172.21.15.143:0/231347503 <== mon.0 v2:172.21.15.143:3300/0 4 ==== mgrmap(e 16) v1 ==== 85954+0+0 (secure 0 0 0) 0x7f3e540b3c90 con 0x7f3e6c13e290 2023-11-26T07:23:51.448 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:51.447+0000 7f3e62ffd640 1 --2- 172.21.15.143:0/231347503 >> [v2:172.21.15.188:6836/3401128553,v1:172.21.15.188:6837/3401128553] conn(0x7f3e4c066590 0x7f3e4c068a50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:23:51.448 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:51.447+0000 7f3e62ffd640 1 -- 172.21.15.143:0/231347503 <== mon.0 v2:172.21.15.143:3300/0 5 ==== osd_map(91..91 src has 1..91) v4 ==== 5795+0+0 (secure 0 0 0) 0x7f3e6c15fb00 con 0x7f3e6c13e290 2023-11-26T07:23:51.448 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:51.447+0000 7f3e6a59c640 1 --2- 172.21.15.143:0/231347503 >> [v2:172.21.15.188:6836/3401128553,v1:172.21.15.188:6837/3401128553] conn(0x7f3e4c066590 0x7f3e4c068a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:23:51.448 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:51.447+0000 7f3e6a59c640 1 --2- 172.21.15.143:0/231347503 >> [v2:172.21.15.188:6836/3401128553,v1:172.21.15.188:6837/3401128553] conn(0x7f3e4c066590 0x7f3e4c068a50 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7f3e5c007fd0 tx=0x7f3e5c007e40 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:23:51.452 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:51.451+0000 7f3e62ffd640 1 -- 172.21.15.143:0/231347503 <== mon.0 v2:172.21.15.143:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172649 (secure 0 0 0) 0x7f3e540eb9f0 con 0x7f3e6c13e290 2023-11-26T07:23:51.613 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:51.611+0000 7f3e708ad640 1 -- 172.21.15.143:0/231347503 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) v1 -- 0x7f3e6c15e980 con 0x7f3e6c13e290 2023-11-26T07:23:51.620 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:51.611+0000 7f3e62ffd640 1 -- 172.21.15.143:0/231347503 <== mon.0 v2:172.21.15.143:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 2 entries v91) v1 ==== 80+0+124 (secure 0 0 0) 0x7f3e540ef010 con 0x7f3e6c13e290 2023-11-26T07:23:51.621 INFO:teuthology.orchestra.run.smithi143.stdout:172.21.15.143:6835/4031724325 2023-11-27T07:23:44.794394+0000 2023-11-26T07:23:51.621 INFO:teuthology.orchestra.run.smithi143.stdout:172.21.15.143:6833/4031724325 2023-11-27T07:23:44.794394+0000 2023-11-26T07:23:51.622 INFO:teuthology.orchestra.run.smithi143.stderr:listed 2 entries 2023-11-26T07:23:51.622 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:51.615+0000 7f3e708ad640 1 -- 172.21.15.143:0/231347503 >> [v2:172.21.15.188:6836/3401128553,v1:172.21.15.188:6837/3401128553] conn(0x7f3e4c066590 msgr2=0x7f3e4c068a50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:23:51.622 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:51.615+0000 7f3e708ad640 1 --2- 172.21.15.143:0/231347503 >> [v2:172.21.15.188:6836/3401128553,v1:172.21.15.188:6837/3401128553] conn(0x7f3e4c066590 0x7f3e4c068a50 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7f3e5c007fd0 tx=0x7f3e5c007e40 comp rx=0 tx=0).stop 2023-11-26T07:23:51.622 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:51.615+0000 7f3e708ad640 1 -- 172.21.15.143:0/231347503 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f3e6c13e290 msgr2=0x7f3e6c15ce80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:23:51.622 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:51.615+0000 7f3e708ad640 1 --2- 172.21.15.143:0/231347503 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f3e6c13e290 0x7f3e6c15ce80 secure :-1 s=READY pgs=1223 cs=0 l=1 rev1=1 crypto rx=0x7f3e540ac040 tx=0x7f3e5401acd0 comp rx=0 tx=0).stop 2023-11-26T07:23:51.622 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:51.615+0000 7f3e708ad640 1 -- 172.21.15.143:0/231347503 shutdown_connections 2023-11-26T07:23:51.622 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:51.615+0000 7f3e708ad640 1 --2- 172.21.15.143:0/231347503 >> [v2:172.21.15.188:6836/3401128553,v1:172.21.15.188:6837/3401128553] conn(0x7f3e4c066590 0x7f3e4c068a50 unknown :-1 s=CLOSED pgs=88 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:23:51.622 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:51.615+0000 7f3e708ad640 1 --2- 172.21.15.143:0/231347503 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f3e6c15f4e0 0x7f3e6c070e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:23:51.622 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:51.615+0000 7f3e708ad640 1 --2- 172.21.15.143:0/231347503 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f3e6c13e290 0x7f3e6c15ce80 unknown :-1 s=CLOSED pgs=1223 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:23:51.622 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:51.615+0000 7f3e708ad640 1 --2- 172.21.15.143:0/231347503 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f3e6c0cf990 0x7f3e6c0ad9b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:23:51.623 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:51.615+0000 7f3e708ad640 1 -- 172.21.15.143:0/231347503 >> 172.21.15.143:0/231347503 conn(0x7f3e6c06a890 msgr2=0x7f3e6c0b0b70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:23:51.623 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:51.615+0000 7f3e708ad640 1 -- 172.21.15.143:0/231347503 shutdown_connections 2023-11-26T07:23:51.623 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:51.615+0000 7f3e708ad640 1 -- 172.21.15.143:0/231347503 wait complete. 2023-11-26T07:23:51.635 INFO:tasks.cephfs.fuse_mount:Running fusermount -u on ubuntu@smithi188.front.sepia.ceph.com... 2023-11-26T07:23:51.635 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-26T07:23:51.635 DEBUG:teuthology.orchestra.run.smithi188:> sudo fusermount -u /home/ubuntu/cephtest/mnt.1 2023-11-26T07:23:51.656 INFO:tasks.cephfs.fuse_mount.ceph-fuse.1.smithi188.stderr:ceph-fuse[45157]: fuse finished with error 0 and tester_r 0 2023-11-26T07:23:51.659 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi188.front.sepia.ceph.com 2023-11-26T07:23:51.659 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-26T07:23:51.659 DEBUG:teuthology.orchestra.run.smithi188:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.1) 2023-11-26T07:23:51.713 DEBUG:teuthology.orchestra.run.smithi143:> set -ex 2023-11-26T07:23:51.713 DEBUG:teuthology.orchestra.run.smithi143:> dd if=/proc/self/mounts of=/dev/stdout 2023-11-26T07:23:51.722 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,relatime,size=16354672k,nr_inodes=4088668,mode=755 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3275384k,mode=755 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=3455 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysusers.service ramfs ro,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3275380k,nr_inodes=818845,mode=700,uid=1000,gid=1271 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3275384k,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2023-11-26T07:23:51.722 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi143.front.sepia.ceph.com 2023-11-26T07:23:51.722 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-26T07:23:51.722 DEBUG:teuthology.orchestra.run.smithi143:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.2) 2023-11-26T07:23:51.773 INFO:teuthology.orchestra.run.smithi143.stderr:rmdir: failed to remove '/home/ubuntu/cephtest/mnt.2': No such file or directory 2023-11-26T07:23:51.774 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-11-26T07:23:51.774 DEBUG:teuthology.orchestra.run.smithi188:> set -ex 2023-11-26T07:23:51.774 DEBUG:teuthology.orchestra.run.smithi188:> dd if=/proc/self/mounts of=/dev/stdout 2023-11-26T07:23:51.783 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,relatime,size=16354628k,nr_inodes=4088657,mode=755 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3275376k,mode=755 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=787 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysusers.service ramfs ro,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3275372k,nr_inodes=818843,mode=700,uid=1000,gid=1271 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3275376k,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2023-11-26T07:23:51.783 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi188.front.sepia.ceph.com 2023-11-26T07:23:51.784 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-26T07:23:51.784 DEBUG:teuthology.orchestra.run.smithi188:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.3) 2023-11-26T07:23:51.840 INFO:teuthology.orchestra.run.smithi188.stderr:rmdir: failed to remove '/home/ubuntu/cephtest/mnt.3': No such file or directory 2023-11-26T07:23:51.840 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-11-26T07:23:51.841 DEBUG:teuthology.orchestra.run.smithi143:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-11-26T07:23:51.979 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:51.979+0000 7f1953d1c640 1 Processor -- start 2023-11-26T07:23:51.979 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:51.979+0000 7f1953d1c640 1 -- start start 2023-11-26T07:23:51.981 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:51.979+0000 7f1953d1c640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f194c06f3e0 0x7f194c161690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:23:51.981 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:51.979+0000 7f1953d1c640 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7f194c099a70 0x7f194c161bd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:23:51.981 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:51.979+0000 7f1953d1c640 1 -- --> v1:172.21.15.188:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f194c0cf260 con 0x7f194c06f000 2023-11-26T07:23:51.981 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:51.979+0000 7f1953d1c640 1 -- --> v2:172.21.15.143:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f194c09a010 con 0x7f194c099a70 2023-11-26T07:23:51.981 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:51.979+0000 7f1953d1c640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f194c09a2f0 con 0x7f194c06f3e0 2023-11-26T07:23:51.981 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:51.979+0000 7f19522b9640 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7f194c099a70 0x7f194c161bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:23:51.981 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:51.979+0000 7f19522b9640 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7f194c099a70 0x7f194c161bd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:36970/0 (socket says 172.21.15.143:36970) 2023-11-26T07:23:51.981 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:51.979+0000 7f19522b9640 1 -- 172.21.15.143:0/4184378584 learned_addr learned my addr 172.21.15.143:0/4184378584 (peer_addr_for_me v2:172.21.15.143:0/0) 2023-11-26T07:23:51.981 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:51.979+0000 7f19512b7640 1 --2- 172.21.15.143:0/4184378584 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f194c06f3e0 0x7f194c161690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:23:51.982 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:51.979+0000 7f19522b9640 1 -- 172.21.15.143:0/4184378584 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f194c06f3e0 msgr2=0x7f194c161690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-26T07:23:51.982 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:51.979+0000 7f19522b9640 1 --2- 172.21.15.143:0/4184378584 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f194c06f3e0 0x7f194c161690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:23:51.982 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:51.979+0000 7f19522b9640 1 -- 172.21.15.143:0/4184378584 >> v1:172.21.15.188:6789/0 conn(0x7f194c06f000 legacy=0x7f194c06d030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:23:51.982 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:51.979+0000 7f19522b9640 1 -- 172.21.15.143:0/4184378584 --> v2:172.21.15.143:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f194c09a5d0 con 0x7f194c099a70 2023-11-26T07:23:51.984 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:51.983+0000 7f19522b9640 1 --2- 172.21.15.143:0/4184378584 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f194c099a70 0x7f194c161bd0 secure :-1 s=READY pgs=1224 cs=0 l=1 rev1=1 crypto rx=0x7f1944009e50 tx=0x7f194401c980 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8393e2523591b436 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:23:51.984 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:51.983+0000 7f1950ab6640 1 -- 172.21.15.143:0/4184378584 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f19440a5640 con 0x7f194c099a70 2023-11-26T07:23:51.984 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:51.983+0000 7f1950ab6640 1 -- 172.21.15.143:0/4184378584 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f19440a57a0 con 0x7f194c099a70 2023-11-26T07:23:51.985 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:51.983+0000 7f1950ab6640 1 -- 172.21.15.143:0/4184378584 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f19440a5b30 con 0x7f194c099a70 2023-11-26T07:23:51.985 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:51.983+0000 7f1953d1c640 1 -- 172.21.15.143:0/4184378584 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f194c099a70 msgr2=0x7f194c161bd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:23:51.985 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:51.983+0000 7f1953d1c640 1 --2- 172.21.15.143:0/4184378584 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f194c099a70 0x7f194c161bd0 secure :-1 s=READY pgs=1224 cs=0 l=1 rev1=1 crypto rx=0x7f1944009e50 tx=0x7f194401c980 comp rx=0 tx=0).stop 2023-11-26T07:23:51.985 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:51.983+0000 7f1953d1c640 1 -- 172.21.15.143:0/4184378584 shutdown_connections 2023-11-26T07:23:51.986 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:51.983+0000 7f1953d1c640 1 --2- 172.21.15.143:0/4184378584 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f194c099a70 0x7f194c161bd0 unknown :-1 s=CLOSED pgs=1224 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:23:51.986 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:51.983+0000 7f1953d1c640 1 --2- 172.21.15.143:0/4184378584 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f194c06f3e0 0x7f194c161690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:23:51.986 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:51.983+0000 7f1953d1c640 1 -- 172.21.15.143:0/4184378584 >> 172.21.15.143:0/4184378584 conn(0x7f194c0b2d10 msgr2=0x7f194c0b3120 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:23:51.986 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:51.983+0000 7f1953d1c640 1 -- 172.21.15.143:0/4184378584 shutdown_connections 2023-11-26T07:23:51.986 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:51.983+0000 7f1953d1c640 1 -- 172.21.15.143:0/4184378584 wait complete. 2023-11-26T07:23:51.987 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:51.987+0000 7f1953d1c640 1 Processor -- start 2023-11-26T07:23:51.987 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:51.987+0000 7f1953d1c640 1 -- start start 2023-11-26T07:23:51.989 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:51.987+0000 7f1953d1c640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f194c06f000 0x7f194c156650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:23:51.989 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:51.987+0000 7f1953d1c640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f194c06f3e0 0x7f194c0b6c20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:23:51.989 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:51.987+0000 7f1953d1c640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f194c099a70 0x7f194c1f6000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:23:51.989 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:51.987+0000 7f1953d1c640 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f194c156b90 con 0x7f194c099a70 2023-11-26T07:23:51.989 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:51.987+0000 7f1953d1c640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f194c0b7160 con 0x7f194c06f000 2023-11-26T07:23:51.989 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:51.987+0000 7f1953d1c640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f194c14efd0 con 0x7f194c06f3e0 2023-11-26T07:23:51.989 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:51.987+0000 7f19522b9640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f194c099a70 0x7f194c1f6000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:23:51.989 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:51.987+0000 7f19522b9640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f194c099a70 0x7f194c1f6000 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:36976/0 (socket says 172.21.15.143:36976) 2023-11-26T07:23:51.989 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:51.987+0000 7f19522b9640 1 -- 172.21.15.143:0/3994490649 learned_addr learned my addr 172.21.15.143:0/3994490649 (peer_addr_for_me v2:172.21.15.143:0/0) 2023-11-26T07:23:51.989 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:51.987+0000 7f1951ab8640 1 --2- 172.21.15.143:0/3994490649 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f194c06f000 0x7f194c156650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:23:51.989 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:51.987+0000 7f19512b7640 1 --2- 172.21.15.143:0/3994490649 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f194c06f3e0 0x7f194c0b6c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:23:51.989 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:51.987+0000 7f19522b9640 1 -- 172.21.15.143:0/3994490649 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f194c06f3e0 msgr2=0x7f194c0b6c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:23:51.990 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:51.987+0000 7f19522b9640 1 --2- 172.21.15.143:0/3994490649 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f194c06f3e0 0x7f194c0b6c20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:23:51.990 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:51.987+0000 7f19522b9640 1 -- 172.21.15.143:0/3994490649 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f194c06f000 msgr2=0x7f194c156650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:23:51.990 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:51.987+0000 7f19522b9640 1 --2- 172.21.15.143:0/3994490649 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f194c06f000 0x7f194c156650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:23:51.990 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:51.987+0000 7f19522b9640 1 -- 172.21.15.143:0/3994490649 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1944009b30 con 0x7f194c099a70 2023-11-26T07:23:51.990 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:51.987+0000 7f19522b9640 1 --2- 172.21.15.143:0/3994490649 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f194c099a70 0x7f194c1f6000 secure :-1 s=READY pgs=1225 cs=0 l=1 rev1=1 crypto rx=0x7f1944009e50 tx=0x7f1944019720 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:23:51.991 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:51.987+0000 7f19427fc640 1 -- 172.21.15.143:0/3994490649 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f19440a5640 con 0x7f194c099a70 2023-11-26T07:23:51.991 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:51.987+0000 7f19427fc640 1 -- 172.21.15.143:0/3994490649 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f19440b88a0 con 0x7f194c099a70 2023-11-26T07:23:51.991 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:51.987+0000 7f19427fc640 1 -- 172.21.15.143:0/3994490649 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f19440b8c50 con 0x7f194c099a70 2023-11-26T07:23:51.991 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:51.987+0000 7f1953d1c640 1 -- 172.21.15.143:0/3994490649 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f194c152a00 con 0x7f194c099a70 2023-11-26T07:23:51.991 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:51.987+0000 7f1953d1c640 1 -- 172.21.15.143:0/3994490649 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f194c157060 con 0x7f194c099a70 2023-11-26T07:23:51.991 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:51.991+0000 7f1953d1c640 1 -- 172.21.15.143:0/3994490649 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f194c152b90 con 0x7f194c099a70 2023-11-26T07:23:51.991 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:51.991+0000 7f19427fc640 1 -- 172.21.15.143:0/3994490649 <== mon.0 v2:172.21.15.143:3300/0 4 ==== mgrmap(e 16) v1 ==== 85954+0+0 (secure 0 0 0) 0x7f19440b6440 con 0x7f194c099a70 2023-11-26T07:23:51.992 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:51.991+0000 7f19427fc640 1 --2- 172.21.15.143:0/3994490649 >> [v2:172.21.15.188:6836/3401128553,v1:172.21.15.188:6837/3401128553] conn(0x7f1930066590 0x7f1930068a50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:23:51.992 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:51.991+0000 7f19427fc640 1 -- 172.21.15.143:0/3994490649 <== mon.0 v2:172.21.15.143:3300/0 5 ==== osd_map(91..91 src has 1..91) v4 ==== 5795+0+0 (secure 0 0 0) 0x7f194411f4f0 con 0x7f194c099a70 2023-11-26T07:23:51.992 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:51.991+0000 7f1951ab8640 1 --2- 172.21.15.143:0/3994490649 >> [v2:172.21.15.188:6836/3401128553,v1:172.21.15.188:6837/3401128553] conn(0x7f1930066590 0x7f1930068a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:23:51.993 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:51.991+0000 7f1951ab8640 1 --2- 172.21.15.143:0/3994490649 >> [v2:172.21.15.188:6836/3401128553,v1:172.21.15.188:6837/3401128553] conn(0x7f1930066590 0x7f1930068a50 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7f1948007fd0 tx=0x7f1948009040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:23:51.996 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:51.995+0000 7f19427fc640 1 -- 172.21.15.143:0/3994490649 <== mon.0 v2:172.21.15.143:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172649 (secure 0 0 0) 0x7f19440ec820 con 0x7f194c099a70 2023-11-26T07:23:52.174 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:52.175+0000 7f1953d1c640 1 -- 172.21.15.143:0/3994490649 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f194c067680 con 0x7f194c099a70 2023-11-26T07:23:52.175 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:52.175+0000 7f19427fc640 1 -- 172.21.15.143:0/3994490649 <== mon.0 v2:172.21.15.143:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 57 v57) v1 ==== 94+0+4924 (secure 0 0 0) 0x7f1944009510 con 0x7f194c099a70 2023-11-26T07:23:52.175 INFO:teuthology.orchestra.run.smithi143.stdout: 2023-11-26T07:23:52.175 INFO:teuthology.orchestra.run.smithi143.stdout:{"epoch":57,"default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6305,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.188:6834/3670325785","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6832","nonce":3670325785},{"type":"v1","addr":"172.21.15.188:6834","nonce":3670325785}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":46},{"gid":7467,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.143:6834/4060405000","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6832","nonce":4060405000},{"type":"v1","addr":"172.21.15.143:6834","nonce":4060405000}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":46},{"gid":7704,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.143:6835/2229435747","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6833","nonce":2229435747},{"type":"v1","addr":"172.21.15.143:6835","nonce":2229435747}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":57}],"filesystems":[{"mdsmap":{"epoch":51,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-11-26T07:23:29.008560+0000","modified":"2023-11-26T07:23:32.033055+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":7476},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7476":{"gid":7476,"name":"b","rank":0,"incarnation":47,"state":"up:active","state_seq":3,"addr":"172.21.15.188:6835/278875865","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6833","nonce":278875865},{"type":"v1","addr":"172.21.15.188:6835","nonce":278875865}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[13],"metadata_pool":12,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":6}]} 2023-11-26T07:23:52.176 INFO:teuthology.orchestra.run.smithi143.stderr:dumped fsmap epoch 57 2023-11-26T07:23:52.177 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:52.175+0000 7f1953d1c640 1 -- 172.21.15.143:0/3994490649 >> [v2:172.21.15.188:6836/3401128553,v1:172.21.15.188:6837/3401128553] conn(0x7f1930066590 msgr2=0x7f1930068a50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:23:52.177 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:52.175+0000 7f1953d1c640 1 --2- 172.21.15.143:0/3994490649 >> [v2:172.21.15.188:6836/3401128553,v1:172.21.15.188:6837/3401128553] conn(0x7f1930066590 0x7f1930068a50 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7f1948007fd0 tx=0x7f1948009040 comp rx=0 tx=0).stop 2023-11-26T07:23:52.177 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:52.175+0000 7f1953d1c640 1 -- 172.21.15.143:0/3994490649 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f194c099a70 msgr2=0x7f194c1f6000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:23:52.178 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:52.175+0000 7f1953d1c640 1 --2- 172.21.15.143:0/3994490649 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f194c099a70 0x7f194c1f6000 secure :-1 s=READY pgs=1225 cs=0 l=1 rev1=1 crypto rx=0x7f1944009e50 tx=0x7f1944019720 comp rx=0 tx=0).stop 2023-11-26T07:23:52.178 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:52.175+0000 7f1953d1c640 1 -- 172.21.15.143:0/3994490649 shutdown_connections 2023-11-26T07:23:52.178 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:52.175+0000 7f1953d1c640 1 --2- 172.21.15.143:0/3994490649 >> [v2:172.21.15.188:6836/3401128553,v1:172.21.15.188:6837/3401128553] conn(0x7f1930066590 0x7f1930068a50 unknown :-1 s=CLOSED pgs=89 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:23:52.178 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:52.175+0000 7f1953d1c640 1 --2- 172.21.15.143:0/3994490649 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f194c099a70 0x7f194c1f6000 unknown :-1 s=CLOSED pgs=1225 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:23:52.178 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:52.175+0000 7f1953d1c640 1 --2- 172.21.15.143:0/3994490649 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f194c06f3e0 0x7f194c0b6c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:23:52.178 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:52.175+0000 7f1953d1c640 1 --2- 172.21.15.143:0/3994490649 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f194c06f000 0x7f194c156650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:23:52.178 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:52.175+0000 7f1953d1c640 1 -- 172.21.15.143:0/3994490649 >> 172.21.15.143:0/3994490649 conn(0x7f194c0b2d10 msgr2=0x7f194c0cc2f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:23:52.178 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:52.175+0000 7f1953d1c640 1 -- 172.21.15.143:0/3994490649 shutdown_connections 2023-11-26T07:23:52.178 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:52.175+0000 7f1953d1c640 1 -- 172.21.15.143:0/3994490649 wait complete. 2023-11-26T07:23:52.196 DEBUG:teuthology.orchestra.run.smithi143:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-11-26T07:23:52.335 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:52.335+0000 7fd421abb640 1 Processor -- start 2023-11-26T07:23:52.335 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:52.335+0000 7fd421abb640 1 -- start start 2023-11-26T07:23:52.337 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:52.335+0000 7fd421abb640 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7fd41c15d6e0 0x7fd41c0d0df0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:23:52.337 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:52.335+0000 7fd421abb640 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7fd41c0d1bc0 0x7fd41c0b0750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:23:52.337 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:52.335+0000 7fd421abb640 1 -- --> v1:172.21.15.188:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd41c0cf0e0 con 0x7fd41c0b2160 2023-11-26T07:23:52.337 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:52.335+0000 7fd421abb640 1 -- --> v2:172.21.15.143:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd41c15dac0 con 0x7fd41c15d6e0 2023-11-26T07:23:52.337 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:52.335+0000 7fd421abb640 1 -- --> v2:172.21.15.188:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd41c0ceb20 con 0x7fd41c0d1bc0 2023-11-26T07:23:52.337 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:52.335+0000 7fd41b7fe640 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7fd41c15d6e0 0x7fd41c0d0df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:23:52.337 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:52.335+0000 7fd41b7fe640 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7fd41c15d6e0 0x7fd41c0d0df0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:36978/0 (socket says 172.21.15.143:36978) 2023-11-26T07:23:52.337 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:52.335+0000 7fd41b7fe640 1 -- 172.21.15.143:0/264853942 learned_addr learned my addr 172.21.15.143:0/264853942 (peer_addr_for_me v2:172.21.15.143:0/0) 2023-11-26T07:23:52.337 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:52.335+0000 7fd41bfff640 1 --2- 172.21.15.143:0/264853942 >> v2:172.21.15.188:3300/0 conn(0x7fd41c0d1bc0 0x7fd41c0b0750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:23:52.338 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:52.335+0000 7fd41b7fe640 1 -- 172.21.15.143:0/264853942 >> v2:172.21.15.188:3300/0 conn(0x7fd41c0d1bc0 msgr2=0x7fd41c0b0750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-26T07:23:52.338 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:52.335+0000 7fd41b7fe640 1 --2- 172.21.15.143:0/264853942 >> v2:172.21.15.188:3300/0 conn(0x7fd41c0d1bc0 0x7fd41c0b0750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:23:52.338 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:52.335+0000 7fd41b7fe640 1 -- 172.21.15.143:0/264853942 >> v1:172.21.15.188:6789/0 conn(0x7fd41c0b2160 legacy=0x7fd41c0d14b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-26T07:23:52.338 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:52.335+0000 7fd41b7fe640 1 -- 172.21.15.143:0/264853942 --> v2:172.21.15.143:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd41c0cee00 con 0x7fd41c15d6e0 2023-11-26T07:23:52.340 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:52.339+0000 7fd41b7fe640 1 --2- 172.21.15.143:0/264853942 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fd41c15d6e0 0x7fd41c0d0df0 secure :-1 s=READY pgs=1226 cs=0 l=1 rev1=1 crypto rx=0x7fd41c0b92d0 tx=0x7fd40c01c4c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7ca5548e221124cb server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:23:52.340 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:52.339+0000 7fd41affd640 1 -- 172.21.15.143:0/264853942 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd40c0ac410 con 0x7fd41c15d6e0 2023-11-26T07:23:52.340 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:52.339+0000 7fd41affd640 1 -- 172.21.15.143:0/264853942 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd40c0ac570 con 0x7fd41c15d6e0 2023-11-26T07:23:52.340 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:52.339+0000 7fd41affd640 1 -- 172.21.15.143:0/264853942 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd40c0ac900 con 0x7fd41c15d6e0 2023-11-26T07:23:52.341 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:52.339+0000 7fd421abb640 1 -- 172.21.15.143:0/264853942 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fd41c15d6e0 msgr2=0x7fd41c0d0df0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:23:52.341 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:52.339+0000 7fd421abb640 1 --2- 172.21.15.143:0/264853942 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fd41c15d6e0 0x7fd41c0d0df0 secure :-1 s=READY pgs=1226 cs=0 l=1 rev1=1 crypto rx=0x7fd41c0b92d0 tx=0x7fd40c01c4c0 comp rx=0 tx=0).stop 2023-11-26T07:23:52.341 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:52.339+0000 7fd421abb640 1 -- 172.21.15.143:0/264853942 shutdown_connections 2023-11-26T07:23:52.341 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:52.339+0000 7fd421abb640 1 --2- 172.21.15.143:0/264853942 >> v2:172.21.15.188:3300/0 conn(0x7fd41c0d1bc0 0x7fd41c0b0750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:23:52.341 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:52.339+0000 7fd421abb640 1 --2- 172.21.15.143:0/264853942 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fd41c15d6e0 0x7fd41c0d0df0 unknown :-1 s=CLOSED pgs=1226 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:23:52.341 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:52.339+0000 7fd421abb640 1 -- 172.21.15.143:0/264853942 >> 172.21.15.143:0/264853942 conn(0x7fd41c09c550 msgr2=0x7fd41c0d3220 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:23:52.342 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:52.339+0000 7fd421abb640 1 -- 172.21.15.143:0/264853942 shutdown_connections 2023-11-26T07:23:52.342 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:52.339+0000 7fd421abb640 1 -- 172.21.15.143:0/264853942 wait complete. 2023-11-26T07:23:52.343 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:52.343+0000 7fd421abb640 1 Processor -- start 2023-11-26T07:23:52.343 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:52.343+0000 7fd421abb640 1 -- start start 2023-11-26T07:23:52.344 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:52.343+0000 7fd421abb640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fd41c0b2160 0x7fd41c0b88e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:23:52.344 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:52.343+0000 7fd421abb640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fd41c0d1bc0 0x7fd41c1606b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:23:52.344 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:52.343+0000 7fd421abb640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fd41c15d6e0 0x7fd41c154750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:23:52.345 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:52.343+0000 7fd421abb640 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd41c155370 con 0x7fd41c0d1bc0 2023-11-26T07:23:52.345 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:52.343+0000 7fd421abb640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd41c0b78e0 con 0x7fd41c0b2160 2023-11-26T07:23:52.345 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:52.343+0000 7fd421abb640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd41c0b7b60 con 0x7fd41c15d6e0 2023-11-26T07:23:52.345 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:52.343+0000 7fd413fff640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fd41c0d1bc0 0x7fd41c1606b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:23:52.345 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:52.343+0000 7fd413fff640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fd41c0d1bc0 0x7fd41c1606b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:36980/0 (socket says 172.21.15.143:36980) 2023-11-26T07:23:52.345 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:52.343+0000 7fd413fff640 1 -- 172.21.15.143:0/814082592 learned_addr learned my addr 172.21.15.143:0/814082592 (peer_addr_for_me v2:172.21.15.143:0/0) 2023-11-26T07:23:52.345 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:52.343+0000 7fd41b7fe640 1 --2- 172.21.15.143:0/814082592 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fd41c0b2160 0x7fd41c0b88e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:23:52.345 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:52.343+0000 7fd41bfff640 1 --2- 172.21.15.143:0/814082592 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fd41c15d6e0 0x7fd41c154750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:23:52.345 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:52.343+0000 7fd413fff640 1 -- 172.21.15.143:0/814082592 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fd41c15d6e0 msgr2=0x7fd41c154750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:23:52.345 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:52.343+0000 7fd413fff640 1 --2- 172.21.15.143:0/814082592 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fd41c15d6e0 0x7fd41c154750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:23:52.345 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:52.343+0000 7fd413fff640 1 -- 172.21.15.143:0/814082592 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fd41c0b2160 msgr2=0x7fd41c0b88e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:23:52.345 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:52.343+0000 7fd413fff640 1 --2- 172.21.15.143:0/814082592 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fd41c0b2160 0x7fd41c0b88e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:23:52.345 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:52.343+0000 7fd413fff640 1 -- 172.21.15.143:0/814082592 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd40c0094d0 con 0x7fd41c0d1bc0 2023-11-26T07:23:52.346 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:52.343+0000 7fd413fff640 1 --2- 172.21.15.143:0/814082592 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fd41c0d1bc0 0x7fd41c1606b0 secure :-1 s=READY pgs=1227 cs=0 l=1 rev1=1 crypto rx=0x7fd40400c450 tx=0x7fd40400c910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:23:52.346 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:52.343+0000 7fd418ff9640 1 -- 172.21.15.143:0/814082592 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd404012830 con 0x7fd41c0d1bc0 2023-11-26T07:23:52.346 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:52.343+0000 7fd418ff9640 1 -- 172.21.15.143:0/814082592 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd4040129b0 con 0x7fd41c0d1bc0 2023-11-26T07:23:52.347 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:52.343+0000 7fd418ff9640 1 -- 172.21.15.143:0/814082592 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd404012d60 con 0x7fd41c0d1bc0 2023-11-26T07:23:52.347 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:52.343+0000 7fd421abb640 1 -- 172.21.15.143:0/814082592 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd41c0b7e40 con 0x7fd41c0d1bc0 2023-11-26T07:23:52.347 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:52.343+0000 7fd421abb640 1 -- 172.21.15.143:0/814082592 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd41c155000 con 0x7fd41c0d1bc0 2023-11-26T07:23:52.347 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:52.347+0000 7fd421abb640 1 -- 172.21.15.143:0/814082592 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd41c067680 con 0x7fd41c0d1bc0 2023-11-26T07:23:52.347 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:52.347+0000 7fd418ff9640 1 -- 172.21.15.143:0/814082592 <== mon.0 v2:172.21.15.143:3300/0 4 ==== mgrmap(e 16) v1 ==== 85954+0+0 (secure 0 0 0) 0x7fd40401c460 con 0x7fd41c0d1bc0 2023-11-26T07:23:52.348 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:52.347+0000 7fd418ff9640 1 --2- 172.21.15.143:0/814082592 >> [v2:172.21.15.188:6836/3401128553,v1:172.21.15.188:6837/3401128553] conn(0x7fd3fc066590 0x7fd3fc068a50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:23:52.348 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:52.347+0000 7fd418ff9640 1 -- 172.21.15.143:0/814082592 <== mon.0 v2:172.21.15.143:3300/0 5 ==== osd_map(91..91 src has 1..91) v4 ==== 5795+0+0 (secure 0 0 0) 0x7fd404083e40 con 0x7fd41c0d1bc0 2023-11-26T07:23:52.349 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:52.347+0000 7fd41b7fe640 1 --2- 172.21.15.143:0/814082592 >> [v2:172.21.15.188:6836/3401128553,v1:172.21.15.188:6837/3401128553] conn(0x7fd3fc066590 0x7fd3fc068a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:23:52.349 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:52.347+0000 7fd41b7fe640 1 --2- 172.21.15.143:0/814082592 >> [v2:172.21.15.188:6836/3401128553,v1:172.21.15.188:6837/3401128553] conn(0x7fd3fc066590 0x7fd3fc068a50 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7fd40c0ae000 tx=0x7fd40c008040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:23:52.352 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:52.351+0000 7fd418ff9640 1 -- 172.21.15.143:0/814082592 <== mon.0 v2:172.21.15.143:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172649 (secure 0 0 0) 0x7fd404051160 con 0x7fd41c0d1bc0 2023-11-26T07:23:52.502 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:52.503+0000 7fd421abb640 1 -- 172.21.15.143:0/814082592 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fd41c0cfe50 con 0x7fd41c0d1bc0 2023-11-26T07:23:52.504 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:52.503+0000 7fd418ff9640 1 -- 172.21.15.143:0/814082592 <== mon.0 v2:172.21.15.143:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v91) v1 ==== 74+0+16727 (secure 0 0 0) 0x7fd404054780 con 0x7fd41c0d1bc0 2023-11-26T07:23:52.504 INFO:teuthology.orchestra.run.smithi143.stdout: 2023-11-26T07:23:52.504 INFO:teuthology.orchestra.run.smithi143.stdout:{"epoch":91,"fsid":"f5033060-2469-4a07-95fe-2b76f5404bc1","created":"2023-11-26T07:17:28.613663+0000","modified":"2023-11-26T07:23:47.807341+0000","last_up_change":"2023-11-26T07:17:41.157813+0000","last_in_change":"2023-11-26T07:17:31.181658+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":15,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-11-26T07:17:41.758597+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":12,"pool_name":"cephfs_metadata","create_time":"2023-11-26T07:23:26.724016+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"83","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":13,"pool_name":"cephfs_data","create_time":"2023-11-26T07:23:28.019663+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"84","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"ebbee2c6-023e-44b9-9f1d-6f7bfd309730","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6817","nonce":639807369},{"type":"v1","addr":"172.21.15.143:6819","nonce":639807369}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6821","nonce":639807369},{"type":"v1","addr":"172.21.15.143:6823","nonce":639807369}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6830","nonce":639807369},{"type":"v1","addr":"172.21.15.143:6831","nonce":639807369}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6825","nonce":639807369},{"type":"v1","addr":"172.21.15.143:6828","nonce":639807369}]},"public_addr":"172.21.15.143:6819/639807369","cluster_addr":"172.21.15.143:6823/639807369","heartbeat_back_addr":"172.21.15.143:6831/639807369","heartbeat_front_addr":"172.21.15.143:6828/639807369","state":["exists","up"]},{"osd":1,"uuid":"fec4b8be-7122-49e9-9a19-5850a0700001","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6816","nonce":1144942530},{"type":"v1","addr":"172.21.15.143:6818","nonce":1144942530}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6820","nonce":1144942530},{"type":"v1","addr":"172.21.15.143:6822","nonce":1144942530}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6827","nonce":1144942530},{"type":"v1","addr":"172.21.15.143:6829","nonce":1144942530}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6824","nonce":1144942530},{"type":"v1","addr":"172.21.15.143:6826","nonce":1144942530}]},"public_addr":"172.21.15.143:6818/1144942530","cluster_addr":"172.21.15.143:6822/1144942530","heartbeat_back_addr":"172.21.15.143:6829/1144942530","heartbeat_front_addr":"172.21.15.143:6826/1144942530","state":["exists","up"]},{"osd":2,"uuid":"2888af5b-f933-46ca-8f95-6dd494ff43f6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6808","nonce":3909095186},{"type":"v1","addr":"172.21.15.143:6809","nonce":3909095186}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6810","nonce":3909095186},{"type":"v1","addr":"172.21.15.143:6811","nonce":3909095186}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6814","nonce":3909095186},{"type":"v1","addr":"172.21.15.143:6815","nonce":3909095186}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6812","nonce":3909095186},{"type":"v1","addr":"172.21.15.143:6813","nonce":3909095186}]},"public_addr":"172.21.15.143:6809/3909095186","cluster_addr":"172.21.15.143:6811/3909095186","heartbeat_back_addr":"172.21.15.143:6815/3909095186","heartbeat_front_addr":"172.21.15.143:6813/3909095186","state":["exists","up"]},{"osd":3,"uuid":"d1f52435-7bc1-4081-9550-9cd1ba55f549","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6800","nonce":3854210991},{"type":"v1","addr":"172.21.15.143:6801","nonce":3854210991}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6802","nonce":3854210991},{"type":"v1","addr":"172.21.15.143:6803","nonce":3854210991}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6806","nonce":3854210991},{"type":"v1","addr":"172.21.15.143:6807","nonce":3854210991}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6804","nonce":3854210991},{"type":"v1","addr":"172.21.15.143:6805","nonce":3854210991}]},"public_addr":"172.21.15.143:6801/3854210991","cluster_addr":"172.21.15.143:6803/3854210991","heartbeat_back_addr":"172.21.15.143:6807/3854210991","heartbeat_front_addr":"172.21.15.143:6805/3854210991","state":["exists","up"]},{"osd":4,"uuid":"e2ae8b6e-6e25-49b2-bcbf-aacce68f06a3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6800","nonce":3924606902},{"type":"v1","addr":"172.21.15.188:6801","nonce":3924606902}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6802","nonce":3924606902},{"type":"v1","addr":"172.21.15.188:6803","nonce":3924606902}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6806","nonce":3924606902},{"type":"v1","addr":"172.21.15.188:6807","nonce":3924606902}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6804","nonce":3924606902},{"type":"v1","addr":"172.21.15.188:6805","nonce":3924606902}]},"public_addr":"172.21.15.188:6801/3924606902","cluster_addr":"172.21.15.188:6803/3924606902","heartbeat_back_addr":"172.21.15.188:6807/3924606902","heartbeat_front_addr":"172.21.15.188:6805/3924606902","state":["exists","up"]},{"osd":5,"uuid":"3cec8a32-7698-4c36-8e89-e4ba11934f33","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6813","nonce":186841544},{"type":"v1","addr":"172.21.15.188:6815","nonce":186841544}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6817","nonce":186841544},{"type":"v1","addr":"172.21.15.188:6818","nonce":186841544}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6822","nonce":186841544},{"type":"v1","addr":"172.21.15.188:6823","nonce":186841544}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6820","nonce":186841544},{"type":"v1","addr":"172.21.15.188:6821","nonce":186841544}]},"public_addr":"172.21.15.188:6815/186841544","cluster_addr":"172.21.15.188:6818/186841544","heartbeat_back_addr":"172.21.15.188:6823/186841544","heartbeat_front_addr":"172.21.15.188:6821/186841544","state":["exists","up"]},{"osd":6,"uuid":"58a65d5d-e090-473a-81b4-ad38e85029f9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6808","nonce":2153499405},{"type":"v1","addr":"172.21.15.188:6809","nonce":2153499405}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6810","nonce":2153499405},{"type":"v1","addr":"172.21.15.188:6811","nonce":2153499405}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6816","nonce":2153499405},{"type":"v1","addr":"172.21.15.188:6819","nonce":2153499405}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6812","nonce":2153499405},{"type":"v1","addr":"172.21.15.188:6814","nonce":2153499405}]},"public_addr":"172.21.15.188:6809/2153499405","cluster_addr":"172.21.15.188:6811/2153499405","heartbeat_back_addr":"172.21.15.188:6819/2153499405","heartbeat_front_addr":"172.21.15.188:6814/2153499405","state":["exists","up"]},{"osd":7,"uuid":"ea5b7f70-3554-4cf8-ac30-f7c59b80678d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6824","nonce":2722096729},{"type":"v1","addr":"172.21.15.188:6825","nonce":2722096729}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6826","nonce":2722096729},{"type":"v1","addr":"172.21.15.188:6827","nonce":2722096729}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6830","nonce":2722096729},{"type":"v1","addr":"172.21.15.188:6831","nonce":2722096729}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6828","nonce":2722096729},{"type":"v1","addr":"172.21.15.188:6829","nonce":2722096729}]},"public_addr":"172.21.15.188:6825/2722096729","cluster_addr":"172.21.15.188:6827/2722096729","heartbeat_back_addr":"172.21.15.188:6831/2722096729","heartbeat_front_addr":"172.21.15.188:6829/2722096729","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-26T07:17:36.995032+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-26T07:17:36.932266+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-26T07:17:36.829304+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-26T07:17:36.804297+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-26T07:17:36.667892+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-26T07:17:36.964868+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-26T07:17:36.827332+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-26T07:17:36.893179+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.143:6835/4031724325":"2023-11-27T07:23:44.794394+0000","172.21.15.143:6833/4031724325":"2023-11-27T07:23:44.794394+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-11-26T07:23:52.506 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:52.503+0000 7fd421abb640 1 -- 172.21.15.143:0/814082592 >> [v2:172.21.15.188:6836/3401128553,v1:172.21.15.188:6837/3401128553] conn(0x7fd3fc066590 msgr2=0x7fd3fc068a50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:23:52.506 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:52.503+0000 7fd421abb640 1 --2- 172.21.15.143:0/814082592 >> [v2:172.21.15.188:6836/3401128553,v1:172.21.15.188:6837/3401128553] conn(0x7fd3fc066590 0x7fd3fc068a50 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7fd40c0ae000 tx=0x7fd40c008040 comp rx=0 tx=0).stop 2023-11-26T07:23:52.506 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:52.503+0000 7fd421abb640 1 -- 172.21.15.143:0/814082592 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fd41c0d1bc0 msgr2=0x7fd41c1606b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:23:52.506 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:52.503+0000 7fd421abb640 1 --2- 172.21.15.143:0/814082592 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fd41c0d1bc0 0x7fd41c1606b0 secure :-1 s=READY pgs=1227 cs=0 l=1 rev1=1 crypto rx=0x7fd40400c450 tx=0x7fd40400c910 comp rx=0 tx=0).stop 2023-11-26T07:23:52.506 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:52.503+0000 7fd421abb640 1 -- 172.21.15.143:0/814082592 shutdown_connections 2023-11-26T07:23:52.506 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:52.503+0000 7fd421abb640 1 --2- 172.21.15.143:0/814082592 >> [v2:172.21.15.188:6836/3401128553,v1:172.21.15.188:6837/3401128553] conn(0x7fd3fc066590 0x7fd3fc068a50 unknown :-1 s=CLOSED pgs=90 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:23:52.506 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:52.503+0000 7fd421abb640 1 --2- 172.21.15.143:0/814082592 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fd41c15d6e0 0x7fd41c154750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:23:52.506 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:52.503+0000 7fd421abb640 1 --2- 172.21.15.143:0/814082592 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fd41c0d1bc0 0x7fd41c1606b0 unknown :-1 s=CLOSED pgs=1227 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:23:52.506 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:52.503+0000 7fd421abb640 1 --2- 172.21.15.143:0/814082592 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fd41c0b2160 0x7fd41c0b88e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:23:52.506 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:52.503+0000 7fd421abb640 1 -- 172.21.15.143:0/814082592 >> 172.21.15.143:0/814082592 conn(0x7fd41c09c550 msgr2=0x7fd41c0b3490 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:23:52.507 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:52.503+0000 7fd421abb640 1 -- 172.21.15.143:0/814082592 shutdown_connections 2023-11-26T07:23:52.507 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:52.503+0000 7fd421abb640 1 -- 172.21.15.143:0/814082592 wait complete. 2023-11-26T07:23:52.525 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-11-26T07:17:41.758597+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 12, 'pool_name': 'cephfs_metadata', 'create_time': '2023-11-26T07:23:26.724016+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '83', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 2, 'score_stable': 2, 'optimal_score': 1, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 13, 'pool_name': 'cephfs_data', 'create_time': '2023-11-26T07:23:28.019663+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '84', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.3799999952316284, 'score_stable': 1.3799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.3799999952316284, 'raw_score_stable': 1.3799999952316284, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2023-11-26T07:23:52.526 DEBUG:teuthology.orchestra.run.smithi143:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-11-26T07:23:52.526 DEBUG:teuthology.orchestra.run.smithi143:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-26T07:23:52.531 DEBUG:teuthology.orchestra.run.smithi188:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-26T07:23:52.670 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:52.667+0000 7f84183c1640 1 Processor -- start 2023-11-26T07:23:52.670 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:52.667+0000 7f84183c1640 1 -- start start 2023-11-26T07:23:52.672 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:52.671+0000 7f84183c1640 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7f8410143ac0 0x7f8410070370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:23:52.672 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:52.671+0000 7f84183c1640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f84100708b0 0x7f84100cf280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:23:52.672 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:52.671+0000 7f84183c1640 1 -- --> v1:172.21.15.143:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f84100cfbf0 con 0x7f84100cf7c0 2023-11-26T07:23:52.672 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:52.671+0000 7f84183c1640 1 -- --> v2:172.21.15.143:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8410143ea0 con 0x7f8410143ac0 2023-11-26T07:23:52.672 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:52.671+0000 7f84183c1640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f84100cc730 con 0x7f84100708b0 2023-11-26T07:23:52.673 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:52.671+0000 7f841615d640 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7f8410143ac0 0x7f8410070370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:23:52.673 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:52.671+0000 7f841695e640 1 --1- >> v1:172.21.15.143:6789/0 conn(0x7f84100cf7c0 0x7f84100b1ef0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.143:6789/0 says I am v1:172.21.15.143:46468/0 (socket says 172.21.15.143:46468) 2023-11-26T07:23:52.673 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:52.671+0000 7f841695e640 1 -- 172.21.15.143:0/3947145328 learned_addr learned my addr 172.21.15.143:0/3947145328 (peer_addr_for_me v1:172.21.15.143:0/0) 2023-11-26T07:23:52.673 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:52.671+0000 7f841595c640 1 --2- 172.21.15.143:0/3947145328 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f84100708b0 0x7f84100cf280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:23:52.673 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:52.671+0000 7f841615d640 1 -- 172.21.15.143:0/3947145328 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f84100708b0 msgr2=0x7f84100cf280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-26T07:23:52.673 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:52.671+0000 7f841615d640 1 --2- 172.21.15.143:0/3947145328 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f84100708b0 0x7f84100cf280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:23:52.673 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:52.671+0000 7f841615d640 1 -- 172.21.15.143:0/3947145328 >> v1:172.21.15.143:6789/0 conn(0x7f84100cf7c0 legacy=0x7f84100b1ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:23:52.673 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:52.671+0000 7f841615d640 1 -- 172.21.15.143:0/3947145328 --> v2:172.21.15.143:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8410070ee0 con 0x7f8410143ac0 2023-11-26T07:23:52.673 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:52.671+0000 7f841515b640 1 -- 172.21.15.143:0/3947145328 <== mon.0 v1:172.21.15.143:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3437089216 0 0) 0x7f84100cfbf0 con 0x7f84100cf7c0 2023-11-26T07:23:52.673 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:52.671+0000 7f841595c640 1 --2- 172.21.15.143:0/3947145328 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f84100708b0 0x7f84100cf280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-26T07:23:52.675 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:52.675+0000 7f841615d640 1 --2- 172.21.15.143:0/3947145328 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f8410143ac0 0x7f8410070370 secure :-1 s=READY pgs=1229 cs=0 l=1 rev1=1 crypto rx=0x7f840c00ba60 tx=0x7f840c01eb50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2b991309eefe4ee9 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:23:52.675 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:52.675+0000 7f841515b640 1 -- 172.21.15.143:0/3947145328 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f840c0a7b30 con 0x7f8410143ac0 2023-11-26T07:23:52.675 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:52.675+0000 7f841515b640 1 -- 172.21.15.143:0/3947145328 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f840c0a7c90 con 0x7f8410143ac0 2023-11-26T07:23:52.675 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:52.675+0000 7f841515b640 1 -- 172.21.15.143:0/3947145328 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f840c0af440 con 0x7f8410143ac0 2023-11-26T07:23:52.676 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:52.675+0000 7f84183c1640 1 -- 172.21.15.143:0/3947145328 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f8410143ac0 msgr2=0x7f8410070370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:23:52.676 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:52.675+0000 7f84183c1640 1 --2- 172.21.15.143:0/3947145328 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f8410143ac0 0x7f8410070370 secure :-1 s=READY pgs=1229 cs=0 l=1 rev1=1 crypto rx=0x7f840c00ba60 tx=0x7f840c01eb50 comp rx=0 tx=0).stop 2023-11-26T07:23:52.676 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:52.675+0000 7f84183c1640 1 -- 172.21.15.143:0/3947145328 shutdown_connections 2023-11-26T07:23:52.676 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:52.675+0000 7f84183c1640 1 --2- 172.21.15.143:0/3947145328 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f84100708b0 0x7f84100cf280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:23:52.677 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:52.675+0000 7f84183c1640 1 --2- 172.21.15.143:0/3947145328 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f8410143ac0 0x7f8410070370 unknown :-1 s=CLOSED pgs=1229 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:23:52.677 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:52.675+0000 7f84183c1640 1 -- 172.21.15.143:0/3947145328 >> 172.21.15.143:0/3947145328 conn(0x7f841006a890 msgr2=0x7f841006aca0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:23:52.677 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:52.675+0000 7f84183c1640 1 -- 172.21.15.143:0/3947145328 shutdown_connections 2023-11-26T07:23:52.677 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:52.675+0000 7f84183c1640 1 -- 172.21.15.143:0/3947145328 wait complete. 2023-11-26T07:23:52.678 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:52.675+0000 7f84183c1640 1 Processor -- start 2023-11-26T07:23:52.678 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:52.675+0000 7f84183c1640 1 -- start start 2023-11-26T07:23:52.684 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:52.679+0000 7f84183c1640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f84100708b0 0x7f841013e850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:23:52.684 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:52.679+0000 7f84183c1640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f84100cf7c0 0x7f841013b010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:23:52.685 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:52.679+0000 7f84183c1640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f8410143ac0 0x7f841013ce40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:23:52.685 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:52.679+0000 7f84183c1640 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f841013b550 con 0x7f84100cf7c0 2023-11-26T07:23:52.685 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:52.679+0000 7f84183c1640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f841013ee30 con 0x7f84100708b0 2023-11-26T07:23:52.685 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:52.679+0000 7f84183c1640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f841013f110 con 0x7f8410143ac0 2023-11-26T07:23:52.685 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:52.679+0000 7f841595c640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f84100cf7c0 0x7f841013b010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:23:52.685 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:52.679+0000 7f841595c640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f84100cf7c0 0x7f841013b010 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:36988/0 (socket says 172.21.15.143:36988) 2023-11-26T07:23:52.685 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:52.679+0000 7f841595c640 1 -- 172.21.15.143:0/3867129830 learned_addr learned my addr 172.21.15.143:0/3867129830 (peer_addr_for_me v2:172.21.15.143:0/0) 2023-11-26T07:23:52.685 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:52.679+0000 7f841615d640 1 --2- 172.21.15.143:0/3867129830 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f84100708b0 0x7f841013e850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:23:52.685 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:52.679+0000 7f841695e640 1 --2- 172.21.15.143:0/3867129830 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f8410143ac0 0x7f841013ce40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:23:52.685 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:52.679+0000 7f841595c640 1 -- 172.21.15.143:0/3867129830 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f8410143ac0 msgr2=0x7f841013ce40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:23:52.685 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:52.679+0000 7f841595c640 1 --2- 172.21.15.143:0/3867129830 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f8410143ac0 0x7f841013ce40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:23:52.685 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:52.679+0000 7f841595c640 1 -- 172.21.15.143:0/3867129830 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f84100708b0 msgr2=0x7f841013e850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:23:52.686 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:52.679+0000 7f841595c640 1 --2- 172.21.15.143:0/3867129830 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f84100708b0 0x7f841013e850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:23:52.686 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:52.679+0000 7f841595c640 1 -- 172.21.15.143:0/3867129830 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f840c00b6c0 con 0x7f84100cf7c0 2023-11-26T07:23:52.686 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:52.679+0000 7f841595c640 1 --2- 172.21.15.143:0/3867129830 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f84100cf7c0 0x7f841013b010 secure :-1 s=READY pgs=1230 cs=0 l=1 rev1=1 crypto rx=0x7f840000e810 tx=0x7f840000ecd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:23:52.686 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:52.679+0000 7f8406ffd640 1 -- 172.21.15.143:0/3867129830 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f840000c8d0 con 0x7f84100cf7c0 2023-11-26T07:23:52.686 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:52.679+0000 7f8406ffd640 1 -- 172.21.15.143:0/3867129830 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f840000ca30 con 0x7f84100cf7c0 2023-11-26T07:23:52.686 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:52.679+0000 7f84183c1640 1 -- 172.21.15.143:0/3867129830 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f841013f3f0 con 0x7f84100cf7c0 2023-11-26T07:23:52.686 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:52.679+0000 7f8406ffd640 1 -- 172.21.15.143:0/3867129830 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f840000cdc0 con 0x7f84100cf7c0 2023-11-26T07:23:52.686 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:52.679+0000 7f84183c1640 1 -- 172.21.15.143:0/3867129830 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f841013d380 con 0x7f84100cf7c0 2023-11-26T07:23:52.686 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:52.679+0000 7f84183c1640 1 -- 172.21.15.143:0/3867129830 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f84100af8d0 con 0x7f84100cf7c0 2023-11-26T07:23:52.686 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:52.683+0000 7f8406ffd640 1 -- 172.21.15.143:0/3867129830 <== mon.0 v2:172.21.15.143:3300/0 4 ==== mgrmap(e 16) v1 ==== 85954+0+0 (secure 0 0 0) 0x7f8400013410 con 0x7f84100cf7c0 2023-11-26T07:23:52.686 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:52.683+0000 7f8406ffd640 1 --2- 172.21.15.143:0/3867129830 >> [v2:172.21.15.188:6836/3401128553,v1:172.21.15.188:6837/3401128553] conn(0x7f83f4066590 0x7f83f4068a50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:23:52.686 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:52.683+0000 7f8406ffd640 1 -- 172.21.15.143:0/3867129830 <== mon.0 v2:172.21.15.143:3300/0 5 ==== osd_map(91..91 src has 1..91) v4 ==== 5795+0+0 (secure 0 0 0) 0x7f8400083e70 con 0x7f84100cf7c0 2023-11-26T07:23:52.687 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:52.683+0000 7f841615d640 1 --2- 172.21.15.143:0/3867129830 >> [v2:172.21.15.188:6836/3401128553,v1:172.21.15.188:6837/3401128553] conn(0x7f83f4066590 0x7f83f4068a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:23:52.687 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:52.683+0000 7f841615d640 1 --2- 172.21.15.143:0/3867129830 >> [v2:172.21.15.188:6836/3401128553,v1:172.21.15.188:6837/3401128553] conn(0x7f83f4066590 0x7f83f4068a50 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7f840c0b1000 tx=0x7f840c0a9070 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:23:52.687 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:52.687+0000 7f8406ffd640 1 -- 172.21.15.143:0/3867129830 <== mon.0 v2:172.21.15.143:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172649 (secure 0 0 0) 0x7f8400051100 con 0x7f84100cf7c0 2023-11-26T07:23:52.837 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:52.835+0000 7f84183c1640 1 -- 172.21.15.143:0/3867129830 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f8410067680 con 0x7f84100cf7c0 2023-11-26T07:23:52.837 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:52.835+0000 7f8406ffd640 1 -- 172.21.15.143:0/3867129830 <== mon.0 v2:172.21.15.143:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v91) v1 ==== 93+0+10 (secure 0 0 0) 0x7f8400009480 con 0x7f84100cf7c0 2023-11-26T07:23:52.837 INFO:teuthology.orchestra.run.smithi143.stdout:pg_num: 1 2023-11-26T07:23:52.840 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:52.839+0000 7f84183c1640 1 -- 172.21.15.143:0/3867129830 >> [v2:172.21.15.188:6836/3401128553,v1:172.21.15.188:6837/3401128553] conn(0x7f83f4066590 msgr2=0x7f83f4068a50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:23:52.840 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:52.839+0000 7f84183c1640 1 --2- 172.21.15.143:0/3867129830 >> [v2:172.21.15.188:6836/3401128553,v1:172.21.15.188:6837/3401128553] conn(0x7f83f4066590 0x7f83f4068a50 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7f840c0b1000 tx=0x7f840c0a9070 comp rx=0 tx=0).stop 2023-11-26T07:23:52.840 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:52.839+0000 7f84183c1640 1 -- 172.21.15.143:0/3867129830 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f84100cf7c0 msgr2=0x7f841013b010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:23:52.840 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:52.839+0000 7f84183c1640 1 --2- 172.21.15.143:0/3867129830 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f84100cf7c0 0x7f841013b010 secure :-1 s=READY pgs=1230 cs=0 l=1 rev1=1 crypto rx=0x7f840000e810 tx=0x7f840000ecd0 comp rx=0 tx=0).stop 2023-11-26T07:23:52.840 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:52.839+0000 7f84183c1640 1 -- 172.21.15.143:0/3867129830 shutdown_connections 2023-11-26T07:23:52.840 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:52.839+0000 7f84183c1640 1 --2- 172.21.15.143:0/3867129830 >> [v2:172.21.15.188:6836/3401128553,v1:172.21.15.188:6837/3401128553] conn(0x7f83f4066590 0x7f83f4068a50 unknown :-1 s=CLOSED pgs=91 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:23:52.840 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:52.839+0000 7f84183c1640 1 --2- 172.21.15.143:0/3867129830 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f8410143ac0 0x7f841013ce40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:23:52.840 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:52.839+0000 7f84183c1640 1 --2- 172.21.15.143:0/3867129830 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f84100cf7c0 0x7f841013b010 unknown :-1 s=CLOSED pgs=1230 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:23:52.840 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:52.839+0000 7f84183c1640 1 --2- 172.21.15.143:0/3867129830 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f84100708b0 0x7f841013e850 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:23:52.840 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:52.839+0000 7f84183c1640 1 -- 172.21.15.143:0/3867129830 >> 172.21.15.143:0/3867129830 conn(0x7f841006a890 msgr2=0x7f84100b2f70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:23:52.840 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:52.839+0000 7f84183c1640 1 -- 172.21.15.143:0/3867129830 shutdown_connections 2023-11-26T07:23:52.841 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:52.839+0000 7f84183c1640 1 -- 172.21.15.143:0/3867129830 wait complete. 2023-11-26T07:23:52.859 DEBUG:teuthology.orchestra.run.smithi143:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-11-26T07:23:53.000 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:52.999+0000 7f806652b640 1 Processor -- start 2023-11-26T07:23:53.000 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:52.999+0000 7f806652b640 1 -- start start 2023-11-26T07:23:53.002 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:52.999+0000 7f806652b640 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7f8060144720 0x7f806006d300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:23:53.002 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:52.999+0000 7f806652b640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f806006d840 0x7f80600714e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:23:53.002 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:52.999+0000 7f806652b640 1 -- --> v1:172.21.15.188:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f806006e3f0 con 0x7f8060071a20 2023-11-26T07:23:53.002 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:52.999+0000 7f806652b640 1 -- --> v2:172.21.15.143:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8060071e90 con 0x7f8060144720 2023-11-26T07:23:53.002 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:52.999+0000 7f806652b640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f80600722f0 con 0x7f806006d840 2023-11-26T07:23:53.003 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:52.999+0000 7f8065529640 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7f8060144720 0x7f806006d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:23:53.003 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:52.999+0000 7f8065529640 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7f8060144720 0x7f806006d300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:37004/0 (socket says 172.21.15.143:37004) 2023-11-26T07:23:53.003 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:52.999+0000 7f8065529640 1 -- 172.21.15.143:0/2469798566 learned_addr learned my addr 172.21.15.143:0/2469798566 (peer_addr_for_me v2:172.21.15.143:0/0) 2023-11-26T07:23:53.003 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:53.003+0000 7f8064d28640 1 --2- 172.21.15.143:0/2469798566 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f806006d840 0x7f80600714e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:23:53.003 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:53.003+0000 7f8065529640 1 -- 172.21.15.143:0/2469798566 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f806006d840 msgr2=0x7f80600714e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-26T07:23:53.003 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:53.003+0000 7f8065529640 1 --2- 172.21.15.143:0/2469798566 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f806006d840 0x7f80600714e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:23:53.003 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:53.003+0000 7f8065529640 1 -- 172.21.15.143:0/2469798566 >> v1:172.21.15.188:6789/0 conn(0x7f8060071a20 legacy=0x7f80600cc170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-26T07:23:53.003 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:53.003+0000 7f8065529640 1 -- 172.21.15.143:0/2469798566 --> v2:172.21.15.143:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f806006de30 con 0x7f8060144720 2023-11-26T07:23:53.005 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:53.003+0000 7f8065529640 1 --2- 172.21.15.143:0/2469798566 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f8060144720 0x7f806006d300 secure :-1 s=READY pgs=1231 cs=0 l=1 rev1=1 crypto rx=0x7f805c009870 tx=0x7f805c01c510 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7e9efb7574a39a97 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:23:53.005 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:53.003+0000 7f8057fff640 1 -- 172.21.15.143:0/2469798566 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f805c0ab620 con 0x7f8060144720 2023-11-26T07:23:53.005 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:53.003+0000 7f8057fff640 1 -- 172.21.15.143:0/2469798566 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f805c0ab780 con 0x7f8060144720 2023-11-26T07:23:53.005 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:53.003+0000 7f8057fff640 1 -- 172.21.15.143:0/2469798566 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f805c0abb10 con 0x7f8060144720 2023-11-26T07:23:53.006 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:53.003+0000 7f806652b640 1 -- 172.21.15.143:0/2469798566 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f8060144720 msgr2=0x7f806006d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:23:53.006 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:53.003+0000 7f806652b640 1 --2- 172.21.15.143:0/2469798566 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f8060144720 0x7f806006d300 secure :-1 s=READY pgs=1231 cs=0 l=1 rev1=1 crypto rx=0x7f805c009870 tx=0x7f805c01c510 comp rx=0 tx=0).stop 2023-11-26T07:23:53.006 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:53.007+0000 7f806652b640 1 -- 172.21.15.143:0/2469798566 shutdown_connections 2023-11-26T07:23:53.007 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:53.007+0000 7f806652b640 1 --2- 172.21.15.143:0/2469798566 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f806006d840 0x7f80600714e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:23:53.007 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:53.007+0000 7f806652b640 1 --2- 172.21.15.143:0/2469798566 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f8060144720 0x7f806006d300 unknown :-1 s=CLOSED pgs=1231 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:23:53.007 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:53.007+0000 7f806652b640 1 -- 172.21.15.143:0/2469798566 >> 172.21.15.143:0/2469798566 conn(0x7f806006a890 msgr2=0x7f806006aca0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:23:53.007 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:53.007+0000 7f806652b640 1 -- 172.21.15.143:0/2469798566 shutdown_connections 2023-11-26T07:23:53.007 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:53.007+0000 7f806652b640 1 -- 172.21.15.143:0/2469798566 wait complete. 2023-11-26T07:23:53.008 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:53.007+0000 7f806652b640 1 Processor -- start 2023-11-26T07:23:53.008 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:53.007+0000 7f806652b640 1 -- start start 2023-11-26T07:23:53.009 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:53.007+0000 7f806652b640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f806006d840 0x7f8060137400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:23:53.009 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:53.007+0000 7f806652b640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f8060071a20 0x7f8060158b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:23:53.009 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:53.007+0000 7f806652b640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f8060144720 0x7f806013c2a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:23:53.009 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:53.007+0000 7f806652b640 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f80601421d0 con 0x7f8060071a20 2023-11-26T07:23:53.009 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:53.007+0000 7f806652b640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f80601418a0 con 0x7f806006d840 2023-11-26T07:23:53.010 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:53.007+0000 7f806652b640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8060141b80 con 0x7f8060144720 2023-11-26T07:23:53.010 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:53.007+0000 7f8064d28640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f8060071a20 0x7f8060158b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:23:53.010 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:53.007+0000 7f8064d28640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f8060071a20 0x7f8060158b50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:37020/0 (socket says 172.21.15.143:37020) 2023-11-26T07:23:53.010 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:53.007+0000 7f8064d28640 1 -- 172.21.15.143:0/1997210873 learned_addr learned my addr 172.21.15.143:0/1997210873 (peer_addr_for_me v2:172.21.15.143:0/0) 2023-11-26T07:23:53.010 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:53.007+0000 7f8065d2a640 1 --2- 172.21.15.143:0/1997210873 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f8060144720 0x7f806013c2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:23:53.010 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:53.007+0000 7f8065529640 1 --2- 172.21.15.143:0/1997210873 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f806006d840 0x7f8060137400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:23:53.012 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:53.007+0000 7f8064d28640 1 -- 172.21.15.143:0/1997210873 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f8060144720 msgr2=0x7f806013c2a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:23:53.012 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:53.007+0000 7f8064d28640 1 --2- 172.21.15.143:0/1997210873 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f8060144720 0x7f806013c2a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:23:53.012 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:53.007+0000 7f8064d28640 1 -- 172.21.15.143:0/1997210873 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f806006d840 msgr2=0x7f8060137400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:23:53.012 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:53.007+0000 7f8064d28640 1 --2- 172.21.15.143:0/1997210873 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f806006d840 0x7f8060137400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:23:53.012 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:53.007+0000 7f8064d28640 1 -- 172.21.15.143:0/1997210873 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f805c0094d0 con 0x7f8060071a20 2023-11-26T07:23:53.013 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:53.007+0000 7f8064d28640 1 --2- 172.21.15.143:0/1997210873 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f8060071a20 0x7f8060158b50 secure :-1 s=READY pgs=1232 cs=0 l=1 rev1=1 crypto rx=0x7f805000d5f0 tx=0x7f805000dab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:23:53.013 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:53.011+0000 7f8055ffb640 1 -- 172.21.15.143:0/1997210873 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f80500125f0 con 0x7f8060071a20 2023-11-26T07:23:53.013 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:53.011+0000 7f8055ffb640 1 -- 172.21.15.143:0/1997210873 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8050012750 con 0x7f8060071a20 2023-11-26T07:23:53.013 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:53.011+0000 7f8055ffb640 1 -- 172.21.15.143:0/1997210873 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8050012ae0 con 0x7f8060071a20 2023-11-26T07:23:53.013 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:53.011+0000 7f806652b640 1 -- 172.21.15.143:0/1997210873 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8060141e60 con 0x7f8060071a20 2023-11-26T07:23:53.013 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:53.011+0000 7f806652b640 1 -- 172.21.15.143:0/1997210873 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8060136c20 con 0x7f8060071a20 2023-11-26T07:23:53.013 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:53.011+0000 7f806652b640 1 -- 172.21.15.143:0/1997210873 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8060067500 con 0x7f8060071a20 2023-11-26T07:23:53.014 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:53.011+0000 7f8055ffb640 1 -- 172.21.15.143:0/1997210873 <== mon.0 v2:172.21.15.143:3300/0 4 ==== mgrmap(e 16) v1 ==== 85954+0+0 (secure 0 0 0) 0x7f8050012c40 con 0x7f8060071a20 2023-11-26T07:23:53.014 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:53.011+0000 7f8055ffb640 1 --2- 172.21.15.143:0/1997210873 >> [v2:172.21.15.188:6836/3401128553,v1:172.21.15.188:6837/3401128553] conn(0x7f8044066590 0x7f8044068a50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:23:53.014 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:53.011+0000 7f8055ffb640 1 -- 172.21.15.143:0/1997210873 <== mon.0 v2:172.21.15.143:3300/0 5 ==== osd_map(91..91 src has 1..91) v4 ==== 5795+0+0 (secure 0 0 0) 0x7f8050084440 con 0x7f8060071a20 2023-11-26T07:23:53.014 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:53.011+0000 7f8065529640 1 --2- 172.21.15.143:0/1997210873 >> [v2:172.21.15.188:6836/3401128553,v1:172.21.15.188:6837/3401128553] conn(0x7f8044066590 0x7f8044068a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:23:53.015 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:53.015+0000 7f8065529640 1 --2- 172.21.15.143:0/1997210873 >> [v2:172.21.15.188:6836/3401128553,v1:172.21.15.188:6837/3401128553] conn(0x7f8044066590 0x7f8044068a50 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7f805c0af000 tx=0x7f805c0b6040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:23:53.017 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:53.015+0000 7f8055ffb640 1 -- 172.21.15.143:0/1997210873 <== mon.0 v2:172.21.15.143:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172649 (secure 0 0 0) 0x7f8050051760 con 0x7f8060071a20 2023-11-26T07:23:53.173 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:53.171+0000 7f806652b640 1 -- 172.21.15.143:0/1997210873 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f80601352f0 con 0x7f8060071a20 2023-11-26T07:23:53.174 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:53.175+0000 7f8055ffb640 1 -- 172.21.15.143:0/1997210873 <== mon.0 v2:172.21.15.143:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v91) v1 ==== 104+0+11 (secure 0 0 0) 0x7f8050007380 con 0x7f8060071a20 2023-11-26T07:23:53.174 INFO:teuthology.orchestra.run.smithi143.stdout:pg_num: 32 2023-11-26T07:23:53.176 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:53.175+0000 7f806652b640 1 -- 172.21.15.143:0/1997210873 >> [v2:172.21.15.188:6836/3401128553,v1:172.21.15.188:6837/3401128553] conn(0x7f8044066590 msgr2=0x7f8044068a50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:23:53.176 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:53.175+0000 7f806652b640 1 --2- 172.21.15.143:0/1997210873 >> [v2:172.21.15.188:6836/3401128553,v1:172.21.15.188:6837/3401128553] conn(0x7f8044066590 0x7f8044068a50 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7f805c0af000 tx=0x7f805c0b6040 comp rx=0 tx=0).stop 2023-11-26T07:23:53.176 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:53.175+0000 7f806652b640 1 -- 172.21.15.143:0/1997210873 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f8060071a20 msgr2=0x7f8060158b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:23:53.177 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:53.175+0000 7f806652b640 1 --2- 172.21.15.143:0/1997210873 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f8060071a20 0x7f8060158b50 secure :-1 s=READY pgs=1232 cs=0 l=1 rev1=1 crypto rx=0x7f805000d5f0 tx=0x7f805000dab0 comp rx=0 tx=0).stop 2023-11-26T07:23:53.177 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:53.175+0000 7f806652b640 1 -- 172.21.15.143:0/1997210873 shutdown_connections 2023-11-26T07:23:53.177 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:53.175+0000 7f806652b640 1 --2- 172.21.15.143:0/1997210873 >> [v2:172.21.15.188:6836/3401128553,v1:172.21.15.188:6837/3401128553] conn(0x7f8044066590 0x7f8044068a50 unknown :-1 s=CLOSED pgs=92 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:23:53.177 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:53.175+0000 7f806652b640 1 --2- 172.21.15.143:0/1997210873 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f8060144720 0x7f806013c2a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:23:53.177 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:53.175+0000 7f806652b640 1 --2- 172.21.15.143:0/1997210873 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f8060071a20 0x7f8060158b50 unknown :-1 s=CLOSED pgs=1232 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:23:53.177 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:53.175+0000 7f806652b640 1 --2- 172.21.15.143:0/1997210873 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f806006d840 0x7f8060137400 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:23:53.177 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:53.175+0000 7f806652b640 1 -- 172.21.15.143:0/1997210873 >> 172.21.15.143:0/1997210873 conn(0x7f806006a890 msgr2=0x7f80600b0af0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:23:53.177 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:53.175+0000 7f806652b640 1 -- 172.21.15.143:0/1997210873 shutdown_connections 2023-11-26T07:23:53.177 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:53.175+0000 7f806652b640 1 -- 172.21.15.143:0/1997210873 wait complete. 2023-11-26T07:23:53.195 DEBUG:teuthology.orchestra.run.smithi143:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-11-26T07:23:53.336 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:53.335+0000 7f3af9bec640 1 Processor -- start 2023-11-26T07:23:53.337 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:53.335+0000 7f3af9bec640 1 -- start start 2023-11-26T07:23:53.339 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:53.335+0000 7f3af9bec640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f3af40cf710 0x7f3af40cd8c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:23:53.339 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:53.335+0000 7f3af9bec640 1 -- --> v1:172.21.15.143:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3af40cc730 con 0x7f3af4143ac0 2023-11-26T07:23:53.339 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:53.335+0000 7f3af9bec640 1 -- --> v1:172.21.15.188:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3af4070ee0 con 0x7f3af40cf280 2023-11-26T07:23:53.339 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:53.335+0000 7f3af9bec640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3af4143ea0 con 0x7f3af40cf710 2023-11-26T07:23:53.339 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:53.339+0000 7f3af37fe640 1 --1- >> v1:172.21.15.143:6789/0 conn(0x7f3af4143ac0 0x7f3af4070370 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.143:6789/0 says I am v1:172.21.15.143:46472/0 (socket says 172.21.15.143:46472) 2023-11-26T07:23:53.339 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:53.339+0000 7f3af37fe640 1 -- 172.21.15.143:0/674449347 learned_addr learned my addr 172.21.15.143:0/674449347 (peer_addr_for_me v1:172.21.15.143:0/0) 2023-11-26T07:23:53.339 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:53.339+0000 7f3af3fff640 1 --2- 172.21.15.143:0/674449347 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f3af40cf710 0x7f3af40cd8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:23:53.340 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:53.339+0000 7f3af2ffd640 1 -- 172.21.15.143:0/674449347 <== mon.0 v1:172.21.15.143:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1441246373 0 0) 0x7f3af40cc730 con 0x7f3af4143ac0 2023-11-26T07:23:53.340 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:53.339+0000 7f3af2ffd640 1 -- 172.21.15.143:0/674449347 --> v1:172.21.15.143:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f3ae0003530 con 0x7f3af4143ac0 2023-11-26T07:23:53.340 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:53.339+0000 7f3af2ffd640 1 -- 172.21.15.143:0/674449347 <== mon.1 v1:172.21.15.188:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2244287701 0 0) 0x7f3af4070ee0 con 0x7f3af40cf280 2023-11-26T07:23:53.340 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:53.339+0000 7f3af2ffd640 1 -- 172.21.15.143:0/674449347 --> v1:172.21.15.188:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f3af40cc730 con 0x7f3af40cf280 2023-11-26T07:23:53.340 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:53.339+0000 7f3af2ffd640 1 -- 172.21.15.143:0/674449347 <== mon.0 v1:172.21.15.143:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3521326571 0 0) 0x7f3ae0003530 con 0x7f3af4143ac0 2023-11-26T07:23:53.340 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:53.339+0000 7f3af3fff640 1 -- 172.21.15.143:0/674449347 >> v1:172.21.15.188:6789/0 conn(0x7f3af40cf280 legacy=0x7f3af40cd1b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:23:53.340 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:53.339+0000 7f3af3fff640 1 -- 172.21.15.143:0/674449347 >> v1:172.21.15.143:6789/0 conn(0x7f3af4143ac0 legacy=0x7f3af4070370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:23:53.340 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:53.339+0000 7f3af3fff640 1 -- 172.21.15.143:0/674449347 --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3af40711c0 con 0x7f3af40cf710 2023-11-26T07:23:53.341 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:53.339+0000 7f3af3fff640 1 --2- 172.21.15.143:0/674449347 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f3af40cf710 0x7f3af40cd8c0 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7f3aec009dc0 tx=0x7f3aec01c890 comp rx=0 tx=0).ready entity=mon.2 client_cookie=cb46c5c1f9aad114 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:23:53.342 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:53.339+0000 7f3af2ffd640 1 -- 172.21.15.143:0/674449347 <== mon.2 v2:172.21.15.188:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3ae4001620 con 0x7f3af40cf710 2023-11-26T07:23:53.342 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:53.339+0000 7f3af2ffd640 1 -- 172.21.15.143:0/674449347 <== mon.2 v2:172.21.15.188:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3aec0a56d0 con 0x7f3af40cf710 2023-11-26T07:23:53.342 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:53.343+0000 7f3af2ffd640 1 -- 172.21.15.143:0/674449347 <== mon.2 v2:172.21.15.188:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3aec0a5a40 con 0x7f3af40cf710 2023-11-26T07:23:53.343 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:53.343+0000 7f3af9bec640 1 -- 172.21.15.143:0/674449347 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f3af40cf710 msgr2=0x7f3af40cd8c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:23:53.343 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:53.343+0000 7f3af9bec640 1 --2- 172.21.15.143:0/674449347 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f3af40cf710 0x7f3af40cd8c0 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7f3aec009dc0 tx=0x7f3aec01c890 comp rx=0 tx=0).stop 2023-11-26T07:23:53.343 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:53.343+0000 7f3af9bec640 1 -- 172.21.15.143:0/674449347 shutdown_connections 2023-11-26T07:23:53.343 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:53.343+0000 7f3af9bec640 1 --2- 172.21.15.143:0/674449347 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f3af40cf710 0x7f3af40cd8c0 unknown :-1 s=CLOSED pgs=149 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:23:53.343 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:53.343+0000 7f3af9bec640 1 -- 172.21.15.143:0/674449347 >> 172.21.15.143:0/674449347 conn(0x7f3af406a890 msgr2=0x7f3af406aca0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:23:53.343 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:53.343+0000 7f3af9bec640 1 -- 172.21.15.143:0/674449347 shutdown_connections 2023-11-26T07:23:53.343 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:53.343+0000 7f3af9bec640 1 -- 172.21.15.143:0/674449347 wait complete. 2023-11-26T07:23:53.344 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:53.343+0000 7f3af9bec640 1 Processor -- start 2023-11-26T07:23:53.344 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:53.343+0000 7f3af9bec640 1 -- start start 2023-11-26T07:23:53.346 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:53.343+0000 7f3af9bec640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f3af40cf280 0x7f3af413a590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:23:53.346 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:53.343+0000 7f3af9bec640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f3af40cf710 0x7f3af4159720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:23:53.346 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:53.343+0000 7f3af9bec640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f3af4143ac0 0x7f3af4157b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:23:53.346 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:53.343+0000 7f3af9bec640 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3af413aad0 con 0x7f3af4143ac0 2023-11-26T07:23:53.346 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:53.343+0000 7f3af9bec640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3af4156c30 con 0x7f3af40cf710 2023-11-26T07:23:53.346 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:53.343+0000 7f3af9bec640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3af4158680 con 0x7f3af40cf280 2023-11-26T07:23:53.346 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:53.343+0000 7f3af3fff640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f3af4143ac0 0x7f3af4157b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:23:53.346 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:53.343+0000 7f3af3fff640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f3af4143ac0 0x7f3af4157b30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:37026/0 (socket says 172.21.15.143:37026) 2023-11-26T07:23:53.346 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:53.343+0000 7f3af3fff640 1 -- 172.21.15.143:0/3362294232 learned_addr learned my addr 172.21.15.143:0/3362294232 (peer_addr_for_me v2:172.21.15.143:0/0) 2023-11-26T07:23:53.347 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:53.343+0000 7f3af37fe640 1 --2- 172.21.15.143:0/3362294232 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f3af40cf280 0x7f3af413a590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:23:53.347 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:53.343+0000 7f3aebfff640 1 --2- 172.21.15.143:0/3362294232 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f3af40cf710 0x7f3af4159720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:23:53.347 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:53.347+0000 7f3af3fff640 1 -- 172.21.15.143:0/3362294232 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f3af40cf280 msgr2=0x7f3af413a590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:23:53.348 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:53.347+0000 7f3af3fff640 1 --2- 172.21.15.143:0/3362294232 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f3af40cf280 0x7f3af413a590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:23:53.348 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:53.347+0000 7f3af3fff640 1 -- 172.21.15.143:0/3362294232 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f3af40cf710 msgr2=0x7f3af4159720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:23:53.348 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:53.347+0000 7f3af3fff640 1 --2- 172.21.15.143:0/3362294232 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f3af40cf710 0x7f3af4159720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:23:53.348 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:53.347+0000 7f3af3fff640 1 -- 172.21.15.143:0/3362294232 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3aec009a70 con 0x7f3af4143ac0 2023-11-26T07:23:53.348 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:53.347+0000 7f3af3fff640 1 --2- 172.21.15.143:0/3362294232 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f3af4143ac0 0x7f3af4157b30 secure :-1 s=READY pgs=1234 cs=0 l=1 rev1=1 crypto rx=0x7f3aec01af40 tx=0x7f3af40ce610 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:23:53.348 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:53.347+0000 7f3af0ff9640 1 -- 172.21.15.143:0/3362294232 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3aec0bf070 con 0x7f3af4143ac0 2023-11-26T07:23:53.348 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:53.347+0000 7f3af0ff9640 1 -- 172.21.15.143:0/3362294232 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3aec009450 con 0x7f3af4143ac0 2023-11-26T07:23:53.348 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:53.347+0000 7f3af0ff9640 1 -- 172.21.15.143:0/3362294232 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3aec007000 con 0x7f3af4143ac0 2023-11-26T07:23:53.348 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:53.347+0000 7f3af9bec640 1 -- 172.21.15.143:0/3362294232 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3af4158960 con 0x7f3af4143ac0 2023-11-26T07:23:53.348 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:53.347+0000 7f3af9bec640 1 -- 172.21.15.143:0/3362294232 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3af4158f20 con 0x7f3af4143ac0 2023-11-26T07:23:53.349 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:53.347+0000 7f3af9bec640 1 -- 172.21.15.143:0/3362294232 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3af4067680 con 0x7f3af4143ac0 2023-11-26T07:23:53.349 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:53.347+0000 7f3af0ff9640 1 -- 172.21.15.143:0/3362294232 <== mon.0 v2:172.21.15.143:3300/0 4 ==== mgrmap(e 16) v1 ==== 85954+0+0 (secure 0 0 0) 0x7f3aec0b4050 con 0x7f3af4143ac0 2023-11-26T07:23:53.350 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:53.347+0000 7f3af0ff9640 1 --2- 172.21.15.143:0/3362294232 >> [v2:172.21.15.188:6836/3401128553,v1:172.21.15.188:6837/3401128553] conn(0x7f3ad4066590 0x7f3ad4068a50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:23:53.350 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:53.347+0000 7f3af0ff9640 1 -- 172.21.15.143:0/3362294232 <== mon.0 v2:172.21.15.143:3300/0 5 ==== osd_map(91..91 src has 1..91) v4 ==== 5795+0+0 (secure 0 0 0) 0x7f3aec11e2b0 con 0x7f3af4143ac0 2023-11-26T07:23:53.350 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:53.351+0000 7f3af37fe640 1 --2- 172.21.15.143:0/3362294232 >> [v2:172.21.15.188:6836/3401128553,v1:172.21.15.188:6837/3401128553] conn(0x7f3ad4066590 0x7f3ad4068a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:23:53.350 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:53.351+0000 7f3af37fe640 1 --2- 172.21.15.143:0/3362294232 >> [v2:172.21.15.188:6836/3401128553,v1:172.21.15.188:6837/3401128553] conn(0x7f3ad4066590 0x7f3ad4068a50 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7f3ae4001b40 tx=0x7f3ae4001ad0 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:23:53.353 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:53.351+0000 7f3af0ff9640 1 -- 172.21.15.143:0/3362294232 <== mon.0 v2:172.21.15.143:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172649 (secure 0 0 0) 0x7f3aec0eb540 con 0x7f3af4143ac0 2023-11-26T07:23:53.506 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:53.503+0000 7f3af9bec640 1 -- 172.21.15.143:0/3362294232 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f3af40b1990 con 0x7f3af4143ac0 2023-11-26T07:23:53.506 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:53.507+0000 7f3af0ff9640 1 -- 172.21.15.143:0/3362294232 <== mon.0 v2:172.21.15.143:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v91) v1 ==== 100+0+11 (secure 0 0 0) 0x7f3aec0eeb60 con 0x7f3af4143ac0 2023-11-26T07:23:53.506 INFO:teuthology.orchestra.run.smithi143.stdout:pg_num: 64 2023-11-26T07:23:53.508 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:53.507+0000 7f3af9bec640 1 -- 172.21.15.143:0/3362294232 >> [v2:172.21.15.188:6836/3401128553,v1:172.21.15.188:6837/3401128553] conn(0x7f3ad4066590 msgr2=0x7f3ad4068a50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:23:53.508 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:53.507+0000 7f3af9bec640 1 --2- 172.21.15.143:0/3362294232 >> [v2:172.21.15.188:6836/3401128553,v1:172.21.15.188:6837/3401128553] conn(0x7f3ad4066590 0x7f3ad4068a50 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7f3ae4001b40 tx=0x7f3ae4001ad0 comp rx=0 tx=0).stop 2023-11-26T07:23:53.508 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:53.507+0000 7f3af9bec640 1 -- 172.21.15.143:0/3362294232 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f3af4143ac0 msgr2=0x7f3af4157b30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:23:53.508 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:53.507+0000 7f3af9bec640 1 --2- 172.21.15.143:0/3362294232 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f3af4143ac0 0x7f3af4157b30 secure :-1 s=READY pgs=1234 cs=0 l=1 rev1=1 crypto rx=0x7f3aec01af40 tx=0x7f3af40ce610 comp rx=0 tx=0).stop 2023-11-26T07:23:53.509 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:53.507+0000 7f3af9bec640 1 -- 172.21.15.143:0/3362294232 shutdown_connections 2023-11-26T07:23:53.509 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:53.507+0000 7f3af9bec640 1 --2- 172.21.15.143:0/3362294232 >> [v2:172.21.15.188:6836/3401128553,v1:172.21.15.188:6837/3401128553] conn(0x7f3ad4066590 0x7f3ad4068a50 unknown :-1 s=CLOSED pgs=93 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:23:53.509 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:53.507+0000 7f3af9bec640 1 --2- 172.21.15.143:0/3362294232 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f3af4143ac0 0x7f3af4157b30 unknown :-1 s=CLOSED pgs=1234 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:23:53.509 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:53.507+0000 7f3af9bec640 1 --2- 172.21.15.143:0/3362294232 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f3af40cf710 0x7f3af4159720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:23:53.509 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:53.507+0000 7f3af9bec640 1 --2- 172.21.15.143:0/3362294232 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f3af40cf280 0x7f3af413a590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:23:53.509 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:53.507+0000 7f3af9bec640 1 -- 172.21.15.143:0/3362294232 >> 172.21.15.143:0/3362294232 conn(0x7f3af406a890 msgr2=0x7f3af40cd1b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:23:53.509 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:53.507+0000 7f3af9bec640 1 -- 172.21.15.143:0/3362294232 shutdown_connections 2023-11-26T07:23:53.509 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:53.507+0000 7f3af9bec640 1 -- 172.21.15.143:0/3362294232 wait complete. 2023-11-26T07:23:53.527 DEBUG:teuthology.orchestra.run.smithi143:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-11-26T07:23:53.667 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:53.667+0000 7fd74502f640 1 Processor -- start 2023-11-26T07:23:53.667 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:53.667+0000 7fd74502f640 1 -- start start 2023-11-26T07:23:53.669 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:53.667+0000 7fd74502f640 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7fd7400b23f0 0x7fd7400ada60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:23:53.669 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:53.667+0000 7fd74502f640 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7fd74006b9d0 0x7fd74006be10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:23:53.669 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:53.667+0000 7fd74502f640 1 -- --> v1:172.21.15.188:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd7400cc050 con 0x7fd7400cfaf0 2023-11-26T07:23:53.669 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:53.667+0000 7fd74502f640 1 -- --> v2:172.21.15.143:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd74006c350 con 0x7fd74006b9d0 2023-11-26T07:23:53.669 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:53.667+0000 7fd74502f640 1 -- --> v2:172.21.15.188:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd7400cb440 con 0x7fd7400b23f0 2023-11-26T07:23:53.669 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:53.667+0000 7fd73f7fe640 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7fd74006b9d0 0x7fd74006be10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:23:53.669 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:53.667+0000 7fd73f7fe640 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7fd74006b9d0 0x7fd74006be10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:37040/0 (socket says 172.21.15.143:37040) 2023-11-26T07:23:53.669 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:53.667+0000 7fd73f7fe640 1 -- 172.21.15.143:0/2427851567 learned_addr learned my addr 172.21.15.143:0/2427851567 (peer_addr_for_me v2:172.21.15.143:0/0) 2023-11-26T07:23:53.669 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:53.667+0000 7fd73ffff640 1 --2- 172.21.15.143:0/2427851567 >> v2:172.21.15.188:3300/0 conn(0x7fd7400b23f0 0x7fd7400ada60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:23:53.669 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:53.667+0000 7fd73f7fe640 1 -- 172.21.15.143:0/2427851567 >> v2:172.21.15.188:3300/0 conn(0x7fd7400b23f0 msgr2=0x7fd7400ada60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-26T07:23:53.670 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:53.667+0000 7fd73f7fe640 1 --2- 172.21.15.143:0/2427851567 >> v2:172.21.15.188:3300/0 conn(0x7fd7400b23f0 0x7fd7400ada60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:23:53.670 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:53.667+0000 7fd73f7fe640 1 -- 172.21.15.143:0/2427851567 >> v1:172.21.15.188:6789/0 conn(0x7fd7400cfaf0 legacy=0x7fd7400cff80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:23:53.670 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:53.667+0000 7fd73f7fe640 1 -- 172.21.15.143:0/2427851567 --> v2:172.21.15.143:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd7400cb720 con 0x7fd74006b9d0 2023-11-26T07:23:53.672 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:53.671+0000 7fd73f7fe640 1 --2- 172.21.15.143:0/2427851567 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fd74006b9d0 0x7fd74006be10 secure :-1 s=READY pgs=1235 cs=0 l=1 rev1=1 crypto rx=0x7fd7400ce8f0 tx=0x7fd73401c500 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5f06d68df7c6171e server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:23:53.672 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:53.671+0000 7fd73effd640 1 -- 172.21.15.143:0/2427851567 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd7340ab5f0 con 0x7fd74006b9d0 2023-11-26T07:23:53.672 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:53.671+0000 7fd73effd640 1 -- 172.21.15.143:0/2427851567 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd7340ab750 con 0x7fd74006b9d0 2023-11-26T07:23:53.672 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:53.671+0000 7fd73effd640 1 -- 172.21.15.143:0/2427851567 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd7340abae0 con 0x7fd74006b9d0 2023-11-26T07:23:53.673 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:53.671+0000 7fd74502f640 1 -- 172.21.15.143:0/2427851567 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fd74006b9d0 msgr2=0x7fd74006be10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:23:53.673 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:53.671+0000 7fd74502f640 1 --2- 172.21.15.143:0/2427851567 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fd74006b9d0 0x7fd74006be10 secure :-1 s=READY pgs=1235 cs=0 l=1 rev1=1 crypto rx=0x7fd7400ce8f0 tx=0x7fd73401c500 comp rx=0 tx=0).stop 2023-11-26T07:23:53.673 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:53.671+0000 7fd74502f640 1 -- 172.21.15.143:0/2427851567 shutdown_connections 2023-11-26T07:23:53.673 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:53.671+0000 7fd74502f640 1 --2- 172.21.15.143:0/2427851567 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fd74006b9d0 0x7fd74006be10 unknown :-1 s=CLOSED pgs=1235 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:23:53.673 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:53.671+0000 7fd74502f640 1 --2- 172.21.15.143:0/2427851567 >> v2:172.21.15.188:3300/0 conn(0x7fd7400b23f0 0x7fd7400ada60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:23:53.673 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:53.671+0000 7fd74502f640 1 -- 172.21.15.143:0/2427851567 >> 172.21.15.143:0/2427851567 conn(0x7fd7400d0df0 msgr2=0x7fd7400d1200 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:23:53.673 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:53.671+0000 7fd74502f640 1 -- 172.21.15.143:0/2427851567 shutdown_connections 2023-11-26T07:23:53.673 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:53.671+0000 7fd74502f640 1 -- 172.21.15.143:0/2427851567 wait complete. 2023-11-26T07:23:53.674 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:53.675+0000 7fd74502f640 1 Processor -- start 2023-11-26T07:23:53.675 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:53.675+0000 7fd74502f640 1 -- start start 2023-11-26T07:23:53.677 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:53.675+0000 7fd74502f640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fd74006b9d0 0x7fd740153790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:23:53.677 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:53.675+0000 7fd74502f640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fd7400b23f0 0x7fd7401f6bc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:23:53.677 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:53.675+0000 7fd74502f640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fd7400cfaf0 0x7fd740158760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:23:53.677 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:53.675+0000 7fd74502f640 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd740157340 con 0x7fd74006b9d0 2023-11-26T07:23:53.677 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:53.675+0000 7fd74502f640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd7401f7100 con 0x7fd7400b23f0 2023-11-26T07:23:53.677 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:53.675+0000 7fd74502f640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd740158ca0 con 0x7fd7400cfaf0 2023-11-26T07:23:53.677 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:53.675+0000 7fd73ffff640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fd74006b9d0 0x7fd740153790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:23:53.678 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:53.675+0000 7fd73ffff640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fd74006b9d0 0x7fd740153790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:37046/0 (socket says 172.21.15.143:37046) 2023-11-26T07:23:53.678 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:53.675+0000 7fd73ffff640 1 -- 172.21.15.143:0/2128412640 learned_addr learned my addr 172.21.15.143:0/2128412640 (peer_addr_for_me v2:172.21.15.143:0/0) 2023-11-26T07:23:53.678 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:53.675+0000 7fd73f7fe640 1 --2- 172.21.15.143:0/2128412640 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fd7400b23f0 0x7fd7401f6bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:23:53.678 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:53.675+0000 7fd74482e640 1 --2- 172.21.15.143:0/2128412640 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fd7400cfaf0 0x7fd740158760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:23:53.678 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:53.675+0000 7fd73ffff640 1 -- 172.21.15.143:0/2128412640 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fd7400cfaf0 msgr2=0x7fd740158760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:23:53.678 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:53.675+0000 7fd73ffff640 1 --2- 172.21.15.143:0/2128412640 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fd7400cfaf0 0x7fd740158760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:23:53.678 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:53.675+0000 7fd73ffff640 1 -- 172.21.15.143:0/2128412640 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fd7400b23f0 msgr2=0x7fd7401f6bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:23:53.678 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:53.675+0000 7fd73ffff640 1 --2- 172.21.15.143:0/2128412640 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fd7400b23f0 0x7fd7401f6bc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:23:53.678 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:53.675+0000 7fd73ffff640 1 -- 172.21.15.143:0/2128412640 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd7340094d0 con 0x7fd74006b9d0 2023-11-26T07:23:53.678 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:53.675+0000 7fd73f7fe640 1 --2- 172.21.15.143:0/2128412640 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fd7400b23f0 0x7fd7401f6bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-26T07:23:53.678 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:53.675+0000 7fd74482e640 1 --2- 172.21.15.143:0/2128412640 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fd7400cfaf0 0x7fd740158760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-26T07:23:53.678 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:53.675+0000 7fd73ffff640 1 --2- 172.21.15.143:0/2128412640 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fd74006b9d0 0x7fd740153790 secure :-1 s=READY pgs=1236 cs=0 l=1 rev1=1 crypto rx=0x7fd73000e870 tx=0x7fd73000ec70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:23:53.678 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:53.675+0000 7fd73cff9640 1 -- 172.21.15.143:0/2128412640 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd730009510 con 0x7fd74006b9d0 2023-11-26T07:23:53.679 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:53.675+0000 7fd73cff9640 1 -- 172.21.15.143:0/2128412640 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd73000eec0 con 0x7fd74006b9d0 2023-11-26T07:23:53.679 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:53.675+0000 7fd73cff9640 1 -- 172.21.15.143:0/2128412640 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd73000cb40 con 0x7fd74006b9d0 2023-11-26T07:23:53.679 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:53.675+0000 7fd74502f640 1 -- 172.21.15.143:0/2128412640 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd740153e00 con 0x7fd74006b9d0 2023-11-26T07:23:53.679 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:53.675+0000 7fd74502f640 1 -- 172.21.15.143:0/2128412640 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd740156c80 con 0x7fd74006b9d0 2023-11-26T07:23:53.679 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:53.679+0000 7fd74502f640 1 -- 172.21.15.143:0/2128412640 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd74006d4f0 con 0x7fd74006b9d0 2023-11-26T07:23:53.679 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:53.679+0000 7fd73cff9640 1 -- 172.21.15.143:0/2128412640 <== mon.0 v2:172.21.15.143:3300/0 4 ==== mgrmap(e 16) v1 ==== 85954+0+0 (secure 0 0 0) 0x7fd73000cca0 con 0x7fd74006b9d0 2023-11-26T07:23:53.680 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:53.679+0000 7fd73cff9640 1 --2- 172.21.15.143:0/2128412640 >> [v2:172.21.15.188:6836/3401128553,v1:172.21.15.188:6837/3401128553] conn(0x7fd72c066590 0x7fd72c068a50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:23:53.680 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:53.679+0000 7fd73cff9640 1 -- 172.21.15.143:0/2128412640 <== mon.0 v2:172.21.15.143:3300/0 5 ==== osd_map(91..91 src has 1..91) v4 ==== 5795+0+0 (secure 0 0 0) 0x7fd730084120 con 0x7fd74006b9d0 2023-11-26T07:23:53.680 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:53.679+0000 7fd73f7fe640 1 --2- 172.21.15.143:0/2128412640 >> [v2:172.21.15.188:6836/3401128553,v1:172.21.15.188:6837/3401128553] conn(0x7fd72c066590 0x7fd72c068a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:23:53.681 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:53.679+0000 7fd73f7fe640 1 --2- 172.21.15.143:0/2128412640 >> [v2:172.21.15.188:6836/3401128553,v1:172.21.15.188:6837/3401128553] conn(0x7fd72c066590 0x7fd72c068a50 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7fd7340af000 tx=0x7fd7340a7000 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:23:53.684 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:53.683+0000 7fd73cff9640 1 -- 172.21.15.143:0/2128412640 <== mon.0 v2:172.21.15.143:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172649 (secure 0 0 0) 0x7fd7300514a0 con 0x7fd74006b9d0 2023-11-26T07:23:53.869 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:53.867+0000 7fd74502f640 1 -- 172.21.15.143:0/2128412640 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fd740067500 con 0x7fd74006b9d0 2023-11-26T07:23:53.870 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:53.867+0000 7fd73cff9640 1 -- 172.21.15.143:0/2128412640 <== mon.0 v2:172.21.15.143:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 57 v57) v1 ==== 94+0+4924 (secure 0 0 0) 0x7fd730054ac0 con 0x7fd74006b9d0 2023-11-26T07:23:53.870 INFO:teuthology.orchestra.run.smithi143.stdout: 2023-11-26T07:23:53.870 INFO:teuthology.orchestra.run.smithi143.stdout:{"epoch":57,"default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6305,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.188:6834/3670325785","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6832","nonce":3670325785},{"type":"v1","addr":"172.21.15.188:6834","nonce":3670325785}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":46},{"gid":7467,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.143:6834/4060405000","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6832","nonce":4060405000},{"type":"v1","addr":"172.21.15.143:6834","nonce":4060405000}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":46},{"gid":7704,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.143:6835/2229435747","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6833","nonce":2229435747},{"type":"v1","addr":"172.21.15.143:6835","nonce":2229435747}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":57}],"filesystems":[{"mdsmap":{"epoch":51,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-11-26T07:23:29.008560+0000","modified":"2023-11-26T07:23:32.033055+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":7476},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7476":{"gid":7476,"name":"b","rank":0,"incarnation":47,"state":"up:active","state_seq":3,"addr":"172.21.15.188:6835/278875865","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6833","nonce":278875865},{"type":"v1","addr":"172.21.15.188:6835","nonce":278875865}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[13],"metadata_pool":12,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":6}]} 2023-11-26T07:23:53.870 INFO:teuthology.orchestra.run.smithi143.stderr:dumped fsmap epoch 57 2023-11-26T07:23:53.872 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:53.871+0000 7fd74502f640 1 -- 172.21.15.143:0/2128412640 >> [v2:172.21.15.188:6836/3401128553,v1:172.21.15.188:6837/3401128553] conn(0x7fd72c066590 msgr2=0x7fd72c068a50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:23:53.872 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:53.871+0000 7fd74502f640 1 --2- 172.21.15.143:0/2128412640 >> [v2:172.21.15.188:6836/3401128553,v1:172.21.15.188:6837/3401128553] conn(0x7fd72c066590 0x7fd72c068a50 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7fd7340af000 tx=0x7fd7340a7000 comp rx=0 tx=0).stop 2023-11-26T07:23:53.872 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:53.871+0000 7fd74502f640 1 -- 172.21.15.143:0/2128412640 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fd74006b9d0 msgr2=0x7fd740153790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:23:53.872 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:53.871+0000 7fd74502f640 1 --2- 172.21.15.143:0/2128412640 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fd74006b9d0 0x7fd740153790 secure :-1 s=READY pgs=1236 cs=0 l=1 rev1=1 crypto rx=0x7fd73000e870 tx=0x7fd73000ec70 comp rx=0 tx=0).stop 2023-11-26T07:23:53.872 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:53.871+0000 7fd74502f640 1 -- 172.21.15.143:0/2128412640 shutdown_connections 2023-11-26T07:23:53.872 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:53.871+0000 7fd74502f640 1 --2- 172.21.15.143:0/2128412640 >> [v2:172.21.15.188:6836/3401128553,v1:172.21.15.188:6837/3401128553] conn(0x7fd72c066590 0x7fd72c068a50 unknown :-1 s=CLOSED pgs=94 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:23:53.872 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:53.871+0000 7fd74502f640 1 --2- 172.21.15.143:0/2128412640 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fd7400cfaf0 0x7fd740158760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:23:53.872 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:53.871+0000 7fd74502f640 1 --2- 172.21.15.143:0/2128412640 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fd7400b23f0 0x7fd7401f6bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:23:53.872 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:53.871+0000 7fd74502f640 1 --2- 172.21.15.143:0/2128412640 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fd74006b9d0 0x7fd740153790 unknown :-1 s=CLOSED pgs=1236 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:23:53.873 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:53.871+0000 7fd74502f640 1 -- 172.21.15.143:0/2128412640 >> 172.21.15.143:0/2128412640 conn(0x7fd7400d0df0 msgr2=0x7fd7400adc70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:23:53.873 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:53.871+0000 7fd74502f640 1 -- 172.21.15.143:0/2128412640 shutdown_connections 2023-11-26T07:23:53.873 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:53.871+0000 7fd74502f640 1 -- 172.21.15.143:0/2128412640 wait complete. 2023-11-26T07:23:53.891 DEBUG:teuthology.orchestra.run.smithi143:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-11-26T07:23:54.029 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:54.027+0000 7f4e92f35640 1 Processor -- start 2023-11-26T07:23:54.029 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:54.027+0000 7f4e92f35640 1 -- start start 2023-11-26T07:23:54.030 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:54.031+0000 7f4e92f35640 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7f4e8c0cf710 0x7f4e8c0cd8c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:23:54.030 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:54.031+0000 7f4e92f35640 1 -- --> v1:172.21.15.143:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4e8c0cc730 con 0x7f4e8c0cf280 2023-11-26T07:23:54.030 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:54.031+0000 7f4e92f35640 1 -- --> v1:172.21.15.188:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4e8c070ee0 con 0x7f4e8c143ac0 2023-11-26T07:23:54.031 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:54.031+0000 7f4e92f35640 1 -- --> v2:172.21.15.143:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4e8c143ea0 con 0x7f4e8c0cf710 2023-11-26T07:23:54.031 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:54.031+0000 7f4e914d2640 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7f4e8c0cf710 0x7f4e8c0cd8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:23:54.031 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:54.031+0000 7f4e8bfff640 1 --1- >> v1:172.21.15.143:6789/0 conn(0x7f4e8c0cf280 0x7f4e8c0cd1b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.143:6789/0 says I am v1:172.21.15.143:46482/0 (socket says 172.21.15.143:46482) 2023-11-26T07:23:54.031 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:54.031+0000 7f4e8bfff640 1 -- 172.21.15.143:0/3412859067 learned_addr learned my addr 172.21.15.143:0/3412859067 (peer_addr_for_me v1:172.21.15.143:0/0) 2023-11-26T07:23:54.031 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:54.031+0000 7f4e914d2640 1 -- 172.21.15.143:0/3412859067 >> v1:172.21.15.188:6789/0 conn(0x7f4e8c143ac0 legacy=0x7f4e8c070370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:23:54.031 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:54.031+0000 7f4e914d2640 1 -- 172.21.15.143:0/3412859067 >> v1:172.21.15.143:6789/0 conn(0x7f4e8c0cf280 legacy=0x7f4e8c0cd1b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:23:54.031 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:54.031+0000 7f4e914d2640 1 -- 172.21.15.143:0/3412859067 --> v2:172.21.15.143:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4e8c0711c0 con 0x7f4e8c0cf710 2023-11-26T07:23:54.033 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:54.031+0000 7f4e914d2640 1 --2- 172.21.15.143:0/3412859067 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f4e8c0cf710 0x7f4e8c0cd8c0 secure :-1 s=READY pgs=1238 cs=0 l=1 rev1=1 crypto rx=0x7f4e84009d90 tx=0x7f4e8401c950 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d20f1114ee1c1707 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:23:54.034 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:54.035+0000 7f4e8b7fe640 1 -- 172.21.15.143:0/3412859067 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4e840a5a10 con 0x7f4e8c0cf710 2023-11-26T07:23:54.034 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:54.035+0000 7f4e8b7fe640 1 -- 172.21.15.143:0/3412859067 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4e840a5b70 con 0x7f4e8c0cf710 2023-11-26T07:23:54.034 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:54.035+0000 7f4e8b7fe640 1 -- 172.21.15.143:0/3412859067 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4e840ad440 con 0x7f4e8c0cf710 2023-11-26T07:23:54.035 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:54.035+0000 7f4e92f35640 1 -- 172.21.15.143:0/3412859067 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f4e8c0cf710 msgr2=0x7f4e8c0cd8c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:23:54.035 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:54.035+0000 7f4e92f35640 1 --2- 172.21.15.143:0/3412859067 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f4e8c0cf710 0x7f4e8c0cd8c0 secure :-1 s=READY pgs=1238 cs=0 l=1 rev1=1 crypto rx=0x7f4e84009d90 tx=0x7f4e8401c950 comp rx=0 tx=0).stop 2023-11-26T07:23:54.035 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:54.035+0000 7f4e92f35640 1 -- 172.21.15.143:0/3412859067 shutdown_connections 2023-11-26T07:23:54.035 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:54.035+0000 7f4e92f35640 1 --2- 172.21.15.143:0/3412859067 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f4e8c0cf710 0x7f4e8c0cd8c0 unknown :-1 s=CLOSED pgs=1238 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:23:54.035 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:54.035+0000 7f4e92f35640 1 -- 172.21.15.143:0/3412859067 >> 172.21.15.143:0/3412859067 conn(0x7f4e8c06a890 msgr2=0x7f4e8c06aca0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:23:54.035 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:54.035+0000 7f4e92f35640 1 -- 172.21.15.143:0/3412859067 shutdown_connections 2023-11-26T07:23:54.035 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:54.035+0000 7f4e92f35640 1 -- 172.21.15.143:0/3412859067 wait complete. 2023-11-26T07:23:54.036 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:54.035+0000 7f4e92f35640 1 Processor -- start 2023-11-26T07:23:54.036 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:54.035+0000 7f4e92f35640 1 -- start start 2023-11-26T07:23:54.039 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:54.035+0000 7f4e92f35640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f4e8c0cf280 0x7f4e8c13d160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:23:54.039 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:54.035+0000 7f4e92f35640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f4e8c0cf710 0x7f4e8c15a600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:23:54.039 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:54.035+0000 7f4e92f35640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f4e8c143ac0 0x7f4e8c1593f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:23:54.039 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:54.035+0000 7f4e92f35640 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4e8c13cd40 con 0x7f4e8c0cf710 2023-11-26T07:23:54.039 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:54.035+0000 7f4e92f35640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4e8c15b050 con 0x7f4e8c143ac0 2023-11-26T07:23:54.039 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:54.035+0000 7f4e92f35640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4e8c13c680 con 0x7f4e8c0cf280 2023-11-26T07:23:54.039 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:54.035+0000 7f4e8bfff640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f4e8c0cf710 0x7f4e8c15a600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:23:54.039 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:54.039+0000 7f4e90cd1640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f4e8c0cf280 0x7f4e8c13d160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:23:54.040 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:54.039+0000 7f4e90cd1640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f4e8c0cf280 0x7f4e8c13d160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.188:3301/0 says I am v2:172.21.15.143:39694/0 (socket says 172.21.15.143:39694) 2023-11-26T07:23:54.040 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:54.039+0000 7f4e90cd1640 1 -- 172.21.15.143:0/1320953589 learned_addr learned my addr 172.21.15.143:0/1320953589 (peer_addr_for_me v2:172.21.15.143:0/0) 2023-11-26T07:23:54.040 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:54.039+0000 7f4e8bfff640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f4e8c0cf710 0x7f4e8c15a600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:37060/0 (socket says 172.21.15.143:37060) 2023-11-26T07:23:54.040 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:54.039+0000 7f4e914d2640 1 --2- 172.21.15.143:0/1320953589 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f4e8c143ac0 0x7f4e8c1593f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:23:54.040 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:54.039+0000 7f4e90cd1640 1 -- 172.21.15.143:0/1320953589 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f4e8c143ac0 msgr2=0x7f4e8c1593f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:23:54.040 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:54.039+0000 7f4e90cd1640 1 --2- 172.21.15.143:0/1320953589 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f4e8c143ac0 0x7f4e8c1593f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:23:54.040 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:54.039+0000 7f4e90cd1640 1 -- 172.21.15.143:0/1320953589 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f4e8c0cf710 msgr2=0x7f4e8c15a600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:23:54.040 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:54.039+0000 7f4e90cd1640 1 --2- 172.21.15.143:0/1320953589 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f4e8c0cf710 0x7f4e8c15a600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:23:54.040 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:54.039+0000 7f4e90cd1640 1 -- 172.21.15.143:0/1320953589 --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4e84009a70 con 0x7f4e8c0cf280 2023-11-26T07:23:54.040 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:54.039+0000 7f4e8bfff640 1 --2- 172.21.15.143:0/1320953589 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f4e8c0cf710 0x7f4e8c15a600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-26T07:23:54.040 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:54.039+0000 7f4e90cd1640 1 --2- 172.21.15.143:0/1320953589 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f4e8c0cf280 0x7f4e8c13d160 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7f4e8000c500 tx=0x7f4e8000c9c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:23:54.040 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:54.039+0000 7f4e897fa640 1 -- 172.21.15.143:0/1320953589 <== mon.2 v2:172.21.15.188:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4e80005220 con 0x7f4e8c0cf280 2023-11-26T07:23:54.040 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:54.039+0000 7f4e897fa640 1 -- 172.21.15.143:0/1320953589 <== mon.2 v2:172.21.15.188:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4e80005380 con 0x7f4e8c0cf280 2023-11-26T07:23:54.041 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:54.039+0000 7f4e897fa640 1 -- 172.21.15.143:0/1320953589 <== mon.2 v2:172.21.15.188:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4e80007270 con 0x7f4e8c0cf280 2023-11-26T07:23:54.041 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:54.039+0000 7f4e92f35640 1 -- 172.21.15.143:0/1320953589 --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4e8c13c960 con 0x7f4e8c0cf280 2023-11-26T07:23:54.041 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:54.039+0000 7f4e92f35640 1 -- 172.21.15.143:0/1320953589 --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4e8c13dd60 con 0x7f4e8c0cf280 2023-11-26T07:23:54.041 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:54.039+0000 7f4e92f35640 1 -- 172.21.15.143:0/1320953589 --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4e8c13cfb0 con 0x7f4e8c0cf280 2023-11-26T07:23:54.041 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:54.039+0000 7f4e897fa640 1 -- 172.21.15.143:0/1320953589 <== mon.2 v2:172.21.15.188:3301/0 4 ==== mgrmap(e 16) v1 ==== 85954+0+0 (secure 0 0 0) 0x7f4e800073d0 con 0x7f4e8c0cf280 2023-11-26T07:23:54.042 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:54.039+0000 7f4e897fa640 1 --2- 172.21.15.143:0/1320953589 >> [v2:172.21.15.188:6836/3401128553,v1:172.21.15.188:6837/3401128553] conn(0x7f4e700665c0 0x7f4e70068a80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:23:54.042 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:54.043+0000 7f4e897fa640 1 -- 172.21.15.143:0/1320953589 <== mon.2 v2:172.21.15.188:3301/0 5 ==== osd_map(91..91 src has 1..91) v4 ==== 5795+0+0 (secure 0 0 0) 0x7f4e80083840 con 0x7f4e8c0cf280 2023-11-26T07:23:54.042 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:54.043+0000 7f4e8bfff640 1 --2- 172.21.15.143:0/1320953589 >> [v2:172.21.15.188:6836/3401128553,v1:172.21.15.188:6837/3401128553] conn(0x7f4e700665c0 0x7f4e70068a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:23:54.043 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:54.043+0000 7f4e8bfff640 1 --2- 172.21.15.143:0/1320953589 >> [v2:172.21.15.188:6836/3401128553,v1:172.21.15.188:6837/3401128553] conn(0x7f4e700665c0 0x7f4e70068a80 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7f4e8c13e6e0 tx=0x7f4e78009040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:23:54.047 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:54.047+0000 7f4e897fa640 1 -- 172.21.15.143:0/1320953589 <== mon.2 v2:172.21.15.188:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172649 (secure 0 0 0) 0x7f4e80050b70 con 0x7f4e8c0cf280 2023-11-26T07:23:54.198 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:54.199+0000 7f4e92f35640 1 -- 172.21.15.143:0/1320953589 --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f4e8c13c7c0 con 0x7f4e8c0cf280 2023-11-26T07:23:54.200 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:54.199+0000 7f4e897fa640 1 -- 172.21.15.143:0/1320953589 <== mon.2 v2:172.21.15.188:3301/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v91) v1 ==== 74+0+16727 (secure 0 0 0) 0x7f4e8c13c7c0 con 0x7f4e8c0cf280 2023-11-26T07:23:54.200 INFO:teuthology.orchestra.run.smithi143.stdout: 2023-11-26T07:23:54.201 INFO:teuthology.orchestra.run.smithi143.stdout:{"epoch":91,"fsid":"f5033060-2469-4a07-95fe-2b76f5404bc1","created":"2023-11-26T07:17:28.613663+0000","modified":"2023-11-26T07:23:47.807341+0000","last_up_change":"2023-11-26T07:17:41.157813+0000","last_in_change":"2023-11-26T07:17:31.181658+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":15,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-11-26T07:17:41.758597+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":12,"pool_name":"cephfs_metadata","create_time":"2023-11-26T07:23:26.724016+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"83","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":13,"pool_name":"cephfs_data","create_time":"2023-11-26T07:23:28.019663+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"84","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"ebbee2c6-023e-44b9-9f1d-6f7bfd309730","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6817","nonce":639807369},{"type":"v1","addr":"172.21.15.143:6819","nonce":639807369}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6821","nonce":639807369},{"type":"v1","addr":"172.21.15.143:6823","nonce":639807369}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6830","nonce":639807369},{"type":"v1","addr":"172.21.15.143:6831","nonce":639807369}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6825","nonce":639807369},{"type":"v1","addr":"172.21.15.143:6828","nonce":639807369}]},"public_addr":"172.21.15.143:6819/639807369","cluster_addr":"172.21.15.143:6823/639807369","heartbeat_back_addr":"172.21.15.143:6831/639807369","heartbeat_front_addr":"172.21.15.143:6828/639807369","state":["exists","up"]},{"osd":1,"uuid":"fec4b8be-7122-49e9-9a19-5850a0700001","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6816","nonce":1144942530},{"type":"v1","addr":"172.21.15.143:6818","nonce":1144942530}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6820","nonce":1144942530},{"type":"v1","addr":"172.21.15.143:6822","nonce":1144942530}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6827","nonce":1144942530},{"type":"v1","addr":"172.21.15.143:6829","nonce":1144942530}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6824","nonce":1144942530},{"type":"v1","addr":"172.21.15.143:6826","nonce":1144942530}]},"public_addr":"172.21.15.143:6818/1144942530","cluster_addr":"172.21.15.143:6822/1144942530","heartbeat_back_addr":"172.21.15.143:6829/1144942530","heartbeat_front_addr":"172.21.15.143:6826/1144942530","state":["exists","up"]},{"osd":2,"uuid":"2888af5b-f933-46ca-8f95-6dd494ff43f6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6808","nonce":3909095186},{"type":"v1","addr":"172.21.15.143:6809","nonce":3909095186}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6810","nonce":3909095186},{"type":"v1","addr":"172.21.15.143:6811","nonce":3909095186}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6814","nonce":3909095186},{"type":"v1","addr":"172.21.15.143:6815","nonce":3909095186}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6812","nonce":3909095186},{"type":"v1","addr":"172.21.15.143:6813","nonce":3909095186}]},"public_addr":"172.21.15.143:6809/3909095186","cluster_addr":"172.21.15.143:6811/3909095186","heartbeat_back_addr":"172.21.15.143:6815/3909095186","heartbeat_front_addr":"172.21.15.143:6813/3909095186","state":["exists","up"]},{"osd":3,"uuid":"d1f52435-7bc1-4081-9550-9cd1ba55f549","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6800","nonce":3854210991},{"type":"v1","addr":"172.21.15.143:6801","nonce":3854210991}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6802","nonce":3854210991},{"type":"v1","addr":"172.21.15.143:6803","nonce":3854210991}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6806","nonce":3854210991},{"type":"v1","addr":"172.21.15.143:6807","nonce":3854210991}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6804","nonce":3854210991},{"type":"v1","addr":"172.21.15.143:6805","nonce":3854210991}]},"public_addr":"172.21.15.143:6801/3854210991","cluster_addr":"172.21.15.143:6803/3854210991","heartbeat_back_addr":"172.21.15.143:6807/3854210991","heartbeat_front_addr":"172.21.15.143:6805/3854210991","state":["exists","up"]},{"osd":4,"uuid":"e2ae8b6e-6e25-49b2-bcbf-aacce68f06a3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6800","nonce":3924606902},{"type":"v1","addr":"172.21.15.188:6801","nonce":3924606902}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6802","nonce":3924606902},{"type":"v1","addr":"172.21.15.188:6803","nonce":3924606902}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6806","nonce":3924606902},{"type":"v1","addr":"172.21.15.188:6807","nonce":3924606902}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6804","nonce":3924606902},{"type":"v1","addr":"172.21.15.188:6805","nonce":3924606902}]},"public_addr":"172.21.15.188:6801/3924606902","cluster_addr":"172.21.15.188:6803/3924606902","heartbeat_back_addr":"172.21.15.188:6807/3924606902","heartbeat_front_addr":"172.21.15.188:6805/3924606902","state":["exists","up"]},{"osd":5,"uuid":"3cec8a32-7698-4c36-8e89-e4ba11934f33","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6813","nonce":186841544},{"type":"v1","addr":"172.21.15.188:6815","nonce":186841544}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6817","nonce":186841544},{"type":"v1","addr":"172.21.15.188:6818","nonce":186841544}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6822","nonce":186841544},{"type":"v1","addr":"172.21.15.188:6823","nonce":186841544}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6820","nonce":186841544},{"type":"v1","addr":"172.21.15.188:6821","nonce":186841544}]},"public_addr":"172.21.15.188:6815/186841544","cluster_addr":"172.21.15.188:6818/186841544","heartbeat_back_addr":"172.21.15.188:6823/186841544","heartbeat_front_addr":"172.21.15.188:6821/186841544","state":["exists","up"]},{"osd":6,"uuid":"58a65d5d-e090-473a-81b4-ad38e85029f9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6808","nonce":2153499405},{"type":"v1","addr":"172.21.15.188:6809","nonce":2153499405}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6810","nonce":2153499405},{"type":"v1","addr":"172.21.15.188:6811","nonce":2153499405}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6816","nonce":2153499405},{"type":"v1","addr":"172.21.15.188:6819","nonce":2153499405}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6812","nonce":2153499405},{"type":"v1","addr":"172.21.15.188:6814","nonce":2153499405}]},"public_addr":"172.21.15.188:6809/2153499405","cluster_addr":"172.21.15.188:6811/2153499405","heartbeat_back_addr":"172.21.15.188:6819/2153499405","heartbeat_front_addr":"172.21.15.188:6814/2153499405","state":["exists","up"]},{"osd":7,"uuid":"ea5b7f70-3554-4cf8-ac30-f7c59b80678d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6824","nonce":2722096729},{"type":"v1","addr":"172.21.15.188:6825","nonce":2722096729}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6826","nonce":2722096729},{"type":"v1","addr":"172.21.15.188:6827","nonce":2722096729}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6830","nonce":2722096729},{"type":"v1","addr":"172.21.15.188:6831","nonce":2722096729}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6828","nonce":2722096729},{"type":"v1","addr":"172.21.15.188:6829","nonce":2722096729}]},"public_addr":"172.21.15.188:6825/2722096729","cluster_addr":"172.21.15.188:6827/2722096729","heartbeat_back_addr":"172.21.15.188:6831/2722096729","heartbeat_front_addr":"172.21.15.188:6829/2722096729","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-26T07:17:36.995032+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-26T07:17:36.932266+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-26T07:17:36.829304+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-26T07:17:36.804297+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-26T07:17:36.667892+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-26T07:17:36.964868+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-26T07:17:36.827332+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-26T07:17:36.893179+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.143:6835/4031724325":"2023-11-27T07:23:44.794394+0000","172.21.15.143:6833/4031724325":"2023-11-27T07:23:44.794394+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-11-26T07:23:54.203 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:54.203+0000 7f4e92f35640 1 -- 172.21.15.143:0/1320953589 >> [v2:172.21.15.188:6836/3401128553,v1:172.21.15.188:6837/3401128553] conn(0x7f4e700665c0 msgr2=0x7f4e70068a80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:23:54.203 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:54.203+0000 7f4e92f35640 1 --2- 172.21.15.143:0/1320953589 >> [v2:172.21.15.188:6836/3401128553,v1:172.21.15.188:6837/3401128553] conn(0x7f4e700665c0 0x7f4e70068a80 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7f4e8c13e6e0 tx=0x7f4e78009040 comp rx=0 tx=0).stop 2023-11-26T07:23:54.203 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:54.203+0000 7f4e92f35640 1 -- 172.21.15.143:0/1320953589 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f4e8c0cf280 msgr2=0x7f4e8c13d160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:23:54.203 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:54.203+0000 7f4e92f35640 1 --2- 172.21.15.143:0/1320953589 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f4e8c0cf280 0x7f4e8c13d160 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7f4e8000c500 tx=0x7f4e8000c9c0 comp rx=0 tx=0).stop 2023-11-26T07:23:54.203 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:54.203+0000 7f4e92f35640 1 -- 172.21.15.143:0/1320953589 shutdown_connections 2023-11-26T07:23:54.203 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:54.203+0000 7f4e92f35640 1 --2- 172.21.15.143:0/1320953589 >> [v2:172.21.15.188:6836/3401128553,v1:172.21.15.188:6837/3401128553] conn(0x7f4e700665c0 0x7f4e70068a80 unknown :-1 s=CLOSED pgs=95 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:23:54.203 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:54.203+0000 7f4e92f35640 1 --2- 172.21.15.143:0/1320953589 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f4e8c143ac0 0x7f4e8c1593f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:23:54.203 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:54.203+0000 7f4e92f35640 1 --2- 172.21.15.143:0/1320953589 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f4e8c0cf710 0x7f4e8c15a600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:23:54.203 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:54.203+0000 7f4e92f35640 1 --2- 172.21.15.143:0/1320953589 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f4e8c0cf280 0x7f4e8c13d160 unknown :-1 s=CLOSED pgs=150 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:23:54.204 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:54.203+0000 7f4e92f35640 1 -- 172.21.15.143:0/1320953589 >> 172.21.15.143:0/1320953589 conn(0x7f4e8c06a890 msgr2=0x7f4e8c0b14f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:23:54.204 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:54.203+0000 7f4e92f35640 1 -- 172.21.15.143:0/1320953589 shutdown_connections 2023-11-26T07:23:54.204 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:54.203+0000 7f4e92f35640 1 -- 172.21.15.143:0/1320953589 wait complete. 2023-11-26T07:23:54.222 INFO:tasks.cephfs.filesystem:Destroying file system cephfs and related pools 2023-11-26T07:23:54.222 DEBUG:teuthology.orchestra.run.smithi143:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-11-26T07:23:54.360 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:54.359+0000 7fee06d85640 1 Processor -- start 2023-11-26T07:23:54.361 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:54.359+0000 7fee06d85640 1 -- start start 2023-11-26T07:23:54.362 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:54.359+0000 7fee06d85640 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7fee00144720 0x7fee0006d300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:23:54.362 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:54.359+0000 7fee06d85640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fee00071bf0 0x7fee0015bde0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:23:54.362 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:54.359+0000 7fee06d85640 1 -- --> v1:172.21.15.143:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fee0006e3f0 con 0x7fee0006d840 2023-11-26T07:23:54.362 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:54.359+0000 7fee06d85640 1 -- --> v2:172.21.15.143:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fee000722f0 con 0x7fee00144720 2023-11-26T07:23:54.362 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:54.359+0000 7fee06d85640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fee0006de30 con 0x7fee00071bf0 2023-11-26T07:23:54.362 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:54.363+0000 7fee05582640 1 --1- >> v1:172.21.15.143:6789/0 conn(0x7fee0006d840 0x7fee000714e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.143:6789/0 says I am v1:172.21.15.143:46490/0 (socket says 172.21.15.143:46490) 2023-11-26T07:23:54.363 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:54.363+0000 7fee05582640 1 -- 172.21.15.143:0/2113379657 learned_addr learned my addr 172.21.15.143:0/2113379657 (peer_addr_for_me v1:172.21.15.143:0/0) 2023-11-26T07:23:54.363 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:54.363+0000 7fee05d83640 1 --2- 172.21.15.143:0/2113379657 >> v2:172.21.15.143:3300/0 conn(0x7fee00144720 0x7fee0006d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:23:54.363 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:54.363+0000 7fee06584640 1 --2- 172.21.15.143:0/2113379657 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fee00071bf0 0x7fee0015bde0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:23:54.363 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:54.363+0000 7fee05d83640 1 -- 172.21.15.143:0/2113379657 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fee00071bf0 msgr2=0x7fee0015bde0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-26T07:23:54.363 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:54.363+0000 7fee05d83640 1 --2- 172.21.15.143:0/2113379657 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fee00071bf0 0x7fee0015bde0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:23:54.364 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:54.363+0000 7fee04d81640 1 -- 172.21.15.143:0/2113379657 <== mon.0 v1:172.21.15.143:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1599167211 0 0) 0x7fee0006e3f0 con 0x7fee0006d840 2023-11-26T07:23:54.364 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:54.363+0000 7fee05d83640 1 -- 172.21.15.143:0/2113379657 >> v1:172.21.15.143:6789/0 conn(0x7fee0006d840 legacy=0x7fee000714e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:23:54.364 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:54.363+0000 7fee05d83640 1 -- 172.21.15.143:0/2113379657 --> v2:172.21.15.143:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fee0006e110 con 0x7fee00144720 2023-11-26T07:23:54.365 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:54.363+0000 7fee05d83640 1 --2- 172.21.15.143:0/2113379657 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fee00144720 0x7fee0006d300 secure :-1 s=READY pgs=1240 cs=0 l=1 rev1=1 crypto rx=0x7fedfc0097c0 tx=0x7fedfc01c450 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f115d3f1dbc7e362 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:23:54.366 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:54.363+0000 7fee04d81640 1 -- 172.21.15.143:0/2113379657 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fedfc0ab650 con 0x7fee00144720 2023-11-26T07:23:54.366 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:54.363+0000 7fee04d81640 1 -- 172.21.15.143:0/2113379657 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fedfc0ab7b0 con 0x7fee00144720 2023-11-26T07:23:54.366 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:54.363+0000 7fee04d81640 1 -- 172.21.15.143:0/2113379657 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fedfc0abb40 con 0x7fee00144720 2023-11-26T07:23:54.366 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:54.367+0000 7fee06d85640 1 -- 172.21.15.143:0/2113379657 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fee00144720 msgr2=0x7fee0006d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:23:54.367 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:54.367+0000 7fee06d85640 1 --2- 172.21.15.143:0/2113379657 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fee00144720 0x7fee0006d300 secure :-1 s=READY pgs=1240 cs=0 l=1 rev1=1 crypto rx=0x7fedfc0097c0 tx=0x7fedfc01c450 comp rx=0 tx=0).stop 2023-11-26T07:23:54.367 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:54.367+0000 7fee06d85640 1 -- 172.21.15.143:0/2113379657 shutdown_connections 2023-11-26T07:23:54.367 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:54.367+0000 7fee06d85640 1 --2- 172.21.15.143:0/2113379657 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fee00071bf0 0x7fee0015bde0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:23:54.367 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:54.367+0000 7fee06d85640 1 --2- 172.21.15.143:0/2113379657 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fee00144720 0x7fee0006d300 unknown :-1 s=CLOSED pgs=1240 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:23:54.367 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:54.367+0000 7fee06d85640 1 -- 172.21.15.143:0/2113379657 >> 172.21.15.143:0/2113379657 conn(0x7fee0006a890 msgr2=0x7fee0006aca0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:23:54.367 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:54.367+0000 7fee06d85640 1 -- 172.21.15.143:0/2113379657 shutdown_connections 2023-11-26T07:23:54.367 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:54.367+0000 7fee06d85640 1 -- 172.21.15.143:0/2113379657 wait complete. 2023-11-26T07:23:54.368 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:54.367+0000 7fee06d85640 1 Processor -- start 2023-11-26T07:23:54.368 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:54.367+0000 7fee06d85640 1 -- start start 2023-11-26T07:23:54.370 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:54.367+0000 7fee06d85640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fee0006d840 0x7fee00137ff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:23:54.370 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:54.367+0000 7fee06d85640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fee00071bf0 0x7fee0015a960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:23:54.370 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:54.367+0000 7fee06d85640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fee00144720 0x7fee001397b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:23:54.370 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:54.367+0000 7fee06d85640 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fee0013b570 con 0x7fee00071bf0 2023-11-26T07:23:54.371 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:54.367+0000 7fee06d85640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fee0015aea0 con 0x7fee0006d840 2023-11-26T07:23:54.371 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:54.367+0000 7fee06d85640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fee0013f1e0 con 0x7fee00144720 2023-11-26T07:23:54.371 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:54.367+0000 7fee05582640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fee00071bf0 0x7fee0015a960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:23:54.371 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:54.367+0000 7fee05582640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fee00071bf0 0x7fee0015a960 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:37080/0 (socket says 172.21.15.143:37080) 2023-11-26T07:23:54.371 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:54.367+0000 7fee05582640 1 -- 172.21.15.143:0/293567812 learned_addr learned my addr 172.21.15.143:0/293567812 (peer_addr_for_me v2:172.21.15.143:0/0) 2023-11-26T07:23:54.371 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:54.367+0000 7fee05d83640 1 --2- 172.21.15.143:0/293567812 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fee0006d840 0x7fee00137ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:23:54.371 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:54.367+0000 7fee06584640 1 --2- 172.21.15.143:0/293567812 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fee00144720 0x7fee001397b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:23:54.371 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:54.371+0000 7fee05582640 1 -- 172.21.15.143:0/293567812 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fee00144720 msgr2=0x7fee001397b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:23:54.371 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:54.371+0000 7fee05582640 1 --2- 172.21.15.143:0/293567812 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fee00144720 0x7fee001397b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:23:54.371 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:54.371+0000 7fee05582640 1 -- 172.21.15.143:0/293567812 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fee0006d840 msgr2=0x7fee00137ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:23:54.371 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:54.371+0000 7fee05582640 1 --2- 172.21.15.143:0/293567812 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fee0006d840 0x7fee00137ff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:23:54.371 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:54.371+0000 7fee05582640 1 -- 172.21.15.143:0/293567812 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fedfc009420 con 0x7fee00071bf0 2023-11-26T07:23:54.372 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:54.371+0000 7fee05582640 1 --2- 172.21.15.143:0/293567812 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fee00071bf0 0x7fee0015a960 secure :-1 s=READY pgs=1241 cs=0 l=1 rev1=1 crypto rx=0x7fedf00011f0 tx=0x7fedf000c9b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:23:54.372 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:54.371+0000 7fedf67fc640 1 -- 172.21.15.143:0/293567812 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fedf00074e0 con 0x7fee00071bf0 2023-11-26T07:23:54.372 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:54.371+0000 7fedf67fc640 1 -- 172.21.15.143:0/293567812 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fedf0007640 con 0x7fee00071bf0 2023-11-26T07:23:54.372 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:54.371+0000 7fee06d85640 1 -- 172.21.15.143:0/293567812 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fee001413f0 con 0x7fee00071bf0 2023-11-26T07:23:54.372 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:54.371+0000 7fedf67fc640 1 -- 172.21.15.143:0/293567812 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fedf0004a10 con 0x7fee00071bf0 2023-11-26T07:23:54.373 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:54.371+0000 7fee06d85640 1 -- 172.21.15.143:0/293567812 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fee0013f760 con 0x7fee00071bf0 2023-11-26T07:23:54.373 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:54.371+0000 7fee06d85640 1 -- 172.21.15.143:0/293567812 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fee0013b6b0 con 0x7fee00071bf0 2023-11-26T07:23:54.373 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:54.371+0000 7fedf67fc640 1 -- 172.21.15.143:0/293567812 <== mon.0 v2:172.21.15.143:3300/0 4 ==== mgrmap(e 16) v1 ==== 85954+0+0 (secure 0 0 0) 0x7fedf0004b70 con 0x7fee00071bf0 2023-11-26T07:23:54.374 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:54.371+0000 7fedf67fc640 1 --2- 172.21.15.143:0/293567812 >> [v2:172.21.15.188:6836/3401128553,v1:172.21.15.188:6837/3401128553] conn(0x7fede4066590 0x7fede4068a50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:23:54.374 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:54.371+0000 7fedf67fc640 1 -- 172.21.15.143:0/293567812 <== mon.0 v2:172.21.15.143:3300/0 5 ==== osd_map(91..91 src has 1..91) v4 ==== 5795+0+0 (secure 0 0 0) 0x7fedf00830b0 con 0x7fee00071bf0 2023-11-26T07:23:54.374 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:54.375+0000 7fee05d83640 1 --2- 172.21.15.143:0/293567812 >> [v2:172.21.15.188:6836/3401128553,v1:172.21.15.188:6837/3401128553] conn(0x7fede4066590 0x7fede4068a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:23:54.375 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:54.375+0000 7fee05d83640 1 --2- 172.21.15.143:0/293567812 >> [v2:172.21.15.188:6836/3401128553,v1:172.21.15.188:6837/3401128553] conn(0x7fede4066590 0x7fede4068a50 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7fedfc0af000 tx=0x7fedfc0a7000 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:23:54.378 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:54.375+0000 7fedf67fc640 1 -- 172.21.15.143:0/293567812 <== mon.0 v2:172.21.15.143:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172649 (secure 0 0 0) 0x7fee0013b6b0 con 0x7fee00071bf0 2023-11-26T07:23:54.556 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:54.555+0000 7fee06d85640 1 -- 172.21.15.143:0/293567812 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fee0013b6b0 con 0x7fee00071bf0 2023-11-26T07:23:54.556 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:54.555+0000 7fedf67fc640 1 -- 172.21.15.143:0/293567812 <== mon.0 v2:172.21.15.143:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 57 v57) v1 ==== 94+0+4924 (secure 0 0 0) 0x7fee0013b6b0 con 0x7fee00071bf0 2023-11-26T07:23:54.556 INFO:teuthology.orchestra.run.smithi143.stdout: 2023-11-26T07:23:54.557 INFO:teuthology.orchestra.run.smithi143.stdout:{"epoch":57,"default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6305,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.188:6834/3670325785","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6832","nonce":3670325785},{"type":"v1","addr":"172.21.15.188:6834","nonce":3670325785}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":46},{"gid":7467,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.143:6834/4060405000","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6832","nonce":4060405000},{"type":"v1","addr":"172.21.15.143:6834","nonce":4060405000}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":46},{"gid":7704,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.143:6835/2229435747","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6833","nonce":2229435747},{"type":"v1","addr":"172.21.15.143:6835","nonce":2229435747}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":57}],"filesystems":[{"mdsmap":{"epoch":51,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-11-26T07:23:29.008560+0000","modified":"2023-11-26T07:23:32.033055+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":7476},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7476":{"gid":7476,"name":"b","rank":0,"incarnation":47,"state":"up:active","state_seq":3,"addr":"172.21.15.188:6835/278875865","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6833","nonce":278875865},{"type":"v1","addr":"172.21.15.188:6835","nonce":278875865}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[13],"metadata_pool":12,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":6}]} 2023-11-26T07:23:54.557 INFO:teuthology.orchestra.run.smithi143.stderr:dumped fsmap epoch 57 2023-11-26T07:23:54.558 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:54.559+0000 7fee06d85640 1 -- 172.21.15.143:0/293567812 >> [v2:172.21.15.188:6836/3401128553,v1:172.21.15.188:6837/3401128553] conn(0x7fede4066590 msgr2=0x7fede4068a50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:23:54.558 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:54.559+0000 7fee06d85640 1 --2- 172.21.15.143:0/293567812 >> [v2:172.21.15.188:6836/3401128553,v1:172.21.15.188:6837/3401128553] conn(0x7fede4066590 0x7fede4068a50 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7fedfc0af000 tx=0x7fedfc0a7000 comp rx=0 tx=0).stop 2023-11-26T07:23:54.558 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:54.559+0000 7fee06d85640 1 -- 172.21.15.143:0/293567812 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fee00071bf0 msgr2=0x7fee0015a960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:23:54.559 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:54.559+0000 7fee06d85640 1 --2- 172.21.15.143:0/293567812 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fee00071bf0 0x7fee0015a960 secure :-1 s=READY pgs=1241 cs=0 l=1 rev1=1 crypto rx=0x7fedf00011f0 tx=0x7fedf000c9b0 comp rx=0 tx=0).stop 2023-11-26T07:23:54.559 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:54.559+0000 7fee06d85640 1 -- 172.21.15.143:0/293567812 shutdown_connections 2023-11-26T07:23:54.559 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:54.559+0000 7fee06d85640 1 --2- 172.21.15.143:0/293567812 >> [v2:172.21.15.188:6836/3401128553,v1:172.21.15.188:6837/3401128553] conn(0x7fede4066590 0x7fede4068a50 unknown :-1 s=CLOSED pgs=96 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:23:54.559 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:54.559+0000 7fee06d85640 1 --2- 172.21.15.143:0/293567812 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fee00144720 0x7fee001397b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:23:54.559 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:54.559+0000 7fee06d85640 1 --2- 172.21.15.143:0/293567812 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fee00071bf0 0x7fee0015a960 unknown :-1 s=CLOSED pgs=1241 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:23:54.559 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:54.559+0000 7fee06d85640 1 --2- 172.21.15.143:0/293567812 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fee0006d840 0x7fee00137ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:23:54.559 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:54.559+0000 7fee06d85640 1 -- 172.21.15.143:0/293567812 >> 172.21.15.143:0/293567812 conn(0x7fee0006a890 msgr2=0x7fee000714e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:23:54.559 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:54.559+0000 7fee06d85640 1 -- 172.21.15.143:0/293567812 shutdown_connections 2023-11-26T07:23:54.559 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:54.559+0000 7fee06d85640 1 -- 172.21.15.143:0/293567812 wait complete. 2023-11-26T07:23:54.578 DEBUG:teuthology.orchestra.run.smithi143:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-11-26T07:23:54.715 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:54.715+0000 7f6f7bd8d640 1 Processor -- start 2023-11-26T07:23:54.715 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:54.715+0000 7f6f7bd8d640 1 -- start start 2023-11-26T07:23:54.716 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:54.715+0000 7f6f7bd8d640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f6f7413e290 0x7f6f740cf280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:23:54.716 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:54.715+0000 7f6f7bd8d640 1 -- --> v1:172.21.15.143:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6f7406c520 con 0x7f6f740cf7c0 2023-11-26T07:23:54.717 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:54.715+0000 7f6f7bd8d640 1 -- --> v1:172.21.15.188:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6f7406c800 con 0x7f6f7415f6b0 2023-11-26T07:23:54.717 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:54.715+0000 7f6f7bd8d640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6f740cfc40 con 0x7f6f7413e290 2023-11-26T07:23:54.717 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:54.715+0000 7f6f79328640 1 --1- >> v1:172.21.15.143:6789/0 conn(0x7f6f740cf7c0 0x7f6f7415efa0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.143:6789/0 says I am v1:172.21.15.143:43280/0 (socket says 172.21.15.143:43280) 2023-11-26T07:23:54.717 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:54.715+0000 7f6f79328640 1 -- 172.21.15.143:0/2834150454 learned_addr learned my addr 172.21.15.143:0/2834150454 (peer_addr_for_me v1:172.21.15.143:0/0) 2023-11-26T07:23:54.718 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:54.715+0000 7f6f79b29640 1 --2- 172.21.15.143:0/2834150454 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f6f7413e290 0x7f6f740cf280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:23:54.718 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:54.715+0000 7f6f78b27640 1 -- 172.21.15.143:0/2834150454 <== mon.0 v1:172.21.15.143:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 251414762 0 0) 0x7f6f7406c520 con 0x7f6f740cf7c0 2023-11-26T07:23:54.718 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:54.715+0000 7f6f78b27640 1 -- 172.21.15.143:0/2834150454 --> v1:172.21.15.143:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f6f60003530 con 0x7f6f740cf7c0 2023-11-26T07:23:54.718 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:54.715+0000 7f6f78b27640 1 -- 172.21.15.143:0/2834150454 <== mon.1 v1:172.21.15.188:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3297983344 0 0) 0x7f6f7406c800 con 0x7f6f7415f6b0 2023-11-26T07:23:54.718 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:54.715+0000 7f6f79b29640 1 -- 172.21.15.143:0/2834150454 >> v1:172.21.15.188:6789/0 conn(0x7f6f7415f6b0 legacy=0x7f6f740cd7b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:23:54.718 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:54.715+0000 7f6f79b29640 1 -- 172.21.15.143:0/2834150454 >> v1:172.21.15.143:6789/0 conn(0x7f6f740cf7c0 legacy=0x7f6f7415efa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:23:54.718 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:54.715+0000 7f6f79b29640 1 -- 172.21.15.143:0/2834150454 --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6f7406cae0 con 0x7f6f7413e290 2023-11-26T07:23:54.720 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:54.719+0000 7f6f79b29640 1 --2- 172.21.15.143:0/2834150454 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f6f7413e290 0x7f6f740cf280 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7f6f700097f0 tx=0x7f6f7001c4c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=4362eb2798ee7ba3 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:23:54.721 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:54.719+0000 7f6f78b27640 1 -- 172.21.15.143:0/2834150454 <== mon.2 v2:172.21.15.188:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6f700ac410 con 0x7f6f7413e290 2023-11-26T07:23:54.721 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:54.719+0000 7f6f78b27640 1 -- 172.21.15.143:0/2834150454 <== mon.2 v2:172.21.15.188:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6f700ac570 con 0x7f6f7413e290 2023-11-26T07:23:54.721 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:54.719+0000 7f6f78b27640 1 -- 172.21.15.143:0/2834150454 <== mon.2 v2:172.21.15.188:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6f700ac900 con 0x7f6f7413e290 2023-11-26T07:23:54.722 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:54.719+0000 7f6f7bd8d640 1 -- 172.21.15.143:0/2834150454 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f6f7413e290 msgr2=0x7f6f740cf280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:23:54.722 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:54.719+0000 7f6f7bd8d640 1 --2- 172.21.15.143:0/2834150454 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f6f7413e290 0x7f6f740cf280 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7f6f700097f0 tx=0x7f6f7001c4c0 comp rx=0 tx=0).stop 2023-11-26T07:23:54.722 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:54.719+0000 7f6f7bd8d640 1 -- 172.21.15.143:0/2834150454 shutdown_connections 2023-11-26T07:23:54.722 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:54.719+0000 7f6f7bd8d640 1 --2- 172.21.15.143:0/2834150454 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f6f7413e290 0x7f6f740cf280 unknown :-1 s=CLOSED pgs=151 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:23:54.722 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:54.719+0000 7f6f7bd8d640 1 -- 172.21.15.143:0/2834150454 >> 172.21.15.143:0/2834150454 conn(0x7f6f7406a890 msgr2=0x7f6f7406aca0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:23:54.722 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:54.719+0000 7f6f7bd8d640 1 -- 172.21.15.143:0/2834150454 shutdown_connections 2023-11-26T07:23:54.722 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:54.719+0000 7f6f7bd8d640 1 -- 172.21.15.143:0/2834150454 wait complete. 2023-11-26T07:23:54.723 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:54.723+0000 7f6f7bd8d640 1 Processor -- start 2023-11-26T07:23:54.723 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:54.723+0000 7f6f7bd8d640 1 -- start start 2023-11-26T07:23:54.725 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:54.723+0000 7f6f7bd8d640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f6f740cf7c0 0x7f6f7415e8c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:23:54.725 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:54.723+0000 7f6f7bd8d640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f6f7413e290 0x7f6f74157800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:23:54.725 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:54.723+0000 7f6f7bd8d640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f6f7415f6b0 0x7f6f7415d920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:23:54.725 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:54.723+0000 7f6f7bd8d640 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6f7415e170 con 0x7f6f740cf7c0 2023-11-26T07:23:54.726 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:54.723+0000 7f6f7bd8d640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6f7415c9b0 con 0x7f6f7413e290 2023-11-26T07:23:54.726 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:54.723+0000 7f6f7bd8d640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6f7415c0c0 con 0x7f6f7415f6b0 2023-11-26T07:23:54.726 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:54.723+0000 7f6f79b29640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f6f740cf7c0 0x7f6f7415e8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:23:54.726 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:54.723+0000 7f6f79b29640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f6f740cf7c0 0x7f6f7415e8c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:38386/0 (socket says 172.21.15.143:38386) 2023-11-26T07:23:54.726 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:54.723+0000 7f6f79b29640 1 -- 172.21.15.143:0/988600137 learned_addr learned my addr 172.21.15.143:0/988600137 (peer_addr_for_me v2:172.21.15.143:0/0) 2023-11-26T07:23:54.726 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:54.723+0000 7f6f79b29640 1 -- 172.21.15.143:0/988600137 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f6f7415f6b0 msgr2=0x7f6f7415d920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:23:54.726 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:54.723+0000 7f6f79b29640 1 --2- 172.21.15.143:0/988600137 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f6f7415f6b0 0x7f6f7415d920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:23:54.726 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:54.723+0000 7f6f79b29640 1 -- 172.21.15.143:0/988600137 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f6f7413e290 msgr2=0x7f6f74157800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:23:54.726 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:54.723+0000 7f6f79b29640 1 --2- 172.21.15.143:0/988600137 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f6f7413e290 0x7f6f74157800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:23:54.726 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:54.723+0000 7f6f79b29640 1 -- 172.21.15.143:0/988600137 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6f70009420 con 0x7f6f740cf7c0 2023-11-26T07:23:54.726 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:54.723+0000 7f6f79b29640 1 --2- 172.21.15.143:0/988600137 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f6f740cf7c0 0x7f6f7415e8c0 secure :-1 s=READY pgs=1243 cs=0 l=1 rev1=1 crypto rx=0x7f6f700af040 tx=0x7f6f70019260 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:23:54.727 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:54.723+0000 7f6f6a7fc640 1 -- 172.21.15.143:0/988600137 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6f700ace90 con 0x7f6f740cf7c0 2023-11-26T07:23:54.727 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:54.723+0000 7f6f6a7fc640 1 -- 172.21.15.143:0/988600137 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6f70006e20 con 0x7f6f740cf7c0 2023-11-26T07:23:54.727 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:54.723+0000 7f6f6a7fc640 1 -- 172.21.15.143:0/988600137 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6f700b5860 con 0x7f6f740cf7c0 2023-11-26T07:23:54.727 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:54.723+0000 7f6f7bd8d640 1 -- 172.21.15.143:0/988600137 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6f7415c360 con 0x7f6f740cf7c0 2023-11-26T07:23:54.727 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:54.723+0000 7f6f7bd8d640 1 -- 172.21.15.143:0/988600137 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6f74157d40 con 0x7f6f740cf7c0 2023-11-26T07:23:54.728 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:54.727+0000 7f6f7bd8d640 1 -- 172.21.15.143:0/988600137 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6f740b1b20 con 0x7f6f740cf7c0 2023-11-26T07:23:54.728 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:54.727+0000 7f6f6a7fc640 1 -- 172.21.15.143:0/988600137 <== mon.0 v2:172.21.15.143:3300/0 4 ==== mgrmap(e 16) v1 ==== 85954+0+0 (secure 0 0 0) 0x7f6f700b3040 con 0x7f6f740cf7c0 2023-11-26T07:23:54.728 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:54.727+0000 7f6f6a7fc640 1 --2- 172.21.15.143:0/988600137 >> [v2:172.21.15.188:6836/3401128553,v1:172.21.15.188:6837/3401128553] conn(0x7f6f58066590 0x7f6f58068a50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:23:54.729 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:54.727+0000 7f6f6a7fc640 1 -- 172.21.15.143:0/988600137 <== mon.0 v2:172.21.15.143:3300/0 5 ==== osd_map(91..91 src has 1..91) v4 ==== 5795+0+0 (secure 0 0 0) 0x7f6f7011e180 con 0x7f6f740cf7c0 2023-11-26T07:23:54.729 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:54.727+0000 7f6f79328640 1 --2- 172.21.15.143:0/988600137 >> [v2:172.21.15.188:6836/3401128553,v1:172.21.15.188:6837/3401128553] conn(0x7f6f58066590 0x7f6f58068a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:23:54.729 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:54.727+0000 7f6f79328640 1 --2- 172.21.15.143:0/988600137 >> [v2:172.21.15.188:6836/3401128553,v1:172.21.15.188:6837/3401128553] conn(0x7f6f58066590 0x7f6f58068a50 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7f6f64007fd0 tx=0x7f6f64007900 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:23:54.732 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:54.731+0000 7f6f6a7fc640 1 -- 172.21.15.143:0/988600137 <== mon.0 v2:172.21.15.143:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172649 (secure 0 0 0) 0x7f6f700eb2e0 con 0x7f6f740cf7c0 2023-11-26T07:23:54.913 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:54.911+0000 7f6f7bd8d640 1 -- 172.21.15.143:0/988600137 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f6f7406d6d0 con 0x7f6f740cf7c0 2023-11-26T07:23:54.914 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:54.915+0000 7f6f6a7fc640 1 -- 172.21.15.143:0/988600137 <== mon.0 v2:172.21.15.143:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 57 v57) v1 ==== 94+0+4924 (secure 0 0 0) 0x7f6f7406d6d0 con 0x7f6f740cf7c0 2023-11-26T07:23:54.914 INFO:teuthology.orchestra.run.smithi143.stdout: 2023-11-26T07:23:54.915 INFO:teuthology.orchestra.run.smithi143.stdout:{"epoch":57,"default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6305,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.188:6834/3670325785","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6832","nonce":3670325785},{"type":"v1","addr":"172.21.15.188:6834","nonce":3670325785}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":46},{"gid":7467,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.143:6834/4060405000","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6832","nonce":4060405000},{"type":"v1","addr":"172.21.15.143:6834","nonce":4060405000}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":46},{"gid":7704,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.143:6835/2229435747","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6833","nonce":2229435747},{"type":"v1","addr":"172.21.15.143:6835","nonce":2229435747}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":57}],"filesystems":[{"mdsmap":{"epoch":51,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-11-26T07:23:29.008560+0000","modified":"2023-11-26T07:23:32.033055+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":7476},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7476":{"gid":7476,"name":"b","rank":0,"incarnation":47,"state":"up:active","state_seq":3,"addr":"172.21.15.188:6835/278875865","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6833","nonce":278875865},{"type":"v1","addr":"172.21.15.188:6835","nonce":278875865}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[13],"metadata_pool":12,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":6}]} 2023-11-26T07:23:54.915 INFO:teuthology.orchestra.run.smithi143.stderr:dumped fsmap epoch 57 2023-11-26T07:23:54.916 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:54.915+0000 7f6f7bd8d640 1 -- 172.21.15.143:0/988600137 >> [v2:172.21.15.188:6836/3401128553,v1:172.21.15.188:6837/3401128553] conn(0x7f6f58066590 msgr2=0x7f6f58068a50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:23:54.916 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:54.915+0000 7f6f7bd8d640 1 --2- 172.21.15.143:0/988600137 >> [v2:172.21.15.188:6836/3401128553,v1:172.21.15.188:6837/3401128553] conn(0x7f6f58066590 0x7f6f58068a50 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7f6f64007fd0 tx=0x7f6f64007900 comp rx=0 tx=0).stop 2023-11-26T07:23:54.917 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:54.915+0000 7f6f7bd8d640 1 -- 172.21.15.143:0/988600137 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f6f740cf7c0 msgr2=0x7f6f7415e8c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:23:54.917 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:54.915+0000 7f6f7bd8d640 1 --2- 172.21.15.143:0/988600137 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f6f740cf7c0 0x7f6f7415e8c0 secure :-1 s=READY pgs=1243 cs=0 l=1 rev1=1 crypto rx=0x7f6f700af040 tx=0x7f6f70019260 comp rx=0 tx=0).stop 2023-11-26T07:23:54.917 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:54.915+0000 7f6f7bd8d640 1 -- 172.21.15.143:0/988600137 shutdown_connections 2023-11-26T07:23:54.917 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:54.915+0000 7f6f7bd8d640 1 --2- 172.21.15.143:0/988600137 >> [v2:172.21.15.188:6836/3401128553,v1:172.21.15.188:6837/3401128553] conn(0x7f6f58066590 0x7f6f58068a50 unknown :-1 s=CLOSED pgs=97 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:23:54.917 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:54.915+0000 7f6f7bd8d640 1 --2- 172.21.15.143:0/988600137 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f6f7415f6b0 0x7f6f7415d920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:23:54.917 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:54.915+0000 7f6f7bd8d640 1 --2- 172.21.15.143:0/988600137 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f6f7413e290 0x7f6f74157800 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:23:54.917 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:54.915+0000 7f6f7bd8d640 1 --2- 172.21.15.143:0/988600137 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f6f740cf7c0 0x7f6f7415e8c0 unknown :-1 s=CLOSED pgs=1243 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:23:54.917 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:54.915+0000 7f6f7bd8d640 1 -- 172.21.15.143:0/988600137 >> 172.21.15.143:0/988600137 conn(0x7f6f7406a890 msgr2=0x7f6f74071d40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:23:54.917 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:54.915+0000 7f6f7bd8d640 1 -- 172.21.15.143:0/988600137 shutdown_connections 2023-11-26T07:23:54.917 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:54.915+0000 7f6f7bd8d640 1 -- 172.21.15.143:0/988600137 wait complete. 2023-11-26T07:23:54.936 DEBUG:teuthology.orchestra.run.smithi143:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-11-26T07:23:55.073 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:55.071+0000 7f034ef65640 1 Processor -- start 2023-11-26T07:23:55.073 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:55.071+0000 7f034ef65640 1 -- start start 2023-11-26T07:23:55.075 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:55.075+0000 7f034ef65640 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7f03480cf280 0x7f03480cf710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:23:55.075 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:55.075+0000 7f034ef65640 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7f03480cd1b0 0x7f03480cd5c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:23:55.075 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:55.075+0000 7f034ef65640 1 -- --> v1:172.21.15.143:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f03480cc730 con 0x7f0348143ac0 2023-11-26T07:23:55.075 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:55.075+0000 7f034ef65640 1 -- --> v2:172.21.15.143:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0348143ea0 con 0x7f03480cf280 2023-11-26T07:23:55.076 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:55.075+0000 7f034ef65640 1 -- --> v2:172.21.15.188:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f03480cfc50 con 0x7f03480cd1b0 2023-11-26T07:23:55.076 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:55.075+0000 7f034cd01640 1 --1- >> v1:172.21.15.143:6789/0 conn(0x7f0348143ac0 0x7f0348070370 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.143:6789/0 says I am v1:172.21.15.143:43284/0 (socket says 172.21.15.143:43284) 2023-11-26T07:23:55.076 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:55.075+0000 7f034cd01640 1 -- 172.21.15.143:0/1653772303 learned_addr learned my addr 172.21.15.143:0/1653772303 (peer_addr_for_me v1:172.21.15.143:0/0) 2023-11-26T07:23:55.076 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:55.075+0000 7f033ffff640 1 --2- 172.21.15.143:0/1653772303 >> v2:172.21.15.143:3300/0 conn(0x7f03480cf280 0x7f03480cf710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:23:55.076 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:55.075+0000 7f034d502640 1 --2- 172.21.15.143:0/1653772303 >> v2:172.21.15.188:3300/0 conn(0x7f03480cd1b0 0x7f03480cd5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:23:55.076 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:55.075+0000 7f033f7fe640 1 -- 172.21.15.143:0/1653772303 <== mon.0 v1:172.21.15.143:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3008472372 0 0) 0x7f03480cc730 con 0x7f0348143ac0 2023-11-26T07:23:55.076 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:55.075+0000 7f033f7fe640 1 -- 172.21.15.143:0/1653772303 --> v1:172.21.15.143:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f0330003530 con 0x7f0348143ac0 2023-11-26T07:23:55.076 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:55.075+0000 7f033ffff640 1 -- 172.21.15.143:0/1653772303 >> v2:172.21.15.188:3300/0 conn(0x7f03480cd1b0 msgr2=0x7f03480cd5c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-26T07:23:55.076 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:55.075+0000 7f033ffff640 1 --2- 172.21.15.143:0/1653772303 >> v2:172.21.15.188:3300/0 conn(0x7f03480cd1b0 0x7f03480cd5c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:23:55.076 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:55.075+0000 7f033ffff640 1 -- 172.21.15.143:0/1653772303 >> v1:172.21.15.143:6789/0 conn(0x7f0348143ac0 legacy=0x7f0348070370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:23:55.076 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:55.075+0000 7f033ffff640 1 -- 172.21.15.143:0/1653772303 --> v2:172.21.15.143:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0348070ee0 con 0x7f03480cf280 2023-11-26T07:23:55.076 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:55.075+0000 7f034d502640 1 --2- 172.21.15.143:0/1653772303 >> v2:172.21.15.188:3300/0 conn(0x7f03480cd1b0 0x7f03480cd5c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-26T07:23:55.078 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:55.079+0000 7f033ffff640 1 --2- 172.21.15.143:0/1653772303 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f03480cf280 0x7f03480cf710 secure :-1 s=READY pgs=1245 cs=0 l=1 rev1=1 crypto rx=0x7f034806e570 tx=0x7f033801c460 comp rx=0 tx=0).ready entity=mon.0 client_cookie=19a81664ee0c842a server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:23:55.079 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:55.079+0000 7f033f7fe640 1 -- 172.21.15.143:0/1653772303 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f03380ab660 con 0x7f03480cf280 2023-11-26T07:23:55.079 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:55.079+0000 7f033f7fe640 1 -- 172.21.15.143:0/1653772303 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f03380ab7c0 con 0x7f03480cf280 2023-11-26T07:23:55.079 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:55.079+0000 7f033f7fe640 1 -- 172.21.15.143:0/1653772303 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f03380abb50 con 0x7f03480cf280 2023-11-26T07:23:55.079 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:55.079+0000 7f034ef65640 1 -- 172.21.15.143:0/1653772303 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f03480cf280 msgr2=0x7f03480cf710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:23:55.079 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:55.079+0000 7f034ef65640 1 --2- 172.21.15.143:0/1653772303 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f03480cf280 0x7f03480cf710 secure :-1 s=READY pgs=1245 cs=0 l=1 rev1=1 crypto rx=0x7f034806e570 tx=0x7f033801c460 comp rx=0 tx=0).stop 2023-11-26T07:23:55.079 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:55.079+0000 7f034ef65640 1 -- 172.21.15.143:0/1653772303 shutdown_connections 2023-11-26T07:23:55.080 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:55.079+0000 7f034ef65640 1 --2- 172.21.15.143:0/1653772303 >> v2:172.21.15.188:3300/0 conn(0x7f03480cd1b0 0x7f03480cd5c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:23:55.080 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:55.079+0000 7f034ef65640 1 --2- 172.21.15.143:0/1653772303 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f03480cf280 0x7f03480cf710 unknown :-1 s=CLOSED pgs=1245 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:23:55.080 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:55.079+0000 7f034ef65640 1 -- 172.21.15.143:0/1653772303 >> 172.21.15.143:0/1653772303 conn(0x7f034806a890 msgr2=0x7f034806aca0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:23:55.080 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:55.079+0000 7f034ef65640 1 -- 172.21.15.143:0/1653772303 shutdown_connections 2023-11-26T07:23:55.080 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:55.079+0000 7f034ef65640 1 -- 172.21.15.143:0/1653772303 wait complete. 2023-11-26T07:23:55.080 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:55.079+0000 7f034ef65640 1 Processor -- start 2023-11-26T07:23:55.081 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:55.079+0000 7f034ef65640 1 -- start start 2023-11-26T07:23:55.083 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:55.079+0000 7f034ef65640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f03480cd1b0 0x7f0348157760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:23:55.083 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:55.079+0000 7f034ef65640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f03480cf280 0x7f034813af10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:23:55.083 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:55.079+0000 7f034ef65640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f0348143ac0 0x7f0348141550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:23:55.083 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:55.079+0000 7f034ef65640 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f034815aaa0 con 0x7f0348143ac0 2023-11-26T07:23:55.083 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:55.079+0000 7f034ef65640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0348157ca0 con 0x7f03480cd1b0 2023-11-26T07:23:55.083 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:55.079+0000 7f034ef65640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0348141130 con 0x7f03480cf280 2023-11-26T07:23:55.083 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:55.083+0000 7f034d502640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f0348143ac0 0x7f0348141550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:23:55.083 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:55.083+0000 7f034d502640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f0348143ac0 0x7f0348141550 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:38414/0 (socket says 172.21.15.143:38414) 2023-11-26T07:23:55.084 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:55.083+0000 7f034d502640 1 -- 172.21.15.143:0/3973700480 learned_addr learned my addr 172.21.15.143:0/3973700480 (peer_addr_for_me v2:172.21.15.143:0/0) 2023-11-26T07:23:55.084 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:55.083+0000 7f033ffff640 1 --2- 172.21.15.143:0/3973700480 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f03480cf280 0x7f034813af10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:23:55.084 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:55.083+0000 7f034d502640 1 -- 172.21.15.143:0/3973700480 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f03480cf280 msgr2=0x7f034813af10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:23:55.084 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:55.083+0000 7f034d502640 1 --2- 172.21.15.143:0/3973700480 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f03480cf280 0x7f034813af10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:23:55.084 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:55.083+0000 7f034d502640 1 -- 172.21.15.143:0/3973700480 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f03480cd1b0 msgr2=0x7f0348157760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:23:55.084 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:55.083+0000 7f034d502640 1 --2- 172.21.15.143:0/3973700480 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f03480cd1b0 0x7f0348157760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:23:55.084 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:55.083+0000 7f034d502640 1 -- 172.21.15.143:0/3973700480 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0338009420 con 0x7f0348143ac0 2023-11-26T07:23:55.084 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:55.083+0000 7f034d502640 1 --2- 172.21.15.143:0/3973700480 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f0348143ac0 0x7f0348141550 secure :-1 s=READY pgs=1246 cs=0 l=1 rev1=1 crypto rx=0x7f034000d830 tx=0x7f034000dcf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:23:55.084 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:55.083+0000 7f033d7fa640 1 -- 172.21.15.143:0/3973700480 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f03400096a0 con 0x7f0348143ac0 2023-11-26T07:23:55.084 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:55.083+0000 7f033d7fa640 1 -- 172.21.15.143:0/3973700480 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0340009800 con 0x7f0348143ac0 2023-11-26T07:23:55.084 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:55.083+0000 7f033d7fa640 1 -- 172.21.15.143:0/3973700480 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0340009bb0 con 0x7f0348143ac0 2023-11-26T07:23:55.084 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:55.083+0000 7f034ef65640 1 -- 172.21.15.143:0/3973700480 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0348140b70 con 0x7f0348143ac0 2023-11-26T07:23:55.084 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:55.083+0000 7f034ef65640 1 -- 172.21.15.143:0/3973700480 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f034815a730 con 0x7f0348143ac0 2023-11-26T07:23:55.085 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:55.083+0000 7f034ef65640 1 -- 172.21.15.143:0/3973700480 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0348067680 con 0x7f0348143ac0 2023-11-26T07:23:55.085 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:55.083+0000 7f033d7fa640 1 -- 172.21.15.143:0/3973700480 <== mon.0 v2:172.21.15.143:3300/0 4 ==== mgrmap(e 16) v1 ==== 85954+0+0 (secure 0 0 0) 0x7f034001e460 con 0x7f0348143ac0 2023-11-26T07:23:55.086 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:55.087+0000 7f033d7fa640 1 --2- 172.21.15.143:0/3973700480 >> [v2:172.21.15.188:6836/3401128553,v1:172.21.15.188:6837/3401128553] conn(0x7f0334066590 0x7f0334068a50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:23:55.086 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:55.087+0000 7f033d7fa640 1 -- 172.21.15.143:0/3973700480 <== mon.0 v2:172.21.15.143:3300/0 5 ==== osd_map(91..91 src has 1..91) v4 ==== 5795+0+0 (secure 0 0 0) 0x7f0340084770 con 0x7f0348143ac0 2023-11-26T07:23:55.086 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:55.087+0000 7f034cd01640 1 --2- 172.21.15.143:0/3973700480 >> [v2:172.21.15.188:6836/3401128553,v1:172.21.15.188:6837/3401128553] conn(0x7f0334066590 0x7f0334068a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:23:55.087 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:55.087+0000 7f034cd01640 1 --2- 172.21.15.143:0/3973700480 >> [v2:172.21.15.188:6836/3401128553,v1:172.21.15.188:6837/3401128553] conn(0x7f0334066590 0x7f0334068a50 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7f03440020b0 tx=0x7f03440016a0 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:23:55.090 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:55.087+0000 7f033d7fa640 1 -- 172.21.15.143:0/3973700480 <== mon.0 v2:172.21.15.143:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172649 (secure 0 0 0) 0x7f0340051a80 con 0x7f0348143ac0 2023-11-26T07:23:55.240 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:55.239+0000 7f034ef65640 1 -- 172.21.15.143:0/3973700480 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f034815abe0 con 0x7f0348143ac0 2023-11-26T07:23:55.241 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:55.239+0000 7f033d7fa640 1 -- 172.21.15.143:0/3973700480 <== mon.0 v2:172.21.15.143:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v91) v1 ==== 74+0+16727 (secure 0 0 0) 0x7f03400550a0 con 0x7f0348143ac0 2023-11-26T07:23:55.242 INFO:teuthology.orchestra.run.smithi143.stdout: 2023-11-26T07:23:55.242 INFO:teuthology.orchestra.run.smithi143.stdout:{"epoch":91,"fsid":"f5033060-2469-4a07-95fe-2b76f5404bc1","created":"2023-11-26T07:17:28.613663+0000","modified":"2023-11-26T07:23:47.807341+0000","last_up_change":"2023-11-26T07:17:41.157813+0000","last_in_change":"2023-11-26T07:17:31.181658+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":15,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-11-26T07:17:41.758597+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":12,"pool_name":"cephfs_metadata","create_time":"2023-11-26T07:23:26.724016+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"83","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":13,"pool_name":"cephfs_data","create_time":"2023-11-26T07:23:28.019663+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"84","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"ebbee2c6-023e-44b9-9f1d-6f7bfd309730","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6817","nonce":639807369},{"type":"v1","addr":"172.21.15.143:6819","nonce":639807369}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6821","nonce":639807369},{"type":"v1","addr":"172.21.15.143:6823","nonce":639807369}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6830","nonce":639807369},{"type":"v1","addr":"172.21.15.143:6831","nonce":639807369}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6825","nonce":639807369},{"type":"v1","addr":"172.21.15.143:6828","nonce":639807369}]},"public_addr":"172.21.15.143:6819/639807369","cluster_addr":"172.21.15.143:6823/639807369","heartbeat_back_addr":"172.21.15.143:6831/639807369","heartbeat_front_addr":"172.21.15.143:6828/639807369","state":["exists","up"]},{"osd":1,"uuid":"fec4b8be-7122-49e9-9a19-5850a0700001","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6816","nonce":1144942530},{"type":"v1","addr":"172.21.15.143:6818","nonce":1144942530}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6820","nonce":1144942530},{"type":"v1","addr":"172.21.15.143:6822","nonce":1144942530}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6827","nonce":1144942530},{"type":"v1","addr":"172.21.15.143:6829","nonce":1144942530}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6824","nonce":1144942530},{"type":"v1","addr":"172.21.15.143:6826","nonce":1144942530}]},"public_addr":"172.21.15.143:6818/1144942530","cluster_addr":"172.21.15.143:6822/1144942530","heartbeat_back_addr":"172.21.15.143:6829/1144942530","heartbeat_front_addr":"172.21.15.143:6826/1144942530","state":["exists","up"]},{"osd":2,"uuid":"2888af5b-f933-46ca-8f95-6dd494ff43f6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6808","nonce":3909095186},{"type":"v1","addr":"172.21.15.143:6809","nonce":3909095186}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6810","nonce":3909095186},{"type":"v1","addr":"172.21.15.143:6811","nonce":3909095186}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6814","nonce":3909095186},{"type":"v1","addr":"172.21.15.143:6815","nonce":3909095186}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6812","nonce":3909095186},{"type":"v1","addr":"172.21.15.143:6813","nonce":3909095186}]},"public_addr":"172.21.15.143:6809/3909095186","cluster_addr":"172.21.15.143:6811/3909095186","heartbeat_back_addr":"172.21.15.143:6815/3909095186","heartbeat_front_addr":"172.21.15.143:6813/3909095186","state":["exists","up"]},{"osd":3,"uuid":"d1f52435-7bc1-4081-9550-9cd1ba55f549","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6800","nonce":3854210991},{"type":"v1","addr":"172.21.15.143:6801","nonce":3854210991}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6802","nonce":3854210991},{"type":"v1","addr":"172.21.15.143:6803","nonce":3854210991}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6806","nonce":3854210991},{"type":"v1","addr":"172.21.15.143:6807","nonce":3854210991}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6804","nonce":3854210991},{"type":"v1","addr":"172.21.15.143:6805","nonce":3854210991}]},"public_addr":"172.21.15.143:6801/3854210991","cluster_addr":"172.21.15.143:6803/3854210991","heartbeat_back_addr":"172.21.15.143:6807/3854210991","heartbeat_front_addr":"172.21.15.143:6805/3854210991","state":["exists","up"]},{"osd":4,"uuid":"e2ae8b6e-6e25-49b2-bcbf-aacce68f06a3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6800","nonce":3924606902},{"type":"v1","addr":"172.21.15.188:6801","nonce":3924606902}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6802","nonce":3924606902},{"type":"v1","addr":"172.21.15.188:6803","nonce":3924606902}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6806","nonce":3924606902},{"type":"v1","addr":"172.21.15.188:6807","nonce":3924606902}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6804","nonce":3924606902},{"type":"v1","addr":"172.21.15.188:6805","nonce":3924606902}]},"public_addr":"172.21.15.188:6801/3924606902","cluster_addr":"172.21.15.188:6803/3924606902","heartbeat_back_addr":"172.21.15.188:6807/3924606902","heartbeat_front_addr":"172.21.15.188:6805/3924606902","state":["exists","up"]},{"osd":5,"uuid":"3cec8a32-7698-4c36-8e89-e4ba11934f33","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6813","nonce":186841544},{"type":"v1","addr":"172.21.15.188:6815","nonce":186841544}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6817","nonce":186841544},{"type":"v1","addr":"172.21.15.188:6818","nonce":186841544}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6822","nonce":186841544},{"type":"v1","addr":"172.21.15.188:6823","nonce":186841544}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6820","nonce":186841544},{"type":"v1","addr":"172.21.15.188:6821","nonce":186841544}]},"public_addr":"172.21.15.188:6815/186841544","cluster_addr":"172.21.15.188:6818/186841544","heartbeat_back_addr":"172.21.15.188:6823/186841544","heartbeat_front_addr":"172.21.15.188:6821/186841544","state":["exists","up"]},{"osd":6,"uuid":"58a65d5d-e090-473a-81b4-ad38e85029f9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6808","nonce":2153499405},{"type":"v1","addr":"172.21.15.188:6809","nonce":2153499405}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6810","nonce":2153499405},{"type":"v1","addr":"172.21.15.188:6811","nonce":2153499405}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6816","nonce":2153499405},{"type":"v1","addr":"172.21.15.188:6819","nonce":2153499405}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6812","nonce":2153499405},{"type":"v1","addr":"172.21.15.188:6814","nonce":2153499405}]},"public_addr":"172.21.15.188:6809/2153499405","cluster_addr":"172.21.15.188:6811/2153499405","heartbeat_back_addr":"172.21.15.188:6819/2153499405","heartbeat_front_addr":"172.21.15.188:6814/2153499405","state":["exists","up"]},{"osd":7,"uuid":"ea5b7f70-3554-4cf8-ac30-f7c59b80678d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6824","nonce":2722096729},{"type":"v1","addr":"172.21.15.188:6825","nonce":2722096729}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6826","nonce":2722096729},{"type":"v1","addr":"172.21.15.188:6827","nonce":2722096729}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6830","nonce":2722096729},{"type":"v1","addr":"172.21.15.188:6831","nonce":2722096729}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6828","nonce":2722096729},{"type":"v1","addr":"172.21.15.188:6829","nonce":2722096729}]},"public_addr":"172.21.15.188:6825/2722096729","cluster_addr":"172.21.15.188:6827/2722096729","heartbeat_back_addr":"172.21.15.188:6831/2722096729","heartbeat_front_addr":"172.21.15.188:6829/2722096729","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-26T07:17:36.995032+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-26T07:17:36.932266+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-26T07:17:36.829304+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-26T07:17:36.804297+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-26T07:17:36.667892+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-26T07:17:36.964868+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-26T07:17:36.827332+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-26T07:17:36.893179+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.143:6835/4031724325":"2023-11-27T07:23:44.794394+0000","172.21.15.143:6833/4031724325":"2023-11-27T07:23:44.794394+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-11-26T07:23:55.244 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:55.243+0000 7f034ef65640 1 -- 172.21.15.143:0/3973700480 >> [v2:172.21.15.188:6836/3401128553,v1:172.21.15.188:6837/3401128553] conn(0x7f0334066590 msgr2=0x7f0334068a50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:23:55.244 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:55.243+0000 7f034ef65640 1 --2- 172.21.15.143:0/3973700480 >> [v2:172.21.15.188:6836/3401128553,v1:172.21.15.188:6837/3401128553] conn(0x7f0334066590 0x7f0334068a50 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7f03440020b0 tx=0x7f03440016a0 comp rx=0 tx=0).stop 2023-11-26T07:23:55.244 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:55.243+0000 7f034ef65640 1 -- 172.21.15.143:0/3973700480 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f0348143ac0 msgr2=0x7f0348141550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:23:55.244 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:55.243+0000 7f034ef65640 1 --2- 172.21.15.143:0/3973700480 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f0348143ac0 0x7f0348141550 secure :-1 s=READY pgs=1246 cs=0 l=1 rev1=1 crypto rx=0x7f034000d830 tx=0x7f034000dcf0 comp rx=0 tx=0).stop 2023-11-26T07:23:55.244 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:55.243+0000 7f034ef65640 1 -- 172.21.15.143:0/3973700480 shutdown_connections 2023-11-26T07:23:55.244 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:55.243+0000 7f034ef65640 1 --2- 172.21.15.143:0/3973700480 >> [v2:172.21.15.188:6836/3401128553,v1:172.21.15.188:6837/3401128553] conn(0x7f0334066590 0x7f0334068a50 unknown :-1 s=CLOSED pgs=98 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:23:55.244 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:55.243+0000 7f034ef65640 1 --2- 172.21.15.143:0/3973700480 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f0348143ac0 0x7f0348141550 unknown :-1 s=CLOSED pgs=1246 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:23:55.244 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:55.243+0000 7f034ef65640 1 --2- 172.21.15.143:0/3973700480 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f03480cf280 0x7f034813af10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:23:55.245 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:55.243+0000 7f034ef65640 1 --2- 172.21.15.143:0/3973700480 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f03480cd1b0 0x7f0348157760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:23:55.245 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:55.243+0000 7f034ef65640 1 -- 172.21.15.143:0/3973700480 >> 172.21.15.143:0/3973700480 conn(0x7f034806a890 msgr2=0x7f03480b3890 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:23:55.245 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:55.243+0000 7f034ef65640 1 -- 172.21.15.143:0/3973700480 shutdown_connections 2023-11-26T07:23:55.245 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:55.243+0000 7f034ef65640 1 -- 172.21.15.143:0/3973700480 wait complete. 2023-11-26T07:23:55.264 DEBUG:teuthology.orchestra.run.smithi143:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs fail cephfs 2023-11-26T07:23:55.401 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:55.399+0000 7f924f7f5640 1 Processor -- start 2023-11-26T07:23:55.402 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:55.399+0000 7f924f7f5640 1 -- start start 2023-11-26T07:23:55.403 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:55.403+0000 7f924f7f5640 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7f92480cf280 0x7f92480cf710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:23:55.403 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:55.403+0000 7f924f7f5640 1 -- --> v1:172.21.15.143:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f92480cc730 con 0x7f9248143ac0 2023-11-26T07:23:55.403 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:55.403+0000 7f924f7f5640 1 -- --> v1:172.21.15.188:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9248070ee0 con 0x7f92480cd1b0 2023-11-26T07:23:55.403 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:55.403+0000 7f924f7f5640 1 -- --> v2:172.21.15.188:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9248143ea0 con 0x7f92480cf280 2023-11-26T07:23:55.404 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:55.403+0000 7f924d591640 1 --1- >> v1:172.21.15.143:6789/0 conn(0x7f9248143ac0 0x7f9248070370 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.143:6789/0 says I am v1:172.21.15.143:43294/0 (socket says 172.21.15.143:43294) 2023-11-26T07:23:55.404 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:55.403+0000 7f924d591640 1 -- 172.21.15.143:0/1857957968 learned_addr learned my addr 172.21.15.143:0/1857957968 (peer_addr_for_me v1:172.21.15.143:0/0) 2023-11-26T07:23:55.406 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:55.403+0000 7f924cd90640 1 --2- 172.21.15.143:0/1857957968 >> v2:172.21.15.188:3300/0 conn(0x7f92480cf280 0x7f92480cf710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:23:55.406 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:55.403+0000 7f923ffff640 1 -- 172.21.15.143:0/1857957968 <== mon.0 v1:172.21.15.143:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1275442285 0 0) 0x7f92480cc730 con 0x7f9248143ac0 2023-11-26T07:23:55.406 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:55.403+0000 7f923ffff640 1 -- 172.21.15.143:0/1857957968 --> v1:172.21.15.143:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f9234003530 con 0x7f9248143ac0 2023-11-26T07:23:55.406 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:55.403+0000 7f923ffff640 1 -- 172.21.15.143:0/1857957968 <== mon.0 v1:172.21.15.143:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3184594448 0 0) 0x7f9234003530 con 0x7f9248143ac0 2023-11-26T07:23:55.406 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:55.403+0000 7f923ffff640 1 -- 172.21.15.143:0/1857957968 >> v2:172.21.15.188:3300/0 conn(0x7f92480cf280 msgr2=0x7f92480cf710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-26T07:23:55.406 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:55.403+0000 7f923ffff640 1 --2- 172.21.15.143:0/1857957968 >> v2:172.21.15.188:3300/0 conn(0x7f92480cf280 0x7f92480cf710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:23:55.406 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:55.403+0000 7f923ffff640 1 -- 172.21.15.143:0/1857957968 >> v1:172.21.15.188:6789/0 conn(0x7f92480cd1b0 legacy=0x7f92480cd5c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:23:55.406 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:55.403+0000 7f923ffff640 1 -- 172.21.15.143:0/1857957968 --> v1:172.21.15.143:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f92480711c0 con 0x7f9248143ac0 2023-11-26T07:23:55.406 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:55.403+0000 7f923ffff640 1 -- 172.21.15.143:0/1857957968 <== mon.0 v1:172.21.15.143:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2137882523 0 0) 0x7f9244001680 con 0x7f9248143ac0 2023-11-26T07:23:55.406 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:55.403+0000 7f923ffff640 1 -- 172.21.15.143:0/1857957968 >> v1:172.21.15.143:6789/0 conn(0x7f9248143ac0 legacy=0x7f9248070370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:23:55.406 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:55.403+0000 7f923ffff640 1 --2- 172.21.15.143:0/1857957968 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f92340039d0 0x7f9234003df0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:23:55.407 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:55.403+0000 7f923ffff640 1 --2- 172.21.15.143:0/1857957968 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f92340047d0 0x7f9234004c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:23:55.407 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:55.403+0000 7f923ffff640 1 --2- 172.21.15.143:0/1857957968 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f9234005190 0x7f923400b990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:23:55.407 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:55.403+0000 7f923ffff640 1 -- 172.21.15.143:0/1857957968 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f923400c060 con 0x7f92340047d0 2023-11-26T07:23:55.407 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:55.403+0000 7f923ffff640 1 -- 172.21.15.143:0/1857957968 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f923400c1d0 con 0x7f92340039d0 2023-11-26T07:23:55.407 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:55.403+0000 7f923ffff640 1 -- 172.21.15.143:0/1857957968 --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f923400c390 con 0x7f9234005190 2023-11-26T07:23:55.407 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:55.403+0000 7f924cd90640 1 --2- 172.21.15.143:0/1857957968 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f92340047d0 0x7f9234004c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:23:55.407 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:55.403+0000 7f924d591640 1 --2- 172.21.15.143:0/1857957968 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f92340039d0 0x7f9234003df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:23:55.407 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:55.403+0000 7f924dd92640 1 --2- 172.21.15.143:0/1857957968 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f9234005190 0x7f923400b990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:23:55.407 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:55.403+0000 7f924cd90640 1 -- 172.21.15.143:0/1857957968 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f9234005190 msgr2=0x7f923400b990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-26T07:23:55.407 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:55.403+0000 7f924cd90640 1 --2- 172.21.15.143:0/1857957968 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f9234005190 0x7f923400b990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:23:55.407 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:55.403+0000 7f924cd90640 1 -- 172.21.15.143:0/1857957968 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f92340039d0 msgr2=0x7f9234003df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-26T07:23:55.407 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:55.403+0000 7f924cd90640 1 --2- 172.21.15.143:0/1857957968 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f92340039d0 0x7f9234003df0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:23:55.408 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:55.403+0000 7f924cd90640 1 -- 172.21.15.143:0/1857957968 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9234003530 con 0x7f92340047d0 2023-11-26T07:23:55.408 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:55.403+0000 7f924d591640 1 --2- 172.21.15.143:0/1857957968 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f92340039d0 0x7f9234003df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-26T07:23:55.408 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:55.403+0000 7f924dd92640 1 -- 172.21.15.143:0/1857957968 reap_dead start 2023-11-26T07:23:55.409 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:55.407+0000 7f924cd90640 1 --2- 172.21.15.143:0/1857957968 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f92340047d0 0x7f9234004c50 secure :-1 s=READY pgs=1248 cs=0 l=1 rev1=1 crypto rx=0x7f9238009ae0 tx=0x7f923801ca50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ff4fc4139e68c1ce server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:23:55.409 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:55.407+0000 7f923ffff640 1 -- 172.21.15.143:0/1857957968 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f92380a5530 con 0x7f92340047d0 2023-11-26T07:23:55.409 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:55.407+0000 7f923ffff640 1 -- 172.21.15.143:0/1857957968 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f92380a5690 con 0x7f92340047d0 2023-11-26T07:23:55.409 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:55.407+0000 7f923ffff640 1 -- 172.21.15.143:0/1857957968 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f92380a5a40 con 0x7f92340047d0 2023-11-26T07:23:55.409 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:55.407+0000 7f924f7f5640 1 -- 172.21.15.143:0/1857957968 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f92340047d0 msgr2=0x7f9234004c50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:23:55.409 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:55.407+0000 7f924f7f5640 1 --2- 172.21.15.143:0/1857957968 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f92340047d0 0x7f9234004c50 secure :-1 s=READY pgs=1248 cs=0 l=1 rev1=1 crypto rx=0x7f9238009ae0 tx=0x7f923801ca50 comp rx=0 tx=0).stop 2023-11-26T07:23:55.409 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:55.407+0000 7f924f7f5640 1 -- 172.21.15.143:0/1857957968 shutdown_connections 2023-11-26T07:23:55.409 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:55.407+0000 7f924f7f5640 1 --2- 172.21.15.143:0/1857957968 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f92340047d0 0x7f9234004c50 unknown :-1 s=CLOSED pgs=1248 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:23:55.410 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:55.407+0000 7f924f7f5640 1 -- 172.21.15.143:0/1857957968 >> 172.21.15.143:0/1857957968 conn(0x7f924806a890 msgr2=0x7f924806aca0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:23:55.410 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:55.407+0000 7f924f7f5640 1 -- 172.21.15.143:0/1857957968 shutdown_connections 2023-11-26T07:23:55.410 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:55.407+0000 7f924f7f5640 1 -- 172.21.15.143:0/1857957968 wait complete. 2023-11-26T07:23:55.410 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:55.411+0000 7f924f7f5640 1 Processor -- start 2023-11-26T07:23:55.410 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:55.411+0000 7f924f7f5640 1 -- start start 2023-11-26T07:23:55.412 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:55.411+0000 7f924f7f5640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f92340047d0 0x7f924813dba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:23:55.412 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:55.411+0000 7f924f7f5640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f924813b0e0 0x7f924813a950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:23:55.412 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:55.411+0000 7f924f7f5640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f92481f0b40 0x7f92481402f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:23:55.412 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:55.411+0000 7f924f7f5640 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f924813ae90 con 0x7f924813b0e0 2023-11-26T07:23:55.413 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:55.411+0000 7f924f7f5640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f924813b8a0 con 0x7f92481f0b40 2023-11-26T07:23:55.413 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:55.411+0000 7f924f7f5640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f924813bb80 con 0x7f92340047d0 2023-11-26T07:23:55.413 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:55.411+0000 7f924cd90640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f924813b0e0 0x7f924813a950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:23:55.413 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:55.411+0000 7f924cd90640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f924813b0e0 0x7f924813a950 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:38428/0 (socket says 172.21.15.143:38428) 2023-11-26T07:23:55.413 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:55.411+0000 7f924cd90640 1 -- 172.21.15.143:0/1650670332 learned_addr learned my addr 172.21.15.143:0/1650670332 (peer_addr_for_me v2:172.21.15.143:0/0) 2023-11-26T07:23:55.413 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:55.411+0000 7f924dd92640 1 --2- 172.21.15.143:0/1650670332 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f92481f0b40 0x7f92481402f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:23:55.413 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:55.411+0000 7f924d591640 1 --2- 172.21.15.143:0/1650670332 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f92340047d0 0x7f924813dba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:23:55.413 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:55.411+0000 7f924cd90640 1 -- 172.21.15.143:0/1650670332 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f92340047d0 msgr2=0x7f924813dba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:23:55.413 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:55.411+0000 7f924cd90640 1 --2- 172.21.15.143:0/1650670332 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f92340047d0 0x7f924813dba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:23:55.413 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:55.411+0000 7f924cd90640 1 -- 172.21.15.143:0/1650670332 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f92481f0b40 msgr2=0x7f92481402f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:23:55.413 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:55.411+0000 7f924cd90640 1 --2- 172.21.15.143:0/1650670332 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f92481f0b40 0x7f92481402f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:23:55.413 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:55.411+0000 7f924cd90640 1 -- 172.21.15.143:0/1650670332 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f924813be60 con 0x7f924813b0e0 2023-11-26T07:23:55.413 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:55.411+0000 7f924dd92640 1 --2- 172.21.15.143:0/1650670332 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f92481f0b40 0x7f92481402f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-26T07:23:55.414 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:55.411+0000 7f924d591640 1 --2- 172.21.15.143:0/1650670332 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f92340047d0 0x7f924813dba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-26T07:23:55.414 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:55.411+0000 7f924cd90640 1 --2- 172.21.15.143:0/1650670332 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f924813b0e0 0x7f924813a950 secure :-1 s=READY pgs=1249 cs=0 l=1 rev1=1 crypto rx=0x7f923801c880 tx=0x7f92380197f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:23:55.414 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:55.411+0000 7f923dffb640 1 -- 172.21.15.143:0/1650670332 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f92380bd070 con 0x7f924813b0e0 2023-11-26T07:23:55.415 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:55.411+0000 7f923dffb640 1 -- 172.21.15.143:0/1650670332 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9238004020 con 0x7f924813b0e0 2023-11-26T07:23:55.415 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:55.411+0000 7f924f7f5640 1 -- 172.21.15.143:0/1650670332 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f924813c140 con 0x7f924813b0e0 2023-11-26T07:23:55.415 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:55.411+0000 7f923dffb640 1 -- 172.21.15.143:0/1650670332 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f92380043d0 con 0x7f924813b0e0 2023-11-26T07:23:55.415 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:55.411+0000 7f924f7f5640 1 -- 172.21.15.143:0/1650670332 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f924813b520 con 0x7f924813b0e0 2023-11-26T07:23:55.415 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:55.415+0000 7f924f7f5640 1 -- 172.21.15.143:0/1650670332 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9248067680 con 0x7f924813b0e0 2023-11-26T07:23:55.415 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:55.415+0000 7f923dffb640 1 -- 172.21.15.143:0/1650670332 <== mon.0 v2:172.21.15.143:3300/0 4 ==== mgrmap(e 16) v1 ==== 85954+0+0 (secure 0 0 0) 0x7f92380b4070 con 0x7f924813b0e0 2023-11-26T07:23:55.416 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:55.415+0000 7f923dffb640 1 --2- 172.21.15.143:0/1650670332 >> [v2:172.21.15.188:6836/3401128553,v1:172.21.15.188:6837/3401128553] conn(0x7f922c066590 0x7f922c068a50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:23:55.416 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:55.415+0000 7f923dffb640 1 -- 172.21.15.143:0/1650670332 <== mon.0 v2:172.21.15.143:3300/0 5 ==== osd_map(91..91 src has 1..91) v4 ==== 5795+0+0 (secure 0 0 0) 0x7f923811e270 con 0x7f924813b0e0 2023-11-26T07:23:55.416 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:55.415+0000 7f924d591640 1 --2- 172.21.15.143:0/1650670332 >> [v2:172.21.15.188:6836/3401128553,v1:172.21.15.188:6837/3401128553] conn(0x7f922c066590 0x7f922c068a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:23:55.417 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:55.415+0000 7f924d591640 1 --2- 172.21.15.143:0/1650670332 >> [v2:172.21.15.188:6836/3401128553,v1:172.21.15.188:6837/3401128553] conn(0x7f922c066590 0x7f922c068a50 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7f9244000c10 tx=0x7f9244009040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:23:55.420 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:55.419+0000 7f923dffb640 1 -- 172.21.15.143:0/1650670332 <== mon.0 v2:172.21.15.143:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172649 (secure 0 0 0) 0x7f92380eb690 con 0x7f924813b0e0 2023-11-26T07:23:55.600 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:55.599+0000 7f924f7f5640 1 -- 172.21.15.143:0/1650670332 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs"} v 0) v1 -- 0x7f92481420d0 con 0x7f924813b0e0 2023-11-26T07:23:55.600 INFO:tasks.ceph.mon.a.smithi143.stderr:2023-11-26T07:23:55.599+0000 7f6946f73640 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2023-11-26T07:23:55.603 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:55.603+0000 7f923dffb640 1 -- 172.21.15.143:0/1650670332 <== mon.0 v2:172.21.15.143:3300/0 7 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs"}]=0 cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v58) v1 ==== 161+0+0 (secure 0 0 0) 0x7f92380eecb0 con 0x7f924813b0e0 2023-11-26T07:23:55.603 INFO:teuthology.orchestra.run.smithi143.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2023-11-26T07:23:55.605 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:55.603+0000 7f924f7f5640 1 -- 172.21.15.143:0/1650670332 >> [v2:172.21.15.188:6836/3401128553,v1:172.21.15.188:6837/3401128553] conn(0x7f922c066590 msgr2=0x7f922c068a50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:23:55.605 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:55.603+0000 7f924f7f5640 1 --2- 172.21.15.143:0/1650670332 >> [v2:172.21.15.188:6836/3401128553,v1:172.21.15.188:6837/3401128553] conn(0x7f922c066590 0x7f922c068a50 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7f9244000c10 tx=0x7f9244009040 comp rx=0 tx=0).stop 2023-11-26T07:23:55.605 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:55.603+0000 7f924f7f5640 1 -- 172.21.15.143:0/1650670332 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f924813b0e0 msgr2=0x7f924813a950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:23:55.605 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:55.603+0000 7f924f7f5640 1 --2- 172.21.15.143:0/1650670332 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f924813b0e0 0x7f924813a950 secure :-1 s=READY pgs=1249 cs=0 l=1 rev1=1 crypto rx=0x7f923801c880 tx=0x7f92380197f0 comp rx=0 tx=0).stop 2023-11-26T07:23:55.605 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:55.603+0000 7f924f7f5640 1 -- 172.21.15.143:0/1650670332 shutdown_connections 2023-11-26T07:23:55.606 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:55.603+0000 7f924f7f5640 1 --2- 172.21.15.143:0/1650670332 >> [v2:172.21.15.188:6836/3401128553,v1:172.21.15.188:6837/3401128553] conn(0x7f922c066590 0x7f922c068a50 unknown :-1 s=CLOSED pgs=99 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:23:55.606 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:55.603+0000 7f924f7f5640 1 --2- 172.21.15.143:0/1650670332 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f92481f0b40 0x7f92481402f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:23:55.606 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:55.603+0000 7f924f7f5640 1 --2- 172.21.15.143:0/1650670332 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f924813b0e0 0x7f924813a950 unknown :-1 s=CLOSED pgs=1249 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:23:55.606 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:55.603+0000 7f924f7f5640 1 --2- 172.21.15.143:0/1650670332 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f92340047d0 0x7f924813dba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:23:55.606 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:55.603+0000 7f924f7f5640 1 -- 172.21.15.143:0/1650670332 >> 172.21.15.143:0/1650670332 conn(0x7f924806a890 msgr2=0x7f92480cd7f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:23:55.606 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:55.603+0000 7f924f7f5640 1 -- 172.21.15.143:0/1650670332 shutdown_connections 2023-11-26T07:23:55.606 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:55.603+0000 7f924f7f5640 1 -- 172.21.15.143:0/1650670332 wait complete. 2023-11-26T07:23:55.612 INFO:tasks.ceph.mds.b.smithi188.stderr: -1313> 2023-11-26T07:23:34.605+0000 7ff8cbe33640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2023-11-26T07:23:55.625 DEBUG:teuthology.orchestra.run.smithi143:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs rm cephfs --yes-i-really-mean-it 2023-11-26T07:23:55.656 INFO:tasks.ceph.mds.b.smithi188.stdout:starting mds.b at 2023-11-26T07:23:55.765 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:55.763+0000 7f552c09a640 1 Processor -- start 2023-11-26T07:23:55.765 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:55.763+0000 7f552c09a640 1 -- start start 2023-11-26T07:23:55.767 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:55.767+0000 7f552c09a640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f552415d6e0 0x7f55240d0df0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:23:55.767 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:55.767+0000 7f552c09a640 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7f55240b2160 0x7f55240d14b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:23:55.768 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:55.767+0000 7f552c09a640 1 -- --> v1:172.21.15.188:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f55240cf0e0 con 0x7f55240d19f0 2023-11-26T07:23:55.768 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:55.767+0000 7f552c09a640 1 -- --> v2:172.21.15.143:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f552415dac0 con 0x7f55240b2160 2023-11-26T07:23:55.768 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:55.767+0000 7f552c09a640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f55240d1e80 con 0x7f552415d6e0 2023-11-26T07:23:55.768 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:55.767+0000 7f5529635640 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7f55240b2160 0x7f55240d14b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:23:55.768 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:55.767+0000 7f5529635640 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7f55240b2160 0x7f55240d14b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:38440/0 (socket says 172.21.15.143:38440) 2023-11-26T07:23:55.768 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:55.767+0000 7f5529635640 1 -- 172.21.15.143:0/2055573591 learned_addr learned my addr 172.21.15.143:0/2055573591 (peer_addr_for_me v2:172.21.15.143:0/0) 2023-11-26T07:23:55.768 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:55.767+0000 7f5529e36640 1 --2- 172.21.15.143:0/2055573591 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f552415d6e0 0x7f55240d0df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:23:55.768 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:55.767+0000 7f5529635640 1 -- 172.21.15.143:0/2055573591 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f552415d6e0 msgr2=0x7f55240d0df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-26T07:23:55.768 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:55.767+0000 7f5529635640 1 --2- 172.21.15.143:0/2055573591 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f552415d6e0 0x7f55240d0df0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:23:55.768 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:55.767+0000 7f5529635640 1 -- 172.21.15.143:0/2055573591 >> v1:172.21.15.188:6789/0 conn(0x7f55240d19f0 legacy=0x7f55240cfbf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:23:55.768 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:55.767+0000 7f5529635640 1 -- 172.21.15.143:0/2055573591 --> v2:172.21.15.143:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f55240ceb20 con 0x7f55240b2160 2023-11-26T07:23:55.770 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:55.771+0000 7f5529635640 1 --2- 172.21.15.143:0/2055573591 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f55240b2160 0x7f55240d14b0 secure :-1 s=READY pgs=1250 cs=0 l=1 rev1=1 crypto rx=0x7f5514009870 tx=0x7f551401c540 comp rx=0 tx=0).ready entity=mon.0 client_cookie=266d37257ecacc0e server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:23:55.771 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:55.771+0000 7f5528e34640 1 -- 172.21.15.143:0/2055573591 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f55140a7070 con 0x7f55240b2160 2023-11-26T07:23:55.771 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:55.771+0000 7f5528e34640 1 -- 172.21.15.143:0/2055573591 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f55140ab430 con 0x7f55240b2160 2023-11-26T07:23:55.771 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:55.771+0000 7f5528e34640 1 -- 172.21.15.143:0/2055573591 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f55140ab7c0 con 0x7f55240b2160 2023-11-26T07:23:55.772 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:55.771+0000 7f552c09a640 1 -- 172.21.15.143:0/2055573591 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f55240b2160 msgr2=0x7f55240d14b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:23:55.772 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:55.771+0000 7f552c09a640 1 --2- 172.21.15.143:0/2055573591 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f55240b2160 0x7f55240d14b0 secure :-1 s=READY pgs=1250 cs=0 l=1 rev1=1 crypto rx=0x7f5514009870 tx=0x7f551401c540 comp rx=0 tx=0).stop 2023-11-26T07:23:55.772 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:55.771+0000 7f552c09a640 1 -- 172.21.15.143:0/2055573591 shutdown_connections 2023-11-26T07:23:55.772 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:55.771+0000 7f552c09a640 1 --2- 172.21.15.143:0/2055573591 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f55240b2160 0x7f55240d14b0 unknown :-1 s=CLOSED pgs=1250 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:23:55.772 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:55.771+0000 7f552c09a640 1 --2- 172.21.15.143:0/2055573591 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f552415d6e0 0x7f55240d0df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:23:55.772 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:55.771+0000 7f552c09a640 1 -- 172.21.15.143:0/2055573591 >> 172.21.15.143:0/2055573591 conn(0x7f552409c550 msgr2=0x7f55240d3220 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:23:55.772 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:55.771+0000 7f552c09a640 1 -- 172.21.15.143:0/2055573591 shutdown_connections 2023-11-26T07:23:55.772 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:55.771+0000 7f552c09a640 1 -- 172.21.15.143:0/2055573591 wait complete. 2023-11-26T07:23:55.773 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:55.771+0000 7f552c09a640 1 Processor -- start 2023-11-26T07:23:55.773 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:55.771+0000 7f552c09a640 1 -- start start 2023-11-26T07:23:55.775 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:55.775+0000 7f552c09a640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f55240b2160 0x7f552415c2d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:23:55.775 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:55.775+0000 7f552c09a640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f55240d19f0 0x7f55240b5310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:23:55.775 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:55.775+0000 7f552c09a640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f552415d6e0 0x7f55240c9750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:23:55.775 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:55.775+0000 7f552c09a640 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f552415c810 con 0x7f55240d19f0 2023-11-26T07:23:55.775 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:55.775+0000 7f552c09a640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f55240c86b0 con 0x7f55240b2160 2023-11-26T07:23:55.775 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:55.775+0000 7f552c09a640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f55240c8990 con 0x7f552415d6e0 2023-11-26T07:23:55.776 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:55.775+0000 7f5529635640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f55240d19f0 0x7f55240b5310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:23:55.776 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:55.775+0000 7f5529635640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f55240d19f0 0x7f55240b5310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:38454/0 (socket says 172.21.15.143:38454) 2023-11-26T07:23:55.776 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:55.775+0000 7f5529635640 1 -- 172.21.15.143:0/550487017 learned_addr learned my addr 172.21.15.143:0/550487017 (peer_addr_for_me v2:172.21.15.143:0/0) 2023-11-26T07:23:55.776 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:55.775+0000 7f5529e36640 1 --2- 172.21.15.143:0/550487017 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f55240b2160 0x7f552415c2d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:23:55.776 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:55.775+0000 7f552a637640 1 --2- 172.21.15.143:0/550487017 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f552415d6e0 0x7f55240c9750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:23:55.776 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:55.775+0000 7f5529635640 1 -- 172.21.15.143:0/550487017 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f552415d6e0 msgr2=0x7f55240c9750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:23:55.776 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:55.775+0000 7f5529635640 1 --2- 172.21.15.143:0/550487017 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f552415d6e0 0x7f55240c9750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:23:55.776 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:55.775+0000 7f5529635640 1 -- 172.21.15.143:0/550487017 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f55240b2160 msgr2=0x7f552415c2d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:23:55.776 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:55.775+0000 7f5529635640 1 --2- 172.21.15.143:0/550487017 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f55240b2160 0x7f552415c2d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:23:55.776 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:55.775+0000 7f5529635640 1 -- 172.21.15.143:0/550487017 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f55140094d0 con 0x7f55240d19f0 2023-11-26T07:23:55.777 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:55.775+0000 7f5529635640 1 --2- 172.21.15.143:0/550487017 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f55240d19f0 0x7f55240b5310 secure :-1 s=READY pgs=1251 cs=0 l=1 rev1=1 crypto rx=0x7f551401a170 tx=0x7f55140192e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:23:55.777 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:55.775+0000 7f551a7fc640 1 -- 172.21.15.143:0/550487017 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f55140ba070 con 0x7f55240d19f0 2023-11-26T07:23:55.777 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:55.775+0000 7f551a7fc640 1 -- 172.21.15.143:0/550487017 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f55140b4420 con 0x7f55240d19f0 2023-11-26T07:23:55.778 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:55.775+0000 7f551a7fc640 1 -- 172.21.15.143:0/550487017 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f55140b47d0 con 0x7f55240d19f0 2023-11-26T07:23:55.778 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:55.775+0000 7f552c09a640 1 -- 172.21.15.143:0/550487017 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f55240c8c70 con 0x7f55240d19f0 2023-11-26T07:23:55.778 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:55.775+0000 7f552c09a640 1 -- 172.21.15.143:0/550487017 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f55240c9230 con 0x7f55240d19f0 2023-11-26T07:23:55.778 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:55.775+0000 7f552c09a640 1 -- 172.21.15.143:0/550487017 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f55240c9420 con 0x7f55240d19f0 2023-11-26T07:23:55.778 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:55.775+0000 7f551a7fc640 1 -- 172.21.15.143:0/550487017 <== mon.0 v2:172.21.15.143:3300/0 4 ==== mgrmap(e 16) v1 ==== 85954+0+0 (secure 0 0 0) 0x7f55140b4930 con 0x7f55240d19f0 2023-11-26T07:23:55.778 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:55.779+0000 7f551a7fc640 1 --2- 172.21.15.143:0/550487017 >> [v2:172.21.15.188:6836/3401128553,v1:172.21.15.188:6837/3401128553] conn(0x7f55080665c0 0x7f5508068a80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:23:55.778 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:55.779+0000 7f551a7fc640 1 -- 172.21.15.143:0/550487017 <== mon.0 v2:172.21.15.143:3300/0 5 ==== osd_map(92..92 src has 1..92) v4 ==== 5881+0+0 (secure 0 0 0) 0x7f551411da00 con 0x7f55240d19f0 2023-11-26T07:23:55.779 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:55.779+0000 7f5529e36640 1 --2- 172.21.15.143:0/550487017 >> [v2:172.21.15.188:6836/3401128553,v1:172.21.15.188:6837/3401128553] conn(0x7f55080665c0 0x7f5508068a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:23:55.779 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:55.779+0000 7f5529e36640 1 --2- 172.21.15.143:0/550487017 >> [v2:172.21.15.188:6836/3401128553,v1:172.21.15.188:6837/3401128553] conn(0x7f55080665c0 0x7f5508068a80 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7f55240c8440 tx=0x7f5520009190 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:23:55.782 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:55.783+0000 7f551a7fc640 1 -- 172.21.15.143:0/550487017 <== mon.0 v2:172.21.15.143:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172649 (secure 0 0 0) 0x7f55140eacd0 con 0x7f55240d19f0 2023-11-26T07:23:55.966 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:55.963+0000 7f552c09a640 1 -- 172.21.15.143:0/550487017 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true} v 0) v1 -- 0x7f5524067680 con 0x7f55240d19f0 2023-11-26T07:23:56.605 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:56.603+0000 7f551a7fc640 1 -- 172.21.15.143:0/550487017 <== mon.0 v2:172.21.15.143:3300/0 7 ==== mon_command_ack([{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]=0 v59) v1 ==== 104+0+0 (secure 0 0 0) 0x7f55140ee2f0 con 0x7f55240d19f0 2023-11-26T07:23:56.607 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:56.607+0000 7f552c09a640 1 -- 172.21.15.143:0/550487017 >> [v2:172.21.15.188:6836/3401128553,v1:172.21.15.188:6837/3401128553] conn(0x7f55080665c0 msgr2=0x7f5508068a80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:23:56.607 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:56.607+0000 7f552c09a640 1 --2- 172.21.15.143:0/550487017 >> [v2:172.21.15.188:6836/3401128553,v1:172.21.15.188:6837/3401128553] conn(0x7f55080665c0 0x7f5508068a80 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7f55240c8440 tx=0x7f5520009190 comp rx=0 tx=0).stop 2023-11-26T07:23:56.607 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:56.607+0000 7f552c09a640 1 -- 172.21.15.143:0/550487017 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f55240d19f0 msgr2=0x7f55240b5310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:23:56.607 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:56.607+0000 7f552c09a640 1 --2- 172.21.15.143:0/550487017 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f55240d19f0 0x7f55240b5310 secure :-1 s=READY pgs=1251 cs=0 l=1 rev1=1 crypto rx=0x7f551401a170 tx=0x7f55140192e0 comp rx=0 tx=0).stop 2023-11-26T07:23:56.607 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:56.607+0000 7f552c09a640 1 -- 172.21.15.143:0/550487017 shutdown_connections 2023-11-26T07:23:56.608 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:56.607+0000 7f552c09a640 1 --2- 172.21.15.143:0/550487017 >> [v2:172.21.15.188:6836/3401128553,v1:172.21.15.188:6837/3401128553] conn(0x7f55080665c0 0x7f5508068a80 unknown :-1 s=CLOSED pgs=100 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:23:56.608 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:56.607+0000 7f552c09a640 1 --2- 172.21.15.143:0/550487017 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f552415d6e0 0x7f55240c9750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:23:56.608 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:56.607+0000 7f552c09a640 1 --2- 172.21.15.143:0/550487017 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f55240d19f0 0x7f55240b5310 unknown :-1 s=CLOSED pgs=1251 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:23:56.608 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:56.607+0000 7f552c09a640 1 --2- 172.21.15.143:0/550487017 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f55240b2160 0x7f552415c2d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:23:56.608 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:56.607+0000 7f552c09a640 1 -- 172.21.15.143:0/550487017 >> 172.21.15.143:0/550487017 conn(0x7f552409c550 msgr2=0x7f55240b7e20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:23:56.608 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:56.607+0000 7f552c09a640 1 -- 172.21.15.143:0/550487017 shutdown_connections 2023-11-26T07:23:56.608 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:56.607+0000 7f552c09a640 1 -- 172.21.15.143:0/550487017 wait complete. 2023-11-26T07:23:56.629 DEBUG:teuthology.orchestra.run.smithi143:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_metadata cephfs_metadata --yes-i-really-really-mean-it 2023-11-26T07:23:56.772 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:56.771+0000 7f99b359e640 1 Processor -- start 2023-11-26T07:23:56.772 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:56.771+0000 7f99b359e640 1 -- start start 2023-11-26T07:23:56.773 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:56.771+0000 7f99b359e640 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7f99b4160850 0x7f99b40d3d30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:23:56.773 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:56.771+0000 7f99b359e640 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7f99b40cf330 0x7f99b40cda90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:23:56.774 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:56.771+0000 7f99b359e640 1 -- --> v1:172.21.15.188:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f99b40cdfd0 con 0x7f99b4160f60 2023-11-26T07:23:56.774 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:56.771+0000 7f99b359e640 1 -- --> v2:172.21.15.143:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f99b40ce290 con 0x7f99b4160850 2023-11-26T07:23:56.774 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:56.771+0000 7f99b359e640 1 -- --> v2:172.21.15.188:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f99b40ce570 con 0x7f99b40cf330 2023-11-26T07:23:56.774 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:56.771+0000 7f99b259c640 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7f99b4160850 0x7f99b40d3d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:23:56.774 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:56.771+0000 7f99b259c640 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7f99b4160850 0x7f99b40d3d30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:38468/0 (socket says 172.21.15.143:38468) 2023-11-26T07:23:56.774 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:56.771+0000 7f99b259c640 1 -- 172.21.15.143:0/3446259645 learned_addr learned my addr 172.21.15.143:0/3446259645 (peer_addr_for_me v2:172.21.15.143:0/0) 2023-11-26T07:23:56.775 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:56.771+0000 7f99abfff640 1 --2- 172.21.15.143:0/3446259645 >> v2:172.21.15.188:3300/0 conn(0x7f99b40cf330 0x7f99b40cda90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:23:56.775 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:56.771+0000 7f99b259c640 1 -- 172.21.15.143:0/3446259645 >> v2:172.21.15.188:3300/0 conn(0x7f99b40cf330 msgr2=0x7f99b40cda90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-26T07:23:56.775 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:56.771+0000 7f99b259c640 1 --2- 172.21.15.143:0/3446259645 >> v2:172.21.15.188:3300/0 conn(0x7f99b40cf330 0x7f99b40cda90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:23:56.776 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:56.771+0000 7f99b259c640 1 -- 172.21.15.143:0/3446259645 >> v1:172.21.15.188:6789/0 conn(0x7f99b4160f60 legacy=0x7f99b40b27e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:23:56.776 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:56.771+0000 7f99b259c640 1 -- 172.21.15.143:0/3446259645 --> v2:172.21.15.143:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f99b40ce850 con 0x7f99b4160850 2023-11-26T07:23:56.777 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:56.775+0000 7f99b259c640 1 --2- 172.21.15.143:0/3446259645 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f99b4160850 0x7f99b40d3d30 secure :-1 s=READY pgs=1252 cs=0 l=1 rev1=1 crypto rx=0x7f99b40b5df0 tx=0x7f99a401c500 comp rx=0 tx=0).ready entity=mon.0 client_cookie=208757d729130dd0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:23:56.777 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:56.775+0000 7f99b1d9b640 1 -- 172.21.15.143:0/3446259645 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f99a40aa680 con 0x7f99b4160850 2023-11-26T07:23:56.777 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:56.775+0000 7f99b1d9b640 1 -- 172.21.15.143:0/3446259645 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f99a40aa7e0 con 0x7f99b4160850 2023-11-26T07:23:56.778 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:56.775+0000 7f99b1d9b640 1 -- 172.21.15.143:0/3446259645 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f99a40aab70 con 0x7f99b4160850 2023-11-26T07:23:56.778 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:56.775+0000 7f99b359e640 1 -- 172.21.15.143:0/3446259645 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f99b4160850 msgr2=0x7f99b40d3d30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:23:56.778 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:56.775+0000 7f99b359e640 1 --2- 172.21.15.143:0/3446259645 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f99b4160850 0x7f99b40d3d30 secure :-1 s=READY pgs=1252 cs=0 l=1 rev1=1 crypto rx=0x7f99b40b5df0 tx=0x7f99a401c500 comp rx=0 tx=0).stop 2023-11-26T07:23:56.778 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:56.775+0000 7f99b359e640 1 -- 172.21.15.143:0/3446259645 shutdown_connections 2023-11-26T07:23:56.779 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:56.775+0000 7f99b359e640 1 --2- 172.21.15.143:0/3446259645 >> v2:172.21.15.188:3300/0 conn(0x7f99b40cf330 0x7f99b40cda90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:23:56.779 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:56.775+0000 7f99b359e640 1 --2- 172.21.15.143:0/3446259645 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f99b4160850 0x7f99b40d3d30 unknown :-1 s=CLOSED pgs=1252 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:23:56.779 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:56.775+0000 7f99b359e640 1 -- 172.21.15.143:0/3446259645 >> 172.21.15.143:0/3446259645 conn(0x7f99b409c5a0 msgr2=0x7f99b40d3260 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:23:56.779 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:56.779+0000 7f99b359e640 1 -- 172.21.15.143:0/3446259645 shutdown_connections 2023-11-26T07:23:56.779 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:56.779+0000 7f99b359e640 1 -- 172.21.15.143:0/3446259645 wait complete. 2023-11-26T07:23:56.780 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:56.779+0000 7f99b359e640 1 Processor -- start 2023-11-26T07:23:56.780 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:56.779+0000 7f99b359e640 1 -- start start 2023-11-26T07:23:56.781 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:56.779+0000 7f99b359e640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f99b40cf330 0x7f99b40c9500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:23:56.782 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:56.779+0000 7f99b359e640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f99b4160850 0x7f99b40cc960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:23:56.782 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:56.779+0000 7f99b359e640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f99b4160f60 0x7f99b40ccea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:23:56.782 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:56.779+0000 7f99b359e640 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f99b40b7b10 con 0x7f99b4160850 2023-11-26T07:23:56.782 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:56.779+0000 7f99b359e640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f99b40c9a40 con 0x7f99b40cf330 2023-11-26T07:23:56.782 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:56.779+0000 7f99b359e640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f99b40cadc0 con 0x7f99b4160f60 2023-11-26T07:23:56.782 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:56.779+0000 7f99abfff640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f99b4160850 0x7f99b40cc960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:23:56.782 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:56.779+0000 7f99abfff640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f99b4160850 0x7f99b40cc960 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:38484/0 (socket says 172.21.15.143:38484) 2023-11-26T07:23:56.782 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:56.779+0000 7f99abfff640 1 -- 172.21.15.143:0/1312656705 learned_addr learned my addr 172.21.15.143:0/1312656705 (peer_addr_for_me v2:172.21.15.143:0/0) 2023-11-26T07:23:56.782 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:56.779+0000 7f99b259c640 1 --2- 172.21.15.143:0/1312656705 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f99b40cf330 0x7f99b40c9500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:23:56.782 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:56.779+0000 7f99b2d9d640 1 --2- 172.21.15.143:0/1312656705 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f99b4160f60 0x7f99b40ccea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:23:56.782 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:56.779+0000 7f99abfff640 1 -- 172.21.15.143:0/1312656705 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f99b4160f60 msgr2=0x7f99b40ccea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:23:56.782 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:56.779+0000 7f99abfff640 1 --2- 172.21.15.143:0/1312656705 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f99b4160f60 0x7f99b40ccea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:23:56.783 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:56.779+0000 7f99abfff640 1 -- 172.21.15.143:0/1312656705 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f99b40cf330 msgr2=0x7f99b40c9500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:23:56.783 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:56.779+0000 7f99abfff640 1 --2- 172.21.15.143:0/1312656705 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f99b40cf330 0x7f99b40c9500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:23:56.783 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:56.779+0000 7f99abfff640 1 -- 172.21.15.143:0/1312656705 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f99a40094d0 con 0x7f99b4160850 2023-11-26T07:23:56.783 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:56.779+0000 7f99abfff640 1 --2- 172.21.15.143:0/1312656705 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f99b4160850 0x7f99b40cc960 secure :-1 s=READY pgs=1253 cs=0 l=1 rev1=1 crypto rx=0x7f999c00d5f0 tx=0x7f999c00dab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:23:56.784 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:56.779+0000 7f99aaffd640 1 -- 172.21.15.143:0/1312656705 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f999c0125f0 con 0x7f99b4160850 2023-11-26T07:23:56.784 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:56.783+0000 7f99aaffd640 1 -- 172.21.15.143:0/1312656705 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f999c012750 con 0x7f99b4160850 2023-11-26T07:23:56.784 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:56.783+0000 7f99b359e640 1 -- 172.21.15.143:0/1312656705 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f99b4161900 con 0x7f99b4160850 2023-11-26T07:23:56.784 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:56.783+0000 7f99aaffd640 1 -- 172.21.15.143:0/1312656705 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f999c012ae0 con 0x7f99b4160850 2023-11-26T07:23:56.784 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:56.783+0000 7f99b359e640 1 -- 172.21.15.143:0/1312656705 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f99b40b6320 con 0x7f99b4160850 2023-11-26T07:23:56.784 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:56.783+0000 7f99b359e640 1 -- 172.21.15.143:0/1312656705 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f99b4067500 con 0x7f99b4160850 2023-11-26T07:23:56.784 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:56.783+0000 7f99aaffd640 1 -- 172.21.15.143:0/1312656705 <== mon.0 v2:172.21.15.143:3300/0 4 ==== mgrmap(e 16) v1 ==== 85954+0+0 (secure 0 0 0) 0x7f999c012c40 con 0x7f99b4160850 2023-11-26T07:23:56.785 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:56.783+0000 7f99aaffd640 1 --2- 172.21.15.143:0/1312656705 >> [v2:172.21.15.188:6836/3401128553,v1:172.21.15.188:6837/3401128553] conn(0x7f99940665c0 0x7f9994068a80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:23:56.785 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:56.783+0000 7f99aaffd640 1 -- 172.21.15.143:0/1312656705 <== mon.0 v2:172.21.15.143:3300/0 5 ==== osd_map(92..92 src has 1..92) v4 ==== 5881+0+0 (secure 0 0 0) 0x7f999c084360 con 0x7f99b4160850 2023-11-26T07:23:56.785 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:56.783+0000 7f99b259c640 1 --2- 172.21.15.143:0/1312656705 >> [v2:172.21.15.188:6836/3401128553,v1:172.21.15.188:6837/3401128553] conn(0x7f99940665c0 0x7f9994068a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:23:56.787 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:56.787+0000 7f99b259c640 1 --2- 172.21.15.143:0/1312656705 >> [v2:172.21.15.188:6836/3401128553,v1:172.21.15.188:6837/3401128553] conn(0x7f99940665c0 0x7f9994068a80 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7f99a40ae000 tx=0x7f99a40b5040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:23:56.789 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:56.787+0000 7f99aaffd640 1 -- 172.21.15.143:0/1312656705 <== mon.0 v2:172.21.15.143:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172649 (secure 0 0 0) 0x7f999c0516d0 con 0x7f99b4160850 2023-11-26T07:23:56.942 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:56.943+0000 7f99b359e640 1 -- 172.21.15.143:0/1312656705 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7f99b40b2cd0 con 0x7f99b4160850 2023-11-26T07:23:57.856 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:57.855+0000 7f99aaffd640 1 -- 172.21.15.143:0/1312656705 <== mon.0 v2:172.21.15.143:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_metadata' removed v93) v1 ==== 181+0+0 (secure 0 0 0) 0x7f999c05b030 con 0x7f99b4160850 2023-11-26T07:23:57.856 INFO:teuthology.orchestra.run.smithi143.stderr:pool 'cephfs_metadata' removed 2023-11-26T07:23:57.862 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:57.859+0000 7f99b359e640 1 -- 172.21.15.143:0/1312656705 >> [v2:172.21.15.188:6836/3401128553,v1:172.21.15.188:6837/3401128553] conn(0x7f99940665c0 msgr2=0x7f9994068a80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:23:57.862 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:57.859+0000 7f99b359e640 1 --2- 172.21.15.143:0/1312656705 >> [v2:172.21.15.188:6836/3401128553,v1:172.21.15.188:6837/3401128553] conn(0x7f99940665c0 0x7f9994068a80 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7f99a40ae000 tx=0x7f99a40b5040 comp rx=0 tx=0).stop 2023-11-26T07:23:57.862 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:57.859+0000 7f99b359e640 1 -- 172.21.15.143:0/1312656705 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f99b4160850 msgr2=0x7f99b40cc960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:23:57.862 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:57.859+0000 7f99b359e640 1 --2- 172.21.15.143:0/1312656705 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f99b4160850 0x7f99b40cc960 secure :-1 s=READY pgs=1253 cs=0 l=1 rev1=1 crypto rx=0x7f999c00d5f0 tx=0x7f999c00dab0 comp rx=0 tx=0).stop 2023-11-26T07:23:57.863 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:57.863+0000 7f99b359e640 1 -- 172.21.15.143:0/1312656705 shutdown_connections 2023-11-26T07:23:57.863 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:57.863+0000 7f99b359e640 1 --2- 172.21.15.143:0/1312656705 >> [v2:172.21.15.188:6836/3401128553,v1:172.21.15.188:6837/3401128553] conn(0x7f99940665c0 0x7f9994068a80 unknown :-1 s=CLOSED pgs=102 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:23:57.863 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:57.863+0000 7f99b359e640 1 --2- 172.21.15.143:0/1312656705 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f99b4160f60 0x7f99b40ccea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:23:57.863 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:57.863+0000 7f99b359e640 1 --2- 172.21.15.143:0/1312656705 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f99b4160850 0x7f99b40cc960 unknown :-1 s=CLOSED pgs=1253 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:23:57.863 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:57.863+0000 7f99b359e640 1 --2- 172.21.15.143:0/1312656705 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f99b40cf330 0x7f99b40c9500 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:23:57.863 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:57.863+0000 7f99b359e640 1 -- 172.21.15.143:0/1312656705 >> 172.21.15.143:0/1312656705 conn(0x7f99b409c5a0 msgr2=0x7f99b40cfae0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:23:57.863 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:57.863+0000 7f99b359e640 1 -- 172.21.15.143:0/1312656705 shutdown_connections 2023-11-26T07:23:57.863 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:57.863+0000 7f99b359e640 1 -- 172.21.15.143:0/1312656705 wait complete. 2023-11-26T07:23:57.888 DEBUG:teuthology.orchestra.run.smithi143:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_data cephfs_data --yes-i-really-really-mean-it 2023-11-26T07:23:58.031 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:58.031+0000 7f32dcf2a640 1 Processor -- start 2023-11-26T07:23:58.031 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:58.031+0000 7f32dcf2a640 1 -- start start 2023-11-26T07:23:58.033 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:58.031+0000 7f32dcf2a640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f32d80cf280 0x7f32d80cf6e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:23:58.033 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:58.031+0000 7f32dcf2a640 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7f32d80cd1b0 0x7f32d80cd5e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:23:58.033 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:58.031+0000 7f32dcf2a640 1 -- --> v1:172.21.15.143:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f32d80cfc20 con 0x7f32d8143ac0 2023-11-26T07:23:58.033 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:58.031+0000 7f32dcf2a640 1 -- --> v2:172.21.15.143:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f32d8143ea0 con 0x7f32d80cd1b0 2023-11-26T07:23:58.033 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:58.031+0000 7f32dcf2a640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f32d80cc730 con 0x7f32d80cf280 2023-11-26T07:23:58.033 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:58.031+0000 7f32d659c640 1 --1- >> v1:172.21.15.143:6789/0 conn(0x7f32d8143ac0 0x7f32d8070370 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.143:6789/0 says I am v1:172.21.15.143:43298/0 (socket says 172.21.15.143:43298) 2023-11-26T07:23:58.033 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:58.031+0000 7f32d659c640 1 -- 172.21.15.143:0/3562395597 learned_addr learned my addr 172.21.15.143:0/3562395597 (peer_addr_for_me v1:172.21.15.143:0/0) 2023-11-26T07:23:58.033 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:58.031+0000 7f32d6d9d640 1 --2- 172.21.15.143:0/3562395597 >> v2:172.21.15.143:3300/0 conn(0x7f32d80cd1b0 0x7f32d80cd5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:23:58.033 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:58.031+0000 7f32cdd9b640 1 --2- 172.21.15.143:0/3562395597 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f32d80cf280 0x7f32d80cf6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:23:58.033 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:58.031+0000 7f32d5d9b640 1 -- 172.21.15.143:0/3562395597 <== mon.0 v1:172.21.15.143:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1205633688 0 0) 0x7f32d80cfc20 con 0x7f32d8143ac0 2023-11-26T07:23:58.034 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:58.031+0000 7f32d5d9b640 1 -- 172.21.15.143:0/3562395597 --> v1:172.21.15.143:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f32c4003530 con 0x7f32d8143ac0 2023-11-26T07:23:58.034 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:58.031+0000 7f32d6d9d640 1 -- 172.21.15.143:0/3562395597 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f32d80cf280 msgr2=0x7f32d80cf6e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-26T07:23:58.034 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:58.031+0000 7f32d6d9d640 1 --2- 172.21.15.143:0/3562395597 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f32d80cf280 0x7f32d80cf6e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:23:58.034 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:58.031+0000 7f32d6d9d640 1 -- 172.21.15.143:0/3562395597 >> v1:172.21.15.143:6789/0 conn(0x7f32d8143ac0 legacy=0x7f32d8070370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:23:58.034 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:58.031+0000 7f32d6d9d640 1 -- 172.21.15.143:0/3562395597 --> v2:172.21.15.143:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f32d8070ee0 con 0x7f32d80cd1b0 2023-11-26T07:23:58.036 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:58.035+0000 7f32d6d9d640 1 --2- 172.21.15.143:0/3562395597 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f32d80cd1b0 0x7f32d80cd5e0 secure :-1 s=READY pgs=1255 cs=0 l=1 rev1=1 crypto rx=0x7f32d0009d90 tx=0x7f32d001c950 comp rx=0 tx=0).ready entity=mon.0 client_cookie=16e65b747f636651 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:23:58.036 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:58.035+0000 7f32d5d9b640 1 -- 172.21.15.143:0/3562395597 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f32d00a5a10 con 0x7f32d80cd1b0 2023-11-26T07:23:58.036 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:58.035+0000 7f32d5d9b640 1 -- 172.21.15.143:0/3562395597 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f32d00a5b70 con 0x7f32d80cd1b0 2023-11-26T07:23:58.036 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:58.035+0000 7f32d5d9b640 1 -- 172.21.15.143:0/3562395597 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f32d00ad440 con 0x7f32d80cd1b0 2023-11-26T07:23:58.037 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:58.035+0000 7f32dcf2a640 1 -- 172.21.15.143:0/3562395597 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f32d80cd1b0 msgr2=0x7f32d80cd5e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:23:58.037 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:58.035+0000 7f32dcf2a640 1 --2- 172.21.15.143:0/3562395597 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f32d80cd1b0 0x7f32d80cd5e0 secure :-1 s=READY pgs=1255 cs=0 l=1 rev1=1 crypto rx=0x7f32d0009d90 tx=0x7f32d001c950 comp rx=0 tx=0).stop 2023-11-26T07:23:58.037 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:58.035+0000 7f32dcf2a640 1 -- 172.21.15.143:0/3562395597 shutdown_connections 2023-11-26T07:23:58.037 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:58.035+0000 7f32dcf2a640 1 --2- 172.21.15.143:0/3562395597 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f32d80cd1b0 0x7f32d80cd5e0 unknown :-1 s=CLOSED pgs=1255 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:23:58.037 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:58.035+0000 7f32dcf2a640 1 --2- 172.21.15.143:0/3562395597 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f32d80cf280 0x7f32d80cf6e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:23:58.037 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:58.035+0000 7f32dcf2a640 1 -- 172.21.15.143:0/3562395597 >> 172.21.15.143:0/3562395597 conn(0x7f32d806a890 msgr2=0x7f32d806aca0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:23:58.037 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:58.035+0000 7f32dcf2a640 1 -- 172.21.15.143:0/3562395597 shutdown_connections 2023-11-26T07:23:58.037 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:58.035+0000 7f32dcf2a640 1 -- 172.21.15.143:0/3562395597 wait complete. 2023-11-26T07:23:58.039 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:58.039+0000 7f32dcf2a640 1 Processor -- start 2023-11-26T07:23:58.039 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:58.039+0000 7f32dcf2a640 1 -- start start 2023-11-26T07:23:58.041 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:58.039+0000 7f32dcf2a640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f32d80cd1b0 0x7f32d8157990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:23:58.041 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:58.039+0000 7f32dcf2a640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f32d80cf280 0x7f32d81582e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:23:58.041 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:58.039+0000 7f32dcf2a640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f32d8143ac0 0x7f32d8158820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:23:58.041 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:58.039+0000 7f32dcf2a640 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f32d8138510 con 0x7f32d80cf280 2023-11-26T07:23:58.041 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:58.039+0000 7f32dcf2a640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f32d81387f0 con 0x7f32d8143ac0 2023-11-26T07:23:58.041 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:58.039+0000 7f32dcf2a640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f32d8138ad0 con 0x7f32d80cd1b0 2023-11-26T07:23:58.041 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:58.039+0000 7f32cdd9b640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f32d80cf280 0x7f32d81582e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:23:58.041 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:58.039+0000 7f32cdd9b640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f32d80cf280 0x7f32d81582e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:38496/0 (socket says 172.21.15.143:38496) 2023-11-26T07:23:58.041 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:58.039+0000 7f32cdd9b640 1 -- 172.21.15.143:0/695185154 learned_addr learned my addr 172.21.15.143:0/695185154 (peer_addr_for_me v2:172.21.15.143:0/0) 2023-11-26T07:23:58.041 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:58.039+0000 7f32d659c640 1 --2- 172.21.15.143:0/695185154 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f32d80cd1b0 0x7f32d8157990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:23:58.041 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:58.039+0000 7f32d6d9d640 1 --2- 172.21.15.143:0/695185154 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f32d8143ac0 0x7f32d8158820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:23:58.041 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:58.039+0000 7f32cdd9b640 1 -- 172.21.15.143:0/695185154 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f32d80cd1b0 msgr2=0x7f32d8157990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:23:58.042 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:58.039+0000 7f32cdd9b640 1 --2- 172.21.15.143:0/695185154 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f32d80cd1b0 0x7f32d8157990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:23:58.042 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:58.039+0000 7f32cdd9b640 1 -- 172.21.15.143:0/695185154 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f32d8143ac0 msgr2=0x7f32d8158820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:23:58.042 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:58.039+0000 7f32cdd9b640 1 --2- 172.21.15.143:0/695185154 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f32d8143ac0 0x7f32d8158820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:23:58.042 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:58.039+0000 7f32cdd9b640 1 -- 172.21.15.143:0/695185154 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f32d0009a70 con 0x7f32d80cf280 2023-11-26T07:23:58.042 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:58.039+0000 7f32cdd9b640 1 --2- 172.21.15.143:0/695185154 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f32d80cf280 0x7f32d81582e0 secure :-1 s=READY pgs=1256 cs=0 l=1 rev1=1 crypto rx=0x7f32c000d5f0 tx=0x7f32c000dab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:23:58.043 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:58.039+0000 7f32cf7fe640 1 -- 172.21.15.143:0/695185154 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f32c00125f0 con 0x7f32d80cf280 2023-11-26T07:23:58.043 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:58.039+0000 7f32cf7fe640 1 -- 172.21.15.143:0/695185154 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f32c0012750 con 0x7f32d80cf280 2023-11-26T07:23:58.043 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:58.039+0000 7f32dcf2a640 1 -- 172.21.15.143:0/695185154 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f32d8138db0 con 0x7f32d80cf280 2023-11-26T07:23:58.043 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:58.039+0000 7f32cf7fe640 1 -- 172.21.15.143:0/695185154 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f32c0012ae0 con 0x7f32d80cf280 2023-11-26T07:23:58.043 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:58.039+0000 7f32dcf2a640 1 -- 172.21.15.143:0/695185154 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f32d81395b0 con 0x7f32d80cf280 2023-11-26T07:23:58.043 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:58.043+0000 7f32dcf2a640 1 -- 172.21.15.143:0/695185154 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f32d8067680 con 0x7f32d80cf280 2023-11-26T07:23:58.043 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:58.043+0000 7f32cf7fe640 1 -- 172.21.15.143:0/695185154 <== mon.0 v2:172.21.15.143:3300/0 4 ==== mgrmap(e 16) v1 ==== 85954+0+0 (secure 0 0 0) 0x7f32c0012c40 con 0x7f32d80cf280 2023-11-26T07:23:58.044 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:58.043+0000 7f32cf7fe640 1 --2- 172.21.15.143:0/695185154 >> [v2:172.21.15.188:6836/3401128553,v1:172.21.15.188:6837/3401128553] conn(0x7f32b8066590 0x7f32b8068a50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:23:58.044 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:58.043+0000 7f32cf7fe640 1 -- 172.21.15.143:0/695185154 <== mon.0 v2:172.21.15.143:3300/0 5 ==== osd_map(93..93 src has 1..93) v4 ==== 5443+0+0 (secure 0 0 0) 0x7f32c0084320 con 0x7f32d80cf280 2023-11-26T07:23:58.044 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:58.043+0000 7f32d659c640 1 --2- 172.21.15.143:0/695185154 >> [v2:172.21.15.188:6836/3401128553,v1:172.21.15.188:6837/3401128553] conn(0x7f32b8066590 0x7f32b8068a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:23:58.045 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:58.043+0000 7f32d659c640 1 --2- 172.21.15.143:0/695185154 >> [v2:172.21.15.188:6836/3401128553,v1:172.21.15.188:6837/3401128553] conn(0x7f32b8066590 0x7f32b8068a50 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7f32c80011f0 tx=0x7f32c8007d40 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:23:58.047 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:58.047+0000 7f32cf7fe640 1 -- 172.21.15.143:0/695185154 <== mon.0 v2:172.21.15.143:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172649 (secure 0 0 0) 0x7f32c00517a0 con 0x7f32d80cf280 2023-11-26T07:23:58.199 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:58.199+0000 7f32dcf2a640 1 -- 172.21.15.143:0/695185154 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7f32d813b150 con 0x7f32d80cf280 2023-11-26T07:23:58.858 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:58.859+0000 7f32cf7fe640 1 -- 172.21.15.143:0/695185154 <== mon.0 v2:172.21.15.143:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_data' removed v94) v1 ==== 169+0+0 (secure 0 0 0) 0x7f32c0054dc0 con 0x7f32d80cf280 2023-11-26T07:23:58.858 INFO:teuthology.orchestra.run.smithi143.stderr:pool 'cephfs_data' removed 2023-11-26T07:23:58.861 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:58.859+0000 7f32dcf2a640 1 -- 172.21.15.143:0/695185154 >> [v2:172.21.15.188:6836/3401128553,v1:172.21.15.188:6837/3401128553] conn(0x7f32b8066590 msgr2=0x7f32b8068a50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:23:58.861 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:58.859+0000 7f32dcf2a640 1 --2- 172.21.15.143:0/695185154 >> [v2:172.21.15.188:6836/3401128553,v1:172.21.15.188:6837/3401128553] conn(0x7f32b8066590 0x7f32b8068a50 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7f32c80011f0 tx=0x7f32c8007d40 comp rx=0 tx=0).stop 2023-11-26T07:23:58.861 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:58.859+0000 7f32dcf2a640 1 -- 172.21.15.143:0/695185154 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f32d80cf280 msgr2=0x7f32d81582e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:23:58.861 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:58.859+0000 7f32dcf2a640 1 --2- 172.21.15.143:0/695185154 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f32d80cf280 0x7f32d81582e0 secure :-1 s=READY pgs=1256 cs=0 l=1 rev1=1 crypto rx=0x7f32c000d5f0 tx=0x7f32c000dab0 comp rx=0 tx=0).stop 2023-11-26T07:23:58.861 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:58.859+0000 7f32dcf2a640 1 -- 172.21.15.143:0/695185154 shutdown_connections 2023-11-26T07:23:58.861 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:58.859+0000 7f32dcf2a640 1 --2- 172.21.15.143:0/695185154 >> [v2:172.21.15.188:6836/3401128553,v1:172.21.15.188:6837/3401128553] conn(0x7f32b8066590 0x7f32b8068a50 unknown :-1 s=CLOSED pgs=103 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:23:58.861 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:58.859+0000 7f32dcf2a640 1 --2- 172.21.15.143:0/695185154 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f32d8143ac0 0x7f32d8158820 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:23:58.862 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:58.859+0000 7f32dcf2a640 1 --2- 172.21.15.143:0/695185154 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f32d80cf280 0x7f32d81582e0 unknown :-1 s=CLOSED pgs=1256 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:23:58.862 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:58.859+0000 7f32dcf2a640 1 --2- 172.21.15.143:0/695185154 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f32d80cd1b0 0x7f32d8157990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:23:58.862 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:58.859+0000 7f32dcf2a640 1 -- 172.21.15.143:0/695185154 >> 172.21.15.143:0/695185154 conn(0x7f32d806a890 msgr2=0x7f32d8072130 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:23:58.862 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:58.859+0000 7f32dcf2a640 1 -- 172.21.15.143:0/695185154 shutdown_connections 2023-11-26T07:23:58.862 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:58.859+0000 7f32dcf2a640 1 -- 172.21.15.143:0/695185154 wait complete. 2023-11-26T07:23:58.889 DEBUG:teuthology.orchestra.run.smithi143:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config rm mon mon_allow_pool_delete 2023-11-26T07:23:59.038 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:59.035+0000 7f9a3e3f8640 1 Processor -- start 2023-11-26T07:23:59.038 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:59.035+0000 7f9a3e3f8640 1 -- start start 2023-11-26T07:23:59.039 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:59.039+0000 7f9a3e3f8640 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7f9a380cf990 0x7f9a3815efa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:23:59.040 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:59.039+0000 7f9a3e3f8640 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7f9a3815f4e0 0x7f9a380cd7b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:23:59.040 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:59.039+0000 7f9a3e3f8640 1 -- --> v1:172.21.15.188:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9a3806c520 con 0x7f9a3813e290 2023-11-26T07:23:59.040 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:59.039+0000 7f9a3e3f8640 1 -- --> v2:172.21.15.143:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9a3806c800 con 0x7f9a3815f4e0 2023-11-26T07:23:59.040 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:59.039+0000 7f9a3e3f8640 1 -- --> v2:172.21.15.188:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9a3806cae0 con 0x7f9a380cf990 2023-11-26T07:23:59.040 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:59.039+0000 7f9a3c995640 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7f9a3815f4e0 0x7f9a380cd7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:23:59.040 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:59.039+0000 7f9a3c995640 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7f9a3815f4e0 0x7f9a380cd7b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:38508/0 (socket says 172.21.15.143:38508) 2023-11-26T07:23:59.040 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:59.039+0000 7f9a3c995640 1 -- 172.21.15.143:0/3963971321 learned_addr learned my addr 172.21.15.143:0/3963971321 (peer_addr_for_me v2:172.21.15.143:0/0) 2023-11-26T07:23:59.040 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:59.039+0000 7f9a377fe640 1 --2- 172.21.15.143:0/3963971321 >> v2:172.21.15.188:3300/0 conn(0x7f9a380cf990 0x7f9a3815efa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:23:59.040 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:59.039+0000 7f9a3c995640 1 -- 172.21.15.143:0/3963971321 >> v2:172.21.15.188:3300/0 conn(0x7f9a380cf990 msgr2=0x7f9a3815efa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-26T07:23:59.040 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:59.039+0000 7f9a3c995640 1 --2- 172.21.15.143:0/3963971321 >> v2:172.21.15.188:3300/0 conn(0x7f9a380cf990 0x7f9a3815efa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:23:59.040 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:59.039+0000 7f9a3c995640 1 -- 172.21.15.143:0/3963971321 >> v1:172.21.15.188:6789/0 conn(0x7f9a3813e290 legacy=0x7f9a380cf280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-26T07:23:59.040 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:59.039+0000 7f9a3c995640 1 -- 172.21.15.143:0/3963971321 --> v2:172.21.15.143:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9a3806cdc0 con 0x7f9a3815f4e0 2023-11-26T07:23:59.043 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:59.043+0000 7f9a3c995640 1 --2- 172.21.15.143:0/3963971321 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f9a3815f4e0 0x7f9a380cd7b0 secure :-1 s=READY pgs=1257 cs=0 l=1 rev1=1 crypto rx=0x7f9a380cfe40 tx=0x7f9a3001c960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ad15d106b2477e1c server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:23:59.043 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:59.043+0000 7f9a36ffd640 1 -- 172.21.15.143:0/3963971321 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9a3815fb60 con 0x7f9a3815f4e0 2023-11-26T07:23:59.043 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:59.043+0000 7f9a36ffd640 1 -- 172.21.15.143:0/3963971321 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9a300a5840 con 0x7f9a3815f4e0 2023-11-26T07:23:59.043 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:59.043+0000 7f9a36ffd640 1 -- 172.21.15.143:0/3963971321 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9a300a5bd0 con 0x7f9a3815f4e0 2023-11-26T07:23:59.044 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:59.043+0000 7f9a3e3f8640 1 -- 172.21.15.143:0/3963971321 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f9a3815f4e0 msgr2=0x7f9a380cd7b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:23:59.044 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:59.043+0000 7f9a3e3f8640 1 --2- 172.21.15.143:0/3963971321 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f9a3815f4e0 0x7f9a380cd7b0 secure :-1 s=READY pgs=1257 cs=0 l=1 rev1=1 crypto rx=0x7f9a380cfe40 tx=0x7f9a3001c960 comp rx=0 tx=0).stop 2023-11-26T07:23:59.044 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:59.043+0000 7f9a3e3f8640 1 -- 172.21.15.143:0/3963971321 shutdown_connections 2023-11-26T07:23:59.044 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:59.043+0000 7f9a3e3f8640 1 --2- 172.21.15.143:0/3963971321 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f9a3815f4e0 0x7f9a380cd7b0 unknown :-1 s=CLOSED pgs=1257 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:23:59.044 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:59.043+0000 7f9a3e3f8640 1 --2- 172.21.15.143:0/3963971321 >> v2:172.21.15.188:3300/0 conn(0x7f9a380cf990 0x7f9a3815efa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:23:59.044 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:59.043+0000 7f9a3e3f8640 1 -- 172.21.15.143:0/3963971321 >> 172.21.15.143:0/3963971321 conn(0x7f9a3806a890 msgr2=0x7f9a3806aca0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:23:59.044 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:59.043+0000 7f9a3e3f8640 1 -- 172.21.15.143:0/3963971321 shutdown_connections 2023-11-26T07:23:59.044 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:59.043+0000 7f9a3e3f8640 1 -- 172.21.15.143:0/3963971321 wait complete. 2023-11-26T07:23:59.045 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:59.043+0000 7f9a3e3f8640 1 Processor -- start 2023-11-26T07:23:59.045 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:59.043+0000 7f9a3e3f8640 1 -- start start 2023-11-26T07:23:59.048 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:59.047+0000 7f9a3e3f8640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f9a380cf990 0x7f9a380b3ec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:23:59.048 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:59.047+0000 7f9a3e3f8640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f9a3813e290 0x7f9a381f8e80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:23:59.048 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:59.047+0000 7f9a3e3f8640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f9a3815f4e0 0x7f9a380b0ee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:23:59.048 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:59.047+0000 7f9a3e3f8640 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9a3815dcb0 con 0x7f9a3813e290 2023-11-26T07:23:59.048 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:59.047+0000 7f9a3e3f8640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9a3815d0a0 con 0x7f9a380cf990 2023-11-26T07:23:59.048 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:59.047+0000 7f9a3e3f8640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9a3815d380 con 0x7f9a3815f4e0 2023-11-26T07:23:59.049 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:59.047+0000 7f9a377fe640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f9a3813e290 0x7f9a381f8e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:23:59.049 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:59.047+0000 7f9a377fe640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f9a3813e290 0x7f9a381f8e80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:38510/0 (socket says 172.21.15.143:38510) 2023-11-26T07:23:59.049 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:59.047+0000 7f9a377fe640 1 -- 172.21.15.143:0/1379639167 learned_addr learned my addr 172.21.15.143:0/1379639167 (peer_addr_for_me v2:172.21.15.143:0/0) 2023-11-26T07:23:59.049 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:59.047+0000 7f9a37fff640 1 --2- 172.21.15.143:0/1379639167 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f9a380cf990 0x7f9a380b3ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:23:59.049 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:59.047+0000 7f9a3c995640 1 --2- 172.21.15.143:0/1379639167 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f9a3815f4e0 0x7f9a380b0ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:23:59.049 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:59.047+0000 7f9a377fe640 1 -- 172.21.15.143:0/1379639167 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f9a3815f4e0 msgr2=0x7f9a380b0ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:23:59.049 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:59.047+0000 7f9a377fe640 1 --2- 172.21.15.143:0/1379639167 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f9a3815f4e0 0x7f9a380b0ee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:23:59.049 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:59.047+0000 7f9a377fe640 1 -- 172.21.15.143:0/1379639167 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f9a380cf990 msgr2=0x7f9a380b3ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:23:59.050 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:59.047+0000 7f9a377fe640 1 --2- 172.21.15.143:0/1379639167 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f9a380cf990 0x7f9a380b3ec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:23:59.050 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:59.047+0000 7f9a377fe640 1 -- 172.21.15.143:0/1379639167 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9a30009b30 con 0x7f9a3813e290 2023-11-26T07:23:59.050 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:59.047+0000 7f9a3c995640 1 --2- 172.21.15.143:0/1379639167 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f9a3815f4e0 0x7f9a380b0ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-26T07:23:59.050 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:59.047+0000 7f9a377fe640 1 --2- 172.21.15.143:0/1379639167 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f9a3813e290 0x7f9a381f8e80 secure :-1 s=READY pgs=1258 cs=0 l=1 rev1=1 crypto rx=0x7f9a2c00d800 tx=0x7f9a2c00dcc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:23:59.050 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:59.047+0000 7f9a34ff9640 1 -- 172.21.15.143:0/1379639167 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9a2c00b6c0 con 0x7f9a3813e290 2023-11-26T07:23:59.050 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:59.047+0000 7f9a34ff9640 1 -- 172.21.15.143:0/1379639167 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9a2c00b820 con 0x7f9a3813e290 2023-11-26T07:23:59.050 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:59.047+0000 7f9a3e3f8640 1 -- 172.21.15.143:0/1379639167 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9a3815d660 con 0x7f9a3813e290 2023-11-26T07:23:59.050 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:59.047+0000 7f9a34ff9640 1 -- 172.21.15.143:0/1379639167 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9a2c00bb90 con 0x7f9a3813e290 2023-11-26T07:23:59.050 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:59.047+0000 7f9a3e3f8640 1 -- 172.21.15.143:0/1379639167 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9a380b1420 con 0x7f9a3813e290 2023-11-26T07:23:59.051 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:59.047+0000 7f9a3e3f8640 1 -- 172.21.15.143:0/1379639167 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9a3815de20 con 0x7f9a3813e290 2023-11-26T07:23:59.051 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:59.047+0000 7f9a34ff9640 1 -- 172.21.15.143:0/1379639167 <== mon.0 v2:172.21.15.143:3300/0 4 ==== mgrmap(e 16) v1 ==== 85954+0+0 (secure 0 0 0) 0x7f9a2c00bcf0 con 0x7f9a3813e290 2023-11-26T07:23:59.051 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:59.051+0000 7f9a34ff9640 1 --2- 172.21.15.143:0/1379639167 >> [v2:172.21.15.188:6836/3401128553,v1:172.21.15.188:6837/3401128553] conn(0x7f9a24066590 0x7f9a24068a50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:23:59.051 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:59.051+0000 7f9a34ff9640 1 -- 172.21.15.143:0/1379639167 <== mon.0 v2:172.21.15.143:3300/0 5 ==== osd_map(94..94 src has 1..94) v4 ==== 5029+0+0 (secure 0 0 0) 0x7f9a2c084260 con 0x7f9a3813e290 2023-11-26T07:23:59.052 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:59.051+0000 7f9a37fff640 1 --2- 172.21.15.143:0/1379639167 >> [v2:172.21.15.188:6836/3401128553,v1:172.21.15.188:6837/3401128553] conn(0x7f9a24066590 0x7f9a24068a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:23:59.052 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:59.051+0000 7f9a37fff640 1 --2- 172.21.15.143:0/1379639167 >> [v2:172.21.15.188:6836/3401128553,v1:172.21.15.188:6837/3401128553] conn(0x7f9a24066590 0x7f9a24068a50 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7f9a38071050 tx=0x7f9a28007e40 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:23:59.055 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:59.055+0000 7f9a34ff9640 1 -- 172.21.15.143:0/1379639167 <== mon.0 v2:172.21.15.143:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172649 (secure 0 0 0) 0x7f9a2c051870 con 0x7f9a3813e290 2023-11-26T07:23:59.206 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:59.207+0000 7f9a3e3f8640 1 -- 172.21.15.143:0/1379639167 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "config rm", "who": "mon", "name": "mon_allow_pool_delete"} v 0) v1 -- 0x7f9a38067680 con 0x7f9a3813e290 2023-11-26T07:23:59.210 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:59.211+0000 7f9a34ff9640 1 -- 172.21.15.143:0/1379639167 <== mon.0 v2:172.21.15.143:3300/0 7 ==== mon_command_ack([{"prefix": "config rm", "who": "mon", "name": "mon_allow_pool_delete"}]=0 v19) v1 ==== 104+0+0 (secure 0 0 0) 0x7f9a2c054e90 con 0x7f9a3813e290 2023-11-26T07:23:59.213 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:59.211+0000 7f9a3e3f8640 1 -- 172.21.15.143:0/1379639167 >> [v2:172.21.15.188:6836/3401128553,v1:172.21.15.188:6837/3401128553] conn(0x7f9a24066590 msgr2=0x7f9a24068a50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:23:59.213 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:59.211+0000 7f9a3e3f8640 1 --2- 172.21.15.143:0/1379639167 >> [v2:172.21.15.188:6836/3401128553,v1:172.21.15.188:6837/3401128553] conn(0x7f9a24066590 0x7f9a24068a50 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7f9a38071050 tx=0x7f9a28007e40 comp rx=0 tx=0).stop 2023-11-26T07:23:59.213 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:59.211+0000 7f9a3e3f8640 1 -- 172.21.15.143:0/1379639167 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f9a3813e290 msgr2=0x7f9a381f8e80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:23:59.213 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:59.211+0000 7f9a3e3f8640 1 --2- 172.21.15.143:0/1379639167 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f9a3813e290 0x7f9a381f8e80 secure :-1 s=READY pgs=1258 cs=0 l=1 rev1=1 crypto rx=0x7f9a2c00d800 tx=0x7f9a2c00dcc0 comp rx=0 tx=0).stop 2023-11-26T07:23:59.213 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:59.211+0000 7f9a3e3f8640 1 -- 172.21.15.143:0/1379639167 shutdown_connections 2023-11-26T07:23:59.213 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:59.211+0000 7f9a3e3f8640 1 --2- 172.21.15.143:0/1379639167 >> [v2:172.21.15.188:6836/3401128553,v1:172.21.15.188:6837/3401128553] conn(0x7f9a24066590 0x7f9a24068a50 unknown :-1 s=CLOSED pgs=104 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:23:59.214 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:59.211+0000 7f9a3e3f8640 1 --2- 172.21.15.143:0/1379639167 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f9a3815f4e0 0x7f9a380b0ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:23:59.214 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:59.211+0000 7f9a3e3f8640 1 --2- 172.21.15.143:0/1379639167 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f9a3813e290 0x7f9a381f8e80 unknown :-1 s=CLOSED pgs=1258 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:23:59.214 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:59.211+0000 7f9a3e3f8640 1 --2- 172.21.15.143:0/1379639167 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f9a380cf990 0x7f9a380b3ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:23:59.214 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:59.211+0000 7f9a3e3f8640 1 -- 172.21.15.143:0/1379639167 >> 172.21.15.143:0/1379639167 conn(0x7f9a3806a890 msgr2=0x7f9a380711f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:23:59.214 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:59.211+0000 7f9a3e3f8640 1 -- 172.21.15.143:0/1379639167 shutdown_connections 2023-11-26T07:23:59.214 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:59.211+0000 7f9a3e3f8640 1 -- 172.21.15.143:0/1379639167 wait complete. 2023-11-26T07:23:59.232 DEBUG:teuthology.orchestra.run.smithi143:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Ended test tasks.cephfs.test_volumes.TestMisc.test_names_can_only_be_goodchars' 2023-11-26T07:23:59.370 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:59.371+0000 7f60f0809640 1 Processor -- start 2023-11-26T07:23:59.370 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:59.371+0000 7f60f0809640 1 -- start start 2023-11-26T07:23:59.371 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:59.371+0000 7f60f0809640 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7f60ec143810 0x7f60ec072140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:23:59.372 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:59.371+0000 7f60f0809640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f60ec15e0f0 0x7f60ec0ce030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:23:59.372 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:59.371+0000 7f60f0809640 1 -- --> v1:172.21.15.143:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f60ec15e860 con 0x7f60ec0b1e10 2023-11-26T07:23:59.372 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:59.371+0000 7f60f0809640 1 -- --> v2:172.21.15.188:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f60ec0cdba0 con 0x7f60ec143810 2023-11-26T07:23:59.372 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:59.371+0000 7f60f0809640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f60ec143bf0 con 0x7f60ec15e0f0 2023-11-26T07:23:59.372 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:59.371+0000 7f60eaffd640 1 --1- >> v1:172.21.15.143:6789/0 conn(0x7f60ec0b1e10 0x7f60ec15d9e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.143:6789/0 says I am v1:172.21.15.143:43308/0 (socket says 172.21.15.143:43308) 2023-11-26T07:23:59.372 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:59.371+0000 7f60eaffd640 1 -- 172.21.15.143:0/4206144273 learned_addr learned my addr 172.21.15.143:0/4206144273 (peer_addr_for_me v1:172.21.15.143:0/0) 2023-11-26T07:23:59.374 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:59.371+0000 7f60eb7fe640 1 --2- 172.21.15.143:0/4206144273 >> v2:172.21.15.188:3300/0 conn(0x7f60ec143810 0x7f60ec072140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:23:59.374 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:59.371+0000 7f60ea7fc640 1 -- 172.21.15.143:0/4206144273 <== mon.0 v1:172.21.15.143:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3135005039 0 0) 0x7f60ec15e860 con 0x7f60ec0b1e10 2023-11-26T07:23:59.374 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:59.371+0000 7f60ea7fc640 1 -- 172.21.15.143:0/4206144273 --> v1:172.21.15.143:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f60d4003530 con 0x7f60ec0b1e10 2023-11-26T07:23:59.374 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:59.371+0000 7f60ebfff640 1 --2- 172.21.15.143:0/4206144273 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f60ec15e0f0 0x7f60ec0ce030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:23:59.374 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:59.371+0000 7f60ea7fc640 1 -- 172.21.15.143:0/4206144273 <== mon.0 v1:172.21.15.143:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 4011463267 0 0) 0x7f60d4003530 con 0x7f60ec0b1e10 2023-11-26T07:23:59.374 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:59.371+0000 7f60ea7fc640 1 -- 172.21.15.143:0/4206144273 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f60ec15e0f0 msgr2=0x7f60ec0ce030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-26T07:23:59.374 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:59.371+0000 7f60ea7fc640 1 --2- 172.21.15.143:0/4206144273 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f60ec15e0f0 0x7f60ec0ce030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:23:59.374 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:59.371+0000 7f60ea7fc640 1 -- 172.21.15.143:0/4206144273 >> v2:172.21.15.188:3300/0 conn(0x7f60ec143810 msgr2=0x7f60ec072140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-26T07:23:59.374 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:59.371+0000 7f60ea7fc640 1 --2- 172.21.15.143:0/4206144273 >> v2:172.21.15.188:3300/0 conn(0x7f60ec143810 0x7f60ec072140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:23:59.375 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:59.371+0000 7f60ea7fc640 1 -- 172.21.15.143:0/4206144273 --> v1:172.21.15.143:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f60ec0714f0 con 0x7f60ec0b1e10 2023-11-26T07:23:59.375 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:59.371+0000 7f60ea7fc640 1 -- 172.21.15.143:0/4206144273 <== mon.0 v1:172.21.15.143:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2137882523 0 0) 0x7f60e0001680 con 0x7f60ec0b1e10 2023-11-26T07:23:59.375 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:59.371+0000 7f60ea7fc640 1 -- 172.21.15.143:0/4206144273 >> v1:172.21.15.143:6789/0 conn(0x7f60ec0b1e10 legacy=0x7f60ec15d9e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:23:59.375 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:59.371+0000 7f60ea7fc640 1 --2- 172.21.15.143:0/4206144273 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f60d4003980 0x7f60d4003da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:23:59.375 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:59.371+0000 7f60ea7fc640 1 --2- 172.21.15.143:0/4206144273 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f60d40047d0 0x7f60d4004c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:23:59.375 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:59.371+0000 7f60ea7fc640 1 --2- 172.21.15.143:0/4206144273 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f60ec15e0f0 0x7f60d4005190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:23:59.375 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:59.371+0000 7f60ea7fc640 1 -- 172.21.15.143:0/4206144273 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f60ec0cdba0 con 0x7f60d4003980 2023-11-26T07:23:59.375 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:59.371+0000 7f60ea7fc640 1 -- 172.21.15.143:0/4206144273 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f60ec143bf0 con 0x7f60d40047d0 2023-11-26T07:23:59.375 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:59.371+0000 7f60ea7fc640 1 -- 172.21.15.143:0/4206144273 --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f60d400b990 con 0x7f60ec15e0f0 2023-11-26T07:23:59.375 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:59.371+0000 7f60eb7fe640 1 --2- 172.21.15.143:0/4206144273 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f60d4003980 0x7f60d4003da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:23:59.375 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:59.371+0000 7f60eaffd640 1 --2- 172.21.15.143:0/4206144273 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f60d40047d0 0x7f60d4004c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:23:59.376 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:59.371+0000 7f60ebfff640 1 --2- 172.21.15.143:0/4206144273 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f60ec15e0f0 0x7f60d4005190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:23:59.376 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:59.371+0000 7f60eb7fe640 1 -- 172.21.15.143:0/4206144273 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f60ec15e0f0 msgr2=0x7f60d4005190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-26T07:23:59.376 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:59.371+0000 7f60eb7fe640 1 --2- 172.21.15.143:0/4206144273 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f60ec15e0f0 0x7f60d4005190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:23:59.376 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:59.371+0000 7f60eb7fe640 1 -- 172.21.15.143:0/4206144273 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f60d40047d0 msgr2=0x7f60d4004c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-26T07:23:59.376 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:59.371+0000 7f60eb7fe640 1 --2- 172.21.15.143:0/4206144273 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f60d40047d0 0x7f60d4004c50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:23:59.376 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:59.371+0000 7f60eb7fe640 1 -- 172.21.15.143:0/4206144273 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f60d4003530 con 0x7f60d4003980 2023-11-26T07:23:59.376 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:59.371+0000 7f60ebfff640 1 --2- 172.21.15.143:0/4206144273 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f60ec15e0f0 0x7f60d4005190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-26T07:23:59.377 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:59.375+0000 7f60eb7fe640 1 --2- 172.21.15.143:0/4206144273 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f60d4003980 0x7f60d4003da0 secure :-1 s=READY pgs=1260 cs=0 l=1 rev1=1 crypto rx=0x7f60dc009ae0 tx=0x7f60dc01ca60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cf0b530e24c990e9 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:23:59.377 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:59.375+0000 7f60ea7fc640 1 -- 172.21.15.143:0/4206144273 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f60dc0a5570 con 0x7f60d4003980 2023-11-26T07:23:59.377 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:59.375+0000 7f60ea7fc640 1 -- 172.21.15.143:0/4206144273 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f60dc0a56d0 con 0x7f60d4003980 2023-11-26T07:23:59.377 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:59.375+0000 7f60ea7fc640 1 -- 172.21.15.143:0/4206144273 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f60dc0a5a80 con 0x7f60d4003980 2023-11-26T07:23:59.377 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:59.375+0000 7f60f0809640 1 -- 172.21.15.143:0/4206144273 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f60d4003980 msgr2=0x7f60d4003da0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:23:59.378 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:59.375+0000 7f60f0809640 1 --2- 172.21.15.143:0/4206144273 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f60d4003980 0x7f60d4003da0 secure :-1 s=READY pgs=1260 cs=0 l=1 rev1=1 crypto rx=0x7f60dc009ae0 tx=0x7f60dc01ca60 comp rx=0 tx=0).stop 2023-11-26T07:23:59.378 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:59.375+0000 7f60ebfff640 1 -- 172.21.15.143:0/4206144273 reap_dead start 2023-11-26T07:23:59.378 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:59.375+0000 7f60f0809640 1 -- 172.21.15.143:0/4206144273 shutdown_connections 2023-11-26T07:23:59.378 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:59.375+0000 7f60f0809640 1 -- 172.21.15.143:0/4206144273 >> 172.21.15.143:0/4206144273 conn(0x7f60ec06a890 msgr2=0x7f60ec06aca0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:23:59.378 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:59.375+0000 7f60f0809640 1 -- 172.21.15.143:0/4206144273 shutdown_connections 2023-11-26T07:23:59.378 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:59.375+0000 7f60f0809640 1 -- 172.21.15.143:0/4206144273 wait complete. 2023-11-26T07:23:59.378 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:59.379+0000 7f60f0809640 1 Processor -- start 2023-11-26T07:23:59.379 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:59.379+0000 7f60f0809640 1 -- start start 2023-11-26T07:23:59.380 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:59.379+0000 7f60f0809640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f60d4003980 0x7f60ec1514e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:23:59.380 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:59.379+0000 7f60f0809640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f60ec13d750 0x7f60ec13fb70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:23:59.380 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:59.379+0000 7f60f0809640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f60ec1506d0 0x7f60ec15ca70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:23:59.380 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:59.379+0000 7f60f0809640 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f60ec1400b0 con 0x7f60d4003980 2023-11-26T07:23:59.380 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:59.379+0000 7f60f0809640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f60ec13dbd0 con 0x7f60ec13d750 2023-11-26T07:23:59.381 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:59.379+0000 7f60f0809640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f60ec1402e0 con 0x7f60ec1506d0 2023-11-26T07:23:59.381 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:59.379+0000 7f60eb7fe640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f60d4003980 0x7f60ec1514e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:23:59.381 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:59.379+0000 7f60eb7fe640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f60d4003980 0x7f60ec1514e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:38528/0 (socket says 172.21.15.143:38528) 2023-11-26T07:23:59.381 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:59.379+0000 7f60eb7fe640 1 -- 172.21.15.143:0/1238104883 learned_addr learned my addr 172.21.15.143:0/1238104883 (peer_addr_for_me v2:172.21.15.143:0/0) 2023-11-26T07:23:59.381 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:59.379+0000 7f60eaffd640 1 --2- 172.21.15.143:0/1238104883 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f60ec13d750 0x7f60ec13fb70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:23:59.381 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:59.379+0000 7f60ebfff640 1 --2- 172.21.15.143:0/1238104883 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f60ec1506d0 0x7f60ec15ca70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:23:59.381 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:59.379+0000 7f60eb7fe640 1 -- 172.21.15.143:0/1238104883 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f60ec1506d0 msgr2=0x7f60ec15ca70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:23:59.381 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:59.379+0000 7f60eb7fe640 1 --2- 172.21.15.143:0/1238104883 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f60ec1506d0 0x7f60ec15ca70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:23:59.381 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:59.379+0000 7f60eb7fe640 1 -- 172.21.15.143:0/1238104883 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f60ec13d750 msgr2=0x7f60ec13fb70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:23:59.381 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:59.379+0000 7f60eb7fe640 1 --2- 172.21.15.143:0/1238104883 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f60ec13d750 0x7f60ec13fb70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:23:59.381 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:59.379+0000 7f60eb7fe640 1 -- 172.21.15.143:0/1238104883 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f60ec142660 con 0x7f60d4003980 2023-11-26T07:23:59.382 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:59.379+0000 7f60eb7fe640 1 --2- 172.21.15.143:0/1238104883 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f60d4003980 0x7f60ec1514e0 secure :-1 s=READY pgs=1261 cs=0 l=1 rev1=1 crypto rx=0x7f60dc009ae0 tx=0x7f60d4005800 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:23:59.382 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:59.379+0000 7f60d3fff640 1 -- 172.21.15.143:0/1238104883 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f60dc0a5dc0 con 0x7f60d4003980 2023-11-26T07:23:59.382 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:59.379+0000 7f60d3fff640 1 -- 172.21.15.143:0/1238104883 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f60dc004020 con 0x7f60d4003980 2023-11-26T07:23:59.383 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:59.379+0000 7f60d3fff640 1 -- 172.21.15.143:0/1238104883 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f60dc004390 con 0x7f60d4003980 2023-11-26T07:23:59.383 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:59.379+0000 7f60f0809640 1 -- 172.21.15.143:0/1238104883 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f60ec142940 con 0x7f60d4003980 2023-11-26T07:23:59.383 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:59.379+0000 7f60f0809640 1 -- 172.21.15.143:0/1238104883 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f60ec140b10 con 0x7f60d4003980 2023-11-26T07:23:59.383 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:59.383+0000 7f60f0809640 1 -- 172.21.15.143:0/1238104883 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f60ec067500 con 0x7f60d4003980 2023-11-26T07:23:59.383 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:59.383+0000 7f60d3fff640 1 -- 172.21.15.143:0/1238104883 <== mon.0 v2:172.21.15.143:3300/0 4 ==== mgrmap(e 16) v1 ==== 85954+0+0 (secure 0 0 0) 0x7f60dc0b4040 con 0x7f60d4003980 2023-11-26T07:23:59.383 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:59.383+0000 7f60d3fff640 1 --2- 172.21.15.143:0/1238104883 >> [v2:172.21.15.188:6836/3401128553,v1:172.21.15.188:6837/3401128553] conn(0x7f60d80665c0 0x7f60d8068a80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:23:59.384 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:59.383+0000 7f60d3fff640 1 -- 172.21.15.143:0/1238104883 <== mon.0 v2:172.21.15.143:3300/0 5 ==== osd_map(94..94 src has 1..94) v4 ==== 5029+0+0 (secure 0 0 0) 0x7f60dc11dbb0 con 0x7f60d4003980 2023-11-26T07:23:59.384 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:59.383+0000 7f60eaffd640 1 --2- 172.21.15.143:0/1238104883 >> [v2:172.21.15.188:6836/3401128553,v1:172.21.15.188:6837/3401128553] conn(0x7f60d80665c0 0x7f60d8068a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:23:59.384 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:59.383+0000 7f60eaffd640 1 --2- 172.21.15.143:0/1238104883 >> [v2:172.21.15.188:6836/3401128553,v1:172.21.15.188:6837/3401128553] conn(0x7f60d80665c0 0x7f60d8068a80 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7f60ec0b3820 tx=0x7f60e0007040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:23:59.387 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:59.387+0000 7f60d3fff640 1 -- 172.21.15.143:0/1238104883 <== mon.0 v2:172.21.15.143:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172649 (secure 0 0 0) 0x7f60dc0eb270 con 0x7f60d4003980 2023-11-26T07:23:59.590 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:59.587+0000 7f60f0809640 1 -- 172.21.15.143:0/1238104883 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_volumes.TestMisc.test_names_can_only_be_goodchars"]} v 0) v1 -- 0x7f60ec15e690 con 0x7f60d4003980 2023-11-26T07:23:59.867 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:59.867+0000 7f60d3fff640 1 -- 172.21.15.143:0/1238104883 <== mon.0 v2:172.21.15.143:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_volumes.TestMisc.test_names_can_only_be_goodchars"]}]=0 v326) v1 ==== 146+0+0 (secure 0 0 0) 0x7f60dc0ee890 con 0x7f60d4003980 2023-11-26T07:23:59.869 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:59.867+0000 7f60f0809640 1 -- 172.21.15.143:0/1238104883 >> [v2:172.21.15.188:6836/3401128553,v1:172.21.15.188:6837/3401128553] conn(0x7f60d80665c0 msgr2=0x7f60d8068a80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:23:59.870 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:59.867+0000 7f60f0809640 1 --2- 172.21.15.143:0/1238104883 >> [v2:172.21.15.188:6836/3401128553,v1:172.21.15.188:6837/3401128553] conn(0x7f60d80665c0 0x7f60d8068a80 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7f60ec0b3820 tx=0x7f60e0007040 comp rx=0 tx=0).stop 2023-11-26T07:23:59.870 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:59.867+0000 7f60f0809640 1 -- 172.21.15.143:0/1238104883 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f60d4003980 msgr2=0x7f60ec1514e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:23:59.870 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:59.867+0000 7f60f0809640 1 --2- 172.21.15.143:0/1238104883 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f60d4003980 0x7f60ec1514e0 secure :-1 s=READY pgs=1261 cs=0 l=1 rev1=1 crypto rx=0x7f60dc009ae0 tx=0x7f60d4005800 comp rx=0 tx=0).stop 2023-11-26T07:23:59.870 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:59.867+0000 7f60f0809640 1 -- 172.21.15.143:0/1238104883 shutdown_connections 2023-11-26T07:23:59.870 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:59.867+0000 7f60f0809640 1 --2- 172.21.15.143:0/1238104883 >> [v2:172.21.15.188:6836/3401128553,v1:172.21.15.188:6837/3401128553] conn(0x7f60d80665c0 0x7f60d8068a80 unknown :-1 s=CLOSED pgs=105 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:23:59.870 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:59.867+0000 7f60f0809640 1 --2- 172.21.15.143:0/1238104883 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f60ec1506d0 0x7f60ec15ca70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:23:59.870 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:59.867+0000 7f60f0809640 1 --2- 172.21.15.143:0/1238104883 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f60ec13d750 0x7f60ec13fb70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:23:59.870 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:59.867+0000 7f60f0809640 1 --2- 172.21.15.143:0/1238104883 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f60d4003980 0x7f60ec1514e0 unknown :-1 s=CLOSED pgs=1261 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:23:59.870 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:59.867+0000 7f60f0809640 1 -- 172.21.15.143:0/1238104883 >> 172.21.15.143:0/1238104883 conn(0x7f60ec06a890 msgr2=0x7f60ec0706e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:23:59.870 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:59.867+0000 7f60f0809640 1 -- 172.21.15.143:0/1238104883 shutdown_connections 2023-11-26T07:23:59.870 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:23:59.867+0000 7f60f0809640 1 -- 172.21.15.143:0/1238104883 wait complete. 2023-11-26T07:23:59.890 INFO:tasks.cephfs_test_runner:Test the creating vols, subvols subvolgroups fails when their names uses ... ok 2023-11-26T07:23:59.890 INFO:tasks.cephfs_test_runner:Starting test: test_subvolume_no_upgrade_v1_sanity (tasks.cephfs.test_volumes.TestMisc) poor man's upgrade test -- theme continues... 2023-11-26T07:23:59.890 INFO:tasks.cephfs_test_runner:test_subvolume_no_upgrade_v1_sanity (tasks.cephfs.test_volumes.TestMisc) 2023-11-26T07:23:59.890 DEBUG:teuthology.orchestra.run.smithi143:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-11-26T07:24:00.028 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:00.027+0000 7ff073679640 1 Processor -- start 2023-11-26T07:24:00.028 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:00.027+0000 7ff073679640 1 -- start start 2023-11-26T07:24:00.029 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:00.027+0000 7ff073679640 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7ff06c1614b0 0x7ff06c0cd610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:24:00.029 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:00.027+0000 7ff073679640 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7ff06c070ba0 0x7ff06c070fd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:24:00.030 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:00.027+0000 7ff073679640 1 -- --> v1:172.21.15.143:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff06c071510 con 0x7ff06c071970 2023-11-26T07:24:00.030 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:00.027+0000 7ff073679640 1 -- --> v2:172.21.15.143:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff06c15d970 con 0x7ff06c070ba0 2023-11-26T07:24:00.030 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:00.027+0000 7ff073679640 1 -- --> v2:172.21.15.188:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff06c15dc30 con 0x7ff06c1614b0 2023-11-26T07:24:00.030 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:00.027+0000 7ff070c14640 1 --1- >> v1:172.21.15.143:6789/0 conn(0x7ff06c071970 0x7ff06c161890 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.143:6789/0 says I am v1:172.21.15.143:43318/0 (socket says 172.21.15.143:43318) 2023-11-26T07:24:00.030 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:00.027+0000 7ff070c14640 1 -- 172.21.15.143:0/1390222320 learned_addr learned my addr 172.21.15.143:0/1390222320 (peer_addr_for_me v1:172.21.15.143:0/0) 2023-11-26T07:24:00.030 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:00.027+0000 7ff071c16640 1 --2- 172.21.15.143:0/1390222320 >> v2:172.21.15.143:3300/0 conn(0x7ff06c070ba0 0x7ff06c070fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:24:00.030 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:00.027+0000 7ff071415640 1 --2- 172.21.15.143:0/1390222320 >> v2:172.21.15.188:3300/0 conn(0x7ff06c1614b0 0x7ff06c0cd610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:24:00.031 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:00.027+0000 7ff063fff640 1 -- 172.21.15.143:0/1390222320 <== mon.0 v1:172.21.15.143:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1134037901 0 0) 0x7ff06c071510 con 0x7ff06c071970 2023-11-26T07:24:00.031 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:00.027+0000 7ff063fff640 1 -- 172.21.15.143:0/1390222320 --> v1:172.21.15.143:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7ff058003530 con 0x7ff06c071970 2023-11-26T07:24:00.031 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:00.031+0000 7ff071c16640 1 -- 172.21.15.143:0/1390222320 >> v2:172.21.15.188:3300/0 conn(0x7ff06c1614b0 msgr2=0x7ff06c0cd610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-26T07:24:00.031 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:00.031+0000 7ff071c16640 1 --2- 172.21.15.143:0/1390222320 >> v2:172.21.15.188:3300/0 conn(0x7ff06c1614b0 0x7ff06c0cd610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:24:00.031 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:00.031+0000 7ff071c16640 1 -- 172.21.15.143:0/1390222320 >> v1:172.21.15.143:6789/0 conn(0x7ff06c071970 legacy=0x7ff06c161890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:24:00.031 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:00.031+0000 7ff071c16640 1 -- 172.21.15.143:0/1390222320 --> v2:172.21.15.143:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff06c15df10 con 0x7ff06c070ba0 2023-11-26T07:24:00.033 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:00.031+0000 7ff071c16640 1 --2- 172.21.15.143:0/1390222320 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7ff06c070ba0 0x7ff06c070fd0 secure :-1 s=READY pgs=1263 cs=0 l=1 rev1=1 crypto rx=0x7ff06c067300 tx=0x7ff06401c8e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cc49456a8bb6f252 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:24:00.033 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:00.031+0000 7ff063fff640 1 -- 172.21.15.143:0/1390222320 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff0640a59a0 con 0x7ff06c070ba0 2023-11-26T07:24:00.033 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:00.031+0000 7ff063fff640 1 -- 172.21.15.143:0/1390222320 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff0640a5b00 con 0x7ff06c070ba0 2023-11-26T07:24:00.033 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:00.031+0000 7ff063fff640 1 -- 172.21.15.143:0/1390222320 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff0640a5e90 con 0x7ff06c070ba0 2023-11-26T07:24:00.034 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:00.031+0000 7ff073679640 1 -- 172.21.15.143:0/1390222320 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7ff06c070ba0 msgr2=0x7ff06c070fd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:24:00.034 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:00.031+0000 7ff073679640 1 --2- 172.21.15.143:0/1390222320 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7ff06c070ba0 0x7ff06c070fd0 secure :-1 s=READY pgs=1263 cs=0 l=1 rev1=1 crypto rx=0x7ff06c067300 tx=0x7ff06401c8e0 comp rx=0 tx=0).stop 2023-11-26T07:24:00.034 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:00.031+0000 7ff073679640 1 -- 172.21.15.143:0/1390222320 shutdown_connections 2023-11-26T07:24:00.034 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:00.031+0000 7ff073679640 1 --2- 172.21.15.143:0/1390222320 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7ff06c070ba0 0x7ff06c070fd0 unknown :-1 s=CLOSED pgs=1263 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:24:00.034 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:00.031+0000 7ff073679640 1 --2- 172.21.15.143:0/1390222320 >> v2:172.21.15.188:3300/0 conn(0x7ff06c1614b0 0x7ff06c0cd610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:24:00.035 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:00.031+0000 7ff073679640 1 -- 172.21.15.143:0/1390222320 >> 172.21.15.143:0/1390222320 conn(0x7ff06c0cf710 msgr2=0x7ff06c0cfb20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:24:00.035 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:00.031+0000 7ff073679640 1 -- 172.21.15.143:0/1390222320 shutdown_connections 2023-11-26T07:24:00.035 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:00.035+0000 7ff073679640 1 -- 172.21.15.143:0/1390222320 wait complete. 2023-11-26T07:24:00.035 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:00.035+0000 7ff073679640 1 Processor -- start 2023-11-26T07:24:00.035 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:00.035+0000 7ff073679640 1 -- start start 2023-11-26T07:24:00.038 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:00.035+0000 7ff073679640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7ff06c070ba0 0x7ff06c14edd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:24:00.038 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:00.035+0000 7ff073679640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7ff06c071970 0x7ff06c0b4000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:24:00.038 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:00.035+0000 7ff073679640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7ff06c1614b0 0x7ff06c15f4b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:24:00.038 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:00.035+0000 7ff073679640 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff06c0adc10 con 0x7ff06c070ba0 2023-11-26T07:24:00.038 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:00.035+0000 7ff073679640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff06c0adef0 con 0x7ff06c071970 2023-11-26T07:24:00.038 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:00.035+0000 7ff073679640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff06c0ae1d0 con 0x7ff06c1614b0 2023-11-26T07:24:00.039 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:00.035+0000 7ff071415640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7ff06c070ba0 0x7ff06c14edd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:24:00.039 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:00.035+0000 7ff071415640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7ff06c070ba0 0x7ff06c14edd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:38544/0 (socket says 172.21.15.143:38544) 2023-11-26T07:24:00.039 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:00.035+0000 7ff071415640 1 -- 172.21.15.143:0/3682221590 learned_addr learned my addr 172.21.15.143:0/3682221590 (peer_addr_for_me v2:172.21.15.143:0/0) 2023-11-26T07:24:00.039 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:00.035+0000 7ff070c14640 1 --2- 172.21.15.143:0/3682221590 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7ff06c071970 0x7ff06c0b4000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:24:00.039 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:00.035+0000 7ff071c16640 1 --2- 172.21.15.143:0/3682221590 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7ff06c1614b0 0x7ff06c15f4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:24:00.039 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:00.035+0000 7ff071415640 1 -- 172.21.15.143:0/3682221590 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7ff06c1614b0 msgr2=0x7ff06c15f4b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:24:00.039 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:00.035+0000 7ff071415640 1 --2- 172.21.15.143:0/3682221590 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7ff06c1614b0 0x7ff06c15f4b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:24:00.039 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:00.035+0000 7ff071415640 1 -- 172.21.15.143:0/3682221590 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7ff06c071970 msgr2=0x7ff06c0b4000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:24:00.040 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:00.035+0000 7ff071415640 1 --2- 172.21.15.143:0/3682221590 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7ff06c071970 0x7ff06c0b4000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:24:00.040 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:00.035+0000 7ff071415640 1 -- 172.21.15.143:0/3682221590 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff064009a70 con 0x7ff06c070ba0 2023-11-26T07:24:00.040 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:00.035+0000 7ff071415640 1 --2- 172.21.15.143:0/3682221590 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7ff06c070ba0 0x7ff06c14edd0 secure :-1 s=READY pgs=1264 cs=0 l=1 rev1=1 crypto rx=0x7ff06800e930 tx=0x7ff06800edf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:24:00.040 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:00.035+0000 7ff061ffb640 1 -- 172.21.15.143:0/3682221590 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff068009690 con 0x7ff06c070ba0 2023-11-26T07:24:00.040 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:00.035+0000 7ff061ffb640 1 -- 172.21.15.143:0/3682221590 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff06800ca00 con 0x7ff06c070ba0 2023-11-26T07:24:00.040 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:00.035+0000 7ff061ffb640 1 -- 172.21.15.143:0/3682221590 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff06800cd70 con 0x7ff06c070ba0 2023-11-26T07:24:00.040 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:00.035+0000 7ff073679640 1 -- 172.21.15.143:0/3682221590 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff06c0ae4b0 con 0x7ff06c070ba0 2023-11-26T07:24:00.040 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:00.039+0000 7ff073679640 1 -- 172.21.15.143:0/3682221590 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff06c0b2350 con 0x7ff06c070ba0 2023-11-26T07:24:00.041 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:00.039+0000 7ff073679640 1 -- 172.21.15.143:0/3682221590 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff06c067680 con 0x7ff06c070ba0 2023-11-26T07:24:00.041 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:00.039+0000 7ff061ffb640 1 -- 172.21.15.143:0/3682221590 <== mon.0 v2:172.21.15.143:3300/0 4 ==== mgrmap(e 16) v1 ==== 85954+0+0 (secure 0 0 0) 0x7ff068013470 con 0x7ff06c070ba0 2023-11-26T07:24:00.041 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:00.039+0000 7ff061ffb640 1 --2- 172.21.15.143:0/3682221590 >> [v2:172.21.15.188:6836/3401128553,v1:172.21.15.188:6837/3401128553] conn(0x7ff050066590 0x7ff050068a50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:24:00.042 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:00.039+0000 7ff061ffb640 1 -- 172.21.15.143:0/3682221590 <== mon.0 v2:172.21.15.143:3300/0 5 ==== osd_map(94..94 src has 1..94) v4 ==== 5029+0+0 (secure 0 0 0) 0x7ff068083ef0 con 0x7ff06c070ba0 2023-11-26T07:24:00.042 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:00.039+0000 7ff070c14640 1 --2- 172.21.15.143:0/3682221590 >> [v2:172.21.15.188:6836/3401128553,v1:172.21.15.188:6837/3401128553] conn(0x7ff050066590 0x7ff050068a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:24:00.042 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:00.039+0000 7ff070c14640 1 --2- 172.21.15.143:0/3682221590 >> [v2:172.21.15.188:6836/3401128553,v1:172.21.15.188:6837/3401128553] conn(0x7ff050066590 0x7ff050068a50 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7ff06c157cc0 tx=0x7ff05c009040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:24:00.044 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:00.043+0000 7ff061ffb640 1 -- 172.21.15.143:0/3682221590 <== mon.0 v2:172.21.15.143:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172649 (secure 0 0 0) 0x7ff068051660 con 0x7ff06c070ba0 2023-11-26T07:24:00.197 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:00.195+0000 7ff073679640 1 -- 172.21.15.143:0/3682221590 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7ff06c06ce30 con 0x7ff06c070ba0 2023-11-26T07:24:00.197 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:00.195+0000 7ff061ffb640 1 -- 172.21.15.143:0/3682221590 <== mon.0 v2:172.21.15.143:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v94) v1 ==== 74+0+13155 (secure 0 0 0) 0x7ff068017090 con 0x7ff06c070ba0 2023-11-26T07:24:00.198 INFO:teuthology.orchestra.run.smithi143.stdout: 2023-11-26T07:24:00.198 INFO:teuthology.orchestra.run.smithi143.stdout:{"epoch":94,"fsid":"f5033060-2469-4a07-95fe-2b76f5404bc1","created":"2023-11-26T07:17:28.613663+0000","modified":"2023-11-26T07:23:58.858835+0000","last_up_change":"2023-11-26T07:17:41.157813+0000","last_in_change":"2023-11-26T07:17:31.181658+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":15,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-11-26T07:17:41.758597+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"ebbee2c6-023e-44b9-9f1d-6f7bfd309730","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6817","nonce":639807369},{"type":"v1","addr":"172.21.15.143:6819","nonce":639807369}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6821","nonce":639807369},{"type":"v1","addr":"172.21.15.143:6823","nonce":639807369}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6830","nonce":639807369},{"type":"v1","addr":"172.21.15.143:6831","nonce":639807369}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6825","nonce":639807369},{"type":"v1","addr":"172.21.15.143:6828","nonce":639807369}]},"public_addr":"172.21.15.143:6819/639807369","cluster_addr":"172.21.15.143:6823/639807369","heartbeat_back_addr":"172.21.15.143:6831/639807369","heartbeat_front_addr":"172.21.15.143:6828/639807369","state":["exists","up"]},{"osd":1,"uuid":"fec4b8be-7122-49e9-9a19-5850a0700001","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6816","nonce":1144942530},{"type":"v1","addr":"172.21.15.143:6818","nonce":1144942530}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6820","nonce":1144942530},{"type":"v1","addr":"172.21.15.143:6822","nonce":1144942530}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6827","nonce":1144942530},{"type":"v1","addr":"172.21.15.143:6829","nonce":1144942530}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6824","nonce":1144942530},{"type":"v1","addr":"172.21.15.143:6826","nonce":1144942530}]},"public_addr":"172.21.15.143:6818/1144942530","cluster_addr":"172.21.15.143:6822/1144942530","heartbeat_back_addr":"172.21.15.143:6829/1144942530","heartbeat_front_addr":"172.21.15.143:6826/1144942530","state":["exists","up"]},{"osd":2,"uuid":"2888af5b-f933-46ca-8f95-6dd494ff43f6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6808","nonce":3909095186},{"type":"v1","addr":"172.21.15.143:6809","nonce":3909095186}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6810","nonce":3909095186},{"type":"v1","addr":"172.21.15.143:6811","nonce":3909095186}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6814","nonce":3909095186},{"type":"v1","addr":"172.21.15.143:6815","nonce":3909095186}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6812","nonce":3909095186},{"type":"v1","addr":"172.21.15.143:6813","nonce":3909095186}]},"public_addr":"172.21.15.143:6809/3909095186","cluster_addr":"172.21.15.143:6811/3909095186","heartbeat_back_addr":"172.21.15.143:6815/3909095186","heartbeat_front_addr":"172.21.15.143:6813/3909095186","state":["exists","up"]},{"osd":3,"uuid":"d1f52435-7bc1-4081-9550-9cd1ba55f549","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6800","nonce":3854210991},{"type":"v1","addr":"172.21.15.143:6801","nonce":3854210991}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6802","nonce":3854210991},{"type":"v1","addr":"172.21.15.143:6803","nonce":3854210991}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6806","nonce":3854210991},{"type":"v1","addr":"172.21.15.143:6807","nonce":3854210991}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6804","nonce":3854210991},{"type":"v1","addr":"172.21.15.143:6805","nonce":3854210991}]},"public_addr":"172.21.15.143:6801/3854210991","cluster_addr":"172.21.15.143:6803/3854210991","heartbeat_back_addr":"172.21.15.143:6807/3854210991","heartbeat_front_addr":"172.21.15.143:6805/3854210991","state":["exists","up"]},{"osd":4,"uuid":"e2ae8b6e-6e25-49b2-bcbf-aacce68f06a3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6800","nonce":3924606902},{"type":"v1","addr":"172.21.15.188:6801","nonce":3924606902}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6802","nonce":3924606902},{"type":"v1","addr":"172.21.15.188:6803","nonce":3924606902}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6806","nonce":3924606902},{"type":"v1","addr":"172.21.15.188:6807","nonce":3924606902}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6804","nonce":3924606902},{"type":"v1","addr":"172.21.15.188:6805","nonce":3924606902}]},"public_addr":"172.21.15.188:6801/3924606902","cluster_addr":"172.21.15.188:6803/3924606902","heartbeat_back_addr":"172.21.15.188:6807/3924606902","heartbeat_front_addr":"172.21.15.188:6805/3924606902","state":["exists","up"]},{"osd":5,"uuid":"3cec8a32-7698-4c36-8e89-e4ba11934f33","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6813","nonce":186841544},{"type":"v1","addr":"172.21.15.188:6815","nonce":186841544}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6817","nonce":186841544},{"type":"v1","addr":"172.21.15.188:6818","nonce":186841544}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6822","nonce":186841544},{"type":"v1","addr":"172.21.15.188:6823","nonce":186841544}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6820","nonce":186841544},{"type":"v1","addr":"172.21.15.188:6821","nonce":186841544}]},"public_addr":"172.21.15.188:6815/186841544","cluster_addr":"172.21.15.188:6818/186841544","heartbeat_back_addr":"172.21.15.188:6823/186841544","heartbeat_front_addr":"172.21.15.188:6821/186841544","state":["exists","up"]},{"osd":6,"uuid":"58a65d5d-e090-473a-81b4-ad38e85029f9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6808","nonce":2153499405},{"type":"v1","addr":"172.21.15.188:6809","nonce":2153499405}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6810","nonce":2153499405},{"type":"v1","addr":"172.21.15.188:6811","nonce":2153499405}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6816","nonce":2153499405},{"type":"v1","addr":"172.21.15.188:6819","nonce":2153499405}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6812","nonce":2153499405},{"type":"v1","addr":"172.21.15.188:6814","nonce":2153499405}]},"public_addr":"172.21.15.188:6809/2153499405","cluster_addr":"172.21.15.188:6811/2153499405","heartbeat_back_addr":"172.21.15.188:6819/2153499405","heartbeat_front_addr":"172.21.15.188:6814/2153499405","state":["exists","up"]},{"osd":7,"uuid":"ea5b7f70-3554-4cf8-ac30-f7c59b80678d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6824","nonce":2722096729},{"type":"v1","addr":"172.21.15.188:6825","nonce":2722096729}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6826","nonce":2722096729},{"type":"v1","addr":"172.21.15.188:6827","nonce":2722096729}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6830","nonce":2722096729},{"type":"v1","addr":"172.21.15.188:6831","nonce":2722096729}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6828","nonce":2722096729},{"type":"v1","addr":"172.21.15.188:6829","nonce":2722096729}]},"public_addr":"172.21.15.188:6825/2722096729","cluster_addr":"172.21.15.188:6827/2722096729","heartbeat_back_addr":"172.21.15.188:6831/2722096729","heartbeat_front_addr":"172.21.15.188:6829/2722096729","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-26T07:17:36.995032+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-26T07:17:36.932266+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-26T07:17:36.829304+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-26T07:17:36.804297+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-26T07:17:36.667892+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-26T07:17:36.964868+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-26T07:17:36.827332+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-26T07:17:36.893179+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.188:6835/278875865":"2023-11-27T07:23:55.603074+0000","172.21.15.188:6833/278875865":"2023-11-27T07:23:55.603074+0000","172.21.15.143:6835/4031724325":"2023-11-27T07:23:44.794394+0000","172.21.15.143:6833/4031724325":"2023-11-27T07:23:44.794394+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-11-26T07:24:00.200 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:00.199+0000 7ff073679640 1 -- 172.21.15.143:0/3682221590 >> [v2:172.21.15.188:6836/3401128553,v1:172.21.15.188:6837/3401128553] conn(0x7ff050066590 msgr2=0x7ff050068a50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:24:00.200 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:00.199+0000 7ff073679640 1 --2- 172.21.15.143:0/3682221590 >> [v2:172.21.15.188:6836/3401128553,v1:172.21.15.188:6837/3401128553] conn(0x7ff050066590 0x7ff050068a50 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7ff06c157cc0 tx=0x7ff05c009040 comp rx=0 tx=0).stop 2023-11-26T07:24:00.200 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:00.199+0000 7ff073679640 1 -- 172.21.15.143:0/3682221590 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7ff06c070ba0 msgr2=0x7ff06c14edd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:24:00.200 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:00.199+0000 7ff073679640 1 --2- 172.21.15.143:0/3682221590 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7ff06c070ba0 0x7ff06c14edd0 secure :-1 s=READY pgs=1264 cs=0 l=1 rev1=1 crypto rx=0x7ff06800e930 tx=0x7ff06800edf0 comp rx=0 tx=0).stop 2023-11-26T07:24:00.200 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:00.199+0000 7ff073679640 1 -- 172.21.15.143:0/3682221590 shutdown_connections 2023-11-26T07:24:00.200 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:00.199+0000 7ff073679640 1 --2- 172.21.15.143:0/3682221590 >> [v2:172.21.15.188:6836/3401128553,v1:172.21.15.188:6837/3401128553] conn(0x7ff050066590 0x7ff050068a50 unknown :-1 s=CLOSED pgs=106 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:24:00.201 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:00.199+0000 7ff073679640 1 --2- 172.21.15.143:0/3682221590 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7ff06c1614b0 0x7ff06c15f4b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:24:00.201 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:00.199+0000 7ff073679640 1 --2- 172.21.15.143:0/3682221590 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7ff06c071970 0x7ff06c0b4000 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:24:00.201 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:00.199+0000 7ff073679640 1 --2- 172.21.15.143:0/3682221590 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7ff06c070ba0 0x7ff06c14edd0 unknown :-1 s=CLOSED pgs=1264 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:24:00.201 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:00.199+0000 7ff073679640 1 -- 172.21.15.143:0/3682221590 >> 172.21.15.143:0/3682221590 conn(0x7ff06c0cf710 msgr2=0x7ff06c070f80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:24:00.201 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:00.199+0000 7ff073679640 1 -- 172.21.15.143:0/3682221590 shutdown_connections 2023-11-26T07:24:00.201 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:00.199+0000 7ff073679640 1 -- 172.21.15.143:0/3682221590 wait complete. 2023-11-26T07:24:00.219 INFO:tasks.ceph_test_case.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-11-26T07:17:41.758597+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2023-11-26T07:24:00.220 DEBUG:teuthology.orchestra.run.smithi143:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-11-26T07:24:00.360 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:00.359+0000 7fb18fbaa640 1 Processor -- start 2023-11-26T07:24:00.360 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:00.359+0000 7fb18fbaa640 1 -- start start 2023-11-26T07:24:00.361 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:00.359+0000 7fb18fbaa640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fb188070d10 0x7fb18815d900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:24:00.361 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:00.359+0000 7fb18fbaa640 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7fb18806cd50 0x7fb1880b2180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:24:00.362 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:00.359+0000 7fb18fbaa640 1 -- --> v1:172.21.15.188:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb18815b2f0 con 0x7fb1880cf280 2023-11-26T07:24:00.362 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:00.359+0000 7fb18fbaa640 1 -- --> v2:172.21.15.188:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb18806c5c0 con 0x7fb18806cd50 2023-11-26T07:24:00.362 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:00.359+0000 7fb18fbaa640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb18815e1a0 con 0x7fb188070d10 2023-11-26T07:24:00.363 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:00.363+0000 7fb18d145640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fb188070d10 0x7fb18815d900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:24:00.363 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:00.363+0000 7fb18e147640 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7fb18806cd50 0x7fb1880b2180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:24:00.363 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:00.363+0000 7fb18d145640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fb188070d10 0x7fb18815d900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.188:3301/0 says I am v2:172.21.15.143:56956/0 (socket says 172.21.15.143:56956) 2023-11-26T07:24:00.363 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:00.363+0000 7fb18d145640 1 -- 172.21.15.143:0/1773433828 learned_addr learned my addr 172.21.15.143:0/1773433828 (peer_addr_for_me v2:172.21.15.143:0/0) 2023-11-26T07:24:00.363 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:00.363+0000 7fb18d145640 1 -- 172.21.15.143:0/1773433828 >> v2:172.21.15.188:3300/0 conn(0x7fb18806cd50 msgr2=0x7fb1880b2180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-26T07:24:00.363 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:00.363+0000 7fb18d145640 1 --2- 172.21.15.143:0/1773433828 >> v2:172.21.15.188:3300/0 conn(0x7fb18806cd50 0x7fb1880b2180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:24:00.363 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:00.363+0000 7fb18d145640 1 -- 172.21.15.143:0/1773433828 >> v1:172.21.15.188:6789/0 conn(0x7fb1880cf280 legacy=0x7fb1880cf660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:24:00.363 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:00.363+0000 7fb18d145640 1 -- 172.21.15.143:0/1773433828 --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb18815e480 con 0x7fb188070d10 2023-11-26T07:24:00.363 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:00.363+0000 7fb18e147640 1 --2- 172.21.15.143:0/1773433828 >> v2:172.21.15.188:3300/0 conn(0x7fb18806cd50 0x7fb1880b2180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-26T07:24:00.365 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:00.363+0000 7fb18d145640 1 --2- 172.21.15.143:0/1773433828 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fb188070d10 0x7fb18815d900 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7fb178009870 tx=0x7fb17801ed30 comp rx=0 tx=0).ready entity=mon.2 client_cookie=d6be8fd896b4cad8 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:24:00.365 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:00.363+0000 7fb18c944640 1 -- 172.21.15.143:0/1773433828 <== mon.2 v2:172.21.15.188:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb1780a8a30 con 0x7fb188070d10 2023-11-26T07:24:00.366 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:00.363+0000 7fb18c944640 1 -- 172.21.15.143:0/1773433828 <== mon.2 v2:172.21.15.188:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb1780a8b90 con 0x7fb188070d10 2023-11-26T07:24:00.366 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:00.363+0000 7fb18c944640 1 -- 172.21.15.143:0/1773433828 <== mon.2 v2:172.21.15.188:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb1780b0440 con 0x7fb188070d10 2023-11-26T07:24:00.366 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:00.367+0000 7fb18fbaa640 1 -- 172.21.15.143:0/1773433828 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fb188070d10 msgr2=0x7fb18815d900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:24:00.366 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:00.367+0000 7fb18fbaa640 1 --2- 172.21.15.143:0/1773433828 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fb188070d10 0x7fb18815d900 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7fb178009870 tx=0x7fb17801ed30 comp rx=0 tx=0).stop 2023-11-26T07:24:00.367 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:00.367+0000 7fb18fbaa640 1 -- 172.21.15.143:0/1773433828 shutdown_connections 2023-11-26T07:24:00.367 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:00.367+0000 7fb18fbaa640 1 --2- 172.21.15.143:0/1773433828 >> v2:172.21.15.188:3300/0 conn(0x7fb18806cd50 0x7fb1880b2180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:24:00.367 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:00.367+0000 7fb18fbaa640 1 --2- 172.21.15.143:0/1773433828 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fb188070d10 0x7fb18815d900 unknown :-1 s=CLOSED pgs=152 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:24:00.367 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:00.367+0000 7fb18fbaa640 1 -- 172.21.15.143:0/1773433828 >> 172.21.15.143:0/1773433828 conn(0x7fb18806a890 msgr2=0x7fb18806aca0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:24:00.367 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:00.367+0000 7fb18fbaa640 1 -- 172.21.15.143:0/1773433828 shutdown_connections 2023-11-26T07:24:00.367 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:00.367+0000 7fb18fbaa640 1 -- 172.21.15.143:0/1773433828 wait complete. 2023-11-26T07:24:00.368 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:00.367+0000 7fb18fbaa640 1 Processor -- start 2023-11-26T07:24:00.368 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:00.367+0000 7fb18fbaa640 1 -- start start 2023-11-26T07:24:00.369 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:00.367+0000 7fb18fbaa640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fb18806cd50 0x7fb1881479c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:24:00.369 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:00.367+0000 7fb18fbaa640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fb188070d10 0x7fb188147280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:24:00.369 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:00.367+0000 7fb18fbaa640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fb1880cf280 0x7fb18814a460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:24:00.370 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:00.367+0000 7fb18fbaa640 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb18814a9a0 con 0x7fb18806cd50 2023-11-26T07:24:00.370 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:00.367+0000 7fb18fbaa640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb188148bb0 con 0x7fb1880cf280 2023-11-26T07:24:00.370 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:00.367+0000 7fb18fbaa640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb188147fa0 con 0x7fb188070d10 2023-11-26T07:24:00.370 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:00.367+0000 7fb18d946640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fb18806cd50 0x7fb1881479c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:24:00.370 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:00.367+0000 7fb18d946640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fb18806cd50 0x7fb1881479c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:38552/0 (socket says 172.21.15.143:38552) 2023-11-26T07:24:00.370 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:00.367+0000 7fb18d946640 1 -- 172.21.15.143:0/2319807736 learned_addr learned my addr 172.21.15.143:0/2319807736 (peer_addr_for_me v2:172.21.15.143:0/0) 2023-11-26T07:24:00.370 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:00.367+0000 7fb18e147640 1 --2- 172.21.15.143:0/2319807736 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fb1880cf280 0x7fb18814a460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:24:00.370 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:00.367+0000 7fb18d145640 1 --2- 172.21.15.143:0/2319807736 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fb188070d10 0x7fb188147280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:24:00.371 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:00.367+0000 7fb18d946640 1 -- 172.21.15.143:0/2319807736 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fb188070d10 msgr2=0x7fb188147280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:24:00.371 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:00.367+0000 7fb18d946640 1 --2- 172.21.15.143:0/2319807736 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fb188070d10 0x7fb188147280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:24:00.371 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:00.367+0000 7fb18d946640 1 -- 172.21.15.143:0/2319807736 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fb1880cf280 msgr2=0x7fb18814a460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:24:00.372 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:00.367+0000 7fb18d946640 1 --2- 172.21.15.143:0/2319807736 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fb1880cf280 0x7fb18814a460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:24:00.372 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:00.367+0000 7fb18d946640 1 -- 172.21.15.143:0/2319807736 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb180009a70 con 0x7fb18806cd50 2023-11-26T07:24:00.372 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:00.371+0000 7fb18d946640 1 --2- 172.21.15.143:0/2319807736 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fb18806cd50 0x7fb1881479c0 secure :-1 s=READY pgs=1265 cs=0 l=1 rev1=1 crypto rx=0x7fb18400c500 tx=0x7fb18400c900 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:24:00.372 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:00.371+0000 7fb17e7fc640 1 -- 172.21.15.143:0/2319807736 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb1840052d0 con 0x7fb18806cd50 2023-11-26T07:24:00.372 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:00.371+0000 7fb17e7fc640 1 -- 172.21.15.143:0/2319807736 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb184005430 con 0x7fb18806cd50 2023-11-26T07:24:00.372 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:00.371+0000 7fb18fbaa640 1 -- 172.21.15.143:0/2319807736 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb1780094d0 con 0x7fb18806cd50 2023-11-26T07:24:00.372 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:00.371+0000 7fb17e7fc640 1 -- 172.21.15.143:0/2319807736 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb184003840 con 0x7fb18806cd50 2023-11-26T07:24:00.372 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:00.371+0000 7fb18fbaa640 1 -- 172.21.15.143:0/2319807736 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb188148560 con 0x7fb18806cd50 2023-11-26T07:24:00.372 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:00.371+0000 7fb18fbaa640 1 -- 172.21.15.143:0/2319807736 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb188067680 con 0x7fb18806cd50 2023-11-26T07:24:00.373 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:00.371+0000 7fb17e7fc640 1 -- 172.21.15.143:0/2319807736 <== mon.0 v2:172.21.15.143:3300/0 4 ==== mgrmap(e 16) v1 ==== 85954+0+0 (secure 0 0 0) 0x7fb1840039a0 con 0x7fb18806cd50 2023-11-26T07:24:00.373 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:00.371+0000 7fb17e7fc640 1 --2- 172.21.15.143:0/2319807736 >> [v2:172.21.15.188:6836/3401128553,v1:172.21.15.188:6837/3401128553] conn(0x7fb16c066590 0x7fb16c068a50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:24:00.373 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:00.371+0000 7fb17e7fc640 1 -- 172.21.15.143:0/2319807736 <== mon.0 v2:172.21.15.143:3300/0 5 ==== osd_map(94..94 src has 1..94) v4 ==== 5029+0+0 (secure 0 0 0) 0x7fb184083080 con 0x7fb18806cd50 2023-11-26T07:24:00.374 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:00.371+0000 7fb18d145640 1 --2- 172.21.15.143:0/2319807736 >> [v2:172.21.15.188:6836/3401128553,v1:172.21.15.188:6837/3401128553] conn(0x7fb16c066590 0x7fb16c068a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:24:00.374 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:00.375+0000 7fb18d145640 1 --2- 172.21.15.143:0/2319807736 >> [v2:172.21.15.188:6836/3401128553,v1:172.21.15.188:6837/3401128553] conn(0x7fb16c066590 0x7fb16c068a50 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7fb1780b2000 tx=0x7fb178009040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:24:00.377 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:00.375+0000 7fb17e7fc640 1 -- 172.21.15.143:0/2319807736 <== mon.0 v2:172.21.15.143:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172649 (secure 0 0 0) 0x7fb184050740 con 0x7fb18806cd50 2023-11-26T07:24:00.528 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:00.527+0000 7fb18fbaa640 1 -- 172.21.15.143:0/2319807736 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7fb18815b2f0 con 0x7fb18806cd50 2023-11-26T07:24:00.529 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:00.527+0000 7fb17e7fc640 1 -- 172.21.15.143:0/2319807736 <== mon.0 v2:172.21.15.143:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v94) v1 ==== 93+0+10 (secure 0 0 0) 0x7fb18815b2f0 con 0x7fb18806cd50 2023-11-26T07:24:00.529 INFO:teuthology.orchestra.run.smithi143.stdout:pg_num: 1 2023-11-26T07:24:00.531 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:00.531+0000 7fb18fbaa640 1 -- 172.21.15.143:0/2319807736 >> [v2:172.21.15.188:6836/3401128553,v1:172.21.15.188:6837/3401128553] conn(0x7fb16c066590 msgr2=0x7fb16c068a50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:24:00.531 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:00.531+0000 7fb18fbaa640 1 --2- 172.21.15.143:0/2319807736 >> [v2:172.21.15.188:6836/3401128553,v1:172.21.15.188:6837/3401128553] conn(0x7fb16c066590 0x7fb16c068a50 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7fb1780b2000 tx=0x7fb178009040 comp rx=0 tx=0).stop 2023-11-26T07:24:00.531 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:00.531+0000 7fb18fbaa640 1 -- 172.21.15.143:0/2319807736 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fb18806cd50 msgr2=0x7fb1881479c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:24:00.531 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:00.531+0000 7fb18fbaa640 1 --2- 172.21.15.143:0/2319807736 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fb18806cd50 0x7fb1881479c0 secure :-1 s=READY pgs=1265 cs=0 l=1 rev1=1 crypto rx=0x7fb18400c500 tx=0x7fb18400c900 comp rx=0 tx=0).stop 2023-11-26T07:24:00.531 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:00.531+0000 7fb18fbaa640 1 -- 172.21.15.143:0/2319807736 shutdown_connections 2023-11-26T07:24:00.531 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:00.531+0000 7fb18fbaa640 1 --2- 172.21.15.143:0/2319807736 >> [v2:172.21.15.188:6836/3401128553,v1:172.21.15.188:6837/3401128553] conn(0x7fb16c066590 0x7fb16c068a50 unknown :-1 s=CLOSED pgs=107 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:24:00.531 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:00.531+0000 7fb18fbaa640 1 --2- 172.21.15.143:0/2319807736 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fb1880cf280 0x7fb18814a460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:24:00.531 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:00.531+0000 7fb18fbaa640 1 --2- 172.21.15.143:0/2319807736 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fb188070d10 0x7fb188147280 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:24:00.532 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:00.531+0000 7fb18fbaa640 1 --2- 172.21.15.143:0/2319807736 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fb18806cd50 0x7fb1881479c0 unknown :-1 s=CLOSED pgs=1265 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:24:00.532 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:00.531+0000 7fb18fbaa640 1 -- 172.21.15.143:0/2319807736 >> 172.21.15.143:0/2319807736 conn(0x7fb18806a890 msgr2=0x7fb1880af4b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:24:00.532 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:00.531+0000 7fb18fbaa640 1 -- 172.21.15.143:0/2319807736 shutdown_connections 2023-11-26T07:24:00.532 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:00.531+0000 7fb18fbaa640 1 -- 172.21.15.143:0/2319807736 wait complete. 2023-11-26T07:24:00.550 DEBUG:teuthology.orchestra.run.smithi143:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Starting test tasks.cephfs.test_volumes.TestMisc.test_subvolume_no_upgrade_v1_sanity' 2023-11-26T07:24:00.688 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:00.687+0000 7f5fee10e640 1 Processor -- start 2023-11-26T07:24:00.688 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:00.687+0000 7f5fee10e640 1 -- start start 2023-11-26T07:24:00.690 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:00.687+0000 7f5fee10e640 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7f5fe80ce7c0 0x7f5fe815f6b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:24:00.690 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:00.687+0000 7f5fee10e640 1 -- --> v1:172.21.15.143:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5fe806c520 con 0x7f5fe813e290 2023-11-26T07:24:00.690 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:00.687+0000 7f5fee10e640 1 -- --> v1:172.21.15.188:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5fe806c800 con 0x7f5fe80cf990 2023-11-26T07:24:00.690 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:00.687+0000 7f5fee10e640 1 -- --> v2:172.21.15.143:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5fe806cae0 con 0x7f5fe80ce7c0 2023-11-26T07:24:00.690 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:00.687+0000 7f5fe77fe640 1 --1- >> v1:172.21.15.143:6789/0 conn(0x7f5fe813e290 0x7f5fe80cf280 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.143:6789/0 says I am v1:172.21.15.143:43334/0 (socket says 172.21.15.143:43334) 2023-11-26T07:24:00.690 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:00.687+0000 7f5fe77fe640 1 -- 172.21.15.143:0/1117721668 learned_addr learned my addr 172.21.15.143:0/1117721668 (peer_addr_for_me v1:172.21.15.143:0/0) 2023-11-26T07:24:00.690 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:00.687+0000 7f5fe7fff640 1 --2- 172.21.15.143:0/1117721668 >> v2:172.21.15.143:3300/0 conn(0x7f5fe80ce7c0 0x7f5fe815f6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:24:00.691 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:00.691+0000 7f5fe7fff640 1 -- 172.21.15.143:0/1117721668 >> v1:172.21.15.188:6789/0 conn(0x7f5fe80cf990 legacy=0x7f5fe815efa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:24:00.691 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:00.691+0000 7f5fe7fff640 1 -- 172.21.15.143:0/1117721668 >> v1:172.21.15.143:6789/0 conn(0x7f5fe813e290 legacy=0x7f5fe80cf280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:24:00.691 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:00.691+0000 7f5fe7fff640 1 -- 172.21.15.143:0/1117721668 --> v2:172.21.15.143:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5fe806cdc0 con 0x7f5fe80ce7c0 2023-11-26T07:24:00.691 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:00.691+0000 7f5fe67fc640 1 -- 172.21.15.143:0/1117721668 <== mon.0 v1:172.21.15.143:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 302892806 0 0) 0x7f5fe806c520 con 0x7f5fe813e290 2023-11-26T07:24:00.693 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:00.691+0000 7f5fe7fff640 1 --2- 172.21.15.143:0/1117721668 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f5fe80ce7c0 0x7f5fe815f6b0 secure :-1 s=READY pgs=1267 cs=0 l=1 rev1=1 crypto rx=0x7f5fe0009d90 tx=0x7f5fe001c890 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d2741faa41c8cf26 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:24:00.693 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:00.691+0000 7f5fe67fc640 1 -- 172.21.15.143:0/1117721668 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5fe00a5970 con 0x7f5fe80ce7c0 2023-11-26T07:24:00.693 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:00.691+0000 7f5fe67fc640 1 -- 172.21.15.143:0/1117721668 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5fe00a5ad0 con 0x7f5fe80ce7c0 2023-11-26T07:24:00.693 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:00.691+0000 7f5fe67fc640 1 -- 172.21.15.143:0/1117721668 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5fe00a5e60 con 0x7f5fe80ce7c0 2023-11-26T07:24:00.694 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:00.691+0000 7f5fee10e640 1 -- 172.21.15.143:0/1117721668 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f5fe80ce7c0 msgr2=0x7f5fe815f6b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:24:00.694 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:00.691+0000 7f5fee10e640 1 --2- 172.21.15.143:0/1117721668 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f5fe80ce7c0 0x7f5fe815f6b0 secure :-1 s=READY pgs=1267 cs=0 l=1 rev1=1 crypto rx=0x7f5fe0009d90 tx=0x7f5fe001c890 comp rx=0 tx=0).stop 2023-11-26T07:24:00.694 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:00.695+0000 7f5fee10e640 1 -- 172.21.15.143:0/1117721668 shutdown_connections 2023-11-26T07:24:00.694 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:00.695+0000 7f5fee10e640 1 --2- 172.21.15.143:0/1117721668 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f5fe80ce7c0 0x7f5fe815f6b0 unknown :-1 s=CLOSED pgs=1267 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:24:00.694 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:00.695+0000 7f5fee10e640 1 -- 172.21.15.143:0/1117721668 >> 172.21.15.143:0/1117721668 conn(0x7f5fe806a890 msgr2=0x7f5fe806aca0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:24:00.695 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:00.695+0000 7f5fee10e640 1 -- 172.21.15.143:0/1117721668 shutdown_connections 2023-11-26T07:24:00.695 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:00.695+0000 7f5fee10e640 1 -- 172.21.15.143:0/1117721668 wait complete. 2023-11-26T07:24:00.695 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:00.695+0000 7f5fee10e640 1 Processor -- start 2023-11-26T07:24:00.696 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:00.695+0000 7f5fee10e640 1 -- start start 2023-11-26T07:24:00.697 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:00.695+0000 7f5fee10e640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f5fe80ce7c0 0x7f5fe80b1ef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:24:00.697 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:00.695+0000 7f5fee10e640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f5fe80cf990 0x7f5fe815b510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:24:00.697 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:00.695+0000 7f5fee10e640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f5fe813e290 0x7f5fe80af570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:24:00.697 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:00.695+0000 7f5fee10e640 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5fe80afab0 con 0x7f5fe80cf990 2023-11-26T07:24:00.697 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:00.695+0000 7f5fee10e640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5fe80b1280 con 0x7f5fe813e290 2023-11-26T07:24:00.697 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:00.695+0000 7f5fee10e640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5fe80b24d0 con 0x7f5fe80ce7c0 2023-11-26T07:24:00.697 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:00.695+0000 7f5fe6ffd640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f5fe80cf990 0x7f5fe815b510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:24:00.698 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:00.695+0000 7f5fe77fe640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f5fe80ce7c0 0x7f5fe80b1ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:24:00.698 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:00.695+0000 7f5fe6ffd640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f5fe80cf990 0x7f5fe815b510 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:38560/0 (socket says 172.21.15.143:38560) 2023-11-26T07:24:00.698 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:00.695+0000 7f5fe6ffd640 1 -- 172.21.15.143:0/538246965 learned_addr learned my addr 172.21.15.143:0/538246965 (peer_addr_for_me v2:172.21.15.143:0/0) 2023-11-26T07:24:00.698 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:00.695+0000 7f5fe7fff640 1 --2- 172.21.15.143:0/538246965 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f5fe813e290 0x7f5fe80af570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:24:00.699 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:00.695+0000 7f5fe6ffd640 1 -- 172.21.15.143:0/538246965 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f5fe80ce7c0 msgr2=0x7f5fe80b1ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:24:00.699 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:00.695+0000 7f5fe6ffd640 1 --2- 172.21.15.143:0/538246965 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f5fe80ce7c0 0x7f5fe80b1ef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:24:00.699 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:00.695+0000 7f5fe6ffd640 1 -- 172.21.15.143:0/538246965 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f5fe813e290 msgr2=0x7f5fe80af570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:24:00.699 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:00.695+0000 7f5fe6ffd640 1 --2- 172.21.15.143:0/538246965 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f5fe813e290 0x7f5fe80af570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:24:00.699 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:00.695+0000 7f5fe6ffd640 1 -- 172.21.15.143:0/538246965 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5fe0009a70 con 0x7f5fe80cf990 2023-11-26T07:24:00.699 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:00.695+0000 7f5fe77fe640 1 --2- 172.21.15.143:0/538246965 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f5fe80ce7c0 0x7f5fe80b1ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-26T07:24:00.699 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:00.695+0000 7f5fe6ffd640 1 --2- 172.21.15.143:0/538246965 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f5fe80cf990 0x7f5fe815b510 secure :-1 s=READY pgs=1268 cs=0 l=1 rev1=1 crypto rx=0x7f5fdc00c500 tx=0x7f5fdc00c9c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:24:00.700 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:00.699+0000 7f5fcffff640 1 -- 172.21.15.143:0/538246965 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5fdc005220 con 0x7f5fe80cf990 2023-11-26T07:24:00.700 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:00.699+0000 7f5fcffff640 1 -- 172.21.15.143:0/538246965 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5fdc005380 con 0x7f5fe80cf990 2023-11-26T07:24:00.700 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:00.699+0000 7f5fcffff640 1 -- 172.21.15.143:0/538246965 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5fdc007270 con 0x7f5fe80cf990 2023-11-26T07:24:00.700 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:00.699+0000 7f5fee10e640 1 -- 172.21.15.143:0/538246965 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5fe80b27b0 con 0x7f5fe80cf990 2023-11-26T07:24:00.700 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:00.699+0000 7f5fee10e640 1 -- 172.21.15.143:0/538246965 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5fe80b2d70 con 0x7f5fe80cf990 2023-11-26T07:24:00.700 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:00.699+0000 7f5fee10e640 1 -- 172.21.15.143:0/538246965 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5fe8067680 con 0x7f5fe80cf990 2023-11-26T07:24:00.700 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:00.699+0000 7f5fcffff640 1 -- 172.21.15.143:0/538246965 <== mon.0 v2:172.21.15.143:3300/0 4 ==== mgrmap(e 16) v1 ==== 85954+0+0 (secure 0 0 0) 0x7f5fdc0073d0 con 0x7f5fe80cf990 2023-11-26T07:24:00.701 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:00.699+0000 7f5fcffff640 1 --2- 172.21.15.143:0/538246965 >> [v2:172.21.15.188:6836/3401128553,v1:172.21.15.188:6837/3401128553] conn(0x7f5fd40665c0 0x7f5fd4068a80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:24:00.701 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:00.699+0000 7f5fcffff640 1 -- 172.21.15.143:0/538246965 <== mon.0 v2:172.21.15.143:3300/0 5 ==== osd_map(94..94 src has 1..94) v4 ==== 5029+0+0 (secure 0 0 0) 0x7f5fdc083560 con 0x7f5fe80cf990 2023-11-26T07:24:00.701 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:00.699+0000 7f5fe77fe640 1 --2- 172.21.15.143:0/538246965 >> [v2:172.21.15.188:6836/3401128553,v1:172.21.15.188:6837/3401128553] conn(0x7f5fd40665c0 0x7f5fd4068a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:24:00.702 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:00.703+0000 7f5fe77fe640 1 --2- 172.21.15.143:0/538246965 >> [v2:172.21.15.188:6836/3401128553,v1:172.21.15.188:6837/3401128553] conn(0x7f5fd40665c0 0x7f5fd4068a80 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7f5fd8007bc0 tx=0x7f5fd8007b10 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:24:00.705 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:00.703+0000 7f5fcffff640 1 -- 172.21.15.143:0/538246965 <== mon.0 v2:172.21.15.143:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172649 (secure 0 0 0) 0x7f5fdc050b90 con 0x7f5fe80cf990 2023-11-26T07:24:00.909 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:00.907+0000 7f5fee10e640 1 -- 172.21.15.143:0/538246965 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_volumes.TestMisc.test_subvolume_no_upgrade_v1_sanity"]} v 0) v1 -- 0x7f5fe806df50 con 0x7f5fe80cf990 2023-11-26T07:24:01.876 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:01.875+0000 7f5fcffff640 1 -- 172.21.15.143:0/538246965 <== mon.0 v2:172.21.15.143:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_volumes.TestMisc.test_subvolume_no_upgrade_v1_sanity"]}]=0 v328) v1 ==== 152+0+0 (secure 0 0 0) 0x7f5fdc0541b0 con 0x7f5fe80cf990 2023-11-26T07:24:01.879 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:01.879+0000 7f5fee10e640 1 -- 172.21.15.143:0/538246965 >> [v2:172.21.15.188:6836/3401128553,v1:172.21.15.188:6837/3401128553] conn(0x7f5fd40665c0 msgr2=0x7f5fd4068a80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:24:01.879 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:01.879+0000 7f5fee10e640 1 --2- 172.21.15.143:0/538246965 >> [v2:172.21.15.188:6836/3401128553,v1:172.21.15.188:6837/3401128553] conn(0x7f5fd40665c0 0x7f5fd4068a80 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7f5fd8007bc0 tx=0x7f5fd8007b10 comp rx=0 tx=0).stop 2023-11-26T07:24:01.879 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:01.879+0000 7f5fee10e640 1 -- 172.21.15.143:0/538246965 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f5fe80cf990 msgr2=0x7f5fe815b510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:24:01.879 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:01.879+0000 7f5fee10e640 1 --2- 172.21.15.143:0/538246965 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f5fe80cf990 0x7f5fe815b510 secure :-1 s=READY pgs=1268 cs=0 l=1 rev1=1 crypto rx=0x7f5fdc00c500 tx=0x7f5fdc00c9c0 comp rx=0 tx=0).stop 2023-11-26T07:24:01.879 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:01.879+0000 7f5fee10e640 1 -- 172.21.15.143:0/538246965 shutdown_connections 2023-11-26T07:24:01.879 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:01.879+0000 7f5fee10e640 1 --2- 172.21.15.143:0/538246965 >> [v2:172.21.15.188:6836/3401128553,v1:172.21.15.188:6837/3401128553] conn(0x7f5fd40665c0 0x7f5fd4068a80 unknown :-1 s=CLOSED pgs=108 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:24:01.879 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:01.879+0000 7f5fee10e640 1 --2- 172.21.15.143:0/538246965 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f5fe813e290 0x7f5fe80af570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:24:01.880 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:01.879+0000 7f5fee10e640 1 --2- 172.21.15.143:0/538246965 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f5fe80cf990 0x7f5fe815b510 unknown :-1 s=CLOSED pgs=1268 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:24:01.880 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:01.879+0000 7f5fee10e640 1 --2- 172.21.15.143:0/538246965 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f5fe80ce7c0 0x7f5fe80b1ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:24:01.880 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:01.879+0000 7f5fee10e640 1 -- 172.21.15.143:0/538246965 >> 172.21.15.143:0/538246965 conn(0x7f5fe806a890 msgr2=0x7f5fe8159f90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:24:01.880 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:01.879+0000 7f5fee10e640 1 -- 172.21.15.143:0/538246965 shutdown_connections 2023-11-26T07:24:01.880 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:01.879+0000 7f5fee10e640 1 -- 172.21.15.143:0/538246965 wait complete. 2023-11-26T07:24:01.899 DEBUG:teuthology.orchestra.run.smithi143:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mon mon_allow_pool_delete True 2023-11-26T07:24:02.041 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:02.039+0000 7efcebf80640 1 Processor -- start 2023-11-26T07:24:02.041 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:02.039+0000 7efcebf80640 1 -- start start 2023-11-26T07:24:02.043 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:02.043+0000 7efcebf80640 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7efce41596d0 0x7efce4071850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:24:02.043 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:02.043+0000 7efcebf80640 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7efce40ce680 0x7efce40b2870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:24:02.043 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:02.043+0000 7efcebf80640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7efce40b2db0 0x7efce415d2c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:24:02.044 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:02.043+0000 7efcebf80640 1 -- --> v2:172.21.15.143:3300/0 -- mon_getmap magic: 0 v1 -- 0x7efce40b3240 con 0x7efce41596d0 2023-11-26T07:24:02.044 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:02.043+0000 7efcebf80640 1 -- --> v2:172.21.15.188:3300/0 -- mon_getmap magic: 0 v1 -- 0x7efce40cbe00 con 0x7efce40ce680 2023-11-26T07:24:02.044 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:02.043+0000 7efcebf80640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7efce406c750 con 0x7efce40b2db0 2023-11-26T07:24:02.044 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:02.043+0000 7efce9d1c640 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7efce41596d0 0x7efce4071850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:24:02.044 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:02.043+0000 7efce9d1c640 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7efce41596d0 0x7efce4071850 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:38572/0 (socket says 172.21.15.143:38572) 2023-11-26T07:24:02.044 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:02.043+0000 7efce9d1c640 1 -- 172.21.15.143:0/2391538810 learned_addr learned my addr 172.21.15.143:0/2391538810 (peer_addr_for_me v2:172.21.15.143:0/0) 2023-11-26T07:24:02.044 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:02.043+0000 7efcea51d640 1 --2- 172.21.15.143:0/2391538810 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7efce40b2db0 0x7efce415d2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:24:02.044 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:02.043+0000 7efce951b640 1 --2- 172.21.15.143:0/2391538810 >> v2:172.21.15.188:3300/0 conn(0x7efce40ce680 0x7efce40b2870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:24:02.044 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:02.043+0000 7efce9d1c640 1 -- 172.21.15.143:0/2391538810 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7efce40b2db0 msgr2=0x7efce415d2c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-26T07:24:02.044 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:02.043+0000 7efce9d1c640 1 --2- 172.21.15.143:0/2391538810 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7efce40b2db0 0x7efce415d2c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:24:02.044 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:02.043+0000 7efce9d1c640 1 -- 172.21.15.143:0/2391538810 >> v2:172.21.15.188:3300/0 conn(0x7efce40ce680 msgr2=0x7efce40b2870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-26T07:24:02.044 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:02.043+0000 7efce9d1c640 1 --2- 172.21.15.143:0/2391538810 >> v2:172.21.15.188:3300/0 conn(0x7efce40ce680 0x7efce40b2870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:24:02.045 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:02.043+0000 7efce9d1c640 1 -- 172.21.15.143:0/2391538810 --> v2:172.21.15.143:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efce40b3550 con 0x7efce41596d0 2023-11-26T07:24:02.046 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:02.047+0000 7efce9d1c640 1 --2- 172.21.15.143:0/2391538810 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7efce41596d0 0x7efce4071850 secure :-1 s=READY pgs=1269 cs=0 l=1 rev1=1 crypto rx=0x7efce40ce440 tx=0x7efce001c500 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a9798cb13023f931 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:24:02.047 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:02.047+0000 7efce8d1a640 1 -- 172.21.15.143:0/2391538810 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efce00ab600 con 0x7efce41596d0 2023-11-26T07:24:02.047 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:02.047+0000 7efce8d1a640 1 -- 172.21.15.143:0/2391538810 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7efce00a7050 con 0x7efce41596d0 2023-11-26T07:24:02.047 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:02.047+0000 7efce8d1a640 1 -- 172.21.15.143:0/2391538810 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efce00ab980 con 0x7efce41596d0 2023-11-26T07:24:02.048 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:02.047+0000 7efcebf80640 1 -- 172.21.15.143:0/2391538810 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7efce41596d0 msgr2=0x7efce4071850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:24:02.048 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:02.047+0000 7efcebf80640 1 --2- 172.21.15.143:0/2391538810 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7efce41596d0 0x7efce4071850 secure :-1 s=READY pgs=1269 cs=0 l=1 rev1=1 crypto rx=0x7efce40ce440 tx=0x7efce001c500 comp rx=0 tx=0).stop 2023-11-26T07:24:02.048 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:02.047+0000 7efcebf80640 1 -- 172.21.15.143:0/2391538810 shutdown_connections 2023-11-26T07:24:02.048 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:02.047+0000 7efcebf80640 1 --2- 172.21.15.143:0/2391538810 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7efce40b2db0 0x7efce415d2c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:24:02.048 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:02.047+0000 7efcebf80640 1 --2- 172.21.15.143:0/2391538810 >> v2:172.21.15.188:3300/0 conn(0x7efce40ce680 0x7efce40b2870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:24:02.048 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:02.047+0000 7efcebf80640 1 --2- 172.21.15.143:0/2391538810 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7efce41596d0 0x7efce4071850 unknown :-1 s=CLOSED pgs=1269 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:24:02.048 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:02.047+0000 7efcebf80640 1 -- 172.21.15.143:0/2391538810 >> 172.21.15.143:0/2391538810 conn(0x7efce406a890 msgr2=0x7efce406aca0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:24:02.048 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:02.047+0000 7efcebf80640 1 -- 172.21.15.143:0/2391538810 shutdown_connections 2023-11-26T07:24:02.048 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:02.047+0000 7efcebf80640 1 -- 172.21.15.143:0/2391538810 wait complete. 2023-11-26T07:24:02.049 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:02.047+0000 7efcebf80640 1 Processor -- start 2023-11-26T07:24:02.049 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:02.047+0000 7efcebf80640 1 -- start start 2023-11-26T07:24:02.051 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:02.051+0000 7efcebf80640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7efce40b2db0 0x7efce414ea70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:24:02.052 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:02.051+0000 7efcebf80640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7efce40ce680 0x7efce40cc7d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:24:02.052 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:02.051+0000 7efcebf80640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7efce41596d0 0x7efce40af7e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:24:02.052 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:02.051+0000 7efcebf80640 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efce40cde30 con 0x7efce41596d0 2023-11-26T07:24:02.052 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:02.051+0000 7efcebf80640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efce414efb0 con 0x7efce40ce680 2023-11-26T07:24:02.052 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:02.051+0000 7efcebf80640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7efce40cd1b0 con 0x7efce40b2db0 2023-11-26T07:24:02.052 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:02.051+0000 7efcea51d640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7efce41596d0 0x7efce40af7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:24:02.052 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:02.051+0000 7efcea51d640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7efce41596d0 0x7efce40af7e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:38576/0 (socket says 172.21.15.143:38576) 2023-11-26T07:24:02.052 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:02.051+0000 7efcea51d640 1 -- 172.21.15.143:0/3732104363 learned_addr learned my addr 172.21.15.143:0/3732104363 (peer_addr_for_me v2:172.21.15.143:0/0) 2023-11-26T07:24:02.052 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:02.051+0000 7efce9d1c640 1 --2- 172.21.15.143:0/3732104363 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7efce40b2db0 0x7efce414ea70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:24:02.052 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:02.051+0000 7efce951b640 1 --2- 172.21.15.143:0/3732104363 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7efce40ce680 0x7efce40cc7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:24:02.053 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:02.051+0000 7efcea51d640 1 -- 172.21.15.143:0/3732104363 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7efce40b2db0 msgr2=0x7efce414ea70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:24:02.053 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:02.051+0000 7efcea51d640 1 --2- 172.21.15.143:0/3732104363 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7efce40b2db0 0x7efce414ea70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:24:02.053 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:02.051+0000 7efcea51d640 1 -- 172.21.15.143:0/3732104363 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7efce40ce680 msgr2=0x7efce40cc7d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:24:02.053 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:02.051+0000 7efcea51d640 1 --2- 172.21.15.143:0/3732104363 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7efce40ce680 0x7efce40cc7d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:24:02.053 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:02.051+0000 7efcea51d640 1 -- 172.21.15.143:0/3732104363 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efce00094d0 con 0x7efce41596d0 2023-11-26T07:24:02.053 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:02.051+0000 7efcea51d640 1 --2- 172.21.15.143:0/3732104363 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7efce41596d0 0x7efce40af7e0 secure :-1 s=READY pgs=1270 cs=0 l=1 rev1=1 crypto rx=0x7efcdc007e60 tx=0x7efcdc00e4b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:24:02.053 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:02.051+0000 7efcda7fc640 1 -- 172.21.15.143:0/3732104363 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efcdc014420 con 0x7efce41596d0 2023-11-26T07:24:02.053 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:02.051+0000 7efcda7fc640 1 -- 172.21.15.143:0/3732104363 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7efcdc0145a0 con 0x7efce41596d0 2023-11-26T07:24:02.053 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:02.051+0000 7efcebf80640 1 -- 172.21.15.143:0/3732104363 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7efce40cd490 con 0x7efce41596d0 2023-11-26T07:24:02.053 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:02.051+0000 7efcda7fc640 1 -- 172.21.15.143:0/3732104363 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efcdc014950 con 0x7efce41596d0 2023-11-26T07:24:02.053 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:02.051+0000 7efcebf80640 1 -- 172.21.15.143:0/3732104363 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7efce40cda50 con 0x7efce41596d0 2023-11-26T07:24:02.054 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:02.051+0000 7efcebf80640 1 -- 172.21.15.143:0/3732104363 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7efce40cdf70 con 0x7efce41596d0 2023-11-26T07:24:02.054 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:02.051+0000 7efcda7fc640 1 -- 172.21.15.143:0/3732104363 <== mon.0 v2:172.21.15.143:3300/0 4 ==== mgrmap(e 16) v1 ==== 85954+0+0 (secure 0 0 0) 0x7efcdc014ab0 con 0x7efce41596d0 2023-11-26T07:24:02.054 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:02.055+0000 7efcda7fc640 1 --2- 172.21.15.143:0/3732104363 >> [v2:172.21.15.188:6836/3401128553,v1:172.21.15.188:6837/3401128553] conn(0x7efcc8066590 0x7efcc8068a50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:24:02.054 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:02.055+0000 7efcda7fc640 1 -- 172.21.15.143:0/3732104363 <== mon.0 v2:172.21.15.143:3300/0 5 ==== osd_map(94..94 src has 1..94) v4 ==== 5029+0+0 (secure 0 0 0) 0x7efcdc0845f0 con 0x7efce41596d0 2023-11-26T07:24:02.055 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:02.055+0000 7efce9d1c640 1 --2- 172.21.15.143:0/3732104363 >> [v2:172.21.15.188:6836/3401128553,v1:172.21.15.188:6837/3401128553] conn(0x7efcc8066590 0x7efcc8068a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:24:02.056 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:02.055+0000 7efce9d1c640 1 --2- 172.21.15.143:0/3732104363 >> [v2:172.21.15.188:6836/3401128553,v1:172.21.15.188:6837/3401128553] conn(0x7efcc8066590 0x7efcc8068a50 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7efce00ad000 tx=0x7efce0008040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:24:02.058 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:02.059+0000 7efcda7fc640 1 -- 172.21.15.143:0/3732104363 <== mon.0 v2:172.21.15.143:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172649 (secure 0 0 0) 0x7efce40cdf70 con 0x7efce41596d0 2023-11-26T07:24:02.208 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:02.207+0000 7efcebf80640 1 -- 172.21.15.143:0/3732104363 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command([{prefix=config set, name=mon_allow_pool_delete}] v 0) v1 -- 0x7efce40cdf70 con 0x7efce41596d0 2023-11-26T07:24:02.214 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:02.215+0000 7efcda7fc640 1 -- 172.21.15.143:0/3732104363 <== mon.0 v2:172.21.15.143:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mon_allow_pool_delete}]=0 v20)=0 v20) v1 ==== 122+0+0 (secure 0 0 0) 0x7efce40cdf70 con 0x7efce41596d0 2023-11-26T07:24:02.216 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:02.215+0000 7efcebf80640 1 -- 172.21.15.143:0/3732104363 >> [v2:172.21.15.188:6836/3401128553,v1:172.21.15.188:6837/3401128553] conn(0x7efcc8066590 msgr2=0x7efcc8068a50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:24:02.216 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:02.215+0000 7efcebf80640 1 --2- 172.21.15.143:0/3732104363 >> [v2:172.21.15.188:6836/3401128553,v1:172.21.15.188:6837/3401128553] conn(0x7efcc8066590 0x7efcc8068a50 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7efce00ad000 tx=0x7efce0008040 comp rx=0 tx=0).stop 2023-11-26T07:24:02.216 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:02.215+0000 7efcebf80640 1 -- 172.21.15.143:0/3732104363 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7efce41596d0 msgr2=0x7efce40af7e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:24:02.216 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:02.215+0000 7efcebf80640 1 --2- 172.21.15.143:0/3732104363 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7efce41596d0 0x7efce40af7e0 secure :-1 s=READY pgs=1270 cs=0 l=1 rev1=1 crypto rx=0x7efcdc007e60 tx=0x7efcdc00e4b0 comp rx=0 tx=0).stop 2023-11-26T07:24:02.217 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:02.215+0000 7efcebf80640 1 -- 172.21.15.143:0/3732104363 shutdown_connections 2023-11-26T07:24:02.217 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:02.215+0000 7efcebf80640 1 --2- 172.21.15.143:0/3732104363 >> [v2:172.21.15.188:6836/3401128553,v1:172.21.15.188:6837/3401128553] conn(0x7efcc8066590 0x7efcc8068a50 unknown :-1 s=CLOSED pgs=109 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:24:02.217 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:02.215+0000 7efcebf80640 1 --2- 172.21.15.143:0/3732104363 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7efce41596d0 0x7efce40af7e0 unknown :-1 s=CLOSED pgs=1270 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:24:02.217 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:02.215+0000 7efcebf80640 1 --2- 172.21.15.143:0/3732104363 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7efce40ce680 0x7efce40cc7d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:24:02.217 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:02.215+0000 7efcebf80640 1 --2- 172.21.15.143:0/3732104363 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7efce40b2db0 0x7efce414ea70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:24:02.217 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:02.215+0000 7efcebf80640 1 -- 172.21.15.143:0/3732104363 >> 172.21.15.143:0/3732104363 conn(0x7efce406a890 msgr2=0x7efce414f7a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:24:02.217 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:02.215+0000 7efcebf80640 1 -- 172.21.15.143:0/3732104363 shutdown_connections 2023-11-26T07:24:02.217 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:02.215+0000 7efcebf80640 1 -- 172.21.15.143:0/3732104363 wait complete. 2023-11-26T07:24:02.235 INFO:teuthology.nuke.actions:Clearing teuthology firewall rules... 2023-11-26T07:24:02.235 DEBUG:teuthology.orchestra.run.smithi143:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2023-11-26T07:24:02.324 DEBUG:teuthology.orchestra.run.smithi188:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2023-11-26T07:24:02.430 INFO:teuthology.nuke.actions:Cleared teuthology firewall rules. 2023-11-26T07:24:02.430 DEBUG:teuthology.orchestra.run.smithi143:> set -ex 2023-11-26T07:24:02.430 DEBUG:teuthology.orchestra.run.smithi143:> dd if=/proc/self/mounts of=/dev/stdout 2023-11-26T07:24:02.439 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,relatime,size=16354672k,nr_inodes=4088668,mode=755 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3275384k,mode=755 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=3455 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysusers.service ramfs ro,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3275380k,nr_inodes=818845,mode=700,uid=1000,gid=1271 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3275384k,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2023-11-26T07:24:02.439 DEBUG:teuthology.orchestra.run.smithi188:> set -ex 2023-11-26T07:24:02.440 DEBUG:teuthology.orchestra.run.smithi188:> dd if=/proc/self/mounts of=/dev/stdout 2023-11-26T07:24:02.449 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,relatime,size=16354628k,nr_inodes=4088657,mode=755 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3275376k,mode=755 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=787 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysusers.service ramfs ro,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3275372k,nr_inodes=818843,mode=700,uid=1000,gid=1271 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3275376k,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2023-11-26T07:24:02.449 DEBUG:teuthology.orchestra.run.smithi143:> set -ex 2023-11-26T07:24:02.449 DEBUG:teuthology.orchestra.run.smithi143:> dd if=/proc/self/mounts of=/dev/stdout 2023-11-26T07:24:02.490 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,relatime,size=16354672k,nr_inodes=4088668,mode=755 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3275384k,mode=755 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=3455 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysusers.service ramfs ro,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3275380k,nr_inodes=818845,mode=700,uid=1000,gid=1271 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3275384k,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2023-11-26T07:24:02.490 DEBUG:teuthology.orchestra.run.smithi188:> set -ex 2023-11-26T07:24:02.490 DEBUG:teuthology.orchestra.run.smithi188:> dd if=/proc/self/mounts of=/dev/stdout 2023-11-26T07:24:02.500 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,relatime,size=16354628k,nr_inodes=4088657,mode=755 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3275376k,mode=755 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=787 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysusers.service ramfs ro,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3275372k,nr_inodes=818843,mode=700,uid=1000,gid=1271 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3275376k,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2023-11-26T07:24:02.500 INFO:tasks.cephfs.cephfs_test_case:[, , , ] 2023-11-26T07:24:02.500 DEBUG:teuthology.orchestra.run.smithi143:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-11-26T07:24:02.672 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:02.671+0000 7f6ad6c9a640 1 Processor -- start 2023-11-26T07:24:02.672 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:02.671+0000 7f6ad6c9a640 1 -- start start 2023-11-26T07:24:02.674 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:02.671+0000 7f6ad6c9a640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f6ad013e290 0x7f6ad00cf280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:24:02.674 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:02.671+0000 7f6ad6c9a640 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7f6ad00cf7c0 0x7f6ad015efa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:24:02.674 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:02.671+0000 7f6ad6c9a640 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7f6ad015f4e0 0x7f6ad01408e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:24:02.674 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:02.671+0000 7f6ad6c9a640 1 -- --> v2:172.21.15.143:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6ad00cfc40 con 0x7f6ad00cf7c0 2023-11-26T07:24:02.674 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:02.671+0000 7f6ad6c9a640 1 -- --> v2:172.21.15.188:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6ad006c520 con 0x7f6ad015f4e0 2023-11-26T07:24:02.674 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:02.671+0000 7f6ad6c9a640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6ad006c800 con 0x7f6ad013e290 2023-11-26T07:24:02.675 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:02.671+0000 7f6acffff640 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7f6ad00cf7c0 0x7f6ad015efa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:24:02.675 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:02.671+0000 7f6acffff640 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7f6ad00cf7c0 0x7f6ad015efa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:38590/0 (socket says 172.21.15.143:38590) 2023-11-26T07:24:02.675 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:02.671+0000 7f6acffff640 1 -- 172.21.15.143:0/3209624228 learned_addr learned my addr 172.21.15.143:0/3209624228 (peer_addr_for_me v2:172.21.15.143:0/0) 2023-11-26T07:24:02.675 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:02.671+0000 7f6ad4a36640 1 --2- 172.21.15.143:0/3209624228 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f6ad013e290 0x7f6ad00cf280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:24:02.675 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:02.671+0000 7f6ad5237640 1 --2- 172.21.15.143:0/3209624228 >> v2:172.21.15.188:3300/0 conn(0x7f6ad015f4e0 0x7f6ad01408e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:24:02.675 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:02.675+0000 7f6acffff640 1 -- 172.21.15.143:0/3209624228 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f6ad013e290 msgr2=0x7f6ad00cf280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-26T07:24:02.675 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:02.675+0000 7f6acffff640 1 --2- 172.21.15.143:0/3209624228 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f6ad013e290 0x7f6ad00cf280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:24:02.675 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:02.675+0000 7f6acffff640 1 -- 172.21.15.143:0/3209624228 >> v2:172.21.15.188:3300/0 conn(0x7f6ad015f4e0 msgr2=0x7f6ad01408e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-26T07:24:02.675 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:02.675+0000 7f6acffff640 1 --2- 172.21.15.143:0/3209624228 >> v2:172.21.15.188:3300/0 conn(0x7f6ad015f4e0 0x7f6ad01408e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:24:02.675 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:02.675+0000 7f6acffff640 1 -- 172.21.15.143:0/3209624228 --> v2:172.21.15.143:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6ad006cae0 con 0x7f6ad00cf7c0 2023-11-26T07:24:02.676 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:02.675+0000 7f6acffff640 1 --2- 172.21.15.143:0/3209624228 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f6ad00cf7c0 0x7f6ad015efa0 secure :-1 s=READY pgs=1271 cs=0 l=1 rev1=1 crypto rx=0x7f6ad0072660 tx=0x7f6abc01c210 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b569467dcf83a1b2 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:24:02.678 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:02.675+0000 7f6acf7fe640 1 -- 172.21.15.143:0/3209624228 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6abc0ab600 con 0x7f6ad00cf7c0 2023-11-26T07:24:02.678 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:02.675+0000 7f6acf7fe640 1 -- 172.21.15.143:0/3209624228 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6abc0a7050 con 0x7f6ad00cf7c0 2023-11-26T07:24:02.678 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:02.675+0000 7f6acf7fe640 1 -- 172.21.15.143:0/3209624228 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6abc0ab980 con 0x7f6ad00cf7c0 2023-11-26T07:24:02.678 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:02.675+0000 7f6ad6c9a640 1 -- 172.21.15.143:0/3209624228 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f6ad00cf7c0 msgr2=0x7f6ad015efa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:24:02.678 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:02.675+0000 7f6ad6c9a640 1 --2- 172.21.15.143:0/3209624228 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f6ad00cf7c0 0x7f6ad015efa0 secure :-1 s=READY pgs=1271 cs=0 l=1 rev1=1 crypto rx=0x7f6ad0072660 tx=0x7f6abc01c210 comp rx=0 tx=0).stop 2023-11-26T07:24:02.678 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:02.675+0000 7f6ad6c9a640 1 -- 172.21.15.143:0/3209624228 shutdown_connections 2023-11-26T07:24:02.678 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:02.675+0000 7f6ad6c9a640 1 --2- 172.21.15.143:0/3209624228 >> v2:172.21.15.188:3300/0 conn(0x7f6ad015f4e0 0x7f6ad01408e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:24:02.678 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:02.675+0000 7f6ad6c9a640 1 --2- 172.21.15.143:0/3209624228 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f6ad00cf7c0 0x7f6ad015efa0 unknown :-1 s=CLOSED pgs=1271 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:24:02.678 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:02.675+0000 7f6ad6c9a640 1 --2- 172.21.15.143:0/3209624228 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f6ad013e290 0x7f6ad00cf280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:24:02.679 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:02.675+0000 7f6ad6c9a640 1 -- 172.21.15.143:0/3209624228 >> 172.21.15.143:0/3209624228 conn(0x7f6ad006a890 msgr2=0x7f6ad006aca0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:24:02.679 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:02.675+0000 7f6ad6c9a640 1 -- 172.21.15.143:0/3209624228 shutdown_connections 2023-11-26T07:24:02.679 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:02.675+0000 7f6ad6c9a640 1 -- 172.21.15.143:0/3209624228 wait complete. 2023-11-26T07:24:02.679 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:02.679+0000 7f6ad6c9a640 1 Processor -- start 2023-11-26T07:24:02.679 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:02.679+0000 7f6ad6c9a640 1 -- start start 2023-11-26T07:24:02.682 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:02.679+0000 7f6ad6c9a640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f6ad00cf7c0 0x7f6ad00b3710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:24:02.682 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:02.679+0000 7f6ad6c9a640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f6ad013e290 0x7f6ad013f0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:24:02.682 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:02.679+0000 7f6ad6c9a640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f6ad015f4e0 0x7f6ad013f5e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:24:02.682 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:02.679+0000 7f6ad6c9a640 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6ad013fb20 con 0x7f6ad013e290 2023-11-26T07:24:02.682 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:02.679+0000 7f6ad6c9a640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6ad00aece0 con 0x7f6ad00cf7c0 2023-11-26T07:24:02.682 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:02.679+0000 7f6ad6c9a640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6ad00aefc0 con 0x7f6ad015f4e0 2023-11-26T07:24:02.682 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:02.679+0000 7f6acffff640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f6ad013e290 0x7f6ad013f0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:24:02.683 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:02.679+0000 7f6acffff640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f6ad013e290 0x7f6ad013f0a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:38594/0 (socket says 172.21.15.143:38594) 2023-11-26T07:24:02.683 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:02.679+0000 7f6acffff640 1 -- 172.21.15.143:0/836000052 learned_addr learned my addr 172.21.15.143:0/836000052 (peer_addr_for_me v2:172.21.15.143:0/0) 2023-11-26T07:24:02.683 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:02.679+0000 7f6ad4a36640 1 --2- 172.21.15.143:0/836000052 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f6ad00cf7c0 0x7f6ad00b3710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:24:02.683 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:02.679+0000 7f6ad5237640 1 --2- 172.21.15.143:0/836000052 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f6ad015f4e0 0x7f6ad013f5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:24:02.683 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:02.679+0000 7f6acffff640 1 -- 172.21.15.143:0/836000052 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f6ad015f4e0 msgr2=0x7f6ad013f5e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:24:02.683 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:02.679+0000 7f6acffff640 1 --2- 172.21.15.143:0/836000052 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f6ad015f4e0 0x7f6ad013f5e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:24:02.683 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:02.679+0000 7f6acffff640 1 -- 172.21.15.143:0/836000052 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f6ad00cf7c0 msgr2=0x7f6ad00b3710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:24:02.683 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:02.679+0000 7f6acffff640 1 --2- 172.21.15.143:0/836000052 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f6ad00cf7c0 0x7f6ad00b3710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:24:02.684 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:02.679+0000 7f6acffff640 1 -- 172.21.15.143:0/836000052 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6abc0094d0 con 0x7f6ad013e290 2023-11-26T07:24:02.684 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:02.679+0000 7f6acffff640 1 --2- 172.21.15.143:0/836000052 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f6ad013e290 0x7f6ad013f0a0 secure :-1 s=READY pgs=1272 cs=0 l=1 rev1=1 crypto rx=0x7f6abc01a8c0 tx=0x7f6abc018fb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:24:02.684 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:02.679+0000 7f6acd7fa640 1 -- 172.21.15.143:0/836000052 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6abc0b37d0 con 0x7f6ad013e290 2023-11-26T07:24:02.684 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:02.679+0000 7f6acd7fa640 1 -- 172.21.15.143:0/836000052 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6abc0b3930 con 0x7f6ad013e290 2023-11-26T07:24:02.684 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:02.679+0000 7f6acd7fa640 1 -- 172.21.15.143:0/836000052 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6abc0b3ca0 con 0x7f6ad013e290 2023-11-26T07:24:02.684 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:02.679+0000 7f6ad6c9a640 1 -- 172.21.15.143:0/836000052 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6ad00af2a0 con 0x7f6ad013e290 2023-11-26T07:24:02.684 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:02.683+0000 7f6ad6c9a640 1 -- 172.21.15.143:0/836000052 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6ad00b3c50 con 0x7f6ad013e290 2023-11-26T07:24:02.685 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:02.683+0000 7f6ad6c9a640 1 -- 172.21.15.143:0/836000052 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6ad0067680 con 0x7f6ad013e290 2023-11-26T07:24:02.685 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:02.683+0000 7f6acd7fa640 1 -- 172.21.15.143:0/836000052 <== mon.0 v2:172.21.15.143:3300/0 4 ==== mgrmap(e 16) v1 ==== 85954+0+0 (secure 0 0 0) 0x7f6abc0b2070 con 0x7f6ad013e290 2023-11-26T07:24:02.685 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:02.683+0000 7f6acd7fa640 1 --2- 172.21.15.143:0/836000052 >> [v2:172.21.15.188:6836/3401128553,v1:172.21.15.188:6837/3401128553] conn(0x7f6ab4066590 0x7f6ab4068a50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:24:02.686 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:02.683+0000 7f6acd7fa640 1 -- 172.21.15.143:0/836000052 <== mon.0 v2:172.21.15.143:3300/0 5 ==== osd_map(94..94 src has 1..94) v4 ==== 5029+0+0 (secure 0 0 0) 0x7f6abc11e9f0 con 0x7f6ad013e290 2023-11-26T07:24:02.686 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:02.683+0000 7f6ad4a36640 1 --2- 172.21.15.143:0/836000052 >> [v2:172.21.15.188:6836/3401128553,v1:172.21.15.188:6837/3401128553] conn(0x7f6ab4066590 0x7f6ab4068a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:24:02.686 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:02.687+0000 7f6ad4a36640 1 --2- 172.21.15.143:0/836000052 >> [v2:172.21.15.188:6836/3401128553,v1:172.21.15.188:6837/3401128553] conn(0x7f6ab4066590 0x7f6ab4068a50 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7f6ac400ba30 tx=0x7f6ac4008040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:24:02.688 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:02.687+0000 7f6acd7fa640 1 -- 172.21.15.143:0/836000052 <== mon.0 v2:172.21.15.143:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172649 (secure 0 0 0) 0x7f6abc0ec060 con 0x7f6ad013e290 2023-11-26T07:24:02.866 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:02.867+0000 7f6ad6c9a640 1 -- 172.21.15.143:0/836000052 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f6ad00af750 con 0x7f6ad013e290 2023-11-26T07:24:02.867 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:02.867+0000 7f6acd7fa640 1 -- 172.21.15.143:0/836000052 <== mon.0 v2:172.21.15.143:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 59 v59) v1 ==== 94+0+3711 (secure 0 0 0) 0x7f6abc0ef680 con 0x7f6ad013e290 2023-11-26T07:24:02.867 INFO:teuthology.orchestra.run.smithi143.stdout: 2023-11-26T07:24:02.868 INFO:teuthology.orchestra.run.smithi143.stdout:{"epoch":59,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6305,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.188:6834/3670325785","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6832","nonce":3670325785},{"type":"v1","addr":"172.21.15.188:6834","nonce":3670325785}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":46},{"gid":7366,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.188:6835/4104350986","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6833","nonce":4104350986},{"type":"v1","addr":"172.21.15.188:6835","nonce":4104350986}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":59},{"gid":7467,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.143:6834/4060405000","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6832","nonce":4060405000},{"type":"v1","addr":"172.21.15.143:6834","nonce":4060405000}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":46},{"gid":7704,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.143:6835/2229435747","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6833","nonce":2229435747},{"type":"v1","addr":"172.21.15.143:6835","nonce":2229435747}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":57}],"filesystems":[]} 2023-11-26T07:24:02.868 INFO:teuthology.orchestra.run.smithi143.stderr:dumped fsmap epoch 59 2023-11-26T07:24:02.869 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:02.867+0000 7f6ad6c9a640 1 -- 172.21.15.143:0/836000052 >> [v2:172.21.15.188:6836/3401128553,v1:172.21.15.188:6837/3401128553] conn(0x7f6ab4066590 msgr2=0x7f6ab4068a50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:24:02.869 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:02.867+0000 7f6ad6c9a640 1 --2- 172.21.15.143:0/836000052 >> [v2:172.21.15.188:6836/3401128553,v1:172.21.15.188:6837/3401128553] conn(0x7f6ab4066590 0x7f6ab4068a50 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7f6ac400ba30 tx=0x7f6ac4008040 comp rx=0 tx=0).stop 2023-11-26T07:24:02.869 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:02.867+0000 7f6ad6c9a640 1 -- 172.21.15.143:0/836000052 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f6ad013e290 msgr2=0x7f6ad013f0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:24:02.870 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:02.867+0000 7f6ad6c9a640 1 --2- 172.21.15.143:0/836000052 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f6ad013e290 0x7f6ad013f0a0 secure :-1 s=READY pgs=1272 cs=0 l=1 rev1=1 crypto rx=0x7f6abc01a8c0 tx=0x7f6abc018fb0 comp rx=0 tx=0).stop 2023-11-26T07:24:02.870 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:02.867+0000 7f6ad6c9a640 1 -- 172.21.15.143:0/836000052 shutdown_connections 2023-11-26T07:24:02.870 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:02.867+0000 7f6ad6c9a640 1 --2- 172.21.15.143:0/836000052 >> [v2:172.21.15.188:6836/3401128553,v1:172.21.15.188:6837/3401128553] conn(0x7f6ab4066590 0x7f6ab4068a50 unknown :-1 s=CLOSED pgs=110 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:24:02.870 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:02.867+0000 7f6ad6c9a640 1 --2- 172.21.15.143:0/836000052 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f6ad015f4e0 0x7f6ad013f5e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:24:02.870 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:02.867+0000 7f6ad6c9a640 1 --2- 172.21.15.143:0/836000052 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f6ad013e290 0x7f6ad013f0a0 unknown :-1 s=CLOSED pgs=1272 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:24:02.870 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:02.867+0000 7f6ad6c9a640 1 --2- 172.21.15.143:0/836000052 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f6ad00cf7c0 0x7f6ad00b3710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:24:02.870 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:02.867+0000 7f6ad6c9a640 1 -- 172.21.15.143:0/836000052 >> 172.21.15.143:0/836000052 conn(0x7f6ad006a890 msgr2=0x7f6ad0139d80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:24:02.871 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:02.867+0000 7f6ad6c9a640 1 -- 172.21.15.143:0/836000052 shutdown_connections 2023-11-26T07:24:02.871 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:02.867+0000 7f6ad6c9a640 1 -- 172.21.15.143:0/836000052 wait complete. 2023-11-26T07:24:02.889 INFO:tasks.ceph.mds.a:Restarting daemon 2023-11-26T07:24:02.889 INFO:tasks.ceph.mds.a:Stopping old one... 2023-11-26T07:24:02.890 DEBUG:tasks.ceph.mds.a:waiting for process to exit 2023-11-26T07:24:02.890 INFO:teuthology.orchestra.run:waiting for 300 2023-11-26T07:24:02.890 INFO:tasks.ceph.mds.c:Restarting daemon 2023-11-26T07:24:02.890 INFO:tasks.ceph.mds.c:Stopping old one... 2023-11-26T07:24:02.890 DEBUG:tasks.ceph.mds.c:waiting for process to exit 2023-11-26T07:24:02.890 INFO:teuthology.orchestra.run:waiting for 300 2023-11-26T07:24:02.890 INFO:tasks.ceph.mds.b:Restarting daemon 2023-11-26T07:24:02.891 INFO:tasks.ceph.mds.b:Stopping old one... 2023-11-26T07:24:02.891 DEBUG:tasks.ceph.mds.b:waiting for process to exit 2023-11-26T07:24:02.891 INFO:teuthology.orchestra.run:waiting for 300 2023-11-26T07:24:02.891 INFO:tasks.ceph.mds.d:Restarting daemon 2023-11-26T07:24:02.891 INFO:tasks.ceph.mds.d:Stopping old one... 2023-11-26T07:24:02.891 DEBUG:tasks.ceph.mds.d:waiting for process to exit 2023-11-26T07:24:02.891 INFO:teuthology.orchestra.run:waiting for 300 2023-11-26T07:24:02.902 INFO:tasks.ceph.mds.a:Stopped 2023-11-26T07:24:02.902 DEBUG:teuthology.orchestra.run.smithi143:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i a 2023-11-26T07:24:02.903 INFO:tasks.ceph.mds.c:Stopped 2023-11-26T07:24:02.904 DEBUG:teuthology.orchestra.run.smithi143:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i c 2023-11-26T07:24:02.908 INFO:tasks.ceph.mds.d:Stopped 2023-11-26T07:24:02.908 DEBUG:teuthology.orchestra.run.smithi188:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i d 2023-11-26T07:24:02.909 INFO:tasks.ceph.mds.b:Stopped 2023-11-26T07:24:02.909 DEBUG:teuthology.orchestra.run.smithi188:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i b 2023-11-26T07:24:02.952 INFO:tasks.ceph.mds.a:Started 2023-11-26T07:24:02.953 DEBUG:teuthology.parallel:result is None 2023-11-26T07:24:02.954 INFO:tasks.ceph.mds.d:Started 2023-11-26T07:24:02.954 DEBUG:teuthology.parallel:result is None 2023-11-26T07:24:02.955 INFO:tasks.ceph.mds.c:Started 2023-11-26T07:24:02.955 DEBUG:teuthology.parallel:result is None 2023-11-26T07:24:02.956 INFO:tasks.ceph.mds.b:Started 2023-11-26T07:24:02.956 DEBUG:teuthology.parallel:result is None 2023-11-26T07:24:02.956 DEBUG:teuthology.orchestra.run.smithi143:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist clear 2023-11-26T07:24:03.042 INFO:tasks.ceph.mds.a.smithi143.stdout:starting mds.a at 2023-11-26T07:24:03.042 INFO:tasks.ceph.mds.c.smithi143.stdout:starting mds.c at 2023-11-26T07:24:03.045 INFO:tasks.ceph.mds.d.smithi188.stdout:starting mds.d at 2023-11-26T07:24:03.047 INFO:tasks.ceph.mds.b.smithi188.stdout:starting mds.b at 2023-11-26T07:24:03.102 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:03.103+0000 7fa4898d6640 1 Processor -- start 2023-11-26T07:24:03.102 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:03.103+0000 7fa4898d6640 1 -- start start 2023-11-26T07:24:03.104 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:03.103+0000 7fa4898d6640 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7fa4841596d0 0x7fa484071850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:24:03.104 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:03.103+0000 7fa4898d6640 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7fa4840ce680 0x7fa4840b2870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:24:03.104 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:03.103+0000 7fa4898d6640 1 -- --> v1:172.21.15.143:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa4840b3200 con 0x7fa4840b2db0 2023-11-26T07:24:03.104 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:03.103+0000 7fa4898d6640 1 -- --> v2:172.21.15.143:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa4840cbe00 con 0x7fa4841596d0 2023-11-26T07:24:03.105 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:03.103+0000 7fa4898d6640 1 -- --> v2:172.21.15.188:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa48406c750 con 0x7fa4840ce680 2023-11-26T07:24:03.105 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:03.103+0000 7fa482ffd640 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7fa4841596d0 0x7fa484071850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:24:03.105 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:03.103+0000 7fa482ffd640 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7fa4841596d0 0x7fa484071850 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:38618/0 (socket says 172.21.15.143:38618) 2023-11-26T07:24:03.105 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:03.103+0000 7fa482ffd640 1 -- 172.21.15.143:0/3897871354 learned_addr learned my addr 172.21.15.143:0/3897871354 (peer_addr_for_me v2:172.21.15.143:0/0) 2023-11-26T07:24:03.105 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:03.103+0000 7fa4827fc640 1 --2- 172.21.15.143:0/3897871354 >> v2:172.21.15.188:3300/0 conn(0x7fa4840ce680 0x7fa4840b2870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:24:03.105 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:03.103+0000 7fa482ffd640 1 -- 172.21.15.143:0/3897871354 >> v2:172.21.15.188:3300/0 conn(0x7fa4840ce680 msgr2=0x7fa4840b2870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-26T07:24:03.105 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:03.103+0000 7fa482ffd640 1 --2- 172.21.15.143:0/3897871354 >> v2:172.21.15.188:3300/0 conn(0x7fa4840ce680 0x7fa4840b2870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:24:03.105 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:03.103+0000 7fa482ffd640 1 -- 172.21.15.143:0/3897871354 >> v1:172.21.15.143:6789/0 conn(0x7fa4840b2db0 legacy=0x7fa48415d2c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:24:03.105 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:03.103+0000 7fa482ffd640 1 -- 172.21.15.143:0/3897871354 --> v2:172.21.15.143:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa4840b3550 con 0x7fa4841596d0 2023-11-26T07:24:03.108 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:03.107+0000 7fa482ffd640 1 --2- 172.21.15.143:0/3897871354 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fa4841596d0 0x7fa484071850 secure :-1 s=READY pgs=1276 cs=0 l=1 rev1=1 crypto rx=0x7fa4840cbf40 tx=0x7fa47401d610 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1d79ed196992f5c2 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:24:03.108 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:03.107+0000 7fa481ffb640 1 -- 172.21.15.143:0/3897871354 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa4740ac670 con 0x7fa4841596d0 2023-11-26T07:24:03.108 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:03.107+0000 7fa481ffb640 1 -- 172.21.15.143:0/3897871354 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa4740ac7d0 con 0x7fa4841596d0 2023-11-26T07:24:03.108 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:03.107+0000 7fa481ffb640 1 -- 172.21.15.143:0/3897871354 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa4740acb60 con 0x7fa4841596d0 2023-11-26T07:24:03.109 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:03.107+0000 7fa4898d6640 1 -- 172.21.15.143:0/3897871354 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fa4841596d0 msgr2=0x7fa484071850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:24:03.109 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:03.107+0000 7fa4898d6640 1 --2- 172.21.15.143:0/3897871354 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fa4841596d0 0x7fa484071850 secure :-1 s=READY pgs=1276 cs=0 l=1 rev1=1 crypto rx=0x7fa4840cbf40 tx=0x7fa47401d610 comp rx=0 tx=0).stop 2023-11-26T07:24:03.109 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:03.107+0000 7fa4898d6640 1 -- 172.21.15.143:0/3897871354 shutdown_connections 2023-11-26T07:24:03.109 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:03.107+0000 7fa4898d6640 1 --2- 172.21.15.143:0/3897871354 >> v2:172.21.15.188:3300/0 conn(0x7fa4840ce680 0x7fa4840b2870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:24:03.109 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:03.107+0000 7fa4898d6640 1 --2- 172.21.15.143:0/3897871354 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fa4841596d0 0x7fa484071850 unknown :-1 s=CLOSED pgs=1276 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:24:03.110 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:03.107+0000 7fa4898d6640 1 -- 172.21.15.143:0/3897871354 >> 172.21.15.143:0/3897871354 conn(0x7fa48406a890 msgr2=0x7fa48406aca0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:24:03.110 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:03.107+0000 7fa4898d6640 1 -- 172.21.15.143:0/3897871354 shutdown_connections 2023-11-26T07:24:03.110 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:03.107+0000 7fa4898d6640 1 -- 172.21.15.143:0/3897871354 wait complete. 2023-11-26T07:24:03.110 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:03.111+0000 7fa4898d6640 1 Processor -- start 2023-11-26T07:24:03.111 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:03.111+0000 7fa4898d6640 1 -- start start 2023-11-26T07:24:03.113 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:03.111+0000 7fa4898d6640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fa4840b2db0 0x7fa48414b9d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:24:03.113 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:03.111+0000 7fa4898d6640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fa4840ce680 0x7fa484167340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:24:03.113 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:03.111+0000 7fa4898d6640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fa4841596d0 0x7fa4841571a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:24:03.113 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:03.111+0000 7fa4898d6640 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa4840afd30 con 0x7fa4840ce680 2023-11-26T07:24:03.113 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:03.111+0000 7fa4898d6640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa4840af420 con 0x7fa4841596d0 2023-11-26T07:24:03.113 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:03.111+0000 7fa4898d6640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa484167c90 con 0x7fa4840b2db0 2023-11-26T07:24:03.114 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:03.111+0000 7fa4827fc640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fa4840ce680 0x7fa484167340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:24:03.114 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:03.111+0000 7fa4827fc640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fa4840ce680 0x7fa484167340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:38628/0 (socket says 172.21.15.143:38628) 2023-11-26T07:24:03.114 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:03.111+0000 7fa4827fc640 1 -- 172.21.15.143:0/529776814 learned_addr learned my addr 172.21.15.143:0/529776814 (peer_addr_for_me v2:172.21.15.143:0/0) 2023-11-26T07:24:03.114 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:03.111+0000 7fa482ffd640 1 --2- 172.21.15.143:0/529776814 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fa4840b2db0 0x7fa48414b9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:24:03.114 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:03.111+0000 7fa4837fe640 1 --2- 172.21.15.143:0/529776814 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fa4841596d0 0x7fa4841571a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:24:03.114 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:03.111+0000 7fa4827fc640 1 -- 172.21.15.143:0/529776814 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fa4840b2db0 msgr2=0x7fa48414b9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:24:03.114 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:03.111+0000 7fa4827fc640 1 --2- 172.21.15.143:0/529776814 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fa4840b2db0 0x7fa48414b9d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:24:03.114 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:03.111+0000 7fa4827fc640 1 -- 172.21.15.143:0/529776814 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fa4841596d0 msgr2=0x7fa4841571a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:24:03.114 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:03.111+0000 7fa4827fc640 1 --2- 172.21.15.143:0/529776814 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fa4841596d0 0x7fa4841571a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:24:03.115 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:03.111+0000 7fa4827fc640 1 -- 172.21.15.143:0/529776814 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa4740094d0 con 0x7fa4840ce680 2023-11-26T07:24:03.115 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:03.111+0000 7fa4827fc640 1 --2- 172.21.15.143:0/529776814 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fa4840ce680 0x7fa484167340 secure :-1 s=READY pgs=1277 cs=0 l=1 rev1=1 crypto rx=0x7fa47800e9e0 tx=0x7fa47800eea0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:24:03.115 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:03.111+0000 7fa46bfff640 1 -- 172.21.15.143:0/529776814 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa4780096b0 con 0x7fa4840ce680 2023-11-26T07:24:03.115 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:03.111+0000 7fa46bfff640 1 -- 172.21.15.143:0/529776814 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa47800c9d0 con 0x7fa4840ce680 2023-11-26T07:24:03.115 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:03.111+0000 7fa4898d6640 1 -- 172.21.15.143:0/529776814 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa4840af670 con 0x7fa4840ce680 2023-11-26T07:24:03.115 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:03.111+0000 7fa46bfff640 1 -- 172.21.15.143:0/529776814 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa47800cd60 con 0x7fa4840ce680 2023-11-26T07:24:03.115 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:03.111+0000 7fa4898d6640 1 -- 172.21.15.143:0/529776814 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa48414bf10 con 0x7fa4840ce680 2023-11-26T07:24:03.116 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:03.111+0000 7fa4898d6640 1 -- 172.21.15.143:0/529776814 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa4840afe70 con 0x7fa4840ce680 2023-11-26T07:24:03.116 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:03.115+0000 7fa46bfff640 1 -- 172.21.15.143:0/529776814 <== mon.0 v2:172.21.15.143:3300/0 4 ==== mgrmap(e 16) v1 ==== 85954+0+0 (secure 0 0 0) 0x7fa478013470 con 0x7fa4840ce680 2023-11-26T07:24:03.116 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:03.115+0000 7fa46bfff640 1 --2- 172.21.15.143:0/529776814 >> [v2:172.21.15.188:6836/3401128553,v1:172.21.15.188:6837/3401128553] conn(0x7fa470066590 0x7fa470068a50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:24:03.116 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:03.115+0000 7fa46bfff640 1 -- 172.21.15.143:0/529776814 <== mon.0 v2:172.21.15.143:3300/0 5 ==== osd_map(94..94 src has 1..94) v4 ==== 5029+0+0 (secure 0 0 0) 0x7fa478083f40 con 0x7fa4840ce680 2023-11-26T07:24:03.116 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:03.115+0000 7fa482ffd640 1 --2- 172.21.15.143:0/529776814 >> [v2:172.21.15.188:6836/3401128553,v1:172.21.15.188:6837/3401128553] conn(0x7fa470066590 0x7fa470068a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:24:03.117 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:03.115+0000 7fa482ffd640 1 --2- 172.21.15.143:0/529776814 >> [v2:172.21.15.188:6836/3401128553,v1:172.21.15.188:6837/3401128553] conn(0x7fa470066590 0x7fa470068a50 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7fa4740b0000 tx=0x7fa474008040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:24:03.119 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:03.119+0000 7fa46bfff640 1 -- 172.21.15.143:0/529776814 <== mon.0 v2:172.21.15.143:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172649 (secure 0 0 0) 0x7fa4840afe70 con 0x7fa4840ce680 2023-11-26T07:24:03.281 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:03.279+0000 7fa4898d6640 1 -- 172.21.15.143:0/529776814 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "osd blocklist clear"} v 0) v1 -- 0x7fa4840afe70 con 0x7fa4840ce680 2023-11-26T07:24:04.531 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:04.531+0000 7fa46bfff640 1 -- 172.21.15.143:0/529776814 <== mon.0 v2:172.21.15.143:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist clear"}]=0 removed all blocklist entries v99) v1 ==== 97+0+0 (secure 0 0 0) 0x7fa4840afe70 con 0x7fa4840ce680 2023-11-26T07:24:04.531 INFO:teuthology.orchestra.run.smithi143.stderr: removed all blocklist entries 2023-11-26T07:24:04.534 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:04.531+0000 7fa4898d6640 1 -- 172.21.15.143:0/529776814 >> [v2:172.21.15.188:6836/3401128553,v1:172.21.15.188:6837/3401128553] conn(0x7fa470066590 msgr2=0x7fa470068a50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:24:04.534 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:04.531+0000 7fa4898d6640 1 --2- 172.21.15.143:0/529776814 >> [v2:172.21.15.188:6836/3401128553,v1:172.21.15.188:6837/3401128553] conn(0x7fa470066590 0x7fa470068a50 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7fa4740b0000 tx=0x7fa474008040 comp rx=0 tx=0).stop 2023-11-26T07:24:04.534 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:04.531+0000 7fa4898d6640 1 -- 172.21.15.143:0/529776814 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fa4840ce680 msgr2=0x7fa484167340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:24:04.535 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:04.531+0000 7fa4898d6640 1 --2- 172.21.15.143:0/529776814 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fa4840ce680 0x7fa484167340 secure :-1 s=READY pgs=1277 cs=0 l=1 rev1=1 crypto rx=0x7fa47800e9e0 tx=0x7fa47800eea0 comp rx=0 tx=0).stop 2023-11-26T07:24:04.535 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:04.531+0000 7fa4898d6640 1 -- 172.21.15.143:0/529776814 shutdown_connections 2023-11-26T07:24:04.535 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:04.531+0000 7fa4898d6640 1 --2- 172.21.15.143:0/529776814 >> [v2:172.21.15.188:6836/3401128553,v1:172.21.15.188:6837/3401128553] conn(0x7fa470066590 0x7fa470068a50 unknown :-1 s=CLOSED pgs=111 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:24:04.535 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:04.531+0000 7fa4898d6640 1 --2- 172.21.15.143:0/529776814 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fa4841596d0 0x7fa4841571a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:24:04.535 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:04.531+0000 7fa4898d6640 1 --2- 172.21.15.143:0/529776814 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fa4840ce680 0x7fa484167340 unknown :-1 s=CLOSED pgs=1277 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:24:04.535 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:04.531+0000 7fa4898d6640 1 --2- 172.21.15.143:0/529776814 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fa4840b2db0 0x7fa48414b9d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:24:04.535 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:04.531+0000 7fa4898d6640 1 -- 172.21.15.143:0/529776814 >> 172.21.15.143:0/529776814 conn(0x7fa48406a890 msgr2=0x7fa4840b01b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:24:04.535 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:04.535+0000 7fa4898d6640 1 -- 172.21.15.143:0/529776814 shutdown_connections 2023-11-26T07:24:04.535 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:04.535+0000 7fa4898d6640 1 -- 172.21.15.143:0/529776814 wait complete. 2023-11-26T07:24:04.554 DEBUG:teuthology.orchestra.run.smithi143:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth ls --format=json-pretty 2023-11-26T07:24:04.692 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:04.691+0000 7f4f309ec640 1 Processor -- start 2023-11-26T07:24:04.692 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:04.691+0000 7f4f309ec640 1 -- start start 2023-11-26T07:24:04.694 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:04.691+0000 7f4f309ec640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f4f2c13e290 0x7f4f2c0cf280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:24:04.694 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:04.691+0000 7f4f309ec640 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7f4f2c0cf7c0 0x7f4f2c15efa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:24:04.694 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:04.691+0000 7f4f309ec640 1 -- --> v1:172.21.15.143:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4f2c06c520 con 0x7f4f2c15f4e0 2023-11-26T07:24:04.694 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:04.691+0000 7f4f309ec640 1 -- --> v2:172.21.15.143:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4f2c0cfc40 con 0x7f4f2c0cf7c0 2023-11-26T07:24:04.694 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:04.691+0000 7f4f309ec640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4f2c06c800 con 0x7f4f2c13e290 2023-11-26T07:24:04.695 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:04.691+0000 7f4f29d9b640 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7f4f2c0cf7c0 0x7f4f2c15efa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:24:04.695 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:04.691+0000 7f4f2ad9d640 1 --1- >> v1:172.21.15.143:6789/0 conn(0x7f4f2c15f4e0 0x7f4f2c1408e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.143:6789/0 says I am v1:172.21.15.143:42378/0 (socket says 172.21.15.143:42378) 2023-11-26T07:24:04.695 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:04.695+0000 7f4f2ad9d640 1 -- 172.21.15.143:0/2899660558 learned_addr learned my addr 172.21.15.143:0/2899660558 (peer_addr_for_me v1:172.21.15.143:0/0) 2023-11-26T07:24:04.695 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:04.695+0000 7f4f2a59c640 1 --2- 172.21.15.143:0/2899660558 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f4f2c13e290 0x7f4f2c0cf280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:24:04.695 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:04.695+0000 7f4f29d9b640 1 -- 172.21.15.143:0/2899660558 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f4f2c13e290 msgr2=0x7f4f2c0cf280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-26T07:24:04.695 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:04.695+0000 7f4f29d9b640 1 --2- 172.21.15.143:0/2899660558 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f4f2c13e290 0x7f4f2c0cf280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:24:04.695 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:04.695+0000 7f4f29d9b640 1 -- 172.21.15.143:0/2899660558 >> v1:172.21.15.143:6789/0 conn(0x7f4f2c15f4e0 legacy=0x7f4f2c1408e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:24:04.695 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:04.695+0000 7f4f29d9b640 1 -- 172.21.15.143:0/2899660558 --> v2:172.21.15.143:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4f2c06cae0 con 0x7f4f2c0cf7c0 2023-11-26T07:24:04.697 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:04.695+0000 7f4f29d9b640 1 --2- 172.21.15.143:0/2899660558 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f4f2c0cf7c0 0x7f4f2c15efa0 secure :-1 s=READY pgs=1279 cs=0 l=1 rev1=1 crypto rx=0x7f4f1c0097c0 tx=0x7f4f1c01c450 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a2a3e2d5e075863b server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:24:04.697 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:04.695+0000 7f4f2959a640 1 -- 172.21.15.143:0/2899660558 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4f1c0ab650 con 0x7f4f2c0cf7c0 2023-11-26T07:24:04.697 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:04.695+0000 7f4f2959a640 1 -- 172.21.15.143:0/2899660558 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4f1c0ab7b0 con 0x7f4f2c0cf7c0 2023-11-26T07:24:04.698 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:04.695+0000 7f4f2959a640 1 -- 172.21.15.143:0/2899660558 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4f1c0abb40 con 0x7f4f2c0cf7c0 2023-11-26T07:24:04.698 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:04.699+0000 7f4f309ec640 1 -- 172.21.15.143:0/2899660558 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f4f2c0cf7c0 msgr2=0x7f4f2c15efa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:24:04.699 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:04.699+0000 7f4f309ec640 1 --2- 172.21.15.143:0/2899660558 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f4f2c0cf7c0 0x7f4f2c15efa0 secure :-1 s=READY pgs=1279 cs=0 l=1 rev1=1 crypto rx=0x7f4f1c0097c0 tx=0x7f4f1c01c450 comp rx=0 tx=0).stop 2023-11-26T07:24:04.699 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:04.699+0000 7f4f309ec640 1 -- 172.21.15.143:0/2899660558 shutdown_connections 2023-11-26T07:24:04.699 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:04.699+0000 7f4f309ec640 1 --2- 172.21.15.143:0/2899660558 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f4f2c0cf7c0 0x7f4f2c15efa0 unknown :-1 s=CLOSED pgs=1279 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:24:04.699 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:04.699+0000 7f4f309ec640 1 --2- 172.21.15.143:0/2899660558 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f4f2c13e290 0x7f4f2c0cf280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:24:04.699 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:04.699+0000 7f4f309ec640 1 -- 172.21.15.143:0/2899660558 >> 172.21.15.143:0/2899660558 conn(0x7f4f2c06a890 msgr2=0x7f4f2c06aca0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:24:04.699 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:04.699+0000 7f4f309ec640 1 -- 172.21.15.143:0/2899660558 shutdown_connections 2023-11-26T07:24:04.699 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:04.699+0000 7f4f309ec640 1 -- 172.21.15.143:0/2899660558 wait complete. 2023-11-26T07:24:04.700 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:04.699+0000 7f4f309ec640 1 Processor -- start 2023-11-26T07:24:04.700 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:04.699+0000 7f4f309ec640 1 -- start start 2023-11-26T07:24:04.702 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:04.699+0000 7f4f309ec640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f4f2c0cf7c0 0x7f4f2c14fe40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:24:04.703 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:04.699+0000 7f4f309ec640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f4f2c13e290 0x7f4f2c128810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:24:04.703 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:04.699+0000 7f4f309ec640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f4f2c15f4e0 0x7f4f2c154750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:24:04.703 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:04.699+0000 7f4f309ec640 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4f2c0aed90 con 0x7f4f2c15f4e0 2023-11-26T07:24:04.703 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:04.699+0000 7f4f309ec640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4f2c15c8f0 con 0x7f4f2c0cf7c0 2023-11-26T07:24:04.703 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:04.699+0000 7f4f309ec640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4f2c150d70 con 0x7f4f2c13e290 2023-11-26T07:24:04.703 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:04.699+0000 7f4f2ad9d640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f4f2c15f4e0 0x7f4f2c154750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:24:04.703 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:04.699+0000 7f4f2ad9d640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f4f2c15f4e0 0x7f4f2c154750 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:33590/0 (socket says 172.21.15.143:33590) 2023-11-26T07:24:04.703 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:04.699+0000 7f4f2ad9d640 1 -- 172.21.15.143:0/3972286403 learned_addr learned my addr 172.21.15.143:0/3972286403 (peer_addr_for_me v2:172.21.15.143:0/0) 2023-11-26T07:24:04.704 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:04.699+0000 7f4f2a59c640 1 --2- 172.21.15.143:0/3972286403 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f4f2c0cf7c0 0x7f4f2c14fe40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:24:04.704 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:04.699+0000 7f4f29d9b640 1 --2- 172.21.15.143:0/3972286403 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f4f2c13e290 0x7f4f2c128810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:24:04.704 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:04.703+0000 7f4f2ad9d640 1 -- 172.21.15.143:0/3972286403 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f4f2c13e290 msgr2=0x7f4f2c128810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:24:04.704 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:04.703+0000 7f4f2ad9d640 1 --2- 172.21.15.143:0/3972286403 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f4f2c13e290 0x7f4f2c128810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:24:04.704 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:04.703+0000 7f4f2ad9d640 1 -- 172.21.15.143:0/3972286403 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f4f2c0cf7c0 msgr2=0x7f4f2c14fe40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:24:04.704 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:04.703+0000 7f4f2ad9d640 1 --2- 172.21.15.143:0/3972286403 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f4f2c0cf7c0 0x7f4f2c14fe40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:24:04.704 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:04.703+0000 7f4f2ad9d640 1 -- 172.21.15.143:0/3972286403 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4f1c009420 con 0x7f4f2c15f4e0 2023-11-26T07:24:04.704 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:04.703+0000 7f4f2ad9d640 1 --2- 172.21.15.143:0/3972286403 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f4f2c15f4e0 0x7f4f2c154750 secure :-1 s=READY pgs=1280 cs=0 l=1 rev1=1 crypto rx=0x7f4f2400ca80 tx=0x7f4f2400cf40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:24:04.705 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:04.703+0000 7f4f22ffd640 1 -- 172.21.15.143:0/3972286403 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4f2400ac30 con 0x7f4f2c15f4e0 2023-11-26T07:24:04.705 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:04.703+0000 7f4f22ffd640 1 -- 172.21.15.143:0/3972286403 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4f2400ad90 con 0x7f4f2c15f4e0 2023-11-26T07:24:04.705 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:04.703+0000 7f4f22ffd640 1 -- 172.21.15.143:0/3972286403 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4f24012600 con 0x7f4f2c15f4e0 2023-11-26T07:24:04.705 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:04.703+0000 7f4f309ec640 1 -- 172.21.15.143:0/3972286403 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4f2c1504d0 con 0x7f4f2c15f4e0 2023-11-26T07:24:04.705 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:04.703+0000 7f4f309ec640 1 -- 172.21.15.143:0/3972286403 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4f2c150a90 con 0x7f4f2c15f4e0 2023-11-26T07:24:04.706 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:04.703+0000 7f4f309ec640 1 -- 172.21.15.143:0/3972286403 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4f2c0aeed0 con 0x7f4f2c15f4e0 2023-11-26T07:24:04.706 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:04.703+0000 7f4f22ffd640 1 -- 172.21.15.143:0/3972286403 <== mon.0 v2:172.21.15.143:3300/0 4 ==== mgrmap(e 16) v1 ==== 85954+0+0 (secure 0 0 0) 0x7f4f24007070 con 0x7f4f2c15f4e0 2023-11-26T07:24:04.707 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:04.703+0000 7f4f22ffd640 1 --2- 172.21.15.143:0/3972286403 >> [v2:172.21.15.188:6836/3401128553,v1:172.21.15.188:6837/3401128553] conn(0x7f4f0c066590 0x7f4f0c068a50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:24:04.707 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:04.703+0000 7f4f22ffd640 1 -- 172.21.15.143:0/3972286403 <== mon.0 v2:172.21.15.143:3300/0 5 ==== osd_map(99..99 src has 1..99) v4 ==== 4857+0+0 (secure 0 0 0) 0x7f4f240839b0 con 0x7f4f2c15f4e0 2023-11-26T07:24:04.707 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:04.703+0000 7f4f2a59c640 1 --2- 172.21.15.143:0/3972286403 >> [v2:172.21.15.188:6836/3401128553,v1:172.21.15.188:6837/3401128553] conn(0x7f4f0c066590 0x7f4f0c068a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:24:04.707 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:04.707+0000 7f4f2a59c640 1 --2- 172.21.15.143:0/3972286403 >> [v2:172.21.15.188:6836/3401128553,v1:172.21.15.188:6837/3401128553] conn(0x7f4f0c066590 0x7f4f0c068a50 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7f4f14009b60 tx=0x7f4f14009190 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:24:04.709 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:04.707+0000 7f4f22ffd640 1 -- 172.21.15.143:0/3972286403 <== mon.0 v2:172.21.15.143:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172649 (secure 0 0 0) 0x7f4f2c0aeed0 con 0x7f4f2c15f4e0 2023-11-26T07:24:04.909 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:04.907+0000 7f4f309ec640 1 -- 172.21.15.143:0/3972286403 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "auth ls", "format": "json-pretty"} v 0) v1 -- 0x7f4f2c0aeed0 con 0x7f4f2c15f4e0 2023-11-26T07:24:04.910 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:04.911+0000 7f4f22ffd640 1 -- 172.21.15.143:0/3972286403 <== mon.0 v2:172.21.15.143:3300/0 7 ==== mon_command_ack([{"prefix": "auth ls", "format": "json-pretty"}]=0 v19) v1 ==== 80+0+5304 (secure 0 0 0) 0x7f4f2c0aeed0 con 0x7f4f2c15f4e0 2023-11-26T07:24:04.910 INFO:teuthology.orchestra.run.smithi143.stdout: 2023-11-26T07:24:04.910 INFO:teuthology.orchestra.run.smithi143.stdout:{ 2023-11-26T07:24:04.911 INFO:teuthology.orchestra.run.smithi143.stdout: "auth_dump": [ 2023-11-26T07:24:04.911 INFO:teuthology.orchestra.run.smithi143.stdout: { 2023-11-26T07:24:04.911 INFO:teuthology.orchestra.run.smithi143.stdout: "entity": "mds.a", 2023-11-26T07:24:04.911 INFO:teuthology.orchestra.run.smithi143.stdout: "key": "AQDl8GJldeBsOBAAhk75XZnQRURKcr0MrW17Lg==", 2023-11-26T07:24:04.911 INFO:teuthology.orchestra.run.smithi143.stdout: "caps": { 2023-11-26T07:24:04.911 INFO:teuthology.orchestra.run.smithi143.stdout: "mds": "allow", 2023-11-26T07:24:04.911 INFO:teuthology.orchestra.run.smithi143.stdout: "mgr": "allow *", 2023-11-26T07:24:04.911 INFO:teuthology.orchestra.run.smithi143.stdout: "mon": "allow *", 2023-11-26T07:24:04.912 INFO:teuthology.orchestra.run.smithi143.stdout: "osd": "allow *" 2023-11-26T07:24:04.912 INFO:teuthology.orchestra.run.smithi143.stdout: } 2023-11-26T07:24:04.912 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2023-11-26T07:24:04.912 INFO:teuthology.orchestra.run.smithi143.stdout: { 2023-11-26T07:24:04.912 INFO:teuthology.orchestra.run.smithi143.stdout: "entity": "mds.b", 2023-11-26T07:24:04.912 INFO:teuthology.orchestra.run.smithi143.stdout: "key": "AQDm8GJl0lvhCBAAXuvHKoB8TAhp0bJ4DcV94g==", 2023-11-26T07:24:04.912 INFO:teuthology.orchestra.run.smithi143.stdout: "caps": { 2023-11-26T07:24:04.912 INFO:teuthology.orchestra.run.smithi143.stdout: "mds": "allow", 2023-11-26T07:24:04.912 INFO:teuthology.orchestra.run.smithi143.stdout: "mgr": "allow *", 2023-11-26T07:24:04.912 INFO:teuthology.orchestra.run.smithi143.stdout: "mon": "allow *", 2023-11-26T07:24:04.913 INFO:teuthology.orchestra.run.smithi143.stdout: "osd": "allow *" 2023-11-26T07:24:04.913 INFO:teuthology.orchestra.run.smithi143.stdout: } 2023-11-26T07:24:04.913 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2023-11-26T07:24:04.913 INFO:teuthology.orchestra.run.smithi143.stdout: { 2023-11-26T07:24:04.913 INFO:teuthology.orchestra.run.smithi143.stdout: "entity": "mds.c", 2023-11-26T07:24:04.913 INFO:teuthology.orchestra.run.smithi143.stdout: "key": "AQDm8GJluQ0PBBAA23EmVdD3bVk52h6izUXDRQ==", 2023-11-26T07:24:04.913 INFO:teuthology.orchestra.run.smithi143.stdout: "caps": { 2023-11-26T07:24:04.913 INFO:teuthology.orchestra.run.smithi143.stdout: "mds": "allow", 2023-11-26T07:24:04.913 INFO:teuthology.orchestra.run.smithi143.stdout: "mgr": "allow *", 2023-11-26T07:24:04.913 INFO:teuthology.orchestra.run.smithi143.stdout: "mon": "allow *", 2023-11-26T07:24:04.914 INFO:teuthology.orchestra.run.smithi143.stdout: "osd": "allow *" 2023-11-26T07:24:04.914 INFO:teuthology.orchestra.run.smithi143.stdout: } 2023-11-26T07:24:04.914 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2023-11-26T07:24:04.914 INFO:teuthology.orchestra.run.smithi143.stdout: { 2023-11-26T07:24:04.914 INFO:teuthology.orchestra.run.smithi143.stdout: "entity": "mds.d", 2023-11-26T07:24:04.914 INFO:teuthology.orchestra.run.smithi143.stdout: "key": "AQDm8GJlkt9nEBAAg+LYtKVNHAN/fwYjEcNJHQ==", 2023-11-26T07:24:04.914 INFO:teuthology.orchestra.run.smithi143.stdout: "caps": { 2023-11-26T07:24:04.914 INFO:teuthology.orchestra.run.smithi143.stdout: "mds": "allow", 2023-11-26T07:24:04.914 INFO:teuthology.orchestra.run.smithi143.stdout: "mgr": "allow *", 2023-11-26T07:24:04.915 INFO:teuthology.orchestra.run.smithi143.stdout: "mon": "allow *", 2023-11-26T07:24:04.915 INFO:teuthology.orchestra.run.smithi143.stdout: "osd": "allow *" 2023-11-26T07:24:04.915 INFO:teuthology.orchestra.run.smithi143.stdout: } 2023-11-26T07:24:04.915 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2023-11-26T07:24:04.915 INFO:teuthology.orchestra.run.smithi143.stdout: { 2023-11-26T07:24:04.915 INFO:teuthology.orchestra.run.smithi143.stdout: "entity": "osd.0", 2023-11-26T07:24:04.915 INFO:teuthology.orchestra.run.smithi143.stdout: "key": "AQDp8GJlNscZLhAA31zK4vTcnxwiRIRgxzewpA==", 2023-11-26T07:24:04.915 INFO:teuthology.orchestra.run.smithi143.stdout: "caps": { 2023-11-26T07:24:04.916 INFO:teuthology.orchestra.run.smithi143.stdout: "mgr": "allow profile osd", 2023-11-26T07:24:04.916 INFO:teuthology.orchestra.run.smithi143.stdout: "mon": "allow profile osd", 2023-11-26T07:24:04.916 INFO:teuthology.orchestra.run.smithi143.stdout: "osd": "allow *" 2023-11-26T07:24:04.916 INFO:teuthology.orchestra.run.smithi143.stdout: } 2023-11-26T07:24:04.916 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2023-11-26T07:24:04.916 INFO:teuthology.orchestra.run.smithi143.stdout: { 2023-11-26T07:24:04.916 INFO:teuthology.orchestra.run.smithi143.stdout: "entity": "osd.1", 2023-11-26T07:24:04.916 INFO:teuthology.orchestra.run.smithi143.stdout: "key": "AQDs8GJlsyKUNBAA+dQ1ElGPrdP1GV7lk4JQjQ==", 2023-11-26T07:24:04.916 INFO:teuthology.orchestra.run.smithi143.stdout: "caps": { 2023-11-26T07:24:04.916 INFO:teuthology.orchestra.run.smithi143.stdout: "mgr": "allow profile osd", 2023-11-26T07:24:04.917 INFO:teuthology.orchestra.run.smithi143.stdout: "mon": "allow profile osd", 2023-11-26T07:24:04.917 INFO:teuthology.orchestra.run.smithi143.stdout: "osd": "allow *" 2023-11-26T07:24:04.917 INFO:teuthology.orchestra.run.smithi143.stdout: } 2023-11-26T07:24:04.917 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2023-11-26T07:24:04.917 INFO:teuthology.orchestra.run.smithi143.stdout: { 2023-11-26T07:24:04.917 INFO:teuthology.orchestra.run.smithi143.stdout: "entity": "osd.2", 2023-11-26T07:24:04.917 INFO:teuthology.orchestra.run.smithi143.stdout: "key": "AQDv8GJlXsy7OhAAJJdldsodgGZVtyV2HuyC4A==", 2023-11-26T07:24:04.917 INFO:teuthology.orchestra.run.smithi143.stdout: "caps": { 2023-11-26T07:24:04.917 INFO:teuthology.orchestra.run.smithi143.stdout: "mgr": "allow profile osd", 2023-11-26T07:24:04.917 INFO:teuthology.orchestra.run.smithi143.stdout: "mon": "allow profile osd", 2023-11-26T07:24:04.917 INFO:teuthology.orchestra.run.smithi143.stdout: "osd": "allow *" 2023-11-26T07:24:04.917 INFO:teuthology.orchestra.run.smithi143.stdout: } 2023-11-26T07:24:04.917 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2023-11-26T07:24:04.918 INFO:teuthology.orchestra.run.smithi143.stdout: { 2023-11-26T07:24:04.918 INFO:teuthology.orchestra.run.smithi143.stdout: "entity": "osd.3", 2023-11-26T07:24:04.918 INFO:teuthology.orchestra.run.smithi143.stdout: "key": "AQDz8GJloLhMBxAAOWD9ec2y4c8nkdJmQnne8Q==", 2023-11-26T07:24:04.918 INFO:teuthology.orchestra.run.smithi143.stdout: "caps": { 2023-11-26T07:24:04.918 INFO:teuthology.orchestra.run.smithi143.stdout: "mgr": "allow profile osd", 2023-11-26T07:24:04.918 INFO:teuthology.orchestra.run.smithi143.stdout: "mon": "allow profile osd", 2023-11-26T07:24:04.918 INFO:teuthology.orchestra.run.smithi143.stdout: "osd": "allow *" 2023-11-26T07:24:04.918 INFO:teuthology.orchestra.run.smithi143.stdout: } 2023-11-26T07:24:04.918 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2023-11-26T07:24:04.918 INFO:teuthology.orchestra.run.smithi143.stdout: { 2023-11-26T07:24:04.918 INFO:teuthology.orchestra.run.smithi143.stdout: "entity": "osd.4", 2023-11-26T07:24:04.918 INFO:teuthology.orchestra.run.smithi143.stdout: "key": "AQD58GJlDlZsDBAAZNauAPPPfAZ+KIqSmgSQhg==", 2023-11-26T07:24:04.919 INFO:teuthology.orchestra.run.smithi143.stdout: "caps": { 2023-11-26T07:24:04.919 INFO:teuthology.orchestra.run.smithi143.stdout: "mgr": "allow profile osd", 2023-11-26T07:24:04.919 INFO:teuthology.orchestra.run.smithi143.stdout: "mon": "allow profile osd", 2023-11-26T07:24:04.919 INFO:teuthology.orchestra.run.smithi143.stdout: "osd": "allow *" 2023-11-26T07:24:04.919 INFO:teuthology.orchestra.run.smithi143.stdout: } 2023-11-26T07:24:04.919 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2023-11-26T07:24:04.919 INFO:teuthology.orchestra.run.smithi143.stdout: { 2023-11-26T07:24:04.919 INFO:teuthology.orchestra.run.smithi143.stdout: "entity": "osd.5", 2023-11-26T07:24:04.919 INFO:teuthology.orchestra.run.smithi143.stdout: "key": "AQD88GJlNE4mExAA5jYkCa0tE3oR6rgK1Dt5cA==", 2023-11-26T07:24:04.919 INFO:teuthology.orchestra.run.smithi143.stdout: "caps": { 2023-11-26T07:24:04.919 INFO:teuthology.orchestra.run.smithi143.stdout: "mgr": "allow profile osd", 2023-11-26T07:24:04.919 INFO:teuthology.orchestra.run.smithi143.stdout: "mon": "allow profile osd", 2023-11-26T07:24:04.920 INFO:teuthology.orchestra.run.smithi143.stdout: "osd": "allow *" 2023-11-26T07:24:04.920 INFO:teuthology.orchestra.run.smithi143.stdout: } 2023-11-26T07:24:04.920 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2023-11-26T07:24:04.920 INFO:teuthology.orchestra.run.smithi143.stdout: { 2023-11-26T07:24:04.920 INFO:teuthology.orchestra.run.smithi143.stdout: "entity": "osd.6", 2023-11-26T07:24:04.920 INFO:teuthology.orchestra.run.smithi143.stdout: "key": "AQD/8GJlh0lYGRAArHalCyZ36UvBOVMZor+N/g==", 2023-11-26T07:24:04.920 INFO:teuthology.orchestra.run.smithi143.stdout: "caps": { 2023-11-26T07:24:04.920 INFO:teuthology.orchestra.run.smithi143.stdout: "mgr": "allow profile osd", 2023-11-26T07:24:04.920 INFO:teuthology.orchestra.run.smithi143.stdout: "mon": "allow profile osd", 2023-11-26T07:24:04.920 INFO:teuthology.orchestra.run.smithi143.stdout: "osd": "allow *" 2023-11-26T07:24:04.920 INFO:teuthology.orchestra.run.smithi143.stdout: } 2023-11-26T07:24:04.920 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2023-11-26T07:24:04.920 INFO:teuthology.orchestra.run.smithi143.stdout: { 2023-11-26T07:24:04.921 INFO:teuthology.orchestra.run.smithi143.stdout: "entity": "osd.7", 2023-11-26T07:24:04.921 INFO:teuthology.orchestra.run.smithi143.stdout: "key": "AQAC8WJlJavtIBAAKMvnR4AnnrDVOXlzN7j05w==", 2023-11-26T07:24:04.921 INFO:teuthology.orchestra.run.smithi143.stdout: "caps": { 2023-11-26T07:24:04.921 INFO:teuthology.orchestra.run.smithi143.stdout: "mgr": "allow profile osd", 2023-11-26T07:24:04.921 INFO:teuthology.orchestra.run.smithi143.stdout: "mon": "allow profile osd", 2023-11-26T07:24:04.921 INFO:teuthology.orchestra.run.smithi143.stdout: "osd": "allow *" 2023-11-26T07:24:04.921 INFO:teuthology.orchestra.run.smithi143.stdout: } 2023-11-26T07:24:04.921 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2023-11-26T07:24:04.921 INFO:teuthology.orchestra.run.smithi143.stdout: { 2023-11-26T07:24:04.921 INFO:teuthology.orchestra.run.smithi143.stdout: "entity": "client.0", 2023-11-26T07:24:04.921 INFO:teuthology.orchestra.run.smithi143.stdout: "key": "AQDm8GJlWMh9FBAAslVzi6fm4/wt7vr/rHCHHQ==", 2023-11-26T07:24:04.921 INFO:teuthology.orchestra.run.smithi143.stdout: "caps": { 2023-11-26T07:24:04.922 INFO:teuthology.orchestra.run.smithi143.stdout: "mds": "allow", 2023-11-26T07:24:04.922 INFO:teuthology.orchestra.run.smithi143.stdout: "mon": "allow r", 2023-11-26T07:24:04.922 INFO:teuthology.orchestra.run.smithi143.stdout: "osd": "allow rw tag cephfs data=cephfs" 2023-11-26T07:24:04.922 INFO:teuthology.orchestra.run.smithi143.stdout: } 2023-11-26T07:24:04.922 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2023-11-26T07:24:04.922 INFO:teuthology.orchestra.run.smithi143.stdout: { 2023-11-26T07:24:04.922 INFO:teuthology.orchestra.run.smithi143.stdout: "entity": "client.1", 2023-11-26T07:24:04.922 INFO:teuthology.orchestra.run.smithi143.stdout: "key": "AQDm8GJl5vkYHhAAgP+gKH2+lpl3aTlgqZNIXg==", 2023-11-26T07:24:04.922 INFO:teuthology.orchestra.run.smithi143.stdout: "caps": { 2023-11-26T07:24:04.922 INFO:teuthology.orchestra.run.smithi143.stdout: "mds": "allow", 2023-11-26T07:24:04.922 INFO:teuthology.orchestra.run.smithi143.stdout: "mgr": "allow r", 2023-11-26T07:24:04.922 INFO:teuthology.orchestra.run.smithi143.stdout: "mon": "allow rw", 2023-11-26T07:24:04.923 INFO:teuthology.orchestra.run.smithi143.stdout: "osd": "allow rwx" 2023-11-26T07:24:04.923 INFO:teuthology.orchestra.run.smithi143.stdout: } 2023-11-26T07:24:04.923 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2023-11-26T07:24:04.923 INFO:teuthology.orchestra.run.smithi143.stdout: { 2023-11-26T07:24:04.923 INFO:teuthology.orchestra.run.smithi143.stdout: "entity": "client.2", 2023-11-26T07:24:04.923 INFO:teuthology.orchestra.run.smithi143.stdout: "key": "AQDm8GJlW4KlGhAANFKsyVJems0ZoE6bo3WmfQ==", 2023-11-26T07:24:04.923 INFO:teuthology.orchestra.run.smithi143.stdout: "caps": { 2023-11-26T07:24:04.923 INFO:teuthology.orchestra.run.smithi143.stdout: "mds": "allow", 2023-11-26T07:24:04.923 INFO:teuthology.orchestra.run.smithi143.stdout: "mgr": "allow r", 2023-11-26T07:24:04.923 INFO:teuthology.orchestra.run.smithi143.stdout: "mon": "allow rw", 2023-11-26T07:24:04.923 INFO:teuthology.orchestra.run.smithi143.stdout: "osd": "allow rwx" 2023-11-26T07:24:04.924 INFO:teuthology.orchestra.run.smithi143.stdout: } 2023-11-26T07:24:04.924 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2023-11-26T07:24:04.924 INFO:teuthology.orchestra.run.smithi143.stdout: { 2023-11-26T07:24:04.924 INFO:teuthology.orchestra.run.smithi143.stdout: "entity": "client.3", 2023-11-26T07:24:04.924 INFO:teuthology.orchestra.run.smithi143.stdout: "key": "AQDm8GJlAJZSJBAAUSUNcwRBY01hOuilBJjlFQ==", 2023-11-26T07:24:04.924 INFO:teuthology.orchestra.run.smithi143.stdout: "caps": { 2023-11-26T07:24:04.924 INFO:teuthology.orchestra.run.smithi143.stdout: "mds": "allow", 2023-11-26T07:24:04.924 INFO:teuthology.orchestra.run.smithi143.stdout: "mgr": "allow r", 2023-11-26T07:24:04.924 INFO:teuthology.orchestra.run.smithi143.stdout: "mon": "allow rw", 2023-11-26T07:24:04.924 INFO:teuthology.orchestra.run.smithi143.stdout: "osd": "allow rwx" 2023-11-26T07:24:04.924 INFO:teuthology.orchestra.run.smithi143.stdout: } 2023-11-26T07:24:04.924 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2023-11-26T07:24:04.925 INFO:teuthology.orchestra.run.smithi143.stdout: { 2023-11-26T07:24:04.925 INFO:teuthology.orchestra.run.smithi143.stdout: "entity": "client.admin", 2023-11-26T07:24:04.925 INFO:teuthology.orchestra.run.smithi143.stdout: "key": "AQDl8GJlbQv7GxAA0Midh8tAO2KO+dH10OV8Xg==", 2023-11-26T07:24:04.925 INFO:teuthology.orchestra.run.smithi143.stdout: "caps": { 2023-11-26T07:24:04.925 INFO:teuthology.orchestra.run.smithi143.stdout: "mds": "allow *", 2023-11-26T07:24:04.925 INFO:teuthology.orchestra.run.smithi143.stdout: "mgr": "allow *", 2023-11-26T07:24:04.925 INFO:teuthology.orchestra.run.smithi143.stdout: "mon": "allow *", 2023-11-26T07:24:04.925 INFO:teuthology.orchestra.run.smithi143.stdout: "osd": "allow *" 2023-11-26T07:24:04.925 INFO:teuthology.orchestra.run.smithi143.stdout: } 2023-11-26T07:24:04.925 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2023-11-26T07:24:04.925 INFO:teuthology.orchestra.run.smithi143.stdout: { 2023-11-26T07:24:04.925 INFO:teuthology.orchestra.run.smithi143.stdout: "entity": "mgr.x", 2023-11-26T07:24:04.925 INFO:teuthology.orchestra.run.smithi143.stdout: "key": "AQDl8GJlXy8cNBAA4f5RDC1MASCqZPcJQvnUDA==", 2023-11-26T07:24:04.926 INFO:teuthology.orchestra.run.smithi143.stdout: "caps": { 2023-11-26T07:24:04.926 INFO:teuthology.orchestra.run.smithi143.stdout: "mds": "allow *", 2023-11-26T07:24:04.926 INFO:teuthology.orchestra.run.smithi143.stdout: "mon": "allow profile mgr", 2023-11-26T07:24:04.926 INFO:teuthology.orchestra.run.smithi143.stdout: "osd": "allow *" 2023-11-26T07:24:04.926 INFO:teuthology.orchestra.run.smithi143.stdout: } 2023-11-26T07:24:04.926 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2023-11-26T07:24:04.926 INFO:teuthology.orchestra.run.smithi143.stdout: { 2023-11-26T07:24:04.926 INFO:teuthology.orchestra.run.smithi143.stdout: "entity": "mgr.y", 2023-11-26T07:24:04.926 INFO:teuthology.orchestra.run.smithi143.stdout: "key": "AQDl8GJlfuFPMBAA3TYEFvdfk4DyGsBkbpv0hg==", 2023-11-26T07:24:04.926 INFO:teuthology.orchestra.run.smithi143.stdout: "caps": { 2023-11-26T07:24:04.926 INFO:teuthology.orchestra.run.smithi143.stdout: "mds": "allow *", 2023-11-26T07:24:04.926 INFO:teuthology.orchestra.run.smithi143.stdout: "mon": "allow profile mgr", 2023-11-26T07:24:04.926 INFO:teuthology.orchestra.run.smithi143.stdout: "osd": "allow *" 2023-11-26T07:24:04.927 INFO:teuthology.orchestra.run.smithi143.stdout: } 2023-11-26T07:24:04.927 INFO:teuthology.orchestra.run.smithi143.stdout: } 2023-11-26T07:24:04.927 INFO:teuthology.orchestra.run.smithi143.stdout: ] 2023-11-26T07:24:04.927 INFO:teuthology.orchestra.run.smithi143.stdout:} 2023-11-26T07:24:04.928 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:04.911+0000 7f4f309ec640 1 -- 172.21.15.143:0/3972286403 >> [v2:172.21.15.188:6836/3401128553,v1:172.21.15.188:6837/3401128553] conn(0x7f4f0c066590 msgr2=0x7f4f0c068a50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:24:04.928 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:04.911+0000 7f4f309ec640 1 --2- 172.21.15.143:0/3972286403 >> [v2:172.21.15.188:6836/3401128553,v1:172.21.15.188:6837/3401128553] conn(0x7f4f0c066590 0x7f4f0c068a50 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7f4f14009b60 tx=0x7f4f14009190 comp rx=0 tx=0).stop 2023-11-26T07:24:04.928 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:04.911+0000 7f4f309ec640 1 -- 172.21.15.143:0/3972286403 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f4f2c15f4e0 msgr2=0x7f4f2c154750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:24:04.928 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:04.911+0000 7f4f309ec640 1 --2- 172.21.15.143:0/3972286403 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f4f2c15f4e0 0x7f4f2c154750 secure :-1 s=READY pgs=1280 cs=0 l=1 rev1=1 crypto rx=0x7f4f2400ca80 tx=0x7f4f2400cf40 comp rx=0 tx=0).stop 2023-11-26T07:24:04.928 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:04.911+0000 7f4f309ec640 1 -- 172.21.15.143:0/3972286403 shutdown_connections 2023-11-26T07:24:04.928 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:04.911+0000 7f4f309ec640 1 --2- 172.21.15.143:0/3972286403 >> [v2:172.21.15.188:6836/3401128553,v1:172.21.15.188:6837/3401128553] conn(0x7f4f0c066590 0x7f4f0c068a50 unknown :-1 s=CLOSED pgs=116 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:24:04.928 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:04.911+0000 7f4f309ec640 1 --2- 172.21.15.143:0/3972286403 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f4f2c15f4e0 0x7f4f2c154750 unknown :-1 s=CLOSED pgs=1280 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:24:04.928 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:04.911+0000 7f4f309ec640 1 --2- 172.21.15.143:0/3972286403 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f4f2c13e290 0x7f4f2c128810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:24:04.928 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:04.911+0000 7f4f309ec640 1 --2- 172.21.15.143:0/3972286403 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f4f2c0cf7c0 0x7f4f2c14fe40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:24:04.928 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:04.911+0000 7f4f309ec640 1 -- 172.21.15.143:0/3972286403 >> 172.21.15.143:0/3972286403 conn(0x7f4f2c06a890 msgr2=0x7f4f2c139060 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:24:04.929 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:04.911+0000 7f4f309ec640 1 -- 172.21.15.143:0/3972286403 shutdown_connections 2023-11-26T07:24:04.929 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:04.911+0000 7f4f309ec640 1 -- 172.21.15.143:0/3972286403 wait complete. 2023-11-26T07:24:04.931 DEBUG:teuthology.orchestra.run.smithi143:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-11-26T07:24:05.069 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:05.067+0000 7f15ce432640 1 Processor -- start 2023-11-26T07:24:05.069 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:05.067+0000 7f15ce432640 1 -- start start 2023-11-26T07:24:05.071 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:05.071+0000 7f15ce432640 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7f15c813e290 0x7f15c80cf280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:24:05.071 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:05.071+0000 7f15ce432640 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7f15c815f6b0 0x7f15c80cd7b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:24:05.071 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:05.071+0000 7f15ce432640 1 -- --> v1:172.21.15.143:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f15c815fb20 con 0x7f15c80cf7c0 2023-11-26T07:24:05.071 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:05.071+0000 7f15ce432640 1 -- --> v2:172.21.15.143:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f15c80cfc40 con 0x7f15c813e290 2023-11-26T07:24:05.071 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:05.071+0000 7f15ce432640 1 -- --> v2:172.21.15.188:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f15c806c520 con 0x7f15c815f6b0 2023-11-26T07:24:05.071 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:05.071+0000 7f15c7fff640 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7f15c813e290 0x7f15c80cf280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:24:05.071 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:05.071+0000 7f15c77fe640 1 --1- >> v1:172.21.15.143:6789/0 conn(0x7f15c80cf7c0 0x7f15c815efa0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.143:6789/0 says I am v1:172.21.15.143:42388/0 (socket says 172.21.15.143:42388) 2023-11-26T07:24:05.072 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:05.071+0000 7f15c7fff640 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7f15c813e290 0x7f15c80cf280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:33596/0 (socket says 172.21.15.143:33596) 2023-11-26T07:24:05.072 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:05.071+0000 7f15c77fe640 1 -- 172.21.15.143:0/2792472181 learned_addr learned my addr 172.21.15.143:0/2792472181 (peer_addr_for_me v1:172.21.15.143:0/0) 2023-11-26T07:24:05.072 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:05.071+0000 7f15cc9cf640 1 --2- 172.21.15.143:0/2792472181 >> v2:172.21.15.188:3300/0 conn(0x7f15c815f6b0 0x7f15c80cd7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:24:05.072 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:05.071+0000 7f15c7fff640 1 -- 172.21.15.143:0/2792472181 >> v2:172.21.15.188:3300/0 conn(0x7f15c815f6b0 msgr2=0x7f15c80cd7b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-26T07:24:05.072 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:05.071+0000 7f15c7fff640 1 --2- 172.21.15.143:0/2792472181 >> v2:172.21.15.188:3300/0 conn(0x7f15c815f6b0 0x7f15c80cd7b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:24:05.072 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:05.071+0000 7f15c7fff640 1 -- 172.21.15.143:0/2792472181 >> v1:172.21.15.143:6789/0 conn(0x7f15c80cf7c0 legacy=0x7f15c815efa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:24:05.072 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:05.071+0000 7f15c7fff640 1 -- 172.21.15.143:0/2792472181 --> v2:172.21.15.143:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f15c806c800 con 0x7f15c813e290 2023-11-26T07:24:05.073 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:05.071+0000 7f15c7fff640 1 --2- 172.21.15.143:0/2792472181 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f15c813e290 0x7f15c80cf280 secure :-1 s=READY pgs=1282 cs=0 l=1 rev1=1 crypto rx=0x7f15c806e780 tx=0x7f15b801c430 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d60e9b6bd9c49d47 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:24:05.074 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:05.075+0000 7f15c6ffd640 1 -- 172.21.15.143:0/2792472181 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f15b80a7070 con 0x7f15c813e290 2023-11-26T07:24:05.074 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:05.075+0000 7f15c6ffd640 1 -- 172.21.15.143:0/2792472181 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f15b80ab430 con 0x7f15c813e290 2023-11-26T07:24:05.074 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:05.075+0000 7f15c6ffd640 1 -- 172.21.15.143:0/2792472181 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f15b80ab7c0 con 0x7f15c813e290 2023-11-26T07:24:05.075 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:05.075+0000 7f15ce432640 1 -- 172.21.15.143:0/2792472181 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f15c813e290 msgr2=0x7f15c80cf280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:24:05.076 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:05.075+0000 7f15ce432640 1 --2- 172.21.15.143:0/2792472181 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f15c813e290 0x7f15c80cf280 secure :-1 s=READY pgs=1282 cs=0 l=1 rev1=1 crypto rx=0x7f15c806e780 tx=0x7f15b801c430 comp rx=0 tx=0).stop 2023-11-26T07:24:05.076 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:05.075+0000 7f15ce432640 1 -- 172.21.15.143:0/2792472181 shutdown_connections 2023-11-26T07:24:05.076 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:05.075+0000 7f15ce432640 1 --2- 172.21.15.143:0/2792472181 >> v2:172.21.15.188:3300/0 conn(0x7f15c815f6b0 0x7f15c80cd7b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:24:05.076 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:05.075+0000 7f15ce432640 1 --2- 172.21.15.143:0/2792472181 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f15c813e290 0x7f15c80cf280 unknown :-1 s=CLOSED pgs=1282 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:24:05.076 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:05.075+0000 7f15ce432640 1 -- 172.21.15.143:0/2792472181 >> 172.21.15.143:0/2792472181 conn(0x7f15c806a890 msgr2=0x7f15c806aca0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:24:05.076 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:05.075+0000 7f15ce432640 1 -- 172.21.15.143:0/2792472181 shutdown_connections 2023-11-26T07:24:05.076 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:05.075+0000 7f15ce432640 1 -- 172.21.15.143:0/2792472181 wait complete. 2023-11-26T07:24:05.077 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:05.075+0000 7f15ce432640 1 Processor -- start 2023-11-26T07:24:05.077 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:05.075+0000 7f15ce432640 1 -- start start 2023-11-26T07:24:05.079 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:05.075+0000 7f15ce432640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f15c80cf7c0 0x7f15c8157540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:24:05.079 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:05.075+0000 7f15ce432640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f15c813e290 0x7f15c8152880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:24:05.079 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:05.075+0000 7f15ce432640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f15c815f6b0 0x7f15c813c670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:24:05.079 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:05.075+0000 7f15ce432640 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f15c815e7c0 con 0x7f15c80cf7c0 2023-11-26T07:24:05.080 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:05.075+0000 7f15ce432640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f15c8157a80 con 0x7f15c813e290 2023-11-26T07:24:05.080 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:05.075+0000 7f15ce432640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f15c815dfb0 con 0x7f15c815f6b0 2023-11-26T07:24:05.080 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:05.075+0000 7f15c7fff640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f15c80cf7c0 0x7f15c8157540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:24:05.080 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:05.079+0000 7f15c7fff640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f15c80cf7c0 0x7f15c8157540 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:33610/0 (socket says 172.21.15.143:33610) 2023-11-26T07:24:05.080 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:05.079+0000 7f15c7fff640 1 -- 172.21.15.143:0/4189063731 learned_addr learned my addr 172.21.15.143:0/4189063731 (peer_addr_for_me v2:172.21.15.143:0/0) 2023-11-26T07:24:05.080 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:05.079+0000 7f15c77fe640 1 --2- 172.21.15.143:0/4189063731 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f15c813e290 0x7f15c8152880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:24:05.080 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:05.079+0000 7f15c7fff640 1 -- 172.21.15.143:0/4189063731 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f15c815f6b0 msgr2=0x7f15c813c670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:24:05.080 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:05.079+0000 7f15cc9cf640 1 --2- 172.21.15.143:0/4189063731 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f15c815f6b0 0x7f15c813c670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:24:05.081 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:05.079+0000 7f15c7fff640 1 --2- 172.21.15.143:0/4189063731 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f15c815f6b0 0x7f15c813c670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:24:05.081 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:05.079+0000 7f15c7fff640 1 -- 172.21.15.143:0/4189063731 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f15c813e290 msgr2=0x7f15c8152880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:24:05.081 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:05.079+0000 7f15c7fff640 1 --2- 172.21.15.143:0/4189063731 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f15c813e290 0x7f15c8152880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:24:05.081 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:05.079+0000 7f15c7fff640 1 -- 172.21.15.143:0/4189063731 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f15b8009420 con 0x7f15c80cf7c0 2023-11-26T07:24:05.081 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:05.079+0000 7f15c77fe640 1 --2- 172.21.15.143:0/4189063731 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f15c813e290 0x7f15c8152880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-26T07:24:05.081 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:05.079+0000 7f15c7fff640 1 --2- 172.21.15.143:0/4189063731 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f15c80cf7c0 0x7f15c8157540 secure :-1 s=READY pgs=1283 cs=0 l=1 rev1=1 crypto rx=0x7f15b80ad040 tx=0x7f15b801ac70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:24:05.081 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:05.079+0000 7f15c4ff9640 1 -- 172.21.15.143:0/4189063731 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f15b80b5810 con 0x7f15c80cf7c0 2023-11-26T07:24:05.081 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:05.079+0000 7f15c4ff9640 1 -- 172.21.15.143:0/4189063731 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f15b80b5970 con 0x7f15c80cf7c0 2023-11-26T07:24:05.081 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:05.079+0000 7f15c4ff9640 1 -- 172.21.15.143:0/4189063731 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f15b80b5d20 con 0x7f15c80cf7c0 2023-11-26T07:24:05.082 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:05.079+0000 7f15ce432640 1 -- 172.21.15.143:0/4189063731 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f15c8152dc0 con 0x7f15c80cf7c0 2023-11-26T07:24:05.082 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:05.079+0000 7f15ce432640 1 -- 172.21.15.143:0/4189063731 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f15c815e200 con 0x7f15c80cf7c0 2023-11-26T07:24:05.083 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:05.079+0000 7f15ce432640 1 -- 172.21.15.143:0/4189063731 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f15c8067680 con 0x7f15c80cf7c0 2023-11-26T07:24:05.083 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:05.079+0000 7f15c4ff9640 1 -- 172.21.15.143:0/4189063731 <== mon.0 v2:172.21.15.143:3300/0 4 ==== mgrmap(e 16) v1 ==== 85954+0+0 (secure 0 0 0) 0x7f15b80bf410 con 0x7f15c80cf7c0 2023-11-26T07:24:05.083 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:05.083+0000 7f15c4ff9640 1 --2- 172.21.15.143:0/4189063731 >> [v2:172.21.15.188:6836/3401128553,v1:172.21.15.188:6837/3401128553] conn(0x7f15b40665c0 0x7f15b4068a80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:24:05.083 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:05.083+0000 7f15c4ff9640 1 -- 172.21.15.143:0/4189063731 <== mon.0 v2:172.21.15.143:3300/0 5 ==== osd_map(99..99 src has 1..99) v4 ==== 4857+0+0 (secure 0 0 0) 0x7f15b811e810 con 0x7f15c80cf7c0 2023-11-26T07:24:05.083 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:05.083+0000 7f15c77fe640 1 --2- 172.21.15.143:0/4189063731 >> [v2:172.21.15.188:6836/3401128553,v1:172.21.15.188:6837/3401128553] conn(0x7f15b40665c0 0x7f15b4068a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:24:05.083 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:05.083+0000 7f15c77fe640 1 --2- 172.21.15.143:0/4189063731 >> [v2:172.21.15.188:6836/3401128553,v1:172.21.15.188:6837/3401128553] conn(0x7f15b40665c0 0x7f15b4068a80 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7f15c815df40 tx=0x7f15bc009040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:24:05.085 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:05.083+0000 7f15c4ff9640 1 -- 172.21.15.143:0/4189063731 <== mon.0 v2:172.21.15.143:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172649 (secure 0 0 0) 0x7f15b80ebf80 con 0x7f15c80cf7c0 2023-11-26T07:24:05.235 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:05.235+0000 7f15ce432640 1 -- 172.21.15.143:0/4189063731 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f15c806df50 con 0x7f15c80cf7c0 2023-11-26T07:24:05.236 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:05.235+0000 7f15c4ff9640 1 -- 172.21.15.143:0/4189063731 <== mon.0 v2:172.21.15.143:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v99) v1 ==== 74+0+12894 (secure 0 0 0) 0x7f15b80c4090 con 0x7f15c80cf7c0 2023-11-26T07:24:05.236 INFO:teuthology.orchestra.run.smithi143.stdout: 2023-11-26T07:24:05.237 INFO:teuthology.orchestra.run.smithi143.stdout:{"epoch":99,"fsid":"f5033060-2469-4a07-95fe-2b76f5404bc1","created":"2023-11-26T07:17:28.613663+0000","modified":"2023-11-26T07:24:04.531571+0000","last_up_change":"2023-11-26T07:17:41.157813+0000","last_in_change":"2023-11-26T07:17:31.181658+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":15,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-11-26T07:17:41.758597+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"ebbee2c6-023e-44b9-9f1d-6f7bfd309730","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6817","nonce":639807369},{"type":"v1","addr":"172.21.15.143:6819","nonce":639807369}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6821","nonce":639807369},{"type":"v1","addr":"172.21.15.143:6823","nonce":639807369}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6830","nonce":639807369},{"type":"v1","addr":"172.21.15.143:6831","nonce":639807369}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6825","nonce":639807369},{"type":"v1","addr":"172.21.15.143:6828","nonce":639807369}]},"public_addr":"172.21.15.143:6819/639807369","cluster_addr":"172.21.15.143:6823/639807369","heartbeat_back_addr":"172.21.15.143:6831/639807369","heartbeat_front_addr":"172.21.15.143:6828/639807369","state":["exists","up"]},{"osd":1,"uuid":"fec4b8be-7122-49e9-9a19-5850a0700001","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6816","nonce":1144942530},{"type":"v1","addr":"172.21.15.143:6818","nonce":1144942530}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6820","nonce":1144942530},{"type":"v1","addr":"172.21.15.143:6822","nonce":1144942530}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6827","nonce":1144942530},{"type":"v1","addr":"172.21.15.143:6829","nonce":1144942530}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6824","nonce":1144942530},{"type":"v1","addr":"172.21.15.143:6826","nonce":1144942530}]},"public_addr":"172.21.15.143:6818/1144942530","cluster_addr":"172.21.15.143:6822/1144942530","heartbeat_back_addr":"172.21.15.143:6829/1144942530","heartbeat_front_addr":"172.21.15.143:6826/1144942530","state":["exists","up"]},{"osd":2,"uuid":"2888af5b-f933-46ca-8f95-6dd494ff43f6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6808","nonce":3909095186},{"type":"v1","addr":"172.21.15.143:6809","nonce":3909095186}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6810","nonce":3909095186},{"type":"v1","addr":"172.21.15.143:6811","nonce":3909095186}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6814","nonce":3909095186},{"type":"v1","addr":"172.21.15.143:6815","nonce":3909095186}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6812","nonce":3909095186},{"type":"v1","addr":"172.21.15.143:6813","nonce":3909095186}]},"public_addr":"172.21.15.143:6809/3909095186","cluster_addr":"172.21.15.143:6811/3909095186","heartbeat_back_addr":"172.21.15.143:6815/3909095186","heartbeat_front_addr":"172.21.15.143:6813/3909095186","state":["exists","up"]},{"osd":3,"uuid":"d1f52435-7bc1-4081-9550-9cd1ba55f549","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6800","nonce":3854210991},{"type":"v1","addr":"172.21.15.143:6801","nonce":3854210991}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6802","nonce":3854210991},{"type":"v1","addr":"172.21.15.143:6803","nonce":3854210991}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6806","nonce":3854210991},{"type":"v1","addr":"172.21.15.143:6807","nonce":3854210991}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6804","nonce":3854210991},{"type":"v1","addr":"172.21.15.143:6805","nonce":3854210991}]},"public_addr":"172.21.15.143:6801/3854210991","cluster_addr":"172.21.15.143:6803/3854210991","heartbeat_back_addr":"172.21.15.143:6807/3854210991","heartbeat_front_addr":"172.21.15.143:6805/3854210991","state":["exists","up"]},{"osd":4,"uuid":"e2ae8b6e-6e25-49b2-bcbf-aacce68f06a3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6800","nonce":3924606902},{"type":"v1","addr":"172.21.15.188:6801","nonce":3924606902}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6802","nonce":3924606902},{"type":"v1","addr":"172.21.15.188:6803","nonce":3924606902}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6806","nonce":3924606902},{"type":"v1","addr":"172.21.15.188:6807","nonce":3924606902}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6804","nonce":3924606902},{"type":"v1","addr":"172.21.15.188:6805","nonce":3924606902}]},"public_addr":"172.21.15.188:6801/3924606902","cluster_addr":"172.21.15.188:6803/3924606902","heartbeat_back_addr":"172.21.15.188:6807/3924606902","heartbeat_front_addr":"172.21.15.188:6805/3924606902","state":["exists","up"]},{"osd":5,"uuid":"3cec8a32-7698-4c36-8e89-e4ba11934f33","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6813","nonce":186841544},{"type":"v1","addr":"172.21.15.188:6815","nonce":186841544}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6817","nonce":186841544},{"type":"v1","addr":"172.21.15.188:6818","nonce":186841544}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6822","nonce":186841544},{"type":"v1","addr":"172.21.15.188:6823","nonce":186841544}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6820","nonce":186841544},{"type":"v1","addr":"172.21.15.188:6821","nonce":186841544}]},"public_addr":"172.21.15.188:6815/186841544","cluster_addr":"172.21.15.188:6818/186841544","heartbeat_back_addr":"172.21.15.188:6823/186841544","heartbeat_front_addr":"172.21.15.188:6821/186841544","state":["exists","up"]},{"osd":6,"uuid":"58a65d5d-e090-473a-81b4-ad38e85029f9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6808","nonce":2153499405},{"type":"v1","addr":"172.21.15.188:6809","nonce":2153499405}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6810","nonce":2153499405},{"type":"v1","addr":"172.21.15.188:6811","nonce":2153499405}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6816","nonce":2153499405},{"type":"v1","addr":"172.21.15.188:6819","nonce":2153499405}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6812","nonce":2153499405},{"type":"v1","addr":"172.21.15.188:6814","nonce":2153499405}]},"public_addr":"172.21.15.188:6809/2153499405","cluster_addr":"172.21.15.188:6811/2153499405","heartbeat_back_addr":"172.21.15.188:6819/2153499405","heartbeat_front_addr":"172.21.15.188:6814/2153499405","state":["exists","up"]},{"osd":7,"uuid":"ea5b7f70-3554-4cf8-ac30-f7c59b80678d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6824","nonce":2722096729},{"type":"v1","addr":"172.21.15.188:6825","nonce":2722096729}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6826","nonce":2722096729},{"type":"v1","addr":"172.21.15.188:6827","nonce":2722096729}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6830","nonce":2722096729},{"type":"v1","addr":"172.21.15.188:6831","nonce":2722096729}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6828","nonce":2722096729},{"type":"v1","addr":"172.21.15.188:6829","nonce":2722096729}]},"public_addr":"172.21.15.188:6825/2722096729","cluster_addr":"172.21.15.188:6827/2722096729","heartbeat_back_addr":"172.21.15.188:6831/2722096729","heartbeat_front_addr":"172.21.15.188:6829/2722096729","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-26T07:17:36.995032+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-26T07:17:36.932266+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-26T07:17:36.829304+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-26T07:17:36.804297+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-26T07:17:36.667892+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-26T07:17:36.964868+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-26T07:17:36.827332+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-26T07:17:36.893179+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-11-26T07:24:05.238 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:05.239+0000 7f15ce432640 1 -- 172.21.15.143:0/4189063731 >> [v2:172.21.15.188:6836/3401128553,v1:172.21.15.188:6837/3401128553] conn(0x7f15b40665c0 msgr2=0x7f15b4068a80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:24:05.239 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:05.239+0000 7f15ce432640 1 --2- 172.21.15.143:0/4189063731 >> [v2:172.21.15.188:6836/3401128553,v1:172.21.15.188:6837/3401128553] conn(0x7f15b40665c0 0x7f15b4068a80 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7f15c815df40 tx=0x7f15bc009040 comp rx=0 tx=0).stop 2023-11-26T07:24:05.239 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:05.239+0000 7f15ce432640 1 -- 172.21.15.143:0/4189063731 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f15c80cf7c0 msgr2=0x7f15c8157540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:24:05.239 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:05.239+0000 7f15ce432640 1 --2- 172.21.15.143:0/4189063731 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f15c80cf7c0 0x7f15c8157540 secure :-1 s=READY pgs=1283 cs=0 l=1 rev1=1 crypto rx=0x7f15b80ad040 tx=0x7f15b801ac70 comp rx=0 tx=0).stop 2023-11-26T07:24:05.239 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:05.239+0000 7f15ce432640 1 -- 172.21.15.143:0/4189063731 shutdown_connections 2023-11-26T07:24:05.239 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:05.239+0000 7f15ce432640 1 --2- 172.21.15.143:0/4189063731 >> [v2:172.21.15.188:6836/3401128553,v1:172.21.15.188:6837/3401128553] conn(0x7f15b40665c0 0x7f15b4068a80 unknown :-1 s=CLOSED pgs=117 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:24:05.239 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:05.239+0000 7f15ce432640 1 --2- 172.21.15.143:0/4189063731 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f15c815f6b0 0x7f15c813c670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:24:05.239 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:05.239+0000 7f15ce432640 1 --2- 172.21.15.143:0/4189063731 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f15c813e290 0x7f15c8152880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:24:05.239 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:05.239+0000 7f15ce432640 1 --2- 172.21.15.143:0/4189063731 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f15c80cf7c0 0x7f15c8157540 unknown :-1 s=CLOSED pgs=1283 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:24:05.239 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:05.239+0000 7f15ce432640 1 -- 172.21.15.143:0/4189063731 >> 172.21.15.143:0/4189063731 conn(0x7f15c806a890 msgr2=0x7f15c80b12a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:24:05.240 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:05.239+0000 7f15ce432640 1 -- 172.21.15.143:0/4189063731 shutdown_connections 2023-11-26T07:24:05.240 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:05.239+0000 7f15ce432640 1 -- 172.21.15.143:0/4189063731 wait complete. 2023-11-26T07:24:05.258 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-11-26T07:17:41.758597+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2023-11-26T07:24:05.259 DEBUG:teuthology.orchestra.run.smithi143:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-11-26T07:24:05.401 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:05.399+0000 7f5ad2232640 1 Processor -- start 2023-11-26T07:24:05.401 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:05.399+0000 7f5ad2232640 1 -- start start 2023-11-26T07:24:05.402 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:05.403+0000 7f5ad2232640 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7f5acc143ac0 0x7f5acc070370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:24:05.402 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:05.403+0000 7f5ad2232640 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7f5acc0cf990 0x7f5acc0cd1b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:24:05.402 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:05.403+0000 7f5ad2232640 1 -- --> v1:172.21.15.188:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5acc0cc730 con 0x7f5acc0708b0 2023-11-26T07:24:05.402 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:05.403+0000 7f5ad2232640 1 -- --> v2:172.21.15.143:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5acc143ea0 con 0x7f5acc143ac0 2023-11-26T07:24:05.402 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:05.403+0000 7f5ad2232640 1 -- --> v2:172.21.15.188:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5acc070ee0 con 0x7f5acc0cf990 2023-11-26T07:24:05.403 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:05.403+0000 7f5acb7fe640 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7f5acc143ac0 0x7f5acc070370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:24:05.403 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:05.403+0000 7f5acb7fe640 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7f5acc143ac0 0x7f5acc070370 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:33616/0 (socket says 172.21.15.143:33616) 2023-11-26T07:24:05.403 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:05.403+0000 7f5acb7fe640 1 -- 172.21.15.143:0/3034247297 learned_addr learned my addr 172.21.15.143:0/3034247297 (peer_addr_for_me v2:172.21.15.143:0/0) 2023-11-26T07:24:05.403 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:05.403+0000 7f5acbfff640 1 --2- 172.21.15.143:0/3034247297 >> v2:172.21.15.188:3300/0 conn(0x7f5acc0cf990 0x7f5acc0cd1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:24:05.403 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:05.403+0000 7f5acb7fe640 1 -- 172.21.15.143:0/3034247297 >> v2:172.21.15.188:3300/0 conn(0x7f5acc0cf990 msgr2=0x7f5acc0cd1b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-26T07:24:05.403 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:05.403+0000 7f5acb7fe640 1 --2- 172.21.15.143:0/3034247297 >> v2:172.21.15.188:3300/0 conn(0x7f5acc0cf990 0x7f5acc0cd1b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:24:05.404 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:05.403+0000 7f5acb7fe640 1 -- 172.21.15.143:0/3034247297 >> v1:172.21.15.188:6789/0 conn(0x7f5acc0708b0 legacy=0x7f5acc0cf280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-26T07:24:05.404 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:05.403+0000 7f5acb7fe640 1 -- 172.21.15.143:0/3034247297 --> v2:172.21.15.143:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5acc0711c0 con 0x7f5acc143ac0 2023-11-26T07:24:05.405 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:05.403+0000 7f5acb7fe640 1 --2- 172.21.15.143:0/3034247297 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f5acc143ac0 0x7f5acc070370 secure :-1 s=READY pgs=1284 cs=0 l=1 rev1=1 crypto rx=0x7f5acc06e570 tx=0x7f5abc01c4c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bb450321728fd977 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:24:05.407 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:05.407+0000 7f5aca7fc640 1 -- 172.21.15.143:0/3034247297 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5abc0aa620 con 0x7f5acc143ac0 2023-11-26T07:24:05.407 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:05.407+0000 7f5aca7fc640 1 -- 172.21.15.143:0/3034247297 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5abc0aa780 con 0x7f5acc143ac0 2023-11-26T07:24:05.407 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:05.407+0000 7f5aca7fc640 1 -- 172.21.15.143:0/3034247297 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5abc0aab10 con 0x7f5acc143ac0 2023-11-26T07:24:05.407 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:05.407+0000 7f5ad2232640 1 -- 172.21.15.143:0/3034247297 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f5acc143ac0 msgr2=0x7f5acc070370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:24:05.407 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:05.407+0000 7f5ad2232640 1 --2- 172.21.15.143:0/3034247297 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f5acc143ac0 0x7f5acc070370 secure :-1 s=READY pgs=1284 cs=0 l=1 rev1=1 crypto rx=0x7f5acc06e570 tx=0x7f5abc01c4c0 comp rx=0 tx=0).stop 2023-11-26T07:24:05.407 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:05.407+0000 7f5ad2232640 1 -- 172.21.15.143:0/3034247297 shutdown_connections 2023-11-26T07:24:05.407 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:05.407+0000 7f5ad2232640 1 --2- 172.21.15.143:0/3034247297 >> v2:172.21.15.188:3300/0 conn(0x7f5acc0cf990 0x7f5acc0cd1b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:24:05.407 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:05.407+0000 7f5ad2232640 1 --2- 172.21.15.143:0/3034247297 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f5acc143ac0 0x7f5acc070370 unknown :-1 s=CLOSED pgs=1284 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:24:05.407 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:05.407+0000 7f5ad2232640 1 -- 172.21.15.143:0/3034247297 >> 172.21.15.143:0/3034247297 conn(0x7f5acc06a890 msgr2=0x7f5acc06aca0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:24:05.407 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:05.407+0000 7f5ad2232640 1 -- 172.21.15.143:0/3034247297 shutdown_connections 2023-11-26T07:24:05.408 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:05.407+0000 7f5ad2232640 1 -- 172.21.15.143:0/3034247297 wait complete. 2023-11-26T07:24:05.408 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:05.407+0000 7f5ad2232640 1 Processor -- start 2023-11-26T07:24:05.408 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:05.407+0000 7f5ad2232640 1 -- start start 2023-11-26T07:24:05.411 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:05.407+0000 7f5ad2232640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f5acc0708b0 0x7f5acc13e630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:24:05.411 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:05.407+0000 7f5ad2232640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f5acc0cf990 0x7f5acc13b480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:24:05.411 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:05.407+0000 7f5ad2232640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f5acc143ac0 0x7f5acc0ab000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:24:05.411 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:05.407+0000 7f5ad2232640 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5acc13eb70 con 0x7f5acc143ac0 2023-11-26T07:24:05.411 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:05.407+0000 7f5ad2232640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5acc0a8dc0 con 0x7f5acc0708b0 2023-11-26T07:24:05.411 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:05.407+0000 7f5ad2232640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5acc13f080 con 0x7f5acc0cf990 2023-11-26T07:24:05.411 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:05.407+0000 7f5acbfff640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f5acc143ac0 0x7f5acc0ab000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:24:05.411 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:05.407+0000 7f5acbfff640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f5acc143ac0 0x7f5acc0ab000 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:33632/0 (socket says 172.21.15.143:33632) 2023-11-26T07:24:05.412 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:05.407+0000 7f5acbfff640 1 -- 172.21.15.143:0/434388087 learned_addr learned my addr 172.21.15.143:0/434388087 (peer_addr_for_me v2:172.21.15.143:0/0) 2023-11-26T07:24:05.412 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:05.411+0000 7f5acb7fe640 1 --2- 172.21.15.143:0/434388087 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f5acc0708b0 0x7f5acc13e630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:24:05.412 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:05.411+0000 7f5acaffd640 1 --2- 172.21.15.143:0/434388087 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f5acc0cf990 0x7f5acc13b480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:24:05.412 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:05.411+0000 7f5acbfff640 1 -- 172.21.15.143:0/434388087 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f5acc0cf990 msgr2=0x7f5acc13b480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:24:05.412 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:05.411+0000 7f5acbfff640 1 --2- 172.21.15.143:0/434388087 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f5acc0cf990 0x7f5acc13b480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:24:05.412 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:05.411+0000 7f5acbfff640 1 -- 172.21.15.143:0/434388087 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f5acc0708b0 msgr2=0x7f5acc13e630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:24:05.412 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:05.411+0000 7f5acbfff640 1 --2- 172.21.15.143:0/434388087 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f5acc0708b0 0x7f5acc13e630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:24:05.412 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:05.411+0000 7f5acbfff640 1 -- 172.21.15.143:0/434388087 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5abc0094d0 con 0x7f5acc143ac0 2023-11-26T07:24:05.412 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:05.411+0000 7f5acbfff640 1 --2- 172.21.15.143:0/434388087 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f5acc143ac0 0x7f5acc0ab000 secure :-1 s=READY pgs=1285 cs=0 l=1 rev1=1 crypto rx=0x7f5ac4007e70 tx=0x7f5ac400e4b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:24:05.412 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:05.411+0000 7f5ab3fff640 1 -- 172.21.15.143:0/434388087 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5ac4014620 con 0x7f5acc143ac0 2023-11-26T07:24:05.413 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:05.411+0000 7f5ab3fff640 1 -- 172.21.15.143:0/434388087 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5ac40147a0 con 0x7f5acc143ac0 2023-11-26T07:24:05.413 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:05.411+0000 7f5ab3fff640 1 -- 172.21.15.143:0/434388087 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5ac4014b50 con 0x7f5acc143ac0 2023-11-26T07:24:05.413 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:05.411+0000 7f5ad2232640 1 -- 172.21.15.143:0/434388087 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5acc13eda0 con 0x7f5acc143ac0 2023-11-26T07:24:05.413 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:05.411+0000 7f5ad2232640 1 -- 172.21.15.143:0/434388087 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5acc0a8a50 con 0x7f5acc143ac0 2023-11-26T07:24:05.414 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:05.411+0000 7f5ad2232640 1 -- 172.21.15.143:0/434388087 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5acc067680 con 0x7f5acc143ac0 2023-11-26T07:24:05.414 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:05.411+0000 7f5ab3fff640 1 -- 172.21.15.143:0/434388087 <== mon.0 v2:172.21.15.143:3300/0 4 ==== mgrmap(e 16) v1 ==== 85954+0+0 (secure 0 0 0) 0x7f5ac4014cb0 con 0x7f5acc143ac0 2023-11-26T07:24:05.414 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:05.411+0000 7f5ab3fff640 1 --2- 172.21.15.143:0/434388087 >> [v2:172.21.15.188:6836/3401128553,v1:172.21.15.188:6837/3401128553] conn(0x7f5ab8066590 0x7f5ab8068a50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:24:05.414 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:05.411+0000 7f5ab3fff640 1 -- 172.21.15.143:0/434388087 <== mon.0 v2:172.21.15.143:3300/0 5 ==== osd_map(99..99 src has 1..99) v4 ==== 4857+0+0 (secure 0 0 0) 0x7f5ac4084540 con 0x7f5acc143ac0 2023-11-26T07:24:05.414 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:05.415+0000 7f5acb7fe640 1 --2- 172.21.15.143:0/434388087 >> [v2:172.21.15.188:6836/3401128553,v1:172.21.15.188:6837/3401128553] conn(0x7f5ab8066590 0x7f5ab8068a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:24:05.415 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:05.415+0000 7f5acb7fe640 1 --2- 172.21.15.143:0/434388087 >> [v2:172.21.15.188:6836/3401128553,v1:172.21.15.188:6837/3401128553] conn(0x7f5ab8066590 0x7f5ab8068a50 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7f5abc0ae000 tx=0x7f5abc0b5040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:24:05.417 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:05.415+0000 7f5ab3fff640 1 -- 172.21.15.143:0/434388087 <== mon.0 v2:172.21.15.143:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172649 (secure 0 0 0) 0x7f5ac4051cb0 con 0x7f5acc143ac0 2023-11-26T07:24:05.571 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:05.571+0000 7f5ad2232640 1 -- 172.21.15.143:0/434388087 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f5acc15a810 con 0x7f5acc143ac0 2023-11-26T07:24:05.572 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:05.571+0000 7f5ab3fff640 1 -- 172.21.15.143:0/434388087 <== mon.0 v2:172.21.15.143:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v99) v1 ==== 93+0+10 (secure 0 0 0) 0x7f5ac40552d0 con 0x7f5acc143ac0 2023-11-26T07:24:05.572 INFO:teuthology.orchestra.run.smithi143.stdout:pg_num: 1 2023-11-26T07:24:05.574 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:05.571+0000 7f5ad2232640 1 -- 172.21.15.143:0/434388087 >> [v2:172.21.15.188:6836/3401128553,v1:172.21.15.188:6837/3401128553] conn(0x7f5ab8066590 msgr2=0x7f5ab8068a50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:24:05.574 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:05.571+0000 7f5ad2232640 1 --2- 172.21.15.143:0/434388087 >> [v2:172.21.15.188:6836/3401128553,v1:172.21.15.188:6837/3401128553] conn(0x7f5ab8066590 0x7f5ab8068a50 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7f5abc0ae000 tx=0x7f5abc0b5040 comp rx=0 tx=0).stop 2023-11-26T07:24:05.574 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:05.571+0000 7f5ad2232640 1 -- 172.21.15.143:0/434388087 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f5acc143ac0 msgr2=0x7f5acc0ab000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:24:05.574 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:05.571+0000 7f5ad2232640 1 --2- 172.21.15.143:0/434388087 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f5acc143ac0 0x7f5acc0ab000 secure :-1 s=READY pgs=1285 cs=0 l=1 rev1=1 crypto rx=0x7f5ac4007e70 tx=0x7f5ac400e4b0 comp rx=0 tx=0).stop 2023-11-26T07:24:05.574 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:05.571+0000 7f5ad2232640 1 -- 172.21.15.143:0/434388087 shutdown_connections 2023-11-26T07:24:05.575 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:05.571+0000 7f5ad2232640 1 --2- 172.21.15.143:0/434388087 >> [v2:172.21.15.188:6836/3401128553,v1:172.21.15.188:6837/3401128553] conn(0x7f5ab8066590 0x7f5ab8068a50 unknown :-1 s=CLOSED pgs=118 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:24:05.575 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:05.571+0000 7f5ad2232640 1 --2- 172.21.15.143:0/434388087 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f5acc143ac0 0x7f5acc0ab000 unknown :-1 s=CLOSED pgs=1285 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:24:05.575 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:05.571+0000 7f5ad2232640 1 --2- 172.21.15.143:0/434388087 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f5acc0cf990 0x7f5acc13b480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:24:05.575 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:05.571+0000 7f5ad2232640 1 --2- 172.21.15.143:0/434388087 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f5acc0708b0 0x7f5acc13e630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:24:05.575 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:05.571+0000 7f5ad2232640 1 -- 172.21.15.143:0/434388087 >> 172.21.15.143:0/434388087 conn(0x7f5acc06a890 msgr2=0x7f5acc0b1410 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:24:05.575 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:05.575+0000 7f5ad2232640 1 -- 172.21.15.143:0/434388087 shutdown_connections 2023-11-26T07:24:05.575 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:05.575+0000 7f5ad2232640 1 -- 172.21.15.143:0/434388087 wait complete. 2023-11-26T07:24:05.593 DEBUG:teuthology.orchestra.run.smithi143:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --format=json-pretty osd lspools 2023-11-26T07:24:05.730 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:05.731+0000 7f0235593640 1 Processor -- start 2023-11-26T07:24:05.730 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:05.731+0000 7f0235593640 1 -- start start 2023-11-26T07:24:05.732 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:05.731+0000 7f0235593640 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7f023015d2a0 0x7f02300d00e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:24:05.732 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:05.731+0000 7f0235593640 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7f023015d680 0x7f02300cec30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:24:05.732 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:05.731+0000 7f0235593640 1 -- --> v1:172.21.15.188:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f02300cdc40 con 0x7f02300d0e80 2023-11-26T07:24:05.732 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:05.731+0000 7f0235593640 1 -- --> v2:172.21.15.143:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f02300ce7a0 con 0x7f023015d680 2023-11-26T07:24:05.732 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:05.731+0000 7f0235593640 1 -- --> v2:172.21.15.188:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f02300d1440 con 0x7f023015d2a0 2023-11-26T07:24:05.732 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:05.731+0000 7f022f7fe640 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7f023015d680 0x7f02300cec30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:24:05.732 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:05.731+0000 7f022ffff640 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7f023015d2a0 0x7f02300d00e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:24:05.732 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:05.731+0000 7f022ffff640 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7f023015d2a0 0x7f02300d00e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.188:3300/0 says I am v2:172.21.15.143:53432/0 (socket says 172.21.15.143:53432) 2023-11-26T07:24:05.732 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:05.731+0000 7f022f7fe640 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7f023015d680 0x7f02300cec30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:33648/0 (socket says 172.21.15.143:33648) 2023-11-26T07:24:05.732 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:05.731+0000 7f022ffff640 1 -- 172.21.15.143:0/3825976811 learned_addr learned my addr 172.21.15.143:0/3825976811 (peer_addr_for_me v2:172.21.15.143:0/0) 2023-11-26T07:24:05.733 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:05.731+0000 7f022ffff640 1 -- 172.21.15.143:0/3825976811 >> v2:172.21.15.143:3300/0 conn(0x7f023015d680 msgr2=0x7f02300cec30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-26T07:24:05.733 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:05.731+0000 7f022ffff640 1 --2- 172.21.15.143:0/3825976811 >> v2:172.21.15.143:3300/0 conn(0x7f023015d680 0x7f02300cec30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:24:05.733 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:05.731+0000 7f022ffff640 1 -- 172.21.15.143:0/3825976811 >> v1:172.21.15.188:6789/0 conn(0x7f02300d0e80 legacy=0x7f02300d1680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:24:05.733 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:05.731+0000 7f022ffff640 1 -- 172.21.15.143:0/3825976811 --> v2:172.21.15.188:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f02300cde70 con 0x7f023015d2a0 2023-11-26T07:24:05.733 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:05.731+0000 7f022f7fe640 1 --2- 172.21.15.143:0/3825976811 >> v2:172.21.15.143:3300/0 conn(0x7f023015d680 0x7f02300cec30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-26T07:24:05.735 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:05.735+0000 7f022ffff640 1 --2- 172.21.15.143:0/3825976811 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f023015d2a0 0x7f02300d00e0 secure :-1 s=READY pgs=482 cs=0 l=1 rev1=1 crypto rx=0x7f023008b990 tx=0x7f022001c4d0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=ddd6677756ef8315 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:24:05.737 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:05.735+0000 7f022effd640 1 -- 172.21.15.143:0/3825976811 <== mon.1 v2:172.21.15.188:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f02200ac640 con 0x7f023015d2a0 2023-11-26T07:24:05.737 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:05.735+0000 7f022effd640 1 -- 172.21.15.143:0/3825976811 <== mon.1 v2:172.21.15.188:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f02200ac7a0 con 0x7f023015d2a0 2023-11-26T07:24:05.737 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:05.735+0000 7f022effd640 1 -- 172.21.15.143:0/3825976811 <== mon.1 v2:172.21.15.188:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f02200acb30 con 0x7f023015d2a0 2023-11-26T07:24:05.737 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:05.735+0000 7f0235593640 1 -- 172.21.15.143:0/3825976811 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f023015d2a0 msgr2=0x7f02300d00e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:24:05.737 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:05.735+0000 7f0235593640 1 --2- 172.21.15.143:0/3825976811 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f023015d2a0 0x7f02300d00e0 secure :-1 s=READY pgs=482 cs=0 l=1 rev1=1 crypto rx=0x7f023008b990 tx=0x7f022001c4d0 comp rx=0 tx=0).stop 2023-11-26T07:24:05.737 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:05.735+0000 7f0235593640 1 -- 172.21.15.143:0/3825976811 shutdown_connections 2023-11-26T07:24:05.738 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:05.735+0000 7f0235593640 1 --2- 172.21.15.143:0/3825976811 >> v2:172.21.15.143:3300/0 conn(0x7f023015d680 0x7f02300cec30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:24:05.738 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:05.735+0000 7f0235593640 1 --2- 172.21.15.143:0/3825976811 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f023015d2a0 0x7f02300d00e0 unknown :-1 s=CLOSED pgs=482 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:24:05.738 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:05.735+0000 7f0235593640 1 -- 172.21.15.143:0/3825976811 >> 172.21.15.143:0/3825976811 conn(0x7f02300d30a0 msgr2=0x7f02300d3480 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:24:05.738 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:05.735+0000 7f0235593640 1 -- 172.21.15.143:0/3825976811 shutdown_connections 2023-11-26T07:24:05.738 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:05.735+0000 7f0235593640 1 -- 172.21.15.143:0/3825976811 wait complete. 2023-11-26T07:24:05.739 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:05.739+0000 7f0235593640 1 Processor -- start 2023-11-26T07:24:05.739 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:05.739+0000 7f0235593640 1 -- start start 2023-11-26T07:24:05.740 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:05.739+0000 7f0235593640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f02300d0e80 0x7f023015df40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:24:05.741 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:05.739+0000 7f0235593640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f023015d2a0 0x7f02300b8390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:24:05.741 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:05.739+0000 7f0235593640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f023015d680 0x7f02301605b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:24:05.741 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:05.739+0000 7f0235593640 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f02300b88d0 con 0x7f02300d0e80 2023-11-26T07:24:05.741 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:05.739+0000 7f0235593640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f023015e520 con 0x7f023015d680 2023-11-26T07:24:05.741 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:05.739+0000 7f0235593640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f023015e800 con 0x7f023015d2a0 2023-11-26T07:24:05.741 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:05.739+0000 7f022ffff640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f02300d0e80 0x7f023015df40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:24:05.741 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:05.739+0000 7f022ffff640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f02300d0e80 0x7f023015df40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:33658/0 (socket says 172.21.15.143:33658) 2023-11-26T07:24:05.741 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:05.739+0000 7f022ffff640 1 -- 172.21.15.143:0/1497104252 learned_addr learned my addr 172.21.15.143:0/1497104252 (peer_addr_for_me v2:172.21.15.143:0/0) 2023-11-26T07:24:05.742 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:05.739+0000 7f022ffff640 1 -- 172.21.15.143:0/1497104252 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f023015d2a0 msgr2=0x7f02300b8390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:24:05.742 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:05.739+0000 7f022ffff640 1 --2- 172.21.15.143:0/1497104252 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f023015d2a0 0x7f02300b8390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:24:05.742 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:05.739+0000 7f022ffff640 1 -- 172.21.15.143:0/1497104252 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f023015d680 msgr2=0x7f02301605b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:24:05.742 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:05.739+0000 7f022ffff640 1 --2- 172.21.15.143:0/1497104252 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f023015d680 0x7f02301605b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:24:05.742 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:05.739+0000 7f022ffff640 1 -- 172.21.15.143:0/1497104252 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0224009420 con 0x7f02300d0e80 2023-11-26T07:24:05.742 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:05.739+0000 7f022ffff640 1 --2- 172.21.15.143:0/1497104252 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f02300d0e80 0x7f023015df40 secure :-1 s=READY pgs=1286 cs=0 l=1 rev1=1 crypto rx=0x7f02200af040 tx=0x7f022001ad10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:24:05.742 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:05.739+0000 7f022cff9640 1 -- 172.21.15.143:0/1497104252 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f02200b7620 con 0x7f02300d0e80 2023-11-26T07:24:05.742 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:05.739+0000 7f022cff9640 1 -- 172.21.15.143:0/1497104252 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f02200b7780 con 0x7f02300d0e80 2023-11-26T07:24:05.742 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:05.739+0000 7f022cff9640 1 -- 172.21.15.143:0/1497104252 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f02200b7b10 con 0x7f02300d0e80 2023-11-26T07:24:05.743 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:05.739+0000 7f0235593640 1 -- 172.21.15.143:0/1497104252 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f02200094d0 con 0x7f02300d0e80 2023-11-26T07:24:05.743 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:05.739+0000 7f0235593640 1 -- 172.21.15.143:0/1497104252 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f02300c8d20 con 0x7f02300d0e80 2023-11-26T07:24:05.744 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:05.739+0000 7f0235593640 1 -- 172.21.15.143:0/1497104252 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0230067500 con 0x7f02300d0e80 2023-11-26T07:24:05.744 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:05.743+0000 7f022cff9640 1 -- 172.21.15.143:0/1497104252 <== mon.0 v2:172.21.15.143:3300/0 4 ==== mgrmap(e 16) v1 ==== 85954+0+0 (secure 0 0 0) 0x7f02200b3040 con 0x7f02300d0e80 2023-11-26T07:24:05.744 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:05.743+0000 7f022cff9640 1 --2- 172.21.15.143:0/1497104252 >> [v2:172.21.15.188:6836/3401128553,v1:172.21.15.188:6837/3401128553] conn(0x7f021c0665c0 0x7f021c068a80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:24:05.744 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:05.743+0000 7f022cff9640 1 -- 172.21.15.143:0/1497104252 <== mon.0 v2:172.21.15.143:3300/0 5 ==== osd_map(99..99 src has 1..99) v4 ==== 4857+0+0 (secure 0 0 0) 0x7f022011e340 con 0x7f02300d0e80 2023-11-26T07:24:05.744 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:05.743+0000 7f022f7fe640 1 --2- 172.21.15.143:0/1497104252 >> [v2:172.21.15.188:6836/3401128553,v1:172.21.15.188:6837/3401128553] conn(0x7f021c0665c0 0x7f021c068a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:24:05.744 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:05.743+0000 7f022f7fe640 1 --2- 172.21.15.143:0/1497104252 >> [v2:172.21.15.188:6836/3401128553,v1:172.21.15.188:6837/3401128553] conn(0x7f021c0665c0 0x7f021c068a80 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7f0224006e90 tx=0x7f0224006de0 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:24:05.747 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:05.747+0000 7f022cff9640 1 -- 172.21.15.143:0/1497104252 <== mon.0 v2:172.21.15.143:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172649 (secure 0 0 0) 0x7f02200eba00 con 0x7f02300d0e80 2023-11-26T07:24:05.898 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:05.899+0000 7f0235593640 1 -- 172.21.15.143:0/1497104252 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "osd lspools", "format": "json-pretty"} v 0) v1 -- 0x7f023008c7f0 con 0x7f02300d0e80 2023-11-26T07:24:05.899 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:05.899+0000 7f022cff9640 1 -- 172.21.15.143:0/1497104252 <== mon.0 v2:172.21.15.143:3300/0 7 ==== mon_command_ack([{"prefix": "osd lspools", "format": "json-pretty"}]=0 v99) v1 ==== 84+0+65 (secure 0 0 0) 0x7f02200ef020 con 0x7f02300d0e80 2023-11-26T07:24:05.899 INFO:teuthology.orchestra.run.smithi143.stdout: 2023-11-26T07:24:05.899 INFO:teuthology.orchestra.run.smithi143.stdout:[ 2023-11-26T07:24:05.899 INFO:teuthology.orchestra.run.smithi143.stdout: { 2023-11-26T07:24:05.899 INFO:teuthology.orchestra.run.smithi143.stdout: "poolnum": 1, 2023-11-26T07:24:05.899 INFO:teuthology.orchestra.run.smithi143.stdout: "poolname": ".mgr" 2023-11-26T07:24:05.899 INFO:teuthology.orchestra.run.smithi143.stdout: } 2023-11-26T07:24:05.900 INFO:teuthology.orchestra.run.smithi143.stdout:] 2023-11-26T07:24:05.901 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:05.899+0000 7f0235593640 1 -- 172.21.15.143:0/1497104252 >> [v2:172.21.15.188:6836/3401128553,v1:172.21.15.188:6837/3401128553] conn(0x7f021c0665c0 msgr2=0x7f021c068a80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:24:05.901 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:05.899+0000 7f0235593640 1 --2- 172.21.15.143:0/1497104252 >> [v2:172.21.15.188:6836/3401128553,v1:172.21.15.188:6837/3401128553] conn(0x7f021c0665c0 0x7f021c068a80 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7f0224006e90 tx=0x7f0224006de0 comp rx=0 tx=0).stop 2023-11-26T07:24:05.901 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:05.899+0000 7f0235593640 1 -- 172.21.15.143:0/1497104252 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f02300d0e80 msgr2=0x7f023015df40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:24:05.901 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:05.899+0000 7f0235593640 1 --2- 172.21.15.143:0/1497104252 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f02300d0e80 0x7f023015df40 secure :-1 s=READY pgs=1286 cs=0 l=1 rev1=1 crypto rx=0x7f02200af040 tx=0x7f022001ad10 comp rx=0 tx=0).stop 2023-11-26T07:24:05.901 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:05.899+0000 7f0235593640 1 -- 172.21.15.143:0/1497104252 shutdown_connections 2023-11-26T07:24:05.902 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:05.899+0000 7f0235593640 1 --2- 172.21.15.143:0/1497104252 >> [v2:172.21.15.188:6836/3401128553,v1:172.21.15.188:6837/3401128553] conn(0x7f021c0665c0 0x7f021c068a80 unknown :-1 s=CLOSED pgs=119 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:24:05.902 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:05.899+0000 7f0235593640 1 --2- 172.21.15.143:0/1497104252 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f023015d680 0x7f02301605b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:24:05.902 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:05.899+0000 7f0235593640 1 --2- 172.21.15.143:0/1497104252 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f023015d2a0 0x7f02300b8390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:24:05.902 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:05.899+0000 7f0235593640 1 --2- 172.21.15.143:0/1497104252 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f02300d0e80 0x7f023015df40 unknown :-1 s=CLOSED pgs=1286 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:24:05.902 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:05.899+0000 7f0235593640 1 -- 172.21.15.143:0/1497104252 >> 172.21.15.143:0/1497104252 conn(0x7f02300d30a0 msgr2=0x7f02300b36f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:24:05.902 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:05.899+0000 7f0235593640 1 -- 172.21.15.143:0/1497104252 shutdown_connections 2023-11-26T07:24:05.902 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:05.899+0000 7f0235593640 1 -- 172.21.15.143:0/1497104252 wait complete. 2023-11-26T07:24:05.920 DEBUG:tasks.cephfs.filesystem:Creating filesystem 'cephfs' 2023-11-26T07:24:05.920 DEBUG:teuthology.orchestra.run.smithi143:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_metadata --pg_num_min 64 2023-11-26T07:24:06.064 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:06.063+0000 7f11cf557640 1 Processor -- start 2023-11-26T07:24:06.064 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:06.063+0000 7f11cf557640 1 -- start start 2023-11-26T07:24:06.067 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:06.063+0000 7f11cf557640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f11c814b1e0 0x7f11c80cf280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:24:06.067 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:06.063+0000 7f11cf557640 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7f11c80cf7c0 0x7f11c80708c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:24:06.067 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:06.067+0000 7f11cf557640 1 -- --> v1:172.21.15.188:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f11c80cfc20 con 0x7f11c8070e00 2023-11-26T07:24:06.067 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:06.067+0000 7f11cf557640 1 -- --> v2:172.21.15.188:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f11c8162080 con 0x7f11c80cf7c0 2023-11-26T07:24:06.067 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:06.067+0000 7f11cf557640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f11c8161d70 con 0x7f11c814b1e0 2023-11-26T07:24:06.067 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:06.067+0000 7f11ccaf2640 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7f11c80cf7c0 0x7f11c80708c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:24:06.067 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:06.067+0000 7f11ccaf2640 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7f11c80cf7c0 0x7f11c80708c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.188:3300/0 says I am v2:172.21.15.143:53452/0 (socket says 172.21.15.143:53452) 2023-11-26T07:24:06.067 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:06.067+0000 7f11ccaf2640 1 -- 172.21.15.143:0/2587295132 learned_addr learned my addr 172.21.15.143:0/2587295132 (peer_addr_for_me v2:172.21.15.143:0/0) 2023-11-26T07:24:06.067 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:06.067+0000 7f11cd2f3640 1 --2- 172.21.15.143:0/2587295132 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f11c814b1e0 0x7f11c80cf280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:24:06.067 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:06.067+0000 7f11ccaf2640 1 -- 172.21.15.143:0/2587295132 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f11c814b1e0 msgr2=0x7f11c80cf280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-26T07:24:06.068 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:06.067+0000 7f11ccaf2640 1 --2- 172.21.15.143:0/2587295132 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f11c814b1e0 0x7f11c80cf280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:24:06.068 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:06.067+0000 7f11ccaf2640 1 -- 172.21.15.143:0/2587295132 >> v1:172.21.15.188:6789/0 conn(0x7f11c8070e00 legacy=0x7f11c80b1ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:24:06.068 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:06.067+0000 7f11ccaf2640 1 -- 172.21.15.143:0/2587295132 --> v2:172.21.15.188:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f11c814b5c0 con 0x7f11c80cf7c0 2023-11-26T07:24:06.068 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:06.067+0000 7f11bffff640 1 -- 172.21.15.143:0/2587295132 <== mon.1 v1:172.21.15.188:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 169882966 0 0) 0x7f11c80cfc20 con 0x7f11c8070e00 2023-11-26T07:24:06.069 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:06.067+0000 7f11ccaf2640 1 --2- 172.21.15.143:0/2587295132 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f11c80cf7c0 0x7f11c80708c0 secure :-1 s=READY pgs=484 cs=0 l=1 rev1=1 crypto rx=0x7f11b8009870 tx=0x7f11b801c510 comp rx=0 tx=0).ready entity=mon.1 client_cookie=de7813b940d114cf server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:24:06.070 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:06.071+0000 7f11bffff640 1 -- 172.21.15.143:0/2587295132 <== mon.1 v2:172.21.15.188:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f11b80aa670 con 0x7f11c80cf7c0 2023-11-26T07:24:06.070 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:06.071+0000 7f11bffff640 1 -- 172.21.15.143:0/2587295132 <== mon.1 v2:172.21.15.188:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f11b80aa7d0 con 0x7f11c80cf7c0 2023-11-26T07:24:06.070 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:06.071+0000 7f11bffff640 1 -- 172.21.15.143:0/2587295132 <== mon.1 v2:172.21.15.188:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f11b80aab60 con 0x7f11c80cf7c0 2023-11-26T07:24:06.071 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:06.071+0000 7f11cf557640 1 -- 172.21.15.143:0/2587295132 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f11c80cf7c0 msgr2=0x7f11c80708c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:24:06.071 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:06.071+0000 7f11cf557640 1 --2- 172.21.15.143:0/2587295132 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f11c80cf7c0 0x7f11c80708c0 secure :-1 s=READY pgs=484 cs=0 l=1 rev1=1 crypto rx=0x7f11b8009870 tx=0x7f11b801c510 comp rx=0 tx=0).stop 2023-11-26T07:24:06.071 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:06.071+0000 7f11cf557640 1 -- 172.21.15.143:0/2587295132 shutdown_connections 2023-11-26T07:24:06.071 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:06.071+0000 7f11cf557640 1 --2- 172.21.15.143:0/2587295132 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f11c80cf7c0 0x7f11c80708c0 unknown :-1 s=CLOSED pgs=484 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:24:06.071 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:06.071+0000 7f11cf557640 1 --2- 172.21.15.143:0/2587295132 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f11c814b1e0 0x7f11c80cf280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:24:06.071 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:06.071+0000 7f11cf557640 1 -- 172.21.15.143:0/2587295132 >> 172.21.15.143:0/2587295132 conn(0x7f11c806a890 msgr2=0x7f11c806aca0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:24:06.072 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:06.071+0000 7f11cf557640 1 -- 172.21.15.143:0/2587295132 shutdown_connections 2023-11-26T07:24:06.072 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:06.071+0000 7f11cf557640 1 -- 172.21.15.143:0/2587295132 wait complete. 2023-11-26T07:24:06.072 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:06.071+0000 7f11cf557640 1 Processor -- start 2023-11-26T07:24:06.072 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:06.071+0000 7f11cf557640 1 -- start start 2023-11-26T07:24:06.074 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:06.071+0000 7f11cf557640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f11c8070e00 0x7f11c8149470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:24:06.074 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:06.071+0000 7f11cf557640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f11c80cf7c0 0x7f11c81fd750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:24:06.074 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:06.071+0000 7f11cf557640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f11c814b1e0 0x7f11c81fdc90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:24:06.075 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:06.071+0000 7f11cf557640 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f11c80a8fe0 con 0x7f11c80cf7c0 2023-11-26T07:24:06.075 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:06.071+0000 7f11cf557640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f11c8149e60 con 0x7f11c814b1e0 2023-11-26T07:24:06.075 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:06.071+0000 7f11cf557640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f11c816b500 con 0x7f11c8070e00 2023-11-26T07:24:06.075 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:06.071+0000 7f11ccaf2640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f11c80cf7c0 0x7f11c81fd750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:24:06.075 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:06.075+0000 7f11ccaf2640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f11c80cf7c0 0x7f11c81fd750 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:33662/0 (socket says 172.21.15.143:33662) 2023-11-26T07:24:06.075 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:06.075+0000 7f11ccaf2640 1 -- 172.21.15.143:0/2828859558 learned_addr learned my addr 172.21.15.143:0/2828859558 (peer_addr_for_me v2:172.21.15.143:0/0) 2023-11-26T07:24:06.075 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:06.075+0000 7f11cd2f3640 1 --2- 172.21.15.143:0/2828859558 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f11c8070e00 0x7f11c8149470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:24:06.075 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:06.075+0000 7f11cdaf4640 1 --2- 172.21.15.143:0/2828859558 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f11c814b1e0 0x7f11c81fdc90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:24:06.075 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:06.075+0000 7f11ccaf2640 1 -- 172.21.15.143:0/2828859558 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f11c8070e00 msgr2=0x7f11c8149470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:24:06.076 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:06.075+0000 7f11ccaf2640 1 --2- 172.21.15.143:0/2828859558 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f11c8070e00 0x7f11c8149470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:24:06.076 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:06.075+0000 7f11ccaf2640 1 -- 172.21.15.143:0/2828859558 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f11c814b1e0 msgr2=0x7f11c81fdc90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:24:06.076 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:06.075+0000 7f11ccaf2640 1 --2- 172.21.15.143:0/2828859558 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f11c814b1e0 0x7f11c81fdc90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:24:06.076 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:06.075+0000 7f11ccaf2640 1 -- 172.21.15.143:0/2828859558 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f11b80094d0 con 0x7f11c80cf7c0 2023-11-26T07:24:06.076 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:06.075+0000 7f11cd2f3640 1 --2- 172.21.15.143:0/2828859558 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f11c8070e00 0x7f11c8149470 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-26T07:24:06.076 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:06.075+0000 7f11ccaf2640 1 --2- 172.21.15.143:0/2828859558 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f11c80cf7c0 0x7f11c81fd750 secure :-1 s=READY pgs=1287 cs=0 l=1 rev1=1 crypto rx=0x7f11b80ac040 tx=0x7f11b801ad50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:24:06.077 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:06.075+0000 7f11bdffb640 1 -- 172.21.15.143:0/2828859558 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f11b80aa670 con 0x7f11c80cf7c0 2023-11-26T07:24:06.077 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:06.075+0000 7f11bdffb640 1 -- 172.21.15.143:0/2828859558 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f11b80b3820 con 0x7f11c80cf7c0 2023-11-26T07:24:06.077 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:06.075+0000 7f11cf557640 1 -- 172.21.15.143:0/2828859558 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f11c816b7e0 con 0x7f11c80cf7c0 2023-11-26T07:24:06.077 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:06.075+0000 7f11bdffb640 1 -- 172.21.15.143:0/2828859558 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f11b80b3bd0 con 0x7f11c80cf7c0 2023-11-26T07:24:06.078 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:06.075+0000 7f11cf557640 1 -- 172.21.15.143:0/2828859558 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f11c8161480 con 0x7f11c80cf7c0 2023-11-26T07:24:06.078 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:06.075+0000 7f11cf557640 1 -- 172.21.15.143:0/2828859558 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f11c80a9120 con 0x7f11c80cf7c0 2023-11-26T07:24:06.078 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:06.075+0000 7f11bdffb640 1 -- 172.21.15.143:0/2828859558 <== mon.0 v2:172.21.15.143:3300/0 4 ==== mgrmap(e 16) v1 ==== 85954+0+0 (secure 0 0 0) 0x7f11b80b3d30 con 0x7f11c80cf7c0 2023-11-26T07:24:06.078 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:06.079+0000 7f11bdffb640 1 --2- 172.21.15.143:0/2828859558 >> [v2:172.21.15.188:6836/3401128553,v1:172.21.15.188:6837/3401128553] conn(0x7f11ac066590 0x7f11ac068a50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:24:06.078 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:06.079+0000 7f11bdffb640 1 -- 172.21.15.143:0/2828859558 <== mon.0 v2:172.21.15.143:3300/0 5 ==== osd_map(99..99 src has 1..99) v4 ==== 4857+0+0 (secure 0 0 0) 0x7f11b811ebd0 con 0x7f11c80cf7c0 2023-11-26T07:24:06.078 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:06.079+0000 7f11cd2f3640 1 --2- 172.21.15.143:0/2828859558 >> [v2:172.21.15.188:6836/3401128553,v1:172.21.15.188:6837/3401128553] conn(0x7f11ac066590 0x7f11ac068a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:24:06.079 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:06.079+0000 7f11cd2f3640 1 --2- 172.21.15.143:0/2828859558 >> [v2:172.21.15.188:6836/3401128553,v1:172.21.15.188:6837/3401128553] conn(0x7f11ac066590 0x7f11ac068a50 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7f11c4009b10 tx=0x7f11c4008040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:24:06.082 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:06.083+0000 7f11bdffb640 1 -- 172.21.15.143:0/2828859558 <== mon.0 v2:172.21.15.143:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172649 (secure 0 0 0) 0x7f11c80a9120 con 0x7f11c80cf7c0 2023-11-26T07:24:06.230 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:06.231+0000 7f11cf557640 1 -- 172.21.15.143:0/2828859558 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64} v 0) v1 -- 0x7f11c80a9120 con 0x7f11c80cf7c0 2023-11-26T07:24:06.904 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:06.903+0000 7f11bdffb640 1 -- 172.21.15.143:0/2828859558 <== mon.0 v2:172.21.15.143:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]=0 pool 'cephfs_metadata' created v100) v1 ==== 138+0+0 (secure 0 0 0) 0x7f11c80a9120 con 0x7f11c80cf7c0 2023-11-26T07:24:06.905 INFO:teuthology.orchestra.run.smithi143.stderr:pool 'cephfs_metadata' created 2023-11-26T07:24:06.908 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:06.907+0000 7f11cf557640 1 -- 172.21.15.143:0/2828859558 >> [v2:172.21.15.188:6836/3401128553,v1:172.21.15.188:6837/3401128553] conn(0x7f11ac066590 msgr2=0x7f11ac068a50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:24:06.908 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:06.907+0000 7f11cf557640 1 --2- 172.21.15.143:0/2828859558 >> [v2:172.21.15.188:6836/3401128553,v1:172.21.15.188:6837/3401128553] conn(0x7f11ac066590 0x7f11ac068a50 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7f11c4009b10 tx=0x7f11c4008040 comp rx=0 tx=0).stop 2023-11-26T07:24:06.908 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:06.907+0000 7f11cf557640 1 -- 172.21.15.143:0/2828859558 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f11c80cf7c0 msgr2=0x7f11c81fd750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:24:06.908 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:06.907+0000 7f11cf557640 1 --2- 172.21.15.143:0/2828859558 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f11c80cf7c0 0x7f11c81fd750 secure :-1 s=READY pgs=1287 cs=0 l=1 rev1=1 crypto rx=0x7f11b80ac040 tx=0x7f11b801ad50 comp rx=0 tx=0).stop 2023-11-26T07:24:06.909 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:06.907+0000 7f11cf557640 1 -- 172.21.15.143:0/2828859558 shutdown_connections 2023-11-26T07:24:06.909 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:06.907+0000 7f11cf557640 1 --2- 172.21.15.143:0/2828859558 >> [v2:172.21.15.188:6836/3401128553,v1:172.21.15.188:6837/3401128553] conn(0x7f11ac066590 0x7f11ac068a50 unknown :-1 s=CLOSED pgs=120 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:24:06.909 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:06.907+0000 7f11cf557640 1 --2- 172.21.15.143:0/2828859558 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f11c814b1e0 0x7f11c81fdc90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:24:06.909 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:06.907+0000 7f11cf557640 1 --2- 172.21.15.143:0/2828859558 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f11c80cf7c0 0x7f11c81fd750 unknown :-1 s=CLOSED pgs=1287 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:24:06.909 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:06.907+0000 7f11cf557640 1 --2- 172.21.15.143:0/2828859558 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f11c8070e00 0x7f11c8149470 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:24:06.909 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:06.907+0000 7f11cf557640 1 -- 172.21.15.143:0/2828859558 >> 172.21.15.143:0/2828859558 conn(0x7f11c806a890 msgr2=0x7f11c80b1420 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:24:06.909 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:06.907+0000 7f11cf557640 1 -- 172.21.15.143:0/2828859558 shutdown_connections 2023-11-26T07:24:06.909 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:06.907+0000 7f11cf557640 1 -- 172.21.15.143:0/2828859558 wait complete. 2023-11-26T07:24:06.934 DEBUG:teuthology.orchestra.run.smithi143:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_data 64 --pg_num_min 64 --target_size_ratio 0.9 2023-11-26T07:24:07.080 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:07.079+0000 7fdf15f1c640 1 Processor -- start 2023-11-26T07:24:07.080 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:07.079+0000 7fdf15f1c640 1 -- start start 2023-11-26T07:24:07.081 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:07.079+0000 7fdf15f1c640 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7fdf10070390 0x7fdf100b1ef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:24:07.081 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:07.079+0000 7fdf15f1c640 1 -- --> v1:172.21.15.143:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fdf100b25f0 con 0x7fdf10143c00 2023-11-26T07:24:07.081 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:07.079+0000 7fdf15f1c640 1 -- --> v1:172.21.15.188:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fdf100b28d0 con 0x7fdf10071550 2023-11-26T07:24:07.081 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:07.079+0000 7fdf15f1c640 1 -- --> v2:172.21.15.188:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fdf100b2bb0 con 0x7fdf10070390 2023-11-26T07:24:07.082 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:07.079+0000 7fdf0f7fe640 1 --1- >> v1:172.21.15.143:6789/0 conn(0x7fdf10143c00 0x7fdf100cc7a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.143:6789/0 says I am v1:172.21.15.143:42392/0 (socket says 172.21.15.143:42392) 2023-11-26T07:24:07.082 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:07.079+0000 7fdf0f7fe640 1 -- 172.21.15.143:0/3741048613 learned_addr learned my addr 172.21.15.143:0/3741048613 (peer_addr_for_me v1:172.21.15.143:0/0) 2023-11-26T07:24:07.083 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:07.079+0000 7fdf0effd640 1 --2- 172.21.15.143:0/3741048613 >> v2:172.21.15.188:3300/0 conn(0x7fdf10070390 0x7fdf100b1ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:24:07.083 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:07.083+0000 7fdf0e7fc640 1 -- 172.21.15.143:0/3741048613 <== mon.0 v1:172.21.15.143:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1537829233 0 0) 0x7fdf100b25f0 con 0x7fdf10143c00 2023-11-26T07:24:07.084 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:07.083+0000 7fdf0e7fc640 1 -- 172.21.15.143:0/3741048613 --> v1:172.21.15.143:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fdef8003530 con 0x7fdf10143c00 2023-11-26T07:24:07.084 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:07.083+0000 7fdf0e7fc640 1 -- 172.21.15.143:0/3741048613 <== mon.0 v1:172.21.15.143:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2508619028 0 0) 0x7fdef8003530 con 0x7fdf10143c00 2023-11-26T07:24:07.084 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:07.083+0000 7fdf0e7fc640 1 -- 172.21.15.143:0/3741048613 >> v2:172.21.15.188:3300/0 conn(0x7fdf10070390 msgr2=0x7fdf100b1ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-26T07:24:07.084 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:07.083+0000 7fdf0e7fc640 1 --2- 172.21.15.143:0/3741048613 >> v2:172.21.15.188:3300/0 conn(0x7fdf10070390 0x7fdf100b1ef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:24:07.084 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:07.083+0000 7fdf0e7fc640 1 -- 172.21.15.143:0/3741048613 >> v1:172.21.15.188:6789/0 conn(0x7fdf10071550 legacy=0x7fdf100b3690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:24:07.084 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:07.083+0000 7fdf0e7fc640 1 -- 172.21.15.143:0/3741048613 --> v1:172.21.15.143:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdf100b2e90 con 0x7fdf10143c00 2023-11-26T07:24:07.084 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:07.083+0000 7fdf0e7fc640 1 -- 172.21.15.143:0/3741048613 <== mon.0 v1:172.21.15.143:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2137882523 0 0) 0x7fdf00001680 con 0x7fdf10143c00 2023-11-26T07:24:07.084 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:07.083+0000 7fdf0e7fc640 1 -- 172.21.15.143:0/3741048613 >> v1:172.21.15.143:6789/0 conn(0x7fdf10143c00 legacy=0x7fdf100cc7a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:24:07.084 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:07.083+0000 7fdf0e7fc640 1 --2- 172.21.15.143:0/3741048613 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fdef8003980 0x7fdef8003da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:24:07.084 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:07.083+0000 7fdf0e7fc640 1 --2- 172.21.15.143:0/3741048613 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fdef80047d0 0x7fdef8004c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:24:07.084 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:07.083+0000 7fdf0e7fc640 1 --2- 172.21.15.143:0/3741048613 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fdef8005190 0x7fdef800b990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:24:07.084 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:07.083+0000 7fdf0e7fc640 1 -- 172.21.15.143:0/3741048613 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdf100b2bb0 con 0x7fdef80047d0 2023-11-26T07:24:07.084 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:07.083+0000 7fdf0e7fc640 1 -- 172.21.15.143:0/3741048613 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdef800c130 con 0x7fdef8003980 2023-11-26T07:24:07.085 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:07.083+0000 7fdf0e7fc640 1 -- 172.21.15.143:0/3741048613 --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdef800c2f0 con 0x7fdef8005190 2023-11-26T07:24:07.085 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:07.083+0000 7fdf0effd640 1 --2- 172.21.15.143:0/3741048613 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fdef80047d0 0x7fdef8004c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:24:07.085 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:07.083+0000 7fdf0f7fe640 1 --2- 172.21.15.143:0/3741048613 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fdef8003980 0x7fdef8003da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:24:07.085 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:07.083+0000 7fdf0ffff640 1 --2- 172.21.15.143:0/3741048613 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fdef8005190 0x7fdef800b990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:24:07.085 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:07.083+0000 7fdf0effd640 1 -- 172.21.15.143:0/3741048613 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fdef8005190 msgr2=0x7fdef800b990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-26T07:24:07.085 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:07.083+0000 7fdf0effd640 1 --2- 172.21.15.143:0/3741048613 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fdef8005190 0x7fdef800b990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:24:07.085 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:07.083+0000 7fdf0effd640 1 -- 172.21.15.143:0/3741048613 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fdef8003980 msgr2=0x7fdef8003da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-26T07:24:07.085 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:07.083+0000 7fdf0effd640 1 --2- 172.21.15.143:0/3741048613 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fdef8003980 0x7fdef8003da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:24:07.085 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:07.083+0000 7fdf0effd640 1 -- 172.21.15.143:0/3741048613 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdef8003530 con 0x7fdef80047d0 2023-11-26T07:24:07.085 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:07.083+0000 7fdf0ffff640 1 -- 172.21.15.143:0/3741048613 reap_dead start 2023-11-26T07:24:07.086 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:07.087+0000 7fdf0effd640 1 --2- 172.21.15.143:0/3741048613 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fdef80047d0 0x7fdef8004c50 secure :-1 s=READY pgs=1289 cs=0 l=1 rev1=1 crypto rx=0x7fdf0400bd80 tx=0x7fdf0401f370 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f460f9a2c3ca5eed server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:24:07.086 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:07.087+0000 7fdf0e7fc640 1 -- 172.21.15.143:0/3741048613 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdf040ae650 con 0x7fdef80047d0 2023-11-26T07:24:07.086 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:07.087+0000 7fdf0e7fc640 1 -- 172.21.15.143:0/3741048613 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdf040ae7b0 con 0x7fdef80047d0 2023-11-26T07:24:07.086 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:07.087+0000 7fdf0e7fc640 1 -- 172.21.15.143:0/3741048613 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdf040aeb60 con 0x7fdef80047d0 2023-11-26T07:24:07.087 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:07.087+0000 7fdf15f1c640 1 -- 172.21.15.143:0/3741048613 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fdef80047d0 msgr2=0x7fdef8004c50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:24:07.087 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:07.087+0000 7fdf15f1c640 1 --2- 172.21.15.143:0/3741048613 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fdef80047d0 0x7fdef8004c50 secure :-1 s=READY pgs=1289 cs=0 l=1 rev1=1 crypto rx=0x7fdf0400bd80 tx=0x7fdf0401f370 comp rx=0 tx=0).stop 2023-11-26T07:24:07.087 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:07.087+0000 7fdf15f1c640 1 -- 172.21.15.143:0/3741048613 shutdown_connections 2023-11-26T07:24:07.087 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:07.087+0000 7fdf15f1c640 1 --2- 172.21.15.143:0/3741048613 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fdef80047d0 0x7fdef8004c50 unknown :-1 s=CLOSED pgs=1289 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:24:07.087 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:07.087+0000 7fdf15f1c640 1 -- 172.21.15.143:0/3741048613 >> 172.21.15.143:0/3741048613 conn(0x7fdf100b0bf0 msgr2=0x7fdf100b1000 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:24:07.087 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:07.087+0000 7fdf15f1c640 1 -- 172.21.15.143:0/3741048613 shutdown_connections 2023-11-26T07:24:07.087 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:07.087+0000 7fdf15f1c640 1 -- 172.21.15.143:0/3741048613 wait complete. 2023-11-26T07:24:07.088 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:07.087+0000 7fdf15f1c640 1 Processor -- start 2023-11-26T07:24:07.088 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:07.087+0000 7fdf15f1c640 1 -- start start 2023-11-26T07:24:07.091 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:07.087+0000 7fdf15f1c640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fdef80047d0 0x7fdf1013fb30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:24:07.091 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:07.087+0000 7fdf15f1c640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fdf10140820 0x7fdf100c92c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:24:07.091 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:07.087+0000 7fdf15f1c640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fdf101402d0 0x7fdf100c9800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:24:07.091 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:07.087+0000 7fdf15f1c640 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdf100ca530 con 0x7fdef80047d0 2023-11-26T07:24:07.091 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:07.087+0000 7fdf15f1c640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdf100c9d40 con 0x7fdf10140820 2023-11-26T07:24:07.091 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:07.087+0000 7fdf15f1c640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdf101f6dd0 con 0x7fdf101402d0 2023-11-26T07:24:07.092 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:07.091+0000 7fdf0f7fe640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fdef80047d0 0x7fdf1013fb30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:24:07.092 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:07.091+0000 7fdf0f7fe640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fdef80047d0 0x7fdf1013fb30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:33676/0 (socket says 172.21.15.143:33676) 2023-11-26T07:24:07.092 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:07.091+0000 7fdf0f7fe640 1 -- 172.21.15.143:0/513393207 learned_addr learned my addr 172.21.15.143:0/513393207 (peer_addr_for_me v2:172.21.15.143:0/0) 2023-11-26T07:24:07.092 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:07.091+0000 7fdf0ffff640 1 --2- 172.21.15.143:0/513393207 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fdf101402d0 0x7fdf100c9800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:24:07.092 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:07.091+0000 7fdf0effd640 1 --2- 172.21.15.143:0/513393207 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fdf10140820 0x7fdf100c92c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:24:07.092 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:07.091+0000 7fdf0f7fe640 1 -- 172.21.15.143:0/513393207 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fdf101402d0 msgr2=0x7fdf100c9800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:24:07.092 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:07.091+0000 7fdf0f7fe640 1 --2- 172.21.15.143:0/513393207 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fdf101402d0 0x7fdf100c9800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:24:07.092 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:07.091+0000 7fdf0f7fe640 1 -- 172.21.15.143:0/513393207 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fdf10140820 msgr2=0x7fdf100c92c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:24:07.092 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:07.091+0000 7fdf0f7fe640 1 --2- 172.21.15.143:0/513393207 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fdf10140820 0x7fdf100c92c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:24:07.092 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:07.091+0000 7fdf0f7fe640 1 -- 172.21.15.143:0/513393207 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdf100b3fb0 con 0x7fdef80047d0 2023-11-26T07:24:07.092 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:07.091+0000 7fdf0ffff640 1 --2- 172.21.15.143:0/513393207 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fdf101402d0 0x7fdf100c9800 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-26T07:24:07.092 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:07.091+0000 7fdf0effd640 1 --2- 172.21.15.143:0/513393207 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fdf10140820 0x7fdf100c92c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-26T07:24:07.092 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:07.091+0000 7fdf0f7fe640 1 --2- 172.21.15.143:0/513393207 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fdef80047d0 0x7fdf1013fb30 secure :-1 s=READY pgs=1290 cs=0 l=1 rev1=1 crypto rx=0x7fdf000072c0 tx=0x7fdf00007780 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:24:07.093 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:07.091+0000 7fdef7fff640 1 -- 172.21.15.143:0/513393207 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdf0000f940 con 0x7fdef80047d0 2023-11-26T07:24:07.093 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:07.091+0000 7fdef7fff640 1 -- 172.21.15.143:0/513393207 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdf0000fac0 con 0x7fdef80047d0 2023-11-26T07:24:07.093 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:07.091+0000 7fdf15f1c640 1 -- 172.21.15.143:0/513393207 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdf100c9f70 con 0x7fdef80047d0 2023-11-26T07:24:07.093 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:07.091+0000 7fdef7fff640 1 -- 172.21.15.143:0/513393207 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdf0000fe70 con 0x7fdef80047d0 2023-11-26T07:24:07.093 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:07.091+0000 7fdf15f1c640 1 -- 172.21.15.143:0/513393207 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdf101f6a60 con 0x7fdef80047d0 2023-11-26T07:24:07.093 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:07.091+0000 7fdf15f1c640 1 -- 172.21.15.143:0/513393207 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fdf101f68c0 con 0x7fdef80047d0 2023-11-26T07:24:07.093 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:07.091+0000 7fdef7fff640 1 -- 172.21.15.143:0/513393207 <== mon.0 v2:172.21.15.143:3300/0 4 ==== mgrmap(e 16) v1 ==== 85954+0+0 (secure 0 0 0) 0x7fdf0001d460 con 0x7fdef80047d0 2023-11-26T07:24:07.094 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:07.095+0000 7fdef7fff640 1 --2- 172.21.15.143:0/513393207 >> [v2:172.21.15.188:6836/3401128553,v1:172.21.15.188:6837/3401128553] conn(0x7fdefc066590 0x7fdefc068a50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:24:07.094 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:07.095+0000 7fdef7fff640 1 -- 172.21.15.143:0/513393207 <== mon.0 v2:172.21.15.143:3300/0 5 ==== osd_map(100..100 src has 1..100) v4 ==== 5227+0+0 (secure 0 0 0) 0x7fdf000847c0 con 0x7fdef80047d0 2023-11-26T07:24:07.095 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:07.095+0000 7fdf0effd640 1 --2- 172.21.15.143:0/513393207 >> [v2:172.21.15.188:6836/3401128553,v1:172.21.15.188:6837/3401128553] conn(0x7fdefc066590 0x7fdefc068a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:24:07.095 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:07.095+0000 7fdf0effd640 1 --2- 172.21.15.143:0/513393207 >> [v2:172.21.15.188:6836/3401128553,v1:172.21.15.188:6837/3401128553] conn(0x7fdefc066590 0x7fdefc068a50 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7fdf040b2000 tx=0x7fdf04008040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:24:07.097 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:07.095+0000 7fdef7fff640 1 -- 172.21.15.143:0/513393207 <== mon.0 v2:172.21.15.143:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172649 (secure 0 0 0) 0x7fdf00051df0 con 0x7fdef80047d0 2023-11-26T07:24:07.246 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:07.247+0000 7fdf15f1c640 1 -- 172.21.15.143:0/513393207 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9} v 0) v1 -- 0x7fdf101f68c0 con 0x7fdef80047d0 2023-11-26T07:24:07.912 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:07.911+0000 7fdef7fff640 1 -- 172.21.15.143:0/513393207 <== mon.0 v2:172.21.15.143:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]=0 pool 'cephfs_data' created v101) v1 ==== 170+0+0 (secure 0 0 0) 0x7fdf101f68c0 con 0x7fdef80047d0 2023-11-26T07:24:07.912 INFO:teuthology.orchestra.run.smithi143.stderr:pool 'cephfs_data' created 2023-11-26T07:24:07.914 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:07.915+0000 7fdf15f1c640 1 -- 172.21.15.143:0/513393207 >> [v2:172.21.15.188:6836/3401128553,v1:172.21.15.188:6837/3401128553] conn(0x7fdefc066590 msgr2=0x7fdefc068a50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:24:07.914 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:07.915+0000 7fdf15f1c640 1 --2- 172.21.15.143:0/513393207 >> [v2:172.21.15.188:6836/3401128553,v1:172.21.15.188:6837/3401128553] conn(0x7fdefc066590 0x7fdefc068a50 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7fdf040b2000 tx=0x7fdf04008040 comp rx=0 tx=0).stop 2023-11-26T07:24:07.914 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:07.915+0000 7fdf15f1c640 1 -- 172.21.15.143:0/513393207 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fdef80047d0 msgr2=0x7fdf1013fb30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:24:07.915 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:07.915+0000 7fdf15f1c640 1 --2- 172.21.15.143:0/513393207 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fdef80047d0 0x7fdf1013fb30 secure :-1 s=READY pgs=1290 cs=0 l=1 rev1=1 crypto rx=0x7fdf000072c0 tx=0x7fdf00007780 comp rx=0 tx=0).stop 2023-11-26T07:24:07.915 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:07.915+0000 7fdf15f1c640 1 -- 172.21.15.143:0/513393207 shutdown_connections 2023-11-26T07:24:07.915 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:07.915+0000 7fdf15f1c640 1 --2- 172.21.15.143:0/513393207 >> [v2:172.21.15.188:6836/3401128553,v1:172.21.15.188:6837/3401128553] conn(0x7fdefc066590 0x7fdefc068a50 unknown :-1 s=CLOSED pgs=121 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:24:07.915 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:07.915+0000 7fdf15f1c640 1 --2- 172.21.15.143:0/513393207 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fdf101402d0 0x7fdf100c9800 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:24:07.915 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:07.915+0000 7fdf15f1c640 1 --2- 172.21.15.143:0/513393207 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fdf10140820 0x7fdf100c92c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:24:07.915 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:07.915+0000 7fdf15f1c640 1 --2- 172.21.15.143:0/513393207 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fdef80047d0 0x7fdf1013fb30 unknown :-1 s=CLOSED pgs=1290 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:24:07.915 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:07.915+0000 7fdf15f1c640 1 -- 172.21.15.143:0/513393207 >> 172.21.15.143:0/513393207 conn(0x7fdf100b0bf0 msgr2=0x7fdf100ce170 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:24:07.915 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:07.915+0000 7fdf15f1c640 1 -- 172.21.15.143:0/513393207 shutdown_connections 2023-11-26T07:24:07.915 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:07.915+0000 7fdf15f1c640 1 -- 172.21.15.143:0/513393207 wait complete. 2023-11-26T07:24:07.954 DEBUG:teuthology.orchestra.run.smithi143:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs new cephfs cephfs_metadata cephfs_data 2023-11-26T07:24:08.095 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:08.095+0000 7f3c26f57640 1 Processor -- start 2023-11-26T07:24:08.096 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:08.095+0000 7f3c26f57640 1 -- start start 2023-11-26T07:24:08.096 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:08.095+0000 7f3c26f57640 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7f3c200cf990 0x7f3c2015efa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:24:08.097 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:08.095+0000 7f3c26f57640 1 -- --> v1:172.21.15.143:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3c2006c520 con 0x7f3c2013e290 2023-11-26T07:24:08.097 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:08.095+0000 7f3c26f57640 1 -- --> v1:172.21.15.188:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3c2006c800 con 0x7f3c2015f4e0 2023-11-26T07:24:08.097 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:08.095+0000 7f3c26f57640 1 -- --> v2:172.21.15.143:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3c2006cae0 con 0x7f3c200cf990 2023-11-26T07:24:08.097 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:08.095+0000 7f3c1ffff640 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7f3c200cf990 0x7f3c2015efa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:24:08.098 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:08.095+0000 7f3c24cf3640 1 --1- >> v1:172.21.15.143:6789/0 conn(0x7f3c2013e290 0x7f3c200cf280 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.143:6789/0 says I am v1:172.21.15.143:42402/0 (socket says 172.21.15.143:42402) 2023-11-26T07:24:08.098 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:08.095+0000 7f3c24cf3640 1 -- 172.21.15.143:0/4235048185 learned_addr learned my addr 172.21.15.143:0/4235048185 (peer_addr_for_me v1:172.21.15.143:0/0) 2023-11-26T07:24:08.098 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:08.095+0000 7f3c1ffff640 1 -- 172.21.15.143:0/4235048185 >> v1:172.21.15.188:6789/0 conn(0x7f3c2015f4e0 legacy=0x7f3c200cd7b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-26T07:24:08.098 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:08.095+0000 7f3c1ffff640 1 -- 172.21.15.143:0/4235048185 >> v1:172.21.15.143:6789/0 conn(0x7f3c2013e290 legacy=0x7f3c200cf280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:24:08.098 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:08.095+0000 7f3c1ffff640 1 -- 172.21.15.143:0/4235048185 --> v2:172.21.15.143:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3c2006cdc0 con 0x7f3c200cf990 2023-11-26T07:24:08.098 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:08.095+0000 7f3c1f7fe640 1 -- 172.21.15.143:0/4235048185 <== mon.0 v1:172.21.15.143:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4289334155 0 0) 0x7f3c2006c520 con 0x7f3c2013e290 2023-11-26T07:24:08.100 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:08.099+0000 7f3c1ffff640 1 --2- 172.21.15.143:0/4235048185 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f3c200cf990 0x7f3c2015efa0 secure :-1 s=READY pgs=1292 cs=0 l=1 rev1=1 crypto rx=0x7f3c0c0097c0 tx=0x7f3c0c01c410 comp rx=0 tx=0).ready entity=mon.0 client_cookie=78e039b12e58460b server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:24:08.100 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:08.099+0000 7f3c1f7fe640 1 -- 172.21.15.143:0/4235048185 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3c0c0a7070 con 0x7f3c200cf990 2023-11-26T07:24:08.100 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:08.099+0000 7f3c1f7fe640 1 -- 172.21.15.143:0/4235048185 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3c0c0ab430 con 0x7f3c200cf990 2023-11-26T07:24:08.101 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:08.099+0000 7f3c1f7fe640 1 -- 172.21.15.143:0/4235048185 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3c2015fb60 con 0x7f3c200cf990 2023-11-26T07:24:08.101 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:08.099+0000 7f3c26f57640 1 -- 172.21.15.143:0/4235048185 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f3c200cf990 msgr2=0x7f3c2015efa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:24:08.101 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:08.099+0000 7f3c26f57640 1 --2- 172.21.15.143:0/4235048185 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f3c200cf990 0x7f3c2015efa0 secure :-1 s=READY pgs=1292 cs=0 l=1 rev1=1 crypto rx=0x7f3c0c0097c0 tx=0x7f3c0c01c410 comp rx=0 tx=0).stop 2023-11-26T07:24:08.101 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:08.099+0000 7f3c26f57640 1 -- 172.21.15.143:0/4235048185 shutdown_connections 2023-11-26T07:24:08.102 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:08.099+0000 7f3c26f57640 1 --2- 172.21.15.143:0/4235048185 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f3c200cf990 0x7f3c2015efa0 unknown :-1 s=CLOSED pgs=1292 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:24:08.102 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:08.099+0000 7f3c26f57640 1 -- 172.21.15.143:0/4235048185 >> 172.21.15.143:0/4235048185 conn(0x7f3c2006a890 msgr2=0x7f3c2006aca0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:24:08.102 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:08.099+0000 7f3c26f57640 1 -- 172.21.15.143:0/4235048185 shutdown_connections 2023-11-26T07:24:08.102 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:08.099+0000 7f3c26f57640 1 -- 172.21.15.143:0/4235048185 wait complete. 2023-11-26T07:24:08.103 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:08.103+0000 7f3c26f57640 1 Processor -- start 2023-11-26T07:24:08.103 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:08.103+0000 7f3c26f57640 1 -- start start 2023-11-26T07:24:08.105 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:08.103+0000 7f3c26f57640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f3c200cf990 0x7f3c201396f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:24:08.105 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:08.103+0000 7f3c26f57640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f3c2013e290 0x7f3c20137030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:24:08.105 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:08.103+0000 7f3c26f57640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f3c2015f4e0 0x7f3c20137570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:24:08.106 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:08.103+0000 7f3c26f57640 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3c2014ea30 con 0x7f3c2015f4e0 2023-11-26T07:24:08.106 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:08.103+0000 7f3c26f57640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3c200b1680 con 0x7f3c200cf990 2023-11-26T07:24:08.106 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:08.103+0000 7f3c26f57640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3c200b0de0 con 0x7f3c2013e290 2023-11-26T07:24:08.106 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:08.103+0000 7f3c254f4640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f3c2015f4e0 0x7f3c20137570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:24:08.106 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:08.103+0000 7f3c254f4640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f3c2015f4e0 0x7f3c20137570 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:33694/0 (socket says 172.21.15.143:33694) 2023-11-26T07:24:08.106 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:08.103+0000 7f3c254f4640 1 -- 172.21.15.143:0/153759229 learned_addr learned my addr 172.21.15.143:0/153759229 (peer_addr_for_me v2:172.21.15.143:0/0) 2023-11-26T07:24:08.106 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:08.103+0000 7f3c24cf3640 1 --2- 172.21.15.143:0/153759229 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f3c200cf990 0x7f3c201396f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:24:08.106 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:08.103+0000 7f3c1ffff640 1 --2- 172.21.15.143:0/153759229 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f3c2013e290 0x7f3c20137030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:24:08.106 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:08.103+0000 7f3c254f4640 1 -- 172.21.15.143:0/153759229 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f3c2013e290 msgr2=0x7f3c20137030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:24:08.106 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:08.103+0000 7f3c254f4640 1 --2- 172.21.15.143:0/153759229 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f3c2013e290 0x7f3c20137030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:24:08.106 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:08.103+0000 7f3c254f4640 1 -- 172.21.15.143:0/153759229 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f3c200cf990 msgr2=0x7f3c201396f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:24:08.106 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:08.103+0000 7f3c254f4640 1 --2- 172.21.15.143:0/153759229 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f3c200cf990 0x7f3c201396f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:24:08.107 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:08.103+0000 7f3c254f4640 1 -- 172.21.15.143:0/153759229 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3c0c009420 con 0x7f3c2015f4e0 2023-11-26T07:24:08.107 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:08.103+0000 7f3c254f4640 1 --2- 172.21.15.143:0/153759229 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f3c2015f4e0 0x7f3c20137570 secure :-1 s=READY pgs=1293 cs=0 l=1 rev1=1 crypto rx=0x7f3c1800b970 tx=0x7f3c1800be30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:24:08.107 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:08.103+0000 7f3c1d7fa640 1 -- 172.21.15.143:0/153759229 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3c1800cb40 con 0x7f3c2015f4e0 2023-11-26T07:24:08.107 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:08.103+0000 7f3c1d7fa640 1 -- 172.21.15.143:0/153759229 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3c1800cca0 con 0x7f3c2015f4e0 2023-11-26T07:24:08.107 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:08.103+0000 7f3c26f57640 1 -- 172.21.15.143:0/153759229 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3c200b10c0 con 0x7f3c2015f4e0 2023-11-26T07:24:08.107 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:08.103+0000 7f3c1d7fa640 1 -- 172.21.15.143:0/153759229 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3c18014680 con 0x7f3c2015f4e0 2023-11-26T07:24:08.107 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:08.103+0000 7f3c26f57640 1 -- 172.21.15.143:0/153759229 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3c20139c30 con 0x7f3c2015f4e0 2023-11-26T07:24:08.107 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:08.107+0000 7f3c26f57640 1 -- 172.21.15.143:0/153759229 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3c2015ac90 con 0x7f3c2015f4e0 2023-11-26T07:24:08.108 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:08.107+0000 7f3c1d7fa640 1 -- 172.21.15.143:0/153759229 <== mon.0 v2:172.21.15.143:3300/0 4 ==== mgrmap(e 16) v1 ==== 85954+0+0 (secure 0 0 0) 0x7f3c18003980 con 0x7f3c2015f4e0 2023-11-26T07:24:08.108 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:08.107+0000 7f3c1d7fa640 1 --2- 172.21.15.143:0/153759229 >> [v2:172.21.15.188:6836/3401128553,v1:172.21.15.188:6837/3401128553] conn(0x7f3c04066590 0x7f3c04068a50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:24:08.109 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:08.107+0000 7f3c1d7fa640 1 -- 172.21.15.143:0/153759229 <== mon.0 v2:172.21.15.143:3300/0 5 ==== osd_map(101..101 src has 1..101) v4 ==== 5609+0+0 (secure 0 0 0) 0x7f3c18083f10 con 0x7f3c2015f4e0 2023-11-26T07:24:08.109 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:08.107+0000 7f3c24cf3640 1 --2- 172.21.15.143:0/153759229 >> [v2:172.21.15.188:6836/3401128553,v1:172.21.15.188:6837/3401128553] conn(0x7f3c04066590 0x7f3c04068a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:24:08.109 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:08.107+0000 7f3c24cf3640 1 --2- 172.21.15.143:0/153759229 >> [v2:172.21.15.188:6836/3401128553,v1:172.21.15.188:6837/3401128553] conn(0x7f3c04066590 0x7f3c04068a50 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7f3c14007fd0 tx=0x7f3c14007d60 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:24:08.112 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:08.111+0000 7f3c1d7fa640 1 -- 172.21.15.143:0/153759229 <== mon.0 v2:172.21.15.143:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172649 (secure 0 0 0) 0x7f3c18051400 con 0x7f3c2015f4e0 2023-11-26T07:24:08.292 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:08.291+0000 7f3c26f57640 1 -- 172.21.15.143:0/153759229 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"} v 0) v1 -- 0x7f3c200cdcf0 con 0x7f3c2015f4e0 2023-11-26T07:24:08.915 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:08.915+0000 7f3c1d7fa640 1 -- 172.21.15.143:0/153759229 <== mon.0 v2:172.21.15.143:3300/0 7 ==== mon_command_ack([{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]=0 new fs with metadata pool 16 and data pool 17 v61) v1 ==== 174+0+0 (secure 0 0 0) 0x7f3c18054a20 con 0x7f3c2015f4e0 2023-11-26T07:24:08.915 INFO:teuthology.orchestra.run.smithi143.stderr:new fs with metadata pool 16 and data pool 17 2023-11-26T07:24:08.918 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:08.915+0000 7f3c26f57640 1 -- 172.21.15.143:0/153759229 >> [v2:172.21.15.188:6836/3401128553,v1:172.21.15.188:6837/3401128553] conn(0x7f3c04066590 msgr2=0x7f3c04068a50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:24:08.918 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:08.915+0000 7f3c26f57640 1 --2- 172.21.15.143:0/153759229 >> [v2:172.21.15.188:6836/3401128553,v1:172.21.15.188:6837/3401128553] conn(0x7f3c04066590 0x7f3c04068a50 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7f3c14007fd0 tx=0x7f3c14007d60 comp rx=0 tx=0).stop 2023-11-26T07:24:08.918 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:08.915+0000 7f3c26f57640 1 -- 172.21.15.143:0/153759229 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f3c2015f4e0 msgr2=0x7f3c20137570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:24:08.918 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:08.915+0000 7f3c26f57640 1 --2- 172.21.15.143:0/153759229 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f3c2015f4e0 0x7f3c20137570 secure :-1 s=READY pgs=1293 cs=0 l=1 rev1=1 crypto rx=0x7f3c1800b970 tx=0x7f3c1800be30 comp rx=0 tx=0).stop 2023-11-26T07:24:08.918 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:08.919+0000 7f3c26f57640 1 -- 172.21.15.143:0/153759229 shutdown_connections 2023-11-26T07:24:08.918 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:08.919+0000 7f3c26f57640 1 --2- 172.21.15.143:0/153759229 >> [v2:172.21.15.188:6836/3401128553,v1:172.21.15.188:6837/3401128553] conn(0x7f3c04066590 0x7f3c04068a50 unknown :-1 s=CLOSED pgs=122 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:24:08.918 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:08.919+0000 7f3c26f57640 1 --2- 172.21.15.143:0/153759229 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f3c2015f4e0 0x7f3c20137570 unknown :-1 s=CLOSED pgs=1293 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:24:08.918 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:08.919+0000 7f3c26f57640 1 --2- 172.21.15.143:0/153759229 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f3c2013e290 0x7f3c20137030 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:24:08.919 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:08.919+0000 7f3c26f57640 1 --2- 172.21.15.143:0/153759229 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f3c200cf990 0x7f3c201396f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:24:08.919 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:08.919+0000 7f3c26f57640 1 -- 172.21.15.143:0/153759229 >> 172.21.15.143:0/153759229 conn(0x7f3c2006a890 msgr2=0x7f3c20154e40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:24:08.919 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:08.919+0000 7f3c26f57640 1 -- 172.21.15.143:0/153759229 shutdown_connections 2023-11-26T07:24:08.919 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:08.919+0000 7f3c26f57640 1 -- 172.21.15.143:0/153759229 wait complete. 2023-11-26T07:24:08.947 DEBUG:teuthology.orchestra.run.smithi143:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-11-26T07:24:09.084 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:09.083+0000 7f2c6ca9c640 1 Processor -- start 2023-11-26T07:24:09.084 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:09.083+0000 7f2c6ca9c640 1 -- start start 2023-11-26T07:24:09.085 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:09.083+0000 7f2c6ca9c640 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7f2c68143ac0 0x7f2c68070370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:24:09.085 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:09.083+0000 7f2c6ca9c640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f2c680708b0 0x7f2c680cf280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:24:09.085 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:09.083+0000 7f2c6ca9c640 1 -- --> v1:172.21.15.188:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2c680cfbf0 con 0x7f2c680cf7c0 2023-11-26T07:24:09.086 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:09.083+0000 7f2c6ca9c640 1 -- --> v2:172.21.15.188:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2c68143ea0 con 0x7f2c68143ac0 2023-11-26T07:24:09.086 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:09.083+0000 7f2c6ca9c640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2c680cc730 con 0x7f2c680708b0 2023-11-26T07:24:09.086 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:09.087+0000 7f2c6659c640 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7f2c68143ac0 0x7f2c68070370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:24:09.087 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:09.087+0000 7f2c66d9d640 1 --1- >> v1:172.21.15.188:6789/0 conn(0x7f2c680cf7c0 0x7f2c680b1ef0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.188:6789/0 says I am v1:172.21.15.143:56522/0 (socket says 172.21.15.143:56522) 2023-11-26T07:24:09.087 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:09.087+0000 7f2c5dd9b640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f2c680708b0 0x7f2c680cf280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:24:09.087 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:09.087+0000 7f2c66d9d640 1 -- 172.21.15.143:0/1244251367 learned_addr learned my addr 172.21.15.143:0/1244251367 (peer_addr_for_me v1:172.21.15.143:0/0) 2023-11-26T07:24:09.087 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:09.087+0000 7f2c65d9b640 1 -- 172.21.15.143:0/1244251367 <== mon.1 v1:172.21.15.188:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1929847349 0 0) 0x7f2c680cfbf0 con 0x7f2c680cf7c0 2023-11-26T07:24:09.087 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:09.087+0000 7f2c65d9b640 1 -- 172.21.15.143:0/1244251367 --> v1:172.21.15.188:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f2c54003530 con 0x7f2c680cf7c0 2023-11-26T07:24:09.087 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:09.087+0000 7f2c5dd9b640 1 -- 172.21.15.143:0/1244251367 >> v2:172.21.15.188:3300/0 conn(0x7f2c68143ac0 msgr2=0x7f2c68070370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-26T07:24:09.087 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:09.087+0000 7f2c5dd9b640 1 --2- 172.21.15.143:0/1244251367 >> v2:172.21.15.188:3300/0 conn(0x7f2c68143ac0 0x7f2c68070370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:24:09.087 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:09.087+0000 7f2c5dd9b640 1 -- 172.21.15.143:0/1244251367 >> v1:172.21.15.188:6789/0 conn(0x7f2c680cf7c0 legacy=0x7f2c680b1ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:24:09.087 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:09.087+0000 7f2c5dd9b640 1 -- 172.21.15.143:0/1244251367 --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2c68070ee0 con 0x7f2c680708b0 2023-11-26T07:24:09.087 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:09.087+0000 7f2c6659c640 1 --2- 172.21.15.143:0/1244251367 >> v2:172.21.15.188:3300/0 conn(0x7f2c68143ac0 0x7f2c68070370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-26T07:24:09.089 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:09.087+0000 7f2c5dd9b640 1 --2- 172.21.15.143:0/1244251367 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f2c680708b0 0x7f2c680cf280 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7f2c50009d90 tx=0x7f2c5001c950 comp rx=0 tx=0).ready entity=mon.2 client_cookie=987e7f8f4fd9b5f2 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:24:09.090 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:09.087+0000 7f2c65d9b640 1 -- 172.21.15.143:0/1244251367 <== mon.2 v2:172.21.15.188:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2c500a5920 con 0x7f2c680708b0 2023-11-26T07:24:09.090 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:09.091+0000 7f2c65d9b640 1 -- 172.21.15.143:0/1244251367 <== mon.2 v2:172.21.15.188:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2c500a5a80 con 0x7f2c680708b0 2023-11-26T07:24:09.090 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:09.091+0000 7f2c65d9b640 1 -- 172.21.15.143:0/1244251367 <== mon.2 v2:172.21.15.188:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2c500a5e10 con 0x7f2c680708b0 2023-11-26T07:24:09.091 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:09.091+0000 7f2c6ca9c640 1 -- 172.21.15.143:0/1244251367 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f2c680708b0 msgr2=0x7f2c680cf280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:24:09.091 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:09.091+0000 7f2c6ca9c640 1 --2- 172.21.15.143:0/1244251367 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f2c680708b0 0x7f2c680cf280 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7f2c50009d90 tx=0x7f2c5001c950 comp rx=0 tx=0).stop 2023-11-26T07:24:09.091 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:09.091+0000 7f2c6ca9c640 1 -- 172.21.15.143:0/1244251367 shutdown_connections 2023-11-26T07:24:09.091 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:09.091+0000 7f2c6ca9c640 1 --2- 172.21.15.143:0/1244251367 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f2c680708b0 0x7f2c680cf280 unknown :-1 s=CLOSED pgs=156 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:24:09.091 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:09.091+0000 7f2c6ca9c640 1 --2- 172.21.15.143:0/1244251367 >> v2:172.21.15.188:3300/0 conn(0x7f2c68143ac0 0x7f2c68070370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:24:09.091 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:09.091+0000 7f2c6ca9c640 1 -- 172.21.15.143:0/1244251367 >> 172.21.15.143:0/1244251367 conn(0x7f2c6806a890 msgr2=0x7f2c6806aca0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:24:09.091 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:09.091+0000 7f2c6ca9c640 1 -- 172.21.15.143:0/1244251367 shutdown_connections 2023-11-26T07:24:09.091 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:09.091+0000 7f2c6ca9c640 1 -- 172.21.15.143:0/1244251367 wait complete. 2023-11-26T07:24:09.092 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:09.091+0000 7f2c6ca9c640 1 Processor -- start 2023-11-26T07:24:09.092 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:09.091+0000 7f2c6ca9c640 1 -- start start 2023-11-26T07:24:09.095 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:09.091+0000 7f2c6ca9c640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f2c680708b0 0x7f2c68158580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:24:09.095 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:09.091+0000 7f2c6ca9c640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f2c680cf7c0 0x7f2c68137770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:24:09.095 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:09.091+0000 7f2c6ca9c640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f2c68143ac0 0x7f2c680cd000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:24:09.095 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:09.091+0000 7f2c6ca9c640 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2c681385d0 con 0x7f2c680708b0 2023-11-26T07:24:09.095 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:09.091+0000 7f2c6ca9c640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2c68159520 con 0x7f2c680cf7c0 2023-11-26T07:24:09.095 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:09.091+0000 7f2c6ca9c640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2c68157230 con 0x7f2c68143ac0 2023-11-26T07:24:09.095 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:09.095+0000 7f2c6659c640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f2c680708b0 0x7f2c68158580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:24:09.095 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:09.095+0000 7f2c6659c640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f2c680708b0 0x7f2c68158580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:33710/0 (socket says 172.21.15.143:33710) 2023-11-26T07:24:09.095 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:09.095+0000 7f2c6659c640 1 -- 172.21.15.143:0/969366689 learned_addr learned my addr 172.21.15.143:0/969366689 (peer_addr_for_me v2:172.21.15.143:0/0) 2023-11-26T07:24:09.095 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:09.095+0000 7f2c66d9d640 1 --2- 172.21.15.143:0/969366689 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f2c68143ac0 0x7f2c680cd000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:24:09.096 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:09.095+0000 7f2c5dd9b640 1 --2- 172.21.15.143:0/969366689 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f2c680cf7c0 0x7f2c68137770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:24:09.096 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:09.095+0000 7f2c6659c640 1 -- 172.21.15.143:0/969366689 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f2c68143ac0 msgr2=0x7f2c680cd000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:24:09.096 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:09.095+0000 7f2c6659c640 1 --2- 172.21.15.143:0/969366689 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f2c68143ac0 0x7f2c680cd000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:24:09.096 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:09.095+0000 7f2c6659c640 1 -- 172.21.15.143:0/969366689 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f2c680cf7c0 msgr2=0x7f2c68137770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:24:09.096 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:09.095+0000 7f2c6659c640 1 --2- 172.21.15.143:0/969366689 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f2c680cf7c0 0x7f2c68137770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:24:09.096 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:09.095+0000 7f2c6659c640 1 -- 172.21.15.143:0/969366689 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2c5800b6c0 con 0x7f2c680708b0 2023-11-26T07:24:09.096 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:09.095+0000 7f2c6659c640 1 --2- 172.21.15.143:0/969366689 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f2c680708b0 0x7f2c68158580 secure :-1 s=READY pgs=1294 cs=0 l=1 rev1=1 crypto rx=0x7f2c58010e60 tx=0x7f2c5800e730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:24:09.096 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:09.095+0000 7f2c5f7fe640 1 -- 172.21.15.143:0/969366689 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2c5800b980 con 0x7f2c680708b0 2023-11-26T07:24:09.096 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:09.095+0000 7f2c5f7fe640 1 -- 172.21.15.143:0/969366689 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2c58011860 con 0x7f2c680708b0 2023-11-26T07:24:09.096 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:09.095+0000 7f2c6ca9c640 1 -- 172.21.15.143:0/969366689 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2c50009a70 con 0x7f2c680708b0 2023-11-26T07:24:09.096 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:09.095+0000 7f2c5f7fe640 1 -- 172.21.15.143:0/969366689 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2c58011bf0 con 0x7f2c680708b0 2023-11-26T07:24:09.096 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:09.095+0000 7f2c6ca9c640 1 -- 172.21.15.143:0/969366689 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2c68158f60 con 0x7f2c680708b0 2023-11-26T07:24:09.097 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:09.095+0000 7f2c6ca9c640 1 -- 172.21.15.143:0/969366689 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2c68067680 con 0x7f2c680708b0 2023-11-26T07:24:09.097 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:09.095+0000 7f2c5f7fe640 1 -- 172.21.15.143:0/969366689 <== mon.0 v2:172.21.15.143:3300/0 4 ==== mgrmap(e 16) v1 ==== 85954+0+0 (secure 0 0 0) 0x7f2c58011d50 con 0x7f2c680708b0 2023-11-26T07:24:09.098 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:09.099+0000 7f2c5f7fe640 1 --2- 172.21.15.143:0/969366689 >> [v2:172.21.15.188:6836/3401128553,v1:172.21.15.188:6837/3401128553] conn(0x7f2c48066590 0x7f2c48068a50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:24:09.098 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:09.099+0000 7f2c5f7fe640 1 -- 172.21.15.143:0/969366689 <== mon.0 v2:172.21.15.143:3300/0 5 ==== osd_map(102..102 src has 1..102) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f2c58086d70 con 0x7f2c680708b0 2023-11-26T07:24:09.099 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:09.099+0000 7f2c5dd9b640 1 --2- 172.21.15.143:0/969366689 >> [v2:172.21.15.188:6836/3401128553,v1:172.21.15.188:6837/3401128553] conn(0x7f2c48066590 0x7f2c48068a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:24:09.099 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:09.099+0000 7f2c5dd9b640 1 --2- 172.21.15.143:0/969366689 >> [v2:172.21.15.188:6836/3401128553,v1:172.21.15.188:6837/3401128553] conn(0x7f2c48066590 0x7f2c48068a50 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7f2c500af000 tx=0x7f2c500a7070 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:24:09.102 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:09.103+0000 7f2c5f7fe640 1 -- 172.21.15.143:0/969366689 <== mon.0 v2:172.21.15.143:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172649 (secure 0 0 0) 0x7f2c58054190 con 0x7f2c680708b0 2023-11-26T07:24:09.253 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:09.251+0000 7f2c6ca9c640 1 -- 172.21.15.143:0/969366689 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f2c68143080 con 0x7f2c680708b0 2023-11-26T07:24:09.254 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:09.255+0000 7f2c5f7fe640 1 -- 172.21.15.143:0/969366689 <== mon.0 v2:172.21.15.143:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v102) v1 ==== 74+0+16576 (secure 0 0 0) 0x7f2c580577b0 con 0x7f2c680708b0 2023-11-26T07:24:09.254 INFO:teuthology.orchestra.run.smithi143.stdout: 2023-11-26T07:24:09.255 INFO:teuthology.orchestra.run.smithi143.stdout:{"epoch":102,"fsid":"f5033060-2469-4a07-95fe-2b76f5404bc1","created":"2023-11-26T07:17:28.613663+0000","modified":"2023-11-26T07:24:08.295548+0000","last_up_change":"2023-11-26T07:17:41.157813+0000","last_in_change":"2023-11-26T07:17:31.181658+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":17,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-11-26T07:17:41.758597+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":16,"pool_name":"cephfs_metadata","create_time":"2023-11-26T07:24:06.278132+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"102","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":17,"pool_name":"cephfs_data","create_time":"2023-11-26T07:24:07.294850+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"102","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"ebbee2c6-023e-44b9-9f1d-6f7bfd309730","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":101,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6817","nonce":639807369},{"type":"v1","addr":"172.21.15.143:6819","nonce":639807369}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6821","nonce":639807369},{"type":"v1","addr":"172.21.15.143:6823","nonce":639807369}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6830","nonce":639807369},{"type":"v1","addr":"172.21.15.143:6831","nonce":639807369}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6825","nonce":639807369},{"type":"v1","addr":"172.21.15.143:6828","nonce":639807369}]},"public_addr":"172.21.15.143:6819/639807369","cluster_addr":"172.21.15.143:6823/639807369","heartbeat_back_addr":"172.21.15.143:6831/639807369","heartbeat_front_addr":"172.21.15.143:6828/639807369","state":["exists","up"]},{"osd":1,"uuid":"fec4b8be-7122-49e9-9a19-5850a0700001","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":101,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6816","nonce":1144942530},{"type":"v1","addr":"172.21.15.143:6818","nonce":1144942530}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6820","nonce":1144942530},{"type":"v1","addr":"172.21.15.143:6822","nonce":1144942530}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6827","nonce":1144942530},{"type":"v1","addr":"172.21.15.143:6829","nonce":1144942530}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6824","nonce":1144942530},{"type":"v1","addr":"172.21.15.143:6826","nonce":1144942530}]},"public_addr":"172.21.15.143:6818/1144942530","cluster_addr":"172.21.15.143:6822/1144942530","heartbeat_back_addr":"172.21.15.143:6829/1144942530","heartbeat_front_addr":"172.21.15.143:6826/1144942530","state":["exists","up"]},{"osd":2,"uuid":"2888af5b-f933-46ca-8f95-6dd494ff43f6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":101,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6808","nonce":3909095186},{"type":"v1","addr":"172.21.15.143:6809","nonce":3909095186}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6810","nonce":3909095186},{"type":"v1","addr":"172.21.15.143:6811","nonce":3909095186}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6814","nonce":3909095186},{"type":"v1","addr":"172.21.15.143:6815","nonce":3909095186}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6812","nonce":3909095186},{"type":"v1","addr":"172.21.15.143:6813","nonce":3909095186}]},"public_addr":"172.21.15.143:6809/3909095186","cluster_addr":"172.21.15.143:6811/3909095186","heartbeat_back_addr":"172.21.15.143:6815/3909095186","heartbeat_front_addr":"172.21.15.143:6813/3909095186","state":["exists","up"]},{"osd":3,"uuid":"d1f52435-7bc1-4081-9550-9cd1ba55f549","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":101,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6800","nonce":3854210991},{"type":"v1","addr":"172.21.15.143:6801","nonce":3854210991}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6802","nonce":3854210991},{"type":"v1","addr":"172.21.15.143:6803","nonce":3854210991}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6806","nonce":3854210991},{"type":"v1","addr":"172.21.15.143:6807","nonce":3854210991}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6804","nonce":3854210991},{"type":"v1","addr":"172.21.15.143:6805","nonce":3854210991}]},"public_addr":"172.21.15.143:6801/3854210991","cluster_addr":"172.21.15.143:6803/3854210991","heartbeat_back_addr":"172.21.15.143:6807/3854210991","heartbeat_front_addr":"172.21.15.143:6805/3854210991","state":["exists","up"]},{"osd":4,"uuid":"e2ae8b6e-6e25-49b2-bcbf-aacce68f06a3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":101,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6800","nonce":3924606902},{"type":"v1","addr":"172.21.15.188:6801","nonce":3924606902}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6802","nonce":3924606902},{"type":"v1","addr":"172.21.15.188:6803","nonce":3924606902}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6806","nonce":3924606902},{"type":"v1","addr":"172.21.15.188:6807","nonce":3924606902}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6804","nonce":3924606902},{"type":"v1","addr":"172.21.15.188:6805","nonce":3924606902}]},"public_addr":"172.21.15.188:6801/3924606902","cluster_addr":"172.21.15.188:6803/3924606902","heartbeat_back_addr":"172.21.15.188:6807/3924606902","heartbeat_front_addr":"172.21.15.188:6805/3924606902","state":["exists","up"]},{"osd":5,"uuid":"3cec8a32-7698-4c36-8e89-e4ba11934f33","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":101,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6813","nonce":186841544},{"type":"v1","addr":"172.21.15.188:6815","nonce":186841544}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6817","nonce":186841544},{"type":"v1","addr":"172.21.15.188:6818","nonce":186841544}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6822","nonce":186841544},{"type":"v1","addr":"172.21.15.188:6823","nonce":186841544}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6820","nonce":186841544},{"type":"v1","addr":"172.21.15.188:6821","nonce":186841544}]},"public_addr":"172.21.15.188:6815/186841544","cluster_addr":"172.21.15.188:6818/186841544","heartbeat_back_addr":"172.21.15.188:6823/186841544","heartbeat_front_addr":"172.21.15.188:6821/186841544","state":["exists","up"]},{"osd":6,"uuid":"58a65d5d-e090-473a-81b4-ad38e85029f9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":101,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6808","nonce":2153499405},{"type":"v1","addr":"172.21.15.188:6809","nonce":2153499405}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6810","nonce":2153499405},{"type":"v1","addr":"172.21.15.188:6811","nonce":2153499405}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6816","nonce":2153499405},{"type":"v1","addr":"172.21.15.188:6819","nonce":2153499405}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6812","nonce":2153499405},{"type":"v1","addr":"172.21.15.188:6814","nonce":2153499405}]},"public_addr":"172.21.15.188:6809/2153499405","cluster_addr":"172.21.15.188:6811/2153499405","heartbeat_back_addr":"172.21.15.188:6819/2153499405","heartbeat_front_addr":"172.21.15.188:6814/2153499405","state":["exists","up"]},{"osd":7,"uuid":"ea5b7f70-3554-4cf8-ac30-f7c59b80678d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":101,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6824","nonce":2722096729},{"type":"v1","addr":"172.21.15.188:6825","nonce":2722096729}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6826","nonce":2722096729},{"type":"v1","addr":"172.21.15.188:6827","nonce":2722096729}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6830","nonce":2722096729},{"type":"v1","addr":"172.21.15.188:6831","nonce":2722096729}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6828","nonce":2722096729},{"type":"v1","addr":"172.21.15.188:6829","nonce":2722096729}]},"public_addr":"172.21.15.188:6825/2722096729","cluster_addr":"172.21.15.188:6827/2722096729","heartbeat_back_addr":"172.21.15.188:6831/2722096729","heartbeat_front_addr":"172.21.15.188:6829/2722096729","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-26T07:17:36.995032+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-26T07:17:36.932266+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-26T07:17:36.829304+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-26T07:17:36.804297+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-26T07:17:36.667892+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-26T07:17:36.964868+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-26T07:17:36.827332+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-26T07:17:36.893179+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-11-26T07:24:09.256 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:09.255+0000 7f2c6ca9c640 1 -- 172.21.15.143:0/969366689 >> [v2:172.21.15.188:6836/3401128553,v1:172.21.15.188:6837/3401128553] conn(0x7f2c48066590 msgr2=0x7f2c48068a50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:24:09.257 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:09.255+0000 7f2c6ca9c640 1 --2- 172.21.15.143:0/969366689 >> [v2:172.21.15.188:6836/3401128553,v1:172.21.15.188:6837/3401128553] conn(0x7f2c48066590 0x7f2c48068a50 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7f2c500af000 tx=0x7f2c500a7070 comp rx=0 tx=0).stop 2023-11-26T07:24:09.257 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:09.255+0000 7f2c6ca9c640 1 -- 172.21.15.143:0/969366689 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f2c680708b0 msgr2=0x7f2c68158580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:24:09.257 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:09.255+0000 7f2c6ca9c640 1 --2- 172.21.15.143:0/969366689 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f2c680708b0 0x7f2c68158580 secure :-1 s=READY pgs=1294 cs=0 l=1 rev1=1 crypto rx=0x7f2c58010e60 tx=0x7f2c5800e730 comp rx=0 tx=0).stop 2023-11-26T07:24:09.257 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:09.255+0000 7f2c6ca9c640 1 -- 172.21.15.143:0/969366689 shutdown_connections 2023-11-26T07:24:09.257 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:09.255+0000 7f2c6ca9c640 1 --2- 172.21.15.143:0/969366689 >> [v2:172.21.15.188:6836/3401128553,v1:172.21.15.188:6837/3401128553] conn(0x7f2c48066590 0x7f2c48068a50 unknown :-1 s=CLOSED pgs=123 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:24:09.257 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:09.255+0000 7f2c6ca9c640 1 --2- 172.21.15.143:0/969366689 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f2c68143ac0 0x7f2c680cd000 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:24:09.257 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:09.255+0000 7f2c6ca9c640 1 --2- 172.21.15.143:0/969366689 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f2c680cf7c0 0x7f2c68137770 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:24:09.257 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:09.255+0000 7f2c6ca9c640 1 --2- 172.21.15.143:0/969366689 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f2c680708b0 0x7f2c68158580 unknown :-1 s=CLOSED pgs=1294 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:24:09.257 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:09.255+0000 7f2c6ca9c640 1 -- 172.21.15.143:0/969366689 >> 172.21.15.143:0/969366689 conn(0x7f2c6806a890 msgr2=0x7f2c6806ac70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:24:09.257 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:09.255+0000 7f2c6ca9c640 1 -- 172.21.15.143:0/969366689 shutdown_connections 2023-11-26T07:24:09.257 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:09.255+0000 7f2c6ca9c640 1 -- 172.21.15.143:0/969366689 wait complete. 2023-11-26T07:24:09.276 DEBUG:teuthology.orchestra.run.smithi143:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-11-26T07:24:09.414 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:09.415+0000 7ff3ec357640 1 Processor -- start 2023-11-26T07:24:09.414 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:09.415+0000 7ff3ec357640 1 -- start start 2023-11-26T07:24:09.416 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:09.415+0000 7ff3ec357640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7ff3e40d1bc0 0x7ff3e40d0df0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:24:09.416 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:09.415+0000 7ff3ec357640 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7ff3e40b2160 0x7ff3e40b0750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:24:09.416 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:09.415+0000 7ff3ec357640 1 -- --> v1:172.21.15.188:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff3e40cf0e0 con 0x7ff3e415d6e0 2023-11-26T07:24:09.416 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:09.415+0000 7ff3ec357640 1 -- --> v2:172.21.15.188:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff3e415dac0 con 0x7ff3e40b2160 2023-11-26T07:24:09.416 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:09.415+0000 7ff3ec357640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff3e40ceb20 con 0x7ff3e40d1bc0 2023-11-26T07:24:09.416 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:09.415+0000 7ff3ea8f4640 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7ff3e40b2160 0x7ff3e40b0750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:24:09.416 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:09.415+0000 7ff3e98f2640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7ff3e40d1bc0 0x7ff3e40d0df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:24:09.417 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:09.415+0000 7ff3ea8f4640 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7ff3e40b2160 0x7ff3e40b0750 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.188:3300/0 says I am v2:172.21.15.143:53510/0 (socket says 172.21.15.143:53510) 2023-11-26T07:24:09.417 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:09.415+0000 7ff3e98f2640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7ff3e40d1bc0 0x7ff3e40d0df0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.188:3301/0 says I am v2:172.21.15.143:45396/0 (socket says 172.21.15.143:45396) 2023-11-26T07:24:09.417 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:09.415+0000 7ff3ea8f4640 1 -- 172.21.15.143:0/1820951825 learned_addr learned my addr 172.21.15.143:0/1820951825 (peer_addr_for_me v2:172.21.15.143:0/0) 2023-11-26T07:24:09.417 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:09.415+0000 7ff3ea0f3640 1 --1- 172.21.15.143:0/1820951825 >> v1:172.21.15.188:6789/0 conn(0x7ff3e415d6e0 0x7ff3e40d14b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.188:6789/0 says I am v1:172.21.15.143:56530/0 (socket says 172.21.15.143:56530) 2023-11-26T07:24:09.417 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:09.415+0000 7ff3ea8f4640 1 -- 172.21.15.143:0/1820951825 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7ff3e40d1bc0 msgr2=0x7ff3e40d0df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-26T07:24:09.417 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:09.415+0000 7ff3ea8f4640 1 --2- 172.21.15.143:0/1820951825 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7ff3e40d1bc0 0x7ff3e40d0df0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:24:09.417 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:09.415+0000 7ff3ea8f4640 1 -- 172.21.15.143:0/1820951825 >> v1:172.21.15.188:6789/0 conn(0x7ff3e415d6e0 legacy=0x7ff3e40d14b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:24:09.417 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:09.415+0000 7ff3ea8f4640 1 -- 172.21.15.143:0/1820951825 --> v2:172.21.15.188:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff3e40cee00 con 0x7ff3e40b2160 2023-11-26T07:24:09.418 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:09.415+0000 7ff3e90f1640 1 -- 172.21.15.143:0/1820951825 <== mon.1 v1:172.21.15.188:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 80640223 0 0) 0x7ff3e40cf0e0 con 0x7ff3e415d6e0 2023-11-26T07:24:09.419 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:09.419+0000 7ff3ea8f4640 1 --2- 172.21.15.143:0/1820951825 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7ff3e40b2160 0x7ff3e40b0750 secure :-1 s=READY pgs=489 cs=0 l=1 rev1=1 crypto rx=0x7ff3dc009e50 tx=0x7ff3dc01ca10 comp rx=0 tx=0).ready entity=mon.1 client_cookie=9c87e88fcb4d55bb server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:24:09.420 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:09.419+0000 7ff3e90f1640 1 -- 172.21.15.143:0/1820951825 <== mon.1 v2:172.21.15.188:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff3dc0a57d0 con 0x7ff3e40b2160 2023-11-26T07:24:09.420 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:09.419+0000 7ff3e90f1640 1 -- 172.21.15.143:0/1820951825 <== mon.1 v2:172.21.15.188:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff3dc0a5930 con 0x7ff3e40b2160 2023-11-26T07:24:09.420 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:09.419+0000 7ff3e90f1640 1 -- 172.21.15.143:0/1820951825 <== mon.1 v2:172.21.15.188:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff3dc0a5cc0 con 0x7ff3e40b2160 2023-11-26T07:24:09.421 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:09.419+0000 7ff3ec357640 1 -- 172.21.15.143:0/1820951825 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7ff3e40b2160 msgr2=0x7ff3e40b0750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:24:09.421 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:09.419+0000 7ff3ec357640 1 --2- 172.21.15.143:0/1820951825 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7ff3e40b2160 0x7ff3e40b0750 secure :-1 s=READY pgs=489 cs=0 l=1 rev1=1 crypto rx=0x7ff3dc009e50 tx=0x7ff3dc01ca10 comp rx=0 tx=0).stop 2023-11-26T07:24:09.421 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:09.419+0000 7ff3ec357640 1 -- 172.21.15.143:0/1820951825 shutdown_connections 2023-11-26T07:24:09.421 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:09.419+0000 7ff3ec357640 1 --2- 172.21.15.143:0/1820951825 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7ff3e40b2160 0x7ff3e40b0750 unknown :-1 s=CLOSED pgs=489 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:24:09.421 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:09.419+0000 7ff3ec357640 1 --2- 172.21.15.143:0/1820951825 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7ff3e40d1bc0 0x7ff3e40d0df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:24:09.421 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:09.419+0000 7ff3ec357640 1 -- 172.21.15.143:0/1820951825 >> 172.21.15.143:0/1820951825 conn(0x7ff3e409c550 msgr2=0x7ff3e40d3220 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:24:09.421 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:09.419+0000 7ff3ec357640 1 -- 172.21.15.143:0/1820951825 shutdown_connections 2023-11-26T07:24:09.421 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:09.419+0000 7ff3ec357640 1 -- 172.21.15.143:0/1820951825 wait complete. 2023-11-26T07:24:09.422 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:09.423+0000 7ff3ec357640 1 Processor -- start 2023-11-26T07:24:09.422 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:09.423+0000 7ff3ec357640 1 -- start start 2023-11-26T07:24:09.423 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:09.423+0000 7ff3ec357640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7ff3e40b2160 0x7ff3e40ca5a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:24:09.424 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:09.423+0000 7ff3ec357640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7ff3e40d1bc0 0x7ff3e40c97b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:24:09.424 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:09.423+0000 7ff3ec357640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7ff3e415d6e0 0x7ff3e40cbd70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:24:09.424 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:09.423+0000 7ff3ec357640 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff3e415f060 con 0x7ff3e415d6e0 2023-11-26T07:24:09.424 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:09.423+0000 7ff3ec357640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff3e4158930 con 0x7ff3e40d1bc0 2023-11-26T07:24:09.424 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:09.423+0000 7ff3ec357640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff3e4158c10 con 0x7ff3e40b2160 2023-11-26T07:24:09.424 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:09.423+0000 7ff3ea8f4640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7ff3e415d6e0 0x7ff3e40cbd70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:24:09.424 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:09.423+0000 7ff3ea8f4640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7ff3e415d6e0 0x7ff3e40cbd70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:33726/0 (socket says 172.21.15.143:33726) 2023-11-26T07:24:09.424 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:09.423+0000 7ff3ea8f4640 1 -- 172.21.15.143:0/4122198198 learned_addr learned my addr 172.21.15.143:0/4122198198 (peer_addr_for_me v2:172.21.15.143:0/0) 2023-11-26T07:24:09.424 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:09.423+0000 7ff3e98f2640 1 --2- 172.21.15.143:0/4122198198 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7ff3e40d1bc0 0x7ff3e40c97b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:24:09.424 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:09.423+0000 7ff3ea0f3640 1 --2- 172.21.15.143:0/4122198198 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7ff3e40b2160 0x7ff3e40ca5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:24:09.425 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:09.423+0000 7ff3ea8f4640 1 -- 172.21.15.143:0/4122198198 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7ff3e40b2160 msgr2=0x7ff3e40ca5a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:24:09.425 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:09.423+0000 7ff3ea8f4640 1 --2- 172.21.15.143:0/4122198198 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7ff3e40b2160 0x7ff3e40ca5a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:24:09.425 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:09.423+0000 7ff3ea8f4640 1 -- 172.21.15.143:0/4122198198 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7ff3e40d1bc0 msgr2=0x7ff3e40c97b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:24:09.426 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:09.423+0000 7ff3ea8f4640 1 --2- 172.21.15.143:0/4122198198 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7ff3e40d1bc0 0x7ff3e40c97b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:24:09.426 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:09.423+0000 7ff3ea8f4640 1 -- 172.21.15.143:0/4122198198 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff3dc009b30 con 0x7ff3e415d6e0 2023-11-26T07:24:09.426 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:09.423+0000 7ff3ea8f4640 1 --2- 172.21.15.143:0/4122198198 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7ff3e415d6e0 0x7ff3e40cbd70 secure :-1 s=READY pgs=1295 cs=0 l=1 rev1=1 crypto rx=0x7ff3dc0af040 tx=0x7ff3dc0197b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:24:09.426 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:09.423+0000 7ff3daffd640 1 -- 172.21.15.143:0/4122198198 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff3dc0b8670 con 0x7ff3e415d6e0 2023-11-26T07:24:09.426 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:09.423+0000 7ff3daffd640 1 -- 172.21.15.143:0/4122198198 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff3dc0b87d0 con 0x7ff3e415d6e0 2023-11-26T07:24:09.426 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:09.423+0000 7ff3daffd640 1 -- 172.21.15.143:0/4122198198 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff3dc0b8b60 con 0x7ff3e415d6e0 2023-11-26T07:24:09.426 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:09.423+0000 7ff3ec357640 1 -- 172.21.15.143:0/4122198198 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff3e415ecf0 con 0x7ff3e415d6e0 2023-11-26T07:24:09.426 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:09.423+0000 7ff3ec357640 1 -- 172.21.15.143:0/4122198198 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff3e4158ef0 con 0x7ff3e415d6e0 2023-11-26T07:24:09.427 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:09.427+0000 7ff3ec357640 1 -- 172.21.15.143:0/4122198198 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff3e415f1d0 con 0x7ff3e415d6e0 2023-11-26T07:24:09.427 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:09.427+0000 7ff3daffd640 1 -- 172.21.15.143:0/4122198198 <== mon.0 v2:172.21.15.143:3300/0 4 ==== mgrmap(e 16) v1 ==== 85954+0+0 (secure 0 0 0) 0x7ff3dc0b8cc0 con 0x7ff3e415d6e0 2023-11-26T07:24:09.427 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:09.427+0000 7ff3daffd640 1 --2- 172.21.15.143:0/4122198198 >> [v2:172.21.15.188:6836/3401128553,v1:172.21.15.188:6837/3401128553] conn(0x7ff3c8066590 0x7ff3c8068a50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:24:09.428 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:09.427+0000 7ff3daffd640 1 -- 172.21.15.143:0/4122198198 <== mon.0 v2:172.21.15.143:3300/0 5 ==== osd_map(102..102 src has 1..102) v4 ==== 5709+0+0 (secure 0 0 0) 0x7ff3dc11f540 con 0x7ff3e415d6e0 2023-11-26T07:24:09.428 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:09.427+0000 7ff3ea0f3640 1 --2- 172.21.15.143:0/4122198198 >> [v2:172.21.15.188:6836/3401128553,v1:172.21.15.188:6837/3401128553] conn(0x7ff3c8066590 0x7ff3c8068a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:24:09.428 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:09.427+0000 7ff3ea0f3640 1 --2- 172.21.15.143:0/4122198198 >> [v2:172.21.15.188:6836/3401128553,v1:172.21.15.188:6837/3401128553] conn(0x7ff3c8066590 0x7ff3c8068a50 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7ff3e0000f70 tx=0x7ff3e0007df0 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:24:09.431 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:09.431+0000 7ff3daffd640 1 -- 172.21.15.143:0/4122198198 <== mon.0 v2:172.21.15.143:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172649 (secure 0 0 0) 0x7ff3dc0eca70 con 0x7ff3e415d6e0 2023-11-26T07:24:09.584 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:09.583+0000 7ff3ec357640 1 -- 172.21.15.143:0/4122198198 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7ff3e4067680 con 0x7ff3e415d6e0 2023-11-26T07:24:09.585 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:09.583+0000 7ff3daffd640 1 -- 172.21.15.143:0/4122198198 <== mon.0 v2:172.21.15.143:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v103) v1 ==== 74+0+16563 (secure 0 0 0) 0x7ff3dc0f0090 con 0x7ff3e415d6e0 2023-11-26T07:24:09.586 INFO:teuthology.orchestra.run.smithi143.stdout: 2023-11-26T07:24:09.586 INFO:teuthology.orchestra.run.smithi143.stdout:{"epoch":103,"fsid":"f5033060-2469-4a07-95fe-2b76f5404bc1","created":"2023-11-26T07:17:28.613663+0000","modified":"2023-11-26T07:24:09.510299+0000","last_up_change":"2023-11-26T07:17:41.157813+0000","last_in_change":"2023-11-26T07:17:31.181658+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":17,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-11-26T07:17:41.758597+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":16,"pool_name":"cephfs_metadata","create_time":"2023-11-26T07:24:06.278132+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"102","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":17,"pool_name":"cephfs_data","create_time":"2023-11-26T07:24:07.294850+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"103","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"ebbee2c6-023e-44b9-9f1d-6f7bfd309730","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":101,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6817","nonce":639807369},{"type":"v1","addr":"172.21.15.143:6819","nonce":639807369}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6821","nonce":639807369},{"type":"v1","addr":"172.21.15.143:6823","nonce":639807369}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6830","nonce":639807369},{"type":"v1","addr":"172.21.15.143:6831","nonce":639807369}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6825","nonce":639807369},{"type":"v1","addr":"172.21.15.143:6828","nonce":639807369}]},"public_addr":"172.21.15.143:6819/639807369","cluster_addr":"172.21.15.143:6823/639807369","heartbeat_back_addr":"172.21.15.143:6831/639807369","heartbeat_front_addr":"172.21.15.143:6828/639807369","state":["exists","up"]},{"osd":1,"uuid":"fec4b8be-7122-49e9-9a19-5850a0700001","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":101,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6816","nonce":1144942530},{"type":"v1","addr":"172.21.15.143:6818","nonce":1144942530}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6820","nonce":1144942530},{"type":"v1","addr":"172.21.15.143:6822","nonce":1144942530}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6827","nonce":1144942530},{"type":"v1","addr":"172.21.15.143:6829","nonce":1144942530}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6824","nonce":1144942530},{"type":"v1","addr":"172.21.15.143:6826","nonce":1144942530}]},"public_addr":"172.21.15.143:6818/1144942530","cluster_addr":"172.21.15.143:6822/1144942530","heartbeat_back_addr":"172.21.15.143:6829/1144942530","heartbeat_front_addr":"172.21.15.143:6826/1144942530","state":["exists","up"]},{"osd":2,"uuid":"2888af5b-f933-46ca-8f95-6dd494ff43f6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":101,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6808","nonce":3909095186},{"type":"v1","addr":"172.21.15.143:6809","nonce":3909095186}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6810","nonce":3909095186},{"type":"v1","addr":"172.21.15.143:6811","nonce":3909095186}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6814","nonce":3909095186},{"type":"v1","addr":"172.21.15.143:6815","nonce":3909095186}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6812","nonce":3909095186},{"type":"v1","addr":"172.21.15.143:6813","nonce":3909095186}]},"public_addr":"172.21.15.143:6809/3909095186","cluster_addr":"172.21.15.143:6811/3909095186","heartbeat_back_addr":"172.21.15.143:6815/3909095186","heartbeat_front_addr":"172.21.15.143:6813/3909095186","state":["exists","up"]},{"osd":3,"uuid":"d1f52435-7bc1-4081-9550-9cd1ba55f549","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":101,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6800","nonce":3854210991},{"type":"v1","addr":"172.21.15.143:6801","nonce":3854210991}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6802","nonce":3854210991},{"type":"v1","addr":"172.21.15.143:6803","nonce":3854210991}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6806","nonce":3854210991},{"type":"v1","addr":"172.21.15.143:6807","nonce":3854210991}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6804","nonce":3854210991},{"type":"v1","addr":"172.21.15.143:6805","nonce":3854210991}]},"public_addr":"172.21.15.143:6801/3854210991","cluster_addr":"172.21.15.143:6803/3854210991","heartbeat_back_addr":"172.21.15.143:6807/3854210991","heartbeat_front_addr":"172.21.15.143:6805/3854210991","state":["exists","up"]},{"osd":4,"uuid":"e2ae8b6e-6e25-49b2-bcbf-aacce68f06a3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":101,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6800","nonce":3924606902},{"type":"v1","addr":"172.21.15.188:6801","nonce":3924606902}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6802","nonce":3924606902},{"type":"v1","addr":"172.21.15.188:6803","nonce":3924606902}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6806","nonce":3924606902},{"type":"v1","addr":"172.21.15.188:6807","nonce":3924606902}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6804","nonce":3924606902},{"type":"v1","addr":"172.21.15.188:6805","nonce":3924606902}]},"public_addr":"172.21.15.188:6801/3924606902","cluster_addr":"172.21.15.188:6803/3924606902","heartbeat_back_addr":"172.21.15.188:6807/3924606902","heartbeat_front_addr":"172.21.15.188:6805/3924606902","state":["exists","up"]},{"osd":5,"uuid":"3cec8a32-7698-4c36-8e89-e4ba11934f33","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":101,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6813","nonce":186841544},{"type":"v1","addr":"172.21.15.188:6815","nonce":186841544}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6817","nonce":186841544},{"type":"v1","addr":"172.21.15.188:6818","nonce":186841544}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6822","nonce":186841544},{"type":"v1","addr":"172.21.15.188:6823","nonce":186841544}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6820","nonce":186841544},{"type":"v1","addr":"172.21.15.188:6821","nonce":186841544}]},"public_addr":"172.21.15.188:6815/186841544","cluster_addr":"172.21.15.188:6818/186841544","heartbeat_back_addr":"172.21.15.188:6823/186841544","heartbeat_front_addr":"172.21.15.188:6821/186841544","state":["exists","up"]},{"osd":6,"uuid":"58a65d5d-e090-473a-81b4-ad38e85029f9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":101,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6808","nonce":2153499405},{"type":"v1","addr":"172.21.15.188:6809","nonce":2153499405}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6810","nonce":2153499405},{"type":"v1","addr":"172.21.15.188:6811","nonce":2153499405}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6816","nonce":2153499405},{"type":"v1","addr":"172.21.15.188:6819","nonce":2153499405}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6812","nonce":2153499405},{"type":"v1","addr":"172.21.15.188:6814","nonce":2153499405}]},"public_addr":"172.21.15.188:6809/2153499405","cluster_addr":"172.21.15.188:6811/2153499405","heartbeat_back_addr":"172.21.15.188:6819/2153499405","heartbeat_front_addr":"172.21.15.188:6814/2153499405","state":["exists","up"]},{"osd":7,"uuid":"ea5b7f70-3554-4cf8-ac30-f7c59b80678d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":101,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6824","nonce":2722096729},{"type":"v1","addr":"172.21.15.188:6825","nonce":2722096729}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6826","nonce":2722096729},{"type":"v1","addr":"172.21.15.188:6827","nonce":2722096729}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6830","nonce":2722096729},{"type":"v1","addr":"172.21.15.188:6831","nonce":2722096729}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6828","nonce":2722096729},{"type":"v1","addr":"172.21.15.188:6829","nonce":2722096729}]},"public_addr":"172.21.15.188:6825/2722096729","cluster_addr":"172.21.15.188:6827/2722096729","heartbeat_back_addr":"172.21.15.188:6831/2722096729","heartbeat_front_addr":"172.21.15.188:6829/2722096729","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-26T07:17:36.995032+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-26T07:17:36.932266+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-26T07:17:36.829304+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-26T07:17:36.804297+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-26T07:17:36.667892+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-26T07:17:36.964868+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-26T07:17:36.827332+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-26T07:17:36.893179+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-11-26T07:24:09.588 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:09.587+0000 7ff3ec357640 1 -- 172.21.15.143:0/4122198198 >> [v2:172.21.15.188:6836/3401128553,v1:172.21.15.188:6837/3401128553] conn(0x7ff3c8066590 msgr2=0x7ff3c8068a50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:24:09.588 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:09.587+0000 7ff3ec357640 1 --2- 172.21.15.143:0/4122198198 >> [v2:172.21.15.188:6836/3401128553,v1:172.21.15.188:6837/3401128553] conn(0x7ff3c8066590 0x7ff3c8068a50 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7ff3e0000f70 tx=0x7ff3e0007df0 comp rx=0 tx=0).stop 2023-11-26T07:24:09.588 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:09.587+0000 7ff3ec357640 1 -- 172.21.15.143:0/4122198198 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7ff3e415d6e0 msgr2=0x7ff3e40cbd70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:24:09.588 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:09.587+0000 7ff3ec357640 1 --2- 172.21.15.143:0/4122198198 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7ff3e415d6e0 0x7ff3e40cbd70 secure :-1 s=READY pgs=1295 cs=0 l=1 rev1=1 crypto rx=0x7ff3dc0af040 tx=0x7ff3dc0197b0 comp rx=0 tx=0).stop 2023-11-26T07:24:09.588 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:09.587+0000 7ff3ec357640 1 -- 172.21.15.143:0/4122198198 shutdown_connections 2023-11-26T07:24:09.588 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:09.587+0000 7ff3ec357640 1 --2- 172.21.15.143:0/4122198198 >> [v2:172.21.15.188:6836/3401128553,v1:172.21.15.188:6837/3401128553] conn(0x7ff3c8066590 0x7ff3c8068a50 unknown :-1 s=CLOSED pgs=124 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:24:09.588 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:09.587+0000 7ff3ec357640 1 --2- 172.21.15.143:0/4122198198 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7ff3e415d6e0 0x7ff3e40cbd70 unknown :-1 s=CLOSED pgs=1295 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:24:09.588 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:09.587+0000 7ff3ec357640 1 --2- 172.21.15.143:0/4122198198 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7ff3e40d1bc0 0x7ff3e40c97b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:24:09.588 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:09.587+0000 7ff3ec357640 1 --2- 172.21.15.143:0/4122198198 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7ff3e40b2160 0x7ff3e40ca5a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:24:09.588 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:09.587+0000 7ff3ec357640 1 -- 172.21.15.143:0/4122198198 >> 172.21.15.143:0/4122198198 conn(0x7ff3e409c550 msgr2=0x7ff3e4159cf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:24:09.588 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:09.587+0000 7ff3ec357640 1 -- 172.21.15.143:0/4122198198 shutdown_connections 2023-11-26T07:24:09.589 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:09.587+0000 7ff3ec357640 1 -- 172.21.15.143:0/4122198198 wait complete. 2023-11-26T07:24:09.607 DEBUG:teuthology.orchestra.run.smithi143:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs standby_count_wanted 0 2023-11-26T07:24:09.748 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:09.747+0000 7f5036068640 1 Processor -- start 2023-11-26T07:24:09.748 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:09.747+0000 7f5036068640 1 -- start start 2023-11-26T07:24:09.749 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:09.747+0000 7f5036068640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f5030071850 0x7f50301596d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:24:09.749 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:09.747+0000 7f5036068640 1 -- --> v1:172.21.15.143:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f503006c750 con 0x7f50300b2870 2023-11-26T07:24:09.749 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:09.747+0000 7f5036068640 1 -- --> v1:172.21.15.188:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f50300b3550 con 0x7f50300ce680 2023-11-26T07:24:09.749 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:09.747+0000 7f5036068640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f50300cbe00 con 0x7f5030071850 2023-11-26T07:24:09.749 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:09.747+0000 7f502ffff640 1 --1- >> v1:172.21.15.143:6789/0 conn(0x7f50300b2870 0x7f5030164650 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.143:6789/0 says I am v1:172.21.15.143:42414/0 (socket says 172.21.15.143:42414) 2023-11-26T07:24:09.749 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:09.747+0000 7f502ffff640 1 -- 172.21.15.143:0/1304529530 learned_addr learned my addr 172.21.15.143:0/1304529530 (peer_addr_for_me v1:172.21.15.143:0/0) 2023-11-26T07:24:09.751 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:09.747+0000 7f502effd640 1 --2- 172.21.15.143:0/1304529530 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f5030071850 0x7f50301596d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:24:09.751 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:09.747+0000 7f502e7fc640 1 -- 172.21.15.143:0/1304529530 <== mon.0 v1:172.21.15.143:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1356302053 0 0) 0x7f503006c750 con 0x7f50300b2870 2023-11-26T07:24:09.751 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:09.747+0000 7f502e7fc640 1 -- 172.21.15.143:0/1304529530 --> v1:172.21.15.143:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f5018003530 con 0x7f50300b2870 2023-11-26T07:24:09.752 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:09.751+0000 7f502e7fc640 1 -- 172.21.15.143:0/1304529530 <== mon.0 v1:172.21.15.143:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3704972784 0 0) 0x7f5018003530 con 0x7f50300b2870 2023-11-26T07:24:09.752 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:09.751+0000 7f502e7fc640 1 -- 172.21.15.143:0/1304529530 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f5030071850 msgr2=0x7f50301596d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-26T07:24:09.752 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:09.751+0000 7f502e7fc640 1 --2- 172.21.15.143:0/1304529530 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f5030071850 0x7f50301596d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:24:09.752 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:09.751+0000 7f502e7fc640 1 -- 172.21.15.143:0/1304529530 >> v1:172.21.15.188:6789/0 conn(0x7f50300ce680 legacy=0x7f50300cf280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:24:09.752 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:09.751+0000 7f502e7fc640 1 -- 172.21.15.143:0/1304529530 --> v1:172.21.15.143:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5030071e80 con 0x7f50300b2870 2023-11-26T07:24:09.752 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:09.751+0000 7f502e7fc640 1 -- 172.21.15.143:0/1304529530 <== mon.0 v1:172.21.15.143:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2137882523 0 0) 0x7f5028001a00 con 0x7f50300b2870 2023-11-26T07:24:09.752 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:09.751+0000 7f502e7fc640 1 -- 172.21.15.143:0/1304529530 >> v1:172.21.15.143:6789/0 conn(0x7f50300b2870 legacy=0x7f5030164650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:24:09.752 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:09.751+0000 7f502e7fc640 1 --2- 172.21.15.143:0/1304529530 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f50180039b0 0x7f5018003d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:24:09.752 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:09.751+0000 7f502e7fc640 1 --2- 172.21.15.143:0/1304529530 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f5030071850 0x7f50180047d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:24:09.752 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:09.751+0000 7f502e7fc640 1 --2- 172.21.15.143:0/1304529530 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f5018004d10 0x7f5018005190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:24:09.752 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:09.751+0000 7f502e7fc640 1 -- 172.21.15.143:0/1304529530 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f50300cbe00 con 0x7f50180039b0 2023-11-26T07:24:09.752 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:09.751+0000 7f502e7fc640 1 -- 172.21.15.143:0/1304529530 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f501800b990 con 0x7f5018004d10 2023-11-26T07:24:09.753 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:09.751+0000 7f502e7fc640 1 -- 172.21.15.143:0/1304529530 --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f501800bb50 con 0x7f5030071850 2023-11-26T07:24:09.753 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:09.751+0000 7f502f7fe640 1 --2- 172.21.15.143:0/1304529530 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f50180039b0 0x7f5018003d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:24:09.753 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:09.751+0000 7f502effd640 1 --2- 172.21.15.143:0/1304529530 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f5030071850 0x7f50180047d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:24:09.753 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:09.751+0000 7f502ffff640 1 --2- 172.21.15.143:0/1304529530 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f5018004d10 0x7f5018005190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:24:09.753 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:09.751+0000 7f502f7fe640 1 -- 172.21.15.143:0/1304529530 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f5030071850 msgr2=0x7f50180047d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-26T07:24:09.753 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:09.751+0000 7f502f7fe640 1 --2- 172.21.15.143:0/1304529530 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f5030071850 0x7f50180047d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:24:09.753 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:09.751+0000 7f502f7fe640 1 -- 172.21.15.143:0/1304529530 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f5018004d10 msgr2=0x7f5018005190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-26T07:24:09.753 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:09.751+0000 7f502f7fe640 1 --2- 172.21.15.143:0/1304529530 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f5018004d10 0x7f5018005190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:24:09.753 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:09.751+0000 7f502f7fe640 1 -- 172.21.15.143:0/1304529530 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5018003530 con 0x7f50180039b0 2023-11-26T07:24:09.754 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:09.751+0000 7f502f7fe640 1 --2- 172.21.15.143:0/1304529530 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f50180039b0 0x7f5018003d90 secure :-1 s=READY pgs=1297 cs=0 l=1 rev1=1 crypto rx=0x7f5020001860 tx=0x7f502001c440 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f893839fd702bd5d server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:24:09.754 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:09.751+0000 7f502e7fc640 1 -- 172.21.15.143:0/1304529530 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f50200ab600 con 0x7f50180039b0 2023-11-26T07:24:09.754 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:09.755+0000 7f502e7fc640 1 -- 172.21.15.143:0/1304529530 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f50200a7050 con 0x7f50180039b0 2023-11-26T07:24:09.754 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:09.755+0000 7f502e7fc640 1 -- 172.21.15.143:0/1304529530 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f50200ab9a0 con 0x7f50180039b0 2023-11-26T07:24:09.755 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:09.755+0000 7f5036068640 1 -- 172.21.15.143:0/1304529530 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f50180039b0 msgr2=0x7f5018003d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:24:09.755 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:09.755+0000 7f5036068640 1 --2- 172.21.15.143:0/1304529530 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f50180039b0 0x7f5018003d90 secure :-1 s=READY pgs=1297 cs=0 l=1 rev1=1 crypto rx=0x7f5020001860 tx=0x7f502001c440 comp rx=0 tx=0).stop 2023-11-26T07:24:09.755 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:09.755+0000 7f502ffff640 1 -- 172.21.15.143:0/1304529530 reap_dead start 2023-11-26T07:24:09.755 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:09.755+0000 7f5036068640 1 -- 172.21.15.143:0/1304529530 shutdown_connections 2023-11-26T07:24:09.755 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:09.755+0000 7f5036068640 1 -- 172.21.15.143:0/1304529530 >> 172.21.15.143:0/1304529530 conn(0x7f503006a890 msgr2=0x7f503006aca0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:24:09.755 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:09.755+0000 7f5036068640 1 -- 172.21.15.143:0/1304529530 shutdown_connections 2023-11-26T07:24:09.755 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:09.755+0000 7f5036068640 1 -- 172.21.15.143:0/1304529530 wait complete. 2023-11-26T07:24:09.756 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:09.755+0000 7f5036068640 1 Processor -- start 2023-11-26T07:24:09.756 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:09.755+0000 7f5036068640 1 -- start start 2023-11-26T07:24:09.758 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:09.755+0000 7f5036068640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f50180039b0 0x7f50301f5c10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:24:09.758 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:09.755+0000 7f5036068640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f503015dca0 0x7f5030159d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:24:09.758 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:09.755+0000 7f5036068640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f503015aa70 0x7f503015f6e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:24:09.758 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:09.755+0000 7f5036068640 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5030153340 con 0x7f503015aa70 2023-11-26T07:24:09.759 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:09.755+0000 7f5036068640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f50301f9580 con 0x7f50180039b0 2023-11-26T07:24:09.759 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:09.755+0000 7f5036068640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f50301f9860 con 0x7f503015dca0 2023-11-26T07:24:09.759 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:09.759+0000 7f502ffff640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f503015aa70 0x7f503015f6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:24:09.759 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:09.759+0000 7f502ffff640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f503015aa70 0x7f503015f6e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:33742/0 (socket says 172.21.15.143:33742) 2023-11-26T07:24:09.759 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:09.759+0000 7f502ffff640 1 -- 172.21.15.143:0/156723242 learned_addr learned my addr 172.21.15.143:0/156723242 (peer_addr_for_me v2:172.21.15.143:0/0) 2023-11-26T07:24:09.759 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:09.759+0000 7f502effd640 1 --2- 172.21.15.143:0/156723242 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f503015dca0 0x7f5030159d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:24:09.759 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:09.759+0000 7f502f7fe640 1 --2- 172.21.15.143:0/156723242 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f50180039b0 0x7f50301f5c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:24:09.759 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:09.759+0000 7f502ffff640 1 -- 172.21.15.143:0/156723242 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f503015dca0 msgr2=0x7f5030159d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:24:09.759 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:09.759+0000 7f502ffff640 1 --2- 172.21.15.143:0/156723242 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f503015dca0 0x7f5030159d40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:24:09.759 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:09.759+0000 7f502ffff640 1 -- 172.21.15.143:0/156723242 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f50180039b0 msgr2=0x7f50301f5c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:24:09.759 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:09.759+0000 7f502ffff640 1 --2- 172.21.15.143:0/156723242 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f50180039b0 0x7f50301f5c10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:24:09.759 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:09.759+0000 7f502ffff640 1 -- 172.21.15.143:0/156723242 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f50300b3210 con 0x7f503015aa70 2023-11-26T07:24:09.759 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:09.759+0000 7f502ffff640 1 --2- 172.21.15.143:0/156723242 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f503015aa70 0x7f503015f6e0 secure :-1 s=READY pgs=1298 cs=0 l=1 rev1=1 crypto rx=0x7f503006e0f0 tx=0x7f502800e530 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:24:09.760 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:09.759+0000 7f5017fff640 1 -- 172.21.15.143:0/156723242 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f502800ec10 con 0x7f503015aa70 2023-11-26T07:24:09.760 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:09.759+0000 7f5017fff640 1 -- 172.21.15.143:0/156723242 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f502800ed90 con 0x7f503015aa70 2023-11-26T07:24:09.760 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:09.759+0000 7f5036068640 1 -- 172.21.15.143:0/156723242 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f50301f9b40 con 0x7f503015aa70 2023-11-26T07:24:09.760 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:09.759+0000 7f5017fff640 1 -- 172.21.15.143:0/156723242 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5028013660 con 0x7f503015aa70 2023-11-26T07:24:09.761 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:09.759+0000 7f5036068640 1 -- 172.21.15.143:0/156723242 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f50301fa100 con 0x7f503015aa70 2023-11-26T07:24:09.761 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:09.759+0000 7f5036068640 1 -- 172.21.15.143:0/156723242 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f503015b830 con 0x7f503015aa70 2023-11-26T07:24:09.761 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:09.759+0000 7f5017fff640 1 -- 172.21.15.143:0/156723242 <== mon.0 v2:172.21.15.143:3300/0 4 ==== mgrmap(e 16) v1 ==== 85954+0+0 (secure 0 0 0) 0x7f50280137c0 con 0x7f503015aa70 2023-11-26T07:24:09.761 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:09.759+0000 7f5017fff640 1 --2- 172.21.15.143:0/156723242 >> [v2:172.21.15.188:6836/3401128553,v1:172.21.15.188:6837/3401128553] conn(0x7f501c0665c0 0x7f501c068a80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:24:09.762 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:09.759+0000 7f5017fff640 1 -- 172.21.15.143:0/156723242 <== mon.0 v2:172.21.15.143:3300/0 5 ==== osd_map(103..103 src has 1..103) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f50280842d0 con 0x7f503015aa70 2023-11-26T07:24:09.762 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:09.763+0000 7f502f7fe640 1 --2- 172.21.15.143:0/156723242 >> [v2:172.21.15.188:6836/3401128553,v1:172.21.15.188:6837/3401128553] conn(0x7f501c0665c0 0x7f501c068a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:24:09.762 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:09.763+0000 7f502f7fe640 1 --2- 172.21.15.143:0/156723242 >> [v2:172.21.15.188:6836/3401128553,v1:172.21.15.188:6837/3401128553] conn(0x7f501c0665c0 0x7f501c068a80 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7f50200ad000 tx=0x7f50200a6000 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:24:09.765 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:09.763+0000 7f5017fff640 1 -- 172.21.15.143:0/156723242 <== mon.0 v2:172.21.15.143:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172649 (secure 0 0 0) 0x7f503015b830 con 0x7f503015aa70 2023-11-26T07:24:09.951 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:09.951+0000 7f5036068640 1 -- 172.21.15.143:0/156723242 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"} v 0) v1 -- 0x7f503015b830 con 0x7f503015aa70 2023-11-26T07:24:10.938 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:10.939+0000 7f5017fff640 1 -- 172.21.15.143:0/156723242 <== mon.0 v2:172.21.15.143:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]=0 v64) v1 ==== 118+0+0 (secure 0 0 0) 0x7f503015b830 con 0x7f503015aa70 2023-11-26T07:24:10.941 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:10.939+0000 7f5036068640 1 -- 172.21.15.143:0/156723242 >> [v2:172.21.15.188:6836/3401128553,v1:172.21.15.188:6837/3401128553] conn(0x7f501c0665c0 msgr2=0x7f501c068a80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:24:10.941 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:10.939+0000 7f5036068640 1 --2- 172.21.15.143:0/156723242 >> [v2:172.21.15.188:6836/3401128553,v1:172.21.15.188:6837/3401128553] conn(0x7f501c0665c0 0x7f501c068a80 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7f50200ad000 tx=0x7f50200a6000 comp rx=0 tx=0).stop 2023-11-26T07:24:10.941 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:10.939+0000 7f5036068640 1 -- 172.21.15.143:0/156723242 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f503015aa70 msgr2=0x7f503015f6e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:24:10.941 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:10.939+0000 7f5036068640 1 --2- 172.21.15.143:0/156723242 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f503015aa70 0x7f503015f6e0 secure :-1 s=READY pgs=1298 cs=0 l=1 rev1=1 crypto rx=0x7f503006e0f0 tx=0x7f502800e530 comp rx=0 tx=0).stop 2023-11-26T07:24:10.941 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:10.939+0000 7f5036068640 1 -- 172.21.15.143:0/156723242 shutdown_connections 2023-11-26T07:24:10.941 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:10.939+0000 7f5036068640 1 --2- 172.21.15.143:0/156723242 >> [v2:172.21.15.188:6836/3401128553,v1:172.21.15.188:6837/3401128553] conn(0x7f501c0665c0 0x7f501c068a80 unknown :-1 s=CLOSED pgs=125 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:24:10.942 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:10.939+0000 7f5036068640 1 --2- 172.21.15.143:0/156723242 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f503015aa70 0x7f503015f6e0 unknown :-1 s=CLOSED pgs=1298 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:24:10.942 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:10.939+0000 7f5036068640 1 --2- 172.21.15.143:0/156723242 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f503015dca0 0x7f5030159d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:24:10.942 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:10.939+0000 7f5036068640 1 --2- 172.21.15.143:0/156723242 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f50180039b0 0x7f50301f5c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:24:10.942 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:10.939+0000 7f5036068640 1 -- 172.21.15.143:0/156723242 >> 172.21.15.143:0/156723242 conn(0x7f503006a890 msgr2=0x7f50300cf250 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:24:10.942 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:10.939+0000 7f5036068640 1 -- 172.21.15.143:0/156723242 shutdown_connections 2023-11-26T07:24:10.942 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:10.939+0000 7f5036068640 1 -- 172.21.15.143:0/156723242 wait complete. 2023-11-26T07:24:10.964 DEBUG:tasks.cephfs.filesystem:fs_config: {} 2023-11-26T07:24:10.964 DEBUG:teuthology.orchestra.run.smithi143:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs allow_standby_replay false 2023-11-26T07:24:11.108 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:11.107+0000 7fc1d7df8640 1 Processor -- start 2023-11-26T07:24:11.108 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:11.107+0000 7fc1d7df8640 1 -- start start 2023-11-26T07:24:11.110 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:11.107+0000 7fc1d7df8640 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7fc1d00cf990 0x7fc1d00708c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:24:11.111 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:11.111+0000 7fc1d7df8640 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7fc1d0070e00 0x7fc1d0162510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:24:11.111 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:11.111+0000 7fc1d7df8640 1 -- --> v1:172.21.15.188:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc1d0162080 con 0x7fc1d014b1e0 2023-11-26T07:24:11.111 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:11.111+0000 7fc1d7df8640 1 -- --> v2:172.21.15.143:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc1d0161d70 con 0x7fc1d0070e00 2023-11-26T07:24:11.111 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:11.111+0000 7fc1d7df8640 1 -- --> v2:172.21.15.188:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc1d014b5c0 con 0x7fc1d00cf990 2023-11-26T07:24:11.111 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:11.111+0000 7fc1d6395640 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7fc1d0070e00 0x7fc1d0162510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:24:11.111 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:11.111+0000 7fc1d6395640 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7fc1d0070e00 0x7fc1d0162510 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:33754/0 (socket says 172.21.15.143:33754) 2023-11-26T07:24:11.111 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:11.111+0000 7fc1d6395640 1 -- 172.21.15.143:0/1039350973 learned_addr learned my addr 172.21.15.143:0/1039350973 (peer_addr_for_me v2:172.21.15.143:0/0) 2023-11-26T07:24:11.111 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:11.111+0000 7fc1d5393640 1 --2- 172.21.15.143:0/1039350973 >> v2:172.21.15.188:3300/0 conn(0x7fc1d00cf990 0x7fc1d00708c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:24:11.111 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:11.111+0000 7fc1d6395640 1 -- 172.21.15.143:0/1039350973 >> v2:172.21.15.188:3300/0 conn(0x7fc1d00cf990 msgr2=0x7fc1d00708c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-26T07:24:11.111 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:11.111+0000 7fc1d6395640 1 --2- 172.21.15.143:0/1039350973 >> v2:172.21.15.188:3300/0 conn(0x7fc1d00cf990 0x7fc1d00708c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:24:11.112 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:11.111+0000 7fc1d6395640 1 -- 172.21.15.143:0/1039350973 >> v1:172.21.15.188:6789/0 conn(0x7fc1d014b1e0 legacy=0x7fc1d00cf280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:24:11.112 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:11.111+0000 7fc1d6395640 1 -- 172.21.15.143:0/1039350973 --> v2:172.21.15.143:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc1d0071580 con 0x7fc1d0070e00 2023-11-26T07:24:11.113 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:11.111+0000 7fc1d6395640 1 --2- 172.21.15.143:0/1039350973 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fc1d0070e00 0x7fc1d0162510 secure :-1 s=READY pgs=1299 cs=0 l=1 rev1=1 crypto rx=0x7fc1d006dfa0 tx=0x7fc1c801c9a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fba3a3f0fa358aeb server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:24:11.113 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:11.111+0000 7fc1d4b92640 1 -- 172.21.15.143:0/1039350973 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc1c80a5860 con 0x7fc1d0070e00 2023-11-26T07:24:11.114 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:11.111+0000 7fc1d4b92640 1 -- 172.21.15.143:0/1039350973 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc1c80a59c0 con 0x7fc1d0070e00 2023-11-26T07:24:11.114 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:11.115+0000 7fc1d4b92640 1 -- 172.21.15.143:0/1039350973 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc1c80a5d50 con 0x7fc1d0070e00 2023-11-26T07:24:11.114 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:11.115+0000 7fc1d7df8640 1 -- 172.21.15.143:0/1039350973 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fc1d0070e00 msgr2=0x7fc1d0162510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:24:11.114 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:11.115+0000 7fc1d7df8640 1 --2- 172.21.15.143:0/1039350973 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fc1d0070e00 0x7fc1d0162510 secure :-1 s=READY pgs=1299 cs=0 l=1 rev1=1 crypto rx=0x7fc1d006dfa0 tx=0x7fc1c801c9a0 comp rx=0 tx=0).stop 2023-11-26T07:24:11.115 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:11.115+0000 7fc1d7df8640 1 -- 172.21.15.143:0/1039350973 shutdown_connections 2023-11-26T07:24:11.115 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:11.115+0000 7fc1d7df8640 1 --2- 172.21.15.143:0/1039350973 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fc1d0070e00 0x7fc1d0162510 unknown :-1 s=CLOSED pgs=1299 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:24:11.115 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:11.115+0000 7fc1d7df8640 1 --2- 172.21.15.143:0/1039350973 >> v2:172.21.15.188:3300/0 conn(0x7fc1d00cf990 0x7fc1d00708c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:24:11.115 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:11.115+0000 7fc1d7df8640 1 -- 172.21.15.143:0/1039350973 >> 172.21.15.143:0/1039350973 conn(0x7fc1d006a890 msgr2=0x7fc1d006aca0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:24:11.115 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:11.115+0000 7fc1d7df8640 1 -- 172.21.15.143:0/1039350973 shutdown_connections 2023-11-26T07:24:11.115 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:11.115+0000 7fc1d7df8640 1 -- 172.21.15.143:0/1039350973 wait complete. 2023-11-26T07:24:11.116 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:11.115+0000 7fc1d7df8640 1 Processor -- start 2023-11-26T07:24:11.116 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:11.115+0000 7fc1d7df8640 1 -- start start 2023-11-26T07:24:11.118 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:11.115+0000 7fc1d7df8640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fc1d0070e00 0x7fc1d01fa350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:24:11.118 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:11.115+0000 7fc1d7df8640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fc1d00cf990 0x7fc1d01429c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:24:11.118 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:11.115+0000 7fc1d7df8640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fc1d014b1e0 0x7fc1d0160be0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:24:11.118 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:11.115+0000 7fc1d7df8640 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc1d0142f00 con 0x7fc1d014b1e0 2023-11-26T07:24:11.118 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:11.115+0000 7fc1d7df8640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc1d01fb5a0 con 0x7fc1d0070e00 2023-11-26T07:24:11.118 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:11.115+0000 7fc1d7df8640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc1d01fa990 con 0x7fc1d00cf990 2023-11-26T07:24:11.118 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:11.115+0000 7fc1d6395640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fc1d014b1e0 0x7fc1d0160be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:24:11.118 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:11.115+0000 7fc1d6395640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fc1d014b1e0 0x7fc1d0160be0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:33766/0 (socket says 172.21.15.143:33766) 2023-11-26T07:24:11.119 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:11.115+0000 7fc1d6395640 1 -- 172.21.15.143:0/105999084 learned_addr learned my addr 172.21.15.143:0/105999084 (peer_addr_for_me v2:172.21.15.143:0/0) 2023-11-26T07:24:11.119 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:11.115+0000 7fc1d5b94640 1 --2- 172.21.15.143:0/105999084 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fc1d0070e00 0x7fc1d01fa350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:24:11.119 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:11.115+0000 7fc1d5393640 1 --2- 172.21.15.143:0/105999084 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fc1d00cf990 0x7fc1d01429c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:24:11.119 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:11.119+0000 7fc1d6395640 1 -- 172.21.15.143:0/105999084 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fc1d00cf990 msgr2=0x7fc1d01429c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:24:11.119 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:11.119+0000 7fc1d6395640 1 --2- 172.21.15.143:0/105999084 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fc1d00cf990 0x7fc1d01429c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:24:11.119 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:11.119+0000 7fc1d6395640 1 -- 172.21.15.143:0/105999084 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fc1d0070e00 msgr2=0x7fc1d01fa350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:24:11.119 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:11.119+0000 7fc1d6395640 1 --2- 172.21.15.143:0/105999084 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fc1d0070e00 0x7fc1d01fa350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:24:11.119 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:11.119+0000 7fc1d6395640 1 -- 172.21.15.143:0/105999084 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc1c8009b30 con 0x7fc1d014b1e0 2023-11-26T07:24:11.119 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:11.119+0000 7fc1d6395640 1 --2- 172.21.15.143:0/105999084 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fc1d014b1e0 0x7fc1d0160be0 secure :-1 s=READY pgs=1300 cs=0 l=1 rev1=1 crypto rx=0x7fc1c80195b0 tx=0x7fc1c8019740 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:24:11.120 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:11.119+0000 7fc1c67fc640 1 -- 172.21.15.143:0/105999084 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc1c80b8670 con 0x7fc1d014b1e0 2023-11-26T07:24:11.120 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:11.119+0000 7fc1c67fc640 1 -- 172.21.15.143:0/105999084 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc1c80b8810 con 0x7fc1d014b1e0 2023-11-26T07:24:11.120 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:11.119+0000 7fc1d7df8640 1 -- 172.21.15.143:0/105999084 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc1d01fac70 con 0x7fc1d014b1e0 2023-11-26T07:24:11.120 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:11.119+0000 7fc1c67fc640 1 -- 172.21.15.143:0/105999084 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc1c80b8bc0 con 0x7fc1d014b1e0 2023-11-26T07:24:11.120 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:11.119+0000 7fc1d7df8640 1 -- 172.21.15.143:0/105999084 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc1d01fb230 con 0x7fc1d014b1e0 2023-11-26T07:24:11.121 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:11.119+0000 7fc1d7df8640 1 -- 172.21.15.143:0/105999084 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc1d01fb750 con 0x7fc1d014b1e0 2023-11-26T07:24:11.121 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:11.119+0000 7fc1c67fc640 1 -- 172.21.15.143:0/105999084 <== mon.0 v2:172.21.15.143:3300/0 4 ==== mgrmap(e 16) v1 ==== 85954+0+0 (secure 0 0 0) 0x7fc1c80b8d20 con 0x7fc1d014b1e0 2023-11-26T07:24:11.122 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:11.119+0000 7fc1c67fc640 1 --2- 172.21.15.143:0/105999084 >> [v2:172.21.15.188:6836/3401128553,v1:172.21.15.188:6837/3401128553] conn(0x7fc1b4066590 0x7fc1b4068a50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:24:11.122 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:11.119+0000 7fc1c67fc640 1 -- 172.21.15.143:0/105999084 <== mon.0 v2:172.21.15.143:3300/0 5 ==== osd_map(103..103 src has 1..103) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fc1c811f5e0 con 0x7fc1d014b1e0 2023-11-26T07:24:11.122 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:11.123+0000 7fc1d5b94640 1 --2- 172.21.15.143:0/105999084 >> [v2:172.21.15.188:6836/3401128553,v1:172.21.15.188:6837/3401128553] conn(0x7fc1b4066590 0x7fc1b4068a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:24:11.122 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:11.123+0000 7fc1d5b94640 1 --2- 172.21.15.143:0/105999084 >> [v2:172.21.15.188:6836/3401128553,v1:172.21.15.188:6837/3401128553] conn(0x7fc1b4066590 0x7fc1b4068a50 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7fc1cc007fd0 tx=0x7fc1cc009040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:24:11.125 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:11.123+0000 7fc1c67fc640 1 -- 172.21.15.143:0/105999084 <== mon.0 v2:172.21.15.143:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172649 (secure 0 0 0) 0x7fc1d01fb750 con 0x7fc1d014b1e0 2023-11-26T07:24:11.305 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:11.303+0000 7fc1d7df8640 1 -- 172.21.15.143:0/105999084 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"} v 0) v1 -- 0x7fc1d01fb710 con 0x7fc1d014b1e0 2023-11-26T07:24:11.946 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:11.947+0000 7fc1c67fc640 1 -- 172.21.15.143:0/105999084 <== mon.0 v2:172.21.15.143:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"}]=0 v65) v1 ==== 122+0+0 (secure 0 0 0) 0x7fc1c80eca90 con 0x7fc1d014b1e0 2023-11-26T07:24:11.949 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:11.947+0000 7fc1d7df8640 1 -- 172.21.15.143:0/105999084 >> [v2:172.21.15.188:6836/3401128553,v1:172.21.15.188:6837/3401128553] conn(0x7fc1b4066590 msgr2=0x7fc1b4068a50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:24:11.949 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:11.947+0000 7fc1d7df8640 1 --2- 172.21.15.143:0/105999084 >> [v2:172.21.15.188:6836/3401128553,v1:172.21.15.188:6837/3401128553] conn(0x7fc1b4066590 0x7fc1b4068a50 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7fc1cc007fd0 tx=0x7fc1cc009040 comp rx=0 tx=0).stop 2023-11-26T07:24:11.949 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:11.947+0000 7fc1d7df8640 1 -- 172.21.15.143:0/105999084 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fc1d014b1e0 msgr2=0x7fc1d0160be0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:24:11.949 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:11.947+0000 7fc1d7df8640 1 --2- 172.21.15.143:0/105999084 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fc1d014b1e0 0x7fc1d0160be0 secure :-1 s=READY pgs=1300 cs=0 l=1 rev1=1 crypto rx=0x7fc1c80195b0 tx=0x7fc1c8019740 comp rx=0 tx=0).stop 2023-11-26T07:24:11.949 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:11.947+0000 7fc1d7df8640 1 -- 172.21.15.143:0/105999084 shutdown_connections 2023-11-26T07:24:11.949 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:11.947+0000 7fc1d7df8640 1 --2- 172.21.15.143:0/105999084 >> [v2:172.21.15.188:6836/3401128553,v1:172.21.15.188:6837/3401128553] conn(0x7fc1b4066590 0x7fc1b4068a50 unknown :-1 s=CLOSED pgs=126 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:24:11.949 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:11.947+0000 7fc1d7df8640 1 --2- 172.21.15.143:0/105999084 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fc1d014b1e0 0x7fc1d0160be0 unknown :-1 s=CLOSED pgs=1300 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:24:11.949 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:11.947+0000 7fc1d7df8640 1 --2- 172.21.15.143:0/105999084 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fc1d00cf990 0x7fc1d01429c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:24:11.949 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:11.947+0000 7fc1d7df8640 1 --2- 172.21.15.143:0/105999084 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fc1d0070e00 0x7fc1d01fa350 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:24:11.950 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:11.947+0000 7fc1d7df8640 1 -- 172.21.15.143:0/105999084 >> 172.21.15.143:0/105999084 conn(0x7fc1d006a890 msgr2=0x7fc1d00a85a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:24:11.950 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:11.947+0000 7fc1d7df8640 1 -- 172.21.15.143:0/105999084 shutdown_connections 2023-11-26T07:24:11.950 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:11.947+0000 7fc1d7df8640 1 -- 172.21.15.143:0/105999084 wait complete. 2023-11-26T07:24:11.970 DEBUG:tasks.cephfs.filesystem:Not Creating any subvols for filesystem 'cephfs' 2023-11-26T07:24:11.970 DEBUG:teuthology.orchestra.run.smithi143:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-11-26T07:24:12.110 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:12.111+0000 7eff8c9f7640 1 Processor -- start 2023-11-26T07:24:12.110 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:12.111+0000 7eff8c9f7640 1 -- start start 2023-11-26T07:24:12.111 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:12.111+0000 7eff8c9f7640 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7eff8815d2a0 0x7eff880d00e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:24:12.111 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:12.111+0000 7eff8c9f7640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7eff880d1d90 0x7eff880cec30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:24:12.112 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:12.111+0000 7eff8c9f7640 1 -- --> v1:172.21.15.143:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7eff880cdc40 con 0x7eff8815d680 2023-11-26T07:24:12.112 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:12.111+0000 7eff8c9f7640 1 -- --> v2:172.21.15.188:3300/0 -- mon_getmap magic: 0 v1 -- 0x7eff880ce7a0 con 0x7eff8815d2a0 2023-11-26T07:24:12.112 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:12.111+0000 7eff8c9f7640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7eff880d1440 con 0x7eff880d1d90 2023-11-26T07:24:12.112 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:12.111+0000 7eff7ffff640 1 --1- >> v1:172.21.15.143:6789/0 conn(0x7eff8815d680 0x7eff880d1680 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.143:6789/0 says I am v1:172.21.15.143:42420/0 (socket says 172.21.15.143:42420) 2023-11-26T07:24:12.112 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:12.111+0000 7eff7ffff640 1 -- 172.21.15.143:0/3974146170 learned_addr learned my addr 172.21.15.143:0/3974146170 (peer_addr_for_me v1:172.21.15.143:0/0) 2023-11-26T07:24:12.112 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:12.111+0000 7eff86d9d640 1 --2- 172.21.15.143:0/3974146170 >> v2:172.21.15.188:3300/0 conn(0x7eff8815d2a0 0x7eff880d00e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:24:12.112 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:12.111+0000 7eff8759e640 1 --2- 172.21.15.143:0/3974146170 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7eff880d1d90 0x7eff880cec30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:24:12.113 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:12.111+0000 7eff8659c640 1 -- 172.21.15.143:0/3974146170 <== mon.0 v1:172.21.15.143:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1030726490 0 0) 0x7eff880cdc40 con 0x7eff8815d680 2023-11-26T07:24:12.113 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:12.111+0000 7eff8659c640 1 -- 172.21.15.143:0/3974146170 --> v1:172.21.15.143:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7eff74003530 con 0x7eff8815d680 2023-11-26T07:24:12.113 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:12.111+0000 7eff8659c640 1 -- 172.21.15.143:0/3974146170 <== mon.0 v1:172.21.15.143:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 4221646189 0 0) 0x7eff74003530 con 0x7eff8815d680 2023-11-26T07:24:12.113 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:12.111+0000 7eff86d9d640 1 -- 172.21.15.143:0/3974146170 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7eff880d1d90 msgr2=0x7eff880cec30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-26T07:24:12.113 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:12.111+0000 7eff86d9d640 1 --2- 172.21.15.143:0/3974146170 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7eff880d1d90 0x7eff880cec30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:24:12.113 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:12.111+0000 7eff86d9d640 1 -- 172.21.15.143:0/3974146170 >> v1:172.21.15.143:6789/0 conn(0x7eff8815d680 legacy=0x7eff880d1680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:24:12.113 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:12.111+0000 7eff86d9d640 1 -- 172.21.15.143:0/3974146170 --> v2:172.21.15.188:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7eff880cde70 con 0x7eff8815d2a0 2023-11-26T07:24:12.115 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:12.115+0000 7eff86d9d640 1 --2- 172.21.15.143:0/3974146170 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7eff8815d2a0 0x7eff880d00e0 secure :-1 s=READY pgs=492 cs=0 l=1 rev1=1 crypto rx=0x7eff780097c0 tx=0x7eff7801c370 comp rx=0 tx=0).ready entity=mon.1 client_cookie=68c54ddd3fe24c98 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:24:12.115 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:12.115+0000 7eff8659c640 1 -- 172.21.15.143:0/3974146170 <== mon.1 v2:172.21.15.188:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7eff70001680 con 0x7eff8815d2a0 2023-11-26T07:24:12.115 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:12.115+0000 7eff8659c640 1 -- 172.21.15.143:0/3974146170 <== mon.1 v2:172.21.15.188:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7eff780a7050 con 0x7eff8815d2a0 2023-11-26T07:24:12.116 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:12.115+0000 7eff8659c640 1 -- 172.21.15.143:0/3974146170 <== mon.1 v2:172.21.15.188:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7eff780ab600 con 0x7eff8815d2a0 2023-11-26T07:24:12.116 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:12.115+0000 7eff8c9f7640 1 -- 172.21.15.143:0/3974146170 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7eff8815d2a0 msgr2=0x7eff880d00e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:24:12.116 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:12.115+0000 7eff8c9f7640 1 --2- 172.21.15.143:0/3974146170 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7eff8815d2a0 0x7eff880d00e0 secure :-1 s=READY pgs=492 cs=0 l=1 rev1=1 crypto rx=0x7eff780097c0 tx=0x7eff7801c370 comp rx=0 tx=0).stop 2023-11-26T07:24:12.116 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:12.115+0000 7eff8c9f7640 1 -- 172.21.15.143:0/3974146170 shutdown_connections 2023-11-26T07:24:12.117 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:12.115+0000 7eff8c9f7640 1 --2- 172.21.15.143:0/3974146170 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7eff880d1d90 0x7eff880cec30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:24:12.117 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:12.115+0000 7eff8c9f7640 1 --2- 172.21.15.143:0/3974146170 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7eff8815d2a0 0x7eff880d00e0 unknown :-1 s=CLOSED pgs=492 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:24:12.117 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:12.115+0000 7eff8c9f7640 1 -- 172.21.15.143:0/3974146170 >> 172.21.15.143:0/3974146170 conn(0x7eff880d30a0 msgr2=0x7eff880d3480 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:24:12.117 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:12.115+0000 7eff8c9f7640 1 -- 172.21.15.143:0/3974146170 shutdown_connections 2023-11-26T07:24:12.117 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:12.115+0000 7eff8c9f7640 1 -- 172.21.15.143:0/3974146170 wait complete. 2023-11-26T07:24:12.118 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:12.119+0000 7eff8c9f7640 1 Processor -- start 2023-11-26T07:24:12.118 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:12.119+0000 7eff8c9f7640 1 -- start start 2023-11-26T07:24:12.120 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:12.119+0000 7eff8c9f7640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7eff880d1d90 0x7eff880cb400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:24:12.120 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:12.119+0000 7eff8c9f7640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7eff8815d2a0 0x7eff88156290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:24:12.120 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:12.119+0000 7eff8c9f7640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7eff8815d680 0x7eff880b1b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:24:12.120 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:12.119+0000 7eff8c9f7640 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x7eff880cb940 con 0x7eff8815d680 2023-11-26T07:24:12.120 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:12.119+0000 7eff8c9f7640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7eff881567d0 con 0x7eff8815d2a0 2023-11-26T07:24:12.120 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:12.119+0000 7eff8c9f7640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7eff88156cf0 con 0x7eff880d1d90 2023-11-26T07:24:12.120 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:12.119+0000 7eff8759e640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7eff8815d680 0x7eff880b1b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:24:12.121 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:12.119+0000 7eff8759e640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7eff8815d680 0x7eff880b1b50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:33770/0 (socket says 172.21.15.143:33770) 2023-11-26T07:24:12.121 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:12.119+0000 7eff8759e640 1 -- 172.21.15.143:0/2503414955 learned_addr learned my addr 172.21.15.143:0/2503414955 (peer_addr_for_me v2:172.21.15.143:0/0) 2023-11-26T07:24:12.121 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:12.119+0000 7eff8759e640 1 -- 172.21.15.143:0/2503414955 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7eff880d1d90 msgr2=0x7eff880cb400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:24:12.121 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:12.119+0000 7eff8759e640 1 --2- 172.21.15.143:0/2503414955 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7eff880d1d90 0x7eff880cb400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:24:12.121 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:12.119+0000 7eff8759e640 1 -- 172.21.15.143:0/2503414955 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7eff8815d2a0 msgr2=0x7eff88156290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:24:12.121 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:12.119+0000 7eff7ffff640 1 --2- 172.21.15.143:0/2503414955 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7eff8815d2a0 0x7eff88156290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:24:12.121 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:12.119+0000 7eff8759e640 1 --2- 172.21.15.143:0/2503414955 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7eff8815d2a0 0x7eff88156290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:24:12.121 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:12.119+0000 7eff8759e640 1 -- 172.21.15.143:0/2503414955 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7eff78009420 con 0x7eff8815d680 2023-11-26T07:24:12.121 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:12.119+0000 7eff7ffff640 1 --2- 172.21.15.143:0/2503414955 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7eff8815d2a0 0x7eff88156290 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-26T07:24:12.121 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:12.119+0000 7eff8759e640 1 --2- 172.21.15.143:0/2503414955 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7eff8815d680 0x7eff880b1b50 secure :-1 s=READY pgs=1302 cs=0 l=1 rev1=1 crypto rx=0x7eff80009cb0 tx=0x7eff8000c490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:24:12.121 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:12.119+0000 7eff7f7fe640 1 -- 172.21.15.143:0/2503414955 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7eff80009b20 con 0x7eff8815d680 2023-11-26T07:24:12.121 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:12.119+0000 7eff7f7fe640 1 -- 172.21.15.143:0/2503414955 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7eff8000f040 con 0x7eff8815d680 2023-11-26T07:24:12.121 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:12.119+0000 7eff7f7fe640 1 -- 172.21.15.143:0/2503414955 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7eff800146c0 con 0x7eff8815d680 2023-11-26T07:24:12.122 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:12.119+0000 7eff8c9f7640 1 -- 172.21.15.143:0/2503414955 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7eff880b0ab0 con 0x7eff8815d680 2023-11-26T07:24:12.122 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:12.119+0000 7eff8c9f7640 1 -- 172.21.15.143:0/2503414955 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7eff880b1070 con 0x7eff8815d680 2023-11-26T07:24:12.122 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:12.119+0000 7eff8c9f7640 1 -- 172.21.15.143:0/2503414955 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7eff88067500 con 0x7eff8815d680 2023-11-26T07:24:12.122 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:12.123+0000 7eff7f7fe640 1 -- 172.21.15.143:0/2503414955 <== mon.0 v2:172.21.15.143:3300/0 4 ==== mgrmap(e 16) v1 ==== 85954+0+0 (secure 0 0 0) 0x7eff80014820 con 0x7eff8815d680 2023-11-26T07:24:12.123 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:12.123+0000 7eff7f7fe640 1 --2- 172.21.15.143:0/2503414955 >> [v2:172.21.15.188:6836/3401128553,v1:172.21.15.188:6837/3401128553] conn(0x7eff68066590 0x7eff68068a50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:24:12.123 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:12.123+0000 7eff7f7fe640 1 -- 172.21.15.143:0/2503414955 <== mon.0 v2:172.21.15.143:3300/0 5 ==== osd_map(103..103 src has 1..103) v4 ==== 5709+0+0 (secure 0 0 0) 0x7eff80084f00 con 0x7eff8815d680 2023-11-26T07:24:12.123 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:12.123+0000 7eff86d9d640 1 --2- 172.21.15.143:0/2503414955 >> [v2:172.21.15.188:6836/3401128553,v1:172.21.15.188:6837/3401128553] conn(0x7eff68066590 0x7eff68068a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:24:12.124 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:12.123+0000 7eff86d9d640 1 --2- 172.21.15.143:0/2503414955 >> [v2:172.21.15.188:6836/3401128553,v1:172.21.15.188:6837/3401128553] conn(0x7eff68066590 0x7eff68068a50 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7eff780ad000 tx=0x7eff78004740 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:24:12.127 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:12.127+0000 7eff7f7fe640 1 -- 172.21.15.143:0/2503414955 <== mon.0 v2:172.21.15.143:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172649 (secure 0 0 0) 0x7eff800520c0 con 0x7eff8815d680 2023-11-26T07:24:12.306 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:12.307+0000 7eff8c9f7640 1 -- 172.21.15.143:0/2503414955 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7eff8808c7f0 con 0x7eff8815d680 2023-11-26T07:24:12.306 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:12.307+0000 7eff7f7fe640 1 -- 172.21.15.143:0/2503414955 <== mon.0 v2:172.21.15.143:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 65 v65) v1 ==== 94+0+4927 (secure 0 0 0) 0x7eff80018090 con 0x7eff8815d680 2023-11-26T07:24:12.307 INFO:teuthology.orchestra.run.smithi143.stdout: 2023-11-26T07:24:12.307 INFO:teuthology.orchestra.run.smithi143.stdout:{"epoch":65,"default_fscid":8,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6629,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.188:6834/1327398471","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6832","nonce":1327398471},{"type":"v1","addr":"172.21.15.188:6834","nonce":1327398471}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":60},{"gid":7453,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.188:6835/1640976426","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6833","nonce":1640976426},{"type":"v1","addr":"172.21.15.188:6835","nonce":1640976426}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":60},{"gid":7941,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.143:6834/2855111671","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6832","nonce":2855111671},{"type":"v1","addr":"172.21.15.143:6834","nonce":2855111671}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":60}],"filesystems":[{"mdsmap":{"epoch":65,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-11-26T07:24:08.295882+0000","modified":"2023-11-26T07:24:11.308119+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":7944},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7944":{"gid":7944,"name":"a","rank":0,"incarnation":61,"state":"up:active","state_seq":3,"addr":"172.21.15.143:6835/2571525464","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6833","nonce":2571525464},{"type":"v1","addr":"172.21.15.143:6835","nonce":2571525464}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[17],"metadata_pool":16,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":8}]} 2023-11-26T07:24:12.307 INFO:teuthology.orchestra.run.smithi143.stderr:dumped fsmap epoch 65 2023-11-26T07:24:12.309 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:12.307+0000 7eff8c9f7640 1 -- 172.21.15.143:0/2503414955 >> [v2:172.21.15.188:6836/3401128553,v1:172.21.15.188:6837/3401128553] conn(0x7eff68066590 msgr2=0x7eff68068a50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:24:12.309 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:12.307+0000 7eff8c9f7640 1 --2- 172.21.15.143:0/2503414955 >> [v2:172.21.15.188:6836/3401128553,v1:172.21.15.188:6837/3401128553] conn(0x7eff68066590 0x7eff68068a50 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7eff780ad000 tx=0x7eff78004740 comp rx=0 tx=0).stop 2023-11-26T07:24:12.309 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:12.307+0000 7eff8c9f7640 1 -- 172.21.15.143:0/2503414955 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7eff8815d680 msgr2=0x7eff880b1b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:24:12.309 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:12.307+0000 7eff8c9f7640 1 --2- 172.21.15.143:0/2503414955 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7eff8815d680 0x7eff880b1b50 secure :-1 s=READY pgs=1302 cs=0 l=1 rev1=1 crypto rx=0x7eff80009cb0 tx=0x7eff8000c490 comp rx=0 tx=0).stop 2023-11-26T07:24:12.309 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:12.307+0000 7eff8c9f7640 1 -- 172.21.15.143:0/2503414955 shutdown_connections 2023-11-26T07:24:12.309 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:12.307+0000 7eff8c9f7640 1 --2- 172.21.15.143:0/2503414955 >> [v2:172.21.15.188:6836/3401128553,v1:172.21.15.188:6837/3401128553] conn(0x7eff68066590 0x7eff68068a50 unknown :-1 s=CLOSED pgs=127 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:24:12.309 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:12.307+0000 7eff8c9f7640 1 --2- 172.21.15.143:0/2503414955 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7eff8815d680 0x7eff880b1b50 unknown :-1 s=CLOSED pgs=1302 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:24:12.309 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:12.307+0000 7eff8c9f7640 1 --2- 172.21.15.143:0/2503414955 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7eff8815d2a0 0x7eff88156290 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:24:12.310 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:12.307+0000 7eff8c9f7640 1 --2- 172.21.15.143:0/2503414955 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7eff880d1d90 0x7eff880cb400 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:24:12.310 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:12.307+0000 7eff8c9f7640 1 -- 172.21.15.143:0/2503414955 >> 172.21.15.143:0/2503414955 conn(0x7eff880d30a0 msgr2=0x7eff880c84b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:24:12.310 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:12.307+0000 7eff8c9f7640 1 -- 172.21.15.143:0/2503414955 shutdown_connections 2023-11-26T07:24:12.310 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:12.307+0000 7eff8c9f7640 1 -- 172.21.15.143:0/2503414955 wait complete. 2023-11-26T07:24:12.328 DEBUG:teuthology.orchestra.run.smithi143:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-11-26T07:24:12.466 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:12.467+0000 7ff7c8965640 1 Processor -- start 2023-11-26T07:24:12.466 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:12.467+0000 7ff7c8965640 1 -- start start 2023-11-26T07:24:12.468 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:12.467+0000 7ff7c8965640 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7ff7c413e290 0x7ff7c40cf280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:24:12.468 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:12.467+0000 7ff7c8965640 1 -- --> v1:172.21.15.143:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff7c415fb20 con 0x7ff7c415f6b0 2023-11-26T07:24:12.468 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:12.467+0000 7ff7c8965640 1 -- --> v1:172.21.15.188:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff7c406c520 con 0x7ff7c40cf7c0 2023-11-26T07:24:12.468 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:12.467+0000 7ff7c8965640 1 -- --> v2:172.21.15.188:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff7c40cfc40 con 0x7ff7c413e290 2023-11-26T07:24:12.468 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:12.467+0000 7ff7c2d9d640 1 --1- >> v1:172.21.15.143:6789/0 conn(0x7ff7c415f6b0 0x7ff7c40cd7b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.143:6789/0 says I am v1:172.21.15.143:42434/0 (socket says 172.21.15.143:42434) 2023-11-26T07:24:12.468 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:12.467+0000 7ff7c2d9d640 1 -- 172.21.15.143:0/2196674457 learned_addr learned my addr 172.21.15.143:0/2196674457 (peer_addr_for_me v1:172.21.15.143:0/0) 2023-11-26T07:24:12.470 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:12.467+0000 7ff7c259c640 1 --2- 172.21.15.143:0/2196674457 >> v2:172.21.15.188:3300/0 conn(0x7ff7c413e290 0x7ff7c40cf280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:24:12.470 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:12.467+0000 7ff7c159a640 1 -- 172.21.15.143:0/2196674457 <== mon.0 v1:172.21.15.143:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3146195291 0 0) 0x7ff7bc0014a0 con 0x7ff7c415f6b0 2023-11-26T07:24:12.470 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:12.467+0000 7ff7c159a640 1 -- 172.21.15.143:0/2196674457 --> v1:172.21.15.143:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7ff7ac003530 con 0x7ff7c415f6b0 2023-11-26T07:24:12.470 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:12.467+0000 7ff7c159a640 1 -- 172.21.15.143:0/2196674457 <== mon.0 v1:172.21.15.143:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2586536634 0 0) 0x7ff7ac003530 con 0x7ff7c415f6b0 2023-11-26T07:24:12.470 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:12.467+0000 7ff7c159a640 1 -- 172.21.15.143:0/2196674457 >> v2:172.21.15.188:3300/0 conn(0x7ff7c413e290 msgr2=0x7ff7c40cf280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-26T07:24:12.470 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:12.467+0000 7ff7c159a640 1 --2- 172.21.15.143:0/2196674457 >> v2:172.21.15.188:3300/0 conn(0x7ff7c413e290 0x7ff7c40cf280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:24:12.470 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:12.467+0000 7ff7c159a640 1 -- 172.21.15.143:0/2196674457 >> v1:172.21.15.188:6789/0 conn(0x7ff7c40cf7c0 legacy=0x7ff7c415efa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:24:12.471 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:12.467+0000 7ff7c159a640 1 -- 172.21.15.143:0/2196674457 --> v1:172.21.15.143:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff7c406c800 con 0x7ff7c415f6b0 2023-11-26T07:24:12.471 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:12.467+0000 7ff7c159a640 1 -- 172.21.15.143:0/2196674457 <== mon.0 v1:172.21.15.143:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2137882523 0 0) 0x7ff7bc001bb0 con 0x7ff7c415f6b0 2023-11-26T07:24:12.471 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:12.467+0000 7ff7c159a640 1 -- 172.21.15.143:0/2196674457 >> v1:172.21.15.143:6789/0 conn(0x7ff7c415f6b0 legacy=0x7ff7c40cd7b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:24:12.471 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:12.467+0000 7ff7c159a640 1 --2- 172.21.15.143:0/2196674457 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7ff7ac003980 0x7ff7ac003da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:24:12.471 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:12.467+0000 7ff7c159a640 1 --2- 172.21.15.143:0/2196674457 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7ff7ac0047d0 0x7ff7ac004c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:24:12.471 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:12.467+0000 7ff7c159a640 1 --2- 172.21.15.143:0/2196674457 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7ff7ac005190 0x7ff7ac00b990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:24:12.471 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:12.467+0000 7ff7c159a640 1 -- 172.21.15.143:0/2196674457 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff7c40cfc40 con 0x7ff7ac0047d0 2023-11-26T07:24:12.471 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:12.467+0000 7ff7c159a640 1 -- 172.21.15.143:0/2196674457 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff7ac00c030 con 0x7ff7ac003980 2023-11-26T07:24:12.471 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:12.467+0000 7ff7c159a640 1 -- 172.21.15.143:0/2196674457 --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff7ac00c1f0 con 0x7ff7ac005190 2023-11-26T07:24:12.471 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:12.467+0000 7ff7c1d9b640 1 --2- 172.21.15.143:0/2196674457 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7ff7ac0047d0 0x7ff7ac004c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:24:12.471 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:12.467+0000 7ff7c259c640 1 --2- 172.21.15.143:0/2196674457 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7ff7ac003980 0x7ff7ac003da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:24:12.471 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:12.467+0000 7ff7c2d9d640 1 --2- 172.21.15.143:0/2196674457 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7ff7ac005190 0x7ff7ac00b990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:24:12.471 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:12.467+0000 7ff7c1d9b640 1 -- 172.21.15.143:0/2196674457 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7ff7ac005190 msgr2=0x7ff7ac00b990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-26T07:24:12.472 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:12.467+0000 7ff7c1d9b640 1 --2- 172.21.15.143:0/2196674457 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7ff7ac005190 0x7ff7ac00b990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:24:12.472 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:12.467+0000 7ff7c1d9b640 1 -- 172.21.15.143:0/2196674457 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7ff7ac003980 msgr2=0x7ff7ac003da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-26T07:24:12.472 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:12.467+0000 7ff7c1d9b640 1 --2- 172.21.15.143:0/2196674457 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7ff7ac003980 0x7ff7ac003da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:24:12.472 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:12.467+0000 7ff7c1d9b640 1 -- 172.21.15.143:0/2196674457 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff7ac003530 con 0x7ff7ac0047d0 2023-11-26T07:24:12.472 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:12.467+0000 7ff7c2d9d640 1 -- 172.21.15.143:0/2196674457 reap_dead start 2023-11-26T07:24:12.472 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:12.471+0000 7ff7c1d9b640 1 --2- 172.21.15.143:0/2196674457 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7ff7ac0047d0 0x7ff7ac004c50 secure :-1 s=READY pgs=1304 cs=0 l=1 rev1=1 crypto rx=0x7ff7b8001780 tx=0x7ff7b801c3e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3ef477f8cc0fdcda server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:24:12.473 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:12.471+0000 7ff7c159a640 1 -- 172.21.15.143:0/2196674457 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff7b80052d0 con 0x7ff7ac0047d0 2023-11-26T07:24:12.473 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:12.471+0000 7ff7c159a640 1 -- 172.21.15.143:0/2196674457 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff7b80a7050 con 0x7ff7ac0047d0 2023-11-26T07:24:12.473 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:12.471+0000 7ff7c159a640 1 -- 172.21.15.143:0/2196674457 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff7b8003600 con 0x7ff7ac0047d0 2023-11-26T07:24:12.473 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:12.471+0000 7ff7c8965640 1 -- 172.21.15.143:0/2196674457 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7ff7ac0047d0 msgr2=0x7ff7ac004c50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:24:12.473 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:12.471+0000 7ff7c8965640 1 --2- 172.21.15.143:0/2196674457 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7ff7ac0047d0 0x7ff7ac004c50 secure :-1 s=READY pgs=1304 cs=0 l=1 rev1=1 crypto rx=0x7ff7b8001780 tx=0x7ff7b801c3e0 comp rx=0 tx=0).stop 2023-11-26T07:24:12.474 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:12.471+0000 7ff7c8965640 1 -- 172.21.15.143:0/2196674457 shutdown_connections 2023-11-26T07:24:12.474 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:12.471+0000 7ff7c8965640 1 --2- 172.21.15.143:0/2196674457 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7ff7ac0047d0 0x7ff7ac004c50 unknown :-1 s=CLOSED pgs=1304 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:24:12.474 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:12.471+0000 7ff7c8965640 1 -- 172.21.15.143:0/2196674457 >> 172.21.15.143:0/2196674457 conn(0x7ff7c406a890 msgr2=0x7ff7c406aca0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:24:12.474 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:12.471+0000 7ff7c8965640 1 -- 172.21.15.143:0/2196674457 shutdown_connections 2023-11-26T07:24:12.474 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:12.471+0000 7ff7c8965640 1 -- 172.21.15.143:0/2196674457 wait complete. 2023-11-26T07:24:12.475 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:12.475+0000 7ff7c8965640 1 Processor -- start 2023-11-26T07:24:12.475 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:12.475+0000 7ff7c8965640 1 -- start start 2023-11-26T07:24:12.477 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:12.475+0000 7ff7c8965640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7ff7ac0047d0 0x7ff7c41554d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:24:12.477 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:12.475+0000 7ff7c8965640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7ff7c4149f00 0x7ff7c415a150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:24:12.477 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:12.475+0000 7ff7c8965640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7ff7c415a690 0x7ff7c4136530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:24:12.477 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:12.475+0000 7ff7c8965640 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff7c415ab40 con 0x7ff7c4149f00 2023-11-26T07:24:12.478 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:12.475+0000 7ff7c8965640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff7c415e2d0 con 0x7ff7ac0047d0 2023-11-26T07:24:12.478 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:12.475+0000 7ff7c8965640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff7c41f3000 con 0x7ff7c415a690 2023-11-26T07:24:12.478 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:12.475+0000 7ff7c1d9b640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7ff7c4149f00 0x7ff7c415a150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:24:12.478 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:12.475+0000 7ff7c1d9b640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7ff7c4149f00 0x7ff7c415a150 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:33788/0 (socket says 172.21.15.143:33788) 2023-11-26T07:24:12.478 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:12.475+0000 7ff7c1d9b640 1 -- 172.21.15.143:0/655954751 learned_addr learned my addr 172.21.15.143:0/655954751 (peer_addr_for_me v2:172.21.15.143:0/0) 2023-11-26T07:24:12.478 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:12.475+0000 7ff7c2d9d640 1 --2- 172.21.15.143:0/655954751 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7ff7c415a690 0x7ff7c4136530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:24:12.478 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:12.475+0000 7ff7c259c640 1 --2- 172.21.15.143:0/655954751 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7ff7ac0047d0 0x7ff7c41554d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:24:12.478 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:12.475+0000 7ff7c1d9b640 1 -- 172.21.15.143:0/655954751 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7ff7c415a690 msgr2=0x7ff7c4136530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:24:12.478 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:12.475+0000 7ff7c1d9b640 1 --2- 172.21.15.143:0/655954751 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7ff7c415a690 0x7ff7c4136530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:24:12.478 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:12.475+0000 7ff7c1d9b640 1 -- 172.21.15.143:0/655954751 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7ff7ac0047d0 msgr2=0x7ff7c41554d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:24:12.478 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:12.475+0000 7ff7c1d9b640 1 --2- 172.21.15.143:0/655954751 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7ff7ac0047d0 0x7ff7c41554d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:24:12.479 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:12.475+0000 7ff7c1d9b640 1 -- 172.21.15.143:0/655954751 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff7c415d6c0 con 0x7ff7c4149f00 2023-11-26T07:24:12.479 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:12.475+0000 7ff7c259c640 1 --2- 172.21.15.143:0/655954751 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7ff7ac0047d0 0x7ff7c41554d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-26T07:24:12.479 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:12.475+0000 7ff7c1d9b640 1 --2- 172.21.15.143:0/655954751 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7ff7c4149f00 0x7ff7c415a150 secure :-1 s=READY pgs=1305 cs=0 l=1 rev1=1 crypto rx=0x7ff7b801c210 tx=0x7ff7ac005800 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:24:12.479 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:12.475+0000 7ff7b2ffd640 1 -- 172.21.15.143:0/655954751 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff7b8003e40 con 0x7ff7c4149f00 2023-11-26T07:24:12.480 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:12.479+0000 7ff7b2ffd640 1 -- 172.21.15.143:0/655954751 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff7b8007a40 con 0x7ff7c4149f00 2023-11-26T07:24:12.480 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:12.479+0000 7ff7c8965640 1 -- 172.21.15.143:0/655954751 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff7c415d9a0 con 0x7ff7c4149f00 2023-11-26T07:24:12.480 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:12.479+0000 7ff7b2ffd640 1 -- 172.21.15.143:0/655954751 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff7b8007270 con 0x7ff7c4149f00 2023-11-26T07:24:12.480 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:12.479+0000 7ff7c8965640 1 -- 172.21.15.143:0/655954751 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff7c415df60 con 0x7ff7c4149f00 2023-11-26T07:24:12.480 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:12.479+0000 7ff7c8965640 1 -- 172.21.15.143:0/655954751 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff7c415aaa0 con 0x7ff7c4149f00 2023-11-26T07:24:12.480 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:12.479+0000 7ff7b2ffd640 1 -- 172.21.15.143:0/655954751 <== mon.0 v2:172.21.15.143:3300/0 4 ==== mgrmap(e 16) v1 ==== 85954+0+0 (secure 0 0 0) 0x7ff7b80073d0 con 0x7ff7c4149f00 2023-11-26T07:24:12.480 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:12.479+0000 7ff7b2ffd640 1 --2- 172.21.15.143:0/655954751 >> [v2:172.21.15.188:6836/3401128553,v1:172.21.15.188:6837/3401128553] conn(0x7ff7a40665c0 0x7ff7a4068a80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:24:12.481 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:12.479+0000 7ff7b2ffd640 1 -- 172.21.15.143:0/655954751 <== mon.0 v2:172.21.15.143:3300/0 5 ==== osd_map(103..103 src has 1..103) v4 ==== 5709+0+0 (secure 0 0 0) 0x7ff7b811e4f0 con 0x7ff7c4149f00 2023-11-26T07:24:12.481 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:12.479+0000 7ff7c259c640 1 --2- 172.21.15.143:0/655954751 >> [v2:172.21.15.188:6836/3401128553,v1:172.21.15.188:6837/3401128553] conn(0x7ff7a40665c0 0x7ff7a4068a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:24:12.481 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:12.483+0000 7ff7c259c640 1 --2- 172.21.15.143:0/655954751 >> [v2:172.21.15.188:6836/3401128553,v1:172.21.15.188:6837/3401128553] conn(0x7ff7a40665c0 0x7ff7a4068a80 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7ff7c4138120 tx=0x7ff7b4008040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:24:12.484 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:12.483+0000 7ff7b2ffd640 1 -- 172.21.15.143:0/655954751 <== mon.0 v2:172.21.15.143:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172649 (secure 0 0 0) 0x7ff7b80eb740 con 0x7ff7c4149f00 2023-11-26T07:24:12.636 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:12.635+0000 7ff7c8965640 1 -- 172.21.15.143:0/655954751 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7ff7c4067680 con 0x7ff7c4149f00 2023-11-26T07:24:12.637 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:12.635+0000 7ff7b2ffd640 1 -- 172.21.15.143:0/655954751 <== mon.0 v2:172.21.15.143:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v103) v1 ==== 74+0+16563 (secure 0 0 0) 0x7ff7b80eed60 con 0x7ff7c4149f00 2023-11-26T07:24:12.637 INFO:teuthology.orchestra.run.smithi143.stdout: 2023-11-26T07:24:12.637 INFO:teuthology.orchestra.run.smithi143.stdout:{"epoch":103,"fsid":"f5033060-2469-4a07-95fe-2b76f5404bc1","created":"2023-11-26T07:17:28.613663+0000","modified":"2023-11-26T07:24:09.510299+0000","last_up_change":"2023-11-26T07:17:41.157813+0000","last_in_change":"2023-11-26T07:17:31.181658+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":17,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-11-26T07:17:41.758597+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":16,"pool_name":"cephfs_metadata","create_time":"2023-11-26T07:24:06.278132+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"102","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":17,"pool_name":"cephfs_data","create_time":"2023-11-26T07:24:07.294850+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"103","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"ebbee2c6-023e-44b9-9f1d-6f7bfd309730","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":101,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6817","nonce":639807369},{"type":"v1","addr":"172.21.15.143:6819","nonce":639807369}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6821","nonce":639807369},{"type":"v1","addr":"172.21.15.143:6823","nonce":639807369}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6830","nonce":639807369},{"type":"v1","addr":"172.21.15.143:6831","nonce":639807369}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6825","nonce":639807369},{"type":"v1","addr":"172.21.15.143:6828","nonce":639807369}]},"public_addr":"172.21.15.143:6819/639807369","cluster_addr":"172.21.15.143:6823/639807369","heartbeat_back_addr":"172.21.15.143:6831/639807369","heartbeat_front_addr":"172.21.15.143:6828/639807369","state":["exists","up"]},{"osd":1,"uuid":"fec4b8be-7122-49e9-9a19-5850a0700001","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":101,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6816","nonce":1144942530},{"type":"v1","addr":"172.21.15.143:6818","nonce":1144942530}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6820","nonce":1144942530},{"type":"v1","addr":"172.21.15.143:6822","nonce":1144942530}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6827","nonce":1144942530},{"type":"v1","addr":"172.21.15.143:6829","nonce":1144942530}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6824","nonce":1144942530},{"type":"v1","addr":"172.21.15.143:6826","nonce":1144942530}]},"public_addr":"172.21.15.143:6818/1144942530","cluster_addr":"172.21.15.143:6822/1144942530","heartbeat_back_addr":"172.21.15.143:6829/1144942530","heartbeat_front_addr":"172.21.15.143:6826/1144942530","state":["exists","up"]},{"osd":2,"uuid":"2888af5b-f933-46ca-8f95-6dd494ff43f6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":101,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6808","nonce":3909095186},{"type":"v1","addr":"172.21.15.143:6809","nonce":3909095186}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6810","nonce":3909095186},{"type":"v1","addr":"172.21.15.143:6811","nonce":3909095186}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6814","nonce":3909095186},{"type":"v1","addr":"172.21.15.143:6815","nonce":3909095186}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6812","nonce":3909095186},{"type":"v1","addr":"172.21.15.143:6813","nonce":3909095186}]},"public_addr":"172.21.15.143:6809/3909095186","cluster_addr":"172.21.15.143:6811/3909095186","heartbeat_back_addr":"172.21.15.143:6815/3909095186","heartbeat_front_addr":"172.21.15.143:6813/3909095186","state":["exists","up"]},{"osd":3,"uuid":"d1f52435-7bc1-4081-9550-9cd1ba55f549","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":101,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6800","nonce":3854210991},{"type":"v1","addr":"172.21.15.143:6801","nonce":3854210991}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6802","nonce":3854210991},{"type":"v1","addr":"172.21.15.143:6803","nonce":3854210991}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6806","nonce":3854210991},{"type":"v1","addr":"172.21.15.143:6807","nonce":3854210991}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6804","nonce":3854210991},{"type":"v1","addr":"172.21.15.143:6805","nonce":3854210991}]},"public_addr":"172.21.15.143:6801/3854210991","cluster_addr":"172.21.15.143:6803/3854210991","heartbeat_back_addr":"172.21.15.143:6807/3854210991","heartbeat_front_addr":"172.21.15.143:6805/3854210991","state":["exists","up"]},{"osd":4,"uuid":"e2ae8b6e-6e25-49b2-bcbf-aacce68f06a3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":101,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6800","nonce":3924606902},{"type":"v1","addr":"172.21.15.188:6801","nonce":3924606902}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6802","nonce":3924606902},{"type":"v1","addr":"172.21.15.188:6803","nonce":3924606902}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6806","nonce":3924606902},{"type":"v1","addr":"172.21.15.188:6807","nonce":3924606902}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6804","nonce":3924606902},{"type":"v1","addr":"172.21.15.188:6805","nonce":3924606902}]},"public_addr":"172.21.15.188:6801/3924606902","cluster_addr":"172.21.15.188:6803/3924606902","heartbeat_back_addr":"172.21.15.188:6807/3924606902","heartbeat_front_addr":"172.21.15.188:6805/3924606902","state":["exists","up"]},{"osd":5,"uuid":"3cec8a32-7698-4c36-8e89-e4ba11934f33","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":101,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6813","nonce":186841544},{"type":"v1","addr":"172.21.15.188:6815","nonce":186841544}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6817","nonce":186841544},{"type":"v1","addr":"172.21.15.188:6818","nonce":186841544}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6822","nonce":186841544},{"type":"v1","addr":"172.21.15.188:6823","nonce":186841544}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6820","nonce":186841544},{"type":"v1","addr":"172.21.15.188:6821","nonce":186841544}]},"public_addr":"172.21.15.188:6815/186841544","cluster_addr":"172.21.15.188:6818/186841544","heartbeat_back_addr":"172.21.15.188:6823/186841544","heartbeat_front_addr":"172.21.15.188:6821/186841544","state":["exists","up"]},{"osd":6,"uuid":"58a65d5d-e090-473a-81b4-ad38e85029f9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":101,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6808","nonce":2153499405},{"type":"v1","addr":"172.21.15.188:6809","nonce":2153499405}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6810","nonce":2153499405},{"type":"v1","addr":"172.21.15.188:6811","nonce":2153499405}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6816","nonce":2153499405},{"type":"v1","addr":"172.21.15.188:6819","nonce":2153499405}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6812","nonce":2153499405},{"type":"v1","addr":"172.21.15.188:6814","nonce":2153499405}]},"public_addr":"172.21.15.188:6809/2153499405","cluster_addr":"172.21.15.188:6811/2153499405","heartbeat_back_addr":"172.21.15.188:6819/2153499405","heartbeat_front_addr":"172.21.15.188:6814/2153499405","state":["exists","up"]},{"osd":7,"uuid":"ea5b7f70-3554-4cf8-ac30-f7c59b80678d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":101,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6824","nonce":2722096729},{"type":"v1","addr":"172.21.15.188:6825","nonce":2722096729}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6826","nonce":2722096729},{"type":"v1","addr":"172.21.15.188:6827","nonce":2722096729}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6830","nonce":2722096729},{"type":"v1","addr":"172.21.15.188:6831","nonce":2722096729}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6828","nonce":2722096729},{"type":"v1","addr":"172.21.15.188:6829","nonce":2722096729}]},"public_addr":"172.21.15.188:6825/2722096729","cluster_addr":"172.21.15.188:6827/2722096729","heartbeat_back_addr":"172.21.15.188:6831/2722096729","heartbeat_front_addr":"172.21.15.188:6829/2722096729","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-26T07:17:36.995032+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-26T07:17:36.932266+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-26T07:17:36.829304+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-26T07:17:36.804297+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-26T07:17:36.667892+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-26T07:17:36.964868+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-26T07:17:36.827332+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-26T07:17:36.893179+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-11-26T07:24:12.639 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:12.639+0000 7ff7c8965640 1 -- 172.21.15.143:0/655954751 >> [v2:172.21.15.188:6836/3401128553,v1:172.21.15.188:6837/3401128553] conn(0x7ff7a40665c0 msgr2=0x7ff7a4068a80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:24:12.639 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:12.639+0000 7ff7c8965640 1 --2- 172.21.15.143:0/655954751 >> [v2:172.21.15.188:6836/3401128553,v1:172.21.15.188:6837/3401128553] conn(0x7ff7a40665c0 0x7ff7a4068a80 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7ff7c4138120 tx=0x7ff7b4008040 comp rx=0 tx=0).stop 2023-11-26T07:24:12.639 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:12.639+0000 7ff7c8965640 1 -- 172.21.15.143:0/655954751 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7ff7c4149f00 msgr2=0x7ff7c415a150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:24:12.640 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:12.639+0000 7ff7c8965640 1 --2- 172.21.15.143:0/655954751 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7ff7c4149f00 0x7ff7c415a150 secure :-1 s=READY pgs=1305 cs=0 l=1 rev1=1 crypto rx=0x7ff7b801c210 tx=0x7ff7ac005800 comp rx=0 tx=0).stop 2023-11-26T07:24:12.640 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:12.639+0000 7ff7c8965640 1 -- 172.21.15.143:0/655954751 shutdown_connections 2023-11-26T07:24:12.640 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:12.639+0000 7ff7c8965640 1 --2- 172.21.15.143:0/655954751 >> [v2:172.21.15.188:6836/3401128553,v1:172.21.15.188:6837/3401128553] conn(0x7ff7a40665c0 0x7ff7a4068a80 unknown :-1 s=CLOSED pgs=128 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:24:12.640 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:12.639+0000 7ff7c8965640 1 --2- 172.21.15.143:0/655954751 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7ff7c415a690 0x7ff7c4136530 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:24:12.640 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:12.639+0000 7ff7c8965640 1 --2- 172.21.15.143:0/655954751 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7ff7c4149f00 0x7ff7c415a150 unknown :-1 s=CLOSED pgs=1305 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:24:12.640 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:12.639+0000 7ff7c8965640 1 --2- 172.21.15.143:0/655954751 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7ff7ac0047d0 0x7ff7c41554d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:24:12.640 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:12.639+0000 7ff7c8965640 1 -- 172.21.15.143:0/655954751 >> 172.21.15.143:0/655954751 conn(0x7ff7c406a890 msgr2=0x7ff7c415eea0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:24:12.640 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:12.639+0000 7ff7c8965640 1 -- 172.21.15.143:0/655954751 shutdown_connections 2023-11-26T07:24:12.640 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:12.639+0000 7ff7c8965640 1 -- 172.21.15.143:0/655954751 wait complete. 2023-11-26T07:24:12.658 INFO:tasks.cephfs.filesystem.ceph_manager:waiting for clean 2023-11-26T07:24:12.658 DEBUG:teuthology.orchestra.run.smithi143:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-11-26T07:24:12.796 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:12.795+0000 7f82c8b72640 1 Processor -- start 2023-11-26T07:24:12.796 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:12.795+0000 7f82c8b72640 1 -- start start 2023-11-26T07:24:12.797 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:12.795+0000 7f82c8b72640 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7f82c40cf990 0x7f82c415efa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:24:12.797 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:12.795+0000 7f82c8b72640 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7f82c415f4e0 0x7f82c40cd7b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:24:12.797 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:12.795+0000 7f82c8b72640 1 -- --> v1:172.21.15.143:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f82c406c520 con 0x7f82c413e290 2023-11-26T07:24:12.797 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:12.795+0000 7f82c8b72640 1 -- --> v2:172.21.15.143:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f82c406c800 con 0x7f82c415f4e0 2023-11-26T07:24:12.798 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:12.795+0000 7f82c8b72640 1 -- --> v2:172.21.15.188:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f82c406cae0 con 0x7f82c40cf990 2023-11-26T07:24:12.798 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:12.795+0000 7f82c259c640 1 --1- >> v1:172.21.15.143:6789/0 conn(0x7f82c413e290 0x7f82c40cf280 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.143:6789/0 says I am v1:172.21.15.143:42442/0 (socket says 172.21.15.143:42442) 2023-11-26T07:24:12.798 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:12.795+0000 7f82c259c640 1 -- 172.21.15.143:0/712623018 learned_addr learned my addr 172.21.15.143:0/712623018 (peer_addr_for_me v1:172.21.15.143:0/0) 2023-11-26T07:24:12.798 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:12.795+0000 7f82c2d9d640 1 --2- 172.21.15.143:0/712623018 >> v2:172.21.15.143:3300/0 conn(0x7f82c415f4e0 0x7f82c40cd7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:24:12.799 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:12.799+0000 7f82c1d9b640 1 --2- 172.21.15.143:0/712623018 >> v2:172.21.15.188:3300/0 conn(0x7f82c40cf990 0x7f82c415efa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:24:12.799 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:12.799+0000 7f82c159a640 1 -- 172.21.15.143:0/712623018 <== mon.0 v1:172.21.15.143:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3469978384 0 0) 0x7f82c406c520 con 0x7f82c413e290 2023-11-26T07:24:12.799 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:12.799+0000 7f82c159a640 1 -- 172.21.15.143:0/712623018 --> v1:172.21.15.143:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f82ac003530 con 0x7f82c413e290 2023-11-26T07:24:12.799 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:12.799+0000 7f82c2d9d640 1 -- 172.21.15.143:0/712623018 >> v2:172.21.15.188:3300/0 conn(0x7f82c40cf990 msgr2=0x7f82c415efa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-26T07:24:12.799 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:12.799+0000 7f82c2d9d640 1 --2- 172.21.15.143:0/712623018 >> v2:172.21.15.188:3300/0 conn(0x7f82c40cf990 0x7f82c415efa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:24:12.799 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:12.799+0000 7f82c2d9d640 1 -- 172.21.15.143:0/712623018 >> v1:172.21.15.143:6789/0 conn(0x7f82c413e290 legacy=0x7f82c40cf280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:24:12.799 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:12.799+0000 7f82c2d9d640 1 -- 172.21.15.143:0/712623018 --> v2:172.21.15.143:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f82c406cdc0 con 0x7f82c415f4e0 2023-11-26T07:24:12.799 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:12.799+0000 7f82c1d9b640 1 --2- 172.21.15.143:0/712623018 >> v2:172.21.15.188:3300/0 conn(0x7f82c40cf990 0x7f82c415efa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-26T07:24:12.801 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:12.799+0000 7f82c2d9d640 1 --2- 172.21.15.143:0/712623018 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f82c415f4e0 0x7f82c40cd7b0 secure :-1 s=READY pgs=1307 cs=0 l=1 rev1=1 crypto rx=0x7f82c40cfe40 tx=0x7f82bc01c8e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=981dbaa32f1c88b server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:24:12.801 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:12.799+0000 7f82c159a640 1 -- 172.21.15.143:0/712623018 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f82c415fb60 con 0x7f82c415f4e0 2023-11-26T07:24:12.801 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:12.799+0000 7f82c159a640 1 -- 172.21.15.143:0/712623018 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f82bc0a59c0 con 0x7f82c415f4e0 2023-11-26T07:24:12.801 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:12.799+0000 7f82c159a640 1 -- 172.21.15.143:0/712623018 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f82bc0a5d50 con 0x7f82c415f4e0 2023-11-26T07:24:12.802 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:12.803+0000 7f82c8b72640 1 -- 172.21.15.143:0/712623018 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f82c415f4e0 msgr2=0x7f82c40cd7b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:24:12.802 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:12.803+0000 7f82c8b72640 1 --2- 172.21.15.143:0/712623018 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f82c415f4e0 0x7f82c40cd7b0 secure :-1 s=READY pgs=1307 cs=0 l=1 rev1=1 crypto rx=0x7f82c40cfe40 tx=0x7f82bc01c8e0 comp rx=0 tx=0).stop 2023-11-26T07:24:12.802 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:12.803+0000 7f82c8b72640 1 -- 172.21.15.143:0/712623018 shutdown_connections 2023-11-26T07:24:12.802 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:12.803+0000 7f82c8b72640 1 --2- 172.21.15.143:0/712623018 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f82c415f4e0 0x7f82c40cd7b0 unknown :-1 s=CLOSED pgs=1307 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:24:12.802 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:12.803+0000 7f82c8b72640 1 --2- 172.21.15.143:0/712623018 >> v2:172.21.15.188:3300/0 conn(0x7f82c40cf990 0x7f82c415efa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:24:12.803 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:12.803+0000 7f82c8b72640 1 -- 172.21.15.143:0/712623018 >> 172.21.15.143:0/712623018 conn(0x7f82c406a890 msgr2=0x7f82c406aca0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:24:12.803 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:12.803+0000 7f82c8b72640 1 -- 172.21.15.143:0/712623018 shutdown_connections 2023-11-26T07:24:12.803 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:12.803+0000 7f82c8b72640 1 -- 172.21.15.143:0/712623018 wait complete. 2023-11-26T07:24:12.804 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:12.803+0000 7f82c8b72640 1 Processor -- start 2023-11-26T07:24:12.804 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:12.803+0000 7f82c8b72640 1 -- start start 2023-11-26T07:24:12.806 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:12.803+0000 7f82c8b72640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f82c40cf990 0x7f82c4134150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:24:12.806 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:12.803+0000 7f82c8b72640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f82c413e290 0x7f82c4159f90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:24:12.806 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:12.803+0000 7f82c8b72640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f82c415f4e0 0x7f82c415a4d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:24:12.806 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:12.803+0000 7f82c8b72640 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f82c4134690 con 0x7f82c413e290 2023-11-26T07:24:12.806 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:12.803+0000 7f82c8b72640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f82c41517f0 con 0x7f82c415f4e0 2023-11-26T07:24:12.806 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:12.803+0000 7f82c8b72640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f82c4154cc0 con 0x7f82c40cf990 2023-11-26T07:24:12.806 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:12.803+0000 7f82c1d9b640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f82c413e290 0x7f82c4159f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:24:12.806 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:12.803+0000 7f82c1d9b640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f82c413e290 0x7f82c4159f90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:33814/0 (socket says 172.21.15.143:33814) 2023-11-26T07:24:12.806 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:12.803+0000 7f82c1d9b640 1 -- 172.21.15.143:0/1258114110 learned_addr learned my addr 172.21.15.143:0/1258114110 (peer_addr_for_me v2:172.21.15.143:0/0) 2023-11-26T07:24:12.806 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:12.803+0000 7f82c259c640 1 --2- 172.21.15.143:0/1258114110 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f82c40cf990 0x7f82c4134150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:24:12.807 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:12.803+0000 7f82c2d9d640 1 --2- 172.21.15.143:0/1258114110 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f82c415f4e0 0x7f82c415a4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:24:12.807 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:12.807+0000 7f82c1d9b640 1 -- 172.21.15.143:0/1258114110 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f82c40cf990 msgr2=0x7f82c4134150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:24:12.807 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:12.807+0000 7f82c1d9b640 1 --2- 172.21.15.143:0/1258114110 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f82c40cf990 0x7f82c4134150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:24:12.807 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:12.807+0000 7f82c1d9b640 1 -- 172.21.15.143:0/1258114110 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f82c415f4e0 msgr2=0x7f82c415a4d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:24:12.807 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:12.807+0000 7f82c1d9b640 1 --2- 172.21.15.143:0/1258114110 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f82c415f4e0 0x7f82c415a4d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:24:12.807 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:12.807+0000 7f82c1d9b640 1 -- 172.21.15.143:0/1258114110 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f82bc009a70 con 0x7f82c413e290 2023-11-26T07:24:12.807 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:12.807+0000 7f82c1d9b640 1 --2- 172.21.15.143:0/1258114110 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f82c413e290 0x7f82c4159f90 secure :-1 s=READY pgs=1308 cs=0 l=1 rev1=1 crypto rx=0x7f82b800e810 tx=0x7f82b800ecd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:24:12.808 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:12.807+0000 7f82b2ffd640 1 -- 172.21.15.143:0/1258114110 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f82b800c8d0 con 0x7f82c413e290 2023-11-26T07:24:12.808 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:12.807+0000 7f82b2ffd640 1 -- 172.21.15.143:0/1258114110 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f82b800ca30 con 0x7f82c413e290 2023-11-26T07:24:12.808 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:12.807+0000 7f82c8b72640 1 -- 172.21.15.143:0/1258114110 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f82c41539c0 con 0x7f82c413e290 2023-11-26T07:24:12.808 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:12.807+0000 7f82b2ffd640 1 -- 172.21.15.143:0/1258114110 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f82b800cdc0 con 0x7f82c413e290 2023-11-26T07:24:12.808 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:12.807+0000 7f82c8b72640 1 -- 172.21.15.143:0/1258114110 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f82c4152ee0 con 0x7f82c413e290 2023-11-26T07:24:12.808 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:12.807+0000 7f82c8b72640 1 -- 172.21.15.143:0/1258114110 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f82c4067680 con 0x7f82c413e290 2023-11-26T07:24:12.809 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:12.807+0000 7f82b2ffd640 1 -- 172.21.15.143:0/1258114110 <== mon.0 v2:172.21.15.143:3300/0 4 ==== mgrmap(e 16) v1 ==== 85954+0+0 (secure 0 0 0) 0x7f82b8013410 con 0x7f82c413e290 2023-11-26T07:24:12.809 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:12.807+0000 7f82b2ffd640 1 --2- 172.21.15.143:0/1258114110 >> [v2:172.21.15.188:6836/3401128553,v1:172.21.15.188:6837/3401128553] conn(0x7f82a40665c0 0x7f82a4068a80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:24:12.809 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:12.807+0000 7f82b2ffd640 1 -- 172.21.15.143:0/1258114110 <== mon.0 v2:172.21.15.143:3300/0 5 ==== osd_map(103..103 src has 1..103) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f82b8083eb0 con 0x7f82c413e290 2023-11-26T07:24:12.810 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:12.811+0000 7f82c259c640 1 --2- 172.21.15.143:0/1258114110 >> [v2:172.21.15.188:6836/3401128553,v1:172.21.15.188:6837/3401128553] conn(0x7f82a40665c0 0x7f82a4068a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:24:12.810 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:12.811+0000 7f82c259c640 1 --2- 172.21.15.143:0/1258114110 >> [v2:172.21.15.188:6836/3401128553,v1:172.21.15.188:6837/3401128553] conn(0x7f82a40665c0 0x7f82a4068a80 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7f82b40011f0 tx=0x7f82b4007d40 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:24:12.813 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:12.811+0000 7f82b2ffd640 1 -- 172.21.15.143:0/1258114110 <== mon.0 v2:172.21.15.143:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172649 (secure 0 0 0) 0x7f82b80511b0 con 0x7f82c413e290 2023-11-26T07:24:12.961 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:12.959+0000 7f82c8b72640 1 -- 172.21.15.143:0/1258114110 --> [v2:172.21.15.188:6836/3401128553,v1:172.21.15.188:6837/3401128553] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f82c4165de0 con 0x7f82a40665c0 2023-11-26T07:24:12.970 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:12.971+0000 7f82b2ffd640 1 -- 172.21.15.143:0/1258114110 <== mgr.4108 v2:172.21.15.188:6836/3401128553 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+277482 (secure 0 0 0) 0x7f82c4165de0 con 0x7f82a40665c0 2023-11-26T07:24:12.973 INFO:teuthology.orchestra.run.smithi143.stdout: 2023-11-26T07:24:12.974 INFO:teuthology.orchestra.run.smithi143.stderr:dumped all 2023-11-26T07:24:12.975 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:12.975+0000 7f82c8b72640 1 -- 172.21.15.143:0/1258114110 >> [v2:172.21.15.188:6836/3401128553,v1:172.21.15.188:6837/3401128553] conn(0x7f82a40665c0 msgr2=0x7f82a4068a80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:24:12.975 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:12.975+0000 7f82c8b72640 1 --2- 172.21.15.143:0/1258114110 >> [v2:172.21.15.188:6836/3401128553,v1:172.21.15.188:6837/3401128553] conn(0x7f82a40665c0 0x7f82a4068a80 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7f82b40011f0 tx=0x7f82b4007d40 comp rx=0 tx=0).stop 2023-11-26T07:24:12.975 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:12.975+0000 7f82c8b72640 1 -- 172.21.15.143:0/1258114110 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f82c413e290 msgr2=0x7f82c4159f90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:24:12.976 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:12.975+0000 7f82c8b72640 1 --2- 172.21.15.143:0/1258114110 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f82c413e290 0x7f82c4159f90 secure :-1 s=READY pgs=1308 cs=0 l=1 rev1=1 crypto rx=0x7f82b800e810 tx=0x7f82b800ecd0 comp rx=0 tx=0).stop 2023-11-26T07:24:12.976 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:12.975+0000 7f82c8b72640 1 -- 172.21.15.143:0/1258114110 shutdown_connections 2023-11-26T07:24:12.976 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:12.975+0000 7f82c8b72640 1 --2- 172.21.15.143:0/1258114110 >> [v2:172.21.15.188:6836/3401128553,v1:172.21.15.188:6837/3401128553] conn(0x7f82a40665c0 0x7f82a4068a80 unknown :-1 s=CLOSED pgs=129 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:24:12.976 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:12.975+0000 7f82c8b72640 1 --2- 172.21.15.143:0/1258114110 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f82c415f4e0 0x7f82c415a4d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:24:12.976 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:12.975+0000 7f82c8b72640 1 --2- 172.21.15.143:0/1258114110 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f82c413e290 0x7f82c4159f90 unknown :-1 s=CLOSED pgs=1308 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:24:12.976 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:12.975+0000 7f82c8b72640 1 --2- 172.21.15.143:0/1258114110 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f82c40cf990 0x7f82c4134150 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:24:12.976 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:12.975+0000 7f82c8b72640 1 -- 172.21.15.143:0/1258114110 >> 172.21.15.143:0/1258114110 conn(0x7f82c406a890 msgr2=0x7f82c40cf250 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:24:12.976 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:12.975+0000 7f82c8b72640 1 -- 172.21.15.143:0/1258114110 shutdown_connections 2023-11-26T07:24:12.976 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:12.975+0000 7f82c8b72640 1 -- 172.21.15.143:0/1258114110 wait complete. 2023-11-26T07:24:12.993 INFO:teuthology.orchestra.run.smithi143.stdout:{"pg_ready":true,"pg_map":{"version":65,"stamp":"2023-11-26T07:24:11.298329+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":212,"num_read_kb":306,"num_write":179,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":128,"ondisk_log_size":128,"up":66,"acting":66,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":5,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":260320,"kb_used_data":10828,"kb_used_omap":12,"kb_used_meta":249331,"kb_avail":754714400,"statfs":{"total":773094113280,"available":772827545600,"internally_reserved":0,"allocated":11087872,"data_stored":5495475,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12749,"internal_metadata":255315507},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":14,"apply_latency_ms":14,"commit_latency_ns":14000000,"apply_latency_ns":14000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"4.391736"},"pg_stats":[{"pgid":"17.3f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-26T07:24:07.907834+0000","last_change":"2023-11-26T07:24:07.907834+0000","last_active":"2023-11-26T07:24:07.907834+0000","last_peered":"2023-11-26T07:24:07.907834+0000","last_clean":"2023-11-26T07:24:07.907834+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-26T07:24:07.907834+0000","last_undegraded":"2023-11-26T07:24:07.907834+0000","last_fullsized":"2023-11-26T07:24:07.907834+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:24:07.907834+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:24:07.907834+0000","last_clean_scrub_stamp":"2023-11-26T07:24:07.907834+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"17.3e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-26T07:24:07.907834+0000","last_change":"2023-11-26T07:24:07.907834+0000","last_active":"2023-11-26T07:24:07.907834+0000","last_peered":"2023-11-26T07:24:07.907834+0000","last_clean":"2023-11-26T07:24:07.907834+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-26T07:24:07.907834+0000","last_undegraded":"2023-11-26T07:24:07.907834+0000","last_fullsized":"2023-11-26T07:24:07.907834+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:24:07.907834+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:24:07.907834+0000","last_clean_scrub_stamp":"2023-11-26T07:24:07.907834+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"17.3d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-26T07:24:07.907834+0000","last_change":"2023-11-26T07:24:07.907834+0000","last_active":"2023-11-26T07:24:07.907834+0000","last_peered":"2023-11-26T07:24:07.907834+0000","last_clean":"2023-11-26T07:24:07.907834+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-26T07:24:07.907834+0000","last_undegraded":"2023-11-26T07:24:07.907834+0000","last_fullsized":"2023-11-26T07:24:07.907834+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:24:07.907834+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:24:07.907834+0000","last_clean_scrub_stamp":"2023-11-26T07:24:07.907834+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"17.3c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-26T07:24:07.907834+0000","last_change":"2023-11-26T07:24:07.907834+0000","last_active":"2023-11-26T07:24:07.907834+0000","last_peered":"2023-11-26T07:24:07.907834+0000","last_clean":"2023-11-26T07:24:07.907834+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-26T07:24:07.907834+0000","last_undegraded":"2023-11-26T07:24:07.907834+0000","last_fullsized":"2023-11-26T07:24:07.907834+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:24:07.907834+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:24:07.907834+0000","last_clean_scrub_stamp":"2023-11-26T07:24:07.907834+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"17.3b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-26T07:24:07.907834+0000","last_change":"2023-11-26T07:24:07.907834+0000","last_active":"2023-11-26T07:24:07.907834+0000","last_peered":"2023-11-26T07:24:07.907834+0000","last_clean":"2023-11-26T07:24:07.907834+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-26T07:24:07.907834+0000","last_undegraded":"2023-11-26T07:24:07.907834+0000","last_fullsized":"2023-11-26T07:24:07.907834+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:24:07.907834+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:24:07.907834+0000","last_clean_scrub_stamp":"2023-11-26T07:24:07.907834+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"17.3a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-26T07:24:07.907834+0000","last_change":"2023-11-26T07:24:07.907834+0000","last_active":"2023-11-26T07:24:07.907834+0000","last_peered":"2023-11-26T07:24:07.907834+0000","last_clean":"2023-11-26T07:24:07.907834+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-26T07:24:07.907834+0000","last_undegraded":"2023-11-26T07:24:07.907834+0000","last_fullsized":"2023-11-26T07:24:07.907834+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:24:07.907834+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:24:07.907834+0000","last_clean_scrub_stamp":"2023-11-26T07:24:07.907834+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"17.39","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-26T07:24:07.907834+0000","last_change":"2023-11-26T07:24:07.907834+0000","last_active":"2023-11-26T07:24:07.907834+0000","last_peered":"2023-11-26T07:24:07.907834+0000","last_clean":"2023-11-26T07:24:07.907834+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-26T07:24:07.907834+0000","last_undegraded":"2023-11-26T07:24:07.907834+0000","last_fullsized":"2023-11-26T07:24:07.907834+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:24:07.907834+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:24:07.907834+0000","last_clean_scrub_stamp":"2023-11-26T07:24:07.907834+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"17.38","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-26T07:24:07.907834+0000","last_change":"2023-11-26T07:24:07.907834+0000","last_active":"2023-11-26T07:24:07.907834+0000","last_peered":"2023-11-26T07:24:07.907834+0000","last_clean":"2023-11-26T07:24:07.907834+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-26T07:24:07.907834+0000","last_undegraded":"2023-11-26T07:24:07.907834+0000","last_fullsized":"2023-11-26T07:24:07.907834+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:24:07.907834+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:24:07.907834+0000","last_clean_scrub_stamp":"2023-11-26T07:24:07.907834+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"17.37","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-26T07:24:07.907834+0000","last_change":"2023-11-26T07:24:07.907834+0000","last_active":"2023-11-26T07:24:07.907834+0000","last_peered":"2023-11-26T07:24:07.907834+0000","last_clean":"2023-11-26T07:24:07.907834+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-26T07:24:07.907834+0000","last_undegraded":"2023-11-26T07:24:07.907834+0000","last_fullsized":"2023-11-26T07:24:07.907834+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:24:07.907834+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:24:07.907834+0000","last_clean_scrub_stamp":"2023-11-26T07:24:07.907834+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"17.36","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-26T07:24:07.907834+0000","last_change":"2023-11-26T07:24:07.907834+0000","last_active":"2023-11-26T07:24:07.907834+0000","last_peered":"2023-11-26T07:24:07.907834+0000","last_clean":"2023-11-26T07:24:07.907834+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-26T07:24:07.907834+0000","last_undegraded":"2023-11-26T07:24:07.907834+0000","last_fullsized":"2023-11-26T07:24:07.907834+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:24:07.907834+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:24:07.907834+0000","last_clean_scrub_stamp":"2023-11-26T07:24:07.907834+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"17.35","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-26T07:24:07.907834+0000","last_change":"2023-11-26T07:24:07.907834+0000","last_active":"2023-11-26T07:24:07.907834+0000","last_peered":"2023-11-26T07:24:07.907834+0000","last_clean":"2023-11-26T07:24:07.907834+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-26T07:24:07.907834+0000","last_undegraded":"2023-11-26T07:24:07.907834+0000","last_fullsized":"2023-11-26T07:24:07.907834+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:24:07.907834+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:24:07.907834+0000","last_clean_scrub_stamp":"2023-11-26T07:24:07.907834+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"17.34","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-26T07:24:07.907834+0000","last_change":"2023-11-26T07:24:07.907834+0000","last_active":"2023-11-26T07:24:07.907834+0000","last_peered":"2023-11-26T07:24:07.907834+0000","last_clean":"2023-11-26T07:24:07.907834+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-26T07:24:07.907834+0000","last_undegraded":"2023-11-26T07:24:07.907834+0000","last_fullsized":"2023-11-26T07:24:07.907834+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:24:07.907834+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:24:07.907834+0000","last_clean_scrub_stamp":"2023-11-26T07:24:07.907834+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"17.33","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-26T07:24:07.907834+0000","last_change":"2023-11-26T07:24:07.907834+0000","last_active":"2023-11-26T07:24:07.907834+0000","last_peered":"2023-11-26T07:24:07.907834+0000","last_clean":"2023-11-26T07:24:07.907834+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-26T07:24:07.907834+0000","last_undegraded":"2023-11-26T07:24:07.907834+0000","last_fullsized":"2023-11-26T07:24:07.907834+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:24:07.907834+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:24:07.907834+0000","last_clean_scrub_stamp":"2023-11-26T07:24:07.907834+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"17.32","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-26T07:24:07.907834+0000","last_change":"2023-11-26T07:24:07.907834+0000","last_active":"2023-11-26T07:24:07.907834+0000","last_peered":"2023-11-26T07:24:07.907834+0000","last_clean":"2023-11-26T07:24:07.907834+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-26T07:24:07.907834+0000","last_undegraded":"2023-11-26T07:24:07.907834+0000","last_fullsized":"2023-11-26T07:24:07.907834+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:24:07.907834+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:24:07.907834+0000","last_clean_scrub_stamp":"2023-11-26T07:24:07.907834+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"17.31","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-26T07:24:07.907834+0000","last_change":"2023-11-26T07:24:07.907834+0000","last_active":"2023-11-26T07:24:07.907834+0000","last_peered":"2023-11-26T07:24:07.907834+0000","last_clean":"2023-11-26T07:24:07.907834+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-26T07:24:07.907834+0000","last_undegraded":"2023-11-26T07:24:07.907834+0000","last_fullsized":"2023-11-26T07:24:07.907834+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:24:07.907834+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:24:07.907834+0000","last_clean_scrub_stamp":"2023-11-26T07:24:07.907834+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"17.30","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-26T07:24:07.907834+0000","last_change":"2023-11-26T07:24:07.907834+0000","last_active":"2023-11-26T07:24:07.907834+0000","last_peered":"2023-11-26T07:24:07.907834+0000","last_clean":"2023-11-26T07:24:07.907834+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-26T07:24:07.907834+0000","last_undegraded":"2023-11-26T07:24:07.907834+0000","last_fullsized":"2023-11-26T07:24:07.907834+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:24:07.907834+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:24:07.907834+0000","last_clean_scrub_stamp":"2023-11-26T07:24:07.907834+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"17.2f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-26T07:24:07.907834+0000","last_change":"2023-11-26T07:24:07.907834+0000","last_active":"2023-11-26T07:24:07.907834+0000","last_peered":"2023-11-26T07:24:07.907834+0000","last_clean":"2023-11-26T07:24:07.907834+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-26T07:24:07.907834+0000","last_undegraded":"2023-11-26T07:24:07.907834+0000","last_fullsized":"2023-11-26T07:24:07.907834+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:24:07.907834+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:24:07.907834+0000","last_clean_scrub_stamp":"2023-11-26T07:24:07.907834+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"17.2e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-26T07:24:07.907834+0000","last_change":"2023-11-26T07:24:07.907834+0000","last_active":"2023-11-26T07:24:07.907834+0000","last_peered":"2023-11-26T07:24:07.907834+0000","last_clean":"2023-11-26T07:24:07.907834+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-26T07:24:07.907834+0000","last_undegraded":"2023-11-26T07:24:07.907834+0000","last_fullsized":"2023-11-26T07:24:07.907834+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:24:07.907834+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:24:07.907834+0000","last_clean_scrub_stamp":"2023-11-26T07:24:07.907834+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"17.2d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-26T07:24:07.907834+0000","last_change":"2023-11-26T07:24:07.907834+0000","last_active":"2023-11-26T07:24:07.907834+0000","last_peered":"2023-11-26T07:24:07.907834+0000","last_clean":"2023-11-26T07:24:07.907834+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-26T07:24:07.907834+0000","last_undegraded":"2023-11-26T07:24:07.907834+0000","last_fullsized":"2023-11-26T07:24:07.907834+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:24:07.907834+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:24:07.907834+0000","last_clean_scrub_stamp":"2023-11-26T07:24:07.907834+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"17.2c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-26T07:24:07.907834+0000","last_change":"2023-11-26T07:24:07.907834+0000","last_active":"2023-11-26T07:24:07.907834+0000","last_peered":"2023-11-26T07:24:07.907834+0000","last_clean":"2023-11-26T07:24:07.907834+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-26T07:24:07.907834+0000","last_undegraded":"2023-11-26T07:24:07.907834+0000","last_fullsized":"2023-11-26T07:24:07.907834+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:24:07.907834+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:24:07.907834+0000","last_clean_scrub_stamp":"2023-11-26T07:24:07.907834+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"17.2b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-26T07:24:07.907834+0000","last_change":"2023-11-26T07:24:07.907834+0000","last_active":"2023-11-26T07:24:07.907834+0000","last_peered":"2023-11-26T07:24:07.907834+0000","last_clean":"2023-11-26T07:24:07.907834+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-26T07:24:07.907834+0000","last_undegraded":"2023-11-26T07:24:07.907834+0000","last_fullsized":"2023-11-26T07:24:07.907834+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:24:07.907834+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:24:07.907834+0000","last_clean_scrub_stamp":"2023-11-26T07:24:07.907834+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"17.2a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-26T07:24:07.907834+0000","last_change":"2023-11-26T07:24:07.907834+0000","last_active":"2023-11-26T07:24:07.907834+0000","last_peered":"2023-11-26T07:24:07.907834+0000","last_clean":"2023-11-26T07:24:07.907834+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-26T07:24:07.907834+0000","last_undegraded":"2023-11-26T07:24:07.907834+0000","last_fullsized":"2023-11-26T07:24:07.907834+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:24:07.907834+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:24:07.907834+0000","last_clean_scrub_stamp":"2023-11-26T07:24:07.907834+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"17.29","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-26T07:24:07.907834+0000","last_change":"2023-11-26T07:24:07.907834+0000","last_active":"2023-11-26T07:24:07.907834+0000","last_peered":"2023-11-26T07:24:07.907834+0000","last_clean":"2023-11-26T07:24:07.907834+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-26T07:24:07.907834+0000","last_undegraded":"2023-11-26T07:24:07.907834+0000","last_fullsized":"2023-11-26T07:24:07.907834+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:24:07.907834+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:24:07.907834+0000","last_clean_scrub_stamp":"2023-11-26T07:24:07.907834+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"17.28","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-26T07:24:07.907834+0000","last_change":"2023-11-26T07:24:07.907834+0000","last_active":"2023-11-26T07:24:07.907834+0000","last_peered":"2023-11-26T07:24:07.907834+0000","last_clean":"2023-11-26T07:24:07.907834+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-26T07:24:07.907834+0000","last_undegraded":"2023-11-26T07:24:07.907834+0000","last_fullsized":"2023-11-26T07:24:07.907834+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:24:07.907834+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:24:07.907834+0000","last_clean_scrub_stamp":"2023-11-26T07:24:07.907834+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"17.27","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-26T07:24:07.907834+0000","last_change":"2023-11-26T07:24:07.907834+0000","last_active":"2023-11-26T07:24:07.907834+0000","last_peered":"2023-11-26T07:24:07.907834+0000","last_clean":"2023-11-26T07:24:07.907834+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-26T07:24:07.907834+0000","last_undegraded":"2023-11-26T07:24:07.907834+0000","last_fullsized":"2023-11-26T07:24:07.907834+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:24:07.907834+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:24:07.907834+0000","last_clean_scrub_stamp":"2023-11-26T07:24:07.907834+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"17.26","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-26T07:24:07.907834+0000","last_change":"2023-11-26T07:24:07.907834+0000","last_active":"2023-11-26T07:24:07.907834+0000","last_peered":"2023-11-26T07:24:07.907834+0000","last_clean":"2023-11-26T07:24:07.907834+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-26T07:24:07.907834+0000","last_undegraded":"2023-11-26T07:24:07.907834+0000","last_fullsized":"2023-11-26T07:24:07.907834+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:24:07.907834+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:24:07.907834+0000","last_clean_scrub_stamp":"2023-11-26T07:24:07.907834+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"17.25","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-26T07:24:07.907834+0000","last_change":"2023-11-26T07:24:07.907834+0000","last_active":"2023-11-26T07:24:07.907834+0000","last_peered":"2023-11-26T07:24:07.907834+0000","last_clean":"2023-11-26T07:24:07.907834+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-26T07:24:07.907834+0000","last_undegraded":"2023-11-26T07:24:07.907834+0000","last_fullsized":"2023-11-26T07:24:07.907834+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:24:07.907834+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:24:07.907834+0000","last_clean_scrub_stamp":"2023-11-26T07:24:07.907834+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"17.24","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-26T07:24:07.907834+0000","last_change":"2023-11-26T07:24:07.907834+0000","last_active":"2023-11-26T07:24:07.907834+0000","last_peered":"2023-11-26T07:24:07.907834+0000","last_clean":"2023-11-26T07:24:07.907834+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-26T07:24:07.907834+0000","last_undegraded":"2023-11-26T07:24:07.907834+0000","last_fullsized":"2023-11-26T07:24:07.907834+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:24:07.907834+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:24:07.907834+0000","last_clean_scrub_stamp":"2023-11-26T07:24:07.907834+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"17.23","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-26T07:24:07.907834+0000","last_change":"2023-11-26T07:24:07.907834+0000","last_active":"2023-11-26T07:24:07.907834+0000","last_peered":"2023-11-26T07:24:07.907834+0000","last_clean":"2023-11-26T07:24:07.907834+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-26T07:24:07.907834+0000","last_undegraded":"2023-11-26T07:24:07.907834+0000","last_fullsized":"2023-11-26T07:24:07.907834+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:24:07.907834+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:24:07.907834+0000","last_clean_scrub_stamp":"2023-11-26T07:24:07.907834+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"17.22","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-26T07:24:07.907834+0000","last_change":"2023-11-26T07:24:07.907834+0000","last_active":"2023-11-26T07:24:07.907834+0000","last_peered":"2023-11-26T07:24:07.907834+0000","last_clean":"2023-11-26T07:24:07.907834+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-26T07:24:07.907834+0000","last_undegraded":"2023-11-26T07:24:07.907834+0000","last_fullsized":"2023-11-26T07:24:07.907834+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:24:07.907834+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:24:07.907834+0000","last_clean_scrub_stamp":"2023-11-26T07:24:07.907834+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"17.21","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-26T07:24:07.907834+0000","last_change":"2023-11-26T07:24:07.907834+0000","last_active":"2023-11-26T07:24:07.907834+0000","last_peered":"2023-11-26T07:24:07.907834+0000","last_clean":"2023-11-26T07:24:07.907834+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-26T07:24:07.907834+0000","last_undegraded":"2023-11-26T07:24:07.907834+0000","last_fullsized":"2023-11-26T07:24:07.907834+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:24:07.907834+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:24:07.907834+0000","last_clean_scrub_stamp":"2023-11-26T07:24:07.907834+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"17.20","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-26T07:24:07.907834+0000","last_change":"2023-11-26T07:24:07.907834+0000","last_active":"2023-11-26T07:24:07.907834+0000","last_peered":"2023-11-26T07:24:07.907834+0000","last_clean":"2023-11-26T07:24:07.907834+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-26T07:24:07.907834+0000","last_undegraded":"2023-11-26T07:24:07.907834+0000","last_fullsized":"2023-11-26T07:24:07.907834+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:24:07.907834+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:24:07.907834+0000","last_clean_scrub_stamp":"2023-11-26T07:24:07.907834+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"17.1e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-26T07:24:07.907834+0000","last_change":"2023-11-26T07:24:07.907834+0000","last_active":"2023-11-26T07:24:07.907834+0000","last_peered":"2023-11-26T07:24:07.907834+0000","last_clean":"2023-11-26T07:24:07.907834+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-26T07:24:07.907834+0000","last_undegraded":"2023-11-26T07:24:07.907834+0000","last_fullsized":"2023-11-26T07:24:07.907834+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:24:07.907834+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:24:07.907834+0000","last_clean_scrub_stamp":"2023-11-26T07:24:07.907834+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"16.1f","version":"0'0","reported_seq":1,"reported_epoch":100,"state":"creating+peering","last_fresh":"2023-11-26T07:24:06.915266+0000","last_change":"2023-11-26T07:24:06.915266+0000","last_active":"2023-11-26T07:24:06.899969+0000","last_peered":"2023-11-26T07:24:06.899969+0000","last_clean":"2023-11-26T07:24:06.899969+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-26T07:24:06.915266+0000","last_undegraded":"2023-11-26T07:24:06.915266+0000","last_fullsized":"2023-11-26T07:24:06.915266+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:24:06.899969+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:24:06.899969+0000","last_clean_scrub_stamp":"2023-11-26T07:24:06.899969+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"17.1f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-26T07:24:07.907834+0000","last_change":"2023-11-26T07:24:07.907834+0000","last_active":"2023-11-26T07:24:07.907834+0000","last_peered":"2023-11-26T07:24:07.907834+0000","last_clean":"2023-11-26T07:24:07.907834+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-26T07:24:07.907834+0000","last_undegraded":"2023-11-26T07:24:07.907834+0000","last_fullsized":"2023-11-26T07:24:07.907834+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:24:07.907834+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:24:07.907834+0000","last_clean_scrub_stamp":"2023-11-26T07:24:07.907834+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"16.1e","version":"0'0","reported_seq":1,"reported_epoch":100,"state":"creating+peering","last_fresh":"2023-11-26T07:24:06.917840+0000","last_change":"2023-11-26T07:24:06.917840+0000","last_active":"2023-11-26T07:24:06.899969+0000","last_peered":"2023-11-26T07:24:06.899969+0000","last_clean":"2023-11-26T07:24:06.899969+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-26T07:24:06.917840+0000","last_undegraded":"2023-11-26T07:24:06.917840+0000","last_fullsized":"2023-11-26T07:24:06.917840+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:24:06.899969+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:24:06.899969+0000","last_clean_scrub_stamp":"2023-11-26T07:24:06.899969+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"17.1c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-26T07:24:07.907834+0000","last_change":"2023-11-26T07:24:07.907834+0000","last_active":"2023-11-26T07:24:07.907834+0000","last_peered":"2023-11-26T07:24:07.907834+0000","last_clean":"2023-11-26T07:24:07.907834+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-26T07:24:07.907834+0000","last_undegraded":"2023-11-26T07:24:07.907834+0000","last_fullsized":"2023-11-26T07:24:07.907834+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:24:07.907834+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:24:07.907834+0000","last_clean_scrub_stamp":"2023-11-26T07:24:07.907834+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"16.1d","version":"0'0","reported_seq":1,"reported_epoch":100,"state":"creating+peering","last_fresh":"2023-11-26T07:24:06.912163+0000","last_change":"2023-11-26T07:24:06.912163+0000","last_active":"2023-11-26T07:24:06.899969+0000","last_peered":"2023-11-26T07:24:06.899969+0000","last_clean":"2023-11-26T07:24:06.899969+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-26T07:24:06.912163+0000","last_undegraded":"2023-11-26T07:24:06.912163+0000","last_fullsized":"2023-11-26T07:24:06.912163+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:24:06.899969+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:24:06.899969+0000","last_clean_scrub_stamp":"2023-11-26T07:24:06.899969+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"17.1d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-26T07:24:07.907834+0000","last_change":"2023-11-26T07:24:07.907834+0000","last_active":"2023-11-26T07:24:07.907834+0000","last_peered":"2023-11-26T07:24:07.907834+0000","last_clean":"2023-11-26T07:24:07.907834+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-26T07:24:07.907834+0000","last_undegraded":"2023-11-26T07:24:07.907834+0000","last_fullsized":"2023-11-26T07:24:07.907834+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:24:07.907834+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:24:07.907834+0000","last_clean_scrub_stamp":"2023-11-26T07:24:07.907834+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"16.1c","version":"0'0","reported_seq":1,"reported_epoch":100,"state":"creating+peering","last_fresh":"2023-11-26T07:24:06.912728+0000","last_change":"2023-11-26T07:24:06.912728+0000","last_active":"2023-11-26T07:24:06.899969+0000","last_peered":"2023-11-26T07:24:06.899969+0000","last_clean":"2023-11-26T07:24:06.899969+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-26T07:24:06.912728+0000","last_undegraded":"2023-11-26T07:24:06.912728+0000","last_fullsized":"2023-11-26T07:24:06.912728+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:24:06.899969+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:24:06.899969+0000","last_clean_scrub_stamp":"2023-11-26T07:24:06.899969+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"17.1a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-26T07:24:07.907834+0000","last_change":"2023-11-26T07:24:07.907834+0000","last_active":"2023-11-26T07:24:07.907834+0000","last_peered":"2023-11-26T07:24:07.907834+0000","last_clean":"2023-11-26T07:24:07.907834+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-26T07:24:07.907834+0000","last_undegraded":"2023-11-26T07:24:07.907834+0000","last_fullsized":"2023-11-26T07:24:07.907834+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:24:07.907834+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:24:07.907834+0000","last_clean_scrub_stamp":"2023-11-26T07:24:07.907834+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"16.1b","version":"0'0","reported_seq":1,"reported_epoch":100,"state":"creating+peering","last_fresh":"2023-11-26T07:24:06.916183+0000","last_change":"2023-11-26T07:24:06.916183+0000","last_active":"2023-11-26T07:24:06.899969+0000","last_peered":"2023-11-26T07:24:06.899969+0000","last_clean":"2023-11-26T07:24:06.899969+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-26T07:24:06.916183+0000","last_undegraded":"2023-11-26T07:24:06.916183+0000","last_fullsized":"2023-11-26T07:24:06.916183+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:24:06.899969+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:24:06.899969+0000","last_clean_scrub_stamp":"2023-11-26T07:24:06.899969+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"17.1b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-26T07:24:07.907834+0000","last_change":"2023-11-26T07:24:07.907834+0000","last_active":"2023-11-26T07:24:07.907834+0000","last_peered":"2023-11-26T07:24:07.907834+0000","last_clean":"2023-11-26T07:24:07.907834+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-26T07:24:07.907834+0000","last_undegraded":"2023-11-26T07:24:07.907834+0000","last_fullsized":"2023-11-26T07:24:07.907834+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:24:07.907834+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:24:07.907834+0000","last_clean_scrub_stamp":"2023-11-26T07:24:07.907834+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"16.1a","version":"0'0","reported_seq":1,"reported_epoch":100,"state":"creating+peering","last_fresh":"2023-11-26T07:24:06.916021+0000","last_change":"2023-11-26T07:24:06.916021+0000","last_active":"2023-11-26T07:24:06.899969+0000","last_peered":"2023-11-26T07:24:06.899969+0000","last_clean":"2023-11-26T07:24:06.899969+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-26T07:24:06.916021+0000","last_undegraded":"2023-11-26T07:24:06.916021+0000","last_fullsized":"2023-11-26T07:24:06.916021+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:24:06.899969+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:24:06.899969+0000","last_clean_scrub_stamp":"2023-11-26T07:24:06.899969+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"17.18","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-26T07:24:07.907834+0000","last_change":"2023-11-26T07:24:07.907834+0000","last_active":"2023-11-26T07:24:07.907834+0000","last_peered":"2023-11-26T07:24:07.907834+0000","last_clean":"2023-11-26T07:24:07.907834+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-26T07:24:07.907834+0000","last_undegraded":"2023-11-26T07:24:07.907834+0000","last_fullsized":"2023-11-26T07:24:07.907834+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:24:07.907834+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:24:07.907834+0000","last_clean_scrub_stamp":"2023-11-26T07:24:07.907834+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"16.19","version":"0'0","reported_seq":1,"reported_epoch":100,"state":"creating+peering","last_fresh":"2023-11-26T07:24:06.909153+0000","last_change":"2023-11-26T07:24:06.909153+0000","last_active":"2023-11-26T07:24:06.899969+0000","last_peered":"2023-11-26T07:24:06.899969+0000","last_clean":"2023-11-26T07:24:06.899969+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-26T07:24:06.909153+0000","last_undegraded":"2023-11-26T07:24:06.909153+0000","last_fullsized":"2023-11-26T07:24:06.909153+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:24:06.899969+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:24:06.899969+0000","last_clean_scrub_stamp":"2023-11-26T07:24:06.899969+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"17.19","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-26T07:24:07.907834+0000","last_change":"2023-11-26T07:24:07.907834+0000","last_active":"2023-11-26T07:24:07.907834+0000","last_peered":"2023-11-26T07:24:07.907834+0000","last_clean":"2023-11-26T07:24:07.907834+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-26T07:24:07.907834+0000","last_undegraded":"2023-11-26T07:24:07.907834+0000","last_fullsized":"2023-11-26T07:24:07.907834+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:24:07.907834+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:24:07.907834+0000","last_clean_scrub_stamp":"2023-11-26T07:24:07.907834+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"16.18","version":"0'0","reported_seq":1,"reported_epoch":100,"state":"creating+peering","last_fresh":"2023-11-26T07:24:06.915893+0000","last_change":"2023-11-26T07:24:06.915893+0000","last_active":"2023-11-26T07:24:06.899969+0000","last_peered":"2023-11-26T07:24:06.899969+0000","last_clean":"2023-11-26T07:24:06.899969+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-26T07:24:06.915893+0000","last_undegraded":"2023-11-26T07:24:06.915893+0000","last_fullsized":"2023-11-26T07:24:06.915893+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:24:06.899969+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:24:06.899969+0000","last_clean_scrub_stamp":"2023-11-26T07:24:06.899969+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.16","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-26T07:24:07.907834+0000","last_change":"2023-11-26T07:24:07.907834+0000","last_active":"2023-11-26T07:24:07.907834+0000","last_peered":"2023-11-26T07:24:07.907834+0000","last_clean":"2023-11-26T07:24:07.907834+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-26T07:24:07.907834+0000","last_undegraded":"2023-11-26T07:24:07.907834+0000","last_fullsized":"2023-11-26T07:24:07.907834+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:24:07.907834+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:24:07.907834+0000","last_clean_scrub_stamp":"2023-11-26T07:24:07.907834+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"16.17","version":"0'0","reported_seq":1,"reported_epoch":100,"state":"creating+peering","last_fresh":"2023-11-26T07:24:06.912138+0000","last_change":"2023-11-26T07:24:06.912138+0000","last_active":"2023-11-26T07:24:06.899969+0000","last_peered":"2023-11-26T07:24:06.899969+0000","last_clean":"2023-11-26T07:24:06.899969+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-26T07:24:06.912138+0000","last_undegraded":"2023-11-26T07:24:06.912138+0000","last_fullsized":"2023-11-26T07:24:06.912138+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:24:06.899969+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:24:06.899969+0000","last_clean_scrub_stamp":"2023-11-26T07:24:06.899969+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"17.17","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-26T07:24:07.907834+0000","last_change":"2023-11-26T07:24:07.907834+0000","last_active":"2023-11-26T07:24:07.907834+0000","last_peered":"2023-11-26T07:24:07.907834+0000","last_clean":"2023-11-26T07:24:07.907834+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-26T07:24:07.907834+0000","last_undegraded":"2023-11-26T07:24:07.907834+0000","last_fullsized":"2023-11-26T07:24:07.907834+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:24:07.907834+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:24:07.907834+0000","last_clean_scrub_stamp":"2023-11-26T07:24:07.907834+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"16.16","version":"0'0","reported_seq":1,"reported_epoch":100,"state":"creating+peering","last_fresh":"2023-11-26T07:24:06.917443+0000","last_change":"2023-11-26T07:24:06.917443+0000","last_active":"2023-11-26T07:24:06.899969+0000","last_peered":"2023-11-26T07:24:06.899969+0000","last_clean":"2023-11-26T07:24:06.899969+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-26T07:24:06.917443+0000","last_undegraded":"2023-11-26T07:24:06.917443+0000","last_fullsized":"2023-11-26T07:24:06.917443+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:24:06.899969+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:24:06.899969+0000","last_clean_scrub_stamp":"2023-11-26T07:24:06.899969+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"17.14","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-26T07:24:07.907834+0000","last_change":"2023-11-26T07:24:07.907834+0000","last_active":"2023-11-26T07:24:07.907834+0000","last_peered":"2023-11-26T07:24:07.907834+0000","last_clean":"2023-11-26T07:24:07.907834+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-26T07:24:07.907834+0000","last_undegraded":"2023-11-26T07:24:07.907834+0000","last_fullsized":"2023-11-26T07:24:07.907834+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:24:07.907834+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:24:07.907834+0000","last_clean_scrub_stamp":"2023-11-26T07:24:07.907834+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"16.15","version":"0'0","reported_seq":1,"reported_epoch":100,"state":"creating+peering","last_fresh":"2023-11-26T07:24:06.915180+0000","last_change":"2023-11-26T07:24:06.915180+0000","last_active":"2023-11-26T07:24:06.899969+0000","last_peered":"2023-11-26T07:24:06.899969+0000","last_clean":"2023-11-26T07:24:06.899969+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-26T07:24:06.915180+0000","last_undegraded":"2023-11-26T07:24:06.915180+0000","last_fullsized":"2023-11-26T07:24:06.915180+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:24:06.899969+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:24:06.899969+0000","last_clean_scrub_stamp":"2023-11-26T07:24:06.899969+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.15","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-26T07:24:07.907834+0000","last_change":"2023-11-26T07:24:07.907834+0000","last_active":"2023-11-26T07:24:07.907834+0000","last_peered":"2023-11-26T07:24:07.907834+0000","last_clean":"2023-11-26T07:24:07.907834+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-26T07:24:07.907834+0000","last_undegraded":"2023-11-26T07:24:07.907834+0000","last_fullsized":"2023-11-26T07:24:07.907834+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:24:07.907834+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:24:07.907834+0000","last_clean_scrub_stamp":"2023-11-26T07:24:07.907834+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"16.14","version":"0'0","reported_seq":1,"reported_epoch":100,"state":"creating+peering","last_fresh":"2023-11-26T07:24:06.914728+0000","last_change":"2023-11-26T07:24:06.914728+0000","last_active":"2023-11-26T07:24:06.899969+0000","last_peered":"2023-11-26T07:24:06.899969+0000","last_clean":"2023-11-26T07:24:06.899969+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-26T07:24:06.914728+0000","last_undegraded":"2023-11-26T07:24:06.914728+0000","last_fullsized":"2023-11-26T07:24:06.914728+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:24:06.899969+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:24:06.899969+0000","last_clean_scrub_stamp":"2023-11-26T07:24:06.899969+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.12","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-26T07:24:07.907834+0000","last_change":"2023-11-26T07:24:07.907834+0000","last_active":"2023-11-26T07:24:07.907834+0000","last_peered":"2023-11-26T07:24:07.907834+0000","last_clean":"2023-11-26T07:24:07.907834+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-26T07:24:07.907834+0000","last_undegraded":"2023-11-26T07:24:07.907834+0000","last_fullsized":"2023-11-26T07:24:07.907834+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:24:07.907834+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:24:07.907834+0000","last_clean_scrub_stamp":"2023-11-26T07:24:07.907834+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"16.13","version":"0'0","reported_seq":1,"reported_epoch":100,"state":"creating+peering","last_fresh":"2023-11-26T07:24:06.916010+0000","last_change":"2023-11-26T07:24:06.916010+0000","last_active":"2023-11-26T07:24:06.899969+0000","last_peered":"2023-11-26T07:24:06.899969+0000","last_clean":"2023-11-26T07:24:06.899969+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-26T07:24:06.916010+0000","last_undegraded":"2023-11-26T07:24:06.916010+0000","last_fullsized":"2023-11-26T07:24:06.916010+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:24:06.899969+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:24:06.899969+0000","last_clean_scrub_stamp":"2023-11-26T07:24:06.899969+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"17.13","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-26T07:24:07.907834+0000","last_change":"2023-11-26T07:24:07.907834+0000","last_active":"2023-11-26T07:24:07.907834+0000","last_peered":"2023-11-26T07:24:07.907834+0000","last_clean":"2023-11-26T07:24:07.907834+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-26T07:24:07.907834+0000","last_undegraded":"2023-11-26T07:24:07.907834+0000","last_fullsized":"2023-11-26T07:24:07.907834+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:24:07.907834+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:24:07.907834+0000","last_clean_scrub_stamp":"2023-11-26T07:24:07.907834+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"16.12","version":"0'0","reported_seq":1,"reported_epoch":100,"state":"creating+peering","last_fresh":"2023-11-26T07:24:06.914513+0000","last_change":"2023-11-26T07:24:06.914513+0000","last_active":"2023-11-26T07:24:06.899969+0000","last_peered":"2023-11-26T07:24:06.899969+0000","last_clean":"2023-11-26T07:24:06.899969+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-26T07:24:06.914513+0000","last_undegraded":"2023-11-26T07:24:06.914513+0000","last_fullsized":"2023-11-26T07:24:06.914513+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:24:06.899969+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:24:06.899969+0000","last_clean_scrub_stamp":"2023-11-26T07:24:06.899969+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"17.11","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-26T07:24:07.907834+0000","last_change":"2023-11-26T07:24:07.907834+0000","last_active":"2023-11-26T07:24:07.907834+0000","last_peered":"2023-11-26T07:24:07.907834+0000","last_clean":"2023-11-26T07:24:07.907834+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-26T07:24:07.907834+0000","last_undegraded":"2023-11-26T07:24:07.907834+0000","last_fullsized":"2023-11-26T07:24:07.907834+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:24:07.907834+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:24:07.907834+0000","last_clean_scrub_stamp":"2023-11-26T07:24:07.907834+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"16.10","version":"0'0","reported_seq":1,"reported_epoch":100,"state":"creating+peering","last_fresh":"2023-11-26T07:24:06.912524+0000","last_change":"2023-11-26T07:24:06.912524+0000","last_active":"2023-11-26T07:24:06.899969+0000","last_peered":"2023-11-26T07:24:06.899969+0000","last_clean":"2023-11-26T07:24:06.899969+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-26T07:24:06.912524+0000","last_undegraded":"2023-11-26T07:24:06.912524+0000","last_fullsized":"2023-11-26T07:24:06.912524+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:24:06.899969+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:24:06.899969+0000","last_clean_scrub_stamp":"2023-11-26T07:24:06.899969+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"17.e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-26T07:24:07.907834+0000","last_change":"2023-11-26T07:24:07.907834+0000","last_active":"2023-11-26T07:24:07.907834+0000","last_peered":"2023-11-26T07:24:07.907834+0000","last_clean":"2023-11-26T07:24:07.907834+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-26T07:24:07.907834+0000","last_undegraded":"2023-11-26T07:24:07.907834+0000","last_fullsized":"2023-11-26T07:24:07.907834+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:24:07.907834+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:24:07.907834+0000","last_clean_scrub_stamp":"2023-11-26T07:24:07.907834+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"16.f","version":"0'0","reported_seq":1,"reported_epoch":100,"state":"creating+peering","last_fresh":"2023-11-26T07:24:06.914722+0000","last_change":"2023-11-26T07:24:06.914722+0000","last_active":"2023-11-26T07:24:06.899969+0000","last_peered":"2023-11-26T07:24:06.899969+0000","last_clean":"2023-11-26T07:24:06.899969+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-26T07:24:06.914722+0000","last_undegraded":"2023-11-26T07:24:06.914722+0000","last_fullsized":"2023-11-26T07:24:06.914722+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:24:06.899969+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:24:06.899969+0000","last_clean_scrub_stamp":"2023-11-26T07:24:06.899969+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-26T07:24:07.907834+0000","last_change":"2023-11-26T07:24:07.907834+0000","last_active":"2023-11-26T07:24:07.907834+0000","last_peered":"2023-11-26T07:24:07.907834+0000","last_clean":"2023-11-26T07:24:07.907834+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-26T07:24:07.907834+0000","last_undegraded":"2023-11-26T07:24:07.907834+0000","last_fullsized":"2023-11-26T07:24:07.907834+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:24:07.907834+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:24:07.907834+0000","last_clean_scrub_stamp":"2023-11-26T07:24:07.907834+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"16.e","version":"0'0","reported_seq":1,"reported_epoch":100,"state":"creating+peering","last_fresh":"2023-11-26T07:24:06.912146+0000","last_change":"2023-11-26T07:24:06.912146+0000","last_active":"2023-11-26T07:24:06.899969+0000","last_peered":"2023-11-26T07:24:06.899969+0000","last_clean":"2023-11-26T07:24:06.899969+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-26T07:24:06.912146+0000","last_undegraded":"2023-11-26T07:24:06.912146+0000","last_fullsized":"2023-11-26T07:24:06.912146+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:24:06.899969+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:24:06.899969+0000","last_clean_scrub_stamp":"2023-11-26T07:24:06.899969+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"17.c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-26T07:24:07.907834+0000","last_change":"2023-11-26T07:24:07.907834+0000","last_active":"2023-11-26T07:24:07.907834+0000","last_peered":"2023-11-26T07:24:07.907834+0000","last_clean":"2023-11-26T07:24:07.907834+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-26T07:24:07.907834+0000","last_undegraded":"2023-11-26T07:24:07.907834+0000","last_fullsized":"2023-11-26T07:24:07.907834+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:24:07.907834+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:24:07.907834+0000","last_clean_scrub_stamp":"2023-11-26T07:24:07.907834+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"16.d","version":"0'0","reported_seq":1,"reported_epoch":100,"state":"creating+peering","last_fresh":"2023-11-26T07:24:06.914509+0000","last_change":"2023-11-26T07:24:06.914509+0000","last_active":"2023-11-26T07:24:06.899969+0000","last_peered":"2023-11-26T07:24:06.899969+0000","last_clean":"2023-11-26T07:24:06.899969+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-26T07:24:06.914509+0000","last_undegraded":"2023-11-26T07:24:06.914509+0000","last_fullsized":"2023-11-26T07:24:06.914509+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:24:06.899969+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:24:06.899969+0000","last_clean_scrub_stamp":"2023-11-26T07:24:06.899969+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"17.d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-26T07:24:07.907834+0000","last_change":"2023-11-26T07:24:07.907834+0000","last_active":"2023-11-26T07:24:07.907834+0000","last_peered":"2023-11-26T07:24:07.907834+0000","last_clean":"2023-11-26T07:24:07.907834+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-26T07:24:07.907834+0000","last_undegraded":"2023-11-26T07:24:07.907834+0000","last_fullsized":"2023-11-26T07:24:07.907834+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:24:07.907834+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:24:07.907834+0000","last_clean_scrub_stamp":"2023-11-26T07:24:07.907834+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"16.c","version":"0'0","reported_seq":1,"reported_epoch":100,"state":"creating+peering","last_fresh":"2023-11-26T07:24:06.915129+0000","last_change":"2023-11-26T07:24:06.915129+0000","last_active":"2023-11-26T07:24:06.899969+0000","last_peered":"2023-11-26T07:24:06.899969+0000","last_clean":"2023-11-26T07:24:06.899969+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-26T07:24:06.915129+0000","last_undegraded":"2023-11-26T07:24:06.915129+0000","last_fullsized":"2023-11-26T07:24:06.915129+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:24:06.899969+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:24:06.899969+0000","last_clean_scrub_stamp":"2023-11-26T07:24:06.899969+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"17.a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-26T07:24:07.907834+0000","last_change":"2023-11-26T07:24:07.907834+0000","last_active":"2023-11-26T07:24:07.907834+0000","last_peered":"2023-11-26T07:24:07.907834+0000","last_clean":"2023-11-26T07:24:07.907834+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-26T07:24:07.907834+0000","last_undegraded":"2023-11-26T07:24:07.907834+0000","last_fullsized":"2023-11-26T07:24:07.907834+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:24:07.907834+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:24:07.907834+0000","last_clean_scrub_stamp":"2023-11-26T07:24:07.907834+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"16.b","version":"0'0","reported_seq":1,"reported_epoch":100,"state":"creating+peering","last_fresh":"2023-11-26T07:24:06.915543+0000","last_change":"2023-11-26T07:24:06.915543+0000","last_active":"2023-11-26T07:24:06.899969+0000","last_peered":"2023-11-26T07:24:06.899969+0000","last_clean":"2023-11-26T07:24:06.899969+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-26T07:24:06.915543+0000","last_undegraded":"2023-11-26T07:24:06.915543+0000","last_fullsized":"2023-11-26T07:24:06.915543+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:24:06.899969+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:24:06.899969+0000","last_clean_scrub_stamp":"2023-11-26T07:24:06.899969+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"17.b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-26T07:24:07.907834+0000","last_change":"2023-11-26T07:24:07.907834+0000","last_active":"2023-11-26T07:24:07.907834+0000","last_peered":"2023-11-26T07:24:07.907834+0000","last_clean":"2023-11-26T07:24:07.907834+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-26T07:24:07.907834+0000","last_undegraded":"2023-11-26T07:24:07.907834+0000","last_fullsized":"2023-11-26T07:24:07.907834+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:24:07.907834+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:24:07.907834+0000","last_clean_scrub_stamp":"2023-11-26T07:24:07.907834+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"16.a","version":"0'0","reported_seq":1,"reported_epoch":100,"state":"creating+peering","last_fresh":"2023-11-26T07:24:06.912245+0000","last_change":"2023-11-26T07:24:06.912245+0000","last_active":"2023-11-26T07:24:06.899969+0000","last_peered":"2023-11-26T07:24:06.899969+0000","last_clean":"2023-11-26T07:24:06.899969+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-26T07:24:06.912245+0000","last_undegraded":"2023-11-26T07:24:06.912245+0000","last_fullsized":"2023-11-26T07:24:06.912245+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:24:06.899969+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:24:06.899969+0000","last_clean_scrub_stamp":"2023-11-26T07:24:06.899969+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"17.8","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-26T07:24:07.907834+0000","last_change":"2023-11-26T07:24:07.907834+0000","last_active":"2023-11-26T07:24:07.907834+0000","last_peered":"2023-11-26T07:24:07.907834+0000","last_clean":"2023-11-26T07:24:07.907834+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-26T07:24:07.907834+0000","last_undegraded":"2023-11-26T07:24:07.907834+0000","last_fullsized":"2023-11-26T07:24:07.907834+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:24:07.907834+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:24:07.907834+0000","last_clean_scrub_stamp":"2023-11-26T07:24:07.907834+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"16.9","version":"0'0","reported_seq":1,"reported_epoch":100,"state":"creating+peering","last_fresh":"2023-11-26T07:24:06.916100+0000","last_change":"2023-11-26T07:24:06.916100+0000","last_active":"2023-11-26T07:24:06.899969+0000","last_peered":"2023-11-26T07:24:06.899969+0000","last_clean":"2023-11-26T07:24:06.899969+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-26T07:24:06.916100+0000","last_undegraded":"2023-11-26T07:24:06.916100+0000","last_fullsized":"2023-11-26T07:24:06.916100+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:24:06.899969+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:24:06.899969+0000","last_clean_scrub_stamp":"2023-11-26T07:24:06.899969+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"17.9","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-26T07:24:07.907834+0000","last_change":"2023-11-26T07:24:07.907834+0000","last_active":"2023-11-26T07:24:07.907834+0000","last_peered":"2023-11-26T07:24:07.907834+0000","last_clean":"2023-11-26T07:24:07.907834+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-26T07:24:07.907834+0000","last_undegraded":"2023-11-26T07:24:07.907834+0000","last_fullsized":"2023-11-26T07:24:07.907834+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:24:07.907834+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:24:07.907834+0000","last_clean_scrub_stamp":"2023-11-26T07:24:07.907834+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"16.8","version":"0'0","reported_seq":1,"reported_epoch":100,"state":"creating+peering","last_fresh":"2023-11-26T07:24:06.915280+0000","last_change":"2023-11-26T07:24:06.915280+0000","last_active":"2023-11-26T07:24:06.899969+0000","last_peered":"2023-11-26T07:24:06.899969+0000","last_clean":"2023-11-26T07:24:06.899969+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-26T07:24:06.915280+0000","last_undegraded":"2023-11-26T07:24:06.915280+0000","last_fullsized":"2023-11-26T07:24:06.915280+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:24:06.899969+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:24:06.899969+0000","last_clean_scrub_stamp":"2023-11-26T07:24:06.899969+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.6","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-26T07:24:07.907834+0000","last_change":"2023-11-26T07:24:07.907834+0000","last_active":"2023-11-26T07:24:07.907834+0000","last_peered":"2023-11-26T07:24:07.907834+0000","last_clean":"2023-11-26T07:24:07.907834+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-26T07:24:07.907834+0000","last_undegraded":"2023-11-26T07:24:07.907834+0000","last_fullsized":"2023-11-26T07:24:07.907834+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:24:07.907834+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:24:07.907834+0000","last_clean_scrub_stamp":"2023-11-26T07:24:07.907834+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"16.7","version":"0'0","reported_seq":1,"reported_epoch":100,"state":"creating+peering","last_fresh":"2023-11-26T07:24:06.916513+0000","last_change":"2023-11-26T07:24:06.916513+0000","last_active":"2023-11-26T07:24:06.899969+0000","last_peered":"2023-11-26T07:24:06.899969+0000","last_clean":"2023-11-26T07:24:06.899969+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-26T07:24:06.916513+0000","last_undegraded":"2023-11-26T07:24:06.916513+0000","last_fullsized":"2023-11-26T07:24:06.916513+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:24:06.899969+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:24:06.899969+0000","last_clean_scrub_stamp":"2023-11-26T07:24:06.899969+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"17.7","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-26T07:24:07.907834+0000","last_change":"2023-11-26T07:24:07.907834+0000","last_active":"2023-11-26T07:24:07.907834+0000","last_peered":"2023-11-26T07:24:07.907834+0000","last_clean":"2023-11-26T07:24:07.907834+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-26T07:24:07.907834+0000","last_undegraded":"2023-11-26T07:24:07.907834+0000","last_fullsized":"2023-11-26T07:24:07.907834+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:24:07.907834+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:24:07.907834+0000","last_clean_scrub_stamp":"2023-11-26T07:24:07.907834+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"16.6","version":"0'0","reported_seq":1,"reported_epoch":100,"state":"creating+peering","last_fresh":"2023-11-26T07:24:06.914505+0000","last_change":"2023-11-26T07:24:06.914505+0000","last_active":"2023-11-26T07:24:06.899969+0000","last_peered":"2023-11-26T07:24:06.899969+0000","last_clean":"2023-11-26T07:24:06.899969+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-26T07:24:06.914505+0000","last_undegraded":"2023-11-26T07:24:06.914505+0000","last_fullsized":"2023-11-26T07:24:06.914505+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:24:06.899969+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:24:06.899969+0000","last_clean_scrub_stamp":"2023-11-26T07:24:06.899969+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"17.4","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-26T07:24:07.907834+0000","last_change":"2023-11-26T07:24:07.907834+0000","last_active":"2023-11-26T07:24:07.907834+0000","last_peered":"2023-11-26T07:24:07.907834+0000","last_clean":"2023-11-26T07:24:07.907834+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-26T07:24:07.907834+0000","last_undegraded":"2023-11-26T07:24:07.907834+0000","last_fullsized":"2023-11-26T07:24:07.907834+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:24:07.907834+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:24:07.907834+0000","last_clean_scrub_stamp":"2023-11-26T07:24:07.907834+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"16.5","version":"0'0","reported_seq":1,"reported_epoch":100,"state":"creating+peering","last_fresh":"2023-11-26T07:24:06.914714+0000","last_change":"2023-11-26T07:24:06.914714+0000","last_active":"2023-11-26T07:24:06.899969+0000","last_peered":"2023-11-26T07:24:06.899969+0000","last_clean":"2023-11-26T07:24:06.899969+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-26T07:24:06.914714+0000","last_undegraded":"2023-11-26T07:24:06.914714+0000","last_fullsized":"2023-11-26T07:24:06.914714+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:24:06.899969+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:24:06.899969+0000","last_clean_scrub_stamp":"2023-11-26T07:24:06.899969+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.5","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-26T07:24:07.907834+0000","last_change":"2023-11-26T07:24:07.907834+0000","last_active":"2023-11-26T07:24:07.907834+0000","last_peered":"2023-11-26T07:24:07.907834+0000","last_clean":"2023-11-26T07:24:07.907834+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-26T07:24:07.907834+0000","last_undegraded":"2023-11-26T07:24:07.907834+0000","last_fullsized":"2023-11-26T07:24:07.907834+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:24:07.907834+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:24:07.907834+0000","last_clean_scrub_stamp":"2023-11-26T07:24:07.907834+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"16.4","version":"0'0","reported_seq":1,"reported_epoch":100,"state":"creating+peering","last_fresh":"2023-11-26T07:24:06.916954+0000","last_change":"2023-11-26T07:24:06.916954+0000","last_active":"2023-11-26T07:24:06.899969+0000","last_peered":"2023-11-26T07:24:06.899969+0000","last_clean":"2023-11-26T07:24:06.899969+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-26T07:24:06.916954+0000","last_undegraded":"2023-11-26T07:24:06.916954+0000","last_fullsized":"2023-11-26T07:24:06.916954+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:24:06.899969+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:24:06.899969+0000","last_clean_scrub_stamp":"2023-11-26T07:24:06.899969+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"17.2","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-26T07:24:07.907834+0000","last_change":"2023-11-26T07:24:07.907834+0000","last_active":"2023-11-26T07:24:07.907834+0000","last_peered":"2023-11-26T07:24:07.907834+0000","last_clean":"2023-11-26T07:24:07.907834+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-26T07:24:07.907834+0000","last_undegraded":"2023-11-26T07:24:07.907834+0000","last_fullsized":"2023-11-26T07:24:07.907834+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:24:07.907834+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:24:07.907834+0000","last_clean_scrub_stamp":"2023-11-26T07:24:07.907834+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"16.3","version":"0'0","reported_seq":1,"reported_epoch":100,"state":"creating+peering","last_fresh":"2023-11-26T07:24:06.914788+0000","last_change":"2023-11-26T07:24:06.914788+0000","last_active":"2023-11-26T07:24:06.899969+0000","last_peered":"2023-11-26T07:24:06.899969+0000","last_clean":"2023-11-26T07:24:06.899969+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-26T07:24:06.914788+0000","last_undegraded":"2023-11-26T07:24:06.914788+0000","last_fullsized":"2023-11-26T07:24:06.914788+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:24:06.899969+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:24:06.899969+0000","last_clean_scrub_stamp":"2023-11-26T07:24:06.899969+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"17.3","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-26T07:24:07.907834+0000","last_change":"2023-11-26T07:24:07.907834+0000","last_active":"2023-11-26T07:24:07.907834+0000","last_peered":"2023-11-26T07:24:07.907834+0000","last_clean":"2023-11-26T07:24:07.907834+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-26T07:24:07.907834+0000","last_undegraded":"2023-11-26T07:24:07.907834+0000","last_fullsized":"2023-11-26T07:24:07.907834+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:24:07.907834+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:24:07.907834+0000","last_clean_scrub_stamp":"2023-11-26T07:24:07.907834+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"16.2","version":"0'0","reported_seq":1,"reported_epoch":100,"state":"creating+peering","last_fresh":"2023-11-26T07:24:06.909153+0000","last_change":"2023-11-26T07:24:06.909153+0000","last_active":"2023-11-26T07:24:06.899969+0000","last_peered":"2023-11-26T07:24:06.899969+0000","last_clean":"2023-11-26T07:24:06.899969+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-26T07:24:06.909153+0000","last_undegraded":"2023-11-26T07:24:06.909153+0000","last_fullsized":"2023-11-26T07:24:06.909153+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:24:06.899969+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:24:06.899969+0000","last_clean_scrub_stamp":"2023-11-26T07:24:06.899969+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"17.0","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-26T07:24:07.907834+0000","last_change":"2023-11-26T07:24:07.907834+0000","last_active":"2023-11-26T07:24:07.907834+0000","last_peered":"2023-11-26T07:24:07.907834+0000","last_clean":"2023-11-26T07:24:07.907834+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-26T07:24:07.907834+0000","last_undegraded":"2023-11-26T07:24:07.907834+0000","last_fullsized":"2023-11-26T07:24:07.907834+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:24:07.907834+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:24:07.907834+0000","last_clean_scrub_stamp":"2023-11-26T07:24:07.907834+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"16.1","version":"0'0","reported_seq":1,"reported_epoch":100,"state":"creating+peering","last_fresh":"2023-11-26T07:24:06.912132+0000","last_change":"2023-11-26T07:24:06.912132+0000","last_active":"2023-11-26T07:24:06.899969+0000","last_peered":"2023-11-26T07:24:06.899969+0000","last_clean":"2023-11-26T07:24:06.899969+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-26T07:24:06.912132+0000","last_undegraded":"2023-11-26T07:24:06.912132+0000","last_fullsized":"2023-11-26T07:24:06.912132+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:24:06.899969+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:24:06.899969+0000","last_clean_scrub_stamp":"2023-11-26T07:24:06.899969+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"17.1","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-26T07:24:07.907834+0000","last_change":"2023-11-26T07:24:07.907834+0000","last_active":"2023-11-26T07:24:07.907834+0000","last_peered":"2023-11-26T07:24:07.907834+0000","last_clean":"2023-11-26T07:24:07.907834+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-26T07:24:07.907834+0000","last_undegraded":"2023-11-26T07:24:07.907834+0000","last_fullsized":"2023-11-26T07:24:07.907834+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:24:07.907834+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:24:07.907834+0000","last_clean_scrub_stamp":"2023-11-26T07:24:07.907834+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"16.0","version":"0'0","reported_seq":1,"reported_epoch":100,"state":"creating+peering","last_fresh":"2023-11-26T07:24:06.914692+0000","last_change":"2023-11-26T07:24:06.914692+0000","last_active":"2023-11-26T07:24:06.899969+0000","last_peered":"2023-11-26T07:24:06.899969+0000","last_clean":"2023-11-26T07:24:06.899969+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-26T07:24:06.914692+0000","last_undegraded":"2023-11-26T07:24:06.914692+0000","last_fullsized":"2023-11-26T07:24:06.914692+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:24:06.899969+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:24:06.899969+0000","last_clean_scrub_stamp":"2023-11-26T07:24:06.899969+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.10","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-26T07:24:07.907834+0000","last_change":"2023-11-26T07:24:07.907834+0000","last_active":"2023-11-26T07:24:07.907834+0000","last_peered":"2023-11-26T07:24:07.907834+0000","last_clean":"2023-11-26T07:24:07.907834+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-26T07:24:07.907834+0000","last_undegraded":"2023-11-26T07:24:07.907834+0000","last_fullsized":"2023-11-26T07:24:07.907834+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:24:07.907834+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:24:07.907834+0000","last_clean_scrub_stamp":"2023-11-26T07:24:07.907834+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"16.11","version":"0'0","reported_seq":1,"reported_epoch":100,"state":"creating+peering","last_fresh":"2023-11-26T07:24:06.914796+0000","last_change":"2023-11-26T07:24:06.914796+0000","last_active":"2023-11-26T07:24:06.899969+0000","last_peered":"2023-11-26T07:24:06.899969+0000","last_clean":"2023-11-26T07:24:06.899969+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-26T07:24:06.914796+0000","last_undegraded":"2023-11-26T07:24:06.914796+0000","last_fullsized":"2023-11-26T07:24:06.914796+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:24:06.899969+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:24:06.899969+0000","last_clean_scrub_stamp":"2023-11-26T07:24:06.899969+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.0","version":"72'128","reported_seq":270,"reported_epoch":99,"state":"active+clean","last_fresh":"2023-11-26T07:24:05.371287+0000","last_change":"2023-11-26T07:17:43.827802+0000","last_active":"2023-11-26T07:24:05.371287+0000","last_peered":"2023-11-26T07:24:05.371287+0000","last_clean":"2023-11-26T07:24:05.371287+0000","last_became_active":"2023-11-26T07:17:43.827308+0000","last_became_peered":"2023-11-26T07:17:43.827308+0000","last_unstale":"2023-11-26T07:24:05.371287+0000","last_undegraded":"2023-11-26T07:24:05.371287+0000","last_fullsized":"2023-11-26T07:24:05.371287+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:17:42.160945+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:17:42.160945+0000","last_clean_scrub_stamp":"2023-11-26T07:17:42.160945+0000","objects_scrubbed":0,"log_size":128,"log_dups_size":0,"ondisk_log_size":128,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T08:17:54.167366+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":212,"num_read_kb":306,"num_write":179,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":17,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0},{"poolid":16,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":212,"num_read_kb":306,"num_write":179,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":128,"ondisk_log_size":128,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574927,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32984,"kb_used_data":1796,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94338856,"statfs":{"total":96636764160,"available":96602988544,"internally_reserved":0,"allocated":1839104,"data_stored":1133346,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":31914442},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Nov 26 07:23:58 2023","interfaces":[{"interface":"back","average":{"1min":0.68200000000000005,"5min":0.70299999999999996,"15min":0.71099999999999997},"min":{"1min":0.47299999999999998,"5min":0.42999999999999999,"15min":0.42999999999999999},"max":{"1min":1.036,"5min":1.036,"15min":1.036},"last":0.70299999999999996},{"interface":"front","average":{"1min":0.70199999999999996,"5min":0.72699999999999998,"15min":0.71499999999999997},"min":{"1min":0.42299999999999999,"5min":0.42299999999999999,"15min":0.42299999999999999},"max":{"1min":1.014,"5min":1.014,"15min":1.0980000000000001},"last":0.78600000000000003}]},{"osd":1,"last update":"Sun Nov 26 07:23:58 2023","interfaces":[{"interface":"back","average":{"1min":0.68400000000000005,"5min":0.68700000000000006,"15min":0.68300000000000005},"min":{"1min":0.45100000000000001,"5min":0.435,"15min":0.435},"max":{"1min":0.85299999999999998,"5min":1.012,"15min":1.0669999999999999},"last":0.67100000000000004},{"interface":"front","average":{"1min":0.66900000000000004,"5min":0.67700000000000005,"15min":0.78500000000000003},"min":{"1min":0.432,"5min":0.432,"15min":0.432},"max":{"1min":0.94699999999999995,"5min":0.94699999999999995,"15min":4.5309999999999997},"last":0.81599999999999995}]},{"osd":2,"last update":"Sun Nov 26 07:23:58 2023","interfaces":[{"interface":"back","average":{"1min":0.69999999999999996,"5min":0.69999999999999996,"15min":0.68400000000000005},"min":{"1min":0.56999999999999995,"5min":0.39700000000000002,"15min":0.39700000000000002},"max":{"1min":0.91600000000000004,"5min":1.0800000000000001,"15min":1.0800000000000001},"last":0.55500000000000005},{"interface":"front","average":{"1min":0.68700000000000006,"5min":0.70499999999999996,"15min":0.70999999999999996},"min":{"1min":0.48999999999999999,"5min":0.40999999999999998,"15min":0.40999999999999998},"max":{"1min":0.97099999999999997,"5min":0.97699999999999998,"15min":1.1339999999999999},"last":0.76800000000000002}]},{"osd":3,"last update":"Sun Nov 26 07:23:58 2023","interfaces":[{"interface":"back","average":{"1min":0.73299999999999998,"5min":0.72799999999999998,"15min":0.70699999999999996},"min":{"1min":0.53200000000000003,"5min":0.47799999999999998,"15min":0.47799999999999998},"max":{"1min":0.99299999999999999,"5min":1.1240000000000001,"15min":1.1240000000000001},"last":0.71599999999999997},{"interface":"front","average":{"1min":0.72399999999999998,"5min":0.74299999999999999,"15min":0.73599999999999999},"min":{"1min":0.50700000000000001,"5min":0.433,"15min":0.433},"max":{"1min":0.93400000000000005,"5min":1.036,"15min":1.4490000000000001},"last":0.58699999999999997}]},{"osd":4,"last update":"Sun Nov 26 07:23:58 2023","interfaces":[{"interface":"back","average":{"1min":0.61099999999999999,"5min":0.61499999999999999,"15min":0.60499999999999998},"min":{"1min":0.48299999999999998,"5min":0.39400000000000002,"15min":0.39400000000000002},"max":{"1min":0.878,"5min":0.96299999999999997,"15min":0.96299999999999997},"last":0.61599999999999999},{"interface":"front","average":{"1min":0.65500000000000003,"5min":0.64400000000000002,"15min":0.66300000000000003},"min":{"1min":0.45800000000000002,"5min":0.35999999999999999,"15min":0.35999999999999999},"max":{"1min":0.83599999999999997,"5min":1.0209999999999999,"15min":1.5509999999999999},"last":0.64200000000000002}]},{"osd":5,"last update":"Sun Nov 26 07:23:58 2023","interfaces":[{"interface":"back","average":{"1min":0.66100000000000003,"5min":0.67400000000000004,"15min":0.66800000000000004},"min":{"1min":0.36599999999999999,"5min":0.36599999999999999,"15min":0.36599999999999999},"max":{"1min":0.86799999999999999,"5min":1.056,"15min":1.415},"last":0.72699999999999998},{"interface":"front","average":{"1min":0.68300000000000005,"5min":0.68200000000000005,"15min":0.70299999999999996},"min":{"1min":0.435,"5min":0.435,"15min":0.435},"max":{"1min":0.873,"5min":1.0840000000000001,"15min":1.3859999999999999},"last":0.68799999999999994}]},{"osd":6,"last update":"Sun Nov 26 07:23:58 2023","interfaces":[{"interface":"back","average":{"1min":0.77300000000000002,"5min":0.78000000000000003,"15min":0.77400000000000002},"min":{"1min":0.53100000000000003,"5min":0.51700000000000002,"15min":0.51700000000000002},"max":{"1min":0.95499999999999996,"5min":1.123,"15min":1.3069999999999999},"last":0.74299999999999999},{"interface":"front","average":{"1min":0.71399999999999997,"5min":0.74399999999999999,"15min":0.76200000000000001},"min":{"1min":0.50600000000000001,"5min":0.46899999999999997,"15min":0.46899999999999997},"max":{"1min":0.92900000000000005,"5min":1.036,"15min":1.6759999999999999},"last":0.752}]}]},{"osd":6,"up_from":13,"seq":55834574929,"num_pgs":3,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32392,"kb_used_data":1204,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339448,"statfs":{"total":96636764160,"available":96603594752,"internally_reserved":0,"allocated":1232896,"data_stored":537161,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Nov 26 07:23:56 2023","interfaces":[{"interface":"back","average":{"1min":0.65400000000000003,"5min":0.67800000000000005,"15min":0.68200000000000005},"min":{"1min":0.48999999999999999,"5min":0.32200000000000001,"15min":0.32200000000000001},"max":{"1min":0.875,"5min":0.91800000000000004,"15min":1.7549999999999999},"last":0.56299999999999994},{"interface":"front","average":{"1min":0.65500000000000003,"5min":0.69599999999999995,"15min":0.87},"min":{"1min":0.36299999999999999,"5min":0.36299999999999999,"15min":0.36299999999999999},"max":{"1min":0.82599999999999996,"5min":0.96199999999999997,"15min":5.7489999999999997},"last":0.61699999999999999}]},{"osd":1,"last update":"Sun Nov 26 07:23:56 2023","interfaces":[{"interface":"back","average":{"1min":0.65000000000000002,"5min":0.70799999999999996,"15min":0.84499999999999997},"min":{"1min":0.54200000000000004,"5min":0.41199999999999998,"15min":0.41199999999999998},"max":{"1min":0.78900000000000003,"5min":0.97799999999999998,"15min":5.4240000000000004},"last":0.67700000000000005},{"interface":"front","average":{"1min":0.68200000000000005,"5min":0.70799999999999996,"15min":0.83399999999999996},"min":{"1min":0.47599999999999998,"5min":0.47599999999999998,"15min":0.47599999999999998},"max":{"1min":0.88600000000000001,"5min":0.96499999999999997,"15min":5.6020000000000003},"last":0.69899999999999995}]},{"osd":2,"last update":"Sun Nov 26 07:23:56 2023","interfaces":[{"interface":"back","average":{"1min":0.623,"5min":0.66600000000000004,"15min":0.68600000000000005},"min":{"1min":0.47999999999999998,"5min":0.41699999999999998,"15min":0.41699999999999998},"max":{"1min":0.89700000000000002,"5min":0.93600000000000005,"15min":1.8520000000000001},"last":0.499},{"interface":"front","average":{"1min":0.71099999999999997,"5min":0.72199999999999998,"15min":0.72999999999999998},"min":{"1min":0.47999999999999998,"5min":0.437,"15min":0.437},"max":{"1min":0.91000000000000003,"5min":0.95199999999999996,"15min":1.9159999999999999},"last":0.63100000000000001}]},{"osd":3,"last update":"Sun Nov 26 07:23:56 2023","interfaces":[{"interface":"back","average":{"1min":0.67900000000000005,"5min":0.70999999999999996,"15min":0.83799999999999997},"min":{"1min":0.47399999999999998,"5min":0.46899999999999997,"15min":0.46899999999999997},"max":{"1min":0.85199999999999998,"5min":0.91500000000000004,"15min":5.4390000000000001},"last":0.57999999999999996},{"interface":"front","average":{"1min":0.65000000000000002,"5min":0.70399999999999996,"15min":0.86399999999999999},"min":{"1min":0.44,"5min":0.44,"15min":0.44},"max":{"1min":0.81299999999999994,"5min":0.90500000000000003,"15min":5.5490000000000004},"last":0.44600000000000001}]},{"osd":4,"last update":"Sun Nov 26 07:23:56 2023","interfaces":[{"interface":"back","average":{"1min":0.61699999999999999,"5min":0.61199999999999999,"15min":0.63200000000000001},"min":{"1min":0.45700000000000002,"5min":0.371,"15min":0.371},"max":{"1min":0.86699999999999999,"5min":1.0129999999999999,"15min":1.0129999999999999},"last":0.71599999999999997},{"interface":"front","average":{"1min":0.623,"5min":0.63400000000000001,"15min":0.66100000000000003},"min":{"1min":0.41299999999999998,"5min":0.38600000000000001,"15min":0.38600000000000001},"max":{"1min":0.85899999999999999,"5min":1.0349999999999999,"15min":1.0349999999999999},"last":0.52400000000000002}]},{"osd":5,"last update":"Sun Nov 26 07:23:56 2023","interfaces":[{"interface":"back","average":{"1min":0.57999999999999996,"5min":0.61899999999999999,"15min":0.64500000000000002},"min":{"1min":0.42799999999999999,"5min":0.38200000000000001,"15min":0.38200000000000001},"max":{"1min":0.85599999999999998,"5min":0.91000000000000003,"15min":0.91000000000000003},"last":0.60299999999999998},{"interface":"front","average":{"1min":0.66100000000000003,"5min":0.68700000000000006,"15min":0.70399999999999996},"min":{"1min":0.501,"5min":0.44,"15min":0.436},"max":{"1min":0.91600000000000004,"5min":1.0960000000000001,"15min":1.0960000000000001},"last":0.47299999999999998}]},{"osd":7,"last update":"Sun Nov 26 07:23:56 2023","interfaces":[{"interface":"back","average":{"1min":0.66000000000000003,"5min":0.69999999999999996,"15min":0.72799999999999998},"min":{"1min":0.497,"5min":0.42599999999999999,"15min":0.42599999999999999},"max":{"1min":0.89400000000000002,"5min":0.95099999999999996,"15min":0.998},"last":0.65500000000000003},{"interface":"front","average":{"1min":0.64800000000000002,"5min":0.68300000000000005,"15min":0.70499999999999996},"min":{"1min":0.55200000000000005,"5min":0.46400000000000002,"15min":0.46400000000000002},"max":{"1min":0.77500000000000002,"5min":0.98799999999999999,"15min":0.98799999999999999},"last":0.54400000000000004}]}]},{"osd":5,"up_from":13,"seq":55834574927,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32392,"kb_used_data":1204,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339448,"statfs":{"total":96636764160,"available":96603594752,"internally_reserved":0,"allocated":1232896,"data_stored":537161,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Nov 26 07:23:49 2023","interfaces":[{"interface":"back","average":{"1min":0.69099999999999995,"5min":0.71999999999999997,"15min":0.68400000000000005},"min":{"1min":0.46899999999999997,"5min":0.36499999999999999,"15min":0.36499999999999999},"max":{"1min":0.876,"5min":2.6949999999999998,"15min":2.6949999999999998},"last":0.623},{"interface":"front","average":{"1min":0.72599999999999998,"5min":0.73099999999999998,"15min":0.68300000000000005},"min":{"1min":0.54100000000000004,"5min":0.38200000000000001,"15min":0.38200000000000001},"max":{"1min":1.0149999999999999,"5min":2.0150000000000001,"15min":2.0150000000000001},"last":0.76500000000000001}]},{"osd":1,"last update":"Sun Nov 26 07:23:49 2023","interfaces":[{"interface":"back","average":{"1min":0.68300000000000005,"5min":0.70699999999999996,"15min":0.68100000000000005},"min":{"1min":0.50900000000000001,"5min":0.41799999999999998,"15min":0.41599999999999998},"max":{"1min":0.97999999999999998,"5min":1.532,"15min":1.532},"last":0.64300000000000002},{"interface":"front","average":{"1min":0.71699999999999997,"5min":0.73699999999999999,"15min":0.70299999999999996},"min":{"1min":0.52100000000000002,"5min":0.443,"15min":0.443},"max":{"1min":0.92900000000000005,"5min":2.6400000000000001,"15min":2.6400000000000001},"last":0.59799999999999998}]},{"osd":2,"last update":"Sun Nov 26 07:23:49 2023","interfaces":[{"interface":"back","average":{"1min":0.72599999999999998,"5min":0.71899999999999997,"15min":0.68899999999999995},"min":{"1min":0.42099999999999999,"5min":0.42099999999999999,"15min":0.42099999999999999},"max":{"1min":0.90200000000000002,"5min":1.665,"15min":1.665},"last":0.54400000000000004},{"interface":"front","average":{"1min":0.72999999999999998,"5min":0.76200000000000001,"15min":0.69999999999999996},"min":{"1min":0.47499999999999998,"5min":0.46200000000000002,"15min":0.46200000000000002},"max":{"1min":1.0009999999999999,"5min":2.125,"15min":2.125},"last":0.70099999999999996}]},{"osd":3,"last update":"Sun Nov 26 07:23:49 2023","interfaces":[{"interface":"back","average":{"1min":0.745,"5min":0.75600000000000001,"15min":0.73199999999999998},"min":{"1min":0.52200000000000002,"5min":0.44700000000000001,"15min":0.43099999999999999},"max":{"1min":0.97699999999999998,"5min":2.0529999999999999,"15min":2.0529999999999999},"last":0.73599999999999999},{"interface":"front","average":{"1min":0.77700000000000002,"5min":0.76100000000000001,"15min":0.70299999999999996},"min":{"1min":0.55700000000000005,"5min":0.48199999999999998,"15min":0.44600000000000001},"max":{"1min":1.022,"5min":1.629,"15min":1.629},"last":0.67600000000000005}]},{"osd":4,"last update":"Sun Nov 26 07:23:49 2023","interfaces":[{"interface":"back","average":{"1min":0.65300000000000002,"5min":0.61599999999999999,"15min":0.628},"min":{"1min":0.39100000000000001,"5min":0.35299999999999998,"15min":0.35299999999999998},"max":{"1min":0.90800000000000003,"5min":2.7429999999999999,"15min":2.7429999999999999},"last":0.56499999999999995},{"interface":"front","average":{"1min":0.70099999999999996,"5min":0.68100000000000005,"15min":0.69899999999999995},"min":{"1min":0.48599999999999999,"5min":0.41499999999999998,"15min":0.41299999999999998},"max":{"1min":0.95299999999999996,"5min":2.4630000000000001,"15min":2.4630000000000001},"last":0.45000000000000001}]},{"osd":6,"last update":"Sun Nov 26 07:23:49 2023","interfaces":[{"interface":"back","average":{"1min":0.68600000000000005,"5min":0.67700000000000005,"15min":0.67300000000000004},"min":{"1min":0.372,"5min":0.372,"15min":0.372},"max":{"1min":0.92400000000000004,"5min":2.1619999999999999,"15min":2.1619999999999999},"last":0.50700000000000001},{"interface":"front","average":{"1min":0.71899999999999997,"5min":0.70999999999999996,"15min":0.72099999999999997},"min":{"1min":0.46500000000000002,"5min":0.436,"15min":0.436},"max":{"1min":0.95799999999999996,"5min":2.7869999999999999,"15min":2.7869999999999999},"last":0.47599999999999998}]},{"osd":7,"last update":"Sun Nov 26 07:23:49 2023","interfaces":[{"interface":"back","average":{"1min":0.73799999999999999,"5min":0.746,"15min":0.78400000000000003},"min":{"1min":0.41399999999999998,"5min":0.41399999999999998,"15min":0.41399999999999998},"max":{"1min":0.94999999999999996,"5min":2.1960000000000002,"15min":2.1960000000000002},"last":0.58599999999999997},{"interface":"front","average":{"1min":0.77100000000000002,"5min":0.76200000000000001,"15min":0.77000000000000002},"min":{"1min":0.44700000000000001,"5min":0.44700000000000001,"15min":0.44700000000000001},"max":{"1min":0.97499999999999998,"5min":1.9510000000000001,"15min":1.9510000000000001},"last":0.60599999999999998}]}]},{"osd":4,"up_from":13,"seq":55834574929,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32396,"kb_used_data":1204,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339444,"statfs":{"total":96636764160,"available":96603590656,"internally_reserved":0,"allocated":1232896,"data_stored":537161,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Nov 26 07:23:53 2023","interfaces":[{"interface":"back","average":{"1min":0.64600000000000002,"5min":0.65800000000000003,"15min":0.748},"min":{"1min":0.443,"5min":0.32100000000000001,"15min":0.32100000000000001},"max":{"1min":0.91100000000000003,"5min":0.96199999999999997,"15min":4.1689999999999996},"last":0.56999999999999995},{"interface":"front","average":{"1min":0.65300000000000002,"5min":0.84699999999999998,"15min":0.94999999999999996},"min":{"1min":0.40799999999999997,"5min":0.40799999999999997,"15min":0.40799999999999997},"max":{"1min":0.83299999999999996,"5min":18.021000000000001,"15min":18.021000000000001},"last":0.70699999999999996}]},{"osd":1,"last update":"Sun Nov 26 07:23:53 2023","interfaces":[{"interface":"back","average":{"1min":0.72699999999999998,"5min":0.73499999999999999,"15min":0.67600000000000005},"min":{"1min":0.55200000000000005,"5min":0.43099999999999999,"15min":0.43099999999999999},"max":{"1min":0.93500000000000005,"5min":4.3159999999999998,"15min":4.3159999999999998},"last":0.54200000000000004},{"interface":"front","average":{"1min":0.67400000000000004,"5min":0.76600000000000001,"15min":0.80200000000000005},"min":{"1min":0.436,"5min":0.39800000000000002,"15min":0.39800000000000002},"max":{"1min":0.94199999999999995,"5min":11.206,"15min":11.206},"last":0.48199999999999998}]},{"osd":2,"last update":"Sun Nov 26 07:23:53 2023","interfaces":[{"interface":"back","average":{"1min":0.71099999999999997,"5min":0.85199999999999998,"15min":0.82199999999999995},"min":{"1min":0.56599999999999995,"5min":0.433,"15min":0.433},"max":{"1min":0.84299999999999997,"5min":15.441000000000001,"15min":15.441000000000001},"last":0.89900000000000002},{"interface":"front","average":{"1min":0.77400000000000002,"5min":0.85599999999999998,"15min":0.996},"min":{"1min":0.59599999999999997,"5min":0.54100000000000004,"15min":0.53400000000000003},"max":{"1min":0.93999999999999995,"5min":9.4800000000000004,"15min":9.4800000000000004},"last":0.80900000000000005}]},{"osd":3,"last update":"Sun Nov 26 07:23:53 2023","interfaces":[{"interface":"back","average":{"1min":0.71199999999999997,"5min":0.82899999999999996,"15min":0.76300000000000001},"min":{"1min":0.53600000000000003,"5min":0.441,"15min":0.441},"max":{"1min":0.95899999999999996,"5min":12.715,"15min":12.715},"last":0.624},{"interface":"front","average":{"1min":0.76500000000000001,"5min":0.81499999999999995,"15min":0.85699999999999998},"min":{"1min":0.63600000000000001,"5min":0.50700000000000001,"15min":0.50700000000000001},"max":{"1min":1.0109999999999999,"5min":8.3239999999999998,"15min":8.3239999999999998},"last":0.84299999999999997}]},{"osd":5,"last update":"Sun Nov 26 07:23:53 2023","interfaces":[{"interface":"back","average":{"1min":0.67300000000000004,"5min":0.629,"15min":0.61699999999999999},"min":{"1min":0.41699999999999998,"5min":0.38200000000000001,"15min":0.38200000000000001},"max":{"1min":0.94199999999999995,"5min":0.94199999999999995,"15min":0.96199999999999997},"last":0.75600000000000001},{"interface":"front","average":{"1min":0.66000000000000003,"5min":0.72399999999999998,"15min":0.66800000000000004},"min":{"1min":0.36399999999999999,"5min":0.35599999999999998,"15min":0.35599999999999998},"max":{"1min":0.88400000000000001,"5min":11.148999999999999,"15min":11.148999999999999},"last":0.72699999999999998}]},{"osd":6,"last update":"Sun Nov 26 07:23:53 2023","interfaces":[{"interface":"back","average":{"1min":0.63400000000000001,"5min":0.60699999999999998,"15min":0.73099999999999998},"min":{"1min":0.48499999999999999,"5min":0.372,"15min":0.372},"max":{"1min":0.76700000000000002,"5min":0.89200000000000002,"15min":2.8330000000000002},"last":0.68400000000000005},{"interface":"front","average":{"1min":0.73799999999999999,"5min":0.71399999999999997,"15min":0.91000000000000003},"min":{"1min":0.47299999999999998,"5min":0.41299999999999998,"15min":0.41299999999999998},"max":{"1min":0.90500000000000003,"5min":2.472,"15min":7.6050000000000004},"last":0.65600000000000003}]},{"osd":7,"last update":"Sun Nov 26 07:23:53 2023","interfaces":[{"interface":"back","average":{"1min":0.71399999999999997,"5min":0.81699999999999995,"15min":0.84599999999999997},"min":{"1min":0.56899999999999995,"5min":0.47699999999999998,"15min":0.44700000000000001},"max":{"1min":0.85699999999999998,"5min":12.653,"15min":12.653},"last":0.77900000000000003},{"interface":"front","average":{"1min":0.78900000000000003,"5min":0.77500000000000002,"15min":0.85299999999999998},"min":{"1min":0.58499999999999996,"5min":0.33300000000000002,"15min":0.33300000000000002},"max":{"1min":0.91300000000000003,"5min":3.8319999999999999,"15min":3.8319999999999999},"last":0.94299999999999995}]}]},{"osd":3,"up_from":13,"seq":55834574930,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32384,"kb_used_data":1204,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339456,"statfs":{"total":96636764160,"available":96603602944,"internally_reserved":0,"allocated":1232896,"data_stored":537161,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1592,"internal_metadata":31914440},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Nov 26 07:23:54 2023","interfaces":[{"interface":"back","average":{"1min":0.40100000000000002,"5min":0.41899999999999998,"15min":0.39200000000000002},"min":{"1min":0.29099999999999998,"5min":0.218,"15min":0.218},"max":{"1min":0.53600000000000003,"5min":0.72799999999999998,"15min":0.72799999999999998},"last":0.33300000000000002},{"interface":"front","average":{"1min":0.47599999999999998,"5min":0.46800000000000003,"15min":0.44400000000000001},"min":{"1min":0.30199999999999999,"5min":0.254,"15min":0.254},"max":{"1min":0.75900000000000001,"5min":0.77300000000000002,"15min":0.77300000000000002},"last":0.437}]},{"osd":1,"last update":"Sun Nov 26 07:23:54 2023","interfaces":[{"interface":"back","average":{"1min":0.44400000000000001,"5min":0.441,"15min":0.42699999999999999},"min":{"1min":0.28799999999999998,"5min":0.28799999999999998,"15min":0.28799999999999998},"max":{"1min":0.59999999999999998,"5min":0.61299999999999999,"15min":0.752},"last":0.35799999999999998},{"interface":"front","average":{"1min":0.51800000000000002,"5min":0.496,"15min":0.48799999999999999},"min":{"1min":0.34000000000000002,"5min":0.29899999999999999,"15min":0.29899999999999999},"max":{"1min":0.80800000000000005,"5min":0.872,"15min":0.872},"last":0.379}]},{"osd":2,"last update":"Sun Nov 26 07:23:54 2023","interfaces":[{"interface":"back","average":{"1min":0.56000000000000005,"5min":0.56599999999999995,"15min":0.54400000000000004},"min":{"1min":0.442,"5min":0.376,"15min":0.376},"max":{"1min":0.77200000000000002,"5min":0.97199999999999998,"15min":0.97199999999999998},"last":0.46800000000000003},{"interface":"front","average":{"1min":0.52100000000000002,"5min":0.53700000000000003,"15min":0.51500000000000001},"min":{"1min":0.32600000000000001,"5min":0.32100000000000001,"15min":0.32100000000000001},"max":{"1min":0.70899999999999996,"5min":0.89700000000000002,"15min":0.89700000000000002},"last":0.45200000000000001}]},{"osd":4,"last update":"Sun Nov 26 07:23:54 2023","interfaces":[{"interface":"back","average":{"1min":0.72599999999999998,"5min":0.71099999999999997,"15min":0.70699999999999996},"min":{"1min":0.51400000000000001,"5min":0.39600000000000002,"15min":0.39600000000000002},"max":{"1min":0.88300000000000001,"5min":1.091,"15min":1.091},"last":0.74399999999999999},{"interface":"front","average":{"1min":0.71699999999999997,"5min":0.72099999999999997,"15min":0.72499999999999998},"min":{"1min":0.52400000000000002,"5min":0.39900000000000002,"15min":0.39900000000000002},"max":{"1min":1.089,"5min":1.242,"15min":1.242},"last":0.56200000000000006}]},{"osd":5,"last update":"Sun Nov 26 07:23:54 2023","interfaces":[{"interface":"back","average":{"1min":0.71599999999999997,"5min":0.70199999999999996,"15min":0.72599999999999998},"min":{"1min":0.48699999999999999,"5min":0.45900000000000002,"15min":0.45900000000000002},"max":{"1min":0.96199999999999997,"5min":1.105,"15min":1.105},"last":0.63700000000000001},{"interface":"front","average":{"1min":0.74399999999999999,"5min":0.73699999999999999,"15min":0.73699999999999999},"min":{"1min":0.54700000000000004,"5min":0.47299999999999998,"15min":0.47299999999999998},"max":{"1min":0.94199999999999995,"5min":1.1319999999999999,"15min":1.1319999999999999},"last":0.57299999999999995}]},{"osd":6,"last update":"Sun Nov 26 07:23:54 2023","interfaces":[{"interface":"back","average":{"1min":0.76900000000000002,"5min":0.75900000000000001,"15min":0.79500000000000004},"min":{"1min":0.51300000000000001,"5min":0.5,"15min":0.5},"max":{"1min":0.96499999999999997,"5min":1.214,"15min":1.2729999999999999},"last":0.69699999999999995},{"interface":"front","average":{"1min":0.76800000000000002,"5min":0.754,"15min":0.78000000000000003},"min":{"1min":0.61799999999999999,"5min":0.48799999999999999,"15min":0.47299999999999998},"max":{"1min":1.1060000000000001,"5min":1.1060000000000001,"15min":1.204},"last":0.68000000000000005}]},{"osd":7,"last update":"Sun Nov 26 07:23:54 2023","interfaces":[{"interface":"back","average":{"1min":0.78200000000000003,"5min":0.755,"15min":0.78500000000000003},"min":{"1min":0.496,"5min":0.437,"15min":0.437},"max":{"1min":1.0349999999999999,"5min":1.125,"15min":1.3600000000000001},"last":0.71199999999999997},{"interface":"front","average":{"1min":0.78100000000000003,"5min":0.78900000000000003,"15min":0.81299999999999994},"min":{"1min":0.627,"5min":0.47299999999999998,"15min":0.47299999999999998},"max":{"1min":1.125,"5min":1.2450000000000001,"15min":1.617},"last":0.72999999999999998}]}]},{"osd":2,"up_from":13,"seq":55834574930,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32404,"kb_used_data":1216,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339436,"statfs":{"total":96636764160,"available":96603582464,"internally_reserved":0,"allocated":1245184,"data_stored":542978,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Nov 26 07:23:52 2023","interfaces":[{"interface":"back","average":{"1min":0.42699999999999999,"5min":0.43099999999999999,"15min":0.61899999999999999},"min":{"1min":0.27200000000000002,"5min":0.27100000000000002,"15min":0.27100000000000002},"max":{"1min":0.67500000000000004,"5min":0.76000000000000001,"15min":4.7030000000000003},"last":0.35899999999999999},{"interface":"front","average":{"1min":0.47299999999999998,"5min":0.46600000000000003,"15min":0.53000000000000003},"min":{"1min":0.378,"5min":0.27300000000000002,"15min":0.27300000000000002},"max":{"1min":0.69799999999999995,"5min":0.77200000000000002,"15min":2.4340000000000002},"last":0.66100000000000003}]},{"osd":1,"last update":"Sun Nov 26 07:23:52 2023","interfaces":[{"interface":"back","average":{"1min":0.48199999999999998,"5min":0.49099999999999999,"15min":0.73899999999999999},"min":{"1min":0.35499999999999998,"5min":0.318,"15min":0.318},"max":{"1min":0.621,"5min":0.86099999999999999,"15min":4.2690000000000001},"last":0.61499999999999999},{"interface":"front","average":{"1min":0.504,"5min":0.501,"15min":0.66100000000000003},"min":{"1min":0.39800000000000002,"5min":0.35099999999999998,"15min":0.31900000000000001},"max":{"1min":0.623,"5min":0.76600000000000001,"15min":3.9940000000000002},"last":0.438}]},{"osd":3,"last update":"Sun Nov 26 07:23:52 2023","interfaces":[{"interface":"back","average":{"1min":0.48199999999999998,"5min":0.47499999999999998,"15min":0.58699999999999997},"min":{"1min":0.311,"5min":0.29699999999999999,"15min":0.29699999999999999},"max":{"1min":0.64700000000000002,"5min":0.82699999999999996,"15min":4.0300000000000002},"last":0.379},{"interface":"front","average":{"1min":0.55500000000000005,"5min":0.55000000000000004,"15min":0.79200000000000004},"min":{"1min":0.443,"5min":0.36099999999999999,"15min":0.36099999999999999},"max":{"1min":0.79700000000000004,"5min":0.86699999999999999,"15min":4.5810000000000004},"last":0.83999999999999997}]},{"osd":4,"last update":"Sun Nov 26 07:23:52 2023","interfaces":[{"interface":"back","average":{"1min":0.73199999999999998,"5min":0.76600000000000001,"15min":0.91200000000000003},"min":{"1min":0.55600000000000005,"5min":0.51400000000000001,"15min":0.51400000000000001},"max":{"1min":0.97499999999999998,"5min":1.01,"15min":4.4269999999999996},"last":0.89700000000000002},{"interface":"front","average":{"1min":0.752,"5min":0.76800000000000002,"15min":0.93200000000000005},"min":{"1min":0.54300000000000004,"5min":0.54000000000000004,"15min":0.54000000000000004},"max":{"1min":1.0009999999999999,"5min":1.0780000000000001,"15min":4.085},"last":0.81999999999999995}]},{"osd":5,"last update":"Sun Nov 26 07:23:52 2023","interfaces":[{"interface":"back","average":{"1min":0.77300000000000002,"5min":0.76600000000000001,"15min":0.83799999999999997},"min":{"1min":0.42099999999999999,"5min":0.42099999999999999,"15min":0.42099999999999999},"max":{"1min":0.97599999999999998,"5min":1.0580000000000001,"15min":3.9590000000000001},"last":0.56200000000000006},{"interface":"front","average":{"1min":0.71999999999999997,"5min":0.78200000000000003,"15min":0.91900000000000004},"min":{"1min":0.51600000000000001,"5min":0.499,"15min":0.499},"max":{"1min":0.96199999999999997,"5min":1.0900000000000001,"15min":4.3849999999999998},"last":1.024}]},{"osd":6,"last update":"Sun Nov 26 07:23:52 2023","interfaces":[{"interface":"back","average":{"1min":0.73699999999999999,"5min":0.76800000000000002,"15min":0.92000000000000004},"min":{"1min":0.45200000000000001,"5min":0.45200000000000001,"15min":0.45200000000000001},"max":{"1min":0.93600000000000005,"5min":1.083,"15min":4.0449999999999999},"last":0.75800000000000001},{"interface":"front","average":{"1min":0.71099999999999997,"5min":0.75900000000000001,"15min":0.86499999999999999},"min":{"1min":0.52700000000000002,"5min":0.48099999999999998,"15min":0.48099999999999998},"max":{"1min":1.0229999999999999,"5min":1.085,"15min":3.992},"last":0.68500000000000005}]},{"osd":7,"last update":"Sun Nov 26 07:23:52 2023","interfaces":[{"interface":"back","average":{"1min":0.76600000000000001,"5min":0.78200000000000003,"15min":0.94999999999999996},"min":{"1min":0.56799999999999995,"5min":0.52200000000000002,"15min":0.52200000000000002},"max":{"1min":1.018,"5min":1.0840000000000001,"15min":4.1219999999999999},"last":0.93899999999999995},{"interface":"front","average":{"1min":0.747,"5min":0.78200000000000003,"15min":0.88600000000000001},"min":{"1min":0.49299999999999999,"5min":0.48199999999999998,"15min":0.48199999999999998},"max":{"1min":0.95399999999999996,"5min":1.036,"15min":3.9159999999999999},"last":0.79800000000000004}]}]},{"osd":1,"up_from":13,"seq":55834574929,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32404,"kb_used_data":1216,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339436,"statfs":{"total":96636764160,"available":96603582464,"internally_reserved":0,"allocated":1245184,"data_stored":542978,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":31914439},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Nov 26 07:23:50 2023","interfaces":[{"interface":"back","average":{"1min":0.41699999999999998,"5min":0.40799999999999997,"15min":0.41399999999999998},"min":{"1min":0.27300000000000002,"5min":0.23599999999999999,"15min":0.23599999999999999},"max":{"1min":0.55600000000000005,"5min":0.59999999999999998,"15min":0.61199999999999999},"last":0.51500000000000001},{"interface":"front","average":{"1min":0.45100000000000001,"5min":0.45000000000000001,"15min":0.47399999999999998},"min":{"1min":0.32800000000000001,"5min":0.26700000000000002,"15min":0.26700000000000002},"max":{"1min":0.65800000000000003,"5min":0.749,"15min":0.78700000000000003},"last":0.34599999999999997}]},{"osd":2,"last update":"Sun Nov 26 07:23:50 2023","interfaces":[{"interface":"back","average":{"1min":0.42999999999999999,"5min":0.437,"15min":0.44900000000000001},"min":{"1min":0.318,"5min":0.27800000000000002,"15min":0.27800000000000002},"max":{"1min":0.58499999999999996,"5min":0.71399999999999997,"15min":0.71399999999999997},"last":0.45700000000000002},{"interface":"front","average":{"1min":0.45500000000000002,"5min":0.47999999999999998,"15min":0.498},"min":{"1min":0.318,"5min":0.27700000000000002,"15min":0.27700000000000002},"max":{"1min":0.60199999999999998,"5min":0.80600000000000005,"15min":0.80600000000000005},"last":0.57199999999999995}]},{"osd":3,"last update":"Sun Nov 26 07:23:50 2023","interfaces":[{"interface":"back","average":{"1min":0.51400000000000001,"5min":0.52500000000000002,"15min":0.52800000000000002},"min":{"1min":0.36099999999999999,"5min":0.36099999999999999,"15min":0.36099999999999999},"max":{"1min":0.66200000000000003,"5min":0.748,"15min":0.748},"last":0.61799999999999999},{"interface":"front","average":{"1min":0.496,"5min":0.499,"15min":0.497},"min":{"1min":0.39500000000000002,"5min":0.31,"15min":0.31},"max":{"1min":0.65800000000000003,"5min":0.73599999999999999,"15min":0.80700000000000005},"last":0.71299999999999997}]},{"osd":4,"last update":"Sun Nov 26 07:23:50 2023","interfaces":[{"interface":"back","average":{"1min":0.79500000000000004,"5min":0.75,"15min":0.71199999999999997},"min":{"1min":0.53700000000000003,"5min":0.41299999999999998,"15min":0.41299999999999998},"max":{"1min":1.319,"5min":1.319,"15min":1.319},"last":0.76900000000000002},{"interface":"front","average":{"1min":0.76500000000000001,"5min":0.72799999999999998,"15min":0.69599999999999995},"min":{"1min":0.52900000000000003,"5min":0.44500000000000001,"15min":0.44500000000000001},"max":{"1min":0.98799999999999999,"5min":1.131,"15min":1.131},"last":0.68899999999999995}]},{"osd":5,"last update":"Sun Nov 26 07:23:50 2023","interfaces":[{"interface":"back","average":{"1min":0.77000000000000002,"5min":0.746,"15min":0.72799999999999998},"min":{"1min":0.47899999999999998,"5min":0.47899999999999998,"15min":0.47899999999999998},"max":{"1min":0.93400000000000005,"5min":1.1919999999999999,"15min":1.1919999999999999},"last":0.748},{"interface":"front","average":{"1min":0.78000000000000003,"5min":0.749,"15min":0.72699999999999998},"min":{"1min":0.59899999999999998,"5min":0.47099999999999997,"15min":0.47099999999999997},"max":{"1min":1.016,"5min":1.0609999999999999,"15min":1.0609999999999999},"last":0.73399999999999999}]},{"osd":6,"last update":"Sun Nov 26 07:23:50 2023","interfaces":[{"interface":"back","average":{"1min":0.79800000000000004,"5min":0.76100000000000001,"15min":0.72299999999999998},"min":{"1min":0.61199999999999999,"5min":0.53200000000000003,"15min":0.50600000000000001},"max":{"1min":1.2190000000000001,"5min":1.2190000000000001,"15min":1.2190000000000001},"last":0.54700000000000004},{"interface":"front","average":{"1min":0.84899999999999998,"5min":0.76900000000000002,"15min":0.72299999999999998},"min":{"1min":0.502,"5min":0.46999999999999997,"15min":0.46999999999999997},"max":{"1min":1.1659999999999999,"5min":1.1659999999999999,"15min":1.1659999999999999},"last":0.78200000000000003}]},{"osd":7,"last update":"Sun Nov 26 07:23:50 2023","interfaces":[{"interface":"back","average":{"1min":0.82899999999999996,"5min":0.76900000000000002,"15min":0.73299999999999998},"min":{"1min":0.627,"5min":0.50600000000000001,"15min":0.50600000000000001},"max":{"1min":1.0920000000000001,"5min":1.0920000000000001,"15min":1.0920000000000001},"last":0.65200000000000002},{"interface":"front","average":{"1min":0.82599999999999996,"5min":0.75700000000000001,"15min":0.754},"min":{"1min":0.58499999999999996,"5min":0.48499999999999999,"15min":0.48499999999999999},"max":{"1min":1.115,"5min":1.115,"15min":1.115},"last":0.80200000000000005}]}]},{"osd":0,"up_from":13,"seq":55834574929,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32964,"kb_used_data":1784,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94338876,"statfs":{"total":96636764160,"available":96603009024,"internally_reserved":0,"allocated":1826816,"data_stored":1127529,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Sun Nov 26 07:23:47 2023","interfaces":[{"interface":"back","average":{"1min":0.41799999999999998,"5min":0.42999999999999999,"15min":0.42999999999999999},"min":{"1min":0.28899999999999998,"5min":0.27000000000000002,"15min":0.27000000000000002},"max":{"1min":0.67300000000000004,"5min":1.26,"15min":1.26},"last":0.313},{"interface":"front","average":{"1min":0.46300000000000002,"5min":0.46800000000000003,"15min":0.47799999999999998},"min":{"1min":0.31,"5min":0.30199999999999999,"15min":0.29899999999999999},"max":{"1min":0.65500000000000003,"5min":1.212,"15min":1.212},"last":0.376}]},{"osd":2,"last update":"Sun Nov 26 07:23:47 2023","interfaces":[{"interface":"back","average":{"1min":0.499,"5min":0.50600000000000001,"15min":0.52600000000000002},"min":{"1min":0.36599999999999999,"5min":0.34699999999999998,"15min":0.34699999999999998},"max":{"1min":0.69099999999999995,"5min":0.84199999999999997,"15min":0.84199999999999997},"last":0.438},{"interface":"front","average":{"1min":0.57799999999999996,"5min":0.55600000000000005,"15min":0.56100000000000005},"min":{"1min":0.40500000000000003,"5min":0.39800000000000002,"15min":0.36399999999999999},"max":{"1min":0.82199999999999995,"5min":1.0800000000000001,"15min":1.0800000000000001},"last":0.48599999999999999}]},{"osd":3,"last update":"Sun Nov 26 07:23:47 2023","interfaces":[{"interface":"back","average":{"1min":0.5,"5min":0.51300000000000001,"15min":0.52500000000000002},"min":{"1min":0.33500000000000002,"5min":0.33500000000000002,"15min":0.33500000000000002},"max":{"1min":0.64100000000000001,"5min":1.3149999999999999,"15min":1.3149999999999999},"last":0.499},{"interface":"front","average":{"1min":0.54200000000000004,"5min":0.52900000000000003,"15min":0.55100000000000005},"min":{"1min":0.35499999999999998,"5min":0.35499999999999998,"15min":0.35499999999999998},"max":{"1min":0.69699999999999995,"5min":0.76400000000000001,"15min":1.2509999999999999},"last":0.42399999999999999}]},{"osd":4,"last update":"Sun Nov 26 07:23:47 2023","interfaces":[{"interface":"back","average":{"1min":0.747,"5min":0.745,"15min":0.754},"min":{"1min":0.52800000000000002,"5min":0.44700000000000001,"15min":0.44700000000000001},"max":{"1min":0.98299999999999998,"5min":1.0900000000000001,"15min":1.2010000000000001},"last":0.73299999999999998},{"interface":"front","average":{"1min":0.78800000000000003,"5min":0.78600000000000003,"15min":0.78800000000000003},"min":{"1min":0.55000000000000004,"5min":0.51100000000000001,"15min":0.51100000000000001},"max":{"1min":1.298,"5min":1.381,"15min":1.381},"last":0.746}]},{"osd":5,"last update":"Sun Nov 26 07:23:47 2023","interfaces":[{"interface":"back","average":{"1min":0.72699999999999998,"5min":0.75,"15min":0.75600000000000001},"min":{"1min":0.45200000000000001,"5min":0.42199999999999999,"15min":0.42199999999999999},"max":{"1min":0.94899999999999995,"5min":1.1819999999999999,"15min":1.1819999999999999},"last":0.65300000000000002},{"interface":"front","average":{"1min":0.78200000000000003,"5min":0.80200000000000005,"15min":0.79800000000000004},"min":{"1min":0.53000000000000003,"5min":0.53000000000000003,"15min":0.53000000000000003},"max":{"1min":1.329,"5min":1.399,"15min":1.399},"last":0.69699999999999995}]},{"osd":6,"last update":"Sun Nov 26 07:23:47 2023","interfaces":[{"interface":"back","average":{"1min":0.72399999999999998,"5min":0.69599999999999995,"15min":0.74099999999999999},"min":{"1min":0.48499999999999999,"5min":0.48499999999999999,"15min":0.48499999999999999},"max":{"1min":1.0740000000000001,"5min":1.0740000000000001,"15min":1.3029999999999999},"last":0.56499999999999995},{"interface":"front","average":{"1min":0.72799999999999998,"5min":0.73699999999999999,"15min":0.75800000000000001},"min":{"1min":0.51200000000000001,"5min":0.47199999999999998,"15min":0.47199999999999998},"max":{"1min":1.2170000000000001,"5min":1.3600000000000001,"15min":1.3600000000000001},"last":0.59599999999999997}]},{"osd":7,"last update":"Sun Nov 26 07:23:47 2023","interfaces":[{"interface":"back","average":{"1min":0.77400000000000002,"5min":0.76800000000000002,"15min":0.79600000000000004},"min":{"1min":0.59799999999999998,"5min":0.56899999999999995,"15min":0.56899999999999995},"max":{"1min":1.1719999999999999,"5min":1.1719999999999999,"15min":1.177},"last":0.71499999999999997},{"interface":"front","average":{"1min":0.76400000000000001,"5min":0.75800000000000001,"15min":0.78500000000000003},"min":{"1min":0.51900000000000002,"5min":0.47899999999999998,"15min":0.47899999999999998},"max":{"1min":1.137,"5min":1.238,"15min":1.2390000000000001},"last":0.68600000000000005}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-11-26T07:24:12.998 DEBUG:teuthology.orchestra.run.smithi143:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-11-26T07:24:13.178 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:13.179+0000 7fde115f2640 1 Processor -- start 2023-11-26T07:24:13.178 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:13.179+0000 7fde115f2640 1 -- start start 2023-11-26T07:24:13.180 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:13.179+0000 7fde115f2640 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7fde0c15d2a0 0x7fde0c0d00e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:24:13.180 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:13.179+0000 7fde115f2640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fde0c15d680 0x7fde0c0cec30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:24:13.180 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:13.179+0000 7fde115f2640 1 -- --> v1:172.21.15.188:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fde0c0cdc40 con 0x7fde0c0d0e80 2023-11-26T07:24:13.180 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:13.179+0000 7fde115f2640 1 -- --> v2:172.21.15.143:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fde0c0ce7a0 con 0x7fde0c15d2a0 2023-11-26T07:24:13.180 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:13.179+0000 7fde115f2640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fde0c0d1440 con 0x7fde0c15d680 2023-11-26T07:24:13.180 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:13.179+0000 7fde0bfff640 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7fde0c15d2a0 0x7fde0c0d00e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:24:13.180 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:13.179+0000 7fde0bfff640 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7fde0c15d2a0 0x7fde0c0d00e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:33830/0 (socket says 172.21.15.143:33830) 2023-11-26T07:24:13.180 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:13.179+0000 7fde0bfff640 1 -- 172.21.15.143:0/4176068607 learned_addr learned my addr 172.21.15.143:0/4176068607 (peer_addr_for_me v2:172.21.15.143:0/0) 2023-11-26T07:24:13.181 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:13.179+0000 7fde0b7fe640 1 --2- 172.21.15.143:0/4176068607 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fde0c15d680 0x7fde0c0cec30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:24:13.181 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:13.179+0000 7fde0bfff640 1 -- 172.21.15.143:0/4176068607 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fde0c15d680 msgr2=0x7fde0c0cec30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-26T07:24:13.181 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:13.179+0000 7fde0bfff640 1 --2- 172.21.15.143:0/4176068607 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fde0c15d680 0x7fde0c0cec30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:24:13.181 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:13.179+0000 7fde0bfff640 1 -- 172.21.15.143:0/4176068607 >> v1:172.21.15.188:6789/0 conn(0x7fde0c0d0e80 legacy=0x7fde0c0d1680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-26T07:24:13.181 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:13.179+0000 7fde0bfff640 1 -- 172.21.15.143:0/4176068607 --> v2:172.21.15.143:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fde0c0cde70 con 0x7fde0c15d2a0 2023-11-26T07:24:13.181 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:13.179+0000 7fde0b7fe640 1 --2- 172.21.15.143:0/4176068607 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fde0c15d680 0x7fde0c0cec30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-26T07:24:13.183 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:13.183+0000 7fde0bfff640 1 --2- 172.21.15.143:0/4176068607 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fde0c15d2a0 0x7fde0c0d00e0 secure :-1 s=READY pgs=1309 cs=0 l=1 rev1=1 crypto rx=0x7fddfc009870 tx=0x7fddfc01c440 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b7ad919ae9b1b4c server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:24:13.183 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:13.183+0000 7fde0affd640 1 -- 172.21.15.143:0/4176068607 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fddfc0ab600 con 0x7fde0c15d2a0 2023-11-26T07:24:13.183 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:13.183+0000 7fde0affd640 1 -- 172.21.15.143:0/4176068607 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fddfc0ab760 con 0x7fde0c15d2a0 2023-11-26T07:24:13.183 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:13.183+0000 7fde0affd640 1 -- 172.21.15.143:0/4176068607 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fddfc0abaf0 con 0x7fde0c15d2a0 2023-11-26T07:24:13.184 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:13.183+0000 7fde115f2640 1 -- 172.21.15.143:0/4176068607 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fde0c15d2a0 msgr2=0x7fde0c0d00e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:24:13.184 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:13.183+0000 7fde115f2640 1 --2- 172.21.15.143:0/4176068607 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fde0c15d2a0 0x7fde0c0d00e0 secure :-1 s=READY pgs=1309 cs=0 l=1 rev1=1 crypto rx=0x7fddfc009870 tx=0x7fddfc01c440 comp rx=0 tx=0).stop 2023-11-26T07:24:13.184 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:13.183+0000 7fde115f2640 1 -- 172.21.15.143:0/4176068607 shutdown_connections 2023-11-26T07:24:13.184 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:13.183+0000 7fde115f2640 1 --2- 172.21.15.143:0/4176068607 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fde0c15d680 0x7fde0c0cec30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:24:13.184 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:13.183+0000 7fde115f2640 1 --2- 172.21.15.143:0/4176068607 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fde0c15d2a0 0x7fde0c0d00e0 unknown :-1 s=CLOSED pgs=1309 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:24:13.184 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:13.183+0000 7fde115f2640 1 -- 172.21.15.143:0/4176068607 >> 172.21.15.143:0/4176068607 conn(0x7fde0c0d30a0 msgr2=0x7fde0c0d3480 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:24:13.185 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:13.183+0000 7fde115f2640 1 -- 172.21.15.143:0/4176068607 shutdown_connections 2023-11-26T07:24:13.185 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:13.183+0000 7fde115f2640 1 -- 172.21.15.143:0/4176068607 wait complete. 2023-11-26T07:24:13.185 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:13.183+0000 7fde115f2640 1 Processor -- start 2023-11-26T07:24:13.185 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:13.187+0000 7fde115f2640 1 -- start start 2023-11-26T07:24:13.187 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:13.187+0000 7fde115f2640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fde0c0d0e80 0x7fde0c0c6230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:24:13.187 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:13.187+0000 7fde115f2640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fde0c15d2a0 0x7fde0c0c8010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:24:13.188 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:13.187+0000 7fde115f2640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fde0c15d680 0x7fde0c0ca170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:24:13.188 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:13.187+0000 7fde115f2640 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fde0c0c6f70 con 0x7fde0c15d680 2023-11-26T07:24:13.188 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:13.187+0000 7fde115f2640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fde0c0c7250 con 0x7fde0c0d0e80 2023-11-26T07:24:13.188 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:13.187+0000 7fde115f2640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fde0c0c7530 con 0x7fde0c15d2a0 2023-11-26T07:24:13.188 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:13.187+0000 7fde10df1640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fde0c15d680 0x7fde0c0ca170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:24:13.188 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:13.187+0000 7fde10df1640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fde0c15d680 0x7fde0c0ca170 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:33838/0 (socket says 172.21.15.143:33838) 2023-11-26T07:24:13.188 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:13.187+0000 7fde10df1640 1 -- 172.21.15.143:0/2745294367 learned_addr learned my addr 172.21.15.143:0/2745294367 (peer_addr_for_me v2:172.21.15.143:0/0) 2023-11-26T07:24:13.188 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:13.187+0000 7fde0bfff640 1 --2- 172.21.15.143:0/2745294367 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fde0c0d0e80 0x7fde0c0c6230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:24:13.188 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:13.187+0000 7fde0b7fe640 1 --2- 172.21.15.143:0/2745294367 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fde0c15d2a0 0x7fde0c0c8010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:24:13.188 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:13.187+0000 7fde10df1640 1 -- 172.21.15.143:0/2745294367 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fde0c15d2a0 msgr2=0x7fde0c0c8010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:24:13.188 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:13.187+0000 7fde10df1640 1 --2- 172.21.15.143:0/2745294367 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fde0c15d2a0 0x7fde0c0c8010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:24:13.188 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:13.187+0000 7fde10df1640 1 -- 172.21.15.143:0/2745294367 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fde0c0d0e80 msgr2=0x7fde0c0c6230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:24:13.188 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:13.187+0000 7fde10df1640 1 --2- 172.21.15.143:0/2745294367 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fde0c0d0e80 0x7fde0c0c6230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:24:13.189 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:13.187+0000 7fde10df1640 1 -- 172.21.15.143:0/2745294367 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fddfc0094d0 con 0x7fde0c15d680 2023-11-26T07:24:13.189 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:13.187+0000 7fde10df1640 1 --2- 172.21.15.143:0/2745294367 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fde0c15d680 0x7fde0c0ca170 secure :-1 s=READY pgs=1310 cs=0 l=1 rev1=1 crypto rx=0x7fde0400c780 tx=0x7fde0400cc40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:24:13.189 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:13.187+0000 7fde08ff9640 1 -- 172.21.15.143:0/2745294367 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fde04004b70 con 0x7fde0c15d680 2023-11-26T07:24:13.189 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:13.187+0000 7fde08ff9640 1 -- 172.21.15.143:0/2745294367 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fde0400ce90 con 0x7fde0c15d680 2023-11-26T07:24:13.190 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:13.187+0000 7fde08ff9640 1 -- 172.21.15.143:0/2745294367 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fde0400a850 con 0x7fde0c15d680 2023-11-26T07:24:13.190 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:13.187+0000 7fde115f2640 1 -- 172.21.15.143:0/2745294367 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fde0c0c7810 con 0x7fde0c15d680 2023-11-26T07:24:13.190 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:13.187+0000 7fde115f2640 1 -- 172.21.15.143:0/2745294367 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fde0c0c6770 con 0x7fde0c15d680 2023-11-26T07:24:13.190 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:13.187+0000 7fde115f2640 1 -- 172.21.15.143:0/2745294367 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fde0c067500 con 0x7fde0c15d680 2023-11-26T07:24:13.190 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:13.191+0000 7fde08ff9640 1 -- 172.21.15.143:0/2745294367 <== mon.0 v2:172.21.15.143:3300/0 4 ==== mgrmap(e 16) v1 ==== 85954+0+0 (secure 0 0 0) 0x7fde0400a9b0 con 0x7fde0c15d680 2023-11-26T07:24:13.191 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:13.191+0000 7fde08ff9640 1 --2- 172.21.15.143:0/2745294367 >> [v2:172.21.15.188:6836/3401128553,v1:172.21.15.188:6837/3401128553] conn(0x7fddf8066590 0x7fddf8068a50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:24:13.191 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:13.191+0000 7fde08ff9640 1 -- 172.21.15.143:0/2745294367 <== mon.0 v2:172.21.15.143:3300/0 5 ==== osd_map(103..103 src has 1..103) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fde04084150 con 0x7fde0c15d680 2023-11-26T07:24:13.191 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:13.191+0000 7fde0bfff640 1 --2- 172.21.15.143:0/2745294367 >> [v2:172.21.15.188:6836/3401128553,v1:172.21.15.188:6837/3401128553] conn(0x7fddf8066590 0x7fddf8068a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:24:13.192 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:13.191+0000 7fde0bfff640 1 --2- 172.21.15.143:0/2745294367 >> [v2:172.21.15.188:6836/3401128553,v1:172.21.15.188:6837/3401128553] conn(0x7fddf8066590 0x7fddf8068a50 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7fddfc0af000 tx=0x7fddfc0b6040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:24:13.194 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:13.195+0000 7fde08ff9640 1 -- 172.21.15.143:0/2745294367 <== mon.0 v2:172.21.15.143:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172649 (secure 0 0 0) 0x7fde040514d0 con 0x7fde0c15d680 2023-11-26T07:24:13.341 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:13.339+0000 7fde115f2640 1 -- 172.21.15.143:0/2745294367 --> [v2:172.21.15.188:6836/3401128553,v1:172.21.15.188:6837/3401128553] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7fde0c0b2c90 con 0x7fddf8066590 2023-11-26T07:24:13.349 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:13.347+0000 7fde08ff9640 1 -- 172.21.15.143:0/2745294367 <== mgr.4108 v2:172.21.15.188:6836/3401128553 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+289651 (secure 0 0 0) 0x7fddfc0ab9d0 con 0x7fddf8066590 2023-11-26T07:24:13.352 INFO:teuthology.orchestra.run.smithi143.stdout: 2023-11-26T07:24:13.353 INFO:teuthology.orchestra.run.smithi143.stderr:dumped all 2023-11-26T07:24:13.355 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:13.351+0000 7fde115f2640 1 -- 172.21.15.143:0/2745294367 >> [v2:172.21.15.188:6836/3401128553,v1:172.21.15.188:6837/3401128553] conn(0x7fddf8066590 msgr2=0x7fddf8068a50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:24:13.355 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:13.351+0000 7fde115f2640 1 --2- 172.21.15.143:0/2745294367 >> [v2:172.21.15.188:6836/3401128553,v1:172.21.15.188:6837/3401128553] conn(0x7fddf8066590 0x7fddf8068a50 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7fddfc0af000 tx=0x7fddfc0b6040 comp rx=0 tx=0).stop 2023-11-26T07:24:13.355 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:13.351+0000 7fde115f2640 1 -- 172.21.15.143:0/2745294367 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fde0c15d680 msgr2=0x7fde0c0ca170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:24:13.355 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:13.351+0000 7fde115f2640 1 --2- 172.21.15.143:0/2745294367 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fde0c15d680 0x7fde0c0ca170 secure :-1 s=READY pgs=1310 cs=0 l=1 rev1=1 crypto rx=0x7fde0400c780 tx=0x7fde0400cc40 comp rx=0 tx=0).stop 2023-11-26T07:24:13.355 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:13.351+0000 7fde115f2640 1 -- 172.21.15.143:0/2745294367 shutdown_connections 2023-11-26T07:24:13.355 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:13.351+0000 7fde115f2640 1 --2- 172.21.15.143:0/2745294367 >> [v2:172.21.15.188:6836/3401128553,v1:172.21.15.188:6837/3401128553] conn(0x7fddf8066590 0x7fddf8068a50 unknown :-1 s=CLOSED pgs=130 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:24:13.355 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:13.351+0000 7fde115f2640 1 --2- 172.21.15.143:0/2745294367 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fde0c15d680 0x7fde0c0ca170 unknown :-1 s=CLOSED pgs=1310 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:24:13.355 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:13.351+0000 7fde115f2640 1 --2- 172.21.15.143:0/2745294367 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fde0c15d2a0 0x7fde0c0c8010 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:24:13.355 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:13.351+0000 7fde115f2640 1 --2- 172.21.15.143:0/2745294367 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fde0c0d0e80 0x7fde0c0c6230 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:24:13.355 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:13.351+0000 7fde115f2640 1 -- 172.21.15.143:0/2745294367 >> 172.21.15.143:0/2745294367 conn(0x7fde0c0d30a0 msgr2=0x7fde0c0b14a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:24:13.355 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:13.351+0000 7fde115f2640 1 -- 172.21.15.143:0/2745294367 shutdown_connections 2023-11-26T07:24:13.355 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:13.355+0000 7fde115f2640 1 -- 172.21.15.143:0/2745294367 wait complete. 2023-11-26T07:24:13.373 INFO:teuthology.orchestra.run.smithi143.stdout:{"pg_ready":true,"pg_map":{"version":66,"stamp":"2023-11-26T07:24:13.298815+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592778,"num_objects":23,"num_object_clones":0,"num_object_copies":46,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":23,"num_whiteouts":0,"num_read":212,"num_read_kb":306,"num_write":223,"num_write_kb":2430,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":166,"ondisk_log_size":166,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":175,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":260724,"kb_used_data":11240,"kb_used_omap":12,"kb_used_meta":249331,"kb_avail":754713996,"statfs":{"total":773094113280,"available":772827131904,"internally_reserved":0,"allocated":11509760,"data_stored":5684180,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12749,"internal_metadata":255315507},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":36,"apply_latency_ms":36,"commit_latency_ns":36000000,"apply_latency_ns":36000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2410,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"6.001530"},"pg_stats":[{"pgid":"17.3f","version":"0'0","reported_seq":11,"reported_epoch":103,"state":"active+clean","last_fresh":"2023-11-26T07:24:09.516298+0000","last_change":"2023-11-26T07:24:08.532738+0000","last_active":"2023-11-26T07:24:09.516298+0000","last_peered":"2023-11-26T07:24:09.516298+0000","last_clean":"2023-11-26T07:24:09.516298+0000","last_became_active":"2023-11-26T07:24:08.532489+0000","last_became_peered":"2023-11-26T07:24:08.532489+0000","last_unstale":"2023-11-26T07:24:09.516298+0000","last_undegraded":"2023-11-26T07:24:09.516298+0000","last_fullsized":"2023-11-26T07:24:09.516298+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:24:07.907834+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:24:07.907834+0000","last_clean_scrub_stamp":"2023-11-26T07:24:07.907834+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T13:20:52.556902+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.3e","version":"0'0","reported_seq":11,"reported_epoch":103,"state":"active+clean","last_fresh":"2023-11-26T07:24:09.514165+0000","last_change":"2023-11-26T07:24:08.532208+0000","last_active":"2023-11-26T07:24:09.514165+0000","last_peered":"2023-11-26T07:24:09.514165+0000","last_clean":"2023-11-26T07:24:09.514165+0000","last_became_active":"2023-11-26T07:24:08.531841+0000","last_became_peered":"2023-11-26T07:24:08.531841+0000","last_unstale":"2023-11-26T07:24:09.514165+0000","last_undegraded":"2023-11-26T07:24:09.514165+0000","last_fullsized":"2023-11-26T07:24:09.514165+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:24:07.907834+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:24:07.907834+0000","last_clean_scrub_stamp":"2023-11-26T07:24:07.907834+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T14:12:27.285662+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"17.3d","version":"0'0","reported_seq":11,"reported_epoch":103,"state":"active+clean","last_fresh":"2023-11-26T07:24:09.915420+0000","last_change":"2023-11-26T07:24:08.537311+0000","last_active":"2023-11-26T07:24:09.915420+0000","last_peered":"2023-11-26T07:24:09.915420+0000","last_clean":"2023-11-26T07:24:09.915420+0000","last_became_active":"2023-11-26T07:24:08.536626+0000","last_became_peered":"2023-11-26T07:24:08.536626+0000","last_unstale":"2023-11-26T07:24:09.915420+0000","last_undegraded":"2023-11-26T07:24:09.915420+0000","last_fullsized":"2023-11-26T07:24:09.915420+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:24:07.907834+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:24:07.907834+0000","last_clean_scrub_stamp":"2023-11-26T07:24:07.907834+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T10:12:44.059607+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"17.3c","version":"0'0","reported_seq":11,"reported_epoch":103,"state":"active+clean","last_fresh":"2023-11-26T07:24:09.515956+0000","last_change":"2023-11-26T07:24:08.521166+0000","last_active":"2023-11-26T07:24:09.515956+0000","last_peered":"2023-11-26T07:24:09.515956+0000","last_clean":"2023-11-26T07:24:09.515956+0000","last_became_active":"2023-11-26T07:24:08.520884+0000","last_became_peered":"2023-11-26T07:24:08.520884+0000","last_unstale":"2023-11-26T07:24:09.515956+0000","last_undegraded":"2023-11-26T07:24:09.515956+0000","last_fullsized":"2023-11-26T07:24:09.515956+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:24:07.907834+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:24:07.907834+0000","last_clean_scrub_stamp":"2023-11-26T07:24:07.907834+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T15:37:05.104632+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.3b","version":"0'0","reported_seq":11,"reported_epoch":103,"state":"active+clean","last_fresh":"2023-11-26T07:24:09.918546+0000","last_change":"2023-11-26T07:24:08.522772+0000","last_active":"2023-11-26T07:24:09.918546+0000","last_peered":"2023-11-26T07:24:09.918546+0000","last_clean":"2023-11-26T07:24:09.918546+0000","last_became_active":"2023-11-26T07:24:08.522493+0000","last_became_peered":"2023-11-26T07:24:08.522493+0000","last_unstale":"2023-11-26T07:24:09.918546+0000","last_undegraded":"2023-11-26T07:24:09.918546+0000","last_fullsized":"2023-11-26T07:24:09.918546+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:24:07.907834+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:24:07.907834+0000","last_clean_scrub_stamp":"2023-11-26T07:24:07.907834+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T11:28:52.589808+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"17.3a","version":"0'0","reported_seq":11,"reported_epoch":103,"state":"active+clean","last_fresh":"2023-11-26T07:24:09.917631+0000","last_change":"2023-11-26T07:24:08.533710+0000","last_active":"2023-11-26T07:24:09.917631+0000","last_peered":"2023-11-26T07:24:09.917631+0000","last_clean":"2023-11-26T07:24:09.917631+0000","last_became_active":"2023-11-26T07:24:08.532418+0000","last_became_peered":"2023-11-26T07:24:08.532418+0000","last_unstale":"2023-11-26T07:24:09.917631+0000","last_undegraded":"2023-11-26T07:24:09.917631+0000","last_fullsized":"2023-11-26T07:24:09.917631+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:24:07.907834+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:24:07.907834+0000","last_clean_scrub_stamp":"2023-11-26T07:24:07.907834+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T10:18:12.472511+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"17.39","version":"0'0","reported_seq":11,"reported_epoch":103,"state":"active+clean","last_fresh":"2023-11-26T07:24:09.693796+0000","last_change":"2023-11-26T07:24:08.529839+0000","last_active":"2023-11-26T07:24:09.693796+0000","last_peered":"2023-11-26T07:24:09.693796+0000","last_clean":"2023-11-26T07:24:09.693796+0000","last_became_active":"2023-11-26T07:24:08.529336+0000","last_became_peered":"2023-11-26T07:24:08.529336+0000","last_unstale":"2023-11-26T07:24:09.693796+0000","last_undegraded":"2023-11-26T07:24:09.693796+0000","last_fullsized":"2023-11-26T07:24:09.693796+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:24:07.907834+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:24:07.907834+0000","last_clean_scrub_stamp":"2023-11-26T07:24:07.907834+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T11:08:47.373995+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"17.38","version":"0'0","reported_seq":11,"reported_epoch":103,"state":"active+clean","last_fresh":"2023-11-26T07:24:09.915788+0000","last_change":"2023-11-26T07:24:08.536959+0000","last_active":"2023-11-26T07:24:09.915788+0000","last_peered":"2023-11-26T07:24:09.915788+0000","last_clean":"2023-11-26T07:24:09.915788+0000","last_became_active":"2023-11-26T07:24:08.535849+0000","last_became_peered":"2023-11-26T07:24:08.535849+0000","last_unstale":"2023-11-26T07:24:09.915788+0000","last_undegraded":"2023-11-26T07:24:09.915788+0000","last_fullsized":"2023-11-26T07:24:09.915788+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:24:07.907834+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:24:07.907834+0000","last_clean_scrub_stamp":"2023-11-26T07:24:07.907834+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T16:47:52.234162+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"17.37","version":"0'0","reported_seq":11,"reported_epoch":103,"state":"active+clean","last_fresh":"2023-11-26T07:24:09.516364+0000","last_change":"2023-11-26T07:24:08.523776+0000","last_active":"2023-11-26T07:24:09.516364+0000","last_peered":"2023-11-26T07:24:09.516364+0000","last_clean":"2023-11-26T07:24:09.516364+0000","last_became_active":"2023-11-26T07:24:08.523509+0000","last_became_peered":"2023-11-26T07:24:08.523509+0000","last_unstale":"2023-11-26T07:24:09.516364+0000","last_undegraded":"2023-11-26T07:24:09.516364+0000","last_fullsized":"2023-11-26T07:24:09.516364+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:24:07.907834+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:24:07.907834+0000","last_clean_scrub_stamp":"2023-11-26T07:24:07.907834+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T16:01:33.915233+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.36","version":"0'0","reported_seq":11,"reported_epoch":103,"state":"active+clean","last_fresh":"2023-11-26T07:24:09.516258+0000","last_change":"2023-11-26T07:24:08.523110+0000","last_active":"2023-11-26T07:24:09.516258+0000","last_peered":"2023-11-26T07:24:09.516258+0000","last_clean":"2023-11-26T07:24:09.516258+0000","last_became_active":"2023-11-26T07:24:08.522851+0000","last_became_peered":"2023-11-26T07:24:08.522851+0000","last_unstale":"2023-11-26T07:24:09.516258+0000","last_undegraded":"2023-11-26T07:24:09.516258+0000","last_fullsized":"2023-11-26T07:24:09.516258+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:24:07.907834+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:24:07.907834+0000","last_clean_scrub_stamp":"2023-11-26T07:24:07.907834+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T07:40:15.539541+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.35","version":"0'0","reported_seq":11,"reported_epoch":103,"state":"active+clean","last_fresh":"2023-11-26T07:24:09.514058+0000","last_change":"2023-11-26T07:24:08.531401+0000","last_active":"2023-11-26T07:24:09.514058+0000","last_peered":"2023-11-26T07:24:09.514058+0000","last_clean":"2023-11-26T07:24:09.514058+0000","last_became_active":"2023-11-26T07:24:08.531096+0000","last_became_peered":"2023-11-26T07:24:08.531096+0000","last_unstale":"2023-11-26T07:24:09.514058+0000","last_undegraded":"2023-11-26T07:24:09.514058+0000","last_fullsized":"2023-11-26T07:24:09.514058+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:24:07.907834+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:24:07.907834+0000","last_clean_scrub_stamp":"2023-11-26T07:24:07.907834+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T19:20:30.259326+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"17.34","version":"0'0","reported_seq":11,"reported_epoch":103,"state":"active+clean","last_fresh":"2023-11-26T07:24:09.515968+0000","last_change":"2023-11-26T07:24:08.523089+0000","last_active":"2023-11-26T07:24:09.515968+0000","last_peered":"2023-11-26T07:24:09.515968+0000","last_clean":"2023-11-26T07:24:09.515968+0000","last_became_active":"2023-11-26T07:24:08.522830+0000","last_became_peered":"2023-11-26T07:24:08.522830+0000","last_unstale":"2023-11-26T07:24:09.515968+0000","last_undegraded":"2023-11-26T07:24:09.515968+0000","last_fullsized":"2023-11-26T07:24:09.515968+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:24:07.907834+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:24:07.907834+0000","last_clean_scrub_stamp":"2023-11-26T07:24:07.907834+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T15:16:38.099438+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.33","version":"0'0","reported_seq":11,"reported_epoch":103,"state":"active+clean","last_fresh":"2023-11-26T07:24:09.915693+0000","last_change":"2023-11-26T07:24:08.538612+0000","last_active":"2023-11-26T07:24:09.915693+0000","last_peered":"2023-11-26T07:24:09.915693+0000","last_clean":"2023-11-26T07:24:09.915693+0000","last_became_active":"2023-11-26T07:24:08.538269+0000","last_became_peered":"2023-11-26T07:24:08.538269+0000","last_unstale":"2023-11-26T07:24:09.915693+0000","last_undegraded":"2023-11-26T07:24:09.915693+0000","last_fullsized":"2023-11-26T07:24:09.915693+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:24:07.907834+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:24:07.907834+0000","last_clean_scrub_stamp":"2023-11-26T07:24:07.907834+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T16:28:02.738541+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"17.32","version":"0'0","reported_seq":11,"reported_epoch":103,"state":"active+clean","last_fresh":"2023-11-26T07:24:09.915596+0000","last_change":"2023-11-26T07:24:08.539043+0000","last_active":"2023-11-26T07:24:09.915596+0000","last_peered":"2023-11-26T07:24:09.915596+0000","last_clean":"2023-11-26T07:24:09.915596+0000","last_became_active":"2023-11-26T07:24:08.538199+0000","last_became_peered":"2023-11-26T07:24:08.538199+0000","last_unstale":"2023-11-26T07:24:09.915596+0000","last_undegraded":"2023-11-26T07:24:09.915596+0000","last_fullsized":"2023-11-26T07:24:09.915596+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:24:07.907834+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:24:07.907834+0000","last_clean_scrub_stamp":"2023-11-26T07:24:07.907834+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T10:24:52.392128+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"17.31","version":"0'0","reported_seq":11,"reported_epoch":103,"state":"active+clean","last_fresh":"2023-11-26T07:24:09.917747+0000","last_change":"2023-11-26T07:24:08.533254+0000","last_active":"2023-11-26T07:24:09.917747+0000","last_peered":"2023-11-26T07:24:09.917747+0000","last_clean":"2023-11-26T07:24:09.917747+0000","last_became_active":"2023-11-26T07:24:08.532721+0000","last_became_peered":"2023-11-26T07:24:08.532721+0000","last_unstale":"2023-11-26T07:24:09.917747+0000","last_undegraded":"2023-11-26T07:24:09.917747+0000","last_fullsized":"2023-11-26T07:24:09.917747+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:24:07.907834+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:24:07.907834+0000","last_clean_scrub_stamp":"2023-11-26T07:24:07.907834+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T08:49:55.531595+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"17.30","version":"0'0","reported_seq":11,"reported_epoch":103,"state":"active+clean","last_fresh":"2023-11-26T07:24:09.515328+0000","last_change":"2023-11-26T07:24:08.535469+0000","last_active":"2023-11-26T07:24:09.515328+0000","last_peered":"2023-11-26T07:24:09.515328+0000","last_clean":"2023-11-26T07:24:09.515328+0000","last_became_active":"2023-11-26T07:24:08.535154+0000","last_became_peered":"2023-11-26T07:24:08.535154+0000","last_unstale":"2023-11-26T07:24:09.515328+0000","last_undegraded":"2023-11-26T07:24:09.515328+0000","last_fullsized":"2023-11-26T07:24:09.515328+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:24:07.907834+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:24:07.907834+0000","last_clean_scrub_stamp":"2023-11-26T07:24:07.907834+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T11:29:20.672235+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.2f","version":"0'0","reported_seq":11,"reported_epoch":103,"state":"active+clean","last_fresh":"2023-11-26T07:24:09.514341+0000","last_change":"2023-11-26T07:24:08.531316+0000","last_active":"2023-11-26T07:24:09.514341+0000","last_peered":"2023-11-26T07:24:09.514341+0000","last_clean":"2023-11-26T07:24:09.514341+0000","last_became_active":"2023-11-26T07:24:08.531000+0000","last_became_peered":"2023-11-26T07:24:08.531000+0000","last_unstale":"2023-11-26T07:24:09.514341+0000","last_undegraded":"2023-11-26T07:24:09.514341+0000","last_fullsized":"2023-11-26T07:24:09.514341+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:24:07.907834+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:24:07.907834+0000","last_clean_scrub_stamp":"2023-11-26T07:24:07.907834+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T13:56:29.957903+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"17.2e","version":"0'0","reported_seq":11,"reported_epoch":103,"state":"active+clean","last_fresh":"2023-11-26T07:24:09.914911+0000","last_change":"2023-11-26T07:24:08.534757+0000","last_active":"2023-11-26T07:24:09.914911+0000","last_peered":"2023-11-26T07:24:09.914911+0000","last_clean":"2023-11-26T07:24:09.914911+0000","last_became_active":"2023-11-26T07:24:08.534462+0000","last_became_peered":"2023-11-26T07:24:08.534462+0000","last_unstale":"2023-11-26T07:24:09.914911+0000","last_undegraded":"2023-11-26T07:24:09.914911+0000","last_fullsized":"2023-11-26T07:24:09.914911+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:24:07.907834+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:24:07.907834+0000","last_clean_scrub_stamp":"2023-11-26T07:24:07.907834+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T12:59:50.984585+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"17.2d","version":"0'0","reported_seq":11,"reported_epoch":103,"state":"active+clean","last_fresh":"2023-11-26T07:24:09.915678+0000","last_change":"2023-11-26T07:24:08.537949+0000","last_active":"2023-11-26T07:24:09.915678+0000","last_peered":"2023-11-26T07:24:09.915678+0000","last_clean":"2023-11-26T07:24:09.915678+0000","last_became_active":"2023-11-26T07:24:08.536969+0000","last_became_peered":"2023-11-26T07:24:08.536969+0000","last_unstale":"2023-11-26T07:24:09.915678+0000","last_undegraded":"2023-11-26T07:24:09.915678+0000","last_fullsized":"2023-11-26T07:24:09.915678+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:24:07.907834+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:24:07.907834+0000","last_clean_scrub_stamp":"2023-11-26T07:24:07.907834+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T11:47:45.347957+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"17.2c","version":"0'0","reported_seq":11,"reported_epoch":103,"state":"active+clean","last_fresh":"2023-11-26T07:24:09.513995+0000","last_change":"2023-11-26T07:24:08.530639+0000","last_active":"2023-11-26T07:24:09.513995+0000","last_peered":"2023-11-26T07:24:09.513995+0000","last_clean":"2023-11-26T07:24:09.513995+0000","last_became_active":"2023-11-26T07:24:08.530373+0000","last_became_peered":"2023-11-26T07:24:08.530373+0000","last_unstale":"2023-11-26T07:24:09.513995+0000","last_undegraded":"2023-11-26T07:24:09.513995+0000","last_fullsized":"2023-11-26T07:24:09.513995+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:24:07.907834+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:24:07.907834+0000","last_clean_scrub_stamp":"2023-11-26T07:24:07.907834+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T18:46:11.782923+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"17.2b","version":"0'0","reported_seq":11,"reported_epoch":103,"state":"active+clean","last_fresh":"2023-11-26T07:24:09.514194+0000","last_change":"2023-11-26T07:24:08.525361+0000","last_active":"2023-11-26T07:24:09.514194+0000","last_peered":"2023-11-26T07:24:09.514194+0000","last_clean":"2023-11-26T07:24:09.514194+0000","last_became_active":"2023-11-26T07:24:08.521682+0000","last_became_peered":"2023-11-26T07:24:08.521682+0000","last_unstale":"2023-11-26T07:24:09.514194+0000","last_undegraded":"2023-11-26T07:24:09.514194+0000","last_fullsized":"2023-11-26T07:24:09.514194+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:24:07.907834+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:24:07.907834+0000","last_clean_scrub_stamp":"2023-11-26T07:24:07.907834+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T07:27:00.160209+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"17.2a","version":"0'0","reported_seq":11,"reported_epoch":103,"state":"active+clean","last_fresh":"2023-11-26T07:24:09.917509+0000","last_change":"2023-11-26T07:24:08.533333+0000","last_active":"2023-11-26T07:24:09.917509+0000","last_peered":"2023-11-26T07:24:09.917509+0000","last_clean":"2023-11-26T07:24:09.917509+0000","last_became_active":"2023-11-26T07:24:08.533057+0000","last_became_peered":"2023-11-26T07:24:08.533057+0000","last_unstale":"2023-11-26T07:24:09.917509+0000","last_undegraded":"2023-11-26T07:24:09.917509+0000","last_fullsized":"2023-11-26T07:24:09.917509+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:24:07.907834+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:24:07.907834+0000","last_clean_scrub_stamp":"2023-11-26T07:24:07.907834+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T10:12:44.059607+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"17.29","version":"0'0","reported_seq":11,"reported_epoch":103,"state":"active+clean","last_fresh":"2023-11-26T07:24:09.513680+0000","last_change":"2023-11-26T07:24:08.530827+0000","last_active":"2023-11-26T07:24:09.513680+0000","last_peered":"2023-11-26T07:24:09.513680+0000","last_clean":"2023-11-26T07:24:09.513680+0000","last_became_active":"2023-11-26T07:24:08.529926+0000","last_became_peered":"2023-11-26T07:24:08.529926+0000","last_unstale":"2023-11-26T07:24:09.513680+0000","last_undegraded":"2023-11-26T07:24:09.513680+0000","last_fullsized":"2023-11-26T07:24:09.513680+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:24:07.907834+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:24:07.907834+0000","last_clean_scrub_stamp":"2023-11-26T07:24:07.907834+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T17:18:45.850481+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"17.28","version":"0'0","reported_seq":11,"reported_epoch":103,"state":"active+clean","last_fresh":"2023-11-26T07:24:09.917012+0000","last_change":"2023-11-26T07:24:08.531407+0000","last_active":"2023-11-26T07:24:09.917012+0000","last_peered":"2023-11-26T07:24:09.917012+0000","last_clean":"2023-11-26T07:24:09.917012+0000","last_became_active":"2023-11-26T07:24:08.531124+0000","last_became_peered":"2023-11-26T07:24:08.531124+0000","last_unstale":"2023-11-26T07:24:09.917012+0000","last_undegraded":"2023-11-26T07:24:09.917012+0000","last_fullsized":"2023-11-26T07:24:09.917012+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:24:07.907834+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:24:07.907834+0000","last_clean_scrub_stamp":"2023-11-26T07:24:07.907834+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T12:59:50.984585+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"17.27","version":"0'0","reported_seq":11,"reported_epoch":103,"state":"active+clean","last_fresh":"2023-11-26T07:24:09.514381+0000","last_change":"2023-11-26T07:24:08.528786+0000","last_active":"2023-11-26T07:24:09.514381+0000","last_peered":"2023-11-26T07:24:09.514381+0000","last_clean":"2023-11-26T07:24:09.514381+0000","last_became_active":"2023-11-26T07:24:08.528462+0000","last_became_peered":"2023-11-26T07:24:08.528462+0000","last_unstale":"2023-11-26T07:24:09.514381+0000","last_undegraded":"2023-11-26T07:24:09.514381+0000","last_fullsized":"2023-11-26T07:24:09.514381+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:24:07.907834+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:24:07.907834+0000","last_clean_scrub_stamp":"2023-11-26T07:24:07.907834+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T12:31:13.988989+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"17.26","version":"0'0","reported_seq":11,"reported_epoch":103,"state":"active+clean","last_fresh":"2023-11-26T07:24:09.516258+0000","last_change":"2023-11-26T07:24:08.530307+0000","last_active":"2023-11-26T07:24:09.516258+0000","last_peered":"2023-11-26T07:24:09.516258+0000","last_clean":"2023-11-26T07:24:09.516258+0000","last_became_active":"2023-11-26T07:24:08.530049+0000","last_became_peered":"2023-11-26T07:24:08.530049+0000","last_unstale":"2023-11-26T07:24:09.516258+0000","last_undegraded":"2023-11-26T07:24:09.516258+0000","last_fullsized":"2023-11-26T07:24:09.516258+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:24:07.907834+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:24:07.907834+0000","last_clean_scrub_stamp":"2023-11-26T07:24:07.907834+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T18:37:49.588926+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.25","version":"0'0","reported_seq":11,"reported_epoch":103,"state":"active+clean","last_fresh":"2023-11-26T07:24:09.918001+0000","last_change":"2023-11-26T07:24:08.534723+0000","last_active":"2023-11-26T07:24:09.918001+0000","last_peered":"2023-11-26T07:24:09.918001+0000","last_clean":"2023-11-26T07:24:09.918001+0000","last_became_active":"2023-11-26T07:24:08.534410+0000","last_became_peered":"2023-11-26T07:24:08.534410+0000","last_unstale":"2023-11-26T07:24:09.918001+0000","last_undegraded":"2023-11-26T07:24:09.918001+0000","last_fullsized":"2023-11-26T07:24:09.918001+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:24:07.907834+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:24:07.907834+0000","last_clean_scrub_stamp":"2023-11-26T07:24:07.907834+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T11:27:29.036348+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"17.24","version":"0'0","reported_seq":11,"reported_epoch":103,"state":"active+clean","last_fresh":"2023-11-26T07:24:09.917927+0000","last_change":"2023-11-26T07:24:08.526650+0000","last_active":"2023-11-26T07:24:09.917927+0000","last_peered":"2023-11-26T07:24:09.917927+0000","last_clean":"2023-11-26T07:24:09.917927+0000","last_became_active":"2023-11-26T07:24:08.524932+0000","last_became_peered":"2023-11-26T07:24:08.524932+0000","last_unstale":"2023-11-26T07:24:09.917927+0000","last_undegraded":"2023-11-26T07:24:09.917927+0000","last_fullsized":"2023-11-26T07:24:09.917927+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:24:07.907834+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:24:07.907834+0000","last_clean_scrub_stamp":"2023-11-26T07:24:07.907834+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T10:24:52.392128+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"17.23","version":"0'0","reported_seq":11,"reported_epoch":103,"state":"active+clean","last_fresh":"2023-11-26T07:24:09.514386+0000","last_change":"2023-11-26T07:24:08.539953+0000","last_active":"2023-11-26T07:24:09.514386+0000","last_peered":"2023-11-26T07:24:09.514386+0000","last_clean":"2023-11-26T07:24:09.514386+0000","last_became_active":"2023-11-26T07:24:08.539659+0000","last_became_peered":"2023-11-26T07:24:08.539659+0000","last_unstale":"2023-11-26T07:24:09.514386+0000","last_undegraded":"2023-11-26T07:24:09.514386+0000","last_fullsized":"2023-11-26T07:24:09.514386+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:24:07.907834+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:24:07.907834+0000","last_clean_scrub_stamp":"2023-11-26T07:24:07.907834+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T16:20:10.561239+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"17.22","version":"0'0","reported_seq":11,"reported_epoch":103,"state":"active+clean","last_fresh":"2023-11-26T07:24:09.693876+0000","last_change":"2023-11-26T07:24:08.531023+0000","last_active":"2023-11-26T07:24:09.693876+0000","last_peered":"2023-11-26T07:24:09.693876+0000","last_clean":"2023-11-26T07:24:09.693876+0000","last_became_active":"2023-11-26T07:24:08.529865+0000","last_became_peered":"2023-11-26T07:24:08.529865+0000","last_unstale":"2023-11-26T07:24:09.693876+0000","last_undegraded":"2023-11-26T07:24:09.693876+0000","last_fullsized":"2023-11-26T07:24:09.693876+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:24:07.907834+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:24:07.907834+0000","last_clean_scrub_stamp":"2023-11-26T07:24:07.907834+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T11:05:48.887033+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"17.21","version":"0'0","reported_seq":11,"reported_epoch":103,"state":"active+clean","last_fresh":"2023-11-26T07:24:09.515393+0000","last_change":"2023-11-26T07:24:08.530520+0000","last_active":"2023-11-26T07:24:09.515393+0000","last_peered":"2023-11-26T07:24:09.515393+0000","last_clean":"2023-11-26T07:24:09.515393+0000","last_became_active":"2023-11-26T07:24:08.530151+0000","last_became_peered":"2023-11-26T07:24:08.530151+0000","last_unstale":"2023-11-26T07:24:09.515393+0000","last_undegraded":"2023-11-26T07:24:09.515393+0000","last_fullsized":"2023-11-26T07:24:09.515393+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:24:07.907834+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:24:07.907834+0000","last_clean_scrub_stamp":"2023-11-26T07:24:07.907834+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T11:48:28.492262+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.20","version":"0'0","reported_seq":11,"reported_epoch":103,"state":"active+clean","last_fresh":"2023-11-26T07:24:09.917252+0000","last_change":"2023-11-26T07:24:08.534264+0000","last_active":"2023-11-26T07:24:09.917252+0000","last_peered":"2023-11-26T07:24:09.917252+0000","last_clean":"2023-11-26T07:24:09.917252+0000","last_became_active":"2023-11-26T07:24:08.534008+0000","last_became_peered":"2023-11-26T07:24:08.534008+0000","last_unstale":"2023-11-26T07:24:09.917252+0000","last_undegraded":"2023-11-26T07:24:09.917252+0000","last_fullsized":"2023-11-26T07:24:09.917252+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:24:07.907834+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:24:07.907834+0000","last_clean_scrub_stamp":"2023-11-26T07:24:07.907834+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T13:02:16.849958+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"17.1e","version":"0'0","reported_seq":11,"reported_epoch":103,"state":"active+clean","last_fresh":"2023-11-26T07:24:09.918142+0000","last_change":"2023-11-26T07:24:08.537449+0000","last_active":"2023-11-26T07:24:09.918142+0000","last_peered":"2023-11-26T07:24:09.918142+0000","last_clean":"2023-11-26T07:24:09.918142+0000","last_became_active":"2023-11-26T07:24:08.536925+0000","last_became_peered":"2023-11-26T07:24:08.536925+0000","last_unstale":"2023-11-26T07:24:09.918142+0000","last_undegraded":"2023-11-26T07:24:09.918142+0000","last_fullsized":"2023-11-26T07:24:09.918142+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:24:07.907834+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:24:07.907834+0000","last_clean_scrub_stamp":"2023-11-26T07:24:07.907834+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T09:49:35.740784+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"16.1f","version":"102'4","reported_seq":16,"reported_epoch":102,"state":"active+clean","last_fresh":"2023-11-26T07:24:08.929463+0000","last_change":"2023-11-26T07:24:07.930036+0000","last_active":"2023-11-26T07:24:08.929463+0000","last_peered":"2023-11-26T07:24:08.929463+0000","last_clean":"2023-11-26T07:24:08.929463+0000","last_became_active":"2023-11-26T07:24:07.929663+0000","last_became_peered":"2023-11-26T07:24:07.929663+0000","last_unstale":"2023-11-26T07:24:08.929463+0000","last_undegraded":"2023-11-26T07:24:08.929463+0000","last_fullsized":"2023-11-26T07:24:08.929463+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":101,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:24:06.899969+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:24:06.899969+0000","last_clean_scrub_stamp":"2023-11-26T07:24:06.899969+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T07:50:34.130970+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"17.1f","version":"0'0","reported_seq":11,"reported_epoch":103,"state":"active+clean","last_fresh":"2023-11-26T07:24:09.915164+0000","last_change":"2023-11-26T07:24:08.539476+0000","last_active":"2023-11-26T07:24:09.915164+0000","last_peered":"2023-11-26T07:24:09.915164+0000","last_clean":"2023-11-26T07:24:09.915164+0000","last_became_active":"2023-11-26T07:24:08.538709+0000","last_became_peered":"2023-11-26T07:24:08.538709+0000","last_unstale":"2023-11-26T07:24:09.915164+0000","last_undegraded":"2023-11-26T07:24:09.915164+0000","last_fullsized":"2023-11-26T07:24:09.915164+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:24:07.907834+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:24:07.907834+0000","last_clean_scrub_stamp":"2023-11-26T07:24:07.907834+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T19:14:15.125936+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.1e","version":"102'1","reported_seq":13,"reported_epoch":102,"state":"active+clean","last_fresh":"2023-11-26T07:24:08.927905+0000","last_change":"2023-11-26T07:24:07.930342+0000","last_active":"2023-11-26T07:24:08.927905+0000","last_peered":"2023-11-26T07:24:08.927905+0000","last_clean":"2023-11-26T07:24:08.927905+0000","last_became_active":"2023-11-26T07:24:07.930060+0000","last_became_peered":"2023-11-26T07:24:07.930060+0000","last_unstale":"2023-11-26T07:24:08.927905+0000","last_undegraded":"2023-11-26T07:24:08.927905+0000","last_fullsized":"2023-11-26T07:24:08.927905+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":101,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:24:06.899969+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:24:06.899969+0000","last_clean_scrub_stamp":"2023-11-26T07:24:06.899969+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T12:31:12.981123+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"17.1c","version":"0'0","reported_seq":11,"reported_epoch":103,"state":"active+clean","last_fresh":"2023-11-26T07:24:09.918043+0000","last_change":"2023-11-26T07:24:08.534692+0000","last_active":"2023-11-26T07:24:09.918043+0000","last_peered":"2023-11-26T07:24:09.918043+0000","last_clean":"2023-11-26T07:24:09.918043+0000","last_became_active":"2023-11-26T07:24:08.532626+0000","last_became_peered":"2023-11-26T07:24:08.532626+0000","last_unstale":"2023-11-26T07:24:09.918043+0000","last_undegraded":"2023-11-26T07:24:09.918043+0000","last_fullsized":"2023-11-26T07:24:09.918043+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:24:07.907834+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:24:07.907834+0000","last_clean_scrub_stamp":"2023-11-26T07:24:07.907834+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T11:31:10.898875+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"16.1d","version":"102'2","reported_seq":14,"reported_epoch":102,"state":"active+clean","last_fresh":"2023-11-26T07:24:08.936674+0000","last_change":"2023-11-26T07:24:07.946117+0000","last_active":"2023-11-26T07:24:08.936674+0000","last_peered":"2023-11-26T07:24:08.936674+0000","last_clean":"2023-11-26T07:24:08.936674+0000","last_became_active":"2023-11-26T07:24:07.945830+0000","last_became_peered":"2023-11-26T07:24:07.945830+0000","last_unstale":"2023-11-26T07:24:08.936674+0000","last_undegraded":"2023-11-26T07:24:08.936674+0000","last_fullsized":"2023-11-26T07:24:08.936674+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":101,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:24:06.899969+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:24:06.899969+0000","last_clean_scrub_stamp":"2023-11-26T07:24:06.899969+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T16:35:35.364264+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"17.1d","version":"0'0","reported_seq":11,"reported_epoch":103,"state":"active+clean","last_fresh":"2023-11-26T07:24:09.915787+0000","last_change":"2023-11-26T07:24:08.537514+0000","last_active":"2023-11-26T07:24:09.915787+0000","last_peered":"2023-11-26T07:24:09.915787+0000","last_clean":"2023-11-26T07:24:09.915787+0000","last_became_active":"2023-11-26T07:24:08.536903+0000","last_became_peered":"2023-11-26T07:24:08.536903+0000","last_unstale":"2023-11-26T07:24:09.915787+0000","last_undegraded":"2023-11-26T07:24:09.915787+0000","last_fullsized":"2023-11-26T07:24:09.915787+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:24:07.907834+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:24:07.907834+0000","last_clean_scrub_stamp":"2023-11-26T07:24:07.907834+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T15:32:24.814118+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"16.1c","version":"102'2","reported_seq":14,"reported_epoch":102,"state":"active+clean","last_fresh":"2023-11-26T07:24:08.936621+0000","last_change":"2023-11-26T07:24:07.947096+0000","last_active":"2023-11-26T07:24:08.936621+0000","last_peered":"2023-11-26T07:24:08.936621+0000","last_clean":"2023-11-26T07:24:08.936621+0000","last_became_active":"2023-11-26T07:24:07.946534+0000","last_became_peered":"2023-11-26T07:24:07.946534+0000","last_unstale":"2023-11-26T07:24:08.936621+0000","last_undegraded":"2023-11-26T07:24:08.936621+0000","last_fullsized":"2023-11-26T07:24:08.936621+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":101,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:24:06.899969+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:24:06.899969+0000","last_clean_scrub_stamp":"2023-11-26T07:24:06.899969+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T15:43:31.139949+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"17.1a","version":"0'0","reported_seq":11,"reported_epoch":103,"state":"active+clean","last_fresh":"2023-11-26T07:24:09.515587+0000","last_change":"2023-11-26T07:24:08.526317+0000","last_active":"2023-11-26T07:24:09.515587+0000","last_peered":"2023-11-26T07:24:09.515587+0000","last_clean":"2023-11-26T07:24:09.515587+0000","last_became_active":"2023-11-26T07:24:08.525452+0000","last_became_peered":"2023-11-26T07:24:08.525452+0000","last_unstale":"2023-11-26T07:24:09.515587+0000","last_undegraded":"2023-11-26T07:24:09.515587+0000","last_fullsized":"2023-11-26T07:24:09.515587+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:24:07.907834+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:24:07.907834+0000","last_clean_scrub_stamp":"2023-11-26T07:24:07.907834+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T12:48:28.952839+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"16.1b","version":"102'1","reported_seq":13,"reported_epoch":102,"state":"active+clean","last_fresh":"2023-11-26T07:24:08.933236+0000","last_change":"2023-11-26T07:24:07.937625+0000","last_active":"2023-11-26T07:24:08.933236+0000","last_peered":"2023-11-26T07:24:08.933236+0000","last_clean":"2023-11-26T07:24:08.933236+0000","last_became_active":"2023-11-26T07:24:07.937266+0000","last_became_peered":"2023-11-26T07:24:07.937266+0000","last_unstale":"2023-11-26T07:24:08.933236+0000","last_undegraded":"2023-11-26T07:24:08.933236+0000","last_fullsized":"2023-11-26T07:24:08.933236+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":101,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:24:06.899969+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:24:06.899969+0000","last_clean_scrub_stamp":"2023-11-26T07:24:06.899969+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T07:36:15.232490+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"17.1b","version":"0'0","reported_seq":11,"reported_epoch":103,"state":"active+clean","last_fresh":"2023-11-26T07:24:09.513949+0000","last_change":"2023-11-26T07:24:08.527270+0000","last_active":"2023-11-26T07:24:09.513949+0000","last_peered":"2023-11-26T07:24:09.513949+0000","last_clean":"2023-11-26T07:24:09.513949+0000","last_became_active":"2023-11-26T07:24:08.526965+0000","last_became_peered":"2023-11-26T07:24:08.526965+0000","last_unstale":"2023-11-26T07:24:09.513949+0000","last_undegraded":"2023-11-26T07:24:09.513949+0000","last_fullsized":"2023-11-26T07:24:09.513949+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:24:07.907834+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:24:07.907834+0000","last_clean_scrub_stamp":"2023-11-26T07:24:07.907834+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T07:57:11.974824+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"16.1a","version":"102'1","reported_seq":13,"reported_epoch":102,"state":"active+clean","last_fresh":"2023-11-26T07:24:08.931110+0000","last_change":"2023-11-26T07:24:07.938011+0000","last_active":"2023-11-26T07:24:08.931110+0000","last_peered":"2023-11-26T07:24:08.931110+0000","last_clean":"2023-11-26T07:24:08.931110+0000","last_became_active":"2023-11-26T07:24:07.937746+0000","last_became_peered":"2023-11-26T07:24:07.937746+0000","last_unstale":"2023-11-26T07:24:08.931110+0000","last_undegraded":"2023-11-26T07:24:08.931110+0000","last_fullsized":"2023-11-26T07:24:08.931110+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":101,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:24:06.899969+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:24:06.899969+0000","last_clean_scrub_stamp":"2023-11-26T07:24:06.899969+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T14:12:26.277796+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"17.18","version":"0'0","reported_seq":11,"reported_epoch":103,"state":"active+clean","last_fresh":"2023-11-26T07:24:09.515328+0000","last_change":"2023-11-26T07:24:08.529637+0000","last_active":"2023-11-26T07:24:09.515328+0000","last_peered":"2023-11-26T07:24:09.515328+0000","last_clean":"2023-11-26T07:24:09.515328+0000","last_became_active":"2023-11-26T07:24:08.529380+0000","last_became_peered":"2023-11-26T07:24:08.529380+0000","last_unstale":"2023-11-26T07:24:09.515328+0000","last_undegraded":"2023-11-26T07:24:09.515328+0000","last_fullsized":"2023-11-26T07:24:09.515328+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:24:07.907834+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:24:07.907834+0000","last_clean_scrub_stamp":"2023-11-26T07:24:07.907834+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T14:03:13.929580+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"16.19","version":"0'0","reported_seq":12,"reported_epoch":102,"state":"active+clean","last_fresh":"2023-11-26T07:24:08.514510+0000","last_change":"2023-11-26T07:24:07.941160+0000","last_active":"2023-11-26T07:24:08.514510+0000","last_peered":"2023-11-26T07:24:08.514510+0000","last_clean":"2023-11-26T07:24:08.514510+0000","last_became_active":"2023-11-26T07:24:07.940790+0000","last_became_peered":"2023-11-26T07:24:07.940790+0000","last_unstale":"2023-11-26T07:24:08.514510+0000","last_undegraded":"2023-11-26T07:24:08.514510+0000","last_fullsized":"2023-11-26T07:24:08.514510+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":101,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:24:06.899969+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:24:06.899969+0000","last_clean_scrub_stamp":"2023-11-26T07:24:06.899969+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T16:42:25.662381+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"17.19","version":"0'0","reported_seq":11,"reported_epoch":103,"state":"active+clean","last_fresh":"2023-11-26T07:24:09.693663+0000","last_change":"2023-11-26T07:24:08.530015+0000","last_active":"2023-11-26T07:24:09.693663+0000","last_peered":"2023-11-26T07:24:09.693663+0000","last_clean":"2023-11-26T07:24:09.693663+0000","last_became_active":"2023-11-26T07:24:08.529364+0000","last_became_peered":"2023-11-26T07:24:08.529364+0000","last_unstale":"2023-11-26T07:24:09.693663+0000","last_undegraded":"2023-11-26T07:24:09.693663+0000","last_fullsized":"2023-11-26T07:24:09.693663+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:24:07.907834+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:24:07.907834+0000","last_clean_scrub_stamp":"2023-11-26T07:24:07.907834+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T13:47:07.749848+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"16.18","version":"0'0","reported_seq":12,"reported_epoch":102,"state":"active+clean","last_fresh":"2023-11-26T07:24:08.513395+0000","last_change":"2023-11-26T07:24:07.936118+0000","last_active":"2023-11-26T07:24:08.513395+0000","last_peered":"2023-11-26T07:24:08.513395+0000","last_clean":"2023-11-26T07:24:08.513395+0000","last_became_active":"2023-11-26T07:24:07.935858+0000","last_became_peered":"2023-11-26T07:24:07.935858+0000","last_unstale":"2023-11-26T07:24:08.513395+0000","last_undegraded":"2023-11-26T07:24:08.513395+0000","last_fullsized":"2023-11-26T07:24:08.513395+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":101,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:24:06.899969+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:24:06.899969+0000","last_clean_scrub_stamp":"2023-11-26T07:24:06.899969+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T15:32:23.806252+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.16","version":"0'0","reported_seq":11,"reported_epoch":103,"state":"active+clean","last_fresh":"2023-11-26T07:24:09.694091+0000","last_change":"2023-11-26T07:24:08.530606+0000","last_active":"2023-11-26T07:24:09.694091+0000","last_peered":"2023-11-26T07:24:09.694091+0000","last_clean":"2023-11-26T07:24:09.694091+0000","last_became_active":"2023-11-26T07:24:08.530278+0000","last_became_peered":"2023-11-26T07:24:08.530278+0000","last_unstale":"2023-11-26T07:24:09.694091+0000","last_undegraded":"2023-11-26T07:24:09.694091+0000","last_fullsized":"2023-11-26T07:24:09.694091+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:24:07.907834+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:24:07.907834+0000","last_clean_scrub_stamp":"2023-11-26T07:24:07.907834+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T12:28:52.949098+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"16.17","version":"0'0","reported_seq":12,"reported_epoch":102,"state":"active+clean","last_fresh":"2023-11-26T07:24:08.520720+0000","last_change":"2023-11-26T07:24:07.941674+0000","last_active":"2023-11-26T07:24:08.520720+0000","last_peered":"2023-11-26T07:24:08.520720+0000","last_clean":"2023-11-26T07:24:08.520720+0000","last_became_active":"2023-11-26T07:24:07.941393+0000","last_became_peered":"2023-11-26T07:24:07.941393+0000","last_unstale":"2023-11-26T07:24:08.520720+0000","last_undegraded":"2023-11-26T07:24:08.520720+0000","last_fullsized":"2023-11-26T07:24:08.520720+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":101,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:24:06.899969+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:24:06.899969+0000","last_clean_scrub_stamp":"2023-11-26T07:24:06.899969+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T13:34:58.409461+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"17.17","version":"0'0","reported_seq":11,"reported_epoch":103,"state":"active+clean","last_fresh":"2023-11-26T07:24:09.915389+0000","last_change":"2023-11-26T07:24:08.531297+0000","last_active":"2023-11-26T07:24:09.915389+0000","last_peered":"2023-11-26T07:24:09.915389+0000","last_clean":"2023-11-26T07:24:09.915389+0000","last_became_active":"2023-11-26T07:24:08.530999+0000","last_became_peered":"2023-11-26T07:24:08.530999+0000","last_unstale":"2023-11-26T07:24:09.915389+0000","last_undegraded":"2023-11-26T07:24:09.915389+0000","last_fullsized":"2023-11-26T07:24:09.915389+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:24:07.907834+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:24:07.907834+0000","last_clean_scrub_stamp":"2023-11-26T07:24:07.907834+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T10:12:44.059607+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.16","version":"0'0","reported_seq":12,"reported_epoch":102,"state":"active+clean","last_fresh":"2023-11-26T07:24:08.519223+0000","last_change":"2023-11-26T07:24:07.936539+0000","last_active":"2023-11-26T07:24:08.519223+0000","last_peered":"2023-11-26T07:24:08.519223+0000","last_clean":"2023-11-26T07:24:08.519223+0000","last_became_active":"2023-11-26T07:24:07.936274+0000","last_became_peered":"2023-11-26T07:24:07.936274+0000","last_unstale":"2023-11-26T07:24:08.519223+0000","last_undegraded":"2023-11-26T07:24:08.519223+0000","last_fullsized":"2023-11-26T07:24:08.519223+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":101,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:24:06.899969+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:24:06.899969+0000","last_clean_scrub_stamp":"2023-11-26T07:24:06.899969+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T13:56:28.950037+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"17.14","version":"0'0","reported_seq":11,"reported_epoch":103,"state":"active+clean","last_fresh":"2023-11-26T07:24:09.918044+0000","last_change":"2023-11-26T07:24:08.526674+0000","last_active":"2023-11-26T07:24:09.918044+0000","last_peered":"2023-11-26T07:24:09.918044+0000","last_clean":"2023-11-26T07:24:09.918044+0000","last_became_active":"2023-11-26T07:24:08.525139+0000","last_became_peered":"2023-11-26T07:24:08.525139+0000","last_unstale":"2023-11-26T07:24:09.918044+0000","last_undegraded":"2023-11-26T07:24:09.918044+0000","last_fullsized":"2023-11-26T07:24:09.918044+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:24:07.907834+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:24:07.907834+0000","last_clean_scrub_stamp":"2023-11-26T07:24:07.907834+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T11:47:45.347957+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"16.15","version":"102'3","reported_seq":15,"reported_epoch":102,"state":"active+clean","last_fresh":"2023-11-26T07:24:08.937366+0000","last_change":"2023-11-26T07:24:07.940674+0000","last_active":"2023-11-26T07:24:08.937366+0000","last_peered":"2023-11-26T07:24:08.937366+0000","last_clean":"2023-11-26T07:24:08.937366+0000","last_became_active":"2023-11-26T07:24:07.940367+0000","last_became_peered":"2023-11-26T07:24:07.940367+0000","last_unstale":"2023-11-26T07:24:08.937366+0000","last_undegraded":"2023-11-26T07:24:08.937366+0000","last_fullsized":"2023-11-26T07:24:08.937366+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":101,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:24:06.899969+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:24:06.899969+0000","last_clean_scrub_stamp":"2023-11-26T07:24:06.899969+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T08:09:42.709854+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.15","version":"0'0","reported_seq":11,"reported_epoch":103,"state":"active+clean","last_fresh":"2023-11-26T07:24:09.694023+0000","last_change":"2023-11-26T07:24:08.536697+0000","last_active":"2023-11-26T07:24:09.694023+0000","last_peered":"2023-11-26T07:24:09.694023+0000","last_clean":"2023-11-26T07:24:09.694023+0000","last_became_active":"2023-11-26T07:24:08.534523+0000","last_became_peered":"2023-11-26T07:24:08.534523+0000","last_unstale":"2023-11-26T07:24:09.694023+0000","last_undegraded":"2023-11-26T07:24:09.694023+0000","last_fullsized":"2023-11-26T07:24:09.694023+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:24:07.907834+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:24:07.907834+0000","last_clean_scrub_stamp":"2023-11-26T07:24:07.907834+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T12:34:46.772607+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"16.14","version":"102'6","reported_seq":18,"reported_epoch":102,"state":"active+clean","last_fresh":"2023-11-26T07:24:08.938305+0000","last_change":"2023-11-26T07:24:07.935288+0000","last_active":"2023-11-26T07:24:08.938305+0000","last_peered":"2023-11-26T07:24:08.938305+0000","last_clean":"2023-11-26T07:24:08.938305+0000","last_became_active":"2023-11-26T07:24:07.935024+0000","last_became_peered":"2023-11-26T07:24:07.935024+0000","last_unstale":"2023-11-26T07:24:08.938305+0000","last_undegraded":"2023-11-26T07:24:08.938305+0000","last_fullsized":"2023-11-26T07:24:08.938305+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":101,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:24:06.899969+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:24:06.899969+0000","last_clean_scrub_stamp":"2023-11-26T07:24:06.899969+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T14:07:47.462732+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.12","version":"0'0","reported_seq":11,"reported_epoch":103,"state":"active+clean","last_fresh":"2023-11-26T07:24:09.917643+0000","last_change":"2023-11-26T07:24:08.533478+0000","last_active":"2023-11-26T07:24:09.917643+0000","last_peered":"2023-11-26T07:24:09.917643+0000","last_clean":"2023-11-26T07:24:09.917643+0000","last_became_active":"2023-11-26T07:24:08.533198+0000","last_became_peered":"2023-11-26T07:24:08.533198+0000","last_unstale":"2023-11-26T07:24:09.917643+0000","last_undegraded":"2023-11-26T07:24:09.917643+0000","last_fullsized":"2023-11-26T07:24:09.917643+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:24:07.907834+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:24:07.907834+0000","last_clean_scrub_stamp":"2023-11-26T07:24:07.907834+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T16:51:06.441296+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"16.13","version":"102'2","reported_seq":14,"reported_epoch":102,"state":"active+clean","last_fresh":"2023-11-26T07:24:08.935751+0000","last_change":"2023-11-26T07:24:07.938716+0000","last_active":"2023-11-26T07:24:08.935751+0000","last_peered":"2023-11-26T07:24:08.935751+0000","last_clean":"2023-11-26T07:24:08.935751+0000","last_became_active":"2023-11-26T07:24:07.938402+0000","last_became_peered":"2023-11-26T07:24:07.938402+0000","last_unstale":"2023-11-26T07:24:08.935751+0000","last_undegraded":"2023-11-26T07:24:08.935751+0000","last_fullsized":"2023-11-26T07:24:08.935751+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":101,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:24:06.899969+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:24:06.899969+0000","last_clean_scrub_stamp":"2023-11-26T07:24:06.899969+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T16:17:17.300998+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"17.13","version":"0'0","reported_seq":11,"reported_epoch":103,"state":"active+clean","last_fresh":"2023-11-26T07:24:09.917455+0000","last_change":"2023-11-26T07:24:08.530960+0000","last_active":"2023-11-26T07:24:09.917455+0000","last_peered":"2023-11-26T07:24:09.917455+0000","last_clean":"2023-11-26T07:24:09.917455+0000","last_became_active":"2023-11-26T07:24:08.530690+0000","last_became_peered":"2023-11-26T07:24:08.530690+0000","last_unstale":"2023-11-26T07:24:09.917455+0000","last_undegraded":"2023-11-26T07:24:09.917455+0000","last_fullsized":"2023-11-26T07:24:09.917455+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:24:07.907834+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:24:07.907834+0000","last_clean_scrub_stamp":"2023-11-26T07:24:07.907834+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T14:25:38.608346+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"16.12","version":"0'0","reported_seq":12,"reported_epoch":102,"state":"active+clean","last_fresh":"2023-11-26T07:24:08.524678+0000","last_change":"2023-11-26T07:24:07.932355+0000","last_active":"2023-11-26T07:24:08.524678+0000","last_peered":"2023-11-26T07:24:08.524678+0000","last_clean":"2023-11-26T07:24:08.524678+0000","last_became_active":"2023-11-26T07:24:07.932002+0000","last_became_peered":"2023-11-26T07:24:07.932002+0000","last_unstale":"2023-11-26T07:24:08.524678+0000","last_undegraded":"2023-11-26T07:24:08.524678+0000","last_fullsized":"2023-11-26T07:24:08.524678+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":101,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:24:06.899969+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:24:06.899969+0000","last_clean_scrub_stamp":"2023-11-26T07:24:06.899969+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T16:20:09.553373+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"17.11","version":"0'0","reported_seq":11,"reported_epoch":103,"state":"active+clean","last_fresh":"2023-11-26T07:24:09.917389+0000","last_change":"2023-11-26T07:24:08.526100+0000","last_active":"2023-11-26T07:24:09.917389+0000","last_peered":"2023-11-26T07:24:09.917389+0000","last_clean":"2023-11-26T07:24:09.917389+0000","last_became_active":"2023-11-26T07:24:08.525789+0000","last_became_peered":"2023-11-26T07:24:08.525789+0000","last_unstale":"2023-11-26T07:24:09.917389+0000","last_undegraded":"2023-11-26T07:24:09.917389+0000","last_fullsized":"2023-11-26T07:24:09.917389+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:24:07.907834+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:24:07.907834+0000","last_clean_scrub_stamp":"2023-11-26T07:24:07.907834+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T19:14:15.125936+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"16.10","version":"102'8","reported_seq":20,"reported_epoch":102,"state":"active+clean","last_fresh":"2023-11-26T07:24:08.936437+0000","last_change":"2023-11-26T07:24:07.934907+0000","last_active":"2023-11-26T07:24:08.936437+0000","last_peered":"2023-11-26T07:24:08.936437+0000","last_clean":"2023-11-26T07:24:08.936437+0000","last_became_active":"2023-11-26T07:24:07.934627+0000","last_became_peered":"2023-11-26T07:24:07.934627+0000","last_unstale":"2023-11-26T07:24:08.936437+0000","last_undegraded":"2023-11-26T07:24:08.936437+0000","last_fullsized":"2023-11-26T07:24:08.936437+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":101,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:24:06.899969+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:24:06.899969+0000","last_clean_scrub_stamp":"2023-11-26T07:24:06.899969+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T17:49:19.086359+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"17.e","version":"0'0","reported_seq":11,"reported_epoch":103,"state":"active+clean","last_fresh":"2023-11-26T07:24:09.918898+0000","last_change":"2023-11-26T07:24:08.529551+0000","last_active":"2023-11-26T07:24:09.918898+0000","last_peered":"2023-11-26T07:24:09.918898+0000","last_clean":"2023-11-26T07:24:09.918898+0000","last_became_active":"2023-11-26T07:24:08.529268+0000","last_became_peered":"2023-11-26T07:24:08.529268+0000","last_unstale":"2023-11-26T07:24:09.918898+0000","last_undegraded":"2023-11-26T07:24:09.918898+0000","last_fullsized":"2023-11-26T07:24:09.918898+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:24:07.907834+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:24:07.907834+0000","last_clean_scrub_stamp":"2023-11-26T07:24:07.907834+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T18:11:46.930613+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"16.f","version":"102'1","reported_seq":13,"reported_epoch":102,"state":"active+clean","last_fresh":"2023-11-26T07:24:08.932819+0000","last_change":"2023-11-26T07:24:07.933475+0000","last_active":"2023-11-26T07:24:08.932819+0000","last_peered":"2023-11-26T07:24:08.932819+0000","last_clean":"2023-11-26T07:24:08.932819+0000","last_became_active":"2023-11-26T07:24:07.933214+0000","last_became_peered":"2023-11-26T07:24:07.933214+0000","last_unstale":"2023-11-26T07:24:08.932819+0000","last_undegraded":"2023-11-26T07:24:08.932819+0000","last_fullsized":"2023-11-26T07:24:08.932819+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":101,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:24:06.899969+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:24:06.899969+0000","last_clean_scrub_stamp":"2023-11-26T07:24:06.899969+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T11:40:22.357576+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.f","version":"0'0","reported_seq":11,"reported_epoch":103,"state":"active+clean","last_fresh":"2023-11-26T07:24:09.918185+0000","last_change":"2023-11-26T07:24:08.533984+0000","last_active":"2023-11-26T07:24:09.918185+0000","last_peered":"2023-11-26T07:24:09.918185+0000","last_clean":"2023-11-26T07:24:09.918185+0000","last_became_active":"2023-11-26T07:24:08.533661+0000","last_became_peered":"2023-11-26T07:24:08.533661+0000","last_unstale":"2023-11-26T07:24:09.918185+0000","last_undegraded":"2023-11-26T07:24:09.918185+0000","last_fullsized":"2023-11-26T07:24:09.918185+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:24:07.907834+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:24:07.907834+0000","last_clean_scrub_stamp":"2023-11-26T07:24:07.907834+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T07:36:16.240356+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"16.e","version":"102'2","reported_seq":14,"reported_epoch":102,"state":"active+clean","last_fresh":"2023-11-26T07:24:08.933834+0000","last_change":"2023-11-26T07:24:07.939514+0000","last_active":"2023-11-26T07:24:08.933834+0000","last_peered":"2023-11-26T07:24:08.933834+0000","last_clean":"2023-11-26T07:24:08.933834+0000","last_became_active":"2023-11-26T07:24:07.939200+0000","last_became_peered":"2023-11-26T07:24:07.939200+0000","last_unstale":"2023-11-26T07:24:08.933834+0000","last_undegraded":"2023-11-26T07:24:08.933834+0000","last_fullsized":"2023-11-26T07:24:08.933834+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":101,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:24:06.899969+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:24:06.899969+0000","last_clean_scrub_stamp":"2023-11-26T07:24:06.899969+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T10:42:19.135108+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"17.c","version":"0'0","reported_seq":11,"reported_epoch":103,"state":"active+clean","last_fresh":"2023-11-26T07:24:09.915847+0000","last_change":"2023-11-26T07:24:08.536505+0000","last_active":"2023-11-26T07:24:09.915847+0000","last_peered":"2023-11-26T07:24:09.915847+0000","last_clean":"2023-11-26T07:24:09.915847+0000","last_became_active":"2023-11-26T07:24:08.536113+0000","last_became_peered":"2023-11-26T07:24:08.536113+0000","last_unstale":"2023-11-26T07:24:09.915847+0000","last_undegraded":"2023-11-26T07:24:09.915847+0000","last_fullsized":"2023-11-26T07:24:09.915847+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:24:07.907834+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:24:07.907834+0000","last_clean_scrub_stamp":"2023-11-26T07:24:07.907834+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T09:58:31.098914+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"16.d","version":"102'1","reported_seq":13,"reported_epoch":102,"state":"active+clean","last_fresh":"2023-11-26T07:24:08.926460+0000","last_change":"2023-11-26T07:24:07.936853+0000","last_active":"2023-11-26T07:24:08.926460+0000","last_peered":"2023-11-26T07:24:08.926460+0000","last_clean":"2023-11-26T07:24:08.926460+0000","last_became_active":"2023-11-26T07:24:07.936507+0000","last_became_peered":"2023-11-26T07:24:07.936507+0000","last_unstale":"2023-11-26T07:24:08.926460+0000","last_undegraded":"2023-11-26T07:24:08.926460+0000","last_fullsized":"2023-11-26T07:24:08.926460+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":101,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:24:06.899969+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:24:06.899969+0000","last_clean_scrub_stamp":"2023-11-26T07:24:06.899969+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T13:56:28.950037+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"17.d","version":"0'0","reported_seq":11,"reported_epoch":103,"state":"active+clean","last_fresh":"2023-11-26T07:24:09.915834+0000","last_change":"2023-11-26T07:24:08.538196+0000","last_active":"2023-11-26T07:24:09.915834+0000","last_peered":"2023-11-26T07:24:09.915834+0000","last_clean":"2023-11-26T07:24:09.915834+0000","last_became_active":"2023-11-26T07:24:08.537273+0000","last_became_peered":"2023-11-26T07:24:08.537273+0000","last_unstale":"2023-11-26T07:24:09.915834+0000","last_undegraded":"2023-11-26T07:24:09.915834+0000","last_fullsized":"2023-11-26T07:24:09.915834+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:24:07.907834+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:24:07.907834+0000","last_clean_scrub_stamp":"2023-11-26T07:24:07.907834+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T08:09:43.717720+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"16.c","version":"0'0","reported_seq":12,"reported_epoch":102,"state":"active+clean","last_fresh":"2023-11-26T07:24:08.518361+0000","last_change":"2023-11-26T07:24:07.929744+0000","last_active":"2023-11-26T07:24:08.518361+0000","last_peered":"2023-11-26T07:24:08.518361+0000","last_clean":"2023-11-26T07:24:08.518361+0000","last_became_active":"2023-11-26T07:24:07.929438+0000","last_became_peered":"2023-11-26T07:24:07.929438+0000","last_unstale":"2023-11-26T07:24:08.518361+0000","last_undegraded":"2023-11-26T07:24:08.518361+0000","last_fullsized":"2023-11-26T07:24:08.518361+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":101,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:24:06.899969+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:24:06.899969+0000","last_clean_scrub_stamp":"2023-11-26T07:24:06.899969+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T09:49:34.732918+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"17.a","version":"0'0","reported_seq":11,"reported_epoch":103,"state":"active+clean","last_fresh":"2023-11-26T07:24:09.915551+0000","last_change":"2023-11-26T07:24:08.537000+0000","last_active":"2023-11-26T07:24:09.915551+0000","last_peered":"2023-11-26T07:24:09.915551+0000","last_clean":"2023-11-26T07:24:09.915551+0000","last_became_active":"2023-11-26T07:24:08.536158+0000","last_became_peered":"2023-11-26T07:24:08.536158+0000","last_unstale":"2023-11-26T07:24:09.915551+0000","last_undegraded":"2023-11-26T07:24:09.915551+0000","last_fullsized":"2023-11-26T07:24:09.915551+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:24:07.907834+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:24:07.907834+0000","last_clean_scrub_stamp":"2023-11-26T07:24:07.907834+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T11:21:18.116956+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"16.b","version":"102'2","reported_seq":14,"reported_epoch":102,"state":"active+clean","last_fresh":"2023-11-26T07:24:08.935652+0000","last_change":"2023-11-26T07:24:07.942234+0000","last_active":"2023-11-26T07:24:08.935652+0000","last_peered":"2023-11-26T07:24:08.935652+0000","last_clean":"2023-11-26T07:24:08.935652+0000","last_became_active":"2023-11-26T07:24:07.941926+0000","last_became_peered":"2023-11-26T07:24:07.941926+0000","last_unstale":"2023-11-26T07:24:08.935652+0000","last_undegraded":"2023-11-26T07:24:08.935652+0000","last_fullsized":"2023-11-26T07:24:08.935652+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":101,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:24:06.899969+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:24:06.899969+0000","last_clean_scrub_stamp":"2023-11-26T07:24:06.899969+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T18:27:27.926651+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"17.b","version":"0'0","reported_seq":11,"reported_epoch":103,"state":"active+clean","last_fresh":"2023-11-26T07:24:09.515815+0000","last_change":"2023-11-26T07:24:08.533331+0000","last_active":"2023-11-26T07:24:09.515815+0000","last_peered":"2023-11-26T07:24:09.515815+0000","last_clean":"2023-11-26T07:24:09.515815+0000","last_became_active":"2023-11-26T07:24:08.533054+0000","last_became_peered":"2023-11-26T07:24:08.533054+0000","last_unstale":"2023-11-26T07:24:09.515815+0000","last_undegraded":"2023-11-26T07:24:09.515815+0000","last_fullsized":"2023-11-26T07:24:09.515815+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:24:07.907834+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:24:07.907834+0000","last_clean_scrub_stamp":"2023-11-26T07:24:07.907834+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T18:49:59.192774+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"16.a","version":"0'0","reported_seq":12,"reported_epoch":102,"state":"active+clean","last_fresh":"2023-11-26T07:24:08.518519+0000","last_change":"2023-11-26T07:24:07.946015+0000","last_active":"2023-11-26T07:24:08.518519+0000","last_peered":"2023-11-26T07:24:08.518519+0000","last_clean":"2023-11-26T07:24:08.518519+0000","last_became_active":"2023-11-26T07:24:07.945744+0000","last_became_peered":"2023-11-26T07:24:07.945744+0000","last_unstale":"2023-11-26T07:24:08.518519+0000","last_undegraded":"2023-11-26T07:24:08.518519+0000","last_fullsized":"2023-11-26T07:24:08.518519+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":101,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:24:06.899969+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:24:06.899969+0000","last_clean_scrub_stamp":"2023-11-26T07:24:06.899969+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T08:37:05.326312+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"17.8","version":"0'0","reported_seq":11,"reported_epoch":103,"state":"active+clean","last_fresh":"2023-11-26T07:24:09.915345+0000","last_change":"2023-11-26T07:24:08.537270+0000","last_active":"2023-11-26T07:24:09.915345+0000","last_peered":"2023-11-26T07:24:09.915345+0000","last_clean":"2023-11-26T07:24:09.915345+0000","last_became_active":"2023-11-26T07:24:08.536252+0000","last_became_peered":"2023-11-26T07:24:08.536252+0000","last_unstale":"2023-11-26T07:24:09.915345+0000","last_undegraded":"2023-11-26T07:24:09.915345+0000","last_fullsized":"2023-11-26T07:24:09.915345+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:24:07.907834+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:24:07.907834+0000","last_clean_scrub_stamp":"2023-11-26T07:24:07.907834+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T14:25:38.608346+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"16.9","version":"0'0","reported_seq":12,"reported_epoch":102,"state":"active+clean","last_fresh":"2023-11-26T07:24:08.525275+0000","last_change":"2023-11-26T07:24:07.942310+0000","last_active":"2023-11-26T07:24:08.525275+0000","last_peered":"2023-11-26T07:24:08.525275+0000","last_clean":"2023-11-26T07:24:08.525275+0000","last_became_active":"2023-11-26T07:24:07.941834+0000","last_became_peered":"2023-11-26T07:24:07.941834+0000","last_unstale":"2023-11-26T07:24:08.525275+0000","last_undegraded":"2023-11-26T07:24:08.525275+0000","last_fullsized":"2023-11-26T07:24:08.525275+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":101,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:24:06.899969+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:24:06.899969+0000","last_clean_scrub_stamp":"2023-11-26T07:24:06.899969+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T17:42:53.603296+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"17.9","version":"0'0","reported_seq":11,"reported_epoch":103,"state":"active+clean","last_fresh":"2023-11-26T07:24:09.917527+0000","last_change":"2023-11-26T07:24:08.526100+0000","last_active":"2023-11-26T07:24:09.917527+0000","last_peered":"2023-11-26T07:24:09.917527+0000","last_clean":"2023-11-26T07:24:09.917527+0000","last_became_active":"2023-11-26T07:24:08.525383+0000","last_became_peered":"2023-11-26T07:24:08.525383+0000","last_unstale":"2023-11-26T07:24:09.917527+0000","last_undegraded":"2023-11-26T07:24:09.917527+0000","last_fullsized":"2023-11-26T07:24:09.917527+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:24:07.907834+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:24:07.907834+0000","last_clean_scrub_stamp":"2023-11-26T07:24:07.907834+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T11:21:18.116956+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"16.8","version":"0'0","reported_seq":12,"reported_epoch":102,"state":"active+clean","last_fresh":"2023-11-26T07:24:08.513516+0000","last_change":"2023-11-26T07:24:07.938408+0000","last_active":"2023-11-26T07:24:08.513516+0000","last_peered":"2023-11-26T07:24:08.513516+0000","last_clean":"2023-11-26T07:24:08.513516+0000","last_became_active":"2023-11-26T07:24:07.937951+0000","last_became_peered":"2023-11-26T07:24:07.937951+0000","last_unstale":"2023-11-26T07:24:08.513516+0000","last_undegraded":"2023-11-26T07:24:08.513516+0000","last_fullsized":"2023-11-26T07:24:08.513516+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":101,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:24:06.899969+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:24:06.899969+0000","last_clean_scrub_stamp":"2023-11-26T07:24:06.899969+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T09:58:30.091049+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.6","version":"0'0","reported_seq":11,"reported_epoch":103,"state":"active+clean","last_fresh":"2023-11-26T07:24:09.915378+0000","last_change":"2023-11-26T07:24:08.533516+0000","last_active":"2023-11-26T07:24:09.915378+0000","last_peered":"2023-11-26T07:24:09.915378+0000","last_clean":"2023-11-26T07:24:09.915378+0000","last_became_active":"2023-11-26T07:24:08.532156+0000","last_became_peered":"2023-11-26T07:24:08.532156+0000","last_unstale":"2023-11-26T07:24:09.915378+0000","last_undegraded":"2023-11-26T07:24:09.915378+0000","last_fullsized":"2023-11-26T07:24:09.915378+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:24:07.907834+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:24:07.907834+0000","last_clean_scrub_stamp":"2023-11-26T07:24:07.907834+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T14:25:38.608346+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.7","version":"102'2","reported_seq":14,"reported_epoch":102,"state":"active+clean","last_fresh":"2023-11-26T07:24:08.936563+0000","last_change":"2023-11-26T07:24:07.934663+0000","last_active":"2023-11-26T07:24:08.936563+0000","last_peered":"2023-11-26T07:24:08.936563+0000","last_clean":"2023-11-26T07:24:08.936563+0000","last_became_active":"2023-11-26T07:24:07.934414+0000","last_became_peered":"2023-11-26T07:24:07.934414+0000","last_unstale":"2023-11-26T07:24:08.936563+0000","last_undegraded":"2023-11-26T07:24:08.936563+0000","last_fullsized":"2023-11-26T07:24:08.936563+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":101,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:24:06.899969+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:24:06.899969+0000","last_clean_scrub_stamp":"2023-11-26T07:24:06.899969+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T12:31:12.981123+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"17.7","version":"0'0","reported_seq":11,"reported_epoch":103,"state":"active+clean","last_fresh":"2023-11-26T07:24:09.516405+0000","last_change":"2023-11-26T07:24:08.521142+0000","last_active":"2023-11-26T07:24:09.516405+0000","last_peered":"2023-11-26T07:24:09.516405+0000","last_clean":"2023-11-26T07:24:09.516405+0000","last_became_active":"2023-11-26T07:24:08.520837+0000","last_became_peered":"2023-11-26T07:24:08.520837+0000","last_unstale":"2023-11-26T07:24:09.516405+0000","last_undegraded":"2023-11-26T07:24:09.516405+0000","last_fullsized":"2023-11-26T07:24:09.516405+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:24:07.907834+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:24:07.907834+0000","last_clean_scrub_stamp":"2023-11-26T07:24:07.907834+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T15:48:32.445844+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"16.6","version":"0'0","reported_seq":12,"reported_epoch":102,"state":"active+clean","last_fresh":"2023-11-26T07:24:08.524476+0000","last_change":"2023-11-26T07:24:07.931951+0000","last_active":"2023-11-26T07:24:08.524476+0000","last_peered":"2023-11-26T07:24:08.524476+0000","last_clean":"2023-11-26T07:24:08.524476+0000","last_became_active":"2023-11-26T07:24:07.931579+0000","last_became_peered":"2023-11-26T07:24:07.931579+0000","last_unstale":"2023-11-26T07:24:08.524476+0000","last_undegraded":"2023-11-26T07:24:08.524476+0000","last_fullsized":"2023-11-26T07:24:08.524476+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":101,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:24:06.899969+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:24:06.899969+0000","last_clean_scrub_stamp":"2023-11-26T07:24:06.899969+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T12:31:12.981123+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"17.4","version":"0'0","reported_seq":11,"reported_epoch":103,"state":"active+clean","last_fresh":"2023-11-26T07:24:09.516091+0000","last_change":"2023-11-26T07:24:08.534301+0000","last_active":"2023-11-26T07:24:09.516091+0000","last_peered":"2023-11-26T07:24:09.516091+0000","last_clean":"2023-11-26T07:24:09.516091+0000","last_became_active":"2023-11-26T07:24:08.534034+0000","last_became_peered":"2023-11-26T07:24:08.534034+0000","last_unstale":"2023-11-26T07:24:09.516091+0000","last_undegraded":"2023-11-26T07:24:09.516091+0000","last_fullsized":"2023-11-26T07:24:09.516091+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:24:07.907834+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:24:07.907834+0000","last_clean_scrub_stamp":"2023-11-26T07:24:07.907834+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T16:16:57.726215+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"16.5","version":"0'0","reported_seq":12,"reported_epoch":102,"state":"active+clean","last_fresh":"2023-11-26T07:24:08.513556+0000","last_change":"2023-11-26T07:24:07.937231+0000","last_active":"2023-11-26T07:24:08.513556+0000","last_peered":"2023-11-26T07:24:08.513556+0000","last_clean":"2023-11-26T07:24:08.513556+0000","last_became_active":"2023-11-26T07:24:07.936996+0000","last_became_peered":"2023-11-26T07:24:07.936996+0000","last_unstale":"2023-11-26T07:24:08.513556+0000","last_undegraded":"2023-11-26T07:24:08.513556+0000","last_fullsized":"2023-11-26T07:24:08.513556+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":101,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:24:06.899969+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:24:06.899969+0000","last_clean_scrub_stamp":"2023-11-26T07:24:06.899969+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T08:37:08.847496+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.5","version":"0'0","reported_seq":11,"reported_epoch":103,"state":"active+clean","last_fresh":"2023-11-26T07:24:09.915045+0000","last_change":"2023-11-26T07:24:08.531326+0000","last_active":"2023-11-26T07:24:09.915045+0000","last_peered":"2023-11-26T07:24:09.915045+0000","last_clean":"2023-11-26T07:24:09.915045+0000","last_became_active":"2023-11-26T07:24:08.531009+0000","last_became_peered":"2023-11-26T07:24:08.531009+0000","last_unstale":"2023-11-26T07:24:09.915045+0000","last_undegraded":"2023-11-26T07:24:09.915045+0000","last_fullsized":"2023-11-26T07:24:09.915045+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:24:07.907834+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:24:07.907834+0000","last_clean_scrub_stamp":"2023-11-26T07:24:07.907834+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T18:09:22.931113+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.4","version":"0'0","reported_seq":12,"reported_epoch":102,"state":"active+clean","last_fresh":"2023-11-26T07:24:08.519165+0000","last_change":"2023-11-26T07:24:07.937865+0000","last_active":"2023-11-26T07:24:08.519165+0000","last_peered":"2023-11-26T07:24:08.519165+0000","last_clean":"2023-11-26T07:24:08.519165+0000","last_became_active":"2023-11-26T07:24:07.937560+0000","last_became_peered":"2023-11-26T07:24:07.937560+0000","last_unstale":"2023-11-26T07:24:08.519165+0000","last_undegraded":"2023-11-26T07:24:08.519165+0000","last_fullsized":"2023-11-26T07:24:08.519165+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":101,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:24:06.899969+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:24:06.899969+0000","last_clean_scrub_stamp":"2023-11-26T07:24:06.899969+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T07:26:59.152344+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"17.2","version":"0'0","reported_seq":11,"reported_epoch":103,"state":"active+clean","last_fresh":"2023-11-26T07:24:09.513757+0000","last_change":"2023-11-26T07:24:08.526662+0000","last_active":"2023-11-26T07:24:09.513757+0000","last_peered":"2023-11-26T07:24:09.513757+0000","last_clean":"2023-11-26T07:24:09.513757+0000","last_became_active":"2023-11-26T07:24:08.526331+0000","last_became_peered":"2023-11-26T07:24:08.526331+0000","last_unstale":"2023-11-26T07:24:09.513757+0000","last_undegraded":"2023-11-26T07:24:09.513757+0000","last_fullsized":"2023-11-26T07:24:09.513757+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:24:07.907834+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:24:07.907834+0000","last_clean_scrub_stamp":"2023-11-26T07:24:07.907834+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T08:55:43.994501+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"16.3","version":"0'0","reported_seq":12,"reported_epoch":102,"state":"active+clean","last_fresh":"2023-11-26T07:24:08.518205+0000","last_change":"2023-11-26T07:24:07.935023+0000","last_active":"2023-11-26T07:24:08.518205+0000","last_peered":"2023-11-26T07:24:08.518205+0000","last_clean":"2023-11-26T07:24:08.518205+0000","last_became_active":"2023-11-26T07:24:07.934662+0000","last_became_peered":"2023-11-26T07:24:07.934662+0000","last_unstale":"2023-11-26T07:24:08.518205+0000","last_undegraded":"2023-11-26T07:24:08.518205+0000","last_fullsized":"2023-11-26T07:24:08.518205+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":101,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:24:06.899969+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:24:06.899969+0000","last_clean_scrub_stamp":"2023-11-26T07:24:06.899969+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T11:31:09.891009+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"17.3","version":"0'0","reported_seq":11,"reported_epoch":103,"state":"active+clean","last_fresh":"2023-11-26T07:24:09.514100+0000","last_change":"2023-11-26T07:24:08.520023+0000","last_active":"2023-11-26T07:24:09.514100+0000","last_peered":"2023-11-26T07:24:09.514100+0000","last_clean":"2023-11-26T07:24:09.514100+0000","last_became_active":"2023-11-26T07:24:08.519730+0000","last_became_peered":"2023-11-26T07:24:08.519730+0000","last_unstale":"2023-11-26T07:24:09.514100+0000","last_undegraded":"2023-11-26T07:24:09.514100+0000","last_fullsized":"2023-11-26T07:24:09.514100+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:24:07.907834+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:24:07.907834+0000","last_clean_scrub_stamp":"2023-11-26T07:24:07.907834+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T15:56:02.937929+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"16.2","version":"0'0","reported_seq":12,"reported_epoch":102,"state":"active+clean","last_fresh":"2023-11-26T07:24:08.514482+0000","last_change":"2023-11-26T07:24:07.943346+0000","last_active":"2023-11-26T07:24:08.514482+0000","last_peered":"2023-11-26T07:24:08.514482+0000","last_clean":"2023-11-26T07:24:08.514482+0000","last_became_active":"2023-11-26T07:24:07.943128+0000","last_became_peered":"2023-11-26T07:24:07.943128+0000","last_unstale":"2023-11-26T07:24:08.514482+0000","last_undegraded":"2023-11-26T07:24:08.514482+0000","last_fullsized":"2023-11-26T07:24:08.514482+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":101,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:24:06.899969+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:24:06.899969+0000","last_clean_scrub_stamp":"2023-11-26T07:24:06.899969+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T10:58:50.728201+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"17.0","version":"0'0","reported_seq":11,"reported_epoch":103,"state":"active+clean","last_fresh":"2023-11-26T07:24:09.915548+0000","last_change":"2023-11-26T07:24:08.537477+0000","last_active":"2023-11-26T07:24:09.915548+0000","last_peered":"2023-11-26T07:24:09.915548+0000","last_clean":"2023-11-26T07:24:09.915548+0000","last_became_active":"2023-11-26T07:24:08.536605+0000","last_became_peered":"2023-11-26T07:24:08.536605+0000","last_unstale":"2023-11-26T07:24:09.915548+0000","last_undegraded":"2023-11-26T07:24:09.915548+0000","last_fullsized":"2023-11-26T07:24:09.915548+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:24:07.907834+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:24:07.907834+0000","last_clean_scrub_stamp":"2023-11-26T07:24:07.907834+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T16:51:06.441296+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"16.1","version":"0'0","reported_seq":12,"reported_epoch":102,"state":"active+clean","last_fresh":"2023-11-26T07:24:08.519620+0000","last_change":"2023-11-26T07:24:07.935017+0000","last_active":"2023-11-26T07:24:08.519620+0000","last_peered":"2023-11-26T07:24:08.519620+0000","last_clean":"2023-11-26T07:24:08.519620+0000","last_became_active":"2023-11-26T07:24:07.934740+0000","last_became_peered":"2023-11-26T07:24:07.934740+0000","last_unstale":"2023-11-26T07:24:08.519620+0000","last_undegraded":"2023-11-26T07:24:08.519620+0000","last_fullsized":"2023-11-26T07:24:08.519620+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":101,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:24:06.899969+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:24:06.899969+0000","last_clean_scrub_stamp":"2023-11-26T07:24:06.899969+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T15:32:54.103996+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"17.1","version":"0'0","reported_seq":11,"reported_epoch":103,"state":"active+clean","last_fresh":"2023-11-26T07:24:09.515645+0000","last_change":"2023-11-26T07:24:08.529643+0000","last_active":"2023-11-26T07:24:09.515645+0000","last_peered":"2023-11-26T07:24:09.515645+0000","last_clean":"2023-11-26T07:24:09.515645+0000","last_became_active":"2023-11-26T07:24:08.529395+0000","last_became_peered":"2023-11-26T07:24:08.529395+0000","last_unstale":"2023-11-26T07:24:09.515645+0000","last_undegraded":"2023-11-26T07:24:09.515645+0000","last_fullsized":"2023-11-26T07:24:09.515645+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:24:07.907834+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:24:07.907834+0000","last_clean_scrub_stamp":"2023-11-26T07:24:07.907834+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T11:19:27.890316+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"16.0","version":"0'0","reported_seq":12,"reported_epoch":102,"state":"active+clean","last_fresh":"2023-11-26T07:24:08.513628+0000","last_change":"2023-11-26T07:24:07.938201+0000","last_active":"2023-11-26T07:24:08.513628+0000","last_peered":"2023-11-26T07:24:08.513628+0000","last_clean":"2023-11-26T07:24:08.513628+0000","last_became_active":"2023-11-26T07:24:07.937485+0000","last_became_peered":"2023-11-26T07:24:07.937485+0000","last_unstale":"2023-11-26T07:24:08.513628+0000","last_undegraded":"2023-11-26T07:24:08.513628+0000","last_fullsized":"2023-11-26T07:24:08.513628+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":101,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:24:06.899969+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:24:06.899969+0000","last_clean_scrub_stamp":"2023-11-26T07:24:06.899969+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T11:44:26.538087+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.10","version":"0'0","reported_seq":11,"reported_epoch":103,"state":"active+clean","last_fresh":"2023-11-26T07:24:09.917174+0000","last_change":"2023-11-26T07:24:08.530925+0000","last_active":"2023-11-26T07:24:09.917174+0000","last_peered":"2023-11-26T07:24:09.917174+0000","last_clean":"2023-11-26T07:24:09.917174+0000","last_became_active":"2023-11-26T07:24:08.530650+0000","last_became_peered":"2023-11-26T07:24:08.530650+0000","last_unstale":"2023-11-26T07:24:09.917174+0000","last_undegraded":"2023-11-26T07:24:09.917174+0000","last_fullsized":"2023-11-26T07:24:09.917174+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:24:07.907834+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:24:07.907834+0000","last_clean_scrub_stamp":"2023-11-26T07:24:07.907834+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T18:09:22.931113+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"16.11","version":"0'0","reported_seq":12,"reported_epoch":102,"state":"active+clean","last_fresh":"2023-11-26T07:24:08.517782+0000","last_change":"2023-11-26T07:24:07.938327+0000","last_active":"2023-11-26T07:24:08.517782+0000","last_peered":"2023-11-26T07:24:08.517782+0000","last_clean":"2023-11-26T07:24:08.517782+0000","last_became_active":"2023-11-26T07:24:07.937824+0000","last_became_peered":"2023-11-26T07:24:07.937824+0000","last_unstale":"2023-11-26T07:24:08.517782+0000","last_undegraded":"2023-11-26T07:24:08.517782+0000","last_fullsized":"2023-11-26T07:24:08.517782+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":101,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:24:06.899969+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:24:06.899969+0000","last_clean_scrub_stamp":"2023-11-26T07:24:06.899969+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T08:49:54.523730+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.0","version":"72'128","reported_seq":273,"reported_epoch":102,"state":"active+clean","last_fresh":"2023-11-26T07:24:08.522394+0000","last_change":"2023-11-26T07:17:43.827802+0000","last_active":"2023-11-26T07:24:08.522394+0000","last_peered":"2023-11-26T07:24:08.522394+0000","last_clean":"2023-11-26T07:24:08.522394+0000","last_became_active":"2023-11-26T07:17:43.827308+0000","last_became_peered":"2023-11-26T07:17:43.827308+0000","last_unstale":"2023-11-26T07:24:08.522394+0000","last_undegraded":"2023-11-26T07:24:08.522394+0000","last_fullsized":"2023-11-26T07:24:08.522394+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:17:42.160945+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:17:42.160945+0000","last_clean_scrub_stamp":"2023-11-26T07:17:42.160945+0000","objects_scrubbed":0,"log_size":128,"log_dups_size":0,"ondisk_log_size":128,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T08:17:54.167366+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":212,"num_read_kb":306,"num_write":179,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":17,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":16,"num_pg":32,"stat_sum":{"num_bytes":2410,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":38,"ondisk_log_size":38,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":212,"num_read_kb":306,"num_write":179,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":128,"ondisk_log_size":128,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574928,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":33028,"kb_used_data":1840,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94338812,"statfs":{"total":96636764160,"available":96602943488,"internally_reserved":0,"allocated":1884160,"data_stored":1153312,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":31914442},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Nov 26 07:23:58 2023","interfaces":[{"interface":"back","average":{"1min":0.68200000000000005,"5min":0.70299999999999996,"15min":0.71099999999999997},"min":{"1min":0.47299999999999998,"5min":0.42999999999999999,"15min":0.42999999999999999},"max":{"1min":1.036,"5min":1.036,"15min":1.036},"last":0.69599999999999995},{"interface":"front","average":{"1min":0.70199999999999996,"5min":0.72699999999999998,"15min":0.71499999999999997},"min":{"1min":0.42299999999999999,"5min":0.42299999999999999,"15min":0.42299999999999999},"max":{"1min":1.014,"5min":1.014,"15min":1.0980000000000001},"last":0.68200000000000005}]},{"osd":1,"last update":"Sun Nov 26 07:23:58 2023","interfaces":[{"interface":"back","average":{"1min":0.68400000000000005,"5min":0.68700000000000006,"15min":0.68300000000000005},"min":{"1min":0.45100000000000001,"5min":0.435,"15min":0.435},"max":{"1min":0.85299999999999998,"5min":1.012,"15min":1.0669999999999999},"last":0.72599999999999998},{"interface":"front","average":{"1min":0.66900000000000004,"5min":0.67700000000000005,"15min":0.78500000000000003},"min":{"1min":0.432,"5min":0.432,"15min":0.432},"max":{"1min":0.94699999999999995,"5min":0.94699999999999995,"15min":4.5309999999999997},"last":0.60499999999999998}]},{"osd":2,"last update":"Sun Nov 26 07:23:58 2023","interfaces":[{"interface":"back","average":{"1min":0.69999999999999996,"5min":0.69999999999999996,"15min":0.68400000000000005},"min":{"1min":0.56999999999999995,"5min":0.39700000000000002,"15min":0.39700000000000002},"max":{"1min":0.91600000000000004,"5min":1.0800000000000001,"15min":1.0800000000000001},"last":0.626},{"interface":"front","average":{"1min":0.68700000000000006,"5min":0.70499999999999996,"15min":0.70999999999999996},"min":{"1min":0.48999999999999999,"5min":0.40999999999999998,"15min":0.40999999999999998},"max":{"1min":0.97099999999999997,"5min":0.97699999999999998,"15min":1.1339999999999999},"last":0.58899999999999997}]},{"osd":3,"last update":"Sun Nov 26 07:23:58 2023","interfaces":[{"interface":"back","average":{"1min":0.73299999999999998,"5min":0.72799999999999998,"15min":0.70699999999999996},"min":{"1min":0.53200000000000003,"5min":0.47799999999999998,"15min":0.47799999999999998},"max":{"1min":0.99299999999999999,"5min":1.1240000000000001,"15min":1.1240000000000001},"last":0.71199999999999997},{"interface":"front","average":{"1min":0.72399999999999998,"5min":0.74299999999999999,"15min":0.73599999999999999},"min":{"1min":0.50700000000000001,"5min":0.433,"15min":0.433},"max":{"1min":0.93400000000000005,"5min":1.036,"15min":1.4490000000000001},"last":0.66700000000000004}]},{"osd":4,"last update":"Sun Nov 26 07:23:58 2023","interfaces":[{"interface":"back","average":{"1min":0.61099999999999999,"5min":0.61499999999999999,"15min":0.60499999999999998},"min":{"1min":0.48299999999999998,"5min":0.39400000000000002,"15min":0.39400000000000002},"max":{"1min":0.878,"5min":0.96299999999999997,"15min":0.96299999999999997},"last":0.36299999999999999},{"interface":"front","average":{"1min":0.65500000000000003,"5min":0.64400000000000002,"15min":0.66300000000000003},"min":{"1min":0.45800000000000002,"5min":0.35999999999999999,"15min":0.35999999999999999},"max":{"1min":0.83599999999999997,"5min":1.0209999999999999,"15min":1.5509999999999999},"last":0.438}]},{"osd":5,"last update":"Sun Nov 26 07:23:58 2023","interfaces":[{"interface":"back","average":{"1min":0.66100000000000003,"5min":0.67400000000000004,"15min":0.66800000000000004},"min":{"1min":0.36599999999999999,"5min":0.36599999999999999,"15min":0.36599999999999999},"max":{"1min":0.86799999999999999,"5min":1.056,"15min":1.415},"last":0.52700000000000002},{"interface":"front","average":{"1min":0.68300000000000005,"5min":0.68200000000000005,"15min":0.70299999999999996},"min":{"1min":0.435,"5min":0.435,"15min":0.435},"max":{"1min":0.873,"5min":1.0840000000000001,"15min":1.3859999999999999},"last":0.48399999999999999}]},{"osd":6,"last update":"Sun Nov 26 07:23:58 2023","interfaces":[{"interface":"back","average":{"1min":0.77300000000000002,"5min":0.78000000000000003,"15min":0.77400000000000002},"min":{"1min":0.53100000000000003,"5min":0.51700000000000002,"15min":0.51700000000000002},"max":{"1min":0.95499999999999996,"5min":1.123,"15min":1.3069999999999999},"last":0.65000000000000002},{"interface":"front","average":{"1min":0.71399999999999997,"5min":0.74399999999999999,"15min":0.76200000000000001},"min":{"1min":0.50600000000000001,"5min":0.46899999999999997,"15min":0.46899999999999997},"max":{"1min":0.92900000000000005,"5min":1.036,"15min":1.6759999999999999},"last":0.55500000000000005}]}]},{"osd":6,"up_from":13,"seq":55834574930,"num_pgs":3,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32444,"kb_used_data":1256,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339396,"statfs":{"total":96636764160,"available":96603541504,"internally_reserved":0,"allocated":1286144,"data_stored":562398,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Nov 26 07:23:56 2023","interfaces":[{"interface":"back","average":{"1min":0.65400000000000003,"5min":0.67800000000000005,"15min":0.68200000000000005},"min":{"1min":0.48999999999999999,"5min":0.32200000000000001,"15min":0.32200000000000001},"max":{"1min":0.875,"5min":0.91800000000000004,"15min":1.7549999999999999},"last":0.56299999999999994},{"interface":"front","average":{"1min":0.65500000000000003,"5min":0.69599999999999995,"15min":0.87},"min":{"1min":0.36299999999999999,"5min":0.36299999999999999,"15min":0.36299999999999999},"max":{"1min":0.82599999999999996,"5min":0.96199999999999997,"15min":5.7489999999999997},"last":0.61699999999999999}]},{"osd":1,"last update":"Sun Nov 26 07:23:56 2023","interfaces":[{"interface":"back","average":{"1min":0.65000000000000002,"5min":0.70799999999999996,"15min":0.84499999999999997},"min":{"1min":0.54200000000000004,"5min":0.41199999999999998,"15min":0.41199999999999998},"max":{"1min":0.78900000000000003,"5min":0.97799999999999998,"15min":5.4240000000000004},"last":0.67700000000000005},{"interface":"front","average":{"1min":0.68200000000000005,"5min":0.70799999999999996,"15min":0.83399999999999996},"min":{"1min":0.47599999999999998,"5min":0.47599999999999998,"15min":0.47599999999999998},"max":{"1min":0.88600000000000001,"5min":0.96499999999999997,"15min":5.6020000000000003},"last":0.69899999999999995}]},{"osd":2,"last update":"Sun Nov 26 07:23:56 2023","interfaces":[{"interface":"back","average":{"1min":0.623,"5min":0.66600000000000004,"15min":0.68600000000000005},"min":{"1min":0.47999999999999998,"5min":0.41699999999999998,"15min":0.41699999999999998},"max":{"1min":0.89700000000000002,"5min":0.93600000000000005,"15min":1.8520000000000001},"last":0.499},{"interface":"front","average":{"1min":0.71099999999999997,"5min":0.72199999999999998,"15min":0.72999999999999998},"min":{"1min":0.47999999999999998,"5min":0.437,"15min":0.437},"max":{"1min":0.91000000000000003,"5min":0.95199999999999996,"15min":1.9159999999999999},"last":0.63100000000000001}]},{"osd":3,"last update":"Sun Nov 26 07:23:56 2023","interfaces":[{"interface":"back","average":{"1min":0.67900000000000005,"5min":0.70999999999999996,"15min":0.83799999999999997},"min":{"1min":0.47399999999999998,"5min":0.46899999999999997,"15min":0.46899999999999997},"max":{"1min":0.85199999999999998,"5min":0.91500000000000004,"15min":5.4390000000000001},"last":0.57999999999999996},{"interface":"front","average":{"1min":0.65000000000000002,"5min":0.70399999999999996,"15min":0.86399999999999999},"min":{"1min":0.44,"5min":0.44,"15min":0.44},"max":{"1min":0.81299999999999994,"5min":0.90500000000000003,"15min":5.5490000000000004},"last":0.44600000000000001}]},{"osd":4,"last update":"Sun Nov 26 07:23:56 2023","interfaces":[{"interface":"back","average":{"1min":0.61699999999999999,"5min":0.61199999999999999,"15min":0.63200000000000001},"min":{"1min":0.45700000000000002,"5min":0.371,"15min":0.371},"max":{"1min":0.86699999999999999,"5min":1.0129999999999999,"15min":1.0129999999999999},"last":0.71599999999999997},{"interface":"front","average":{"1min":0.623,"5min":0.63400000000000001,"15min":0.66100000000000003},"min":{"1min":0.41299999999999998,"5min":0.38600000000000001,"15min":0.38600000000000001},"max":{"1min":0.85899999999999999,"5min":1.0349999999999999,"15min":1.0349999999999999},"last":0.52400000000000002}]},{"osd":5,"last update":"Sun Nov 26 07:23:56 2023","interfaces":[{"interface":"back","average":{"1min":0.57999999999999996,"5min":0.61899999999999999,"15min":0.64500000000000002},"min":{"1min":0.42799999999999999,"5min":0.38200000000000001,"15min":0.38200000000000001},"max":{"1min":0.85599999999999998,"5min":0.91000000000000003,"15min":0.91000000000000003},"last":0.60299999999999998},{"interface":"front","average":{"1min":0.66100000000000003,"5min":0.68700000000000006,"15min":0.70399999999999996},"min":{"1min":0.501,"5min":0.44,"15min":0.436},"max":{"1min":0.91600000000000004,"5min":1.0960000000000001,"15min":1.0960000000000001},"last":0.47299999999999998}]},{"osd":7,"last update":"Sun Nov 26 07:23:56 2023","interfaces":[{"interface":"back","average":{"1min":0.66000000000000003,"5min":0.69999999999999996,"15min":0.72799999999999998},"min":{"1min":0.497,"5min":0.42599999999999999,"15min":0.42599999999999999},"max":{"1min":0.89400000000000002,"5min":0.95099999999999996,"15min":0.998},"last":0.65500000000000003},{"interface":"front","average":{"1min":0.64800000000000002,"5min":0.68300000000000005,"15min":0.70499999999999996},"min":{"1min":0.55200000000000005,"5min":0.46400000000000002,"15min":0.46400000000000002},"max":{"1min":0.77500000000000002,"5min":0.98799999999999999,"15min":0.98799999999999999},"last":0.54400000000000004}]}]},{"osd":5,"up_from":13,"seq":55834574928,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32448,"kb_used_data":1260,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339392,"statfs":{"total":96636764160,"available":96603537408,"internally_reserved":0,"allocated":1290240,"data_stored":562932,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Nov 26 07:23:49 2023","interfaces":[{"interface":"back","average":{"1min":0.69099999999999995,"5min":0.71999999999999997,"15min":0.68400000000000005},"min":{"1min":0.46899999999999997,"5min":0.36499999999999999,"15min":0.36499999999999999},"max":{"1min":0.876,"5min":2.6949999999999998,"15min":2.6949999999999998},"last":0.80800000000000005},{"interface":"front","average":{"1min":0.72599999999999998,"5min":0.73099999999999998,"15min":0.68300000000000005},"min":{"1min":0.54100000000000004,"5min":0.38200000000000001,"15min":0.38200000000000001},"max":{"1min":1.0149999999999999,"5min":2.0150000000000001,"15min":2.0150000000000001},"last":0.65600000000000003}]},{"osd":1,"last update":"Sun Nov 26 07:23:49 2023","interfaces":[{"interface":"back","average":{"1min":0.68300000000000005,"5min":0.70699999999999996,"15min":0.68100000000000005},"min":{"1min":0.50900000000000001,"5min":0.41799999999999998,"15min":0.41599999999999998},"max":{"1min":0.97999999999999998,"5min":1.532,"15min":1.532},"last":0.77000000000000002},{"interface":"front","average":{"1min":0.71699999999999997,"5min":0.73699999999999999,"15min":0.70299999999999996},"min":{"1min":0.52100000000000002,"5min":0.443,"15min":0.443},"max":{"1min":0.92900000000000005,"5min":2.6400000000000001,"15min":2.6400000000000001},"last":0.71699999999999997}]},{"osd":2,"last update":"Sun Nov 26 07:23:49 2023","interfaces":[{"interface":"back","average":{"1min":0.72599999999999998,"5min":0.71899999999999997,"15min":0.68899999999999995},"min":{"1min":0.42099999999999999,"5min":0.42099999999999999,"15min":0.42099999999999999},"max":{"1min":0.90200000000000002,"5min":1.665,"15min":1.665},"last":0.91500000000000004},{"interface":"front","average":{"1min":0.72999999999999998,"5min":0.76200000000000001,"15min":0.69999999999999996},"min":{"1min":0.47499999999999998,"5min":0.46200000000000002,"15min":0.46200000000000002},"max":{"1min":1.0009999999999999,"5min":2.125,"15min":2.125},"last":0.69599999999999995}]},{"osd":3,"last update":"Sun Nov 26 07:23:49 2023","interfaces":[{"interface":"back","average":{"1min":0.745,"5min":0.75600000000000001,"15min":0.73199999999999998},"min":{"1min":0.52200000000000002,"5min":0.44700000000000001,"15min":0.43099999999999999},"max":{"1min":0.97699999999999998,"5min":2.0529999999999999,"15min":2.0529999999999999},"last":0.879},{"interface":"front","average":{"1min":0.77700000000000002,"5min":0.76100000000000001,"15min":0.70299999999999996},"min":{"1min":0.55700000000000005,"5min":0.48199999999999998,"15min":0.44600000000000001},"max":{"1min":1.022,"5min":1.629,"15min":1.629},"last":0.67700000000000005}]},{"osd":4,"last update":"Sun Nov 26 07:23:49 2023","interfaces":[{"interface":"back","average":{"1min":0.65300000000000002,"5min":0.61599999999999999,"15min":0.628},"min":{"1min":0.39100000000000001,"5min":0.35299999999999998,"15min":0.35299999999999998},"max":{"1min":0.90800000000000003,"5min":2.7429999999999999,"15min":2.7429999999999999},"last":0.626},{"interface":"front","average":{"1min":0.70099999999999996,"5min":0.68100000000000005,"15min":0.69899999999999995},"min":{"1min":0.48599999999999999,"5min":0.41499999999999998,"15min":0.41299999999999998},"max":{"1min":0.95299999999999996,"5min":2.4630000000000001,"15min":2.4630000000000001},"last":0.85999999999999999}]},{"osd":6,"last update":"Sun Nov 26 07:23:49 2023","interfaces":[{"interface":"back","average":{"1min":0.68600000000000005,"5min":0.67700000000000005,"15min":0.67300000000000004},"min":{"1min":0.372,"5min":0.372,"15min":0.372},"max":{"1min":0.92400000000000004,"5min":2.1619999999999999,"15min":2.1619999999999999},"last":0.84199999999999997},{"interface":"front","average":{"1min":0.71899999999999997,"5min":0.70999999999999996,"15min":0.72099999999999997},"min":{"1min":0.46500000000000002,"5min":0.436,"15min":0.436},"max":{"1min":0.95799999999999996,"5min":2.7869999999999999,"15min":2.7869999999999999},"last":0.89500000000000002}]},{"osd":7,"last update":"Sun Nov 26 07:23:49 2023","interfaces":[{"interface":"back","average":{"1min":0.73799999999999999,"5min":0.746,"15min":0.78400000000000003},"min":{"1min":0.41399999999999998,"5min":0.41399999999999998,"15min":0.41399999999999998},"max":{"1min":0.94999999999999996,"5min":2.1960000000000002,"15min":2.1960000000000002},"last":0.747},{"interface":"front","average":{"1min":0.77100000000000002,"5min":0.76200000000000001,"15min":0.77000000000000002},"min":{"1min":0.44700000000000001,"5min":0.44700000000000001,"15min":0.44700000000000001},"max":{"1min":0.97499999999999998,"5min":1.9510000000000001,"15min":1.9510000000000001},"last":0.94899999999999995}]}]},{"osd":4,"up_from":13,"seq":55834574930,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32432,"kb_used_data":1252,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339408,"statfs":{"total":96636764160,"available":96603553792,"internally_reserved":0,"allocated":1282048,"data_stored":562328,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Nov 26 07:23:53 2023","interfaces":[{"interface":"back","average":{"1min":0.64600000000000002,"5min":0.65800000000000003,"15min":0.748},"min":{"1min":0.443,"5min":0.32100000000000001,"15min":0.32100000000000001},"max":{"1min":0.91100000000000003,"5min":0.96199999999999997,"15min":4.1689999999999996},"last":0.67800000000000005},{"interface":"front","average":{"1min":0.65300000000000002,"5min":0.84699999999999998,"15min":0.94999999999999996},"min":{"1min":0.40799999999999997,"5min":0.40799999999999997,"15min":0.40799999999999997},"max":{"1min":0.83299999999999996,"5min":18.021000000000001,"15min":18.021000000000001},"last":0.65400000000000003}]},{"osd":1,"last update":"Sun Nov 26 07:23:53 2023","interfaces":[{"interface":"back","average":{"1min":0.72699999999999998,"5min":0.73499999999999999,"15min":0.67600000000000005},"min":{"1min":0.55200000000000005,"5min":0.43099999999999999,"15min":0.43099999999999999},"max":{"1min":0.93500000000000005,"5min":4.3159999999999998,"15min":4.3159999999999998},"last":0.79400000000000004},{"interface":"front","average":{"1min":0.67400000000000004,"5min":0.76600000000000001,"15min":0.80200000000000005},"min":{"1min":0.436,"5min":0.39800000000000002,"15min":0.39800000000000002},"max":{"1min":0.94199999999999995,"5min":11.206,"15min":11.206},"last":0.76100000000000001}]},{"osd":2,"last update":"Sun Nov 26 07:23:53 2023","interfaces":[{"interface":"back","average":{"1min":0.71099999999999997,"5min":0.85199999999999998,"15min":0.82199999999999995},"min":{"1min":0.56599999999999995,"5min":0.433,"15min":0.433},"max":{"1min":0.84299999999999997,"5min":15.441000000000001,"15min":15.441000000000001},"last":0.73699999999999999},{"interface":"front","average":{"1min":0.77400000000000002,"5min":0.85599999999999998,"15min":0.996},"min":{"1min":0.59599999999999997,"5min":0.54100000000000004,"15min":0.53400000000000003},"max":{"1min":0.93999999999999995,"5min":9.4800000000000004,"15min":9.4800000000000004},"last":0.871}]},{"osd":3,"last update":"Sun Nov 26 07:23:53 2023","interfaces":[{"interface":"back","average":{"1min":0.71199999999999997,"5min":0.82899999999999996,"15min":0.76300000000000001},"min":{"1min":0.53600000000000003,"5min":0.441,"15min":0.441},"max":{"1min":0.95899999999999996,"5min":12.715,"15min":12.715},"last":0.84499999999999997},{"interface":"front","average":{"1min":0.76500000000000001,"5min":0.81499999999999995,"15min":0.85699999999999998},"min":{"1min":0.63600000000000001,"5min":0.50700000000000001,"15min":0.50700000000000001},"max":{"1min":1.0109999999999999,"5min":8.3239999999999998,"15min":8.3239999999999998},"last":0.82899999999999996}]},{"osd":5,"last update":"Sun Nov 26 07:23:53 2023","interfaces":[{"interface":"back","average":{"1min":0.67300000000000004,"5min":0.629,"15min":0.61699999999999999},"min":{"1min":0.41699999999999998,"5min":0.38200000000000001,"15min":0.38200000000000001},"max":{"1min":0.94199999999999995,"5min":0.94199999999999995,"15min":0.96199999999999997},"last":0.92100000000000004},{"interface":"front","average":{"1min":0.66000000000000003,"5min":0.72399999999999998,"15min":0.66800000000000004},"min":{"1min":0.36399999999999999,"5min":0.35599999999999998,"15min":0.35599999999999998},"max":{"1min":0.88400000000000001,"5min":11.148999999999999,"15min":11.148999999999999},"last":0.94599999999999995}]},{"osd":6,"last update":"Sun Nov 26 07:23:53 2023","interfaces":[{"interface":"back","average":{"1min":0.63400000000000001,"5min":0.60699999999999998,"15min":0.73099999999999998},"min":{"1min":0.48499999999999999,"5min":0.372,"15min":0.372},"max":{"1min":0.76700000000000002,"5min":0.89200000000000002,"15min":2.8330000000000002},"last":0.78200000000000003},{"interface":"front","average":{"1min":0.73799999999999999,"5min":0.71399999999999997,"15min":0.91000000000000003},"min":{"1min":0.47299999999999998,"5min":0.41299999999999998,"15min":0.41299999999999998},"max":{"1min":0.90500000000000003,"5min":2.472,"15min":7.6050000000000004},"last":0.71699999999999997}]},{"osd":7,"last update":"Sun Nov 26 07:23:53 2023","interfaces":[{"interface":"back","average":{"1min":0.71399999999999997,"5min":0.81699999999999995,"15min":0.84599999999999997},"min":{"1min":0.56899999999999995,"5min":0.47699999999999998,"15min":0.44700000000000001},"max":{"1min":0.85699999999999998,"5min":12.653,"15min":12.653},"last":0.90000000000000002},{"interface":"front","average":{"1min":0.78900000000000003,"5min":0.77500000000000002,"15min":0.85299999999999998},"min":{"1min":0.58499999999999996,"5min":0.33300000000000002,"15min":0.33300000000000002},"max":{"1min":0.91300000000000003,"5min":3.8319999999999999,"15min":3.8319999999999999},"last":0.95999999999999996}]}]},{"osd":3,"up_from":13,"seq":55834574931,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32448,"kb_used_data":1268,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339392,"statfs":{"total":96636764160,"available":96603537408,"internally_reserved":0,"allocated":1298432,"data_stored":564010,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1592,"internal_metadata":31914440},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Nov 26 07:23:54 2023","interfaces":[{"interface":"back","average":{"1min":0.40100000000000002,"5min":0.41899999999999998,"15min":0.39200000000000002},"min":{"1min":0.29099999999999998,"5min":0.218,"15min":0.218},"max":{"1min":0.53600000000000003,"5min":0.72799999999999998,"15min":0.72799999999999998},"last":0.29399999999999998},{"interface":"front","average":{"1min":0.47599999999999998,"5min":0.46800000000000003,"15min":0.44400000000000001},"min":{"1min":0.30199999999999999,"5min":0.254,"15min":0.254},"max":{"1min":0.75900000000000001,"5min":0.77300000000000002,"15min":0.77300000000000002},"last":0.32600000000000001}]},{"osd":1,"last update":"Sun Nov 26 07:23:54 2023","interfaces":[{"interface":"back","average":{"1min":0.44400000000000001,"5min":0.441,"15min":0.42699999999999999},"min":{"1min":0.28799999999999998,"5min":0.28799999999999998,"15min":0.28799999999999998},"max":{"1min":0.59999999999999998,"5min":0.61299999999999999,"15min":0.752},"last":0.34599999999999997},{"interface":"front","average":{"1min":0.51800000000000002,"5min":0.496,"15min":0.48799999999999999},"min":{"1min":0.34000000000000002,"5min":0.29899999999999999,"15min":0.29899999999999999},"max":{"1min":0.80800000000000005,"5min":0.872,"15min":0.872},"last":0.378}]},{"osd":2,"last update":"Sun Nov 26 07:23:54 2023","interfaces":[{"interface":"back","average":{"1min":0.56000000000000005,"5min":0.56599999999999995,"15min":0.54400000000000004},"min":{"1min":0.442,"5min":0.376,"15min":0.376},"max":{"1min":0.77200000000000002,"5min":0.97199999999999998,"15min":0.97199999999999998},"last":0.53100000000000003},{"interface":"front","average":{"1min":0.52100000000000002,"5min":0.53700000000000003,"15min":0.51500000000000001},"min":{"1min":0.32600000000000001,"5min":0.32100000000000001,"15min":0.32100000000000001},"max":{"1min":0.70899999999999996,"5min":0.89700000000000002,"15min":0.89700000000000002},"last":0.48299999999999998}]},{"osd":4,"last update":"Sun Nov 26 07:23:54 2023","interfaces":[{"interface":"back","average":{"1min":0.72599999999999998,"5min":0.71099999999999997,"15min":0.70699999999999996},"min":{"1min":0.51400000000000001,"5min":0.39600000000000002,"15min":0.39600000000000002},"max":{"1min":0.88300000000000001,"5min":1.091,"15min":1.091},"last":0.91000000000000003},{"interface":"front","average":{"1min":0.71699999999999997,"5min":0.72099999999999997,"15min":0.72499999999999998},"min":{"1min":0.52400000000000002,"5min":0.39900000000000002,"15min":0.39900000000000002},"max":{"1min":1.089,"5min":1.242,"15min":1.242},"last":0.77400000000000002}]},{"osd":5,"last update":"Sun Nov 26 07:23:54 2023","interfaces":[{"interface":"back","average":{"1min":0.71599999999999997,"5min":0.70199999999999996,"15min":0.72599999999999998},"min":{"1min":0.48699999999999999,"5min":0.45900000000000002,"15min":0.45900000000000002},"max":{"1min":0.96199999999999997,"5min":1.105,"15min":1.105},"last":0.85399999999999998},{"interface":"front","average":{"1min":0.74399999999999999,"5min":0.73699999999999999,"15min":0.73699999999999999},"min":{"1min":0.54700000000000004,"5min":0.47299999999999998,"15min":0.47299999999999998},"max":{"1min":0.94199999999999995,"5min":1.1319999999999999,"15min":1.1319999999999999},"last":0.81299999999999994}]},{"osd":6,"last update":"Sun Nov 26 07:23:54 2023","interfaces":[{"interface":"back","average":{"1min":0.76900000000000002,"5min":0.75900000000000001,"15min":0.79500000000000004},"min":{"1min":0.51300000000000001,"5min":0.5,"15min":0.5},"max":{"1min":0.96499999999999997,"5min":1.214,"15min":1.2729999999999999},"last":0.83899999999999997},{"interface":"front","average":{"1min":0.76800000000000002,"5min":0.754,"15min":0.78000000000000003},"min":{"1min":0.61799999999999999,"5min":0.48799999999999999,"15min":0.47299999999999998},"max":{"1min":1.1060000000000001,"5min":1.1060000000000001,"15min":1.204},"last":0.82399999999999995}]},{"osd":7,"last update":"Sun Nov 26 07:23:54 2023","interfaces":[{"interface":"back","average":{"1min":0.78200000000000003,"5min":0.755,"15min":0.78500000000000003},"min":{"1min":0.496,"5min":0.437,"15min":0.437},"max":{"1min":1.0349999999999999,"5min":1.125,"15min":1.3600000000000001},"last":0.86699999999999999},{"interface":"front","average":{"1min":0.78100000000000003,"5min":0.78900000000000003,"15min":0.81299999999999994},"min":{"1min":0.627,"5min":0.47299999999999998,"15min":0.47299999999999998},"max":{"1min":1.125,"5min":1.2450000000000001,"15min":1.617},"last":0.88700000000000001}]}]},{"osd":2,"up_from":13,"seq":55834574931,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32440,"kb_used_data":1252,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339400,"statfs":{"total":96636764160,"available":96603545600,"internally_reserved":0,"allocated":1282048,"data_stored":562328,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Nov 26 07:23:52 2023","interfaces":[{"interface":"back","average":{"1min":0.42699999999999999,"5min":0.43099999999999999,"15min":0.61899999999999999},"min":{"1min":0.27200000000000002,"5min":0.27100000000000002,"15min":0.27100000000000002},"max":{"1min":0.67500000000000004,"5min":0.76000000000000001,"15min":4.7030000000000003},"last":0.496},{"interface":"front","average":{"1min":0.47299999999999998,"5min":0.46600000000000003,"15min":0.53000000000000003},"min":{"1min":0.378,"5min":0.27300000000000002,"15min":0.27300000000000002},"max":{"1min":0.69799999999999995,"5min":0.77200000000000002,"15min":2.4340000000000002},"last":0.28399999999999997}]},{"osd":1,"last update":"Sun Nov 26 07:23:52 2023","interfaces":[{"interface":"back","average":{"1min":0.48199999999999998,"5min":0.49099999999999999,"15min":0.73899999999999999},"min":{"1min":0.35499999999999998,"5min":0.318,"15min":0.318},"max":{"1min":0.621,"5min":0.86099999999999999,"15min":4.2690000000000001},"last":0.58599999999999997},{"interface":"front","average":{"1min":0.504,"5min":0.501,"15min":0.66100000000000003},"min":{"1min":0.39800000000000002,"5min":0.35099999999999998,"15min":0.31900000000000001},"max":{"1min":0.623,"5min":0.76600000000000001,"15min":3.9940000000000002},"last":0.439}]},{"osd":3,"last update":"Sun Nov 26 07:23:52 2023","interfaces":[{"interface":"back","average":{"1min":0.48199999999999998,"5min":0.47499999999999998,"15min":0.58699999999999997},"min":{"1min":0.311,"5min":0.29699999999999999,"15min":0.29699999999999999},"max":{"1min":0.64700000000000002,"5min":0.82699999999999996,"15min":4.0300000000000002},"last":0.73299999999999998},{"interface":"front","average":{"1min":0.55500000000000005,"5min":0.55000000000000004,"15min":0.79200000000000004},"min":{"1min":0.443,"5min":0.36099999999999999,"15min":0.36099999999999999},"max":{"1min":0.79700000000000004,"5min":0.86699999999999999,"15min":4.5810000000000004},"last":0.71999999999999997}]},{"osd":4,"last update":"Sun Nov 26 07:23:52 2023","interfaces":[{"interface":"back","average":{"1min":0.73199999999999998,"5min":0.76600000000000001,"15min":0.91200000000000003},"min":{"1min":0.55600000000000005,"5min":0.51400000000000001,"15min":0.51400000000000001},"max":{"1min":0.97499999999999998,"5min":1.01,"15min":4.4269999999999996},"last":0.68799999999999994},{"interface":"front","average":{"1min":0.752,"5min":0.76800000000000002,"15min":0.93200000000000005},"min":{"1min":0.54300000000000004,"5min":0.54000000000000004,"15min":0.54000000000000004},"max":{"1min":1.0009999999999999,"5min":1.0780000000000001,"15min":4.085},"last":0.56499999999999995}]},{"osd":5,"last update":"Sun Nov 26 07:23:52 2023","interfaces":[{"interface":"back","average":{"1min":0.77300000000000002,"5min":0.76600000000000001,"15min":0.83799999999999997},"min":{"1min":0.42099999999999999,"5min":0.42099999999999999,"15min":0.42099999999999999},"max":{"1min":0.97599999999999998,"5min":1.0580000000000001,"15min":3.9590000000000001},"last":0.81399999999999995},{"interface":"front","average":{"1min":0.71999999999999997,"5min":0.78200000000000003,"15min":0.91900000000000004},"min":{"1min":0.51600000000000001,"5min":0.499,"15min":0.499},"max":{"1min":0.96199999999999997,"5min":1.0900000000000001,"15min":4.3849999999999998},"last":0.64700000000000002}]},{"osd":6,"last update":"Sun Nov 26 07:23:52 2023","interfaces":[{"interface":"back","average":{"1min":0.73699999999999999,"5min":0.76800000000000002,"15min":0.92000000000000004},"min":{"1min":0.45200000000000001,"5min":0.45200000000000001,"15min":0.45200000000000001},"max":{"1min":0.93600000000000005,"5min":1.083,"15min":4.0449999999999999},"last":0.66600000000000004},{"interface":"front","average":{"1min":0.71099999999999997,"5min":0.75900000000000001,"15min":0.86499999999999999},"min":{"1min":0.52700000000000002,"5min":0.48099999999999998,"15min":0.48099999999999998},"max":{"1min":1.0229999999999999,"5min":1.085,"15min":3.992},"last":0.77800000000000002}]},{"osd":7,"last update":"Sun Nov 26 07:23:52 2023","interfaces":[{"interface":"back","average":{"1min":0.76600000000000001,"5min":0.78200000000000003,"15min":0.94999999999999996},"min":{"1min":0.56799999999999995,"5min":0.52200000000000002,"15min":0.52200000000000002},"max":{"1min":1.018,"5min":1.0840000000000001,"15min":4.1219999999999999},"last":0.627},{"interface":"front","average":{"1min":0.747,"5min":0.78200000000000003,"15min":0.88600000000000001},"min":{"1min":0.49299999999999999,"5min":0.48199999999999998,"15min":0.48199999999999998},"max":{"1min":0.95399999999999996,"5min":1.036,"15min":3.9159999999999999},"last":0.85099999999999998}]}]},{"osd":1,"up_from":13,"seq":55834574930,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32448,"kb_used_data":1260,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339392,"statfs":{"total":96636764160,"available":96603537408,"internally_reserved":0,"allocated":1290240,"data_stored":562452,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":31914439},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Nov 26 07:23:50 2023","interfaces":[{"interface":"back","average":{"1min":0.41699999999999998,"5min":0.40799999999999997,"15min":0.41399999999999998},"min":{"1min":0.27300000000000002,"5min":0.23599999999999999,"15min":0.23599999999999999},"max":{"1min":0.55600000000000005,"5min":0.59999999999999998,"15min":0.61199999999999999},"last":0.45300000000000001},{"interface":"front","average":{"1min":0.45100000000000001,"5min":0.45000000000000001,"15min":0.47399999999999998},"min":{"1min":0.32800000000000001,"5min":0.26700000000000002,"15min":0.26700000000000002},"max":{"1min":0.65800000000000003,"5min":0.749,"15min":0.78700000000000003},"last":0.40600000000000003}]},{"osd":2,"last update":"Sun Nov 26 07:23:50 2023","interfaces":[{"interface":"back","average":{"1min":0.42999999999999999,"5min":0.437,"15min":0.44900000000000001},"min":{"1min":0.318,"5min":0.27800000000000002,"15min":0.27800000000000002},"max":{"1min":0.58499999999999996,"5min":0.71399999999999997,"15min":0.71399999999999997},"last":0.38100000000000001},{"interface":"front","average":{"1min":0.45500000000000002,"5min":0.47999999999999998,"15min":0.498},"min":{"1min":0.318,"5min":0.27700000000000002,"15min":0.27700000000000002},"max":{"1min":0.60199999999999998,"5min":0.80600000000000005,"15min":0.80600000000000005},"last":0.499}]},{"osd":3,"last update":"Sun Nov 26 07:23:50 2023","interfaces":[{"interface":"back","average":{"1min":0.51400000000000001,"5min":0.52500000000000002,"15min":0.52800000000000002},"min":{"1min":0.36099999999999999,"5min":0.36099999999999999,"15min":0.36099999999999999},"max":{"1min":0.66200000000000003,"5min":0.748,"15min":0.748},"last":0.53500000000000003},{"interface":"front","average":{"1min":0.496,"5min":0.499,"15min":0.497},"min":{"1min":0.39500000000000002,"5min":0.31,"15min":0.31},"max":{"1min":0.65800000000000003,"5min":0.73599999999999999,"15min":0.80700000000000005},"last":0.48899999999999999}]},{"osd":4,"last update":"Sun Nov 26 07:23:50 2023","interfaces":[{"interface":"back","average":{"1min":0.79500000000000004,"5min":0.75,"15min":0.71199999999999997},"min":{"1min":0.53700000000000003,"5min":0.41299999999999998,"15min":0.41299999999999998},"max":{"1min":1.319,"5min":1.319,"15min":1.319},"last":0.67800000000000005},{"interface":"front","average":{"1min":0.76500000000000001,"5min":0.72799999999999998,"15min":0.69599999999999995},"min":{"1min":0.52900000000000003,"5min":0.44500000000000001,"15min":0.44500000000000001},"max":{"1min":0.98799999999999999,"5min":1.131,"15min":1.131},"last":0.63}]},{"osd":5,"last update":"Sun Nov 26 07:23:50 2023","interfaces":[{"interface":"back","average":{"1min":0.77000000000000002,"5min":0.746,"15min":0.72799999999999998},"min":{"1min":0.47899999999999998,"5min":0.47899999999999998,"15min":0.47899999999999998},"max":{"1min":0.93400000000000005,"5min":1.1919999999999999,"15min":1.1919999999999999},"last":0.66800000000000004},{"interface":"front","average":{"1min":0.78000000000000003,"5min":0.749,"15min":0.72699999999999998},"min":{"1min":0.59899999999999998,"5min":0.47099999999999997,"15min":0.47099999999999997},"max":{"1min":1.016,"5min":1.0609999999999999,"15min":1.0609999999999999},"last":0.72199999999999998}]},{"osd":6,"last update":"Sun Nov 26 07:23:50 2023","interfaces":[{"interface":"back","average":{"1min":0.79800000000000004,"5min":0.76100000000000001,"15min":0.72299999999999998},"min":{"1min":0.61199999999999999,"5min":0.53200000000000003,"15min":0.50600000000000001},"max":{"1min":1.2190000000000001,"5min":1.2190000000000001,"15min":1.2190000000000001},"last":0.76200000000000001},{"interface":"front","average":{"1min":0.84899999999999998,"5min":0.76900000000000002,"15min":0.72299999999999998},"min":{"1min":0.502,"5min":0.46999999999999997,"15min":0.46999999999999997},"max":{"1min":1.1659999999999999,"5min":1.1659999999999999,"15min":1.1659999999999999},"last":0.70899999999999996}]},{"osd":7,"last update":"Sun Nov 26 07:23:50 2023","interfaces":[{"interface":"back","average":{"1min":0.82899999999999996,"5min":0.76900000000000002,"15min":0.73299999999999998},"min":{"1min":0.627,"5min":0.50600000000000001,"15min":0.50600000000000001},"max":{"1min":1.0920000000000001,"5min":1.0920000000000001,"15min":1.0920000000000001},"last":0.76800000000000002},{"interface":"front","average":{"1min":0.82599999999999996,"5min":0.75700000000000001,"15min":0.754},"min":{"1min":0.58499999999999996,"5min":0.48499999999999999,"15min":0.48499999999999999},"max":{"1min":1.115,"5min":1.115,"15min":1.115},"last":0.73999999999999999}]}]},{"osd":0,"up_from":13,"seq":55834574930,"num_pgs":36,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":33036,"kb_used_data":1852,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94338804,"statfs":{"total":96636764160,"available":96602935296,"internally_reserved":0,"allocated":1896448,"data_stored":1154420,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Sun Nov 26 07:23:47 2023","interfaces":[{"interface":"back","average":{"1min":0.41799999999999998,"5min":0.42999999999999999,"15min":0.42999999999999999},"min":{"1min":0.28899999999999998,"5min":0.27000000000000002,"15min":0.27000000000000002},"max":{"1min":0.67300000000000004,"5min":1.26,"15min":1.26},"last":0.56299999999999994},{"interface":"front","average":{"1min":0.46300000000000002,"5min":0.46800000000000003,"15min":0.47799999999999998},"min":{"1min":0.31,"5min":0.30199999999999999,"15min":0.29899999999999999},"max":{"1min":0.65500000000000003,"5min":1.212,"15min":1.212},"last":0.34599999999999997}]},{"osd":2,"last update":"Sun Nov 26 07:23:47 2023","interfaces":[{"interface":"back","average":{"1min":0.499,"5min":0.50600000000000001,"15min":0.52600000000000002},"min":{"1min":0.36599999999999999,"5min":0.34699999999999998,"15min":0.34699999999999998},"max":{"1min":0.69099999999999995,"5min":0.84199999999999997,"15min":0.84199999999999997},"last":0.38200000000000001},{"interface":"front","average":{"1min":0.57799999999999996,"5min":0.55600000000000005,"15min":0.56100000000000005},"min":{"1min":0.40500000000000003,"5min":0.39800000000000002,"15min":0.36399999999999999},"max":{"1min":0.82199999999999995,"5min":1.0800000000000001,"15min":1.0800000000000001},"last":0.498}]},{"osd":3,"last update":"Sun Nov 26 07:23:47 2023","interfaces":[{"interface":"back","average":{"1min":0.5,"5min":0.51300000000000001,"15min":0.52500000000000002},"min":{"1min":0.33500000000000002,"5min":0.33500000000000002,"15min":0.33500000000000002},"max":{"1min":0.64100000000000001,"5min":1.3149999999999999,"15min":1.3149999999999999},"last":0.434},{"interface":"front","average":{"1min":0.54200000000000004,"5min":0.52900000000000003,"15min":0.55100000000000005},"min":{"1min":0.35499999999999998,"5min":0.35499999999999998,"15min":0.35499999999999998},"max":{"1min":0.69699999999999995,"5min":0.76400000000000001,"15min":1.2509999999999999},"last":0.61299999999999999}]},{"osd":4,"last update":"Sun Nov 26 07:23:47 2023","interfaces":[{"interface":"back","average":{"1min":0.747,"5min":0.745,"15min":0.754},"min":{"1min":0.52800000000000002,"5min":0.44700000000000001,"15min":0.44700000000000001},"max":{"1min":0.98299999999999998,"5min":1.0900000000000001,"15min":1.2010000000000001},"last":0.58199999999999996},{"interface":"front","average":{"1min":0.78800000000000003,"5min":0.78600000000000003,"15min":0.78800000000000003},"min":{"1min":0.55000000000000004,"5min":0.51100000000000001,"15min":0.51100000000000001},"max":{"1min":1.298,"5min":1.381,"15min":1.381},"last":0.76000000000000001}]},{"osd":5,"last update":"Sun Nov 26 07:23:47 2023","interfaces":[{"interface":"back","average":{"1min":0.72699999999999998,"5min":0.75,"15min":0.75600000000000001},"min":{"1min":0.45200000000000001,"5min":0.42199999999999999,"15min":0.42199999999999999},"max":{"1min":0.94899999999999995,"5min":1.1819999999999999,"15min":1.1819999999999999},"last":0.69699999999999995},{"interface":"front","average":{"1min":0.78200000000000003,"5min":0.80200000000000005,"15min":0.79800000000000004},"min":{"1min":0.53000000000000003,"5min":0.53000000000000003,"15min":0.53000000000000003},"max":{"1min":1.329,"5min":1.399,"15min":1.399},"last":0.80000000000000004}]},{"osd":6,"last update":"Sun Nov 26 07:23:47 2023","interfaces":[{"interface":"back","average":{"1min":0.72399999999999998,"5min":0.69599999999999995,"15min":0.74099999999999999},"min":{"1min":0.48499999999999999,"5min":0.48499999999999999,"15min":0.48499999999999999},"max":{"1min":1.0740000000000001,"5min":1.0740000000000001,"15min":1.3029999999999999},"last":0.71299999999999997},{"interface":"front","average":{"1min":0.72799999999999998,"5min":0.73699999999999999,"15min":0.75800000000000001},"min":{"1min":0.51200000000000001,"5min":0.47199999999999998,"15min":0.47199999999999998},"max":{"1min":1.2170000000000001,"5min":1.3600000000000001,"15min":1.3600000000000001},"last":0.66300000000000003}]},{"osd":7,"last update":"Sun Nov 26 07:23:47 2023","interfaces":[{"interface":"back","average":{"1min":0.77400000000000002,"5min":0.76800000000000002,"15min":0.79600000000000004},"min":{"1min":0.59799999999999998,"5min":0.56899999999999995,"15min":0.56899999999999995},"max":{"1min":1.1719999999999999,"5min":1.1719999999999999,"15min":1.177},"last":0.78000000000000003},{"interface":"front","average":{"1min":0.76400000000000001,"5min":0.75800000000000001,"15min":0.78500000000000003},"min":{"1min":0.51900000000000002,"5min":0.47899999999999998,"15min":0.47899999999999998},"max":{"1min":1.137,"5min":1.238,"15min":1.2390000000000001},"last":0.629}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":1724,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":124,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1682,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":604,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":70,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":616,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":17,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":17,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":17,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":17,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":17,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":17,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":17,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":17,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-11-26T07:24:13.378 INFO:tasks.cephfs.filesystem.ceph_manager:clean! 2023-11-26T07:24:13.378 DEBUG:teuthology.orchestra.run.smithi143:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth caps client.0 mon 'allow r' osd 'allow rw tag cephfs data=cephfs' mds allow 2023-11-26T07:24:13.558 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:13.559+0000 7f7d07b65640 1 Processor -- start 2023-11-26T07:24:13.559 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:13.559+0000 7f7d07b65640 1 -- start start 2023-11-26T07:24:13.560 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:13.559+0000 7f7d07b65640 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7f7d000d1840 0x7f7d000d0720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:24:13.560 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:13.559+0000 7f7d07b65640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f7d001608f0 0x7f7d00160d60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:24:13.560 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:13.559+0000 7f7d07b65640 1 -- --> v1:172.21.15.143:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7d000b2510 con 0x7f7d000d1c20 2023-11-26T07:24:13.560 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:13.559+0000 7f7d07b65640 1 -- --> v2:172.21.15.143:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7d00161f80 con 0x7f7d000d1840 2023-11-26T07:24:13.560 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:13.559+0000 7f7d07b65640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7d000cfc40 con 0x7f7d001608f0 2023-11-26T07:24:13.561 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:13.559+0000 7f7d05901640 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7f7d000d1840 0x7f7d000d0720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:24:13.561 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:13.559+0000 7f7d05100640 1 --1- >> v1:172.21.15.143:6789/0 conn(0x7f7d000d1c20 0x7f7d00161410 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.143:6789/0 says I am v1:172.21.15.143:42452/0 (socket says 172.21.15.143:42452) 2023-11-26T07:24:13.561 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:13.559+0000 7f7d05100640 1 -- 172.21.15.143:0/4036223616 learned_addr learned my addr 172.21.15.143:0/4036223616 (peer_addr_for_me v1:172.21.15.143:0/0) 2023-11-26T07:24:13.561 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:13.559+0000 7f7d06102640 1 --2- 172.21.15.143:0/4036223616 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f7d001608f0 0x7f7d00160d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:24:13.561 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:13.559+0000 7f7d05901640 1 -- 172.21.15.143:0/4036223616 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f7d001608f0 msgr2=0x7f7d00160d60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-26T07:24:13.561 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:13.559+0000 7f7d05901640 1 --2- 172.21.15.143:0/4036223616 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f7d001608f0 0x7f7d00160d60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:24:13.562 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:13.559+0000 7f7d05901640 1 -- 172.21.15.143:0/4036223616 >> v1:172.21.15.143:6789/0 conn(0x7f7d000d1c20 legacy=0x7f7d00161410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:24:13.562 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:13.559+0000 7f7d05901640 1 -- 172.21.15.143:0/4036223616 --> v2:172.21.15.143:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7d000cff20 con 0x7f7d000d1840 2023-11-26T07:24:13.562 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:13.559+0000 7f7d048ff640 1 -- 172.21.15.143:0/4036223616 <== mon.0 v1:172.21.15.143:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2454065941 0 0) 0x7f7d000b2510 con 0x7f7d000d1c20 2023-11-26T07:24:13.564 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:13.563+0000 7f7d05901640 1 --2- 172.21.15.143:0/4036223616 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f7d000d1840 0x7f7d000d0720 secure :-1 s=READY pgs=1312 cs=0 l=1 rev1=1 crypto rx=0x7f7cfc0097f0 tx=0x7f7cfc01c3f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7bd0ac86929cbb46 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:24:13.564 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:13.563+0000 7f7d048ff640 1 -- 172.21.15.143:0/4036223616 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7cfc0ab600 con 0x7f7d000d1840 2023-11-26T07:24:13.564 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:13.563+0000 7f7d048ff640 1 -- 172.21.15.143:0/4036223616 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7cfc0a7050 con 0x7f7d000d1840 2023-11-26T07:24:13.564 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:13.563+0000 7f7d048ff640 1 -- 172.21.15.143:0/4036223616 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7cfc0ab980 con 0x7f7d000d1840 2023-11-26T07:24:13.565 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:13.563+0000 7f7d07b65640 1 -- 172.21.15.143:0/4036223616 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f7d000d1840 msgr2=0x7f7d000d0720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:24:13.565 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:13.563+0000 7f7d07b65640 1 --2- 172.21.15.143:0/4036223616 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f7d000d1840 0x7f7d000d0720 secure :-1 s=READY pgs=1312 cs=0 l=1 rev1=1 crypto rx=0x7f7cfc0097f0 tx=0x7f7cfc01c3f0 comp rx=0 tx=0).stop 2023-11-26T07:24:13.565 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:13.563+0000 7f7d07b65640 1 -- 172.21.15.143:0/4036223616 shutdown_connections 2023-11-26T07:24:13.565 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:13.563+0000 7f7d07b65640 1 --2- 172.21.15.143:0/4036223616 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f7d001608f0 0x7f7d00160d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:24:13.565 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:13.563+0000 7f7d07b65640 1 --2- 172.21.15.143:0/4036223616 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f7d000d1840 0x7f7d000d0720 unknown :-1 s=CLOSED pgs=1312 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:24:13.565 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:13.563+0000 7f7d07b65640 1 -- 172.21.15.143:0/4036223616 >> 172.21.15.143:0/4036223616 conn(0x7f7d0009c570 msgr2=0x7f7d000d3240 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:24:13.565 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:13.563+0000 7f7d07b65640 1 -- 172.21.15.143:0/4036223616 shutdown_connections 2023-11-26T07:24:13.565 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:13.563+0000 7f7d07b65640 1 -- 172.21.15.143:0/4036223616 wait complete. 2023-11-26T07:24:13.566 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:13.567+0000 7f7d07b65640 1 Processor -- start 2023-11-26T07:24:13.566 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:13.567+0000 7f7d07b65640 1 -- start start 2023-11-26T07:24:13.568 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:13.567+0000 7f7d07b65640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f7d000d1840 0x7f7d00153b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:24:13.568 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:13.567+0000 7f7d07b65640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f7d000d1c20 0x7f7d000ccb90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:24:13.569 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:13.567+0000 7f7d07b65640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f7d001608f0 0x7f7d000ca310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:24:13.569 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:13.567+0000 7f7d07b65640 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7d00156710 con 0x7f7d001608f0 2023-11-26T07:24:13.569 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:13.567+0000 7f7d07b65640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7d001540e0 con 0x7f7d000d1c20 2023-11-26T07:24:13.569 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:13.567+0000 7f7d07b65640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7d001543c0 con 0x7f7d000d1840 2023-11-26T07:24:13.569 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:13.567+0000 7f7d06102640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f7d001608f0 0x7f7d000ca310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:24:13.569 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:13.567+0000 7f7d06102640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f7d001608f0 0x7f7d000ca310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:33856/0 (socket says 172.21.15.143:33856) 2023-11-26T07:24:13.569 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:13.567+0000 7f7d06102640 1 -- 172.21.15.143:0/1565938463 learned_addr learned my addr 172.21.15.143:0/1565938463 (peer_addr_for_me v2:172.21.15.143:0/0) 2023-11-26T07:24:13.569 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:13.567+0000 7f7d05901640 1 --2- 172.21.15.143:0/1565938463 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f7d000d1840 0x7f7d00153b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:24:13.569 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:13.567+0000 7f7d05100640 1 --2- 172.21.15.143:0/1565938463 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f7d000d1c20 0x7f7d000ccb90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:24:13.569 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:13.567+0000 7f7d06102640 1 -- 172.21.15.143:0/1565938463 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f7d000d1840 msgr2=0x7f7d00153b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:24:13.569 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:13.567+0000 7f7d06102640 1 --2- 172.21.15.143:0/1565938463 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f7d000d1840 0x7f7d00153b00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:24:13.569 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:13.567+0000 7f7d06102640 1 -- 172.21.15.143:0/1565938463 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f7d000d1c20 msgr2=0x7f7d000ccb90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:24:13.570 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:13.567+0000 7f7d06102640 1 --2- 172.21.15.143:0/1565938463 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f7d000d1c20 0x7f7d000ccb90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:24:13.570 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:13.567+0000 7f7d06102640 1 -- 172.21.15.143:0/1565938463 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7cfc009420 con 0x7f7d001608f0 2023-11-26T07:24:13.570 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:13.567+0000 7f7d05100640 1 --2- 172.21.15.143:0/1565938463 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f7d000d1c20 0x7f7d000ccb90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-26T07:24:13.570 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:13.567+0000 7f7d06102640 1 --2- 172.21.15.143:0/1565938463 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f7d001608f0 0x7f7d000ca310 secure :-1 s=READY pgs=1313 cs=0 l=1 rev1=1 crypto rx=0x7f7cf8007e60 tx=0x7f7cf800e4b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:24:13.570 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:13.567+0000 7f7cf67fc640 1 -- 172.21.15.143:0/1565938463 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7cf8014420 con 0x7f7d001608f0 2023-11-26T07:24:13.570 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:13.567+0000 7f7cf67fc640 1 -- 172.21.15.143:0/1565938463 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7cf80145a0 con 0x7f7d001608f0 2023-11-26T07:24:13.570 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:13.567+0000 7f7cf67fc640 1 -- 172.21.15.143:0/1565938463 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7cf8014950 con 0x7f7d001608f0 2023-11-26T07:24:13.570 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:13.567+0000 7f7d07b65640 1 -- 172.21.15.143:0/1565938463 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7d001546a0 con 0x7f7d001608f0 2023-11-26T07:24:13.570 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:13.567+0000 7f7d07b65640 1 -- 172.21.15.143:0/1565938463 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7d00154980 con 0x7f7d001608f0 2023-11-26T07:24:13.571 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:13.567+0000 7f7d07b65640 1 -- 172.21.15.143:0/1565938463 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7d001568e0 con 0x7f7d001608f0 2023-11-26T07:24:13.571 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:13.571+0000 7f7cf67fc640 1 -- 172.21.15.143:0/1565938463 <== mon.0 v2:172.21.15.143:3300/0 4 ==== mgrmap(e 16) v1 ==== 85954+0+0 (secure 0 0 0) 0x7f7cf8014ab0 con 0x7f7d001608f0 2023-11-26T07:24:13.571 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:13.571+0000 7f7cf67fc640 1 --2- 172.21.15.143:0/1565938463 >> [v2:172.21.15.188:6836/3401128553,v1:172.21.15.188:6837/3401128553] conn(0x7f7ce4066590 0x7f7ce4068a50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:24:13.571 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:13.571+0000 7f7cf67fc640 1 -- 172.21.15.143:0/1565938463 <== mon.0 v2:172.21.15.143:3300/0 5 ==== osd_map(103..103 src has 1..103) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f7cf8084930 con 0x7f7d001608f0 2023-11-26T07:24:13.572 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:13.571+0000 7f7d05901640 1 --2- 172.21.15.143:0/1565938463 >> [v2:172.21.15.188:6836/3401128553,v1:172.21.15.188:6837/3401128553] conn(0x7f7ce4066590 0x7f7ce4068a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:24:13.572 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:13.571+0000 7f7d05901640 1 --2- 172.21.15.143:0/1565938463 >> [v2:172.21.15.188:6836/3401128553,v1:172.21.15.188:6837/3401128553] conn(0x7f7ce4066590 0x7f7ce4068a50 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7f7cfc0ad000 tx=0x7f7cfc0a6000 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:24:13.575 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:13.575+0000 7f7cf67fc640 1 -- 172.21.15.143:0/1565938463 <== mon.0 v2:172.21.15.143:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172649 (secure 0 0 0) 0x7f7d001568e0 con 0x7f7d001608f0 2023-11-26T07:24:13.778 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:13.779+0000 7f7d07b65640 1 -- 172.21.15.143:0/1565938463 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]} v 0) v1 -- 0x7f7d001568e0 con 0x7f7d001608f0 2023-11-26T07:24:13.781 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:13.779+0000 7f7cf67fc640 1 -- 172.21.15.143:0/1565938463 <== mon.0 v2:172.21.15.143:3300/0 7 ==== mon_command_ack([{"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]}]=0 updated caps for client.0 v20) v1 ==== 190+0+147 (secure 0 0 0) 0x7f7d001568e0 con 0x7f7d001608f0 2023-11-26T07:24:13.781 INFO:teuthology.orchestra.run.smithi143.stdout:[client.0] 2023-11-26T07:24:13.781 INFO:teuthology.orchestra.run.smithi143.stdout: key = AQDm8GJlWMh9FBAAslVzi6fm4/wt7vr/rHCHHQ== 2023-11-26T07:24:13.782 INFO:teuthology.orchestra.run.smithi143.stdout: caps mds = "allow" 2023-11-26T07:24:13.782 INFO:teuthology.orchestra.run.smithi143.stdout: caps mon = "allow r" 2023-11-26T07:24:13.782 INFO:teuthology.orchestra.run.smithi143.stdout: caps osd = "allow rw tag cephfs data=cephfs" 2023-11-26T07:24:13.782 INFO:teuthology.orchestra.run.smithi143.stderr:updated caps for client.0 2023-11-26T07:24:13.783 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:13.783+0000 7f7d07b65640 1 -- 172.21.15.143:0/1565938463 >> [v2:172.21.15.188:6836/3401128553,v1:172.21.15.188:6837/3401128553] conn(0x7f7ce4066590 msgr2=0x7f7ce4068a50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:24:13.784 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:13.783+0000 7f7d07b65640 1 --2- 172.21.15.143:0/1565938463 >> [v2:172.21.15.188:6836/3401128553,v1:172.21.15.188:6837/3401128553] conn(0x7f7ce4066590 0x7f7ce4068a50 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7f7cfc0ad000 tx=0x7f7cfc0a6000 comp rx=0 tx=0).stop 2023-11-26T07:24:13.784 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:13.783+0000 7f7d07b65640 1 -- 172.21.15.143:0/1565938463 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f7d001608f0 msgr2=0x7f7d000ca310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:24:13.784 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:13.783+0000 7f7d07b65640 1 --2- 172.21.15.143:0/1565938463 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f7d001608f0 0x7f7d000ca310 secure :-1 s=READY pgs=1313 cs=0 l=1 rev1=1 crypto rx=0x7f7cf8007e60 tx=0x7f7cf800e4b0 comp rx=0 tx=0).stop 2023-11-26T07:24:13.784 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:13.783+0000 7f7d07b65640 1 -- 172.21.15.143:0/1565938463 shutdown_connections 2023-11-26T07:24:13.784 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:13.783+0000 7f7d07b65640 1 --2- 172.21.15.143:0/1565938463 >> [v2:172.21.15.188:6836/3401128553,v1:172.21.15.188:6837/3401128553] conn(0x7f7ce4066590 0x7f7ce4068a50 unknown :-1 s=CLOSED pgs=131 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:24:13.784 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:13.783+0000 7f7d07b65640 1 --2- 172.21.15.143:0/1565938463 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f7d001608f0 0x7f7d000ca310 unknown :-1 s=CLOSED pgs=1313 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:24:13.784 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:13.783+0000 7f7d07b65640 1 --2- 172.21.15.143:0/1565938463 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f7d000d1c20 0x7f7d000ccb90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:24:13.784 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:13.783+0000 7f7d07b65640 1 --2- 172.21.15.143:0/1565938463 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f7d000d1840 0x7f7d00153b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:24:13.784 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:13.783+0000 7f7d07b65640 1 -- 172.21.15.143:0/1565938463 >> 172.21.15.143:0/1565938463 conn(0x7f7d0009c570 msgr2=0x7f7d000d3110 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:24:13.784 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:13.783+0000 7f7d07b65640 1 -- 172.21.15.143:0/1565938463 shutdown_connections 2023-11-26T07:24:13.784 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:13.783+0000 7f7d07b65640 1 -- 172.21.15.143:0/1565938463 wait complete. 2023-11-26T07:24:13.803 DEBUG:teuthology.orchestra.run.smithi143:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-11-26T07:24:13.895 INFO:tasks.ceph.mds.a.smithi143.stderr:2023-11-26T07:24:13.895+0000 7f533d54f640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2023-11-26T07:24:13.941 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:13.939+0000 7f47f6157640 1 Processor -- start 2023-11-26T07:24:13.941 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:13.939+0000 7f47f6157640 1 -- start start 2023-11-26T07:24:13.943 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:13.943+0000 7f47f6157640 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7f47f01596d0 0x7f47f0071850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:24:13.943 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:13.943+0000 7f47f6157640 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7f47f00ce680 0x7f47f00b2870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:24:13.943 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:13.943+0000 7f47f6157640 1 -- --> v1:172.21.15.143:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f47f00b3200 con 0x7f47f00b2db0 2023-11-26T07:24:13.943 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:13.943+0000 7f47f6157640 1 -- --> v2:172.21.15.143:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f47f00cbe00 con 0x7f47f00ce680 2023-11-26T07:24:13.943 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:13.943+0000 7f47f6157640 1 -- --> v2:172.21.15.188:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f47f006c750 con 0x7f47f01596d0 2023-11-26T07:24:13.943 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:13.943+0000 7f47eeffd640 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7f47f00ce680 0x7f47f00b2870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:24:13.943 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:13.943+0000 7f47effff640 1 --1- >> v1:172.21.15.143:6789/0 conn(0x7f47f00b2db0 0x7f47f015d2c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.143:6789/0 says I am v1:172.21.15.143:42468/0 (socket says 172.21.15.143:42468) 2023-11-26T07:24:13.943 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:13.943+0000 7f47effff640 1 -- 172.21.15.143:0/2912696536 learned_addr learned my addr 172.21.15.143:0/2912696536 (peer_addr_for_me v1:172.21.15.143:0/0) 2023-11-26T07:24:13.944 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:13.943+0000 7f47ef7fe640 1 --2- 172.21.15.143:0/2912696536 >> v2:172.21.15.188:3300/0 conn(0x7f47f01596d0 0x7f47f0071850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:24:13.944 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:13.943+0000 7f47eeffd640 1 -- 172.21.15.143:0/2912696536 >> v2:172.21.15.188:3300/0 conn(0x7f47f01596d0 msgr2=0x7f47f0071850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-26T07:24:13.944 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:13.943+0000 7f47eeffd640 1 --2- 172.21.15.143:0/2912696536 >> v2:172.21.15.188:3300/0 conn(0x7f47f01596d0 0x7f47f0071850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:24:13.944 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:13.943+0000 7f47eeffd640 1 -- 172.21.15.143:0/2912696536 >> v1:172.21.15.143:6789/0 conn(0x7f47f00b2db0 legacy=0x7f47f015d2c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:24:13.944 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:13.943+0000 7f47eeffd640 1 -- 172.21.15.143:0/2912696536 --> v2:172.21.15.143:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f47f00b3550 con 0x7f47f00ce680 2023-11-26T07:24:13.944 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:13.943+0000 7f47ee7fc640 1 -- 172.21.15.143:0/2912696536 <== mon.0 v1:172.21.15.143:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2318183757 0 0) 0x7f47e80014a0 con 0x7f47f00b2db0 2023-11-26T07:24:13.946 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:13.947+0000 7f47eeffd640 1 --2- 172.21.15.143:0/2912696536 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f47f00ce680 0x7f47f00b2870 secure :-1 s=READY pgs=1315 cs=0 l=1 rev1=1 crypto rx=0x7f47f00cbf40 tx=0x7f47e401c440 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3965d3fcb2ea7c4f server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:24:13.946 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:13.947+0000 7f47ee7fc640 1 -- 172.21.15.143:0/2912696536 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f47e40ab600 con 0x7f47f00ce680 2023-11-26T07:24:13.946 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:13.947+0000 7f47ee7fc640 1 -- 172.21.15.143:0/2912696536 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f47e40a7050 con 0x7f47f00ce680 2023-11-26T07:24:13.946 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:13.947+0000 7f47ee7fc640 1 -- 172.21.15.143:0/2912696536 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f47e40ab980 con 0x7f47f00ce680 2023-11-26T07:24:13.947 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:13.947+0000 7f47f6157640 1 -- 172.21.15.143:0/2912696536 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f47f00ce680 msgr2=0x7f47f00b2870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:24:13.947 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:13.947+0000 7f47f6157640 1 --2- 172.21.15.143:0/2912696536 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f47f00ce680 0x7f47f00b2870 secure :-1 s=READY pgs=1315 cs=0 l=1 rev1=1 crypto rx=0x7f47f00cbf40 tx=0x7f47e401c440 comp rx=0 tx=0).stop 2023-11-26T07:24:13.947 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:13.947+0000 7f47f6157640 1 -- 172.21.15.143:0/2912696536 shutdown_connections 2023-11-26T07:24:13.947 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:13.947+0000 7f47f6157640 1 --2- 172.21.15.143:0/2912696536 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f47f00ce680 0x7f47f00b2870 unknown :-1 s=CLOSED pgs=1315 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:24:13.948 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:13.947+0000 7f47f6157640 1 --2- 172.21.15.143:0/2912696536 >> v2:172.21.15.188:3300/0 conn(0x7f47f01596d0 0x7f47f0071850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:24:13.948 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:13.947+0000 7f47f6157640 1 -- 172.21.15.143:0/2912696536 >> 172.21.15.143:0/2912696536 conn(0x7f47f006a890 msgr2=0x7f47f006aca0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:24:13.948 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:13.947+0000 7f47f6157640 1 -- 172.21.15.143:0/2912696536 shutdown_connections 2023-11-26T07:24:13.948 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:13.947+0000 7f47f6157640 1 -- 172.21.15.143:0/2912696536 wait complete. 2023-11-26T07:24:13.949 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:13.947+0000 7f47f6157640 1 Processor -- start 2023-11-26T07:24:13.949 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:13.947+0000 7f47f6157640 1 -- start start 2023-11-26T07:24:13.951 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:13.951+0000 7f47f6157640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f47f00b2db0 0x7f47f015c810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:24:13.951 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:13.951+0000 7f47f6157640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f47f00ce680 0x7f47f01483a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:24:13.951 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:13.951+0000 7f47f6157640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f47f01596d0 0x7f47f0149a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:24:13.951 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:13.951+0000 7f47f6157640 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f47f014cd70 con 0x7f47f00b2db0 2023-11-26T07:24:13.951 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:13.951+0000 7f47f6157640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f47f0149fb0 con 0x7f47f00ce680 2023-11-26T07:24:13.951 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:13.951+0000 7f47f6157640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f47f014c0f0 con 0x7f47f01596d0 2023-11-26T07:24:13.951 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:13.951+0000 7f47ef7fe640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f47f00b2db0 0x7f47f015c810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:24:13.952 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:13.951+0000 7f47ef7fe640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f47f00b2db0 0x7f47f015c810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:33864/0 (socket says 172.21.15.143:33864) 2023-11-26T07:24:13.952 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:13.951+0000 7f47ef7fe640 1 -- 172.21.15.143:0/443578573 learned_addr learned my addr 172.21.15.143:0/443578573 (peer_addr_for_me v2:172.21.15.143:0/0) 2023-11-26T07:24:13.952 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:13.951+0000 7f47eeffd640 1 --2- 172.21.15.143:0/443578573 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f47f00ce680 0x7f47f01483a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:24:13.952 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:13.951+0000 7f47effff640 1 --2- 172.21.15.143:0/443578573 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f47f01596d0 0x7f47f0149a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:24:13.952 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:13.951+0000 7f47ef7fe640 1 -- 172.21.15.143:0/443578573 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f47f01596d0 msgr2=0x7f47f0149a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:24:13.952 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:13.951+0000 7f47ef7fe640 1 --2- 172.21.15.143:0/443578573 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f47f01596d0 0x7f47f0149a70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:24:13.952 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:13.951+0000 7f47ef7fe640 1 -- 172.21.15.143:0/443578573 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f47f00ce680 msgr2=0x7f47f01483a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:24:13.952 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:13.951+0000 7f47ef7fe640 1 --2- 172.21.15.143:0/443578573 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f47f00ce680 0x7f47f01483a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:24:13.952 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:13.951+0000 7f47ef7fe640 1 -- 172.21.15.143:0/443578573 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f47e4009420 con 0x7f47f00b2db0 2023-11-26T07:24:13.952 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:13.951+0000 7f47ef7fe640 1 --2- 172.21.15.143:0/443578573 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f47f00b2db0 0x7f47f015c810 secure :-1 s=READY pgs=1316 cs=0 l=1 rev1=1 crypto rx=0x7f47e000d7c0 tx=0x7f47e000dc80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:24:13.953 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:13.951+0000 7f47d7fff640 1 -- 172.21.15.143:0/443578573 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f47e000a8c0 con 0x7f47f00b2db0 2023-11-26T07:24:13.953 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:13.951+0000 7f47d7fff640 1 -- 172.21.15.143:0/443578573 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f47e000aa20 con 0x7f47f00b2db0 2023-11-26T07:24:13.953 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:13.951+0000 7f47d7fff640 1 -- 172.21.15.143:0/443578573 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f47e000ad90 con 0x7f47f00b2db0 2023-11-26T07:24:13.953 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:13.951+0000 7f47f6157640 1 -- 172.21.15.143:0/443578573 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f47f014c3d0 con 0x7f47f00b2db0 2023-11-26T07:24:13.953 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:13.951+0000 7f47f6157640 1 -- 172.21.15.143:0/443578573 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f47f014c990 con 0x7f47f00b2db0 2023-11-26T07:24:13.953 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:13.951+0000 7f47f6157640 1 -- 172.21.15.143:0/443578573 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f47f0159dc0 con 0x7f47f00b2db0 2023-11-26T07:24:13.954 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:13.955+0000 7f47d7fff640 1 -- 172.21.15.143:0/443578573 <== mon.0 v2:172.21.15.143:3300/0 4 ==== mgrmap(e 16) v1 ==== 85954+0+0 (secure 0 0 0) 0x7f47e00123f0 con 0x7f47f00b2db0 2023-11-26T07:24:13.954 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:13.955+0000 7f47d7fff640 1 --2- 172.21.15.143:0/443578573 >> [v2:172.21.15.188:6836/3401128553,v1:172.21.15.188:6837/3401128553] conn(0x7f47dc066590 0x7f47dc068a50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:24:13.954 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:13.955+0000 7f47d7fff640 1 -- 172.21.15.143:0/443578573 <== mon.0 v2:172.21.15.143:3300/0 5 ==== osd_map(103..103 src has 1..103) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f47e0084170 con 0x7f47f00b2db0 2023-11-26T07:24:13.955 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:13.955+0000 7f47eeffd640 1 --2- 172.21.15.143:0/443578573 >> [v2:172.21.15.188:6836/3401128553,v1:172.21.15.188:6837/3401128553] conn(0x7f47dc066590 0x7f47dc068a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:24:13.955 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:13.955+0000 7f47eeffd640 1 --2- 172.21.15.143:0/443578573 >> [v2:172.21.15.188:6836/3401128553,v1:172.21.15.188:6837/3401128553] conn(0x7f47dc066590 0x7f47dc068a50 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7f47e40ad000 tx=0x7f47e4008040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:24:13.958 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:13.959+0000 7f47d7fff640 1 -- 172.21.15.143:0/443578573 <== mon.0 v2:172.21.15.143:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172649 (secure 0 0 0) 0x7f47e00513e0 con 0x7f47f00b2db0 2023-11-26T07:24:14.142 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:14.139+0000 7f47f6157640 1 -- 172.21.15.143:0/443578573 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f47f0067680 con 0x7f47f00b2db0 2023-11-26T07:24:14.142 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:14.143+0000 7f47d7fff640 1 -- 172.21.15.143:0/443578573 <== mon.0 v2:172.21.15.143:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 65 v65) v1 ==== 94+0+4927 (secure 0 0 0) 0x7f47e0054a00 con 0x7f47f00b2db0 2023-11-26T07:24:14.142 INFO:teuthology.orchestra.run.smithi143.stdout: 2023-11-26T07:24:14.143 INFO:teuthology.orchestra.run.smithi143.stdout:{"epoch":65,"default_fscid":8,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6629,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.188:6834/1327398471","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6832","nonce":1327398471},{"type":"v1","addr":"172.21.15.188:6834","nonce":1327398471}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":60},{"gid":7453,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.188:6835/1640976426","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6833","nonce":1640976426},{"type":"v1","addr":"172.21.15.188:6835","nonce":1640976426}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":60},{"gid":7941,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.143:6834/2855111671","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6832","nonce":2855111671},{"type":"v1","addr":"172.21.15.143:6834","nonce":2855111671}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":60}],"filesystems":[{"mdsmap":{"epoch":65,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-11-26T07:24:08.295882+0000","modified":"2023-11-26T07:24:11.308119+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":7944},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7944":{"gid":7944,"name":"a","rank":0,"incarnation":61,"state":"up:active","state_seq":3,"addr":"172.21.15.143:6835/2571525464","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6833","nonce":2571525464},{"type":"v1","addr":"172.21.15.143:6835","nonce":2571525464}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[17],"metadata_pool":16,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":8}]} 2023-11-26T07:24:14.143 INFO:teuthology.orchestra.run.smithi143.stderr:dumped fsmap epoch 65 2023-11-26T07:24:14.144 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:14.143+0000 7f47f6157640 1 -- 172.21.15.143:0/443578573 >> [v2:172.21.15.188:6836/3401128553,v1:172.21.15.188:6837/3401128553] conn(0x7f47dc066590 msgr2=0x7f47dc068a50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:24:14.144 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:14.143+0000 7f47f6157640 1 --2- 172.21.15.143:0/443578573 >> [v2:172.21.15.188:6836/3401128553,v1:172.21.15.188:6837/3401128553] conn(0x7f47dc066590 0x7f47dc068a50 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7f47e40ad000 tx=0x7f47e4008040 comp rx=0 tx=0).stop 2023-11-26T07:24:14.145 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:14.143+0000 7f47f6157640 1 -- 172.21.15.143:0/443578573 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f47f00b2db0 msgr2=0x7f47f015c810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:24:14.145 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:14.143+0000 7f47f6157640 1 --2- 172.21.15.143:0/443578573 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f47f00b2db0 0x7f47f015c810 secure :-1 s=READY pgs=1316 cs=0 l=1 rev1=1 crypto rx=0x7f47e000d7c0 tx=0x7f47e000dc80 comp rx=0 tx=0).stop 2023-11-26T07:24:14.145 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:14.143+0000 7f47f6157640 1 -- 172.21.15.143:0/443578573 shutdown_connections 2023-11-26T07:24:14.145 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:14.143+0000 7f47f6157640 1 --2- 172.21.15.143:0/443578573 >> [v2:172.21.15.188:6836/3401128553,v1:172.21.15.188:6837/3401128553] conn(0x7f47dc066590 0x7f47dc068a50 unknown :-1 s=CLOSED pgs=132 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:24:14.145 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:14.143+0000 7f47f6157640 1 --2- 172.21.15.143:0/443578573 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f47f01596d0 0x7f47f0149a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:24:14.145 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:14.143+0000 7f47f6157640 1 --2- 172.21.15.143:0/443578573 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f47f00ce680 0x7f47f01483a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:24:14.145 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:14.143+0000 7f47f6157640 1 --2- 172.21.15.143:0/443578573 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f47f00b2db0 0x7f47f015c810 unknown :-1 s=CLOSED pgs=1316 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:24:14.145 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:14.143+0000 7f47f6157640 1 -- 172.21.15.143:0/443578573 >> 172.21.15.143:0/443578573 conn(0x7f47f006a890 msgr2=0x7f47f00b03d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:24:14.145 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:14.143+0000 7f47f6157640 1 -- 172.21.15.143:0/443578573 shutdown_connections 2023-11-26T07:24:14.145 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:14.143+0000 7f47f6157640 1 -- 172.21.15.143:0/443578573 wait complete. 2023-11-26T07:24:14.565 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 65, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2023-11-26T07:24:08.295882+0000', 'modified': '2023-11-26T07:24:11.308119+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 7944}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_7944': {'gid': 7944, 'name': 'a', 'rank': 0, 'incarnation': 61, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.143:6835/2571525464', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.143:6833', 'nonce': 2571525464}, {'type': 'v1', 'addr': '172.21.15.143:6835', 'nonce': 2571525464}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138322906710015, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}}}, 'data_pools': [17], 'metadata_pool': 16, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0} 2023-11-26T07:24:14.566 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2023-11-26T07:24:14.566 DEBUG:teuthology.orchestra.run.smithi143:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.a status 2023-11-26T07:24:14.705 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:14.703+0000 7f82493b8640 1 Processor -- start 2023-11-26T07:24:14.705 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:14.703+0000 7f82493b8640 1 -- start start 2023-11-26T07:24:14.706 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:14.707+0000 7f82493b8640 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7f8244143780 0x7f82440cdfe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:24:14.707 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:14.707+0000 7f82493b8640 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7f82440722c0 0x7f824415c5a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:24:14.707 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:14.707+0000 7f82493b8640 1 -- --> v1:172.21.15.188:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f82440ceb50 con 0x7f82440ce520 2023-11-26T07:24:14.707 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:14.707+0000 7f82493b8640 1 -- --> v2:172.21.15.143:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f824415c110 con 0x7f82440722c0 2023-11-26T07:24:14.707 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:14.707+0000 7f82493b8640 1 -- --> v2:172.21.15.188:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f824415b7e0 con 0x7f8244143780 2023-11-26T07:24:14.707 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:14.707+0000 7f82437fe640 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7f82440722c0 0x7f824415c5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:24:14.707 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:14.707+0000 7f82437fe640 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7f82440722c0 0x7f824415c5a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:54488/0 (socket says 172.21.15.143:54488) 2023-11-26T07:24:14.707 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:14.707+0000 7f82437fe640 1 -- 172.21.15.143:0/3393216598 learned_addr learned my addr 172.21.15.143:0/3393216598 (peer_addr_for_me v2:172.21.15.143:0/0) 2023-11-26T07:24:14.708 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:14.707+0000 7f8242ffd640 1 --2- 172.21.15.143:0/3393216598 >> v2:172.21.15.188:3300/0 conn(0x7f8244143780 0x7f82440cdfe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:24:14.708 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:14.707+0000 7f82437fe640 1 -- 172.21.15.143:0/3393216598 >> v2:172.21.15.188:3300/0 conn(0x7f8244143780 msgr2=0x7f82440cdfe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-26T07:24:14.708 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:14.707+0000 7f82437fe640 1 --2- 172.21.15.143:0/3393216598 >> v2:172.21.15.188:3300/0 conn(0x7f8244143780 0x7f82440cdfe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:24:14.708 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:14.707+0000 7f82437fe640 1 -- 172.21.15.143:0/3393216598 >> v1:172.21.15.188:6789/0 conn(0x7f82440ce520 legacy=0x7f8244071bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-26T07:24:14.708 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:14.707+0000 7f82437fe640 1 -- 172.21.15.143:0/3393216598 --> v2:172.21.15.143:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f824415bac0 con 0x7f82440722c0 2023-11-26T07:24:14.708 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:14.707+0000 7f8242ffd640 1 --2- 172.21.15.143:0/3393216598 >> v2:172.21.15.188:3300/0 conn(0x7f8244143780 0x7f82440cdfe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-26T07:24:14.710 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:14.711+0000 7f82437fe640 1 --2- 172.21.15.143:0/3393216598 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f82440722c0 0x7f824415c5a0 secure :-1 s=READY pgs=1317 cs=0 l=1 rev1=1 crypto rx=0x7f824406be30 tx=0x7f823c01c4c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=915977d3cc307feb server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:24:14.710 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:14.711+0000 7f82427fc640 1 -- 172.21.15.143:0/3393216598 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f823c0ac630 con 0x7f82440722c0 2023-11-26T07:24:14.710 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:14.711+0000 7f82427fc640 1 -- 172.21.15.143:0/3393216598 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f823c0ac790 con 0x7f82440722c0 2023-11-26T07:24:14.710 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:14.711+0000 7f82427fc640 1 -- 172.21.15.143:0/3393216598 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f823c0acb20 con 0x7f82440722c0 2023-11-26T07:24:14.711 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:14.711+0000 7f82493b8640 1 -- 172.21.15.143:0/3393216598 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f82440722c0 msgr2=0x7f824415c5a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:24:14.711 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:14.711+0000 7f82493b8640 1 --2- 172.21.15.143:0/3393216598 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f82440722c0 0x7f824415c5a0 secure :-1 s=READY pgs=1317 cs=0 l=1 rev1=1 crypto rx=0x7f824406be30 tx=0x7f823c01c4c0 comp rx=0 tx=0).stop 2023-11-26T07:24:14.711 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:14.711+0000 7f82493b8640 1 -- 172.21.15.143:0/3393216598 shutdown_connections 2023-11-26T07:24:14.712 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:14.711+0000 7f82493b8640 1 --2- 172.21.15.143:0/3393216598 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f82440722c0 0x7f824415c5a0 unknown :-1 s=CLOSED pgs=1317 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:24:14.712 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:14.711+0000 7f82493b8640 1 --2- 172.21.15.143:0/3393216598 >> v2:172.21.15.188:3300/0 conn(0x7f8244143780 0x7f82440cdfe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:24:14.712 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:14.711+0000 7f82493b8640 1 -- 172.21.15.143:0/3393216598 >> 172.21.15.143:0/3393216598 conn(0x7f824406c3f0 msgr2=0x7f824406c800 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:24:14.712 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:14.711+0000 7f82493b8640 1 -- 172.21.15.143:0/3393216598 shutdown_connections 2023-11-26T07:24:14.712 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:14.711+0000 7f82493b8640 1 -- 172.21.15.143:0/3393216598 wait complete. 2023-11-26T07:24:14.712 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:14.711+0000 7f82493b8640 1 Processor -- start 2023-11-26T07:24:14.713 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:14.711+0000 7f82493b8640 1 -- start start 2023-11-26T07:24:14.715 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:14.715+0000 7f82493b8640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f82440722c0 0x7f82441592c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:24:14.715 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:14.715+0000 7f82493b8640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f82440ce520 0x7f8244139420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:24:14.715 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:14.715+0000 7f82493b8640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f8244143780 0x7f8244132780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:24:14.715 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:14.715+0000 7f82493b8640 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8244138060 con 0x7f8244143780 2023-11-26T07:24:14.715 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:14.715+0000 7f82493b8640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8244159800 con 0x7f82440722c0 2023-11-26T07:24:14.715 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:14.715+0000 7f82493b8640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8244131490 con 0x7f82440ce520 2023-11-26T07:24:14.715 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:14.715+0000 7f823a7fc640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f82440ce520 0x7f8244139420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:24:14.715 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:14.715+0000 7f82437fe640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f8244143780 0x7f8244132780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:24:14.715 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:14.715+0000 7f8242ffd640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f82440722c0 0x7f82441592c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:24:14.715 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:14.715+0000 7f823a7fc640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f82440ce520 0x7f8244139420 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.188:3301/0 says I am v2:172.21.15.143:36762/0 (socket says 172.21.15.143:36762) 2023-11-26T07:24:14.716 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:14.715+0000 7f82437fe640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f8244143780 0x7f8244132780 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:54502/0 (socket says 172.21.15.143:54502) 2023-11-26T07:24:14.716 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:14.715+0000 7f823a7fc640 1 -- 172.21.15.143:0/3741046815 learned_addr learned my addr 172.21.15.143:0/3741046815 (peer_addr_for_me v2:172.21.15.143:0/0) 2023-11-26T07:24:14.716 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:14.715+0000 7f82437fe640 1 -- 172.21.15.143:0/3741046815 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f82440ce520 msgr2=0x7f8244139420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:24:14.716 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:14.715+0000 7f82437fe640 1 --2- 172.21.15.143:0/3741046815 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f82440ce520 0x7f8244139420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:24:14.716 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:14.715+0000 7f82437fe640 1 -- 172.21.15.143:0/3741046815 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f82440722c0 msgr2=0x7f82441592c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:24:14.716 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:14.715+0000 7f82437fe640 1 --2- 172.21.15.143:0/3741046815 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f82440722c0 0x7f82441592c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:24:14.716 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:14.715+0000 7f82437fe640 1 -- 172.21.15.143:0/3741046815 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f823c0094d0 con 0x7f8244143780 2023-11-26T07:24:14.716 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:14.715+0000 7f82437fe640 1 --2- 172.21.15.143:0/3741046815 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f8244143780 0x7f8244132780 secure :-1 s=READY pgs=1318 cs=0 l=1 rev1=1 crypto rx=0x7f823c0ac000 tx=0x7f823c019260 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:24:14.717 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:14.715+0000 7f823bfff640 1 -- 172.21.15.143:0/3741046815 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f823c0b5640 con 0x7f8244143780 2023-11-26T07:24:14.717 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:14.715+0000 7f823bfff640 1 -- 172.21.15.143:0/3741046815 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f823c0b57a0 con 0x7f8244143780 2023-11-26T07:24:14.717 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:14.715+0000 7f82493b8640 1 -- 172.21.15.143:0/3741046815 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f82441322f0 con 0x7f8244143780 2023-11-26T07:24:14.717 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:14.715+0000 7f823bfff640 1 -- 172.21.15.143:0/3741046815 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f823c0b5b50 con 0x7f8244143780 2023-11-26T07:24:14.717 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:14.715+0000 7f82493b8640 1 -- 172.21.15.143:0/3741046815 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8244131ca0 con 0x7f8244143780 2023-11-26T07:24:14.718 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:14.719+0000 7f823bfff640 1 -- 172.21.15.143:0/3741046815 <== mon.0 v2:172.21.15.143:3300/0 4 ==== mgrmap(e 16) v1 ==== 85954+0+0 (secure 0 0 0) 0x7f823c0b3070 con 0x7f8244143780 2023-11-26T07:24:14.719 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:14.719+0000 7f823bfff640 1 --2- 172.21.15.143:0/3741046815 >> [v2:172.21.15.188:6836/3401128553,v1:172.21.15.188:6837/3401128553] conn(0x7f82240665c0 0x7f8224068a80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:24:14.719 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:14.719+0000 7f823bfff640 1 -- 172.21.15.143:0/3741046815 <== mon.0 v2:172.21.15.143:3300/0 5 ==== osd_map(103..103 src has 1..103) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f823c11dea0 con 0x7f8244143780 2023-11-26T07:24:14.719 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:14.719+0000 7f824c71c000 1 Processor -- start 2023-11-26T07:24:14.719 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:14.719+0000 7f8242ffd640 1 --2- 172.21.15.143:0/3741046815 >> [v2:172.21.15.188:6836/3401128553,v1:172.21.15.188:6837/3401128553] conn(0x7f82240665c0 0x7f8224068a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:24:14.720 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:14.719+0000 7f824c71c000 1 -- start start 2023-11-26T07:24:14.720 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:14.719+0000 7f8242ffd640 1 --2- 172.21.15.143:0/3741046815 >> [v2:172.21.15.188:6836/3401128553,v1:172.21.15.188:6837/3401128553] conn(0x7f82240665c0 0x7f8224068a80 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7f824415a370 tx=0x7f8234007770 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:24:14.722 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:14.719+0000 7f824c71c000 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55f126c14c90 0x55f126c15070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:24:14.722 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:14.719+0000 7f824c71c000 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x55f126c155b0 0x55f126bcade0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:24:14.722 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:14.719+0000 7f824c71c000 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55f126bcb320 0x55f126bcb700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:24:14.722 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:14.719+0000 7f824c71c000 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f126a35130 con 0x55f126c14c90 2023-11-26T07:24:14.722 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:14.719+0000 7f824c71c000 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f126a229a0 con 0x55f126bcb320 2023-11-26T07:24:14.722 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:14.719+0000 7f824c71c000 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x55f126a1fed0 con 0x55f126c155b0 2023-11-26T07:24:14.722 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:14.719+0000 7f8242ffd640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55f126c14c90 0x55f126c15070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:24:14.722 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:14.719+0000 7f8242ffd640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55f126c14c90 0x55f126c15070 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:54512/0 (socket says 172.21.15.143:54512) 2023-11-26T07:24:14.722 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:14.719+0000 7f8242ffd640 1 -- 172.21.15.143:0/3046112063 learned_addr learned my addr 172.21.15.143:0/3046112063 (peer_addr_for_me v2:172.21.15.143:0/0) 2023-11-26T07:24:14.722 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:14.719+0000 7f82437fe640 1 --2- 172.21.15.143:0/3046112063 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55f126bcb320 0x55f126bcb700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:24:14.723 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:14.719+0000 7f823a7fc640 1 --2- 172.21.15.143:0/3046112063 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x55f126c155b0 0x55f126bcade0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:24:14.723 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:14.719+0000 7f8242ffd640 1 -- 172.21.15.143:0/3046112063 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x55f126c155b0 msgr2=0x55f126bcade0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-26T07:24:14.723 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:14.719+0000 7f8242ffd640 1 --2- 172.21.15.143:0/3046112063 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x55f126c155b0 0x55f126bcade0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:24:14.723 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:14.719+0000 7f8242ffd640 1 -- 172.21.15.143:0/3046112063 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55f126bcb320 msgr2=0x55f126bcb700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-26T07:24:14.723 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:14.719+0000 7f8242ffd640 1 --2- 172.21.15.143:0/3046112063 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55f126bcb320 0x55f126bcb700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:24:14.723 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:14.719+0000 7f8242ffd640 1 -- 172.21.15.143:0/3046112063 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55f126713890 con 0x55f126c14c90 2023-11-26T07:24:14.723 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:14.719+0000 7f8242ffd640 1 --2- 172.21.15.143:0/3046112063 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55f126c14c90 0x55f126c15070 secure :-1 s=READY pgs=1319 cs=0 l=1 rev1=1 crypto rx=0x7f823400d620 tx=0x7f8234012000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c183902fc2d35337 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:24:14.723 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:14.723+0000 7f8238ff9640 1 -- 172.21.15.143:0/3046112063 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8234024430 con 0x55f126c14c90 2023-11-26T07:24:14.723 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:14.723+0000 7f8238ff9640 1 -- 172.21.15.143:0/3046112063 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f823401f070 con 0x55f126c14c90 2023-11-26T07:24:14.723 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:14.723+0000 7f8238ff9640 1 -- 172.21.15.143:0/3046112063 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f82340247b0 con 0x55f126c14c90 2023-11-26T07:24:14.727 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:14.723+0000 7f824c71c000 1 -- 172.21.15.143:0/3046112063 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55f126c14c90 msgr2=0x55f126c15070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:24:14.727 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:14.723+0000 7f824c71c000 1 --2- 172.21.15.143:0/3046112063 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55f126c14c90 0x55f126c15070 secure :-1 s=READY pgs=1319 cs=0 l=1 rev1=1 crypto rx=0x7f823400d620 tx=0x7f8234012000 comp rx=0 tx=0).stop 2023-11-26T07:24:14.727 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:14.723+0000 7f824c71c000 1 -- 172.21.15.143:0/3046112063 shutdown_connections 2023-11-26T07:24:14.727 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:14.723+0000 7f824c71c000 1 --2- 172.21.15.143:0/3046112063 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55f126bcb320 0x55f126bcb700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:24:14.727 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:14.723+0000 7f824c71c000 1 --2- 172.21.15.143:0/3046112063 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x55f126c155b0 0x55f126bcade0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:24:14.727 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:14.723+0000 7f824c71c000 1 --2- 172.21.15.143:0/3046112063 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55f126c14c90 0x55f126c15070 unknown :-1 s=CLOSED pgs=1319 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:24:14.727 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:14.723+0000 7f824c71c000 1 -- 172.21.15.143:0/3046112063 >> 172.21.15.143:0/3046112063 conn(0x55f126bc72d0 msgr2=0x55f126bc96c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:24:14.727 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:14.723+0000 7f824c71c000 1 -- 172.21.15.143:0/3046112063 shutdown_connections 2023-11-26T07:24:14.727 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:14.723+0000 7f824c71c000 1 -- 172.21.15.143:0/3046112063 wait complete. 2023-11-26T07:24:14.727 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:14.723+0000 7f824c71c000 1 -- start start 2023-11-26T07:24:14.728 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:14.723+0000 7f824c71c000 1 Processor -- start 2023-11-26T07:24:14.728 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:14.723+0000 7f824c71c000 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55f126bcb320 0x55f126bf9ab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:24:14.728 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:14.723+0000 7f824c71c000 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55f126c14c90 0x55f126bf9ff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:24:14.728 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:14.723+0000 7f824c71c000 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x55f126c155b0 0x55f126bfa530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:24:14.728 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:14.723+0000 7f824c71c000 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f126a1b7c0 con 0x55f126bcb320 2023-11-26T07:24:14.728 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:14.723+0000 7f824c71c000 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f1269d3bf0 con 0x55f126c14c90 2023-11-26T07:24:14.728 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:14.723+0000 7f824c71c000 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x55f1269d9a90 con 0x55f126c155b0 2023-11-26T07:24:14.728 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:14.723+0000 7f824c71c000 10 client.0 _ll_register_callbacks cb 0x55f126bb63c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-11-26T07:24:14.728 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:14.723+0000 7f8242ffd640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55f126bcb320 0x55f126bf9ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:24:14.728 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:14.723+0000 7f8242ffd640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55f126bcb320 0x55f126bf9ab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:54524/0 (socket says 172.21.15.143:54524) 2023-11-26T07:24:14.728 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:14.723+0000 7f8242ffd640 1 -- 172.21.15.143:0/1700708177 learned_addr learned my addr 172.21.15.143:0/1700708177 (peer_addr_for_me v2:172.21.15.143:0/0) 2023-11-26T07:24:14.728 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:14.723+0000 7f82437fe640 1 --2- 172.21.15.143:0/1700708177 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x55f126c155b0 0x55f126bfa530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:24:14.729 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:14.723+0000 7f823a7fc640 1 --2- 172.21.15.143:0/1700708177 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55f126c14c90 0x55f126bf9ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:24:14.729 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:14.723+0000 7f8242ffd640 1 -- 172.21.15.143:0/1700708177 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x55f126c155b0 msgr2=0x55f126bfa530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-26T07:24:14.729 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:14.723+0000 7f8242ffd640 1 --2- 172.21.15.143:0/1700708177 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x55f126c155b0 0x55f126bfa530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:24:14.729 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:14.723+0000 7f8242ffd640 1 -- 172.21.15.143:0/1700708177 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55f126c14c90 msgr2=0x55f126bf9ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-26T07:24:14.729 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:14.723+0000 7f8242ffd640 1 --2- 172.21.15.143:0/1700708177 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55f126c14c90 0x55f126bf9ff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:24:14.729 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:14.723+0000 7f8242ffd640 1 -- 172.21.15.143:0/1700708177 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8234015070 con 0x55f126bcb320 2023-11-26T07:24:14.729 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:14.723+0000 7f8242ffd640 1 --2- 172.21.15.143:0/1700708177 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55f126bcb320 0x55f126bf9ab0 secure :-1 s=READY pgs=1320 cs=0 l=1 rev1=1 crypto rx=0x7f823400fd60 tx=0x7f8234004ac0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=12f827ca781b05b server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:24:14.729 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:14.723+0000 7f8221ffb640 10 client.0 ms_handle_connect on v2:172.21.15.143:3300/0 2023-11-26T07:24:14.729 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:14.723+0000 7f8221ffb640 1 -- 172.21.15.143:0/1700708177 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f82340257d0 con 0x55f126bcb320 2023-11-26T07:24:14.729 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:14.727+0000 7f8221ffb640 1 -- 172.21.15.143:0/1700708177 <== mon.0 v2:172.21.15.143:3300/0 2 ==== osd_map(103..103 src has 1..103) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f8234025930 con 0x55f126bcb320 2023-11-26T07:24:14.729 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:14.727+0000 7f824c71c000 1 -- 172.21.15.143:0/1700708177 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55f1269dd470 con 0x55f126bcb320 2023-11-26T07:24:14.729 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:14.727+0000 7f8221ffb640 1 -- 172.21.15.143:0/1700708177 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_get_version_reply(handle=1 version=65) v2 ==== 24+0+0 (secure 0 0 0) 0x7f8234025af0 con 0x55f126bcb320 2023-11-26T07:24:14.729 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:14.727+0000 7f824c71c000 10 client.8082 fetch_fsmap learned FSMap version 65 2023-11-26T07:24:14.730 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:14.727+0000 7f824c71c000 1 -- 172.21.15.143:0/1700708177 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,fsmap=65,monmap=2+}) v3 -- 0x55f1269dc190 con 0x55f126bcb320 2023-11-26T07:24:14.730 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:14.727+0000 7f8221ffb640 1 -- 172.21.15.143:0/1700708177 <== mon.0 v2:172.21.15.143:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x55f1269dd470 con 0x55f126bcb320 2023-11-26T07:24:14.730 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:14.727+0000 7f8221ffb640 1 -- 172.21.15.143:0/1700708177 <== mon.0 v2:172.21.15.143:3300/0 5 ==== fsmap(e 65) v1 ==== 3294+0+0 (secure 0 0 0) 0x7f8234025c50 con 0x55f126bcb320 2023-11-26T07:24:14.730 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:14.727+0000 7f824c71c000 10 client.8082 fetch_fsmap finished waiting for FSMap version 65 2023-11-26T07:24:14.730 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:14.727+0000 7f824c71c000 10 client.8082 resolve_mds: resolved name 'a' to daemon mds.a 2023-11-26T07:24:14.730 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:14.727+0000 7f824c71c000 20 client.8082 populate_metadata read hostname 'smithi143' 2023-11-26T07:24:14.730 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:14.727+0000 7f824c71c000 1 --2- 172.21.15.143:0/1700708177 >> [v2:172.21.15.143:6833/2571525464,v1:172.21.15.143:6835/2571525464] conn(0x55f126bfaf40 0x55f126c26f50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:24:14.730 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:14.727+0000 7f824c71c000 4 client.8082 mds_command: new command op to 7944 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2023-11-26T07:24:14.730 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:14.727+0000 7f824c71c000 1 -- 172.21.15.143:0/1700708177 --> [v2:172.21.15.143:6833/2571525464,v1:172.21.15.143:6835/2571525464] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55f1269e0b50 con 0x55f126bfaf40 2023-11-26T07:24:14.730 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:14.727+0000 7f823a7fc640 1 --2- 172.21.15.143:0/1700708177 >> [v2:172.21.15.143:6833/2571525464,v1:172.21.15.143:6835/2571525464] conn(0x55f126bfaf40 0x55f126c26f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:24:14.730 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:14.727+0000 7f823a7fc640 1 --2- 172.21.15.143:0/1700708177 >> [v2:172.21.15.143:6833/2571525464,v1:172.21.15.143:6835/2571525464] conn(0x55f126bfaf40 0x55f126c26f50 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a906d0c958d1efab server_cookie=6d60e8abb514bc83 in_seq=0 out_seq=0 2023-11-26T07:24:14.730 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:14.727+0000 7f8221ffb640 10 client.8082 ms_handle_connect on v2:172.21.15.143:6833/2571525464 2023-11-26T07:24:14.731 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:14.727+0000 7f8221ffb640 1 -- 172.21.15.143:0/1700708177 <== mds.0 v2:172.21.15.143:6833/2571525464 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x55f1269e0b50 con 0x55f126bfaf40 2023-11-26T07:24:14.732 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:14.727+0000 7f8221ffb640 10 client.8082 handle_command_reply: tid=0 2023-11-26T07:24:14.732 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:14.727+0000 7f824c71c000 1 client.8082 shutdown 2023-11-26T07:24:14.732 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:14.727+0000 7f824c71c000 1 -- 172.21.15.143:0/1700708177 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55f126bcb320 msgr2=0x55f126bf9ab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:24:14.732 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:14.727+0000 7f824c71c000 1 --2- 172.21.15.143:0/1700708177 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55f126bcb320 0x55f126bf9ab0 secure :-1 s=READY pgs=1320 cs=0 l=1 rev1=1 crypto rx=0x7f823400fd60 tx=0x7f8234004ac0 comp rx=0 tx=0).stop 2023-11-26T07:24:14.732 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:14.727+0000 7f824c71c000 1 -- 172.21.15.143:0/1700708177 shutdown_connections 2023-11-26T07:24:14.732 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:14.727+0000 7f824c71c000 1 --2- 172.21.15.143:0/1700708177 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x55f126c155b0 0x55f126bfa530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:24:14.732 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:14.727+0000 7f824c71c000 1 --2- 172.21.15.143:0/1700708177 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55f126c14c90 0x55f126bf9ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:24:14.732 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:14.727+0000 7f824c71c000 1 --2- 172.21.15.143:0/1700708177 >> [v2:172.21.15.143:6833/2571525464,v1:172.21.15.143:6835/2571525464] conn(0x55f126bfaf40 0x55f126c26f50 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:24:14.732 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:14.727+0000 7f824c71c000 1 --2- 172.21.15.143:0/1700708177 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55f126bcb320 0x55f126bf9ab0 unknown :-1 s=CLOSED pgs=1320 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:24:14.732 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:14.727+0000 7f8221ffb640 0 client.8082 ms_handle_reset on v2:172.21.15.143:6833/2571525464 2023-11-26T07:24:14.732 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:14.727+0000 7f824c71c000 1 -- 172.21.15.143:0/1700708177 >> 172.21.15.143:0/1700708177 conn(0x55f126bc72d0 msgr2=0x55f126bc96c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:24:14.732 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:14.727+0000 7f824c71c000 1 -- 172.21.15.143:0/1700708177 shutdown_connections 2023-11-26T07:24:14.732 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:14.727+0000 7f824c71c000 1 -- 172.21.15.143:0/1700708177 wait complete. 2023-11-26T07:24:14.733 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:14.727+0000 7f824c71c000 20 client.8082 trim_cache size 0 max 16384 2023-11-26T07:24:14.747 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:14.743+0000 7f824c71c000 1 Processor -- start 2023-11-26T07:24:14.747 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:14.743+0000 7f824c71c000 1 -- start start 2023-11-26T07:24:14.747 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:14.743+0000 7f824c71c000 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55f126bc72d0 0x55f126c133e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:24:14.747 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:14.743+0000 7f824c71c000 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x55f126bcb320 0x55f126c13920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:24:14.747 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:14.743+0000 7f824c71c000 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55f126c14c90 0x55f126bcb700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:24:14.747 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:14.743+0000 7f824c71c000 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f126bc7190 con 0x55f126bc72d0 2023-11-26T07:24:14.747 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:14.743+0000 7f824c71c000 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f1266d4940 con 0x55f126c14c90 2023-11-26T07:24:14.748 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:14.743+0000 7f824c71c000 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x55f126a1e320 con 0x55f126bcb320 2023-11-26T07:24:14.748 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:14.743+0000 7f8242ffd640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55f126bc72d0 0x55f126c133e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:24:14.748 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:14.743+0000 7f8242ffd640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55f126bc72d0 0x55f126c133e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:54530/0 (socket says 172.21.15.143:54530) 2023-11-26T07:24:14.748 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:14.743+0000 7f8242ffd640 1 -- 172.21.15.143:0/3715477545 learned_addr learned my addr 172.21.15.143:0/3715477545 (peer_addr_for_me v2:172.21.15.143:0/0) 2023-11-26T07:24:14.748 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:14.743+0000 7f82437fe640 1 --2- 172.21.15.143:0/3715477545 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55f126c14c90 0x55f126bcb700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:24:14.748 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:14.743+0000 7f823a7fc640 1 --2- 172.21.15.143:0/3715477545 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x55f126bcb320 0x55f126c13920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:24:14.748 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:14.743+0000 7f8242ffd640 1 -- 172.21.15.143:0/3715477545 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x55f126bcb320 msgr2=0x55f126c13920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-26T07:24:14.748 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:14.743+0000 7f8242ffd640 1 --2- 172.21.15.143:0/3715477545 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x55f126bcb320 0x55f126c13920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:24:14.748 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:14.743+0000 7f8242ffd640 1 -- 172.21.15.143:0/3715477545 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55f126c14c90 msgr2=0x55f126bcb700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-26T07:24:14.748 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:14.743+0000 7f8242ffd640 1 --2- 172.21.15.143:0/3715477545 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55f126c14c90 0x55f126bcb700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:24:14.748 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:14.743+0000 7f8242ffd640 1 -- 172.21.15.143:0/3715477545 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55f1269c2490 con 0x55f126bc72d0 2023-11-26T07:24:14.749 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:14.743+0000 7f8242ffd640 1 --2- 172.21.15.143:0/3715477545 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55f126bc72d0 0x55f126c133e0 secure :-1 s=READY pgs=1321 cs=0 l=1 rev1=1 crypto rx=0x7f8234027000 tx=0x7f8234029000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5158e7c6824eeceb server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:24:14.749 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:14.747+0000 7f82397fa640 1 -- 172.21.15.143:0/3715477545 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f823400dae0 con 0x55f126bc72d0 2023-11-26T07:24:14.749 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:14.747+0000 7f82397fa640 1 -- 172.21.15.143:0/3715477545 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f823401f040 con 0x55f126bc72d0 2023-11-26T07:24:14.749 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:14.747+0000 7f82397fa640 1 -- 172.21.15.143:0/3715477545 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8234025260 con 0x55f126bc72d0 2023-11-26T07:24:14.749 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:14.747+0000 7f824c71c000 1 -- 172.21.15.143:0/3715477545 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55f126bc72d0 msgr2=0x55f126c133e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:24:14.749 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:14.747+0000 7f824c71c000 1 --2- 172.21.15.143:0/3715477545 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55f126bc72d0 0x55f126c133e0 secure :-1 s=READY pgs=1321 cs=0 l=1 rev1=1 crypto rx=0x7f8234027000 tx=0x7f8234029000 comp rx=0 tx=0).stop 2023-11-26T07:24:14.749 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:14.747+0000 7f824c71c000 1 -- 172.21.15.143:0/3715477545 shutdown_connections 2023-11-26T07:24:14.749 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:14.747+0000 7f824c71c000 1 --2- 172.21.15.143:0/3715477545 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55f126c14c90 0x55f126bcb700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:24:14.749 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:14.747+0000 7f824c71c000 1 --2- 172.21.15.143:0/3715477545 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x55f126bcb320 0x55f126c13920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:24:14.749 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:14.747+0000 7f824c71c000 1 --2- 172.21.15.143:0/3715477545 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55f126bc72d0 0x55f126c133e0 unknown :-1 s=CLOSED pgs=1321 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:24:14.749 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:14.747+0000 7f824c71c000 1 -- 172.21.15.143:0/3715477545 >> 172.21.15.143:0/3715477545 conn(0x55f126bf7aa0 msgr2=0x55f126bc96c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:24:14.749 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:14.747+0000 7f824c71c000 1 -- 172.21.15.143:0/3715477545 shutdown_connections 2023-11-26T07:24:14.749 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:14.747+0000 7f824c71c000 1 -- 172.21.15.143:0/3715477545 wait complete. 2023-11-26T07:24:14.750 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:14.747+0000 7f824c71c000 1 -- start start 2023-11-26T07:24:14.753 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:14.747+0000 7f824c71c000 1 Processor -- start 2023-11-26T07:24:14.753 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:14.747+0000 7f824c71c000 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x55f126bc72d0 0x55f126c27fa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:24:14.753 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:14.747+0000 7f824c71c000 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55f126bcb320 0x55f126c284e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:24:14.753 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:14.747+0000 7f824c71c000 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55f126c14c90 0x55f126c329e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:24:14.753 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:14.747+0000 7f824c71c000 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f126a17130 con 0x55f126c14c90 2023-11-26T07:24:14.753 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:14.747+0000 7f824c71c000 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f1269da8f0 con 0x55f126bcb320 2023-11-26T07:24:14.753 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:14.747+0000 7f824c71c000 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x55f126aa2fa0 con 0x55f126bc72d0 2023-11-26T07:24:14.753 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:14.747+0000 7f824c71c000 10 client.0 _ll_register_callbacks cb 0x55f126a69c00 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-11-26T07:24:14.753 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:14.747+0000 7f82437fe640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55f126c14c90 0x55f126c329e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:24:14.754 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:14.747+0000 7f82437fe640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55f126c14c90 0x55f126c329e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:54538/0 (socket says 172.21.15.143:54538) 2023-11-26T07:24:14.754 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:14.747+0000 7f82437fe640 1 -- 172.21.15.143:0/3408546160 learned_addr learned my addr 172.21.15.143:0/3408546160 (peer_addr_for_me v2:172.21.15.143:0/0) 2023-11-26T07:24:14.754 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:14.747+0000 7f8242ffd640 1 --2- 172.21.15.143:0/3408546160 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x55f126bc72d0 0x55f126c27fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:24:14.754 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:14.747+0000 7f823a7fc640 1 --2- 172.21.15.143:0/3408546160 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55f126bcb320 0x55f126c284e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:24:14.754 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:14.747+0000 7f82437fe640 1 -- 172.21.15.143:0/3408546160 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x55f126bc72d0 msgr2=0x55f126c27fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-26T07:24:14.754 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:14.747+0000 7f82437fe640 1 --2- 172.21.15.143:0/3408546160 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x55f126bc72d0 0x55f126c27fa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:24:14.754 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:14.747+0000 7f82437fe640 1 -- 172.21.15.143:0/3408546160 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55f126bcb320 msgr2=0x55f126c284e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-26T07:24:14.754 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:14.747+0000 7f82437fe640 1 --2- 172.21.15.143:0/3408546160 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55f126bcb320 0x55f126c284e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:24:14.754 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:14.747+0000 7f82437fe640 1 -- 172.21.15.143:0/3408546160 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55f1269dc190 con 0x55f126c14c90 2023-11-26T07:24:14.754 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:14.747+0000 7f82437fe640 1 --2- 172.21.15.143:0/3408546160 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55f126c14c90 0x55f126c329e0 secure :-1 s=READY pgs=1322 cs=0 l=1 rev1=1 crypto rx=0x7f823c0eb370 tx=0x7f823c0f2fd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c718faa3557d38e server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:24:14.754 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:14.747+0000 7f8221ffb640 10 client.0 ms_handle_connect on v2:172.21.15.143:3300/0 2023-11-26T07:24:14.754 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:14.747+0000 7f8221ffb640 1 -- 172.21.15.143:0/3408546160 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f823c0ee560 con 0x55f126c14c90 2023-11-26T07:24:14.755 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:14.747+0000 7f8221ffb640 1 -- 172.21.15.143:0/3408546160 <== mon.0 v2:172.21.15.143:3300/0 2 ==== osd_map(103..103 src has 1..103) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f823c0f2070 con 0x55f126c14c90 2023-11-26T07:24:14.755 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:14.747+0000 7f824c71c000 1 -- 172.21.15.143:0/3408546160 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55f126a1a110 con 0x55f126c14c90 2023-11-26T07:24:14.755 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:14.747+0000 7f8221ffb640 1 -- 172.21.15.143:0/3408546160 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_get_version_reply(handle=1 version=65) v2 ==== 24+0+0 (secure 0 0 0) 0x7f823c11c530 con 0x55f126c14c90 2023-11-26T07:24:14.755 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:14.747+0000 7f824c71c000 10 client.8088 fetch_fsmap learned FSMap version 65 2023-11-26T07:24:14.755 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:14.747+0000 7f824c71c000 1 -- 172.21.15.143:0/3408546160 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,fsmap=65,monmap=2+}) v3 -- 0x55f126713cc0 con 0x55f126c14c90 2023-11-26T07:24:14.755 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:14.751+0000 7f8221ffb640 1 -- 172.21.15.143:0/3408546160 <== mon.0 v2:172.21.15.143:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x55f126a1a110 con 0x55f126c14c90 2023-11-26T07:24:14.755 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:14.751+0000 7f8221ffb640 1 -- 172.21.15.143:0/3408546160 <== mon.0 v2:172.21.15.143:3300/0 5 ==== fsmap(e 65) v1 ==== 3294+0+0 (secure 0 0 0) 0x7f823c0c5020 con 0x55f126c14c90 2023-11-26T07:24:14.755 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:14.751+0000 7f824c71c000 10 client.8088 fetch_fsmap finished waiting for FSMap version 65 2023-11-26T07:24:14.755 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:14.751+0000 7f824c71c000 10 client.8088 resolve_mds: resolved name 'a' to daemon mds.a 2023-11-26T07:24:14.755 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:14.751+0000 7f824c71c000 20 client.8088 populate_metadata read hostname 'smithi143' 2023-11-26T07:24:14.755 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:14.751+0000 7f824c71c000 1 --2- 172.21.15.143:0/3408546160 >> [v2:172.21.15.143:6833/2571525464,v1:172.21.15.143:6835/2571525464] conn(0x55f126c155b0 0x55f126c33370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:24:14.755 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:14.751+0000 7f824c71c000 4 client.8088 mds_command: new command op to 7944 tid=0 multi_id=0 [{"prefix": "status"}] 2023-11-26T07:24:14.755 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:14.751+0000 7f824c71c000 1 -- 172.21.15.143:0/3408546160 --> [v2:172.21.15.143:6833/2571525464,v1:172.21.15.143:6835/2571525464] -- command(tid 0: {"prefix": "status"}) v1 -- 0x55f126a499a0 con 0x55f126c155b0 2023-11-26T07:24:14.756 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:14.751+0000 7f8242ffd640 1 --2- 172.21.15.143:0/3408546160 >> [v2:172.21.15.143:6833/2571525464,v1:172.21.15.143:6835/2571525464] conn(0x55f126c155b0 0x55f126c33370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:24:14.756 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:14.751+0000 7f8221ffb640 10 client.8088 ms_handle_connect on v2:172.21.15.143:6833/2571525464 2023-11-26T07:24:14.756 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:14.751+0000 7f8242ffd640 1 --2- 172.21.15.143:0/3408546160 >> [v2:172.21.15.143:6833/2571525464,v1:172.21.15.143:6835/2571525464] conn(0x55f126c155b0 0x55f126c33370 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f1589f861d43786e server_cookie=ba81c63bd1f6cdd8 in_seq=0 out_seq=0 2023-11-26T07:24:14.756 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:14.751+0000 7f8221ffb640 1 -- 172.21.15.143:0/3408546160 <== mds.0 v2:172.21.15.143:6833/2571525464 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+333 (crc 0 0 0) 0x55f126a499a0 con 0x55f126c155b0 2023-11-26T07:24:14.756 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:14.751+0000 7f8221ffb640 10 client.8088 handle_command_reply: tid=0 2023-11-26T07:24:14.756 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:14.751+0000 7f824c71c000 1 client.8088 shutdown 2023-11-26T07:24:14.756 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:14.751+0000 7f824c71c000 1 -- 172.21.15.143:0/3408546160 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55f126c14c90 msgr2=0x55f126c329e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:24:14.756 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:14.751+0000 7f824c71c000 1 --2- 172.21.15.143:0/3408546160 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55f126c14c90 0x55f126c329e0 secure :-1 s=READY pgs=1322 cs=0 l=1 rev1=1 crypto rx=0x7f823c0eb370 tx=0x7f823c0f2fd0 comp rx=0 tx=0).stop 2023-11-26T07:24:14.756 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:14.751+0000 7f824c71c000 1 -- 172.21.15.143:0/3408546160 shutdown_connections 2023-11-26T07:24:14.756 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:14.751+0000 7f824c71c000 1 --2- 172.21.15.143:0/3408546160 >> [v2:172.21.15.143:6833/2571525464,v1:172.21.15.143:6835/2571525464] conn(0x55f126c155b0 0x55f126c33370 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:24:14.756 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:14.751+0000 7f824c71c000 1 --2- 172.21.15.143:0/3408546160 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55f126c14c90 0x55f126c329e0 unknown :-1 s=CLOSED pgs=1322 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:24:14.756 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:14.751+0000 7f824c71c000 1 --2- 172.21.15.143:0/3408546160 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55f126bcb320 0x55f126c284e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:24:14.757 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:14.751+0000 7f8221ffb640 0 client.8088 ms_handle_reset on v2:172.21.15.143:6833/2571525464 2023-11-26T07:24:14.757 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:14.751+0000 7f824c71c000 1 --2- 172.21.15.143:0/3408546160 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x55f126bc72d0 0x55f126c27fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:24:14.757 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:14.751+0000 7f824c71c000 1 -- 172.21.15.143:0/3408546160 >> 172.21.15.143:0/3408546160 conn(0x55f126bf7aa0 msgr2=0x55f126bc9f50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:24:14.757 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:14.751+0000 7f824c71c000 1 -- 172.21.15.143:0/3408546160 shutdown_connections 2023-11-26T07:24:14.757 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:14.751+0000 7f824c71c000 1 -- 172.21.15.143:0/3408546160 wait complete. 2023-11-26T07:24:14.757 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:14.751+0000 7f824c71c000 20 client.8088 trim_cache size 0 max 16384 2023-11-26T07:24:14.757 INFO:teuthology.orchestra.run.smithi143.stdout:{ 2023-11-26T07:24:14.757 INFO:teuthology.orchestra.run.smithi143.stdout: "cluster_fsid": "f5033060-2469-4a07-95fe-2b76f5404bc1", 2023-11-26T07:24:14.757 INFO:teuthology.orchestra.run.smithi143.stdout: "whoami": 0, 2023-11-26T07:24:14.757 INFO:teuthology.orchestra.run.smithi143.stdout: "id": 7944, 2023-11-26T07:24:14.758 INFO:teuthology.orchestra.run.smithi143.stdout: "want_state": "up:active", 2023-11-26T07:24:14.758 INFO:teuthology.orchestra.run.smithi143.stdout: "state": "up:active", 2023-11-26T07:24:14.758 INFO:teuthology.orchestra.run.smithi143.stdout: "fs_name": "cephfs", 2023-11-26T07:24:14.758 INFO:teuthology.orchestra.run.smithi143.stdout: "rank_uptime": 5.8350278979999999, 2023-11-26T07:24:14.758 INFO:teuthology.orchestra.run.smithi143.stdout: "mdsmap_epoch": 65, 2023-11-26T07:24:14.758 INFO:teuthology.orchestra.run.smithi143.stdout: "osdmap_epoch": 103, 2023-11-26T07:24:14.758 INFO:teuthology.orchestra.run.smithi143.stdout: "osdmap_epoch_barrier": 103, 2023-11-26T07:24:14.758 INFO:teuthology.orchestra.run.smithi143.stdout: "uptime": 11.704572301000001 2023-11-26T07:24:14.758 INFO:teuthology.orchestra.run.smithi143.stdout:} 2023-11-26T07:24:14.759 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:14.751+0000 7f82493b8640 1 -- 172.21.15.143:0/3741046815 >> [v2:172.21.15.188:6836/3401128553,v1:172.21.15.188:6837/3401128553] conn(0x7f82240665c0 msgr2=0x7f8224068a80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:24:14.759 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:14.751+0000 7f82493b8640 1 --2- 172.21.15.143:0/3741046815 >> [v2:172.21.15.188:6836/3401128553,v1:172.21.15.188:6837/3401128553] conn(0x7f82240665c0 0x7f8224068a80 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7f824415a370 tx=0x7f8234007770 comp rx=0 tx=0).stop 2023-11-26T07:24:14.759 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:14.751+0000 7f82493b8640 1 -- 172.21.15.143:0/3741046815 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f8244143780 msgr2=0x7f8244132780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:24:14.759 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:14.751+0000 7f82493b8640 1 --2- 172.21.15.143:0/3741046815 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f8244143780 0x7f8244132780 secure :-1 s=READY pgs=1318 cs=0 l=1 rev1=1 crypto rx=0x7f823c0ac000 tx=0x7f823c019260 comp rx=0 tx=0).stop 2023-11-26T07:24:14.759 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:14.751+0000 7f82493b8640 1 -- 172.21.15.143:0/3741046815 shutdown_connections 2023-11-26T07:24:14.760 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:14.751+0000 7f82493b8640 1 --2- 172.21.15.143:0/3741046815 >> [v2:172.21.15.188:6836/3401128553,v1:172.21.15.188:6837/3401128553] conn(0x7f82240665c0 0x7f8224068a80 unknown :-1 s=CLOSED pgs=133 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:24:14.760 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:14.751+0000 7f82493b8640 1 --2- 172.21.15.143:0/3741046815 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f8244143780 0x7f8244132780 unknown :-1 s=CLOSED pgs=1318 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:24:14.760 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:14.751+0000 7f82493b8640 1 --2- 172.21.15.143:0/3741046815 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f82440ce520 0x7f8244139420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:24:14.760 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:14.751+0000 7f82493b8640 1 --2- 172.21.15.143:0/3741046815 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f82440722c0 0x7f82441592c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:24:14.760 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:14.751+0000 7f82493b8640 1 -- 172.21.15.143:0/3741046815 >> 172.21.15.143:0/3741046815 conn(0x7f824406c3f0 msgr2=0x7f824406dcc0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:24:14.760 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:14.751+0000 7f82493b8640 1 -- 172.21.15.143:0/3741046815 shutdown_connections 2023-11-26T07:24:14.760 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:14.751+0000 7f82493b8640 1 -- 172.21.15.143:0/3741046815 wait complete. 2023-11-26T07:24:14.771 DEBUG:teuthology.orchestra.run.smithi143:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2023-11-26T07:24:14.908 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:14.907+0000 7efebf2c9640 1 Processor -- start 2023-11-26T07:24:14.908 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:14.907+0000 7efebf2c9640 1 -- start start 2023-11-26T07:24:14.910 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:14.911+0000 7efebf2c9640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7efeb814b1e0 0x7efeb80cf280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:24:14.910 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:14.911+0000 7efebf2c9640 1 -- --> v1:172.21.15.143:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7efeb80cfc20 con 0x7efeb80cf7c0 2023-11-26T07:24:14.910 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:14.911+0000 7efebf2c9640 1 -- --> v1:172.21.15.188:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7efeb8162080 con 0x7efeb8070fd0 2023-11-26T07:24:14.910 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:14.911+0000 7efebf2c9640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7efeb8161d70 con 0x7efeb814b1e0 2023-11-26T07:24:14.910 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:14.911+0000 7efebc864640 1 --1- >> v1:172.21.15.143:6789/0 conn(0x7efeb80cf7c0 0x7efeb80708c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.143:6789/0 says I am v1:172.21.15.143:40192/0 (socket says 172.21.15.143:40192) 2023-11-26T07:24:14.910 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:14.911+0000 7efebc864640 1 -- 172.21.15.143:0/2386640370 learned_addr learned my addr 172.21.15.143:0/2386640370 (peer_addr_for_me v1:172.21.15.143:0/0) 2023-11-26T07:24:14.911 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:14.911+0000 7efebd065640 1 --2- 172.21.15.143:0/2386640370 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7efeb814b1e0 0x7efeb80cf280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:24:14.911 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:14.911+0000 7efeaffff640 1 -- 172.21.15.143:0/2386640370 <== mon.0 v1:172.21.15.143:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2876010167 0 0) 0x7efeb80cfc20 con 0x7efeb80cf7c0 2023-11-26T07:24:14.911 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:14.911+0000 7efeaffff640 1 -- 172.21.15.143:0/2386640370 --> v1:172.21.15.143:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7efea4003530 con 0x7efeb80cf7c0 2023-11-26T07:24:14.911 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:14.911+0000 7efebd065640 1 -- 172.21.15.143:0/2386640370 >> v1:172.21.15.188:6789/0 conn(0x7efeb8070fd0 legacy=0x7efeb8162510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:24:14.911 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:14.911+0000 7efebd065640 1 -- 172.21.15.143:0/2386640370 >> v1:172.21.15.143:6789/0 conn(0x7efeb80cf7c0 legacy=0x7efeb80708c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:24:14.912 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:14.911+0000 7efebd065640 1 -- 172.21.15.143:0/2386640370 --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efeb814b5c0 con 0x7efeb814b1e0 2023-11-26T07:24:14.914 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:14.911+0000 7efebd065640 1 --2- 172.21.15.143:0/2386640370 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7efeb814b1e0 0x7efeb80cf280 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7efeb40097f0 tx=0x7efeb401c490 comp rx=0 tx=0).ready entity=mon.2 client_cookie=817d77e422a0a446 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:24:14.914 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:14.915+0000 7efeaffff640 1 -- 172.21.15.143:0/2386640370 <== mon.2 v2:172.21.15.188:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efeb40aa5f0 con 0x7efeb814b1e0 2023-11-26T07:24:14.914 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:14.915+0000 7efeaffff640 1 -- 172.21.15.143:0/2386640370 <== mon.2 v2:172.21.15.188:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7efeb40aa750 con 0x7efeb814b1e0 2023-11-26T07:24:14.914 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:14.915+0000 7efeaffff640 1 -- 172.21.15.143:0/2386640370 <== mon.2 v2:172.21.15.188:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efeb40aaae0 con 0x7efeb814b1e0 2023-11-26T07:24:14.915 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:14.915+0000 7efebf2c9640 1 -- 172.21.15.143:0/2386640370 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7efeb814b1e0 msgr2=0x7efeb80cf280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:24:14.915 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:14.915+0000 7efebf2c9640 1 --2- 172.21.15.143:0/2386640370 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7efeb814b1e0 0x7efeb80cf280 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7efeb40097f0 tx=0x7efeb401c490 comp rx=0 tx=0).stop 2023-11-26T07:24:14.915 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:14.915+0000 7efebf2c9640 1 -- 172.21.15.143:0/2386640370 shutdown_connections 2023-11-26T07:24:14.915 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:14.915+0000 7efebf2c9640 1 --2- 172.21.15.143:0/2386640370 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7efeb814b1e0 0x7efeb80cf280 unknown :-1 s=CLOSED pgs=157 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:24:14.915 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:14.915+0000 7efebf2c9640 1 -- 172.21.15.143:0/2386640370 >> 172.21.15.143:0/2386640370 conn(0x7efeb806a890 msgr2=0x7efeb806aca0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:24:14.915 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:14.915+0000 7efebf2c9640 1 -- 172.21.15.143:0/2386640370 shutdown_connections 2023-11-26T07:24:14.915 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:14.915+0000 7efebf2c9640 1 -- 172.21.15.143:0/2386640370 wait complete. 2023-11-26T07:24:14.916 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:14.915+0000 7efebf2c9640 1 Processor -- start 2023-11-26T07:24:14.916 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:14.915+0000 7efebf2c9640 1 -- start start 2023-11-26T07:24:14.919 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:14.915+0000 7efebf2c9640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7efeb8070fd0 0x7efeb80b0e80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:24:14.919 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:14.915+0000 7efebf2c9640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7efeb80cf7c0 0x7efeb8160770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:24:14.919 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:14.915+0000 7efebf2c9640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7efeb814b1e0 0x7efeb8160cb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:24:14.919 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:14.915+0000 7efebf2c9640 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efeb80ad370 con 0x7efeb814b1e0 2023-11-26T07:24:14.919 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:14.915+0000 7efebf2c9640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efeb8161610 con 0x7efeb80cf7c0 2023-11-26T07:24:14.919 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:14.915+0000 7efebf2c9640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7efeb80ac6f0 con 0x7efeb8070fd0 2023-11-26T07:24:14.919 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:14.919+0000 7efebd866640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7efeb814b1e0 0x7efeb8160cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:24:14.919 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:14.919+0000 7efebd866640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7efeb814b1e0 0x7efeb8160cb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:54542/0 (socket says 172.21.15.143:54542) 2023-11-26T07:24:14.919 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:14.919+0000 7efebd866640 1 -- 172.21.15.143:0/389704060 learned_addr learned my addr 172.21.15.143:0/389704060 (peer_addr_for_me v2:172.21.15.143:0/0) 2023-11-26T07:24:14.919 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:14.919+0000 7efebd065640 1 --2- 172.21.15.143:0/389704060 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7efeb8070fd0 0x7efeb80b0e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:24:14.920 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:14.919+0000 7efebc864640 1 --2- 172.21.15.143:0/389704060 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7efeb80cf7c0 0x7efeb8160770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:24:14.920 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:14.919+0000 7efebd866640 1 -- 172.21.15.143:0/389704060 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7efeb8070fd0 msgr2=0x7efeb80b0e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:24:14.920 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:14.919+0000 7efebd866640 1 --2- 172.21.15.143:0/389704060 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7efeb8070fd0 0x7efeb80b0e80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:24:14.920 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:14.919+0000 7efebd866640 1 -- 172.21.15.143:0/389704060 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7efeb80cf7c0 msgr2=0x7efeb8160770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:24:14.920 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:14.919+0000 7efebd866640 1 --2- 172.21.15.143:0/389704060 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7efeb80cf7c0 0x7efeb8160770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:24:14.920 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:14.919+0000 7efebd866640 1 -- 172.21.15.143:0/389704060 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efeb4009420 con 0x7efeb814b1e0 2023-11-26T07:24:14.920 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:14.919+0000 7efebd065640 1 --2- 172.21.15.143:0/389704060 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7efeb8070fd0 0x7efeb80b0e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-26T07:24:14.920 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:14.919+0000 7efebd866640 1 --2- 172.21.15.143:0/389704060 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7efeb814b1e0 0x7efeb8160cb0 secure :-1 s=READY pgs=1324 cs=0 l=1 rev1=1 crypto rx=0x7efeb000c770 tx=0x7efeb000cc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:24:14.920 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:14.919+0000 7efeadffb640 1 -- 172.21.15.143:0/389704060 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efeb0003b80 con 0x7efeb814b1e0 2023-11-26T07:24:14.920 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:14.919+0000 7efeadffb640 1 -- 172.21.15.143:0/389704060 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7efeb0003ce0 con 0x7efeb814b1e0 2023-11-26T07:24:14.921 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:14.919+0000 7efebf2c9640 1 -- 172.21.15.143:0/389704060 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7efeb80ac9d0 con 0x7efeb814b1e0 2023-11-26T07:24:14.921 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:14.919+0000 7efeadffb640 1 -- 172.21.15.143:0/389704060 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efeb000aa90 con 0x7efeb814b1e0 2023-11-26T07:24:14.921 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:14.919+0000 7efebf2c9640 1 -- 172.21.15.143:0/389704060 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7efeb80acf90 con 0x7efeb814b1e0 2023-11-26T07:24:14.921 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:14.919+0000 7efebf2c9640 1 -- 172.21.15.143:0/389704060 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7efeb80ad4b0 con 0x7efeb814b1e0 2023-11-26T07:24:14.921 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:14.919+0000 7efeadffb640 1 -- 172.21.15.143:0/389704060 <== mon.0 v2:172.21.15.143:3300/0 4 ==== mgrmap(e 16) v1 ==== 85954+0+0 (secure 0 0 0) 0x7efeb000abf0 con 0x7efeb814b1e0 2023-11-26T07:24:14.922 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:14.923+0000 7efeadffb640 1 --2- 172.21.15.143:0/389704060 >> [v2:172.21.15.188:6836/3401128553,v1:172.21.15.188:6837/3401128553] conn(0x7efe9c066590 0x7efe9c068a50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:24:14.922 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:14.923+0000 7efeadffb640 1 -- 172.21.15.143:0/389704060 <== mon.0 v2:172.21.15.143:3300/0 5 ==== osd_map(103..103 src has 1..103) v4 ==== 5709+0+0 (secure 0 0 0) 0x7efeb0083e50 con 0x7efeb814b1e0 2023-11-26T07:24:14.922 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:14.923+0000 7efebd065640 1 --2- 172.21.15.143:0/389704060 >> [v2:172.21.15.188:6836/3401128553,v1:172.21.15.188:6837/3401128553] conn(0x7efe9c066590 0x7efe9c068a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:24:14.922 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:14.923+0000 7efebd065640 1 --2- 172.21.15.143:0/389704060 >> [v2:172.21.15.188:6836/3401128553,v1:172.21.15.188:6837/3401128553] conn(0x7efe9c066590 0x7efe9c068a50 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7efeb40ae000 tx=0x7efeb40a7000 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:24:14.925 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:14.923+0000 7efeadffb640 1 -- 172.21.15.143:0/389704060 <== mon.0 v2:172.21.15.143:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172649 (secure 0 0 0) 0x7efeb80ad4b0 con 0x7efeb814b1e0 2023-11-26T07:24:15.107 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:15.107+0000 7efebf2c9640 1 -- 172.21.15.143:0/389704060 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "fs ls"} v 0) v1 -- 0x7efeb80ad4b0 con 0x7efeb814b1e0 2023-11-26T07:24:15.107 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:15.107+0000 7efeadffb640 1 -- 172.21.15.143:0/389704060 <== mon.0 v2:172.21.15.143:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v65) v1 ==== 53+0+73 (secure 0 0 0) 0x7efeb80ad4b0 con 0x7efeb814b1e0 2023-11-26T07:24:15.108 INFO:teuthology.orchestra.run.smithi143.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2023-11-26T07:24:15.110 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:15.107+0000 7efebf2c9640 1 -- 172.21.15.143:0/389704060 >> [v2:172.21.15.188:6836/3401128553,v1:172.21.15.188:6837/3401128553] conn(0x7efe9c066590 msgr2=0x7efe9c068a50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:24:15.110 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:15.111+0000 7efebf2c9640 1 --2- 172.21.15.143:0/389704060 >> [v2:172.21.15.188:6836/3401128553,v1:172.21.15.188:6837/3401128553] conn(0x7efe9c066590 0x7efe9c068a50 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7efeb40ae000 tx=0x7efeb40a7000 comp rx=0 tx=0).stop 2023-11-26T07:24:15.110 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:15.111+0000 7efebf2c9640 1 -- 172.21.15.143:0/389704060 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7efeb814b1e0 msgr2=0x7efeb8160cb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:24:15.110 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:15.111+0000 7efebf2c9640 1 --2- 172.21.15.143:0/389704060 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7efeb814b1e0 0x7efeb8160cb0 secure :-1 s=READY pgs=1324 cs=0 l=1 rev1=1 crypto rx=0x7efeb000c770 tx=0x7efeb000cc30 comp rx=0 tx=0).stop 2023-11-26T07:24:15.110 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:15.111+0000 7efebf2c9640 1 -- 172.21.15.143:0/389704060 shutdown_connections 2023-11-26T07:24:15.110 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:15.111+0000 7efebf2c9640 1 --2- 172.21.15.143:0/389704060 >> [v2:172.21.15.188:6836/3401128553,v1:172.21.15.188:6837/3401128553] conn(0x7efe9c066590 0x7efe9c068a50 unknown :-1 s=CLOSED pgs=134 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:24:15.110 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:15.111+0000 7efebf2c9640 1 --2- 172.21.15.143:0/389704060 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7efeb814b1e0 0x7efeb8160cb0 unknown :-1 s=CLOSED pgs=1324 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:24:15.110 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:15.111+0000 7efebf2c9640 1 --2- 172.21.15.143:0/389704060 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7efeb80cf7c0 0x7efeb8160770 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:24:15.111 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:15.111+0000 7efebf2c9640 1 --2- 172.21.15.143:0/389704060 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7efeb8070fd0 0x7efeb80b0e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:24:15.111 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:15.111+0000 7efebf2c9640 1 -- 172.21.15.143:0/389704060 >> 172.21.15.143:0/389704060 conn(0x7efeb806a890 msgr2=0x7efeb8146480 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:24:15.111 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:15.111+0000 7efebf2c9640 1 -- 172.21.15.143:0/389704060 shutdown_connections 2023-11-26T07:24:15.111 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:15.111+0000 7efebf2c9640 1 -- 172.21.15.143:0/389704060 wait complete. 2023-11-26T07:24:15.129 INFO:tasks.cephfs.mount:Mounting default Ceph FS; just confirmed its presence on cluster 2023-11-26T07:24:15.129 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2023-11-26T07:24:15.129 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi143.front.sepia.ceph.com 2023-11-26T07:24:15.130 INFO:tasks.cephfs.mount:self.client.name = client.0 2023-11-26T07:24:15.130 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2023-11-26T07:24:15.130 INFO:tasks.cephfs.mount:self.cephfs_name = None 2023-11-26T07:24:15.130 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2023-11-26T07:24:15.130 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2023-11-26T07:24:15.130 INFO:tasks.cephfs.mount:Setting the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2023-11-26T07:24:15.130 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-26T07:24:15.130 DEBUG:teuthology.orchestra.run.smithi143:> ip addr 2023-11-26T07:24:15.140 INFO:teuthology.orchestra.run.smithi143.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2023-11-26T07:24:15.140 INFO:teuthology.orchestra.run.smithi143.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2023-11-26T07:24:15.140 INFO:teuthology.orchestra.run.smithi143.stdout: inet 127.0.0.1/8 scope host lo 2023-11-26T07:24:15.140 INFO:teuthology.orchestra.run.smithi143.stdout: valid_lft forever preferred_lft forever 2023-11-26T07:24:15.140 INFO:teuthology.orchestra.run.smithi143.stdout: inet6 ::1/128 scope host 2023-11-26T07:24:15.140 INFO:teuthology.orchestra.run.smithi143.stdout: valid_lft forever preferred_lft forever 2023-11-26T07:24:15.140 INFO:teuthology.orchestra.run.smithi143.stdout:2: eno1: mtu 1500 qdisc noop state DOWN group default qlen 1000 2023-11-26T07:24:15.141 INFO:teuthology.orchestra.run.smithi143.stdout: link/ether 0c:c4:7a:d9:3b:94 brd ff:ff:ff:ff:ff:ff 2023-11-26T07:24:15.141 INFO:teuthology.orchestra.run.smithi143.stdout: altname enp7s0f0 2023-11-26T07:24:15.141 INFO:teuthology.orchestra.run.smithi143.stdout:3: eno2: mtu 1500 qdisc noop state DOWN group default qlen 1000 2023-11-26T07:24:15.141 INFO:teuthology.orchestra.run.smithi143.stdout: link/ether 0c:c4:7a:d9:3b:95 brd ff:ff:ff:ff:ff:ff 2023-11-26T07:24:15.141 INFO:teuthology.orchestra.run.smithi143.stdout: altname enp7s0f1 2023-11-26T07:24:15.141 INFO:teuthology.orchestra.run.smithi143.stdout:4: enp3s0f0: mtu 1500 qdisc noop state DOWN group default qlen 1000 2023-11-26T07:24:15.141 INFO:teuthology.orchestra.run.smithi143.stdout: link/ether 0c:c4:7a:88:77:82 brd ff:ff:ff:ff:ff:ff 2023-11-26T07:24:15.141 INFO:teuthology.orchestra.run.smithi143.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2023-11-26T07:24:15.141 INFO:teuthology.orchestra.run.smithi143.stdout: link/ether 0c:c4:7a:88:77:83 brd ff:ff:ff:ff:ff:ff 2023-11-26T07:24:15.141 INFO:teuthology.orchestra.run.smithi143.stdout: inet 172.21.15.143/20 metric 100 brd 172.21.15.255 scope global dynamic enp3s0f1 2023-11-26T07:24:15.141 INFO:teuthology.orchestra.run.smithi143.stdout: valid_lft 42019sec preferred_lft 42019sec 2023-11-26T07:24:15.141 INFO:teuthology.orchestra.run.smithi143.stdout: inet6 fe80::ec4:7aff:fe88:7783/64 scope link 2023-11-26T07:24:15.141 INFO:teuthology.orchestra.run.smithi143.stdout: valid_lft forever preferred_lft forever 2023-11-26T07:24:15.142 INFO:teuthology.orchestra.run.smithi143.stdout:6: docker0: mtu 1500 qdisc noqueue state DOWN group default 2023-11-26T07:24:15.142 INFO:teuthology.orchestra.run.smithi143.stdout: link/ether 02:42:c4:b4:2f:eb brd ff:ff:ff:ff:ff:ff 2023-11-26T07:24:15.142 INFO:teuthology.orchestra.run.smithi143.stdout: inet 172.17.0.1/16 brd 172.17.255.255 scope global docker0 2023-11-26T07:24:15.142 INFO:teuthology.orchestra.run.smithi143.stdout: valid_lft forever preferred_lft forever 2023-11-26T07:24:15.142 INFO:teuthology.orchestra.run.smithi143.stdout:7: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2023-11-26T07:24:15.142 INFO:teuthology.orchestra.run.smithi143.stdout: link/ether 5e:1f:84:6d:79:ae brd ff:ff:ff:ff:ff:ff 2023-11-26T07:24:15.142 INFO:teuthology.orchestra.run.smithi143.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2023-11-26T07:24:15.142 INFO:teuthology.orchestra.run.smithi143.stdout: valid_lft forever preferred_lft forever 2023-11-26T07:24:15.142 INFO:teuthology.orchestra.run.smithi143.stdout: inet6 fe80::5c1f:84ff:fe6d:79ae/64 scope link 2023-11-26T07:24:15.142 INFO:teuthology.orchestra.run.smithi143.stdout: valid_lft forever preferred_lft forever 2023-11-26T07:24:15.142 INFO:teuthology.orchestra.run.smithi143.stdout:8: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2023-11-26T07:24:15.142 INFO:teuthology.orchestra.run.smithi143.stdout: link/ether 0e:5c:f6:1f:82:2f brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.0 2023-11-26T07:24:15.143 INFO:teuthology.orchestra.run.smithi143.stdout: inet6 fe80::c5c:f6ff:fe1f:822f/64 scope link 2023-11-26T07:24:15.143 INFO:teuthology.orchestra.run.smithi143.stdout: valid_lft forever preferred_lft forever 2023-11-26T07:24:15.143 INFO:teuthology.orchestra.run.smithi143.stdout:9: brx.1@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2023-11-26T07:24:15.143 INFO:teuthology.orchestra.run.smithi143.stdout: link/ether c6:a6:3d:69:d0:db brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.2 2023-11-26T07:24:15.143 INFO:teuthology.orchestra.run.smithi143.stdout: inet6 fe80::c4a6:3dff:fe69:d0db/64 scope link 2023-11-26T07:24:15.143 INFO:teuthology.orchestra.run.smithi143.stdout: valid_lft forever preferred_lft forever 2023-11-26T07:24:15.143 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-26T07:24:15.143 DEBUG:teuthology.orchestra.run.smithi143:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2023-11-26T07:24:15.199 INFO:teuthology.orchestra.run.smithi143.stdout:1 2023-11-26T07:24:15.202 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-26T07:24:15.202 DEBUG:teuthology.orchestra.run.smithi143:> route 2023-11-26T07:24:15.260 INFO:teuthology.orchestra.run.smithi143.stdout:Kernel IP routing table 2023-11-26T07:24:15.260 INFO:teuthology.orchestra.run.smithi143.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2023-11-26T07:24:15.260 INFO:teuthology.orchestra.run.smithi143.stdout:default _gateway 0.0.0.0 UG 0 0 0 enp3s0f1 2023-11-26T07:24:15.260 INFO:teuthology.orchestra.run.smithi143.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2023-11-26T07:24:15.260 INFO:teuthology.orchestra.run.smithi143.stdout:172.17.0.0 0.0.0.0 255.255.0.0 U 0 0 0 docker0 2023-11-26T07:24:15.260 INFO:teuthology.orchestra.run.smithi143.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 enp3s0f1 2023-11-26T07:24:15.260 INFO:teuthology.orchestra.run.smithi143.stdout:ns1.front.sepia 0.0.0.0 255.255.255.255 UH 100 0 0 enp3s0f1 2023-11-26T07:24:15.260 INFO:teuthology.orchestra.run.smithi143.stdout:172.21.0.2 0.0.0.0 255.255.255.255 UH 100 0 0 enp3s0f1 2023-11-26T07:24:15.260 INFO:teuthology.orchestra.run.smithi143.stdout:_gateway 0.0.0.0 255.255.255.255 UH 100 0 0 enp3s0f1 2023-11-26T07:24:15.260 INFO:teuthology.orchestra.run.smithi143.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2023-11-26T07:24:15.261 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-26T07:24:15.261 DEBUG:teuthology.orchestra.run.smithi143:> (cd / && exec bash -c ' 2023-11-26T07:24:15.261 DEBUG:teuthology.orchestra.run.smithi143:> set -e 2023-11-26T07:24:15.261 DEBUG:teuthology.orchestra.run.smithi143:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2023-11-26T07:24:15.261 DEBUG:teuthology.orchestra.run.smithi143:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2023-11-26T07:24:15.261 DEBUG:teuthology.orchestra.run.smithi143:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2023-11-26T07:24:15.261 DEBUG:teuthology.orchestra.run.smithi143:> ') 2023-11-26T07:24:15.356 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-26T07:24:15.356 DEBUG:teuthology.orchestra.run.smithi143:> ip netns list 2023-11-26T07:24:15.365 INFO:teuthology.orchestra.run.smithi143.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2023-11-26T07:24:15.365 INFO:teuthology.orchestra.run.smithi143.stdout:ceph-ns--home-ubuntu-cephtest-mnt.2 (id: 1) 2023-11-26T07:24:15.365 INFO:tasks.cephfs.mount:Resuming the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2023-11-26T07:24:15.366 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-26T07:24:15.366 DEBUG:teuthology.orchestra.run.smithi143:> sudo ip link set brx.0 up 2023-11-26T07:24:15.433 INFO:tasks.cephfs.fuse_mount:Client client.0 config is {} 2023-11-26T07:24:15.433 INFO:teuthology.orchestra.run:Running command with timeout 60 2023-11-26T07:24:15.433 DEBUG:teuthology.orchestra.run.smithi143:> mkdir -p -v /home/ubuntu/cephtest/mnt.0 2023-11-26T07:24:15.489 INFO:teuthology.orchestra.run.smithi143.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.0' 2023-11-26T07:24:15.490 INFO:teuthology.orchestra.run:Running command with timeout 60 2023-11-26T07:24:15.490 DEBUG:teuthology.orchestra.run.smithi143:> chmod 0000 /home/ubuntu/cephtest/mnt.0 2023-11-26T07:24:15.545 DEBUG:teuthology.orchestra.run.smithi143:> sudo modprobe fuse 2023-11-26T07:24:15.610 DEBUG:teuthology.orchestra.run.smithi143:> cat /proc/self/mounts | awk '{print $2}' 2023-11-26T07:24:15.667 INFO:teuthology.orchestra.run.smithi143.stdout:/sys 2023-11-26T07:24:15.667 INFO:teuthology.orchestra.run.smithi143.stdout:/proc 2023-11-26T07:24:15.667 INFO:teuthology.orchestra.run.smithi143.stdout:/dev 2023-11-26T07:24:15.667 INFO:teuthology.orchestra.run.smithi143.stdout:/dev/pts 2023-11-26T07:24:15.667 INFO:teuthology.orchestra.run.smithi143.stdout:/run 2023-11-26T07:24:15.667 INFO:teuthology.orchestra.run.smithi143.stdout:/ 2023-11-26T07:24:15.667 INFO:teuthology.orchestra.run.smithi143.stdout:/sys/kernel/security 2023-11-26T07:24:15.667 INFO:teuthology.orchestra.run.smithi143.stdout:/dev/shm 2023-11-26T07:24:15.667 INFO:teuthology.orchestra.run.smithi143.stdout:/run/lock 2023-11-26T07:24:15.667 INFO:teuthology.orchestra.run.smithi143.stdout:/sys/fs/cgroup 2023-11-26T07:24:15.668 INFO:teuthology.orchestra.run.smithi143.stdout:/sys/fs/pstore 2023-11-26T07:24:15.668 INFO:teuthology.orchestra.run.smithi143.stdout:/sys/fs/bpf 2023-11-26T07:24:15.668 INFO:teuthology.orchestra.run.smithi143.stdout:/proc/sys/fs/binfmt_misc 2023-11-26T07:24:15.668 INFO:teuthology.orchestra.run.smithi143.stdout:/dev/hugepages 2023-11-26T07:24:15.668 INFO:teuthology.orchestra.run.smithi143.stdout:/dev/mqueue 2023-11-26T07:24:15.668 INFO:teuthology.orchestra.run.smithi143.stdout:/sys/kernel/debug 2023-11-26T07:24:15.668 INFO:teuthology.orchestra.run.smithi143.stdout:/sys/kernel/tracing 2023-11-26T07:24:15.668 INFO:teuthology.orchestra.run.smithi143.stdout:/proc/fs/nfsd 2023-11-26T07:24:15.668 INFO:teuthology.orchestra.run.smithi143.stdout:/sys/fs/fuse/connections 2023-11-26T07:24:15.668 INFO:teuthology.orchestra.run.smithi143.stdout:/sys/kernel/config 2023-11-26T07:24:15.668 INFO:teuthology.orchestra.run.smithi143.stdout:/run/credentials/systemd-sysusers.service 2023-11-26T07:24:15.668 INFO:teuthology.orchestra.run.smithi143.stdout:/run/rpc_pipefs 2023-11-26T07:24:15.668 INFO:teuthology.orchestra.run.smithi143.stdout:/run/user/1000 2023-11-26T07:24:15.669 INFO:teuthology.orchestra.run.smithi143.stdout:/var/lib/ceph 2023-11-26T07:24:15.669 INFO:teuthology.orchestra.run.smithi143.stdout:/var/lib/ceph/osd/ceph-0 2023-11-26T07:24:15.669 INFO:teuthology.orchestra.run.smithi143.stdout:/var/lib/ceph/osd/ceph-1 2023-11-26T07:24:15.669 INFO:teuthology.orchestra.run.smithi143.stdout:/var/lib/ceph/osd/ceph-2 2023-11-26T07:24:15.669 INFO:teuthology.orchestra.run.smithi143.stdout:/var/lib/ceph/osd/ceph-3 2023-11-26T07:24:15.669 INFO:teuthology.orchestra.run.smithi143.stdout:/run/netns 2023-11-26T07:24:15.669 INFO:teuthology.orchestra.run.smithi143.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2023-11-26T07:24:15.669 INFO:teuthology.orchestra.run.smithi143.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2023-11-26T07:24:15.669 INFO:teuthology.orchestra.run.smithi143.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 2023-11-26T07:24:15.669 INFO:teuthology.orchestra.run.smithi143.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 2023-11-26T07:24:15.670 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-26T07:24:15.670 DEBUG:teuthology.orchestra.run.smithi143:> ls /sys/fs/fuse/connections 2023-11-26T07:24:15.725 INFO:tasks.cephfs.fuse_mount:Pre-mount connections: [] 2023-11-26T07:24:15.725 DEBUG:teuthology.orchestra.run.smithi143:> (cd /home/ubuntu/cephtest && exec sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-fuse -f --admin-socket '/var/run/ceph/$cluster-$name.$pid.asok' /home/ubuntu/cephtest/mnt.0 --id 0) 2023-11-26T07:24:15.776 DEBUG:teuthology.orchestra.run.smithi143:> sudo modprobe fuse 2023-11-26T07:24:15.797 DEBUG:teuthology.orchestra.run.smithi143:> cat /proc/self/mounts | awk '{print $2}' 2023-11-26T07:24:15.847 INFO:teuthology.orchestra.run.smithi143.stdout:/sys 2023-11-26T07:24:15.847 INFO:teuthology.orchestra.run.smithi143.stdout:/proc 2023-11-26T07:24:15.847 INFO:teuthology.orchestra.run.smithi143.stdout:/dev 2023-11-26T07:24:15.847 INFO:teuthology.orchestra.run.smithi143.stdout:/dev/pts 2023-11-26T07:24:15.848 INFO:teuthology.orchestra.run.smithi143.stdout:/run 2023-11-26T07:24:15.848 INFO:teuthology.orchestra.run.smithi143.stdout:/ 2023-11-26T07:24:15.848 INFO:teuthology.orchestra.run.smithi143.stdout:/sys/kernel/security 2023-11-26T07:24:15.848 INFO:teuthology.orchestra.run.smithi143.stdout:/dev/shm 2023-11-26T07:24:15.848 INFO:teuthology.orchestra.run.smithi143.stdout:/run/lock 2023-11-26T07:24:15.848 INFO:teuthology.orchestra.run.smithi143.stdout:/sys/fs/cgroup 2023-11-26T07:24:15.848 INFO:teuthology.orchestra.run.smithi143.stdout:/sys/fs/pstore 2023-11-26T07:24:15.848 INFO:teuthology.orchestra.run.smithi143.stdout:/sys/fs/bpf 2023-11-26T07:24:15.848 INFO:teuthology.orchestra.run.smithi143.stdout:/proc/sys/fs/binfmt_misc 2023-11-26T07:24:15.848 INFO:teuthology.orchestra.run.smithi143.stdout:/dev/hugepages 2023-11-26T07:24:15.848 INFO:teuthology.orchestra.run.smithi143.stdout:/dev/mqueue 2023-11-26T07:24:15.848 INFO:teuthology.orchestra.run.smithi143.stdout:/sys/kernel/debug 2023-11-26T07:24:15.848 INFO:teuthology.orchestra.run.smithi143.stdout:/sys/kernel/tracing 2023-11-26T07:24:15.849 INFO:teuthology.orchestra.run.smithi143.stdout:/proc/fs/nfsd 2023-11-26T07:24:15.849 INFO:teuthology.orchestra.run.smithi143.stdout:/sys/fs/fuse/connections 2023-11-26T07:24:15.849 INFO:teuthology.orchestra.run.smithi143.stdout:/sys/kernel/config 2023-11-26T07:24:15.849 INFO:teuthology.orchestra.run.smithi143.stdout:/run/credentials/systemd-sysusers.service 2023-11-26T07:24:15.849 INFO:teuthology.orchestra.run.smithi143.stdout:/run/rpc_pipefs 2023-11-26T07:24:15.849 INFO:teuthology.orchestra.run.smithi143.stdout:/run/user/1000 2023-11-26T07:24:15.849 INFO:teuthology.orchestra.run.smithi143.stdout:/var/lib/ceph 2023-11-26T07:24:15.849 INFO:teuthology.orchestra.run.smithi143.stdout:/var/lib/ceph/osd/ceph-0 2023-11-26T07:24:15.849 INFO:teuthology.orchestra.run.smithi143.stdout:/var/lib/ceph/osd/ceph-1 2023-11-26T07:24:15.849 INFO:teuthology.orchestra.run.smithi143.stdout:/var/lib/ceph/osd/ceph-2 2023-11-26T07:24:15.849 INFO:teuthology.orchestra.run.smithi143.stdout:/var/lib/ceph/osd/ceph-3 2023-11-26T07:24:15.849 INFO:teuthology.orchestra.run.smithi143.stdout:/run/netns 2023-11-26T07:24:15.849 INFO:teuthology.orchestra.run.smithi143.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2023-11-26T07:24:15.850 INFO:teuthology.orchestra.run.smithi143.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2023-11-26T07:24:15.850 INFO:teuthology.orchestra.run.smithi143.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 2023-11-26T07:24:15.850 INFO:teuthology.orchestra.run.smithi143.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 2023-11-26T07:24:15.851 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-26T07:24:15.851 DEBUG:teuthology.orchestra.run.smithi143:> ls /sys/fs/fuse/connections 2023-11-26T07:24:15.872 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi143.stderr:2023-11-26T07:24:15.871+0000 7f0380c33580 -1 init, newargv = 0x55ff5efbc1e0 newargc=15 2023-11-26T07:24:15.872 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi143.stderr:ceph-fuse[59135]: starting ceph client 2023-11-26T07:24:15.878 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi143.stderr:ceph-fuse[59135]: starting fuse 2023-11-26T07:24:15.881 INFO:teuthology.orchestra.run.smithi143.stdout:43 2023-11-26T07:24:15.882 INFO:tasks.cephfs.fuse_mount:Post-mount connections: [43] 2023-11-26T07:24:15.882 DEBUG:teuthology.orchestra.run.smithi143:> sudo stdin-killer -- python3 -c ' 2023-11-26T07:24:15.882 DEBUG:teuthology.orchestra.run.smithi143:> import glob 2023-11-26T07:24:15.882 DEBUG:teuthology.orchestra.run.smithi143:> import re 2023-11-26T07:24:15.882 DEBUG:teuthology.orchestra.run.smithi143:> import os 2023-11-26T07:24:15.882 DEBUG:teuthology.orchestra.run.smithi143:> import subprocess 2023-11-26T07:24:15.882 DEBUG:teuthology.orchestra.run.smithi143:> 2023-11-26T07:24:15.883 DEBUG:teuthology.orchestra.run.smithi143:> def _find_admin_socket(client_name): 2023-11-26T07:24:15.883 DEBUG:teuthology.orchestra.run.smithi143:> asok_path = "/var/run/ceph/ceph-client.0.*.asok" 2023-11-26T07:24:15.883 DEBUG:teuthology.orchestra.run.smithi143:> files = glob.glob(asok_path) 2023-11-26T07:24:15.883 DEBUG:teuthology.orchestra.run.smithi143:> mountpoint = "/home/ubuntu/cephtest/mnt.0" 2023-11-26T07:24:15.883 DEBUG:teuthology.orchestra.run.smithi143:> 2023-11-26T07:24:15.883 DEBUG:teuthology.orchestra.run.smithi143:> # Given a non-glob path, it better be there 2023-11-26T07:24:15.883 DEBUG:teuthology.orchestra.run.smithi143:> if "*" not in asok_path: 2023-11-26T07:24:15.883 DEBUG:teuthology.orchestra.run.smithi143:> assert(len(files) == 1) 2023-11-26T07:24:15.883 DEBUG:teuthology.orchestra.run.smithi143:> return files[0] 2023-11-26T07:24:15.883 DEBUG:teuthology.orchestra.run.smithi143:> 2023-11-26T07:24:15.883 DEBUG:teuthology.orchestra.run.smithi143:> for f in files: 2023-11-26T07:24:15.883 DEBUG:teuthology.orchestra.run.smithi143:> pid = re.match(".*\.(\d+)\.asok$", f).group(1) 2023-11-26T07:24:15.883 DEBUG:teuthology.orchestra.run.smithi143:> if os.path.exists("/proc/{0}".format(pid)): 2023-11-26T07:24:15.883 DEBUG:teuthology.orchestra.run.smithi143:> with open("/proc/{0}/cmdline".format(pid), '"'"'r'"'"') as proc_f: 2023-11-26T07:24:15.883 DEBUG:teuthology.orchestra.run.smithi143:> contents = proc_f.read() 2023-11-26T07:24:15.884 DEBUG:teuthology.orchestra.run.smithi143:> if mountpoint in contents: 2023-11-26T07:24:15.884 DEBUG:teuthology.orchestra.run.smithi143:> return f 2023-11-26T07:24:15.884 DEBUG:teuthology.orchestra.run.smithi143:> raise RuntimeError("Client socket {0} not found".format(client_name)) 2023-11-26T07:24:15.884 DEBUG:teuthology.orchestra.run.smithi143:> 2023-11-26T07:24:15.884 DEBUG:teuthology.orchestra.run.smithi143:> print(_find_admin_socket("client.0")) 2023-11-26T07:24:15.884 DEBUG:teuthology.orchestra.run.smithi143:> ' 2023-11-26T07:24:16.012 INFO:teuthology.orchestra.run.smithi143.stdout:/var/run/ceph/ceph-client.0.59135.asok 2023-11-26T07:24:16.015 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:16 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2023-11-26T07:24:16.023 INFO:tasks.cephfs.fuse_mount:Found client admin socket at /var/run/ceph/ceph-client.0.59135.asok 2023-11-26T07:24:16.023 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-26T07:24:16.023 DEBUG:teuthology.orchestra.run.smithi143:> sudo ceph --admin-daemon /var/run/ceph/ceph-client.0.59135.asok status 2023-11-26T07:24:16.193 INFO:teuthology.orchestra.run.smithi143.stdout:{ 2023-11-26T07:24:16.193 INFO:teuthology.orchestra.run.smithi143.stdout: "metadata": { 2023-11-26T07:24:16.193 INFO:teuthology.orchestra.run.smithi143.stdout: "ceph_sha1": "44c2237fe77713214148c1fd622c872f346f1b99", 2023-11-26T07:24:16.193 INFO:teuthology.orchestra.run.smithi143.stdout: "ceph_version": "ceph version 18.0.0-7484-g44c2237f (44c2237fe77713214148c1fd622c872f346f1b99) reef (dev)", 2023-11-26T07:24:16.193 INFO:teuthology.orchestra.run.smithi143.stdout: "entity_id": "0", 2023-11-26T07:24:16.193 INFO:teuthology.orchestra.run.smithi143.stdout: "hostname": "smithi143", 2023-11-26T07:24:16.193 INFO:teuthology.orchestra.run.smithi143.stdout: "mount_point": "/home/ubuntu/cephtest/mnt.0", 2023-11-26T07:24:16.194 INFO:teuthology.orchestra.run.smithi143.stdout: "pid": "59135", 2023-11-26T07:24:16.194 INFO:teuthology.orchestra.run.smithi143.stdout: "root": "/" 2023-11-26T07:24:16.194 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2023-11-26T07:24:16.194 INFO:teuthology.orchestra.run.smithi143.stdout: "dentry_count": 0, 2023-11-26T07:24:16.194 INFO:teuthology.orchestra.run.smithi143.stdout: "dentry_pinned_count": 0, 2023-11-26T07:24:16.194 INFO:teuthology.orchestra.run.smithi143.stdout: "id": 8103, 2023-11-26T07:24:16.194 INFO:teuthology.orchestra.run.smithi143.stdout: "inst": { 2023-11-26T07:24:16.194 INFO:teuthology.orchestra.run.smithi143.stdout: "name": { 2023-11-26T07:24:16.194 INFO:teuthology.orchestra.run.smithi143.stdout: "type": "client", 2023-11-26T07:24:16.194 INFO:teuthology.orchestra.run.smithi143.stdout: "num": 8103 2023-11-26T07:24:16.194 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2023-11-26T07:24:16.194 INFO:teuthology.orchestra.run.smithi143.stdout: "addr": { 2023-11-26T07:24:16.194 INFO:teuthology.orchestra.run.smithi143.stdout: "type": "v1", 2023-11-26T07:24:16.195 INFO:teuthology.orchestra.run.smithi143.stdout: "addr": "192.168.0.1:0", 2023-11-26T07:24:16.195 INFO:teuthology.orchestra.run.smithi143.stdout: "nonce": 4214332927 2023-11-26T07:24:16.195 INFO:teuthology.orchestra.run.smithi143.stdout: } 2023-11-26T07:24:16.195 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2023-11-26T07:24:16.195 INFO:teuthology.orchestra.run.smithi143.stdout: "addr": { 2023-11-26T07:24:16.195 INFO:teuthology.orchestra.run.smithi143.stdout: "type": "v1", 2023-11-26T07:24:16.195 INFO:teuthology.orchestra.run.smithi143.stdout: "addr": "192.168.0.1:0", 2023-11-26T07:24:16.195 INFO:teuthology.orchestra.run.smithi143.stdout: "nonce": 4214332927 2023-11-26T07:24:16.195 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2023-11-26T07:24:16.195 INFO:teuthology.orchestra.run.smithi143.stdout: "inst_str": "client.8103 192.168.0.1:0/4214332927", 2023-11-26T07:24:16.195 INFO:teuthology.orchestra.run.smithi143.stdout: "addr_str": "192.168.0.1:0/4214332927", 2023-11-26T07:24:16.195 INFO:teuthology.orchestra.run.smithi143.stdout: "inode_count": 1, 2023-11-26T07:24:16.195 INFO:teuthology.orchestra.run.smithi143.stdout: "mds_epoch": 65, 2023-11-26T07:24:16.195 INFO:teuthology.orchestra.run.smithi143.stdout: "osd_epoch": 103, 2023-11-26T07:24:16.196 INFO:teuthology.orchestra.run.smithi143.stdout: "osd_epoch_barrier": 0, 2023-11-26T07:24:16.196 INFO:teuthology.orchestra.run.smithi143.stdout: "blocklisted": false, 2023-11-26T07:24:16.196 INFO:teuthology.orchestra.run.smithi143.stdout: "fs_name": "cephfs" 2023-11-26T07:24:16.196 INFO:teuthology.orchestra.run.smithi143.stdout:} 2023-11-26T07:24:16.209 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-26T07:24:16.209 DEBUG:teuthology.orchestra.run.smithi143:> stat --file-system '--printf=%T 2023-11-26T07:24:16.209 DEBUG:teuthology.orchestra.run.smithi143:> ' -- /home/ubuntu/cephtest/mnt.0 2023-11-26T07:24:16.218 INFO:teuthology.orchestra.run.smithi143.stdout:fuseblk 2023-11-26T07:24:16.218 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.0 2023-11-26T07:24:16.218 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-26T07:24:16.218 DEBUG:teuthology.orchestra.run.smithi143:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.0 2023-11-26T07:24:16.282 DEBUG:teuthology.orchestra.run.smithi143:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2023-11-26T07:24:16.466 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:16.467+0000 7f6b2532b640 1 Processor -- start 2023-11-26T07:24:16.466 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:16.467+0000 7f6b2532b640 1 -- start start 2023-11-26T07:24:16.467 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:16.467+0000 7f6b2532b640 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7f6b200d01b0 0x7f6b200d3be0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:24:16.467 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:16.467+0000 7f6b2532b640 1 -- --> v1:172.21.15.143:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6b200ce3b0 con 0x7f6b2015d610 2023-11-26T07:24:16.467 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:16.467+0000 7f6b2532b640 1 -- --> v1:172.21.15.188:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6b200ce690 con 0x7f6b2015d230 2023-11-26T07:24:16.468 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:16.467+0000 7f6b2532b640 1 -- --> v2:172.21.15.143:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6b200ce970 con 0x7f6b200d01b0 2023-11-26T07:24:16.468 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:16.467+0000 7f6b1f7fe640 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7f6b200d01b0 0x7f6b200d3be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:24:16.468 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:16.467+0000 7f6b24b2a640 1 --1- >> v1:172.21.15.143:6789/0 conn(0x7f6b2015d610 0x7f6b200b3cc0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.143:6789/0 says I am v1:172.21.15.143:40206/0 (socket says 172.21.15.143:40206) 2023-11-26T07:24:16.468 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:16.467+0000 7f6b24b2a640 1 -- 172.21.15.143:0/413553274 learned_addr learned my addr 172.21.15.143:0/413553274 (peer_addr_for_me v1:172.21.15.143:0/0) 2023-11-26T07:24:16.468 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:16.467+0000 7f6b1f7fe640 1 -- 172.21.15.143:0/413553274 >> v1:172.21.15.188:6789/0 conn(0x7f6b2015d230 legacy=0x7f6b200cfaa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-26T07:24:16.468 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:16.467+0000 7f6b1f7fe640 1 -- 172.21.15.143:0/413553274 >> v1:172.21.15.143:6789/0 conn(0x7f6b2015d610 legacy=0x7f6b200b3cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:24:16.469 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:16.467+0000 7f6b1f7fe640 1 -- 172.21.15.143:0/413553274 --> v2:172.21.15.143:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6b200cec50 con 0x7f6b200d01b0 2023-11-26T07:24:16.469 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:16.467+0000 7f6b1effd640 1 -- 172.21.15.143:0/413553274 <== mon.0 v1:172.21.15.143:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3122499948 0 0) 0x7f6b200ce3b0 con 0x7f6b2015d610 2023-11-26T07:24:16.471 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:16.471+0000 7f6b1f7fe640 1 --2- 172.21.15.143:0/413553274 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f6b200d01b0 0x7f6b200d3be0 secure :-1 s=READY pgs=1329 cs=0 l=1 rev1=1 crypto rx=0x7f6b140097c0 tx=0x7f6b1401c3b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=363bfe90609f97b server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:24:16.471 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:16.471+0000 7f6b1effd640 1 -- 172.21.15.143:0/413553274 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6b140ac630 con 0x7f6b200d01b0 2023-11-26T07:24:16.471 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:16.471+0000 7f6b1effd640 1 -- 172.21.15.143:0/413553274 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6b140ac790 con 0x7f6b200d01b0 2023-11-26T07:24:16.471 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:16.471+0000 7f6b1effd640 1 -- 172.21.15.143:0/413553274 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6b140acb20 con 0x7f6b200d01b0 2023-11-26T07:24:16.471 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:16.471+0000 7f6b2532b640 1 -- 172.21.15.143:0/413553274 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f6b200d01b0 msgr2=0x7f6b200d3be0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:24:16.472 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:16.471+0000 7f6b2532b640 1 --2- 172.21.15.143:0/413553274 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f6b200d01b0 0x7f6b200d3be0 secure :-1 s=READY pgs=1329 cs=0 l=1 rev1=1 crypto rx=0x7f6b140097c0 tx=0x7f6b1401c3b0 comp rx=0 tx=0).stop 2023-11-26T07:24:16.472 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:16.471+0000 7f6b2532b640 1 -- 172.21.15.143:0/413553274 shutdown_connections 2023-11-26T07:24:16.472 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:16.471+0000 7f6b2532b640 1 --2- 172.21.15.143:0/413553274 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f6b200d01b0 0x7f6b200d3be0 unknown :-1 s=CLOSED pgs=1329 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:24:16.472 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:16.471+0000 7f6b2532b640 1 -- 172.21.15.143:0/413553274 >> 172.21.15.143:0/413553274 conn(0x7f6b200d30c0 msgr2=0x7f6b200d34d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:24:16.472 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:16.471+0000 7f6b2532b640 1 -- 172.21.15.143:0/413553274 shutdown_connections 2023-11-26T07:24:16.472 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:16.471+0000 7f6b2532b640 1 -- 172.21.15.143:0/413553274 wait complete. 2023-11-26T07:24:16.473 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:16.471+0000 7f6b2532b640 1 Processor -- start 2023-11-26T07:24:16.473 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:16.475+0000 7f6b2532b640 1 -- start start 2023-11-26T07:24:16.475 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:16.475+0000 7f6b2532b640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f6b200d01b0 0x7f6b200c8d30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:24:16.475 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:16.475+0000 7f6b2532b640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f6b2015d230 0x7f6b2015e2d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:24:16.475 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:16.475+0000 7f6b2532b640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f6b2015d610 0x7f6b200ca190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:24:16.475 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:16.475+0000 7f6b2532b640 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6b200c9270 con 0x7f6b2015d230 2023-11-26T07:24:16.475 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:16.475+0000 7f6b2532b640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6b2015f710 con 0x7f6b2015d610 2023-11-26T07:24:16.475 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:16.475+0000 7f6b2532b640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6b2015eb00 con 0x7f6b200d01b0 2023-11-26T07:24:16.476 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:16.475+0000 7f6b1f7fe640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f6b2015d230 0x7f6b2015e2d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:24:16.476 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:16.475+0000 7f6b1f7fe640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f6b2015d230 0x7f6b2015e2d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:54562/0 (socket says 172.21.15.143:54562) 2023-11-26T07:24:16.476 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:16.475+0000 7f6b1f7fe640 1 -- 172.21.15.143:0/3979279794 learned_addr learned my addr 172.21.15.143:0/3979279794 (peer_addr_for_me v2:172.21.15.143:0/0) 2023-11-26T07:24:16.476 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:16.475+0000 7f6b1ffff640 1 --2- 172.21.15.143:0/3979279794 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f6b200d01b0 0x7f6b200c8d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:24:16.476 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:16.475+0000 7f6b24b2a640 1 --2- 172.21.15.143:0/3979279794 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f6b2015d610 0x7f6b200ca190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:24:16.476 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:16.475+0000 7f6b1f7fe640 1 -- 172.21.15.143:0/3979279794 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f6b200d01b0 msgr2=0x7f6b200c8d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:24:16.476 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:16.475+0000 7f6b1f7fe640 1 --2- 172.21.15.143:0/3979279794 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f6b200d01b0 0x7f6b200c8d30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:24:16.476 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:16.475+0000 7f6b1f7fe640 1 -- 172.21.15.143:0/3979279794 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f6b2015d610 msgr2=0x7f6b200ca190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:24:16.476 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:16.475+0000 7f6b1f7fe640 1 --2- 172.21.15.143:0/3979279794 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f6b2015d610 0x7f6b200ca190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:24:16.476 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:16.475+0000 7f6b1f7fe640 1 -- 172.21.15.143:0/3979279794 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6b14009420 con 0x7f6b2015d230 2023-11-26T07:24:16.477 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:16.475+0000 7f6b1f7fe640 1 --2- 172.21.15.143:0/3979279794 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f6b2015d230 0x7f6b2015e2d0 secure :-1 s=READY pgs=1330 cs=0 l=1 rev1=1 crypto rx=0x7f6b14018fc0 tx=0x7f6b1401abf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:24:16.477 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:16.475+0000 7f6b1cff9640 1 -- 172.21.15.143:0/3979279794 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6b14004230 con 0x7f6b2015d230 2023-11-26T07:24:16.477 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:16.475+0000 7f6b1cff9640 1 -- 172.21.15.143:0/3979279794 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6b14004390 con 0x7f6b2015d230 2023-11-26T07:24:16.477 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:16.475+0000 7f6b1cff9640 1 -- 172.21.15.143:0/3979279794 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6b140b7830 con 0x7f6b2015d230 2023-11-26T07:24:16.477 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:16.475+0000 7f6b2532b640 1 -- 172.21.15.143:0/3979279794 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6b2015ede0 con 0x7f6b2015d230 2023-11-26T07:24:16.477 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:16.475+0000 7f6b2532b640 1 -- 172.21.15.143:0/3979279794 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6b2015f3a0 con 0x7f6b2015d230 2023-11-26T07:24:16.478 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:16.475+0000 7f6b2532b640 1 -- 172.21.15.143:0/3979279794 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6b20067c10 con 0x7f6b2015d230 2023-11-26T07:24:16.478 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:16.479+0000 7f6b1cff9640 1 -- 172.21.15.143:0/3979279794 <== mon.0 v2:172.21.15.143:3300/0 4 ==== mgrmap(e 16) v1 ==== 85954+0+0 (secure 0 0 0) 0x7f6b140b7990 con 0x7f6b2015d230 2023-11-26T07:24:16.479 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:16.479+0000 7f6b1cff9640 1 --2- 172.21.15.143:0/3979279794 >> [v2:172.21.15.188:6836/3401128553,v1:172.21.15.188:6837/3401128553] conn(0x7f6b0c066590 0x7f6b0c068a50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:24:16.479 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:16.479+0000 7f6b1cff9640 1 -- 172.21.15.143:0/3979279794 <== mon.0 v2:172.21.15.143:3300/0 5 ==== osd_map(103..103 src has 1..103) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f6b1411ed20 con 0x7f6b2015d230 2023-11-26T07:24:16.479 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:16.479+0000 7f6b1ffff640 1 --2- 172.21.15.143:0/3979279794 >> [v2:172.21.15.188:6836/3401128553,v1:172.21.15.188:6837/3401128553] conn(0x7f6b0c066590 0x7f6b0c068a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:24:16.480 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:16.479+0000 7f6b1ffff640 1 --2- 172.21.15.143:0/3979279794 >> [v2:172.21.15.188:6836/3401128553,v1:172.21.15.188:6837/3401128553] conn(0x7f6b0c066590 0x7f6b0c068a50 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7f6b20160060 tx=0x7f6b10007e40 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:24:16.483 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:16.483+0000 7f6b1cff9640 1 -- 172.21.15.143:0/3979279794 <== mon.0 v2:172.21.15.143:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172649 (secure 0 0 0) 0x7f6b140ebf90 con 0x7f6b2015d230 2023-11-26T07:24:16.669 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:16.667+0000 7f6b2532b640 1 -- 172.21.15.143:0/3979279794 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "fs ls"} v 0) v1 -- 0x7f6b200d1480 con 0x7f6b2015d230 2023-11-26T07:24:16.670 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:16.671+0000 7f6b1cff9640 1 -- 172.21.15.143:0/3979279794 <== mon.0 v2:172.21.15.143:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v65) v1 ==== 53+0+73 (secure 0 0 0) 0x7f6b140a8070 con 0x7f6b2015d230 2023-11-26T07:24:16.670 INFO:teuthology.orchestra.run.smithi143.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2023-11-26T07:24:16.672 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:16.671+0000 7f6b2532b640 1 -- 172.21.15.143:0/3979279794 >> [v2:172.21.15.188:6836/3401128553,v1:172.21.15.188:6837/3401128553] conn(0x7f6b0c066590 msgr2=0x7f6b0c068a50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:24:16.672 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:16.671+0000 7f6b2532b640 1 --2- 172.21.15.143:0/3979279794 >> [v2:172.21.15.188:6836/3401128553,v1:172.21.15.188:6837/3401128553] conn(0x7f6b0c066590 0x7f6b0c068a50 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7f6b20160060 tx=0x7f6b10007e40 comp rx=0 tx=0).stop 2023-11-26T07:24:16.672 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:16.671+0000 7f6b2532b640 1 -- 172.21.15.143:0/3979279794 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f6b2015d230 msgr2=0x7f6b2015e2d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:24:16.673 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:16.671+0000 7f6b2532b640 1 --2- 172.21.15.143:0/3979279794 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f6b2015d230 0x7f6b2015e2d0 secure :-1 s=READY pgs=1330 cs=0 l=1 rev1=1 crypto rx=0x7f6b14018fc0 tx=0x7f6b1401abf0 comp rx=0 tx=0).stop 2023-11-26T07:24:16.673 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:16.671+0000 7f6b2532b640 1 -- 172.21.15.143:0/3979279794 shutdown_connections 2023-11-26T07:24:16.673 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:16.671+0000 7f6b2532b640 1 --2- 172.21.15.143:0/3979279794 >> [v2:172.21.15.188:6836/3401128553,v1:172.21.15.188:6837/3401128553] conn(0x7f6b0c066590 0x7f6b0c068a50 unknown :-1 s=CLOSED pgs=135 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:24:16.673 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:16.671+0000 7f6b2532b640 1 --2- 172.21.15.143:0/3979279794 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f6b2015d610 0x7f6b200ca190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:24:16.673 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:16.671+0000 7f6b2532b640 1 --2- 172.21.15.143:0/3979279794 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f6b2015d230 0x7f6b2015e2d0 unknown :-1 s=CLOSED pgs=1330 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:24:16.673 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:16.671+0000 7f6b2532b640 1 --2- 172.21.15.143:0/3979279794 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f6b200d01b0 0x7f6b200c8d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:24:16.673 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:16.671+0000 7f6b2532b640 1 -- 172.21.15.143:0/3979279794 >> 172.21.15.143:0/3979279794 conn(0x7f6b200d30c0 msgr2=0x7f6b2006c7e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:24:16.673 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:16.671+0000 7f6b2532b640 1 -- 172.21.15.143:0/3979279794 shutdown_connections 2023-11-26T07:24:16.673 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:16.671+0000 7f6b2532b640 1 -- 172.21.15.143:0/3979279794 wait complete. 2023-11-26T07:24:16.691 INFO:tasks.cephfs.mount:Mounting default Ceph FS; just confirmed its presence on cluster 2023-11-26T07:24:16.691 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2023-11-26T07:24:16.691 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi188.front.sepia.ceph.com 2023-11-26T07:24:16.691 INFO:tasks.cephfs.mount:self.client.name = client.1 2023-11-26T07:24:16.691 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.1 2023-11-26T07:24:16.691 INFO:tasks.cephfs.mount:self.cephfs_name = None 2023-11-26T07:24:16.691 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2023-11-26T07:24:16.692 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2023-11-26T07:24:16.692 INFO:tasks.cephfs.mount:Setting the 'ceph-ns--home-ubuntu-cephtest-mnt.1' netns for '/home/ubuntu/cephtest/mnt.1' 2023-11-26T07:24:16.692 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-26T07:24:16.692 DEBUG:teuthology.orchestra.run.smithi188:> ip addr 2023-11-26T07:24:16.702 INFO:teuthology.orchestra.run.smithi188.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2023-11-26T07:24:16.703 INFO:teuthology.orchestra.run.smithi188.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2023-11-26T07:24:16.703 INFO:teuthology.orchestra.run.smithi188.stdout: inet 127.0.0.1/8 scope host lo 2023-11-26T07:24:16.703 INFO:teuthology.orchestra.run.smithi188.stdout: valid_lft forever preferred_lft forever 2023-11-26T07:24:16.703 INFO:teuthology.orchestra.run.smithi188.stdout: inet6 ::1/128 scope host 2023-11-26T07:24:16.703 INFO:teuthology.orchestra.run.smithi188.stdout: valid_lft forever preferred_lft forever 2023-11-26T07:24:16.703 INFO:teuthology.orchestra.run.smithi188.stdout:2: eno1: mtu 1500 qdisc noop state DOWN group default qlen 1000 2023-11-26T07:24:16.703 INFO:teuthology.orchestra.run.smithi188.stdout: link/ether 0c:c4:7a:d9:37:48 brd ff:ff:ff:ff:ff:ff 2023-11-26T07:24:16.703 INFO:teuthology.orchestra.run.smithi188.stdout: altname enp7s0f0 2023-11-26T07:24:16.703 INFO:teuthology.orchestra.run.smithi188.stdout:3: eno2: mtu 1500 qdisc noop state DOWN group default qlen 1000 2023-11-26T07:24:16.703 INFO:teuthology.orchestra.run.smithi188.stdout: link/ether 0c:c4:7a:d9:37:49 brd ff:ff:ff:ff:ff:ff 2023-11-26T07:24:16.703 INFO:teuthology.orchestra.run.smithi188.stdout: altname enp7s0f1 2023-11-26T07:24:16.704 INFO:teuthology.orchestra.run.smithi188.stdout:4: enp3s0f0: mtu 1500 qdisc noop state DOWN group default qlen 1000 2023-11-26T07:24:16.704 INFO:teuthology.orchestra.run.smithi188.stdout: link/ether 0c:c4:7a:88:80:38 brd ff:ff:ff:ff:ff:ff 2023-11-26T07:24:16.704 INFO:teuthology.orchestra.run.smithi188.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2023-11-26T07:24:16.704 INFO:teuthology.orchestra.run.smithi188.stdout: link/ether 0c:c4:7a:88:80:39 brd ff:ff:ff:ff:ff:ff 2023-11-26T07:24:16.704 INFO:teuthology.orchestra.run.smithi188.stdout: inet 172.21.15.188/20 metric 100 brd 172.21.15.255 scope global dynamic enp3s0f1 2023-11-26T07:24:16.704 INFO:teuthology.orchestra.run.smithi188.stdout: valid_lft 42011sec preferred_lft 42011sec 2023-11-26T07:24:16.704 INFO:teuthology.orchestra.run.smithi188.stdout: inet6 fe80::ec4:7aff:fe88:8039/64 scope link 2023-11-26T07:24:16.704 INFO:teuthology.orchestra.run.smithi188.stdout: valid_lft forever preferred_lft forever 2023-11-26T07:24:16.704 INFO:teuthology.orchestra.run.smithi188.stdout:6: docker0: mtu 1500 qdisc noqueue state DOWN group default 2023-11-26T07:24:16.704 INFO:teuthology.orchestra.run.smithi188.stdout: link/ether 02:42:d8:0d:a0:fa brd ff:ff:ff:ff:ff:ff 2023-11-26T07:24:16.704 INFO:teuthology.orchestra.run.smithi188.stdout: inet 172.17.0.1/16 brd 172.17.255.255 scope global docker0 2023-11-26T07:24:16.704 INFO:teuthology.orchestra.run.smithi188.stdout: valid_lft forever preferred_lft forever 2023-11-26T07:24:16.705 INFO:teuthology.orchestra.run.smithi188.stdout:7: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2023-11-26T07:24:16.705 INFO:teuthology.orchestra.run.smithi188.stdout: link/ether 5e:1f:84:6d:79:ae brd ff:ff:ff:ff:ff:ff 2023-11-26T07:24:16.705 INFO:teuthology.orchestra.run.smithi188.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2023-11-26T07:24:16.705 INFO:teuthology.orchestra.run.smithi188.stdout: valid_lft forever preferred_lft forever 2023-11-26T07:24:16.705 INFO:teuthology.orchestra.run.smithi188.stdout: inet6 fe80::5c1f:84ff:fe6d:79ae/64 scope link 2023-11-26T07:24:16.705 INFO:teuthology.orchestra.run.smithi188.stdout: valid_lft forever preferred_lft forever 2023-11-26T07:24:16.705 INFO:teuthology.orchestra.run.smithi188.stdout:8: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2023-11-26T07:24:16.705 INFO:teuthology.orchestra.run.smithi188.stdout: link/ether 0e:5c:f6:1f:82:2f brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.1 2023-11-26T07:24:16.705 INFO:teuthology.orchestra.run.smithi188.stdout: inet6 fe80::c5c:f6ff:fe1f:822f/64 scope link 2023-11-26T07:24:16.705 INFO:teuthology.orchestra.run.smithi188.stdout: valid_lft forever preferred_lft forever 2023-11-26T07:24:16.705 INFO:teuthology.orchestra.run.smithi188.stdout:9: brx.1@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2023-11-26T07:24:16.705 INFO:teuthology.orchestra.run.smithi188.stdout: link/ether c6:a6:3d:69:d0:db brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.3 2023-11-26T07:24:16.705 INFO:teuthology.orchestra.run.smithi188.stdout: inet6 fe80::c4a6:3dff:fe69:d0db/64 scope link 2023-11-26T07:24:16.706 INFO:teuthology.orchestra.run.smithi188.stdout: valid_lft forever preferred_lft forever 2023-11-26T07:24:16.706 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-26T07:24:16.706 DEBUG:teuthology.orchestra.run.smithi188:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2023-11-26T07:24:16.768 INFO:teuthology.orchestra.run.smithi188.stdout:1 2023-11-26T07:24:16.770 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-26T07:24:16.770 DEBUG:teuthology.orchestra.run.smithi188:> route 2023-11-26T07:24:16.822 INFO:teuthology.orchestra.run.smithi188.stdout:Kernel IP routing table 2023-11-26T07:24:16.822 INFO:teuthology.orchestra.run.smithi188.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2023-11-26T07:24:16.823 INFO:teuthology.orchestra.run.smithi188.stdout:default _gateway 0.0.0.0 UG 0 0 0 enp3s0f1 2023-11-26T07:24:16.823 INFO:teuthology.orchestra.run.smithi188.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2023-11-26T07:24:16.823 INFO:teuthology.orchestra.run.smithi188.stdout:172.17.0.0 0.0.0.0 255.255.0.0 U 0 0 0 docker0 2023-11-26T07:24:16.823 INFO:teuthology.orchestra.run.smithi188.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 enp3s0f1 2023-11-26T07:24:16.823 INFO:teuthology.orchestra.run.smithi188.stdout:ns1.front.sepia 0.0.0.0 255.255.255.255 UH 100 0 0 enp3s0f1 2023-11-26T07:24:16.823 INFO:teuthology.orchestra.run.smithi188.stdout:172.21.0.2 0.0.0.0 255.255.255.255 UH 100 0 0 enp3s0f1 2023-11-26T07:24:16.823 INFO:teuthology.orchestra.run.smithi188.stdout:_gateway 0.0.0.0 255.255.255.255 UH 100 0 0 enp3s0f1 2023-11-26T07:24:16.823 INFO:teuthology.orchestra.run.smithi188.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2023-11-26T07:24:16.824 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-26T07:24:16.824 DEBUG:teuthology.orchestra.run.smithi188:> (cd / && exec bash -c ' 2023-11-26T07:24:16.824 DEBUG:teuthology.orchestra.run.smithi188:> set -e 2023-11-26T07:24:16.824 DEBUG:teuthology.orchestra.run.smithi188:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2023-11-26T07:24:16.824 DEBUG:teuthology.orchestra.run.smithi188:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2023-11-26T07:24:16.825 DEBUG:teuthology.orchestra.run.smithi188:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2023-11-26T07:24:16.825 DEBUG:teuthology.orchestra.run.smithi188:> ') 2023-11-26T07:24:16.923 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-26T07:24:16.923 DEBUG:teuthology.orchestra.run.smithi188:> ip netns list 2023-11-26T07:24:16.933 INFO:teuthology.orchestra.run.smithi188.stdout:ceph-ns--home-ubuntu-cephtest-mnt.1 (id: 0) 2023-11-26T07:24:16.933 INFO:teuthology.orchestra.run.smithi188.stdout:ceph-ns--home-ubuntu-cephtest-mnt.3 (id: 1) 2023-11-26T07:24:16.934 INFO:tasks.cephfs.mount:Resuming the 'ceph-ns--home-ubuntu-cephtest-mnt.1' netns for '/home/ubuntu/cephtest/mnt.1' 2023-11-26T07:24:16.934 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-26T07:24:16.934 DEBUG:teuthology.orchestra.run.smithi188:> sudo ip link set brx.0 up 2023-11-26T07:24:17.002 INFO:tasks.cephfs.fuse_mount:Client client.1 config is {} 2023-11-26T07:24:17.002 INFO:teuthology.orchestra.run:Running command with timeout 60 2023-11-26T07:24:17.002 DEBUG:teuthology.orchestra.run.smithi188:> mkdir -p -v /home/ubuntu/cephtest/mnt.1 2023-11-26T07:24:17.051 INFO:teuthology.orchestra.run.smithi188.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.1' 2023-11-26T07:24:17.052 INFO:teuthology.orchestra.run:Running command with timeout 60 2023-11-26T07:24:17.052 DEBUG:teuthology.orchestra.run.smithi188:> chmod 0000 /home/ubuntu/cephtest/mnt.1 2023-11-26T07:24:17.103 DEBUG:teuthology.orchestra.run.smithi188:> sudo modprobe fuse 2023-11-26T07:24:17.169 DEBUG:teuthology.orchestra.run.smithi188:> cat /proc/self/mounts | awk '{print $2}' 2023-11-26T07:24:17.221 INFO:teuthology.orchestra.run.smithi188.stdout:/sys 2023-11-26T07:24:17.221 INFO:teuthology.orchestra.run.smithi188.stdout:/proc 2023-11-26T07:24:17.221 INFO:teuthology.orchestra.run.smithi188.stdout:/dev 2023-11-26T07:24:17.221 INFO:teuthology.orchestra.run.smithi188.stdout:/dev/pts 2023-11-26T07:24:17.221 INFO:teuthology.orchestra.run.smithi188.stdout:/run 2023-11-26T07:24:17.221 INFO:teuthology.orchestra.run.smithi188.stdout:/ 2023-11-26T07:24:17.221 INFO:teuthology.orchestra.run.smithi188.stdout:/sys/kernel/security 2023-11-26T07:24:17.221 INFO:teuthology.orchestra.run.smithi188.stdout:/dev/shm 2023-11-26T07:24:17.221 INFO:teuthology.orchestra.run.smithi188.stdout:/run/lock 2023-11-26T07:24:17.221 INFO:teuthology.orchestra.run.smithi188.stdout:/sys/fs/cgroup 2023-11-26T07:24:17.222 INFO:teuthology.orchestra.run.smithi188.stdout:/sys/fs/pstore 2023-11-26T07:24:17.222 INFO:teuthology.orchestra.run.smithi188.stdout:/sys/fs/bpf 2023-11-26T07:24:17.222 INFO:teuthology.orchestra.run.smithi188.stdout:/proc/sys/fs/binfmt_misc 2023-11-26T07:24:17.222 INFO:teuthology.orchestra.run.smithi188.stdout:/dev/hugepages 2023-11-26T07:24:17.222 INFO:teuthology.orchestra.run.smithi188.stdout:/dev/mqueue 2023-11-26T07:24:17.222 INFO:teuthology.orchestra.run.smithi188.stdout:/sys/kernel/debug 2023-11-26T07:24:17.222 INFO:teuthology.orchestra.run.smithi188.stdout:/sys/kernel/tracing 2023-11-26T07:24:17.222 INFO:teuthology.orchestra.run.smithi188.stdout:/proc/fs/nfsd 2023-11-26T07:24:17.222 INFO:teuthology.orchestra.run.smithi188.stdout:/sys/fs/fuse/connections 2023-11-26T07:24:17.222 INFO:teuthology.orchestra.run.smithi188.stdout:/sys/kernel/config 2023-11-26T07:24:17.222 INFO:teuthology.orchestra.run.smithi188.stdout:/run/credentials/systemd-sysusers.service 2023-11-26T07:24:17.222 INFO:teuthology.orchestra.run.smithi188.stdout:/run/rpc_pipefs 2023-11-26T07:24:17.223 INFO:teuthology.orchestra.run.smithi188.stdout:/run/user/1000 2023-11-26T07:24:17.223 INFO:teuthology.orchestra.run.smithi188.stdout:/var/lib/ceph 2023-11-26T07:24:17.223 INFO:teuthology.orchestra.run.smithi188.stdout:/var/lib/ceph/osd/ceph-4 2023-11-26T07:24:17.223 INFO:teuthology.orchestra.run.smithi188.stdout:/var/lib/ceph/osd/ceph-5 2023-11-26T07:24:17.223 INFO:teuthology.orchestra.run.smithi188.stdout:/var/lib/ceph/osd/ceph-6 2023-11-26T07:24:17.223 INFO:teuthology.orchestra.run.smithi188.stdout:/var/lib/ceph/osd/ceph-7 2023-11-26T07:24:17.223 INFO:teuthology.orchestra.run.smithi188.stdout:/run/netns 2023-11-26T07:24:17.223 INFO:teuthology.orchestra.run.smithi188.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 2023-11-26T07:24:17.223 INFO:teuthology.orchestra.run.smithi188.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 2023-11-26T07:24:17.223 INFO:teuthology.orchestra.run.smithi188.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 2023-11-26T07:24:17.223 INFO:teuthology.orchestra.run.smithi188.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 2023-11-26T07:24:17.224 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-26T07:24:17.224 DEBUG:teuthology.orchestra.run.smithi188:> ls /sys/fs/fuse/connections 2023-11-26T07:24:17.275 INFO:tasks.cephfs.fuse_mount:Pre-mount connections: [] 2023-11-26T07:24:17.275 DEBUG:teuthology.orchestra.run.smithi188:> (cd /home/ubuntu/cephtest && exec sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-fuse -f --admin-socket '/var/run/ceph/$cluster-$name.$pid.asok' /home/ubuntu/cephtest/mnt.1 --id 1) 2023-11-26T07:24:17.322 DEBUG:teuthology.orchestra.run.smithi188:> sudo modprobe fuse 2023-11-26T07:24:17.344 DEBUG:teuthology.orchestra.run.smithi188:> cat /proc/self/mounts | awk '{print $2}' 2023-11-26T07:24:17.397 INFO:teuthology.orchestra.run.smithi188.stdout:/sys 2023-11-26T07:24:17.397 INFO:teuthology.orchestra.run.smithi188.stdout:/proc 2023-11-26T07:24:17.397 INFO:teuthology.orchestra.run.smithi188.stdout:/dev 2023-11-26T07:24:17.397 INFO:teuthology.orchestra.run.smithi188.stdout:/dev/pts 2023-11-26T07:24:17.397 INFO:teuthology.orchestra.run.smithi188.stdout:/run 2023-11-26T07:24:17.397 INFO:teuthology.orchestra.run.smithi188.stdout:/ 2023-11-26T07:24:17.397 INFO:teuthology.orchestra.run.smithi188.stdout:/sys/kernel/security 2023-11-26T07:24:17.398 INFO:teuthology.orchestra.run.smithi188.stdout:/dev/shm 2023-11-26T07:24:17.398 INFO:teuthology.orchestra.run.smithi188.stdout:/run/lock 2023-11-26T07:24:17.398 INFO:teuthology.orchestra.run.smithi188.stdout:/sys/fs/cgroup 2023-11-26T07:24:17.398 INFO:teuthology.orchestra.run.smithi188.stdout:/sys/fs/pstore 2023-11-26T07:24:17.398 INFO:teuthology.orchestra.run.smithi188.stdout:/sys/fs/bpf 2023-11-26T07:24:17.398 INFO:teuthology.orchestra.run.smithi188.stdout:/proc/sys/fs/binfmt_misc 2023-11-26T07:24:17.398 INFO:teuthology.orchestra.run.smithi188.stdout:/dev/hugepages 2023-11-26T07:24:17.398 INFO:teuthology.orchestra.run.smithi188.stdout:/dev/mqueue 2023-11-26T07:24:17.398 INFO:teuthology.orchestra.run.smithi188.stdout:/sys/kernel/debug 2023-11-26T07:24:17.398 INFO:teuthology.orchestra.run.smithi188.stdout:/sys/kernel/tracing 2023-11-26T07:24:17.398 INFO:teuthology.orchestra.run.smithi188.stdout:/proc/fs/nfsd 2023-11-26T07:24:17.398 INFO:teuthology.orchestra.run.smithi188.stdout:/sys/fs/fuse/connections 2023-11-26T07:24:17.399 INFO:teuthology.orchestra.run.smithi188.stdout:/sys/kernel/config 2023-11-26T07:24:17.399 INFO:teuthology.orchestra.run.smithi188.stdout:/run/credentials/systemd-sysusers.service 2023-11-26T07:24:17.399 INFO:teuthology.orchestra.run.smithi188.stdout:/run/rpc_pipefs 2023-11-26T07:24:17.399 INFO:teuthology.orchestra.run.smithi188.stdout:/run/user/1000 2023-11-26T07:24:17.399 INFO:teuthology.orchestra.run.smithi188.stdout:/var/lib/ceph 2023-11-26T07:24:17.399 INFO:teuthology.orchestra.run.smithi188.stdout:/var/lib/ceph/osd/ceph-4 2023-11-26T07:24:17.399 INFO:teuthology.orchestra.run.smithi188.stdout:/var/lib/ceph/osd/ceph-5 2023-11-26T07:24:17.399 INFO:teuthology.orchestra.run.smithi188.stdout:/var/lib/ceph/osd/ceph-6 2023-11-26T07:24:17.399 INFO:teuthology.orchestra.run.smithi188.stdout:/var/lib/ceph/osd/ceph-7 2023-11-26T07:24:17.399 INFO:teuthology.orchestra.run.smithi188.stdout:/run/netns 2023-11-26T07:24:17.399 INFO:teuthology.orchestra.run.smithi188.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 2023-11-26T07:24:17.399 INFO:teuthology.orchestra.run.smithi188.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 2023-11-26T07:24:17.400 INFO:teuthology.orchestra.run.smithi188.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 2023-11-26T07:24:17.400 INFO:teuthology.orchestra.run.smithi188.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 2023-11-26T07:24:17.400 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-26T07:24:17.401 DEBUG:teuthology.orchestra.run.smithi188:> ls /sys/fs/fuse/connections 2023-11-26T07:24:17.419 INFO:tasks.cephfs.fuse_mount.ceph-fuse.1.smithi188.stderr:2023-11-26T07:24:17.413+0000 7f561b6dc580 -1 init, newargv = 0x56232a407f70 newargc=15 2023-11-26T07:24:17.419 INFO:tasks.cephfs.fuse_mount.ceph-fuse.1.smithi188.stderr:ceph-fuse[45326]: starting ceph client 2023-11-26T07:24:17.426 INFO:tasks.cephfs.fuse_mount.ceph-fuse.1.smithi188.stderr:ceph-fuse[45326]: starting fuse 2023-11-26T07:24:17.427 INFO:teuthology.orchestra.run.smithi188.stdout:43 2023-11-26T07:24:17.428 INFO:tasks.cephfs.fuse_mount:Post-mount connections: [43] 2023-11-26T07:24:17.428 DEBUG:teuthology.orchestra.run.smithi188:> sudo stdin-killer -- python3 -c ' 2023-11-26T07:24:17.428 DEBUG:teuthology.orchestra.run.smithi188:> import glob 2023-11-26T07:24:17.428 DEBUG:teuthology.orchestra.run.smithi188:> import re 2023-11-26T07:24:17.429 DEBUG:teuthology.orchestra.run.smithi188:> import os 2023-11-26T07:24:17.429 DEBUG:teuthology.orchestra.run.smithi188:> import subprocess 2023-11-26T07:24:17.429 DEBUG:teuthology.orchestra.run.smithi188:> 2023-11-26T07:24:17.429 DEBUG:teuthology.orchestra.run.smithi188:> def _find_admin_socket(client_name): 2023-11-26T07:24:17.429 DEBUG:teuthology.orchestra.run.smithi188:> asok_path = "/var/run/ceph/ceph-client.1.*.asok" 2023-11-26T07:24:17.429 DEBUG:teuthology.orchestra.run.smithi188:> files = glob.glob(asok_path) 2023-11-26T07:24:17.429 DEBUG:teuthology.orchestra.run.smithi188:> mountpoint = "/home/ubuntu/cephtest/mnt.1" 2023-11-26T07:24:17.429 DEBUG:teuthology.orchestra.run.smithi188:> 2023-11-26T07:24:17.429 DEBUG:teuthology.orchestra.run.smithi188:> # Given a non-glob path, it better be there 2023-11-26T07:24:17.429 DEBUG:teuthology.orchestra.run.smithi188:> if "*" not in asok_path: 2023-11-26T07:24:17.429 DEBUG:teuthology.orchestra.run.smithi188:> assert(len(files) == 1) 2023-11-26T07:24:17.429 DEBUG:teuthology.orchestra.run.smithi188:> return files[0] 2023-11-26T07:24:17.429 DEBUG:teuthology.orchestra.run.smithi188:> 2023-11-26T07:24:17.429 DEBUG:teuthology.orchestra.run.smithi188:> for f in files: 2023-11-26T07:24:17.430 DEBUG:teuthology.orchestra.run.smithi188:> pid = re.match(".*\.(\d+)\.asok$", f).group(1) 2023-11-26T07:24:17.430 DEBUG:teuthology.orchestra.run.smithi188:> if os.path.exists("/proc/{0}".format(pid)): 2023-11-26T07:24:17.430 DEBUG:teuthology.orchestra.run.smithi188:> with open("/proc/{0}/cmdline".format(pid), '"'"'r'"'"') as proc_f: 2023-11-26T07:24:17.430 DEBUG:teuthology.orchestra.run.smithi188:> contents = proc_f.read() 2023-11-26T07:24:17.430 DEBUG:teuthology.orchestra.run.smithi188:> if mountpoint in contents: 2023-11-26T07:24:17.430 DEBUG:teuthology.orchestra.run.smithi188:> return f 2023-11-26T07:24:17.430 DEBUG:teuthology.orchestra.run.smithi188:> raise RuntimeError("Client socket {0} not found".format(client_name)) 2023-11-26T07:24:17.430 DEBUG:teuthology.orchestra.run.smithi188:> 2023-11-26T07:24:17.430 DEBUG:teuthology.orchestra.run.smithi188:> print(_find_admin_socket("client.1")) 2023-11-26T07:24:17.430 DEBUG:teuthology.orchestra.run.smithi188:> ' 2023-11-26T07:24:17.554 INFO:teuthology.orchestra.run.smithi188.stdout:/var/run/ceph/ceph-client.1.45326.asok 2023-11-26T07:24:17.558 INFO:teuthology.orchestra.run.smithi188.stderr:2023-11-26T07:24:17 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2023-11-26T07:24:17.565 INFO:tasks.cephfs.fuse_mount:Found client admin socket at /var/run/ceph/ceph-client.1.45326.asok 2023-11-26T07:24:17.566 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-26T07:24:17.566 DEBUG:teuthology.orchestra.run.smithi188:> sudo ceph --admin-daemon /var/run/ceph/ceph-client.1.45326.asok status 2023-11-26T07:24:17.741 INFO:teuthology.orchestra.run.smithi188.stdout:{ 2023-11-26T07:24:17.741 INFO:teuthology.orchestra.run.smithi188.stdout: "metadata": { 2023-11-26T07:24:17.741 INFO:teuthology.orchestra.run.smithi188.stdout: "ceph_sha1": "44c2237fe77713214148c1fd622c872f346f1b99", 2023-11-26T07:24:17.741 INFO:teuthology.orchestra.run.smithi188.stdout: "ceph_version": "ceph version 18.0.0-7484-g44c2237f (44c2237fe77713214148c1fd622c872f346f1b99) reef (dev)", 2023-11-26T07:24:17.742 INFO:teuthology.orchestra.run.smithi188.stdout: "entity_id": "1", 2023-11-26T07:24:17.742 INFO:teuthology.orchestra.run.smithi188.stdout: "hostname": "smithi188", 2023-11-26T07:24:17.742 INFO:teuthology.orchestra.run.smithi188.stdout: "mount_point": "/home/ubuntu/cephtest/mnt.1", 2023-11-26T07:24:17.742 INFO:teuthology.orchestra.run.smithi188.stdout: "pid": "45326", 2023-11-26T07:24:17.742 INFO:teuthology.orchestra.run.smithi188.stdout: "root": "/" 2023-11-26T07:24:17.742 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2023-11-26T07:24:17.742 INFO:teuthology.orchestra.run.smithi188.stdout: "dentry_count": 0, 2023-11-26T07:24:17.742 INFO:teuthology.orchestra.run.smithi188.stdout: "dentry_pinned_count": 0, 2023-11-26T07:24:17.742 INFO:teuthology.orchestra.run.smithi188.stdout: "id": 7588, 2023-11-26T07:24:17.742 INFO:teuthology.orchestra.run.smithi188.stdout: "inst": { 2023-11-26T07:24:17.742 INFO:teuthology.orchestra.run.smithi188.stdout: "name": { 2023-11-26T07:24:17.742 INFO:teuthology.orchestra.run.smithi188.stdout: "type": "client", 2023-11-26T07:24:17.743 INFO:teuthology.orchestra.run.smithi188.stdout: "num": 7588 2023-11-26T07:24:17.743 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2023-11-26T07:24:17.743 INFO:teuthology.orchestra.run.smithi188.stdout: "addr": { 2023-11-26T07:24:17.743 INFO:teuthology.orchestra.run.smithi188.stdout: "type": "v1", 2023-11-26T07:24:17.743 INFO:teuthology.orchestra.run.smithi188.stdout: "addr": "192.168.0.1:0", 2023-11-26T07:24:17.743 INFO:teuthology.orchestra.run.smithi188.stdout: "nonce": 1232094689 2023-11-26T07:24:17.743 INFO:teuthology.orchestra.run.smithi188.stdout: } 2023-11-26T07:24:17.743 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2023-11-26T07:24:17.743 INFO:teuthology.orchestra.run.smithi188.stdout: "addr": { 2023-11-26T07:24:17.743 INFO:teuthology.orchestra.run.smithi188.stdout: "type": "v1", 2023-11-26T07:24:17.743 INFO:teuthology.orchestra.run.smithi188.stdout: "addr": "192.168.0.1:0", 2023-11-26T07:24:17.743 INFO:teuthology.orchestra.run.smithi188.stdout: "nonce": 1232094689 2023-11-26T07:24:17.743 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2023-11-26T07:24:17.744 INFO:teuthology.orchestra.run.smithi188.stdout: "inst_str": "client.7588 192.168.0.1:0/1232094689", 2023-11-26T07:24:17.744 INFO:teuthology.orchestra.run.smithi188.stdout: "addr_str": "192.168.0.1:0/1232094689", 2023-11-26T07:24:17.744 INFO:teuthology.orchestra.run.smithi188.stdout: "inode_count": 1, 2023-11-26T07:24:17.744 INFO:teuthology.orchestra.run.smithi188.stdout: "mds_epoch": 65, 2023-11-26T07:24:17.744 INFO:teuthology.orchestra.run.smithi188.stdout: "osd_epoch": 103, 2023-11-26T07:24:17.744 INFO:teuthology.orchestra.run.smithi188.stdout: "osd_epoch_barrier": 0, 2023-11-26T07:24:17.744 INFO:teuthology.orchestra.run.smithi188.stdout: "blocklisted": false, 2023-11-26T07:24:17.744 INFO:teuthology.orchestra.run.smithi188.stdout: "fs_name": "cephfs" 2023-11-26T07:24:17.744 INFO:teuthology.orchestra.run.smithi188.stdout:} 2023-11-26T07:24:17.756 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-26T07:24:17.756 DEBUG:teuthology.orchestra.run.smithi188:> stat --file-system '--printf=%T 2023-11-26T07:24:17.756 DEBUG:teuthology.orchestra.run.smithi188:> ' -- /home/ubuntu/cephtest/mnt.1 2023-11-26T07:24:17.765 INFO:teuthology.orchestra.run.smithi188.stdout:fuseblk 2023-11-26T07:24:17.766 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.1 2023-11-26T07:24:17.766 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-26T07:24:17.766 DEBUG:teuthology.orchestra.run.smithi188:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.1 2023-11-26T07:24:17.827 DEBUG:teuthology.orchestra.run.smithi143:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs flag set enable_multiple true --yes-i-really-mean-it 2023-11-26T07:24:17.979 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:17.979+0000 7f70aaddf640 1 Processor -- start 2023-11-26T07:24:17.979 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:17.979+0000 7f70aaddf640 1 -- start start 2023-11-26T07:24:17.981 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:17.979+0000 7f70aaddf640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f70a413e290 0x7f70a40cf280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:24:17.981 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:17.979+0000 7f70aaddf640 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7f70a40cf7c0 0x7f70a415efa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:24:17.981 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:17.979+0000 7f70aaddf640 1 -- --> v1:172.21.15.143:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f70a406c520 con 0x7f70a415f4e0 2023-11-26T07:24:17.981 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:17.979+0000 7f70aaddf640 1 -- --> v2:172.21.15.188:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f70a40cfc40 con 0x7f70a40cf7c0 2023-11-26T07:24:17.981 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:17.979+0000 7f70aaddf640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f70a406c800 con 0x7f70a413e290 2023-11-26T07:24:17.981 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:17.979+0000 7f70a937c640 1 --1- >> v1:172.21.15.143:6789/0 conn(0x7f70a415f4e0 0x7f70a41408e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.143:6789/0 says I am v1:172.21.15.143:40222/0 (socket says 172.21.15.143:40222) 2023-11-26T07:24:17.981 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:17.979+0000 7f70a937c640 1 -- 172.21.15.143:0/2157073809 learned_addr learned my addr 172.21.15.143:0/2157073809 (peer_addr_for_me v1:172.21.15.143:0/0) 2023-11-26T07:24:17.983 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:17.979+0000 7f70a8b7b640 1 --2- 172.21.15.143:0/2157073809 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f70a413e290 0x7f70a40cf280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:24:17.983 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:17.979+0000 7f709bfff640 1 --2- 172.21.15.143:0/2157073809 >> v2:172.21.15.188:3300/0 conn(0x7f70a40cf7c0 0x7f70a415efa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:24:17.983 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:17.979+0000 7f709b7fe640 1 -- 172.21.15.143:0/2157073809 <== mon.0 v1:172.21.15.143:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1714351086 0 0) 0x7f70a406c520 con 0x7f70a415f4e0 2023-11-26T07:24:17.984 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:17.979+0000 7f709b7fe640 1 -- 172.21.15.143:0/2157073809 --> v1:172.21.15.143:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f708c003530 con 0x7f70a415f4e0 2023-11-26T07:24:17.984 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:17.983+0000 7f709b7fe640 1 -- 172.21.15.143:0/2157073809 <== mon.0 v1:172.21.15.143:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3096620532 0 0) 0x7f708c003530 con 0x7f70a415f4e0 2023-11-26T07:24:17.984 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:17.983+0000 7f709b7fe640 1 -- 172.21.15.143:0/2157073809 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f70a413e290 msgr2=0x7f70a40cf280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-26T07:24:17.984 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:17.983+0000 7f709b7fe640 1 --2- 172.21.15.143:0/2157073809 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f70a413e290 0x7f70a40cf280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:24:17.984 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:17.983+0000 7f709b7fe640 1 -- 172.21.15.143:0/2157073809 >> v2:172.21.15.188:3300/0 conn(0x7f70a40cf7c0 msgr2=0x7f70a415efa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-26T07:24:17.984 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:17.983+0000 7f709b7fe640 1 --2- 172.21.15.143:0/2157073809 >> v2:172.21.15.188:3300/0 conn(0x7f70a40cf7c0 0x7f70a415efa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:24:17.984 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:17.983+0000 7f709b7fe640 1 -- 172.21.15.143:0/2157073809 --> v1:172.21.15.143:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f70a406cae0 con 0x7f70a415f4e0 2023-11-26T07:24:17.984 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:17.983+0000 7f709b7fe640 1 -- 172.21.15.143:0/2157073809 <== mon.0 v1:172.21.15.143:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2137882523 0 0) 0x7f709c001a00 con 0x7f70a415f4e0 2023-11-26T07:24:17.984 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:17.983+0000 7f709b7fe640 1 -- 172.21.15.143:0/2157073809 >> v1:172.21.15.143:6789/0 conn(0x7f70a415f4e0 legacy=0x7f70a41408e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:24:17.984 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:17.983+0000 7f709b7fe640 1 --2- 172.21.15.143:0/2157073809 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f708c003980 0x7f708c003da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:24:17.984 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:17.983+0000 7f709b7fe640 1 --2- 172.21.15.143:0/2157073809 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f708c0047d0 0x7f708c004c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:24:17.984 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:17.983+0000 7f709b7fe640 1 --2- 172.21.15.143:0/2157073809 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f708c005190 0x7f708c00b990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:24:17.985 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:17.983+0000 7f709b7fe640 1 -- 172.21.15.143:0/2157073809 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f70a40cfc40 con 0x7f708c005190 2023-11-26T07:24:17.985 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:17.983+0000 7f709b7fe640 1 -- 172.21.15.143:0/2157073809 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f70a406c800 con 0x7f708c003980 2023-11-26T07:24:17.985 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:17.983+0000 7f709b7fe640 1 -- 172.21.15.143:0/2157073809 --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f708c00bed0 con 0x7f708c0047d0 2023-11-26T07:24:17.985 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:17.983+0000 7f70a8b7b640 1 --2- 172.21.15.143:0/2157073809 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f70a413e290 0x7f70a40cf280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-26T07:24:17.985 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:17.983+0000 7f709bfff640 1 --2- 172.21.15.143:0/2157073809 >> v2:172.21.15.188:3300/0 conn(0x7f70a40cf7c0 0x7f70a415efa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-26T07:24:17.985 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:17.983+0000 7f70a937c640 1 --2- 172.21.15.143:0/2157073809 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f708c005190 0x7f708c00b990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:24:17.985 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:17.983+0000 7f70a8b7b640 1 --2- 172.21.15.143:0/2157073809 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f708c003980 0x7f708c003da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:24:17.985 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:17.983+0000 7f709bfff640 1 --2- 172.21.15.143:0/2157073809 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f708c0047d0 0x7f708c004c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:24:17.985 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:17.983+0000 7f70a8b7b640 1 -- 172.21.15.143:0/2157073809 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f708c0047d0 msgr2=0x7f708c004c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-26T07:24:17.985 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:17.983+0000 7f70a8b7b640 1 --2- 172.21.15.143:0/2157073809 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f708c0047d0 0x7f708c004c50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:24:17.985 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:17.983+0000 7f70a8b7b640 1 -- 172.21.15.143:0/2157073809 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f708c005190 msgr2=0x7f708c00b990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-26T07:24:17.985 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:17.983+0000 7f70a8b7b640 1 --2- 172.21.15.143:0/2157073809 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f708c005190 0x7f708c00b990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:24:17.985 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:17.983+0000 7f70a8b7b640 1 -- 172.21.15.143:0/2157073809 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f708c003530 con 0x7f708c003980 2023-11-26T07:24:17.986 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:17.983+0000 7f70a937c640 1 --2- 172.21.15.143:0/2157073809 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f708c005190 0x7f708c00b990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-26T07:24:17.986 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:17.983+0000 7f70a8b7b640 1 --2- 172.21.15.143:0/2157073809 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f708c003980 0x7f708c003da0 secure :-1 s=READY pgs=501 cs=0 l=1 rev1=1 crypto rx=0x7f70a0009b40 tx=0x7f70a001cab0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=e4a643aa2cb349d0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:24:17.987 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:17.987+0000 7f709b7fe640 1 -- 172.21.15.143:0/2157073809 <== mon.1 v2:172.21.15.188:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f70a0004030 con 0x7f708c003980 2023-11-26T07:24:17.987 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:17.987+0000 7f709b7fe640 1 -- 172.21.15.143:0/2157073809 <== mon.1 v2:172.21.15.188:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f70a0004190 con 0x7f708c003980 2023-11-26T07:24:17.987 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:17.987+0000 7f709b7fe640 1 -- 172.21.15.143:0/2157073809 <== mon.1 v2:172.21.15.188:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f70a0004500 con 0x7f708c003980 2023-11-26T07:24:17.987 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:17.987+0000 7f70aaddf640 1 -- 172.21.15.143:0/2157073809 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f708c003980 msgr2=0x7f708c003da0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:24:17.987 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:17.987+0000 7f70aaddf640 1 --2- 172.21.15.143:0/2157073809 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f708c003980 0x7f708c003da0 secure :-1 s=READY pgs=501 cs=0 l=1 rev1=1 crypto rx=0x7f70a0009b40 tx=0x7f70a001cab0 comp rx=0 tx=0).stop 2023-11-26T07:24:17.987 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:17.987+0000 7f70a937c640 1 -- 172.21.15.143:0/2157073809 reap_dead start 2023-11-26T07:24:17.987 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:17.987+0000 7f70aaddf640 1 -- 172.21.15.143:0/2157073809 shutdown_connections 2023-11-26T07:24:17.987 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:17.987+0000 7f70aaddf640 1 -- 172.21.15.143:0/2157073809 >> 172.21.15.143:0/2157073809 conn(0x7f70a406a890 msgr2=0x7f70a406aca0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:24:17.987 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:17.987+0000 7f70aaddf640 1 -- 172.21.15.143:0/2157073809 shutdown_connections 2023-11-26T07:24:17.987 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:17.987+0000 7f70aaddf640 1 -- 172.21.15.143:0/2157073809 wait complete. 2023-11-26T07:24:17.988 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:17.987+0000 7f70aaddf640 1 Processor -- start 2023-11-26T07:24:17.988 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:17.987+0000 7f70aaddf640 1 -- start start 2023-11-26T07:24:17.990 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:17.987+0000 7f70aaddf640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f708c003980 0x7f70a4154d20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:24:17.990 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:17.987+0000 7f70aaddf640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f70a4139450 0x7f70a41398d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:24:17.990 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:17.987+0000 7f70aaddf640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f70a40b2d00 0x7f70a40b3180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:24:17.990 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:17.987+0000 7f70aaddf640 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f70a4133280 con 0x7f708c003980 2023-11-26T07:24:17.990 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:17.987+0000 7f70aaddf640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f70a40b36c0 con 0x7f70a4139450 2023-11-26T07:24:17.990 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:17.987+0000 7f70aaddf640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f70a4155260 con 0x7f70a40b2d00 2023-11-26T07:24:17.990 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:17.991+0000 7f70a8b7b640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f708c003980 0x7f70a4154d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:24:17.990 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:17.991+0000 7f70a8b7b640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f708c003980 0x7f70a4154d20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:54572/0 (socket says 172.21.15.143:54572) 2023-11-26T07:24:17.990 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:17.991+0000 7f70a8b7b640 1 -- 172.21.15.143:0/1963402706 learned_addr learned my addr 172.21.15.143:0/1963402706 (peer_addr_for_me v2:172.21.15.143:0/0) 2023-11-26T07:24:17.991 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:17.991+0000 7f70a8b7b640 1 -- 172.21.15.143:0/1963402706 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f70a40b2d00 msgr2=0x7f70a40b3180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:24:17.991 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:17.991+0000 7f70a8b7b640 1 --2- 172.21.15.143:0/1963402706 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f70a40b2d00 0x7f70a40b3180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:24:17.991 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:17.991+0000 7f70a8b7b640 1 -- 172.21.15.143:0/1963402706 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f70a4139450 msgr2=0x7f70a41398d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:24:17.992 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:17.991+0000 7f709bfff640 1 --2- 172.21.15.143:0/1963402706 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f70a4139450 0x7f70a41398d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:24:17.992 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:17.991+0000 7f70a8b7b640 1 --2- 172.21.15.143:0/1963402706 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f70a4139450 0x7f70a41398d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:24:17.992 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:17.991+0000 7f70a8b7b640 1 -- 172.21.15.143:0/1963402706 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f70a4133490 con 0x7f708c003980 2023-11-26T07:24:17.992 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:17.991+0000 7f709bfff640 1 --2- 172.21.15.143:0/1963402706 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f70a4139450 0x7f70a41398d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-26T07:24:17.992 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:17.991+0000 7f70a8b7b640 1 --2- 172.21.15.143:0/1963402706 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f708c003980 0x7f70a4154d20 secure :-1 s=READY pgs=1332 cs=0 l=1 rev1=1 crypto rx=0x7f70a0009b40 tx=0x7f708c005800 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:24:17.992 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:17.991+0000 7f70997fa640 1 -- 172.21.15.143:0/1963402706 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f70a00a58a0 con 0x7f708c003980 2023-11-26T07:24:17.992 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:17.991+0000 7f70997fa640 1 -- 172.21.15.143:0/1963402706 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f70a00a5a00 con 0x7f708c003980 2023-11-26T07:24:17.992 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:17.991+0000 7f70997fa640 1 -- 172.21.15.143:0/1963402706 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f70a00a5d70 con 0x7f708c003980 2023-11-26T07:24:17.992 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:17.991+0000 7f70aaddf640 1 -- 172.21.15.143:0/1963402706 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f70a4133770 con 0x7f708c003980 2023-11-26T07:24:17.992 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:17.991+0000 7f70aaddf640 1 -- 172.21.15.143:0/1963402706 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f70a4133d30 con 0x7f708c003980 2023-11-26T07:24:17.993 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:17.991+0000 7f70aaddf640 1 -- 172.21.15.143:0/1963402706 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f70a4133ef0 con 0x7f708c003980 2023-11-26T07:24:17.993 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:17.991+0000 7f70997fa640 1 -- 172.21.15.143:0/1963402706 <== mon.0 v2:172.21.15.143:3300/0 4 ==== mgrmap(e 16) v1 ==== 85954+0+0 (secure 0 0 0) 0x7f70a0003620 con 0x7f708c003980 2023-11-26T07:24:17.994 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:17.995+0000 7f70997fa640 1 --2- 172.21.15.143:0/1963402706 >> [v2:172.21.15.188:6836/3401128553,v1:172.21.15.188:6837/3401128553] conn(0x7f7090066590 0x7f7090068a50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:24:17.994 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:17.995+0000 7f70997fa640 1 -- 172.21.15.143:0/1963402706 <== mon.0 v2:172.21.15.143:3300/0 5 ==== osd_map(103..103 src has 1..103) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f70a011f190 con 0x7f708c003980 2023-11-26T07:24:17.994 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:17.995+0000 7f709bfff640 1 --2- 172.21.15.143:0/1963402706 >> [v2:172.21.15.188:6836/3401128553,v1:172.21.15.188:6837/3401128553] conn(0x7f7090066590 0x7f7090068a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:24:17.995 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:17.995+0000 7f709bfff640 1 --2- 172.21.15.143:0/1963402706 >> [v2:172.21.15.188:6836/3401128553,v1:172.21.15.188:6837/3401128553] conn(0x7f7090066590 0x7f7090068a50 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7f70a40b3c70 tx=0x7f7094008040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:24:17.998 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:17.999+0000 7f70997fa640 1 -- 172.21.15.143:0/1963402706 <== mon.0 v2:172.21.15.143:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172649 (secure 0 0 0) 0x7f70a00ec590 con 0x7f708c003980 2023-11-26T07:24:18.180 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:18.179+0000 7f70aaddf640 1 -- 172.21.15.143:0/1963402706 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "fs flag set", "flag_name": "enable_multiple", "val": "true", "yes_i_really_mean_it": true} v 0) v1 -- 0x7f70a4133ef0 con 0x7f708c003980 2023-11-26T07:24:18.539 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:18.539+0000 7f70997fa640 1 -- 172.21.15.143:0/1963402706 <== mon.0 v2:172.21.15.143:3300/0 7 ==== mon_command_ack([{"prefix": "fs flag set", "flag_name": "enable_multiple", "val": "true", "yes_i_really_mean_it": true}]=0 v66) v1 ==== 136+0+0 (secure 0 0 0) 0x7f70a00efbb0 con 0x7f708c003980 2023-11-26T07:24:18.541 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:18.539+0000 7f70aaddf640 1 -- 172.21.15.143:0/1963402706 >> [v2:172.21.15.188:6836/3401128553,v1:172.21.15.188:6837/3401128553] conn(0x7f7090066590 msgr2=0x7f7090068a50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:24:18.541 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:18.539+0000 7f70aaddf640 1 --2- 172.21.15.143:0/1963402706 >> [v2:172.21.15.188:6836/3401128553,v1:172.21.15.188:6837/3401128553] conn(0x7f7090066590 0x7f7090068a50 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7f70a40b3c70 tx=0x7f7094008040 comp rx=0 tx=0).stop 2023-11-26T07:24:18.541 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:18.539+0000 7f70aaddf640 1 -- 172.21.15.143:0/1963402706 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f708c003980 msgr2=0x7f70a4154d20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:24:18.542 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:18.539+0000 7f70aaddf640 1 --2- 172.21.15.143:0/1963402706 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f708c003980 0x7f70a4154d20 secure :-1 s=READY pgs=1332 cs=0 l=1 rev1=1 crypto rx=0x7f70a0009b40 tx=0x7f708c005800 comp rx=0 tx=0).stop 2023-11-26T07:24:18.542 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:18.539+0000 7f70aaddf640 1 -- 172.21.15.143:0/1963402706 shutdown_connections 2023-11-26T07:24:18.542 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:18.539+0000 7f70aaddf640 1 --2- 172.21.15.143:0/1963402706 >> [v2:172.21.15.188:6836/3401128553,v1:172.21.15.188:6837/3401128553] conn(0x7f7090066590 0x7f7090068a50 unknown :-1 s=CLOSED pgs=136 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:24:18.542 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:18.539+0000 7f70aaddf640 1 --2- 172.21.15.143:0/1963402706 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f70a40b2d00 0x7f70a40b3180 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:24:18.542 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:18.539+0000 7f70aaddf640 1 --2- 172.21.15.143:0/1963402706 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f70a4139450 0x7f70a41398d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:24:18.542 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:18.539+0000 7f70aaddf640 1 --2- 172.21.15.143:0/1963402706 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f708c003980 0x7f70a4154d20 unknown :-1 s=CLOSED pgs=1332 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:24:18.542 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:18.539+0000 7f70aaddf640 1 -- 172.21.15.143:0/1963402706 >> 172.21.15.143:0/1963402706 conn(0x7f70a406a890 msgr2=0x7f70a415d520 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:24:18.542 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:18.539+0000 7f70aaddf640 1 -- 172.21.15.143:0/1963402706 shutdown_connections 2023-11-26T07:24:18.542 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:18.539+0000 7f70aaddf640 1 -- 172.21.15.143:0/1963402706 wait complete. 2023-11-26T07:24:18.561 DEBUG:teuthology.orchestra.run.smithi143:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs volume ls 2023-11-26T07:24:18.699 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:18.699+0000 7f6a71013640 1 Processor -- start 2023-11-26T07:24:18.699 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:18.699+0000 7f6a71013640 1 -- start start 2023-11-26T07:24:18.701 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:18.699+0000 7f6a71013640 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7f6a6c15d8f0 0x7f6a6c0cec40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:24:18.701 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:18.699+0000 7f6a71013640 1 -- --> v1:172.21.15.143:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6a6c0ccfa0 con 0x7f6a6c0745d0 2023-11-26T07:24:18.701 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:18.699+0000 7f6a71013640 1 -- --> v1:172.21.15.188:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6a6c06e280 con 0x7f6a6c1614f0 2023-11-26T07:24:18.701 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:18.699+0000 7f6a71013640 1 -- --> v2:172.21.15.188:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6a6c06e8b0 con 0x7f6a6c15d8f0 2023-11-26T07:24:18.701 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:18.699+0000 7f6a6a59c640 1 --1- >> v1:172.21.15.143:6789/0 conn(0x7f6a6c0745d0 0x7f6a6c15d1e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.143:6789/0 says I am v1:172.21.15.143:40224/0 (socket says 172.21.15.143:40224) 2023-11-26T07:24:18.701 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:18.699+0000 7f6a6a59c640 1 -- 172.21.15.143:0/3402184928 learned_addr learned my addr 172.21.15.143:0/3402184928 (peer_addr_for_me v1:172.21.15.143:0/0) 2023-11-26T07:24:18.703 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:18.699+0000 7f6a6b59e640 1 --2- 172.21.15.143:0/3402184928 >> v2:172.21.15.188:3300/0 conn(0x7f6a6c15d8f0 0x7f6a6c0cec40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:24:18.703 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:18.699+0000 7f6a69d9b640 1 -- 172.21.15.143:0/3402184928 <== mon.0 v1:172.21.15.143:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3923810777 0 0) 0x7f6a6c0ccfa0 con 0x7f6a6c0745d0 2023-11-26T07:24:18.703 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:18.699+0000 7f6a69d9b640 1 -- 172.21.15.143:0/3402184928 --> v1:172.21.15.143:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f6a54003530 con 0x7f6a6c0745d0 2023-11-26T07:24:18.703 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:18.699+0000 7f6a69d9b640 1 -- 172.21.15.143:0/3402184928 <== mon.0 v1:172.21.15.143:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 4000215256 0 0) 0x7f6a54003530 con 0x7f6a6c0745d0 2023-11-26T07:24:18.703 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:18.699+0000 7f6a69d9b640 1 -- 172.21.15.143:0/3402184928 >> v2:172.21.15.188:3300/0 conn(0x7f6a6c15d8f0 msgr2=0x7f6a6c0cec40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-26T07:24:18.703 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:18.703+0000 7f6a69d9b640 1 --2- 172.21.15.143:0/3402184928 >> v2:172.21.15.188:3300/0 conn(0x7f6a6c15d8f0 0x7f6a6c0cec40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:24:18.703 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:18.703+0000 7f6a69d9b640 1 -- 172.21.15.143:0/3402184928 >> v1:172.21.15.188:6789/0 conn(0x7f6a6c1614f0 legacy=0x7f6a6c1618d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:24:18.703 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:18.703+0000 7f6a69d9b640 1 -- 172.21.15.143:0/3402184928 --> v1:172.21.15.143:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6a6c0cd240 con 0x7f6a6c0745d0 2023-11-26T07:24:18.703 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:18.703+0000 7f6a69d9b640 1 -- 172.21.15.143:0/3402184928 <== mon.0 v1:172.21.15.143:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2137882523 0 0) 0x7f6a60001680 con 0x7f6a6c0745d0 2023-11-26T07:24:18.703 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:18.703+0000 7f6a69d9b640 1 -- 172.21.15.143:0/3402184928 >> v1:172.21.15.143:6789/0 conn(0x7f6a6c0745d0 legacy=0x7f6a6c15d1e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:24:18.703 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:18.703+0000 7f6a69d9b640 1 --2- 172.21.15.143:0/3402184928 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f6a54003980 0x7f6a54003d60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:24:18.704 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:18.703+0000 7f6a69d9b640 1 --2- 172.21.15.143:0/3402184928 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f6a54004380 0x7f6a54004850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:24:18.704 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:18.703+0000 7f6a69d9b640 1 --2- 172.21.15.143:0/3402184928 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f6a54005050 0x7f6a5400b990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:24:18.704 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:18.703+0000 7f6a69d9b640 1 -- 172.21.15.143:0/3402184928 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6a6c06e8b0 con 0x7f6a54003980 2023-11-26T07:24:18.704 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:18.703+0000 7f6a69d9b640 1 -- 172.21.15.143:0/3402184928 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6a5400bfb0 con 0x7f6a54005050 2023-11-26T07:24:18.704 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:18.703+0000 7f6a69d9b640 1 -- 172.21.15.143:0/3402184928 --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6a5400c170 con 0x7f6a54004380 2023-11-26T07:24:18.704 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:18.703+0000 7f6a6ad9d640 1 --2- 172.21.15.143:0/3402184928 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f6a54003980 0x7f6a54003d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:24:18.704 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:18.703+0000 7f6a6b59e640 1 --2- 172.21.15.143:0/3402184928 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f6a54005050 0x7f6a5400b990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:24:18.704 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:18.703+0000 7f6a6a59c640 1 --2- 172.21.15.143:0/3402184928 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f6a54004380 0x7f6a54004850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:24:18.704 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:18.703+0000 7f6a6ad9d640 1 -- 172.21.15.143:0/3402184928 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f6a54004380 msgr2=0x7f6a54004850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-26T07:24:18.704 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:18.703+0000 7f6a6ad9d640 1 --2- 172.21.15.143:0/3402184928 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f6a54004380 0x7f6a54004850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:24:18.704 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:18.703+0000 7f6a6ad9d640 1 -- 172.21.15.143:0/3402184928 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f6a54005050 msgr2=0x7f6a5400b990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-26T07:24:18.704 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:18.703+0000 7f6a6ad9d640 1 --2- 172.21.15.143:0/3402184928 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f6a54005050 0x7f6a5400b990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:24:18.705 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:18.703+0000 7f6a6ad9d640 1 -- 172.21.15.143:0/3402184928 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6a54003530 con 0x7f6a54003980 2023-11-26T07:24:18.705 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:18.703+0000 7f6a6b59e640 1 -- 172.21.15.143:0/3402184928 reap_dead start 2023-11-26T07:24:18.705 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:18.703+0000 7f6a6ad9d640 1 --2- 172.21.15.143:0/3402184928 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f6a54003980 0x7f6a54003d60 secure :-1 s=READY pgs=1334 cs=0 l=1 rev1=1 crypto rx=0x7f6a5c001860 tx=0x7f6a5c01c4b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=86f24111f9555b2b server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:24:18.706 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:18.707+0000 7f6a69d9b640 1 -- 172.21.15.143:0/3402184928 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6a5c0ab600 con 0x7f6a54003980 2023-11-26T07:24:18.706 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:18.707+0000 7f6a69d9b640 1 -- 172.21.15.143:0/3402184928 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6a5c0a7050 con 0x7f6a54003980 2023-11-26T07:24:18.706 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:18.707+0000 7f6a69d9b640 1 -- 172.21.15.143:0/3402184928 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6a5c0ab9e0 con 0x7f6a54003980 2023-11-26T07:24:18.707 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:18.707+0000 7f6a71013640 1 -- 172.21.15.143:0/3402184928 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f6a54003980 msgr2=0x7f6a54003d60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:24:18.707 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:18.707+0000 7f6a71013640 1 --2- 172.21.15.143:0/3402184928 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f6a54003980 0x7f6a54003d60 secure :-1 s=READY pgs=1334 cs=0 l=1 rev1=1 crypto rx=0x7f6a5c001860 tx=0x7f6a5c01c4b0 comp rx=0 tx=0).stop 2023-11-26T07:24:18.707 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:18.707+0000 7f6a71013640 1 -- 172.21.15.143:0/3402184928 shutdown_connections 2023-11-26T07:24:18.707 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:18.707+0000 7f6a71013640 1 --2- 172.21.15.143:0/3402184928 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f6a54003980 0x7f6a54003d60 unknown :-1 s=CLOSED pgs=1334 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:24:18.707 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:18.707+0000 7f6a71013640 1 -- 172.21.15.143:0/3402184928 >> 172.21.15.143:0/3402184928 conn(0x7f6a6c0b0d80 msgr2=0x7f6a6c0b1190 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:24:18.707 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:18.707+0000 7f6a71013640 1 -- 172.21.15.143:0/3402184928 shutdown_connections 2023-11-26T07:24:18.707 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:18.707+0000 7f6a71013640 1 -- 172.21.15.143:0/3402184928 wait complete. 2023-11-26T07:24:18.708 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:18.707+0000 7f6a71013640 1 Processor -- start 2023-11-26T07:24:18.708 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:18.707+0000 7f6a71013640 1 -- start start 2023-11-26T07:24:18.710 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:18.707+0000 7f6a71013640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f6a54003980 0x7f6a6c0c5620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:24:18.711 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:18.707+0000 7f6a71013640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f6a6c1467e0 0x7f6a6c1eb4e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:24:18.711 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:18.707+0000 7f6a71013640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f6a6c146c60 0x7f6a6c1eba20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:24:18.711 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:18.707+0000 7f6a71013640 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6a6c15ff50 con 0x7f6a6c146c60 2023-11-26T07:24:18.711 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:18.707+0000 7f6a71013640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6a6c0c5b60 con 0x7f6a54003980 2023-11-26T07:24:18.711 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:18.707+0000 7f6a71013640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6a6c15e0a0 con 0x7f6a6c1467e0 2023-11-26T07:24:18.711 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:18.711+0000 7f6a6b59e640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f6a6c146c60 0x7f6a6c1eba20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:24:18.711 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:18.711+0000 7f6a6b59e640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f6a6c146c60 0x7f6a6c1eba20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:54592/0 (socket says 172.21.15.143:54592) 2023-11-26T07:24:18.711 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:18.711+0000 7f6a6b59e640 1 -- 172.21.15.143:0/3433484416 learned_addr learned my addr 172.21.15.143:0/3433484416 (peer_addr_for_me v2:172.21.15.143:0/0) 2023-11-26T07:24:18.711 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:18.711+0000 7f6a6a59c640 1 --2- 172.21.15.143:0/3433484416 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f6a6c1467e0 0x7f6a6c1eb4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:24:18.711 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:18.711+0000 7f6a6ad9d640 1 --2- 172.21.15.143:0/3433484416 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f6a54003980 0x7f6a6c0c5620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:24:18.711 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:18.711+0000 7f6a6b59e640 1 -- 172.21.15.143:0/3433484416 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f6a6c1467e0 msgr2=0x7f6a6c1eb4e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:24:18.711 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:18.711+0000 7f6a6b59e640 1 --2- 172.21.15.143:0/3433484416 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f6a6c1467e0 0x7f6a6c1eb4e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:24:18.712 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:18.711+0000 7f6a6b59e640 1 -- 172.21.15.143:0/3433484416 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f6a54003980 msgr2=0x7f6a6c0c5620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:24:18.712 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:18.711+0000 7f6a6b59e640 1 --2- 172.21.15.143:0/3433484416 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f6a54003980 0x7f6a6c0c5620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:24:18.712 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:18.711+0000 7f6a6b59e640 1 -- 172.21.15.143:0/3433484416 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6a6c149450 con 0x7f6a6c146c60 2023-11-26T07:24:18.712 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:18.711+0000 7f6a6ad9d640 1 --2- 172.21.15.143:0/3433484416 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f6a54003980 0x7f6a6c0c5620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-26T07:24:18.712 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:18.711+0000 7f6a6b59e640 1 --2- 172.21.15.143:0/3433484416 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f6a6c146c60 0x7f6a6c1eba20 secure :-1 s=READY pgs=1335 cs=0 l=1 rev1=1 crypto rx=0x7f6a6c0b1e10 tx=0x7f6a6400f580 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:24:18.712 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:18.711+0000 7f6a537fe640 1 -- 172.21.15.143:0/3433484416 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6a6400e040 con 0x7f6a6c146c60 2023-11-26T07:24:18.712 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:18.711+0000 7f6a537fe640 1 -- 172.21.15.143:0/3433484416 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6a6400f040 con 0x7f6a6c146c60 2023-11-26T07:24:18.712 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:18.711+0000 7f6a537fe640 1 -- 172.21.15.143:0/3433484416 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6a640039a0 con 0x7f6a6c146c60 2023-11-26T07:24:18.712 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:18.711+0000 7f6a71013640 1 -- 172.21.15.143:0/3433484416 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6a6c15f890 con 0x7f6a6c146c60 2023-11-26T07:24:18.712 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:18.711+0000 7f6a71013640 1 -- 172.21.15.143:0/3433484416 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6a6c149730 con 0x7f6a6c146c60 2023-11-26T07:24:18.713 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:18.711+0000 7f6a71013640 1 -- 172.21.15.143:0/3433484416 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6a6c15f6f0 con 0x7f6a6c146c60 2023-11-26T07:24:18.713 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:18.711+0000 7f6a537fe640 1 -- 172.21.15.143:0/3433484416 <== mon.0 v2:172.21.15.143:3300/0 4 ==== mgrmap(e 16) v1 ==== 85954+0+0 (secure 0 0 0) 0x7f6a64003b00 con 0x7f6a6c146c60 2023-11-26T07:24:18.714 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:18.715+0000 7f6a537fe640 1 --2- 172.21.15.143:0/3433484416 >> [v2:172.21.15.188:6836/3401128553,v1:172.21.15.188:6837/3401128553] conn(0x7f6a580665c0 0x7f6a58068a80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:24:18.714 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:18.715+0000 7f6a537fe640 1 -- 172.21.15.143:0/3433484416 <== mon.0 v2:172.21.15.143:3300/0 5 ==== osd_map(103..103 src has 1..103) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f6a640848f0 con 0x7f6a6c146c60 2023-11-26T07:24:18.715 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:18.715+0000 7f6a6ad9d640 1 --2- 172.21.15.143:0/3433484416 >> [v2:172.21.15.188:6836/3401128553,v1:172.21.15.188:6837/3401128553] conn(0x7f6a580665c0 0x7f6a58068a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:24:18.715 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:18.715+0000 7f6a6ad9d640 1 --2- 172.21.15.143:0/3433484416 >> [v2:172.21.15.188:6836/3401128553,v1:172.21.15.188:6837/3401128553] conn(0x7f6a580665c0 0x7f6a58068a80 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7f6a5c0ad000 tx=0x7f6a5c0a6000 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:24:18.719 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:18.719+0000 7f6a537fe640 1 -- 172.21.15.143:0/3433484416 <== mon.0 v2:172.21.15.143:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172649 (secure 0 0 0) 0x7f6a6c15f6f0 con 0x7f6a6c146c60 2023-11-26T07:24:18.887 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:18.887+0000 7f6a71013640 1 -- 172.21.15.143:0/3433484416 --> [v2:172.21.15.188:6836/3401128553,v1:172.21.15.188:6837/3401128553] -- mgr_command(tid 0: {"prefix": "fs volume ls", "target": ["mon-mgr", ""]}) v1 -- 0x7f6a6c1620e0 con 0x7f6a580665c0 2023-11-26T07:24:18.889 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:18.887+0000 7f6a537fe640 1 -- 172.21.15.143:0/3433484416 <== mgr.4108 v2:172.21.15.188:6836/3401128553 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+41 (secure 0 0 0) 0x7f6a5c006b80 con 0x7f6a580665c0 2023-11-26T07:24:18.889 INFO:teuthology.orchestra.run.smithi143.stdout:[ 2023-11-26T07:24:18.890 INFO:teuthology.orchestra.run.smithi143.stdout: { 2023-11-26T07:24:18.890 INFO:teuthology.orchestra.run.smithi143.stdout: "name": "cephfs" 2023-11-26T07:24:18.890 INFO:teuthology.orchestra.run.smithi143.stdout: } 2023-11-26T07:24:18.890 INFO:teuthology.orchestra.run.smithi143.stdout:] 2023-11-26T07:24:18.892 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:18.891+0000 7f6a71013640 1 -- 172.21.15.143:0/3433484416 >> [v2:172.21.15.188:6836/3401128553,v1:172.21.15.188:6837/3401128553] conn(0x7f6a580665c0 msgr2=0x7f6a58068a80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:24:18.892 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:18.891+0000 7f6a71013640 1 --2- 172.21.15.143:0/3433484416 >> [v2:172.21.15.188:6836/3401128553,v1:172.21.15.188:6837/3401128553] conn(0x7f6a580665c0 0x7f6a58068a80 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7f6a5c0ad000 tx=0x7f6a5c0a6000 comp rx=0 tx=0).stop 2023-11-26T07:24:18.892 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:18.891+0000 7f6a71013640 1 -- 172.21.15.143:0/3433484416 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f6a6c146c60 msgr2=0x7f6a6c1eba20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:24:18.892 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:18.891+0000 7f6a71013640 1 --2- 172.21.15.143:0/3433484416 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f6a6c146c60 0x7f6a6c1eba20 secure :-1 s=READY pgs=1335 cs=0 l=1 rev1=1 crypto rx=0x7f6a6c0b1e10 tx=0x7f6a6400f580 comp rx=0 tx=0).stop 2023-11-26T07:24:18.892 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:18.891+0000 7f6a71013640 1 -- 172.21.15.143:0/3433484416 shutdown_connections 2023-11-26T07:24:18.893 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:18.891+0000 7f6a71013640 1 --2- 172.21.15.143:0/3433484416 >> [v2:172.21.15.188:6836/3401128553,v1:172.21.15.188:6837/3401128553] conn(0x7f6a580665c0 0x7f6a58068a80 unknown :-1 s=CLOSED pgs=137 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:24:18.893 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:18.891+0000 7f6a71013640 1 --2- 172.21.15.143:0/3433484416 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f6a6c146c60 0x7f6a6c1eba20 unknown :-1 s=CLOSED pgs=1335 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:24:18.893 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:18.891+0000 7f6a71013640 1 --2- 172.21.15.143:0/3433484416 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f6a6c1467e0 0x7f6a6c1eb4e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:24:18.893 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:18.891+0000 7f6a71013640 1 --2- 172.21.15.143:0/3433484416 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f6a54003980 0x7f6a6c0c5620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:24:18.893 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:18.891+0000 7f6a71013640 1 -- 172.21.15.143:0/3433484416 >> 172.21.15.143:0/3433484416 conn(0x7f6a6c0b0d80 msgr2=0x7f6a6c15c160 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:24:18.893 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:18.891+0000 7f6a71013640 1 -- 172.21.15.143:0/3433484416 shutdown_connections 2023-11-26T07:24:18.893 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:18.891+0000 7f6a71013640 1 -- 172.21.15.143:0/3433484416 wait complete. 2023-11-26T07:24:18.911 DEBUG:teuthology.orchestra.run.smithi143:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mon mon_allow_pool_delete True 2023-11-26T07:24:19.052 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:19.051+0000 7f70acbdf640 1 Processor -- start 2023-11-26T07:24:19.053 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:19.051+0000 7f70acbdf640 1 -- start start 2023-11-26T07:24:19.054 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:19.051+0000 7f70acbdf640 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7f70a8143ac0 0x7f70a8070370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:24:19.054 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:19.051+0000 7f70acbdf640 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7f70a80cf990 0x7f70a80cd1b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:24:19.054 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:19.051+0000 7f70acbdf640 1 -- --> v1:172.21.15.188:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f70a80cc730 con 0x7f70a80708b0 2023-11-26T07:24:19.054 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:19.051+0000 7f70acbdf640 1 -- --> v2:172.21.15.143:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f70a8143ea0 con 0x7f70a8143ac0 2023-11-26T07:24:19.054 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:19.051+0000 7f70acbdf640 1 -- --> v2:172.21.15.188:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f70a8070ee0 con 0x7f70a80cf990 2023-11-26T07:24:19.054 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:19.055+0000 7f70a659c640 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7f70a8143ac0 0x7f70a8070370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:24:19.055 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:19.055+0000 7f70a659c640 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7f70a8143ac0 0x7f70a8070370 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:54602/0 (socket says 172.21.15.143:54602) 2023-11-26T07:24:19.055 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:19.055+0000 7f70a659c640 1 -- 172.21.15.143:0/939365490 learned_addr learned my addr 172.21.15.143:0/939365490 (peer_addr_for_me v2:172.21.15.143:0/0) 2023-11-26T07:24:19.055 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:19.055+0000 7f70a6d9d640 1 --2- 172.21.15.143:0/939365490 >> v2:172.21.15.188:3300/0 conn(0x7f70a80cf990 0x7f70a80cd1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:24:19.055 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:19.055+0000 7f70a659c640 1 -- 172.21.15.143:0/939365490 >> v2:172.21.15.188:3300/0 conn(0x7f70a80cf990 msgr2=0x7f70a80cd1b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-26T07:24:19.055 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:19.055+0000 7f70a659c640 1 --2- 172.21.15.143:0/939365490 >> v2:172.21.15.188:3300/0 conn(0x7f70a80cf990 0x7f70a80cd1b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:24:19.055 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:19.055+0000 7f70a659c640 1 -- 172.21.15.143:0/939365490 >> v1:172.21.15.188:6789/0 conn(0x7f70a80708b0 legacy=0x7f70a80cf280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:24:19.055 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:19.055+0000 7f70a659c640 1 -- 172.21.15.143:0/939365490 --> v2:172.21.15.143:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f70a80711c0 con 0x7f70a8143ac0 2023-11-26T07:24:19.055 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:19.055+0000 7f70a6d9d640 1 --2- 172.21.15.143:0/939365490 >> v2:172.21.15.188:3300/0 conn(0x7f70a80cf990 0x7f70a80cd1b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-26T07:24:19.057 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:19.055+0000 7f70a659c640 1 --2- 172.21.15.143:0/939365490 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f70a8143ac0 0x7f70a8070370 secure :-1 s=READY pgs=1336 cs=0 l=1 rev1=1 crypto rx=0x7f70a806e570 tx=0x7f709801c9d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=316052641001b2da server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:24:19.057 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:19.055+0000 7f70a5d9b640 1 -- 172.21.15.143:0/939365490 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f70980a57a0 con 0x7f70a8143ac0 2023-11-26T07:24:19.057 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:19.055+0000 7f70a5d9b640 1 -- 172.21.15.143:0/939365490 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f70980a5900 con 0x7f70a8143ac0 2023-11-26T07:24:19.057 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:19.055+0000 7f70a5d9b640 1 -- 172.21.15.143:0/939365490 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f70980a5c90 con 0x7f70a8143ac0 2023-11-26T07:24:19.058 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:19.059+0000 7f70acbdf640 1 -- 172.21.15.143:0/939365490 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f70a8143ac0 msgr2=0x7f70a8070370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:24:19.058 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:19.059+0000 7f70acbdf640 1 --2- 172.21.15.143:0/939365490 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f70a8143ac0 0x7f70a8070370 secure :-1 s=READY pgs=1336 cs=0 l=1 rev1=1 crypto rx=0x7f70a806e570 tx=0x7f709801c9d0 comp rx=0 tx=0).stop 2023-11-26T07:24:19.058 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:19.059+0000 7f70acbdf640 1 -- 172.21.15.143:0/939365490 shutdown_connections 2023-11-26T07:24:19.058 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:19.059+0000 7f70acbdf640 1 --2- 172.21.15.143:0/939365490 >> v2:172.21.15.188:3300/0 conn(0x7f70a80cf990 0x7f70a80cd1b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:24:19.059 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:19.059+0000 7f70acbdf640 1 --2- 172.21.15.143:0/939365490 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f70a8143ac0 0x7f70a8070370 unknown :-1 s=CLOSED pgs=1336 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:24:19.059 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:19.059+0000 7f70acbdf640 1 -- 172.21.15.143:0/939365490 >> 172.21.15.143:0/939365490 conn(0x7f70a806a890 msgr2=0x7f70a806aca0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:24:19.059 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:19.059+0000 7f70acbdf640 1 -- 172.21.15.143:0/939365490 shutdown_connections 2023-11-26T07:24:19.059 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:19.059+0000 7f70acbdf640 1 -- 172.21.15.143:0/939365490 wait complete. 2023-11-26T07:24:19.060 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:19.059+0000 7f70acbdf640 1 Processor -- start 2023-11-26T07:24:19.060 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:19.059+0000 7f70acbdf640 1 -- start start 2023-11-26T07:24:19.062 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:19.059+0000 7f70acbdf640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f70a80708b0 0x7f70a814b8b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:24:19.062 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:19.059+0000 7f70acbdf640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f70a80cf990 0x7f70a80ab020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:24:19.062 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:19.059+0000 7f70acbdf640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f70a8143ac0 0x7f70a815c9d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:24:19.062 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:19.059+0000 7f70acbdf640 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f70a80a84d0 con 0x7f70a8143ac0 2023-11-26T07:24:19.062 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:19.059+0000 7f70acbdf640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f70a814bdf0 con 0x7f70a80cf990 2023-11-26T07:24:19.062 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:19.059+0000 7f70acbdf640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f70a80adbf0 con 0x7f70a80708b0 2023-11-26T07:24:19.062 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:19.059+0000 7f70a6d9d640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f70a8143ac0 0x7f70a815c9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:24:19.062 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:19.059+0000 7f70a6d9d640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f70a8143ac0 0x7f70a815c9d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:54616/0 (socket says 172.21.15.143:54616) 2023-11-26T07:24:19.063 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:19.059+0000 7f70a6d9d640 1 -- 172.21.15.143:0/1965244087 learned_addr learned my addr 172.21.15.143:0/1965244087 (peer_addr_for_me v2:172.21.15.143:0/0) 2023-11-26T07:24:19.063 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:19.059+0000 7f70a659c640 1 --2- 172.21.15.143:0/1965244087 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f70a80708b0 0x7f70a814b8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:24:19.063 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:19.063+0000 7f709dd9b640 1 --2- 172.21.15.143:0/1965244087 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f70a80cf990 0x7f70a80ab020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:24:19.063 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:19.063+0000 7f70a6d9d640 1 -- 172.21.15.143:0/1965244087 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f70a80708b0 msgr2=0x7f70a814b8b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:24:19.063 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:19.063+0000 7f70a6d9d640 1 --2- 172.21.15.143:0/1965244087 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f70a80708b0 0x7f70a814b8b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:24:19.063 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:19.063+0000 7f70a6d9d640 1 -- 172.21.15.143:0/1965244087 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f70a80cf990 msgr2=0x7f70a80ab020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:24:19.063 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:19.063+0000 7f70a6d9d640 1 --2- 172.21.15.143:0/1965244087 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f70a80cf990 0x7f70a80ab020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:24:19.063 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:19.063+0000 7f70a6d9d640 1 -- 172.21.15.143:0/1965244087 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7098009b30 con 0x7f70a8143ac0 2023-11-26T07:24:19.063 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:19.063+0000 7f70a6d9d640 1 --2- 172.21.15.143:0/1965244087 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f70a8143ac0 0x7f70a815c9d0 secure :-1 s=READY pgs=1337 cs=0 l=1 rev1=1 crypto rx=0x7f70a000e810 tx=0x7f70a000ecd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:24:19.064 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:19.063+0000 7f709f7fe640 1 -- 172.21.15.143:0/1965244087 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f70a000c670 con 0x7f70a8143ac0 2023-11-26T07:24:19.064 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:19.063+0000 7f709f7fe640 1 -- 172.21.15.143:0/1965244087 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f70a000c7d0 con 0x7f70a8143ac0 2023-11-26T07:24:19.064 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:19.063+0000 7f70acbdf640 1 -- 172.21.15.143:0/1965244087 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f70a80aded0 con 0x7f70a8143ac0 2023-11-26T07:24:19.064 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:19.063+0000 7f709f7fe640 1 -- 172.21.15.143:0/1965244087 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f70a000cb80 con 0x7f70a8143ac0 2023-11-26T07:24:19.064 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:19.063+0000 7f70acbdf640 1 -- 172.21.15.143:0/1965244087 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f70a80a86b0 con 0x7f70a8143ac0 2023-11-26T07:24:19.064 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:19.063+0000 7f70acbdf640 1 -- 172.21.15.143:0/1965244087 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f70a80ae3f0 con 0x7f70a8143ac0 2023-11-26T07:24:19.065 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:19.063+0000 7f709f7fe640 1 -- 172.21.15.143:0/1965244087 <== mon.0 v2:172.21.15.143:3300/0 4 ==== mgrmap(e 16) v1 ==== 85954+0+0 (secure 0 0 0) 0x7f70a000cce0 con 0x7f70a8143ac0 2023-11-26T07:24:19.065 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:19.063+0000 7f709f7fe640 1 --2- 172.21.15.143:0/1965244087 >> [v2:172.21.15.188:6836/3401128553,v1:172.21.15.188:6837/3401128553] conn(0x7f7088066590 0x7f7088068a50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:24:19.065 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:19.063+0000 7f709f7fe640 1 -- 172.21.15.143:0/1965244087 <== mon.0 v2:172.21.15.143:3300/0 5 ==== osd_map(103..103 src has 1..103) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f70a0083e60 con 0x7f70a8143ac0 2023-11-26T07:24:19.066 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:19.067+0000 7f70a659c640 1 --2- 172.21.15.143:0/1965244087 >> [v2:172.21.15.188:6836/3401128553,v1:172.21.15.188:6837/3401128553] conn(0x7f7088066590 0x7f7088068a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:24:19.066 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:19.067+0000 7f70a659c640 1 --2- 172.21.15.143:0/1965244087 >> [v2:172.21.15.188:6836/3401128553,v1:172.21.15.188:6837/3401128553] conn(0x7f7088066590 0x7f7088068a50 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7f70980ae000 tx=0x7f70980a7070 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:24:19.069 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:19.067+0000 7f709f7fe640 1 -- 172.21.15.143:0/1965244087 <== mon.0 v2:172.21.15.143:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172649 (secure 0 0 0) 0x7f70a80ae3f0 con 0x7f70a8143ac0 2023-11-26T07:24:19.219 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:19.219+0000 7f70acbdf640 1 -- 172.21.15.143:0/1965244087 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command([{prefix=config set, name=mon_allow_pool_delete}] v 0) v1 -- 0x7f70a80ae3f0 con 0x7f70a8143ac0 2023-11-26T07:24:19.219 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:19.219+0000 7f709f7fe640 1 -- 172.21.15.143:0/1965244087 <== mon.0 v2:172.21.15.143:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mon_allow_pool_delete}]=0 v20)=0 v20) v1 ==== 122+0+0 (secure 0 0 0) 0x7f70a80ae3f0 con 0x7f70a8143ac0 2023-11-26T07:24:19.221 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:19.219+0000 7f70acbdf640 1 -- 172.21.15.143:0/1965244087 >> [v2:172.21.15.188:6836/3401128553,v1:172.21.15.188:6837/3401128553] conn(0x7f7088066590 msgr2=0x7f7088068a50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:24:19.221 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:19.219+0000 7f70acbdf640 1 --2- 172.21.15.143:0/1965244087 >> [v2:172.21.15.188:6836/3401128553,v1:172.21.15.188:6837/3401128553] conn(0x7f7088066590 0x7f7088068a50 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7f70980ae000 tx=0x7f70980a7070 comp rx=0 tx=0).stop 2023-11-26T07:24:19.222 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:19.219+0000 7f70acbdf640 1 -- 172.21.15.143:0/1965244087 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f70a8143ac0 msgr2=0x7f70a815c9d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:24:19.222 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:19.219+0000 7f70acbdf640 1 --2- 172.21.15.143:0/1965244087 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f70a8143ac0 0x7f70a815c9d0 secure :-1 s=READY pgs=1337 cs=0 l=1 rev1=1 crypto rx=0x7f70a000e810 tx=0x7f70a000ecd0 comp rx=0 tx=0).stop 2023-11-26T07:24:19.222 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:19.219+0000 7f70acbdf640 1 -- 172.21.15.143:0/1965244087 shutdown_connections 2023-11-26T07:24:19.222 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:19.219+0000 7f70acbdf640 1 --2- 172.21.15.143:0/1965244087 >> [v2:172.21.15.188:6836/3401128553,v1:172.21.15.188:6837/3401128553] conn(0x7f7088066590 0x7f7088068a50 unknown :-1 s=CLOSED pgs=138 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:24:19.222 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:19.219+0000 7f70acbdf640 1 --2- 172.21.15.143:0/1965244087 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f70a8143ac0 0x7f70a815c9d0 unknown :-1 s=CLOSED pgs=1337 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:24:19.222 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:19.219+0000 7f70acbdf640 1 --2- 172.21.15.143:0/1965244087 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f70a80cf990 0x7f70a80ab020 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:24:19.222 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:19.219+0000 7f70acbdf640 1 --2- 172.21.15.143:0/1965244087 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f70a80708b0 0x7f70a814b8b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:24:19.222 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:19.219+0000 7f70acbdf640 1 -- 172.21.15.143:0/1965244087 >> 172.21.15.143:0/1965244087 conn(0x7f70a806a890 msgr2=0x7f70a806c520 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:24:19.222 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:19.219+0000 7f70acbdf640 1 -- 172.21.15.143:0/1965244087 shutdown_connections 2023-11-26T07:24:19.222 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:19.219+0000 7f70acbdf640 1 -- 172.21.15.143:0/1965244087 wait complete. 2023-11-26T07:24:19.241 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-26T07:24:19.241 DEBUG:teuthology.orchestra.run.smithi143:> (cd /home/ubuntu/cephtest/mnt.0 && exec sudo mkdir -p volumes/_nogroup/subvol24520/f57b3d1b-7328-4835-8e29-9827b7800bab) 2023-11-26T07:24:19.273 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-26T07:24:19.273 DEBUG:teuthology.orchestra.run.smithi143:> (cd /home/ubuntu/cephtest/mnt.0 && exec sudo mkdir -p volumes/_nogroup/subvol24520/f57b3d1b-7328-4835-8e29-9827b7800bab/.snap/fake) 2023-11-26T07:24:19.348 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-26T07:24:19.348 DEBUG:teuthology.orchestra.run.smithi143:> (cd /home/ubuntu/cephtest/mnt.0 && exec getfattr --only-values -n ceph.dir.layout.pool .) 2023-11-26T07:24:19.404 INFO:teuthology.orchestra.run.smithi143.stdout:cephfs_data 2023-11-26T07:24:19.404 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-26T07:24:19.404 DEBUG:teuthology.orchestra.run.smithi143:> (cd /home/ubuntu/cephtest/mnt.0 && exec sudo setfattr -n ceph.dir.layout.pool -v cephfs_data volumes/_nogroup/subvol24520/f57b3d1b-7328-4835-8e29-9827b7800bab) 2023-11-26T07:24:19.471 DEBUG:teuthology.orchestra.run.smithi143:> set -ex 2023-11-26T07:24:19.471 DEBUG:teuthology.orchestra.run.smithi143:> sudo dd of=/home/ubuntu/cephtest/mnt.0/volumes/_nogroup/subvol24520/.meta 2023-11-26T07:24:19.543 DEBUG:teuthology.orchestra.run.smithi143:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume getpath cephfs subvol24520 2023-11-26T07:24:19.725 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:19.723+0000 7f1205b57640 1 Processor -- start 2023-11-26T07:24:19.725 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:19.723+0000 7f1205b57640 1 -- start start 2023-11-26T07:24:19.727 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:19.727+0000 7f1205b57640 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7f120013e290 0x7f12000cf280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:24:19.727 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:19.727+0000 7f1205b57640 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7f12000cf7c0 0x7f120015efa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:24:19.727 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:19.727+0000 7f1205b57640 1 -- --> v1:172.21.15.143:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f120006c520 con 0x7f120015f4e0 2023-11-26T07:24:19.727 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:19.727+0000 7f1205b57640 1 -- --> v2:172.21.15.143:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f12000cfc40 con 0x7f12000cf7c0 2023-11-26T07:24:19.727 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:19.727+0000 7f1205b57640 1 -- --> v2:172.21.15.188:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f120006c800 con 0x7f120013e290 2023-11-26T07:24:19.727 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:19.727+0000 7f11f7fff640 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7f12000cf7c0 0x7f120015efa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:24:19.727 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:19.727+0000 7f11fffff640 1 --1- >> v1:172.21.15.143:6789/0 conn(0x7f120015f4e0 0x7f12001408e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.143:6789/0 says I am v1:172.21.15.143:40232/0 (socket says 172.21.15.143:40232) 2023-11-26T07:24:19.727 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:19.727+0000 7f11fffff640 1 -- 172.21.15.143:0/2110328802 learned_addr learned my addr 172.21.15.143:0/2110328802 (peer_addr_for_me v1:172.21.15.143:0/0) 2023-11-26T07:24:19.727 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:19.727+0000 7f11ff7fe640 1 --2- 172.21.15.143:0/2110328802 >> v2:172.21.15.188:3300/0 conn(0x7f120013e290 0x7f12000cf280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:24:19.728 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:19.727+0000 7f11f7fff640 1 -- 172.21.15.143:0/2110328802 >> v2:172.21.15.188:3300/0 conn(0x7f120013e290 msgr2=0x7f12000cf280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-26T07:24:19.728 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:19.727+0000 7f11f7fff640 1 --2- 172.21.15.143:0/2110328802 >> v2:172.21.15.188:3300/0 conn(0x7f120013e290 0x7f12000cf280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:24:19.728 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:19.727+0000 7f11f7fff640 1 -- 172.21.15.143:0/2110328802 >> v1:172.21.15.143:6789/0 conn(0x7f120015f4e0 legacy=0x7f12001408e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:24:19.728 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:19.727+0000 7f11feffd640 1 -- 172.21.15.143:0/2110328802 <== mon.0 v1:172.21.15.143:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3473659325 0 0) 0x7f120006c520 con 0x7f120015f4e0 2023-11-26T07:24:19.728 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:19.727+0000 7f11f7fff640 1 -- 172.21.15.143:0/2110328802 --> v2:172.21.15.143:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f120006cae0 con 0x7f12000cf7c0 2023-11-26T07:24:19.730 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:19.731+0000 7f11f7fff640 1 --2- 172.21.15.143:0/2110328802 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f12000cf7c0 0x7f120015efa0 secure :-1 s=READY pgs=1339 cs=0 l=1 rev1=1 crypto rx=0x7f12000725a0 tx=0x7f11e801c430 comp rx=0 tx=0).ready entity=mon.0 client_cookie=26f3396f38b31256 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:24:19.730 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:19.731+0000 7f11feffd640 1 -- 172.21.15.143:0/2110328802 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f11e80a7070 con 0x7f12000cf7c0 2023-11-26T07:24:19.730 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:19.731+0000 7f11feffd640 1 -- 172.21.15.143:0/2110328802 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f11e80ab430 con 0x7f12000cf7c0 2023-11-26T07:24:19.730 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:19.731+0000 7f11feffd640 1 -- 172.21.15.143:0/2110328802 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f11e80ab7c0 con 0x7f12000cf7c0 2023-11-26T07:24:19.731 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:19.731+0000 7f1205b57640 1 -- 172.21.15.143:0/2110328802 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f12000cf7c0 msgr2=0x7f120015efa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:24:19.731 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:19.731+0000 7f1205b57640 1 --2- 172.21.15.143:0/2110328802 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f12000cf7c0 0x7f120015efa0 secure :-1 s=READY pgs=1339 cs=0 l=1 rev1=1 crypto rx=0x7f12000725a0 tx=0x7f11e801c430 comp rx=0 tx=0).stop 2023-11-26T07:24:19.731 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:19.731+0000 7f1205b57640 1 -- 172.21.15.143:0/2110328802 shutdown_connections 2023-11-26T07:24:19.731 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:19.731+0000 7f1205b57640 1 --2- 172.21.15.143:0/2110328802 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f12000cf7c0 0x7f120015efa0 unknown :-1 s=CLOSED pgs=1339 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:24:19.731 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:19.731+0000 7f1205b57640 1 --2- 172.21.15.143:0/2110328802 >> v2:172.21.15.188:3300/0 conn(0x7f120013e290 0x7f12000cf280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:24:19.731 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:19.731+0000 7f1205b57640 1 -- 172.21.15.143:0/2110328802 >> 172.21.15.143:0/2110328802 conn(0x7f120006a890 msgr2=0x7f120006aca0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:24:19.731 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:19.731+0000 7f1205b57640 1 -- 172.21.15.143:0/2110328802 shutdown_connections 2023-11-26T07:24:19.732 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:19.731+0000 7f1205b57640 1 -- 172.21.15.143:0/2110328802 wait complete. 2023-11-26T07:24:19.732 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:19.731+0000 7f1205b57640 1 Processor -- start 2023-11-26T07:24:19.732 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:19.731+0000 7f1205b57640 1 -- start start 2023-11-26T07:24:19.734 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:19.735+0000 7f1205b57640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f12000cf7c0 0x7f1200154d60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:24:19.734 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:19.735+0000 7f1205b57640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f120013e290 0x7f1200155d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:24:19.734 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:19.735+0000 7f1205b57640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f120015f4e0 0x7f12000aeb60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:24:19.734 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:19.735+0000 7f1205b57640 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f120015c700 con 0x7f120015f4e0 2023-11-26T07:24:19.734 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:19.735+0000 7f1205b57640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f120015c9e0 con 0x7f12000cf7c0 2023-11-26T07:24:19.734 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:19.735+0000 7f1205b57640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f120015ccc0 con 0x7f120013e290 2023-11-26T07:24:19.734 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:19.735+0000 7f11fffff640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f120015f4e0 0x7f12000aeb60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:24:19.735 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:19.735+0000 7f11fffff640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f120015f4e0 0x7f12000aeb60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:54620/0 (socket says 172.21.15.143:54620) 2023-11-26T07:24:19.735 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:19.735+0000 7f11fffff640 1 -- 172.21.15.143:0/1801817267 learned_addr learned my addr 172.21.15.143:0/1801817267 (peer_addr_for_me v2:172.21.15.143:0/0) 2023-11-26T07:24:19.735 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:19.735+0000 7f11f7fff640 1 --2- 172.21.15.143:0/1801817267 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f120013e290 0x7f1200155d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:24:19.735 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:19.735+0000 7f11ff7fe640 1 --2- 172.21.15.143:0/1801817267 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f12000cf7c0 0x7f1200154d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:24:19.736 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:19.735+0000 7f11fffff640 1 -- 172.21.15.143:0/1801817267 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f120013e290 msgr2=0x7f1200155d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:24:19.736 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:19.735+0000 7f11fffff640 1 --2- 172.21.15.143:0/1801817267 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f120013e290 0x7f1200155d10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:24:19.736 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:19.735+0000 7f11fffff640 1 -- 172.21.15.143:0/1801817267 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f12000cf7c0 msgr2=0x7f1200154d60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:24:19.736 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:19.735+0000 7f11fffff640 1 --2- 172.21.15.143:0/1801817267 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f12000cf7c0 0x7f1200154d60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:24:19.736 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:19.735+0000 7f11fffff640 1 -- 172.21.15.143:0/1801817267 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f11e8009420 con 0x7f120015f4e0 2023-11-26T07:24:19.736 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:19.735+0000 7f11fffff640 1 --2- 172.21.15.143:0/1801817267 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f120015f4e0 0x7f12000aeb60 secure :-1 s=READY pgs=1340 cs=0 l=1 rev1=1 crypto rx=0x7f11f8002240 tx=0x7f11f800cf50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:24:19.736 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:19.735+0000 7f11fcff9640 1 -- 172.21.15.143:0/1801817267 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f11f800ac30 con 0x7f120015f4e0 2023-11-26T07:24:19.736 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:19.735+0000 7f11fcff9640 1 -- 172.21.15.143:0/1801817267 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f11f800ad90 con 0x7f120015f4e0 2023-11-26T07:24:19.736 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:19.735+0000 7f1205b57640 1 -- 172.21.15.143:0/1801817267 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f120015cfa0 con 0x7f120015f4e0 2023-11-26T07:24:19.736 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:19.735+0000 7f11fcff9640 1 -- 172.21.15.143:0/1801817267 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f11f8012620 con 0x7f120015f4e0 2023-11-26T07:24:19.737 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:19.735+0000 7f1205b57640 1 -- 172.21.15.143:0/1801817267 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1200156250 con 0x7f120015f4e0 2023-11-26T07:24:19.737 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:19.735+0000 7f1205b57640 1 -- 172.21.15.143:0/1801817267 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f120015fb20 con 0x7f120015f4e0 2023-11-26T07:24:19.737 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:19.735+0000 7f11fcff9640 1 -- 172.21.15.143:0/1801817267 <== mon.0 v2:172.21.15.143:3300/0 4 ==== mgrmap(e 16) v1 ==== 85954+0+0 (secure 0 0 0) 0x7f11f8007070 con 0x7f120015f4e0 2023-11-26T07:24:19.738 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:19.739+0000 7f11fcff9640 1 --2- 172.21.15.143:0/1801817267 >> [v2:172.21.15.188:6836/3401128553,v1:172.21.15.188:6837/3401128553] conn(0x7f11e0066590 0x7f11e0068a50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:24:19.738 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:19.739+0000 7f11fcff9640 1 -- 172.21.15.143:0/1801817267 <== mon.0 v2:172.21.15.143:3300/0 5 ==== osd_map(103..103 src has 1..103) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f11f8083fe0 con 0x7f120015f4e0 2023-11-26T07:24:19.738 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:19.739+0000 7f11ff7fe640 1 --2- 172.21.15.143:0/1801817267 >> [v2:172.21.15.188:6836/3401128553,v1:172.21.15.188:6837/3401128553] conn(0x7f11e0066590 0x7f11e0068a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:24:19.739 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:19.739+0000 7f11ff7fe640 1 --2- 172.21.15.143:0/1801817267 >> [v2:172.21.15.188:6836/3401128553,v1:172.21.15.188:6837/3401128553] conn(0x7f11e0066590 0x7f11e0068a50 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7f11f0007a60 tx=0x7f11f0008040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:24:19.741 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:19.743+0000 7f11fcff9640 1 -- 172.21.15.143:0/1801817267 <== mon.0 v2:172.21.15.143:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172649 (secure 0 0 0) 0x7f120015fb20 con 0x7f120015f4e0 2023-11-26T07:24:19.913 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:19.911+0000 7f1205b57640 1 -- 172.21.15.143:0/1801817267 --> [v2:172.21.15.188:6836/3401128553,v1:172.21.15.188:6837/3401128553] -- mgr_command(tid 0: {"prefix": "fs subvolume getpath", "vol_name": "cephfs", "sub_name": "subvol24520", "target": ["mon-mgr", ""]}) v1 -- 0x7f12001564a0 con 0x7f11e0066590 2023-11-26T07:24:19.920 INFO:tasks.ceph.mgr.x.smithi188.stderr:2023-11-26T07:24:19.917+0000 7f04adb83640 -1 client.0 error registering admin socket command: (17) File exists 2023-11-26T07:24:19.920 INFO:tasks.ceph.mgr.x.smithi188.stderr:2023-11-26T07:24:19.917+0000 7f04adb83640 -1 client.0 error registering admin socket command: (17) File exists 2023-11-26T07:24:19.920 INFO:tasks.ceph.mgr.x.smithi188.stderr:2023-11-26T07:24:19.917+0000 7f04adb83640 -1 client.0 error registering admin socket command: (17) File exists 2023-11-26T07:24:19.920 INFO:tasks.ceph.mgr.x.smithi188.stderr:2023-11-26T07:24:19.917+0000 7f04adb83640 -1 client.0 error registering admin socket command: (17) File exists 2023-11-26T07:24:19.920 INFO:tasks.ceph.mgr.x.smithi188.stderr:2023-11-26T07:24:19.917+0000 7f04adb83640 -1 client.0 error registering admin socket command: (17) File exists 2023-11-26T07:24:20.074 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:20.075+0000 7f11fcff9640 1 -- 172.21.15.143:0/1801817267 <== mgr.4108 v2:172.21.15.188:6836/3401128553 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+67 (secure 0 0 0) 0x7f12001564a0 con 0x7f11e0066590 2023-11-26T07:24:20.075 INFO:teuthology.orchestra.run.smithi143.stdout:/volumes/_nogroup/subvol24520/f57b3d1b-7328-4835-8e29-9827b7800bab 2023-11-26T07:24:20.077 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:20.075+0000 7f1205b57640 1 -- 172.21.15.143:0/1801817267 >> [v2:172.21.15.188:6836/3401128553,v1:172.21.15.188:6837/3401128553] conn(0x7f11e0066590 msgr2=0x7f11e0068a50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:24:20.077 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:20.075+0000 7f1205b57640 1 --2- 172.21.15.143:0/1801817267 >> [v2:172.21.15.188:6836/3401128553,v1:172.21.15.188:6837/3401128553] conn(0x7f11e0066590 0x7f11e0068a50 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7f11f0007a60 tx=0x7f11f0008040 comp rx=0 tx=0).stop 2023-11-26T07:24:20.077 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:20.075+0000 7f1205b57640 1 -- 172.21.15.143:0/1801817267 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f120015f4e0 msgr2=0x7f12000aeb60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:24:20.077 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:20.075+0000 7f1205b57640 1 --2- 172.21.15.143:0/1801817267 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f120015f4e0 0x7f12000aeb60 secure :-1 s=READY pgs=1340 cs=0 l=1 rev1=1 crypto rx=0x7f11f8002240 tx=0x7f11f800cf50 comp rx=0 tx=0).stop 2023-11-26T07:24:20.077 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:20.075+0000 7f1205b57640 1 -- 172.21.15.143:0/1801817267 shutdown_connections 2023-11-26T07:24:20.077 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:20.075+0000 7f1205b57640 1 --2- 172.21.15.143:0/1801817267 >> [v2:172.21.15.188:6836/3401128553,v1:172.21.15.188:6837/3401128553] conn(0x7f11e0066590 0x7f11e0068a50 unknown :-1 s=CLOSED pgs=139 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:24:20.077 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:20.075+0000 7f1205b57640 1 --2- 172.21.15.143:0/1801817267 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f120015f4e0 0x7f12000aeb60 unknown :-1 s=CLOSED pgs=1340 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:24:20.077 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:20.075+0000 7f1205b57640 1 --2- 172.21.15.143:0/1801817267 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f120013e290 0x7f1200155d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:24:20.078 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:20.075+0000 7f1205b57640 1 --2- 172.21.15.143:0/1801817267 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f12000cf7c0 0x7f1200154d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:24:20.078 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:20.075+0000 7f1205b57640 1 -- 172.21.15.143:0/1801817267 >> 172.21.15.143:0/1801817267 conn(0x7f120006a890 msgr2=0x7f1200138e90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:24:20.078 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:20.075+0000 7f1205b57640 1 -- 172.21.15.143:0/1801817267 shutdown_connections 2023-11-26T07:24:20.078 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:20.075+0000 7f1205b57640 1 -- 172.21.15.143:0/1801817267 wait complete. 2023-11-26T07:24:20.096 DEBUG:teuthology.orchestra.run.smithi143:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume ls cephfs 2023-11-26T07:24:20.235 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:20.235+0000 7f9a8dfdf640 1 Processor -- start 2023-11-26T07:24:20.235 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:20.235+0000 7f9a8dfdf640 1 -- start start 2023-11-26T07:24:20.236 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:20.235+0000 7f9a8dfdf640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f9a8813e290 0x7f9a880cf280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:24:20.237 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:20.235+0000 7f9a8dfdf640 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7f9a880cf7c0 0x7f9a8815efa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:24:20.237 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:20.235+0000 7f9a8dfdf640 1 -- --> v1:172.21.15.143:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9a8806c520 con 0x7f9a8815f4e0 2023-11-26T07:24:20.237 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:20.235+0000 7f9a8dfdf640 1 -- --> v2:172.21.15.143:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9a880cfc40 con 0x7f9a880cf7c0 2023-11-26T07:24:20.237 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:20.235+0000 7f9a8dfdf640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9a8806c800 con 0x7f9a8813e290 2023-11-26T07:24:20.237 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:20.235+0000 7f9a7ffff640 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7f9a880cf7c0 0x7f9a8815efa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:24:20.237 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:20.235+0000 7f9a87fff640 1 --1- >> v1:172.21.15.143:6789/0 conn(0x7f9a8815f4e0 0x7f9a881408e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.143:6789/0 says I am v1:172.21.15.143:40246/0 (socket says 172.21.15.143:40246) 2023-11-26T07:24:20.237 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:20.235+0000 7f9a7ffff640 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7f9a880cf7c0 0x7f9a8815efa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:54628/0 (socket says 172.21.15.143:54628) 2023-11-26T07:24:20.237 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:20.235+0000 7f9a87fff640 1 -- 172.21.15.143:0/459730989 learned_addr learned my addr 172.21.15.143:0/459730989 (peer_addr_for_me v1:172.21.15.143:0/0) 2023-11-26T07:24:20.237 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:20.235+0000 7f9a877fe640 1 --2- 172.21.15.143:0/459730989 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f9a8813e290 0x7f9a880cf280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:24:20.238 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:20.235+0000 7f9a7ffff640 1 -- 172.21.15.143:0/459730989 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f9a8813e290 msgr2=0x7f9a880cf280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-26T07:24:20.238 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:20.235+0000 7f9a7ffff640 1 --2- 172.21.15.143:0/459730989 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f9a8813e290 0x7f9a880cf280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:24:20.238 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:20.235+0000 7f9a7ffff640 1 -- 172.21.15.143:0/459730989 >> v1:172.21.15.143:6789/0 conn(0x7f9a8815f4e0 legacy=0x7f9a881408e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:24:20.238 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:20.235+0000 7f9a7ffff640 1 -- 172.21.15.143:0/459730989 --> v2:172.21.15.143:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9a8806cae0 con 0x7f9a880cf7c0 2023-11-26T07:24:20.238 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:20.235+0000 7f9a86ffd640 1 -- 172.21.15.143:0/459730989 <== mon.0 v1:172.21.15.143:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 384039000 0 0) 0x7f9a8806c520 con 0x7f9a8815f4e0 2023-11-26T07:24:20.240 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:20.239+0000 7f9a7ffff640 1 --2- 172.21.15.143:0/459730989 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f9a880cf7c0 0x7f9a8815efa0 secure :-1 s=READY pgs=1342 cs=0 l=1 rev1=1 crypto rx=0x7f9a700097c0 tx=0x7f9a7001c450 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8fe01108bf7ea514 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:24:20.240 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:20.239+0000 7f9a86ffd640 1 -- 172.21.15.143:0/459730989 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9a700ab650 con 0x7f9a880cf7c0 2023-11-26T07:24:20.240 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:20.239+0000 7f9a86ffd640 1 -- 172.21.15.143:0/459730989 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9a700ab7b0 con 0x7f9a880cf7c0 2023-11-26T07:24:20.240 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:20.239+0000 7f9a86ffd640 1 -- 172.21.15.143:0/459730989 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9a700abb40 con 0x7f9a880cf7c0 2023-11-26T07:24:20.241 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:20.239+0000 7f9a8dfdf640 1 -- 172.21.15.143:0/459730989 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f9a880cf7c0 msgr2=0x7f9a8815efa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:24:20.241 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:20.239+0000 7f9a8dfdf640 1 --2- 172.21.15.143:0/459730989 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f9a880cf7c0 0x7f9a8815efa0 secure :-1 s=READY pgs=1342 cs=0 l=1 rev1=1 crypto rx=0x7f9a700097c0 tx=0x7f9a7001c450 comp rx=0 tx=0).stop 2023-11-26T07:24:20.241 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:20.239+0000 7f9a8dfdf640 1 -- 172.21.15.143:0/459730989 shutdown_connections 2023-11-26T07:24:20.241 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:20.239+0000 7f9a8dfdf640 1 --2- 172.21.15.143:0/459730989 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f9a880cf7c0 0x7f9a8815efa0 unknown :-1 s=CLOSED pgs=1342 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:24:20.242 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:20.239+0000 7f9a8dfdf640 1 --2- 172.21.15.143:0/459730989 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f9a8813e290 0x7f9a880cf280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:24:20.242 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:20.239+0000 7f9a8dfdf640 1 -- 172.21.15.143:0/459730989 >> 172.21.15.143:0/459730989 conn(0x7f9a8806a890 msgr2=0x7f9a8806aca0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:24:20.242 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:20.239+0000 7f9a8dfdf640 1 -- 172.21.15.143:0/459730989 shutdown_connections 2023-11-26T07:24:20.242 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:20.239+0000 7f9a8dfdf640 1 -- 172.21.15.143:0/459730989 wait complete. 2023-11-26T07:24:20.242 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:20.243+0000 7f9a8dfdf640 1 Processor -- start 2023-11-26T07:24:20.242 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:20.243+0000 7f9a8dfdf640 1 -- start start 2023-11-26T07:24:20.245 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:20.243+0000 7f9a8dfdf640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f9a880cf7c0 0x7f9a88154b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:24:20.245 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:20.243+0000 7f9a8dfdf640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f9a8813e290 0x7f9a88136e90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:24:20.245 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:20.243+0000 7f9a8dfdf640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f9a8815f4e0 0x7f9a881373d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:24:20.245 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:20.243+0000 7f9a8dfdf640 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9a88137910 con 0x7f9a8815f4e0 2023-11-26T07:24:20.245 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:20.243+0000 7f9a8dfdf640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9a8813fad0 con 0x7f9a880cf7c0 2023-11-26T07:24:20.245 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:20.243+0000 7f9a8dfdf640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9a8813fdb0 con 0x7f9a8813e290 2023-11-26T07:24:20.245 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:20.243+0000 7f9a87fff640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f9a8815f4e0 0x7f9a881373d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:24:20.245 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:20.243+0000 7f9a87fff640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f9a8815f4e0 0x7f9a881373d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:54634/0 (socket says 172.21.15.143:54634) 2023-11-26T07:24:20.245 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:20.243+0000 7f9a87fff640 1 -- 172.21.15.143:0/1925777187 learned_addr learned my addr 172.21.15.143:0/1925777187 (peer_addr_for_me v2:172.21.15.143:0/0) 2023-11-26T07:24:20.246 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:20.243+0000 7f9a7ffff640 1 --2- 172.21.15.143:0/1925777187 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f9a8813e290 0x7f9a88136e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:24:20.246 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:20.243+0000 7f9a877fe640 1 --2- 172.21.15.143:0/1925777187 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f9a880cf7c0 0x7f9a88154b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:24:20.246 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:20.243+0000 7f9a87fff640 1 -- 172.21.15.143:0/1925777187 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f9a8813e290 msgr2=0x7f9a88136e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:24:20.246 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:20.243+0000 7f9a87fff640 1 --2- 172.21.15.143:0/1925777187 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f9a8813e290 0x7f9a88136e90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:24:20.246 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:20.243+0000 7f9a87fff640 1 -- 172.21.15.143:0/1925777187 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f9a880cf7c0 msgr2=0x7f9a88154b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:24:20.246 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:20.243+0000 7f9a87fff640 1 --2- 172.21.15.143:0/1925777187 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f9a880cf7c0 0x7f9a88154b20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:24:20.246 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:20.243+0000 7f9a87fff640 1 -- 172.21.15.143:0/1925777187 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9a70009420 con 0x7f9a8815f4e0 2023-11-26T07:24:20.246 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:20.243+0000 7f9a7ffff640 1 --2- 172.21.15.143:0/1925777187 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f9a8813e290 0x7f9a88136e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-26T07:24:20.246 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:20.243+0000 7f9a87fff640 1 --2- 172.21.15.143:0/1925777187 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f9a8815f4e0 0x7f9a881373d0 secure :-1 s=READY pgs=1343 cs=0 l=1 rev1=1 crypto rx=0x7f9a8000c700 tx=0x7f9a8000cbc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:24:20.247 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:20.243+0000 7f9a84ff9640 1 -- 172.21.15.143:0/1925777187 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9a8000ce10 con 0x7f9a8815f4e0 2023-11-26T07:24:20.247 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:20.243+0000 7f9a84ff9640 1 -- 172.21.15.143:0/1925777187 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9a8000a640 con 0x7f9a8815f4e0 2023-11-26T07:24:20.247 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:20.243+0000 7f9a8dfdf640 1 -- 172.21.15.143:0/1925777187 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9a88140090 con 0x7f9a8815f4e0 2023-11-26T07:24:20.247 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:20.243+0000 7f9a84ff9640 1 -- 172.21.15.143:0/1925777187 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9a8000aa70 con 0x7f9a8815f4e0 2023-11-26T07:24:20.247 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:20.243+0000 7f9a8dfdf640 1 -- 172.21.15.143:0/1925777187 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9a8813f5d0 con 0x7f9a8815f4e0 2023-11-26T07:24:20.248 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:20.247+0000 7f9a8dfdf640 1 -- 172.21.15.143:0/1925777187 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9a881405b0 con 0x7f9a8815f4e0 2023-11-26T07:24:20.248 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:20.247+0000 7f9a84ff9640 1 -- 172.21.15.143:0/1925777187 <== mon.0 v2:172.21.15.143:3300/0 4 ==== mgrmap(e 16) v1 ==== 85954+0+0 (secure 0 0 0) 0x7f9a8000abd0 con 0x7f9a8815f4e0 2023-11-26T07:24:20.248 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:20.247+0000 7f9a84ff9640 1 --2- 172.21.15.143:0/1925777187 >> [v2:172.21.15.188:6836/3401128553,v1:172.21.15.188:6837/3401128553] conn(0x7f9a68066590 0x7f9a68068a50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:24:20.248 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:20.247+0000 7f9a84ff9640 1 -- 172.21.15.143:0/1925777187 <== mon.0 v2:172.21.15.143:3300/0 5 ==== osd_map(103..103 src has 1..103) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f9a80083730 con 0x7f9a8815f4e0 2023-11-26T07:24:20.248 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:20.247+0000 7f9a877fe640 1 --2- 172.21.15.143:0/1925777187 >> [v2:172.21.15.188:6836/3401128553,v1:172.21.15.188:6837/3401128553] conn(0x7f9a68066590 0x7f9a68068a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:24:20.249 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:20.247+0000 7f9a877fe640 1 --2- 172.21.15.143:0/1925777187 >> [v2:172.21.15.188:6836/3401128553,v1:172.21.15.188:6837/3401128553] conn(0x7f9a68066590 0x7f9a68068a50 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7f9a78009af0 tx=0x7f9a78008040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:24:20.252 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:20.251+0000 7f9a84ff9640 1 -- 172.21.15.143:0/1925777187 <== mon.0 v2:172.21.15.143:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172649 (secure 0 0 0) 0x7f9a881405b0 con 0x7f9a8815f4e0 2023-11-26T07:24:20.422 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:20.423+0000 7f9a8dfdf640 1 -- 172.21.15.143:0/1925777187 --> [v2:172.21.15.188:6836/3401128553,v1:172.21.15.188:6837/3401128553] -- mgr_command(tid 0: {"prefix": "fs subvolume ls", "vol_name": "cephfs", "target": ["mon-mgr", ""]}) v1 -- 0x7f9a8815fb40 con 0x7f9a68066590 2023-11-26T07:24:20.427 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:20.427+0000 7f9a84ff9640 1 -- 172.21.15.143:0/1925777187 <== mgr.4108 v2:172.21.15.188:6836/3401128553 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+46 (secure 0 0 0) 0x7f9a78009820 con 0x7f9a68066590 2023-11-26T07:24:20.427 INFO:teuthology.orchestra.run.smithi143.stdout:[ 2023-11-26T07:24:20.427 INFO:teuthology.orchestra.run.smithi143.stdout: { 2023-11-26T07:24:20.427 INFO:teuthology.orchestra.run.smithi143.stdout: "name": "subvol24520" 2023-11-26T07:24:20.427 INFO:teuthology.orchestra.run.smithi143.stdout: } 2023-11-26T07:24:20.427 INFO:teuthology.orchestra.run.smithi143.stdout:] 2023-11-26T07:24:20.429 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:20.427+0000 7f9a8dfdf640 1 -- 172.21.15.143:0/1925777187 >> [v2:172.21.15.188:6836/3401128553,v1:172.21.15.188:6837/3401128553] conn(0x7f9a68066590 msgr2=0x7f9a68068a50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:24:20.429 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:20.427+0000 7f9a8dfdf640 1 --2- 172.21.15.143:0/1925777187 >> [v2:172.21.15.188:6836/3401128553,v1:172.21.15.188:6837/3401128553] conn(0x7f9a68066590 0x7f9a68068a50 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7f9a78009af0 tx=0x7f9a78008040 comp rx=0 tx=0).stop 2023-11-26T07:24:20.429 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:20.427+0000 7f9a8dfdf640 1 -- 172.21.15.143:0/1925777187 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f9a8815f4e0 msgr2=0x7f9a881373d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:24:20.429 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:20.427+0000 7f9a8dfdf640 1 --2- 172.21.15.143:0/1925777187 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f9a8815f4e0 0x7f9a881373d0 secure :-1 s=READY pgs=1343 cs=0 l=1 rev1=1 crypto rx=0x7f9a8000c700 tx=0x7f9a8000cbc0 comp rx=0 tx=0).stop 2023-11-26T07:24:20.429 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:20.427+0000 7f9a8dfdf640 1 -- 172.21.15.143:0/1925777187 shutdown_connections 2023-11-26T07:24:20.429 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:20.427+0000 7f9a8dfdf640 1 --2- 172.21.15.143:0/1925777187 >> [v2:172.21.15.188:6836/3401128553,v1:172.21.15.188:6837/3401128553] conn(0x7f9a68066590 0x7f9a68068a50 unknown :-1 s=CLOSED pgs=140 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:24:20.430 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:20.427+0000 7f9a8dfdf640 1 --2- 172.21.15.143:0/1925777187 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f9a8815f4e0 0x7f9a881373d0 unknown :-1 s=CLOSED pgs=1343 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:24:20.430 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:20.427+0000 7f9a8dfdf640 1 --2- 172.21.15.143:0/1925777187 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f9a8813e290 0x7f9a88136e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:24:20.430 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:20.427+0000 7f9a8dfdf640 1 --2- 172.21.15.143:0/1925777187 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f9a880cf7c0 0x7f9a88154b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:24:20.430 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:20.427+0000 7f9a8dfdf640 1 -- 172.21.15.143:0/1925777187 >> 172.21.15.143:0/1925777187 conn(0x7f9a8806a890 msgr2=0x7f9a880b39f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:24:20.430 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:20.427+0000 7f9a8dfdf640 1 -- 172.21.15.143:0/1925777187 shutdown_connections 2023-11-26T07:24:20.430 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:20.427+0000 7f9a8dfdf640 1 -- 172.21.15.143:0/1925777187 wait complete. 2023-11-26T07:24:20.448 DEBUG:teuthology.orchestra.run.smithi143:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume info cephfs subvol24520 2023-11-26T07:24:20.591 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:20.591+0000 7f96a3a46640 1 Processor -- start 2023-11-26T07:24:20.591 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:20.591+0000 7f96a3a46640 1 -- start start 2023-11-26T07:24:20.592 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:20.591+0000 7f96a3a46640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f969c13e290 0x7f969c0cf280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:24:20.593 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:20.591+0000 7f96a3a46640 1 -- --> v1:172.21.15.143:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f969c06c520 con 0x7f969c0cf7c0 2023-11-26T07:24:20.593 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:20.591+0000 7f96a3a46640 1 -- --> v1:172.21.15.188:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f969c06c800 con 0x7f969c15f6b0 2023-11-26T07:24:20.593 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:20.591+0000 7f96a3a46640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f969c0cfc40 con 0x7f969c13e290 2023-11-26T07:24:20.593 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:20.591+0000 7f96a0fe1640 1 --1- >> v1:172.21.15.143:6789/0 conn(0x7f969c0cf7c0 0x7f969c15efa0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.143:6789/0 says I am v1:172.21.15.143:40252/0 (socket says 172.21.15.143:40252) 2023-11-26T07:24:20.593 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:20.591+0000 7f96a0fe1640 1 -- 172.21.15.143:0/2311941498 learned_addr learned my addr 172.21.15.143:0/2311941498 (peer_addr_for_me v1:172.21.15.143:0/0) 2023-11-26T07:24:20.594 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:20.591+0000 7f96a17e2640 1 --2- 172.21.15.143:0/2311941498 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f969c13e290 0x7f969c0cf280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:24:20.594 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:20.591+0000 7f9693fff640 1 -- 172.21.15.143:0/2311941498 <== mon.0 v1:172.21.15.143:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 379997923 0 0) 0x7f969c06c520 con 0x7f969c0cf7c0 2023-11-26T07:24:20.594 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:20.591+0000 7f9693fff640 1 -- 172.21.15.143:0/2311941498 --> v1:172.21.15.143:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f9688003530 con 0x7f969c0cf7c0 2023-11-26T07:24:20.594 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:20.595+0000 7f96a17e2640 1 -- 172.21.15.143:0/2311941498 >> v1:172.21.15.188:6789/0 conn(0x7f969c15f6b0 legacy=0x7f969c0cd7b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:24:20.594 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:20.595+0000 7f9693fff640 1 -- 172.21.15.143:0/2311941498 <== mon.1 v1:172.21.15.188:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 396598208 0 0) 0x7f969c06c800 con 0x7f969c15f6b0 2023-11-26T07:24:20.594 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:20.595+0000 7f96a17e2640 1 -- 172.21.15.143:0/2311941498 >> v1:172.21.15.143:6789/0 conn(0x7f969c0cf7c0 legacy=0x7f969c15efa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:24:20.594 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:20.595+0000 7f96a17e2640 1 -- 172.21.15.143:0/2311941498 --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f969c06cae0 con 0x7f969c13e290 2023-11-26T07:24:20.596 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:20.595+0000 7f96a17e2640 1 --2- 172.21.15.143:0/2311941498 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f969c13e290 0x7f969c0cf280 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7f96980097f0 tx=0x7f969801c4c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=53524afab22937f9 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:24:20.597 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:20.595+0000 7f9693fff640 1 -- 172.21.15.143:0/2311941498 <== mon.2 v2:172.21.15.188:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f96980ac410 con 0x7f969c13e290 2023-11-26T07:24:20.597 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:20.595+0000 7f9693fff640 1 -- 172.21.15.143:0/2311941498 <== mon.2 v2:172.21.15.188:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f96980ac570 con 0x7f969c13e290 2023-11-26T07:24:20.597 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:20.595+0000 7f9693fff640 1 -- 172.21.15.143:0/2311941498 <== mon.2 v2:172.21.15.188:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f96980ac900 con 0x7f969c13e290 2023-11-26T07:24:20.597 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:20.595+0000 7f96a3a46640 1 -- 172.21.15.143:0/2311941498 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f969c13e290 msgr2=0x7f969c0cf280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:24:20.598 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:20.595+0000 7f96a3a46640 1 --2- 172.21.15.143:0/2311941498 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f969c13e290 0x7f969c0cf280 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7f96980097f0 tx=0x7f969801c4c0 comp rx=0 tx=0).stop 2023-11-26T07:24:20.598 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:20.595+0000 7f96a3a46640 1 -- 172.21.15.143:0/2311941498 shutdown_connections 2023-11-26T07:24:20.598 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:20.595+0000 7f96a3a46640 1 --2- 172.21.15.143:0/2311941498 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f969c13e290 0x7f969c0cf280 unknown :-1 s=CLOSED pgs=160 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:24:20.598 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:20.595+0000 7f96a3a46640 1 -- 172.21.15.143:0/2311941498 >> 172.21.15.143:0/2311941498 conn(0x7f969c06a890 msgr2=0x7f969c06aca0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:24:20.598 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:20.599+0000 7f96a3a46640 1 -- 172.21.15.143:0/2311941498 shutdown_connections 2023-11-26T07:24:20.598 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:20.599+0000 7f96a3a46640 1 -- 172.21.15.143:0/2311941498 wait complete. 2023-11-26T07:24:20.599 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:20.599+0000 7f96a3a46640 1 Processor -- start 2023-11-26T07:24:20.599 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:20.599+0000 7f96a3a46640 1 -- start start 2023-11-26T07:24:20.600 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:20.599+0000 7f96a3a46640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f969c0cf7c0 0x7f969c154b10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:24:20.600 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:20.599+0000 7f96a3a46640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f969c13e290 0x7f969c15dbf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:24:20.600 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:20.599+0000 7f96a3a46640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f969c15f6b0 0x7f969c15e130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:24:20.600 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:20.599+0000 7f96a3a46640 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f969c15bf20 con 0x7f969c15f6b0 2023-11-26T07:24:20.600 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:20.599+0000 7f96a3a46640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f969c15c200 con 0x7f969c0cf7c0 2023-11-26T07:24:20.600 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:20.599+0000 7f96a3a46640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f969c15c4e0 con 0x7f969c13e290 2023-11-26T07:24:20.602 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:20.599+0000 7f96a1fe3640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f969c15f6b0 0x7f969c15e130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:24:20.602 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:20.599+0000 7f96a1fe3640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f969c15f6b0 0x7f969c15e130 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:54638/0 (socket says 172.21.15.143:54638) 2023-11-26T07:24:20.602 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:20.599+0000 7f96a1fe3640 1 -- 172.21.15.143:0/2535532285 learned_addr learned my addr 172.21.15.143:0/2535532285 (peer_addr_for_me v2:172.21.15.143:0/0) 2023-11-26T07:24:20.602 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:20.599+0000 7f96a0fe1640 1 --2- 172.21.15.143:0/2535532285 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f969c13e290 0x7f969c15dbf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:24:20.602 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:20.599+0000 7f96a17e2640 1 --2- 172.21.15.143:0/2535532285 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f969c0cf7c0 0x7f969c154b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:24:20.602 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:20.599+0000 7f96a1fe3640 1 -- 172.21.15.143:0/2535532285 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f969c13e290 msgr2=0x7f969c15dbf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:24:20.602 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:20.599+0000 7f96a1fe3640 1 --2- 172.21.15.143:0/2535532285 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f969c13e290 0x7f969c15dbf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:24:20.602 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:20.599+0000 7f96a1fe3640 1 -- 172.21.15.143:0/2535532285 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f969c0cf7c0 msgr2=0x7f969c154b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:24:20.603 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:20.599+0000 7f96a1fe3640 1 --2- 172.21.15.143:0/2535532285 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f969c0cf7c0 0x7f969c154b10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:24:20.603 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:20.599+0000 7f96a1fe3640 1 -- 172.21.15.143:0/2535532285 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9698009420 con 0x7f969c15f6b0 2023-11-26T07:24:20.603 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:20.599+0000 7f96a1fe3640 1 --2- 172.21.15.143:0/2535532285 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f969c15f6b0 0x7f969c15e130 secure :-1 s=READY pgs=1345 cs=0 l=1 rev1=1 crypto rx=0x7f9694002240 tx=0x7f969400cbe0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:24:20.603 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:20.603+0000 7f9691ffb640 1 -- 172.21.15.143:0/2535532285 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9694003db0 con 0x7f969c15f6b0 2023-11-26T07:24:20.603 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:20.603+0000 7f9691ffb640 1 -- 172.21.15.143:0/2535532285 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f969400ce30 con 0x7f969c15f6b0 2023-11-26T07:24:20.603 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:20.603+0000 7f9691ffb640 1 -- 172.21.15.143:0/2535532285 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9694005270 con 0x7f969c15f6b0 2023-11-26T07:24:20.603 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:20.603+0000 7f96a3a46640 1 -- 172.21.15.143:0/2535532285 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f969c15c7c0 con 0x7f969c15f6b0 2023-11-26T07:24:20.603 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:20.603+0000 7f96a3a46640 1 -- 172.21.15.143:0/2535532285 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f969c15cd80 con 0x7f969c15f6b0 2023-11-26T07:24:20.604 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:20.603+0000 7f96a3a46640 1 -- 172.21.15.143:0/2535532285 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f969c0b1b20 con 0x7f969c15f6b0 2023-11-26T07:24:20.604 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:20.603+0000 7f9691ffb640 1 -- 172.21.15.143:0/2535532285 <== mon.0 v2:172.21.15.143:3300/0 4 ==== mgrmap(e 17) v1 ==== 86005+0+0 (secure 0 0 0) 0x7f96940053d0 con 0x7f969c15f6b0 2023-11-26T07:24:20.605 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:20.603+0000 7f9691ffb640 1 --2- 172.21.15.143:0/2535532285 >> [v2:172.21.15.188:6836/3401128553,v1:172.21.15.188:6837/3401128553] conn(0x7f9680066620 0x7f9680068ae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:24:20.605 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:20.603+0000 7f9691ffb640 1 -- 172.21.15.143:0/2535532285 <== mon.0 v2:172.21.15.143:3300/0 5 ==== osd_map(103..103 src has 1..103) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f9694083830 con 0x7f969c15f6b0 2023-11-26T07:24:20.605 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:20.603+0000 7f96a17e2640 1 --2- 172.21.15.143:0/2535532285 >> [v2:172.21.15.188:6836/3401128553,v1:172.21.15.188:6837/3401128553] conn(0x7f9680066620 0x7f9680068ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:24:20.606 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:20.607+0000 7f96a17e2640 1 --2- 172.21.15.143:0/2535532285 >> [v2:172.21.15.188:6836/3401128553,v1:172.21.15.188:6837/3401128553] conn(0x7f9680066620 0x7f9680068ae0 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7f96980ae000 tx=0x7f9698008040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:24:20.609 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:20.607+0000 7f9691ffb640 1 -- 172.21.15.143:0/2535532285 <== mon.0 v2:172.21.15.143:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172649 (secure 0 0 0) 0x7f9694050b90 con 0x7f969c15f6b0 2023-11-26T07:24:20.782 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:20.783+0000 7f96a3a46640 1 -- 172.21.15.143:0/2535532285 --> [v2:172.21.15.188:6836/3401128553,v1:172.21.15.188:6837/3401128553] -- mgr_command(tid 0: {"prefix": "fs subvolume info", "vol_name": "cephfs", "sub_name": "subvol24520", "target": ["mon-mgr", ""]}) v1 -- 0x7f969c15c650 con 0x7f9680066620 2023-11-26T07:24:20.794 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:20.795+0000 7f9691ffb640 1 -- 172.21.15.143:0/2535532285 <== mgr.4108 v2:172.21.15.188:6836/3401128553 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+670 (secure 0 0 0) 0x7f969c15c650 con 0x7f9680066620 2023-11-26T07:24:20.794 INFO:teuthology.orchestra.run.smithi143.stdout:{ 2023-11-26T07:24:20.795 INFO:teuthology.orchestra.run.smithi143.stdout: "atime": "2023-11-26 07:24:19", 2023-11-26T07:24:20.795 INFO:teuthology.orchestra.run.smithi143.stdout: "bytes_pcent": "undefined", 2023-11-26T07:24:20.795 INFO:teuthology.orchestra.run.smithi143.stdout: "bytes_quota": "infinite", 2023-11-26T07:24:20.795 INFO:teuthology.orchestra.run.smithi143.stdout: "bytes_used": 0, 2023-11-26T07:24:20.795 INFO:teuthology.orchestra.run.smithi143.stdout: "created_at": "2023-11-26 07:24:19", 2023-11-26T07:24:20.795 INFO:teuthology.orchestra.run.smithi143.stdout: "ctime": "2023-11-26 07:24:20", 2023-11-26T07:24:20.795 INFO:teuthology.orchestra.run.smithi143.stdout: "data_pool": "cephfs_data", 2023-11-26T07:24:20.795 INFO:teuthology.orchestra.run.smithi143.stdout: "features": [ 2023-11-26T07:24:20.795 INFO:teuthology.orchestra.run.smithi143.stdout: "snapshot-clone", 2023-11-26T07:24:20.795 INFO:teuthology.orchestra.run.smithi143.stdout: "snapshot-autoprotect" 2023-11-26T07:24:20.795 INFO:teuthology.orchestra.run.smithi143.stdout: ], 2023-11-26T07:24:20.796 INFO:teuthology.orchestra.run.smithi143.stdout: "gid": 0, 2023-11-26T07:24:20.796 INFO:teuthology.orchestra.run.smithi143.stdout: "mode": 16877, 2023-11-26T07:24:20.796 INFO:teuthology.orchestra.run.smithi143.stdout: "mon_addrs": [ 2023-11-26T07:24:20.796 INFO:teuthology.orchestra.run.smithi143.stdout: "172.21.15.143:6789", 2023-11-26T07:24:20.796 INFO:teuthology.orchestra.run.smithi143.stdout: "172.21.15.188:6789", 2023-11-26T07:24:20.796 INFO:teuthology.orchestra.run.smithi143.stdout: "172.21.15.188:6790" 2023-11-26T07:24:20.796 INFO:teuthology.orchestra.run.smithi143.stdout: ], 2023-11-26T07:24:20.796 INFO:teuthology.orchestra.run.smithi143.stdout: "mtime": "2023-11-26 07:24:19", 2023-11-26T07:24:20.796 INFO:teuthology.orchestra.run.smithi143.stdout: "path": "/volumes/_nogroup/subvol24520/f57b3d1b-7328-4835-8e29-9827b7800bab", 2023-11-26T07:24:20.796 INFO:teuthology.orchestra.run.smithi143.stdout: "pool_namespace": "", 2023-11-26T07:24:20.797 INFO:teuthology.orchestra.run.smithi143.stdout: "state": "complete", 2023-11-26T07:24:20.797 INFO:teuthology.orchestra.run.smithi143.stdout: "type": "subvolume", 2023-11-26T07:24:20.797 INFO:teuthology.orchestra.run.smithi143.stdout: "uid": 0 2023-11-26T07:24:20.797 INFO:teuthology.orchestra.run.smithi143.stdout:} 2023-11-26T07:24:20.798 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:20.795+0000 7f96a3a46640 1 -- 172.21.15.143:0/2535532285 >> [v2:172.21.15.188:6836/3401128553,v1:172.21.15.188:6837/3401128553] conn(0x7f9680066620 msgr2=0x7f9680068ae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:24:20.798 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:20.795+0000 7f96a3a46640 1 --2- 172.21.15.143:0/2535532285 >> [v2:172.21.15.188:6836/3401128553,v1:172.21.15.188:6837/3401128553] conn(0x7f9680066620 0x7f9680068ae0 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7f96980ae000 tx=0x7f9698008040 comp rx=0 tx=0).stop 2023-11-26T07:24:20.798 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:20.795+0000 7f96a3a46640 1 -- 172.21.15.143:0/2535532285 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f969c15f6b0 msgr2=0x7f969c15e130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:24:20.798 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:20.795+0000 7f96a3a46640 1 --2- 172.21.15.143:0/2535532285 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f969c15f6b0 0x7f969c15e130 secure :-1 s=READY pgs=1345 cs=0 l=1 rev1=1 crypto rx=0x7f9694002240 tx=0x7f969400cbe0 comp rx=0 tx=0).stop 2023-11-26T07:24:20.798 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:20.795+0000 7f96a3a46640 1 -- 172.21.15.143:0/2535532285 shutdown_connections 2023-11-26T07:24:20.799 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:20.795+0000 7f96a3a46640 1 --2- 172.21.15.143:0/2535532285 >> [v2:172.21.15.188:6836/3401128553,v1:172.21.15.188:6837/3401128553] conn(0x7f9680066620 0x7f9680068ae0 unknown :-1 s=CLOSED pgs=141 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:24:20.799 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:20.795+0000 7f96a3a46640 1 --2- 172.21.15.143:0/2535532285 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f969c15f6b0 0x7f969c15e130 unknown :-1 s=CLOSED pgs=1345 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:24:20.799 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:20.795+0000 7f96a3a46640 1 --2- 172.21.15.143:0/2535532285 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f969c13e290 0x7f969c15dbf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:24:20.799 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:20.795+0000 7f96a3a46640 1 --2- 172.21.15.143:0/2535532285 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f969c0cf7c0 0x7f969c154b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:24:20.799 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:20.795+0000 7f96a3a46640 1 -- 172.21.15.143:0/2535532285 >> 172.21.15.143:0/2535532285 conn(0x7f969c06a890 msgr2=0x7f969c071d40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:24:20.799 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:20.795+0000 7f96a3a46640 1 -- 172.21.15.143:0/2535532285 shutdown_connections 2023-11-26T07:24:20.799 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:20.795+0000 7f96a3a46640 1 -- 172.21.15.143:0/2535532285 wait complete. 2023-11-26T07:24:20.816 DEBUG:teuthology.orchestra.run.smithi143:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume resize cephfs subvol24520 10485760 2023-11-26T07:24:20.960 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:20.959+0000 7f50ed7df640 1 Processor -- start 2023-11-26T07:24:20.960 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:20.959+0000 7f50ed7df640 1 -- start start 2023-11-26T07:24:20.962 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:20.963+0000 7f50ed7df640 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7f50e80cf280 0x7f50e80cf710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:24:20.962 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:20.963+0000 7f50ed7df640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f50e80cd1b0 0x7f50e80cd5c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:24:20.962 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:20.963+0000 7f50ed7df640 1 -- --> v1:172.21.15.143:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f50e80cc730 con 0x7f50e8143ac0 2023-11-26T07:24:20.962 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:20.963+0000 7f50ed7df640 1 -- --> v2:172.21.15.188:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f50e8143ea0 con 0x7f50e80cf280 2023-11-26T07:24:20.962 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:20.963+0000 7f50ed7df640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f50e80cfc50 con 0x7f50e80cd1b0 2023-11-26T07:24:20.962 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:20.963+0000 7f50e6ffd640 1 --1- >> v1:172.21.15.143:6789/0 conn(0x7f50e8143ac0 0x7f50e8070370 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.143:6789/0 says I am v1:172.21.15.143:40258/0 (socket says 172.21.15.143:40258) 2023-11-26T07:24:20.962 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:20.963+0000 7f50e6ffd640 1 -- 172.21.15.143:0/782545841 learned_addr learned my addr 172.21.15.143:0/782545841 (peer_addr_for_me v1:172.21.15.143:0/0) 2023-11-26T07:24:20.962 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:20.963+0000 7f50de7fc640 1 --2- 172.21.15.143:0/782545841 >> v2:172.21.15.188:3300/0 conn(0x7f50e80cf280 0x7f50e80cf710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:24:20.962 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:20.963+0000 7f50e77fe640 1 --2- 172.21.15.143:0/782545841 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f50e80cd1b0 0x7f50e80cd5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:24:20.963 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:20.963+0000 7f50e67fc640 1 -- 172.21.15.143:0/782545841 <== mon.0 v1:172.21.15.143:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1962597682 0 0) 0x7f50e80cc730 con 0x7f50e8143ac0 2023-11-26T07:24:20.963 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:20.963+0000 7f50e67fc640 1 -- 172.21.15.143:0/782545841 --> v1:172.21.15.143:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f50d4003530 con 0x7f50e8143ac0 2023-11-26T07:24:20.963 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:20.963+0000 7f50de7fc640 1 -- 172.21.15.143:0/782545841 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f50e80cd1b0 msgr2=0x7f50e80cd5c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-26T07:24:20.963 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:20.963+0000 7f50de7fc640 1 --2- 172.21.15.143:0/782545841 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f50e80cd1b0 0x7f50e80cd5c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:24:20.963 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:20.963+0000 7f50de7fc640 1 -- 172.21.15.143:0/782545841 >> v1:172.21.15.143:6789/0 conn(0x7f50e8143ac0 legacy=0x7f50e8070370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:24:20.964 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:20.963+0000 7f50de7fc640 1 -- 172.21.15.143:0/782545841 --> v2:172.21.15.188:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f50e8070ee0 con 0x7f50e80cf280 2023-11-26T07:24:20.965 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:20.967+0000 7f50de7fc640 1 --2- 172.21.15.143:0/782545841 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f50e80cf280 0x7f50e80cf710 secure :-1 s=READY pgs=506 cs=0 l=1 rev1=1 crypto rx=0x7f50d80097c0 tx=0x7f50d801c480 comp rx=0 tx=0).ready entity=mon.1 client_cookie=b49ab459df04f676 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:24:20.966 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:20.967+0000 7f50e67fc640 1 -- 172.21.15.143:0/782545841 <== mon.1 v2:172.21.15.188:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f50d80ab680 con 0x7f50e80cf280 2023-11-26T07:24:20.966 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:20.967+0000 7f50e67fc640 1 -- 172.21.15.143:0/782545841 <== mon.1 v2:172.21.15.188:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f50d80ab7e0 con 0x7f50e80cf280 2023-11-26T07:24:20.966 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:20.967+0000 7f50e67fc640 1 -- 172.21.15.143:0/782545841 <== mon.1 v2:172.21.15.188:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f50d80abb50 con 0x7f50e80cf280 2023-11-26T07:24:20.967 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:20.967+0000 7f50ed7df640 1 -- 172.21.15.143:0/782545841 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f50e80cf280 msgr2=0x7f50e80cf710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:24:20.967 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:20.967+0000 7f50ed7df640 1 --2- 172.21.15.143:0/782545841 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f50e80cf280 0x7f50e80cf710 secure :-1 s=READY pgs=506 cs=0 l=1 rev1=1 crypto rx=0x7f50d80097c0 tx=0x7f50d801c480 comp rx=0 tx=0).stop 2023-11-26T07:24:20.967 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:20.967+0000 7f50ed7df640 1 -- 172.21.15.143:0/782545841 shutdown_connections 2023-11-26T07:24:20.967 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:20.967+0000 7f50ed7df640 1 --2- 172.21.15.143:0/782545841 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f50e80cd1b0 0x7f50e80cd5c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:24:20.968 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:20.967+0000 7f50ed7df640 1 --2- 172.21.15.143:0/782545841 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f50e80cf280 0x7f50e80cf710 unknown :-1 s=CLOSED pgs=506 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:24:20.968 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:20.967+0000 7f50ed7df640 1 -- 172.21.15.143:0/782545841 >> 172.21.15.143:0/782545841 conn(0x7f50e806a890 msgr2=0x7f50e806aca0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:24:20.968 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:20.967+0000 7f50ed7df640 1 -- 172.21.15.143:0/782545841 shutdown_connections 2023-11-26T07:24:20.968 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:20.967+0000 7f50ed7df640 1 -- 172.21.15.143:0/782545841 wait complete. 2023-11-26T07:24:20.968 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:20.967+0000 7f50ed7df640 1 Processor -- start 2023-11-26T07:24:20.969 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:20.967+0000 7f50ed7df640 1 -- start start 2023-11-26T07:24:20.971 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:20.967+0000 7f50ed7df640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f50e80cd1b0 0x7f50e80b0f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:24:20.971 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:20.967+0000 7f50ed7df640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f50e80cf280 0x7f50e80acf40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:24:20.971 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:20.967+0000 7f50ed7df640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f50e8143ac0 0x7f50e8139d00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:24:20.971 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:20.967+0000 7f50ed7df640 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f50e80ad480 con 0x7f50e8143ac0 2023-11-26T07:24:20.971 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:20.967+0000 7f50ed7df640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f50e815a590 con 0x7f50e80cd1b0 2023-11-26T07:24:20.971 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:20.967+0000 7f50ed7df640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f50e80b1590 con 0x7f50e80cf280 2023-11-26T07:24:20.971 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:20.971+0000 7f50e77fe640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f50e8143ac0 0x7f50e8139d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:24:20.972 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:20.971+0000 7f50e77fe640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f50e8143ac0 0x7f50e8139d00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:54650/0 (socket says 172.21.15.143:54650) 2023-11-26T07:24:20.972 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:20.971+0000 7f50e77fe640 1 -- 172.21.15.143:0/723027127 learned_addr learned my addr 172.21.15.143:0/723027127 (peer_addr_for_me v2:172.21.15.143:0/0) 2023-11-26T07:24:20.972 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:20.971+0000 7f50e6ffd640 1 --2- 172.21.15.143:0/723027127 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f50e80cd1b0 0x7f50e80b0f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:24:20.972 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:20.971+0000 7f50de7fc640 1 --2- 172.21.15.143:0/723027127 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f50e80cf280 0x7f50e80acf40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:24:20.972 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:20.971+0000 7f50e77fe640 1 -- 172.21.15.143:0/723027127 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f50e80cf280 msgr2=0x7f50e80acf40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:24:20.972 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:20.971+0000 7f50e77fe640 1 --2- 172.21.15.143:0/723027127 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f50e80cf280 0x7f50e80acf40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:24:20.972 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:20.971+0000 7f50e77fe640 1 -- 172.21.15.143:0/723027127 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f50e80cd1b0 msgr2=0x7f50e80b0f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:24:20.972 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:20.971+0000 7f50e77fe640 1 --2- 172.21.15.143:0/723027127 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f50e80cd1b0 0x7f50e80b0f00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:24:20.972 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:20.971+0000 7f50e77fe640 1 -- 172.21.15.143:0/723027127 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f50d8009420 con 0x7f50e8143ac0 2023-11-26T07:24:20.972 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:20.971+0000 7f50e77fe640 1 --2- 172.21.15.143:0/723027127 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f50e8143ac0 0x7f50e8139d00 secure :-1 s=READY pgs=1347 cs=0 l=1 rev1=1 crypto rx=0x7f50e000efc0 tx=0x7f50e000c490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:24:20.973 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:20.971+0000 7f50dffff640 1 -- 172.21.15.143:0/723027127 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f50e0009ba0 con 0x7f50e8143ac0 2023-11-26T07:24:20.973 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:20.971+0000 7f50dffff640 1 -- 172.21.15.143:0/723027127 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f50e000f040 con 0x7f50e8143ac0 2023-11-26T07:24:20.973 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:20.971+0000 7f50ed7df640 1 -- 172.21.15.143:0/723027127 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f50e80b1870 con 0x7f50e8143ac0 2023-11-26T07:24:20.973 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:20.971+0000 7f50dffff640 1 -- 172.21.15.143:0/723027127 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f50e00146c0 con 0x7f50e8143ac0 2023-11-26T07:24:20.973 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:20.971+0000 7f50ed7df640 1 -- 172.21.15.143:0/723027127 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f50e815aac0 con 0x7f50e8143ac0 2023-11-26T07:24:20.974 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:20.971+0000 7f50ed7df640 1 -- 172.21.15.143:0/723027127 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f50e806df80 con 0x7f50e8143ac0 2023-11-26T07:24:20.974 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:20.971+0000 7f50dffff640 1 -- 172.21.15.143:0/723027127 <== mon.0 v2:172.21.15.143:3300/0 4 ==== mgrmap(e 17) v1 ==== 86005+0+0 (secure 0 0 0) 0x7f50e0014820 con 0x7f50e8143ac0 2023-11-26T07:24:20.974 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:20.975+0000 7f50dffff640 1 --2- 172.21.15.143:0/723027127 >> [v2:172.21.15.188:6836/3401128553,v1:172.21.15.188:6837/3401128553] conn(0x7f50c8066650 0x7f50c8068b10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:24:20.974 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:20.975+0000 7f50dffff640 1 -- 172.21.15.143:0/723027127 <== mon.0 v2:172.21.15.143:3300/0 5 ==== osd_map(103..103 src has 1..103) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f50e0084b70 con 0x7f50e8143ac0 2023-11-26T07:24:20.974 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:20.975+0000 7f50e6ffd640 1 --2- 172.21.15.143:0/723027127 >> [v2:172.21.15.188:6836/3401128553,v1:172.21.15.188:6837/3401128553] conn(0x7f50c8066650 0x7f50c8068b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:24:20.975 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:20.975+0000 7f50e6ffd640 1 --2- 172.21.15.143:0/723027127 >> [v2:172.21.15.188:6836/3401128553,v1:172.21.15.188:6837/3401128553] conn(0x7f50c8066650 0x7f50c8068b10 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7f50d0007fd0 tx=0x7f50d0000bd0 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:24:20.977 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:20.975+0000 7f50dffff640 1 -- 172.21.15.143:0/723027127 <== mon.0 v2:172.21.15.143:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172649 (secure 0 0 0) 0x7f50e0051d80 con 0x7f50e8143ac0 2023-11-26T07:24:21.155 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:21.155+0000 7f50ed7df640 1 -- 172.21.15.143:0/723027127 --> [v2:172.21.15.188:6836/3401128553,v1:172.21.15.188:6837/3401128553] -- mgr_command(tid 0: {"prefix": "fs subvolume resize", "vol_name": "cephfs", "sub_name": "subvol24520", "new_size": "10485760", "target": ["mon-mgr", ""]}) v1 -- 0x7f50e8071c30 con 0x7f50c8066650 2023-11-26T07:24:21.171 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:21.171+0000 7f50dffff640 1 -- 172.21.15.143:0/723027127 <== mgr.4108 v2:172.21.15.188:6836/3401128553 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+128 (secure 0 0 0) 0x7f50d0001570 con 0x7f50c8066650 2023-11-26T07:24:21.171 INFO:teuthology.orchestra.run.smithi143.stdout:[ 2023-11-26T07:24:21.171 INFO:teuthology.orchestra.run.smithi143.stdout: { 2023-11-26T07:24:21.171 INFO:teuthology.orchestra.run.smithi143.stdout: "bytes_used": 0 2023-11-26T07:24:21.171 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2023-11-26T07:24:21.171 INFO:teuthology.orchestra.run.smithi143.stdout: { 2023-11-26T07:24:21.171 INFO:teuthology.orchestra.run.smithi143.stdout: "bytes_quota": 10485760 2023-11-26T07:24:21.171 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2023-11-26T07:24:21.171 INFO:teuthology.orchestra.run.smithi143.stdout: { 2023-11-26T07:24:21.171 INFO:teuthology.orchestra.run.smithi143.stdout: "bytes_pcent": "0.00" 2023-11-26T07:24:21.171 INFO:teuthology.orchestra.run.smithi143.stdout: } 2023-11-26T07:24:21.172 INFO:teuthology.orchestra.run.smithi143.stdout:] 2023-11-26T07:24:21.173 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:21.171+0000 7f50ed7df640 1 -- 172.21.15.143:0/723027127 >> [v2:172.21.15.188:6836/3401128553,v1:172.21.15.188:6837/3401128553] conn(0x7f50c8066650 msgr2=0x7f50c8068b10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:24:21.173 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:21.171+0000 7f50ed7df640 1 --2- 172.21.15.143:0/723027127 >> [v2:172.21.15.188:6836/3401128553,v1:172.21.15.188:6837/3401128553] conn(0x7f50c8066650 0x7f50c8068b10 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7f50d0007fd0 tx=0x7f50d0000bd0 comp rx=0 tx=0).stop 2023-11-26T07:24:21.173 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:21.171+0000 7f50ed7df640 1 -- 172.21.15.143:0/723027127 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f50e8143ac0 msgr2=0x7f50e8139d00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:24:21.173 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:21.171+0000 7f50ed7df640 1 --2- 172.21.15.143:0/723027127 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f50e8143ac0 0x7f50e8139d00 secure :-1 s=READY pgs=1347 cs=0 l=1 rev1=1 crypto rx=0x7f50e000efc0 tx=0x7f50e000c490 comp rx=0 tx=0).stop 2023-11-26T07:24:21.173 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:21.171+0000 7f50ed7df640 1 -- 172.21.15.143:0/723027127 shutdown_connections 2023-11-26T07:24:21.173 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:21.171+0000 7f50ed7df640 1 --2- 172.21.15.143:0/723027127 >> [v2:172.21.15.188:6836/3401128553,v1:172.21.15.188:6837/3401128553] conn(0x7f50c8066650 0x7f50c8068b10 unknown :-1 s=CLOSED pgs=142 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:24:21.173 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:21.171+0000 7f50ed7df640 1 --2- 172.21.15.143:0/723027127 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f50e8143ac0 0x7f50e8139d00 unknown :-1 s=CLOSED pgs=1347 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:24:21.174 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:21.171+0000 7f50ed7df640 1 --2- 172.21.15.143:0/723027127 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f50e80cf280 0x7f50e80acf40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:24:21.174 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:21.171+0000 7f50ed7df640 1 --2- 172.21.15.143:0/723027127 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f50e80cd1b0 0x7f50e80b0f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:24:21.174 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:21.171+0000 7f50ed7df640 1 -- 172.21.15.143:0/723027127 >> 172.21.15.143:0/723027127 conn(0x7f50e806a890 msgr2=0x7f50e8072150 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:24:21.174 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:21.171+0000 7f50ed7df640 1 -- 172.21.15.143:0/723027127 shutdown_connections 2023-11-26T07:24:21.174 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:21.171+0000 7f50ed7df640 1 -- 172.21.15.143:0/723027127 wait complete. 2023-11-26T07:24:21.192 DEBUG:teuthology.orchestra.run.smithi143:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume info cephfs subvol24520 2023-11-26T07:24:21.332 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:21.331+0000 7fe5ee562640 1 Processor -- start 2023-11-26T07:24:21.333 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:21.331+0000 7fe5ee562640 1 -- start start 2023-11-26T07:24:21.334 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:21.335+0000 7fe5ee562640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fe5e8144720 0x7fe5e806d300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:24:21.334 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:21.335+0000 7fe5ee562640 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7fe5e8071bf0 0x7fe5e815bde0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:24:21.334 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:21.335+0000 7fe5ee562640 1 -- --> v1:172.21.15.143:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe5e806e3f0 con 0x7fe5e806d840 2023-11-26T07:24:21.334 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:21.335+0000 7fe5ee562640 1 -- --> v2:172.21.15.188:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe5e80722f0 con 0x7fe5e8071bf0 2023-11-26T07:24:21.335 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:21.335+0000 7fe5ee562640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe5e806de30 con 0x7fe5e8144720 2023-11-26T07:24:21.335 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:21.335+0000 7fe5ecd5f640 1 --1- >> v1:172.21.15.143:6789/0 conn(0x7fe5e806d840 0x7fe5e80714e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.143:6789/0 says I am v1:172.21.15.143:40274/0 (socket says 172.21.15.143:40274) 2023-11-26T07:24:21.335 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:21.335+0000 7fe5ecd5f640 1 -- 172.21.15.143:0/4194147704 learned_addr learned my addr 172.21.15.143:0/4194147704 (peer_addr_for_me v1:172.21.15.143:0/0) 2023-11-26T07:24:21.335 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:21.335+0000 7fe5edd61640 1 --2- 172.21.15.143:0/4194147704 >> v2:172.21.15.188:3300/0 conn(0x7fe5e8071bf0 0x7fe5e815bde0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:24:21.336 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:21.335+0000 7fe5ed560640 1 --2- 172.21.15.143:0/4194147704 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fe5e8144720 0x7fe5e806d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:24:21.336 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:21.335+0000 7fe5dffff640 1 -- 172.21.15.143:0/4194147704 <== mon.0 v1:172.21.15.143:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1560152850 0 0) 0x7fe5e806e3f0 con 0x7fe5e806d840 2023-11-26T07:24:21.336 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:21.335+0000 7fe5dffff640 1 -- 172.21.15.143:0/4194147704 --> v1:172.21.15.143:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fe5d4003530 con 0x7fe5e806d840 2023-11-26T07:24:21.336 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:21.335+0000 7fe5ed560640 1 -- 172.21.15.143:0/4194147704 >> v2:172.21.15.188:3300/0 conn(0x7fe5e8071bf0 msgr2=0x7fe5e815bde0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-26T07:24:21.336 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:21.335+0000 7fe5ed560640 1 --2- 172.21.15.143:0/4194147704 >> v2:172.21.15.188:3300/0 conn(0x7fe5e8071bf0 0x7fe5e815bde0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:24:21.336 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:21.335+0000 7fe5ed560640 1 -- 172.21.15.143:0/4194147704 >> v1:172.21.15.143:6789/0 conn(0x7fe5e806d840 legacy=0x7fe5e80714e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:24:21.336 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:21.335+0000 7fe5ed560640 1 -- 172.21.15.143:0/4194147704 --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe5e806e110 con 0x7fe5e8144720 2023-11-26T07:24:21.336 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:21.335+0000 7fe5edd61640 1 --2- 172.21.15.143:0/4194147704 >> v2:172.21.15.188:3300/0 conn(0x7fe5e8071bf0 0x7fe5e815bde0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-26T07:24:21.338 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:21.339+0000 7fe5ed560640 1 --2- 172.21.15.143:0/4194147704 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fe5e8144720 0x7fe5e806d300 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7fe5e40097c0 tx=0x7fe5e401c450 comp rx=0 tx=0).ready entity=mon.2 client_cookie=636e0d8e0ec14348 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:24:21.338 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:21.339+0000 7fe5dffff640 1 -- 172.21.15.143:0/4194147704 <== mon.2 v2:172.21.15.188:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe5e40ad660 con 0x7fe5e8144720 2023-11-26T07:24:21.338 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:21.339+0000 7fe5dffff640 1 -- 172.21.15.143:0/4194147704 <== mon.2 v2:172.21.15.188:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe5e40ad7c0 con 0x7fe5e8144720 2023-11-26T07:24:21.339 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:21.339+0000 7fe5dffff640 1 -- 172.21.15.143:0/4194147704 <== mon.2 v2:172.21.15.188:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe5e40adb50 con 0x7fe5e8144720 2023-11-26T07:24:21.339 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:21.339+0000 7fe5ee562640 1 -- 172.21.15.143:0/4194147704 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fe5e8144720 msgr2=0x7fe5e806d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:24:21.339 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:21.339+0000 7fe5ee562640 1 --2- 172.21.15.143:0/4194147704 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fe5e8144720 0x7fe5e806d300 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7fe5e40097c0 tx=0x7fe5e401c450 comp rx=0 tx=0).stop 2023-11-26T07:24:21.339 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:21.339+0000 7fe5ee562640 1 -- 172.21.15.143:0/4194147704 shutdown_connections 2023-11-26T07:24:21.340 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:21.339+0000 7fe5ee562640 1 --2- 172.21.15.143:0/4194147704 >> v2:172.21.15.188:3300/0 conn(0x7fe5e8071bf0 0x7fe5e815bde0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:24:21.340 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:21.339+0000 7fe5ee562640 1 --2- 172.21.15.143:0/4194147704 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fe5e8144720 0x7fe5e806d300 unknown :-1 s=CLOSED pgs=161 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:24:21.340 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:21.339+0000 7fe5ee562640 1 -- 172.21.15.143:0/4194147704 >> 172.21.15.143:0/4194147704 conn(0x7fe5e806a890 msgr2=0x7fe5e806aca0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:24:21.340 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:21.339+0000 7fe5ee562640 1 -- 172.21.15.143:0/4194147704 shutdown_connections 2023-11-26T07:24:21.340 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:21.339+0000 7fe5ee562640 1 -- 172.21.15.143:0/4194147704 wait complete. 2023-11-26T07:24:21.340 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:21.339+0000 7fe5ee562640 1 Processor -- start 2023-11-26T07:24:21.341 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:21.339+0000 7fe5ee562640 1 -- start start 2023-11-26T07:24:21.343 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:21.343+0000 7fe5ee562640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fe5e806d840 0x7fe5e80ccb50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:24:21.343 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:21.343+0000 7fe5ee562640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fe5e8071bf0 0x7fe5e80b3150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:24:21.343 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:21.343+0000 7fe5ee562640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fe5e8144720 0x7fe5e81403c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:24:21.343 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:21.343+0000 7fe5ee562640 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe5e80cd090 con 0x7fe5e8071bf0 2023-11-26T07:24:21.343 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:21.343+0000 7fe5ee562640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe5e80cbe20 con 0x7fe5e806d840 2023-11-26T07:24:21.343 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:21.343+0000 7fe5ee562640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe5e80cc6c0 con 0x7fe5e8144720 2023-11-26T07:24:21.343 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:21.343+0000 7fe5ecd5f640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fe5e8071bf0 0x7fe5e80b3150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:24:21.343 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:21.343+0000 7fe5ecd5f640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fe5e8071bf0 0x7fe5e80b3150 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:54654/0 (socket says 172.21.15.143:54654) 2023-11-26T07:24:21.344 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:21.343+0000 7fe5ecd5f640 1 -- 172.21.15.143:0/4038126155 learned_addr learned my addr 172.21.15.143:0/4038126155 (peer_addr_for_me v2:172.21.15.143:0/0) 2023-11-26T07:24:21.344 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:21.343+0000 7fe5ed560640 1 --2- 172.21.15.143:0/4038126155 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fe5e806d840 0x7fe5e80ccb50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:24:21.344 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:21.343+0000 7fe5edd61640 1 --2- 172.21.15.143:0/4038126155 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fe5e8144720 0x7fe5e81403c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:24:21.344 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:21.343+0000 7fe5ecd5f640 1 -- 172.21.15.143:0/4038126155 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fe5e8144720 msgr2=0x7fe5e81403c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:24:21.344 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:21.343+0000 7fe5ecd5f640 1 --2- 172.21.15.143:0/4038126155 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fe5e8144720 0x7fe5e81403c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:24:21.344 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:21.343+0000 7fe5ecd5f640 1 -- 172.21.15.143:0/4038126155 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fe5e806d840 msgr2=0x7fe5e80ccb50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:24:21.344 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:21.343+0000 7fe5ecd5f640 1 --2- 172.21.15.143:0/4038126155 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fe5e806d840 0x7fe5e80ccb50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:24:21.344 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:21.343+0000 7fe5ecd5f640 1 -- 172.21.15.143:0/4038126155 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe5e0009a70 con 0x7fe5e8071bf0 2023-11-26T07:24:21.344 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:21.343+0000 7fe5ecd5f640 1 --2- 172.21.15.143:0/4038126155 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fe5e8071bf0 0x7fe5e80b3150 secure :-1 s=READY pgs=1349 cs=0 l=1 rev1=1 crypto rx=0x7fe5d800c500 tx=0x7fe5d800c9c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:24:21.344 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:21.343+0000 7fe5ddffb640 1 -- 172.21.15.143:0/4038126155 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe5d8005cf0 con 0x7fe5e8071bf0 2023-11-26T07:24:21.344 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:21.343+0000 7fe5ddffb640 1 -- 172.21.15.143:0/4038126155 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe5d8005e50 con 0x7fe5e8071bf0 2023-11-26T07:24:21.345 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:21.343+0000 7fe5ddffb640 1 -- 172.21.15.143:0/4038126155 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe5d80072f0 con 0x7fe5e8071bf0 2023-11-26T07:24:21.345 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:21.343+0000 7fe5ee562640 1 -- 172.21.15.143:0/4038126155 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe5e4009420 con 0x7fe5e8071bf0 2023-11-26T07:24:21.345 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:21.343+0000 7fe5ee562640 1 -- 172.21.15.143:0/4038126155 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe5e80cb950 con 0x7fe5e8071bf0 2023-11-26T07:24:21.345 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:21.343+0000 7fe5ee562640 1 -- 172.21.15.143:0/4038126155 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe5e8067500 con 0x7fe5e8071bf0 2023-11-26T07:24:21.345 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:21.343+0000 7fe5ddffb640 1 -- 172.21.15.143:0/4038126155 <== mon.0 v2:172.21.15.143:3300/0 4 ==== mgrmap(e 17) v1 ==== 86005+0+0 (secure 0 0 0) 0x7fe5d8007450 con 0x7fe5e8071bf0 2023-11-26T07:24:21.346 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:21.347+0000 7fe5ddffb640 1 --2- 172.21.15.143:0/4038126155 >> [v2:172.21.15.188:6836/3401128553,v1:172.21.15.188:6837/3401128553] conn(0x7fe5cc066620 0x7fe5cc068ae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:24:21.346 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:21.347+0000 7fe5ddffb640 1 -- 172.21.15.143:0/4038126155 <== mon.0 v2:172.21.15.143:3300/0 5 ==== osd_map(103..103 src has 1..103) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fe5d8083610 con 0x7fe5e8071bf0 2023-11-26T07:24:21.346 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:21.347+0000 7fe5ed560640 1 --2- 172.21.15.143:0/4038126155 >> [v2:172.21.15.188:6836/3401128553,v1:172.21.15.188:6837/3401128553] conn(0x7fe5cc066620 0x7fe5cc068ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:24:21.347 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:21.347+0000 7fe5ed560640 1 --2- 172.21.15.143:0/4038126155 >> [v2:172.21.15.188:6836/3401128553,v1:172.21.15.188:6837/3401128553] conn(0x7fe5cc066620 0x7fe5cc068ae0 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7fe5e40af000 tx=0x7fe5e40a7000 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:24:21.350 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:21.351+0000 7fe5ddffb640 1 -- 172.21.15.143:0/4038126155 <== mon.0 v2:172.21.15.143:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172649 (secure 0 0 0) 0x7fe5d8050960 con 0x7fe5e8071bf0 2023-11-26T07:24:21.523 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:21.523+0000 7fe5ee562640 1 -- 172.21.15.143:0/4038126155 --> [v2:172.21.15.188:6836/3401128553,v1:172.21.15.188:6837/3401128553] -- mgr_command(tid 0: {"prefix": "fs subvolume info", "vol_name": "cephfs", "sub_name": "subvol24520", "target": ["mon-mgr", ""]}) v1 -- 0x7fe5e80cc800 con 0x7fe5cc066620 2023-11-26T07:24:21.537 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:21.535+0000 7fe5ddffb640 1 -- 172.21.15.143:0/4038126155 <== mgr.4108 v2:172.21.15.188:6836/3401128553 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+663 (secure 0 0 0) 0x7fe5e80cc800 con 0x7fe5cc066620 2023-11-26T07:24:21.537 INFO:teuthology.orchestra.run.smithi143.stdout:{ 2023-11-26T07:24:21.537 INFO:teuthology.orchestra.run.smithi143.stdout: "atime": "2023-11-26 07:24:19", 2023-11-26T07:24:21.537 INFO:teuthology.orchestra.run.smithi143.stdout: "bytes_pcent": "0.00", 2023-11-26T07:24:21.537 INFO:teuthology.orchestra.run.smithi143.stdout: "bytes_quota": 10485760, 2023-11-26T07:24:21.538 INFO:teuthology.orchestra.run.smithi143.stdout: "bytes_used": 0, 2023-11-26T07:24:21.538 INFO:teuthology.orchestra.run.smithi143.stdout: "created_at": "2023-11-26 07:24:19", 2023-11-26T07:24:21.538 INFO:teuthology.orchestra.run.smithi143.stdout: "ctime": "2023-11-26 07:24:21", 2023-11-26T07:24:21.538 INFO:teuthology.orchestra.run.smithi143.stdout: "data_pool": "cephfs_data", 2023-11-26T07:24:21.538 INFO:teuthology.orchestra.run.smithi143.stdout: "features": [ 2023-11-26T07:24:21.538 INFO:teuthology.orchestra.run.smithi143.stdout: "snapshot-clone", 2023-11-26T07:24:21.538 INFO:teuthology.orchestra.run.smithi143.stdout: "snapshot-autoprotect" 2023-11-26T07:24:21.538 INFO:teuthology.orchestra.run.smithi143.stdout: ], 2023-11-26T07:24:21.538 INFO:teuthology.orchestra.run.smithi143.stdout: "gid": 0, 2023-11-26T07:24:21.538 INFO:teuthology.orchestra.run.smithi143.stdout: "mode": 16877, 2023-11-26T07:24:21.538 INFO:teuthology.orchestra.run.smithi143.stdout: "mon_addrs": [ 2023-11-26T07:24:21.539 INFO:teuthology.orchestra.run.smithi143.stdout: "172.21.15.143:6789", 2023-11-26T07:24:21.539 INFO:teuthology.orchestra.run.smithi143.stdout: "172.21.15.188:6789", 2023-11-26T07:24:21.539 INFO:teuthology.orchestra.run.smithi143.stdout: "172.21.15.188:6790" 2023-11-26T07:24:21.539 INFO:teuthology.orchestra.run.smithi143.stdout: ], 2023-11-26T07:24:21.539 INFO:teuthology.orchestra.run.smithi143.stdout: "mtime": "2023-11-26 07:24:19", 2023-11-26T07:24:21.539 INFO:teuthology.orchestra.run.smithi143.stdout: "path": "/volumes/_nogroup/subvol24520/f57b3d1b-7328-4835-8e29-9827b7800bab", 2023-11-26T07:24:21.539 INFO:teuthology.orchestra.run.smithi143.stdout: "pool_namespace": "", 2023-11-26T07:24:21.539 INFO:teuthology.orchestra.run.smithi143.stdout: "state": "complete", 2023-11-26T07:24:21.539 INFO:teuthology.orchestra.run.smithi143.stdout: "type": "subvolume", 2023-11-26T07:24:21.539 INFO:teuthology.orchestra.run.smithi143.stdout: "uid": 0 2023-11-26T07:24:21.539 INFO:teuthology.orchestra.run.smithi143.stdout:} 2023-11-26T07:24:21.540 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:21.539+0000 7fe5ee562640 1 -- 172.21.15.143:0/4038126155 >> [v2:172.21.15.188:6836/3401128553,v1:172.21.15.188:6837/3401128553] conn(0x7fe5cc066620 msgr2=0x7fe5cc068ae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:24:21.540 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:21.539+0000 7fe5ee562640 1 --2- 172.21.15.143:0/4038126155 >> [v2:172.21.15.188:6836/3401128553,v1:172.21.15.188:6837/3401128553] conn(0x7fe5cc066620 0x7fe5cc068ae0 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7fe5e40af000 tx=0x7fe5e40a7000 comp rx=0 tx=0).stop 2023-11-26T07:24:21.540 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:21.539+0000 7fe5ee562640 1 -- 172.21.15.143:0/4038126155 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fe5e8071bf0 msgr2=0x7fe5e80b3150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:24:21.540 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:21.539+0000 7fe5ee562640 1 --2- 172.21.15.143:0/4038126155 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fe5e8071bf0 0x7fe5e80b3150 secure :-1 s=READY pgs=1349 cs=0 l=1 rev1=1 crypto rx=0x7fe5d800c500 tx=0x7fe5d800c9c0 comp rx=0 tx=0).stop 2023-11-26T07:24:21.541 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:21.539+0000 7fe5ee562640 1 -- 172.21.15.143:0/4038126155 shutdown_connections 2023-11-26T07:24:21.541 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:21.539+0000 7fe5ee562640 1 --2- 172.21.15.143:0/4038126155 >> [v2:172.21.15.188:6836/3401128553,v1:172.21.15.188:6837/3401128553] conn(0x7fe5cc066620 0x7fe5cc068ae0 unknown :-1 s=CLOSED pgs=143 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:24:21.541 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:21.539+0000 7fe5ee562640 1 --2- 172.21.15.143:0/4038126155 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fe5e8144720 0x7fe5e81403c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:24:21.541 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:21.539+0000 7fe5ee562640 1 --2- 172.21.15.143:0/4038126155 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fe5e8071bf0 0x7fe5e80b3150 unknown :-1 s=CLOSED pgs=1349 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:24:21.541 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:21.539+0000 7fe5ee562640 1 --2- 172.21.15.143:0/4038126155 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fe5e806d840 0x7fe5e80ccb50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:24:21.541 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:21.539+0000 7fe5ee562640 1 -- 172.21.15.143:0/4038126155 >> 172.21.15.143:0/4038126155 conn(0x7fe5e806a890 msgr2=0x7fe5e80b1380 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:24:21.541 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:21.539+0000 7fe5ee562640 1 -- 172.21.15.143:0/4038126155 shutdown_connections 2023-11-26T07:24:21.541 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:21.539+0000 7fe5ee562640 1 -- 172.21.15.143:0/4038126155 wait complete. 2023-11-26T07:24:21.558 DEBUG:teuthology.orchestra.run.smithi143:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume create cephfs subvol24520 --mode 777 --uid 1000 --gid 1000 2023-11-26T07:24:21.706 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:21.707+0000 7f380d002640 1 Processor -- start 2023-11-26T07:24:21.706 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:21.707+0000 7f380d002640 1 -- start start 2023-11-26T07:24:21.707 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:21.707+0000 7f380d002640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f38080cddd0 0x7f38080cf170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:24:21.707 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:21.707+0000 7f380d002640 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7f38080b6050 0x7f38080b64c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:24:21.707 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:21.707+0000 7f380d002640 1 -- --> v1:172.21.15.188:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f380815d7e0 con 0x7f380815d400 2023-11-26T07:24:21.708 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:21.707+0000 7f380d002640 1 -- --> v2:172.21.15.188:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f38080d0530 con 0x7f38080b6050 2023-11-26T07:24:21.708 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:21.707+0000 7f380d002640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f38080cb5c0 con 0x7f38080cddd0 2023-11-26T07:24:21.709 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:21.707+0000 7f380759e640 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7f38080b6050 0x7f38080b64c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:24:21.709 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:21.707+0000 7f380659c640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f38080cddd0 0x7f38080cf170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:24:21.709 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:21.707+0000 7f380659c640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f38080cddd0 0x7f38080cf170 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.188:3301/0 says I am v2:172.21.15.143:36962/0 (socket says 172.21.15.143:36962) 2023-11-26T07:24:21.709 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:21.707+0000 7f380759e640 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7f38080b6050 0x7f38080b64c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.188:3300/0 says I am v2:172.21.15.143:57386/0 (socket says 172.21.15.143:57386) 2023-11-26T07:24:21.709 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:21.707+0000 7f380659c640 1 -- 172.21.15.143:0/4091179467 learned_addr learned my addr 172.21.15.143:0/4091179467 (peer_addr_for_me v2:172.21.15.143:0/0) 2023-11-26T07:24:21.710 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:21.707+0000 7f380659c640 1 -- 172.21.15.143:0/4091179467 >> v2:172.21.15.188:3300/0 conn(0x7f38080b6050 msgr2=0x7f38080b64c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-26T07:24:21.710 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:21.707+0000 7f380659c640 1 --2- 172.21.15.143:0/4091179467 >> v2:172.21.15.188:3300/0 conn(0x7f38080b6050 0x7f38080b64c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:24:21.710 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:21.707+0000 7f380659c640 1 -- 172.21.15.143:0/4091179467 >> v1:172.21.15.188:6789/0 conn(0x7f380815d400 legacy=0x7f38080cd6c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:24:21.710 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:21.707+0000 7f380659c640 1 -- 172.21.15.143:0/4091179467 --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f38080cb8a0 con 0x7f38080cddd0 2023-11-26T07:24:21.710 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:21.707+0000 7f380759e640 1 --2- 172.21.15.143:0/4091179467 >> v2:172.21.15.188:3300/0 conn(0x7f38080b6050 0x7f38080b64c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-26T07:24:21.711 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:21.711+0000 7f380659c640 1 --2- 172.21.15.143:0/4091179467 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f38080cddd0 0x7f38080cf170 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7f37fc009870 tx=0x7f37fc01c4b0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=68dcedef5da0098a server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:24:21.712 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:21.711+0000 7f3805d9b640 1 -- 172.21.15.143:0/4091179467 <== mon.2 v2:172.21.15.188:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f37fc0ac620 con 0x7f38080cddd0 2023-11-26T07:24:21.712 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:21.711+0000 7f3805d9b640 1 -- 172.21.15.143:0/4091179467 <== mon.2 v2:172.21.15.188:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f37fc0ac780 con 0x7f38080cddd0 2023-11-26T07:24:21.712 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:21.711+0000 7f3805d9b640 1 -- 172.21.15.143:0/4091179467 <== mon.2 v2:172.21.15.188:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f37fc0acb10 con 0x7f38080cddd0 2023-11-26T07:24:21.713 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:21.711+0000 7f380d002640 1 -- 172.21.15.143:0/4091179467 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f38080cddd0 msgr2=0x7f38080cf170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:24:21.713 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:21.711+0000 7f380d002640 1 --2- 172.21.15.143:0/4091179467 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f38080cddd0 0x7f38080cf170 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7f37fc009870 tx=0x7f37fc01c4b0 comp rx=0 tx=0).stop 2023-11-26T07:24:21.713 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:21.711+0000 7f380d002640 1 -- 172.21.15.143:0/4091179467 shutdown_connections 2023-11-26T07:24:21.713 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:21.711+0000 7f380d002640 1 --2- 172.21.15.143:0/4091179467 >> v2:172.21.15.188:3300/0 conn(0x7f38080b6050 0x7f38080b64c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:24:21.713 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:21.711+0000 7f380d002640 1 --2- 172.21.15.143:0/4091179467 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f38080cddd0 0x7f38080cf170 unknown :-1 s=CLOSED pgs=162 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:24:21.713 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:21.711+0000 7f380d002640 1 -- 172.21.15.143:0/4091179467 >> 172.21.15.143:0/4091179467 conn(0x7f38080d0e40 msgr2=0x7f38080d1250 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:24:21.713 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:21.711+0000 7f380d002640 1 -- 172.21.15.143:0/4091179467 shutdown_connections 2023-11-26T07:24:21.714 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:21.711+0000 7f380d002640 1 -- 172.21.15.143:0/4091179467 wait complete. 2023-11-26T07:24:21.714 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:21.715+0000 7f380d002640 1 Processor -- start 2023-11-26T07:24:21.715 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:21.715+0000 7f380d002640 1 -- start start 2023-11-26T07:24:21.717 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:21.715+0000 7f380d002640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f38080b6050 0x7f38081491e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:24:21.717 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:21.715+0000 7f380d002640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f38080cddd0 0x7f3808160280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:24:21.717 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:21.715+0000 7f380d002640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f380815d400 0x7f38081607c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:24:21.717 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:21.715+0000 7f380d002640 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f38080ca550 con 0x7f38080b6050 2023-11-26T07:24:21.717 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:21.715+0000 7f380d002640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f38080ca830 con 0x7f38080cddd0 2023-11-26T07:24:21.718 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:21.715+0000 7f380d002640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f38080cab10 con 0x7f380815d400 2023-11-26T07:24:21.718 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:21.715+0000 7f3806d9d640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f38080b6050 0x7f38081491e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:24:21.718 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:21.715+0000 7f3806d9d640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f38080b6050 0x7f38081491e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:54658/0 (socket says 172.21.15.143:54658) 2023-11-26T07:24:21.718 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:21.715+0000 7f3806d9d640 1 -- 172.21.15.143:0/2642594935 learned_addr learned my addr 172.21.15.143:0/2642594935 (peer_addr_for_me v2:172.21.15.143:0/0) 2023-11-26T07:24:21.718 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:21.715+0000 7f380659c640 1 --2- 172.21.15.143:0/2642594935 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f38080cddd0 0x7f3808160280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:24:21.719 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:21.715+0000 7f380759e640 1 --2- 172.21.15.143:0/2642594935 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f380815d400 0x7f38081607c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:24:21.719 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:21.715+0000 7f3806d9d640 1 -- 172.21.15.143:0/2642594935 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f380815d400 msgr2=0x7f38081607c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:24:21.719 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:21.715+0000 7f3806d9d640 1 --2- 172.21.15.143:0/2642594935 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f380815d400 0x7f38081607c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:24:21.719 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:21.715+0000 7f3806d9d640 1 -- 172.21.15.143:0/2642594935 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f38080cddd0 msgr2=0x7f3808160280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:24:21.719 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:21.715+0000 7f3806d9d640 1 --2- 172.21.15.143:0/2642594935 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f38080cddd0 0x7f3808160280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:24:21.719 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:21.715+0000 7f3806d9d640 1 -- 172.21.15.143:0/2642594935 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3800009a70 con 0x7f38080b6050 2023-11-26T07:24:21.719 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:21.715+0000 7f380759e640 1 --2- 172.21.15.143:0/2642594935 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f380815d400 0x7f38081607c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-26T07:24:21.720 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:21.715+0000 7f3806d9d640 1 --2- 172.21.15.143:0/2642594935 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f38080b6050 0x7f38081491e0 secure :-1 s=READY pgs=1350 cs=0 l=1 rev1=1 crypto rx=0x7f37f800c3c0 tx=0x7f37f800c880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:24:21.720 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:21.715+0000 7f37ef7fe640 1 -- 172.21.15.143:0/2642594935 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f37f80052d0 con 0x7f38080b6050 2023-11-26T07:24:21.720 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:21.715+0000 7f37ef7fe640 1 -- 172.21.15.143:0/2642594935 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f37f8005430 con 0x7f38080b6050 2023-11-26T07:24:21.720 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:21.715+0000 7f380d002640 1 -- 172.21.15.143:0/2642594935 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f37fc0094d0 con 0x7f38080b6050 2023-11-26T07:24:21.720 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:21.715+0000 7f37ef7fe640 1 -- 172.21.15.143:0/2642594935 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f37f8003840 con 0x7f38080b6050 2023-11-26T07:24:21.720 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:21.715+0000 7f380d002640 1 -- 172.21.15.143:0/2642594935 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f38080cb0d0 con 0x7f38080b6050 2023-11-26T07:24:21.721 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:21.719+0000 7f380d002640 1 -- 172.21.15.143:0/2642594935 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3808067570 con 0x7f38080b6050 2023-11-26T07:24:21.722 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:21.719+0000 7f37ef7fe640 1 -- 172.21.15.143:0/2642594935 <== mon.0 v2:172.21.15.143:3300/0 4 ==== mgrmap(e 17) v1 ==== 86005+0+0 (secure 0 0 0) 0x7f37f80039a0 con 0x7f38080b6050 2023-11-26T07:24:21.722 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:21.719+0000 7f37ef7fe640 1 --2- 172.21.15.143:0/2642594935 >> [v2:172.21.15.188:6836/3401128553,v1:172.21.15.188:6837/3401128553] conn(0x7f37f4066620 0x7f37f4068ae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:24:21.722 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:21.719+0000 7f37ef7fe640 1 -- 172.21.15.143:0/2642594935 <== mon.0 v2:172.21.15.143:3300/0 5 ==== osd_map(103..103 src has 1..103) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f37f8083760 con 0x7f38080b6050 2023-11-26T07:24:21.722 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:21.719+0000 7f380659c640 1 --2- 172.21.15.143:0/2642594935 >> [v2:172.21.15.188:6836/3401128553,v1:172.21.15.188:6837/3401128553] conn(0x7f37f4066620 0x7f37f4068ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:24:21.722 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:21.719+0000 7f380659c640 1 --2- 172.21.15.143:0/2642594935 >> [v2:172.21.15.188:6836/3401128553,v1:172.21.15.188:6837/3401128553] conn(0x7f37f4066620 0x7f37f4068ae0 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7f37fc0ae000 tx=0x7f37fc008040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:24:21.723 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:21.723+0000 7f37ef7fe640 1 -- 172.21.15.143:0/2642594935 <== mon.0 v2:172.21.15.143:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172649 (secure 0 0 0) 0x7f37f80508c0 con 0x7f38080b6050 2023-11-26T07:24:21.893 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:21.891+0000 7f380d002640 1 -- 172.21.15.143:0/2642594935 --> [v2:172.21.15.188:6836/3401128553,v1:172.21.15.188:6837/3401128553] -- mgr_command(tid 0: {"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "subvol24520", "mode": "777", "uid": 1000, "gid": 1000, "target": ["mon-mgr", ""]}) v1 -- 0x7f38080667a0 con 0x7f37f4066620 2023-11-26T07:24:21.910 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:21.911+0000 7f37ef7fe640 1 -- 172.21.15.143:0/2642594935 <== mgr.4108 v2:172.21.15.188:6836/3401128553 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+0 (secure 0 0 0) 0x7f38080667a0 con 0x7f37f4066620 2023-11-26T07:24:21.913 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:21.911+0000 7f380d002640 1 -- 172.21.15.143:0/2642594935 >> [v2:172.21.15.188:6836/3401128553,v1:172.21.15.188:6837/3401128553] conn(0x7f37f4066620 msgr2=0x7f37f4068ae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:24:21.913 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:21.911+0000 7f380d002640 1 --2- 172.21.15.143:0/2642594935 >> [v2:172.21.15.188:6836/3401128553,v1:172.21.15.188:6837/3401128553] conn(0x7f37f4066620 0x7f37f4068ae0 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7f37fc0ae000 tx=0x7f37fc008040 comp rx=0 tx=0).stop 2023-11-26T07:24:21.913 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:21.911+0000 7f380d002640 1 -- 172.21.15.143:0/2642594935 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f38080b6050 msgr2=0x7f38081491e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:24:21.913 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:21.911+0000 7f380d002640 1 --2- 172.21.15.143:0/2642594935 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f38080b6050 0x7f38081491e0 secure :-1 s=READY pgs=1350 cs=0 l=1 rev1=1 crypto rx=0x7f37f800c3c0 tx=0x7f37f800c880 comp rx=0 tx=0).stop 2023-11-26T07:24:21.913 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:21.911+0000 7f380d002640 1 -- 172.21.15.143:0/2642594935 shutdown_connections 2023-11-26T07:24:21.913 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:21.911+0000 7f380d002640 1 --2- 172.21.15.143:0/2642594935 >> [v2:172.21.15.188:6836/3401128553,v1:172.21.15.188:6837/3401128553] conn(0x7f37f4066620 0x7f37f4068ae0 unknown :-1 s=CLOSED pgs=144 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:24:21.913 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:21.911+0000 7f380d002640 1 --2- 172.21.15.143:0/2642594935 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f380815d400 0x7f38081607c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:24:21.914 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:21.911+0000 7f380d002640 1 --2- 172.21.15.143:0/2642594935 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f38080cddd0 0x7f3808160280 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:24:21.914 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:21.911+0000 7f380d002640 1 --2- 172.21.15.143:0/2642594935 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f38080b6050 0x7f38081491e0 unknown :-1 s=CLOSED pgs=1350 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:24:21.914 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:21.911+0000 7f380d002640 1 -- 172.21.15.143:0/2642594935 >> 172.21.15.143:0/2642594935 conn(0x7f38080d0e40 msgr2=0x7f38080b31c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:24:21.914 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:21.911+0000 7f380d002640 1 -- 172.21.15.143:0/2642594935 shutdown_connections 2023-11-26T07:24:21.914 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:21.911+0000 7f380d002640 1 -- 172.21.15.143:0/2642594935 wait complete. 2023-11-26T07:24:21.933 DEBUG:teuthology.orchestra.run.smithi143:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume getpath cephfs subvol24520 2023-11-26T07:24:22.073 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:22.071+0000 7f498b5f5640 1 Processor -- start 2023-11-26T07:24:22.074 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:22.075+0000 7f498b5f5640 1 -- start start 2023-11-26T07:24:22.076 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:22.075+0000 7f498b5f5640 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7f49840cf280 0x7f49840cf710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:24:22.076 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:22.075+0000 7f498b5f5640 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7f49840cd1b0 0x7f49840cd5c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:24:22.076 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:22.075+0000 7f498b5f5640 1 -- --> v1:172.21.15.143:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f49840cc730 con 0x7f4984143ac0 2023-11-26T07:24:22.076 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:22.075+0000 7f498b5f5640 1 -- --> v2:172.21.15.143:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4984143ea0 con 0x7f49840cf280 2023-11-26T07:24:22.076 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:22.075+0000 7f498b5f5640 1 -- --> v2:172.21.15.188:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f49840cfc50 con 0x7f49840cd1b0 2023-11-26T07:24:22.076 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:22.075+0000 7f4988b90640 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7f49840cf280 0x7f49840cf710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:24:22.076 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:22.075+0000 7f4989391640 1 --1- >> v1:172.21.15.143:6789/0 conn(0x7f4984143ac0 0x7f4984070370 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.143:6789/0 says I am v1:172.21.15.143:40286/0 (socket says 172.21.15.143:40286) 2023-11-26T07:24:22.076 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:22.075+0000 7f4989391640 1 -- 172.21.15.143:0/1375263955 learned_addr learned my addr 172.21.15.143:0/1375263955 (peer_addr_for_me v1:172.21.15.143:0/0) 2023-11-26T07:24:22.076 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:22.075+0000 7f4989b92640 1 --2- 172.21.15.143:0/1375263955 >> v2:172.21.15.188:3300/0 conn(0x7f49840cd1b0 0x7f49840cd5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:24:22.076 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:22.075+0000 7f4988b90640 1 -- 172.21.15.143:0/1375263955 >> v2:172.21.15.188:3300/0 conn(0x7f49840cd1b0 msgr2=0x7f49840cd5c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-26T07:24:22.076 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:22.075+0000 7f4988b90640 1 --2- 172.21.15.143:0/1375263955 >> v2:172.21.15.188:3300/0 conn(0x7f49840cd1b0 0x7f49840cd5c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:24:22.077 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:22.075+0000 7f4988b90640 1 -- 172.21.15.143:0/1375263955 >> v1:172.21.15.143:6789/0 conn(0x7f4984143ac0 legacy=0x7f4984070370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:24:22.077 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:22.075+0000 7f4988b90640 1 -- 172.21.15.143:0/1375263955 --> v2:172.21.15.143:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4984070ee0 con 0x7f49840cf280 2023-11-26T07:24:22.077 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:22.075+0000 7f4989b92640 1 --2- 172.21.15.143:0/1375263955 >> v2:172.21.15.188:3300/0 conn(0x7f49840cd1b0 0x7f49840cd5c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-26T07:24:22.078 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:22.079+0000 7f4988b90640 1 --2- 172.21.15.143:0/1375263955 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f49840cf280 0x7f49840cf710 secure :-1 s=READY pgs=1352 cs=0 l=1 rev1=1 crypto rx=0x7f498406e570 tx=0x7f497401c460 comp rx=0 tx=0).ready entity=mon.0 client_cookie=30f901a593d2dfdf server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:24:22.079 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:22.079+0000 7f497bfff640 1 -- 172.21.15.143:0/1375263955 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f49740ab660 con 0x7f49840cf280 2023-11-26T07:24:22.079 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:22.079+0000 7f497bfff640 1 -- 172.21.15.143:0/1375263955 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f49740ab7c0 con 0x7f49840cf280 2023-11-26T07:24:22.079 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:22.079+0000 7f497bfff640 1 -- 172.21.15.143:0/1375263955 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f49740abb50 con 0x7f49840cf280 2023-11-26T07:24:22.080 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:22.079+0000 7f498b5f5640 1 -- 172.21.15.143:0/1375263955 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f49840cf280 msgr2=0x7f49840cf710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:24:22.080 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:22.079+0000 7f498b5f5640 1 --2- 172.21.15.143:0/1375263955 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f49840cf280 0x7f49840cf710 secure :-1 s=READY pgs=1352 cs=0 l=1 rev1=1 crypto rx=0x7f498406e570 tx=0x7f497401c460 comp rx=0 tx=0).stop 2023-11-26T07:24:22.080 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:22.079+0000 7f498b5f5640 1 -- 172.21.15.143:0/1375263955 shutdown_connections 2023-11-26T07:24:22.080 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:22.079+0000 7f498b5f5640 1 --2- 172.21.15.143:0/1375263955 >> v2:172.21.15.188:3300/0 conn(0x7f49840cd1b0 0x7f49840cd5c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:24:22.081 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:22.079+0000 7f498b5f5640 1 --2- 172.21.15.143:0/1375263955 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f49840cf280 0x7f49840cf710 unknown :-1 s=CLOSED pgs=1352 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:24:22.081 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:22.079+0000 7f498b5f5640 1 -- 172.21.15.143:0/1375263955 >> 172.21.15.143:0/1375263955 conn(0x7f498406a890 msgr2=0x7f498406aca0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:24:22.081 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:22.079+0000 7f498b5f5640 1 -- 172.21.15.143:0/1375263955 shutdown_connections 2023-11-26T07:24:22.081 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:22.079+0000 7f498b5f5640 1 -- 172.21.15.143:0/1375263955 wait complete. 2023-11-26T07:24:22.081 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:22.079+0000 7f498b5f5640 1 Processor -- start 2023-11-26T07:24:22.082 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:22.079+0000 7f498b5f5640 1 -- start start 2023-11-26T07:24:22.083 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:22.083+0000 7f498b5f5640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f49840cd1b0 0x7f49840b1200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:24:22.084 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:22.083+0000 7f498b5f5640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f49840cf280 0x7f4984142680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:24:22.084 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:22.083+0000 7f498b5f5640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f4984143ac0 0x7f4984159520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:24:22.084 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:22.083+0000 7f498b5f5640 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f49840ae190 con 0x7f4984143ac0 2023-11-26T07:24:22.084 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:22.083+0000 7f498b5f5640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f498415acf0 con 0x7f49840cf280 2023-11-26T07:24:22.084 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:22.083+0000 7f498b5f5640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f49840ad3c0 con 0x7f49840cd1b0 2023-11-26T07:24:22.084 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:22.083+0000 7f4989b92640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f4984143ac0 0x7f4984159520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:24:22.084 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:22.083+0000 7f4989b92640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f4984143ac0 0x7f4984159520 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:54682/0 (socket says 172.21.15.143:54682) 2023-11-26T07:24:22.084 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:22.083+0000 7f4989b92640 1 -- 172.21.15.143:0/1041779914 learned_addr learned my addr 172.21.15.143:0/1041779914 (peer_addr_for_me v2:172.21.15.143:0/0) 2023-11-26T07:24:22.084 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:22.083+0000 7f4988b90640 1 --2- 172.21.15.143:0/1041779914 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f49840cf280 0x7f4984142680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:24:22.084 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:22.083+0000 7f4989391640 1 --2- 172.21.15.143:0/1041779914 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f49840cd1b0 0x7f49840b1200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:24:22.085 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:22.083+0000 7f4989b92640 1 -- 172.21.15.143:0/1041779914 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f49840cd1b0 msgr2=0x7f49840b1200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:24:22.085 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:22.083+0000 7f4989b92640 1 --2- 172.21.15.143:0/1041779914 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f49840cd1b0 0x7f49840b1200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:24:22.085 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:22.083+0000 7f4989b92640 1 -- 172.21.15.143:0/1041779914 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f49840cf280 msgr2=0x7f4984142680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:24:22.085 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:22.083+0000 7f4989b92640 1 --2- 172.21.15.143:0/1041779914 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f49840cf280 0x7f4984142680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:24:22.085 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:22.083+0000 7f4989b92640 1 -- 172.21.15.143:0/1041779914 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4974009420 con 0x7f4984143ac0 2023-11-26T07:24:22.085 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:22.083+0000 7f4989391640 1 --2- 172.21.15.143:0/1041779914 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f49840cd1b0 0x7f49840b1200 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-26T07:24:22.085 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:22.083+0000 7f4989b92640 1 --2- 172.21.15.143:0/1041779914 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f4984143ac0 0x7f4984159520 secure :-1 s=READY pgs=1353 cs=0 l=1 rev1=1 crypto rx=0x7f497c00cd50 tx=0x7f497c00a650 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:24:22.085 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:22.083+0000 7f4979ffb640 1 -- 172.21.15.143:0/1041779914 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f497c00f680 con 0x7f4984143ac0 2023-11-26T07:24:22.085 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:22.083+0000 7f4979ffb640 1 -- 172.21.15.143:0/1041779914 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f497c00f800 con 0x7f4984143ac0 2023-11-26T07:24:22.085 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:22.083+0000 7f4979ffb640 1 -- 172.21.15.143:0/1041779914 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f497c00fbb0 con 0x7f4984143ac0 2023-11-26T07:24:22.085 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:22.083+0000 7f498b5f5640 1 -- 172.21.15.143:0/1041779914 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f49840ad6a0 con 0x7f4984143ac0 2023-11-26T07:24:22.086 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:22.083+0000 7f498b5f5640 1 -- 172.21.15.143:0/1041779914 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f498415a980 con 0x7f4984143ac0 2023-11-26T07:24:22.086 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:22.083+0000 7f498b5f5640 1 -- 172.21.15.143:0/1041779914 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f498415ae60 con 0x7f4984143ac0 2023-11-26T07:24:22.086 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:22.087+0000 7f4979ffb640 1 -- 172.21.15.143:0/1041779914 <== mon.0 v2:172.21.15.143:3300/0 4 ==== mgrmap(e 17) v1 ==== 86005+0+0 (secure 0 0 0) 0x7f497c00b040 con 0x7f4984143ac0 2023-11-26T07:24:22.087 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:22.087+0000 7f4979ffb640 1 --2- 172.21.15.143:0/1041779914 >> [v2:172.21.15.188:6836/3401128553,v1:172.21.15.188:6837/3401128553] conn(0x7f4968066620 0x7f4968068ae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:24:22.087 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:22.087+0000 7f4979ffb640 1 -- 172.21.15.143:0/1041779914 <== mon.0 v2:172.21.15.143:3300/0 5 ==== osd_map(103..103 src has 1..103) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f497c084c20 con 0x7f4984143ac0 2023-11-26T07:24:22.087 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:22.087+0000 7f4989391640 1 --2- 172.21.15.143:0/1041779914 >> [v2:172.21.15.188:6836/3401128553,v1:172.21.15.188:6837/3401128553] conn(0x7f4968066620 0x7f4968068ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:24:22.087 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:22.087+0000 7f4989391640 1 --2- 172.21.15.143:0/1041779914 >> [v2:172.21.15.188:6836/3401128553,v1:172.21.15.188:6837/3401128553] conn(0x7f4968066620 0x7f4968068ae0 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7f4980005890 tx=0x7f4980009040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:24:22.090 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:22.091+0000 7f4979ffb640 1 -- 172.21.15.143:0/1041779914 <== mon.0 v2:172.21.15.143:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172649 (secure 0 0 0) 0x7f497c051c80 con 0x7f4984143ac0 2023-11-26T07:24:22.263 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:22.263+0000 7f498b5f5640 1 -- 172.21.15.143:0/1041779914 --> [v2:172.21.15.188:6836/3401128553,v1:172.21.15.188:6837/3401128553] -- mgr_command(tid 0: {"prefix": "fs subvolume getpath", "vol_name": "cephfs", "sub_name": "subvol24520", "target": ["mon-mgr", ""]}) v1 -- 0x7f4984071c30 con 0x7f4968066620 2023-11-26T07:24:22.280 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:22.279+0000 7f4979ffb640 1 -- 172.21.15.143:0/1041779914 <== mgr.4108 v2:172.21.15.188:6836/3401128553 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+67 (secure 0 0 0) 0x7f4980001430 con 0x7f4968066620 2023-11-26T07:24:22.280 INFO:teuthology.orchestra.run.smithi143.stdout:/volumes/_nogroup/subvol24520/f57b3d1b-7328-4835-8e29-9827b7800bab 2023-11-26T07:24:22.283 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:22.283+0000 7f498b5f5640 1 -- 172.21.15.143:0/1041779914 >> [v2:172.21.15.188:6836/3401128553,v1:172.21.15.188:6837/3401128553] conn(0x7f4968066620 msgr2=0x7f4968068ae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:24:22.283 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:22.283+0000 7f498b5f5640 1 --2- 172.21.15.143:0/1041779914 >> [v2:172.21.15.188:6836/3401128553,v1:172.21.15.188:6837/3401128553] conn(0x7f4968066620 0x7f4968068ae0 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7f4980005890 tx=0x7f4980009040 comp rx=0 tx=0).stop 2023-11-26T07:24:22.283 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:22.283+0000 7f498b5f5640 1 -- 172.21.15.143:0/1041779914 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f4984143ac0 msgr2=0x7f4984159520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:24:22.283 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:22.283+0000 7f498b5f5640 1 --2- 172.21.15.143:0/1041779914 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f4984143ac0 0x7f4984159520 secure :-1 s=READY pgs=1353 cs=0 l=1 rev1=1 crypto rx=0x7f497c00cd50 tx=0x7f497c00a650 comp rx=0 tx=0).stop 2023-11-26T07:24:22.283 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:22.283+0000 7f498b5f5640 1 -- 172.21.15.143:0/1041779914 shutdown_connections 2023-11-26T07:24:22.283 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:22.283+0000 7f498b5f5640 1 --2- 172.21.15.143:0/1041779914 >> [v2:172.21.15.188:6836/3401128553,v1:172.21.15.188:6837/3401128553] conn(0x7f4968066620 0x7f4968068ae0 unknown :-1 s=CLOSED pgs=145 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:24:22.284 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:22.283+0000 7f498b5f5640 1 --2- 172.21.15.143:0/1041779914 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f4984143ac0 0x7f4984159520 unknown :-1 s=CLOSED pgs=1353 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:24:22.284 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:22.283+0000 7f498b5f5640 1 --2- 172.21.15.143:0/1041779914 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f49840cf280 0x7f4984142680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:24:22.284 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:22.283+0000 7f498b5f5640 1 --2- 172.21.15.143:0/1041779914 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f49840cd1b0 0x7f49840b1200 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:24:22.284 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:22.283+0000 7f498b5f5640 1 -- 172.21.15.143:0/1041779914 >> 172.21.15.143:0/1041779914 conn(0x7f498406a890 msgr2=0x7f49840b1f90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:24:22.284 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:22.283+0000 7f498b5f5640 1 -- 172.21.15.143:0/1041779914 shutdown_connections 2023-11-26T07:24:22.284 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:22.283+0000 7f498b5f5640 1 -- 172.21.15.143:0/1041779914 wait complete. 2023-11-26T07:24:22.302 DEBUG:tasks.cephfs.test_volumes:filling subvolume subvol24520 with 8 files each 1MB size under directory volumes/_nogroup/subvol24520/f57b3d1b-7328-4835-8e29-9827b7800bab 2023-11-26T07:24:22.302 DEBUG:teuthology.orchestra.run.smithi143:> set -ex 2023-11-26T07:24:22.302 DEBUG:teuthology.orchestra.run.smithi143:> dd if=/proc/self/mounts of=/dev/stdout 2023-11-26T07:24:22.311 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-26T07:24:22.312 DEBUG:teuthology.orchestra.run.smithi143:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol24520/f57b3d1b-7328-4835-8e29-9827b7800bab/subvolume_file.0 bs=1M conv=fdatasync count=1 seek=0) 2023-11-26T07:24:22.385 INFO:teuthology.orchestra.run.smithi143.stderr:1+0 records in 2023-11-26T07:24:22.385 INFO:teuthology.orchestra.run.smithi143.stderr:1+0 records out 2023-11-26T07:24:22.385 INFO:teuthology.orchestra.run.smithi143.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0170432 s, 61.5 MB/s 2023-11-26T07:24:22.386 DEBUG:teuthology.orchestra.run.smithi143:> set -ex 2023-11-26T07:24:22.386 DEBUG:teuthology.orchestra.run.smithi143:> dd if=/proc/self/mounts of=/dev/stdout 2023-11-26T07:24:22.438 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-26T07:24:22.438 DEBUG:teuthology.orchestra.run.smithi143:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol24520/f57b3d1b-7328-4835-8e29-9827b7800bab/subvolume_file.1 bs=1M conv=fdatasync count=1 seek=0) 2023-11-26T07:24:22.512 INFO:teuthology.orchestra.run.smithi143.stderr:1+0 records in 2023-11-26T07:24:22.720 INFO:teuthology.orchestra.run.smithi143.stderr:1+0 records out 2023-11-26T07:24:22.721 INFO:teuthology.orchestra.run.smithi143.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0165425 s, 63.4 MB/s 2023-11-26T07:24:22.722 DEBUG:teuthology.orchestra.run.smithi143:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-26T07:24:22.724 DEBUG:teuthology.orchestra.run.smithi143:> set -ex 2023-11-26T07:24:22.724 DEBUG:teuthology.orchestra.run.smithi143:> dd if=/proc/self/mounts of=/dev/stdout 2023-11-26T07:24:22.768 DEBUG:teuthology.orchestra.run.smithi188:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-26T07:24:22.779 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-26T07:24:22.779 DEBUG:teuthology.orchestra.run.smithi143:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol24520/f57b3d1b-7328-4835-8e29-9827b7800bab/subvolume_file.2 bs=1M conv=fdatasync count=1 seek=0) 2023-11-26T07:24:22.856 INFO:teuthology.orchestra.run.smithi143.stderr:1+0 records in 2023-11-26T07:24:22.856 INFO:teuthology.orchestra.run.smithi143.stderr:1+0 records out 2023-11-26T07:24:22.856 INFO:teuthology.orchestra.run.smithi143.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0199178 s, 52.6 MB/s 2023-11-26T07:24:22.858 DEBUG:teuthology.orchestra.run.smithi143:> set -ex 2023-11-26T07:24:22.858 DEBUG:teuthology.orchestra.run.smithi143:> dd if=/proc/self/mounts of=/dev/stdout 2023-11-26T07:24:22.911 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-26T07:24:22.911 DEBUG:teuthology.orchestra.run.smithi143:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol24520/f57b3d1b-7328-4835-8e29-9827b7800bab/subvolume_file.3 bs=1M conv=fdatasync count=1 seek=0) 2023-11-26T07:24:22.990 INFO:teuthology.orchestra.run.smithi143.stderr:1+0 records in 2023-11-26T07:24:22.991 INFO:teuthology.orchestra.run.smithi143.stderr:1+0 records out 2023-11-26T07:24:22.991 INFO:teuthology.orchestra.run.smithi143.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0176828 s, 59.3 MB/s 2023-11-26T07:24:22.993 DEBUG:teuthology.orchestra.run.smithi143:> set -ex 2023-11-26T07:24:22.993 DEBUG:teuthology.orchestra.run.smithi143:> dd if=/proc/self/mounts of=/dev/stdout 2023-11-26T07:24:23.042 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-26T07:24:23.043 DEBUG:teuthology.orchestra.run.smithi143:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol24520/f57b3d1b-7328-4835-8e29-9827b7800bab/subvolume_file.4 bs=1M conv=fdatasync count=1 seek=0) 2023-11-26T07:24:23.119 INFO:teuthology.orchestra.run.smithi143.stderr:1+0 records in 2023-11-26T07:24:23.119 INFO:teuthology.orchestra.run.smithi143.stderr:1+0 records out 2023-11-26T07:24:23.119 INFO:teuthology.orchestra.run.smithi143.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0162766 s, 64.4 MB/s 2023-11-26T07:24:23.121 DEBUG:teuthology.orchestra.run.smithi143:> set -ex 2023-11-26T07:24:23.121 DEBUG:teuthology.orchestra.run.smithi143:> dd if=/proc/self/mounts of=/dev/stdout 2023-11-26T07:24:23.171 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-26T07:24:23.172 DEBUG:teuthology.orchestra.run.smithi143:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol24520/f57b3d1b-7328-4835-8e29-9827b7800bab/subvolume_file.5 bs=1M conv=fdatasync count=1 seek=0) 2023-11-26T07:24:23.245 INFO:teuthology.orchestra.run.smithi143.stderr:1+0 records in 2023-11-26T07:24:23.245 INFO:teuthology.orchestra.run.smithi143.stderr:1+0 records out 2023-11-26T07:24:23.245 INFO:teuthology.orchestra.run.smithi143.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.016247 s, 64.5 MB/s 2023-11-26T07:24:23.246 DEBUG:teuthology.orchestra.run.smithi143:> set -ex 2023-11-26T07:24:23.247 DEBUG:teuthology.orchestra.run.smithi143:> dd if=/proc/self/mounts of=/dev/stdout 2023-11-26T07:24:23.299 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-26T07:24:23.299 DEBUG:teuthology.orchestra.run.smithi143:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol24520/f57b3d1b-7328-4835-8e29-9827b7800bab/subvolume_file.6 bs=1M conv=fdatasync count=1 seek=0) 2023-11-26T07:24:23.373 INFO:teuthology.orchestra.run.smithi143.stderr:1+0 records in 2023-11-26T07:24:23.373 INFO:teuthology.orchestra.run.smithi143.stderr:1+0 records out 2023-11-26T07:24:23.373 INFO:teuthology.orchestra.run.smithi143.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0172601 s, 60.8 MB/s 2023-11-26T07:24:23.374 DEBUG:teuthology.orchestra.run.smithi143:> set -ex 2023-11-26T07:24:23.374 DEBUG:teuthology.orchestra.run.smithi143:> dd if=/proc/self/mounts of=/dev/stdout 2023-11-26T07:24:23.426 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-26T07:24:23.426 DEBUG:teuthology.orchestra.run.smithi143:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol24520/f57b3d1b-7328-4835-8e29-9827b7800bab/subvolume_file.7 bs=1M conv=fdatasync count=1 seek=0) 2023-11-26T07:24:23.502 INFO:teuthology.orchestra.run.smithi143.stderr:1+0 records in 2023-11-26T07:24:23.503 INFO:teuthology.orchestra.run.smithi143.stderr:1+0 records out 2023-11-26T07:24:23.503 INFO:teuthology.orchestra.run.smithi143.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0181311 s, 57.8 MB/s 2023-11-26T07:24:23.504 DEBUG:teuthology.orchestra.run.smithi143:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume snapshot create cephfs subvol24520 subvol_snap55360 2023-11-26T07:24:23.688 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:23.687+0000 7fe66c91d640 1 Processor -- start 2023-11-26T07:24:23.689 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:23.687+0000 7fe66c91d640 1 -- start start 2023-11-26T07:24:23.690 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:23.691+0000 7fe66c91d640 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7fe668143ac0 0x7fe668070370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:24:23.690 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:23.691+0000 7fe66c91d640 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7fe6680708b0 0x7fe6680cf280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:24:23.690 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:23.691+0000 7fe66c91d640 1 -- --> v1:172.21.15.143:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe6680cc730 con 0x7fe6680cf7c0 2023-11-26T07:24:23.690 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:23.691+0000 7fe66c91d640 1 -- --> v2:172.21.15.143:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe668143ea0 con 0x7fe668143ac0 2023-11-26T07:24:23.690 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:23.691+0000 7fe66c91d640 1 -- --> v2:172.21.15.188:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe6680cfbd0 con 0x7fe6680708b0 2023-11-26T07:24:23.690 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:23.691+0000 7fe66659c640 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7fe668143ac0 0x7fe668070370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:24:23.690 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:23.691+0000 7fe666d9d640 1 --1- >> v1:172.21.15.143:6789/0 conn(0x7fe6680cf7c0 0x7fe6680b1ef0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.143:6789/0 says I am v1:172.21.15.143:40292/0 (socket says 172.21.15.143:40292) 2023-11-26T07:24:23.690 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:23.691+0000 7fe666d9d640 1 -- 172.21.15.143:0/3223801306 learned_addr learned my addr 172.21.15.143:0/3223801306 (peer_addr_for_me v1:172.21.15.143:0/0) 2023-11-26T07:24:23.691 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:23.691+0000 7fe65dd9b640 1 --2- 172.21.15.143:0/3223801306 >> v2:172.21.15.188:3300/0 conn(0x7fe6680708b0 0x7fe6680cf280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:24:23.691 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:23.691+0000 7fe66659c640 1 -- 172.21.15.143:0/3223801306 >> v2:172.21.15.188:3300/0 conn(0x7fe6680708b0 msgr2=0x7fe6680cf280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-26T07:24:23.691 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:23.691+0000 7fe66659c640 1 --2- 172.21.15.143:0/3223801306 >> v2:172.21.15.188:3300/0 conn(0x7fe6680708b0 0x7fe6680cf280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:24:23.691 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:23.691+0000 7fe66659c640 1 -- 172.21.15.143:0/3223801306 >> v1:172.21.15.143:6789/0 conn(0x7fe6680cf7c0 legacy=0x7fe6680b1ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:24:23.691 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:23.691+0000 7fe66659c640 1 -- 172.21.15.143:0/3223801306 --> v2:172.21.15.143:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe668070ee0 con 0x7fe668143ac0 2023-11-26T07:24:23.691 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:23.691+0000 7fe665d9b640 1 -- 172.21.15.143:0/3223801306 <== mon.0 v1:172.21.15.143:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4145507537 0 0) 0x7fe6680cc730 con 0x7fe6680cf7c0 2023-11-26T07:24:23.694 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:23.695+0000 7fe66659c640 1 --2- 172.21.15.143:0/3223801306 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fe668143ac0 0x7fe668070370 secure :-1 s=READY pgs=1355 cs=0 l=1 rev1=1 crypto rx=0x7fe6680aeb10 tx=0x7fe65801c460 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bd9239c51fd7933c server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:24:23.694 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:23.695+0000 7fe665d9b640 1 -- 172.21.15.143:0/3223801306 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe6580ab600 con 0x7fe668143ac0 2023-11-26T07:24:23.694 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:23.695+0000 7fe665d9b640 1 -- 172.21.15.143:0/3223801306 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe6580a7050 con 0x7fe668143ac0 2023-11-26T07:24:23.694 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:23.695+0000 7fe665d9b640 1 -- 172.21.15.143:0/3223801306 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe6580ab980 con 0x7fe668143ac0 2023-11-26T07:24:23.695 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:23.695+0000 7fe66c91d640 1 -- 172.21.15.143:0/3223801306 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fe668143ac0 msgr2=0x7fe668070370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:24:23.695 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:23.695+0000 7fe66c91d640 1 --2- 172.21.15.143:0/3223801306 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fe668143ac0 0x7fe668070370 secure :-1 s=READY pgs=1355 cs=0 l=1 rev1=1 crypto rx=0x7fe6680aeb10 tx=0x7fe65801c460 comp rx=0 tx=0).stop 2023-11-26T07:24:23.695 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:23.695+0000 7fe66c91d640 1 -- 172.21.15.143:0/3223801306 shutdown_connections 2023-11-26T07:24:23.695 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:23.695+0000 7fe66c91d640 1 --2- 172.21.15.143:0/3223801306 >> v2:172.21.15.188:3300/0 conn(0x7fe6680708b0 0x7fe6680cf280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:24:23.695 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:23.695+0000 7fe66c91d640 1 --2- 172.21.15.143:0/3223801306 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fe668143ac0 0x7fe668070370 unknown :-1 s=CLOSED pgs=1355 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:24:23.695 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:23.695+0000 7fe66c91d640 1 -- 172.21.15.143:0/3223801306 >> 172.21.15.143:0/3223801306 conn(0x7fe66806a890 msgr2=0x7fe66806aca0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:24:23.695 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:23.695+0000 7fe66c91d640 1 -- 172.21.15.143:0/3223801306 shutdown_connections 2023-11-26T07:24:23.695 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:23.695+0000 7fe66c91d640 1 -- 172.21.15.143:0/3223801306 wait complete. 2023-11-26T07:24:23.696 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:23.695+0000 7fe66c91d640 1 Processor -- start 2023-11-26T07:24:23.696 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:23.695+0000 7fe66c91d640 1 -- start start 2023-11-26T07:24:23.698 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:23.699+0000 7fe66c91d640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fe6680708b0 0x7fe6680addb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:24:23.698 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:23.699+0000 7fe66c91d640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fe6680cf7c0 0x7fe668159a60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:24:23.698 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:23.699+0000 7fe66c91d640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fe668143ac0 0x7fe66813e0e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:24:23.698 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:23.699+0000 7fe66c91d640 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe668159fa0 con 0x7fe668143ac0 2023-11-26T07:24:23.698 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:23.699+0000 7fe66c91d640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe66813e900 con 0x7fe6680cf7c0 2023-11-26T07:24:23.698 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:23.699+0000 7fe66c91d640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe66813ebe0 con 0x7fe6680708b0 2023-11-26T07:24:23.698 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:23.699+0000 7fe666d9d640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fe668143ac0 0x7fe66813e0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:24:23.699 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:23.699+0000 7fe66659c640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fe6680708b0 0x7fe6680addb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:24:23.699 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:23.699+0000 7fe666d9d640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fe668143ac0 0x7fe66813e0e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:54688/0 (socket says 172.21.15.143:54688) 2023-11-26T07:24:23.699 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:23.699+0000 7fe666d9d640 1 -- 172.21.15.143:0/2006661108 learned_addr learned my addr 172.21.15.143:0/2006661108 (peer_addr_for_me v2:172.21.15.143:0/0) 2023-11-26T07:24:23.699 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:23.699+0000 7fe65dd9b640 1 --2- 172.21.15.143:0/2006661108 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fe6680cf7c0 0x7fe668159a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:24:23.700 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:23.699+0000 7fe666d9d640 1 -- 172.21.15.143:0/2006661108 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fe6680708b0 msgr2=0x7fe6680addb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:24:23.700 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:23.699+0000 7fe666d9d640 1 --2- 172.21.15.143:0/2006661108 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fe6680708b0 0x7fe6680addb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:24:23.700 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:23.699+0000 7fe666d9d640 1 -- 172.21.15.143:0/2006661108 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fe6680cf7c0 msgr2=0x7fe668159a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:24:23.700 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:23.699+0000 7fe666d9d640 1 --2- 172.21.15.143:0/2006661108 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fe6680cf7c0 0x7fe668159a60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:24:23.700 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:23.699+0000 7fe666d9d640 1 -- 172.21.15.143:0/2006661108 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe658009420 con 0x7fe668143ac0 2023-11-26T07:24:23.700 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:23.699+0000 7fe666d9d640 1 --2- 172.21.15.143:0/2006661108 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fe668143ac0 0x7fe66813e0e0 secure :-1 s=READY pgs=1356 cs=0 l=1 rev1=1 crypto rx=0x7fe66000c480 tx=0x7fe66000c940 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:24:23.700 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:23.699+0000 7fe65f7fe640 1 -- 172.21.15.143:0/2006661108 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe6600074e0 con 0x7fe668143ac0 2023-11-26T07:24:23.700 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:23.699+0000 7fe65f7fe640 1 -- 172.21.15.143:0/2006661108 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe660007640 con 0x7fe668143ac0 2023-11-26T07:24:23.700 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:23.699+0000 7fe66c91d640 1 -- 172.21.15.143:0/2006661108 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe66813eec0 con 0x7fe668143ac0 2023-11-26T07:24:23.700 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:23.699+0000 7fe65f7fe640 1 -- 172.21.15.143:0/2006661108 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe660004a10 con 0x7fe668143ac0 2023-11-26T07:24:23.701 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:23.699+0000 7fe66c91d640 1 -- 172.21.15.143:0/2006661108 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe66813e620 con 0x7fe668143ac0 2023-11-26T07:24:23.701 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:23.699+0000 7fe66c91d640 1 -- 172.21.15.143:0/2006661108 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe6680ae2f0 con 0x7fe668143ac0 2023-11-26T07:24:23.701 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:23.699+0000 7fe65f7fe640 1 -- 172.21.15.143:0/2006661108 <== mon.0 v2:172.21.15.143:3300/0 4 ==== mgrmap(e 17) v1 ==== 86005+0+0 (secure 0 0 0) 0x7fe660004b70 con 0x7fe668143ac0 2023-11-26T07:24:23.702 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:23.703+0000 7fe65f7fe640 1 --2- 172.21.15.143:0/2006661108 >> [v2:172.21.15.188:6836/3401128553,v1:172.21.15.188:6837/3401128553] conn(0x7fe648066650 0x7fe648068b10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:24:23.702 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:23.703+0000 7fe65f7fe640 1 -- 172.21.15.143:0/2006661108 <== mon.0 v2:172.21.15.143:3300/0 5 ==== osd_map(103..103 src has 1..103) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fe6600834e0 con 0x7fe668143ac0 2023-11-26T07:24:23.702 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:23.703+0000 7fe66659c640 1 --2- 172.21.15.143:0/2006661108 >> [v2:172.21.15.188:6836/3401128553,v1:172.21.15.188:6837/3401128553] conn(0x7fe648066650 0x7fe648068b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:24:23.703 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:23.703+0000 7fe66659c640 1 --2- 172.21.15.143:0/2006661108 >> [v2:172.21.15.188:6836/3401128553,v1:172.21.15.188:6837/3401128553] conn(0x7fe648066650 0x7fe648068b10 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7fe6580ad000 tx=0x7fe6580a6000 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:24:23.706 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:23.707+0000 7fe65f7fe640 1 -- 172.21.15.143:0/2006661108 <== mon.0 v2:172.21.15.143:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172649 (secure 0 0 0) 0x7fe660050830 con 0x7fe668143ac0 2023-11-26T07:24:23.881 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:23.879+0000 7fe66c91d640 1 -- 172.21.15.143:0/2006661108 --> [v2:172.21.15.188:6836/3401128553,v1:172.21.15.188:6837/3401128553] -- mgr_command(tid 0: {"prefix": "fs subvolume snapshot create", "vol_name": "cephfs", "sub_name": "subvol24520", "snap_name": "subvol_snap55360", "target": ["mon-mgr", ""]}) v1 -- 0x7fe668071c30 con 0x7fe648066650 2023-11-26T07:24:23.905 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:23.903+0000 7fe65f7fe640 1 -- 172.21.15.143:0/2006661108 <== mgr.4108 v2:172.21.15.188:6836/3401128553 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+0 (secure 0 0 0) 0x7fe6580bb070 con 0x7fe648066650 2023-11-26T07:24:23.907 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:23.907+0000 7fe66c91d640 1 -- 172.21.15.143:0/2006661108 >> [v2:172.21.15.188:6836/3401128553,v1:172.21.15.188:6837/3401128553] conn(0x7fe648066650 msgr2=0x7fe648068b10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:24:23.908 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:23.907+0000 7fe66c91d640 1 --2- 172.21.15.143:0/2006661108 >> [v2:172.21.15.188:6836/3401128553,v1:172.21.15.188:6837/3401128553] conn(0x7fe648066650 0x7fe648068b10 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7fe6580ad000 tx=0x7fe6580a6000 comp rx=0 tx=0).stop 2023-11-26T07:24:23.908 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:23.907+0000 7fe66c91d640 1 -- 172.21.15.143:0/2006661108 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fe668143ac0 msgr2=0x7fe66813e0e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:24:23.908 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:23.907+0000 7fe66c91d640 1 --2- 172.21.15.143:0/2006661108 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fe668143ac0 0x7fe66813e0e0 secure :-1 s=READY pgs=1356 cs=0 l=1 rev1=1 crypto rx=0x7fe66000c480 tx=0x7fe66000c940 comp rx=0 tx=0).stop 2023-11-26T07:24:23.908 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:23.907+0000 7fe66c91d640 1 -- 172.21.15.143:0/2006661108 shutdown_connections 2023-11-26T07:24:23.908 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:23.907+0000 7fe66c91d640 1 --2- 172.21.15.143:0/2006661108 >> [v2:172.21.15.188:6836/3401128553,v1:172.21.15.188:6837/3401128553] conn(0x7fe648066650 0x7fe648068b10 unknown :-1 s=CLOSED pgs=146 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:24:23.908 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:23.907+0000 7fe66c91d640 1 --2- 172.21.15.143:0/2006661108 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fe668143ac0 0x7fe66813e0e0 unknown :-1 s=CLOSED pgs=1356 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:24:23.908 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:23.907+0000 7fe66c91d640 1 --2- 172.21.15.143:0/2006661108 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fe6680cf7c0 0x7fe668159a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:24:23.908 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:23.907+0000 7fe66c91d640 1 --2- 172.21.15.143:0/2006661108 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fe6680708b0 0x7fe6680addb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:24:23.908 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:23.907+0000 7fe66c91d640 1 -- 172.21.15.143:0/2006661108 >> 172.21.15.143:0/2006661108 conn(0x7fe66806a890 msgr2=0x7fe6680b1410 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:24:23.908 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:23.907+0000 7fe66c91d640 1 -- 172.21.15.143:0/2006661108 shutdown_connections 2023-11-26T07:24:23.908 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:23.907+0000 7fe66c91d640 1 -- 172.21.15.143:0/2006661108 wait complete. 2023-11-26T07:24:23.926 DEBUG:teuthology.orchestra.run.smithi143:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume snapshot clone cephfs subvol24520 subvol_snap55360 subvol_clone30650 2023-11-26T07:24:24.072 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:24.071+0000 7fbf2909a640 1 Processor -- start 2023-11-26T07:24:24.072 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:24.071+0000 7fbf2909a640 1 -- start start 2023-11-26T07:24:24.074 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:24.075+0000 7fbf2909a640 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7fbf2413e290 0x7fbf240cf280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:24:24.075 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:24.075+0000 7fbf2909a640 1 -- --> v1:172.21.15.143:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbf2415fb20 con 0x7fbf240cf7c0 2023-11-26T07:24:24.075 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:24.075+0000 7fbf2909a640 1 -- --> v1:172.21.15.188:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbf2406c520 con 0x7fbf2415f6b0 2023-11-26T07:24:24.075 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:24.075+0000 7fbf2909a640 1 -- --> v2:172.21.15.188:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fbf240cfc40 con 0x7fbf2413e290 2023-11-26T07:24:24.075 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:24.075+0000 7fbf1bfff640 1 --1- >> v1:172.21.15.143:6789/0 conn(0x7fbf240cf7c0 0x7fbf2415efa0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.143:6789/0 says I am v1:172.21.15.143:40298/0 (socket says 172.21.15.143:40298) 2023-11-26T07:24:24.075 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:24.075+0000 7fbf1bfff640 1 -- 172.21.15.143:0/2204943523 learned_addr learned my addr 172.21.15.143:0/2204943523 (peer_addr_for_me v1:172.21.15.143:0/0) 2023-11-26T07:24:24.075 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:24.075+0000 7fbf22d9d640 1 --2- 172.21.15.143:0/2204943523 >> v2:172.21.15.188:3300/0 conn(0x7fbf2413e290 0x7fbf240cf280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:24:24.075 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:24.075+0000 7fbf2259c640 1 -- 172.21.15.143:0/2204943523 <== mon.0 v1:172.21.15.143:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1068810463 0 0) 0x7fbf0c001120 con 0x7fbf240cf7c0 2023-11-26T07:24:24.075 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:24.075+0000 7fbf2259c640 1 -- 172.21.15.143:0/2204943523 --> v1:172.21.15.143:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fbf10003530 con 0x7fbf240cf7c0 2023-11-26T07:24:24.075 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:24.075+0000 7fbf2259c640 1 -- 172.21.15.143:0/2204943523 <== mon.1 v1:172.21.15.188:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2612425145 0 0) 0x7fbf2406c520 con 0x7fbf2415f6b0 2023-11-26T07:24:24.075 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:24.075+0000 7fbf2259c640 1 -- 172.21.15.143:0/2204943523 --> v1:172.21.15.188:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fbf0c001120 con 0x7fbf2415f6b0 2023-11-26T07:24:24.075 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:24.075+0000 7fbf2259c640 1 -- 172.21.15.143:0/2204943523 <== mon.0 v1:172.21.15.143:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1164371081 0 0) 0x7fbf10003530 con 0x7fbf240cf7c0 2023-11-26T07:24:24.075 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:24.075+0000 7fbf22d9d640 1 -- 172.21.15.143:0/2204943523 >> v1:172.21.15.188:6789/0 conn(0x7fbf2415f6b0 legacy=0x7fbf240cd7b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:24:24.076 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:24.075+0000 7fbf22d9d640 1 -- 172.21.15.143:0/2204943523 >> v1:172.21.15.143:6789/0 conn(0x7fbf240cf7c0 legacy=0x7fbf2415efa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:24:24.076 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:24.075+0000 7fbf22d9d640 1 -- 172.21.15.143:0/2204943523 --> v2:172.21.15.188:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbf2406c800 con 0x7fbf2413e290 2023-11-26T07:24:24.078 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:24.079+0000 7fbf22d9d640 1 --2- 172.21.15.143:0/2204943523 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fbf2413e290 0x7fbf240cf280 secure :-1 s=READY pgs=509 cs=0 l=1 rev1=1 crypto rx=0x7fbf140097c0 tx=0x7fbf1401c370 comp rx=0 tx=0).ready entity=mon.1 client_cookie=e2af733f4eb447de server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:24:24.078 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:24.079+0000 7fbf2259c640 1 -- 172.21.15.143:0/2204943523 <== mon.1 v2:172.21.15.188:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbf0c001830 con 0x7fbf2413e290 2023-11-26T07:24:24.078 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:24.079+0000 7fbf2259c640 1 -- 172.21.15.143:0/2204943523 <== mon.1 v2:172.21.15.188:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbf140a7050 con 0x7fbf2413e290 2023-11-26T07:24:24.078 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:24.079+0000 7fbf2259c640 1 -- 172.21.15.143:0/2204943523 <== mon.1 v2:172.21.15.188:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbf140ab620 con 0x7fbf2413e290 2023-11-26T07:24:24.079 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:24.079+0000 7fbf2909a640 1 -- 172.21.15.143:0/2204943523 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fbf2413e290 msgr2=0x7fbf240cf280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:24:24.079 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:24.079+0000 7fbf2909a640 1 --2- 172.21.15.143:0/2204943523 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fbf2413e290 0x7fbf240cf280 secure :-1 s=READY pgs=509 cs=0 l=1 rev1=1 crypto rx=0x7fbf140097c0 tx=0x7fbf1401c370 comp rx=0 tx=0).stop 2023-11-26T07:24:24.079 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:24.079+0000 7fbf2909a640 1 -- 172.21.15.143:0/2204943523 shutdown_connections 2023-11-26T07:24:24.079 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:24.079+0000 7fbf2909a640 1 --2- 172.21.15.143:0/2204943523 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fbf2413e290 0x7fbf240cf280 unknown :-1 s=CLOSED pgs=509 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:24:24.079 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:24.079+0000 7fbf2909a640 1 -- 172.21.15.143:0/2204943523 >> 172.21.15.143:0/2204943523 conn(0x7fbf2406a890 msgr2=0x7fbf2406aca0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:24:24.079 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:24.079+0000 7fbf2909a640 1 -- 172.21.15.143:0/2204943523 shutdown_connections 2023-11-26T07:24:24.079 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:24.079+0000 7fbf2909a640 1 -- 172.21.15.143:0/2204943523 wait complete. 2023-11-26T07:24:24.080 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:24.079+0000 7fbf2909a640 1 Processor -- start 2023-11-26T07:24:24.080 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:24.079+0000 7fbf2909a640 1 -- start start 2023-11-26T07:24:24.082 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:24.079+0000 7fbf2909a640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fbf240cf7c0 0x7fbf2413b160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:24:24.082 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:24.079+0000 7fbf2909a640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fbf2413e290 0x7fbf2413a880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:24:24.082 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:24.083+0000 7fbf2909a640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fbf2415f6b0 0x7fbf240ab7e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:24:24.082 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:24.083+0000 7fbf2909a640 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbf240a9370 con 0x7fbf240cf7c0 2023-11-26T07:24:24.083 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:24.083+0000 7fbf2909a640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbf240adfa0 con 0x7fbf2413e290 2023-11-26T07:24:24.083 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:24.083+0000 7fbf2909a640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbf240a95e0 con 0x7fbf2415f6b0 2023-11-26T07:24:24.083 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:24.083+0000 7fbf22d9d640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fbf240cf7c0 0x7fbf2413b160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:24:24.083 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:24.083+0000 7fbf22d9d640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fbf240cf7c0 0x7fbf2413b160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:54694/0 (socket says 172.21.15.143:54694) 2023-11-26T07:24:24.083 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:24.083+0000 7fbf22d9d640 1 -- 172.21.15.143:0/3879661386 learned_addr learned my addr 172.21.15.143:0/3879661386 (peer_addr_for_me v2:172.21.15.143:0/0) 2023-11-26T07:24:24.083 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:24.083+0000 7fbf22d9d640 1 -- 172.21.15.143:0/3879661386 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fbf2415f6b0 msgr2=0x7fbf240ab7e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:24:24.083 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:24.083+0000 7fbf22d9d640 1 --2- 172.21.15.143:0/3879661386 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fbf2415f6b0 0x7fbf240ab7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:24:24.083 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:24.083+0000 7fbf22d9d640 1 -- 172.21.15.143:0/3879661386 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fbf2413e290 msgr2=0x7fbf2413a880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:24:24.083 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:24.083+0000 7fbf22d9d640 1 --2- 172.21.15.143:0/3879661386 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fbf2413e290 0x7fbf2413a880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:24:24.083 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:24.083+0000 7fbf22d9d640 1 -- 172.21.15.143:0/3879661386 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbf14009420 con 0x7fbf240cf7c0 2023-11-26T07:24:24.083 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:24.083+0000 7fbf22d9d640 1 --2- 172.21.15.143:0/3879661386 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fbf240cf7c0 0x7fbf2413b160 secure :-1 s=READY pgs=1358 cs=0 l=1 rev1=1 crypto rx=0x7fbf140097c0 tx=0x7fbf1401abb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:24:24.084 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:24.083+0000 7fbf1b7fe640 1 -- 172.21.15.143:0/3879661386 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbf140abc50 con 0x7fbf240cf7c0 2023-11-26T07:24:24.084 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:24.083+0000 7fbf1b7fe640 1 -- 172.21.15.143:0/3879661386 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbf140abdb0 con 0x7fbf240cf7c0 2023-11-26T07:24:24.084 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:24.083+0000 7fbf2909a640 1 -- 172.21.15.143:0/3879661386 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbf240adc70 con 0x7fbf240cf7c0 2023-11-26T07:24:24.084 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:24.083+0000 7fbf1b7fe640 1 -- 172.21.15.143:0/3879661386 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbf14004260 con 0x7fbf240cf7c0 2023-11-26T07:24:24.084 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:24.083+0000 7fbf2909a640 1 -- 172.21.15.143:0/3879661386 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbf2413b880 con 0x7fbf240cf7c0 2023-11-26T07:24:24.084 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:24.083+0000 7fbf2909a640 1 -- 172.21.15.143:0/3879661386 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbf24067680 con 0x7fbf240cf7c0 2023-11-26T07:24:24.085 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:24.083+0000 7fbf1b7fe640 1 -- 172.21.15.143:0/3879661386 <== mon.0 v2:172.21.15.143:3300/0 4 ==== mgrmap(e 17) v1 ==== 86005+0+0 (secure 0 0 0) 0x7fbf140043c0 con 0x7fbf240cf7c0 2023-11-26T07:24:24.086 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:24.087+0000 7fbf1b7fe640 1 --2- 172.21.15.143:0/3879661386 >> [v2:172.21.15.188:6836/3401128553,v1:172.21.15.188:6837/3401128553] conn(0x7fbf04066620 0x7fbf04068ae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:24:24.086 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:24.087+0000 7fbf1b7fe640 1 -- 172.21.15.143:0/3879661386 <== mon.0 v2:172.21.15.143:3300/0 5 ==== osd_map(103..103 src has 1..103) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fbf1411e980 con 0x7fbf240cf7c0 2023-11-26T07:24:24.087 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:24.087+0000 7fbf1bfff640 1 --2- 172.21.15.143:0/3879661386 >> [v2:172.21.15.188:6836/3401128553,v1:172.21.15.188:6837/3401128553] conn(0x7fbf04066620 0x7fbf04068ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:24:24.087 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:24.087+0000 7fbf1bfff640 1 --2- 172.21.15.143:0/3879661386 >> [v2:172.21.15.188:6836/3401128553,v1:172.21.15.188:6837/3401128553] conn(0x7fbf04066620 0x7fbf04068ae0 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7fbf0c001500 tx=0x7fbf0c001b40 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:24:24.090 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:24.091+0000 7fbf1b7fe640 1 -- 172.21.15.143:0/3879661386 <== mon.0 v2:172.21.15.143:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172649 (secure 0 0 0) 0x7fbf140ebc20 con 0x7fbf240cf7c0 2023-11-26T07:24:24.267 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:24.267+0000 7fbf2909a640 1 -- 172.21.15.143:0/3879661386 --> [v2:172.21.15.188:6836/3401128553,v1:172.21.15.188:6837/3401128553] -- mgr_command(tid 0: {"prefix": "fs subvolume snapshot clone", "vol_name": "cephfs", "sub_name": "subvol24520", "snap_name": "subvol_snap55360", "target_sub_name": "subvol_clone30650", "target": ["mon-mgr", ""]}) v1 -- 0x7fbf2406df50 con 0x7fbf04066620 2023-11-26T07:24:24.338 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:24.339+0000 7fbf1b7fe640 1 -- 172.21.15.143:0/3879661386 <== mgr.4108 v2:172.21.15.188:6836/3401128553 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+0 (secure 0 0 0) 0x7fbf2406df50 con 0x7fbf04066620 2023-11-26T07:24:24.340 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:24.339+0000 7fbf2909a640 1 -- 172.21.15.143:0/3879661386 >> [v2:172.21.15.188:6836/3401128553,v1:172.21.15.188:6837/3401128553] conn(0x7fbf04066620 msgr2=0x7fbf04068ae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:24:24.340 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:24.339+0000 7fbf2909a640 1 --2- 172.21.15.143:0/3879661386 >> [v2:172.21.15.188:6836/3401128553,v1:172.21.15.188:6837/3401128553] conn(0x7fbf04066620 0x7fbf04068ae0 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7fbf0c001500 tx=0x7fbf0c001b40 comp rx=0 tx=0).stop 2023-11-26T07:24:24.340 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:24.339+0000 7fbf2909a640 1 -- 172.21.15.143:0/3879661386 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fbf240cf7c0 msgr2=0x7fbf2413b160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:24:24.341 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:24.339+0000 7fbf2909a640 1 --2- 172.21.15.143:0/3879661386 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fbf240cf7c0 0x7fbf2413b160 secure :-1 s=READY pgs=1358 cs=0 l=1 rev1=1 crypto rx=0x7fbf140097c0 tx=0x7fbf1401abb0 comp rx=0 tx=0).stop 2023-11-26T07:24:24.341 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:24.339+0000 7fbf2909a640 1 -- 172.21.15.143:0/3879661386 shutdown_connections 2023-11-26T07:24:24.341 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:24.339+0000 7fbf2909a640 1 --2- 172.21.15.143:0/3879661386 >> [v2:172.21.15.188:6836/3401128553,v1:172.21.15.188:6837/3401128553] conn(0x7fbf04066620 0x7fbf04068ae0 unknown :-1 s=CLOSED pgs=147 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:24:24.341 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:24.339+0000 7fbf2909a640 1 --2- 172.21.15.143:0/3879661386 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fbf2415f6b0 0x7fbf240ab7e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:24:24.341 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:24.339+0000 7fbf2909a640 1 --2- 172.21.15.143:0/3879661386 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fbf2413e290 0x7fbf2413a880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:24:24.341 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:24.339+0000 7fbf2909a640 1 --2- 172.21.15.143:0/3879661386 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fbf240cf7c0 0x7fbf2413b160 unknown :-1 s=CLOSED pgs=1358 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:24:24.341 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:24.339+0000 7fbf2909a640 1 -- 172.21.15.143:0/3879661386 >> 172.21.15.143:0/3879661386 conn(0x7fbf2406a890 msgr2=0x7fbf24071680 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:24:24.341 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:24.339+0000 7fbf2909a640 1 -- 172.21.15.143:0/3879661386 shutdown_connections 2023-11-26T07:24:24.341 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:24.339+0000 7fbf2909a640 1 -- 172.21.15.143:0/3879661386 wait complete. 2023-11-26T07:24:24.344 INFO:tasks.ceph.mgr.x.smithi188.stderr:2023-11-26T07:24:24.341+0000 7f04b338e640 -1 client.0 error registering admin socket command: (17) File exists 2023-11-26T07:24:24.344 INFO:tasks.ceph.mgr.x.smithi188.stderr:2023-11-26T07:24:24.341+0000 7f04b338e640 -1 client.0 error registering admin socket command: (17) File exists 2023-11-26T07:24:24.344 INFO:tasks.ceph.mgr.x.smithi188.stderr:2023-11-26T07:24:24.341+0000 7f04b338e640 -1 client.0 error registering admin socket command: (17) File exists 2023-11-26T07:24:24.344 INFO:tasks.ceph.mgr.x.smithi188.stderr:2023-11-26T07:24:24.341+0000 7f04b338e640 -1 client.0 error registering admin socket command: (17) File exists 2023-11-26T07:24:24.344 INFO:tasks.ceph.mgr.x.smithi188.stderr:2023-11-26T07:24:24.341+0000 7f04b338e640 -1 client.0 error registering admin socket command: (17) File exists 2023-11-26T07:24:24.362 INFO:tasks.ceph.mgr.x.smithi188.stderr:2023-11-26T07:24:24.357+0000 7f04b2b8d640 -1 client.0 error registering admin socket command: (17) File exists 2023-11-26T07:24:24.362 INFO:tasks.ceph.mgr.x.smithi188.stderr:2023-11-26T07:24:24.357+0000 7f04b2b8d640 -1 client.0 error registering admin socket command: (17) File exists 2023-11-26T07:24:24.362 INFO:tasks.ceph.mgr.x.smithi188.stderr:2023-11-26T07:24:24.357+0000 7f04b2b8d640 -1 client.0 error registering admin socket command: (17) File exists 2023-11-26T07:24:24.362 INFO:tasks.ceph.mgr.x.smithi188.stderr:2023-11-26T07:24:24.357+0000 7f04b2b8d640 -1 client.0 error registering admin socket command: (17) File exists 2023-11-26T07:24:24.362 INFO:tasks.ceph.mgr.x.smithi188.stderr:2023-11-26T07:24:24.357+0000 7f04b2b8d640 -1 client.0 error registering admin socket command: (17) File exists 2023-11-26T07:24:24.363 DEBUG:teuthology.orchestra.run.smithi143:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs clone status cephfs subvol_clone30650 2023-11-26T07:24:24.532 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:24.531+0000 7feb54cc8640 1 Processor -- start 2023-11-26T07:24:24.532 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:24.531+0000 7feb54cc8640 1 -- start start 2023-11-26T07:24:24.534 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:24.535+0000 7feb54cc8640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7feb500d01b0 0x7feb500d3be0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:24:24.534 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:24.535+0000 7feb54cc8640 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7feb5015d610 0x7feb500cdc40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:24:24.534 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:24.535+0000 7feb54cc8640 1 -- --> v1:172.21.15.143:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7feb500ce180 con 0x7feb5015d230 2023-11-26T07:24:24.534 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:24.535+0000 7feb54cc8640 1 -- --> v2:172.21.15.188:3300/0 -- mon_getmap magic: 0 v1 -- 0x7feb500ce3b0 con 0x7feb5015d610 2023-11-26T07:24:24.534 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:24.535+0000 7feb54cc8640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7feb500ce690 con 0x7feb500d01b0 2023-11-26T07:24:24.534 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:24.535+0000 7feb4effd640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7feb500d01b0 0x7feb500d3be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:24:24.535 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:24.535+0000 7feb4ffff640 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7feb5015d610 0x7feb500cdc40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:24:24.535 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:24.535+0000 7feb4effd640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7feb500d01b0 0x7feb500d3be0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.188:3301/0 says I am v2:172.21.15.143:40704/0 (socket says 172.21.15.143:40704) 2023-11-26T07:24:24.535 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:24.535+0000 7feb4effd640 1 -- 172.21.15.143:0/351952915 learned_addr learned my addr 172.21.15.143:0/351952915 (peer_addr_for_me v2:172.21.15.143:0/0) 2023-11-26T07:24:24.535 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:24.535+0000 7feb4f7fe640 1 --1- 172.21.15.143:0/351952915 >> v1:172.21.15.143:6789/0 conn(0x7feb5015d230 0x7feb500cfaa0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.143:6789/0 says I am v1:172.21.15.143:39362/0 (socket says 172.21.15.143:39362) 2023-11-26T07:24:24.535 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:24.535+0000 7feb4e7fc640 1 -- 172.21.15.143:0/351952915 <== mon.0 v1:172.21.15.143:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 189383993 0 0) 0x7feb500ce180 con 0x7feb5015d230 2023-11-26T07:24:24.536 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:24.535+0000 7feb4e7fc640 1 -- 172.21.15.143:0/351952915 --> v1:172.21.15.143:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7feb38003530 con 0x7feb5015d230 2023-11-26T07:24:24.536 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:24.535+0000 7feb4effd640 1 -- 172.21.15.143:0/351952915 >> v2:172.21.15.188:3300/0 conn(0x7feb5015d610 msgr2=0x7feb500cdc40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-26T07:24:24.536 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:24.535+0000 7feb4effd640 1 --2- 172.21.15.143:0/351952915 >> v2:172.21.15.188:3300/0 conn(0x7feb5015d610 0x7feb500cdc40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:24:24.536 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:24.535+0000 7feb4effd640 1 -- 172.21.15.143:0/351952915 >> v1:172.21.15.143:6789/0 conn(0x7feb5015d230 legacy=0x7feb500cfaa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:24:24.536 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:24.535+0000 7feb4effd640 1 -- 172.21.15.143:0/351952915 --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7feb500ce970 con 0x7feb500d01b0 2023-11-26T07:24:24.537 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:24.539+0000 7feb4effd640 1 --2- 172.21.15.143:0/351952915 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7feb500d01b0 0x7feb500d3be0 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7feb44009870 tx=0x7feb4401c4b0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=1c6922bc832adec8 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:24:24.538 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:24.539+0000 7feb4e7fc640 1 -- 172.21.15.143:0/351952915 <== mon.2 v2:172.21.15.188:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feb440aa610 con 0x7feb500d01b0 2023-11-26T07:24:24.538 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:24.539+0000 7feb4e7fc640 1 -- 172.21.15.143:0/351952915 <== mon.2 v2:172.21.15.188:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7feb440a6040 con 0x7feb500d01b0 2023-11-26T07:24:24.538 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:24.539+0000 7feb4e7fc640 1 -- 172.21.15.143:0/351952915 <== mon.2 v2:172.21.15.188:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feb440aa990 con 0x7feb500d01b0 2023-11-26T07:24:24.539 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:24.539+0000 7feb54cc8640 1 -- 172.21.15.143:0/351952915 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7feb500d01b0 msgr2=0x7feb500d3be0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:24:24.539 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:24.539+0000 7feb54cc8640 1 --2- 172.21.15.143:0/351952915 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7feb500d01b0 0x7feb500d3be0 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7feb44009870 tx=0x7feb4401c4b0 comp rx=0 tx=0).stop 2023-11-26T07:24:24.539 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:24.539+0000 7feb54cc8640 1 -- 172.21.15.143:0/351952915 shutdown_connections 2023-11-26T07:24:24.539 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:24.539+0000 7feb54cc8640 1 --2- 172.21.15.143:0/351952915 >> v2:172.21.15.188:3300/0 conn(0x7feb5015d610 0x7feb500cdc40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:24:24.539 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:24.539+0000 7feb54cc8640 1 --2- 172.21.15.143:0/351952915 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7feb500d01b0 0x7feb500d3be0 unknown :-1 s=CLOSED pgs=164 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:24:24.539 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:24.539+0000 7feb54cc8640 1 -- 172.21.15.143:0/351952915 >> 172.21.15.143:0/351952915 conn(0x7feb500d30c0 msgr2=0x7feb500d34d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:24:24.539 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:24.539+0000 7feb54cc8640 1 -- 172.21.15.143:0/351952915 shutdown_connections 2023-11-26T07:24:24.539 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:24.539+0000 7feb54cc8640 1 -- 172.21.15.143:0/351952915 wait complete. 2023-11-26T07:24:24.540 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:24.539+0000 7feb54cc8640 1 Processor -- start 2023-11-26T07:24:24.540 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:24.539+0000 7feb54cc8640 1 -- start start 2023-11-26T07:24:24.542 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:24.539+0000 7feb54cc8640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7feb500d01b0 0x7feb50153400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:24:24.542 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:24.539+0000 7feb54cc8640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7feb5015d230 0x7feb50153940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:24:24.542 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:24.539+0000 7feb54cc8640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7feb5015d610 0x7feb50156590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:24:24.542 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:24.539+0000 7feb54cc8640 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x7feb5015e790 con 0x7feb500d01b0 2023-11-26T07:24:24.542 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:24.539+0000 7feb54cc8640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7feb5015ea70 con 0x7feb5015d230 2023-11-26T07:24:24.542 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:24.539+0000 7feb54cc8640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7feb5015ed50 con 0x7feb5015d610 2023-11-26T07:24:24.542 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:24.543+0000 7feb4f7fe640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7feb500d01b0 0x7feb50153400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:24:24.542 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:24.543+0000 7feb4effd640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7feb5015d230 0x7feb50153940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:24:24.542 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:24.543+0000 7feb4effd640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7feb5015d230 0x7feb50153940 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.188:3300/0 says I am v2:172.21.15.143:52050/0 (socket says 172.21.15.143:52050) 2023-11-26T07:24:24.543 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:24.543+0000 7feb4f7fe640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7feb500d01b0 0x7feb50153400 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:38478/0 (socket says 172.21.15.143:38478) 2023-11-26T07:24:24.543 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:24.543+0000 7feb4effd640 1 -- 172.21.15.143:0/338403992 learned_addr learned my addr 172.21.15.143:0/338403992 (peer_addr_for_me v2:172.21.15.143:0/0) 2023-11-26T07:24:24.543 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:24.543+0000 7feb4ffff640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7feb5015d610 0x7feb50156590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:24:24.544 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:24.543+0000 7feb4f7fe640 1 -- 172.21.15.143:0/338403992 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7feb5015d610 msgr2=0x7feb50156590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:24:24.544 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:24.543+0000 7feb4f7fe640 1 --2- 172.21.15.143:0/338403992 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7feb5015d610 0x7feb50156590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:24:24.544 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:24.543+0000 7feb4f7fe640 1 -- 172.21.15.143:0/338403992 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7feb5015d230 msgr2=0x7feb50153940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:24:24.544 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:24.543+0000 7feb4f7fe640 1 --2- 172.21.15.143:0/338403992 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7feb5015d230 0x7feb50153940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:24:24.544 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:24.543+0000 7feb4f7fe640 1 -- 172.21.15.143:0/338403992 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7feb440094d0 con 0x7feb500d01b0 2023-11-26T07:24:24.544 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:24.543+0000 7feb4ffff640 1 --2- 172.21.15.143:0/338403992 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7feb5015d610 0x7feb50156590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-26T07:24:24.544 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:24.543+0000 7feb4effd640 1 --2- 172.21.15.143:0/338403992 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7feb5015d230 0x7feb50153940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-26T07:24:24.544 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:24.543+0000 7feb4f7fe640 1 --2- 172.21.15.143:0/338403992 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7feb500d01b0 0x7feb50153400 secure :-1 s=READY pgs=1361 cs=0 l=1 rev1=1 crypto rx=0x7feb4000c400 tx=0x7feb4000c8c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:24:24.544 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:24.543+0000 7feb37fff640 1 -- 172.21.15.143:0/338403992 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feb400074e0 con 0x7feb500d01b0 2023-11-26T07:24:24.545 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:24.543+0000 7feb37fff640 1 -- 172.21.15.143:0/338403992 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7feb40007660 con 0x7feb500d01b0 2023-11-26T07:24:24.545 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:24.543+0000 7feb54cc8640 1 -- 172.21.15.143:0/338403992 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7feb500b4a20 con 0x7feb500d01b0 2023-11-26T07:24:24.545 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:24.543+0000 7feb37fff640 1 -- 172.21.15.143:0/338403992 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feb40004a30 con 0x7feb500d01b0 2023-11-26T07:24:24.545 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:24.543+0000 7feb54cc8640 1 -- 172.21.15.143:0/338403992 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7feb5015e460 con 0x7feb500d01b0 2023-11-26T07:24:24.545 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:24.543+0000 7feb54cc8640 1 -- 172.21.15.143:0/338403992 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7feb50067c10 con 0x7feb500d01b0 2023-11-26T07:24:24.545 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:24.547+0000 7feb37fff640 1 -- 172.21.15.143:0/338403992 <== mon.0 v2:172.21.15.143:3300/0 4 ==== mgrmap(e 17) v1 ==== 86005+0+0 (secure 0 0 0) 0x7feb40004b90 con 0x7feb500d01b0 2023-11-26T07:24:24.546 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:24.547+0000 7feb37fff640 1 --2- 172.21.15.143:0/338403992 >> [v2:172.21.15.188:6836/3401128553,v1:172.21.15.188:6837/3401128553] conn(0x7feb3c066620 0x7feb3c068ae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:24:24.546 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:24.547+0000 7feb37fff640 1 -- 172.21.15.143:0/338403992 <== mon.0 v2:172.21.15.143:3300/0 5 ==== osd_map(103..103 src has 1..103) v4 ==== 5709+0+0 (secure 0 0 0) 0x7feb40083b70 con 0x7feb500d01b0 2023-11-26T07:24:24.547 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:24.547+0000 7feb4effd640 1 --2- 172.21.15.143:0/338403992 >> [v2:172.21.15.188:6836/3401128553,v1:172.21.15.188:6837/3401128553] conn(0x7feb3c066620 0x7feb3c068ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:24:24.547 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:24.547+0000 7feb4effd640 1 --2- 172.21.15.143:0/338403992 >> [v2:172.21.15.188:6836/3401128553,v1:172.21.15.188:6837/3401128553] conn(0x7feb3c066620 0x7feb3c068ae0 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7feb4401c4f0 tx=0x7feb440b5040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:24:24.550 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:24.551+0000 7feb37fff640 1 -- 172.21.15.143:0/338403992 <== mon.0 v2:172.21.15.143:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172649 (secure 0 0 0) 0x7feb40050da0 con 0x7feb500d01b0 2023-11-26T07:24:24.749 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:24.747+0000 7feb54cc8640 1 -- 172.21.15.143:0/338403992 --> [v2:172.21.15.188:6836/3401128553,v1:172.21.15.188:6837/3401128553] -- mgr_command(tid 0: {"prefix": "fs clone status", "vol_name": "cephfs", "clone_name": "subvol_clone30650", "target": ["mon-mgr", ""]}) v1 -- 0x7feb5015ebe0 con 0x7feb3c066620 2023-11-26T07:24:26.597 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:26.595+0000 7feb37fff640 1 -- 172.21.15.143:0/338403992 <== mon.0 v2:172.21.15.143:3300/0 7 ==== mgrmap(e 18) v1 ==== 86107+0+0 (secure 0 0 0) 0x7feb40051170 con 0x7feb500d01b0 2023-11-26T07:24:28.061 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:28.059+0000 7feb37fff640 1 -- 172.21.15.143:0/338403992 <== mgr.4108 v2:172.21.15.188:6836/3401128553 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+46 (secure 0 0 0) 0x7feb5015ebe0 con 0x7feb3c066620 2023-11-26T07:24:28.063 INFO:teuthology.orchestra.run.smithi143.stdout:{ 2023-11-26T07:24:28.063 INFO:teuthology.orchestra.run.smithi143.stdout: "status": { 2023-11-26T07:24:28.064 INFO:teuthology.orchestra.run.smithi143.stdout: "state": "complete" 2023-11-26T07:24:28.064 INFO:teuthology.orchestra.run.smithi143.stdout: } 2023-11-26T07:24:28.064 INFO:teuthology.orchestra.run.smithi143.stdout:} 2023-11-26T07:24:28.064 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:28.063+0000 7feb54cc8640 1 -- 172.21.15.143:0/338403992 >> [v2:172.21.15.188:6836/3401128553,v1:172.21.15.188:6837/3401128553] conn(0x7feb3c066620 msgr2=0x7feb3c068ae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:24:28.064 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:28.063+0000 7feb54cc8640 1 --2- 172.21.15.143:0/338403992 >> [v2:172.21.15.188:6836/3401128553,v1:172.21.15.188:6837/3401128553] conn(0x7feb3c066620 0x7feb3c068ae0 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7feb4401c4f0 tx=0x7feb440b5040 comp rx=0 tx=0).stop 2023-11-26T07:24:28.064 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:28.063+0000 7feb54cc8640 1 -- 172.21.15.143:0/338403992 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7feb500d01b0 msgr2=0x7feb50153400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:24:28.065 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:28.063+0000 7feb54cc8640 1 --2- 172.21.15.143:0/338403992 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7feb500d01b0 0x7feb50153400 secure :-1 s=READY pgs=1361 cs=0 l=1 rev1=1 crypto rx=0x7feb4000c400 tx=0x7feb4000c8c0 comp rx=0 tx=0).stop 2023-11-26T07:24:28.065 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:28.063+0000 7feb54cc8640 1 -- 172.21.15.143:0/338403992 shutdown_connections 2023-11-26T07:24:28.065 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:28.063+0000 7feb54cc8640 1 --2- 172.21.15.143:0/338403992 >> [v2:172.21.15.188:6836/3401128553,v1:172.21.15.188:6837/3401128553] conn(0x7feb3c066620 0x7feb3c068ae0 unknown :-1 s=CLOSED pgs=148 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:24:28.065 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:28.063+0000 7feb54cc8640 1 --2- 172.21.15.143:0/338403992 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7feb5015d610 0x7feb50156590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:24:28.065 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:28.063+0000 7feb54cc8640 1 --2- 172.21.15.143:0/338403992 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7feb5015d230 0x7feb50153940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:24:28.065 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:28.063+0000 7feb54cc8640 1 --2- 172.21.15.143:0/338403992 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7feb500d01b0 0x7feb50153400 unknown :-1 s=CLOSED pgs=1361 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:24:28.065 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:28.063+0000 7feb54cc8640 1 -- 172.21.15.143:0/338403992 >> 172.21.15.143:0/338403992 conn(0x7feb500d30c0 msgr2=0x7feb500d34a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:24:28.065 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:28.063+0000 7feb54cc8640 1 -- 172.21.15.143:0/338403992 shutdown_connections 2023-11-26T07:24:28.066 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:28.063+0000 7feb54cc8640 1 -- 172.21.15.143:0/338403992 wait complete. 2023-11-26T07:24:28.089 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-26T07:24:28.089 DEBUG:teuthology.orchestra.run.smithi143:> (cd /home/ubuntu/cephtest/mnt.0 && exec sudo cat ./volumes/_nogroup/subvol_clone30650/.meta) 2023-11-26T07:24:28.115 INFO:teuthology.orchestra.run.smithi143.stdout:[GLOBAL] 2023-11-26T07:24:28.115 INFO:teuthology.orchestra.run.smithi143.stdout:version = 2 2023-11-26T07:24:28.115 INFO:teuthology.orchestra.run.smithi143.stdout:type = clone 2023-11-26T07:24:28.115 INFO:teuthology.orchestra.run.smithi143.stdout:path = /volumes/_nogroup/subvol_clone30650/b942b8f0-8eab-4a75-9972-e4fe55fa1e63 2023-11-26T07:24:28.115 INFO:teuthology.orchestra.run.smithi143.stdout:state = complete 2023-11-26T07:24:28.115 INFO:teuthology.orchestra.run.smithi143.stdout: 2023-11-26T07:24:28.115 INFO:teuthology.orchestra.run.smithi143.stdout:[source] 2023-11-26T07:24:28.115 INFO:teuthology.orchestra.run.smithi143.stdout:volume = cephfs 2023-11-26T07:24:28.116 INFO:teuthology.orchestra.run.smithi143.stdout:subvolume = subvol24520 2023-11-26T07:24:28.116 INFO:teuthology.orchestra.run.smithi143.stdout:snapshot = subvol_snap55360 2023-11-26T07:24:28.116 INFO:teuthology.orchestra.run.smithi143.stdout: 2023-11-26T07:24:28.117 DEBUG:teuthology.orchestra.run.smithi143:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume getpath cephfs subvol24520 2023-11-26T07:24:28.305 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:28.303+0000 7fd864ae3640 1 Processor -- start 2023-11-26T07:24:28.305 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:28.303+0000 7fd864ae3640 1 -- start start 2023-11-26T07:24:28.306 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:28.307+0000 7fd864ae3640 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7fd860143ac0 0x7fd860070370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:24:28.306 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:28.307+0000 7fd864ae3640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fd8600cf990 0x7fd8600cd1b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:24:28.306 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:28.307+0000 7fd864ae3640 1 -- --> v1:172.21.15.188:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd8600cc730 con 0x7fd8600708b0 2023-11-26T07:24:28.307 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:28.307+0000 7fd864ae3640 1 -- --> v2:172.21.15.188:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd860143ea0 con 0x7fd860143ac0 2023-11-26T07:24:28.307 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:28.307+0000 7fd864ae3640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd860070ee0 con 0x7fd8600cf990 2023-11-26T07:24:28.307 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:28.307+0000 7fd85ed9d640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fd8600cf990 0x7fd8600cd1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:24:28.308 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:28.307+0000 7fd85e59c640 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7fd860143ac0 0x7fd860070370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:24:28.308 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:28.307+0000 7fd85ed9d640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fd8600cf990 0x7fd8600cd1b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.188:3301/0 says I am v2:172.21.15.143:40722/0 (socket says 172.21.15.143:40722) 2023-11-26T07:24:28.308 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:28.307+0000 7fd85ed9d640 1 -- 172.21.15.143:0/4212675834 learned_addr learned my addr 172.21.15.143:0/4212675834 (peer_addr_for_me v2:172.21.15.143:0/0) 2023-11-26T07:24:28.308 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:28.307+0000 7fd85e59c640 1 -- 172.21.15.143:0/4212675834 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fd8600cf990 msgr2=0x7fd8600cd1b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-26T07:24:28.308 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:28.307+0000 7fd85e59c640 1 --2- 172.21.15.143:0/4212675834 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fd8600cf990 0x7fd8600cd1b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:24:28.308 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:28.307+0000 7fd85e59c640 1 -- 172.21.15.143:0/4212675834 >> v1:172.21.15.188:6789/0 conn(0x7fd8600708b0 legacy=0x7fd8600cf280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:24:28.308 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:28.307+0000 7fd85dd9b640 1 -- 172.21.15.143:0/4212675834 <== mon.1 v1:172.21.15.188:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 882958591 0 0) 0x7fd8600cc730 con 0x7fd8600708b0 2023-11-26T07:24:28.308 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:28.307+0000 7fd85e59c640 1 -- 172.21.15.143:0/4212675834 --> v2:172.21.15.188:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd8600711c0 con 0x7fd860143ac0 2023-11-26T07:24:28.308 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:28.307+0000 7fd85ed9d640 1 --2- 172.21.15.143:0/4212675834 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fd8600cf990 0x7fd8600cd1b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-26T07:24:28.310 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:28.311+0000 7fd85e59c640 1 --2- 172.21.15.143:0/4212675834 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fd860143ac0 0x7fd860070370 secure :-1 s=READY pgs=513 cs=0 l=1 rev1=1 crypto rx=0x7fd850009d90 tx=0x7fd85001c950 comp rx=0 tx=0).ready entity=mon.1 client_cookie=14e782088276470d server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:24:28.311 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:28.311+0000 7fd85dd9b640 1 -- 172.21.15.143:0/4212675834 <== mon.1 v2:172.21.15.188:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd8500a5920 con 0x7fd860143ac0 2023-11-26T07:24:28.311 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:28.311+0000 7fd85dd9b640 1 -- 172.21.15.143:0/4212675834 <== mon.1 v2:172.21.15.188:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd8500a5a80 con 0x7fd860143ac0 2023-11-26T07:24:28.311 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:28.311+0000 7fd85dd9b640 1 -- 172.21.15.143:0/4212675834 <== mon.1 v2:172.21.15.188:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd8500a5e10 con 0x7fd860143ac0 2023-11-26T07:24:28.312 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:28.311+0000 7fd864ae3640 1 -- 172.21.15.143:0/4212675834 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fd860143ac0 msgr2=0x7fd860070370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:24:28.312 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:28.311+0000 7fd864ae3640 1 --2- 172.21.15.143:0/4212675834 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fd860143ac0 0x7fd860070370 secure :-1 s=READY pgs=513 cs=0 l=1 rev1=1 crypto rx=0x7fd850009d90 tx=0x7fd85001c950 comp rx=0 tx=0).stop 2023-11-26T07:24:28.312 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:28.311+0000 7fd864ae3640 1 -- 172.21.15.143:0/4212675834 shutdown_connections 2023-11-26T07:24:28.312 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:28.311+0000 7fd864ae3640 1 --2- 172.21.15.143:0/4212675834 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fd8600cf990 0x7fd8600cd1b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:24:28.312 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:28.311+0000 7fd864ae3640 1 --2- 172.21.15.143:0/4212675834 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fd860143ac0 0x7fd860070370 unknown :-1 s=CLOSED pgs=513 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:24:28.312 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:28.311+0000 7fd864ae3640 1 -- 172.21.15.143:0/4212675834 >> 172.21.15.143:0/4212675834 conn(0x7fd86006a890 msgr2=0x7fd86006aca0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:24:28.312 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:28.311+0000 7fd864ae3640 1 -- 172.21.15.143:0/4212675834 shutdown_connections 2023-11-26T07:24:28.312 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:28.311+0000 7fd864ae3640 1 -- 172.21.15.143:0/4212675834 wait complete. 2023-11-26T07:24:28.314 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:28.315+0000 7fd864ae3640 1 Processor -- start 2023-11-26T07:24:28.314 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:28.315+0000 7fd864ae3640 1 -- start start 2023-11-26T07:24:28.317 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:28.315+0000 7fd864ae3640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fd8600708b0 0x7fd8600ade90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:24:28.317 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:28.315+0000 7fd864ae3640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fd8600cf990 0x7fd8600aa740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:24:28.317 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:28.315+0000 7fd864ae3640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fd860143ac0 0x7fd8600af1b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:24:28.317 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:28.315+0000 7fd864ae3640 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd8600ac300 con 0x7fd8600708b0 2023-11-26T07:24:28.317 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:28.315+0000 7fd864ae3640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd860159d70 con 0x7fd8600cf990 2023-11-26T07:24:28.317 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:28.315+0000 7fd864ae3640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd8600ada00 con 0x7fd860143ac0 2023-11-26T07:24:28.317 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:28.315+0000 7fd85e59c640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fd8600708b0 0x7fd8600ade90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:24:28.318 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:28.315+0000 7fd85e59c640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fd8600708b0 0x7fd8600ade90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:38492/0 (socket says 172.21.15.143:38492) 2023-11-26T07:24:28.318 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:28.315+0000 7fd85e59c640 1 -- 172.21.15.143:0/3309576749 learned_addr learned my addr 172.21.15.143:0/3309576749 (peer_addr_for_me v2:172.21.15.143:0/0) 2023-11-26T07:24:28.318 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:28.315+0000 7fd85ed9d640 1 --2- 172.21.15.143:0/3309576749 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fd860143ac0 0x7fd8600af1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:24:28.318 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:28.315+0000 7fd855d9b640 1 --2- 172.21.15.143:0/3309576749 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fd8600cf990 0x7fd8600aa740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:24:28.318 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:28.315+0000 7fd85e59c640 1 -- 172.21.15.143:0/3309576749 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fd860143ac0 msgr2=0x7fd8600af1b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:24:28.318 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:28.315+0000 7fd85e59c640 1 --2- 172.21.15.143:0/3309576749 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fd860143ac0 0x7fd8600af1b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:24:28.319 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:28.315+0000 7fd85e59c640 1 -- 172.21.15.143:0/3309576749 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fd8600cf990 msgr2=0x7fd8600aa740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:24:28.319 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:28.315+0000 7fd85e59c640 1 --2- 172.21.15.143:0/3309576749 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fd8600cf990 0x7fd8600aa740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:24:28.319 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:28.315+0000 7fd85e59c640 1 -- 172.21.15.143:0/3309576749 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd8580094d0 con 0x7fd8600708b0 2023-11-26T07:24:28.319 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:28.315+0000 7fd85e59c640 1 --2- 172.21.15.143:0/3309576749 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fd8600708b0 0x7fd8600ade90 secure :-1 s=READY pgs=1362 cs=0 l=1 rev1=1 crypto rx=0x7fd85001ce30 tx=0x7fd8500196f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:24:28.319 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:28.315+0000 7fd8577fe640 1 -- 172.21.15.143:0/3309576749 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd8500aed60 con 0x7fd8600708b0 2023-11-26T07:24:28.319 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:28.315+0000 7fd8577fe640 1 -- 172.21.15.143:0/3309576749 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd8500b7410 con 0x7fd8600708b0 2023-11-26T07:24:28.319 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:28.315+0000 7fd8577fe640 1 -- 172.21.15.143:0/3309576749 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd8500b77c0 con 0x7fd8600708b0 2023-11-26T07:24:28.319 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:28.315+0000 7fd864ae3640 1 -- 172.21.15.143:0/3309576749 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd850009a70 con 0x7fd8600708b0 2023-11-26T07:24:28.320 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:28.315+0000 7fd864ae3640 1 -- 172.21.15.143:0/3309576749 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd8600b1990 con 0x7fd8600708b0 2023-11-26T07:24:28.321 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:28.315+0000 7fd864ae3640 1 -- 172.21.15.143:0/3309576749 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd8600ae3d0 con 0x7fd8600708b0 2023-11-26T07:24:28.321 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:28.319+0000 7fd8577fe640 1 -- 172.21.15.143:0/3309576749 <== mon.0 v2:172.21.15.143:3300/0 4 ==== mgrmap(e 18) v1 ==== 86107+0+0 (secure 0 0 0) 0x7fd8500b4050 con 0x7fd8600708b0 2023-11-26T07:24:28.321 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:28.319+0000 7fd8577fe640 1 --2- 172.21.15.143:0/3309576749 >> [v2:172.21.15.188:6836/3401128553,v1:172.21.15.188:6837/3401128553] conn(0x7fd840066770 0x7fd840068c30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:24:28.321 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:28.319+0000 7fd8577fe640 1 -- 172.21.15.143:0/3309576749 <== mon.0 v2:172.21.15.143:3300/0 5 ==== osd_map(103..103 src has 1..103) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fd85011f400 con 0x7fd8600708b0 2023-11-26T07:24:28.321 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:28.319+0000 7fd855d9b640 1 --2- 172.21.15.143:0/3309576749 >> [v2:172.21.15.188:6836/3401128553,v1:172.21.15.188:6837/3401128553] conn(0x7fd840066770 0x7fd840068c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:24:28.321 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:28.319+0000 7fd855d9b640 1 --2- 172.21.15.143:0/3309576749 >> [v2:172.21.15.188:6836/3401128553,v1:172.21.15.188:6837/3401128553] conn(0x7fd840066770 0x7fd840068c30 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7fd860140480 tx=0x7fd848009040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:24:28.323 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:28.323+0000 7fd8577fe640 1 -- 172.21.15.143:0/3309576749 <== mon.0 v2:172.21.15.143:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172649 (secure 0 0 0) 0x7fd8500ec6f0 con 0x7fd8600708b0 2023-11-26T07:24:28.494 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:28.495+0000 7fd864ae3640 1 -- 172.21.15.143:0/3309576749 --> [v2:172.21.15.188:6836/3401128553,v1:172.21.15.188:6837/3401128553] -- mgr_command(tid 0: {"prefix": "fs subvolume getpath", "vol_name": "cephfs", "sub_name": "subvol24520", "target": ["mon-mgr", ""]}) v1 -- 0x7fd860067680 con 0x7fd840066770 2023-11-26T07:24:28.509 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:28.507+0000 7fd8577fe640 1 -- 172.21.15.143:0/3309576749 <== mgr.4108 v2:172.21.15.188:6836/3401128553 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+67 (secure 0 0 0) 0x7fd860067680 con 0x7fd840066770 2023-11-26T07:24:28.510 INFO:teuthology.orchestra.run.smithi143.stdout:/volumes/_nogroup/subvol24520/f57b3d1b-7328-4835-8e29-9827b7800bab 2023-11-26T07:24:28.512 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:28.511+0000 7fd864ae3640 1 -- 172.21.15.143:0/3309576749 >> [v2:172.21.15.188:6836/3401128553,v1:172.21.15.188:6837/3401128553] conn(0x7fd840066770 msgr2=0x7fd840068c30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:24:28.512 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:28.511+0000 7fd864ae3640 1 --2- 172.21.15.143:0/3309576749 >> [v2:172.21.15.188:6836/3401128553,v1:172.21.15.188:6837/3401128553] conn(0x7fd840066770 0x7fd840068c30 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7fd860140480 tx=0x7fd848009040 comp rx=0 tx=0).stop 2023-11-26T07:24:28.512 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:28.511+0000 7fd864ae3640 1 -- 172.21.15.143:0/3309576749 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fd8600708b0 msgr2=0x7fd8600ade90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:24:28.512 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:28.511+0000 7fd864ae3640 1 --2- 172.21.15.143:0/3309576749 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fd8600708b0 0x7fd8600ade90 secure :-1 s=READY pgs=1362 cs=0 l=1 rev1=1 crypto rx=0x7fd85001ce30 tx=0x7fd8500196f0 comp rx=0 tx=0).stop 2023-11-26T07:24:28.513 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:28.511+0000 7fd864ae3640 1 -- 172.21.15.143:0/3309576749 shutdown_connections 2023-11-26T07:24:28.513 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:28.511+0000 7fd864ae3640 1 --2- 172.21.15.143:0/3309576749 >> [v2:172.21.15.188:6836/3401128553,v1:172.21.15.188:6837/3401128553] conn(0x7fd840066770 0x7fd840068c30 unknown :-1 s=CLOSED pgs=149 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:24:28.513 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:28.511+0000 7fd864ae3640 1 --2- 172.21.15.143:0/3309576749 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fd860143ac0 0x7fd8600af1b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:24:28.513 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:28.511+0000 7fd864ae3640 1 --2- 172.21.15.143:0/3309576749 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fd8600cf990 0x7fd8600aa740 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:24:28.513 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:28.511+0000 7fd864ae3640 1 --2- 172.21.15.143:0/3309576749 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fd8600708b0 0x7fd8600ade90 unknown :-1 s=CLOSED pgs=1362 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:24:28.513 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:28.511+0000 7fd864ae3640 1 -- 172.21.15.143:0/3309576749 >> 172.21.15.143:0/3309576749 conn(0x7fd86006a890 msgr2=0x7fd86015c7a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:24:28.513 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:28.511+0000 7fd864ae3640 1 -- 172.21.15.143:0/3309576749 shutdown_connections 2023-11-26T07:24:28.513 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:28.511+0000 7fd864ae3640 1 -- 172.21.15.143:0/3309576749 wait complete. 2023-11-26T07:24:28.532 DEBUG:teuthology.orchestra.run.smithi143:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume getpath cephfs subvol_clone30650 2023-11-26T07:24:28.675 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:28.675+0000 7f2a89965640 1 Processor -- start 2023-11-26T07:24:28.675 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:28.675+0000 7f2a89965640 1 -- start start 2023-11-26T07:24:28.676 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:28.675+0000 7f2a89965640 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7f2a84143780 0x7f2a840cdfe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:24:28.676 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:28.675+0000 7f2a89965640 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7f2a840ce520 0x7f2a84071bb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:24:28.676 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:28.675+0000 7f2a89965640 1 -- --> v1:172.21.15.143:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2a840ceb50 con 0x7f2a840720f0 2023-11-26T07:24:28.676 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:28.675+0000 7f2a89965640 1 -- --> v2:172.21.15.143:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2a8415c110 con 0x7f2a840ce520 2023-11-26T07:24:28.676 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:28.675+0000 7f2a89965640 1 -- --> v2:172.21.15.188:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2a84072530 con 0x7f2a84143780 2023-11-26T07:24:28.676 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:28.675+0000 7f2a7a7fc640 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7f2a840ce520 0x7f2a84071bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:24:28.676 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:28.675+0000 7f2a837fe640 1 --1- >> v1:172.21.15.143:6789/0 conn(0x7f2a840720f0 0x7f2a8415a450 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.143:6789/0 says I am v1:172.21.15.143:39370/0 (socket says 172.21.15.143:39370) 2023-11-26T07:24:28.676 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:28.675+0000 7f2a837fe640 1 -- 172.21.15.143:0/2606131775 learned_addr learned my addr 172.21.15.143:0/2606131775 (peer_addr_for_me v1:172.21.15.143:0/0) 2023-11-26T07:24:28.677 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:28.675+0000 7f2a82ffd640 1 --2- 172.21.15.143:0/2606131775 >> v2:172.21.15.188:3300/0 conn(0x7f2a84143780 0x7f2a840cdfe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:24:28.677 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:28.675+0000 7f2a7a7fc640 1 -- 172.21.15.143:0/2606131775 >> v2:172.21.15.188:3300/0 conn(0x7f2a84143780 msgr2=0x7f2a840cdfe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-26T07:24:28.677 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:28.675+0000 7f2a7a7fc640 1 --2- 172.21.15.143:0/2606131775 >> v2:172.21.15.188:3300/0 conn(0x7f2a84143780 0x7f2a840cdfe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:24:28.678 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:28.675+0000 7f2a7a7fc640 1 -- 172.21.15.143:0/2606131775 >> v1:172.21.15.143:6789/0 conn(0x7f2a840720f0 legacy=0x7f2a8415a450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:24:28.678 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:28.675+0000 7f2a7a7fc640 1 -- 172.21.15.143:0/2606131775 --> v2:172.21.15.143:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2a8415b7e0 con 0x7f2a840ce520 2023-11-26T07:24:28.678 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:28.675+0000 7f2a827fc640 1 -- 172.21.15.143:0/2606131775 <== mon.0 v1:172.21.15.143:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3962723980 0 0) 0x7f2a7c001120 con 0x7f2a840720f0 2023-11-26T07:24:28.678 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:28.675+0000 7f2a82ffd640 1 --2- 172.21.15.143:0/2606131775 >> v2:172.21.15.188:3300/0 conn(0x7f2a84143780 0x7f2a840cdfe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-26T07:24:28.679 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:28.679+0000 7f2a7a7fc640 1 --2- 172.21.15.143:0/2606131775 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f2a840ce520 0x7f2a84071bb0 secure :-1 s=READY pgs=1364 cs=0 l=1 rev1=1 crypto rx=0x7f2a8406be30 tx=0x7f2a6c01c460 comp rx=0 tx=0).ready entity=mon.0 client_cookie=da3aec4bbbf6a24f server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:24:28.679 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:28.679+0000 7f2a827fc640 1 -- 172.21.15.143:0/2606131775 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2a6c0ab600 con 0x7f2a840ce520 2023-11-26T07:24:28.679 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:28.679+0000 7f2a827fc640 1 -- 172.21.15.143:0/2606131775 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2a6c0a7050 con 0x7f2a840ce520 2023-11-26T07:24:28.680 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:28.679+0000 7f2a827fc640 1 -- 172.21.15.143:0/2606131775 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2a6c0ab980 con 0x7f2a840ce520 2023-11-26T07:24:28.680 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:28.679+0000 7f2a89965640 1 -- 172.21.15.143:0/2606131775 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f2a840ce520 msgr2=0x7f2a84071bb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:24:28.680 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:28.679+0000 7f2a89965640 1 --2- 172.21.15.143:0/2606131775 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f2a840ce520 0x7f2a84071bb0 secure :-1 s=READY pgs=1364 cs=0 l=1 rev1=1 crypto rx=0x7f2a8406be30 tx=0x7f2a6c01c460 comp rx=0 tx=0).stop 2023-11-26T07:24:28.680 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:28.679+0000 7f2a89965640 1 -- 172.21.15.143:0/2606131775 shutdown_connections 2023-11-26T07:24:28.681 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:28.679+0000 7f2a89965640 1 --2- 172.21.15.143:0/2606131775 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f2a840ce520 0x7f2a84071bb0 unknown :-1 s=CLOSED pgs=1364 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:24:28.681 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:28.679+0000 7f2a89965640 1 --2- 172.21.15.143:0/2606131775 >> v2:172.21.15.188:3300/0 conn(0x7f2a84143780 0x7f2a840cdfe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:24:28.681 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:28.679+0000 7f2a89965640 1 -- 172.21.15.143:0/2606131775 >> 172.21.15.143:0/2606131775 conn(0x7f2a8406c3f0 msgr2=0x7f2a8406c800 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:24:28.681 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:28.679+0000 7f2a89965640 1 -- 172.21.15.143:0/2606131775 shutdown_connections 2023-11-26T07:24:28.681 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:28.679+0000 7f2a89965640 1 -- 172.21.15.143:0/2606131775 wait complete. 2023-11-26T07:24:28.681 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:28.683+0000 7f2a89965640 1 Processor -- start 2023-11-26T07:24:28.682 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:28.683+0000 7f2a89965640 1 -- start start 2023-11-26T07:24:28.684 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:28.683+0000 7f2a89965640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f2a840720f0 0x7f2a84157d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:24:28.684 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:28.683+0000 7f2a89965640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f2a840ce520 0x7f2a841653b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:24:28.684 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:28.683+0000 7f2a89965640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f2a84143780 0x7f2a8413bea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:24:28.684 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:28.683+0000 7f2a89965640 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2a84158290 con 0x7f2a840720f0 2023-11-26T07:24:28.685 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:28.683+0000 7f2a89965640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2a84165b60 con 0x7f2a840ce520 2023-11-26T07:24:28.685 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:28.683+0000 7f2a89965640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2a84137f10 con 0x7f2a84143780 2023-11-26T07:24:28.685 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:28.683+0000 7f2a82ffd640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f2a840720f0 0x7f2a84157d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:24:28.685 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:28.683+0000 7f2a82ffd640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f2a840720f0 0x7f2a84157d50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:38506/0 (socket says 172.21.15.143:38506) 2023-11-26T07:24:28.685 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:28.683+0000 7f2a82ffd640 1 -- 172.21.15.143:0/2046797442 learned_addr learned my addr 172.21.15.143:0/2046797442 (peer_addr_for_me v2:172.21.15.143:0/0) 2023-11-26T07:24:28.685 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:28.683+0000 7f2a837fe640 1 --2- 172.21.15.143:0/2046797442 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f2a84143780 0x7f2a8413bea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:24:28.685 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:28.683+0000 7f2a7a7fc640 1 --2- 172.21.15.143:0/2046797442 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f2a840ce520 0x7f2a841653b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:24:28.685 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:28.683+0000 7f2a82ffd640 1 -- 172.21.15.143:0/2046797442 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f2a84143780 msgr2=0x7f2a8413bea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:24:28.685 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:28.683+0000 7f2a82ffd640 1 --2- 172.21.15.143:0/2046797442 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f2a84143780 0x7f2a8413bea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:24:28.685 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:28.683+0000 7f2a82ffd640 1 -- 172.21.15.143:0/2046797442 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f2a840ce520 msgr2=0x7f2a841653b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:24:28.685 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:28.683+0000 7f2a82ffd640 1 --2- 172.21.15.143:0/2046797442 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f2a840ce520 0x7f2a841653b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:24:28.685 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:28.683+0000 7f2a82ffd640 1 -- 172.21.15.143:0/2046797442 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2a6c009420 con 0x7f2a840720f0 2023-11-26T07:24:28.686 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:28.683+0000 7f2a82ffd640 1 --2- 172.21.15.143:0/2046797442 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f2a840720f0 0x7f2a84157d50 secure :-1 s=READY pgs=1365 cs=0 l=1 rev1=1 crypto rx=0x7f2a7400ce00 tx=0x7f2a7400a650 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:24:28.686 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:28.683+0000 7f2a80ff9640 1 -- 172.21.15.143:0/2046797442 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2a7400f890 con 0x7f2a840720f0 2023-11-26T07:24:28.686 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:28.683+0000 7f2a80ff9640 1 -- 172.21.15.143:0/2046797442 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2a7400f9f0 con 0x7f2a840720f0 2023-11-26T07:24:28.686 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:28.683+0000 7f2a89965640 1 -- 172.21.15.143:0/2046797442 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2a84159040 con 0x7f2a840720f0 2023-11-26T07:24:28.686 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:28.683+0000 7f2a80ff9640 1 -- 172.21.15.143:0/2046797442 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2a7400fdc0 con 0x7f2a840720f0 2023-11-26T07:24:28.686 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:28.683+0000 7f2a89965640 1 -- 172.21.15.143:0/2046797442 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2a84159600 con 0x7f2a840720f0 2023-11-26T07:24:28.687 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:28.687+0000 7f2a89965640 1 -- 172.21.15.143:0/2046797442 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2a841597f0 con 0x7f2a840720f0 2023-11-26T07:24:28.687 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:28.687+0000 7f2a80ff9640 1 -- 172.21.15.143:0/2046797442 <== mon.0 v2:172.21.15.143:3300/0 4 ==== mgrmap(e 18) v1 ==== 86107+0+0 (secure 0 0 0) 0x7f2a7400b040 con 0x7f2a840720f0 2023-11-26T07:24:28.688 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:28.687+0000 7f2a80ff9640 1 --2- 172.21.15.143:0/2046797442 >> [v2:172.21.15.188:6836/3401128553,v1:172.21.15.188:6837/3401128553] conn(0x7f2a64066740 0x7f2a64068c00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:24:28.688 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:28.687+0000 7f2a80ff9640 1 -- 172.21.15.143:0/2046797442 <== mon.0 v2:172.21.15.143:3300/0 5 ==== osd_map(103..103 src has 1..103) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f2a74084dc0 con 0x7f2a840720f0 2023-11-26T07:24:28.688 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:28.687+0000 7f2a7a7fc640 1 --2- 172.21.15.143:0/2046797442 >> [v2:172.21.15.188:6836/3401128553,v1:172.21.15.188:6837/3401128553] conn(0x7f2a64066740 0x7f2a64068c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:24:28.688 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:28.687+0000 7f2a7a7fc640 1 --2- 172.21.15.143:0/2046797442 >> [v2:172.21.15.188:6836/3401128553,v1:172.21.15.188:6837/3401128553] conn(0x7f2a64066740 0x7f2a64068c00 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7f2a6c0ad000 tx=0x7f2a6c008040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:24:28.692 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:28.691+0000 7f2a80ff9640 1 -- 172.21.15.143:0/2046797442 <== mon.0 v2:172.21.15.143:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172649 (secure 0 0 0) 0x7f2a74051e30 con 0x7f2a840720f0 2023-11-26T07:24:28.865 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:28.867+0000 7f2a89965640 1 -- 172.21.15.143:0/2046797442 --> [v2:172.21.15.188:6836/3401128553,v1:172.21.15.188:6837/3401128553] -- mgr_command(tid 0: {"prefix": "fs subvolume getpath", "vol_name": "cephfs", "sub_name": "subvol_clone30650", "target": ["mon-mgr", ""]}) v1 -- 0x7f2a84067500 con 0x7f2a64066740 2023-11-26T07:24:28.872 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:28.871+0000 7f2a80ff9640 1 -- 172.21.15.143:0/2046797442 <== mgr.4108 v2:172.21.15.188:6836/3401128553 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+73 (secure 0 0 0) 0x7f2a84067500 con 0x7f2a64066740 2023-11-26T07:24:28.872 INFO:teuthology.orchestra.run.smithi143.stdout:/volumes/_nogroup/subvol_clone30650/b942b8f0-8eab-4a75-9972-e4fe55fa1e63 2023-11-26T07:24:28.875 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:28.875+0000 7f2a89965640 1 -- 172.21.15.143:0/2046797442 >> [v2:172.21.15.188:6836/3401128553,v1:172.21.15.188:6837/3401128553] conn(0x7f2a64066740 msgr2=0x7f2a64068c00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:24:28.875 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:28.875+0000 7f2a89965640 1 --2- 172.21.15.143:0/2046797442 >> [v2:172.21.15.188:6836/3401128553,v1:172.21.15.188:6837/3401128553] conn(0x7f2a64066740 0x7f2a64068c00 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7f2a6c0ad000 tx=0x7f2a6c008040 comp rx=0 tx=0).stop 2023-11-26T07:24:28.875 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:28.875+0000 7f2a89965640 1 -- 172.21.15.143:0/2046797442 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f2a840720f0 msgr2=0x7f2a84157d50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:24:28.875 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:28.875+0000 7f2a89965640 1 --2- 172.21.15.143:0/2046797442 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f2a840720f0 0x7f2a84157d50 secure :-1 s=READY pgs=1365 cs=0 l=1 rev1=1 crypto rx=0x7f2a7400ce00 tx=0x7f2a7400a650 comp rx=0 tx=0).stop 2023-11-26T07:24:28.875 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:28.875+0000 7f2a89965640 1 -- 172.21.15.143:0/2046797442 shutdown_connections 2023-11-26T07:24:28.875 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:28.875+0000 7f2a89965640 1 --2- 172.21.15.143:0/2046797442 >> [v2:172.21.15.188:6836/3401128553,v1:172.21.15.188:6837/3401128553] conn(0x7f2a64066740 0x7f2a64068c00 unknown :-1 s=CLOSED pgs=150 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:24:28.876 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:28.875+0000 7f2a89965640 1 --2- 172.21.15.143:0/2046797442 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f2a84143780 0x7f2a8413bea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:24:28.876 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:28.875+0000 7f2a89965640 1 --2- 172.21.15.143:0/2046797442 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f2a840ce520 0x7f2a841653b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:24:28.876 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:28.875+0000 7f2a89965640 1 --2- 172.21.15.143:0/2046797442 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f2a840720f0 0x7f2a84157d50 unknown :-1 s=CLOSED pgs=1365 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:24:28.876 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:28.875+0000 7f2a89965640 1 -- 172.21.15.143:0/2046797442 >> 172.21.15.143:0/2046797442 conn(0x7f2a8406c3f0 msgr2=0x7f2a840ad580 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:24:28.876 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:28.875+0000 7f2a89965640 1 -- 172.21.15.143:0/2046797442 shutdown_connections 2023-11-26T07:24:28.876 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:28.875+0000 7f2a89965640 1 -- 172.21.15.143:0/2046797442 wait complete. 2023-11-26T07:24:28.893 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-26T07:24:28.893 DEBUG:teuthology.orchestra.run.smithi143:> (cd /home/ubuntu/cephtest/mnt.0 && exec getfattr --only-values -n ceph.dir.rentries volumes/_nogroup/subvol24520/f57b3d1b-7328-4835-8e29-9827b7800bab/.snap/subvol_snap55360) 2023-11-26T07:24:28.911 INFO:teuthology.orchestra.run.smithi143.stdout:9 2023-11-26T07:24:28.912 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-26T07:24:28.912 DEBUG:teuthology.orchestra.run.smithi143:> (cd /home/ubuntu/cephtest/mnt.0 && exec getfattr --only-values -n ceph.dir.rentries volumes/_nogroup/subvol_clone30650/b942b8f0-8eab-4a75-9972-e4fe55fa1e63) 2023-11-26T07:24:28.966 INFO:teuthology.orchestra.run.smithi143.stdout:9 2023-11-26T07:24:28.966 DEBUG:teuthology.orchestra.run.smithi143:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume info cephfs subvol_clone30650 2023-11-26T07:24:29.154 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:29.155+0000 7f1e06cfe640 1 Processor -- start 2023-11-26T07:24:29.154 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:29.155+0000 7f1e06cfe640 1 -- start start 2023-11-26T07:24:29.156 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:29.155+0000 7f1e06cfe640 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7f1e00143ac0 0x7f1e00070370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:24:29.156 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:29.155+0000 7f1e06cfe640 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7f1e000708b0 0x7f1e000cf280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:24:29.156 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:29.155+0000 7f1e06cfe640 1 -- --> v1:172.21.15.143:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1e000cc730 con 0x7f1e000cf7c0 2023-11-26T07:24:29.156 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:29.155+0000 7f1e06cfe640 1 -- --> v2:172.21.15.143:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1e00143ea0 con 0x7f1e000708b0 2023-11-26T07:24:29.156 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:29.155+0000 7f1e06cfe640 1 -- --> v2:172.21.15.188:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1e000cfbd0 con 0x7f1e00143ac0 2023-11-26T07:24:29.156 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:29.155+0000 7f1dfffff640 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7f1e000708b0 0x7f1e000cf280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:24:29.156 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:29.155+0000 7f1e0529b640 1 --1- >> v1:172.21.15.143:6789/0 conn(0x7f1e000cf7c0 0x7f1e000b1ef0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.143:6789/0 says I am v1:172.21.15.143:39376/0 (socket says 172.21.15.143:39376) 2023-11-26T07:24:29.156 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:29.155+0000 7f1e0529b640 1 -- 172.21.15.143:0/711516103 learned_addr learned my addr 172.21.15.143:0/711516103 (peer_addr_for_me v1:172.21.15.143:0/0) 2023-11-26T07:24:29.156 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:29.155+0000 7f1dfffff640 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7f1e000708b0 0x7f1e000cf280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:38516/0 (socket says 172.21.15.143:38516) 2023-11-26T07:24:29.157 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:29.155+0000 7f1e04a9a640 1 --2- 172.21.15.143:0/711516103 >> v2:172.21.15.188:3300/0 conn(0x7f1e00143ac0 0x7f1e00070370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:24:29.157 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:29.155+0000 7f1dfffff640 1 -- 172.21.15.143:0/711516103 >> v2:172.21.15.188:3300/0 conn(0x7f1e00143ac0 msgr2=0x7f1e00070370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-26T07:24:29.157 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:29.155+0000 7f1dfffff640 1 --2- 172.21.15.143:0/711516103 >> v2:172.21.15.188:3300/0 conn(0x7f1e00143ac0 0x7f1e00070370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:24:29.157 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:29.155+0000 7f1dfffff640 1 -- 172.21.15.143:0/711516103 >> v1:172.21.15.143:6789/0 conn(0x7f1e000cf7c0 legacy=0x7f1e000b1ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:24:29.157 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:29.155+0000 7f1dfffff640 1 -- 172.21.15.143:0/711516103 --> v2:172.21.15.143:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1e00070ee0 con 0x7f1e000708b0 2023-11-26T07:24:29.157 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:29.155+0000 7f1dff7fe640 1 -- 172.21.15.143:0/711516103 <== mon.0 v1:172.21.15.143:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2378901957 0 0) 0x7f1e000cc730 con 0x7f1e000cf7c0 2023-11-26T07:24:29.159 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:29.159+0000 7f1dfffff640 1 --2- 172.21.15.143:0/711516103 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f1e000708b0 0x7f1e000cf280 secure :-1 s=READY pgs=1367 cs=0 l=1 rev1=1 crypto rx=0x7f1e0006dca0 tx=0x7f1dec01c460 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fdf7583ffbac1924 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:24:29.159 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:29.159+0000 7f1dff7fe640 1 -- 172.21.15.143:0/711516103 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1dec0ab660 con 0x7f1e000708b0 2023-11-26T07:24:29.159 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:29.159+0000 7f1dff7fe640 1 -- 172.21.15.143:0/711516103 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1dec0ab7c0 con 0x7f1e000708b0 2023-11-26T07:24:29.159 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:29.159+0000 7f1dff7fe640 1 -- 172.21.15.143:0/711516103 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1dec0abb50 con 0x7f1e000708b0 2023-11-26T07:24:29.160 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:29.159+0000 7f1e06cfe640 1 -- 172.21.15.143:0/711516103 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f1e000708b0 msgr2=0x7f1e000cf280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:24:29.160 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:29.159+0000 7f1e06cfe640 1 --2- 172.21.15.143:0/711516103 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f1e000708b0 0x7f1e000cf280 secure :-1 s=READY pgs=1367 cs=0 l=1 rev1=1 crypto rx=0x7f1e0006dca0 tx=0x7f1dec01c460 comp rx=0 tx=0).stop 2023-11-26T07:24:29.160 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:29.159+0000 7f1e06cfe640 1 -- 172.21.15.143:0/711516103 shutdown_connections 2023-11-26T07:24:29.160 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:29.159+0000 7f1e06cfe640 1 --2- 172.21.15.143:0/711516103 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f1e000708b0 0x7f1e000cf280 unknown :-1 s=CLOSED pgs=1367 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:24:29.160 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:29.159+0000 7f1e06cfe640 1 --2- 172.21.15.143:0/711516103 >> v2:172.21.15.188:3300/0 conn(0x7f1e00143ac0 0x7f1e00070370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:24:29.160 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:29.159+0000 7f1e06cfe640 1 -- 172.21.15.143:0/711516103 >> 172.21.15.143:0/711516103 conn(0x7f1e0006a890 msgr2=0x7f1e0006aca0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:24:29.160 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:29.159+0000 7f1e06cfe640 1 -- 172.21.15.143:0/711516103 shutdown_connections 2023-11-26T07:24:29.161 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:29.159+0000 7f1e06cfe640 1 -- 172.21.15.143:0/711516103 wait complete. 2023-11-26T07:24:29.161 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:29.163+0000 7f1e06cfe640 1 Processor -- start 2023-11-26T07:24:29.161 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:29.163+0000 7f1e06cfe640 1 -- start start 2023-11-26T07:24:29.164 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:29.163+0000 7f1e06cfe640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f1e000708b0 0x7f1e00159b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:24:29.164 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:29.163+0000 7f1e06cfe640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f1e000cf7c0 0x7f1e000aefe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:24:29.164 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:29.163+0000 7f1e06cfe640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f1e00143ac0 0x7f1e000af520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:24:29.164 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:29.163+0000 7f1e06cfe640 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1e0013c690 con 0x7f1e000708b0 2023-11-26T07:24:29.164 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:29.163+0000 7f1e06cfe640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1e0013bd60 con 0x7f1e00143ac0 2023-11-26T07:24:29.164 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:29.163+0000 7f1e06cfe640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1e0013c040 con 0x7f1e000cf7c0 2023-11-26T07:24:29.164 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:29.163+0000 7f1e04a9a640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f1e000708b0 0x7f1e00159b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:24:29.164 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:29.163+0000 7f1e04a9a640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f1e000708b0 0x7f1e00159b20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:38522/0 (socket says 172.21.15.143:38522) 2023-11-26T07:24:29.164 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:29.163+0000 7f1e04a9a640 1 -- 172.21.15.143:0/3073908660 learned_addr learned my addr 172.21.15.143:0/3073908660 (peer_addr_for_me v2:172.21.15.143:0/0) 2023-11-26T07:24:29.165 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:29.163+0000 7f1dfffff640 1 --2- 172.21.15.143:0/3073908660 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f1e000cf7c0 0x7f1e000aefe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:24:29.165 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:29.163+0000 7f1e0529b640 1 --2- 172.21.15.143:0/3073908660 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f1e00143ac0 0x7f1e000af520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:24:29.165 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:29.163+0000 7f1e04a9a640 1 -- 172.21.15.143:0/3073908660 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f1e000cf7c0 msgr2=0x7f1e000aefe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:24:29.165 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:29.163+0000 7f1e04a9a640 1 --2- 172.21.15.143:0/3073908660 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f1e000cf7c0 0x7f1e000aefe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:24:29.165 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:29.163+0000 7f1e04a9a640 1 -- 172.21.15.143:0/3073908660 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f1e00143ac0 msgr2=0x7f1e000af520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:24:29.165 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:29.163+0000 7f1e04a9a640 1 --2- 172.21.15.143:0/3073908660 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f1e00143ac0 0x7f1e000af520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:24:29.165 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:29.163+0000 7f1e04a9a640 1 -- 172.21.15.143:0/3073908660 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1dec009420 con 0x7f1e000708b0 2023-11-26T07:24:29.165 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:29.163+0000 7f1e0529b640 1 --2- 172.21.15.143:0/3073908660 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f1e00143ac0 0x7f1e000af520 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-26T07:24:29.165 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:29.163+0000 7f1e04a9a640 1 --2- 172.21.15.143:0/3073908660 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f1e000708b0 0x7f1e00159b20 secure :-1 s=READY pgs=1368 cs=0 l=1 rev1=1 crypto rx=0x7f1df400d5f0 tx=0x7f1df400dab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:24:29.165 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:29.163+0000 7f1dfd7fa640 1 -- 172.21.15.143:0/3073908660 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1df40125f0 con 0x7f1e000708b0 2023-11-26T07:24:29.165 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:29.163+0000 7f1dfd7fa640 1 -- 172.21.15.143:0/3073908660 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1df4012750 con 0x7f1e000708b0 2023-11-26T07:24:29.165 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:29.163+0000 7f1dfd7fa640 1 -- 172.21.15.143:0/3073908660 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1df4012ae0 con 0x7f1e000708b0 2023-11-26T07:24:29.166 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:29.163+0000 7f1e06cfe640 1 -- 172.21.15.143:0/3073908660 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1e0013c320 con 0x7f1e000708b0 2023-11-26T07:24:29.166 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:29.163+0000 7f1e06cfe640 1 -- 172.21.15.143:0/3073908660 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1e000afc60 con 0x7f1e000708b0 2023-11-26T07:24:29.166 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:29.163+0000 7f1e06cfe640 1 -- 172.21.15.143:0/3073908660 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1e00067680 con 0x7f1e000708b0 2023-11-26T07:24:29.166 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:29.167+0000 7f1dfd7fa640 1 -- 172.21.15.143:0/3073908660 <== mon.0 v2:172.21.15.143:3300/0 4 ==== mgrmap(e 18) v1 ==== 86107+0+0 (secure 0 0 0) 0x7f1df4012c40 con 0x7f1e000708b0 2023-11-26T07:24:29.167 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:29.167+0000 7f1dfd7fa640 1 --2- 172.21.15.143:0/3073908660 >> [v2:172.21.15.188:6836/3401128553,v1:172.21.15.188:6837/3401128553] conn(0x7f1de4066740 0x7f1de4068c00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:24:29.167 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:29.167+0000 7f1dfd7fa640 1 -- 172.21.15.143:0/3073908660 <== mon.0 v2:172.21.15.143:3300/0 5 ==== osd_map(103..103 src has 1..103) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f1df4084850 con 0x7f1e000708b0 2023-11-26T07:24:29.167 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:29.167+0000 7f1dfffff640 1 --2- 172.21.15.143:0/3073908660 >> [v2:172.21.15.188:6836/3401128553,v1:172.21.15.188:6837/3401128553] conn(0x7f1de4066740 0x7f1de4068c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:24:29.168 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:29.167+0000 7f1dfffff640 1 --2- 172.21.15.143:0/3073908660 >> [v2:172.21.15.188:6836/3401128553,v1:172.21.15.188:6837/3401128553] conn(0x7f1de4066740 0x7f1de4068c00 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7f1dec0af000 tx=0x7f1dec0a7000 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:24:29.171 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:29.171+0000 7f1dfd7fa640 1 -- 172.21.15.143:0/3073908660 <== mon.0 v2:172.21.15.143:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172649 (secure 0 0 0) 0x7f1df4051940 con 0x7f1e000708b0 2023-11-26T07:24:29.342 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:29.343+0000 7f1e06cfe640 1 -- 172.21.15.143:0/3073908660 --> [v2:172.21.15.188:6836/3401128553,v1:172.21.15.188:6837/3401128553] -- mgr_command(tid 0: {"prefix": "fs subvolume info", "vol_name": "cephfs", "sub_name": "subvol_clone30650", "target": ["mon-mgr", ""]}) v1 -- 0x7f1e00071c30 con 0x7f1de4066740 2023-11-26T07:24:29.353 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:29.351+0000 7f1dfd7fa640 1 -- 172.21.15.143:0/3073908660 <== mgr.4108 v2:172.21.15.188:6836/3401128553 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+708 (secure 0 0 0) 0x7f1dec006e20 con 0x7f1de4066740 2023-11-26T07:24:29.353 INFO:teuthology.orchestra.run.smithi143.stdout:{ 2023-11-26T07:24:29.353 INFO:teuthology.orchestra.run.smithi143.stdout: "atime": "2023-11-26 07:24:19", 2023-11-26T07:24:29.353 INFO:teuthology.orchestra.run.smithi143.stdout: "bytes_pcent": "80.00", 2023-11-26T07:24:29.353 INFO:teuthology.orchestra.run.smithi143.stdout: "bytes_quota": 10485760, 2023-11-26T07:24:29.353 INFO:teuthology.orchestra.run.smithi143.stdout: "bytes_used": 8388608, 2023-11-26T07:24:29.353 INFO:teuthology.orchestra.run.smithi143.stdout: "created_at": "2023-11-26 07:24:24", 2023-11-26T07:24:29.354 INFO:teuthology.orchestra.run.smithi143.stdout: "ctime": "2023-11-26 07:24:24", 2023-11-26T07:24:29.354 INFO:teuthology.orchestra.run.smithi143.stdout: "data_pool": "cephfs_data", 2023-11-26T07:24:29.354 INFO:teuthology.orchestra.run.smithi143.stdout: "features": [ 2023-11-26T07:24:29.354 INFO:teuthology.orchestra.run.smithi143.stdout: "snapshot-clone", 2023-11-26T07:24:29.354 INFO:teuthology.orchestra.run.smithi143.stdout: "snapshot-autoprotect", 2023-11-26T07:24:29.354 INFO:teuthology.orchestra.run.smithi143.stdout: "snapshot-retention" 2023-11-26T07:24:29.354 INFO:teuthology.orchestra.run.smithi143.stdout: ], 2023-11-26T07:24:29.354 INFO:teuthology.orchestra.run.smithi143.stdout: "gid": 1000, 2023-11-26T07:24:29.354 INFO:teuthology.orchestra.run.smithi143.stdout: "mode": 16895, 2023-11-26T07:24:29.354 INFO:teuthology.orchestra.run.smithi143.stdout: "mon_addrs": [ 2023-11-26T07:24:29.354 INFO:teuthology.orchestra.run.smithi143.stdout: "172.21.15.143:6789", 2023-11-26T07:24:29.354 INFO:teuthology.orchestra.run.smithi143.stdout: "172.21.15.188:6789", 2023-11-26T07:24:29.354 INFO:teuthology.orchestra.run.smithi143.stdout: "172.21.15.188:6790" 2023-11-26T07:24:29.355 INFO:teuthology.orchestra.run.smithi143.stdout: ], 2023-11-26T07:24:29.355 INFO:teuthology.orchestra.run.smithi143.stdout: "mtime": "2023-11-26 07:24:23", 2023-11-26T07:24:29.355 INFO:teuthology.orchestra.run.smithi143.stdout: "path": "/volumes/_nogroup/subvol_clone30650/b942b8f0-8eab-4a75-9972-e4fe55fa1e63", 2023-11-26T07:24:29.355 INFO:teuthology.orchestra.run.smithi143.stdout: "pool_namespace": "", 2023-11-26T07:24:29.355 INFO:teuthology.orchestra.run.smithi143.stdout: "state": "complete", 2023-11-26T07:24:29.355 INFO:teuthology.orchestra.run.smithi143.stdout: "type": "clone", 2023-11-26T07:24:29.355 INFO:teuthology.orchestra.run.smithi143.stdout: "uid": 1000 2023-11-26T07:24:29.355 INFO:teuthology.orchestra.run.smithi143.stdout:} 2023-11-26T07:24:29.356 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:29.355+0000 7f1e06cfe640 1 -- 172.21.15.143:0/3073908660 >> [v2:172.21.15.188:6836/3401128553,v1:172.21.15.188:6837/3401128553] conn(0x7f1de4066740 msgr2=0x7f1de4068c00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:24:29.356 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:29.355+0000 7f1e06cfe640 1 --2- 172.21.15.143:0/3073908660 >> [v2:172.21.15.188:6836/3401128553,v1:172.21.15.188:6837/3401128553] conn(0x7f1de4066740 0x7f1de4068c00 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7f1dec0af000 tx=0x7f1dec0a7000 comp rx=0 tx=0).stop 2023-11-26T07:24:29.356 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:29.355+0000 7f1e06cfe640 1 -- 172.21.15.143:0/3073908660 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f1e000708b0 msgr2=0x7f1e00159b20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:24:29.356 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:29.355+0000 7f1e06cfe640 1 --2- 172.21.15.143:0/3073908660 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f1e000708b0 0x7f1e00159b20 secure :-1 s=READY pgs=1368 cs=0 l=1 rev1=1 crypto rx=0x7f1df400d5f0 tx=0x7f1df400dab0 comp rx=0 tx=0).stop 2023-11-26T07:24:29.356 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:29.355+0000 7f1e06cfe640 1 -- 172.21.15.143:0/3073908660 shutdown_connections 2023-11-26T07:24:29.356 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:29.355+0000 7f1e06cfe640 1 --2- 172.21.15.143:0/3073908660 >> [v2:172.21.15.188:6836/3401128553,v1:172.21.15.188:6837/3401128553] conn(0x7f1de4066740 0x7f1de4068c00 unknown :-1 s=CLOSED pgs=151 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:24:29.356 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:29.355+0000 7f1e06cfe640 1 --2- 172.21.15.143:0/3073908660 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f1e00143ac0 0x7f1e000af520 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:24:29.356 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:29.355+0000 7f1e06cfe640 1 --2- 172.21.15.143:0/3073908660 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f1e000cf7c0 0x7f1e000aefe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:24:29.357 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:29.355+0000 7f1e06cfe640 1 --2- 172.21.15.143:0/3073908660 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f1e000708b0 0x7f1e00159b20 unknown :-1 s=CLOSED pgs=1368 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:24:29.357 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:29.355+0000 7f1e06cfe640 1 -- 172.21.15.143:0/3073908660 >> 172.21.15.143:0/3073908660 conn(0x7f1e0006a890 msgr2=0x7f1e00140c50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:24:29.357 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:29.355+0000 7f1e06cfe640 1 -- 172.21.15.143:0/3073908660 shutdown_connections 2023-11-26T07:24:29.357 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:29.355+0000 7f1e06cfe640 1 -- 172.21.15.143:0/3073908660 wait complete. 2023-11-26T07:24:29.374 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-26T07:24:29.375 DEBUG:teuthology.orchestra.run.smithi143:> (cd /home/ubuntu/cephtest/mnt.0 && exec getfattr --only-values -n ceph.quota.max_bytes volumes/_nogroup/subvol24520/f57b3d1b-7328-4835-8e29-9827b7800bab/.snap/subvol_snap55360) 2023-11-26T07:24:29.391 INFO:teuthology.orchestra.run.smithi143.stdout:10485760 2023-11-26T07:24:29.391 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-26T07:24:29.391 DEBUG:teuthology.orchestra.run.smithi143:> (cd /home/ubuntu/cephtest/mnt.0 && exec getfattr --only-values -n ceph.dir.layout.pool volumes/_nogroup/subvol24520/f57b3d1b-7328-4835-8e29-9827b7800bab/.snap/subvol_snap55360) 2023-11-26T07:24:29.449 INFO:teuthology.orchestra.run.smithi143.stdout:cephfs_data 2023-11-26T07:24:29.449 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-26T07:24:29.449 DEBUG:teuthology.orchestra.run.smithi143:> (cd /home/ubuntu/cephtest/mnt.0 && exec getfattr --only-values -n ceph.dir.layout.pool_namespace volumes/_nogroup/subvol24520/f57b3d1b-7328-4835-8e29-9827b7800bab/.snap/subvol_snap55360) 2023-11-26T07:24:29.504 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-26T07:24:29.505 DEBUG:teuthology.orchestra.run.smithi143:> (cd /home/ubuntu/cephtest/mnt.0 && exec find volumes/_nogroup/subvol24520/f57b3d1b-7328-4835-8e29-9827b7800bab/.snap/subvol_snap55360) 2023-11-26T07:24:29.563 INFO:teuthology.orchestra.run.smithi143.stdout:volumes/_nogroup/subvol24520/f57b3d1b-7328-4835-8e29-9827b7800bab/.snap/subvol_snap55360 2023-11-26T07:24:29.563 INFO:teuthology.orchestra.run.smithi143.stdout:volumes/_nogroup/subvol24520/f57b3d1b-7328-4835-8e29-9827b7800bab/.snap/subvol_snap55360/subvolume_file.2 2023-11-26T07:24:29.563 INFO:teuthology.orchestra.run.smithi143.stdout:volumes/_nogroup/subvol24520/f57b3d1b-7328-4835-8e29-9827b7800bab/.snap/subvol_snap55360/subvolume_file.5 2023-11-26T07:24:29.563 INFO:teuthology.orchestra.run.smithi143.stdout:volumes/_nogroup/subvol24520/f57b3d1b-7328-4835-8e29-9827b7800bab/.snap/subvol_snap55360/subvolume_file.4 2023-11-26T07:24:29.563 INFO:teuthology.orchestra.run.smithi143.stdout:volumes/_nogroup/subvol24520/f57b3d1b-7328-4835-8e29-9827b7800bab/.snap/subvol_snap55360/subvolume_file.7 2023-11-26T07:24:29.563 INFO:teuthology.orchestra.run.smithi143.stdout:volumes/_nogroup/subvol24520/f57b3d1b-7328-4835-8e29-9827b7800bab/.snap/subvol_snap55360/subvolume_file.6 2023-11-26T07:24:29.563 INFO:teuthology.orchestra.run.smithi143.stdout:volumes/_nogroup/subvol24520/f57b3d1b-7328-4835-8e29-9827b7800bab/.snap/subvol_snap55360/subvolume_file.0 2023-11-26T07:24:29.563 INFO:teuthology.orchestra.run.smithi143.stdout:volumes/_nogroup/subvol24520/f57b3d1b-7328-4835-8e29-9827b7800bab/.snap/subvol_snap55360/subvolume_file.3 2023-11-26T07:24:29.564 INFO:teuthology.orchestra.run.smithi143.stdout:volumes/_nogroup/subvol24520/f57b3d1b-7328-4835-8e29-9827b7800bab/.snap/subvol_snap55360/subvolume_file.1 2023-11-26T07:24:29.564 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-26T07:24:29.565 DEBUG:teuthology.orchestra.run.smithi143:> (cd /home/ubuntu/cephtest/mnt.0 && exec stat -c%f volumes/_nogroup/subvol24520/f57b3d1b-7328-4835-8e29-9827b7800bab/.snap/subvol_snap55360) 2023-11-26T07:24:29.616 INFO:teuthology.orchestra.run.smithi143.stdout:41ff 2023-11-26T07:24:29.617 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-26T07:24:29.617 DEBUG:teuthology.orchestra.run.smithi143:> (cd /home/ubuntu/cephtest/mnt.0 && exec stat -c%f volumes/_nogroup/subvol_clone30650/b942b8f0-8eab-4a75-9972-e4fe55fa1e63/) 2023-11-26T07:24:29.670 INFO:teuthology.orchestra.run.smithi143.stdout:41ff 2023-11-26T07:24:29.671 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-26T07:24:29.672 DEBUG:teuthology.orchestra.run.smithi143:> (cd /home/ubuntu/cephtest/mnt.0 && exec stat -c%u volumes/_nogroup/subvol24520/f57b3d1b-7328-4835-8e29-9827b7800bab/.snap/subvol_snap55360) 2023-11-26T07:24:29.724 INFO:teuthology.orchestra.run.smithi143.stdout:1000 2023-11-26T07:24:29.725 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-26T07:24:29.725 DEBUG:teuthology.orchestra.run.smithi143:> (cd /home/ubuntu/cephtest/mnt.0 && exec stat -c%u volumes/_nogroup/subvol_clone30650/b942b8f0-8eab-4a75-9972-e4fe55fa1e63/) 2023-11-26T07:24:29.778 INFO:teuthology.orchestra.run.smithi143.stdout:1000 2023-11-26T07:24:29.779 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-26T07:24:29.779 DEBUG:teuthology.orchestra.run.smithi143:> (cd /home/ubuntu/cephtest/mnt.0 && exec stat -c%g volumes/_nogroup/subvol24520/f57b3d1b-7328-4835-8e29-9827b7800bab/.snap/subvol_snap55360) 2023-11-26T07:24:29.836 INFO:teuthology.orchestra.run.smithi143.stdout:1000 2023-11-26T07:24:29.837 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-26T07:24:29.837 DEBUG:teuthology.orchestra.run.smithi143:> (cd /home/ubuntu/cephtest/mnt.0 && exec stat -c%g volumes/_nogroup/subvol_clone30650/b942b8f0-8eab-4a75-9972-e4fe55fa1e63/) 2023-11-26T07:24:29.886 INFO:teuthology.orchestra.run.smithi143.stdout:1000 2023-11-26T07:24:29.887 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-26T07:24:29.887 DEBUG:teuthology.orchestra.run.smithi143:> (cd /home/ubuntu/cephtest/mnt.0 && exec stat -c%Y volumes/_nogroup/subvol24520/f57b3d1b-7328-4835-8e29-9827b7800bab/.snap/subvol_snap55360) 2023-11-26T07:24:29.939 INFO:teuthology.orchestra.run.smithi143.stdout:1700983463 2023-11-26T07:24:29.940 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-26T07:24:29.940 DEBUG:teuthology.orchestra.run.smithi143:> (cd /home/ubuntu/cephtest/mnt.0 && exec stat -c%Y volumes/_nogroup/subvol_clone30650/b942b8f0-8eab-4a75-9972-e4fe55fa1e63/) 2023-11-26T07:24:29.990 INFO:teuthology.orchestra.run.smithi143.stdout:1700983463 2023-11-26T07:24:29.991 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-26T07:24:29.991 DEBUG:teuthology.orchestra.run.smithi143:> (cd /home/ubuntu/cephtest/mnt.0 && exec stat -c%f volumes/_nogroup/subvol24520/f57b3d1b-7328-4835-8e29-9827b7800bab/.snap/subvol_snap55360/subvolume_file.2) 2023-11-26T07:24:30.048 INFO:teuthology.orchestra.run.smithi143.stdout:81b4 2023-11-26T07:24:30.049 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-26T07:24:30.049 DEBUG:teuthology.orchestra.run.smithi143:> (cd /home/ubuntu/cephtest/mnt.0 && exec stat -c%f volumes/_nogroup/subvol_clone30650/b942b8f0-8eab-4a75-9972-e4fe55fa1e63/subvolume_file.2) 2023-11-26T07:24:30.109 INFO:teuthology.orchestra.run.smithi143.stdout:81b4 2023-11-26T07:24:30.110 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-26T07:24:30.111 DEBUG:teuthology.orchestra.run.smithi143:> (cd /home/ubuntu/cephtest/mnt.0 && exec stat -c%u volumes/_nogroup/subvol24520/f57b3d1b-7328-4835-8e29-9827b7800bab/.snap/subvol_snap55360/subvolume_file.2) 2023-11-26T07:24:30.164 INFO:teuthology.orchestra.run.smithi143.stdout:1000 2023-11-26T07:24:30.165 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-26T07:24:30.165 DEBUG:teuthology.orchestra.run.smithi143:> (cd /home/ubuntu/cephtest/mnt.0 && exec stat -c%u volumes/_nogroup/subvol_clone30650/b942b8f0-8eab-4a75-9972-e4fe55fa1e63/subvolume_file.2) 2023-11-26T07:24:30.219 INFO:teuthology.orchestra.run.smithi143.stdout:1000 2023-11-26T07:24:30.220 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-26T07:24:30.220 DEBUG:teuthology.orchestra.run.smithi143:> (cd /home/ubuntu/cephtest/mnt.0 && exec stat -c%g volumes/_nogroup/subvol24520/f57b3d1b-7328-4835-8e29-9827b7800bab/.snap/subvol_snap55360/subvolume_file.2) 2023-11-26T07:24:30.272 INFO:teuthology.orchestra.run.smithi143.stdout:1271 2023-11-26T07:24:30.273 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-26T07:24:30.273 DEBUG:teuthology.orchestra.run.smithi143:> (cd /home/ubuntu/cephtest/mnt.0 && exec stat -c%g volumes/_nogroup/subvol_clone30650/b942b8f0-8eab-4a75-9972-e4fe55fa1e63/subvolume_file.2) 2023-11-26T07:24:30.323 INFO:teuthology.orchestra.run.smithi143.stdout:1271 2023-11-26T07:24:30.323 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-26T07:24:30.324 DEBUG:teuthology.orchestra.run.smithi143:> (cd /home/ubuntu/cephtest/mnt.0 && exec stat -c%Y volumes/_nogroup/subvol24520/f57b3d1b-7328-4835-8e29-9827b7800bab/.snap/subvol_snap55360/subvolume_file.2) 2023-11-26T07:24:30.376 INFO:teuthology.orchestra.run.smithi143.stdout:1700983462 2023-11-26T07:24:30.377 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-26T07:24:30.377 DEBUG:teuthology.orchestra.run.smithi143:> (cd /home/ubuntu/cephtest/mnt.0 && exec stat -c%Y volumes/_nogroup/subvol_clone30650/b942b8f0-8eab-4a75-9972-e4fe55fa1e63/subvolume_file.2) 2023-11-26T07:24:30.431 INFO:teuthology.orchestra.run.smithi143.stdout:1700983462 2023-11-26T07:24:30.432 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-26T07:24:30.432 DEBUG:teuthology.orchestra.run.smithi143:> (cd /home/ubuntu/cephtest/mnt.0 && exec stat -c%f volumes/_nogroup/subvol24520/f57b3d1b-7328-4835-8e29-9827b7800bab/.snap/subvol_snap55360/subvolume_file.5) 2023-11-26T07:24:30.484 INFO:teuthology.orchestra.run.smithi143.stdout:81b4 2023-11-26T07:24:30.485 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-26T07:24:30.485 DEBUG:teuthology.orchestra.run.smithi143:> (cd /home/ubuntu/cephtest/mnt.0 && exec stat -c%f volumes/_nogroup/subvol_clone30650/b942b8f0-8eab-4a75-9972-e4fe55fa1e63/subvolume_file.5) 2023-11-26T07:24:30.546 INFO:teuthology.orchestra.run.smithi143.stdout:81b4 2023-11-26T07:24:30.547 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-26T07:24:30.547 DEBUG:teuthology.orchestra.run.smithi143:> (cd /home/ubuntu/cephtest/mnt.0 && exec stat -c%u volumes/_nogroup/subvol24520/f57b3d1b-7328-4835-8e29-9827b7800bab/.snap/subvol_snap55360/subvolume_file.5) 2023-11-26T07:24:30.600 INFO:teuthology.orchestra.run.smithi143.stdout:1000 2023-11-26T07:24:30.601 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-26T07:24:30.602 DEBUG:teuthology.orchestra.run.smithi143:> (cd /home/ubuntu/cephtest/mnt.0 && exec stat -c%u volumes/_nogroup/subvol_clone30650/b942b8f0-8eab-4a75-9972-e4fe55fa1e63/subvolume_file.5) 2023-11-26T07:24:30.655 INFO:teuthology.orchestra.run.smithi143.stdout:1000 2023-11-26T07:24:30.656 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-26T07:24:30.656 DEBUG:teuthology.orchestra.run.smithi143:> (cd /home/ubuntu/cephtest/mnt.0 && exec stat -c%g volumes/_nogroup/subvol24520/f57b3d1b-7328-4835-8e29-9827b7800bab/.snap/subvol_snap55360/subvolume_file.5) 2023-11-26T07:24:30.712 INFO:teuthology.orchestra.run.smithi143.stdout:1271 2023-11-26T07:24:30.713 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-26T07:24:30.713 DEBUG:teuthology.orchestra.run.smithi143:> (cd /home/ubuntu/cephtest/mnt.0 && exec stat -c%g volumes/_nogroup/subvol_clone30650/b942b8f0-8eab-4a75-9972-e4fe55fa1e63/subvolume_file.5) 2023-11-26T07:24:30.767 INFO:teuthology.orchestra.run.smithi143.stdout:1271 2023-11-26T07:24:30.768 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-26T07:24:30.768 DEBUG:teuthology.orchestra.run.smithi143:> (cd /home/ubuntu/cephtest/mnt.0 && exec stat -c%Y volumes/_nogroup/subvol24520/f57b3d1b-7328-4835-8e29-9827b7800bab/.snap/subvol_snap55360/subvolume_file.5) 2023-11-26T07:24:30.820 INFO:teuthology.orchestra.run.smithi143.stdout:1700983463 2023-11-26T07:24:30.821 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-26T07:24:30.821 DEBUG:teuthology.orchestra.run.smithi143:> (cd /home/ubuntu/cephtest/mnt.0 && exec stat -c%Y volumes/_nogroup/subvol_clone30650/b942b8f0-8eab-4a75-9972-e4fe55fa1e63/subvolume_file.5) 2023-11-26T07:24:30.874 INFO:teuthology.orchestra.run.smithi143.stdout:1700983463 2023-11-26T07:24:30.875 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-26T07:24:30.875 DEBUG:teuthology.orchestra.run.smithi143:> (cd /home/ubuntu/cephtest/mnt.0 && exec stat -c%f volumes/_nogroup/subvol24520/f57b3d1b-7328-4835-8e29-9827b7800bab/.snap/subvol_snap55360/subvolume_file.4) 2023-11-26T07:24:30.928 INFO:teuthology.orchestra.run.smithi143.stdout:81b4 2023-11-26T07:24:30.929 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-26T07:24:30.929 DEBUG:teuthology.orchestra.run.smithi143:> (cd /home/ubuntu/cephtest/mnt.0 && exec stat -c%f volumes/_nogroup/subvol_clone30650/b942b8f0-8eab-4a75-9972-e4fe55fa1e63/subvolume_file.4) 2023-11-26T07:24:30.980 INFO:teuthology.orchestra.run.smithi143.stdout:81b4 2023-11-26T07:24:30.981 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-26T07:24:30.981 DEBUG:teuthology.orchestra.run.smithi143:> (cd /home/ubuntu/cephtest/mnt.0 && exec stat -c%u volumes/_nogroup/subvol24520/f57b3d1b-7328-4835-8e29-9827b7800bab/.snap/subvol_snap55360/subvolume_file.4) 2023-11-26T07:24:31.031 INFO:teuthology.orchestra.run.smithi143.stdout:1000 2023-11-26T07:24:31.032 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-26T07:24:31.033 DEBUG:teuthology.orchestra.run.smithi143:> (cd /home/ubuntu/cephtest/mnt.0 && exec stat -c%u volumes/_nogroup/subvol_clone30650/b942b8f0-8eab-4a75-9972-e4fe55fa1e63/subvolume_file.4) 2023-11-26T07:24:31.086 INFO:teuthology.orchestra.run.smithi143.stdout:1000 2023-11-26T07:24:31.087 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-26T07:24:31.087 DEBUG:teuthology.orchestra.run.smithi143:> (cd /home/ubuntu/cephtest/mnt.0 && exec stat -c%g volumes/_nogroup/subvol24520/f57b3d1b-7328-4835-8e29-9827b7800bab/.snap/subvol_snap55360/subvolume_file.4) 2023-11-26T07:24:31.140 INFO:teuthology.orchestra.run.smithi143.stdout:1271 2023-11-26T07:24:31.141 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-26T07:24:31.141 DEBUG:teuthology.orchestra.run.smithi143:> (cd /home/ubuntu/cephtest/mnt.0 && exec stat -c%g volumes/_nogroup/subvol_clone30650/b942b8f0-8eab-4a75-9972-e4fe55fa1e63/subvolume_file.4) 2023-11-26T07:24:31.195 INFO:teuthology.orchestra.run.smithi143.stdout:1271 2023-11-26T07:24:31.195 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-26T07:24:31.196 DEBUG:teuthology.orchestra.run.smithi143:> (cd /home/ubuntu/cephtest/mnt.0 && exec stat -c%Y volumes/_nogroup/subvol24520/f57b3d1b-7328-4835-8e29-9827b7800bab/.snap/subvol_snap55360/subvolume_file.4) 2023-11-26T07:24:31.252 INFO:teuthology.orchestra.run.smithi143.stdout:1700983463 2023-11-26T07:24:31.253 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-26T07:24:31.253 DEBUG:teuthology.orchestra.run.smithi143:> (cd /home/ubuntu/cephtest/mnt.0 && exec stat -c%Y volumes/_nogroup/subvol_clone30650/b942b8f0-8eab-4a75-9972-e4fe55fa1e63/subvolume_file.4) 2023-11-26T07:24:31.307 INFO:teuthology.orchestra.run.smithi143.stdout:1700983463 2023-11-26T07:24:31.307 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-26T07:24:31.308 DEBUG:teuthology.orchestra.run.smithi143:> (cd /home/ubuntu/cephtest/mnt.0 && exec stat -c%f volumes/_nogroup/subvol24520/f57b3d1b-7328-4835-8e29-9827b7800bab/.snap/subvol_snap55360/subvolume_file.7) 2023-11-26T07:24:31.360 INFO:teuthology.orchestra.run.smithi143.stdout:81b4 2023-11-26T07:24:31.361 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-26T07:24:31.361 DEBUG:teuthology.orchestra.run.smithi143:> (cd /home/ubuntu/cephtest/mnt.0 && exec stat -c%f volumes/_nogroup/subvol_clone30650/b942b8f0-8eab-4a75-9972-e4fe55fa1e63/subvolume_file.7) 2023-11-26T07:24:31.419 INFO:teuthology.orchestra.run.smithi143.stdout:81b4 2023-11-26T07:24:31.420 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-26T07:24:31.420 DEBUG:teuthology.orchestra.run.smithi143:> (cd /home/ubuntu/cephtest/mnt.0 && exec stat -c%u volumes/_nogroup/subvol24520/f57b3d1b-7328-4835-8e29-9827b7800bab/.snap/subvol_snap55360/subvolume_file.7) 2023-11-26T07:24:31.476 INFO:teuthology.orchestra.run.smithi143.stdout:1000 2023-11-26T07:24:31.477 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-26T07:24:31.477 DEBUG:teuthology.orchestra.run.smithi143:> (cd /home/ubuntu/cephtest/mnt.0 && exec stat -c%u volumes/_nogroup/subvol_clone30650/b942b8f0-8eab-4a75-9972-e4fe55fa1e63/subvolume_file.7) 2023-11-26T07:24:31.530 INFO:teuthology.orchestra.run.smithi143.stdout:1000 2023-11-26T07:24:31.531 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-26T07:24:31.532 DEBUG:teuthology.orchestra.run.smithi143:> (cd /home/ubuntu/cephtest/mnt.0 && exec stat -c%g volumes/_nogroup/subvol24520/f57b3d1b-7328-4835-8e29-9827b7800bab/.snap/subvol_snap55360/subvolume_file.7) 2023-11-26T07:24:31.584 INFO:teuthology.orchestra.run.smithi143.stdout:1271 2023-11-26T07:24:31.585 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-26T07:24:31.585 DEBUG:teuthology.orchestra.run.smithi143:> (cd /home/ubuntu/cephtest/mnt.0 && exec stat -c%g volumes/_nogroup/subvol_clone30650/b942b8f0-8eab-4a75-9972-e4fe55fa1e63/subvolume_file.7) 2023-11-26T07:24:31.635 INFO:teuthology.orchestra.run.smithi143.stdout:1271 2023-11-26T07:24:31.636 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-26T07:24:31.636 DEBUG:teuthology.orchestra.run.smithi143:> (cd /home/ubuntu/cephtest/mnt.0 && exec stat -c%Y volumes/_nogroup/subvol24520/f57b3d1b-7328-4835-8e29-9827b7800bab/.snap/subvol_snap55360/subvolume_file.7) 2023-11-26T07:24:31.692 INFO:teuthology.orchestra.run.smithi143.stdout:1700983463 2023-11-26T07:24:31.693 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-26T07:24:31.693 DEBUG:teuthology.orchestra.run.smithi143:> (cd /home/ubuntu/cephtest/mnt.0 && exec stat -c%Y volumes/_nogroup/subvol_clone30650/b942b8f0-8eab-4a75-9972-e4fe55fa1e63/subvolume_file.7) 2023-11-26T07:24:31.743 INFO:teuthology.orchestra.run.smithi143.stdout:1700983463 2023-11-26T07:24:31.743 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-26T07:24:31.744 DEBUG:teuthology.orchestra.run.smithi143:> (cd /home/ubuntu/cephtest/mnt.0 && exec stat -c%f volumes/_nogroup/subvol24520/f57b3d1b-7328-4835-8e29-9827b7800bab/.snap/subvol_snap55360/subvolume_file.6) 2023-11-26T07:24:31.800 INFO:teuthology.orchestra.run.smithi143.stdout:81b4 2023-11-26T07:24:31.801 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-26T07:24:31.801 DEBUG:teuthology.orchestra.run.smithi143:> (cd /home/ubuntu/cephtest/mnt.0 && exec stat -c%f volumes/_nogroup/subvol_clone30650/b942b8f0-8eab-4a75-9972-e4fe55fa1e63/subvolume_file.6) 2023-11-26T07:24:31.855 INFO:teuthology.orchestra.run.smithi143.stdout:81b4 2023-11-26T07:24:31.856 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-26T07:24:31.856 DEBUG:teuthology.orchestra.run.smithi143:> (cd /home/ubuntu/cephtest/mnt.0 && exec stat -c%u volumes/_nogroup/subvol24520/f57b3d1b-7328-4835-8e29-9827b7800bab/.snap/subvol_snap55360/subvolume_file.6) 2023-11-26T07:24:31.912 INFO:teuthology.orchestra.run.smithi143.stdout:1000 2023-11-26T07:24:31.912 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-26T07:24:31.913 DEBUG:teuthology.orchestra.run.smithi143:> (cd /home/ubuntu/cephtest/mnt.0 && exec stat -c%u volumes/_nogroup/subvol_clone30650/b942b8f0-8eab-4a75-9972-e4fe55fa1e63/subvolume_file.6) 2023-11-26T07:24:31.962 INFO:teuthology.orchestra.run.smithi143.stdout:1000 2023-11-26T07:24:31.963 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-26T07:24:31.964 DEBUG:teuthology.orchestra.run.smithi143:> (cd /home/ubuntu/cephtest/mnt.0 && exec stat -c%g volumes/_nogroup/subvol24520/f57b3d1b-7328-4835-8e29-9827b7800bab/.snap/subvol_snap55360/subvolume_file.6) 2023-11-26T07:24:32.016 INFO:teuthology.orchestra.run.smithi143.stdout:1271 2023-11-26T07:24:32.017 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-26T07:24:32.017 DEBUG:teuthology.orchestra.run.smithi143:> (cd /home/ubuntu/cephtest/mnt.0 && exec stat -c%g volumes/_nogroup/subvol_clone30650/b942b8f0-8eab-4a75-9972-e4fe55fa1e63/subvolume_file.6) 2023-11-26T07:24:32.071 INFO:teuthology.orchestra.run.smithi143.stdout:1271 2023-11-26T07:24:32.072 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-26T07:24:32.072 DEBUG:teuthology.orchestra.run.smithi143:> (cd /home/ubuntu/cephtest/mnt.0 && exec stat -c%Y volumes/_nogroup/subvol24520/f57b3d1b-7328-4835-8e29-9827b7800bab/.snap/subvol_snap55360/subvolume_file.6) 2023-11-26T07:24:32.124 INFO:teuthology.orchestra.run.smithi143.stdout:1700983463 2023-11-26T07:24:32.125 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-26T07:24:32.125 DEBUG:teuthology.orchestra.run.smithi143:> (cd /home/ubuntu/cephtest/mnt.0 && exec stat -c%Y volumes/_nogroup/subvol_clone30650/b942b8f0-8eab-4a75-9972-e4fe55fa1e63/subvolume_file.6) 2023-11-26T07:24:32.175 INFO:teuthology.orchestra.run.smithi143.stdout:1700983463 2023-11-26T07:24:32.176 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-26T07:24:32.176 DEBUG:teuthology.orchestra.run.smithi143:> (cd /home/ubuntu/cephtest/mnt.0 && exec stat -c%f volumes/_nogroup/subvol24520/f57b3d1b-7328-4835-8e29-9827b7800bab/.snap/subvol_snap55360/subvolume_file.0) 2023-11-26T07:24:32.228 INFO:teuthology.orchestra.run.smithi143.stdout:81b4 2023-11-26T07:24:32.229 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-26T07:24:32.229 DEBUG:teuthology.orchestra.run.smithi143:> (cd /home/ubuntu/cephtest/mnt.0 && exec stat -c%f volumes/_nogroup/subvol_clone30650/b942b8f0-8eab-4a75-9972-e4fe55fa1e63/subvolume_file.0) 2023-11-26T07:24:32.279 INFO:teuthology.orchestra.run.smithi143.stdout:81b4 2023-11-26T07:24:32.280 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-26T07:24:32.280 DEBUG:teuthology.orchestra.run.smithi143:> (cd /home/ubuntu/cephtest/mnt.0 && exec stat -c%u volumes/_nogroup/subvol24520/f57b3d1b-7328-4835-8e29-9827b7800bab/.snap/subvol_snap55360/subvolume_file.0) 2023-11-26T07:24:32.332 INFO:teuthology.orchestra.run.smithi143.stdout:1000 2023-11-26T07:24:32.333 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-26T07:24:32.333 DEBUG:teuthology.orchestra.run.smithi143:> (cd /home/ubuntu/cephtest/mnt.0 && exec stat -c%u volumes/_nogroup/subvol_clone30650/b942b8f0-8eab-4a75-9972-e4fe55fa1e63/subvolume_file.0) 2023-11-26T07:24:32.382 INFO:teuthology.orchestra.run.smithi143.stdout:1000 2023-11-26T07:24:32.383 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-26T07:24:32.383 DEBUG:teuthology.orchestra.run.smithi143:> (cd /home/ubuntu/cephtest/mnt.0 && exec stat -c%g volumes/_nogroup/subvol24520/f57b3d1b-7328-4835-8e29-9827b7800bab/.snap/subvol_snap55360/subvolume_file.0) 2023-11-26T07:24:32.439 INFO:teuthology.orchestra.run.smithi143.stdout:1271 2023-11-26T07:24:32.440 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-26T07:24:32.441 DEBUG:teuthology.orchestra.run.smithi143:> (cd /home/ubuntu/cephtest/mnt.0 && exec stat -c%g volumes/_nogroup/subvol_clone30650/b942b8f0-8eab-4a75-9972-e4fe55fa1e63/subvolume_file.0) 2023-11-26T07:24:32.490 INFO:teuthology.orchestra.run.smithi143.stdout:1271 2023-11-26T07:24:32.491 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-26T07:24:32.491 DEBUG:teuthology.orchestra.run.smithi143:> (cd /home/ubuntu/cephtest/mnt.0 && exec stat -c%Y volumes/_nogroup/subvol24520/f57b3d1b-7328-4835-8e29-9827b7800bab/.snap/subvol_snap55360/subvolume_file.0) 2023-11-26T07:24:32.548 INFO:teuthology.orchestra.run.smithi143.stdout:1700983462 2023-11-26T07:24:32.549 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-26T07:24:32.549 DEBUG:teuthology.orchestra.run.smithi143:> (cd /home/ubuntu/cephtest/mnt.0 && exec stat -c%Y volumes/_nogroup/subvol_clone30650/b942b8f0-8eab-4a75-9972-e4fe55fa1e63/subvolume_file.0) 2023-11-26T07:24:32.602 INFO:teuthology.orchestra.run.smithi143.stdout:1700983462 2023-11-26T07:24:32.603 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-26T07:24:32.603 DEBUG:teuthology.orchestra.run.smithi143:> (cd /home/ubuntu/cephtest/mnt.0 && exec stat -c%f volumes/_nogroup/subvol24520/f57b3d1b-7328-4835-8e29-9827b7800bab/.snap/subvol_snap55360/subvolume_file.3) 2023-11-26T07:24:32.656 INFO:teuthology.orchestra.run.smithi143.stdout:81b4 2023-11-26T07:24:32.657 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-26T07:24:32.657 DEBUG:teuthology.orchestra.run.smithi143:> (cd /home/ubuntu/cephtest/mnt.0 && exec stat -c%f volumes/_nogroup/subvol_clone30650/b942b8f0-8eab-4a75-9972-e4fe55fa1e63/subvolume_file.3) 2023-11-26T07:24:32.715 INFO:teuthology.orchestra.run.smithi143.stdout:81b4 2023-11-26T07:24:32.716 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-26T07:24:32.716 DEBUG:teuthology.orchestra.run.smithi143:> (cd /home/ubuntu/cephtest/mnt.0 && exec stat -c%u volumes/_nogroup/subvol24520/f57b3d1b-7328-4835-8e29-9827b7800bab/.snap/subvol_snap55360/subvolume_file.3) 2023-11-26T07:24:32.767 INFO:teuthology.orchestra.run.smithi143.stdout:1000 2023-11-26T07:24:32.768 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-26T07:24:32.768 DEBUG:teuthology.orchestra.run.smithi143:> (cd /home/ubuntu/cephtest/mnt.0 && exec stat -c%u volumes/_nogroup/subvol_clone30650/b942b8f0-8eab-4a75-9972-e4fe55fa1e63/subvolume_file.3) 2023-11-26T07:24:32.818 INFO:teuthology.orchestra.run.smithi143.stdout:1000 2023-11-26T07:24:32.819 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-26T07:24:32.820 DEBUG:teuthology.orchestra.run.smithi143:> (cd /home/ubuntu/cephtest/mnt.0 && exec stat -c%g volumes/_nogroup/subvol24520/f57b3d1b-7328-4835-8e29-9827b7800bab/.snap/subvol_snap55360/subvolume_file.3) 2023-11-26T07:24:32.871 INFO:teuthology.orchestra.run.smithi143.stdout:1271 2023-11-26T07:24:32.873 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-26T07:24:32.873 DEBUG:teuthology.orchestra.run.smithi143:> (cd /home/ubuntu/cephtest/mnt.0 && exec stat -c%g volumes/_nogroup/subvol_clone30650/b942b8f0-8eab-4a75-9972-e4fe55fa1e63/subvolume_file.3) 2023-11-26T07:24:32.922 INFO:teuthology.orchestra.run.smithi143.stdout:1271 2023-11-26T07:24:32.923 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-26T07:24:32.923 DEBUG:teuthology.orchestra.run.smithi143:> (cd /home/ubuntu/cephtest/mnt.0 && exec stat -c%Y volumes/_nogroup/subvol24520/f57b3d1b-7328-4835-8e29-9827b7800bab/.snap/subvol_snap55360/subvolume_file.3) 2023-11-26T07:24:32.976 INFO:teuthology.orchestra.run.smithi143.stdout:1700983462 2023-11-26T07:24:32.977 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-26T07:24:32.977 DEBUG:teuthology.orchestra.run.smithi143:> (cd /home/ubuntu/cephtest/mnt.0 && exec stat -c%Y volumes/_nogroup/subvol_clone30650/b942b8f0-8eab-4a75-9972-e4fe55fa1e63/subvolume_file.3) 2023-11-26T07:24:33.030 INFO:teuthology.orchestra.run.smithi143.stdout:1700983462 2023-11-26T07:24:33.031 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-26T07:24:33.032 DEBUG:teuthology.orchestra.run.smithi143:> (cd /home/ubuntu/cephtest/mnt.0 && exec stat -c%f volumes/_nogroup/subvol24520/f57b3d1b-7328-4835-8e29-9827b7800bab/.snap/subvol_snap55360/subvolume_file.1) 2023-11-26T07:24:33.088 INFO:teuthology.orchestra.run.smithi143.stdout:81b4 2023-11-26T07:24:33.089 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-26T07:24:33.089 DEBUG:teuthology.orchestra.run.smithi143:> (cd /home/ubuntu/cephtest/mnt.0 && exec stat -c%f volumes/_nogroup/subvol_clone30650/b942b8f0-8eab-4a75-9972-e4fe55fa1e63/subvolume_file.1) 2023-11-26T07:24:33.147 INFO:teuthology.orchestra.run.smithi143.stdout:81b4 2023-11-26T07:24:33.148 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-26T07:24:33.148 DEBUG:teuthology.orchestra.run.smithi143:> (cd /home/ubuntu/cephtest/mnt.0 && exec stat -c%u volumes/_nogroup/subvol24520/f57b3d1b-7328-4835-8e29-9827b7800bab/.snap/subvol_snap55360/subvolume_file.1) 2023-11-26T07:24:33.204 INFO:teuthology.orchestra.run.smithi143.stdout:1000 2023-11-26T07:24:33.205 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-26T07:24:33.205 DEBUG:teuthology.orchestra.run.smithi143:> (cd /home/ubuntu/cephtest/mnt.0 && exec stat -c%u volumes/_nogroup/subvol_clone30650/b942b8f0-8eab-4a75-9972-e4fe55fa1e63/subvolume_file.1) 2023-11-26T07:24:33.262 INFO:teuthology.orchestra.run.smithi143.stdout:1000 2023-11-26T07:24:33.263 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-26T07:24:33.263 DEBUG:teuthology.orchestra.run.smithi143:> (cd /home/ubuntu/cephtest/mnt.0 && exec stat -c%g volumes/_nogroup/subvol24520/f57b3d1b-7328-4835-8e29-9827b7800bab/.snap/subvol_snap55360/subvolume_file.1) 2023-11-26T07:24:33.316 INFO:teuthology.orchestra.run.smithi143.stdout:1271 2023-11-26T07:24:33.317 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-26T07:24:33.317 DEBUG:teuthology.orchestra.run.smithi143:> (cd /home/ubuntu/cephtest/mnt.0 && exec stat -c%g volumes/_nogroup/subvol_clone30650/b942b8f0-8eab-4a75-9972-e4fe55fa1e63/subvolume_file.1) 2023-11-26T07:24:33.374 INFO:teuthology.orchestra.run.smithi143.stdout:1271 2023-11-26T07:24:33.375 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-26T07:24:33.375 DEBUG:teuthology.orchestra.run.smithi143:> (cd /home/ubuntu/cephtest/mnt.0 && exec stat -c%Y volumes/_nogroup/subvol24520/f57b3d1b-7328-4835-8e29-9827b7800bab/.snap/subvol_snap55360/subvolume_file.1) 2023-11-26T07:24:33.428 INFO:teuthology.orchestra.run.smithi143.stdout:1700983462 2023-11-26T07:24:33.428 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-26T07:24:33.429 DEBUG:teuthology.orchestra.run.smithi143:> (cd /home/ubuntu/cephtest/mnt.0 && exec stat -c%Y volumes/_nogroup/subvol_clone30650/b942b8f0-8eab-4a75-9972-e4fe55fa1e63/subvolume_file.1) 2023-11-26T07:24:33.482 INFO:teuthology.orchestra.run.smithi143.stdout:1700983462 2023-11-26T07:24:33.483 DEBUG:teuthology.orchestra.run.smithi143:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume snapshot clone cephfs subvol24520 fake subvol_clone15961 2023-11-26T07:24:33.668 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:33.667+0000 7fbd614c9640 1 Processor -- start 2023-11-26T07:24:33.669 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:33.667+0000 7fbd614c9640 1 -- start start 2023-11-26T07:24:33.670 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:33.671+0000 7fbd614c9640 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7fbd5c0d17f0 0x7fbd5c0d1c30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:24:33.670 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:33.671+0000 7fbd614c9640 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7fbd5c0b0490 0x7fbd5c0b0920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:24:33.670 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:33.671+0000 7fbd614c9640 1 -- --> v1:172.21.15.143:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbd5c0d0bc0 con 0x7fbd5c15d580 2023-11-26T07:24:33.670 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:33.671+0000 7fbd614c9640 1 -- --> v2:172.21.15.143:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fbd5c0d0320 con 0x7fbd5c0d17f0 2023-11-26T07:24:33.670 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:33.671+0000 7fbd614c9640 1 -- --> v2:172.21.15.188:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fbd5c0d0600 con 0x7fbd5c0b0490 2023-11-26T07:24:33.670 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:33.671+0000 7fbd53fff640 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7fbd5c0d17f0 0x7fbd5c0d1c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:24:33.670 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:33.671+0000 7fbd5affd640 1 --1- >> v1:172.21.15.143:6789/0 conn(0x7fbd5c15d580 0x7fbd5c0d3920 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.143:6789/0 says I am v1:172.21.15.143:39378/0 (socket says 172.21.15.143:39378) 2023-11-26T07:24:33.670 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:33.671+0000 7fbd5affd640 1 -- 172.21.15.143:0/1011258385 learned_addr learned my addr 172.21.15.143:0/1011258385 (peer_addr_for_me v1:172.21.15.143:0/0) 2023-11-26T07:24:33.671 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:33.671+0000 7fbd53fff640 1 -- 172.21.15.143:0/1011258385 >> v2:172.21.15.188:3300/0 conn(0x7fbd5c0b0490 msgr2=0x7fbd5c0b0920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-26T07:24:33.671 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:33.671+0000 7fbd53fff640 1 --2- 172.21.15.143:0/1011258385 >> v2:172.21.15.188:3300/0 conn(0x7fbd5c0b0490 0x7fbd5c0b0920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:24:33.671 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:33.671+0000 7fbd53fff640 1 -- 172.21.15.143:0/1011258385 >> v1:172.21.15.143:6789/0 conn(0x7fbd5c15d580 legacy=0x7fbd5c0d3920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:24:33.671 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:33.671+0000 7fbd53fff640 1 -- 172.21.15.143:0/1011258385 --> v2:172.21.15.143:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbd5c0d08e0 con 0x7fbd5c0d17f0 2023-11-26T07:24:33.671 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:33.671+0000 7fbd5a7fc640 1 -- 172.21.15.143:0/1011258385 <== mon.0 v1:172.21.15.143:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 540513263 0 0) 0x7fbd5c0d0bc0 con 0x7fbd5c15d580 2023-11-26T07:24:33.673 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:33.671+0000 7fbd53fff640 1 --2- 172.21.15.143:0/1011258385 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fbd5c0d17f0 0x7fbd5c0d1c30 secure :-1 s=READY pgs=1370 cs=0 l=1 rev1=1 crypto rx=0x7fbd5c08d400 tx=0x7fbd4401c430 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dc4ddb9385e1240 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:24:33.673 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:33.675+0000 7fbd5a7fc640 1 -- 172.21.15.143:0/1011258385 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbd440ab610 con 0x7fbd5c0d17f0 2023-11-26T07:24:33.673 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:33.675+0000 7fbd5a7fc640 1 -- 172.21.15.143:0/1011258385 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbd440ab770 con 0x7fbd5c0d17f0 2023-11-26T07:24:33.673 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:33.675+0000 7fbd5a7fc640 1 -- 172.21.15.143:0/1011258385 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbd440abb00 con 0x7fbd5c0d17f0 2023-11-26T07:24:33.674 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:33.675+0000 7fbd614c9640 1 -- 172.21.15.143:0/1011258385 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fbd5c0d17f0 msgr2=0x7fbd5c0d1c30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:24:33.674 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:33.675+0000 7fbd614c9640 1 --2- 172.21.15.143:0/1011258385 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fbd5c0d17f0 0x7fbd5c0d1c30 secure :-1 s=READY pgs=1370 cs=0 l=1 rev1=1 crypto rx=0x7fbd5c08d400 tx=0x7fbd4401c430 comp rx=0 tx=0).stop 2023-11-26T07:24:33.674 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:33.675+0000 7fbd614c9640 1 -- 172.21.15.143:0/1011258385 shutdown_connections 2023-11-26T07:24:33.674 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:33.675+0000 7fbd614c9640 1 --2- 172.21.15.143:0/1011258385 >> v2:172.21.15.188:3300/0 conn(0x7fbd5c0b0490 0x7fbd5c0b0920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:24:33.675 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:33.675+0000 7fbd614c9640 1 --2- 172.21.15.143:0/1011258385 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fbd5c0d17f0 0x7fbd5c0d1c30 unknown :-1 s=CLOSED pgs=1370 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:24:33.675 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:33.675+0000 7fbd614c9640 1 -- 172.21.15.143:0/1011258385 >> 172.21.15.143:0/1011258385 conn(0x7fbd5c09c550 msgr2=0x7fbd5c0d3210 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:24:33.675 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:33.675+0000 7fbd614c9640 1 -- 172.21.15.143:0/1011258385 shutdown_connections 2023-11-26T07:24:33.675 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:33.675+0000 7fbd614c9640 1 -- 172.21.15.143:0/1011258385 wait complete. 2023-11-26T07:24:33.675 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:33.675+0000 7fbd614c9640 1 Processor -- start 2023-11-26T07:24:33.676 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:33.675+0000 7fbd614c9640 1 -- start start 2023-11-26T07:24:33.678 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:33.675+0000 7fbd614c9640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fbd5c0b0490 0x7fbd5c153e00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:24:33.678 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:33.675+0000 7fbd614c9640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fbd5c0d17f0 0x7fbd5c06b1a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:24:33.678 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:33.675+0000 7fbd614c9640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fbd5c15d580 0x7fbd5c06b6e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:24:33.679 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:33.675+0000 7fbd614c9640 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbd5c154f70 con 0x7fbd5c0b0490 2023-11-26T07:24:33.679 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:33.675+0000 7fbd614c9640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbd5c154340 con 0x7fbd5c0d17f0 2023-11-26T07:24:33.679 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:33.675+0000 7fbd614c9640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbd5c0b7830 con 0x7fbd5c15d580 2023-11-26T07:24:33.679 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:33.675+0000 7fbd5affd640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fbd5c0b0490 0x7fbd5c153e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:24:33.679 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:33.675+0000 7fbd5affd640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fbd5c0b0490 0x7fbd5c153e00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:38540/0 (socket says 172.21.15.143:38540) 2023-11-26T07:24:33.679 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:33.675+0000 7fbd5affd640 1 -- 172.21.15.143:0/2192167107 learned_addr learned my addr 172.21.15.143:0/2192167107 (peer_addr_for_me v2:172.21.15.143:0/0) 2023-11-26T07:24:33.679 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:33.679+0000 7fbd53fff640 1 --2- 172.21.15.143:0/2192167107 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fbd5c0d17f0 0x7fbd5c06b1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:24:33.679 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:33.679+0000 7fbd5b7fe640 1 --2- 172.21.15.143:0/2192167107 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fbd5c15d580 0x7fbd5c06b6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:24:33.679 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:33.679+0000 7fbd5affd640 1 -- 172.21.15.143:0/2192167107 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fbd5c15d580 msgr2=0x7fbd5c06b6e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:24:33.679 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:33.679+0000 7fbd5affd640 1 --2- 172.21.15.143:0/2192167107 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fbd5c15d580 0x7fbd5c06b6e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:24:33.679 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:33.679+0000 7fbd5affd640 1 -- 172.21.15.143:0/2192167107 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fbd5c0d17f0 msgr2=0x7fbd5c06b1a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:24:33.680 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:33.679+0000 7fbd5affd640 1 --2- 172.21.15.143:0/2192167107 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fbd5c0d17f0 0x7fbd5c06b1a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:24:33.680 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:33.679+0000 7fbd5affd640 1 -- 172.21.15.143:0/2192167107 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbd44009420 con 0x7fbd5c0b0490 2023-11-26T07:24:33.680 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:33.679+0000 7fbd5b7fe640 1 --2- 172.21.15.143:0/2192167107 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fbd5c15d580 0x7fbd5c06b6e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-26T07:24:33.680 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:33.679+0000 7fbd5affd640 1 --2- 172.21.15.143:0/2192167107 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fbd5c0b0490 0x7fbd5c153e00 secure :-1 s=READY pgs=1371 cs=0 l=1 rev1=1 crypto rx=0x7fbd4c00c480 tx=0x7fbd4c00c940 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:24:33.680 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:33.679+0000 7fbd537fe640 1 -- 172.21.15.143:0/2192167107 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbd4c0074e0 con 0x7fbd5c0b0490 2023-11-26T07:24:33.680 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:33.679+0000 7fbd537fe640 1 -- 172.21.15.143:0/2192167107 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbd4c007660 con 0x7fbd5c0b0490 2023-11-26T07:24:33.680 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:33.679+0000 7fbd537fe640 1 -- 172.21.15.143:0/2192167107 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbd4c004a30 con 0x7fbd5c0b0490 2023-11-26T07:24:33.680 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:33.679+0000 7fbd614c9640 1 -- 172.21.15.143:0/2192167107 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbd5c0b7b10 con 0x7fbd5c0b0490 2023-11-26T07:24:33.680 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:33.679+0000 7fbd614c9640 1 -- 172.21.15.143:0/2192167107 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbd5c1551c0 con 0x7fbd5c0b0490 2023-11-26T07:24:33.681 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:33.679+0000 7fbd614c9640 1 -- 172.21.15.143:0/2192167107 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbd5c0b8030 con 0x7fbd5c0b0490 2023-11-26T07:24:33.681 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:33.679+0000 7fbd537fe640 1 -- 172.21.15.143:0/2192167107 <== mon.0 v2:172.21.15.143:3300/0 4 ==== mgrmap(e 18) v1 ==== 86107+0+0 (secure 0 0 0) 0x7fbd4c004b90 con 0x7fbd5c0b0490 2023-11-26T07:24:33.681 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:33.679+0000 7fbd537fe640 1 --2- 172.21.15.143:0/2192167107 >> [v2:172.21.15.188:6836/3401128553,v1:172.21.15.188:6837/3401128553] conn(0x7fbd3c066740 0x7fbd3c068c00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:24:33.681 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:33.679+0000 7fbd537fe640 1 -- 172.21.15.143:0/2192167107 <== mon.0 v2:172.21.15.143:3300/0 5 ==== osd_map(103..103 src has 1..103) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fbd4c083340 con 0x7fbd5c0b0490 2023-11-26T07:24:33.681 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:33.683+0000 7fbd53fff640 1 --2- 172.21.15.143:0/2192167107 >> [v2:172.21.15.188:6836/3401128553,v1:172.21.15.188:6837/3401128553] conn(0x7fbd3c066740 0x7fbd3c068c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:24:33.682 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:33.683+0000 7fbd53fff640 1 --2- 172.21.15.143:0/2192167107 >> [v2:172.21.15.188:6836/3401128553,v1:172.21.15.188:6837/3401128553] conn(0x7fbd3c066740 0x7fbd3c068c00 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7fbd440af000 tx=0x7fbd440a7000 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:24:33.685 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:33.683+0000 7fbd537fe640 1 -- 172.21.15.143:0/2192167107 <== mon.0 v2:172.21.15.143:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172649 (secure 0 0 0) 0x7fbd4c054040 con 0x7fbd5c0b0490 2023-11-26T07:24:33.863 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:33.863+0000 7fbd614c9640 1 -- 172.21.15.143:0/2192167107 --> [v2:172.21.15.188:6836/3401128553,v1:172.21.15.188:6837/3401128553] -- mgr_command(tid 0: {"prefix": "fs subvolume snapshot clone", "vol_name": "cephfs", "sub_name": "subvol24520", "snap_name": "fake", "target_sub_name": "subvol_clone15961", "target": ["mon-mgr", ""]}) v1 -- 0x7fbd5c0d13b0 con 0x7fbd3c066740 2023-11-26T07:24:33.937 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:33.939+0000 7fbd537fe640 1 -- 172.21.15.143:0/2192167107 <== mgr.4108 v2:172.21.15.188:6836/3401128553 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+0 (secure 0 0 0) 0x7fbd440ab9e0 con 0x7fbd3c066740 2023-11-26T07:24:33.940 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:33.939+0000 7fbd614c9640 1 -- 172.21.15.143:0/2192167107 >> [v2:172.21.15.188:6836/3401128553,v1:172.21.15.188:6837/3401128553] conn(0x7fbd3c066740 msgr2=0x7fbd3c068c00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:24:33.940 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:33.939+0000 7fbd614c9640 1 --2- 172.21.15.143:0/2192167107 >> [v2:172.21.15.188:6836/3401128553,v1:172.21.15.188:6837/3401128553] conn(0x7fbd3c066740 0x7fbd3c068c00 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7fbd440af000 tx=0x7fbd440a7000 comp rx=0 tx=0).stop 2023-11-26T07:24:33.940 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:33.939+0000 7fbd614c9640 1 -- 172.21.15.143:0/2192167107 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fbd5c0b0490 msgr2=0x7fbd5c153e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:24:33.940 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:33.939+0000 7fbd614c9640 1 --2- 172.21.15.143:0/2192167107 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fbd5c0b0490 0x7fbd5c153e00 secure :-1 s=READY pgs=1371 cs=0 l=1 rev1=1 crypto rx=0x7fbd4c00c480 tx=0x7fbd4c00c940 comp rx=0 tx=0).stop 2023-11-26T07:24:33.940 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:33.939+0000 7fbd614c9640 1 -- 172.21.15.143:0/2192167107 shutdown_connections 2023-11-26T07:24:33.940 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:33.939+0000 7fbd614c9640 1 --2- 172.21.15.143:0/2192167107 >> [v2:172.21.15.188:6836/3401128553,v1:172.21.15.188:6837/3401128553] conn(0x7fbd3c066740 0x7fbd3c068c00 unknown :-1 s=CLOSED pgs=152 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:24:33.940 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:33.939+0000 7fbd614c9640 1 --2- 172.21.15.143:0/2192167107 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fbd5c15d580 0x7fbd5c06b6e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:24:33.940 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:33.939+0000 7fbd614c9640 1 --2- 172.21.15.143:0/2192167107 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fbd5c0d17f0 0x7fbd5c06b1a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:24:33.940 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:33.939+0000 7fbd614c9640 1 --2- 172.21.15.143:0/2192167107 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fbd5c0b0490 0x7fbd5c153e00 unknown :-1 s=CLOSED pgs=1371 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:24:33.940 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:33.939+0000 7fbd614c9640 1 -- 172.21.15.143:0/2192167107 >> 172.21.15.143:0/2192167107 conn(0x7fbd5c09c550 msgr2=0x7fbd5c06c2d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:24:33.941 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:33.939+0000 7fbd614c9640 1 -- 172.21.15.143:0/2192167107 shutdown_connections 2023-11-26T07:24:33.941 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:33.939+0000 7fbd614c9640 1 -- 172.21.15.143:0/2192167107 wait complete. 2023-11-26T07:24:33.961 DEBUG:teuthology.orchestra.run.smithi143:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs clone status cephfs subvol_clone15961 2023-11-26T07:24:34.102 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:34.103+0000 7f122686b640 1 Processor -- start 2023-11-26T07:24:34.102 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:34.103+0000 7f122686b640 1 -- start start 2023-11-26T07:24:34.104 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:34.103+0000 7f122686b640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f12201596d0 0x7f1220071850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:24:34.104 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:34.103+0000 7f122686b640 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7f12200ce680 0x7f12200b2870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:24:34.104 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:34.103+0000 7f122686b640 1 -- --> v1:172.21.15.143:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f122006c750 con 0x7f12200b2db0 2023-11-26T07:24:34.104 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:34.103+0000 7f122686b640 1 -- --> v2:172.21.15.188:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f12200b3240 con 0x7f12200ce680 2023-11-26T07:24:34.104 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:34.103+0000 7f122686b640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f12200cbe00 con 0x7f12201596d0 2023-11-26T07:24:34.105 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:34.103+0000 7f1224e08640 1 --1- >> v1:172.21.15.143:6789/0 conn(0x7f12200b2db0 0x7f122015d2c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.143:6789/0 says I am v1:172.21.15.143:39380/0 (socket says 172.21.15.143:39380) 2023-11-26T07:24:34.105 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:34.103+0000 7f1224e08640 1 -- 172.21.15.143:0/3764473149 learned_addr learned my addr 172.21.15.143:0/3764473149 (peer_addr_for_me v1:172.21.15.143:0/0) 2023-11-26T07:24:34.105 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:34.103+0000 7f121ffff640 1 --2- 172.21.15.143:0/3764473149 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f12201596d0 0x7f1220071850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:24:34.105 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:34.103+0000 7f121f7fe640 1 --2- 172.21.15.143:0/3764473149 >> v2:172.21.15.188:3300/0 conn(0x7f12200ce680 0x7f12200b2870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:24:34.105 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:34.103+0000 7f121effd640 1 -- 172.21.15.143:0/3764473149 <== mon.0 v1:172.21.15.143:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2564974392 0 0) 0x7f122006c750 con 0x7f12200b2db0 2023-11-26T07:24:34.105 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:34.103+0000 7f121effd640 1 -- 172.21.15.143:0/3764473149 --> v1:172.21.15.143:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f1208003530 con 0x7f12200b2db0 2023-11-26T07:24:34.105 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:34.103+0000 7f121f7fe640 1 -- 172.21.15.143:0/3764473149 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f12201596d0 msgr2=0x7f1220071850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-26T07:24:34.105 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:34.103+0000 7f121f7fe640 1 --2- 172.21.15.143:0/3764473149 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f12201596d0 0x7f1220071850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:24:34.105 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:34.103+0000 7f121f7fe640 1 -- 172.21.15.143:0/3764473149 >> v1:172.21.15.143:6789/0 conn(0x7f12200b2db0 legacy=0x7f122015d2c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:24:34.105 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:34.103+0000 7f121f7fe640 1 -- 172.21.15.143:0/3764473149 --> v2:172.21.15.188:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f12200b3550 con 0x7f12200ce680 2023-11-26T07:24:34.105 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:34.103+0000 7f121ffff640 1 --2- 172.21.15.143:0/3764473149 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f12201596d0 0x7f1220071850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-26T07:24:34.107 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:34.107+0000 7f121f7fe640 1 --2- 172.21.15.143:0/3764473149 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f12200ce680 0x7f12200b2870 secure :-1 s=READY pgs=514 cs=0 l=1 rev1=1 crypto rx=0x7f12140097c0 tx=0x7f121401e690 comp rx=0 tx=0).ready entity=mon.1 client_cookie=2e7ba190336a169a server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:24:34.108 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:34.107+0000 7f121effd640 1 -- 172.21.15.143:0/3764473149 <== mon.1 v2:172.21.15.188:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f12140ac670 con 0x7f12200ce680 2023-11-26T07:24:34.108 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:34.107+0000 7f121effd640 1 -- 172.21.15.143:0/3764473149 <== mon.1 v2:172.21.15.188:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f12140ac7d0 con 0x7f12200ce680 2023-11-26T07:24:34.108 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:34.107+0000 7f121effd640 1 -- 172.21.15.143:0/3764473149 <== mon.1 v2:172.21.15.188:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f12140acb60 con 0x7f12200ce680 2023-11-26T07:24:34.108 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:34.107+0000 7f122686b640 1 -- 172.21.15.143:0/3764473149 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f12200ce680 msgr2=0x7f12200b2870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:24:34.108 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:34.107+0000 7f122686b640 1 --2- 172.21.15.143:0/3764473149 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f12200ce680 0x7f12200b2870 secure :-1 s=READY pgs=514 cs=0 l=1 rev1=1 crypto rx=0x7f12140097c0 tx=0x7f121401e690 comp rx=0 tx=0).stop 2023-11-26T07:24:34.109 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:34.107+0000 7f122686b640 1 -- 172.21.15.143:0/3764473149 shutdown_connections 2023-11-26T07:24:34.109 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:34.107+0000 7f122686b640 1 --2- 172.21.15.143:0/3764473149 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f12200ce680 0x7f12200b2870 unknown :-1 s=CLOSED pgs=514 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:24:34.109 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:34.107+0000 7f122686b640 1 --2- 172.21.15.143:0/3764473149 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f12201596d0 0x7f1220071850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:24:34.109 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:34.107+0000 7f122686b640 1 -- 172.21.15.143:0/3764473149 >> 172.21.15.143:0/3764473149 conn(0x7f122006a890 msgr2=0x7f122006aca0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:24:34.109 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:34.107+0000 7f122686b640 1 -- 172.21.15.143:0/3764473149 shutdown_connections 2023-11-26T07:24:34.109 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:34.107+0000 7f122686b640 1 -- 172.21.15.143:0/3764473149 wait complete. 2023-11-26T07:24:34.110 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:34.111+0000 7f122686b640 1 Processor -- start 2023-11-26T07:24:34.110 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:34.111+0000 7f122686b640 1 -- start start 2023-11-26T07:24:34.111 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:34.111+0000 7f122686b640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f12200b2db0 0x7f12201f9d30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:24:34.111 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:34.111+0000 7f122686b640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f12200ce680 0x7f12200cce50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:24:34.111 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:34.111+0000 7f122686b640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f12201596d0 0x7f12200cd390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:24:34.112 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:34.111+0000 7f122686b640 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1220148410 con 0x7f12200ce680 2023-11-26T07:24:34.112 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:34.111+0000 7f122686b640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1220147800 con 0x7f12201596d0 2023-11-26T07:24:34.112 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:34.111+0000 7f122686b640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1220147ae0 con 0x7f12200b2db0 2023-11-26T07:24:34.113 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:34.111+0000 7f121f7fe640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f12200ce680 0x7f12200cce50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:24:34.113 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:34.111+0000 7f1224e08640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f12201596d0 0x7f12200cd390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:24:34.113 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:34.111+0000 7f121ffff640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f12200b2db0 0x7f12201f9d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:24:34.113 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:34.111+0000 7f121ffff640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f12200b2db0 0x7f12201f9d30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.188:3301/0 says I am v2:172.21.15.143:40782/0 (socket says 172.21.15.143:40782) 2023-11-26T07:24:34.113 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:34.111+0000 7f121ffff640 1 -- 172.21.15.143:0/763342544 learned_addr learned my addr 172.21.15.143:0/763342544 (peer_addr_for_me v2:172.21.15.143:0/0) 2023-11-26T07:24:34.113 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:34.111+0000 7f1224e08640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f12201596d0 0x7f12200cd390 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.188:3300/0 says I am v2:172.21.15.143:52138/0 (socket says 172.21.15.143:52138) 2023-11-26T07:24:34.113 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:34.111+0000 7f121f7fe640 1 -- 172.21.15.143:0/763342544 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f12200b2db0 msgr2=0x7f12201f9d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:24:34.114 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:34.111+0000 7f121f7fe640 1 --2- 172.21.15.143:0/763342544 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f12200b2db0 0x7f12201f9d30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:24:34.114 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:34.111+0000 7f121f7fe640 1 -- 172.21.15.143:0/763342544 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f12201596d0 msgr2=0x7f12200cd390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:24:34.114 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:34.111+0000 7f121f7fe640 1 --2- 172.21.15.143:0/763342544 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f12201596d0 0x7f12200cd390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:24:34.114 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:34.111+0000 7f121f7fe640 1 -- 172.21.15.143:0/763342544 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f121000a5c0 con 0x7f12200ce680 2023-11-26T07:24:34.114 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:34.111+0000 7f121f7fe640 1 --2- 172.21.15.143:0/763342544 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f12200ce680 0x7f12200cce50 secure :-1 s=READY pgs=1373 cs=0 l=1 rev1=1 crypto rx=0x7f12140ad000 tx=0x7f121401ced0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:24:34.114 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:34.111+0000 7f121cff9640 1 -- 172.21.15.143:0/763342544 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f12140b5620 con 0x7f12200ce680 2023-11-26T07:24:34.114 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:34.111+0000 7f121cff9640 1 -- 172.21.15.143:0/763342544 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f12140b5780 con 0x7f12200ce680 2023-11-26T07:24:34.114 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:34.111+0000 7f122686b640 1 -- 172.21.15.143:0/763342544 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1214009420 con 0x7f12200ce680 2023-11-26T07:24:34.114 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:34.111+0000 7f121cff9640 1 -- 172.21.15.143:0/763342544 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f12140b5b30 con 0x7f12200ce680 2023-11-26T07:24:34.114 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:34.111+0000 7f122686b640 1 -- 172.21.15.143:0/763342544 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f12201480a0 con 0x7f12200ce680 2023-11-26T07:24:34.115 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:34.115+0000 7f122686b640 1 -- 172.21.15.143:0/763342544 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1220067680 con 0x7f12200ce680 2023-11-26T07:24:34.115 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:34.115+0000 7f121cff9640 1 -- 172.21.15.143:0/763342544 <== mon.0 v2:172.21.15.143:3300/0 4 ==== mgrmap(e 18) v1 ==== 86107+0+0 (secure 0 0 0) 0x7f12140b5c90 con 0x7f12200ce680 2023-11-26T07:24:34.115 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:34.115+0000 7f121cff9640 1 --2- 172.21.15.143:0/763342544 >> [v2:172.21.15.188:6836/3401128553,v1:172.21.15.188:6837/3401128553] conn(0x7f120c066740 0x7f120c068c00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:24:34.116 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:34.115+0000 7f121cff9640 1 -- 172.21.15.143:0/763342544 <== mon.0 v2:172.21.15.143:3300/0 5 ==== osd_map(103..103 src has 1..103) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f121411ebf0 con 0x7f12200ce680 2023-11-26T07:24:34.116 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:34.115+0000 7f121ffff640 1 --2- 172.21.15.143:0/763342544 >> [v2:172.21.15.188:6836/3401128553,v1:172.21.15.188:6837/3401128553] conn(0x7f120c066740 0x7f120c068c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:24:34.117 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:34.115+0000 7f121ffff640 1 --2- 172.21.15.143:0/763342544 >> [v2:172.21.15.188:6836/3401128553,v1:172.21.15.188:6837/3401128553] conn(0x7f120c066740 0x7f120c068c00 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7f121000ae50 tx=0x7f121000a2a0 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:24:34.119 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:34.119+0000 7f121cff9640 1 -- 172.21.15.143:0/763342544 <== mon.0 v2:172.21.15.143:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172649 (secure 0 0 0) 0x7f12140ebd10 con 0x7f12200ce680 2023-11-26T07:24:34.298 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:34.299+0000 7f122686b640 1 -- 172.21.15.143:0/763342544 --> [v2:172.21.15.188:6836/3401128553,v1:172.21.15.188:6837/3401128553] -- mgr_command(tid 0: {"prefix": "fs clone status", "vol_name": "cephfs", "clone_name": "subvol_clone15961", "target": ["mon-mgr", ""]}) v1 -- 0x7f1220159ab0 con 0x7f120c066740 2023-11-26T07:24:38.090 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:38.091+0000 7f121cff9640 1 -- 172.21.15.143:0/763342544 <== mgr.4108 v2:172.21.15.188:6836/3401128553 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+153 (secure 0 0 0) 0x7f121000aaa0 con 0x7f120c066740 2023-11-26T07:24:38.090 INFO:teuthology.orchestra.run.smithi143.stdout:{ 2023-11-26T07:24:38.091 INFO:teuthology.orchestra.run.smithi143.stdout: "status": { 2023-11-26T07:24:38.091 INFO:teuthology.orchestra.run.smithi143.stdout: "state": "pending", 2023-11-26T07:24:38.091 INFO:teuthology.orchestra.run.smithi143.stdout: "source": { 2023-11-26T07:24:38.091 INFO:teuthology.orchestra.run.smithi143.stdout: "volume": "cephfs", 2023-11-26T07:24:38.091 INFO:teuthology.orchestra.run.smithi143.stdout: "subvolume": "subvol24520", 2023-11-26T07:24:38.091 INFO:teuthology.orchestra.run.smithi143.stdout: "snapshot": "fake" 2023-11-26T07:24:38.091 INFO:teuthology.orchestra.run.smithi143.stdout: } 2023-11-26T07:24:38.091 INFO:teuthology.orchestra.run.smithi143.stdout: } 2023-11-26T07:24:38.091 INFO:teuthology.orchestra.run.smithi143.stdout:} 2023-11-26T07:24:38.093 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:38.091+0000 7f122686b640 1 -- 172.21.15.143:0/763342544 >> [v2:172.21.15.188:6836/3401128553,v1:172.21.15.188:6837/3401128553] conn(0x7f120c066740 msgr2=0x7f120c068c00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:24:38.093 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:38.091+0000 7f122686b640 1 --2- 172.21.15.143:0/763342544 >> [v2:172.21.15.188:6836/3401128553,v1:172.21.15.188:6837/3401128553] conn(0x7f120c066740 0x7f120c068c00 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7f121000ae50 tx=0x7f121000a2a0 comp rx=0 tx=0).stop 2023-11-26T07:24:38.093 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:38.091+0000 7f122686b640 1 -- 172.21.15.143:0/763342544 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f12200ce680 msgr2=0x7f12200cce50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:24:38.093 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:38.091+0000 7f122686b640 1 --2- 172.21.15.143:0/763342544 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f12200ce680 0x7f12200cce50 secure :-1 s=READY pgs=1373 cs=0 l=1 rev1=1 crypto rx=0x7f12140ad000 tx=0x7f121401ced0 comp rx=0 tx=0).stop 2023-11-26T07:24:38.094 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:38.091+0000 7f122686b640 1 -- 172.21.15.143:0/763342544 shutdown_connections 2023-11-26T07:24:38.094 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:38.091+0000 7f122686b640 1 --2- 172.21.15.143:0/763342544 >> [v2:172.21.15.188:6836/3401128553,v1:172.21.15.188:6837/3401128553] conn(0x7f120c066740 0x7f120c068c00 unknown :-1 s=CLOSED pgs=153 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:24:38.094 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:38.091+0000 7f122686b640 1 --2- 172.21.15.143:0/763342544 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f12201596d0 0x7f12200cd390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:24:38.094 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:38.091+0000 7f122686b640 1 --2- 172.21.15.143:0/763342544 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f12200ce680 0x7f12200cce50 unknown :-1 s=CLOSED pgs=1373 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:24:38.094 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:38.091+0000 7f122686b640 1 --2- 172.21.15.143:0/763342544 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f12200b2db0 0x7f12201f9d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:24:38.094 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:38.091+0000 7f122686b640 1 -- 172.21.15.143:0/763342544 >> 172.21.15.143:0/763342544 conn(0x7f122006a890 msgr2=0x7f12200cdbb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:24:38.094 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:38.095+0000 7f122686b640 1 -- 172.21.15.143:0/763342544 shutdown_connections 2023-11-26T07:24:38.094 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:38.095+0000 7f122686b640 1 -- 172.21.15.143:0/763342544 wait complete. 2023-11-26T07:24:39.120 DEBUG:teuthology.orchestra.run.smithi143:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs clone status cephfs subvol_clone15961 2023-11-26T07:24:39.273 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:39.271+0000 7fdb49de7640 1 Processor -- start 2023-11-26T07:24:39.273 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:39.271+0000 7fdb49de7640 1 -- start start 2023-11-26T07:24:39.274 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:39.275+0000 7fdb49de7640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fdb440cf280 0x7fdb440cf6e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:24:39.274 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:39.275+0000 7fdb49de7640 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7fdb440cd1b0 0x7fdb440cd5e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:24:39.275 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:39.275+0000 7fdb49de7640 1 -- --> v1:172.21.15.143:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fdb440cfc20 con 0x7fdb44143ac0 2023-11-26T07:24:39.275 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:39.275+0000 7fdb49de7640 1 -- --> v2:172.21.15.188:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fdb44143ea0 con 0x7fdb440cd1b0 2023-11-26T07:24:39.275 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:39.275+0000 7fdb49de7640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdb440cc730 con 0x7fdb440cf280 2023-11-26T07:24:39.275 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:39.275+0000 7fdb437fe640 1 --1- >> v1:172.21.15.143:6789/0 conn(0x7fdb44143ac0 0x7fdb44070370 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.143:6789/0 says I am v1:172.21.15.143:55348/0 (socket says 172.21.15.143:55348) 2023-11-26T07:24:39.275 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:39.275+0000 7fdb437fe640 1 -- 172.21.15.143:0/4152496446 learned_addr learned my addr 172.21.15.143:0/4152496446 (peer_addr_for_me v1:172.21.15.143:0/0) 2023-11-26T07:24:39.277 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:39.275+0000 7fdb42ffd640 1 --2- 172.21.15.143:0/4152496446 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fdb440cf280 0x7fdb440cf6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:24:39.277 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:39.275+0000 7fdb43fff640 1 --2- 172.21.15.143:0/4152496446 >> v2:172.21.15.188:3300/0 conn(0x7fdb440cd1b0 0x7fdb440cd5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:24:39.277 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:39.275+0000 7fdb427fc640 1 -- 172.21.15.143:0/4152496446 <== mon.0 v1:172.21.15.143:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 106992588 0 0) 0x7fdb440cfc20 con 0x7fdb44143ac0 2023-11-26T07:24:39.277 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:39.275+0000 7fdb427fc640 1 -- 172.21.15.143:0/4152496446 --> v1:172.21.15.143:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fdb2c003530 con 0x7fdb44143ac0 2023-11-26T07:24:39.277 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:39.275+0000 7fdb427fc640 1 -- 172.21.15.143:0/4152496446 <== mon.0 v1:172.21.15.143:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2992825756 0 0) 0x7fdb2c003530 con 0x7fdb44143ac0 2023-11-26T07:24:39.277 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:39.275+0000 7fdb427fc640 1 -- 172.21.15.143:0/4152496446 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fdb440cf280 msgr2=0x7fdb440cf6e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-26T07:24:39.277 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:39.275+0000 7fdb427fc640 1 --2- 172.21.15.143:0/4152496446 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fdb440cf280 0x7fdb440cf6e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:24:39.277 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:39.275+0000 7fdb427fc640 1 -- 172.21.15.143:0/4152496446 >> v2:172.21.15.188:3300/0 conn(0x7fdb440cd1b0 msgr2=0x7fdb440cd5e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-26T07:24:39.278 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:39.275+0000 7fdb427fc640 1 --2- 172.21.15.143:0/4152496446 >> v2:172.21.15.188:3300/0 conn(0x7fdb440cd1b0 0x7fdb440cd5e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:24:39.278 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:39.275+0000 7fdb427fc640 1 -- 172.21.15.143:0/4152496446 --> v1:172.21.15.143:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdb44070ee0 con 0x7fdb44143ac0 2023-11-26T07:24:39.278 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:39.275+0000 7fdb427fc640 1 -- 172.21.15.143:0/4152496446 <== mon.0 v1:172.21.15.143:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2137882523 0 0) 0x7fdb34001680 con 0x7fdb44143ac0 2023-11-26T07:24:39.278 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:39.275+0000 7fdb427fc640 1 -- 172.21.15.143:0/4152496446 >> v1:172.21.15.143:6789/0 conn(0x7fdb44143ac0 legacy=0x7fdb44070370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:24:39.278 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:39.275+0000 7fdb427fc640 1 --2- 172.21.15.143:0/4152496446 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fdb2c003980 0x7fdb2c003da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:24:39.278 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:39.275+0000 7fdb427fc640 1 --2- 172.21.15.143:0/4152496446 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fdb2c0047d0 0x7fdb2c004c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:24:39.278 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:39.275+0000 7fdb427fc640 1 --2- 172.21.15.143:0/4152496446 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fdb440cf280 0x7fdb2c005190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:24:39.278 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:39.275+0000 7fdb427fc640 1 -- 172.21.15.143:0/4152496446 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdb440cc730 con 0x7fdb2c0047d0 2023-11-26T07:24:39.278 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:39.275+0000 7fdb427fc640 1 -- 172.21.15.143:0/4152496446 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdb2c00b990 con 0x7fdb2c003980 2023-11-26T07:24:39.278 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:39.275+0000 7fdb427fc640 1 -- 172.21.15.143:0/4152496446 --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdb2c00bad0 con 0x7fdb440cf280 2023-11-26T07:24:39.278 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:39.275+0000 7fdb42ffd640 1 --2- 172.21.15.143:0/4152496446 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fdb2c0047d0 0x7fdb2c004c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:24:39.279 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:39.275+0000 7fdb437fe640 1 --2- 172.21.15.143:0/4152496446 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fdb2c003980 0x7fdb2c003da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:24:39.279 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:39.275+0000 7fdb43fff640 1 --2- 172.21.15.143:0/4152496446 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fdb440cf280 0x7fdb2c005190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:24:39.279 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:39.275+0000 7fdb42ffd640 1 -- 172.21.15.143:0/4152496446 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fdb440cf280 msgr2=0x7fdb2c005190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-26T07:24:39.279 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:39.275+0000 7fdb42ffd640 1 --2- 172.21.15.143:0/4152496446 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fdb440cf280 0x7fdb2c005190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:24:39.279 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:39.275+0000 7fdb42ffd640 1 -- 172.21.15.143:0/4152496446 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fdb2c003980 msgr2=0x7fdb2c003da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-26T07:24:39.279 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:39.275+0000 7fdb42ffd640 1 --2- 172.21.15.143:0/4152496446 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fdb2c003980 0x7fdb2c003da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:24:39.279 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:39.275+0000 7fdb42ffd640 1 -- 172.21.15.143:0/4152496446 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdb2c003530 con 0x7fdb2c0047d0 2023-11-26T07:24:39.280 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:39.279+0000 7fdb42ffd640 1 --2- 172.21.15.143:0/4152496446 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fdb2c0047d0 0x7fdb2c004c50 secure :-1 s=READY pgs=1375 cs=0 l=1 rev1=1 crypto rx=0x7fdb38009670 tx=0x7fdb3801ec80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ca842c82acff0571 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:24:39.280 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:39.279+0000 7fdb427fc640 1 -- 172.21.15.143:0/4152496446 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdb380a77a0 con 0x7fdb2c0047d0 2023-11-26T07:24:39.280 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:39.279+0000 7fdb427fc640 1 -- 172.21.15.143:0/4152496446 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdb380a7900 con 0x7fdb2c0047d0 2023-11-26T07:24:39.281 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:39.279+0000 7fdb427fc640 1 -- 172.21.15.143:0/4152496446 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdb380a7cb0 con 0x7fdb2c0047d0 2023-11-26T07:24:39.281 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:39.279+0000 7fdb49de7640 1 -- 172.21.15.143:0/4152496446 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fdb2c0047d0 msgr2=0x7fdb2c004c50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:24:39.281 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:39.279+0000 7fdb49de7640 1 --2- 172.21.15.143:0/4152496446 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fdb2c0047d0 0x7fdb2c004c50 secure :-1 s=READY pgs=1375 cs=0 l=1 rev1=1 crypto rx=0x7fdb38009670 tx=0x7fdb3801ec80 comp rx=0 tx=0).stop 2023-11-26T07:24:39.281 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:39.279+0000 7fdb43fff640 1 -- 172.21.15.143:0/4152496446 reap_dead start 2023-11-26T07:24:39.281 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:39.279+0000 7fdb49de7640 1 -- 172.21.15.143:0/4152496446 shutdown_connections 2023-11-26T07:24:39.281 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:39.279+0000 7fdb49de7640 1 -- 172.21.15.143:0/4152496446 >> 172.21.15.143:0/4152496446 conn(0x7fdb4406a890 msgr2=0x7fdb4406aca0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:24:39.281 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:39.279+0000 7fdb49de7640 1 -- 172.21.15.143:0/4152496446 shutdown_connections 2023-11-26T07:24:39.281 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:39.279+0000 7fdb49de7640 1 -- 172.21.15.143:0/4152496446 wait complete. 2023-11-26T07:24:39.281 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:39.283+0000 7fdb49de7640 1 Processor -- start 2023-11-26T07:24:39.282 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:39.283+0000 7fdb49de7640 1 -- start start 2023-11-26T07:24:39.283 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:39.283+0000 7fdb49de7640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fdb2c0047d0 0x7fdb4413e330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:24:39.283 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:39.283+0000 7fdb49de7640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fdb4413b550 0x7fdb441f3490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:24:39.283 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:39.283+0000 7fdb49de7640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fdb4415c0a0 0x7fdb440ac8e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:24:39.283 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:39.283+0000 7fdb49de7640 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdb441f2cd0 con 0x7fdb4415c0a0 2023-11-26T07:24:39.283 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:39.283+0000 7fdb49de7640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdb4413ea10 con 0x7fdb2c0047d0 2023-11-26T07:24:39.283 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:39.283+0000 7fdb49de7640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdb4413ecf0 con 0x7fdb4413b550 2023-11-26T07:24:39.285 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:39.283+0000 7fdb43fff640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fdb4415c0a0 0x7fdb440ac8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:24:39.285 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:39.283+0000 7fdb43fff640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fdb4415c0a0 0x7fdb440ac8e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:33272/0 (socket says 172.21.15.143:33272) 2023-11-26T07:24:39.285 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:39.283+0000 7fdb43fff640 1 -- 172.21.15.143:0/3660006086 learned_addr learned my addr 172.21.15.143:0/3660006086 (peer_addr_for_me v2:172.21.15.143:0/0) 2023-11-26T07:24:39.285 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:39.283+0000 7fdb42ffd640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fdb4413b550 0x7fdb441f3490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:24:39.285 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:39.283+0000 7fdb437fe640 1 --2- 172.21.15.143:0/3660006086 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fdb2c0047d0 0x7fdb4413e330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:24:39.285 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:39.283+0000 7fdb43fff640 1 -- 172.21.15.143:0/3660006086 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fdb4413b550 msgr2=0x7fdb441f3490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:24:39.285 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:39.283+0000 7fdb43fff640 1 --2- 172.21.15.143:0/3660006086 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fdb4413b550 0x7fdb441f3490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:24:39.285 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:39.283+0000 7fdb43fff640 1 -- 172.21.15.143:0/3660006086 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fdb2c0047d0 msgr2=0x7fdb4413e330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:24:39.285 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:39.283+0000 7fdb43fff640 1 --2- 172.21.15.143:0/3660006086 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fdb2c0047d0 0x7fdb4413e330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:24:39.285 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:39.283+0000 7fdb43fff640 1 -- 172.21.15.143:0/3660006086 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdb441f2f70 con 0x7fdb4415c0a0 2023-11-26T07:24:39.286 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:39.283+0000 7fdb43fff640 1 --2- 172.21.15.143:0/3660006086 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fdb4415c0a0 0x7fdb440ac8e0 secure :-1 s=READY pgs=1376 cs=0 l=1 rev1=1 crypto rx=0x7fdb4406e880 tx=0x7fdb3c00f580 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:24:39.286 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:39.283+0000 7fdb2bfff640 1 -- 172.21.15.143:0/3660006086 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdb3c00e040 con 0x7fdb4415c0a0 2023-11-26T07:24:39.286 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:39.283+0000 7fdb2bfff640 1 -- 172.21.15.143:0/3660006086 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdb3c00f040 con 0x7fdb4415c0a0 2023-11-26T07:24:39.286 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:39.283+0000 7fdb2bfff640 1 -- 172.21.15.143:0/3660006086 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdb3c015670 con 0x7fdb4415c0a0 2023-11-26T07:24:39.286 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:39.283+0000 7fdb49de7640 1 -- 172.21.15.143:0/3660006086 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdb4413efd0 con 0x7fdb4415c0a0 2023-11-26T07:24:39.286 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:39.283+0000 7fdb49de7640 1 -- 172.21.15.143:0/3660006086 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdb441f39d0 con 0x7fdb4415c0a0 2023-11-26T07:24:39.286 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:39.287+0000 7fdb49de7640 1 -- 172.21.15.143:0/3660006086 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fdb441f3160 con 0x7fdb4415c0a0 2023-11-26T07:24:39.287 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:39.287+0000 7fdb2bfff640 1 -- 172.21.15.143:0/3660006086 <== mon.0 v2:172.21.15.143:3300/0 4 ==== mgrmap(e 18) v1 ==== 86107+0+0 (secure 0 0 0) 0x7fdb3c0157d0 con 0x7fdb4415c0a0 2023-11-26T07:24:39.287 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:39.287+0000 7fdb2bfff640 1 --2- 172.21.15.143:0/3660006086 >> [v2:172.21.15.188:6836/3401128553,v1:172.21.15.188:6837/3401128553] conn(0x7fdb30066740 0x7fdb30068c00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:24:39.287 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:39.287+0000 7fdb2bfff640 1 -- 172.21.15.143:0/3660006086 <== mon.0 v2:172.21.15.143:3300/0 5 ==== osd_map(103..103 src has 1..103) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fdb3c084d50 con 0x7fdb4415c0a0 2023-11-26T07:24:39.288 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:39.287+0000 7fdb437fe640 1 --2- 172.21.15.143:0/3660006086 >> [v2:172.21.15.188:6836/3401128553,v1:172.21.15.188:6837/3401128553] conn(0x7fdb30066740 0x7fdb30068c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:24:39.288 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:39.287+0000 7fdb437fe640 1 --2- 172.21.15.143:0/3660006086 >> [v2:172.21.15.188:6836/3401128553,v1:172.21.15.188:6837/3401128553] conn(0x7fdb30066740 0x7fdb30068c00 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7fdb4413bfb0 tx=0x7fdb34007900 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:24:39.291 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:39.291+0000 7fdb2bfff640 1 -- 172.21.15.143:0/3660006086 <== mon.0 v2:172.21.15.143:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172649 (secure 0 0 0) 0x7fdb3c051e00 con 0x7fdb4415c0a0 2023-11-26T07:24:39.475 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:39.474+0000 7fdb49de7640 1 -- 172.21.15.143:0/3660006086 --> [v2:172.21.15.188:6836/3401128553,v1:172.21.15.188:6837/3401128553] -- mgr_command(tid 0: {"prefix": "fs clone status", "vol_name": "cephfs", "clone_name": "subvol_clone15961", "target": ["mon-mgr", ""]}) v1 -- 0x7fdb440ccc90 con 0x7fdb30066740 2023-11-26T07:24:43.060 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:43.058+0000 7fdb2bfff640 1 -- 172.21.15.143:0/3660006086 <== mgr.4108 v2:172.21.15.188:6836/3401128553 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+46 (secure 0 0 0) 0x7fdb340015a0 con 0x7fdb30066740 2023-11-26T07:24:43.060 INFO:teuthology.orchestra.run.smithi143.stdout:{ 2023-11-26T07:24:43.060 INFO:teuthology.orchestra.run.smithi143.stdout: "status": { 2023-11-26T07:24:43.060 INFO:teuthology.orchestra.run.smithi143.stdout: "state": "complete" 2023-11-26T07:24:43.060 INFO:teuthology.orchestra.run.smithi143.stdout: } 2023-11-26T07:24:43.060 INFO:teuthology.orchestra.run.smithi143.stdout:} 2023-11-26T07:24:43.063 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:43.062+0000 7fdb49de7640 1 -- 172.21.15.143:0/3660006086 >> [v2:172.21.15.188:6836/3401128553,v1:172.21.15.188:6837/3401128553] conn(0x7fdb30066740 msgr2=0x7fdb30068c00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:24:43.063 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:43.062+0000 7fdb49de7640 1 --2- 172.21.15.143:0/3660006086 >> [v2:172.21.15.188:6836/3401128553,v1:172.21.15.188:6837/3401128553] conn(0x7fdb30066740 0x7fdb30068c00 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7fdb4413bfb0 tx=0x7fdb34007900 comp rx=0 tx=0).stop 2023-11-26T07:24:43.063 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:43.062+0000 7fdb49de7640 1 -- 172.21.15.143:0/3660006086 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fdb4415c0a0 msgr2=0x7fdb440ac8e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:24:43.063 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:43.062+0000 7fdb49de7640 1 --2- 172.21.15.143:0/3660006086 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fdb4415c0a0 0x7fdb440ac8e0 secure :-1 s=READY pgs=1376 cs=0 l=1 rev1=1 crypto rx=0x7fdb4406e880 tx=0x7fdb3c00f580 comp rx=0 tx=0).stop 2023-11-26T07:24:43.063 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:43.062+0000 7fdb49de7640 1 -- 172.21.15.143:0/3660006086 shutdown_connections 2023-11-26T07:24:43.063 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:43.062+0000 7fdb49de7640 1 --2- 172.21.15.143:0/3660006086 >> [v2:172.21.15.188:6836/3401128553,v1:172.21.15.188:6837/3401128553] conn(0x7fdb30066740 0x7fdb30068c00 unknown :-1 s=CLOSED pgs=154 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:24:43.063 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:43.062+0000 7fdb49de7640 1 --2- 172.21.15.143:0/3660006086 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fdb4415c0a0 0x7fdb440ac8e0 unknown :-1 s=CLOSED pgs=1376 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:24:43.063 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:43.062+0000 7fdb49de7640 1 --2- 172.21.15.143:0/3660006086 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fdb4413b550 0x7fdb441f3490 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:24:43.064 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:43.062+0000 7fdb49de7640 1 --2- 172.21.15.143:0/3660006086 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fdb2c0047d0 0x7fdb4413e330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:24:43.064 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:43.062+0000 7fdb49de7640 1 -- 172.21.15.143:0/3660006086 >> 172.21.15.143:0/3660006086 conn(0x7fdb4406a890 msgr2=0x7fdb440cf6e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:24:43.064 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:43.062+0000 7fdb49de7640 1 -- 172.21.15.143:0/3660006086 shutdown_connections 2023-11-26T07:24:43.064 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:43.062+0000 7fdb49de7640 1 -- 172.21.15.143:0/3660006086 wait complete. 2023-11-26T07:24:43.086 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-26T07:24:43.087 DEBUG:teuthology.orchestra.run.smithi143:> (cd /home/ubuntu/cephtest/mnt.0 && exec sudo cat ./volumes/_nogroup/subvol_clone15961/.meta) 2023-11-26T07:24:43.111 INFO:teuthology.orchestra.run.smithi143.stdout:[GLOBAL] 2023-11-26T07:24:43.112 INFO:teuthology.orchestra.run.smithi143.stdout:version = 2 2023-11-26T07:24:43.112 INFO:teuthology.orchestra.run.smithi143.stdout:type = clone 2023-11-26T07:24:43.112 INFO:teuthology.orchestra.run.smithi143.stdout:path = /volumes/_nogroup/subvol_clone15961/91b92a2d-42eb-4fb9-b67d-993d9e51e972 2023-11-26T07:24:43.112 INFO:teuthology.orchestra.run.smithi143.stdout:state = complete 2023-11-26T07:24:43.112 INFO:teuthology.orchestra.run.smithi143.stdout: 2023-11-26T07:24:43.112 INFO:teuthology.orchestra.run.smithi143.stdout:[source] 2023-11-26T07:24:43.112 INFO:teuthology.orchestra.run.smithi143.stdout:volume = cephfs 2023-11-26T07:24:43.112 INFO:teuthology.orchestra.run.smithi143.stdout:subvolume = subvol24520 2023-11-26T07:24:43.112 INFO:teuthology.orchestra.run.smithi143.stdout:snapshot = fake 2023-11-26T07:24:43.112 INFO:teuthology.orchestra.run.smithi143.stdout: 2023-11-26T07:24:43.114 DEBUG:teuthology.orchestra.run.smithi143:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume snapshot info cephfs subvol24520 subvol_snap55360 2023-11-26T07:24:43.300 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:43.298+0000 7f7d540e3640 1 Processor -- start 2023-11-26T07:24:43.300 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:43.298+0000 7f7d540e3640 1 -- start start 2023-11-26T07:24:43.301 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:43.302+0000 7f7d540e3640 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7f7d4c163520 0x7f7d4c16c3a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:24:43.301 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:43.302+0000 7f7d540e3640 1 -- --> v1:172.21.15.143:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7d4c162080 con 0x7f7d4c0cf990 2023-11-26T07:24:43.302 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:43.302+0000 7f7d540e3640 1 -- --> v1:172.21.15.188:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7d4c161d70 con 0x7f7d4c14b1e0 2023-11-26T07:24:43.302 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:43.302+0000 7f7d540e3640 1 -- --> v2:172.21.15.143:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7d4c14b5c0 con 0x7f7d4c163520 2023-11-26T07:24:43.302 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:43.302+0000 7f7d52680640 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7f7d4c163520 0x7f7d4c16c3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:24:43.302 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:43.302+0000 7f7d5167e640 1 --1- >> v1:172.21.15.143:6789/0 conn(0x7f7d4c0cf990 0x7f7d4c0708c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.143:6789/0 says I am v1:172.21.15.143:55358/0 (socket says 172.21.15.143:55358) 2023-11-26T07:24:43.302 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:43.302+0000 7f7d5167e640 1 -- 172.21.15.143:0/1535681957 learned_addr learned my addr 172.21.15.143:0/1535681957 (peer_addr_for_me v1:172.21.15.143:0/0) 2023-11-26T07:24:43.302 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:43.302+0000 7f7d52680640 1 -- 172.21.15.143:0/1535681957 >> v1:172.21.15.188:6789/0 conn(0x7f7d4c14b1e0 legacy=0x7f7d4c0cf280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:24:43.303 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:43.302+0000 7f7d52680640 1 -- 172.21.15.143:0/1535681957 >> v1:172.21.15.143:6789/0 conn(0x7f7d4c0cf990 legacy=0x7f7d4c0708c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:24:43.303 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:43.302+0000 7f7d52680640 1 -- 172.21.15.143:0/1535681957 --> v2:172.21.15.143:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7d4c071580 con 0x7f7d4c163520 2023-11-26T07:24:43.304 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:43.302+0000 7f7d52680640 1 --2- 172.21.15.143:0/1535681957 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f7d4c163520 0x7f7d4c16c3a0 secure :-1 s=READY pgs=1378 cs=0 l=1 rev1=1 crypto rx=0x7f7d44009d90 tx=0x7f7d4401c920 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f6454e7827342715 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:24:43.305 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:43.302+0000 7f7d50e7d640 1 -- 172.21.15.143:0/1535681957 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7d440a59e0 con 0x7f7d4c163520 2023-11-26T07:24:43.305 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:43.306+0000 7f7d50e7d640 1 -- 172.21.15.143:0/1535681957 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7d440a5b40 con 0x7f7d4c163520 2023-11-26T07:24:43.305 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:43.306+0000 7f7d50e7d640 1 -- 172.21.15.143:0/1535681957 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7d440ad440 con 0x7f7d4c163520 2023-11-26T07:24:43.306 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:43.306+0000 7f7d540e3640 1 -- 172.21.15.143:0/1535681957 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f7d4c163520 msgr2=0x7f7d4c16c3a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:24:43.306 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:43.306+0000 7f7d540e3640 1 --2- 172.21.15.143:0/1535681957 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f7d4c163520 0x7f7d4c16c3a0 secure :-1 s=READY pgs=1378 cs=0 l=1 rev1=1 crypto rx=0x7f7d44009d90 tx=0x7f7d4401c920 comp rx=0 tx=0).stop 2023-11-26T07:24:43.306 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:43.306+0000 7f7d540e3640 1 -- 172.21.15.143:0/1535681957 shutdown_connections 2023-11-26T07:24:43.306 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:43.306+0000 7f7d540e3640 1 --2- 172.21.15.143:0/1535681957 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f7d4c163520 0x7f7d4c16c3a0 unknown :-1 s=CLOSED pgs=1378 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:24:43.306 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:43.306+0000 7f7d540e3640 1 -- 172.21.15.143:0/1535681957 >> 172.21.15.143:0/1535681957 conn(0x7f7d4c06a890 msgr2=0x7f7d4c06aca0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:24:43.307 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:43.306+0000 7f7d540e3640 1 -- 172.21.15.143:0/1535681957 shutdown_connections 2023-11-26T07:24:43.307 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:43.306+0000 7f7d540e3640 1 -- 172.21.15.143:0/1535681957 wait complete. 2023-11-26T07:24:43.307 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:43.306+0000 7f7d540e3640 1 Processor -- start 2023-11-26T07:24:43.307 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:43.306+0000 7f7d540e3640 1 -- start start 2023-11-26T07:24:43.309 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:43.306+0000 7f7d540e3640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f7d4c0cf990 0x7f7d4c1602f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:24:43.309 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:43.306+0000 7f7d540e3640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f7d4c14b1e0 0x7f7d4c0a84d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:24:43.309 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:43.306+0000 7f7d540e3640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f7d4c163520 0x7f7d4c145a50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:24:43.309 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:43.306+0000 7f7d540e3640 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7d4c161310 con 0x7f7d4c14b1e0 2023-11-26T07:24:43.310 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:43.306+0000 7f7d540e3640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7d4c160830 con 0x7f7d4c163520 2023-11-26T07:24:43.310 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:43.306+0000 7f7d540e3640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7d4c1faac0 con 0x7f7d4c0cf990 2023-11-26T07:24:43.310 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:43.306+0000 7f7d5167e640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f7d4c14b1e0 0x7f7d4c0a84d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:24:43.310 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:43.306+0000 7f7d5167e640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f7d4c14b1e0 0x7f7d4c0a84d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:33280/0 (socket says 172.21.15.143:33280) 2023-11-26T07:24:43.310 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:43.306+0000 7f7d5167e640 1 -- 172.21.15.143:0/2999763050 learned_addr learned my addr 172.21.15.143:0/2999763050 (peer_addr_for_me v2:172.21.15.143:0/0) 2023-11-26T07:24:43.310 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:43.306+0000 7f7d51e7f640 1 --2- 172.21.15.143:0/2999763050 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f7d4c0cf990 0x7f7d4c1602f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:24:43.310 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:43.306+0000 7f7d52680640 1 --2- 172.21.15.143:0/2999763050 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f7d4c163520 0x7f7d4c145a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:24:43.310 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:43.310+0000 7f7d5167e640 1 -- 172.21.15.143:0/2999763050 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f7d4c0cf990 msgr2=0x7f7d4c1602f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:24:43.310 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:43.310+0000 7f7d5167e640 1 --2- 172.21.15.143:0/2999763050 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f7d4c0cf990 0x7f7d4c1602f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:24:43.310 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:43.310+0000 7f7d5167e640 1 -- 172.21.15.143:0/2999763050 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f7d4c163520 msgr2=0x7f7d4c145a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:24:43.310 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:43.310+0000 7f7d5167e640 1 --2- 172.21.15.143:0/2999763050 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f7d4c163520 0x7f7d4c145a50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:24:43.310 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:43.310+0000 7f7d5167e640 1 -- 172.21.15.143:0/2999763050 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7d44009a70 con 0x7f7d4c14b1e0 2023-11-26T07:24:43.311 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:43.310+0000 7f7d5167e640 1 --2- 172.21.15.143:0/2999763050 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f7d4c14b1e0 0x7f7d4c0a84d0 secure :-1 s=READY pgs=1379 cs=0 l=1 rev1=1 crypto rx=0x7f7d3c00c500 tx=0x7f7d3c00c9c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:24:43.312 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:43.310+0000 7f7d427fc640 1 -- 172.21.15.143:0/2999763050 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7d3c005cf0 con 0x7f7d4c14b1e0 2023-11-26T07:24:43.312 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:43.310+0000 7f7d427fc640 1 -- 172.21.15.143:0/2999763050 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7d3c005e50 con 0x7f7d4c14b1e0 2023-11-26T07:24:43.312 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:43.310+0000 7f7d540e3640 1 -- 172.21.15.143:0/2999763050 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7d4c160a70 con 0x7f7d4c14b1e0 2023-11-26T07:24:43.312 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:43.310+0000 7f7d427fc640 1 -- 172.21.15.143:0/2999763050 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7d3c0072f0 con 0x7f7d4c14b1e0 2023-11-26T07:24:43.312 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:43.310+0000 7f7d540e3640 1 -- 172.21.15.143:0/2999763050 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7d4c161030 con 0x7f7d4c14b1e0 2023-11-26T07:24:43.312 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:43.310+0000 7f7d540e3640 1 -- 172.21.15.143:0/2999763050 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7d4c0cebc0 con 0x7f7d4c14b1e0 2023-11-26T07:24:43.312 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:43.310+0000 7f7d427fc640 1 -- 172.21.15.143:0/2999763050 <== mon.0 v2:172.21.15.143:3300/0 4 ==== mgrmap(e 18) v1 ==== 86107+0+0 (secure 0 0 0) 0x7f7d3c007450 con 0x7f7d4c14b1e0 2023-11-26T07:24:43.313 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:43.314+0000 7f7d427fc640 1 --2- 172.21.15.143:0/2999763050 >> [v2:172.21.15.188:6836/3401128553,v1:172.21.15.188:6837/3401128553] conn(0x7f7d30066740 0x7f7d30068c00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:24:43.313 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:43.314+0000 7f7d427fc640 1 -- 172.21.15.143:0/2999763050 <== mon.0 v2:172.21.15.143:3300/0 5 ==== osd_map(103..103 src has 1..103) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f7d3c083720 con 0x7f7d4c14b1e0 2023-11-26T07:24:43.313 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:43.314+0000 7f7d51e7f640 1 --2- 172.21.15.143:0/2999763050 >> [v2:172.21.15.188:6836/3401128553,v1:172.21.15.188:6837/3401128553] conn(0x7f7d30066740 0x7f7d30068c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:24:43.314 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:43.314+0000 7f7d51e7f640 1 --2- 172.21.15.143:0/2999763050 >> [v2:172.21.15.188:6836/3401128553,v1:172.21.15.188:6837/3401128553] conn(0x7f7d30066740 0x7f7d30068c00 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7f7d48007fd0 tx=0x7f7d48009040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:24:43.317 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:43.318+0000 7f7d427fc640 1 -- 172.21.15.143:0/2999763050 <== mon.0 v2:172.21.15.143:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172649 (secure 0 0 0) 0x7f7d3c050950 con 0x7f7d4c14b1e0 2023-11-26T07:24:43.491 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:43.490+0000 7f7d540e3640 1 -- 172.21.15.143:0/2999763050 --> [v2:172.21.15.188:6836/3401128553,v1:172.21.15.188:6837/3401128553] -- mgr_command(tid 0: {"prefix": "fs subvolume snapshot info", "vol_name": "cephfs", "sub_name": "subvol24520", "snap_name": "subvol_snap55360", "target": ["mon-mgr", ""]}) v1 -- 0x7f7d4c067680 con 0x7f7d30066740 2023-11-26T07:24:43.510 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:43.510+0000 7f7d427fc640 1 -- 172.21.15.143:0/2999763050 <== mgr.4108 v2:172.21.15.188:6836/3401128553 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+207 (secure 0 0 0) 0x7f7d4c067680 con 0x7f7d30066740 2023-11-26T07:24:43.510 INFO:teuthology.orchestra.run.smithi143.stdout:{ 2023-11-26T07:24:43.511 INFO:teuthology.orchestra.run.smithi143.stdout: "created_at": "2023-11-26 07:24:23.899685", 2023-11-26T07:24:43.511 INFO:teuthology.orchestra.run.smithi143.stdout: "data_pool": "cephfs_data", 2023-11-26T07:24:43.511 INFO:teuthology.orchestra.run.smithi143.stdout: "has_pending_clones": "yes", 2023-11-26T07:24:43.511 INFO:teuthology.orchestra.run.smithi143.stdout: "pending_clones": [ 2023-11-26T07:24:43.511 INFO:teuthology.orchestra.run.smithi143.stdout: { 2023-11-26T07:24:43.511 INFO:teuthology.orchestra.run.smithi143.stdout: "name": "subvol_clone30650" 2023-11-26T07:24:43.511 INFO:teuthology.orchestra.run.smithi143.stdout: } 2023-11-26T07:24:43.511 INFO:teuthology.orchestra.run.smithi143.stdout: ] 2023-11-26T07:24:43.511 INFO:teuthology.orchestra.run.smithi143.stdout:} 2023-11-26T07:24:43.512 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:43.510+0000 7f7d540e3640 1 -- 172.21.15.143:0/2999763050 >> [v2:172.21.15.188:6836/3401128553,v1:172.21.15.188:6837/3401128553] conn(0x7f7d30066740 msgr2=0x7f7d30068c00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:24:43.512 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:43.510+0000 7f7d540e3640 1 --2- 172.21.15.143:0/2999763050 >> [v2:172.21.15.188:6836/3401128553,v1:172.21.15.188:6837/3401128553] conn(0x7f7d30066740 0x7f7d30068c00 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7f7d48007fd0 tx=0x7f7d48009040 comp rx=0 tx=0).stop 2023-11-26T07:24:43.512 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:43.510+0000 7f7d540e3640 1 -- 172.21.15.143:0/2999763050 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f7d4c14b1e0 msgr2=0x7f7d4c0a84d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:24:43.512 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:43.510+0000 7f7d540e3640 1 --2- 172.21.15.143:0/2999763050 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f7d4c14b1e0 0x7f7d4c0a84d0 secure :-1 s=READY pgs=1379 cs=0 l=1 rev1=1 crypto rx=0x7f7d3c00c500 tx=0x7f7d3c00c9c0 comp rx=0 tx=0).stop 2023-11-26T07:24:43.512 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:43.510+0000 7f7d540e3640 1 -- 172.21.15.143:0/2999763050 shutdown_connections 2023-11-26T07:24:43.512 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:43.510+0000 7f7d540e3640 1 --2- 172.21.15.143:0/2999763050 >> [v2:172.21.15.188:6836/3401128553,v1:172.21.15.188:6837/3401128553] conn(0x7f7d30066740 0x7f7d30068c00 unknown :-1 s=CLOSED pgs=155 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:24:43.513 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:43.510+0000 7f7d540e3640 1 --2- 172.21.15.143:0/2999763050 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f7d4c163520 0x7f7d4c145a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:24:43.513 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:43.510+0000 7f7d540e3640 1 --2- 172.21.15.143:0/2999763050 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f7d4c14b1e0 0x7f7d4c0a84d0 unknown :-1 s=CLOSED pgs=1379 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:24:43.513 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:43.510+0000 7f7d540e3640 1 --2- 172.21.15.143:0/2999763050 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f7d4c0cf990 0x7f7d4c1602f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:24:43.513 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:43.510+0000 7f7d540e3640 1 -- 172.21.15.143:0/2999763050 >> 172.21.15.143:0/2999763050 conn(0x7f7d4c06a890 msgr2=0x7f7d4c143370 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:24:43.513 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:43.510+0000 7f7d540e3640 1 -- 172.21.15.143:0/2999763050 shutdown_connections 2023-11-26T07:24:43.513 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:43.510+0000 7f7d540e3640 1 -- 172.21.15.143:0/2999763050 wait complete. 2023-11-26T07:24:43.533 INFO:teuthology.nuke.actions:Clearing teuthology firewall rules... 2023-11-26T07:24:43.533 DEBUG:teuthology.orchestra.run.smithi143:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2023-11-26T07:24:43.623 DEBUG:teuthology.orchestra.run.smithi188:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2023-11-26T07:24:43.710 INFO:teuthology.nuke.actions:Cleared teuthology firewall rules. 2023-11-26T07:24:43.710 DEBUG:teuthology.orchestra.run.smithi143:> set -ex 2023-11-26T07:24:43.710 DEBUG:teuthology.orchestra.run.smithi143:> dd if=/proc/self/mounts of=/dev/stdout 2023-11-26T07:24:43.720 DEBUG:teuthology.orchestra.run.smithi143:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-11-26T07:24:43.898 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:43.898+0000 7f2c9adf5640 1 Processor -- start 2023-11-26T07:24:43.898 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:43.898+0000 7f2c9adf5640 1 -- start start 2023-11-26T07:24:43.900 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:43.898+0000 7f2c9adf5640 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7f2c9413e290 0x7f2c940cf280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:24:43.900 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:43.898+0000 7f2c9adf5640 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7f2c9415f6b0 0x7f2c940cd7b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:24:43.900 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:43.898+0000 7f2c9adf5640 1 -- --> v1:172.21.15.188:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2c9415fb20 con 0x7f2c940cf7c0 2023-11-26T07:24:43.900 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:43.898+0000 7f2c9adf5640 1 -- --> v2:172.21.15.143:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2c940cfc40 con 0x7f2c9413e290 2023-11-26T07:24:43.900 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:43.898+0000 7f2c9adf5640 1 -- --> v2:172.21.15.188:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2c9406c520 con 0x7f2c9415f6b0 2023-11-26T07:24:43.900 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:43.898+0000 7f2c98b91640 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7f2c9413e290 0x7f2c940cf280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:24:43.900 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:43.898+0000 7f2c98b91640 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7f2c9413e290 0x7f2c940cf280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:33282/0 (socket says 172.21.15.143:33282) 2023-11-26T07:24:43.900 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:43.898+0000 7f2c98b91640 1 -- 172.21.15.143:0/2543680865 learned_addr learned my addr 172.21.15.143:0/2543680865 (peer_addr_for_me v2:172.21.15.143:0/0) 2023-11-26T07:24:43.900 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:43.898+0000 7f2c99392640 1 --2- 172.21.15.143:0/2543680865 >> v2:172.21.15.188:3300/0 conn(0x7f2c9415f6b0 0x7f2c940cd7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:24:43.900 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:43.898+0000 7f2c98b91640 1 -- 172.21.15.143:0/2543680865 >> v2:172.21.15.188:3300/0 conn(0x7f2c9415f6b0 msgr2=0x7f2c940cd7b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-26T07:24:43.900 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:43.898+0000 7f2c98b91640 1 --2- 172.21.15.143:0/2543680865 >> v2:172.21.15.188:3300/0 conn(0x7f2c9415f6b0 0x7f2c940cd7b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:24:43.901 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:43.898+0000 7f2c98b91640 1 -- 172.21.15.143:0/2543680865 >> v1:172.21.15.188:6789/0 conn(0x7f2c940cf7c0 legacy=0x7f2c9415efa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:24:43.901 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:43.898+0000 7f2c98b91640 1 -- 172.21.15.143:0/2543680865 --> v2:172.21.15.143:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2c9406c800 con 0x7f2c9413e290 2023-11-26T07:24:43.901 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:43.898+0000 7f2c99392640 1 --2- 172.21.15.143:0/2543680865 >> v2:172.21.15.188:3300/0 conn(0x7f2c9415f6b0 0x7f2c940cd7b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-26T07:24:43.902 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:43.902+0000 7f2c98b91640 1 --2- 172.21.15.143:0/2543680865 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f2c9413e290 0x7f2c940cf280 secure :-1 s=READY pgs=1380 cs=0 l=1 rev1=1 crypto rx=0x7f2c9406e780 tx=0x7f2c8801c4d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=787247cd4bc8adb2 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:24:43.903 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:43.902+0000 7f2c937fe640 1 -- 172.21.15.143:0/2543680865 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2c880aa430 con 0x7f2c9413e290 2023-11-26T07:24:43.903 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:43.902+0000 7f2c937fe640 1 -- 172.21.15.143:0/2543680865 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2c880aa590 con 0x7f2c9413e290 2023-11-26T07:24:43.903 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:43.902+0000 7f2c937fe640 1 -- 172.21.15.143:0/2543680865 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2c880aa920 con 0x7f2c9413e290 2023-11-26T07:24:43.904 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:43.902+0000 7f2c9adf5640 1 -- 172.21.15.143:0/2543680865 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f2c9413e290 msgr2=0x7f2c940cf280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:24:43.904 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:43.902+0000 7f2c9adf5640 1 --2- 172.21.15.143:0/2543680865 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f2c9413e290 0x7f2c940cf280 secure :-1 s=READY pgs=1380 cs=0 l=1 rev1=1 crypto rx=0x7f2c9406e780 tx=0x7f2c8801c4d0 comp rx=0 tx=0).stop 2023-11-26T07:24:43.904 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:43.902+0000 7f2c9adf5640 1 -- 172.21.15.143:0/2543680865 shutdown_connections 2023-11-26T07:24:43.904 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:43.902+0000 7f2c9adf5640 1 --2- 172.21.15.143:0/2543680865 >> v2:172.21.15.188:3300/0 conn(0x7f2c9415f6b0 0x7f2c940cd7b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:24:43.904 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:43.902+0000 7f2c9adf5640 1 --2- 172.21.15.143:0/2543680865 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f2c9413e290 0x7f2c940cf280 unknown :-1 s=CLOSED pgs=1380 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:24:43.904 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:43.902+0000 7f2c9adf5640 1 -- 172.21.15.143:0/2543680865 >> 172.21.15.143:0/2543680865 conn(0x7f2c9406a890 msgr2=0x7f2c9406aca0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:24:43.904 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:43.902+0000 7f2c9adf5640 1 -- 172.21.15.143:0/2543680865 shutdown_connections 2023-11-26T07:24:43.904 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:43.902+0000 7f2c9adf5640 1 -- 172.21.15.143:0/2543680865 wait complete. 2023-11-26T07:24:43.905 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:43.906+0000 7f2c9adf5640 1 Processor -- start 2023-11-26T07:24:43.905 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:43.906+0000 7f2c9adf5640 1 -- start start 2023-11-26T07:24:43.907 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:43.906+0000 7f2c9adf5640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f2c940cf7c0 0x7f2c9413e930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:24:43.907 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:43.906+0000 7f2c9adf5640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f2c9413e290 0x7f2c941f7450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:24:43.908 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:43.906+0000 7f2c9adf5640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f2c9415f6b0 0x7f2c941f7990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:24:43.908 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:43.906+0000 7f2c9adf5640 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2c9415db70 con 0x7f2c940cf7c0 2023-11-26T07:24:43.908 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:43.906+0000 7f2c9adf5640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2c9415cc80 con 0x7f2c9415f6b0 2023-11-26T07:24:43.908 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:43.906+0000 7f2c9adf5640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2c9415cf60 con 0x7f2c9413e290 2023-11-26T07:24:43.908 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:43.906+0000 7f2c98b91640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f2c940cf7c0 0x7f2c9413e930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:24:43.908 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:43.906+0000 7f2c98b91640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f2c940cf7c0 0x7f2c9413e930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:33294/0 (socket says 172.21.15.143:33294) 2023-11-26T07:24:43.908 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:43.906+0000 7f2c98b91640 1 -- 172.21.15.143:0/2155241653 learned_addr learned my addr 172.21.15.143:0/2155241653 (peer_addr_for_me v2:172.21.15.143:0/0) 2023-11-26T07:24:43.908 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:43.906+0000 7f2c99392640 1 --2- 172.21.15.143:0/2155241653 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f2c9415f6b0 0x7f2c941f7990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:24:43.908 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:43.906+0000 7f2c93fff640 1 --2- 172.21.15.143:0/2155241653 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f2c9413e290 0x7f2c941f7450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:24:43.908 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:43.906+0000 7f2c98b91640 1 -- 172.21.15.143:0/2155241653 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f2c9413e290 msgr2=0x7f2c941f7450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:24:43.908 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:43.906+0000 7f2c98b91640 1 --2- 172.21.15.143:0/2155241653 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f2c9413e290 0x7f2c941f7450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:24:43.908 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:43.906+0000 7f2c98b91640 1 -- 172.21.15.143:0/2155241653 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f2c9415f6b0 msgr2=0x7f2c941f7990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:24:43.908 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:43.906+0000 7f2c98b91640 1 --2- 172.21.15.143:0/2155241653 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f2c9415f6b0 0x7f2c941f7990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:24:43.909 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:43.906+0000 7f2c98b91640 1 -- 172.21.15.143:0/2155241653 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2c880094d0 con 0x7f2c940cf7c0 2023-11-26T07:24:43.909 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:43.906+0000 7f2c98b91640 1 --2- 172.21.15.143:0/2155241653 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f2c940cf7c0 0x7f2c9413e930 secure :-1 s=READY pgs=1381 cs=0 l=1 rev1=1 crypto rx=0x7f2c880ac040 tx=0x7f2c8801ad10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:24:43.909 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:43.906+0000 7f2c917fa640 1 -- 172.21.15.143:0/2155241653 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2c880b3800 con 0x7f2c940cf7c0 2023-11-26T07:24:43.909 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:43.906+0000 7f2c917fa640 1 -- 172.21.15.143:0/2155241653 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2c880b3960 con 0x7f2c940cf7c0 2023-11-26T07:24:43.910 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:43.906+0000 7f2c917fa640 1 -- 172.21.15.143:0/2155241653 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2c880b3d10 con 0x7f2c940cf7c0 2023-11-26T07:24:43.910 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:43.906+0000 7f2c9adf5640 1 -- 172.21.15.143:0/2155241653 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2c9415d240 con 0x7f2c940cf7c0 2023-11-26T07:24:43.910 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:43.906+0000 7f2c9adf5640 1 -- 172.21.15.143:0/2155241653 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2c9415d800 con 0x7f2c940cf7c0 2023-11-26T07:24:43.910 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:43.906+0000 7f2c9adf5640 1 -- 172.21.15.143:0/2155241653 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2c9415d990 con 0x7f2c940cf7c0 2023-11-26T07:24:43.910 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:43.910+0000 7f2c917fa640 1 -- 172.21.15.143:0/2155241653 <== mon.0 v2:172.21.15.143:3300/0 4 ==== mgrmap(e 18) v1 ==== 86107+0+0 (secure 0 0 0) 0x7f2c880b2380 con 0x7f2c940cf7c0 2023-11-26T07:24:43.911 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:43.910+0000 7f2c917fa640 1 --2- 172.21.15.143:0/2155241653 >> [v2:172.21.15.188:6836/3401128553,v1:172.21.15.188:6837/3401128553] conn(0x7f2c78066740 0x7f2c78068c00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:24:43.911 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:43.910+0000 7f2c917fa640 1 -- 172.21.15.143:0/2155241653 <== mon.0 v2:172.21.15.143:3300/0 5 ==== osd_map(103..103 src has 1..103) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f2c8811e990 con 0x7f2c940cf7c0 2023-11-26T07:24:43.911 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:43.910+0000 7f2c93fff640 1 --2- 172.21.15.143:0/2155241653 >> [v2:172.21.15.188:6836/3401128553,v1:172.21.15.188:6837/3401128553] conn(0x7f2c78066740 0x7f2c78068c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:24:43.912 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:43.910+0000 7f2c93fff640 1 --2- 172.21.15.143:0/2155241653 >> [v2:172.21.15.188:6836/3401128553,v1:172.21.15.188:6837/3401128553] conn(0x7f2c78066740 0x7f2c78068c00 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7f2c941550f0 tx=0x7f2c80009040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:24:43.915 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:43.914+0000 7f2c917fa640 1 -- 172.21.15.143:0/2155241653 <== mon.0 v2:172.21.15.143:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172649 (secure 0 0 0) 0x7f2c880ebab0 con 0x7f2c940cf7c0 2023-11-26T07:24:44.066 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:44.066+0000 7f2c9adf5640 1 -- 172.21.15.143:0/2155241653 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f2c94067680 con 0x7f2c940cf7c0 2023-11-26T07:24:44.067 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:44.066+0000 7f2c917fa640 1 -- 172.21.15.143:0/2155241653 <== mon.0 v2:172.21.15.143:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v103) v1 ==== 74+0+16563 (secure 0 0 0) 0x7f2c880ef0d0 con 0x7f2c940cf7c0 2023-11-26T07:24:44.067 INFO:teuthology.orchestra.run.smithi143.stdout: 2023-11-26T07:24:44.067 INFO:teuthology.orchestra.run.smithi143.stdout:{"epoch":103,"fsid":"f5033060-2469-4a07-95fe-2b76f5404bc1","created":"2023-11-26T07:17:28.613663+0000","modified":"2023-11-26T07:24:09.510299+0000","last_up_change":"2023-11-26T07:17:41.157813+0000","last_in_change":"2023-11-26T07:17:31.181658+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":17,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-11-26T07:17:41.758597+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":16,"pool_name":"cephfs_metadata","create_time":"2023-11-26T07:24:06.278132+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"102","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":17,"pool_name":"cephfs_data","create_time":"2023-11-26T07:24:07.294850+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"103","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"ebbee2c6-023e-44b9-9f1d-6f7bfd309730","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":101,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6817","nonce":639807369},{"type":"v1","addr":"172.21.15.143:6819","nonce":639807369}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6821","nonce":639807369},{"type":"v1","addr":"172.21.15.143:6823","nonce":639807369}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6830","nonce":639807369},{"type":"v1","addr":"172.21.15.143:6831","nonce":639807369}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6825","nonce":639807369},{"type":"v1","addr":"172.21.15.143:6828","nonce":639807369}]},"public_addr":"172.21.15.143:6819/639807369","cluster_addr":"172.21.15.143:6823/639807369","heartbeat_back_addr":"172.21.15.143:6831/639807369","heartbeat_front_addr":"172.21.15.143:6828/639807369","state":["exists","up"]},{"osd":1,"uuid":"fec4b8be-7122-49e9-9a19-5850a0700001","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":101,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6816","nonce":1144942530},{"type":"v1","addr":"172.21.15.143:6818","nonce":1144942530}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6820","nonce":1144942530},{"type":"v1","addr":"172.21.15.143:6822","nonce":1144942530}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6827","nonce":1144942530},{"type":"v1","addr":"172.21.15.143:6829","nonce":1144942530}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6824","nonce":1144942530},{"type":"v1","addr":"172.21.15.143:6826","nonce":1144942530}]},"public_addr":"172.21.15.143:6818/1144942530","cluster_addr":"172.21.15.143:6822/1144942530","heartbeat_back_addr":"172.21.15.143:6829/1144942530","heartbeat_front_addr":"172.21.15.143:6826/1144942530","state":["exists","up"]},{"osd":2,"uuid":"2888af5b-f933-46ca-8f95-6dd494ff43f6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":101,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6808","nonce":3909095186},{"type":"v1","addr":"172.21.15.143:6809","nonce":3909095186}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6810","nonce":3909095186},{"type":"v1","addr":"172.21.15.143:6811","nonce":3909095186}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6814","nonce":3909095186},{"type":"v1","addr":"172.21.15.143:6815","nonce":3909095186}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6812","nonce":3909095186},{"type":"v1","addr":"172.21.15.143:6813","nonce":3909095186}]},"public_addr":"172.21.15.143:6809/3909095186","cluster_addr":"172.21.15.143:6811/3909095186","heartbeat_back_addr":"172.21.15.143:6815/3909095186","heartbeat_front_addr":"172.21.15.143:6813/3909095186","state":["exists","up"]},{"osd":3,"uuid":"d1f52435-7bc1-4081-9550-9cd1ba55f549","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":101,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6800","nonce":3854210991},{"type":"v1","addr":"172.21.15.143:6801","nonce":3854210991}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6802","nonce":3854210991},{"type":"v1","addr":"172.21.15.143:6803","nonce":3854210991}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6806","nonce":3854210991},{"type":"v1","addr":"172.21.15.143:6807","nonce":3854210991}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6804","nonce":3854210991},{"type":"v1","addr":"172.21.15.143:6805","nonce":3854210991}]},"public_addr":"172.21.15.143:6801/3854210991","cluster_addr":"172.21.15.143:6803/3854210991","heartbeat_back_addr":"172.21.15.143:6807/3854210991","heartbeat_front_addr":"172.21.15.143:6805/3854210991","state":["exists","up"]},{"osd":4,"uuid":"e2ae8b6e-6e25-49b2-bcbf-aacce68f06a3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":101,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6800","nonce":3924606902},{"type":"v1","addr":"172.21.15.188:6801","nonce":3924606902}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6802","nonce":3924606902},{"type":"v1","addr":"172.21.15.188:6803","nonce":3924606902}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6806","nonce":3924606902},{"type":"v1","addr":"172.21.15.188:6807","nonce":3924606902}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6804","nonce":3924606902},{"type":"v1","addr":"172.21.15.188:6805","nonce":3924606902}]},"public_addr":"172.21.15.188:6801/3924606902","cluster_addr":"172.21.15.188:6803/3924606902","heartbeat_back_addr":"172.21.15.188:6807/3924606902","heartbeat_front_addr":"172.21.15.188:6805/3924606902","state":["exists","up"]},{"osd":5,"uuid":"3cec8a32-7698-4c36-8e89-e4ba11934f33","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":101,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6813","nonce":186841544},{"type":"v1","addr":"172.21.15.188:6815","nonce":186841544}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6817","nonce":186841544},{"type":"v1","addr":"172.21.15.188:6818","nonce":186841544}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6822","nonce":186841544},{"type":"v1","addr":"172.21.15.188:6823","nonce":186841544}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6820","nonce":186841544},{"type":"v1","addr":"172.21.15.188:6821","nonce":186841544}]},"public_addr":"172.21.15.188:6815/186841544","cluster_addr":"172.21.15.188:6818/186841544","heartbeat_back_addr":"172.21.15.188:6823/186841544","heartbeat_front_addr":"172.21.15.188:6821/186841544","state":["exists","up"]},{"osd":6,"uuid":"58a65d5d-e090-473a-81b4-ad38e85029f9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":101,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6808","nonce":2153499405},{"type":"v1","addr":"172.21.15.188:6809","nonce":2153499405}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6810","nonce":2153499405},{"type":"v1","addr":"172.21.15.188:6811","nonce":2153499405}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6816","nonce":2153499405},{"type":"v1","addr":"172.21.15.188:6819","nonce":2153499405}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6812","nonce":2153499405},{"type":"v1","addr":"172.21.15.188:6814","nonce":2153499405}]},"public_addr":"172.21.15.188:6809/2153499405","cluster_addr":"172.21.15.188:6811/2153499405","heartbeat_back_addr":"172.21.15.188:6819/2153499405","heartbeat_front_addr":"172.21.15.188:6814/2153499405","state":["exists","up"]},{"osd":7,"uuid":"ea5b7f70-3554-4cf8-ac30-f7c59b80678d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":101,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6824","nonce":2722096729},{"type":"v1","addr":"172.21.15.188:6825","nonce":2722096729}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6826","nonce":2722096729},{"type":"v1","addr":"172.21.15.188:6827","nonce":2722096729}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6830","nonce":2722096729},{"type":"v1","addr":"172.21.15.188:6831","nonce":2722096729}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6828","nonce":2722096729},{"type":"v1","addr":"172.21.15.188:6829","nonce":2722096729}]},"public_addr":"172.21.15.188:6825/2722096729","cluster_addr":"172.21.15.188:6827/2722096729","heartbeat_back_addr":"172.21.15.188:6831/2722096729","heartbeat_front_addr":"172.21.15.188:6829/2722096729","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-26T07:17:36.995032+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-26T07:17:36.932266+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-26T07:17:36.829304+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-26T07:17:36.804297+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-26T07:17:36.667892+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-26T07:17:36.964868+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-26T07:17:36.827332+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-26T07:17:36.893179+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-11-26T07:24:44.069 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:44.066+0000 7f2c9adf5640 1 -- 172.21.15.143:0/2155241653 >> [v2:172.21.15.188:6836/3401128553,v1:172.21.15.188:6837/3401128553] conn(0x7f2c78066740 msgr2=0x7f2c78068c00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:24:44.069 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:44.066+0000 7f2c9adf5640 1 --2- 172.21.15.143:0/2155241653 >> [v2:172.21.15.188:6836/3401128553,v1:172.21.15.188:6837/3401128553] conn(0x7f2c78066740 0x7f2c78068c00 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7f2c941550f0 tx=0x7f2c80009040 comp rx=0 tx=0).stop 2023-11-26T07:24:44.069 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:44.066+0000 7f2c9adf5640 1 -- 172.21.15.143:0/2155241653 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f2c940cf7c0 msgr2=0x7f2c9413e930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:24:44.069 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:44.066+0000 7f2c9adf5640 1 --2- 172.21.15.143:0/2155241653 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f2c940cf7c0 0x7f2c9413e930 secure :-1 s=READY pgs=1381 cs=0 l=1 rev1=1 crypto rx=0x7f2c880ac040 tx=0x7f2c8801ad10 comp rx=0 tx=0).stop 2023-11-26T07:24:44.069 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:44.066+0000 7f2c9adf5640 1 -- 172.21.15.143:0/2155241653 shutdown_connections 2023-11-26T07:24:44.069 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:44.066+0000 7f2c9adf5640 1 --2- 172.21.15.143:0/2155241653 >> [v2:172.21.15.188:6836/3401128553,v1:172.21.15.188:6837/3401128553] conn(0x7f2c78066740 0x7f2c78068c00 unknown :-1 s=CLOSED pgs=156 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:24:44.070 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:44.066+0000 7f2c9adf5640 1 --2- 172.21.15.143:0/2155241653 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f2c9415f6b0 0x7f2c941f7990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:24:44.070 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:44.066+0000 7f2c9adf5640 1 --2- 172.21.15.143:0/2155241653 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f2c9413e290 0x7f2c941f7450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:24:44.070 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:44.066+0000 7f2c9adf5640 1 --2- 172.21.15.143:0/2155241653 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f2c940cf7c0 0x7f2c9413e930 unknown :-1 s=CLOSED pgs=1381 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:24:44.070 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:44.066+0000 7f2c9adf5640 1 -- 172.21.15.143:0/2155241653 >> 172.21.15.143:0/2155241653 conn(0x7f2c9406a890 msgr2=0x7f2c940b0fc0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:24:44.070 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:44.070+0000 7f2c9adf5640 1 -- 172.21.15.143:0/2155241653 shutdown_connections 2023-11-26T07:24:44.070 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:44.070+0000 7f2c9adf5640 1 -- 172.21.15.143:0/2155241653 wait complete. 2023-11-26T07:24:44.088 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-11-26T07:17:41.758597+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 16, 'pool_name': 'cephfs_metadata', 'create_time': '2023-11-26T07:24:06.278132+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '102', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 1.75, 'score_stable': 1.75, 'optimal_score': 1, 'raw_score_acting': 1.75, 'raw_score_stable': 1.75, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 17, 'pool_name': 'cephfs_data', 'create_time': '2023-11-26T07:24:07.294850+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '103', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.75, 'score_stable': 1.75, 'optimal_score': 1, 'raw_score_acting': 1.75, 'raw_score_stable': 1.75, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2023-11-26T07:24:44.089 DEBUG:teuthology.orchestra.run.smithi143:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-11-26T07:24:44.232 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:44.230+0000 7ff384a6b640 1 Processor -- start 2023-11-26T07:24:44.233 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:44.230+0000 7ff384a6b640 1 -- start start 2023-11-26T07:24:44.234 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:44.234+0000 7ff384a6b640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7ff38013e290 0x7ff3800cf280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:24:44.235 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:44.234+0000 7ff384a6b640 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7ff38015f6b0 0x7ff3800cd7b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:24:44.235 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:44.234+0000 7ff384a6b640 1 -- --> v1:172.21.15.188:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff38006c520 con 0x7ff3800cf7c0 2023-11-26T07:24:44.235 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:44.234+0000 7ff384a6b640 1 -- --> v2:172.21.15.143:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff3800cfc40 con 0x7ff38015f6b0 2023-11-26T07:24:44.235 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:44.234+0000 7ff384a6b640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff38015fb40 con 0x7ff38013e290 2023-11-26T07:24:44.235 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:44.234+0000 7ff37ed9d640 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7ff38015f6b0 0x7ff3800cd7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:24:44.235 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:44.234+0000 7ff37ed9d640 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7ff38015f6b0 0x7ff3800cd7b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:33302/0 (socket says 172.21.15.143:33302) 2023-11-26T07:24:44.235 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:44.234+0000 7ff37ed9d640 1 -- 172.21.15.143:0/1625423417 learned_addr learned my addr 172.21.15.143:0/1625423417 (peer_addr_for_me v2:172.21.15.143:0/0) 2023-11-26T07:24:44.235 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:44.234+0000 7ff37e59c640 1 --2- 172.21.15.143:0/1625423417 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7ff38013e290 0x7ff3800cf280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:24:44.235 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:44.234+0000 7ff37ed9d640 1 -- 172.21.15.143:0/1625423417 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7ff38013e290 msgr2=0x7ff3800cf280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-26T07:24:44.235 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:44.234+0000 7ff37ed9d640 1 --2- 172.21.15.143:0/1625423417 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7ff38013e290 0x7ff3800cf280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:24:44.235 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:44.234+0000 7ff37ed9d640 1 -- 172.21.15.143:0/1625423417 >> v1:172.21.15.188:6789/0 conn(0x7ff3800cf7c0 legacy=0x7ff38015efa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-26T07:24:44.236 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:44.234+0000 7ff37ed9d640 1 -- 172.21.15.143:0/1625423417 --> v2:172.21.15.143:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff38006c800 con 0x7ff38015f6b0 2023-11-26T07:24:44.237 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:44.238+0000 7ff37ed9d640 1 --2- 172.21.15.143:0/1625423417 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7ff38015f6b0 0x7ff3800cd7b0 secure :-1 s=READY pgs=1382 cs=0 l=1 rev1=1 crypto rx=0x7ff378009e50 tx=0x7ff37801c9a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=768622146638193f server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:24:44.238 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:44.238+0000 7ff37d59a640 1 -- 172.21.15.143:0/1625423417 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff3780a5820 con 0x7ff38015f6b0 2023-11-26T07:24:44.238 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:44.238+0000 7ff37d59a640 1 -- 172.21.15.143:0/1625423417 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff3780a5980 con 0x7ff38015f6b0 2023-11-26T07:24:44.238 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:44.238+0000 7ff37d59a640 1 -- 172.21.15.143:0/1625423417 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff3780a5d10 con 0x7ff38015f6b0 2023-11-26T07:24:44.239 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:44.238+0000 7ff384a6b640 1 -- 172.21.15.143:0/1625423417 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7ff38015f6b0 msgr2=0x7ff3800cd7b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:24:44.239 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:44.238+0000 7ff384a6b640 1 --2- 172.21.15.143:0/1625423417 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7ff38015f6b0 0x7ff3800cd7b0 secure :-1 s=READY pgs=1382 cs=0 l=1 rev1=1 crypto rx=0x7ff378009e50 tx=0x7ff37801c9a0 comp rx=0 tx=0).stop 2023-11-26T07:24:44.239 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:44.238+0000 7ff384a6b640 1 -- 172.21.15.143:0/1625423417 shutdown_connections 2023-11-26T07:24:44.239 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:44.238+0000 7ff384a6b640 1 --2- 172.21.15.143:0/1625423417 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7ff38015f6b0 0x7ff3800cd7b0 unknown :-1 s=CLOSED pgs=1382 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:24:44.239 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:44.238+0000 7ff384a6b640 1 --2- 172.21.15.143:0/1625423417 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7ff38013e290 0x7ff3800cf280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:24:44.239 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:44.238+0000 7ff384a6b640 1 -- 172.21.15.143:0/1625423417 >> 172.21.15.143:0/1625423417 conn(0x7ff38006a890 msgr2=0x7ff38006aca0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:24:44.239 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:44.238+0000 7ff384a6b640 1 -- 172.21.15.143:0/1625423417 shutdown_connections 2023-11-26T07:24:44.239 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:44.238+0000 7ff384a6b640 1 -- 172.21.15.143:0/1625423417 wait complete. 2023-11-26T07:24:44.240 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:44.238+0000 7ff384a6b640 1 Processor -- start 2023-11-26T07:24:44.240 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:44.238+0000 7ff384a6b640 1 -- start start 2023-11-26T07:24:44.242 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:44.242+0000 7ff384a6b640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7ff3800cf7c0 0x7ff380135a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:24:44.242 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:44.242+0000 7ff384a6b640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7ff38013e290 0x7ff380137160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:24:44.242 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:44.242+0000 7ff384a6b640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7ff38015f6b0 0x7ff3801376a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:24:44.242 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:44.242+0000 7ff384a6b640 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff3801361c0 con 0x7ff38015f6b0 2023-11-26T07:24:44.242 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:44.242+0000 7ff384a6b640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff380136d40 con 0x7ff38013e290 2023-11-26T07:24:44.242 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:44.242+0000 7ff384a6b640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff3801363a0 con 0x7ff3800cf7c0 2023-11-26T07:24:44.242 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:44.242+0000 7ff37ed9d640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7ff38015f6b0 0x7ff3801376a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:24:44.242 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:44.242+0000 7ff37ed9d640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7ff38015f6b0 0x7ff3801376a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:33318/0 (socket says 172.21.15.143:33318) 2023-11-26T07:24:44.242 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:44.242+0000 7ff37ed9d640 1 -- 172.21.15.143:0/3041694966 learned_addr learned my addr 172.21.15.143:0/3041694966 (peer_addr_for_me v2:172.21.15.143:0/0) 2023-11-26T07:24:44.243 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:44.242+0000 7ff37e59c640 1 --2- 172.21.15.143:0/3041694966 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7ff3800cf7c0 0x7ff380135a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:24:44.243 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:44.242+0000 7ff37dd9b640 1 --2- 172.21.15.143:0/3041694966 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7ff38013e290 0x7ff380137160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:24:44.244 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:44.242+0000 7ff37ed9d640 1 -- 172.21.15.143:0/3041694966 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7ff3800cf7c0 msgr2=0x7ff380135a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:24:44.244 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:44.242+0000 7ff37ed9d640 1 --2- 172.21.15.143:0/3041694966 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7ff3800cf7c0 0x7ff380135a00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:24:44.244 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:44.242+0000 7ff37ed9d640 1 -- 172.21.15.143:0/3041694966 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7ff38013e290 msgr2=0x7ff380137160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:24:44.244 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:44.242+0000 7ff37ed9d640 1 --2- 172.21.15.143:0/3041694966 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7ff38013e290 0x7ff380137160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:24:44.244 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:44.242+0000 7ff37ed9d640 1 -- 172.21.15.143:0/3041694966 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff378009b30 con 0x7ff38015f6b0 2023-11-26T07:24:44.244 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:44.242+0000 7ff37e59c640 1 --2- 172.21.15.143:0/3041694966 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7ff3800cf7c0 0x7ff380135a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-26T07:24:44.244 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:44.242+0000 7ff37ed9d640 1 --2- 172.21.15.143:0/3041694966 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7ff38015f6b0 0x7ff3801376a0 secure :-1 s=READY pgs=1383 cs=0 l=1 rev1=1 crypto rx=0x7ff37801b050 tx=0x7ff378019740 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:24:44.244 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:44.242+0000 7ff36effd640 1 -- 172.21.15.143:0/3041694966 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff3780b8670 con 0x7ff38015f6b0 2023-11-26T07:24:44.244 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:44.242+0000 7ff36effd640 1 -- 172.21.15.143:0/3041694966 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff3780b87f0 con 0x7ff38015f6b0 2023-11-26T07:24:44.244 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:44.242+0000 7ff384a6b640 1 -- 172.21.15.143:0/3041694966 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff380136680 con 0x7ff38015f6b0 2023-11-26T07:24:44.244 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:44.242+0000 7ff36effd640 1 -- 172.21.15.143:0/3041694966 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff3780b8ba0 con 0x7ff38015f6b0 2023-11-26T07:24:44.245 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:44.242+0000 7ff384a6b640 1 -- 172.21.15.143:0/3041694966 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff380135f40 con 0x7ff38015f6b0 2023-11-26T07:24:44.245 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:44.242+0000 7ff384a6b640 1 -- 172.21.15.143:0/3041694966 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff380136e80 con 0x7ff38015f6b0 2023-11-26T07:24:44.245 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:44.242+0000 7ff36effd640 1 -- 172.21.15.143:0/3041694966 <== mon.0 v2:172.21.15.143:3300/0 4 ==== mgrmap(e 18) v1 ==== 86107+0+0 (secure 0 0 0) 0x7ff3780b8d00 con 0x7ff38015f6b0 2023-11-26T07:24:44.245 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:44.246+0000 7ff36effd640 1 --2- 172.21.15.143:0/3041694966 >> [v2:172.21.15.188:6836/3401128553,v1:172.21.15.188:6837/3401128553] conn(0x7ff360066740 0x7ff360068c00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:24:44.246 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:44.246+0000 7ff36effd640 1 -- 172.21.15.143:0/3041694966 <== mon.0 v2:172.21.15.143:3300/0 5 ==== osd_map(103..103 src has 1..103) v4 ==== 5709+0+0 (secure 0 0 0) 0x7ff37811f850 con 0x7ff38015f6b0 2023-11-26T07:24:44.246 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:44.246+0000 7ff37e59c640 1 --2- 172.21.15.143:0/3041694966 >> [v2:172.21.15.188:6836/3401128553,v1:172.21.15.188:6837/3401128553] conn(0x7ff360066740 0x7ff360068c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:24:44.246 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:44.246+0000 7ff37e59c640 1 --2- 172.21.15.143:0/3041694966 >> [v2:172.21.15.188:6836/3401128553,v1:172.21.15.188:6837/3401128553] conn(0x7ff360066740 0x7ff360068c00 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7ff370007a70 tx=0x7ff370008040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:24:44.249 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:44.250+0000 7ff36effd640 1 -- 172.21.15.143:0/3041694966 <== mon.0 v2:172.21.15.143:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172649 (secure 0 0 0) 0x7ff380136e80 con 0x7ff38015f6b0 2023-11-26T07:24:44.400 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:44.398+0000 7ff384a6b640 1 -- 172.21.15.143:0/3041694966 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7ff380136e80 con 0x7ff38015f6b0 2023-11-26T07:24:44.400 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:44.398+0000 7ff36effd640 1 -- 172.21.15.143:0/3041694966 <== mon.0 v2:172.21.15.143:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v103) v1 ==== 93+0+10 (secure 0 0 0) 0x7ff380136e80 con 0x7ff38015f6b0 2023-11-26T07:24:44.400 INFO:teuthology.orchestra.run.smithi143.stdout:pg_num: 1 2023-11-26T07:24:44.402 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:44.402+0000 7ff384a6b640 1 -- 172.21.15.143:0/3041694966 >> [v2:172.21.15.188:6836/3401128553,v1:172.21.15.188:6837/3401128553] conn(0x7ff360066740 msgr2=0x7ff360068c00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:24:44.402 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:44.402+0000 7ff384a6b640 1 --2- 172.21.15.143:0/3041694966 >> [v2:172.21.15.188:6836/3401128553,v1:172.21.15.188:6837/3401128553] conn(0x7ff360066740 0x7ff360068c00 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7ff370007a70 tx=0x7ff370008040 comp rx=0 tx=0).stop 2023-11-26T07:24:44.402 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:44.402+0000 7ff384a6b640 1 -- 172.21.15.143:0/3041694966 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7ff38015f6b0 msgr2=0x7ff3801376a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:24:44.403 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:44.402+0000 7ff384a6b640 1 --2- 172.21.15.143:0/3041694966 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7ff38015f6b0 0x7ff3801376a0 secure :-1 s=READY pgs=1383 cs=0 l=1 rev1=1 crypto rx=0x7ff37801b050 tx=0x7ff378019740 comp rx=0 tx=0).stop 2023-11-26T07:24:44.403 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:44.402+0000 7ff384a6b640 1 -- 172.21.15.143:0/3041694966 shutdown_connections 2023-11-26T07:24:44.403 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:44.402+0000 7ff384a6b640 1 --2- 172.21.15.143:0/3041694966 >> [v2:172.21.15.188:6836/3401128553,v1:172.21.15.188:6837/3401128553] conn(0x7ff360066740 0x7ff360068c00 unknown :-1 s=CLOSED pgs=157 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:24:44.403 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:44.402+0000 7ff384a6b640 1 --2- 172.21.15.143:0/3041694966 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7ff38015f6b0 0x7ff3801376a0 unknown :-1 s=CLOSED pgs=1383 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:24:44.403 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:44.402+0000 7ff384a6b640 1 --2- 172.21.15.143:0/3041694966 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7ff38013e290 0x7ff380137160 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:24:44.403 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:44.402+0000 7ff384a6b640 1 --2- 172.21.15.143:0/3041694966 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7ff3800cf7c0 0x7ff380135a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:24:44.403 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:44.402+0000 7ff384a6b640 1 -- 172.21.15.143:0/3041694966 >> 172.21.15.143:0/3041694966 conn(0x7ff38006a890 msgr2=0x7ff38015eea0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:24:44.404 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:44.402+0000 7ff384a6b640 1 -- 172.21.15.143:0/3041694966 shutdown_connections 2023-11-26T07:24:44.404 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:44.402+0000 7ff384a6b640 1 -- 172.21.15.143:0/3041694966 wait complete. 2023-11-26T07:24:44.421 DEBUG:teuthology.orchestra.run.smithi143:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-11-26T07:24:44.560 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:44.558+0000 7f1d358ad640 1 Processor -- start 2023-11-26T07:24:44.561 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:44.558+0000 7f1d358ad640 1 -- start start 2023-11-26T07:24:44.562 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:44.562+0000 7f1d358ad640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f1d30160850 0x7f1d300d3d30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:24:44.562 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:44.562+0000 7f1d358ad640 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7f1d30160f60 0x7f1d301613f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:24:44.562 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:44.562+0000 7f1d358ad640 1 -- --> v1:172.21.15.143:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1d300ce290 con 0x7f1d300cf330 2023-11-26T07:24:44.562 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:44.562+0000 7f1d358ad640 1 -- --> v2:172.21.15.143:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1d30161930 con 0x7f1d30160f60 2023-11-26T07:24:44.562 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:44.562+0000 7f1d358ad640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1d300ce570 con 0x7f1d30160850 2023-11-26T07:24:44.563 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:44.562+0000 7f1d350ac640 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7f1d30160f60 0x7f1d301613f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:24:44.563 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:44.562+0000 7f1d2ffff640 1 --1- >> v1:172.21.15.143:6789/0 conn(0x7f1d300cf330 0x7f1d300cda90 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.143:6789/0 says I am v1:172.21.15.143:49634/0 (socket says 172.21.15.143:49634) 2023-11-26T07:24:44.563 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:44.562+0000 7f1d2ffff640 1 -- 172.21.15.143:0/2200854251 learned_addr learned my addr 172.21.15.143:0/2200854251 (peer_addr_for_me v1:172.21.15.143:0/0) 2023-11-26T07:24:44.563 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:44.562+0000 7f1d348ab640 1 --2- 172.21.15.143:0/2200854251 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f1d30160850 0x7f1d300d3d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:24:44.563 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:44.562+0000 7f1d350ac640 1 -- 172.21.15.143:0/2200854251 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f1d30160850 msgr2=0x7f1d300d3d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-26T07:24:44.563 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:44.562+0000 7f1d350ac640 1 --2- 172.21.15.143:0/2200854251 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f1d30160850 0x7f1d300d3d30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:24:44.564 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:44.562+0000 7f1d350ac640 1 -- 172.21.15.143:0/2200854251 >> v1:172.21.15.143:6789/0 conn(0x7f1d300cf330 legacy=0x7f1d300cda90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:24:44.564 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:44.562+0000 7f1d350ac640 1 -- 172.21.15.143:0/2200854251 --> v2:172.21.15.143:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1d300ce850 con 0x7f1d30160f60 2023-11-26T07:24:44.564 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:44.562+0000 7f1d348ab640 1 --2- 172.21.15.143:0/2200854251 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f1d30160850 0x7f1d300d3d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-26T07:24:44.565 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:44.566+0000 7f1d350ac640 1 --2- 172.21.15.143:0/2200854251 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f1d30160f60 0x7f1d301613f0 secure :-1 s=READY pgs=1385 cs=0 l=1 rev1=1 crypto rx=0x7f1d28009d90 tx=0x7f1d2801c950 comp rx=0 tx=0).ready entity=mon.0 client_cookie=70dc1c75c232581f server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:24:44.565 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:44.566+0000 7f1d2f7fe640 1 -- 172.21.15.143:0/2200854251 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1d280a5a10 con 0x7f1d30160f60 2023-11-26T07:24:44.565 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:44.566+0000 7f1d2f7fe640 1 -- 172.21.15.143:0/2200854251 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1d280a5b70 con 0x7f1d30160f60 2023-11-26T07:24:44.565 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:44.566+0000 7f1d2f7fe640 1 -- 172.21.15.143:0/2200854251 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1d280ad440 con 0x7f1d30160f60 2023-11-26T07:24:44.566 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:44.566+0000 7f1d358ad640 1 -- 172.21.15.143:0/2200854251 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f1d30160f60 msgr2=0x7f1d301613f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:24:44.566 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:44.566+0000 7f1d358ad640 1 --2- 172.21.15.143:0/2200854251 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f1d30160f60 0x7f1d301613f0 secure :-1 s=READY pgs=1385 cs=0 l=1 rev1=1 crypto rx=0x7f1d28009d90 tx=0x7f1d2801c950 comp rx=0 tx=0).stop 2023-11-26T07:24:44.566 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:44.566+0000 7f1d358ad640 1 -- 172.21.15.143:0/2200854251 shutdown_connections 2023-11-26T07:24:44.567 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:44.566+0000 7f1d358ad640 1 --2- 172.21.15.143:0/2200854251 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f1d30160f60 0x7f1d301613f0 unknown :-1 s=CLOSED pgs=1385 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:24:44.567 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:44.566+0000 7f1d358ad640 1 --2- 172.21.15.143:0/2200854251 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f1d30160850 0x7f1d300d3d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:24:44.567 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:44.566+0000 7f1d358ad640 1 -- 172.21.15.143:0/2200854251 >> 172.21.15.143:0/2200854251 conn(0x7f1d3009c5a0 msgr2=0x7f1d300d3260 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:24:44.567 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:44.566+0000 7f1d358ad640 1 -- 172.21.15.143:0/2200854251 shutdown_connections 2023-11-26T07:24:44.567 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:44.566+0000 7f1d358ad640 1 -- 172.21.15.143:0/2200854251 wait complete. 2023-11-26T07:24:44.568 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:44.566+0000 7f1d358ad640 1 Processor -- start 2023-11-26T07:24:44.568 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:44.566+0000 7f1d358ad640 1 -- start start 2023-11-26T07:24:44.572 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:44.566+0000 7f1d358ad640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f1d300cf330 0x7f1d301572e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:24:44.572 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:44.566+0000 7f1d358ad640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f1d30160850 0x7f1d300cc340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:24:44.572 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:44.566+0000 7f1d358ad640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f1d30160f60 0x7f1d300b7fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:24:44.572 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:44.566+0000 7f1d358ad640 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1d300cce80 con 0x7f1d30160f60 2023-11-26T07:24:44.572 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:44.566+0000 7f1d358ad640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1d300cd0f0 con 0x7f1d300cf330 2023-11-26T07:24:44.573 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:44.566+0000 7f1d358ad640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1d300cd3d0 con 0x7f1d30160850 2023-11-26T07:24:44.573 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:44.570+0000 7f1d350ac640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f1d30160f60 0x7f1d300b7fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:24:44.573 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:44.570+0000 7f1d350ac640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f1d30160f60 0x7f1d300b7fe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:58472/0 (socket says 172.21.15.143:58472) 2023-11-26T07:24:44.573 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:44.570+0000 7f1d350ac640 1 -- 172.21.15.143:0/304784366 learned_addr learned my addr 172.21.15.143:0/304784366 (peer_addr_for_me v2:172.21.15.143:0/0) 2023-11-26T07:24:44.573 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:44.570+0000 7f1d2ffff640 1 --2- 172.21.15.143:0/304784366 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f1d30160850 0x7f1d300cc340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:24:44.574 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:44.570+0000 7f1d348ab640 1 --2- 172.21.15.143:0/304784366 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f1d300cf330 0x7f1d301572e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:24:44.574 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:44.570+0000 7f1d350ac640 1 -- 172.21.15.143:0/304784366 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f1d30160850 msgr2=0x7f1d300cc340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:24:44.574 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:44.570+0000 7f1d350ac640 1 --2- 172.21.15.143:0/304784366 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f1d30160850 0x7f1d300cc340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:24:44.575 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:44.570+0000 7f1d350ac640 1 -- 172.21.15.143:0/304784366 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f1d300cf330 msgr2=0x7f1d301572e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:24:44.575 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:44.570+0000 7f1d350ac640 1 --2- 172.21.15.143:0/304784366 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f1d300cf330 0x7f1d301572e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:24:44.575 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:44.570+0000 7f1d350ac640 1 -- 172.21.15.143:0/304784366 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1d28009a70 con 0x7f1d30160f60 2023-11-26T07:24:44.575 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:44.570+0000 7f1d348ab640 1 --2- 172.21.15.143:0/304784366 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f1d300cf330 0x7f1d301572e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-26T07:24:44.575 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:44.570+0000 7f1d350ac640 1 --2- 172.21.15.143:0/304784366 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f1d30160f60 0x7f1d300b7fe0 secure :-1 s=READY pgs=1386 cs=0 l=1 rev1=1 crypto rx=0x7f1d280b0040 tx=0x7f1d280196f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:24:44.576 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:44.570+0000 7f1d2d7fa640 1 -- 172.21.15.143:0/304784366 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1d280ade40 con 0x7f1d30160f60 2023-11-26T07:24:44.576 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:44.570+0000 7f1d2d7fa640 1 -- 172.21.15.143:0/304784366 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1d28009450 con 0x7f1d30160f60 2023-11-26T07:24:44.576 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:44.570+0000 7f1d2d7fa640 1 -- 172.21.15.143:0/304784366 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1d280b9690 con 0x7f1d30160f60 2023-11-26T07:24:44.576 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:44.570+0000 7f1d358ad640 1 -- 172.21.15.143:0/304784366 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1d3015c1b0 con 0x7f1d30160f60 2023-11-26T07:24:44.576 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:44.570+0000 7f1d358ad640 1 -- 172.21.15.143:0/304784366 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1d30157820 con 0x7f1d30160f60 2023-11-26T07:24:44.576 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:44.570+0000 7f1d358ad640 1 -- 172.21.15.143:0/304784366 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1d30067500 con 0x7f1d30160f60 2023-11-26T07:24:44.577 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:44.570+0000 7f1d2d7fa640 1 -- 172.21.15.143:0/304784366 <== mon.0 v2:172.21.15.143:3300/0 4 ==== mgrmap(e 18) v1 ==== 86107+0+0 (secure 0 0 0) 0x7f1d280b97f0 con 0x7f1d30160f60 2023-11-26T07:24:44.578 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:44.574+0000 7f1d2d7fa640 1 --2- 172.21.15.143:0/304784366 >> [v2:172.21.15.188:6836/3401128553,v1:172.21.15.188:6837/3401128553] conn(0x7f1d14066740 0x7f1d14068c00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:24:44.578 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:44.574+0000 7f1d2d7fa640 1 -- 172.21.15.143:0/304784366 <== mon.0 v2:172.21.15.143:3300/0 5 ==== osd_map(103..103 src has 1..103) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f1d2811f8e0 con 0x7f1d30160f60 2023-11-26T07:24:44.578 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:44.574+0000 7f1d348ab640 1 --2- 172.21.15.143:0/304784366 >> [v2:172.21.15.188:6836/3401128553,v1:172.21.15.188:6837/3401128553] conn(0x7f1d14066740 0x7f1d14068c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:24:44.578 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:44.574+0000 7f1d348ab640 1 --2- 172.21.15.143:0/304784366 >> [v2:172.21.15.188:6836/3401128553,v1:172.21.15.188:6837/3401128553] conn(0x7f1d14066740 0x7f1d14068c00 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7f1d24009970 tx=0x7f1d24008040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:24:44.578 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:44.574+0000 7f1d2d7fa640 1 -- 172.21.15.143:0/304784366 <== mon.0 v2:172.21.15.143:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172649 (secure 0 0 0) 0x7f1d280eca10 con 0x7f1d30160f60 2023-11-26T07:24:44.730 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:44.730+0000 7f1d358ad640 1 -- 172.21.15.143:0/304784366 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f1d3009ca80 con 0x7f1d30160f60 2023-11-26T07:24:44.731 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:44.730+0000 7f1d2d7fa640 1 -- 172.21.15.143:0/304784366 <== mon.0 v2:172.21.15.143:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v103) v1 ==== 104+0+11 (secure 0 0 0) 0x7f1d280f0030 con 0x7f1d30160f60 2023-11-26T07:24:44.731 INFO:teuthology.orchestra.run.smithi143.stdout:pg_num: 32 2023-11-26T07:24:44.733 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:44.730+0000 7f1d358ad640 1 -- 172.21.15.143:0/304784366 >> [v2:172.21.15.188:6836/3401128553,v1:172.21.15.188:6837/3401128553] conn(0x7f1d14066740 msgr2=0x7f1d14068c00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:24:44.733 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:44.730+0000 7f1d358ad640 1 --2- 172.21.15.143:0/304784366 >> [v2:172.21.15.188:6836/3401128553,v1:172.21.15.188:6837/3401128553] conn(0x7f1d14066740 0x7f1d14068c00 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7f1d24009970 tx=0x7f1d24008040 comp rx=0 tx=0).stop 2023-11-26T07:24:44.733 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:44.730+0000 7f1d358ad640 1 -- 172.21.15.143:0/304784366 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f1d30160f60 msgr2=0x7f1d300b7fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:24:44.733 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:44.730+0000 7f1d358ad640 1 --2- 172.21.15.143:0/304784366 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f1d30160f60 0x7f1d300b7fe0 secure :-1 s=READY pgs=1386 cs=0 l=1 rev1=1 crypto rx=0x7f1d280b0040 tx=0x7f1d280196f0 comp rx=0 tx=0).stop 2023-11-26T07:24:44.733 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:44.730+0000 7f1d358ad640 1 -- 172.21.15.143:0/304784366 shutdown_connections 2023-11-26T07:24:44.733 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:44.730+0000 7f1d358ad640 1 --2- 172.21.15.143:0/304784366 >> [v2:172.21.15.188:6836/3401128553,v1:172.21.15.188:6837/3401128553] conn(0x7f1d14066740 0x7f1d14068c00 unknown :-1 s=CLOSED pgs=158 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:24:44.733 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:44.730+0000 7f1d358ad640 1 --2- 172.21.15.143:0/304784366 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f1d30160f60 0x7f1d300b7fe0 unknown :-1 s=CLOSED pgs=1386 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:24:44.733 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:44.730+0000 7f1d358ad640 1 --2- 172.21.15.143:0/304784366 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f1d30160850 0x7f1d300cc340 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:24:44.734 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:44.730+0000 7f1d358ad640 1 --2- 172.21.15.143:0/304784366 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f1d300cf330 0x7f1d301572e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:24:44.734 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:44.730+0000 7f1d358ad640 1 -- 172.21.15.143:0/304784366 >> 172.21.15.143:0/304784366 conn(0x7f1d3009c5a0 msgr2=0x7f1d300b4c20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:24:44.734 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:44.730+0000 7f1d358ad640 1 -- 172.21.15.143:0/304784366 shutdown_connections 2023-11-26T07:24:44.734 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:44.730+0000 7f1d358ad640 1 -- 172.21.15.143:0/304784366 wait complete. 2023-11-26T07:24:44.751 DEBUG:teuthology.orchestra.run.smithi143:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-11-26T07:24:44.891 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:44.890+0000 7fa2b8980640 1 Processor -- start 2023-11-26T07:24:44.891 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:44.890+0000 7fa2b8980640 1 -- start start 2023-11-26T07:24:44.893 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:44.890+0000 7fa2b8980640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fa2b4143ac0 0x7fa2b4070370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:24:44.893 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:44.890+0000 7fa2b8980640 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7fa2b40708b0 0x7fa2b40cf280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:24:44.893 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:44.890+0000 7fa2b8980640 1 -- --> v1:172.21.15.143:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa2b40cfbf0 con 0x7fa2b40cf7c0 2023-11-26T07:24:44.893 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:44.890+0000 7fa2b8980640 1 -- --> v2:172.21.15.143:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa2b4143ea0 con 0x7fa2b40708b0 2023-11-26T07:24:44.893 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:44.890+0000 7fa2b8980640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa2b40cc730 con 0x7fa2b4143ac0 2023-11-26T07:24:44.894 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:44.894+0000 7fa2b1d9b640 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7fa2b40708b0 0x7fa2b40cf280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:24:44.894 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:44.894+0000 7fa2b1d9b640 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7fa2b40708b0 0x7fa2b40cf280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:58488/0 (socket says 172.21.15.143:58488) 2023-11-26T07:24:44.894 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:44.894+0000 7fa2b1d9b640 1 -- 172.21.15.143:0/3618010514 learned_addr learned my addr 172.21.15.143:0/3618010514 (peer_addr_for_me v2:172.21.15.143:0/0) 2023-11-26T07:24:44.894 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:44.894+0000 7fa2b2d9d640 1 --1- 172.21.15.143:0/3618010514 >> v1:172.21.15.143:6789/0 conn(0x7fa2b40cf7c0 0x7fa2b40b1ef0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.143:6789/0 says I am v1:172.21.15.143:49640/0 (socket says 172.21.15.143:49640) 2023-11-26T07:24:44.894 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:44.894+0000 7fa2b259c640 1 --2- 172.21.15.143:0/3618010514 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fa2b4143ac0 0x7fa2b4070370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:24:44.895 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:44.894+0000 7fa2b1d9b640 1 -- 172.21.15.143:0/3618010514 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fa2b4143ac0 msgr2=0x7fa2b4070370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-26T07:24:44.895 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:44.894+0000 7fa2b1d9b640 1 --2- 172.21.15.143:0/3618010514 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fa2b4143ac0 0x7fa2b4070370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:24:44.895 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:44.894+0000 7fa2b1d9b640 1 -- 172.21.15.143:0/3618010514 >> v1:172.21.15.143:6789/0 conn(0x7fa2b40cf7c0 legacy=0x7fa2b40b1ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:24:44.895 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:44.894+0000 7fa2b1d9b640 1 -- 172.21.15.143:0/3618010514 --> v2:172.21.15.143:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa2b4070ee0 con 0x7fa2b40708b0 2023-11-26T07:24:44.896 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:44.894+0000 7fa2b1d9b640 1 --2- 172.21.15.143:0/3618010514 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fa2b40708b0 0x7fa2b40cf280 secure :-1 s=READY pgs=1388 cs=0 l=1 rev1=1 crypto rx=0x7fa2a8009870 tx=0x7fa2a801c540 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dfca36df2a8bb126 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:24:44.897 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:44.894+0000 7fa2b159a640 1 -- 172.21.15.143:0/3618010514 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa2a80ab650 con 0x7fa2b40708b0 2023-11-26T07:24:44.897 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:44.894+0000 7fa2b159a640 1 -- 172.21.15.143:0/3618010514 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa2a80ab7b0 con 0x7fa2b40708b0 2023-11-26T07:24:44.897 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:44.894+0000 7fa2b159a640 1 -- 172.21.15.143:0/3618010514 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa2a80abb40 con 0x7fa2b40708b0 2023-11-26T07:24:44.898 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:44.898+0000 7fa2b8980640 1 -- 172.21.15.143:0/3618010514 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fa2b40708b0 msgr2=0x7fa2b40cf280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:24:44.898 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:44.898+0000 7fa2b8980640 1 --2- 172.21.15.143:0/3618010514 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fa2b40708b0 0x7fa2b40cf280 secure :-1 s=READY pgs=1388 cs=0 l=1 rev1=1 crypto rx=0x7fa2a8009870 tx=0x7fa2a801c540 comp rx=0 tx=0).stop 2023-11-26T07:24:44.898 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:44.898+0000 7fa2b8980640 1 -- 172.21.15.143:0/3618010514 shutdown_connections 2023-11-26T07:24:44.898 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:44.898+0000 7fa2b8980640 1 --2- 172.21.15.143:0/3618010514 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fa2b40708b0 0x7fa2b40cf280 unknown :-1 s=CLOSED pgs=1388 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:24:44.898 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:44.898+0000 7fa2b8980640 1 --2- 172.21.15.143:0/3618010514 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fa2b4143ac0 0x7fa2b4070370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:24:44.898 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:44.898+0000 7fa2b8980640 1 -- 172.21.15.143:0/3618010514 >> 172.21.15.143:0/3618010514 conn(0x7fa2b406a890 msgr2=0x7fa2b406aca0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:24:44.898 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:44.898+0000 7fa2b8980640 1 -- 172.21.15.143:0/3618010514 shutdown_connections 2023-11-26T07:24:44.898 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:44.898+0000 7fa2b8980640 1 -- 172.21.15.143:0/3618010514 wait complete. 2023-11-26T07:24:44.899 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:44.898+0000 7fa2b8980640 1 Processor -- start 2023-11-26T07:24:44.899 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:44.898+0000 7fa2b8980640 1 -- start start 2023-11-26T07:24:44.902 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:44.898+0000 7fa2b8980640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fa2b40708b0 0x7fa2b4153a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:24:44.902 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:44.898+0000 7fa2b8980640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fa2b40cf7c0 0x7fa2b40adc20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:24:44.902 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:44.898+0000 7fa2b8980640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fa2b4143ac0 0x7fa2b41386a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:24:44.902 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:44.898+0000 7fa2b8980640 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa2b4153f70 con 0x7fa2b40708b0 2023-11-26T07:24:44.902 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:44.898+0000 7fa2b8980640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa2b4157690 con 0x7fa2b40cf7c0 2023-11-26T07:24:44.902 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:44.898+0000 7fa2b8980640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa2b41570d0 con 0x7fa2b4143ac0 2023-11-26T07:24:44.902 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:44.898+0000 7fa2b259c640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fa2b40708b0 0x7fa2b4153a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:24:44.902 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:44.898+0000 7fa2b259c640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fa2b40708b0 0x7fa2b4153a30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:58494/0 (socket says 172.21.15.143:58494) 2023-11-26T07:24:44.903 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:44.898+0000 7fa2b259c640 1 -- 172.21.15.143:0/1770233548 learned_addr learned my addr 172.21.15.143:0/1770233548 (peer_addr_for_me v2:172.21.15.143:0/0) 2023-11-26T07:24:44.903 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:44.902+0000 7fa2b2d9d640 1 --2- 172.21.15.143:0/1770233548 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fa2b4143ac0 0x7fa2b41386a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:24:44.903 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:44.902+0000 7fa2b1d9b640 1 --2- 172.21.15.143:0/1770233548 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fa2b40cf7c0 0x7fa2b40adc20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:24:44.903 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:44.902+0000 7fa2b259c640 1 -- 172.21.15.143:0/1770233548 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fa2b4143ac0 msgr2=0x7fa2b41386a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:24:44.903 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:44.902+0000 7fa2b259c640 1 --2- 172.21.15.143:0/1770233548 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fa2b4143ac0 0x7fa2b41386a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:24:44.903 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:44.902+0000 7fa2b259c640 1 -- 172.21.15.143:0/1770233548 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fa2b40cf7c0 msgr2=0x7fa2b40adc20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:24:44.903 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:44.902+0000 7fa2b259c640 1 --2- 172.21.15.143:0/1770233548 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fa2b40cf7c0 0x7fa2b40adc20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:24:44.903 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:44.902+0000 7fa2b259c640 1 -- 172.21.15.143:0/1770233548 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa2a80094d0 con 0x7fa2b40708b0 2023-11-26T07:24:44.903 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:44.902+0000 7fa2b1d9b640 1 --2- 172.21.15.143:0/1770233548 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fa2b40cf7c0 0x7fa2b40adc20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-26T07:24:44.903 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:44.902+0000 7fa2b259c640 1 --2- 172.21.15.143:0/1770233548 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fa2b40708b0 0x7fa2b4153a30 secure :-1 s=READY pgs=1389 cs=0 l=1 rev1=1 crypto rx=0x7fa2a400d5f0 tx=0x7fa2a400dab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:24:44.903 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:44.902+0000 7fa2a2ffd640 1 -- 172.21.15.143:0/1770233548 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa2a40125f0 con 0x7fa2b40708b0 2023-11-26T07:24:44.903 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:44.902+0000 7fa2a2ffd640 1 -- 172.21.15.143:0/1770233548 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa2a4012750 con 0x7fa2b40708b0 2023-11-26T07:24:44.904 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:44.902+0000 7fa2b8980640 1 -- 172.21.15.143:0/1770233548 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa2b41573b0 con 0x7fa2b40708b0 2023-11-26T07:24:44.904 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:44.902+0000 7fa2a2ffd640 1 -- 172.21.15.143:0/1770233548 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa2a4012ae0 con 0x7fa2b40708b0 2023-11-26T07:24:44.904 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:44.902+0000 7fa2b8980640 1 -- 172.21.15.143:0/1770233548 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa2b40ad700 con 0x7fa2b40708b0 2023-11-26T07:24:44.904 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:44.902+0000 7fa2b8980640 1 -- 172.21.15.143:0/1770233548 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa2b4067680 con 0x7fa2b40708b0 2023-11-26T07:24:44.904 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:44.902+0000 7fa2a2ffd640 1 -- 172.21.15.143:0/1770233548 <== mon.0 v2:172.21.15.143:3300/0 4 ==== mgrmap(e 18) v1 ==== 86107+0+0 (secure 0 0 0) 0x7fa2a4012c40 con 0x7fa2b40708b0 2023-11-26T07:24:44.905 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:44.906+0000 7fa2a2ffd640 1 --2- 172.21.15.143:0/1770233548 >> [v2:172.21.15.188:6836/3401128553,v1:172.21.15.188:6837/3401128553] conn(0x7fa294066740 0x7fa294068c00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:24:44.905 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:44.906+0000 7fa2a2ffd640 1 -- 172.21.15.143:0/1770233548 <== mon.0 v2:172.21.15.143:3300/0 5 ==== osd_map(103..103 src has 1..103) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fa2a4084700 con 0x7fa2b40708b0 2023-11-26T07:24:44.906 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:44.906+0000 7fa2b1d9b640 1 --2- 172.21.15.143:0/1770233548 >> [v2:172.21.15.188:6836/3401128553,v1:172.21.15.188:6837/3401128553] conn(0x7fa294066740 0x7fa294068c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:24:44.906 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:44.906+0000 7fa2b1d9b640 1 --2- 172.21.15.143:0/1770233548 >> [v2:172.21.15.188:6836/3401128553,v1:172.21.15.188:6837/3401128553] conn(0x7fa294066740 0x7fa294068c00 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7fa2a80af000 tx=0x7fa2a8008040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:24:44.909 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:44.910+0000 7fa2a2ffd640 1 -- 172.21.15.143:0/1770233548 <== mon.0 v2:172.21.15.143:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172649 (secure 0 0 0) 0x7fa2a4051900 con 0x7fa2b40708b0 2023-11-26T07:24:45.059 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:45.058+0000 7fa2b8980640 1 -- 172.21.15.143:0/1770233548 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7fa2b406af50 con 0x7fa2b40708b0 2023-11-26T07:24:45.059 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:45.058+0000 7fa2a2ffd640 1 -- 172.21.15.143:0/1770233548 <== mon.0 v2:172.21.15.143:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v103) v1 ==== 100+0+11 (secure 0 0 0) 0x7fa2a4054f20 con 0x7fa2b40708b0 2023-11-26T07:24:45.059 INFO:teuthology.orchestra.run.smithi143.stdout:pg_num: 64 2023-11-26T07:24:45.061 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:45.058+0000 7fa2b8980640 1 -- 172.21.15.143:0/1770233548 >> [v2:172.21.15.188:6836/3401128553,v1:172.21.15.188:6837/3401128553] conn(0x7fa294066740 msgr2=0x7fa294068c00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:24:45.061 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:45.062+0000 7fa2b8980640 1 --2- 172.21.15.143:0/1770233548 >> [v2:172.21.15.188:6836/3401128553,v1:172.21.15.188:6837/3401128553] conn(0x7fa294066740 0x7fa294068c00 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7fa2a80af000 tx=0x7fa2a8008040 comp rx=0 tx=0).stop 2023-11-26T07:24:45.061 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:45.062+0000 7fa2b8980640 1 -- 172.21.15.143:0/1770233548 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fa2b40708b0 msgr2=0x7fa2b4153a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:24:45.062 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:45.062+0000 7fa2b8980640 1 --2- 172.21.15.143:0/1770233548 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fa2b40708b0 0x7fa2b4153a30 secure :-1 s=READY pgs=1389 cs=0 l=1 rev1=1 crypto rx=0x7fa2a400d5f0 tx=0x7fa2a400dab0 comp rx=0 tx=0).stop 2023-11-26T07:24:45.062 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:45.062+0000 7fa2b8980640 1 -- 172.21.15.143:0/1770233548 shutdown_connections 2023-11-26T07:24:45.062 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:45.062+0000 7fa2b8980640 1 --2- 172.21.15.143:0/1770233548 >> [v2:172.21.15.188:6836/3401128553,v1:172.21.15.188:6837/3401128553] conn(0x7fa294066740 0x7fa294068c00 unknown :-1 s=CLOSED pgs=159 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:24:45.062 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:45.062+0000 7fa2b8980640 1 --2- 172.21.15.143:0/1770233548 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fa2b4143ac0 0x7fa2b41386a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:24:45.062 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:45.062+0000 7fa2b8980640 1 --2- 172.21.15.143:0/1770233548 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fa2b40cf7c0 0x7fa2b40adc20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:24:45.062 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:45.062+0000 7fa2b8980640 1 --2- 172.21.15.143:0/1770233548 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fa2b40708b0 0x7fa2b4153a30 unknown :-1 s=CLOSED pgs=1389 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:24:45.062 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:45.062+0000 7fa2b8980640 1 -- 172.21.15.143:0/1770233548 >> 172.21.15.143:0/1770233548 conn(0x7fa2b406a890 msgr2=0x7fa2b413dee0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:24:45.062 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:45.062+0000 7fa2b8980640 1 -- 172.21.15.143:0/1770233548 shutdown_connections 2023-11-26T07:24:45.062 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:45.062+0000 7fa2b8980640 1 -- 172.21.15.143:0/1770233548 wait complete. 2023-11-26T07:24:45.080 DEBUG:teuthology.orchestra.run.smithi143:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2023-11-26T07:24:45.219 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:45.218+0000 7fae7f1c4640 1 Processor -- start 2023-11-26T07:24:45.219 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:45.218+0000 7fae7f1c4640 1 -- start start 2023-11-26T07:24:45.221 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:45.218+0000 7fae7f1c4640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fae780cf990 0x7fae7815efa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:24:45.221 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:45.218+0000 7fae7f1c4640 1 -- --> v1:172.21.15.143:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fae7806c520 con 0x7fae7815f4e0 2023-11-26T07:24:45.221 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:45.218+0000 7fae7f1c4640 1 -- --> v1:172.21.15.188:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fae7806c800 con 0x7fae7813e290 2023-11-26T07:24:45.221 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:45.218+0000 7fae7f1c4640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fae7806cae0 con 0x7fae780cf990 2023-11-26T07:24:45.221 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:45.218+0000 7fae7d761640 1 --1- >> v1:172.21.15.143:6789/0 conn(0x7fae7815f4e0 0x7fae780cd7b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.143:6789/0 says I am v1:172.21.15.143:49656/0 (socket says 172.21.15.143:49656) 2023-11-26T07:24:45.221 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:45.218+0000 7fae7d761640 1 -- 172.21.15.143:0/38439469 learned_addr learned my addr 172.21.15.143:0/38439469 (peer_addr_for_me v1:172.21.15.143:0/0) 2023-11-26T07:24:45.223 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:45.222+0000 7fae6ffff640 1 --2- 172.21.15.143:0/38439469 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fae780cf990 0x7fae7815efa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:24:45.223 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:45.222+0000 7fae6f7fe640 1 -- 172.21.15.143:0/38439469 <== mon.0 v1:172.21.15.143:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1131811316 0 0) 0x7fae7806c520 con 0x7fae7815f4e0 2023-11-26T07:24:45.223 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:45.222+0000 7fae6f7fe640 1 -- 172.21.15.143:0/38439469 --> v1:172.21.15.143:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fae60003530 con 0x7fae7815f4e0 2023-11-26T07:24:45.223 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:45.222+0000 7fae6f7fe640 1 -- 172.21.15.143:0/38439469 <== mon.0 v1:172.21.15.143:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3676528190 0 0) 0x7fae60003530 con 0x7fae7815f4e0 2023-11-26T07:24:45.223 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:45.222+0000 7fae6f7fe640 1 -- 172.21.15.143:0/38439469 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fae780cf990 msgr2=0x7fae7815efa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-26T07:24:45.223 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:45.222+0000 7fae6f7fe640 1 --2- 172.21.15.143:0/38439469 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fae780cf990 0x7fae7815efa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:24:45.223 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:45.222+0000 7fae6f7fe640 1 -- 172.21.15.143:0/38439469 >> v1:172.21.15.188:6789/0 conn(0x7fae7813e290 legacy=0x7fae780cf280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:24:45.224 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:45.222+0000 7fae6f7fe640 1 -- 172.21.15.143:0/38439469 --> v1:172.21.15.143:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fae7806cdc0 con 0x7fae7815f4e0 2023-11-26T07:24:45.224 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:45.222+0000 7fae6f7fe640 1 -- 172.21.15.143:0/38439469 <== mon.0 v1:172.21.15.143:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2137882523 0 0) 0x7fae70001a00 con 0x7fae7815f4e0 2023-11-26T07:24:45.224 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:45.222+0000 7fae6f7fe640 1 -- 172.21.15.143:0/38439469 >> v1:172.21.15.143:6789/0 conn(0x7fae7815f4e0 legacy=0x7fae780cd7b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:24:45.224 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:45.222+0000 7fae6f7fe640 1 --2- 172.21.15.143:0/38439469 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fae780cf990 0x7fae600039b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:24:45.224 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:45.222+0000 7fae6f7fe640 1 --2- 172.21.15.143:0/38439469 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fae60003f30 0x7fae600047d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:24:45.224 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:45.222+0000 7fae6f7fe640 1 --2- 172.21.15.143:0/38439469 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fae60004d10 0x7fae60005190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:24:45.224 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:45.222+0000 7fae6f7fe640 1 -- 172.21.15.143:0/38439469 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fae7806cae0 con 0x7fae60004d10 2023-11-26T07:24:45.224 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:45.222+0000 7fae6f7fe640 1 -- 172.21.15.143:0/38439469 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fae6000ba70 con 0x7fae60003f30 2023-11-26T07:24:45.224 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:45.222+0000 7fae6f7fe640 1 -- 172.21.15.143:0/38439469 --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fae6000bc30 con 0x7fae780cf990 2023-11-26T07:24:45.224 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:45.222+0000 7fae7d761640 1 --2- 172.21.15.143:0/38439469 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fae60004d10 0x7fae60005190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:24:45.224 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:45.222+0000 7fae7cf60640 1 --2- 172.21.15.143:0/38439469 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fae780cf990 0x7fae600039b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:24:45.225 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:45.222+0000 7fae6ffff640 1 --2- 172.21.15.143:0/38439469 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fae60003f30 0x7fae600047d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:24:45.225 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:45.222+0000 7fae7d761640 1 -- 172.21.15.143:0/38439469 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fae780cf990 msgr2=0x7fae600039b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-26T07:24:45.225 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:45.222+0000 7fae7d761640 1 --2- 172.21.15.143:0/38439469 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fae780cf990 0x7fae600039b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:24:45.225 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:45.222+0000 7fae7d761640 1 -- 172.21.15.143:0/38439469 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fae60003f30 msgr2=0x7fae600047d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-26T07:24:45.225 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:45.222+0000 7fae7d761640 1 --2- 172.21.15.143:0/38439469 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fae60003f30 0x7fae600047d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:24:45.225 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:45.222+0000 7fae7d761640 1 -- 172.21.15.143:0/38439469 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fae60003530 con 0x7fae60004d10 2023-11-26T07:24:45.225 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:45.226+0000 7fae7d761640 1 --2- 172.21.15.143:0/38439469 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fae60004d10 0x7fae60005190 secure :-1 s=READY pgs=1391 cs=0 l=1 rev1=1 crypto rx=0x7fae7000a830 tx=0x7fae7001d340 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9e7acf26a7834461 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:24:45.226 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:45.226+0000 7fae6f7fe640 1 -- 172.21.15.143:0/38439469 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fae700072d0 con 0x7fae60004d10 2023-11-26T07:24:45.226 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:45.226+0000 7fae6f7fe640 1 -- 172.21.15.143:0/38439469 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fae700a8070 con 0x7fae60004d10 2023-11-26T07:24:45.226 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:45.226+0000 7fae6f7fe640 1 -- 172.21.15.143:0/38439469 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fae700076f0 con 0x7fae60004d10 2023-11-26T07:24:45.227 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:45.226+0000 7fae7f1c4640 1 -- 172.21.15.143:0/38439469 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fae60004d10 msgr2=0x7fae60005190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:24:45.227 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:45.226+0000 7fae7f1c4640 1 --2- 172.21.15.143:0/38439469 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fae60004d10 0x7fae60005190 secure :-1 s=READY pgs=1391 cs=0 l=1 rev1=1 crypto rx=0x7fae7000a830 tx=0x7fae7001d340 comp rx=0 tx=0).stop 2023-11-26T07:24:45.227 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:45.226+0000 7fae7d761640 1 -- 172.21.15.143:0/38439469 reap_dead start 2023-11-26T07:24:45.227 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:45.226+0000 7fae7f1c4640 1 -- 172.21.15.143:0/38439469 shutdown_connections 2023-11-26T07:24:45.227 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:45.226+0000 7fae7f1c4640 1 -- 172.21.15.143:0/38439469 >> 172.21.15.143:0/38439469 conn(0x7fae7806a890 msgr2=0x7fae7806aca0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:24:45.227 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:45.226+0000 7fae7f1c4640 1 -- 172.21.15.143:0/38439469 shutdown_connections 2023-11-26T07:24:45.227 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:45.226+0000 7fae7f1c4640 1 -- 172.21.15.143:0/38439469 wait complete. 2023-11-26T07:24:45.228 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:45.226+0000 7fae7f1c4640 1 Processor -- start 2023-11-26T07:24:45.228 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:45.226+0000 7fae7f1c4640 1 -- start start 2023-11-26T07:24:45.229 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:45.226+0000 7fae7f1c4640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fae60004d10 0x7fae781f0570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:24:45.229 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:45.226+0000 7fae7f1c4640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fae781f3af0 0x7fae781f2300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:24:45.229 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:45.226+0000 7fae7f1c4640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fae781f3360 0x7fae7815d6e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:24:45.230 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:45.230+0000 7fae7f1c4640 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fae781f2840 con 0x7fae781f3af0 2023-11-26T07:24:45.230 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:45.230+0000 7fae7f1c4640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fae780a87d0 con 0x7fae60004d10 2023-11-26T07:24:45.230 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:45.230+0000 7fae7f1c4640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fae781f3fb0 con 0x7fae781f3360 2023-11-26T07:24:45.230 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:45.230+0000 7fae6ffff640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fae781f3af0 0x7fae781f2300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:24:45.230 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:45.230+0000 7fae6ffff640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fae781f3af0 0x7fae781f2300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:58518/0 (socket says 172.21.15.143:58518) 2023-11-26T07:24:45.230 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:45.230+0000 7fae6ffff640 1 -- 172.21.15.143:0/3871263992 learned_addr learned my addr 172.21.15.143:0/3871263992 (peer_addr_for_me v2:172.21.15.143:0/0) 2023-11-26T07:24:45.230 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:45.230+0000 7fae7d761640 1 --2- 172.21.15.143:0/3871263992 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fae781f3360 0x7fae7815d6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:24:45.230 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:45.230+0000 7fae7cf60640 1 --2- 172.21.15.143:0/3871263992 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fae60004d10 0x7fae781f0570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:24:45.231 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:45.230+0000 7fae6ffff640 1 -- 172.21.15.143:0/3871263992 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fae781f3360 msgr2=0x7fae7815d6e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:24:45.231 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:45.230+0000 7fae6ffff640 1 --2- 172.21.15.143:0/3871263992 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fae781f3360 0x7fae7815d6e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:24:45.231 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:45.230+0000 7fae6ffff640 1 -- 172.21.15.143:0/3871263992 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fae60004d10 msgr2=0x7fae781f0570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:24:45.232 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:45.230+0000 7fae6ffff640 1 --2- 172.21.15.143:0/3871263992 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fae60004d10 0x7fae781f0570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:24:45.232 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:45.230+0000 7fae6ffff640 1 -- 172.21.15.143:0/3871263992 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fae781f4290 con 0x7fae781f3af0 2023-11-26T07:24:45.232 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:45.230+0000 7fae6ffff640 1 --2- 172.21.15.143:0/3871263992 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fae781f3af0 0x7fae781f2300 secure :-1 s=READY pgs=1392 cs=0 l=1 rev1=1 crypto rx=0x7fae680119d0 tx=0x7fae68011e90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:24:45.232 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:45.230+0000 7fae6d7fa640 1 -- 172.21.15.143:0/3871263992 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fae6800fb50 con 0x7fae781f3af0 2023-11-26T07:24:45.232 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:45.230+0000 7fae6d7fa640 1 -- 172.21.15.143:0/3871263992 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fae6800fcb0 con 0x7fae781f3af0 2023-11-26T07:24:45.232 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:45.230+0000 7fae7f1c4640 1 -- 172.21.15.143:0/3871263992 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fae781f4570 con 0x7fae781f3af0 2023-11-26T07:24:45.232 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:45.230+0000 7fae6d7fa640 1 -- 172.21.15.143:0/3871263992 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fae68014440 con 0x7fae781f3af0 2023-11-26T07:24:45.232 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:45.230+0000 7fae7f1c4640 1 -- 172.21.15.143:0/3871263992 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fae781f4b30 con 0x7fae781f3af0 2023-11-26T07:24:45.232 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:45.230+0000 7fae7f1c4640 1 -- 172.21.15.143:0/3871263992 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fae78067680 con 0x7fae781f3af0 2023-11-26T07:24:45.233 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:45.230+0000 7fae6d7fa640 1 -- 172.21.15.143:0/3871263992 <== mon.0 v2:172.21.15.143:3300/0 4 ==== mgrmap(e 18) v1 ==== 86107+0+0 (secure 0 0 0) 0x7fae680145a0 con 0x7fae781f3af0 2023-11-26T07:24:45.233 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:45.234+0000 7fae6d7fa640 1 --2- 172.21.15.143:0/3871263992 >> [v2:172.21.15.188:6836/3401128553,v1:172.21.15.188:6837/3401128553] conn(0x7fae64066740 0x7fae64068c00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:24:45.233 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:45.234+0000 7fae6d7fa640 1 -- 172.21.15.143:0/3871263992 <== mon.0 v2:172.21.15.143:3300/0 5 ==== osd_map(103..103 src has 1..103) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fae68084a10 con 0x7fae781f3af0 2023-11-26T07:24:45.234 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:45.234+0000 7fae7cf60640 1 --2- 172.21.15.143:0/3871263992 >> [v2:172.21.15.188:6836/3401128553,v1:172.21.15.188:6837/3401128553] conn(0x7fae64066740 0x7fae64068c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:24:45.234 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:45.234+0000 7fae7cf60640 1 --2- 172.21.15.143:0/3871263992 >> [v2:172.21.15.188:6836/3401128553,v1:172.21.15.188:6837/3401128553] conn(0x7fae64066740 0x7fae64068c00 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7fae7813d9f0 tx=0x7fae74009040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:24:45.237 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:45.238+0000 7fae6d7fa640 1 -- 172.21.15.143:0/3871263992 <== mon.0 v2:172.21.15.143:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172649 (secure 0 0 0) 0x7fae68051cb0 con 0x7fae781f3af0 2023-11-26T07:24:45.399 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:45.398+0000 7fae7f1c4640 1 -- 172.21.15.143:0/3871263992 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) v1 -- 0x7fae7806df50 con 0x7fae781f3af0 2023-11-26T07:24:45.399 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:45.398+0000 7fae6d7fa640 1 -- 172.21.15.143:0/3871263992 <== mon.0 v2:172.21.15.143:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v103) v1 ==== 80+0+0 (secure 0 0 0) 0x7fae68022020 con 0x7fae781f3af0 2023-11-26T07:24:45.399 INFO:teuthology.orchestra.run.smithi143.stderr:listed 0 entries 2023-11-26T07:24:45.401 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:45.398+0000 7fae7f1c4640 1 -- 172.21.15.143:0/3871263992 >> [v2:172.21.15.188:6836/3401128553,v1:172.21.15.188:6837/3401128553] conn(0x7fae64066740 msgr2=0x7fae64068c00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:24:45.401 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:45.398+0000 7fae7f1c4640 1 --2- 172.21.15.143:0/3871263992 >> [v2:172.21.15.188:6836/3401128553,v1:172.21.15.188:6837/3401128553] conn(0x7fae64066740 0x7fae64068c00 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7fae7813d9f0 tx=0x7fae74009040 comp rx=0 tx=0).stop 2023-11-26T07:24:45.401 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:45.398+0000 7fae7f1c4640 1 -- 172.21.15.143:0/3871263992 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fae781f3af0 msgr2=0x7fae781f2300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:24:45.402 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:45.398+0000 7fae7f1c4640 1 --2- 172.21.15.143:0/3871263992 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fae781f3af0 0x7fae781f2300 secure :-1 s=READY pgs=1392 cs=0 l=1 rev1=1 crypto rx=0x7fae680119d0 tx=0x7fae68011e90 comp rx=0 tx=0).stop 2023-11-26T07:24:45.402 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:45.402+0000 7fae7f1c4640 1 -- 172.21.15.143:0/3871263992 shutdown_connections 2023-11-26T07:24:45.402 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:45.402+0000 7fae7f1c4640 1 --2- 172.21.15.143:0/3871263992 >> [v2:172.21.15.188:6836/3401128553,v1:172.21.15.188:6837/3401128553] conn(0x7fae64066740 0x7fae64068c00 unknown :-1 s=CLOSED pgs=160 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:24:45.402 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:45.402+0000 7fae7f1c4640 1 --2- 172.21.15.143:0/3871263992 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fae781f3360 0x7fae7815d6e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:24:45.402 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:45.402+0000 7fae7f1c4640 1 --2- 172.21.15.143:0/3871263992 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fae781f3af0 0x7fae781f2300 unknown :-1 s=CLOSED pgs=1392 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:24:45.402 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:45.402+0000 7fae7f1c4640 1 --2- 172.21.15.143:0/3871263992 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fae60004d10 0x7fae781f0570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:24:45.402 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:45.402+0000 7fae7f1c4640 1 -- 172.21.15.143:0/3871263992 >> 172.21.15.143:0/3871263992 conn(0x7fae7806a890 msgr2=0x7fae78159f90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:24:45.402 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:45.402+0000 7fae7f1c4640 1 -- 172.21.15.143:0/3871263992 shutdown_connections 2023-11-26T07:24:45.403 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:45.402+0000 7fae7f1c4640 1 -- 172.21.15.143:0/3871263992 wait complete. 2023-11-26T07:24:45.420 INFO:tasks.cephfs.fuse_mount:Running fusermount -u on ubuntu@smithi143.front.sepia.ceph.com... 2023-11-26T07:24:45.420 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-26T07:24:45.421 DEBUG:teuthology.orchestra.run.smithi143:> sudo fusermount -u /home/ubuntu/cephtest/mnt.0 2023-11-26T07:24:45.439 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi143.stderr:ceph-fuse[59135]: fuse finished with error 0 and tester_r 0 2023-11-26T07:24:45.444 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi143.front.sepia.ceph.com 2023-11-26T07:24:45.444 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-26T07:24:45.444 DEBUG:teuthology.orchestra.run.smithi143:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2023-11-26T07:24:45.612 DEBUG:teuthology.orchestra.run.smithi188:> set -ex 2023-11-26T07:24:45.612 DEBUG:teuthology.orchestra.run.smithi188:> dd if=/proc/self/mounts of=/dev/stdout 2023-11-26T07:24:45.622 DEBUG:teuthology.orchestra.run.smithi143:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-11-26T07:24:45.795 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:45.794+0000 7f3e1c3aa640 1 Processor -- start 2023-11-26T07:24:45.795 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:45.794+0000 7f3e1c3aa640 1 -- start start 2023-11-26T07:24:45.796 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:45.794+0000 7f3e1c3aa640 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7f3e1415d6e0 0x7f3e140d0df0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:24:45.796 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:45.794+0000 7f3e1c3aa640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f3e140b2160 0x7f3e140d14b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:24:45.796 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:45.794+0000 7f3e1c3aa640 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7f3e140d19f0 0x7f3e140cfbf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:24:45.796 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:45.794+0000 7f3e1c3aa640 1 -- --> v2:172.21.15.143:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3e1415dac0 con 0x7f3e140d19f0 2023-11-26T07:24:45.797 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:45.794+0000 7f3e1c3aa640 1 -- --> v2:172.21.15.188:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3e140d1e80 con 0x7f3e1415d6e0 2023-11-26T07:24:45.797 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:45.794+0000 7f3e1c3aa640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3e140cf0e0 con 0x7f3e140b2160 2023-11-26T07:24:45.797 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:45.794+0000 7f3e1a947640 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7f3e140d19f0 0x7f3e140cfbf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:24:45.797 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:45.794+0000 7f3e1a947640 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7f3e140d19f0 0x7f3e140cfbf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:58534/0 (socket says 172.21.15.143:58534) 2023-11-26T07:24:45.797 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:45.794+0000 7f3e1a947640 1 -- 172.21.15.143:0/2483212724 learned_addr learned my addr 172.21.15.143:0/2483212724 (peer_addr_for_me v2:172.21.15.143:0/0) 2023-11-26T07:24:45.797 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:45.794+0000 7f3e19945640 1 --2- 172.21.15.143:0/2483212724 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f3e140b2160 0x7f3e140d14b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:24:45.797 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:45.794+0000 7f3e1a146640 1 --2- 172.21.15.143:0/2483212724 >> v2:172.21.15.188:3300/0 conn(0x7f3e1415d6e0 0x7f3e140d0df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:24:45.798 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:45.794+0000 7f3e1a947640 1 -- 172.21.15.143:0/2483212724 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f3e140b2160 msgr2=0x7f3e140d14b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-26T07:24:45.798 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:45.794+0000 7f3e1a947640 1 --2- 172.21.15.143:0/2483212724 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f3e140b2160 0x7f3e140d14b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:24:45.798 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:45.794+0000 7f3e1a947640 1 -- 172.21.15.143:0/2483212724 >> v2:172.21.15.188:3300/0 conn(0x7f3e1415d6e0 msgr2=0x7f3e140d0df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-26T07:24:45.798 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:45.794+0000 7f3e1a947640 1 --2- 172.21.15.143:0/2483212724 >> v2:172.21.15.188:3300/0 conn(0x7f3e1415d6e0 0x7f3e140d0df0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:24:45.798 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:45.794+0000 7f3e1a947640 1 -- 172.21.15.143:0/2483212724 --> v2:172.21.15.143:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3e140ceb20 con 0x7f3e140d19f0 2023-11-26T07:24:45.798 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:45.794+0000 7f3e19945640 1 --2- 172.21.15.143:0/2483212724 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f3e140b2160 0x7f3e140d14b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-26T07:24:45.799 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:45.798+0000 7f3e1a947640 1 --2- 172.21.15.143:0/2483212724 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f3e140d19f0 0x7f3e140cfbf0 secure :-1 s=READY pgs=1393 cs=0 l=1 rev1=1 crypto rx=0x7f3e140b5db0 tx=0x7f3e0c01c710 comp rx=0 tx=0).ready entity=mon.0 client_cookie=195e25ba828843a server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:24:45.800 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:45.798+0000 7f3e19144640 1 -- 172.21.15.143:0/2483212724 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3e0c0a6490 con 0x7f3e140d19f0 2023-11-26T07:24:45.800 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:45.798+0000 7f3e19144640 1 -- 172.21.15.143:0/2483212724 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3e0c0a65f0 con 0x7f3e140d19f0 2023-11-26T07:24:45.800 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:45.798+0000 7f3e19144640 1 -- 172.21.15.143:0/2483212724 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3e0c0a6980 con 0x7f3e140d19f0 2023-11-26T07:24:45.801 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:45.798+0000 7f3e1c3aa640 1 -- 172.21.15.143:0/2483212724 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f3e140d19f0 msgr2=0x7f3e140cfbf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:24:45.801 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:45.798+0000 7f3e1c3aa640 1 --2- 172.21.15.143:0/2483212724 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f3e140d19f0 0x7f3e140cfbf0 secure :-1 s=READY pgs=1393 cs=0 l=1 rev1=1 crypto rx=0x7f3e140b5db0 tx=0x7f3e0c01c710 comp rx=0 tx=0).stop 2023-11-26T07:24:45.801 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:45.798+0000 7f3e1c3aa640 1 -- 172.21.15.143:0/2483212724 shutdown_connections 2023-11-26T07:24:45.801 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:45.798+0000 7f3e1c3aa640 1 --2- 172.21.15.143:0/2483212724 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f3e140d19f0 0x7f3e140cfbf0 unknown :-1 s=CLOSED pgs=1393 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:24:45.801 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:45.798+0000 7f3e1c3aa640 1 --2- 172.21.15.143:0/2483212724 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f3e140b2160 0x7f3e140d14b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:24:45.801 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:45.798+0000 7f3e1c3aa640 1 --2- 172.21.15.143:0/2483212724 >> v2:172.21.15.188:3300/0 conn(0x7f3e1415d6e0 0x7f3e140d0df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:24:45.801 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:45.798+0000 7f3e1c3aa640 1 -- 172.21.15.143:0/2483212724 >> 172.21.15.143:0/2483212724 conn(0x7f3e1409c550 msgr2=0x7f3e140d3220 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:24:45.801 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:45.798+0000 7f3e1c3aa640 1 -- 172.21.15.143:0/2483212724 shutdown_connections 2023-11-26T07:24:45.801 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:45.798+0000 7f3e1c3aa640 1 -- 172.21.15.143:0/2483212724 wait complete. 2023-11-26T07:24:45.802 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:45.802+0000 7f3e1c3aa640 1 Processor -- start 2023-11-26T07:24:45.802 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:45.802+0000 7f3e1c3aa640 1 -- start start 2023-11-26T07:24:45.804 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:45.802+0000 7f3e1c3aa640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f3e140b2160 0x7f3e140c8e90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:24:45.804 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:45.802+0000 7f3e1c3aa640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f3e140d19f0 0x7f3e140ca840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:24:45.804 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:45.802+0000 7f3e1c3aa640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f3e1415d6e0 0x7f3e140b7f50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:24:45.804 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:45.802+0000 7f3e1c3aa640 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3e140c93d0 con 0x7f3e140b2160 2023-11-26T07:24:45.804 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:45.802+0000 7f3e1c3aa640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3e140b29a0 con 0x7f3e1415d6e0 2023-11-26T07:24:45.805 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:45.802+0000 7f3e1c3aa640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3e140b2c80 con 0x7f3e140d19f0 2023-11-26T07:24:45.805 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:45.802+0000 7f3e1a146640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f3e140b2160 0x7f3e140c8e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:24:45.805 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:45.802+0000 7f3e1a146640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f3e140b2160 0x7f3e140c8e90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:58548/0 (socket says 172.21.15.143:58548) 2023-11-26T07:24:45.805 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:45.802+0000 7f3e1a146640 1 -- 172.21.15.143:0/231045333 learned_addr learned my addr 172.21.15.143:0/231045333 (peer_addr_for_me v2:172.21.15.143:0/0) 2023-11-26T07:24:45.805 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:45.802+0000 7f3e19945640 1 --2- 172.21.15.143:0/231045333 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f3e140d19f0 0x7f3e140ca840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:24:45.805 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:45.802+0000 7f3e1a947640 1 --2- 172.21.15.143:0/231045333 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f3e1415d6e0 0x7f3e140b7f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:24:45.805 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:45.802+0000 7f3e1a146640 1 -- 172.21.15.143:0/231045333 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f3e140d19f0 msgr2=0x7f3e140ca840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:24:45.805 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:45.802+0000 7f3e1a146640 1 --2- 172.21.15.143:0/231045333 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f3e140d19f0 0x7f3e140ca840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:24:45.805 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:45.802+0000 7f3e1a146640 1 -- 172.21.15.143:0/231045333 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f3e1415d6e0 msgr2=0x7f3e140b7f50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:24:45.805 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:45.802+0000 7f3e1a146640 1 --2- 172.21.15.143:0/231045333 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f3e1415d6e0 0x7f3e140b7f50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:24:45.805 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:45.802+0000 7f3e1a146640 1 -- 172.21.15.143:0/231045333 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3e0c009b30 con 0x7f3e140b2160 2023-11-26T07:24:45.805 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:45.802+0000 7f3e1a146640 1 --2- 172.21.15.143:0/231045333 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f3e140b2160 0x7f3e140c8e90 secure :-1 s=READY pgs=1394 cs=0 l=1 rev1=1 crypto rx=0x7f3e1000d5f0 tx=0x7f3e1000dab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:24:45.806 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:45.802+0000 7f3e0affd640 1 -- 172.21.15.143:0/231045333 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3e100125f0 con 0x7f3e140b2160 2023-11-26T07:24:45.806 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:45.802+0000 7f3e0affd640 1 -- 172.21.15.143:0/231045333 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3e10012750 con 0x7f3e140b2160 2023-11-26T07:24:45.806 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:45.802+0000 7f3e1c3aa640 1 -- 172.21.15.143:0/231045333 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3e140b2f60 con 0x7f3e140b2160 2023-11-26T07:24:45.807 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:45.802+0000 7f3e0affd640 1 -- 172.21.15.143:0/231045333 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3e10012ae0 con 0x7f3e140b2160 2023-11-26T07:24:45.807 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:45.802+0000 7f3e1c3aa640 1 -- 172.21.15.143:0/231045333 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3e140b3520 con 0x7f3e140b2160 2023-11-26T07:24:45.807 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:45.806+0000 7f3e1c3aa640 1 -- 172.21.15.143:0/231045333 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3e14067680 con 0x7f3e140b2160 2023-11-26T07:24:45.807 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:45.806+0000 7f3e0affd640 1 -- 172.21.15.143:0/231045333 <== mon.0 v2:172.21.15.143:3300/0 4 ==== mgrmap(e 18) v1 ==== 86107+0+0 (secure 0 0 0) 0x7f3e10012c40 con 0x7f3e140b2160 2023-11-26T07:24:45.807 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:45.806+0000 7f3e0affd640 1 --2- 172.21.15.143:0/231045333 >> [v2:172.21.15.188:6836/3401128553,v1:172.21.15.188:6837/3401128553] conn(0x7f3df8066740 0x7f3df8068c00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:24:45.808 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:45.806+0000 7f3e0affd640 1 -- 172.21.15.143:0/231045333 <== mon.0 v2:172.21.15.143:3300/0 5 ==== osd_map(103..103 src has 1..103) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f3e10084610 con 0x7f3e140b2160 2023-11-26T07:24:45.808 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:45.806+0000 7f3e19945640 1 --2- 172.21.15.143:0/231045333 >> [v2:172.21.15.188:6836/3401128553,v1:172.21.15.188:6837/3401128553] conn(0x7f3df8066740 0x7f3df8068c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:24:45.809 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:45.806+0000 7f3e19945640 1 --2- 172.21.15.143:0/231045333 >> [v2:172.21.15.188:6836/3401128553,v1:172.21.15.188:6837/3401128553] conn(0x7f3df8066740 0x7f3df8068c00 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7f3e1413ee90 tx=0x7f3e04008040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:24:45.811 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:45.810+0000 7f3e0affd640 1 -- 172.21.15.143:0/231045333 <== mon.0 v2:172.21.15.143:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172649 (secure 0 0 0) 0x7f3e10051840 con 0x7f3e140b2160 2023-11-26T07:24:45.962 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:45.962+0000 7f3e1c3aa640 1 -- 172.21.15.143:0/231045333 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f3e140b59c0 con 0x7f3e140b2160 2023-11-26T07:24:45.963 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:45.962+0000 7f3e0affd640 1 -- 172.21.15.143:0/231045333 <== mon.0 v2:172.21.15.143:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v103) v1 ==== 74+0+16563 (secure 0 0 0) 0x7f3e10054e60 con 0x7f3e140b2160 2023-11-26T07:24:45.964 INFO:teuthology.orchestra.run.smithi143.stdout: 2023-11-26T07:24:45.964 INFO:teuthology.orchestra.run.smithi143.stdout:{"epoch":103,"fsid":"f5033060-2469-4a07-95fe-2b76f5404bc1","created":"2023-11-26T07:17:28.613663+0000","modified":"2023-11-26T07:24:09.510299+0000","last_up_change":"2023-11-26T07:17:41.157813+0000","last_in_change":"2023-11-26T07:17:31.181658+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":17,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-11-26T07:17:41.758597+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":16,"pool_name":"cephfs_metadata","create_time":"2023-11-26T07:24:06.278132+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"102","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":17,"pool_name":"cephfs_data","create_time":"2023-11-26T07:24:07.294850+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"103","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"ebbee2c6-023e-44b9-9f1d-6f7bfd309730","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":101,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6817","nonce":639807369},{"type":"v1","addr":"172.21.15.143:6819","nonce":639807369}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6821","nonce":639807369},{"type":"v1","addr":"172.21.15.143:6823","nonce":639807369}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6830","nonce":639807369},{"type":"v1","addr":"172.21.15.143:6831","nonce":639807369}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6825","nonce":639807369},{"type":"v1","addr":"172.21.15.143:6828","nonce":639807369}]},"public_addr":"172.21.15.143:6819/639807369","cluster_addr":"172.21.15.143:6823/639807369","heartbeat_back_addr":"172.21.15.143:6831/639807369","heartbeat_front_addr":"172.21.15.143:6828/639807369","state":["exists","up"]},{"osd":1,"uuid":"fec4b8be-7122-49e9-9a19-5850a0700001","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":101,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6816","nonce":1144942530},{"type":"v1","addr":"172.21.15.143:6818","nonce":1144942530}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6820","nonce":1144942530},{"type":"v1","addr":"172.21.15.143:6822","nonce":1144942530}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6827","nonce":1144942530},{"type":"v1","addr":"172.21.15.143:6829","nonce":1144942530}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6824","nonce":1144942530},{"type":"v1","addr":"172.21.15.143:6826","nonce":1144942530}]},"public_addr":"172.21.15.143:6818/1144942530","cluster_addr":"172.21.15.143:6822/1144942530","heartbeat_back_addr":"172.21.15.143:6829/1144942530","heartbeat_front_addr":"172.21.15.143:6826/1144942530","state":["exists","up"]},{"osd":2,"uuid":"2888af5b-f933-46ca-8f95-6dd494ff43f6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":101,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6808","nonce":3909095186},{"type":"v1","addr":"172.21.15.143:6809","nonce":3909095186}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6810","nonce":3909095186},{"type":"v1","addr":"172.21.15.143:6811","nonce":3909095186}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6814","nonce":3909095186},{"type":"v1","addr":"172.21.15.143:6815","nonce":3909095186}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6812","nonce":3909095186},{"type":"v1","addr":"172.21.15.143:6813","nonce":3909095186}]},"public_addr":"172.21.15.143:6809/3909095186","cluster_addr":"172.21.15.143:6811/3909095186","heartbeat_back_addr":"172.21.15.143:6815/3909095186","heartbeat_front_addr":"172.21.15.143:6813/3909095186","state":["exists","up"]},{"osd":3,"uuid":"d1f52435-7bc1-4081-9550-9cd1ba55f549","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":101,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6800","nonce":3854210991},{"type":"v1","addr":"172.21.15.143:6801","nonce":3854210991}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6802","nonce":3854210991},{"type":"v1","addr":"172.21.15.143:6803","nonce":3854210991}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6806","nonce":3854210991},{"type":"v1","addr":"172.21.15.143:6807","nonce":3854210991}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6804","nonce":3854210991},{"type":"v1","addr":"172.21.15.143:6805","nonce":3854210991}]},"public_addr":"172.21.15.143:6801/3854210991","cluster_addr":"172.21.15.143:6803/3854210991","heartbeat_back_addr":"172.21.15.143:6807/3854210991","heartbeat_front_addr":"172.21.15.143:6805/3854210991","state":["exists","up"]},{"osd":4,"uuid":"e2ae8b6e-6e25-49b2-bcbf-aacce68f06a3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":101,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6800","nonce":3924606902},{"type":"v1","addr":"172.21.15.188:6801","nonce":3924606902}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6802","nonce":3924606902},{"type":"v1","addr":"172.21.15.188:6803","nonce":3924606902}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6806","nonce":3924606902},{"type":"v1","addr":"172.21.15.188:6807","nonce":3924606902}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6804","nonce":3924606902},{"type":"v1","addr":"172.21.15.188:6805","nonce":3924606902}]},"public_addr":"172.21.15.188:6801/3924606902","cluster_addr":"172.21.15.188:6803/3924606902","heartbeat_back_addr":"172.21.15.188:6807/3924606902","heartbeat_front_addr":"172.21.15.188:6805/3924606902","state":["exists","up"]},{"osd":5,"uuid":"3cec8a32-7698-4c36-8e89-e4ba11934f33","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":101,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6813","nonce":186841544},{"type":"v1","addr":"172.21.15.188:6815","nonce":186841544}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6817","nonce":186841544},{"type":"v1","addr":"172.21.15.188:6818","nonce":186841544}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6822","nonce":186841544},{"type":"v1","addr":"172.21.15.188:6823","nonce":186841544}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6820","nonce":186841544},{"type":"v1","addr":"172.21.15.188:6821","nonce":186841544}]},"public_addr":"172.21.15.188:6815/186841544","cluster_addr":"172.21.15.188:6818/186841544","heartbeat_back_addr":"172.21.15.188:6823/186841544","heartbeat_front_addr":"172.21.15.188:6821/186841544","state":["exists","up"]},{"osd":6,"uuid":"58a65d5d-e090-473a-81b4-ad38e85029f9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":101,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6808","nonce":2153499405},{"type":"v1","addr":"172.21.15.188:6809","nonce":2153499405}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6810","nonce":2153499405},{"type":"v1","addr":"172.21.15.188:6811","nonce":2153499405}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6816","nonce":2153499405},{"type":"v1","addr":"172.21.15.188:6819","nonce":2153499405}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6812","nonce":2153499405},{"type":"v1","addr":"172.21.15.188:6814","nonce":2153499405}]},"public_addr":"172.21.15.188:6809/2153499405","cluster_addr":"172.21.15.188:6811/2153499405","heartbeat_back_addr":"172.21.15.188:6819/2153499405","heartbeat_front_addr":"172.21.15.188:6814/2153499405","state":["exists","up"]},{"osd":7,"uuid":"ea5b7f70-3554-4cf8-ac30-f7c59b80678d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":101,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6824","nonce":2722096729},{"type":"v1","addr":"172.21.15.188:6825","nonce":2722096729}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6826","nonce":2722096729},{"type":"v1","addr":"172.21.15.188:6827","nonce":2722096729}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6830","nonce":2722096729},{"type":"v1","addr":"172.21.15.188:6831","nonce":2722096729}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6828","nonce":2722096729},{"type":"v1","addr":"172.21.15.188:6829","nonce":2722096729}]},"public_addr":"172.21.15.188:6825/2722096729","cluster_addr":"172.21.15.188:6827/2722096729","heartbeat_back_addr":"172.21.15.188:6831/2722096729","heartbeat_front_addr":"172.21.15.188:6829/2722096729","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-26T07:17:36.995032+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-26T07:17:36.932266+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-26T07:17:36.829304+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-26T07:17:36.804297+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-26T07:17:36.667892+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-26T07:17:36.964868+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-26T07:17:36.827332+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-26T07:17:36.893179+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-11-26T07:24:45.966 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:45.966+0000 7f3e1c3aa640 1 -- 172.21.15.143:0/231045333 >> [v2:172.21.15.188:6836/3401128553,v1:172.21.15.188:6837/3401128553] conn(0x7f3df8066740 msgr2=0x7f3df8068c00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:24:45.966 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:45.966+0000 7f3e1c3aa640 1 --2- 172.21.15.143:0/231045333 >> [v2:172.21.15.188:6836/3401128553,v1:172.21.15.188:6837/3401128553] conn(0x7f3df8066740 0x7f3df8068c00 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7f3e1413ee90 tx=0x7f3e04008040 comp rx=0 tx=0).stop 2023-11-26T07:24:45.966 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:45.966+0000 7f3e1c3aa640 1 -- 172.21.15.143:0/231045333 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f3e140b2160 msgr2=0x7f3e140c8e90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:24:45.966 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:45.966+0000 7f3e1c3aa640 1 --2- 172.21.15.143:0/231045333 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f3e140b2160 0x7f3e140c8e90 secure :-1 s=READY pgs=1394 cs=0 l=1 rev1=1 crypto rx=0x7f3e1000d5f0 tx=0x7f3e1000dab0 comp rx=0 tx=0).stop 2023-11-26T07:24:45.966 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:45.966+0000 7f3e1c3aa640 1 -- 172.21.15.143:0/231045333 shutdown_connections 2023-11-26T07:24:45.966 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:45.966+0000 7f3e1c3aa640 1 --2- 172.21.15.143:0/231045333 >> [v2:172.21.15.188:6836/3401128553,v1:172.21.15.188:6837/3401128553] conn(0x7f3df8066740 0x7f3df8068c00 unknown :-1 s=CLOSED pgs=161 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:24:45.967 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:45.966+0000 7f3e1c3aa640 1 --2- 172.21.15.143:0/231045333 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f3e1415d6e0 0x7f3e140b7f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:24:45.967 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:45.966+0000 7f3e1c3aa640 1 --2- 172.21.15.143:0/231045333 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f3e140d19f0 0x7f3e140ca840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:24:45.967 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:45.966+0000 7f3e1c3aa640 1 --2- 172.21.15.143:0/231045333 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f3e140b2160 0x7f3e140c8e90 unknown :-1 s=CLOSED pgs=1394 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:24:45.967 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:45.966+0000 7f3e1c3aa640 1 -- 172.21.15.143:0/231045333 >> 172.21.15.143:0/231045333 conn(0x7f3e1409c550 msgr2=0x7f3e140d3220 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:24:45.967 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:45.966+0000 7f3e1c3aa640 1 -- 172.21.15.143:0/231045333 shutdown_connections 2023-11-26T07:24:45.967 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:45.966+0000 7f3e1c3aa640 1 -- 172.21.15.143:0/231045333 wait complete. 2023-11-26T07:24:45.985 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-11-26T07:17:41.758597+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 16, 'pool_name': 'cephfs_metadata', 'create_time': '2023-11-26T07:24:06.278132+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '102', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 1.75, 'score_stable': 1.75, 'optimal_score': 1, 'raw_score_acting': 1.75, 'raw_score_stable': 1.75, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 17, 'pool_name': 'cephfs_data', 'create_time': '2023-11-26T07:24:07.294850+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '103', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.75, 'score_stable': 1.75, 'optimal_score': 1, 'raw_score_acting': 1.75, 'raw_score_stable': 1.75, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2023-11-26T07:24:45.986 DEBUG:teuthology.orchestra.run.smithi143:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-11-26T07:24:46.126 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:46.126+0000 7f0039365640 1 Processor -- start 2023-11-26T07:24:46.126 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:46.126+0000 7f0039365640 1 -- start start 2023-11-26T07:24:46.128 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:46.126+0000 7f0039365640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f003406f000 0x7f003406d030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:24:46.128 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:46.126+0000 7f0039365640 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7f003406f3e0 0x7f003406b5a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:24:46.128 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:46.126+0000 7f0039365640 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7f0034099a70 0x7f003406bae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:24:46.128 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:46.126+0000 7f0039365640 1 -- --> v2:172.21.15.143:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f00340cf260 con 0x7f0034099a70 2023-11-26T07:24:46.128 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:46.126+0000 7f0039365640 1 -- --> v2:172.21.15.188:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f003406d570 con 0x7f003406f3e0 2023-11-26T07:24:46.129 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:46.126+0000 7f0039365640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f003409a010 con 0x7f003406f000 2023-11-26T07:24:46.129 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:46.126+0000 7f00337fe640 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7f0034099a70 0x7f003406bae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:24:46.129 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:46.126+0000 7f00337fe640 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7f0034099a70 0x7f003406bae0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:58554/0 (socket says 172.21.15.143:58554) 2023-11-26T07:24:46.129 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:46.126+0000 7f00337fe640 1 -- 172.21.15.143:0/3987317034 learned_addr learned my addr 172.21.15.143:0/3987317034 (peer_addr_for_me v2:172.21.15.143:0/0) 2023-11-26T07:24:46.129 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:46.126+0000 7f0032ffd640 1 --2- 172.21.15.143:0/3987317034 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f003406f000 0x7f003406d030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:24:46.129 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:46.126+0000 7f002a7fc640 1 --2- 172.21.15.143:0/3987317034 >> v2:172.21.15.188:3300/0 conn(0x7f003406f3e0 0x7f003406b5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:24:46.129 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:46.126+0000 7f00337fe640 1 -- 172.21.15.143:0/3987317034 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f003406f000 msgr2=0x7f003406d030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-26T07:24:46.129 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:46.126+0000 7f00337fe640 1 --2- 172.21.15.143:0/3987317034 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f003406f000 0x7f003406d030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:24:46.129 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:46.126+0000 7f00337fe640 1 -- 172.21.15.143:0/3987317034 >> v2:172.21.15.188:3300/0 conn(0x7f003406f3e0 msgr2=0x7f003406b5a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-26T07:24:46.129 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:46.126+0000 7f00337fe640 1 --2- 172.21.15.143:0/3987317034 >> v2:172.21.15.188:3300/0 conn(0x7f003406f3e0 0x7f003406b5a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:24:46.129 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:46.126+0000 7f00337fe640 1 -- 172.21.15.143:0/3987317034 --> v2:172.21.15.143:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f003409a2f0 con 0x7f0034099a70 2023-11-26T07:24:46.129 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:46.126+0000 7f0032ffd640 1 --2- 172.21.15.143:0/3987317034 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f003406f000 0x7f003406d030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-26T07:24:46.131 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:46.130+0000 7f00337fe640 1 --2- 172.21.15.143:0/3987317034 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f0034099a70 0x7f003406bae0 secure :-1 s=READY pgs=1395 cs=0 l=1 rev1=1 crypto rx=0x7f0034074a80 tx=0x7f002c01c9d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=79d5591b6447a05 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:24:46.131 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:46.130+0000 7f00327fc640 1 -- 172.21.15.143:0/3987317034 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f002c0a5650 con 0x7f0034099a70 2023-11-26T07:24:46.131 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:46.130+0000 7f00327fc640 1 -- 172.21.15.143:0/3987317034 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f002c0a57b0 con 0x7f0034099a70 2023-11-26T07:24:46.132 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:46.130+0000 7f00327fc640 1 -- 172.21.15.143:0/3987317034 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f002c0a5b40 con 0x7f0034099a70 2023-11-26T07:24:46.132 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:46.130+0000 7f0039365640 1 -- 172.21.15.143:0/3987317034 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f0034099a70 msgr2=0x7f003406bae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:24:46.132 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:46.130+0000 7f0039365640 1 --2- 172.21.15.143:0/3987317034 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f0034099a70 0x7f003406bae0 secure :-1 s=READY pgs=1395 cs=0 l=1 rev1=1 crypto rx=0x7f0034074a80 tx=0x7f002c01c9d0 comp rx=0 tx=0).stop 2023-11-26T07:24:46.133 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:46.130+0000 7f0039365640 1 -- 172.21.15.143:0/3987317034 shutdown_connections 2023-11-26T07:24:46.133 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:46.130+0000 7f0039365640 1 --2- 172.21.15.143:0/3987317034 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f0034099a70 0x7f003406bae0 unknown :-1 s=CLOSED pgs=1395 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:24:46.133 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:46.130+0000 7f0039365640 1 --2- 172.21.15.143:0/3987317034 >> v2:172.21.15.188:3300/0 conn(0x7f003406f3e0 0x7f003406b5a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:24:46.133 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:46.130+0000 7f0039365640 1 --2- 172.21.15.143:0/3987317034 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f003406f000 0x7f003406d030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:24:46.133 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:46.130+0000 7f0039365640 1 -- 172.21.15.143:0/3987317034 >> 172.21.15.143:0/3987317034 conn(0x7f00340b2d10 msgr2=0x7f00340b3120 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:24:46.133 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:46.130+0000 7f0039365640 1 -- 172.21.15.143:0/3987317034 shutdown_connections 2023-11-26T07:24:46.133 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:46.130+0000 7f0039365640 1 -- 172.21.15.143:0/3987317034 wait complete. 2023-11-26T07:24:46.133 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:46.134+0000 7f0039365640 1 Processor -- start 2023-11-26T07:24:46.134 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:46.134+0000 7f0039365640 1 -- start start 2023-11-26T07:24:46.136 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:46.134+0000 7f0039365640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f003406f000 0x7f00340c3610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:24:46.136 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:46.134+0000 7f0039365640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f003406f3e0 0x7f0034153a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:24:46.136 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:46.134+0000 7f0039365640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f0034099a70 0x7f0034154690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:24:46.137 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:46.134+0000 7f0039365640 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f003414a870 con 0x7f003406f000 2023-11-26T07:24:46.137 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:46.134+0000 7f0039365640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f00340b57e0 con 0x7f003406f3e0 2023-11-26T07:24:46.137 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:46.134+0000 7f0039365640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0034153f60 con 0x7f0034099a70 2023-11-26T07:24:46.137 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:46.134+0000 7f0032ffd640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f003406f000 0x7f00340c3610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:24:46.137 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:46.134+0000 7f0032ffd640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f003406f000 0x7f00340c3610 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:58566/0 (socket says 172.21.15.143:58566) 2023-11-26T07:24:46.137 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:46.134+0000 7f0032ffd640 1 -- 172.21.15.143:0/171002208 learned_addr learned my addr 172.21.15.143:0/171002208 (peer_addr_for_me v2:172.21.15.143:0/0) 2023-11-26T07:24:46.137 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:46.134+0000 7f002a7fc640 1 --2- 172.21.15.143:0/171002208 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f003406f3e0 0x7f0034153a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:24:46.137 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:46.134+0000 7f00337fe640 1 --2- 172.21.15.143:0/171002208 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f0034099a70 0x7f0034154690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:24:46.137 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:46.134+0000 7f0032ffd640 1 -- 172.21.15.143:0/171002208 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f0034099a70 msgr2=0x7f0034154690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:24:46.137 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:46.134+0000 7f0032ffd640 1 --2- 172.21.15.143:0/171002208 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f0034099a70 0x7f0034154690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:24:46.137 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:46.134+0000 7f0032ffd640 1 -- 172.21.15.143:0/171002208 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f003406f3e0 msgr2=0x7f0034153a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:24:46.137 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:46.134+0000 7f0032ffd640 1 --2- 172.21.15.143:0/171002208 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f003406f3e0 0x7f0034153a20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:24:46.137 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:46.134+0000 7f0032ffd640 1 -- 172.21.15.143:0/171002208 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f002c009b30 con 0x7f003406f000 2023-11-26T07:24:46.138 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:46.134+0000 7f002a7fc640 1 --2- 172.21.15.143:0/171002208 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f003406f3e0 0x7f0034153a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-26T07:24:46.138 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:46.134+0000 7f00337fe640 1 --2- 172.21.15.143:0/171002208 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f0034099a70 0x7f0034154690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-26T07:24:46.138 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:46.134+0000 7f0032ffd640 1 --2- 172.21.15.143:0/171002208 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f003406f000 0x7f00340c3610 secure :-1 s=READY pgs=1396 cs=0 l=1 rev1=1 crypto rx=0x7f002400d800 tx=0x7f002400dcc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:24:46.138 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:46.134+0000 7f002bfff640 1 -- 172.21.15.143:0/171002208 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0024004030 con 0x7f003406f000 2023-11-26T07:24:46.138 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:46.134+0000 7f002bfff640 1 -- 172.21.15.143:0/171002208 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0024004190 con 0x7f003406f000 2023-11-26T07:24:46.138 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:46.134+0000 7f0039365640 1 -- 172.21.15.143:0/171002208 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f003415aed0 con 0x7f003406f000 2023-11-26T07:24:46.138 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:46.134+0000 7f002bfff640 1 -- 172.21.15.143:0/171002208 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0024004520 con 0x7f003406f000 2023-11-26T07:24:46.138 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:46.134+0000 7f0039365640 1 -- 172.21.15.143:0/171002208 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f003414ada0 con 0x7f003406f000 2023-11-26T07:24:46.139 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:46.138+0000 7f0039365640 1 -- 172.21.15.143:0/171002208 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f00340b5950 con 0x7f003406f000 2023-11-26T07:24:46.139 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:46.138+0000 7f002bfff640 1 -- 172.21.15.143:0/171002208 <== mon.0 v2:172.21.15.143:3300/0 4 ==== mgrmap(e 18) v1 ==== 86107+0+0 (secure 0 0 0) 0x7f00240096a0 con 0x7f003406f000 2023-11-26T07:24:46.139 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:46.138+0000 7f002bfff640 1 --2- 172.21.15.143:0/171002208 >> [v2:172.21.15.188:6836/3401128553,v1:172.21.15.188:6837/3401128553] conn(0x7f0014066770 0x7f0014068c30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:24:46.139 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:46.138+0000 7f002bfff640 1 -- 172.21.15.143:0/171002208 <== mon.0 v2:172.21.15.143:3300/0 5 ==== osd_map(103..103 src has 1..103) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f0024084580 con 0x7f003406f000 2023-11-26T07:24:46.139 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:46.138+0000 7f002a7fc640 1 --2- 172.21.15.143:0/171002208 >> [v2:172.21.15.188:6836/3401128553,v1:172.21.15.188:6837/3401128553] conn(0x7f0014066770 0x7f0014068c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:24:46.140 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:46.138+0000 7f002a7fc640 1 --2- 172.21.15.143:0/171002208 >> [v2:172.21.15.188:6836/3401128553,v1:172.21.15.188:6837/3401128553] conn(0x7f0014066770 0x7f0014068c30 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7f003414e760 tx=0x7f001c008040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:24:46.144 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:46.142+0000 7f002bfff640 1 -- 172.21.15.143:0/171002208 <== mon.0 v2:172.21.15.143:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172649 (secure 0 0 0) 0x7f00340b5950 con 0x7f003406f000 2023-11-26T07:24:46.300 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:46.298+0000 7f0039365640 1 -- 172.21.15.143:0/171002208 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f00340b5950 con 0x7f003406f000 2023-11-26T07:24:46.301 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:46.298+0000 7f002bfff640 1 -- 172.21.15.143:0/171002208 <== mon.0 v2:172.21.15.143:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v103) v1 ==== 93+0+10 (secure 0 0 0) 0x7f0024018020 con 0x7f003406f000 2023-11-26T07:24:46.301 INFO:teuthology.orchestra.run.smithi143.stdout:pg_num: 1 2023-11-26T07:24:46.303 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:46.302+0000 7f0039365640 1 -- 172.21.15.143:0/171002208 >> [v2:172.21.15.188:6836/3401128553,v1:172.21.15.188:6837/3401128553] conn(0x7f0014066770 msgr2=0x7f0014068c30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:24:46.303 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:46.302+0000 7f0039365640 1 --2- 172.21.15.143:0/171002208 >> [v2:172.21.15.188:6836/3401128553,v1:172.21.15.188:6837/3401128553] conn(0x7f0014066770 0x7f0014068c30 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7f003414e760 tx=0x7f001c008040 comp rx=0 tx=0).stop 2023-11-26T07:24:46.303 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:46.302+0000 7f0039365640 1 -- 172.21.15.143:0/171002208 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f003406f000 msgr2=0x7f00340c3610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:24:46.303 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:46.302+0000 7f0039365640 1 --2- 172.21.15.143:0/171002208 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f003406f000 0x7f00340c3610 secure :-1 s=READY pgs=1396 cs=0 l=1 rev1=1 crypto rx=0x7f002400d800 tx=0x7f002400dcc0 comp rx=0 tx=0).stop 2023-11-26T07:24:46.303 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:46.302+0000 7f0039365640 1 -- 172.21.15.143:0/171002208 shutdown_connections 2023-11-26T07:24:46.304 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:46.302+0000 7f0039365640 1 --2- 172.21.15.143:0/171002208 >> [v2:172.21.15.188:6836/3401128553,v1:172.21.15.188:6837/3401128553] conn(0x7f0014066770 0x7f0014068c30 unknown :-1 s=CLOSED pgs=162 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:24:46.304 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:46.302+0000 7f0039365640 1 --2- 172.21.15.143:0/171002208 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f0034099a70 0x7f0034154690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:24:46.304 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:46.302+0000 7f0039365640 1 --2- 172.21.15.143:0/171002208 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f003406f3e0 0x7f0034153a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:24:46.304 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:46.302+0000 7f0039365640 1 --2- 172.21.15.143:0/171002208 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f003406f000 0x7f00340c3610 unknown :-1 s=CLOSED pgs=1396 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:24:46.304 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:46.302+0000 7f0039365640 1 -- 172.21.15.143:0/171002208 >> 172.21.15.143:0/171002208 conn(0x7f00340b2d10 msgr2=0x7f00340cb830 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:24:46.304 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:46.302+0000 7f0039365640 1 -- 172.21.15.143:0/171002208 shutdown_connections 2023-11-26T07:24:46.304 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:46.302+0000 7f0039365640 1 -- 172.21.15.143:0/171002208 wait complete. 2023-11-26T07:24:46.322 DEBUG:teuthology.orchestra.run.smithi143:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-11-26T07:24:46.464 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:46.462+0000 7f06d3e5a640 1 Processor -- start 2023-11-26T07:24:46.464 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:46.462+0000 7f06d3e5a640 1 -- start start 2023-11-26T07:24:46.465 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:46.462+0000 7f06d3e5a640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f06cc0b2f80 0x7f06cc071850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:24:46.465 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:46.462+0000 7f06d3e5a640 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7f06cc0ce680 0x7f06cc06cbe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:24:46.465 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:46.466+0000 7f06d3e5a640 1 -- --> v1:172.21.15.188:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f06cc06c750 con 0x7f06cc1596d0 2023-11-26T07:24:46.465 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:46.466+0000 7f06d3e5a640 1 -- --> v2:172.21.15.188:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f06cc0cbe00 con 0x7f06cc0ce680 2023-11-26T07:24:46.465 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:46.466+0000 7f06d3e5a640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f06cc0b3550 con 0x7f06cc0b2f80 2023-11-26T07:24:46.466 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:46.466+0000 7f06d23f7640 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7f06cc0ce680 0x7f06cc06cbe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:24:46.466 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:46.466+0000 7f06d1bf6640 1 --1- >> v1:172.21.15.188:6789/0 conn(0x7f06cc1596d0 0x7f06cc0b2870 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.188:6789/0 says I am v1:172.21.15.143:52396/0 (socket says 172.21.15.143:52396) 2023-11-26T07:24:46.466 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:46.466+0000 7f06d1bf6640 1 -- 172.21.15.143:0/1978464793 learned_addr learned my addr 172.21.15.143:0/1978464793 (peer_addr_for_me v1:172.21.15.143:0/0) 2023-11-26T07:24:46.466 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:46.466+0000 7f06d13f5640 1 --2- 172.21.15.143:0/1978464793 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f06cc0b2f80 0x7f06cc071850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:24:46.467 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:46.466+0000 7f06d23f7640 1 -- 172.21.15.143:0/1978464793 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f06cc0b2f80 msgr2=0x7f06cc071850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-26T07:24:46.467 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:46.466+0000 7f06d23f7640 1 --2- 172.21.15.143:0/1978464793 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f06cc0b2f80 0x7f06cc071850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:24:46.467 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:46.466+0000 7f06d23f7640 1 -- 172.21.15.143:0/1978464793 >> v1:172.21.15.188:6789/0 conn(0x7f06cc1596d0 legacy=0x7f06cc0b2870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:24:46.467 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:46.466+0000 7f06d23f7640 1 -- 172.21.15.143:0/1978464793 --> v2:172.21.15.188:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f06cc071e80 con 0x7f06cc0ce680 2023-11-26T07:24:46.469 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:46.470+0000 7f06d23f7640 1 --2- 172.21.15.143:0/1978464793 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f06cc0ce680 0x7f06cc06cbe0 secure :-1 s=READY pgs=520 cs=0 l=1 rev1=1 crypto rx=0x7f06c400bfa0 tx=0x7f06c401eb50 comp rx=0 tx=0).ready entity=mon.1 client_cookie=7650ae6cfc95f41d server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:24:46.469 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:46.470+0000 7f06d0bf4640 1 -- 172.21.15.143:0/1978464793 <== mon.1 v2:172.21.15.188:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f06c400b2d0 con 0x7f06cc0ce680 2023-11-26T07:24:46.469 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:46.470+0000 7f06d0bf4640 1 -- 172.21.15.143:0/1978464793 <== mon.1 v2:172.21.15.188:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f06c40a7890 con 0x7f06cc0ce680 2023-11-26T07:24:46.469 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:46.470+0000 7f06d0bf4640 1 -- 172.21.15.143:0/1978464793 <== mon.1 v2:172.21.15.188:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f06c40a7c20 con 0x7f06cc0ce680 2023-11-26T07:24:46.470 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:46.470+0000 7f06d3e5a640 1 -- 172.21.15.143:0/1978464793 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f06cc0ce680 msgr2=0x7f06cc06cbe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:24:46.470 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:46.470+0000 7f06d3e5a640 1 --2- 172.21.15.143:0/1978464793 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f06cc0ce680 0x7f06cc06cbe0 secure :-1 s=READY pgs=520 cs=0 l=1 rev1=1 crypto rx=0x7f06c400bfa0 tx=0x7f06c401eb50 comp rx=0 tx=0).stop 2023-11-26T07:24:46.470 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:46.470+0000 7f06d3e5a640 1 -- 172.21.15.143:0/1978464793 shutdown_connections 2023-11-26T07:24:46.471 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:46.470+0000 7f06d3e5a640 1 --2- 172.21.15.143:0/1978464793 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f06cc0ce680 0x7f06cc06cbe0 unknown :-1 s=CLOSED pgs=520 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:24:46.471 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:46.470+0000 7f06d3e5a640 1 --2- 172.21.15.143:0/1978464793 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f06cc0b2f80 0x7f06cc071850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:24:46.471 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:46.470+0000 7f06d3e5a640 1 -- 172.21.15.143:0/1978464793 >> 172.21.15.143:0/1978464793 conn(0x7f06cc06a890 msgr2=0x7f06cc06aca0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:24:46.471 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:46.470+0000 7f06d3e5a640 1 -- 172.21.15.143:0/1978464793 shutdown_connections 2023-11-26T07:24:46.471 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:46.470+0000 7f06d3e5a640 1 -- 172.21.15.143:0/1978464793 wait complete. 2023-11-26T07:24:46.472 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:46.470+0000 7f06d3e5a640 1 Processor -- start 2023-11-26T07:24:46.472 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:46.470+0000 7f06d3e5a640 1 -- start start 2023-11-26T07:24:46.474 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:46.474+0000 7f06d3e5a640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f06cc0b2f80 0x7f06cc0a8540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:24:46.474 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:46.474+0000 7f06d3e5a640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f06cc0ce680 0x7f06cc14b1a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:24:46.474 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:46.474+0000 7f06d3e5a640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f06cc1596d0 0x7f06cc070c10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:24:46.474 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:46.474+0000 7f06d3e5a640 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f06cc14bdb0 con 0x7f06cc0ce680 2023-11-26T07:24:46.474 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:46.474+0000 7f06d3e5a640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f06cc0a8a80 con 0x7f06cc1596d0 2023-11-26T07:24:46.474 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:46.474+0000 7f06d3e5a640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f06cc15b5e0 con 0x7f06cc0b2f80 2023-11-26T07:24:46.474 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:46.474+0000 7f06d13f5640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f06cc0ce680 0x7f06cc14b1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:24:46.474 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:46.474+0000 7f06d13f5640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f06cc0ce680 0x7f06cc14b1a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:58578/0 (socket says 172.21.15.143:58578) 2023-11-26T07:24:46.475 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:46.474+0000 7f06d13f5640 1 -- 172.21.15.143:0/1788129731 learned_addr learned my addr 172.21.15.143:0/1788129731 (peer_addr_for_me v2:172.21.15.143:0/0) 2023-11-26T07:24:46.475 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:46.474+0000 7f06d23f7640 1 --2- 172.21.15.143:0/1788129731 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f06cc1596d0 0x7f06cc070c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:24:46.475 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:46.474+0000 7f06d1bf6640 1 --2- 172.21.15.143:0/1788129731 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f06cc0b2f80 0x7f06cc0a8540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:24:46.475 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:46.474+0000 7f06d13f5640 1 -- 172.21.15.143:0/1788129731 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f06cc0b2f80 msgr2=0x7f06cc0a8540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:24:46.475 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:46.474+0000 7f06d13f5640 1 --2- 172.21.15.143:0/1788129731 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f06cc0b2f80 0x7f06cc0a8540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:24:46.475 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:46.474+0000 7f06d13f5640 1 -- 172.21.15.143:0/1788129731 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f06cc1596d0 msgr2=0x7f06cc070c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:24:46.475 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:46.474+0000 7f06d13f5640 1 --2- 172.21.15.143:0/1788129731 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f06cc1596d0 0x7f06cc070c10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:24:46.475 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:46.474+0000 7f06d13f5640 1 -- 172.21.15.143:0/1788129731 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f06c400bc80 con 0x7f06cc0ce680 2023-11-26T07:24:46.475 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:46.474+0000 7f06d1bf6640 1 --2- 172.21.15.143:0/1788129731 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f06cc0b2f80 0x7f06cc0a8540 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-26T07:24:46.475 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:46.474+0000 7f06d13f5640 1 --2- 172.21.15.143:0/1788129731 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f06cc0ce680 0x7f06cc14b1a0 secure :-1 s=READY pgs=1397 cs=0 l=1 rev1=1 crypto rx=0x7f06bc00e930 tx=0x7f06bc00edf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:24:46.476 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:46.474+0000 7f06c27fc640 1 -- 172.21.15.143:0/1788129731 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f06bc009690 con 0x7f06cc0ce680 2023-11-26T07:24:46.476 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:46.474+0000 7f06c27fc640 1 -- 172.21.15.143:0/1788129731 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f06bc00c9a0 con 0x7f06cc0ce680 2023-11-26T07:24:46.476 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:46.474+0000 7f06d3e5a640 1 -- 172.21.15.143:0/1788129731 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f06cc15ca70 con 0x7f06cc0ce680 2023-11-26T07:24:46.476 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:46.474+0000 7f06c27fc640 1 -- 172.21.15.143:0/1788129731 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f06bc00cd10 con 0x7f06cc0ce680 2023-11-26T07:24:46.476 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:46.474+0000 7f06d3e5a640 1 -- 172.21.15.143:0/1788129731 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f06cc15b040 con 0x7f06cc0ce680 2023-11-26T07:24:46.477 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:46.474+0000 7f06d3e5a640 1 -- 172.21.15.143:0/1788129731 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f06cc15b790 con 0x7f06cc0ce680 2023-11-26T07:24:46.477 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:46.474+0000 7f06c27fc640 1 -- 172.21.15.143:0/1788129731 <== mon.0 v2:172.21.15.143:3300/0 4 ==== mgrmap(e 18) v1 ==== 86107+0+0 (secure 0 0 0) 0x7f06bc013470 con 0x7f06cc0ce680 2023-11-26T07:24:46.478 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:46.478+0000 7f06c27fc640 1 --2- 172.21.15.143:0/1788129731 >> [v2:172.21.15.188:6836/3401128553,v1:172.21.15.188:6837/3401128553] conn(0x7f06b0066740 0x7f06b0068c00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:24:46.478 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:46.478+0000 7f06c27fc640 1 -- 172.21.15.143:0/1788129731 <== mon.0 v2:172.21.15.143:3300/0 5 ==== osd_map(103..103 src has 1..103) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f06bc0843e0 con 0x7f06cc0ce680 2023-11-26T07:24:46.478 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:46.478+0000 7f06d1bf6640 1 --2- 172.21.15.143:0/1788129731 >> [v2:172.21.15.188:6836/3401128553,v1:172.21.15.188:6837/3401128553] conn(0x7f06b0066740 0x7f06b0068c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:24:46.479 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:46.478+0000 7f06d1bf6640 1 --2- 172.21.15.143:0/1788129731 >> [v2:172.21.15.188:6836/3401128553,v1:172.21.15.188:6837/3401128553] conn(0x7f06b0066740 0x7f06b0068c00 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7f06c8005890 tx=0x7f06c8009040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:24:46.482 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:46.482+0000 7f06c27fc640 1 -- 172.21.15.143:0/1788129731 <== mon.0 v2:172.21.15.143:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172649 (secure 0 0 0) 0x7f06cc15b790 con 0x7f06cc0ce680 2023-11-26T07:24:46.631 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:46.630+0000 7f06d3e5a640 1 -- 172.21.15.143:0/1788129731 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f06cc15b750 con 0x7f06cc0ce680 2023-11-26T07:24:46.631 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:46.630+0000 7f06c27fc640 1 -- 172.21.15.143:0/1788129731 <== mon.0 v2:172.21.15.143:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v103) v1 ==== 104+0+11 (secure 0 0 0) 0x7f06bc05b070 con 0x7f06cc0ce680 2023-11-26T07:24:46.631 INFO:teuthology.orchestra.run.smithi143.stdout:pg_num: 32 2023-11-26T07:24:46.634 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:46.634+0000 7f06d3e5a640 1 -- 172.21.15.143:0/1788129731 >> [v2:172.21.15.188:6836/3401128553,v1:172.21.15.188:6837/3401128553] conn(0x7f06b0066740 msgr2=0x7f06b0068c00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:24:46.634 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:46.634+0000 7f06d3e5a640 1 --2- 172.21.15.143:0/1788129731 >> [v2:172.21.15.188:6836/3401128553,v1:172.21.15.188:6837/3401128553] conn(0x7f06b0066740 0x7f06b0068c00 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7f06c8005890 tx=0x7f06c8009040 comp rx=0 tx=0).stop 2023-11-26T07:24:46.634 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:46.634+0000 7f06d3e5a640 1 -- 172.21.15.143:0/1788129731 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f06cc0ce680 msgr2=0x7f06cc14b1a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:24:46.634 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:46.634+0000 7f06d3e5a640 1 --2- 172.21.15.143:0/1788129731 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f06cc0ce680 0x7f06cc14b1a0 secure :-1 s=READY pgs=1397 cs=0 l=1 rev1=1 crypto rx=0x7f06bc00e930 tx=0x7f06bc00edf0 comp rx=0 tx=0).stop 2023-11-26T07:24:46.634 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:46.634+0000 7f06d3e5a640 1 -- 172.21.15.143:0/1788129731 shutdown_connections 2023-11-26T07:24:46.634 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:46.634+0000 7f06d3e5a640 1 --2- 172.21.15.143:0/1788129731 >> [v2:172.21.15.188:6836/3401128553,v1:172.21.15.188:6837/3401128553] conn(0x7f06b0066740 0x7f06b0068c00 unknown :-1 s=CLOSED pgs=163 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:24:46.634 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:46.634+0000 7f06d3e5a640 1 --2- 172.21.15.143:0/1788129731 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f06cc1596d0 0x7f06cc070c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:24:46.634 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:46.634+0000 7f06d3e5a640 1 --2- 172.21.15.143:0/1788129731 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f06cc0ce680 0x7f06cc14b1a0 unknown :-1 s=CLOSED pgs=1397 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:24:46.634 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:46.634+0000 7f06d3e5a640 1 --2- 172.21.15.143:0/1788129731 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f06cc0b2f80 0x7f06cc0a8540 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:24:46.634 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:46.634+0000 7f06d3e5a640 1 -- 172.21.15.143:0/1788129731 >> 172.21.15.143:0/1788129731 conn(0x7f06cc06a890 msgr2=0x7f06cc14e190 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:24:46.635 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:46.634+0000 7f06d3e5a640 1 -- 172.21.15.143:0/1788129731 shutdown_connections 2023-11-26T07:24:46.635 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:46.634+0000 7f06d3e5a640 1 -- 172.21.15.143:0/1788129731 wait complete. 2023-11-26T07:24:46.653 DEBUG:teuthology.orchestra.run.smithi143:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-11-26T07:24:46.794 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:46.794+0000 7f7fe8165640 1 Processor -- start 2023-11-26T07:24:46.794 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:46.794+0000 7f7fe8165640 1 -- start start 2023-11-26T07:24:46.796 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:46.794+0000 7f7fe8165640 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7f7fe014b1e0 0x7f7fe00cf280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:24:46.796 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:46.794+0000 7f7fe8165640 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7f7fe00cf7c0 0x7f7fe00708c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:24:46.796 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:46.794+0000 7f7fe8165640 1 -- --> v1:172.21.15.143:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7fe00cfc20 con 0x7f7fe0070e00 2023-11-26T07:24:46.796 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:46.794+0000 7f7fe8165640 1 -- --> v2:172.21.15.143:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7fe0162080 con 0x7f7fe00cf7c0 2023-11-26T07:24:46.796 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:46.794+0000 7f7fe8165640 1 -- --> v2:172.21.15.188:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7fe0161d70 con 0x7f7fe014b1e0 2023-11-26T07:24:46.797 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:46.794+0000 7f7fe6702640 1 --1- >> v1:172.21.15.143:6789/0 conn(0x7f7fe0070e00 0x7f7fe00b1ef0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.143:6789/0 says I am v1:172.21.15.143:49672/0 (socket says 172.21.15.143:49672) 2023-11-26T07:24:46.797 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:46.794+0000 7f7fe6702640 1 -- 172.21.15.143:0/1303139552 learned_addr learned my addr 172.21.15.143:0/1303139552 (peer_addr_for_me v1:172.21.15.143:0/0) 2023-11-26T07:24:46.797 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:46.794+0000 7f7fe5700640 1 --2- 172.21.15.143:0/1303139552 >> v2:172.21.15.143:3300/0 conn(0x7f7fe00cf7c0 0x7f7fe00708c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:24:46.797 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:46.794+0000 7f7fe5f01640 1 --2- 172.21.15.143:0/1303139552 >> v2:172.21.15.188:3300/0 conn(0x7f7fe014b1e0 0x7f7fe00cf280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:24:46.797 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:46.794+0000 7f7fe4eff640 1 -- 172.21.15.143:0/1303139552 <== mon.0 v1:172.21.15.143:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2909598106 0 0) 0x7f7fe00cfc20 con 0x7f7fe0070e00 2023-11-26T07:24:46.797 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:46.794+0000 7f7fe4eff640 1 -- 172.21.15.143:0/1303139552 --> v1:172.21.15.143:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f7fcc003530 con 0x7f7fe0070e00 2023-11-26T07:24:46.797 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:46.794+0000 7f7fe5700640 1 -- 172.21.15.143:0/1303139552 >> v2:172.21.15.188:3300/0 conn(0x7f7fe014b1e0 msgr2=0x7f7fe00cf280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-26T07:24:46.797 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:46.794+0000 7f7fe5700640 1 --2- 172.21.15.143:0/1303139552 >> v2:172.21.15.188:3300/0 conn(0x7f7fe014b1e0 0x7f7fe00cf280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:24:46.797 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:46.794+0000 7f7fe5700640 1 -- 172.21.15.143:0/1303139552 >> v1:172.21.15.143:6789/0 conn(0x7f7fe0070e00 legacy=0x7f7fe00b1ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:24:46.797 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:46.794+0000 7f7fe5700640 1 -- 172.21.15.143:0/1303139552 --> v2:172.21.15.143:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7fe014b5c0 con 0x7f7fe00cf7c0 2023-11-26T07:24:46.799 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:46.798+0000 7f7fe5700640 1 --2- 172.21.15.143:0/1303139552 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f7fe00cf7c0 0x7f7fe00708c0 secure :-1 s=READY pgs=1399 cs=0 l=1 rev1=1 crypto rx=0x7f7fe006e480 tx=0x7f7fd001c430 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bace6dbf21d784aa server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:24:46.799 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:46.798+0000 7f7fe4eff640 1 -- 172.21.15.143:0/1303139552 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7fd00a7070 con 0x7f7fe00cf7c0 2023-11-26T07:24:46.799 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:46.798+0000 7f7fe4eff640 1 -- 172.21.15.143:0/1303139552 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7fd00ab430 con 0x7f7fe00cf7c0 2023-11-26T07:24:46.799 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:46.798+0000 7f7fe4eff640 1 -- 172.21.15.143:0/1303139552 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7fd00ab7c0 con 0x7f7fe00cf7c0 2023-11-26T07:24:46.800 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:46.798+0000 7f7fe8165640 1 -- 172.21.15.143:0/1303139552 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f7fe00cf7c0 msgr2=0x7f7fe00708c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:24:46.800 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:46.798+0000 7f7fe8165640 1 --2- 172.21.15.143:0/1303139552 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f7fe00cf7c0 0x7f7fe00708c0 secure :-1 s=READY pgs=1399 cs=0 l=1 rev1=1 crypto rx=0x7f7fe006e480 tx=0x7f7fd001c430 comp rx=0 tx=0).stop 2023-11-26T07:24:46.800 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:46.798+0000 7f7fe8165640 1 -- 172.21.15.143:0/1303139552 shutdown_connections 2023-11-26T07:24:46.800 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:46.798+0000 7f7fe8165640 1 --2- 172.21.15.143:0/1303139552 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f7fe00cf7c0 0x7f7fe00708c0 unknown :-1 s=CLOSED pgs=1399 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:24:46.800 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:46.798+0000 7f7fe8165640 1 --2- 172.21.15.143:0/1303139552 >> v2:172.21.15.188:3300/0 conn(0x7f7fe014b1e0 0x7f7fe00cf280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:24:46.801 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:46.798+0000 7f7fe8165640 1 -- 172.21.15.143:0/1303139552 >> 172.21.15.143:0/1303139552 conn(0x7f7fe006a890 msgr2=0x7f7fe006aca0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:24:46.801 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:46.798+0000 7f7fe8165640 1 -- 172.21.15.143:0/1303139552 shutdown_connections 2023-11-26T07:24:46.801 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:46.798+0000 7f7fe8165640 1 -- 172.21.15.143:0/1303139552 wait complete. 2023-11-26T07:24:46.801 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:46.802+0000 7f7fe8165640 1 Processor -- start 2023-11-26T07:24:46.802 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:46.802+0000 7f7fe8165640 1 -- start start 2023-11-26T07:24:46.803 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:46.802+0000 7f7fe8165640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f7fe0070e00 0x7f7fe00ac270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:24:46.803 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:46.802+0000 7f7fe8165640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f7fe00cf7c0 0x7f7fe01fd640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:24:46.803 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:46.802+0000 7f7fe8165640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f7fe014b1e0 0x7f7fe01fdb80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:24:46.803 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:46.802+0000 7f7fe8165640 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7fe0160840 con 0x7f7fe014b1e0 2023-11-26T07:24:46.803 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:46.802+0000 7f7fe8165640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7fe015fc30 con 0x7f7fe0070e00 2023-11-26T07:24:46.803 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:46.802+0000 7f7fe8165640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7fe015ff10 con 0x7f7fe00cf7c0 2023-11-26T07:24:46.803 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:46.802+0000 7f7fe6702640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f7fe014b1e0 0x7f7fe01fdb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:24:46.803 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:46.802+0000 7f7fe6702640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f7fe014b1e0 0x7f7fe01fdb80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:58600/0 (socket says 172.21.15.143:58600) 2023-11-26T07:24:46.803 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:46.802+0000 7f7fe6702640 1 -- 172.21.15.143:0/3741759871 learned_addr learned my addr 172.21.15.143:0/3741759871 (peer_addr_for_me v2:172.21.15.143:0/0) 2023-11-26T07:24:46.805 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:46.802+0000 7f7fe5f01640 1 --2- 172.21.15.143:0/3741759871 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f7fe0070e00 0x7f7fe00ac270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:24:46.805 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:46.802+0000 7f7fe6702640 1 -- 172.21.15.143:0/3741759871 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f7fe00cf7c0 msgr2=0x7f7fe01fd640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:24:46.805 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:46.802+0000 7f7fe5700640 1 --2- 172.21.15.143:0/3741759871 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f7fe00cf7c0 0x7f7fe01fd640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:24:46.805 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:46.802+0000 7f7fe6702640 1 --2- 172.21.15.143:0/3741759871 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f7fe00cf7c0 0x7f7fe01fd640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:24:46.805 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:46.802+0000 7f7fe6702640 1 -- 172.21.15.143:0/3741759871 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f7fe0070e00 msgr2=0x7f7fe00ac270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:24:46.805 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:46.802+0000 7f7fe6702640 1 --2- 172.21.15.143:0/3741759871 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f7fe0070e00 0x7f7fe00ac270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:24:46.805 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:46.802+0000 7f7fe6702640 1 -- 172.21.15.143:0/3741759871 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7fd0009420 con 0x7f7fe014b1e0 2023-11-26T07:24:46.805 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:46.802+0000 7f7fe5f01640 1 --2- 172.21.15.143:0/3741759871 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f7fe0070e00 0x7f7fe00ac270 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-26T07:24:46.805 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:46.802+0000 7f7fe5700640 1 --2- 172.21.15.143:0/3741759871 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f7fe00cf7c0 0x7f7fe01fd640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-26T07:24:46.806 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:46.802+0000 7f7fe6702640 1 --2- 172.21.15.143:0/3741759871 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f7fe014b1e0 0x7f7fe01fdb80 secure :-1 s=READY pgs=1400 cs=0 l=1 rev1=1 crypto rx=0x7f7fd800cb40 tx=0x7f7fd800a060 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:24:46.806 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:46.802+0000 7f7fd67fc640 1 -- 172.21.15.143:0/3741759871 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7fd800aca0 con 0x7f7fe014b1e0 2023-11-26T07:24:46.806 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:46.802+0000 7f7fd67fc640 1 -- 172.21.15.143:0/3741759871 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7fd800ae00 con 0x7f7fe014b1e0 2023-11-26T07:24:46.806 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:46.802+0000 7f7fd67fc640 1 -- 172.21.15.143:0/3741759871 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7fd8012620 con 0x7f7fe014b1e0 2023-11-26T07:24:46.806 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:46.802+0000 7f7fe8165640 1 -- 172.21.15.143:0/3741759871 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7fe01601f0 con 0x7f7fe014b1e0 2023-11-26T07:24:46.806 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:46.802+0000 7f7fe8165640 1 -- 172.21.15.143:0/3741759871 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7fe015f850 con 0x7f7fe014b1e0 2023-11-26T07:24:46.806 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:46.802+0000 7f7fe8165640 1 -- 172.21.15.143:0/3741759871 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7fe01609f0 con 0x7f7fe014b1e0 2023-11-26T07:24:46.807 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:46.806+0000 7f7fd67fc640 1 -- 172.21.15.143:0/3741759871 <== mon.0 v2:172.21.15.143:3300/0 4 ==== mgrmap(e 18) v1 ==== 86107+0+0 (secure 0 0 0) 0x7f7fd8007070 con 0x7f7fe014b1e0 2023-11-26T07:24:46.807 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:46.806+0000 7f7fd67fc640 1 --2- 172.21.15.143:0/3741759871 >> [v2:172.21.15.188:6836/3401128553,v1:172.21.15.188:6837/3401128553] conn(0x7f7fc4066770 0x7f7fc4068c30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:24:46.807 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:46.806+0000 7f7fd67fc640 1 -- 172.21.15.143:0/3741759871 <== mon.0 v2:172.21.15.143:3300/0 5 ==== osd_map(103..103 src has 1..103) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f7fd8084430 con 0x7f7fe014b1e0 2023-11-26T07:24:46.807 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:46.806+0000 7f7fe5f01640 1 --2- 172.21.15.143:0/3741759871 >> [v2:172.21.15.188:6836/3401128553,v1:172.21.15.188:6837/3401128553] conn(0x7f7fc4066770 0x7f7fc4068c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:24:46.808 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:46.806+0000 7f7fe5f01640 1 --2- 172.21.15.143:0/3741759871 >> [v2:172.21.15.188:6836/3401128553,v1:172.21.15.188:6837/3401128553] conn(0x7f7fc4066770 0x7f7fc4068c30 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7f7fdc009a40 tx=0x7f7fdc008040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:24:46.810 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:46.810+0000 7f7fd67fc640 1 -- 172.21.15.143:0/3741759871 <== mon.0 v2:172.21.15.143:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172649 (secure 0 0 0) 0x7f7fe01609f0 con 0x7f7fe014b1e0 2023-11-26T07:24:46.961 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:46.962+0000 7f7fe8165640 1 -- 172.21.15.143:0/3741759871 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f7fe01609f0 con 0x7f7fe014b1e0 2023-11-26T07:24:46.962 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:46.962+0000 7f7fd67fc640 1 -- 172.21.15.143:0/3741759871 <== mon.0 v2:172.21.15.143:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v103) v1 ==== 100+0+11 (secure 0 0 0) 0x7f7fe01609f0 con 0x7f7fe014b1e0 2023-11-26T07:24:46.962 INFO:teuthology.orchestra.run.smithi143.stdout:pg_num: 64 2023-11-26T07:24:46.964 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:46.962+0000 7f7fe8165640 1 -- 172.21.15.143:0/3741759871 >> [v2:172.21.15.188:6836/3401128553,v1:172.21.15.188:6837/3401128553] conn(0x7f7fc4066770 msgr2=0x7f7fc4068c30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:24:46.964 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:46.962+0000 7f7fe8165640 1 --2- 172.21.15.143:0/3741759871 >> [v2:172.21.15.188:6836/3401128553,v1:172.21.15.188:6837/3401128553] conn(0x7f7fc4066770 0x7f7fc4068c30 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7f7fdc009a40 tx=0x7f7fdc008040 comp rx=0 tx=0).stop 2023-11-26T07:24:46.964 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:46.962+0000 7f7fe8165640 1 -- 172.21.15.143:0/3741759871 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f7fe014b1e0 msgr2=0x7f7fe01fdb80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:24:46.964 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:46.962+0000 7f7fe8165640 1 --2- 172.21.15.143:0/3741759871 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f7fe014b1e0 0x7f7fe01fdb80 secure :-1 s=READY pgs=1400 cs=0 l=1 rev1=1 crypto rx=0x7f7fd800cb40 tx=0x7f7fd800a060 comp rx=0 tx=0).stop 2023-11-26T07:24:46.964 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:46.962+0000 7f7fe8165640 1 -- 172.21.15.143:0/3741759871 shutdown_connections 2023-11-26T07:24:46.964 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:46.962+0000 7f7fe8165640 1 --2- 172.21.15.143:0/3741759871 >> [v2:172.21.15.188:6836/3401128553,v1:172.21.15.188:6837/3401128553] conn(0x7f7fc4066770 0x7f7fc4068c30 unknown :-1 s=CLOSED pgs=164 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:24:46.964 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:46.962+0000 7f7fe8165640 1 --2- 172.21.15.143:0/3741759871 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f7fe014b1e0 0x7f7fe01fdb80 unknown :-1 s=CLOSED pgs=1400 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:24:46.965 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:46.962+0000 7f7fe8165640 1 --2- 172.21.15.143:0/3741759871 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f7fe00cf7c0 0x7f7fe01fd640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:24:46.965 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:46.962+0000 7f7fe8165640 1 --2- 172.21.15.143:0/3741759871 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f7fe0070e00 0x7f7fe00ac270 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:24:46.965 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:46.962+0000 7f7fe8165640 1 -- 172.21.15.143:0/3741759871 >> 172.21.15.143:0/3741759871 conn(0x7f7fe006a890 msgr2=0x7f7fe016bc50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:24:46.965 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:46.962+0000 7f7fe8165640 1 -- 172.21.15.143:0/3741759871 shutdown_connections 2023-11-26T07:24:46.965 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:46.962+0000 7f7fe8165640 1 -- 172.21.15.143:0/3741759871 wait complete. 2023-11-26T07:24:46.983 DEBUG:teuthology.orchestra.run.smithi143:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2023-11-26T07:24:47.123 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:47.122+0000 7f7f47e9c640 1 Processor -- start 2023-11-26T07:24:47.124 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:47.122+0000 7f7f47e9c640 1 -- start start 2023-11-26T07:24:47.125 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:47.122+0000 7f7f47e9c640 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7f7f4015d580 0x7f7f400d3920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:24:47.125 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:47.122+0000 7f7f47e9c640 1 -- --> v1:172.21.15.143:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7f400d0bc0 con 0x7f7f400d3e60 2023-11-26T07:24:47.125 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:47.122+0000 7f7f47e9c640 1 -- --> v1:172.21.15.188:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7f400d0320 con 0x7f7f400d1f00 2023-11-26T07:24:47.125 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:47.126+0000 7f7f47e9c640 1 -- --> v2:172.21.15.143:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7f400d0600 con 0x7f7f4015d580 2023-11-26T07:24:47.125 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:47.126+0000 7f7f45c38640 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7f7f4015d580 0x7f7f400d3920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:24:47.125 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:47.126+0000 7f7f45437640 1 --1- >> v1:172.21.15.143:6789/0 conn(0x7f7f400d3e60 0x7f7f400d17f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.143:6789/0 says I am v1:172.21.15.143:49682/0 (socket says 172.21.15.143:49682) 2023-11-26T07:24:47.126 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:47.126+0000 7f7f45437640 1 -- 172.21.15.143:0/1440593071 learned_addr learned my addr 172.21.15.143:0/1440593071 (peer_addr_for_me v1:172.21.15.143:0/0) 2023-11-26T07:24:47.126 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:47.126+0000 7f7f45c38640 1 -- 172.21.15.143:0/1440593071 >> v1:172.21.15.188:6789/0 conn(0x7f7f400d1f00 legacy=0x7f7f400b0490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-26T07:24:47.126 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:47.126+0000 7f7f45c38640 1 -- 172.21.15.143:0/1440593071 >> v1:172.21.15.143:6789/0 conn(0x7f7f400d3e60 legacy=0x7f7f400d17f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:24:47.126 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:47.126+0000 7f7f45c38640 1 -- 172.21.15.143:0/1440593071 --> v2:172.21.15.143:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7f400d08e0 con 0x7f7f4015d580 2023-11-26T07:24:47.126 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:47.126+0000 7f7f44c36640 1 -- 172.21.15.143:0/1440593071 <== mon.0 v1:172.21.15.143:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 927481482 0 0) 0x7f7f400d0bc0 con 0x7f7f400d3e60 2023-11-26T07:24:47.129 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:47.126+0000 7f7f45c38640 1 --2- 172.21.15.143:0/1440593071 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f7f4015d580 0x7f7f400d3920 secure :-1 s=READY pgs=1402 cs=0 l=1 rev1=1 crypto rx=0x7f7f3c0097c0 tx=0x7f7f3c01c410 comp rx=0 tx=0).ready entity=mon.0 client_cookie=88a8aaa4d015d8c5 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:24:47.129 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:47.126+0000 7f7f44c36640 1 -- 172.21.15.143:0/1440593071 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7f3c0ab600 con 0x7f7f4015d580 2023-11-26T07:24:47.129 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:47.126+0000 7f7f44c36640 1 -- 172.21.15.143:0/1440593071 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7f3c0a7050 con 0x7f7f4015d580 2023-11-26T07:24:47.129 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:47.126+0000 7f7f44c36640 1 -- 172.21.15.143:0/1440593071 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7f3c0ab980 con 0x7f7f4015d580 2023-11-26T07:24:47.130 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:47.130+0000 7f7f47e9c640 1 -- 172.21.15.143:0/1440593071 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f7f4015d580 msgr2=0x7f7f400d3920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:24:47.130 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:47.130+0000 7f7f47e9c640 1 --2- 172.21.15.143:0/1440593071 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f7f4015d580 0x7f7f400d3920 secure :-1 s=READY pgs=1402 cs=0 l=1 rev1=1 crypto rx=0x7f7f3c0097c0 tx=0x7f7f3c01c410 comp rx=0 tx=0).stop 2023-11-26T07:24:47.130 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:47.130+0000 7f7f47e9c640 1 -- 172.21.15.143:0/1440593071 shutdown_connections 2023-11-26T07:24:47.130 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:47.130+0000 7f7f47e9c640 1 --2- 172.21.15.143:0/1440593071 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f7f4015d580 0x7f7f400d3920 unknown :-1 s=CLOSED pgs=1402 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:24:47.130 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:47.130+0000 7f7f47e9c640 1 -- 172.21.15.143:0/1440593071 >> 172.21.15.143:0/1440593071 conn(0x7f7f4009c550 msgr2=0x7f7f400d3210 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:24:47.130 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:47.130+0000 7f7f47e9c640 1 -- 172.21.15.143:0/1440593071 shutdown_connections 2023-11-26T07:24:47.130 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:47.130+0000 7f7f47e9c640 1 -- 172.21.15.143:0/1440593071 wait complete. 2023-11-26T07:24:47.131 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:47.130+0000 7f7f47e9c640 1 Processor -- start 2023-11-26T07:24:47.131 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:47.130+0000 7f7f47e9c640 1 -- start start 2023-11-26T07:24:47.133 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:47.130+0000 7f7f47e9c640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f7f400d1f00 0x7f7f40154d00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:24:47.134 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:47.130+0000 7f7f47e9c640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f7f400d3e60 0x7f7f4015bfa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:24:47.134 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:47.130+0000 7f7f47e9c640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f7f4015d580 0x7f7f400b7e70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:24:47.134 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:47.130+0000 7f7f47e9c640 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7f4015cd20 con 0x7f7f4015d580 2023-11-26T07:24:47.134 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:47.130+0000 7f7f47e9c640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7f4015ca40 con 0x7f7f400d3e60 2023-11-26T07:24:47.134 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:47.130+0000 7f7f47e9c640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7f40155fe0 con 0x7f7f400d1f00 2023-11-26T07:24:47.134 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:47.130+0000 7f7f46439640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f7f4015d580 0x7f7f400b7e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:24:47.134 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:47.130+0000 7f7f46439640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f7f4015d580 0x7f7f400b7e70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:58614/0 (socket says 172.21.15.143:58614) 2023-11-26T07:24:47.134 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:47.130+0000 7f7f46439640 1 -- 172.21.15.143:0/2214604214 learned_addr learned my addr 172.21.15.143:0/2214604214 (peer_addr_for_me v2:172.21.15.143:0/0) 2023-11-26T07:24:47.134 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:47.130+0000 7f7f45c38640 1 --2- 172.21.15.143:0/2214604214 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f7f400d1f00 0x7f7f40154d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:24:47.134 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:47.130+0000 7f7f45437640 1 --2- 172.21.15.143:0/2214604214 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f7f400d3e60 0x7f7f4015bfa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:24:47.134 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:47.130+0000 7f7f46439640 1 -- 172.21.15.143:0/2214604214 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f7f400d1f00 msgr2=0x7f7f40154d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:24:47.135 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:47.130+0000 7f7f46439640 1 --2- 172.21.15.143:0/2214604214 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f7f400d1f00 0x7f7f40154d00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:24:47.135 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:47.130+0000 7f7f46439640 1 -- 172.21.15.143:0/2214604214 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f7f400d3e60 msgr2=0x7f7f4015bfa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:24:47.135 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:47.130+0000 7f7f46439640 1 --2- 172.21.15.143:0/2214604214 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f7f400d3e60 0x7f7f4015bfa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:24:47.135 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:47.130+0000 7f7f46439640 1 -- 172.21.15.143:0/2214604214 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7f3c009420 con 0x7f7f4015d580 2023-11-26T07:24:47.135 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:47.130+0000 7f7f45437640 1 --2- 172.21.15.143:0/2214604214 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f7f400d3e60 0x7f7f4015bfa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-26T07:24:47.135 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:47.134+0000 7f7f46439640 1 --2- 172.21.15.143:0/2214604214 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f7f4015d580 0x7f7f400b7e70 secure :-1 s=READY pgs=1403 cs=0 l=1 rev1=1 crypto rx=0x7f7f3800c780 tx=0x7f7f3800cc40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:24:47.135 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:47.134+0000 7f7f367fc640 1 -- 172.21.15.143:0/2214604214 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7f3800a9f0 con 0x7f7f4015d580 2023-11-26T07:24:47.135 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:47.134+0000 7f7f367fc640 1 -- 172.21.15.143:0/2214604214 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7f3800ce90 con 0x7f7f4015d580 2023-11-26T07:24:47.135 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:47.134+0000 7f7f47e9c640 1 -- 172.21.15.143:0/2214604214 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7f401562c0 con 0x7f7f4015d580 2023-11-26T07:24:47.135 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:47.134+0000 7f7f367fc640 1 -- 172.21.15.143:0/2214604214 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7f3800add0 con 0x7f7f4015d580 2023-11-26T07:24:47.135 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:47.134+0000 7f7f47e9c640 1 -- 172.21.15.143:0/2214604214 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7f40155240 con 0x7f7f4015d580 2023-11-26T07:24:47.136 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:47.134+0000 7f7f47e9c640 1 -- 172.21.15.143:0/2214604214 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7f40067680 con 0x7f7f4015d580 2023-11-26T07:24:47.136 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:47.134+0000 7f7f367fc640 1 -- 172.21.15.143:0/2214604214 <== mon.0 v2:172.21.15.143:3300/0 4 ==== mgrmap(e 18) v1 ==== 86107+0+0 (secure 0 0 0) 0x7f7f38012420 con 0x7f7f4015d580 2023-11-26T07:24:47.136 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:47.134+0000 7f7f367fc640 1 --2- 172.21.15.143:0/2214604214 >> [v2:172.21.15.188:6836/3401128553,v1:172.21.15.188:6837/3401128553] conn(0x7f7f24066740 0x7f7f24068c00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:24:47.137 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:47.134+0000 7f7f367fc640 1 -- 172.21.15.143:0/2214604214 <== mon.0 v2:172.21.15.143:3300/0 5 ==== osd_map(103..103 src has 1..103) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f7f38084220 con 0x7f7f4015d580 2023-11-26T07:24:47.137 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:47.138+0000 7f7f45c38640 1 --2- 172.21.15.143:0/2214604214 >> [v2:172.21.15.188:6836/3401128553,v1:172.21.15.188:6837/3401128553] conn(0x7f7f24066740 0x7f7f24068c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:24:47.137 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:47.138+0000 7f7f45c38640 1 --2- 172.21.15.143:0/2214604214 >> [v2:172.21.15.188:6836/3401128553,v1:172.21.15.188:6837/3401128553] conn(0x7f7f24066740 0x7f7f24068c00 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7f7f400a1190 tx=0x7f7f3c0a6000 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:24:47.140 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:47.138+0000 7f7f367fc640 1 -- 172.21.15.143:0/2214604214 <== mon.0 v2:172.21.15.143:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172649 (secure 0 0 0) 0x7f7f380512c0 con 0x7f7f4015d580 2023-11-26T07:24:47.308 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:47.306+0000 7f7f47e9c640 1 -- 172.21.15.143:0/2214604214 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) v1 -- 0x7f7f4015f8b0 con 0x7f7f4015d580 2023-11-26T07:24:47.308 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:47.306+0000 7f7f367fc640 1 -- 172.21.15.143:0/2214604214 <== mon.0 v2:172.21.15.143:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v103) v1 ==== 80+0+0 (secure 0 0 0) 0x7f7f380548e0 con 0x7f7f4015d580 2023-11-26T07:24:47.308 INFO:teuthology.orchestra.run.smithi143.stderr:listed 0 entries 2023-11-26T07:24:47.310 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:47.310+0000 7f7f47e9c640 1 -- 172.21.15.143:0/2214604214 >> [v2:172.21.15.188:6836/3401128553,v1:172.21.15.188:6837/3401128553] conn(0x7f7f24066740 msgr2=0x7f7f24068c00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:24:47.310 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:47.310+0000 7f7f47e9c640 1 --2- 172.21.15.143:0/2214604214 >> [v2:172.21.15.188:6836/3401128553,v1:172.21.15.188:6837/3401128553] conn(0x7f7f24066740 0x7f7f24068c00 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7f7f400a1190 tx=0x7f7f3c0a6000 comp rx=0 tx=0).stop 2023-11-26T07:24:47.310 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:47.310+0000 7f7f47e9c640 1 -- 172.21.15.143:0/2214604214 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f7f4015d580 msgr2=0x7f7f400b7e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:24:47.310 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:47.310+0000 7f7f47e9c640 1 --2- 172.21.15.143:0/2214604214 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f7f4015d580 0x7f7f400b7e70 secure :-1 s=READY pgs=1403 cs=0 l=1 rev1=1 crypto rx=0x7f7f3800c780 tx=0x7f7f3800cc40 comp rx=0 tx=0).stop 2023-11-26T07:24:47.311 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:47.310+0000 7f7f47e9c640 1 -- 172.21.15.143:0/2214604214 shutdown_connections 2023-11-26T07:24:47.311 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:47.310+0000 7f7f47e9c640 1 --2- 172.21.15.143:0/2214604214 >> [v2:172.21.15.188:6836/3401128553,v1:172.21.15.188:6837/3401128553] conn(0x7f7f24066740 0x7f7f24068c00 unknown :-1 s=CLOSED pgs=165 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:24:47.311 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:47.310+0000 7f7f47e9c640 1 --2- 172.21.15.143:0/2214604214 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f7f4015d580 0x7f7f400b7e70 unknown :-1 s=CLOSED pgs=1403 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:24:47.311 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:47.310+0000 7f7f47e9c640 1 --2- 172.21.15.143:0/2214604214 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f7f400d3e60 0x7f7f4015bfa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:24:47.311 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:47.310+0000 7f7f47e9c640 1 --2- 172.21.15.143:0/2214604214 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f7f400d1f00 0x7f7f40154d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:24:47.311 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:47.310+0000 7f7f47e9c640 1 -- 172.21.15.143:0/2214604214 >> 172.21.15.143:0/2214604214 conn(0x7f7f4009c550 msgr2=0x7f7f4015f1a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:24:47.311 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:47.310+0000 7f7f47e9c640 1 -- 172.21.15.143:0/2214604214 shutdown_connections 2023-11-26T07:24:47.311 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:47.310+0000 7f7f47e9c640 1 -- 172.21.15.143:0/2214604214 wait complete. 2023-11-26T07:24:47.329 INFO:tasks.cephfs.fuse_mount:Running fusermount -u on ubuntu@smithi188.front.sepia.ceph.com... 2023-11-26T07:24:47.329 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-26T07:24:47.330 DEBUG:teuthology.orchestra.run.smithi188:> sudo fusermount -u /home/ubuntu/cephtest/mnt.1 2023-11-26T07:24:47.350 INFO:tasks.cephfs.fuse_mount.ceph-fuse.1.smithi188.stderr:ceph-fuse[45326]: fuse finished with error 0 and tester_r 0 2023-11-26T07:24:47.353 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi188.front.sepia.ceph.com 2023-11-26T07:24:47.354 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-26T07:24:47.354 DEBUG:teuthology.orchestra.run.smithi188:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.1) 2023-11-26T07:24:47.446 DEBUG:teuthology.orchestra.run.smithi143:> set -ex 2023-11-26T07:24:47.446 DEBUG:teuthology.orchestra.run.smithi143:> dd if=/proc/self/mounts of=/dev/stdout 2023-11-26T07:24:47.455 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,relatime,size=16354672k,nr_inodes=4088668,mode=755 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3275384k,mode=755 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=3455 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysusers.service ramfs ro,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3275380k,nr_inodes=818845,mode=700,uid=1000,gid=1271 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3275384k,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2023-11-26T07:24:47.455 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi143.front.sepia.ceph.com 2023-11-26T07:24:47.455 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-26T07:24:47.455 DEBUG:teuthology.orchestra.run.smithi143:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.2) 2023-11-26T07:24:47.504 INFO:teuthology.orchestra.run.smithi143.stderr:rmdir: failed to remove '/home/ubuntu/cephtest/mnt.2': No such file or directory 2023-11-26T07:24:47.505 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-11-26T07:24:47.505 DEBUG:teuthology.orchestra.run.smithi188:> set -ex 2023-11-26T07:24:47.506 DEBUG:teuthology.orchestra.run.smithi188:> dd if=/proc/self/mounts of=/dev/stdout 2023-11-26T07:24:47.515 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,relatime,size=16354628k,nr_inodes=4088657,mode=755 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3275376k,mode=755 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=787 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysusers.service ramfs ro,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3275372k,nr_inodes=818843,mode=700,uid=1000,gid=1271 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3275376k,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2023-11-26T07:24:47.515 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi188.front.sepia.ceph.com 2023-11-26T07:24:47.515 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-26T07:24:47.515 DEBUG:teuthology.orchestra.run.smithi188:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.3) 2023-11-26T07:24:47.567 INFO:teuthology.orchestra.run.smithi188.stderr:rmdir: failed to remove '/home/ubuntu/cephtest/mnt.3': No such file or directory 2023-11-26T07:24:47.568 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-11-26T07:24:47.568 DEBUG:teuthology.orchestra.run.smithi143:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-11-26T07:24:47.708 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:47.706+0000 7f660829c640 1 Processor -- start 2023-11-26T07:24:47.708 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:47.706+0000 7f660829c640 1 -- start start 2023-11-26T07:24:47.710 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:47.710+0000 7f660829c640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f6600143ac0 0x7f6600070370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:24:47.710 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:47.710+0000 7f660829c640 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7f66000708b0 0x7f66000cf280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:24:47.710 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:47.710+0000 7f660829c640 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7f66000cf7c0 0x7f66000b1ef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:24:47.710 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:47.710+0000 7f660829c640 1 -- --> v2:172.21.15.143:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6600143ea0 con 0x7f66000cf7c0 2023-11-26T07:24:47.711 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:47.710+0000 7f660829c640 1 -- --> v2:172.21.15.188:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f66000cfbf0 con 0x7f66000708b0 2023-11-26T07:24:47.711 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:47.710+0000 7f660829c640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f66000cc730 con 0x7f6600143ac0 2023-11-26T07:24:47.711 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:47.710+0000 7f6606839640 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7f66000cf7c0 0x7f66000b1ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:24:47.711 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:47.710+0000 7f6606839640 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7f66000cf7c0 0x7f66000b1ef0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:58618/0 (socket says 172.21.15.143:58618) 2023-11-26T07:24:47.711 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:47.710+0000 7f6606839640 1 -- 172.21.15.143:0/4282571196 learned_addr learned my addr 172.21.15.143:0/4282571196 (peer_addr_for_me v2:172.21.15.143:0/0) 2023-11-26T07:24:47.711 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:47.710+0000 7f6606038640 1 --2- 172.21.15.143:0/4282571196 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f6600143ac0 0x7f6600070370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:24:47.711 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:47.710+0000 7f6605837640 1 --2- 172.21.15.143:0/4282571196 >> v2:172.21.15.188:3300/0 conn(0x7f66000708b0 0x7f66000cf280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:24:47.711 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:47.710+0000 7f6606839640 1 -- 172.21.15.143:0/4282571196 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f6600143ac0 msgr2=0x7f6600070370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-26T07:24:47.711 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:47.710+0000 7f6606839640 1 --2- 172.21.15.143:0/4282571196 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f6600143ac0 0x7f6600070370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:24:47.711 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:47.710+0000 7f6606839640 1 -- 172.21.15.143:0/4282571196 >> v2:172.21.15.188:3300/0 conn(0x7f66000708b0 msgr2=0x7f66000cf280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-26T07:24:47.711 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:47.710+0000 7f6606839640 1 --2- 172.21.15.143:0/4282571196 >> v2:172.21.15.188:3300/0 conn(0x7f66000708b0 0x7f66000cf280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:24:47.711 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:47.710+0000 7f6606839640 1 -- 172.21.15.143:0/4282571196 --> v2:172.21.15.143:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6600070ee0 con 0x7f66000cf7c0 2023-11-26T07:24:47.711 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:47.710+0000 7f6605837640 1 --2- 172.21.15.143:0/4282571196 >> v2:172.21.15.188:3300/0 conn(0x7f66000708b0 0x7f66000cf280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-26T07:24:47.713 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:47.714+0000 7f6606839640 1 --2- 172.21.15.143:0/4282571196 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f66000cf7c0 0x7f66000b1ef0 secure :-1 s=READY pgs=1404 cs=0 l=1 rev1=1 crypto rx=0x7f66000cfec0 tx=0x7f65f801c710 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8438e5c7560993b1 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:24:47.713 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:47.714+0000 7f6605036640 1 -- 172.21.15.143:0/4282571196 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f65f80a64b0 con 0x7f66000cf7c0 2023-11-26T07:24:47.714 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:47.714+0000 7f6605036640 1 -- 172.21.15.143:0/4282571196 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f65f80a6610 con 0x7f66000cf7c0 2023-11-26T07:24:47.714 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:47.714+0000 7f6605036640 1 -- 172.21.15.143:0/4282571196 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f65f80a69a0 con 0x7f66000cf7c0 2023-11-26T07:24:47.714 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:47.714+0000 7f660829c640 1 -- 172.21.15.143:0/4282571196 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f66000cf7c0 msgr2=0x7f66000b1ef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:24:47.715 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:47.714+0000 7f660829c640 1 --2- 172.21.15.143:0/4282571196 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f66000cf7c0 0x7f66000b1ef0 secure :-1 s=READY pgs=1404 cs=0 l=1 rev1=1 crypto rx=0x7f66000cfec0 tx=0x7f65f801c710 comp rx=0 tx=0).stop 2023-11-26T07:24:47.715 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:47.714+0000 7f660829c640 1 -- 172.21.15.143:0/4282571196 shutdown_connections 2023-11-26T07:24:47.715 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:47.714+0000 7f660829c640 1 --2- 172.21.15.143:0/4282571196 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f66000cf7c0 0x7f66000b1ef0 unknown :-1 s=CLOSED pgs=1404 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:24:47.715 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:47.714+0000 7f660829c640 1 --2- 172.21.15.143:0/4282571196 >> v2:172.21.15.188:3300/0 conn(0x7f66000708b0 0x7f66000cf280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:24:47.715 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:47.714+0000 7f660829c640 1 --2- 172.21.15.143:0/4282571196 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f6600143ac0 0x7f6600070370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:24:47.715 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:47.714+0000 7f660829c640 1 -- 172.21.15.143:0/4282571196 >> 172.21.15.143:0/4282571196 conn(0x7f660006a890 msgr2=0x7f660006aca0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:24:47.715 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:47.714+0000 7f660829c640 1 -- 172.21.15.143:0/4282571196 shutdown_connections 2023-11-26T07:24:47.715 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:47.714+0000 7f660829c640 1 -- 172.21.15.143:0/4282571196 wait complete. 2023-11-26T07:24:47.716 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:47.714+0000 7f660829c640 1 Processor -- start 2023-11-26T07:24:47.716 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:47.714+0000 7f660829c640 1 -- start start 2023-11-26T07:24:47.718 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:47.714+0000 7f660829c640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f66000708b0 0x7f6600159b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:24:47.718 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:47.714+0000 7f660829c640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f66000cf7c0 0x7f66000b0520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:24:47.718 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:47.714+0000 7f660829c640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f6600143ac0 0x7f66000b0a60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:24:47.718 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:47.714+0000 7f660829c640 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f66000ac8e0 con 0x7f6600143ac0 2023-11-26T07:24:47.718 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:47.714+0000 7f660829c640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6600139820 con 0x7f66000cf7c0 2023-11-26T07:24:47.718 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:47.714+0000 7f660829c640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6600139b00 con 0x7f66000708b0 2023-11-26T07:24:47.718 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:47.718+0000 7f6606839640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f6600143ac0 0x7f66000b0a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:24:47.719 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:47.718+0000 7f6606839640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f6600143ac0 0x7f66000b0a60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:58620/0 (socket says 172.21.15.143:58620) 2023-11-26T07:24:47.719 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:47.718+0000 7f6606839640 1 -- 172.21.15.143:0/139780254 learned_addr learned my addr 172.21.15.143:0/139780254 (peer_addr_for_me v2:172.21.15.143:0/0) 2023-11-26T07:24:47.719 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:47.718+0000 7f6606038640 1 --2- 172.21.15.143:0/139780254 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f66000708b0 0x7f6600159b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:24:47.719 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:47.718+0000 7f6605837640 1 --2- 172.21.15.143:0/139780254 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f66000cf7c0 0x7f66000b0520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:24:47.719 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:47.718+0000 7f6606839640 1 -- 172.21.15.143:0/139780254 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f66000708b0 msgr2=0x7f6600159b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:24:47.719 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:47.718+0000 7f6606839640 1 --2- 172.21.15.143:0/139780254 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f66000708b0 0x7f6600159b90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:24:47.719 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:47.718+0000 7f6606839640 1 -- 172.21.15.143:0/139780254 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f66000cf7c0 msgr2=0x7f66000b0520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:24:47.719 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:47.718+0000 7f6606839640 1 --2- 172.21.15.143:0/139780254 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f66000cf7c0 0x7f66000b0520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:24:47.719 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:47.718+0000 7f6606839640 1 -- 172.21.15.143:0/139780254 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f65f8009b30 con 0x7f6600143ac0 2023-11-26T07:24:47.719 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:47.718+0000 7f6606839640 1 --2- 172.21.15.143:0/139780254 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f6600143ac0 0x7f66000b0a60 secure :-1 s=READY pgs=1405 cs=0 l=1 rev1=1 crypto rx=0x7f65f801adc0 tx=0x7f65f80194b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:24:47.719 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:47.718+0000 7f65f6ffd640 1 -- 172.21.15.143:0/139780254 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f65f80b4070 con 0x7f6600143ac0 2023-11-26T07:24:47.719 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:47.718+0000 7f65f6ffd640 1 -- 172.21.15.143:0/139780254 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f65f8009150 con 0x7f6600143ac0 2023-11-26T07:24:47.720 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:47.718+0000 7f660829c640 1 -- 172.21.15.143:0/139780254 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6600139de0 con 0x7f6600143ac0 2023-11-26T07:24:47.720 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:47.718+0000 7f65f6ffd640 1 -- 172.21.15.143:0/139780254 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f65f80a69a0 con 0x7f6600143ac0 2023-11-26T07:24:47.720 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:47.718+0000 7f660829c640 1 -- 172.21.15.143:0/139780254 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f660015a0d0 con 0x7f6600143ac0 2023-11-26T07:24:47.720 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:47.718+0000 7f660829c640 1 -- 172.21.15.143:0/139780254 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6600067680 con 0x7f6600143ac0 2023-11-26T07:24:47.721 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:47.718+0000 7f65f6ffd640 1 -- 172.21.15.143:0/139780254 <== mon.0 v2:172.21.15.143:3300/0 4 ==== mgrmap(e 18) v1 ==== 86107+0+0 (secure 0 0 0) 0x7f65f80b8070 con 0x7f6600143ac0 2023-11-26T07:24:47.722 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:47.722+0000 7f65f6ffd640 1 --2- 172.21.15.143:0/139780254 >> [v2:172.21.15.188:6836/3401128553,v1:172.21.15.188:6837/3401128553] conn(0x7f65e4066740 0x7f65e4068c00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:24:47.722 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:47.722+0000 7f65f6ffd640 1 -- 172.21.15.143:0/139780254 <== mon.0 v2:172.21.15.143:3300/0 5 ==== osd_map(103..103 src has 1..103) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f65f80aa070 con 0x7f6600143ac0 2023-11-26T07:24:47.722 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:47.722+0000 7f6606038640 1 --2- 172.21.15.143:0/139780254 >> [v2:172.21.15.188:6836/3401128553,v1:172.21.15.188:6837/3401128553] conn(0x7f65e4066740 0x7f65e4068c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:24:47.723 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:47.722+0000 7f6606038640 1 --2- 172.21.15.143:0/139780254 >> [v2:172.21.15.188:6836/3401128553,v1:172.21.15.188:6837/3401128553] conn(0x7f65e4066740 0x7f65e4068c00 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7f660013a410 tx=0x7f65fc008040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:24:47.724 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:47.722+0000 7f65f6ffd640 1 -- 172.21.15.143:0/139780254 <== mon.0 v2:172.21.15.143:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172649 (secure 0 0 0) 0x7f65f80eb250 con 0x7f6600143ac0 2023-11-26T07:24:47.904 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:47.902+0000 7f660829c640 1 -- 172.21.15.143:0/139780254 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f66000acd00 con 0x7f6600143ac0 2023-11-26T07:24:47.905 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:47.902+0000 7f65f6ffd640 1 -- 172.21.15.143:0/139780254 <== mon.0 v2:172.21.15.143:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 66 v66) v1 ==== 94+0+4927 (secure 0 0 0) 0x7f65f80ee870 con 0x7f6600143ac0 2023-11-26T07:24:47.905 INFO:teuthology.orchestra.run.smithi143.stdout: 2023-11-26T07:24:47.905 INFO:teuthology.orchestra.run.smithi143.stdout:{"epoch":66,"default_fscid":8,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6629,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.188:6834/1327398471","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6832","nonce":1327398471},{"type":"v1","addr":"172.21.15.188:6834","nonce":1327398471}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":60},{"gid":7453,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.188:6835/1640976426","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6833","nonce":1640976426},{"type":"v1","addr":"172.21.15.188:6835","nonce":1640976426}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":60},{"gid":7941,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.143:6834/2855111671","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6832","nonce":2855111671},{"type":"v1","addr":"172.21.15.143:6834","nonce":2855111671}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":60}],"filesystems":[{"mdsmap":{"epoch":65,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-11-26T07:24:08.295882+0000","modified":"2023-11-26T07:24:11.308119+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":7944},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7944":{"gid":7944,"name":"a","rank":0,"incarnation":61,"state":"up:active","state_seq":3,"addr":"172.21.15.143:6835/2571525464","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6833","nonce":2571525464},{"type":"v1","addr":"172.21.15.143:6835","nonce":2571525464}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[17],"metadata_pool":16,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":8}]} 2023-11-26T07:24:47.905 INFO:teuthology.orchestra.run.smithi143.stderr:dumped fsmap epoch 66 2023-11-26T07:24:47.907 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:47.906+0000 7f660829c640 1 -- 172.21.15.143:0/139780254 >> [v2:172.21.15.188:6836/3401128553,v1:172.21.15.188:6837/3401128553] conn(0x7f65e4066740 msgr2=0x7f65e4068c00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:24:47.907 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:47.906+0000 7f660829c640 1 --2- 172.21.15.143:0/139780254 >> [v2:172.21.15.188:6836/3401128553,v1:172.21.15.188:6837/3401128553] conn(0x7f65e4066740 0x7f65e4068c00 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7f660013a410 tx=0x7f65fc008040 comp rx=0 tx=0).stop 2023-11-26T07:24:47.907 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:47.906+0000 7f660829c640 1 -- 172.21.15.143:0/139780254 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f6600143ac0 msgr2=0x7f66000b0a60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:24:47.907 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:47.906+0000 7f660829c640 1 --2- 172.21.15.143:0/139780254 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f6600143ac0 0x7f66000b0a60 secure :-1 s=READY pgs=1405 cs=0 l=1 rev1=1 crypto rx=0x7f65f801adc0 tx=0x7f65f80194b0 comp rx=0 tx=0).stop 2023-11-26T07:24:47.907 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:47.906+0000 7f660829c640 1 -- 172.21.15.143:0/139780254 shutdown_connections 2023-11-26T07:24:47.907 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:47.906+0000 7f660829c640 1 --2- 172.21.15.143:0/139780254 >> [v2:172.21.15.188:6836/3401128553,v1:172.21.15.188:6837/3401128553] conn(0x7f65e4066740 0x7f65e4068c00 unknown :-1 s=CLOSED pgs=166 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:24:47.907 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:47.906+0000 7f660829c640 1 --2- 172.21.15.143:0/139780254 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f6600143ac0 0x7f66000b0a60 unknown :-1 s=CLOSED pgs=1405 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:24:47.907 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:47.906+0000 7f660829c640 1 --2- 172.21.15.143:0/139780254 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f66000cf7c0 0x7f66000b0520 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:24:47.907 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:47.906+0000 7f660829c640 1 --2- 172.21.15.143:0/139780254 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f66000708b0 0x7f6600159b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:24:47.907 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:47.906+0000 7f660829c640 1 -- 172.21.15.143:0/139780254 >> 172.21.15.143:0/139780254 conn(0x7f660006a890 msgr2=0x7f66000ce720 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:24:47.908 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:47.906+0000 7f660829c640 1 -- 172.21.15.143:0/139780254 shutdown_connections 2023-11-26T07:24:47.908 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:47.906+0000 7f660829c640 1 -- 172.21.15.143:0/139780254 wait complete. 2023-11-26T07:24:47.926 DEBUG:teuthology.orchestra.run.smithi143:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-11-26T07:24:48.067 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:48.066+0000 7f2f49a6b640 1 Processor -- start 2023-11-26T07:24:48.067 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:48.066+0000 7f2f49a6b640 1 -- start start 2023-11-26T07:24:48.069 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:48.066+0000 7f2f49a6b640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f2f440ce680 0x7f2f44071850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:24:48.069 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:48.066+0000 7f2f49a6b640 1 -- --> v1:172.21.15.143:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2f4406c750 con 0x7f2f440b2f80 2023-11-26T07:24:48.069 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:48.066+0000 7f2f49a6b640 1 -- --> v1:172.21.15.188:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2f440b3550 con 0x7f2f441596d0 2023-11-26T07:24:48.069 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:48.066+0000 7f2f49a6b640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2f440cbe00 con 0x7f2f440ce680 2023-11-26T07:24:48.069 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:48.066+0000 7f2f42ffd640 1 --1- >> v1:172.21.15.143:6789/0 conn(0x7f2f440b2f80 0x7f2f4406cbe0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.143:6789/0 says I am v1:172.21.15.143:49698/0 (socket says 172.21.15.143:49698) 2023-11-26T07:24:48.069 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:48.066+0000 7f2f42ffd640 1 -- 172.21.15.143:0/259001189 learned_addr learned my addr 172.21.15.143:0/259001189 (peer_addr_for_me v1:172.21.15.143:0/0) 2023-11-26T07:24:48.069 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:48.070+0000 7f2f43fff640 1 --2- 172.21.15.143:0/259001189 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f2f440ce680 0x7f2f44071850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:24:48.070 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:48.070+0000 7f2f427fc640 1 -- 172.21.15.143:0/259001189 <== mon.0 v1:172.21.15.143:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3666466042 0 0) 0x7f2f4406c750 con 0x7f2f440b2f80 2023-11-26T07:24:48.070 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:48.070+0000 7f2f427fc640 1 -- 172.21.15.143:0/259001189 --> v1:172.21.15.143:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f2f2c004640 con 0x7f2f440b2f80 2023-11-26T07:24:48.070 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:48.070+0000 7f2f427fc640 1 -- 172.21.15.143:0/259001189 <== mon.1 v1:172.21.15.188:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1986775027 0 0) 0x7f2f440b3550 con 0x7f2f441596d0 2023-11-26T07:24:48.070 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:48.070+0000 7f2f427fc640 1 -- 172.21.15.143:0/259001189 --> v1:172.21.15.188:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f2f4406c750 con 0x7f2f441596d0 2023-11-26T07:24:48.070 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:48.070+0000 7f2f43fff640 1 -- 172.21.15.143:0/259001189 >> v1:172.21.15.188:6789/0 conn(0x7f2f441596d0 legacy=0x7f2f440b2870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:24:48.070 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:48.070+0000 7f2f43fff640 1 -- 172.21.15.143:0/259001189 >> v1:172.21.15.143:6789/0 conn(0x7f2f440b2f80 legacy=0x7f2f4406cbe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:24:48.070 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:48.070+0000 7f2f43fff640 1 -- 172.21.15.143:0/259001189 --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2f44071e80 con 0x7f2f440ce680 2023-11-26T07:24:48.072 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:48.070+0000 7f2f43fff640 1 --2- 172.21.15.143:0/259001189 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f2f440ce680 0x7f2f44071850 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7f2f3c009dc0 tx=0x7f2f3c01c970 comp rx=0 tx=0).ready entity=mon.2 client_cookie=18588c15d9481afc server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:24:48.073 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:48.070+0000 7f2f427fc640 1 -- 172.21.15.143:0/259001189 <== mon.2 v2:172.21.15.188:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2f3c0a58a0 con 0x7f2f440ce680 2023-11-26T07:24:48.073 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:48.074+0000 7f2f427fc640 1 -- 172.21.15.143:0/259001189 <== mon.2 v2:172.21.15.188:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2f3c0a5a00 con 0x7f2f440ce680 2023-11-26T07:24:48.074 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:48.074+0000 7f2f427fc640 1 -- 172.21.15.143:0/259001189 <== mon.2 v2:172.21.15.188:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2f3c0a5d70 con 0x7f2f440ce680 2023-11-26T07:24:48.074 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:48.074+0000 7f2f49a6b640 1 -- 172.21.15.143:0/259001189 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f2f440ce680 msgr2=0x7f2f44071850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:24:48.074 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:48.074+0000 7f2f49a6b640 1 --2- 172.21.15.143:0/259001189 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f2f440ce680 0x7f2f44071850 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7f2f3c009dc0 tx=0x7f2f3c01c970 comp rx=0 tx=0).stop 2023-11-26T07:24:48.074 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:48.074+0000 7f2f49a6b640 1 -- 172.21.15.143:0/259001189 shutdown_connections 2023-11-26T07:24:48.074 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:48.074+0000 7f2f49a6b640 1 --2- 172.21.15.143:0/259001189 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f2f440ce680 0x7f2f44071850 unknown :-1 s=CLOSED pgs=165 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:24:48.074 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:48.074+0000 7f2f49a6b640 1 -- 172.21.15.143:0/259001189 >> 172.21.15.143:0/259001189 conn(0x7f2f4406a890 msgr2=0x7f2f4406aca0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:24:48.074 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:48.074+0000 7f2f49a6b640 1 -- 172.21.15.143:0/259001189 shutdown_connections 2023-11-26T07:24:48.074 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:48.074+0000 7f2f49a6b640 1 -- 172.21.15.143:0/259001189 wait complete. 2023-11-26T07:24:48.075 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:48.074+0000 7f2f49a6b640 1 Processor -- start 2023-11-26T07:24:48.075 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:48.074+0000 7f2f49a6b640 1 -- start start 2023-11-26T07:24:48.077 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:48.074+0000 7f2f49a6b640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f2f440b2f80 0x7f2f4414ea00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:24:48.077 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:48.074+0000 7f2f49a6b640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f2f440ce680 0x7f2f44151580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:24:48.077 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:48.074+0000 7f2f49a6b640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f2f441596d0 0x7f2f440adad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:24:48.077 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:48.074+0000 7f2f49a6b640 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2f440aca30 con 0x7f2f440ce680 2023-11-26T07:24:48.077 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:48.074+0000 7f2f49a6b640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2f440acd10 con 0x7f2f440b2f80 2023-11-26T07:24:48.077 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:48.074+0000 7f2f49a6b640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2f440acff0 con 0x7f2f441596d0 2023-11-26T07:24:48.078 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:48.074+0000 7f2f42ffd640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f2f440ce680 0x7f2f44151580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:24:48.078 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:48.074+0000 7f2f42ffd640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f2f440ce680 0x7f2f44151580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:58622/0 (socket says 172.21.15.143:58622) 2023-11-26T07:24:48.078 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:48.074+0000 7f2f42ffd640 1 -- 172.21.15.143:0/4218207254 learned_addr learned my addr 172.21.15.143:0/4218207254 (peer_addr_for_me v2:172.21.15.143:0/0) 2023-11-26T07:24:48.078 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:48.074+0000 7f2f437fe640 1 --2- 172.21.15.143:0/4218207254 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f2f440b2f80 0x7f2f4414ea00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:24:48.078 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:48.078+0000 7f2f43fff640 1 --2- 172.21.15.143:0/4218207254 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f2f441596d0 0x7f2f440adad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:24:48.078 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:48.078+0000 7f2f42ffd640 1 -- 172.21.15.143:0/4218207254 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f2f441596d0 msgr2=0x7f2f440adad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:24:48.078 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:48.078+0000 7f2f42ffd640 1 --2- 172.21.15.143:0/4218207254 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f2f441596d0 0x7f2f440adad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:24:48.078 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:48.078+0000 7f2f42ffd640 1 -- 172.21.15.143:0/4218207254 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f2f440b2f80 msgr2=0x7f2f4414ea00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:24:48.078 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:48.078+0000 7f2f42ffd640 1 --2- 172.21.15.143:0/4218207254 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f2f440b2f80 0x7f2f4414ea00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:24:48.078 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:48.078+0000 7f2f42ffd640 1 -- 172.21.15.143:0/4218207254 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2f3c009a70 con 0x7f2f440ce680 2023-11-26T07:24:48.079 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:48.078+0000 7f2f42ffd640 1 --2- 172.21.15.143:0/4218207254 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f2f440ce680 0x7f2f44151580 secure :-1 s=READY pgs=1407 cs=0 l=1 rev1=1 crypto rx=0x7f2f3800c4d0 tx=0x7f2f3800c990 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:24:48.079 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:48.078+0000 7f2f2bfff640 1 -- 172.21.15.143:0/4218207254 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2f38005cf0 con 0x7f2f440ce680 2023-11-26T07:24:48.079 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:48.078+0000 7f2f2bfff640 1 -- 172.21.15.143:0/4218207254 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2f38005e50 con 0x7f2f440ce680 2023-11-26T07:24:48.079 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:48.078+0000 7f2f2bfff640 1 -- 172.21.15.143:0/4218207254 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2f380072b0 con 0x7f2f440ce680 2023-11-26T07:24:48.079 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:48.078+0000 7f2f49a6b640 1 -- 172.21.15.143:0/4218207254 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2f440ad2d0 con 0x7f2f440ce680 2023-11-26T07:24:48.080 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:48.078+0000 7f2f49a6b640 1 -- 172.21.15.143:0/4218207254 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2f4414dd60 con 0x7f2f440ce680 2023-11-26T07:24:48.080 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:48.078+0000 7f2f49a6b640 1 -- 172.21.15.143:0/4218207254 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2f44067680 con 0x7f2f440ce680 2023-11-26T07:24:48.080 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:48.078+0000 7f2f2bfff640 1 -- 172.21.15.143:0/4218207254 <== mon.0 v2:172.21.15.143:3300/0 4 ==== mgrmap(e 18) v1 ==== 86107+0+0 (secure 0 0 0) 0x7f2f38007410 con 0x7f2f440ce680 2023-11-26T07:24:48.081 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:48.078+0000 7f2f2bfff640 1 --2- 172.21.15.143:0/4218207254 >> [v2:172.21.15.188:6836/3401128553,v1:172.21.15.188:6837/3401128553] conn(0x7f2f30066740 0x7f2f30068c00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:24:48.081 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:48.078+0000 7f2f2bfff640 1 -- 172.21.15.143:0/4218207254 <== mon.0 v2:172.21.15.143:3300/0 5 ==== osd_map(103..103 src has 1..103) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f2f38083890 con 0x7f2f440ce680 2023-11-26T07:24:48.081 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:48.082+0000 7f2f437fe640 1 --2- 172.21.15.143:0/4218207254 >> [v2:172.21.15.188:6836/3401128553,v1:172.21.15.188:6837/3401128553] conn(0x7f2f30066740 0x7f2f30068c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:24:48.082 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:48.082+0000 7f2f437fe640 1 --2- 172.21.15.143:0/4218207254 >> [v2:172.21.15.188:6836/3401128553,v1:172.21.15.188:6837/3401128553] conn(0x7f2f30066740 0x7f2f30068c00 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7f2f340011f0 tx=0x7f2f34009040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:24:48.085 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:48.082+0000 7f2f2bfff640 1 -- 172.21.15.143:0/4218207254 <== mon.0 v2:172.21.15.143:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172649 (secure 0 0 0) 0x7f2f38050a30 con 0x7f2f440ce680 2023-11-26T07:24:48.246 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:48.246+0000 7f2f49a6b640 1 -- 172.21.15.143:0/4218207254 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f2f440ad780 con 0x7f2f440ce680 2023-11-26T07:24:48.247 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:48.246+0000 7f2f2bfff640 1 -- 172.21.15.143:0/4218207254 <== mon.0 v2:172.21.15.143:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v103) v1 ==== 74+0+16563 (secure 0 0 0) 0x7f2f38017020 con 0x7f2f440ce680 2023-11-26T07:24:48.248 INFO:teuthology.orchestra.run.smithi143.stdout: 2023-11-26T07:24:48.248 INFO:teuthology.orchestra.run.smithi143.stdout:{"epoch":103,"fsid":"f5033060-2469-4a07-95fe-2b76f5404bc1","created":"2023-11-26T07:17:28.613663+0000","modified":"2023-11-26T07:24:09.510299+0000","last_up_change":"2023-11-26T07:17:41.157813+0000","last_in_change":"2023-11-26T07:17:31.181658+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":17,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-11-26T07:17:41.758597+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":16,"pool_name":"cephfs_metadata","create_time":"2023-11-26T07:24:06.278132+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"102","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":17,"pool_name":"cephfs_data","create_time":"2023-11-26T07:24:07.294850+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"103","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"ebbee2c6-023e-44b9-9f1d-6f7bfd309730","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":101,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6817","nonce":639807369},{"type":"v1","addr":"172.21.15.143:6819","nonce":639807369}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6821","nonce":639807369},{"type":"v1","addr":"172.21.15.143:6823","nonce":639807369}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6830","nonce":639807369},{"type":"v1","addr":"172.21.15.143:6831","nonce":639807369}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6825","nonce":639807369},{"type":"v1","addr":"172.21.15.143:6828","nonce":639807369}]},"public_addr":"172.21.15.143:6819/639807369","cluster_addr":"172.21.15.143:6823/639807369","heartbeat_back_addr":"172.21.15.143:6831/639807369","heartbeat_front_addr":"172.21.15.143:6828/639807369","state":["exists","up"]},{"osd":1,"uuid":"fec4b8be-7122-49e9-9a19-5850a0700001","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":101,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6816","nonce":1144942530},{"type":"v1","addr":"172.21.15.143:6818","nonce":1144942530}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6820","nonce":1144942530},{"type":"v1","addr":"172.21.15.143:6822","nonce":1144942530}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6827","nonce":1144942530},{"type":"v1","addr":"172.21.15.143:6829","nonce":1144942530}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6824","nonce":1144942530},{"type":"v1","addr":"172.21.15.143:6826","nonce":1144942530}]},"public_addr":"172.21.15.143:6818/1144942530","cluster_addr":"172.21.15.143:6822/1144942530","heartbeat_back_addr":"172.21.15.143:6829/1144942530","heartbeat_front_addr":"172.21.15.143:6826/1144942530","state":["exists","up"]},{"osd":2,"uuid":"2888af5b-f933-46ca-8f95-6dd494ff43f6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":101,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6808","nonce":3909095186},{"type":"v1","addr":"172.21.15.143:6809","nonce":3909095186}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6810","nonce":3909095186},{"type":"v1","addr":"172.21.15.143:6811","nonce":3909095186}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6814","nonce":3909095186},{"type":"v1","addr":"172.21.15.143:6815","nonce":3909095186}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6812","nonce":3909095186},{"type":"v1","addr":"172.21.15.143:6813","nonce":3909095186}]},"public_addr":"172.21.15.143:6809/3909095186","cluster_addr":"172.21.15.143:6811/3909095186","heartbeat_back_addr":"172.21.15.143:6815/3909095186","heartbeat_front_addr":"172.21.15.143:6813/3909095186","state":["exists","up"]},{"osd":3,"uuid":"d1f52435-7bc1-4081-9550-9cd1ba55f549","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":101,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6800","nonce":3854210991},{"type":"v1","addr":"172.21.15.143:6801","nonce":3854210991}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6802","nonce":3854210991},{"type":"v1","addr":"172.21.15.143:6803","nonce":3854210991}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6806","nonce":3854210991},{"type":"v1","addr":"172.21.15.143:6807","nonce":3854210991}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6804","nonce":3854210991},{"type":"v1","addr":"172.21.15.143:6805","nonce":3854210991}]},"public_addr":"172.21.15.143:6801/3854210991","cluster_addr":"172.21.15.143:6803/3854210991","heartbeat_back_addr":"172.21.15.143:6807/3854210991","heartbeat_front_addr":"172.21.15.143:6805/3854210991","state":["exists","up"]},{"osd":4,"uuid":"e2ae8b6e-6e25-49b2-bcbf-aacce68f06a3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":101,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6800","nonce":3924606902},{"type":"v1","addr":"172.21.15.188:6801","nonce":3924606902}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6802","nonce":3924606902},{"type":"v1","addr":"172.21.15.188:6803","nonce":3924606902}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6806","nonce":3924606902},{"type":"v1","addr":"172.21.15.188:6807","nonce":3924606902}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6804","nonce":3924606902},{"type":"v1","addr":"172.21.15.188:6805","nonce":3924606902}]},"public_addr":"172.21.15.188:6801/3924606902","cluster_addr":"172.21.15.188:6803/3924606902","heartbeat_back_addr":"172.21.15.188:6807/3924606902","heartbeat_front_addr":"172.21.15.188:6805/3924606902","state":["exists","up"]},{"osd":5,"uuid":"3cec8a32-7698-4c36-8e89-e4ba11934f33","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":101,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6813","nonce":186841544},{"type":"v1","addr":"172.21.15.188:6815","nonce":186841544}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6817","nonce":186841544},{"type":"v1","addr":"172.21.15.188:6818","nonce":186841544}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6822","nonce":186841544},{"type":"v1","addr":"172.21.15.188:6823","nonce":186841544}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6820","nonce":186841544},{"type":"v1","addr":"172.21.15.188:6821","nonce":186841544}]},"public_addr":"172.21.15.188:6815/186841544","cluster_addr":"172.21.15.188:6818/186841544","heartbeat_back_addr":"172.21.15.188:6823/186841544","heartbeat_front_addr":"172.21.15.188:6821/186841544","state":["exists","up"]},{"osd":6,"uuid":"58a65d5d-e090-473a-81b4-ad38e85029f9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":101,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6808","nonce":2153499405},{"type":"v1","addr":"172.21.15.188:6809","nonce":2153499405}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6810","nonce":2153499405},{"type":"v1","addr":"172.21.15.188:6811","nonce":2153499405}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6816","nonce":2153499405},{"type":"v1","addr":"172.21.15.188:6819","nonce":2153499405}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6812","nonce":2153499405},{"type":"v1","addr":"172.21.15.188:6814","nonce":2153499405}]},"public_addr":"172.21.15.188:6809/2153499405","cluster_addr":"172.21.15.188:6811/2153499405","heartbeat_back_addr":"172.21.15.188:6819/2153499405","heartbeat_front_addr":"172.21.15.188:6814/2153499405","state":["exists","up"]},{"osd":7,"uuid":"ea5b7f70-3554-4cf8-ac30-f7c59b80678d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":101,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6824","nonce":2722096729},{"type":"v1","addr":"172.21.15.188:6825","nonce":2722096729}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6826","nonce":2722096729},{"type":"v1","addr":"172.21.15.188:6827","nonce":2722096729}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6830","nonce":2722096729},{"type":"v1","addr":"172.21.15.188:6831","nonce":2722096729}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6828","nonce":2722096729},{"type":"v1","addr":"172.21.15.188:6829","nonce":2722096729}]},"public_addr":"172.21.15.188:6825/2722096729","cluster_addr":"172.21.15.188:6827/2722096729","heartbeat_back_addr":"172.21.15.188:6831/2722096729","heartbeat_front_addr":"172.21.15.188:6829/2722096729","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-26T07:17:36.995032+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-26T07:17:36.932266+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-26T07:17:36.829304+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-26T07:17:36.804297+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-26T07:17:36.667892+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-26T07:17:36.964868+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-26T07:17:36.827332+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-26T07:17:36.893179+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-11-26T07:24:48.250 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:48.250+0000 7f2f49a6b640 1 -- 172.21.15.143:0/4218207254 >> [v2:172.21.15.188:6836/3401128553,v1:172.21.15.188:6837/3401128553] conn(0x7f2f30066740 msgr2=0x7f2f30068c00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:24:48.250 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:48.250+0000 7f2f49a6b640 1 --2- 172.21.15.143:0/4218207254 >> [v2:172.21.15.188:6836/3401128553,v1:172.21.15.188:6837/3401128553] conn(0x7f2f30066740 0x7f2f30068c00 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7f2f340011f0 tx=0x7f2f34009040 comp rx=0 tx=0).stop 2023-11-26T07:24:48.250 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:48.250+0000 7f2f49a6b640 1 -- 172.21.15.143:0/4218207254 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f2f440ce680 msgr2=0x7f2f44151580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:24:48.250 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:48.250+0000 7f2f49a6b640 1 --2- 172.21.15.143:0/4218207254 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f2f440ce680 0x7f2f44151580 secure :-1 s=READY pgs=1407 cs=0 l=1 rev1=1 crypto rx=0x7f2f3800c4d0 tx=0x7f2f3800c990 comp rx=0 tx=0).stop 2023-11-26T07:24:48.250 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:48.250+0000 7f2f49a6b640 1 -- 172.21.15.143:0/4218207254 shutdown_connections 2023-11-26T07:24:48.250 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:48.250+0000 7f2f49a6b640 1 --2- 172.21.15.143:0/4218207254 >> [v2:172.21.15.188:6836/3401128553,v1:172.21.15.188:6837/3401128553] conn(0x7f2f30066740 0x7f2f30068c00 unknown :-1 s=CLOSED pgs=167 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:24:48.250 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:48.250+0000 7f2f49a6b640 1 --2- 172.21.15.143:0/4218207254 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f2f441596d0 0x7f2f440adad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:24:48.250 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:48.250+0000 7f2f49a6b640 1 --2- 172.21.15.143:0/4218207254 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f2f440ce680 0x7f2f44151580 unknown :-1 s=CLOSED pgs=1407 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:24:48.250 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:48.250+0000 7f2f49a6b640 1 --2- 172.21.15.143:0/4218207254 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f2f440b2f80 0x7f2f4414ea00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:24:48.251 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:48.250+0000 7f2f49a6b640 1 -- 172.21.15.143:0/4218207254 >> 172.21.15.143:0/4218207254 conn(0x7f2f4406a890 msgr2=0x7f2f44150890 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:24:48.251 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:48.250+0000 7f2f49a6b640 1 -- 172.21.15.143:0/4218207254 shutdown_connections 2023-11-26T07:24:48.251 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:48.250+0000 7f2f49a6b640 1 -- 172.21.15.143:0/4218207254 wait complete. 2023-11-26T07:24:48.269 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-11-26T07:17:41.758597+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 16, 'pool_name': 'cephfs_metadata', 'create_time': '2023-11-26T07:24:06.278132+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '102', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 1.75, 'score_stable': 1.75, 'optimal_score': 1, 'raw_score_acting': 1.75, 'raw_score_stable': 1.75, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 17, 'pool_name': 'cephfs_data', 'create_time': '2023-11-26T07:24:07.294850+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '103', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.75, 'score_stable': 1.75, 'optimal_score': 1, 'raw_score_acting': 1.75, 'raw_score_stable': 1.75, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2023-11-26T07:24:48.269 DEBUG:teuthology.orchestra.run.smithi143:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-11-26T07:24:48.409 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:48.410+0000 7fc0e4a6b640 1 Processor -- start 2023-11-26T07:24:48.410 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:48.410+0000 7fc0e4a6b640 1 -- start start 2023-11-26T07:24:48.411 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:48.410+0000 7fc0e4a6b640 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7fc0e0145fa0 0x7fc0e0072420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:24:48.411 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:48.410+0000 7fc0e4a6b640 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7fc0e006a490 0x7fc0e006a8e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:24:48.411 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:48.410+0000 7fc0e4a6b640 1 -- --> v1:172.21.15.188:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc0e0146380 con 0x7fc0e006b190 2023-11-26T07:24:48.412 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:48.410+0000 7fc0e4a6b640 1 -- --> v2:172.21.15.143:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc0e0071480 con 0x7fc0e006a490 2023-11-26T07:24:48.412 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:48.410+0000 7fc0e4a6b640 1 -- --> v2:172.21.15.188:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc0e0072000 con 0x7fc0e0145fa0 2023-11-26T07:24:48.412 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:48.410+0000 7fc0ded9d640 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7fc0e006a490 0x7fc0e006a8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:24:48.413 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:48.410+0000 7fc0ded9d640 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7fc0e006a490 0x7fc0e006a8e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:58630/0 (socket says 172.21.15.143:58630) 2023-11-26T07:24:48.413 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:48.410+0000 7fc0ded9d640 1 -- 172.21.15.143:0/3159710315 learned_addr learned my addr 172.21.15.143:0/3159710315 (peer_addr_for_me v2:172.21.15.143:0/0) 2023-11-26T07:24:48.413 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:48.410+0000 7fc0de59c640 1 --2- 172.21.15.143:0/3159710315 >> v2:172.21.15.188:3300/0 conn(0x7fc0e0145fa0 0x7fc0e0072420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:24:48.413 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:48.410+0000 7fc0ded9d640 1 -- 172.21.15.143:0/3159710315 >> v2:172.21.15.188:3300/0 conn(0x7fc0e0145fa0 msgr2=0x7fc0e0072420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-26T07:24:48.414 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:48.410+0000 7fc0ded9d640 1 --2- 172.21.15.143:0/3159710315 >> v2:172.21.15.188:3300/0 conn(0x7fc0e0145fa0 0x7fc0e0072420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:24:48.414 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:48.410+0000 7fc0ded9d640 1 -- 172.21.15.143:0/3159710315 >> v1:172.21.15.188:6789/0 conn(0x7fc0e006b190 legacy=0x7fc0e0146530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:24:48.414 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:48.410+0000 7fc0ded9d640 1 -- 172.21.15.143:0/3159710315 --> v2:172.21.15.143:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc0e0146c40 con 0x7fc0e006a490 2023-11-26T07:24:48.414 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:48.414+0000 7fc0ded9d640 1 --2- 172.21.15.143:0/3159710315 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fc0e006a490 0x7fc0e006a8e0 secure :-1 s=READY pgs=1408 cs=0 l=1 rev1=1 crypto rx=0x7fc0e0072b60 tx=0x7fc0d801c9d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f4eaf677080df0c2 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:24:48.415 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:48.414+0000 7fc0ddd9b640 1 -- 172.21.15.143:0/3159710315 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc0d80a5690 con 0x7fc0e006a490 2023-11-26T07:24:48.415 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:48.414+0000 7fc0ddd9b640 1 -- 172.21.15.143:0/3159710315 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc0d80a57f0 con 0x7fc0e006a490 2023-11-26T07:24:48.415 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:48.414+0000 7fc0ddd9b640 1 -- 172.21.15.143:0/3159710315 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc0d80a5b80 con 0x7fc0e006a490 2023-11-26T07:24:48.415 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:48.414+0000 7fc0e4a6b640 1 -- 172.21.15.143:0/3159710315 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fc0e006a490 msgr2=0x7fc0e006a8e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:24:48.416 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:48.414+0000 7fc0e4a6b640 1 --2- 172.21.15.143:0/3159710315 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fc0e006a490 0x7fc0e006a8e0 secure :-1 s=READY pgs=1408 cs=0 l=1 rev1=1 crypto rx=0x7fc0e0072b60 tx=0x7fc0d801c9d0 comp rx=0 tx=0).stop 2023-11-26T07:24:48.416 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:48.414+0000 7fc0e4a6b640 1 -- 172.21.15.143:0/3159710315 shutdown_connections 2023-11-26T07:24:48.416 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:48.414+0000 7fc0e4a6b640 1 --2- 172.21.15.143:0/3159710315 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fc0e006a490 0x7fc0e006a8e0 unknown :-1 s=CLOSED pgs=1408 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:24:48.416 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:48.414+0000 7fc0e4a6b640 1 --2- 172.21.15.143:0/3159710315 >> v2:172.21.15.188:3300/0 conn(0x7fc0e0145fa0 0x7fc0e0072420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:24:48.416 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:48.414+0000 7fc0e4a6b640 1 -- 172.21.15.143:0/3159710315 >> 172.21.15.143:0/3159710315 conn(0x7fc0e00cff00 msgr2=0x7fc0e00d0310 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:24:48.416 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:48.414+0000 7fc0e4a6b640 1 -- 172.21.15.143:0/3159710315 shutdown_connections 2023-11-26T07:24:48.416 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:48.414+0000 7fc0e4a6b640 1 -- 172.21.15.143:0/3159710315 wait complete. 2023-11-26T07:24:48.417 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:48.418+0000 7fc0e4a6b640 1 Processor -- start 2023-11-26T07:24:48.417 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:48.418+0000 7fc0e4a6b640 1 -- start start 2023-11-26T07:24:48.419 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:48.418+0000 7fc0e4a6b640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fc0e006a490 0x7fc0e0140ba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:24:48.419 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:48.418+0000 7fc0e4a6b640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fc0e006b190 0x7fc0e013b980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:24:48.419 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:48.418+0000 7fc0e4a6b640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fc0e0145fa0 0x7fc0e0142500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:24:48.419 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:48.418+0000 7fc0e4a6b640 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc0e01399e0 con 0x7fc0e006a490 2023-11-26T07:24:48.419 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:48.418+0000 7fc0e4a6b640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc0e01603d0 con 0x7fc0e0145fa0 2023-11-26T07:24:48.419 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:48.418+0000 7fc0e4a6b640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc0e01410e0 con 0x7fc0e006b190 2023-11-26T07:24:48.419 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:48.418+0000 7fc0de59c640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fc0e006a490 0x7fc0e0140ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:24:48.419 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:48.418+0000 7fc0de59c640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fc0e006a490 0x7fc0e0140ba0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:58642/0 (socket says 172.21.15.143:58642) 2023-11-26T07:24:48.419 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:48.418+0000 7fc0de59c640 1 -- 172.21.15.143:0/2033631122 learned_addr learned my addr 172.21.15.143:0/2033631122 (peer_addr_for_me v2:172.21.15.143:0/0) 2023-11-26T07:24:48.419 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:48.418+0000 7fc0d7fff640 1 --2- 172.21.15.143:0/2033631122 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fc0e006b190 0x7fc0e013b980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:24:48.420 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:48.418+0000 7fc0ded9d640 1 --2- 172.21.15.143:0/2033631122 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fc0e0145fa0 0x7fc0e0142500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:24:48.420 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:48.418+0000 7fc0de59c640 1 -- 172.21.15.143:0/2033631122 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fc0e006b190 msgr2=0x7fc0e013b980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:24:48.420 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:48.418+0000 7fc0de59c640 1 --2- 172.21.15.143:0/2033631122 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fc0e006b190 0x7fc0e013b980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:24:48.420 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:48.418+0000 7fc0de59c640 1 -- 172.21.15.143:0/2033631122 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fc0e0145fa0 msgr2=0x7fc0e0142500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:24:48.420 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:48.418+0000 7fc0de59c640 1 --2- 172.21.15.143:0/2033631122 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fc0e0145fa0 0x7fc0e0142500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:24:48.420 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:48.418+0000 7fc0de59c640 1 -- 172.21.15.143:0/2033631122 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc0d8009b30 con 0x7fc0e006a490 2023-11-26T07:24:48.421 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:48.418+0000 7fc0de59c640 1 --2- 172.21.15.143:0/2033631122 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fc0e006a490 0x7fc0e0140ba0 secure :-1 s=READY pgs=1409 cs=0 l=1 rev1=1 crypto rx=0x7fc0d000e930 tx=0x7fc0d000edf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:24:48.421 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:48.418+0000 7fc0d6ffd640 1 -- 172.21.15.143:0/2033631122 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc0d0009690 con 0x7fc0e006a490 2023-11-26T07:24:48.421 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:48.418+0000 7fc0d6ffd640 1 -- 172.21.15.143:0/2033631122 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc0d000c9a0 con 0x7fc0e006a490 2023-11-26T07:24:48.421 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:48.418+0000 7fc0d6ffd640 1 -- 172.21.15.143:0/2033631122 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc0d000cd10 con 0x7fc0e006a490 2023-11-26T07:24:48.421 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:48.418+0000 7fc0e4a6b640 1 -- 172.21.15.143:0/2033631122 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc0e013bec0 con 0x7fc0e006a490 2023-11-26T07:24:48.421 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:48.418+0000 7fc0e4a6b640 1 -- 172.21.15.143:0/2033631122 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc0e0139320 con 0x7fc0e006a490 2023-11-26T07:24:48.421 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:48.418+0000 7fc0e4a6b640 1 -- 172.21.15.143:0/2033631122 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc0e0139c50 con 0x7fc0e006a490 2023-11-26T07:24:48.422 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:48.422+0000 7fc0d6ffd640 1 -- 172.21.15.143:0/2033631122 <== mon.0 v2:172.21.15.143:3300/0 4 ==== mgrmap(e 18) v1 ==== 86107+0+0 (secure 0 0 0) 0x7fc0d0013470 con 0x7fc0e006a490 2023-11-26T07:24:48.423 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:48.422+0000 7fc0d6ffd640 1 --2- 172.21.15.143:0/2033631122 >> [v2:172.21.15.188:6836/3401128553,v1:172.21.15.188:6837/3401128553] conn(0x7fc0c0066740 0x7fc0c0068c00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:24:48.423 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:48.422+0000 7fc0d6ffd640 1 -- 172.21.15.143:0/2033631122 <== mon.0 v2:172.21.15.143:3300/0 5 ==== osd_map(103..103 src has 1..103) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fc0d00843a0 con 0x7fc0e006a490 2023-11-26T07:24:48.423 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:48.422+0000 7fc0d7fff640 1 --2- 172.21.15.143:0/2033631122 >> [v2:172.21.15.188:6836/3401128553,v1:172.21.15.188:6837/3401128553] conn(0x7fc0c0066740 0x7fc0c0068c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:24:48.423 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:48.422+0000 7fc0d7fff640 1 --2- 172.21.15.143:0/2033631122 >> [v2:172.21.15.188:6836/3401128553,v1:172.21.15.188:6837/3401128553] conn(0x7fc0c0066740 0x7fc0c0068c00 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7fc0e00ad750 tx=0x7fc0c8009040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:24:48.427 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:48.426+0000 7fc0d6ffd640 1 -- 172.21.15.143:0/2033631122 <== mon.0 v2:172.21.15.143:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172649 (secure 0 0 0) 0x7fc0d00516d0 con 0x7fc0e006a490 2023-11-26T07:24:48.581 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:48.582+0000 7fc0e4a6b640 1 -- 172.21.15.143:0/2033631122 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7fc0e0067680 con 0x7fc0e006a490 2023-11-26T07:24:48.582 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:48.582+0000 7fc0d6ffd640 1 -- 172.21.15.143:0/2033631122 <== mon.0 v2:172.21.15.143:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v103) v1 ==== 93+0+10 (secure 0 0 0) 0x7fc0d005b030 con 0x7fc0e006a490 2023-11-26T07:24:48.582 INFO:teuthology.orchestra.run.smithi143.stdout:pg_num: 1 2023-11-26T07:24:48.584 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:48.582+0000 7fc0e4a6b640 1 -- 172.21.15.143:0/2033631122 >> [v2:172.21.15.188:6836/3401128553,v1:172.21.15.188:6837/3401128553] conn(0x7fc0c0066740 msgr2=0x7fc0c0068c00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:24:48.584 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:48.582+0000 7fc0e4a6b640 1 --2- 172.21.15.143:0/2033631122 >> [v2:172.21.15.188:6836/3401128553,v1:172.21.15.188:6837/3401128553] conn(0x7fc0c0066740 0x7fc0c0068c00 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7fc0e00ad750 tx=0x7fc0c8009040 comp rx=0 tx=0).stop 2023-11-26T07:24:48.584 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:48.582+0000 7fc0e4a6b640 1 -- 172.21.15.143:0/2033631122 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fc0e006a490 msgr2=0x7fc0e0140ba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:24:48.584 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:48.582+0000 7fc0e4a6b640 1 --2- 172.21.15.143:0/2033631122 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fc0e006a490 0x7fc0e0140ba0 secure :-1 s=READY pgs=1409 cs=0 l=1 rev1=1 crypto rx=0x7fc0d000e930 tx=0x7fc0d000edf0 comp rx=0 tx=0).stop 2023-11-26T07:24:48.584 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:48.582+0000 7fc0e4a6b640 1 -- 172.21.15.143:0/2033631122 shutdown_connections 2023-11-26T07:24:48.585 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:48.582+0000 7fc0e4a6b640 1 --2- 172.21.15.143:0/2033631122 >> [v2:172.21.15.188:6836/3401128553,v1:172.21.15.188:6837/3401128553] conn(0x7fc0c0066740 0x7fc0c0068c00 unknown :-1 s=CLOSED pgs=168 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:24:48.585 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:48.582+0000 7fc0e4a6b640 1 --2- 172.21.15.143:0/2033631122 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fc0e0145fa0 0x7fc0e0142500 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:24:48.585 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:48.582+0000 7fc0e4a6b640 1 --2- 172.21.15.143:0/2033631122 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fc0e006b190 0x7fc0e013b980 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:24:48.585 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:48.582+0000 7fc0e4a6b640 1 --2- 172.21.15.143:0/2033631122 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fc0e006a490 0x7fc0e0140ba0 unknown :-1 s=CLOSED pgs=1409 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:24:48.585 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:48.582+0000 7fc0e4a6b640 1 -- 172.21.15.143:0/2033631122 >> 172.21.15.143:0/2033631122 conn(0x7fc0e00cff00 msgr2=0x7fc0e0146510 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:24:48.585 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:48.582+0000 7fc0e4a6b640 1 -- 172.21.15.143:0/2033631122 shutdown_connections 2023-11-26T07:24:48.585 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:48.582+0000 7fc0e4a6b640 1 -- 172.21.15.143:0/2033631122 wait complete. 2023-11-26T07:24:48.603 DEBUG:teuthology.orchestra.run.smithi143:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-11-26T07:24:48.746 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:48.746+0000 7f316c679640 1 Processor -- start 2023-11-26T07:24:48.746 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:48.746+0000 7f316c679640 1 -- start start 2023-11-26T07:24:48.747 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:48.746+0000 7f316c679640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f3164143ac0 0x7f3164070370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:24:48.748 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:48.746+0000 7f316c679640 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7f31640cf990 0x7f31640cd1b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:24:48.748 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:48.746+0000 7f316c679640 1 -- --> v1:172.21.15.143:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f31640cc730 con 0x7f31640708b0 2023-11-26T07:24:48.748 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:48.746+0000 7f316c679640 1 -- --> v2:172.21.15.188:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3164143ea0 con 0x7f31640cf990 2023-11-26T07:24:48.748 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:48.746+0000 7f316c679640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3164070ee0 con 0x7f3164143ac0 2023-11-26T07:24:48.748 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:48.746+0000 7f3169c14640 1 --1- >> v1:172.21.15.143:6789/0 conn(0x7f31640708b0 0x7f31640cf280 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.143:6789/0 says I am v1:172.21.15.143:49704/0 (socket says 172.21.15.143:49704) 2023-11-26T07:24:48.748 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:48.746+0000 7f3169c14640 1 -- 172.21.15.143:0/805651263 learned_addr learned my addr 172.21.15.143:0/805651263 (peer_addr_for_me v1:172.21.15.143:0/0) 2023-11-26T07:24:48.750 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:48.746+0000 7f316a415640 1 --2- 172.21.15.143:0/805651263 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f3164143ac0 0x7f3164070370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:24:48.750 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:48.746+0000 7f316ac16640 1 --2- 172.21.15.143:0/805651263 >> v2:172.21.15.188:3300/0 conn(0x7f31640cf990 0x7f31640cd1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:24:48.750 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:48.746+0000 7f3169413640 1 -- 172.21.15.143:0/805651263 <== mon.0 v1:172.21.15.143:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4120594907 0 0) 0x7f31640cc730 con 0x7f31640708b0 2023-11-26T07:24:48.750 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:48.746+0000 7f3169413640 1 -- 172.21.15.143:0/805651263 --> v1:172.21.15.143:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f3150003530 con 0x7f31640708b0 2023-11-26T07:24:48.750 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:48.746+0000 7f3169413640 1 -- 172.21.15.143:0/805651263 <== mon.0 v1:172.21.15.143:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2518457507 0 0) 0x7f3150003530 con 0x7f31640708b0 2023-11-26T07:24:48.750 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:48.746+0000 7f3169413640 1 -- 172.21.15.143:0/805651263 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f3164143ac0 msgr2=0x7f3164070370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-26T07:24:48.750 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:48.746+0000 7f3169413640 1 --2- 172.21.15.143:0/805651263 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f3164143ac0 0x7f3164070370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:24:48.751 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:48.746+0000 7f3169413640 1 -- 172.21.15.143:0/805651263 >> v2:172.21.15.188:3300/0 conn(0x7f31640cf990 msgr2=0x7f31640cd1b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-26T07:24:48.751 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:48.746+0000 7f3169413640 1 --2- 172.21.15.143:0/805651263 >> v2:172.21.15.188:3300/0 conn(0x7f31640cf990 0x7f31640cd1b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:24:48.751 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:48.746+0000 7f3169413640 1 -- 172.21.15.143:0/805651263 --> v1:172.21.15.143:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f31640711c0 con 0x7f31640708b0 2023-11-26T07:24:48.751 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:48.746+0000 7f3169413640 1 -- 172.21.15.143:0/805651263 <== mon.0 v1:172.21.15.143:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2137882523 0 0) 0x7f31540016a0 con 0x7f31640708b0 2023-11-26T07:24:48.751 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:48.746+0000 7f3169413640 1 -- 172.21.15.143:0/805651263 >> v1:172.21.15.143:6789/0 conn(0x7f31640708b0 legacy=0x7f31640cf280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:24:48.751 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:48.746+0000 7f3169413640 1 --2- 172.21.15.143:0/805651263 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f3164143ac0 0x7f3150003980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:24:48.751 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:48.746+0000 7f3169413640 1 --2- 172.21.15.143:0/805651263 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f3150003f50 0x7f31500047d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:24:48.751 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:48.746+0000 7f3169413640 1 --2- 172.21.15.143:0/805651263 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f3150004d10 0x7f3150005190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:24:48.752 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:48.746+0000 7f3169413640 1 -- 172.21.15.143:0/805651263 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3164070ee0 con 0x7f3150004d10 2023-11-26T07:24:48.752 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:48.746+0000 7f3169413640 1 -- 172.21.15.143:0/805651263 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f315000b990 con 0x7f3150003f50 2023-11-26T07:24:48.752 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:48.746+0000 7f3169413640 1 -- 172.21.15.143:0/805651263 --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f315000bad0 con 0x7f3164143ac0 2023-11-26T07:24:48.752 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:48.746+0000 7f316a415640 1 --2- 172.21.15.143:0/805651263 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f3164143ac0 0x7f3150003980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:24:48.752 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:48.746+0000 7f316ac16640 1 --2- 172.21.15.143:0/805651263 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f3150004d10 0x7f3150005190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:24:48.752 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:48.746+0000 7f3169c14640 1 --2- 172.21.15.143:0/805651263 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f3150003f50 0x7f31500047d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:24:48.752 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:48.750+0000 7f316a415640 1 -- 172.21.15.143:0/805651263 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f3150003f50 msgr2=0x7f31500047d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-26T07:24:48.752 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:48.750+0000 7f316a415640 1 --2- 172.21.15.143:0/805651263 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f3150003f50 0x7f31500047d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:24:48.752 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:48.750+0000 7f316a415640 1 -- 172.21.15.143:0/805651263 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f3150004d10 msgr2=0x7f3150005190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-26T07:24:48.752 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:48.750+0000 7f316a415640 1 --2- 172.21.15.143:0/805651263 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f3150004d10 0x7f3150005190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:24:48.752 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:48.750+0000 7f316a415640 1 -- 172.21.15.143:0/805651263 --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3150003530 con 0x7f3164143ac0 2023-11-26T07:24:48.753 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:48.750+0000 7f3169c14640 1 --2- 172.21.15.143:0/805651263 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f3150003f50 0x7f31500047d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-26T07:24:48.753 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:48.750+0000 7f316a415640 1 --2- 172.21.15.143:0/805651263 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f3164143ac0 0x7f3150003980 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7f3160009b10 tx=0x7f316001ed00 comp rx=0 tx=0).ready entity=mon.2 client_cookie=81937a166c1cdcd0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:24:48.754 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:48.750+0000 7f3169413640 1 -- 172.21.15.143:0/805651263 <== mon.2 v2:172.21.15.188:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f31600a7b00 con 0x7f3164143ac0 2023-11-26T07:24:48.754 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:48.750+0000 7f3169413640 1 -- 172.21.15.143:0/805651263 <== mon.2 v2:172.21.15.188:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f31600a7c60 con 0x7f3164143ac0 2023-11-26T07:24:48.754 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:48.750+0000 7f3169413640 1 -- 172.21.15.143:0/805651263 <== mon.2 v2:172.21.15.188:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f31600af440 con 0x7f3164143ac0 2023-11-26T07:24:48.754 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:48.754+0000 7f316c679640 1 -- 172.21.15.143:0/805651263 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f3164143ac0 msgr2=0x7f3150003980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:24:48.754 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:48.754+0000 7f316c679640 1 --2- 172.21.15.143:0/805651263 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f3164143ac0 0x7f3150003980 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7f3160009b10 tx=0x7f316001ed00 comp rx=0 tx=0).stop 2023-11-26T07:24:48.754 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:48.754+0000 7f316ac16640 1 -- 172.21.15.143:0/805651263 reap_dead start 2023-11-26T07:24:48.754 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:48.754+0000 7f316c679640 1 -- 172.21.15.143:0/805651263 shutdown_connections 2023-11-26T07:24:48.755 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:48.754+0000 7f316c679640 1 -- 172.21.15.143:0/805651263 >> 172.21.15.143:0/805651263 conn(0x7f316406a890 msgr2=0x7f316406aca0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:24:48.755 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:48.754+0000 7f316c679640 1 -- 172.21.15.143:0/805651263 shutdown_connections 2023-11-26T07:24:48.755 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:48.754+0000 7f316c679640 1 -- 172.21.15.143:0/805651263 wait complete. 2023-11-26T07:24:48.755 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:48.754+0000 7f316c679640 1 Processor -- start 2023-11-26T07:24:48.755 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:48.754+0000 7f316c679640 1 -- start start 2023-11-26T07:24:48.756 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:48.754+0000 7f316c679640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f3164143ac0 0x7f316415c160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:24:48.756 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:48.754+0000 7f316c679640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f31640b0ff0 0x7f31640b1470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:24:48.757 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:48.754+0000 7f316c679640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f31640aecd0 0x7f31640af150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:24:48.757 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:48.754+0000 7f316c679640 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f31640b0610 con 0x7f31640b0ff0 2023-11-26T07:24:48.757 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:48.754+0000 7f316c679640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f31640afa90 con 0x7f31640aecd0 2023-11-26T07:24:48.757 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:48.754+0000 7f316c679640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f31640afd70 con 0x7f3164143ac0 2023-11-26T07:24:48.757 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:48.754+0000 7f3169c14640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f31640b0ff0 0x7f31640b1470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:24:48.757 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:48.754+0000 7f3169c14640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f31640b0ff0 0x7f31640b1470 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:58650/0 (socket says 172.21.15.143:58650) 2023-11-26T07:24:48.757 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:48.754+0000 7f3169c14640 1 -- 172.21.15.143:0/2910919576 learned_addr learned my addr 172.21.15.143:0/2910919576 (peer_addr_for_me v2:172.21.15.143:0/0) 2023-11-26T07:24:48.757 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:48.754+0000 7f316a415640 1 --2- 172.21.15.143:0/2910919576 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f3164143ac0 0x7f316415c160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:24:48.758 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:48.754+0000 7f316ac16640 1 --2- 172.21.15.143:0/2910919576 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f31640aecd0 0x7f31640af150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:24:48.759 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:48.754+0000 7f3169c14640 1 -- 172.21.15.143:0/2910919576 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f3164143ac0 msgr2=0x7f316415c160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:24:48.759 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:48.758+0000 7f3169c14640 1 --2- 172.21.15.143:0/2910919576 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f3164143ac0 0x7f316415c160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:24:48.759 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:48.758+0000 7f3169c14640 1 -- 172.21.15.143:0/2910919576 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f31640aecd0 msgr2=0x7f31640af150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:24:48.759 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:48.758+0000 7f3169c14640 1 --2- 172.21.15.143:0/2910919576 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f31640aecd0 0x7f31640af150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:24:48.759 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:48.758+0000 7f3169c14640 1 -- 172.21.15.143:0/2910919576 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f31640b0050 con 0x7f31640b0ff0 2023-11-26T07:24:48.759 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:48.758+0000 7f3169c14640 1 --2- 172.21.15.143:0/2910919576 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f31640b0ff0 0x7f31640b1470 secure :-1 s=READY pgs=1411 cs=0 l=1 rev1=1 crypto rx=0x7f3154007310 tx=0x7f31540077d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:24:48.760 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:48.758+0000 7f315affd640 1 -- 172.21.15.143:0/2910919576 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f315400fa80 con 0x7f31640b0ff0 2023-11-26T07:24:48.760 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:48.758+0000 7f315affd640 1 -- 172.21.15.143:0/2910919576 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f315400fbe0 con 0x7f31640b0ff0 2023-11-26T07:24:48.760 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:48.758+0000 7f316c679640 1 -- 172.21.15.143:0/2910919576 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f31640b0330 con 0x7f31640b0ff0 2023-11-26T07:24:48.760 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:48.758+0000 7f315affd640 1 -- 172.21.15.143:0/2910919576 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f31540133f0 con 0x7f31640b0ff0 2023-11-26T07:24:48.760 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:48.758+0000 7f316c679640 1 -- 172.21.15.143:0/2910919576 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f31640af690 con 0x7f31640b0ff0 2023-11-26T07:24:48.760 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:48.758+0000 7f316c679640 1 -- 172.21.15.143:0/2910919576 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3164067680 con 0x7f31640b0ff0 2023-11-26T07:24:48.761 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:48.758+0000 7f315affd640 1 -- 172.21.15.143:0/2910919576 <== mon.0 v2:172.21.15.143:3300/0 4 ==== mgrmap(e 18) v1 ==== 86107+0+0 (secure 0 0 0) 0x7f3154013550 con 0x7f31640b0ff0 2023-11-26T07:24:48.761 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:48.758+0000 7f315affd640 1 --2- 172.21.15.143:0/2910919576 >> [v2:172.21.15.188:6836/3401128553,v1:172.21.15.188:6837/3401128553] conn(0x7f3148066740 0x7f3148068c00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:24:48.761 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:48.758+0000 7f315affd640 1 -- 172.21.15.143:0/2910919576 <== mon.0 v2:172.21.15.143:3300/0 5 ==== osd_map(103..103 src has 1..103) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f3154084470 con 0x7f31640b0ff0 2023-11-26T07:24:48.761 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:48.758+0000 7f316a415640 1 --2- 172.21.15.143:0/2910919576 >> [v2:172.21.15.188:6836/3401128553,v1:172.21.15.188:6837/3401128553] conn(0x7f3148066740 0x7f3148068c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:24:48.761 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:48.762+0000 7f316a415640 1 --2- 172.21.15.143:0/2910919576 >> [v2:172.21.15.188:6836/3401128553,v1:172.21.15.188:6837/3401128553] conn(0x7f3148066740 0x7f3148068c00 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7f31600b1000 tx=0x7f31600046d0 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:24:48.764 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:48.762+0000 7f315affd640 1 -- 172.21.15.143:0/2910919576 <== mon.0 v2:172.21.15.143:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172649 (secure 0 0 0) 0x7f31540516a0 con 0x7f31640b0ff0 2023-11-26T07:24:48.913 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:48.914+0000 7f316c679640 1 -- 172.21.15.143:0/2910919576 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f3164067fc0 con 0x7f31640b0ff0 2023-11-26T07:24:48.914 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:48.914+0000 7f315affd640 1 -- 172.21.15.143:0/2910919576 <== mon.0 v2:172.21.15.143:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v103) v1 ==== 104+0+11 (secure 0 0 0) 0x7f3154054cc0 con 0x7f31640b0ff0 2023-11-26T07:24:48.914 INFO:teuthology.orchestra.run.smithi143.stdout:pg_num: 32 2023-11-26T07:24:48.916 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:48.914+0000 7f316c679640 1 -- 172.21.15.143:0/2910919576 >> [v2:172.21.15.188:6836/3401128553,v1:172.21.15.188:6837/3401128553] conn(0x7f3148066740 msgr2=0x7f3148068c00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:24:48.916 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:48.914+0000 7f316c679640 1 --2- 172.21.15.143:0/2910919576 >> [v2:172.21.15.188:6836/3401128553,v1:172.21.15.188:6837/3401128553] conn(0x7f3148066740 0x7f3148068c00 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7f31600b1000 tx=0x7f31600046d0 comp rx=0 tx=0).stop 2023-11-26T07:24:48.916 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:48.914+0000 7f316c679640 1 -- 172.21.15.143:0/2910919576 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f31640b0ff0 msgr2=0x7f31640b1470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:24:48.916 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:48.914+0000 7f316c679640 1 --2- 172.21.15.143:0/2910919576 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f31640b0ff0 0x7f31640b1470 secure :-1 s=READY pgs=1411 cs=0 l=1 rev1=1 crypto rx=0x7f3154007310 tx=0x7f31540077d0 comp rx=0 tx=0).stop 2023-11-26T07:24:48.916 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:48.914+0000 7f316c679640 1 -- 172.21.15.143:0/2910919576 shutdown_connections 2023-11-26T07:24:48.916 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:48.914+0000 7f316c679640 1 --2- 172.21.15.143:0/2910919576 >> [v2:172.21.15.188:6836/3401128553,v1:172.21.15.188:6837/3401128553] conn(0x7f3148066740 0x7f3148068c00 unknown :-1 s=CLOSED pgs=169 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:24:48.916 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:48.914+0000 7f316c679640 1 --2- 172.21.15.143:0/2910919576 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f31640aecd0 0x7f31640af150 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:24:48.917 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:48.914+0000 7f316c679640 1 --2- 172.21.15.143:0/2910919576 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f31640b0ff0 0x7f31640b1470 unknown :-1 s=CLOSED pgs=1411 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:24:48.917 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:48.914+0000 7f316c679640 1 --2- 172.21.15.143:0/2910919576 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f3164143ac0 0x7f316415c160 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:24:48.917 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:48.914+0000 7f316c679640 1 -- 172.21.15.143:0/2910919576 >> 172.21.15.143:0/2910919576 conn(0x7f316406a890 msgr2=0x7f31640b1ce0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:24:48.917 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:48.914+0000 7f316c679640 1 -- 172.21.15.143:0/2910919576 shutdown_connections 2023-11-26T07:24:48.917 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:48.914+0000 7f316c679640 1 -- 172.21.15.143:0/2910919576 wait complete. 2023-11-26T07:24:48.935 DEBUG:teuthology.orchestra.run.smithi143:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-11-26T07:24:49.076 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:49.074+0000 7f6d7ce59640 1 Processor -- start 2023-11-26T07:24:49.076 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:49.074+0000 7f6d7ce59640 1 -- start start 2023-11-26T07:24:49.078 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:49.078+0000 7f6d7ce59640 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7f6d7806d820 0x7f6d78164500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:24:49.078 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:49.078+0000 7f6d7ce59640 1 -- --> v1:172.21.15.143:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6d780b1e10 con 0x7f6d78153b90 2023-11-26T07:24:49.078 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:49.078+0000 7f6d7ce59640 1 -- --> v1:172.21.15.188:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6d780b4160 con 0x7f6d780b2b40 2023-11-26T07:24:49.078 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:49.078+0000 7f6d7ce59640 1 -- --> v2:172.21.15.143:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6d780cea00 con 0x7f6d7806d820 2023-11-26T07:24:49.078 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:49.078+0000 7f6d777fe640 1 --1- >> v1:172.21.15.143:6789/0 conn(0x7f6d78153b90 0x7f6d7815a600 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.143:6789/0 says I am v1:172.21.15.143:49706/0 (socket says 172.21.15.143:49706) 2023-11-26T07:24:49.078 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:49.078+0000 7f6d777fe640 1 -- 172.21.15.143:0/2824557796 learned_addr learned my addr 172.21.15.143:0/2824557796 (peer_addr_for_me v1:172.21.15.143:0/0) 2023-11-26T07:24:49.078 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:49.078+0000 7f6d77fff640 1 --2- 172.21.15.143:0/2824557796 >> v2:172.21.15.143:3300/0 conn(0x7f6d7806d820 0x7f6d78164500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:24:49.078 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:49.078+0000 7f6d76ffd640 1 -- 172.21.15.143:0/2824557796 <== mon.0 v1:172.21.15.143:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3065498522 0 0) 0x7f6d780b1e10 con 0x7f6d78153b90 2023-11-26T07:24:49.078 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:49.078+0000 7f6d77fff640 1 -- 172.21.15.143:0/2824557796 >> v1:172.21.15.188:6789/0 conn(0x7f6d780b2b40 legacy=0x7f6d7806d110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:24:49.079 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:49.078+0000 7f6d77fff640 1 -- 172.21.15.143:0/2824557796 >> v1:172.21.15.143:6789/0 conn(0x7f6d78153b90 legacy=0x7f6d7815a600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:24:49.079 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:49.078+0000 7f6d77fff640 1 -- 172.21.15.143:0/2824557796 --> v2:172.21.15.143:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6d780b2060 con 0x7f6d7806d820 2023-11-26T07:24:49.081 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:49.078+0000 7f6d77fff640 1 --2- 172.21.15.143:0/2824557796 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f6d7806d820 0x7f6d78164500 secure :-1 s=READY pgs=1413 cs=0 l=1 rev1=1 crypto rx=0x7f6d7000aea0 tx=0x7f6d7001da20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f516c467ff19cb99 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:24:49.081 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:49.078+0000 7f6d76ffd640 1 -- 172.21.15.143:0/2824557796 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6d7000a2d0 con 0x7f6d7806d820 2023-11-26T07:24:49.081 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:49.078+0000 7f6d76ffd640 1 -- 172.21.15.143:0/2824557796 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6d700a6860 con 0x7f6d7806d820 2023-11-26T07:24:49.081 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:49.078+0000 7f6d76ffd640 1 -- 172.21.15.143:0/2824557796 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6d700a6bf0 con 0x7f6d7806d820 2023-11-26T07:24:49.082 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:49.082+0000 7f6d7ce59640 1 -- 172.21.15.143:0/2824557796 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f6d7806d820 msgr2=0x7f6d78164500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:24:49.082 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:49.082+0000 7f6d7ce59640 1 --2- 172.21.15.143:0/2824557796 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f6d7806d820 0x7f6d78164500 secure :-1 s=READY pgs=1413 cs=0 l=1 rev1=1 crypto rx=0x7f6d7000aea0 tx=0x7f6d7001da20 comp rx=0 tx=0).stop 2023-11-26T07:24:49.082 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:49.082+0000 7f6d7ce59640 1 -- 172.21.15.143:0/2824557796 shutdown_connections 2023-11-26T07:24:49.082 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:49.082+0000 7f6d7ce59640 1 --2- 172.21.15.143:0/2824557796 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f6d7806d820 0x7f6d78164500 unknown :-1 s=CLOSED pgs=1413 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:24:49.082 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:49.082+0000 7f6d7ce59640 1 -- 172.21.15.143:0/2824557796 >> 172.21.15.143:0/2824557796 conn(0x7f6d7806a890 msgr2=0x7f6d7806aca0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:24:49.083 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:49.082+0000 7f6d7ce59640 1 -- 172.21.15.143:0/2824557796 shutdown_connections 2023-11-26T07:24:49.083 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:49.082+0000 7f6d7ce59640 1 -- 172.21.15.143:0/2824557796 wait complete. 2023-11-26T07:24:49.083 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:49.082+0000 7f6d7ce59640 1 Processor -- start 2023-11-26T07:24:49.083 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:49.082+0000 7f6d7ce59640 1 -- start start 2023-11-26T07:24:49.085 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:49.082+0000 7f6d7ce59640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f6d7806d820 0x7f6d780affd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:24:49.085 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:49.082+0000 7f6d7ce59640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f6d780b2b40 0x7f6d7813b1e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:24:49.085 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:49.082+0000 7f6d7ce59640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f6d78153b90 0x7f6d7813b720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:24:49.086 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:49.082+0000 7f6d7ce59640 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6d7814ee30 con 0x7f6d78153b90 2023-11-26T07:24:49.086 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:49.082+0000 7f6d7ce59640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6d7814e1f0 con 0x7f6d780b2b40 2023-11-26T07:24:49.086 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:49.082+0000 7f6d7ce59640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6d7814e490 con 0x7f6d7806d820 2023-11-26T07:24:49.086 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:49.082+0000 7f6d77fff640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f6d78153b90 0x7f6d7813b720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:24:49.086 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:49.082+0000 7f6d77fff640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f6d78153b90 0x7f6d7813b720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:58670/0 (socket says 172.21.15.143:58670) 2023-11-26T07:24:49.086 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:49.082+0000 7f6d77fff640 1 -- 172.21.15.143:0/2514688548 learned_addr learned my addr 172.21.15.143:0/2514688548 (peer_addr_for_me v2:172.21.15.143:0/0) 2023-11-26T07:24:49.086 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:49.082+0000 7f6d777fe640 1 --2- 172.21.15.143:0/2514688548 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f6d7806d820 0x7f6d780affd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:24:49.086 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:49.082+0000 7f6d6ffff640 1 --2- 172.21.15.143:0/2514688548 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f6d780b2b40 0x7f6d7813b1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:24:49.086 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:49.086+0000 7f6d77fff640 1 -- 172.21.15.143:0/2514688548 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f6d7806d820 msgr2=0x7f6d780affd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:24:49.087 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:49.086+0000 7f6d77fff640 1 --2- 172.21.15.143:0/2514688548 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f6d7806d820 0x7f6d780affd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:24:49.087 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:49.086+0000 7f6d77fff640 1 -- 172.21.15.143:0/2514688548 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f6d780b2b40 msgr2=0x7f6d7813b1e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:24:49.087 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:49.086+0000 7f6d77fff640 1 --2- 172.21.15.143:0/2514688548 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f6d780b2b40 0x7f6d7813b1e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:24:49.087 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:49.086+0000 7f6d77fff640 1 -- 172.21.15.143:0/2514688548 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6d7000ab80 con 0x7f6d78153b90 2023-11-26T07:24:49.087 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:49.086+0000 7f6d77fff640 1 --2- 172.21.15.143:0/2514688548 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f6d78153b90 0x7f6d7813b720 secure :-1 s=READY pgs=1414 cs=0 l=1 rev1=1 crypto rx=0x7f6d700b0040 tx=0x7f6d7001a7c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:24:49.088 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:49.086+0000 7f6d74ff9640 1 -- 172.21.15.143:0/2514688548 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6d700b7680 con 0x7f6d78153b90 2023-11-26T07:24:49.088 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:49.086+0000 7f6d74ff9640 1 -- 172.21.15.143:0/2514688548 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6d700b7820 con 0x7f6d78153b90 2023-11-26T07:24:49.088 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:49.086+0000 7f6d7ce59640 1 -- 172.21.15.143:0/2514688548 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6d7814e770 con 0x7f6d78153b90 2023-11-26T07:24:49.088 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:49.086+0000 7f6d74ff9640 1 -- 172.21.15.143:0/2514688548 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6d700b7bd0 con 0x7f6d78153b90 2023-11-26T07:24:49.088 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:49.086+0000 7f6d7ce59640 1 -- 172.21.15.143:0/2514688548 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6d780b0510 con 0x7f6d78153b90 2023-11-26T07:24:49.088 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:49.086+0000 7f6d7ce59640 1 -- 172.21.15.143:0/2514688548 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6d7814ef70 con 0x7f6d78153b90 2023-11-26T07:24:49.088 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:49.086+0000 7f6d74ff9640 1 -- 172.21.15.143:0/2514688548 <== mon.0 v2:172.21.15.143:3300/0 4 ==== mgrmap(e 18) v1 ==== 86107+0+0 (secure 0 0 0) 0x7f6d700b7d30 con 0x7f6d78153b90 2023-11-26T07:24:49.089 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:49.086+0000 7f6d74ff9640 1 --2- 172.21.15.143:0/2514688548 >> [v2:172.21.15.188:6836/3401128553,v1:172.21.15.188:6837/3401128553] conn(0x7f6d58066770 0x7f6d58068c30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:24:49.089 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:49.086+0000 7f6d74ff9640 1 -- 172.21.15.143:0/2514688548 <== mon.0 v2:172.21.15.143:3300/0 5 ==== osd_map(103..103 src has 1..103) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f6d7011f0d0 con 0x7f6d78153b90 2023-11-26T07:24:49.089 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:49.090+0000 7f6d777fe640 1 --2- 172.21.15.143:0/2514688548 >> [v2:172.21.15.188:6836/3401128553,v1:172.21.15.188:6837/3401128553] conn(0x7f6d58066770 0x7f6d58068c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:24:49.090 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:49.090+0000 7f6d777fe640 1 --2- 172.21.15.143:0/2514688548 >> [v2:172.21.15.188:6836/3401128553,v1:172.21.15.188:6837/3401128553] conn(0x7f6d58066770 0x7f6d58068c30 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7f6d68000f70 tx=0x7f6d68007240 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:24:49.092 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:49.090+0000 7f6d74ff9640 1 -- 172.21.15.143:0/2514688548 <== mon.0 v2:172.21.15.143:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172649 (secure 0 0 0) 0x7f6d7814ef70 con 0x7f6d78153b90 2023-11-26T07:24:49.248 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:49.246+0000 7f6d7ce59640 1 -- 172.21.15.143:0/2514688548 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f6d7814ef70 con 0x7f6d78153b90 2023-11-26T07:24:49.249 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:49.246+0000 7f6d74ff9640 1 -- 172.21.15.143:0/2514688548 <== mon.0 v2:172.21.15.143:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v103) v1 ==== 100+0+11 (secure 0 0 0) 0x7f6d7814ef70 con 0x7f6d78153b90 2023-11-26T07:24:49.249 INFO:teuthology.orchestra.run.smithi143.stdout:pg_num: 64 2023-11-26T07:24:49.251 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:49.250+0000 7f6d7ce59640 1 -- 172.21.15.143:0/2514688548 >> [v2:172.21.15.188:6836/3401128553,v1:172.21.15.188:6837/3401128553] conn(0x7f6d58066770 msgr2=0x7f6d58068c30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:24:49.252 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:49.250+0000 7f6d7ce59640 1 --2- 172.21.15.143:0/2514688548 >> [v2:172.21.15.188:6836/3401128553,v1:172.21.15.188:6837/3401128553] conn(0x7f6d58066770 0x7f6d58068c30 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7f6d68000f70 tx=0x7f6d68007240 comp rx=0 tx=0).stop 2023-11-26T07:24:49.252 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:49.250+0000 7f6d7ce59640 1 -- 172.21.15.143:0/2514688548 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f6d78153b90 msgr2=0x7f6d7813b720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:24:49.252 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:49.250+0000 7f6d7ce59640 1 --2- 172.21.15.143:0/2514688548 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f6d78153b90 0x7f6d7813b720 secure :-1 s=READY pgs=1414 cs=0 l=1 rev1=1 crypto rx=0x7f6d700b0040 tx=0x7f6d7001a7c0 comp rx=0 tx=0).stop 2023-11-26T07:24:49.252 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:49.250+0000 7f6d7ce59640 1 -- 172.21.15.143:0/2514688548 shutdown_connections 2023-11-26T07:24:49.252 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:49.250+0000 7f6d7ce59640 1 --2- 172.21.15.143:0/2514688548 >> [v2:172.21.15.188:6836/3401128553,v1:172.21.15.188:6837/3401128553] conn(0x7f6d58066770 0x7f6d58068c30 unknown :-1 s=CLOSED pgs=170 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:24:49.252 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:49.250+0000 7f6d7ce59640 1 --2- 172.21.15.143:0/2514688548 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f6d78153b90 0x7f6d7813b720 unknown :-1 s=CLOSED pgs=1414 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:24:49.252 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:49.250+0000 7f6d7ce59640 1 --2- 172.21.15.143:0/2514688548 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f6d780b2b40 0x7f6d7813b1e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:24:49.252 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:49.250+0000 7f6d7ce59640 1 --2- 172.21.15.143:0/2514688548 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f6d7806d820 0x7f6d780affd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:24:49.252 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:49.250+0000 7f6d7ce59640 1 -- 172.21.15.143:0/2514688548 >> 172.21.15.143:0/2514688548 conn(0x7f6d7806a890 msgr2=0x7f6d7806ac70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:24:49.252 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:49.250+0000 7f6d7ce59640 1 -- 172.21.15.143:0/2514688548 shutdown_connections 2023-11-26T07:24:49.253 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:49.250+0000 7f6d7ce59640 1 -- 172.21.15.143:0/2514688548 wait complete. 2023-11-26T07:24:49.271 DEBUG:teuthology.orchestra.run.smithi143:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-11-26T07:24:49.408 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:49.406+0000 7f1e11972640 1 Processor -- start 2023-11-26T07:24:49.408 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:49.406+0000 7f1e11972640 1 -- start start 2023-11-26T07:24:49.411 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:49.410+0000 7f1e11972640 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7f1e0c143ac0 0x7f1e0c070370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:24:49.411 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:49.410+0000 7f1e11972640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f1e0c0708b0 0x7f1e0c0cf280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:24:49.412 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:49.410+0000 7f1e11972640 1 -- --> v1:172.21.15.143:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1e0c0cfbf0 con 0x7f1e0c0cf7c0 2023-11-26T07:24:49.412 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:49.410+0000 7f1e11972640 1 -- --> v2:172.21.15.188:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1e0c143ea0 con 0x7f1e0c143ac0 2023-11-26T07:24:49.412 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:49.410+0000 7f1e11972640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1e0c0cc730 con 0x7f1e0c0708b0 2023-11-26T07:24:49.412 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:49.410+0000 7f1e0b7fe640 1 --1- >> v1:172.21.15.143:6789/0 conn(0x7f1e0c0cf7c0 0x7f1e0c0b1ef0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.143:6789/0 says I am v1:172.21.15.143:49708/0 (socket says 172.21.15.143:49708) 2023-11-26T07:24:49.412 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:49.410+0000 7f1e0b7fe640 1 -- 172.21.15.143:0/4109834467 learned_addr learned my addr 172.21.15.143:0/4109834467 (peer_addr_for_me v1:172.21.15.143:0/0) 2023-11-26T07:24:49.412 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:49.410+0000 7f1e03fff640 1 --2- 172.21.15.143:0/4109834467 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f1e0c0708b0 0x7f1e0c0cf280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:24:49.412 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:49.410+0000 7f1e0affd640 1 --2- 172.21.15.143:0/4109834467 >> v2:172.21.15.188:3300/0 conn(0x7f1e0c143ac0 0x7f1e0c070370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:24:49.412 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:49.410+0000 7f1e0a7fc640 1 -- 172.21.15.143:0/4109834467 <== mon.0 v1:172.21.15.143:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3065382501 0 0) 0x7f1e0c0cfbf0 con 0x7f1e0c0cf7c0 2023-11-26T07:24:49.412 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:49.410+0000 7f1e0a7fc640 1 -- 172.21.15.143:0/4109834467 --> v1:172.21.15.143:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f1df8005740 con 0x7f1e0c0cf7c0 2023-11-26T07:24:49.412 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:49.410+0000 7f1e0a7fc640 1 -- 172.21.15.143:0/4109834467 <== mon.0 v1:172.21.15.143:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 4251303444 0 0) 0x7f1df8005740 con 0x7f1e0c0cf7c0 2023-11-26T07:24:49.412 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:49.410+0000 7f1e0a7fc640 1 -- 172.21.15.143:0/4109834467 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f1e0c0708b0 msgr2=0x7f1e0c0cf280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-26T07:24:49.413 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:49.410+0000 7f1e0a7fc640 1 --2- 172.21.15.143:0/4109834467 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f1e0c0708b0 0x7f1e0c0cf280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:24:49.413 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:49.410+0000 7f1e0a7fc640 1 -- 172.21.15.143:0/4109834467 >> v2:172.21.15.188:3300/0 conn(0x7f1e0c143ac0 msgr2=0x7f1e0c070370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-26T07:24:49.413 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:49.410+0000 7f1e0a7fc640 1 --2- 172.21.15.143:0/4109834467 >> v2:172.21.15.188:3300/0 conn(0x7f1e0c143ac0 0x7f1e0c070370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:24:49.413 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:49.410+0000 7f1e0a7fc640 1 -- 172.21.15.143:0/4109834467 --> v1:172.21.15.143:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1e0c070ee0 con 0x7f1e0c0cf7c0 2023-11-26T07:24:49.413 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:49.410+0000 7f1e0a7fc640 1 -- 172.21.15.143:0/4109834467 <== mon.0 v1:172.21.15.143:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2137882523 0 0) 0x7f1e04001a20 con 0x7f1e0c0cf7c0 2023-11-26T07:24:49.413 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:49.410+0000 7f1e0affd640 1 --2- 172.21.15.143:0/4109834467 >> v2:172.21.15.188:3300/0 conn(0x7f1e0c143ac0 0x7f1e0c070370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-26T07:24:49.413 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:49.410+0000 7f1e0a7fc640 1 -- 172.21.15.143:0/4109834467 >> v1:172.21.15.143:6789/0 conn(0x7f1e0c0cf7c0 legacy=0x7f1e0c0b1ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:24:49.413 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:49.410+0000 7f1e0a7fc640 1 --2- 172.21.15.143:0/4109834467 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f1e0c0708b0 0x7f1df8005b90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:24:49.413 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:49.410+0000 7f1e0a7fc640 1 --2- 172.21.15.143:0/4109834467 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f1df8006110 0x7f1df80069e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:24:49.413 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:49.410+0000 7f1e0a7fc640 1 --2- 172.21.15.143:0/4109834467 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f1df8006f20 0x7f1df80073a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:24:49.413 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:49.410+0000 7f1e0a7fc640 1 -- 172.21.15.143:0/4109834467 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1e0c0cc730 con 0x7f1df8006f20 2023-11-26T07:24:49.413 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:49.410+0000 7f1e0a7fc640 1 -- 172.21.15.143:0/4109834467 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1df800dba0 con 0x7f1df8006110 2023-11-26T07:24:49.413 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:49.410+0000 7f1e0a7fc640 1 -- 172.21.15.143:0/4109834467 --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1df800dce0 con 0x7f1e0c0708b0 2023-11-26T07:24:49.413 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:49.410+0000 7f1e0b7fe640 1 --2- 172.21.15.143:0/4109834467 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f1df8006f20 0x7f1df80073a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:24:49.414 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:49.410+0000 7f1e0affd640 1 --2- 172.21.15.143:0/4109834467 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f1e0c0708b0 0x7f1df8005b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:24:49.414 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:49.410+0000 7f1e03fff640 1 --2- 172.21.15.143:0/4109834467 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f1df8006110 0x7f1df80069e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:24:49.414 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:49.410+0000 7f1e0b7fe640 1 -- 172.21.15.143:0/4109834467 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f1e0c0708b0 msgr2=0x7f1df8005b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-26T07:24:49.414 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:49.410+0000 7f1e0b7fe640 1 --2- 172.21.15.143:0/4109834467 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f1e0c0708b0 0x7f1df8005b90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:24:49.414 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:49.410+0000 7f1e0b7fe640 1 -- 172.21.15.143:0/4109834467 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f1df8006110 msgr2=0x7f1df80069e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-26T07:24:49.414 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:49.410+0000 7f1e0b7fe640 1 --2- 172.21.15.143:0/4109834467 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f1df8006110 0x7f1df80069e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:24:49.414 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:49.410+0000 7f1e0b7fe640 1 -- 172.21.15.143:0/4109834467 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1df8005740 con 0x7f1df8006f20 2023-11-26T07:24:49.414 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:49.410+0000 7f1e0affd640 1 --2- 172.21.15.143:0/4109834467 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f1e0c0708b0 0x7f1df8005b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-26T07:24:49.415 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:49.414+0000 7f1e0b7fe640 1 --2- 172.21.15.143:0/4109834467 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f1df8006f20 0x7f1df80073a0 secure :-1 s=READY pgs=1416 cs=0 l=1 rev1=1 crypto rx=0x7f1e0c0cfd80 tx=0x7f1e0401d2c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=79f4c93883bbd92c server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:24:49.415 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:49.414+0000 7f1e0a7fc640 1 -- 172.21.15.143:0/4109834467 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1e040ad630 con 0x7f1df8006f20 2023-11-26T07:24:49.415 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:49.414+0000 7f1e0a7fc640 1 -- 172.21.15.143:0/4109834467 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1e040a8070 con 0x7f1df8006f20 2023-11-26T07:24:49.415 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:49.414+0000 7f1e0a7fc640 1 -- 172.21.15.143:0/4109834467 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1e040ad9d0 con 0x7f1df8006f20 2023-11-26T07:24:49.416 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:49.414+0000 7f1e11972640 1 -- 172.21.15.143:0/4109834467 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f1df8006f20 msgr2=0x7f1df80073a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:24:49.416 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:49.414+0000 7f1e11972640 1 --2- 172.21.15.143:0/4109834467 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f1df8006f20 0x7f1df80073a0 secure :-1 s=READY pgs=1416 cs=0 l=1 rev1=1 crypto rx=0x7f1e0c0cfd80 tx=0x7f1e0401d2c0 comp rx=0 tx=0).stop 2023-11-26T07:24:49.416 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:49.414+0000 7f1e0b7fe640 1 -- 172.21.15.143:0/4109834467 reap_dead start 2023-11-26T07:24:49.416 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:49.414+0000 7f1e11972640 1 -- 172.21.15.143:0/4109834467 shutdown_connections 2023-11-26T07:24:49.416 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:49.414+0000 7f1e11972640 1 -- 172.21.15.143:0/4109834467 >> 172.21.15.143:0/4109834467 conn(0x7f1e0c06a890 msgr2=0x7f1e0c06aca0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:24:49.416 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:49.414+0000 7f1e11972640 1 -- 172.21.15.143:0/4109834467 shutdown_connections 2023-11-26T07:24:49.416 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:49.414+0000 7f1e11972640 1 -- 172.21.15.143:0/4109834467 wait complete. 2023-11-26T07:24:49.417 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:49.418+0000 7f1e11972640 1 Processor -- start 2023-11-26T07:24:49.417 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:49.418+0000 7f1e11972640 1 -- start start 2023-11-26T07:24:49.419 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:49.418+0000 7f1e11972640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f1df8006f20 0x7f1e0c159480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:24:49.419 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:49.418+0000 7f1e11972640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f1e0c137e20 0x7f1e0c13a3a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:24:49.419 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:49.418+0000 7f1e11972640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f1e0c15baa0 0x7f1e0c15bf20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:24:49.419 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:49.418+0000 7f1e11972640 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1e0c1ee350 con 0x7f1e0c15baa0 2023-11-26T07:24:49.419 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:49.418+0000 7f1e11972640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1e0c15c460 con 0x7f1e0c137e20 2023-11-26T07:24:49.419 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:49.418+0000 7f1e11972640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1e0c0aebd0 con 0x7f1df8006f20 2023-11-26T07:24:49.420 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:49.418+0000 7f1e0b7fe640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f1e0c15baa0 0x7f1e0c15bf20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:24:49.420 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:49.418+0000 7f1e0b7fe640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f1e0c15baa0 0x7f1e0c15bf20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:58694/0 (socket says 172.21.15.143:58694) 2023-11-26T07:24:49.420 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:49.418+0000 7f1e0b7fe640 1 -- 172.21.15.143:0/2875419808 learned_addr learned my addr 172.21.15.143:0/2875419808 (peer_addr_for_me v2:172.21.15.143:0/0) 2023-11-26T07:24:49.420 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:49.418+0000 7f1e03fff640 1 --2- 172.21.15.143:0/2875419808 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f1e0c137e20 0x7f1e0c13a3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:24:49.420 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:49.418+0000 7f1e0affd640 1 --2- 172.21.15.143:0/2875419808 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f1df8006f20 0x7f1e0c159480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:24:49.420 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:49.418+0000 7f1e0b7fe640 1 -- 172.21.15.143:0/2875419808 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f1df8006f20 msgr2=0x7f1e0c159480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:24:49.420 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:49.418+0000 7f1e0b7fe640 1 --2- 172.21.15.143:0/2875419808 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f1df8006f20 0x7f1e0c159480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:24:49.421 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:49.418+0000 7f1e0b7fe640 1 -- 172.21.15.143:0/2875419808 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f1e0c137e20 msgr2=0x7f1e0c13a3a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:24:49.421 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:49.418+0000 7f1e0b7fe640 1 --2- 172.21.15.143:0/2875419808 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f1e0c137e20 0x7f1e0c13a3a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:24:49.421 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:49.418+0000 7f1e0b7fe640 1 -- 172.21.15.143:0/2875419808 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1e0c0aeeb0 con 0x7f1e0c15baa0 2023-11-26T07:24:49.421 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:49.418+0000 7f1e0b7fe640 1 --2- 172.21.15.143:0/2875419808 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f1e0c15baa0 0x7f1e0c15bf20 secure :-1 s=READY pgs=1417 cs=0 l=1 rev1=1 crypto rx=0x7f1e04005c30 tx=0x7f1df8007a10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:24:49.422 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:49.418+0000 7f1e08ff9640 1 -- 172.21.15.143:0/2875419808 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1e040019e0 con 0x7f1e0c15baa0 2023-11-26T07:24:49.422 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:49.418+0000 7f1e08ff9640 1 -- 172.21.15.143:0/2875419808 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1e04001ff0 con 0x7f1e0c15baa0 2023-11-26T07:24:49.422 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:49.418+0000 7f1e08ff9640 1 -- 172.21.15.143:0/2875419808 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1e040ad630 con 0x7f1e0c15baa0 2023-11-26T07:24:49.422 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:49.418+0000 7f1e11972640 1 -- 172.21.15.143:0/2875419808 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1e0c0af190 con 0x7f1e0c15baa0 2023-11-26T07:24:49.423 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:49.418+0000 7f1e11972640 1 -- 172.21.15.143:0/2875419808 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1e0c0af750 con 0x7f1e0c15baa0 2023-11-26T07:24:49.423 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:49.418+0000 7f1e11972640 1 -- 172.21.15.143:0/2875419808 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1e0c067680 con 0x7f1e0c15baa0 2023-11-26T07:24:49.423 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:49.422+0000 7f1e08ff9640 1 -- 172.21.15.143:0/2875419808 <== mon.0 v2:172.21.15.143:3300/0 4 ==== mgrmap(e 18) v1 ==== 86107+0+0 (secure 0 0 0) 0x7f1e040aa030 con 0x7f1e0c15baa0 2023-11-26T07:24:49.423 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:49.422+0000 7f1e08ff9640 1 --2- 172.21.15.143:0/2875419808 >> [v2:172.21.15.188:6836/3401128553,v1:172.21.15.188:6837/3401128553] conn(0x7f1dec066740 0x7f1dec068c00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:24:49.423 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:49.422+0000 7f1e08ff9640 1 -- 172.21.15.143:0/2875419808 <== mon.0 v2:172.21.15.143:3300/0 5 ==== osd_map(103..103 src has 1..103) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f1e0411ea70 con 0x7f1e0c15baa0 2023-11-26T07:24:49.424 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:49.422+0000 7f1e0affd640 1 --2- 172.21.15.143:0/2875419808 >> [v2:172.21.15.188:6836/3401128553,v1:172.21.15.188:6837/3401128553] conn(0x7f1dec066740 0x7f1dec068c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:24:49.424 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:49.422+0000 7f1e0affd640 1 --2- 172.21.15.143:0/2875419808 >> [v2:172.21.15.188:6836/3401128553,v1:172.21.15.188:6837/3401128553] conn(0x7f1dec066740 0x7f1dec068c00 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7f1dfc009de0 tx=0x7f1dfc009190 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:24:49.426 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:49.426+0000 7f1e08ff9640 1 -- 172.21.15.143:0/2875419808 <== mon.0 v2:172.21.15.143:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172649 (secure 0 0 0) 0x7f1e040ef640 con 0x7f1e0c15baa0 2023-11-26T07:24:49.608 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:49.606+0000 7f1e11972640 1 -- 172.21.15.143:0/2875419808 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f1e0c067680 con 0x7f1e0c15baa0 2023-11-26T07:24:49.608 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:49.606+0000 7f1e08ff9640 1 -- 172.21.15.143:0/2875419808 <== mon.0 v2:172.21.15.143:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 66 v66) v1 ==== 94+0+4927 (secure 0 0 0) 0x7f1e040ef1d0 con 0x7f1e0c15baa0 2023-11-26T07:24:49.608 INFO:teuthology.orchestra.run.smithi143.stdout: 2023-11-26T07:24:49.609 INFO:teuthology.orchestra.run.smithi143.stdout:{"epoch":66,"default_fscid":8,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6629,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.188:6834/1327398471","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6832","nonce":1327398471},{"type":"v1","addr":"172.21.15.188:6834","nonce":1327398471}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":60},{"gid":7453,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.188:6835/1640976426","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6833","nonce":1640976426},{"type":"v1","addr":"172.21.15.188:6835","nonce":1640976426}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":60},{"gid":7941,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.143:6834/2855111671","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6832","nonce":2855111671},{"type":"v1","addr":"172.21.15.143:6834","nonce":2855111671}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":60}],"filesystems":[{"mdsmap":{"epoch":65,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-11-26T07:24:08.295882+0000","modified":"2023-11-26T07:24:11.308119+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":7944},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7944":{"gid":7944,"name":"a","rank":0,"incarnation":61,"state":"up:active","state_seq":3,"addr":"172.21.15.143:6835/2571525464","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6833","nonce":2571525464},{"type":"v1","addr":"172.21.15.143:6835","nonce":2571525464}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[17],"metadata_pool":16,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":8}]} 2023-11-26T07:24:49.609 INFO:teuthology.orchestra.run.smithi143.stderr:dumped fsmap epoch 66 2023-11-26T07:24:49.610 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:49.610+0000 7f1e11972640 1 -- 172.21.15.143:0/2875419808 >> [v2:172.21.15.188:6836/3401128553,v1:172.21.15.188:6837/3401128553] conn(0x7f1dec066740 msgr2=0x7f1dec068c00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:24:49.611 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:49.610+0000 7f1e11972640 1 --2- 172.21.15.143:0/2875419808 >> [v2:172.21.15.188:6836/3401128553,v1:172.21.15.188:6837/3401128553] conn(0x7f1dec066740 0x7f1dec068c00 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7f1dfc009de0 tx=0x7f1dfc009190 comp rx=0 tx=0).stop 2023-11-26T07:24:49.611 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:49.610+0000 7f1e11972640 1 -- 172.21.15.143:0/2875419808 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f1e0c15baa0 msgr2=0x7f1e0c15bf20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:24:49.611 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:49.610+0000 7f1e11972640 1 --2- 172.21.15.143:0/2875419808 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f1e0c15baa0 0x7f1e0c15bf20 secure :-1 s=READY pgs=1417 cs=0 l=1 rev1=1 crypto rx=0x7f1e04005c30 tx=0x7f1df8007a10 comp rx=0 tx=0).stop 2023-11-26T07:24:49.611 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:49.610+0000 7f1e11972640 1 -- 172.21.15.143:0/2875419808 shutdown_connections 2023-11-26T07:24:49.611 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:49.610+0000 7f1e11972640 1 --2- 172.21.15.143:0/2875419808 >> [v2:172.21.15.188:6836/3401128553,v1:172.21.15.188:6837/3401128553] conn(0x7f1dec066740 0x7f1dec068c00 unknown :-1 s=CLOSED pgs=171 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:24:49.611 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:49.610+0000 7f1e11972640 1 --2- 172.21.15.143:0/2875419808 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f1e0c15baa0 0x7f1e0c15bf20 unknown :-1 s=CLOSED pgs=1417 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:24:49.611 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:49.610+0000 7f1e11972640 1 --2- 172.21.15.143:0/2875419808 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f1e0c137e20 0x7f1e0c13a3a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:24:49.612 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:49.610+0000 7f1e11972640 1 --2- 172.21.15.143:0/2875419808 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f1df8006f20 0x7f1e0c159480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:24:49.612 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:49.610+0000 7f1e11972640 1 -- 172.21.15.143:0/2875419808 >> 172.21.15.143:0/2875419808 conn(0x7f1e0c06a890 msgr2=0x7f1e0c0ce070 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:24:49.612 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:49.610+0000 7f1e11972640 1 -- 172.21.15.143:0/2875419808 shutdown_connections 2023-11-26T07:24:49.612 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:49.610+0000 7f1e11972640 1 -- 172.21.15.143:0/2875419808 wait complete. 2023-11-26T07:24:49.630 DEBUG:teuthology.orchestra.run.smithi143:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-11-26T07:24:49.771 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:49.770+0000 7f8b1d846640 1 Processor -- start 2023-11-26T07:24:49.771 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:49.770+0000 7f8b1d846640 1 -- start start 2023-11-26T07:24:49.772 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:49.770+0000 7f8b1d846640 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7f8b18143ac0 0x7f8b18070370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:24:49.772 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:49.770+0000 7f8b1d846640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f8b180708b0 0x7f8b180cf280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:24:49.773 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:49.770+0000 7f8b1d846640 1 -- --> v1:172.21.15.143:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8b180cfbf0 con 0x7f8b180cf7c0 2023-11-26T07:24:49.773 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:49.770+0000 7f8b1d846640 1 -- --> v2:172.21.15.143:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8b18143ea0 con 0x7f8b18143ac0 2023-11-26T07:24:49.773 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:49.770+0000 7f8b1d846640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8b180cc730 con 0x7f8b180708b0 2023-11-26T07:24:49.773 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:49.770+0000 7f8b177fe640 1 --1- >> v1:172.21.15.143:6789/0 conn(0x7f8b180cf7c0 0x7f8b180b1ef0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.143:6789/0 says I am v1:172.21.15.143:49716/0 (socket says 172.21.15.143:49716) 2023-11-26T07:24:49.773 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:49.770+0000 7f8b177fe640 1 -- 172.21.15.143:0/3157906780 learned_addr learned my addr 172.21.15.143:0/3157906780 (peer_addr_for_me v1:172.21.15.143:0/0) 2023-11-26T07:24:49.773 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:49.770+0000 7f8b16ffd640 1 --2- 172.21.15.143:0/3157906780 >> v2:172.21.15.143:3300/0 conn(0x7f8b18143ac0 0x7f8b18070370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:24:49.774 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:49.770+0000 7f8b0e7fc640 1 --2- 172.21.15.143:0/3157906780 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f8b180708b0 0x7f8b180cf280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:24:49.774 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:49.770+0000 7f8b16ffd640 1 -- 172.21.15.143:0/3157906780 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f8b180708b0 msgr2=0x7f8b180cf280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-26T07:24:49.774 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:49.774+0000 7f8b16ffd640 1 --2- 172.21.15.143:0/3157906780 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f8b180708b0 0x7f8b180cf280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:24:49.774 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:49.774+0000 7f8b16ffd640 1 -- 172.21.15.143:0/3157906780 >> v1:172.21.15.143:6789/0 conn(0x7f8b180cf7c0 legacy=0x7f8b180b1ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:24:49.774 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:49.774+0000 7f8b16ffd640 1 -- 172.21.15.143:0/3157906780 --> v2:172.21.15.143:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8b18070ee0 con 0x7f8b18143ac0 2023-11-26T07:24:49.774 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:49.774+0000 7f8b167fc640 1 -- 172.21.15.143:0/3157906780 <== mon.0 v1:172.21.15.143:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 141090445 0 0) 0x7f8b180cfbf0 con 0x7f8b180cf7c0 2023-11-26T07:24:49.776 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:49.774+0000 7f8b16ffd640 1 --2- 172.21.15.143:0/3157906780 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f8b18143ac0 0x7f8b18070370 secure :-1 s=READY pgs=1419 cs=0 l=1 rev1=1 crypto rx=0x7f8b080097c0 tx=0x7f8b0801c480 comp rx=0 tx=0).ready entity=mon.0 client_cookie=887eeab366cc09f1 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:24:49.776 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:49.774+0000 7f8b167fc640 1 -- 172.21.15.143:0/3157906780 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8b080ad640 con 0x7f8b18143ac0 2023-11-26T07:24:49.776 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:49.774+0000 7f8b167fc640 1 -- 172.21.15.143:0/3157906780 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8b080ad7a0 con 0x7f8b18143ac0 2023-11-26T07:24:49.776 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:49.774+0000 7f8b167fc640 1 -- 172.21.15.143:0/3157906780 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8b080adb30 con 0x7f8b18143ac0 2023-11-26T07:24:49.777 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:49.774+0000 7f8b1d846640 1 -- 172.21.15.143:0/3157906780 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f8b18143ac0 msgr2=0x7f8b18070370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:24:49.777 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:49.774+0000 7f8b1d846640 1 --2- 172.21.15.143:0/3157906780 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f8b18143ac0 0x7f8b18070370 secure :-1 s=READY pgs=1419 cs=0 l=1 rev1=1 crypto rx=0x7f8b080097c0 tx=0x7f8b0801c480 comp rx=0 tx=0).stop 2023-11-26T07:24:49.777 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:49.774+0000 7f8b1d846640 1 -- 172.21.15.143:0/3157906780 shutdown_connections 2023-11-26T07:24:49.777 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:49.774+0000 7f8b1d846640 1 --2- 172.21.15.143:0/3157906780 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f8b180708b0 0x7f8b180cf280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:24:49.777 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:49.774+0000 7f8b1d846640 1 --2- 172.21.15.143:0/3157906780 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f8b18143ac0 0x7f8b18070370 unknown :-1 s=CLOSED pgs=1419 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:24:49.777 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:49.774+0000 7f8b1d846640 1 -- 172.21.15.143:0/3157906780 >> 172.21.15.143:0/3157906780 conn(0x7f8b1806a890 msgr2=0x7f8b1806aca0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:24:49.777 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:49.774+0000 7f8b1d846640 1 -- 172.21.15.143:0/3157906780 shutdown_connections 2023-11-26T07:24:49.778 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:49.774+0000 7f8b1d846640 1 -- 172.21.15.143:0/3157906780 wait complete. 2023-11-26T07:24:49.778 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:49.778+0000 7f8b1d846640 1 Processor -- start 2023-11-26T07:24:49.779 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:49.778+0000 7f8b1d846640 1 -- start start 2023-11-26T07:24:49.780 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:49.778+0000 7f8b1d846640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f8b180708b0 0x7f8b18140480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:24:49.781 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:49.778+0000 7f8b1d846640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f8b180cf7c0 0x7f8b180b1170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:24:49.781 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:49.778+0000 7f8b1d846640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f8b18143ac0 0x7f8b181f6170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:24:49.781 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:49.778+0000 7f8b1d846640 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8b1813f3e0 con 0x7f8b18143ac0 2023-11-26T07:24:49.781 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:49.778+0000 7f8b1d846640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8b1813f6c0 con 0x7f8b180708b0 2023-11-26T07:24:49.781 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:49.778+0000 7f8b1d846640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8b1813f9a0 con 0x7f8b180cf7c0 2023-11-26T07:24:49.781 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:49.778+0000 7f8b177fe640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f8b18143ac0 0x7f8b181f6170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:24:49.781 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:49.778+0000 7f8b177fe640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f8b18143ac0 0x7f8b181f6170 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:58706/0 (socket says 172.21.15.143:58706) 2023-11-26T07:24:49.781 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:49.778+0000 7f8b177fe640 1 -- 172.21.15.143:0/3119368749 learned_addr learned my addr 172.21.15.143:0/3119368749 (peer_addr_for_me v2:172.21.15.143:0/0) 2023-11-26T07:24:49.781 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:49.778+0000 7f8b0e7fc640 1 --2- 172.21.15.143:0/3119368749 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f8b180cf7c0 0x7f8b180b1170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:24:49.781 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:49.778+0000 7f8b16ffd640 1 --2- 172.21.15.143:0/3119368749 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f8b180708b0 0x7f8b18140480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:24:49.781 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:49.778+0000 7f8b177fe640 1 -- 172.21.15.143:0/3119368749 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f8b180cf7c0 msgr2=0x7f8b180b1170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:24:49.782 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:49.778+0000 7f8b177fe640 1 --2- 172.21.15.143:0/3119368749 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f8b180cf7c0 0x7f8b180b1170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:24:49.782 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:49.778+0000 7f8b177fe640 1 -- 172.21.15.143:0/3119368749 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f8b180708b0 msgr2=0x7f8b18140480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:24:49.782 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:49.778+0000 7f8b177fe640 1 --2- 172.21.15.143:0/3119368749 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f8b180708b0 0x7f8b18140480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:24:49.782 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:49.778+0000 7f8b177fe640 1 -- 172.21.15.143:0/3119368749 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8b08009420 con 0x7f8b18143ac0 2023-11-26T07:24:49.783 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:49.778+0000 7f8b177fe640 1 --2- 172.21.15.143:0/3119368749 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f8b18143ac0 0x7f8b181f6170 secure :-1 s=READY pgs=1420 cs=0 l=1 rev1=1 crypto rx=0x7f8b1000c430 tx=0x7f8b1000c8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:24:49.783 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:49.782+0000 7f8b14ff9640 1 -- 172.21.15.143:0/3119368749 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8b10005cf0 con 0x7f8b18143ac0 2023-11-26T07:24:49.783 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:49.782+0000 7f8b14ff9640 1 -- 172.21.15.143:0/3119368749 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8b10005e50 con 0x7f8b18143ac0 2023-11-26T07:24:49.783 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:49.782+0000 7f8b14ff9640 1 -- 172.21.15.143:0/3119368749 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8b100072f0 con 0x7f8b18143ac0 2023-11-26T07:24:49.783 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:49.782+0000 7f8b1d846640 1 -- 172.21.15.143:0/3119368749 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8b1813fc80 con 0x7f8b18143ac0 2023-11-26T07:24:49.783 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:49.782+0000 7f8b1d846640 1 -- 172.21.15.143:0/3119368749 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8b180b16b0 con 0x7f8b18143ac0 2023-11-26T07:24:49.783 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:49.782+0000 7f8b1d846640 1 -- 172.21.15.143:0/3119368749 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8b181409c0 con 0x7f8b18143ac0 2023-11-26T07:24:49.784 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:49.782+0000 7f8b14ff9640 1 -- 172.21.15.143:0/3119368749 <== mon.0 v2:172.21.15.143:3300/0 4 ==== mgrmap(e 18) v1 ==== 86107+0+0 (secure 0 0 0) 0x7f8b10007450 con 0x7f8b18143ac0 2023-11-26T07:24:49.784 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:49.782+0000 7f8b14ff9640 1 --2- 172.21.15.143:0/3119368749 >> [v2:172.21.15.188:6836/3401128553,v1:172.21.15.188:6837/3401128553] conn(0x7f8af8066740 0x7f8af8068c00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:24:49.784 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:49.782+0000 7f8b14ff9640 1 -- 172.21.15.143:0/3119368749 <== mon.0 v2:172.21.15.143:3300/0 5 ==== osd_map(103..103 src has 1..103) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f8b10083890 con 0x7f8b18143ac0 2023-11-26T07:24:49.785 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:49.782+0000 7f8b16ffd640 1 --2- 172.21.15.143:0/3119368749 >> [v2:172.21.15.188:6836/3401128553,v1:172.21.15.188:6837/3401128553] conn(0x7f8af8066740 0x7f8af8068c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:24:49.785 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:49.786+0000 7f8b16ffd640 1 --2- 172.21.15.143:0/3119368749 >> [v2:172.21.15.188:6836/3401128553,v1:172.21.15.188:6837/3401128553] conn(0x7f8af8066740 0x7f8af8068c00 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7f8b080af000 tx=0x7f8b08008040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:24:49.788 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:49.786+0000 7f8b14ff9640 1 -- 172.21.15.143:0/3119368749 <== mon.0 v2:172.21.15.143:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172649 (secure 0 0 0) 0x7f8b10050a30 con 0x7f8b18143ac0 2023-11-26T07:24:49.938 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:49.938+0000 7f8b1d846640 1 -- 172.21.15.143:0/3119368749 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f8b18067680 con 0x7f8b18143ac0 2023-11-26T07:24:49.939 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:49.938+0000 7f8b14ff9640 1 -- 172.21.15.143:0/3119368749 <== mon.0 v2:172.21.15.143:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v103) v1 ==== 74+0+16563 (secure 0 0 0) 0x7f8b10017090 con 0x7f8b18143ac0 2023-11-26T07:24:49.939 INFO:teuthology.orchestra.run.smithi143.stdout: 2023-11-26T07:24:49.939 INFO:teuthology.orchestra.run.smithi143.stdout:{"epoch":103,"fsid":"f5033060-2469-4a07-95fe-2b76f5404bc1","created":"2023-11-26T07:17:28.613663+0000","modified":"2023-11-26T07:24:09.510299+0000","last_up_change":"2023-11-26T07:17:41.157813+0000","last_in_change":"2023-11-26T07:17:31.181658+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":17,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-11-26T07:17:41.758597+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":16,"pool_name":"cephfs_metadata","create_time":"2023-11-26T07:24:06.278132+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"102","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":17,"pool_name":"cephfs_data","create_time":"2023-11-26T07:24:07.294850+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"103","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"ebbee2c6-023e-44b9-9f1d-6f7bfd309730","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":101,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6817","nonce":639807369},{"type":"v1","addr":"172.21.15.143:6819","nonce":639807369}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6821","nonce":639807369},{"type":"v1","addr":"172.21.15.143:6823","nonce":639807369}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6830","nonce":639807369},{"type":"v1","addr":"172.21.15.143:6831","nonce":639807369}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6825","nonce":639807369},{"type":"v1","addr":"172.21.15.143:6828","nonce":639807369}]},"public_addr":"172.21.15.143:6819/639807369","cluster_addr":"172.21.15.143:6823/639807369","heartbeat_back_addr":"172.21.15.143:6831/639807369","heartbeat_front_addr":"172.21.15.143:6828/639807369","state":["exists","up"]},{"osd":1,"uuid":"fec4b8be-7122-49e9-9a19-5850a0700001","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":101,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6816","nonce":1144942530},{"type":"v1","addr":"172.21.15.143:6818","nonce":1144942530}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6820","nonce":1144942530},{"type":"v1","addr":"172.21.15.143:6822","nonce":1144942530}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6827","nonce":1144942530},{"type":"v1","addr":"172.21.15.143:6829","nonce":1144942530}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6824","nonce":1144942530},{"type":"v1","addr":"172.21.15.143:6826","nonce":1144942530}]},"public_addr":"172.21.15.143:6818/1144942530","cluster_addr":"172.21.15.143:6822/1144942530","heartbeat_back_addr":"172.21.15.143:6829/1144942530","heartbeat_front_addr":"172.21.15.143:6826/1144942530","state":["exists","up"]},{"osd":2,"uuid":"2888af5b-f933-46ca-8f95-6dd494ff43f6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":101,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6808","nonce":3909095186},{"type":"v1","addr":"172.21.15.143:6809","nonce":3909095186}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6810","nonce":3909095186},{"type":"v1","addr":"172.21.15.143:6811","nonce":3909095186}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6814","nonce":3909095186},{"type":"v1","addr":"172.21.15.143:6815","nonce":3909095186}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6812","nonce":3909095186},{"type":"v1","addr":"172.21.15.143:6813","nonce":3909095186}]},"public_addr":"172.21.15.143:6809/3909095186","cluster_addr":"172.21.15.143:6811/3909095186","heartbeat_back_addr":"172.21.15.143:6815/3909095186","heartbeat_front_addr":"172.21.15.143:6813/3909095186","state":["exists","up"]},{"osd":3,"uuid":"d1f52435-7bc1-4081-9550-9cd1ba55f549","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":101,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6800","nonce":3854210991},{"type":"v1","addr":"172.21.15.143:6801","nonce":3854210991}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6802","nonce":3854210991},{"type":"v1","addr":"172.21.15.143:6803","nonce":3854210991}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6806","nonce":3854210991},{"type":"v1","addr":"172.21.15.143:6807","nonce":3854210991}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6804","nonce":3854210991},{"type":"v1","addr":"172.21.15.143:6805","nonce":3854210991}]},"public_addr":"172.21.15.143:6801/3854210991","cluster_addr":"172.21.15.143:6803/3854210991","heartbeat_back_addr":"172.21.15.143:6807/3854210991","heartbeat_front_addr":"172.21.15.143:6805/3854210991","state":["exists","up"]},{"osd":4,"uuid":"e2ae8b6e-6e25-49b2-bcbf-aacce68f06a3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":101,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6800","nonce":3924606902},{"type":"v1","addr":"172.21.15.188:6801","nonce":3924606902}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6802","nonce":3924606902},{"type":"v1","addr":"172.21.15.188:6803","nonce":3924606902}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6806","nonce":3924606902},{"type":"v1","addr":"172.21.15.188:6807","nonce":3924606902}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6804","nonce":3924606902},{"type":"v1","addr":"172.21.15.188:6805","nonce":3924606902}]},"public_addr":"172.21.15.188:6801/3924606902","cluster_addr":"172.21.15.188:6803/3924606902","heartbeat_back_addr":"172.21.15.188:6807/3924606902","heartbeat_front_addr":"172.21.15.188:6805/3924606902","state":["exists","up"]},{"osd":5,"uuid":"3cec8a32-7698-4c36-8e89-e4ba11934f33","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":101,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6813","nonce":186841544},{"type":"v1","addr":"172.21.15.188:6815","nonce":186841544}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6817","nonce":186841544},{"type":"v1","addr":"172.21.15.188:6818","nonce":186841544}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6822","nonce":186841544},{"type":"v1","addr":"172.21.15.188:6823","nonce":186841544}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6820","nonce":186841544},{"type":"v1","addr":"172.21.15.188:6821","nonce":186841544}]},"public_addr":"172.21.15.188:6815/186841544","cluster_addr":"172.21.15.188:6818/186841544","heartbeat_back_addr":"172.21.15.188:6823/186841544","heartbeat_front_addr":"172.21.15.188:6821/186841544","state":["exists","up"]},{"osd":6,"uuid":"58a65d5d-e090-473a-81b4-ad38e85029f9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":101,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6808","nonce":2153499405},{"type":"v1","addr":"172.21.15.188:6809","nonce":2153499405}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6810","nonce":2153499405},{"type":"v1","addr":"172.21.15.188:6811","nonce":2153499405}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6816","nonce":2153499405},{"type":"v1","addr":"172.21.15.188:6819","nonce":2153499405}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6812","nonce":2153499405},{"type":"v1","addr":"172.21.15.188:6814","nonce":2153499405}]},"public_addr":"172.21.15.188:6809/2153499405","cluster_addr":"172.21.15.188:6811/2153499405","heartbeat_back_addr":"172.21.15.188:6819/2153499405","heartbeat_front_addr":"172.21.15.188:6814/2153499405","state":["exists","up"]},{"osd":7,"uuid":"ea5b7f70-3554-4cf8-ac30-f7c59b80678d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":101,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6824","nonce":2722096729},{"type":"v1","addr":"172.21.15.188:6825","nonce":2722096729}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6826","nonce":2722096729},{"type":"v1","addr":"172.21.15.188:6827","nonce":2722096729}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6830","nonce":2722096729},{"type":"v1","addr":"172.21.15.188:6831","nonce":2722096729}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6828","nonce":2722096729},{"type":"v1","addr":"172.21.15.188:6829","nonce":2722096729}]},"public_addr":"172.21.15.188:6825/2722096729","cluster_addr":"172.21.15.188:6827/2722096729","heartbeat_back_addr":"172.21.15.188:6831/2722096729","heartbeat_front_addr":"172.21.15.188:6829/2722096729","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-26T07:17:36.995032+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-26T07:17:36.932266+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-26T07:17:36.829304+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-26T07:17:36.804297+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-26T07:17:36.667892+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-26T07:17:36.964868+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-26T07:17:36.827332+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-26T07:17:36.893179+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-11-26T07:24:49.941 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:49.938+0000 7f8b1d846640 1 -- 172.21.15.143:0/3119368749 >> [v2:172.21.15.188:6836/3401128553,v1:172.21.15.188:6837/3401128553] conn(0x7f8af8066740 msgr2=0x7f8af8068c00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:24:49.941 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:49.938+0000 7f8b1d846640 1 --2- 172.21.15.143:0/3119368749 >> [v2:172.21.15.188:6836/3401128553,v1:172.21.15.188:6837/3401128553] conn(0x7f8af8066740 0x7f8af8068c00 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7f8b080af000 tx=0x7f8b08008040 comp rx=0 tx=0).stop 2023-11-26T07:24:49.941 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:49.938+0000 7f8b1d846640 1 -- 172.21.15.143:0/3119368749 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f8b18143ac0 msgr2=0x7f8b181f6170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:24:49.941 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:49.938+0000 7f8b1d846640 1 --2- 172.21.15.143:0/3119368749 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f8b18143ac0 0x7f8b181f6170 secure :-1 s=READY pgs=1420 cs=0 l=1 rev1=1 crypto rx=0x7f8b1000c430 tx=0x7f8b1000c8f0 comp rx=0 tx=0).stop 2023-11-26T07:24:49.941 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:49.938+0000 7f8b1d846640 1 -- 172.21.15.143:0/3119368749 shutdown_connections 2023-11-26T07:24:49.941 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:49.938+0000 7f8b1d846640 1 --2- 172.21.15.143:0/3119368749 >> [v2:172.21.15.188:6836/3401128553,v1:172.21.15.188:6837/3401128553] conn(0x7f8af8066740 0x7f8af8068c00 unknown :-1 s=CLOSED pgs=172 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:24:49.942 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:49.938+0000 7f8b1d846640 1 --2- 172.21.15.143:0/3119368749 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f8b18143ac0 0x7f8b181f6170 unknown :-1 s=CLOSED pgs=1420 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:24:49.942 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:49.938+0000 7f8b1d846640 1 --2- 172.21.15.143:0/3119368749 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f8b180cf7c0 0x7f8b180b1170 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:24:49.942 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:49.938+0000 7f8b1d846640 1 --2- 172.21.15.143:0/3119368749 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f8b180708b0 0x7f8b18140480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:24:49.942 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:49.938+0000 7f8b1d846640 1 -- 172.21.15.143:0/3119368749 >> 172.21.15.143:0/3119368749 conn(0x7f8b1806a890 msgr2=0x7f8b180b3950 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:24:49.942 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:49.942+0000 7f8b1d846640 1 -- 172.21.15.143:0/3119368749 shutdown_connections 2023-11-26T07:24:49.942 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:49.942+0000 7f8b1d846640 1 -- 172.21.15.143:0/3119368749 wait complete. 2023-11-26T07:24:49.961 INFO:tasks.cephfs.filesystem:Destroying file system cephfs and related pools 2023-11-26T07:24:49.962 DEBUG:teuthology.orchestra.run.smithi143:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-11-26T07:24:50.100 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:50.098+0000 7f16c7079640 1 Processor -- start 2023-11-26T07:24:50.100 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:50.098+0000 7f16c7079640 1 -- start start 2023-11-26T07:24:50.102 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:50.102+0000 7f16c7079640 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7f16c015d6e0 0x7f16c00d0df0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:24:50.102 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:50.102+0000 7f16c7079640 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7f16c00b2160 0x7f16c00d14b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:24:50.102 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:50.102+0000 7f16c7079640 1 -- --> v1:172.21.15.143:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f16c00cf0e0 con 0x7f16c00d19f0 2023-11-26T07:24:50.102 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:50.102+0000 7f16c7079640 1 -- --> v2:172.21.15.143:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f16c015dac0 con 0x7f16c015d6e0 2023-11-26T07:24:50.102 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:50.102+0000 7f16c7079640 1 -- --> v2:172.21.15.188:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f16c00d1e80 con 0x7f16c00b2160 2023-11-26T07:24:50.102 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:50.102+0000 7f16c4e15640 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7f16c015d6e0 0x7f16c00d0df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:24:50.103 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:50.102+0000 7f16c5616640 1 --1- >> v1:172.21.15.143:6789/0 conn(0x7f16c00d19f0 0x7f16c00cfbf0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.143:6789/0 says I am v1:172.21.15.143:49720/0 (socket says 172.21.15.143:49720) 2023-11-26T07:24:50.103 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:50.102+0000 7f16c5616640 1 -- 172.21.15.143:0/2505935202 learned_addr learned my addr 172.21.15.143:0/2505935202 (peer_addr_for_me v1:172.21.15.143:0/0) 2023-11-26T07:24:50.103 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:50.102+0000 7f16bffff640 1 --2- 172.21.15.143:0/2505935202 >> v2:172.21.15.188:3300/0 conn(0x7f16c00b2160 0x7f16c00d14b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:24:50.103 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:50.102+0000 7f16c4e15640 1 -- 172.21.15.143:0/2505935202 >> v2:172.21.15.188:3300/0 conn(0x7f16c00b2160 msgr2=0x7f16c00d14b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-26T07:24:50.103 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:50.102+0000 7f16c4e15640 1 --2- 172.21.15.143:0/2505935202 >> v2:172.21.15.188:3300/0 conn(0x7f16c00b2160 0x7f16c00d14b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:24:50.103 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:50.102+0000 7f16c4e15640 1 -- 172.21.15.143:0/2505935202 >> v1:172.21.15.143:6789/0 conn(0x7f16c00d19f0 legacy=0x7f16c00cfbf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:24:50.103 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:50.102+0000 7f16c4e15640 1 -- 172.21.15.143:0/2505935202 --> v2:172.21.15.143:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f16c00ceb20 con 0x7f16c015d6e0 2023-11-26T07:24:50.103 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:50.102+0000 7f16bffff640 1 --2- 172.21.15.143:0/2505935202 >> v2:172.21.15.188:3300/0 conn(0x7f16c00b2160 0x7f16c00d14b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-26T07:24:50.103 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:50.102+0000 7f16bf7fe640 1 -- 172.21.15.143:0/2505935202 <== mon.0 v1:172.21.15.143:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4143894104 0 0) 0x7f16c00cf0e0 con 0x7f16c00d19f0 2023-11-26T07:24:50.105 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:50.102+0000 7f16c4e15640 1 --2- 172.21.15.143:0/2505935202 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f16c015d6e0 0x7f16c00d0df0 secure :-1 s=READY pgs=1422 cs=0 l=1 rev1=1 crypto rx=0x7f16c00ca2a0 tx=0x7f16b401c460 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1af856eae043be3f server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:24:50.105 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:50.106+0000 7f16bf7fe640 1 -- 172.21.15.143:0/2505935202 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f16b4004230 con 0x7f16c015d6e0 2023-11-26T07:24:50.105 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:50.106+0000 7f16bf7fe640 1 -- 172.21.15.143:0/2505935202 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f16b40a7050 con 0x7f16c015d6e0 2023-11-26T07:24:50.105 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:50.106+0000 7f16bf7fe640 1 -- 172.21.15.143:0/2505935202 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f16b4003620 con 0x7f16c015d6e0 2023-11-26T07:24:50.106 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:50.106+0000 7f16c7079640 1 -- 172.21.15.143:0/2505935202 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f16c015d6e0 msgr2=0x7f16c00d0df0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:24:50.106 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:50.106+0000 7f16c7079640 1 --2- 172.21.15.143:0/2505935202 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f16c015d6e0 0x7f16c00d0df0 secure :-1 s=READY pgs=1422 cs=0 l=1 rev1=1 crypto rx=0x7f16c00ca2a0 tx=0x7f16b401c460 comp rx=0 tx=0).stop 2023-11-26T07:24:50.106 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:50.106+0000 7f16c7079640 1 -- 172.21.15.143:0/2505935202 shutdown_connections 2023-11-26T07:24:50.106 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:50.106+0000 7f16c7079640 1 --2- 172.21.15.143:0/2505935202 >> v2:172.21.15.188:3300/0 conn(0x7f16c00b2160 0x7f16c00d14b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:24:50.106 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:50.106+0000 7f16c7079640 1 --2- 172.21.15.143:0/2505935202 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f16c015d6e0 0x7f16c00d0df0 unknown :-1 s=CLOSED pgs=1422 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:24:50.106 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:50.106+0000 7f16c7079640 1 -- 172.21.15.143:0/2505935202 >> 172.21.15.143:0/2505935202 conn(0x7f16c009c550 msgr2=0x7f16c00d3220 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:24:50.106 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:50.106+0000 7f16c7079640 1 -- 172.21.15.143:0/2505935202 shutdown_connections 2023-11-26T07:24:50.107 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:50.106+0000 7f16c7079640 1 -- 172.21.15.143:0/2505935202 wait complete. 2023-11-26T07:24:50.107 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:50.106+0000 7f16c7079640 1 Processor -- start 2023-11-26T07:24:50.107 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:50.106+0000 7f16c7079640 1 -- start start 2023-11-26T07:24:50.110 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:50.106+0000 7f16c7079640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f16c00b2160 0x7f16c00c9600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:24:50.110 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:50.106+0000 7f16c7079640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f16c00d19f0 0x7f16c00b5ff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:24:50.110 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:50.106+0000 7f16c7079640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f16c015d6e0 0x7f16c00b2690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:24:50.110 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:50.106+0000 7f16c7079640 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f16c0153fe0 con 0x7f16c015d6e0 2023-11-26T07:24:50.110 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:50.106+0000 7f16c7079640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f16c0154230 con 0x7f16c00b2160 2023-11-26T07:24:50.110 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:50.106+0000 7f16c7079640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f16c0154510 con 0x7f16c00d19f0 2023-11-26T07:24:50.111 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:50.106+0000 7f16c5616640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f16c015d6e0 0x7f16c00b2690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:24:50.111 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:50.110+0000 7f16c5616640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f16c015d6e0 0x7f16c00b2690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:58722/0 (socket says 172.21.15.143:58722) 2023-11-26T07:24:50.111 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:50.110+0000 7f16c5616640 1 -- 172.21.15.143:0/806968232 learned_addr learned my addr 172.21.15.143:0/806968232 (peer_addr_for_me v2:172.21.15.143:0/0) 2023-11-26T07:24:50.111 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:50.110+0000 7f16c4e15640 1 --2- 172.21.15.143:0/806968232 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f16c00b2160 0x7f16c00c9600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:24:50.111 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:50.110+0000 7f16bffff640 1 --2- 172.21.15.143:0/806968232 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f16c00d19f0 0x7f16c00b5ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:24:50.111 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:50.110+0000 7f16c5616640 1 -- 172.21.15.143:0/806968232 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f16c00d19f0 msgr2=0x7f16c00b5ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:24:50.111 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:50.110+0000 7f16c5616640 1 --2- 172.21.15.143:0/806968232 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f16c00d19f0 0x7f16c00b5ff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:24:50.111 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:50.110+0000 7f16c5616640 1 -- 172.21.15.143:0/806968232 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f16c00b2160 msgr2=0x7f16c00c9600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:24:50.111 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:50.110+0000 7f16c5616640 1 --2- 172.21.15.143:0/806968232 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f16c00b2160 0x7f16c00c9600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:24:50.111 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:50.110+0000 7f16c5616640 1 -- 172.21.15.143:0/806968232 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f16b4009420 con 0x7f16c015d6e0 2023-11-26T07:24:50.111 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:50.110+0000 7f16c4e15640 1 --2- 172.21.15.143:0/806968232 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f16c00b2160 0x7f16c00c9600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-26T07:24:50.111 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:50.110+0000 7f16bffff640 1 --2- 172.21.15.143:0/806968232 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f16c00d19f0 0x7f16c00b5ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-26T07:24:50.112 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:50.110+0000 7f16c5616640 1 --2- 172.21.15.143:0/806968232 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f16c015d6e0 0x7f16c00b2690 secure :-1 s=READY pgs=1423 cs=0 l=1 rev1=1 crypto rx=0x7f16b8001570 tx=0x7f16b800cca0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:24:50.112 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:50.110+0000 7f16bd7fa640 1 -- 172.21.15.143:0/806968232 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f16b800a640 con 0x7f16c015d6e0 2023-11-26T07:24:50.112 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:50.110+0000 7f16bd7fa640 1 -- 172.21.15.143:0/806968232 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f16b800a7c0 con 0x7f16c015d6e0 2023-11-26T07:24:50.112 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:50.110+0000 7f16c7079640 1 -- 172.21.15.143:0/806968232 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f16c01547f0 con 0x7f16c015d6e0 2023-11-26T07:24:50.112 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:50.110+0000 7f16bd7fa640 1 -- 172.21.15.143:0/806968232 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f16b800ab70 con 0x7f16c015d6e0 2023-11-26T07:24:50.112 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:50.110+0000 7f16c7079640 1 -- 172.21.15.143:0/806968232 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f16c0154db0 con 0x7f16c015d6e0 2023-11-26T07:24:50.113 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:50.110+0000 7f16c7079640 1 -- 172.21.15.143:0/806968232 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f16c0154fd0 con 0x7f16c015d6e0 2023-11-26T07:24:50.113 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:50.110+0000 7f16bd7fa640 1 -- 172.21.15.143:0/806968232 <== mon.0 v2:172.21.15.143:3300/0 4 ==== mgrmap(e 18) v1 ==== 86107+0+0 (secure 0 0 0) 0x7f16b800acd0 con 0x7f16c015d6e0 2023-11-26T07:24:50.113 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:50.110+0000 7f16bd7fa640 1 --2- 172.21.15.143:0/806968232 >> [v2:172.21.15.188:6836/3401128553,v1:172.21.15.188:6837/3401128553] conn(0x7f16a4066740 0x7f16a4068c00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:24:50.113 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:50.110+0000 7f16bd7fa640 1 -- 172.21.15.143:0/806968232 <== mon.0 v2:172.21.15.143:3300/0 5 ==== osd_map(103..103 src has 1..103) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f16b8083c00 con 0x7f16c015d6e0 2023-11-26T07:24:50.113 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:50.114+0000 7f16c4e15640 1 --2- 172.21.15.143:0/806968232 >> [v2:172.21.15.188:6836/3401128553,v1:172.21.15.188:6837/3401128553] conn(0x7f16a4066740 0x7f16a4068c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:24:50.114 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:50.114+0000 7f16c4e15640 1 --2- 172.21.15.143:0/806968232 >> [v2:172.21.15.188:6836/3401128553,v1:172.21.15.188:6837/3401128553] conn(0x7f16a4066740 0x7f16a4068c00 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7f16b40ad000 tx=0x7f16b40a6000 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:24:50.117 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:50.118+0000 7f16bd7fa640 1 -- 172.21.15.143:0/806968232 <== mon.0 v2:172.21.15.143:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172649 (secure 0 0 0) 0x7f16b8050c10 con 0x7f16c015d6e0 2023-11-26T07:24:50.301 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:50.302+0000 7f16c7079640 1 -- 172.21.15.143:0/806968232 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f16c0067680 con 0x7f16c015d6e0 2023-11-26T07:24:50.302 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:50.302+0000 7f16bd7fa640 1 -- 172.21.15.143:0/806968232 <== mon.0 v2:172.21.15.143:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 66 v66) v1 ==== 94+0+4927 (secure 0 0 0) 0x7f16b8018020 con 0x7f16c015d6e0 2023-11-26T07:24:50.302 INFO:teuthology.orchestra.run.smithi143.stdout: 2023-11-26T07:24:50.302 INFO:teuthology.orchestra.run.smithi143.stdout:{"epoch":66,"default_fscid":8,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6629,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.188:6834/1327398471","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6832","nonce":1327398471},{"type":"v1","addr":"172.21.15.188:6834","nonce":1327398471}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":60},{"gid":7453,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.188:6835/1640976426","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6833","nonce":1640976426},{"type":"v1","addr":"172.21.15.188:6835","nonce":1640976426}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":60},{"gid":7941,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.143:6834/2855111671","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6832","nonce":2855111671},{"type":"v1","addr":"172.21.15.143:6834","nonce":2855111671}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":60}],"filesystems":[{"mdsmap":{"epoch":65,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-11-26T07:24:08.295882+0000","modified":"2023-11-26T07:24:11.308119+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":7944},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7944":{"gid":7944,"name":"a","rank":0,"incarnation":61,"state":"up:active","state_seq":3,"addr":"172.21.15.143:6835/2571525464","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6833","nonce":2571525464},{"type":"v1","addr":"172.21.15.143:6835","nonce":2571525464}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[17],"metadata_pool":16,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":8}]} 2023-11-26T07:24:50.303 INFO:teuthology.orchestra.run.smithi143.stderr:dumped fsmap epoch 66 2023-11-26T07:24:50.304 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:50.302+0000 7f16c7079640 1 -- 172.21.15.143:0/806968232 >> [v2:172.21.15.188:6836/3401128553,v1:172.21.15.188:6837/3401128553] conn(0x7f16a4066740 msgr2=0x7f16a4068c00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:24:50.304 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:50.302+0000 7f16c7079640 1 --2- 172.21.15.143:0/806968232 >> [v2:172.21.15.188:6836/3401128553,v1:172.21.15.188:6837/3401128553] conn(0x7f16a4066740 0x7f16a4068c00 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7f16b40ad000 tx=0x7f16b40a6000 comp rx=0 tx=0).stop 2023-11-26T07:24:50.304 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:50.302+0000 7f16c7079640 1 -- 172.21.15.143:0/806968232 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f16c015d6e0 msgr2=0x7f16c00b2690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:24:50.304 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:50.302+0000 7f16c7079640 1 --2- 172.21.15.143:0/806968232 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f16c015d6e0 0x7f16c00b2690 secure :-1 s=READY pgs=1423 cs=0 l=1 rev1=1 crypto rx=0x7f16b8001570 tx=0x7f16b800cca0 comp rx=0 tx=0).stop 2023-11-26T07:24:50.304 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:50.302+0000 7f16c7079640 1 -- 172.21.15.143:0/806968232 shutdown_connections 2023-11-26T07:24:50.304 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:50.302+0000 7f16c7079640 1 --2- 172.21.15.143:0/806968232 >> [v2:172.21.15.188:6836/3401128553,v1:172.21.15.188:6837/3401128553] conn(0x7f16a4066740 0x7f16a4068c00 unknown :-1 s=CLOSED pgs=173 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:24:50.304 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:50.302+0000 7f16c7079640 1 --2- 172.21.15.143:0/806968232 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f16c015d6e0 0x7f16c00b2690 unknown :-1 s=CLOSED pgs=1423 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:24:50.305 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:50.302+0000 7f16c7079640 1 --2- 172.21.15.143:0/806968232 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f16c00d19f0 0x7f16c00b5ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:24:50.305 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:50.302+0000 7f16c7079640 1 --2- 172.21.15.143:0/806968232 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f16c00b2160 0x7f16c00c9600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:24:50.305 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:50.302+0000 7f16c7079640 1 -- 172.21.15.143:0/806968232 >> 172.21.15.143:0/806968232 conn(0x7f16c009c550 msgr2=0x7f16c00d3220 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:24:50.305 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:50.302+0000 7f16c7079640 1 -- 172.21.15.143:0/806968232 shutdown_connections 2023-11-26T07:24:50.305 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:50.302+0000 7f16c7079640 1 -- 172.21.15.143:0/806968232 wait complete. 2023-11-26T07:24:50.323 DEBUG:teuthology.orchestra.run.smithi143:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-11-26T07:24:50.461 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:50.462+0000 7fc99b5aa640 1 Processor -- start 2023-11-26T07:24:50.462 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:50.462+0000 7fc99b5aa640 1 -- start start 2023-11-26T07:24:50.465 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:50.462+0000 7fc99b5aa640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fc99406f000 0x7fc99406d030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:24:50.465 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:50.462+0000 7fc99b5aa640 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7fc99406f3e0 0x7fc99406b5a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:24:50.465 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:50.462+0000 7fc99b5aa640 1 -- --> v1:172.21.15.143:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc9940cf260 con 0x7fc994099a70 2023-11-26T07:24:50.465 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:50.462+0000 7fc99b5aa640 1 -- --> v2:172.21.15.188:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc99406d570 con 0x7fc99406f3e0 2023-11-26T07:24:50.465 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:50.462+0000 7fc99b5aa640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc99409a010 con 0x7fc99406f000 2023-11-26T07:24:50.465 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:50.462+0000 7fc999b47640 1 --1- >> v1:172.21.15.143:6789/0 conn(0x7fc994099a70 0x7fc99406bae0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.143:6789/0 says I am v1:172.21.15.143:49726/0 (socket says 172.21.15.143:49726) 2023-11-26T07:24:50.465 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:50.462+0000 7fc999b47640 1 -- 172.21.15.143:0/2193302306 learned_addr learned my addr 172.21.15.143:0/2193302306 (peer_addr_for_me v1:172.21.15.143:0/0) 2023-11-26T07:24:50.465 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:50.462+0000 7fc998b45640 1 --2- 172.21.15.143:0/2193302306 >> v2:172.21.15.188:3300/0 conn(0x7fc99406f3e0 0x7fc99406b5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:24:50.466 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:50.462+0000 7fc999346640 1 --2- 172.21.15.143:0/2193302306 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fc99406f000 0x7fc99406d030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:24:50.466 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:50.462+0000 7fc98bfff640 1 -- 172.21.15.143:0/2193302306 <== mon.0 v1:172.21.15.143:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1718301286 0 0) 0x7fc9940cf260 con 0x7fc994099a70 2023-11-26T07:24:50.466 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:50.462+0000 7fc98bfff640 1 -- 172.21.15.143:0/2193302306 --> v1:172.21.15.143:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fc980003530 con 0x7fc994099a70 2023-11-26T07:24:50.466 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:50.462+0000 7fc98bfff640 1 -- 172.21.15.143:0/2193302306 <== mon.0 v1:172.21.15.143:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3957887172 0 0) 0x7fc980003530 con 0x7fc994099a70 2023-11-26T07:24:50.466 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:50.462+0000 7fc98bfff640 1 -- 172.21.15.143:0/2193302306 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fc99406f000 msgr2=0x7fc99406d030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-26T07:24:50.466 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:50.462+0000 7fc98bfff640 1 --2- 172.21.15.143:0/2193302306 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fc99406f000 0x7fc99406d030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:24:50.466 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:50.462+0000 7fc98bfff640 1 -- 172.21.15.143:0/2193302306 >> v2:172.21.15.188:3300/0 conn(0x7fc99406f3e0 msgr2=0x7fc99406b5a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-26T07:24:50.466 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:50.462+0000 7fc98bfff640 1 --2- 172.21.15.143:0/2193302306 >> v2:172.21.15.188:3300/0 conn(0x7fc99406f3e0 0x7fc99406b5a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:24:50.466 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:50.462+0000 7fc98bfff640 1 -- 172.21.15.143:0/2193302306 --> v1:172.21.15.143:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc99409a2f0 con 0x7fc994099a70 2023-11-26T07:24:50.466 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:50.462+0000 7fc98bfff640 1 -- 172.21.15.143:0/2193302306 <== mon.0 v1:172.21.15.143:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2137882523 0 0) 0x7fc98c001a00 con 0x7fc994099a70 2023-11-26T07:24:50.466 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:50.462+0000 7fc98bfff640 1 -- 172.21.15.143:0/2193302306 >> v1:172.21.15.143:6789/0 conn(0x7fc994099a70 legacy=0x7fc99406bae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:24:50.467 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:50.462+0000 7fc98bfff640 1 --2- 172.21.15.143:0/2193302306 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fc980003980 0x7fc980003da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:24:50.467 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:50.462+0000 7fc98bfff640 1 --2- 172.21.15.143:0/2193302306 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fc99406f000 0x7fc9800047d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:24:50.467 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:50.462+0000 7fc98bfff640 1 --2- 172.21.15.143:0/2193302306 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fc980004d10 0x7fc980005190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:24:50.467 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:50.462+0000 7fc98bfff640 1 -- 172.21.15.143:0/2193302306 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc99406d570 con 0x7fc980003980 2023-11-26T07:24:50.467 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:50.462+0000 7fc98bfff640 1 -- 172.21.15.143:0/2193302306 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc99409a010 con 0x7fc980004d10 2023-11-26T07:24:50.467 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:50.462+0000 7fc98bfff640 1 -- 172.21.15.143:0/2193302306 --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc98000b990 con 0x7fc99406f000 2023-11-26T07:24:50.467 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:50.462+0000 7fc998b45640 1 --2- 172.21.15.143:0/2193302306 >> v2:172.21.15.188:3300/0 conn(0x7fc99406f3e0 0x7fc99406b5a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-26T07:24:50.467 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:50.462+0000 7fc999346640 1 --2- 172.21.15.143:0/2193302306 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fc980003980 0x7fc980003da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:24:50.467 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:50.462+0000 7fc999b47640 1 --2- 172.21.15.143:0/2193302306 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fc980004d10 0x7fc980005190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:24:50.467 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:50.462+0000 7fc998b45640 1 --2- 172.21.15.143:0/2193302306 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fc99406f000 0x7fc9800047d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:24:50.467 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:50.462+0000 7fc999346640 1 -- 172.21.15.143:0/2193302306 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fc99406f000 msgr2=0x7fc9800047d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-26T07:24:50.467 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:50.462+0000 7fc999346640 1 --2- 172.21.15.143:0/2193302306 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fc99406f000 0x7fc9800047d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:24:50.468 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:50.462+0000 7fc999346640 1 -- 172.21.15.143:0/2193302306 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fc980004d10 msgr2=0x7fc980005190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-26T07:24:50.468 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:50.462+0000 7fc999346640 1 --2- 172.21.15.143:0/2193302306 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fc980004d10 0x7fc980005190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:24:50.468 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:50.462+0000 7fc999346640 1 -- 172.21.15.143:0/2193302306 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc980003530 con 0x7fc980003980 2023-11-26T07:24:50.469 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:50.466+0000 7fc999346640 1 --2- 172.21.15.143:0/2193302306 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fc980003980 0x7fc980003da0 secure :-1 s=READY pgs=1425 cs=0 l=1 rev1=1 crypto rx=0x7fc990009ae0 tx=0x7fc99001ca50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=20a8484a0da36830 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:24:50.469 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:50.466+0000 7fc98bfff640 1 -- 172.21.15.143:0/2193302306 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc9900a6430 con 0x7fc980003980 2023-11-26T07:24:50.469 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:50.466+0000 7fc98bfff640 1 -- 172.21.15.143:0/2193302306 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc9900a6590 con 0x7fc980003980 2023-11-26T07:24:50.469 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:50.466+0000 7fc98bfff640 1 -- 172.21.15.143:0/2193302306 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc9900a6940 con 0x7fc980003980 2023-11-26T07:24:50.469 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:50.466+0000 7fc99b5aa640 1 -- 172.21.15.143:0/2193302306 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fc980003980 msgr2=0x7fc980003da0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:24:50.469 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:50.466+0000 7fc99b5aa640 1 --2- 172.21.15.143:0/2193302306 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fc980003980 0x7fc980003da0 secure :-1 s=READY pgs=1425 cs=0 l=1 rev1=1 crypto rx=0x7fc990009ae0 tx=0x7fc99001ca50 comp rx=0 tx=0).stop 2023-11-26T07:24:50.469 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:50.466+0000 7fc999b47640 1 -- 172.21.15.143:0/2193302306 reap_dead start 2023-11-26T07:24:50.469 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:50.466+0000 7fc99b5aa640 1 -- 172.21.15.143:0/2193302306 shutdown_connections 2023-11-26T07:24:50.469 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:50.466+0000 7fc99b5aa640 1 -- 172.21.15.143:0/2193302306 >> 172.21.15.143:0/2193302306 conn(0x7fc9940b2d10 msgr2=0x7fc9940b3120 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:24:50.470 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:50.466+0000 7fc99b5aa640 1 -- 172.21.15.143:0/2193302306 shutdown_connections 2023-11-26T07:24:50.470 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:50.466+0000 7fc99b5aa640 1 -- 172.21.15.143:0/2193302306 wait complete. 2023-11-26T07:24:50.470 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:50.470+0000 7fc99b5aa640 1 Processor -- start 2023-11-26T07:24:50.470 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:50.470+0000 7fc99b5aa640 1 -- start start 2023-11-26T07:24:50.472 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:50.470+0000 7fc99b5aa640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fc980003980 0x7fc994157d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:24:50.472 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:50.470+0000 7fc99b5aa640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fc9941eb5b0 0x7fc9941eba10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:24:50.472 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:50.470+0000 7fc99b5aa640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fc9940c9ed0 0x7fc9940ca350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:24:50.472 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:50.470+0000 7fc99b5aa640 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc9940cb4d0 con 0x7fc980003980 2023-11-26T07:24:50.473 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:50.470+0000 7fc99b5aa640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc9941e9920 con 0x7fc9941eb5b0 2023-11-26T07:24:50.473 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:50.470+0000 7fc99b5aa640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc9941ebf50 con 0x7fc9940c9ed0 2023-11-26T07:24:50.473 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:50.470+0000 7fc999346640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fc980003980 0x7fc994157d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:24:50.473 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:50.470+0000 7fc999346640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fc980003980 0x7fc994157d10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:58736/0 (socket says 172.21.15.143:58736) 2023-11-26T07:24:50.473 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:50.470+0000 7fc999346640 1 -- 172.21.15.143:0/562983309 learned_addr learned my addr 172.21.15.143:0/562983309 (peer_addr_for_me v2:172.21.15.143:0/0) 2023-11-26T07:24:50.473 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:50.470+0000 7fc999b47640 1 --2- 172.21.15.143:0/562983309 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fc9940c9ed0 0x7fc9940ca350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:24:50.473 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:50.470+0000 7fc998b45640 1 --2- 172.21.15.143:0/562983309 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fc9941eb5b0 0x7fc9941eba10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:24:50.473 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:50.470+0000 7fc999346640 1 -- 172.21.15.143:0/562983309 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fc9940c9ed0 msgr2=0x7fc9940ca350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:24:50.473 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:50.470+0000 7fc999346640 1 --2- 172.21.15.143:0/562983309 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fc9940c9ed0 0x7fc9940ca350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:24:50.473 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:50.470+0000 7fc999346640 1 -- 172.21.15.143:0/562983309 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fc9941eb5b0 msgr2=0x7fc9941eba10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:24:50.473 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:50.470+0000 7fc999346640 1 --2- 172.21.15.143:0/562983309 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fc9941eb5b0 0x7fc9941eba10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:24:50.474 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:50.470+0000 7fc999346640 1 -- 172.21.15.143:0/562983309 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc9940cac30 con 0x7fc980003980 2023-11-26T07:24:50.474 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:50.470+0000 7fc998b45640 1 --2- 172.21.15.143:0/562983309 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fc9941eb5b0 0x7fc9941eba10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-26T07:24:50.474 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:50.470+0000 7fc999346640 1 --2- 172.21.15.143:0/562983309 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fc980003980 0x7fc994157d10 secure :-1 s=READY pgs=1426 cs=0 l=1 rev1=1 crypto rx=0x7fc99001c880 tx=0x7fc980005800 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:24:50.474 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:50.470+0000 7fc989ffb640 1 -- 172.21.15.143:0/562983309 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc99000f000 con 0x7fc980003980 2023-11-26T07:24:50.474 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:50.470+0000 7fc989ffb640 1 -- 172.21.15.143:0/562983309 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc9900a6d80 con 0x7fc980003980 2023-11-26T07:24:50.474 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:50.470+0000 7fc99b5aa640 1 -- 172.21.15.143:0/562983309 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc9940caf10 con 0x7fc980003980 2023-11-26T07:24:50.474 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:50.470+0000 7fc989ffb640 1 -- 172.21.15.143:0/562983309 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc9900b9630 con 0x7fc980003980 2023-11-26T07:24:50.475 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:50.470+0000 7fc99b5aa640 1 -- 172.21.15.143:0/562983309 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc9940ca890 con 0x7fc980003980 2023-11-26T07:24:50.475 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:50.474+0000 7fc99b5aa640 1 -- 172.21.15.143:0/562983309 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc9941e96f0 con 0x7fc980003980 2023-11-26T07:24:50.475 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:50.474+0000 7fc989ffb640 1 -- 172.21.15.143:0/562983309 <== mon.0 v2:172.21.15.143:3300/0 4 ==== mgrmap(e 18) v1 ==== 86107+0+0 (secure 0 0 0) 0x7fc9900a7410 con 0x7fc980003980 2023-11-26T07:24:50.476 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:50.474+0000 7fc989ffb640 1 --2- 172.21.15.143:0/562983309 >> [v2:172.21.15.188:6836/3401128553,v1:172.21.15.188:6837/3401128553] conn(0x7fc978066740 0x7fc978068c00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:24:50.476 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:50.474+0000 7fc989ffb640 1 -- 172.21.15.143:0/562983309 <== mon.0 v2:172.21.15.143:3300/0 5 ==== osd_map(103..103 src has 1..103) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fc99011dd60 con 0x7fc980003980 2023-11-26T07:24:50.476 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:50.474+0000 7fc998b45640 1 --2- 172.21.15.143:0/562983309 >> [v2:172.21.15.188:6836/3401128553,v1:172.21.15.188:6837/3401128553] conn(0x7fc978066740 0x7fc978068c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:24:50.477 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:50.478+0000 7fc998b45640 1 --2- 172.21.15.143:0/562983309 >> [v2:172.21.15.188:6836/3401128553,v1:172.21.15.188:6837/3401128553] conn(0x7fc978066740 0x7fc978068c00 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7fc9941f91d0 tx=0x7fc984008040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:24:50.480 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:50.478+0000 7fc989ffb640 1 -- 172.21.15.143:0/562983309 <== mon.0 v2:172.21.15.143:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172649 (secure 0 0 0) 0x7fc9941e96f0 con 0x7fc980003980 2023-11-26T07:24:50.661 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:50.662+0000 7fc99b5aa640 1 -- 172.21.15.143:0/562983309 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fc9941e96f0 con 0x7fc980003980 2023-11-26T07:24:50.662 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:50.662+0000 7fc989ffb640 1 -- 172.21.15.143:0/562983309 <== mon.0 v2:172.21.15.143:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 66 v66) v1 ==== 94+0+4927 (secure 0 0 0) 0x7fc9941e96f0 con 0x7fc980003980 2023-11-26T07:24:50.662 INFO:teuthology.orchestra.run.smithi143.stdout: 2023-11-26T07:24:50.662 INFO:teuthology.orchestra.run.smithi143.stdout:{"epoch":66,"default_fscid":8,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6629,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.188:6834/1327398471","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6832","nonce":1327398471},{"type":"v1","addr":"172.21.15.188:6834","nonce":1327398471}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":60},{"gid":7453,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.188:6835/1640976426","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6833","nonce":1640976426},{"type":"v1","addr":"172.21.15.188:6835","nonce":1640976426}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":60},{"gid":7941,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.143:6834/2855111671","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6832","nonce":2855111671},{"type":"v1","addr":"172.21.15.143:6834","nonce":2855111671}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":60}],"filesystems":[{"mdsmap":{"epoch":65,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-11-26T07:24:08.295882+0000","modified":"2023-11-26T07:24:11.308119+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":7944},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7944":{"gid":7944,"name":"a","rank":0,"incarnation":61,"state":"up:active","state_seq":3,"addr":"172.21.15.143:6835/2571525464","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6833","nonce":2571525464},{"type":"v1","addr":"172.21.15.143:6835","nonce":2571525464}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[17],"metadata_pool":16,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":8}]} 2023-11-26T07:24:50.663 INFO:teuthology.orchestra.run.smithi143.stderr:dumped fsmap epoch 66 2023-11-26T07:24:50.664 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:50.662+0000 7fc99b5aa640 1 -- 172.21.15.143:0/562983309 >> [v2:172.21.15.188:6836/3401128553,v1:172.21.15.188:6837/3401128553] conn(0x7fc978066740 msgr2=0x7fc978068c00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:24:50.664 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:50.662+0000 7fc99b5aa640 1 --2- 172.21.15.143:0/562983309 >> [v2:172.21.15.188:6836/3401128553,v1:172.21.15.188:6837/3401128553] conn(0x7fc978066740 0x7fc978068c00 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7fc9941f91d0 tx=0x7fc984008040 comp rx=0 tx=0).stop 2023-11-26T07:24:50.664 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:50.662+0000 7fc99b5aa640 1 -- 172.21.15.143:0/562983309 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fc980003980 msgr2=0x7fc994157d10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:24:50.664 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:50.662+0000 7fc99b5aa640 1 --2- 172.21.15.143:0/562983309 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fc980003980 0x7fc994157d10 secure :-1 s=READY pgs=1426 cs=0 l=1 rev1=1 crypto rx=0x7fc99001c880 tx=0x7fc980005800 comp rx=0 tx=0).stop 2023-11-26T07:24:50.664 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:50.662+0000 7fc99b5aa640 1 -- 172.21.15.143:0/562983309 shutdown_connections 2023-11-26T07:24:50.664 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:50.662+0000 7fc99b5aa640 1 --2- 172.21.15.143:0/562983309 >> [v2:172.21.15.188:6836/3401128553,v1:172.21.15.188:6837/3401128553] conn(0x7fc978066740 0x7fc978068c00 unknown :-1 s=CLOSED pgs=174 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:24:50.665 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:50.662+0000 7fc99b5aa640 1 --2- 172.21.15.143:0/562983309 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fc9940c9ed0 0x7fc9940ca350 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:24:50.665 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:50.662+0000 7fc99b5aa640 1 --2- 172.21.15.143:0/562983309 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fc9941eb5b0 0x7fc9941eba10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:24:50.665 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:50.662+0000 7fc99b5aa640 1 --2- 172.21.15.143:0/562983309 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fc980003980 0x7fc994157d10 unknown :-1 s=CLOSED pgs=1426 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:24:50.665 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:50.662+0000 7fc99b5aa640 1 -- 172.21.15.143:0/562983309 >> 172.21.15.143:0/562983309 conn(0x7fc9940b2d10 msgr2=0x7fc9940b4070 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:24:50.665 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:50.662+0000 7fc99b5aa640 1 -- 172.21.15.143:0/562983309 shutdown_connections 2023-11-26T07:24:50.665 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:50.662+0000 7fc99b5aa640 1 -- 172.21.15.143:0/562983309 wait complete. 2023-11-26T07:24:50.684 DEBUG:teuthology.orchestra.run.smithi143:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-11-26T07:24:50.821 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:50.818+0000 7fd9bc335640 1 Processor -- start 2023-11-26T07:24:50.821 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:50.818+0000 7fd9bc335640 1 -- start start 2023-11-26T07:24:50.823 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:50.822+0000 7fd9bc335640 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7fd9b413e290 0x7fd9b40cf280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:24:50.823 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:50.822+0000 7fd9bc335640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fd9b40cf7c0 0x7fd9b415efa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:24:50.823 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:50.822+0000 7fd9bc335640 1 -- --> v1:172.21.15.188:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd9b406c520 con 0x7fd9b415f4e0 2023-11-26T07:24:50.823 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:50.822+0000 7fd9bc335640 1 -- --> v2:172.21.15.143:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd9b40cfc40 con 0x7fd9b413e290 2023-11-26T07:24:50.823 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:50.822+0000 7fd9bc335640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd9b406c800 con 0x7fd9b40cf7c0 2023-11-26T07:24:50.823 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:50.822+0000 7fd9ba0d1640 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7fd9b413e290 0x7fd9b40cf280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:24:50.823 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:50.822+0000 7fd9ba0d1640 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7fd9b413e290 0x7fd9b40cf280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:58752/0 (socket says 172.21.15.143:58752) 2023-11-26T07:24:50.824 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:50.822+0000 7fd9ba0d1640 1 -- 172.21.15.143:0/428168892 learned_addr learned my addr 172.21.15.143:0/428168892 (peer_addr_for_me v2:172.21.15.143:0/0) 2023-11-26T07:24:50.824 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:50.822+0000 7fd9b98d0640 1 --2- 172.21.15.143:0/428168892 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fd9b40cf7c0 0x7fd9b415efa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:24:50.824 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:50.822+0000 7fd9ba0d1640 1 -- 172.21.15.143:0/428168892 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fd9b40cf7c0 msgr2=0x7fd9b415efa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-26T07:24:50.824 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:50.822+0000 7fd9ba0d1640 1 --2- 172.21.15.143:0/428168892 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fd9b40cf7c0 0x7fd9b415efa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:24:50.824 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:50.822+0000 7fd9ba0d1640 1 -- 172.21.15.143:0/428168892 >> v1:172.21.15.188:6789/0 conn(0x7fd9b415f4e0 legacy=0x7fd9b41408e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:24:50.824 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:50.822+0000 7fd9ba0d1640 1 -- 172.21.15.143:0/428168892 --> v2:172.21.15.143:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd9b406cae0 con 0x7fd9b413e290 2023-11-26T07:24:50.824 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:50.822+0000 7fd9b98d0640 1 --2- 172.21.15.143:0/428168892 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fd9b40cf7c0 0x7fd9b415efa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-26T07:24:50.826 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:50.826+0000 7fd9ba0d1640 1 --2- 172.21.15.143:0/428168892 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fd9b413e290 0x7fd9b40cf280 secure :-1 s=READY pgs=1427 cs=0 l=1 rev1=1 crypto rx=0x7fd9b0009870 tx=0x7fd9b001c510 comp rx=0 tx=0).ready entity=mon.0 client_cookie=275c213d05a47e4e server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:24:50.826 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:50.826+0000 7fd9b90cf640 1 -- 172.21.15.143:0/428168892 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd9b00aa670 con 0x7fd9b413e290 2023-11-26T07:24:50.826 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:50.826+0000 7fd9b90cf640 1 -- 172.21.15.143:0/428168892 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd9b00aa7d0 con 0x7fd9b413e290 2023-11-26T07:24:50.826 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:50.826+0000 7fd9b90cf640 1 -- 172.21.15.143:0/428168892 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd9b00aab60 con 0x7fd9b413e290 2023-11-26T07:24:50.827 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:50.826+0000 7fd9bc335640 1 -- 172.21.15.143:0/428168892 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fd9b413e290 msgr2=0x7fd9b40cf280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:24:50.827 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:50.826+0000 7fd9bc335640 1 --2- 172.21.15.143:0/428168892 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fd9b413e290 0x7fd9b40cf280 secure :-1 s=READY pgs=1427 cs=0 l=1 rev1=1 crypto rx=0x7fd9b0009870 tx=0x7fd9b001c510 comp rx=0 tx=0).stop 2023-11-26T07:24:50.827 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:50.826+0000 7fd9bc335640 1 -- 172.21.15.143:0/428168892 shutdown_connections 2023-11-26T07:24:50.828 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:50.826+0000 7fd9bc335640 1 --2- 172.21.15.143:0/428168892 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fd9b40cf7c0 0x7fd9b415efa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:24:50.828 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:50.826+0000 7fd9bc335640 1 --2- 172.21.15.143:0/428168892 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fd9b413e290 0x7fd9b40cf280 unknown :-1 s=CLOSED pgs=1427 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:24:50.828 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:50.826+0000 7fd9bc335640 1 -- 172.21.15.143:0/428168892 >> 172.21.15.143:0/428168892 conn(0x7fd9b406a890 msgr2=0x7fd9b406aca0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:24:50.828 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:50.826+0000 7fd9bc335640 1 -- 172.21.15.143:0/428168892 shutdown_connections 2023-11-26T07:24:50.828 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:50.826+0000 7fd9bc335640 1 -- 172.21.15.143:0/428168892 wait complete. 2023-11-26T07:24:50.829 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:50.826+0000 7fd9bc335640 1 Processor -- start 2023-11-26T07:24:50.829 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:50.826+0000 7fd9bc335640 1 -- start start 2023-11-26T07:24:50.830 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:50.830+0000 7fd9bc335640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fd9b40cf7c0 0x7fd9b415daf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:24:50.831 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:50.830+0000 7fd9bc335640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fd9b413e290 0x7fd9b4137770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:24:50.831 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:50.830+0000 7fd9bc335640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fd9b415f4e0 0x7fd9b41f9300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:24:50.831 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:50.830+0000 7fd9bc335640 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd9b4141c60 con 0x7fd9b413e290 2023-11-26T07:24:50.831 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:50.830+0000 7fd9bc335640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd9b415d010 con 0x7fd9b40cf7c0 2023-11-26T07:24:50.831 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:50.830+0000 7fd9bc335640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd9b415d2f0 con 0x7fd9b415f4e0 2023-11-26T07:24:50.831 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:50.830+0000 7fd9b98d0640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fd9b413e290 0x7fd9b4137770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:24:50.831 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:50.830+0000 7fd9b98d0640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fd9b413e290 0x7fd9b4137770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:58756/0 (socket says 172.21.15.143:58756) 2023-11-26T07:24:50.831 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:50.830+0000 7fd9b98d0640 1 -- 172.21.15.143:0/128673901 learned_addr learned my addr 172.21.15.143:0/128673901 (peer_addr_for_me v2:172.21.15.143:0/0) 2023-11-26T07:24:50.831 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:50.830+0000 7fd9ba0d1640 1 --2- 172.21.15.143:0/128673901 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fd9b40cf7c0 0x7fd9b415daf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:24:50.831 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:50.830+0000 7fd9ba8d2640 1 --2- 172.21.15.143:0/128673901 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fd9b415f4e0 0x7fd9b41f9300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:24:50.831 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:50.830+0000 7fd9b98d0640 1 -- 172.21.15.143:0/128673901 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fd9b415f4e0 msgr2=0x7fd9b41f9300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:24:50.832 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:50.830+0000 7fd9b98d0640 1 --2- 172.21.15.143:0/128673901 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fd9b415f4e0 0x7fd9b41f9300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:24:50.832 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:50.830+0000 7fd9b98d0640 1 -- 172.21.15.143:0/128673901 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fd9b40cf7c0 msgr2=0x7fd9b415daf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:24:50.832 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:50.830+0000 7fd9b98d0640 1 --2- 172.21.15.143:0/128673901 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fd9b40cf7c0 0x7fd9b415daf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:24:50.832 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:50.830+0000 7fd9b98d0640 1 -- 172.21.15.143:0/128673901 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd9b00094d0 con 0x7fd9b413e290 2023-11-26T07:24:50.832 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:50.830+0000 7fd9b98d0640 1 --2- 172.21.15.143:0/128673901 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fd9b413e290 0x7fd9b4137770 secure :-1 s=READY pgs=1428 cs=0 l=1 rev1=1 crypto rx=0x7fd9a400e810 tx=0x7fd9a400ecd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:24:50.833 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:50.830+0000 7fd9aaffd640 1 -- 172.21.15.143:0/128673901 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd9a400c8d0 con 0x7fd9b413e290 2023-11-26T07:24:50.833 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:50.830+0000 7fd9aaffd640 1 -- 172.21.15.143:0/128673901 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd9a400ca30 con 0x7fd9b413e290 2023-11-26T07:24:50.833 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:50.830+0000 7fd9aaffd640 1 -- 172.21.15.143:0/128673901 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd9a400cdc0 con 0x7fd9b413e290 2023-11-26T07:24:50.833 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:50.830+0000 7fd9bc335640 1 -- 172.21.15.143:0/128673901 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd9b415d5d0 con 0x7fd9b413e290 2023-11-26T07:24:50.833 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:50.830+0000 7fd9bc335640 1 -- 172.21.15.143:0/128673901 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd9b4141f40 con 0x7fd9b413e290 2023-11-26T07:24:50.833 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:50.830+0000 7fd9bc335640 1 -- 172.21.15.143:0/128673901 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd9b4067680 con 0x7fd9b413e290 2023-11-26T07:24:50.833 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:50.830+0000 7fd9aaffd640 1 -- 172.21.15.143:0/128673901 <== mon.0 v2:172.21.15.143:3300/0 4 ==== mgrmap(e 18) v1 ==== 86107+0+0 (secure 0 0 0) 0x7fd9a4013410 con 0x7fd9b413e290 2023-11-26T07:24:50.834 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:50.834+0000 7fd9aaffd640 1 --2- 172.21.15.143:0/128673901 >> [v2:172.21.15.188:6836/3401128553,v1:172.21.15.188:6837/3401128553] conn(0x7fd998066740 0x7fd998068c00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:24:50.834 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:50.834+0000 7fd9aaffd640 1 -- 172.21.15.143:0/128673901 <== mon.0 v2:172.21.15.143:3300/0 5 ==== osd_map(103..103 src has 1..103) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fd9a40841b0 con 0x7fd9b413e290 2023-11-26T07:24:50.834 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:50.834+0000 7fd9ba0d1640 1 --2- 172.21.15.143:0/128673901 >> [v2:172.21.15.188:6836/3401128553,v1:172.21.15.188:6837/3401128553] conn(0x7fd998066740 0x7fd998068c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:24:50.835 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:50.834+0000 7fd9ba0d1640 1 --2- 172.21.15.143:0/128673901 >> [v2:172.21.15.188:6836/3401128553,v1:172.21.15.188:6837/3401128553] conn(0x7fd998066740 0x7fd998068c00 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7fd9b00ae000 tx=0x7fd9b00a7000 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:24:50.837 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:50.838+0000 7fd9aaffd640 1 -- 172.21.15.143:0/128673901 <== mon.0 v2:172.21.15.143:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172649 (secure 0 0 0) 0x7fd9a40511d0 con 0x7fd9b413e290 2023-11-26T07:24:50.988 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:50.986+0000 7fd9bc335640 1 -- 172.21.15.143:0/128673901 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fd9b406df50 con 0x7fd9b413e290 2023-11-26T07:24:50.990 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:50.990+0000 7fd9aaffd640 1 -- 172.21.15.143:0/128673901 <== mon.0 v2:172.21.15.143:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v103) v1 ==== 74+0+16563 (secure 0 0 0) 0x7fd9a4009480 con 0x7fd9b413e290 2023-11-26T07:24:50.990 INFO:teuthology.orchestra.run.smithi143.stdout: 2023-11-26T07:24:50.990 INFO:teuthology.orchestra.run.smithi143.stdout:{"epoch":103,"fsid":"f5033060-2469-4a07-95fe-2b76f5404bc1","created":"2023-11-26T07:17:28.613663+0000","modified":"2023-11-26T07:24:09.510299+0000","last_up_change":"2023-11-26T07:17:41.157813+0000","last_in_change":"2023-11-26T07:17:31.181658+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":17,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-11-26T07:17:41.758597+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":16,"pool_name":"cephfs_metadata","create_time":"2023-11-26T07:24:06.278132+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"102","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":17,"pool_name":"cephfs_data","create_time":"2023-11-26T07:24:07.294850+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"103","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"ebbee2c6-023e-44b9-9f1d-6f7bfd309730","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":101,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6817","nonce":639807369},{"type":"v1","addr":"172.21.15.143:6819","nonce":639807369}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6821","nonce":639807369},{"type":"v1","addr":"172.21.15.143:6823","nonce":639807369}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6830","nonce":639807369},{"type":"v1","addr":"172.21.15.143:6831","nonce":639807369}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6825","nonce":639807369},{"type":"v1","addr":"172.21.15.143:6828","nonce":639807369}]},"public_addr":"172.21.15.143:6819/639807369","cluster_addr":"172.21.15.143:6823/639807369","heartbeat_back_addr":"172.21.15.143:6831/639807369","heartbeat_front_addr":"172.21.15.143:6828/639807369","state":["exists","up"]},{"osd":1,"uuid":"fec4b8be-7122-49e9-9a19-5850a0700001","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":101,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6816","nonce":1144942530},{"type":"v1","addr":"172.21.15.143:6818","nonce":1144942530}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6820","nonce":1144942530},{"type":"v1","addr":"172.21.15.143:6822","nonce":1144942530}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6827","nonce":1144942530},{"type":"v1","addr":"172.21.15.143:6829","nonce":1144942530}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6824","nonce":1144942530},{"type":"v1","addr":"172.21.15.143:6826","nonce":1144942530}]},"public_addr":"172.21.15.143:6818/1144942530","cluster_addr":"172.21.15.143:6822/1144942530","heartbeat_back_addr":"172.21.15.143:6829/1144942530","heartbeat_front_addr":"172.21.15.143:6826/1144942530","state":["exists","up"]},{"osd":2,"uuid":"2888af5b-f933-46ca-8f95-6dd494ff43f6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":101,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6808","nonce":3909095186},{"type":"v1","addr":"172.21.15.143:6809","nonce":3909095186}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6810","nonce":3909095186},{"type":"v1","addr":"172.21.15.143:6811","nonce":3909095186}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6814","nonce":3909095186},{"type":"v1","addr":"172.21.15.143:6815","nonce":3909095186}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6812","nonce":3909095186},{"type":"v1","addr":"172.21.15.143:6813","nonce":3909095186}]},"public_addr":"172.21.15.143:6809/3909095186","cluster_addr":"172.21.15.143:6811/3909095186","heartbeat_back_addr":"172.21.15.143:6815/3909095186","heartbeat_front_addr":"172.21.15.143:6813/3909095186","state":["exists","up"]},{"osd":3,"uuid":"d1f52435-7bc1-4081-9550-9cd1ba55f549","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":101,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6800","nonce":3854210991},{"type":"v1","addr":"172.21.15.143:6801","nonce":3854210991}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6802","nonce":3854210991},{"type":"v1","addr":"172.21.15.143:6803","nonce":3854210991}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6806","nonce":3854210991},{"type":"v1","addr":"172.21.15.143:6807","nonce":3854210991}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6804","nonce":3854210991},{"type":"v1","addr":"172.21.15.143:6805","nonce":3854210991}]},"public_addr":"172.21.15.143:6801/3854210991","cluster_addr":"172.21.15.143:6803/3854210991","heartbeat_back_addr":"172.21.15.143:6807/3854210991","heartbeat_front_addr":"172.21.15.143:6805/3854210991","state":["exists","up"]},{"osd":4,"uuid":"e2ae8b6e-6e25-49b2-bcbf-aacce68f06a3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":101,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6800","nonce":3924606902},{"type":"v1","addr":"172.21.15.188:6801","nonce":3924606902}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6802","nonce":3924606902},{"type":"v1","addr":"172.21.15.188:6803","nonce":3924606902}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6806","nonce":3924606902},{"type":"v1","addr":"172.21.15.188:6807","nonce":3924606902}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6804","nonce":3924606902},{"type":"v1","addr":"172.21.15.188:6805","nonce":3924606902}]},"public_addr":"172.21.15.188:6801/3924606902","cluster_addr":"172.21.15.188:6803/3924606902","heartbeat_back_addr":"172.21.15.188:6807/3924606902","heartbeat_front_addr":"172.21.15.188:6805/3924606902","state":["exists","up"]},{"osd":5,"uuid":"3cec8a32-7698-4c36-8e89-e4ba11934f33","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":101,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6813","nonce":186841544},{"type":"v1","addr":"172.21.15.188:6815","nonce":186841544}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6817","nonce":186841544},{"type":"v1","addr":"172.21.15.188:6818","nonce":186841544}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6822","nonce":186841544},{"type":"v1","addr":"172.21.15.188:6823","nonce":186841544}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6820","nonce":186841544},{"type":"v1","addr":"172.21.15.188:6821","nonce":186841544}]},"public_addr":"172.21.15.188:6815/186841544","cluster_addr":"172.21.15.188:6818/186841544","heartbeat_back_addr":"172.21.15.188:6823/186841544","heartbeat_front_addr":"172.21.15.188:6821/186841544","state":["exists","up"]},{"osd":6,"uuid":"58a65d5d-e090-473a-81b4-ad38e85029f9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":101,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6808","nonce":2153499405},{"type":"v1","addr":"172.21.15.188:6809","nonce":2153499405}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6810","nonce":2153499405},{"type":"v1","addr":"172.21.15.188:6811","nonce":2153499405}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6816","nonce":2153499405},{"type":"v1","addr":"172.21.15.188:6819","nonce":2153499405}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6812","nonce":2153499405},{"type":"v1","addr":"172.21.15.188:6814","nonce":2153499405}]},"public_addr":"172.21.15.188:6809/2153499405","cluster_addr":"172.21.15.188:6811/2153499405","heartbeat_back_addr":"172.21.15.188:6819/2153499405","heartbeat_front_addr":"172.21.15.188:6814/2153499405","state":["exists","up"]},{"osd":7,"uuid":"ea5b7f70-3554-4cf8-ac30-f7c59b80678d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":101,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6824","nonce":2722096729},{"type":"v1","addr":"172.21.15.188:6825","nonce":2722096729}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6826","nonce":2722096729},{"type":"v1","addr":"172.21.15.188:6827","nonce":2722096729}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6830","nonce":2722096729},{"type":"v1","addr":"172.21.15.188:6831","nonce":2722096729}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6828","nonce":2722096729},{"type":"v1","addr":"172.21.15.188:6829","nonce":2722096729}]},"public_addr":"172.21.15.188:6825/2722096729","cluster_addr":"172.21.15.188:6827/2722096729","heartbeat_back_addr":"172.21.15.188:6831/2722096729","heartbeat_front_addr":"172.21.15.188:6829/2722096729","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-26T07:17:36.995032+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-26T07:17:36.932266+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-26T07:17:36.829304+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-26T07:17:36.804297+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-26T07:17:36.667892+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-26T07:17:36.964868+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-26T07:17:36.827332+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-26T07:17:36.893179+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-11-26T07:24:50.992 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:50.990+0000 7fd9bc335640 1 -- 172.21.15.143:0/128673901 >> [v2:172.21.15.188:6836/3401128553,v1:172.21.15.188:6837/3401128553] conn(0x7fd998066740 msgr2=0x7fd998068c00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:24:50.992 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:50.990+0000 7fd9bc335640 1 --2- 172.21.15.143:0/128673901 >> [v2:172.21.15.188:6836/3401128553,v1:172.21.15.188:6837/3401128553] conn(0x7fd998066740 0x7fd998068c00 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7fd9b00ae000 tx=0x7fd9b00a7000 comp rx=0 tx=0).stop 2023-11-26T07:24:50.992 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:50.990+0000 7fd9bc335640 1 -- 172.21.15.143:0/128673901 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fd9b413e290 msgr2=0x7fd9b4137770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:24:50.992 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:50.990+0000 7fd9bc335640 1 --2- 172.21.15.143:0/128673901 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fd9b413e290 0x7fd9b4137770 secure :-1 s=READY pgs=1428 cs=0 l=1 rev1=1 crypto rx=0x7fd9a400e810 tx=0x7fd9a400ecd0 comp rx=0 tx=0).stop 2023-11-26T07:24:50.992 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:50.990+0000 7fd9bc335640 1 -- 172.21.15.143:0/128673901 shutdown_connections 2023-11-26T07:24:50.992 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:50.990+0000 7fd9bc335640 1 --2- 172.21.15.143:0/128673901 >> [v2:172.21.15.188:6836/3401128553,v1:172.21.15.188:6837/3401128553] conn(0x7fd998066740 0x7fd998068c00 unknown :-1 s=CLOSED pgs=175 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:24:50.992 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:50.990+0000 7fd9bc335640 1 --2- 172.21.15.143:0/128673901 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fd9b415f4e0 0x7fd9b41f9300 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:24:50.992 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:50.990+0000 7fd9bc335640 1 --2- 172.21.15.143:0/128673901 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fd9b413e290 0x7fd9b4137770 unknown :-1 s=CLOSED pgs=1428 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:24:50.993 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:50.990+0000 7fd9bc335640 1 --2- 172.21.15.143:0/128673901 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fd9b40cf7c0 0x7fd9b415daf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:24:50.993 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:50.990+0000 7fd9bc335640 1 -- 172.21.15.143:0/128673901 >> 172.21.15.143:0/128673901 conn(0x7fd9b406a890 msgr2=0x7fd9b415a150 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:24:50.993 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:50.990+0000 7fd9bc335640 1 -- 172.21.15.143:0/128673901 shutdown_connections 2023-11-26T07:24:50.993 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:50.990+0000 7fd9bc335640 1 -- 172.21.15.143:0/128673901 wait complete. 2023-11-26T07:24:51.011 DEBUG:teuthology.orchestra.run.smithi143:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs fail cephfs 2023-11-26T07:24:51.149 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:51.150+0000 7f49ecf1d640 1 Processor -- start 2023-11-26T07:24:51.149 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:51.150+0000 7f49ecf1d640 1 -- start start 2023-11-26T07:24:51.151 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:51.150+0000 7f49ecf1d640 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7f49e8160aa0 0x7f49e80b2b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:24:51.151 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:51.150+0000 7f49ecf1d640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f49e80b2240 0x7f49e80d0590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:24:51.151 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:51.150+0000 7f49ecf1d640 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7f49e80d14e0 0x7f49e80d1970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:24:51.152 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:51.150+0000 7f49ecf1d640 1 -- --> v2:172.21.15.143:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f49e80ce440 con 0x7f49e80d14e0 2023-11-26T07:24:51.152 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:51.150+0000 7f49ecf1d640 1 -- --> v2:172.21.15.188:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f49e80d1eb0 con 0x7f49e8160aa0 2023-11-26T07:24:51.152 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:51.150+0000 7f49ecf1d640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f49e80d4140 con 0x7f49e80b2240 2023-11-26T07:24:51.152 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:51.150+0000 7f49e6d9d640 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7f49e80d14e0 0x7f49e80d1970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:24:51.152 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:51.150+0000 7f49e6d9d640 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7f49e80d14e0 0x7f49e80d1970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:58768/0 (socket says 172.21.15.143:58768) 2023-11-26T07:24:51.152 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:51.150+0000 7f49e6d9d640 1 -- 172.21.15.143:0/164615758 learned_addr learned my addr 172.21.15.143:0/164615758 (peer_addr_for_me v2:172.21.15.143:0/0) 2023-11-26T07:24:51.152 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:51.150+0000 7f49dffff640 1 --2- 172.21.15.143:0/164615758 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f49e80b2240 0x7f49e80d0590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:24:51.152 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:51.150+0000 7f49e659c640 1 --2- 172.21.15.143:0/164615758 >> v2:172.21.15.188:3300/0 conn(0x7f49e8160aa0 0x7f49e80b2b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:24:51.152 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:51.150+0000 7f49e6d9d640 1 -- 172.21.15.143:0/164615758 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f49e80b2240 msgr2=0x7f49e80d0590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-26T07:24:51.152 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:51.150+0000 7f49e6d9d640 1 --2- 172.21.15.143:0/164615758 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f49e80b2240 0x7f49e80d0590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:24:51.152 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:51.150+0000 7f49e6d9d640 1 -- 172.21.15.143:0/164615758 >> v2:172.21.15.188:3300/0 conn(0x7f49e8160aa0 msgr2=0x7f49e80b2b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-26T07:24:51.152 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:51.150+0000 7f49e6d9d640 1 --2- 172.21.15.143:0/164615758 >> v2:172.21.15.188:3300/0 conn(0x7f49e8160aa0 0x7f49e80b2b80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:24:51.152 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:51.150+0000 7f49e6d9d640 1 -- 172.21.15.143:0/164615758 --> v2:172.21.15.143:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f49e80d3930 con 0x7f49e80d14e0 2023-11-26T07:24:51.153 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:51.150+0000 7f49dffff640 1 --2- 172.21.15.143:0/164615758 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f49e80b2240 0x7f49e80d0590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-26T07:24:51.154 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:51.154+0000 7f49e6d9d640 1 --2- 172.21.15.143:0/164615758 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f49e80d14e0 0x7f49e80d1970 secure :-1 s=READY pgs=1429 cs=0 l=1 rev1=1 crypto rx=0x7f49e808b590 tx=0x7f49e001c470 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a70c4ce094e7d2ef server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:24:51.154 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:51.154+0000 7f49e5d9b640 1 -- 172.21.15.143:0/164615758 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f49e00ab600 con 0x7f49e80d14e0 2023-11-26T07:24:51.154 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:51.154+0000 7f49e5d9b640 1 -- 172.21.15.143:0/164615758 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f49e00a7050 con 0x7f49e80d14e0 2023-11-26T07:24:51.155 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:51.154+0000 7f49e5d9b640 1 -- 172.21.15.143:0/164615758 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f49e00ab980 con 0x7f49e80d14e0 2023-11-26T07:24:51.155 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:51.154+0000 7f49ecf1d640 1 -- 172.21.15.143:0/164615758 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f49e80d14e0 msgr2=0x7f49e80d1970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:24:51.155 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:51.154+0000 7f49ecf1d640 1 --2- 172.21.15.143:0/164615758 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f49e80d14e0 0x7f49e80d1970 secure :-1 s=READY pgs=1429 cs=0 l=1 rev1=1 crypto rx=0x7f49e808b590 tx=0x7f49e001c470 comp rx=0 tx=0).stop 2023-11-26T07:24:51.155 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:51.154+0000 7f49ecf1d640 1 -- 172.21.15.143:0/164615758 shutdown_connections 2023-11-26T07:24:51.156 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:51.154+0000 7f49ecf1d640 1 --2- 172.21.15.143:0/164615758 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f49e80d14e0 0x7f49e80d1970 unknown :-1 s=CLOSED pgs=1429 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:24:51.156 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:51.154+0000 7f49ecf1d640 1 --2- 172.21.15.143:0/164615758 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f49e80b2240 0x7f49e80d0590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:24:51.156 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:51.154+0000 7f49ecf1d640 1 --2- 172.21.15.143:0/164615758 >> v2:172.21.15.188:3300/0 conn(0x7f49e8160aa0 0x7f49e80b2b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:24:51.156 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:51.154+0000 7f49ecf1d640 1 -- 172.21.15.143:0/164615758 >> 172.21.15.143:0/164615758 conn(0x7f49e809c550 msgr2=0x7f49e80d3220 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:24:51.156 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:51.154+0000 7f49ecf1d640 1 -- 172.21.15.143:0/164615758 shutdown_connections 2023-11-26T07:24:51.156 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:51.154+0000 7f49ecf1d640 1 -- 172.21.15.143:0/164615758 wait complete. 2023-11-26T07:24:51.157 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:51.158+0000 7f49ecf1d640 1 Processor -- start 2023-11-26T07:24:51.157 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:51.158+0000 7f49ecf1d640 1 -- start start 2023-11-26T07:24:51.159 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:51.158+0000 7f49ecf1d640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f49e80b2240 0x7f49e81554b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:24:51.159 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:51.158+0000 7f49ecf1d640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f49e80d14e0 0x7f49e80ccef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:24:51.159 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:51.158+0000 7f49ecf1d640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f49e8160aa0 0x7f49e8154270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:24:51.159 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:51.158+0000 7f49ecf1d640 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f49e81563c0 con 0x7f49e80d14e0 2023-11-26T07:24:51.159 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:51.158+0000 7f49ecf1d640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f49e80cd430 con 0x7f49e80b2240 2023-11-26T07:24:51.159 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:51.158+0000 7f49ecf1d640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f49e8155e20 con 0x7f49e8160aa0 2023-11-26T07:24:51.159 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:51.158+0000 7f49dffff640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f49e80d14e0 0x7f49e80ccef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:24:51.159 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:51.158+0000 7f49dffff640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f49e80d14e0 0x7f49e80ccef0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:58770/0 (socket says 172.21.15.143:58770) 2023-11-26T07:24:51.159 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:51.158+0000 7f49dffff640 1 -- 172.21.15.143:0/2900732819 learned_addr learned my addr 172.21.15.143:0/2900732819 (peer_addr_for_me v2:172.21.15.143:0/0) 2023-11-26T07:24:51.159 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:51.158+0000 7f49e659c640 1 --2- 172.21.15.143:0/2900732819 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f49e80b2240 0x7f49e81554b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:24:51.159 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:51.158+0000 7f49e6d9d640 1 --2- 172.21.15.143:0/2900732819 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f49e8160aa0 0x7f49e8154270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:24:51.160 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:51.158+0000 7f49dffff640 1 -- 172.21.15.143:0/2900732819 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f49e8160aa0 msgr2=0x7f49e8154270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:24:51.160 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:51.158+0000 7f49dffff640 1 --2- 172.21.15.143:0/2900732819 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f49e8160aa0 0x7f49e8154270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:24:51.160 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:51.158+0000 7f49dffff640 1 -- 172.21.15.143:0/2900732819 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f49e80b2240 msgr2=0x7f49e81554b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:24:51.160 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:51.158+0000 7f49dffff640 1 --2- 172.21.15.143:0/2900732819 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f49e80b2240 0x7f49e81554b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:24:51.160 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:51.158+0000 7f49dffff640 1 -- 172.21.15.143:0/2900732819 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f49e00094d0 con 0x7f49e80d14e0 2023-11-26T07:24:51.161 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:51.158+0000 7f49dffff640 1 --2- 172.21.15.143:0/2900732819 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f49e80d14e0 0x7f49e80ccef0 secure :-1 s=READY pgs=1430 cs=0 l=1 rev1=1 crypto rx=0x7f49d000d830 tx=0x7f49d000dcf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:24:51.161 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:51.158+0000 7f49deffd640 1 -- 172.21.15.143:0/2900732819 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f49d000be10 con 0x7f49e80d14e0 2023-11-26T07:24:51.161 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:51.158+0000 7f49deffd640 1 -- 172.21.15.143:0/2900732819 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f49d0004030 con 0x7f49e80d14e0 2023-11-26T07:24:51.161 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:51.158+0000 7f49deffd640 1 -- 172.21.15.143:0/2900732819 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f49d00043c0 con 0x7f49e80d14e0 2023-11-26T07:24:51.161 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:51.158+0000 7f49ecf1d640 1 -- 172.21.15.143:0/2900732819 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f49e8155b40 con 0x7f49e80d14e0 2023-11-26T07:24:51.161 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:51.158+0000 7f49ecf1d640 1 -- 172.21.15.143:0/2900732819 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f49e80ca910 con 0x7f49e80d14e0 2023-11-26T07:24:51.161 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:51.158+0000 7f49ecf1d640 1 -- 172.21.15.143:0/2900732819 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f49e8067680 con 0x7f49e80d14e0 2023-11-26T07:24:51.162 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:51.162+0000 7f49deffd640 1 -- 172.21.15.143:0/2900732819 <== mon.0 v2:172.21.15.143:3300/0 4 ==== mgrmap(e 18) v1 ==== 86107+0+0 (secure 0 0 0) 0x7f49d00098b0 con 0x7f49e80d14e0 2023-11-26T07:24:51.162 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:51.162+0000 7f49deffd640 1 --2- 172.21.15.143:0/2900732819 >> [v2:172.21.15.188:6836/3401128553,v1:172.21.15.188:6837/3401128553] conn(0x7f49c8066770 0x7f49c8068c30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:24:51.163 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:51.162+0000 7f49deffd640 1 -- 172.21.15.143:0/2900732819 <== mon.0 v2:172.21.15.143:3300/0 5 ==== osd_map(103..103 src has 1..103) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f49d00842a0 con 0x7f49e80d14e0 2023-11-26T07:24:51.163 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:51.162+0000 7f49e659c640 1 --2- 172.21.15.143:0/2900732819 >> [v2:172.21.15.188:6836/3401128553,v1:172.21.15.188:6837/3401128553] conn(0x7f49c8066770 0x7f49c8068c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:24:51.163 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:51.162+0000 7f49e659c640 1 --2- 172.21.15.143:0/2900732819 >> [v2:172.21.15.188:6836/3401128553,v1:172.21.15.188:6837/3401128553] conn(0x7f49c8066770 0x7f49c8068c30 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7f49e80c6b60 tx=0x7f49d8009830 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:24:51.166 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:51.166+0000 7f49deffd640 1 -- 172.21.15.143:0/2900732819 <== mon.0 v2:172.21.15.143:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172649 (secure 0 0 0) 0x7f49d0051510 con 0x7f49e80d14e0 2023-11-26T07:24:51.346 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:51.346+0000 7f49ecf1d640 1 -- 172.21.15.143:0/2900732819 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs"} v 0) v1 -- 0x7f49e809ca30 con 0x7f49e80d14e0 2023-11-26T07:24:51.347 INFO:tasks.ceph.mon.a.smithi143.stderr:2023-11-26T07:24:51.346+0000 7f6946f73640 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2023-11-26T07:24:51.349 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:51.350+0000 7f49deffd640 1 -- 172.21.15.143:0/2900732819 <== mon.0 v2:172.21.15.143:3300/0 7 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs"}]=0 cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v67) v1 ==== 161+0+0 (secure 0 0 0) 0x7f49d0054b30 con 0x7f49e80d14e0 2023-11-26T07:24:51.349 INFO:teuthology.orchestra.run.smithi143.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2023-11-26T07:24:51.352 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:51.350+0000 7f49ecf1d640 1 -- 172.21.15.143:0/2900732819 >> [v2:172.21.15.188:6836/3401128553,v1:172.21.15.188:6837/3401128553] conn(0x7f49c8066770 msgr2=0x7f49c8068c30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:24:51.352 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:51.350+0000 7f49ecf1d640 1 --2- 172.21.15.143:0/2900732819 >> [v2:172.21.15.188:6836/3401128553,v1:172.21.15.188:6837/3401128553] conn(0x7f49c8066770 0x7f49c8068c30 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7f49e80c6b60 tx=0x7f49d8009830 comp rx=0 tx=0).stop 2023-11-26T07:24:51.352 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:51.350+0000 7f49ecf1d640 1 -- 172.21.15.143:0/2900732819 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f49e80d14e0 msgr2=0x7f49e80ccef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:24:51.352 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:51.350+0000 7f49ecf1d640 1 --2- 172.21.15.143:0/2900732819 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f49e80d14e0 0x7f49e80ccef0 secure :-1 s=READY pgs=1430 cs=0 l=1 rev1=1 crypto rx=0x7f49d000d830 tx=0x7f49d000dcf0 comp rx=0 tx=0).stop 2023-11-26T07:24:51.352 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:51.350+0000 7f49ecf1d640 1 -- 172.21.15.143:0/2900732819 shutdown_connections 2023-11-26T07:24:51.352 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:51.350+0000 7f49ecf1d640 1 --2- 172.21.15.143:0/2900732819 >> [v2:172.21.15.188:6836/3401128553,v1:172.21.15.188:6837/3401128553] conn(0x7f49c8066770 0x7f49c8068c30 unknown :-1 s=CLOSED pgs=176 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:24:51.352 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:51.350+0000 7f49ecf1d640 1 --2- 172.21.15.143:0/2900732819 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f49e8160aa0 0x7f49e8154270 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:24:51.352 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:51.350+0000 7f49ecf1d640 1 --2- 172.21.15.143:0/2900732819 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f49e80d14e0 0x7f49e80ccef0 unknown :-1 s=CLOSED pgs=1430 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:24:51.352 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:51.350+0000 7f49ecf1d640 1 --2- 172.21.15.143:0/2900732819 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f49e80b2240 0x7f49e81554b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:24:51.352 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:51.350+0000 7f49ecf1d640 1 -- 172.21.15.143:0/2900732819 >> 172.21.15.143:0/2900732819 conn(0x7f49e809c550 msgr2=0x7f49e80b02b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:24:51.353 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:51.350+0000 7f49ecf1d640 1 -- 172.21.15.143:0/2900732819 shutdown_connections 2023-11-26T07:24:51.353 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:51.350+0000 7f49ecf1d640 1 -- 172.21.15.143:0/2900732819 wait complete. 2023-11-26T07:24:51.374 DEBUG:teuthology.orchestra.run.smithi143:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs rm cephfs --yes-i-really-mean-it 2023-11-26T07:24:51.420 INFO:tasks.ceph.mds.a.smithi143.stdout:starting mds.a at 2023-11-26T07:24:51.517 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:51.518+0000 7f93002fe640 1 Processor -- start 2023-11-26T07:24:51.518 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:51.518+0000 7f93002fe640 1 -- start start 2023-11-26T07:24:51.519 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:51.518+0000 7f93002fe640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f92f813e290 0x7f92f80cf280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:24:51.519 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:51.518+0000 7f93002fe640 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7f92f80cf7c0 0x7f92f815efa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:24:51.519 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:51.518+0000 7f93002fe640 1 -- --> v1:172.21.15.143:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f92f806c520 con 0x7f92f815f4e0 2023-11-26T07:24:51.519 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:51.518+0000 7f93002fe640 1 -- --> v2:172.21.15.143:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f92f80cfc40 con 0x7f92f80cf7c0 2023-11-26T07:24:51.519 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:51.518+0000 7f93002fe640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f92f806c800 con 0x7f92f813e290 2023-11-26T07:24:51.519 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:51.518+0000 7f92fd899640 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7f92f80cf7c0 0x7f92f815efa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:24:51.520 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:51.518+0000 7f92fe89b640 1 --1- >> v1:172.21.15.143:6789/0 conn(0x7f92f815f4e0 0x7f92f81408e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.143:6789/0 says I am v1:172.21.15.143:49748/0 (socket says 172.21.15.143:49748) 2023-11-26T07:24:51.520 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:51.518+0000 7f92fe89b640 1 -- 172.21.15.143:0/2014100313 learned_addr learned my addr 172.21.15.143:0/2014100313 (peer_addr_for_me v1:172.21.15.143:0/0) 2023-11-26T07:24:51.520 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:51.518+0000 7f92fe09a640 1 --2- 172.21.15.143:0/2014100313 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f92f813e290 0x7f92f80cf280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:24:51.520 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:51.518+0000 7f92fd899640 1 -- 172.21.15.143:0/2014100313 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f92f813e290 msgr2=0x7f92f80cf280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-26T07:24:51.521 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:51.518+0000 7f92fd899640 1 --2- 172.21.15.143:0/2014100313 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f92f813e290 0x7f92f80cf280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:24:51.521 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:51.518+0000 7f92fd899640 1 -- 172.21.15.143:0/2014100313 >> v1:172.21.15.143:6789/0 conn(0x7f92f815f4e0 legacy=0x7f92f81408e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:24:51.521 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:51.518+0000 7f92fd899640 1 -- 172.21.15.143:0/2014100313 --> v2:172.21.15.143:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f92f806cae0 con 0x7f92f80cf7c0 2023-11-26T07:24:51.521 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:51.518+0000 7f92fd098640 1 -- 172.21.15.143:0/2014100313 <== mon.0 v1:172.21.15.143:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 949924650 0 0) 0x7f92f806c520 con 0x7f92f815f4e0 2023-11-26T07:24:51.523 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:51.522+0000 7f92fd899640 1 --2- 172.21.15.143:0/2014100313 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f92f80cf7c0 0x7f92f815efa0 secure :-1 s=READY pgs=1435 cs=0 l=1 rev1=1 crypto rx=0x7f92e80097c0 tx=0x7f92e801c450 comp rx=0 tx=0).ready entity=mon.0 client_cookie=735ba2b93cc3b4b2 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:24:51.523 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:51.522+0000 7f92fd098640 1 -- 172.21.15.143:0/2014100313 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f92e80ab650 con 0x7f92f80cf7c0 2023-11-26T07:24:51.523 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:51.522+0000 7f92fd098640 1 -- 172.21.15.143:0/2014100313 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f92e80ab7b0 con 0x7f92f80cf7c0 2023-11-26T07:24:51.523 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:51.522+0000 7f92fd098640 1 -- 172.21.15.143:0/2014100313 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f92e80abb40 con 0x7f92f80cf7c0 2023-11-26T07:24:51.524 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:51.522+0000 7f93002fe640 1 -- 172.21.15.143:0/2014100313 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f92f80cf7c0 msgr2=0x7f92f815efa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:24:51.524 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:51.522+0000 7f93002fe640 1 --2- 172.21.15.143:0/2014100313 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f92f80cf7c0 0x7f92f815efa0 secure :-1 s=READY pgs=1435 cs=0 l=1 rev1=1 crypto rx=0x7f92e80097c0 tx=0x7f92e801c450 comp rx=0 tx=0).stop 2023-11-26T07:24:51.524 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:51.522+0000 7f93002fe640 1 -- 172.21.15.143:0/2014100313 shutdown_connections 2023-11-26T07:24:51.524 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:51.522+0000 7f93002fe640 1 --2- 172.21.15.143:0/2014100313 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f92f80cf7c0 0x7f92f815efa0 unknown :-1 s=CLOSED pgs=1435 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:24:51.524 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:51.522+0000 7f93002fe640 1 --2- 172.21.15.143:0/2014100313 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f92f813e290 0x7f92f80cf280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:24:51.524 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:51.522+0000 7f93002fe640 1 -- 172.21.15.143:0/2014100313 >> 172.21.15.143:0/2014100313 conn(0x7f92f806a890 msgr2=0x7f92f806aca0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:24:51.524 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:51.522+0000 7f93002fe640 1 -- 172.21.15.143:0/2014100313 shutdown_connections 2023-11-26T07:24:51.524 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:51.522+0000 7f93002fe640 1 -- 172.21.15.143:0/2014100313 wait complete. 2023-11-26T07:24:51.525 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:51.526+0000 7f93002fe640 1 Processor -- start 2023-11-26T07:24:51.525 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:51.526+0000 7f93002fe640 1 -- start start 2023-11-26T07:24:51.527 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:51.526+0000 7f93002fe640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f92f80cf7c0 0x7f92f812a780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:24:51.527 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:51.526+0000 7f93002fe640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f92f813e290 0x7f92f81440a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:24:51.527 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:51.526+0000 7f93002fe640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f92f815f4e0 0x7f92f80b0320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:24:51.527 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:51.526+0000 7f93002fe640 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f92f812acc0 con 0x7f92f813e290 2023-11-26T07:24:51.527 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:51.526+0000 7f93002fe640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f92f81445e0 con 0x7f92f815f4e0 2023-11-26T07:24:51.527 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:51.526+0000 7f93002fe640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f92f80b1130 con 0x7f92f80cf7c0 2023-11-26T07:24:51.528 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:51.526+0000 7f92fd899640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f92f813e290 0x7f92f81440a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:24:51.528 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:51.526+0000 7f92fd899640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f92f813e290 0x7f92f81440a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:58826/0 (socket says 172.21.15.143:58826) 2023-11-26T07:24:51.528 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:51.526+0000 7f92fd899640 1 -- 172.21.15.143:0/2538061420 learned_addr learned my addr 172.21.15.143:0/2538061420 (peer_addr_for_me v2:172.21.15.143:0/0) 2023-11-26T07:24:51.528 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:51.526+0000 7f92fe89b640 1 --2- 172.21.15.143:0/2538061420 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f92f815f4e0 0x7f92f80b0320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:24:51.528 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:51.526+0000 7f92fe09a640 1 --2- 172.21.15.143:0/2538061420 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f92f80cf7c0 0x7f92f812a780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:24:51.528 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:51.526+0000 7f92fd899640 1 -- 172.21.15.143:0/2538061420 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f92f80cf7c0 msgr2=0x7f92f812a780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:24:51.528 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:51.526+0000 7f92fd899640 1 --2- 172.21.15.143:0/2538061420 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f92f80cf7c0 0x7f92f812a780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:24:51.528 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:51.526+0000 7f92fd899640 1 -- 172.21.15.143:0/2538061420 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f92f815f4e0 msgr2=0x7f92f80b0320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:24:51.528 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:51.526+0000 7f92fd899640 1 --2- 172.21.15.143:0/2538061420 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f92f815f4e0 0x7f92f80b0320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:24:51.528 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:51.526+0000 7f92fd899640 1 -- 172.21.15.143:0/2538061420 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f92e8009420 con 0x7f92f813e290 2023-11-26T07:24:51.528 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:51.526+0000 7f92fd899640 1 --2- 172.21.15.143:0/2538061420 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f92f813e290 0x7f92f81440a0 secure :-1 s=READY pgs=1436 cs=0 l=1 rev1=1 crypto rx=0x7f92e80ad040 tx=0x7f92e801ac90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:24:51.529 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:51.526+0000 7f92eeffd640 1 -- 172.21.15.143:0/2538061420 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f92e80b5810 con 0x7f92f813e290 2023-11-26T07:24:51.529 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:51.526+0000 7f92eeffd640 1 -- 172.21.15.143:0/2538061420 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f92e80b5990 con 0x7f92f813e290 2023-11-26T07:24:51.529 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:51.526+0000 7f92eeffd640 1 -- 172.21.15.143:0/2538061420 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f92e80b5d40 con 0x7f92f813e290 2023-11-26T07:24:51.529 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:51.526+0000 7f93002fe640 1 -- 172.21.15.143:0/2538061420 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f92f80b1410 con 0x7f92f813e290 2023-11-26T07:24:51.529 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:51.526+0000 7f93002fe640 1 -- 172.21.15.143:0/2538061420 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f92f81418f0 con 0x7f92f813e290 2023-11-26T07:24:51.530 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:51.526+0000 7f93002fe640 1 -- 172.21.15.143:0/2538061420 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f92f8067680 con 0x7f92f813e290 2023-11-26T07:24:51.530 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:51.530+0000 7f92eeffd640 1 -- 172.21.15.143:0/2538061420 <== mon.0 v2:172.21.15.143:3300/0 4 ==== mgrmap(e 18) v1 ==== 86107+0+0 (secure 0 0 0) 0x7f92e80bf410 con 0x7f92f813e290 2023-11-26T07:24:51.530 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:51.530+0000 7f92eeffd640 1 --2- 172.21.15.143:0/2538061420 >> [v2:172.21.15.188:6836/3401128553,v1:172.21.15.188:6837/3401128553] conn(0x7f92dc066740 0x7f92dc068c00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:24:51.531 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:51.530+0000 7f92eeffd640 1 -- 172.21.15.143:0/2538061420 <== mon.0 v2:172.21.15.143:3300/0 5 ==== osd_map(104..104 src has 1..104) v4 ==== 5795+0+0 (secure 0 0 0) 0x7f92e811f100 con 0x7f92f813e290 2023-11-26T07:24:51.531 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:51.530+0000 7f92fe09a640 1 --2- 172.21.15.143:0/2538061420 >> [v2:172.21.15.188:6836/3401128553,v1:172.21.15.188:6837/3401128553] conn(0x7f92dc066740 0x7f92dc068c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:24:51.531 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:51.530+0000 7f92fe09a640 1 --2- 172.21.15.143:0/2538061420 >> [v2:172.21.15.188:6836/3401128553,v1:172.21.15.188:6837/3401128553] conn(0x7f92dc066740 0x7f92dc068c00 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7f92f400b9d0 tx=0x7f92f4008040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:24:51.534 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:51.534+0000 7f92eeffd640 1 -- 172.21.15.143:0/2538061420 <== mon.0 v2:172.21.15.143:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172649 (secure 0 0 0) 0x7f92e80ec340 con 0x7f92f813e290 2023-11-26T07:24:51.718 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:51.718+0000 7f93002fe640 1 -- 172.21.15.143:0/2538061420 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true} v 0) v1 -- 0x7f92f80cc0f0 con 0x7f92f813e290 2023-11-26T07:24:52.351 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:52.350+0000 7f92eeffd640 1 -- 172.21.15.143:0/2538061420 <== mon.0 v2:172.21.15.143:3300/0 7 ==== mon_command_ack([{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]=0 v68) v1 ==== 104+0+0 (secure 0 0 0) 0x7f92e80ef960 con 0x7f92f813e290 2023-11-26T07:24:52.353 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:52.354+0000 7f93002fe640 1 -- 172.21.15.143:0/2538061420 >> [v2:172.21.15.188:6836/3401128553,v1:172.21.15.188:6837/3401128553] conn(0x7f92dc066740 msgr2=0x7f92dc068c00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:24:52.354 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:52.354+0000 7f93002fe640 1 --2- 172.21.15.143:0/2538061420 >> [v2:172.21.15.188:6836/3401128553,v1:172.21.15.188:6837/3401128553] conn(0x7f92dc066740 0x7f92dc068c00 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7f92f400b9d0 tx=0x7f92f4008040 comp rx=0 tx=0).stop 2023-11-26T07:24:52.354 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:52.354+0000 7f93002fe640 1 -- 172.21.15.143:0/2538061420 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f92f813e290 msgr2=0x7f92f81440a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:24:52.354 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:52.354+0000 7f93002fe640 1 --2- 172.21.15.143:0/2538061420 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f92f813e290 0x7f92f81440a0 secure :-1 s=READY pgs=1436 cs=0 l=1 rev1=1 crypto rx=0x7f92e80ad040 tx=0x7f92e801ac90 comp rx=0 tx=0).stop 2023-11-26T07:24:52.354 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:52.354+0000 7f93002fe640 1 -- 172.21.15.143:0/2538061420 shutdown_connections 2023-11-26T07:24:52.354 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:52.354+0000 7f93002fe640 1 --2- 172.21.15.143:0/2538061420 >> [v2:172.21.15.188:6836/3401128553,v1:172.21.15.188:6837/3401128553] conn(0x7f92dc066740 0x7f92dc068c00 unknown :-1 s=CLOSED pgs=177 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:24:52.354 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:52.354+0000 7f93002fe640 1 --2- 172.21.15.143:0/2538061420 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f92f815f4e0 0x7f92f80b0320 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:24:52.354 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:52.354+0000 7f93002fe640 1 --2- 172.21.15.143:0/2538061420 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f92f813e290 0x7f92f81440a0 unknown :-1 s=CLOSED pgs=1436 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:24:52.354 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:52.354+0000 7f93002fe640 1 --2- 172.21.15.143:0/2538061420 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f92f80cf7c0 0x7f92f812a780 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:24:52.354 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:52.354+0000 7f93002fe640 1 -- 172.21.15.143:0/2538061420 >> 172.21.15.143:0/2538061420 conn(0x7f92f806a890 msgr2=0x7f92f80cdf00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:24:52.354 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:52.354+0000 7f93002fe640 1 -- 172.21.15.143:0/2538061420 shutdown_connections 2023-11-26T07:24:52.354 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:52.354+0000 7f93002fe640 1 -- 172.21.15.143:0/2538061420 wait complete. 2023-11-26T07:24:52.374 DEBUG:teuthology.orchestra.run.smithi143:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_metadata cephfs_metadata --yes-i-really-really-mean-it 2023-11-26T07:24:52.557 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:52.554+0000 7fdacddf8640 1 Processor -- start 2023-11-26T07:24:52.557 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:52.558+0000 7fdacddf8640 1 -- start start 2023-11-26T07:24:52.559 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:52.558+0000 7fdacddf8640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fdac80cf280 0x7fdac80cf6e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:24:52.559 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:52.558+0000 7fdacddf8640 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7fdac80cd1b0 0x7fdac80cd5e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:24:52.559 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:52.558+0000 7fdacddf8640 1 -- --> v1:172.21.15.188:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fdac80cfc20 con 0x7fdac8143ac0 2023-11-26T07:24:52.559 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:52.558+0000 7fdacddf8640 1 -- --> v2:172.21.15.188:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fdac8143ea0 con 0x7fdac80cd1b0 2023-11-26T07:24:52.559 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:52.558+0000 7fdacddf8640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdac80cc730 con 0x7fdac80cf280 2023-11-26T07:24:52.559 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:52.558+0000 7fdabffff640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fdac80cf280 0x7fdac80cf6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:24:52.559 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:52.558+0000 7fdac7fff640 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7fdac80cd1b0 0x7fdac80cd5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:24:52.559 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:52.558+0000 7fdabffff640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fdac80cf280 0x7fdac80cf6e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.188:3301/0 says I am v2:172.21.15.143:37150/0 (socket says 172.21.15.143:37150) 2023-11-26T07:24:52.560 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:52.558+0000 7fdabffff640 1 -- 172.21.15.143:0/1677903908 learned_addr learned my addr 172.21.15.143:0/1677903908 (peer_addr_for_me v2:172.21.15.143:0/0) 2023-11-26T07:24:52.560 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:52.558+0000 7fdabffff640 1 -- 172.21.15.143:0/1677903908 >> v2:172.21.15.188:3300/0 conn(0x7fdac80cd1b0 msgr2=0x7fdac80cd5e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-26T07:24:52.560 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:52.558+0000 7fdabffff640 1 --2- 172.21.15.143:0/1677903908 >> v2:172.21.15.188:3300/0 conn(0x7fdac80cd1b0 0x7fdac80cd5e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:24:52.560 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:52.558+0000 7fdabffff640 1 -- 172.21.15.143:0/1677903908 >> v1:172.21.15.188:6789/0 conn(0x7fdac8143ac0 legacy=0x7fdac8070370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:24:52.560 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:52.558+0000 7fdabffff640 1 -- 172.21.15.143:0/1677903908 --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdac8070ee0 con 0x7fdac80cf280 2023-11-26T07:24:52.560 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:52.558+0000 7fdac7fff640 1 --2- 172.21.15.143:0/1677903908 >> v2:172.21.15.188:3300/0 conn(0x7fdac80cd1b0 0x7fdac80cd5e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-26T07:24:52.562 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:52.562+0000 7fdabffff640 1 --2- 172.21.15.143:0/1677903908 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fdac80cf280 0x7fdac80cf6e0 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7fdab0009870 tx=0x7fdab001c540 comp rx=0 tx=0).ready entity=mon.2 client_cookie=f16de85c87b04a05 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:24:52.563 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:52.562+0000 7fdac6ffd640 1 -- 172.21.15.143:0/1677903908 <== mon.2 v2:172.21.15.188:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdab00a9400 con 0x7fdac80cf280 2023-11-26T07:24:52.563 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:52.562+0000 7fdac6ffd640 1 -- 172.21.15.143:0/1677903908 <== mon.2 v2:172.21.15.188:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdab00a9560 con 0x7fdac80cf280 2023-11-26T07:24:52.563 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:52.562+0000 7fdac6ffd640 1 -- 172.21.15.143:0/1677903908 <== mon.2 v2:172.21.15.188:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdab00a98f0 con 0x7fdac80cf280 2023-11-26T07:24:52.564 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:52.562+0000 7fdacddf8640 1 -- 172.21.15.143:0/1677903908 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fdac80cf280 msgr2=0x7fdac80cf6e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:24:52.564 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:52.562+0000 7fdacddf8640 1 --2- 172.21.15.143:0/1677903908 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fdac80cf280 0x7fdac80cf6e0 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7fdab0009870 tx=0x7fdab001c540 comp rx=0 tx=0).stop 2023-11-26T07:24:52.564 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:52.562+0000 7fdacddf8640 1 -- 172.21.15.143:0/1677903908 shutdown_connections 2023-11-26T07:24:52.564 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:52.562+0000 7fdacddf8640 1 --2- 172.21.15.143:0/1677903908 >> v2:172.21.15.188:3300/0 conn(0x7fdac80cd1b0 0x7fdac80cd5e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:24:52.564 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:52.562+0000 7fdacddf8640 1 --2- 172.21.15.143:0/1677903908 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fdac80cf280 0x7fdac80cf6e0 unknown :-1 s=CLOSED pgs=167 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:24:52.564 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:52.562+0000 7fdacddf8640 1 -- 172.21.15.143:0/1677903908 >> 172.21.15.143:0/1677903908 conn(0x7fdac806a890 msgr2=0x7fdac806aca0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:24:52.564 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:52.562+0000 7fdacddf8640 1 -- 172.21.15.143:0/1677903908 shutdown_connections 2023-11-26T07:24:52.564 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:52.562+0000 7fdacddf8640 1 -- 172.21.15.143:0/1677903908 wait complete. 2023-11-26T07:24:52.565 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:52.566+0000 7fdacddf8640 1 Processor -- start 2023-11-26T07:24:52.565 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:52.566+0000 7fdacddf8640 1 -- start start 2023-11-26T07:24:52.567 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:52.566+0000 7fdacddf8640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fdac80cd1b0 0x7fdac80afc90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:24:52.567 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:52.566+0000 7fdacddf8640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fdac80cf280 0x7fdac8159180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:24:52.567 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:52.566+0000 7fdacddf8640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fdac8143ac0 0x7fdac81414f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:24:52.567 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:52.566+0000 7fdacddf8640 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdac81f5ef0 con 0x7fdac80cd1b0 2023-11-26T07:24:52.567 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:52.566+0000 7fdacddf8640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdac813a920 con 0x7fdac8143ac0 2023-11-26T07:24:52.567 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:52.566+0000 7fdacddf8640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdac813afe0 con 0x7fdac80cf280 2023-11-26T07:24:52.568 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:52.566+0000 7fdac77fe640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fdac80cd1b0 0x7fdac80afc90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:24:52.568 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:52.566+0000 7fdac77fe640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fdac80cd1b0 0x7fdac80afc90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:58842/0 (socket says 172.21.15.143:58842) 2023-11-26T07:24:52.568 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:52.566+0000 7fdac77fe640 1 -- 172.21.15.143:0/3661431374 learned_addr learned my addr 172.21.15.143:0/3661431374 (peer_addr_for_me v2:172.21.15.143:0/0) 2023-11-26T07:24:52.568 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:52.566+0000 7fdac7fff640 1 --2- 172.21.15.143:0/3661431374 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fdac8143ac0 0x7fdac81414f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:24:52.568 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:52.566+0000 7fdabffff640 1 --2- 172.21.15.143:0/3661431374 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fdac80cf280 0x7fdac8159180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:24:52.568 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:52.566+0000 7fdac77fe640 1 -- 172.21.15.143:0/3661431374 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fdac80cf280 msgr2=0x7fdac8159180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:24:52.568 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:52.566+0000 7fdac77fe640 1 --2- 172.21.15.143:0/3661431374 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fdac80cf280 0x7fdac8159180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:24:52.568 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:52.566+0000 7fdac77fe640 1 -- 172.21.15.143:0/3661431374 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fdac8143ac0 msgr2=0x7fdac81414f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:24:52.568 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:52.566+0000 7fdac77fe640 1 --2- 172.21.15.143:0/3661431374 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fdac8143ac0 0x7fdac81414f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:24:52.568 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:52.566+0000 7fdac77fe640 1 -- 172.21.15.143:0/3661431374 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdac0009a70 con 0x7fdac80cd1b0 2023-11-26T07:24:52.568 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:52.566+0000 7fdac77fe640 1 --2- 172.21.15.143:0/3661431374 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fdac80cd1b0 0x7fdac80afc90 secure :-1 s=READY pgs=1437 cs=0 l=1 rev1=1 crypto rx=0x7fdab800c3c0 tx=0x7fdab800c880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:24:52.569 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:52.566+0000 7fdac4ff9640 1 -- 172.21.15.143:0/3661431374 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdab80052d0 con 0x7fdac80cd1b0 2023-11-26T07:24:52.569 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:52.566+0000 7fdac4ff9640 1 -- 172.21.15.143:0/3661431374 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdab8005430 con 0x7fdac80cd1b0 2023-11-26T07:24:52.569 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:52.566+0000 7fdac4ff9640 1 -- 172.21.15.143:0/3661431374 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdab8003840 con 0x7fdac80cd1b0 2023-11-26T07:24:52.569 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:52.566+0000 7fdacddf8640 1 -- 172.21.15.143:0/3661431374 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdab00094d0 con 0x7fdac80cd1b0 2023-11-26T07:24:52.569 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:52.566+0000 7fdacddf8640 1 -- 172.21.15.143:0/3661431374 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdac813b280 con 0x7fdac80cd1b0 2023-11-26T07:24:52.570 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:52.566+0000 7fdacddf8640 1 -- 172.21.15.143:0/3661431374 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fdac81f6030 con 0x7fdac80cd1b0 2023-11-26T07:24:52.570 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:52.570+0000 7fdac4ff9640 1 -- 172.21.15.143:0/3661431374 <== mon.0 v2:172.21.15.143:3300/0 4 ==== mgrmap(e 18) v1 ==== 86107+0+0 (secure 0 0 0) 0x7fdab80039a0 con 0x7fdac80cd1b0 2023-11-26T07:24:52.570 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:52.570+0000 7fdac4ff9640 1 --2- 172.21.15.143:0/3661431374 >> [v2:172.21.15.188:6836/3401128553,v1:172.21.15.188:6837/3401128553] conn(0x7fdaa8066740 0x7fdaa8068c00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:24:52.571 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:52.570+0000 7fdac4ff9640 1 -- 172.21.15.143:0/3661431374 <== mon.0 v2:172.21.15.143:3300/0 5 ==== osd_map(104..104 src has 1..104) v4 ==== 5795+0+0 (secure 0 0 0) 0x7fdab8083820 con 0x7fdac80cd1b0 2023-11-26T07:24:52.571 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:52.570+0000 7fdabffff640 1 --2- 172.21.15.143:0/3661431374 >> [v2:172.21.15.188:6836/3401128553,v1:172.21.15.188:6837/3401128553] conn(0x7fdaa8066740 0x7fdaa8068c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:24:52.571 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:52.570+0000 7fdabffff640 1 --2- 172.21.15.143:0/3661431374 >> [v2:172.21.15.188:6836/3401128553,v1:172.21.15.188:6837/3401128553] conn(0x7fdaa8066740 0x7fdaa8068c00 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7fdab00ae000 tx=0x7fdab00b3040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:24:52.574 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:52.574+0000 7fdac4ff9640 1 -- 172.21.15.143:0/3661431374 <== mon.0 v2:172.21.15.143:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172649 (secure 0 0 0) 0x7fdab8050940 con 0x7fdac80cd1b0 2023-11-26T07:24:52.729 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:52.726+0000 7fdacddf8640 1 -- 172.21.15.143:0/3661431374 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7fdac8067680 con 0x7fdac80cd1b0 2023-11-26T07:24:52.835 DEBUG:teuthology.orchestra.run.smithi143:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-26T07:24:52.838 DEBUG:teuthology.orchestra.run.smithi188:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-26T07:24:53.686 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:53.686+0000 7fdac4ff9640 1 -- 172.21.15.143:0/3661431374 <== mon.0 v2:172.21.15.143:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_metadata' removed v105) v1 ==== 181+0+0 (secure 0 0 0) 0x7fdab8053f60 con 0x7fdac80cd1b0 2023-11-26T07:24:53.686 INFO:teuthology.orchestra.run.smithi143.stderr:pool 'cephfs_metadata' removed 2023-11-26T07:24:53.688 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:53.686+0000 7fdacddf8640 1 -- 172.21.15.143:0/3661431374 >> [v2:172.21.15.188:6836/3401128553,v1:172.21.15.188:6837/3401128553] conn(0x7fdaa8066740 msgr2=0x7fdaa8068c00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:24:53.688 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:53.686+0000 7fdacddf8640 1 --2- 172.21.15.143:0/3661431374 >> [v2:172.21.15.188:6836/3401128553,v1:172.21.15.188:6837/3401128553] conn(0x7fdaa8066740 0x7fdaa8068c00 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7fdab00ae000 tx=0x7fdab00b3040 comp rx=0 tx=0).stop 2023-11-26T07:24:53.688 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:53.686+0000 7fdacddf8640 1 -- 172.21.15.143:0/3661431374 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fdac80cd1b0 msgr2=0x7fdac80afc90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:24:53.688 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:53.686+0000 7fdacddf8640 1 --2- 172.21.15.143:0/3661431374 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fdac80cd1b0 0x7fdac80afc90 secure :-1 s=READY pgs=1437 cs=0 l=1 rev1=1 crypto rx=0x7fdab800c3c0 tx=0x7fdab800c880 comp rx=0 tx=0).stop 2023-11-26T07:24:53.690 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:53.690+0000 7fdacddf8640 1 -- 172.21.15.143:0/3661431374 shutdown_connections 2023-11-26T07:24:53.690 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:53.690+0000 7fdacddf8640 1 --2- 172.21.15.143:0/3661431374 >> [v2:172.21.15.188:6836/3401128553,v1:172.21.15.188:6837/3401128553] conn(0x7fdaa8066740 0x7fdaa8068c00 unknown :-1 s=CLOSED pgs=179 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:24:53.690 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:53.690+0000 7fdacddf8640 1 --2- 172.21.15.143:0/3661431374 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fdac8143ac0 0x7fdac81414f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:24:53.690 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:53.690+0000 7fdacddf8640 1 --2- 172.21.15.143:0/3661431374 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fdac80cf280 0x7fdac8159180 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:24:53.691 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:53.690+0000 7fdacddf8640 1 --2- 172.21.15.143:0/3661431374 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fdac80cd1b0 0x7fdac80afc90 unknown :-1 s=CLOSED pgs=1437 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:24:53.691 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:53.690+0000 7fdacddf8640 1 -- 172.21.15.143:0/3661431374 >> 172.21.15.143:0/3661431374 conn(0x7fdac806a890 msgr2=0x7fdac8070370 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:24:53.692 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:53.690+0000 7fdacddf8640 1 -- 172.21.15.143:0/3661431374 shutdown_connections 2023-11-26T07:24:53.692 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:53.690+0000 7fdacddf8640 1 -- 172.21.15.143:0/3661431374 wait complete. 2023-11-26T07:24:53.720 DEBUG:teuthology.orchestra.run.smithi143:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_data cephfs_data --yes-i-really-really-mean-it 2023-11-26T07:24:53.862 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:53.862+0000 7f5a16232640 1 Processor -- start 2023-11-26T07:24:53.862 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:53.862+0000 7f5a16232640 1 -- start start 2023-11-26T07:24:53.863 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:53.862+0000 7f5a16232640 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7f5a100cf280 0x7f5a100cf710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:24:53.863 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:53.862+0000 7f5a16232640 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7f5a100cd1b0 0x7f5a100cd5c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:24:53.863 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:53.862+0000 7f5a16232640 1 -- --> v1:172.21.15.188:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5a100cc730 con 0x7f5a10143ac0 2023-11-26T07:24:53.863 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:53.862+0000 7f5a16232640 1 -- --> v2:172.21.15.143:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5a10143ea0 con 0x7f5a100cf280 2023-11-26T07:24:53.863 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:53.862+0000 7f5a16232640 1 -- --> v2:172.21.15.188:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5a100cfc50 con 0x7f5a100cd1b0 2023-11-26T07:24:53.864 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:53.862+0000 7f5a0effd640 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7f5a100cf280 0x7f5a100cf710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:24:53.864 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:53.862+0000 7f5a0effd640 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7f5a100cf280 0x7f5a100cf710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:58856/0 (socket says 172.21.15.143:58856) 2023-11-26T07:24:53.864 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:53.862+0000 7f5a0effd640 1 -- 172.21.15.143:0/1631122873 learned_addr learned my addr 172.21.15.143:0/1631122873 (peer_addr_for_me v2:172.21.15.143:0/0) 2023-11-26T07:24:53.864 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:53.862+0000 7f5a0ffff640 1 --2- 172.21.15.143:0/1631122873 >> v2:172.21.15.188:3300/0 conn(0x7f5a100cd1b0 0x7f5a100cd5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:24:53.865 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:53.862+0000 7f5a0effd640 1 -- 172.21.15.143:0/1631122873 >> v2:172.21.15.188:3300/0 conn(0x7f5a100cd1b0 msgr2=0x7f5a100cd5c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-26T07:24:53.865 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:53.862+0000 7f5a0effd640 1 --2- 172.21.15.143:0/1631122873 >> v2:172.21.15.188:3300/0 conn(0x7f5a100cd1b0 0x7f5a100cd5c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:24:53.865 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:53.862+0000 7f5a0effd640 1 -- 172.21.15.143:0/1631122873 >> v1:172.21.15.188:6789/0 conn(0x7f5a10143ac0 legacy=0x7f5a10070370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-26T07:24:53.865 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:53.862+0000 7f5a0effd640 1 -- 172.21.15.143:0/1631122873 --> v2:172.21.15.143:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5a10070ee0 con 0x7f5a100cf280 2023-11-26T07:24:53.866 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:53.866+0000 7f5a0effd640 1 --2- 172.21.15.143:0/1631122873 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f5a100cf280 0x7f5a100cf710 secure :-1 s=READY pgs=1438 cs=0 l=1 rev1=1 crypto rx=0x7f5a1006e570 tx=0x7f5a0401c4c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b23644b6c8dc45e4 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:24:53.867 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:53.866+0000 7f5a0e7fc640 1 -- 172.21.15.143:0/1631122873 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5a040aa620 con 0x7f5a100cf280 2023-11-26T07:24:53.867 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:53.866+0000 7f5a0e7fc640 1 -- 172.21.15.143:0/1631122873 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5a040aa780 con 0x7f5a100cf280 2023-11-26T07:24:53.867 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:53.866+0000 7f5a0e7fc640 1 -- 172.21.15.143:0/1631122873 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5a040aab10 con 0x7f5a100cf280 2023-11-26T07:24:53.868 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:53.866+0000 7f5a16232640 1 -- 172.21.15.143:0/1631122873 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f5a100cf280 msgr2=0x7f5a100cf710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:24:53.868 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:53.866+0000 7f5a16232640 1 --2- 172.21.15.143:0/1631122873 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f5a100cf280 0x7f5a100cf710 secure :-1 s=READY pgs=1438 cs=0 l=1 rev1=1 crypto rx=0x7f5a1006e570 tx=0x7f5a0401c4c0 comp rx=0 tx=0).stop 2023-11-26T07:24:53.868 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:53.866+0000 7f5a16232640 1 -- 172.21.15.143:0/1631122873 shutdown_connections 2023-11-26T07:24:53.868 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:53.866+0000 7f5a16232640 1 --2- 172.21.15.143:0/1631122873 >> v2:172.21.15.188:3300/0 conn(0x7f5a100cd1b0 0x7f5a100cd5c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:24:53.868 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:53.866+0000 7f5a16232640 1 --2- 172.21.15.143:0/1631122873 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f5a100cf280 0x7f5a100cf710 unknown :-1 s=CLOSED pgs=1438 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:24:53.868 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:53.866+0000 7f5a16232640 1 -- 172.21.15.143:0/1631122873 >> 172.21.15.143:0/1631122873 conn(0x7f5a1006a890 msgr2=0x7f5a1006aca0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:24:53.868 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:53.866+0000 7f5a16232640 1 -- 172.21.15.143:0/1631122873 shutdown_connections 2023-11-26T07:24:53.868 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:53.866+0000 7f5a16232640 1 -- 172.21.15.143:0/1631122873 wait complete. 2023-11-26T07:24:53.869 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:53.870+0000 7f5a16232640 1 Processor -- start 2023-11-26T07:24:53.869 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:53.870+0000 7f5a16232640 1 -- start start 2023-11-26T07:24:53.871 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:53.870+0000 7f5a16232640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f5a100cd1b0 0x7f5a1015a600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:24:53.871 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:53.870+0000 7f5a16232640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f5a100cf280 0x7f5a100aee90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:24:53.872 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:53.870+0000 7f5a16232640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f5a10143ac0 0x7f5a10158870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:24:53.872 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:53.870+0000 7f5a16232640 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5a100afa50 con 0x7f5a10143ac0 2023-11-26T07:24:53.872 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:53.870+0000 7f5a16232640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5a100af770 con 0x7f5a100cf280 2023-11-26T07:24:53.872 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:53.870+0000 7f5a16232640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5a10166d70 con 0x7f5a100cd1b0 2023-11-26T07:24:53.872 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:53.870+0000 7f5a0ffff640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f5a10143ac0 0x7f5a10158870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:24:53.872 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:53.870+0000 7f5a0f7fe640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f5a100cd1b0 0x7f5a1015a600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:24:53.872 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:53.870+0000 7f5a0ffff640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f5a10143ac0 0x7f5a10158870 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:58864/0 (socket says 172.21.15.143:58864) 2023-11-26T07:24:53.872 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:53.870+0000 7f5a0ffff640 1 -- 172.21.15.143:0/3706636874 learned_addr learned my addr 172.21.15.143:0/3706636874 (peer_addr_for_me v2:172.21.15.143:0/0) 2023-11-26T07:24:53.872 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:53.870+0000 7f5a0effd640 1 --2- 172.21.15.143:0/3706636874 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f5a100cf280 0x7f5a100aee90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:24:53.872 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:53.870+0000 7f5a0ffff640 1 -- 172.21.15.143:0/3706636874 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f5a100cd1b0 msgr2=0x7f5a1015a600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:24:53.872 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:53.870+0000 7f5a0ffff640 1 --2- 172.21.15.143:0/3706636874 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f5a100cd1b0 0x7f5a1015a600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:24:53.872 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:53.870+0000 7f5a0ffff640 1 -- 172.21.15.143:0/3706636874 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f5a100cf280 msgr2=0x7f5a100aee90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:24:53.873 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:53.870+0000 7f5a0ffff640 1 --2- 172.21.15.143:0/3706636874 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f5a100cf280 0x7f5a100aee90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:24:53.873 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:53.870+0000 7f5a0ffff640 1 -- 172.21.15.143:0/3706636874 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5a040094d0 con 0x7f5a10143ac0 2023-11-26T07:24:53.873 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:53.870+0000 7f5a0ffff640 1 --2- 172.21.15.143:0/3706636874 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f5a10143ac0 0x7f5a10158870 secure :-1 s=READY pgs=1439 cs=0 l=1 rev1=1 crypto rx=0x7f5a08007e60 tx=0x7f5a0800e4b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:24:53.874 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:53.870+0000 7f59f7fff640 1 -- 172.21.15.143:0/3706636874 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5a08014420 con 0x7f5a10143ac0 2023-11-26T07:24:53.874 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:53.870+0000 7f59f7fff640 1 -- 172.21.15.143:0/3706636874 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5a080145a0 con 0x7f5a10143ac0 2023-11-26T07:24:53.874 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:53.870+0000 7f5a16232640 1 -- 172.21.15.143:0/3706636874 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5a10167050 con 0x7f5a10143ac0 2023-11-26T07:24:53.874 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:53.870+0000 7f59f7fff640 1 -- 172.21.15.143:0/3706636874 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5a08014950 con 0x7f5a10143ac0 2023-11-26T07:24:53.874 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:53.870+0000 7f5a16232640 1 -- 172.21.15.143:0/3706636874 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5a100af3d0 con 0x7f5a10143ac0 2023-11-26T07:24:53.874 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:53.874+0000 7f5a16232640 1 -- 172.21.15.143:0/3706636874 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5a10166bd0 con 0x7f5a10143ac0 2023-11-26T07:24:53.874 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:53.874+0000 7f59f7fff640 1 -- 172.21.15.143:0/3706636874 <== mon.0 v2:172.21.15.143:3300/0 4 ==== mgrmap(e 18) v1 ==== 86107+0+0 (secure 0 0 0) 0x7f5a08014ab0 con 0x7f5a10143ac0 2023-11-26T07:24:53.875 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:53.874+0000 7f59f7fff640 1 --2- 172.21.15.143:0/3706636874 >> [v2:172.21.15.188:6836/3401128553,v1:172.21.15.188:6837/3401128553] conn(0x7f59fc066740 0x7f59fc068c00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:24:53.875 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:53.874+0000 7f59f7fff640 1 -- 172.21.15.143:0/3706636874 <== mon.0 v2:172.21.15.143:3300/0 5 ==== osd_map(105..105 src has 1..105) v4 ==== 5357+0+0 (secure 0 0 0) 0x7f5a08084910 con 0x7f5a10143ac0 2023-11-26T07:24:53.876 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:53.874+0000 7f5a0f7fe640 1 --2- 172.21.15.143:0/3706636874 >> [v2:172.21.15.188:6836/3401128553,v1:172.21.15.188:6837/3401128553] conn(0x7f59fc066740 0x7f59fc068c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:24:53.876 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:53.874+0000 7f5a0f7fe640 1 --2- 172.21.15.143:0/3706636874 >> [v2:172.21.15.188:6836/3401128553,v1:172.21.15.188:6837/3401128553] conn(0x7f59fc066740 0x7f59fc068c00 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7f5a00007fd0 tx=0x7f5a00007e40 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:24:53.880 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:53.878+0000 7f59f7fff640 1 -- 172.21.15.143:0/3706636874 <== mon.0 v2:172.21.15.143:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172649 (secure 0 0 0) 0x7f5a10166bd0 con 0x7f5a10143ac0 2023-11-26T07:24:54.030 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:54.030+0000 7f5a16232640 1 -- 172.21.15.143:0/3706636874 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7f5a10166bd0 con 0x7f5a10143ac0 2023-11-26T07:24:54.688 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:54.686+0000 7f59f7fff640 1 -- 172.21.15.143:0/3706636874 <== mon.0 v2:172.21.15.143:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_data' removed v106) v1 ==== 169+0+0 (secure 0 0 0) 0x7f5a10166bd0 con 0x7f5a10143ac0 2023-11-26T07:24:54.689 INFO:teuthology.orchestra.run.smithi143.stderr:pool 'cephfs_data' removed 2023-11-26T07:24:54.691 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:54.690+0000 7f5a16232640 1 -- 172.21.15.143:0/3706636874 >> [v2:172.21.15.188:6836/3401128553,v1:172.21.15.188:6837/3401128553] conn(0x7f59fc066740 msgr2=0x7f59fc068c00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:24:54.691 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:54.690+0000 7f5a16232640 1 --2- 172.21.15.143:0/3706636874 >> [v2:172.21.15.188:6836/3401128553,v1:172.21.15.188:6837/3401128553] conn(0x7f59fc066740 0x7f59fc068c00 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7f5a00007fd0 tx=0x7f5a00007e40 comp rx=0 tx=0).stop 2023-11-26T07:24:54.691 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:54.690+0000 7f5a16232640 1 -- 172.21.15.143:0/3706636874 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f5a10143ac0 msgr2=0x7f5a10158870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:24:54.691 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:54.690+0000 7f5a16232640 1 --2- 172.21.15.143:0/3706636874 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f5a10143ac0 0x7f5a10158870 secure :-1 s=READY pgs=1439 cs=0 l=1 rev1=1 crypto rx=0x7f5a08007e60 tx=0x7f5a0800e4b0 comp rx=0 tx=0).stop 2023-11-26T07:24:54.691 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:54.690+0000 7f5a16232640 1 -- 172.21.15.143:0/3706636874 shutdown_connections 2023-11-26T07:24:54.691 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:54.690+0000 7f5a16232640 1 --2- 172.21.15.143:0/3706636874 >> [v2:172.21.15.188:6836/3401128553,v1:172.21.15.188:6837/3401128553] conn(0x7f59fc066740 0x7f59fc068c00 unknown :-1 s=CLOSED pgs=180 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:24:54.691 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:54.690+0000 7f5a16232640 1 --2- 172.21.15.143:0/3706636874 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f5a10143ac0 0x7f5a10158870 unknown :-1 s=CLOSED pgs=1439 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:24:54.691 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:54.690+0000 7f5a16232640 1 --2- 172.21.15.143:0/3706636874 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f5a100cf280 0x7f5a100aee90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:24:54.691 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:54.690+0000 7f5a16232640 1 --2- 172.21.15.143:0/3706636874 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f5a100cd1b0 0x7f5a1015a600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:24:54.691 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:54.690+0000 7f5a16232640 1 -- 172.21.15.143:0/3706636874 >> 172.21.15.143:0/3706636874 conn(0x7f5a1006a890 msgr2=0x7f5a10070370 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:24:54.691 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:54.690+0000 7f5a16232640 1 -- 172.21.15.143:0/3706636874 shutdown_connections 2023-11-26T07:24:54.692 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:54.690+0000 7f5a16232640 1 -- 172.21.15.143:0/3706636874 wait complete. 2023-11-26T07:24:54.719 DEBUG:teuthology.orchestra.run.smithi143:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config rm mon mon_allow_pool_delete 2023-11-26T07:24:54.860 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:54.858+0000 7fa7f7b55640 1 Processor -- start 2023-11-26T07:24:54.860 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:54.858+0000 7fa7f7b55640 1 -- start start 2023-11-26T07:24:54.862 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:54.862+0000 7fa7f7b55640 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7fa7f013e290 0x7fa7f00cf280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:24:54.862 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:54.862+0000 7fa7f7b55640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fa7f00cf7c0 0x7fa7f015efa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:24:54.862 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:54.862+0000 7fa7f7b55640 1 -- --> v1:172.21.15.188:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa7f006c520 con 0x7fa7f015f4e0 2023-11-26T07:24:54.862 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:54.862+0000 7fa7f7b55640 1 -- --> v2:172.21.15.143:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa7f00cfc40 con 0x7fa7f013e290 2023-11-26T07:24:54.862 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:54.862+0000 7fa7f7b55640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa7f006c800 con 0x7fa7f00cf7c0 2023-11-26T07:24:54.862 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:54.862+0000 7fa7f58f1640 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7fa7f013e290 0x7fa7f00cf280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:24:54.863 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:54.862+0000 7fa7f58f1640 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7fa7f013e290 0x7fa7f00cf280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:45808/0 (socket says 172.21.15.143:45808) 2023-11-26T07:24:54.863 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:54.862+0000 7fa7f58f1640 1 -- 172.21.15.143:0/2175798159 learned_addr learned my addr 172.21.15.143:0/2175798159 (peer_addr_for_me v2:172.21.15.143:0/0) 2023-11-26T07:24:54.863 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:54.862+0000 7fa7f50f0640 1 --2- 172.21.15.143:0/2175798159 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fa7f00cf7c0 0x7fa7f015efa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:24:54.863 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:54.862+0000 7fa7f58f1640 1 -- 172.21.15.143:0/2175798159 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fa7f00cf7c0 msgr2=0x7fa7f015efa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-26T07:24:54.863 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:54.862+0000 7fa7f58f1640 1 --2- 172.21.15.143:0/2175798159 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fa7f00cf7c0 0x7fa7f015efa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:24:54.863 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:54.862+0000 7fa7f58f1640 1 -- 172.21.15.143:0/2175798159 >> v1:172.21.15.188:6789/0 conn(0x7fa7f015f4e0 legacy=0x7fa7f01408e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:24:54.864 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:54.862+0000 7fa7f58f1640 1 -- 172.21.15.143:0/2175798159 --> v2:172.21.15.143:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa7f006cae0 con 0x7fa7f013e290 2023-11-26T07:24:54.865 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:54.866+0000 7fa7f58f1640 1 --2- 172.21.15.143:0/2175798159 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fa7f013e290 0x7fa7f00cf280 secure :-1 s=READY pgs=1440 cs=0 l=1 rev1=1 crypto rx=0x7fa7ec009870 tx=0x7fa7ec01c510 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6a19684eabba0744 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:24:54.865 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:54.866+0000 7fa7f48ef640 1 -- 172.21.15.143:0/2175798159 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa7ec0aa670 con 0x7fa7f013e290 2023-11-26T07:24:54.865 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:54.866+0000 7fa7f48ef640 1 -- 172.21.15.143:0/2175798159 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa7ec0aa7d0 con 0x7fa7f013e290 2023-11-26T07:24:54.865 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:54.866+0000 7fa7f48ef640 1 -- 172.21.15.143:0/2175798159 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa7ec0aab60 con 0x7fa7f013e290 2023-11-26T07:24:54.866 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:54.866+0000 7fa7f7b55640 1 -- 172.21.15.143:0/2175798159 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fa7f013e290 msgr2=0x7fa7f00cf280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:24:54.866 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:54.866+0000 7fa7f7b55640 1 --2- 172.21.15.143:0/2175798159 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fa7f013e290 0x7fa7f00cf280 secure :-1 s=READY pgs=1440 cs=0 l=1 rev1=1 crypto rx=0x7fa7ec009870 tx=0x7fa7ec01c510 comp rx=0 tx=0).stop 2023-11-26T07:24:54.867 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:54.866+0000 7fa7f7b55640 1 -- 172.21.15.143:0/2175798159 shutdown_connections 2023-11-26T07:24:54.867 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:54.866+0000 7fa7f7b55640 1 --2- 172.21.15.143:0/2175798159 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fa7f00cf7c0 0x7fa7f015efa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:24:54.867 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:54.866+0000 7fa7f7b55640 1 --2- 172.21.15.143:0/2175798159 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fa7f013e290 0x7fa7f00cf280 unknown :-1 s=CLOSED pgs=1440 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:24:54.867 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:54.866+0000 7fa7f7b55640 1 -- 172.21.15.143:0/2175798159 >> 172.21.15.143:0/2175798159 conn(0x7fa7f006a890 msgr2=0x7fa7f006aca0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:24:54.867 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:54.866+0000 7fa7f7b55640 1 -- 172.21.15.143:0/2175798159 shutdown_connections 2023-11-26T07:24:54.867 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:54.866+0000 7fa7f7b55640 1 -- 172.21.15.143:0/2175798159 wait complete. 2023-11-26T07:24:54.868 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:54.866+0000 7fa7f7b55640 1 Processor -- start 2023-11-26T07:24:54.868 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:54.866+0000 7fa7f7b55640 1 -- start start 2023-11-26T07:24:54.870 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:54.866+0000 7fa7f7b55640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fa7f00cf7c0 0x7fa7f0139520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:24:54.871 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:54.866+0000 7fa7f7b55640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fa7f013e290 0x7fa7f0138ef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:24:54.871 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:54.866+0000 7fa7f7b55640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fa7f015f4e0 0x7fa7f00b1490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:24:54.871 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:54.866+0000 7fa7f7b55640 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa7f00b19d0 con 0x7fa7f013e290 2023-11-26T07:24:54.871 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:54.866+0000 7fa7f7b55640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa7f01f74b0 con 0x7fa7f00cf7c0 2023-11-26T07:24:54.871 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:54.866+0000 7fa7f7b55640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa7f01f66f0 con 0x7fa7f015f4e0 2023-11-26T07:24:54.871 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:54.870+0000 7fa7f50f0640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fa7f013e290 0x7fa7f0138ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:24:54.872 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:54.870+0000 7fa7f58f1640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fa7f00cf7c0 0x7fa7f0139520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:24:54.872 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:54.870+0000 7fa7f50f0640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fa7f013e290 0x7fa7f0138ef0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:45814/0 (socket says 172.21.15.143:45814) 2023-11-26T07:24:54.872 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:54.870+0000 7fa7f50f0640 1 -- 172.21.15.143:0/3545273443 learned_addr learned my addr 172.21.15.143:0/3545273443 (peer_addr_for_me v2:172.21.15.143:0/0) 2023-11-26T07:24:54.872 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:54.870+0000 7fa7f60f2640 1 --2- 172.21.15.143:0/3545273443 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fa7f015f4e0 0x7fa7f00b1490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:24:54.872 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:54.870+0000 7fa7f50f0640 1 -- 172.21.15.143:0/3545273443 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fa7f015f4e0 msgr2=0x7fa7f00b1490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:24:54.872 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:54.870+0000 7fa7f50f0640 1 --2- 172.21.15.143:0/3545273443 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fa7f015f4e0 0x7fa7f00b1490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:24:54.873 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:54.870+0000 7fa7f50f0640 1 -- 172.21.15.143:0/3545273443 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fa7f00cf7c0 msgr2=0x7fa7f0139520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:24:54.873 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:54.870+0000 7fa7f50f0640 1 --2- 172.21.15.143:0/3545273443 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fa7f00cf7c0 0x7fa7f0139520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:24:54.873 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:54.870+0000 7fa7f50f0640 1 -- 172.21.15.143:0/3545273443 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa7ec0094d0 con 0x7fa7f013e290 2023-11-26T07:24:54.873 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:54.870+0000 7fa7f50f0640 1 --2- 172.21.15.143:0/3545273443 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fa7f013e290 0x7fa7f0138ef0 secure :-1 s=READY pgs=1441 cs=0 l=1 rev1=1 crypto rx=0x7fa7e000e930 tx=0x7fa7e000edf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:24:54.873 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:54.870+0000 7fa7e67fc640 1 -- 172.21.15.143:0/3545273443 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa7e0009690 con 0x7fa7f013e290 2023-11-26T07:24:54.873 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:54.870+0000 7fa7e67fc640 1 -- 172.21.15.143:0/3545273443 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa7e000ca00 con 0x7fa7f013e290 2023-11-26T07:24:54.873 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:54.870+0000 7fa7f7b55640 1 -- 172.21.15.143:0/3545273443 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa7f01f69d0 con 0x7fa7f013e290 2023-11-26T07:24:54.874 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:54.870+0000 7fa7e67fc640 1 -- 172.21.15.143:0/3545273443 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa7e000cd70 con 0x7fa7f013e290 2023-11-26T07:24:54.874 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:54.870+0000 7fa7f7b55640 1 -- 172.21.15.143:0/3545273443 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa7f01f6f90 con 0x7fa7f013e290 2023-11-26T07:24:54.875 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:54.870+0000 7fa7f7b55640 1 -- 172.21.15.143:0/3545273443 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa7f0067680 con 0x7fa7f013e290 2023-11-26T07:24:54.875 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:54.870+0000 7fa7e67fc640 1 -- 172.21.15.143:0/3545273443 <== mon.0 v2:172.21.15.143:3300/0 4 ==== mgrmap(e 18) v1 ==== 86107+0+0 (secure 0 0 0) 0x7fa7e0013470 con 0x7fa7f013e290 2023-11-26T07:24:54.875 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:54.874+0000 7fa7e67fc640 1 --2- 172.21.15.143:0/3545273443 >> [v2:172.21.15.188:6836/3401128553,v1:172.21.15.188:6837/3401128553] conn(0x7fa7d4066770 0x7fa7d4068c30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:24:54.875 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:54.874+0000 7fa7e67fc640 1 -- 172.21.15.143:0/3545273443 <== mon.0 v2:172.21.15.143:3300/0 5 ==== osd_map(106..106 src has 1..106) v4 ==== 4943+0+0 (secure 0 0 0) 0x7fa7e00840c0 con 0x7fa7f013e290 2023-11-26T07:24:54.875 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:54.874+0000 7fa7f58f1640 1 --2- 172.21.15.143:0/3545273443 >> [v2:172.21.15.188:6836/3401128553,v1:172.21.15.188:6837/3401128553] conn(0x7fa7d4066770 0x7fa7d4068c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:24:54.876 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:54.874+0000 7fa7f58f1640 1 --2- 172.21.15.143:0/3545273443 >> [v2:172.21.15.188:6836/3401128553,v1:172.21.15.188:6837/3401128553] conn(0x7fa7d4066770 0x7fa7d4068c30 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7fa7ec0ae000 tx=0x7fa7ec0b5040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:24:54.877 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:54.878+0000 7fa7e67fc640 1 -- 172.21.15.143:0/3545273443 <== mon.0 v2:172.21.15.143:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172649 (secure 0 0 0) 0x7fa7e00516f0 con 0x7fa7f013e290 2023-11-26T07:24:55.028 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:55.026+0000 7fa7f7b55640 1 -- 172.21.15.143:0/3545273443 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "config rm", "who": "mon", "name": "mon_allow_pool_delete"} v 0) v1 -- 0x7fa7f006df50 con 0x7fa7f013e290 2023-11-26T07:24:55.033 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:55.030+0000 7fa7e67fc640 1 -- 172.21.15.143:0/3545273443 <== mon.0 v2:172.21.15.143:3300/0 7 ==== mon_command_ack([{"prefix": "config rm", "who": "mon", "name": "mon_allow_pool_delete"}]=0 v21) v1 ==== 104+0+0 (secure 0 0 0) 0x7fa7e0054d10 con 0x7fa7f013e290 2023-11-26T07:24:55.035 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:55.034+0000 7fa7f7b55640 1 -- 172.21.15.143:0/3545273443 >> [v2:172.21.15.188:6836/3401128553,v1:172.21.15.188:6837/3401128553] conn(0x7fa7d4066770 msgr2=0x7fa7d4068c30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:24:55.035 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:55.034+0000 7fa7f7b55640 1 --2- 172.21.15.143:0/3545273443 >> [v2:172.21.15.188:6836/3401128553,v1:172.21.15.188:6837/3401128553] conn(0x7fa7d4066770 0x7fa7d4068c30 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7fa7ec0ae000 tx=0x7fa7ec0b5040 comp rx=0 tx=0).stop 2023-11-26T07:24:55.035 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:55.034+0000 7fa7f7b55640 1 -- 172.21.15.143:0/3545273443 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fa7f013e290 msgr2=0x7fa7f0138ef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:24:55.035 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:55.034+0000 7fa7f7b55640 1 --2- 172.21.15.143:0/3545273443 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fa7f013e290 0x7fa7f0138ef0 secure :-1 s=READY pgs=1441 cs=0 l=1 rev1=1 crypto rx=0x7fa7e000e930 tx=0x7fa7e000edf0 comp rx=0 tx=0).stop 2023-11-26T07:24:55.036 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:55.034+0000 7fa7f7b55640 1 -- 172.21.15.143:0/3545273443 shutdown_connections 2023-11-26T07:24:55.036 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:55.034+0000 7fa7f7b55640 1 --2- 172.21.15.143:0/3545273443 >> [v2:172.21.15.188:6836/3401128553,v1:172.21.15.188:6837/3401128553] conn(0x7fa7d4066770 0x7fa7d4068c30 unknown :-1 s=CLOSED pgs=181 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:24:55.036 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:55.034+0000 7fa7f7b55640 1 --2- 172.21.15.143:0/3545273443 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fa7f015f4e0 0x7fa7f00b1490 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:24:55.036 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:55.034+0000 7fa7f7b55640 1 --2- 172.21.15.143:0/3545273443 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fa7f013e290 0x7fa7f0138ef0 unknown :-1 s=CLOSED pgs=1441 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:24:55.036 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:55.034+0000 7fa7f7b55640 1 --2- 172.21.15.143:0/3545273443 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fa7f00cf7c0 0x7fa7f0139520 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:24:55.036 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:55.034+0000 7fa7f7b55640 1 -- 172.21.15.143:0/3545273443 >> 172.21.15.143:0/3545273443 conn(0x7fa7f006a890 msgr2=0x7fa7f006ac70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:24:55.036 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:55.034+0000 7fa7f7b55640 1 -- 172.21.15.143:0/3545273443 shutdown_connections 2023-11-26T07:24:55.037 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:55.034+0000 7fa7f7b55640 1 -- 172.21.15.143:0/3545273443 wait complete. 2023-11-26T07:24:55.054 DEBUG:teuthology.orchestra.run.smithi143:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Ended test tasks.cephfs.test_volumes.TestMisc.test_subvolume_no_upgrade_v1_sanity' 2023-11-26T07:24:55.192 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:55.190+0000 7f51acdc4640 1 Processor -- start 2023-11-26T07:24:55.192 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:55.190+0000 7f51acdc4640 1 -- start start 2023-11-26T07:24:55.194 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:55.194+0000 7f51acdc4640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f51a81596d0 0x7f51a8071850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:24:55.194 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:55.194+0000 7f51acdc4640 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7f51a80b2f80 0x7f51a806cbe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:24:55.195 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:55.194+0000 7f51acdc4640 1 -- --> v1:172.21.15.188:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f51a806c750 con 0x7f51a80ce680 2023-11-26T07:24:55.195 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:55.194+0000 7f51acdc4640 1 -- --> v2:172.21.15.143:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f51a80cbe00 con 0x7f51a80b2f80 2023-11-26T07:24:55.195 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:55.194+0000 7f51acdc4640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f51a80b3550 con 0x7f51a81596d0 2023-11-26T07:24:55.195 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:55.194+0000 7f51a6d9d640 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7f51a80b2f80 0x7f51a806cbe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:24:55.195 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:55.194+0000 7f51a6d9d640 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7f51a80b2f80 0x7f51a806cbe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:45828/0 (socket says 172.21.15.143:45828) 2023-11-26T07:24:55.195 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:55.194+0000 7f51a6d9d640 1 -- 172.21.15.143:0/3575690943 learned_addr learned my addr 172.21.15.143:0/3575690943 (peer_addr_for_me v2:172.21.15.143:0/0) 2023-11-26T07:24:55.195 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:55.194+0000 7f51a659c640 1 --2- 172.21.15.143:0/3575690943 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f51a81596d0 0x7f51a8071850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:24:55.195 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:55.194+0000 7f51a6d9d640 1 -- 172.21.15.143:0/3575690943 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f51a81596d0 msgr2=0x7f51a8071850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-26T07:24:55.195 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:55.194+0000 7f51a6d9d640 1 --2- 172.21.15.143:0/3575690943 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f51a81596d0 0x7f51a8071850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:24:55.195 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:55.194+0000 7f51a6d9d640 1 -- 172.21.15.143:0/3575690943 >> v1:172.21.15.188:6789/0 conn(0x7f51a80ce680 legacy=0x7f51a80b2870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-26T07:24:55.195 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:55.194+0000 7f51a6d9d640 1 -- 172.21.15.143:0/3575690943 --> v2:172.21.15.143:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f51a8071e80 con 0x7f51a80b2f80 2023-11-26T07:24:55.195 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:55.194+0000 7f51a659c640 1 --2- 172.21.15.143:0/3575690943 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f51a81596d0 0x7f51a8071850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-26T07:24:55.197 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:55.198+0000 7f51a6d9d640 1 --2- 172.21.15.143:0/3575690943 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f51a80b2f80 0x7f51a806cbe0 secure :-1 s=READY pgs=1442 cs=0 l=1 rev1=1 crypto rx=0x7f51a0009e50 tx=0x7f51a001c9d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e57519243fba6de8 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:24:55.197 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:55.198+0000 7f51a559a640 1 -- 172.21.15.143:0/3575690943 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f51a00a56c0 con 0x7f51a80b2f80 2023-11-26T07:24:55.197 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:55.198+0000 7f51a559a640 1 -- 172.21.15.143:0/3575690943 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f51a00a5820 con 0x7f51a80b2f80 2023-11-26T07:24:55.197 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:55.198+0000 7f51a559a640 1 -- 172.21.15.143:0/3575690943 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f51a00a5bb0 con 0x7f51a80b2f80 2023-11-26T07:24:55.198 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:55.198+0000 7f51acdc4640 1 -- 172.21.15.143:0/3575690943 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f51a80b2f80 msgr2=0x7f51a806cbe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:24:55.198 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:55.198+0000 7f51acdc4640 1 --2- 172.21.15.143:0/3575690943 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f51a80b2f80 0x7f51a806cbe0 secure :-1 s=READY pgs=1442 cs=0 l=1 rev1=1 crypto rx=0x7f51a0009e50 tx=0x7f51a001c9d0 comp rx=0 tx=0).stop 2023-11-26T07:24:55.198 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:55.198+0000 7f51acdc4640 1 -- 172.21.15.143:0/3575690943 shutdown_connections 2023-11-26T07:24:55.199 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:55.198+0000 7f51acdc4640 1 --2- 172.21.15.143:0/3575690943 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f51a80b2f80 0x7f51a806cbe0 unknown :-1 s=CLOSED pgs=1442 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:24:55.199 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:55.198+0000 7f51acdc4640 1 --2- 172.21.15.143:0/3575690943 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f51a81596d0 0x7f51a8071850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:24:55.199 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:55.198+0000 7f51acdc4640 1 -- 172.21.15.143:0/3575690943 >> 172.21.15.143:0/3575690943 conn(0x7f51a806a890 msgr2=0x7f51a806aca0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:24:55.199 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:55.198+0000 7f51acdc4640 1 -- 172.21.15.143:0/3575690943 shutdown_connections 2023-11-26T07:24:55.199 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:55.198+0000 7f51acdc4640 1 -- 172.21.15.143:0/3575690943 wait complete. 2023-11-26T07:24:55.200 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:55.198+0000 7f51acdc4640 1 Processor -- start 2023-11-26T07:24:55.200 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:55.198+0000 7f51acdc4640 1 -- start start 2023-11-26T07:24:55.202 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:55.202+0000 7f51acdc4640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f51a80b2f80 0x7f51a815c5b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:24:55.202 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:55.202+0000 7f51acdc4640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f51a80ce680 0x7f51a8153f10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:24:55.202 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:55.202+0000 7f51acdc4640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f51a81596d0 0x7f51a814f050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:24:55.202 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:55.202+0000 7f51acdc4640 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f51a815caf0 con 0x7f51a81596d0 2023-11-26T07:24:55.202 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:55.202+0000 7f51acdc4640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f51a8147500 con 0x7f51a80b2f80 2023-11-26T07:24:55.202 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:55.202+0000 7f51acdc4640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f51a81477e0 con 0x7f51a80ce680 2023-11-26T07:24:55.202 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:55.202+0000 7f51a6d9d640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f51a81596d0 0x7f51a814f050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:24:55.202 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:55.202+0000 7f51a6d9d640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f51a81596d0 0x7f51a814f050 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:45838/0 (socket says 172.21.15.143:45838) 2023-11-26T07:24:55.202 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:55.202+0000 7f51a6d9d640 1 -- 172.21.15.143:0/1659962789 learned_addr learned my addr 172.21.15.143:0/1659962789 (peer_addr_for_me v2:172.21.15.143:0/0) 2023-11-26T07:24:55.203 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:55.202+0000 7f51a659c640 1 --2- 172.21.15.143:0/1659962789 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f51a80b2f80 0x7f51a815c5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:24:55.203 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:55.202+0000 7f51a5d9b640 1 --2- 172.21.15.143:0/1659962789 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f51a80ce680 0x7f51a8153f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:24:55.203 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:55.202+0000 7f51a6d9d640 1 -- 172.21.15.143:0/1659962789 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f51a80ce680 msgr2=0x7f51a8153f10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:24:55.203 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:55.202+0000 7f51a6d9d640 1 --2- 172.21.15.143:0/1659962789 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f51a80ce680 0x7f51a8153f10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:24:55.203 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:55.202+0000 7f51a6d9d640 1 -- 172.21.15.143:0/1659962789 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f51a80b2f80 msgr2=0x7f51a815c5b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:24:55.203 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:55.202+0000 7f51a6d9d640 1 --2- 172.21.15.143:0/1659962789 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f51a80b2f80 0x7f51a815c5b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:24:55.203 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:55.202+0000 7f51a6d9d640 1 -- 172.21.15.143:0/1659962789 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f51a0009b30 con 0x7f51a81596d0 2023-11-26T07:24:55.203 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:55.202+0000 7f51a6d9d640 1 --2- 172.21.15.143:0/1659962789 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f51a81596d0 0x7f51a814f050 secure :-1 s=READY pgs=1443 cs=0 l=1 rev1=1 crypto rx=0x7f51a001b080 tx=0x7f51a0019770 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:24:55.204 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:55.202+0000 7f5196ffd640 1 -- 172.21.15.143:0/1659962789 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f51a00b8670 con 0x7f51a81596d0 2023-11-26T07:24:55.204 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:55.202+0000 7f5196ffd640 1 -- 172.21.15.143:0/1659962789 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f51a00b87d0 con 0x7f51a81596d0 2023-11-26T07:24:55.204 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:55.202+0000 7f51acdc4640 1 -- 172.21.15.143:0/1659962789 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f51a8147ac0 con 0x7f51a81596d0 2023-11-26T07:24:55.204 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:55.202+0000 7f5196ffd640 1 -- 172.21.15.143:0/1659962789 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f51a00b8b80 con 0x7f51a81596d0 2023-11-26T07:24:55.204 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:55.202+0000 7f51acdc4640 1 -- 172.21.15.143:0/1659962789 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f51a8148080 con 0x7f51a81596d0 2023-11-26T07:24:55.204 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:55.202+0000 7f51acdc4640 1 -- 172.21.15.143:0/1659962789 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f51a8148270 con 0x7f51a81596d0 2023-11-26T07:24:55.205 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:55.202+0000 7f5196ffd640 1 -- 172.21.15.143:0/1659962789 <== mon.0 v2:172.21.15.143:3300/0 4 ==== mgrmap(e 18) v1 ==== 86107+0+0 (secure 0 0 0) 0x7f51a00b8ce0 con 0x7f51a81596d0 2023-11-26T07:24:55.205 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:55.206+0000 7f5196ffd640 1 --2- 172.21.15.143:0/1659962789 >> [v2:172.21.15.188:6836/3401128553,v1:172.21.15.188:6837/3401128553] conn(0x7f5188066740 0x7f5188068c00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:24:55.205 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:55.206+0000 7f5196ffd640 1 -- 172.21.15.143:0/1659962789 <== mon.0 v2:172.21.15.143:3300/0 5 ==== osd_map(106..106 src has 1..106) v4 ==== 4943+0+0 (secure 0 0 0) 0x7f51a011f440 con 0x7f51a81596d0 2023-11-26T07:24:55.205 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:55.206+0000 7f51a659c640 1 --2- 172.21.15.143:0/1659962789 >> [v2:172.21.15.188:6836/3401128553,v1:172.21.15.188:6837/3401128553] conn(0x7f5188066740 0x7f5188068c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:24:55.206 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:55.206+0000 7f51a659c640 1 --2- 172.21.15.143:0/1659962789 >> [v2:172.21.15.188:6836/3401128553,v1:172.21.15.188:6837/3401128553] conn(0x7f5188066740 0x7f5188068c00 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7f5198006fd0 tx=0x7f5198008040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:24:55.209 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:55.210+0000 7f5196ffd640 1 -- 172.21.15.143:0/1659962789 <== mon.0 v2:172.21.15.143:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172649 (secure 0 0 0) 0x7f51a00ecb40 con 0x7f51a81596d0 2023-11-26T07:24:55.409 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:55.406+0000 7f51acdc4640 1 -- 172.21.15.143:0/1659962789 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_volumes.TestMisc.test_subvolume_no_upgrade_v1_sanity"]} v 0) v1 -- 0x7f51a8148270 con 0x7f51a81596d0 2023-11-26T07:24:55.699 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:55.698+0000 7f5196ffd640 1 -- 172.21.15.143:0/1659962789 <== mon.0 v2:172.21.15.143:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_volumes.TestMisc.test_subvolume_no_upgrade_v1_sanity"]}]=0 v375) v1 ==== 149+0+0 (secure 0 0 0) 0x7f51a00f0160 con 0x7f51a81596d0 2023-11-26T07:24:55.701 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:55.702+0000 7f51acdc4640 1 -- 172.21.15.143:0/1659962789 >> [v2:172.21.15.188:6836/3401128553,v1:172.21.15.188:6837/3401128553] conn(0x7f5188066740 msgr2=0x7f5188068c00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:24:55.702 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:55.702+0000 7f51acdc4640 1 --2- 172.21.15.143:0/1659962789 >> [v2:172.21.15.188:6836/3401128553,v1:172.21.15.188:6837/3401128553] conn(0x7f5188066740 0x7f5188068c00 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7f5198006fd0 tx=0x7f5198008040 comp rx=0 tx=0).stop 2023-11-26T07:24:55.702 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:55.702+0000 7f51acdc4640 1 -- 172.21.15.143:0/1659962789 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f51a81596d0 msgr2=0x7f51a814f050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:24:55.702 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:55.702+0000 7f51acdc4640 1 --2- 172.21.15.143:0/1659962789 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f51a81596d0 0x7f51a814f050 secure :-1 s=READY pgs=1443 cs=0 l=1 rev1=1 crypto rx=0x7f51a001b080 tx=0x7f51a0019770 comp rx=0 tx=0).stop 2023-11-26T07:24:55.702 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:55.702+0000 7f51acdc4640 1 -- 172.21.15.143:0/1659962789 shutdown_connections 2023-11-26T07:24:55.702 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:55.702+0000 7f51acdc4640 1 --2- 172.21.15.143:0/1659962789 >> [v2:172.21.15.188:6836/3401128553,v1:172.21.15.188:6837/3401128553] conn(0x7f5188066740 0x7f5188068c00 unknown :-1 s=CLOSED pgs=182 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:24:55.702 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:55.702+0000 7f51acdc4640 1 --2- 172.21.15.143:0/1659962789 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f51a81596d0 0x7f51a814f050 unknown :-1 s=CLOSED pgs=1443 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:24:55.702 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:55.702+0000 7f51acdc4640 1 --2- 172.21.15.143:0/1659962789 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f51a80ce680 0x7f51a8153f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:24:55.702 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:55.702+0000 7f51acdc4640 1 --2- 172.21.15.143:0/1659962789 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f51a80b2f80 0x7f51a815c5b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:24:55.702 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:55.702+0000 7f51acdc4640 1 -- 172.21.15.143:0/1659962789 >> 172.21.15.143:0/1659962789 conn(0x7f51a806a890 msgr2=0x7f51a80714b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:24:55.702 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:55.702+0000 7f51acdc4640 1 -- 172.21.15.143:0/1659962789 shutdown_connections 2023-11-26T07:24:55.702 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:55.702+0000 7f51acdc4640 1 -- 172.21.15.143:0/1659962789 wait complete. 2023-11-26T07:24:55.723 INFO:tasks.cephfs_test_runner:poor man's upgrade test -- theme continues... ... FAIL 2023-11-26T07:24:55.723 INFO:tasks.cephfs_test_runner: 2023-11-26T07:24:55.723 INFO:tasks.cephfs_test_runner:====================================================================== 2023-11-26T07:24:55.723 INFO:tasks.cephfs_test_runner:FAIL: test_subvolume_no_upgrade_v1_sanity (tasks.cephfs.test_volumes.TestMisc) 2023-11-26T07:24:55.723 INFO:tasks.cephfs_test_runner:poor man's upgrade test -- theme continues... 2023-11-26T07:24:55.723 INFO:tasks.cephfs_test_runner:---------------------------------------------------------------------- 2023-11-26T07:24:55.724 INFO:tasks.cephfs_test_runner:Traceback (most recent call last): 2023-11-26T07:24:55.724 INFO:tasks.cephfs_test_runner: File "/home/teuthworker/src/git.ceph.com_ceph-c_44c2237fe77713214148c1fd622c872f346f1b99/qa/tasks/cephfs/test_volumes.py", line 7763, in test_subvolume_no_upgrade_v1_sanity 2023-11-26T07:24:55.724 INFO:tasks.cephfs_test_runner: self.assertEqual(snap_info["has_pending_clones"], "no") 2023-11-26T07:24:55.724 INFO:tasks.cephfs_test_runner:AssertionError: 'yes' != 'no' 2023-11-26T07:24:55.724 INFO:tasks.cephfs_test_runner:- yes 2023-11-26T07:24:55.724 INFO:tasks.cephfs_test_runner:+ no 2023-11-26T07:24:55.724 INFO:tasks.cephfs_test_runner: 2023-11-26T07:24:55.724 INFO:tasks.cephfs_test_runner: 2023-11-26T07:24:55.724 INFO:tasks.cephfs_test_runner:---------------------------------------------------------------------- 2023-11-26T07:24:55.724 INFO:tasks.cephfs_test_runner:Ran 6 tests in 393.888s 2023-11-26T07:24:55.724 INFO:tasks.cephfs_test_runner: 2023-11-26T07:24:55.724 INFO:tasks.cephfs_test_runner:FAILED (failures=1) 2023-11-26T07:24:55.724 INFO:tasks.cephfs_test_runner: 2023-11-26T07:24:55.725 INFO:tasks.cephfs_test_runner:====================================================================== 2023-11-26T07:24:55.725 INFO:tasks.cephfs_test_runner:FAIL: test_subvolume_no_upgrade_v1_sanity (tasks.cephfs.test_volumes.TestMisc) 2023-11-26T07:24:55.725 INFO:tasks.cephfs_test_runner:poor man's upgrade test -- theme continues... 2023-11-26T07:24:55.725 INFO:tasks.cephfs_test_runner:---------------------------------------------------------------------- 2023-11-26T07:24:55.725 INFO:tasks.cephfs_test_runner:Traceback (most recent call last): 2023-11-26T07:24:55.725 INFO:tasks.cephfs_test_runner: File "/home/teuthworker/src/git.ceph.com_ceph-c_44c2237fe77713214148c1fd622c872f346f1b99/qa/tasks/cephfs/test_volumes.py", line 7763, in test_subvolume_no_upgrade_v1_sanity 2023-11-26T07:24:55.725 INFO:tasks.cephfs_test_runner: self.assertEqual(snap_info["has_pending_clones"], "no") 2023-11-26T07:24:55.725 INFO:tasks.cephfs_test_runner:AssertionError: 'yes' != 'no' 2023-11-26T07:24:55.725 INFO:tasks.cephfs_test_runner:- yes 2023-11-26T07:24:55.725 INFO:tasks.cephfs_test_runner:+ no 2023-11-26T07:24:55.725 INFO:tasks.cephfs_test_runner: 2023-11-26T07:24:55.725 INFO:tasks.cephfs_test_runner: 2023-11-26T07:24:55.726 ERROR:teuthology.run_tasks:Saw exception from tasks. Traceback (most recent call last): File "/home/teuthworker/src/git.ceph.com_teuthology_2442239d2653456406c25ae0c71b689c8f2657b6/teuthology/run_tasks.py", line 109, in run_tasks manager.__enter__() File "/usr/lib/python3.8/contextlib.py", line 113, in __enter__ return next(self.gen) File "/home/teuthworker/src/git.ceph.com_ceph-c_44c2237fe77713214148c1fd622c872f346f1b99/qa/tasks/cephfs_test_runner.py", line 211, in task raise RuntimeError("Test failure: {0}".format(", ".join(bad_tests))) RuntimeError: Test failure: test_subvolume_no_upgrade_v1_sanity (tasks.cephfs.test_volumes.TestMisc) 2023-11-26T07:24:55.829 ERROR:teuthology.util.sentry: Sentry event: https://sentry.ceph.com/organizations/ceph/?query=0df9442983ae458daf9b9b579fdbfc90 Traceback (most recent call last): File "/home/teuthworker/src/git.ceph.com_teuthology_2442239d2653456406c25ae0c71b689c8f2657b6/teuthology/run_tasks.py", line 109, in run_tasks manager.__enter__() File "/usr/lib/python3.8/contextlib.py", line 113, in __enter__ return next(self.gen) File "/home/teuthworker/src/git.ceph.com_ceph-c_44c2237fe77713214148c1fd622c872f346f1b99/qa/tasks/cephfs_test_runner.py", line 211, in task raise RuntimeError("Test failure: {0}".format(", ".join(bad_tests))) RuntimeError: Test failure: test_subvolume_no_upgrade_v1_sanity (tasks.cephfs.test_volumes.TestMisc) 2023-11-26T07:24:55.831 DEBUG:teuthology.run_tasks:Unwinding manager cephfs_test_runner 2023-11-26T07:24:55.840 DEBUG:teuthology.run_tasks:Unwinding manager ceph-fuse 2023-11-26T07:24:55.883 INFO:tasks.ceph_fuse:Unmounting ceph-fuse clients... 2023-11-26T07:24:55.883 DEBUG:teuthology.orchestra.run.smithi143:> set -ex 2023-11-26T07:24:55.883 DEBUG:teuthology.orchestra.run.smithi143:> dd if=/proc/self/mounts of=/dev/stdout 2023-11-26T07:24:55.896 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,relatime,size=16354672k,nr_inodes=4088668,mode=755 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3275384k,mode=755 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=3455 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysusers.service ramfs ro,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3275380k,nr_inodes=818845,mode=700,uid=1000,gid=1271 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3275384k,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2023-11-26T07:24:55.896 DEBUG:teuthology.orchestra.run.smithi143:> set -ex 2023-11-26T07:24:55.896 DEBUG:teuthology.orchestra.run.smithi143:> dd if=/proc/self/mounts of=/dev/stdout 2023-11-26T07:24:55.945 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,relatime,size=16354672k,nr_inodes=4088668,mode=755 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3275384k,mode=755 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=3455 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysusers.service ramfs ro,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3275380k,nr_inodes=818845,mode=700,uid=1000,gid=1271 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3275384k,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2023-11-26T07:24:55.945 DEBUG:teuthology.orchestra.run.smithi188:> set -ex 2023-11-26T07:24:55.945 DEBUG:teuthology.orchestra.run.smithi188:> dd if=/proc/self/mounts of=/dev/stdout 2023-11-26T07:24:55.955 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,relatime,size=16354628k,nr_inodes=4088657,mode=755 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3275376k,mode=755 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=787 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysusers.service ramfs ro,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3275372k,nr_inodes=818843,mode=700,uid=1000,gid=1271 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3275376k,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2023-11-26T07:24:55.955 DEBUG:teuthology.orchestra.run.smithi188:> set -ex 2023-11-26T07:24:55.956 DEBUG:teuthology.orchestra.run.smithi188:> dd if=/proc/self/mounts of=/dev/stdout 2023-11-26T07:24:56.007 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,relatime,size=16354628k,nr_inodes=4088657,mode=755 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3275376k,mode=755 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=787 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysusers.service ramfs ro,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3275372k,nr_inodes=818843,mode=700,uid=1000,gid=1271 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3275376k,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2023-11-26T07:24:56.007 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-26T07:24:56.007 DEBUG:teuthology.orchestra.run.smithi188:> ip netns list 2023-11-26T07:24:56.060 INFO:teuthology.orchestra.run.smithi188.stdout:ceph-ns--home-ubuntu-cephtest-mnt.1 (id: 0) 2023-11-26T07:24:56.060 INFO:teuthology.orchestra.run.smithi188.stdout:ceph-ns--home-ubuntu-cephtest-mnt.3 (id: 1) 2023-11-26T07:24:56.061 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-26T07:24:56.061 DEBUG:teuthology.orchestra.run.smithi188:> sudo ip netns delete ceph-ns--home-ubuntu-cephtest-mnt.1 2023-11-26T07:24:56.124 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-26T07:24:56.125 DEBUG:teuthology.orchestra.run.smithi188:> sudo ip netns delete ceph-ns--home-ubuntu-cephtest-mnt.3 2023-11-26T07:24:56.214 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-26T07:24:56.214 DEBUG:teuthology.orchestra.run.smithi188:> sudo ip link delete ceph-brx 2023-11-26T07:24:56.394 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-26T07:24:56.394 DEBUG:teuthology.orchestra.run.smithi143:> ip netns list 2023-11-26T07:24:56.403 INFO:teuthology.orchestra.run.smithi143.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2023-11-26T07:24:56.404 INFO:teuthology.orchestra.run.smithi143.stdout:ceph-ns--home-ubuntu-cephtest-mnt.2 (id: 1) 2023-11-26T07:24:56.404 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-26T07:24:56.405 DEBUG:teuthology.orchestra.run.smithi143:> sudo ip netns delete ceph-ns--home-ubuntu-cephtest-mnt.0 2023-11-26T07:24:56.474 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-26T07:24:56.474 DEBUG:teuthology.orchestra.run.smithi143:> sudo ip netns delete ceph-ns--home-ubuntu-cephtest-mnt.2 2023-11-26T07:24:56.557 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-26T07:24:56.557 DEBUG:teuthology.orchestra.run.smithi143:> sudo ip link delete ceph-brx 2023-11-26T07:24:56.752 DEBUG:teuthology.run_tasks:Unwinding manager ceph 2023-11-26T07:24:56.761 INFO:tasks.ceph.ceph_manager.ceph:waiting for clean 2023-11-26T07:24:56.762 DEBUG:teuthology.orchestra.run.smithi143:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-11-26T07:24:56.902 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:56.902+0000 7f0c91e5a640 1 Processor -- start 2023-11-26T07:24:56.903 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:56.902+0000 7f0c91e5a640 1 -- start start 2023-11-26T07:24:56.904 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:56.902+0000 7f0c91e5a640 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7f0c8c070d10 0x7f0c8c15d900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:24:56.904 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:56.902+0000 7f0c91e5a640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f0c8c06cd50 0x7f0c8c0b2180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:24:56.904 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:56.902+0000 7f0c91e5a640 1 -- --> v1:172.21.15.143:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0c8c15b2f0 con 0x7f0c8c0cf280 2023-11-26T07:24:56.904 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:56.902+0000 7f0c91e5a640 1 -- --> v2:172.21.15.143:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0c8c06c5c0 con 0x7f0c8c070d10 2023-11-26T07:24:56.904 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:56.902+0000 7f0c91e5a640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0c8c15e1a0 con 0x7f0c8c06cd50 2023-11-26T07:24:56.904 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:56.902+0000 7f0c8affd640 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7f0c8c070d10 0x7f0c8c15d900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:24:56.905 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:56.902+0000 7f0c8b7fe640 1 --1- >> v1:172.21.15.143:6789/0 conn(0x7f0c8c0cf280 0x7f0c8c0cf660 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.143:6789/0 says I am v1:172.21.15.143:36272/0 (socket says 172.21.15.143:36272) 2023-11-26T07:24:56.905 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:56.902+0000 7f0c8b7fe640 1 -- 172.21.15.143:0/3566331065 learned_addr learned my addr 172.21.15.143:0/3566331065 (peer_addr_for_me v1:172.21.15.143:0/0) 2023-11-26T07:24:56.906 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:56.902+0000 7f0c8bfff640 1 --2- 172.21.15.143:0/3566331065 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f0c8c06cd50 0x7f0c8c0b2180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:24:56.906 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:56.902+0000 7f0c8affd640 1 -- 172.21.15.143:0/3566331065 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f0c8c06cd50 msgr2=0x7f0c8c0b2180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-26T07:24:56.906 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:56.902+0000 7f0c8affd640 1 --2- 172.21.15.143:0/3566331065 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f0c8c06cd50 0x7f0c8c0b2180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:24:56.906 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:56.902+0000 7f0c8affd640 1 -- 172.21.15.143:0/3566331065 >> v1:172.21.15.143:6789/0 conn(0x7f0c8c0cf280 legacy=0x7f0c8c0cf660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:24:56.906 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:56.902+0000 7f0c8affd640 1 -- 172.21.15.143:0/3566331065 --> v2:172.21.15.143:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0c8c15e480 con 0x7f0c8c070d10 2023-11-26T07:24:56.906 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:56.902+0000 7f0c8bfff640 1 --2- 172.21.15.143:0/3566331065 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f0c8c06cd50 0x7f0c8c0b2180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-26T07:24:56.907 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:56.902+0000 7f0c8a7fc640 1 -- 172.21.15.143:0/3566331065 <== mon.0 v1:172.21.15.143:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 95771699 0 0) 0x7f0c7c001120 con 0x7f0c8c0cf280 2023-11-26T07:24:56.908 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:56.906+0000 7f0c8affd640 1 --2- 172.21.15.143:0/3566331065 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f0c8c070d10 0x7f0c8c15d900 secure :-1 s=READY pgs=1445 cs=0 l=1 rev1=1 crypto rx=0x7f0c800097c0 tx=0x7f0c8001ec70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=30821849980956cd server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:24:56.908 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:56.906+0000 7f0c8a7fc640 1 -- 172.21.15.143:0/3566331065 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0c800a8970 con 0x7f0c8c070d10 2023-11-26T07:24:56.908 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:56.906+0000 7f0c8a7fc640 1 -- 172.21.15.143:0/3566331065 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0c800a8ad0 con 0x7f0c8c070d10 2023-11-26T07:24:56.908 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:56.906+0000 7f0c8a7fc640 1 -- 172.21.15.143:0/3566331065 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0c800a8e60 con 0x7f0c8c070d10 2023-11-26T07:24:56.909 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:56.906+0000 7f0c91e5a640 1 -- 172.21.15.143:0/3566331065 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f0c8c070d10 msgr2=0x7f0c8c15d900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:24:56.910 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:56.906+0000 7f0c91e5a640 1 --2- 172.21.15.143:0/3566331065 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f0c8c070d10 0x7f0c8c15d900 secure :-1 s=READY pgs=1445 cs=0 l=1 rev1=1 crypto rx=0x7f0c800097c0 tx=0x7f0c8001ec70 comp rx=0 tx=0).stop 2023-11-26T07:24:56.910 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:56.906+0000 7f0c91e5a640 1 -- 172.21.15.143:0/3566331065 shutdown_connections 2023-11-26T07:24:56.910 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:56.906+0000 7f0c91e5a640 1 --2- 172.21.15.143:0/3566331065 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f0c8c06cd50 0x7f0c8c0b2180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:24:56.910 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:56.906+0000 7f0c91e5a640 1 --2- 172.21.15.143:0/3566331065 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f0c8c070d10 0x7f0c8c15d900 unknown :-1 s=CLOSED pgs=1445 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:24:56.910 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:56.906+0000 7f0c91e5a640 1 -- 172.21.15.143:0/3566331065 >> 172.21.15.143:0/3566331065 conn(0x7f0c8c06a890 msgr2=0x7f0c8c06aca0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:24:56.911 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:56.906+0000 7f0c91e5a640 1 -- 172.21.15.143:0/3566331065 shutdown_connections 2023-11-26T07:24:56.911 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:56.906+0000 7f0c91e5a640 1 -- 172.21.15.143:0/3566331065 wait complete. 2023-11-26T07:24:56.913 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:56.910+0000 7f0c91e5a640 1 Processor -- start 2023-11-26T07:24:56.913 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:56.910+0000 7f0c91e5a640 1 -- start start 2023-11-26T07:24:56.913 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:56.910+0000 7f0c91e5a640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f0c8c06cd50 0x7f0c8c0a9850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:24:56.913 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:56.910+0000 7f0c91e5a640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f0c8c070d10 0x7f0c8c0abba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:24:56.913 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:56.910+0000 7f0c91e5a640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f0c8c0cf280 0x7f0c8c0b0630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:24:56.913 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:56.910+0000 7f0c91e5a640 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0c8c1f82b0 con 0x7f0c8c06cd50 2023-11-26T07:24:56.914 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:56.910+0000 7f0c91e5a640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0c8c0ac6b0 con 0x7f0c8c070d10 2023-11-26T07:24:56.914 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:56.910+0000 7f0c91e5a640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0c8c0ac3d0 con 0x7f0c8c0cf280 2023-11-26T07:24:56.914 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:56.910+0000 7f0c8b7fe640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f0c8c06cd50 0x7f0c8c0a9850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:24:56.914 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:56.910+0000 7f0c8b7fe640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f0c8c06cd50 0x7f0c8c0a9850 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:45858/0 (socket says 172.21.15.143:45858) 2023-11-26T07:24:56.914 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:56.910+0000 7f0c8b7fe640 1 -- 172.21.15.143:0/2483919222 learned_addr learned my addr 172.21.15.143:0/2483919222 (peer_addr_for_me v2:172.21.15.143:0/0) 2023-11-26T07:24:56.914 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:56.910+0000 7f0c8bfff640 1 --2- 172.21.15.143:0/2483919222 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f0c8c0cf280 0x7f0c8c0b0630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:24:56.914 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:56.910+0000 7f0c8affd640 1 --2- 172.21.15.143:0/2483919222 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f0c8c070d10 0x7f0c8c0abba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:24:56.915 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:56.910+0000 7f0c8b7fe640 1 -- 172.21.15.143:0/2483919222 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f0c8c0cf280 msgr2=0x7f0c8c0b0630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:24:56.915 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:56.910+0000 7f0c8b7fe640 1 --2- 172.21.15.143:0/2483919222 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f0c8c0cf280 0x7f0c8c0b0630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:24:56.915 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:56.910+0000 7f0c8b7fe640 1 -- 172.21.15.143:0/2483919222 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f0c8c070d10 msgr2=0x7f0c8c0abba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:24:56.915 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:56.910+0000 7f0c8b7fe640 1 --2- 172.21.15.143:0/2483919222 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f0c8c070d10 0x7f0c8c0abba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:24:56.915 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:56.910+0000 7f0c8b7fe640 1 -- 172.21.15.143:0/2483919222 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0c80009420 con 0x7f0c8c06cd50 2023-11-26T07:24:56.915 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:56.910+0000 7f0c8b7fe640 1 --2- 172.21.15.143:0/2483919222 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f0c8c06cd50 0x7f0c8c0a9850 secure :-1 s=READY pgs=1446 cs=0 l=1 rev1=1 crypto rx=0x7f0c7c00c780 tx=0x7f0c7c00cc40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:24:56.915 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:56.910+0000 7f0c73fff640 1 -- 172.21.15.143:0/2483919222 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0c7c0072d0 con 0x7f0c8c06cd50 2023-11-26T07:24:56.916 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:56.910+0000 7f0c73fff640 1 -- 172.21.15.143:0/2483919222 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0c7c007450 con 0x7f0c8c06cd50 2023-11-26T07:24:56.916 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:56.910+0000 7f0c73fff640 1 -- 172.21.15.143:0/2483919222 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0c7c0047f0 con 0x7f0c8c06cd50 2023-11-26T07:24:56.916 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:56.910+0000 7f0c91e5a640 1 -- 172.21.15.143:0/2483919222 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0c8c1585b0 con 0x7f0c8c06cd50 2023-11-26T07:24:56.916 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:56.910+0000 7f0c91e5a640 1 -- 172.21.15.143:0/2483919222 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0c8c158b70 con 0x7f0c8c06cd50 2023-11-26T07:24:56.917 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:56.914+0000 7f0c91e5a640 1 -- 172.21.15.143:0/2483919222 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0c8c067680 con 0x7f0c8c06cd50 2023-11-26T07:24:56.917 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:56.914+0000 7f0c73fff640 1 -- 172.21.15.143:0/2483919222 <== mon.0 v2:172.21.15.143:3300/0 4 ==== mgrmap(e 18) v1 ==== 86107+0+0 (secure 0 0 0) 0x7f0c7c004950 con 0x7f0c8c06cd50 2023-11-26T07:24:56.917 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:56.914+0000 7f0c73fff640 1 --2- 172.21.15.143:0/2483919222 >> [v2:172.21.15.188:6836/3401128553,v1:172.21.15.188:6837/3401128553] conn(0x7f0c78066740 0x7f0c78068c00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:24:56.917 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:56.914+0000 7f0c73fff640 1 -- 172.21.15.143:0/2483919222 <== mon.0 v2:172.21.15.143:3300/0 5 ==== osd_map(106..106 src has 1..106) v4 ==== 4943+0+0 (secure 0 0 0) 0x7f0c7c083060 con 0x7f0c8c06cd50 2023-11-26T07:24:56.917 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:56.914+0000 7f0c8affd640 1 --2- 172.21.15.143:0/2483919222 >> [v2:172.21.15.188:6836/3401128553,v1:172.21.15.188:6837/3401128553] conn(0x7f0c78066740 0x7f0c78068c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:24:56.918 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:56.918+0000 7f0c8affd640 1 --2- 172.21.15.143:0/2483919222 >> [v2:172.21.15.188:6836/3401128553,v1:172.21.15.188:6837/3401128553] conn(0x7f0c78066740 0x7f0c78068c00 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7f0c800b2000 tx=0x7f0c80008040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:24:56.920 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:56.918+0000 7f0c73fff640 1 -- 172.21.15.143:0/2483919222 <== mon.0 v2:172.21.15.143:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172649 (secure 0 0 0) 0x7f0c7c050540 con 0x7f0c8c06cd50 2023-11-26T07:24:57.066 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:57.066+0000 7f0c91e5a640 1 -- 172.21.15.143:0/2483919222 --> [v2:172.21.15.188:6836/3401128553,v1:172.21.15.188:6837/3401128553] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f0c8c15a6c0 con 0x7f0c78066740 2023-11-26T07:24:57.069 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:57.070+0000 7f0c73fff640 1 -- 172.21.15.143:0/2483919222 <== mgr.4108 v2:172.21.15.188:6836/3401128553 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+50755 (secure 0 0 0) 0x7f0c8c15a6c0 con 0x7f0c78066740 2023-11-26T07:24:57.070 INFO:teuthology.orchestra.run.smithi143.stdout: 2023-11-26T07:24:57.071 INFO:teuthology.orchestra.run.smithi143.stderr:dumped all 2023-11-26T07:24:57.072 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:57.070+0000 7f0c91e5a640 1 -- 172.21.15.143:0/2483919222 >> [v2:172.21.15.188:6836/3401128553,v1:172.21.15.188:6837/3401128553] conn(0x7f0c78066740 msgr2=0x7f0c78068c00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:24:57.072 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:57.070+0000 7f0c91e5a640 1 --2- 172.21.15.143:0/2483919222 >> [v2:172.21.15.188:6836/3401128553,v1:172.21.15.188:6837/3401128553] conn(0x7f0c78066740 0x7f0c78068c00 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7f0c800b2000 tx=0x7f0c80008040 comp rx=0 tx=0).stop 2023-11-26T07:24:57.072 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:57.070+0000 7f0c91e5a640 1 -- 172.21.15.143:0/2483919222 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f0c8c06cd50 msgr2=0x7f0c8c0a9850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:24:57.072 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:57.070+0000 7f0c91e5a640 1 --2- 172.21.15.143:0/2483919222 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f0c8c06cd50 0x7f0c8c0a9850 secure :-1 s=READY pgs=1446 cs=0 l=1 rev1=1 crypto rx=0x7f0c7c00c780 tx=0x7f0c7c00cc40 comp rx=0 tx=0).stop 2023-11-26T07:24:57.072 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:57.070+0000 7f0c91e5a640 1 -- 172.21.15.143:0/2483919222 shutdown_connections 2023-11-26T07:24:57.072 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:57.070+0000 7f0c91e5a640 1 --2- 172.21.15.143:0/2483919222 >> [v2:172.21.15.188:6836/3401128553,v1:172.21.15.188:6837/3401128553] conn(0x7f0c78066740 0x7f0c78068c00 unknown :-1 s=CLOSED pgs=183 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:24:57.073 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:57.070+0000 7f0c91e5a640 1 --2- 172.21.15.143:0/2483919222 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f0c8c0cf280 0x7f0c8c0b0630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:24:57.073 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:57.070+0000 7f0c91e5a640 1 --2- 172.21.15.143:0/2483919222 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f0c8c070d10 0x7f0c8c0abba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:24:57.073 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:57.070+0000 7f0c91e5a640 1 --2- 172.21.15.143:0/2483919222 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f0c8c06cd50 0x7f0c8c0a9850 unknown :-1 s=CLOSED pgs=1446 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:24:57.073 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:57.070+0000 7f0c91e5a640 1 -- 172.21.15.143:0/2483919222 >> 172.21.15.143:0/2483919222 conn(0x7f0c8c06a890 msgr2=0x7f0c8c06d1e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:24:57.073 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:57.070+0000 7f0c91e5a640 1 -- 172.21.15.143:0/2483919222 shutdown_connections 2023-11-26T07:24:57.073 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:57.070+0000 7f0c91e5a640 1 -- 172.21.15.143:0/2483919222 wait complete. 2023-11-26T07:24:57.091 INFO:teuthology.orchestra.run.smithi143.stdout:{"pg_ready":true,"pg_map":{"version":90,"stamp":"2023-11-26T07:24:55.310127+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":212,"num_read_kb":306,"num_write":179,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":128,"ondisk_log_size":128,"up":2,"acting":2,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":194,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":294328,"kb_used_data":44908,"kb_used_omap":12,"kb_used_meta":249331,"kb_avail":754680392,"statfs":{"total":773094113280,"available":772792721408,"internally_reserved":0,"allocated":45985792,"data_stored":40116539,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12749,"internal_metadata":255315507},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-17207712,"num_objects":-42,"num_object_clones":0,"num_object_copies":-84,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":-42,"num_whiteouts":0,"num_read":-23,"num_read_kb":-8207,"num_write":-184,"num_write_kb":-16883,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":-14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"6.001813"},"pg_stats":[{"pgid":"1.0","version":"72'128","reported_seq":273,"reported_epoch":102,"state":"active+clean","last_fresh":"2023-11-26T07:24:08.522394+0000","last_change":"2023-11-26T07:17:43.827802+0000","last_active":"2023-11-26T07:24:08.522394+0000","last_peered":"2023-11-26T07:24:08.522394+0000","last_clean":"2023-11-26T07:24:08.522394+0000","last_became_active":"2023-11-26T07:17:43.827308+0000","last_became_peered":"2023-11-26T07:17:43.827308+0000","last_unstale":"2023-11-26T07:24:08.522394+0000","last_undegraded":"2023-11-26T07:24:08.522394+0000","last_fullsized":"2023-11-26T07:24:08.522394+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:17:42.160945+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:17:42.160945+0000","last_clean_scrub_stamp":"2023-11-26T07:17:42.160945+0000","objects_scrubbed":0,"log_size":128,"log_dups_size":0,"ondisk_log_size":128,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T08:17:54.167366+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":212,"num_read_kb":306,"num_write":179,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":212,"num_read_kb":306,"num_write":179,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":128,"ondisk_log_size":128,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574936,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":39172,"kb_used_data":7996,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94332668,"statfs":{"total":96636764160,"available":96596652032,"internally_reserved":0,"allocated":8187904,"data_stored":7445421,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":31914442},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Nov 26 07:23:58 2023","interfaces":[{"interface":"back","average":{"1min":0.68200000000000005,"5min":0.70299999999999996,"15min":0.71099999999999997},"min":{"1min":0.47299999999999998,"5min":0.42999999999999999,"15min":0.42999999999999999},"max":{"1min":1.036,"5min":1.036,"15min":1.036},"last":0.64300000000000002},{"interface":"front","average":{"1min":0.70199999999999996,"5min":0.72699999999999998,"15min":0.71499999999999997},"min":{"1min":0.42299999999999999,"5min":0.42299999999999999,"15min":0.42299999999999999},"max":{"1min":1.014,"5min":1.014,"15min":1.0980000000000001},"last":0.63}]},{"osd":1,"last update":"Sun Nov 26 07:23:58 2023","interfaces":[{"interface":"back","average":{"1min":0.68400000000000005,"5min":0.68700000000000006,"15min":0.68300000000000005},"min":{"1min":0.45100000000000001,"5min":0.435,"15min":0.435},"max":{"1min":0.85299999999999998,"5min":1.012,"15min":1.0669999999999999},"last":0.75600000000000001},{"interface":"front","average":{"1min":0.66900000000000004,"5min":0.67700000000000005,"15min":0.78500000000000003},"min":{"1min":0.432,"5min":0.432,"15min":0.432},"max":{"1min":0.94699999999999995,"5min":0.94699999999999995,"15min":4.5309999999999997},"last":0.59699999999999998}]},{"osd":2,"last update":"Sun Nov 26 07:23:58 2023","interfaces":[{"interface":"back","average":{"1min":0.69999999999999996,"5min":0.69999999999999996,"15min":0.68400000000000005},"min":{"1min":0.56999999999999995,"5min":0.39700000000000002,"15min":0.39700000000000002},"max":{"1min":0.91600000000000004,"5min":1.0800000000000001,"15min":1.0800000000000001},"last":0.61399999999999999},{"interface":"front","average":{"1min":0.68700000000000006,"5min":0.70499999999999996,"15min":0.70999999999999996},"min":{"1min":0.48999999999999999,"5min":0.40999999999999998,"15min":0.40999999999999998},"max":{"1min":0.97099999999999997,"5min":0.97699999999999998,"15min":1.1339999999999999},"last":0.71699999999999997}]},{"osd":3,"last update":"Sun Nov 26 07:23:58 2023","interfaces":[{"interface":"back","average":{"1min":0.73299999999999998,"5min":0.72799999999999998,"15min":0.70699999999999996},"min":{"1min":0.53200000000000003,"5min":0.47799999999999998,"15min":0.47799999999999998},"max":{"1min":0.99299999999999999,"5min":1.1240000000000001,"15min":1.1240000000000001},"last":0.73299999999999998},{"interface":"front","average":{"1min":0.72399999999999998,"5min":0.74299999999999999,"15min":0.73599999999999999},"min":{"1min":0.50700000000000001,"5min":0.433,"15min":0.433},"max":{"1min":0.93400000000000005,"5min":1.036,"15min":1.4490000000000001},"last":0.69299999999999995}]},{"osd":4,"last update":"Sun Nov 26 07:23:58 2023","interfaces":[{"interface":"back","average":{"1min":0.61099999999999999,"5min":0.61499999999999999,"15min":0.60499999999999998},"min":{"1min":0.48299999999999998,"5min":0.39400000000000002,"15min":0.39400000000000002},"max":{"1min":0.878,"5min":0.96299999999999997,"15min":0.96299999999999997},"last":0.495},{"interface":"front","average":{"1min":0.65500000000000003,"5min":0.64400000000000002,"15min":0.66300000000000003},"min":{"1min":0.45800000000000002,"5min":0.35999999999999999,"15min":0.35999999999999999},"max":{"1min":0.83599999999999997,"5min":1.0209999999999999,"15min":1.5509999999999999},"last":0.67200000000000004}]},{"osd":5,"last update":"Sun Nov 26 07:23:58 2023","interfaces":[{"interface":"back","average":{"1min":0.66100000000000003,"5min":0.67400000000000004,"15min":0.66800000000000004},"min":{"1min":0.36599999999999999,"5min":0.36599999999999999,"15min":0.36599999999999999},"max":{"1min":0.86799999999999999,"5min":1.056,"15min":1.415},"last":0.56899999999999995},{"interface":"front","average":{"1min":0.68300000000000005,"5min":0.68200000000000005,"15min":0.70299999999999996},"min":{"1min":0.435,"5min":0.435,"15min":0.435},"max":{"1min":0.873,"5min":1.0840000000000001,"15min":1.3859999999999999},"last":0.55100000000000005}]},{"osd":6,"last update":"Sun Nov 26 07:23:58 2023","interfaces":[{"interface":"back","average":{"1min":0.77300000000000002,"5min":0.78000000000000003,"15min":0.77400000000000002},"min":{"1min":0.53100000000000003,"5min":0.51700000000000002,"15min":0.51700000000000002},"max":{"1min":0.95499999999999996,"5min":1.123,"15min":1.3069999999999999},"last":0.65400000000000003},{"interface":"front","average":{"1min":0.71399999999999997,"5min":0.74399999999999999,"15min":0.76200000000000001},"min":{"1min":0.50600000000000001,"5min":0.46899999999999997,"15min":0.46899999999999997},"max":{"1min":0.92900000000000005,"5min":1.036,"15min":1.6759999999999999},"last":0.52100000000000002}]}]},{"osd":6,"up_from":13,"seq":55834574938,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":35504,"kb_used_data":4328,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94336336,"statfs":{"total":96636764160,"available":96600408064,"internally_reserved":0,"allocated":4431872,"data_stored":3708126,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Nov 26 07:23:56 2023","interfaces":[{"interface":"back","average":{"1min":0.65400000000000003,"5min":0.67800000000000005,"15min":0.68200000000000005},"min":{"1min":0.48999999999999999,"5min":0.32200000000000001,"15min":0.32200000000000001},"max":{"1min":0.875,"5min":0.91800000000000004,"15min":1.7549999999999999},"last":0.67100000000000004},{"interface":"front","average":{"1min":0.65500000000000003,"5min":0.69599999999999995,"15min":0.87},"min":{"1min":0.36299999999999999,"5min":0.36299999999999999,"15min":0.36299999999999999},"max":{"1min":0.82599999999999996,"5min":0.96199999999999997,"15min":5.7489999999999997},"last":0.65700000000000003}]},{"osd":1,"last update":"Sun Nov 26 07:23:56 2023","interfaces":[{"interface":"back","average":{"1min":0.65000000000000002,"5min":0.70799999999999996,"15min":0.84499999999999997},"min":{"1min":0.54200000000000004,"5min":0.41199999999999998,"15min":0.41199999999999998},"max":{"1min":0.78900000000000003,"5min":0.97799999999999998,"15min":5.4240000000000004},"last":0.72399999999999998},{"interface":"front","average":{"1min":0.68200000000000005,"5min":0.70799999999999996,"15min":0.83399999999999996},"min":{"1min":0.47599999999999998,"5min":0.47599999999999998,"15min":0.47599999999999998},"max":{"1min":0.88600000000000001,"5min":0.96499999999999997,"15min":5.6020000000000003},"last":0.59899999999999998}]},{"osd":2,"last update":"Sun Nov 26 07:23:56 2023","interfaces":[{"interface":"back","average":{"1min":0.623,"5min":0.66600000000000004,"15min":0.68600000000000005},"min":{"1min":0.47999999999999998,"5min":0.41699999999999998,"15min":0.41699999999999998},"max":{"1min":0.89700000000000002,"5min":0.93600000000000005,"15min":1.8520000000000001},"last":0.88},{"interface":"front","average":{"1min":0.71099999999999997,"5min":0.72199999999999998,"15min":0.72999999999999998},"min":{"1min":0.47999999999999998,"5min":0.437,"15min":0.437},"max":{"1min":0.91000000000000003,"5min":0.95199999999999996,"15min":1.9159999999999999},"last":0.82299999999999995}]},{"osd":3,"last update":"Sun Nov 26 07:23:56 2023","interfaces":[{"interface":"back","average":{"1min":0.67900000000000005,"5min":0.70999999999999996,"15min":0.83799999999999997},"min":{"1min":0.47399999999999998,"5min":0.46899999999999997,"15min":0.46899999999999997},"max":{"1min":0.85199999999999998,"5min":0.91500000000000004,"15min":5.4390000000000001},"last":0.753},{"interface":"front","average":{"1min":0.65000000000000002,"5min":0.70399999999999996,"15min":0.86399999999999999},"min":{"1min":0.44,"5min":0.44,"15min":0.44},"max":{"1min":0.81299999999999994,"5min":0.90500000000000003,"15min":5.5490000000000004},"last":0.84099999999999997}]},{"osd":4,"last update":"Sun Nov 26 07:23:56 2023","interfaces":[{"interface":"back","average":{"1min":0.61699999999999999,"5min":0.61199999999999999,"15min":0.63200000000000001},"min":{"1min":0.45700000000000002,"5min":0.371,"15min":0.371},"max":{"1min":0.86699999999999999,"5min":1.0129999999999999,"15min":1.0129999999999999},"last":0.61899999999999999},{"interface":"front","average":{"1min":0.623,"5min":0.63400000000000001,"15min":0.66100000000000003},"min":{"1min":0.41299999999999998,"5min":0.38600000000000001,"15min":0.38600000000000001},"max":{"1min":0.85899999999999999,"5min":1.0349999999999999,"15min":1.0349999999999999},"last":0.69699999999999995}]},{"osd":5,"last update":"Sun Nov 26 07:23:56 2023","interfaces":[{"interface":"back","average":{"1min":0.57999999999999996,"5min":0.61899999999999999,"15min":0.64500000000000002},"min":{"1min":0.42799999999999999,"5min":0.38200000000000001,"15min":0.38200000000000001},"max":{"1min":0.85599999999999998,"5min":0.91000000000000003,"15min":0.91000000000000003},"last":0.93500000000000005},{"interface":"front","average":{"1min":0.66100000000000003,"5min":0.68700000000000006,"15min":0.70399999999999996},"min":{"1min":0.501,"5min":0.44,"15min":0.436},"max":{"1min":0.91600000000000004,"5min":1.0960000000000001,"15min":1.0960000000000001},"last":0.77500000000000002}]},{"osd":7,"last update":"Sun Nov 26 07:23:56 2023","interfaces":[{"interface":"back","average":{"1min":0.66000000000000003,"5min":0.69999999999999996,"15min":0.72799999999999998},"min":{"1min":0.497,"5min":0.42599999999999999,"15min":0.42599999999999999},"max":{"1min":0.89400000000000002,"5min":0.95099999999999996,"15min":0.998},"last":0.80300000000000005},{"interface":"front","average":{"1min":0.64800000000000002,"5min":0.68300000000000005,"15min":0.70499999999999996},"min":{"1min":0.55200000000000005,"5min":0.46400000000000002,"15min":0.46400000000000002},"max":{"1min":0.77500000000000002,"5min":0.98799999999999999,"15min":0.98799999999999999},"last":0.91000000000000003}]}]},{"osd":5,"up_from":13,"seq":55834574936,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":34484,"kb_used_data":3308,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94337356,"statfs":{"total":96636764160,"available":96601452544,"internally_reserved":0,"allocated":3387392,"data_stored":2660084,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Nov 26 07:24:50 2023","interfaces":[{"interface":"back","average":{"1min":0.61899999999999999,"5min":0.70499999999999996,"15min":0.68100000000000005},"min":{"1min":0.29699999999999999,"5min":0.29699999999999999,"15min":0.29699999999999999},"max":{"1min":0.80800000000000005,"5min":2.6949999999999998,"15min":2.6949999999999998},"last":0.57199999999999995},{"interface":"front","average":{"1min":0.64800000000000002,"5min":0.72399999999999998,"15min":0.68200000000000005},"min":{"1min":0.40200000000000002,"5min":0.38200000000000001,"15min":0.38200000000000001},"max":{"1min":0.88600000000000001,"5min":2.0150000000000001,"15min":2.0150000000000001},"last":0.67300000000000004}]},{"osd":1,"last update":"Sun Nov 26 07:24:50 2023","interfaces":[{"interface":"back","average":{"1min":0.60999999999999999,"5min":0.69099999999999995,"15min":0.67700000000000005},"min":{"1min":0.45900000000000002,"5min":0.41799999999999998,"15min":0.41599999999999998},"max":{"1min":0.77000000000000002,"5min":1.532,"15min":1.532},"last":0.61199999999999999},{"interface":"front","average":{"1min":0.64200000000000002,"5min":0.72399999999999998,"15min":0.69999999999999996},"min":{"1min":0.51800000000000002,"5min":0.443,"15min":0.443},"max":{"1min":0.81100000000000005,"5min":2.6400000000000001,"15min":2.6400000000000001},"last":0.621}]},{"osd":2,"last update":"Sun Nov 26 07:24:50 2023","interfaces":[{"interface":"back","average":{"1min":0.65000000000000002,"5min":0.71299999999999997,"15min":0.68799999999999994},"min":{"1min":0.42799999999999999,"5min":0.42099999999999999,"15min":0.42099999999999999},"max":{"1min":0.91500000000000004,"5min":1.665,"15min":1.665},"last":0.71099999999999997},{"interface":"front","average":{"1min":0.66300000000000003,"5min":0.745,"15min":0.69999999999999996},"min":{"1min":0.45700000000000002,"5min":0.45700000000000002,"15min":0.45700000000000002},"max":{"1min":0.85199999999999998,"5min":2.125,"15min":2.125},"last":0.47499999999999998}]},{"osd":3,"last update":"Sun Nov 26 07:24:50 2023","interfaces":[{"interface":"back","average":{"1min":0.68400000000000005,"5min":0.752,"15min":0.72899999999999998},"min":{"1min":0.52800000000000002,"5min":0.44700000000000001,"15min":0.43099999999999999},"max":{"1min":0.879,"5min":2.0529999999999999,"15min":2.0529999999999999},"last":0.59799999999999998},{"interface":"front","average":{"1min":0.67800000000000005,"5min":0.755,"15min":0.70299999999999996},"min":{"1min":0.55500000000000005,"5min":0.51900000000000002,"15min":0.44600000000000001},"max":{"1min":0.94099999999999995,"5min":1.629,"15min":1.629},"last":0.68500000000000005}]},{"osd":4,"last update":"Sun Nov 26 07:24:50 2023","interfaces":[{"interface":"back","average":{"1min":0.58499999999999996,"5min":0.61199999999999999,"15min":0.625},"min":{"1min":0.42199999999999999,"5min":0.35299999999999998,"15min":0.35299999999999998},"max":{"1min":0.84699999999999998,"5min":2.7429999999999999,"15min":2.7429999999999999},"last":0.42199999999999999},{"interface":"front","average":{"1min":0.60999999999999999,"5min":0.67300000000000004,"15min":0.69299999999999995},"min":{"1min":0.44400000000000001,"5min":0.44400000000000001,"15min":0.41299999999999998},"max":{"1min":0.85999999999999999,"5min":2.4630000000000001,"15min":2.4630000000000001},"last":0.54500000000000004}]},{"osd":6,"last update":"Sun Nov 26 07:24:50 2023","interfaces":[{"interface":"back","average":{"1min":0.63800000000000001,"5min":0.66600000000000004,"15min":0.67100000000000004},"min":{"1min":0.38300000000000001,"5min":0.372,"15min":0.372},"max":{"1min":1.5409999999999999,"5min":2.1619999999999999,"15min":2.1619999999999999},"last":0.52200000000000002},{"interface":"front","average":{"1min":0.63,"5min":0.69099999999999995,"15min":0.71499999999999997},"min":{"1min":0.47599999999999998,"5min":0.436,"15min":0.436},"max":{"1min":0.89500000000000002,"5min":2.7869999999999999,"15min":2.7869999999999999},"last":0.499}]},{"osd":7,"last update":"Sun Nov 26 07:24:50 2023","interfaces":[{"interface":"back","average":{"1min":0.66000000000000003,"5min":0.73099999999999998,"15min":0.77500000000000002},"min":{"1min":0.46999999999999997,"5min":0.41399999999999998,"15min":0.41399999999999998},"max":{"1min":0.878,"5min":2.1960000000000002,"15min":2.1960000000000002},"last":0.64500000000000002},{"interface":"front","average":{"1min":0.67700000000000005,"5min":0.748,"15min":0.76400000000000001},"min":{"1min":0.51100000000000001,"5min":0.44700000000000001,"15min":0.44700000000000001},"max":{"1min":0.94899999999999995,"5min":1.9510000000000001,"15min":1.9510000000000001},"last":0.65700000000000003}]}]},{"osd":4,"up_from":13,"seq":55834574938,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":35524,"kb_used_data":4344,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94336316,"statfs":{"total":96636764160,"available":96600387584,"internally_reserved":0,"allocated":4448256,"data_stored":3714592,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Nov 26 07:23:53 2023","interfaces":[{"interface":"back","average":{"1min":0.64600000000000002,"5min":0.65800000000000003,"15min":0.748},"min":{"1min":0.443,"5min":0.32100000000000001,"15min":0.32100000000000001},"max":{"1min":0.91100000000000003,"5min":0.96199999999999997,"15min":4.1689999999999996},"last":0.70599999999999996},{"interface":"front","average":{"1min":0.65300000000000002,"5min":0.84699999999999998,"15min":0.94999999999999996},"min":{"1min":0.40799999999999997,"5min":0.40799999999999997,"15min":0.40799999999999997},"max":{"1min":0.83299999999999996,"5min":18.021000000000001,"15min":18.021000000000001},"last":0.89200000000000002}]},{"osd":1,"last update":"Sun Nov 26 07:23:53 2023","interfaces":[{"interface":"back","average":{"1min":0.72699999999999998,"5min":0.73499999999999999,"15min":0.67600000000000005},"min":{"1min":0.55200000000000005,"5min":0.43099999999999999,"15min":0.43099999999999999},"max":{"1min":0.93500000000000005,"5min":4.3159999999999998,"15min":4.3159999999999998},"last":0.73199999999999998},{"interface":"front","average":{"1min":0.67400000000000004,"5min":0.76600000000000001,"15min":0.80200000000000005},"min":{"1min":0.436,"5min":0.39800000000000002,"15min":0.39800000000000002},"max":{"1min":0.94199999999999995,"5min":11.206,"15min":11.206},"last":0.79100000000000004}]},{"osd":2,"last update":"Sun Nov 26 07:23:53 2023","interfaces":[{"interface":"back","average":{"1min":0.71099999999999997,"5min":0.85199999999999998,"15min":0.82199999999999995},"min":{"1min":0.56599999999999995,"5min":0.433,"15min":0.433},"max":{"1min":0.84299999999999997,"5min":15.441000000000001,"15min":15.441000000000001},"last":0.874},{"interface":"front","average":{"1min":0.77400000000000002,"5min":0.85599999999999998,"15min":0.996},"min":{"1min":0.59599999999999997,"5min":0.54100000000000004,"15min":0.53400000000000003},"max":{"1min":0.93999999999999995,"5min":9.4800000000000004,"15min":9.4800000000000004},"last":0.95799999999999996}]},{"osd":3,"last update":"Sun Nov 26 07:23:53 2023","interfaces":[{"interface":"back","average":{"1min":0.71199999999999997,"5min":0.82899999999999996,"15min":0.76300000000000001},"min":{"1min":0.53600000000000003,"5min":0.441,"15min":0.441},"max":{"1min":0.95899999999999996,"5min":12.715,"15min":12.715},"last":0.75700000000000001},{"interface":"front","average":{"1min":0.76500000000000001,"5min":0.81499999999999995,"15min":0.85699999999999998},"min":{"1min":0.63600000000000001,"5min":0.50700000000000001,"15min":0.50700000000000001},"max":{"1min":1.0109999999999999,"5min":8.3239999999999998,"15min":8.3239999999999998},"last":0.81100000000000005}]},{"osd":5,"last update":"Sun Nov 26 07:23:53 2023","interfaces":[{"interface":"back","average":{"1min":0.67300000000000004,"5min":0.629,"15min":0.61699999999999999},"min":{"1min":0.41699999999999998,"5min":0.38200000000000001,"15min":0.38200000000000001},"max":{"1min":0.94199999999999995,"5min":0.94199999999999995,"15min":0.96199999999999997},"last":0.82899999999999996},{"interface":"front","average":{"1min":0.66000000000000003,"5min":0.72399999999999998,"15min":0.66800000000000004},"min":{"1min":0.36399999999999999,"5min":0.35599999999999998,"15min":0.35599999999999998},"max":{"1min":0.88400000000000001,"5min":11.148999999999999,"15min":11.148999999999999},"last":0.67300000000000004}]},{"osd":6,"last update":"Sun Nov 26 07:23:53 2023","interfaces":[{"interface":"back","average":{"1min":0.63400000000000001,"5min":0.60699999999999998,"15min":0.73099999999999998},"min":{"1min":0.48499999999999999,"5min":0.372,"15min":0.372},"max":{"1min":0.76700000000000002,"5min":0.89200000000000002,"15min":2.8330000000000002},"last":0.77400000000000002},{"interface":"front","average":{"1min":0.73799999999999999,"5min":0.71399999999999997,"15min":0.91000000000000003},"min":{"1min":0.47299999999999998,"5min":0.41299999999999998,"15min":0.41299999999999998},"max":{"1min":0.90500000000000003,"5min":2.472,"15min":7.6050000000000004},"last":0.92900000000000005}]},{"osd":7,"last update":"Sun Nov 26 07:23:53 2023","interfaces":[{"interface":"back","average":{"1min":0.71399999999999997,"5min":0.81699999999999995,"15min":0.84599999999999997},"min":{"1min":0.56899999999999995,"5min":0.47699999999999998,"15min":0.44700000000000001},"max":{"1min":0.85699999999999998,"5min":12.653,"15min":12.653},"last":0.85699999999999998},{"interface":"front","average":{"1min":0.78900000000000003,"5min":0.77500000000000002,"15min":0.85299999999999998},"min":{"1min":0.58499999999999996,"5min":0.33300000000000002,"15min":0.33300000000000002},"max":{"1min":0.91300000000000003,"5min":3.8319999999999999,"15min":3.8319999999999999},"last":0.91700000000000004}]}]},{"osd":3,"up_from":13,"seq":55834574939,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":36956,"kb_used_data":5780,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94334884,"statfs":{"total":96636764160,"available":96598921216,"internally_reserved":0,"allocated":5918720,"data_stored":5186897,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1592,"internal_metadata":31914440},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Nov 26 07:23:54 2023","interfaces":[{"interface":"back","average":{"1min":0.40100000000000002,"5min":0.41899999999999998,"15min":0.39200000000000002},"min":{"1min":0.29099999999999998,"5min":0.218,"15min":0.218},"max":{"1min":0.53600000000000003,"5min":0.72799999999999998,"15min":0.72799999999999998},"last":0.48099999999999998},{"interface":"front","average":{"1min":0.47599999999999998,"5min":0.46800000000000003,"15min":0.44400000000000001},"min":{"1min":0.30199999999999999,"5min":0.254,"15min":0.254},"max":{"1min":0.75900000000000001,"5min":0.77300000000000002,"15min":0.77300000000000002},"last":0.34799999999999998}]},{"osd":1,"last update":"Sun Nov 26 07:23:54 2023","interfaces":[{"interface":"back","average":{"1min":0.44400000000000001,"5min":0.441,"15min":0.42699999999999999},"min":{"1min":0.28799999999999998,"5min":0.28799999999999998,"15min":0.28799999999999998},"max":{"1min":0.59999999999999998,"5min":0.61299999999999999,"15min":0.752},"last":0.42399999999999999},{"interface":"front","average":{"1min":0.51800000000000002,"5min":0.496,"15min":0.48799999999999999},"min":{"1min":0.34000000000000002,"5min":0.29899999999999999,"15min":0.29899999999999999},"max":{"1min":0.80800000000000005,"5min":0.872,"15min":0.872},"last":0.501}]},{"osd":2,"last update":"Sun Nov 26 07:23:54 2023","interfaces":[{"interface":"back","average":{"1min":0.56000000000000005,"5min":0.56599999999999995,"15min":0.54400000000000004},"min":{"1min":0.442,"5min":0.376,"15min":0.376},"max":{"1min":0.77200000000000002,"5min":0.97199999999999998,"15min":0.97199999999999998},"last":0.36599999999999999},{"interface":"front","average":{"1min":0.52100000000000002,"5min":0.53700000000000003,"15min":0.51500000000000001},"min":{"1min":0.32600000000000001,"5min":0.32100000000000001,"15min":0.32100000000000001},"max":{"1min":0.70899999999999996,"5min":0.89700000000000002,"15min":0.89700000000000002},"last":0.52500000000000002}]},{"osd":4,"last update":"Sun Nov 26 07:23:54 2023","interfaces":[{"interface":"back","average":{"1min":0.72599999999999998,"5min":0.71099999999999997,"15min":0.70699999999999996},"min":{"1min":0.51400000000000001,"5min":0.39600000000000002,"15min":0.39600000000000002},"max":{"1min":0.88300000000000001,"5min":1.091,"15min":1.091},"last":0.94399999999999995},{"interface":"front","average":{"1min":0.71699999999999997,"5min":0.72099999999999997,"15min":0.72499999999999998},"min":{"1min":0.52400000000000002,"5min":0.39900000000000002,"15min":0.39900000000000002},"max":{"1min":1.089,"5min":1.242,"15min":1.242},"last":0.71199999999999997}]},{"osd":5,"last update":"Sun Nov 26 07:23:54 2023","interfaces":[{"interface":"back","average":{"1min":0.71599999999999997,"5min":0.70199999999999996,"15min":0.72599999999999998},"min":{"1min":0.48699999999999999,"5min":0.45900000000000002,"15min":0.45900000000000002},"max":{"1min":0.96199999999999997,"5min":1.105,"15min":1.105},"last":0.68999999999999995},{"interface":"front","average":{"1min":0.74399999999999999,"5min":0.73699999999999999,"15min":0.73699999999999999},"min":{"1min":0.54700000000000004,"5min":0.47299999999999998,"15min":0.47299999999999998},"max":{"1min":0.94199999999999995,"5min":1.1319999999999999,"15min":1.1319999999999999},"last":0.747}]},{"osd":6,"last update":"Sun Nov 26 07:23:54 2023","interfaces":[{"interface":"back","average":{"1min":0.76900000000000002,"5min":0.75900000000000001,"15min":0.79500000000000004},"min":{"1min":0.51300000000000001,"5min":0.5,"15min":0.5},"max":{"1min":0.96499999999999997,"5min":1.214,"15min":1.2729999999999999},"last":0.92500000000000004},{"interface":"front","average":{"1min":0.76800000000000002,"5min":0.754,"15min":0.78000000000000003},"min":{"1min":0.61799999999999999,"5min":0.48799999999999999,"15min":0.47299999999999998},"max":{"1min":1.1060000000000001,"5min":1.1060000000000001,"15min":1.204},"last":0.97399999999999998}]},{"osd":7,"last update":"Sun Nov 26 07:23:54 2023","interfaces":[{"interface":"back","average":{"1min":0.78200000000000003,"5min":0.755,"15min":0.78500000000000003},"min":{"1min":0.496,"5min":0.437,"15min":0.437},"max":{"1min":1.0349999999999999,"5min":1.125,"15min":1.3600000000000001},"last":0.79200000000000004},{"interface":"front","average":{"1min":0.78100000000000003,"5min":0.78900000000000003,"15min":0.81299999999999994},"min":{"1min":0.627,"5min":0.47299999999999998,"15min":0.47299999999999998},"max":{"1min":1.125,"5min":1.2450000000000001,"15min":1.617},"last":0.88300000000000001}]}]},{"osd":2,"up_from":13,"seq":55834574939,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":35508,"kb_used_data":4332,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94336332,"statfs":{"total":96636764160,"available":96600403968,"internally_reserved":0,"allocated":4435968,"data_stored":3708860,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Nov 26 07:23:52 2023","interfaces":[{"interface":"back","average":{"1min":0.42699999999999999,"5min":0.43099999999999999,"15min":0.61899999999999999},"min":{"1min":0.27200000000000002,"5min":0.27100000000000002,"15min":0.27100000000000002},"max":{"1min":0.67500000000000004,"5min":0.76000000000000001,"15min":4.7030000000000003},"last":0.48799999999999999},{"interface":"front","average":{"1min":0.47299999999999998,"5min":0.46600000000000003,"15min":0.53000000000000003},"min":{"1min":0.378,"5min":0.27300000000000002,"15min":0.27300000000000002},"max":{"1min":0.69799999999999995,"5min":0.77200000000000002,"15min":2.4340000000000002},"last":0.28599999999999998}]},{"osd":1,"last update":"Sun Nov 26 07:23:52 2023","interfaces":[{"interface":"back","average":{"1min":0.48199999999999998,"5min":0.49099999999999999,"15min":0.73899999999999999},"min":{"1min":0.35499999999999998,"5min":0.318,"15min":0.318},"max":{"1min":0.621,"5min":0.86099999999999999,"15min":4.2690000000000001},"last":0.40500000000000003},{"interface":"front","average":{"1min":0.504,"5min":0.501,"15min":0.66100000000000003},"min":{"1min":0.39800000000000002,"5min":0.35099999999999998,"15min":0.31900000000000001},"max":{"1min":0.623,"5min":0.76600000000000001,"15min":3.9940000000000002},"last":0.46300000000000002}]},{"osd":3,"last update":"Sun Nov 26 07:23:52 2023","interfaces":[{"interface":"back","average":{"1min":0.48199999999999998,"5min":0.47499999999999998,"15min":0.58699999999999997},"min":{"1min":0.311,"5min":0.29699999999999999,"15min":0.29699999999999999},"max":{"1min":0.64700000000000002,"5min":0.82699999999999996,"15min":4.0300000000000002},"last":0.434},{"interface":"front","average":{"1min":0.55500000000000005,"5min":0.55000000000000004,"15min":0.79200000000000004},"min":{"1min":0.443,"5min":0.36099999999999999,"15min":0.36099999999999999},"max":{"1min":0.79700000000000004,"5min":0.86699999999999999,"15min":4.5810000000000004},"last":0.52200000000000002}]},{"osd":4,"last update":"Sun Nov 26 07:23:52 2023","interfaces":[{"interface":"back","average":{"1min":0.73199999999999998,"5min":0.76600000000000001,"15min":0.91200000000000003},"min":{"1min":0.55600000000000005,"5min":0.51400000000000001,"15min":0.51400000000000001},"max":{"1min":0.97499999999999998,"5min":1.01,"15min":4.4269999999999996},"last":0.56699999999999995},{"interface":"front","average":{"1min":0.752,"5min":0.76800000000000002,"15min":0.93200000000000005},"min":{"1min":0.54300000000000004,"5min":0.54000000000000004,"15min":0.54000000000000004},"max":{"1min":1.0009999999999999,"5min":1.0780000000000001,"15min":4.085},"last":0.53500000000000003}]},{"osd":5,"last update":"Sun Nov 26 07:23:52 2023","interfaces":[{"interface":"back","average":{"1min":0.77300000000000002,"5min":0.76600000000000001,"15min":0.83799999999999997},"min":{"1min":0.42099999999999999,"5min":0.42099999999999999,"15min":0.42099999999999999},"max":{"1min":0.97599999999999998,"5min":1.0580000000000001,"15min":3.9590000000000001},"last":0.55200000000000005},{"interface":"front","average":{"1min":0.71999999999999997,"5min":0.78200000000000003,"15min":0.91900000000000004},"min":{"1min":0.51600000000000001,"5min":0.499,"15min":0.499},"max":{"1min":0.96199999999999997,"5min":1.0900000000000001,"15min":4.3849999999999998},"last":0.60499999999999998}]},{"osd":6,"last update":"Sun Nov 26 07:23:52 2023","interfaces":[{"interface":"back","average":{"1min":0.73699999999999999,"5min":0.76800000000000002,"15min":0.92000000000000004},"min":{"1min":0.45200000000000001,"5min":0.45200000000000001,"15min":0.45200000000000001},"max":{"1min":0.93600000000000005,"5min":1.083,"15min":4.0449999999999999},"last":0.58499999999999996},{"interface":"front","average":{"1min":0.71099999999999997,"5min":0.75900000000000001,"15min":0.86499999999999999},"min":{"1min":0.52700000000000002,"5min":0.48099999999999998,"15min":0.48099999999999998},"max":{"1min":1.0229999999999999,"5min":1.085,"15min":3.992},"last":0.70799999999999996}]},{"osd":7,"last update":"Sun Nov 26 07:23:52 2023","interfaces":[{"interface":"back","average":{"1min":0.76600000000000001,"5min":0.78200000000000003,"15min":0.94999999999999996},"min":{"1min":0.56799999999999995,"5min":0.52200000000000002,"15min":0.52200000000000002},"max":{"1min":1.018,"5min":1.0840000000000001,"15min":4.1219999999999999},"last":0.623},{"interface":"front","average":{"1min":0.747,"5min":0.78200000000000003,"15min":0.88600000000000001},"min":{"1min":0.49299999999999999,"5min":0.48199999999999998,"15min":0.48199999999999998},"max":{"1min":0.95399999999999996,"5min":1.036,"15min":3.9159999999999999},"last":0.73499999999999999}]}]},{"osd":1,"up_from":13,"seq":55834574938,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":36552,"kb_used_data":5372,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94335288,"statfs":{"total":96636764160,"available":96599334912,"internally_reserved":0,"allocated":5500928,"data_stored":4763443,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":31914439},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Nov 26 07:24:51 2023","interfaces":[{"interface":"back","average":{"1min":0.38400000000000001,"5min":0.40500000000000003,"15min":0.41099999999999998},"min":{"1min":0.26600000000000001,"5min":0.26600000000000001,"15min":0.23599999999999999},"max":{"1min":0.58399999999999996,"5min":0.58399999999999996,"15min":0.61199999999999999},"last":0.42799999999999999},{"interface":"front","average":{"1min":0.41199999999999998,"5min":0.44400000000000001,"15min":0.46899999999999997},"min":{"1min":0.28000000000000003,"5min":0.28000000000000003,"15min":0.26700000000000002},"max":{"1min":0.52100000000000002,"5min":0.749,"15min":0.78700000000000003},"last":0.52100000000000002}]},{"osd":2,"last update":"Sun Nov 26 07:24:51 2023","interfaces":[{"interface":"back","average":{"1min":0.42099999999999999,"5min":0.436,"15min":0.44600000000000001},"min":{"1min":0.30499999999999999,"5min":0.29899999999999999,"15min":0.27800000000000002},"max":{"1min":0.63400000000000001,"5min":0.71399999999999997,"15min":0.71399999999999997},"last":0.34399999999999997},{"interface":"front","average":{"1min":0.46200000000000002,"5min":0.47599999999999998,"15min":0.495},"min":{"1min":0.33100000000000002,"5min":0.318,"15min":0.27700000000000002},"max":{"1min":0.61799999999999999,"5min":0.80600000000000005,"15min":0.80600000000000005},"last":0.38}]},{"osd":3,"last update":"Sun Nov 26 07:24:51 2023","interfaces":[{"interface":"back","average":{"1min":0.48499999999999999,"5min":0.52100000000000002,"15min":0.52500000000000002},"min":{"1min":0.33200000000000002,"5min":0.33200000000000002,"15min":0.33200000000000002},"max":{"1min":0.72699999999999998,"5min":0.748,"15min":0.748},"last":0.46999999999999997},{"interface":"front","average":{"1min":0.47499999999999998,"5min":0.498,"15min":0.496},"min":{"1min":0.38100000000000001,"5min":0.35499999999999998,"15min":0.31},"max":{"1min":0.71299999999999997,"5min":0.71299999999999997,"15min":0.80700000000000005},"last":0.41099999999999998}]},{"osd":4,"last update":"Sun Nov 26 07:24:51 2023","interfaces":[{"interface":"back","average":{"1min":0.72999999999999998,"5min":0.75800000000000001,"15min":0.71499999999999997},"min":{"1min":0.50600000000000001,"5min":0.50600000000000001,"15min":0.41299999999999998},"max":{"1min":0.95699999999999996,"5min":1.319,"15min":1.319},"last":0.91300000000000003},{"interface":"front","average":{"1min":0.71899999999999997,"5min":0.72499999999999998,"15min":0.69799999999999995},"min":{"1min":0.54300000000000004,"5min":0.44500000000000001,"15min":0.44500000000000001},"max":{"1min":0.97999999999999998,"5min":1.006,"15min":1.131},"last":0.93999999999999995}]},{"osd":5,"last update":"Sun Nov 26 07:24:51 2023","interfaces":[{"interface":"back","average":{"1min":0.73299999999999998,"5min":0.747,"15min":0.72899999999999998},"min":{"1min":0.501,"5min":0.47899999999999998,"15min":0.47899999999999998},"max":{"1min":0.95699999999999996,"5min":1.0409999999999999,"15min":1.1919999999999999},"last":0.75700000000000001},{"interface":"front","average":{"1min":0.73499999999999999,"5min":0.75,"15min":0.72899999999999998},"min":{"1min":0.53100000000000003,"5min":0.47099999999999997,"15min":0.47099999999999997},"max":{"1min":0.86299999999999999,"5min":1.0389999999999999,"15min":1.0609999999999999},"last":0.82699999999999996}]},{"osd":6,"last update":"Sun Nov 26 07:24:51 2023","interfaces":[{"interface":"back","average":{"1min":0.72499999999999998,"5min":0.76000000000000001,"15min":0.72399999999999998},"min":{"1min":0.53800000000000003,"5min":0.53200000000000003,"15min":0.50600000000000001},"max":{"1min":0.89600000000000002,"5min":1.2190000000000001,"15min":1.2190000000000001},"last":0.80600000000000005},{"interface":"front","average":{"1min":0.72199999999999998,"5min":0.76200000000000001,"15min":0.72499999999999998},"min":{"1min":0.52200000000000002,"5min":0.46999999999999997,"15min":0.46999999999999997},"max":{"1min":0.91800000000000004,"5min":1.1659999999999999,"15min":1.1659999999999999},"last":0.73599999999999999}]},{"osd":7,"last update":"Sun Nov 26 07:24:51 2023","interfaces":[{"interface":"back","average":{"1min":0.76700000000000002,"5min":0.77400000000000002,"15min":0.73599999999999999},"min":{"1min":0.57099999999999995,"5min":0.50600000000000001,"15min":0.50600000000000001},"max":{"1min":0.97299999999999998,"5min":1.0920000000000001,"15min":1.0920000000000001},"last":0.88600000000000001},{"interface":"front","average":{"1min":0.76200000000000001,"5min":0.76500000000000001,"15min":0.755},"min":{"1min":0.54000000000000004,"5min":0.53700000000000003,"15min":0.48499999999999999},"max":{"1min":1.006,"5min":1.115,"15min":1.115},"last":0.77000000000000002}]}]},{"osd":0,"up_from":13,"seq":55834574938,"num_pgs":36,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":40628,"kb_used_data":9448,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94331212,"statfs":{"total":96636764160,"available":96595161088,"internally_reserved":0,"allocated":9674752,"data_stored":8929116,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Sun Nov 26 07:24:50 2023","interfaces":[{"interface":"back","average":{"1min":0.439,"5min":0.437,"15min":0.43099999999999999},"min":{"1min":0.25800000000000001,"5min":0.25800000000000001,"15min":0.25800000000000001},"max":{"1min":0.70499999999999996,"5min":1.26,"15min":1.26},"last":0.434},{"interface":"front","average":{"1min":0.45100000000000001,"5min":0.47299999999999998,"15min":0.47499999999999998},"min":{"1min":0.29899999999999999,"5min":0.29899999999999999,"15min":0.29899999999999999},"max":{"1min":0.72099999999999997,"5min":1.212,"15min":1.212},"last":0.33400000000000002}]},{"osd":2,"last update":"Sun Nov 26 07:24:50 2023","interfaces":[{"interface":"back","average":{"1min":0.47199999999999998,"5min":0.497,"15min":0.52200000000000002},"min":{"1min":0.33000000000000002,"5min":0.33000000000000002,"15min":0.33000000000000002},"max":{"1min":0.68100000000000005,"5min":0.84199999999999997,"15min":0.84199999999999997},"last":0.50800000000000001},{"interface":"front","average":{"1min":0.53900000000000003,"5min":0.55800000000000005,"15min":0.55900000000000005},"min":{"1min":0.45200000000000001,"5min":0.40100000000000002,"15min":0.36399999999999999},"max":{"1min":0.81000000000000005,"5min":1.0800000000000001,"15min":1.0800000000000001},"last":0.58899999999999997}]},{"osd":3,"last update":"Sun Nov 26 07:24:50 2023","interfaces":[{"interface":"back","average":{"1min":0.51600000000000001,"5min":0.51300000000000001,"15min":0.52400000000000002},"min":{"1min":0.41099999999999998,"5min":0.33500000000000002,"15min":0.33500000000000002},"max":{"1min":0.72899999999999998,"5min":0.873,"15min":1.3149999999999999},"last":0.47399999999999998},{"interface":"front","average":{"1min":0.53000000000000003,"5min":0.53600000000000003,"15min":0.54900000000000004},"min":{"1min":0.377,"5min":0.35499999999999998,"15min":0.35499999999999998},"max":{"1min":0.74299999999999999,"5min":0.74299999999999999,"15min":1.2509999999999999},"last":0.41499999999999998}]},{"osd":4,"last update":"Sun Nov 26 07:24:50 2023","interfaces":[{"interface":"back","average":{"1min":0.68500000000000005,"5min":0.73899999999999999,"15min":0.749},"min":{"1min":0.40100000000000002,"5min":0.40100000000000002,"15min":0.40100000000000002},"max":{"1min":0.91300000000000003,"5min":1.0900000000000001,"15min":1.2010000000000001},"last":0.45400000000000001},{"interface":"front","average":{"1min":0.76000000000000001,"5min":0.78000000000000003,"15min":0.78600000000000003},"min":{"1min":0.55900000000000005,"5min":0.51100000000000001,"15min":0.51100000000000001},"max":{"1min":1.012,"5min":1.298,"15min":1.381},"last":0.65200000000000002}]},{"osd":5,"last update":"Sun Nov 26 07:24:50 2023","interfaces":[{"interface":"back","average":{"1min":0.66800000000000004,"5min":0.73599999999999999,"15min":0.75},"min":{"1min":0.50800000000000001,"5min":0.45200000000000001,"15min":0.42199999999999999},"max":{"1min":0.91600000000000004,"5min":1.1819999999999999,"15min":1.1819999999999999},"last":0.52400000000000002},{"interface":"front","average":{"1min":0.77500000000000002,"5min":0.79800000000000004,"15min":0.79700000000000004},"min":{"1min":0.59799999999999998,"5min":0.53000000000000003,"15min":0.53000000000000003},"max":{"1min":1.0609999999999999,"5min":1.329,"15min":1.399},"last":0.67100000000000004}]},{"osd":6,"last update":"Sun Nov 26 07:24:50 2023","interfaces":[{"interface":"back","average":{"1min":0.70599999999999996,"5min":0.70599999999999996,"15min":0.73699999999999999},"min":{"1min":0.48999999999999999,"5min":0.48499999999999999,"15min":0.48499999999999999},"max":{"1min":0.97199999999999998,"5min":1.0740000000000001,"15min":1.3029999999999999},"last":0.48999999999999999},{"interface":"front","average":{"1min":0.71099999999999997,"5min":0.72899999999999998,"15min":0.754},"min":{"1min":0.52100000000000002,"5min":0.47199999999999998,"15min":0.47199999999999998},"max":{"1min":0.93600000000000005,"5min":1.2170000000000001,"15min":1.3600000000000001},"last":0.622}]},{"osd":7,"last update":"Sun Nov 26 07:24:50 2023","interfaces":[{"interface":"back","average":{"1min":0.75600000000000001,"5min":0.76800000000000002,"15min":0.79300000000000004},"min":{"1min":0.60099999999999998,"5min":0.56899999999999995,"15min":0.56899999999999995},"max":{"1min":0.97999999999999998,"5min":1.1719999999999999,"15min":1.177},"last":0.63900000000000001},{"interface":"front","average":{"1min":0.69899999999999995,"5min":0.76500000000000001,"15min":0.77800000000000002},"min":{"1min":0.56899999999999995,"5min":0.47899999999999998,"15min":0.47899999999999998},"max":{"1min":0.92400000000000004,"5min":1.238,"15min":1.2390000000000001},"last":0.60899999999999999}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-11-26T07:24:57.093 DEBUG:teuthology.orchestra.run.smithi143:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-11-26T07:24:57.231 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:57.230+0000 7fa7009df640 1 Processor -- start 2023-11-26T07:24:57.232 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:57.230+0000 7fa7009df640 1 -- start start 2023-11-26T07:24:57.235 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:57.230+0000 7fa7009df640 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7fa6fc0cf280 0x7fa6fc0cf710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:24:57.235 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:57.230+0000 7fa7009df640 1 -- --> v1:172.21.15.143:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa6fc0cc730 con 0x7fa6fc143ac0 2023-11-26T07:24:57.235 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:57.230+0000 7fa7009df640 1 -- --> v1:172.21.15.188:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa6fc070ee0 con 0x7fa6fc0cd1b0 2023-11-26T07:24:57.236 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:57.230+0000 7fa7009df640 1 -- --> v2:172.21.15.188:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa6fc143ea0 con 0x7fa6fc0cf280 2023-11-26T07:24:57.236 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:57.234+0000 7fa6fa59c640 1 --1- >> v1:172.21.15.143:6789/0 conn(0x7fa6fc143ac0 0x7fa6fc070370 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.143:6789/0 says I am v1:172.21.15.143:36278/0 (socket says 172.21.15.143:36278) 2023-11-26T07:24:57.236 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:57.234+0000 7fa6fa59c640 1 -- 172.21.15.143:0/2943351725 learned_addr learned my addr 172.21.15.143:0/2943351725 (peer_addr_for_me v1:172.21.15.143:0/0) 2023-11-26T07:24:57.236 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:57.234+0000 7fa6f9d9b640 1 -- 172.21.15.143:0/2943351725 <== mon.0 v1:172.21.15.143:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 981105225 0 0) 0x7fa6fc0cc730 con 0x7fa6fc143ac0 2023-11-26T07:24:57.236 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:57.234+0000 7fa6f9d9b640 1 -- 172.21.15.143:0/2943351725 --> v1:172.21.15.143:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fa6e8003530 con 0x7fa6fc143ac0 2023-11-26T07:24:57.236 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:57.234+0000 7fa6f3fff640 1 --2- 172.21.15.143:0/2943351725 >> v2:172.21.15.188:3300/0 conn(0x7fa6fc0cf280 0x7fa6fc0cf710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:24:57.236 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:57.234+0000 7fa6f9d9b640 1 -- 172.21.15.143:0/2943351725 <== mon.0 v1:172.21.15.143:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3836714377 0 0) 0x7fa6e8003530 con 0x7fa6fc143ac0 2023-11-26T07:24:57.236 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:57.234+0000 7fa6f9d9b640 1 -- 172.21.15.143:0/2943351725 >> v2:172.21.15.188:3300/0 conn(0x7fa6fc0cf280 msgr2=0x7fa6fc0cf710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-26T07:24:57.237 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:57.234+0000 7fa6f9d9b640 1 --2- 172.21.15.143:0/2943351725 >> v2:172.21.15.188:3300/0 conn(0x7fa6fc0cf280 0x7fa6fc0cf710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:24:57.237 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:57.234+0000 7fa6f9d9b640 1 -- 172.21.15.143:0/2943351725 >> v1:172.21.15.188:6789/0 conn(0x7fa6fc0cd1b0 legacy=0x7fa6fc0cd5c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-26T07:24:57.237 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:57.234+0000 7fa6f9d9b640 1 -- 172.21.15.143:0/2943351725 --> v1:172.21.15.143:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa6fc0711c0 con 0x7fa6fc143ac0 2023-11-26T07:24:57.237 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:57.234+0000 7fa6f9d9b640 1 -- 172.21.15.143:0/2943351725 <== mon.0 v1:172.21.15.143:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2137882523 0 0) 0x7fa6ec001680 con 0x7fa6fc143ac0 2023-11-26T07:24:57.237 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:57.234+0000 7fa6f9d9b640 1 -- 172.21.15.143:0/2943351725 >> v1:172.21.15.143:6789/0 conn(0x7fa6fc143ac0 legacy=0x7fa6fc070370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:24:57.237 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:57.234+0000 7fa6f9d9b640 1 --2- 172.21.15.143:0/2943351725 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fa6e8003980 0x7fa6e8003d80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:24:57.237 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:57.234+0000 7fa6f9d9b640 1 --2- 172.21.15.143:0/2943351725 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fa6e80047d0 0x7fa6e8004c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:24:57.238 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:57.234+0000 7fa6f9d9b640 1 --2- 172.21.15.143:0/2943351725 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fa6e8005190 0x7fa6e800b990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:24:57.238 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:57.234+0000 7fa6f9d9b640 1 -- 172.21.15.143:0/2943351725 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa6e800c090 con 0x7fa6e8005190 2023-11-26T07:24:57.238 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:57.234+0000 7fa6f9d9b640 1 -- 172.21.15.143:0/2943351725 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa6e800c200 con 0x7fa6e80047d0 2023-11-26T07:24:57.238 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:57.234+0000 7fa6f9d9b640 1 -- 172.21.15.143:0/2943351725 --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa6e800e5d0 con 0x7fa6e8003980 2023-11-26T07:24:57.238 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:57.234+0000 7fa6fad9d640 1 --2- 172.21.15.143:0/2943351725 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fa6e8005190 0x7fa6e800b990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:24:57.238 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:57.234+0000 7fa6f3fff640 1 --2- 172.21.15.143:0/2943351725 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fa6e80047d0 0x7fa6e8004c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:24:57.238 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:57.234+0000 7fa6fa59c640 1 --2- 172.21.15.143:0/2943351725 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fa6e8003980 0x7fa6e8003d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:24:57.239 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:57.234+0000 7fa6fad9d640 1 -- 172.21.15.143:0/2943351725 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fa6e8003980 msgr2=0x7fa6e8003d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-26T07:24:57.239 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:57.234+0000 7fa6fad9d640 1 --2- 172.21.15.143:0/2943351725 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fa6e8003980 0x7fa6e8003d80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:24:57.239 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:57.234+0000 7fa6fad9d640 1 -- 172.21.15.143:0/2943351725 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fa6e80047d0 msgr2=0x7fa6e8004c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-26T07:24:57.239 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:57.234+0000 7fa6fad9d640 1 --2- 172.21.15.143:0/2943351725 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fa6e80047d0 0x7fa6e8004c50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:24:57.239 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:57.234+0000 7fa6fad9d640 1 -- 172.21.15.143:0/2943351725 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa6e8003530 con 0x7fa6e8005190 2023-11-26T07:24:57.240 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:57.238+0000 7fa6fad9d640 1 -- 172.21.15.143:0/2943351725 reap_dead start 2023-11-26T07:24:57.241 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:57.238+0000 7fa6fad9d640 1 --2- 172.21.15.143:0/2943351725 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fa6e8005190 0x7fa6e800b990 secure :-1 s=READY pgs=1448 cs=0 l=1 rev1=1 crypto rx=0x7fa6f4007fd0 tx=0x7fa6f401c660 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e268c83510c38ae server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:24:57.241 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:57.238+0000 7fa6f9d9b640 1 -- 172.21.15.143:0/2943351725 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa6f40a9660 con 0x7fa6e8005190 2023-11-26T07:24:57.241 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:57.238+0000 7fa6f9d9b640 1 -- 172.21.15.143:0/2943351725 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa6f40a97e0 con 0x7fa6e8005190 2023-11-26T07:24:57.241 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:57.238+0000 7fa6f9d9b640 1 -- 172.21.15.143:0/2943351725 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa6f40a9b90 con 0x7fa6e8005190 2023-11-26T07:24:57.241 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:57.238+0000 7fa7009df640 1 -- 172.21.15.143:0/2943351725 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fa6e8005190 msgr2=0x7fa6e800b990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:24:57.241 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:57.238+0000 7fa7009df640 1 --2- 172.21.15.143:0/2943351725 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fa6e8005190 0x7fa6e800b990 secure :-1 s=READY pgs=1448 cs=0 l=1 rev1=1 crypto rx=0x7fa6f4007fd0 tx=0x7fa6f401c660 comp rx=0 tx=0).stop 2023-11-26T07:24:57.242 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:57.238+0000 7fa7009df640 1 -- 172.21.15.143:0/2943351725 shutdown_connections 2023-11-26T07:24:57.242 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:57.238+0000 7fa7009df640 1 --2- 172.21.15.143:0/2943351725 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fa6e8005190 0x7fa6e800b990 unknown :-1 s=CLOSED pgs=1448 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:24:57.242 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:57.238+0000 7fa7009df640 1 -- 172.21.15.143:0/2943351725 >> 172.21.15.143:0/2943351725 conn(0x7fa6fc06a890 msgr2=0x7fa6fc06aca0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:24:57.242 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:57.238+0000 7fa7009df640 1 -- 172.21.15.143:0/2943351725 shutdown_connections 2023-11-26T07:24:57.242 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:57.238+0000 7fa7009df640 1 -- 172.21.15.143:0/2943351725 wait complete. 2023-11-26T07:24:57.242 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:57.238+0000 7fa7009df640 1 Processor -- start 2023-11-26T07:24:57.242 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:57.238+0000 7fa7009df640 1 -- start start 2023-11-26T07:24:57.245 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:57.242+0000 7fa7009df640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fa6e8005190 0x7fa6fc1396f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:24:57.245 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:57.242+0000 7fa7009df640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fa6fc0ad2b0 0x7fa6fc0b12f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:24:57.245 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:57.242+0000 7fa7009df640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fa6fc0c4d10 0x7fa6fc0c5190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:24:57.245 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:57.242+0000 7fa7009df640 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa6fc0c56d0 con 0x7fa6fc0c4d10 2023-11-26T07:24:57.246 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:57.242+0000 7fa7009df640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa6fc139c30 con 0x7fa6e8005190 2023-11-26T07:24:57.246 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:57.242+0000 7fa7009df640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa6fc1f2d40 con 0x7fa6fc0ad2b0 2023-11-26T07:24:57.246 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:57.242+0000 7fa6fad9d640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fa6fc0c4d10 0x7fa6fc0c5190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:24:57.246 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:57.242+0000 7fa6fad9d640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fa6fc0c4d10 0x7fa6fc0c5190 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:45876/0 (socket says 172.21.15.143:45876) 2023-11-26T07:24:57.246 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:57.242+0000 7fa6fad9d640 1 -- 172.21.15.143:0/456758135 learned_addr learned my addr 172.21.15.143:0/456758135 (peer_addr_for_me v2:172.21.15.143:0/0) 2023-11-26T07:24:57.246 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:57.242+0000 7fa6fa59c640 1 --2- 172.21.15.143:0/456758135 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fa6e8005190 0x7fa6fc1396f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:24:57.247 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:57.242+0000 7fa6f3fff640 1 --2- 172.21.15.143:0/456758135 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fa6fc0ad2b0 0x7fa6fc0b12f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:24:57.247 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:57.242+0000 7fa6fad9d640 1 -- 172.21.15.143:0/456758135 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fa6fc0ad2b0 msgr2=0x7fa6fc0b12f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:24:57.247 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:57.242+0000 7fa6fad9d640 1 --2- 172.21.15.143:0/456758135 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fa6fc0ad2b0 0x7fa6fc0b12f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:24:57.247 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:57.242+0000 7fa6fad9d640 1 -- 172.21.15.143:0/456758135 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fa6e8005190 msgr2=0x7fa6fc1396f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:24:57.247 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:57.242+0000 7fa6fad9d640 1 --2- 172.21.15.143:0/456758135 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fa6e8005190 0x7fa6fc1396f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:24:57.247 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:57.242+0000 7fa6fad9d640 1 -- 172.21.15.143:0/456758135 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa6fc0ad7f0 con 0x7fa6fc0c4d10 2023-11-26T07:24:57.248 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:57.242+0000 7fa6fad9d640 1 --2- 172.21.15.143:0/456758135 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fa6fc0c4d10 0x7fa6fc0c5190 secure :-1 s=READY pgs=1449 cs=0 l=1 rev1=1 crypto rx=0x7fa6f401cf00 tx=0x7fa6e8005800 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:24:57.248 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:57.242+0000 7fa6f2ffd640 1 -- 172.21.15.143:0/456758135 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa6f40a9d60 con 0x7fa6fc0c4d10 2023-11-26T07:24:57.248 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:57.242+0000 7fa6f2ffd640 1 -- 172.21.15.143:0/456758135 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa6f40b4470 con 0x7fa6fc0c4d10 2023-11-26T07:24:57.248 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:57.242+0000 7fa7009df640 1 -- 172.21.15.143:0/456758135 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa6fc0adad0 con 0x7fa6fc0c4d10 2023-11-26T07:24:57.248 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:57.242+0000 7fa6f2ffd640 1 -- 172.21.15.143:0/456758135 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa6f40b47e0 con 0x7fa6fc0c4d10 2023-11-26T07:24:57.248 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:57.242+0000 7fa7009df640 1 -- 172.21.15.143:0/456758135 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa6fc0ae090 con 0x7fa6fc0c4d10 2023-11-26T07:24:57.249 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:57.242+0000 7fa7009df640 1 -- 172.21.15.143:0/456758135 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa6fc0ae280 con 0x7fa6fc0c4d10 2023-11-26T07:24:57.249 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:57.242+0000 7fa6f2ffd640 1 -- 172.21.15.143:0/456758135 <== mon.0 v2:172.21.15.143:3300/0 4 ==== mgrmap(e 18) v1 ==== 86107+0+0 (secure 0 0 0) 0x7fa6f40b4940 con 0x7fa6fc0c4d10 2023-11-26T07:24:57.250 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:57.246+0000 7fa6f2ffd640 1 --2- 172.21.15.143:0/456758135 >> [v2:172.21.15.188:6836/3401128553,v1:172.21.15.188:6837/3401128553] conn(0x7fa6dc066770 0x7fa6dc068c30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:24:57.250 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:57.246+0000 7fa6f2ffd640 1 -- 172.21.15.143:0/456758135 <== mon.0 v2:172.21.15.143:3300/0 5 ==== osd_map(106..106 src has 1..106) v4 ==== 4943+0+0 (secure 0 0 0) 0x7fa6f411ed10 con 0x7fa6fc0c4d10 2023-11-26T07:24:57.250 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:57.246+0000 7fa6fa59c640 1 --2- 172.21.15.143:0/456758135 >> [v2:172.21.15.188:6836/3401128553,v1:172.21.15.188:6837/3401128553] conn(0x7fa6dc066770 0x7fa6dc068c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:24:57.250 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:57.246+0000 7fa6fa59c640 1 --2- 172.21.15.143:0/456758135 >> [v2:172.21.15.188:6836/3401128553,v1:172.21.15.188:6837/3401128553] conn(0x7fa6dc066770 0x7fa6dc068c30 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7fa6ec000c10 tx=0x7fa6ec001c50 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:24:57.250 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:57.250+0000 7fa6f2ffd640 1 -- 172.21.15.143:0/456758135 <== mon.0 v2:172.21.15.143:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172649 (secure 0 0 0) 0x7fa6f40ec020 con 0x7fa6fc0c4d10 2023-11-26T07:24:57.397 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:57.394+0000 7fa7009df640 1 -- 172.21.15.143:0/456758135 --> [v2:172.21.15.188:6836/3401128553,v1:172.21.15.188:6837/3401128553] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7fa6fc071c30 con 0x7fa6dc066770 2023-11-26T07:24:57.400 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:57.398+0000 7fa6f2ffd640 1 -- 172.21.15.143:0/456758135 <== mgr.4108 v2:172.21.15.188:6836/3401128553 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+50820 (secure 0 0 0) 0x7fa6ec0015e0 con 0x7fa6dc066770 2023-11-26T07:24:57.400 INFO:teuthology.orchestra.run.smithi143.stdout: 2023-11-26T07:24:57.401 INFO:teuthology.orchestra.run.smithi143.stderr:dumped all 2023-11-26T07:24:57.402 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:57.402+0000 7fa7009df640 1 -- 172.21.15.143:0/456758135 >> [v2:172.21.15.188:6836/3401128553,v1:172.21.15.188:6837/3401128553] conn(0x7fa6dc066770 msgr2=0x7fa6dc068c30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:24:57.402 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:57.402+0000 7fa7009df640 1 --2- 172.21.15.143:0/456758135 >> [v2:172.21.15.188:6836/3401128553,v1:172.21.15.188:6837/3401128553] conn(0x7fa6dc066770 0x7fa6dc068c30 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7fa6ec000c10 tx=0x7fa6ec001c50 comp rx=0 tx=0).stop 2023-11-26T07:24:57.402 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:57.402+0000 7fa7009df640 1 -- 172.21.15.143:0/456758135 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fa6fc0c4d10 msgr2=0x7fa6fc0c5190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:24:57.402 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:57.402+0000 7fa7009df640 1 --2- 172.21.15.143:0/456758135 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fa6fc0c4d10 0x7fa6fc0c5190 secure :-1 s=READY pgs=1449 cs=0 l=1 rev1=1 crypto rx=0x7fa6f401cf00 tx=0x7fa6e8005800 comp rx=0 tx=0).stop 2023-11-26T07:24:57.402 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:57.402+0000 7fa7009df640 1 -- 172.21.15.143:0/456758135 shutdown_connections 2023-11-26T07:24:57.403 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:57.402+0000 7fa7009df640 1 --2- 172.21.15.143:0/456758135 >> [v2:172.21.15.188:6836/3401128553,v1:172.21.15.188:6837/3401128553] conn(0x7fa6dc066770 0x7fa6dc068c30 unknown :-1 s=CLOSED pgs=184 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:24:57.403 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:57.402+0000 7fa7009df640 1 --2- 172.21.15.143:0/456758135 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fa6fc0c4d10 0x7fa6fc0c5190 unknown :-1 s=CLOSED pgs=1449 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:24:57.403 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:57.402+0000 7fa7009df640 1 --2- 172.21.15.143:0/456758135 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fa6fc0ad2b0 0x7fa6fc0b12f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:24:57.403 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:57.402+0000 7fa7009df640 1 --2- 172.21.15.143:0/456758135 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fa6e8005190 0x7fa6fc1396f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:24:57.403 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:57.402+0000 7fa7009df640 1 -- 172.21.15.143:0/456758135 >> 172.21.15.143:0/456758135 conn(0x7fa6fc06a890 msgr2=0x7fa6fc06ac70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:24:57.403 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:57.402+0000 7fa7009df640 1 -- 172.21.15.143:0/456758135 shutdown_connections 2023-11-26T07:24:57.403 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:57.402+0000 7fa7009df640 1 -- 172.21.15.143:0/456758135 wait complete. 2023-11-26T07:24:57.421 INFO:teuthology.orchestra.run.smithi143.stdout:{"pg_ready":true,"pg_map":{"version":91,"stamp":"2023-11-26T07:24:57.310516+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":212,"num_read_kb":306,"num_write":179,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":128,"ondisk_log_size":128,"up":2,"acting":2,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":69,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":2,"kb":754974720,"kb_used":260972,"kb_used_data":11528,"kb_used_omap":12,"kb_used_meta":249331,"kb_avail":754713748,"statfs":{"total":773094113280,"available":772826877952,"internally_reserved":0,"allocated":11804672,"data_stored":5820552,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12749,"internal_metadata":255315507},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":31,"apply_latency_ms":31,"commit_latency_ns":31000000,"apply_latency_ns":31000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-17207712,"num_objects":-42,"num_object_clones":0,"num_object_copies":-84,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":-42,"num_whiteouts":0,"num_read":-23,"num_read_kb":-8207,"num_write":-184,"num_write_kb":-16883,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":-14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"6.001598"},"pg_stats":[{"pgid":"1.0","version":"72'128","reported_seq":275,"reported_epoch":104,"state":"active+clean","last_fresh":"2023-11-26T07:24:53.071827+0000","last_change":"2023-11-26T07:17:43.827802+0000","last_active":"2023-11-26T07:24:53.071827+0000","last_peered":"2023-11-26T07:24:53.071827+0000","last_clean":"2023-11-26T07:24:53.071827+0000","last_became_active":"2023-11-26T07:17:43.827308+0000","last_became_peered":"2023-11-26T07:17:43.827308+0000","last_unstale":"2023-11-26T07:24:53.071827+0000","last_undegraded":"2023-11-26T07:24:53.071827+0000","last_fullsized":"2023-11-26T07:24:53.071827+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:17:42.160945+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:17:42.160945+0000","last_clean_scrub_stamp":"2023-11-26T07:17:42.160945+0000","objects_scrubbed":0,"log_size":128,"log_dups_size":0,"ondisk_log_size":128,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T08:17:54.167366+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":212,"num_read_kb":306,"num_write":179,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":212,"num_read_kb":306,"num_write":179,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":128,"ondisk_log_size":128,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574937,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":33056,"kb_used_data":1876,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94338784,"statfs":{"total":96636764160,"available":96602914816,"internally_reserved":0,"allocated":1921024,"data_stored":1170345,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":31914442},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Nov 26 07:23:58 2023","interfaces":[{"interface":"back","average":{"1min":0.68200000000000005,"5min":0.70299999999999996,"15min":0.71099999999999997},"min":{"1min":0.47299999999999998,"5min":0.42999999999999999,"15min":0.42999999999999999},"max":{"1min":1.036,"5min":1.036,"15min":1.036},"last":0.53700000000000003},{"interface":"front","average":{"1min":0.70199999999999996,"5min":0.72699999999999998,"15min":0.71499999999999997},"min":{"1min":0.42299999999999999,"5min":0.42299999999999999,"15min":0.42299999999999999},"max":{"1min":1.014,"5min":1.014,"15min":1.0980000000000001},"last":0.55600000000000005}]},{"osd":1,"last update":"Sun Nov 26 07:23:58 2023","interfaces":[{"interface":"back","average":{"1min":0.68400000000000005,"5min":0.68700000000000006,"15min":0.68300000000000005},"min":{"1min":0.45100000000000001,"5min":0.435,"15min":0.435},"max":{"1min":0.85299999999999998,"5min":1.012,"15min":1.0669999999999999},"last":0.66500000000000004},{"interface":"front","average":{"1min":0.66900000000000004,"5min":0.67700000000000005,"15min":0.78500000000000003},"min":{"1min":0.432,"5min":0.432,"15min":0.432},"max":{"1min":0.94699999999999995,"5min":0.94699999999999995,"15min":4.5309999999999997},"last":0.44600000000000001}]},{"osd":2,"last update":"Sun Nov 26 07:23:58 2023","interfaces":[{"interface":"back","average":{"1min":0.69999999999999996,"5min":0.69999999999999996,"15min":0.68400000000000005},"min":{"1min":0.56999999999999995,"5min":0.39700000000000002,"15min":0.39700000000000002},"max":{"1min":0.91600000000000004,"5min":1.0800000000000001,"15min":1.0800000000000001},"last":0.60599999999999998},{"interface":"front","average":{"1min":0.68700000000000006,"5min":0.70499999999999996,"15min":0.70999999999999996},"min":{"1min":0.48999999999999999,"5min":0.40999999999999998,"15min":0.40999999999999998},"max":{"1min":0.97099999999999997,"5min":0.97699999999999998,"15min":1.1339999999999999},"last":0.47299999999999998}]},{"osd":3,"last update":"Sun Nov 26 07:23:58 2023","interfaces":[{"interface":"back","average":{"1min":0.73299999999999998,"5min":0.72799999999999998,"15min":0.70699999999999996},"min":{"1min":0.53200000000000003,"5min":0.47799999999999998,"15min":0.47799999999999998},"max":{"1min":0.99299999999999999,"5min":1.1240000000000001,"15min":1.1240000000000001},"last":0.57499999999999996},{"interface":"front","average":{"1min":0.72399999999999998,"5min":0.74299999999999999,"15min":0.73599999999999999},"min":{"1min":0.50700000000000001,"5min":0.433,"15min":0.433},"max":{"1min":0.93400000000000005,"5min":1.036,"15min":1.4490000000000001},"last":0.68799999999999994}]},{"osd":4,"last update":"Sun Nov 26 07:23:58 2023","interfaces":[{"interface":"back","average":{"1min":0.61099999999999999,"5min":0.61499999999999999,"15min":0.60499999999999998},"min":{"1min":0.48299999999999998,"5min":0.39400000000000002,"15min":0.39400000000000002},"max":{"1min":0.878,"5min":0.96299999999999997,"15min":0.96299999999999997},"last":0.59299999999999997},{"interface":"front","average":{"1min":0.65500000000000003,"5min":0.64400000000000002,"15min":0.66300000000000003},"min":{"1min":0.45800000000000002,"5min":0.35999999999999999,"15min":0.35999999999999999},"max":{"1min":0.83599999999999997,"5min":1.0209999999999999,"15min":1.5509999999999999},"last":0.64800000000000002}]},{"osd":5,"last update":"Sun Nov 26 07:23:58 2023","interfaces":[{"interface":"back","average":{"1min":0.66100000000000003,"5min":0.67400000000000004,"15min":0.66800000000000004},"min":{"1min":0.36599999999999999,"5min":0.36599999999999999,"15min":0.36599999999999999},"max":{"1min":0.86799999999999999,"5min":1.056,"15min":1.415},"last":0.628},{"interface":"front","average":{"1min":0.68300000000000005,"5min":0.68200000000000005,"15min":0.70299999999999996},"min":{"1min":0.435,"5min":0.435,"15min":0.435},"max":{"1min":0.873,"5min":1.0840000000000001,"15min":1.3859999999999999},"last":0.497}]},{"osd":6,"last update":"Sun Nov 26 07:23:58 2023","interfaces":[{"interface":"back","average":{"1min":0.77300000000000002,"5min":0.78000000000000003,"15min":0.77400000000000002},"min":{"1min":0.53100000000000003,"5min":0.51700000000000002,"15min":0.51700000000000002},"max":{"1min":0.95499999999999996,"5min":1.123,"15min":1.3069999999999999},"last":0.61599999999999999},{"interface":"front","average":{"1min":0.71399999999999997,"5min":0.74399999999999999,"15min":0.76200000000000001},"min":{"1min":0.50600000000000001,"5min":0.46899999999999997,"15min":0.46899999999999997},"max":{"1min":0.92900000000000005,"5min":1.036,"15min":1.6759999999999999},"last":0.51600000000000001}]}]},{"osd":6,"up_from":13,"seq":55834574939,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":32476,"kb_used_data":1296,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339364,"statfs":{"total":96636764160,"available":96603508736,"internally_reserved":0,"allocated":1327104,"data_stored":579977,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Nov 26 07:23:56 2023","interfaces":[{"interface":"back","average":{"1min":0.65400000000000003,"5min":0.67800000000000005,"15min":0.68200000000000005},"min":{"1min":0.48999999999999999,"5min":0.32200000000000001,"15min":0.32200000000000001},"max":{"1min":0.875,"5min":0.91800000000000004,"15min":1.7549999999999999},"last":0.78800000000000003},{"interface":"front","average":{"1min":0.65500000000000003,"5min":0.69599999999999995,"15min":0.87},"min":{"1min":0.36299999999999999,"5min":0.36299999999999999,"15min":0.36299999999999999},"max":{"1min":0.82599999999999996,"5min":0.96199999999999997,"15min":5.7489999999999997},"last":0.75}]},{"osd":1,"last update":"Sun Nov 26 07:23:56 2023","interfaces":[{"interface":"back","average":{"1min":0.65000000000000002,"5min":0.70799999999999996,"15min":0.84499999999999997},"min":{"1min":0.54200000000000004,"5min":0.41199999999999998,"15min":0.41199999999999998},"max":{"1min":0.78900000000000003,"5min":0.97799999999999998,"15min":5.4240000000000004},"last":0.73499999999999999},{"interface":"front","average":{"1min":0.68200000000000005,"5min":0.70799999999999996,"15min":0.83399999999999996},"min":{"1min":0.47599999999999998,"5min":0.47599999999999998,"15min":0.47599999999999998},"max":{"1min":0.88600000000000001,"5min":0.96499999999999997,"15min":5.6020000000000003},"last":0.80500000000000005}]},{"osd":2,"last update":"Sun Nov 26 07:23:56 2023","interfaces":[{"interface":"back","average":{"1min":0.623,"5min":0.66600000000000004,"15min":0.68600000000000005},"min":{"1min":0.47999999999999998,"5min":0.41699999999999998,"15min":0.41699999999999998},"max":{"1min":0.89700000000000002,"5min":0.93600000000000005,"15min":1.8520000000000001},"last":0.69499999999999995},{"interface":"front","average":{"1min":0.71099999999999997,"5min":0.72199999999999998,"15min":0.72999999999999998},"min":{"1min":0.47999999999999998,"5min":0.437,"15min":0.437},"max":{"1min":0.91000000000000003,"5min":0.95199999999999996,"15min":1.9159999999999999},"last":0.76300000000000001}]},{"osd":3,"last update":"Sun Nov 26 07:23:56 2023","interfaces":[{"interface":"back","average":{"1min":0.67900000000000005,"5min":0.70999999999999996,"15min":0.83799999999999997},"min":{"1min":0.47399999999999998,"5min":0.46899999999999997,"15min":0.46899999999999997},"max":{"1min":0.85199999999999998,"5min":0.91500000000000004,"15min":5.4390000000000001},"last":0.77400000000000002},{"interface":"front","average":{"1min":0.65000000000000002,"5min":0.70399999999999996,"15min":0.86399999999999999},"min":{"1min":0.44,"5min":0.44,"15min":0.44},"max":{"1min":0.81299999999999994,"5min":0.90500000000000003,"15min":5.5490000000000004},"last":0.65400000000000003}]},{"osd":4,"last update":"Sun Nov 26 07:23:56 2023","interfaces":[{"interface":"back","average":{"1min":0.61699999999999999,"5min":0.61199999999999999,"15min":0.63200000000000001},"min":{"1min":0.45700000000000002,"5min":0.371,"15min":0.371},"max":{"1min":0.86699999999999999,"5min":1.0129999999999999,"15min":1.0129999999999999},"last":0.57699999999999996},{"interface":"front","average":{"1min":0.623,"5min":0.63400000000000001,"15min":0.66100000000000003},"min":{"1min":0.41299999999999998,"5min":0.38600000000000001,"15min":0.38600000000000001},"max":{"1min":0.85899999999999999,"5min":1.0349999999999999,"15min":1.0349999999999999},"last":0.624}]},{"osd":5,"last update":"Sun Nov 26 07:23:56 2023","interfaces":[{"interface":"back","average":{"1min":0.57999999999999996,"5min":0.61899999999999999,"15min":0.64500000000000002},"min":{"1min":0.42799999999999999,"5min":0.38200000000000001,"15min":0.38200000000000001},"max":{"1min":0.85599999999999998,"5min":0.91000000000000003,"15min":0.91000000000000003},"last":0.60199999999999998},{"interface":"front","average":{"1min":0.66100000000000003,"5min":0.68700000000000006,"15min":0.70399999999999996},"min":{"1min":0.501,"5min":0.44,"15min":0.436},"max":{"1min":0.91600000000000004,"5min":1.0960000000000001,"15min":1.0960000000000001},"last":0.67700000000000005}]},{"osd":7,"last update":"Sun Nov 26 07:23:56 2023","interfaces":[{"interface":"back","average":{"1min":0.66000000000000003,"5min":0.69999999999999996,"15min":0.72799999999999998},"min":{"1min":0.497,"5min":0.42599999999999999,"15min":0.42599999999999999},"max":{"1min":0.89400000000000002,"5min":0.95099999999999996,"15min":0.998},"last":0.71299999999999997},{"interface":"front","average":{"1min":0.64800000000000002,"5min":0.68300000000000005,"15min":0.70499999999999996},"min":{"1min":0.55200000000000005,"5min":0.46400000000000002,"15min":0.46400000000000002},"max":{"1min":0.77500000000000002,"5min":0.98799999999999999,"15min":0.98799999999999999},"last":0.53900000000000003}]}]},{"osd":5,"up_from":13,"seq":55834574937,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":32476,"kb_used_data":1296,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339364,"statfs":{"total":96636764160,"available":96603508736,"internally_reserved":0,"allocated":1327104,"data_stored":579977,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Nov 26 07:24:50 2023","interfaces":[{"interface":"back","average":{"1min":0.61899999999999999,"5min":0.70499999999999996,"15min":0.68100000000000005},"min":{"1min":0.29699999999999999,"5min":0.29699999999999999,"15min":0.29699999999999999},"max":{"1min":0.80800000000000005,"5min":2.6949999999999998,"15min":2.6949999999999998},"last":0.84599999999999997},{"interface":"front","average":{"1min":0.64800000000000002,"5min":0.72399999999999998,"15min":0.68200000000000005},"min":{"1min":0.40200000000000002,"5min":0.38200000000000001,"15min":0.38200000000000001},"max":{"1min":0.88600000000000001,"5min":2.0150000000000001,"15min":2.0150000000000001},"last":0.77800000000000002}]},{"osd":1,"last update":"Sun Nov 26 07:24:50 2023","interfaces":[{"interface":"back","average":{"1min":0.60999999999999999,"5min":0.69099999999999995,"15min":0.67700000000000005},"min":{"1min":0.45900000000000002,"5min":0.41799999999999998,"15min":0.41599999999999998},"max":{"1min":0.77000000000000002,"5min":1.532,"15min":1.532},"last":0.89300000000000002},{"interface":"front","average":{"1min":0.64200000000000002,"5min":0.72399999999999998,"15min":0.69999999999999996},"min":{"1min":0.51800000000000002,"5min":0.443,"15min":0.443},"max":{"1min":0.81100000000000005,"5min":2.6400000000000001,"15min":2.6400000000000001},"last":0.79600000000000004}]},{"osd":2,"last update":"Sun Nov 26 07:24:50 2023","interfaces":[{"interface":"back","average":{"1min":0.65000000000000002,"5min":0.71299999999999997,"15min":0.68799999999999994},"min":{"1min":0.42799999999999999,"5min":0.42099999999999999,"15min":0.42099999999999999},"max":{"1min":0.91500000000000004,"5min":1.665,"15min":1.665},"last":0.82199999999999995},{"interface":"front","average":{"1min":0.66300000000000003,"5min":0.745,"15min":0.69999999999999996},"min":{"1min":0.45700000000000002,"5min":0.45700000000000002,"15min":0.45700000000000002},"max":{"1min":0.85199999999999998,"5min":2.125,"15min":2.125},"last":0.83299999999999996}]},{"osd":3,"last update":"Sun Nov 26 07:24:50 2023","interfaces":[{"interface":"back","average":{"1min":0.68400000000000005,"5min":0.752,"15min":0.72899999999999998},"min":{"1min":0.52800000000000002,"5min":0.44700000000000001,"15min":0.43099999999999999},"max":{"1min":0.879,"5min":2.0529999999999999,"15min":2.0529999999999999},"last":0.80900000000000005},{"interface":"front","average":{"1min":0.67800000000000005,"5min":0.755,"15min":0.70299999999999996},"min":{"1min":0.55500000000000005,"5min":0.51900000000000002,"15min":0.44600000000000001},"max":{"1min":0.94099999999999995,"5min":1.629,"15min":1.629},"last":0.86499999999999999}]},{"osd":4,"last update":"Sun Nov 26 07:24:50 2023","interfaces":[{"interface":"back","average":{"1min":0.58499999999999996,"5min":0.61199999999999999,"15min":0.625},"min":{"1min":0.42199999999999999,"5min":0.35299999999999998,"15min":0.35299999999999998},"max":{"1min":0.84699999999999998,"5min":2.7429999999999999,"15min":2.7429999999999999},"last":0.629},{"interface":"front","average":{"1min":0.60999999999999999,"5min":0.67300000000000004,"15min":0.69299999999999995},"min":{"1min":0.44400000000000001,"5min":0.44400000000000001,"15min":0.41299999999999998},"max":{"1min":0.85999999999999999,"5min":2.4630000000000001,"15min":2.4630000000000001},"last":0.65500000000000003}]},{"osd":6,"last update":"Sun Nov 26 07:24:50 2023","interfaces":[{"interface":"back","average":{"1min":0.63800000000000001,"5min":0.66600000000000004,"15min":0.67100000000000004},"min":{"1min":0.38300000000000001,"5min":0.372,"15min":0.372},"max":{"1min":1.5409999999999999,"5min":2.1619999999999999,"15min":2.1619999999999999},"last":0.70499999999999996},{"interface":"front","average":{"1min":0.63,"5min":0.69099999999999995,"15min":0.71499999999999997},"min":{"1min":0.47599999999999998,"5min":0.436,"15min":0.436},"max":{"1min":0.89500000000000002,"5min":2.7869999999999999,"15min":2.7869999999999999},"last":0.68400000000000005}]},{"osd":7,"last update":"Sun Nov 26 07:24:50 2023","interfaces":[{"interface":"back","average":{"1min":0.66000000000000003,"5min":0.73099999999999998,"15min":0.77500000000000002},"min":{"1min":0.46999999999999997,"5min":0.41399999999999998,"15min":0.41399999999999998},"max":{"1min":0.878,"5min":2.1960000000000002,"15min":2.1960000000000002},"last":0.753},{"interface":"front","average":{"1min":0.67700000000000005,"5min":0.748,"15min":0.76400000000000001},"min":{"1min":0.51100000000000001,"5min":0.44700000000000001,"15min":0.44700000000000001},"max":{"1min":0.94899999999999995,"5min":1.9510000000000001,"15min":1.9510000000000001},"last":0.76200000000000001}]}]},{"osd":4,"up_from":13,"seq":55834574939,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":32476,"kb_used_data":1296,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339364,"statfs":{"total":96636764160,"available":96603508736,"internally_reserved":0,"allocated":1327104,"data_stored":579977,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Nov 26 07:24:55 2023","interfaces":[{"interface":"back","average":{"1min":0.65000000000000002,"5min":0.66600000000000004,"15min":0.73899999999999999},"min":{"1min":0.46600000000000003,"5min":0.443,"15min":0.32100000000000001},"max":{"1min":0.80000000000000004,"5min":0.96199999999999997,"15min":4.1689999999999996},"last":0.61399999999999999},{"interface":"front","average":{"1min":0.67500000000000004,"5min":0.85999999999999999,"15min":0.92800000000000005},"min":{"1min":0.49399999999999999,"5min":0.40799999999999997,"15min":0.40799999999999997},"max":{"1min":1.0209999999999999,"5min":18.021000000000001,"15min":18.021000000000001},"last":0.49399999999999999}]},{"osd":1,"last update":"Sun Nov 26 07:24:55 2023","interfaces":[{"interface":"back","average":{"1min":0.70599999999999996,"5min":0.75,"15min":0.68000000000000005},"min":{"1min":0.54200000000000004,"5min":0.43099999999999999,"15min":0.43099999999999999},"max":{"1min":0.95199999999999996,"5min":4.3159999999999998,"15min":4.3159999999999998},"last":0.58999999999999997},{"interface":"front","average":{"1min":0.63500000000000001,"5min":0.77100000000000002,"15min":0.78900000000000003},"min":{"1min":0.438,"5min":0.436,"15min":0.39800000000000002},"max":{"1min":0.89800000000000002,"5min":11.206,"15min":11.206},"last":0.46500000000000002}]},{"osd":2,"last update":"Sun Nov 26 07:24:55 2023","interfaces":[{"interface":"back","average":{"1min":0.72199999999999998,"5min":0.86499999999999999,"15min":0.81599999999999995},"min":{"1min":0.47399999999999998,"5min":0.46600000000000003,"15min":0.433},"max":{"1min":0.89900000000000002,"5min":15.441000000000001,"15min":15.441000000000001},"last":0.51600000000000001},{"interface":"front","average":{"1min":0.79200000000000004,"5min":0.871,"15min":0.97799999999999998},"min":{"1min":0.54300000000000004,"5min":0.54300000000000004,"15min":0.53400000000000003},"max":{"1min":1.071,"5min":9.4800000000000004,"15min":9.4800000000000004},"last":0.54300000000000004}]},{"osd":3,"last update":"Sun Nov 26 07:24:55 2023","interfaces":[{"interface":"back","average":{"1min":0.69499999999999995,"5min":0.83899999999999997,"15min":0.76000000000000001},"min":{"1min":0.51200000000000001,"5min":0.498,"15min":0.441},"max":{"1min":0.84699999999999998,"5min":12.715,"15min":12.715},"last":0.64900000000000002},{"interface":"front","average":{"1min":0.74399999999999999,"5min":0.82799999999999996,"15min":0.84799999999999998},"min":{"1min":0.55400000000000005,"5min":0.50700000000000001,"15min":0.50700000000000001},"max":{"1min":0.878,"5min":8.3239999999999998,"15min":8.3239999999999998},"last":0.66500000000000004}]},{"osd":5,"last update":"Sun Nov 26 07:24:55 2023","interfaces":[{"interface":"back","average":{"1min":0.68799999999999994,"5min":0.65100000000000002,"15min":0.622},"min":{"1min":0.41299999999999998,"5min":0.40500000000000003,"15min":0.38200000000000001},"max":{"1min":1.0229999999999999,"5min":1.0229999999999999,"15min":1.0229999999999999},"last":0.434},{"interface":"front","average":{"1min":0.67400000000000004,"5min":0.74199999999999999,"15min":0.67000000000000004},"min":{"1min":0.41999999999999998,"5min":0.36399999999999999,"15min":0.35599999999999998},"max":{"1min":0.96099999999999997,"5min":11.148999999999999,"15min":11.148999999999999},"last":0.56799999999999995}]},{"osd":6,"last update":"Sun Nov 26 07:24:55 2023","interfaces":[{"interface":"back","average":{"1min":0.68600000000000005,"5min":0.626,"15min":0.72399999999999998},"min":{"1min":0.40000000000000002,"5min":0.38400000000000001,"15min":0.372},"max":{"1min":0.91600000000000004,"5min":0.91600000000000004,"15min":2.8330000000000002},"last":0.70899999999999996},{"interface":"front","average":{"1min":0.71999999999999997,"5min":0.72399999999999998,"15min":0.89100000000000001},"min":{"1min":0.443,"5min":0.42699999999999999,"15min":0.41299999999999998},"max":{"1min":1.046,"5min":2.472,"15min":7.6050000000000004},"last":0.63100000000000001}]},{"osd":7,"last update":"Sun Nov 26 07:24:55 2023","interfaces":[{"interface":"back","average":{"1min":0.75900000000000001,"5min":0.83799999999999997,"15min":0.83899999999999997},"min":{"1min":0.59199999999999997,"5min":0.48699999999999999,"15min":0.44700000000000001},"max":{"1min":0.95999999999999996,"5min":12.653,"15min":12.653},"last":0.69199999999999995},{"interface":"front","average":{"1min":0.81599999999999995,"5min":0.79700000000000004,"15min":0.84799999999999998},"min":{"1min":0.56499999999999995,"5min":0.439,"15min":0.33300000000000002},"max":{"1min":0.97599999999999998,"5min":3.8319999999999999,"15min":3.8319999999999999},"last":0.74199999999999999}]}]},{"osd":3,"up_from":13,"seq":55834574940,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":32480,"kb_used_data":1296,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339360,"statfs":{"total":96636764160,"available":96603504640,"internally_reserved":0,"allocated":1327104,"data_stored":579977,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1592,"internal_metadata":31914440},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Nov 26 07:24:55 2023","interfaces":[{"interface":"back","average":{"1min":0.377,"5min":0.41099999999999998,"15min":0.39200000000000002},"min":{"1min":0.25900000000000001,"5min":0.218,"15min":0.218},"max":{"1min":0.58199999999999996,"5min":0.72799999999999998,"15min":0.72799999999999998},"last":0.38800000000000001},{"interface":"front","average":{"1min":0.41099999999999998,"5min":0.46000000000000002,"15min":0.443},"min":{"1min":0.27700000000000002,"5min":0.254,"15min":0.254},"max":{"1min":0.56599999999999995,"5min":0.77300000000000002,"15min":0.77300000000000002},"last":0.54600000000000004}]},{"osd":1,"last update":"Sun Nov 26 07:24:55 2023","interfaces":[{"interface":"back","average":{"1min":0.39200000000000002,"5min":0.435,"15min":0.42499999999999999},"min":{"1min":0.29899999999999999,"5min":0.28799999999999998,"15min":0.28799999999999998},"max":{"1min":0.53300000000000003,"5min":0.61299999999999999,"15min":0.752},"last":0.32900000000000001},{"interface":"front","average":{"1min":0.437,"5min":0.48599999999999999,"15min":0.48399999999999999},"min":{"1min":0.315,"5min":0.29899999999999999,"15min":0.29899999999999999},"max":{"1min":0.53300000000000003,"5min":0.872,"15min":0.872},"last":0.45000000000000001}]},{"osd":2,"last update":"Sun Nov 26 07:24:55 2023","interfaces":[{"interface":"back","average":{"1min":0.504,"5min":0.55500000000000005,"15min":0.54200000000000004},"min":{"1min":0.36599999999999999,"5min":0.36599999999999999,"15min":0.36599999999999999},"max":{"1min":0.63200000000000001,"5min":0.97199999999999998,"15min":0.97199999999999998},"last":0.52100000000000002},{"interface":"front","average":{"1min":0.48999999999999999,"5min":0.52300000000000002,"15min":0.51400000000000001},"min":{"1min":0.36299999999999999,"5min":0.32100000000000001,"15min":0.32100000000000001},"max":{"1min":0.65100000000000002,"5min":0.89700000000000002,"15min":0.89700000000000002},"last":0.47699999999999998}]},{"osd":4,"last update":"Sun Nov 26 07:24:55 2023","interfaces":[{"interface":"back","average":{"1min":0.72399999999999998,"5min":0.71099999999999997,"15min":0.70799999999999996},"min":{"1min":0.502,"5min":0.39600000000000002,"15min":0.39600000000000002},"max":{"1min":0.94399999999999995,"5min":0.94399999999999995,"15min":1.091},"last":0.77900000000000003},{"interface":"front","average":{"1min":0.72899999999999998,"5min":0.72299999999999998,"15min":0.72499999999999998},"min":{"1min":0.48999999999999999,"5min":0.39900000000000002,"15min":0.39900000000000002},"max":{"1min":1.0309999999999999,"5min":1.242,"15min":1.242},"last":1.0309999999999999}]},{"osd":5,"last update":"Sun Nov 26 07:24:55 2023","interfaces":[{"interface":"back","average":{"1min":0.70599999999999996,"5min":0.70099999999999996,"15min":0.72399999999999998},"min":{"1min":0.505,"5min":0.45900000000000002,"15min":0.45900000000000002},"max":{"1min":0.86499999999999999,"5min":1.105,"15min":1.105},"last":0.81000000000000005},{"interface":"front","average":{"1min":0.70999999999999996,"5min":0.73299999999999998,"15min":0.73499999999999999},"min":{"1min":0.53100000000000003,"5min":0.47299999999999998,"15min":0.47299999999999998},"max":{"1min":0.94599999999999995,"5min":1.1319999999999999,"15min":1.1319999999999999},"last":0.94599999999999995}]},{"osd":6,"last update":"Sun Nov 26 07:24:55 2023","interfaces":[{"interface":"back","average":{"1min":0.79000000000000004,"5min":0.76800000000000002,"15min":0.79400000000000004},"min":{"1min":0.57099999999999995,"5min":0.5,"15min":0.5},"max":{"1min":1.0640000000000001,"5min":1.214,"15min":1.2729999999999999},"last":0.91300000000000003},{"interface":"front","average":{"1min":0.77200000000000002,"5min":0.75900000000000001,"15min":0.77800000000000002},"min":{"1min":0.53700000000000003,"5min":0.48799999999999999,"15min":0.47299999999999998},"max":{"1min":1.0509999999999999,"5min":1.1060000000000001,"15min":1.204},"last":0.74299999999999999}]},{"osd":7,"last update":"Sun Nov 26 07:24:55 2023","interfaces":[{"interface":"back","average":{"1min":0.73899999999999999,"5min":0.755,"15min":0.78100000000000003},"min":{"1min":0.55700000000000005,"5min":0.437,"15min":0.437},"max":{"1min":1.0069999999999999,"5min":1.0349999999999999,"15min":1.3600000000000001},"last":0.86299999999999999},{"interface":"front","average":{"1min":0.80300000000000005,"5min":0.79000000000000004,"15min":0.81200000000000006},"min":{"1min":0.52600000000000002,"5min":0.47299999999999998,"15min":0.47299999999999998},"max":{"1min":1.0529999999999999,"5min":1.2450000000000001,"15min":1.617},"last":0.99199999999999999}]}]},{"osd":2,"up_from":13,"seq":55834574940,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":32476,"kb_used_data":1296,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339364,"statfs":{"total":96636764160,"available":96603508736,"internally_reserved":0,"allocated":1327104,"data_stored":579977,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Nov 26 07:24:52 2023","interfaces":[{"interface":"back","average":{"1min":0.435,"5min":0.42599999999999999,"15min":0.60099999999999998},"min":{"1min":0.26600000000000001,"5min":0.26600000000000001,"15min":0.26600000000000001},"max":{"1min":0.65400000000000003,"5min":0.72899999999999998,"15min":4.7030000000000003},"last":0.501},{"interface":"front","average":{"1min":0.41799999999999998,"5min":0.46000000000000002,"15min":0.52000000000000002},"min":{"1min":0.28399999999999997,"5min":0.27300000000000002,"15min":0.27300000000000002},"max":{"1min":0.66100000000000003,"5min":0.77200000000000002,"15min":2.4340000000000002},"last":0.36399999999999999}]},{"osd":1,"last update":"Sun Nov 26 07:24:52 2023","interfaces":[{"interface":"back","average":{"1min":0.47199999999999998,"5min":0.48699999999999999,"15min":0.71299999999999997},"min":{"1min":0.33000000000000002,"5min":0.318,"15min":0.318},"max":{"1min":0.61499999999999999,"5min":0.86099999999999999,"15min":4.2690000000000001},"last":0.34100000000000003},{"interface":"front","average":{"1min":0.47299999999999998,"5min":0.502,"15min":0.64300000000000002},"min":{"1min":0.33200000000000002,"5min":0.33200000000000002,"15min":0.31900000000000001},"max":{"1min":0.68100000000000005,"5min":0.76600000000000001,"15min":3.9940000000000002},"last":0.42399999999999999}]},{"osd":3,"last update":"Sun Nov 26 07:24:52 2023","interfaces":[{"interface":"back","average":{"1min":0.46500000000000002,"5min":0.47699999999999998,"15min":0.57499999999999996},"min":{"1min":0.314,"5min":0.29699999999999999,"15min":0.29699999999999999},"max":{"1min":0.73299999999999998,"5min":0.82699999999999996,"15min":4.0300000000000002},"last":0.46700000000000003},{"interface":"front","average":{"1min":0.53000000000000003,"5min":0.54700000000000004,"15min":0.76700000000000002},"min":{"1min":0.374,"5min":0.36099999999999999,"15min":0.36099999999999999},"max":{"1min":0.83999999999999997,"5min":0.86699999999999999,"15min":4.5810000000000004},"last":0.56100000000000005}]},{"osd":4,"last update":"Sun Nov 26 07:24:52 2023","interfaces":[{"interface":"back","average":{"1min":0.72999999999999998,"5min":0.75800000000000001,"15min":0.89500000000000002},"min":{"1min":0.56100000000000005,"5min":0.51400000000000001,"15min":0.51400000000000001},"max":{"1min":1.0189999999999999,"5min":1.0189999999999999,"15min":4.4269999999999996},"last":0.69299999999999995},{"interface":"front","average":{"1min":0.73899999999999999,"5min":0.77100000000000002,"15min":0.91400000000000003},"min":{"1min":0.53500000000000003,"5min":0.53500000000000003,"15min":0.53500000000000003},"max":{"1min":0.96199999999999997,"5min":1.0780000000000001,"15min":4.085},"last":0.73899999999999999}]},{"osd":5,"last update":"Sun Nov 26 07:24:52 2023","interfaces":[{"interface":"back","average":{"1min":0.76700000000000002,"5min":0.77400000000000002,"15min":0.83099999999999996},"min":{"1min":0.53800000000000003,"5min":0.42099999999999999,"15min":0.42099999999999999},"max":{"1min":1.2090000000000001,"5min":1.2090000000000001,"15min":3.9590000000000001},"last":0.71999999999999997},{"interface":"front","average":{"1min":0.74399999999999999,"5min":0.77400000000000002,"15min":0.90300000000000002},"min":{"1min":0.54500000000000004,"5min":0.499,"15min":0.499},"max":{"1min":1.024,"5min":1.0900000000000001,"15min":4.3849999999999998},"last":0.77800000000000002}]},{"osd":6,"last update":"Sun Nov 26 07:24:52 2023","interfaces":[{"interface":"back","average":{"1min":0.77500000000000002,"5min":0.77100000000000002,"15min":0.90500000000000003},"min":{"1min":0.58499999999999996,"5min":0.45200000000000001,"15min":0.45200000000000001},"max":{"1min":1.1160000000000001,"5min":1.1160000000000001,"15min":4.0449999999999999},"last":0.67300000000000004},{"interface":"front","average":{"1min":0.748,"5min":0.755,"15min":0.85299999999999998},"min":{"1min":0.56799999999999995,"5min":0.48099999999999998,"15min":0.48099999999999998},"max":{"1min":1.093,"5min":1.093,"15min":3.992},"last":0.75900000000000001}]},{"osd":7,"last update":"Sun Nov 26 07:24:52 2023","interfaces":[{"interface":"back","average":{"1min":0.79700000000000004,"5min":0.79300000000000004,"15min":0.93500000000000005},"min":{"1min":0.61399999999999999,"5min":0.56799999999999995,"15min":0.52200000000000002},"max":{"1min":1.0609999999999999,"5min":1.0840000000000001,"15min":4.1219999999999999},"last":0.79300000000000004},{"interface":"front","average":{"1min":0.77500000000000002,"5min":0.78100000000000003,"15min":0.875},"min":{"1min":0.61499999999999999,"5min":0.48199999999999998,"15min":0.48199999999999998},"max":{"1min":1.151,"5min":1.151,"15min":3.9159999999999999},"last":0.81200000000000006}]}]},{"osd":1,"up_from":13,"seq":55834574939,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":32476,"kb_used_data":1296,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339364,"statfs":{"total":96636764160,"available":96603508736,"internally_reserved":0,"allocated":1327104,"data_stored":579977,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":31914439},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Nov 26 07:24:51 2023","interfaces":[{"interface":"back","average":{"1min":0.38400000000000001,"5min":0.40500000000000003,"15min":0.41099999999999998},"min":{"1min":0.26600000000000001,"5min":0.26600000000000001,"15min":0.23599999999999999},"max":{"1min":0.58399999999999996,"5min":0.58399999999999996,"15min":0.61199999999999999},"last":0.48199999999999998},{"interface":"front","average":{"1min":0.41199999999999998,"5min":0.44400000000000001,"15min":0.46899999999999997},"min":{"1min":0.28000000000000003,"5min":0.28000000000000003,"15min":0.26700000000000002},"max":{"1min":0.52100000000000002,"5min":0.749,"15min":0.78700000000000003},"last":0.28000000000000003}]},{"osd":2,"last update":"Sun Nov 26 07:24:51 2023","interfaces":[{"interface":"back","average":{"1min":0.42099999999999999,"5min":0.436,"15min":0.44600000000000001},"min":{"1min":0.30499999999999999,"5min":0.29899999999999999,"15min":0.27800000000000002},"max":{"1min":0.63400000000000001,"5min":0.71399999999999997,"15min":0.71399999999999997},"last":0.31900000000000001},{"interface":"front","average":{"1min":0.46200000000000002,"5min":0.47599999999999998,"15min":0.495},"min":{"1min":0.33100000000000002,"5min":0.318,"15min":0.27700000000000002},"max":{"1min":0.61799999999999999,"5min":0.80600000000000005,"15min":0.80600000000000005},"last":0.44400000000000001}]},{"osd":3,"last update":"Sun Nov 26 07:24:51 2023","interfaces":[{"interface":"back","average":{"1min":0.48499999999999999,"5min":0.52100000000000002,"15min":0.52500000000000002},"min":{"1min":0.33200000000000002,"5min":0.33200000000000002,"15min":0.33200000000000002},"max":{"1min":0.72699999999999998,"5min":0.748,"15min":0.748},"last":0.40100000000000002},{"interface":"front","average":{"1min":0.47499999999999998,"5min":0.498,"15min":0.496},"min":{"1min":0.38100000000000001,"5min":0.35499999999999998,"15min":0.31},"max":{"1min":0.71299999999999997,"5min":0.71299999999999997,"15min":0.80700000000000005},"last":0.36299999999999999}]},{"osd":4,"last update":"Sun Nov 26 07:24:51 2023","interfaces":[{"interface":"back","average":{"1min":0.72999999999999998,"5min":0.75800000000000001,"15min":0.71499999999999997},"min":{"1min":0.50600000000000001,"5min":0.50600000000000001,"15min":0.41299999999999998},"max":{"1min":0.95699999999999996,"5min":1.319,"15min":1.319},"last":0.51500000000000001},{"interface":"front","average":{"1min":0.71899999999999997,"5min":0.72499999999999998,"15min":0.69799999999999995},"min":{"1min":0.54300000000000004,"5min":0.44500000000000001,"15min":0.44500000000000001},"max":{"1min":0.97999999999999998,"5min":1.006,"15min":1.131},"last":0.67500000000000004}]},{"osd":5,"last update":"Sun Nov 26 07:24:51 2023","interfaces":[{"interface":"back","average":{"1min":0.73299999999999998,"5min":0.747,"15min":0.72899999999999998},"min":{"1min":0.501,"5min":0.47899999999999998,"15min":0.47899999999999998},"max":{"1min":0.95699999999999996,"5min":1.0409999999999999,"15min":1.1919999999999999},"last":0.74299999999999999},{"interface":"front","average":{"1min":0.73499999999999999,"5min":0.75,"15min":0.72899999999999998},"min":{"1min":0.53100000000000003,"5min":0.47099999999999997,"15min":0.47099999999999997},"max":{"1min":0.86299999999999999,"5min":1.0389999999999999,"15min":1.0609999999999999},"last":0.52300000000000002}]},{"osd":6,"last update":"Sun Nov 26 07:24:51 2023","interfaces":[{"interface":"back","average":{"1min":0.72499999999999998,"5min":0.76000000000000001,"15min":0.72399999999999998},"min":{"1min":0.53800000000000003,"5min":0.53200000000000003,"15min":0.50600000000000001},"max":{"1min":0.89600000000000002,"5min":1.2190000000000001,"15min":1.2190000000000001},"last":0.72199999999999998},{"interface":"front","average":{"1min":0.72199999999999998,"5min":0.76200000000000001,"15min":0.72499999999999998},"min":{"1min":0.52200000000000002,"5min":0.46999999999999997,"15min":0.46999999999999997},"max":{"1min":0.91800000000000004,"5min":1.1659999999999999,"15min":1.1659999999999999},"last":0.59999999999999998}]},{"osd":7,"last update":"Sun Nov 26 07:24:51 2023","interfaces":[{"interface":"back","average":{"1min":0.76700000000000002,"5min":0.77400000000000002,"15min":0.73599999999999999},"min":{"1min":0.57099999999999995,"5min":0.50600000000000001,"15min":0.50600000000000001},"max":{"1min":0.97299999999999998,"5min":1.0920000000000001,"15min":1.0920000000000001},"last":0.65500000000000003},{"interface":"front","average":{"1min":0.76200000000000001,"5min":0.76500000000000001,"15min":0.755},"min":{"1min":0.54000000000000004,"5min":0.53700000000000003,"15min":0.48499999999999999},"max":{"1min":1.006,"5min":1.115,"15min":1.115},"last":0.625}]}]},{"osd":0,"up_from":13,"seq":55834574939,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":33056,"kb_used_data":1876,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94338784,"statfs":{"total":96636764160,"available":96602914816,"internally_reserved":0,"allocated":1921024,"data_stored":1170345,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Sun Nov 26 07:24:50 2023","interfaces":[{"interface":"back","average":{"1min":0.439,"5min":0.437,"15min":0.43099999999999999},"min":{"1min":0.25800000000000001,"5min":0.25800000000000001,"15min":0.25800000000000001},"max":{"1min":0.70499999999999996,"5min":1.26,"15min":1.26},"last":0.45100000000000001},{"interface":"front","average":{"1min":0.45100000000000001,"5min":0.47299999999999998,"15min":0.47499999999999998},"min":{"1min":0.29899999999999999,"5min":0.29899999999999999,"15min":0.29899999999999999},"max":{"1min":0.72099999999999997,"5min":1.212,"15min":1.212},"last":0.47199999999999998}]},{"osd":2,"last update":"Sun Nov 26 07:24:50 2023","interfaces":[{"interface":"back","average":{"1min":0.47199999999999998,"5min":0.497,"15min":0.52200000000000002},"min":{"1min":0.33000000000000002,"5min":0.33000000000000002,"15min":0.33000000000000002},"max":{"1min":0.68100000000000005,"5min":0.84199999999999997,"15min":0.84199999999999997},"last":0.52900000000000003},{"interface":"front","average":{"1min":0.53900000000000003,"5min":0.55800000000000005,"15min":0.55900000000000005},"min":{"1min":0.45200000000000001,"5min":0.40100000000000002,"15min":0.36399999999999999},"max":{"1min":0.81000000000000005,"5min":1.0800000000000001,"15min":1.0800000000000001},"last":0.62}]},{"osd":3,"last update":"Sun Nov 26 07:24:50 2023","interfaces":[{"interface":"back","average":{"1min":0.51600000000000001,"5min":0.51300000000000001,"15min":0.52400000000000002},"min":{"1min":0.41099999999999998,"5min":0.33500000000000002,"15min":0.33500000000000002},"max":{"1min":0.72899999999999998,"5min":0.873,"15min":1.3149999999999999},"last":0.60899999999999999},{"interface":"front","average":{"1min":0.53000000000000003,"5min":0.53600000000000003,"15min":0.54900000000000004},"min":{"1min":0.377,"5min":0.35499999999999998,"15min":0.35499999999999998},"max":{"1min":0.74299999999999999,"5min":0.74299999999999999,"15min":1.2509999999999999},"last":0.58099999999999996}]},{"osd":4,"last update":"Sun Nov 26 07:24:50 2023","interfaces":[{"interface":"back","average":{"1min":0.68500000000000005,"5min":0.73899999999999999,"15min":0.749},"min":{"1min":0.40100000000000002,"5min":0.40100000000000002,"15min":0.40100000000000002},"max":{"1min":0.91300000000000003,"5min":1.0900000000000001,"15min":1.2010000000000001},"last":1.0640000000000001},{"interface":"front","average":{"1min":0.76000000000000001,"5min":0.78000000000000003,"15min":0.78600000000000003},"min":{"1min":0.55900000000000005,"5min":0.51100000000000001,"15min":0.51100000000000001},"max":{"1min":1.012,"5min":1.298,"15min":1.381},"last":1.004}]},{"osd":5,"last update":"Sun Nov 26 07:24:50 2023","interfaces":[{"interface":"back","average":{"1min":0.66800000000000004,"5min":0.73599999999999999,"15min":0.75},"min":{"1min":0.50800000000000001,"5min":0.45200000000000001,"15min":0.42199999999999999},"max":{"1min":0.91600000000000004,"5min":1.1819999999999999,"15min":1.1819999999999999},"last":1.0169999999999999},{"interface":"front","average":{"1min":0.77500000000000002,"5min":0.79800000000000004,"15min":0.79700000000000004},"min":{"1min":0.59799999999999998,"5min":0.53000000000000003,"15min":0.53000000000000003},"max":{"1min":1.0609999999999999,"5min":1.329,"15min":1.399},"last":1.038}]},{"osd":6,"last update":"Sun Nov 26 07:24:50 2023","interfaces":[{"interface":"back","average":{"1min":0.70599999999999996,"5min":0.70599999999999996,"15min":0.73699999999999999},"min":{"1min":0.48999999999999999,"5min":0.48499999999999999,"15min":0.48499999999999999},"max":{"1min":0.97199999999999998,"5min":1.0740000000000001,"15min":1.3029999999999999},"last":1.0529999999999999},{"interface":"front","average":{"1min":0.71099999999999997,"5min":0.72899999999999998,"15min":0.754},"min":{"1min":0.52100000000000002,"5min":0.47199999999999998,"15min":0.47199999999999998},"max":{"1min":0.93600000000000005,"5min":1.2170000000000001,"15min":1.3600000000000001},"last":0.92700000000000005}]},{"osd":7,"last update":"Sun Nov 26 07:24:50 2023","interfaces":[{"interface":"back","average":{"1min":0.75600000000000001,"5min":0.76800000000000002,"15min":0.79300000000000004},"min":{"1min":0.60099999999999998,"5min":0.56899999999999995,"15min":0.56899999999999995},"max":{"1min":0.97999999999999998,"5min":1.1719999999999999,"15min":1.177},"last":0.98799999999999999},{"interface":"front","average":{"1min":0.69899999999999995,"5min":0.76500000000000001,"15min":0.77800000000000002},"min":{"1min":0.56899999999999995,"5min":0.47899999999999998,"15min":0.47899999999999998},"max":{"1min":0.92400000000000004,"5min":1.238,"15min":1.2390000000000001},"last":1.097}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-11-26T07:24:57.423 INFO:tasks.ceph.ceph_manager.ceph:clean! 2023-11-26T07:24:57.423 DEBUG:teuthology.orchestra.run.smithi143:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-11-26T07:24:57.561 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:57.558+0000 7f54ea957640 1 Processor -- start 2023-11-26T07:24:57.561 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:57.558+0000 7f54ea957640 1 -- start start 2023-11-26T07:24:57.562 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:57.562+0000 7f54ea957640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f54e40cf280 0x7f54e40cf6e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:24:57.563 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:57.562+0000 7f54ea957640 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7f54e40cd1b0 0x7f54e40cd5e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:24:57.563 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:57.562+0000 7f54ea957640 1 -- --> v1:172.21.15.188:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f54e40cfc20 con 0x7f54e4143ac0 2023-11-26T07:24:57.563 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:57.562+0000 7f54ea957640 1 -- --> v2:172.21.15.188:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f54e4143ea0 con 0x7f54e40cd1b0 2023-11-26T07:24:57.563 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:57.562+0000 7f54ea957640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f54e40cc730 con 0x7f54e40cf280 2023-11-26T07:24:57.563 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:57.562+0000 7f54e37fe640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f54e40cf280 0x7f54e40cf6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:24:57.563 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:57.562+0000 7f54e8ef4640 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7f54e40cd1b0 0x7f54e40cd5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:24:57.563 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:57.562+0000 7f54e37fe640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f54e40cf280 0x7f54e40cf6e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.188:3301/0 says I am v2:172.21.15.143:46780/0 (socket says 172.21.15.143:46780) 2023-11-26T07:24:57.564 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:57.562+0000 7f54e37fe640 1 -- 172.21.15.143:0/2977390975 learned_addr learned my addr 172.21.15.143:0/2977390975 (peer_addr_for_me v2:172.21.15.143:0/0) 2023-11-26T07:24:57.564 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:57.562+0000 7f54e37fe640 1 -- 172.21.15.143:0/2977390975 >> v2:172.21.15.188:3300/0 conn(0x7f54e40cd1b0 msgr2=0x7f54e40cd5e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-26T07:24:57.564 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:57.562+0000 7f54e37fe640 1 --2- 172.21.15.143:0/2977390975 >> v2:172.21.15.188:3300/0 conn(0x7f54e40cd1b0 0x7f54e40cd5e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:24:57.564 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:57.562+0000 7f54e37fe640 1 -- 172.21.15.143:0/2977390975 >> v1:172.21.15.188:6789/0 conn(0x7f54e4143ac0 legacy=0x7f54e4070370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:24:57.564 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:57.562+0000 7f54e37fe640 1 -- 172.21.15.143:0/2977390975 --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f54e4070ee0 con 0x7f54e40cf280 2023-11-26T07:24:57.566 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:57.566+0000 7f54e37fe640 1 --2- 172.21.15.143:0/2977390975 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f54e40cf280 0x7f54e40cf6e0 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7f54d8009870 tx=0x7f54d801c540 comp rx=0 tx=0).ready entity=mon.2 client_cookie=a64b83c9b646ec83 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:24:57.567 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:57.566+0000 7f54e2ffd640 1 -- 172.21.15.143:0/2977390975 <== mon.2 v2:172.21.15.188:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f54d80a9400 con 0x7f54e40cf280 2023-11-26T07:24:57.567 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:57.566+0000 7f54e2ffd640 1 -- 172.21.15.143:0/2977390975 <== mon.2 v2:172.21.15.188:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f54d80a9560 con 0x7f54e40cf280 2023-11-26T07:24:57.567 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:57.566+0000 7f54e2ffd640 1 -- 172.21.15.143:0/2977390975 <== mon.2 v2:172.21.15.188:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f54d80a98f0 con 0x7f54e40cf280 2023-11-26T07:24:57.567 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:57.566+0000 7f54ea957640 1 -- 172.21.15.143:0/2977390975 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f54e40cf280 msgr2=0x7f54e40cf6e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:24:57.568 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:57.566+0000 7f54ea957640 1 --2- 172.21.15.143:0/2977390975 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f54e40cf280 0x7f54e40cf6e0 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7f54d8009870 tx=0x7f54d801c540 comp rx=0 tx=0).stop 2023-11-26T07:24:57.568 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:57.566+0000 7f54ea957640 1 -- 172.21.15.143:0/2977390975 shutdown_connections 2023-11-26T07:24:57.568 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:57.566+0000 7f54ea957640 1 --2- 172.21.15.143:0/2977390975 >> v2:172.21.15.188:3300/0 conn(0x7f54e40cd1b0 0x7f54e40cd5e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:24:57.568 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:57.566+0000 7f54ea957640 1 --2- 172.21.15.143:0/2977390975 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f54e40cf280 0x7f54e40cf6e0 unknown :-1 s=CLOSED pgs=168 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:24:57.568 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:57.566+0000 7f54ea957640 1 -- 172.21.15.143:0/2977390975 >> 172.21.15.143:0/2977390975 conn(0x7f54e406a890 msgr2=0x7f54e406aca0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:24:57.568 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:57.566+0000 7f54ea957640 1 -- 172.21.15.143:0/2977390975 shutdown_connections 2023-11-26T07:24:57.568 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:57.566+0000 7f54ea957640 1 -- 172.21.15.143:0/2977390975 wait complete. 2023-11-26T07:24:57.569 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:57.566+0000 7f54ea957640 1 Processor -- start 2023-11-26T07:24:57.569 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:57.570+0000 7f54ea957640 1 -- start start 2023-11-26T07:24:57.571 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:57.570+0000 7f54ea957640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f54e40cd1b0 0x7f54e41f71b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:24:57.572 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:57.570+0000 7f54ea957640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f54e40cf280 0x7f54e413fa50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:24:57.572 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:57.570+0000 7f54ea957640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f54e4143ac0 0x7f54e413ff90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:24:57.572 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:57.570+0000 7f54ea957640 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f54e40b0530 con 0x7f54e4143ac0 2023-11-26T07:24:57.572 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:57.570+0000 7f54ea957640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f54e41f76f0 con 0x7f54e40cd1b0 2023-11-26T07:24:57.572 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:57.570+0000 7f54ea957640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f54e40af8b0 con 0x7f54e40cf280 2023-11-26T07:24:57.572 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:57.570+0000 7f54e37fe640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f54e40cf280 0x7f54e413fa50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:24:57.572 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:57.570+0000 7f54e8ef4640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f54e4143ac0 0x7f54e413ff90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:24:57.573 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:57.570+0000 7f54e37fe640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f54e40cf280 0x7f54e413fa50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.188:3301/0 says I am v2:172.21.15.143:46794/0 (socket says 172.21.15.143:46794) 2023-11-26T07:24:57.573 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:57.570+0000 7f54e37fe640 1 -- 172.21.15.143:0/1385686218 learned_addr learned my addr 172.21.15.143:0/1385686218 (peer_addr_for_me v2:172.21.15.143:0/0) 2023-11-26T07:24:57.573 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:57.570+0000 7f54e3fff640 1 --2- 172.21.15.143:0/1385686218 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f54e40cd1b0 0x7f54e41f71b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:24:57.573 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:57.570+0000 7f54e8ef4640 1 -- 172.21.15.143:0/1385686218 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f54e40cf280 msgr2=0x7f54e413fa50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:24:57.573 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:57.570+0000 7f54e8ef4640 1 --2- 172.21.15.143:0/1385686218 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f54e40cf280 0x7f54e413fa50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:24:57.573 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:57.570+0000 7f54e8ef4640 1 -- 172.21.15.143:0/1385686218 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f54e40cd1b0 msgr2=0x7f54e41f71b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:24:57.573 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:57.570+0000 7f54e8ef4640 1 --2- 172.21.15.143:0/1385686218 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f54e40cd1b0 0x7f54e41f71b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:24:57.574 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:57.570+0000 7f54e8ef4640 1 -- 172.21.15.143:0/1385686218 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f54d80094d0 con 0x7f54e4143ac0 2023-11-26T07:24:57.574 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:57.570+0000 7f54e37fe640 1 --2- 172.21.15.143:0/1385686218 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f54e40cf280 0x7f54e413fa50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-26T07:24:57.574 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:57.570+0000 7f54e8ef4640 1 --2- 172.21.15.143:0/1385686218 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f54e4143ac0 0x7f54e413ff90 secure :-1 s=READY pgs=1450 cs=0 l=1 rev1=1 crypto rx=0x7f54dc00eba0 tx=0x7f54dc00c490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:24:57.574 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:57.570+0000 7f54e0ff9640 1 -- 172.21.15.143:0/1385686218 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f54dc00cd20 con 0x7f54e4143ac0 2023-11-26T07:24:57.574 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:57.570+0000 7f54e0ff9640 1 -- 172.21.15.143:0/1385686218 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f54dc00f040 con 0x7f54e4143ac0 2023-11-26T07:24:57.574 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:57.570+0000 7f54ea957640 1 -- 172.21.15.143:0/1385686218 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f54e40afb90 con 0x7f54e4143ac0 2023-11-26T07:24:57.575 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:57.570+0000 7f54e0ff9640 1 -- 172.21.15.143:0/1385686218 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f54dc0146c0 con 0x7f54e4143ac0 2023-11-26T07:24:57.575 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:57.570+0000 7f54ea957640 1 -- 172.21.15.143:0/1385686218 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f54e40b0150 con 0x7f54e4143ac0 2023-11-26T07:24:57.576 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:57.570+0000 7f54ea957640 1 -- 172.21.15.143:0/1385686218 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f54e40b0670 con 0x7f54e4143ac0 2023-11-26T07:24:57.576 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:57.574+0000 7f54e0ff9640 1 -- 172.21.15.143:0/1385686218 <== mon.0 v2:172.21.15.143:3300/0 4 ==== mgrmap(e 18) v1 ==== 86107+0+0 (secure 0 0 0) 0x7f54dc014820 con 0x7f54e4143ac0 2023-11-26T07:24:57.576 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:57.574+0000 7f54e0ff9640 1 --2- 172.21.15.143:0/1385686218 >> [v2:172.21.15.188:6836/3401128553,v1:172.21.15.188:6837/3401128553] conn(0x7f54d0066740 0x7f54d0068c00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:24:57.576 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:57.574+0000 7f54e0ff9640 1 -- 172.21.15.143:0/1385686218 <== mon.0 v2:172.21.15.143:3300/0 5 ==== osd_map(106..106 src has 1..106) v4 ==== 4943+0+0 (secure 0 0 0) 0x7f54dc084c40 con 0x7f54e4143ac0 2023-11-26T07:24:57.576 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:57.574+0000 7f54e3fff640 1 --2- 172.21.15.143:0/1385686218 >> [v2:172.21.15.188:6836/3401128553,v1:172.21.15.188:6837/3401128553] conn(0x7f54d0066740 0x7f54d0068c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:24:57.576 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:57.574+0000 7f54e3fff640 1 --2- 172.21.15.143:0/1385686218 >> [v2:172.21.15.188:6836/3401128553,v1:172.21.15.188:6837/3401128553] conn(0x7f54d0066740 0x7f54d0068c00 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7f54d4001810 tx=0x7f54d4009040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:24:57.578 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:57.578+0000 7f54e0ff9640 1 -- 172.21.15.143:0/1385686218 <== mon.0 v2:172.21.15.143:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172649 (secure 0 0 0) 0x7f54e40b0670 con 0x7f54e4143ac0 2023-11-26T07:24:57.728 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:57.726+0000 7f54ea957640 1 -- 172.21.15.143:0/1385686218 --> [v2:172.21.15.188:6836/3401128553,v1:172.21.15.188:6837/3401128553] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f54e4071c30 con 0x7f54d0066740 2023-11-26T07:24:57.730 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:57.730+0000 7f54e0ff9640 1 -- 172.21.15.143:0/1385686218 <== mgr.4108 v2:172.21.15.188:6836/3401128553 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+50820 (secure 0 0 0) 0x7f54d4001450 con 0x7f54d0066740 2023-11-26T07:24:57.731 INFO:teuthology.orchestra.run.smithi143.stdout: 2023-11-26T07:24:57.732 INFO:teuthology.orchestra.run.smithi143.stderr:dumped all 2023-11-26T07:24:57.733 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:57.730+0000 7f54ea957640 1 -- 172.21.15.143:0/1385686218 >> [v2:172.21.15.188:6836/3401128553,v1:172.21.15.188:6837/3401128553] conn(0x7f54d0066740 msgr2=0x7f54d0068c00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:24:57.733 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:57.730+0000 7f54ea957640 1 --2- 172.21.15.143:0/1385686218 >> [v2:172.21.15.188:6836/3401128553,v1:172.21.15.188:6837/3401128553] conn(0x7f54d0066740 0x7f54d0068c00 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7f54d4001810 tx=0x7f54d4009040 comp rx=0 tx=0).stop 2023-11-26T07:24:57.733 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:57.730+0000 7f54ea957640 1 -- 172.21.15.143:0/1385686218 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f54e4143ac0 msgr2=0x7f54e413ff90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:24:57.733 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:57.730+0000 7f54ea957640 1 --2- 172.21.15.143:0/1385686218 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f54e4143ac0 0x7f54e413ff90 secure :-1 s=READY pgs=1450 cs=0 l=1 rev1=1 crypto rx=0x7f54dc00eba0 tx=0x7f54dc00c490 comp rx=0 tx=0).stop 2023-11-26T07:24:57.733 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:57.730+0000 7f54ea957640 1 -- 172.21.15.143:0/1385686218 shutdown_connections 2023-11-26T07:24:57.734 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:57.730+0000 7f54ea957640 1 --2- 172.21.15.143:0/1385686218 >> [v2:172.21.15.188:6836/3401128553,v1:172.21.15.188:6837/3401128553] conn(0x7f54d0066740 0x7f54d0068c00 unknown :-1 s=CLOSED pgs=185 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:24:57.734 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:57.730+0000 7f54ea957640 1 --2- 172.21.15.143:0/1385686218 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f54e4143ac0 0x7f54e413ff90 unknown :-1 s=CLOSED pgs=1450 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:24:57.734 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:57.730+0000 7f54ea957640 1 --2- 172.21.15.143:0/1385686218 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f54e40cf280 0x7f54e413fa50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:24:57.734 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:57.730+0000 7f54ea957640 1 --2- 172.21.15.143:0/1385686218 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f54e40cd1b0 0x7f54e41f71b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:24:57.734 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:57.730+0000 7f54ea957640 1 -- 172.21.15.143:0/1385686218 >> 172.21.15.143:0/1385686218 conn(0x7f54e406a890 msgr2=0x7f54e406ac70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:24:57.734 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:57.730+0000 7f54ea957640 1 -- 172.21.15.143:0/1385686218 shutdown_connections 2023-11-26T07:24:57.734 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:57.734+0000 7f54ea957640 1 -- 172.21.15.143:0/1385686218 wait complete. 2023-11-26T07:24:57.752 INFO:teuthology.orchestra.run.smithi143.stdout:{"pg_ready":true,"pg_map":{"version":91,"stamp":"2023-11-26T07:24:57.310516+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":212,"num_read_kb":306,"num_write":179,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":128,"ondisk_log_size":128,"up":2,"acting":2,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":69,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":2,"kb":754974720,"kb_used":260972,"kb_used_data":11528,"kb_used_omap":12,"kb_used_meta":249331,"kb_avail":754713748,"statfs":{"total":773094113280,"available":772826877952,"internally_reserved":0,"allocated":11804672,"data_stored":5820552,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12749,"internal_metadata":255315507},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":31,"apply_latency_ms":31,"commit_latency_ns":31000000,"apply_latency_ns":31000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-17207712,"num_objects":-42,"num_object_clones":0,"num_object_copies":-84,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":-42,"num_whiteouts":0,"num_read":-23,"num_read_kb":-8207,"num_write":-184,"num_write_kb":-16883,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":-14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"6.001598"},"pg_stats":[{"pgid":"1.0","version":"72'128","reported_seq":275,"reported_epoch":104,"state":"active+clean","last_fresh":"2023-11-26T07:24:53.071827+0000","last_change":"2023-11-26T07:17:43.827802+0000","last_active":"2023-11-26T07:24:53.071827+0000","last_peered":"2023-11-26T07:24:53.071827+0000","last_clean":"2023-11-26T07:24:53.071827+0000","last_became_active":"2023-11-26T07:17:43.827308+0000","last_became_peered":"2023-11-26T07:17:43.827308+0000","last_unstale":"2023-11-26T07:24:53.071827+0000","last_undegraded":"2023-11-26T07:24:53.071827+0000","last_fullsized":"2023-11-26T07:24:53.071827+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:17:42.160945+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:17:42.160945+0000","last_clean_scrub_stamp":"2023-11-26T07:17:42.160945+0000","objects_scrubbed":0,"log_size":128,"log_dups_size":0,"ondisk_log_size":128,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T08:17:54.167366+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":212,"num_read_kb":306,"num_write":179,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":212,"num_read_kb":306,"num_write":179,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":128,"ondisk_log_size":128,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574937,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":33056,"kb_used_data":1876,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94338784,"statfs":{"total":96636764160,"available":96602914816,"internally_reserved":0,"allocated":1921024,"data_stored":1170345,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":31914442},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Nov 26 07:23:58 2023","interfaces":[{"interface":"back","average":{"1min":0.68200000000000005,"5min":0.70299999999999996,"15min":0.71099999999999997},"min":{"1min":0.47299999999999998,"5min":0.42999999999999999,"15min":0.42999999999999999},"max":{"1min":1.036,"5min":1.036,"15min":1.036},"last":0.53700000000000003},{"interface":"front","average":{"1min":0.70199999999999996,"5min":0.72699999999999998,"15min":0.71499999999999997},"min":{"1min":0.42299999999999999,"5min":0.42299999999999999,"15min":0.42299999999999999},"max":{"1min":1.014,"5min":1.014,"15min":1.0980000000000001},"last":0.55600000000000005}]},{"osd":1,"last update":"Sun Nov 26 07:23:58 2023","interfaces":[{"interface":"back","average":{"1min":0.68400000000000005,"5min":0.68700000000000006,"15min":0.68300000000000005},"min":{"1min":0.45100000000000001,"5min":0.435,"15min":0.435},"max":{"1min":0.85299999999999998,"5min":1.012,"15min":1.0669999999999999},"last":0.66500000000000004},{"interface":"front","average":{"1min":0.66900000000000004,"5min":0.67700000000000005,"15min":0.78500000000000003},"min":{"1min":0.432,"5min":0.432,"15min":0.432},"max":{"1min":0.94699999999999995,"5min":0.94699999999999995,"15min":4.5309999999999997},"last":0.44600000000000001}]},{"osd":2,"last update":"Sun Nov 26 07:23:58 2023","interfaces":[{"interface":"back","average":{"1min":0.69999999999999996,"5min":0.69999999999999996,"15min":0.68400000000000005},"min":{"1min":0.56999999999999995,"5min":0.39700000000000002,"15min":0.39700000000000002},"max":{"1min":0.91600000000000004,"5min":1.0800000000000001,"15min":1.0800000000000001},"last":0.60599999999999998},{"interface":"front","average":{"1min":0.68700000000000006,"5min":0.70499999999999996,"15min":0.70999999999999996},"min":{"1min":0.48999999999999999,"5min":0.40999999999999998,"15min":0.40999999999999998},"max":{"1min":0.97099999999999997,"5min":0.97699999999999998,"15min":1.1339999999999999},"last":0.47299999999999998}]},{"osd":3,"last update":"Sun Nov 26 07:23:58 2023","interfaces":[{"interface":"back","average":{"1min":0.73299999999999998,"5min":0.72799999999999998,"15min":0.70699999999999996},"min":{"1min":0.53200000000000003,"5min":0.47799999999999998,"15min":0.47799999999999998},"max":{"1min":0.99299999999999999,"5min":1.1240000000000001,"15min":1.1240000000000001},"last":0.57499999999999996},{"interface":"front","average":{"1min":0.72399999999999998,"5min":0.74299999999999999,"15min":0.73599999999999999},"min":{"1min":0.50700000000000001,"5min":0.433,"15min":0.433},"max":{"1min":0.93400000000000005,"5min":1.036,"15min":1.4490000000000001},"last":0.68799999999999994}]},{"osd":4,"last update":"Sun Nov 26 07:23:58 2023","interfaces":[{"interface":"back","average":{"1min":0.61099999999999999,"5min":0.61499999999999999,"15min":0.60499999999999998},"min":{"1min":0.48299999999999998,"5min":0.39400000000000002,"15min":0.39400000000000002},"max":{"1min":0.878,"5min":0.96299999999999997,"15min":0.96299999999999997},"last":0.59299999999999997},{"interface":"front","average":{"1min":0.65500000000000003,"5min":0.64400000000000002,"15min":0.66300000000000003},"min":{"1min":0.45800000000000002,"5min":0.35999999999999999,"15min":0.35999999999999999},"max":{"1min":0.83599999999999997,"5min":1.0209999999999999,"15min":1.5509999999999999},"last":0.64800000000000002}]},{"osd":5,"last update":"Sun Nov 26 07:23:58 2023","interfaces":[{"interface":"back","average":{"1min":0.66100000000000003,"5min":0.67400000000000004,"15min":0.66800000000000004},"min":{"1min":0.36599999999999999,"5min":0.36599999999999999,"15min":0.36599999999999999},"max":{"1min":0.86799999999999999,"5min":1.056,"15min":1.415},"last":0.628},{"interface":"front","average":{"1min":0.68300000000000005,"5min":0.68200000000000005,"15min":0.70299999999999996},"min":{"1min":0.435,"5min":0.435,"15min":0.435},"max":{"1min":0.873,"5min":1.0840000000000001,"15min":1.3859999999999999},"last":0.497}]},{"osd":6,"last update":"Sun Nov 26 07:23:58 2023","interfaces":[{"interface":"back","average":{"1min":0.77300000000000002,"5min":0.78000000000000003,"15min":0.77400000000000002},"min":{"1min":0.53100000000000003,"5min":0.51700000000000002,"15min":0.51700000000000002},"max":{"1min":0.95499999999999996,"5min":1.123,"15min":1.3069999999999999},"last":0.61599999999999999},{"interface":"front","average":{"1min":0.71399999999999997,"5min":0.74399999999999999,"15min":0.76200000000000001},"min":{"1min":0.50600000000000001,"5min":0.46899999999999997,"15min":0.46899999999999997},"max":{"1min":0.92900000000000005,"5min":1.036,"15min":1.6759999999999999},"last":0.51600000000000001}]}]},{"osd":6,"up_from":13,"seq":55834574939,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":32476,"kb_used_data":1296,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339364,"statfs":{"total":96636764160,"available":96603508736,"internally_reserved":0,"allocated":1327104,"data_stored":579977,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Nov 26 07:23:56 2023","interfaces":[{"interface":"back","average":{"1min":0.65400000000000003,"5min":0.67800000000000005,"15min":0.68200000000000005},"min":{"1min":0.48999999999999999,"5min":0.32200000000000001,"15min":0.32200000000000001},"max":{"1min":0.875,"5min":0.91800000000000004,"15min":1.7549999999999999},"last":0.78800000000000003},{"interface":"front","average":{"1min":0.65500000000000003,"5min":0.69599999999999995,"15min":0.87},"min":{"1min":0.36299999999999999,"5min":0.36299999999999999,"15min":0.36299999999999999},"max":{"1min":0.82599999999999996,"5min":0.96199999999999997,"15min":5.7489999999999997},"last":0.75}]},{"osd":1,"last update":"Sun Nov 26 07:23:56 2023","interfaces":[{"interface":"back","average":{"1min":0.65000000000000002,"5min":0.70799999999999996,"15min":0.84499999999999997},"min":{"1min":0.54200000000000004,"5min":0.41199999999999998,"15min":0.41199999999999998},"max":{"1min":0.78900000000000003,"5min":0.97799999999999998,"15min":5.4240000000000004},"last":0.73499999999999999},{"interface":"front","average":{"1min":0.68200000000000005,"5min":0.70799999999999996,"15min":0.83399999999999996},"min":{"1min":0.47599999999999998,"5min":0.47599999999999998,"15min":0.47599999999999998},"max":{"1min":0.88600000000000001,"5min":0.96499999999999997,"15min":5.6020000000000003},"last":0.80500000000000005}]},{"osd":2,"last update":"Sun Nov 26 07:23:56 2023","interfaces":[{"interface":"back","average":{"1min":0.623,"5min":0.66600000000000004,"15min":0.68600000000000005},"min":{"1min":0.47999999999999998,"5min":0.41699999999999998,"15min":0.41699999999999998},"max":{"1min":0.89700000000000002,"5min":0.93600000000000005,"15min":1.8520000000000001},"last":0.69499999999999995},{"interface":"front","average":{"1min":0.71099999999999997,"5min":0.72199999999999998,"15min":0.72999999999999998},"min":{"1min":0.47999999999999998,"5min":0.437,"15min":0.437},"max":{"1min":0.91000000000000003,"5min":0.95199999999999996,"15min":1.9159999999999999},"last":0.76300000000000001}]},{"osd":3,"last update":"Sun Nov 26 07:23:56 2023","interfaces":[{"interface":"back","average":{"1min":0.67900000000000005,"5min":0.70999999999999996,"15min":0.83799999999999997},"min":{"1min":0.47399999999999998,"5min":0.46899999999999997,"15min":0.46899999999999997},"max":{"1min":0.85199999999999998,"5min":0.91500000000000004,"15min":5.4390000000000001},"last":0.77400000000000002},{"interface":"front","average":{"1min":0.65000000000000002,"5min":0.70399999999999996,"15min":0.86399999999999999},"min":{"1min":0.44,"5min":0.44,"15min":0.44},"max":{"1min":0.81299999999999994,"5min":0.90500000000000003,"15min":5.5490000000000004},"last":0.65400000000000003}]},{"osd":4,"last update":"Sun Nov 26 07:23:56 2023","interfaces":[{"interface":"back","average":{"1min":0.61699999999999999,"5min":0.61199999999999999,"15min":0.63200000000000001},"min":{"1min":0.45700000000000002,"5min":0.371,"15min":0.371},"max":{"1min":0.86699999999999999,"5min":1.0129999999999999,"15min":1.0129999999999999},"last":0.57699999999999996},{"interface":"front","average":{"1min":0.623,"5min":0.63400000000000001,"15min":0.66100000000000003},"min":{"1min":0.41299999999999998,"5min":0.38600000000000001,"15min":0.38600000000000001},"max":{"1min":0.85899999999999999,"5min":1.0349999999999999,"15min":1.0349999999999999},"last":0.624}]},{"osd":5,"last update":"Sun Nov 26 07:23:56 2023","interfaces":[{"interface":"back","average":{"1min":0.57999999999999996,"5min":0.61899999999999999,"15min":0.64500000000000002},"min":{"1min":0.42799999999999999,"5min":0.38200000000000001,"15min":0.38200000000000001},"max":{"1min":0.85599999999999998,"5min":0.91000000000000003,"15min":0.91000000000000003},"last":0.60199999999999998},{"interface":"front","average":{"1min":0.66100000000000003,"5min":0.68700000000000006,"15min":0.70399999999999996},"min":{"1min":0.501,"5min":0.44,"15min":0.436},"max":{"1min":0.91600000000000004,"5min":1.0960000000000001,"15min":1.0960000000000001},"last":0.67700000000000005}]},{"osd":7,"last update":"Sun Nov 26 07:23:56 2023","interfaces":[{"interface":"back","average":{"1min":0.66000000000000003,"5min":0.69999999999999996,"15min":0.72799999999999998},"min":{"1min":0.497,"5min":0.42599999999999999,"15min":0.42599999999999999},"max":{"1min":0.89400000000000002,"5min":0.95099999999999996,"15min":0.998},"last":0.71299999999999997},{"interface":"front","average":{"1min":0.64800000000000002,"5min":0.68300000000000005,"15min":0.70499999999999996},"min":{"1min":0.55200000000000005,"5min":0.46400000000000002,"15min":0.46400000000000002},"max":{"1min":0.77500000000000002,"5min":0.98799999999999999,"15min":0.98799999999999999},"last":0.53900000000000003}]}]},{"osd":5,"up_from":13,"seq":55834574937,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":32476,"kb_used_data":1296,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339364,"statfs":{"total":96636764160,"available":96603508736,"internally_reserved":0,"allocated":1327104,"data_stored":579977,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Nov 26 07:24:50 2023","interfaces":[{"interface":"back","average":{"1min":0.61899999999999999,"5min":0.70499999999999996,"15min":0.68100000000000005},"min":{"1min":0.29699999999999999,"5min":0.29699999999999999,"15min":0.29699999999999999},"max":{"1min":0.80800000000000005,"5min":2.6949999999999998,"15min":2.6949999999999998},"last":0.84599999999999997},{"interface":"front","average":{"1min":0.64800000000000002,"5min":0.72399999999999998,"15min":0.68200000000000005},"min":{"1min":0.40200000000000002,"5min":0.38200000000000001,"15min":0.38200000000000001},"max":{"1min":0.88600000000000001,"5min":2.0150000000000001,"15min":2.0150000000000001},"last":0.77800000000000002}]},{"osd":1,"last update":"Sun Nov 26 07:24:50 2023","interfaces":[{"interface":"back","average":{"1min":0.60999999999999999,"5min":0.69099999999999995,"15min":0.67700000000000005},"min":{"1min":0.45900000000000002,"5min":0.41799999999999998,"15min":0.41599999999999998},"max":{"1min":0.77000000000000002,"5min":1.532,"15min":1.532},"last":0.89300000000000002},{"interface":"front","average":{"1min":0.64200000000000002,"5min":0.72399999999999998,"15min":0.69999999999999996},"min":{"1min":0.51800000000000002,"5min":0.443,"15min":0.443},"max":{"1min":0.81100000000000005,"5min":2.6400000000000001,"15min":2.6400000000000001},"last":0.79600000000000004}]},{"osd":2,"last update":"Sun Nov 26 07:24:50 2023","interfaces":[{"interface":"back","average":{"1min":0.65000000000000002,"5min":0.71299999999999997,"15min":0.68799999999999994},"min":{"1min":0.42799999999999999,"5min":0.42099999999999999,"15min":0.42099999999999999},"max":{"1min":0.91500000000000004,"5min":1.665,"15min":1.665},"last":0.82199999999999995},{"interface":"front","average":{"1min":0.66300000000000003,"5min":0.745,"15min":0.69999999999999996},"min":{"1min":0.45700000000000002,"5min":0.45700000000000002,"15min":0.45700000000000002},"max":{"1min":0.85199999999999998,"5min":2.125,"15min":2.125},"last":0.83299999999999996}]},{"osd":3,"last update":"Sun Nov 26 07:24:50 2023","interfaces":[{"interface":"back","average":{"1min":0.68400000000000005,"5min":0.752,"15min":0.72899999999999998},"min":{"1min":0.52800000000000002,"5min":0.44700000000000001,"15min":0.43099999999999999},"max":{"1min":0.879,"5min":2.0529999999999999,"15min":2.0529999999999999},"last":0.80900000000000005},{"interface":"front","average":{"1min":0.67800000000000005,"5min":0.755,"15min":0.70299999999999996},"min":{"1min":0.55500000000000005,"5min":0.51900000000000002,"15min":0.44600000000000001},"max":{"1min":0.94099999999999995,"5min":1.629,"15min":1.629},"last":0.86499999999999999}]},{"osd":4,"last update":"Sun Nov 26 07:24:50 2023","interfaces":[{"interface":"back","average":{"1min":0.58499999999999996,"5min":0.61199999999999999,"15min":0.625},"min":{"1min":0.42199999999999999,"5min":0.35299999999999998,"15min":0.35299999999999998},"max":{"1min":0.84699999999999998,"5min":2.7429999999999999,"15min":2.7429999999999999},"last":0.629},{"interface":"front","average":{"1min":0.60999999999999999,"5min":0.67300000000000004,"15min":0.69299999999999995},"min":{"1min":0.44400000000000001,"5min":0.44400000000000001,"15min":0.41299999999999998},"max":{"1min":0.85999999999999999,"5min":2.4630000000000001,"15min":2.4630000000000001},"last":0.65500000000000003}]},{"osd":6,"last update":"Sun Nov 26 07:24:50 2023","interfaces":[{"interface":"back","average":{"1min":0.63800000000000001,"5min":0.66600000000000004,"15min":0.67100000000000004},"min":{"1min":0.38300000000000001,"5min":0.372,"15min":0.372},"max":{"1min":1.5409999999999999,"5min":2.1619999999999999,"15min":2.1619999999999999},"last":0.70499999999999996},{"interface":"front","average":{"1min":0.63,"5min":0.69099999999999995,"15min":0.71499999999999997},"min":{"1min":0.47599999999999998,"5min":0.436,"15min":0.436},"max":{"1min":0.89500000000000002,"5min":2.7869999999999999,"15min":2.7869999999999999},"last":0.68400000000000005}]},{"osd":7,"last update":"Sun Nov 26 07:24:50 2023","interfaces":[{"interface":"back","average":{"1min":0.66000000000000003,"5min":0.73099999999999998,"15min":0.77500000000000002},"min":{"1min":0.46999999999999997,"5min":0.41399999999999998,"15min":0.41399999999999998},"max":{"1min":0.878,"5min":2.1960000000000002,"15min":2.1960000000000002},"last":0.753},{"interface":"front","average":{"1min":0.67700000000000005,"5min":0.748,"15min":0.76400000000000001},"min":{"1min":0.51100000000000001,"5min":0.44700000000000001,"15min":0.44700000000000001},"max":{"1min":0.94899999999999995,"5min":1.9510000000000001,"15min":1.9510000000000001},"last":0.76200000000000001}]}]},{"osd":4,"up_from":13,"seq":55834574939,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":32476,"kb_used_data":1296,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339364,"statfs":{"total":96636764160,"available":96603508736,"internally_reserved":0,"allocated":1327104,"data_stored":579977,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Nov 26 07:24:55 2023","interfaces":[{"interface":"back","average":{"1min":0.65000000000000002,"5min":0.66600000000000004,"15min":0.73899999999999999},"min":{"1min":0.46600000000000003,"5min":0.443,"15min":0.32100000000000001},"max":{"1min":0.80000000000000004,"5min":0.96199999999999997,"15min":4.1689999999999996},"last":0.61399999999999999},{"interface":"front","average":{"1min":0.67500000000000004,"5min":0.85999999999999999,"15min":0.92800000000000005},"min":{"1min":0.49399999999999999,"5min":0.40799999999999997,"15min":0.40799999999999997},"max":{"1min":1.0209999999999999,"5min":18.021000000000001,"15min":18.021000000000001},"last":0.49399999999999999}]},{"osd":1,"last update":"Sun Nov 26 07:24:55 2023","interfaces":[{"interface":"back","average":{"1min":0.70599999999999996,"5min":0.75,"15min":0.68000000000000005},"min":{"1min":0.54200000000000004,"5min":0.43099999999999999,"15min":0.43099999999999999},"max":{"1min":0.95199999999999996,"5min":4.3159999999999998,"15min":4.3159999999999998},"last":0.58999999999999997},{"interface":"front","average":{"1min":0.63500000000000001,"5min":0.77100000000000002,"15min":0.78900000000000003},"min":{"1min":0.438,"5min":0.436,"15min":0.39800000000000002},"max":{"1min":0.89800000000000002,"5min":11.206,"15min":11.206},"last":0.46500000000000002}]},{"osd":2,"last update":"Sun Nov 26 07:24:55 2023","interfaces":[{"interface":"back","average":{"1min":0.72199999999999998,"5min":0.86499999999999999,"15min":0.81599999999999995},"min":{"1min":0.47399999999999998,"5min":0.46600000000000003,"15min":0.433},"max":{"1min":0.89900000000000002,"5min":15.441000000000001,"15min":15.441000000000001},"last":0.51600000000000001},{"interface":"front","average":{"1min":0.79200000000000004,"5min":0.871,"15min":0.97799999999999998},"min":{"1min":0.54300000000000004,"5min":0.54300000000000004,"15min":0.53400000000000003},"max":{"1min":1.071,"5min":9.4800000000000004,"15min":9.4800000000000004},"last":0.54300000000000004}]},{"osd":3,"last update":"Sun Nov 26 07:24:55 2023","interfaces":[{"interface":"back","average":{"1min":0.69499999999999995,"5min":0.83899999999999997,"15min":0.76000000000000001},"min":{"1min":0.51200000000000001,"5min":0.498,"15min":0.441},"max":{"1min":0.84699999999999998,"5min":12.715,"15min":12.715},"last":0.64900000000000002},{"interface":"front","average":{"1min":0.74399999999999999,"5min":0.82799999999999996,"15min":0.84799999999999998},"min":{"1min":0.55400000000000005,"5min":0.50700000000000001,"15min":0.50700000000000001},"max":{"1min":0.878,"5min":8.3239999999999998,"15min":8.3239999999999998},"last":0.66500000000000004}]},{"osd":5,"last update":"Sun Nov 26 07:24:55 2023","interfaces":[{"interface":"back","average":{"1min":0.68799999999999994,"5min":0.65100000000000002,"15min":0.622},"min":{"1min":0.41299999999999998,"5min":0.40500000000000003,"15min":0.38200000000000001},"max":{"1min":1.0229999999999999,"5min":1.0229999999999999,"15min":1.0229999999999999},"last":0.434},{"interface":"front","average":{"1min":0.67400000000000004,"5min":0.74199999999999999,"15min":0.67000000000000004},"min":{"1min":0.41999999999999998,"5min":0.36399999999999999,"15min":0.35599999999999998},"max":{"1min":0.96099999999999997,"5min":11.148999999999999,"15min":11.148999999999999},"last":0.56799999999999995}]},{"osd":6,"last update":"Sun Nov 26 07:24:55 2023","interfaces":[{"interface":"back","average":{"1min":0.68600000000000005,"5min":0.626,"15min":0.72399999999999998},"min":{"1min":0.40000000000000002,"5min":0.38400000000000001,"15min":0.372},"max":{"1min":0.91600000000000004,"5min":0.91600000000000004,"15min":2.8330000000000002},"last":0.70899999999999996},{"interface":"front","average":{"1min":0.71999999999999997,"5min":0.72399999999999998,"15min":0.89100000000000001},"min":{"1min":0.443,"5min":0.42699999999999999,"15min":0.41299999999999998},"max":{"1min":1.046,"5min":2.472,"15min":7.6050000000000004},"last":0.63100000000000001}]},{"osd":7,"last update":"Sun Nov 26 07:24:55 2023","interfaces":[{"interface":"back","average":{"1min":0.75900000000000001,"5min":0.83799999999999997,"15min":0.83899999999999997},"min":{"1min":0.59199999999999997,"5min":0.48699999999999999,"15min":0.44700000000000001},"max":{"1min":0.95999999999999996,"5min":12.653,"15min":12.653},"last":0.69199999999999995},{"interface":"front","average":{"1min":0.81599999999999995,"5min":0.79700000000000004,"15min":0.84799999999999998},"min":{"1min":0.56499999999999995,"5min":0.439,"15min":0.33300000000000002},"max":{"1min":0.97599999999999998,"5min":3.8319999999999999,"15min":3.8319999999999999},"last":0.74199999999999999}]}]},{"osd":3,"up_from":13,"seq":55834574940,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":32480,"kb_used_data":1296,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339360,"statfs":{"total":96636764160,"available":96603504640,"internally_reserved":0,"allocated":1327104,"data_stored":579977,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1592,"internal_metadata":31914440},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Nov 26 07:24:55 2023","interfaces":[{"interface":"back","average":{"1min":0.377,"5min":0.41099999999999998,"15min":0.39200000000000002},"min":{"1min":0.25900000000000001,"5min":0.218,"15min":0.218},"max":{"1min":0.58199999999999996,"5min":0.72799999999999998,"15min":0.72799999999999998},"last":0.38800000000000001},{"interface":"front","average":{"1min":0.41099999999999998,"5min":0.46000000000000002,"15min":0.443},"min":{"1min":0.27700000000000002,"5min":0.254,"15min":0.254},"max":{"1min":0.56599999999999995,"5min":0.77300000000000002,"15min":0.77300000000000002},"last":0.54600000000000004}]},{"osd":1,"last update":"Sun Nov 26 07:24:55 2023","interfaces":[{"interface":"back","average":{"1min":0.39200000000000002,"5min":0.435,"15min":0.42499999999999999},"min":{"1min":0.29899999999999999,"5min":0.28799999999999998,"15min":0.28799999999999998},"max":{"1min":0.53300000000000003,"5min":0.61299999999999999,"15min":0.752},"last":0.32900000000000001},{"interface":"front","average":{"1min":0.437,"5min":0.48599999999999999,"15min":0.48399999999999999},"min":{"1min":0.315,"5min":0.29899999999999999,"15min":0.29899999999999999},"max":{"1min":0.53300000000000003,"5min":0.872,"15min":0.872},"last":0.45000000000000001}]},{"osd":2,"last update":"Sun Nov 26 07:24:55 2023","interfaces":[{"interface":"back","average":{"1min":0.504,"5min":0.55500000000000005,"15min":0.54200000000000004},"min":{"1min":0.36599999999999999,"5min":0.36599999999999999,"15min":0.36599999999999999},"max":{"1min":0.63200000000000001,"5min":0.97199999999999998,"15min":0.97199999999999998},"last":0.52100000000000002},{"interface":"front","average":{"1min":0.48999999999999999,"5min":0.52300000000000002,"15min":0.51400000000000001},"min":{"1min":0.36299999999999999,"5min":0.32100000000000001,"15min":0.32100000000000001},"max":{"1min":0.65100000000000002,"5min":0.89700000000000002,"15min":0.89700000000000002},"last":0.47699999999999998}]},{"osd":4,"last update":"Sun Nov 26 07:24:55 2023","interfaces":[{"interface":"back","average":{"1min":0.72399999999999998,"5min":0.71099999999999997,"15min":0.70799999999999996},"min":{"1min":0.502,"5min":0.39600000000000002,"15min":0.39600000000000002},"max":{"1min":0.94399999999999995,"5min":0.94399999999999995,"15min":1.091},"last":0.77900000000000003},{"interface":"front","average":{"1min":0.72899999999999998,"5min":0.72299999999999998,"15min":0.72499999999999998},"min":{"1min":0.48999999999999999,"5min":0.39900000000000002,"15min":0.39900000000000002},"max":{"1min":1.0309999999999999,"5min":1.242,"15min":1.242},"last":1.0309999999999999}]},{"osd":5,"last update":"Sun Nov 26 07:24:55 2023","interfaces":[{"interface":"back","average":{"1min":0.70599999999999996,"5min":0.70099999999999996,"15min":0.72399999999999998},"min":{"1min":0.505,"5min":0.45900000000000002,"15min":0.45900000000000002},"max":{"1min":0.86499999999999999,"5min":1.105,"15min":1.105},"last":0.81000000000000005},{"interface":"front","average":{"1min":0.70999999999999996,"5min":0.73299999999999998,"15min":0.73499999999999999},"min":{"1min":0.53100000000000003,"5min":0.47299999999999998,"15min":0.47299999999999998},"max":{"1min":0.94599999999999995,"5min":1.1319999999999999,"15min":1.1319999999999999},"last":0.94599999999999995}]},{"osd":6,"last update":"Sun Nov 26 07:24:55 2023","interfaces":[{"interface":"back","average":{"1min":0.79000000000000004,"5min":0.76800000000000002,"15min":0.79400000000000004},"min":{"1min":0.57099999999999995,"5min":0.5,"15min":0.5},"max":{"1min":1.0640000000000001,"5min":1.214,"15min":1.2729999999999999},"last":0.91300000000000003},{"interface":"front","average":{"1min":0.77200000000000002,"5min":0.75900000000000001,"15min":0.77800000000000002},"min":{"1min":0.53700000000000003,"5min":0.48799999999999999,"15min":0.47299999999999998},"max":{"1min":1.0509999999999999,"5min":1.1060000000000001,"15min":1.204},"last":0.74299999999999999}]},{"osd":7,"last update":"Sun Nov 26 07:24:55 2023","interfaces":[{"interface":"back","average":{"1min":0.73899999999999999,"5min":0.755,"15min":0.78100000000000003},"min":{"1min":0.55700000000000005,"5min":0.437,"15min":0.437},"max":{"1min":1.0069999999999999,"5min":1.0349999999999999,"15min":1.3600000000000001},"last":0.86299999999999999},{"interface":"front","average":{"1min":0.80300000000000005,"5min":0.79000000000000004,"15min":0.81200000000000006},"min":{"1min":0.52600000000000002,"5min":0.47299999999999998,"15min":0.47299999999999998},"max":{"1min":1.0529999999999999,"5min":1.2450000000000001,"15min":1.617},"last":0.99199999999999999}]}]},{"osd":2,"up_from":13,"seq":55834574940,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":32476,"kb_used_data":1296,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339364,"statfs":{"total":96636764160,"available":96603508736,"internally_reserved":0,"allocated":1327104,"data_stored":579977,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Nov 26 07:24:52 2023","interfaces":[{"interface":"back","average":{"1min":0.435,"5min":0.42599999999999999,"15min":0.60099999999999998},"min":{"1min":0.26600000000000001,"5min":0.26600000000000001,"15min":0.26600000000000001},"max":{"1min":0.65400000000000003,"5min":0.72899999999999998,"15min":4.7030000000000003},"last":0.501},{"interface":"front","average":{"1min":0.41799999999999998,"5min":0.46000000000000002,"15min":0.52000000000000002},"min":{"1min":0.28399999999999997,"5min":0.27300000000000002,"15min":0.27300000000000002},"max":{"1min":0.66100000000000003,"5min":0.77200000000000002,"15min":2.4340000000000002},"last":0.36399999999999999}]},{"osd":1,"last update":"Sun Nov 26 07:24:52 2023","interfaces":[{"interface":"back","average":{"1min":0.47199999999999998,"5min":0.48699999999999999,"15min":0.71299999999999997},"min":{"1min":0.33000000000000002,"5min":0.318,"15min":0.318},"max":{"1min":0.61499999999999999,"5min":0.86099999999999999,"15min":4.2690000000000001},"last":0.34100000000000003},{"interface":"front","average":{"1min":0.47299999999999998,"5min":0.502,"15min":0.64300000000000002},"min":{"1min":0.33200000000000002,"5min":0.33200000000000002,"15min":0.31900000000000001},"max":{"1min":0.68100000000000005,"5min":0.76600000000000001,"15min":3.9940000000000002},"last":0.42399999999999999}]},{"osd":3,"last update":"Sun Nov 26 07:24:52 2023","interfaces":[{"interface":"back","average":{"1min":0.46500000000000002,"5min":0.47699999999999998,"15min":0.57499999999999996},"min":{"1min":0.314,"5min":0.29699999999999999,"15min":0.29699999999999999},"max":{"1min":0.73299999999999998,"5min":0.82699999999999996,"15min":4.0300000000000002},"last":0.46700000000000003},{"interface":"front","average":{"1min":0.53000000000000003,"5min":0.54700000000000004,"15min":0.76700000000000002},"min":{"1min":0.374,"5min":0.36099999999999999,"15min":0.36099999999999999},"max":{"1min":0.83999999999999997,"5min":0.86699999999999999,"15min":4.5810000000000004},"last":0.56100000000000005}]},{"osd":4,"last update":"Sun Nov 26 07:24:52 2023","interfaces":[{"interface":"back","average":{"1min":0.72999999999999998,"5min":0.75800000000000001,"15min":0.89500000000000002},"min":{"1min":0.56100000000000005,"5min":0.51400000000000001,"15min":0.51400000000000001},"max":{"1min":1.0189999999999999,"5min":1.0189999999999999,"15min":4.4269999999999996},"last":0.69299999999999995},{"interface":"front","average":{"1min":0.73899999999999999,"5min":0.77100000000000002,"15min":0.91400000000000003},"min":{"1min":0.53500000000000003,"5min":0.53500000000000003,"15min":0.53500000000000003},"max":{"1min":0.96199999999999997,"5min":1.0780000000000001,"15min":4.085},"last":0.73899999999999999}]},{"osd":5,"last update":"Sun Nov 26 07:24:52 2023","interfaces":[{"interface":"back","average":{"1min":0.76700000000000002,"5min":0.77400000000000002,"15min":0.83099999999999996},"min":{"1min":0.53800000000000003,"5min":0.42099999999999999,"15min":0.42099999999999999},"max":{"1min":1.2090000000000001,"5min":1.2090000000000001,"15min":3.9590000000000001},"last":0.71999999999999997},{"interface":"front","average":{"1min":0.74399999999999999,"5min":0.77400000000000002,"15min":0.90300000000000002},"min":{"1min":0.54500000000000004,"5min":0.499,"15min":0.499},"max":{"1min":1.024,"5min":1.0900000000000001,"15min":4.3849999999999998},"last":0.77800000000000002}]},{"osd":6,"last update":"Sun Nov 26 07:24:52 2023","interfaces":[{"interface":"back","average":{"1min":0.77500000000000002,"5min":0.77100000000000002,"15min":0.90500000000000003},"min":{"1min":0.58499999999999996,"5min":0.45200000000000001,"15min":0.45200000000000001},"max":{"1min":1.1160000000000001,"5min":1.1160000000000001,"15min":4.0449999999999999},"last":0.67300000000000004},{"interface":"front","average":{"1min":0.748,"5min":0.755,"15min":0.85299999999999998},"min":{"1min":0.56799999999999995,"5min":0.48099999999999998,"15min":0.48099999999999998},"max":{"1min":1.093,"5min":1.093,"15min":3.992},"last":0.75900000000000001}]},{"osd":7,"last update":"Sun Nov 26 07:24:52 2023","interfaces":[{"interface":"back","average":{"1min":0.79700000000000004,"5min":0.79300000000000004,"15min":0.93500000000000005},"min":{"1min":0.61399999999999999,"5min":0.56799999999999995,"15min":0.52200000000000002},"max":{"1min":1.0609999999999999,"5min":1.0840000000000001,"15min":4.1219999999999999},"last":0.79300000000000004},{"interface":"front","average":{"1min":0.77500000000000002,"5min":0.78100000000000003,"15min":0.875},"min":{"1min":0.61499999999999999,"5min":0.48199999999999998,"15min":0.48199999999999998},"max":{"1min":1.151,"5min":1.151,"15min":3.9159999999999999},"last":0.81200000000000006}]}]},{"osd":1,"up_from":13,"seq":55834574939,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":32476,"kb_used_data":1296,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339364,"statfs":{"total":96636764160,"available":96603508736,"internally_reserved":0,"allocated":1327104,"data_stored":579977,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":31914439},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Nov 26 07:24:51 2023","interfaces":[{"interface":"back","average":{"1min":0.38400000000000001,"5min":0.40500000000000003,"15min":0.41099999999999998},"min":{"1min":0.26600000000000001,"5min":0.26600000000000001,"15min":0.23599999999999999},"max":{"1min":0.58399999999999996,"5min":0.58399999999999996,"15min":0.61199999999999999},"last":0.48199999999999998},{"interface":"front","average":{"1min":0.41199999999999998,"5min":0.44400000000000001,"15min":0.46899999999999997},"min":{"1min":0.28000000000000003,"5min":0.28000000000000003,"15min":0.26700000000000002},"max":{"1min":0.52100000000000002,"5min":0.749,"15min":0.78700000000000003},"last":0.28000000000000003}]},{"osd":2,"last update":"Sun Nov 26 07:24:51 2023","interfaces":[{"interface":"back","average":{"1min":0.42099999999999999,"5min":0.436,"15min":0.44600000000000001},"min":{"1min":0.30499999999999999,"5min":0.29899999999999999,"15min":0.27800000000000002},"max":{"1min":0.63400000000000001,"5min":0.71399999999999997,"15min":0.71399999999999997},"last":0.31900000000000001},{"interface":"front","average":{"1min":0.46200000000000002,"5min":0.47599999999999998,"15min":0.495},"min":{"1min":0.33100000000000002,"5min":0.318,"15min":0.27700000000000002},"max":{"1min":0.61799999999999999,"5min":0.80600000000000005,"15min":0.80600000000000005},"last":0.44400000000000001}]},{"osd":3,"last update":"Sun Nov 26 07:24:51 2023","interfaces":[{"interface":"back","average":{"1min":0.48499999999999999,"5min":0.52100000000000002,"15min":0.52500000000000002},"min":{"1min":0.33200000000000002,"5min":0.33200000000000002,"15min":0.33200000000000002},"max":{"1min":0.72699999999999998,"5min":0.748,"15min":0.748},"last":0.40100000000000002},{"interface":"front","average":{"1min":0.47499999999999998,"5min":0.498,"15min":0.496},"min":{"1min":0.38100000000000001,"5min":0.35499999999999998,"15min":0.31},"max":{"1min":0.71299999999999997,"5min":0.71299999999999997,"15min":0.80700000000000005},"last":0.36299999999999999}]},{"osd":4,"last update":"Sun Nov 26 07:24:51 2023","interfaces":[{"interface":"back","average":{"1min":0.72999999999999998,"5min":0.75800000000000001,"15min":0.71499999999999997},"min":{"1min":0.50600000000000001,"5min":0.50600000000000001,"15min":0.41299999999999998},"max":{"1min":0.95699999999999996,"5min":1.319,"15min":1.319},"last":0.51500000000000001},{"interface":"front","average":{"1min":0.71899999999999997,"5min":0.72499999999999998,"15min":0.69799999999999995},"min":{"1min":0.54300000000000004,"5min":0.44500000000000001,"15min":0.44500000000000001},"max":{"1min":0.97999999999999998,"5min":1.006,"15min":1.131},"last":0.67500000000000004}]},{"osd":5,"last update":"Sun Nov 26 07:24:51 2023","interfaces":[{"interface":"back","average":{"1min":0.73299999999999998,"5min":0.747,"15min":0.72899999999999998},"min":{"1min":0.501,"5min":0.47899999999999998,"15min":0.47899999999999998},"max":{"1min":0.95699999999999996,"5min":1.0409999999999999,"15min":1.1919999999999999},"last":0.74299999999999999},{"interface":"front","average":{"1min":0.73499999999999999,"5min":0.75,"15min":0.72899999999999998},"min":{"1min":0.53100000000000003,"5min":0.47099999999999997,"15min":0.47099999999999997},"max":{"1min":0.86299999999999999,"5min":1.0389999999999999,"15min":1.0609999999999999},"last":0.52300000000000002}]},{"osd":6,"last update":"Sun Nov 26 07:24:51 2023","interfaces":[{"interface":"back","average":{"1min":0.72499999999999998,"5min":0.76000000000000001,"15min":0.72399999999999998},"min":{"1min":0.53800000000000003,"5min":0.53200000000000003,"15min":0.50600000000000001},"max":{"1min":0.89600000000000002,"5min":1.2190000000000001,"15min":1.2190000000000001},"last":0.72199999999999998},{"interface":"front","average":{"1min":0.72199999999999998,"5min":0.76200000000000001,"15min":0.72499999999999998},"min":{"1min":0.52200000000000002,"5min":0.46999999999999997,"15min":0.46999999999999997},"max":{"1min":0.91800000000000004,"5min":1.1659999999999999,"15min":1.1659999999999999},"last":0.59999999999999998}]},{"osd":7,"last update":"Sun Nov 26 07:24:51 2023","interfaces":[{"interface":"back","average":{"1min":0.76700000000000002,"5min":0.77400000000000002,"15min":0.73599999999999999},"min":{"1min":0.57099999999999995,"5min":0.50600000000000001,"15min":0.50600000000000001},"max":{"1min":0.97299999999999998,"5min":1.0920000000000001,"15min":1.0920000000000001},"last":0.65500000000000003},{"interface":"front","average":{"1min":0.76200000000000001,"5min":0.76500000000000001,"15min":0.755},"min":{"1min":0.54000000000000004,"5min":0.53700000000000003,"15min":0.48499999999999999},"max":{"1min":1.006,"5min":1.115,"15min":1.115},"last":0.625}]}]},{"osd":0,"up_from":13,"seq":55834574939,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":33056,"kb_used_data":1876,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94338784,"statfs":{"total":96636764160,"available":96602914816,"internally_reserved":0,"allocated":1921024,"data_stored":1170345,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Sun Nov 26 07:24:50 2023","interfaces":[{"interface":"back","average":{"1min":0.439,"5min":0.437,"15min":0.43099999999999999},"min":{"1min":0.25800000000000001,"5min":0.25800000000000001,"15min":0.25800000000000001},"max":{"1min":0.70499999999999996,"5min":1.26,"15min":1.26},"last":0.45100000000000001},{"interface":"front","average":{"1min":0.45100000000000001,"5min":0.47299999999999998,"15min":0.47499999999999998},"min":{"1min":0.29899999999999999,"5min":0.29899999999999999,"15min":0.29899999999999999},"max":{"1min":0.72099999999999997,"5min":1.212,"15min":1.212},"last":0.47199999999999998}]},{"osd":2,"last update":"Sun Nov 26 07:24:50 2023","interfaces":[{"interface":"back","average":{"1min":0.47199999999999998,"5min":0.497,"15min":0.52200000000000002},"min":{"1min":0.33000000000000002,"5min":0.33000000000000002,"15min":0.33000000000000002},"max":{"1min":0.68100000000000005,"5min":0.84199999999999997,"15min":0.84199999999999997},"last":0.52900000000000003},{"interface":"front","average":{"1min":0.53900000000000003,"5min":0.55800000000000005,"15min":0.55900000000000005},"min":{"1min":0.45200000000000001,"5min":0.40100000000000002,"15min":0.36399999999999999},"max":{"1min":0.81000000000000005,"5min":1.0800000000000001,"15min":1.0800000000000001},"last":0.62}]},{"osd":3,"last update":"Sun Nov 26 07:24:50 2023","interfaces":[{"interface":"back","average":{"1min":0.51600000000000001,"5min":0.51300000000000001,"15min":0.52400000000000002},"min":{"1min":0.41099999999999998,"5min":0.33500000000000002,"15min":0.33500000000000002},"max":{"1min":0.72899999999999998,"5min":0.873,"15min":1.3149999999999999},"last":0.60899999999999999},{"interface":"front","average":{"1min":0.53000000000000003,"5min":0.53600000000000003,"15min":0.54900000000000004},"min":{"1min":0.377,"5min":0.35499999999999998,"15min":0.35499999999999998},"max":{"1min":0.74299999999999999,"5min":0.74299999999999999,"15min":1.2509999999999999},"last":0.58099999999999996}]},{"osd":4,"last update":"Sun Nov 26 07:24:50 2023","interfaces":[{"interface":"back","average":{"1min":0.68500000000000005,"5min":0.73899999999999999,"15min":0.749},"min":{"1min":0.40100000000000002,"5min":0.40100000000000002,"15min":0.40100000000000002},"max":{"1min":0.91300000000000003,"5min":1.0900000000000001,"15min":1.2010000000000001},"last":1.0640000000000001},{"interface":"front","average":{"1min":0.76000000000000001,"5min":0.78000000000000003,"15min":0.78600000000000003},"min":{"1min":0.55900000000000005,"5min":0.51100000000000001,"15min":0.51100000000000001},"max":{"1min":1.012,"5min":1.298,"15min":1.381},"last":1.004}]},{"osd":5,"last update":"Sun Nov 26 07:24:50 2023","interfaces":[{"interface":"back","average":{"1min":0.66800000000000004,"5min":0.73599999999999999,"15min":0.75},"min":{"1min":0.50800000000000001,"5min":0.45200000000000001,"15min":0.42199999999999999},"max":{"1min":0.91600000000000004,"5min":1.1819999999999999,"15min":1.1819999999999999},"last":1.0169999999999999},{"interface":"front","average":{"1min":0.77500000000000002,"5min":0.79800000000000004,"15min":0.79700000000000004},"min":{"1min":0.59799999999999998,"5min":0.53000000000000003,"15min":0.53000000000000003},"max":{"1min":1.0609999999999999,"5min":1.329,"15min":1.399},"last":1.038}]},{"osd":6,"last update":"Sun Nov 26 07:24:50 2023","interfaces":[{"interface":"back","average":{"1min":0.70599999999999996,"5min":0.70599999999999996,"15min":0.73699999999999999},"min":{"1min":0.48999999999999999,"5min":0.48499999999999999,"15min":0.48499999999999999},"max":{"1min":0.97199999999999998,"5min":1.0740000000000001,"15min":1.3029999999999999},"last":1.0529999999999999},{"interface":"front","average":{"1min":0.71099999999999997,"5min":0.72899999999999998,"15min":0.754},"min":{"1min":0.52100000000000002,"5min":0.47199999999999998,"15min":0.47199999999999998},"max":{"1min":0.93600000000000005,"5min":1.2170000000000001,"15min":1.3600000000000001},"last":0.92700000000000005}]},{"osd":7,"last update":"Sun Nov 26 07:24:50 2023","interfaces":[{"interface":"back","average":{"1min":0.75600000000000001,"5min":0.76800000000000002,"15min":0.79300000000000004},"min":{"1min":0.60099999999999998,"5min":0.56899999999999995,"15min":0.56899999999999995},"max":{"1min":0.97999999999999998,"5min":1.1719999999999999,"15min":1.177},"last":0.98799999999999999},{"interface":"front","average":{"1min":0.69899999999999995,"5min":0.76500000000000001,"15min":0.77800000000000002},"min":{"1min":0.56899999999999995,"5min":0.47899999999999998,"15min":0.47899999999999998},"max":{"1min":0.92400000000000004,"5min":1.238,"15min":1.2390000000000001},"last":1.097}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-11-26T07:24:57.753 DEBUG:teuthology.orchestra.run.smithi143:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-11-26T07:24:57.890 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:57.890+0000 7fb433d01640 1 Processor -- start 2023-11-26T07:24:57.891 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:57.890+0000 7fb433d01640 1 -- start start 2023-11-26T07:24:57.892 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:57.890+0000 7fb433d01640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fb42c0cf280 0x7fb42c0cf6e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:24:57.892 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:57.890+0000 7fb433d01640 1 -- --> v1:172.21.15.143:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb42c0cfc20 con 0x7fb42c143ac0 2023-11-26T07:24:57.892 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:57.890+0000 7fb433d01640 1 -- --> v1:172.21.15.188:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb42c0cc730 con 0x7fb42c0cd1b0 2023-11-26T07:24:57.892 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:57.890+0000 7fb433d01640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb42c143ea0 con 0x7fb42c0cf280 2023-11-26T07:24:57.893 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:57.890+0000 7fb431a9d640 1 --1- >> v1:172.21.15.143:6789/0 conn(0x7fb42c143ac0 0x7fb42c070370 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.143:6789/0 says I am v1:172.21.15.143:36286/0 (socket says 172.21.15.143:36286) 2023-11-26T07:24:57.893 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:57.890+0000 7fb431a9d640 1 -- 172.21.15.143:0/4112698687 learned_addr learned my addr 172.21.15.143:0/4112698687 (peer_addr_for_me v1:172.21.15.143:0/0) 2023-11-26T07:24:57.894 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:57.890+0000 7fb43129c640 1 --2- 172.21.15.143:0/4112698687 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fb42c0cf280 0x7fb42c0cf6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:24:57.894 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:57.890+0000 7fb430a9b640 1 -- 172.21.15.143:0/4112698687 <== mon.0 v1:172.21.15.143:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3437401927 0 0) 0x7fb42c0cfc20 con 0x7fb42c143ac0 2023-11-26T07:24:57.894 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:57.894+0000 7fb430a9b640 1 -- 172.21.15.143:0/4112698687 --> v1:172.21.15.143:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fb418005740 con 0x7fb42c143ac0 2023-11-26T07:24:57.894 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:57.894+0000 7fb430a9b640 1 -- 172.21.15.143:0/4112698687 <== mon.1 v1:172.21.15.188:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1045542085 0 0) 0x7fb42c0cc730 con 0x7fb42c0cd1b0 2023-11-26T07:24:57.894 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:57.894+0000 7fb43129c640 1 -- 172.21.15.143:0/4112698687 >> v1:172.21.15.188:6789/0 conn(0x7fb42c0cd1b0 legacy=0x7fb42c0cd5e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:24:57.894 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:57.894+0000 7fb43129c640 1 -- 172.21.15.143:0/4112698687 >> v1:172.21.15.143:6789/0 conn(0x7fb42c143ac0 legacy=0x7fb42c070370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:24:57.894 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:57.894+0000 7fb43129c640 1 -- 172.21.15.143:0/4112698687 --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb42c070ee0 con 0x7fb42c0cf280 2023-11-26T07:24:57.896 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:57.894+0000 7fb43129c640 1 --2- 172.21.15.143:0/4112698687 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fb42c0cf280 0x7fb42c0cf6e0 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7fb41c0097f0 tx=0x7fb41c01c300 comp rx=0 tx=0).ready entity=mon.2 client_cookie=3ba72052c3da917b server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:24:57.897 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:57.894+0000 7fb430a9b640 1 -- 172.21.15.143:0/4112698687 <== mon.2 v2:172.21.15.188:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb428001680 con 0x7fb42c0cf280 2023-11-26T07:24:57.897 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:57.894+0000 7fb430a9b640 1 -- 172.21.15.143:0/4112698687 <== mon.2 v2:172.21.15.188:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb41c0a7050 con 0x7fb42c0cf280 2023-11-26T07:24:57.897 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:57.894+0000 7fb430a9b640 1 -- 172.21.15.143:0/4112698687 <== mon.2 v2:172.21.15.188:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb41c0ab600 con 0x7fb42c0cf280 2023-11-26T07:24:57.897 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:57.898+0000 7fb433d01640 1 -- 172.21.15.143:0/4112698687 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fb42c0cf280 msgr2=0x7fb42c0cf6e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:24:57.898 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:57.898+0000 7fb433d01640 1 --2- 172.21.15.143:0/4112698687 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fb42c0cf280 0x7fb42c0cf6e0 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7fb41c0097f0 tx=0x7fb41c01c300 comp rx=0 tx=0).stop 2023-11-26T07:24:57.898 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:57.898+0000 7fb433d01640 1 -- 172.21.15.143:0/4112698687 shutdown_connections 2023-11-26T07:24:57.898 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:57.898+0000 7fb433d01640 1 --2- 172.21.15.143:0/4112698687 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fb42c0cf280 0x7fb42c0cf6e0 unknown :-1 s=CLOSED pgs=169 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:24:57.898 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:57.898+0000 7fb433d01640 1 -- 172.21.15.143:0/4112698687 >> 172.21.15.143:0/4112698687 conn(0x7fb42c06a890 msgr2=0x7fb42c06aca0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:24:57.898 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:57.898+0000 7fb433d01640 1 -- 172.21.15.143:0/4112698687 shutdown_connections 2023-11-26T07:24:57.898 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:57.898+0000 7fb433d01640 1 -- 172.21.15.143:0/4112698687 wait complete. 2023-11-26T07:24:57.899 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:57.898+0000 7fb433d01640 1 Processor -- start 2023-11-26T07:24:57.899 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:57.898+0000 7fb433d01640 1 -- start start 2023-11-26T07:24:57.900 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:57.898+0000 7fb433d01640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fb42c0cd1b0 0x7fb42c13b600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:24:57.901 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:57.898+0000 7fb433d01640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fb42c0cf280 0x7fb42c13bce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:24:57.901 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:57.898+0000 7fb433d01640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fb42c143ac0 0x7fb42c13c220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:24:57.901 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:57.898+0000 7fb433d01640 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb42c1596e0 con 0x7fb42c0cf280 2023-11-26T07:24:57.901 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:57.898+0000 7fb433d01640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb42c13bb40 con 0x7fb42c0cd1b0 2023-11-26T07:24:57.901 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:57.898+0000 7fb433d01640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb42c13cdd0 con 0x7fb42c143ac0 2023-11-26T07:24:57.901 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:57.898+0000 7fb43129c640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fb42c0cf280 0x7fb42c13bce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:24:57.902 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:57.898+0000 7fb43129c640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fb42c0cf280 0x7fb42c13bce0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:45906/0 (socket says 172.21.15.143:45906) 2023-11-26T07:24:57.902 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:57.898+0000 7fb43129c640 1 -- 172.21.15.143:0/2560848173 learned_addr learned my addr 172.21.15.143:0/2560848173 (peer_addr_for_me v2:172.21.15.143:0/0) 2023-11-26T07:24:57.902 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:57.898+0000 7fb431a9d640 1 --2- 172.21.15.143:0/2560848173 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fb42c0cd1b0 0x7fb42c13b600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:24:57.902 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:57.898+0000 7fb43229e640 1 --2- 172.21.15.143:0/2560848173 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fb42c143ac0 0x7fb42c13c220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:24:57.902 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:57.898+0000 7fb43129c640 1 -- 172.21.15.143:0/2560848173 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fb42c143ac0 msgr2=0x7fb42c13c220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:24:57.902 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:57.898+0000 7fb43129c640 1 --2- 172.21.15.143:0/2560848173 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fb42c143ac0 0x7fb42c13c220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:24:57.902 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:57.898+0000 7fb43129c640 1 -- 172.21.15.143:0/2560848173 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fb42c0cd1b0 msgr2=0x7fb42c13b600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:24:57.902 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:57.898+0000 7fb43129c640 1 --2- 172.21.15.143:0/2560848173 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fb42c0cd1b0 0x7fb42c13b600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:24:57.903 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:57.898+0000 7fb43129c640 1 -- 172.21.15.143:0/2560848173 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb41c009420 con 0x7fb42c0cf280 2023-11-26T07:24:57.904 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:57.902+0000 7fb43129c640 1 --2- 172.21.15.143:0/2560848173 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fb42c0cf280 0x7fb42c13bce0 secure :-1 s=READY pgs=1452 cs=0 l=1 rev1=1 crypto rx=0x7fb41c000c10 tx=0x7fb41c0190a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:24:57.904 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:57.902+0000 7fb4227fc640 1 -- 172.21.15.143:0/2560848173 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb41c0b4400 con 0x7fb42c0cf280 2023-11-26T07:24:57.904 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:57.902+0000 7fb4227fc640 1 -- 172.21.15.143:0/2560848173 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb41c0b4560 con 0x7fb42c0cf280 2023-11-26T07:24:57.904 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:57.902+0000 7fb4227fc640 1 -- 172.21.15.143:0/2560848173 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb41c0b48d0 con 0x7fb42c0cf280 2023-11-26T07:24:57.904 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:57.902+0000 7fb433d01640 1 -- 172.21.15.143:0/2560848173 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb42c13caf0 con 0x7fb42c0cf280 2023-11-26T07:24:57.904 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:57.902+0000 7fb433d01640 1 -- 172.21.15.143:0/2560848173 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb42c159300 con 0x7fb42c0cf280 2023-11-26T07:24:57.904 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:57.902+0000 7fb433d01640 1 -- 172.21.15.143:0/2560848173 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb42c067680 con 0x7fb42c0cf280 2023-11-26T07:24:57.905 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:57.902+0000 7fb4227fc640 1 -- 172.21.15.143:0/2560848173 <== mon.0 v2:172.21.15.143:3300/0 4 ==== mgrmap(e 18) v1 ==== 86107+0+0 (secure 0 0 0) 0x7fb41c0b2040 con 0x7fb42c0cf280 2023-11-26T07:24:57.905 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:57.902+0000 7fb4227fc640 1 --2- 172.21.15.143:0/2560848173 >> [v2:172.21.15.188:6836/3401128553,v1:172.21.15.188:6837/3401128553] conn(0x7fb410066740 0x7fb410068c00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:24:57.906 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:57.902+0000 7fb4227fc640 1 -- 172.21.15.143:0/2560848173 <== mon.0 v2:172.21.15.143:3300/0 5 ==== osd_map(106..106 src has 1..106) v4 ==== 4943+0+0 (secure 0 0 0) 0x7fb41c11ddd0 con 0x7fb42c0cf280 2023-11-26T07:24:57.906 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:57.902+0000 7fb431a9d640 1 --2- 172.21.15.143:0/2560848173 >> [v2:172.21.15.188:6836/3401128553,v1:172.21.15.188:6837/3401128553] conn(0x7fb410066740 0x7fb410068c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:24:57.906 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:57.906+0000 7fb431a9d640 1 --2- 172.21.15.143:0/2560848173 >> [v2:172.21.15.188:6836/3401128553,v1:172.21.15.188:6837/3401128553] conn(0x7fb410066740 0x7fb410068c00 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7fb428001d10 tx=0x7fb428001ca0 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:24:57.908 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:57.906+0000 7fb4227fc640 1 -- 172.21.15.143:0/2560848173 <== mon.0 v2:172.21.15.143:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172649 (secure 0 0 0) 0x7fb41c0eb160 con 0x7fb42c0cf280 2023-11-26T07:24:58.058 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:58.058+0000 7fb433d01640 1 -- 172.21.15.143:0/2560848173 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fb42c155150 con 0x7fb42c0cf280 2023-11-26T07:24:58.059 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:58.058+0000 7fb4227fc640 1 -- 172.21.15.143:0/2560848173 <== mon.0 v2:172.21.15.143:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v106) v1 ==== 74+0+13034 (secure 0 0 0) 0x7fb41c0b0020 con 0x7fb42c0cf280 2023-11-26T07:24:58.060 INFO:teuthology.orchestra.run.smithi143.stdout: 2023-11-26T07:24:58.060 INFO:teuthology.orchestra.run.smithi143.stdout:{"epoch":106,"fsid":"f5033060-2469-4a07-95fe-2b76f5404bc1","created":"2023-11-26T07:17:28.613663+0000","modified":"2023-11-26T07:24:54.689681+0000","last_up_change":"2023-11-26T07:17:41.157813+0000","last_in_change":"2023-11-26T07:17:31.181658+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":17,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-11-26T07:17:41.758597+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"ebbee2c6-023e-44b9-9f1d-6f7bfd309730","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":101,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6817","nonce":639807369},{"type":"v1","addr":"172.21.15.143:6819","nonce":639807369}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6821","nonce":639807369},{"type":"v1","addr":"172.21.15.143:6823","nonce":639807369}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6830","nonce":639807369},{"type":"v1","addr":"172.21.15.143:6831","nonce":639807369}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6825","nonce":639807369},{"type":"v1","addr":"172.21.15.143:6828","nonce":639807369}]},"public_addr":"172.21.15.143:6819/639807369","cluster_addr":"172.21.15.143:6823/639807369","heartbeat_back_addr":"172.21.15.143:6831/639807369","heartbeat_front_addr":"172.21.15.143:6828/639807369","state":["exists","up"]},{"osd":1,"uuid":"fec4b8be-7122-49e9-9a19-5850a0700001","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":101,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6816","nonce":1144942530},{"type":"v1","addr":"172.21.15.143:6818","nonce":1144942530}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6820","nonce":1144942530},{"type":"v1","addr":"172.21.15.143:6822","nonce":1144942530}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6827","nonce":1144942530},{"type":"v1","addr":"172.21.15.143:6829","nonce":1144942530}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6824","nonce":1144942530},{"type":"v1","addr":"172.21.15.143:6826","nonce":1144942530}]},"public_addr":"172.21.15.143:6818/1144942530","cluster_addr":"172.21.15.143:6822/1144942530","heartbeat_back_addr":"172.21.15.143:6829/1144942530","heartbeat_front_addr":"172.21.15.143:6826/1144942530","state":["exists","up"]},{"osd":2,"uuid":"2888af5b-f933-46ca-8f95-6dd494ff43f6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":101,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6808","nonce":3909095186},{"type":"v1","addr":"172.21.15.143:6809","nonce":3909095186}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6810","nonce":3909095186},{"type":"v1","addr":"172.21.15.143:6811","nonce":3909095186}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6814","nonce":3909095186},{"type":"v1","addr":"172.21.15.143:6815","nonce":3909095186}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6812","nonce":3909095186},{"type":"v1","addr":"172.21.15.143:6813","nonce":3909095186}]},"public_addr":"172.21.15.143:6809/3909095186","cluster_addr":"172.21.15.143:6811/3909095186","heartbeat_back_addr":"172.21.15.143:6815/3909095186","heartbeat_front_addr":"172.21.15.143:6813/3909095186","state":["exists","up"]},{"osd":3,"uuid":"d1f52435-7bc1-4081-9550-9cd1ba55f549","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":101,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6800","nonce":3854210991},{"type":"v1","addr":"172.21.15.143:6801","nonce":3854210991}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6802","nonce":3854210991},{"type":"v1","addr":"172.21.15.143:6803","nonce":3854210991}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6806","nonce":3854210991},{"type":"v1","addr":"172.21.15.143:6807","nonce":3854210991}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6804","nonce":3854210991},{"type":"v1","addr":"172.21.15.143:6805","nonce":3854210991}]},"public_addr":"172.21.15.143:6801/3854210991","cluster_addr":"172.21.15.143:6803/3854210991","heartbeat_back_addr":"172.21.15.143:6807/3854210991","heartbeat_front_addr":"172.21.15.143:6805/3854210991","state":["exists","up"]},{"osd":4,"uuid":"e2ae8b6e-6e25-49b2-bcbf-aacce68f06a3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":101,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6800","nonce":3924606902},{"type":"v1","addr":"172.21.15.188:6801","nonce":3924606902}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6802","nonce":3924606902},{"type":"v1","addr":"172.21.15.188:6803","nonce":3924606902}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6806","nonce":3924606902},{"type":"v1","addr":"172.21.15.188:6807","nonce":3924606902}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6804","nonce":3924606902},{"type":"v1","addr":"172.21.15.188:6805","nonce":3924606902}]},"public_addr":"172.21.15.188:6801/3924606902","cluster_addr":"172.21.15.188:6803/3924606902","heartbeat_back_addr":"172.21.15.188:6807/3924606902","heartbeat_front_addr":"172.21.15.188:6805/3924606902","state":["exists","up"]},{"osd":5,"uuid":"3cec8a32-7698-4c36-8e89-e4ba11934f33","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":101,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6813","nonce":186841544},{"type":"v1","addr":"172.21.15.188:6815","nonce":186841544}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6817","nonce":186841544},{"type":"v1","addr":"172.21.15.188:6818","nonce":186841544}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6822","nonce":186841544},{"type":"v1","addr":"172.21.15.188:6823","nonce":186841544}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6820","nonce":186841544},{"type":"v1","addr":"172.21.15.188:6821","nonce":186841544}]},"public_addr":"172.21.15.188:6815/186841544","cluster_addr":"172.21.15.188:6818/186841544","heartbeat_back_addr":"172.21.15.188:6823/186841544","heartbeat_front_addr":"172.21.15.188:6821/186841544","state":["exists","up"]},{"osd":6,"uuid":"58a65d5d-e090-473a-81b4-ad38e85029f9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":101,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6808","nonce":2153499405},{"type":"v1","addr":"172.21.15.188:6809","nonce":2153499405}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6810","nonce":2153499405},{"type":"v1","addr":"172.21.15.188:6811","nonce":2153499405}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6816","nonce":2153499405},{"type":"v1","addr":"172.21.15.188:6819","nonce":2153499405}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6812","nonce":2153499405},{"type":"v1","addr":"172.21.15.188:6814","nonce":2153499405}]},"public_addr":"172.21.15.188:6809/2153499405","cluster_addr":"172.21.15.188:6811/2153499405","heartbeat_back_addr":"172.21.15.188:6819/2153499405","heartbeat_front_addr":"172.21.15.188:6814/2153499405","state":["exists","up"]},{"osd":7,"uuid":"ea5b7f70-3554-4cf8-ac30-f7c59b80678d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":101,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6824","nonce":2722096729},{"type":"v1","addr":"172.21.15.188:6825","nonce":2722096729}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6826","nonce":2722096729},{"type":"v1","addr":"172.21.15.188:6827","nonce":2722096729}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6830","nonce":2722096729},{"type":"v1","addr":"172.21.15.188:6831","nonce":2722096729}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6828","nonce":2722096729},{"type":"v1","addr":"172.21.15.188:6829","nonce":2722096729}]},"public_addr":"172.21.15.188:6825/2722096729","cluster_addr":"172.21.15.188:6827/2722096729","heartbeat_back_addr":"172.21.15.188:6831/2722096729","heartbeat_front_addr":"172.21.15.188:6829/2722096729","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-26T07:17:36.995032+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-26T07:17:36.932266+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-26T07:17:36.829304+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-26T07:17:36.804297+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-26T07:17:36.667892+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-26T07:17:36.964868+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-26T07:17:36.827332+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-26T07:17:36.893179+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.143:6835/2571525464":"2023-11-27T07:24:51.350494+0000","172.21.15.143:6833/2571525464":"2023-11-27T07:24:51.350494+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-11-26T07:24:58.061 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:58.062+0000 7fb433d01640 1 -- 172.21.15.143:0/2560848173 >> [v2:172.21.15.188:6836/3401128553,v1:172.21.15.188:6837/3401128553] conn(0x7fb410066740 msgr2=0x7fb410068c00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:24:58.062 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:58.062+0000 7fb433d01640 1 --2- 172.21.15.143:0/2560848173 >> [v2:172.21.15.188:6836/3401128553,v1:172.21.15.188:6837/3401128553] conn(0x7fb410066740 0x7fb410068c00 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7fb428001d10 tx=0x7fb428001ca0 comp rx=0 tx=0).stop 2023-11-26T07:24:58.062 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:58.062+0000 7fb433d01640 1 -- 172.21.15.143:0/2560848173 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fb42c0cf280 msgr2=0x7fb42c13bce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:24:58.062 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:58.062+0000 7fb433d01640 1 --2- 172.21.15.143:0/2560848173 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fb42c0cf280 0x7fb42c13bce0 secure :-1 s=READY pgs=1452 cs=0 l=1 rev1=1 crypto rx=0x7fb41c000c10 tx=0x7fb41c0190a0 comp rx=0 tx=0).stop 2023-11-26T07:24:58.062 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:58.062+0000 7fb433d01640 1 -- 172.21.15.143:0/2560848173 shutdown_connections 2023-11-26T07:24:58.062 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:58.062+0000 7fb433d01640 1 --2- 172.21.15.143:0/2560848173 >> [v2:172.21.15.188:6836/3401128553,v1:172.21.15.188:6837/3401128553] conn(0x7fb410066740 0x7fb410068c00 unknown :-1 s=CLOSED pgs=186 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:24:58.062 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:58.062+0000 7fb433d01640 1 --2- 172.21.15.143:0/2560848173 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fb42c143ac0 0x7fb42c13c220 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:24:58.062 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:58.062+0000 7fb433d01640 1 --2- 172.21.15.143:0/2560848173 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fb42c0cf280 0x7fb42c13bce0 unknown :-1 s=CLOSED pgs=1452 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:24:58.063 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:58.062+0000 7fb433d01640 1 --2- 172.21.15.143:0/2560848173 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fb42c0cd1b0 0x7fb42c13b600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:24:58.063 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:58.062+0000 7fb433d01640 1 -- 172.21.15.143:0/2560848173 >> 172.21.15.143:0/2560848173 conn(0x7fb42c06a890 msgr2=0x7fb42c140c60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:24:58.063 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:58.062+0000 7fb433d01640 1 -- 172.21.15.143:0/2560848173 shutdown_connections 2023-11-26T07:24:58.063 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:58.062+0000 7fb433d01640 1 -- 172.21.15.143:0/2560848173 wait complete. 2023-11-26T07:24:59.082 INFO:tasks.ceph:Scrubbing osd.0 2023-11-26T07:24:59.082 DEBUG:teuthology.orchestra.run.smithi143:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.0 config set osd_debug_deep_scrub_sleep 0 2023-11-26T07:24:59.227 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:59.226+0000 7ff5a1b2a640 1 Processor -- start 2023-11-26T07:24:59.228 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:59.226+0000 7ff5a1b2a640 1 -- start start 2023-11-26T07:24:59.229 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:59.226+0000 7ff5a1b2a640 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7ff59c1596d0 0x7ff59c071850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:24:59.229 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:59.226+0000 7ff5a1b2a640 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7ff59c0b2f80 0x7ff59c06cbe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:24:59.230 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:59.226+0000 7ff5a1b2a640 1 -- --> v1:172.21.15.188:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff59c06c750 con 0x7ff59c0ce680 2023-11-26T07:24:59.230 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:59.226+0000 7ff5a1b2a640 1 -- --> v2:172.21.15.143:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff59c0cbe00 con 0x7ff59c0b2f80 2023-11-26T07:24:59.230 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:59.226+0000 7ff5a1b2a640 1 -- --> v2:172.21.15.188:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff59c0b3550 con 0x7ff59c1596d0 2023-11-26T07:24:59.230 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:59.226+0000 7ff59bfff640 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7ff59c0b2f80 0x7ff59c06cbe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:24:59.230 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:59.226+0000 7ff59bfff640 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7ff59c0b2f80 0x7ff59c06cbe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:45910/0 (socket says 172.21.15.143:45910) 2023-11-26T07:24:59.230 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:59.226+0000 7ff59bfff640 1 -- 172.21.15.143:0/1507403430 learned_addr learned my addr 172.21.15.143:0/1507403430 (peer_addr_for_me v2:172.21.15.143:0/0) 2023-11-26T07:24:59.230 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:59.230+0000 7ff59b7fe640 1 --2- 172.21.15.143:0/1507403430 >> v2:172.21.15.188:3300/0 conn(0x7ff59c1596d0 0x7ff59c071850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:24:59.231 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:59.230+0000 7ff59bfff640 1 -- 172.21.15.143:0/1507403430 >> v2:172.21.15.188:3300/0 conn(0x7ff59c1596d0 msgr2=0x7ff59c071850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-26T07:24:59.231 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:59.230+0000 7ff59bfff640 1 --2- 172.21.15.143:0/1507403430 >> v2:172.21.15.188:3300/0 conn(0x7ff59c1596d0 0x7ff59c071850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:24:59.231 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:59.230+0000 7ff59bfff640 1 -- 172.21.15.143:0/1507403430 >> v1:172.21.15.188:6789/0 conn(0x7ff59c0ce680 legacy=0x7ff59c0b2870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-26T07:24:59.231 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:59.230+0000 7ff59bfff640 1 -- 172.21.15.143:0/1507403430 --> v2:172.21.15.143:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff59c071e80 con 0x7ff59c0b2f80 2023-11-26T07:24:59.232 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:59.230+0000 7ff59bfff640 1 --2- 172.21.15.143:0/1507403430 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7ff59c0b2f80 0x7ff59c06cbe0 secure :-1 s=READY pgs=1453 cs=0 l=1 rev1=1 crypto rx=0x7ff59c06e4d0 tx=0x7ff59401c990 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e7ba9b95f9927b31 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:24:59.232 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:59.230+0000 7ff59a7fc640 1 -- 172.21.15.143:0/1507403430 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff5940a55b0 con 0x7ff59c0b2f80 2023-11-26T07:24:59.233 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:59.230+0000 7ff59a7fc640 1 -- 172.21.15.143:0/1507403430 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff5940a5710 con 0x7ff59c0b2f80 2023-11-26T07:24:59.233 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:59.230+0000 7ff59a7fc640 1 -- 172.21.15.143:0/1507403430 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff5940a5aa0 con 0x7ff59c0b2f80 2023-11-26T07:24:59.233 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:59.230+0000 7ff5a1b2a640 1 -- 172.21.15.143:0/1507403430 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7ff59c0b2f80 msgr2=0x7ff59c06cbe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:24:59.234 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:59.230+0000 7ff5a1b2a640 1 --2- 172.21.15.143:0/1507403430 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7ff59c0b2f80 0x7ff59c06cbe0 secure :-1 s=READY pgs=1453 cs=0 l=1 rev1=1 crypto rx=0x7ff59c06e4d0 tx=0x7ff59401c990 comp rx=0 tx=0).stop 2023-11-26T07:24:59.234 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:59.230+0000 7ff5a1b2a640 1 -- 172.21.15.143:0/1507403430 shutdown_connections 2023-11-26T07:24:59.234 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:59.230+0000 7ff5a1b2a640 1 --2- 172.21.15.143:0/1507403430 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7ff59c0b2f80 0x7ff59c06cbe0 unknown :-1 s=CLOSED pgs=1453 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:24:59.234 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:59.230+0000 7ff5a1b2a640 1 --2- 172.21.15.143:0/1507403430 >> v2:172.21.15.188:3300/0 conn(0x7ff59c1596d0 0x7ff59c071850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:24:59.234 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:59.230+0000 7ff5a1b2a640 1 -- 172.21.15.143:0/1507403430 >> 172.21.15.143:0/1507403430 conn(0x7ff59c06a890 msgr2=0x7ff59c06aca0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:24:59.235 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:59.234+0000 7ff5a1b2a640 1 -- 172.21.15.143:0/1507403430 shutdown_connections 2023-11-26T07:24:59.235 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:59.234+0000 7ff5a1b2a640 1 -- 172.21.15.143:0/1507403430 wait complete. 2023-11-26T07:24:59.235 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:59.234+0000 7ff5a1b2a640 1 Processor -- start 2023-11-26T07:24:59.235 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:59.234+0000 7ff5a1b2a640 1 -- start start 2023-11-26T07:24:59.238 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:59.234+0000 7ff5a1b2a640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7ff59c0b2f80 0x7ff59c1f9be0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:24:59.238 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:59.234+0000 7ff5a1b2a640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7ff59c0ce680 0x7ff59c14bc70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:24:59.238 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:59.234+0000 7ff5a1b2a640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7ff59c1596d0 0x7ff59c0aeba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:24:59.238 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:59.234+0000 7ff5a1b2a640 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff59c1fa120 con 0x7ff59c0b2f80 2023-11-26T07:24:59.238 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:59.234+0000 7ff5a1b2a640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff59c13e720 con 0x7ff59c0ce680 2023-11-26T07:24:59.238 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:59.234+0000 7ff5a1b2a640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff59c166770 con 0x7ff59c1596d0 2023-11-26T07:24:59.239 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:59.234+0000 7ff59b7fe640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7ff59c0b2f80 0x7ff59c1f9be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:24:59.239 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:59.234+0000 7ff59b7fe640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7ff59c0b2f80 0x7ff59c1f9be0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:45924/0 (socket says 172.21.15.143:45924) 2023-11-26T07:24:59.239 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:59.234+0000 7ff59b7fe640 1 -- 172.21.15.143:0/2871953472 learned_addr learned my addr 172.21.15.143:0/2871953472 (peer_addr_for_me v2:172.21.15.143:0/0) 2023-11-26T07:24:59.239 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:59.234+0000 7ff59affd640 1 --2- 172.21.15.143:0/2871953472 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7ff59c0ce680 0x7ff59c14bc70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:24:59.239 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:59.234+0000 7ff59bfff640 1 --2- 172.21.15.143:0/2871953472 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7ff59c1596d0 0x7ff59c0aeba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:24:59.239 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:59.234+0000 7ff59b7fe640 1 -- 172.21.15.143:0/2871953472 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7ff59c1596d0 msgr2=0x7ff59c0aeba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:24:59.239 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:59.234+0000 7ff59b7fe640 1 --2- 172.21.15.143:0/2871953472 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7ff59c1596d0 0x7ff59c0aeba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:24:59.240 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:59.234+0000 7ff59b7fe640 1 -- 172.21.15.143:0/2871953472 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7ff59c0ce680 msgr2=0x7ff59c14bc70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:24:59.240 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:59.234+0000 7ff59b7fe640 1 --2- 172.21.15.143:0/2871953472 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7ff59c0ce680 0x7ff59c14bc70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:24:59.240 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:59.234+0000 7ff59b7fe640 1 -- 172.21.15.143:0/2871953472 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff594009b30 con 0x7ff59c0b2f80 2023-11-26T07:24:59.240 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:59.238+0000 7ff59b7fe640 1 --2- 172.21.15.143:0/2871953472 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7ff59c0b2f80 0x7ff59c1f9be0 secure :-1 s=READY pgs=1454 cs=0 l=1 rev1=1 crypto rx=0x7ff58c00d5f0 tx=0x7ff58c00dab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:24:59.240 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:59.238+0000 7ff583fff640 1 -- 172.21.15.143:0/2871953472 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff58c0125f0 con 0x7ff59c0b2f80 2023-11-26T07:24:59.240 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:59.238+0000 7ff583fff640 1 -- 172.21.15.143:0/2871953472 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff58c012750 con 0x7ff59c0b2f80 2023-11-26T07:24:59.240 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:59.238+0000 7ff583fff640 1 -- 172.21.15.143:0/2871953472 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff58c012ae0 con 0x7ff59c0b2f80 2023-11-26T07:24:59.240 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:59.238+0000 7ff5a1b2a640 1 -- 172.21.15.143:0/2871953472 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff59c14c780 con 0x7ff59c0b2f80 2023-11-26T07:24:59.241 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:59.238+0000 7ff5a1b2a640 1 -- 172.21.15.143:0/2871953472 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff59c166a30 con 0x7ff59c0b2f80 2023-11-26T07:24:59.242 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:59.238+0000 7ff5a1b2a640 1 -- 172.21.15.143:0/2871953472 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7ff59c067680 con 0x7ff59c0b2f80 2023-11-26T07:24:59.242 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:59.238+0000 7ff583fff640 1 -- 172.21.15.143:0/2871953472 <== mon.0 v2:172.21.15.143:3300/0 4 ==== mgrmap(e 18) v1 ==== 86107+0+0 (secure 0 0 0) 0x7ff58c012c40 con 0x7ff59c0b2f80 2023-11-26T07:24:59.242 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:59.238+0000 7ff583fff640 1 --2- 172.21.15.143:0/2871953472 >> [v2:172.21.15.188:6836/3401128553,v1:172.21.15.188:6837/3401128553] conn(0x7ff588066740 0x7ff588068c00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:24:59.242 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:59.238+0000 7ff583fff640 1 -- 172.21.15.143:0/2871953472 <== mon.0 v2:172.21.15.143:3300/0 5 ==== osd_map(106..106 src has 1..106) v4 ==== 4943+0+0 (secure 0 0 0) 0x7ff58c084300 con 0x7ff59c0b2f80 2023-11-26T07:24:59.242 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:59.238+0000 7ff583fff640 1 --2- 172.21.15.143:0/2871953472 >> [v2:172.21.15.143:6817/639807369,v1:172.21.15.143:6819/639807369] conn(0x7ff58806c5a0 0x7ff58806e9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:24:59.242 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:59.238+0000 7ff583fff640 1 -- 172.21.15.143:0/2871953472 --> [v2:172.21.15.143:6817/639807369,v1:172.21.15.143:6819/639807369] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7ff58806f070 con 0x7ff58806c5a0 2023-11-26T07:24:59.243 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:59.238+0000 7ff583fff640 1 -- 172.21.15.143:0/2871953472 <== mon.0 v2:172.21.15.143:3300/0 6 ==== mon_get_version_reply(handle=1 version=106) v2 ==== 24+0+0 (secure 0 0 0) 0x7ff58c0868e0 con 0x7ff59c0b2f80 2023-11-26T07:24:59.243 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:59.238+0000 7ff59affd640 1 --2- 172.21.15.143:0/2871953472 >> [v2:172.21.15.188:6836/3401128553,v1:172.21.15.188:6837/3401128553] conn(0x7ff588066740 0x7ff588068c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:24:59.243 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:59.238+0000 7ff59bfff640 1 --2- 172.21.15.143:0/2871953472 >> [v2:172.21.15.143:6817/639807369,v1:172.21.15.143:6819/639807369] conn(0x7ff58806c5a0 0x7ff58806e9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:24:59.243 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:59.242+0000 7ff59affd640 1 --2- 172.21.15.143:0/2871953472 >> [v2:172.21.15.188:6836/3401128553,v1:172.21.15.188:6837/3401128553] conn(0x7ff588066740 0x7ff588068c00 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7ff59c06e660 tx=0x7ff590007e40 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:24:59.243 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:59.242+0000 7ff59bfff640 1 --2- 172.21.15.143:0/2871953472 >> [v2:172.21.15.143:6817/639807369,v1:172.21.15.143:6819/639807369] conn(0x7ff58806c5a0 0x7ff58806e9c0 crc :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:24:59.244 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:59.242+0000 7ff583fff640 1 -- 172.21.15.143:0/2871953472 <== osd.0 v2:172.21.15.143:6817/639807369 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+26965 (crc 0 0 0) 0x7ff58806f070 con 0x7ff58806c5a0 2023-11-26T07:24:59.255 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:59.254+0000 7ff5a1b2a640 1 -- 172.21.15.143:0/2871953472 --> [v2:172.21.15.143:6817/639807369,v1:172.21.15.143:6819/639807369] -- command(tid 2: {"prefix": "config set", "var": "osd_debug_deep_scrub_sleep", "val": ["0"]}) v1 -- 0x7ff59c0709f0 con 0x7ff58806c5a0 2023-11-26T07:24:59.255 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:59.254+0000 7ff583fff640 1 -- 172.21.15.143:0/2871953472 <== osd.0 v2:172.21.15.143:6817/639807369 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+97 (crc 0 0 0) 0x7ff59c0709f0 con 0x7ff58806c5a0 2023-11-26T07:24:59.255 INFO:teuthology.orchestra.run.smithi143.stdout:{ 2023-11-26T07:24:59.256 INFO:teuthology.orchestra.run.smithi143.stdout: "success": "osd_debug_deep_scrub_sleep = '' (not observed, change may require restart) " 2023-11-26T07:24:59.256 INFO:teuthology.orchestra.run.smithi143.stdout:} 2023-11-26T07:24:59.257 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:59.254+0000 7ff5a1b2a640 1 -- 172.21.15.143:0/2871953472 >> [v2:172.21.15.143:6817/639807369,v1:172.21.15.143:6819/639807369] conn(0x7ff58806c5a0 msgr2=0x7ff58806e9c0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:24:59.257 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:59.254+0000 7ff5a1b2a640 1 --2- 172.21.15.143:0/2871953472 >> [v2:172.21.15.143:6817/639807369,v1:172.21.15.143:6819/639807369] conn(0x7ff58806c5a0 0x7ff58806e9c0 crc :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:24:59.257 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:59.254+0000 7ff5a1b2a640 1 -- 172.21.15.143:0/2871953472 >> [v2:172.21.15.188:6836/3401128553,v1:172.21.15.188:6837/3401128553] conn(0x7ff588066740 msgr2=0x7ff588068c00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:24:59.257 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:59.254+0000 7ff5a1b2a640 1 --2- 172.21.15.143:0/2871953472 >> [v2:172.21.15.188:6836/3401128553,v1:172.21.15.188:6837/3401128553] conn(0x7ff588066740 0x7ff588068c00 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7ff59c06e660 tx=0x7ff590007e40 comp rx=0 tx=0).stop 2023-11-26T07:24:59.257 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:59.254+0000 7ff5a1b2a640 1 -- 172.21.15.143:0/2871953472 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7ff59c0b2f80 msgr2=0x7ff59c1f9be0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:24:59.257 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:59.254+0000 7ff5a1b2a640 1 --2- 172.21.15.143:0/2871953472 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7ff59c0b2f80 0x7ff59c1f9be0 secure :-1 s=READY pgs=1454 cs=0 l=1 rev1=1 crypto rx=0x7ff58c00d5f0 tx=0x7ff58c00dab0 comp rx=0 tx=0).stop 2023-11-26T07:24:59.258 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:59.254+0000 7ff59bfff640 1 -- 172.21.15.143:0/2871953472 reap_dead start 2023-11-26T07:24:59.258 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:59.254+0000 7ff5a1b2a640 1 -- 172.21.15.143:0/2871953472 shutdown_connections 2023-11-26T07:24:59.258 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:59.254+0000 7ff5a1b2a640 1 -- 172.21.15.143:0/2871953472 >> 172.21.15.143:0/2871953472 conn(0x7ff59c06a890 msgr2=0x7ff59c15dc00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:24:59.258 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:59.254+0000 7ff5a1b2a640 1 -- 172.21.15.143:0/2871953472 shutdown_connections 2023-11-26T07:24:59.258 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:59.254+0000 7ff5a1b2a640 1 -- 172.21.15.143:0/2871953472 wait complete. 2023-11-26T07:24:59.274 DEBUG:teuthology.orchestra.run.smithi143:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 0 2023-11-26T07:24:59.432 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:59.418+0000 7f57c3c17640 1 Processor -- start 2023-11-26T07:24:59.432 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:59.418+0000 7f57c3c17640 1 -- start start 2023-11-26T07:24:59.432 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:59.418+0000 7f57c3c17640 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7f57bc15d6e0 0x7f57bc0d0df0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:24:59.432 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:59.418+0000 7f57c3c17640 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7f57bc0b2160 0x7f57bc0d14b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:24:59.433 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:59.418+0000 7f57c3c17640 1 -- --> v1:172.21.15.143:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f57bc0cf0e0 con 0x7f57bc0d19f0 2023-11-26T07:24:59.433 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:59.418+0000 7f57c3c17640 1 -- --> v2:172.21.15.143:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f57bc15dac0 con 0x7f57bc0b2160 2023-11-26T07:24:59.433 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:59.418+0000 7f57c3c17640 1 -- --> v2:172.21.15.188:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f57bc0d1e80 con 0x7f57bc15d6e0 2023-11-26T07:24:59.433 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:59.418+0000 7f57c11b2640 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7f57bc0b2160 0x7f57bc0d14b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:24:59.433 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:59.418+0000 7f57c21b4640 1 --1- >> v1:172.21.15.143:6789/0 conn(0x7f57bc0d19f0 0x7f57bc0cfbf0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.143:6789/0 says I am v1:172.21.15.143:36298/0 (socket says 172.21.15.143:36298) 2023-11-26T07:24:59.433 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:59.418+0000 7f57c21b4640 1 -- 172.21.15.143:0/1102943261 learned_addr learned my addr 172.21.15.143:0/1102943261 (peer_addr_for_me v1:172.21.15.143:0/0) 2023-11-26T07:24:59.433 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:59.418+0000 7f57c19b3640 1 --2- 172.21.15.143:0/1102943261 >> v2:172.21.15.188:3300/0 conn(0x7f57bc15d6e0 0x7f57bc0d0df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:24:59.433 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:59.422+0000 7f57c11b2640 1 -- 172.21.15.143:0/1102943261 >> v2:172.21.15.188:3300/0 conn(0x7f57bc15d6e0 msgr2=0x7f57bc0d0df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-26T07:24:59.434 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:59.422+0000 7f57c11b2640 1 --2- 172.21.15.143:0/1102943261 >> v2:172.21.15.188:3300/0 conn(0x7f57bc15d6e0 0x7f57bc0d0df0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:24:59.434 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:59.422+0000 7f57c11b2640 1 -- 172.21.15.143:0/1102943261 >> v1:172.21.15.143:6789/0 conn(0x7f57bc0d19f0 legacy=0x7f57bc0cfbf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:24:59.434 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:59.422+0000 7f57c11b2640 1 -- 172.21.15.143:0/1102943261 --> v2:172.21.15.143:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f57bc0ceb20 con 0x7f57bc0b2160 2023-11-26T07:24:59.434 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:59.422+0000 7f57c09b1640 1 -- 172.21.15.143:0/1102943261 <== mon.0 v1:172.21.15.143:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4187476440 0 0) 0x7f57bc0cf0e0 con 0x7f57bc0d19f0 2023-11-26T07:24:59.434 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:59.422+0000 7f57c11b2640 1 --2- 172.21.15.143:0/1102943261 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f57bc0b2160 0x7f57bc0d14b0 secure :-1 s=READY pgs=1456 cs=0 l=1 rev1=1 crypto rx=0x7f57bc0b5db0 tx=0x7f57ac01c460 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b92d689fc3ace963 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:24:59.434 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:59.422+0000 7f57c09b1640 1 -- 172.21.15.143:0/1102943261 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f57ac004230 con 0x7f57bc0b2160 2023-11-26T07:24:59.434 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:59.422+0000 7f57c09b1640 1 -- 172.21.15.143:0/1102943261 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f57ac0a7050 con 0x7f57bc0b2160 2023-11-26T07:24:59.435 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:59.422+0000 7f57c09b1640 1 -- 172.21.15.143:0/1102943261 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f57ac003620 con 0x7f57bc0b2160 2023-11-26T07:24:59.435 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:59.422+0000 7f57c3c17640 1 -- 172.21.15.143:0/1102943261 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f57bc0b2160 msgr2=0x7f57bc0d14b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:24:59.435 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:59.422+0000 7f57c3c17640 1 --2- 172.21.15.143:0/1102943261 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f57bc0b2160 0x7f57bc0d14b0 secure :-1 s=READY pgs=1456 cs=0 l=1 rev1=1 crypto rx=0x7f57bc0b5db0 tx=0x7f57ac01c460 comp rx=0 tx=0).stop 2023-11-26T07:24:59.435 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:59.426+0000 7f57c3c17640 1 -- 172.21.15.143:0/1102943261 shutdown_connections 2023-11-26T07:24:59.435 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:59.426+0000 7f57c3c17640 1 --2- 172.21.15.143:0/1102943261 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f57bc0b2160 0x7f57bc0d14b0 unknown :-1 s=CLOSED pgs=1456 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:24:59.435 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:59.426+0000 7f57c3c17640 1 --2- 172.21.15.143:0/1102943261 >> v2:172.21.15.188:3300/0 conn(0x7f57bc15d6e0 0x7f57bc0d0df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:24:59.435 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:59.426+0000 7f57c3c17640 1 -- 172.21.15.143:0/1102943261 >> 172.21.15.143:0/1102943261 conn(0x7f57bc09c550 msgr2=0x7f57bc0d3220 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:24:59.436 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:59.426+0000 7f57c3c17640 1 -- 172.21.15.143:0/1102943261 shutdown_connections 2023-11-26T07:24:59.436 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:59.426+0000 7f57c3c17640 1 -- 172.21.15.143:0/1102943261 wait complete. 2023-11-26T07:24:59.436 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:59.426+0000 7f57c3c17640 1 Processor -- start 2023-11-26T07:24:59.436 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:59.426+0000 7f57c3c17640 1 -- start start 2023-11-26T07:24:59.436 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:59.426+0000 7f57c3c17640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f57bc0b2160 0x7f57bc0ccd00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:24:59.436 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:59.426+0000 7f57c3c17640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f57bc0d19f0 0x7f57bc0b6e90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:24:59.436 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:59.426+0000 7f57c3c17640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f57bc15d6e0 0x7f57bc0b73d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:24:59.436 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:59.426+0000 7f57c3c17640 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f57bc0cc870 con 0x7f57bc15d6e0 2023-11-26T07:24:59.437 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:59.426+0000 7f57c3c17640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f57bc0cbc60 con 0x7f57bc0d19f0 2023-11-26T07:24:59.437 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:59.426+0000 7f57c3c17640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f57bc0cbf40 con 0x7f57bc0b2160 2023-11-26T07:24:59.437 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:59.426+0000 7f57c21b4640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f57bc15d6e0 0x7f57bc0b73d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:24:59.437 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:59.426+0000 7f57c21b4640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f57bc15d6e0 0x7f57bc0b73d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:45944/0 (socket says 172.21.15.143:45944) 2023-11-26T07:24:59.437 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:59.426+0000 7f57c21b4640 1 -- 172.21.15.143:0/1148243198 learned_addr learned my addr 172.21.15.143:0/1148243198 (peer_addr_for_me v2:172.21.15.143:0/0) 2023-11-26T07:24:59.437 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:59.426+0000 7f57c19b3640 1 --2- 172.21.15.143:0/1148243198 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f57bc0b2160 0x7f57bc0ccd00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:24:59.437 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:59.426+0000 7f57c11b2640 1 --2- 172.21.15.143:0/1148243198 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f57bc0d19f0 0x7f57bc0b6e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:24:59.438 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:59.426+0000 7f57c21b4640 1 -- 172.21.15.143:0/1148243198 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f57bc0b2160 msgr2=0x7f57bc0ccd00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:24:59.438 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:59.426+0000 7f57c21b4640 1 --2- 172.21.15.143:0/1148243198 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f57bc0b2160 0x7f57bc0ccd00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:24:59.438 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:59.426+0000 7f57c21b4640 1 -- 172.21.15.143:0/1148243198 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f57bc0d19f0 msgr2=0x7f57bc0b6e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:24:59.438 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:59.426+0000 7f57c21b4640 1 --2- 172.21.15.143:0/1148243198 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f57bc0d19f0 0x7f57bc0b6e90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:24:59.438 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:59.426+0000 7f57c21b4640 1 -- 172.21.15.143:0/1148243198 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f57ac009420 con 0x7f57bc15d6e0 2023-11-26T07:24:59.438 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:59.426+0000 7f57c21b4640 1 --2- 172.21.15.143:0/1148243198 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f57bc15d6e0 0x7f57bc0b73d0 secure :-1 s=READY pgs=1457 cs=0 l=1 rev1=1 crypto rx=0x7f57b400c7a0 tx=0x7f57b400cc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:24:59.438 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:59.430+0000 7f57b27fc640 1 -- 172.21.15.143:0/1148243198 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f57b400a640 con 0x7f57bc15d6e0 2023-11-26T07:24:59.438 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:59.430+0000 7f57b27fc640 1 -- 172.21.15.143:0/1148243198 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f57b400a7a0 con 0x7f57bc15d6e0 2023-11-26T07:24:59.439 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:59.430+0000 7f57b27fc640 1 -- 172.21.15.143:0/1148243198 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f57b400abd0 con 0x7f57bc15d6e0 2023-11-26T07:24:59.439 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:59.430+0000 7f57c3c17640 1 -- 172.21.15.143:0/1148243198 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f57bc0cc220 con 0x7f57bc15d6e0 2023-11-26T07:24:59.439 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:59.430+0000 7f57c3c17640 1 -- 172.21.15.143:0/1148243198 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f57bc0b3ce0 con 0x7f57bc15d6e0 2023-11-26T07:24:59.439 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:59.430+0000 7f57c3c17640 1 -- 172.21.15.143:0/1148243198 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f57bc067680 con 0x7f57bc15d6e0 2023-11-26T07:24:59.439 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:59.430+0000 7f57b27fc640 1 -- 172.21.15.143:0/1148243198 <== mon.0 v2:172.21.15.143:3300/0 4 ==== mgrmap(e 18) v1 ==== 86107+0+0 (secure 0 0 0) 0x7f57b400ad30 con 0x7f57bc15d6e0 2023-11-26T07:24:59.440 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:59.430+0000 7f57b27fc640 1 --2- 172.21.15.143:0/1148243198 >> [v2:172.21.15.188:6836/3401128553,v1:172.21.15.188:6837/3401128553] conn(0x7f57a0066740 0x7f57a0068c00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:24:59.440 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:59.430+0000 7f57b27fc640 1 -- 172.21.15.143:0/1148243198 <== mon.0 v2:172.21.15.143:3300/0 5 ==== osd_map(106..106 src has 1..106) v4 ==== 4943+0+0 (secure 0 0 0) 0x7f57b4083500 con 0x7f57bc15d6e0 2023-11-26T07:24:59.440 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:59.430+0000 7f57c19b3640 1 --2- 172.21.15.143:0/1148243198 >> [v2:172.21.15.188:6836/3401128553,v1:172.21.15.188:6837/3401128553] conn(0x7f57a0066740 0x7f57a0068c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:24:59.440 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:59.434+0000 7f57c19b3640 1 --2- 172.21.15.143:0/1148243198 >> [v2:172.21.15.188:6836/3401128553,v1:172.21.15.188:6837/3401128553] conn(0x7f57a0066740 0x7f57a0068c00 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7f57bc15a930 tx=0x7f57b8008040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:24:59.440 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:59.434+0000 7f57b27fc640 1 -- 172.21.15.143:0/1148243198 <== mon.0 v2:172.21.15.143:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172649 (secure 0 0 0) 0x7f57b4050980 con 0x7f57bc15d6e0 2023-11-26T07:24:59.584 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:59.582+0000 7f57c3c17640 1 -- 172.21.15.143:0/1148243198 --> [v2:172.21.15.188:6836/3401128553,v1:172.21.15.188:6837/3401128553] -- mgr_command(tid 0: {"prefix": "osd deep-scrub", "who": "0", "target": ["mon-mgr", ""]}) v1 -- 0x7f57bc0b3f90 con 0x7f57a0066740 2023-11-26T07:24:59.585 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:59.586+0000 7f57b27fc640 1 -- 172.21.15.143:0/1148243198 <== mgr.4108 v2:172.21.15.188:6836/3401128553 1 ==== mgr_command_reply(tid 0: 0 instructed osd(s) 0 to deep-scrub) v1 ==== 41+0+0 (secure 0 0 0) 0x7f57bc0b3f90 con 0x7f57a0066740 2023-11-26T07:24:59.585 INFO:teuthology.orchestra.run.smithi143.stderr:instructed osd(s) 0 to deep-scrub 2023-11-26T07:24:59.587 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:59.586+0000 7f57c3c17640 1 -- 172.21.15.143:0/1148243198 >> [v2:172.21.15.188:6836/3401128553,v1:172.21.15.188:6837/3401128553] conn(0x7f57a0066740 msgr2=0x7f57a0068c00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:24:59.587 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:59.586+0000 7f57c3c17640 1 --2- 172.21.15.143:0/1148243198 >> [v2:172.21.15.188:6836/3401128553,v1:172.21.15.188:6837/3401128553] conn(0x7f57a0066740 0x7f57a0068c00 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7f57bc15a930 tx=0x7f57b8008040 comp rx=0 tx=0).stop 2023-11-26T07:24:59.587 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:59.586+0000 7f57c3c17640 1 -- 172.21.15.143:0/1148243198 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f57bc15d6e0 msgr2=0x7f57bc0b73d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:24:59.588 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:59.586+0000 7f57c3c17640 1 --2- 172.21.15.143:0/1148243198 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f57bc15d6e0 0x7f57bc0b73d0 secure :-1 s=READY pgs=1457 cs=0 l=1 rev1=1 crypto rx=0x7f57b400c7a0 tx=0x7f57b400cc60 comp rx=0 tx=0).stop 2023-11-26T07:24:59.588 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:59.586+0000 7f57c3c17640 1 -- 172.21.15.143:0/1148243198 shutdown_connections 2023-11-26T07:24:59.588 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:59.586+0000 7f57c3c17640 1 --2- 172.21.15.143:0/1148243198 >> [v2:172.21.15.188:6836/3401128553,v1:172.21.15.188:6837/3401128553] conn(0x7f57a0066740 0x7f57a0068c00 unknown :-1 s=CLOSED pgs=188 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:24:59.588 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:59.586+0000 7f57c3c17640 1 --2- 172.21.15.143:0/1148243198 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f57bc15d6e0 0x7f57bc0b73d0 unknown :-1 s=CLOSED pgs=1457 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:24:59.589 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:59.586+0000 7f57c3c17640 1 --2- 172.21.15.143:0/1148243198 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f57bc0d19f0 0x7f57bc0b6e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:24:59.589 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:59.586+0000 7f57c3c17640 1 --2- 172.21.15.143:0/1148243198 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f57bc0b2160 0x7f57bc0ccd00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:24:59.589 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:59.586+0000 7f57c3c17640 1 -- 172.21.15.143:0/1148243198 >> 172.21.15.143:0/1148243198 conn(0x7f57bc09c550 msgr2=0x7f57bc0cfbe0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:24:59.589 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:59.586+0000 7f57c3c17640 1 -- 172.21.15.143:0/1148243198 shutdown_connections 2023-11-26T07:24:59.589 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:59.586+0000 7f57c3c17640 1 -- 172.21.15.143:0/1148243198 wait complete. 2023-11-26T07:24:59.607 INFO:tasks.ceph:Scrubbing osd.1 2023-11-26T07:24:59.607 DEBUG:teuthology.orchestra.run.smithi143:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.1 config set osd_debug_deep_scrub_sleep 0 2023-11-26T07:24:59.753 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:59.750+0000 7fdd6713e640 1 Processor -- start 2023-11-26T07:24:59.753 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:59.750+0000 7fdd6713e640 1 -- start start 2023-11-26T07:24:59.755 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:59.754+0000 7fdd6713e640 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7fdd600cf710 0x7fdd600cd8c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:24:59.755 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:59.754+0000 7fdd6713e640 1 -- --> v1:172.21.15.143:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fdd600cc730 con 0x7fdd600cf280 2023-11-26T07:24:59.755 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:59.754+0000 7fdd6713e640 1 -- --> v1:172.21.15.188:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fdd60070ee0 con 0x7fdd60143ac0 2023-11-26T07:24:59.755 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:59.754+0000 7fdd6713e640 1 -- --> v2:172.21.15.188:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fdd60143ea0 con 0x7fdd600cf710 2023-11-26T07:24:59.756 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:59.754+0000 7fdd5ffff640 1 --1- >> v1:172.21.15.143:6789/0 conn(0x7fdd600cf280 0x7fdd600cd1b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.143:6789/0 says I am v1:172.21.15.143:36308/0 (socket says 172.21.15.143:36308) 2023-11-26T07:24:59.756 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:59.754+0000 7fdd5ffff640 1 -- 172.21.15.143:0/3422254065 learned_addr learned my addr 172.21.15.143:0/3422254065 (peer_addr_for_me v1:172.21.15.143:0/0) 2023-11-26T07:24:59.756 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:59.754+0000 7fdd656db640 1 --2- 172.21.15.143:0/3422254065 >> v2:172.21.15.188:3300/0 conn(0x7fdd600cf710 0x7fdd600cd8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:24:59.756 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:59.754+0000 7fdd5f7fe640 1 -- 172.21.15.143:0/3422254065 <== mon.0 v1:172.21.15.143:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1895945206 0 0) 0x7fdd600cc730 con 0x7fdd600cf280 2023-11-26T07:24:59.757 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:59.754+0000 7fdd5f7fe640 1 -- 172.21.15.143:0/3422254065 --> v1:172.21.15.143:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fdd50003530 con 0x7fdd600cf280 2023-11-26T07:24:59.757 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:59.754+0000 7fdd5f7fe640 1 -- 172.21.15.143:0/3422254065 <== mon.0 v1:172.21.15.143:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3706259369 0 0) 0x7fdd50003530 con 0x7fdd600cf280 2023-11-26T07:24:59.757 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:59.754+0000 7fdd656db640 1 -- 172.21.15.143:0/3422254065 >> v1:172.21.15.188:6789/0 conn(0x7fdd60143ac0 legacy=0x7fdd60070370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:24:59.757 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:59.754+0000 7fdd656db640 1 -- 172.21.15.143:0/3422254065 >> v1:172.21.15.143:6789/0 conn(0x7fdd600cf280 legacy=0x7fdd600cd1b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:24:59.757 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:59.754+0000 7fdd656db640 1 -- 172.21.15.143:0/3422254065 --> v2:172.21.15.188:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdd600711c0 con 0x7fdd600cf710 2023-11-26T07:24:59.758 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:59.758+0000 7fdd656db640 1 --2- 172.21.15.143:0/3422254065 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fdd600cf710 0x7fdd600cd8c0 secure :-1 s=READY pgs=535 cs=0 l=1 rev1=1 crypto rx=0x7fdd58009d90 tx=0x7fdd5801c870 comp rx=0 tx=0).ready entity=mon.1 client_cookie=b0d048a4b163ef9b server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:24:59.759 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:59.758+0000 7fdd5f7fe640 1 -- 172.21.15.143:0/3422254065 <== mon.1 v2:172.21.15.188:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdd4c001680 con 0x7fdd600cf710 2023-11-26T07:24:59.759 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:59.758+0000 7fdd5f7fe640 1 -- 172.21.15.143:0/3422254065 <== mon.1 v2:172.21.15.188:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdd580a56b0 con 0x7fdd600cf710 2023-11-26T07:24:59.759 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:59.758+0000 7fdd5f7fe640 1 -- 172.21.15.143:0/3422254065 <== mon.1 v2:172.21.15.188:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdd580a5a40 con 0x7fdd600cf710 2023-11-26T07:24:59.760 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:59.758+0000 7fdd6713e640 1 -- 172.21.15.143:0/3422254065 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fdd600cf710 msgr2=0x7fdd600cd8c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:24:59.760 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:59.758+0000 7fdd6713e640 1 --2- 172.21.15.143:0/3422254065 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fdd600cf710 0x7fdd600cd8c0 secure :-1 s=READY pgs=535 cs=0 l=1 rev1=1 crypto rx=0x7fdd58009d90 tx=0x7fdd5801c870 comp rx=0 tx=0).stop 2023-11-26T07:24:59.760 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:59.758+0000 7fdd6713e640 1 -- 172.21.15.143:0/3422254065 shutdown_connections 2023-11-26T07:24:59.761 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:59.758+0000 7fdd6713e640 1 --2- 172.21.15.143:0/3422254065 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fdd600cf710 0x7fdd600cd8c0 unknown :-1 s=CLOSED pgs=535 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:24:59.761 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:59.758+0000 7fdd6713e640 1 -- 172.21.15.143:0/3422254065 >> 172.21.15.143:0/3422254065 conn(0x7fdd6006a890 msgr2=0x7fdd6006aca0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:24:59.761 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:59.758+0000 7fdd6713e640 1 -- 172.21.15.143:0/3422254065 shutdown_connections 2023-11-26T07:24:59.761 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:59.758+0000 7fdd6713e640 1 -- 172.21.15.143:0/3422254065 wait complete. 2023-11-26T07:24:59.764 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:59.762+0000 7fdd6713e640 1 Processor -- start 2023-11-26T07:24:59.764 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:59.762+0000 7fdd6713e640 1 -- start start 2023-11-26T07:24:59.764 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:59.762+0000 7fdd6713e640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fdd600cf280 0x7fdd60138910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:24:59.764 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:59.762+0000 7fdd6713e640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fdd600cf710 0x7fdd60154750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:24:59.765 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:59.762+0000 7fdd6713e640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fdd60143ac0 0x7fdd601593f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:24:59.765 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:59.762+0000 7fdd6713e640 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdd60138e50 con 0x7fdd60143ac0 2023-11-26T07:24:59.765 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:59.762+0000 7fdd6713e640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdd60154fa0 con 0x7fdd600cf280 2023-11-26T07:24:59.765 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:59.762+0000 7fdd6713e640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdd60155280 con 0x7fdd600cf710 2023-11-26T07:24:59.765 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:59.762+0000 7fdd656db640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fdd60143ac0 0x7fdd601593f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:24:59.766 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:59.762+0000 7fdd656db640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fdd60143ac0 0x7fdd601593f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:45954/0 (socket says 172.21.15.143:45954) 2023-11-26T07:24:59.766 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:59.762+0000 7fdd656db640 1 -- 172.21.15.143:0/1946428809 learned_addr learned my addr 172.21.15.143:0/1946428809 (peer_addr_for_me v2:172.21.15.143:0/0) 2023-11-26T07:24:59.766 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:59.762+0000 7fdd64eda640 1 --2- 172.21.15.143:0/1946428809 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fdd600cf280 0x7fdd60138910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:24:59.766 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:59.762+0000 7fdd5ffff640 1 --2- 172.21.15.143:0/1946428809 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fdd600cf710 0x7fdd60154750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:24:59.767 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:59.762+0000 7fdd656db640 1 -- 172.21.15.143:0/1946428809 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fdd600cf710 msgr2=0x7fdd60154750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:24:59.767 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:59.762+0000 7fdd656db640 1 --2- 172.21.15.143:0/1946428809 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fdd600cf710 0x7fdd60154750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:24:59.767 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:59.762+0000 7fdd656db640 1 -- 172.21.15.143:0/1946428809 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fdd600cf280 msgr2=0x7fdd60138910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:24:59.767 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:59.762+0000 7fdd656db640 1 --2- 172.21.15.143:0/1946428809 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fdd600cf280 0x7fdd60138910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:24:59.767 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:59.762+0000 7fdd656db640 1 -- 172.21.15.143:0/1946428809 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdd58009a70 con 0x7fdd60143ac0 2023-11-26T07:24:59.767 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:59.762+0000 7fdd656db640 1 --2- 172.21.15.143:0/1946428809 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fdd60143ac0 0x7fdd601593f0 secure :-1 s=READY pgs=1459 cs=0 l=1 rev1=1 crypto rx=0x7fdd580b0040 tx=0x7fdd600ce610 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:24:59.767 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:59.762+0000 7fdd5d7fa640 1 -- 172.21.15.143:0/1946428809 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdd580be070 con 0x7fdd60143ac0 2023-11-26T07:24:59.768 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:59.762+0000 7fdd5d7fa640 1 -- 172.21.15.143:0/1946428809 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdd58009450 con 0x7fdd60143ac0 2023-11-26T07:24:59.768 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:59.762+0000 7fdd6713e640 1 -- 172.21.15.143:0/1946428809 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdd60155560 con 0x7fdd60143ac0 2023-11-26T07:24:59.768 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:59.762+0000 7fdd5d7fa640 1 -- 172.21.15.143:0/1946428809 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdd58003be0 con 0x7fdd60143ac0 2023-11-26T07:24:59.768 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:59.762+0000 7fdd6713e640 1 -- 172.21.15.143:0/1946428809 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdd60154c90 con 0x7fdd60143ac0 2023-11-26T07:24:59.769 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:59.762+0000 7fdd6713e640 1 -- 172.21.15.143:0/1946428809 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7fdd600cfb20 con 0x7fdd60143ac0 2023-11-26T07:24:59.770 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:59.766+0000 7fdd5d7fa640 1 -- 172.21.15.143:0/1946428809 <== mon.0 v2:172.21.15.143:3300/0 4 ==== mgrmap(e 18) v1 ==== 86107+0+0 (secure 0 0 0) 0x7fdd580b4040 con 0x7fdd60143ac0 2023-11-26T07:24:59.770 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:59.766+0000 7fdd5d7fa640 1 --2- 172.21.15.143:0/1946428809 >> [v2:172.21.15.188:6836/3401128553,v1:172.21.15.188:6837/3401128553] conn(0x7fdd44066740 0x7fdd44068c00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:24:59.770 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:59.766+0000 7fdd5d7fa640 1 -- 172.21.15.143:0/1946428809 <== mon.0 v2:172.21.15.143:3300/0 5 ==== osd_map(106..106 src has 1..106) v4 ==== 4943+0+0 (secure 0 0 0) 0x7fdd5811ec70 con 0x7fdd60143ac0 2023-11-26T07:24:59.770 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:59.766+0000 7fdd5d7fa640 1 --2- 172.21.15.143:0/1946428809 >> [v2:172.21.15.143:6816/1144942530,v1:172.21.15.143:6818/1144942530] conn(0x7fdd4406c5a0 0x7fdd4406e9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:24:59.770 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:59.766+0000 7fdd5d7fa640 1 -- 172.21.15.143:0/1946428809 --> [v2:172.21.15.143:6816/1144942530,v1:172.21.15.143:6818/1144942530] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7fdd4406f070 con 0x7fdd4406c5a0 2023-11-26T07:24:59.770 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:59.766+0000 7fdd5d7fa640 1 -- 172.21.15.143:0/1946428809 <== mon.0 v2:172.21.15.143:3300/0 6 ==== mon_get_version_reply(handle=1 version=106) v2 ==== 24+0+0 (secure 0 0 0) 0x7fdd5811f060 con 0x7fdd60143ac0 2023-11-26T07:24:59.771 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:59.766+0000 7fdd64eda640 1 --2- 172.21.15.143:0/1946428809 >> [v2:172.21.15.188:6836/3401128553,v1:172.21.15.188:6837/3401128553] conn(0x7fdd44066740 0x7fdd44068c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:24:59.771 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:59.766+0000 7fdd5ffff640 1 --2- 172.21.15.143:0/1946428809 >> [v2:172.21.15.143:6816/1144942530,v1:172.21.15.143:6818/1144942530] conn(0x7fdd4406c5a0 0x7fdd4406e9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:24:59.771 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:59.766+0000 7fdd5ffff640 1 --2- 172.21.15.143:0/1946428809 >> [v2:172.21.15.143:6816/1144942530,v1:172.21.15.143:6818/1144942530] conn(0x7fdd4406c5a0 0x7fdd4406e9c0 crc :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:24:59.771 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:59.766+0000 7fdd64eda640 1 --2- 172.21.15.143:0/1946428809 >> [v2:172.21.15.188:6836/3401128553,v1:172.21.15.188:6837/3401128553] conn(0x7fdd44066740 0x7fdd44068c00 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7fdd54007fd0 tx=0x7fdd54009040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:24:59.771 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:59.766+0000 7fdd5d7fa640 1 -- 172.21.15.143:0/1946428809 <== osd.1 v2:172.21.15.143:6816/1144942530 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+26965 (crc 0 0 0) 0x7fdd4406f070 con 0x7fdd4406c5a0 2023-11-26T07:24:59.781 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:59.782+0000 7fdd6713e640 1 -- 172.21.15.143:0/1946428809 --> [v2:172.21.15.143:6816/1144942530,v1:172.21.15.143:6818/1144942530] -- command(tid 2: {"prefix": "config set", "var": "osd_debug_deep_scrub_sleep", "val": ["0"]}) v1 -- 0x7fdd60067680 con 0x7fdd4406c5a0 2023-11-26T07:24:59.782 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:59.782+0000 7fdd5d7fa640 1 -- 172.21.15.143:0/1946428809 <== osd.1 v2:172.21.15.143:6816/1144942530 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+97 (crc 0 0 0) 0x7fdd60067680 con 0x7fdd4406c5a0 2023-11-26T07:24:59.782 INFO:teuthology.orchestra.run.smithi143.stdout:{ 2023-11-26T07:24:59.782 INFO:teuthology.orchestra.run.smithi143.stdout: "success": "osd_debug_deep_scrub_sleep = '' (not observed, change may require restart) " 2023-11-26T07:24:59.783 INFO:teuthology.orchestra.run.smithi143.stdout:} 2023-11-26T07:24:59.783 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:59.782+0000 7fdd6713e640 1 -- 172.21.15.143:0/1946428809 >> [v2:172.21.15.143:6816/1144942530,v1:172.21.15.143:6818/1144942530] conn(0x7fdd4406c5a0 msgr2=0x7fdd4406e9c0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:24:59.784 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:59.782+0000 7fdd6713e640 1 --2- 172.21.15.143:0/1946428809 >> [v2:172.21.15.143:6816/1144942530,v1:172.21.15.143:6818/1144942530] conn(0x7fdd4406c5a0 0x7fdd4406e9c0 crc :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:24:59.784 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:59.782+0000 7fdd6713e640 1 -- 172.21.15.143:0/1946428809 >> [v2:172.21.15.188:6836/3401128553,v1:172.21.15.188:6837/3401128553] conn(0x7fdd44066740 msgr2=0x7fdd44068c00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:24:59.784 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:59.782+0000 7fdd6713e640 1 --2- 172.21.15.143:0/1946428809 >> [v2:172.21.15.188:6836/3401128553,v1:172.21.15.188:6837/3401128553] conn(0x7fdd44066740 0x7fdd44068c00 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7fdd54007fd0 tx=0x7fdd54009040 comp rx=0 tx=0).stop 2023-11-26T07:24:59.784 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:59.782+0000 7fdd6713e640 1 -- 172.21.15.143:0/1946428809 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fdd60143ac0 msgr2=0x7fdd601593f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:24:59.785 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:59.782+0000 7fdd6713e640 1 --2- 172.21.15.143:0/1946428809 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fdd60143ac0 0x7fdd601593f0 secure :-1 s=READY pgs=1459 cs=0 l=1 rev1=1 crypto rx=0x7fdd580b0040 tx=0x7fdd600ce610 comp rx=0 tx=0).stop 2023-11-26T07:24:59.785 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:59.782+0000 7fdd656db640 1 -- 172.21.15.143:0/1946428809 reap_dead start 2023-11-26T07:24:59.785 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:59.782+0000 7fdd6713e640 1 -- 172.21.15.143:0/1946428809 shutdown_connections 2023-11-26T07:24:59.785 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:59.782+0000 7fdd6713e640 1 -- 172.21.15.143:0/1946428809 >> 172.21.15.143:0/1946428809 conn(0x7fdd6006a890 msgr2=0x7fdd600cd1b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:24:59.786 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:59.782+0000 7fdd6713e640 1 -- 172.21.15.143:0/1946428809 shutdown_connections 2023-11-26T07:24:59.786 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:59.782+0000 7fdd6713e640 1 -- 172.21.15.143:0/1946428809 wait complete. 2023-11-26T07:24:59.801 DEBUG:teuthology.orchestra.run.smithi143:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 1 2023-11-26T07:24:59.939 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:59.938+0000 7f3bf3668640 1 Processor -- start 2023-11-26T07:24:59.940 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:59.938+0000 7f3bf3668640 1 -- start start 2023-11-26T07:24:59.940 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:59.938+0000 7f3bf3668640 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7f3bec14b1e0 0x7f3bec0cf280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:24:59.941 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:59.938+0000 7f3bf3668640 1 -- --> v1:172.21.15.143:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3bec0cfc20 con 0x7f3bec0cf7c0 2023-11-26T07:24:59.941 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:59.938+0000 7f3bf3668640 1 -- --> v1:172.21.15.188:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3bec162080 con 0x7f3bec070fd0 2023-11-26T07:24:59.941 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:59.938+0000 7f3bf3668640 1 -- --> v2:172.21.15.143:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3bec161d70 con 0x7f3bec14b1e0 2023-11-26T07:24:59.942 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:59.942+0000 7f3bf1404640 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7f3bec14b1e0 0x7f3bec0cf280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:24:59.942 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:59.942+0000 7f3bf0c03640 1 --1- >> v1:172.21.15.143:6789/0 conn(0x7f3bec0cf7c0 0x7f3bec0708c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.143:6789/0 says I am v1:172.21.15.143:36312/0 (socket says 172.21.15.143:36312) 2023-11-26T07:24:59.942 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:59.942+0000 7f3bf0c03640 1 -- 172.21.15.143:0/3442592330 learned_addr learned my addr 172.21.15.143:0/3442592330 (peer_addr_for_me v1:172.21.15.143:0/0) 2023-11-26T07:24:59.942 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:59.942+0000 7f3bf1404640 1 -- 172.21.15.143:0/3442592330 >> v1:172.21.15.188:6789/0 conn(0x7f3bec070fd0 legacy=0x7f3bec162510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:24:59.942 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:59.942+0000 7f3bf1404640 1 -- 172.21.15.143:0/3442592330 >> v1:172.21.15.143:6789/0 conn(0x7f3bec0cf7c0 legacy=0x7f3bec0708c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:24:59.942 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:59.942+0000 7f3bf1404640 1 -- 172.21.15.143:0/3442592330 --> v2:172.21.15.143:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3bec14b5c0 con 0x7f3bec14b1e0 2023-11-26T07:24:59.944 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:59.942+0000 7f3bf1404640 1 --2- 172.21.15.143:0/3442592330 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f3bec14b1e0 0x7f3bec0cf280 secure :-1 s=READY pgs=1461 cs=0 l=1 rev1=1 crypto rx=0x7f3be80097c0 tx=0x7f3be801c450 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8de16aefdce41674 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:24:59.945 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:59.942+0000 7f3be3fff640 1 -- 172.21.15.143:0/3442592330 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3be80ab650 con 0x7f3bec14b1e0 2023-11-26T07:24:59.945 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:59.942+0000 7f3be3fff640 1 -- 172.21.15.143:0/3442592330 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3be80ab7b0 con 0x7f3bec14b1e0 2023-11-26T07:24:59.945 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:59.942+0000 7f3be3fff640 1 -- 172.21.15.143:0/3442592330 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3be80abb40 con 0x7f3bec14b1e0 2023-11-26T07:24:59.946 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:59.946+0000 7f3bf3668640 1 -- 172.21.15.143:0/3442592330 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f3bec14b1e0 msgr2=0x7f3bec0cf280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:24:59.946 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:59.946+0000 7f3bf3668640 1 --2- 172.21.15.143:0/3442592330 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f3bec14b1e0 0x7f3bec0cf280 secure :-1 s=READY pgs=1461 cs=0 l=1 rev1=1 crypto rx=0x7f3be80097c0 tx=0x7f3be801c450 comp rx=0 tx=0).stop 2023-11-26T07:24:59.946 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:59.946+0000 7f3bf3668640 1 -- 172.21.15.143:0/3442592330 shutdown_connections 2023-11-26T07:24:59.946 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:59.946+0000 7f3bf3668640 1 --2- 172.21.15.143:0/3442592330 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f3bec14b1e0 0x7f3bec0cf280 unknown :-1 s=CLOSED pgs=1461 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:24:59.947 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:59.946+0000 7f3bf3668640 1 -- 172.21.15.143:0/3442592330 >> 172.21.15.143:0/3442592330 conn(0x7f3bec06a890 msgr2=0x7f3bec06aca0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:24:59.947 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:59.946+0000 7f3bf3668640 1 -- 172.21.15.143:0/3442592330 shutdown_connections 2023-11-26T07:24:59.947 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:59.946+0000 7f3bf3668640 1 -- 172.21.15.143:0/3442592330 wait complete. 2023-11-26T07:24:59.948 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:59.946+0000 7f3bf3668640 1 Processor -- start 2023-11-26T07:24:59.948 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:59.946+0000 7f3bf3668640 1 -- start start 2023-11-26T07:24:59.950 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:59.946+0000 7f3bf3668640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f3bec070fd0 0x7f3bec0ad950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:24:59.950 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:59.946+0000 7f3bf3668640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f3bec0cf7c0 0x7f3bec0aaeb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:24:59.950 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:59.946+0000 7f3bf3668640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f3bec14b1e0 0x7f3bec0ab3f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:24:59.950 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:59.946+0000 7f3bf3668640 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3bec0aa5e0 con 0x7f3bec14b1e0 2023-11-26T07:24:59.950 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:59.946+0000 7f3bf3668640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3bec0a9a60 con 0x7f3bec0cf7c0 2023-11-26T07:24:59.950 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:59.946+0000 7f3bf3668640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3bec0a9d40 con 0x7f3bec070fd0 2023-11-26T07:24:59.951 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:59.946+0000 7f3bf1c05640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f3bec14b1e0 0x7f3bec0ab3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:24:59.951 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:59.946+0000 7f3bf1c05640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f3bec14b1e0 0x7f3bec0ab3f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:45980/0 (socket says 172.21.15.143:45980) 2023-11-26T07:24:59.951 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:59.946+0000 7f3bf1c05640 1 -- 172.21.15.143:0/776788476 learned_addr learned my addr 172.21.15.143:0/776788476 (peer_addr_for_me v2:172.21.15.143:0/0) 2023-11-26T07:24:59.951 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:59.950+0000 7f3bf1404640 1 --2- 172.21.15.143:0/776788476 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f3bec070fd0 0x7f3bec0ad950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:24:59.951 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:59.950+0000 7f3bf0c03640 1 --2- 172.21.15.143:0/776788476 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f3bec0cf7c0 0x7f3bec0aaeb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:24:59.952 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:59.950+0000 7f3bf1c05640 1 -- 172.21.15.143:0/776788476 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f3bec070fd0 msgr2=0x7f3bec0ad950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:24:59.952 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:59.950+0000 7f3bf1c05640 1 --2- 172.21.15.143:0/776788476 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f3bec070fd0 0x7f3bec0ad950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:24:59.952 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:59.950+0000 7f3bf1c05640 1 -- 172.21.15.143:0/776788476 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f3bec0cf7c0 msgr2=0x7f3bec0aaeb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:24:59.952 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:59.950+0000 7f3bf1c05640 1 --2- 172.21.15.143:0/776788476 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f3bec0cf7c0 0x7f3bec0aaeb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:24:59.952 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:59.950+0000 7f3bf1c05640 1 -- 172.21.15.143:0/776788476 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3be8009420 con 0x7f3bec14b1e0 2023-11-26T07:24:59.952 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:59.950+0000 7f3bf1c05640 1 --2- 172.21.15.143:0/776788476 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f3bec14b1e0 0x7f3bec0ab3f0 secure :-1 s=READY pgs=1462 cs=0 l=1 rev1=1 crypto rx=0x7f3be400c770 tx=0x7f3be400cc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:24:59.953 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:59.950+0000 7f3be1ffb640 1 -- 172.21.15.143:0/776788476 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3be4003b80 con 0x7f3bec14b1e0 2023-11-26T07:24:59.953 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:59.950+0000 7f3be1ffb640 1 -- 172.21.15.143:0/776788476 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3be4003ce0 con 0x7f3bec14b1e0 2023-11-26T07:24:59.953 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:59.950+0000 7f3be1ffb640 1 -- 172.21.15.143:0/776788476 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3be400aa90 con 0x7f3bec14b1e0 2023-11-26T07:24:59.953 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:59.950+0000 7f3bf3668640 1 -- 172.21.15.143:0/776788476 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3bec0aa020 con 0x7f3bec14b1e0 2023-11-26T07:24:59.954 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:59.950+0000 7f3bf3668640 1 -- 172.21.15.143:0/776788476 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3bec0ae370 con 0x7f3bec14b1e0 2023-11-26T07:24:59.955 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:59.950+0000 7f3bf3668640 1 -- 172.21.15.143:0/776788476 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3bec067680 con 0x7f3bec14b1e0 2023-11-26T07:24:59.955 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:59.950+0000 7f3be1ffb640 1 -- 172.21.15.143:0/776788476 <== mon.0 v2:172.21.15.143:3300/0 4 ==== mgrmap(e 18) v1 ==== 86107+0+0 (secure 0 0 0) 0x7f3be400abf0 con 0x7f3bec14b1e0 2023-11-26T07:24:59.955 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:59.950+0000 7f3be1ffb640 1 --2- 172.21.15.143:0/776788476 >> [v2:172.21.15.188:6836/3401128553,v1:172.21.15.188:6837/3401128553] conn(0x7f3bd0066740 0x7f3bd0068c00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:24:59.955 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:59.954+0000 7f3be1ffb640 1 -- 172.21.15.143:0/776788476 <== mon.0 v2:172.21.15.143:3300/0 5 ==== osd_map(106..106 src has 1..106) v4 ==== 4943+0+0 (secure 0 0 0) 0x7f3be4083cc0 con 0x7f3bec14b1e0 2023-11-26T07:24:59.955 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:59.954+0000 7f3bf1404640 1 --2- 172.21.15.143:0/776788476 >> [v2:172.21.15.188:6836/3401128553,v1:172.21.15.188:6837/3401128553] conn(0x7f3bd0066740 0x7f3bd0068c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:24:59.955 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:59.954+0000 7f3bf1404640 1 --2- 172.21.15.143:0/776788476 >> [v2:172.21.15.188:6836/3401128553,v1:172.21.15.188:6837/3401128553] conn(0x7f3bd0066740 0x7f3bd0068c00 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7f3be80af000 tx=0x7f3be80a7000 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:24:59.956 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:24:59.954+0000 7f3be1ffb640 1 -- 172.21.15.143:0/776788476 <== mon.0 v2:172.21.15.143:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172649 (secure 0 0 0) 0x7f3be40511a0 con 0x7f3bec14b1e0 2023-11-26T07:25:00.105 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:25:00.106+0000 7f3bf3668640 1 -- 172.21.15.143:0/776788476 --> [v2:172.21.15.188:6836/3401128553,v1:172.21.15.188:6837/3401128553] -- mgr_command(tid 0: {"prefix": "osd deep-scrub", "who": "1", "target": ["mon-mgr", ""]}) v1 -- 0x7f3bec06d920 con 0x7f3bd0066740 2023-11-26T07:25:00.106 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:25:00.106+0000 7f3be1ffb640 1 -- 172.21.15.143:0/776788476 <== mgr.4108 v2:172.21.15.188:6836/3401128553 1 ==== mgr_command_reply(tid 0: 0 instructed osd(s) 1 to deep-scrub) v1 ==== 41+0+0 (secure 0 0 0) 0x7f3bec06d920 con 0x7f3bd0066740 2023-11-26T07:25:00.106 INFO:teuthology.orchestra.run.smithi143.stderr:instructed osd(s) 1 to deep-scrub 2023-11-26T07:25:00.108 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:25:00.106+0000 7f3bf3668640 1 -- 172.21.15.143:0/776788476 >> [v2:172.21.15.188:6836/3401128553,v1:172.21.15.188:6837/3401128553] conn(0x7f3bd0066740 msgr2=0x7f3bd0068c00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:25:00.108 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:25:00.106+0000 7f3bf3668640 1 --2- 172.21.15.143:0/776788476 >> [v2:172.21.15.188:6836/3401128553,v1:172.21.15.188:6837/3401128553] conn(0x7f3bd0066740 0x7f3bd0068c00 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7f3be80af000 tx=0x7f3be80a7000 comp rx=0 tx=0).stop 2023-11-26T07:25:00.108 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:25:00.106+0000 7f3bf3668640 1 -- 172.21.15.143:0/776788476 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f3bec14b1e0 msgr2=0x7f3bec0ab3f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:25:00.108 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:25:00.106+0000 7f3bf3668640 1 --2- 172.21.15.143:0/776788476 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f3bec14b1e0 0x7f3bec0ab3f0 secure :-1 s=READY pgs=1462 cs=0 l=1 rev1=1 crypto rx=0x7f3be400c770 tx=0x7f3be400cc30 comp rx=0 tx=0).stop 2023-11-26T07:25:00.109 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:25:00.106+0000 7f3bf3668640 1 -- 172.21.15.143:0/776788476 shutdown_connections 2023-11-26T07:25:00.109 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:25:00.106+0000 7f3bf3668640 1 --2- 172.21.15.143:0/776788476 >> [v2:172.21.15.188:6836/3401128553,v1:172.21.15.188:6837/3401128553] conn(0x7f3bd0066740 0x7f3bd0068c00 unknown :-1 s=CLOSED pgs=190 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:25:00.109 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:25:00.106+0000 7f3bf3668640 1 --2- 172.21.15.143:0/776788476 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f3bec14b1e0 0x7f3bec0ab3f0 unknown :-1 s=CLOSED pgs=1462 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:25:00.109 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:25:00.106+0000 7f3bf3668640 1 --2- 172.21.15.143:0/776788476 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f3bec0cf7c0 0x7f3bec0aaeb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:25:00.109 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:25:00.106+0000 7f3bf3668640 1 --2- 172.21.15.143:0/776788476 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f3bec070fd0 0x7f3bec0ad950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:25:00.109 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:25:00.106+0000 7f3bf3668640 1 -- 172.21.15.143:0/776788476 >> 172.21.15.143:0/776788476 conn(0x7f3bec06a890 msgr2=0x7f3bec0b0730 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:25:00.109 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:25:00.106+0000 7f3bf3668640 1 -- 172.21.15.143:0/776788476 shutdown_connections 2023-11-26T07:25:00.109 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:25:00.106+0000 7f3bf3668640 1 -- 172.21.15.143:0/776788476 wait complete. 2023-11-26T07:25:00.129 INFO:tasks.ceph:Scrubbing osd.2 2023-11-26T07:25:00.129 DEBUG:teuthology.orchestra.run.smithi143:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.2 config set osd_debug_deep_scrub_sleep 0 2023-11-26T07:25:00.272 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:25:00.270+0000 7f1086557640 1 Processor -- start 2023-11-26T07:25:00.273 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:25:00.270+0000 7f1086557640 1 -- start start 2023-11-26T07:25:00.274 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:25:00.274+0000 7f1086557640 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7f1080070d10 0x7f108015d900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:25:00.275 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:25:00.274+0000 7f1086557640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f108006cd50 0x7f10800b2180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:25:00.275 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:25:00.274+0000 7f1086557640 1 -- --> v1:172.21.15.143:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f108015b2f0 con 0x7f10800cf280 2023-11-26T07:25:00.275 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:25:00.274+0000 7f1086557640 1 -- --> v2:172.21.15.143:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f108006c5c0 con 0x7f1080070d10 2023-11-26T07:25:00.275 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:25:00.274+0000 7f1086557640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f108015e1a0 con 0x7f108006cd50 2023-11-26T07:25:00.275 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:25:00.274+0000 7f107f7fe640 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7f1080070d10 0x7f108015d900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:25:00.275 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:25:00.274+0000 7f107ffff640 1 --1- >> v1:172.21.15.143:6789/0 conn(0x7f10800cf280 0x7f10800cf660 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.143:6789/0 says I am v1:172.21.15.143:36322/0 (socket says 172.21.15.143:36322) 2023-11-26T07:25:00.275 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:25:00.274+0000 7f107ffff640 1 -- 172.21.15.143:0/3785111020 learned_addr learned my addr 172.21.15.143:0/3785111020 (peer_addr_for_me v1:172.21.15.143:0/0) 2023-11-26T07:25:00.275 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:25:00.274+0000 7f107f7fe640 1 -- 172.21.15.143:0/3785111020 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f108006cd50 msgr2=0x7f10800b2180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-26T07:25:00.276 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:25:00.274+0000 7f1084af4640 1 --2- 172.21.15.143:0/3785111020 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f108006cd50 0x7f10800b2180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:25:00.276 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:25:00.274+0000 7f107effd640 1 -- 172.21.15.143:0/3785111020 <== mon.0 v1:172.21.15.143:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1027270927 0 0) 0x7f1070001120 con 0x7f10800cf280 2023-11-26T07:25:00.276 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:25:00.274+0000 7f107f7fe640 1 --2- 172.21.15.143:0/3785111020 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f108006cd50 0x7f10800b2180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:25:00.276 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:25:00.274+0000 7f107f7fe640 1 -- 172.21.15.143:0/3785111020 >> v1:172.21.15.143:6789/0 conn(0x7f10800cf280 legacy=0x7f10800cf660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:25:00.276 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:25:00.274+0000 7f107f7fe640 1 -- 172.21.15.143:0/3785111020 --> v2:172.21.15.143:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f108015e480 con 0x7f1080070d10 2023-11-26T07:25:00.276 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:25:00.274+0000 7f1084af4640 1 --2- 172.21.15.143:0/3785111020 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f108006cd50 0x7f10800b2180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-26T07:25:00.277 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:25:00.278+0000 7f107f7fe640 1 --2- 172.21.15.143:0/3785111020 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f1080070d10 0x7f108015d900 secure :-1 s=READY pgs=1464 cs=0 l=1 rev1=1 crypto rx=0x7f10740097c0 tx=0x7f107401ec70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b033b68de56030fa server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:25:00.278 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:25:00.278+0000 7f107effd640 1 -- 172.21.15.143:0/3785111020 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f10740a8970 con 0x7f1080070d10 2023-11-26T07:25:00.278 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:25:00.278+0000 7f107effd640 1 -- 172.21.15.143:0/3785111020 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f10740a8ad0 con 0x7f1080070d10 2023-11-26T07:25:00.278 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:25:00.278+0000 7f107effd640 1 -- 172.21.15.143:0/3785111020 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f10740a8e60 con 0x7f1080070d10 2023-11-26T07:25:00.279 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:25:00.278+0000 7f1086557640 1 -- 172.21.15.143:0/3785111020 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f1080070d10 msgr2=0x7f108015d900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:25:00.279 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:25:00.278+0000 7f1086557640 1 --2- 172.21.15.143:0/3785111020 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f1080070d10 0x7f108015d900 secure :-1 s=READY pgs=1464 cs=0 l=1 rev1=1 crypto rx=0x7f10740097c0 tx=0x7f107401ec70 comp rx=0 tx=0).stop 2023-11-26T07:25:00.279 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:25:00.278+0000 7f1086557640 1 -- 172.21.15.143:0/3785111020 shutdown_connections 2023-11-26T07:25:00.279 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:25:00.278+0000 7f1086557640 1 --2- 172.21.15.143:0/3785111020 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f108006cd50 0x7f10800b2180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:25:00.279 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:25:00.278+0000 7f1086557640 1 --2- 172.21.15.143:0/3785111020 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f1080070d10 0x7f108015d900 unknown :-1 s=CLOSED pgs=1464 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:25:00.279 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:25:00.278+0000 7f1086557640 1 -- 172.21.15.143:0/3785111020 >> 172.21.15.143:0/3785111020 conn(0x7f108006a890 msgr2=0x7f108006aca0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:25:00.280 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:25:00.278+0000 7f1086557640 1 -- 172.21.15.143:0/3785111020 shutdown_connections 2023-11-26T07:25:00.280 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:25:00.278+0000 7f1086557640 1 -- 172.21.15.143:0/3785111020 wait complete. 2023-11-26T07:25:00.280 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:25:00.278+0000 7f1086557640 1 Processor -- start 2023-11-26T07:25:00.280 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:25:00.278+0000 7f1086557640 1 -- start start 2023-11-26T07:25:00.282 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:25:00.282+0000 7f1086557640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f108006cd50 0x7f10801e6d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:25:00.283 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:25:00.282+0000 7f1086557640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f1080070d10 0x7f108014c950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:25:00.283 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:25:00.282+0000 7f1086557640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f10800cf280 0x7f10801e51e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:25:00.283 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:25:00.282+0000 7f1086557640 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f10801e5ca0 con 0x7f1080070d10 2023-11-26T07:25:00.283 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:25:00.282+0000 7f1086557640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f10801e5f80 con 0x7f10800cf280 2023-11-26T07:25:00.283 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:25:00.282+0000 7f1086557640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f10801e6260 con 0x7f108006cd50 2023-11-26T07:25:00.283 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:25:00.282+0000 7f107f7fe640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f1080070d10 0x7f108014c950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:25:00.283 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:25:00.282+0000 7f107f7fe640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f1080070d10 0x7f108014c950 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:46000/0 (socket says 172.21.15.143:46000) 2023-11-26T07:25:00.283 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:25:00.282+0000 7f107f7fe640 1 -- 172.21.15.143:0/2695129864 learned_addr learned my addr 172.21.15.143:0/2695129864 (peer_addr_for_me v2:172.21.15.143:0/0) 2023-11-26T07:25:00.284 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:25:00.282+0000 7f1084af4640 1 --2- 172.21.15.143:0/2695129864 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f10800cf280 0x7f10801e51e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:25:00.284 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:25:00.282+0000 7f107ffff640 1 --2- 172.21.15.143:0/2695129864 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f108006cd50 0x7f10801e6d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:25:00.284 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:25:00.282+0000 7f107f7fe640 1 -- 172.21.15.143:0/2695129864 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f108006cd50 msgr2=0x7f10801e6d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:25:00.284 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:25:00.282+0000 7f107f7fe640 1 --2- 172.21.15.143:0/2695129864 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f108006cd50 0x7f10801e6d40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:25:00.284 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:25:00.282+0000 7f107f7fe640 1 -- 172.21.15.143:0/2695129864 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f10800cf280 msgr2=0x7f10801e51e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:25:00.284 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:25:00.282+0000 7f107f7fe640 1 --2- 172.21.15.143:0/2695129864 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f10800cf280 0x7f10801e51e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:25:00.284 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:25:00.282+0000 7f107f7fe640 1 -- 172.21.15.143:0/2695129864 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1074009420 con 0x7f1080070d10 2023-11-26T07:25:00.285 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:25:00.282+0000 7f107f7fe640 1 --2- 172.21.15.143:0/2695129864 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f1080070d10 0x7f108014c950 secure :-1 s=READY pgs=1465 cs=0 l=1 rev1=1 crypto rx=0x7f10740b3040 tx=0x7f107401ba10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:25:00.285 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:25:00.282+0000 7f107cff9640 1 -- 172.21.15.143:0/2695129864 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f10740b9630 con 0x7f1080070d10 2023-11-26T07:25:00.285 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:25:00.282+0000 7f107cff9640 1 -- 172.21.15.143:0/2695129864 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f10740b97b0 con 0x7f1080070d10 2023-11-26T07:25:00.285 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:25:00.282+0000 7f107cff9640 1 -- 172.21.15.143:0/2695129864 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f10740b9b60 con 0x7f1080070d10 2023-11-26T07:25:00.285 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:25:00.282+0000 7f1086557640 1 -- 172.21.15.143:0/2695129864 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f10801e6540 con 0x7f1080070d10 2023-11-26T07:25:00.285 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:25:00.282+0000 7f1086557640 1 -- 172.21.15.143:0/2695129864 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f10801e7280 con 0x7f1080070d10 2023-11-26T07:25:00.287 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:25:00.282+0000 7f1086557640 1 -- 172.21.15.143:0/2695129864 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f108015e760 con 0x7f1080070d10 2023-11-26T07:25:00.287 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:25:00.282+0000 7f107cff9640 1 -- 172.21.15.143:0/2695129864 <== mon.0 v2:172.21.15.143:3300/0 4 ==== mgrmap(e 18) v1 ==== 86107+0+0 (secure 0 0 0) 0x7f10740b7040 con 0x7f1080070d10 2023-11-26T07:25:00.287 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:25:00.286+0000 7f107cff9640 1 --2- 172.21.15.143:0/2695129864 >> [v2:172.21.15.188:6836/3401128553,v1:172.21.15.188:6837/3401128553] conn(0x7f106c066740 0x7f106c068c00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:25:00.287 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:25:00.286+0000 7f107cff9640 1 -- 172.21.15.143:0/2695129864 <== mon.0 v2:172.21.15.143:3300/0 5 ==== osd_map(106..106 src has 1..106) v4 ==== 4943+0+0 (secure 0 0 0) 0x7f1074122a60 con 0x7f1080070d10 2023-11-26T07:25:00.287 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:25:00.286+0000 7f107cff9640 1 --2- 172.21.15.143:0/2695129864 >> [v2:172.21.15.143:6808/3909095186,v1:172.21.15.143:6809/3909095186] conn(0x7f106c06c5a0 0x7f106c06e9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:25:00.288 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:25:00.286+0000 7f107cff9640 1 -- 172.21.15.143:0/2695129864 --> [v2:172.21.15.143:6808/3909095186,v1:172.21.15.143:6809/3909095186] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f106c06f070 con 0x7f106c06c5a0 2023-11-26T07:25:00.288 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:25:00.286+0000 7f107cff9640 1 -- 172.21.15.143:0/2695129864 <== mon.0 v2:172.21.15.143:3300/0 6 ==== mon_get_version_reply(handle=1 version=106) v2 ==== 24+0+0 (secure 0 0 0) 0x7f1074122dd0 con 0x7f1080070d10 2023-11-26T07:25:00.288 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:25:00.286+0000 7f107ffff640 1 --2- 172.21.15.143:0/2695129864 >> [v2:172.21.15.188:6836/3401128553,v1:172.21.15.188:6837/3401128553] conn(0x7f106c066740 0x7f106c068c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:25:00.288 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:25:00.286+0000 7f1084af4640 1 --2- 172.21.15.143:0/2695129864 >> [v2:172.21.15.143:6808/3909095186,v1:172.21.15.143:6809/3909095186] conn(0x7f106c06c5a0 0x7f106c06e9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:25:00.288 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:25:00.286+0000 7f1084af4640 1 --2- 172.21.15.143:0/2695129864 >> [v2:172.21.15.143:6808/3909095186,v1:172.21.15.143:6809/3909095186] conn(0x7f106c06c5a0 0x7f106c06e9c0 crc :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:25:00.288 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:25:00.286+0000 7f107ffff640 1 --2- 172.21.15.143:0/2695129864 >> [v2:172.21.15.188:6836/3401128553,v1:172.21.15.188:6837/3401128553] conn(0x7f106c066740 0x7f106c068c00 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7f1070007fd0 tx=0x7f10700079c0 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:25:00.288 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:25:00.286+0000 7f107cff9640 1 -- 172.21.15.143:0/2695129864 <== osd.2 v2:172.21.15.143:6808/3909095186 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+26965 (crc 0 0 0) 0x7f106c06f070 con 0x7f106c06c5a0 2023-11-26T07:25:00.300 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:25:00.298+0000 7f1086557640 1 -- 172.21.15.143:0/2695129864 --> [v2:172.21.15.143:6808/3909095186,v1:172.21.15.143:6809/3909095186] -- command(tid 2: {"prefix": "config set", "var": "osd_debug_deep_scrub_sleep", "val": ["0"]}) v1 -- 0x7f108015de40 con 0x7f106c06c5a0 2023-11-26T07:25:00.300 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:25:00.298+0000 7f107cff9640 1 -- 172.21.15.143:0/2695129864 <== osd.2 v2:172.21.15.143:6808/3909095186 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+97 (crc 0 0 0) 0x7f107800e460 con 0x7f106c06c5a0 2023-11-26T07:25:00.300 INFO:teuthology.orchestra.run.smithi143.stdout:{ 2023-11-26T07:25:00.301 INFO:teuthology.orchestra.run.smithi143.stdout: "success": "osd_debug_deep_scrub_sleep = '' (not observed, change may require restart) " 2023-11-26T07:25:00.301 INFO:teuthology.orchestra.run.smithi143.stdout:} 2023-11-26T07:25:00.302 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:25:00.298+0000 7f1086557640 1 -- 172.21.15.143:0/2695129864 >> [v2:172.21.15.143:6808/3909095186,v1:172.21.15.143:6809/3909095186] conn(0x7f106c06c5a0 msgr2=0x7f106c06e9c0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:25:00.302 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:25:00.298+0000 7f1086557640 1 --2- 172.21.15.143:0/2695129864 >> [v2:172.21.15.143:6808/3909095186,v1:172.21.15.143:6809/3909095186] conn(0x7f106c06c5a0 0x7f106c06e9c0 crc :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:25:00.302 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:25:00.298+0000 7f1086557640 1 -- 172.21.15.143:0/2695129864 >> [v2:172.21.15.188:6836/3401128553,v1:172.21.15.188:6837/3401128553] conn(0x7f106c066740 msgr2=0x7f106c068c00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:25:00.302 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:25:00.298+0000 7f1086557640 1 --2- 172.21.15.143:0/2695129864 >> [v2:172.21.15.188:6836/3401128553,v1:172.21.15.188:6837/3401128553] conn(0x7f106c066740 0x7f106c068c00 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7f1070007fd0 tx=0x7f10700079c0 comp rx=0 tx=0).stop 2023-11-26T07:25:00.302 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:25:00.302+0000 7f1086557640 1 -- 172.21.15.143:0/2695129864 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f1080070d10 msgr2=0x7f108014c950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:25:00.302 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:25:00.302+0000 7f1086557640 1 --2- 172.21.15.143:0/2695129864 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f1080070d10 0x7f108014c950 secure :-1 s=READY pgs=1465 cs=0 l=1 rev1=1 crypto rx=0x7f10740b3040 tx=0x7f107401ba10 comp rx=0 tx=0).stop 2023-11-26T07:25:00.303 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:25:00.302+0000 7f1084af4640 1 -- 172.21.15.143:0/2695129864 reap_dead start 2023-11-26T07:25:00.303 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:25:00.302+0000 7f1086557640 1 -- 172.21.15.143:0/2695129864 shutdown_connections 2023-11-26T07:25:00.303 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:25:00.302+0000 7f1086557640 1 -- 172.21.15.143:0/2695129864 >> 172.21.15.143:0/2695129864 conn(0x7f108006a890 msgr2=0x7f10800aea60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:25:00.303 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:25:00.302+0000 7f1086557640 1 -- 172.21.15.143:0/2695129864 shutdown_connections 2023-11-26T07:25:00.303 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:25:00.302+0000 7f1086557640 1 -- 172.21.15.143:0/2695129864 wait complete. 2023-11-26T07:25:00.319 DEBUG:teuthology.orchestra.run.smithi143:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 2 2023-11-26T07:25:00.457 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:25:00.458+0000 7ff270816640 1 Processor -- start 2023-11-26T07:25:00.457 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:25:00.458+0000 7ff270816640 1 -- start start 2023-11-26T07:25:00.459 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:25:00.458+0000 7ff270816640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7ff26c0ce7c0 0x7ff26c15f6b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:25:00.459 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:25:00.458+0000 7ff270816640 1 -- --> v1:172.21.15.143:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff26c06c520 con 0x7ff26c13e290 2023-11-26T07:25:00.459 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:25:00.458+0000 7ff270816640 1 -- --> v1:172.21.15.188:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff26c06c800 con 0x7ff26c0cf990 2023-11-26T07:25:00.459 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:25:00.458+0000 7ff270816640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff26c06cae0 con 0x7ff26c0ce7c0 2023-11-26T07:25:00.459 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:25:00.458+0000 7ff26a59c640 1 --1- >> v1:172.21.15.143:6789/0 conn(0x7ff26c13e290 0x7ff26c0cf280 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.143:6789/0 says I am v1:172.21.15.143:36332/0 (socket says 172.21.15.143:36332) 2023-11-26T07:25:00.459 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:25:00.458+0000 7ff26a59c640 1 -- 172.21.15.143:0/1723391159 learned_addr learned my addr 172.21.15.143:0/1723391159 (peer_addr_for_me v1:172.21.15.143:0/0) 2023-11-26T07:25:00.461 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:25:00.458+0000 7ff26ad9d640 1 --2- 172.21.15.143:0/1723391159 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7ff26c0ce7c0 0x7ff26c15f6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:25:00.461 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:25:00.458+0000 7ff26959a640 1 -- 172.21.15.143:0/1723391159 <== mon.0 v1:172.21.15.143:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 307667618 0 0) 0x7ff26c06c520 con 0x7ff26c13e290 2023-11-26T07:25:00.461 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:25:00.458+0000 7ff26959a640 1 -- 172.21.15.143:0/1723391159 --> v1:172.21.15.143:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7ff254003530 con 0x7ff26c13e290 2023-11-26T07:25:00.461 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:25:00.458+0000 7ff26959a640 1 -- 172.21.15.143:0/1723391159 <== mon.0 v1:172.21.15.143:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 400185599 0 0) 0x7ff254003530 con 0x7ff26c13e290 2023-11-26T07:25:00.462 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:25:00.458+0000 7ff26959a640 1 -- 172.21.15.143:0/1723391159 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7ff26c0ce7c0 msgr2=0x7ff26c15f6b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-26T07:25:00.462 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:25:00.458+0000 7ff26959a640 1 --2- 172.21.15.143:0/1723391159 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7ff26c0ce7c0 0x7ff26c15f6b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:25:00.462 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:25:00.458+0000 7ff26959a640 1 -- 172.21.15.143:0/1723391159 >> v1:172.21.15.188:6789/0 conn(0x7ff26c0cf990 legacy=0x7ff26c15efa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:25:00.462 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:25:00.458+0000 7ff26959a640 1 -- 172.21.15.143:0/1723391159 --> v1:172.21.15.143:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff26c06cdc0 con 0x7ff26c13e290 2023-11-26T07:25:00.462 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:25:00.458+0000 7ff26959a640 1 -- 172.21.15.143:0/1723391159 <== mon.0 v1:172.21.15.143:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2137882523 0 0) 0x7ff25c001680 con 0x7ff26c13e290 2023-11-26T07:25:00.462 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:25:00.458+0000 7ff26959a640 1 -- 172.21.15.143:0/1723391159 >> v1:172.21.15.143:6789/0 conn(0x7ff26c13e290 legacy=0x7ff26c0cf280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:25:00.462 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:25:00.458+0000 7ff26959a640 1 --2- 172.21.15.143:0/1723391159 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7ff26c0ce7c0 0x7ff2540039b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:25:00.463 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:25:00.458+0000 7ff26959a640 1 --2- 172.21.15.143:0/1723391159 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7ff254003f30 0x7ff2540047d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:25:00.463 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:25:00.458+0000 7ff26959a640 1 --2- 172.21.15.143:0/1723391159 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7ff254004d10 0x7ff254005190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:25:00.463 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:25:00.458+0000 7ff26959a640 1 -- 172.21.15.143:0/1723391159 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff26c06cae0 con 0x7ff254003f30 2023-11-26T07:25:00.463 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:25:00.458+0000 7ff26959a640 1 -- 172.21.15.143:0/1723391159 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff25400ba70 con 0x7ff254004d10 2023-11-26T07:25:00.463 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:25:00.458+0000 7ff26959a640 1 -- 172.21.15.143:0/1723391159 --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff25400bc30 con 0x7ff26c0ce7c0 2023-11-26T07:25:00.463 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:25:00.458+0000 7ff269d9b640 1 --2- 172.21.15.143:0/1723391159 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7ff254003f30 0x7ff2540047d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:25:00.463 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:25:00.458+0000 7ff26a59c640 1 --2- 172.21.15.143:0/1723391159 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7ff26c0ce7c0 0x7ff2540039b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:25:00.463 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:25:00.458+0000 7ff26ad9d640 1 --2- 172.21.15.143:0/1723391159 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7ff254004d10 0x7ff254005190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:25:00.464 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:25:00.458+0000 7ff269d9b640 1 -- 172.21.15.143:0/1723391159 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7ff26c0ce7c0 msgr2=0x7ff2540039b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-26T07:25:00.464 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:25:00.458+0000 7ff269d9b640 1 --2- 172.21.15.143:0/1723391159 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7ff26c0ce7c0 0x7ff2540039b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:25:00.464 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:25:00.458+0000 7ff269d9b640 1 -- 172.21.15.143:0/1723391159 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7ff254004d10 msgr2=0x7ff254005190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-26T07:25:00.464 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:25:00.458+0000 7ff269d9b640 1 --2- 172.21.15.143:0/1723391159 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7ff254004d10 0x7ff254005190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:25:00.464 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:25:00.458+0000 7ff269d9b640 1 -- 172.21.15.143:0/1723391159 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff254003530 con 0x7ff254003f30 2023-11-26T07:25:00.465 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:25:00.462+0000 7ff269d9b640 1 --2- 172.21.15.143:0/1723391159 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7ff254003f30 0x7ff2540047d0 secure :-1 s=READY pgs=1467 cs=0 l=1 rev1=1 crypto rx=0x7ff260001780 tx=0x7ff26001c400 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b75b7a52d00c9e43 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:25:00.465 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:25:00.462+0000 7ff26959a640 1 -- 172.21.15.143:0/1723391159 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff2600052d0 con 0x7ff254003f30 2023-11-26T07:25:00.465 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:25:00.462+0000 7ff26959a640 1 -- 172.21.15.143:0/1723391159 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff2600a7050 con 0x7ff254003f30 2023-11-26T07:25:00.465 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:25:00.462+0000 7ff26959a640 1 -- 172.21.15.143:0/1723391159 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff260003600 con 0x7ff254003f30 2023-11-26T07:25:00.466 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:25:00.462+0000 7ff270816640 1 -- 172.21.15.143:0/1723391159 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7ff254003f30 msgr2=0x7ff2540047d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:25:00.466 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:25:00.462+0000 7ff270816640 1 --2- 172.21.15.143:0/1723391159 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7ff254003f30 0x7ff2540047d0 secure :-1 s=READY pgs=1467 cs=0 l=1 rev1=1 crypto rx=0x7ff260001780 tx=0x7ff26001c400 comp rx=0 tx=0).stop 2023-11-26T07:25:00.466 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:25:00.462+0000 7ff26ad9d640 1 -- 172.21.15.143:0/1723391159 reap_dead start 2023-11-26T07:25:00.466 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:25:00.462+0000 7ff270816640 1 -- 172.21.15.143:0/1723391159 shutdown_connections 2023-11-26T07:25:00.466 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:25:00.462+0000 7ff270816640 1 -- 172.21.15.143:0/1723391159 >> 172.21.15.143:0/1723391159 conn(0x7ff26c06a890 msgr2=0x7ff26c06aca0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:25:00.466 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:25:00.462+0000 7ff270816640 1 -- 172.21.15.143:0/1723391159 shutdown_connections 2023-11-26T07:25:00.466 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:25:00.462+0000 7ff270816640 1 -- 172.21.15.143:0/1723391159 wait complete. 2023-11-26T07:25:00.468 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:25:00.466+0000 7ff270816640 1 Processor -- start 2023-11-26T07:25:00.468 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:25:00.466+0000 7ff270816640 1 -- start start 2023-11-26T07:25:00.468 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:25:00.466+0000 7ff270816640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7ff254003f30 0x7ff26c138f80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:25:00.469 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:25:00.466+0000 7ff270816640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7ff26c0afe80 0x7ff26c0b02e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:25:00.469 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:25:00.466+0000 7ff270816640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7ff26c070370 0x7ff26c0707f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:25:00.469 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:25:00.466+0000 7ff270816640 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff26c0b15c0 con 0x7ff26c070370 2023-11-26T07:25:00.469 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:25:00.466+0000 7ff270816640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff26c070d30 con 0x7ff254003f30 2023-11-26T07:25:00.469 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:25:00.466+0000 7ff270816640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff26c0b19e0 con 0x7ff26c0afe80 2023-11-26T07:25:00.469 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:25:00.466+0000 7ff26ad9d640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7ff26c070370 0x7ff26c0707f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:25:00.469 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:25:00.466+0000 7ff26ad9d640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7ff26c070370 0x7ff26c0707f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:46018/0 (socket says 172.21.15.143:46018) 2023-11-26T07:25:00.470 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:25:00.466+0000 7ff26ad9d640 1 -- 172.21.15.143:0/3024115503 learned_addr learned my addr 172.21.15.143:0/3024115503 (peer_addr_for_me v2:172.21.15.143:0/0) 2023-11-26T07:25:00.470 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:25:00.466+0000 7ff269d9b640 1 --2- 172.21.15.143:0/3024115503 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7ff26c0afe80 0x7ff26c0b02e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:25:00.470 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:25:00.466+0000 7ff26a59c640 1 --2- 172.21.15.143:0/3024115503 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7ff254003f30 0x7ff26c138f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:25:00.470 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:25:00.466+0000 7ff26ad9d640 1 -- 172.21.15.143:0/3024115503 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7ff26c0afe80 msgr2=0x7ff26c0b02e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:25:00.470 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:25:00.466+0000 7ff26ad9d640 1 --2- 172.21.15.143:0/3024115503 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7ff26c0afe80 0x7ff26c0b02e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:25:00.470 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:25:00.466+0000 7ff26ad9d640 1 -- 172.21.15.143:0/3024115503 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7ff254003f30 msgr2=0x7ff26c138f80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:25:00.470 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:25:00.466+0000 7ff26ad9d640 1 --2- 172.21.15.143:0/3024115503 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7ff254003f30 0x7ff26c138f80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:25:00.471 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:25:00.466+0000 7ff26ad9d640 1 -- 172.21.15.143:0/3024115503 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff26c0b0940 con 0x7ff26c070370 2023-11-26T07:25:00.471 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:25:00.466+0000 7ff26ad9d640 1 --2- 172.21.15.143:0/3024115503 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7ff26c070370 0x7ff26c0707f0 secure :-1 s=READY pgs=1468 cs=0 l=1 rev1=1 crypto rx=0x7ff26c0b1c00 tx=0x7ff2640115a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:25:00.471 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:25:00.466+0000 7ff25affd640 1 -- 172.21.15.143:0/3024115503 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff26400e070 con 0x7ff26c070370 2023-11-26T07:25:00.471 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:25:00.466+0000 7ff25affd640 1 -- 172.21.15.143:0/3024115503 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff264011db0 con 0x7ff26c070370 2023-11-26T07:25:00.471 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:25:00.466+0000 7ff25affd640 1 -- 172.21.15.143:0/3024115503 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff2640039a0 con 0x7ff26c070370 2023-11-26T07:25:00.471 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:25:00.466+0000 7ff270816640 1 -- 172.21.15.143:0/3024115503 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff26c0b0c20 con 0x7ff26c070370 2023-11-26T07:25:00.471 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:25:00.466+0000 7ff270816640 1 -- 172.21.15.143:0/3024115503 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff26c0b11e0 con 0x7ff26c070370 2023-11-26T07:25:00.472 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:25:00.470+0000 7ff270816640 1 -- 172.21.15.143:0/3024115503 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff26c0b1700 con 0x7ff26c070370 2023-11-26T07:25:00.472 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:25:00.470+0000 7ff25affd640 1 -- 172.21.15.143:0/3024115503 <== mon.0 v2:172.21.15.143:3300/0 4 ==== mgrmap(e 18) v1 ==== 86107+0+0 (secure 0 0 0) 0x7ff264003b00 con 0x7ff26c070370 2023-11-26T07:25:00.472 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:25:00.470+0000 7ff25affd640 1 --2- 172.21.15.143:0/3024115503 >> [v2:172.21.15.188:6836/3401128553,v1:172.21.15.188:6837/3401128553] conn(0x7ff24c066770 0x7ff24c068c30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:25:00.473 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:25:00.470+0000 7ff25affd640 1 -- 172.21.15.143:0/3024115503 <== mon.0 v2:172.21.15.143:3300/0 5 ==== osd_map(106..106 src has 1..106) v4 ==== 4943+0+0 (secure 0 0 0) 0x7ff2640845a0 con 0x7ff26c070370 2023-11-26T07:25:00.473 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:25:00.470+0000 7ff26a59c640 1 --2- 172.21.15.143:0/3024115503 >> [v2:172.21.15.188:6836/3401128553,v1:172.21.15.188:6837/3401128553] conn(0x7ff24c066770 0x7ff24c068c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:25:00.473 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:25:00.470+0000 7ff26a59c640 1 --2- 172.21.15.143:0/3024115503 >> [v2:172.21.15.188:6836/3401128553,v1:172.21.15.188:6837/3401128553] conn(0x7ff24c066770 0x7ff24c068c30 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7ff25c001060 tx=0x7ff25c001c20 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:25:00.475 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:25:00.474+0000 7ff25affd640 1 -- 172.21.15.143:0/3024115503 <== mon.0 v2:172.21.15.143:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172649 (secure 0 0 0) 0x7ff26c0b1700 con 0x7ff26c070370 2023-11-26T07:25:00.624 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:25:00.622+0000 7ff270816640 1 -- 172.21.15.143:0/3024115503 --> [v2:172.21.15.188:6836/3401128553,v1:172.21.15.188:6837/3401128553] -- mgr_command(tid 0: {"prefix": "osd deep-scrub", "who": "2", "target": ["mon-mgr", ""]}) v1 -- 0x7ff26c0b1700 con 0x7ff24c066770 2023-11-26T07:25:00.625 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:25:00.626+0000 7ff25affd640 1 -- 172.21.15.143:0/3024115503 <== mgr.4108 v2:172.21.15.188:6836/3401128553 1 ==== mgr_command_reply(tid 0: 0 instructed osd(s) 2 to deep-scrub) v1 ==== 41+0+0 (secure 0 0 0) 0x7ff26c0b1700 con 0x7ff24c066770 2023-11-26T07:25:00.625 INFO:teuthology.orchestra.run.smithi143.stderr:instructed osd(s) 2 to deep-scrub 2023-11-26T07:25:00.627 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:25:00.626+0000 7ff270816640 1 -- 172.21.15.143:0/3024115503 >> [v2:172.21.15.188:6836/3401128553,v1:172.21.15.188:6837/3401128553] conn(0x7ff24c066770 msgr2=0x7ff24c068c30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:25:00.628 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:25:00.626+0000 7ff270816640 1 --2- 172.21.15.143:0/3024115503 >> [v2:172.21.15.188:6836/3401128553,v1:172.21.15.188:6837/3401128553] conn(0x7ff24c066770 0x7ff24c068c30 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7ff25c001060 tx=0x7ff25c001c20 comp rx=0 tx=0).stop 2023-11-26T07:25:00.628 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:25:00.626+0000 7ff270816640 1 -- 172.21.15.143:0/3024115503 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7ff26c070370 msgr2=0x7ff26c0707f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:25:00.628 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:25:00.626+0000 7ff270816640 1 --2- 172.21.15.143:0/3024115503 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7ff26c070370 0x7ff26c0707f0 secure :-1 s=READY pgs=1468 cs=0 l=1 rev1=1 crypto rx=0x7ff26c0b1c00 tx=0x7ff2640115a0 comp rx=0 tx=0).stop 2023-11-26T07:25:00.628 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:25:00.626+0000 7ff270816640 1 -- 172.21.15.143:0/3024115503 shutdown_connections 2023-11-26T07:25:00.628 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:25:00.626+0000 7ff270816640 1 --2- 172.21.15.143:0/3024115503 >> [v2:172.21.15.188:6836/3401128553,v1:172.21.15.188:6837/3401128553] conn(0x7ff24c066770 0x7ff24c068c30 unknown :-1 s=CLOSED pgs=192 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:25:00.629 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:25:00.626+0000 7ff270816640 1 --2- 172.21.15.143:0/3024115503 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7ff26c070370 0x7ff26c0707f0 unknown :-1 s=CLOSED pgs=1468 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:25:00.629 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:25:00.626+0000 7ff270816640 1 --2- 172.21.15.143:0/3024115503 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7ff26c0afe80 0x7ff26c0b02e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:25:00.629 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:25:00.626+0000 7ff270816640 1 --2- 172.21.15.143:0/3024115503 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7ff254003f30 0x7ff26c138f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:25:00.629 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:25:00.626+0000 7ff270816640 1 -- 172.21.15.143:0/3024115503 >> 172.21.15.143:0/3024115503 conn(0x7ff26c06a890 msgr2=0x7ff26c071d90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:25:00.629 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:25:00.626+0000 7ff270816640 1 -- 172.21.15.143:0/3024115503 shutdown_connections 2023-11-26T07:25:00.629 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:25:00.626+0000 7ff270816640 1 -- 172.21.15.143:0/3024115503 wait complete. 2023-11-26T07:25:00.646 INFO:tasks.ceph:Scrubbing osd.3 2023-11-26T07:25:00.647 DEBUG:teuthology.orchestra.run.smithi143:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.3 config set osd_debug_deep_scrub_sleep 0 2023-11-26T07:25:00.789 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:25:00.786+0000 7fa2c6a17640 1 Processor -- start 2023-11-26T07:25:00.789 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:25:00.790+0000 7fa2c6a17640 1 -- start start 2023-11-26T07:25:00.790 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:25:00.790+0000 7fa2c6a17640 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7fa2c00d3d30 0x7fa2c00cda90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:25:00.790 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:25:00.790+0000 7fa2c6a17640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fa2c0160f60 0x7fa2c01613f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:25:00.790 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:25:00.790+0000 7fa2c6a17640 1 -- --> v1:172.21.15.188:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa2c00d4170 con 0x7fa2c0160850 2023-11-26T07:25:00.791 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:25:00.790+0000 7fa2c6a17640 1 -- --> v2:172.21.15.188:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa2c0161930 con 0x7fa2c00d3d30 2023-11-26T07:25:00.791 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:25:00.790+0000 7fa2c6a17640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa2c00cdfd0 con 0x7fa2c0160f60 2023-11-26T07:25:00.792 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:25:00.790+0000 7fa2c6216640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fa2c0160f60 0x7fa2c01613f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:25:00.792 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:25:00.790+0000 7fa2c5214640 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7fa2c00d3d30 0x7fa2c00cda90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:25:00.792 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:25:00.790+0000 7fa2c6216640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fa2c0160f60 0x7fa2c01613f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.188:3301/0 says I am v2:172.21.15.143:46922/0 (socket says 172.21.15.143:46922) 2023-11-26T07:25:00.793 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:25:00.790+0000 7fa2c6216640 1 -- 172.21.15.143:0/3170368083 learned_addr learned my addr 172.21.15.143:0/3170368083 (peer_addr_for_me v2:172.21.15.143:0/0) 2023-11-26T07:25:00.793 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:25:00.790+0000 7fa2c6216640 1 -- 172.21.15.143:0/3170368083 >> v2:172.21.15.188:3300/0 conn(0x7fa2c00d3d30 msgr2=0x7fa2c00cda90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-26T07:25:00.793 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:25:00.790+0000 7fa2c6216640 1 --2- 172.21.15.143:0/3170368083 >> v2:172.21.15.188:3300/0 conn(0x7fa2c00d3d30 0x7fa2c00cda90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:25:00.793 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:25:00.790+0000 7fa2c6216640 1 -- 172.21.15.143:0/3170368083 >> v1:172.21.15.188:6789/0 conn(0x7fa2c0160850 legacy=0x7fa2c00cf330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:25:00.793 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:25:00.790+0000 7fa2c6216640 1 -- 172.21.15.143:0/3170368083 --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa2c00ce290 con 0x7fa2c0160f60 2023-11-26T07:25:00.795 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:25:00.794+0000 7fa2c6216640 1 --2- 172.21.15.143:0/3170368083 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fa2c0160f60 0x7fa2c01613f0 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7fa2b8009e50 tx=0x7fa2b801c9e0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=b33351088f4f8af6 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:25:00.795 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:25:00.794+0000 7fa2c4a13640 1 -- 172.21.15.143:0/3170368083 <== mon.2 v2:172.21.15.188:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa2b80a58a0 con 0x7fa2c0160f60 2023-11-26T07:25:00.795 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:25:00.794+0000 7fa2c4a13640 1 -- 172.21.15.143:0/3170368083 <== mon.2 v2:172.21.15.188:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa2b80a5a00 con 0x7fa2c0160f60 2023-11-26T07:25:00.795 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:25:00.794+0000 7fa2c4a13640 1 -- 172.21.15.143:0/3170368083 <== mon.2 v2:172.21.15.188:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa2b80a5d90 con 0x7fa2c0160f60 2023-11-26T07:25:00.796 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:25:00.794+0000 7fa2c6a17640 1 -- 172.21.15.143:0/3170368083 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fa2c0160f60 msgr2=0x7fa2c01613f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:25:00.796 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:25:00.794+0000 7fa2c6a17640 1 --2- 172.21.15.143:0/3170368083 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fa2c0160f60 0x7fa2c01613f0 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7fa2b8009e50 tx=0x7fa2b801c9e0 comp rx=0 tx=0).stop 2023-11-26T07:25:00.797 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:25:00.794+0000 7fa2c6a17640 1 -- 172.21.15.143:0/3170368083 shutdown_connections 2023-11-26T07:25:00.797 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:25:00.794+0000 7fa2c6a17640 1 --2- 172.21.15.143:0/3170368083 >> v2:172.21.15.188:3300/0 conn(0x7fa2c00d3d30 0x7fa2c00cda90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:25:00.797 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:25:00.794+0000 7fa2c6a17640 1 --2- 172.21.15.143:0/3170368083 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fa2c0160f60 0x7fa2c01613f0 unknown :-1 s=CLOSED pgs=170 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:25:00.797 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:25:00.794+0000 7fa2c6a17640 1 -- 172.21.15.143:0/3170368083 >> 172.21.15.143:0/3170368083 conn(0x7fa2c009c5a0 msgr2=0x7fa2c00d3260 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:25:00.797 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:25:00.794+0000 7fa2c6a17640 1 -- 172.21.15.143:0/3170368083 shutdown_connections 2023-11-26T07:25:00.797 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:25:00.794+0000 7fa2c6a17640 1 -- 172.21.15.143:0/3170368083 wait complete. 2023-11-26T07:25:00.800 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:25:00.798+0000 7fa2c6a17640 1 Processor -- start 2023-11-26T07:25:00.800 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:25:00.798+0000 7fa2c6a17640 1 -- start start 2023-11-26T07:25:00.800 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:25:00.798+0000 7fa2c6a17640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fa2c00d3d30 0x7fa2c015b540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:25:00.800 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:25:00.798+0000 7fa2c6a17640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fa2c0160850 0x7fa2c0158940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:25:00.801 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:25:00.798+0000 7fa2c6a17640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fa2c0160f60 0x7fa2c00cb5b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:25:00.801 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:25:00.798+0000 7fa2c6a17640 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa2c015ba80 con 0x7fa2c00d3d30 2023-11-26T07:25:00.801 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:25:00.798+0000 7fa2c6a17640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa2c00b89c0 con 0x7fa2c0160850 2023-11-26T07:25:00.801 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:25:00.798+0000 7fa2c6a17640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa2c00b8150 con 0x7fa2c0160f60 2023-11-26T07:25:00.801 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:25:00.798+0000 7fa2c5a15640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fa2c00d3d30 0x7fa2c015b540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:25:00.801 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:25:00.798+0000 7fa2c5214640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fa2c0160850 0x7fa2c0158940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:25:00.802 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:25:00.798+0000 7fa2c5a15640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fa2c00d3d30 0x7fa2c015b540 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:46020/0 (socket says 172.21.15.143:46020) 2023-11-26T07:25:00.802 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:25:00.798+0000 7fa2c5a15640 1 -- 172.21.15.143:0/2788363447 learned_addr learned my addr 172.21.15.143:0/2788363447 (peer_addr_for_me v2:172.21.15.143:0/0) 2023-11-26T07:25:00.802 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:25:00.798+0000 7fa2c6216640 1 --2- 172.21.15.143:0/2788363447 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fa2c0160f60 0x7fa2c00cb5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:25:00.802 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:25:00.798+0000 7fa2c5a15640 1 -- 172.21.15.143:0/2788363447 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fa2c0160f60 msgr2=0x7fa2c00cb5b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:25:00.802 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:25:00.798+0000 7fa2c5a15640 1 --2- 172.21.15.143:0/2788363447 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fa2c0160f60 0x7fa2c00cb5b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:25:00.803 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:25:00.798+0000 7fa2c5a15640 1 -- 172.21.15.143:0/2788363447 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fa2c0160850 msgr2=0x7fa2c0158940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:25:00.803 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:25:00.798+0000 7fa2c5a15640 1 --2- 172.21.15.143:0/2788363447 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fa2c0160850 0x7fa2c0158940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:25:00.803 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:25:00.798+0000 7fa2c5a15640 1 -- 172.21.15.143:0/2788363447 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa2b8009b30 con 0x7fa2c00d3d30 2023-11-26T07:25:00.803 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:25:00.798+0000 7fa2c5a15640 1 --2- 172.21.15.143:0/2788363447 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fa2c00d3d30 0x7fa2c015b540 secure :-1 s=READY pgs=1469 cs=0 l=1 rev1=1 crypto rx=0x7fa2bc00c500 tx=0x7fa2bc00c9c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:25:00.803 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:25:00.798+0000 7fa2b67fc640 1 -- 172.21.15.143:0/2788363447 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa2bc005220 con 0x7fa2c00d3d30 2023-11-26T07:25:00.803 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:25:00.798+0000 7fa2b67fc640 1 -- 172.21.15.143:0/2788363447 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa2bc005380 con 0x7fa2c00d3d30 2023-11-26T07:25:00.803 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:25:00.798+0000 7fa2b67fc640 1 -- 172.21.15.143:0/2788363447 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa2bc007270 con 0x7fa2c00d3d30 2023-11-26T07:25:00.804 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:25:00.798+0000 7fa2c6a17640 1 -- 172.21.15.143:0/2788363447 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa2c00b8400 con 0x7fa2c00d3d30 2023-11-26T07:25:00.804 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:25:00.798+0000 7fa2c6a17640 1 -- 172.21.15.143:0/2788363447 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa2c00b5ff0 con 0x7fa2c00d3d30 2023-11-26T07:25:00.805 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:25:00.798+0000 7fa2c6a17640 1 -- 172.21.15.143:0/2788363447 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7fa2c00d2390 con 0x7fa2c00d3d30 2023-11-26T07:25:00.805 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:25:00.802+0000 7fa2b67fc640 1 -- 172.21.15.143:0/2788363447 <== mon.0 v2:172.21.15.143:3300/0 4 ==== mgrmap(e 18) v1 ==== 86107+0+0 (secure 0 0 0) 0x7fa2bc0073d0 con 0x7fa2c00d3d30 2023-11-26T07:25:00.805 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:25:00.802+0000 7fa2b67fc640 1 --2- 172.21.15.143:0/2788363447 >> [v2:172.21.15.188:6836/3401128553,v1:172.21.15.188:6837/3401128553] conn(0x7fa2a4066740 0x7fa2a4068c00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:25:00.805 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:25:00.802+0000 7fa2b67fc640 1 -- 172.21.15.143:0/2788363447 <== mon.0 v2:172.21.15.143:3300/0 5 ==== osd_map(106..106 src has 1..106) v4 ==== 4943+0+0 (secure 0 0 0) 0x7fa2bc083810 con 0x7fa2c00d3d30 2023-11-26T07:25:00.806 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:25:00.802+0000 7fa2b67fc640 1 --2- 172.21.15.143:0/2788363447 >> [v2:172.21.15.143:6800/3854210991,v1:172.21.15.143:6801/3854210991] conn(0x7fa2a406c5a0 0x7fa2a406e9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:25:00.806 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:25:00.802+0000 7fa2b67fc640 1 -- 172.21.15.143:0/2788363447 --> [v2:172.21.15.143:6800/3854210991,v1:172.21.15.143:6801/3854210991] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7fa2a406f070 con 0x7fa2a406c5a0 2023-11-26T07:25:00.806 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:25:00.802+0000 7fa2b67fc640 1 -- 172.21.15.143:0/2788363447 <== mon.0 v2:172.21.15.143:3300/0 6 ==== mon_get_version_reply(handle=1 version=106) v2 ==== 24+0+0 (secure 0 0 0) 0x7fa2bc0882a0 con 0x7fa2c00d3d30 2023-11-26T07:25:00.806 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:25:00.802+0000 7fa2c5214640 1 --2- 172.21.15.143:0/2788363447 >> [v2:172.21.15.188:6836/3401128553,v1:172.21.15.188:6837/3401128553] conn(0x7fa2a4066740 0x7fa2a4068c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:25:00.806 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:25:00.802+0000 7fa2c6216640 1 --2- 172.21.15.143:0/2788363447 >> [v2:172.21.15.143:6800/3854210991,v1:172.21.15.143:6801/3854210991] conn(0x7fa2a406c5a0 0x7fa2a406e9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:25:00.806 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:25:00.802+0000 7fa2c5214640 1 --2- 172.21.15.143:0/2788363447 >> [v2:172.21.15.188:6836/3401128553,v1:172.21.15.188:6837/3401128553] conn(0x7fa2a4066740 0x7fa2a4068c00 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7fa2c0159670 tx=0x7fa2b0009190 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:25:00.807 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:25:00.802+0000 7fa2c6216640 1 --2- 172.21.15.143:0/2788363447 >> [v2:172.21.15.143:6800/3854210991,v1:172.21.15.143:6801/3854210991] conn(0x7fa2a406c5a0 0x7fa2a406e9c0 crc :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:25:00.807 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:25:00.802+0000 7fa2b67fc640 1 -- 172.21.15.143:0/2788363447 <== osd.3 v2:172.21.15.143:6800/3854210991 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+26965 (crc 0 0 0) 0x7fa2a406f070 con 0x7fa2a406c5a0 2023-11-26T07:25:00.817 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:25:00.818+0000 7fa2c6a17640 1 -- 172.21.15.143:0/2788363447 --> [v2:172.21.15.143:6800/3854210991,v1:172.21.15.143:6801/3854210991] -- command(tid 2: {"prefix": "config set", "var": "osd_debug_deep_scrub_sleep", "val": ["0"]}) v1 -- 0x7fa2c0161370 con 0x7fa2a406c5a0 2023-11-26T07:25:00.817 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:25:00.818+0000 7fa2b67fc640 1 -- 172.21.15.143:0/2788363447 <== osd.3 v2:172.21.15.143:6800/3854210991 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+97 (crc 0 0 0) 0x7fa2c0161370 con 0x7fa2a406c5a0 2023-11-26T07:25:00.818 INFO:teuthology.orchestra.run.smithi143.stdout:{ 2023-11-26T07:25:00.818 INFO:teuthology.orchestra.run.smithi143.stdout: "success": "osd_debug_deep_scrub_sleep = '' (not observed, change may require restart) " 2023-11-26T07:25:00.818 INFO:teuthology.orchestra.run.smithi143.stdout:} 2023-11-26T07:25:00.819 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:25:00.818+0000 7fa2c6a17640 1 -- 172.21.15.143:0/2788363447 >> [v2:172.21.15.143:6800/3854210991,v1:172.21.15.143:6801/3854210991] conn(0x7fa2a406c5a0 msgr2=0x7fa2a406e9c0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:25:00.819 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:25:00.818+0000 7fa2c6a17640 1 --2- 172.21.15.143:0/2788363447 >> [v2:172.21.15.143:6800/3854210991,v1:172.21.15.143:6801/3854210991] conn(0x7fa2a406c5a0 0x7fa2a406e9c0 crc :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:25:00.819 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:25:00.818+0000 7fa2c6a17640 1 -- 172.21.15.143:0/2788363447 >> [v2:172.21.15.188:6836/3401128553,v1:172.21.15.188:6837/3401128553] conn(0x7fa2a4066740 msgr2=0x7fa2a4068c00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:25:00.819 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:25:00.818+0000 7fa2c6a17640 1 --2- 172.21.15.143:0/2788363447 >> [v2:172.21.15.188:6836/3401128553,v1:172.21.15.188:6837/3401128553] conn(0x7fa2a4066740 0x7fa2a4068c00 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7fa2c0159670 tx=0x7fa2b0009190 comp rx=0 tx=0).stop 2023-11-26T07:25:00.819 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:25:00.818+0000 7fa2c6a17640 1 -- 172.21.15.143:0/2788363447 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fa2c00d3d30 msgr2=0x7fa2c015b540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:25:00.820 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:25:00.818+0000 7fa2c6a17640 1 --2- 172.21.15.143:0/2788363447 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fa2c00d3d30 0x7fa2c015b540 secure :-1 s=READY pgs=1469 cs=0 l=1 rev1=1 crypto rx=0x7fa2bc00c500 tx=0x7fa2bc00c9c0 comp rx=0 tx=0).stop 2023-11-26T07:25:00.820 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:25:00.818+0000 7fa2c6216640 1 -- 172.21.15.143:0/2788363447 reap_dead start 2023-11-26T07:25:00.820 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:25:00.818+0000 7fa2c6a17640 1 -- 172.21.15.143:0/2788363447 shutdown_connections 2023-11-26T07:25:00.820 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:25:00.818+0000 7fa2c6a17640 1 -- 172.21.15.143:0/2788363447 >> 172.21.15.143:0/2788363447 conn(0x7fa2c009c5a0 msgr2=0x7fa2c015f990 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:25:00.820 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:25:00.818+0000 7fa2c6a17640 1 -- 172.21.15.143:0/2788363447 shutdown_connections 2023-11-26T07:25:00.820 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:25:00.818+0000 7fa2c6a17640 1 -- 172.21.15.143:0/2788363447 wait complete. 2023-11-26T07:25:00.836 DEBUG:teuthology.orchestra.run.smithi143:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 3 2023-11-26T07:25:00.974 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:25:00.974+0000 7f88bf3aa640 1 Processor -- start 2023-11-26T07:25:00.975 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:25:00.974+0000 7f88bf3aa640 1 -- start start 2023-11-26T07:25:00.976 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:25:00.974+0000 7f88bf3aa640 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7f88b80b2160 0x7f88b80d0df0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:25:00.976 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:25:00.974+0000 7f88bf3aa640 1 -- --> v1:172.21.15.143:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f88b80cf0e0 con 0x7f88b80d1bc0 2023-11-26T07:25:00.976 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:25:00.974+0000 7f88bf3aa640 1 -- --> v1:172.21.15.188:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f88b80ceb20 con 0x7f88b815d6e0 2023-11-26T07:25:00.976 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:25:00.974+0000 7f88bf3aa640 1 -- --> v2:172.21.15.143:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f88b815dac0 con 0x7f88b80b2160 2023-11-26T07:25:00.977 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:25:00.974+0000 7f88bd947640 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7f88b80b2160 0x7f88b80d0df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:25:00.977 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:25:00.974+0000 7f88bc945640 1 --1- >> v1:172.21.15.143:6789/0 conn(0x7f88b80d1bc0 0x7f88b80b0750 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.143:6789/0 says I am v1:172.21.15.143:36346/0 (socket says 172.21.15.143:36346) 2023-11-26T07:25:00.977 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:25:00.974+0000 7f88bc945640 1 -- 172.21.15.143:0/2835774458 learned_addr learned my addr 172.21.15.143:0/2835774458 (peer_addr_for_me v1:172.21.15.143:0/0) 2023-11-26T07:25:00.977 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:25:00.974+0000 7f88bd947640 1 -- 172.21.15.143:0/2835774458 >> v1:172.21.15.188:6789/0 conn(0x7f88b815d6e0 legacy=0x7f88b80d14b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:25:00.978 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:25:00.974+0000 7f88bd947640 1 -- 172.21.15.143:0/2835774458 >> v1:172.21.15.143:6789/0 conn(0x7f88b80d1bc0 legacy=0x7f88b80b0750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:25:00.978 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:25:00.974+0000 7f88bd947640 1 -- 172.21.15.143:0/2835774458 --> v2:172.21.15.143:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f88b80cee00 con 0x7f88b80b2160 2023-11-26T07:25:00.978 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:25:00.974+0000 7f88affff640 1 -- 172.21.15.143:0/2835774458 <== mon.0 v1:172.21.15.143:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2597757094 0 0) 0x7f88b80cf0e0 con 0x7f88b80d1bc0 2023-11-26T07:25:00.979 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:25:00.978+0000 7f88bd947640 1 --2- 172.21.15.143:0/2835774458 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f88b80b2160 0x7f88b80d0df0 secure :-1 s=READY pgs=1471 cs=0 l=1 rev1=1 crypto rx=0x7f88b0009d90 tx=0x7f88b001c950 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a5d8c5705db419ae server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:25:00.980 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:25:00.978+0000 7f88affff640 1 -- 172.21.15.143:0/2835774458 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f88b00a5900 con 0x7f88b80b2160 2023-11-26T07:25:00.980 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:25:00.978+0000 7f88affff640 1 -- 172.21.15.143:0/2835774458 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f88b00a5a60 con 0x7f88b80b2160 2023-11-26T07:25:00.980 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:25:00.978+0000 7f88affff640 1 -- 172.21.15.143:0/2835774458 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f88b00a5df0 con 0x7f88b80b2160 2023-11-26T07:25:00.981 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:25:00.978+0000 7f88bf3aa640 1 -- 172.21.15.143:0/2835774458 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f88b80b2160 msgr2=0x7f88b80d0df0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:25:00.981 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:25:00.978+0000 7f88bf3aa640 1 --2- 172.21.15.143:0/2835774458 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f88b80b2160 0x7f88b80d0df0 secure :-1 s=READY pgs=1471 cs=0 l=1 rev1=1 crypto rx=0x7f88b0009d90 tx=0x7f88b001c950 comp rx=0 tx=0).stop 2023-11-26T07:25:00.981 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:25:00.978+0000 7f88bf3aa640 1 -- 172.21.15.143:0/2835774458 shutdown_connections 2023-11-26T07:25:00.981 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:25:00.978+0000 7f88bf3aa640 1 --2- 172.21.15.143:0/2835774458 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f88b80b2160 0x7f88b80d0df0 unknown :-1 s=CLOSED pgs=1471 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:25:00.981 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:25:00.978+0000 7f88bf3aa640 1 -- 172.21.15.143:0/2835774458 >> 172.21.15.143:0/2835774458 conn(0x7f88b809c550 msgr2=0x7f88b80d3220 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:25:00.981 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:25:00.978+0000 7f88bf3aa640 1 -- 172.21.15.143:0/2835774458 shutdown_connections 2023-11-26T07:25:00.982 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:25:00.978+0000 7f88bf3aa640 1 -- 172.21.15.143:0/2835774458 wait complete. 2023-11-26T07:25:00.982 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:25:00.982+0000 7f88bf3aa640 1 Processor -- start 2023-11-26T07:25:00.982 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:25:00.982+0000 7f88bf3aa640 1 -- start start 2023-11-26T07:25:00.984 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:25:00.982+0000 7f88bf3aa640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f88b80b2160 0x7f88b8153d00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:25:00.984 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:25:00.982+0000 7f88bf3aa640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f88b80d1bc0 0x7f88b80cb510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:25:00.985 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:25:00.982+0000 7f88bf3aa640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f88b815d6e0 0x7f88b80c8bc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:25:00.985 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:25:00.982+0000 7f88bf3aa640 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f88b80cb080 con 0x7f88b815d6e0 2023-11-26T07:25:00.986 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:25:00.982+0000 7f88bf3aa640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f88b8154240 con 0x7f88b80d1bc0 2023-11-26T07:25:00.986 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:25:00.982+0000 7f88bf3aa640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f88b80ca470 con 0x7f88b80b2160 2023-11-26T07:25:00.986 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:25:00.982+0000 7f88bd947640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f88b815d6e0 0x7f88b80c8bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:25:00.986 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:25:00.982+0000 7f88bd947640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f88b815d6e0 0x7f88b80c8bc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:46036/0 (socket says 172.21.15.143:46036) 2023-11-26T07:25:00.987 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:25:00.982+0000 7f88bd947640 1 -- 172.21.15.143:0/1021995614 learned_addr learned my addr 172.21.15.143:0/1021995614 (peer_addr_for_me v2:172.21.15.143:0/0) 2023-11-26T07:25:00.987 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:25:00.982+0000 7f88bc945640 1 --2- 172.21.15.143:0/1021995614 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f88b80d1bc0 0x7f88b80cb510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:25:00.987 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:25:00.982+0000 7f88bd146640 1 --2- 172.21.15.143:0/1021995614 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f88b80b2160 0x7f88b8153d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:25:00.987 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:25:00.982+0000 7f88bd947640 1 -- 172.21.15.143:0/1021995614 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f88b80b2160 msgr2=0x7f88b8153d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:25:00.987 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:25:00.982+0000 7f88bd947640 1 --2- 172.21.15.143:0/1021995614 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f88b80b2160 0x7f88b8153d00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:25:00.987 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:25:00.982+0000 7f88bd947640 1 -- 172.21.15.143:0/1021995614 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f88b80d1bc0 msgr2=0x7f88b80cb510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:25:00.988 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:25:00.982+0000 7f88bd947640 1 --2- 172.21.15.143:0/1021995614 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f88b80d1bc0 0x7f88b80cb510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:25:00.988 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:25:00.982+0000 7f88bd947640 1 -- 172.21.15.143:0/1021995614 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f88b0009a70 con 0x7f88b815d6e0 2023-11-26T07:25:00.988 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:25:00.982+0000 7f88bc945640 1 --2- 172.21.15.143:0/1021995614 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f88b80d1bc0 0x7f88b80cb510 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-26T07:25:00.988 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:25:00.982+0000 7f88bd947640 1 --2- 172.21.15.143:0/1021995614 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f88b815d6e0 0x7f88b80c8bc0 secure :-1 s=READY pgs=1472 cs=0 l=1 rev1=1 crypto rx=0x7f88b001a580 tx=0x7f88b00196f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:25:00.988 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:25:00.982+0000 7f88adffb640 1 -- 172.21.15.143:0/1021995614 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f88b00ad950 con 0x7f88b815d6e0 2023-11-26T07:25:00.988 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:25:00.982+0000 7f88adffb640 1 -- 172.21.15.143:0/1021995614 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f88b00adab0 con 0x7f88b815d6e0 2023-11-26T07:25:00.988 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:25:00.982+0000 7f88adffb640 1 -- 172.21.15.143:0/1021995614 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f88b00ade40 con 0x7f88b815d6e0 2023-11-26T07:25:00.988 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:25:00.982+0000 7f88bf3aa640 1 -- 172.21.15.143:0/1021995614 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f88b80ca750 con 0x7f88b815d6e0 2023-11-26T07:25:00.989 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:25:00.982+0000 7f88bf3aa640 1 -- 172.21.15.143:0/1021995614 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f88b80cad10 con 0x7f88b815d6e0 2023-11-26T07:25:00.990 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:25:00.986+0000 7f88bf3aa640 1 -- 172.21.15.143:0/1021995614 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f88b8067680 con 0x7f88b815d6e0 2023-11-26T07:25:00.990 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:25:00.986+0000 7f88adffb640 1 -- 172.21.15.143:0/1021995614 <== mon.0 v2:172.21.15.143:3300/0 4 ==== mgrmap(e 18) v1 ==== 86107+0+0 (secure 0 0 0) 0x7f88b00c4460 con 0x7f88b815d6e0 2023-11-26T07:25:00.990 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:25:00.986+0000 7f88adffb640 1 --2- 172.21.15.143:0/1021995614 >> [v2:172.21.15.188:6836/3401128553,v1:172.21.15.188:6837/3401128553] conn(0x7f889c066740 0x7f889c068c00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:25:00.990 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:25:00.986+0000 7f88adffb640 1 -- 172.21.15.143:0/1021995614 <== mon.0 v2:172.21.15.143:3300/0 5 ==== osd_map(106..106 src has 1..106) v4 ==== 4943+0+0 (secure 0 0 0) 0x7f88b011f4e0 con 0x7f88b815d6e0 2023-11-26T07:25:00.990 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:25:00.986+0000 7f88bd146640 1 --2- 172.21.15.143:0/1021995614 >> [v2:172.21.15.188:6836/3401128553,v1:172.21.15.188:6837/3401128553] conn(0x7f889c066740 0x7f889c068c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:25:00.990 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:25:00.986+0000 7f88bd146640 1 --2- 172.21.15.143:0/1021995614 >> [v2:172.21.15.188:6836/3401128553,v1:172.21.15.188:6837/3401128553] conn(0x7f889c066740 0x7f889c068c00 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7f88b80b29f0 tx=0x7f88b4009040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:25:00.991 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:25:00.990+0000 7f88adffb640 1 -- 172.21.15.143:0/1021995614 <== mon.0 v2:172.21.15.143:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172649 (secure 0 0 0) 0x7f88b00ec9a0 con 0x7f88b815d6e0 2023-11-26T07:25:01.141 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:25:01.138+0000 7f88bf3aa640 1 -- 172.21.15.143:0/1021995614 --> [v2:172.21.15.188:6836/3401128553,v1:172.21.15.188:6837/3401128553] -- mgr_command(tid 0: {"prefix": "osd deep-scrub", "who": "3", "target": ["mon-mgr", ""]}) v1 -- 0x7f88b80cb1c0 con 0x7f889c066740 2023-11-26T07:25:01.142 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:25:01.138+0000 7f88adffb640 1 -- 172.21.15.143:0/1021995614 <== mgr.4108 v2:172.21.15.188:6836/3401128553 1 ==== mgr_command_reply(tid 0: 0 instructed osd(s) 3 to deep-scrub) v1 ==== 41+0+0 (secure 0 0 0) 0x7f88b80cb1c0 con 0x7f889c066740 2023-11-26T07:25:01.142 INFO:teuthology.orchestra.run.smithi143.stderr:instructed osd(s) 3 to deep-scrub 2023-11-26T07:25:01.143 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:25:01.142+0000 7f88bf3aa640 1 -- 172.21.15.143:0/1021995614 >> [v2:172.21.15.188:6836/3401128553,v1:172.21.15.188:6837/3401128553] conn(0x7f889c066740 msgr2=0x7f889c068c00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:25:01.143 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:25:01.142+0000 7f88bf3aa640 1 --2- 172.21.15.143:0/1021995614 >> [v2:172.21.15.188:6836/3401128553,v1:172.21.15.188:6837/3401128553] conn(0x7f889c066740 0x7f889c068c00 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7f88b80b29f0 tx=0x7f88b4009040 comp rx=0 tx=0).stop 2023-11-26T07:25:01.143 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:25:01.142+0000 7f88bf3aa640 1 -- 172.21.15.143:0/1021995614 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f88b815d6e0 msgr2=0x7f88b80c8bc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:25:01.143 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:25:01.142+0000 7f88bf3aa640 1 --2- 172.21.15.143:0/1021995614 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f88b815d6e0 0x7f88b80c8bc0 secure :-1 s=READY pgs=1472 cs=0 l=1 rev1=1 crypto rx=0x7f88b001a580 tx=0x7f88b00196f0 comp rx=0 tx=0).stop 2023-11-26T07:25:01.143 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:25:01.142+0000 7f88bf3aa640 1 -- 172.21.15.143:0/1021995614 shutdown_connections 2023-11-26T07:25:01.144 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:25:01.142+0000 7f88bf3aa640 1 --2- 172.21.15.143:0/1021995614 >> [v2:172.21.15.188:6836/3401128553,v1:172.21.15.188:6837/3401128553] conn(0x7f889c066740 0x7f889c068c00 unknown :-1 s=CLOSED pgs=194 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:25:01.144 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:25:01.142+0000 7f88bf3aa640 1 --2- 172.21.15.143:0/1021995614 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f88b815d6e0 0x7f88b80c8bc0 unknown :-1 s=CLOSED pgs=1472 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:25:01.144 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:25:01.142+0000 7f88bf3aa640 1 --2- 172.21.15.143:0/1021995614 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f88b80d1bc0 0x7f88b80cb510 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:25:01.144 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:25:01.142+0000 7f88bf3aa640 1 --2- 172.21.15.143:0/1021995614 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f88b80b2160 0x7f88b8153d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:25:01.144 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:25:01.142+0000 7f88bf3aa640 1 -- 172.21.15.143:0/1021995614 >> 172.21.15.143:0/1021995614 conn(0x7f88b809c550 msgr2=0x7f88b80d3220 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:25:01.144 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:25:01.142+0000 7f88bf3aa640 1 -- 172.21.15.143:0/1021995614 shutdown_connections 2023-11-26T07:25:01.144 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:25:01.142+0000 7f88bf3aa640 1 -- 172.21.15.143:0/1021995614 wait complete. 2023-11-26T07:25:01.162 INFO:tasks.ceph:Scrubbing osd.4 2023-11-26T07:25:01.162 DEBUG:teuthology.orchestra.run.smithi143:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.4 config set osd_debug_deep_scrub_sleep 0 2023-11-26T07:25:01.303 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:25:01.302+0000 7f6be1772640 1 Processor -- start 2023-11-26T07:25:01.304 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:25:01.302+0000 7f6be1772640 1 -- start start 2023-11-26T07:25:01.305 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:25:01.302+0000 7f6be1772640 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7f6bdc0cf990 0x7f6bdc15efa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:25:01.305 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:25:01.302+0000 7f6be1772640 1 -- --> v1:172.21.15.143:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6bdc06c520 con 0x7f6bdc13e290 2023-11-26T07:25:01.305 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:25:01.302+0000 7f6be1772640 1 -- --> v1:172.21.15.188:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6bdc06c800 con 0x7f6bdc15f4e0 2023-11-26T07:25:01.305 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:25:01.302+0000 7f6be1772640 1 -- --> v2:172.21.15.188:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6bdc06cae0 con 0x7f6bdc0cf990 2023-11-26T07:25:01.305 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:25:01.306+0000 7f6bdaffd640 1 --1- >> v1:172.21.15.143:6789/0 conn(0x7f6bdc13e290 0x7f6bdc0cf280 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.143:6789/0 says I am v1:172.21.15.143:36360/0 (socket says 172.21.15.143:36360) 2023-11-26T07:25:01.305 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:25:01.306+0000 7f6bdaffd640 1 -- 172.21.15.143:0/3837635584 learned_addr learned my addr 172.21.15.143:0/3837635584 (peer_addr_for_me v1:172.21.15.143:0/0) 2023-11-26T07:25:01.307 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:25:01.306+0000 7f6bd3fff640 1 --2- 172.21.15.143:0/3837635584 >> v2:172.21.15.188:3300/0 conn(0x7f6bdc0cf990 0x7f6bdc15efa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:25:01.307 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:25:01.306+0000 7f6bda7fc640 1 -- 172.21.15.143:0/3837635584 <== mon.0 v1:172.21.15.143:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4250124475 0 0) 0x7f6bdc06c520 con 0x7f6bdc13e290 2023-11-26T07:25:01.307 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:25:01.306+0000 7f6bda7fc640 1 -- 172.21.15.143:0/3837635584 --> v1:172.21.15.143:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f6bc8003530 con 0x7f6bdc13e290 2023-11-26T07:25:01.307 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:25:01.306+0000 7f6bda7fc640 1 -- 172.21.15.143:0/3837635584 <== mon.0 v1:172.21.15.143:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1383654407 0 0) 0x7f6bc8003530 con 0x7f6bdc13e290 2023-11-26T07:25:01.308 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:25:01.306+0000 7f6bda7fc640 1 -- 172.21.15.143:0/3837635584 >> v2:172.21.15.188:3300/0 conn(0x7f6bdc0cf990 msgr2=0x7f6bdc15efa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-26T07:25:01.308 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:25:01.306+0000 7f6bda7fc640 1 --2- 172.21.15.143:0/3837635584 >> v2:172.21.15.188:3300/0 conn(0x7f6bdc0cf990 0x7f6bdc15efa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:25:01.308 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:25:01.306+0000 7f6bda7fc640 1 -- 172.21.15.143:0/3837635584 >> v1:172.21.15.188:6789/0 conn(0x7f6bdc15f4e0 legacy=0x7f6bdc0cd7b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:25:01.308 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:25:01.306+0000 7f6bda7fc640 1 -- 172.21.15.143:0/3837635584 --> v1:172.21.15.143:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6bdc06cdc0 con 0x7f6bdc13e290 2023-11-26T07:25:01.308 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:25:01.306+0000 7f6bda7fc640 1 -- 172.21.15.143:0/3837635584 <== mon.0 v1:172.21.15.143:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2137882523 0 0) 0x7f6bcc001680 con 0x7f6bdc13e290 2023-11-26T07:25:01.308 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:25:01.306+0000 7f6bda7fc640 1 -- 172.21.15.143:0/3837635584 >> v1:172.21.15.143:6789/0 conn(0x7f6bdc13e290 legacy=0x7f6bdc0cf280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:25:01.308 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:25:01.306+0000 7f6bda7fc640 1 --2- 172.21.15.143:0/3837635584 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f6bc8003980 0x7f6bc8003da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:25:01.309 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:25:01.306+0000 7f6bda7fc640 1 --2- 172.21.15.143:0/3837635584 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f6bc80047d0 0x7f6bc8004c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:25:01.309 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:25:01.306+0000 7f6bda7fc640 1 --2- 172.21.15.143:0/3837635584 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f6bc8005190 0x7f6bc800b990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:25:01.309 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:25:01.306+0000 7f6bda7fc640 1 -- 172.21.15.143:0/3837635584 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6bdc06cae0 con 0x7f6bc8005190 2023-11-26T07:25:01.309 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:25:01.306+0000 7f6bda7fc640 1 -- 172.21.15.143:0/3837635584 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6bc800c030 con 0x7f6bc80047d0 2023-11-26T07:25:01.309 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:25:01.306+0000 7f6bda7fc640 1 -- 172.21.15.143:0/3837635584 --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6bc800c1f0 con 0x7f6bc8003980 2023-11-26T07:25:01.309 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:25:01.306+0000 7f6bdb7fe640 1 --2- 172.21.15.143:0/3837635584 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f6bc8005190 0x7f6bc800b990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:25:01.309 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:25:01.306+0000 7f6bdaffd640 1 --2- 172.21.15.143:0/3837635584 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f6bc8003980 0x7f6bc8003da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:25:01.309 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:25:01.306+0000 7f6bd3fff640 1 --2- 172.21.15.143:0/3837635584 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f6bc80047d0 0x7f6bc8004c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:25:01.310 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:25:01.306+0000 7f6bdb7fe640 1 -- 172.21.15.143:0/3837635584 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f6bc8003980 msgr2=0x7f6bc8003da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-26T07:25:01.310 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:25:01.306+0000 7f6bdb7fe640 1 --2- 172.21.15.143:0/3837635584 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f6bc8003980 0x7f6bc8003da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:25:01.310 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:25:01.306+0000 7f6bdb7fe640 1 -- 172.21.15.143:0/3837635584 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f6bc80047d0 msgr2=0x7f6bc8004c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-26T07:25:01.310 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:25:01.306+0000 7f6bdb7fe640 1 --2- 172.21.15.143:0/3837635584 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f6bc80047d0 0x7f6bc8004c50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:25:01.310 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:25:01.306+0000 7f6bdb7fe640 1 -- 172.21.15.143:0/3837635584 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6bc8003530 con 0x7f6bc8005190 2023-11-26T07:25:01.310 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:25:01.306+0000 7f6bd3fff640 1 --2- 172.21.15.143:0/3837635584 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f6bc80047d0 0x7f6bc8004c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-26T07:25:01.311 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:25:01.310+0000 7f6bdb7fe640 1 -- 172.21.15.143:0/3837635584 reap_dead start 2023-11-26T07:25:01.311 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:25:01.310+0000 7f6bdb7fe640 1 --2- 172.21.15.143:0/3837635584 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f6bc8005190 0x7f6bc800b990 secure :-1 s=READY pgs=1474 cs=0 l=1 rev1=1 crypto rx=0x7f6bd4001e70 tx=0x7f6bd401c760 comp rx=0 tx=0).ready entity=mon.0 client_cookie=43c92fcd0b9f0463 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:25:01.312 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:25:01.310+0000 7f6bda7fc640 1 -- 172.21.15.143:0/3837635584 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6bd4003b80 con 0x7f6bc8005190 2023-11-26T07:25:01.312 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:25:01.310+0000 7f6bda7fc640 1 -- 172.21.15.143:0/3837635584 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6bd4003d00 con 0x7f6bc8005190 2023-11-26T07:25:01.312 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:25:01.310+0000 7f6bda7fc640 1 -- 172.21.15.143:0/3837635584 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6bd4005260 con 0x7f6bc8005190 2023-11-26T07:25:01.312 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:25:01.310+0000 7f6be1772640 1 -- 172.21.15.143:0/3837635584 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f6bc8005190 msgr2=0x7f6bc800b990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:25:01.312 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:25:01.310+0000 7f6be1772640 1 --2- 172.21.15.143:0/3837635584 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f6bc8005190 0x7f6bc800b990 secure :-1 s=READY pgs=1474 cs=0 l=1 rev1=1 crypto rx=0x7f6bd4001e70 tx=0x7f6bd401c760 comp rx=0 tx=0).stop 2023-11-26T07:25:01.312 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:25:01.310+0000 7f6be1772640 1 -- 172.21.15.143:0/3837635584 shutdown_connections 2023-11-26T07:25:01.312 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:25:01.310+0000 7f6be1772640 1 --2- 172.21.15.143:0/3837635584 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f6bc8005190 0x7f6bc800b990 unknown :-1 s=CLOSED pgs=1474 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:25:01.313 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:25:01.310+0000 7f6be1772640 1 -- 172.21.15.143:0/3837635584 >> 172.21.15.143:0/3837635584 conn(0x7f6bdc06a890 msgr2=0x7f6bdc06aca0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:25:01.313 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:25:01.310+0000 7f6be1772640 1 -- 172.21.15.143:0/3837635584 shutdown_connections 2023-11-26T07:25:01.313 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:25:01.310+0000 7f6be1772640 1 -- 172.21.15.143:0/3837635584 wait complete. 2023-11-26T07:25:01.315 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:25:01.310+0000 7f6be1772640 1 Processor -- start 2023-11-26T07:25:01.315 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:25:01.310+0000 7f6be1772640 1 -- start start 2023-11-26T07:25:01.315 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:25:01.314+0000 7f6be1772640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f6bc8005190 0x7f6bdc15d6a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:25:01.315 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:25:01.314+0000 7f6be1772640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f6bdc15c100 0x7f6bdc1544e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:25:01.315 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:25:01.314+0000 7f6be1772640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f6bdc071410 0x7f6bdc0b0a40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:25:01.315 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:25:01.314+0000 7f6be1772640 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6bdc15d210 con 0x7f6bc8005190 2023-11-26T07:25:01.315 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:25:01.314+0000 7f6be1772640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6bdc154a20 con 0x7f6bdc071410 2023-11-26T07:25:01.316 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:25:01.314+0000 7f6be1772640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6bdc15c600 con 0x7f6bdc15c100 2023-11-26T07:25:01.316 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:25:01.314+0000 7f6bdaffd640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f6bc8005190 0x7f6bdc15d6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:25:01.316 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:25:01.314+0000 7f6bdaffd640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f6bc8005190 0x7f6bdc15d6a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:46048/0 (socket says 172.21.15.143:46048) 2023-11-26T07:25:01.316 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:25:01.314+0000 7f6bdaffd640 1 -- 172.21.15.143:0/1344769824 learned_addr learned my addr 172.21.15.143:0/1344769824 (peer_addr_for_me v2:172.21.15.143:0/0) 2023-11-26T07:25:01.316 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:25:01.314+0000 7f6bd3fff640 1 --2- 172.21.15.143:0/1344769824 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f6bdc15c100 0x7f6bdc1544e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:25:01.316 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:25:01.314+0000 7f6bdb7fe640 1 --2- 172.21.15.143:0/1344769824 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f6bdc071410 0x7f6bdc0b0a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:25:01.316 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:25:01.314+0000 7f6bdaffd640 1 -- 172.21.15.143:0/1344769824 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f6bdc15c100 msgr2=0x7f6bdc1544e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:25:01.317 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:25:01.314+0000 7f6bdaffd640 1 --2- 172.21.15.143:0/1344769824 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f6bdc15c100 0x7f6bdc1544e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:25:01.317 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:25:01.314+0000 7f6bdaffd640 1 -- 172.21.15.143:0/1344769824 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f6bdc071410 msgr2=0x7f6bdc0b0a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:25:01.317 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:25:01.314+0000 7f6bdaffd640 1 --2- 172.21.15.143:0/1344769824 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f6bdc071410 0x7f6bdc0b0a40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:25:01.317 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:25:01.314+0000 7f6bdaffd640 1 -- 172.21.15.143:0/1344769824 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6bdc15c8e0 con 0x7f6bc8005190 2023-11-26T07:25:01.317 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:25:01.314+0000 7f6bd3fff640 1 --2- 172.21.15.143:0/1344769824 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f6bdc15c100 0x7f6bdc1544e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-26T07:25:01.317 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:25:01.314+0000 7f6bdaffd640 1 --2- 172.21.15.143:0/1344769824 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f6bc8005190 0x7f6bdc15d6a0 secure :-1 s=READY pgs=1475 cs=0 l=1 rev1=1 crypto rx=0x7f6bcc0072c0 tx=0x7f6bcc007780 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:25:01.317 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:25:01.314+0000 7f6bd37fe640 1 -- 172.21.15.143:0/1344769824 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6bcc00f940 con 0x7f6bc8005190 2023-11-26T07:25:01.317 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:25:01.314+0000 7f6bd37fe640 1 -- 172.21.15.143:0/1344769824 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6bcc00faa0 con 0x7f6bc8005190 2023-11-26T07:25:01.318 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:25:01.314+0000 7f6bd37fe640 1 -- 172.21.15.143:0/1344769824 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6bcc00fe50 con 0x7f6bc8005190 2023-11-26T07:25:01.318 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:25:01.314+0000 7f6be1772640 1 -- 172.21.15.143:0/1344769824 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6bdc15cbc0 con 0x7f6bc8005190 2023-11-26T07:25:01.318 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:25:01.314+0000 7f6be1772640 1 -- 172.21.15.143:0/1344769824 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6bdc071890 con 0x7f6bc8005190 2023-11-26T07:25:01.319 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:25:01.314+0000 7f6be1772640 1 -- 172.21.15.143:0/1344769824 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f6bdc1448e0 con 0x7f6bc8005190 2023-11-26T07:25:01.319 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:25:01.314+0000 7f6bd37fe640 1 -- 172.21.15.143:0/1344769824 <== mon.0 v2:172.21.15.143:3300/0 4 ==== mgrmap(e 18) v1 ==== 86107+0+0 (secure 0 0 0) 0x7f6bcc01d460 con 0x7f6bc8005190 2023-11-26T07:25:01.319 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:25:01.318+0000 7f6bd37fe640 1 --2- 172.21.15.143:0/1344769824 >> [v2:172.21.15.188:6836/3401128553,v1:172.21.15.188:6837/3401128553] conn(0x7f6bbc066770 0x7f6bbc068c30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:25:01.320 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:25:01.318+0000 7f6bd37fe640 1 -- 172.21.15.143:0/1344769824 <== mon.0 v2:172.21.15.143:3300/0 5 ==== osd_map(106..106 src has 1..106) v4 ==== 4943+0+0 (secure 0 0 0) 0x7f6bcc0840a0 con 0x7f6bc8005190 2023-11-26T07:25:01.320 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:25:01.318+0000 7f6bd37fe640 1 --2- 172.21.15.143:0/1344769824 >> [v2:172.21.15.188:6800/3924606902,v1:172.21.15.188:6801/3924606902] conn(0x7f6bbc06c5d0 0x7f6bbc06e9f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:25:01.320 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:25:01.318+0000 7f6bd37fe640 1 -- 172.21.15.143:0/1344769824 --> [v2:172.21.15.188:6800/3924606902,v1:172.21.15.188:6801/3924606902] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f6bbc06f0a0 con 0x7f6bbc06c5d0 2023-11-26T07:25:01.320 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:25:01.318+0000 7f6bd37fe640 1 -- 172.21.15.143:0/1344769824 <== mon.0 v2:172.21.15.143:3300/0 6 ==== mon_get_version_reply(handle=1 version=106) v2 ==== 24+0+0 (secure 0 0 0) 0x7f6bcc086680 con 0x7f6bc8005190 2023-11-26T07:25:01.320 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:25:01.318+0000 7f6bd3fff640 1 --2- 172.21.15.143:0/1344769824 >> [v2:172.21.15.188:6836/3401128553,v1:172.21.15.188:6837/3401128553] conn(0x7f6bbc066770 0x7f6bbc068c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:25:01.320 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:25:01.318+0000 7f6bdb7fe640 1 --2- 172.21.15.143:0/1344769824 >> [v2:172.21.15.188:6800/3924606902,v1:172.21.15.188:6801/3924606902] conn(0x7f6bbc06c5d0 0x7f6bbc06e9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:25:01.320 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:25:01.318+0000 7f6bd3fff640 1 --2- 172.21.15.143:0/1344769824 >> [v2:172.21.15.188:6836/3401128553,v1:172.21.15.188:6837/3401128553] conn(0x7f6bbc066770 0x7f6bbc068c30 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7f6bdc134190 tx=0x7f6bc4008040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:25:01.320 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:25:01.318+0000 7f6bdb7fe640 1 --2- 172.21.15.143:0/1344769824 >> [v2:172.21.15.188:6800/3924606902,v1:172.21.15.188:6801/3924606902] conn(0x7f6bbc06c5d0 0x7f6bbc06e9f0 crc :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.4 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:25:01.321 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:25:01.318+0000 7f6bd37fe640 1 -- 172.21.15.143:0/1344769824 <== osd.4 v2:172.21.15.188:6800/3924606902 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+26965 (crc 0 0 0) 0x7f6bbc06f0a0 con 0x7f6bbc06c5d0 2023-11-26T07:25:01.332 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:25:01.330+0000 7f6be1772640 1 -- 172.21.15.143:0/1344769824 --> [v2:172.21.15.188:6800/3924606902,v1:172.21.15.188:6801/3924606902] -- command(tid 2: {"prefix": "config set", "var": "osd_debug_deep_scrub_sleep", "val": ["0"]}) v1 -- 0x7f6bdc15efa0 con 0x7f6bbc06c5d0 2023-11-26T07:25:01.333 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:25:01.334+0000 7f6bd37fe640 1 -- 172.21.15.143:0/1344769824 <== osd.4 v2:172.21.15.188:6800/3924606902 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+97 (crc 0 0 0) 0x7f6bd4003980 con 0x7f6bbc06c5d0 2023-11-26T07:25:01.333 INFO:teuthology.orchestra.run.smithi143.stdout:{ 2023-11-26T07:25:01.333 INFO:teuthology.orchestra.run.smithi143.stdout: "success": "osd_debug_deep_scrub_sleep = '' (not observed, change may require restart) " 2023-11-26T07:25:01.333 INFO:teuthology.orchestra.run.smithi143.stdout:} 2023-11-26T07:25:01.334 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:25:01.334+0000 7f6be1772640 1 -- 172.21.15.143:0/1344769824 >> [v2:172.21.15.188:6800/3924606902,v1:172.21.15.188:6801/3924606902] conn(0x7f6bbc06c5d0 msgr2=0x7f6bbc06e9f0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:25:01.335 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:25:01.334+0000 7f6be1772640 1 --2- 172.21.15.143:0/1344769824 >> [v2:172.21.15.188:6800/3924606902,v1:172.21.15.188:6801/3924606902] conn(0x7f6bbc06c5d0 0x7f6bbc06e9f0 crc :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:25:01.335 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:25:01.334+0000 7f6be1772640 1 -- 172.21.15.143:0/1344769824 >> [v2:172.21.15.188:6836/3401128553,v1:172.21.15.188:6837/3401128553] conn(0x7f6bbc066770 msgr2=0x7f6bbc068c30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:25:01.335 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:25:01.334+0000 7f6be1772640 1 --2- 172.21.15.143:0/1344769824 >> [v2:172.21.15.188:6836/3401128553,v1:172.21.15.188:6837/3401128553] conn(0x7f6bbc066770 0x7f6bbc068c30 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7f6bdc134190 tx=0x7f6bc4008040 comp rx=0 tx=0).stop 2023-11-26T07:25:01.335 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:25:01.334+0000 7f6be1772640 1 -- 172.21.15.143:0/1344769824 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f6bc8005190 msgr2=0x7f6bdc15d6a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:25:01.335 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:25:01.334+0000 7f6be1772640 1 --2- 172.21.15.143:0/1344769824 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f6bc8005190 0x7f6bdc15d6a0 secure :-1 s=READY pgs=1475 cs=0 l=1 rev1=1 crypto rx=0x7f6bcc0072c0 tx=0x7f6bcc007780 comp rx=0 tx=0).stop 2023-11-26T07:25:01.335 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:25:01.334+0000 7f6bdb7fe640 1 -- 172.21.15.143:0/1344769824 reap_dead start 2023-11-26T07:25:01.335 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:25:01.334+0000 7f6be1772640 1 -- 172.21.15.143:0/1344769824 shutdown_connections 2023-11-26T07:25:01.336 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:25:01.334+0000 7f6be1772640 1 -- 172.21.15.143:0/1344769824 >> 172.21.15.143:0/1344769824 conn(0x7f6bdc06a890 msgr2=0x7f6bdc0ce640 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:25:01.336 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:25:01.334+0000 7f6be1772640 1 -- 172.21.15.143:0/1344769824 shutdown_connections 2023-11-26T07:25:01.336 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:25:01.334+0000 7f6be1772640 1 -- 172.21.15.143:0/1344769824 wait complete. 2023-11-26T07:25:01.352 DEBUG:teuthology.orchestra.run.smithi143:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 4 2023-11-26T07:25:01.490 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:25:01.490+0000 7f4407067640 1 Processor -- start 2023-11-26T07:25:01.490 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:25:01.490+0000 7f4407067640 1 -- start start 2023-11-26T07:25:01.491 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:25:01.490+0000 7f4407067640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f4400160850 0x7f44000d3d30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:25:01.491 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:25:01.490+0000 7f4407067640 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7f44000cf330 0x7f44000cda90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:25:01.491 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:25:01.490+0000 7f4407067640 1 -- --> v1:172.21.15.188:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f44000cdfd0 con 0x7f4400160f60 2023-11-26T07:25:01.492 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:25:01.490+0000 7f4407067640 1 -- --> v2:172.21.15.188:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f44000ce290 con 0x7f44000cf330 2023-11-26T07:25:01.492 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:25:01.490+0000 7f4407067640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f44000ce570 con 0x7f4400160850 2023-11-26T07:25:01.492 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:25:01.490+0000 7f4405864640 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7f44000cf330 0x7f44000cda90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:25:01.493 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:25:01.490+0000 7f4406065640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f4400160850 0x7f44000d3d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:25:01.493 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:25:01.490+0000 7f4405864640 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7f44000cf330 0x7f44000cda90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.188:3300/0 says I am v2:172.21.15.143:55690/0 (socket says 172.21.15.143:55690) 2023-11-26T07:25:01.493 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:25:01.490+0000 7f4405864640 1 -- 172.21.15.143:0/768046155 learned_addr learned my addr 172.21.15.143:0/768046155 (peer_addr_for_me v2:172.21.15.143:0/0) 2023-11-26T07:25:01.493 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:25:01.490+0000 7f4406065640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f4400160850 0x7f44000d3d30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.188:3301/0 says I am v2:172.21.15.143:46960/0 (socket says 172.21.15.143:46960) 2023-11-26T07:25:01.493 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:25:01.490+0000 7f4406065640 1 -- 172.21.15.143:0/768046155 >> v2:172.21.15.188:3300/0 conn(0x7f44000cf330 msgr2=0x7f44000cda90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-26T07:25:01.493 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:25:01.490+0000 7f4406065640 1 --2- 172.21.15.143:0/768046155 >> v2:172.21.15.188:3300/0 conn(0x7f44000cf330 0x7f44000cda90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:25:01.494 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:25:01.490+0000 7f4406065640 1 -- 172.21.15.143:0/768046155 >> v1:172.21.15.188:6789/0 conn(0x7f4400160f60 legacy=0x7f44000b27e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:25:01.494 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:25:01.490+0000 7f4406065640 1 -- 172.21.15.143:0/768046155 --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f44000ce850 con 0x7f4400160850 2023-11-26T07:25:01.495 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:25:01.494+0000 7f4406065640 1 --2- 172.21.15.143:0/768046155 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f4400160850 0x7f44000d3d30 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7f43fc0097c0 tx=0x7f43fc01c480 comp rx=0 tx=0).ready entity=mon.2 client_cookie=aea308e9e8b311f7 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:25:01.496 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:25:01.494+0000 7f4405063640 1 -- 172.21.15.143:0/768046155 <== mon.2 v2:172.21.15.188:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f43fc0a6040 con 0x7f4400160850 2023-11-26T07:25:01.496 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:25:01.494+0000 7f4405063640 1 -- 172.21.15.143:0/768046155 <== mon.2 v2:172.21.15.188:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f43fc0aa600 con 0x7f4400160850 2023-11-26T07:25:01.496 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:25:01.494+0000 7f4405063640 1 -- 172.21.15.143:0/768046155 <== mon.2 v2:172.21.15.188:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f43fc0aa990 con 0x7f4400160850 2023-11-26T07:25:01.496 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:25:01.494+0000 7f4407067640 1 -- 172.21.15.143:0/768046155 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f4400160850 msgr2=0x7f44000d3d30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:25:01.496 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:25:01.494+0000 7f4407067640 1 --2- 172.21.15.143:0/768046155 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f4400160850 0x7f44000d3d30 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7f43fc0097c0 tx=0x7f43fc01c480 comp rx=0 tx=0).stop 2023-11-26T07:25:01.497 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:25:01.494+0000 7f4407067640 1 -- 172.21.15.143:0/768046155 shutdown_connections 2023-11-26T07:25:01.497 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:25:01.494+0000 7f4407067640 1 --2- 172.21.15.143:0/768046155 >> v2:172.21.15.188:3300/0 conn(0x7f44000cf330 0x7f44000cda90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:25:01.497 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:25:01.494+0000 7f4407067640 1 --2- 172.21.15.143:0/768046155 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f4400160850 0x7f44000d3d30 unknown :-1 s=CLOSED pgs=171 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:25:01.498 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:25:01.494+0000 7f4407067640 1 -- 172.21.15.143:0/768046155 >> 172.21.15.143:0/768046155 conn(0x7f440009c5a0 msgr2=0x7f44000d3260 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:25:01.498 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:25:01.494+0000 7f4407067640 1 -- 172.21.15.143:0/768046155 shutdown_connections 2023-11-26T07:25:01.498 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:25:01.494+0000 7f4407067640 1 -- 172.21.15.143:0/768046155 wait complete. 2023-11-26T07:25:01.498 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:25:01.498+0000 7f4407067640 1 Processor -- start 2023-11-26T07:25:01.499 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:25:01.498+0000 7f4407067640 1 -- start start 2023-11-26T07:25:01.500 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:25:01.498+0000 7f4407067640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f44000cf330 0x7f44000cb790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:25:01.501 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:25:01.498+0000 7f4407067640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f4400160850 0x7f4400157b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:25:01.501 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:25:01.498+0000 7f4407067640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f4400160f60 0x7f4400159c20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:25:01.501 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:25:01.498+0000 7f4407067640 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f44000cbcd0 con 0x7f4400160f60 2023-11-26T07:25:01.501 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:25:01.498+0000 7f4407067640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f44000ca6f0 con 0x7f44000cf330 2023-11-26T07:25:01.501 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:25:01.498+0000 7f4407067640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f44000ca9d0 con 0x7f4400160850 2023-11-26T07:25:01.501 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:25:01.498+0000 7f4406866640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f4400160f60 0x7f4400159c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:25:01.501 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:25:01.498+0000 7f4406065640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f44000cf330 0x7f44000cb790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:25:01.501 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:25:01.498+0000 7f4406866640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f4400160f60 0x7f4400159c20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:46056/0 (socket says 172.21.15.143:46056) 2023-11-26T07:25:01.502 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:25:01.498+0000 7f4406065640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f44000cf330 0x7f44000cb790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.188:3300/0 says I am v2:172.21.15.143:55692/0 (socket says 172.21.15.143:55692) 2023-11-26T07:25:01.502 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:25:01.498+0000 7f4406866640 1 -- 172.21.15.143:0/1882734022 learned_addr learned my addr 172.21.15.143:0/1882734022 (peer_addr_for_me v2:172.21.15.143:0/0) 2023-11-26T07:25:01.502 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:25:01.498+0000 7f4405864640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f4400160850 0x7f4400157b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:25:01.502 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:25:01.498+0000 7f4406866640 1 -- 172.21.15.143:0/1882734022 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f4400160850 msgr2=0x7f4400157b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:25:01.502 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:25:01.498+0000 7f4406866640 1 --2- 172.21.15.143:0/1882734022 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f4400160850 0x7f4400157b50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:25:01.502 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:25:01.498+0000 7f4406866640 1 -- 172.21.15.143:0/1882734022 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f44000cf330 msgr2=0x7f44000cb790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:25:01.502 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:25:01.498+0000 7f4406866640 1 --2- 172.21.15.143:0/1882734022 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f44000cf330 0x7f44000cb790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:25:01.503 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:25:01.498+0000 7f4406866640 1 -- 172.21.15.143:0/1882734022 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f43fc009420 con 0x7f4400160f60 2023-11-26T07:25:01.503 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:25:01.498+0000 7f4406866640 1 --2- 172.21.15.143:0/1882734022 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f4400160f60 0x7f4400159c20 secure :-1 s=READY pgs=1476 cs=0 l=1 rev1=1 crypto rx=0x7f43f800c770 tx=0x7f43f800cc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:25:01.503 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:25:01.498+0000 7f43f6ffd640 1 -- 172.21.15.143:0/1882734022 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f43f8003b80 con 0x7f4400160f60 2023-11-26T07:25:01.503 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:25:01.498+0000 7f43f6ffd640 1 -- 172.21.15.143:0/1882734022 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f43f8003ce0 con 0x7f4400160f60 2023-11-26T07:25:01.503 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:25:01.498+0000 7f43f6ffd640 1 -- 172.21.15.143:0/1882734022 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f43f800aa90 con 0x7f4400160f60 2023-11-26T07:25:01.503 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:25:01.498+0000 7f4407067640 1 -- 172.21.15.143:0/1882734022 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4400161900 con 0x7f4400160f60 2023-11-26T07:25:01.503 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:25:01.498+0000 7f4407067640 1 -- 172.21.15.143:0/1882734022 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f44000caf90 con 0x7f4400160f60 2023-11-26T07:25:01.504 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:25:01.502+0000 7f4407067640 1 -- 172.21.15.143:0/1882734022 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4400067500 con 0x7f4400160f60 2023-11-26T07:25:01.504 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:25:01.502+0000 7f43f6ffd640 1 -- 172.21.15.143:0/1882734022 <== mon.0 v2:172.21.15.143:3300/0 4 ==== mgrmap(e 18) v1 ==== 86107+0+0 (secure 0 0 0) 0x7f43f800abf0 con 0x7f4400160f60 2023-11-26T07:25:01.504 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:25:01.502+0000 7f43f6ffd640 1 --2- 172.21.15.143:0/1882734022 >> [v2:172.21.15.188:6836/3401128553,v1:172.21.15.188:6837/3401128553] conn(0x7f43e4066740 0x7f43e4068c00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:25:01.505 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:25:01.502+0000 7f43f6ffd640 1 -- 172.21.15.143:0/1882734022 <== mon.0 v2:172.21.15.143:3300/0 5 ==== osd_map(106..106 src has 1..106) v4 ==== 4943+0+0 (secure 0 0 0) 0x7f43f8083d00 con 0x7f4400160f60 2023-11-26T07:25:01.505 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:25:01.502+0000 7f4406065640 1 --2- 172.21.15.143:0/1882734022 >> [v2:172.21.15.188:6836/3401128553,v1:172.21.15.188:6837/3401128553] conn(0x7f43e4066740 0x7f43e4068c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:25:01.505 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:25:01.502+0000 7f4406065640 1 --2- 172.21.15.143:0/1882734022 >> [v2:172.21.15.188:6836/3401128553,v1:172.21.15.188:6837/3401128553] conn(0x7f43e4066740 0x7f43e4068c00 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7f43fc0ae000 tx=0x7f43fc0b5040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:25:01.507 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:25:01.506+0000 7f43f6ffd640 1 -- 172.21.15.143:0/1882734022 <== mon.0 v2:172.21.15.143:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172649 (secure 0 0 0) 0x7f43f80511a0 con 0x7f4400160f60 2023-11-26T07:25:01.658 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:25:01.658+0000 7f4407067640 1 -- 172.21.15.143:0/1882734022 --> [v2:172.21.15.188:6836/3401128553,v1:172.21.15.188:6837/3401128553] -- mgr_command(tid 0: {"prefix": "osd deep-scrub", "who": "4", "target": ["mon-mgr", ""]}) v1 -- 0x7f4400161780 con 0x7f43e4066740 2023-11-26T07:25:01.659 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:25:01.658+0000 7f43f6ffd640 1 -- 172.21.15.143:0/1882734022 <== mgr.4108 v2:172.21.15.188:6836/3401128553 1 ==== mgr_command_reply(tid 0: 0 instructed osd(s) 4 to deep-scrub) v1 ==== 41+0+0 (secure 0 0 0) 0x7f43fc006e20 con 0x7f43e4066740 2023-11-26T07:25:01.659 INFO:teuthology.orchestra.run.smithi143.stderr:instructed osd(s) 4 to deep-scrub 2023-11-26T07:25:01.661 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:25:01.662+0000 7f4407067640 1 -- 172.21.15.143:0/1882734022 >> [v2:172.21.15.188:6836/3401128553,v1:172.21.15.188:6837/3401128553] conn(0x7f43e4066740 msgr2=0x7f43e4068c00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:25:01.662 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:25:01.662+0000 7f4407067640 1 --2- 172.21.15.143:0/1882734022 >> [v2:172.21.15.188:6836/3401128553,v1:172.21.15.188:6837/3401128553] conn(0x7f43e4066740 0x7f43e4068c00 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7f43fc0ae000 tx=0x7f43fc0b5040 comp rx=0 tx=0).stop 2023-11-26T07:25:01.662 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:25:01.662+0000 7f4407067640 1 -- 172.21.15.143:0/1882734022 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f4400160f60 msgr2=0x7f4400159c20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:25:01.662 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:25:01.662+0000 7f4407067640 1 --2- 172.21.15.143:0/1882734022 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f4400160f60 0x7f4400159c20 secure :-1 s=READY pgs=1476 cs=0 l=1 rev1=1 crypto rx=0x7f43f800c770 tx=0x7f43f800cc30 comp rx=0 tx=0).stop 2023-11-26T07:25:01.662 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:25:01.662+0000 7f4407067640 1 -- 172.21.15.143:0/1882734022 shutdown_connections 2023-11-26T07:25:01.662 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:25:01.662+0000 7f4407067640 1 --2- 172.21.15.143:0/1882734022 >> [v2:172.21.15.188:6836/3401128553,v1:172.21.15.188:6837/3401128553] conn(0x7f43e4066740 0x7f43e4068c00 unknown :-1 s=CLOSED pgs=196 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:25:01.662 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:25:01.662+0000 7f4407067640 1 --2- 172.21.15.143:0/1882734022 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f4400160f60 0x7f4400159c20 unknown :-1 s=CLOSED pgs=1476 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:25:01.663 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:25:01.662+0000 7f4407067640 1 --2- 172.21.15.143:0/1882734022 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f4400160850 0x7f4400157b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:25:01.663 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:25:01.662+0000 7f4407067640 1 --2- 172.21.15.143:0/1882734022 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f44000cf330 0x7f44000cb790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:25:01.663 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:25:01.662+0000 7f4407067640 1 -- 172.21.15.143:0/1882734022 >> 172.21.15.143:0/1882734022 conn(0x7f440009c5a0 msgr2=0x7f44000b3510 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:25:01.663 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:25:01.662+0000 7f4407067640 1 -- 172.21.15.143:0/1882734022 shutdown_connections 2023-11-26T07:25:01.663 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:25:01.662+0000 7f4407067640 1 -- 172.21.15.143:0/1882734022 wait complete. 2023-11-26T07:25:01.681 INFO:tasks.ceph:Scrubbing osd.5 2023-11-26T07:25:01.681 DEBUG:teuthology.orchestra.run.smithi143:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.5 config set osd_debug_deep_scrub_sleep 0 2023-11-26T07:25:01.826 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:25:01.826+0000 7f7f72c9c640 1 Processor -- start 2023-11-26T07:25:01.826 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:25:01.826+0000 7f7f72c9c640 1 -- start start 2023-11-26T07:25:01.827 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:25:01.826+0000 7f7f72c9c640 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7f7f6c0cf280 0x7f7f6c0cf710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:25:01.827 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:25:01.826+0000 7f7f72c9c640 1 -- --> v1:172.21.15.143:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7f6c0cc730 con 0x7f7f6c143ac0 2023-11-26T07:25:01.827 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:25:01.826+0000 7f7f72c9c640 1 -- --> v1:172.21.15.188:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7f6c070ee0 con 0x7f7f6c0cd1b0 2023-11-26T07:25:01.827 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:25:01.826+0000 7f7f72c9c640 1 -- --> v2:172.21.15.188:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7f6c143ea0 con 0x7f7f6c0cf280 2023-11-26T07:25:01.827 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:25:01.826+0000 7f7f70a38640 1 --1- >> v1:172.21.15.143:6789/0 conn(0x7f7f6c143ac0 0x7f7f6c070370 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.143:6789/0 says I am v1:172.21.15.143:36374/0 (socket says 172.21.15.143:36374) 2023-11-26T07:25:01.828 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:25:01.826+0000 7f7f70a38640 1 -- 172.21.15.143:0/1633366869 learned_addr learned my addr 172.21.15.143:0/1633366869 (peer_addr_for_me v1:172.21.15.143:0/0) 2023-11-26T07:25:01.830 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:25:01.826+0000 7f7f63fff640 1 --2- 172.21.15.143:0/1633366869 >> v2:172.21.15.188:3300/0 conn(0x7f7f6c0cf280 0x7f7f6c0cf710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:25:01.830 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:25:01.826+0000 7f7f637fe640 1 -- 172.21.15.143:0/1633366869 <== mon.0 v1:172.21.15.143:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2148706399 0 0) 0x7f7f6c0cc730 con 0x7f7f6c143ac0 2023-11-26T07:25:01.830 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:25:01.826+0000 7f7f637fe640 1 -- 172.21.15.143:0/1633366869 --> v1:172.21.15.143:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f7f54003530 con 0x7f7f6c143ac0 2023-11-26T07:25:01.830 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:25:01.826+0000 7f7f637fe640 1 -- 172.21.15.143:0/1633366869 <== mon.0 v1:172.21.15.143:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3123386201 0 0) 0x7f7f54003530 con 0x7f7f6c143ac0 2023-11-26T07:25:01.830 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:25:01.826+0000 7f7f637fe640 1 -- 172.21.15.143:0/1633366869 >> v2:172.21.15.188:3300/0 conn(0x7f7f6c0cf280 msgr2=0x7f7f6c0cf710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-26T07:25:01.830 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:25:01.826+0000 7f7f637fe640 1 --2- 172.21.15.143:0/1633366869 >> v2:172.21.15.188:3300/0 conn(0x7f7f6c0cf280 0x7f7f6c0cf710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:25:01.831 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:25:01.826+0000 7f7f637fe640 1 -- 172.21.15.143:0/1633366869 >> v1:172.21.15.188:6789/0 conn(0x7f7f6c0cd1b0 legacy=0x7f7f6c0cd5c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:25:01.831 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:25:01.826+0000 7f7f637fe640 1 -- 172.21.15.143:0/1633366869 --> v1:172.21.15.143:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7f6c0711c0 con 0x7f7f6c143ac0 2023-11-26T07:25:01.831 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:25:01.826+0000 7f7f637fe640 1 -- 172.21.15.143:0/1633366869 <== mon.0 v1:172.21.15.143:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2137882523 0 0) 0x7f7f68001680 con 0x7f7f6c143ac0 2023-11-26T07:25:01.831 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:25:01.826+0000 7f7f637fe640 1 -- 172.21.15.143:0/1633366869 >> v1:172.21.15.143:6789/0 conn(0x7f7f6c143ac0 legacy=0x7f7f6c070370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:25:01.831 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:25:01.826+0000 7f7f637fe640 1 --2- 172.21.15.143:0/1633366869 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f7f54003980 0x7f7f54003da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:25:01.832 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:25:01.826+0000 7f7f637fe640 1 --2- 172.21.15.143:0/1633366869 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f7f540047d0 0x7f7f54004c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:25:01.832 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:25:01.826+0000 7f7f637fe640 1 --2- 172.21.15.143:0/1633366869 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f7f54005190 0x7f7f5400dba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:25:01.832 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:25:01.826+0000 7f7f637fe640 1 -- 172.21.15.143:0/1633366869 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7f5400e210 con 0x7f7f540047d0 2023-11-26T07:25:01.832 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:25:01.826+0000 7f7f637fe640 1 -- 172.21.15.143:0/1633366869 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7f5400e380 con 0x7f7f54003980 2023-11-26T07:25:01.833 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:25:01.826+0000 7f7f637fe640 1 -- 172.21.15.143:0/1633366869 --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7f5400e540 con 0x7f7f54005190 2023-11-26T07:25:01.833 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:25:01.826+0000 7f7f63fff640 1 --2- 172.21.15.143:0/1633366869 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f7f540047d0 0x7f7f54004c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:25:01.833 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:25:01.826+0000 7f7f70a38640 1 --2- 172.21.15.143:0/1633366869 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f7f54003980 0x7f7f54003da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:25:01.833 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:25:01.826+0000 7f7f71239640 1 --2- 172.21.15.143:0/1633366869 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f7f54005190 0x7f7f5400dba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:25:01.834 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:25:01.830+0000 7f7f63fff640 1 -- 172.21.15.143:0/1633366869 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f7f54005190 msgr2=0x7f7f5400dba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-26T07:25:01.834 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:25:01.830+0000 7f7f63fff640 1 --2- 172.21.15.143:0/1633366869 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f7f54005190 0x7f7f5400dba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:25:01.834 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:25:01.830+0000 7f7f63fff640 1 -- 172.21.15.143:0/1633366869 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f7f54003980 msgr2=0x7f7f54003da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-26T07:25:01.834 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:25:01.830+0000 7f7f63fff640 1 --2- 172.21.15.143:0/1633366869 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f7f54003980 0x7f7f54003da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:25:01.835 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:25:01.830+0000 7f7f63fff640 1 -- 172.21.15.143:0/1633366869 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7f54003530 con 0x7f7f540047d0 2023-11-26T07:25:01.835 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:25:01.830+0000 7f7f70a38640 1 --2- 172.21.15.143:0/1633366869 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f7f54003980 0x7f7f54003da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-26T07:25:01.835 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:25:01.830+0000 7f7f71239640 1 -- 172.21.15.143:0/1633366869 reap_dead start 2023-11-26T07:25:01.840 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:25:01.830+0000 7f7f63fff640 1 --2- 172.21.15.143:0/1633366869 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f7f540047d0 0x7f7f54004c50 secure :-1 s=READY pgs=1478 cs=0 l=1 rev1=1 crypto rx=0x7f7f5c009ae0 tx=0x7f7f5c01ca50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7ebe39167a34ce41 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:25:01.841 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:25:01.830+0000 7f7f637fe640 1 -- 172.21.15.143:0/1633366869 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7f5c0a5640 con 0x7f7f540047d0 2023-11-26T07:25:01.841 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:25:01.830+0000 7f7f637fe640 1 -- 172.21.15.143:0/1633366869 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7f5c0a57a0 con 0x7f7f540047d0 2023-11-26T07:25:01.841 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:25:01.830+0000 7f7f637fe640 1 -- 172.21.15.143:0/1633366869 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7f5c0a5b50 con 0x7f7f540047d0 2023-11-26T07:25:01.842 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:25:01.830+0000 7f7f72c9c640 1 -- 172.21.15.143:0/1633366869 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f7f540047d0 msgr2=0x7f7f54004c50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:25:01.842 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:25:01.830+0000 7f7f72c9c640 1 --2- 172.21.15.143:0/1633366869 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f7f540047d0 0x7f7f54004c50 secure :-1 s=READY pgs=1478 cs=0 l=1 rev1=1 crypto rx=0x7f7f5c009ae0 tx=0x7f7f5c01ca50 comp rx=0 tx=0).stop 2023-11-26T07:25:01.842 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:25:01.830+0000 7f7f72c9c640 1 -- 172.21.15.143:0/1633366869 shutdown_connections 2023-11-26T07:25:01.843 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:25:01.830+0000 7f7f72c9c640 1 --2- 172.21.15.143:0/1633366869 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f7f540047d0 0x7f7f54004c50 unknown :-1 s=CLOSED pgs=1478 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:25:01.843 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:25:01.830+0000 7f7f72c9c640 1 -- 172.21.15.143:0/1633366869 >> 172.21.15.143:0/1633366869 conn(0x7f7f6c06a890 msgr2=0x7f7f6c06aca0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:25:01.844 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:25:01.830+0000 7f7f72c9c640 1 -- 172.21.15.143:0/1633366869 shutdown_connections 2023-11-26T07:25:01.844 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:25:01.830+0000 7f7f72c9c640 1 -- 172.21.15.143:0/1633366869 wait complete. 2023-11-26T07:25:01.844 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:25:01.834+0000 7f7f72c9c640 1 Processor -- start 2023-11-26T07:25:01.845 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:25:01.834+0000 7f7f72c9c640 1 -- start start 2023-11-26T07:25:01.845 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:25:01.834+0000 7f7f72c9c640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f7f540047d0 0x7f7f6c1591c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:25:01.845 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:25:01.834+0000 7f7f72c9c640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f7f6c13bd90 0x7f7f6c13e5a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:25:01.846 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:25:01.834+0000 7f7f72c9c640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f7f6c13eae0 0x7f7f6c0aeb40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:25:01.846 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:25:01.834+0000 7f7f72c9c640 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7f6c159bd0 con 0x7f7f6c13bd90 2023-11-26T07:25:01.846 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:25:01.834+0000 7f7f72c9c640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7f6c13e110 con 0x7f7f540047d0 2023-11-26T07:25:01.846 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:25:01.834+0000 7f7f72c9c640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7f6c13b120 con 0x7f7f6c13eae0 2023-11-26T07:25:01.847 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:25:01.834+0000 7f7f63fff640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f7f6c13bd90 0x7f7f6c13e5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:25:01.847 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:25:01.834+0000 7f7f63fff640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f7f6c13bd90 0x7f7f6c13e5a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:46072/0 (socket says 172.21.15.143:46072) 2023-11-26T07:25:01.847 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:25:01.834+0000 7f7f63fff640 1 -- 172.21.15.143:0/3931907731 learned_addr learned my addr 172.21.15.143:0/3931907731 (peer_addr_for_me v2:172.21.15.143:0/0) 2023-11-26T07:25:01.847 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:25:01.834+0000 7f7f71239640 1 --2- 172.21.15.143:0/3931907731 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f7f6c13eae0 0x7f7f6c0aeb40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:25:01.847 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:25:01.834+0000 7f7f70a38640 1 --2- 172.21.15.143:0/3931907731 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f7f540047d0 0x7f7f6c1591c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:25:01.847 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:25:01.834+0000 7f7f63fff640 1 -- 172.21.15.143:0/3931907731 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f7f6c13eae0 msgr2=0x7f7f6c0aeb40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:25:01.847 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:25:01.834+0000 7f7f63fff640 1 --2- 172.21.15.143:0/3931907731 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f7f6c13eae0 0x7f7f6c0aeb40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:25:01.848 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:25:01.834+0000 7f7f63fff640 1 -- 172.21.15.143:0/3931907731 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f7f540047d0 msgr2=0x7f7f6c1591c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:25:01.848 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:25:01.834+0000 7f7f63fff640 1 --2- 172.21.15.143:0/3931907731 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f7f540047d0 0x7f7f6c1591c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:25:01.848 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:25:01.834+0000 7f7f63fff640 1 -- 172.21.15.143:0/3931907731 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7f6c13ab60 con 0x7f7f6c13bd90 2023-11-26T07:25:01.848 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:25:01.834+0000 7f7f63fff640 1 --2- 172.21.15.143:0/3931907731 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f7f6c13bd90 0x7f7f6c13e5a0 secure :-1 s=READY pgs=1479 cs=0 l=1 rev1=1 crypto rx=0x7f7f5c01c880 tx=0x7f7f5c0197f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:25:01.848 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:25:01.834+0000 7f7f617fa640 1 -- 172.21.15.143:0/3931907731 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7f5c004020 con 0x7f7f6c13bd90 2023-11-26T07:25:01.848 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:25:01.838+0000 7f7f617fa640 1 -- 172.21.15.143:0/3931907731 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7f5c0041c0 con 0x7f7f6c13bd90 2023-11-26T07:25:01.848 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:25:01.838+0000 7f7f72c9c640 1 -- 172.21.15.143:0/3931907731 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7f6c13ae40 con 0x7f7f6c13bd90 2023-11-26T07:25:01.848 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:25:01.838+0000 7f7f617fa640 1 -- 172.21.15.143:0/3931907731 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7f5c0b8660 con 0x7f7f6c13bd90 2023-11-26T07:25:01.848 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:25:01.838+0000 7f7f72c9c640 1 -- 172.21.15.143:0/3931907731 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7f6c1599a0 con 0x7f7f6c13bd90 2023-11-26T07:25:01.849 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:25:01.838+0000 7f7f72c9c640 1 -- 172.21.15.143:0/3931907731 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f7f6c071c30 con 0x7f7f6c13bd90 2023-11-26T07:25:01.849 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:25:01.838+0000 7f7f617fa640 1 -- 172.21.15.143:0/3931907731 <== mon.0 v2:172.21.15.143:3300/0 4 ==== mgrmap(e 18) v1 ==== 86107+0+0 (secure 0 0 0) 0x7f7f5c0b4070 con 0x7f7f6c13bd90 2023-11-26T07:25:01.849 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:25:01.838+0000 7f7f617fa640 1 --2- 172.21.15.143:0/3931907731 >> [v2:172.21.15.188:6836/3401128553,v1:172.21.15.188:6837/3401128553] conn(0x7f7f58066740 0x7f7f58068c00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:25:01.849 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:25:01.838+0000 7f7f617fa640 1 -- 172.21.15.143:0/3931907731 <== mon.0 v2:172.21.15.143:3300/0 5 ==== osd_map(106..106 src has 1..106) v4 ==== 4943+0+0 (secure 0 0 0) 0x7f7f5c11e880 con 0x7f7f6c13bd90 2023-11-26T07:25:01.849 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:25:01.838+0000 7f7f617fa640 1 --2- 172.21.15.143:0/3931907731 >> [v2:172.21.15.188:6813/186841544,v1:172.21.15.188:6815/186841544] conn(0x7f7f5806c5a0 0x7f7f5806e9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:25:01.849 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:25:01.838+0000 7f7f617fa640 1 -- 172.21.15.143:0/3931907731 --> [v2:172.21.15.188:6813/186841544,v1:172.21.15.188:6815/186841544] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f7f5806f070 con 0x7f7f5806c5a0 2023-11-26T07:25:01.849 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:25:01.838+0000 7f7f617fa640 1 -- 172.21.15.143:0/3931907731 <== mon.0 v2:172.21.15.143:3300/0 6 ==== mon_get_version_reply(handle=1 version=106) v2 ==== 24+0+0 (secure 0 0 0) 0x7f7f5c120e60 con 0x7f7f6c13bd90 2023-11-26T07:25:01.849 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:25:01.838+0000 7f7f70a38640 1 --2- 172.21.15.143:0/3931907731 >> [v2:172.21.15.188:6836/3401128553,v1:172.21.15.188:6837/3401128553] conn(0x7f7f58066740 0x7f7f58068c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:25:01.850 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:25:01.838+0000 7f7f71239640 1 --2- 172.21.15.143:0/3931907731 >> [v2:172.21.15.188:6813/186841544,v1:172.21.15.188:6815/186841544] conn(0x7f7f5806c5a0 0x7f7f5806e9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:25:01.850 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:25:01.842+0000 7f7f70a38640 1 --2- 172.21.15.143:0/3931907731 >> [v2:172.21.15.188:6836/3401128553,v1:172.21.15.188:6837/3401128553] conn(0x7f7f58066740 0x7f7f58068c00 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7f7f6c0b0fe0 tx=0x7f7f68001480 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:25:01.850 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:25:01.842+0000 7f7f71239640 1 --2- 172.21.15.143:0/3931907731 >> [v2:172.21.15.188:6813/186841544,v1:172.21.15.188:6815/186841544] conn(0x7f7f5806c5a0 0x7f7f5806e9c0 crc :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.5 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:25:01.851 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:25:01.842+0000 7f7f617fa640 1 -- 172.21.15.143:0/3931907731 <== osd.5 v2:172.21.15.188:6813/186841544 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+26965 (crc 0 0 0) 0x7f7f5806f070 con 0x7f7f5806c5a0 2023-11-26T07:25:01.854 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:25:01.854+0000 7f7f72c9c640 1 -- 172.21.15.143:0/3931907731 --> [v2:172.21.15.188:6813/186841544,v1:172.21.15.188:6815/186841544] -- command(tid 2: {"prefix": "config set", "var": "osd_debug_deep_scrub_sleep", "val": ["0"]}) v1 -- 0x7f7f6c0ccc90 con 0x7f7f5806c5a0 2023-11-26T07:25:01.855 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:25:01.854+0000 7f7f617fa640 1 -- 172.21.15.143:0/3931907731 <== osd.5 v2:172.21.15.188:6813/186841544 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+97 (crc 0 0 0) 0x7f7f64001980 con 0x7f7f5806c5a0 2023-11-26T07:25:01.855 INFO:teuthology.orchestra.run.smithi143.stdout:{ 2023-11-26T07:25:01.856 INFO:teuthology.orchestra.run.smithi143.stdout: "success": "osd_debug_deep_scrub_sleep = '' (not observed, change may require restart) " 2023-11-26T07:25:01.856 INFO:teuthology.orchestra.run.smithi143.stdout:} 2023-11-26T07:25:01.856 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:25:01.854+0000 7f7f72c9c640 1 -- 172.21.15.143:0/3931907731 >> [v2:172.21.15.188:6813/186841544,v1:172.21.15.188:6815/186841544] conn(0x7f7f5806c5a0 msgr2=0x7f7f5806e9c0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:25:01.857 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:25:01.854+0000 7f7f72c9c640 1 --2- 172.21.15.143:0/3931907731 >> [v2:172.21.15.188:6813/186841544,v1:172.21.15.188:6815/186841544] conn(0x7f7f5806c5a0 0x7f7f5806e9c0 crc :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:25:01.857 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:25:01.854+0000 7f7f72c9c640 1 -- 172.21.15.143:0/3931907731 >> [v2:172.21.15.188:6836/3401128553,v1:172.21.15.188:6837/3401128553] conn(0x7f7f58066740 msgr2=0x7f7f58068c00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:25:01.857 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:25:01.854+0000 7f7f72c9c640 1 --2- 172.21.15.143:0/3931907731 >> [v2:172.21.15.188:6836/3401128553,v1:172.21.15.188:6837/3401128553] conn(0x7f7f58066740 0x7f7f58068c00 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7f7f6c0b0fe0 tx=0x7f7f68001480 comp rx=0 tx=0).stop 2023-11-26T07:25:01.857 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:25:01.854+0000 7f7f72c9c640 1 -- 172.21.15.143:0/3931907731 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f7f6c13bd90 msgr2=0x7f7f6c13e5a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:25:01.857 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:25:01.854+0000 7f7f72c9c640 1 --2- 172.21.15.143:0/3931907731 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f7f6c13bd90 0x7f7f6c13e5a0 secure :-1 s=READY pgs=1479 cs=0 l=1 rev1=1 crypto rx=0x7f7f5c01c880 tx=0x7f7f5c0197f0 comp rx=0 tx=0).stop 2023-11-26T07:25:01.857 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:25:01.854+0000 7f7f71239640 1 -- 172.21.15.143:0/3931907731 reap_dead start 2023-11-26T07:25:01.857 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:25:01.854+0000 7f7f72c9c640 1 -- 172.21.15.143:0/3931907731 shutdown_connections 2023-11-26T07:25:01.858 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:25:01.854+0000 7f7f72c9c640 1 -- 172.21.15.143:0/3931907731 >> 172.21.15.143:0/3931907731 conn(0x7f7f6c06a890 msgr2=0x7f7f6c142420 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:25:01.858 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:25:01.854+0000 7f7f72c9c640 1 -- 172.21.15.143:0/3931907731 shutdown_connections 2023-11-26T07:25:01.858 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:25:01.854+0000 7f7f72c9c640 1 -- 172.21.15.143:0/3931907731 wait complete. 2023-11-26T07:25:01.874 DEBUG:teuthology.orchestra.run.smithi143:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 5 2023-11-26T07:25:02.013 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:25:02.010+0000 7fa908357640 1 Processor -- start 2023-11-26T07:25:02.013 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:25:02.014+0000 7fa908357640 1 -- start start 2023-11-26T07:25:02.015 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:25:02.014+0000 7fa908357640 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7fa9000cf990 0x7fa90015efa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:25:02.015 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:25:02.014+0000 7fa908357640 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7fa90015f4e0 0x7fa9000cd7b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:25:02.015 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:25:02.014+0000 7fa908357640 1 -- --> v1:172.21.15.188:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa90006c520 con 0x7fa90013e290 2023-11-26T07:25:02.015 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:25:02.014+0000 7fa908357640 1 -- --> v2:172.21.15.143:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa90006c800 con 0x7fa9000cf990 2023-11-26T07:25:02.015 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:25:02.014+0000 7fa908357640 1 -- --> v2:172.21.15.188:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa90006cae0 con 0x7fa90015f4e0 2023-11-26T07:25:02.016 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:25:02.014+0000 7fa9058f2640 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7fa9000cf990 0x7fa90015efa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:25:02.016 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:25:02.014+0000 7fa9058f2640 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7fa9000cf990 0x7fa90015efa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:46076/0 (socket says 172.21.15.143:46076) 2023-11-26T07:25:02.016 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:25:02.014+0000 7fa9058f2640 1 -- 172.21.15.143:0/2717305679 learned_addr learned my addr 172.21.15.143:0/2717305679 (peer_addr_for_me v2:172.21.15.143:0/0) 2023-11-26T07:25:02.016 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:25:02.014+0000 7fa9068f4640 1 --2- 172.21.15.143:0/2717305679 >> v2:172.21.15.188:3300/0 conn(0x7fa90015f4e0 0x7fa9000cd7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:25:02.016 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:25:02.014+0000 7fa9058f2640 1 -- 172.21.15.143:0/2717305679 >> v2:172.21.15.188:3300/0 conn(0x7fa90015f4e0 msgr2=0x7fa9000cd7b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-26T07:25:02.016 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:25:02.014+0000 7fa9058f2640 1 --2- 172.21.15.143:0/2717305679 >> v2:172.21.15.188:3300/0 conn(0x7fa90015f4e0 0x7fa9000cd7b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:25:02.016 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:25:02.014+0000 7fa9058f2640 1 -- 172.21.15.143:0/2717305679 >> v1:172.21.15.188:6789/0 conn(0x7fa90013e290 legacy=0x7fa9000cf280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:25:02.016 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:25:02.014+0000 7fa9058f2640 1 -- 172.21.15.143:0/2717305679 --> v2:172.21.15.143:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa90006cdc0 con 0x7fa9000cf990 2023-11-26T07:25:02.018 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:25:02.018+0000 7fa9058f2640 1 --2- 172.21.15.143:0/2717305679 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fa9000cf990 0x7fa90015efa0 secure :-1 s=READY pgs=1480 cs=0 l=1 rev1=1 crypto rx=0x7fa900155cf0 tx=0x7fa8f001c4d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ab829dbaea7b062c server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:25:02.018 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:25:02.018+0000 7fa9050f1640 1 -- 172.21.15.143:0/2717305679 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa8f00aa650 con 0x7fa9000cf990 2023-11-26T07:25:02.018 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:25:02.018+0000 7fa9050f1640 1 -- 172.21.15.143:0/2717305679 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa8f00aa7b0 con 0x7fa9000cf990 2023-11-26T07:25:02.018 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:25:02.018+0000 7fa9050f1640 1 -- 172.21.15.143:0/2717305679 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa8f00aab40 con 0x7fa9000cf990 2023-11-26T07:25:02.019 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:25:02.018+0000 7fa908357640 1 -- 172.21.15.143:0/2717305679 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fa9000cf990 msgr2=0x7fa90015efa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:25:02.019 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:25:02.018+0000 7fa908357640 1 --2- 172.21.15.143:0/2717305679 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fa9000cf990 0x7fa90015efa0 secure :-1 s=READY pgs=1480 cs=0 l=1 rev1=1 crypto rx=0x7fa900155cf0 tx=0x7fa8f001c4d0 comp rx=0 tx=0).stop 2023-11-26T07:25:02.019 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:25:02.018+0000 7fa908357640 1 -- 172.21.15.143:0/2717305679 shutdown_connections 2023-11-26T07:25:02.020 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:25:02.018+0000 7fa908357640 1 --2- 172.21.15.143:0/2717305679 >> v2:172.21.15.188:3300/0 conn(0x7fa90015f4e0 0x7fa9000cd7b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:25:02.020 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:25:02.018+0000 7fa908357640 1 --2- 172.21.15.143:0/2717305679 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fa9000cf990 0x7fa90015efa0 unknown :-1 s=CLOSED pgs=1480 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:25:02.020 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:25:02.018+0000 7fa908357640 1 -- 172.21.15.143:0/2717305679 >> 172.21.15.143:0/2717305679 conn(0x7fa90006a890 msgr2=0x7fa90006aca0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:25:02.020 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:25:02.018+0000 7fa908357640 1 -- 172.21.15.143:0/2717305679 shutdown_connections 2023-11-26T07:25:02.020 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:25:02.018+0000 7fa908357640 1 -- 172.21.15.143:0/2717305679 wait complete. 2023-11-26T07:25:02.021 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:25:02.018+0000 7fa908357640 1 Processor -- start 2023-11-26T07:25:02.021 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:25:02.022+0000 7fa908357640 1 -- start start 2023-11-26T07:25:02.022 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:25:02.022+0000 7fa908357640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fa9000cf990 0x7fa900137070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:25:02.022 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:25:02.022+0000 7fa908357640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fa90013e290 0x7fa900138d20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:25:02.022 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:25:02.022+0000 7fa908357640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fa90015f4e0 0x7fa9001351b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:25:02.022 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:25:02.022+0000 7fa908357640 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa9001375b0 con 0x7fa90013e290 2023-11-26T07:25:02.023 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:25:02.022+0000 7fa908357640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa9000aed90 con 0x7fa9000cf990 2023-11-26T07:25:02.023 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:25:02.022+0000 7fa908357640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa900070370 con 0x7fa90015f4e0 2023-11-26T07:25:02.023 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:25:02.022+0000 7fa9058f2640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fa90013e290 0x7fa900138d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:25:02.023 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:25:02.022+0000 7fa9058f2640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fa90013e290 0x7fa900138d20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:46084/0 (socket says 172.21.15.143:46084) 2023-11-26T07:25:02.023 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:25:02.022+0000 7fa9058f2640 1 -- 172.21.15.143:0/1656848043 learned_addr learned my addr 172.21.15.143:0/1656848043 (peer_addr_for_me v2:172.21.15.143:0/0) 2023-11-26T07:25:02.023 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:25:02.022+0000 7fa9060f3640 1 --2- 172.21.15.143:0/1656848043 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fa9000cf990 0x7fa900137070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:25:02.023 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:25:02.022+0000 7fa9068f4640 1 --2- 172.21.15.143:0/1656848043 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fa90015f4e0 0x7fa9001351b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:25:02.024 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:25:02.022+0000 7fa9058f2640 1 -- 172.21.15.143:0/1656848043 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fa90015f4e0 msgr2=0x7fa9001351b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:25:02.025 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:25:02.022+0000 7fa9058f2640 1 --2- 172.21.15.143:0/1656848043 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fa90015f4e0 0x7fa9001351b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:25:02.025 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:25:02.022+0000 7fa9058f2640 1 -- 172.21.15.143:0/1656848043 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fa9000cf990 msgr2=0x7fa900137070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:25:02.025 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:25:02.022+0000 7fa9058f2640 1 --2- 172.21.15.143:0/1656848043 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fa9000cf990 0x7fa900137070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:25:02.025 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:25:02.022+0000 7fa9058f2640 1 -- 172.21.15.143:0/1656848043 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa8f00094d0 con 0x7fa90013e290 2023-11-26T07:25:02.025 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:25:02.022+0000 7fa9058f2640 1 --2- 172.21.15.143:0/1656848043 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fa90013e290 0x7fa900138d20 secure :-1 s=READY pgs=1481 cs=0 l=1 rev1=1 crypto rx=0x7fa8f00190e0 tx=0x7fa8f001ad10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:25:02.025 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:25:02.022+0000 7fa8f6ffd640 1 -- 172.21.15.143:0/1656848043 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa8f00b3800 con 0x7fa90013e290 2023-11-26T07:25:02.025 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:25:02.022+0000 7fa8f6ffd640 1 -- 172.21.15.143:0/1656848043 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa8f00b3960 con 0x7fa90013e290 2023-11-26T07:25:02.025 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:25:02.022+0000 7fa8f6ffd640 1 -- 172.21.15.143:0/1656848043 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa8f00b3cd0 con 0x7fa90013e290 2023-11-26T07:25:02.026 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:25:02.022+0000 7fa908357640 1 -- 172.21.15.143:0/1656848043 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa900070650 con 0x7fa90013e290 2023-11-26T07:25:02.026 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:25:02.022+0000 7fa908357640 1 -- 172.21.15.143:0/1656848043 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa900070c10 con 0x7fa90013e290 2023-11-26T07:25:02.026 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:25:02.022+0000 7fa908357640 1 -- 172.21.15.143:0/1656848043 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa900067680 con 0x7fa90013e290 2023-11-26T07:25:02.026 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:25:02.026+0000 7fa8f6ffd640 1 -- 172.21.15.143:0/1656848043 <== mon.0 v2:172.21.15.143:3300/0 4 ==== mgrmap(e 18) v1 ==== 86107+0+0 (secure 0 0 0) 0x7fa8f00b2380 con 0x7fa90013e290 2023-11-26T07:25:02.027 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:25:02.026+0000 7fa8f6ffd640 1 --2- 172.21.15.143:0/1656848043 >> [v2:172.21.15.188:6836/3401128553,v1:172.21.15.188:6837/3401128553] conn(0x7fa8e4066740 0x7fa8e4068c00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:25:02.027 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:25:02.026+0000 7fa8f6ffd640 1 -- 172.21.15.143:0/1656848043 <== mon.0 v2:172.21.15.143:3300/0 5 ==== osd_map(106..106 src has 1..106) v4 ==== 4943+0+0 (secure 0 0 0) 0x7fa90015fb00 con 0x7fa90013e290 2023-11-26T07:25:02.027 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:25:02.026+0000 7fa9060f3640 1 --2- 172.21.15.143:0/1656848043 >> [v2:172.21.15.188:6836/3401128553,v1:172.21.15.188:6837/3401128553] conn(0x7fa8e4066740 0x7fa8e4068c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:25:02.027 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:25:02.026+0000 7fa9060f3640 1 --2- 172.21.15.143:0/1656848043 >> [v2:172.21.15.188:6836/3401128553,v1:172.21.15.188:6837/3401128553] conn(0x7fa8e4066740 0x7fa8e4068c00 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7fa8fc007fd0 tx=0x7fa8fc009040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:25:02.030 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:25:02.030+0000 7fa8f6ffd640 1 -- 172.21.15.143:0/1656848043 <== mon.0 v2:172.21.15.143:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172649 (secure 0 0 0) 0x7fa8f00eb860 con 0x7fa90013e290 2023-11-26T07:25:02.183 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:25:02.182+0000 7fa908357640 1 -- 172.21.15.143:0/1656848043 --> [v2:172.21.15.188:6836/3401128553,v1:172.21.15.188:6837/3401128553] -- mgr_command(tid 0: {"prefix": "osd deep-scrub", "who": "5", "target": ["mon-mgr", ""]}) v1 -- 0x7fa90013da50 con 0x7fa8e4066740 2023-11-26T07:25:02.184 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:25:02.182+0000 7fa8f6ffd640 1 -- 172.21.15.143:0/1656848043 <== mgr.4108 v2:172.21.15.188:6836/3401128553 1 ==== mgr_command_reply(tid 0: 0 instructed osd(s) 5 to deep-scrub) v1 ==== 41+0+0 (secure 0 0 0) 0x7fa90013da50 con 0x7fa8e4066740 2023-11-26T07:25:02.184 INFO:teuthology.orchestra.run.smithi143.stderr:instructed osd(s) 5 to deep-scrub 2023-11-26T07:25:02.185 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:25:02.186+0000 7fa908357640 1 -- 172.21.15.143:0/1656848043 >> [v2:172.21.15.188:6836/3401128553,v1:172.21.15.188:6837/3401128553] conn(0x7fa8e4066740 msgr2=0x7fa8e4068c00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:25:02.186 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:25:02.186+0000 7fa908357640 1 --2- 172.21.15.143:0/1656848043 >> [v2:172.21.15.188:6836/3401128553,v1:172.21.15.188:6837/3401128553] conn(0x7fa8e4066740 0x7fa8e4068c00 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7fa8fc007fd0 tx=0x7fa8fc009040 comp rx=0 tx=0).stop 2023-11-26T07:25:02.186 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:25:02.186+0000 7fa908357640 1 -- 172.21.15.143:0/1656848043 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fa90013e290 msgr2=0x7fa900138d20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:25:02.186 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:25:02.186+0000 7fa908357640 1 --2- 172.21.15.143:0/1656848043 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fa90013e290 0x7fa900138d20 secure :-1 s=READY pgs=1481 cs=0 l=1 rev1=1 crypto rx=0x7fa8f00190e0 tx=0x7fa8f001ad10 comp rx=0 tx=0).stop 2023-11-26T07:25:02.186 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:25:02.186+0000 7fa908357640 1 -- 172.21.15.143:0/1656848043 shutdown_connections 2023-11-26T07:25:02.186 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:25:02.186+0000 7fa908357640 1 --2- 172.21.15.143:0/1656848043 >> [v2:172.21.15.188:6836/3401128553,v1:172.21.15.188:6837/3401128553] conn(0x7fa8e4066740 0x7fa8e4068c00 unknown :-1 s=CLOSED pgs=198 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:25:02.186 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:25:02.186+0000 7fa908357640 1 --2- 172.21.15.143:0/1656848043 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fa90015f4e0 0x7fa9001351b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:25:02.187 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:25:02.186+0000 7fa908357640 1 --2- 172.21.15.143:0/1656848043 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fa90013e290 0x7fa900138d20 unknown :-1 s=CLOSED pgs=1481 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:25:02.187 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:25:02.186+0000 7fa908357640 1 --2- 172.21.15.143:0/1656848043 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fa9000cf990 0x7fa900137070 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:25:02.187 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:25:02.186+0000 7fa908357640 1 -- 172.21.15.143:0/1656848043 >> 172.21.15.143:0/1656848043 conn(0x7fa90006a890 msgr2=0x7fa90013c720 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:25:02.187 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:25:02.186+0000 7fa908357640 1 -- 172.21.15.143:0/1656848043 shutdown_connections 2023-11-26T07:25:02.187 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:25:02.186+0000 7fa908357640 1 -- 172.21.15.143:0/1656848043 wait complete. 2023-11-26T07:25:02.205 INFO:tasks.ceph:Scrubbing osd.6 2023-11-26T07:25:02.205 DEBUG:teuthology.orchestra.run.smithi143:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.6 config set osd_debug_deep_scrub_sleep 0 2023-11-26T07:25:02.347 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:25:02.346+0000 7fb81b017640 1 Processor -- start 2023-11-26T07:25:02.347 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:25:02.346+0000 7fb81b017640 1 -- start start 2023-11-26T07:25:02.348 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:25:02.346+0000 7fb81b017640 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7fb814143ac0 0x7fb814070370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:25:02.349 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:25:02.346+0000 7fb81b017640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fb8140708b0 0x7fb8140cf280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:25:02.349 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:25:02.346+0000 7fb81b017640 1 -- --> v1:172.21.15.143:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb8140cfbf0 con 0x7fb8140cf7c0 2023-11-26T07:25:02.349 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:25:02.346+0000 7fb81b017640 1 -- --> v2:172.21.15.143:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb814143ea0 con 0x7fb814143ac0 2023-11-26T07:25:02.349 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:25:02.346+0000 7fb81b017640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb8140cc730 con 0x7fb8140708b0 2023-11-26T07:25:02.349 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:25:02.346+0000 7fb818db3640 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7fb814143ac0 0x7fb814070370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:25:02.349 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:25:02.346+0000 7fb8195b4640 1 --1- >> v1:172.21.15.143:6789/0 conn(0x7fb8140cf7c0 0x7fb8140b1ef0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.143:6789/0 says I am v1:172.21.15.143:36390/0 (socket says 172.21.15.143:36390) 2023-11-26T07:25:02.349 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:25:02.346+0000 7fb8195b4640 1 -- 172.21.15.143:0/3435730402 learned_addr learned my addr 172.21.15.143:0/3435730402 (peer_addr_for_me v1:172.21.15.143:0/0) 2023-11-26T07:25:02.350 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:25:02.350+0000 7fb80bfff640 1 --2- 172.21.15.143:0/3435730402 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fb8140708b0 0x7fb8140cf280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:25:02.350 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:25:02.350+0000 7fb818db3640 1 -- 172.21.15.143:0/3435730402 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fb8140708b0 msgr2=0x7fb8140cf280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-26T07:25:02.350 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:25:02.350+0000 7fb818db3640 1 --2- 172.21.15.143:0/3435730402 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fb8140708b0 0x7fb8140cf280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:25:02.350 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:25:02.350+0000 7fb818db3640 1 -- 172.21.15.143:0/3435730402 >> v1:172.21.15.143:6789/0 conn(0x7fb8140cf7c0 legacy=0x7fb8140b1ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:25:02.351 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:25:02.350+0000 7fb818db3640 1 -- 172.21.15.143:0/3435730402 --> v2:172.21.15.143:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb814070ee0 con 0x7fb814143ac0 2023-11-26T07:25:02.352 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:25:02.350+0000 7fb818db3640 1 --2- 172.21.15.143:0/3435730402 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fb814143ac0 0x7fb814070370 secure :-1 s=READY pgs=1483 cs=0 l=1 rev1=1 crypto rx=0x7fb8100097c0 tx=0x7fb81001c480 comp rx=0 tx=0).ready entity=mon.0 client_cookie=87f5eb96a4a67402 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:25:02.352 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:25:02.350+0000 7fb80b7fe640 1 -- 172.21.15.143:0/3435730402 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb8100ad640 con 0x7fb814143ac0 2023-11-26T07:25:02.352 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:25:02.350+0000 7fb80b7fe640 1 -- 172.21.15.143:0/3435730402 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb8100ad7a0 con 0x7fb814143ac0 2023-11-26T07:25:02.353 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:25:02.350+0000 7fb80b7fe640 1 -- 172.21.15.143:0/3435730402 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb8100adb30 con 0x7fb814143ac0 2023-11-26T07:25:02.353 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:25:02.354+0000 7fb81b017640 1 -- 172.21.15.143:0/3435730402 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fb814143ac0 msgr2=0x7fb814070370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:25:02.353 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:25:02.354+0000 7fb81b017640 1 --2- 172.21.15.143:0/3435730402 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fb814143ac0 0x7fb814070370 secure :-1 s=READY pgs=1483 cs=0 l=1 rev1=1 crypto rx=0x7fb8100097c0 tx=0x7fb81001c480 comp rx=0 tx=0).stop 2023-11-26T07:25:02.353 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:25:02.354+0000 7fb81b017640 1 -- 172.21.15.143:0/3435730402 shutdown_connections 2023-11-26T07:25:02.354 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:25:02.354+0000 7fb81b017640 1 --2- 172.21.15.143:0/3435730402 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fb8140708b0 0x7fb8140cf280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:25:02.354 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:25:02.354+0000 7fb81b017640 1 --2- 172.21.15.143:0/3435730402 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fb814143ac0 0x7fb814070370 unknown :-1 s=CLOSED pgs=1483 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:25:02.354 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:25:02.354+0000 7fb81b017640 1 -- 172.21.15.143:0/3435730402 >> 172.21.15.143:0/3435730402 conn(0x7fb81406a890 msgr2=0x7fb81406aca0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:25:02.354 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:25:02.354+0000 7fb81b017640 1 -- 172.21.15.143:0/3435730402 shutdown_connections 2023-11-26T07:25:02.354 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:25:02.354+0000 7fb81b017640 1 -- 172.21.15.143:0/3435730402 wait complete. 2023-11-26T07:25:02.355 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:25:02.354+0000 7fb81b017640 1 Processor -- start 2023-11-26T07:25:02.355 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:25:02.354+0000 7fb81b017640 1 -- start start 2023-11-26T07:25:02.356 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:25:02.354+0000 7fb81b017640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fb8140708b0 0x7fb814159bb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:25:02.356 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:25:02.354+0000 7fb81b017640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fb8140cf7c0 0x7fb81413ad90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:25:02.356 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:25:02.354+0000 7fb81b017640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fb814143ac0 0x7fb81413b2d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:25:02.357 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:25:02.354+0000 7fb81b017640 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb814168640 con 0x7fb814143ac0 2023-11-26T07:25:02.357 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:25:02.354+0000 7fb81b017640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb814167da0 con 0x7fb8140cf7c0 2023-11-26T07:25:02.357 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:25:02.354+0000 7fb81b017640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb814168080 con 0x7fb8140708b0 2023-11-26T07:25:02.357 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:25:02.354+0000 7fb8195b4640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fb814143ac0 0x7fb81413b2d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:25:02.357 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:25:02.354+0000 7fb8195b4640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fb814143ac0 0x7fb81413b2d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:46104/0 (socket says 172.21.15.143:46104) 2023-11-26T07:25:02.357 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:25:02.354+0000 7fb8195b4640 1 -- 172.21.15.143:0/191649557 learned_addr learned my addr 172.21.15.143:0/191649557 (peer_addr_for_me v2:172.21.15.143:0/0) 2023-11-26T07:25:02.357 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:25:02.354+0000 7fb818db3640 1 --2- 172.21.15.143:0/191649557 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fb8140708b0 0x7fb814159bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:25:02.357 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:25:02.354+0000 7fb80bfff640 1 --2- 172.21.15.143:0/191649557 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fb8140cf7c0 0x7fb81413ad90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:25:02.358 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:25:02.354+0000 7fb8195b4640 1 -- 172.21.15.143:0/191649557 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fb8140708b0 msgr2=0x7fb814159bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:25:02.358 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:25:02.354+0000 7fb8195b4640 1 --2- 172.21.15.143:0/191649557 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fb8140708b0 0x7fb814159bb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:25:02.358 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:25:02.354+0000 7fb8195b4640 1 -- 172.21.15.143:0/191649557 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fb8140cf7c0 msgr2=0x7fb81413ad90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:25:02.358 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:25:02.354+0000 7fb8195b4640 1 --2- 172.21.15.143:0/191649557 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fb8140cf7c0 0x7fb81413ad90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:25:02.358 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:25:02.354+0000 7fb8195b4640 1 -- 172.21.15.143:0/191649557 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb810009420 con 0x7fb814143ac0 2023-11-26T07:25:02.359 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:25:02.354+0000 7fb8195b4640 1 --2- 172.21.15.143:0/191649557 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fb814143ac0 0x7fb81413b2d0 secure :-1 s=READY pgs=1484 cs=0 l=1 rev1=1 crypto rx=0x7fb80c00cae0 tx=0x7fb80c00a060 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:25:02.359 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:25:02.358+0000 7fb8097fa640 1 -- 172.21.15.143:0/191649557 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb80c00ac40 con 0x7fb814143ac0 2023-11-26T07:25:02.359 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:25:02.358+0000 7fb8097fa640 1 -- 172.21.15.143:0/191649557 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb80c00ada0 con 0x7fb814143ac0 2023-11-26T07:25:02.360 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:25:02.358+0000 7fb8097fa640 1 -- 172.21.15.143:0/191649557 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb80c012620 con 0x7fb814143ac0 2023-11-26T07:25:02.360 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:25:02.358+0000 7fb81b017640 1 -- 172.21.15.143:0/191649557 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb814168360 con 0x7fb814143ac0 2023-11-26T07:25:02.360 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:25:02.358+0000 7fb81b017640 1 -- 172.21.15.143:0/191649557 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb814167030 con 0x7fb814143ac0 2023-11-26T07:25:02.360 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:25:02.358+0000 7fb81b017640 1 -- 172.21.15.143:0/191649557 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7fb814071c30 con 0x7fb814143ac0 2023-11-26T07:25:02.360 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:25:02.358+0000 7fb8097fa640 1 -- 172.21.15.143:0/191649557 <== mon.0 v2:172.21.15.143:3300/0 4 ==== mgrmap(e 18) v1 ==== 86107+0+0 (secure 0 0 0) 0x7fb80c007070 con 0x7fb814143ac0 2023-11-26T07:25:02.361 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:25:02.358+0000 7fb8097fa640 1 --2- 172.21.15.143:0/191649557 >> [v2:172.21.15.188:6836/3401128553,v1:172.21.15.188:6837/3401128553] conn(0x7fb800066770 0x7fb800068c30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:25:02.361 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:25:02.358+0000 7fb8097fa640 1 -- 172.21.15.143:0/191649557 <== mon.0 v2:172.21.15.143:3300/0 5 ==== osd_map(106..106 src has 1..106) v4 ==== 4943+0+0 (secure 0 0 0) 0x7fb80c083e20 con 0x7fb814143ac0 2023-11-26T07:25:02.362 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:25:02.358+0000 7fb8097fa640 1 --2- 172.21.15.143:0/191649557 >> [v2:172.21.15.188:6808/2153499405,v1:172.21.15.188:6809/2153499405] conn(0x7fb80006c5d0 0x7fb80006e9f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:25:02.362 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:25:02.358+0000 7fb8097fa640 1 -- 172.21.15.143:0/191649557 --> [v2:172.21.15.188:6808/2153499405,v1:172.21.15.188:6809/2153499405] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7fb80006f0a0 con 0x7fb80006c5d0 2023-11-26T07:25:02.362 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:25:02.358+0000 7fb8097fa640 1 -- 172.21.15.143:0/191649557 <== mon.0 v2:172.21.15.143:3300/0 6 ==== mon_get_version_reply(handle=1 version=106) v2 ==== 24+0+0 (secure 0 0 0) 0x7fb80c086400 con 0x7fb814143ac0 2023-11-26T07:25:02.362 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:25:02.358+0000 7fb818db3640 1 --2- 172.21.15.143:0/191649557 >> [v2:172.21.15.188:6836/3401128553,v1:172.21.15.188:6837/3401128553] conn(0x7fb800066770 0x7fb800068c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:25:02.362 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:25:02.358+0000 7fb80bfff640 1 --2- 172.21.15.143:0/191649557 >> [v2:172.21.15.188:6808/2153499405,v1:172.21.15.188:6809/2153499405] conn(0x7fb80006c5d0 0x7fb80006e9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:25:02.362 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:25:02.362+0000 7fb818db3640 1 --2- 172.21.15.143:0/191649557 >> [v2:172.21.15.188:6836/3401128553,v1:172.21.15.188:6837/3401128553] conn(0x7fb800066770 0x7fb800068c30 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7fb8100af000 tx=0x7fb810008040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:25:02.362 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:25:02.362+0000 7fb80bfff640 1 --2- 172.21.15.143:0/191649557 >> [v2:172.21.15.188:6808/2153499405,v1:172.21.15.188:6809/2153499405] conn(0x7fb80006c5d0 0x7fb80006e9f0 crc :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:25:02.363 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:25:02.362+0000 7fb8097fa640 1 -- 172.21.15.143:0/191649557 <== osd.6 v2:172.21.15.188:6808/2153499405 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+26965 (crc 0 0 0) 0x7fb80006f0a0 con 0x7fb80006c5d0 2023-11-26T07:25:02.375 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:25:02.374+0000 7fb81b017640 1 -- 172.21.15.143:0/191649557 --> [v2:172.21.15.188:6808/2153499405,v1:172.21.15.188:6809/2153499405] -- command(tid 2: {"prefix": "config set", "var": "osd_debug_deep_scrub_sleep", "val": ["0"]}) v1 -- 0x7fb8140af180 con 0x7fb80006c5d0 2023-11-26T07:25:02.375 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:25:02.374+0000 7fb8097fa640 1 -- 172.21.15.143:0/191649557 <== osd.6 v2:172.21.15.188:6808/2153499405 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+97 (crc 0 0 0) 0x7fb8140af180 con 0x7fb80006c5d0 2023-11-26T07:25:02.376 INFO:teuthology.orchestra.run.smithi143.stdout:{ 2023-11-26T07:25:02.376 INFO:teuthology.orchestra.run.smithi143.stdout: "success": "osd_debug_deep_scrub_sleep = '' (not observed, change may require restart) " 2023-11-26T07:25:02.376 INFO:teuthology.orchestra.run.smithi143.stdout:} 2023-11-26T07:25:02.377 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:25:02.374+0000 7fb81b017640 1 -- 172.21.15.143:0/191649557 >> [v2:172.21.15.188:6808/2153499405,v1:172.21.15.188:6809/2153499405] conn(0x7fb80006c5d0 msgr2=0x7fb80006e9f0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:25:02.377 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:25:02.374+0000 7fb81b017640 1 --2- 172.21.15.143:0/191649557 >> [v2:172.21.15.188:6808/2153499405,v1:172.21.15.188:6809/2153499405] conn(0x7fb80006c5d0 0x7fb80006e9f0 crc :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:25:02.377 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:25:02.374+0000 7fb81b017640 1 -- 172.21.15.143:0/191649557 >> [v2:172.21.15.188:6836/3401128553,v1:172.21.15.188:6837/3401128553] conn(0x7fb800066770 msgr2=0x7fb800068c30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:25:02.377 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:25:02.374+0000 7fb81b017640 1 --2- 172.21.15.143:0/191649557 >> [v2:172.21.15.188:6836/3401128553,v1:172.21.15.188:6837/3401128553] conn(0x7fb800066770 0x7fb800068c30 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7fb8100af000 tx=0x7fb810008040 comp rx=0 tx=0).stop 2023-11-26T07:25:02.378 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:25:02.374+0000 7fb81b017640 1 -- 172.21.15.143:0/191649557 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fb814143ac0 msgr2=0x7fb81413b2d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:25:02.378 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:25:02.374+0000 7fb81b017640 1 --2- 172.21.15.143:0/191649557 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fb814143ac0 0x7fb81413b2d0 secure :-1 s=READY pgs=1484 cs=0 l=1 rev1=1 crypto rx=0x7fb80c00cae0 tx=0x7fb80c00a060 comp rx=0 tx=0).stop 2023-11-26T07:25:02.378 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:25:02.374+0000 7fb8195b4640 1 -- 172.21.15.143:0/191649557 reap_dead start 2023-11-26T07:25:02.378 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:25:02.374+0000 7fb81b017640 1 -- 172.21.15.143:0/191649557 shutdown_connections 2023-11-26T07:25:02.378 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:25:02.374+0000 7fb81b017640 1 -- 172.21.15.143:0/191649557 >> 172.21.15.143:0/191649557 conn(0x7fb81406a890 msgr2=0x7fb81406ac70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:25:02.378 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:25:02.374+0000 7fb81b017640 1 -- 172.21.15.143:0/191649557 shutdown_connections 2023-11-26T07:25:02.378 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:25:02.374+0000 7fb81b017640 1 -- 172.21.15.143:0/191649557 wait complete. 2023-11-26T07:25:02.394 DEBUG:teuthology.orchestra.run.smithi143:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 6 2023-11-26T07:25:02.533 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:25:02.530+0000 7fe7c4632640 1 Processor -- start 2023-11-26T07:25:02.533 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:25:02.534+0000 7fe7c4632640 1 -- start start 2023-11-26T07:25:02.534 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:25:02.534+0000 7fe7c4632640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fe7bc161cf0 0x7fe7bc06bae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:25:02.534 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:25:02.534+0000 7fe7c4632640 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7fe7bc0cdd10 0x7fe7bc06d950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:25:02.535 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:25:02.534+0000 7fe7c4632640 1 -- --> v1:172.21.15.143:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe7bc0cd610 con 0x7fe7bc070ba0 2023-11-26T07:25:02.535 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:25:02.534+0000 7fe7c4632640 1 -- --> v2:172.21.15.188:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe7bc0cd8f0 con 0x7fe7bc0cdd10 2023-11-26T07:25:02.535 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:25:02.534+0000 7fe7c4632640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe7bc161210 con 0x7fe7bc161cf0 2023-11-26T07:25:02.535 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:25:02.534+0000 7fe7c23ce640 1 --1- >> v1:172.21.15.143:6789/0 conn(0x7fe7bc070ba0 0x7fe7bc070f80 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.143:6789/0 says I am v1:172.21.15.143:36398/0 (socket says 172.21.15.143:36398) 2023-11-26T07:25:02.535 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:25:02.534+0000 7fe7c23ce640 1 -- 172.21.15.143:0/1370911488 learned_addr learned my addr 172.21.15.143:0/1370911488 (peer_addr_for_me v1:172.21.15.143:0/0) 2023-11-26T07:25:02.537 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:25:02.534+0000 7fe7c1bcd640 1 --2- 172.21.15.143:0/1370911488 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fe7bc161cf0 0x7fe7bc06bae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:25:02.537 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:25:02.534+0000 7fe7c2bcf640 1 --2- 172.21.15.143:0/1370911488 >> v2:172.21.15.188:3300/0 conn(0x7fe7bc0cdd10 0x7fe7bc06d950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:25:02.537 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:25:02.534+0000 7fe7c13cc640 1 -- 172.21.15.143:0/1370911488 <== mon.0 v1:172.21.15.143:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2701836885 0 0) 0x7fe7bc0cd610 con 0x7fe7bc070ba0 2023-11-26T07:25:02.537 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:25:02.534+0000 7fe7c13cc640 1 -- 172.21.15.143:0/1370911488 --> v1:172.21.15.143:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fe7a8003530 con 0x7fe7bc070ba0 2023-11-26T07:25:02.537 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:25:02.534+0000 7fe7c13cc640 1 -- 172.21.15.143:0/1370911488 <== mon.0 v1:172.21.15.143:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1515384787 0 0) 0x7fe7a8003530 con 0x7fe7bc070ba0 2023-11-26T07:25:02.538 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:25:02.534+0000 7fe7c13cc640 1 -- 172.21.15.143:0/1370911488 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fe7bc161cf0 msgr2=0x7fe7bc06bae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-26T07:25:02.538 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:25:02.534+0000 7fe7c13cc640 1 --2- 172.21.15.143:0/1370911488 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fe7bc161cf0 0x7fe7bc06bae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:25:02.538 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:25:02.534+0000 7fe7c13cc640 1 -- 172.21.15.143:0/1370911488 >> v2:172.21.15.188:3300/0 conn(0x7fe7bc0cdd10 msgr2=0x7fe7bc06d950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-26T07:25:02.538 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:25:02.534+0000 7fe7c13cc640 1 --2- 172.21.15.143:0/1370911488 >> v2:172.21.15.188:3300/0 conn(0x7fe7bc0cdd10 0x7fe7bc06d950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:25:02.538 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:25:02.534+0000 7fe7c13cc640 1 -- 172.21.15.143:0/1370911488 --> v1:172.21.15.143:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe7bc1614f0 con 0x7fe7bc070ba0 2023-11-26T07:25:02.538 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:25:02.534+0000 7fe7c13cc640 1 -- 172.21.15.143:0/1370911488 <== mon.0 v1:172.21.15.143:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2137882523 0 0) 0x7fe7b8001680 con 0x7fe7bc070ba0 2023-11-26T07:25:02.538 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:25:02.534+0000 7fe7c13cc640 1 -- 172.21.15.143:0/1370911488 >> v1:172.21.15.143:6789/0 conn(0x7fe7bc070ba0 legacy=0x7fe7bc070f80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:25:02.539 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:25:02.534+0000 7fe7c13cc640 1 --2- 172.21.15.143:0/1370911488 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fe7a8003980 0x7fe7a8003da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:25:02.539 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:25:02.534+0000 7fe7c13cc640 1 --2- 172.21.15.143:0/1370911488 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fe7a80047d0 0x7fe7a8004c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:25:02.539 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:25:02.534+0000 7fe7c13cc640 1 --2- 172.21.15.143:0/1370911488 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fe7bc161cf0 0x7fe7a8005190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:25:02.539 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:25:02.534+0000 7fe7c13cc640 1 -- 172.21.15.143:0/1370911488 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe7bc0cd8f0 con 0x7fe7a8003980 2023-11-26T07:25:02.539 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:25:02.534+0000 7fe7c13cc640 1 -- 172.21.15.143:0/1370911488 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe7bc161210 con 0x7fe7a80047d0 2023-11-26T07:25:02.539 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:25:02.534+0000 7fe7c13cc640 1 -- 172.21.15.143:0/1370911488 --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe7a800b9e0 con 0x7fe7bc161cf0 2023-11-26T07:25:02.539 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:25:02.534+0000 7fe7c23ce640 1 --2- 172.21.15.143:0/1370911488 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fe7a8003980 0x7fe7a8003da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:25:02.539 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:25:02.534+0000 7fe7c2bcf640 1 --2- 172.21.15.143:0/1370911488 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fe7bc161cf0 0x7fe7a8005190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:25:02.540 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:25:02.534+0000 7fe7c1bcd640 1 --2- 172.21.15.143:0/1370911488 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fe7a80047d0 0x7fe7a8004c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:25:02.540 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:25:02.534+0000 7fe7c23ce640 1 -- 172.21.15.143:0/1370911488 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fe7bc161cf0 msgr2=0x7fe7a8005190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-26T07:25:02.540 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:25:02.534+0000 7fe7c23ce640 1 --2- 172.21.15.143:0/1370911488 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fe7bc161cf0 0x7fe7a8005190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:25:02.540 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:25:02.534+0000 7fe7c23ce640 1 -- 172.21.15.143:0/1370911488 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fe7a80047d0 msgr2=0x7fe7a8004c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-26T07:25:02.540 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:25:02.534+0000 7fe7c23ce640 1 --2- 172.21.15.143:0/1370911488 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fe7a80047d0 0x7fe7a8004c50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:25:02.540 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:25:02.534+0000 7fe7c23ce640 1 -- 172.21.15.143:0/1370911488 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe7a8003530 con 0x7fe7a8003980 2023-11-26T07:25:02.540 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:25:02.534+0000 7fe7c1bcd640 1 --2- 172.21.15.143:0/1370911488 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fe7a80047d0 0x7fe7a8004c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-26T07:25:02.542 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:25:02.538+0000 7fe7c23ce640 1 --2- 172.21.15.143:0/1370911488 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fe7a8003980 0x7fe7a8003da0 secure :-1 s=READY pgs=1486 cs=0 l=1 rev1=1 crypto rx=0x7fe7b800c240 tx=0x7fe7b801cc40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=513c30fdc8797d59 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:25:02.542 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:25:02.538+0000 7fe7c13cc640 1 -- 172.21.15.143:0/1370911488 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe7b8005ae0 con 0x7fe7a8003980 2023-11-26T07:25:02.542 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:25:02.538+0000 7fe7c13cc640 1 -- 172.21.15.143:0/1370911488 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe7b8005c40 con 0x7fe7a8003980 2023-11-26T07:25:02.543 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:25:02.538+0000 7fe7c13cc640 1 -- 172.21.15.143:0/1370911488 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe7b80a5860 con 0x7fe7a8003980 2023-11-26T07:25:02.543 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:25:02.538+0000 7fe7c4632640 1 -- 172.21.15.143:0/1370911488 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fe7a8003980 msgr2=0x7fe7a8003da0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:25:02.543 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:25:02.538+0000 7fe7c4632640 1 --2- 172.21.15.143:0/1370911488 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fe7a8003980 0x7fe7a8003da0 secure :-1 s=READY pgs=1486 cs=0 l=1 rev1=1 crypto rx=0x7fe7b800c240 tx=0x7fe7b801cc40 comp rx=0 tx=0).stop 2023-11-26T07:25:02.543 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:25:02.538+0000 7fe7c2bcf640 1 -- 172.21.15.143:0/1370911488 reap_dead start 2023-11-26T07:25:02.543 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:25:02.538+0000 7fe7c4632640 1 -- 172.21.15.143:0/1370911488 shutdown_connections 2023-11-26T07:25:02.543 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:25:02.538+0000 7fe7c4632640 1 -- 172.21.15.143:0/1370911488 >> 172.21.15.143:0/1370911488 conn(0x7fe7bc0cf710 msgr2=0x7fe7bc0cfb20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:25:02.543 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:25:02.538+0000 7fe7c4632640 1 -- 172.21.15.143:0/1370911488 shutdown_connections 2023-11-26T07:25:02.543 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:25:02.538+0000 7fe7c4632640 1 -- 172.21.15.143:0/1370911488 wait complete. 2023-11-26T07:25:02.544 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:25:02.542+0000 7fe7c4632640 1 Processor -- start 2023-11-26T07:25:02.544 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:25:02.542+0000 7fe7c4632640 1 -- start start 2023-11-26T07:25:02.546 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:25:02.542+0000 7fe7c4632640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fe7a8003980 0x7fe7bc1532f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:25:02.546 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:25:02.542+0000 7fe7c4632640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fe7bc0ae2f0 0x7fe7bc157810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:25:02.547 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:25:02.542+0000 7fe7c4632640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fe7bc157d50 0x7fe7bc158b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:25:02.547 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:25:02.542+0000 7fe7c4632640 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe7bc1573f0 con 0x7fe7bc157d50 2023-11-26T07:25:02.547 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:25:02.542+0000 7fe7c4632640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe7bc1570a0 con 0x7fe7bc0ae2f0 2023-11-26T07:25:02.547 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:25:02.542+0000 7fe7c4632640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe7bc153830 con 0x7fe7a8003980 2023-11-26T07:25:02.547 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:25:02.542+0000 7fe7c2bcf640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fe7bc157d50 0x7fe7bc158b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:25:02.547 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:25:02.542+0000 7fe7c2bcf640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fe7bc157d50 0x7fe7bc158b20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:46118/0 (socket says 172.21.15.143:46118) 2023-11-26T07:25:02.548 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:25:02.542+0000 7fe7c2bcf640 1 -- 172.21.15.143:0/2699456943 learned_addr learned my addr 172.21.15.143:0/2699456943 (peer_addr_for_me v2:172.21.15.143:0/0) 2023-11-26T07:25:02.548 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:25:02.542+0000 7fe7c23ce640 1 --2- 172.21.15.143:0/2699456943 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fe7a8003980 0x7fe7bc1532f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:25:02.548 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:25:02.542+0000 7fe7c1bcd640 1 --2- 172.21.15.143:0/2699456943 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fe7bc0ae2f0 0x7fe7bc157810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:25:02.548 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:25:02.542+0000 7fe7c2bcf640 1 -- 172.21.15.143:0/2699456943 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fe7a8003980 msgr2=0x7fe7bc1532f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:25:02.548 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:25:02.542+0000 7fe7c2bcf640 1 --2- 172.21.15.143:0/2699456943 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fe7a8003980 0x7fe7bc1532f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:25:02.548 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:25:02.542+0000 7fe7c2bcf640 1 -- 172.21.15.143:0/2699456943 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fe7bc0ae2f0 msgr2=0x7fe7bc157810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:25:02.549 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:25:02.542+0000 7fe7c2bcf640 1 --2- 172.21.15.143:0/2699456943 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fe7bc0ae2f0 0x7fe7bc157810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:25:02.549 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:25:02.542+0000 7fe7c2bcf640 1 -- 172.21.15.143:0/2699456943 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe7bc0ae7d0 con 0x7fe7bc157d50 2023-11-26T07:25:02.549 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:25:02.542+0000 7fe7c23ce640 1 --2- 172.21.15.143:0/2699456943 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fe7a8003980 0x7fe7bc1532f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-26T07:25:02.549 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:25:02.542+0000 7fe7c2bcf640 1 --2- 172.21.15.143:0/2699456943 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fe7bc157d50 0x7fe7bc158b20 secure :-1 s=READY pgs=1487 cs=0 l=1 rev1=1 crypto rx=0x7fe7bc070b50 tx=0x7fe7b400f580 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:25:02.549 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:25:02.542+0000 7fe7b2ffd640 1 -- 172.21.15.143:0/2699456943 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe7b400e040 con 0x7fe7bc157d50 2023-11-26T07:25:02.549 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:25:02.542+0000 7fe7b2ffd640 1 -- 172.21.15.143:0/2699456943 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe7b400f040 con 0x7fe7bc157d50 2023-11-26T07:25:02.550 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:25:02.542+0000 7fe7c4632640 1 -- 172.21.15.143:0/2699456943 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe7bc0aeab0 con 0x7fe7bc157d50 2023-11-26T07:25:02.550 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:25:02.542+0000 7fe7b2ffd640 1 -- 172.21.15.143:0/2699456943 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe7b4015670 con 0x7fe7bc157d50 2023-11-26T07:25:02.550 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:25:02.542+0000 7fe7c4632640 1 -- 172.21.15.143:0/2699456943 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe7bc159060 con 0x7fe7bc157d50 2023-11-26T07:25:02.550 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:25:02.546+0000 7fe7c4632640 1 -- 172.21.15.143:0/2699456943 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe7bc157530 con 0x7fe7bc157d50 2023-11-26T07:25:02.550 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:25:02.546+0000 7fe7b2ffd640 1 -- 172.21.15.143:0/2699456943 <== mon.0 v2:172.21.15.143:3300/0 4 ==== mgrmap(e 18) v1 ==== 86107+0+0 (secure 0 0 0) 0x7fe7b40157d0 con 0x7fe7bc157d50 2023-11-26T07:25:02.551 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:25:02.546+0000 7fe7b2ffd640 1 --2- 172.21.15.143:0/2699456943 >> [v2:172.21.15.188:6836/3401128553,v1:172.21.15.188:6837/3401128553] conn(0x7fe7a0066740 0x7fe7a0068c00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:25:02.552 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:25:02.546+0000 7fe7b2ffd640 1 -- 172.21.15.143:0/2699456943 <== mon.0 v2:172.21.15.143:3300/0 5 ==== osd_map(106..106 src has 1..106) v4 ==== 4943+0+0 (secure 0 0 0) 0x7fe7b4084a60 con 0x7fe7bc157d50 2023-11-26T07:25:02.552 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:25:02.546+0000 7fe7c23ce640 1 --2- 172.21.15.143:0/2699456943 >> [v2:172.21.15.188:6836/3401128553,v1:172.21.15.188:6837/3401128553] conn(0x7fe7a0066740 0x7fe7a0068c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:25:02.552 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:25:02.546+0000 7fe7c23ce640 1 --2- 172.21.15.143:0/2699456943 >> [v2:172.21.15.188:6836/3401128553,v1:172.21.15.188:6837/3401128553] conn(0x7fe7a0066740 0x7fe7a0068c00 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7fe7b80ab000 tx=0x7fe7b80a7070 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:25:02.552 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:25:02.550+0000 7fe7b2ffd640 1 -- 172.21.15.143:0/2699456943 <== mon.0 v2:172.21.15.143:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172649 (secure 0 0 0) 0x7fe7bc157530 con 0x7fe7bc157d50 2023-11-26T07:25:02.701 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:25:02.702+0000 7fe7c4632640 1 -- 172.21.15.143:0/2699456943 --> [v2:172.21.15.188:6836/3401128553,v1:172.21.15.188:6837/3401128553] -- mgr_command(tid 0: {"prefix": "osd deep-scrub", "who": "6", "target": ["mon-mgr", ""]}) v1 -- 0x7fe7bc157530 con 0x7fe7a0066740 2023-11-26T07:25:02.702 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:25:02.702+0000 7fe7b2ffd640 1 -- 172.21.15.143:0/2699456943 <== mgr.4108 v2:172.21.15.188:6836/3401128553 1 ==== mgr_command_reply(tid 0: 0 instructed osd(s) 6 to deep-scrub) v1 ==== 41+0+0 (secure 0 0 0) 0x7fe7bc157530 con 0x7fe7a0066740 2023-11-26T07:25:02.703 INFO:teuthology.orchestra.run.smithi143.stderr:instructed osd(s) 6 to deep-scrub 2023-11-26T07:25:02.705 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:25:02.702+0000 7fe7c4632640 1 -- 172.21.15.143:0/2699456943 >> [v2:172.21.15.188:6836/3401128553,v1:172.21.15.188:6837/3401128553] conn(0x7fe7a0066740 msgr2=0x7fe7a0068c00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:25:02.705 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:25:02.702+0000 7fe7c4632640 1 --2- 172.21.15.143:0/2699456943 >> [v2:172.21.15.188:6836/3401128553,v1:172.21.15.188:6837/3401128553] conn(0x7fe7a0066740 0x7fe7a0068c00 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7fe7b80ab000 tx=0x7fe7b80a7070 comp rx=0 tx=0).stop 2023-11-26T07:25:02.705 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:25:02.702+0000 7fe7c4632640 1 -- 172.21.15.143:0/2699456943 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fe7bc157d50 msgr2=0x7fe7bc158b20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:25:02.705 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:25:02.702+0000 7fe7c4632640 1 --2- 172.21.15.143:0/2699456943 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fe7bc157d50 0x7fe7bc158b20 secure :-1 s=READY pgs=1487 cs=0 l=1 rev1=1 crypto rx=0x7fe7bc070b50 tx=0x7fe7b400f580 comp rx=0 tx=0).stop 2023-11-26T07:25:02.705 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:25:02.702+0000 7fe7c4632640 1 -- 172.21.15.143:0/2699456943 shutdown_connections 2023-11-26T07:25:02.705 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:25:02.702+0000 7fe7c4632640 1 --2- 172.21.15.143:0/2699456943 >> [v2:172.21.15.188:6836/3401128553,v1:172.21.15.188:6837/3401128553] conn(0x7fe7a0066740 0x7fe7a0068c00 unknown :-1 s=CLOSED pgs=200 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:25:02.705 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:25:02.702+0000 7fe7c4632640 1 --2- 172.21.15.143:0/2699456943 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fe7bc157d50 0x7fe7bc158b20 unknown :-1 s=CLOSED pgs=1487 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:25:02.706 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:25:02.702+0000 7fe7c4632640 1 --2- 172.21.15.143:0/2699456943 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fe7bc0ae2f0 0x7fe7bc157810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:25:02.706 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:25:02.702+0000 7fe7c4632640 1 --2- 172.21.15.143:0/2699456943 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fe7a8003980 0x7fe7bc1532f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:25:02.706 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:25:02.702+0000 7fe7c4632640 1 -- 172.21.15.143:0/2699456943 >> 172.21.15.143:0/2699456943 conn(0x7fe7bc0cf710 msgr2=0x7fe7bc163c30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:25:02.706 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:25:02.702+0000 7fe7c4632640 1 -- 172.21.15.143:0/2699456943 shutdown_connections 2023-11-26T07:25:02.706 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:25:02.706+0000 7fe7c4632640 1 -- 172.21.15.143:0/2699456943 wait complete. 2023-11-26T07:25:02.725 INFO:tasks.ceph:Scrubbing osd.7 2023-11-26T07:25:02.725 DEBUG:teuthology.orchestra.run.smithi143:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.7 config set osd_debug_deep_scrub_sleep 0 2023-11-26T07:25:02.866 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:25:02.866+0000 7fd33509c640 1 Processor -- start 2023-11-26T07:25:02.867 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:25:02.866+0000 7fd33509c640 1 -- start start 2023-11-26T07:25:02.868 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:25:02.866+0000 7fd33509c640 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7fd33013e290 0x7fd3300cf280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:25:02.868 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:25:02.866+0000 7fd33509c640 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7fd33015f6b0 0x7fd3300cd7b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:25:02.868 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:25:02.866+0000 7fd33509c640 1 -- --> v1:172.21.15.188:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd33015fb20 con 0x7fd3300cf7c0 2023-11-26T07:25:02.868 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:25:02.866+0000 7fd33509c640 1 -- --> v2:172.21.15.143:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd3300cfc40 con 0x7fd33015f6b0 2023-11-26T07:25:02.868 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:25:02.866+0000 7fd33509c640 1 -- --> v2:172.21.15.188:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd33006c520 con 0x7fd33013e290 2023-11-26T07:25:02.868 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:25:02.866+0000 7fd32f59e640 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7fd33015f6b0 0x7fd3300cd7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:25:02.869 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:25:02.866+0000 7fd32f59e640 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7fd33015f6b0 0x7fd3300cd7b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:46122/0 (socket says 172.21.15.143:46122) 2023-11-26T07:25:02.869 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:25:02.866+0000 7fd32f59e640 1 -- 172.21.15.143:0/3852831822 learned_addr learned my addr 172.21.15.143:0/3852831822 (peer_addr_for_me v2:172.21.15.143:0/0) 2023-11-26T07:25:02.869 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:25:02.866+0000 7fd32ed9d640 1 --2- 172.21.15.143:0/3852831822 >> v2:172.21.15.188:3300/0 conn(0x7fd33013e290 0x7fd3300cf280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:25:02.870 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:25:02.866+0000 7fd32f59e640 1 -- 172.21.15.143:0/3852831822 >> v2:172.21.15.188:3300/0 conn(0x7fd33013e290 msgr2=0x7fd3300cf280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-26T07:25:02.870 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:25:02.866+0000 7fd32f59e640 1 --2- 172.21.15.143:0/3852831822 >> v2:172.21.15.188:3300/0 conn(0x7fd33013e290 0x7fd3300cf280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:25:02.870 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:25:02.866+0000 7fd32f59e640 1 -- 172.21.15.143:0/3852831822 >> v1:172.21.15.188:6789/0 conn(0x7fd3300cf7c0 legacy=0x7fd33015efa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-26T07:25:02.870 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:25:02.866+0000 7fd32f59e640 1 -- 172.21.15.143:0/3852831822 --> v2:172.21.15.143:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd33006c800 con 0x7fd33015f6b0 2023-11-26T07:25:02.871 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:25:02.870+0000 7fd32f59e640 1 --2- 172.21.15.143:0/3852831822 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fd33015f6b0 0x7fd3300cd7b0 secure :-1 s=READY pgs=1488 cs=0 l=1 rev1=1 crypto rx=0x7fd33006e780 tx=0x7fd32801c940 comp rx=0 tx=0).ready entity=mon.0 client_cookie=333914d8842b87ec server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:25:02.872 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:25:02.870+0000 7fd32e59c640 1 -- 172.21.15.143:0/3852831822 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd3280a5800 con 0x7fd33015f6b0 2023-11-26T07:25:02.872 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:25:02.870+0000 7fd32e59c640 1 -- 172.21.15.143:0/3852831822 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd3280a5960 con 0x7fd33015f6b0 2023-11-26T07:25:02.872 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:25:02.870+0000 7fd32e59c640 1 -- 172.21.15.143:0/3852831822 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd3280a5cf0 con 0x7fd33015f6b0 2023-11-26T07:25:02.873 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:25:02.870+0000 7fd33509c640 1 -- 172.21.15.143:0/3852831822 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fd33015f6b0 msgr2=0x7fd3300cd7b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:25:02.873 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:25:02.870+0000 7fd33509c640 1 --2- 172.21.15.143:0/3852831822 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fd33015f6b0 0x7fd3300cd7b0 secure :-1 s=READY pgs=1488 cs=0 l=1 rev1=1 crypto rx=0x7fd33006e780 tx=0x7fd32801c940 comp rx=0 tx=0).stop 2023-11-26T07:25:02.873 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:25:02.870+0000 7fd33509c640 1 -- 172.21.15.143:0/3852831822 shutdown_connections 2023-11-26T07:25:02.873 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:25:02.870+0000 7fd33509c640 1 --2- 172.21.15.143:0/3852831822 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fd33015f6b0 0x7fd3300cd7b0 unknown :-1 s=CLOSED pgs=1488 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:25:02.873 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:25:02.870+0000 7fd33509c640 1 --2- 172.21.15.143:0/3852831822 >> v2:172.21.15.188:3300/0 conn(0x7fd33013e290 0x7fd3300cf280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:25:02.873 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:25:02.870+0000 7fd33509c640 1 -- 172.21.15.143:0/3852831822 >> 172.21.15.143:0/3852831822 conn(0x7fd33006a890 msgr2=0x7fd33006aca0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:25:02.873 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:25:02.870+0000 7fd33509c640 1 -- 172.21.15.143:0/3852831822 shutdown_connections 2023-11-26T07:25:02.874 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:25:02.870+0000 7fd33509c640 1 -- 172.21.15.143:0/3852831822 wait complete. 2023-11-26T07:25:02.874 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:25:02.874+0000 7fd33509c640 1 Processor -- start 2023-11-26T07:25:02.874 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:25:02.874+0000 7fd33509c640 1 -- start start 2023-11-26T07:25:02.876 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:25:02.874+0000 7fd33509c640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fd3300cf7c0 0x7fd3300b1290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:25:02.876 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:25:02.874+0000 7fd33509c640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fd33013e290 0x7fd330137970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:25:02.876 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:25:02.874+0000 7fd33509c640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fd33015f6b0 0x7fd33015ca70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:25:02.876 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:25:02.874+0000 7fd33509c640 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd330153ec0 con 0x7fd33015f6b0 2023-11-26T07:25:02.876 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:25:02.874+0000 7fd33509c640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd3301541a0 con 0x7fd3300cf7c0 2023-11-26T07:25:02.876 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:25:02.874+0000 7fd33509c640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd330154480 con 0x7fd33013e290 2023-11-26T07:25:02.877 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:25:02.874+0000 7fd32f59e640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fd33015f6b0 0x7fd33015ca70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:25:02.877 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:25:02.874+0000 7fd32ed9d640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fd3300cf7c0 0x7fd3300b1290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:25:02.877 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:25:02.874+0000 7fd32f59e640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fd33015f6b0 0x7fd33015ca70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:46126/0 (socket says 172.21.15.143:46126) 2023-11-26T07:25:02.877 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:25:02.874+0000 7fd32ed9d640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fd3300cf7c0 0x7fd3300b1290 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.188:3300/0 says I am v2:172.21.15.143:55798/0 (socket says 172.21.15.143:55798) 2023-11-26T07:25:02.877 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:25:02.874+0000 7fd32f59e640 1 -- 172.21.15.143:0/339996158 learned_addr learned my addr 172.21.15.143:0/339996158 (peer_addr_for_me v2:172.21.15.143:0/0) 2023-11-26T07:25:02.877 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:25:02.874+0000 7fd327fff640 1 --2- 172.21.15.143:0/339996158 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fd33013e290 0x7fd330137970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:25:02.879 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:25:02.874+0000 7fd32f59e640 1 -- 172.21.15.143:0/339996158 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fd33013e290 msgr2=0x7fd330137970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:25:02.879 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:25:02.874+0000 7fd32f59e640 1 --2- 172.21.15.143:0/339996158 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fd33013e290 0x7fd330137970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:25:02.879 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:25:02.874+0000 7fd32f59e640 1 -- 172.21.15.143:0/339996158 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fd3300cf7c0 msgr2=0x7fd3300b1290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:25:02.880 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:25:02.874+0000 7fd32f59e640 1 --2- 172.21.15.143:0/339996158 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fd3300cf7c0 0x7fd3300b1290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:25:02.880 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:25:02.874+0000 7fd32f59e640 1 -- 172.21.15.143:0/339996158 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd328009b30 con 0x7fd33015f6b0 2023-11-26T07:25:02.880 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:25:02.874+0000 7fd32f59e640 1 --2- 172.21.15.143:0/339996158 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fd33015f6b0 0x7fd33015ca70 secure :-1 s=READY pgs=1489 cs=0 l=1 rev1=1 crypto rx=0x7fd32801a570 tx=0x7fd3280196e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:25:02.880 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:25:02.874+0000 7fd3277fe640 1 -- 172.21.15.143:0/339996158 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd3280b8670 con 0x7fd33015f6b0 2023-11-26T07:25:02.881 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:25:02.878+0000 7fd3277fe640 1 -- 172.21.15.143:0/339996158 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd3280b87d0 con 0x7fd33015f6b0 2023-11-26T07:25:02.881 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:25:02.878+0000 7fd3277fe640 1 -- 172.21.15.143:0/339996158 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd3280b8b60 con 0x7fd33015f6b0 2023-11-26T07:25:02.881 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:25:02.878+0000 7fd33509c640 1 -- 172.21.15.143:0/339996158 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd330154760 con 0x7fd33015f6b0 2023-11-26T07:25:02.881 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:25:02.878+0000 7fd33509c640 1 -- 172.21.15.143:0/339996158 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd330155200 con 0x7fd33015f6b0 2023-11-26T07:25:02.881 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:25:02.878+0000 7fd33509c640 1 -- 172.21.15.143:0/339996158 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7fd330154030 con 0x7fd33015f6b0 2023-11-26T07:25:02.881 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:25:02.878+0000 7fd3277fe640 1 -- 172.21.15.143:0/339996158 <== mon.0 v2:172.21.15.143:3300/0 4 ==== mgrmap(e 18) v1 ==== 86107+0+0 (secure 0 0 0) 0x7fd3280b8cc0 con 0x7fd33015f6b0 2023-11-26T07:25:02.883 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:25:02.878+0000 7fd3277fe640 1 --2- 172.21.15.143:0/339996158 >> [v2:172.21.15.188:6836/3401128553,v1:172.21.15.188:6837/3401128553] conn(0x7fd310066740 0x7fd310068c00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:25:02.883 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:25:02.878+0000 7fd3277fe640 1 -- 172.21.15.143:0/339996158 <== mon.0 v2:172.21.15.143:3300/0 5 ==== osd_map(106..106 src has 1..106) v4 ==== 4943+0+0 (secure 0 0 0) 0x7fd32811f810 con 0x7fd33015f6b0 2023-11-26T07:25:02.883 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:25:02.878+0000 7fd3277fe640 1 --2- 172.21.15.143:0/339996158 >> [v2:172.21.15.188:6824/2722096729,v1:172.21.15.188:6825/2722096729] conn(0x7fd31006c5a0 0x7fd31006e9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:25:02.884 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:25:02.878+0000 7fd3277fe640 1 -- 172.21.15.143:0/339996158 --> [v2:172.21.15.188:6824/2722096729,v1:172.21.15.188:6825/2722096729] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7fd31006f070 con 0x7fd31006c5a0 2023-11-26T07:25:02.884 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:25:02.878+0000 7fd3277fe640 1 -- 172.21.15.143:0/339996158 <== mon.0 v2:172.21.15.143:3300/0 6 ==== mon_get_version_reply(handle=1 version=106) v2 ==== 24+0+0 (secure 0 0 0) 0x7fd32811fb80 con 0x7fd33015f6b0 2023-11-26T07:25:02.884 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:25:02.878+0000 7fd32ed9d640 1 --2- 172.21.15.143:0/339996158 >> [v2:172.21.15.188:6836/3401128553,v1:172.21.15.188:6837/3401128553] conn(0x7fd310066740 0x7fd310068c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:25:02.884 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:25:02.878+0000 7fd327fff640 1 --2- 172.21.15.143:0/339996158 >> [v2:172.21.15.188:6824/2722096729,v1:172.21.15.188:6825/2722096729] conn(0x7fd31006c5a0 0x7fd31006e9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:25:02.884 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:25:02.882+0000 7fd32ed9d640 1 --2- 172.21.15.143:0/339996158 >> [v2:172.21.15.188:6836/3401128553,v1:172.21.15.188:6837/3401128553] conn(0x7fd310066740 0x7fd310068c00 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7fd320007a60 tx=0x7fd320008040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:25:02.884 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:25:02.882+0000 7fd327fff640 1 --2- 172.21.15.143:0/339996158 >> [v2:172.21.15.188:6824/2722096729,v1:172.21.15.188:6825/2722096729] conn(0x7fd31006c5a0 0x7fd31006e9c0 crc :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.7 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:25:02.885 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:25:02.882+0000 7fd3277fe640 1 -- 172.21.15.143:0/339996158 <== osd.7 v2:172.21.15.188:6824/2722096729 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+26965 (crc 0 0 0) 0x7fd31006f070 con 0x7fd31006c5a0 2023-11-26T07:25:02.894 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:25:02.894+0000 7fd33509c640 1 -- 172.21.15.143:0/339996158 --> [v2:172.21.15.188:6824/2722096729,v1:172.21.15.188:6825/2722096729] -- command(tid 2: {"prefix": "config set", "var": "osd_debug_deep_scrub_sleep", "val": ["0"]}) v1 -- 0x7fd33015fb20 con 0x7fd31006c5a0 2023-11-26T07:25:02.895 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:25:02.894+0000 7fd3277fe640 1 -- 172.21.15.143:0/339996158 <== osd.7 v2:172.21.15.188:6824/2722096729 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+97 (crc 0 0 0) 0x7fd33015fb20 con 0x7fd31006c5a0 2023-11-26T07:25:02.895 INFO:teuthology.orchestra.run.smithi143.stdout:{ 2023-11-26T07:25:02.895 INFO:teuthology.orchestra.run.smithi143.stdout: "success": "osd_debug_deep_scrub_sleep = '' (not observed, change may require restart) " 2023-11-26T07:25:02.896 INFO:teuthology.orchestra.run.smithi143.stdout:} 2023-11-26T07:25:02.896 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:25:02.894+0000 7fd33509c640 1 -- 172.21.15.143:0/339996158 >> [v2:172.21.15.188:6824/2722096729,v1:172.21.15.188:6825/2722096729] conn(0x7fd31006c5a0 msgr2=0x7fd31006e9c0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:25:02.897 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:25:02.894+0000 7fd33509c640 1 --2- 172.21.15.143:0/339996158 >> [v2:172.21.15.188:6824/2722096729,v1:172.21.15.188:6825/2722096729] conn(0x7fd31006c5a0 0x7fd31006e9c0 crc :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:25:02.897 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:25:02.894+0000 7fd33509c640 1 -- 172.21.15.143:0/339996158 >> [v2:172.21.15.188:6836/3401128553,v1:172.21.15.188:6837/3401128553] conn(0x7fd310066740 msgr2=0x7fd310068c00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:25:02.897 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:25:02.894+0000 7fd33509c640 1 --2- 172.21.15.143:0/339996158 >> [v2:172.21.15.188:6836/3401128553,v1:172.21.15.188:6837/3401128553] conn(0x7fd310066740 0x7fd310068c00 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7fd320007a60 tx=0x7fd320008040 comp rx=0 tx=0).stop 2023-11-26T07:25:02.897 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:25:02.894+0000 7fd33509c640 1 -- 172.21.15.143:0/339996158 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fd33015f6b0 msgr2=0x7fd33015ca70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:25:02.897 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:25:02.894+0000 7fd33509c640 1 --2- 172.21.15.143:0/339996158 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fd33015f6b0 0x7fd33015ca70 secure :-1 s=READY pgs=1489 cs=0 l=1 rev1=1 crypto rx=0x7fd32801a570 tx=0x7fd3280196e0 comp rx=0 tx=0).stop 2023-11-26T07:25:02.897 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:25:02.894+0000 7fd32f59e640 1 -- 172.21.15.143:0/339996158 reap_dead start 2023-11-26T07:25:02.898 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:25:02.894+0000 7fd33509c640 1 -- 172.21.15.143:0/339996158 shutdown_connections 2023-11-26T07:25:02.898 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:25:02.894+0000 7fd33509c640 1 -- 172.21.15.143:0/339996158 >> 172.21.15.143:0/339996158 conn(0x7fd33006a890 msgr2=0x7fd330138310 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:25:02.898 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:25:02.894+0000 7fd33509c640 1 -- 172.21.15.143:0/339996158 shutdown_connections 2023-11-26T07:25:02.898 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:25:02.894+0000 7fd33509c640 1 -- 172.21.15.143:0/339996158 wait complete. 2023-11-26T07:25:02.914 DEBUG:teuthology.orchestra.run.smithi143:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 7 2023-11-26T07:25:03.053 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:25:03.054+0000 7ff19d832640 1 Processor -- start 2023-11-26T07:25:03.054 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:25:03.054+0000 7ff19d832640 1 -- start start 2023-11-26T07:25:03.056 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:25:03.054+0000 7ff19d832640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7ff1980ce560 0x7ff19815ec80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:25:03.056 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:25:03.054+0000 7ff19d832640 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7ff198071d00 0x7ff198072150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:25:03.056 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:25:03.054+0000 7ff19d832640 1 -- --> v1:172.21.15.143:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff19815f310 con 0x7ff198141660 2023-11-26T07:25:03.056 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:25:03.054+0000 7ff19d832640 1 -- --> v2:172.21.15.188:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff19815fbb0 con 0x7ff198071d00 2023-11-26T07:25:03.056 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:25:03.054+0000 7ff19d832640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff1980ce9e0 con 0x7ff1980ce560 2023-11-26T07:25:03.056 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:25:03.054+0000 7ff196ffd640 1 --1- >> v1:172.21.15.143:6789/0 conn(0x7ff198141660 0x7ff198141f30 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.143:6789/0 says I am v1:172.21.15.143:36400/0 (socket says 172.21.15.143:36400) 2023-11-26T07:25:03.057 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:25:03.054+0000 7ff196ffd640 1 -- 172.21.15.143:0/3294889436 learned_addr learned my addr 172.21.15.143:0/3294889436 (peer_addr_for_me v1:172.21.15.143:0/0) 2023-11-26T07:25:03.057 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:25:03.054+0000 7ff1977fe640 1 --2- 172.21.15.143:0/3294889436 >> v2:172.21.15.188:3300/0 conn(0x7ff198071d00 0x7ff198072150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:25:03.057 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:25:03.054+0000 7ff18ffff640 1 --2- 172.21.15.143:0/3294889436 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7ff1980ce560 0x7ff19815ec80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:25:03.057 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:25:03.054+0000 7ff1967fc640 1 -- 172.21.15.143:0/3294889436 <== mon.0 v1:172.21.15.143:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2763880735 0 0) 0x7ff19815f310 con 0x7ff198141660 2023-11-26T07:25:03.057 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:25:03.054+0000 7ff1967fc640 1 -- 172.21.15.143:0/3294889436 --> v1:172.21.15.143:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7ff184003530 con 0x7ff198141660 2023-11-26T07:25:03.057 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:25:03.054+0000 7ff1977fe640 1 -- 172.21.15.143:0/3294889436 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7ff1980ce560 msgr2=0x7ff19815ec80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-26T07:25:03.058 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:25:03.054+0000 7ff1977fe640 1 --2- 172.21.15.143:0/3294889436 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7ff1980ce560 0x7ff19815ec80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:25:03.058 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:25:03.054+0000 7ff1977fe640 1 -- 172.21.15.143:0/3294889436 >> v1:172.21.15.143:6789/0 conn(0x7ff198141660 legacy=0x7ff198141f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:25:03.058 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:25:03.054+0000 7ff1977fe640 1 -- 172.21.15.143:0/3294889436 --> v2:172.21.15.188:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff19815f5f0 con 0x7ff198071d00 2023-11-26T07:25:03.059 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:25:03.058+0000 7ff1977fe640 1 --2- 172.21.15.143:0/3294889436 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7ff198071d00 0x7ff198072150 secure :-1 s=READY pgs=545 cs=0 l=1 rev1=1 crypto rx=0x7ff190009d90 tx=0x7ff19001c950 comp rx=0 tx=0).ready entity=mon.1 client_cookie=f064788ee6bfcf0e server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:25:03.059 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:25:03.058+0000 7ff1967fc640 1 -- 172.21.15.143:0/3294889436 <== mon.1 v2:172.21.15.188:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff1900a5900 con 0x7ff198071d00 2023-11-26T07:25:03.060 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:25:03.058+0000 7ff1967fc640 1 -- 172.21.15.143:0/3294889436 <== mon.1 v2:172.21.15.188:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff1900a5a60 con 0x7ff198071d00 2023-11-26T07:25:03.060 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:25:03.058+0000 7ff1967fc640 1 -- 172.21.15.143:0/3294889436 <== mon.1 v2:172.21.15.188:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff1900a5dd0 con 0x7ff198071d00 2023-11-26T07:25:03.060 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:25:03.058+0000 7ff19d832640 1 -- 172.21.15.143:0/3294889436 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7ff198071d00 msgr2=0x7ff198072150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:25:03.060 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:25:03.058+0000 7ff19d832640 1 --2- 172.21.15.143:0/3294889436 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7ff198071d00 0x7ff198072150 secure :-1 s=READY pgs=545 cs=0 l=1 rev1=1 crypto rx=0x7ff190009d90 tx=0x7ff19001c950 comp rx=0 tx=0).stop 2023-11-26T07:25:03.060 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:25:03.058+0000 7ff19d832640 1 -- 172.21.15.143:0/3294889436 shutdown_connections 2023-11-26T07:25:03.060 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:25:03.058+0000 7ff19d832640 1 --2- 172.21.15.143:0/3294889436 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7ff198071d00 0x7ff198072150 unknown :-1 s=CLOSED pgs=545 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:25:03.061 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:25:03.058+0000 7ff19d832640 1 --2- 172.21.15.143:0/3294889436 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7ff1980ce560 0x7ff19815ec80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:25:03.061 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:25:03.058+0000 7ff19d832640 1 -- 172.21.15.143:0/3294889436 >> 172.21.15.143:0/3294889436 conn(0x7ff19806c3f0 msgr2=0x7ff19806c800 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:25:03.061 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:25:03.058+0000 7ff19d832640 1 -- 172.21.15.143:0/3294889436 shutdown_connections 2023-11-26T07:25:03.061 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:25:03.058+0000 7ff19d832640 1 -- 172.21.15.143:0/3294889436 wait complete. 2023-11-26T07:25:03.061 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:25:03.062+0000 7ff19d832640 1 Processor -- start 2023-11-26T07:25:03.062 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:25:03.062+0000 7ff19d832640 1 -- start start 2023-11-26T07:25:03.063 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:25:03.062+0000 7ff19d832640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7ff198071d00 0x7ff1980b3740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:25:03.063 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:25:03.062+0000 7ff19d832640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7ff1980ce560 0x7ff198159e80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:25:03.063 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:25:03.062+0000 7ff19d832640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7ff198141660 0x7ff1981427c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:25:03.064 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:25:03.062+0000 7ff19d832640 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff19815a3c0 con 0x7ff1980ce560 2023-11-26T07:25:03.064 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:25:03.062+0000 7ff19d832640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff1980b26a0 con 0x7ff198071d00 2023-11-26T07:25:03.064 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:25:03.062+0000 7ff19d832640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff1980b2980 con 0x7ff198141660 2023-11-26T07:25:03.064 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:25:03.062+0000 7ff18ffff640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7ff1980ce560 0x7ff198159e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:25:03.064 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:25:03.062+0000 7ff18ffff640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7ff1980ce560 0x7ff198159e80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:46128/0 (socket says 172.21.15.143:46128) 2023-11-26T07:25:03.064 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:25:03.062+0000 7ff18ffff640 1 -- 172.21.15.143:0/2855590810 learned_addr learned my addr 172.21.15.143:0/2855590810 (peer_addr_for_me v2:172.21.15.143:0/0) 2023-11-26T07:25:03.064 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:25:03.062+0000 7ff196ffd640 1 --2- 172.21.15.143:0/2855590810 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7ff198071d00 0x7ff1980b3740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:25:03.065 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:25:03.062+0000 7ff1977fe640 1 --2- 172.21.15.143:0/2855590810 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7ff198141660 0x7ff1981427c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:25:03.065 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:25:03.062+0000 7ff18ffff640 1 -- 172.21.15.143:0/2855590810 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7ff198141660 msgr2=0x7ff1981427c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:25:03.065 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:25:03.062+0000 7ff18ffff640 1 --2- 172.21.15.143:0/2855590810 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7ff198141660 0x7ff1981427c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:25:03.065 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:25:03.062+0000 7ff18ffff640 1 -- 172.21.15.143:0/2855590810 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7ff198071d00 msgr2=0x7ff1980b3740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:25:03.065 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:25:03.062+0000 7ff18ffff640 1 --2- 172.21.15.143:0/2855590810 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7ff198071d00 0x7ff1980b3740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:25:03.065 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:25:03.062+0000 7ff18ffff640 1 -- 172.21.15.143:0/2855590810 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff190009a70 con 0x7ff1980ce560 2023-11-26T07:25:03.065 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:25:03.062+0000 7ff1977fe640 1 --2- 172.21.15.143:0/2855590810 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7ff198141660 0x7ff1981427c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-26T07:25:03.065 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:25:03.062+0000 7ff18ffff640 1 --2- 172.21.15.143:0/2855590810 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7ff1980ce560 0x7ff198159e80 secure :-1 s=READY pgs=1491 cs=0 l=1 rev1=1 crypto rx=0x7ff18000e930 tx=0x7ff18000edf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:25:03.065 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:25:03.062+0000 7ff194ff9640 1 -- 172.21.15.143:0/2855590810 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff180009690 con 0x7ff1980ce560 2023-11-26T07:25:03.066 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:25:03.062+0000 7ff194ff9640 1 -- 172.21.15.143:0/2855590810 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff18000ca00 con 0x7ff1980ce560 2023-11-26T07:25:03.066 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:25:03.062+0000 7ff194ff9640 1 -- 172.21.15.143:0/2855590810 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff18000cd70 con 0x7ff1980ce560 2023-11-26T07:25:03.066 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:25:03.062+0000 7ff19d832640 1 -- 172.21.15.143:0/2855590810 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff1980cd150 con 0x7ff1980ce560 2023-11-26T07:25:03.066 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:25:03.062+0000 7ff19d832640 1 -- 172.21.15.143:0/2855590810 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff1980b2f40 con 0x7ff1980ce560 2023-11-26T07:25:03.067 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:25:03.062+0000 7ff19d832640 1 -- 172.21.15.143:0/2855590810 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff198067500 con 0x7ff1980ce560 2023-11-26T07:25:03.067 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:25:03.066+0000 7ff194ff9640 1 -- 172.21.15.143:0/2855590810 <== mon.0 v2:172.21.15.143:3300/0 4 ==== mgrmap(e 18) v1 ==== 86107+0+0 (secure 0 0 0) 0x7ff180013470 con 0x7ff1980ce560 2023-11-26T07:25:03.067 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:25:03.066+0000 7ff194ff9640 1 --2- 172.21.15.143:0/2855590810 >> [v2:172.21.15.188:6836/3401128553,v1:172.21.15.188:6837/3401128553] conn(0x7ff178066740 0x7ff178068c00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:25:03.067 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:25:03.066+0000 7ff194ff9640 1 -- 172.21.15.143:0/2855590810 <== mon.0 v2:172.21.15.143:3300/0 5 ==== osd_map(106..106 src has 1..106) v4 ==== 4943+0+0 (secure 0 0 0) 0x7ff180084170 con 0x7ff1980ce560 2023-11-26T07:25:03.067 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:25:03.066+0000 7ff196ffd640 1 --2- 172.21.15.143:0/2855590810 >> [v2:172.21.15.188:6836/3401128553,v1:172.21.15.188:6837/3401128553] conn(0x7ff178066740 0x7ff178068c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:25:03.068 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:25:03.066+0000 7ff196ffd640 1 --2- 172.21.15.143:0/2855590810 >> [v2:172.21.15.188:6836/3401128553,v1:172.21.15.188:6837/3401128553] conn(0x7ff178066740 0x7ff178068c00 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7ff188007fd0 tx=0x7ff188009040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:25:03.070 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:25:03.070+0000 7ff194ff9640 1 -- 172.21.15.143:0/2855590810 <== mon.0 v2:172.21.15.143:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172649 (secure 0 0 0) 0x7ff180051760 con 0x7ff1980ce560 2023-11-26T07:25:03.218 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:25:03.218+0000 7ff19d832640 1 -- 172.21.15.143:0/2855590810 --> [v2:172.21.15.188:6836/3401128553,v1:172.21.15.188:6837/3401128553] -- mgr_command(tid 0: {"prefix": "osd deep-scrub", "who": "7", "target": ["mon-mgr", ""]}) v1 -- 0x7ff1980ccfd0 con 0x7ff178066740 2023-11-26T07:25:03.219 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:25:03.218+0000 7ff194ff9640 1 -- 172.21.15.143:0/2855590810 <== mgr.4108 v2:172.21.15.188:6836/3401128553 1 ==== mgr_command_reply(tid 0: 0 instructed osd(s) 7 to deep-scrub) v1 ==== 41+0+0 (secure 0 0 0) 0x7ff18800f820 con 0x7ff178066740 2023-11-26T07:25:03.219 INFO:teuthology.orchestra.run.smithi143.stderr:instructed osd(s) 7 to deep-scrub 2023-11-26T07:25:03.220 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:25:03.218+0000 7ff19d832640 1 -- 172.21.15.143:0/2855590810 >> [v2:172.21.15.188:6836/3401128553,v1:172.21.15.188:6837/3401128553] conn(0x7ff178066740 msgr2=0x7ff178068c00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:25:03.221 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:25:03.218+0000 7ff19d832640 1 --2- 172.21.15.143:0/2855590810 >> [v2:172.21.15.188:6836/3401128553,v1:172.21.15.188:6837/3401128553] conn(0x7ff178066740 0x7ff178068c00 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7ff188007fd0 tx=0x7ff188009040 comp rx=0 tx=0).stop 2023-11-26T07:25:03.221 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:25:03.218+0000 7ff19d832640 1 -- 172.21.15.143:0/2855590810 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7ff1980ce560 msgr2=0x7ff198159e80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:25:03.221 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:25:03.218+0000 7ff19d832640 1 --2- 172.21.15.143:0/2855590810 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7ff1980ce560 0x7ff198159e80 secure :-1 s=READY pgs=1491 cs=0 l=1 rev1=1 crypto rx=0x7ff18000e930 tx=0x7ff18000edf0 comp rx=0 tx=0).stop 2023-11-26T07:25:03.221 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:25:03.218+0000 7ff19d832640 1 -- 172.21.15.143:0/2855590810 shutdown_connections 2023-11-26T07:25:03.221 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:25:03.218+0000 7ff19d832640 1 --2- 172.21.15.143:0/2855590810 >> [v2:172.21.15.188:6836/3401128553,v1:172.21.15.188:6837/3401128553] conn(0x7ff178066740 0x7ff178068c00 unknown :-1 s=CLOSED pgs=202 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:25:03.221 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:25:03.218+0000 7ff19d832640 1 --2- 172.21.15.143:0/2855590810 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7ff198141660 0x7ff1981427c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:25:03.222 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:25:03.218+0000 7ff19d832640 1 --2- 172.21.15.143:0/2855590810 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7ff1980ce560 0x7ff198159e80 unknown :-1 s=CLOSED pgs=1491 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:25:03.222 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:25:03.218+0000 7ff19d832640 1 --2- 172.21.15.143:0/2855590810 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7ff198071d00 0x7ff1980b3740 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:25:03.222 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:25:03.218+0000 7ff19d832640 1 -- 172.21.15.143:0/2855590810 >> 172.21.15.143:0/2855590810 conn(0x7ff19806c3f0 msgr2=0x7ff19813c4f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:25:03.222 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:25:03.218+0000 7ff19d832640 1 -- 172.21.15.143:0/2855590810 shutdown_connections 2023-11-26T07:25:03.222 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:25:03.222+0000 7ff19d832640 1 -- 172.21.15.143:0/2855590810 wait complete. 2023-11-26T07:25:03.240 DEBUG:teuthology.orchestra.run.smithi143:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-11-26T07:25:03.378 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:25:03.378+0000 7fc6ba5a1640 1 Processor -- start 2023-11-26T07:25:03.378 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:25:03.378+0000 7fc6ba5a1640 1 -- start start 2023-11-26T07:25:03.379 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:25:03.378+0000 7fc6ba5a1640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fc6b41596d0 0x7fc6b4071850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:25:03.379 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:25:03.378+0000 7fc6ba5a1640 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7fc6b40b2f80 0x7fc6b406cbe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:25:03.379 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:25:03.378+0000 7fc6ba5a1640 1 -- --> v1:172.21.15.143:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc6b406c750 con 0x7fc6b40ce680 2023-11-26T07:25:03.379 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:25:03.378+0000 7fc6ba5a1640 1 -- --> v2:172.21.15.143:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc6b40cbe00 con 0x7fc6b40b2f80 2023-11-26T07:25:03.379 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:25:03.378+0000 7fc6ba5a1640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc6b40b3550 con 0x7fc6b41596d0 2023-11-26T07:25:03.380 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:25:03.378+0000 7fc6b8b3e640 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7fc6b40b2f80 0x7fc6b406cbe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:25:03.380 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:25:03.378+0000 7fc6b37fe640 1 --1- >> v1:172.21.15.143:6789/0 conn(0x7fc6b40ce680 0x7fc6b40b2870 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.143:6789/0 says I am v1:172.21.15.143:36406/0 (socket says 172.21.15.143:36406) 2023-11-26T07:25:03.380 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:25:03.378+0000 7fc6b37fe640 1 -- 172.21.15.143:0/2309239563 learned_addr learned my addr 172.21.15.143:0/2309239563 (peer_addr_for_me v1:172.21.15.143:0/0) 2023-11-26T07:25:03.380 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:25:03.378+0000 7fc6b3fff640 1 --2- 172.21.15.143:0/2309239563 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fc6b41596d0 0x7fc6b4071850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:25:03.381 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:25:03.378+0000 7fc6b8b3e640 1 -- 172.21.15.143:0/2309239563 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fc6b41596d0 msgr2=0x7fc6b4071850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-26T07:25:03.381 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:25:03.378+0000 7fc6b8b3e640 1 --2- 172.21.15.143:0/2309239563 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fc6b41596d0 0x7fc6b4071850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:25:03.381 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:25:03.378+0000 7fc6b8b3e640 1 -- 172.21.15.143:0/2309239563 >> v1:172.21.15.143:6789/0 conn(0x7fc6b40ce680 legacy=0x7fc6b40b2870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:25:03.381 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:25:03.378+0000 7fc6b8b3e640 1 -- 172.21.15.143:0/2309239563 --> v2:172.21.15.143:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc6b4071e80 con 0x7fc6b40b2f80 2023-11-26T07:25:03.382 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:25:03.382+0000 7fc6b8b3e640 1 --2- 172.21.15.143:0/2309239563 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fc6b40b2f80 0x7fc6b406cbe0 secure :-1 s=READY pgs=1493 cs=0 l=1 rev1=1 crypto rx=0x7fc6ac009d90 tx=0x7fc6ac01c950 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f99528cdf90307b1 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:25:03.383 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:25:03.382+0000 7fc6b2ffd640 1 -- 172.21.15.143:0/2309239563 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc6ac0a5880 con 0x7fc6b40b2f80 2023-11-26T07:25:03.383 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:25:03.382+0000 7fc6b2ffd640 1 -- 172.21.15.143:0/2309239563 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc6ac0a59e0 con 0x7fc6b40b2f80 2023-11-26T07:25:03.383 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:25:03.382+0000 7fc6b2ffd640 1 -- 172.21.15.143:0/2309239563 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc6ac0a5d70 con 0x7fc6b40b2f80 2023-11-26T07:25:03.384 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:25:03.382+0000 7fc6ba5a1640 1 -- 172.21.15.143:0/2309239563 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fc6b40b2f80 msgr2=0x7fc6b406cbe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:25:03.384 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:25:03.382+0000 7fc6ba5a1640 1 --2- 172.21.15.143:0/2309239563 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fc6b40b2f80 0x7fc6b406cbe0 secure :-1 s=READY pgs=1493 cs=0 l=1 rev1=1 crypto rx=0x7fc6ac009d90 tx=0x7fc6ac01c950 comp rx=0 tx=0).stop 2023-11-26T07:25:03.384 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:25:03.382+0000 7fc6ba5a1640 1 -- 172.21.15.143:0/2309239563 shutdown_connections 2023-11-26T07:25:03.384 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:25:03.382+0000 7fc6ba5a1640 1 --2- 172.21.15.143:0/2309239563 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fc6b40b2f80 0x7fc6b406cbe0 unknown :-1 s=CLOSED pgs=1493 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:25:03.384 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:25:03.382+0000 7fc6ba5a1640 1 --2- 172.21.15.143:0/2309239563 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fc6b41596d0 0x7fc6b4071850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:25:03.384 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:25:03.382+0000 7fc6ba5a1640 1 -- 172.21.15.143:0/2309239563 >> 172.21.15.143:0/2309239563 conn(0x7fc6b406a890 msgr2=0x7fc6b406aca0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:25:03.385 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:25:03.382+0000 7fc6ba5a1640 1 -- 172.21.15.143:0/2309239563 shutdown_connections 2023-11-26T07:25:03.385 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:25:03.382+0000 7fc6ba5a1640 1 -- 172.21.15.143:0/2309239563 wait complete. 2023-11-26T07:25:03.385 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:25:03.386+0000 7fc6ba5a1640 1 Processor -- start 2023-11-26T07:25:03.385 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:25:03.386+0000 7fc6ba5a1640 1 -- start start 2023-11-26T07:25:03.386 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:25:03.386+0000 7fc6ba5a1640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fc6b40b2f80 0x7fc6b40cc7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:25:03.387 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:25:03.386+0000 7fc6ba5a1640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fc6b40ce680 0x7fc6b40cd5a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:25:03.387 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:25:03.386+0000 7fc6ba5a1640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fc6b41596d0 0x7fc6b40cdae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:25:03.387 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:25:03.386+0000 7fc6ba5a1640 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc6b414a4a0 con 0x7fc6b40ce680 2023-11-26T07:25:03.387 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:25:03.386+0000 7fc6ba5a1640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc6b41f6c10 con 0x7fc6b41596d0 2023-11-26T07:25:03.387 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:25:03.386+0000 7fc6ba5a1640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc6b40ccce0 con 0x7fc6b40b2f80 2023-11-26T07:25:03.387 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:25:03.386+0000 7fc6b37fe640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fc6b40ce680 0x7fc6b40cd5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:25:03.387 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:25:03.386+0000 7fc6b37fe640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fc6b40ce680 0x7fc6b40cd5a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:46146/0 (socket says 172.21.15.143:46146) 2023-11-26T07:25:03.388 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:25:03.386+0000 7fc6b37fe640 1 -- 172.21.15.143:0/3729954635 learned_addr learned my addr 172.21.15.143:0/3729954635 (peer_addr_for_me v2:172.21.15.143:0/0) 2023-11-26T07:25:03.388 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:25:03.386+0000 7fc6b8b3e640 1 --2- 172.21.15.143:0/3729954635 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fc6b41596d0 0x7fc6b40cdae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:25:03.388 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:25:03.386+0000 7fc6b3fff640 1 --2- 172.21.15.143:0/3729954635 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fc6b40b2f80 0x7fc6b40cc7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:25:03.389 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:25:03.386+0000 7fc6b37fe640 1 -- 172.21.15.143:0/3729954635 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fc6b40b2f80 msgr2=0x7fc6b40cc7a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:25:03.389 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:25:03.386+0000 7fc6b37fe640 1 --2- 172.21.15.143:0/3729954635 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fc6b40b2f80 0x7fc6b40cc7a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:25:03.389 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:25:03.386+0000 7fc6b37fe640 1 -- 172.21.15.143:0/3729954635 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fc6b41596d0 msgr2=0x7fc6b40cdae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:25:03.389 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:25:03.386+0000 7fc6b37fe640 1 --2- 172.21.15.143:0/3729954635 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fc6b41596d0 0x7fc6b40cdae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:25:03.390 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:25:03.386+0000 7fc6b37fe640 1 -- 172.21.15.143:0/3729954635 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc6ac009a70 con 0x7fc6b40ce680 2023-11-26T07:25:03.390 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:25:03.386+0000 7fc6b37fe640 1 --2- 172.21.15.143:0/3729954635 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fc6b40ce680 0x7fc6b40cd5a0 secure :-1 s=READY pgs=1494 cs=0 l=1 rev1=1 crypto rx=0x7fc6a800c500 tx=0x7fc6a800c9c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:25:03.390 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:25:03.386+0000 7fc6b0ff9640 1 -- 172.21.15.143:0/3729954635 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc6a8005cf0 con 0x7fc6b40ce680 2023-11-26T07:25:03.390 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:25:03.386+0000 7fc6b0ff9640 1 -- 172.21.15.143:0/3729954635 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc6a8005e50 con 0x7fc6b40ce680 2023-11-26T07:25:03.390 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:25:03.386+0000 7fc6b0ff9640 1 -- 172.21.15.143:0/3729954635 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc6a80072f0 con 0x7fc6b40ce680 2023-11-26T07:25:03.390 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:25:03.386+0000 7fc6ba5a1640 1 -- 172.21.15.143:0/3729954635 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc6b41f6000 con 0x7fc6b40ce680 2023-11-26T07:25:03.390 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:25:03.386+0000 7fc6ba5a1640 1 -- 172.21.15.143:0/3729954635 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc6b41f65c0 con 0x7fc6b40ce680 2023-11-26T07:25:03.390 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:25:03.390+0000 7fc6ba5a1640 1 -- 172.21.15.143:0/3729954635 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc6b4067680 con 0x7fc6b40ce680 2023-11-26T07:25:03.391 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:25:03.390+0000 7fc6b0ff9640 1 -- 172.21.15.143:0/3729954635 <== mon.0 v2:172.21.15.143:3300/0 4 ==== mgrmap(e 18) v1 ==== 86107+0+0 (secure 0 0 0) 0x7fc6a8007450 con 0x7fc6b40ce680 2023-11-26T07:25:03.391 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:25:03.390+0000 7fc6b0ff9640 1 --2- 172.21.15.143:0/3729954635 >> [v2:172.21.15.188:6836/3401128553,v1:172.21.15.188:6837/3401128553] conn(0x7fc6a0066740 0x7fc6a0068c00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:25:03.392 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:25:03.390+0000 7fc6b0ff9640 1 -- 172.21.15.143:0/3729954635 <== mon.0 v2:172.21.15.143:3300/0 5 ==== osd_map(106..106 src has 1..106) v4 ==== 4943+0+0 (secure 0 0 0) 0x7fc6a8083450 con 0x7fc6b40ce680 2023-11-26T07:25:03.392 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:25:03.390+0000 7fc6b3fff640 1 --2- 172.21.15.143:0/3729954635 >> [v2:172.21.15.188:6836/3401128553,v1:172.21.15.188:6837/3401128553] conn(0x7fc6a0066740 0x7fc6a0068c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:25:03.392 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:25:03.390+0000 7fc6b3fff640 1 --2- 172.21.15.143:0/3729954635 >> [v2:172.21.15.188:6836/3401128553,v1:172.21.15.188:6837/3401128553] conn(0x7fc6a0066740 0x7fc6a0068c00 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7fc6a400d980 tx=0x7fc6a4008040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:25:03.394 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:25:03.394+0000 7fc6b0ff9640 1 -- 172.21.15.143:0/3729954635 <== mon.0 v2:172.21.15.143:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172649 (secure 0 0 0) 0x7fc6a8050a00 con 0x7fc6b40ce680 2023-11-26T07:25:03.543 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:25:03.542+0000 7fc6ba5a1640 1 -- 172.21.15.143:0/3729954635 --> [v2:172.21.15.188:6836/3401128553,v1:172.21.15.188:6837/3401128553] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7fc6b40709f0 con 0x7fc6a0066740 2023-11-26T07:25:03.545 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:25:03.546+0000 7fc6b0ff9640 1 -- 172.21.15.143:0/3729954635 <== mgr.4108 v2:172.21.15.188:6836/3401128553 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+50834 (secure 0 0 0) 0x7fc6b40709f0 con 0x7fc6a0066740 2023-11-26T07:25:03.546 INFO:teuthology.orchestra.run.smithi143.stdout: 2023-11-26T07:25:03.546 INFO:teuthology.orchestra.run.smithi143.stderr:dumped all 2023-11-26T07:25:03.547 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:25:03.546+0000 7fc6ba5a1640 1 -- 172.21.15.143:0/3729954635 >> [v2:172.21.15.188:6836/3401128553,v1:172.21.15.188:6837/3401128553] conn(0x7fc6a0066740 msgr2=0x7fc6a0068c00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:25:03.548 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:25:03.546+0000 7fc6ba5a1640 1 --2- 172.21.15.143:0/3729954635 >> [v2:172.21.15.188:6836/3401128553,v1:172.21.15.188:6837/3401128553] conn(0x7fc6a0066740 0x7fc6a0068c00 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7fc6a400d980 tx=0x7fc6a4008040 comp rx=0 tx=0).stop 2023-11-26T07:25:03.548 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:25:03.546+0000 7fc6ba5a1640 1 -- 172.21.15.143:0/3729954635 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fc6b40ce680 msgr2=0x7fc6b40cd5a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:25:03.548 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:25:03.546+0000 7fc6ba5a1640 1 --2- 172.21.15.143:0/3729954635 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fc6b40ce680 0x7fc6b40cd5a0 secure :-1 s=READY pgs=1494 cs=0 l=1 rev1=1 crypto rx=0x7fc6a800c500 tx=0x7fc6a800c9c0 comp rx=0 tx=0).stop 2023-11-26T07:25:03.548 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:25:03.546+0000 7fc6ba5a1640 1 -- 172.21.15.143:0/3729954635 shutdown_connections 2023-11-26T07:25:03.548 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:25:03.546+0000 7fc6ba5a1640 1 --2- 172.21.15.143:0/3729954635 >> [v2:172.21.15.188:6836/3401128553,v1:172.21.15.188:6837/3401128553] conn(0x7fc6a0066740 0x7fc6a0068c00 unknown :-1 s=CLOSED pgs=203 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:25:03.548 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:25:03.546+0000 7fc6ba5a1640 1 --2- 172.21.15.143:0/3729954635 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fc6b41596d0 0x7fc6b40cdae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:25:03.548 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:25:03.546+0000 7fc6ba5a1640 1 --2- 172.21.15.143:0/3729954635 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fc6b40ce680 0x7fc6b40cd5a0 unknown :-1 s=CLOSED pgs=1494 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:25:03.549 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:25:03.546+0000 7fc6ba5a1640 1 --2- 172.21.15.143:0/3729954635 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fc6b40b2f80 0x7fc6b40cc7a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:25:03.549 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:25:03.546+0000 7fc6ba5a1640 1 -- 172.21.15.143:0/3729954635 >> 172.21.15.143:0/3729954635 conn(0x7fc6b406a890 msgr2=0x7fc6b40aeb60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:25:03.549 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:25:03.546+0000 7fc6ba5a1640 1 -- 172.21.15.143:0/3729954635 shutdown_connections 2023-11-26T07:25:03.549 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:25:03.546+0000 7fc6ba5a1640 1 -- 172.21.15.143:0/3729954635 wait complete. 2023-11-26T07:25:03.567 INFO:teuthology.orchestra.run.smithi143.stdout:{"pg_ready":true,"pg_map":{"version":94,"stamp":"2023-11-26T07:25:03.311543+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":212,"num_read_kb":306,"num_write":179,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":128,"ondisk_log_size":128,"up":2,"acting":2,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":2,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":2,"kb":754974720,"kb_used":260972,"kb_used_data":11528,"kb_used_omap":12,"kb_used_meta":249331,"kb_avail":754713748,"statfs":{"total":773094113280,"available":772826877952,"internally_reserved":0,"allocated":11804672,"data_stored":5820552,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12749,"internal_metadata":255315507},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-16777869,"num_objects":-19,"num_object_clones":0,"num_object_copies":-38,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":-19,"num_whiteouts":0,"num_read":-17,"num_read_kb":-8201,"num_write":-31,"num_write_kb":-16393,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"9.623732"},"pg_stats":[{"pgid":"1.0","version":"72'128","reported_seq":275,"reported_epoch":104,"state":"active+clean","last_fresh":"2023-11-26T07:24:53.071827+0000","last_change":"2023-11-26T07:17:43.827802+0000","last_active":"2023-11-26T07:24:53.071827+0000","last_peered":"2023-11-26T07:24:53.071827+0000","last_clean":"2023-11-26T07:24:53.071827+0000","last_became_active":"2023-11-26T07:17:43.827308+0000","last_became_peered":"2023-11-26T07:17:43.827308+0000","last_unstale":"2023-11-26T07:24:53.071827+0000","last_undegraded":"2023-11-26T07:24:53.071827+0000","last_fullsized":"2023-11-26T07:24:53.071827+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:17:42.160945+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:17:42.160945+0000","last_clean_scrub_stamp":"2023-11-26T07:17:42.160945+0000","objects_scrubbed":0,"log_size":128,"log_dups_size":0,"ondisk_log_size":128,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T08:17:54.167366+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":212,"num_read_kb":306,"num_write":179,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":212,"num_read_kb":306,"num_write":179,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":128,"ondisk_log_size":128,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574938,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":33056,"kb_used_data":1876,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94338784,"statfs":{"total":96636764160,"available":96602914816,"internally_reserved":0,"allocated":1921024,"data_stored":1170345,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":31914442},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Nov 26 07:25:01 2023","interfaces":[{"interface":"back","average":{"1min":0.66700000000000004,"5min":0.70399999999999996,"15min":0.70799999999999996},"min":{"1min":0.53700000000000003,"5min":0.42999999999999999,"15min":0.42999999999999999},"max":{"1min":0.79300000000000004,"5min":1.036,"15min":1.036},"last":0.70899999999999996},{"interface":"front","average":{"1min":0.68400000000000005,"5min":0.71999999999999997,"15min":0.71299999999999997},"min":{"1min":0.47499999999999998,"5min":0.42299999999999999,"15min":0.42299999999999999},"max":{"1min":0.85799999999999998,"5min":1.014,"15min":1.0980000000000001},"last":0.79400000000000004}]},{"osd":1,"last update":"Sun Nov 26 07:25:01 2023","interfaces":[{"interface":"back","average":{"1min":0.67400000000000004,"5min":0.68899999999999995,"15min":0.68300000000000005},"min":{"1min":0.442,"5min":0.435,"15min":0.435},"max":{"1min":0.82299999999999995,"5min":1.012,"15min":1.0669999999999999},"last":0.77400000000000002},{"interface":"front","average":{"1min":0.61799999999999999,"5min":0.67300000000000004,"15min":0.77000000000000002},"min":{"1min":0.44600000000000001,"5min":0.432,"15min":0.432},"max":{"1min":0.81599999999999995,"5min":0.94699999999999995,"15min":4.5309999999999997},"last":0.752}]},{"osd":2,"last update":"Sun Nov 26 07:25:01 2023","interfaces":[{"interface":"back","average":{"1min":0.61599999999999999,"5min":0.68899999999999995,"15min":0.68000000000000005},"min":{"1min":0.51500000000000001,"5min":0.39700000000000002,"15min":0.39700000000000002},"max":{"1min":0.73999999999999999,"5min":1.0800000000000001,"15min":1.0800000000000001},"last":0.72799999999999998},{"interface":"front","average":{"1min":0.67200000000000004,"5min":0.70499999999999996,"15min":0.70699999999999996},"min":{"1min":0.47299999999999998,"5min":0.40999999999999998,"15min":0.40999999999999998},"max":{"1min":0.89300000000000002,"5min":0.97699999999999998,"15min":1.1339999999999999},"last":0.60999999999999999}]},{"osd":3,"last update":"Sun Nov 26 07:25:01 2023","interfaces":[{"interface":"back","average":{"1min":0.65400000000000003,"5min":0.71499999999999997,"15min":0.70399999999999996},"min":{"1min":0.48999999999999999,"5min":0.48999999999999999,"15min":0.47799999999999998},"max":{"1min":0.81000000000000005,"5min":1.1240000000000001,"15min":1.1240000000000001},"last":0.63700000000000001},{"interface":"front","average":{"1min":0.70199999999999996,"5min":0.73899999999999999,"15min":0.73399999999999999},"min":{"1min":0.58699999999999997,"5min":0.433,"15min":0.433},"max":{"1min":0.91900000000000004,"5min":1.036,"15min":1.4490000000000001},"last":0.83099999999999996}]},{"osd":4,"last update":"Sun Nov 26 07:25:01 2023","interfaces":[{"interface":"back","average":{"1min":0.58999999999999997,"5min":0.60099999999999998,"15min":0.60399999999999998},"min":{"1min":0.36299999999999999,"5min":0.36299999999999999,"15min":0.36299999999999999},"max":{"1min":0.84699999999999998,"5min":0.96299999999999997,"15min":0.96299999999999997},"last":0.58799999999999997},{"interface":"front","average":{"1min":0.60299999999999998,"5min":0.63500000000000001,"15min":0.65800000000000003},"min":{"1min":0.42899999999999999,"5min":0.35999999999999999,"15min":0.35999999999999999},"max":{"1min":0.75900000000000001,"5min":1.0209999999999999,"15min":1.5509999999999999},"last":0.76500000000000001}]},{"osd":5,"last update":"Sun Nov 26 07:25:01 2023","interfaces":[{"interface":"back","average":{"1min":0.65000000000000002,"5min":0.65800000000000003,"15min":0.66700000000000004},"min":{"1min":0.495,"5min":0.36599999999999999,"15min":0.36599999999999999},"max":{"1min":0.82299999999999995,"5min":1.0029999999999999,"15min":1.415},"last":0.66400000000000003},{"interface":"front","average":{"1min":0.63600000000000001,"5min":0.67100000000000004,"15min":0.69799999999999995},"min":{"1min":0.48399999999999999,"5min":0.435,"15min":0.435},"max":{"1min":0.89900000000000002,"5min":1.0840000000000001,"15min":1.3859999999999999},"last":0.68500000000000005}]},{"osd":6,"last update":"Sun Nov 26 07:25:01 2023","interfaces":[{"interface":"back","average":{"1min":0.72699999999999998,"5min":0.76800000000000002,"15min":0.77100000000000002},"min":{"1min":0.55400000000000005,"5min":0.52400000000000002,"15min":0.51700000000000002},"max":{"1min":0.93899999999999995,"5min":1.123,"15min":1.3069999999999999},"last":0.84199999999999997},{"interface":"front","average":{"1min":0.66100000000000003,"5min":0.72299999999999998,"15min":0.754},"min":{"1min":0.51600000000000001,"5min":0.46899999999999997,"15min":0.46899999999999997},"max":{"1min":0.86899999999999999,"5min":1.032,"15min":1.6759999999999999},"last":0.81399999999999995}]}]},{"osd":6,"up_from":13,"seq":55834574940,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":32476,"kb_used_data":1296,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339364,"statfs":{"total":96636764160,"available":96603508736,"internally_reserved":0,"allocated":1327104,"data_stored":579977,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Nov 26 07:24:58 2023","interfaces":[{"interface":"back","average":{"1min":0.67800000000000005,"5min":0.67600000000000005,"15min":0.68100000000000005},"min":{"1min":0.56299999999999994,"5min":0.32200000000000001,"15min":0.32200000000000001},"max":{"1min":0.88700000000000001,"5min":0.91800000000000004,"15min":1.7549999999999999},"last":0.75},{"interface":"front","average":{"1min":0.69799999999999995,"5min":0.69799999999999995,"15min":0.85299999999999998},"min":{"1min":0.44900000000000001,"5min":0.36299999999999999,"15min":0.36299999999999999},"max":{"1min":0.89200000000000002,"5min":0.90500000000000003,"15min":5.7489999999999997},"last":0.73199999999999998}]},{"osd":1,"last update":"Sun Nov 26 07:24:58 2023","interfaces":[{"interface":"back","average":{"1min":0.68000000000000005,"5min":0.70499999999999996,"15min":0.82899999999999996},"min":{"1min":0.41599999999999998,"5min":0.41599999999999998,"15min":0.41199999999999998},"max":{"1min":0.94699999999999995,"5min":0.97799999999999998,"15min":5.4240000000000004},"last":0.70499999999999996},{"interface":"front","average":{"1min":0.71299999999999997,"5min":0.70999999999999996,"15min":0.82199999999999995},"min":{"1min":0.497,"5min":0.47599999999999998,"15min":0.47599999999999998},"max":{"1min":0.88900000000000001,"5min":0.96499999999999997,"15min":5.6020000000000003},"last":0.68400000000000005}]},{"osd":2,"last update":"Sun Nov 26 07:24:58 2023","interfaces":[{"interface":"back","average":{"1min":0.67700000000000005,"5min":0.67300000000000004,"15min":0.68500000000000005},"min":{"1min":0.46200000000000002,"5min":0.41699999999999998,"15min":0.41699999999999998},"max":{"1min":1.002,"5min":1.002,"15min":1.8520000000000001},"last":0.57499999999999996},{"interface":"front","average":{"1min":0.70399999999999996,"5min":0.72099999999999997,"15min":0.72799999999999998},"min":{"1min":0.44900000000000001,"5min":0.44900000000000001,"15min":0.437},"max":{"1min":0.86399999999999999,"5min":0.95199999999999996,"15min":1.9159999999999999},"last":0.79200000000000004}]},{"osd":3,"last update":"Sun Nov 26 07:24:58 2023","interfaces":[{"interface":"back","average":{"1min":0.69999999999999996,"5min":0.70899999999999996,"15min":0.82399999999999995},"min":{"1min":0.42499999999999999,"5min":0.42499999999999999,"15min":0.42499999999999999},"max":{"1min":0.93500000000000005,"5min":0.93500000000000005,"15min":5.4390000000000001},"last":0.77200000000000002},{"interface":"front","average":{"1min":0.66100000000000003,"5min":0.69899999999999995,"15min":0.84499999999999997},"min":{"1min":0.44600000000000001,"5min":0.44,"15min":0.44},"max":{"1min":0.96499999999999997,"5min":0.96499999999999997,"15min":5.5490000000000004},"last":0.66700000000000004}]},{"osd":4,"last update":"Sun Nov 26 07:24:58 2023","interfaces":[{"interface":"back","average":{"1min":0.56299999999999994,"5min":0.58599999999999997,"15min":0.626},"min":{"1min":0.39500000000000002,"5min":0.39500000000000002,"15min":0.371},"max":{"1min":0.75900000000000001,"5min":0.95599999999999996,"15min":1.0129999999999999},"last":0.54400000000000004},{"interface":"front","average":{"1min":0.59199999999999997,"5min":0.61099999999999999,"15min":0.65600000000000003},"min":{"1min":0.44,"5min":0.38600000000000001,"15min":0.38600000000000001},"max":{"1min":0.751,"5min":0.86399999999999999,"15min":1.0349999999999999},"last":0.59499999999999997}]},{"osd":5,"last update":"Sun Nov 26 07:24:58 2023","interfaces":[{"interface":"back","average":{"1min":0.60699999999999998,"5min":0.60999999999999999,"15min":0.64200000000000002},"min":{"1min":0.46800000000000003,"5min":0.40699999999999997,"15min":0.38200000000000001},"max":{"1min":0.93500000000000005,"5min":0.93500000000000005,"15min":0.93500000000000005},"last":0.51200000000000001},{"interface":"front","average":{"1min":0.64800000000000002,"5min":0.66400000000000003,"15min":0.69999999999999996},"min":{"1min":0.46600000000000003,"5min":0.44,"15min":0.436},"max":{"1min":0.84299999999999997,"5min":1.014,"15min":1.0960000000000001},"last":0.63900000000000001}]},{"osd":7,"last update":"Sun Nov 26 07:24:58 2023","interfaces":[{"interface":"back","average":{"1min":0.67400000000000004,"5min":0.67800000000000005,"15min":0.72299999999999998},"min":{"1min":0.52100000000000002,"5min":0.42599999999999999,"15min":0.42599999999999999},"max":{"1min":0.82299999999999995,"5min":0.95099999999999996,"15min":0.998},"last":0.81499999999999995},{"interface":"front","average":{"1min":0.65900000000000003,"5min":0.66600000000000004,"15min":0.70099999999999996},"min":{"1min":0.46300000000000002,"5min":0.46300000000000002,"15min":0.46300000000000002},"max":{"1min":0.91200000000000003,"5min":0.91200000000000003,"15min":0.98799999999999999},"last":0.72199999999999998}]}]},{"osd":5,"up_from":13,"seq":55834574938,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":32476,"kb_used_data":1296,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339364,"statfs":{"total":96636764160,"available":96603508736,"internally_reserved":0,"allocated":1327104,"data_stored":579977,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Nov 26 07:24:50 2023","interfaces":[{"interface":"back","average":{"1min":0.61899999999999999,"5min":0.70499999999999996,"15min":0.68100000000000005},"min":{"1min":0.29699999999999999,"5min":0.29699999999999999,"15min":0.29699999999999999},"max":{"1min":0.80800000000000005,"5min":2.6949999999999998,"15min":2.6949999999999998},"last":0.73899999999999999},{"interface":"front","average":{"1min":0.64800000000000002,"5min":0.72399999999999998,"15min":0.68200000000000005},"min":{"1min":0.40200000000000002,"5min":0.38200000000000001,"15min":0.38200000000000001},"max":{"1min":0.88600000000000001,"5min":2.0150000000000001,"15min":2.0150000000000001},"last":0.63800000000000001}]},{"osd":1,"last update":"Sun Nov 26 07:24:50 2023","interfaces":[{"interface":"back","average":{"1min":0.60999999999999999,"5min":0.69099999999999995,"15min":0.67700000000000005},"min":{"1min":0.45900000000000002,"5min":0.41799999999999998,"15min":0.41599999999999998},"max":{"1min":0.77000000000000002,"5min":1.532,"15min":1.532},"last":0.76200000000000001},{"interface":"front","average":{"1min":0.64200000000000002,"5min":0.72399999999999998,"15min":0.69999999999999996},"min":{"1min":0.51800000000000002,"5min":0.443,"15min":0.443},"max":{"1min":0.81100000000000005,"5min":2.6400000000000001,"15min":2.6400000000000001},"last":0.70099999999999996}]},{"osd":2,"last update":"Sun Nov 26 07:24:50 2023","interfaces":[{"interface":"back","average":{"1min":0.65000000000000002,"5min":0.71299999999999997,"15min":0.68799999999999994},"min":{"1min":0.42799999999999999,"5min":0.42099999999999999,"15min":0.42099999999999999},"max":{"1min":0.91500000000000004,"5min":1.665,"15min":1.665},"last":0.68300000000000005},{"interface":"front","average":{"1min":0.66300000000000003,"5min":0.745,"15min":0.69999999999999996},"min":{"1min":0.45700000000000002,"5min":0.45700000000000002,"15min":0.45700000000000002},"max":{"1min":0.85199999999999998,"5min":2.125,"15min":2.125},"last":0.54000000000000004}]},{"osd":3,"last update":"Sun Nov 26 07:24:50 2023","interfaces":[{"interface":"back","average":{"1min":0.68400000000000005,"5min":0.752,"15min":0.72899999999999998},"min":{"1min":0.52800000000000002,"5min":0.44700000000000001,"15min":0.43099999999999999},"max":{"1min":0.879,"5min":2.0529999999999999,"15min":2.0529999999999999},"last":0.78900000000000003},{"interface":"front","average":{"1min":0.67800000000000005,"5min":0.755,"15min":0.70299999999999996},"min":{"1min":0.55500000000000005,"5min":0.51900000000000002,"15min":0.44600000000000001},"max":{"1min":0.94099999999999995,"5min":1.629,"15min":1.629},"last":0.61199999999999999}]},{"osd":4,"last update":"Sun Nov 26 07:24:50 2023","interfaces":[{"interface":"back","average":{"1min":0.58499999999999996,"5min":0.61199999999999999,"15min":0.625},"min":{"1min":0.42199999999999999,"5min":0.35299999999999998,"15min":0.35299999999999998},"max":{"1min":0.84699999999999998,"5min":2.7429999999999999,"15min":2.7429999999999999},"last":0.77600000000000002},{"interface":"front","average":{"1min":0.60999999999999999,"5min":0.67300000000000004,"15min":0.69299999999999995},"min":{"1min":0.44400000000000001,"5min":0.44400000000000001,"15min":0.41299999999999998},"max":{"1min":0.85999999999999999,"5min":2.4630000000000001,"15min":2.4630000000000001},"last":0.84999999999999998}]},{"osd":6,"last update":"Sun Nov 26 07:24:50 2023","interfaces":[{"interface":"back","average":{"1min":0.63800000000000001,"5min":0.66600000000000004,"15min":0.67100000000000004},"min":{"1min":0.38300000000000001,"5min":0.372,"15min":0.372},"max":{"1min":1.5409999999999999,"5min":2.1619999999999999,"15min":2.1619999999999999},"last":0.71999999999999997},{"interface":"front","average":{"1min":0.63,"5min":0.69099999999999995,"15min":0.71499999999999997},"min":{"1min":0.47599999999999998,"5min":0.436,"15min":0.436},"max":{"1min":0.89500000000000002,"5min":2.7869999999999999,"15min":2.7869999999999999},"last":0.80200000000000005}]},{"osd":7,"last update":"Sun Nov 26 07:24:50 2023","interfaces":[{"interface":"back","average":{"1min":0.66000000000000003,"5min":0.73099999999999998,"15min":0.77500000000000002},"min":{"1min":0.46999999999999997,"5min":0.41399999999999998,"15min":0.41399999999999998},"max":{"1min":0.878,"5min":2.1960000000000002,"15min":2.1960000000000002},"last":0.83099999999999996},{"interface":"front","average":{"1min":0.67700000000000005,"5min":0.748,"15min":0.76400000000000001},"min":{"1min":0.51100000000000001,"5min":0.44700000000000001,"15min":0.44700000000000001},"max":{"1min":0.94899999999999995,"5min":1.9510000000000001,"15min":1.9510000000000001},"last":0.81599999999999995}]}]},{"osd":4,"up_from":13,"seq":55834574940,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":32476,"kb_used_data":1296,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339364,"statfs":{"total":96636764160,"available":96603508736,"internally_reserved":0,"allocated":1327104,"data_stored":579977,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Nov 26 07:24:55 2023","interfaces":[{"interface":"back","average":{"1min":0.65000000000000002,"5min":0.66600000000000004,"15min":0.73899999999999999},"min":{"1min":0.46600000000000003,"5min":0.443,"15min":0.32100000000000001},"max":{"1min":0.80000000000000004,"5min":0.96199999999999997,"15min":4.1689999999999996},"last":0.59399999999999997},{"interface":"front","average":{"1min":0.67500000000000004,"5min":0.85999999999999999,"15min":0.92800000000000005},"min":{"1min":0.49399999999999999,"5min":0.40799999999999997,"15min":0.40799999999999997},"max":{"1min":1.0209999999999999,"5min":18.021000000000001,"15min":18.021000000000001},"last":0.74299999999999999}]},{"osd":1,"last update":"Sun Nov 26 07:24:55 2023","interfaces":[{"interface":"back","average":{"1min":0.70599999999999996,"5min":0.75,"15min":0.68000000000000005},"min":{"1min":0.54200000000000004,"5min":0.43099999999999999,"15min":0.43099999999999999},"max":{"1min":0.95199999999999996,"5min":4.3159999999999998,"15min":4.3159999999999998},"last":0.68300000000000005},{"interface":"front","average":{"1min":0.63500000000000001,"5min":0.77100000000000002,"15min":0.78900000000000003},"min":{"1min":0.438,"5min":0.436,"15min":0.39800000000000002},"max":{"1min":0.89800000000000002,"5min":11.206,"15min":11.206},"last":0.53500000000000003}]},{"osd":2,"last update":"Sun Nov 26 07:24:55 2023","interfaces":[{"interface":"back","average":{"1min":0.72199999999999998,"5min":0.86499999999999999,"15min":0.81599999999999995},"min":{"1min":0.47399999999999998,"5min":0.46600000000000003,"15min":0.433},"max":{"1min":0.89900000000000002,"5min":15.441000000000001,"15min":15.441000000000001},"last":0.55600000000000005},{"interface":"front","average":{"1min":0.79200000000000004,"5min":0.871,"15min":0.97799999999999998},"min":{"1min":0.54300000000000004,"5min":0.54300000000000004,"15min":0.53400000000000003},"max":{"1min":1.071,"5min":9.4800000000000004,"15min":9.4800000000000004},"last":0.78300000000000003}]},{"osd":3,"last update":"Sun Nov 26 07:24:55 2023","interfaces":[{"interface":"back","average":{"1min":0.69499999999999995,"5min":0.83899999999999997,"15min":0.76000000000000001},"min":{"1min":0.51200000000000001,"5min":0.498,"15min":0.441},"max":{"1min":0.84699999999999998,"5min":12.715,"15min":12.715},"last":0.58399999999999996},{"interface":"front","average":{"1min":0.74399999999999999,"5min":0.82799999999999996,"15min":0.84799999999999998},"min":{"1min":0.55400000000000005,"5min":0.50700000000000001,"15min":0.50700000000000001},"max":{"1min":0.878,"5min":8.3239999999999998,"15min":8.3239999999999998},"last":0.65000000000000002}]},{"osd":5,"last update":"Sun Nov 26 07:24:55 2023","interfaces":[{"interface":"back","average":{"1min":0.68799999999999994,"5min":0.65100000000000002,"15min":0.622},"min":{"1min":0.41299999999999998,"5min":0.40500000000000003,"15min":0.38200000000000001},"max":{"1min":1.0229999999999999,"5min":1.0229999999999999,"15min":1.0229999999999999},"last":0.51700000000000002},{"interface":"front","average":{"1min":0.67400000000000004,"5min":0.74199999999999999,"15min":0.67000000000000004},"min":{"1min":0.41999999999999998,"5min":0.36399999999999999,"15min":0.35599999999999998},"max":{"1min":0.96099999999999997,"5min":11.148999999999999,"15min":11.148999999999999},"last":0.624}]},{"osd":6,"last update":"Sun Nov 26 07:24:55 2023","interfaces":[{"interface":"back","average":{"1min":0.68600000000000005,"5min":0.626,"15min":0.72399999999999998},"min":{"1min":0.40000000000000002,"5min":0.38400000000000001,"15min":0.372},"max":{"1min":0.91600000000000004,"5min":0.91600000000000004,"15min":2.8330000000000002},"last":0.72699999999999998},{"interface":"front","average":{"1min":0.71999999999999997,"5min":0.72399999999999998,"15min":0.89100000000000001},"min":{"1min":0.443,"5min":0.42699999999999999,"15min":0.41299999999999998},"max":{"1min":1.046,"5min":2.472,"15min":7.6050000000000004},"last":0.81100000000000005}]},{"osd":7,"last update":"Sun Nov 26 07:24:55 2023","interfaces":[{"interface":"back","average":{"1min":0.75900000000000001,"5min":0.83799999999999997,"15min":0.83899999999999997},"min":{"1min":0.59199999999999997,"5min":0.48699999999999999,"15min":0.44700000000000001},"max":{"1min":0.95999999999999996,"5min":12.653,"15min":12.653},"last":0.70499999999999996},{"interface":"front","average":{"1min":0.81599999999999995,"5min":0.79700000000000004,"15min":0.84799999999999998},"min":{"1min":0.56499999999999995,"5min":0.439,"15min":0.33300000000000002},"max":{"1min":0.97599999999999998,"5min":3.8319999999999999,"15min":3.8319999999999999},"last":0.76100000000000001}]}]},{"osd":3,"up_from":13,"seq":55834574941,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":32480,"kb_used_data":1296,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339360,"statfs":{"total":96636764160,"available":96603504640,"internally_reserved":0,"allocated":1327104,"data_stored":579977,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1592,"internal_metadata":31914440},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Nov 26 07:24:55 2023","interfaces":[{"interface":"back","average":{"1min":0.377,"5min":0.41099999999999998,"15min":0.39200000000000002},"min":{"1min":0.25900000000000001,"5min":0.218,"15min":0.218},"max":{"1min":0.58199999999999996,"5min":0.72799999999999998,"15min":0.72799999999999998},"last":0.41299999999999998},{"interface":"front","average":{"1min":0.41099999999999998,"5min":0.46000000000000002,"15min":0.443},"min":{"1min":0.27700000000000002,"5min":0.254,"15min":0.254},"max":{"1min":0.56599999999999995,"5min":0.77300000000000002,"15min":0.77300000000000002},"last":0.39400000000000002}]},{"osd":1,"last update":"Sun Nov 26 07:24:55 2023","interfaces":[{"interface":"back","average":{"1min":0.39200000000000002,"5min":0.435,"15min":0.42499999999999999},"min":{"1min":0.29899999999999999,"5min":0.28799999999999998,"15min":0.28799999999999998},"max":{"1min":0.53300000000000003,"5min":0.61299999999999999,"15min":0.752},"last":0.30299999999999999},{"interface":"front","average":{"1min":0.437,"5min":0.48599999999999999,"15min":0.48399999999999999},"min":{"1min":0.315,"5min":0.29899999999999999,"15min":0.29899999999999999},"max":{"1min":0.53300000000000003,"5min":0.872,"15min":0.872},"last":0.438}]},{"osd":2,"last update":"Sun Nov 26 07:24:55 2023","interfaces":[{"interface":"back","average":{"1min":0.504,"5min":0.55500000000000005,"15min":0.54200000000000004},"min":{"1min":0.36599999999999999,"5min":0.36599999999999999,"15min":0.36599999999999999},"max":{"1min":0.63200000000000001,"5min":0.97199999999999998,"15min":0.97199999999999998},"last":0.46899999999999997},{"interface":"front","average":{"1min":0.48999999999999999,"5min":0.52300000000000002,"15min":0.51400000000000001},"min":{"1min":0.36299999999999999,"5min":0.32100000000000001,"15min":0.32100000000000001},"max":{"1min":0.65100000000000002,"5min":0.89700000000000002,"15min":0.89700000000000002},"last":0.45300000000000001}]},{"osd":4,"last update":"Sun Nov 26 07:24:55 2023","interfaces":[{"interface":"back","average":{"1min":0.72399999999999998,"5min":0.71099999999999997,"15min":0.70799999999999996},"min":{"1min":0.502,"5min":0.39600000000000002,"15min":0.39600000000000002},"max":{"1min":0.94399999999999995,"5min":0.94399999999999995,"15min":1.091},"last":0.56999999999999995},{"interface":"front","average":{"1min":0.72899999999999998,"5min":0.72299999999999998,"15min":0.72499999999999998},"min":{"1min":0.48999999999999999,"5min":0.39900000000000002,"15min":0.39900000000000002},"max":{"1min":1.0309999999999999,"5min":1.242,"15min":1.242},"last":0.51500000000000001}]},{"osd":5,"last update":"Sun Nov 26 07:24:55 2023","interfaces":[{"interface":"back","average":{"1min":0.70599999999999996,"5min":0.70099999999999996,"15min":0.72399999999999998},"min":{"1min":0.505,"5min":0.45900000000000002,"15min":0.45900000000000002},"max":{"1min":0.86499999999999999,"5min":1.105,"15min":1.105},"last":0.68600000000000005},{"interface":"front","average":{"1min":0.70999999999999996,"5min":0.73299999999999998,"15min":0.73499999999999999},"min":{"1min":0.53100000000000003,"5min":0.47299999999999998,"15min":0.47299999999999998},"max":{"1min":0.94599999999999995,"5min":1.1319999999999999,"15min":1.1319999999999999},"last":0.53000000000000003}]},{"osd":6,"last update":"Sun Nov 26 07:24:55 2023","interfaces":[{"interface":"back","average":{"1min":0.79000000000000004,"5min":0.76800000000000002,"15min":0.79400000000000004},"min":{"1min":0.57099999999999995,"5min":0.5,"15min":0.5},"max":{"1min":1.0640000000000001,"5min":1.214,"15min":1.2729999999999999},"last":0.63900000000000001},{"interface":"front","average":{"1min":0.77200000000000002,"5min":0.75900000000000001,"15min":0.77800000000000002},"min":{"1min":0.53700000000000003,"5min":0.48799999999999999,"15min":0.47299999999999998},"max":{"1min":1.0509999999999999,"5min":1.1060000000000001,"15min":1.204},"last":0.54700000000000004}]},{"osd":7,"last update":"Sun Nov 26 07:24:55 2023","interfaces":[{"interface":"back","average":{"1min":0.73899999999999999,"5min":0.755,"15min":0.78100000000000003},"min":{"1min":0.55700000000000005,"5min":0.437,"15min":0.437},"max":{"1min":1.0069999999999999,"5min":1.0349999999999999,"15min":1.3600000000000001},"last":0.69699999999999995},{"interface":"front","average":{"1min":0.80300000000000005,"5min":0.79000000000000004,"15min":0.81200000000000006},"min":{"1min":0.52600000000000002,"5min":0.47299999999999998,"15min":0.47299999999999998},"max":{"1min":1.0529999999999999,"5min":1.2450000000000001,"15min":1.617},"last":0.70799999999999996}]}]},{"osd":2,"up_from":13,"seq":55834574941,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":32476,"kb_used_data":1296,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339364,"statfs":{"total":96636764160,"available":96603508736,"internally_reserved":0,"allocated":1327104,"data_stored":579977,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Nov 26 07:24:52 2023","interfaces":[{"interface":"back","average":{"1min":0.435,"5min":0.42599999999999999,"15min":0.60099999999999998},"min":{"1min":0.26600000000000001,"5min":0.26600000000000001,"15min":0.26600000000000001},"max":{"1min":0.65400000000000003,"5min":0.72899999999999998,"15min":4.7030000000000003},"last":0.32600000000000001},{"interface":"front","average":{"1min":0.41799999999999998,"5min":0.46000000000000002,"15min":0.52000000000000002},"min":{"1min":0.28399999999999997,"5min":0.27300000000000002,"15min":0.27300000000000002},"max":{"1min":0.66100000000000003,"5min":0.77200000000000002,"15min":2.4340000000000002},"last":0.499}]},{"osd":1,"last update":"Sun Nov 26 07:24:52 2023","interfaces":[{"interface":"back","average":{"1min":0.47199999999999998,"5min":0.48699999999999999,"15min":0.71299999999999997},"min":{"1min":0.33000000000000002,"5min":0.318,"15min":0.318},"max":{"1min":0.61499999999999999,"5min":0.86099999999999999,"15min":4.2690000000000001},"last":0.36599999999999999},{"interface":"front","average":{"1min":0.47299999999999998,"5min":0.502,"15min":0.64300000000000002},"min":{"1min":0.33200000000000002,"5min":0.33200000000000002,"15min":0.31900000000000001},"max":{"1min":0.68100000000000005,"5min":0.76600000000000001,"15min":3.9940000000000002},"last":0.46800000000000003}]},{"osd":3,"last update":"Sun Nov 26 07:24:52 2023","interfaces":[{"interface":"back","average":{"1min":0.46500000000000002,"5min":0.47699999999999998,"15min":0.57499999999999996},"min":{"1min":0.314,"5min":0.29699999999999999,"15min":0.29699999999999999},"max":{"1min":0.73299999999999998,"5min":0.82699999999999996,"15min":4.0300000000000002},"last":0.44400000000000001},{"interface":"front","average":{"1min":0.53000000000000003,"5min":0.54700000000000004,"15min":0.76700000000000002},"min":{"1min":0.374,"5min":0.36099999999999999,"15min":0.36099999999999999},"max":{"1min":0.83999999999999997,"5min":0.86699999999999999,"15min":4.5810000000000004},"last":0.40600000000000003}]},{"osd":4,"last update":"Sun Nov 26 07:24:52 2023","interfaces":[{"interface":"back","average":{"1min":0.72999999999999998,"5min":0.75800000000000001,"15min":0.89500000000000002},"min":{"1min":0.56100000000000005,"5min":0.51400000000000001,"15min":0.51400000000000001},"max":{"1min":1.0189999999999999,"5min":1.0189999999999999,"15min":4.4269999999999996},"last":0.748},{"interface":"front","average":{"1min":0.73899999999999999,"5min":0.77100000000000002,"15min":0.91400000000000003},"min":{"1min":0.53500000000000003,"5min":0.53500000000000003,"15min":0.53500000000000003},"max":{"1min":0.96199999999999997,"5min":1.0780000000000001,"15min":4.085},"last":0.76300000000000001}]},{"osd":5,"last update":"Sun Nov 26 07:24:52 2023","interfaces":[{"interface":"back","average":{"1min":0.76700000000000002,"5min":0.77400000000000002,"15min":0.83099999999999996},"min":{"1min":0.53800000000000003,"5min":0.42099999999999999,"15min":0.42099999999999999},"max":{"1min":1.2090000000000001,"5min":1.2090000000000001,"15min":3.9590000000000001},"last":0.68100000000000005},{"interface":"front","average":{"1min":0.74399999999999999,"5min":0.77400000000000002,"15min":0.90300000000000002},"min":{"1min":0.54500000000000004,"5min":0.499,"15min":0.499},"max":{"1min":1.024,"5min":1.0900000000000001,"15min":4.3849999999999998},"last":0.71399999999999997}]},{"osd":6,"last update":"Sun Nov 26 07:24:52 2023","interfaces":[{"interface":"back","average":{"1min":0.77500000000000002,"5min":0.77100000000000002,"15min":0.90500000000000003},"min":{"1min":0.58499999999999996,"5min":0.45200000000000001,"15min":0.45200000000000001},"max":{"1min":1.1160000000000001,"5min":1.1160000000000001,"15min":4.0449999999999999},"last":0.69799999999999995},{"interface":"front","average":{"1min":0.748,"5min":0.755,"15min":0.85299999999999998},"min":{"1min":0.56799999999999995,"5min":0.48099999999999998,"15min":0.48099999999999998},"max":{"1min":1.093,"5min":1.093,"15min":3.992},"last":0.77500000000000002}]},{"osd":7,"last update":"Sun Nov 26 07:24:52 2023","interfaces":[{"interface":"back","average":{"1min":0.79700000000000004,"5min":0.79300000000000004,"15min":0.93500000000000005},"min":{"1min":0.61399999999999999,"5min":0.56799999999999995,"15min":0.52200000000000002},"max":{"1min":1.0609999999999999,"5min":1.0840000000000001,"15min":4.1219999999999999},"last":0.66200000000000003},{"interface":"front","average":{"1min":0.77500000000000002,"5min":0.78100000000000003,"15min":0.875},"min":{"1min":0.61499999999999999,"5min":0.48199999999999998,"15min":0.48199999999999998},"max":{"1min":1.151,"5min":1.151,"15min":3.9159999999999999},"last":0.72999999999999998}]}]},{"osd":1,"up_from":13,"seq":55834574940,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":32476,"kb_used_data":1296,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339364,"statfs":{"total":96636764160,"available":96603508736,"internally_reserved":0,"allocated":1327104,"data_stored":579977,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":31914439},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Nov 26 07:24:51 2023","interfaces":[{"interface":"back","average":{"1min":0.38400000000000001,"5min":0.40500000000000003,"15min":0.41099999999999998},"min":{"1min":0.26600000000000001,"5min":0.26600000000000001,"15min":0.23599999999999999},"max":{"1min":0.58399999999999996,"5min":0.58399999999999996,"15min":0.61199999999999999},"last":0.32000000000000001},{"interface":"front","average":{"1min":0.41199999999999998,"5min":0.44400000000000001,"15min":0.46899999999999997},"min":{"1min":0.28000000000000003,"5min":0.28000000000000003,"15min":0.26700000000000002},"max":{"1min":0.52100000000000002,"5min":0.749,"15min":0.78700000000000003},"last":0.47799999999999998}]},{"osd":2,"last update":"Sun Nov 26 07:24:51 2023","interfaces":[{"interface":"back","average":{"1min":0.42099999999999999,"5min":0.436,"15min":0.44600000000000001},"min":{"1min":0.30499999999999999,"5min":0.29899999999999999,"15min":0.27800000000000002},"max":{"1min":0.63400000000000001,"5min":0.71399999999999997,"15min":0.71399999999999997},"last":0.45700000000000002},{"interface":"front","average":{"1min":0.46200000000000002,"5min":0.47599999999999998,"15min":0.495},"min":{"1min":0.33100000000000002,"5min":0.318,"15min":0.27700000000000002},"max":{"1min":0.61799999999999999,"5min":0.80600000000000005,"15min":0.80600000000000005},"last":0.36199999999999999}]},{"osd":3,"last update":"Sun Nov 26 07:24:51 2023","interfaces":[{"interface":"back","average":{"1min":0.48499999999999999,"5min":0.52100000000000002,"15min":0.52500000000000002},"min":{"1min":0.33200000000000002,"5min":0.33200000000000002,"15min":0.33200000000000002},"max":{"1min":0.72699999999999998,"5min":0.748,"15min":0.748},"last":0.39800000000000002},{"interface":"front","average":{"1min":0.47499999999999998,"5min":0.498,"15min":0.496},"min":{"1min":0.38100000000000001,"5min":0.35499999999999998,"15min":0.31},"max":{"1min":0.71299999999999997,"5min":0.71299999999999997,"15min":0.80700000000000005},"last":0.55300000000000005}]},{"osd":4,"last update":"Sun Nov 26 07:24:51 2023","interfaces":[{"interface":"back","average":{"1min":0.72999999999999998,"5min":0.75800000000000001,"15min":0.71499999999999997},"min":{"1min":0.50600000000000001,"5min":0.50600000000000001,"15min":0.41299999999999998},"max":{"1min":0.95699999999999996,"5min":1.319,"15min":1.319},"last":0.79800000000000004},{"interface":"front","average":{"1min":0.71899999999999997,"5min":0.72499999999999998,"15min":0.69799999999999995},"min":{"1min":0.54300000000000004,"5min":0.44500000000000001,"15min":0.44500000000000001},"max":{"1min":0.97999999999999998,"5min":1.006,"15min":1.131},"last":0.82399999999999995}]},{"osd":5,"last update":"Sun Nov 26 07:24:51 2023","interfaces":[{"interface":"back","average":{"1min":0.73299999999999998,"5min":0.747,"15min":0.72899999999999998},"min":{"1min":0.501,"5min":0.47899999999999998,"15min":0.47899999999999998},"max":{"1min":0.95699999999999996,"5min":1.0409999999999999,"15min":1.1919999999999999},"last":0.73099999999999998},{"interface":"front","average":{"1min":0.73499999999999999,"5min":0.75,"15min":0.72899999999999998},"min":{"1min":0.53100000000000003,"5min":0.47099999999999997,"15min":0.47099999999999997},"max":{"1min":0.86299999999999999,"5min":1.0389999999999999,"15min":1.0609999999999999},"last":0.71299999999999997}]},{"osd":6,"last update":"Sun Nov 26 07:24:51 2023","interfaces":[{"interface":"back","average":{"1min":0.72499999999999998,"5min":0.76000000000000001,"15min":0.72399999999999998},"min":{"1min":0.53800000000000003,"5min":0.53200000000000003,"15min":0.50600000000000001},"max":{"1min":0.89600000000000002,"5min":1.2190000000000001,"15min":1.2190000000000001},"last":0.88},{"interface":"front","average":{"1min":0.72199999999999998,"5min":0.76200000000000001,"15min":0.72499999999999998},"min":{"1min":0.52200000000000002,"5min":0.46999999999999997,"15min":0.46999999999999997},"max":{"1min":0.91800000000000004,"5min":1.1659999999999999,"15min":1.1659999999999999},"last":0.69399999999999995}]},{"osd":7,"last update":"Sun Nov 26 07:24:51 2023","interfaces":[{"interface":"back","average":{"1min":0.76700000000000002,"5min":0.77400000000000002,"15min":0.73599999999999999},"min":{"1min":0.57099999999999995,"5min":0.50600000000000001,"15min":0.50600000000000001},"max":{"1min":0.97299999999999998,"5min":1.0920000000000001,"15min":1.0920000000000001},"last":0.751},{"interface":"front","average":{"1min":0.76200000000000001,"5min":0.76500000000000001,"15min":0.755},"min":{"1min":0.54000000000000004,"5min":0.53700000000000003,"15min":0.48499999999999999},"max":{"1min":1.006,"5min":1.115,"15min":1.115},"last":0.84199999999999997}]}]},{"osd":0,"up_from":13,"seq":55834574940,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":33056,"kb_used_data":1876,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94338784,"statfs":{"total":96636764160,"available":96602914816,"internally_reserved":0,"allocated":1921024,"data_stored":1170345,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Sun Nov 26 07:24:50 2023","interfaces":[{"interface":"back","average":{"1min":0.439,"5min":0.437,"15min":0.43099999999999999},"min":{"1min":0.25800000000000001,"5min":0.25800000000000001,"15min":0.25800000000000001},"max":{"1min":0.70499999999999996,"5min":1.26,"15min":1.26},"last":0.57899999999999996},{"interface":"front","average":{"1min":0.45100000000000001,"5min":0.47299999999999998,"15min":0.47499999999999998},"min":{"1min":0.29899999999999999,"5min":0.29899999999999999,"15min":0.29899999999999999},"max":{"1min":0.72099999999999997,"5min":1.212,"15min":1.212},"last":0.35199999999999998}]},{"osd":2,"last update":"Sun Nov 26 07:24:50 2023","interfaces":[{"interface":"back","average":{"1min":0.47199999999999998,"5min":0.497,"15min":0.52200000000000002},"min":{"1min":0.33000000000000002,"5min":0.33000000000000002,"15min":0.33000000000000002},"max":{"1min":0.68100000000000005,"5min":0.84199999999999997,"15min":0.84199999999999997},"last":0.45500000000000002},{"interface":"front","average":{"1min":0.53900000000000003,"5min":0.55800000000000005,"15min":0.55900000000000005},"min":{"1min":0.45200000000000001,"5min":0.40100000000000002,"15min":0.36399999999999999},"max":{"1min":0.81000000000000005,"5min":1.0800000000000001,"15min":1.0800000000000001},"last":0.51900000000000002}]},{"osd":3,"last update":"Sun Nov 26 07:24:50 2023","interfaces":[{"interface":"back","average":{"1min":0.51600000000000001,"5min":0.51300000000000001,"15min":0.52400000000000002},"min":{"1min":0.41099999999999998,"5min":0.33500000000000002,"15min":0.33500000000000002},"max":{"1min":0.72899999999999998,"5min":0.873,"15min":1.3149999999999999},"last":0.495},{"interface":"front","average":{"1min":0.53000000000000003,"5min":0.53600000000000003,"15min":0.54900000000000004},"min":{"1min":0.377,"5min":0.35499999999999998,"15min":0.35499999999999998},"max":{"1min":0.74299999999999999,"5min":0.74299999999999999,"15min":1.2509999999999999},"last":0.438}]},{"osd":4,"last update":"Sun Nov 26 07:24:50 2023","interfaces":[{"interface":"back","average":{"1min":0.68500000000000005,"5min":0.73899999999999999,"15min":0.749},"min":{"1min":0.40100000000000002,"5min":0.40100000000000002,"15min":0.40100000000000002},"max":{"1min":0.91300000000000003,"5min":1.0900000000000001,"15min":1.2010000000000001},"last":0.73499999999999999},{"interface":"front","average":{"1min":0.76000000000000001,"5min":0.78000000000000003,"15min":0.78600000000000003},"min":{"1min":0.55900000000000005,"5min":0.51100000000000001,"15min":0.51100000000000001},"max":{"1min":1.012,"5min":1.298,"15min":1.381},"last":0.68700000000000006}]},{"osd":5,"last update":"Sun Nov 26 07:24:50 2023","interfaces":[{"interface":"back","average":{"1min":0.66800000000000004,"5min":0.73599999999999999,"15min":0.75},"min":{"1min":0.50800000000000001,"5min":0.45200000000000001,"15min":0.42199999999999999},"max":{"1min":0.91600000000000004,"5min":1.1819999999999999,"15min":1.1819999999999999},"last":0.56999999999999995},{"interface":"front","average":{"1min":0.77500000000000002,"5min":0.79800000000000004,"15min":0.79700000000000004},"min":{"1min":0.59799999999999998,"5min":0.53000000000000003,"15min":0.53000000000000003},"max":{"1min":1.0609999999999999,"5min":1.329,"15min":1.399},"last":0.65200000000000002}]},{"osd":6,"last update":"Sun Nov 26 07:24:50 2023","interfaces":[{"interface":"back","average":{"1min":0.70599999999999996,"5min":0.70599999999999996,"15min":0.73699999999999999},"min":{"1min":0.48999999999999999,"5min":0.48499999999999999,"15min":0.48499999999999999},"max":{"1min":0.97199999999999998,"5min":1.0740000000000001,"15min":1.3029999999999999},"last":0.63600000000000001},{"interface":"front","average":{"1min":0.71099999999999997,"5min":0.72899999999999998,"15min":0.754},"min":{"1min":0.52100000000000002,"5min":0.47199999999999998,"15min":0.47199999999999998},"max":{"1min":0.93600000000000005,"5min":1.2170000000000001,"15min":1.3600000000000001},"last":0.61799999999999999}]},{"osd":7,"last update":"Sun Nov 26 07:24:50 2023","interfaces":[{"interface":"back","average":{"1min":0.75600000000000001,"5min":0.76800000000000002,"15min":0.79300000000000004},"min":{"1min":0.60099999999999998,"5min":0.56899999999999995,"15min":0.56899999999999995},"max":{"1min":0.97999999999999998,"5min":1.1719999999999999,"15min":1.177},"last":0.71899999999999997},{"interface":"front","average":{"1min":0.69899999999999995,"5min":0.76500000000000001,"15min":0.77800000000000002},"min":{"1min":0.56899999999999995,"5min":0.47899999999999998,"15min":0.47899999999999998},"max":{"1min":0.92400000000000004,"5min":1.238,"15min":1.2390000000000001},"last":0.76600000000000001}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-11-26T07:25:03.571 INFO:tasks.ceph:pgid 1.0 last_scrub_stamp 2023-11-26T07:17:42.160945+0000 time.struct_time(tm_year=2023, tm_mon=11, tm_mday=26, tm_hour=7, tm_min=17, tm_sec=42, tm_wday=6, tm_yday=330, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=11, tm_mday=26, tm_hour=7, tm_min=24, tm_sec=58, tm_wday=6, tm_yday=330, tm_isdst=0) 2023-11-26T07:25:03.572 INFO:tasks.ceph:Still waiting for all pgs to be scrubbed. 2023-11-26T07:25:22.897 DEBUG:teuthology.orchestra.run.smithi143:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-26T07:25:22.902 DEBUG:teuthology.orchestra.run.smithi188:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-26T07:25:23.573 DEBUG:teuthology.orchestra.run.smithi143:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-11-26T07:25:23.722 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:25:23.722+0000 7f8f19832640 1 Processor -- start 2023-11-26T07:25:23.723 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:25:23.722+0000 7f8f19832640 1 -- start start 2023-11-26T07:25:23.724 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:25:23.722+0000 7f8f19832640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f8f1406f3e0 0x7f8f14161690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:25:23.724 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:25:23.722+0000 7f8f19832640 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7f8f14099a70 0x7f8f14161bd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:25:23.724 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:25:23.722+0000 7f8f19832640 1 -- --> v1:172.21.15.143:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8f140cf260 con 0x7f8f1406f000 2023-11-26T07:25:23.724 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:25:23.722+0000 7f8f19832640 1 -- --> v2:172.21.15.143:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8f1409a010 con 0x7f8f14099a70 2023-11-26T07:25:23.724 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:25:23.722+0000 7f8f19832640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8f1409a2f0 con 0x7f8f1406f3e0 2023-11-26T07:25:23.725 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:25:23.722+0000 7f8f137fe640 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7f8f14099a70 0x7f8f14161bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:25:23.725 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:25:23.722+0000 7f8f12ffd640 1 --1- >> v1:172.21.15.143:6789/0 conn(0x7f8f1406f000 0x7f8f1406d030 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.143:6789/0 says I am v1:172.21.15.143:54496/0 (socket says 172.21.15.143:54496) 2023-11-26T07:25:23.725 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:25:23.722+0000 7f8f12ffd640 1 -- 172.21.15.143:0/1439043514 learned_addr learned my addr 172.21.15.143:0/1439043514 (peer_addr_for_me v1:172.21.15.143:0/0) 2023-11-26T07:25:23.725 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:25:23.722+0000 7f8f0bfff640 1 --2- 172.21.15.143:0/1439043514 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f8f1406f3e0 0x7f8f14161690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:25:23.726 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:25:23.726+0000 7f8f137fe640 1 -- 172.21.15.143:0/1439043514 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f8f1406f3e0 msgr2=0x7f8f14161690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-26T07:25:23.726 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:25:23.726+0000 7f8f137fe640 1 --2- 172.21.15.143:0/1439043514 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f8f1406f3e0 0x7f8f14161690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:25:23.726 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:25:23.726+0000 7f8f137fe640 1 -- 172.21.15.143:0/1439043514 >> v1:172.21.15.143:6789/0 conn(0x7f8f1406f000 legacy=0x7f8f1406d030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:25:23.726 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:25:23.726+0000 7f8f137fe640 1 -- 172.21.15.143:0/1439043514 --> v2:172.21.15.143:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8f1409a5d0 con 0x7f8f14099a70 2023-11-26T07:25:23.726 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:25:23.726+0000 7f8f127fc640 1 -- 172.21.15.143:0/1439043514 <== mon.0 v1:172.21.15.143:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3377490826 0 0) 0x7f8f140cf260 con 0x7f8f1406f000 2023-11-26T07:25:23.727 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:25:23.726+0000 7f8f137fe640 1 --2- 172.21.15.143:0/1439043514 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f8f14099a70 0x7f8f14161bd0 secure :-1 s=READY pgs=1496 cs=0 l=1 rev1=1 crypto rx=0x7f8f0c009d90 tx=0x7f8f0c01c8c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1e43d31a74a1f5db server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:25:23.727 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:25:23.726+0000 7f8f127fc640 1 -- 172.21.15.143:0/1439043514 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8f0c0a5780 con 0x7f8f14099a70 2023-11-26T07:25:23.728 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:25:23.726+0000 7f8f127fc640 1 -- 172.21.15.143:0/1439043514 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8f0c0a58e0 con 0x7f8f14099a70 2023-11-26T07:25:23.728 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:25:23.726+0000 7f8f127fc640 1 -- 172.21.15.143:0/1439043514 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8f0c0a5c70 con 0x7f8f14099a70 2023-11-26T07:25:23.728 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:25:23.726+0000 7f8f19832640 1 -- 172.21.15.143:0/1439043514 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f8f14099a70 msgr2=0x7f8f14161bd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:25:23.729 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:25:23.726+0000 7f8f19832640 1 --2- 172.21.15.143:0/1439043514 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f8f14099a70 0x7f8f14161bd0 secure :-1 s=READY pgs=1496 cs=0 l=1 rev1=1 crypto rx=0x7f8f0c009d90 tx=0x7f8f0c01c8c0 comp rx=0 tx=0).stop 2023-11-26T07:25:23.729 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:25:23.726+0000 7f8f19832640 1 -- 172.21.15.143:0/1439043514 shutdown_connections 2023-11-26T07:25:23.729 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:25:23.726+0000 7f8f19832640 1 --2- 172.21.15.143:0/1439043514 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f8f14099a70 0x7f8f14161bd0 unknown :-1 s=CLOSED pgs=1496 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:25:23.729 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:25:23.726+0000 7f8f19832640 1 --2- 172.21.15.143:0/1439043514 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f8f1406f3e0 0x7f8f14161690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:25:23.729 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:25:23.726+0000 7f8f19832640 1 -- 172.21.15.143:0/1439043514 >> 172.21.15.143:0/1439043514 conn(0x7f8f140b2d10 msgr2=0x7f8f140b3120 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:25:23.729 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:25:23.730+0000 7f8f19832640 1 -- 172.21.15.143:0/1439043514 shutdown_connections 2023-11-26T07:25:23.729 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:25:23.730+0000 7f8f19832640 1 -- 172.21.15.143:0/1439043514 wait complete. 2023-11-26T07:25:23.730 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:25:23.730+0000 7f8f19832640 1 Processor -- start 2023-11-26T07:25:23.730 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:25:23.730+0000 7f8f19832640 1 -- start start 2023-11-26T07:25:23.731 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:25:23.730+0000 7f8f19832640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f8f1406f000 0x7f8f14160360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:25:23.731 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:25:23.730+0000 7f8f19832640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f8f1406f3e0 0x7f8f140c82f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:25:23.731 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:25:23.730+0000 7f8f19832640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f8f14099a70 0x7f8f140c8830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:25:23.732 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:25:23.730+0000 7f8f19832640 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8f14161950 con 0x7f8f1406f000 2023-11-26T07:25:23.732 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:25:23.730+0000 7f8f19832640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8f14160d40 con 0x7f8f1406f3e0 2023-11-26T07:25:23.732 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:25:23.730+0000 7f8f19832640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8f14161020 con 0x7f8f14099a70 2023-11-26T07:25:23.732 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:25:23.730+0000 7f8f12ffd640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f8f1406f000 0x7f8f14160360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:25:23.732 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:25:23.730+0000 7f8f12ffd640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f8f1406f000 0x7f8f14160360 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:51152/0 (socket says 172.21.15.143:51152) 2023-11-26T07:25:23.732 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:25:23.730+0000 7f8f12ffd640 1 -- 172.21.15.143:0/3596552199 learned_addr learned my addr 172.21.15.143:0/3596552199 (peer_addr_for_me v2:172.21.15.143:0/0) 2023-11-26T07:25:23.734 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:25:23.730+0000 7f8f0bfff640 1 --2- 172.21.15.143:0/3596552199 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f8f1406f3e0 0x7f8f140c82f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:25:23.734 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:25:23.730+0000 7f8f137fe640 1 --2- 172.21.15.143:0/3596552199 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f8f14099a70 0x7f8f140c8830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:25:23.734 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:25:23.730+0000 7f8f12ffd640 1 -- 172.21.15.143:0/3596552199 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f8f14099a70 msgr2=0x7f8f140c8830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:25:23.734 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:25:23.730+0000 7f8f12ffd640 1 --2- 172.21.15.143:0/3596552199 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f8f14099a70 0x7f8f140c8830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:25:23.734 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:25:23.730+0000 7f8f12ffd640 1 -- 172.21.15.143:0/3596552199 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f8f1406f3e0 msgr2=0x7f8f140c82f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:25:23.734 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:25:23.730+0000 7f8f12ffd640 1 --2- 172.21.15.143:0/3596552199 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f8f1406f3e0 0x7f8f140c82f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:25:23.734 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:25:23.730+0000 7f8f12ffd640 1 -- 172.21.15.143:0/3596552199 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8f0c009a70 con 0x7f8f1406f000 2023-11-26T07:25:23.735 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:25:23.730+0000 7f8f0bfff640 1 --2- 172.21.15.143:0/3596552199 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f8f1406f3e0 0x7f8f140c82f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-26T07:25:23.735 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:25:23.730+0000 7f8f137fe640 1 --2- 172.21.15.143:0/3596552199 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f8f14099a70 0x7f8f140c8830 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-26T07:25:23.735 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:25:23.730+0000 7f8f12ffd640 1 --2- 172.21.15.143:0/3596552199 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f8f1406f000 0x7f8f14160360 secure :-1 s=READY pgs=1497 cs=0 l=1 rev1=1 crypto rx=0x7f8f040011f0 tx=0x7f8f0400c9b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:25:23.735 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:25:23.730+0000 7f8f10ff9640 1 -- 172.21.15.143:0/3596552199 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8f040074e0 con 0x7f8f1406f000 2023-11-26T07:25:23.735 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:25:23.734+0000 7f8f10ff9640 1 -- 172.21.15.143:0/3596552199 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8f04007640 con 0x7f8f1406f000 2023-11-26T07:25:23.735 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:25:23.734+0000 7f8f10ff9640 1 -- 172.21.15.143:0/3596552199 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8f04004a10 con 0x7f8f1406f000 2023-11-26T07:25:23.735 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:25:23.734+0000 7f8f19832640 1 -- 172.21.15.143:0/3596552199 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8f14161300 con 0x7f8f1406f000 2023-11-26T07:25:23.735 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:25:23.734+0000 7f8f19832640 1 -- 172.21.15.143:0/3596552199 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8f1414b7a0 con 0x7f8f1406f000 2023-11-26T07:25:23.736 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:25:23.734+0000 7f8f19832640 1 -- 172.21.15.143:0/3596552199 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8f14067680 con 0x7f8f1406f000 2023-11-26T07:25:23.736 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:25:23.734+0000 7f8f10ff9640 1 -- 172.21.15.143:0/3596552199 <== mon.0 v2:172.21.15.143:3300/0 4 ==== mgrmap(e 18) v1 ==== 86107+0+0 (secure 0 0 0) 0x7f8f04004b70 con 0x7f8f1406f000 2023-11-26T07:25:23.737 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:25:23.734+0000 7f8f10ff9640 1 --2- 172.21.15.143:0/3596552199 >> [v2:172.21.15.188:6836/3401128553,v1:172.21.15.188:6837/3401128553] conn(0x7f8ef4066740 0x7f8ef4068c00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:25:23.737 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:25:23.734+0000 7f8f10ff9640 1 -- 172.21.15.143:0/3596552199 <== mon.0 v2:172.21.15.143:3300/0 5 ==== osd_map(106..106 src has 1..106) v4 ==== 4943+0+0 (secure 0 0 0) 0x7f8f04083400 con 0x7f8f1406f000 2023-11-26T07:25:23.737 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:25:23.734+0000 7f8f0bfff640 1 --2- 172.21.15.143:0/3596552199 >> [v2:172.21.15.188:6836/3401128553,v1:172.21.15.188:6837/3401128553] conn(0x7f8ef4066740 0x7f8ef4068c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:25:23.737 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:25:23.734+0000 7f8f0bfff640 1 --2- 172.21.15.143:0/3596552199 >> [v2:172.21.15.188:6836/3401128553,v1:172.21.15.188:6837/3401128553] conn(0x7f8ef4066740 0x7f8ef4068c00 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7f8f1414bd10 tx=0x7f8efc008040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:25:23.739 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:25:23.738+0000 7f8f10ff9640 1 -- 172.21.15.143:0/3596552199 <== mon.0 v2:172.21.15.143:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172649 (secure 0 0 0) 0x7f8f040506c0 con 0x7f8f1406f000 2023-11-26T07:25:23.884 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:25:23.882+0000 7f8f19832640 1 -- 172.21.15.143:0/3596552199 --> [v2:172.21.15.188:6836/3401128553,v1:172.21.15.188:6837/3401128553] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f8f14161a90 con 0x7f8ef4066740 2023-11-26T07:25:23.887 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:25:23.886+0000 7f8f10ff9640 1 -- 172.21.15.143:0/3596552199 <== mgr.4108 v2:172.21.15.188:6836/3401128553 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+50698 (secure 0 0 0) 0x7f8efc010900 con 0x7f8ef4066740 2023-11-26T07:25:23.887 INFO:teuthology.orchestra.run.smithi143.stdout: 2023-11-26T07:25:23.888 INFO:teuthology.orchestra.run.smithi143.stderr:dumped all 2023-11-26T07:25:23.889 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:25:23.890+0000 7f8f19832640 1 -- 172.21.15.143:0/3596552199 >> [v2:172.21.15.188:6836/3401128553,v1:172.21.15.188:6837/3401128553] conn(0x7f8ef4066740 msgr2=0x7f8ef4068c00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:25:23.889 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:25:23.890+0000 7f8f19832640 1 --2- 172.21.15.143:0/3596552199 >> [v2:172.21.15.188:6836/3401128553,v1:172.21.15.188:6837/3401128553] conn(0x7f8ef4066740 0x7f8ef4068c00 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7f8f1414bd10 tx=0x7f8efc008040 comp rx=0 tx=0).stop 2023-11-26T07:25:23.889 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:25:23.890+0000 7f8f19832640 1 -- 172.21.15.143:0/3596552199 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f8f1406f000 msgr2=0x7f8f14160360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:25:23.889 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:25:23.890+0000 7f8f19832640 1 --2- 172.21.15.143:0/3596552199 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f8f1406f000 0x7f8f14160360 secure :-1 s=READY pgs=1497 cs=0 l=1 rev1=1 crypto rx=0x7f8f040011f0 tx=0x7f8f0400c9b0 comp rx=0 tx=0).stop 2023-11-26T07:25:23.890 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:25:23.890+0000 7f8f19832640 1 -- 172.21.15.143:0/3596552199 shutdown_connections 2023-11-26T07:25:23.890 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:25:23.890+0000 7f8f19832640 1 --2- 172.21.15.143:0/3596552199 >> [v2:172.21.15.188:6836/3401128553,v1:172.21.15.188:6837/3401128553] conn(0x7f8ef4066740 0x7f8ef4068c00 unknown :-1 s=CLOSED pgs=204 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:25:23.890 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:25:23.890+0000 7f8f19832640 1 --2- 172.21.15.143:0/3596552199 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f8f14099a70 0x7f8f140c8830 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:25:23.890 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:25:23.890+0000 7f8f19832640 1 --2- 172.21.15.143:0/3596552199 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f8f1406f3e0 0x7f8f140c82f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:25:23.890 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:25:23.890+0000 7f8f19832640 1 --2- 172.21.15.143:0/3596552199 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f8f1406f000 0x7f8f14160360 unknown :-1 s=CLOSED pgs=1497 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:25:23.890 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:25:23.890+0000 7f8f19832640 1 -- 172.21.15.143:0/3596552199 >> 172.21.15.143:0/3596552199 conn(0x7f8f140b2d10 msgr2=0x7f8f140ccd10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:25:23.890 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:25:23.890+0000 7f8f19832640 1 -- 172.21.15.143:0/3596552199 shutdown_connections 2023-11-26T07:25:23.890 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:25:23.890+0000 7f8f19832640 1 -- 172.21.15.143:0/3596552199 wait complete. 2023-11-26T07:25:23.909 INFO:teuthology.orchestra.run.smithi143.stdout:{"pg_ready":true,"pg_map":{"version":104,"stamp":"2023-11-26T07:25:23.315624+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":212,"num_read_kb":306,"num_write":179,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":130,"ondisk_log_size":130,"up":2,"acting":2,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":2,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":2,"kb":754974720,"kb_used":260964,"kb_used_data":11528,"kb_used_omap":12,"kb_used_meta":249331,"kb_avail":754713756,"statfs":{"total":773094113280,"available":772826886144,"internally_reserved":0,"allocated":11804672,"data_stored":5820552,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12749,"internal_metadata":255315507},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.002583"},"pg_stats":[{"pgid":"1.0","version":"106'130","reported_seq":286,"reported_epoch":106,"state":"active+clean","last_fresh":"2023-11-26T07:25:04.266476+0000","last_change":"2023-11-26T07:25:04.266508+0000","last_active":"2023-11-26T07:25:04.266476+0000","last_peered":"2023-11-26T07:25:04.266476+0000","last_clean":"2023-11-26T07:25:04.266476+0000","last_became_active":"2023-11-26T07:17:43.827308+0000","last_became_peered":"2023-11-26T07:17:43.827308+0000","last_unstale":"2023-11-26T07:25:04.266476+0000","last_undegraded":"2023-11-26T07:25:04.266476+0000","last_fullsized":"2023-11-26T07:25:04.266476+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"106'130","last_scrub_stamp":"2023-11-26T07:25:04.266372+0000","last_deep_scrub":"106'130","last_deep_scrub_stamp":"2023-11-26T07:25:04.266372+0000","last_clean_scrub_stamp":"2023-11-26T07:25:04.266372+0000","objects_scrubbed":2,"log_size":130,"log_dups_size":0,"ondisk_log_size":130,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T11:49:24.850799+0000","scrub_duration":0.0076308000000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":212,"num_read_kb":306,"num_write":179,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":212,"num_read_kb":306,"num_write":179,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":130,"ondisk_log_size":130,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574942,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":33052,"kb_used_data":1876,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94338788,"statfs":{"total":96636764160,"available":96602918912,"internally_reserved":0,"allocated":1921024,"data_stored":1170345,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":31914442},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Nov 26 07:25:01 2023","interfaces":[{"interface":"back","average":{"1min":0.66700000000000004,"5min":0.70399999999999996,"15min":0.70799999999999996},"min":{"1min":0.53700000000000003,"5min":0.42999999999999999,"15min":0.42999999999999999},"max":{"1min":0.79300000000000004,"5min":1.036,"15min":1.036},"last":0.80900000000000005},{"interface":"front","average":{"1min":0.68400000000000005,"5min":0.71999999999999997,"15min":0.71299999999999997},"min":{"1min":0.47499999999999998,"5min":0.42299999999999999,"15min":0.42299999999999999},"max":{"1min":0.85799999999999998,"5min":1.014,"15min":1.0980000000000001},"last":0.82599999999999996}]},{"osd":1,"last update":"Sun Nov 26 07:25:01 2023","interfaces":[{"interface":"back","average":{"1min":0.67400000000000004,"5min":0.68899999999999995,"15min":0.68300000000000005},"min":{"1min":0.442,"5min":0.435,"15min":0.435},"max":{"1min":0.82299999999999995,"5min":1.012,"15min":1.0669999999999999},"last":0.90800000000000003},{"interface":"front","average":{"1min":0.61799999999999999,"5min":0.67300000000000004,"15min":0.77000000000000002},"min":{"1min":0.44600000000000001,"5min":0.432,"15min":0.432},"max":{"1min":0.81599999999999995,"5min":0.94699999999999995,"15min":4.5309999999999997},"last":0.74399999999999999}]},{"osd":2,"last update":"Sun Nov 26 07:25:01 2023","interfaces":[{"interface":"back","average":{"1min":0.61599999999999999,"5min":0.68899999999999995,"15min":0.68000000000000005},"min":{"1min":0.51500000000000001,"5min":0.39700000000000002,"15min":0.39700000000000002},"max":{"1min":0.73999999999999999,"5min":1.0800000000000001,"15min":1.0800000000000001},"last":0.78200000000000003},{"interface":"front","average":{"1min":0.67200000000000004,"5min":0.70499999999999996,"15min":0.70699999999999996},"min":{"1min":0.47299999999999998,"5min":0.40999999999999998,"15min":0.40999999999999998},"max":{"1min":0.89300000000000002,"5min":0.97699999999999998,"15min":1.1339999999999999},"last":0.76200000000000001}]},{"osd":3,"last update":"Sun Nov 26 07:25:01 2023","interfaces":[{"interface":"back","average":{"1min":0.65400000000000003,"5min":0.71499999999999997,"15min":0.70399999999999996},"min":{"1min":0.48999999999999999,"5min":0.48999999999999999,"15min":0.47799999999999998},"max":{"1min":0.81000000000000005,"5min":1.1240000000000001,"15min":1.1240000000000001},"last":0.85199999999999998},{"interface":"front","average":{"1min":0.70199999999999996,"5min":0.73899999999999999,"15min":0.73399999999999999},"min":{"1min":0.58699999999999997,"5min":0.433,"15min":0.433},"max":{"1min":0.91900000000000004,"5min":1.036,"15min":1.4490000000000001},"last":0.86899999999999999}]},{"osd":4,"last update":"Sun Nov 26 07:25:01 2023","interfaces":[{"interface":"back","average":{"1min":0.58999999999999997,"5min":0.60099999999999998,"15min":0.60399999999999998},"min":{"1min":0.36299999999999999,"5min":0.36299999999999999,"15min":0.36299999999999999},"max":{"1min":0.84699999999999998,"5min":0.96299999999999997,"15min":0.96299999999999997},"last":0.60899999999999999},{"interface":"front","average":{"1min":0.60299999999999998,"5min":0.63500000000000001,"15min":0.65800000000000003},"min":{"1min":0.42899999999999999,"5min":0.35999999999999999,"15min":0.35999999999999999},"max":{"1min":0.75900000000000001,"5min":1.0209999999999999,"15min":1.5509999999999999},"last":0.48099999999999998}]},{"osd":5,"last update":"Sun Nov 26 07:25:01 2023","interfaces":[{"interface":"back","average":{"1min":0.65000000000000002,"5min":0.65800000000000003,"15min":0.66700000000000004},"min":{"1min":0.495,"5min":0.36599999999999999,"15min":0.36599999999999999},"max":{"1min":0.82299999999999995,"5min":1.0029999999999999,"15min":1.415},"last":0.501},{"interface":"front","average":{"1min":0.63600000000000001,"5min":0.67100000000000004,"15min":0.69799999999999995},"min":{"1min":0.48399999999999999,"5min":0.435,"15min":0.435},"max":{"1min":0.89900000000000002,"5min":1.0840000000000001,"15min":1.3859999999999999},"last":0.55200000000000005}]},{"osd":6,"last update":"Sun Nov 26 07:25:01 2023","interfaces":[{"interface":"back","average":{"1min":0.72699999999999998,"5min":0.76800000000000002,"15min":0.77100000000000002},"min":{"1min":0.55400000000000005,"5min":0.52400000000000002,"15min":0.51700000000000002},"max":{"1min":0.93899999999999995,"5min":1.123,"15min":1.3069999999999999},"last":0.65500000000000003},{"interface":"front","average":{"1min":0.66100000000000003,"5min":0.72299999999999998,"15min":0.754},"min":{"1min":0.51600000000000001,"5min":0.46899999999999997,"15min":0.46899999999999997},"max":{"1min":0.86899999999999999,"5min":1.032,"15min":1.6759999999999999},"last":0.53600000000000003}]}]},{"osd":6,"up_from":13,"seq":55834574944,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":32476,"kb_used_data":1296,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339364,"statfs":{"total":96636764160,"available":96603508736,"internally_reserved":0,"allocated":1327104,"data_stored":579977,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Nov 26 07:24:58 2023","interfaces":[{"interface":"back","average":{"1min":0.67800000000000005,"5min":0.67600000000000005,"15min":0.68100000000000005},"min":{"1min":0.56299999999999994,"5min":0.32200000000000001,"15min":0.32200000000000001},"max":{"1min":0.88700000000000001,"5min":0.91800000000000004,"15min":1.7549999999999999},"last":0.81799999999999995},{"interface":"front","average":{"1min":0.69799999999999995,"5min":0.69799999999999995,"15min":0.85299999999999998},"min":{"1min":0.44900000000000001,"5min":0.36299999999999999,"15min":0.36299999999999999},"max":{"1min":0.89200000000000002,"5min":0.90500000000000003,"15min":5.7489999999999997},"last":1.0469999999999999}]},{"osd":1,"last update":"Sun Nov 26 07:24:58 2023","interfaces":[{"interface":"back","average":{"1min":0.68000000000000005,"5min":0.70499999999999996,"15min":0.82899999999999996},"min":{"1min":0.41599999999999998,"5min":0.41599999999999998,"15min":0.41199999999999998},"max":{"1min":0.94699999999999995,"5min":0.97799999999999998,"15min":5.4240000000000004},"last":0.91300000000000003},{"interface":"front","average":{"1min":0.71299999999999997,"5min":0.70999999999999996,"15min":0.82199999999999995},"min":{"1min":0.497,"5min":0.47599999999999998,"15min":0.47599999999999998},"max":{"1min":0.88900000000000001,"5min":0.96499999999999997,"15min":5.6020000000000003},"last":0.89400000000000002}]},{"osd":2,"last update":"Sun Nov 26 07:24:58 2023","interfaces":[{"interface":"back","average":{"1min":0.67700000000000005,"5min":0.67300000000000004,"15min":0.68500000000000005},"min":{"1min":0.46200000000000002,"5min":0.41699999999999998,"15min":0.41699999999999998},"max":{"1min":1.002,"5min":1.002,"15min":1.8520000000000001},"last":0.875},{"interface":"front","average":{"1min":0.70399999999999996,"5min":0.72099999999999997,"15min":0.72799999999999998},"min":{"1min":0.44900000000000001,"5min":0.44900000000000001,"15min":0.437},"max":{"1min":0.86399999999999999,"5min":0.95199999999999996,"15min":1.9159999999999999},"last":1.008}]},{"osd":3,"last update":"Sun Nov 26 07:24:58 2023","interfaces":[{"interface":"back","average":{"1min":0.69999999999999996,"5min":0.70899999999999996,"15min":0.82399999999999995},"min":{"1min":0.42499999999999999,"5min":0.42499999999999999,"15min":0.42499999999999999},"max":{"1min":0.93500000000000005,"5min":0.93500000000000005,"15min":5.4390000000000001},"last":0.98899999999999999},{"interface":"front","average":{"1min":0.66100000000000003,"5min":0.69899999999999995,"15min":0.84499999999999997},"min":{"1min":0.44600000000000001,"5min":0.44,"15min":0.44},"max":{"1min":0.96499999999999997,"5min":0.96499999999999997,"15min":5.5490000000000004},"last":1.026}]},{"osd":4,"last update":"Sun Nov 26 07:24:58 2023","interfaces":[{"interface":"back","average":{"1min":0.56299999999999994,"5min":0.58599999999999997,"15min":0.626},"min":{"1min":0.39500000000000002,"5min":0.39500000000000002,"15min":0.371},"max":{"1min":0.75900000000000001,"5min":0.95599999999999996,"15min":1.0129999999999999},"last":0.64500000000000002},{"interface":"front","average":{"1min":0.59199999999999997,"5min":0.61099999999999999,"15min":0.65600000000000003},"min":{"1min":0.44,"5min":0.38600000000000001,"15min":0.38600000000000001},"max":{"1min":0.751,"5min":0.86399999999999999,"15min":1.0349999999999999},"last":0.83099999999999996}]},{"osd":5,"last update":"Sun Nov 26 07:24:58 2023","interfaces":[{"interface":"back","average":{"1min":0.60699999999999998,"5min":0.60999999999999999,"15min":0.64200000000000002},"min":{"1min":0.46800000000000003,"5min":0.40699999999999997,"15min":0.38200000000000001},"max":{"1min":0.93500000000000005,"5min":0.93500000000000005,"15min":0.93500000000000005},"last":0.70099999999999996},{"interface":"front","average":{"1min":0.64800000000000002,"5min":0.66400000000000003,"15min":0.69999999999999996},"min":{"1min":0.46600000000000003,"5min":0.44,"15min":0.436},"max":{"1min":0.84299999999999997,"5min":1.014,"15min":1.0960000000000001},"last":0.72799999999999998}]},{"osd":7,"last update":"Sun Nov 26 07:24:58 2023","interfaces":[{"interface":"back","average":{"1min":0.67400000000000004,"5min":0.67800000000000005,"15min":0.72299999999999998},"min":{"1min":0.52100000000000002,"5min":0.42599999999999999,"15min":0.42599999999999999},"max":{"1min":0.82299999999999995,"5min":0.95099999999999996,"15min":0.998},"last":0.755},{"interface":"front","average":{"1min":0.65900000000000003,"5min":0.66600000000000004,"15min":0.70099999999999996},"min":{"1min":0.46300000000000002,"5min":0.46300000000000002,"15min":0.46300000000000002},"max":{"1min":0.91200000000000003,"5min":0.91200000000000003,"15min":0.98799999999999999},"last":0.78800000000000003}]}]},{"osd":5,"up_from":13,"seq":55834574942,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":32476,"kb_used_data":1296,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339364,"statfs":{"total":96636764160,"available":96603508736,"internally_reserved":0,"allocated":1327104,"data_stored":579977,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Nov 26 07:24:50 2023","interfaces":[{"interface":"back","average":{"1min":0.61899999999999999,"5min":0.70499999999999996,"15min":0.68100000000000005},"min":{"1min":0.29699999999999999,"5min":0.29699999999999999,"15min":0.29699999999999999},"max":{"1min":0.80800000000000005,"5min":2.6949999999999998,"15min":2.6949999999999998},"last":0.77500000000000002},{"interface":"front","average":{"1min":0.64800000000000002,"5min":0.72399999999999998,"15min":0.68200000000000005},"min":{"1min":0.40200000000000002,"5min":0.38200000000000001,"15min":0.38200000000000001},"max":{"1min":0.88600000000000001,"5min":2.0150000000000001,"15min":2.0150000000000001},"last":0.63300000000000001}]},{"osd":1,"last update":"Sun Nov 26 07:24:50 2023","interfaces":[{"interface":"back","average":{"1min":0.60999999999999999,"5min":0.69099999999999995,"15min":0.67700000000000005},"min":{"1min":0.45900000000000002,"5min":0.41799999999999998,"15min":0.41599999999999998},"max":{"1min":0.77000000000000002,"5min":1.532,"15min":1.532},"last":0.93400000000000005},{"interface":"front","average":{"1min":0.64200000000000002,"5min":0.72399999999999998,"15min":0.69999999999999996},"min":{"1min":0.51800000000000002,"5min":0.443,"15min":0.443},"max":{"1min":0.81100000000000005,"5min":2.6400000000000001,"15min":2.6400000000000001},"last":0.86099999999999999}]},{"osd":2,"last update":"Sun Nov 26 07:24:50 2023","interfaces":[{"interface":"back","average":{"1min":0.65000000000000002,"5min":0.71299999999999997,"15min":0.68799999999999994},"min":{"1min":0.42799999999999999,"5min":0.42099999999999999,"15min":0.42099999999999999},"max":{"1min":0.91500000000000004,"5min":1.665,"15min":1.665},"last":0.878},{"interface":"front","average":{"1min":0.66300000000000003,"5min":0.745,"15min":0.69999999999999996},"min":{"1min":0.45700000000000002,"5min":0.45700000000000002,"15min":0.45700000000000002},"max":{"1min":0.85199999999999998,"5min":2.125,"15min":2.125},"last":0.90900000000000003}]},{"osd":3,"last update":"Sun Nov 26 07:24:50 2023","interfaces":[{"interface":"back","average":{"1min":0.68400000000000005,"5min":0.752,"15min":0.72899999999999998},"min":{"1min":0.52800000000000002,"5min":0.44700000000000001,"15min":0.43099999999999999},"max":{"1min":0.879,"5min":2.0529999999999999,"15min":2.0529999999999999},"last":0.83299999999999996},{"interface":"front","average":{"1min":0.67800000000000005,"5min":0.755,"15min":0.70299999999999996},"min":{"1min":0.55500000000000005,"5min":0.51900000000000002,"15min":0.44600000000000001},"max":{"1min":0.94099999999999995,"5min":1.629,"15min":1.629},"last":0.97799999999999998}]},{"osd":4,"last update":"Sun Nov 26 07:24:50 2023","interfaces":[{"interface":"back","average":{"1min":0.58499999999999996,"5min":0.61199999999999999,"15min":0.625},"min":{"1min":0.42199999999999999,"5min":0.35299999999999998,"15min":0.35299999999999998},"max":{"1min":0.84699999999999998,"5min":2.7429999999999999,"15min":2.7429999999999999},"last":0.68799999999999994},{"interface":"front","average":{"1min":0.60999999999999999,"5min":0.67300000000000004,"15min":0.69299999999999995},"min":{"1min":0.44400000000000001,"5min":0.44400000000000001,"15min":0.41299999999999998},"max":{"1min":0.85999999999999999,"5min":2.4630000000000001,"15min":2.4630000000000001},"last":0.74399999999999999}]},{"osd":6,"last update":"Sun Nov 26 07:24:50 2023","interfaces":[{"interface":"back","average":{"1min":0.63800000000000001,"5min":0.66600000000000004,"15min":0.67100000000000004},"min":{"1min":0.38300000000000001,"5min":0.372,"15min":0.372},"max":{"1min":1.5409999999999999,"5min":2.1619999999999999,"15min":2.1619999999999999},"last":0.997},{"interface":"front","average":{"1min":0.63,"5min":0.69099999999999995,"15min":0.71499999999999997},"min":{"1min":0.47599999999999998,"5min":0.436,"15min":0.436},"max":{"1min":0.89500000000000002,"5min":2.7869999999999999,"15min":2.7869999999999999},"last":0.89700000000000002}]},{"osd":7,"last update":"Sun Nov 26 07:24:50 2023","interfaces":[{"interface":"back","average":{"1min":0.66000000000000003,"5min":0.73099999999999998,"15min":0.77500000000000002},"min":{"1min":0.46999999999999997,"5min":0.41399999999999998,"15min":0.41399999999999998},"max":{"1min":0.878,"5min":2.1960000000000002,"15min":2.1960000000000002},"last":0.95399999999999996},{"interface":"front","average":{"1min":0.67700000000000005,"5min":0.748,"15min":0.76400000000000001},"min":{"1min":0.51100000000000001,"5min":0.44700000000000001,"15min":0.44700000000000001},"max":{"1min":0.94899999999999995,"5min":1.9510000000000001,"15min":1.9510000000000001},"last":0.80700000000000005}]}]},{"osd":4,"up_from":13,"seq":55834574944,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":32476,"kb_used_data":1296,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339364,"statfs":{"total":96636764160,"available":96603508736,"internally_reserved":0,"allocated":1327104,"data_stored":579977,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Nov 26 07:24:55 2023","interfaces":[{"interface":"back","average":{"1min":0.65000000000000002,"5min":0.66600000000000004,"15min":0.73899999999999999},"min":{"1min":0.46600000000000003,"5min":0.443,"15min":0.32100000000000001},"max":{"1min":0.80000000000000004,"5min":0.96199999999999997,"15min":4.1689999999999996},"last":0.76800000000000002},{"interface":"front","average":{"1min":0.67500000000000004,"5min":0.85999999999999999,"15min":0.92800000000000005},"min":{"1min":0.49399999999999999,"5min":0.40799999999999997,"15min":0.40799999999999997},"max":{"1min":1.0209999999999999,"5min":18.021000000000001,"15min":18.021000000000001},"last":0.96899999999999997}]},{"osd":1,"last update":"Sun Nov 26 07:24:55 2023","interfaces":[{"interface":"back","average":{"1min":0.70599999999999996,"5min":0.75,"15min":0.68000000000000005},"min":{"1min":0.54200000000000004,"5min":0.43099999999999999,"15min":0.43099999999999999},"max":{"1min":0.95199999999999996,"5min":4.3159999999999998,"15min":4.3159999999999998},"last":0.876},{"interface":"front","average":{"1min":0.63500000000000001,"5min":0.77100000000000002,"15min":0.78900000000000003},"min":{"1min":0.438,"5min":0.436,"15min":0.39800000000000002},"max":{"1min":0.89800000000000002,"5min":11.206,"15min":11.206},"last":0.74399999999999999}]},{"osd":2,"last update":"Sun Nov 26 07:24:55 2023","interfaces":[{"interface":"back","average":{"1min":0.72199999999999998,"5min":0.86499999999999999,"15min":0.81599999999999995},"min":{"1min":0.47399999999999998,"5min":0.46600000000000003,"15min":0.433},"max":{"1min":0.89900000000000002,"5min":15.441000000000001,"15min":15.441000000000001},"last":0.89800000000000002},{"interface":"front","average":{"1min":0.79200000000000004,"5min":0.871,"15min":0.97799999999999998},"min":{"1min":0.54300000000000004,"5min":0.54300000000000004,"15min":0.53400000000000003},"max":{"1min":1.071,"5min":9.4800000000000004,"15min":9.4800000000000004},"last":0.93600000000000005}]},{"osd":3,"last update":"Sun Nov 26 07:24:55 2023","interfaces":[{"interface":"back","average":{"1min":0.69499999999999995,"5min":0.83899999999999997,"15min":0.76000000000000001},"min":{"1min":0.51200000000000001,"5min":0.498,"15min":0.441},"max":{"1min":0.84699999999999998,"5min":12.715,"15min":12.715},"last":0.91300000000000003},{"interface":"front","average":{"1min":0.74399999999999999,"5min":0.82799999999999996,"15min":0.84799999999999998},"min":{"1min":0.55400000000000005,"5min":0.50700000000000001,"15min":0.50700000000000001},"max":{"1min":0.878,"5min":8.3239999999999998,"15min":8.3239999999999998},"last":0.95299999999999996}]},{"osd":5,"last update":"Sun Nov 26 07:24:55 2023","interfaces":[{"interface":"back","average":{"1min":0.68799999999999994,"5min":0.65100000000000002,"15min":0.622},"min":{"1min":0.41299999999999998,"5min":0.40500000000000003,"15min":0.38200000000000001},"max":{"1min":1.0229999999999999,"5min":1.0229999999999999,"15min":1.0229999999999999},"last":0.55100000000000005},{"interface":"front","average":{"1min":0.67400000000000004,"5min":0.74199999999999999,"15min":0.67000000000000004},"min":{"1min":0.41999999999999998,"5min":0.36399999999999999,"15min":0.35599999999999998},"max":{"1min":0.96099999999999997,"5min":11.148999999999999,"15min":11.148999999999999},"last":0.59699999999999998}]},{"osd":6,"last update":"Sun Nov 26 07:24:55 2023","interfaces":[{"interface":"back","average":{"1min":0.68600000000000005,"5min":0.626,"15min":0.72399999999999998},"min":{"1min":0.40000000000000002,"5min":0.38400000000000001,"15min":0.372},"max":{"1min":0.91600000000000004,"5min":0.91600000000000004,"15min":2.8330000000000002},"last":0.66800000000000004},{"interface":"front","average":{"1min":0.71999999999999997,"5min":0.72399999999999998,"15min":0.89100000000000001},"min":{"1min":0.443,"5min":0.42699999999999999,"15min":0.41299999999999998},"max":{"1min":1.046,"5min":2.472,"15min":7.6050000000000004},"last":0.626}]},{"osd":7,"last update":"Sun Nov 26 07:24:55 2023","interfaces":[{"interface":"back","average":{"1min":0.75900000000000001,"5min":0.83799999999999997,"15min":0.83899999999999997},"min":{"1min":0.59199999999999997,"5min":0.48699999999999999,"15min":0.44700000000000001},"max":{"1min":0.95999999999999996,"5min":12.653,"15min":12.653},"last":0.65400000000000003},{"interface":"front","average":{"1min":0.81599999999999995,"5min":0.79700000000000004,"15min":0.84799999999999998},"min":{"1min":0.56499999999999995,"5min":0.439,"15min":0.33300000000000002},"max":{"1min":0.97599999999999998,"5min":3.8319999999999999,"15min":3.8319999999999999},"last":0.72099999999999997}]}]},{"osd":3,"up_from":13,"seq":55834574945,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":32480,"kb_used_data":1296,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339360,"statfs":{"total":96636764160,"available":96603504640,"internally_reserved":0,"allocated":1327104,"data_stored":579977,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1592,"internal_metadata":31914440},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Nov 26 07:24:55 2023","interfaces":[{"interface":"back","average":{"1min":0.377,"5min":0.41099999999999998,"15min":0.39200000000000002},"min":{"1min":0.25900000000000001,"5min":0.218,"15min":0.218},"max":{"1min":0.58199999999999996,"5min":0.72799999999999998,"15min":0.72799999999999998},"last":0.41499999999999998},{"interface":"front","average":{"1min":0.41099999999999998,"5min":0.46000000000000002,"15min":0.443},"min":{"1min":0.27700000000000002,"5min":0.254,"15min":0.254},"max":{"1min":0.56599999999999995,"5min":0.77300000000000002,"15min":0.77300000000000002},"last":0.34699999999999998}]},{"osd":1,"last update":"Sun Nov 26 07:24:55 2023","interfaces":[{"interface":"back","average":{"1min":0.39200000000000002,"5min":0.435,"15min":0.42499999999999999},"min":{"1min":0.29899999999999999,"5min":0.28799999999999998,"15min":0.28799999999999998},"max":{"1min":0.53300000000000003,"5min":0.61299999999999999,"15min":0.752},"last":0.46600000000000003},{"interface":"front","average":{"1min":0.437,"5min":0.48599999999999999,"15min":0.48399999999999999},"min":{"1min":0.315,"5min":0.29899999999999999,"15min":0.29899999999999999},"max":{"1min":0.53300000000000003,"5min":0.872,"15min":0.872},"last":0.504}]},{"osd":2,"last update":"Sun Nov 26 07:24:55 2023","interfaces":[{"interface":"back","average":{"1min":0.504,"5min":0.55500000000000005,"15min":0.54200000000000004},"min":{"1min":0.36599999999999999,"5min":0.36599999999999999,"15min":0.36599999999999999},"max":{"1min":0.63200000000000001,"5min":0.97199999999999998,"15min":0.97199999999999998},"last":0.621},{"interface":"front","average":{"1min":0.48999999999999999,"5min":0.52300000000000002,"15min":0.51400000000000001},"min":{"1min":0.36299999999999999,"5min":0.32100000000000001,"15min":0.32100000000000001},"max":{"1min":0.65100000000000002,"5min":0.89700000000000002,"15min":0.89700000000000002},"last":0.58099999999999996}]},{"osd":4,"last update":"Sun Nov 26 07:24:55 2023","interfaces":[{"interface":"back","average":{"1min":0.72399999999999998,"5min":0.71099999999999997,"15min":0.70799999999999996},"min":{"1min":0.502,"5min":0.39600000000000002,"15min":0.39600000000000002},"max":{"1min":0.94399999999999995,"5min":0.94399999999999995,"15min":1.091},"last":0.92400000000000004},{"interface":"front","average":{"1min":0.72899999999999998,"5min":0.72299999999999998,"15min":0.72499999999999998},"min":{"1min":0.48999999999999999,"5min":0.39900000000000002,"15min":0.39900000000000002},"max":{"1min":1.0309999999999999,"5min":1.242,"15min":1.242},"last":0.80300000000000005}]},{"osd":5,"last update":"Sun Nov 26 07:24:55 2023","interfaces":[{"interface":"back","average":{"1min":0.70599999999999996,"5min":0.70099999999999996,"15min":0.72399999999999998},"min":{"1min":0.505,"5min":0.45900000000000002,"15min":0.45900000000000002},"max":{"1min":0.86499999999999999,"5min":1.105,"15min":1.105},"last":0.86799999999999999},{"interface":"front","average":{"1min":0.70999999999999996,"5min":0.73299999999999998,"15min":0.73499999999999999},"min":{"1min":0.53100000000000003,"5min":0.47299999999999998,"15min":0.47299999999999998},"max":{"1min":0.94599999999999995,"5min":1.1319999999999999,"15min":1.1319999999999999},"last":0.68400000000000005}]},{"osd":6,"last update":"Sun Nov 26 07:24:55 2023","interfaces":[{"interface":"back","average":{"1min":0.79000000000000004,"5min":0.76800000000000002,"15min":0.79400000000000004},"min":{"1min":0.57099999999999995,"5min":0.5,"15min":0.5},"max":{"1min":1.0640000000000001,"5min":1.214,"15min":1.2729999999999999},"last":0.84799999999999998},{"interface":"front","average":{"1min":0.77200000000000002,"5min":0.75900000000000001,"15min":0.77800000000000002},"min":{"1min":0.53700000000000003,"5min":0.48799999999999999,"15min":0.47299999999999998},"max":{"1min":1.0509999999999999,"5min":1.1060000000000001,"15min":1.204},"last":0.94899999999999995}]},{"osd":7,"last update":"Sun Nov 26 07:24:55 2023","interfaces":[{"interface":"back","average":{"1min":0.73899999999999999,"5min":0.755,"15min":0.78100000000000003},"min":{"1min":0.55700000000000005,"5min":0.437,"15min":0.437},"max":{"1min":1.0069999999999999,"5min":1.0349999999999999,"15min":1.3600000000000001},"last":0.78700000000000003},{"interface":"front","average":{"1min":0.80300000000000005,"5min":0.79000000000000004,"15min":0.81200000000000006},"min":{"1min":0.52600000000000002,"5min":0.47299999999999998,"15min":0.47299999999999998},"max":{"1min":1.0529999999999999,"5min":1.2450000000000001,"15min":1.617},"last":0.89800000000000002}]}]},{"osd":2,"up_from":13,"seq":55834574945,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":32476,"kb_used_data":1296,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339364,"statfs":{"total":96636764160,"available":96603508736,"internally_reserved":0,"allocated":1327104,"data_stored":579977,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Nov 26 07:24:52 2023","interfaces":[{"interface":"back","average":{"1min":0.435,"5min":0.42599999999999999,"15min":0.60099999999999998},"min":{"1min":0.26600000000000001,"5min":0.26600000000000001,"15min":0.26600000000000001},"max":{"1min":0.65400000000000003,"5min":0.72899999999999998,"15min":4.7030000000000003},"last":0.55600000000000005},{"interface":"front","average":{"1min":0.41799999999999998,"5min":0.46000000000000002,"15min":0.52000000000000002},"min":{"1min":0.28399999999999997,"5min":0.27300000000000002,"15min":0.27300000000000002},"max":{"1min":0.66100000000000003,"5min":0.77200000000000002,"15min":2.4340000000000002},"last":0.44800000000000001}]},{"osd":1,"last update":"Sun Nov 26 07:24:52 2023","interfaces":[{"interface":"back","average":{"1min":0.47199999999999998,"5min":0.48699999999999999,"15min":0.71299999999999997},"min":{"1min":0.33000000000000002,"5min":0.318,"15min":0.318},"max":{"1min":0.61499999999999999,"5min":0.86099999999999999,"15min":4.2690000000000001},"last":0.66500000000000004},{"interface":"front","average":{"1min":0.47299999999999998,"5min":0.502,"15min":0.64300000000000002},"min":{"1min":0.33200000000000002,"5min":0.33200000000000002,"15min":0.31900000000000001},"max":{"1min":0.68100000000000005,"5min":0.76600000000000001,"15min":3.9940000000000002},"last":0.499}]},{"osd":3,"last update":"Sun Nov 26 07:24:52 2023","interfaces":[{"interface":"back","average":{"1min":0.46500000000000002,"5min":0.47699999999999998,"15min":0.57499999999999996},"min":{"1min":0.314,"5min":0.29699999999999999,"15min":0.29699999999999999},"max":{"1min":0.73299999999999998,"5min":0.82699999999999996,"15min":4.0300000000000002},"last":0.58399999999999996},{"interface":"front","average":{"1min":0.53000000000000003,"5min":0.54700000000000004,"15min":0.76700000000000002},"min":{"1min":0.374,"5min":0.36099999999999999,"15min":0.36099999999999999},"max":{"1min":0.83999999999999997,"5min":0.86699999999999999,"15min":4.5810000000000004},"last":0.61599999999999999}]},{"osd":4,"last update":"Sun Nov 26 07:24:52 2023","interfaces":[{"interface":"back","average":{"1min":0.72999999999999998,"5min":0.75800000000000001,"15min":0.89500000000000002},"min":{"1min":0.56100000000000005,"5min":0.51400000000000001,"15min":0.51400000000000001},"max":{"1min":1.0189999999999999,"5min":1.0189999999999999,"15min":4.4269999999999996},"last":0.72599999999999998},{"interface":"front","average":{"1min":0.73899999999999999,"5min":0.77100000000000002,"15min":0.91400000000000003},"min":{"1min":0.53500000000000003,"5min":0.53500000000000003,"15min":0.53500000000000003},"max":{"1min":0.96199999999999997,"5min":1.0780000000000001,"15min":4.085},"last":0.86599999999999999}]},{"osd":5,"last update":"Sun Nov 26 07:24:52 2023","interfaces":[{"interface":"back","average":{"1min":0.76700000000000002,"5min":0.77400000000000002,"15min":0.83099999999999996},"min":{"1min":0.53800000000000003,"5min":0.42099999999999999,"15min":0.42099999999999999},"max":{"1min":1.2090000000000001,"5min":1.2090000000000001,"15min":3.9590000000000001},"last":0.81899999999999995},{"interface":"front","average":{"1min":0.74399999999999999,"5min":0.77400000000000002,"15min":0.90300000000000002},"min":{"1min":0.54500000000000004,"5min":0.499,"15min":0.499},"max":{"1min":1.024,"5min":1.0900000000000001,"15min":4.3849999999999998},"last":0.69699999999999995}]},{"osd":6,"last update":"Sun Nov 26 07:24:52 2023","interfaces":[{"interface":"back","average":{"1min":0.77500000000000002,"5min":0.77100000000000002,"15min":0.90500000000000003},"min":{"1min":0.58499999999999996,"5min":0.45200000000000001,"15min":0.45200000000000001},"max":{"1min":1.1160000000000001,"5min":1.1160000000000001,"15min":4.0449999999999999},"last":0.83499999999999996},{"interface":"front","average":{"1min":0.748,"5min":0.755,"15min":0.85299999999999998},"min":{"1min":0.56799999999999995,"5min":0.48099999999999998,"15min":0.48099999999999998},"max":{"1min":1.093,"5min":1.093,"15min":3.992},"last":0.70999999999999996}]},{"osd":7,"last update":"Sun Nov 26 07:24:52 2023","interfaces":[{"interface":"back","average":{"1min":0.79700000000000004,"5min":0.79300000000000004,"15min":0.93500000000000005},"min":{"1min":0.61399999999999999,"5min":0.56799999999999995,"15min":0.52200000000000002},"max":{"1min":1.0609999999999999,"5min":1.0840000000000001,"15min":4.1219999999999999},"last":0.89100000000000001},{"interface":"front","average":{"1min":0.77500000000000002,"5min":0.78100000000000003,"15min":0.875},"min":{"1min":0.61499999999999999,"5min":0.48199999999999998,"15min":0.48199999999999998},"max":{"1min":1.151,"5min":1.151,"15min":3.9159999999999999},"last":0.75800000000000001}]}]},{"osd":1,"up_from":13,"seq":55834574944,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":32476,"kb_used_data":1296,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339364,"statfs":{"total":96636764160,"available":96603508736,"internally_reserved":0,"allocated":1327104,"data_stored":579977,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":31914439},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Nov 26 07:24:51 2023","interfaces":[{"interface":"back","average":{"1min":0.38400000000000001,"5min":0.40500000000000003,"15min":0.41099999999999998},"min":{"1min":0.26600000000000001,"5min":0.26600000000000001,"15min":0.23599999999999999},"max":{"1min":0.58399999999999996,"5min":0.58399999999999996,"15min":0.61199999999999999},"last":0.499},{"interface":"front","average":{"1min":0.41199999999999998,"5min":0.44400000000000001,"15min":0.46899999999999997},"min":{"1min":0.28000000000000003,"5min":0.28000000000000003,"15min":0.26700000000000002},"max":{"1min":0.52100000000000002,"5min":0.749,"15min":0.78700000000000003},"last":0.57599999999999996}]},{"osd":2,"last update":"Sun Nov 26 07:24:51 2023","interfaces":[{"interface":"back","average":{"1min":0.42099999999999999,"5min":0.436,"15min":0.44600000000000001},"min":{"1min":0.30499999999999999,"5min":0.29899999999999999,"15min":0.27800000000000002},"max":{"1min":0.63400000000000001,"5min":0.71399999999999997,"15min":0.71399999999999997},"last":0.41699999999999998},{"interface":"front","average":{"1min":0.46200000000000002,"5min":0.47599999999999998,"15min":0.495},"min":{"1min":0.33100000000000002,"5min":0.318,"15min":0.27700000000000002},"max":{"1min":0.61799999999999999,"5min":0.80600000000000005,"15min":0.80600000000000005},"last":0.55800000000000005}]},{"osd":3,"last update":"Sun Nov 26 07:24:51 2023","interfaces":[{"interface":"back","average":{"1min":0.48499999999999999,"5min":0.52100000000000002,"15min":0.52500000000000002},"min":{"1min":0.33200000000000002,"5min":0.33200000000000002,"15min":0.33200000000000002},"max":{"1min":0.72699999999999998,"5min":0.748,"15min":0.748},"last":0.58899999999999997},{"interface":"front","average":{"1min":0.47499999999999998,"5min":0.498,"15min":0.496},"min":{"1min":0.38100000000000001,"5min":0.35499999999999998,"15min":0.31},"max":{"1min":0.71299999999999997,"5min":0.71299999999999997,"15min":0.80700000000000005},"last":0.54100000000000004}]},{"osd":4,"last update":"Sun Nov 26 07:24:51 2023","interfaces":[{"interface":"back","average":{"1min":0.72999999999999998,"5min":0.75800000000000001,"15min":0.71499999999999997},"min":{"1min":0.50600000000000001,"5min":0.50600000000000001,"15min":0.41299999999999998},"max":{"1min":0.95699999999999996,"5min":1.319,"15min":1.319},"last":1.167},{"interface":"front","average":{"1min":0.71899999999999997,"5min":0.72499999999999998,"15min":0.69799999999999995},"min":{"1min":0.54300000000000004,"5min":0.44500000000000001,"15min":0.44500000000000001},"max":{"1min":0.97999999999999998,"5min":1.006,"15min":1.131},"last":1.052}]},{"osd":5,"last update":"Sun Nov 26 07:24:51 2023","interfaces":[{"interface":"back","average":{"1min":0.73299999999999998,"5min":0.747,"15min":0.72899999999999998},"min":{"1min":0.501,"5min":0.47899999999999998,"15min":0.47899999999999998},"max":{"1min":0.95699999999999996,"5min":1.0409999999999999,"15min":1.1919999999999999},"last":1.139},{"interface":"front","average":{"1min":0.73499999999999999,"5min":0.75,"15min":0.72899999999999998},"min":{"1min":0.53100000000000003,"5min":0.47099999999999997,"15min":0.47099999999999997},"max":{"1min":0.86299999999999999,"5min":1.0389999999999999,"15min":1.0609999999999999},"last":0.83799999999999997}]},{"osd":6,"last update":"Sun Nov 26 07:24:51 2023","interfaces":[{"interface":"back","average":{"1min":0.72499999999999998,"5min":0.76000000000000001,"15min":0.72399999999999998},"min":{"1min":0.53800000000000003,"5min":0.53200000000000003,"15min":0.50600000000000001},"max":{"1min":0.89600000000000002,"5min":1.2190000000000001,"15min":1.2190000000000001},"last":1.105},{"interface":"front","average":{"1min":0.72199999999999998,"5min":0.76200000000000001,"15min":0.72499999999999998},"min":{"1min":0.52200000000000002,"5min":0.46999999999999997,"15min":0.46999999999999997},"max":{"1min":0.91800000000000004,"5min":1.1659999999999999,"15min":1.1659999999999999},"last":0.98299999999999998}]},{"osd":7,"last update":"Sun Nov 26 07:24:51 2023","interfaces":[{"interface":"back","average":{"1min":0.76700000000000002,"5min":0.77400000000000002,"15min":0.73599999999999999},"min":{"1min":0.57099999999999995,"5min":0.50600000000000001,"15min":0.50600000000000001},"max":{"1min":0.97299999999999998,"5min":1.0920000000000001,"15min":1.0920000000000001},"last":1.0780000000000001},{"interface":"front","average":{"1min":0.76200000000000001,"5min":0.76500000000000001,"15min":0.755},"min":{"1min":0.54000000000000004,"5min":0.53700000000000003,"15min":0.48499999999999999},"max":{"1min":1.006,"5min":1.115,"15min":1.115},"last":1.024}]}]},{"osd":0,"up_from":13,"seq":55834574944,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":33052,"kb_used_data":1876,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94338788,"statfs":{"total":96636764160,"available":96602918912,"internally_reserved":0,"allocated":1921024,"data_stored":1170345,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Sun Nov 26 07:24:50 2023","interfaces":[{"interface":"back","average":{"1min":0.439,"5min":0.437,"15min":0.43099999999999999},"min":{"1min":0.25800000000000001,"5min":0.25800000000000001,"15min":0.25800000000000001},"max":{"1min":0.70499999999999996,"5min":1.26,"15min":1.26},"last":0.502},{"interface":"front","average":{"1min":0.45100000000000001,"5min":0.47299999999999998,"15min":0.47499999999999998},"min":{"1min":0.29899999999999999,"5min":0.29899999999999999,"15min":0.29899999999999999},"max":{"1min":0.72099999999999997,"5min":1.212,"15min":1.212},"last":0.59099999999999997}]},{"osd":2,"last update":"Sun Nov 26 07:24:50 2023","interfaces":[{"interface":"back","average":{"1min":0.47199999999999998,"5min":0.497,"15min":0.52200000000000002},"min":{"1min":0.33000000000000002,"5min":0.33000000000000002,"15min":0.33000000000000002},"max":{"1min":0.68100000000000005,"5min":0.84199999999999997,"15min":0.84199999999999997},"last":0.66300000000000003},{"interface":"front","average":{"1min":0.53900000000000003,"5min":0.55800000000000005,"15min":0.55900000000000005},"min":{"1min":0.45200000000000001,"5min":0.40100000000000002,"15min":0.36399999999999999},"max":{"1min":0.81000000000000005,"5min":1.0800000000000001,"15min":1.0800000000000001},"last":0.82599999999999996}]},{"osd":3,"last update":"Sun Nov 26 07:24:50 2023","interfaces":[{"interface":"back","average":{"1min":0.51600000000000001,"5min":0.51300000000000001,"15min":0.52400000000000002},"min":{"1min":0.41099999999999998,"5min":0.33500000000000002,"15min":0.33500000000000002},"max":{"1min":0.72899999999999998,"5min":0.873,"15min":1.3149999999999999},"last":0.56000000000000005},{"interface":"front","average":{"1min":0.53000000000000003,"5min":0.53600000000000003,"15min":0.54900000000000004},"min":{"1min":0.377,"5min":0.35499999999999998,"15min":0.35499999999999998},"max":{"1min":0.74299999999999999,"5min":0.74299999999999999,"15min":1.2509999999999999},"last":0.53600000000000003}]},{"osd":4,"last update":"Sun Nov 26 07:24:50 2023","interfaces":[{"interface":"back","average":{"1min":0.68500000000000005,"5min":0.73899999999999999,"15min":0.749},"min":{"1min":0.40100000000000002,"5min":0.40100000000000002,"15min":0.40100000000000002},"max":{"1min":0.91300000000000003,"5min":1.0900000000000001,"15min":1.2010000000000001},"last":0.67900000000000005},{"interface":"front","average":{"1min":0.76000000000000001,"5min":0.78000000000000003,"15min":0.78600000000000003},"min":{"1min":0.55900000000000005,"5min":0.51100000000000001,"15min":0.51100000000000001},"max":{"1min":1.012,"5min":1.298,"15min":1.381},"last":0.74299999999999999}]},{"osd":5,"last update":"Sun Nov 26 07:24:50 2023","interfaces":[{"interface":"back","average":{"1min":0.66800000000000004,"5min":0.73599999999999999,"15min":0.75},"min":{"1min":0.50800000000000001,"5min":0.45200000000000001,"15min":0.42199999999999999},"max":{"1min":0.91600000000000004,"5min":1.1819999999999999,"15min":1.1819999999999999},"last":0.72699999999999998},{"interface":"front","average":{"1min":0.77500000000000002,"5min":0.79800000000000004,"15min":0.79700000000000004},"min":{"1min":0.59799999999999998,"5min":0.53000000000000003,"15min":0.53000000000000003},"max":{"1min":1.0609999999999999,"5min":1.329,"15min":1.399},"last":0.84599999999999997}]},{"osd":6,"last update":"Sun Nov 26 07:24:50 2023","interfaces":[{"interface":"back","average":{"1min":0.70599999999999996,"5min":0.70599999999999996,"15min":0.73699999999999999},"min":{"1min":0.48999999999999999,"5min":0.48499999999999999,"15min":0.48499999999999999},"max":{"1min":0.97199999999999998,"5min":1.0740000000000001,"15min":1.3029999999999999},"last":0.70099999999999996},{"interface":"front","average":{"1min":0.71099999999999997,"5min":0.72899999999999998,"15min":0.754},"min":{"1min":0.52100000000000002,"5min":0.47199999999999998,"15min":0.47199999999999998},"max":{"1min":0.93600000000000005,"5min":1.2170000000000001,"15min":1.3600000000000001},"last":0.79500000000000004}]},{"osd":7,"last update":"Sun Nov 26 07:24:50 2023","interfaces":[{"interface":"back","average":{"1min":0.75600000000000001,"5min":0.76800000000000002,"15min":0.79300000000000004},"min":{"1min":0.60099999999999998,"5min":0.56899999999999995,"15min":0.56899999999999995},"max":{"1min":0.97999999999999998,"5min":1.1719999999999999,"15min":1.177},"last":0.76700000000000002},{"interface":"front","average":{"1min":0.69899999999999995,"5min":0.76500000000000001,"15min":0.77800000000000002},"min":{"1min":0.56899999999999995,"5min":0.47899999999999998,"15min":0.47899999999999998},"max":{"1min":0.92400000000000004,"5min":1.238,"15min":1.2390000000000001},"last":0.622}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-11-26T07:25:23.912 DEBUG:teuthology.orchestra.run.smithi143:> sudo ceph --cluster ceph config set global mon_health_to_clog false 2023-11-26T07:25:24.048 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:25:24.046+0000 7f486552a640 1 Processor -- start 2023-11-26T07:25:24.048 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:25:24.046+0000 7f486552a640 1 -- start start 2023-11-26T07:25:24.049 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:25:24.050+0000 7f486552a640 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7f4860161cf0 0x7f486006bae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:25:24.050 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:25:24.050+0000 7f486552a640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f48600cdd10 0x7f486006d950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:25:24.050 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:25:24.050+0000 7f486552a640 1 -- --> v1:172.21.15.188:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f48600cd610 con 0x7f4860070ba0 2023-11-26T07:25:24.050 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:25:24.050+0000 7f486552a640 1 -- --> v2:172.21.15.188:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f48600cd8f0 con 0x7f4860161cf0 2023-11-26T07:25:24.050 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:25:24.050+0000 7f486552a640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4860161210 con 0x7f48600cdd10 2023-11-26T07:25:24.051 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:25:24.050+0000 7f48567fc640 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7f4860161cf0 0x7f486006bae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:25:24.051 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:25:24.050+0000 7f485f7fe640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f48600cdd10 0x7f486006d950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:25:24.051 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:25:24.050+0000 7f48567fc640 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7f4860161cf0 0x7f486006bae0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.188:3300/0 says I am v2:172.21.15.143:51294/0 (socket says 172.21.15.143:51294) 2023-11-26T07:25:24.051 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:25:24.050+0000 7f48567fc640 1 -- 172.21.15.143:0/4183232426 learned_addr learned my addr 172.21.15.143:0/4183232426 (peer_addr_for_me v2:172.21.15.143:0/0) 2023-11-26T07:25:24.051 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:25:24.050+0000 7f48567fc640 1 -- 172.21.15.143:0/4183232426 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f48600cdd10 msgr2=0x7f486006d950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-26T07:25:24.051 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:25:24.050+0000 7f48567fc640 1 --2- 172.21.15.143:0/4183232426 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f48600cdd10 0x7f486006d950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:25:24.052 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:25:24.050+0000 7f48567fc640 1 -- 172.21.15.143:0/4183232426 >> v1:172.21.15.188:6789/0 conn(0x7f4860070ba0 legacy=0x7f4860070f80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:25:24.052 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:25:24.050+0000 7f48567fc640 1 -- 172.21.15.143:0/4183232426 --> v2:172.21.15.188:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f48601614f0 con 0x7f4860161cf0 2023-11-26T07:25:24.052 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:25:24.050+0000 7f485f7fe640 1 --2- 172.21.15.143:0/4183232426 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f48600cdd10 0x7f486006d950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-26T07:25:24.053 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:25:24.054+0000 7f48567fc640 1 --2- 172.21.15.143:0/4183232426 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f4860161cf0 0x7f486006bae0 secure :-1 s=READY pgs=547 cs=0 l=1 rev1=1 crypto rx=0x7f4848009870 tx=0x7f484801c540 comp rx=0 tx=0).ready entity=mon.1 client_cookie=2852a884f421721b server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:25:24.054 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:25:24.054+0000 7f485e7fc640 1 -- 172.21.15.143:0/4183232426 <== mon.1 v2:172.21.15.188:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f48480a9620 con 0x7f4860161cf0 2023-11-26T07:25:24.054 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:25:24.054+0000 7f485e7fc640 1 -- 172.21.15.143:0/4183232426 <== mon.1 v2:172.21.15.188:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f48480a9780 con 0x7f4860161cf0 2023-11-26T07:25:24.054 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:25:24.054+0000 7f485e7fc640 1 -- 172.21.15.143:0/4183232426 <== mon.1 v2:172.21.15.188:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f48480a9b10 con 0x7f4860161cf0 2023-11-26T07:25:24.054 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:25:24.054+0000 7f486552a640 1 -- 172.21.15.143:0/4183232426 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f4860161cf0 msgr2=0x7f486006bae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:25:24.055 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:25:24.054+0000 7f486552a640 1 --2- 172.21.15.143:0/4183232426 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f4860161cf0 0x7f486006bae0 secure :-1 s=READY pgs=547 cs=0 l=1 rev1=1 crypto rx=0x7f4848009870 tx=0x7f484801c540 comp rx=0 tx=0).stop 2023-11-26T07:25:24.055 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:25:24.054+0000 7f486552a640 1 -- 172.21.15.143:0/4183232426 shutdown_connections 2023-11-26T07:25:24.055 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:25:24.054+0000 7f486552a640 1 --2- 172.21.15.143:0/4183232426 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f4860161cf0 0x7f486006bae0 unknown :-1 s=CLOSED pgs=547 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:25:24.056 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:25:24.054+0000 7f486552a640 1 --2- 172.21.15.143:0/4183232426 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f48600cdd10 0x7f486006d950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:25:24.056 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:25:24.054+0000 7f486552a640 1 -- 172.21.15.143:0/4183232426 >> 172.21.15.143:0/4183232426 conn(0x7f48600cf710 msgr2=0x7f48600cfb20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:25:24.056 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:25:24.054+0000 7f486552a640 1 -- 172.21.15.143:0/4183232426 shutdown_connections 2023-11-26T07:25:24.056 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:25:24.054+0000 7f486552a640 1 -- 172.21.15.143:0/4183232426 wait complete. 2023-11-26T07:25:24.056 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:25:24.054+0000 7f486552a640 1 Processor -- start 2023-11-26T07:25:24.057 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:25:24.058+0000 7f486552a640 1 -- start start 2023-11-26T07:25:24.059 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:25:24.058+0000 7f486552a640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f4860070ba0 0x7f48601507d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:25:24.059 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:25:24.058+0000 7f486552a640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f48600cdd10 0x7f4860156be0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:25:24.059 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:25:24.058+0000 7f486552a640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f4860161cf0 0x7f48600b0290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:25:24.059 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:25:24.058+0000 7f486552a640 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4860155b40 con 0x7f4860070ba0 2023-11-26T07:25:24.059 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:25:24.058+0000 7f486552a640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4860155e20 con 0x7f48600cdd10 2023-11-26T07:25:24.060 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:25:24.058+0000 7f486552a640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4860156100 con 0x7f4860161cf0 2023-11-26T07:25:24.060 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:25:24.058+0000 7f485effd640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f4860070ba0 0x7f48601507d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:25:24.060 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:25:24.058+0000 7f485effd640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f4860070ba0 0x7f48601507d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:51156/0 (socket says 172.21.15.143:51156) 2023-11-26T07:25:24.060 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:25:24.058+0000 7f485effd640 1 -- 172.21.15.143:0/1568437578 learned_addr learned my addr 172.21.15.143:0/1568437578 (peer_addr_for_me v2:172.21.15.143:0/0) 2023-11-26T07:25:24.060 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:25:24.058+0000 7f48567fc640 1 --2- 172.21.15.143:0/1568437578 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f48600cdd10 0x7f4860156be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:25:24.060 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:25:24.058+0000 7f485f7fe640 1 --2- 172.21.15.143:0/1568437578 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f4860161cf0 0x7f48600b0290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:25:24.060 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:25:24.058+0000 7f485effd640 1 -- 172.21.15.143:0/1568437578 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f4860161cf0 msgr2=0x7f48600b0290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:25:24.060 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:25:24.058+0000 7f485effd640 1 --2- 172.21.15.143:0/1568437578 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f4860161cf0 0x7f48600b0290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:25:24.061 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:25:24.058+0000 7f485effd640 1 -- 172.21.15.143:0/1568437578 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f48600cdd10 msgr2=0x7f4860156be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:25:24.061 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:25:24.058+0000 7f485effd640 1 --2- 172.21.15.143:0/1568437578 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f48600cdd10 0x7f4860156be0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:25:24.061 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:25:24.058+0000 7f485effd640 1 -- 172.21.15.143:0/1568437578 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4858009a70 con 0x7f4860070ba0 2023-11-26T07:25:24.061 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:25:24.058+0000 7f485effd640 1 --2- 172.21.15.143:0/1568437578 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f4860070ba0 0x7f48601507d0 secure :-1 s=READY pgs=1498 cs=0 l=1 rev1=1 crypto rx=0x7f485000c500 tx=0x7f485000c900 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:25:24.061 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:25:24.058+0000 7f4857fff640 1 -- 172.21.15.143:0/1568437578 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f48500052d0 con 0x7f4860070ba0 2023-11-26T07:25:24.061 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:25:24.058+0000 7f4857fff640 1 -- 172.21.15.143:0/1568437578 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4850005430 con 0x7f4860070ba0 2023-11-26T07:25:24.061 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:25:24.058+0000 7f4857fff640 1 -- 172.21.15.143:0/1568437578 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4850003840 con 0x7f4860070ba0 2023-11-26T07:25:24.062 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:25:24.058+0000 7f486552a640 1 -- 172.21.15.143:0/1568437578 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f48480094d0 con 0x7f4860070ba0 2023-11-26T07:25:24.062 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:25:24.058+0000 7f486552a640 1 -- 172.21.15.143:0/1568437578 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f48601566c0 con 0x7f4860070ba0 2023-11-26T07:25:24.063 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:25:24.058+0000 7f486552a640 1 -- 172.21.15.143:0/1568437578 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4860156900 con 0x7f4860070ba0 2023-11-26T07:25:24.063 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:25:24.062+0000 7f4857fff640 1 -- 172.21.15.143:0/1568437578 <== mon.0 v2:172.21.15.143:3300/0 4 ==== mgrmap(e 18) v1 ==== 86107+0+0 (secure 0 0 0) 0x7f48500039a0 con 0x7f4860070ba0 2023-11-26T07:25:24.063 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:25:24.062+0000 7f4857fff640 1 --2- 172.21.15.143:0/1568437578 >> [v2:172.21.15.188:6836/3401128553,v1:172.21.15.188:6837/3401128553] conn(0x7f4840066770 0x7f4840068c30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:25:24.063 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:25:24.062+0000 7f4857fff640 1 -- 172.21.15.143:0/1568437578 <== mon.0 v2:172.21.15.143:3300/0 5 ==== osd_map(106..106 src has 1..106) v4 ==== 4943+0+0 (secure 0 0 0) 0x7f4850083430 con 0x7f4860070ba0 2023-11-26T07:25:24.063 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:25:24.062+0000 7f48567fc640 1 --2- 172.21.15.143:0/1568437578 >> [v2:172.21.15.188:6836/3401128553,v1:172.21.15.188:6837/3401128553] conn(0x7f4840066770 0x7f4840068c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:25:24.063 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:25:24.062+0000 7f48567fc640 1 --2- 172.21.15.143:0/1568437578 >> [v2:172.21.15.188:6836/3401128553,v1:172.21.15.188:6837/3401128553] conn(0x7f4840066770 0x7f4840068c30 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7f48480ae000 tx=0x7f48480a7000 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:25:24.065 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:25:24.066+0000 7f4857fff640 1 -- 172.21.15.143:0/1568437578 <== mon.0 v2:172.21.15.143:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172649 (secure 0 0 0) 0x7f4860156900 con 0x7f4860070ba0 2023-11-26T07:25:24.215 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:25:24.214+0000 7f486552a640 1 -- 172.21.15.143:0/1568437578 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command([{prefix=config set, name=mon_health_to_clog}] v 0) v1 -- 0x7f4860156900 con 0x7f4860070ba0 2023-11-26T07:25:24.220 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:25:24.218+0000 7f4857fff640 1 -- 172.21.15.143:0/1568437578 <== mon.0 v2:172.21.15.143:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mon_health_to_clog}]=0 v22)=0 v22) v1 ==== 123+0+0 (secure 0 0 0) 0x7f4860156900 con 0x7f4860070ba0 2023-11-26T07:25:24.221 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:25:24.222+0000 7f4857fff640 1 -- 172.21.15.143:0/1568437578 <== mon.0 v2:172.21.15.143:3300/0 8 ==== config(1 keys) v1 ==== 35+0+0 (secure 0 0 0) 0x7f48500011d0 con 0x7f4860070ba0 2023-11-26T07:25:24.223 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:25:24.222+0000 7f486552a640 1 -- 172.21.15.143:0/1568437578 >> [v2:172.21.15.188:6836/3401128553,v1:172.21.15.188:6837/3401128553] conn(0x7f4840066770 msgr2=0x7f4840068c30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:25:24.223 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:25:24.222+0000 7f486552a640 1 --2- 172.21.15.143:0/1568437578 >> [v2:172.21.15.188:6836/3401128553,v1:172.21.15.188:6837/3401128553] conn(0x7f4840066770 0x7f4840068c30 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7f48480ae000 tx=0x7f48480a7000 comp rx=0 tx=0).stop 2023-11-26T07:25:24.223 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:25:24.222+0000 7f486552a640 1 -- 172.21.15.143:0/1568437578 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f4860070ba0 msgr2=0x7f48601507d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:25:24.223 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:25:24.222+0000 7f486552a640 1 --2- 172.21.15.143:0/1568437578 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f4860070ba0 0x7f48601507d0 secure :-1 s=READY pgs=1498 cs=0 l=1 rev1=1 crypto rx=0x7f485000c500 tx=0x7f485000c900 comp rx=0 tx=0).stop 2023-11-26T07:25:24.223 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:25:24.222+0000 7f486552a640 1 -- 172.21.15.143:0/1568437578 shutdown_connections 2023-11-26T07:25:24.224 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:25:24.222+0000 7f486552a640 1 --2- 172.21.15.143:0/1568437578 >> [v2:172.21.15.188:6836/3401128553,v1:172.21.15.188:6837/3401128553] conn(0x7f4840066770 0x7f4840068c30 unknown :-1 s=CLOSED pgs=205 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:25:24.224 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:25:24.222+0000 7f486552a640 1 --2- 172.21.15.143:0/1568437578 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f4860161cf0 0x7f48600b0290 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:25:24.224 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:25:24.222+0000 7f486552a640 1 --2- 172.21.15.143:0/1568437578 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f48600cdd10 0x7f4860156be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:25:24.224 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:25:24.222+0000 7f486552a640 1 --2- 172.21.15.143:0/1568437578 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f4860070ba0 0x7f48601507d0 unknown :-1 s=CLOSED pgs=1498 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:25:24.224 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:25:24.222+0000 7f486552a640 1 -- 172.21.15.143:0/1568437578 >> 172.21.15.143:0/1568437578 conn(0x7f48600cf710 msgr2=0x7f486015cc00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:25:24.224 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:25:24.222+0000 7f486552a640 1 -- 172.21.15.143:0/1568437578 shutdown_connections 2023-11-26T07:25:24.224 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:25:24.222+0000 7f486552a640 1 -- 172.21.15.143:0/1568437578 wait complete. 2023-11-26T07:25:24.242 INFO:tasks.cephfs.filesystem:Destroying file system cephfs and related pools 2023-11-26T07:25:24.242 DEBUG:teuthology.orchestra.run.smithi143:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-11-26T07:25:24.379 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:25:24.378+0000 7f4d8932e640 1 Processor -- start 2023-11-26T07:25:24.379 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:25:24.378+0000 7f4d8932e640 1 -- start start 2023-11-26T07:25:24.381 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:25:24.378+0000 7f4d8932e640 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7f4d8415d580 0x7f4d840d3920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:25:24.381 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:25:24.378+0000 7f4d8932e640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f4d840d1f00 0x7f4d840b0490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:25:24.381 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:25:24.378+0000 7f4d8932e640 1 -- --> v1:172.21.15.188:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4d840d0bc0 con 0x7f4d840d3e60 2023-11-26T07:25:24.381 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:25:24.382+0000 7f4d8932e640 1 -- --> v2:172.21.15.143:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4d840d0320 con 0x7f4d8415d580 2023-11-26T07:25:24.381 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:25:24.382+0000 7f4d8932e640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4d840d0600 con 0x7f4d840d1f00 2023-11-26T07:25:24.382 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:25:24.382+0000 7f4d82ffd640 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7f4d8415d580 0x7f4d840d3920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:25:24.382 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:25:24.382+0000 7f4d82ffd640 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7f4d8415d580 0x7f4d840d3920 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:51158/0 (socket says 172.21.15.143:51158) 2023-11-26T07:25:24.382 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:25:24.382+0000 7f4d82ffd640 1 -- 172.21.15.143:0/1665389654 learned_addr learned my addr 172.21.15.143:0/1665389654 (peer_addr_for_me v2:172.21.15.143:0/0) 2023-11-26T07:25:24.382 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:25:24.382+0000 7f4d837fe640 1 --2- 172.21.15.143:0/1665389654 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f4d840d1f00 0x7f4d840b0490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:25:24.382 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:25:24.382+0000 7f4d82ffd640 1 -- 172.21.15.143:0/1665389654 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f4d840d1f00 msgr2=0x7f4d840b0490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-26T07:25:24.382 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:25:24.382+0000 7f4d82ffd640 1 --2- 172.21.15.143:0/1665389654 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f4d840d1f00 0x7f4d840b0490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:25:24.382 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:25:24.382+0000 7f4d82ffd640 1 -- 172.21.15.143:0/1665389654 >> v1:172.21.15.188:6789/0 conn(0x7f4d840d3e60 legacy=0x7f4d840d17f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-26T07:25:24.383 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:25:24.382+0000 7f4d82ffd640 1 -- 172.21.15.143:0/1665389654 --> v2:172.21.15.143:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4d840d08e0 con 0x7f4d8415d580 2023-11-26T07:25:24.383 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:25:24.382+0000 7f4d82ffd640 1 --2- 172.21.15.143:0/1665389654 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f4d8415d580 0x7f4d840d3920 secure :-1 s=READY pgs=1499 cs=0 l=1 rev1=1 crypto rx=0x7f4d74009e50 tx=0x7f4d7401c9a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a92a6b9d9b576ebd server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:25:24.384 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:25:24.382+0000 7f4d827fc640 1 -- 172.21.15.143:0/1665389654 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4d740a5750 con 0x7f4d8415d580 2023-11-26T07:25:24.384 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:25:24.382+0000 7f4d827fc640 1 -- 172.21.15.143:0/1665389654 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(1 keys) v1 ==== 35+0+0 (secure 0 0 0) 0x7f4d740a58b0 con 0x7f4d8415d580 2023-11-26T07:25:24.384 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:25:24.382+0000 7f4d827fc640 1 -- 172.21.15.143:0/1665389654 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4d740a5c60 con 0x7f4d8415d580 2023-11-26T07:25:24.385 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:25:24.386+0000 7f4d8932e640 1 -- 172.21.15.143:0/1665389654 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f4d8415d580 msgr2=0x7f4d840d3920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:25:24.385 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:25:24.386+0000 7f4d8932e640 1 --2- 172.21.15.143:0/1665389654 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f4d8415d580 0x7f4d840d3920 secure :-1 s=READY pgs=1499 cs=0 l=1 rev1=1 crypto rx=0x7f4d74009e50 tx=0x7f4d7401c9a0 comp rx=0 tx=0).stop 2023-11-26T07:25:24.386 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:25:24.386+0000 7f4d8932e640 1 -- 172.21.15.143:0/1665389654 shutdown_connections 2023-11-26T07:25:24.386 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:25:24.386+0000 7f4d8932e640 1 --2- 172.21.15.143:0/1665389654 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f4d840d1f00 0x7f4d840b0490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:25:24.386 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:25:24.386+0000 7f4d8932e640 1 --2- 172.21.15.143:0/1665389654 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f4d8415d580 0x7f4d840d3920 unknown :-1 s=CLOSED pgs=1499 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:25:24.386 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:25:24.386+0000 7f4d8932e640 1 -- 172.21.15.143:0/1665389654 >> 172.21.15.143:0/1665389654 conn(0x7f4d8409c550 msgr2=0x7f4d840d3210 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:25:24.386 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:25:24.386+0000 7f4d8932e640 1 -- 172.21.15.143:0/1665389654 shutdown_connections 2023-11-26T07:25:24.386 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:25:24.386+0000 7f4d8932e640 1 -- 172.21.15.143:0/1665389654 wait complete. 2023-11-26T07:25:24.387 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:25:24.386+0000 7f4d8932e640 1 Processor -- start 2023-11-26T07:25:24.387 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:25:24.386+0000 7f4d8932e640 1 -- start start 2023-11-26T07:25:24.388 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:25:24.386+0000 7f4d8932e640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f4d840d1f00 0x7f4d84156500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:25:24.388 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:25:24.386+0000 7f4d8932e640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f4d840d3e60 0x7f4d84154840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:25:24.388 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:25:24.386+0000 7f4d8932e640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f4d8415d580 0x7f4d840b86e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:25:24.388 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:25:24.386+0000 7f4d8932e640 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4d840c7710 con 0x7f4d8415d580 2023-11-26T07:25:24.388 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:25:24.386+0000 7f4d8932e640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4d84154d80 con 0x7f4d840d3e60 2023-11-26T07:25:24.388 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:25:24.386+0000 7f4d8932e640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4d840c6de0 con 0x7f4d840d1f00 2023-11-26T07:25:24.390 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:25:24.386+0000 7f4d837fe640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f4d8415d580 0x7f4d840b86e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:25:24.390 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:25:24.386+0000 7f4d837fe640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f4d8415d580 0x7f4d840b86e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:51164/0 (socket says 172.21.15.143:51164) 2023-11-26T07:25:24.390 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:25:24.386+0000 7f4d837fe640 1 -- 172.21.15.143:0/3296154808 learned_addr learned my addr 172.21.15.143:0/3296154808 (peer_addr_for_me v2:172.21.15.143:0/0) 2023-11-26T07:25:24.390 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:25:24.386+0000 7f4d82ffd640 1 --2- 172.21.15.143:0/3296154808 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f4d840d1f00 0x7f4d84156500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:25:24.390 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:25:24.386+0000 7f4d7a7fc640 1 --2- 172.21.15.143:0/3296154808 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f4d840d3e60 0x7f4d84154840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:25:24.390 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:25:24.390+0000 7f4d837fe640 1 -- 172.21.15.143:0/3296154808 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f4d840d1f00 msgr2=0x7f4d84156500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:25:24.391 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:25:24.390+0000 7f4d837fe640 1 --2- 172.21.15.143:0/3296154808 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f4d840d1f00 0x7f4d84156500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:25:24.391 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:25:24.390+0000 7f4d837fe640 1 -- 172.21.15.143:0/3296154808 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f4d840d3e60 msgr2=0x7f4d84154840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:25:24.391 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:25:24.390+0000 7f4d837fe640 1 --2- 172.21.15.143:0/3296154808 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f4d840d3e60 0x7f4d84154840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:25:24.391 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:25:24.390+0000 7f4d837fe640 1 -- 172.21.15.143:0/3296154808 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4d74009b30 con 0x7f4d8415d580 2023-11-26T07:25:24.391 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:25:24.390+0000 7f4d82ffd640 1 --2- 172.21.15.143:0/3296154808 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f4d840d1f00 0x7f4d84156500 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-26T07:25:24.391 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:25:24.390+0000 7f4d837fe640 1 --2- 172.21.15.143:0/3296154808 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f4d8415d580 0x7f4d840b86e0 secure :-1 s=READY pgs=1500 cs=0 l=1 rev1=1 crypto rx=0x7f4d7c00d5d0 tx=0x7f4d7c00da90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:25:24.391 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:25:24.390+0000 7f4d7bfff640 1 -- 172.21.15.143:0/3296154808 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4d7c0125f0 con 0x7f4d8415d580 2023-11-26T07:25:24.391 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:25:24.390+0000 7f4d7bfff640 1 -- 172.21.15.143:0/3296154808 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(1 keys) v1 ==== 35+0+0 (secure 0 0 0) 0x7f4d7c012770 con 0x7f4d8415d580 2023-11-26T07:25:24.392 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:25:24.390+0000 7f4d7bfff640 1 -- 172.21.15.143:0/3296154808 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4d7c012b40 con 0x7f4d8415d580 2023-11-26T07:25:24.392 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:25:24.390+0000 7f4d8932e640 1 -- 172.21.15.143:0/3296154808 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4d840c70c0 con 0x7f4d8415d580 2023-11-26T07:25:24.392 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:25:24.390+0000 7f4d8932e640 1 -- 172.21.15.143:0/3296154808 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4d840b8c20 con 0x7f4d8415d580 2023-11-26T07:25:24.393 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:25:24.390+0000 7f4d8932e640 1 -- 172.21.15.143:0/3296154808 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4d840c7880 con 0x7f4d8415d580 2023-11-26T07:25:24.393 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:25:24.390+0000 7f4d7bfff640 1 -- 172.21.15.143:0/3296154808 <== mon.0 v2:172.21.15.143:3300/0 4 ==== mgrmap(e 18) v1 ==== 86107+0+0 (secure 0 0 0) 0x7f4d7c012ca0 con 0x7f4d8415d580 2023-11-26T07:25:24.393 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:25:24.394+0000 7f4d7bfff640 1 --2- 172.21.15.143:0/3296154808 >> [v2:172.21.15.188:6836/3401128553,v1:172.21.15.188:6837/3401128553] conn(0x7f4d64066740 0x7f4d64068c00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:25:24.393 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:25:24.394+0000 7f4d7bfff640 1 -- 172.21.15.143:0/3296154808 <== mon.0 v2:172.21.15.143:3300/0 5 ==== osd_map(106..106 src has 1..106) v4 ==== 4943+0+0 (secure 0 0 0) 0x7f4d7c0844c0 con 0x7f4d8415d580 2023-11-26T07:25:24.393 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:25:24.394+0000 7f4d82ffd640 1 --2- 172.21.15.143:0/3296154808 >> [v2:172.21.15.188:6836/3401128553,v1:172.21.15.188:6837/3401128553] conn(0x7f4d64066740 0x7f4d64068c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:25:24.394 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:25:24.394+0000 7f4d82ffd640 1 --2- 172.21.15.143:0/3296154808 >> [v2:172.21.15.188:6836/3401128553,v1:172.21.15.188:6837/3401128553] conn(0x7f4d64066740 0x7f4d64068c00 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7f4d740ae000 tx=0x7f4d740a7070 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:25:24.396 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:25:24.394+0000 7f4d7bfff640 1 -- 172.21.15.143:0/3296154808 <== mon.0 v2:172.21.15.143:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172649 (secure 0 0 0) 0x7f4d7c0518d0 con 0x7f4d8415d580 2023-11-26T07:25:24.577 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:25:24.578+0000 7f4d8932e640 1 -- 172.21.15.143:0/3296154808 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f4d84067680 con 0x7f4d8415d580 2023-11-26T07:25:24.577 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:25:24.578+0000 7f4d7bfff640 1 -- 172.21.15.143:0/3296154808 <== mon.0 v2:172.21.15.143:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 68 v68) v1 ==== 94+0+3711 (secure 0 0 0) 0x7f4d7c054ef0 con 0x7f4d8415d580 2023-11-26T07:25:24.578 INFO:teuthology.orchestra.run.smithi143.stdout: 2023-11-26T07:25:24.578 INFO:teuthology.orchestra.run.smithi143.stdout:{"epoch":68,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6629,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.188:6834/1327398471","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6832","nonce":1327398471},{"type":"v1","addr":"172.21.15.188:6834","nonce":1327398471}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":60},{"gid":7453,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.188:6835/1640976426","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6833","nonce":1640976426},{"type":"v1","addr":"172.21.15.188:6835","nonce":1640976426}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":60},{"gid":7941,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.143:6834/2855111671","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6832","nonce":2855111671},{"type":"v1","addr":"172.21.15.143:6834","nonce":2855111671}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":60},{"gid":8424,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.143:6835/2771527063","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6833","nonce":2771527063},{"type":"v1","addr":"172.21.15.143:6835","nonce":2771527063}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":68}],"filesystems":[]} 2023-11-26T07:25:24.578 INFO:teuthology.orchestra.run.smithi143.stderr:dumped fsmap epoch 68 2023-11-26T07:25:24.579 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:25:24.578+0000 7f4d8932e640 1 -- 172.21.15.143:0/3296154808 >> [v2:172.21.15.188:6836/3401128553,v1:172.21.15.188:6837/3401128553] conn(0x7f4d64066740 msgr2=0x7f4d64068c00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:25:24.579 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:25:24.578+0000 7f4d8932e640 1 --2- 172.21.15.143:0/3296154808 >> [v2:172.21.15.188:6836/3401128553,v1:172.21.15.188:6837/3401128553] conn(0x7f4d64066740 0x7f4d64068c00 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7f4d740ae000 tx=0x7f4d740a7070 comp rx=0 tx=0).stop 2023-11-26T07:25:24.580 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:25:24.578+0000 7f4d8932e640 1 -- 172.21.15.143:0/3296154808 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f4d8415d580 msgr2=0x7f4d840b86e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:25:24.580 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:25:24.578+0000 7f4d8932e640 1 --2- 172.21.15.143:0/3296154808 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f4d8415d580 0x7f4d840b86e0 secure :-1 s=READY pgs=1500 cs=0 l=1 rev1=1 crypto rx=0x7f4d7c00d5d0 tx=0x7f4d7c00da90 comp rx=0 tx=0).stop 2023-11-26T07:25:24.580 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:25:24.578+0000 7f4d8932e640 1 -- 172.21.15.143:0/3296154808 shutdown_connections 2023-11-26T07:25:24.580 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:25:24.578+0000 7f4d8932e640 1 --2- 172.21.15.143:0/3296154808 >> [v2:172.21.15.188:6836/3401128553,v1:172.21.15.188:6837/3401128553] conn(0x7f4d64066740 0x7f4d64068c00 unknown :-1 s=CLOSED pgs=206 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:25:24.580 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:25:24.578+0000 7f4d8932e640 1 --2- 172.21.15.143:0/3296154808 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f4d8415d580 0x7f4d840b86e0 unknown :-1 s=CLOSED pgs=1500 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:25:24.580 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:25:24.578+0000 7f4d8932e640 1 --2- 172.21.15.143:0/3296154808 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f4d840d3e60 0x7f4d84154840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:25:24.580 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:25:24.578+0000 7f4d8932e640 1 --2- 172.21.15.143:0/3296154808 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f4d840d1f00 0x7f4d84156500 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:25:24.580 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:25:24.578+0000 7f4d8932e640 1 -- 172.21.15.143:0/3296154808 >> 172.21.15.143:0/3296154808 conn(0x7f4d8409c550 msgr2=0x7f4d840cf750 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:25:24.581 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:25:24.578+0000 7f4d8932e640 1 -- 172.21.15.143:0/3296154808 shutdown_connections 2023-11-26T07:25:24.581 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-26T07:25:24.578+0000 7f4d8932e640 1 -- 172.21.15.143:0/3296154808 wait complete. 2023-11-26T07:25:24.598 DEBUG:tasks.cephfs.filesystem:already dead... 2023-11-26T07:25:24.599 INFO:teuthology.misc:Shutting down mds daemons... 2023-11-26T07:25:24.599 DEBUG:tasks.ceph.mds.a:waiting for process to exit 2023-11-26T07:25:24.599 INFO:teuthology.orchestra.run:waiting for 300 2023-11-26T07:25:24.609 INFO:tasks.ceph.mds.a:Stopped 2023-11-26T07:25:24.610 DEBUG:tasks.ceph.mds.c:waiting for process to exit 2023-11-26T07:25:24.610 INFO:teuthology.orchestra.run:waiting for 300 2023-11-26T07:25:24.668 INFO:tasks.ceph.mds.c:Stopped 2023-11-26T07:25:24.668 DEBUG:tasks.ceph.mds.b:waiting for process to exit 2023-11-26T07:25:24.668 INFO:teuthology.orchestra.run:waiting for 300 2023-11-26T07:25:24.684 INFO:tasks.ceph.mds.b:Stopped 2023-11-26T07:25:24.684 DEBUG:tasks.ceph.mds.d:waiting for process to exit 2023-11-26T07:25:24.684 INFO:teuthology.orchestra.run:waiting for 300 2023-11-26T07:25:24.744 INFO:tasks.ceph.mds.d:Stopped 2023-11-26T07:25:24.744 INFO:teuthology.misc:Shutting down osd daemons... 2023-11-26T07:25:24.745 DEBUG:tasks.ceph.osd.0:waiting for process to exit 2023-11-26T07:25:24.745 INFO:teuthology.orchestra.run:waiting for 300 2023-11-26T07:25:24.805 INFO:tasks.ceph.osd.0:Stopped 2023-11-26T07:25:24.805 DEBUG:tasks.ceph.osd.1:waiting for process to exit 2023-11-26T07:25:24.805 INFO:teuthology.orchestra.run:waiting for 300 2023-11-26T07:25:24.904 INFO:tasks.ceph.osd.1:Stopped 2023-11-26T07:25:24.904 DEBUG:tasks.ceph.osd.2:waiting for process to exit 2023-11-26T07:25:24.904 INFO:teuthology.orchestra.run:waiting for 300 2023-11-26T07:25:24.976 INFO:tasks.ceph.osd.2:Stopped 2023-11-26T07:25:24.976 DEBUG:tasks.ceph.osd.3:waiting for process to exit 2023-11-26T07:25:24.976 INFO:teuthology.orchestra.run:waiting for 300 2023-11-26T07:25:25.040 INFO:tasks.ceph.osd.3:Stopped 2023-11-26T07:25:25.040 DEBUG:tasks.ceph.osd.4:waiting for process to exit 2023-11-26T07:25:25.040 INFO:teuthology.orchestra.run:waiting for 300 2023-11-26T07:25:25.129 INFO:tasks.ceph.osd.4:Stopped 2023-11-26T07:25:25.129 DEBUG:tasks.ceph.osd.5:waiting for process to exit 2023-11-26T07:25:25.129 INFO:teuthology.orchestra.run:waiting for 300 2023-11-26T07:25:25.204 INFO:tasks.ceph.osd.5:Stopped 2023-11-26T07:25:25.205 DEBUG:tasks.ceph.osd.6:waiting for process to exit 2023-11-26T07:25:25.205 INFO:teuthology.orchestra.run:waiting for 300 2023-11-26T07:25:25.275 INFO:tasks.ceph.osd.6:Stopped 2023-11-26T07:25:25.276 DEBUG:tasks.ceph.osd.7:waiting for process to exit 2023-11-26T07:25:25.276 INFO:teuthology.orchestra.run:waiting for 300 2023-11-26T07:25:29.441 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.7 is failed for ~0s 2023-11-26T07:25:31.378 INFO:tasks.ceph.osd.7:Stopped 2023-11-26T07:25:31.378 INFO:teuthology.misc:Shutting down mgr daemons... 2023-11-26T07:25:31.379 DEBUG:tasks.ceph.mgr.y:waiting for process to exit 2023-11-26T07:25:31.379 INFO:teuthology.orchestra.run:waiting for 300 2023-11-26T07:25:31.435 INFO:tasks.ceph.mgr.y:Stopped 2023-11-26T07:25:31.436 DEBUG:tasks.ceph.mgr.x:waiting for process to exit 2023-11-26T07:25:31.436 INFO:teuthology.orchestra.run:waiting for 300 2023-11-26T07:25:31.495 INFO:tasks.ceph.mgr.x:Stopped 2023-11-26T07:25:31.495 INFO:teuthology.misc:Shutting down mon daemons... 2023-11-26T07:25:31.495 DEBUG:tasks.ceph.mon.a:waiting for process to exit 2023-11-26T07:25:31.495 INFO:teuthology.orchestra.run:waiting for 300 2023-11-26T07:25:31.517 INFO:tasks.ceph.mon.a:Stopped 2023-11-26T07:25:31.517 DEBUG:tasks.ceph.mon.b:waiting for process to exit 2023-11-26T07:25:31.517 INFO:teuthology.orchestra.run:waiting for 300 2023-11-26T07:25:31.534 INFO:tasks.ceph.mon.b:Stopped 2023-11-26T07:25:31.534 DEBUG:tasks.ceph.mon.c:waiting for process to exit 2023-11-26T07:25:31.534 INFO:teuthology.orchestra.run:waiting for 300 2023-11-26T07:25:31.595 INFO:tasks.ceph.mon.c:Stopped 2023-11-26T07:25:31.595 INFO:tasks.ceph:Checking cluster log for badness... 2023-11-26T07:25:31.595 DEBUG:teuthology.orchestra.run.smithi143:> sudo egrep '\[ERR\]|\[WRN\]|\[SEC\]' /var/log/ceph/ceph.log | egrep -v '\(MDS_ALL_DOWN\)' | egrep -v '\(MDS_UP_LESS_THAN_MAX\)' | egrep -v 'overall HEALTH_' | egrep -v '\(FS_DEGRADED\)' | egrep -v '\(MDS_FAILED\)' | egrep -v '\(MDS_DEGRADED\)' | egrep -v '\(FS_WITH_FAILED_MDS\)' | egrep -v '\(MDS_DAMAGE\)' | egrep -v '\(MDS_ALL_DOWN\)' | egrep -v '\(MDS_UP_LESS_THAN_MAX\)' | egrep -v '\(FS_INLINE_DATA_DEPRECATED\)' | egrep -v '\(POOL_APP_NOT_ENABLED\)' | egrep -v 'overall HEALTH_' | egrep -v '\(OSD_DOWN\)' | egrep -v '\(OSD_' | egrep -v 'but it is still running' | egrep -v 'is not responding' | egrep -v 'OSD full dropping all updates' | egrep -v 'OSD near full' | egrep -v 'pausewr flag' | egrep -v 'failsafe engaged, dropping updates' | egrep -v 'failsafe disengaged, no longer dropping' | egrep -v 'is full \(reached quota' | egrep -v POOL_FULL | egrep -v POOL_BACKFILLFULL | head -n 1 2023-11-26T07:25:31.629 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-0 on ubuntu@smithi143.front.sepia.ceph.com 2023-11-26T07:25:31.630 DEBUG:teuthology.orchestra.run.smithi143:> sync && sudo umount -f /var/lib/ceph/osd/ceph-0 2023-11-26T07:25:32.077 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-1 on ubuntu@smithi143.front.sepia.ceph.com 2023-11-26T07:25:32.077 DEBUG:teuthology.orchestra.run.smithi143:> sync && sudo umount -f /var/lib/ceph/osd/ceph-1 2023-11-26T07:25:32.271 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-2 on ubuntu@smithi143.front.sepia.ceph.com 2023-11-26T07:25:32.272 DEBUG:teuthology.orchestra.run.smithi143:> sync && sudo umount -f /var/lib/ceph/osd/ceph-2 2023-11-26T07:25:32.440 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-3 on ubuntu@smithi143.front.sepia.ceph.com 2023-11-26T07:25:32.440 DEBUG:teuthology.orchestra.run.smithi143:> sync && sudo umount -f /var/lib/ceph/osd/ceph-3 2023-11-26T07:25:32.627 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-4 on ubuntu@smithi188.front.sepia.ceph.com 2023-11-26T07:25:32.627 DEBUG:teuthology.orchestra.run.smithi188:> sync && sudo umount -f /var/lib/ceph/osd/ceph-4 2023-11-26T07:25:32.924 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-5 on ubuntu@smithi188.front.sepia.ceph.com 2023-11-26T07:25:32.924 DEBUG:teuthology.orchestra.run.smithi188:> sync && sudo umount -f /var/lib/ceph/osd/ceph-5 2023-11-26T07:25:33.100 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-6 on ubuntu@smithi188.front.sepia.ceph.com 2023-11-26T07:25:33.100 DEBUG:teuthology.orchestra.run.smithi188:> sync && sudo umount -f /var/lib/ceph/osd/ceph-6 2023-11-26T07:25:33.327 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-7 on ubuntu@smithi188.front.sepia.ceph.com 2023-11-26T07:25:33.328 DEBUG:teuthology.orchestra.run.smithi188:> sync && sudo umount -f /var/lib/ceph/osd/ceph-7 2023-11-26T07:25:33.514 INFO:tasks.ceph:Archiving mon data... 2023-11-26T07:25:33.516 DEBUG:teuthology.misc:Transferring archived files from smithi143:/var/lib/ceph/mon/ceph-a to /home/teuthworker/archive/rishabh-2023-11-26_06:51:12-fs:volumes-rishabh-mgr-vol-misc-testing-default-smithi/7467343/data/mon.a.tgz 2023-11-26T07:25:33.516 DEBUG:teuthology.orchestra.run.smithi143:> mktemp 2023-11-26T07:25:33.525 INFO:teuthology.orchestra.run.smithi143.stdout:/tmp/tmp.PZ2QH33KNm 2023-11-26T07:25:33.525 DEBUG:teuthology.orchestra.run.smithi143:> sudo tar cz -f - -C /var/lib/ceph/mon/ceph-a -- . > /tmp/tmp.PZ2QH33KNm 2023-11-26T07:25:33.720 DEBUG:teuthology.orchestra.run.smithi143:> sudo chmod 0666 /tmp/tmp.PZ2QH33KNm 2023-11-26T07:25:33.796 DEBUG:teuthology.orchestra.remote:smithi143:/tmp/tmp.PZ2QH33KNm is 770KB 2023-11-26T07:25:33.862 DEBUG:teuthology.orchestra.run.smithi143:> rm -fr /tmp/tmp.PZ2QH33KNm 2023-11-26T07:25:33.871 DEBUG:teuthology.misc:Transferring archived files from smithi188:/var/lib/ceph/mon/ceph-b to /home/teuthworker/archive/rishabh-2023-11-26_06:51:12-fs:volumes-rishabh-mgr-vol-misc-testing-default-smithi/7467343/data/mon.b.tgz 2023-11-26T07:25:33.871 DEBUG:teuthology.orchestra.run.smithi188:> mktemp 2023-11-26T07:25:33.880 INFO:teuthology.orchestra.run.smithi188.stdout:/tmp/tmp.FEIjFkk4RC 2023-11-26T07:25:33.880 DEBUG:teuthology.orchestra.run.smithi188:> sudo tar cz -f - -C /var/lib/ceph/mon/ceph-b -- . > /tmp/tmp.FEIjFkk4RC 2023-11-26T07:25:34.086 DEBUG:teuthology.orchestra.run.smithi188:> sudo chmod 0666 /tmp/tmp.FEIjFkk4RC 2023-11-26T07:25:34.159 DEBUG:teuthology.orchestra.remote:smithi188:/tmp/tmp.FEIjFkk4RC is 811KB 2023-11-26T07:25:34.229 DEBUG:teuthology.orchestra.run.smithi188:> rm -fr /tmp/tmp.FEIjFkk4RC 2023-11-26T07:25:34.238 DEBUG:teuthology.misc:Transferring archived files from smithi188:/var/lib/ceph/mon/ceph-c to /home/teuthworker/archive/rishabh-2023-11-26_06:51:12-fs:volumes-rishabh-mgr-vol-misc-testing-default-smithi/7467343/data/mon.c.tgz 2023-11-26T07:25:34.238 DEBUG:teuthology.orchestra.run.smithi188:> mktemp 2023-11-26T07:25:34.287 INFO:teuthology.orchestra.run.smithi188.stdout:/tmp/tmp.b3gvIDWPdL 2023-11-26T07:25:34.287 DEBUG:teuthology.orchestra.run.smithi188:> sudo tar cz -f - -C /var/lib/ceph/mon/ceph-c -- . > /tmp/tmp.b3gvIDWPdL 2023-11-26T07:25:34.442 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.7 has been restored 2023-11-26T07:25:34.488 DEBUG:teuthology.orchestra.run.smithi188:> sudo chmod 0666 /tmp/tmp.b3gvIDWPdL 2023-11-26T07:25:34.563 DEBUG:teuthology.orchestra.remote:smithi188:/tmp/tmp.b3gvIDWPdL is 811KB 2023-11-26T07:25:34.633 DEBUG:teuthology.orchestra.run.smithi188:> rm -fr /tmp/tmp.b3gvIDWPdL 2023-11-26T07:25:34.642 INFO:tasks.ceph:Cleaning ceph cluster... 2023-11-26T07:25:34.642 DEBUG:teuthology.orchestra.run.smithi143:> sudo rm -rf -- /etc/ceph/ceph.conf /etc/ceph/ceph.keyring /home/ubuntu/cephtest/ceph.data /home/ubuntu/cephtest/ceph.monmap /home/ubuntu/cephtest/../*.pid 2023-11-26T07:25:34.645 DEBUG:teuthology.orchestra.run.smithi188:> sudo rm -rf -- /etc/ceph/ceph.conf /etc/ceph/ceph.keyring /home/ubuntu/cephtest/ceph.data /home/ubuntu/cephtest/ceph.monmap /home/ubuntu/cephtest/../*.pid 2023-11-26T07:25:34.702 INFO:tasks.ceph:Checking for errors in any valgrind logs... 2023-11-26T07:25:34.703 DEBUG:teuthology.orchestra.run.smithi143:> sudo zgrep '' /var/log/ceph/valgrind/* /dev/null | sort | uniq 2023-11-26T07:25:34.711 DEBUG:teuthology.orchestra.run.smithi188:> sudo zgrep '' /var/log/ceph/valgrind/* /dev/null | sort | uniq 2023-11-26T07:25:34.747 INFO:teuthology.orchestra.run.smithi143.stderr:gzip: /var/log/ceph/valgrind/*.gz: No such file or directory 2023-11-26T07:25:34.776 INFO:teuthology.orchestra.run.smithi188.stderr:gzip: /var/log/ceph/valgrind/*.gz: No such file or directory 2023-11-26T07:25:34.782 INFO:tasks.ceph:Archiving crash dumps... 2023-11-26T07:25:34.784 DEBUG:teuthology.misc:Transferring archived files from smithi143:/var/lib/ceph/crash to /home/teuthworker/archive/rishabh-2023-11-26_06:51:12-fs:volumes-rishabh-mgr-vol-misc-testing-default-smithi/7467343/remote/smithi143/crash 2023-11-26T07:25:34.785 DEBUG:teuthology.orchestra.run.smithi143:> sudo tar c -f - -C /var/lib/ceph/crash -- . 2023-11-26T07:25:34.820 DEBUG:teuthology.misc:Transferring archived files from smithi188:/var/lib/ceph/crash to /home/teuthworker/archive/rishabh-2023-11-26_06:51:12-fs:volumes-rishabh-mgr-vol-misc-testing-default-smithi/7467343/remote/smithi188/crash 2023-11-26T07:25:34.821 DEBUG:teuthology.orchestra.run.smithi188:> sudo tar c -f - -C /var/lib/ceph/crash -- . 2023-11-26T07:25:34.845 INFO:tasks.ceph:Shutting down logrotate 2023-11-26T07:25:34.845 DEBUG:teuthology.orchestra.run.smithi143:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-26T07:25:34.867 DEBUG:teuthology.orchestra.run.smithi188:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-26T07:25:34.941 DEBUG:teuthology.orchestra.run.smithi143:> sudo rm /etc/logrotate.d/ceph-test.conf 2023-11-26T07:25:34.962 DEBUG:teuthology.orchestra.run.smithi188:> sudo rm /etc/logrotate.d/ceph-test.conf 2023-11-26T07:25:34.981 INFO:tasks.ceph:Compressing logs... 2023-11-26T07:25:34.981 DEBUG:teuthology.orchestra.run.smithi143:> time sudo find /var/log/ceph -name '*.log' -print0 | sudo xargs --max-args=1 --max-procs=0 --verbose -0 --no-run-if-empty -- gzip -5 --verbose -- 2023-11-26T07:25:35.007 DEBUG:teuthology.orchestra.run.smithi188:> time sudo find /var/log/ceph -name '*.log' -print0 | sudo xargs --max-args=1 --max-procs=0 --verbose -0 --no-run-if-empty -- gzip -5 --verbose -- 2023-11-26T07:25:35.040 INFO:teuthology.orchestra.run.smithi143.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.0.50877.log 2023-11-26T07:25:35.041 INFO:teuthology.orchestra.run.smithi143.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54946.log 2023-11-26T07:25:35.042 INFO:teuthology.orchestra.run.smithi143.stderr:/var/log/ceph/ceph-client.0.50877.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.56065.log 2023-11-26T07:25:35.042 INFO:teuthology.orchestra.run.smithi143.stderr:/var/log/ceph/ceph-client.admin.54946.log: 89.2% -- replaced with /var/log/ceph/ceph-client.0.50877.log.gz 2023-11-26T07:25:35.043 INFO:teuthology.orchestra.run.smithi143.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44282.log 2023-11-26T07:25:35.043 INFO:teuthology.orchestra.run.smithi143.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.54946.log.gz 2023-11-26T07:25:35.044 INFO:teuthology.orchestra.run.smithi143.stderr:/var/log/ceph/ceph-client.admin.56065.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.60899.log 2023-11-26T07:25:35.044 INFO:teuthology.orchestra.run.smithi143.stderr: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.56065.log.gz 2023-11-26T07:25:35.045 INFO:teuthology.orchestra.run.smithi143.stderr:/var/log/ceph/ceph-client.admin.44282.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46159.log 2023-11-26T07:25:35.045 INFO:teuthology.orchestra.run.smithi143.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.44282.log.gz 2023-11-26T07:25:35.046 INFO:teuthology.orchestra.run.smithi143.stderr:/var/log/ceph/ceph-client.admin.60899.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46342.log 2023-11-26T07:25:35.046 INFO:teuthology.orchestra.run.smithi143.stderr: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.60899.log.gz 2023-11-26T07:25:35.047 INFO:teuthology.orchestra.run.smithi143.stderr:/var/log/ceph/ceph-client.admin.46159.log: gzip 88.7% -- replaced with /var/log/ceph/ceph-client.admin.46159.log.gz -5 2023-11-26T07:25:35.047 INFO:teuthology.orchestra.run.smithi143.stderr: --verbose -- /var/log/ceph/ceph-client.admin.53880.log 2023-11-26T07:25:35.048 INFO:teuthology.orchestra.run.smithi143.stderr:/var/log/ceph/ceph-client.admin.46342.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.56182.log 2023-11-26T07:25:35.048 INFO:teuthology.orchestra.run.smithi143.stderr: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.46342.log.gz 2023-11-26T07:25:35.049 INFO:teuthology.orchestra.run.smithi143.stderr:/var/log/ceph/ceph-client.admin.53880.log: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.53880.log.gz 2023-11-26T07:25:35.049 INFO:teuthology.orchestra.run.smithi143.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47111.log 2023-11-26T07:25:35.050 INFO:teuthology.orchestra.run.smithi143.stderr:/var/log/ceph/ceph-client.admin.56182.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54203.log 2023-11-26T07:25:35.050 INFO:teuthology.orchestra.run.smithi143.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.56182.log.gz 2023-11-26T07:25:35.051 INFO:teuthology.orchestra.run.smithi143.stderr:/var/log/ceph/ceph-client.admin.47111.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52445.log 2023-11-26T07:25:35.051 INFO:teuthology.orchestra.run.smithi143.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.47111.log.gz 2023-11-26T07:25:35.052 INFO:teuthology.orchestra.run.smithi188.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.1.45157.log 2023-11-26T07:25:35.053 INFO:teuthology.orchestra.run.smithi143.stderr:/var/log/ceph/ceph-client.admin.54203.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.55784.log 2023-11-26T07:25:35.053 INFO:teuthology.orchestra.run.smithi143.stderr: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.54203.log.gz 2023-11-26T07:25:35.054 INFO:teuthology.orchestra.run.smithi188.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.1.44390.log 2023-11-26T07:25:35.054 INFO:teuthology.orchestra.run.smithi143.stderr:/var/log/ceph/ceph-client.admin.52445.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.57624.log 2023-11-26T07:25:35.055 INFO:teuthology.orchestra.run.smithi143.stderr: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.52445.log.gz 2023-11-26T07:25:35.055 INFO:teuthology.orchestra.run.smithi143.stderr:/var/log/ceph/ceph-client.admin.55784.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46846.log 2023-11-26T07:25:35.055 INFO:teuthology.orchestra.run.smithi143.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.55784.log.gz 2023-11-26T07:25:35.056 INFO:teuthology.orchestra.run.smithi143.stderr:/var/log/ceph/ceph-client.admin.57624.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.60337.log 2023-11-26T07:25:35.056 INFO:teuthology.orchestra.run.smithi143.stderr: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.57624.log.gz 2023-11-26T07:25:35.057 INFO:teuthology.orchestra.run.smithi188.stderr:/var/log/ceph/ceph-client.1.45157.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38200.log 2023-11-26T07:25:35.057 INFO:teuthology.orchestra.run.smithi188.stderr:/var/log/ceph/ceph-client.1.44390.log: 90.1% -- replaced with /var/log/ceph/ceph-client.1.45157.log.gz 2023-11-26T07:25:35.057 INFO:teuthology.orchestra.run.smithi188.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38272.log 2023-11-26T07:25:35.057 INFO:teuthology.orchestra.run.smithi188.stderr:/var/log/ceph/ceph-client.admin.38200.log: 89.1% -- replaced with /var/log/ceph/ceph-client.1.44390.log.gz 2023-11-26T07:25:35.057 INFO:teuthology.orchestra.run.smithi188.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38176.log 2023-11-26T07:25:35.057 INFO:teuthology.orchestra.run.smithi188.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.38200.log.gz 2023-11-26T07:25:35.059 INFO:teuthology.orchestra.run.smithi188.stderr:/var/log/ceph/ceph-client.admin.38272.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.1.44878.log 2023-11-26T07:25:35.059 INFO:teuthology.orchestra.run.smithi188.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.38272.log.gz 2023-11-26T07:25:35.059 INFO:teuthology.orchestra.run.smithi188.stderr:/var/log/ceph/ceph-client.admin.38176.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.1.44712.log 2023-11-26T07:25:35.059 INFO:teuthology.orchestra.run.smithi188.stderr: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.38176.log.gz 2023-11-26T07:25:35.060 INFO:teuthology.orchestra.run.smithi188.stderr:/var/log/ceph/ceph-client.1.44878.log: gzip -5 --verbose -- /var/log/ceph/ceph-osd.7.log 2023-11-26T07:25:35.061 INFO:teuthology.orchestra.run.smithi143.stderr:/var/log/ceph/ceph-client.admin.46846.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52263.log 2023-11-26T07:25:35.061 INFO:teuthology.orchestra.run.smithi143.stderr: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.46846.log.gz 2023-11-26T07:25:35.061 INFO:teuthology.orchestra.run.smithi143.stderr:/var/log/ceph/ceph-client.admin.60337.log: 89.5% -- replaced with /var/log/ceph/ceph-client.admin.60337.log.gz 2023-11-26T07:25:35.061 INFO:teuthology.orchestra.run.smithi143.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54411.log 2023-11-26T07:25:35.061 INFO:teuthology.orchestra.run.smithi143.stderr:/var/log/ceph/ceph-client.admin.52263.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50288.log 2023-11-26T07:25:35.061 INFO:teuthology.orchestra.run.smithi143.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.52263.log.gz 2023-11-26T07:25:35.062 INFO:teuthology.orchestra.run.smithi143.stderr:/var/log/ceph/ceph-client.admin.54411.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51134.log 2023-11-26T07:25:35.062 INFO:teuthology.orchestra.run.smithi143.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.54411.log.gz 2023-11-26T07:25:35.062 INFO:teuthology.orchestra.run.smithi143.stderr:/var/log/ceph/ceph-client.admin.50288.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51108.log 2023-11-26T07:25:35.062 INFO:teuthology.orchestra.run.smithi143.stderr: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.50288.log.gz 2023-11-26T07:25:35.064 INFO:teuthology.orchestra.run.smithi143.stderr:/var/log/ceph/ceph-client.admin.51134.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48178.log 2023-11-26T07:25:35.065 INFO:teuthology.orchestra.run.smithi143.stderr: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.51134.log.gz 2023-11-26T07:25:35.065 INFO:teuthology.orchestra.run.smithi143.stderr:/var/log/ceph/ceph-client.admin.51108.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46368.log 2023-11-26T07:25:35.065 INFO:teuthology.orchestra.run.smithi143.stderr: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.51108.log.gz 2023-11-26T07:25:35.065 INFO:teuthology.orchestra.run.smithi143.stderr:/var/log/ceph/ceph-client.admin.48178.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54281.log 2023-11-26T07:25:35.065 INFO:teuthology.orchestra.run.smithi143.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.48178.log.gz 2023-11-26T07:25:35.065 INFO:teuthology.orchestra.run.smithi143.stderr:/var/log/ceph/ceph-client.admin.46368.log: gzip -5 --verbose -- /var/log/ceph/ceph-mgr.y.log 2023-11-26T07:25:35.065 INFO:teuthology.orchestra.run.smithi143.stderr: 89.5% -- replaced with /var/log/ceph/ceph-client.admin.46368.log.gz 2023-11-26T07:25:35.066 INFO:teuthology.orchestra.run.smithi188.stderr: 90.3% -- replaced with /var/log/ceph/ceph-client.1.44878.log.gz 2023-11-26T07:25:35.066 INFO:teuthology.orchestra.run.smithi188.stderr:/var/log/ceph/ceph-client.1.44712.log: gzip -5 --verbose -- /var/log/ceph/ceph-osd.6.log 2023-11-26T07:25:35.069 INFO:teuthology.orchestra.run.smithi188.stderr:/var/log/ceph/ceph-osd.7.log: 89.9% -- replaced with /var/log/ceph/ceph-client.1.44712.log.gz 2023-11-26T07:25:35.069 INFO:teuthology.orchestra.run.smithi188.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38248.log 2023-11-26T07:25:35.069 INFO:teuthology.orchestra.run.smithi188.stderr:/var/log/ceph/ceph-osd.6.log: gzip -5 --verbose -- /var/log/ceph/ceph-osd.5.log 2023-11-26T07:25:35.069 INFO:teuthology.orchestra.run.smithi188.stderr:/var/log/ceph/ceph-client.admin.38248.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.1.44538.log 2023-11-26T07:25:35.069 INFO:teuthology.orchestra.run.smithi188.stderr: 87.1% -- replaced with /var/log/ceph/ceph-client.admin.38248.log.gz 2023-11-26T07:25:35.069 INFO:teuthology.orchestra.run.smithi188.stderr:/var/log/ceph/ceph-osd.5.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.3.44220.log 2023-11-26T07:25:35.069 INFO:teuthology.orchestra.run.smithi188.stderr:/var/log/ceph/ceph-client.1.44538.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.1.45326.log 2023-11-26T07:25:35.070 INFO:teuthology.orchestra.run.smithi188.stderr: 90.2% -- replaced with /var/log/ceph/ceph-client.1.44538.log.gz 2023-11-26T07:25:35.073 INFO:teuthology.orchestra.run.smithi188.stderr:/var/log/ceph/ceph-client.3.44220.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38224.log 2023-11-26T07:25:35.073 INFO:teuthology.orchestra.run.smithi188.stderr: 90.7% -- replaced with /var/log/ceph/ceph-client.3.44220.log.gz 2023-11-26T07:25:35.073 INFO:teuthology.orchestra.run.smithi188.stderr:/var/log/ceph/ceph-client.1.45326.log: gzip -5 --verbose -- /var/log/ceph/ceph-mgr.x.log 2023-11-26T07:25:35.073 INFO:teuthology.orchestra.run.smithi188.stderr:/var/log/ceph/ceph-client.admin.38224.log: 87.0% -- replaced with /var/log/ceph/ceph-client.admin.38224.log.gz 2023-11-26T07:25:35.073 INFO:teuthology.orchestra.run.smithi188.stderr: 91.4% -- replaced with /var/log/ceph/ceph-client.1.45326.log.gz 2023-11-26T07:25:35.073 INFO:teuthology.orchestra.run.smithi188.stderr:gzip -5 --verbose -- /var/log/ceph/ceph.log 2023-11-26T07:25:35.073 INFO:teuthology.orchestra.run.smithi188.stderr:/var/log/ceph/ceph-mgr.x.log: gzip -5 --verbose -- /var/log/ceph/ceph-mon.c.log 2023-11-26T07:25:35.073 INFO:teuthology.orchestra.run.smithi188.stderr:/var/log/ceph/ceph.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38296.log 2023-11-26T07:25:35.076 INFO:teuthology.orchestra.run.smithi143.stderr:/var/log/ceph/ceph-client.admin.54281.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49102.log 2023-11-26T07:25:35.076 INFO:teuthology.orchestra.run.smithi143.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.54281.log.gz 2023-11-26T07:25:35.076 INFO:teuthology.orchestra.run.smithi143.stderr:/var/log/ceph/ceph-mgr.y.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51686.log 2023-11-26T07:25:35.076 INFO:teuthology.orchestra.run.smithi143.stderr:/var/log/ceph/ceph-client.admin.49102.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.55706.log 2023-11-26T07:25:35.076 INFO:teuthology.orchestra.run.smithi143.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.49102.log.gz 2023-11-26T07:25:35.076 INFO:teuthology.orchestra.run.smithi143.stderr:/var/log/ceph/ceph-client.admin.51686.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.55836.log 2023-11-26T07:25:35.077 INFO:teuthology.orchestra.run.smithi143.stderr: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.51686.log.gz 2023-11-26T07:25:35.077 INFO:teuthology.orchestra.run.smithi143.stderr:/var/log/ceph/ceph-client.admin.55706.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46898.log 2023-11-26T07:25:35.077 INFO:teuthology.orchestra.run.smithi143.stderr: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.55706.log.gz 2023-11-26T07:25:35.077 INFO:teuthology.orchestra.run.smithi143.stderr:/var/log/ceph/ceph-client.admin.55836.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46630.log 2023-11-26T07:25:35.077 INFO:teuthology.orchestra.run.smithi143.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.55836.log.gz 2023-11-26T07:25:35.077 INFO:teuthology.orchestra.run.smithi143.stderr:/var/log/ceph/ceph-client.admin.46898.log: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.46898.log.gz 2023-11-26T07:25:35.077 INFO:teuthology.orchestra.run.smithi143.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53990.log 2023-11-26T07:25:35.077 INFO:teuthology.orchestra.run.smithi143.stderr:/var/log/ceph/ceph-client.admin.46630.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.57838.log 2023-11-26T07:25:35.078 INFO:teuthology.orchestra.run.smithi143.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.46630.log.gz 2023-11-26T07:25:35.078 INFO:teuthology.orchestra.run.smithi143.stderr:/var/log/ceph/ceph-client.admin.53990.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50694.log 2023-11-26T07:25:35.078 INFO:teuthology.orchestra.run.smithi143.stderr: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.53990.log.gz 2023-11-26T07:25:35.078 INFO:teuthology.orchestra.run.smithi143.stderr:/var/log/ceph/ceph-client.admin.57838.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45510.log 2023-11-26T07:25:35.078 INFO:teuthology.orchestra.run.smithi143.stderr: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.57838.log.gz 2023-11-26T07:25:35.078 INFO:teuthology.orchestra.run.smithi143.stderr:/var/log/ceph/ceph-client.admin.50694.log: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.50694.log.gz 2023-11-26T07:25:35.078 INFO:teuthology.orchestra.run.smithi143.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45536.log 2023-11-26T07:25:35.079 INFO:teuthology.orchestra.run.smithi188.stderr:/var/log/ceph/ceph-mon.c.log: gzip -5 --verbose -- /var/log/ceph/ceph.audit.log 2023-11-26T07:25:35.079 INFO:teuthology.orchestra.run.smithi188.stderr:/var/log/ceph/ceph-client.admin.38296.log: 92.4% -- replaced with /var/log/ceph/ceph.log.gz 2023-11-26T07:25:35.080 INFO:teuthology.orchestra.run.smithi188.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.38296.log.gz 2023-11-26T07:25:35.080 INFO:teuthology.orchestra.run.smithi188.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38152.log 2023-11-26T07:25:35.080 INFO:teuthology.orchestra.run.smithi188.stderr:/var/log/ceph/ceph.audit.log: /var/log/ceph/ceph-client.admin.38152.log: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.38152.log.gz 2023-11-26T07:25:35.080 INFO:teuthology.orchestra.run.smithi188.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-mds.d.log 2023-11-26T07:25:35.081 INFO:teuthology.orchestra.run.smithi188.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38320.log 2023-11-26T07:25:35.081 INFO:teuthology.orchestra.run.smithi188.stderr:/var/log/ceph/ceph-mds.d.log: 93.5% -- replaced with /var/log/ceph/ceph.audit.log.gz 2023-11-26T07:25:35.083 INFO:teuthology.orchestra.run.smithi143.stderr:/var/log/ceph/ceph-client.admin.45510.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46741.log 2023-11-26T07:25:35.084 INFO:teuthology.orchestra.run.smithi143.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.45510.log.gz 2023-11-26T07:25:35.084 INFO:teuthology.orchestra.run.smithi143.stderr:/var/log/ceph/ceph-client.admin.45536.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.59327.log 2023-11-26T07:25:35.084 INFO:teuthology.orchestra.run.smithi143.stderr: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.45536.log.gz 2023-11-26T07:25:35.084 INFO:teuthology.orchestra.run.smithi143.stderr:/var/log/ceph/ceph-client.admin.46741.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.55128.log 2023-11-26T07:25:35.084 INFO:teuthology.orchestra.run.smithi143.stderr: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.46741.log.gz 2023-11-26T07:25:35.084 INFO:teuthology.orchestra.run.smithi143.stderr:/var/log/ceph/ceph-client.admin.59327.log: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.59327.log.gz 2023-11-26T07:25:35.084 INFO:teuthology.orchestra.run.smithi143.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.55888.log 2023-11-26T07:25:35.085 INFO:teuthology.orchestra.run.smithi143.stderr:/var/log/ceph/ceph-client.admin.55128.log: gzip 87.6% -5 -- replaced with /var/log/ceph/ceph-client.admin.55128.log.gz --verbose 2023-11-26T07:25:35.085 INFO:teuthology.orchestra.run.smithi143.stderr: -- /var/log/ceph/ceph-client.admin.53483.log 2023-11-26T07:25:35.085 INFO:teuthology.orchestra.run.smithi143.stderr:/var/log/ceph/ceph-client.admin.55888.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47400.log 2023-11-26T07:25:35.085 INFO:teuthology.orchestra.run.smithi143.stderr: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.55888.log.gz 2023-11-26T07:25:35.085 INFO:teuthology.orchestra.run.smithi143.stderr:/var/log/ceph/ceph-client.admin.53483.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54071.log 2023-11-26T07:25:35.085 INFO:teuthology.orchestra.run.smithi143.stderr: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.53483.log.gz 2023-11-26T07:25:35.085 INFO:teuthology.orchestra.run.smithi143.stderr:/var/log/ceph/ceph-client.admin.47400.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49692.log 2023-11-26T07:25:35.085 INFO:teuthology.orchestra.run.smithi143.stderr: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.47400.log.gz 2023-11-26T07:25:35.087 INFO:teuthology.orchestra.run.smithi188.stderr:/var/log/ceph/ceph-client.admin.38320.log: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.38320.log.gz 2023-11-26T07:25:35.088 INFO:teuthology.orchestra.run.smithi188.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-mds.b.log 2023-11-26T07:25:35.088 INFO:teuthology.orchestra.run.smithi188.stderr: 93.1% -- replaced with /var/log/ceph/ceph-mds.d.log.gz 2023-11-26T07:25:35.088 INFO:teuthology.orchestra.run.smithi188.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-mon.b.log 2023-11-26T07:25:35.088 INFO:teuthology.orchestra.run.smithi188.stderr:/var/log/ceph/ceph-mds.b.log: gzip -5 --verbose -- /var/log/ceph/ceph-osd.4.log 2023-11-26T07:25:35.089 INFO:teuthology.orchestra.run.smithi143.stderr:/var/log/ceph/ceph-client.admin.54071.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.57101.log 2023-11-26T07:25:35.089 INFO:teuthology.orchestra.run.smithi143.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.54071.log.gz 2023-11-26T07:25:35.089 INFO:teuthology.orchestra.run.smithi143.stderr:/var/log/ceph/ceph-client.admin.49692.log: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.49692.log.gz 2023-11-26T07:25:35.089 INFO:teuthology.orchestra.run.smithi143.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49770.log 2023-11-26T07:25:35.090 INFO:teuthology.orchestra.run.smithi143.stderr:/var/log/ceph/ceph-client.admin.57101.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.58767.log 2023-11-26T07:25:35.090 INFO:teuthology.orchestra.run.smithi143.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.57101.log.gz 2023-11-26T07:25:35.090 INFO:teuthology.orchestra.run.smithi143.stderr:/var/log/ceph/ceph-client.admin.49770.log: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.49770.log.gz 2023-11-26T07:25:35.090 INFO:teuthology.orchestra.run.smithi143.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49429.log 2023-11-26T07:25:35.090 INFO:teuthology.orchestra.run.smithi143.stderr:/var/log/ceph/ceph-client.admin.58767.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54858.log 2023-11-26T07:25:35.090 INFO:teuthology.orchestra.run.smithi143.stderr: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.58767.log.gz 2023-11-26T07:25:35.091 INFO:teuthology.orchestra.run.smithi143.stderr:/var/log/ceph/ceph-client.admin.49429.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50977.log 88.2% 2023-11-26T07:25:35.092 INFO:teuthology.orchestra.run.smithi143.stderr: -- replaced with /var/log/ceph/ceph-client.admin.49429.log.gz 2023-11-26T07:25:35.092 INFO:teuthology.orchestra.run.smithi143.stderr:/var/log/ceph/ceph-client.admin.54858.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51839.log 2023-11-26T07:25:35.092 INFO:teuthology.orchestra.run.smithi143.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.54858.log.gz 2023-11-26T07:25:35.092 INFO:teuthology.orchestra.run.smithi143.stderr:/var/log/ceph/ceph-client.admin.50977.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.56009.log 2023-11-26T07:25:35.092 INFO:teuthology.orchestra.run.smithi143.stderr: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.50977.log.gz 2023-11-26T07:25:35.093 INFO:teuthology.orchestra.run.smithi188.stderr:/var/log/ceph/ceph-mon.b.log: /var/log/ceph/ceph-osd.4.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.1.44114.log 2023-11-26T07:25:35.093 INFO:teuthology.orchestra.run.smithi143.stderr:/var/log/ceph/ceph-client.admin.51839.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45614.log 2023-11-26T07:25:35.094 INFO:teuthology.orchestra.run.smithi143.stderr: 89.4% -- replaced with /var/log/ceph/ceph-client.admin.51839.log.gz 2023-11-26T07:25:35.094 INFO:teuthology.orchestra.run.smithi143.stderr:/var/log/ceph/ceph-client.admin.56009.log: gzip -5 --verbose -- 88.8% /var/log/ceph/ceph-client.admin.54463.log -- replaced with /var/log/ceph/ceph-client.admin.56009.log.gz 2023-11-26T07:25:35.094 INFO:teuthology.orchestra.run.smithi143.stderr: 2023-11-26T07:25:35.095 INFO:teuthology.orchestra.run.smithi143.stderr:/var/log/ceph/ceph-client.admin.45614.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48283.log 2023-11-26T07:25:35.095 INFO:teuthology.orchestra.run.smithi143.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.45614.log.gz 2023-11-26T07:25:35.096 INFO:teuthology.orchestra.run.smithi143.stderr:/var/log/ceph/ceph-client.admin.54463.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53828.log 2023-11-26T07:25:35.096 INFO:teuthology.orchestra.run.smithi143.stderr: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.54463.log.gz 2023-11-26T07:25:35.096 INFO:teuthology.orchestra.run.smithi143.stderr:/var/log/ceph/ceph-client.admin.48283.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.58077.log 2023-11-26T07:25:35.096 INFO:teuthology.orchestra.run.smithi143.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.48283.log.gz 2023-11-26T07:25:35.097 INFO:teuthology.orchestra.run.smithi143.stderr:/var/log/ceph/ceph-client.admin.53828.log: 89.5%gzip -- replaced with /var/log/ceph/ceph-client.admin.53828.log.gz 2023-11-26T07:25:35.097 INFO:teuthology.orchestra.run.smithi143.stderr: -5 --verbose -- /var/log/ceph/ceph-osd.0.log 2023-11-26T07:25:35.098 INFO:teuthology.orchestra.run.smithi143.stderr:/var/log/ceph/ceph-client.admin.58077.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45380.log 2023-11-26T07:25:35.099 INFO:teuthology.orchestra.run.smithi143.stderr: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.58077.log.gz 2023-11-26T07:25:35.099 INFO:teuthology.orchestra.run.smithi188.stderr:/var/log/ceph/ceph-client.1.44114.log: 89.5% -- replaced with /var/log/ceph/ceph-client.1.44114.log.gz 2023-11-26T07:25:35.099 INFO:teuthology.orchestra.run.smithi143.stderr:/var/log/ceph/ceph-osd.0.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44866.log 2023-11-26T07:25:35.100 INFO:teuthology.orchestra.run.smithi143.stderr:/var/log/ceph/ceph-client.admin.45380.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48469.log 2023-11-26T07:25:35.101 INFO:teuthology.orchestra.run.smithi143.stderr: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.45380.log.gz 2023-11-26T07:25:35.102 INFO:teuthology.orchestra.run.smithi143.stderr:/var/log/ceph/ceph-client.admin.44866.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.44866.log.gz 2023-11-26T07:25:35.102 INFO:teuthology.orchestra.run.smithi143.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.55490.log 2023-11-26T07:25:35.103 INFO:teuthology.orchestra.run.smithi143.stderr:/var/log/ceph/ceph-client.admin.48469.log: gzip -5 --verbose -- 87.7% /var/log/ceph/ceph-client.admin.46315.log -- replaced with /var/log/ceph/ceph-client.admin.48469.log.gz 2023-11-26T07:25:35.103 INFO:teuthology.orchestra.run.smithi143.stderr: 2023-11-26T07:25:35.104 INFO:teuthology.orchestra.run.smithi143.stderr:/var/log/ceph/ceph-client.admin.55490.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44392.log 2023-11-26T07:25:35.104 INFO:teuthology.orchestra.run.smithi143.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.55490.log.gz 2023-11-26T07:25:35.105 INFO:teuthology.orchestra.run.smithi143.stderr:/var/log/ceph/ceph-client.admin.46315.log: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.46315.log.gz 2023-11-26T07:25:35.105 INFO:teuthology.orchestra.run.smithi143.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50562.log 2023-11-26T07:25:35.106 INFO:teuthology.orchestra.run.smithi143.stderr:/var/log/ceph/ceph-client.admin.44392.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46715.log 2023-11-26T07:25:35.106 INFO:teuthology.orchestra.run.smithi143.stderr: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.44392.log.gz 2023-11-26T07:25:35.107 INFO:teuthology.orchestra.run.smithi143.stderr:/var/log/ceph/ceph-client.admin.50562.log: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.50562.log.gz 2023-11-26T07:25:35.107 INFO:teuthology.orchestra.run.smithi143.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47085.log 2023-11-26T07:25:35.108 INFO:teuthology.orchestra.run.smithi143.stderr:/var/log/ceph/ceph-client.admin.46715.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48335.log 2023-11-26T07:25:35.108 INFO:teuthology.orchestra.run.smithi143.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.46715.log.gz 2023-11-26T07:25:35.109 INFO:teuthology.orchestra.run.smithi143.stderr:/var/log/ceph/ceph-client.admin.47085.log: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.47085.log.gz 2023-11-26T07:25:35.110 INFO:teuthology.orchestra.run.smithi143.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51161.log 2023-11-26T07:25:35.111 INFO:teuthology.orchestra.run.smithi143.stderr:/var/log/ceph/ceph-client.admin.48335.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47535.log 2023-11-26T07:25:35.111 INFO:teuthology.orchestra.run.smithi143.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.48335.log.gz 2023-11-26T07:25:35.112 INFO:teuthology.orchestra.run.smithi143.stderr:/var/log/ceph/ceph-client.admin.51161.log: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.51161.log.gz 2023-11-26T07:25:35.112 INFO:teuthology.orchestra.run.smithi143.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50510.log 2023-11-26T07:25:35.113 INFO:teuthology.orchestra.run.smithi143.stderr:/var/log/ceph/ceph-client.admin.47535.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.56997.log 2023-11-26T07:25:35.113 INFO:teuthology.orchestra.run.smithi143.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.47535.log.gz 2023-11-26T07:25:35.114 INFO:teuthology.orchestra.run.smithi143.stderr:/var/log/ceph/ceph-client.admin.50510.log: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.50510.log.gz 2023-11-26T07:25:35.114 INFO:teuthology.orchestra.run.smithi143.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.60566.log 2023-11-26T07:25:35.115 INFO:teuthology.orchestra.run.smithi143.stderr:/var/log/ceph/ceph-client.admin.56997.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47665.log 2023-11-26T07:25:35.115 INFO:teuthology.orchestra.run.smithi143.stderr: 89.4% -- replaced with /var/log/ceph/ceph-client.admin.56997.log.gz 2023-11-26T07:25:35.116 INFO:teuthology.orchestra.run.smithi143.stderr:/var/log/ceph/ceph-client.admin.60566.log: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.60566.log.gz 2023-11-26T07:25:35.116 INFO:teuthology.orchestra.run.smithi143.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52764.log 2023-11-26T07:25:35.118 INFO:teuthology.orchestra.run.smithi143.stderr:/var/log/ceph/ceph-client.admin.47665.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.57257.log 2023-11-26T07:25:35.118 INFO:teuthology.orchestra.run.smithi143.stderr: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.47665.log.gz 2023-11-26T07:25:35.119 INFO:teuthology.orchestra.run.smithi143.stderr:/var/log/ceph/ceph-client.admin.52764.log: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.52764.log.gz 2023-11-26T07:25:35.119 INFO:teuthology.orchestra.run.smithi143.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48819.log 2023-11-26T07:25:35.120 INFO:teuthology.orchestra.run.smithi143.stderr:/var/log/ceph/ceph-client.admin.57257.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44678.log 2023-11-26T07:25:35.120 INFO:teuthology.orchestra.run.smithi143.stderr:/var/log/ceph/ceph-client.admin.48819.log: 91.0% -- replaced with /var/log/ceph/ceph-client.admin.57257.log.gz 2023-11-26T07:25:35.121 INFO:teuthology.orchestra.run.smithi143.stderr: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.48819.log.gz 2023-11-26T07:25:35.121 INFO:teuthology.orchestra.run.smithi143.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49139.log 2023-11-26T07:25:35.122 INFO:teuthology.orchestra.run.smithi143.stderr:/var/log/ceph/ceph-client.admin.44678.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.60769.log 2023-11-26T07:25:35.122 INFO:teuthology.orchestra.run.smithi143.stderr: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.44678.log.gz 2023-11-26T07:25:35.123 INFO:teuthology.orchestra.run.smithi143.stderr:/var/log/ceph/ceph-client.admin.49139.log: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.49139.log.gz 2023-11-26T07:25:35.123 INFO:teuthology.orchestra.run.smithi143.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51558.log 2023-11-26T07:25:35.125 INFO:teuthology.orchestra.run.smithi143.stderr:/var/log/ceph/ceph-client.admin.60769.log: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.60769.log.gz 2023-11-26T07:25:35.125 INFO:teuthology.orchestra.run.smithi143.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46289.log 2023-11-26T07:25:35.126 INFO:teuthology.orchestra.run.smithi143.stderr:/var/log/ceph/ceph-client.admin.51558.log: gzip 88.1% -5 -- replaced with /var/log/ceph/ceph-client.admin.51558.log.gz --verbose 2023-11-26T07:25:35.126 INFO:teuthology.orchestra.run.smithi143.stderr: -- /var/log/ceph/ceph-client.admin.50370.log 2023-11-26T07:25:35.127 INFO:teuthology.orchestra.run.smithi143.stderr:/var/log/ceph/ceph-client.admin.46289.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50668.log 2023-11-26T07:25:35.127 INFO:teuthology.orchestra.run.smithi143.stderr: 89.6% -- replaced with /var/log/ceph/ceph-client.admin.46289.log.gz 2023-11-26T07:25:35.128 INFO:teuthology.orchestra.run.smithi143.stderr:/var/log/ceph/ceph-client.admin.50370.log: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.50370.log.gz 2023-11-26T07:25:35.128 INFO:teuthology.orchestra.run.smithi143.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.58625.log 2023-11-26T07:25:35.129 INFO:teuthology.orchestra.run.smithi143.stderr:/var/log/ceph/ceph-client.admin.50668.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.58873.log 2023-11-26T07:25:35.129 INFO:teuthology.orchestra.run.smithi143.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.50668.log.gz 2023-11-26T07:25:35.130 INFO:teuthology.orchestra.run.smithi143.stderr:/var/log/ceph/ceph-client.admin.58625.log: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.58625.log.gz 2023-11-26T07:25:35.130 INFO:teuthology.orchestra.run.smithi143.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48413.log 2023-11-26T07:25:35.131 INFO:teuthology.orchestra.run.smithi143.stderr:/var/log/ceph/ceph-client.admin.58873.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44574.log 2023-11-26T07:25:35.131 INFO:teuthology.orchestra.run.smithi143.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.58873.log.gz 2023-11-26T07:25:35.132 INFO:teuthology.orchestra.run.smithi143.stderr:/var/log/ceph/ceph-client.admin.48413.log: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.48413.log.gz 2023-11-26T07:25:35.132 INFO:teuthology.orchestra.run.smithi143.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48204.log 2023-11-26T07:25:35.134 INFO:teuthology.orchestra.run.smithi143.stderr:/var/log/ceph/ceph-client.admin.44574.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47509.log 2023-11-26T07:25:35.134 INFO:teuthology.orchestra.run.smithi143.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.44574.log.gz 2023-11-26T07:25:35.135 INFO:teuthology.orchestra.run.smithi143.stderr:/var/log/ceph/ceph-client.admin.48204.log: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.48204.log.gz 2023-11-26T07:25:35.135 INFO:teuthology.orchestra.run.smithi143.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51213.log 2023-11-26T07:25:35.136 INFO:teuthology.orchestra.run.smithi143.stderr:/var/log/ceph/ceph-client.admin.47509.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44027.log 2023-11-26T07:25:35.136 INFO:teuthology.orchestra.run.smithi143.stderr: 89.6% -- replaced with /var/log/ceph/ceph-client.admin.47509.log.gz 2023-11-26T07:25:35.137 INFO:teuthology.orchestra.run.smithi143.stderr:/var/log/ceph/ceph-client.admin.51213.log: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.51213.log.gz 2023-11-26T07:25:35.137 INFO:teuthology.orchestra.run.smithi143.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49294.log 2023-11-26T07:25:35.138 INFO:teuthology.orchestra.run.smithi143.stderr:/var/log/ceph/ceph-client.admin.44027.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.56547.log 2023-11-26T07:25:35.138 INFO:teuthology.orchestra.run.smithi143.stderr: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.44027.log.gz 2023-11-26T07:25:35.139 INFO:teuthology.orchestra.run.smithi143.stderr:/var/log/ceph/ceph-client.admin.49294.log: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.49294.log.gz 2023-11-26T07:25:35.139 INFO:teuthology.orchestra.run.smithi143.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.55464.log 2023-11-26T07:25:35.141 INFO:teuthology.orchestra.run.smithi143.stderr:/var/log/ceph/ceph-client.admin.56547.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.60441.log 2023-11-26T07:25:35.141 INFO:teuthology.orchestra.run.smithi143.stderr: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.56547.log.gz 2023-11-26T07:25:35.142 INFO:teuthology.orchestra.run.smithi143.stderr:/var/log/ceph/ceph-client.admin.55464.log: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.55464.log.gz 2023-11-26T07:25:35.142 INFO:teuthology.orchestra.run.smithi143.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.55569.log 2023-11-26T07:25:35.143 INFO:teuthology.orchestra.run.smithi143.stderr:/var/log/ceph/ceph-client.admin.60441.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46500.log 2023-11-26T07:25:35.143 INFO:teuthology.orchestra.run.smithi143.stderr: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.60441.log.gz 2023-11-26T07:25:35.144 INFO:teuthology.orchestra.run.smithi143.stderr:/var/log/ceph/ceph-client.admin.55569.log: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.55569.log.gz 2023-11-26T07:25:35.144 INFO:teuthology.orchestra.run.smithi143.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.60043.log 2023-11-26T07:25:35.145 INFO:teuthology.orchestra.run.smithi143.stderr:/var/log/ceph/ceph-client.admin.46500.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47483.log 2023-11-26T07:25:35.145 INFO:teuthology.orchestra.run.smithi143.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.46500.log.gz 2023-11-26T07:25:35.146 INFO:teuthology.orchestra.run.smithi143.stderr:/var/log/ceph/ceph-client.admin.60043.log: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.60043.log.gz 2023-11-26T07:25:35.146 INFO:teuthology.orchestra.run.smithi143.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48361.log 2023-11-26T07:25:35.147 INFO:teuthology.orchestra.run.smithi143.stderr:/var/log/ceph/ceph-client.admin.47483.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53431.log 2023-11-26T07:25:35.147 INFO:teuthology.orchestra.run.smithi143.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.47483.log.gz 2023-11-26T07:25:35.149 INFO:teuthology.orchestra.run.smithi143.stderr:/var/log/ceph/ceph-client.admin.48361.log: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.48361.log.gz 2023-11-26T07:25:35.149 INFO:teuthology.orchestra.run.smithi143.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.59907.log 2023-11-26T07:25:35.150 INFO:teuthology.orchestra.run.smithi143.stderr:/var/log/ceph/ceph-client.admin.53431.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49975.log 2023-11-26T07:25:35.150 INFO:teuthology.orchestra.run.smithi143.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.53431.log.gz 2023-11-26T07:25:35.151 INFO:teuthology.orchestra.run.smithi143.stderr:/var/log/ceph/ceph-client.admin.59907.log: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.59907.log.gz 2023-11-26T07:25:35.151 INFO:teuthology.orchestra.run.smithi143.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.59182.log 2023-11-26T07:25:35.152 INFO:teuthology.orchestra.run.smithi143.stderr:/var/log/ceph/ceph-client.admin.49975.log: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.49975.log.gz 2023-11-26T07:25:35.153 INFO:teuthology.orchestra.run.smithi143.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53174.log 2023-11-26T07:25:35.154 INFO:teuthology.orchestra.run.smithi143.stderr:/var/log/ceph/ceph-client.admin.59182.log: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.59182.log.gz 2023-11-26T07:25:35.154 INFO:teuthology.orchestra.run.smithi143.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54515.log 2023-11-26T07:25:35.155 INFO:teuthology.orchestra.run.smithi143.stderr:/var/log/ceph/ceph-client.admin.53174.log: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.53174.log.gz 2023-11-26T07:25:35.155 INFO:teuthology.orchestra.run.smithi143.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46185.log 2023-11-26T07:25:35.156 INFO:teuthology.orchestra.run.smithi143.stderr:/var/log/ceph/ceph-client.admin.54515.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50053.log 2023-11-26T07:25:35.156 INFO:teuthology.orchestra.run.smithi143.stderr: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.54515.log.gz 2023-11-26T07:25:35.157 INFO:teuthology.orchestra.run.smithi143.stderr:/var/log/ceph/ceph-client.admin.46185.log: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.46185.log.gz 2023-11-26T07:25:35.157 INFO:teuthology.orchestra.run.smithi143.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.58977.log 2023-11-26T07:25:35.158 INFO:teuthology.orchestra.run.smithi143.stderr:/var/log/ceph/ceph-client.admin.50053.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54830.log 2023-11-26T07:25:35.158 INFO:teuthology.orchestra.run.smithi143.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.50053.log.gz 2023-11-26T07:25:35.159 INFO:teuthology.orchestra.run.smithi143.stderr:/var/log/ceph/ceph-client.admin.58977.log: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.58977.log.gz 2023-11-26T07:25:35.159 INFO:teuthology.orchestra.run.smithi143.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54574.log 2023-11-26T07:25:35.160 INFO:teuthology.orchestra.run.smithi143.stderr:/var/log/ceph/ceph-client.admin.54830.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49744.log 2023-11-26T07:25:35.161 INFO:teuthology.orchestra.run.smithi143.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.54830.log.gz 2023-11-26T07:25:35.161 INFO:teuthology.orchestra.run.smithi143.stderr:/var/log/ceph/ceph-client.admin.54574.log: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.54574.log.gz 2023-11-26T07:25:35.162 INFO:teuthology.orchestra.run.smithi143.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.57729.log 2023-11-26T07:25:35.163 INFO:teuthology.orchestra.run.smithi143.stderr:/var/log/ceph/ceph-client.admin.49744.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52732.log 2023-11-26T07:25:35.163 INFO:teuthology.orchestra.run.smithi143.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.49744.log.gz 2023-11-26T07:25:35.164 INFO:teuthology.orchestra.run.smithi143.stderr:/var/log/ceph/ceph-client.admin.57729.log: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.57729.log.gz 2023-11-26T07:25:35.164 INFO:teuthology.orchestra.run.smithi143.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48087.log 2023-11-26T07:25:35.165 INFO:teuthology.orchestra.run.smithi143.stderr:/var/log/ceph/ceph-client.admin.52732.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.55412.log 2023-11-26T07:25:35.165 INFO:teuthology.orchestra.run.smithi143.stderr: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.52732.log.gz 2023-11-26T07:25:35.166 INFO:teuthology.orchestra.run.smithi143.stderr:/var/log/ceph/ceph-client.admin.48087.log: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.48087.log.gz 2023-11-26T07:25:35.166 INFO:teuthology.orchestra.run.smithi143.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.58129.log 2023-11-26T07:25:35.167 INFO:teuthology.orchestra.run.smithi143.stderr:/var/log/ceph/ceph-client.admin.55412.log: 89.4% -- replaced with /var/log/ceph/ceph-client.admin.55412.log.gz 2023-11-26T07:25:35.167 INFO:teuthology.orchestra.run.smithi143.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46552.log 2023-11-26T07:25:35.168 INFO:teuthology.orchestra.run.smithi143.stderr:/var/log/ceph/ceph-client.admin.58129.log: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.58129.log.gz 2023-11-26T07:25:35.168 INFO:teuthology.orchestra.run.smithi143.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.60074.log 2023-11-26T07:25:35.169 INFO:teuthology.orchestra.run.smithi143.stderr:/var/log/ceph/ceph-client.admin.46552.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.60285.log 2023-11-26T07:25:35.170 INFO:teuthology.orchestra.run.smithi143.stderr: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.46552.log.gz 2023-11-26T07:25:35.170 INFO:teuthology.orchestra.run.smithi143.stderr:/var/log/ceph/ceph-client.admin.60074.log: 89.5% -- replaced with /var/log/ceph/ceph-client.admin.60074.log.gz 2023-11-26T07:25:35.171 INFO:teuthology.orchestra.run.smithi143.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.56398.log 2023-11-26T07:25:35.172 INFO:teuthology.orchestra.run.smithi143.stderr:/var/log/ceph/ceph-client.admin.60285.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44418.log 2023-11-26T07:25:35.172 INFO:teuthology.orchestra.run.smithi143.stderr: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.60285.log.gz 2023-11-26T07:25:35.173 INFO:teuthology.orchestra.run.smithi143.stderr:/var/log/ceph/ceph-client.admin.56398.log: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.56398.log.gz 2023-11-26T07:25:35.173 INFO:teuthology.orchestra.run.smithi143.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.55154.log 2023-11-26T07:25:35.174 INFO:teuthology.orchestra.run.smithi143.stderr:/var/log/ceph/ceph-client.admin.44418.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45083.log 2023-11-26T07:25:35.174 INFO:teuthology.orchestra.run.smithi143.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.44418.log.gz 2023-11-26T07:25:35.175 INFO:teuthology.orchestra.run.smithi143.stderr:/var/log/ceph/ceph-client.admin.55154.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50822.log 2023-11-26T07:25:35.175 INFO:teuthology.orchestra.run.smithi143.stderr: 91.2% -- replaced with /var/log/ceph/ceph-client.admin.55154.log.gz 2023-11-26T07:25:35.176 INFO:teuthology.orchestra.run.smithi143.stderr:/var/log/ceph/ceph-client.admin.45083.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44340.log 2023-11-26T07:25:35.176 INFO:teuthology.orchestra.run.smithi143.stderr: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.45083.log.gz 2023-11-26T07:25:35.177 INFO:teuthology.orchestra.run.smithi143.stderr:/var/log/ceph/ceph-client.admin.50822.log: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.50822.log.gz 2023-11-26T07:25:35.178 INFO:teuthology.orchestra.run.smithi143.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.56600.log 2023-11-26T07:25:35.179 INFO:teuthology.orchestra.run.smithi143.stderr:/var/log/ceph/ceph-client.admin.44340.log: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.44340.log.gz 2023-11-26T07:25:35.179 INFO:teuthology.orchestra.run.smithi143.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49718.log 2023-11-26T07:25:35.180 INFO:teuthology.orchestra.run.smithi143.stderr:/var/log/ceph/ceph-client.admin.56600.log: 89.4% -- replaced with /var/log/ceph/ceph-client.admin.56600.log.gz 2023-11-26T07:25:35.180 INFO:teuthology.orchestra.run.smithi143.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44522.log 2023-11-26T07:25:35.181 INFO:teuthology.orchestra.run.smithi143.stderr:/var/log/ceph/ceph-client.admin.49718.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44665.log 2023-11-26T07:25:35.181 INFO:teuthology.orchestra.run.smithi143.stderr: 89.5% -- replaced with /var/log/ceph/ceph-client.admin.49718.log.gz 2023-11-26T07:25:35.182 INFO:teuthology.orchestra.run.smithi143.stderr:/var/log/ceph/ceph-client.admin.44522.log: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.44522.log.gz 2023-11-26T07:25:35.182 INFO:teuthology.orchestra.run.smithi143.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49210.log 2023-11-26T07:25:35.183 INFO:teuthology.orchestra.run.smithi143.stderr:/var/log/ceph/ceph-client.admin.44665.log: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.44665.log.gz 2023-11-26T07:25:35.183 INFO:teuthology.orchestra.run.smithi143.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52830.log 2023-11-26T07:25:35.184 INFO:teuthology.orchestra.run.smithi143.stderr:/var/log/ceph/ceph-client.admin.49210.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.57488.log 2023-11-26T07:25:35.184 INFO:teuthology.orchestra.run.smithi143.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.49210.log.gz 2023-11-26T07:25:35.185 INFO:teuthology.orchestra.run.smithi143.stderr:/var/log/ceph/ceph-client.admin.52830.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38304.log 2023-11-26T07:25:35.186 INFO:teuthology.orchestra.run.smithi143.stderr: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.52830.log.gz 2023-11-26T07:25:35.186 INFO:teuthology.orchestra.run.smithi143.stderr:/var/log/ceph/ceph-client.admin.57488.log: 92.4% -- replaced with /var/log/ceph/ceph-mgr.y.log.gz 2023-11-26T07:25:35.186 INFO:teuthology.orchestra.run.smithi143.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.57488.log.gz 2023-11-26T07:25:35.187 INFO:teuthology.orchestra.run.smithi143.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44366.log 2023-11-26T07:25:35.187 INFO:teuthology.orchestra.run.smithi143.stderr:/var/log/ceph/ceph-client.admin.38304.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47977.log 2023-11-26T07:25:35.188 INFO:teuthology.orchestra.run.smithi143.stderr: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.38304.log.gz 2023-11-26T07:25:35.188 INFO:teuthology.orchestra.run.smithi143.stderr:/var/log/ceph/ceph-client.admin.44366.log: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.44366.log.gz 2023-11-26T07:25:35.189 INFO:teuthology.orchestra.run.smithi143.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47951.log 2023-11-26T07:25:35.190 INFO:teuthology.orchestra.run.smithi143.stderr:/var/log/ceph/ceph-client.admin.47977.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51428.log 2023-11-26T07:25:35.190 INFO:teuthology.orchestra.run.smithi143.stderr: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.47977.log.gz 2023-11-26T07:25:35.191 INFO:teuthology.orchestra.run.smithi143.stderr:/var/log/ceph/ceph-client.admin.47951.log: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.47951.log.gz 2023-11-26T07:25:35.191 INFO:teuthology.orchestra.run.smithi143.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-osd.2.log 2023-11-26T07:25:35.192 INFO:teuthology.orchestra.run.smithi143.stderr:/var/log/ceph/ceph-client.admin.51428.log: gzip 89.2% -5 -- replaced with /var/log/ceph/ceph-client.admin.51428.log.gz --verbose 2023-11-26T07:25:35.192 INFO:teuthology.orchestra.run.smithi143.stderr: -- /var/log/ceph/ceph-client.admin.49320.log 2023-11-26T07:25:35.193 INFO:teuthology.orchestra.run.smithi143.stderr:/var/log/ceph/ceph-osd.2.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48921.log 2023-11-26T07:25:35.194 INFO:teuthology.orchestra.run.smithi143.stderr:/var/log/ceph/ceph-client.admin.49320.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52699.log 2023-11-26T07:25:35.194 INFO:teuthology.orchestra.run.smithi143.stderr: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.49320.log.gz 2023-11-26T07:25:35.195 INFO:teuthology.orchestra.run.smithi143.stderr:/var/log/ceph/ceph-client.admin.48921.log: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.48921.log.gz 2023-11-26T07:25:35.195 INFO:teuthology.orchestra.run.smithi143.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46767.log 2023-11-26T07:25:35.196 INFO:teuthology.orchestra.run.smithi143.stderr:/var/log/ceph/ceph-client.admin.52699.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.57576.log 2023-11-26T07:25:35.197 INFO:teuthology.orchestra.run.smithi143.stderr: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.52699.log.gz 2023-11-26T07:25:35.197 INFO:teuthology.orchestra.run.smithi143.stderr:/var/log/ceph/ceph-client.admin.46767.log: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.46767.log.gz 2023-11-26T07:25:35.198 INFO:teuthology.orchestra.run.smithi143.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44496.log 2023-11-26T07:25:35.199 INFO:teuthology.orchestra.run.smithi143.stderr:/var/log/ceph/ceph-client.admin.57576.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.56971.log 2023-11-26T07:25:35.199 INFO:teuthology.orchestra.run.smithi143.stderr: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.57576.log.gz 2023-11-26T07:25:35.200 INFO:teuthology.orchestra.run.smithi143.stderr:/var/log/ceph/ceph-client.admin.44496.log: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.44496.log.gz 2023-11-26T07:25:35.200 INFO:teuthology.orchestra.run.smithi143.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45088.log 2023-11-26T07:25:35.201 INFO:teuthology.orchestra.run.smithi143.stderr:/var/log/ceph/ceph-client.admin.56971.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.56889.log 2023-11-26T07:25:35.201 INFO:teuthology.orchestra.run.smithi143.stderr: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.56971.log.gz 2023-11-26T07:25:35.202 INFO:teuthology.orchestra.run.smithi143.stderr:/var/log/ceph/ceph-client.admin.45088.log: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.45088.log.gz 2023-11-26T07:25:35.202 INFO:teuthology.orchestra.run.smithi143.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.58337.log 2023-11-26T07:25:35.203 INFO:teuthology.orchestra.run.smithi143.stderr:/var/log/ceph/ceph-client.admin.56889.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.57864.log 2023-11-26T07:25:35.203 INFO:teuthology.orchestra.run.smithi143.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.56889.log.gz 2023-11-26T07:25:35.205 INFO:teuthology.orchestra.run.smithi143.stderr:/var/log/ceph/ceph-client.admin.58337.log: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.58337.log.gz 2023-11-26T07:25:35.205 INFO:teuthology.orchestra.run.smithi143.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53963.log 2023-11-26T07:25:35.206 INFO:teuthology.orchestra.run.smithi143.stderr:/var/log/ceph/ceph-client.admin.57864.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.58599.log 2023-11-26T07:25:35.206 INFO:teuthology.orchestra.run.smithi143.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.57864.log.gz 2023-11-26T07:25:35.207 INFO:teuthology.orchestra.run.smithi143.stderr:/var/log/ceph/ceph-client.admin.53963.log: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.53963.log.gz 2023-11-26T07:25:35.207 INFO:teuthology.orchestra.run.smithi143.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52646.log 2023-11-26T07:25:35.208 INFO:teuthology.orchestra.run.smithi143.stderr:/var/log/ceph/ceph-client.admin.58599.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54606.log 2023-11-26T07:25:35.208 INFO:teuthology.orchestra.run.smithi143.stderr: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.58599.log.gz 2023-11-26T07:25:35.209 INFO:teuthology.orchestra.run.smithi143.stderr:/var/log/ceph/ceph-client.admin.52646.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.52646.log.gz 2023-11-26T07:25:35.209 INFO:teuthology.orchestra.run.smithi143.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49181.log 2023-11-26T07:25:35.210 INFO:teuthology.orchestra.run.smithi143.stderr:/var/log/ceph/ceph-client.admin.54606.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.55024.log 2023-11-26T07:25:35.211 INFO:teuthology.orchestra.run.smithi143.stderr: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.54606.log.gz 2023-11-26T07:25:35.211 INFO:teuthology.orchestra.run.smithi143.stderr:/var/log/ceph/ceph-client.admin.49181.log: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.49181.log.gz 2023-11-26T07:25:35.212 INFO:teuthology.orchestra.run.smithi143.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.55076.log 2023-11-26T07:25:35.213 INFO:teuthology.orchestra.run.smithi143.stderr:/var/log/ceph/ceph-client.admin.55024.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.61136.log 2023-11-26T07:25:35.213 INFO:teuthology.orchestra.run.smithi143.stderr: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.55024.log.gz 2023-11-26T07:25:35.214 INFO:teuthology.orchestra.run.smithi143.stderr:/var/log/ceph/ceph-client.admin.55076.log: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.55076.log.gz 2023-11-26T07:25:35.214 INFO:teuthology.orchestra.run.smithi143.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45640.log 2023-11-26T07:25:35.215 INFO:teuthology.orchestra.run.smithi143.stderr:/var/log/ceph/ceph-client.admin.61136.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54750.log 2023-11-26T07:25:35.215 INFO:teuthology.orchestra.run.smithi143.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.61136.log.gz 2023-11-26T07:25:35.216 INFO:teuthology.orchestra.run.smithi143.stderr:/var/log/ceph/ceph-client.admin.45640.log: 89.5% -- replaced with /var/log/ceph/ceph-client.admin.45640.log.gz 2023-11-26T07:25:35.216 INFO:teuthology.orchestra.run.smithi143.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54776.log 2023-11-26T07:25:35.217 INFO:teuthology.orchestra.run.smithi143.stderr:/var/log/ceph/ceph-client.admin.54750.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.57049.log 2023-11-26T07:25:35.217 INFO:teuthology.orchestra.run.smithi143.stderr: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.54750.log.gz 2023-11-26T07:25:35.218 INFO:teuthology.orchestra.run.smithi143.stderr:/var/log/ceph/ceph-client.admin.54776.log: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.54776.log.gz 2023-11-26T07:25:35.218 INFO:teuthology.orchestra.run.smithi143.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.56574.log 2023-11-26T07:25:35.219 INFO:teuthology.orchestra.run.smithi143.stderr:/var/log/ceph/ceph-client.admin.57049.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.61032.log 2023-11-26T07:25:35.219 INFO:teuthology.orchestra.run.smithi143.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.57049.log.gz 2023-11-26T07:25:35.220 INFO:teuthology.orchestra.run.smithi143.stderr:/var/log/ceph/ceph-client.admin.56574.log: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.56574.log.gz 2023-11-26T07:25:35.220 INFO:teuthology.orchestra.run.smithi143.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.60691.log 2023-11-26T07:25:35.222 INFO:teuthology.orchestra.run.smithi143.stderr:/var/log/ceph/ceph-client.admin.61032.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.0.45847.log 2023-11-26T07:25:35.222 INFO:teuthology.orchestra.run.smithi143.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.61032.log.gz 2023-11-26T07:25:35.223 INFO:teuthology.orchestra.run.smithi143.stderr:/var/log/ceph/ceph-client.admin.60691.log: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.60691.log.gz 2023-11-26T07:25:35.223 INFO:teuthology.orchestra.run.smithi143.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.55204.log 2023-11-26T07:25:35.224 INFO:teuthology.orchestra.run.smithi143.stderr:/var/log/ceph/ceph-client.0.45847.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.0.57362.log 2023-11-26T07:25:35.225 INFO:teuthology.orchestra.run.smithi143.stderr: 89.9% -- replaced with /var/log/ceph/ceph-client.0.45847.log.gz 2023-11-26T07:25:35.225 INFO:teuthology.orchestra.run.smithi143.stderr:/var/log/ceph/ceph-client.admin.55204.log: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.55204.log.gz 2023-11-26T07:25:35.226 INFO:teuthology.orchestra.run.smithi143.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.58311.log 2023-11-26T07:25:35.227 INFO:teuthology.orchestra.run.smithi143.stderr:/var/log/ceph/ceph-client.0.57362.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51809.log 2023-11-26T07:25:35.227 INFO:teuthology.orchestra.run.smithi143.stderr:/var/log/ceph/ceph-client.admin.58311.log: 90.4% -- replaced with /var/log/ceph/ceph-client.0.57362.log.gz 2023-11-26T07:25:35.228 INFO:teuthology.orchestra.run.smithi143.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.58311.log.gz 2023-11-26T07:25:35.228 INFO:teuthology.orchestra.run.smithi143.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-mon.a.log 2023-11-26T07:25:35.229 INFO:teuthology.orchestra.run.smithi143.stderr:/var/log/ceph/ceph-client.admin.51809.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.56091.log 2023-11-26T07:25:35.229 INFO:teuthology.orchestra.run.smithi143.stderr: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.51809.log.gz 2023-11-26T07:25:35.230 INFO:teuthology.orchestra.run.smithi143.stderr:/var/log/ceph/ceph-mon.a.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.59379.log 2023-11-26T07:25:35.231 INFO:teuthology.orchestra.run.smithi143.stderr:/var/log/ceph/ceph-client.admin.56091.log: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.56091.log.gz 2023-11-26T07:25:35.232 INFO:teuthology.orchestra.run.smithi143.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50404.log 2023-11-26T07:25:35.233 INFO:teuthology.orchestra.run.smithi143.stderr:/var/log/ceph/ceph-client.admin.59379.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50456.log 2023-11-26T07:25:35.233 INFO:teuthology.orchestra.run.smithi143.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.59379.log.gz 2023-11-26T07:25:35.234 INFO:teuthology.orchestra.run.smithi143.stderr:/var/log/ceph/ceph-client.admin.50404.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.50404.log.gz 2023-11-26T07:25:35.234 INFO:teuthology.orchestra.run.smithi143.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.56652.log 2023-11-26T07:25:35.235 INFO:teuthology.orchestra.run.smithi143.stderr:/var/log/ceph/ceph-client.admin.50456.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48441.log 2023-11-26T07:25:35.235 INFO:teuthology.orchestra.run.smithi143.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.50456.log.gz 2023-11-26T07:25:35.236 INFO:teuthology.orchestra.run.smithi143.stderr:/var/log/ceph/ceph-client.admin.56652.log: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.56652.log.gz 2023-11-26T07:25:35.237 INFO:teuthology.orchestra.run.smithi143.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44871.log 2023-11-26T07:25:35.237 INFO:teuthology.orchestra.run.smithi143.stderr:/var/log/ceph/ceph-client.admin.48441.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.58573.log 2023-11-26T07:25:35.238 INFO:teuthology.orchestra.run.smithi143.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.48441.log.gz 2023-11-26T07:25:35.238 INFO:teuthology.orchestra.run.smithi143.stderr:/var/log/ceph/ceph-client.admin.44871.log: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.44871.log.gz 2023-11-26T07:25:35.239 INFO:teuthology.orchestra.run.smithi143.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46578.log 2023-11-26T07:25:35.240 INFO:teuthology.orchestra.run.smithi143.stderr:/var/log/ceph/ceph-client.admin.58573.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.59965.log 2023-11-26T07:25:35.240 INFO:teuthology.orchestra.run.smithi143.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.58573.log.gz 2023-11-26T07:25:35.241 INFO:teuthology.orchestra.run.smithi143.stderr:/var/log/ceph/ceph-client.admin.46578.log: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.46578.log.gz 2023-11-26T07:25:35.241 INFO:teuthology.orchestra.run.smithi143.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52021.log 2023-11-26T07:25:35.242 INFO:teuthology.orchestra.run.smithi143.stderr:/var/log/ceph/ceph-client.admin.59965.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.56521.log 2023-11-26T07:25:35.243 INFO:teuthology.orchestra.run.smithi143.stderr: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.59965.log.gz 2023-11-26T07:25:35.243 INFO:teuthology.orchestra.run.smithi143.stderr:/var/log/ceph/ceph-client.admin.52021.log: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.52021.log.gz 2023-11-26T07:25:35.244 INFO:teuthology.orchestra.run.smithi143.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.57436.log 2023-11-26T07:25:35.245 INFO:teuthology.orchestra.run.smithi143.stderr:/var/log/ceph/ceph-client.admin.56521.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47059.log 2023-11-26T07:25:35.245 INFO:teuthology.orchestra.run.smithi143.stderr: 89.4% -- replaced with /var/log/ceph/ceph-client.admin.56521.log.gz 2023-11-26T07:25:35.246 INFO:teuthology.orchestra.run.smithi143.stderr:/var/log/ceph/ceph-client.admin.57436.log: 89.5% -- replaced with /var/log/ceph/ceph-client.admin.57436.log.gz 2023-11-26T07:25:35.246 INFO:teuthology.orchestra.run.smithi143.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.60126.log 2023-11-26T07:25:35.247 INFO:teuthology.orchestra.run.smithi143.stderr:/var/log/ceph/ceph-client.admin.47059.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.60100.log 2023-11-26T07:25:35.248 INFO:teuthology.orchestra.run.smithi143.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.47059.log.gz 2023-11-26T07:25:35.248 INFO:teuthology.orchestra.run.smithi143.stderr:/var/log/ceph/ceph-client.admin.60126.log: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.60126.log.gz 2023-11-26T07:25:35.249 INFO:teuthology.orchestra.run.smithi143.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.55359.log 2023-11-26T07:25:35.249 INFO:teuthology.orchestra.run.smithi143.stderr:/var/log/ceph/ceph-client.admin.60100.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50616.log 2023-11-26T07:25:35.250 INFO:teuthology.orchestra.run.smithi143.stderr: 89.7% -- replaced with /var/log/ceph/ceph-client.admin.60100.log.gz 2023-11-26T07:25:35.250 INFO:teuthology.orchestra.run.smithi143.stderr:/var/log/ceph/ceph-client.admin.55359.log: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.55359.log.gz 2023-11-26T07:25:35.251 INFO:teuthology.orchestra.run.smithi143.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.59260.log 2023-11-26T07:25:35.252 INFO:teuthology.orchestra.run.smithi143.stderr:/var/log/ceph/ceph-client.admin.50616.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48231.log 2023-11-26T07:25:35.253 INFO:teuthology.orchestra.run.smithi143.stderr: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.50616.log.gz 2023-11-26T07:25:35.253 INFO:teuthology.orchestra.run.smithi143.stderr:/var/log/ceph/ceph-client.admin.59260.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.59260.log.gz 2023-11-26T07:25:35.253 INFO:teuthology.orchestra.run.smithi143.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46049.log 2023-11-26T07:25:35.254 INFO:teuthology.orchestra.run.smithi143.stderr:/var/log/ceph/ceph-client.admin.48231.log: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.48231.log.gz 2023-11-26T07:25:35.255 INFO:teuthology.orchestra.run.smithi143.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47873.log 2023-11-26T07:25:35.256 INFO:teuthology.orchestra.run.smithi143.stderr:/var/log/ceph/ceph-client.admin.46049.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.59878.log 2023-11-26T07:25:35.256 INFO:teuthology.orchestra.run.smithi143.stderr: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.46049.log.gz 2023-11-26T07:25:35.257 INFO:teuthology.orchestra.run.smithi143.stderr:/var/log/ceph/ceph-client.admin.47873.log: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.47873.log.gz 2023-11-26T07:25:35.257 INFO:teuthology.orchestra.run.smithi143.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.55758.log 2023-11-26T07:25:35.259 INFO:teuthology.orchestra.run.smithi143.stderr:/var/log/ceph/ceph-client.admin.59878.log: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.59878.log.gz 2023-11-26T07:25:35.259 INFO:teuthology.orchestra.run.smithi143.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46955.log 2023-11-26T07:25:35.260 INFO:teuthology.orchestra.run.smithi143.stderr:/var/log/ceph/ceph-client.admin.55758.log: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.55758.log.gz 2023-11-26T07:25:35.260 INFO:teuthology.orchestra.run.smithi143.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.60592.log 2023-11-26T07:25:35.262 INFO:teuthology.orchestra.run.smithi143.stderr:/var/log/ceph/ceph-client.admin.46955.log: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.46955.log.gzgzip 2023-11-26T07:25:35.262 INFO:teuthology.orchestra.run.smithi143.stderr: -5 --verbose -- /var/log/ceph/ceph-client.admin.47847.log 2023-11-26T07:25:35.263 INFO:teuthology.orchestra.run.smithi143.stderr:/var/log/ceph/ceph-client.admin.60592.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.60821.log 2023-11-26T07:25:35.263 INFO:teuthology.orchestra.run.smithi143.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.60592.log.gz 2023-11-26T07:25:35.265 INFO:teuthology.orchestra.run.smithi143.stderr:/var/log/ceph/ceph-client.admin.47847.log: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.47847.log.gz 2023-11-26T07:25:35.265 INFO:teuthology.orchestra.run.smithi143.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.60152.log 2023-11-26T07:25:35.266 INFO:teuthology.orchestra.run.smithi143.stderr:/var/log/ceph/ceph-client.admin.60821.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49822.log 2023-11-26T07:25:35.266 INFO:teuthology.orchestra.run.smithi143.stderr: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.60821.log.gz 2023-11-26T07:25:35.267 INFO:teuthology.orchestra.run.smithi143.stderr:/var/log/ceph/ceph-client.admin.60152.log: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.60152.log.gz 2023-11-26T07:25:35.267 INFO:teuthology.orchestra.run.smithi143.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49076.log 2023-11-26T07:25:35.269 INFO:teuthology.orchestra.run.smithi143.stderr:/var/log/ceph/ceph-client.admin.49822.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.49822.log.gz 2023-11-26T07:25:35.269 INFO:teuthology.orchestra.run.smithi143.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52471.log 2023-11-26T07:25:35.269 INFO:teuthology.orchestra.run.smithi143.stderr:/var/log/ceph/ceph-client.admin.49076.log: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.49076.log.gz 2023-11-26T07:25:35.270 INFO:teuthology.orchestra.run.smithi143.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.56208.log 2023-11-26T07:25:35.271 INFO:teuthology.orchestra.run.smithi188.stderr: 92.9% -- replaced with /var/log/ceph/ceph-mds.b.log.gz 2023-11-26T07:25:35.271 INFO:teuthology.orchestra.run.smithi143.stderr:/var/log/ceph/ceph-client.admin.52471.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52051.log 2023-11-26T07:25:35.272 INFO:teuthology.orchestra.run.smithi143.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.52471.log.gz 2023-11-26T07:25:35.272 INFO:teuthology.orchestra.run.smithi143.stderr:/var/log/ceph/ceph-client.admin.56208.log: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.56208.log.gz 2023-11-26T07:25:35.273 INFO:teuthology.orchestra.run.smithi143.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49050.log 2023-11-26T07:25:35.274 INFO:teuthology.orchestra.run.smithi143.stderr:/var/log/ceph/ceph-client.admin.52051.log: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.52051.log.gz 2023-11-26T07:25:35.274 INFO:teuthology.orchestra.run.smithi143.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54359.log 2023-11-26T07:25:35.275 INFO:teuthology.orchestra.run.smithi143.stderr:/var/log/ceph/ceph-client.admin.49050.log: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.49050.log.gz 2023-11-26T07:25:35.276 INFO:teuthology.orchestra.run.smithi143.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51506.log 2023-11-26T07:25:35.276 INFO:teuthology.orchestra.run.smithi143.stderr:/var/log/ceph/ceph-client.admin.54359.log: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.54359.log.gz 2023-11-26T07:25:35.277 INFO:teuthology.orchestra.run.smithi143.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44672.log 2023-11-26T07:25:35.278 INFO:teuthology.orchestra.run.smithi143.stderr:/var/log/ceph/ceph-client.admin.51506.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.56156.log 2023-11-26T07:25:35.278 INFO:teuthology.orchestra.run.smithi143.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.51506.log.gz 2023-11-26T07:25:35.279 INFO:teuthology.orchestra.run.smithi143.stderr:/var/log/ceph/ceph-client.admin.44672.log: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.44672.log.gz 2023-11-26T07:25:35.279 INFO:teuthology.orchestra.run.smithi143.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47925.log 2023-11-26T07:25:35.280 INFO:teuthology.orchestra.run.smithi143.stderr:/var/log/ceph/ceph-client.admin.56156.log: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.56156.log.gz 2023-11-26T07:25:35.281 INFO:teuthology.orchestra.run.smithi143.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.58441.log 2023-11-26T07:25:35.282 INFO:teuthology.orchestra.run.smithi143.stderr:/var/log/ceph/ceph-client.admin.47925.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53544.log 2023-11-26T07:25:35.282 INFO:teuthology.orchestra.run.smithi143.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.47925.log.gz 2023-11-26T07:25:35.283 INFO:teuthology.orchestra.run.smithi143.stderr:/var/log/ceph/ceph-client.admin.58441.log: 89.5% -- replaced with /var/log/ceph/ceph-client.admin.58441.log.gz 2023-11-26T07:25:35.283 INFO:teuthology.orchestra.run.smithi143.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46604.log 2023-11-26T07:25:35.284 INFO:teuthology.orchestra.run.smithi143.stderr:/var/log/ceph/ceph-client.admin.53544.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.57075.log 2023-11-26T07:25:35.285 INFO:teuthology.orchestra.run.smithi143.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.53544.log.gz 2023-11-26T07:25:35.285 INFO:teuthology.orchestra.run.smithi143.stderr:/var/log/ceph/ceph-client.admin.46604.log: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.46604.log.gz 2023-11-26T07:25:35.286 INFO:teuthology.orchestra.run.smithi143.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47717.log 2023-11-26T07:25:35.287 INFO:teuthology.orchestra.run.smithi143.stderr:/var/log/ceph/ceph-client.admin.57075.log: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.57075.log.gz 2023-11-26T07:25:35.287 INFO:teuthology.orchestra.run.smithi143.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54177.log 2023-11-26T07:25:35.289 INFO:teuthology.orchestra.run.smithi143.stderr:/var/log/ceph/ceph-client.admin.47717.log: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.47717.log.gz 2023-11-26T07:25:35.289 INFO:teuthology.orchestra.run.smithi143.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45588.log 2023-11-26T07:25:35.290 INFO:teuthology.orchestra.run.smithi143.stderr:/var/log/ceph/ceph-client.admin.54177.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.57514.log 2023-11-26T07:25:35.290 INFO:teuthology.orchestra.run.smithi143.stderr: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.54177.log.gz 2023-11-26T07:25:35.291 INFO:teuthology.orchestra.run.smithi143.stderr:/var/log/ceph/ceph-client.admin.45588.log: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.45588.log.gz 2023-11-26T07:25:35.292 INFO:teuthology.orchestra.run.smithi143.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48715.log 2023-11-26T07:25:35.293 INFO:teuthology.orchestra.run.smithi143.stderr:/var/log/ceph/ceph-client.admin.57514.log: 86.3% -- replaced with /var/log/ceph/ceph-client.admin.57514.log.gz 2023-11-26T07:25:35.293 INFO:teuthology.orchestra.run.smithi143.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.58233.log 2023-11-26T07:25:35.294 INFO:teuthology.orchestra.run.smithi143.stderr:/var/log/ceph/ceph-client.admin.48715.log: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.48715.log.gz 2023-11-26T07:25:35.294 INFO:teuthology.orchestra.run.smithi143.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45096.log 2023-11-26T07:25:35.295 INFO:teuthology.orchestra.run.smithi143.stderr:/var/log/ceph/ceph-client.admin.58233.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45095.log 2023-11-26T07:25:35.296 INFO:teuthology.orchestra.run.smithi143.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.58233.log.gz 2023-11-26T07:25:35.297 INFO:teuthology.orchestra.run.smithi143.stderr:/var/log/ceph/ceph-client.admin.45096.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49024.log 2023-11-26T07:25:35.297 INFO:teuthology.orchestra.run.smithi143.stderr: 89.4% -- replaced with /var/log/ceph/ceph-client.admin.45096.log.gz 2023-11-26T07:25:35.298 INFO:teuthology.orchestra.run.smithi143.stderr:/var/log/ceph/ceph-client.admin.45095.log: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.45095.log.gz 2023-11-26T07:25:35.298 INFO:teuthology.orchestra.run.smithi143.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48029.log 2023-11-26T07:25:35.299 INFO:teuthology.orchestra.run.smithi143.stderr:/var/log/ceph/ceph-client.admin.49024.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53911.log 2023-11-26T07:25:35.300 INFO:teuthology.orchestra.run.smithi143.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.49024.log.gz 2023-11-26T07:25:35.301 INFO:teuthology.orchestra.run.smithi143.stderr:/var/log/ceph/ceph-client.admin.48029.log: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.48029.log.gz 2023-11-26T07:25:35.301 INFO:teuthology.orchestra.run.smithi143.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51376.log 2023-11-26T07:25:35.302 INFO:teuthology.orchestra.run.smithi143.stderr:/var/log/ceph/ceph-client.admin.53911.log: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.53911.log.gz 2023-11-26T07:25:35.303 INFO:teuthology.orchestra.run.smithi143.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.60493.log 2023-11-26T07:25:35.304 INFO:teuthology.orchestra.run.smithi143.stderr:/var/log/ceph/ceph-client.admin.51376.log: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.51376.log.gz 2023-11-26T07:25:35.304 INFO:teuthology.orchestra.run.smithi143.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54802.log 2023-11-26T07:25:35.306 INFO:teuthology.orchestra.run.smithi143.stderr:/var/log/ceph/ceph-client.admin.60493.log: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.60493.log.gz 2023-11-26T07:25:35.306 INFO:teuthology.orchestra.run.smithi143.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.60951.log 2023-11-26T07:25:35.306 INFO:teuthology.orchestra.run.smithi143.stderr:/var/log/ceph/ceph-client.admin.54802.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.56372.log 2023-11-26T07:25:35.307 INFO:teuthology.orchestra.run.smithi143.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.54802.log.gz 2023-11-26T07:25:35.308 INFO:teuthology.orchestra.run.smithi143.stderr:/var/log/ceph/ceph-client.admin.60951.log: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.60951.log.gz 2023-11-26T07:25:35.308 INFO:teuthology.orchestra.run.smithi143.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.60259.log 2023-11-26T07:25:35.309 INFO:teuthology.orchestra.run.smithi143.stderr:/var/log/ceph/ceph-client.admin.56372.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45086.log 2023-11-26T07:25:35.310 INFO:teuthology.orchestra.run.smithi143.stderr: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.56372.log.gz 2023-11-26T07:25:35.310 INFO:teuthology.orchestra.run.smithi143.stderr:/var/log/ceph/ceph-client.admin.60259.log: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.60259.log.gz 2023-11-26T07:25:35.311 INFO:teuthology.orchestra.run.smithi143.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52289.log 2023-11-26T07:25:35.312 INFO:teuthology.orchestra.run.smithi143.stderr:/var/log/ceph/ceph-client.admin.45086.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54698.log 2023-11-26T07:25:35.312 INFO:teuthology.orchestra.run.smithi143.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.45086.log.gz 2023-11-26T07:25:35.313 INFO:teuthology.orchestra.run.smithi143.stderr:/var/log/ceph/ceph-client.admin.52289.log: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.52289.log.gz 2023-11-26T07:25:35.314 INFO:teuthology.orchestra.run.smithi143.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50001.log 2023-11-26T07:25:35.315 INFO:teuthology.orchestra.run.smithi143.stderr:/var/log/ceph/ceph-client.admin.54698.log: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.54698.log.gz 2023-11-26T07:25:35.315 INFO:teuthology.orchestra.run.smithi143.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48767.log 2023-11-26T07:25:35.316 INFO:teuthology.orchestra.run.smithi143.stderr:/var/log/ceph/ceph-client.admin.50001.log: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.50001.log.gz 2023-11-26T07:25:35.317 INFO:teuthology.orchestra.run.smithi143.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44548.log 2023-11-26T07:25:35.318 INFO:teuthology.orchestra.run.smithi143.stderr:/var/log/ceph/ceph-client.admin.48767.log: 89.5% -- replaced with /var/log/ceph/ceph-client.admin.48767.log.gz 2023-11-26T07:25:35.318 INFO:teuthology.orchestra.run.smithi143.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46820.log 2023-11-26T07:25:35.320 INFO:teuthology.orchestra.run.smithi143.stderr:/var/log/ceph/ceph-client.admin.44548.log: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.44548.log.gz 2023-11-26T07:25:35.320 INFO:teuthology.orchestra.run.smithi143.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.57600.log 2023-11-26T07:25:35.321 INFO:teuthology.orchestra.run.smithi143.stderr:/var/log/ceph/ceph-client.admin.46820.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.46820.log.gz 2023-11-26T07:25:35.322 INFO:teuthology.orchestra.run.smithi143.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53776.log 2023-11-26T07:25:35.323 INFO:teuthology.orchestra.run.smithi143.stderr:/var/log/ceph/ceph-client.admin.57600.log: 86.2% -- replaced with /var/log/ceph/ceph-client.admin.57600.log.gz 2023-11-26T07:25:35.323 INFO:teuthology.orchestra.run.smithi143.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44883.log 2023-11-26T07:25:35.324 INFO:teuthology.orchestra.run.smithi143.stderr:/var/log/ceph/ceph-client.admin.53776.log: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.53776.log.gz 2023-11-26T07:25:35.324 INFO:teuthology.orchestra.run.smithi143.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43960.log 2023-11-26T07:25:35.326 INFO:teuthology.orchestra.run.smithi143.stderr:/var/log/ceph/ceph-client.admin.44883.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.44883.log.gz 2023-11-26T07:25:35.326 INFO:teuthology.orchestra.run.smithi143.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52144.log 2023-11-26T07:25:35.327 INFO:teuthology.orchestra.run.smithi143.stderr:/var/log/ceph/ceph-client.admin.43960.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46526.log 2023-11-26T07:25:35.328 INFO:teuthology.orchestra.run.smithi143.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.43960.log.gz 2023-11-26T07:25:35.329 INFO:teuthology.orchestra.run.smithi143.stderr:/var/log/ceph/ceph-client.admin.52144.log: gzip -5 --verbose -- /var/log/ceph/ceph-osd.3.log 2023-11-26T07:25:35.329 INFO:teuthology.orchestra.run.smithi143.stderr: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.52144.log.gz 2023-11-26T07:25:35.330 INFO:teuthology.orchestra.run.smithi143.stderr:/var/log/ceph/ceph-client.admin.46526.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51899.log 2023-11-26T07:25:35.330 INFO:teuthology.orchestra.run.smithi143.stderr: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.46526.log.gz 2023-11-26T07:25:35.332 INFO:teuthology.orchestra.run.smithi143.stderr:/var/log/ceph/ceph-osd.3.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50236.log 2023-11-26T07:25:35.333 INFO:teuthology.orchestra.run.smithi143.stderr:/var/log/ceph/ceph-client.admin.51899.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52419.log 2023-11-26T07:25:35.333 INFO:teuthology.orchestra.run.smithi143.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.51899.log.gz 2023-11-26T07:25:35.334 INFO:teuthology.orchestra.run.smithi143.stderr:/var/log/ceph/ceph-client.admin.50236.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.56039.log 2023-11-26T07:25:35.335 INFO:teuthology.orchestra.run.smithi143.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.50236.log.gz 2023-11-26T07:25:35.336 INFO:teuthology.orchestra.run.smithi143.stderr:/var/log/ceph/ceph-client.admin.52419.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.57410.log 2023-11-26T07:25:35.336 INFO:teuthology.orchestra.run.smithi143.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.52419.log.gz 2023-11-26T07:25:35.337 INFO:teuthology.orchestra.run.smithi143.stderr:/var/log/ceph/ceph-client.admin.56039.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45354.log 2023-11-26T07:25:35.337 INFO:teuthology.orchestra.run.smithi143.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.56039.log.gz 2023-11-26T07:25:35.339 INFO:teuthology.orchestra.run.smithi143.stderr:/var/log/ceph/ceph-client.admin.57410.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46023.log 2023-11-26T07:25:35.339 INFO:teuthology.orchestra.run.smithi143.stderr: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.57410.log.gz 2023-11-26T07:25:35.340 INFO:teuthology.orchestra.run.smithi143.stderr:/var/log/ceph/ceph-client.admin.45354.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48055.log 2023-11-26T07:25:35.341 INFO:teuthology.orchestra.run.smithi143.stderr: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.45354.log.gz 2023-11-26T07:25:35.342 INFO:teuthology.orchestra.run.smithi143.stderr:/var/log/ceph/ceph-client.admin.46023.log: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.46023.log.gz 2023-11-26T07:25:35.342 INFO:teuthology.orchestra.run.smithi143.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44676.log 2023-11-26T07:25:35.343 INFO:teuthology.orchestra.run.smithi143.stderr:/var/log/ceph/ceph-client.admin.48055.log: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.48055.log.gz 2023-11-26T07:25:35.343 INFO:teuthology.orchestra.run.smithi143.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49949.log 2023-11-26T07:25:35.344 INFO:teuthology.orchestra.run.smithi143.stderr:/var/log/ceph/ceph-client.admin.44676.log: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.44676.log.gz 2023-11-26T07:25:35.345 INFO:teuthology.orchestra.run.smithi143.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51931.log 2023-11-26T07:25:35.347 INFO:teuthology.orchestra.run.smithi143.stderr:/var/log/ceph/ceph-client.admin.49949.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48663.log 2023-11-26T07:25:35.347 INFO:teuthology.orchestra.run.smithi143.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.49949.log.gz 2023-11-26T07:25:35.348 INFO:teuthology.orchestra.run.smithi143.stderr:/var/log/ceph/ceph-client.admin.51931.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.56626.log 2023-11-26T07:25:35.348 INFO:teuthology.orchestra.run.smithi143.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.51931.log.gz 2023-11-26T07:25:35.350 INFO:teuthology.orchestra.run.smithi143.stderr:/var/log/ceph/ceph-client.admin.48663.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52620.log 2023-11-26T07:25:35.350 INFO:teuthology.orchestra.run.smithi143.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.48663.log.gz 2023-11-26T07:25:35.351 INFO:teuthology.orchestra.run.smithi143.stderr:/var/log/ceph/ceph-client.admin.56626.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46211.log 2023-11-26T07:25:35.351 INFO:teuthology.orchestra.run.smithi143.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.56626.log.gz 2023-11-26T07:25:35.352 INFO:teuthology.orchestra.run.smithi143.stderr:/var/log/ceph/ceph-client.admin.52620.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.60977.log 2023-11-26T07:25:35.353 INFO:teuthology.orchestra.run.smithi143.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.52620.log.gz 2023-11-26T07:25:35.354 INFO:teuthology.orchestra.run.smithi143.stderr:/var/log/ceph/ceph-client.admin.46211.log: gzip 89.5% -- replaced with /var/log/ceph/ceph-client.admin.46211.log.gz -5 2023-11-26T07:25:35.354 INFO:teuthology.orchestra.run.smithi143.stderr: --verbose -- /var/log/ceph/ceph-client.admin.52964.log 2023-11-26T07:25:35.356 INFO:teuthology.orchestra.run.smithi143.stderr:/var/log/ceph/ceph-client.admin.60977.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53509.log 2023-11-26T07:25:35.356 INFO:teuthology.orchestra.run.smithi143.stderr: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.60977.log.gz 2023-11-26T07:25:35.357 INFO:teuthology.orchestra.run.smithi143.stderr:/var/log/ceph/ceph-client.admin.52964.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51656.log 2023-11-26T07:25:35.357 INFO:teuthology.orchestra.run.smithi143.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.52964.log.gz 2023-11-26T07:25:35.358 INFO:teuthology.orchestra.run.smithi143.stderr:/var/log/ceph/ceph-client.admin.53509.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.55102.log 2023-11-26T07:25:35.359 INFO:teuthology.orchestra.run.smithi143.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.53509.log.gz 2023-11-26T07:25:35.360 INFO:teuthology.orchestra.run.smithi143.stderr:/var/log/ceph/ceph-client.admin.51656.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51588.log 2023-11-26T07:25:35.360 INFO:teuthology.orchestra.run.smithi143.stderr: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.51656.log.gz 2023-11-26T07:25:35.361 INFO:teuthology.orchestra.run.smithi143.stderr:/var/log/ceph/ceph-client.admin.55102.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54672.log 2023-11-26T07:25:35.361 INFO:teuthology.orchestra.run.smithi143.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.55102.log.gz 2023-11-26T07:25:35.362 INFO:teuthology.orchestra.run.smithi143.stderr:/var/log/ceph/ceph-client.admin.51588.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47426.log 2023-11-26T07:25:35.363 INFO:teuthology.orchestra.run.smithi143.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.51588.log.gz 2023-11-26T07:25:35.364 INFO:teuthology.orchestra.run.smithi143.stderr:/var/log/ceph/ceph-client.admin.54672.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45458.log 2023-11-26T07:25:35.364 INFO:teuthology.orchestra.run.smithi143.stderr: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.54672.log.gz 2023-11-26T07:25:35.365 INFO:teuthology.orchestra.run.smithi143.stderr:/var/log/ceph/ceph-client.admin.47426.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45888.log 2023-11-26T07:25:35.366 INFO:teuthology.orchestra.run.smithi143.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.47426.log.gz 2023-11-26T07:25:35.367 INFO:teuthology.orchestra.run.smithi143.stderr:/var/log/ceph/ceph-client.admin.45458.log: 88.4%gzip -- replaced with /var/log/ceph/ceph-client.admin.45458.log.gz -5 2023-11-26T07:25:35.367 INFO:teuthology.orchestra.run.smithi143.stderr: --verbose -- /var/log/ceph/ceph-client.admin.58025.log 2023-11-26T07:25:35.368 INFO:teuthology.orchestra.run.smithi143.stderr:/var/log/ceph/ceph-client.admin.45888.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50720.log 2023-11-26T07:25:35.368 INFO:teuthology.orchestra.run.smithi143.stderr: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.45888.log.gz 2023-11-26T07:25:35.370 INFO:teuthology.orchestra.run.smithi143.stderr:/var/log/ceph/ceph-client.admin.58025.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53640.log 2023-11-26T07:25:35.370 INFO:teuthology.orchestra.run.smithi143.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.58025.log.gz 2023-11-26T07:25:35.371 INFO:teuthology.orchestra.run.smithi143.stderr:/var/log/ceph/ceph-client.admin.50720.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54998.log 2023-11-26T07:25:35.372 INFO:teuthology.orchestra.run.smithi143.stderr: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.50720.log.gz 2023-11-26T07:25:35.373 INFO:teuthology.orchestra.run.smithi143.stderr:/var/log/ceph/ceph-client.admin.53640.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44125.log 2023-11-26T07:25:35.373 INFO:teuthology.orchestra.run.smithi143.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.53640.log.gz 2023-11-26T07:25:35.374 INFO:teuthology.orchestra.run.smithi143.stderr:/var/log/ceph/ceph-client.admin.54998.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.59405.log 2023-11-26T07:25:35.374 INFO:teuthology.orchestra.run.smithi143.stderr: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.54998.log.gz 2023-11-26T07:25:35.375 INFO:teuthology.orchestra.run.smithi188.stderr: 91.3% -- replaced with /var/log/ceph/ceph-mon.b.log.gz 2023-11-26T07:25:35.375 INFO:teuthology.orchestra.run.smithi143.stderr:/var/log/ceph/ceph-client.admin.44125.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.61110.log 2023-11-26T07:25:35.376 INFO:teuthology.orchestra.run.smithi143.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.44125.log.gz 2023-11-26T07:25:35.377 INFO:teuthology.orchestra.run.smithi143.stderr:/var/log/ceph/ceph-client.admin.59405.log: gzip -5 87.8% --verbose -- -- replaced with /var/log/ceph/ceph-client.admin.59405.log.gz /var/log/ceph/ceph-client.admin.47769.log 2023-11-26T07:25:35.377 INFO:teuthology.orchestra.run.smithi143.stderr: 2023-11-26T07:25:35.378 INFO:teuthology.orchestra.run.smithi143.stderr:/var/log/ceph/ceph-client.admin.61110.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.2.45979.log 2023-11-26T07:25:35.379 INFO:teuthology.orchestra.run.smithi143.stderr: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.61110.log.gz 2023-11-26T07:25:35.380 INFO:teuthology.orchestra.run.smithi143.stderr:/var/log/ceph/ceph-client.admin.47769.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.57462.log 2023-11-26T07:25:35.380 INFO:teuthology.orchestra.run.smithi143.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.47769.log.gz 2023-11-26T07:25:35.381 INFO:teuthology.orchestra.run.smithi188.stderr: 92.0% -- replaced with /var/log/ceph/ceph-mon.c.log.gz 2023-11-26T07:25:35.381 INFO:teuthology.orchestra.run.smithi143.stderr:/var/log/ceph/ceph-client.2.45979.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53692.log 2023-11-26T07:25:35.382 INFO:teuthology.orchestra.run.smithi143.stderr: 90.7% -- replaced with /var/log/ceph/ceph-client.2.45979.log.gz 2023-11-26T07:25:35.383 INFO:teuthology.orchestra.run.smithi143.stderr:/var/log/ceph/ceph-client.admin.57462.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53226.log 2023-11-26T07:25:35.383 INFO:teuthology.orchestra.run.smithi143.stderr: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.57462.log.gz 2023-11-26T07:25:35.384 INFO:teuthology.orchestra.run.smithi143.stderr:/var/log/ceph/ceph-client.admin.53692.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.58181.log 2023-11-26T07:25:35.385 INFO:teuthology.orchestra.run.smithi143.stderr: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.53692.log.gz 2023-11-26T07:25:35.386 INFO:teuthology.orchestra.run.smithi143.stderr:/var/log/ceph/ceph-client.admin.53226.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46929.log 2023-11-26T07:25:35.386 INFO:teuthology.orchestra.run.smithi143.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.53226.log.gz 2023-11-26T07:25:35.387 INFO:teuthology.orchestra.run.smithi143.stderr:/var/log/ceph/ceph-client.admin.58181.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44151.log 2023-11-26T07:25:35.387 INFO:teuthology.orchestra.run.smithi143.stderr: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.58181.log.gz 2023-11-26T07:25:35.389 INFO:teuthology.orchestra.run.smithi143.stderr:/var/log/ceph/ceph-client.admin.46929.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50210.log 2023-11-26T07:25:35.389 INFO:teuthology.orchestra.run.smithi143.stderr: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.46929.log.gz 2023-11-26T07:25:35.390 INFO:teuthology.orchestra.run.smithi143.stderr:/var/log/ceph/ceph-client.admin.44151.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45562.log 2023-11-26T07:25:35.390 INFO:teuthology.orchestra.run.smithi143.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.44151.log.gz 2023-11-26T07:25:35.391 INFO:teuthology.orchestra.run.smithi143.stderr:/var/log/ceph/ceph-client.admin.50210.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44887.log 2023-11-26T07:25:35.391 INFO:teuthology.orchestra.run.smithi143.stderr: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.50210.log.gz 2023-11-26T07:25:35.393 INFO:teuthology.orchestra.run.smithi143.stderr:/var/log/ceph/ceph-client.admin.45562.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.60389.log 2023-11-26T07:25:35.393 INFO:teuthology.orchestra.run.smithi143.stderr: 89.5% -- replaced with /var/log/ceph/ceph-client.admin.45562.log.gz 2023-11-26T07:25:35.394 INFO:teuthology.orchestra.run.smithi143.stderr:/var/log/ceph/ceph-client.admin.44887.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.58951.log 2023-11-26T07:25:35.395 INFO:teuthology.orchestra.run.smithi143.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.44887.log.gz 2023-11-26T07:25:35.396 INFO:teuthology.orchestra.run.smithi143.stderr:/var/log/ceph/ceph-client.admin.60389.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54541.log 2023-11-26T07:25:35.396 INFO:teuthology.orchestra.run.smithi143.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.60389.log.gz 2023-11-26T07:25:35.397 INFO:teuthology.orchestra.run.smithi143.stderr:/var/log/ceph/ceph-client.admin.58951.log: gzip -5 --verbose -- /var/log/ceph/ceph-osd.1.log 2023-11-26T07:25:35.397 INFO:teuthology.orchestra.run.smithi143.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.58951.log.gz 2023-11-26T07:25:35.399 INFO:teuthology.orchestra.run.smithi143.stderr:/var/log/ceph/ceph-client.admin.54541.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.57127.log 2023-11-26T07:25:35.399 INFO:teuthology.orchestra.run.smithi143.stderr: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.54541.log.gz 2023-11-26T07:25:35.400 INFO:teuthology.orchestra.run.smithi143.stderr:/var/log/ceph/ceph-osd.1.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45666.log 2023-11-26T07:25:35.402 INFO:teuthology.orchestra.run.smithi143.stderr:/var/log/ceph/ceph-client.admin.57127.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53328.log 2023-11-26T07:25:35.402 INFO:teuthology.orchestra.run.smithi143.stderr: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.57127.log.gz 2023-11-26T07:25:35.403 INFO:teuthology.orchestra.run.smithi143.stderr:/var/log/ceph/ceph-client.admin.45666.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45077.log 2023-11-26T07:25:35.403 INFO:teuthology.orchestra.run.smithi143.stderr: 91.5% -- replaced with /var/log/ceph/ceph-client.admin.45666.log.gz 2023-11-26T07:25:35.404 INFO:teuthology.orchestra.run.smithi143.stderr:/var/log/ceph/ceph-client.admin.53328.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.59616.log 2023-11-26T07:25:35.405 INFO:teuthology.orchestra.run.smithi143.stderr: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.53328.log.gz 2023-11-26T07:25:35.405 INFO:teuthology.orchestra.run.smithi143.stderr:/var/log/ceph/ceph-client.admin.45077.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.45077.log.gz 2023-11-26T07:25:35.407 INFO:teuthology.orchestra.run.smithi143.stderr:/var/log/ceph/ceph-client.admin.59616.log: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.59616.log.gz 2023-11-26T07:25:35.407 INFO:teuthology.orchestra.run.smithi143.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.57781.log 2023-11-26T07:25:35.409 INFO:teuthology.orchestra.run.smithi143.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.60311.log 2023-11-26T07:25:35.411 INFO:teuthology.orchestra.run.smithi143.stderr:/var/log/ceph/ceph-client.admin.57781.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50144.log 2023-11-26T07:25:35.411 INFO:teuthology.orchestra.run.smithi143.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.57781.log.gz 2023-11-26T07:25:35.412 INFO:teuthology.orchestra.run.smithi143.stderr:/var/log/ceph/ceph-client.admin.60311.log: gzip 87.8% -5 --verbose -- replaced with /var/log/ceph/ceph-client.admin.60311.log.gz -- 2023-11-26T07:25:35.412 INFO:teuthology.orchestra.run.smithi143.stderr: /var/log/ceph/ceph-client.admin.46263.log 2023-11-26T07:25:35.413 INFO:teuthology.orchestra.run.smithi143.stderr:/var/log/ceph/ceph-client.admin.50144.log: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.50144.log.gz 2023-11-26T07:25:35.414 INFO:teuthology.orchestra.run.smithi143.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51454.log 2023-11-26T07:25:35.415 INFO:teuthology.orchestra.run.smithi143.stderr:/var/log/ceph/ceph-client.admin.46263.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48257.log 2023-11-26T07:25:35.416 INFO:teuthology.orchestra.run.smithi143.stderr: 89.6% -- replaced with /var/log/ceph/ceph-client.admin.46263.log.gz 2023-11-26T07:25:35.417 INFO:teuthology.orchestra.run.smithi143.stderr:/var/log/ceph/ceph-client.admin.51454.log: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.51454.log.gz 2023-11-26T07:25:35.418 INFO:teuthology.orchestra.run.smithi143.stderr:/var/log/ceph/ceph-client.admin.48257.log: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.48257.log.gz 2023-11-26T07:25:35.419 INFO:teuthology.orchestra.run.smithi143.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.58677.log 2023-11-26T07:25:35.420 INFO:teuthology.orchestra.run.smithi143.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50590.log 2023-11-26T07:25:35.422 INFO:teuthology.orchestra.run.smithi143.stderr:/var/log/ceph/ceph-client.admin.58677.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47163.log 2023-11-26T07:25:35.422 INFO:teuthology.orchestra.run.smithi143.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.58677.log.gz 2023-11-26T07:25:35.423 INFO:teuthology.orchestra.run.smithi143.stderr:/var/log/ceph/ceph-client.admin.50590.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52084.log 2023-11-26T07:25:35.423 INFO:teuthology.orchestra.run.smithi143.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.50590.log.gz 2023-11-26T07:25:35.424 INFO:teuthology.orchestra.run.smithi143.stderr:/var/log/ceph/ceph-client.admin.47163.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51716.log 2023-11-26T07:25:35.425 INFO:teuthology.orchestra.run.smithi143.stderr: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.47163.log.gz 2023-11-26T07:25:35.426 INFO:teuthology.orchestra.run.smithi143.stderr:/var/log/ceph/ceph-client.admin.52084.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47795.log 2023-11-26T07:25:35.426 INFO:teuthology.orchestra.run.smithi143.stderr: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.52084.log.gz 2023-11-26T07:25:35.427 INFO:teuthology.orchestra.run.smithi143.stderr:/var/log/ceph/ceph-client.admin.51716.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.60717.log 2023-11-26T07:25:35.428 INFO:teuthology.orchestra.run.smithi143.stderr: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.51716.log.gz 2023-11-26T07:25:35.428 INFO:teuthology.orchestra.run.smithi143.stderr:/var/log/ceph/ceph-client.admin.47795.log: 89.4% -- replaced with /var/log/ceph/ceph-client.admin.47795.log.gz 2023-11-26T07:25:35.430 INFO:teuthology.orchestra.run.smithi188.stderr: 94.4% -- replaced with /var/log/ceph/ceph-mgr.x.log.gz 2023-11-26T07:25:35.430 INFO:teuthology.orchestra.run.smithi143.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44060.log 2023-11-26T07:25:35.431 INFO:teuthology.orchestra.run.smithi143.stderr:/var/log/ceph/ceph-client.admin.60717.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.56241.log 2023-11-26T07:25:35.431 INFO:teuthology.orchestra.run.smithi143.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.60717.log.gz 2023-11-26T07:25:35.432 INFO:teuthology.orchestra.run.smithi143.stderr:/var/log/ceph/ceph-client.admin.44060.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38185.log 2023-11-26T07:25:35.432 INFO:teuthology.orchestra.run.smithi143.stderr: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.44060.log.gz 2023-11-26T07:25:35.433 INFO:teuthology.orchestra.run.smithi143.stderr:/var/log/ceph/ceph-client.admin.56241.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.60925.log 2023-11-26T07:25:35.434 INFO:teuthology.orchestra.run.smithi143.stderr: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.56241.log.gz 2023-11-26T07:25:35.435 INFO:teuthology.orchestra.run.smithi143.stderr:/var/log/ceph/ceph-client.admin.38185.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.56495.log 2023-11-26T07:25:35.435 INFO:teuthology.orchestra.run.smithi143.stderr: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.38185.log.gz 2023-11-26T07:25:35.436 INFO:teuthology.orchestra.run.smithi143.stderr:/var/log/ceph/ceph-client.admin.60925.log: gzip -5 --verbose -- 88.4% /var/log/ceph/ceph-client.admin.53148.log -- replaced with /var/log/ceph/ceph-client.admin.60925.log.gz 2023-11-26T07:25:35.436 INFO:teuthology.orchestra.run.smithi143.stderr: 2023-11-26T07:25:35.438 INFO:teuthology.orchestra.run.smithi143.stderr:/var/log/ceph/ceph-client.admin.56495.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.55050.log 2023-11-26T07:25:35.438 INFO:teuthology.orchestra.run.smithi143.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.56495.log.gz 2023-11-26T07:25:35.439 INFO:teuthology.orchestra.run.smithi143.stderr:/var/log/ceph/ceph-client.admin.53148.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.53148.log.gz 2023-11-26T07:25:35.440 INFO:teuthology.orchestra.run.smithi143.stderr:/var/log/ceph/ceph-client.admin.55050.log: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.55050.log.gz 2023-11-26T07:25:35.441 INFO:teuthology.orchestra.run.smithi143.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.55680.log 2023-11-26T07:25:35.442 INFO:teuthology.orchestra.run.smithi143.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53122.log 2023-11-26T07:25:35.444 INFO:teuthology.orchestra.run.smithi143.stderr:/var/log/ceph/ceph-client.admin.55680.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52542.log 2023-11-26T07:25:35.444 INFO:teuthology.orchestra.run.smithi143.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.55680.log.gz 2023-11-26T07:25:35.446 INFO:teuthology.orchestra.run.smithi143.stderr:/var/log/ceph/ceph-client.admin.53122.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49533.log 2023-11-26T07:25:35.446 INFO:teuthology.orchestra.run.smithi143.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.53122.log.gz 2023-11-26T07:25:35.447 INFO:teuthology.orchestra.run.smithi143.stderr:/var/log/ceph/ceph-client.admin.52542.log: 87.7%gzip -- replaced with /var/log/ceph/ceph-client.admin.52542.log.gz -5 2023-11-26T07:25:35.447 INFO:teuthology.orchestra.run.smithi143.stderr: --verbose -- /var/log/ceph/ceph-client.admin.55438.log 2023-11-26T07:25:35.448 INFO:teuthology.orchestra.run.smithi143.stderr:/var/log/ceph/ceph-client.admin.49533.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51350.log 2023-11-26T07:25:35.448 INFO:teuthology.orchestra.run.smithi143.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.49533.log.gz 2023-11-26T07:25:35.450 INFO:teuthology.orchestra.run.smithi143.stderr:/var/log/ceph/ceph-client.admin.55438.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.57755.log 2023-11-26T07:25:35.450 INFO:teuthology.orchestra.run.smithi143.stderr: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.55438.log.gz 2023-11-26T07:25:35.451 INFO:teuthology.orchestra.run.smithi143.stderr:/var/log/ceph/ceph-client.admin.51350.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.58925.log 2023-11-26T07:25:35.451 INFO:teuthology.orchestra.run.smithi143.stderr: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.51350.log.gz 2023-11-26T07:25:35.452 INFO:teuthology.orchestra.run.smithi143.stderr:/var/log/ceph/ceph-client.admin.57755.log: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.57755.log.gz 2023-11-26T07:25:35.454 INFO:teuthology.orchestra.run.smithi143.stderr:/var/log/ceph/ceph-client.admin.58925.log: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.58925.log.gz 2023-11-26T07:25:35.454 INFO:teuthology.orchestra.run.smithi143.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.57677.log 2023-11-26T07:25:35.456 INFO:teuthology.orchestra.run.smithi143.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.59353.log 2023-11-26T07:25:35.457 INFO:teuthology.orchestra.run.smithi143.stderr:/var/log/ceph/ceph-client.admin.57677.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48003.log 2023-11-26T07:25:35.457 INFO:teuthology.orchestra.run.smithi143.stderr: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.57677.log.gz 2023-11-26T07:25:35.459 INFO:teuthology.orchestra.run.smithi143.stderr:/var/log/ceph/ceph-client.admin.59353.log: gzip 87.4% -- replaced with /var/log/ceph/ceph-client.admin.59353.log.gz -5 2023-11-26T07:25:35.459 INFO:teuthology.orchestra.run.smithi143.stderr: --verbose -- /var/log/ceph/ceph-client.admin.54307.log 2023-11-26T07:25:35.460 INFO:teuthology.orchestra.run.smithi143.stderr:/var/log/ceph/ceph-client.admin.48003.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52936.log 2023-11-26T07:25:35.460 INFO:teuthology.orchestra.run.smithi143.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.48003.log.gz 2023-11-26T07:25:35.462 INFO:teuthology.orchestra.run.smithi143.stderr:/var/log/ceph/ceph-client.admin.54307.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54045.log 2023-11-26T07:25:35.462 INFO:teuthology.orchestra.run.smithi143.stderr: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.54307.log.gz 2023-11-26T07:25:35.463 INFO:teuthology.orchestra.run.smithi143.stderr:/var/log/ceph/ceph-client.admin.52936.log: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.52936.log.gz 2023-11-26T07:25:35.464 INFO:teuthology.orchestra.run.smithi143.stderr:/var/log/ceph/ceph-client.admin.54045.log: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.54045.log.gz 2023-11-26T07:25:35.465 INFO:teuthology.orchestra.run.smithi143.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49455.log 2023-11-26T07:25:35.466 INFO:teuthology.orchestra.run.smithi143.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53278.log 2023-11-26T07:25:35.468 INFO:teuthology.orchestra.run.smithi143.stderr:/var/log/ceph/ceph-client.admin.49455.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.59234.log 2023-11-26T07:25:35.468 INFO:teuthology.orchestra.run.smithi143.stderr: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.49455.log.gz 2023-11-26T07:25:35.470 INFO:teuthology.orchestra.run.smithi143.stderr:/var/log/ceph/ceph-client.admin.53278.log: /var/log/ceph/ceph-client.admin.59234.log: 91.3% -- replaced with /var/log/ceph/ceph-client.admin.53278.log.gz 2023-11-26T07:25:35.470 INFO:teuthology.orchestra.run.smithi143.stderr: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.59234.log.gz 2023-11-26T07:25:35.471 INFO:teuthology.orchestra.run.smithi143.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.57971.log 2023-11-26T07:25:35.472 INFO:teuthology.orchestra.run.smithi143.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.56811.log 2023-11-26T07:25:35.474 INFO:teuthology.orchestra.run.smithi143.stderr:/var/log/ceph/ceph-client.admin.57971.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.55862.log 2023-11-26T07:25:35.474 INFO:teuthology.orchestra.run.smithi143.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.57971.log.gz 2023-11-26T07:25:35.475 INFO:teuthology.orchestra.run.smithi143.stderr:/var/log/ceph/ceph-client.admin.56811.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47033.log 2023-11-26T07:25:35.476 INFO:teuthology.orchestra.run.smithi143.stderr: 89.5% -- replaced with /var/log/ceph/ceph-client.admin.56811.log.gz 2023-11-26T07:25:35.476 INFO:teuthology.orchestra.run.smithi188.stderr: 93.8% -- replaced with /var/log/ceph/ceph-osd.5.log.gz 2023-11-26T07:25:35.477 INFO:teuthology.orchestra.run.smithi143.stderr:/var/log/ceph/ceph-client.admin.55862.log: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.55862.log.gz 2023-11-26T07:25:35.478 INFO:teuthology.orchestra.run.smithi143.stderr:/var/log/ceph/ceph-client.admin.47033.log: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.47033.log.gz 2023-11-26T07:25:35.478 INFO:teuthology.orchestra.run.smithi143.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53937.log 2023-11-26T07:25:35.480 INFO:teuthology.orchestra.run.smithi143.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53096.log 2023-11-26T07:25:35.482 INFO:teuthology.orchestra.run.smithi143.stderr:/var/log/ceph/ceph-client.admin.53937.log: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.53937.log.gz 2023-11-26T07:25:35.483 INFO:teuthology.orchestra.run.smithi143.stderr:/var/log/ceph/ceph-client.admin.53096.log: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.53096.log.gz 2023-11-26T07:25:35.483 INFO:teuthology.orchestra.run.smithi143.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52211.log 2023-11-26T07:25:35.485 INFO:teuthology.orchestra.run.smithi143.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51239.log 2023-11-26T07:25:35.486 INFO:teuthology.orchestra.run.smithi143.stderr:/var/log/ceph/ceph-client.admin.52211.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53018.log 2023-11-26T07:25:35.486 INFO:teuthology.orchestra.run.smithi143.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.52211.log.gz 2023-11-26T07:25:35.488 INFO:teuthology.orchestra.run.smithi143.stderr:/var/log/ceph/ceph-client.admin.51239.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52237.log 2023-11-26T07:25:35.488 INFO:teuthology.orchestra.run.smithi143.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.51239.log.gz 2023-11-26T07:25:35.489 INFO:teuthology.orchestra.run.smithi143.stderr:/var/log/ceph/ceph-client.admin.53018.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50772.log 2023-11-26T07:25:35.489 INFO:teuthology.orchestra.run.smithi143.stderr: 89.6% -- replaced with /var/log/ceph/ceph-client.admin.53018.log.gz 2023-11-26T07:25:35.490 INFO:teuthology.orchestra.run.smithi143.stderr:/var/log/ceph/ceph-client.admin.52237.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49346.log 2023-11-26T07:25:35.491 INFO:teuthology.orchestra.run.smithi143.stderr: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.52237.log.gz 2023-11-26T07:25:35.492 INFO:teuthology.orchestra.run.smithi143.stderr:/var/log/ceph/ceph-client.admin.50772.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.55621.log 2023-11-26T07:25:35.493 INFO:teuthology.orchestra.run.smithi143.stderr: 91.3% -- replaced with /var/log/ceph/ceph-client.admin.50772.log.gz/var/log/ceph/ceph-client.admin.49346.log: 2023-11-26T07:25:35.493 INFO:teuthology.orchestra.run.smithi143.stderr: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.49346.log.gz 2023-11-26T07:25:35.494 INFO:teuthology.orchestra.run.smithi143.stderr:/var/log/ceph/ceph-client.admin.55621.log: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.55621.log.gz 2023-11-26T07:25:35.495 INFO:teuthology.orchestra.run.smithi143.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44888.log 2023-11-26T07:25:35.497 INFO:teuthology.orchestra.run.smithi143.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50925.log 2023-11-26T07:25:35.498 INFO:teuthology.orchestra.run.smithi143.stderr:/var/log/ceph/ceph-client.admin.44888.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.59457.log 2023-11-26T07:25:35.498 INFO:teuthology.orchestra.run.smithi143.stderr: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.44888.log.gz 2023-11-26T07:25:35.499 INFO:teuthology.orchestra.run.smithi143.stderr:/var/log/ceph/ceph-client.admin.50925.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45328.log 2023-11-26T07:25:35.500 INFO:teuthology.orchestra.run.smithi143.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.50925.log.gz 2023-11-26T07:25:35.501 INFO:teuthology.orchestra.run.smithi143.stderr:/var/log/ceph/ceph-client.admin.59457.log: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.59457.log.gz 2023-11-26T07:25:35.502 INFO:teuthology.orchestra.run.smithi143.stderr:/var/log/ceph/ceph-client.admin.45328.log: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.45328.log.gz 2023-11-26T07:25:35.503 INFO:teuthology.orchestra.run.smithi143.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47457.log 2023-11-26T07:25:35.504 INFO:teuthology.orchestra.run.smithi143.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.56267.log 2023-11-26T07:25:35.506 INFO:teuthology.orchestra.run.smithi143.stderr:/var/log/ceph/ceph-client.admin.47457.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48309.log 2023-11-26T07:25:35.506 INFO:teuthology.orchestra.run.smithi143.stderr: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.47457.log.gz 2023-11-26T07:25:35.507 INFO:teuthology.orchestra.run.smithi143.stderr:/var/log/ceph/ceph-client.admin.56267.log: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.56267.log.gz 2023-11-26T07:25:35.509 INFO:teuthology.orchestra.run.smithi143.stderr:/var/log/ceph/ceph-client.admin.48309.log: 89.7% -- replaced with /var/log/ceph/ceph-client.admin.48309.log.gz 2023-11-26T07:25:35.509 INFO:teuthology.orchestra.run.smithi143.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49262.log 2023-11-26T07:25:35.510 INFO:teuthology.orchestra.run.smithi143.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.58155.log 2023-11-26T07:25:35.512 INFO:teuthology.orchestra.run.smithi143.stderr:/var/log/ceph/ceph-client.admin.49262.log: gzip -5 --verbose -- 87.5% /var/log/ceph/ceph-client.admin.56863.log 2023-11-26T07:25:35.512 INFO:teuthology.orchestra.run.smithi143.stderr: -- replaced with /var/log/ceph/ceph-client.admin.49262.log.gz 2023-11-26T07:25:35.513 INFO:teuthology.orchestra.run.smithi143.stderr:/var/log/ceph/ceph-client.admin.58155.log: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.58155.log.gz 2023-11-26T07:25:35.513 INFO:teuthology.orchestra.run.smithi143.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.56915.log 2023-11-26T07:25:35.515 INFO:teuthology.orchestra.run.smithi143.stderr:/var/log/ceph/ceph-client.admin.56863.log: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.56863.log.gz 2023-11-26T07:25:35.515 INFO:teuthology.orchestra.run.smithi143.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.58651.log 2023-11-26T07:25:35.516 INFO:teuthology.orchestra.run.smithi143.stderr:/var/log/ceph/ceph-client.admin.56915.log: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.56915.log.gz 2023-11-26T07:25:35.517 INFO:teuthology.orchestra.run.smithi143.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49640.log 2023-11-26T07:25:35.518 INFO:teuthology.orchestra.run.smithi143.stderr:/var/log/ceph/ceph-client.admin.58651.log: 88.7%gzip -- replaced with /var/log/ceph/ceph-client.admin.58651.log.gz -5 2023-11-26T07:25:35.518 INFO:teuthology.orchestra.run.smithi143.stderr: --verbose -- /var/log/ceph/ceph-client.admin.55385.log 2023-11-26T07:25:35.520 INFO:teuthology.orchestra.run.smithi143.stderr:/var/log/ceph/ceph-client.admin.49640.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.57890.log 2023-11-26T07:25:35.520 INFO:teuthology.orchestra.run.smithi143.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.49640.log.gz 2023-11-26T07:25:35.521 INFO:teuthology.orchestra.run.smithi143.stderr:/var/log/ceph/ceph-client.admin.55385.log: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.55385.log.gz 2023-11-26T07:25:35.522 INFO:teuthology.orchestra.run.smithi143.stderr:/var/log/ceph/ceph-client.admin.57890.log: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.57890.log.gz 2023-11-26T07:25:35.523 INFO:teuthology.orchestra.run.smithi143.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48505.log 2023-11-26T07:25:35.524 INFO:teuthology.orchestra.run.smithi143.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47899.log 2023-11-26T07:25:35.526 INFO:teuthology.orchestra.run.smithi143.stderr:/var/log/ceph/ceph-client.admin.48505.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.57703.log 2023-11-26T07:25:35.526 INFO:teuthology.orchestra.run.smithi143.stderr: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.48505.log.gz 2023-11-26T07:25:35.527 INFO:teuthology.orchestra.run.smithi143.stderr:/var/log/ceph/ceph-client.admin.47899.log: gzip -5 --verbose -- /var/log/ceph/ceph.log 2023-11-26T07:25:35.527 INFO:teuthology.orchestra.run.smithi143.stderr: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.47899.log.gz 2023-11-26T07:25:35.529 INFO:teuthology.orchestra.run.smithi143.stderr:/var/log/ceph/ceph-client.admin.57703.log: gzip -5 88.3% --verbose -- -- replaced with /var/log/ceph/ceph-client.admin.57703.log.gz /var/log/ceph/ceph-client.admin.44626.log 2023-11-26T07:25:35.529 INFO:teuthology.orchestra.run.smithi143.stderr: 2023-11-26T07:25:35.532 INFO:teuthology.orchestra.run.smithi143.stderr:/var/log/ceph/ceph.log: /var/log/ceph/ceph-client.admin.44626.log: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.44626.log.gz 2023-11-26T07:25:35.532 INFO:teuthology.orchestra.run.smithi143.stderr: 86.4% -- replaced with /var/log/ceph/ceph.log.gz 2023-11-26T07:25:35.532 INFO:teuthology.orchestra.run.smithi143.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.58389.log 2023-11-26T07:25:35.533 INFO:teuthology.orchestra.run.smithi143.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.60618.log 2023-11-26T07:25:35.535 INFO:teuthology.orchestra.run.smithi143.stderr:/var/log/ceph/ceph-client.admin.58389.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53854.log 2023-11-26T07:25:35.535 INFO:teuthology.orchestra.run.smithi143.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.58389.log.gz 2023-11-26T07:25:35.537 INFO:teuthology.orchestra.run.smithi143.stderr:/var/log/ceph/ceph-client.admin.60618.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48637.log 2023-11-26T07:25:35.537 INFO:teuthology.orchestra.run.smithi143.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.60618.log.gz 2023-11-26T07:25:35.538 INFO:teuthology.orchestra.run.smithi143.stderr:/var/log/ceph/ceph-client.admin.53854.log: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.53854.log.gz 2023-11-26T07:25:35.538 INFO:teuthology.orchestra.run.smithi143.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.60467.log 2023-11-26T07:25:35.540 INFO:teuthology.orchestra.run.smithi143.stderr:/var/log/ceph/ceph-client.admin.48637.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54097.log 2023-11-26T07:25:35.540 INFO:teuthology.orchestra.run.smithi143.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.48637.log.gz 2023-11-26T07:25:35.541 INFO:teuthology.orchestra.run.smithi143.stderr:/var/log/ceph/ceph-client.admin.60467.log: gzip -5 --verbose -- /var/log/ceph/ceph.tmp-client.admin.33215.log 2023-11-26T07:25:35.541 INFO:teuthology.orchestra.run.smithi143.stderr: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.60467.log.gz 2023-11-26T07:25:35.542 INFO:teuthology.orchestra.run.smithi143.stderr:/var/log/ceph/ceph-client.admin.54097.log: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.54097.log.gz 2023-11-26T07:25:35.543 INFO:teuthology.orchestra.run.smithi143.stderr:/var/log/ceph/ceph.tmp-client.admin.33215.log: 0.0% -- replaced with /var/log/ceph/ceph.tmp-client.admin.33215.log.gz 2023-11-26T07:25:35.544 INFO:teuthology.orchestra.run.smithi143.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.55307.log 2023-11-26T07:25:35.545 INFO:teuthology.orchestra.run.smithi143.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51324.log 2023-11-26T07:25:35.547 INFO:teuthology.orchestra.run.smithi143.stderr:/var/log/ceph/ceph-client.admin.55307.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44870.log 2023-11-26T07:25:35.547 INFO:teuthology.orchestra.run.smithi143.stderr: 89.7% -- replaced with /var/log/ceph/ceph-client.admin.55307.log.gz 2023-11-26T07:25:35.548 INFO:teuthology.orchestra.run.smithi143.stderr:/var/log/ceph/ceph-client.admin.51324.log: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.51324.log.gz 2023-11-26T07:25:35.549 INFO:teuthology.orchestra.run.smithi143.stderr:/var/log/ceph/ceph-client.admin.44870.log: 89.7% -- replaced with /var/log/ceph/ceph-client.admin.44870.log.gz 2023-11-26T07:25:35.550 INFO:teuthology.orchestra.run.smithi143.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52393.log 2023-11-26T07:25:35.551 INFO:teuthology.orchestra.run.smithi143.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.59029.log 2023-11-26T07:25:35.553 INFO:teuthology.orchestra.run.smithi143.stderr:/var/log/ceph/ceph-client.admin.52393.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44675.log 2023-11-26T07:25:35.553 INFO:teuthology.orchestra.run.smithi143.stderr: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.52393.log.gz 2023-11-26T07:25:35.554 INFO:teuthology.orchestra.run.smithi143.stderr:/var/log/ceph/ceph-client.admin.59029.log: 91.4% -- replaced with /var/log/ceph/ceph-client.admin.59029.log.gz 2023-11-26T07:25:35.556 INFO:teuthology.orchestra.run.smithi143.stderr:/var/log/ceph/ceph-client.admin.44675.log: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.44675.log.gz 2023-11-26T07:25:35.556 INFO:teuthology.orchestra.run.smithi143.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53802.log 2023-11-26T07:25:35.557 INFO:teuthology.orchestra.run.smithi188.stderr: 94.0% -- replaced with /var/log/ceph/ceph-osd.6.log.gz 2023-11-26T07:25:35.558 INFO:teuthology.orchestra.run.smithi143.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52367.log 2023-11-26T07:25:35.559 INFO:teuthology.orchestra.run.smithi143.stderr:/var/log/ceph/ceph-client.admin.53802.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44178.log 2023-11-26T07:25:35.559 INFO:teuthology.orchestra.run.smithi143.stderr: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.53802.log.gz 2023-11-26T07:25:35.560 INFO:teuthology.orchestra.run.smithi143.stderr:/var/log/ceph/ceph-client.admin.52367.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.59208.log 2023-11-26T07:25:35.561 INFO:teuthology.orchestra.run.smithi143.stderr: 89.7% -- replaced with /var/log/ceph/ceph-client.admin.52367.log.gz 2023-11-26T07:25:35.561 INFO:teuthology.orchestra.run.smithi143.stderr:/var/log/ceph/ceph-client.admin.44178.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.44178.log.gz 2023-11-26T07:25:35.563 INFO:teuthology.orchestra.run.smithi143.stderr:/var/log/ceph/ceph-client.admin.59208.log: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.59208.log.gz 2023-11-26T07:25:35.564 INFO:teuthology.orchestra.run.smithi143.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49796.log 2023-11-26T07:25:35.565 INFO:teuthology.orchestra.run.smithi143.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.0.55259.log 2023-11-26T07:25:35.567 INFO:teuthology.orchestra.run.smithi143.stderr:/var/log/ceph/ceph-client.admin.49796.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.56745.log 2023-11-26T07:25:35.567 INFO:teuthology.orchestra.run.smithi143.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.49796.log.gz 2023-11-26T07:25:35.568 INFO:teuthology.orchestra.run.smithi143.stderr:/var/log/ceph/ceph-client.0.55259.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45721.log 2023-11-26T07:25:35.569 INFO:teuthology.orchestra.run.smithi143.stderr: 89.2% -- replaced with /var/log/ceph/ceph-client.0.55259.log.gz 2023-11-26T07:25:35.569 INFO:teuthology.orchestra.run.smithi143.stderr:/var/log/ceph/ceph-client.admin.56745.log: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.56745.log.gz 2023-11-26T07:25:35.571 INFO:teuthology.orchestra.run.smithi143.stderr:/var/log/ceph/ceph-client.admin.45721.log: 89.4% -- replaced with /var/log/ceph/ceph-client.admin.45721.log.gz 2023-11-26T07:25:35.571 INFO:teuthology.orchestra.run.smithi143.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.60847.log 2023-11-26T07:25:35.573 INFO:teuthology.orchestra.run.smithi143.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.57231.log 2023-11-26T07:25:35.575 INFO:teuthology.orchestra.run.smithi143.stderr:/var/log/ceph/ceph-client.admin.60847.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45276.log 2023-11-26T07:25:35.575 INFO:teuthology.orchestra.run.smithi143.stderr: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.60847.log.gz 2023-11-26T07:25:35.576 INFO:teuthology.orchestra.run.smithi143.stderr:/var/log/ceph/ceph-client.admin.57231.log: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.57231.log.gz 2023-11-26T07:25:35.577 INFO:teuthology.orchestra.run.smithi143.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51056.log 2023-11-26T07:25:35.579 INFO:teuthology.orchestra.run.smithi143.stderr:/var/log/ceph/ceph-client.admin.45276.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54972.log 2023-11-26T07:25:35.579 INFO:teuthology.orchestra.run.smithi143.stderr: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.45276.log.gz 2023-11-26T07:25:35.580 INFO:teuthology.orchestra.run.smithi143.stderr:/var/log/ceph/ceph-client.admin.51056.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51779.log 2023-11-26T07:25:35.580 INFO:teuthology.orchestra.run.smithi143.stderr: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.51056.log.gz 2023-11-26T07:25:35.581 INFO:teuthology.orchestra.run.smithi143.stderr:/var/log/ceph/ceph-client.admin.54972.log: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.54972.log.gz 2023-11-26T07:25:35.582 INFO:teuthology.orchestra.run.smithi143.stderr:/var/log/ceph/ceph-client.admin.51779.log: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.51779.log.gz 2023-11-26T07:25:35.583 INFO:teuthology.orchestra.run.smithi143.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50430.log 2023-11-26T07:25:35.585 INFO:teuthology.orchestra.run.smithi143.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.55941.log 2023-11-26T07:25:35.586 INFO:teuthology.orchestra.run.smithi143.stderr:/var/log/ceph/ceph-client.admin.50430.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.60873.log 2023-11-26T07:25:35.586 INFO:teuthology.orchestra.run.smithi143.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.50430.log.gz 2023-11-26T07:25:35.587 INFO:teuthology.orchestra.run.smithi143.stderr:/var/log/ceph/ceph-client.admin.55941.log: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.55941.log.gz 2023-11-26T07:25:35.588 INFO:teuthology.orchestra.run.smithi143.stderr:/var/log/ceph/ceph-client.admin.60873.log: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.60873.log.gz 2023-11-26T07:25:35.589 INFO:teuthology.orchestra.run.smithi143.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53457.log 2023-11-26T07:25:35.590 INFO:teuthology.orchestra.run.smithi143.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49920.log 2023-11-26T07:25:35.592 INFO:teuthology.orchestra.run.smithi143.stderr:/var/log/ceph/ceph-client.admin.53457.log: gzip 89.4% -5 -- replaced with /var/log/ceph/ceph-client.admin.53457.log.gz --verbose 2023-11-26T07:25:35.592 INFO:teuthology.orchestra.run.smithi143.stderr: -- /var/log/ceph/ceph-client.admin.59850.log 2023-11-26T07:25:35.594 INFO:teuthology.orchestra.run.smithi143.stderr:/var/log/ceph/ceph-client.admin.49920.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.61162.log 2023-11-26T07:25:35.594 INFO:teuthology.orchestra.run.smithi143.stderr: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.49920.log.gz 2023-11-26T07:25:35.595 INFO:teuthology.orchestra.run.smithi143.stderr:/var/log/ceph/ceph-client.admin.59850.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38069.log 2023-11-26T07:25:35.595 INFO:teuthology.orchestra.run.smithi143.stderr: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.59850.log.gz 2023-11-26T07:25:35.597 INFO:teuthology.orchestra.run.smithi143.stderr:/var/log/ceph/ceph-client.admin.61162.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44662.log 2023-11-26T07:25:35.597 INFO:teuthology.orchestra.run.smithi143.stderr: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.61162.log.gz 2023-11-26T07:25:35.598 INFO:teuthology.orchestra.run.smithi143.stderr:/var/log/ceph/ceph-client.admin.38069.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.58051.log 2023-11-26T07:25:35.598 INFO:teuthology.orchestra.run.smithi143.stderr: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.38069.log.gz 2023-11-26T07:25:35.599 INFO:teuthology.orchestra.run.smithi143.stderr:/var/log/ceph/ceph-client.admin.44662.log: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.44662.log.gz 2023-11-26T07:25:35.601 INFO:teuthology.orchestra.run.smithi143.stderr:/var/log/ceph/ceph-client.admin.58051.log: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.58051.log.gz 2023-11-26T07:25:35.601 INFO:teuthology.orchestra.run.smithi143.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51626.log 2023-11-26T07:25:35.602 INFO:teuthology.orchestra.run.smithi188.stderr: 93.9% -- replaced with /var/log/ceph/ceph-osd.4.log.gz 2023-11-26T07:25:35.602 INFO:teuthology.orchestra.run.smithi143.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47007.log 2023-11-26T07:25:35.604 INFO:teuthology.orchestra.run.smithi143.stderr:/var/log/ceph/ceph-client.admin.51626.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.56713.log 2023-11-26T07:25:35.604 INFO:teuthology.orchestra.run.smithi143.stderr: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.51626.log.gz 2023-11-26T07:25:35.605 INFO:teuthology.orchestra.run.smithi143.stderr:/var/log/ceph/ceph-client.admin.47007.log: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.47007.log.gz 2023-11-26T07:25:35.607 INFO:teuthology.orchestra.run.smithi143.stderr:/var/log/ceph/ceph-client.admin.56713.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.56713.log.gz 2023-11-26T07:25:35.607 INFO:teuthology.orchestra.run.smithi143.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53742.log 2023-11-26T07:25:35.609 INFO:teuthology.orchestra.run.smithi143.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47217.log 2023-11-26T07:25:35.611 INFO:teuthology.orchestra.run.smithi143.stderr:/var/log/ceph/ceph-client.admin.53742.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47587.log 2023-11-26T07:25:35.611 INFO:teuthology.orchestra.run.smithi143.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.53742.log.gz 2023-11-26T07:25:35.611 INFO:teuthology.orchestra.run.smithi143.stderr:/var/log/ceph/ceph-client.admin.47217.log: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.47217.log.gz 2023-11-26T07:25:35.612 INFO:teuthology.orchestra.run.smithi143.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.60538.log 2023-11-26T07:25:35.614 INFO:teuthology.orchestra.run.smithi143.stderr:/var/log/ceph/ceph-client.admin.47587.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.47587.log.gz 2023-11-26T07:25:35.614 INFO:teuthology.orchestra.run.smithi143.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.61003.log 2023-11-26T07:25:35.615 INFO:teuthology.orchestra.run.smithi143.stderr:/var/log/ceph/ceph-client.admin.60538.log: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.60538.log.gz 2023-11-26T07:25:35.615 INFO:teuthology.orchestra.run.smithi143.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50536.log 2023-11-26T07:25:35.616 INFO:teuthology.orchestra.run.smithi188.stderr: 94.0% -- replaced with /var/log/ceph/ceph-osd.7.log.gz 2023-11-26T07:25:35.617 INFO:teuthology.orchestra.run.smithi143.stderr:/var/log/ceph/ceph-client.admin.61003.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54016.log 2023-11-26T07:25:35.617 INFO:teuthology.orchestra.run.smithi143.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.61003.log.gz 2023-11-26T07:25:35.618 INFO:teuthology.orchestra.run.smithi188.stderr: 2023-11-26T07:25:35.618 INFO:teuthology.orchestra.run.smithi188.stderr:real 0m0.589s 2023-11-26T07:25:35.618 INFO:teuthology.orchestra.run.smithi188.stderr:user 0m2.881s 2023-11-26T07:25:35.618 INFO:teuthology.orchestra.run.smithi188.stderr:sys 0m0.345s 2023-11-26T07:25:35.619 INFO:teuthology.orchestra.run.smithi143.stderr:/var/log/ceph/ceph-client.admin.50536.log: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.50536.log.gz 2023-11-26T07:25:35.620 INFO:teuthology.orchestra.run.smithi143.stderr:/var/log/ceph/ceph-client.admin.54016.log: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.54016.log.gz 2023-11-26T07:25:35.620 INFO:teuthology.orchestra.run.smithi143.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48387.log 2023-11-26T07:25:35.622 INFO:teuthology.orchestra.run.smithi143.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.59590.log 2023-11-26T07:25:35.623 INFO:teuthology.orchestra.run.smithi143.stderr:/var/log/ceph/ceph-client.admin.48387.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54437.log 2023-11-26T07:25:35.624 INFO:teuthology.orchestra.run.smithi143.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.48387.log.gz 2023-11-26T07:25:35.625 INFO:teuthology.orchestra.run.smithi143.stderr:/var/log/ceph/ceph-client.admin.59590.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.55732.log 2023-11-26T07:25:35.625 INFO:teuthology.orchestra.run.smithi143.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.59590.log.gz 2023-11-26T07:25:35.626 INFO:teuthology.orchestra.run.smithi143.stderr:/var/log/ceph/ceph-client.admin.54437.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52315.log 2023-11-26T07:25:35.626 INFO:teuthology.orchestra.run.smithi143.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.54437.log.gz 2023-11-26T07:25:35.628 INFO:teuthology.orchestra.run.smithi143.stderr:/var/log/ceph/ceph-client.admin.55732.log: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.55732.log.gz 2023-11-26T07:25:35.628 INFO:teuthology.orchestra.run.smithi143.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.58415.log 2023-11-26T07:25:35.630 INFO:teuthology.orchestra.run.smithi143.stderr:/var/log/ceph/ceph-client.admin.52315.log: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.52315.log.gz 2023-11-26T07:25:35.630 INFO:teuthology.orchestra.run.smithi143.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.57179.log 2023-11-26T07:25:35.632 INFO:teuthology.orchestra.run.smithi143.stderr:/var/log/ceph/ceph-client.admin.58415.log: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.58415.log.gz 2023-11-26T07:25:35.632 INFO:teuthology.orchestra.run.smithi143.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49614.log 2023-11-26T07:25:35.633 INFO:teuthology.orchestra.run.smithi143.stderr:/var/log/ceph/ceph-client.admin.57179.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.55647.log 2023-11-26T07:25:35.634 INFO:teuthology.orchestra.run.smithi143.stderr: 93.9% -- replaced with /var/log/ceph/ceph-osd.0.log.gz 2023-11-26T07:25:35.634 INFO:teuthology.orchestra.run.smithi143.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.57179.log.gz 2023-11-26T07:25:35.634 INFO:teuthology.orchestra.run.smithi143.stderr:/var/log/ceph/ceph-client.admin.49614.log: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.49614.log.gz 2023-11-26T07:25:35.635 INFO:teuthology.orchestra.run.smithi143.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48585.log 2023-11-26T07:25:35.635 INFO:teuthology.orchestra.run.smithi143.stderr:/var/log/ceph/ceph-client.admin.55647.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46237.log 2023-11-26T07:25:35.636 INFO:teuthology.orchestra.run.smithi143.stderr: 89.7% -- replaced with /var/log/ceph/ceph-client.admin.55647.log.gz 2023-11-26T07:25:35.637 INFO:teuthology.orchestra.run.smithi143.stderr:/var/log/ceph/ceph-client.admin.48585.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.48585.log.gz 2023-11-26T07:25:35.637 INFO:teuthology.orchestra.run.smithi143.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44872.log 2023-11-26T07:25:35.639 INFO:teuthology.orchestra.run.smithi143.stderr:/var/log/ceph/ceph-client.admin.46237.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54123.log 2023-11-26T07:25:35.639 INFO:teuthology.orchestra.run.smithi143.stderr: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.46237.log.gz 2023-11-26T07:25:35.639 INFO:teuthology.orchestra.run.smithi143.stderr:/var/log/ceph/ceph-client.admin.44872.log: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.44872.log.gz 2023-11-26T07:25:35.640 INFO:teuthology.orchestra.run.smithi143.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.59939.log 2023-11-26T07:25:35.640 INFO:teuthology.orchestra.run.smithi143.stderr: 93.8% -- replaced with /var/log/ceph/ceph-osd.2.log.gz 2023-11-26T07:25:35.641 INFO:teuthology.orchestra.run.smithi143.stderr:/var/log/ceph/ceph-client.admin.54123.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45484.log 2023-11-26T07:25:35.641 INFO:teuthology.orchestra.run.smithi143.stderr: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.54123.log.gz 2023-11-26T07:25:35.642 INFO:teuthology.orchestra.run.smithi143.stderr:/var/log/ceph/ceph-client.admin.59939.log: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.59939.log.gz 2023-11-26T07:25:35.643 INFO:teuthology.orchestra.run.smithi143.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.57999.log 2023-11-26T07:25:35.644 INFO:teuthology.orchestra.run.smithi143.stderr:/var/log/ceph/ceph-client.admin.45484.log: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.45484.log.gz 2023-11-26T07:25:35.644 INFO:teuthology.orchestra.run.smithi143.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50314.log 2023-11-26T07:25:35.645 INFO:teuthology.orchestra.run.smithi143.stderr:/var/log/ceph/ceph-client.admin.57999.log: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.57999.log.gz 2023-11-26T07:25:35.646 INFO:teuthology.orchestra.run.smithi143.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.58474.log 2023-11-26T07:25:35.647 INFO:teuthology.orchestra.run.smithi143.stderr:/var/log/ceph/ceph-client.admin.50314.log: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.50314.log.gz 2023-11-26T07:25:35.647 INFO:teuthology.orchestra.run.smithi143.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.59300.log 2023-11-26T07:25:35.648 INFO:teuthology.orchestra.run.smithi143.stderr:/var/log/ceph/ceph-client.admin.58474.log: 89.4% -- replaced with /var/log/ceph/ceph-client.admin.58474.log.gz 2023-11-26T07:25:35.648 INFO:teuthology.orchestra.run.smithi143.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48845.log 2023-11-26T07:25:35.649 INFO:teuthology.orchestra.run.smithi143.stderr:/var/log/ceph/ceph-client.admin.59300.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49398.log 2023-11-26T07:25:35.650 INFO:teuthology.orchestra.run.smithi143.stderr: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.59300.log.gz 2023-11-26T07:25:35.651 INFO:teuthology.orchestra.run.smithi143.stderr:/var/log/ceph/ceph-client.admin.48845.log: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.48845.log.gz 2023-11-26T07:25:35.651 INFO:teuthology.orchestra.run.smithi143.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44600.log 2023-11-26T07:25:35.652 INFO:teuthology.orchestra.run.smithi143.stderr:/var/log/ceph/ceph-client.admin.49398.log: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.49398.log.gz 2023-11-26T07:25:35.652 INFO:teuthology.orchestra.run.smithi143.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44204.log 2023-11-26T07:25:35.653 INFO:teuthology.orchestra.run.smithi143.stderr:/var/log/ceph/ceph-client.admin.44600.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44312.log 2023-11-26T07:25:35.654 INFO:teuthology.orchestra.run.smithi143.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.44600.log.gz 2023-11-26T07:25:35.655 INFO:teuthology.orchestra.run.smithi143.stderr:/var/log/ceph/ceph-client.admin.44204.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.44204.log.gz 2023-11-26T07:25:35.655 INFO:teuthology.orchestra.run.smithi143.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.59079.log 2023-11-26T07:25:35.656 INFO:teuthology.orchestra.run.smithi143.stderr:/var/log/ceph/ceph-client.admin.44312.log: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.44312.log.gz 2023-11-26T07:25:35.656 INFO:teuthology.orchestra.run.smithi143.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48741.log 2023-11-26T07:25:35.657 INFO:teuthology.orchestra.run.smithi143.stderr:/var/log/ceph/ceph-client.admin.59079.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.59509.log 2023-11-26T07:25:35.658 INFO:teuthology.orchestra.run.smithi143.stderr: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.59079.log.gz 2023-11-26T07:25:35.659 INFO:teuthology.orchestra.run.smithi143.stderr:/var/log/ceph/ceph-client.admin.48741.log: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.48741.log.gz 2023-11-26T07:25:35.659 INFO:teuthology.orchestra.run.smithi143.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47691.log 2023-11-26T07:25:35.661 INFO:teuthology.orchestra.run.smithi143.stderr:/var/log/ceph/ceph-client.admin.59509.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48531.log 2023-11-26T07:25:35.661 INFO:teuthology.orchestra.run.smithi143.stderr: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.59509.log.gz 2023-11-26T07:25:35.662 INFO:teuthology.orchestra.run.smithi143.stderr:/var/log/ceph/ceph-client.admin.47691.log: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.47691.log.gz 2023-11-26T07:25:35.662 INFO:teuthology.orchestra.run.smithi143.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.58793.log 2023-11-26T07:25:35.663 INFO:teuthology.orchestra.run.smithi143.stderr:/var/log/ceph/ceph-client.admin.48531.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.59824.log 2023-11-26T07:25:35.663 INFO:teuthology.orchestra.run.smithi143.stderr: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.48531.log.gz 2023-11-26T07:25:35.665 INFO:teuthology.orchestra.run.smithi143.stderr:/var/log/ceph/ceph-client.admin.58793.log: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.58793.log.gz 2023-11-26T07:25:35.665 INFO:teuthology.orchestra.run.smithi143.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.57812.log 2023-11-26T07:25:35.667 INFO:teuthology.orchestra.run.smithi143.stderr:/var/log/ceph/ceph-client.admin.59824.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47137.log 2023-11-26T07:25:35.667 INFO:teuthology.orchestra.run.smithi143.stderr: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.59824.log.gz 2023-11-26T07:25:35.667 INFO:teuthology.orchestra.run.smithi143.stderr:/var/log/ceph/ceph-client.admin.57812.log: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.57812.log.gz 2023-11-26T07:25:35.668 INFO:teuthology.orchestra.run.smithi143.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.57205.log 2023-11-26T07:25:35.668 INFO:teuthology.orchestra.run.smithi143.stderr:gzip -5 --verbose --/var/log/ceph/ceph-client.admin.47137.log: /var/log/ceph/ceph.audit.log 2023-11-26T07:25:35.669 INFO:teuthology.orchestra.run.smithi143.stderr:/var/log/ceph/ceph-client.admin.57205.log: 89.7% -- replaced with /var/log/ceph/ceph-client.admin.47137.log.gz 2023-11-26T07:25:35.670 INFO:teuthology.orchestra.run.smithi143.stderr: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.57205.log.gz 2023-11-26T07:25:35.670 INFO:teuthology.orchestra.run.smithi143.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48557.log 2023-11-26T07:25:35.672 INFO:teuthology.orchestra.run.smithi143.stderr:/var/log/ceph/ceph.audit.log: gzip -5 --verbose -- /var/log/ceph/ceph-mds.c.log 2023-11-26T07:25:35.673 INFO:teuthology.orchestra.run.smithi143.stderr:/var/log/ceph/ceph-client.admin.48557.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.56450.log 2023-11-26T07:25:35.673 INFO:teuthology.orchestra.run.smithi143.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.48557.log.gz 2023-11-26T07:25:35.674 INFO:teuthology.orchestra.run.smithi143.stderr:/var/log/ceph/ceph-mds.c.log: 89.3% -- replaced with /var/log/ceph/ceph.audit.log.gz 2023-11-26T07:25:35.674 INFO:teuthology.orchestra.run.smithi143.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46689.log 2023-11-26T07:25:35.675 INFO:teuthology.orchestra.run.smithi143.stderr:/var/log/ceph/ceph-client.admin.56450.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.57916.log 2023-11-26T07:25:35.676 INFO:teuthology.orchestra.run.smithi143.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.56450.log.gz 2023-11-26T07:25:35.677 INFO:teuthology.orchestra.run.smithi143.stderr:/var/log/ceph/ceph-client.admin.46689.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50642.log 2023-11-26T07:25:35.677 INFO:teuthology.orchestra.run.smithi143.stderr: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.46689.log.gz 2023-11-26T07:25:35.679 INFO:teuthology.orchestra.run.smithi143.stderr:/var/log/ceph/ceph-client.admin.57916.log: gzip -5 --verbose -- /var/log/ceph/ceph-mds.a.log 2023-11-26T07:25:35.679 INFO:teuthology.orchestra.run.smithi143.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.57916.log.gz 2023-11-26T07:25:35.680 INFO:teuthology.orchestra.run.smithi143.stderr:/var/log/ceph/ceph-client.admin.50642.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.57538.log 2023-11-26T07:25:35.680 INFO:teuthology.orchestra.run.smithi143.stderr: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.50642.log.gz 2023-11-26T07:25:35.681 INFO:teuthology.orchestra.run.smithi143.stderr:/var/log/ceph/ceph-mds.a.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.58506.log 2023-11-26T07:25:35.685 INFO:teuthology.orchestra.run.smithi143.stderr:/var/log/ceph/ceph-client.admin.57538.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.58207.log 2023-11-26T07:25:35.685 INFO:teuthology.orchestra.run.smithi143.stderr: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.57538.log.gz 2023-11-26T07:25:35.686 INFO:teuthology.orchestra.run.smithi143.stderr:/var/log/ceph/ceph-client.admin.58506.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48611.log 2023-11-26T07:25:35.686 INFO:teuthology.orchestra.run.smithi143.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.58506.log.gz 2023-11-26T07:25:35.686 INFO:teuthology.orchestra.run.smithi143.stderr:/var/log/ceph/ceph-client.admin.58207.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.57023.log 2023-11-26T07:25:35.687 INFO:teuthology.orchestra.run.smithi143.stderr: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.58207.log.gz 2023-11-26T07:25:35.687 INFO:teuthology.orchestra.run.smithi143.stderr:/var/log/ceph/ceph-client.admin.48611.log: gzip 88.3% -- replaced with /var/log/ceph/ceph-client.admin.48611.log.gz -5 2023-11-26T07:25:35.687 INFO:teuthology.orchestra.run.smithi143.stderr: --verbose -- /var/log/ceph/ceph-client.admin.52884.log 2023-11-26T07:25:35.688 INFO:teuthology.orchestra.run.smithi143.stderr:/var/log/ceph/ceph-client.admin.57023.log: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.57023.log.gz 2023-11-26T07:25:35.689 INFO:teuthology.orchestra.run.smithi143.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45082.log 2023-11-26T07:25:35.690 INFO:teuthology.orchestra.run.smithi143.stderr:/var/log/ceph/ceph-client.admin.52884.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54892.log 2023-11-26T07:25:35.691 INFO:teuthology.orchestra.run.smithi143.stderr: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.52884.log.gz 2023-11-26T07:25:35.692 INFO:teuthology.orchestra.run.smithi143.stderr:/var/log/ceph/ceph-client.admin.45082.log: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.45082.log.gz 2023-11-26T07:25:35.692 INFO:teuthology.orchestra.run.smithi143.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.56293.log 2023-11-26T07:25:35.693 INFO:teuthology.orchestra.run.smithi143.stderr:/var/log/ceph/ceph-client.admin.54892.log: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.54892.log.gz 2023-11-26T07:25:35.693 INFO:teuthology.orchestra.run.smithi143.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.58285.log 2023-11-26T07:25:35.695 INFO:teuthology.orchestra.run.smithi143.stderr:/var/log/ceph/ceph-client.admin.56293.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.61084.log 2023-11-26T07:25:35.695 INFO:teuthology.orchestra.run.smithi143.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.56293.log.gz 2023-11-26T07:25:35.696 INFO:teuthology.orchestra.run.smithi143.stderr:/var/log/ceph/ceph-client.admin.58285.log: 88.0%gzip -- replaced with /var/log/ceph/ceph-client.admin.58285.log.gz 2023-11-26T07:25:35.696 INFO:teuthology.orchestra.run.smithi143.stderr: -5 --verbose -- /var/log/ceph/ceph-client.admin.60743.log 2023-11-26T07:25:35.698 INFO:teuthology.orchestra.run.smithi143.stderr:/var/log/ceph/ceph-client.admin.61084.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.0.59135.log 2023-11-26T07:25:35.698 INFO:teuthology.orchestra.run.smithi143.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.61084.log.gz 2023-11-26T07:25:35.699 INFO:teuthology.orchestra.run.smithi143.stderr:/var/log/ceph/ceph-client.admin.60743.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50079.log 2023-11-26T07:25:35.699 INFO:teuthology.orchestra.run.smithi143.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.60743.log.gz 2023-11-26T07:25:35.701 INFO:teuthology.orchestra.run.smithi143.stderr:/var/log/ceph/ceph-client.0.59135.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54489.log 2023-11-26T07:25:35.702 INFO:teuthology.orchestra.run.smithi143.stderr:/var/log/ceph/ceph-client.admin.50079.log: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.50079.log.gz 2023-11-26T07:25:35.703 INFO:teuthology.orchestra.run.smithi143.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46133.log 2023-11-26T07:25:35.704 INFO:teuthology.orchestra.run.smithi143.stderr:/var/log/ceph/ceph-client.admin.54489.log: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.54489.log.gz 2023-11-26T07:25:35.704 INFO:teuthology.orchestra.run.smithi143.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50746.log 2023-11-26T07:25:35.706 INFO:teuthology.orchestra.run.smithi143.stderr:/var/log/ceph/ceph-client.admin.46133.log: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.46133.log.gz 2023-11-26T07:25:35.706 INFO:teuthology.orchestra.run.smithi143.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46420.log 2023-11-26T07:25:35.707 INFO:teuthology.orchestra.run.smithi143.stderr:/var/log/ceph/ceph-client.admin.50746.log: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.50746.log.gz 2023-11-26T07:25:35.709 INFO:teuthology.orchestra.run.smithi143.stderr:/var/log/ceph/ceph-client.admin.46420.log: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.46420.log.gz 2023-11-26T07:25:35.709 INFO:teuthology.orchestra.run.smithi143.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.59561.log 2023-11-26T07:25:35.711 INFO:teuthology.orchestra.run.smithi143.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54333.log 2023-11-26T07:25:35.713 INFO:teuthology.orchestra.run.smithi143.stderr:/var/log/ceph/ceph-client.admin.59561.log: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.59561.log.gz 2023-11-26T07:25:35.714 INFO:teuthology.orchestra.run.smithi143.stderr:/var/log/ceph/ceph-client.admin.54333.log: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.54333.log.gz 2023-11-26T07:25:35.714 INFO:teuthology.orchestra.run.smithi143.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51030.log 2023-11-26T07:25:35.716 INFO:teuthology.orchestra.run.smithi143.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47821.log 2023-11-26T07:25:35.717 INFO:teuthology.orchestra.run.smithi143.stderr:/var/log/ceph/ceph-client.admin.51030.log: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.51030.log.gz 2023-11-26T07:25:35.719 INFO:teuthology.orchestra.run.smithi143.stderr:/var/log/ceph/ceph-client.admin.47821.log: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.47821.log.gz 2023-11-26T07:25:35.719 INFO:teuthology.orchestra.run.smithi143.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47243.log 2023-11-26T07:25:35.721 INFO:teuthology.orchestra.run.smithi143.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51991.log 2023-11-26T07:25:35.722 INFO:teuthology.orchestra.run.smithi143.stderr:/var/log/ceph/ceph-client.admin.47243.log: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.47243.log.gz 2023-11-26T07:25:35.724 INFO:teuthology.orchestra.run.smithi143.stderr:/var/log/ceph/ceph-client.admin.51991.log: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.51991.log.gz 2023-11-26T07:25:35.724 INFO:teuthology.orchestra.run.smithi143.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49507.log 2023-11-26T07:25:35.726 INFO:teuthology.orchestra.run.smithi143.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.0.53383.log 2023-11-26T07:25:35.727 INFO:teuthology.orchestra.run.smithi143.stderr:/var/log/ceph/ceph-client.admin.49507.log: 89.6% -- replaced with /var/log/ceph/ceph-client.admin.49507.log.gz 2023-11-26T07:25:35.732 INFO:teuthology.orchestra.run.smithi143.stderr:/var/log/ceph/ceph-client.0.53383.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45432.log 2023-11-26T07:25:35.735 INFO:teuthology.orchestra.run.smithi143.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.61241.log 2023-11-26T07:25:35.736 INFO:teuthology.orchestra.run.smithi143.stderr:/var/log/ceph/ceph-client.admin.45432.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47269.log 2023-11-26T07:25:35.736 INFO:teuthology.orchestra.run.smithi143.stderr: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.45432.log.gz 2023-11-26T07:25:35.737 INFO:teuthology.orchestra.run.smithi143.stderr:/var/log/ceph/ceph-client.admin.61241.log: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.61241.log.gz 2023-11-26T07:25:35.738 INFO:teuthology.orchestra.run.smithi143.stderr: 92.7% -- replaced with /var/log/ceph/ceph-client.0.53383.log.gz 2023-11-26T07:25:35.740 INFO:teuthology.orchestra.run.smithi143.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47191.log 2023-11-26T07:25:35.742 INFO:teuthology.orchestra.run.smithi143.stderr:/var/log/ceph/ceph-client.admin.47269.log: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.47269.log.gz 2023-11-26T07:25:35.743 INFO:teuthology.orchestra.run.smithi143.stderr:/var/log/ceph/ceph-client.admin.47191.log: 89.7% -- replaced with /var/log/ceph/ceph-client.admin.47191.log.gz 2023-11-26T07:25:35.743 INFO:teuthology.orchestra.run.smithi143.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.55516.log 2023-11-26T07:25:35.745 INFO:teuthology.orchestra.run.smithi143.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.56320.log 2023-11-26T07:25:35.746 INFO:teuthology.orchestra.run.smithi143.stderr:/var/log/ceph/ceph-client.admin.55516.log: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.55516.log.gz 2023-11-26T07:25:35.747 INFO:teuthology.orchestra.run.smithi143.stderr:/var/log/ceph/ceph-client.admin.56320.log: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.56320.log.gz 2023-11-26T07:25:35.748 INFO:teuthology.orchestra.run.smithi143.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54918.log 2023-11-26T07:25:35.750 INFO:teuthology.orchestra.run.smithi143.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53044.log 2023-11-26T07:25:35.751 INFO:teuthology.orchestra.run.smithi143.stderr:/var/log/ceph/ceph-client.admin.54918.log: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.54918.log.gz 2023-11-26T07:25:35.753 INFO:teuthology.orchestra.run.smithi143.stderr:/var/log/ceph/ceph-client.admin.53044.log: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.53044.log.gz 2023-11-26T07:25:35.753 INFO:teuthology.orchestra.run.smithi143.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51402.log 2023-11-26T07:25:35.754 INFO:teuthology.orchestra.run.smithi143.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45068.log 2023-11-26T07:25:35.756 INFO:teuthology.orchestra.run.smithi143.stderr:/var/log/ceph/ceph-client.admin.51402.log: 89.4% -- replaced with /var/log/ceph/ceph-client.admin.51402.log.gz 2023-11-26T07:25:35.757 INFO:teuthology.orchestra.run.smithi143.stderr:/var/log/ceph/ceph-client.admin.45068.log: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.45068.log.gz 2023-11-26T07:25:35.757 INFO:teuthology.orchestra.run.smithi143.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51298.log 2023-11-26T07:25:35.759 INFO:teuthology.orchestra.run.smithi143.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47561.log 2023-11-26T07:25:35.761 INFO:teuthology.orchestra.run.smithi143.stderr: 93.4%/var/log/ceph/ceph-client.admin.51298.log: -- replaced with /var/log/ceph/ceph-client.0.59135.log.gz 2023-11-26T07:25:35.761 INFO:teuthology.orchestra.run.smithi143.stderr: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.51298.log.gz 2023-11-26T07:25:35.762 INFO:teuthology.orchestra.run.smithi143.stderr:/var/log/ceph/ceph-client.admin.47561.log: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.47561.log.gz 2023-11-26T07:25:35.762 INFO:teuthology.orchestra.run.smithi143.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.60363.log 2023-11-26T07:25:35.764 INFO:teuthology.orchestra.run.smithi143.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49848.log 2023-11-26T07:25:35.765 INFO:teuthology.orchestra.run.smithi143.stderr:/var/log/ceph/ceph-client.admin.60363.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.57153.log 2023-11-26T07:25:35.766 INFO:teuthology.orchestra.run.smithi143.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.60363.log.gz 2023-11-26T07:25:35.767 INFO:teuthology.orchestra.run.smithi143.stderr:/var/log/ceph/ceph-client.admin.49848.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.58899.log 2023-11-26T07:25:35.767 INFO:teuthology.orchestra.run.smithi143.stderr: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.49848.log.gz 2023-11-26T07:25:35.768 INFO:teuthology.orchestra.run.smithi143.stderr:/var/log/ceph/ceph-client.admin.57153.log: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.57153.log.gz 2023-11-26T07:25:35.768 INFO:teuthology.orchestra.run.smithi143.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.0.48976.log 2023-11-26T07:25:35.769 INFO:teuthology.orchestra.run.smithi143.stderr:/var/log/ceph/ceph-client.admin.58899.log: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.58899.log.gz 2023-11-26T07:25:35.770 INFO:teuthology.orchestra.run.smithi143.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53252.log 2023-11-26T07:25:35.771 INFO:teuthology.orchestra.run.smithi143.stderr:/var/log/ceph/ceph-client.0.48976.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53200.log 2023-11-26T07:25:35.773 INFO:teuthology.orchestra.run.smithi143.stderr:/var/log/ceph/ceph-client.admin.53252.log: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.53252.log.gz 2023-11-26T07:25:35.774 INFO:teuthology.orchestra.run.smithi143.stderr:/var/log/ceph/ceph-client.admin.53200.log: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.53200.log.gz 2023-11-26T07:25:35.774 INFO:teuthology.orchestra.run.smithi143.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.58847.log 2023-11-26T07:25:35.776 INFO:teuthology.orchestra.run.smithi143.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51265.log 2023-11-26T07:25:35.777 INFO:teuthology.orchestra.run.smithi143.stderr:/var/log/ceph/ceph-client.admin.58847.log: 92.6% -- replaced with /var/log/ceph/ceph-client.0.48976.log.gz 2023-11-26T07:25:35.778 INFO:teuthology.orchestra.run.smithi143.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.58847.log.gz 2023-11-26T07:25:35.778 INFO:teuthology.orchestra.run.smithi143.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.60795.log 2023-11-26T07:25:35.779 INFO:teuthology.orchestra.run.smithi143.stderr:/var/log/ceph/ceph-client.admin.51265.log: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.51265.log.gz 2023-11-26T07:25:35.779 INFO:teuthology.orchestra.run.smithi143.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49588.log 2023-11-26T07:25:35.781 INFO:teuthology.orchestra.run.smithi143.stderr:/var/log/ceph/ceph-client.admin.60795.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48152.log 2023-11-26T07:25:35.781 INFO:teuthology.orchestra.run.smithi143.stderr: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.60795.log.gz 2023-11-26T07:25:35.782 INFO:teuthology.orchestra.run.smithi143.stderr:/var/log/ceph/ceph-client.admin.49588.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.60207.log 2023-11-26T07:25:35.782 INFO:teuthology.orchestra.run.smithi143.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.49588.log.gz 2023-11-26T07:25:35.783 INFO:teuthology.orchestra.run.smithi143.stderr:/var/log/ceph/ceph-client.admin.48152.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.60017.log 2023-11-26T07:25:35.784 INFO:teuthology.orchestra.run.smithi143.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.48152.log.gz 2023-11-26T07:25:35.785 INFO:teuthology.orchestra.run.smithi143.stderr:/var/log/ceph/ceph-client.admin.60207.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.61217.log 2023-11-26T07:25:35.785 INFO:teuthology.orchestra.run.smithi143.stderr: 89.4% -- replaced with /var/log/ceph/ceph-client.admin.60207.log.gz 2023-11-26T07:25:35.786 INFO:teuthology.orchestra.run.smithi143.stderr:/var/log/ceph/ceph-client.admin.60017.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50262.log 2023-11-26T07:25:35.786 INFO:teuthology.orchestra.run.smithi143.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.60017.log.gz 2023-11-26T07:25:35.788 INFO:teuthology.orchestra.run.smithi143.stderr:/var/log/ceph/ceph-client.admin.61217.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52910.log 2023-11-26T07:25:35.788 INFO:teuthology.orchestra.run.smithi143.stderr: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.61217.log.gz 2023-11-26T07:25:35.789 INFO:teuthology.orchestra.run.smithi143.stderr:/var/log/ceph/ceph-client.admin.50262.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46081.log 2023-11-26T07:25:35.789 INFO:teuthology.orchestra.run.smithi143.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.50262.log.gz 2023-11-26T07:25:35.790 INFO:teuthology.orchestra.run.smithi143.stderr:/var/log/ceph/ceph-client.admin.52910.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54229.log 2023-11-26T07:25:35.791 INFO:teuthology.orchestra.run.smithi143.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.52910.log.gz 2023-11-26T07:25:35.792 INFO:teuthology.orchestra.run.smithi143.stderr:/var/log/ceph/ceph-client.admin.46081.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44680.log 2023-11-26T07:25:35.792 INFO:teuthology.orchestra.run.smithi143.stderr: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.46081.log.gz 2023-11-26T07:25:35.793 INFO:teuthology.orchestra.run.smithi143.stderr:/var/log/ceph/ceph-client.admin.54229.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.56424.log 2023-11-26T07:25:35.794 INFO:teuthology.orchestra.run.smithi143.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.54229.log.gz 2023-11-26T07:25:35.795 INFO:teuthology.orchestra.run.smithi143.stderr:/var/log/ceph/ceph-client.admin.44680.log: gzip 88.4% -5 --verbose -- replaced with /var/log/ceph/ceph-client.admin.44680.log.gz -- 2023-11-26T07:25:35.795 INFO:teuthology.orchestra.run.smithi143.stderr: /var/log/ceph/ceph-client.admin.54385.log 2023-11-26T07:25:35.797 INFO:teuthology.orchestra.run.smithi143.stderr:/var/log/ceph/ceph-client.admin.56424.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52992.log 2023-11-26T07:25:35.797 INFO:teuthology.orchestra.run.smithi143.stderr: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.56424.log.gz 2023-11-26T07:25:35.797 INFO:teuthology.orchestra.run.smithi143.stderr:/var/log/ceph/ceph-client.admin.54385.log: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.54385.log.gz 2023-11-26T07:25:35.798 INFO:teuthology.orchestra.run.smithi143.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52594.log 2023-11-26T07:25:35.799 INFO:teuthology.orchestra.run.smithi143.stderr:/var/log/ceph/ceph-client.admin.52992.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.57307.log 2023-11-26T07:25:35.799 INFO:teuthology.orchestra.run.smithi143.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.52992.log.gz 2023-11-26T07:25:35.801 INFO:teuthology.orchestra.run.smithi143.stderr:/var/log/ceph/ceph-client.admin.52594.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44470.log 2023-11-26T07:25:35.801 INFO:teuthology.orchestra.run.smithi143.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.52594.log.gz 2023-11-26T07:25:35.802 INFO:teuthology.orchestra.run.smithi143.stderr:/var/log/ceph/ceph-client.admin.57307.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44230.log 2023-11-26T07:25:35.802 INFO:teuthology.orchestra.run.smithi143.stderr: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.57307.log.gz 2023-11-26T07:25:35.804 INFO:teuthology.orchestra.run.smithi143.stderr:/var/log/ceph/ceph-client.admin.44470.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.59431.log 2023-11-26T07:25:35.804 INFO:teuthology.orchestra.run.smithi143.stderr: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.44470.log.gz 2023-11-26T07:25:35.805 INFO:teuthology.orchestra.run.smithi143.stderr:/var/log/ceph/ceph-client.admin.44230.log: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.44230.log.gz 2023-11-26T07:25:35.805 INFO:teuthology.orchestra.run.smithi143.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.61267.log 2023-11-26T07:25:35.807 INFO:teuthology.orchestra.run.smithi143.stderr:/var/log/ceph/ceph-client.admin.59431.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52672.log 2023-11-26T07:25:35.807 INFO:teuthology.orchestra.run.smithi143.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.59431.log.gz 2023-11-26T07:25:35.809 INFO:teuthology.orchestra.run.smithi143.stderr:/var/log/ceph/ceph-client.admin.61267.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46107.log 2023-11-26T07:25:35.809 INFO:teuthology.orchestra.run.smithi143.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.61267.log.gz 2023-11-26T07:25:35.810 INFO:teuthology.orchestra.run.smithi143.stderr:/var/log/ceph/ceph-client.admin.52672.log: 88.0%gzip -- replaced with /var/log/ceph/ceph-client.admin.52672.log.gz 2023-11-26T07:25:35.810 INFO:teuthology.orchestra.run.smithi143.stderr: -5 --verbose -- /var/log/ceph/ceph-client.admin.61058.log 2023-11-26T07:25:35.811 INFO:teuthology.orchestra.run.smithi143.stderr:/var/log/ceph/ceph-client.admin.46107.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47374.log 2023-11-26T07:25:35.811 INFO:teuthology.orchestra.run.smithi143.stderr: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.46107.log.gz 2023-11-26T07:25:35.813 INFO:teuthology.orchestra.run.smithi143.stderr:/var/log/ceph/ceph-client.admin.61058.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.55810.log 2023-11-26T07:25:35.813 INFO:teuthology.orchestra.run.smithi143.stderr: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.61058.log.gz 2023-11-26T07:25:35.814 INFO:teuthology.orchestra.run.smithi143.stderr:/var/log/ceph/ceph-client.admin.47374.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51869.log 2023-11-26T07:25:35.814 INFO:teuthology.orchestra.run.smithi143.stderr: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.47374.log.gz 2023-11-26T07:25:35.815 INFO:teuthology.orchestra.run.smithi143.stderr:/var/log/ceph/ceph-client.admin.55810.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46446.log 2023-11-26T07:25:35.816 INFO:teuthology.orchestra.run.smithi143.stderr: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.55810.log.gz 2023-11-26T07:25:35.817 INFO:teuthology.orchestra.run.smithi143.stderr:/var/log/ceph/ceph-client.admin.51869.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47322.log 2023-11-26T07:25:35.817 INFO:teuthology.orchestra.run.smithi143.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.51869.log.gz 2023-11-26T07:25:35.818 INFO:teuthology.orchestra.run.smithi143.stderr:/var/log/ceph/ceph-client.admin.46446.log: gzip -5 88.1% --verbose -- replaced with /var/log/ceph/ceph-client.admin.46446.log.gz -- 2023-11-26T07:25:35.818 INFO:teuthology.orchestra.run.smithi143.stderr: /var/log/ceph/ceph-client.admin.50951.log 2023-11-26T07:25:35.820 INFO:teuthology.orchestra.run.smithi143.stderr:/var/log/ceph/ceph-client.admin.47322.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46981.log 2023-11-26T07:25:35.820 INFO:teuthology.orchestra.run.smithi143.stderr: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.47322.log.gz 2023-11-26T07:25:35.821 INFO:teuthology.orchestra.run.smithi143.stderr:/var/log/ceph/ceph-client.admin.50951.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51480.log 2023-11-26T07:25:35.821 INFO:teuthology.orchestra.run.smithi143.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.50951.log.gz 2023-11-26T07:25:35.822 INFO:teuthology.orchestra.run.smithi143.stderr:/var/log/ceph/ceph-client.admin.46981.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49481.log 2023-11-26T07:25:35.823 INFO:teuthology.orchestra.run.smithi143.stderr: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.46981.log.gz 2023-11-26T07:25:35.824 INFO:teuthology.orchestra.run.smithi143.stderr:/var/log/ceph/ceph-client.admin.51480.log: gzip -5 89.0% --verbose -- -- replaced with /var/log/ceph/ceph-client.admin.51480.log.gz 2023-11-26T07:25:35.824 INFO:teuthology.orchestra.run.smithi143.stderr: /var/log/ceph/ceph-client.admin.56837.log 2023-11-26T07:25:35.825 INFO:teuthology.orchestra.run.smithi143.stderr:/var/log/ceph/ceph-client.admin.49481.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.55595.log 2023-11-26T07:25:35.826 INFO:teuthology.orchestra.run.smithi143.stderr: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.49481.log.gz 2023-11-26T07:25:35.827 INFO:teuthology.orchestra.run.smithi143.stderr:/var/log/ceph/ceph-client.admin.56837.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.58259.log 2023-11-26T07:25:35.827 INFO:teuthology.orchestra.run.smithi143.stderr: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.56837.log.gz 2023-11-26T07:25:35.828 INFO:teuthology.orchestra.run.smithi143.stderr:/var/log/ceph/ceph-client.admin.55595.log: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.55595.log.gz 2023-11-26T07:25:35.829 INFO:teuthology.orchestra.run.smithi143.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49894.log 2023-11-26T07:25:35.830 INFO:teuthology.orchestra.run.smithi143.stderr:/var/log/ceph/ceph-client.admin.58259.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53585.log 2023-11-26T07:25:35.830 INFO:teuthology.orchestra.run.smithi143.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.58259.log.gz 2023-11-26T07:25:35.832 INFO:teuthology.orchestra.run.smithi143.stderr:/var/log/ceph/ceph-client.admin.49894.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.56680.log 2023-11-26T07:25:35.832 INFO:teuthology.orchestra.run.smithi143.stderr: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.49894.log.gz 2023-11-26T07:25:35.833 INFO:teuthology.orchestra.run.smithi143.stderr:/var/log/ceph/ceph-client.admin.53585.log: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.53585.log.gz 2023-11-26T07:25:35.833 INFO:teuthology.orchestra.run.smithi143.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47613.log 2023-11-26T07:25:35.835 INFO:teuthology.orchestra.run.smithi143.stderr:/var/log/ceph/ceph-client.admin.56680.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.58819.log 2023-11-26T07:25:35.835 INFO:teuthology.orchestra.run.smithi143.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.56680.log.gz 2023-11-26T07:25:35.836 INFO:teuthology.orchestra.run.smithi143.stderr:/var/log/ceph/ceph-client.admin.47613.log: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.47613.log.gz 2023-11-26T07:25:35.836 INFO:teuthology.orchestra.run.smithi143.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51749.log 2023-11-26T07:25:35.838 INFO:teuthology.orchestra.run.smithi143.stderr:/var/log/ceph/ceph-client.admin.58819.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52568.log 2023-11-26T07:25:35.838 INFO:teuthology.orchestra.run.smithi143.stderr: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.58819.log.gz 2023-11-26T07:25:35.839 INFO:teuthology.orchestra.run.smithi143.stderr:/var/log/ceph/ceph-client.admin.51749.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.51749.log.gz 2023-11-26T07:25:35.840 INFO:teuthology.orchestra.run.smithi143.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46794.log 2023-11-26T07:25:35.841 INFO:teuthology.orchestra.run.smithi143.stderr:/var/log/ceph/ceph-client.admin.52568.log: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.52568.log.gz 2023-11-26T07:25:35.841 INFO:teuthology.orchestra.run.smithi143.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50342.log 2023-11-26T07:25:35.842 INFO:teuthology.orchestra.run.smithi143.stderr:/var/log/ceph/ceph-client.admin.46794.log: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.46794.log.gz 2023-11-26T07:25:35.843 INFO:teuthology.orchestra.run.smithi143.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47295.log 2023-11-26T07:25:35.844 INFO:teuthology.orchestra.run.smithi143.stderr:/var/log/ceph/ceph-client.admin.50342.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47639.log 2023-11-26T07:25:35.844 INFO:teuthology.orchestra.run.smithi143.stderr: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.50342.log.gz 2023-11-26T07:25:35.845 INFO:teuthology.orchestra.run.smithi143.stderr: 92.8% -- replaced with /var/log/ceph/ceph-mds.c.log.gz 2023-11-26T07:25:35.846 INFO:teuthology.orchestra.run.smithi143.stderr:/var/log/ceph/ceph-client.admin.47295.log: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.47295.log.gz 2023-11-26T07:25:35.846 INFO:teuthology.orchestra.run.smithi143.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53666.log 2023-11-26T07:25:35.847 INFO:teuthology.orchestra.run.smithi143.stderr:/var/log/ceph/ceph-client.admin.47639.log: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.47639.log.gz 2023-11-26T07:25:35.848 INFO:teuthology.orchestra.run.smithi143.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51082.log 2023-11-26T07:25:35.849 INFO:teuthology.orchestra.run.smithi143.stderr:/var/log/ceph/ceph-client.admin.53666.log: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.53666.log.gz 2023-11-26T07:25:35.849 INFO:teuthology.orchestra.run.smithi143.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.56346.log 2023-11-26T07:25:35.850 INFO:teuthology.orchestra.run.smithi143.stderr:/var/log/ceph/ceph-client.admin.51082.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54151.log 2023-11-26T07:25:35.851 INFO:teuthology.orchestra.run.smithi143.stderr: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.51082.log.gz 2023-11-26T07:25:35.852 INFO:teuthology.orchestra.run.smithi143.stderr:/var/log/ceph/ceph-client.admin.56346.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52856.log 2023-11-26T07:25:35.852 INFO:teuthology.orchestra.run.smithi143.stderr: 89.4% -- replaced with /var/log/ceph/ceph-client.admin.56346.log.gz 2023-11-26T07:25:35.854 INFO:teuthology.orchestra.run.smithi143.stderr:/var/log/ceph/ceph-client.admin.54151.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44885.log 2023-11-26T07:25:35.854 INFO:teuthology.orchestra.run.smithi143.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.54151.log.gz 2023-11-26T07:25:35.855 INFO:teuthology.orchestra.run.smithi143.stderr:/var/log/ceph/ceph-client.admin.52856.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44444.log 2023-11-26T07:25:35.855 INFO:teuthology.orchestra.run.smithi143.stderr: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.52856.log.gz 2023-11-26T07:25:35.856 INFO:teuthology.orchestra.run.smithi143.stderr:/var/log/ceph/ceph-client.admin.44885.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.58705.log 2023-11-26T07:25:35.856 INFO:teuthology.orchestra.run.smithi143.stderr: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.44885.log.gz 2023-11-26T07:25:35.858 INFO:teuthology.orchestra.run.smithi143.stderr:/var/log/ceph/ceph-client.admin.44444.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.59991.log 2023-11-26T07:25:35.858 INFO:teuthology.orchestra.run.smithi143.stderr: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.44444.log.gz 2023-11-26T07:25:35.859 INFO:teuthology.orchestra.run.smithi143.stderr:/var/log/ceph/ceph-client.admin.58705.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.55543.log 2023-11-26T07:25:35.859 INFO:teuthology.orchestra.run.smithi143.stderr: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.58705.log.gz 2023-11-26T07:25:35.861 INFO:teuthology.orchestra.run.smithi143.stderr:/var/log/ceph/ceph-client.admin.59991.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51961.log 2023-11-26T07:25:35.861 INFO:teuthology.orchestra.run.smithi143.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.59991.log.gz 2023-11-26T07:25:35.861 INFO:teuthology.orchestra.run.smithi143.stderr: 94.0% -- replaced with /var/log/ceph/ceph-osd.3.log.gz 2023-11-26T07:25:35.861 INFO:teuthology.orchestra.run.smithi143.stderr:/var/log/ceph/ceph-client.admin.55543.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50027.log 2023-11-26T07:25:35.862 INFO:teuthology.orchestra.run.smithi143.stderr: 89.5% -- replaced with /var/log/ceph/ceph-client.admin.55543.log.gz 2023-11-26T07:25:35.863 INFO:teuthology.orchestra.run.smithi143.stderr:/var/log/ceph/ceph-client.admin.51961.log: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.51961.log.gz 2023-11-26T07:25:35.863 INFO:teuthology.orchestra.run.smithi143.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.57945.log 2023-11-26T07:25:35.864 INFO:teuthology.orchestra.run.smithi143.stderr:/var/log/ceph/ceph-client.admin.50027.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.60662.log 2023-11-26T07:25:35.865 INFO:teuthology.orchestra.run.smithi143.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.50027.log.gz 2023-11-26T07:25:35.865 INFO:teuthology.orchestra.run.smithi143.stderr:/var/log/ceph/ceph-client.admin.57945.log: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.57945.log.gz 2023-11-26T07:25:35.866 INFO:teuthology.orchestra.run.smithi143.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53611.log 2023-11-26T07:25:35.867 INFO:teuthology.orchestra.run.smithi143.stderr:/var/log/ceph/ceph-client.admin.60662.log: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.60662.log.gz 2023-11-26T07:25:35.867 INFO:teuthology.orchestra.run.smithi143.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49562.log 2023-11-26T07:25:35.869 INFO:teuthology.orchestra.run.smithi143.stderr:/var/log/ceph/ceph-client.admin.53611.log: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.53611.log.gz 2023-11-26T07:25:35.869 INFO:teuthology.orchestra.run.smithi143.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47743.log 2023-11-26T07:25:35.870 INFO:teuthology.orchestra.run.smithi143.stderr:/var/log/ceph/ceph-client.admin.49562.log: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.49562.log.gz 2023-11-26T07:25:35.871 INFO:teuthology.orchestra.run.smithi143.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.58363.log 2023-11-26T07:25:35.872 INFO:teuthology.orchestra.run.smithi143.stderr:/var/log/ceph/ceph-client.admin.47743.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.47743.log.gz 2023-11-26T07:25:35.872 INFO:teuthology.orchestra.run.smithi143.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.59646.log 2023-11-26T07:25:35.873 INFO:teuthology.orchestra.run.smithi143.stderr:/var/log/ceph/ceph-client.admin.58363.log: gzip 88.3% -5 -- replaced with /var/log/ceph/ceph-client.admin.58363.log.gz --verbose 2023-11-26T07:25:35.873 INFO:teuthology.orchestra.run.smithi143.stderr: -- /var/log/ceph/ceph-client.admin.48793.log 2023-11-26T07:25:35.875 INFO:teuthology.orchestra.run.smithi143.stderr:/var/log/ceph/ceph-client.admin.59646.log: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.59646.log.gz 2023-11-26T07:25:35.875 INFO:teuthology.orchestra.run.smithi143.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.59535.log 2023-11-26T07:25:35.876 INFO:teuthology.orchestra.run.smithi143.stderr:/var/log/ceph/ceph-client.admin.48793.log: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.48793.log.gz 2023-11-26T07:25:35.876 INFO:teuthology.orchestra.run.smithi143.stderr: 90.8% -- replaced with /var/log/ceph/ceph-mon.a.log.gz 2023-11-26T07:25:35.877 INFO:teuthology.orchestra.run.smithi143.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.58733.log 2023-11-26T07:25:35.878 INFO:teuthology.orchestra.run.smithi143.stderr:/var/log/ceph/ceph-client.admin.59535.log: 87.1% -- replaced with /var/log/ceph/ceph-client.admin.59535.log.gz 2023-11-26T07:25:35.878 INFO:teuthology.orchestra.run.smithi143.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46872.log 2023-11-26T07:25:35.879 INFO:teuthology.orchestra.run.smithi143.stderr:/var/log/ceph/ceph-client.admin.58733.log: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.58733.log.gz 2023-11-26T07:25:35.879 INFO:teuthology.orchestra.run.smithi143.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45302.log 2023-11-26T07:25:35.880 INFO:teuthology.orchestra.run.smithi143.stderr:/var/log/ceph/ceph-client.admin.46872.log: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.46872.log.gz 2023-11-26T07:25:35.881 INFO:teuthology.orchestra.run.smithi143.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.60233.log 2023-11-26T07:25:35.882 INFO:teuthology.orchestra.run.smithi143.stderr:/var/log/ceph/ceph-client.admin.45302.log: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.45302.log.gz 2023-11-26T07:25:35.882 INFO:teuthology.orchestra.run.smithi143.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.56943.log 2023-11-26T07:25:35.883 INFO:teuthology.orchestra.run.smithi143.stderr:/var/log/ceph/ceph-client.admin.60233.log: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.60233.log.gz 2023-11-26T07:25:35.883 INFO:teuthology.orchestra.run.smithi143.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49372.log 2023-11-26T07:25:35.884 INFO:teuthology.orchestra.run.smithi143.stderr:/var/log/ceph/ceph-client.admin.56943.log: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.56943.log.gz 2023-11-26T07:25:35.884 INFO:teuthology.orchestra.run.smithi143.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.60415.log 2023-11-26T07:25:35.885 INFO:teuthology.orchestra.run.smithi143.stderr:/var/log/ceph/ceph-client.admin.49372.log: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.49372.log.gz 2023-11-26T07:25:35.885 INFO:teuthology.orchestra.run.smithi143.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52114.log 2023-11-26T07:25:35.886 INFO:teuthology.orchestra.run.smithi143.stderr:/var/log/ceph/ceph-client.admin.60415.log: gzip -5 89.4% --verbose -- replaced with /var/log/ceph/ceph-client.admin.60415.log.gz 2023-11-26T07:25:35.886 INFO:teuthology.orchestra.run.smithi143.stderr: -- /var/log/ceph/ceph-client.admin.55333.log 2023-11-26T07:25:35.887 INFO:teuthology.orchestra.run.smithi143.stderr:/var/log/ceph/ceph-client.admin.52114.log: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.52114.log.gz 2023-11-26T07:25:35.887 INFO:teuthology.orchestra.run.smithi143.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49666.log 2023-11-26T07:25:35.888 INFO:teuthology.orchestra.run.smithi143.stderr:/var/log/ceph/ceph-client.admin.55333.log: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.55333.log.gz 2023-11-26T07:25:35.888 INFO:teuthology.orchestra.run.smithi143.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46394.log 2023-11-26T07:25:35.889 INFO:teuthology.orchestra.run.smithi143.stderr:/var/log/ceph/ceph-client.admin.49666.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.49666.log.gz 2023-11-26T07:25:35.889 INFO:teuthology.orchestra.run.smithi143.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46663.log 2023-11-26T07:25:35.890 INFO:teuthology.orchestra.run.smithi143.stderr:/var/log/ceph/ceph-client.admin.46394.log: 89.4% -- replaced with /var/log/ceph/ceph-client.admin.46394.log.gz 2023-11-26T07:25:35.890 INFO:teuthology.orchestra.run.smithi143.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48689.log 2023-11-26T07:25:35.891 INFO:teuthology.orchestra.run.smithi143.stderr:/var/log/ceph/ceph-client.admin.46663.log: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.46663.log.gz 2023-11-26T07:25:35.891 INFO:teuthology.orchestra.run.smithi143.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50112.log 2023-11-26T07:25:35.892 INFO:teuthology.orchestra.run.smithi143.stderr:/var/log/ceph/ceph-client.admin.48689.log: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.48689.log.gz 2023-11-26T07:25:35.893 INFO:teuthology.orchestra.run.smithi143.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52185.log 2023-11-26T07:25:35.894 INFO:teuthology.orchestra.run.smithi143.stderr:/var/log/ceph/ceph-client.admin.50112.log: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.50112.log.gz 2023-11-26T07:25:35.894 INFO:teuthology.orchestra.run.smithi143.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44256.log 2023-11-26T07:25:35.895 INFO:teuthology.orchestra.run.smithi143.stderr:/var/log/ceph/ceph-client.admin.52185.log: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.52185.log.gz 2023-11-26T07:25:35.895 INFO:teuthology.orchestra.run.smithi143.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.58103.log 2023-11-26T07:25:35.896 INFO:teuthology.orchestra.run.smithi143.stderr:/var/log/ceph/ceph-client.admin.44256.log: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.44256.log.gz 2023-11-26T07:25:35.896 INFO:teuthology.orchestra.run.smithi143.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51532.log 2023-11-26T07:25:35.897 INFO:teuthology.orchestra.run.smithi143.stderr:/var/log/ceph/ceph-client.admin.58103.log: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.58103.log.gz 2023-11-26T07:25:35.897 INFO:teuthology.orchestra.run.smithi143.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.55971.log 2023-11-26T07:25:35.898 INFO:teuthology.orchestra.run.smithi143.stderr:/var/log/ceph/ceph-client.admin.51532.log: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.51532.log.gz 2023-11-26T07:25:35.898 INFO:teuthology.orchestra.run.smithi143.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44667.log 2023-11-26T07:25:35.899 INFO:teuthology.orchestra.run.smithi143.stderr:/var/log/ceph/ceph-client.admin.55971.log: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.55971.log.gz 2023-11-26T07:25:35.899 INFO:teuthology.orchestra.run.smithi143.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51003.log 2023-11-26T07:25:35.900 INFO:teuthology.orchestra.run.smithi143.stderr:/var/log/ceph/ceph-client.admin.44667.log: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.44667.log.gz 2023-11-26T07:25:35.900 INFO:teuthology.orchestra.run.smithi143.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.55914.log 2023-11-26T07:25:35.901 INFO:teuthology.orchestra.run.smithi143.stderr:/var/log/ceph/ceph-client.admin.51003.log: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.51003.log.gz 2023-11-26T07:25:35.901 INFO:teuthology.orchestra.run.smithi143.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.60178.log 2023-11-26T07:25:35.902 INFO:teuthology.orchestra.run.smithi143.stderr:/var/log/ceph/ceph-client.admin.55914.log: 89.6% -- replaced with /var/log/ceph/ceph-client.admin.55914.log.gz 2023-11-26T07:25:35.903 INFO:teuthology.orchestra.run.smithi143.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47348.log 2023-11-26T07:25:35.903 INFO:teuthology.orchestra.run.smithi143.stderr:/var/log/ceph/ceph-client.admin.60178.log: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.60178.log.gz 2023-11-26T07:25:35.904 INFO:teuthology.orchestra.run.smithi143.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52516.log 2023-11-26T07:25:35.905 INFO:teuthology.orchestra.run.smithi143.stderr:/var/log/ceph/ceph-client.admin.47348.log: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.47348.log.gz 2023-11-26T07:25:35.905 INFO:teuthology.orchestra.run.smithi143.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.59003.log 2023-11-26T07:25:35.907 INFO:teuthology.orchestra.run.smithi143.stderr:/var/log/ceph/ceph-client.admin.52516.log: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.52516.log.gz 2023-11-26T07:25:35.907 INFO:teuthology.orchestra.run.smithi143.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46472.log 2023-11-26T07:25:35.907 INFO:teuthology.orchestra.run.smithi143.stderr:/var/log/ceph/ceph-client.admin.59003.log: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.59003.log.gz 2023-11-26T07:25:35.908 INFO:teuthology.orchestra.run.smithi143.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48871.log 2023-11-26T07:25:35.908 INFO:teuthology.orchestra.run.smithi143.stderr:/var/log/ceph/ceph-client.admin.46472.log: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.46472.log.gz 2023-11-26T07:25:35.908 INFO:teuthology.orchestra.run.smithi143.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50484.log 2023-11-26T07:25:35.909 INFO:teuthology.orchestra.run.smithi143.stderr:/var/log/ceph/ceph-client.admin.48871.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53070.log 2023-11-26T07:25:35.909 INFO:teuthology.orchestra.run.smithi143.stderr: 91.4% -- replaced with /var/log/ceph/ceph-client.admin.48871.log.gz 2023-11-26T07:25:35.910 INFO:teuthology.orchestra.run.smithi143.stderr:/var/log/ceph/ceph-client.admin.50484.log: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.50484.log.gz 2023-11-26T07:25:35.910 INFO:teuthology.orchestra.run.smithi143.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45406.log 2023-11-26T07:25:35.911 INFO:teuthology.orchestra.run.smithi143.stderr:/var/log/ceph/ceph-client.admin.53070.log: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.53070.log.gz 2023-11-26T07:25:35.911 INFO:teuthology.orchestra.run.smithi143.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52341.log 2023-11-26T07:25:35.912 INFO:teuthology.orchestra.run.smithi143.stderr: 93.8% -- replaced with /var/log/ceph/ceph-osd.1.log.gz 2023-11-26T07:25:35.912 INFO:teuthology.orchestra.run.smithi143.stderr:/var/log/ceph/ceph-client.admin.45406.log: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.45406.log.gz 2023-11-26T07:25:35.913 INFO:teuthology.orchestra.run.smithi143.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51187.log 2023-11-26T07:25:35.913 INFO:teuthology.orchestra.run.smithi143.stderr:/var/log/ceph/ceph-client.admin.52341.log: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.52341.log.gz 2023-11-26T07:25:35.913 INFO:teuthology.orchestra.run.smithi143.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54724.log 2023-11-26T07:25:35.914 INFO:teuthology.orchestra.run.smithi143.stderr:/var/log/ceph/ceph-client.admin.51187.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.61188.log 2023-11-26T07:25:35.914 INFO:teuthology.orchestra.run.smithi143.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.51187.log.gz 2023-11-26T07:25:35.915 INFO:teuthology.orchestra.run.smithi143.stderr:/var/log/ceph/ceph-client.admin.54724.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54255.log 2023-11-26T07:25:35.915 INFO:teuthology.orchestra.run.smithi143.stderr: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.54724.log.gz 2023-11-26T07:25:35.916 INFO:teuthology.orchestra.run.smithi143.stderr:/var/log/ceph/ceph-client.admin.61188.log: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.61188.log.gz 2023-11-26T07:25:35.917 INFO:teuthology.orchestra.run.smithi143.stderr:/var/log/ceph/ceph-client.admin.54255.log: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.54255.log.gz 2023-11-26T07:25:36.051 INFO:teuthology.orchestra.run.smithi143.stderr: 94.2% -- replaced with /var/log/ceph/ceph-mds.a.log.gz 2023-11-26T07:25:36.053 INFO:teuthology.orchestra.run.smithi143.stderr: 2023-11-26T07:25:36.053 INFO:teuthology.orchestra.run.smithi143.stderr:real 0m1.042s 2023-11-26T07:25:36.053 INFO:teuthology.orchestra.run.smithi143.stderr:user 0m3.795s 2023-11-26T07:25:36.053 INFO:teuthology.orchestra.run.smithi143.stderr:sys 0m1.426s 2023-11-26T07:25:36.054 INFO:tasks.ceph:Archiving logs... 2023-11-26T07:25:36.054 DEBUG:teuthology.misc:Transferring archived files from smithi143:/var/log/ceph to /home/teuthworker/archive/rishabh-2023-11-26_06:51:12-fs:volumes-rishabh-mgr-vol-misc-testing-default-smithi/7467343/remote/smithi143/log 2023-11-26T07:25:36.056 DEBUG:teuthology.orchestra.run.smithi143:> sudo tar c -f - -C /var/log/ceph -- . 2023-11-26T07:25:36.855 DEBUG:teuthology.misc:Transferring archived files from smithi188:/var/log/ceph to /home/teuthworker/archive/rishabh-2023-11-26_06:51:12-fs:volumes-rishabh-mgr-vol-misc-testing-default-smithi/7467343/remote/smithi188/log 2023-11-26T07:25:36.856 DEBUG:teuthology.orchestra.run.smithi188:> sudo tar c -f - -C /var/log/ceph -- . 2023-11-26T07:25:37.154 DEBUG:teuthology.run_tasks:Unwinding manager install 2023-11-26T07:25:37.163 INFO:teuthology.task.install.util:Removing shipped files: /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer... 2023-11-26T07:25:37.163 DEBUG:teuthology.orchestra.run.smithi143:> sudo rm -f -- /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer 2023-11-26T07:25:37.167 DEBUG:teuthology.orchestra.run.smithi188:> sudo rm -f -- /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer 2023-11-26T07:25:37.251 DEBUG:teuthology.run_tasks:Unwinding manager clock 2023-11-26T07:25:37.260 INFO:teuthology.task.clock:Checking final clock skew... 2023-11-26T07:25:37.260 DEBUG:teuthology.orchestra.run.smithi143:> PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2023-11-26T07:25:37.264 DEBUG:teuthology.orchestra.run.smithi188:> PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2023-11-26T07:25:37.275 INFO:teuthology.orchestra.run.smithi143.stdout: remote refid st t when poll reach delay offset jitter 2023-11-26T07:25:37.275 INFO:teuthology.orchestra.run.smithi143.stdout:============================================================================== 2023-11-26T07:25:37.275 INFO:teuthology.orchestra.run.smithi143.stdout:*hv01.front.sepi 44.4.53.2 2 u 4 64 377 0.088 -2.436 2.045 2023-11-26T07:25:37.276 INFO:teuthology.orchestra.run.smithi143.stdout:+hv02.front.sepi 63.231.80.2 3 u 8 64 377 0.094 -1.826 1.581 2023-11-26T07:25:37.276 INFO:teuthology.orchestra.run.smithi143.stdout:+hv03.front.sepi 74.6.168.72 3 u 15 64 377 0.073 -1.166 1.811 2023-11-26T07:25:37.276 INFO:teuthology.orchestra.run.smithi143.stdout: hv04.front.sepi .INIT. 16 u - 256 0 0.000 +0.000 0.000 2023-11-26T07:25:37.279 INFO:teuthology.orchestra.run.smithi188.stdout: remote refid st t when poll reach delay offset jitter 2023-11-26T07:25:37.279 INFO:teuthology.orchestra.run.smithi188.stdout:============================================================================== 2023-11-26T07:25:37.280 INFO:teuthology.orchestra.run.smithi188.stdout:*hv01.front.sepi 44.4.53.2 2 u 12 64 377 0.108 -2.564 1.253 2023-11-26T07:25:37.280 INFO:teuthology.orchestra.run.smithi188.stdout:+hv02.front.sepi 63.231.80.2 3 u 6 64 377 0.086 +1.153 0.979 2023-11-26T07:25:37.280 INFO:teuthology.orchestra.run.smithi188.stdout:+hv03.front.sepi 74.6.168.72 3 u 5 64 377 0.079 -0.672 1.052 2023-11-26T07:25:37.280 INFO:teuthology.orchestra.run.smithi188.stdout: hv04.front.sepi .INIT. 16 u - 256 0 0.000 +0.000 0.000 2023-11-26T07:25:37.281 DEBUG:teuthology.run_tasks:Unwinding manager ansible.cephlab 2023-11-26T07:25:37.289 INFO:teuthology.task.ansible:Skipping ansible cleanup... 2023-11-26T07:25:37.290 DEBUG:teuthology.run_tasks:Unwinding manager selinux 2023-11-26T07:25:37.298 DEBUG:teuthology.run_tasks:Unwinding manager pcp 2023-11-26T07:25:37.307 DEBUG:teuthology.run_tasks:Unwinding manager internal.timer 2023-11-26T07:25:37.318 INFO:teuthology.task.internal:Duration was 1258.169618 seconds 2023-11-26T07:25:37.318 DEBUG:teuthology.run_tasks:Unwinding manager internal.syslog 2023-11-26T07:25:37.326 INFO:teuthology.task.internal.syslog:Shutting down syslog monitoring... 2023-11-26T07:25:37.326 DEBUG:teuthology.orchestra.run.smithi143:> sudo rm -f -- /etc/rsyslog.d/80-cephtest.conf && sudo service rsyslog restart 2023-11-26T07:25:37.331 DEBUG:teuthology.orchestra.run.smithi188:> sudo rm -f -- /etc/rsyslog.d/80-cephtest.conf && sudo service rsyslog restart 2023-11-26T07:25:37.446 INFO:teuthology.task.internal.syslog:Checking logs for errors... 2023-11-26T07:25:37.446 DEBUG:teuthology.task.internal.syslog:Checking ubuntu@smithi143.front.sepia.ceph.com 2023-11-26T07:25:37.446 DEBUG:teuthology.orchestra.run.smithi143:> egrep --binary-files=text '\bBUG\b|\bINFO\b|\bDEADLOCK\b' /home/ubuntu/cephtest/archive/syslog/kern.log | grep -v 'task .* blocked for more than .* seconds' | grep -v 'lockdep is turned off' | grep -v 'trying to register non-static key' | grep -v 'DEBUG: fsize' | grep -v CRON | grep -v 'BUG: bad unlock balance detected' | grep -v 'inconsistent lock state' | grep -v '*** DEADLOCK ***' | grep -v 'INFO: possible irq lock inversion dependency detected' | grep -v 'INFO: NMI handler (perf_event_nmi_handler) took too long to run' | grep -v 'INFO: recovery required on readonly' | grep -v 'ceph-create-keys: INFO' | grep -v INFO:ceph-create-keys | grep -v 'Loaded datasource DataSourceOpenStack' | grep -v 'container-storage-setup: INFO: Volume group backing root filesystem could not be determined' | egrep -v '\bsalt-master\b|\bsalt-minion\b|\bsalt-api\b' | grep -v ceph-crash | egrep -v '\btcmu-runner\b.*\bINFO\b' | head -n 1 2023-11-26T07:25:37.468 DEBUG:teuthology.task.internal.syslog:Checking ubuntu@smithi188.front.sepia.ceph.com 2023-11-26T07:25:37.469 DEBUG:teuthology.orchestra.run.smithi188:> egrep --binary-files=text '\bBUG\b|\bINFO\b|\bDEADLOCK\b' /home/ubuntu/cephtest/archive/syslog/kern.log | grep -v 'task .* blocked for more than .* seconds' | grep -v 'lockdep is turned off' | grep -v 'trying to register non-static key' | grep -v 'DEBUG: fsize' | grep -v CRON | grep -v 'BUG: bad unlock balance detected' | grep -v 'inconsistent lock state' | grep -v '*** DEADLOCK ***' | grep -v 'INFO: possible irq lock inversion dependency detected' | grep -v 'INFO: NMI handler (perf_event_nmi_handler) took too long to run' | grep -v 'INFO: recovery required on readonly' | grep -v 'ceph-create-keys: INFO' | grep -v INFO:ceph-create-keys | grep -v 'Loaded datasource DataSourceOpenStack' | grep -v 'container-storage-setup: INFO: Volume group backing root filesystem could not be determined' | egrep -v '\bsalt-master\b|\bsalt-minion\b|\bsalt-api\b' | grep -v ceph-crash | egrep -v '\btcmu-runner\b.*\bINFO\b' | head -n 1 2023-11-26T07:25:37.490 INFO:teuthology.task.internal.syslog:Compressing syslogs... 2023-11-26T07:25:37.490 DEBUG:teuthology.orchestra.run.smithi143:> find /home/ubuntu/cephtest/archive/syslog -name '*.log' -print0 | sudo xargs -0 --no-run-if-empty -- gzip -- 2023-11-26T07:25:37.519 DEBUG:teuthology.orchestra.run.smithi188:> find /home/ubuntu/cephtest/archive/syslog -name '*.log' -print0 | sudo xargs -0 --no-run-if-empty -- gzip -- 2023-11-26T07:25:37.568 INFO:teuthology.task.internal.syslog:Gathering journactl -b0... 2023-11-26T07:25:37.568 DEBUG:teuthology.orchestra.run.smithi143:> sudo journalctl -b0 | gzip -9 > /home/ubuntu/cephtest/archive/syslog/journalctl-b0.gz 2023-11-26T07:25:37.599 DEBUG:teuthology.orchestra.run.smithi188:> sudo journalctl -b0 | gzip -9 > /home/ubuntu/cephtest/archive/syslog/journalctl-b0.gz 2023-11-26T07:25:37.679 DEBUG:teuthology.run_tasks:Unwinding manager internal.sudo 2023-11-26T07:25:37.689 INFO:teuthology.task.internal:Restoring /etc/sudoers... 2023-11-26T07:25:37.689 DEBUG:teuthology.orchestra.run.smithi143:> sudo mv -f /etc/sudoers.orig.teuthology /etc/sudoers 2023-11-26T07:25:37.710 DEBUG:teuthology.orchestra.run.smithi188:> sudo mv -f /etc/sudoers.orig.teuthology /etc/sudoers 2023-11-26T07:25:37.731 DEBUG:teuthology.run_tasks:Unwinding manager internal.coredump 2023-11-26T07:25:37.740 DEBUG:teuthology.orchestra.run.smithi143:> sudo sysctl -w kernel.core_pattern=core && sudo bash -c 'for f in `find /home/ubuntu/cephtest/archive/coredump -type f`; do file $f | grep -q systemd-sysusers && rm $f || true ; done' && rmdir --ignore-fail-on-non-empty -- /home/ubuntu/cephtest/archive/coredump 2023-11-26T07:25:37.759 DEBUG:teuthology.orchestra.run.smithi188:> sudo sysctl -w kernel.core_pattern=core && sudo bash -c 'for f in `find /home/ubuntu/cephtest/archive/coredump -type f`; do file $f | grep -q systemd-sysusers && rm $f || true ; done' && rmdir --ignore-fail-on-non-empty -- /home/ubuntu/cephtest/archive/coredump 2023-11-26T07:25:37.773 INFO:teuthology.orchestra.run.smithi143.stdout:kernel.core_pattern = core 2023-11-26T07:25:37.792 INFO:teuthology.orchestra.run.smithi188.stdout:kernel.core_pattern = core 2023-11-26T07:25:37.810 DEBUG:teuthology.orchestra.run.smithi143:> test -e /home/ubuntu/cephtest/archive/coredump 2023-11-26T07:25:37.842 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-11-26T07:25:37.843 DEBUG:teuthology.orchestra.run.smithi188:> test -e /home/ubuntu/cephtest/archive/coredump 2023-11-26T07:25:37.857 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-11-26T07:25:37.858 DEBUG:teuthology.run_tasks:Unwinding manager internal.archive 2023-11-26T07:25:37.868 INFO:teuthology.task.internal:Transferring archived files... 2023-11-26T07:25:37.868 DEBUG:teuthology.misc:Transferring archived files from smithi143:/home/ubuntu/cephtest/archive to /home/teuthworker/archive/rishabh-2023-11-26_06:51:12-fs:volumes-rishabh-mgr-vol-misc-testing-default-smithi/7467343/remote/smithi143 2023-11-26T07:25:37.869 DEBUG:teuthology.orchestra.run.smithi143:> sudo tar c -f - -C /home/ubuntu/cephtest/archive -- . 2023-11-26T07:25:37.909 DEBUG:teuthology.misc:Transferring archived files from smithi188:/home/ubuntu/cephtest/archive to /home/teuthworker/archive/rishabh-2023-11-26_06:51:12-fs:volumes-rishabh-mgr-vol-misc-testing-default-smithi/7467343/remote/smithi188 2023-11-26T07:25:37.909 DEBUG:teuthology.orchestra.run.smithi188:> sudo tar c -f - -C /home/ubuntu/cephtest/archive -- . 2023-11-26T07:25:37.934 INFO:teuthology.task.internal:Removing archive directory... 2023-11-26T07:25:37.935 DEBUG:teuthology.orchestra.run.smithi143:> rm -rf -- /home/ubuntu/cephtest/archive 2023-11-26T07:25:37.955 DEBUG:teuthology.orchestra.run.smithi188:> rm -rf -- /home/ubuntu/cephtest/archive 2023-11-26T07:25:37.979 DEBUG:teuthology.run_tasks:Unwinding manager internal.archive_upload 2023-11-26T07:25:37.990 INFO:teuthology.task.internal:Not uploading archives. 2023-11-26T07:25:37.990 DEBUG:teuthology.run_tasks:Unwinding manager internal.base 2023-11-26T07:25:38.039 INFO:teuthology.task.internal:Tidying up after the test... 2023-11-26T07:25:38.039 DEBUG:teuthology.orchestra.run.smithi143:> find /home/ubuntu/cephtest -ls ; rmdir -- /home/ubuntu/cephtest 2023-11-26T07:25:38.043 DEBUG:teuthology.orchestra.run.smithi188:> find /home/ubuntu/cephtest -ls ; rmdir -- /home/ubuntu/cephtest 2023-11-26T07:25:38.048 INFO:teuthology.orchestra.run.smithi143.stdout: 658040 4 drwxr-xr-x 2 ubuntu ubuntu 4096 Nov 26 07:25 /home/ubuntu/cephtest 2023-11-26T07:25:38.051 INFO:teuthology.orchestra.run.smithi188.stdout: 655441 4 drwxr-xr-x 2 ubuntu ubuntu 4096 Nov 26 07:25 /home/ubuntu/cephtest 2023-11-26T07:25:38.053 DEBUG:teuthology.run_tasks:Unwinding manager kernel 2023-11-26T07:25:38.064 DEBUG:teuthology.run_tasks:Unwinding manager console_log 2023-11-26T07:25:38.085 INFO:teuthology.nuke:Checking targets against current locks 2023-11-26T07:25:38.113 DEBUG:teuthology.nuke:shortname: smithi143 2023-11-26T07:25:38.113 INFO:teuthology.task.internal.check_lock:Checking locks... 2023-11-26T07:25:38.136 DEBUG:teuthology.nuke:shortname: smithi188 2023-11-26T07:25:38.136 INFO:teuthology.task.internal.check_lock:Checking locks... 2023-11-26T07:25:38.140 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi143.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/rishabh-2023-11-26_06:51:12-fs:volumes-rishabh-mgr-vol-misc-testing-default-smithi/7467343', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'ubuntu', 'os_version': '22.04', 'arch': 'x86_64', 'locked': True, 'locked_since': '2023-11-26 06:52:34.444332', 'locked_by': 'scheduled_rishabh@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBLMlsYrpI20D7nmuV4Ss3kS8JeSY9qF58zM5ezfrs6+JcgHI73OPCbgwt0ossMtulfaME6pGevN9hNlafUPf9wQ='} 2023-11-26T07:25:38.157 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi188.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/rishabh-2023-11-26_06:51:12-fs:volumes-rishabh-mgr-vol-misc-testing-default-smithi/7467343', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'ubuntu', 'os_version': '22.04', 'arch': 'x86_64', 'locked': True, 'locked_since': '2023-11-26 06:52:34.445826', 'locked_by': 'scheduled_rishabh@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBJdbw4I9Ou0oA1kXwuSM4nPA+qQBLn4/ORmmu+HODcBiXZaQj7dzvrfCaWf6g8HzUkhcLo6Ef+7Yycrf+9QMqsM='} 2023-11-26T07:25:38.188 INFO:teuthology.orchestra.console.smithi143:Power off 2023-11-26T07:25:38.189 DEBUG:teuthology.orchestra.console.smithi143:pexpect command: ipmitool -H smithi143.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power off 2023-11-26T07:25:38.204 INFO:teuthology.orchestra.console.smithi188:Power off 2023-11-26T07:25:38.204 DEBUG:teuthology.orchestra.console.smithi188:pexpect command: ipmitool -H smithi188.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power off 2023-11-26T07:25:38.213 DEBUG:teuthology.orchestra.console.smithi143:power off output: Chassis Power Control: Down/Off 2023-11-26T07:25:38.214 DEBUG:teuthology.orchestra.console.smithi143:pexpect command: ipmitool -H smithi143.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-11-26T07:25:38.226 DEBUG:teuthology.orchestra.console.smithi188:power off output: Chassis Power Control: Down/Off 2023-11-26T07:25:38.226 DEBUG:teuthology.orchestra.console.smithi188:pexpect command: ipmitool -H smithi188.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-11-26T07:25:38.234 DEBUG:teuthology.orchestra.console.smithi143:check power output: Chassis Power is on 2023-11-26T07:25:38.247 DEBUG:teuthology.orchestra.console.smithi188:check power output: Chassis Power is on 2023-11-26T07:25:42.235 DEBUG:teuthology.orchestra.console.smithi143:pexpect command: ipmitool -H smithi143.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-11-26T07:25:42.248 DEBUG:teuthology.orchestra.console.smithi188:pexpect command: ipmitool -H smithi188.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-11-26T07:25:42.346 DEBUG:teuthology.orchestra.console.smithi143:check power output: Chassis Power is on 2023-11-26T07:25:42.356 DEBUG:teuthology.orchestra.console.smithi188:check power output: Chassis Power is on 2023-11-26T07:25:46.348 DEBUG:teuthology.orchestra.console.smithi143:pexpect command: ipmitool -H smithi143.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-11-26T07:25:46.357 DEBUG:teuthology.orchestra.console.smithi188:pexpect command: ipmitool -H smithi188.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-11-26T07:25:46.459 DEBUG:teuthology.orchestra.console.smithi143:check power output: Chassis Power is off 2023-11-26T07:25:46.466 DEBUG:teuthology.orchestra.console.smithi188:check power output: Chassis Power is off 2023-11-26T07:25:46.561 INFO:teuthology.orchestra.console.smithi143:Power off completed 2023-11-26T07:25:46.567 INFO:teuthology.orchestra.console.smithi188:Power off completed 2023-11-26T07:25:46.672 INFO:teuthology.run:Summary data: description: fs:volumes/{begin/{0-install 1-ceph 2-logrotate} clusters/1a3s-mds-4c-client conf/{client mds mon osd} distro/{ubuntu_latest} mount/fuse objectstore/bluestore-bitmap overrides/{ignorelist_health ignorelist_wrongly_marked_down no_client_pidfile} tasks/volumes/{overrides test/misc}} duration: 1258.1696183681488 failure_reason: 'Test failure: test_subvolume_no_upgrade_v1_sanity (tasks.cephfs.test_volumes.TestMisc)' flavor: default mon.a-kernel-sha1: 8247bbb5b470f7e2a1f3eefcc16e24cf1338f606 mon.b-kernel-sha1: 8247bbb5b470f7e2a1f3eefcc16e24cf1338f606 owner: scheduled_rishabh@teuthology sentry_event: https://sentry.ceph.com/organizations/ceph/?query=0df9442983ae458daf9b9b579fdbfc90 status: fail success: false 2023-11-26T07:25:46.672 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2023-11-26T07:25:46.756 INFO:teuthology.run:FAIL